UPDATED 08:30 EDT / SEPTEMBER 21 2015

NEWS

What you missed in Big Data: Unlocking the potential of Hadoop

The industry effort to make Hadoop more viable for everyday enterprises gained a surprising new supporter last week in Pinterest Inc., which released an internally-developed tool that helps make insights produced in the framework accessible to other applications. Its entry into the fray adds another name to the already lengthy list of web companies involved in the project.

Hadoop started its life at Yahoo! Inc. as an implementation of the MapReduce data processing model developed by Google Inc. in 2004 and has since seen its capabilities augmented with additions from a number of other online giants. Pinterest’s new technology, aptly called Terrapin, combines several of those latter contributions to provide low-latency access to terabyte-sized datasets from the framework.

Achieving that previously required complicated workarounds that proved far too inefficient for its purposes, not too mention taxing on the engineering team, a challenge that is compounded severalfold for the majority of organizations that don’t have access to talent of the same caliber. Making it easier to export files from Hadoop thus allows Terrapin to significantly lower the entry barrier to the project.

But there are still numerous issues standing in the way of the platform from reaching its full adoption potential, first and foremost the difficulty of producing the insights that Pinterest’s tool is designed to help make more accessible. That issue also received some much-needed attention last week after Arcadia Data Inc. launched a commercial version of its Hadoop-based business intelligence solution.

Arcadia Enterprise provides a visual analytics workbench that makes it possible to create visualizations from information stored in their organizations’ analytics clusters using a drag-and-drop builder that doesn’t require any knowledge of the underlying cogwork. That expands the accessibility of platform beyond data scientists to everyday business users.

For all the focus on Hadoop, however, it’s hardly the only tool in an organization’s analytics arsenal. The growing amounts of data coming from new sources such as connected devices is also driving enterprises to adopt solutions like Iron.io Inc.’s namesake event processing service, which netted its creator $8 million in funding against the backdrop of Arcadia’s launch.

The capital will help add more capabilities to its offering, an alternative to the Lambda engine that Amazon Inc. added to its cloud platform last year with the same value proposition. Developers can use Iron.io’s technology to automatically execute algorithms against incoming real-time data and feed the results into their applications for use cases such as alert delivery and traffic monitoring.

Photo via Yolanda

Since you’re here …

… We’d like to tell you about our mission and how you can help us fulfill it. SiliconANGLE Media Inc.’s business model is based on the intrinsic value of the content, not advertising. Unlike many online publications, we don’t have a paywall or run banner advertising, because we want to keep our journalism open, without influence or the need to chase traffic.The journalism, reporting and commentary on SiliconANGLE — along with live, unscripted video from our Silicon Valley studio and globe-trotting video teams at theCUBE — take a lot of hard work, time and money. Keeping the quality high requires the support of sponsors who are aligned with our vision of ad-free journalism content.

If you like the reporting, video interviews and other ad-free content here, please take a moment to check out a sample of the video content supported by our sponsors, tweet your support, and keep coming back to SiliconANGLE.