The Data Economy: With GemFire XD, Pivotal closes the Big Data analytics loop
The team at Pivotal made a handful of announcements today focused on the Big Data and analytics layer of its enterprise platform play. Namely, the company announced Pivotal HD 2, the latest version of its Hadoop distribution that incorporates HAWQ, a modified version of the Greenplum database, for SQL analytics. Version 2 is based on Hadoop 2.2, which as you’ll remember is the remastered version of Apache Hadoop that leverages YARN to enable multiple types of applications (not just MapReduce applications) to run on top of HDFS.
In addition, Pivotal also announced the general availability of GemFire XD, in which its in-memory GemFire database is now able to – like HAWQ – run on Pivotal HD. Finally, the company announced that it has integrated GraphLabs, a library of graph analytics algorithms, and MadLIB, the machine learning library, with HAWQ, making it easier for Data Scientists to perform large-scale analytics on Hadoop.
The GemFire XD announcement is a particularly important one for Pivotal, in my opinion. The company is betting big on its grand vision of a data layer that, among other things, allows for not just Big Data application development, but (as it calls it) Fast Data applications, including transactional applications. The vision is a single data layer that closes the analytics loop by leveraging real-time analytics for intelligent transactional workloads and large-scale Big Data analytics for historical analysis and model building that learns from and feeds the front-end transactions. David Floyer and I developed the below model to illustrate the general concept (not Pivotal’s vision specifically) of a close-loop Big Data architecture.
Integration of Big Streams, Big Systems & Big Data for Implementation of Enterprise-wide Integrated Applications; Source: Wikibon 2013
GemFire XD is the final piece of the Big Data closed-loop puzzle for Pivotal, with Hadoop and HAWQ being the other two. With all three in place, the data fabric layer of Pivotal’s platform is capable of Big Data analytics, Fast Data analytics, and everything in between. The addition of GraphLab and MadLIB should make Pivotal HD even that much more attractive to enterprise Data Scientists.
But that still leaves plenty of work on the cloud and application layers of Pivotal’s platform. Pivotal still needs to show it can attract developers to the platform, because application developers are the ones that put those theoretical models built by Data Scientists into production. On this from, a lot will depend on how well the company integrates the Spring Framework for Java application development with the data layer. And then there’s the cloud layer. Pivotal must seamlessly integrate its PaaS, which the company is touting as a multi-cloud PaaS, with the data fabric layer above it.
photo: mrjoro via photopin cc
Since you’re here …
… We’d like to tell you about our mission and how you can help us fulfill it. SiliconANGLE Media Inc.’s business model is based on the intrinsic value of the content, not advertising. Unlike many online publications, we don’t have a paywall or run banner advertising, because we want to keep our journalism open, without influence or the need to chase traffic.The journalism, reporting and commentary on SiliconANGLE — along with live, unscripted video from our Silicon Valley studio and globe-trotting video teams at theCUBE — take a lot of hard work, time and money. Keeping the quality high requires the support of sponsors who are aligned with our vision of ad-free journalism content.
If you like the reporting, video interviews and other ad-free content here, please take a moment to check out a sample of the video content supported by our sponsors, tweet your support, and keep coming back to SiliconANGLE.
