UPDATED 14:30 EST / FEBRUARY 27 2012

Outline of Hadoop’s Problems is Shortsighted

Jessica Vascellaro of WSJ published a piece in which she engaged the buzz around Hadoop from the opposite angle, highlighting some of the reasons the open-source parallel analytics platform may not be the universal cure of massive datasets.

One of her main points is that the fact Hadoop is free doesn’t necessary translate into a drastic cut in overhead on the bottom line when taking into account a couple factors. For one, data scientists – the people tasked with managing what is unquestionably a very complicated operation, especially at scale – are currently a scarce resource in the job market. The second was laid out by the head of Hortonworks, also described it as an “aspirin for a broken leg” scenario by Collectivei co-founder co-founder Stephen Messer.

“Rob Bearden, CEO of Hortonworks in Sunnyvale, says the impulse to store lots of data because it can be cheap can lead to storing too much and make answering simple questions harder. “You want to have some sort of control over what data you push into an application,” he says. “Otherwise, your juice isn’t really worth the squeeze.”

However, this perspective addresses only a small aspect of the market. The counter-argument comes from a broader point of view, one that includes Cloudera as well. Chief executive Mike Olson attributed the challenge Bearden outlined above to the relatively small Hadoop ecosystem, specifically to a shortage in applications that “make the data they [companies] are storing useful.”

Nevertheless, the reality is that the market still has a lot of room to mature, though the actual numbers showing that the universal cure may not be completely unachievable after all. Cloudera’s sales doubled in the past 12 months according to the company, and recent data by Wikibon seems to point towards a a similar phenomenon occurring market-wide. In 2012 big data analytics will be worth a total of roughly $5 billion, and in just five years this demand will grow tenfold.


Since you’re here …

… We’d like to tell you about our mission and how you can help us fulfill it. SiliconANGLE Media Inc.’s business model is based on the intrinsic value of the content, not advertising. Unlike many online publications, we don’t have a paywall or run banner advertising, because we want to keep our journalism open, without influence or the need to chase traffic.The journalism, reporting and commentary on SiliconANGLE — along with live, unscripted video from our Silicon Valley studio and globe-trotting video teams at theCUBE — take a lot of hard work, time and money. Keeping the quality high requires the support of sponsors who are aligned with our vision of ad-free journalism content.

If you like the reporting, video interviews and other ad-free content here, please take a moment to check out a sample of the video content supported by our sponsors, tweet your support, and keep coming back to SiliconANGLE.