Last year was the year of validation for Hadoop, when the endorsements by a growing list of vendors and users showed that it was for more than a toy for Yahoo, Google, Facebook, and Twitter, says Tresata Founder and Big Data Visionary Abhishek Mehta. This year’s Strata conference shows that it has entered the proof-of-concept stage, when businesses identify problems that they cannot solve with traditional tools and do proof-of-concepts focused on applying Hadoop to those problems at enterprise scale. In the process, they will rearchitect their data stack around Hadoop and ultimately change their business models to drive business with big data analysis.

Where they are headed, he says, is a new way of doing business in which “the business user in touch with direct end customers – who may be bank tellers – has ubiquitous real-time access to insights where all you have to do is swipe something. So it will look more like customer apps than business applications where you need an act of God and 5,000 engineers to write a report in six months.”

The killer app, he told SiliconAngle CEO John Furrier and Wikibon.org Chief Analyst David Vellante in a live webcast (see below) from Strata 2012 on SiliconAngle.tv, is natural language search on big data sets in near real time. That, he says, will be the key to the full potential for Hadoop, which “is not a $50 billion market, it’s more like a $50 trillion market. It will resize and reshape the industry.”

To get there, however, Hadoop has to become enterprise-ready. “You should not expect the Open Source community to solve all the problems, because they won’t. What’s on the roadmap is not what customers are asking for. You will see companies like ours innovate around the imperfect pieces of open source technology to make it more friendly and more productive.”

Hadoop needs to scale to the enterprise platform, he said. One major problem is that today “we have platforms built around every flavor of Apache Hadoop since its inception. If Hadoop is to become what it should be, the industry needs to coalesce around one player that can be the Red Hat of big data.”

Successful big data platforms have to meet three requirements. First they have to provide both data processing and analytics, the two core processes of big data. Second they have to build on standard APIs, because the evolution of the application ecosystem will always outpace the platform just because of the time it takes to move any platform forward. So it has to be designed to allow unforeseen applications to work on top of it. Third, it must be designed to support verticalization around data types rather than industry verticals. For instance, 80% of retail data is financial, which makes this a natural combination with financial market applications.

We are big believers in HBase at Tresata,” Mehta said. “HBase works, and it is a massive disrupter. It has the kinds of functionality users expect, in particular very quick writes and reads. It is definitely the winner.”