Infochimps, known for its data market, today announced a big data management tool called Infochimps Platform, a suite for managing Hadoop and other big data tools that can be hosted in the cloud or on-premise. Infochimps will now provide customers with the technology stack that Infochimps has developed for its data market. Customers can use that infrastructure to manage their own data sets as well as data purchased from the data market.

Infochimps is also getting into the services business, offering data scientists for hire for custom projects, training and support. Infochimps can support companies that have existing big data infrastructure, whether that’s Hadoop, MongoDB or something else, or companies that are starting from the ground level and aren’t even sure what data they have available within their enterprise.

The features of Infochimps Platform include, according to the company’s announcement:

• Ironfan: simply, powerfully describe your architecture: stack components, the machines
  they run on and how they connect.
• Schedulable, Elastic Hadoop: A full suite of data science tools, out of the box
• Data Delivery Service: scalable, legacy and existing systems
• Real-time Dashboarding
• In-stream data augmentation
• Integration with your existing stack

“Many companies don’t even know what to do with the data they already have, let alone what to do with a data market,” Infochimps CEO Joe Kelly told me in an interview. We’ve written frequently about the need businesses have for big data guidance, the complexity of Hadoop and the shortage of data scientists. Infochimps is the latest player in this $15 billion market.

According to the company’s announcement initial customers include Runa and Black Locus. The companies are using Infochimps Platform to “ingest data from legacydatabases, data feeds, data from the Web, and from the Infochimps Data Marketplace.” This is a move towards what I call data-as-a-service, a concept I will explore in a forthcoming Trends 2012 post. Data markets, APIs for accessing data sets hosted data management tools and proprietary algorithms can converge to create new hosted services that enable radical new uses of data.

ServicesAngle

Infochimps is getting into the big data consulting business with this move. While many support companies like Cloudera and 10Gen are providing services for software that they are intimately involved in developing, Infochimps is taking a step back and looking at the broader big data landscape. The downside is that the Infochimps team won’t have the depth of knowledge in Hadoop that Hortonworks has, or the depth of knowledge in MongoDB that 10gen has. But they will have the advantage of not seeing every data nail as a problem for one particular type of big data hammer. Other providers in this space include Datasalt and our sponsor EMC.