Cloudera founder Christophe Bisciglia unveiled his new startup Odiago this morning, giving to the business side scoop to TechCrunch and the technical details to Curt Monash. The company is launching a product called Wibidata (“we be data”) specializing in data management and what it Monash calls “investigative analytics.”

Wibidata is built on Apache Hadoop and HBase, which Cloudera specializes in supporting and developing. Monash defines investigative analytics as “seeking (previously unknown) patterns in data,” as opposed to monitoring-oriented analytics that aren’t investigative and don’t necessarily yield new patterns.

Monash describes how Wibidata works:

• ALL data pertaining to a single user (or mobile device) is kept in a single, possibly very long, HBase row.
• There are two primary operators in WibiData, Produce and Gather.
  • Produce operates on single rows. It can operate on one row at HBase speed (milliseconds) if you need to inform an interactive user response. Or it can operate on the whole database in batch via Hadoop MapReduce.
  • It is reasonable to think of Produce as mainly doing two things. One is the aforementioned serving of data out of WibiData into interactive applications. The other is scoring, classifying, recommending, etc. on individual users (i.e. rows), in line with an analytic model.
  • Gather typically operates on all your rows at once, and emits suitable input for a MapReduce Reduce step. It is reasonable to think of Gather as being a key cog in the training of analytic models.
• HBase schema management is done at the WibiData system level, not directly in applications. There’s a WibiData HBase data dictionary, powered by a set of system tables, that specifies cell data types/record types and, in effect, primitive schemas.

The company is funded by Google Chairman Eric Schmidt, Cloudera CEO Mike Olsen, SV Angel and others.

Services Angle

We’ve been writing recently about both the difficulty of putting Apache Hadoop to good use due to its complexity, and about the the need for new data management and governance tools. It seems that Odiago is working on solving some of those problems by building tools on top of the Hadoop platform and by creating ways to centrally manage data.