Big Data is not a product; it’s an ecosystem, and “the ecosystem is really just one big mess,” announced Dave Vellante, cohost of theCUBE, from the SiliconANGLE Media team. With Big Data infrastructure software under going a “slow and steady collapse,” Vellante said that there is an “interesting rift going on, lots of jockeying for position.”

“The technology under the hood is changing,” added John Furrier, who joined Vellante and Wikibon’s George Gilbert, cohosts of theCUBE, at BigDataNYC 2015 during Strata + Hadoop World. There are big questions about what is going on in Big Data, and conference presenters Cloudera, Inc. and O’Reilly Media, Inc. are at the center.

Wikibon Survey: Data warehouse still a mainstay

In this kick-off discussion, Gilbert stated how he sees the Big Data ecosystem spinning out of control, with even the core becoming fragmented. A recent Wikibon survey showed the data warehouse is still a mainstay, and Vellante considers how the slow decline is allowing companies, such as Oracle and Teradata Corp. to make investments and adjust to the changes.

“The ecosystem used to be, ‘Let’s play nice.’ Now it’s getting very competitive,” said Vellante, who thinks that the market is heading to play out so that the number one company will make a lot of money, number two will do OK, number three will break even and the rest will be gone. Discussing the momentum behind Apache Spark (for Big Data processing), Gilbert quoted a Databricks, Inc. survey that showed 48% of Spark customers now run outside the Hadoop ecosystem.

Vellante summed up the discussion by asking: “How are people going to make money? By identifying ways they can differentiate from the competition.” He sees this as a long game, with the key for the winners being differentiation and adoption, particularly moving into the application space.

Watch the full interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of BigDataNYC 2015.

Photo by SiliconANGLE