The term “big data” may have lost favor in recent years, but its byproducts have quietly been cultivating a revival. Through mainstream fascination (and fear) of machine intelligence, that swamp of stockpiled data is finally finding its footing in the business world. Now, eight years after SiliconANGLE’s first coverage of a dedicated big-data event, the excitement for data’s overhyped potential is returning.

Broadcasting live from Silicon Valley, theCUBE, SiliconANGLE’s roving news desk, celebrates Big Data Week with a two-day event, BigData SV. With exclusive commentary and a special report from SiliconANGLE Media-owned research group Wikibon, this week’s coverage reflects on nearly a decade of big data successes, failures and points of cultural evolution. Kicking off the event is theCUBE host Dave Vellante (@dvellante, pictured, left), who sits down with Wikibon analysts Peter Burris (@plburris, pictured, center) and George Gilbert (@ggilbert41, pictured, right). The trio discussed the latest trends in the big data market, noting the historical developments enabling big data’s current revival.

“The most important trend we’re seeing is that users are starting to drive what’s happening in the big data universe,” Burris said. After years of experimentation, big data has worked out enough kinks to deliver user-friendly software applications that require less expertise at the infrastructure level, and Burris expects the enterprise will restructure itself to serve both the software and the hardware needs of democratized data services.

Digital transformation requires big data expertise

This market maturity is crucial to big data’s role in the larger trend toward digital transformations. Having been disrupted by big data’s biggest practitioners, companies like Amazon.com Inc. and Netflix Inc. are forcing entire industries to better employ data analytics at the heart of their business models. This demand has sustained interest in big data deployments despite an 85 percent failure rate, according to some estimates, as insights drawn from massive data pools comprise the most important aspect of a successful digital transformation.

“The whole concept of the digital transformation starts with data,” Burris said. “Our view at Wikibon is [that] a digital business uses data as an asset. That core concept of using data differently is essential because big data is the process by which you create business value out of data.”

From this perspective, Burris sees the big data market as having a pretty normal adoption process. Looking back at the companies that initiated modern big data management processes through open-source offerings, including Yahoo (Oath Inc.) and Google LLC, the objective was to solve a specific business problem. As these companies tried to diffuse big data technologies into other industries, the market got caught up in the buzz. Ultimately recognizing the need to better integrate technologies at the infrastructure level, domain expertise is returning to the market to support the new trend for prepackaged big data services, according to Wikibon analysts.

“On the infrastructure side, what we’re seeing now is a convergence where pieces are fitting together easily enough so that mortal administrators are able to work on them, but [software] applications will take [big data] mainstream,” Gilbert said.

The public cloud has truly enabled much of big data’s progress to date, according to Wikibon analysts. And those cloud providers are also the ones now baking in big data services, a necessary development for mainstream adoption. In a report released in conjunction with BigData SV, Wikibon expects the big data market to exceed $100 billion in the next 10 years, centered on software services. The report identifies the top players in big data services as IBM Corp., Splunk Inc., Dell EMC and Oracle Corp.

Wikibon will be presenting its findings at BigData SV tomorrow morning, and the report can be downloaded in its entirety here (available with a Wikibon subscription).

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of BigData SV.

Photo: SiliconANGLE