Arcadia Data Inc., developer of a business intelligence platform that works directly on unstructured and semistructured data stored in massive “data lakes,” is adding machine learning-aided visualization recommendations and support for complex data types to the latest release of its platform.

The new release of Arcadia Enterprise software, announced today, is different from standard BI tools in that runs directly on compute nodes rather than requiring a separate BI server or edge nodes.

The software combines the necessary facilities for reading and transforming data with an extensive set of visualizations delivered via the browser. The company claimed its approach can all but eliminate the need for extract tables and data transformation, enabling business users to get directly to information in their data lakes quickly and with minimal overhead.

Arcadia positioned its product as a companion to traditional BI platforms. “Data lakes have gotten a bad rap because they’ve been limited to use by data scientists,” said Steve Wooledge, the company’s vice president of marketing. “Legacy BI tools were never built to scale with data lakes. With Arcadia, you don’t make any compromises, as you do with middleware.”

The new release incorporates a feature called Instant Visuals that uses machine learning to recommend the best representation of live data along with side-by-side comparisons of visuals. Recommendations are based on standard best practices for visualizations and the machine learning adjusts for different use scenarios, the company said.

Support for complex data types has been expanded to include arrays, maps and a data type called “structs,” thereby reducing or eliminating the need for preprocessing prior to use in BI. “This release cuts way down on joins and reformatting,” Wooledge said. Arrays are mathematical expressions of data that use a row-and-column format for easier sorting and searching. Maps are another way of creating hierarchy using key-value pairs. Structs are a nested hierarchical data type with elements and sub-elements commonly used in unstructured data formats such as JavaScript Object Notation and Parquet.

Arcadia Enterprise provides a drag-and-drop interface for interrogating data contained in schemas with these nested and complex data types using Structured Query Language, reducing the need for information technology organizations to create logical views or flatten nested data in advance. “In the past you wouldn’t have seen arrays or structs broken out for drag-and-drop processing,” said Dale Kim, senior director of products and solutions at Arcadia Data. “We now understand the underlying data types so you can select individual elements within the cells without a delay in loading.”

Other new features include acceleration for a wider variety of complex queries, support for the Microsoft Azure Data Lake Store for data analysis in the cloud and support for Confluent Inc.’s KSQL, which enables visualization of Apache Kafka topics for real-time analytics. “Confluent just announced the  general availability of KSQL and we’re the first and only vendor to use it,” Wooledge said. “This means we can support queries in memory from real-time data streams.”

Image: Arcadia Data