Pentaho Corp. is giving data engineers and analysts access to the same tools as business end-users in Pentaho Business Analytics 7.0, which is launching today.

The new release also features simplified deployment options, improved capabilities for handling Spark data streams and expanded Hadoop data security. The most significant improvement is that Pentaho will now make it possible for data to be visualized at any stage in the data preparation process without the need for separate data integration and visualization tools. Pentaho isn’t adding any new visualizations to its existing reporting suite; rather, it’s spreading them across the data preparation lifecycle.

That’s a big deal, said Chuck Yarbrough, senior director of solutions marketing at the Hitachi Corp. subsidiary. Loading data into a data warehouse usually involves a painful process called extract/transfer/load in which source data is examined, cleansed, normalized and ultimately loaded into the warehouse, a process that can take days. A lot of potential problems turn up during ETL. For example, location data may be missing or may be coded as geo-coordinates rather than ZIP codes. Those inconsistencies need to be fixed in the transformation process.

“Often you don’t know what’s wrong with your data set until you get to the end, because that’s when you can put it into a multidimensional model to see the holes,” he said. With the new release, “I can pivot and interrogate that data with my visual toolset at any point along the data workflow. I can do the same exact process that the end user would do at any point in the process.”

The result is less time spent addressing data quality problems after the fact because engineers and analysts can see how the data will finally be used. Some surveys have established that as much as 80 percent of time required for big data analytics projects is spent on preparation. “As the ETL person is interrogating the data, they’re working with the business users to validate that what is there is appropriate,” Yarbrough said.

Pentaho 7.0 will be generally available in November as a no-cost upgrade to existing subscribers. Pentaho Business Analytics has about 1,500 paying customers, according to the company.

Image via Pixabay CC