Hortonworks Inc. today announced a new version of its data platform, boosting query performance, strengthening security and streamlining data management operations.

With version 2.6 of the Hortonworks Data Platform, based on the open-source Hadoop software framework for handling very large datasets distributed on many computers, the company claims to be the first Hadoop vendor to fully support Apache Hive 2.0 with Live Long and Process functionality for intelligent in-memory processing. LLAP replaces direct interactions with the HDFS DataNode with a long-lived daemon that performs caching, pre-fetching, some query processing and access control. This reduces the overhead of running massively parallel queries upfront and also runs multiple queries against same data. The company said the combination translates into a significant performance boost for SQL interactive queries.

“This enables real-time tactical query response times that are consistent and scalable across a scale-out architecture,” said Scott Gnau (pictured), chief technology officer at Hortonworks. “The neat thing about this is that we’ve done [the acceleration] inside of Hive so everyone can get the advantage and existing applications don’t have to be rewritten.”

HDP 2.6 also introduces ACID merge functionality, which enables customers to perform incremental data maintenance without reloading the entire table. “If you’ve already got a primary key in the table, you can update the attributes without duplicating the key,” Gnau said.

Support for Apache Spark 2.1 has been added, as well as the latest version of the Apache Zeppelin notebook, a favorite tool of data scientists. Hortonworks will also continue to support Spark 1.6.3 for customers who aren’t yet ready to migrate. The new version of Zeppelin “creates much better collaboration capabilities,” Gnau said. “It’s making Spark more usable, consumable and productive for data scientists.”

Enhancements to the Apache Ranger security framework and Apache Atlas governance framework reduce synchronization time for companies with a lot of users. Hortonworks is also enhancing support for the bulk addition of policies from one environment to another via expanded tag-based policy support for Spark, Zeppelin, HDFS, Apache Kafka and Hbase.

Hortonworks has also improved Apache Ambari, a suite of tools for provisioning, managing and monitoring Hadoop clusters. The enhancements “extend the breadth and depth of coverage and add functionality to help operations teams manage restarts and upgrades,” Gnau said. Hortonworks’ SmartSense cluster performance monitor has been enhanced to make it easier to automate the application of recommendations for cluster improvement.

The new release is also available on IBM Power Systems. This is an outcome of the work the two companies have been doing together on the Open Data Platform Initiative, which is creating a standardized Hadoop framework.

Finally, HDP 2.6 is the first release that will be available first in the cloud. The new functionality was live on Microsoft Azure, the Amazon Web Services cloud and the Hortonworks cloud several days before today’s formal release. Gnau said that in the future the company intends to “bring cloud previews to market faster so customers can try them out.”

Photo: theCUBE