Hadoop vendor Hortonworks Inc. has rolled out an update to its Hortonworks Data Platform (HDP) for the first time in six months, with the biggest change being that it can now be provisioned automatically in any cloud environment, the company claims. In addition, Hortonworks’ Hadoop is now easier for enterprises to adopt and administer, and has been made more secure.

Hortonworks’ main selling point is that it adds value to Apache Hadoop, and is focused on making the software easier to deploy across distributed clusters. As such, one of the big new features in HDP 2.3 is proactive cluster monitoring.

In addition, Hortonworks says it’s integrated a number of open-source technologies into its latest release, relating to data governance and security.

The new release is the first since Hortonworks sparked controversy in the Hadoop world by founding the Open Data Platform alongside Pivotal Software Inc. and IBM, among others, but without the cooperation of Hadoop rivals Cloudera Inc. and MapR Technologies Inc. At around the same time, Hortonworks also teamed up with Aetna Inc., Merck & Co., Inc., Target Corp., and SAS Institute Inc., to create the Data Governance Initiative (DGI) in a much less-publicized move to tackle the problems of data governance in Hadoop.

Now it seems we’re finally seeing the fruits of that initiative with the inclusion of Apache Atlas, an open-source project that builds data governance services directly into Hortonworks’s Hadoop distro. According to the company, Atlas brings a scalable metadata service, integration with Hive and SQL metrics, and a new user interface for searching metadata and lineage.

Meanwhile, Cloudbreak is the name of Hortonworks’ new tool that automatically provisions HDP clusters in any cloud environment. The technology is a result of the company’s recent acquisition of SequenceIQ. The new feature comes after Hortonworks said it was now an official service on Google’s cloud platform.

Hortonworks has also added a service called SmartSense proactive monitoring for its subscription customers only. This service provides recommendations and insights about cluster utilization and health, and can help users to utilize cluster resources more efficiently, as well as providing quick and easy log-file capturing.

Other new features in HDP 2.3 include better data protection, alongside more transparent data encryption. There’s also a new encryption key management store provided by Apache Ranger, which manages authorization and audit policies. Meanwhile, Apache Knox looks after authentication with LDAP data caching and bi-directional SSL support.

“EMC and Hortonworks have a shared vision of a Business Data Lake which provides the ability to bring together data, analytics, and applications to deliver meaningful business outcomes for companies,” said Aidan O’Brien, senior director for Big Data solutions at EMC, a Hortonworks partner. “The new security and data governance capabilities and improved user interface in the latest release of the Hortonworks Data Platform [are] going to make it easier to achieve these outcomes in a more sustainable and secure way.”

One example of this improved user experience is a new set of guided configurations for HDFS, YARN, Hive, and HBase, which makes each component more predictable and easier to use. In addition, Hortonworks is also making it easier to optimize clusters with a new customizable dashboard showing key performance indicators.

Image credit: ejaugsburg via Pixabay.com