Hortonworks CEO Rob Bearden In theCUBE

Hot on the heels of Cloudera Inc. upgrading its version of the batch processing framework, rival Hortonworks Inc. is following suite with a new version of its own distribution that focuses on many of the same core areas as its better-funded competitor.

The new release of the Hortonworks Data Platform (HDP) places the emphasis not on Hadoop itself but rather the surrounding ecosystem of components, adding support for two of the most strategically significant ones in the bunch: Apache Spark, which the firm recently commited to helping bring up to par with enterprise standards, and which Cloudera already supports, and the Apache Kafka message broker, for which the latter distributor pledged to do the same yesterday as part of its announcement.

In a response to its rival’s sudden interest in greater community involvement, Hortonworks is upping the ante with a new initiative that moves the fight to the data management front. Ozone, as the project is called, aims to extend the Hadoop File System with capabilities for storing information as objects, an approach that is highly complementary to media, text and many of the other uninstructed workloads that the batch processing framework is built to handle.

It should be awhile before object storage comes to HDP, but until then customers can settle for the fruits of Hortonworks’ first major Hadoop initiative, which is currently in its second phase. Stinger.next aims to make the Facebook-created Apache Hive more competitive against other structured query tools for Hadoop, particularly Cloudera’s newly upgraded Impala, through a number of unique features that have now been incorporated into HDP. The most notable addition is a cost-based optimizer, Apache Calcite (whose creator joined Hortonworks last October), that tailors execution based on the specific requirements of a particular workloads.

Continuing the integration theme, HDP 2.2 also broadens the component coverage of Apache Argus, the open-source version of the access control technology that the company obtained through the acquisition of XA Secure Inc. earlier this year. The project is a response to Cloudera’s Apache Sentry, which launched in 2013 and was extended to Hive and Impala as part of the company’s update yesterday. Similarly, HDP 2.2 makes the capabilities of Argues available for Hive as well as the Apache Knox authentication service, HBase data warehouse, Spark and Apache Storm, an ultra-fast engine for ingesting real-time streams that graduated from incubation earlier this month.

The emphasis on security is part of Hortonworks’ broader push to address the strict requirements of production environments, an effort that also encompasses delivering management automation and monitoring functionality. To that end, HDP 2.2 taps into the architecture reuse and log visualization capabilities of Apache Ambari and makes it possible to roll out upgrades without incurring any downtime, according to the firm. And to further improve reliability, the new version also allows users to back up data in their environments to Amazon Web Services or Microsoft Corp.’s rivaling Azure platform.

Although the smaller player in the infrastructure-as-a-service space, the Redmond giant currently plays the biggest role in Hortonworks’ cloud strategy thanks to a long-running partnership dating back to 2012. As part of today’s launch, the companies are expanding that relationship by certifying Azure to run HDP, which already powers the native HDInsight service on the platform. Hortonworks is also partnering with Avanade Inc., a consultancy focusing on Microsoft shops, to help customers implement the option that best suits their requirements.