Yahoo and Benchmark have teamed up to form an independent company Hortonworks. This initiative further increases the investment of developing Apache Hadoop. Horton Works will concentrate its efforts in making Hadoop an enterprise-ready solution, and providing assistance to enterprise vendors.

Since its inception, Apache Hadoop has been the leading open source technology in big data and cloud computing. Yahoo has pioneered in developing Hadoop and used it in its own web operations. Big companies like IBM and Faebook have also utilized Apache Hadoop to efficiently and cost-effectively store and manage big data.

“Apache Hadoop has been and will continue to be an important area of investment for Yahoo!. The creation of Hortonworks will enable Yahoo! to leverage a commercial partnership in addition to our continued internal investment to accelerate the evolution of the technology and its use to power Yahoo!’s business,” said Jay Rossiter, senior vice president, Cloud Platform Group at Yahoo.

“Forming Hortonworks with Benchmark Capital is the natural next step in the evolution of Apache Hadoop. As Hadoop demand continues to increase, Hortonworks will help organizations more rapidly and effectively implement the technology to derive value and insights from large data sets.”

Hortonworks will be taking Apache Hadoop to the next level. With increased investment, the company focuses on the standardization and commercialization of Hadoop. They projected that half of the world’s data would be stored in Apache Hadoop in the next five years.

“We anticipate that within five years, more than half the world’s data will be stored in Apache Hadoop. We’ve assembled a top caliber team committed to the Apache open source community and with the technology and business expertise to deliver value to the big data market,” said Eric Baldeschwieler, named CEO of Hortonworks and formerly VP of software engineering for the Hadoop team at Yahoo!.

“Apache Hadoop is an exceptionally valuable technology and its growing adoption during the last five years demonstrates its potential to quickly become the de facto platform for managing big data.”

This move by Yahoo and Benchmark has sparked interest from other Hadoop developers. One of those is NetApp, who actively endorsed Hortonworks. “As businesses generate, organize and process more data than ever before, the importance of Apache Hadoop in the enterprise has never been more apparent,” said Val Bercovici, cloud Czar at NetApp. “NetApp is focused on working together with Hortonworks to empower our customers with the advanced tools and services required for managing big data in enterprise environments. We are excited to collaborate with the Hortonworks team, the extended Apache-Hadoop community and our extensive ecosystem of solution partners to accelerate the development and adoption of Apache Hadoop.”

This joint venture by Yahoo and Benchmark is still subject to customary closing conditions and is expected to close in July 2011.