Dell Open-Sources Code for Hadoop Deployment Tool

dell logo

Crowbar is an open source deployment tool developed by Dell originally as part of its Dell OpenStack Cloud service. It started as a tool for installing Open Stack, but can deploy other software through the use of plug-in modules called “barclamps.” Today Dell released the source code for its Apache Hadoop barclamp on Github.

The goal of the Hadoop barclamp is to reduce Hadoop deployment time from weeks to a single day.

Crowbar is based on Opscode Chef Server, an automated configuration manager. Opscode competes with Puppet, which we reported earlier has raised another $8.5 million.

Dell has invested significantly in Crowbar. It’s an example of Dell’s focus on developing open-source tools that fits with its own server technology. For example, the Dell Open-Stack Cloud Solution integrates OpenStack with Dell’s PowerEdge C servers, the Crowbar OpenStack installer and services from Dell and Rackspace Cloud. Crowbar manages the OpenStack deployment from the initial server boot to the configuration of the primary OpenStack components. In addition to installation it also proves monitoring, network discovery and performance data gathering features.

Crowbar does not compete with Cloudera’s line of Hadoop management tools included in Cloudera Enterprise. Dell is a Cloudera partner and resells Cloudera Enterprise.

In addition to Hadoop and OpenStack, Crowbar can be used with Cloud Foundry. Expect to see more software supported in the future.

Dell’s support for this project is a validation of both Hadoop and the DevOps approach. I’ve been saying for a while that tools like Hadoop necessitate DevOps – you just can’t manage huge clusters of development-oriented servers without good automation tools and a strong relationship between the operations and development staffs.