ClusterHQ offers developers GitHub-like data repository

ClusterHQ FlockerHub

ClusterHQ Inc. is attacking a critical shortcoming of container technology – statelessness – with two new container data management products intended for use by application developers.

The company likens its new FlockerHub hosted data management repository to GitHub, the repository hosting service that is wildly popular with developers. A companion open-source tool called Fli lets developers snapshot, clone, push and pull data volumes to FlockerHub. The combination enables teams that employ the DevOps style of rapid application development to “seamlessly move data between laptops, test environments, data centers and clouds, all with version and access controls,” the company said.

Statelessness has been a major barrier to container, adoption, said Michael Ferranti, ClusterHQ’s vice president of marketing. “All of the core tooling is optimized for containers that don’t have to remember anything between being turned on and shut down,” he said. “If you’re running a database and the server dies, you can spin up a new container but it won’t have any of that data you were using.”

As a result, developers are forced to work with temporary extracts of production data instead of live data. That results in lost time due to bug fixing late in the development process. “We’re spending a massive amount of time and money doing continuous integration and staging,” Ferranti said. “One reason bugs make it into production is because people test with unrealistic data. With FlockerHub and Fli that should never happen.”

GitHub for data

ClusterHQ describes FlockerHub as a version-control repository for Docker data volumes in the same way that GitHub is a version-control repository for source code. DevOps teams can use it to keep track of and distribute data volumes to any host.

Fli is the command line interface for FlockerHub. It enables DevOps teams to version-control Docker data volumes in the same way Git version-controls code. Fli can take incremental snapshots of any database or data volume on Linux and make them available on FlockerHub. Incremental snapshots save space and bandwidth by only capturing changes to the data, not the entire data set. Snapshots also enable developers to roll back to a previous point in time, since original data sets aren’t overwritten.

Fli can clone data multiple times without increasing storage consumption, which enables parallel testing without additional storage overhead. Fli also provides the ability to organize data into volume sets and to create branches, which are useful in managing large test libraries.

“Fli lets you make a copy of any Docker volume, create branches and only keep the ones you need,” Ferranti said. Data volumes that are pushed to Flockerhub have unique identifiers, enabling any other Fli client to access them by a unique name.

FlockerHub will launch into beta test on Nov. 8, with free access during the approximately six-month beta period. Pricing has not been set. Developers can sign up here to receive instructions on how to access their account at launch. Fli will be available as an Apache 2.0 download on Nov. 8.

Image courtesy ClusterHQ