Pivotal Software Inc. has made good on an earlier promise to open-source some of its Big Data software, releasing the source code behind the GemFire, one of the integral parts of its Big Data Suite Hadoop product.

Released under the name Project Geode, Pivotal said its decision means customers will have more insight into what features are being planned for newer versions of the software. In addition, Pivotal says it has submitted an official proposal to the Apache Software Foundation to incubate Project Geode under its governance, in order to grow an open-source community around the software.

GemFire is distributed in-memory database software, which gives users a way to store large amounts of data in the working memory of multiple nodes. GemFire works to balance data across hundreds of nodes, which means it can potentially manage terabytes of data. The benefit is it gives enterprise applications low-latency access to datasets that would otherwise be too large to fit into the memory of a single server. GemFire also offers ACID (atomicity, consistency, isolation and durability) properties, plus fail-over capabilities, which means it will still be responsive even if one or several nodes in the system fail.

The move to open-source GemFire is just the first step in Pivotal’s plan, revealed earlier this year, which calls for it to open-source all of the major components of its Big Data suite. In addition to GemFire, Pivotal says it will release the code for its Hawq SQL engine for Hadoop and Greenplum Database later this year.

Pivotal isn’t releasing the entirety of GemFire’s code however. It’s keeping some of the more advanced features closed-source for the commercial version of the software, including the ability to establish wide-area network connectivity between clusters and stage continuous queries. Paying customers also receive enterprise-level support, Pivotal said.

Nevertheless, Pivotal’s director of open source strategy Roman Shaposhnik said the move would likely aid enterprise adoption while encouraging further innovation with the proprietary products it offers.

“You may have a killer feature today,” Shaposhnik told Infoworld, “but if the bulk of the core of your product is actually coming from an open source project, you can be guaranteed that killer feature will be re-implemented by the open source community at some point.”

GemFire was originally developed by the independent software vendor GemStone over a decade ago, and has grown to more than one million lines of code. GemStone was acquired by EMC Federation member VMware Inc. back in 2010, before its in-memory database technology was transferred to Pivotal in 2013. Rival in-memory databases include SAP’s HANA, the AeroSpike NoSQL database, and Tachyon, an open-source memory-pooling tool that’s currently under development for Hadoop and Spark.

photo: mrjoro via photopin cc