Google is said to be mulling the idea of commercializing a highly scalable, globally distributed database technology known as “Spanner”, which the company has been using in its own data centers for several years to deliver many of its core services.

The challenge for Google’s Cloud Platform Group now is to work out how to transform the technology into a product that others can use to build applications, reported The Information. It adds that one of the main obstacles Google’s engineers must overcome is working out how to decouple Spanner from its own infrastructure and create a version that will work in other environments.

The Information quotes two anonymous sources as saying that Google is driven to create something out of Spanner due to increased competition from cloud rivals like Amazon Web Services and Microsoft Azure. The hope is that Spanner, described by the sources as a highly-scalable, high-performance database technology, will give Google a significant advantage over its rivals.

Google has gone on the record to talk about Spanner in the past, saying its an SQL-like database that can run across multiple data centers, and is capable of scaling up to millions of machines in hundreds of data centers and trillions of database rows. It is “the first system to distribute data at global scale and support externally-consistent distributed transactions,” Google has said.

Spanner’s most appealing feature is that it supports synchronous replication, which means that any changes made to the database will automatically be replicated across every data center in real-time, so the data stays consistent regardless of where it’s accessed from.

“Applications can use Spanner for high availability, even in the face of wide-area natural disasters, by replicating their data within or even across continents,” according to Google.

Google first put Spanner to use with F1, a relational database management system that supports Google’s Adwords platform. Thanks to Spanner’s support for synchronous replication, Google was able to maintain five replicas of F1’s database, each of them working in perfect tandem with one another in data centers located across the world, thereby ensuring continuous availability of the platform.

It’s believed that Google’s Gmail, Search and Picasa services all rely on Spanner’s technology, though Google has never confirmed this.

Because Google has declined to comment on The Information’s story, it’s impossible to know what kinds of applications Google intends to run on Spanner. However, we can get some insights from a startup founded by two ex-Google engineers called Cockroach Labs, which is believed to be building an open-source version of Spanner.

Cockroach Labs’ main product is CockroachDB, currently in beta, which is said to be inspired by Spanner. The startup describes its platform as a distributed SQL database that’s able to scale horizontally and survive machine, disk, rack and data center failures with minimal latency or application disruption. It adds that the database is particularly useful for applications that demand a high-degree of survivability and horizontal scalability.

Photo Credit: matt-artz via Compfight cc