There’s plenty of tension in the database management world, and it’s not for the faint of heart.

Vertica Systems Inc. has been playing in a high stakes poker game that, over its 15-year history, involved the world’s three largest public cloud providers, critical decisions by major enterprises over migrating data away from on-premises networks, and two major U.S. presidential campaigns. Big bets have big consequences, and that’s where tension comes into play.

Vertica, now part of Micro Focus International PLC, launched its technology in 2005 to address scaling and performance issues with traditional enterprise data warehouses, a year before Amazon.com Inc. introduced its public cloud.

As Amazon’s cloud grew to become a disruptive enterprise colossus, it also developed its own cloud-native database Redshift. This spawned competition, most significantly from Snowflake Inc., and Vertica was left with a key decision to make. Should it join the cloud-native party or stay firmly planted in the on-premises world?

The answer came in 2018 when Vertica introduced Eon Mode to support running on Amazon EC2 compute and S3 storage. It was the first data store that allowed compute and storage to be scaled independently for cloud and on-prem workloads.

To make things more interesting, Snowflake and Vertica quickly formed agreements with Amazon Web Services Inc., Microsoft Corp. and Google LLC to integrate services, even though the three major cloud providers had internal data warehousing platforms of their own.

AWS had Redshift, Microsoft ran Azure SQL Data Warehouse, and Google offered BigQuery. This created tension, which Snowflake chief executive officer Frank Slootman has readily acknowledged.

“It’s unavoidable for us to contest these first-party, first-generation cloud products,” said Slootman, during an interview with Diginomica during the company’s Summit event in October. “Does that create tension? Yes, it does. We do compete against them, and that creates a very complex dynamic and, depending on what day of the week it is, things can get a little testy.”

Cloud-native vs. hybrid

Vertica must not only compete against Snowflake, it has to also fight for market share against the products of its own partners. This has placed the company squarely in the middle of a debate within the enterprise computing community: should data-driven workloads remain on-prem, migrated wholesale to the cloud, or run in both?

Recent survey data from the Cloud Native Computing Foundation shows signs of movement toward a hybrid solution. While 62% preferred public cloud, this was down from 77% during the previous year. Now, 38% prefer a hybrid model, and CNCF said it “fully expects this trend to continue.”

Fortunately for Vertica, its decision to launch Eon Mode two years ago positioned the firm to capitalize on growing interest in hybrid. The technology’s ability to separate compute from storage has proven to be valuable for businesses that depend on rapid elasticity to address changing workloads.

A good example of this can be found at The Trade Desk, an online advertising placement firm that bids on hundreds of thousands of ad auctions per second, reaching billions of users around the globe. The Trade Desk broke Rackspace Inc.’s cloud twice and was more than happy to do the same for Eon Mode when Vertica was testing it in beta.

“Vertica said, ‘Are you guys willing to break it for us?’ And we’re like, ‘No problem! Can do!,’” said Dave Pickles, chief technology officer and co-founder at The Trade Desk during an interview last year. “We broke 25 versions of it. But they kept dialing it in and it got real good by the end of it, so we’re really happy how that turned out.”

Beyond integration

Breaking systems over and over again may create its own special form of tension, but there is a method to Vertica’s madness. With Eon Mode, the company is recognizing that developers want to go beyond building applications that merely operate effectively on a cloud platform.

The real value comes when solutions are so ingrained in the design process, developers can take full advantage of the range of cloud services being offered. Merely integrating is not enough. The database management solution must be an architecture designed from scratch to work at maximum efficiency in the cloud.

This critical distinction was highlighted by Ben Vandiver, former chief technical officer at Vertica and the person primarily responsible for Eon Mode’s design and deployment.

“With Eon Mode, Vertica is moving from simply integrating with cloud services to introducing a core architecture optimized specifically for the cloud so customers can capitalize on the economics of compute and storage separation,” said Vandiver during an interview in 2018.

The separation of compute and storage was a key advancement for Vertica, because it gave enterprise customers granular scaling. If they ran out of storage, customers didn’t have to also purchase unnecessary compute to fix the problem.

Eon Mode was the first data store that allowed the independent scaling of compute and storage in both on-premises and cloud environments, but Snowflake provided this early on in cloud-native environments, as did Google with BigQuery and Amazon with its RA3 nodes for Redshift.

Campaign issues

This competitive tension between partners has spilled over into an unlikely but important space: national politics. In 2012, Vertica played a crucial role in the successful re-election effort of President Barack Obama when it was selected as the campaign’s primary analytics database management system.

However, according to a description published in Wired, the data operation for Hillary Clinton’s campaign in 2016 experienced numerous information-technology problems, some of which were attributed by DNC personnel to Vertica. In 2019, the Democratic National Committee replaced Vertica with a database driven by Google’s BigQuery.

A person familiar with the Democratic campaign operation in 2016, told SiliconANGLE that the party had not significantly patched or upgraded its hardware following Obama’s re-election in 2012. Vertica’s software had to run on a system that had not been upgraded in four years, according to this source.

The problems were also compounded by the departure of many key figures in the IT team that ran the database operation for Obama in 2012, according to a trusted source, intimately familiar with the Vertica project at the time, who spoke with SiliconANGLE.

Vertica’s experience highlights a simple reality: It’s not just enterprises that need database management technology in a cloud-driven world. AWS now powers websites for the Federal Election Commission, the Republican National Committee, and the DNC, in addition to providing services to state and country election offices in over 40 states.

There is a lot of tension around data these days. Is it useful or irrelevant, protected or unprotected, too little or too much? Vertica finds itself in the eye of the hurricane, and there is no other place its top executive would rather be.

“If you look at the disruption that’s happening with data, companies that compete with data are not only out-competing everyone in their space, in some cases they’re reinventing and opening up massive new opportunities,” said Colin Mahony, Vertica’s senior vice president and general manager, during an interview in December. “Vertica is really at the heart of that disruption.”

Image: Pixabay