Unable to keep pace with Moore’s law, traditional disk storage has become the weak link in corporate  IT environments, with the ever-widening I/O gap between processors and mechanical drivers (and historically just as significantly, the absence of a genuinely viable alternative) leaving developers no choice but to build artificial restrictions into their software.  Flash has the potential to change all that, removing a fundamental hardware limitation that has been holding back application performance for the last 50 years, but there is still a long way to go until the benefits of the underlying speed improvements can be truly realized at the upper layers of the enterprise stack.

Fusion-io, the market leader in server-side flash, has a vested interest in seeing that happen sooner rather than later and has been actively working towards that goal in recent years. The company made its first major contribution to the flash-aware application cause in July 2013 with the release of its non-volatile memory (NVM) API for Atomic Writes, a feature that makes it possible to simultaneously write multiple independent storage sectors as a single storage transaction instead of the traditional two. That significantly improves database performance and doubles the life of flash drives without compromising data integrity.

At the Percona Live conference earlier this month, the company announced another flash interface called NVM Compression that delivers equally impressive results on the data management front, drastically reducing the performance overhead associated with disk-era compression algorithms in MySQL environments. Wikibon co-founder and CTO David Floyer appeared on the latest episode of SiliconANGLE’s CubeConversations to drill down into the technology and what it means for CIOs with theCUBE host Jeff Frick.

MySQL includes a built-in data compression capability, Floyer details, but most admins opt to turn it off  because it uses up too much CPU power to be practical in transaction-intensive enterprise environments. Not utilizing compression increases throughput by about 64 percent, according to a recent Wikibon study, but the tradeoff is that the raw data requires twice as much storage capacity and bandwidth.

NVM Compression provides an alternative that essentially allows users to have their cake and eat it too, doubling usable capacity on supporting SSDs with minimal impact on overall performance.  “That’s a pretty big deal because for the first time, the ability to do some of the ‘fancy’ things with flash – making applications flash-aware – has gone from the universities and the people who are bleeding edge to a standard NVM extension for all of the MySQL distributions,” Floyer says.

Assuming a base scenario where processing speed measures at 100 transactions per second with native compression and double writes enabled, just installing Atomic Writes and NVM Compression would provide a 10 percent increase in total throughout thanks to nifty under-the-hood tricks courtesy of Fusion-io, Floyer explains. Activating Atomic Writes would add another 60 percent for a 170 percent boost, which can be pushed up to 278 percent using NVM Compression.

The net benefit scales with the environment, he continues, meaning that the bigger the database the bigger the impact on the bottom line. Therefore, the companies that have the most to gain from the technology are large Internet firms without significant legacy baggage that use MySQL as a backend for their web-scale applications, one of the most popular use cases for the open source database.

“The people that can benefit the most are people with large SQL applications,” Floyer elaborates. “They’re often the cloud providers, they haven’t had the resources and they don’t want the overhead of investing in very large Oracle or SQL Server or DB2 databases, so they’ve been using MySQL.”

The NVM extensions already work with three major MySQL distributions, namely Oracle,  MariaDB and Percona, with limited support in MongoDB and other NoSQL databases. Floyer predicts that the technology will become ubiquitous in time, simply because the advantages  are too compelling to ignore. Not only do Atomic Writes and NVM Compression improve performance but they increase hardware longevity as well, greatly reducing costs.

“Every time you have an investment in a database, especially in a fast growing area, you will need to re-architect it, you will need to shard it maybe, you need to do things that extend that database,” he notes. “If you can get three times the throughput, you’ve essentially enabled three times the life.”

Most important of all, the interfaces represent a step closer to eliminating the disk bottleneck inherent in traditional enterprise storage architectures. The end-goal is to enable the creation of next-generation applications that can handle more information than ever before while incurring only a fraction of the database licensing fees organizations have to pay today.

“We’ve had such a built-in restriction on the number of database calls you can make in a transaction, that’s been built into every applications that’s been built in the last 50 years essentially,” Floyer says. “This allows ten to a hundred times the number of database calls per transaction.  That’s a huge difference, a huge difference in how you design applications and how you can make them usable by everybody.”

photo credit: ViaMoi via photopin cc