UPDATED 14:52 EDT / OCTOBER 02 2013

The Storage Holy Grail: Decoupling Performance from Capacity

Virtualization has been the biggest trend in data centers over the last 10 years, resulting in enormous cost savings and increased operational efficiencies.  But as more applications become virtualized, many companies are experiencing an interesting and discouraging trend – a decrease in application performance.  The root cause of this problem is commonly being tracked back to a single source, the storage infrastructure.   In fact, internal estimates at one of the largest virtualization vendors indicate that over 70% of application performance support calls are tied to storage related issues.

I/O bottlenecks in primary storage can add significant latency to virtual applications, resulting in slow response times at best and unusable applications at worst.  Aside from frustrated end users, this creates numerous problems for IT such as unpredictable expenses, increased vendor lock-in, hardware sprawl, and increased help desk calls.

To date, the only option when faced with the above challenge is to throw storage hardware at the problem.  Storage administrators can improve the capabilities of their Storage Area Network (SAN) by moving to faster interconnects and by upgrading the SAN itself with faster disks and processors or more capacity.  Unfortunately, these are all very expensive and disruptive solutions, which don’t even guarantee an improvement in application performance.

What companies need is a way to get dramatic improvement and scale for storage performance separate from storage capacity.   And they need a software solution that lets them break free from the never ending shackles of storage hardware upgrades.  This has created an enormous market demand for server side flash, which in turn has created a need for new “Flash Hypervisors”.

Flash to the Rescue

 

Many companies are looking at flash as the savior for their storage performance woes.  Flash has microsecond response times (compared to milliseconds for disks) and delivers over 10x the performance (measured in I/O per Second, or IOPS) as spinning disk.  As the price of flash continues to come down, it is increasingly looked upon as an intrinsic solution for storage performance challenges going forward.

  • But what is the best way to implement flash?

One approach is to use flash in the storage array, either in conjunction with spinning disk or as a replacement.  First, it is costly.  As a new array can cost tens of thousands of dollars, throwing SAN hardware at storage performance problems is an expensive endeavor.  Second, it is very disruptive.  Any time that changes are made to the SAN, the new device needs to provide the same level of data services (data protection, disaster recovery, etc) as the old device, and applications need to be migrated to the new device. Finally, upgrading SAN hardware is usually just a temporary fix.  At some point, even the new array will reach peak capacity, resulting in the need for constant future upgrades.

An alternative approach is to use server side flash to accelerate storage performance. Like flash in the SAN, this approach leverages the massive IOPS capabilities of flash to increase storage performance.  But unlike the SAN approach, this is much more cost effective, easy to scale-out along with CPU and memory, and promises better response times by co-locating the flash in the hosts, directly next to your application.

However, server side flash, by itself, is not without its own challenges.  Server flash cannot be leveraged transparently in a way that will let applications continue leveraging the backend storage system as a capacity and data services tier. It is often difficult to leverage flash efficiently across multiple hosts, which is required when Virtual Machines (VM) move around.  In addition, it is often only suitable for a limited set of use cases – i.e. workloads that are primarily read-only or read-intensive.  That is because using server side flash for writes creates a potential fault tolerance problem whereby if a host (or the flash device on the host) becomes unavailable, data can be lost.

The Need for a Flash Hypervisor

 

From a performance, ease and cost standpoint, it is clear that server side flash shows the most potential for solving storage performance problems.  But the issues raised above need to be addressed for server side flash to be implemented as a strategic, enterprise class solution.  This has given birth to a new technology, called the Flash Hypervisor.

A Flash Hypervisor virtualizes all server side flash into a clustered acceleration tier that enables IT administrators to quickly, easily and cost-effectively scale-out storage performance completely independent of storage capacity. Just like traditional hypervisors abstract physical CPU and RAM into a logical pool of resources, a Flash Hypervisor does the same for all server flash devices across a data center.  More specifically, the Flash Hypervisor provides a resource management scheme that multiplexes multiple VMs to a set of flash devices according to user-specified policies. The result is dramatically faster and truly scale-out read and write performance for all VMs, without the need to change existing storage infrastructure.

VM mobility is addressed with a Flash Hypervisor by clustering all available flash into a single pool that can be accessed by any host.  In other words, each host running the Flash Hypervisor communicates with other hosts to provide on-demand access to the data sets on their respective flash devices.

In addition, Flash Hypervisors make server side flash applicable to all virtual applications, regardless of read/write behavior.  Since the Flash Hypervisor is clustered, writes are synchronously replicated between hosts in the cluster until they can be destaged to the storage array.  This means that every write has at least one copy on another host in the cluster, which protects against unforeseen outages.  This level of fault tolerance cannot be attained without a Flash Hypervisor.

 

A Revolution in Storage Design

 

Flash Hypervisors fundamentally change data center design.  In the past, storage was designed with performance and capacity in one tier, both being serviced by a monolithic (expensive) piece of SAN hardware.  With a Flash Hypervisor, storage performance is now decoupled from capacity so that it can cost effectively scale-out with demand.

Image: Charlotte Bartlett

For example, a single mid-range SAN costs about $60,000 and delivers about 50,000 IOPS.  Need more IOPS?  You have to buy another storage array.  In contrast, a Flash Hypervisor deployed across just two hosts can deliver 2x more IOPS (approximately 100,000) using the same back-end storage device.  Need more performance?  Just add more inexpensive flash.  For example, six hosts (approximately $6,000) will deliver over 350,000 IOPS!  This is an order of magnitude more storage performance at a fraction of the cost.

A few years ago, it was considered “normal” for application provisioning to take a few weeks. Overnight, virtualization changed everything, slashing the normal to just a few minutes.  The same thing is happening with Flash Hypervisors, where microsecond response times and millions of IOPS are becoming the new normal for storage performance.

Are Flash Hypervisors the “holy grail” of storage – i.e. a simple and cost effective way of decoupling storage performance from capacity?  Once you see the technology in action, you will become a true believer.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU