There’s been a lot of talk lately about storage hypervisors.  Server hypervisors are allowing us to wring a lot more out of our server resources, but the storage in virtual environments is being used even less efficiently than it was when it was attached only to physical servers.  The goal of a storage hypervisor is to bring the same types of improvements to storage that server hypervisors brought to servers.

What Is A Storage Hypervisor?

The concept of a storage hypervisor tracks pretty closely with that of a server hypervisor.  Server hypervisors did two things:  they allowed us to get much higher utilization out of our existing server resources, and they virtualized those resources to provide management flexibility just not available with physical servers.  By enabling much higher utilization of server resources, we needed a lot less hardware to meet any given computing requirement, lowering the cost/server pretty significantly.  And by decoupling servers from physical hardware, we opened up a whole new range of options in dealing with many of IT’s challenges in the areas of responsiveness, high availability, data protection and disaster recovery.

Why Do We Need A Storage Hypervisor?

In pursuing this analogy, storage in virtualized environments falls short in several areas.  Because of the significantly increased randomness and write-intensive of the I/O patterns in virtual environments, legacy storage architectures deliver fewer usable IOPS per spindle than they do in physical environments, which means we need to buy more of them to meet any given performance requirement, increasing costs.  We can spin a new virtual machine (VM) up in seconds, but it still takes minutes or hours to provision high performance storage for it, holding us back from achieving the full promise of server virtualization to rapidly provision new servers.  Despite the availability of thin provisioning technology, we’re still wasting a lot of physical storage capacity in virtual environments primarily because the performance hit of thin provisioned storage is driving us to generally use “thickly” provisioned storage for most VMs.

What Should We Expect From A Storage Hypervisor?

Conceptually, we’d like a storage hypervisor to remove the challenges presented by physical storage so that we can better achieve the full promise of virtualization technology.  Those challenges include performance, space efficiency and provisioning times.

In the performance area, the storage hypervisor should use the existing storage hardware more efficiently to increase performance without requiring the purchase of any additional or faster storage.  It is the very random, very write-intensive I/O patterns of virtual environments that is the problem – what is the storage hypervisor doing to address that and deliver more usable IOPS per spindle?  With increased IOPS per spindle, you’ll need fewer spindles to meet any given performance requirement, and there will be a cost savings.

Server hypervisors effectively “thin provision” servers with hardly a performance hit (on a per CPU basis).  Yet when we thin provision storage in a virtual environment using the native hypervisor “thin” virtual disk options, that device can run seven to nine times slower than a thick provisioned virtual disk.  If that thin virtual disk stays around long enough to have all its blocks written to at least once, the performance differential between it and a thick virtual disk will narrow.  DRAM or SSD caches can be used to front end these devices to address this performance issue, but it’s an expensive way to solve the problem.  A storage hypervisor should provide options to deliver high performance, thin provisioned virtual disk without requiring that you buy any additional hardware to address the issue.

Thin provisioning is inextricably linked with how fast storage can be provisioned.  A thin provisioned virtual disk can be created very quickly, regardless of its size, but imposes a performance hit when you write to it during runtime operations.  A storage hypervisor should allow high performance storage to be rapidly provisioned so that a new VM could be spun up literally within seconds, and yet still support the need for high performance.  Since administrators today are primarily using thick provisioned virtual disks for high performance VMs and just accepting the “provisioning time” hit, the ability to instantly provision high performance storage moves us a lot closer to the full promise of server virtualization technology.

We should expect a storage hypervisor to accommodate existing storage hardware, whatever it is.  Clearly it has to provide its benefits for spinning disk, but solid state disk (SSD) is quickly becoming a part of many computing environments as well.  A storage hypervisor should provide benefits across both, but not require higher performance storage to be able to wring the most out of what customers already own.  SSD is not cheap, and a storage hypervisor’s ability to use it very efficiently when it is present fits very nicely with the overall goal of getting the most out of existing storage resources.

Fitting In

Ideally, a storage hypervisor would work so closely with your server hypervisor that you would hardly know that its there.  Following the “embrace and extend” model is a very efficient way to implement a storage hypervisor.  Certain types of critical functionality, such as failover, snapshots, live migration and replication, are being implemented at the hypervisor level by the major hypervisor vendors.  Implementing the storage hypervisor to deliver the needed performance, space-efficiency and rapid provisioning benefits while leveraging the infrastructure that the hypervisor vendors already provide results in the lightest weight implementation and the greatest overall value.  It’s also preferable from an administrative point of view, since it delivers its functionality with minimal to no change in current workflows.  Those storage hypervisors that integrate most transparently into the existing management tools and workflows are likely to be most successful.

Every day, how we’re using virtualized server resources acts as a constant reminder of what we’re missing on the storage side.  The storage hypervisor fills that gap.  In the long run, storage hypervisor technology will undoubtedly become a part of the server hypervisor itself.

About Mark Davis

Mark has been at the center of the networked storage and virtualization revolutions since their inception. He launched the industry’s first Fibre Channel disk array in 1994, and was instrumental in growing Sun Microsystems from a non-player to the largest Unix storage vendor within five years. After a stint as VP of marketing at $2B storage vendor StorageTek, Mark repositioned ConvergeNet as the inventor of SAN-based storage virtualization, resulting in a $340M acquisition by Dell. After an instrumental role in the IPO of Evolve Software, Mark returned to the storage industry, working on multiple virtualization projects. Before co-founding Virsto, he was CEO of storage resource management vendor Creekpath, where he engineered its acquisition by Opsware (now HP).