NetApp today announced a deal with Fusion-io. Industry news coverage has called this announcement a “mystery,” citing excerpts from the press release as follows:

“… working closely with storage industry leader NetApp to provide solutions using server-side flash and caching software products when used in conjunction with the NetApp Virtual Storage Tier. The two companies are collaborating on low-latency, high-performance solutions for compatibility between the Fusion ioMemory platform and NetApp’s Data ONTAP operating system, as well as key caching solutions, including NetApp Flash Cache, NetApp Flash Pool and Fusion-io caching software.”

 According to All Things D:

“It’s kind of a weird announcement, where neither company is disclosing what it’s all about.”

I’m not sure why this is such a mystery to people. What’s really going on that the mainstream trade press doesn’t understand can be seen by connecting the dots. Let’s start with a bit of review:

1. Fusion-io entered the market for server side flash about five years ago. Around the same time that EMC had the idea to stuff flash cards into conventional disk arrays.
2. Fusion-io, began to market its product to speed up database-bound application and sold its solutions through a variety of OEMs, including IBM, Dell, HP and the channel. It also sold direct to big Internet Giants like Apple and Facebook and to some smaller innovators such as Kaminario (all flash arrays) and Datacore (hybrid arrays). The company exploded and had a very successful IPO last summer.
3. It started to become clear that Fusion-io was more interested in software than hardware. Check out this piece from Wikibon which explains Fusion’s Virtual Storage Layer (VSL) strategy.  Fusion was heading in the direction to eliminate what Intel’s Pauline Nist called on theCUBE, “the horrible storage stack,” meaning spinning disk and storage protocols are painfully slow. Fusion-io set out to completely change the storage game.
4. Meanwhile it became clear that storage function (which had been moving away from the server for fifteen years) was starting to move back to the server.
5. Storage giant EMC, realizing that it was way behind Fusion-io; and realizing that you can’t win by just stuffing fast flash cards into a conventional RAID controller, initiated Project Lightning – a version of the Fusion-io card that hit the market about 4 years after Fusion-io started shipping. EMC at that time also announced Thunder, a cache coherent distributed flash product designed to freeze the market on Fusion-io and attack it’s main weakness – i.e. the Fusion-io card is a single point of failure.
6. NetApp, realizing it was way behind, did a deal with Fusion-io as a time-to-market move.

Battle for the New Storage Stack

Here’s the problem. Disk is painfully slow. It’s the only mechanical bottleneck in systems. Storage capacity has been doubling every 18 months for decades but storage performance hasn’t increased much at all. Sure systems designers insert caching and smart algorithms where possible but generally as data grows, storage becomes more of a bottleneck.

Flash changes everything. Because we now have a persistent medium on the other side of the channel (i.e. internal to the server complex) we have the potential to dramatically increase application performance. Fusion-io wants to do this by completely changing the storage game by reaching around the storage stack and developing a new way to perform writes and manage storage. EMC, of course, wants to keep the status quo as long as possible and drive as much margin out of its legacy RAID products as it can. The wildcard for EMC is it owns VMware (more on that in a minute).

As Wikibon’s David Floyer has pointed out numerous times, “you can’t manage fast servers from slow disks arrays.” But that’s exactly what virtually every storage company has tried to do. EMC’s Fully Automated Storage Tiering strategy essentially tries to make the disk array the point of control by “sending hints” to the server. NetApp for its part has resisted flash and basically had a flash cache strategy within its storage arrays. It’s the wrong approach and the storage world is finally waking up to this.

The shot across the bow was when Pat Gelsinger was named CEO of VMware. As Wikibon’s David Floyer points out:

One simple strategy for Pat Gelsinger would be to introduce EMC VFCache and VMware kernel APIs to drive a top-down data acceleration and management strategy. This tight integration could provide atomic write capabilities to the flash card and management of metadata and the total storage stack, enabling both transactional and analytic big data applications. More APIs would allow key file and database systems to integrate with VMware and flash and provide a seamless integration to the shared flash-only storage array layer based on the XtremIO acquisition. This goes right at the heart (for example) of Fusion-io’s value proposition. Despite Fusion-io’s enormous lead, we believe VMware will move down this path. The potential performance of these systems would be in billions of IOs per second, enabling a new generation of applications with far higher data integration. The result would be applications that deliver far greater productivity to users and better utilization of resources. This IO-Centric era would also result in higher overall spending on IT.

So what’s happening here is Fusion-io is trying to become the next Veritas of storage – an independent storage company managing a new and emerging storage stack. For it’s part, EMC is going to use VMware to try and counter these moves, probably in partnership with Intel, Gelsinger’s former company. Meanwhile, NetApp seeing that it was falling behind in flash had to make a time-to-market move or risk getting edged out of the conversation with CIOs and the channel.

So what’s this new emerging storage stack look like?

Most of the innovation will be on the server side, with persistent, high availability storage as close to the server as possible. You’re seeing the next layer as a persistent flash cache at the server level (like Fusion-io’s card and EMC’s VF Cache). The next piece is a metadata layer for files and blocks. This will be the key to the future. Whoever owns the metadata management will win the game because they will control the placement of data. Suffice to say that layer will be controlled by fast servers, not slow storage arrays. The next layer down will be a shared high speed persistent distributed caching layer, connected by some type of server to server interconnect such as infiniband. This is similar to EMC’s Thunder, IBM’s BlueHawk and other distributed caching systems which will hit the market later this year. And my bet is Fusion-io will probably get to market first in this space, despite EMC and IBM’s pre-announcements. Next you’ll see all-flash arrays emerge like Kaminario and XtremeIO (the company EMC purchased for $400+M). Then Hybrid arrays such as Datacore, Tintri and Nimble and finally – the “bit bucket” layer of the stack – traditional RAID and JBOD.

Fusion-io will largely partner (with guys like NetApp) for this lowest layer. The rest of the stack is Fusion’s sweetspot with the IO Turbine acquisition and future innovations filling the gaps.

So there’s no mystery in this announcement. NetApp is playing catchup and trying to remain relevant to the channel and to CIOs. Fusion-io is trying to compete at all levels of the stack and will partner with anyone that is willing to help it expand the TAM and at the same time take on EMC. Meanwhile, EMC maneuvers to use it’s greatest asset, VMware to exercise its dominant position.

The key for EMC will be integration with VMware and demonstrating it can run any application, any time. The key for Fusion-io will be to get software ISVs to write to its standard and continue delivering more function faster than the other guys (including EMC and Intel).

Watch this space – the storage landscape is changing.