Storage experts often say that NAND flash is a disruptive technology. Certainly flash is much faster than disk, but to be disruptive, it needs to be more than a faster disk drive. So I asked Wikibon co-founder and CTO David Floyer how flash will disrupt the data center.

“My vision of the flash architecture is complex,” he said. First, the order-of magnitude speed increase in flash mandates a total rearchitecting of the way data is processed. As processing power has increased and networking bandwidth has grown, spinning disk has become the gating factor of system performance. Applications were written around the limits of disk speed and I/O, controllers were designed to work at the pace of disk and strategies like caching data in memory were developed to minimize its impact.

All that changes with all-flash arrays. Flash storage is nearly as fast as processor speed, and the new gating factor is the network and the speed of light itself.

To get the optimal performance, storage and processing, which have been separate for decades, are being reunited. All-flash arrays need to be as close as possible to the servers to minimize lag times, as detailed in Floyer’s recent analysis presenting the Flash as Memory Extension (FaME) architecture. That, he says is a higher level view of ServerSAN.  The purpose is to get the storage close enough to the servers to connect with a PCIe bus rather than a storage network. But increasing amounts of flash will also move into the server, onto the memory bus and as close as it can get to the processor. Eventually it is not impossible that all storage for some applications will move back into the server.

This has important implications for the IT organization as well as the physical architecture. The locus of storage control will move from the storage arrays to the server. And many of the technical strategies developed around spinning disk will become obsolete as storage becomes a slightly slower but persistent form of memory. The only really separate storage will be the data archive, which may run on disk and tape.

The organizational implication of this is that the separate storage and server groups in IT will be combined to reflect the physical and logical integration of storage and server. Applications will also have to be rewritten or replaced to take advantage of the order-of-magnitude increase in performance of flash over disk.

Less is more

Flash also provides huge amounts of I/O, which means that today’s common practice of making multiple copies of databases to support non-production operations such as data analysis and application development will be largely eliminated. The huge I/O gain, says Floyer, will allow these operations to use logical copies of the data that draw directly on the production database without slowing transactional performance. This will allow the users to access the latest business data and eliminate the need for additional infrastructure to accommodate multiple database copies. This will bring down the total cost of flash to a leverl that is already competitive with disk. Over time, flash will be cheaper.

The real revolution, however, will be in how flash will allow businesses to do things that are impossible with spinning disk, Floyer says. For example, corporate leaders will be able to check sales every morning to identify developing trends and take action immediately rather than having to wait for a monthly or quarterly report that comes long after any chance of heading off a problem has passed. Another possibility may be combining data from multiple physical databases, both internal and external, in a single logical database running on a NoSQL engine for in-depth business analysis.

All of this will revolutionize not just IT but the business itself, which is why Floyer maintains that flash is one of the major disruptive forces in business today.