For a year Wikibon CTO David Floyer has argued that flash memory is a disruptive technology. But to be disruptive a technology needs to make entirely new processes or applications that have major real-world implications possible, not just make existing ones faster or easier. In his latest in-depth technology cost analysis, “Designing Systems and Infrastructure in the Big Data IO Centric Era,” Floyer documents exactly that – how flash memory, in this case the Fusion-io PCIe card – enables near-real time analysis of huge amounts of data from automated sources for a variety of valuable applications. He argues that this marks the start of what he as named the Big Data, IO-Centric Era, not just in technology but in business, medicine, and government and, in a larger sense, society and culture as a whole.

Floyer focused on the demonstration earlier this month by HP and Fusion-io of a system that reached 1 billion IOPS using eight servers and 64 flash PCIe cards. His analysis shows that this configuration, in which huge volumes of individually small chunks of real-time data are written straight onto persistent flash storage cards treated as an extension of memory in a non-locking system, reduces the cost of the total system by an order of magnitude – from $34 million using a traditional IO software stack with PCIe flash to $3 million. For comparison he also priced theoretical equivalent systems using disk and SSDs at $220 million and disk only at $2,276 million. Obviously while technically possible using these older systems, this has been made practical for large numbers of applications for the first time with the HP/Fusion-io architecture, which eliminates both disk and the traditional IO software stack to boost write ioDrive performance from 1 million to more than 15 million IOs a second.

This makes real-time analysis of a whole new class of applications possible, Floyer says. Real-world applications include:

1. The capture and analysis of real-time performance data from large populations smart machines such as refrigerators, washing machines, driers, etc., to identify developing problems early, allowing service providers to intervene before the unit fails.
2. Real-time monitoring of tens of thousands patients with smart implantable medical devices such as heart defibrillators to identify early signs of heart or other medical emergencies and literally save people’s lives.
3. Monitoring of large transportation system such as railroads to identify problems ranging from “hot boxes” (locked air brakes on a freight car) to incipient failures of a track joint to head off derailments.
4. Analysis of credit card use over large populations to identify fraud.

All of these and others are already in use in small scale experiments but need the kind of very large scale, near real-time, atomic write data capture that persistent flash memory provides in this configuration for the first time. The implications, Floyer argues, are staggering.