In his latest article on how persistent flash memory will disrupt the IT architecture in the era of real-time, IO-centric big data processing, Wikibon CTO David Floyer looks at the issue of “Real-time IO Centric Processing for Big Data.” The article focuses on a comparison of DRAM, popular in very high value, real-time applications such as currency trading, with persistent flash memory. He finds that DRAM remains faster in overall IO and will hold onto its existing high-demand markets. However, it will remain too expensive to compete with flash memory for the developing market for near real-time big data capture and analysis.

This article, built on his seminal paper on “Designing Systems and Infrastructure in the Big Data IO Centric Era,” he finds that DRAM has several disadvantages that will keep its costs much higher than flash, including that it is unlikely to attract wide-scale consumer market applications to drive its base price down; its power protection systems are bulky and add expense and complexity; and it is byte addressable which makes it expensive to scale. Flash, by comparison, is experiencing steady price decreases driven by its heavy use in consumer mobile devices, does not require power backup, and is block oriented, which makes it much easier to scale.

The one disadvantage of flash versus DRAM is IO speed. However, Floyer says, flash is fast enough to meet the needs of near real-time big data analysis applications, and the large extra cost of using DRAM, and the complexity of maintaining its battery backup, make it an impractical alternative.

However, Floyer does not expect flash to invade DRAM’s established transactional systems market. Some applications, particularly in the financial industry, demand the absolute speed advantage of DRAM, despite its higher cost. He suggests that some real-time systems may evolve into hybrid DRAM-flash environments.