Data center infrastructure provider Pure Storage Inc. says it’s stepping up to take care of the industry’s rising demand for high-performance storage systems that can scale to keep up with the most advanced artificial intelligence models.

The company today announced a new and dedicated data storage software platform called FlashBlade//EXA, saying it’s designed to solve the “metadata bottleneck” problem that holds back a growing number of enterprises from scaling up their AI workloads.

Designed for high concurrency, FlashBlade//EXA is uniquely able to scale regular data and its associated metadata independently. Even better, it works with any kind of storage array, while deployment is simplified through the use of standard protocols and networking.

The company is trying to fix a major bottleneck in data center storage today amid the realization that legacy storage systems are simply unable to process data for AI workloads efficiently. Storage needs to be able to keep up with the computational intensity and volume of the graphics processing units that process the data needed for AI, but existing systems have been left wanting. The issue is that they face major limitations in terms of parallel and concurrent reads and writes.

The metadata bottleneck

Existing storage systems have been optimized for traditional application environments, where the workloads are more predictable, so the focus has been on scaling up raw performance. But the vast majority of new AI workloads are more complex and require multimodal data, including text, images and videos that must be processed simultaneously by thousands of GPUs all working together.

As such, AI workloads require not just massive performance, but also a massively parallel architecture that can keep pace with the underlying metadata associated with those massive volumes of multimodal data.

The problem stems from the fact that AI simply doesn’t look anything like traditional enterprises workloads, which are what almost every storage array is designed to address today, said Steve McDowell of NAND Research Inc.

“AI requires a massively parallel architecture with a global namespace that can serve data up to thousands of GPUs simultaneously,” he explained. “You never want to starve an expensive GPU of data. It requires a fundamentally different approach.”

McDowell said it’s a problem that’s well-understood by every player in the storage industry, with the likes of Dell Technologies Inc. and NetApp Inc. announcing solutions that intend to address it. But with FlashBlade//EXA set to launch in the summer, it looks like Pure Storage has beaten them in the race to bring its product to the market.

Built for AI training

The company explained that FlashBlade//EXA is purpose-built to address AI workloads, offering unmatched performance alongside sophisticated metadata management capabilities. With its disaggregated, massively parallel architecture, it solves the problem of scaling up AI workloads, eliminating idle GPU time so enterprises can accelerate AI training and inference.

What’s most surprising is that Pure Storage has made FlashBlade//EXA compatible with third-party data nodes, which means customers won’t have to buy the company’s expensive, flash-based storage arrays to benefit from it.

“This will make adoption easy for hyperscalers, specialty-cloud providers and big enterprises that already have a heavy investment in storage,” McDowell said. “It’s a smart play for Pure that blends the best of hardware and software-first approaches. It also teases a future where Pure is as equally focused on selling into the hyperscale world as traditional enterprise.”

Customers have not yet had a chance to verify FlashBlade//EXA’s performance capabilities, but Pure Storage said preliminary tests by early adopters show it can deliver more than 10 terabytes per second of read performance in a single namespace, dramatically improving on what traditional systems are capable of.

“With FlashBlade//EXA, Pure is building a system that separates out metadata handling from the data itself,” McDowell said. “This looks like a system built for AI training and it’s delivering some stellar initial performance numbers in the process.”

Photo: Pure Storage