AI storage infrastructure is becoming as strategically important as compute in the next phase of the industry’s evolution. As models grow larger and workloads spread across more environments, the industry is moving toward data architectures designed to deliver higher performance without letting cost and complexity spiral.

In AI infrastructure, every enterprise is searching for an edge that can squeeze more value out of every layer of the stack. The upside of better storage is not just solving shortages, but unlocking stronger economics by helping GPUs run more efficiently and improving margins, according to Brennen Smith (pictured, right), chief technology officer of Runpod Inc.

“We use [Vast Data Inc.] very heavily — that increases our margins by 12%. That’s a very repeatable playbook. It’s very consistent in our data,” Smith said. “GPUs make more money when they have storage attached to them, and that’s because it unlocks so many different use cases that realistically are very difficult to achieve otherwise.”

Smith, Andy Pernsteiner (center), field chief technology officer of Vast Data, and Ace Stryker (left), director of AI and ecosystem marketing at Solidigm, a trademark of SK Hynix NAND Product Solutions Corp., spoke with theCUBE’s John Furrier at the Nvidia GTC AI Conference & Expo, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how AI storage infrastructure is shifting from an afterthought to a core part of modern AI systems, along with the broader push toward denser and more distributed architectures. (* Disclosure below.)

AI storage infrastructure in focus

As AI becomes more widely adopted, organizations are realizing they need access to a much larger share of their data to generate useful insights. That shift is driving demand for infrastructure that can store that data in a way that is fast and cost-effective, according to Pernsteiner.

“We rely on our partners to make sure that we’re getting the best-in-breed, not only from an efficiency standpoint for cost, but also from a density standpoint,” Pernsteiner said. “Because part of the layer cake is power, and people don’t want to spend money on power if they don’t have to.”

Solidigm’s launch of a 122-TB SSD last year showed how quickly storage density is advancing, with the company announcing its ambitions to double that in the near future. That jump in capacity can improve energy efficiency while easing rack space constraints and reducing infrastructure complexity, according to Stryker.

“Whether it’s the stuff that lives in a GPU server where you care about really high bandwidth and getting that hot data served to the GPUs as quickly as possible, or whether it’s more of a shared storage across the network where density is the name of the game, Solidigm’s always worried about those things and about how folks are using these kinds of solutions in the real world,” Stryker said.

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Nvidia GTC AI Conference & Expo: 

(* Disclosure: Solidigm, a trademark of SK Hynix NAND Product Solutions Corp., sponsored this segment of theCUBE. Neither Solidigm nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE