Artificial intelligence data platform startup Vast Data Inc. today announced new AI cloud architecture in partnership with Nvidia Corp., using the company’s BlueField-3 data processing unit technology to host Vast’s operating system and turn supercomputers into AI data engines.

The company also announced a partnership with Super Micro Computer Inc., a major global supplier of computing hardware solutions for AI, cloud and storage that will help simplify the deployment of full-stack, end-to-end AI solutions at scale. Through this collaboration, Vast Data and Supermicro are offering large cloud service providers the ability to build data-centric AI-powered solutions powered by systems embedded with Nvidia DPUs.

The Nvidia BlueField networking platform provides a software-defined accelerated computing infrastructure for AI by combining compute power and integrated hardware accelerators to secure networking environments. By embedding Vast’s operating system into each DPU running in a graphics processing unit server, Vast can make it possible to parallelize its services to embed storage and database at extremely large scale.

Vast’s Data Platform is a software infrastructure framework that captures, catalogues and preserves unstructured data for AI processes, which need to synthesize vast amounts of data for training, fine tuning and during deployment in order to process queries from users. Putting Vast’s services onto the BlueField DPUs businesses can greatly accelerate the movement of data between AI processing and storage, John Mao, vice president of business development and technology alliances at Vast Data explained to SiliconANGLE in an interview. This comes with a large number of benefits for training, deploying and scaling AI applications in data centers, which require large clusters of GPUs for AI training and deployment purposes.

“The benefit for most organizations is that it radically reduces the amount of equipment required,” Mao said. “So, there’s obviously context savings. But I think the more interesting thing is that these big GPU clusters if you’ve been following, they’re very power-hungry to take up a lot of power in data centers. It’s not it doesn’t sound like a lot where we did a, we did an analysis that looks at basically about 5% of total datacenter power savings.”

That may not sound like a lot, Mao said, however many of these data centers run on multimegawatt scales. That means a 5% savings can save enormous amounts of power over months or a year.

The other big benefit, he said, is that by putting the software onto BlueField DPUs, it’s easier for AI data center developers to scale out quickly without needing to reengineer anything. “If you start with something relatively small, a few hundred GPUs, as you add 1,000 GPUs and then 10,000 GPUs, this new architecture allows those clusters to be able to scale performance of those new storage services completely linearly with no thinking and no rearchitecting,” Mao said.

Vast is already working with CoreWeave Inc., a GPU-accelerated cloud compute infrastructure provider that focuses on AI training and deployment. It’s using Vast’s operating software on Nvidia BlueField DPUs in production.

The company also announced a partnership with Supermicro that will provide full-stack solutions for large-scale AI infrastructure deployments, and Mao said for the first time a Vast Data Platform offering that will run on industry-standard servers. This has been a paradigm shift for the company, since it was running on more bespoke hardware. It was created to host the Vast software, but now, with industry-standard hardware, customers can manage their own supply chains for much larger scales.

Over the last year, Mao said, customers have been making deals with Vast in the petabyte and exabyte deployment sizes, and in some cases, the multiple exabyte scales. At these scales a customer isn’t worried so much about a single server failing anymore. They’re much more concerned about an entire rack – an entire group of servers – failing all at once.

“Their concept of what they’re optimizing for is different,” said Mao. “They also don’t have an appetite — those customers running at that scale — they need to be able to optimize their own supply chains from a hardware perspective.”

This will come in quite handy for companies as well because the Supermicro offering will come with the BlueField DPUs on board, since Nvidia doesn’t just sell them separately. They’re part of the technology stack that ships with the clustered GPU servers. That means hyperscale data center providers working with Vast will be able to take advantage of that solution as well, said Mao.

Image: Pixabay