As enterprises move AI initiatives from proof of concept to production, attention is shifting toward modular AI infrastructure that can simplify deployment and scaling. Controlling costs and simplifying operations are emerging as the defining challenges of enterprise AI adoption.

The central challenge for enterprises is making the jump from proof of concept to production. While AI projects often work well with a small group of users, scaling them to enterprise-wide deployment introduces a new set of challenges, according to Varun Chhabra (pictured), senior vice president of product marketing and infrastructure solutions group at Dell Technologies Inc.

“One is, the cost of native token-based pricing is just out of control. It can go really, really out of control, especially as your top users start to see big, big, big value from the workflows,” Chhabra said.”How do you take control of that? How do you budget for that?”

Chhabra spoke with theCUBE’s Dave Vellante at the AMD Advancing AI event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the challenges of scaling AI from proof of concept to production and the growing need for modular AI infrastructure to support enterprise deployments. (* Disclosure below.)

Modular AI infrastructure for scale

In addition to the cost of native token-based pricing, another challenge involves data. Beyond placing compute close to enterprise data, organizations must establish governance policies and ensure AI models are fed the right data through technologies such as RAG pipelines.

“The third one, security and governance. When you scale, how do you make sure that you’re thinking about unintended consequences?” Chhabra said. “All of these things, I think, are part of the recipe that you need to be able to scale enterprise workloads for [generative] AI.”

As inference workloads continue to grow and consume an increasing share of AI compute, enterprises are exploring on-premises deployments to become their own token generators instead of depending on cloud-based APIs. Rather than relying on a single approach, large enterprises are expected to use different AI infrastructure and deployment models for different workloads.

“What we’re finding is customers want modular solutions. They want to think about across the whole platform,” Chhabra said. “Compute, storage, networking, GPU. The software framework on top of it, the models, have they all been tested, have they all been validated?”

Dell and Advanced Micro Devices Inc. designed the Dell AI Platform with AMD to let customers start with small AI deployments and scale on the same platform without rearchitecting their infrastructure, according to Chhabra. The modular AI Factory is part of the approach.

“You can start small with these composable units of storage, compute, networking, pre-integrated with AMD Instinct and EPYC CPUs with the ROCm software as well as the open ecosystem that sits on top of it,” Chhabra said. “We test it, we validate it, and the customers can use it and once they start seeing value in it, they know they can scale in a modular way with the same frameworks.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the AMD Advancing AI event:

(* Disclosure: TheCUBE is a paid media partner for the AMD Advancing AI event. Neither AMD, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE