Enterprises need practical solutions when it comes to artificial intelligence implementation.

As organizations look to move AI projects from proof of concept into production, navigating the challenges of integrating AI across an entire organization is becoming a priority. The hardest part? Making all of an organization’s data accessible and usable.

“Businesses are in the business of business,” said Nicola Tan (pictured, right), director of market development, Enterprise AI, at Advanced Micro Devices Inc. “They’re not making models. Unlike hyperscalers, they’re trying to maximize their core businesses. Everyone has access to ChatGPT or Claude or open-source models. So, the model itself is table stakes. Really where the value for enterprises is in their data.”

Tan, alongside Junxia Zhou (left), senior product manager at Super Micro Computer Inc., and Mayank Gupta (middle), director of solutions marketing at Nutanix Inc., spoke with Rob Strechay, for the Supermicro Open Storage Summit interview series, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed their collaboration and the practicalities of AI adoption. (* Disclosure below.)

Moving beyond proof of concept

POCs are not a roadmap for AI implementation, according to Zhou. Businesses need to think holistically and figure out how their entire organization can be oriented toward the infrastructure and data requirements of production AI. Supermicro provides GPU systems powered by AMD technology while working with Nutanix on software-defined infrastructure and management capabilities.

“The production and the pilot are totally different,” Zhou said. “[Pilots are] just designed for a single use case, a small data set and serve for a small number of team members. But for the production, the customer may need to support thousands of end users that generate lots of inference requests.”

Other roadblocks to moving AI into production include token costs and fragmented data. An organization’s data might reside across many different systems, while existing software and data layers may not be equipped for the demands of agentic AI.

“What we are realizing is people’s software layers are not ready to provide and connect the agentic environment,” Gupta said. “While a regular chatbot query would have consumed a thousand or a hundred tokens, agents are talking to agents, to data, to other agents, to models, and … they consume thousands of tokens within a fraction of seconds.”

One solution, which Supermicro and its partners facilitate, is a two-tier storage architecture. Clients can use flash memory (SSDs) to store active data sets and large-scale object storage to store archival data that is less frequently accessed. The approach can balance high-performance data access with the cost efficiencies of capacity-optimized storage.

“From Supermicro’s perspective, the performance always starts with keeping the expensive GPU resources fully utilized,” Zhou said. “Today’s infrastructure is not constrained by hardware availability, it’s constrained by how quickly data can be delivered to the GPUs. That’s why we are focusing on optimizing the entire AI stack. Compute performance is not enough. Data storage and networking are becoming a very critical part of AI pipelines.”

Secure AI implementation

Autonomous agents have raised new security and governance concerns because of the level of access they can have to enterprise systems and data. Data needs to be visible to AI, but at the same time sensitive data and workloads may need to remain within controlled environments.

“Security is paramount, especially as agents are … accessing data at a speedy pace and they’re consuming tokens at a pace which has never been seen before,” Gupta said. “From my perspective, [and] a CEO perspective, governance is very important. How these agents talk to each other is very important. From the CFO perspective, the token costs [are most important.]”

Companies are increasingly seeking sovereign AI solutions because of security concerns and a desire to protect proprietary data. Organizations are also looking to deploy AI models and agents on-premises, although the speakers highlighted the importance of connecting to hybrid and edge environments.

“The differentiator for enterprises is their sensitive data,” Zhou said. “Enterprises do not feel comfortable with sending their sensitive data, like intellectual property, customer information, into external AI providers. There is a growing demand for sovereign AI and private AI because enterprises want to build their AI infrastructure closer to where their data is located.”

Although private AI environments can play an important role in that strategy, Tan emphasized the value of having an “open strategy” that encompasses open-source solutions, open standards and an open platform. The ability to flexibly move between hybrid environments could give companies greater choice over where AI workloads run.

“I’ve been really impressed with local models, open weight models that are delivering really near frontier intelligence,” she said. “You don’t necessarily have to do a trade-off of a smart model here or a dumb model here. You can get almost as good as frontier level intelligence and now run that on-prem. More and more, you can bring a lot of those capabilities in house.”

Stay tuned for the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Supermicro Open Storage Summit interview series.

(* Disclosure: TheCUBE is a paid media partner for the Supermicro Open Storage Summit interview series. Neither Supermicro, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE