Enterprise experimentation with cloud AI has run into a wall of data-control, sovereignty and token-cost questions, pushing intelligence back toward infrastructure enterprises own. The result is a rebuild of the private cloud as an AI platform, where production inference, not experimentation, sets the requirements.

That shift lands squarely on information technology operations teams, which must serve frontier models, small local models and swarms of agents from the same pool of hardware. Mixing those workloads without driving up server, energy and licensing costs is now the central architectural problem, according to Chris Wolf (pictured), global head of AI and advanced services, VMware Cloud Foundation Division, at Broadcom Inc.

“You have sovereignty considerations. You have tokenomics considerations as well. This doesn’t mean don’t use frontier models. It means be practical,” Wolf said. “Use frontier models where it makes sense, where [you] need deep reasoning. Use specialized models, local SLMs, where they make sense as well. You’re really seeing this breadth of coverage happening in the industry — and now IT operations is caught in the middle of all of this.”

Wolf spoke with theCUBE’s John Furrier at VMware Explore 2026, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the move from private cloud to private AI cloud, AI factory operations, sovereignty and how enterprises should plan the next 18 months. (* Disclosure below.)

Cloud AI economics put software ahead of hardware

Agentic workloads have added new demands to that mix, including warm pools of isolated virtual machines that can spin up agents without allowing escapes or privilege escalations. Memory is the other constraint, spanning key-value cache placement across graphics processing units and tiering across storage classes, which is why Broadcom has been positioning VMware Cloud Foundation as the pooling layer. But too many buyers still start at the wrong end of the stack, Wolf said.

“‘Buy your hardware first, figure out the software later’ — no, that’s a horrible idea, because you have to make sure that your software choices are compatible with the hardware you bought,” he said. “Software is what’s giving you the ability to have autonomy in terms of the accelerators you use, to have the flexibility to ensure that I can use cloud models when I need to [or] use local models when I need to.”

That gap is what Broadcom is targeting with its AI factory approach, which provisions from bare metal through model runtimes and then exports a YAML file to clone additional clusters. Customers arriving from earlier deployments tend to describe the same experience, Wolf noted.

“People were running into buyer’s remorse,” he said. “They bought what they thought was this full turnkey solution, and as it turns out, it wasn’t.”

Sovereignty has become the other driver, with governments in North America, Europe and Asia demanding localized models, data planes, encryption keys and control planes. For enterprises weighing cloud AI against on-premises builds, the idea is to slow down before committing, Wolf explained.

“More than ever, they have to architect for the expectation of change. They can’t architect based on what looks good today, because the space is moving too fast,” he said. “Make sure software is at the forefront of your architecture and decision-making, and then go from there.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of VMware Explore 2026:

(* Disclosure: TheCUBE is a paid media partner for the VMware Explore 2026 event. Neither Broadcom, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE