AI infrastructure clouds have multiplied as GPU capital pours into new data centers, but standing up capacity is proving easier than running it at scale. With Kubernetes now hardened into the operating layer for AI factories, the cloud-native community holds expertise that many of these new providers lack.

KubeCon has been shifting toward an AI infrastructure event, with container orchestration for AI and distributed inference now central to the conversation. The newer AI clouds run on the same computing fundamentals as the hyperscalers before them, yet many specialize narrowly and lack a bridge to the enterprise systems that will ultimately consume their capacity, according to Sarbjeet Johal (pictured), chief executive officer of Stackpane.

“I’m afraid that the neoclouds, if they don’t partner with the ones who are appreciating the brownfield, they will suffer,” Johal said. “They can’t survive on their own because, who is going to take that compute from them? It’s the hyperscalers. The neoclouds are the overflow.”

Johal spoke with theCUBE Research’s John Furrier and Rebecca Knight at KubeCon + CloudNativeCon NA, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the event’s first-day takeaways, from AI-written code and token economics to the maturing neocloud market and the skills needed to run it. (* Disclosure below.)

Software chops will sort the winners among AI infrastructure clouds

Kubernetes spent more than a decade evolving in fits and starts, and the payoff is now showing up in AI. The Cloud Native Computing Foundation added a dedicated AI inference and agentic track to this year’s KubeCon North America schedule, a sign of where practitioner attention has moved. Kubernetes now runs across about two-thirds of the enterprise, sits inside Nvidia Corp.’s reference architectures and underpins the neoclouds, Furrier noted. Skills honed through years of operating Kubernetes are now being applied directly to the commercial buildout of AI factories, where most of the capital is going.

“Now it’s hardened. So when you have hardened Kubernetes, we’re seeing inference being talked about,” Furrier said. “Who would have thought at this show inference would be discussed? Well, guess what? Distributed inference is the hottest area right now.”

That maturity matters because so much AI capital spending is flowing to new providers. SoftBank Group Corp. unveiled plans to enter the U.S. neocloud business this summer, joining a crowded field. Many existing operators started out building data centers for bitcoin mining or arbitraging spot pricing rather than running large-scale software, according to Furrier, who sees a reckoning ahead. The hard part is less about acquiring GPUs than operating them as a true cloud.

“Half the neoclouds will fail,” Furrier said. “To run a big neocloud, you’ve got to look like CoreWeave. Well, guess what CoreWeave is? The founders know data centers. … People who know large-scale software stacks at scale win.”

The rescue, in Furrier’s view, will come from the cloud-native community. He pointed to CoreWeave Inc., whose engineering is run by former Google Cloud LLC executive Chen Goldberg, as the model for pairing data center know-how with large-scale software skills. He also suggested private equity investors could buy smaller Kubernetes companies to fill the gap. Those companies could command billion-dollar valuations as data center owners with land, power and shell seek help with connectivity, multigeneration Nvidia hardware and workloads that span data centers, he explained. Johal, who watched a similar story play out at Rackspace Technology Inc., agreed that software will decide which providers last.

“We have seen this movie earlier. So if you want to be a good hyperscaler, if you don’t have the software chops, you’re dead,” Johal said of the new AI infrastructure clouds. “We knew how to run data centers at Rackspace, but we didn’t have software chops. … Without software chops, you’re dead in the water.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of KubeCon + CloudNativeCon NA:

(* Disclosure: TheCUBE is a paid media partner for KubeCon + CloudNativeCon NA. Sponsors of theCUBE’s event coverage do not have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE