Unpredictable demand, specialized hardware, production-scale complexity — all of it is making AI inference harder to run at enterprise scale. Now, cloud-native open-source infrastructure is emerging as the answer to inference chaos.

That shift is already showing up in the Kubernetes ecosystem. In fact, the Cloud Native Computing Foundation has almost doubled the number of approved platforms in its Kubernetes AI Conformance Program, following an over 70% surge in certified offerings, according to Jonathan Bryce (pictured), executive director of cloud and infrastructure at the Linux Foundation. The program creates open, community-defined standards for running AI workloads on Kubernetes, and as organizations increasingly move those workloads into production, they need consistent and interoperable infrastructure.

“AI is going to be something that is [going to] drive the next 10, 20 years of technology, the way that cloud did the last 10 or 20 years,” Bryce told theCUBE, SiliconANGLE Media’s livestreaming studio. “But it’s also a very different workload than what we’ve ever run before. It requires specialized hardware. The usage patterns are super unpredictable … When you talk about bursty [demand spikes] and AI, you could need a thousand times as much capacity and then it goes away.”

Bryce spoke with theCUBE’s Rebecca Knight and Rob Strechay at KubeCon + CloudNativeCon EU 2026, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how the CNCF is evolving cloud-native open-source infrastructure to meet the demands of AI inference at global scale. (* Disclosure below.)

Cloud-native open-source infrastructure steps up for AI

The CNCF is tackling the AI infrastructure challenge on different fronts: evolving existing projects such as Kubernetes with dynamic resource allocation and an inference gateway, onboarding new projects such as llm-d for horizontally scaled inference and integrating with open-source AI tools such as PyTorch and vLLM, Bryce explained. Those three prongs reflect a broader push to make AI infrastructure more operational.

“We’re bringing the best — the innovation that’s coming out of the AI experts’ work — and we’re turning it into something that we can operate,” Bryce said. “We’re kind of bringing it out of the lab and turning it into a factory, and we’re doing it in the open.”

AI inference is projected to jump from 20.9 gigawatts in 2025 to 93.3 gigawatts by 2030 — surpassing training to become the dominant workload in AI data centers. But inference demand could exceed all current data center workloads combined by 2030, Bryce noted. That pressure motivates the search for smaller, more specialized models that can deliver better economics at scale.

“When I have talked with companies who are creating specialized models, they’re seeing that they’re way faster, way more accurate and way cheaper,” he said. “If you can increase efficiency, then you can really get to that ROI and you can bring forward AI adoption.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of KubeCon + CloudNativeCon EU 2026:

(* Disclosure: The Cloud Native Computing Foundation sponsored this segment of theCUBE. Neither the CNCF nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE