For years, cloud computing’s entire pitch was “forget about hardware.” Kubernetes doubled down on that promise, abstracting infrastructure into something developers could safely ignore. But AI workloads don’t play by those rules. Inference engines, agentic systems and foundation models are pulling hardware back into the conversation — and this time, ignoring it isn’t an option.

At KubeCon + CloudNativeCon, that tension surfaced in nearly every conversation. Attendees kept circling back to the same conclusion: Scaling AI inference, training pipelines and agentic workloads requires confronting hardware constraints that cloud promised to eliminate. The demand for inference engines is surging across customer-facing apps and developer tools, putting pressure on latency, costs and accelerator management, according to Joep Piscaer, analyst at TLA Tech B.V.

“We’ve gone away from hardware; everything was cloud,” he told theCUBE. “We’re now in a place where we have to consider 400-gig networking because the models need stuff like that.”

Piscaer, Ned Bellavance, independent consultant and technical educator, and industry analyst Rob Strechay spoke with Savannah Peterson at the KubeCon + CloudNativeCon NA event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how Kubernetes is evolving to handle AI workloads and the infrastructure demands reshaping cloud-native architectures. (* Disclosure below.)

Here’s theCUBE’s complete video interview with Savannah Peterson, Rob Strechay, Joep Piscaer and Ned Bellavance:

Check out three insights you may have missed during theCUBE’s coverage of KubeCon + CloudNativeCon NA:

Insight #1: Sovereign control and zero-trust architecture secure AI workloads.

Data sovereignty is emerging as the foundation for enterprise artificial intelligence, not only for compliance, but as the control layer that keeps innovation grounded in security and governance. Red Hat Inc.’s OpenShift AI strategy addresses this need directly, providing full-stack oversight from platform to containers while ensuring workloads remain sovereign whether running on-premises or across clouds, according to Red Hat’s Shane Utt (pictured, left), senior principal software engineer, and Jimmy Alvarez (right), senior principal product marketing manager.

“Security is perennial; it’s the job that’s never done,” Utt said during the event. “We’re pretty convinced that quantum is going to happen in the next five-plus years. We’re doing PQC, post-quantum cryptography, across the entire stack.”

That forward-looking security posture extends into zero-trust frameworks, where continuous verification replaces assumption. Red Hat is applying zero-trust principles to AI workloads, ensuring that users, machines and agents are authenticated at every layer — particularly critical as agentic systems introduce new attack surfaces, according to Red Hat’s Anjali Telang, senior principal product manager of OpenShift Security and Identity, and Roman Zhukov, principal security community architect.

“We hear these so-called digital sovereignty concerns,” Zhukov told theCUBE. “Everybody wants to make sure that the technology that they build and use, they can control them, and this notion expands to AI as well. That’s why technologies like confidential computing, for example, come into play. Confidential computing is all about securing data in use when you can protect your workloads while in use, so nobody — including the cloud providers or administrators  — can access your data because it’s encrypted.”

Here’s theCUBE’s complete video interview with Shane Utt and Jimmy Alvarez:

Insight #2: Infrastructure economics demand self-service operations and cost optimization.

Self-service infrastructure is reshaping how enterprises support developers at scale, most notably in research and development environments where thousands of engineers need resources without waiting on tickets or risking downtime. Nvidia Corp.’s platform teams use Portworx to deliver multi-tenant data management across Kubernetes clusters, enabling teams running AI workloads — from chip design to firmware development to AI training — while maintaining availability, according to Brian Monroe, senior software engineer at Nvidia, and Venkat Ramakrishnan, vice president and general manager of Portworx by Pure Storage.

“We need to be able to … take down a cluster, do maintenance,” Monroe said during the event. “We need to be able to shift our workloads. We try to generally do a zero-downtime maintenance, so we basically would take down one node in a cluster, do the upgrades, things like that. We shift the workloads around … the Portworx storage infrastructure with replication spread across multiple nodes [that] allows us to move those workloads around in various locations without having to worry about taking down a specific business process or function.”

Price-performance economics are becoming just as critical as operational agility, especially as enterprises balance AI workload demands against infrastructure costs. Vultr’s latest cloud compute series, built on American Micro Devices Inc.’s architecture, delivers significant cost advantages while maintaining data sovereignty, offering 82% better performance per dollar compared to previous generations, according to Kevin Cochrane, chief product officer at Vultr, and Aleks Shargorodskiy, senior director of product management at AMD.

“The models that are needed in life sciences are really, really big,” Shargorodskiy told theCUBE. “One big benefit of AMD GPUs is [that] we have a competitive advantage in our high-bandwidth memory. All these large models that do genomic prediction for folding proteins and drug discovery, they can all fit on a single GPU, and I’ll be able to run on a single one.”

Here’s theCUBE’s complete video interview with Kevin Cochrane and Aleks Shargorodskiy:

Insight #3: Kubernetes evolves from orchestration platform to AI operational layer.

Google Kubernetes Engine’s 10-year evolution mirrors the broader shift toward production-ready AI infrastructure, normalizing cloud environments through a common API abstraction that united previously proprietary ecosystems. In addition to standardizing container orchestration, GKE created the foundation for AI workloads to run consistently across platforms, according to Kelsey Hightower, distinguished engineer at Google Cloud, Bobby Allen, cloud therapist at Google Cloud, and Eric Hanselman, chief analyst at S&P Global Market Intelligence.

“It was the first time that I saw Google Cloud Platform inspire and influence all the other cloud providers,” Hightower said during the event. “It takes what was traditionally infrastructure-as-a-service and makes it one thing. We almost normalized the cloud around this one [Application programming interface] abstraction that no one thought was possible, because before that, everything was proprietary. We finally got the cloud providers to agree on something.”

That standardization enabled community-driven innovation at scale, with developers leveraging GKE’s enhancements — including Autopilot and Inference Gateway — to build intelligent infrastructure that supports AI workloads in production, according to Allen.

“You couldn’t argue with the number of Google products that are powered by containers or Kubernetes, directly or indirectly,” Allen told theCUBE. “I’ve turned the corner because I’ve seen what the community’s done. I’ve seen what people have built on it, and I’ve seen the openness that Google really gave it away to the community. It’s not about what we’re building; it’s about what we’re empowering you to build.”

Agentic AI introduces new operational challenges as autonomous systems gain the ability to execute tasks across Kubernetes clusters. Google’s Agent Sandbox addresses this issue by creating controlled environments where AI agents can operate with precision while staying within defined boundaries, preventing security breaches and infrastructure destabilization, according to theCUBE Research’s Rob Strechay; Mofi Rahman, senior developer relations engineer at Google; and Brandon Royal, senior product manager of Google Kubernetes Engine at Google.

“You start to look at all of the different pieces going from chatbots to agents to agentic workflows,” Strechay said during the event. “There’s a lot of people who are on different parts of their [agentic] journey. This is why the infrastructure portion getting solved is super interesting and important, because it’s complex and complexity kills. That’s why it needs to get simpler.”

Guardrails became the first priority as developers recognized the risks of letting language models execute code unchecked in production environments, according to Rahman. Agent Sandbox provides pre-configured containers where agents can safely test generated code, making failures cheaper and less disruptive.

“I want a sandbox environment for my agent to do things,” Rahman told theCUBE. “I don’t want to run random code that a large language model generated in my code environment. Let’s say you have certain libraries you want to use. You can create this sandbox that has those libraries pre-installed and if you try to run the code that doesn’t follow that guideline, it’s going to just fail. It’s much easier and much cheaper to fail in a container.”

The open-source collaboration that launched Kubernetes a decade ago continues to drive its evolution into an AI-ready platform. As enterprises demand sovereignty and control over their infrastructure, Kubernetes has become the foundation for building flexible, independent systems that can handle AI workloads across clouds and edge environments, according to Leon Bouwmeester, director of engineering and head of Hue Platform at Signify N.V.; Tim Hockin, Kubernetes co-founder and distinguished software engineer at Google; and Lee Sustar, principal analyst at Forrester Research.

“The foundational elements that are coming together in the community are making that innovation happen,” Sustar said during the event. “I feel that it’s much more open and dynamic. And there’s a sense of … there’s competition in other fronts, but when it comes to making this work, the collaboration is there. The way people are coming together right now to try and tackle some of the problems around AI is interesting.”

Here’s theCUBE’s complete video interview with Leon Bouwmeester, Tim Hockin and Lee Sustar:

For more from theCUBE’s coverage of KubeCon + CloudNativeCon NA, check out these segments:

• Alois Reitbauer, chief technology strategist of Dynatrace Inc., talked about the impact AI adoption has on observability.
• Kelsey Hightower, distinguished engineer at Google Cloud; Bobby Allen, cloud therapist at Google Cloud; and Eric Hanselman, chief analyst at S&P Global Market Intelligence, discussed the evolution of GKE as a key resource in enterprise computing.
• Jago Macleod, director of engineering, Kubernetes, at Google Cloud; Gari Singh, product manager at Google Cloud; and Kate Holerhoff, senior industry analyst at RedMonk, talked with theCUBE about GKE scalability milestones in light of added node capabilities, control-plane resilience and AI-driven operations.
• Jennifer Vargas, senior principal marketing manager at Red Hat, and James Harmison, senior principal technical marketing manager at Red Hat talked about the company’s strategy for scaling AI inference in enterprise Kubernetes environments and its approach to open-source contributions across the cloud-native ecosystem.
• Shubhika Taneja, senior product marketing manager, AI/ML, at Google; Kaslin Fields, manager of cloud runtimes developer relations at Google Cloud; and Amie Wei, GKE Global Hackathon Winner, discussed trends that emerged from the hackathon and Google’s recent announcements for Kubernetes.
• Rodrigo Ramalho, principal solutions architect at Red Hat, and Banco do Brasil’s Gustavo Fiuza, DevOps engineer, and Welton Danyel Felipe, system analyst, talked about how the bank has leveraged OpenShift to modernize its application development process.
• Diana Todea, developer experience engineer at VictoriaMetrics and co-lead of the Neurodiversity Group, and Jay Jackson, senior software engineer at CallRevu, Deaf and Hard of Hearing WG member, and sign language team at Cloud Native Glossary, discussed the Merge Forward initiative, the cloud-native glossary project, spectrum-based approaches to inclusion and the importance of centering underrepresented voices in accessibility work.

To watch more of theCUBE’s coverage of KubeCon + CloudNativeCon NA, here’s our complete video playlist:

(* Disclosure: TheCUBE is a paid media partner for the KubeCon + CloudNativeCon NA event. Neither Red Hat, the headline sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE