Build it … secure it … launch it … observe it. In the AI era, these four key tasks capture the essence of the cloud-native world’s focus as autonomous platforms play more of a role in running today’s enterprise.

For cloud-native leaders such as Red Hat Inc., the challenge is to provide flexible infrastructure, zero-trust security, operational capabilities and observability tools that can channel the tsunami of data that AI requires into meaningful business results. A steady stream of news and announcements emerging from the KubeCon + CloudNativeCon NA gathering in November underscored how Red Hat has been positioning its offerings to keep pace with rapid changes in the field. And there may be no faster changes occurring than in the quality of AI models, especially in the open-source world.

“We’ve seen huge innovations over the course of the past three years,” said Robert Shaw, director of engineering at Red Hat, in an interview with theCUBE, SiliconANGLE Media’s livestreaming studio. “It’s just stunning, the amount of progress that has happened over just a very, very short period of time. Reasoning capabilities that have been added to some of the proprietary models are starting to show up in the open-source models. [There is] continued improvement in the overall quality of those open-source models, which is what actually unlocks the business use cases and consumer use cases that are powering all this frenzy.”

This feature is part of SiliconANGLE Media’s ongoing exploration into the evolution of cloud-native computing, open-source innovation and the future of Kubernetes. Be sure to check out theCUBE’s recent analyst-led coverage of KubeCon + CloudNativeCon NA. (* Disclosure below.)

Cloud-native fabric for AI

A key part of Red Hat’s strategy is to position itself as a foundational enabler of AI-driven digital innovation based on open-source and cloud-native tenets. The company’s release of Red Hat AI 3 in October brought its open-source distributed inference engine into general availability, with native integration into Kubernetes.

Red Hat AI 3 provides a unified collaboration layer designed to bridge platform engineering and AI development. This includes the launch of tools such as AI Hub, Gen AI Studio and Model as a Service to support shared workflows across teams.

“These combined features transform Red Hat AI 3 into an enterprise collaboration fabric for AI, where developers, operators and data teams share one control plane, one governance model and one source of truth,” said Paul Nashawaty, practice lead and principal analyst at theCUBE Research.

Red Hat AI 3 is also designed to provide a level of interoperability that allows enterprise to integrate their own tools and frameworks. In the complicated and crowded world of AI providers, Red Hat is seeking to balance tool recommendations with the freedom for users to bring preferred open-source technologies into the enterprise stack.

“One of the things that we do is to make sure that we give customers interoperability,” explained Jennifer Vargas, senior principal marketing manager at Red Hat, during a conference interview with theCUBE. “You need applications or technologies that work together that are resilient, that are reliable. That’s the part where we put the spark on our products … when everyone’s looking for an AI that’s transparent, that is trustworthy, that is reliable.”

Optimization for OpenShift

Red Hat’s interest in providing frameworks where applications can work together became more apparent in early November with the release of OpenShift 4.20. The latest version includes an optimization of OpenShift Virtualization that enables customers to manage virtual machines alongside containers and cloud-native applications from a single platform.

OpenShift 4.20’s latest features have provided support for Red Hat customers such as Banco do Brasil. In a recent interview with theCUBE, Welton Danyel Felipe, system analyst at Banco do Brasil, described how OpenShift was supporting the institution’s ongoing work to embrace GitOps, applying DevOps practices to infrastructure automation.

“Working on that platform, we can deliver more applications in a short time,” Felipe said. “We have all these delivery processes, standardizing, GitOps, etc. We could use all that to deliver more applications and more value to the customer in the end.”

The release also included an expansion of Red Hat OpenShift Lightspeed, a virtual assistant built into the OpenShift console. Integration with Red Hat Advanced Cluster Manager for Kubernetes allows Lightspeed to support multiple clusters and enables administrators to troubleshoot issues and navigate system configurations using natural language prompts.

This focus on the user experience by embedding practical AI across developer tools has put a tailwind behind OpenShift for Red Hat. Revenue from the product stood at $100 million at the time that Red Hat was acquired by IBM Corp. in 2019. That figure is significantly higher now and user demand continues to climb, according to Stu Miniman, senior director of market insights for hybrid platforms at Red Hat.

“OpenShift is now a $1.8 billion annual recurring revenue product growing at over 30%,” Miniman recently told SiliconANGLE.

Exploring vLLM and llm-d for AI

Red Hat has been focused on helping its customers meet the need for AI infrastructure. One of the emerging technologies with momentum behind it to meet this need is the virtual large language model or vLLM. As an open-source, high-throughput and memory efficient serving engine to accommodate large language models, vLLM offers enterprises a way to generate per-token efficiency and smart, targeted compute use.

Red Hat has described vLLM as “the decision-maker for your LLM inference pipeline.” The vLLM Sematic Router, which is supported by Red Hat, can intelligently route incoming queries to the most suitable LLM based on the query’s intent or complexity.

“The way AI is heading is a very fragmented kind of world,” said Brian Stevens, senior vice president and AI chief technology officer at Red Hat, in conversation with theCUBE. “Our vision’s really been like, ‘How do we unify that into a common platform like we did with Linux, where there could be one core … vLLM that can run all models and run all accelerators?’ In doing so, think about what that means to end users. They can just have one vLLM platform and get to use all the best in future models and all the accelerators seamlessly.”

The ability for vLLM to map an entire ecosystem of open-source models onto hardware accelerators provides an integration point for AI participants, but there is a next step. Organizations using AI want to scale beyond individual nodes to entire clusters, and this is where llm-d plays an important role.

Llm-d, large language model-distributed, is an open-source, Kubernetes-native framework designed to make LLM inferencing more scalable and cost-effective. The project is backed by a powerhouse group of contributors that includes Red Hat, IBM and Google LLC, and it can maximize throughput across multi-node clusters and enable enterprises to harness AI at scale.

“There’s a lot of work, not just at llm-d, but all the Kubernetes pieces underneath,” Miniman told theCUBE. “How do pipelines and GitOps play into this new world? There’s dozens of projects there. We know the CNCF has been great at pulling in a lot of these projects to make sure that they work there, because the infrastructure and the applications have to play well together in this fast-moving space.”

KubeVirt gains momentum

Red Hat’s work to leverage the Kubernetes pieces underneath involves finding new ways to extend the container orchestration tool’s API so that it can interact with virtual machines. A key part of this effort is KubeVirt, a popular solution for bringing VM workloads onto Kubernetes clusters. KubeVirt’s flexible infrastructure integrates both containerized and traditional virtual machine workloads for a more resilient and versatile IT environment, an increasingly important consideration in the implementation of AI.

Red Hat’s OpenShift Virtualization offering is anchored by the KubeVirt project, and there are signs that the platform-level solution is gaining momentum. More than half of organizations in one survey indicated plans to migrate virtual machines to Kubernetes-based technologies like KubeVirt, and 65% said they would do so in the next two years.

“We’re one of the creators of the KubeVirt project … and that forms a basis of our OpenShift Virtualization capability that’s available as part of our OpenShift platform,” said Ashesh Badani, chief product officer of Red Hat, during an interview with theCUBE. “My guess is … the number of customers that we’ve got probably [has] tripled … and there are customers of all kinds who are embracing this. Traditional financial services, manufacturers, telcos [and] public sector organizations. Across the world, we are seeing a huge amount of interest in this area.”

Securing models and infrastructure

Building the AI infrastructure also means finding the tools and services to properly secure it. Last month’s release of Red Hat OpenShift 4.20 included advanced cluster security and enhancements to the company’s Trusted Artifact Signer and Trusted Profile Analyzer

“With AI adoption accelerating — 83% of enterprises expected to use AI in production by 2026 — security risks are evolving,” said Rob Strechay, managing director and principal analyst at theCUBE Research. “Red Hat looks to unveil a strategy to secure AI models throughout the entire lifecycle.”

Red Hat has also been focused on pioneering zero-trust AI, a continuous process to boost safety, accountability and reliability in critical environments. This includes practices such as enforcing ongoing verification, strict access controls and compartmentalization to protect sensitive data and preserve model integrity, according to Anjali Telang, senior principal product manager of OpenShift Security and Identity at Red Hat, in a recent conversation with theCUBE.

“Zero trust in general means that you trust no one, you always verify, and then you base that verification on an identity, and then you trust the person,” Telang said. “With AI, we want to sort of bring in the same trust that we already have built into the system. We want to make sure that the users, the machine, all the trust that we have brought in with the best practices around that, translates to AI workloads, AI agents.”

Red Hat’s continued focus on building enterprise frameworks for AI infrastructure has been driven by the current experience of its customers. Enterprises are moving past model training and into the inference phase, seeking performance, governance and effective cost optimization.

This will take a production-grade platform with predictable performance, lower cost per token and simplified orchestration across hybrid environments. Enterprises are transitioning from the experimentation phase with AI. It’s time to get things done.

“Enterprises are moving beyond model training to focus on inference (the ‘doing’ phase of AI) where performance, governance and cost optimization determine success,” according to theCUBE Research’s Nashawaty and Sam Weston. “For CIOs and IT leaders, the focus now shifts from experimentation to infrastructure efficiency through maximizing accelerator utilization, maintaining sovereignty and ensuring reproducibility. By bridging inference scalability with agentic readiness, Red Hat AI 3 sets the template for how open-source ecosystems can rival closed-loop AI stacks without sacrificing flexibility or control.”

(* Disclosure: TheCUBE is a paid media partner for the KubeCon + CloudNativeCon NA event. Neither Red Hat, the headline sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Image: Microsoft Designer/SiliconANGLE