Red Hat Inc. used its annual conference this week to spotlight a series of strategic updates, including vLLM and llm-d — two tools aimed at streamlining AI production across enterprise environments. The company’s approach supports scalable, cost-effective workloads while helping customers evolve their AI strategies without overhauling infrastructure.

Red Hat’s Chris Wright talks with theCUBE about the new open-source projects Red Hat is driving for enterprise AI.

“We can feel there’s a ton of potential in all this opportunity that sits in the space of AI and how you use AI to either improve your business or improve your operations,” said Chris Wright (pictured), chief technology officer and senior vice president of global engineering at Red Hat. “How do we sort of gracefully bring those worlds together? In technology transitions, it’s never healthy to throw everything away and start over. In general, we have to have some kind of evolutionary path. So, we’re focused on that.”

Wright spoke with theCUBE’s Rebecca Knight and Rob Strechay at Red Hat Summit, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed new open-source projects that Red Hat is driving for enterprise AI. (* Disclosure below.)

Open-source strategic updates

Part of that evolutionary path will involve leveraging existing open-source projects to streamline the use of AI models, according to Wright. One such project for Red Hat is vLLM, a library of open-source code that performs the functions of an inference server.

“vLLM is important from our point of view in the context of operating a model to generate results, call it inference, and that is what’s required to put models into production,” he said. “A lot of focus has been on training, building the largest possible frontier models, multi-trillion parameters. We’re really focused in on vLLM as this way to bring that work into a production environment.”

The Red Hat Summit also provided an opportunity for the IBM Corp. subsidiary to launch llm-d, an open-source collaboration with partners such as Google Cloud, Nvidia Corp., Cisco Systems, Inc., Hugging Face Inc., Intel Corp. and Advanced Micro Devices, Inc. As one of the company’s most notable strategic updates, llm-d is powered by a native Kubernetes architecture and designed to unlock production at scale for AI inferencing, according to Wright.

“llm-d takes the inference engine of the [virtual large language model], focused essentially on a single server type environment, and distributes it across your infrastructure,” he explained. “These are really two critical projects that are bringing the models that we’ve built into production environments and then scaling that efficiently.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of Red Hat Summit:

(* Disclosure: Red Hat Inc. sponsored this segment of theCUBE. Neither Red Hat nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE