

Open-source software giant Red Hat Inc. announced today that it has agreed to acquire Neural Magic Inc., a machine learning startup that optimizes AI models to run whatever hardware is available, including central processing units or graphics processing units.
As AI large language models grow in size and complexity, they also consume more compute power and energy. As a result, businesses have seen the need to build more cost-efficient LLMs designed for efficiency as well as accuracy that take advantage of the hardware they’re deployed on.
Founded in 2018 by Massachusetts Institute of Technology Research Scientist Alex Matveev and Professor Nir Shavit, Neural Magic’s technology allows for the optimization of LLMs to run on off-the-shelf CPUs and GPUs at similar speeds to specialized AI chips. That means a company can fine-tune and optimize AI models to maximize for hardware efficiency across numerous hardware architectures.
“AI workloads need to run wherever customer data lives across the hybrid cloud,” said Matt Hicks, president and chief executive of Red Hat. “This makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs.”
Neural Magic has contributed significantly to an open-source project for model deployment called vLLM, which Red Hat said brought the company to its attention. The project provides a fast, efficient library for serving and deploying AI models optimized for the cloud that supports numerous processor and GPU architectures. The company has also designed a specialized LLM compressor that provides faster inference with vLLM allowing faster delivery of AI models.
The startup also keeps a repository of prebuilt, pre-optimized open-source LLMs ready to deploy with vLLM so businesses can quickly deploy high-performance models at scale.
Red Hat said Neural Magic’s leadership in vLLM will help build on the company’s ability to support LLM deployments for the hybrid cloud with any hardware back end.
This acquisition represents an important move for Red Hat’s open-source AI strategy. The company recently launched Red Hat Enterprise Linux AI, which brings together the open-source Granite family of LLMs developed by IBM Corp. with Red Hat’s InstructLab open-source community-driven model alignment tools, packaged as optimized, bootable server deployments.
“Open source has proven time and again to drive innovation through the power of community collaboration,” said Neural Magic CEO Brian Stevens. “At Neural Magic, we’ve assembled some of the industry’s top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities.”
Support our open free content by sharing and engaging with our content and community.
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.