The open-source software giant Red Hat Inc. is strengthening the case for its platforms to become the foundation of enterprises’ artificial intelligence systems with a host of new features announced today aimed at accelerating development and deployment.

The updates to the “Red Hat AI” suite that launched in September are meant to improve the efficiency of AI training and inference while simplifying the user experience with flexible deployment across hybrid cloud environments, the company said.

Red Hat AI encompasses two of Red Hat’s key offerings. The first is Red Hat Enterprise Linux AI, which is a specialized version of the standard RHEL operating system that’s geared for the deployment of foundational large language models.

Developers can use it to deploy AI applications based on Red Hat parent company IBM Corp.’s Granite LLMs and others, such as OpenAI’s GPT family. The platform is packaged as an optimized, bootable RHEL image that can be deployed on individual servers on-premises or in hybrid cloud environments.

The other major component of Red Hat AI is Red Hat OpenShift AI, which is a scalable AI and machine learning development platform that’s used to create, test and launch AI applications at scale.

According to Red Hat, enterprises are still struggling to integrate their AI applications and models with the proprietary data that’s essential for training them to handle more specific use cases, partly because that information is siloed in all manner of different places, including on-premises servers, cloud infrastructure platforms and even at the network edge, where it’s generated by sensors and other devices.

Enterprise Strategy Group Principal Analyst Torsten Volks said the ability to develop, deploy, integrate and scale up AI rapidly and cost-effectively has become a critical success factor for organizations today.

“Establishing this capability requires an open and extensible AI foundation that ensures seamless integration with existing systems and processes, operational agility and continuous governance,” he said.

Red Hat said today’s updates to Red Hat OpenShift AI and RHEL AI help cement this foundation, paving the way for the development of more efficient and optimized models that are fine-tuned on business-specific data in a way that ensures the security of that data. They can then be deployed in any location on a range of accelerated compute architectures, including Nvidia Corp.’s graphics processing units.

Red Hat OpenShift AI

Generally available from today, Red Hat OpenShift AI 2.18 adds new capabilities including support for distributed serving, enabling teams to split model serving across multiple GPUs to reduce the operational burden on their server infrastructure. By deploying AI across clusters of GPUs, teams can dramatically speed up their training and inference processes and maximize the efficiency of this underlying infrastructure.

OpenShift AI also adds support for Red Hat AI InstructLab and OpenShift AI data science pipelines to create an “end-to-end model tuning experience.” It’s more scalable and auditable in large production environments, helping to boost AI application security, the company said.

Moreover, there are enhanced safety guardrails for LLMs that aim to improve their performance and the accuracy of their outputs while increasing transparency. The new guardrails monitor both inputs and outputs, enabling users to identify and mitigate so-called “prompt injection attacks” that aim to manipulate AI systems into generating abusive, hateful or profane speech or leak sensitive information.

Additionally, the platform is getting a new language model evaluation component that provides vital insights on the overall quality of LLMs, helping data scientists to benchmark their performance in various different tasks, such as mathematical and logical reasoning.

Red Hat Enterprise Linux AI

As for RHEL AI, the main update here is support for the latest Granite 3.1 8B model, which is the most powerful yet within the open-source and hardware-efficient Granite LLM family. It’s the first to come with multilingual support for inference and taxonomy and knowledge customization, and features a larger 128K context window that allows for improved summarization outputs.

RHEL AI also gets an improved graphical user interface to support skills and knowledge contributions to AI models. Available as a developer preview, it’s meant to simplify data ingestion and chunking tasks, making it much simpler for developers to enhance the knowledge of the LLMs they’re working with.

Lastly, there’s a new document knowledge-bench that makes it easier for developers to compare the performance of different fine-tuned LLMs trained on private data.

Joe Fernandes, Red Hat’s vice president and general manager of AI, said enterprises are still seeking ways to ease the integration of private data with their AI models and manage the rising cost of these activities.

“Red Hat AI helps enterprises to address these challenges by enabling them to leverage more efficient, purpose-built models trained on their data, as well as flexible inference across on-premises, cloud and edge environments,” he said.

In some other updates announced today, Red Hat said it’ll soon be bringing the popular InstructLab tool to the IBM Cloud, enabling teams to simplify, scale up and boost their security footprints when training LLMs on that platform. Set to launch in the spring, it should be especially useful for teams that need to fine-tune AI models using private data, the company said.

Finally, Red Hat said, its customers will now be able to access its AI Foundations online training courses at no cost. The courses, available at a cost for non-customers, provide two AI learning certificate paths, for experienced professionals and AI novices, educating learners on how AI can improve business operations and decision-making and accelerate innovation.

Image: SiliconANGLE/Dreamina