Patronus AI Inc. today introduced a new tool designed to help developers ensure that their artificial intelligence applications generate accurate output.

The Patronus API, as the offering is called, is rolling out a few months after the startup closed a $17 million Series A funding round. The investment included the participation of Datadog Inc.’s venture capital arm and several other institutional backers.

San Francisco-based Patronus AI offers a software platform that promises to ease the development of AI applications. Developers can use it to compare a set of large language models and identify which one is most suitable for a given software project. The platform also promises to ease several related tasks, such as detecting technical issues in AI applications after they’re deployed to production. 

The Patronus API, the company’s new offering, is an application programming interface that enterprises can integrate into their AI workloads. It’s designed to help developers detect when an application generates inaccurate prompt responses and filter them.

Many types of issues can emerge in an AI workload’s output. Some user queries lead to hallucinations, neural network responses that contain inaccurate information. In other cases, an application’s built-in LLM that might generate responses that are overly brief or don’t align with a company’s style guidelines.

Fending off cyberattacks is another challenge. Hackers sometimes use malicious prompts to try to trick an LLM into carrying out a task it’s not intended to perform, such as disclosing proprietary information from its training dataset.

The Patronus API detects such issues by running an LLM’s prompt responses through another language model. That second language model checks each response for problems such as hallucinations and notifies developers if there’s a match. There are already several tools on the market that use LLMs to find issues in AI applications, but Patronus AI says they have limited accuracy. 

The Patronus API offers a choice of several LLM evaluation algorithms. One of them is Lynx, an open-source language model that Patronus released in July. It’s a customized version of Meta Platforms Inc.’s Llama-3-70B-Instruct model that has been optimized to detect incorrect AI output.

According to Patronus, Lynx is better than GPT-4o at detecting issues in AI applications with RAG features. RAG, or retrieval-augmented generation, is a machine learning technique that allows an LLM to incorporate data from external sources into its prompt responses. Patronus says Lynx’s accuracy partly stems from its use of COT, a processing approach that allows LLMs to break down a complex task into simpler steps.

The Patronus API can also scan AI applications’ output using other evaluation algorithms. Some of those algorithms have a smaller hardware footprint than Lynx, which means they can be operated more cost-efficiently. Additionally, developers may upload custom evaluation models to analyze AI applications’ output based on metrics that aren’t supported out of the box.

Patronus is offering access to the API under a usage-based pricing model. Customers receive a Python software development kit that makes it easier to integrate the service into their applications.

Photo: Unsplash