Patronus AI Inc., a startup helping companies detect and fix reliability issues in their large language models, today announced that it has closed a $17 million investment.

Notable Capital led the Series A round. It was joined by publicly traded observability provider Datadog Inc., Lightspeed Venture Partners, Factorial Capital and several angel investors from the tech industry. The cash infusion brings Patronus AI’s total outside funding to $20 million.

Inaccurate information in prompt responses is not the only risk that companies must address before deploying an LLM to production. In some cases, user prompts can cause a model to output copyrighted material or sensitive business data. There are also more subtle issues to address, such as situations where LLM output doesn’t align with a company’s text style guidelines.

San Francisco-based Patronus AI has developed a platform that promises to help developers address those challenges. According to the company, its software uses AI to automatically generate adversarial prompts. Those are prompts that test an LLM’s reliability by attempting to trick it into generating unwanted output.

The platform also provides prepackaged reliability evaluations created by the company. For added measure, Patronus AI has included a dashboard that visualizes the results of reliability tests in charts. If a given evaluation comprises 100 prompts designed to assess the accuracy of an LLM’s responses, the dashboard can display how many of the prompts were processed incorrectly.

“Model hallucinations and safety risks are here to stay,” said Chief Executive Anand Kannappan. “What enterprises need is transparency into model performance and accuracy in order to circumvent risks. For the first time, we’re giving companies a way to truly understand what they are working with so they can deploy LLMs with confidence.”

One of the use cases Patronus AI is targeting with its platform is improving the reliability of LLMs with RAG, or retrieval-augmented generation, features. Standard language models can only generate responses based on information in their training datasets. A RAG-enabled LLM, in contrast, can expand its knowledge base by drawing on external data sources, which improves response quality.

The process through which an LLM incorporates data from external sources into prompts comprises multiple steps. According to Patronus AI, developers can use its platform to check that those steps are carried out correctly. The company says its software provides at least 20% better “evaluation performance” than competing methods.

Developers can also use Patronus AI to determine which of the LLMs at their disposal are most suitable for a given software project. Using the platform, an application team can test several models on the same set of prompts and identify the one that generates the most accurate responses. The company says its platform supports both off-the-shelf and customized LLMs.

In some cases, a language model that performs well when it’s deployed to production can become less accurate down the road. The issue emerges when the types of prompts that users enter into an LLM change over time. Patronus’ platform includes an application programming interface that allows developers to continuously evaluate a deployed LLM for such gradual accuracy declines.

The company will use the proceeds from its newly announced funding round to support product development initiatives. It will grow its AI research and engineering teams to that end, as well as expand its go-to-market efforts in parallel. The hiring push will reportedly double its headcount to about 24 employees by year’s end.

Photo: Patronus AI