Automated evaluation and security platform Patronus AI Inc. today launched out of stealth mode and announced that it has raised $3 million in seed funding.

Founded by machine learning experts Anand Kannappan and Rebecca Qian, both former Meta Platforms Inc. employees, Patronus has been designed to tackle the difficulties of evaluating artificial intelligence outputs and the challenge of large language model evaluation for enterprises.

The Patronus platform is claimed to be the first automated evaluation and security platform that helps companies use LLMs safely. The platform uses AI to enable enterprise development teams to score model performance and generate adversarial test cases and benchmark models.

Patronus AI automates and scales the manual and costly model evaluation methods commonly found in enterprises to enable organizations to confidently deploy LLMs while minimizing the risk of model failures and misaligned outputs. The platform uses machine learning technology to test and score any language model in order to identify potential failures.

The platform automates scoring, ranking model performance in real-world scenarios and key criteria like hallucinations and safety. Patronus AI also automatically gnerates adversarial test suites at large scale and benchmarking compares models to help customers identify the best model for specific use cases.

“Every company is looking for ways to use LLMs today, yet they are concerned that unexpected model behavior, incorrect outputs and hallucinations will put their business and customers at risk,” Chief Executive Kannappan said in a statement. “Whether off-the-shelf, open-source or custom, models today remain inadequately vetted and tested in real-world scenarios. And until now, the process of evaluating LLMs has been extremely inefficient and unscalable, producing unreliable results.”

The company launched with $3 million in seed funding in a round led by Lightspeed Venture Partners L.P. Factorial Capital LLC, the CEO of Replit Inc. Amjad Masad, Gokul Rajaram and several Fortune 500 executives and board members also participated in the round.

Image: Patronus AI