Generative artificial intelligence threat detection startup Lakera AI AG, which is notable for creating one of the world’s largest “AI red-teams” for testing generative AI models, said today it has closed on a $20 million early-stage round of funding.

Today’s Series A round was led by the European venture capital firm Atomico and saw participation from Citi Ventures, Dropbox Ventures and Redalpine, bringing the startup’s total funding to $30 million.

Zurich, Switzerland-based Lakera is the creator of a cloud-based service called Lakera Guard that helps companies spot when an individual tries to enter malicious prompts into a large language model. It’s designed to protect against prompts that aim to trick AI models into generating harmful outputs, and prevent attempts to extract sensitive information from them.

In addition, Lakera has a second product called MLTest, which is designed for developers that need to test the performance of their computer vision models before they deploy them in production. MLTest can be integrated within developer’s continuous integration/continuous deployment workflows and automatically scan AI code for any issues they need to know about.

The startup believes that demand for its tools is set to grow rapidly, citing data from Gartner Inc. that shows how 80% of all enterprises will have generative AI applications up and running in production by 2026, compared with less than 5% in 2023.

Those enterprises will need a platform such as Lakera’s because cybersecurity remains one of the biggest challenges for AI practitioners, mainly because traditional security tools are not designed for this new technology. What companies need are generative AI-specific security tools, it says.

Lakera says generative AI poses a number of risks, with one of the biggest being the danger of malicious prompts. These are dangerous because anyone can try to trick an LLM into saying something compromising or harmful, because there’s no need to write any code. LLMs understand natural language, and that opens the door for anyone to try to use them to get access to confidential data or cause mischief.

Other threats include what Lakera calls “AI sleeper agents,” which refers to the more sophisticated jailbreaking techniques used by coders to compromise AI models. There’s also a growing danger around “AI-targeted worms,” it says, which are designed to bypass security measures and harvest confidential information stored within AI systems.

Lakera founder and Chief Executive David Haber says that old cybersecurity techniques simply aren’t effective enough for generative AI applications. “Enterprises now operate in a world where anyone who knows how to talk knows how to hack,” he pointed out. “Security solutions need to change but they can’t get in the way of user experience.”

Lakera says its generative AI security tools have several benefits. Perhaps the most significant of these is that its cloud tools are powered by a repository of tens of millions of data points about AI-focused cyberattacks. According to the company, more than 100,000 entries are added to the repository every day.

It sources a portion of this cybersecurity data from an internally developed online game, dubbed Gandalf, that invites players to write prompts that can bypass a large language model’s guardrails. With more than 250,000 users, Lakera claims that Gandalf is effectively the world’s largest “red-team.” Red-teaming refers to a coordinated effort to discover flaws and vulnerabilities within AI systems.

Another advantage of Lakera’s tools, the company says, is that they’re easy to deploy. It has created an application programming interface that enables Lakera Guard and MLTest to be deployed into developer’s workflows with a single line of code.

Atomico Partner Sasha Vidoborskiy said he invested in Lakera because it has already shown “impressive commercial pull,” winning over customers such as Dropbox Inc. and one of the largest banks in the U.S. since its last funding round in October.

“More than 35% of Fortune 100 companies are knocking on their door [because] they have an urgency to deploy generative AI applications into production but can’t do it without protection in place,” Vidoborskiy said.

Lakera said the money from today’s round will go toward expanding its presence in the U.S. and accelerating its product development efforts.

Image: SiliconANGLE/Microsoft Designer