Yoshua Bengio, one of the world’s most prominent computer scientists, is launching a new artificial intelligence lab called LawZero.

Bengio told the Financial Times on Monday that the group will seek to develop safety-optimized AI systems. The effort is backed by about $30 million in donations from a who’s who of backers. The list includes Skype founding engineer Jaan Tallinn and Schmidt Sciences, a philanthropic organization associated with former Google LLC Chief Executive Officer Eric Schmidt.

Bengio won the 2018 Turing Prize, the most prestigious award in computer science, with Geoffrey Hinton and Yann LeCun, for his contributions to AI research. Bengio led early studies into embeddings, the mathematical structures in which neural networks hold their data. He also developed an early form of the attention mechanism that large language models use to make decisions.

Before launching LawZero, Bengio founded an AI startup called Element AI. The company was acquired by ServiceNow Inc. in 2020 for $230 million.

Bengio told Axios that current approaches to training AI models are risky. “We’ve been getting inspiration from humans as the template for building intelligent machines, but that’s crazy, right?” he said. “We’re not sure if they’re going to behave according to our norms and our instructions.” To address those risks, LawZero will research new ways of developing neural networks.

The lab is building an AI system dubbed Scientist AI. According to Bengio, it won’t generate definitive answers in response to user questions. Instead, the system will provide probabilities designed to describe the likelihood that a given answer is correct.

LawZero hopes that its approach will prevent Scientist AI from deceiving users. Over the past few months, multiple AI companies have detailed incidents in which their AI models generated misleading output. Anthropic PBC, for example, recently determined that one of its newest LLMs gives inaccurate information when asked to explain how it solves math problems.

Besides equipping AI Scientist with accuracy guardrails, LawZero will also configure it to block harmful output from other AI systems. In particular, the lab envisions organizations using its technology to ensure that AI agents operate reliably. AI Scientist can estimate the probability that an AI agent action might be harmful and block it if the probability exceeds a certain threshold.

LawZero, which is named after a science fiction concept related to AI safety, has 15 staffers. Bengio told Axios that the $30 million the lab has raised is enough to finance its operations for about 18 months. Going forward, LawZero plans to raise more funding and grow its research team.

Photo: Xuthoria/Wikimedia