AI researchers call for new tools that can slow automated model development
A group of tech workers today published an open letter that calls for a new approach to regulating automated artificial intelligence development initiatives.
All the document’s 1,134 signatories work at companies that are building frontier AI models. Anthropic PBC, OpenAI Group PBC, Google LLC and Meta Platforms Inc. are among the industry players on the list. The letter was also signed by staffers at venture-backed AI labs such as Thinking Machines Inc. and London-based Inherent Laboratories Ltd.
The initiative’s backers include some of the AI field’s most prominent figures. Anthropic Chief Executive Officer Dario Dario Amodei signed the letter along with several other of the company’s co-founders. They were joined by the chief scientists of OpenAI, Meta’s Meta AI unit and Thinking Machines.
The letter focuses on the efforts of major tech firms to automate their model development programs with AI. The signatories are concerned that advances in this field could give rise to new safety risks.
“It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems,” reads the letter.
The document goes on to argue that the industry needs a way to “deliberately pace” automated AI development initiatives. According to the signatories, the tools necessary to implement a research slowdown are currently unavailable. The letter asks the U.S. government to help develop the necessary technologies.
“We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development,” the signatories wrote.
In April, OpenAI revealed that its GPT-5.5 large language model had helped optimize the cluster of Nvidia Corp. graphics cards on which it runs. One of the optimizations developed by the LLM increased token generation speeds up more than 20%. OpenAI has since released a newer model lineup, the GPT-5.6 series, that is significantly more adept at coding tasks.
Also in April, Anthropic disclosed that it’s using Claude to speed up its AI safety research. The company developed a Claude-powered AI agent to study a machine learning technique called weak-to-strong supervision. Anthropic’s researchers believe that the technique might hold the key to effectively supervising future AI models.
Today’s public letter continues a recent trend of researchers issuing calls for AI policy changes.
On Friday, Nvidia Corp. and other large tech firms urged U.S. policymakers not to ban open-source AI models. A few days earlier, a group of economists issued a statement that calls for stronger AI regulation.
Google DeepMind CEO Demis Hassabis joined the fray too earlier this month with a Substack essay. Like today’s open letter, the document suggests that the U.S. government help launch a new initiative to address AI safety risks. Hassabis is proposing the creation of a standards body that would develop AI risk benchmarks and use them to evaluate new LLMs before they’re broadly released.