The Department of Commerce today announced that it has added five people to the AI Safety Institute’s leadership team, including a man who once predicted that there was a 50% chance that artificial intelligence development would end in doom for humanity.

As part of the National Institute of Standards and Technology, or NIST, the new team will help encourage AI development while at the same time ensuring it benefits humanity rather than harms it. Over the years, there has been much talk about AI becoming an existential threat, although many others have said that’s overblown and such stark predictions could stifle development.

The new members to the team include Mara Quintero Campbell, deputy chief operating officer of Commerce’s Economic Development Administration, who will act as the chief operating officer and chief of staff. Adam Russell, director of the AI division of University of California’s Information Sciences Institute, will become the chief vision officer.

Rob Reich, a professor of political science and associate director of the Institute for Human-Centered AI at Stanford, will become the senior adviser. Mark Latonero, deputy director of the National AI Initiative Office at the White House Office of Science and Technology Policy, will become head of international engagement.

The member who has been referred to as a former AI doomer is Paul Christiano. He founded the nonprofit Alignment Research Center and used to run OpenAI’s language model alignment team. He will become the team’s head of AI safety. Last year, he was asked in a podcast what the chance that our AI ambitions could end in doom was, to which he replied, “A 50/50 chance of doom shortly after you have AI systems that are human level.”

“I think maybe there’s something like a 10-20% chance of AI takeover, [with] many [or] most humans dead,” he added. “I take it quite seriously.” He explained that if this scenario presented itself, it would only be after several periods of “acceleration” in AI development rather than an extremely fast transformation.

“To safeguard our global leadership on responsible AI and ensure we’re equipped to fulfill our mission to mitigate the risks of AI and harness its benefits, we need the top talent our nation has to offer,” said U.S. Secretary of Commerce Gina Raimondo about the new appointments. “Developing guidelines that will strengthen our safety and security, engaging with civil society and business, and working in lockstep with our allies are fundamental to addressing this generation-defining technology.”

Photo: Freepik