Anthropic’s Dario Amodei calls for slower AI race: What will it take, and can it be done?

Amid renewed discussions of human extinction at the hands of AI, Anthropic CEO Dario Amodei has suggested tapping the brakes on frontier AI progress, but stopped short of advocating for a complete halt.

In a lengthy essay on Saturday, September 12, Amodei said that developments over the past few months have convinced him that AI risk prevention needs time to catch up with rapidly advancing AI capabilities. He pointed to two key warning signs behind his call for a slowdown: early signs of self-improving AI and the recent OpenAI-Hugging Face incident.

Amodei also laid out a three-step plan to put his proposal into action, including allowing third-party evaluators to assess Anthropic’s AI systems with access comparable to that of employees. “To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this,” Amodei said.

It is an extraordinary decision that comes as Anthropic gears up for an anticipated IPO amid a highly competitive race with arch-rival OpenAI, and as researchers grapple with rapid advancements in AI capabilities that have left industry leaders worried about their ability to control them.

Earlier this week, an Anthropic researcher made the news by quitting his job because he believes that there is a greater than 10 per cent chance that “AI could kill all humans” within the next decade. Anthropic published a report last month assessing the risk that its AI models will go off the rails as ‘low’ – up from ‘very low’ which means that the threat level has increased. These dire warnings have snowballed into calls from various stakeholders for increased caution.

“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei wrote in his blog post.