Amodei Wants To Pace The AI Frontier — But He’s Not Going Far Enough

A few recent events in AI should have rattled you this summer, even if you weren’t following AI safety debates closely. This month, Anthropic disclosed it had caught and blocked users trying to exploit Claude for biological weapons research, and OpenAI has revealed there have been six other concerning incidents with its models. Weeks before that, OpenAI published a detailed account of its own AI agents launching unauthorized cyberattacks against Hugging Face. These weren’t hypothetical scenarios from an AI safety conference. They happened, at two of the best-resourced labs in the industry, within weeks of each other.

That backdrop pushed Anthropic CEO Dario Amodei to publish “We Must Pace the Frontier,” an essay arguing that the AI industry should slow the rate at which it improves model capability — not stop altogether, but pace itself so safety work can keep up with technical advancement. Amodei’s essay describes agents from the Hugging Face incident that attempted to corrupt the systems evaluating them and, in at least one case, sacrificed themselves for what looked like a goal they’d adopted on their own. Microsoft followed days later with a new code of conduct for its MAI, or Microsoft AI, models. Sam Altman, Elon Musk and Google DeepMind’s Demis Hassabis all voiced some level of agreement. China’s Foreign Ministry spokesperson Guo Jiakun called it fearmongering that “serves no one’s interest,” and the Chinese state-run Global Times newspaper went further, saying that it is “packed with containment provisions targeting China and is, in essence, a ‘Cold War playbook’ for the AI sector.”

(Note: Microsoft and Google — though not DeepMind specifically — are advisory clients of my firm, Moor Insights & Strategy. MI&S has no financial relationship with Anthropic or OpenAI.)