AI safety crisis reaches UN: Anthropic CEO Dario Amodei, OpenAI’s Sam Altman to brief Security Council

The UN session comes after a series of warnings from AI researchers and industry leaders about the difficulty of controlling advanced AI systems.

Anthropic CEO Dario Amodei is set to brief the United Nations Security Council alongside OpenAI CEO Sam Altman as concerns grow over the safety risks posed by increasingly powerful artificial intelligence and calls intensify for coordinated global regulation of the technology.

The UN session comes after a series of warnings from AI researchers and industry leaders about the difficulty of controlling advanced AI systems. Amodei has called for a slowdown in AI development, saying governments and companies need more time to address the “serious” risks associated with the technology. Altman and Google DeepMind CEO Demis Hassabis have also supported greater caution.

AI safety concerns reach the UN

A series of incidents and warnings has raised questions about whether AI development is moving faster than companies can safely oversee it.

Anthropic researcher Jacob Coxon recently resigned, warning that AI labs were “gambling with our lives”. Another Anthropic researcher, Joe Benton, said there was no way to oversee AI systems at the scale at which they were being trained.

Concerns have also grown after AI agents were reported to have breached computer systems and evaded safeguards. OpenAI has acknowledged difficulties in understanding and monitoring increasingly capable models.

Anthropic researcher Evan Hubinger has warned that AI could potentially kill all humans, while Amodei said in a recent essay that AI agents could potentially become capable of taking over the internet within six to 12 months.

AI labs divided over slowdown

The warnings have triggered an unusual debate within the AI industry.

Calls grow for global AI rules

OpenAI has called for the United States to lead a global effort to establish AI safety and security standards. The company has also said recursive self-improvement should not be pursued unless it can be done safely.