California moves to explore ‘kill switch’ for advanced AI models | What is it and how it works

California Governor Gavin Newsom signed an executive order to explore the feasibility of a mandatory kill switch for advanced AI models, citing federal inaction on AI oversight. 

California Governor Gavin Newsom signed an executive order Friday directing state officials to explore the feasibility of imposing a mandatory "kill switch" for the most advanced AI models.

Newsom, widely seen as a possible Democratic presidential contender in 2028, said he was making the move because US President Donald Trump was doing nothing on the issue.

"The federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation," Newsom said in a statement.

"We're not waiting to act -- we're going to speed up our work on substantial and responsible AI oversight before it's too late."

The order comes days after dire warnings from industry insiders about the threat the technology poses to humanity.

California is home to Silicon Valley and the AI firms at the forefront of the sector.

What is an AI kill switch, and does it work?

The term "kill switch" refers to the ability to halt an AI system when it is deemed to have malfunctioned or gone off the rails.

The order signed by Newsom asks state officials to weigh requiring AI companies to host independent auditors in their labs.

Further, it asks officials to consider requiring companies to report loss-of-control incidents, such as those recently disclosed by major developers.

The incident includes a July breach of the AI platform Hugging Face by autonomous agents built on OpenAI models, which escaped a testing environment during an internal evaluation. The incident, and others like it, have raised fears that the latest AI programs have slipped out of the control of the companies building them

Warning from the AI world

Anthropic chief executive Dario Amodei published an essay soon after urging caution.

.