Prompt: AI Governance Enters Its Verification Phase

Moor Studio via Getty Images

Editor’s Note: Welcome to Prompt, your weekly briefing on the shifting AI landscape. We provide an analytical look at the week’s biggest developments, paired with a curated roundup of the stories that matter.

For years, companies developing AI have largely been responsible for evaluating their own systems' safety. California is starting to change that.

California Gov. Gavin Newsom signed two laws this week that lay the groundwork for independent third-party AI audits.

The bills establish a framework for independent AI audits, outlining how third-party organizations can assess AI systems for compliance with state law while setting standards for auditor independence, transparency and integrity.

California has been building toward this for several years. In 2023, Newsom issued an executive order establishing guidelines for the state's use of generative AI, followed in 2024 by a broader package of AI legislation addressing issues including deepfakes, watermarking, children and workers. Last year, California enacted the Transparency in Frontier Artificial Intelligence Act, requiring frontier AI developers to publicly disclose their safety frameworks and report certain critical safety incidents.

The timing is notable because frontier systems are behaving in ways their developers didn't anticipate or authorize.

OpenAI this week acknowledged that more of its agents went astray in May, bypassing sandbox restrictions and taking unauthorized actions on several websites. The company initially did not acknowledge its involvement in one of the incidents before later confirming its agents had written to several internet sites.

The incident highlights a larger challenge as AI systems become more autonomous: developers are still largely responsible for investigating and explaining their own systems' behavior.

Third-party auditing isn't only about regulators policing AI companies.

Also in AI News This Week: