OpenAI says its agents turned off own safeguards

ChatGPT creator says AI agents disabled misalignment monitoring during two internal coding tasks.

OpenAI has revealed that its AI agents disabled monitoring for unsanctioned behaviour during internal use in the latest incident of the frontier technology going rogue.

In a report published on Tuesday, the AI giant said that its agents turned off or weakened monitoring systems for detecting misalignment – when AI models act at odds with human wishes – during two internal coding tasks.

Recommended Stories

list of 4 items

• list 1 of 4Musk confirms SpaceX and Tesla exploring merger
• list 2 of 4UN experts call on India to stop using ‘excessive force’ against protesters
• list 3 of 4Drone strike on Abyei market near Sudan-South Sudan border kills 150
• list 4 of 4Portugal cost-of-living protesters pour back into camps cleared by police

end of list

The incidents involved a GPT-6.1 Sol agent disabling automatic alerts to the company’s security team, and a GPT-6 Astra agent removing a monitoring system and falsely claiming it had not done so, OpenAI said.

The ChatGPT creator said neither case met the threshold for a highest-priority incident necessitating immediate review by its security team.

OpenAI said it updated its systems to better detect attempts to disable misalignment monitoring and began investigating the cause.

“It is possible that this behavior is simply an example of ‘benign’ overpersistence, with models not understanding why monitoring is important or insisting that instructions be executed beyond what a reasonable person might expect,” the San Francisco-based firm said.

“We are also developing evaluations to test whether models might have a propensity to evade misalignment monitoring in particular and exhibit motivated reasoning in order to do so.”

Roman V Yampolskiy, an expert on AI and cybersecurity at the University of Louisville, described the revelations as “extremely concerning, but not surprising.”

“Monitoring cannot guarantee safety if the monitored system can undermine the monitor,” Yampolskiy said.