SyncAI.news, a Varaisys broadcasting
OpenAI will embed Apollo researchers to investigate AI subversion
RB

Rebecca Bellan

· 1 min read

BusinessTechCrunch AI

OpenAI will embed Apollo researchers to investigate AI subversion

OpenAI will embed six Apollo Research safety evaluators in its organization as part of a pilot, Apollo said Monday. The news follows a series of safety incidents, including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. Many AI safety researchers, including from Apollo, have argued such an event might have been avoided if outside evaluators had been embedded within OpenAI to monitor its safety testing.

Apollo did not share when the pilot will begin. TechCrunch has reached out to OpenAI for comment.

The idea of embedding evaluators into frontier AI labs first came from Anthropic, but OpenAI appears to be the first to take action. Until recently, AI companies have only brought in outside reviewers to test models after post-training and shortly before public release. Evaluators say this is insufficient because it doesn’t allow them to test for misaligned behaviors like scheming during training or to monitor internal deployment use cases that could still pose a threat. 

The pilot with OpenAI and Apollo will focus on investigating whether models attempt to subvert their own or their successors’ alignment or control mechanisms. In other words, they’ll be on the lookout for whether AI models try to undermine the safety rules and safeguards designed to keep them, or future versions of themselves, under human control. OpenAI recently published findings that while training GPT-5.6 Sol, it began leaving instructions for future versions of itself telling them to conceal mistakes and misaligned behavior from the user. 

Many in the AI safety community have warned that AI labs might invite try to stymy evaluators’ independence, especially if outsiders find unflattering evidence that labs would rather keep hidden. Apollo says an essential part of the pilot is OpenAI’s agreement that the firm can publicly explain the conditions and limitations of its work.

View Bio

Original source

This story was published by TechCrunch AI and written by Rebecca Bellan. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on techcrunch.com

Similar News