OpenAI resumes AI model training offline after series of breakout events by agents force it to pause

OpenAI says it has resumed training its most capable models after security changes and tests by AI agents. Those tests revealed a vulnerability in Brotli, which has since issued updated versions, as well as flaws in OpenAI safeguards. 

OpenAI has said that it had resumed training of its most advanced AI models after tightening security and barring internet access, and disclosed a flaw in a widely used software that it found while testing its defenses. The disclosure on Friday comes amid concerns over the ability of increasingly capable AI agents to identify and exploit software flaws, including in the guardrails meant to contain them.

Why OpenAI paused training

OpenAI had paused training of its most capable models after one of its research models used a configuration error to gain limited internet access in September. The company said it contained the incident and used the pause to strengthen its defenses.

To test them, OpenAI had teams of AI agents try to break out of their restricted environments. It said the agents found previously unknown flaws, which it had fixed, and training resumed on October 1.

During that exercise, the ChatGPT maker found a flaw in Brotli, a widely used data compression tool.

What OpenAI said

The company said attackers could exploit the flaw by sending malicious data that lets them run their own code on affected systems, including those using popular web software such as Node.js, Bun and Envoy.

OpenAI said the root cause was fixed in Brotli's source code in February, before it was known to create a security vulnerability, but no new version was released at the time. Brotli has since released updated versions and published a public record of the flaw after OpenAI disclosed it.

OpenAI told the safety organizations it has since patched all vulnerabilities affecting its production safeguards, although METR and Redwood said they had not verified the fixes.