Third-party evaluators find vulnerabilities in OpenAI monitoring system during test pilot

OpenAI CEO Sam Altman delivers a keynote speech during OpenAI's annual DevDay event in San Francisco, California, US, September 29, 2026.

Carlos Barria | Reuters

Nonprofit evaluators METR and Redwood Research on Thursday said they conducted a pilot review of an internal monitoring system at OpenAI, the artificial intelligence company's latest effort to determine how it will deepen its work with third-party safety organizations.

METR and Redwood said the exercise was designed for OpenAI to test what it would look like to give external researchers "a deep level of access" to its internal infrastructure.

From July 21 to Aug. 11th, an employee from each organization had access to OpenAI's internal codebase, documentation and some technical staffers. The employees were chaperoned while on site and used OpenAI-issued devices that "never left the premises," according to a release.

The evaluators discovered several "novel vulnerabilities," including an exploit that would allow agents to prevent "hundreds" of recent actions from being monitored. OpenAI told METR and Redwood that it has patched the identified vulnerabilities that impact its production safeguards, according to the release.

METR and Redwood said they agreed to only publish a high-level public post about their findings since the pilot was not designed to "provide robust public transparency about OpenAI's monitoring system." They shared a 61-page internal report with OpenAI, and said they expect to publicly share more details about future exercises.

"This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators," an OpenAI spokesperson told CNBC in a statement.

The evaluators assessed the monitoring system as it existed on and before July 21, and noted that OpenAI was expanding the system during the exercise.

watch now