Strengthening our safety ecosystem with external testing

At OpenAI, we believe that independent, trusted third party assessments play a critical role in strengthening the safety ecosystem of frontier AI. Third party assessments are evaluations conducted on frontier models to confirm or provide additional evidence to claims about critical safety capabilities and mitigations. These evaluations help validate safety claims, protect against blind spots, and increase transparency around capabilities and risks. By inviting external experts to test our frontier models, we also aim to foster trust in the depth of our capability evaluations and safeguards, and help uplift the broader safety ecosystem.

Since the launch of GPT‑4, OpenAI has collaborated with a range of external partners to test and assess our models. Broadly, our third-party collaborations take three forms:

• Independent evaluations of key frontier capability and risk areas such as biosecurity, cybersecurity, AI self-improvement, and scheming

• Methodology reviews that assess how we evaluate and interpret risk

• Subject-matter expert (SME) probing, where experts evaluate the model directly on real world SME tasks and provide structured input into our assessment of its capabilities and associated safeguards¹

This blog outlines how we use each of these forms of external assessment, why they matter, how they have shaped deployment decisions, and the principles we use to structure these collaborations. In the spirit of transparency, we’re also sharing more about the confidentiality and publication terms that govern our collaborations with third party testers. 

Why is this important? 

Third party assessors add an independent layer of evaluation alongside our internal work, strengthening rigor and providing additional protections against self-confirmation. Their input provides additional evidence alongside our own assessments, helping to inform responsible deployment decisions for powerful systems.