OpenAI, Anthropic negotiate landmark deal to stress-test each other’s AI models for safety risks: Report

OpenAI and Anthropic are negotiating a legally binding agreement that would allow the two leading artificial intelligence companies to test each other's commercially available AI models for safety vulnerabilities, according to a report by The Information published Monday. Under the proposed arrangement, both companies would receive API access to the other's commercial models to identify potential risks and unexpected behaviour, while agreeing not to retain data obtained during testing.

The discussions come as increasingly capable AI systems raise concerns over safety, cyber risks and the ability of developers to monitor models as they become more autonomous.

The proposed agreement would allow OpenAI and Anthropic to conduct cross-company safety assessments rather than relying solely on internal testing.

Through API access, the companies could probe each other's commercial AI models for vulnerabilities and behaviours that might not emerge during conventional evaluations. A key condition under discussion is that neither side would retain the other's data.

The Information reported that the arrangement is intended to help identify safety risks and unexpected behaviours as the companies develop increasingly advanced AI systems.

Dario Amodei, Sam Altman call for stronger AI safeguards

The negotiations follow a broader debate among AI leaders over whether the industry's rapid development is outpacing safety measures.

Anthropic CEO Dario Amodei recently called for greater caution and stronger guardrails as competition between AI companies accelerates the development of increasingly capable systems.

OpenAI CEO Sam Altman subsequently echoed the call, pledging to match Anthropic's move to give independent evaluators employee-like access to its systems.

SpaceAI founder Elon Musk, whose public disagreements with both executives are well documented, responded to the exchange in three words: “Dario is right.”

Sayantani Biswas