Nvidia releases software platform to stop AI agents from misbehaving

Nvidia CEO Jensen Huang speaks during the keynote address at Salesforce's Dreamforce conference at the Moscone Center on Sept. 15, 2026 in San Francisco, California.

Benjamin Fanjoy | Getty Images

Nvidia is rolling out a new software platform to allow AI developers to set safeguards for agents and prevent them from breaking out of containment.

The release on Monday of Nvidia's Open Agent Safety Platform comes after companies including OpenAI, Anthropic, Meta, and Google disclosed recent incidents in which their artificial intelligence models escaped their sandboxes and attempted to hack other companies and access their computer systems.

An Nvidia representative told reporters on a call on Sunday that its platform could have prevented OpenAI's HuggingFace incident in July. That's when OpenAI models escaped containment, accessed the open internet and breached Hugging Face, which operates an open-source developer platform. 

"Each security incident is unique, and we have to look at all of them in detail," said Justin Boitano, vice president of enterprise AI at Nvidia, the world's most valuable company. "From what we know, Hugging Face reported over 17,000 agents attacking their infrastructure that went on for days and weeks."

Nvidia has been at the center of the generative AI boom since the launch of ChatGPT almost four years ago, as the chipmaker's graphics processing units are critical to the development of large language models and to the AI services offered by hyperscalers. But CEO Jensen Huang has more recently emerged as a key voice in the AI safety debate, arguing that many security concerns are engineering issues that can be solved through computer science and product development.

"You have to think about what you could have done, what's the solution for it," Huang said in a podcast with The New York Times' Ezra Klein released last week, referring to recent incidents. "In the future, improve your process so that you could avoid this from happening again."