Continuously hardening ChatGPT Atlas against prompt injection

Agent mode in ChatGPT Atlas is one of the most general-purpose agentic features we’ve released to date. In this mode, the browser agent views webpages and takes actions, clicks, and keystrokes inside your browser, just as you would. This allows ChatGPT to work directly on many of your day-to-day workflows using the same space, context, and data.

As the browser agent helps you get more done, it also becomes a higher-value target of adversarial attacks. This makes AI security especially important. Long before we launched ChatGPT Atlas, we’ve been continuously building and hardening defenses against emerging threats that specifically target this new “agent in the browser” paradigm. Prompt injection⁠ is one of the most significant risks we actively defend against to help ensure ChatGPT Atlas can operate securely on your behalf. 

As part of this effort, we recently shipped a security update to Atlas’s browser agent, including a newly adversarially trained model and strengthened surrounding safeguards. This update was prompted by a new class of prompt-injection attacks uncovered through our internal automated red teaming.

In this post, we explain how prompt-injection risk can arise for web-based agents, and we share a rapid response loop we’ve been building to continuously discover new attacks and ship mitigations quickly—illustrated by this recent security update.

Prompt injection as an open challenge for agent security

A prompt injection attack targets AI agents by embedding malicious instructions into content the agent processes. Those instructions are crafted to override or redirect the agent’s behavior—hijacking it into following an attacker’s intent, rather than the user’s.

We’ve made progress defending against prompt injection through multiple layers of safeguards, as we shared in an earlier post⁠. However, prompt injection remains an open challenge for agent security, and one we expect to continue working on for years to come.