When AI Agents And Attackers Look Alike, What Should Defenders Test?

Jon Baker is VP, Threat-Informed Defense at AttackIQ.

An AI agent attempts to access a sensitive system outside its assigned role. Its credentials are valid, and it belongs to the organization. Is it trying to complete a legitimate task, following an instruction an attacker planted or running under credentials an attacker now controls?

While that distinction matters for the investigation, it should not determine whether defenders take the access attempt seriously. An internal agent can create an opportunity for an external adversary, and a trusted identity can perform an unauthorized action.

The immediate questions are operational: What could this identity reach? What happened around the attempted access? Which control should have interrupted the activity? Each can be answered with evidence rather than a theory about intent.

Start With The Authority The Agent Actually Has

An agent’s assigned task and its technical permissions are two different things. A task description might tell an agent to summarize support cases, while the account behind its tools can access a much wider set of records. The task describes intent, but permissions determine what actions connected systems allow.

OWASP identifies excessive functionality, permissions and autonomy as root causes of excessive agency. Its guidance calls for enforcing authorization in downstream systems rather than leaving the model to decide whether an action is allowed.

For defenders, the useful starting point is a map of the agent’s effective access. Include every account’s tools use, along with delegated access and shared service accounts. A dedicated agent identity provides little isolation if one of its tools connects through a broadly privileged account. Then compare that access with the business task. Any difference gives the team a specific boundary to test.