Trusting AI Agents Means Preserving Human Intent

Arun Kumar Elengovan is director, engineering security at Okta.

The transition from AI making recommendations to decision-making and action itself transforms the equation of trust. In the case when AI makes a recommendation, the human is still responsible for analyzing the recommendation and making further decisions based on it. Nevertheless, an autonomous agent is able to make its own decisions, gain access to sensitive information, pay and interact with other agents without any human intervention.

Without the right safeguards, we risk creating agents that have authority to act on our behalf but lack the boundaries required to protect our interests. That could lead to privacy violations, financial losses and, ultimately, a loss of human control.

As I see it, the future of agentic computing should be oriented toward intent-based AI authorization, whereby any action taken autonomously should always be in accordance with the original intent of the human.

Permission Is No Longer Enough

Traditional identity and authorization systems were designed primarily to determine who can access a resource and what operations they’re permitted to perform. Identity and access management, role-based access control and protocols such as OAuth determine whether you are authorized to access information or perform certain actions. That model made sense because the human was doing the work.

Agentic AI introduces a different challenge. An AI agent might have legitimate permission to access financial records or make a payment, yet still use that access in a way the user never intended. Imagine asking an agent to book a rental car. It might also have access that allows it to book your flight or hotel. It may have permission to perform those actions, but that does not mean it should.

The next generation of authorization therefore has to go beyond verifying whether an agent can perform an action. It needs to determine whether that action should be performed in the context of the human’s instructions.