SyncAI.news, a Varaisys broadcasting
Beyond Token Counts: A Framework For Measuring AI ROI
MB

Monica Bajaj, Forbes Councils Member

· 1 min read

World NewsForbes: Innovation

Beyond Token Counts: A Framework For Measuring AI ROI

Monica Bajaj is the Senior Vice President of Engineering at Okta.

Engineering leaders are facing an uncomfortable AI reality. Token consumption is soaring, while true business impact often remains unclear. Companies want their teams to embrace AI, but there’s growing pressure to demonstrate a return on that investment. Engineering leaders want to see greater velocity. Security leaders want to ensure that all systems are safe. Finance leaders want to know how dollars are spent and if they are getting the proper return. It’s a three-edged sword, and that tension is building every day.​

Part of the problem is that our systems, architectures and workflows were designed for humans. AI agents are the new kid on the block. Humans find workarounds when processes aren’t working. Agents can continue looping, consuming resources without producing meaningful results. We need to rethink how we measure AI’s value, distinguishing between consumption and outcomes.

Token Consumption Is Not Business Value

In my opinion, token consumption measures system activity, not business value. It tells you what you’re spending, not what you’re getting in return. A sharp increase in token volume can reflect inefficient system design, bloated prompts, repetitive trial and error or agents caught in unnecessary loops. Someone might repeatedly prompt an AI tool because the output isn’t useful, consuming more tokens without improving productivity.

That’s why I believe we need to shift our mindset from cost per token to cost per successful outcome. Instead of measuring consumption alone, evaluate the cost per resolved task, quality and defect rates, task completion velocity and caching efficiency. Evaluate whether a use case needs AI or would be better served by traditional automation.

We should always be asking whether we’re getting meaningful outcomes at a reasonable cost.

Original source

This story was published by Forbes: Innovation and written by Monica Bajaj, Forbes Councils Member. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on forbes.com

Similar News