The Hidden Tax In How Most Enterprises Orchestrate AI

Giles Whiting is CEO of Decisions, a global AI orchestration and process automation company.

​Most enterprise AI pilots start in a similar way. A small team launches a promising use case, watches it work and checks what it costs to run. The number is trivial. The technology appears powerful and inexpensive.

Then the use case reaches production. Volume climbs, the invoice becomes material and the architecture is already difficult to change. What looked cheap in a pilot can become an expensive design choice at scale.

This is not simply a token pricing problem. In its May 2026 report, “Taming the AI Cost Curve: AI Cost Optimization Strategies,” Gartner observed the erosion of expected AI value through cost creep and what it calls a silent “token tax.” The more important question is what is creating that cost in the first place.

Often, the answer is architecture. Organizations are using probabilistic models to repeatedly make decisions that are already known.

Paying To Rethink What You Already Know

Language models are valuable when a task requires interpretation, synthesis or judgment. The problem begins when the model is also asked to manage routine control flow. It decides what comes next—routes requests, checks conditions and applies established business logic every time a process runs.

That can look elegant in a demonstration because one model appears to handle the entire flow. In production, however, the organization may be paying inference costs to answer the same fixed questions thousands or millions of times.

The cost can also grow as workflows become more complex. In some architectures, a model repeatedly rereads prior context to determine the next action. The longer the process runs, the more context it may need to process. That means additional steps can increase cost for reasons that have little to do with new reasoning and much more to do with repeatedly reconstructing the state of the process.