The Two Worst Words In Enterprise AI

Daniel Fallmann is founder and CEO of Mindbreeze, a leader in enterprise search, applied artificial intelligence and knowledge management.

​Two words are starting to dominate enterprise AI usage: with review.

The agent will open the service ticket with review. Update the record, issue the credit, adjust the order, with review. Those two words are why the risk committee signed off. They are also, in most cases, why the project will not produce the promised return.

Here is the uncomfortable arithmetic. Automation only creates value when checking the work costs meaningfully less than doing it. If a person must open the system, reconstruct the context and confirm each action the agent took, you have not removed labor. You have converted it from execution into verification and moved it to a different desk. Verification is expensive not because humans are cautious but because they are uncertain.

A study published in June examined what happens when agent guardrails escalate decisions to a human. Oversight capacity is finite and depletes. By the 300th approval of a benign action, a reviewer is fatigued and primed to keep clicking approve, so a dangerous action buried in that stream gets waved through. A guard who escalates only five times a day gets a reviewer who reads carefully. The system with more oversight produced the worst outcome because its trivia escalations drew from the same finite pool of attention the serious matter needed.

Attention is not a policy setting; it is a consumable. Every low-stakes approval spends down a budget you will need later, for the one decision that actually mattered.

The Reviewer Who Is Present But Powerless

Most enterprises have deployed the first and assumed the second, and field research on how people actually oversee software agents suggests the assumption is doing a lot of unearned work. A human who can understand what the system did in time to act on it is a control; a human merely sitting near it is not. Only one is on your org chart.