The Balance Sheet Liability Of 'Mostly Right' AI

Raju Malhotra is Chief Product & Technology Officer at Certinia where he leads product strategy and AI innovation for professional services.

​Ask any enterprise software vendor if their AI is accurate, and the answer is almost always yes. Ask what “accurate” means, and the conversation gets vague fast. A chatbot that gets something wrong 1% of the time in a consumer setting is a minor annoyance. That same 1% behaves very differently in the parts of a business that touch cash, contracts and compliance.

The Math Behind ‘99% Accurate’

Consider a mid-sized professional services firm processing 10,000 invoices a month through an AI agent running at 99% accuracy. That sounds like a strong number. It means 100 invoices a month with an error a human has to catch, correct or explain to an auditor. Multiply that across a fiscal year, and you get over 1,000 small failures sitting in the general ledger, waiting to surface at close, in an audit or in a client dispute.

The trouble starts when that response is one step in a longer process, because step-level accuracy doesn’t carry over to the whole task. It multiplies against itself. An agent that is 99.9% accurate at each step of a 20-step invoicing workflow (matching line items, flagging exceptions, routing approvals and applying the right rate card) isn’t 99.9% accurate on the finished invoice. Run the math (0.999 raised to the 20th power), and the task-level success rate drops to about 98%.

Two percent sounds small until you apply it at volume. A firm processing 10,000 invoices a month at that rate generates roughly 200 invoices a month with an error buried somewhere in the chain, more than 2,000 a year sitting in the general ledger waiting to surface at close, in an audit or in a client dispute.

The Reliability Gap In The Data

Fixing The Problem Upstream

What To Ask Before You Trust An Agent With The Ledger

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?