
Sidhant Kabra, Forbes Councils Member
· 2 min read
Voice AI Benchmarks Are Not Scoreboards
Sidhant Kabra is a Co-Founder of Cekura, a company that is pioneering Voice AI evals and benchmarks.
Every few days, a new voice AI model ships with a chart that puts it on top: fastest reply, lowest error rate, most human.
As the co-founder of Cekura, a company that tests voice agents and publishes voice AI benchmarks, my advice is to use the rank to build a shortlist, not to make the decision. Picking the top three to five is a good starting point. Past that, in my view, the ranking is about 2% of a benchmark’s value. The other 98% is the specific finding: where an agent breaks, compared with whom and under what conditions.
Benchmarks are experiments. Change the line quality, the caller’s accent, the workflow or how slowly a tool responds, and the order changes. Absolute numbers don’t transfer to your business. Specific gaps do.
Why A Benchmark Is An Experiment
Start by holding the agent constant. In one public experiment, we deployed the same Medicare insurance agent on six voice platforms and ran 23 scenarios three times on each. Workflow scores ranged from 65.2% to 95.7%. Same prompt, same tools, same workflow.
Now change what you count. One platform passed 69.6% of scenarios when we scored only whether the agent took the right actions. Once we also counted infrastructure failures, such as long silences during tool calls that left callers unable to finish, it passed 8.7%. Neither number is wrong. They answer different questions.
Swap the language model and the story flips again. With everything else held fixed, an open-source model beat an established proprietary one on reliability (88.1% vs. 76.3%) and replied a second faster at the median. But the proprietary model won red-team safety, 100% vs. 80%. Which is “better” depends on whether your agent books haircuts or moves money.
Where Agents Lag: Against Each Other And Against Humans
None of that shows up in a single score. All of it decides whether a caller trusts the agent.
Original source
This story was published by Forbes: Innovation and written by Sidhant Kabra, Forbes Councils Member. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on forbes.com


