AI Voice Agents Are Easy To Demo; Production Is Where The Engineering Begins

Alexandro Picolini is a technology executive and software architect focused on telecom, cloud and AI.

Voice AI demos have become surprisingly easy to build. With a real-time model, a microphone and a good prompt, a team can create something impressive quickly. But a demo is usually a controlled environment: clean audio, a stable network, cooperative users and readily available data.

A real telephone operation is very different.

I am a CTO and software architect with more than 25 years of experience in telecommunications and software development. My company develops contact-center platforms that connect telephony, digital channels and customer management. Our work integrating voice AI into these environments has shown me how much production depends on systems beyond the model.

The AI Only Hears The Audio It Receives

One of the first problems we faced was telephone audio quality. As a call passes between carriers, its audio may be compressed and converted along the way. Receiving it in a higher-quality format at the final connection does not restore detail already lost. A person may still understand the conversation while a speech-recognition system struggles.

Speakerphone calls add background conversations, television and traffic. Voice activity detection—the technology that detects when someone is speaking—does not always distinguish the caller from surrounding voices.

It’s important to test through the actual telephone routes customers use, not only a headset connected to a computer. Teams should include mobile phones, speakerphones and noisy environments. They should listen to the audio reaching the agent, not just the voice it generates.

Turn-Taking Is Still Difficult

In some of our implementations, around two seconds has been a workable compromise. It is not a universal setting: Turn detection requires more than detecting sound.