Voice agents in noisy environments

Reddit r/AI_Agents Models
voice-agents noise-cancellation speech-recognition word-error-rate primary-speaker asr

Summary

A speech company trained a model that cancels noise and identifies the primary speaker, achieving 50% lower word error rate on leading ASR models in noisy environments.

I have been working with Voice agents since more than a year now. And they always break in certain environments - noises, people talking around, etc. I looked for solutions, even asked on a few subreddits, and found nothing that worked well. Being a Speech company, we decided to solve this. We trained a model that identifies the primary speaker and cancels out everything else. Months of exploration, data sourcing, and multiple model iterations later, we built something that is now outperforming industry benchmarks- 50% lower WER on leading ASR models. Since I actively follow and learn from this group, just wanted to share the journey. :)
Original Article

Similar Articles

I evaluated top STT models on large Real-World data

Reddit r/AI_Agents

An evaluation of leading STT models on 1000+ noisy real-world clips reveals most perform poorly in noisy environments, with DG Nova performing best. Applying noise cancellation significantly improves accuracy.

@MaxForAI: If you are working on voice agents, you should try this project. A team from NTU, NUS, and Shanghai AI Lab released: Mega-ASR. This fully open-source ASR is built on Qwen3-ASR, aiming to break the long-standing bottleneck of ASR performance in noisy, reverberant, or other impaired real-world environments...

X AI KOLs Timeline

NTU, NUS, and Shanghai AI Lab jointly released Mega-ASR, a fully open-source ASR model built on Qwen3-ASR. Using the Voices-in-the-Wild-2M dataset and progressive acoustic-to-semantic optimization, it achieves up to 30% relative Word Error Rate (WER) reduction in real-world noisy environments. With only 1.7B parameters, it enables efficient inference on consumer-grade hardware.

Can Voice Agents Handle Bilingual Customers? Benchmarking Frontier ASR on Code-Switched Speech

Hugging Face Blog

ServiceNow AI releases a benchmark and dataset for evaluating automatic speech recognition (ASR) on code-switched speech across four language pairs (Spanish-English, French-English, Canadian French-English, German-English) in enterprise HR and IT scenarios, finding that current frontier ASR models still struggle with code-switching, leading to higher error rates.

Room reverberation and low SNR hurt STT accuracy far more than model size

Reddit r/ArtificialInteligence

Room reverberation and low-frequency noise from the environment hurt speech-to-text accuracy far more than the choice of model size; front-end audio preprocessing like adaptive spectral subtraction can recover masked phonemes and reduce word error rate more effectively than upgrading the model backend.

My voice agent sounded smart until one phone number was transcribed wrong.

Reddit r/AI_Agents

The article argues that voice agent STT should be evaluated on entity accuracy (e.g., phone numbers, dates) rather than general word error rate, because missing critical fields can break workflows. It mentions testing with HubSpot fields and notes Smallest AI Pulse as an interesting tool for capturing workflow-critical entities in real time.