vad

The author measured the response latency on xAI's realtime voice engine, finding a floor of around 1.3 seconds, and resolved fixed first message issues by composing the opener in the prompt.

Voice agents often face latency issues not mainly from the model but from components like endpointing; optimizing VAD and using proper benchmarking can reduce turn latency effectively.

LangChain launches LangSmith tracing for voice frameworks (Pipecat, LiveKit, OpenAI Realtime, Gemini Live), enabling full audio monitoring, STT/TTS latency tracking, interruption detection, and VAD analysis with minimal code.

This article shares hard-won lessons from building real-time voice AI agents, highlighting the importance of proper turn-taking, VAD handling, billing awareness, and avoiding echo loops.

Recommends using Qwen3-ASR instead of Whisper to reduce hallucinations, using LattifAI tools for precise audio-text alignment and subtitle generation, and introducing their own OmniVAD-Kit project for voice activity detection.

After 6 months running a voice AI agent for service businesses, the author reveals that real-world latency is bimodal (median ~800ms, p95 ~2.4s) and this p95 determines user perception. Issues like VAD misfires, function call degradation with long prompts, and TTS quality matter more than LLM choice, with multilingual support adding significant costs.