Best STT API for voice agents? I’d test latency before accuracy

I used to think the “best STT for voice agents” question meant: which one has the best transcription accuracy? I don’t think that anymore. For live agents, the transcript can be technically accurate and still ruin the call if it arrives late or keeps changing. The user doesn’t care that your WER is good. They feel: “why did the bot pause?” “why did it answer before I finished?” “why did it miss the number I corrected?” “why is it talking over me?” So my current test is less “which STT is most accurate?” and more: can the rest of the agent safely use the text fast enough? I’m trying a LiveKit + Langfuse setup where I log every turn: user starts talking first transcript fragment usable transcript LLM starts tool call voice starts user interrupts agent shuts up Smallest AI Pulse is on my shortlist here for a specific reason: I don’t want to evaluate it like a file transcription tool. I want to see whether it behaves like a real-time listening layer for a voice agent. For this use case, my scorecard would be: first usable text final transcript delay partial rewrite chaos endpointing barge-in behavior phone audio names/numbers/dates p95 turn latency Accuracy still matters, obviously. But for voice agents, latency decides whether the whole thing feels alive or fake. Anyone else measuring STT this way?