Tag
The paper introduces tau-Multilingual, a benchmark extending tau-Voice to Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin for evaluating voice agents across task completion, interaction quality, and language generation. Results show Spanish, Portuguese, and Hindi perform close to English, while Korean and Mandarin drop significantly, and the authors release language packs, validated judges, and tools for community-built multilingual voice-agent evaluation.
This paper introduces Inquesto Score, a reliability protocol for voice agents that measures interaction success by defining failure events and severity levels, with diagnostic views for behavior and acoustic robustness.
This paper introduces a lightweight backchannel head for full-duplex spoken dialogue models to predict and control the timing of backchannels, improving natural conversation dynamics.
The paper introduces TACT, a benchmark for evaluating turn-taking in full-duplex spoken dialogue models that uses intent-conditioned continuous scoring to replace binary rules, demonstrating better alignment with human judgments.
This paper evaluates full-duplex speech models' ability to decide when to speak, finding that models like Moshi and PersonaPlex primarily respond to being addressed or silence rather than content-driven triggers such as false claims or hazards, identifying a gap in content understanding.
RoleBreak is an open benchmark for evaluating long-horizon role-playing robustness in spoken dialogue systems, revealing gaps in current models' ability to maintain role consistency and vocal emotion over extended interactions.
The paper introduces τ-Elicitation, a benchmark for evaluating multi-turn entity extraction in voice agents, identifying strategy selection and error recovery as key bottlenecks.
TurnBench introduces a multi-domain benchmark for assessing turn-taking dynamics in spoken dialogue, featuring a hand-labeled corpus and standardized evaluation protocols for end-of-turn and interruption detection.
M3-DuplexBench is a new multi-turn, multilingual, multidomain benchmark for evaluating full-duplex spoken dialogue systems, supporting English and Japanese across casual conversation and question answering domains.
TurnNat is a likelihood-based framework for automatically evaluating turn-taking naturalness in dyadic spoken dialogue, using a causal turn-taking prediction model trained on natural conversations to measure timing atypicality via negative log-likelihood.
This paper introduces MAD2, a new benchmark for multimodal claim verification in spoken dialogues, and proposes a calibrated fusion of audio and text models that leverages conversational context to improve verification accuracy.
This paper analyzes synchronization and turn-taking dynamics in full-duplex speech dialogue models by simulating conversations between two instances of the Moshi model, measuring representational alignment via CKA and predicting turn boundaries with LSTM probes.
VoxMind introduces an end-to-end spoken dialogue system with agentic capabilities, using a "Think-before-Speak" mechanism and dynamic tool management to boost task completion from 34.88% to 74.57% while preserving conversational quality.
WavAlign introduces a modality-aware adaptive post-training method that uses constrained preference updates and explicit anchoring to boost both semantic quality and speech expressiveness in end-to-end spoken dialogue models.