Tag
The paper introduces StyleInstructCaps, the first standardized benchmark for controllable speaking-style captioning (SSC) in audio-language models, evaluating groundedness, hallucination, instruction-following, and speaker consistency across nine state-of-the-art models.
SEA-SpeechBench is the first large-scale multitask benchmark for evaluating speech understanding in 11 Southeast Asian languages, highlighting performance gaps in current models.
Real World VoiceEQ is a new benchmark for evaluating the human quality of voice AI, based on over a million human ratings, assessing models across speech recognition, synthesis, and understanding in real-world conditions.
This paper identifies a localized 'entity binding failure' in Speech Large Language Models (SLLMs) where logical reasoning involving entity tracking collapses to chance-level accuracy, and proposes Entity-Aware Chain-of-Thought (EA-CoT) prompting to resolve this, achieving up to 24.4% absolute accuracy improvement.