Tag
SpeechSense is a novel dataset for fine-grained speech sentiment analysis, focusing on paralinguistic cues to address limitations in text-centric approaches and validate the importance of acoustic features.
Introduces counterfactual audits to test whether audio language model judges actually use paralinguistic evidence when evaluating speech-to-speech responses, finding that contrastive success often overstates native reliability and similar accuracies can hide different failure modes across Gemini, GPT, and open models.
Discusses how voice agents lose paralinguistic signals like tone, hesitation, and speaker identity when transcribing to text, and questions whether and how these features are captured and used downstream.
Introduces MMEE, a multilingual multi-emotion emphasis corpus of 10,000 utterances across 7 languages and 34 emotions, and benchmarks emphasis detection models under various transfer settings, finding that multilingual training improves robustness while monolingual models show limited zero-shot transfer.
SpeechEQ introduces a benchmark and dataset for evaluating emotional intelligence in speech-language models, covering 15 EQ subscales across 2,265 dialogues. Experiments reveal current models struggle with paralinguistic cues, exhibiting text-reliant shortcuts and other limitations.
ParaBridge is an on-policy self-distillation method that bridges the gap between paralinguistic perception and dialogue behavior in speech language models, significantly improving safety and empathy without external rewards.