Tag
X-AuT is a progressive framework for compressing audio-encoder layers in speech large language models, reducing inference cost while restoring accuracy via techniques like cross-scale distillation and LoRA adaptation.
This paper introduces a method for word-level timestamping in Speech Large Language Models using relative time intervals and masked training to enhance prediction accuracy and robustness against noisy real-world annotations.
VoiceChat-TTS is a low-latency, continuous text-to-speech model designed for interactive agents, enabling real-time streaming and interruption handling without compromising speech quality.
This paper presents the first systematic study of federated training for SpeechLLM-based end-to-end ASR systems, evaluating on English and Italian tasks and achieving competitive word error rates with reduced communication costs.
This paper describes TalTech's systems for generating SOAP notes directly from doctor-patient conversation audio, using Voxtral models fine-tuned with supervised learning and DAPO reinforcement learning. Their submissions ranked first in both tracks of the BeTraC challenge, achieving high concept accuracy and low hallucination rates.
This paper introduces Lychee-FD, a native end-to-end full-duplex spoken language model that mitigates modality interference through a hierarchical parameter separation strategy, achieving significant improvements in speech intelligence and interaction fluidity.
This paper presents an open-source re-implementation of the NAVER LABS instruction-following pipeline for IWSLT 2026, using SeamlessM4T-v2-large and Qwen3-4B-Instruct, with 100k synthetic examples.
This paper proposes Joint Speech-Text Interleaved Pretraining (JSTIP), a pretraining strategy that constructs word-level and segment-level interleaved speech-text sequences to improve ASR entity accuracy and reduce the modality gap between speech and text, showing competitive performance on domain adaptation and zero-shot speech question answering.
This paper describes FBK's submission to the IWSLT 2026 Instruction Following shared task, developing SpeechLLMs for short-form and long-form speech instruction following, exploring segmentation methods and achieving robust long-form performance with fixed 30-second segmentation.
This paper identifies a localized 'entity binding failure' in Speech Large Language Models (SLLMs) where logical reasoning involving entity tracking collapses to chance-level accuracy, and proposes Entity-Aware Chain-of-Thought (EA-CoT) prompting to resolve this, achieving up to 24.4% absolute accuracy improvement.
SpeechEditBench is a bilingual multi-attribute benchmark for evaluating instruction-guided speech editing across seven atomic tasks and compositional tasks, using an anchor-based evaluation protocol with three metrics. Evaluation of mainstream Speech LLMs reveals no single model excels across all dimensions, and compositional editing remains highly challenging.
LaSR proposes a latent reasoning training paradigm for context-aware speech recognition, aligning chain-of-thought supervision around acoustic features to improve terminology recognition without added latency, outperforming standard fine-tuning on Fun-Audio-Chat.
Proposes S2ST-Omni 2, a many-to-one compositional speech-to-speech translation framework that replaces flat language labels with structured typological priors to improve multilingual adaptation, achieving superior performance on CVSS-C.
Proposes TextPro-SLM, a speech large language model that minimizes the modality gap by processing spoken input to resemble prosody-aware text input, achieving strong paralinguistic understanding with low training data.
Proposes Listen-Write-Speak (LWS), a text-first tri-channel paradigm that allows a single autoregressive LLM to continuously listen, write visible text, and speak in real-time, enabling full-duplex speech interaction without architectural modifications.