MoVE: Translating Laughter and Tears via Mixture of Vocalization Experts in Speech-to-Speech Translation
Summary
MoVE proposes a Mixture-of-LoRA-Experts architecture that preserves laughter and crying in speech-to-speech translation, achieving 76% NV retention with only 30 minutes of curated data.
View Cached Full Text
Cached at: 04/22/26, 02:41 PM
Paper page - MoVE: Translating Laughter and Tears via Mixture of Vocalization Experts in Speech-to-Speech Translation
Source: https://huggingface.co/papers/2604.17435 Published on Apr 19
·
Submitted byhttps://huggingface.co/47z
陳思齊on Apr 22
Abstract
MoVE, a Mixture-of-LoRA-Experts architecture with expressive-specialized adapters and a soft-weighting router, enables effective speech-to-speech translation with preserved non-verbal vocalizations while achieving high naturalness and emotional fidelity using minimal curated data.
RecentSpeech-to-Speech Translation(S2ST) systems achieve strong semantic accuracy yet consistently strip awaynon-verbal vocalizations(NVs), such as laughter and crying that convey pragmatic intent, which severely limits real-world utility. We address this via three contributions. First, we propose a synthesis pipeline for building scalableexpressive datasetsto overcome the data scarcity limitation. Second, we propose MoVE, aMixture-of-LoRA-Expertsarchitecture withexpressive-specialized adaptersand asoft-weighting routerthat blends experts for capturing hybrid expressive states. Third, we show pretrainedAudioLLMsenable striking data efficiency: 30 minutes of curated data is enough for strong performance. On English-Chinese S2ST, while comparing with strong baselines, MoVE reproduces target NVs in 76% of cases and achieves the highest human-rated naturalness and emotional fidelity among all compared systems, where existing S2ST systems preserve at most 14% of NVs.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2604\.17435
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2604.17435 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2604.17435 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2604.17435 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Breaking the Curse ofMultilinguality inMany-to-Many Speech-to-Text Translation via a Resource-AwareMixture of Speech Encoders
This paper introduces MSRT, a framework with a resource-aware Mixture of Speech Encoders (MoSE) to overcome the curse of multilinguality in many-to-many speech-to-text translation. The 4B-parameter model achieves state-of-the-art results across 45 languages, particularly improving low-resource speech translation with only 10 hours of paired data per language.
MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition
MoLGE assigns dedicated expert modules to clusters of similar languages in a mixture-of-experts framework for large-scale multilingual ASR, achieving improvements across 495 languages with minimal parameter increase.
Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
Mix-MoE proposes a mixed Mixture-of-Experts framework with specialized expert groups and Fourier-transform-enhanced routing to mitigate parameter interference in multilingual machine translation, achieving significant improvements over baselines.
VTaMo: Video-Text Alignment Model for Sign Language Translation
VTaMo introduces explicit multi-granularity video-text alignment for sign language translation using optimal transport and contrastive learning, achieving state-of-the-art performance on four benchmarks.
Mixture-of-Translators: Translating KV Caches Across Heterogeneous Large Language Models
This paper introduces Mixture-of-Translators (MoT), a framework for translating KV caches across heterogeneous LLMs, enabling cache reuse between different architectures. Experiments show preserved QA performance and long-context quality across Qwen2.5, GPT-2, and OPT models.