Convex Low-resource Accent-Robust Language Detection in Speech Recognition

Hugging Face Daily Papers Papers
speech-recognition low-resource accent-robust convex-optimization language-detection asr

Summary

This paper introduces CLD, a lightweight convex optimization-based language detection head for ASR that achieves 97-98% accuracy with under 100 training samples while reducing compute costs by 13x, addressing accent and dialect robustness across 5 languages and 24 sub-dialects.

Globalization and multiculturalism continue to produce increasingly diverse speech varieties. Yet current spoken dialogue systems frequently fail on under-represented dialects and accents, often misidentifying the input language and causing cascading failures in downstream dialogue tasks. Addressing this dialectal variance under low-resource constraints remains an open challenge, as standard fine-tuning is computationally expensive and prone to overfitting on high-dimensional speech data. We propose Convex Language Detection (CLD), a novel framework that integrates theoretically grounded convex optimization techniques into the spoken dialogue systems pipeline. Our method is efficiently implemented via multi-GPU Alternating Direction Method of Multipliers (ADMM) in JAX, thus providing global optimality guarantees and fast training in polynomial time. Theoretically, we prove that our convex objective induces certified margin stability and provide guarantees against feature perturbations. Empirically, we demonstrate sample efficiency and robustness to input dialectical variation, achieving 97-98% accuracy in challenging low-resource regimes. Our open-source package is available at https://pypi.org/project/jaxcld/
Original Article
View Cached Full Text

Cached at: 05/29/26, 11:04 PM

Paper page - Convex Low-resource Accent-Robust Language Detection in Speech Recognition

Source: https://huggingface.co/papers/2605.23235 🎵 Meet Convex Language Detection (CLD)!

Automatic Speech Recognition (ASR) frequently exhibits failures on accents and dialects. But collecting more data to retrain a larger model is slow and expensive. CLD solves this—not by grid-searching hyperparameters or collecting massive datasets, but through the elegant geometry of convex optimization.

🌐🎙️ Instead of relying on unpredictable large-scale neural networks that struggle with accent variance, CLD introduces a lightweight, pluggable detection head that yields mathematically certified margin stability.

We benchmarked CLD across 5 languages, 24 unique sub-dialects (including highly challenging regimes like Singaporean English and regional Mandarin), and foundational models like Whisper and MMS-1B. The results: Even with under 100 training samples, CLD locks in 97–98% accuracy, reduces cross-lingual decoding failures, and cuts compute costs by a massive 13x.

The structural shift is fundamentally distinct: Current multilingual ASR models are heavily imbalanced toward standard, high-resource speech datasets, leaving millions of global speakers facing cascading errors. By recasting language identification as a convex program solved via parallelized ADMM in JAX, we don’t just guess a boundary—we calculate a verifiable radius of label invariance with guarantees. We see this as a highly scalable, theoretically backed plug-and-play module which aims to bring equity, speed, and reliability to global speech systems.

🛠️ Open-Source Code:https://github.com/pilancilab/CLD 📦 JAX Package: pip install jaxcld (https://pypi.org/project/jaxcld/) 📄 Full Paper:https://arxiv.org/abs/2605.23235

Similar Articles

Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs

arXiv cs.CL

This paper applies Direct Preference Optimization (DPO) to align Audio LLMs for transcribing English-Mandarin code-switching speech, achieving up to 89.6% MER reduction in-distribution and 20% out-of-distribution. It identifies three failure modes—language omission, translation instead of transcription, and hallucination—and shows that preference-based alignment effectively elicits correct code-switching behavior from multilingual Audio LLMs.

CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language

arXiv cs.CL

CRoCoDiL proposes a continuous and robust conditioned diffusion approach for language that shifts masked diffusion models into a continuous semantic space, achieving superior generation quality and 10x faster sampling speeds compared to discrete methods like LLaDA.

LaSR: Context-Aware Speech Recognition via Latent Reasoning

arXiv cs.CL

LaSR proposes a latent reasoning training paradigm for context-aware speech recognition, aligning chain-of-thought supervision around acoustic features to improve terminology recognition without added latency, outperforming standard fine-tuning on Fun-Audio-Chat.