Tag
Introduces a reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR, achieving significant gains across 10 language pairs with minimal data.
A significant breakthrough in Levantine Arabic speech synthesis and English code-switching, achieving a 76% improvement using a single RTX 3060 in an evening.
The author shares their experience swapping the TTS in their voice agent to a custom model (Banter 1) designed for bilingual Arabic-English conversations, which significantly reduced perceived lag.
This paper introduces STEER, a gradient-guided attack that exploits LLMs' safety training distribution by translating high-attribution words into low-resource languages to bypass refusal mechanisms, achieving up to 96.7% attack success rate on AdvBench and transferring to GPT-4o-mini at 35.5% ASR.
This paper introduces Progressive Code-Switching (PCS), a reinforcement learning approach with curriculum learning that gradually increases code-switching in LLMs to efficiently transfer multilingual reasoning capabilities.
Universal-3.5 Pro improves native code switching, diarization, and adds more languages, enhancing speech recognition capabilities.
Introduces MultiClin, a benchmark for evaluating ASR in multiscript clinical settings, showing that script unification improves performance over conventional single-reference metrics.
ServiceNow AI releases a benchmark and dataset for evaluating automatic speech recognition (ASR) on code-switched speech across four language pairs (Spanish-English, French-English, Canadian French-English, German-English) in enterprise HR and IT scenarios, finding that current frontier ASR models still struggle with code-switching, leading to higher error rates.
Proposes a POI-aware contrastive training framework using LLM-generated near-misses to improve ASR robustness at code-switching regions, achieving consistent error reductions on two benchmarks.
This paper investigates whether code-switching ASR capabilities learned from limited seen language pairs can generalize to unseen pairs using model merging and domain generalization methods, finding only modest transfer.
This paper introduces BOUTEF, a large-scale multilingual corpus for studying fake news in Algeria and Tunisia, covering Arabic dialects, Arabizi, French, English, and code-switching. It includes empirical analysis of linguistic strategies and engagement dynamics.
This paper applies Direct Preference Optimization (DPO) to align Audio LLMs for transcribing English-Mandarin code-switching speech, achieving up to 89.6% MER reduction in-distribution and 20% out-of-distribution. It identifies three failure modes—language omission, translation instead of transcription, and hallucination—and shows that preference-based alignment effectively elicits correct code-switching behavior from multilingual Audio LLMs.
This paper presents a benchmark evaluating five commercial ASR systems on code-switching speech across Arabic-English, Persian-English, and German-English pairs, using a two-stage pipeline to select 300 samples per pair and assessing performance with WER and BERTScore. ElevenLabs Scribe v2 achieves the lowest overall WER (13.2%) and highest BERTScore (0.936), with public dataset available.
MUSCAT is a new multilingual, scientific conversation benchmark dataset for evaluating ASR systems on challenging multilingual scenarios including code-switching, domain-specific vocabulary, and mixed language input. The dataset consists of bilingual discussions on scientific papers between speakers using different languages, with results showing current state-of-the-art systems struggle with these multilingual challenges.
This paper introduces a data-efficient fine-tuning framework for teaching reasoning models to code-switch (mix languages) effectively, demonstrating that strategic code-switching can improve reasoning capabilities for lower-resource languages. The work analyzes code-switching behaviors in large language models across diverse languages, tasks, and domains, then develops interventions to promote beneficial code-switching patterns.
Researchers introduce CSR-L and CS-MTEB benchmarks showing that code-switching queries degrade IR system performance by up to 27%, revealing embedding-space divergence that current multilingual techniques cannot fix.