Reinforcement Learning for Data-Efficient Code-Switched ASR

arXiv cs.CL Papers

Summary

Introduces a reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR, achieving significant gains across 10 language pairs with minimal data.

arXiv:2607.02757v1 Announce Type: new Abstract: Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries. We propose a practical reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR using group relative policy optimization, combining an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure. Using Qwen2-Audio as a reproducible testbed across 10 language pairs, training on only TTS code-switched speech, we show that RLVR with 10% of the data matches LoRA supervised fine-tuning trained on the full dataset, with the largest gains on typologically distant pairs. The error rate reward eliminates translation errors while the script fidelity reward separately reduces script contamination without degradation. These gains transfer zero-shot to a human-recorded code-switching corpus.
Original Article
View Cached Full Text

Cached at: 07/07/26, 04:36 AM

# Reinforcement Learning for Data-Efficient Code-Switched ASR
Source: [https://arxiv.org/html/2607.02757](https://arxiv.org/html/2607.02757)
Ye Vickers

###### Abstract

Audio\-language models can be prompted for code\-switched speech, but their decoding is not optimized for code\-switching and often fails at language boundaries\. We propose a practical reinforcement learning with verifiable rewards recipe for data\-efficient adaptation of audio\-language models to code\-switched ASR using group relative policy optimization, combining an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two\-pass draft\-and\-refinement procedure\. Using Qwen2\-Audio as a reproducible testbed across 10 language pairs, training on only TTS code\-switched speech, we show that RLVR with 10% of the data matches LoRA supervised fine\-tuning trained on the full dataset, with the largest gains on typologically distant pairs\. The error rate reward eliminates translation errors while the script fidelity reward separately reduces script contamination without degradation\. These gains transfer zero\-shot to a human\-recorded code\-switching corpus\.

###### keywords:

reinforcement learning, code\-switching, automatic speech recognition, reward design

## 1Introduction

Code\-switching is a natural communicative behavior for the majority of the world's multilingual population\[poplack1980codeswitching,dogruoz2021survey\]\. Despite its prevalence, code\-switched automatic speech recognition \(CS\-ASR\) remains challenging due to the need to model rapidly shifting acoustic and linguistic cues, compounded by the scarcity of labeled code\-switched training data\[winata2023decades,sitaram2019survey\]\. Since large\-scale speech models are typically pre\-trained on monolingual corpora\[radford2022whisper,baevski2020wav2vec,chu2023qwenaudio\], data\-efficient adaptation methods are needed to endow speech\-augmented large language models \(speech\-LLMs\) with robustness to code\-switching\.

Instruction\-following speech\-LLMs can ingest language context via prompting for flexible output control, yet their autoregressive decoding is trained with token\-level cross\-entropy, which does not directly optimize sequence\-level metrics such as CER and is subject to exposure bias\[ranzato2016sequence\]\. These issues can be amplified at code\-switch boundaries, where language confusion and script hallucination have been observed\[liu2023language\_confusion\]\.

We frame CS\-ASR as a verifiable reward problem: given reference transcripts, we can compute automatic rewards and use RL to directly optimize sequence\-level transcription quality\. Rather than targeting state\-of\-the\-art performance, we use an established, publicly available speech\-LLM \(Qwen2\-Audio\) as a controlled testbed to study how reward design and data efficiency interact for code\-switched adaptation\. In this paper, we make the following contributions:

- •We applyReinforcement Learning with Verifiable Rewards \(RLVR\)\[shao2024deepseekmath\]on code\-switched ASR, using rewards based on CER and ascript fidelity rewardto directly encourage correct writing systems at code\-switch boundaries\.
- •We introduce atraining\-time two\-pass draft and refinementprocedure that conditions a second decoding pass on the best draft to encourage \`\`listen again and fix'' behavior\.
- •We evaluate on the human\-read subset of CS\-FLEURS\[yan2025csfleurs\]across 10 language pairs, showing data\-efficient gains over LoRA supervised fine\-tuning \(SFT\), and demonstrate zero\-shot transfer to SwitchLingua\[xie2025switchlingua\], a human\-recorded code\-switching corpus\.

Reference\(cmn\-eng, SwitchLingua\)这种科学发现真是太fascinating了Format\-only baselineCER 1\.81This kind of scientific discovery is reallyfascinatingChinese translated to English太…fascinating…了 frame code\-switching lostLoRA SFT\(100% data\)CER 1\.33This scientific discovery is sofascinating\.RLVR\(10% data\)CER 0\.00这种科学发现真是太fascinating了Chinese preservedEnglish insertion intactCode\-switch boundary correctFigure 1:Chinese\-English code\-switch sample in SwitchLingua
## 2Related work

Code\-switched ASR\.Code\-switching ASR must jointly model acoustics and rapidly shifting language priors, and is challenged by language confusion errors and limited labeled code\-switched data\[mustafa2022codeswitchreview\]\. Recent work incorporates explicit language tag signals and language\-aware decoding\[wang2023textderivedlid\]\. FLEURS provides a standardized multilingual benchmark\[conneau2022fleurs\], and CS\-FLEURS extends it to code\-switched pairs\[yan2025csfleurs\]\.

Speech\-LLMs for ASR\.Large\-scale speech encoder\-decoder models such as Whisper\[radford2022whisper\]have motivated speech\-language models that pair speech encoders with LLM decoders for flexible instruction following\[chu2023qwenaudio,lai2023instructionfollowing\]\. A recurring challenge is hallucination, and in neural machine translation, exposure bias has been linked to hallucination under domain shift\[wang2020exposure\_bias\], an analogy that motivates RL\-based training for code\-switching, where the language distribution shifts abruptly at boundaries\.

Sequence\-level objectives and RLVR\.Directly optimizing ASR evaluation metrics has a long history, including minimum word error rate training and minimum Bayes risk\[prabhavalkar2018mwer,shannon2017embr\]\. Reinforcement learning with verifiable rewards \(RLVR\) extends this idea using reward signals in domains where correctness can be automatically checked\[wen2025rlvr\]\. Group Relative Policy Optimization \(GRPO\) normalizes rewards within groups of sampled completions to avoid an explicit learned critic\[shao2024deepseekmath\], and has been used in reasoning RL\[guo2025deepseekr1\]\. Concurrent work has applied GRPO to monolingual ASR\[shivakumar2025grpo\_asr\]and to audio question\-answering\[rouditchenko2025omnir1\]\. Our approach adapts RLVR to code\-switched ASR with transcription rewards augmented by code\-switch consistency signals\.

## 3Method

### 3\.1Problem Setup

Given an audio segment𝐱\\mathbf\{x\}, a language context𝒞\\mathcal\{C\}, and a reference transcripty∗y^\{\*\}, the model produces a transcripty^∼πθ\(⋅∣𝐱,𝒞\)\\hat\{y\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},\\mathcal\{C\}\)\. Our goal is to improve both transcription accuracy and code\-switch fidelity\. The language context𝒞\\mathcal\{C\}consists of a instruction specifying the language pair \(\`\`The audio contains speech mixing Japanese and English''\) and format constraints\.

### 3\.2RLVR with GRPO

We train the model with GRPO, a RL method that learns from relative preferences among a small set of sampled candidate transcriptions\.

For each input\(𝐱,𝒞\)\(\\mathbf\{x\},\\mathcal\{C\}\), we sampleGGcandidate transcriptions\{y^i\}i=1G\\\{\\hat\{y\}\_\{i\}\\\}\_\{i=1\}^\{G\}from the current policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}and score each one with a verifiable rewardRi=R​\(y^i,y∗,𝒞\)R\_\{i\}=R\(\\hat\{y\}\_\{i\},y^\{\*\},\\mathcal\{C\}\)\. We then z\-score rewards within the group to obtain advantagesAiA\_\{i\}, so candidates that score above the group mean receive positive advantage and those below receive negative advantage\. This encourages the model to increase the probability of higher scoring candidates relative to their peers, without requiring a learned value function\.

We update the policy using a clipped surrogate objective with KL regularization to prevent distribution collapse:

ℒ\(θ\)=−𝔼i\[min\(ρiAi,clip\(ρi,1−ε,1\+ε\)Ai\)\]\+λKL​KL​\(πθ∥πref\)\\begin\{split\}\\mathcal\{L\}\(\\theta\)=\-\\mathbb\{E\}\_\{i\}\\Big\[\\min\\Big\(\\rho\_\{i\}A\_\{i\},\\text\{clip\}&\(\\rho\_\{i\},1\-\\varepsilon,1\+\\varepsilon\)A\_\{i\}\\Big\)\\Big\]\\\\ &\+\\lambda\_\{\\text\{KL\}\}\\text\{KL\}\(\\pi\_\{\\theta\}\\parallel\\pi\_\{\\text\{ref\}\}\)\\end\{split\}\(1\)
where

ρi=πθ​\(y^i∣𝐱,𝒞\)πθold​\(y^i∣𝐱,𝒞\)\\rho\_\{i\}=\\frac\{\\pi\_\{\\theta\}\(\\hat\{y\}\_\{i\}\\mid\\mathbf\{x\},\\mathcal\{C\}\)\}\{\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(\\hat\{y\}\_\{i\}\\mid\\mathbf\{x\},\\mathcal\{C\}\)\}\(2\)is the importance ratio between the updated policy and the sampling policy\.

### 3\.3Script Fidelity \(SHR\) Reward

When the base model is tuned for structured output, it exhibits pervasive script contamination: the output contains characters from a writing system unrelated to either language in the pair\. On SwitchLingua, 18% of non\-Latin\-script samples contain wrong\-script characters, rising to 27% for Japanese and 27% for Arabic\. To discourage this, we augment the CER reward with a binary bonus for clean outputs:

R​\(y^i,y∗,𝒞\)=−CER​\(y^i,y∗\)\+βsf⋅Script​\(y^i,𝒞\)R\(\\hat\{y\}\_\{i\},y^\{\*\},\\mathcal\{C\}\)=\-\\mathrm\{CER\}\(\\hat\{y\}\_\{i\},\\,y^\{\*\}\)\+\\beta\_\{\\mathrm\{sf\}\}\\cdot\\mathrm\{Script\}\(\\hat\{y\}\_\{i\},\\mathcal\{C\}\), whereScript​\(y^i,𝒞\)=1\\mathrm\{Script\}\(\\hat\{y\}\_\{i\},\\mathcal\{C\}\)=1if all characters iny^i\\hat\{y\}\_\{i\}belong to the allowed scripts and0otherwise\. The set of allowed scripts is𝒮=𝒮L1∪𝒮L2\\mathcal\{S\}=\\mathcal\{S\}\_\{L\_\{1\}\}\\cup\\mathcal\{S\}\_\{L\_\{2\}\}, the union of Unicode scripts used by both languages in the pair\. Non\-alphanumeric characters such as punctuation, digits, and whitespace are always permitted\. We setβsf=0\.05\\beta\_\{\\mathrm\{sf\}\}=0\.05\. Because GRPO normalizes rewards within each group, this bonus only affects the advantage when generations differ in script contamination\.

### 3\.4Two\-Step Draft Refinement

Beyond script contamination, the base model reveals further failures, On SwitchLingua, 37% of non\-Latin\-script samples are fully translated into English, and 36% of code\-switched utterances retain only the English portion\. These errors suggest that single pass decoding can commit early to an incorrect language mode\. Inspired by iterative self\-refinement in LLMs\[madaan2023selfrefine\], we address this with a two\-step procedure in which the first pass generatesGGdrafts and applies a GRPO update, then refines the highest reward draft in a second GRPO pass conditioned on the original audio and the draft \(Algorithm[1](https://arxiv.org/html/2607.02757#alg1)\)\.

This two\-step approach trains the model to both transcribe and self\-correct within a single training iteration, without requiring any external correction signal beyond the existing reward function\. We do single pass inference as with the other models during testing time\.

Algorithm 1Two\-Pass GRPO with Draft Refinement1:Audio

𝐱\\mathbf\{x\}, reference

y∗y^\{\*\}, language context

𝒞\\mathcal\{C\}, number of generations

GG
2:

3:Pass 1 \(Draft\):

4:Sample

GGdrafts:

y^\(g\)∼πθ\(⋅∣𝐱,𝒞\)\\hat\{y\}^\{\(g\)\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},\\mathcal\{C\}\)for

g=1,…,Gg=1,\\ldots,G
5:Compute rewards

R\(g\)=R​\(y^\(g\),y∗,𝒞\)R^\{\(g\)\}=R\(\\hat\{y\}^\{\(g\)\},y^\{\*\},\\mathcal\{C\}\)and GRPO loss

ℒ1\\mathcal\{L\}\_\{1\}
6:Select best draft:

y^†=y^\(arg⁡maxg⁡R\(g\)\)\\hat\{y\}^\{\\dagger\}=\\hat\{y\}^\{\(\\arg\\max\_\{g\}R^\{\(g\)\}\)\}
7:Update

θ\\thetawith

∇θℒ1\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{1\}
8:

9:Pass 2 \(Refine\):

10:Construct prompt:

\(𝐱,y^†,𝒞\)\(\\mathbf\{x\},\\hat\{y\}^\{\\dagger\},\\mathcal\{C\}\)
11:Sample

GGrefinements:

y~\(g\)∼πθ\(⋅∣𝐱,y^†,𝒞\)\\tilde\{y\}^\{\(g\)\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid\\mathbf\{x\},\\hat\{y\}^\{\\dagger\},\\mathcal\{C\}\)
12:Compute rewards

R~\(g\)=R​\(y~\(g\),y∗,𝒞\)\\tilde\{R\}^\{\(g\)\}=R\(\\tilde\{y\}^\{\(g\)\},y^\{\*\},\\mathcal\{C\}\)and GRPO loss

ℒ2\\mathcal\{L\}\_\{2\}
13:Update

θ\\thetawith

∇θℒ2\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{2\}

Table 1:CER on CS\-FLEURSread\_testand SwitchLingua \(zero\-shot transfer\)\. For each data setting, the best per language pair is inbold\. Multi\-seed columns \(nn=3\) report mean±\\pmstd \(SHR\)\. All models are trained on CS\-FLEURSXTTS\-Train\.

## 4Experimental setup

### 4\.1Datasets

CS\-FLEURS\.CS\-FLEURS\[yan2025csfleurs\]is a massively multilingual code\-switched speech dataset built on FLEURS\[conneau2022fleurs\], covering 52 languages across 113 code\-switched pairs\. Code\-switched text is generated with an align\-then\-swap procedure that replaces 30% of content words using word alignments from AwesomeAlign\[dou2021word\], yielding consistent code\-mixing across pairs\. We use two of its splits:XTTS\-Train\(128 h, 16 X\-English pairs\), where speech is synthesized with XTTS\-v2\[casanova2024xtts\], for training, andRead\-Test\(3\.3 h, 14 X\-English pairs\) which contains human\-recorded bilingual speech for evaluation\. Text includes English markers denoting code\-switched spans and our normalization strips Unicode punctuation\.

SwitchLingua\.We additionally evaluate on SwitchLingua\[xie2025switchlingua\], a multilingual code\-switching dataset with over 80 hours of human\-recorded audio across 12 X\-English pairs in single\- and multi\-turn dialogue formats\. We restrict evaluation to the 8 pairs supported by Qwen2\-Audio \(ara, cmn, deu, fra, ita, jpn, kor, rus\), excluding Cantonese, Hindi \(unsupported by the base model\) and Spanish \(mislabeled in the Huggingface dataset\)\. SwitchLingua complements CS\-FLEURS by providing naturally structured code\-switching with human\-recorded audio rather than TTS\-synthesized speech\.

### 4\.2Baselines

We compare against two baselines that isolate the contribution of RLVR:

- •Prompt\-only \(format\): A base Qwen2\-Audio model trained for a small number of iterations with a format\-only reward that encourages outputting transcripts inside<answer\>tags, without any transcription\-quality signal\.
- •LoRA SFT: LoRA\[hu2022lora\]fine\-tuning on the full CS\-FLEURS training set, providing an upper bound on what supervised training alone can achieve at this scale\.

### 4\.3Training details

Base model\.All experiments start from Qwen2\-Audio\-7B\-Instruct\[chu2023qwenaudio\], a 7B\-parameter audio\-language model with a Whisper\-based audio encoder\[radford2022whisper\]and a Qwen\-2 LLM decoder\[yang2024qwen2\], trained inbfloat16\. We choose Qwen2\-Audio as a controlled, publicly available testbed: its open weights enable full reproducibility, and its weaknesses on code\-switched speech provide a starting point for studying reward design\. Because Qwen2\-Audio does not cover all CS\-FLEURS language pairs, we restrict experiments to the 10 pairs where the base model has non\-trivial recognition capability\.

GRPO \(RLVR\)\.Both RLVR and LoRA keep the audio encoder frozen and update only the LLM decoder\. RLVR updates all decoder parameters, while LoRA updates low\-rank adapters\. We train with GRPO using DeepSpeed ZeRO\-2\[rasley2020deepspeed\]on 8 GPUs \(clipε=0\.2\\varepsilon\{=\}0\.2, initial checkpoint asπref\\pi\_\{\\text\{ref\}\}\)\. We sampleG=8G\{=\}8completions per prompt at temperatureτ=1\.0\\tau\{=\}1\.0, max length 512, with per\-device batch size 1, gradient accumulation 8, and a constant learning rate of10−610^\{\-6\}\. Two\-pass models train for 4 epochs \(2 updates per step\)\. Single\-pass models train for 8 epochs, equalizing gradient updates\. All evaluation uses single\-pass greedy decoding\. A format reward requiring<answer\>tags is shared across all configurations\. We subsample 2 310 utterances \(10%\) or 4 625 \(20%\) fromXTTS\-Trainacross 10 language pairs\. Both RLVR and LoRA experiments are run with 3 identical random seeds\.

LoRA SFT\.The LoRA baseline\[hu2022lora\]applies low\-rank adapters \(r=64r\{=\}64,α=128\\alpha\{=\}128, dropout0\.050\.05\) to all attention and MLP projections of the LLM decoder\. Training matches GRPO in batch size, accumulation, and learning rate, with 4 epochs on the fullXTTS\-Trainsplit \(10×10\{\\times\}the 10% RLVR data\)\.

### 4\.4Metrics

Character Error Rate \(CER\): the Levenshtein edit distance between the hypothesis and reference sequences, divided by the reference length\. Both strings are lowercased, stripped of punctuation, and have whitespace removed before comparison:CER=Lev​\(y^,y∗\)/\|y∗\|\\mathrm\{CER\}=\\mathrm\{Lev\}\(\\hat\{y\},\\,y^\{\*\}\)\\,/\\,\|y^\{\*\}\|\. We report both macro\-averaged CER over language pairs and micro\-averaged CER, as the two can diverge when pairs differ in reference length\.

Script Hallucination Rate \(SHR\): the fraction of samples containing any character from a disallowed Unicode script\.SHR=1N∑n𝟏\[∃c∈y^n:script\(c\)∉𝒮\]\\mathrm\{SHR\}=\\frac\{1\}\{N\}\\sum\_\{n\}\\mathbf\{1\}\[\\exists\\,c\\in\\hat\{y\}\_\{n\}:\\mathrm\{script\}\(c\)\\notin\\mathcal\{S\}\]\. This is the evaluation\-time counterpart of the SHR reward \(Section 3\.3\)\.

## 5Results

CS\-FLEURS human\-read speech\.Table[1](https://arxiv.org/html/2607.02757#S3.T1)shows that with only 20% of the training data, RLVR with two\-pass refinement and the SHR reward achieves a micro\-averaged CER of0\.1470\.147, outperforming 100% LoRA SFT \(0\.1590\.159\) despite using5×5\{\\times\}less data\. Even at 10% data, RLVR matches or surpasses LoRA trained on the full dataset\. Two\-pass refinement provides a clear gain over single\-pass CER training at 10% data\. The SHR reward substantially reduces script hallucinations \(SHR0\.0250\.025vs\.0\.0680\.068for CER\-only at 10%\) without degrading CER\. LoRA's advantage is limited to European pairs \(deu, fra, ita, por, spa\), cmn\-eng, and kor\-eng, where CER is already below 10%, while RLVR yields the largest gains on typologically distant pairs \(ara, jpn, rus\) where code\-switch transitions are hardest\.

SwitchLingua \(Cross\-dataset transfer\)\.To test whether RLVR generalizes beyond TTS\-synthesized speech, we evaluate all models zero\-shot on SwitchLingua, a human\-recorded code\-switching corpus with naturally structured multi\-turn dialogues and diverse speaker backgrounds\. Table[1](https://arxiv.org/html/2607.02757#S3.T1)shows that 20% RLVR with the SHR reward \(0\.2190\.219micro\-averaged CER\) again outperforms 100% LoRA SFT \(0\.2460\.246\) despite using5×5\{\\times\}less data, demonstrating that the gains transfer across domains\. Unlike CS\-FLEURS, two\-pass refinement alone does not improve over single\-pass CER training at 10% data\. However, adding the SHR reward recovers the gain and substantially reduces SHR \(0\.0610\.061vs\.0\.1140\.114for CER\-only\)\. The largest RLVR gains appear on ara\-eng \(0\.3930\.393vs\.0\.6520\.652for LoRA\) and rus\-eng, both typologically distant pairs where LoRA's translation failure mode is most pronounced\. LoRA retains an advantage on European pairs \(fra, deu, ita\) and cmn\-eng, consistent with the CS\-FLEURS pattern\. Overall CER are higher than on CS\-FLEURSread\_test, reflecting the greater difficulty of SwitchLingua's long\-form, multi\-turn recordings\.

## 6Analysis and Discussion

### 6\.1Do the rewards address code\-switching failures?

To isolate the contribution of each reward component, we measure translation rate \(non\-English portion rendered as English\) and wrong\-script rate \(characters from an unrelated writing system\) across the ablation chain on non\-Latin\-script samples\. Figure[2](https://arxiv.org/html/2607.02757#S6.F2)summarizes the results\.

BaseCER\+refine\+ref\+rew01010202030304040Rate \(%\)Transl\. \(SL\)Script \(SL\)Transl\. \(CF\)Script \(CF\)Figure 2:Translation and wrong\-script rates across the 10% RLVR ablation chain on non\-Latin\-script samples\. SL = SwitchLingua, CF = CS\-FLEURS\. CER reward eliminates translation\. The SHR reward is needed to reduce script contamination\.CER reward alone virtually eliminates translation errors \(37%→\\to1\.1% on SwitchLingua, 30%→\\to3\.2% on CS\-FLEURS\), since translating produces high edit distance from the code\-switched reference\. However, CER reward does not fix script contamination: wrong\-script rates remain at 18\.6% on SwitchLingua with CER\-only training, nearly unchanged from the base model \(18\.1%\)\. Adding the SHR reward reduces this to 11\.0%, a 41% relative reduction, without degrading CER\. On CS\-FLEURS the effect is stronger: wrong\-script rates fall from 9\.9% to 3\.4%\. For Latin\-script pairs, the SHR reward also improves CER indirectly by regularizing surface\-form fidelity \(\`\`seventy km''→\\to\`\`70 km''\), demonstrating the need for better evaluation metrics\.

Two\-pass refinement reduces wrong\-script rates on CS\-FLEURS \(from 9\.9% to 7\.5%\) even without the SHR reward, suggesting the second pass provides an opportunity to correct first\-pass script errors\. On SwitchLingua, refinement alone does not improve CER or wrong\-script rate\. A per\-pair breakdown shows refinement helps most pairs \(e\.g\., jpn\-eng−\.014\-\.014, cmn\-eng−\.011\-\.011on CS\-FLEURS\) but consistently hurts Arabic\-English on both datasets \(\+\.068\+\.068on CS\-FLEURS,\+\.070\+\.070on SwitchLingua\)\. On SwitchLingua, 5 of 8 pairs improve but the Arabic degradation tips the overall average negative\. This suggests the second pass can overcorrect on pairs where script boundaries are most ambiguous, and the SHR reward is needed to constrain such overcorrection\.

### 6\.2What are the cases where LoRA or RLVR fails?

We compare 20% RLVR \(with refinement and SHR reward\) against 100% LoRA SFT\. On CS\-FLEURS, RLVR outperforms LoRA by\>\>0\.2 CER on 275 samples, while LoRA wins on 132\. RLVR's advantage concentrates on typologically distant pairs: ara\-eng \(158 vs\. 19 wins\) and jpn\-eng \(24 vs\. 4\)\. LoRA's advantage appears on kor\-eng \(26 vs\. 17 wins on CS\-FLEURS, 143 vs\. 64 on SwitchLingua\) and cmn\-eng\.

In scenarios where RLVR wins, LoRA translates code\-switched spans into a single language \(CER\>\>0\.8\), while RLVR preserves code\-switch boundaries \(CER<<0\.15\)\. Conversely, RLVR garbles text or fails to preserve exact technical tokens that LoRA reproduces faithfully in cases where LoRA wins\.

LoRA achieves more near\-perfect transcriptions \(29\.3% with CER<<0\.05 vs\. 23\.1% for RLVR on SwitchLingua\), but RLVR produces fewer catastrophic errors \(10\.1% with CER\>\>0\.5 vs\. 11\.8%\), consistent with RL optimizing expected reward rather than memorizing individual examples\.

## 7Conclusion

We showed that RLVR with GRPO enables highly data\-efficient adaptation of speech\-LLMs to code\-switched ASR\. With only 10% of the training data and∼40×\{\\sim\}40\{\\times\}fewer gradient steps, RLVR matches LoRA SFT trained on the full dataset across 10 language pairs, and at 20% outperforms it, with the largest gains on typologically distant pairs where code\-switch transitions are hardest\. A CER reward eliminates translation errors, while a dedicated SHR reward is needed to reduce script contamination, and a two\-pass draft refinement paradigm has potential to mitigate both error types\. These gains transfer zero\-shot from TTS\-synthesized training data to human\-recorded code\-switching speech, demonstrating robustness across acoustic domains\.

## 8Generative AI Use Disclosure

LLMs were used to produce a fraction of the source codes for experiment orchestration\. Additionally, LLMs were used for spellchecking and suggesting edits to further improve the fluency and conciseness of the paper during the drafting process\.

## References

Similar Articles

Benchmarking Commercial ASR Systems on Code-Switching Speech: Arabic, Persian, and German

arXiv cs.CL

This paper presents a benchmark evaluating five commercial ASR systems on code-switching speech across Arabic-English, Persian-English, and German-English pairs, using a two-stage pipeline to select 300 samples per pair and assessing performance with WER and BERTScore. ElevenLabs Scribe v2 achieves the lowest overall WER (13.2%) and highest BERTScore (0.936), with public dataset available.