Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR
Summary
This paper investigates the impact of data scale versus latency on cross-lingual transfer for streaming ASR, finding that multilingual initialization benefits are data-limited, not latency-limited, and diminish as target-language data increases.
View Cached Full Text
Cached at: 06/24/26, 07:45 AM
# Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR
Source: [https://arxiv.org/html/2606.24169](https://arxiv.org/html/2606.24169)
Nenad BanficThis work has been submitted to the IEEE for possible publication\. Copyright may be transferred without notice, after which this version may no longer be accessible\.
###### Abstract
Adapting a streaming speech recognition model to a new language requires choosing between two plausible warm starts: a multilingual \(ML\) encoder or an English\-only \(EN\) encoder\. The common intuition is that the multilingual encoder should help most at low data, but it is unclear how long that advantage persists, whether tight streaming latency amplifies it, and whether it survives deployment quantization\. We answer these questions with a controlled sweep of a 0\.6 B\-parameter cache\-aware FastConformer transducer across eight European languages, up to five target\-language data scales \(100 h to 2500 h\), three streaming tiers plus offline decoding, and up to four public test sets\.
The main result is that multilingual initialization is a*data\-limited*advantage, not a*latency\-limited*one\. On FLEURS at 160 ms, the mean EN–ML word error rate \(WER\) gap falls from\+4\.21\+4\.21percentage points \(pp\) at 100 h to\+0\.20\+0\.20pp at 2500 h; a power\-law fit summarizes this decay, with each doubling of target\-language data roughly halving the remaining advantage\. Across the three streaming tiers, the across\-language mean EN–ML gap is approximately stable at each scale from 100 to 1000 h, and is near zero by 2500 h\. Finally, 4\-bit weight\-only encoder quantization at the matched 560 ms streaming tier reduces the encoder footprint by about3×3\{\\times\}, with an average FLEURS WER increase of about0\.50\.5pp\. The resulting guideline is simple: use multilingual initialization in low\-data regimes, treat the choice as effectively irrelevant at large data, and make latency and quantization decisions independently\.
## IIntroduction
Production automatic speech recognition \(ASR\) increasingly requires*streaming*operation: emitting partial hypotheses with bounded look\-ahead so that voice assistants, live captioning, and real\-time translation feel responsive\. Modern cache\-aware Conformer architectures\[[1](https://arxiv.org/html/2606.24169#bib.bib1),[2](https://arxiv.org/html/2606.24169#bib.bib2),[3](https://arxiv.org/html/2606.24169#bib.bib3)\]satisfy this requirement by constraining the encoder’s left and right contexts, maintaining activation caches across non\-overlapping audio chunks, and supporting multi\-lookahead training so that a single model can be deployed at multiple latency operating points\.
Bringing such a model to a new language is, in practice, a fine\-tuning problem\. An organization holding both a 0\.6\-billion\-parameter \(0\.6 B\) English streaming encoder and a 0\.6 B\-parameter multilingual streaming encoder must choose, per target language and per data budget, which to initialize from\. Prior work on cross\-lingual transfer for streaming ASR establishes that pretraining helps over random initialization\[[4](https://arxiv.org/html/2606.24169#bib.bib4),[5](https://arxiv.org/html/2606.24169#bib.bib5)\], and multilingual speech foundation models continue to push the state of the art\[[6](https://arxiv.org/html/2606.24169#bib.bib6),[7](https://arxiv.org/html/2606.24169#bib.bib7)\]\. However, the practically interesting comparison is*between two viable pretrained encoders*that differ in pretraining language coverage, not between pretraining and random initialization; a random\-encoder sanity check under the same recipe was not competitive and is reported in the supplement\. Throughout this paper we use*multilingual*\(ML\) for the encoder trained jointly on multiple languages, and*English\-only*\(EN\) for the encoder trained on English data only\. Existing transfer studies for streaming ASR cover one or two target languages \(Hindi\[[4](https://arxiv.org/html/2606.24169#bib.bib4)\], French/Italian\[[5](https://arxiv.org/html/2606.24169#bib.bib5)\]\), do not vary streaming latency at evaluation, and do not include cache\-aware Conformers; multilingual foundation\-model work \(XLSR\[[6](https://arxiv.org/html/2606.24169#bib.bib6)\], USM\[[7](https://arxiv.org/html/2606.24169#bib.bib7)\]\) reports many languages but evaluates in offline/non\-streaming settings rather than across streaming\-latency tiers\.
We sweep four axes simultaneously — target language \(eight European languages spanning Germanic, Romance and Slavic\), training data scale \(up to five levels per language, 100 h to 2500 h\), streaming\-inference latency \(three streaming tiers at 160, 560 and 1120 ms plus an offline batch reference; we reserve “streaming tier” for the three throughout, naming offline explicitly when it is included\), and evaluation domain \(up to four public test sets\) — for both ML and EN initializations\. The eight target languages are German \(DE\), Dutch \(NL\) and Icelandic \(IS\) from Germanic; Spanish \(ES\), French \(FR\) and Portuguese \(PT\) from Romance; and Croatian \(HR\) and Polish \(PL\) from Slavic\. The result is, to our knowledge, the largest published cross\-lingual transfer study for a streaming\-trained ASR architecture, spanning seed\-reproducibility, Slavic\-pivot and hybrid\-encoder runs, plus INT4 quantization sweeps over the main grid\.
The headline finding is that multilingual initialization is a*data\-limited*advantage, not a*latency\-limited*one: across the eight languages the mean EN–ML FLEURS gap closes monotonically with target\-language hours and is well summarized by a power law \(Sec\.[IV\-A](https://arxiv.org/html/2606.24169#S4.SS1)\), while the across\-language mean gap is approximately stable across the three streaming\-latency tiers at every data scale where it is non\-trivial \(Sec\.[IV\-B](https://arxiv.org/html/2606.24169#S4.SS2)\)\. 4\-bit weight\-only encoder quantization at the matched 560 ms tier costs roughly half a WER point on FLEURS \(Sec\.[V](https://arxiv.org/html/2606.24169#S5)\), small compared to the EN–ML gap at low data\. Per\-language and per\-corpus breakdowns \(Sec\.[IV\-C](https://arxiv.org/html/2606.24169#S4.SS3)\), a Slavic\-pivot warm start, and a hybrid\-encoder ablation that localizes the transferable signal to the upper half of the encoder stack \(both in Sec\.[IV\-D](https://arxiv.org/html/2606.24169#S4.SS4)\), together with a two\-seed reproducibility re\-run on all HR/PT cells \(Sec\.[III\-C](https://arxiv.org/html/2606.24169#S3.SS3),[IV\-A](https://arxiv.org/html/2606.24169#S4.SS1)\), extend this picture without changing the headline conclusion\.
## IIRelated Work
Transfer learning for streaming ASR\.Joshi et al\.\[[4](https://arxiv.org/html/2606.24169#bib.bib4)\]studied transfer for a recurrent neural network transducer \(RNN\-T\)\[[8](https://arxiv.org/html/2606.24169#bib.bib8)\]to Hindi at 50–1,000 h, showing that pretraining is most useful at low data and accelerates convergence\. Pandey et al\.\[[5](https://arxiv.org/html/2606.24169#bib.bib5)\]extended this line of work to Emformer on\-device models in French and Italian\. Neither study varies streaming latency at evaluation, and both compare transfer\-initialized models against non\-pretrained target\-language baselines rather than two pretrained initializations\.
Multilingual ASR at scale\.Multilingual end\-to\-end models\[[9](https://arxiv.org/html/2606.24169#bib.bib9),[10](https://arxiv.org/html/2606.24169#bib.bib10)\], large\-scale multilingual systems\[[7](https://arxiv.org/html/2606.24169#bib.bib7)\], and self\-supervised representations\[[6](https://arxiv.org/html/2606.24169#bib.bib6),[11](https://arxiv.org/html/2606.24169#bib.bib11)\]show that learning across many languages can match or beat per\-language monolingual baselines\. OWLS\[[12](https://arxiv.org/html/2606.24169#bib.bib12)\]derives neural scaling laws across model, data and compute scale, complementing earlier acoustic\-model scaling laws\[[13](https://arxiv.org/html/2606.24169#bib.bib13)\]; we instead hold model size fixed and study how downstream fine\-tuning data and latency interact with encoder initialization\. Hernandez et al\.\[[14](https://arxiv.org/html/2606.24169#bib.bib14)\]showed that, in the low\-data regime, the effective data transferred from text pretraining to code fine\-tuning follows a power law in model size and fine\-tuning data size\.
Cache\-aware streaming\.The stateful Conformer work of Noroozi et al\.\[[3](https://arxiv.org/html/2606.24169#bib.bib3)\]adapts FastConformer to streaming ASR with constrained attention context and multi\-latency training, enabling a single model to support multiple inference latencies\. Related cascaded\-encoder architectures\[[15](https://arxiv.org/html/2606.24169#bib.bib15)\]pursue the complementary goal of unifying streaming and non\-streaming operation within one model\. To our knowledge, ours is the first transfer\-learning study to treat streaming\-latency tiers as an explicit evaluation axis\.
## IIIExperimental Setup
### III\-AModel and initializations
We use a 0\.6 B\-parameter cache\-aware FastConformer RNN\-T with 24 encoder layers,dmodel=1024d\_\{\\text\{model\}\}\{=\}1024, an 8×\\timessubsampling pre\-encoder, and a long short\-term memory \(LSTM\)\-based RNN\-T decoder with joint network\[[3](https://arxiv.org/html/2606.24169#bib.bib3)\], trained with the NeMo toolkit\[[16](https://arxiv.org/html/2606.24169#bib.bib16)\]\. The encoder is trained with multipleatt\_context\_sizeconfigurations so it can be deployed at any of the latency tiers in Table[I](https://arxiv.org/html/2606.24169#S3.T1)\. We compare two initializations: \(i\)MLloads all weights from the multilingual streaming checkpoint described below; and \(ii\)ENloads decoder and joint from that same multilingual baseline, but replaces the encoder layer\-for\-layer with the English\-only Nemotron streaming encoder\[[3](https://arxiv.org/html/2606.24169#bib.bib3),[17](https://arxiv.org/html/2606.24169#bib.bib17)\]\. Both arms share the same multilingual byte\-pair\-encoding \(BPE\) tokenizer\[[18](https://arxiv.org/html/2606.24169#bib.bib18)\], so the comparison isolates the encoder\.
TABLE I:Streaming\-inference operating points used in evaluation\. All three streaming tiers share the same 70\-frame left context \(≈5\.6\\approx 5\.6s of audio\); each chunk spans the right context plus the current frame, and the chunk size equals the emission latency\. The encoder runs at 80 ms/frame after 8×\\timessubsampling\.Offlinedecodes the full utterance in a single pass with the same context caps as the 1120 ms tier\.TierChunk sizeLeft ctxRight ctx160 ms2 frames70 frames1 frame560 ms7 frames70 frames6 frames1120 ms14 frames70 frames13 framesoffline–70 frames13 framesMultilingual baseline\.The ML initialization is a multilingual checkpoint, trained once and then frozen as the starting point for every ML fine\-tuning run\. We start from the same English encoder used as the EN initialization\[[17](https://arxiv.org/html/2606.24169#bib.bib17)\],nvidia/nemotron\-speech\-streaming\-en\-0\.6b, replace its tokenizer and joint network with those ofnvidia/parakeet\-tdt\-0\.6b\-v3\[[19](https://arxiv.org/html/2606.24169#bib.bib19)\]so that the vocabulary covers the target European languages, and adapt the full model to five European languages \(DE, ES, FR, NL and Italian IT\) with standard RNN\-T training\. The corpus is the union of the public training portions of Common Voice \(CV\)\[[20](https://arxiv.org/html/2606.24169#bib.bib20)\], Multilingual LibriSpeech \(MLS\)\[[21](https://arxiv.org/html/2606.24169#bib.bib21)\], VoxPopuli \(VP\)\[[22](https://arxiv.org/html/2606.24169#bib.bib22)\], CML\-TTS\[[23](https://arxiv.org/html/2606.24169#bib.bib23)\]and crawled YODAS\-Granary\[[24](https://arxiv.org/html/2606.24169#bib.bib24)\]subsets,∼9\\sim 9k hours\. Training uses an effective batch size of 448 on 7 NVIDIA H100 GPUs\. The downstream fine\-tuning manifests of Sec\.[III\-B](https://arxiv.org/html/2606.24169#S3.SS2)are constructed independently and verified utterance\-disjoint from this corpus; FLEURS is excluded from both training stages\.
### III\-BTarget languages and corpora
We fine\-tune on eight European languages spanning three families: Germanic \(DE, NL, IS\), Romance \(ES, FR, PT\) and Slavic \(HR, PL\)\. We further partition them by their representation in the multilingual pretraining mix:
- •Seen\(DE, ES, FR, NL\): well covered by the ML pretrain corpus; the comparison against EN init mainly quantifies the head\-room left\.
- •Unseen / under\-represented\(PT, HR, PL, IS\): absent from, or only marginally represented in, the ML pretrain mix\. These are the cases that test whether the multilingual prior survives outside its training distribution\.
Fine\-tuning corpora\.Training data is drawn from the training splits of the same public corpora as the multilingual baseline above \(CV, MLS, VoxPopuli, CML\-TTS and YODAS\-Granary\)\. Two of our unseen languages have limited coverage in these corpora and use additional language\-specific sources: Croatian draws the bulk of its hours from the ParlaSpeech\-HR parliamentary corpus\[[25](https://arxiv.org/html/2606.24169#bib.bib25)\], and Icelandic combines the Althingi Parliamentary Speech Corpus\[[26](https://arxiv.org/html/2606.24169#bib.bib26)\]with the Samrómur\[[27](https://arxiv.org/html/2606.24169#bib.bib27)\]and Málrómur\[[28](https://arxiv.org/html/2606.24169#bib.bib28)\]crowdsourced read\-speech corpora\. For each language we construct nested manifests at 100, 250, 500, 1000 and 2500 h where sufficient data exists; HR, IS and PL stop at 1000 h\.
Evaluation sets\.We evaluate on the official test splits of CV, MLS, VoxPopuli and FLEURS\[[29](https://arxiv.org/html/2606.24169#bib.bib29)\]\. The first three are*in\-domain*\(their training splits feed the fine\-tuning manifests\); FLEURS is*out\-of\-domain*, held out from fine\-tuning at every scale, and is our headline set\. All evaluations report WER over hypotheses and references normalized with theBasicMultilingualTextNormalizerfrom the Open ASR Leaderboard\[[30](https://arxiv.org/html/2606.24169#bib.bib30)\]\. For each per\-cellΔ\\Deltaon FLEURS we run a per\-utterance paired bootstrap\[[31](https://arxiv.org/html/2606.24169#bib.bib31)\]\(B=1000B\{=\}1000\) and markΔ\\Deltawith∗when its 95% CI excludes 0 \(Table[II](https://arxiv.org/html/2606.24169#S4.T2)\)\.
### III\-CTraining recipe and seeds
All fine\-tuning runs use the same recipe: RNN\-T loss with FastEmit regularization\[[32](https://arxiv.org/html/2606.24169#bib.bib32)\]\(λ=0\.005\\lambda\{=\}0\.005\), AdamW\[[33](https://arxiv.org/html/2606.24169#bib.bib33)\], learning rate1×10−41\{\\times\}10^\{\-4\}with cosine decay to10−610^\{\-6\}over the first 25 epochs of a 30\-epoch schedule, effective batch size 48 on NVIDIA H100 GPUs in bf16 mixed precision, weight decay10−310^\{\-3\}, decayed SpecAugment\[[34](https://arxiv.org/html/2606.24169#bib.bib34)\]\(f=2×27f\{=\}2\{\\times\}27,t=10×0\.05t\{=\}10\{\\times\}0\.05\), early stopping with patience 8\. Checkpoint selection and early stopping use WER on the target\-language FLEURS dev split\. This recipe was fixed by a small pilot sweep and then frozen for every \(lang, hours, initialization\) cell\. Training samples encoder context uniformly over\[70,13\]\[70,13\],\[70,6\]\[70,6\],\[70,1\]\[70,1\], and\[70,0\]\[70,0\]\.
Every \(init, hours\) cell of HR and PT was trained both with the default training seed \(42\) and with seed 45; additionally, the 100 h cell of each of the other six languages was re\-trained with seed 45 for both initializations\.
## IVMain Results
### IV\-AML’s advantage closes with hours
TABLE II:FLEURS WER \(%\) at the 160 ms streaming\-latency tier\.Δ=WEREN−WERML\\Delta=\\mathrm\{WER\}\_\{\\mathrm\{EN\}\}\-\\mathrm\{WER\}\_\{\\mathrm\{ML\}\}\(positive: ML init better\)\.∗marks cells where the paired per\-utterance bootstrap 95% CI forΔ\\Deltaexcludes 0 \(B=1000B\{=\}1000\)\. Best per \(lang, hours\) inbold\. “–”: data not available\. The bottommeanrow reports the unweighted across\-language macro mean \(K=8K\{=\}8for 100–1000 h,K=5K\{=\}5for 2500 h\)\.Δ\\Deltais computed at full precision, so it can differ from the difference of the rounded WERs by0\.010\.01\.100 h250 h500 h1000 h2500 hLangMLEN𝚫\\boldsymbol\{\\Delta\}MLEN𝚫\\boldsymbol\{\\Delta\}MLEN𝚫\\boldsymbol\{\\Delta\}MLEN𝚫\\boldsymbol\{\\Delta\}MLEN𝚫\\boldsymbol\{\\Delta\}DE16\.5721\.73\+5\.16∗14\.9316\.72\+1\.78∗13\.0413\.98\+0\.94∗13\.3313\.89\+0\.56∗11\.3711\.07\-0\.30ES11\.1216\.43\+5\.31∗10\.0512\.08\+2\.03∗8\.709\.90\+1\.20∗8\.979\.76\+0\.79∗7\.387\.80\+0\.43∗FR19\.5125\.38\+5\.87∗16\.8419\.87\+3\.02∗15\.5516\.81\+1\.26∗15\.4916\.47\+0\.97∗13\.7513\.81\+0\.06HR36\.3638\.29\+1\.92∗28\.9430\.08\+1\.14∗24\.9125\.50\+0\.5925\.0325\.19\+0\.16–––IS26\.2327\.88\+1\.6522\.8523\.04\+0\.1920\.6220\.33\-0\.2919\.2619\.07\-0\.19–––NL23\.0030\.80\+7\.80∗20\.9423\.81\+2\.87∗18\.8320\.08\+1\.25∗17\.8519\.23\+1\.38∗15\.0415\.95\+0\.91∗PL35\.9237\.42\+1\.50∗27\.2827\.78\+0\.5021\.3922\.33\+0\.93∗21\.7322\.48\+0\.75∗–––PT15\.2719\.74\+4\.48∗12\.4514\.52\+2\.06∗11\.7812\.18\+0\.4012\.0512\.13\+0\.0810\.2410\.17\-0\.07mean23\.0027\.21\+4\.21\+4\.2119\.2920\.99\+1\.70\+1\.7016\.8517\.64\+0\.79\+0\.7916\.7117\.28\+0\.56\+0\.5611\.5611\.76\+0\.20\+0\.20Table[II](https://arxiv.org/html/2606.24169#S4.T2)reports FLEURS WER at the most demanding latency tier \(160 ms\) for both initializations across all available \(lang, hours\) cells, together with paired\-bootstrap significance markers\. The meanΔ\\Deltadrops monotonically from\+4\.21\+4\.21pp at 100 h \(n=8n\{=\}8\) to\+0\.79\+0\.79pp at 500 h,\+0\.56\+0\.56pp at 1000 h and\+0\.20\+0\.20pp at 2500 h \(n=5n\{=\}5\)\. The decay is monotone within each language group: seen languages average\+6\.03→\+1\.16→\+0\.93\+6\.03\\to\+1\.16\\to\+0\.93pp across 100/500/1000 h, while the four unseen languages average\+2\.39→\+0\.41→\+0\.20\+2\.39\\to\+0\.41\\to\+0\.20pp over the same scales — a smaller absolute advantage that disappears into the noise by 1000 h\. At 2500 h no per\-language\|Δ\|\|\\Delta\|exceeds 1 pp and several are not statistically distinguishable from zero under the paired bootstrap\.
Transfer\-gap decay\.A natural practitioner question is: givenhhhours of target\-language data, how much initialization advantage remains from a multilingual checkpoint? WritingΔ¯\(h\)\\bar\{\\Delta\}\(h\)for the across\-language mean EN–ML WER gap \(pp\) athhhours andh0=100h\_\{0\}\{=\}100h for the lowest scale, we summarize the decay with a power\-law fit
Δ¯\(h\)=Δ¯\(h0\)\(h/h0\)−βTG,\\bar\{\\Delta\}\(h\)=\\bar\{\\Delta\}\(h\_\{0\}\)\\,\(h/h\_\{0\}\)^\{\-\\beta\_\{\\mathrm\{TG\}\}\},\(1\)whose exponentβTG\\beta\_\{\\mathrm\{TG\}\}we call the*transfer\-gap exponent*: it sets how fast the EN–ML advantage decays with target\-language data\. Fitting on FLEURS at 160 ms gives
Δ¯\(h\)≈4\(h/100\)−0\.92,R2=0\.99\\bar\{\\Delta\}\(h\)\\approx 4\\,\(h/100\)^\{\-0\.92\},\\quad R^\{2\}=0\.99\(2\)\(Fig\.[1](https://arxiv.org/html/2606.24169#S4.F1)\)\. Power\-law behavior is familiar from the transfer\-learning scaling\-laws literature\[[14](https://arxiv.org/html/2606.24169#bib.bib14)\], although Hernandez et al\. model effective transferred data rather than the WER gap between two pretrained initializations\. Withβ^TG≈0\.92\\hat\{\\beta\}\_\{\\mathrm\{TG\}\}\{\\approx\}0\.92, doubling the amount of target\-language data multiplies the remaining ML\-init advantage by2−0\.92≈0\.532^\{\-0\.92\}\{\\approx\}0\.53, i\.e\., it roughly halves the gap\.
A language\-resampling bootstrap \(B=10000B\{=\}10000\) at 160 ms gives a 95% CI of\[0\.59,1\.52\]\[0\.59,\\,1\.52\]forβ^TG\\hat\{\\beta\}\_\{\\mathrm\{TG\}\}, with fewer than 1% of resamples yieldingβ^TG<0\.5\\hat\{\\beta\}\_\{\\mathrm\{TG\}\}\{<\}0\.5\. Refitting the same form on the three in\-domain test sets givesβ^TG∈\{0\.79\(VP\),0\.97\(MLS\),1\.09\(CV\)\}\\hat\{\\beta\}\_\{\\mathrm\{TG\}\}\{\\in\}\\\{0\.79~\(\\text\{VP\}\),\\,0\.97~\(\\text\{MLS\}\),\\,1\.09~\(\\text\{CV\}\)\\\}\(R2≥0\.93R^\{2\}\{\\geq\}0\.93\), and applying the FLEURS@160 ms fit unchanged to the other latency tiers predicts each per\-tier mean gap to within∼0\.5\\sim 0\.5pp on average\. A single fixed exponent therefore captures the across\-language mean tightly across all four test sets and all four latency settings with no retuning; only the per\-language fits scatter more widely\.
Figure 1:Per\-languageEN−ML\\mathrm\{EN\}\-\\mathrm\{ML\}FLEURS gapΔℓ\(h\)\\Delta\_\{\\ell\}\(h\)\(pp\) at 160 ms versus training hourshh, split into*seen*\(DE, ES, FR, NL; left\) and*unseen*\(HR, IS, PL, PT; right\) languages\. Black curve: the power\-law fitΔ¯\(h\)≈4\(h/100\)−0\.92\\bar\{\\Delta\}\(h\)\{\\approx\}4\\,\(h/100\)^\{\-0\.92\}\([2](https://arxiv.org/html/2606.24169#S4.E2)\) to the mean over all eight languages \(R2=0\.99R^\{2\}\{=\}0\.99\), drawn identically in both panels for reference\.Seed stability\.Re\-running each cell with a second seed, the mean absolute seed\-to\-seed FLEURS WER difference at 560 ms is0\.210\.21pp across the HR/PT cells and0\.440\.44pp across the 100 h cells of all eight languages — well below the 100 h EN–ML gaps at the same tier \(up to\+7\.43\+7\.43pp for NL\)\. The EN–ML gap keeps the same sign at both seeds for all eight languages\.
Convergence speed\.The multilingual prior also accelerates fine\-tuning: at 1000 h, ML reaches the EN arm’s final running\-best validation WER \(ENfinal\\mathrm\{EN\}\_\{\\mathrm\{final\}\}\) 4–11 epochs sooner in 6/8 languages, and reaches a looser1\.1×ENfinal1\.1\{\\times\}\\mathrm\{EN\}\_\{\\mathrm\{final\}\}target \(achievable by both arms in every language\) a mean of3\.53\.5epochs sooner \(Fig\.[2](https://arxiv.org/html/2606.24169#S4.F2)\)\.
Figure 2:1000 h fine\-tuning convergence speed: epochs each arm needs to reach1\.1×ENfinal1\.1\{\\times\}\\mathrm\{EN\}\_\{\\mathrm\{final\}\}, a WER target reachable by both arms in every language\.Single\-language extension to 5000 h\.Since the EN–ML gap is already near zero on average at 2500 h, we extended only a single ES ML\-init checkpoint to 5000 h\. At 560 ms streaming it gains∼0\.7\\sim 0\.7–1\.41\.4pp on every ES test set over the 2500 h cell and stays competitive with the contemporaneous NVIDIA Nemotron\-3\.5\-ASR\-Streaming\-0\.6B checkpoint\[[35](https://arxiv.org/html/2606.24169#bib.bib35)\]\(same size and streaming family\[[3](https://arxiv.org/html/2606.24169#bib.bib3)\], but multilingual\) to within11pp on three of the four splits and1\.021\.02pp on VoxPopuli — so absolute WER keeps improving past the point where the initialization gap has closed\.
### IV\-BThe EN–ML gap is latency\-invariant
TABLE III:Unweighted across\-language \(macro\) meanEN−ML\\mathrm\{EN\}\-\\mathrm\{ML\}gap \(pp\) on FLEURS at each streaming\-latency tier \(and offline batch decoding\) versus target\-language data scale\. Positive: ML init better\. Each cell is the arithmetic mean across the available languages \(K=8K\{=\}8for 100–1000 h,K=5K\{=\}5for 2500 h\) of the per\-languageΔ\\Delta\(EN−\-ML\) at that tier and data scale; the 160 ms row aggregates theΔ\\Deltacolumn of Table[II](https://arxiv.org/html/2606.24169#S4.T2)\.Tier100 h250 h500 h1000 h2500 h160 ms\+4\.21\+4\.21\+1\.70\+1\.70\+0\.79\+0\.79\+0\.56\+0\.56\+0\.20\+0\.20560 ms\+4\.48\+4\.48\+1\.77\+1\.77\+0\.85\+0\.85\+0\.40\+0\.40\+0\.05\+0\.051120 ms\+4\.56\+4\.56\+1\.43\+1\.43\+0\.96\+0\.96\+0\.27\+0\.27−0\.58\-0\.58offline\+4\.19\+4\.19\+1\.61\+1\.61\+0\.81\+0\.81\+0\.45\+0\.45\+0\.09\+0\.09Multi\-latency cache\-aware training makes cross\-tier robustness of*absolute*WER plausible, but does not by itself imply that the*EN–ML transfer gap*should be latency\-stable\. Our three streaming tiers share the same 70\-frame left context and differ only in chunk size \(Table[I](https://arxiv.org/html/2606.24169#S3.T1)\), but initialization and chunking can still interact: at 160 ms each chunk holds only two encoder frames, so the model sees less right\-context evidence per emission step\. A priori, a reasonable hypothesis is therefore that the multilingual prior might matter*more*at tight latencies\.
We quantify cross\-tier stability with the*Latency Sensitivity of Transfer*\(LST\)\. For languageℓ\\ellat data scalehh,
Δτ\(ℓ,h\)\\displaystyle\\Delta\_\{\\tau\}\(\\ell,h\)=WEREN,τ\(ℓ,h\)−WERML,τ\(ℓ,h\),\\displaystyle=\\mathrm\{WER\}\_\{\\mathrm\{EN\},\\tau\}\(\\ell,h\)\-\\mathrm\{WER\}\_\{\\mathrm\{ML\},\\tau\}\(\\ell,h\),\(3\)LST\(ℓ,h\)\\displaystyle\\mathrm\{LST\}\(\\ell,h\)=maxτΔτ\(ℓ,h\)−minτΔτ\(ℓ,h\),\\displaystyle=\\max\_\{\\tau\}\\Delta\_\{\\tau\}\(\\ell,h\)\-\\min\_\{\\tau\}\\Delta\_\{\\tau\}\(\\ell,h\),\(4\)whereτ∈\{160,560,1120ms\}\\tau\\in\\\{160,\\,560,\\,1120\\,\\mathrm\{ms\}\\\}\. LST is thus the spread of the EN–ML gap across the three streaming tiers; a small LST means the transfer advantage is nearly the same at every latency\. LST and the across\-language macro\-mean gapΔ¯\\bar\{\\Delta\}\(Table[III](https://arxiv.org/html/2606.24169#S4.T3)\) are complementary: the macro\-mean averages over languages*before*comparing tiers, so opposite\-sign per\-language shifts can cancel, whereas LST takes each language’s own across\-tier spread*before*averaging and so cannot be hidden by such cancellation\. Offline decoding is excluded from LST but reported alongside the streaming tiers as a latency\-unconstrained reference\.
Figure 3:Per\-language LST \([4](https://arxiv.org/html/2606.24169#S4.E4)\) on FLEURS versus target\-language training hours for the four*unseen*languages \(HR, IS, PL, PT\); computed over the three streaming tiers \(160/560/1120 ms\)\. IS shows the largest unseen\-language LST values at 100 h and 250 h; otherwise most unseen\-language cells remain near or below 1 pp \(HR, IS and PL have no 2500 h cell\)\.Averaged over the eight languages, LST \([4](https://arxiv.org/html/2606.24169#S4.E4)\) is1\.001\.00pp at 100 h,1\.131\.13at 250 h,0\.610\.61at 500 h and0\.660\.66at 1000 h: at every data scale the EN–ML gap moves by on the order of a point as latency varies across the three streaming tiers\. A paired per\-utterance bootstrap \(B=1000B\{=\}1000\) confirms this is not a small\-sample artifact, with the 95% CI upper bound on the across\-language mean LST staying≤1\.6\\leq 1\.6pp at every data scale\. Per\-language exceptions exist \(Fig\.[3](https://arxiv.org/html/2606.24169#S4.F3)\), most visibly IS at 100 h, so this stability is an aggregate pattern rather than a property of every \(lang, hours\) cell\. The hypothesis that the smaller per\-chunk look\-ahead at 160 ms should amplify the value of the initialization is therefore*not*supported\.
Table[III](https://arxiv.org/html/2606.24169#S4.T3), which collapses each tier to its across\-language mean gap, tells the same story at the aggregate level and lets us fold in offline decoding\. Within a fixed data scale this macro\-mean gapΔ¯\\bar\{\\Delta\}, now read per tier, is approximately constant across the three streaming tiers — its range is at most0\.350\.35pp wherever the gap is non\-trivial \(100–1000 h\) — and at 2500 h all four values \(the three streaming tiers and offline\) lie within11pp of zero, as expected once the gap has effectively closed\. Offline tracks the streaming\-tier mean to within0\.230\.23pp at every data scale, and the absolute streaming penalty itself \(streaming WER minus offline WER\) is small and roughly equal for ML and EN init:∼1\.1\\sim 1\.1–1\.21\.2pp at 160 ms across 100–1000 h for both arms\.
### IV\-CPer\-language pattern: the unseen four
Per\-languageΔ\\Deltavalues group cleanly by representation in the multilingual pretraining mix\. The largest 100 h gains accrue to the seen languages: NL \(\+7\.80\+7\.80pp\), FR \(\+5\.87\+5\.87pp\), ES \(\+5\.31\+5\.31pp\), DE \(\+5\.16\+5\.16pp\); seen\-group mean\+6\.03\+6\.03pp\. The four unseen languages show much smaller gains: PT \(\+4\.48\+4\.48pp\), HR \(\+1\.92\+1\.92pp\), IS \(\+1\.65\+1\.65pp\), PL \(\+1\.50\+1\.50pp\); group mean\+2\.39\+2\.39pp\. Both groups’ gaps shrink sharply by 1000 h: the unseen group as a whole is statistically indistinguishable from EN init \(95% bootstrap CI on the group\-meanΔ\\Deltacontains zero\), with HR, IS and PT individually non\-significant and only PL retaining a small but significant gap \(\+0\.75\+0\.75pp\)\. IS has by far the smallest FLEURS test set \(46 utterances\), giving it the widest bootstrap interval and the only non\-significant 100 hΔ\\Deltaof the eight languages; we treat its 100 h gap as a low\-confidence signal\. The unseen group still benefits modestly from the multilingual prior despite the target language being out of the pretraining distribution: a hybrid\-encoder ablation \(Sec\.[IV\-D](https://arxiv.org/html/2606.24169#S4.SS4)\) localizes this transferable component to the upper half of the encoder\.
### IV\-DSupporting transfer analyses
Slavic pivot: HR fine\-tuned from PL\.A cheaper alternative to rebuilding a multilingual pretraining mix is to warm\-start from a model already fine\-tuned on a related language\. We test this for HR by pivoting through the PLML\{\}\_\{\\text\{ML\}\}1000 h checkpoint, the only same\-family Slavic pivot available among our targets\. The pivot does not improve over direct ML initialization: on FLEURS it is\+0\.56\+0\.56to\+1\.05\+1\.05pp worse across the four scales, significant at three of them, while on VoxPopuli the differences are smaller and mostly within noise, significant only at 250 h, where the pivot is instead1\.341\.34pp better\. This negative result suggests the broad multilingual encoder is already at least as useful a prior as this PL\-specialized checkpoint, and that same\-family relatedness alone does not determine the best pivot\.
Layer\-wise localization\.A hybrid\-encoder ablation combines layers from the ML and EN checkpoints while keeping the multilingual decoder, joint and pre\-encoder; we fine\-tune on 100 h of the target language with the recipe of Sec\.[III\-C](https://arxiv.org/html/2606.24169#S3.SS3)and evaluate at 560 ms streaming \(single seed throughout\)\. The splice that takes the bottom 12 encoder layers from English \(input\-proximal layers\) but keeps the top 12 multilingual layers \(decoder\-proximal layers\) matches full\-ML accuracy across*all eight*languages: across\-language mean−0\.07\-0\.07pp on FLEURS, with seen\-group mean17\.0817\.08vs17\.0717\.07% and unseen\-group27\.3027\.30vs27\.4527\.45%\. Other splices — keeping only the bottom, middle, or top third of the encoder from ML \(the rest from EN\) — all regress, the middle\-third variant essentially back to the full\-EN baseline\. This matches the layer\-wise probing literature on speech encoders\[[36](https://arxiv.org/html/2606.24169#bib.bib36),[37](https://arxiv.org/html/2606.24169#bib.bib37)\]and gives direct ASR\-side confirmation that the transferable multilingual\-pretraining signal sits in the higher encoder layers, not the acoustic front\-end\. A complementary joint\-re\-initialization ablation at the same 100 h cells \(both encoders intact, the same freshly initialized joint for both arms\) hurts the EN\-init arm substantially more than the ML\-init arm, even though the reset joint is fine\-tuned together with its encoder\. This argues against a simple joint–encoder mismatch explanation for the EN–ML gap and instead suggests that the ML encoder exposes representations that are easier for the RNN\-T decoder/joint stack to use\. Both ablations still require the full multilingual checkpoint as a source, so these are representational rather than deployment findings\.
## VEncoder Quantization
For on\-device deployment we additionally quantize the encoder weights to 4\-bit integers using the k\-quant weight\-only scheme of ONNX Runtime’sMatMulNBitsQuantizer\(block size 32, symmetric\)\[[38](https://arxiv.org/html/2606.24169#bib.bib38)\]; prior work has reported that this same INT4 k\-quant recipe preserves WER within∼1\\sim 1pp on a 0\.6 B\-parameter English Nemotron streaming\-ASR model\[[39](https://arxiv.org/html/2606.24169#bib.bib39)\], and we apply it unchanged here, without per\-language or per\-cell quantization tuning\. Only the encoder’sMatMulweights are quantized; the decoder, joint network, all activations, and the audio front\-end remain FP32\. Since the encoder holds the large majority of model parameters, quantizing it captures most of the realizable footprint reduction\.
The INT4 ONNX models are deployed viaonnxruntime\-genai\[[40](https://arxiv.org/html/2606.24169#bib.bib40)\]at the 560 ms streaming tier\. To make the quantization comparison fair we rebuild the FP32 reference at the same tier from the NeMo checkpoint of every fine\-tuned cell\.
TABLE IV:INT4 weight\-only encoder quantization on FLEURS at 560 ms streaming latency, against an FP32 NeMo baseline at the same latency\.Δ¯Q=WERINT4−WERFP32\\bar\{\\Delta\}\_\{\\mathrm\{Q\}\}=\\mathrm\{WER\}\_\{\\mathrm\{INT4\}\}\-\\mathrm\{WER\}\_\{\\mathrm\{FP32\}\}\(pp, positive = INT4 worse\), averaged across the available languages\. Decoder and joint are kept FP32\.HoursΔ¯Q,ML\\bar\{\\Delta\}\_\{\\mathrm\{Q,ML\}\}\(pp\)nMLn\_\{\\mathrm\{ML\}\}Δ¯Q,EN\\bar\{\\Delta\}\_\{\\mathrm\{Q,EN\}\}\(pp\)nENn\_\{\\mathrm\{EN\}\}100\+0\.88\+0\.888\+0\.70\+0\.708250\+0\.34\+0\.348\+0\.65\+0\.658500\+0\.15\+0\.158\+0\.63\+0\.6381000\+0\.26\+0\.268\+0\.57\+0\.5782500\+0\.17\+0\.175\+0\.41\+0\.415pooled\+0\.49\+0\.49\(pp\), median\+0\.42\+0\.42,n=74n\{=\}74cellsAcross the 74 matched cells \(Table[IV](https://arxiv.org/html/2606.24169#S5.T4)\) the mean change is\+0\.49\+0\.49pp and the median is\+0\.42\+0\.42pp; 60/74 cells lie within±1\.0\\pm 1\.0pp of their FP32 counterpart and INT4 is worse than FP32 in 65/74 cells\. The eight cells where INT4 is nominally better \(the remaining cell is an exact tie\) are all small in magnitude \(within0\.680\.68pp\), comparable to the seed\-to\-seed variation reported in Sec\.[III\-C](https://arxiv.org/html/2606.24169#S3.SS3)\. The per\-cell cost is small but*not zero*: it buys roughly3×3\{\\times\}encoder file\-size reduction at a typical cost of about half a WER point on FLEURS, with no systematic dependence on data scale\. A within\-\(lang, hours\) paired comparison gives a meanΔ¯Q,EN−Δ¯Q,ML=\+0\.23\\bar\{\\Delta\}\_\{\\mathrm\{Q,EN\}\}\-\\bar\{\\Delta\}\_\{\\mathrm\{Q,ML\}\}\{=\}\+0\.23pp across the 37 paired cells: the EN arm is more often costlier to quantize than the ML arm, but the effect is inconsistent across cells, so we treat this as a weak trend rather than a robust effect\.
## VIConclusion
We presented a systematic study of how multilingual versus English\-only encoder initialization interacts with target\-language data scale, streaming latency and target language for a cache\-aware streaming ASR architecture, with particular attention to the four languages \(PT, HR, PL, IS\) outside the multilingual pretraining mix\. Across eight languages the multilingual advantage closes monotonically with data and is approximately latency\-invariant: the across\-language mean gap varies by≤0\.35\\leq 0\.35pp across the three streaming tiers wherever it is non\-trivial, and the latency sensitivity of transfer \(LST\), averaged over languages, remains small, ranging from0\.610\.61to1\.131\.13pp across the 100–1000 h scales, with the largest value at 250 h driven by a small number of language\-specific outliers\. A power law with exponentβ^TG≈0\.92\\hat\{\\beta\}\_\{\\mathrm\{TG\}\}\{\\approx\}0\.92summarizes the gap decay, and INT4 weight\-only encoder quantization at 560 ms shifts FLEURS WER by\+0\.49\+0\.49pp on average across all 74 \(lang, hours, init\) cells, so the FP32 conclusions carry over to the INT4\-deployed model\.
More broadly, these results suggest that multilingual encoder pretraining should be viewed less as a permanent architectural advantage and more as a data\-efficiency mechanism: it changes how quickly a target language reaches a given WER, but its effect can be washed out by sufficient supervised target\-language data\. In this setting, latency and INT4 encoder quantization affect the absolute operating point, but they do not substantially change the transfer\-learning decision\. This separates three choices that are often coupled in streaming\-ASR deployment discussions: use multilingual initialization to reduce the data needed for adaptation, choose the streaming latency from product constraints, and apply weight\-only quantization with a small measured WER budget\.
Scope and reproducibility\.Our study uses eight European target languages and a single 0\.6 B cache\-aware FastConformer RNN\-T family with a fixed fine\-tuning recipe and multi\-latency training\. We expect the qualitative conclusion—that multilingual initialization mainly improves data efficiency rather than changing the latency decision—to generalize beyond this model, while the exact EN–ML cross\-over points and fitted exponent are empirical properties of this sweep\. The multilingual pretraining checkpoint is released publicly; training and evaluation code is included as supplementary material, and fine\-tuned checkpoints can be shared upon request\.
## Acknowledgment
The authors used ChatGPT and Claude to assist with the scripts for model training, evaluation and figure rendering \(Sections[III](https://arxiv.org/html/2606.24169#S3)–[V](https://arxiv.org/html/2606.24169#S5)\) and with grammar editing of the manuscript\. AI was not used to generate the experimental data, numerical results, scientific claims, or conclusions; all code, results and figures were reviewed, validated and finalized by the authors, who take full responsibility for the content of this article\.
## References
- \[1\]A\. Gulati, J\. Qin, C\.\-C\. Chiu, et al\., “Conformer: Convolution\-augmented transformer for speech recognition,” in*Proc\. Interspeech*, 2020, pp\. 5036–5040\.
- \[2\]D\. Rekesh, N\. R\. Koluguri, S\. Kriman, et al\., “Fast Conformer with linearly scalable attention for efficient speech recognition,” in*Proc\. ASRU*, 2023, pp\. 1–8\.
- \[3\]V\. Noroozi, S\. Majumdar, A\. Kumar, et al\., “Stateful conformer with cache\-based inference for streaming automatic speech recognition,” in*Proc\. ICASSP*, 2024, pp\. 12041–12045\.
- \[4\]V\. Joshi, R\. Zhao, R\. R\. Mehta, K\. Kumar, and J\. Li, “Transfer learning approaches for streaming end\-to\-end speech recognition system,” in*Proc\. Interspeech*, 2020, pp\. 2152–2156\.
- \[5\]L\. Pandey, K\. Li, J\. Guo, D\. Paul, A\. Guo, J\. Mahadeokar, and X\. Zhang, “Towards scalable efficient on\-device ASR with transfer learning,” arXiv:2407\.16664, 2024\.
- \[6\]A\. Conneau, A\. Baevski, R\. Collobert, A\. Mohamed, and M\. Auli, “Unsupervised cross\-lingual representation learning for speech recognition,” in*Proc\. Interspeech*, 2021, pp\. 2426–2430\.
- \[7\]Y\. Zhang, W\. Han, J\. Qin, et al\., “Google USM: Scaling automatic speech recognition beyond 100 languages,” arXiv:2303\.01037, 2023\.
- \[8\]A\. Graves, “Sequence transduction with recurrent neural networks,” arXiv:1211\.3711, 2012\.
- \[9\]A\. Kannan, A\. Datta, T\. Sainath, et al\., “Large\-scale multilingual speech recognition with a streaming end\-to\-end model,” in*Proc\. Interspeech*, 2019, pp\. 2130–2134\.
- \[10\]B\. Li, R\. Pang, T\. N\. Sainath, et al\., “Scaling end\-to\-end models for large\-scale multilingual ASR,” in*Proc\. ASRU*, 2021, pp\. 1011–1018\.
- \[11\]V\. Pratap, A\. Tjandra, B\. Shi, et al\., “Scaling speech technology to 1,000\+ languages,”*J\. Mach\. Learn\. Res\.*, vol\. 25, no\. 97, pp\. 1–52, 2024\.
- \[12\]W\. Chen, J\. Tian, Y\. Peng, B\. Yan, C\.\-H\. H\. Yang, and S\. Watanabe, “OWLS: Scaling laws for multilingual speech recognition and translation models,” in*Proc\. ICML*, 2025, pp\. 9121–9145\.
- \[13\]J\. Droppo and O\. Elibol, “Scaling laws for acoustic models,” in*Proc\. Interspeech*, 2021, pp\. 2576–2580\.
- \[14\]D\. Hernandez, J\. Kaplan, T\. Henighan, and S\. McCandlish, “Scaling laws for transfer,” arXiv:2102\.01293, 2021\.
- \[15\]A\. Narayanan, T\. N\. Sainath, R\. Pang, et al\., “Cascaded encoders for unifying streaming and non\-streaming ASR,” in*Proc\. ICASSP*, 2021, pp\. 5629–5633\.
- \[16\]O\. Kuchaiev, J\. Li, H\. Nguyen, et al\., “NeMo: A toolkit for building AI applications using neural modules,” arXiv:1909\.09577, 2019\.
- \[17\]NVIDIA, “Nemotron\-Speech\-Streaming\-En\-0\.6B,” Hugging Face model card\. \[Online\]\. Available:https://huggingface\.co/nvidia/nemotron\-speech\-streaming\-en\-0\.6b, accessed Jun\. 11, 2026\.
- \[18\]R\. Sennrich, B\. Haddow, and A\. Birch, “Neural machine translation of rare words with subword units,” in*Proc\. ACL*, 2016, pp\. 1715–1725\.
- \[19\]NVIDIA, “Parakeet\-TDT\-0\.6B\-v3,” Hugging Face model card\. \[Online\]\. Available:https://huggingface\.co/nvidia/parakeet\-tdt\-0\.6b\-v3, accessed Jun\. 11, 2026\.
- \[20\]R\. Ardila, M\. Branson, K\. Davis, et al\., “Common Voice: A massively\-multilingual speech corpus,” in*Proc\. LREC*, 2020, pp\. 4218–4222\.
- \[21\]V\. Pratap, Q\. Xu, A\. Sriram, G\. Synnaeve, and R\. Collobert, “MLS: A large\-scale multilingual dataset for speech research,” in*Proc\. Interspeech*, 2020, pp\. 2757–2761\.
- \[22\]C\. Wang, A\. Riviere, A\. Lee, et al\., “VoxPopuli: A large\-scale multilingual speech corpus for representation learning, semi\-supervised learning and interpretation,” in*Proc\. ACL\-IJCNLP*, 2021, pp\. 993–1003\.
- \[23\]F\. S\. Oliveira, E\. Casanova, A\. Cândido Júnior, A\. S\. Soares, and A\. R\. Galvão Filho, “CML\-TTS: A multilingual dataset for speech synthesis in low\-resource languages,” in*Text, Speech, and Dialogue*, 2023, pp\. 188–199\.
- \[24\]N\. Rao Koluguri, M\. Sekoyan, G\. Zelenfroynd, et al\., “Granary: Speech recognition and translation dataset in 25 European languages,” in*Proc\. Interspeech*, 2025, pp\. 3923–3927\.
- \[25\]N\. Ljubešić, D\. Koržinek, P\. Rupnik, and I\.\-P\. Jazbec, “ParlaSpeech\-HR: A freely available ASR dataset for Croatian bootstrapped from the ParlaMint corpus,” in*Proc\. Workshop ParlaCLARIN III at LREC*, 2022, pp\. 111–116\.
- \[26\]I\. R\. Helgadóttir, R\. Kjaran, A\. B\. Nikulasdottir, and J\. Gudnason, “Building an ASR corpus using Althingi’s parliamentary speeches,” in*Proc\. Interspeech*, 2017, pp\. 2163–2167\.
- \[27\]D\. E\. Mollberg, Ó\. H\. Jónsson, S\. Thorsteinsdóttir, S\. Steingrímsson, E\. H\. Magnúsdóttir, and J\. Gudnason, “Samrómur: Crowd\-sourcing data collection for Icelandic speech recognition,” in*Proc\. LREC*, 2020, pp\. 3463–3467\.
- \[28\]S\. Steingrímsson, J\. Gudnason, S\. Helgadóttir, and E\. Rögnvaldsson, “Málrómur: A manually verified corpus of recorded Icelandic speech,” in*Proc\. NODALIDA*, 2017, pp\. 237–240\.
- \[29\]A\. Conneau, M\. Ma, S\. Khanuja, et al\., “FLEURS: Few\-shot learning evaluation of universal representations of speech,” in*Proc\. SLT*, 2023, pp\. 798–805\.
- \[30\]Hugging Face, “Open ASR Leaderboard\.” \[Online\]\. Available:https://github\.com/huggingface/open\_asr\_leaderboard, accessed Jun\. 12, 2026\.
- \[31\]M\. Bisani and H\. Ney, “Bootstrap estimates for confidence intervals in ASR performance evaluation,” in*Proc\. ICASSP*, 2004, vol\. 1, pp\. I\-409–I\-412\.
- \[32\]J\. Yu, C\.\-C\. Chiu, B\. Li, et al\., “FastEmit: Low\-latency streaming ASR with sequence\-level emission regularization,” in*Proc\. ICASSP*, 2021, pp\. 6004–6008\.
- \[33\]I\. Loshchilov and F\. Hutter, “Decoupled weight decay regularization,” in*Proc\. ICLR*, 2019\.
- \[34\]D\. S\. Park, W\. Chan, Y\. Zhang, C\.\-C\. Chiu, B\. Zoph, E\. D\. Cubuk, and Q\. V\. Le, “SpecAugment: A simple data augmentation method for automatic speech recognition,” in*Proc\. Interspeech*, 2019, pp\. 2613–2617\.
- \[35\]NVIDIA, “Nemotron\-3\.5\-ASR\-Streaming\-0\.6B,” Hugging Face model card\. \[Online\]\. Available:https://huggingface\.co/nvidia/nemotron\-3\.5\-asr\-streaming\-0\.6b, accessed Jun\. 12, 2026\.
- \[36\]A\. Pasad, J\.\-C\. Chou, and K\. Livescu, “Layer\-wise analysis of a self\-supervised speech representation model,” in*Proc\. ASRU*, 2021, pp\. 914–921\.
- \[37\]S\.\-w\. Yang, P\.\-H\. Chi, Y\.\-S\. Chuang, et al\., “SUPERB: Speech processing universal performance benchmark,” in*Proc\. Interspeech*, 2021, pp\. 1194–1198\.
- \[38\]Microsoft, “ONNX Runtime: cross\-platform accelerated machine learning\.” \[Online\]\. Available:https://onnxruntime\.ai, accessed Jun\. 12, 2026\.
- \[39\]N\. Banfic, D\. Fan, et al\., “Pushing the limits of on\-device streaming ASR: A compact, high\-accuracy English model for low\-latency inference,” arXiv:2604\.14493, 2026\.
- \[40\]Microsoft, “onnxruntime\-genai: Generative AI extensions for ONNX Runtime\.” \[Online\]\. Available:https://github\.com/microsoft/onnxruntime\-genai, accessed Jun\. 12, 2026\.Similar Articles
Scaling laws for neural language models
Foundational empirical study demonstrating power-law scaling relationships between language model performance and model size, dataset size, and compute budget, with implications for optimal training allocation and sample efficiency.
Agentic Data Sampling Framework for Multilingual ASR via Linguistic-Aware Error-Driven Optimization
提出一个基于LLM代理的多语言ASR数据采样框架,通过语言学感知的错误驱动优化在12种Common Voice语言上以更少数据和算力超越全数据训练效果,并显著加速收敛。
Structural priors for data-efficient language learning
This paper investigates structural transfer from non-language data to improve data efficiency in language modeling, finding that while it reduces next-token prediction loss, it does not reliably enhance downstream linguistic performance.
Towards Truly Multilingual ASR: Generalizing Code-Switching ASR to Unseen Language Pairs
This paper investigates whether code-switching ASR capabilities learned from limited seen language pairs can generalize to unseen pairs using model merging and domain generalization methods, finding only modest transfer.
Latency matters more than model selection when building AI tutoring systems
A practitioner argues that speech start latency—not model selection—is the critical factor in AI tutoring systems, recommending targets under 1 second for speech start and highlighting streaming TTS as the highest-leverage optimization. The post outlines a full pipeline from ASR through TTS and avatar sync, identifying where latency compounds most.