Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition

# Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition
Source: [https://arxiv.org/html/2607.05612](https://arxiv.org/html/2607.05612)
Mohammad Zeineldeen12, Albert Zeyer12, Haoran Zhang2, Robin Schmitt12, Ralf Schlüter12, Hermann Ney12

###### Abstract

Language model \(LM\) perplexity \(PPL\) has historically been used as a proxy for automatic speech recognition \(ASR\) word error rate \(WER\), with prior work reporting an approximately linear relation in log\-log space\. Modern end\-to\-end ASR systems challenge this assumption because they already contain internal language modeling capacity, are often evaluated without external language models, and can now be combined with neural LMs and large language models \(LLMs\) through different recognition strategies\. This paper revisits the relation between PPL and WER for modern ASR systems\. We study whether external LMs still improve current end\-to\-end ASR systems, whether the PPL\-WER relation remains linear in log\-log space, how encoder context length affects this relation, and how LLM perplexities fit into the trend observed for standard neural LMs\. We further investigate internal language modeling \(ILM\) in attention\-based encoder\-decoder systems and show that ILM subtraction changes the observed PPL\-WER relation, indicating that the decoder’s internal LM must be considered when interpreting the effect of external LM quality\.

## IIntroduction & Related Work

Language models \(LMs\) have long been central to automatic speech recognition \(ASR\)\. Perplexity \(PPL\) is commonly used as an intrinsic LM metric, tracing back to its introduction as a measure of speech task difficulty\[[12](https://arxiv.org/html/2607.05612#bib.bib1)\]\. However, ASR performance is measured by word error rate \(WER\), which also depends on the acoustic model, decoding strategy, pruning, and scale tuning\. Thus, lower LM perplexity does not automatically translate into lower WER\. Earlier studies, mostly using n\-gram LMs\[[15](https://arxiv.org/html/2607.05612#bib.bib21)\], reported experimental evidence for a correlation between perplexity and WER across several ASR tasks\[[3](https://arxiv.org/html/2607.05612#bib.bib3),[14](https://arxiv.org/html/2607.05612#bib.bib2),[8](https://arxiv.org/html/2607.05612#bib.bib20)\]\. The work in\[[14](https://arxiv.org/html/2607.05612#bib.bib2)\]systematically studied this question and found that WER and PPL are approximately linearly related in log\-log space\. The same log\-log linear relation was later observed in\[[11](https://arxiv.org/html/2607.05612#bib.bib14)\]for hybrid ASR models on Quaero and LibriSpeech dev\-clean using both n\-gram and long short\-term memory \(LSTM\) LMs\[[24](https://arxiv.org/html/2607.05612#bib.bib22)\]\. This observation has been influential because it suggests that relative PPL improvements can be translated into expected WER improvements through a power\-law relation\.

Large\-scale ASR studies also support stronger external LMs\. The study in\[[2](https://arxiv.org/html/2607.05612#bib.bib4)\]showed that increasing LM scale and using large text corpora yields consistent WER reductions\. Later work extended LM integration to neural LMs and end\-to\-end ASR\. Shallow fusion combines an external LM with the ASR model during beam search\[[13](https://arxiv.org/html/2607.05612#bib.bib8)\], while cold fusion incorporates a pretrained LM during training\[[23](https://arxiv.org/html/2607.05612#bib.bib9)\]\. Several integration strategies were compared in\[[25](https://arxiv.org/html/2607.05612#bib.bib10)\]for attention\-based encoder\-decoder \(AED\) ASR and found shallow fusion to be a strong and simple first\-pass decoding method across multiple conditions\. Neural Transformer LMs\[[26](https://arxiv.org/html/2607.05612#bib.bib27)\]have also been applied successfully to hybrid and end\-to\-end ASR, including lattice rescoring and shallow fusion\[[10](https://arxiv.org/html/2607.05612#bib.bib11)\]\.

Modern end\-to\-end ASR systems make the PPL\-WER relation less direct\. In Connectionist temporal classification \(CTC\)\[[5](https://arxiv.org/html/2607.05612#bib.bib5)\], recurrent neural network transducer \(RNN\-T\)\[[6](https://arxiv.org/html/2607.05612#bib.bib6)\], and AED\[[1](https://arxiv.org/html/2607.05612#bib.bib7)\]systems, acoustic and language modeling components are no longer cleanly separated as in classical hybrid ASR\. AED and transducer decoders, in particular, learn an internal LM from paired speech\-text training data\. Moreover,\[[28](https://arxiv.org/html/2607.05612#bib.bib18)\]shows that the CTC encoder also learns an internal LM, which can affect the benefit of external LMs specifically on cross\-domain tasks\. This internal LM can interact with an external LM during recognition and can lead to domain\-prior mismatch\. Internal LM estimation and subtraction have therefore been proposed to improve external LM integration\[[29](https://arxiv.org/html/2607.05612#bib.bib12),[30](https://arxiv.org/html/2607.05612#bib.bib19),[28](https://arxiv.org/html/2607.05612#bib.bib18)\]\. These methods suggest that the observed benefit of an external LM may depend not only on the external LM’s PPL, but also on the strength of the internal language model of the ASR system\.

Recent ASR work often reports greedy recognition or beam search without an external LM, since LM gains can be small for strong end\-to\-end models and no\-LM recognition is simpler and faster\. This makes it unclear whether external neural LMs still provide meaningful gains and whether PPL predicts them\. The question becomes more important with large language models \(LLMs\), which can be used for rescoring, prompting, correction, or one\-pass search integration, but raise issues such as inference cost and vocabulary mismatch\. Recent work\[[9](https://arxiv.org/html/2607.05612#bib.bib13)\]addresses some of these issues by applying LLM scores during one\-pass recognition with delayed scoring and re\-tokenization, and shows improvements over CTC and attention\-based baselines\. However, such work typically does not analyze how LLM PPL fits into the classical PPL\-WER relation\.

This motivates a systematic re\-examination of the PPL\-WER relation for modern ASR\. In this work, we study the following questions:

- •Do external neural LMs still improve modern end\-to\-end ASR over a no\-LM baseline?
- •Is the relation between LM perplexity and ASR WER still approximately linear in log\-log space?
- •For AED systems, how does internal LM subtraction affect the strength and shape of the perplexity\-WER relation?
- •How does encoder context length affect both the absolute gain from external LMs and the correlation between perplexity and WER?
- •How do LLM perplexities fit into the PPL\-WER relation observed for standard neural LMs, and do LLMs follow the same trend when used as external LMs?

We address these questions across different ASR architectures, encoder context settings, recognition modes, and LM families\.

## IIProblem Formulation

### II\-APerplexity and WER

Leta1Sa\_\{1\}^\{S\}denote the label output sequence, excluding special beginning\-of\-sentence \(BOS\) and end\-of\-sentence \(EOS\) symbols\. For autoregressive LMs, we augment the sequence witha0=⟨bos⟩a\_\{0\}=\\langle\\mathrm\{bos\}\\rangleandaS\+1=⟨eos⟩a\_\{S\+1\}=\\langle\\mathrm\{eos\}\\rangle\. Perplexity is computed as

PPL=exp⁡\(−1S\+1​∑s=1S\+1log⁡p​\(as∣a0s−1\)\),\\mathrm\{PPL\}=\\exp\\left\(\-\\frac\{1\}\{S\+1\}\\sum\_\{s=1\}^\{S\+1\}\\log p\(a\_\{s\}\\mid a\_\{0\}^\{s\-1\}\)\\right\),where the final prediction is the EOS label\. The labelasa\_\{s\}can denote words, single characters, or subword tokens\[[22](https://arxiv.org/html/2607.05612#bib.bib15),[17](https://arxiv.org/html/2607.05612#bib.bib16)\]\. PPL values are only directly comparable when the scoring label unit, normalization \(e\.g\., cased vs\. uncased\), and evaluation text are fixed\. In experiments where all LMs share the same 10k SentencePiece \(SPM\) vocabulary\[[17](https://arxiv.org/html/2607.05612#bib.bib16)\], we report subword\-level PPL\. When comparing LMs with different tokenizations, such as LLMs with different vocabularies, we instead report word\-level PPL to make the perplexities comparable across models\.

The ASR metric of interest is WER\. Following prior work, we study the relation between LM perplexity and WER in log\-log space:

log⁡\(WER\)=α⋅log⁡\(PPL\)\+β\.\\log\(\\mathrm\{WER\}\)=\\alpha\\cdot\\log\(\\mathrm\{PPL\}\)\+\\beta\.\(1\)The slopeα\\alphameasures how sensitive WER is to relative changes in PPL, while the shiftβ\\betacaptures the recognition setup\-dependent offset\. For example, if the relation holds locally, reducing PPL by a factorrrchanges WER by approximately a factorrαr^\{\\alpha\}\.

### II\-BRegression Protocol

For each ASR condition, we fit[Equation1](https://arxiv.org/html/2607.05612#S2.E1)using the LM variants evaluated under the same recognition setup\. When the scatter plot shows a clear change in slope, we additionally fit a piecewise relation with a fixed split point and report the slopes of both regions\.

## IIIASR and LM Integration

### III\-ACTC ASR

For an input acoustic sequencex1Tx\_\{1\}^\{T\}and output label sequencea1Sa\_\{1\}^\{S\}, CTC models the posterior probability by summing over all frame\-level alignmentsy1Ty\_\{1\}^\{T\}that collapse toa1Sa\_\{1\}^\{S\}\. Unlike autoregressive LMs and AED decoders, CTC does not use an EOS label; its frame\-level label set instead includes the blank label\.

pCTC​\(a1S∣x1T\)=∑y1T∈ℬ−1​\(a1S\)∏t=1Tp​\(yt∣x1T\)\.p\_\{\\mathrm\{CTC\}\}\(a\_\{1\}^\{S\}\\mid x\_\{1\}^\{T\}\)=\\sum\_\{y\_\{1\}^\{T\}\\in\\mathcal\{B\}^\{\-1\}\(a\_\{1\}^\{S\}\)\}\\prod\_\{t=1\}^\{T\}p\(y\_\{t\}\\mid x\_\{1\}^\{T\}\)\.The model is trained by minimizing−log⁡pCTC​\(a1S∣x1T\)\-\\log p\_\{\\mathrm\{CTC\}\}\(a\_\{1\}^\{S\}\\mid x\_\{1\}^\{T\}\)\. In recognition, we approximate the CTC sum over alignments by the best alignment score\. Without an external LM, this gives

a^1S^=arg⁡maxa1S⁡\{maxy1T∈ℬ−1​\(a1S\)​∑t=1Tlog⁡p​\(yt∣x1T\)\}\\hat\{a\}\_\{1\}^\{\\hat\{S\}\}=\\arg\\max\_\{a\_\{1\}^\{S\}\}\\bigg\\\{\\max\_\{y\_\{1\}^\{T\}\\in\\mathcal\{B\}^\{\-1\}\(a\_\{1\}^\{S\}\)\}\\sum\_\{t=1\}^\{T\}\\log p\(y\_\{t\}\\mid x\_\{1\}^\{T\}\)\\bigg\\\}With an external LM, we use frame\-synchronous decoding with a log\-linear score over frame\-level CTC labels and a sequence\-level LM score:

a^1S^=argmaxa1S\{\\displaystyle\\hat\{a\}\_\{1\}^\{\\hat\{S\}\}=\\arg\\max\_\{a\_\{1\}^\{S\}\}\\Bigg\\\{maxy1T∈ℬ−1​\(a1S\)∑t=1T\[logp\(yt∣x1T\)\\displaystyle\\max\_\{y\_\{1\}^\{T\}\\in\\mathcal\{B\}^\{\-1\}\(a\_\{1\}^\{S\}\)\}\\sum\_\{t=1\}^\{T\}\\Big\[\\log p\(y\_\{t\}\\mid x\_\{1\}^\{T\}\)−μlogpprior\(yt\)\]\+λlogpLM\(a1S\)\}\\displaystyle\\quad\-\\mu\\log p\_\{\\mathrm\{prior\}\}\(y\_\{t\}\)\\Big\]\+\\lambda\\log p\_\{\\mathrm\{LM\}\}\(a\_\{1\}^\{S\}\)\\Bigg\\\}whereλ\\lambdais the LM scale andμ\\muis the frame\-wise prior scale\. The LM state is advanced only when non\-blank labels are emitted; blank labels do not receive an LM score\. The LM history is initialized with the BOS tokena0a\_\{0\}\. The frame\-wise CTC prior is estimated from posteriors computed over the training data using the softmax average method:pprior​\(y\)=1∑nTn​∑n∑t=1Tnp​\(yt=y∣xn,1Tn\),p\_\{\\mathrm\{prior\}\}\(y\)=\\frac\{1\}\{\\sum\_\{n\}T\_\{n\}\}\\sum\_\{n\}\\sum\_\{t=1\}^\{T\_\{n\}\}p\(y\_\{t\}=y\\mid x\_\{n,1\}^\{T\_\{n\}\}\),wherennindexes training utterances\.

### III\-BAED ASR

Attention\-based encoder\-decoder \(AED\) models directly factorize the output posterior as

pAED​\(a1S∣x1T\)=∏s=1S\+1p​\(as∣a0s−1,x1T\),p\_\{\\mathrm\{AED\}\}\(a\_\{1\}^\{S\}\\mid x\_\{1\}^\{T\}\)=\\prod\_\{s=1\}^\{S\+1\}p\(a\_\{s\}\\mid a\_\{0\}^\{s\-1\},x\_\{1\}^\{T\}\),wherea0=⟨bos⟩a\_\{0\}=\\langle\\mathrm\{bos\}\\rangleandaS\+1=⟨eos⟩a\_\{S\+1\}=\\langle\\mathrm\{eos\}\\rangle\. The encoder maps the acoustic sequence to hidden representations, and the autoregressive decoder predicts each output token conditioned on previous tokens and the encoder context\. The model is trained by minimizing the negative log\-likelihood of the reference sequence, including the EOS label\.

With an external LM, AED recognition uses a log\-linear score

a^1S^=argmaxa1S\[\\displaystyle\\hat\{a\}\_\{1\}^\{\\hat\{S\}\}=\\arg\\max\_\{a\_\{1\}^\{S\}\}\\big\[log⁡pAED​\(a1S∣x1T\)\+λ​log⁡pLM​\(a1S\)\\displaystyle\\log p\_\{\\mathrm\{AED\}\}\(a\_\{1\}^\{S\}\\mid x\_\{1\}^\{T\}\)\+\\lambda\\log p\_\{\\mathrm\{LM\}\}\(a\_\{1\}^\{S\}\)−γlogpILM\(a1S\)\],\\displaystyle\-\\gamma\\log p\_\{\\mathrm\{ILM\}\}\(a\_\{1\}^\{S\}\)\\big\],wherepILMp\_\{\\mathrm\{ILM\}\}denotes an estimate of the AED decoder’s internal language model andγ\\gammais the ILM subtraction scale\. In the ILM experiments, we compare decoding withγ=0\\gamma=0against decoding with tuned ILM subtraction using the Mini\-LSTM ILM estimate\[[29](https://arxiv.org/html/2607.05612#bib.bib12)\]\. Here, sequence\-level LM probabilities are factorized with the same BOS and EOS convention:pLM​\(a1S\)=∏s=1S\+1pLM​\(as∣a0s−1\)p\_\{\\mathrm\{LM\}\}\(a\_\{1\}^\{S\}\)=\\prod\_\{s=1\}^\{S\+1\}p\_\{\\mathrm\{LM\}\}\(a\_\{s\}\\mid a\_\{0\}^\{s\-1\}\)\.

## IVExperimental Setup

### IV\-ACorpora

We evaluate two ASR tasks that differ in speaking style, domain coverage, and the amount of external LM text\.

LibriSpeech\.For read English speech, we use the LibriSpeech 960h corpus\[[19](https://arxiv.org/html/2607.05612#bib.bib23)\]and report results on the standard dev/test clean/other sets, focusing mainly on dev\-other and test\-other\. The LibriSpeech LM corpus contains roughly 800M running words, about 80 times more than the 10M words in the acoustic transcriptions\.

AppTek Spanish\.For spontaneous Spanish speech, we use an in\-house AppTek mixed\-bandwidth task with 8 kHz and 16 kHz data\. It contains about 12k hours of speech, 120M transcription words, and 700M external LM training words\. We report average WER over multiple in\-house test sets covering open\-domain speech and conversational call\-center domains\. Recognition scales are tuned on matching development sets\. Text is lowercased and punctuation is removed\.

The external\-text/transcription\-text ratio is much larger for LibriSpeech than for AppTek Spanish, approximately 80 versus 6\. This is relevant for interpreting PPL\-WER slopes because it affects how much additional information the external LM provides beyond the ASR training transcriptions\.

### IV\-BASR Systems

Our LibriSpeech CTC ASR model uses a 16\-layer Conformer encoder\[[7](https://arxiv.org/html/2607.05612#bib.bib24)\]with 1024 dimensions and CTC auxiliary losses on top of layers 4, 10, and 16 \(406M parameters\)\. For AED models, we use a 12\-layer Conformer encoder with an LSTM decoder with dimension 1024 \(130M parameters\)\. For AppTek Spanish, the CTC ASR model uses a 20\-layer Conformer encoder with dimension 896 \(438M parameters\)\. All models are trained for 100 epochs on LibriSpeech and 8 epochs on AppTek Spanish using the AdamW optimizer\[[18](https://arxiv.org/html/2607.05612#bib.bib25)\]\. For both LibriSpeech and AppTek Spanish, the ASR systems use 10k SPM subword label units and apply on\-the\-fly speed perturbation\[[16](https://arxiv.org/html/2607.05612#bib.bib26)\]and SpecAugment\[[20](https://arxiv.org/html/2607.05612#bib.bib17)\]\.

### IV\-CLanguage Models

We vary LM architecture and size to obtain a broad PPL range\. Unless stated otherwise, LMs use the same 10k SPM vocabulary and text normalization as the corresponding ASR system\.

For LibriSpeech, we train Transformer LMs\[[26](https://arxiv.org/html/2607.05612#bib.bib27)\]on the LibriSpeech LM corpus, varying layers from 2 to 96 and dimensions from 128 to 1280\. The strongest recognition LM has 32 layers, dimension 1024, and about 422M parameters\.

For AppTek Spanish, we train Transformer, LSTM, and n\-gram SPM LMs\. The LSTM variants use 2 or 4 layers and dimensions 256, 512, 1024, and 2048\. The n\-gram LMs use ordersn∈\{2,3,4,5,6\}n\\in\\\{2,3,4,5,6\\\}with different pruning thresholds; larger thresholds remove more low\-impact n\-grams and usually increase PPL\.

### IV\-DRecognition and Tuning

We tune recognition hyperparameters and scales by grid search on the corresponding development sets\. CTC systems use time\-synchronous search\[[5](https://arxiv.org/html/2607.05612#bib.bib5)\], whereas AED systems use label\-synchronous beam search\[[1](https://arxiv.org/html/2607.05612#bib.bib7)\]\. Unless stated otherwise, we use beam size 64\. We also check search errors to ensure that they do not mask the effect of the external LM; they are in the range of 0\.1–0\.3% absolute and are therefore negligible for the trends studied here\.

## VExperiments

### V\-APerplexity\-WER Relation

![Refer to caption](https://arxiv.org/html/2607.05612v1/x1.png)Figure 1:Relation between subword\-level PPL and WER for the LibriSpeech CTC system using Transformer LMs\. The panels show dev\-other and test\-other\. Both axes are on the natural log scale\. We fitlog⁡\(WER\)=α​log⁡\(PPL\)\+β\\log\(\\mathrm\{WER\}\)=\\alpha\\log\(\\mathrm\{PPL\}\)\+\\betabelow and above a PPL split point of 50\. The dashed horizontal line marks CTC recognition without an external LM and is not included in the fits\. On dev\-other, Fit\-1/Fit\-2 have\(α,β\)=\(0\.267,0\.413\)/\(0\.092,1\.111\)\(\\alpha,\\beta\)=\(0\.267,0\.413\)/\(0\.092,1\.111\)\. On test\-other, Fit\-1/Fit\-2 have\(α,β\)=\(0\.246,0\.588\)/\(0\.103,1\.142\)\(\\alpha,\\beta\)=\(0\.246,0\.588\)/\(0\.103,1\.142\)\.[Figure1](https://arxiv.org/html/2607.05612#S5.F1)shows the PPL\-WER relation for the LibriSpeech CTC system with Transformer LMs on the more challenging “other” subsets\. The PPL\-WER trend is steeper in the low\-PPL region and becomes flatter at higher PPLs\. With a split at subword\-level PPL 50, the fitted slope on dev\-other decreases fromα=0\.267\\alpha=0\.267toα=0\.092\\alpha=0\.092, while on test\-other it decreases fromα=0\.246\\alpha=0\.246toα=0\.103\\alpha=0\.103\. Thus, LibriSpeech shows a clear reduction in slope, but not a fully saturated high\-PPL region\. The relation remains approximately linear up to around this split point, and the high\-PPL slopes remain above zero, consistent with CTC still benefiting from external LM information beyond this range\.

![Refer to caption](https://arxiv.org/html/2607.05612v1/x2.png)Figure 2:Relation between subword\-level PPL and WER for the AppTek Spanish CTC system using Transformer, LSTM, and n\-gram LMs\. Both axes are on the natural log scale\. We fitlog⁡\(WER\)=α​log⁡\(PPL\)\+β\\log\(\\mathrm\{WER\}\)=\\alpha\\log\(\\mathrm\{PPL\}\)\+\\betabelow and above a PPL split point of 450\. The dashed horizontal line marks recognition without an external LM and is not included in the fits\. Fit\-1 has\(α,β\)=\(0\.046,1\.964\)\(\\alpha,\\beta\)=\(0\.046,1\.964\), and Fit\-2 has\(α,β\)=\(0\.014,2\.147\)\(\\alpha,\\beta\)=\(0\.014,2\.147\)\.[Figure2](https://arxiv.org/html/2607.05612#S5.F2)shows the relation between external LM perplexity and WER for the AppTek Spanish CTC system\. The relation is monotonic: stronger Transformer LMs give the lowest WERs, followed by LSTM and n\-gram LMs\. At the same time, the slope becomes smaller in the high\-PPL region\. The fit up to a PPL of 450 givesα=0\.046\\alpha=0\.046, whereas the fit above the split givesα=0\.014\\alpha=0\.014\. This suggests a strong flattening at high perplexities\. Compared to LibriSpeech, the AppTek Spanish slopes are substantially smaller\. One likely reason is the ratio between external LM text and ASR transcription text: for LibriSpeech, the LM corpus is about 80 times larger than the acoustic transcriptions, whereas for AppTek Spanish this ratio is about 6\. Thus, the external LM can provide more additional information beyond the transcriptions on LibriSpeech, leading to a stronger dependence of WER on LM perplexity\.

### V\-BEffect of Encoder Context Length

We next study whether the benefit from an external LM depends on the amount of encoder context\. For this experiment, we use a CTC Conformer model trained on LibriSpeech 960h and compare greedy CTC decoding, i\.e\., no external LM, against recognition with the best Transformer LM available in our setup\. To control the encoder context, we restrict both convolution and self\-attention in the Conformer encoder to a fixed center window\. We do not use lookahead or history chunks\. Thus, each model has access only to the acoustic frames inside the current center window\.

![Refer to caption](https://arxiv.org/html/2607.05612v1/x3.png)Figure 3:Effect of encoder context length on CTC recognition with and without an external LM\. We compare CTC greedy decoding against CTC\+LM recognition\. WERs are averaged over LibriSpeech dev\-other and test\-other\. The external LM gives larger relative improvements for shorter context lengths, indicating that LM information becomes more important when the encoder has less acoustic context\.[Figure3](https://arxiv.org/html/2607.05612#S5.F3)shows the average WER on LibriSpeech dev\-other and test\-other for different center\-window sizes\. The results show a clear interaction between encoder context and external LM gain\. With full encoder context, the Transformer LM improves WER from 5\.00% to 3\.88%, corresponding to a relative improvement of 22\.2%\. When the center window is reduced to 0\.3 seconds, greedy CTC decoding degrades strongly from 5\.00% to 40\.83%, while CTC\+LM recognition reduces WER to 18\.05%\. This is a much larger relative improvement of 55\.8%\. Similar behavior is observed for 0\.6\- and 1\.2\-second windows, where the LM gives 50\.8% and 47\.7% relative improvements, respectively\.

These results suggest that the external LM becomes more important when the encoder has limited acoustic context\. In the short\-context regime, the CTC model has less information to disambiguate acoustically plausible hypotheses, and the external LM can compensate for part of this missing context during search\. This may also be interpreted as indirectly weakening the encoder’s internal language modeling capacity\[[28](https://arxiv.org/html/2607.05612#bib.bib18)\], which makes the external LM more beneficial\. However, the LM does not fully recover the full\-context baseline for very short windows\. For example, the 0\.3 second model with an LM still has 18\.05% WER, which is far above the 3\.88% WER of the full\-context model with an LM\.

The effect becomes smaller as the center window increases\. Around 4\.8 seconds and beyond, the gap between chunked and full\-context recognition is much smaller, and the relative LM gain also decreases\. For example, the LM improves the 4\.8 second model by 31\.8% relative, but the gain decreases to about 24% for 9\.6–15\.6 second windows, close to the 22\.2% gain observed for the full\-context model\. This indicates that once the encoder sees several seconds of speech, the external LM is still useful but no longer compensates for a large context deficit\.

### V\-CEffect of Temperature\-Based LM Probability Smoothing

We smooth the external LM distribution by a softmax*temperature*TT, i\.e\., we score the LM aslog⁡softmax⁡\(z/T\)\\log\\operatorname\{softmax\}\(z/T\)from the LM logitszz, so thatT\>1T\>1flattens the distribution andT<1T<1sharpens it\. We apply this to the LibriSpeech CTC system with a Transformer LM, sweepTT, re\-tune the log\-linear fusion scale for eachTT, and report the subword\-level perplexity and the WER\.

TABLE I:Effect of LM temperature \(T\) smoothing for the LibriSpeech CTC system on LibriSpeech dev\-other and test\-other\. Decoding scales are re\-tuned for each temperature\.TTdev\-othertest\-otherPPLWER\[%\]PPLWER\[%\]0\.50135\.13\.81134\.94\.140\.7051\.23\.7951\.14\.090\.8537\.93\.8137\.64\.051\.0034\.13\.7533\.84\.021\.2035\.93\.7135\.64\.061\.5049\.23\.8648\.54\.102\.00100\.23\.8798\.74\.05[TableI](https://arxiv.org/html/2607.05612#S5.T1)shows a controlled case where PPL and WER decouple\. On dev\-other, subword\-level PPL varies by a factor of 3\.96 across the temperature sweep, from 34\.1 to 135\.1, while WER only changes between 3\.71% and 3\.87%\. On test\-other, subword\-level PPL varies by a factor of 3\.99, from 33\.8 to 134\.9, while WER remains between 4\.02% and 4\.14%\. The best PPL also does not always correspond to the best WER: on dev\-other, the lowest PPL is obtained atT=1\.0T=1\.0, whereas the lowest WER is obtained atT=1\.2T=1\.2\.

This suggests that PPL changes caused by post\-hoc probability smoothing mainly reflect changes in calibration or entropy, not necessarily changes in the ranking of recognition hypotheses\. After re\-tuning the decoding scales, WER becomes nearly insensitive to these PPL changes\. Therefore, PPL\-WER comparisons should distinguish between genuinely different LMs and post\-hoc transformations of the same LM distribution\.

### V\-DEffect of Internal LM Subtraction on PPL\-WER Relation for AED ASR Models

![Refer to caption](https://arxiv.org/html/2607.05612v1/x4.png)Figure 4:Relation between subword\-level PPL and WER for a Conformer AED model on LibriSpeech dev\-other, without and with ILM subtraction\. Both axes are on the natural log scale\. We fitlog⁡\(WER\)=α​log⁡\(PPL\)\+β\\log\(\\mathrm\{WER\}\)=\\alpha\\log\(\\mathrm\{PPL\}\)\+\\betabelow and above a PPL split point of 100\. The dashed horizontal line marks recognition without an external LM and is not included in the fits\. Without ILM subtraction, Fit\-1/Fit\-2 have\(α,β\)=\(0\.200,0\.732\)/\(0\.011,1\.603\)\(\\alpha,\\beta\)=\(0\.200,0\.732\)/\(0\.011,1\.603\)\. With ILM subtraction, Fit\-1/Fit\-2 have\(α,β\)=\(0\.294,0\.275\)/\(0\.007,1\.618\)\(\\alpha,\\beta\)=\(0\.294,0\.275\)/\(0\.007,1\.618\)\.For attention\-based encoder\-decoder \(AED\) models, the decoder learns an internal language model \(ILM\)\[[29](https://arxiv.org/html/2607.05612#bib.bib12)\]\. The ILM acts as a strong prior that can reduce the gain from external LMs and also weaken the correlation between perplexity and WER\. Thus, we want to investigate how the perplexity\-WER relation changes when we apply ILM subtraction during recognition\. We use the Mini\-LSTM ILM\[[29](https://arxiv.org/html/2607.05612#bib.bib12)\]as the ILM estimation method\.[Figure4](https://arxiv.org/html/2607.05612#S5.F4)shows the correlation between subword\-level perplexity and WER using a Conformer AED model on the LibriSpeech dev\-other set without and with ILM subtraction, respectively\. Both axes are logarithmic, and the regression is performed using natural logarithms\. In both cases, the relation is described by two regimes\. For perplexity below approximately 100, the relation is approximately linear in log\-log space\. However, with higher perplexities, the slope is close to 0, indicating a saturation effect and a weak correlation between perplexity and WER\. This contrasts with the LibriSpeech CTC experiment, where the high\-PPL slope is reduced but remains non\-negligible over the evaluated range\.

Applying ILM subtraction increases the slope from 0\.19 to 0\.29 in the low\-perplexity regime\. This indicates that ILM subtraction strengthens the correlation between perplexity and WER, and also increases the gain from external LMs, so it is an important factor when studying AED systems\.

Moreover, we computed the ILM perplexity of the model on the dev\-other set, and it is 110, which is close to the empirical transition point between the low\-perplexity and high\-perplexity regimes\. This suggests that the saturation effect is likely due to the ILM dominating the predictions when the external LM perplexity is higher than the ILM perplexity, which makes the fitted slope close to 0\.

### V\-EHow does LLM PPL fit into the PPL\-WER relation?

![Refer to caption](https://arxiv.org/html/2607.05612v1/x5.png)Figure 5:Relation between word\-level PPL and WER for LLM\-based recognition on the LibriSpeech CTC system, evaluated on dev\-other\. In the legend, ASR vocab denotes the 10k SentencePiece vocabulary of the ASR system\. Both axes are on the natural log scale\. The dashed horizontal line marks recognition without an external LM\.We next compare standard neural LMs with pretrained text\-based Qwen2 LLMs\[[27](https://arxiv.org/html/2607.05612#bib.bib29)\]on the LibriSpeech CTC system\. Because the LLMs and ASR system may use different tokenizations, we report word\-level PPL in this experiment\. We evaluate Qwen2 base models, variants where newline is used as the EOS label, and fully fine\-tuned models on LibriSpeech transcriptions and external LM text\. For fine\-tuning with the ASR SPM vocabulary, we initialize SPM embeddings from the LLM vocabulary following\[[4](https://arxiv.org/html/2607.05612#bib.bib30)\]: matching labels reuse the corresponding embedding, while non\-matching labels are initialized by averaging the embeddings of their LLM\-tokenized pieces\. This full fine\-tuning setup follows our previous work\[[21](https://arxiv.org/html/2607.05612#bib.bib28)\]\. For recognition, models with the ASR SPM vocabulary use time\-synchronous one\-pass search\. Models with the original LLM vocabulary use delayed fusion\[[9](https://arxiv.org/html/2607.05612#bib.bib13)\], where LLM scores are fused every 10 frames\. We tune the beam size and observe saturation at beam size 64\.

[Figure5](https://arxiv.org/html/2607.05612#S5.F5)shows that word\-level PPL for LLMs must be interpreted carefully\. Word\-level PPL is on a different scale from the subword\-level PPL used in the previous experiments, since the LibriSpeech word vocabulary has about 200k entries and is much larger than the 10k SPM vocabulary\. Thus, the absolute PPL values in this figure should not be compared directly to the subword\-level PPL\-WER curves\. Under the common word\-level scoring used here, the base Qwen2 models have higher PPL than the Transformer LM, e\.g\. 250–356 versus 54\.5, but still improve over no\-LM recognition, reducing WER from 5\.02% to 4\.29–4\.42%\.

The EOS choice mainly affects PPL, not WER\. Using newline as EOS reduces PPL substantially, e\.g\. from 355\.5 to 167\.4 for Qwen2\-0\.5B and from 250\.6 to 122\.4 for Qwen2\-1\.5B, while WER remains unchanged at 4\.42% and 4\.29%, respectively\. This suggests that EOS handling can strongly affect intrinsic PPL without changing the recognition ranking\. Across comparable settings, the larger 1\.5B model consistently gives lower WER than the 0\.5B model\.

Fine\-tuning moves the LLMs closer to the standard LM region under word\-level scoring, but vocabulary and integration still matter\. With the original LLM vocabulary, fine\-tuned Qwen2 models reach PPLs comparable to the Transformer LM but remain worse in WER\. In contrast, fine\-tuning with the ASR SPM vocabulary gives the best LLM results: Qwen2\-0\.5B\-FT\-SPM matches the Transformer LM \(3\.76% vs\. 3\.75%\), and Qwen2\-1\.5B\-FT\-SPM improves further to 3\.65%\. Thus, this experiment is best interpreted as showing the effect of tokenization, fine\-tuning, and search integration on how LLM PPL relates to WER, rather than as a direct comparison to the subword\-level PPL\-WER trends\.

## VIConclusion

We revisited the relation between language model \(LM\) perplexity \(PPL\) and automatic speech recognition \(ASR\) word error rate \(WER\) for modern end\-to\-end ASR\. The relation remains informative, but is not always a single global log\-log trend: several setups show a low\-PPL region with larger WER sensitivity and a high\-PPL region with smaller slope\. For Connectionist temporal classification \(CTC\), external LMs still help, especially with restricted encoder context\. For attention\-based encoder\-decoder \(AED\) ASR, the internal language model \(ILM\) affects how external LM quality maps to WER, and ILM subtraction increases the low\-PPL slope\. Finally, the large language model \(LLM\) experiments show that word\-level PPL is harder to interpret across tokenizations and integration methods: under word\-level scoring, Qwen2 LLMs improve over recognition without an external LM even when its PPL is higher than the standard LM, newline\-based end\-of\-sentence \(EOS\) handling changes PPL without changing WER, and Qwen2 LLMs with the ASR SentencePiece \(SPM\) vocabulary give the best LLM results\.

## VIIAcknowledgments

The authors gratefully acknowledge the computing time provided to them at the NHR Center NHR4CES at RWTH Aachen University \(project number p0023999\)\. This is funded by the Federal Ministry of Education and Research, and the state governments participating on the basis of the resolutions of the GWK for national high performance computing at universities \(www\.nhr\-verein\.de/unsere\-partner\)\.

## VIIIGenerative AI Use Disclosure

We use LLMs to improve the formulations and grammar of the paper\.

## References

- \[1\]\(2016\)Listen, attend and spell: a neural network for large vocabulary conversational speech recognition\.In2016 IEEE International Conference on Acoustics, Speech and Signal Processing,pp\. 4960–4964\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP.2016.7472621)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p3.1),[§IV\-D](https://arxiv.org/html/2607.05612#S4.SS4.p1.1)\.
- \[2\]C\. Chelba, D\. Bikel, M\. Shugrina, P\. Nguyen, and S\. Kumar\(2012\)Large scale language modeling in automatic speech recognition\.CoRRabs/1210\.8440\.External Links:[Link](http://arxiv.org/abs/1210.8440),1210\.8440Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p2.1)\.
- \[3\]S\. F\. Chen, D\. Beeferman, and R\. Rosenfeld\(1998\-02\)Evaluation metrics for language models\.InProceedings of the DARPA Broadcast News Transcription and Understanding Workshop,Lansdowne, VA, USA,pp\. 275–280\.Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p1.1)\.
- \[4\]K\. Deng, J\. Guo, Y\. Ma, N\. Moritz, P\. C\. Woodland, O\. Kalinli, and M\. Seltzer\(2025\)Transducer\-llama: integrating llms into streamable transducer\-based speech recognition\.In2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025, Hyderabad, India, April 6\-11, 2025,pp\. 1–5\.External Links:[Link](https://doi.org/10.1109/ICASSP49660.2025.10889530),[Document](https://dx.doi.org/10.1109/ICASSP49660.2025.10889530)Cited by:[§V\-E](https://arxiv.org/html/2607.05612#S5.SS5.p1.1)\.
- \[5\]A\. Graves, S\. Fernández, F\. Gomez, and J\. Schmidhuber\(2006\)Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks\.InProceedings of the 23rd International Conference on Machine Learning,pp\. 369–376\.External Links:[Document](https://dx.doi.org/10.1145/1143844.1143891)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p3.1),[§IV\-D](https://arxiv.org/html/2607.05612#S4.SS4.p1.1)\.
- \[6\]A\. Graves\(2012\)Sequence transduction with recurrent neural networks\.InICML Workshop on Representation Learning,External Links:[Link](https://arxiv.org/abs/1211.3711)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p3.1)\.
- \[7\]A\. Gulati, J\. Qin, C\. Chiu, N\. Parmar, Y\. Zhang, J\. Yu, W\. Han, S\. Wang, Z\. Zhang, Y\. Wu, and R\. Pang\(2020\)Conformer: convolution\-augmented transformer for speech recognition\.In21st Annual Conference of the International Speech Communication Association, Interspeech 2020, Shanghai, China, October 25\-29, 2020,pp\. 5036–5040\.External Links:[Link](https://doi.org/10.21437/Interspeech.2020-3015),[Document](https://dx.doi.org/10.21437/Interspeech.2020-3015)Cited by:[§IV\-B](https://arxiv.org/html/2607.05612#S4.SS2.p1.1)\.
- \[8\]Y\. Halpern, K\. B\. Hall, V\. Schogol, M\. Riley, B\. Roark, G\. Skobeltsyn, and M\. Bäuml\(2016\)Contextual prediction models for speech recognition\.In17th Annual Conference of the International Speech Communication Association, Interspeech 2016, San Francisco, CA, USA, September 8\-12, 2016,N\. Morgan \(Ed\.\),pp\. 2338–2342\.External Links:[Link](https://doi.org/10.21437/Interspeech.2016-1358),[Document](https://dx.doi.org/10.21437/INTERSPEECH.2016-1358)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p1.1)\.
- \[9\]T\. Hori, M\. Kocour, A\. Haider, E\. McDermott, and X\. Zhuang\(2025\)Delayed fusion: integrating large language models into first\-pass decoding in end\-to\-end speech recognition\.In2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025, Hyderabad, India, April 6\-11, 2025,pp\. 1–5\.External Links:[Link](https://doi.org/10.1109/ICASSP49660.2025.10890391),[Document](https://dx.doi.org/10.1109/ICASSP49660.2025.10890391)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p4.1),[§V\-E](https://arxiv.org/html/2607.05612#S5.SS5.p1.1)\.
- \[10\]K\. Irie, A\. Zeyer, R\. Schlüter, and H\. Ney\(2019\)Language modeling with deep transformers\.In20th Annual Conference of the International Speech Communication Association, Interspeech 2019, Graz, Austria, September 15\-19, 2019,G\. Kubin and Z\. Kacic \(Eds\.\),pp\. 3905–3909\.External Links:[Link](https://doi.org/10.21437/Interspeech.2019-2225),[Document](https://dx.doi.org/10.21437/INTERSPEECH.2019-2225)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p2.1)\.
- \[11\]K\. Irie\(2020\)Advancing neural language modeling in automatic speech recognition\.Ph\.D\. Thesis,RWTH Aachen University, Germany\.External Links:[Link](https://publications.rwth-aachen.de/record/789081)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p1.1)\.
- \[12\]F\. Jelinek, R\. L\. Mercer, L\. R\. Bahl, and J\. K\. Baker\(1977\)Perplexity—a measure of the difficulty of speech recognition tasks\.The Journal of the Acoustical Society of America62\(S1\),pp\. S63–S63\.External Links:[Document](https://dx.doi.org/10.1121/1.2016299)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p1.1)\.
- \[13\]A\. Kannan, Y\. Wu, P\. Nguyen, T\. N\. Sainath, Z\. Chen, and R\. Prabhavalkar\(2018\)An analysis of incorporating an external language model into a sequence\-to\-sequence model\.In2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018, Calgary, AB, Canada, April 15\-20, 2018,pp\. 5824–5828\.External Links:[Link](https://doi.org/10.1109/ICASSP.2018.8462682),[Document](https://dx.doi.org/10.1109/ICASSP.2018.8462682)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p2.1)\.
- \[14\]D\. Klakow and J\. Peters\(2002\)Testing the correlation of word error rate and perplexity\.Speech Commun\.38\(1\-2\),pp\. 19–28\.External Links:[Link](https://doi.org/10.1016/S0167-6393(01)00041-3),[Document](https://dx.doi.org/10.1016/S0167-6393%2801%2900041-3)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p1.1)\.
- \[15\]R\. Kneser and H\. Ney\(1995\)Improved backing\-off for m\-gram language modeling\.In1995 International Conference on Acoustics, Speech, and Signal Processing, ICASSP ’95, Detroit, Michigan, USA, May 08\-12, 1995,pp\. 181–184\.External Links:[Link](https://doi.org/10.1109/ICASSP.1995.479394),[Document](https://dx.doi.org/10.1109/ICASSP.1995.479394)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p1.1)\.
- \[16\]T\. Ko, V\. Peddinti, D\. Povey, and S\. Khudanpur\(2015\)Audio augmentation for speech recognition\.In16th Annual Conference of the International Speech Communication Association, Interspeech 2015, Dresden, Germany, September 6\-10, 2015,pp\. 3586–3589\.External Links:[Link](https://doi.org/10.21437/Interspeech.2015-711),[Document](https://dx.doi.org/10.21437/Interspeech.2015-711)Cited by:[§IV\-B](https://arxiv.org/html/2607.05612#S4.SS2.p1.1)\.
- \[17\]T\. Kudo and J\. Richardson\(2018\)SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018: System Demonstrations, Brussels, Belgium, October 31 \- November 4, 2018,E\. Blanco and W\. Lu \(Eds\.\),pp\. 66–71\.External Links:[Link](https://doi.org/10.18653/v1/d18-2012),[Document](https://dx.doi.org/10.18653/V1/D18-2012)Cited by:[§II\-A](https://arxiv.org/html/2607.05612#S2.SS1.p1.4)\.
- \[18\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\-9, 2019,External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[§IV\-B](https://arxiv.org/html/2607.05612#S4.SS2.p1.1)\.
- \[19\]V\. Panayotov, G\. Chen, D\. Povey, and S\. Khudanpur\(2015\)Librispeech: an ASR corpus based on public domain audio books\.InIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2015, South Brisbane, Queensland, Australia, April 19\-24, 2015,pp\. 5206–5210\.External Links:[Link](https://doi.org/10.1109/ICASSP.2015.7178964),[Document](https://dx.doi.org/10.1109/ICASSP.2015.7178964)Cited by:[§IV\-A](https://arxiv.org/html/2607.05612#S4.SS1.p2.1)\.
- \[20\]D\. S\. Park, W\. Chan, Y\. Zhang, C\. Chiu, B\. Zoph, E\. D\. Cubuk, and Q\. V\. Le\(2019\)SpecAugment: a simple data augmentation method for automatic speech recognition\.InInterspeech,pp\. 2613–2617\.Cited by:[§IV\-B](https://arxiv.org/html/2607.05612#S4.SS2.p1.1)\.
- \[21\]R\. Schmitt, A\. Zeyer, M\. Zeineldeen, R\. Schlüter, and H\. Ney\(2026\)LLMs and speech: integration vs\. combination\.External Links:2603\.15045,[Link](https://arxiv.org/abs/2603.15045)Cited by:[§V\-E](https://arxiv.org/html/2607.05612#S5.SS5.p1.1)\.
- \[22\]R\. Sennrich, B\. Haddow, and A\. Birch\(2016\)Neural machine translation of rare words with subword units\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7\-12, 2016, Berlin, Germany, Volume 1: Long Papers,External Links:[Link](https://doi.org/10.18653/v1/p16-1162),[Document](https://dx.doi.org/10.18653/V1/P16-1162)Cited by:[§II\-A](https://arxiv.org/html/2607.05612#S2.SS1.p1.4)\.
- \[23\]A\. Sriram, H\. Jun, S\. Satheesh, and A\. Coates\(2018\)Cold fusion: training seq2seq models together with language models\.In19th Annual Conference of the International Speech Communication Association, Interspeech 2018, Hyderabad, India, September 2\-6, 2018,B\. Yegnanarayana \(Ed\.\),pp\. 387–391\.External Links:[Link](https://doi.org/10.21437/Interspeech.2018-1392),[Document](https://dx.doi.org/10.21437/INTERSPEECH.2018-1392)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p2.1)\.
- \[24\]M\. Sundermeyer, H\. Ney, and R\. Schlüter\(2015\)From feedforward to recurrent LSTM neural networks for language modeling\.IEEE ACM Trans\. Audio Speech Lang\. Process\.23\(3\),pp\. 517–529\.External Links:[Link](https://doi.org/10.1109/TASLP.2015.2400218),[Document](https://dx.doi.org/10.1109/TASLP.2015.2400218)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p1.1)\.
- \[25\]S\. Toshniwal, A\. Kannan, C\. Chiu, Y\. Wu, T\. N\. Sainath, and K\. Livescu\(2018\)A comparison of techniques for language model integration in encoder\-decoder speech recognition\.In2018 IEEE Spoken Language Technology Workshop, SLT 2018, Athens, Greece, December 18\-21, 2018,pp\. 369–375\.External Links:[Link](https://doi.org/10.1109/SLT.2018.8639038),[Document](https://dx.doi.org/10.1109/SLT.2018.8639038)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p2.1)\.
- \[26\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA, December 4\-9, 2017,pp\. 5998–6008\.External Links:[Link](https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p2.1),[§IV\-C](https://arxiv.org/html/2607.05612#S4.SS3.p2.1)\.
- \[27\]A\. Yang, B\. Yang, B\. Hui, B\. Zheng, B\. Yu, C\. Zhou, C\. Li, C\. Li, D\. Liu, F\. Huang, G\. Dong, H\. Wei, H\. Lin, J\. Tang, J\. Wang, J\. Yang, J\. Tu, J\. Zhang, J\. Ma, J\. Yang, J\. Xu, J\. Zhou, J\. Bai, J\. He, J\. Lin, K\. Dang, K\. Lu, K\. Chen, K\. Yang, M\. Li, M\. Xue, N\. Ni, P\. Zhang, P\. Wang, R\. Peng, R\. Men, R\. Gao, R\. Lin, S\. Wang, S\. Bai, S\. Tan, T\. Zhu, T\. Li, T\. Liu, W\. Ge, X\. Deng, X\. Zhou, X\. Ren, X\. Zhang, X\. Wei, X\. Ren, X\. Liu, Y\. Fan, Y\. Yao, Y\. Zhang, Y\. Wan, Y\. Chu, Y\. Liu, Z\. Cui, Z\. Zhang, Z\. Guo, and Z\. Fan\(2024\)Qwen2 technical report\.CoRRabs/2407\.10671\.External Links:[Link](https://doi.org/10.48550/arXiv.2407.10671),[Document](https://dx.doi.org/10.48550/ARXIV.2407.10671),2407\.10671Cited by:[§V\-E](https://arxiv.org/html/2607.05612#S5.SS5.p1.1)\.
- \[28\]Z\. Yang, M\. Phan, R\. Schlüter, and H\. Ney\(2025\)Label\-context\-dependent internal language model estimation for CTC\.In26th Annual Conference of the International Speech Communication Association, Interspeech 2025, Rotterdam, The Netherlands, 17\-21 August 2025,O\. Scharenborg, C\. Oertel, and K\. Truong \(Eds\.\),External Links:[Link](https://doi.org/10.21437/Interspeech.2025-1378),[Document](https://dx.doi.org/10.21437/INTERSPEECH.2025-1378)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p3.1),[§V\-B](https://arxiv.org/html/2607.05612#S5.SS2.p3.1)\.
- \[29\]M\. Zeineldeen, A\. Glushko, W\. Michel, A\. Zeyer, R\. Schlüter, and H\. Ney\(2021\)Investigating methods to improve language model integration for attention\-based encoder\-decoder ASR models\.In22nd Annual Conference of the International Speech Communication Association, Interspeech 2021, Brno, Czechia, August 30 \- September 3, 2021,H\. Hermansky, H\. Cernocký, L\. Burget, L\. Lamel, O\. Scharenborg, and P\. Motlícek \(Eds\.\),pp\. 2856–2860\.External Links:[Link](https://doi.org/10.21437/Interspeech.2021-1255),[Document](https://dx.doi.org/10.21437/INTERSPEECH.2021-1255)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p3.1),[§III\-B](https://arxiv.org/html/2607.05612#S3.SS2.p2.4),[§V\-D](https://arxiv.org/html/2607.05612#S5.SS4.p1.1)\.
- \[30\]W\. Zhou, Z\. Zheng, R\. Schlüter, and H\. Ney\(2022\)On language model integration for RNN transducer based speech recognition\.InIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singapore, 23\-27 May 2022,pp\. 8407–8411\.External Links:[Link](https://doi.org/10.1109/ICASSP43922.2022.9746948),[Document](https://dx.doi.org/10.1109/ICASSP43922.2022.9746948)Cited by:[§I](https://arxiv.org/html/2607.05612#S1.p3.1)\.