Dual-Form ASR: Semantics-Aware Inverse Text Normalization for Chinese Speech Recognition

# Dual-Form ASR: Semantics-Aware Inverse Text Normalization for Chinese Speech Recognition
Source: [https://arxiv.org/html/2609.02901](https://arxiv.org/html/2609.02901)
###### Abstract

Modern automatic speech recognition \(ASR\) scenarios require both spoken\-form transcripts for faithful transcription and readable written\-form transcripts with inverse text normalization \(ITN\)\. However, these forms are typically produced by cascaded modules, where a spoken\-form ASR output is rewritten by a separate ITN component, making written\-form ASR\-ITN vulnerable to recognition errors and decoupling normalization from acoustic\-contextual modeling, especially for semantically dependent numeric expressions\. In this paper, we propose Dual\-Form ASR \(DF\-ASR\), a framework that extends spoken\-form ASR capability to semantics\-aware written\-form ITN through paired spoken\-form and written\-form supervision while retaining prompt\-level selection between transcript forms\. The dual\-form supervision is constructed via a large language model \(LLM\)\-driven generate\-and\-judge workflow, and training is further enhanced by ITN\-MWER, a sequence\-level objective that assigns higher cost to errors on normalization\-sensitive spans\. We also introduce a decision\-awareRequire\-ITN/Forbid\-ITNprotocol to separately measure required normalization and forbidden\-span preservation\. On manually annotated Chinese subsets fromSpeechIO,DF\-ASRconsistently outperforms open\-source ASR\-ITN systems, remains competitive with strong closed\-source references, and preserves reliable prompt\-level control between spoken\-form and written\-form outputs\.

## IIntroduction

Recent advances in large language models \(LLMs\)\[[21](https://arxiv.org/html/2609.02901#bib.bib6),[6](https://arxiv.org/html/2609.02901#bib.bib5),[28](https://arxiv.org/html/2609.02901#bib.bib1),[27](https://arxiv.org/html/2609.02901#bib.bib2),[13](https://arxiv.org/html/2609.02901#bib.bib7)\]are enabling automatic speech recognition \(ASR\) systems to generate information\-rich transcripts rather than verbatim text\[[31](https://arxiv.org/html/2609.02901#bib.bib9),[12](https://arxiv.org/html/2609.02901#bib.bib10),[8](https://arxiv.org/html/2609.02901#bib.bib11),[19](https://arxiv.org/html/2609.02901#bib.bib12)\]\. In practical ASR deployment, readable display, captions, and meeting minutes often require written\-form transcripts with inverse text normalization \(ITN\), whereas corpus annotation, linguistic analysis, and some downstream modules require faithful spoken\-form transcription\. Open Chinese ASR already provides a strong spoken\-form recognition foundation, but extending this ability to written\-form ASR\-ITN remains difficult since ITN is not simple digit replacement: it depends on the semantic role of the expression in context\.

Three gaps limit reliable open Chinese ASR\-ITN\. First, cascaded ITN lacks joint optimization\. As shown in Fig\.[1](https://arxiv.org/html/2609.02901#S1.F1)\(a\), practical systems usually recognize speech first and then apply a text\-level ITN module\[[10](https://arxiv.org/html/2609.02901#bib.bib35),[37](https://arxiv.org/html/2609.02901#bib.bib34),[2](https://arxiv.org/html/2609.02901#bib.bib41),[4](https://arxiv.org/html/2609.02901#bib.bib39),[5](https://arxiv.org/html/2609.02901#bib.bib40),[25](https://arxiv.org/html/2609.02901#bib.bib37),[26](https://arxiv.org/html/2609.02901#bib.bib38),[14](https://arxiv.org/html/2609.02901#bib.bib42)\]\. The normalizer only observes recognized text, so ASR errors propagate into the ITN stage and numeric\-expression normalization decisions cannot be optimized with acoustic recognition\.

Second, written\-form ASR\-ITN is a context\-sensitive semantic decision rather than a local formatting operation\. The same spoken numeric expression may need to be normalized or preserved in spoken form depending on its semantic role\. For example, “ten to one” may be written as “10:1” when it denotes a ratio, but should remain spoken in “it is ten to one that …”, where the phrase means “very likely”\. Such cases are difficult to maintain with rigid rule\-based ITN systems, since context\-dependent exceptions are hard to scale\. Reliable ASR\-ITN therefore requires semantics\-aware normalization decisions grounded in the surrounding linguistic context\.

Third, evaluation must measure over\-normalization, not only required normalization\. A system that aggressively converts every numeric\-looking span can improve readability on ordinary quantities but damage semantically constrained expressions that should remain in spoken form\. Therefore, ASR\-ITN needs a protocol that jointly evaluates whether required spans are normalized and whether protected spans are preserved\.

![Refer to caption](https://arxiv.org/html/2609.02901v1/x1.png)Figure 1:Comparison of ASR\-ITN paradigms\. \(a\) Cascaded ASR decouples spoken\-form recognition from written\-form normalization by applying a separate ITN module after recognition\. \(b\) Dual\-Form ASR jointly trains one prompt\-conditioned recognizer with paired spoken\-form and written\-form targets, enabling prompt\-level selection between transcript forms\.To address these gaps, we introduce Dual\-Form ASR \(DF\-ASR\), a framework that extends spoken\-form ASR capability to semantics\-aware written\-form ITN\. As illustrated in Fig\.[1](https://arxiv.org/html/2609.02901#S1.F1)\(b\),DF\-ASRis not designed to replace spoken\-form ASR or relearn acoustic recognition from scratch\. Instead, the spoken\-form target preserves faithful recognition behavior, while the written\-form target teaches when numeric expressions should be normalized or preserved in written\-form transcription\. The prompt selects the transcript form, and semantic interpretation determines the numeric\-expression normalization policy within written\-form mode\. Our contributions are summarized as follows:

- •We propose Dual\-Form ASR \(DF\-ASR\), a prompt\-conditioned ASR\-ITN framework that extends spoken\-form ASR capability to semantics\-aware written\-form ITN while preserving prompt\-level selection between spoken\-form and written\-form transcripts\.
- •We construct dual\-form supervision from spoken\-form ASR corpora with an LLM\-driven generate\-and\-judge workflow, and further optimize the model with ITN\-MWER, an ITN\-aware objective that assigns higher cost to errors on normalization\-sensitive spans\.
- •We design a decision\-awareRequire\-ITN/Forbid\-ITNevaluation protocol and show thatDF\-ASRachieves 4\.64% I\-CER and 94\.85% keyword F1 onRequire\-ITN, while obtaining a 95\.18% forbidden\-span preservation rate \(FSPR\) onForbid\-ITN\.

## IIRelated Work

### II\-AFormatted and LLM\-based ASR

LLM\-based ASR combines speech encoders with powerful text decoders and has been extended to contextual biasing\[[33](https://arxiv.org/html/2609.02901#bib.bib17),[9](https://arxiv.org/html/2609.02901#bib.bib19)\], speaker\-aware recognition\[[24](https://arxiv.org/html/2609.02901#bib.bib20),[34](https://arxiv.org/html/2609.02901#bib.bib21)\], code\-switching\[[36](https://arxiv.org/html/2609.02901#bib.bib23),[17](https://arxiv.org/html/2609.02901#bib.bib24)\], and multilingual recognition\[[3](https://arxiv.org/html/2609.02901#bib.bib25),[22](https://arxiv.org/html/2609.02901#bib.bib27)\]\. These studies show that ASR is moving beyond plain transcription toward speech\-language generation with richer formatting and contextual modeling\[[31](https://arxiv.org/html/2609.02901#bib.bib9),[8](https://arxiv.org/html/2609.02901#bib.bib11),[19](https://arxiv.org/html/2609.02901#bib.bib12)\]\.

Formatted transcription can also be learned implicitly\. Whisper predicts raw Internet transcripts without heavy text standardization and therefore acquires orthographic formatting abilities such as punctuation, casing, and numeric\-expression normalization\[[23](https://arxiv.org/html/2609.02901#bib.bib28)\]\. Some open LLM\-based ASR systems further provide prompt interfaces for requesting formatted or normalized transcripts\[[1](https://arxiv.org/html/2609.02901#bib.bib4)\]\. However, implicit formatting and prompt\-elicited normalization provide only weak control over the boundary between spoken\-form transcription and written\-form rewriting\. Without explicit supervision for both output forms, ITN behavior can become unstable and unreliable, particularly when numeric expressions require context\-sensitive semantic decisions\.

### II\-BStandalone ITN and Spoken\-Written Conversion

Standalone ITN is typically treated as text\-level spoken\-to\-written conversion\. Rule\-based systems based on hand\-crafted grammars and weighted finite\-state transducers \(WFSTs\)\[[37](https://arxiv.org/html/2609.02901#bib.bib34),[30](https://arxiv.org/html/2609.02901#bib.bib36)\]are efficient on covered semiotic classes, but are difficult to maintain for open\-domain, context\-dependent numeric expressions\. Neural converters, including sequence\-to\-sequence models\[[25](https://arxiv.org/html/2609.02901#bib.bib37)\], copy\-or\-rewrite taggers\[[2](https://arxiv.org/html/2609.02901#bib.bib41)\], streaming transducers\[[10](https://arxiv.org/html/2609.02901#bib.bib35),[26](https://arxiv.org/html/2609.02901#bib.bib38),[14](https://arxiv.org/html/2609.02901#bib.bib42)\], and decoder\-only pretrained language models\[[4](https://arxiv.org/html/2609.02901#bib.bib39),[5](https://arxiv.org/html/2609.02901#bib.bib40)\], reduce rule engineering but still operate after recognition\. Thus, recognition errors have already occurred, and normalization decisions cannot be optimized jointly with acoustic recognition\. Since normalization is a conditional presentation decision rather than a universal action after numeric detection, these limitations motivate our dual\-form setting with both faithful spoken\-form and application\-oriented written\-form targets\.

### II\-CLLM\-generated Supervision and ITN Evaluation

LLM\-generated supervision and LLM\-as\-a\-judge paradigms have been used to synthesize training signals or provide scalable quality control\[[29](https://arxiv.org/html/2609.02901#bib.bib29),[38](https://arxiv.org/html/2609.02901#bib.bib31),[18](https://arxiv.org/html/2609.02901#bib.bib32)\]\. For ASR\-ITN, however, the target is not open\-ended instruction following but constrained spoken\-to\-written transformation: the written\-form target should change appropriate numeric spans while preserving all non\-ITN content\.

Evaluation also needs to reflect this constrained decision\. Conventional ASR metrics measure overall transcription quality, while standalone ITN metrics often emphasize whether required numeric spans are converted\. They do not isolate harmful over\-normalization on idioms, names, approximate quantities, and other protected spans\. We therefore use LLMs to generate and judge written\-form candidates, apply lightweight sanity checks for obvious invalid cases, and evaluate required normalization and forbidden\-span preservation separately\.

## IIIMethod

DF\-ASRformulates ASR\-ITN as prompt\-conditioned dual\-form generation\. Given speechxxand a promptpp, the model estimatesPθ​\(y∣x,p\)P\_\{\\theta\}\(y\\mid x,p\)\. We usepsp^\{s\}for spoken\-form ASR andpwp^\{w\}for written\-form ASR\-ITN\. The spoken\-form target preserves faithful recognition behavior, while the written\-form target teaches semantics\-aware ITN\. The prompt selects the transcript form, and semantic interpretation determines whether spoken semiotic expressions should be normalized or preserved in written\-form mode\.

### III\-ALLM\-driven Generate\-and\-Judge Dual\-form Supervision

Fig\.[2](https://arxiv.org/html/2609.02901#S3.F2)illustrates the proposed LLM\-driven generate\-and\-judge workflow for dual\-form supervision\. Starting from an existing ASR corpus with speechxix\_\{i\}and a spoken\-form transcriptyisy\_\{i\}^\{s\}, our goal is to generate a high\-confidence written\-form counterpartyiwy\_\{i\}^\{w\}\. The workflow follows the general idea of LLM\-generated supervision and LLM\-based verification, but differs from open\-ended instruction\-data generation: the source transcript is fixed, and only local ITN\-related spans are intended to change\. LLMs are used to generate candidates and score pair quality, while lightweight deterministic sanity checks remove obvious invalid outputs before training\.

First, a guideline synthesizer operationalizes the transcription standard into a generation promptpgp\_\{g\}and a judgment promptpjp\_\{j\}, specifying how normalization\-sensitive expressions should be rewritten or preserved under the written\-form condition\. a written\-form generatorGGis a text\-only rewriter prompted to modify only ITN\-related spans and propose a candidate

y~iw=G​\(yis;pg\)\.\\tilde\{y\}\_\{i\}^\{w\}=G\(y\_\{i\}^\{s\};p\_\{g\}\)\.\(1\)Second, a pairwise quality verifierJJis a text\-only pairwise scorer that checks whether\(yis,y~iw\)\(y\_\{i\}^\{s\},\\tilde\{y\}\_\{i\}^\{w\}\)preserves the original semantics, normalizes only appropriate spans, and avoids non\-local rewriting:

si=J​\(yis,y~iw;pj\)\.s\_\{i\}=J\(y\_\{i\}^\{s\},\\tilde\{y\}\_\{i\}^\{w\};p\_\{j\}\)\.\(2\)A pair is retained if the verifier score exceeds a thresholdτ\\tau\. Before finalizing the retained set, we apply lightweight deterministic sanity checks to remove candidates with non\-ITN rewriting artifacts\. For simplicity, we denote the accepted candidatey~iw\\tilde\{y\}\_\{i\}^\{w\}asyiwy\_\{i\}^\{w\}and re\-index the retained triples as

𝒯=\{\(xi,yis,yiw\)\}i=1MT,MT=\|𝒯\|\.\\mathcal\{T\}=\\\{\(x\_\{i\},y\_\{i\}^\{s\},y\_\{i\}^\{w\}\)\\\}\_\{i=1\}^\{M\_\{T\}\},\\quad M\_\{T\}=\|\\mathcal\{T\}\|\.\(3\)
This design makes the workflow auditable: the pairwise quality verifier provides scalable quality control and checks obvious invalid samples\. Each retained example contributes two training instances,\(xi,ps,yis\)\(x\_\{i\},p^\{s\},y\_\{i\}^\{s\}\)and\(xi,pw,yiw\)\(x\_\{i\},p^\{w\},y\_\{i\}^\{w\}\), so transcript form is specified by the prompt\.

![Refer to caption](https://arxiv.org/html/2609.02901v1/x2.png)Figure 2:LLM\-driven generate\-and\-judge workflow for dual\-form supervision\. An LLM generator proposes written\-form candidates, an LLM verifier scores candidate quality, and lightweight deterministic checks remove obvious invalid or non\-ITN rewriting artifacts\.
### III\-BPrompt\-conditioned ASR\-ITN Model

As shown in Fig\.[3](https://arxiv.org/html/2609.02901#S3.F3),DF\-ASRfollows an LLM\-based ASR architecture with a speech encoder, an adaptor, and an LLM decoder\. Acoustic representations are projected into the LLM embedding space and concatenated with textual prompt embeddings\. The same model then generates either spoken\-form or written\-form transcripts according to the prompt\.

For paired supervision𝒯\\mathcal\{T\}, the dual\-form cross\-entropy \(CE\) objective is

ℒCE=−∑i=1MT∑b∈\{s,w\}∑t=1\|yib\|log⁡Pθ​\(yi,tb∣yi,<tb,xi,pb\)\.\\mathcal\{L\}\_\{\\mathrm\{CE\}\}=\-\\sum\_\{i=1\}^\{M\_\{T\}\}\\sum\_\{b\\in\\\{s,w\\\}\}\\sum\_\{t=1\}^\{\|y\_\{i\}^\{b\}\|\}\\log P\_\{\\theta\}\(y\_\{i,t\}^\{b\}\\mid y\_\{i,<t\}^\{b\},x\_\{i\},p^\{b\}\)\.\(4\)
This objective prevents the model from treating spoken and written formats as noisy alternatives under the same condition\. Instead, the target form is made explicit by the prompt, allowing shared acoustic recognition ability to support two deployment modes: faithful spoken\-form transcription and semantics\-aware written\-form ASR\-ITN\.

![Refer to caption](https://arxiv.org/html/2609.02901v1/x3.png)

Figure 3:DF\-ASRtraining with dual\-form CE and ITN\-MWER\. ITN\-MWER adds sequence\-level feedback that emphasizes numeric values and units, whose errors may cause larger semantic loss than ordinary character substitutions\. The figure shows 4 candidates for readability; training uses 16 candidates\.
### III\-CITN\-MWER for Numeric Information Loss

While dual\-form supervised fine\-tuning \(SFT\) establishes the prompt\-to\-form mapping, we further refine the model with sequence\-level optimization\. Standard CE training optimizes token\-level likelihood, and conventional minimum word error rate \(MWER\) training\[[20](https://arxiv.org/html/2609.02901#bib.bib43)\]optimizes sequence\-level character errors; however, neither explicitly accounts for the unequal semantic cost of different token types\. In ASR\-ITN, numeric errors are often more damaging than ordinary character substitutions: dropping a decimal point, changing a percentage sign, or substituting a unit can alter the transmitted value rather than only the surface wording\.

We therefore introduce ITN\-MWER, a sequence\-level objective that combines general transcription quality with numeric\-keyword correctness\. For each prompt conditionb∈\{s,w\}b\\in\\\{s,w\\\}, the model produces anNN\-best set\{y^i,nb\}n=1N\\\{\\hat\{y\}\_\{i,n\}^\{b\}\\\}\_\{n=1\}^\{N\}with normalized probabilitiesP~θ​\(y^i,nb∣xi,pb\)\\tilde\{P\}\_\{\\theta\}\(\\hat\{y\}\_\{i,n\}^\{b\}\\mid x\_\{i\},p^\{b\}\)\. The character error rate \(CER\)\-based reward used in vanilla MWER is

RCER​\(y^,y\)=1−CER​\(y^,y\)\.R\_\{\\mathrm\{CER\}\}\(\\hat\{y\},y\)=1\-\\mathrm\{CER\}\(\\hat\{y\},y\)\.\(5\)
To introduce ITN\-specific feedback, let𝒦​\(⋅\)\\mathcal\{K\}\(\\cdot\)extract numeric keywords, including Arabic numbers and their attached units or symbols\. Given a hypothesisy^\\hat\{y\}and a referenceyy, we compute matched, missing, and extra numeric keywords as TP, FN, and FP, respectively\. The numeric\-keyword reward is defined as

Rkey​\(y^,y\)=2​T​P2​T​P\+FN\+FP\.R\_\{\\mathrm\{key\}\}\(\\hat\{y\},y\)=\\frac\{2\\mathrm\{TP\}\}\{2\\mathrm\{TP\}\+\\mathrm\{FN\}\+\\mathrm\{FP\}\}\.\(6\)If neither side contains numeric keywords,RkeyR\_\{\\mathrm\{key\}\}is set to 1\. The final ITN\-MWER reward combines general transcription quality and numeric\-keyword correctness:

RITN​\(y^,y\)=α​RCER​\(y^,y\)\+β​Rkey​\(y^,y\)α\+β\.R\_\{\\mathrm\{ITN\}\}\(\\hat\{y\},y\)=\\frac\{\\alpha R\_\{\\mathrm\{CER\}\}\(\\hat\{y\},y\)\+\\beta R\_\{\\mathrm\{key\}\}\(\\hat\{y\},y\)\}\{\\alpha\+\\beta\}\.\(7\)The ITN\-MWER loss uses the average rewardR¯ib\\bar\{R\}\_\{i\}^\{b\}over theNN\-best set as a variance\-reducing baseline:

ℒITN​\-​MWER=\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{ITN\\mbox\{\-\}MWER\}\}=−∑i=1MT∑b∈\{s,w\}∑n=1NP~θ​\(y^i,nb∣xi,pb\)\\displaystyle\-\\sum\_\{i=1\}^\{M\_\{T\}\}\\sum\_\{b\\in\\\{s,w\\\}\}\\sum\_\{n=1\}^\{N\}\\tilde\{P\}\_\{\\theta\}\(\\hat\{y\}\_\{i,n\}^\{b\}\\mid x\_\{i\},p^\{b\}\)\(8\)⋅\(RITN​\(y^i,nb,yib\)−R¯ib\)\.\\displaystyle\\cdot\\left\(R\_\{\\mathrm\{ITN\}\}\(\\hat\{y\}\_\{i,n\}^\{b\},y\_\{i\}^\{b\}\)\-\\bar\{R\}\_\{i\}^\{b\}\\right\)\.The final objective retains the CE loss as a stabilizing term:

ℒ=ℒITN​\-​MWER\+λ​ℒCE\.\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{ITN\\mbox\{\-\}MWER\}\}\+\\lambda\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\.\(9\)
The CER\-based reward preserves general transcription fidelity, while the numeric\-keyword reward provides ITN\-specific feedback for normalization\-sensitive spans\. The reward is applied under both prompt conditions so that sequence\-level optimization does not collapse the dual\-form behavior into a single preferred transcript style\. Under the spoken\-form prompt, the CER term preserves faithful transcription and prevents unnecessary rewriting\. Under the written\-form prompt, the numeric\-keyword term provides additional feedback for normalization\-sensitive spans, while the CER term constrains the model from changing unrelated context\. This design aligns ITN\-MWER with the dual\-form objective, allowing sequence\-level optimization to support written\-form ASR\-ITN while maintaining spoken\-form transcription behavior\.

## IVExperimental Setup

### IV\-ATraining Data and Evaluation Set

DF\-ASRis trained from a Chinese LLM\-ASR backbone usingWenetSpeech\[[35](https://arxiv.org/html/2609.02901#bib.bib44)\], covering 14\.61M utterances and 9992\.6 hours of speech\. For dual\-form supervision, we construct high\-confidence targets through the proposed LLM\-driven generate\-and\-judge workflow\. The final dual\-form training set contains 14\.23M utterances and 9574\.7 hours, where each utterance contains both a spoken\-form target and a written\-form target\. The written\-form targets include both ITN\-required utterances and no\-number utterances whose written targets remain identical to the spoken transcripts after basic text normalization\. Among the written\-form targets, 393\.6K utterances, corresponding to 417\.2 hours, contain Arabic numerals, accounting for 4\.4% of the written\-form hours\.

For evaluation, we construct a manually annotated and reviewed Chinese ASR\-ITN benchmark fromSpeechIO\[[7](https://arxiv.org/html/2609.02901#bib.bib47)\]based on the Chinese national standard GB/T 15835–2011 for writing numerals in public texts\[[11](https://arxiv.org/html/2609.02901#bib.bib48)\]\. The benchmark separates two complementary deployment requirements\.Require\-ITNcontains 772 utterances where numeric expressions should be normalized into written form for readability\.Forbid\-ITNcontains 309 utterances with 332 manually marked forbidden spans that should preserve spoken or lexicalized form, such as idioms, fixed expressions, proper nouns, historical terms, approximate numbers, and lexicalized numeric phrases\. This split directly tests the two errors that matter in semantics\-aware written\-form ITN: missing required normalization and over\-normalizing protected spans\. We also sample 1,000 utterances without numeric spans as a no\-number control set to check whether the written\-form prompt affects ordinary ASR behavior or introduces false digit insertions when ITN is not required\.

### IV\-BSystems Compared

We compareDF\-ASRwith cascaded systems, open\-source direct ASR systems, and closed\-source models\. Cascaded denotes spoken\-form ASR followed byWeTextProcessing\. Cascaded\-LLM denotes FireRedASR2 spoken\-form ASR followed by a Qwen3\.5\-35B\-A3B post\-processor conditioned onpgp\_\{g\}\. For open\-source ASR\-ITN references, we evaluateFunASR\-NanoandWhisper\-large\-v3\.FunASR\-Nanois treated as an off\-the\-shelf promptable open ASR reference; its internal normalization policy and training recipe are not publicly specified, so we compare it by behavior under the same protocol rather than treating it as an architectural baseline\. For closed\-source models, we evaluateDoubao\-LLM ASR\[[3](https://arxiv.org/html/2609.02901#bib.bib25)\]andFun\-ASR 1\.5\[[1](https://arxiv.org/html/2609.02901#bib.bib4)\]\. Both systems are evaluated via their public APIs on June 10, 2026\.

TABLE I:Main results onRequire\-ITNandForbid\-ITN\.Require\-ITNevaluates required numeric\-expression normalization with I\-CER, NI\-CER, overall CER, and numeric keyword F1;Forbid\-ITNevaluates protected\-span preservation with CER and forbidden\-span preservation rate \(FSPR\)\.
### IV\-CImplementation Details

The transcription guideline also follows the standard document GB/T 15835–2011\[[11](https://arxiv.org/html/2609.02901#bib.bib48)\]\. Gemini 3\.0 Flash is used only to operationalize this standard into generation and judgment prompt templates\. The LLM generatorGGand verifierJJare both instantiated with Qwen3\.5\-35B\-A3B served byvLLM\[[16](https://arxiv.org/html/2609.02901#bib.bib45)\]\. The generator produces written\-form candidates from spoken\-form transcripts, while the verifier assigns a 1–10 quality score based on semantic preservation, appropriate numeric\-expression normalization, and non\-ITN text invariance\. We set the threshold toτ=9\\tau=9\. Before finalizing retained pairs, lightweight deterministic sanity checks normalize punctuation, spaces, full\-/half\-width symbols, and English letter case, and remove candidates with invalid symbols or obvious non\-ITN rewriting artifacts\. Rejected candidates are not used as written\-form supervision, since they may involve unverifiable changes beyond local numeric\-expression normalization, and automatically repairing such cases would introduce additional assumptions into the training targets\.

DF\-ASRis initialized with FireRedASR2, an LLM\-based ASR model\[[32](https://arxiv.org/html/2609.02901#bib.bib46)\], which contains a FireRedASR2 encoder, a linear adaptor with downsampling rate 2, and a Qwen2\-7B\-Instruct decoder\. The LLM decoder is frozen except for low\-rank adaptation \(LoRA\) adapters\[[15](https://arxiv.org/html/2609.02901#bib.bib49)\]applied to all modules with rank 64, scaling factor 16, and dropout 0\. The speech encoder and adaptor are trainable\. We train with AdamW, learning rate2×10−52\\times 10^\{\-5\}, warmup of 4,000 steps, dynamic length batching with batch size 18,000 tokens, bf16 mixed precision, gradient clipping at 5 for 1 epoch on 8 NVIDIA H200 GPUs \(140GB\)\. To disentangle the contributions of supervision construction and sequence\-level optimization, we conduct two groups of ablations\. For supervision construction, we train SFT\-only variants with rule\-derived targets, unverified LLM\-generated targets, and verified dual\-form targets, respectively\. These variants isolate the effect of target quality and LLM\-based verification\. We further fine\-tune the verified dual\-form SFT model with MWER using 16 candidates\. The full ITN\-MWER system sets\(α,β\)=\(0\.5,0\.5\)\(\\alpha,\\beta\)=\(0\.5,0\.5\), giving equal weight to the CER\-based reward and the numeric\-keyword reward, and sets the CE stabilizing weight toλ=0\.2\\lambda=0\.2\. We also evaluate\(α,β\)=\(1,0\)\(\\alpha,\\beta\)=\(1,0\)as vanilla MWER and\(α,β\)=\(0,1\)\(\\alpha,\\beta\)=\(0,1\)as keyword\-only MWER to isolate the effect of each reward component\. English letters are normalized to uppercase during evaluation to avoid case\-only penalties\.

### IV\-DMetrics

Following prior works\[[4](https://arxiv.org/html/2609.02901#bib.bib39),[5](https://arxiv.org/html/2609.02901#bib.bib40),[14](https://arxiv.org/html/2609.02901#bib.bib42)\], onRequire\-ITNwe report character error rate \(CER\), Inverse Character Error Rate \(I\-CER\), Non\-Inverse Character Error Rate \(NI\-CER\), and numeric keyword F1\. I\-CER measures errors on ITN\-related regions, while NI\-CER measures non\-ITN regions and is used to detect unwanted rewriting\. Keyword F1 evaluates whether numeric values and units are correctly produced\.

OnForbid\-ITN, we report CER and forbidden\-span preservation rate \(FSPR\)\. LetNspanfN\_\{\\mathrm\{span\}\}^\{f\}be the total number of forbidden spans,NitnfN\_\{\\mathrm\{itn\}\}^\{f\}the number incorrectly normalized into digit\-bearing written forms, andNothfN\_\{\\mathrm\{oth\}\}^\{f\}the number otherwise deleted, substituted, or incompletely preserved\. We define

F​S​P​R=1−Nitnf\+NothfNspanf\.FSPR\{\}=1\-\\frac\{N\_\{\\mathrm\{itn\}\}^\{f\}\+N\_\{\\mathrm\{oth\}\}^\{f\}\}\{N\_\{\\mathrm\{span\}\}^\{f\}\}\.\(10\)A higher FSPR indicates better preservation of protected spoken\-form spans\.

## VResults and Analysis

### V\-ARequired Normalization

TheRequire\-ITNcolumns in Table[I](https://arxiv.org/html/2609.02901#S4.T1)evaluate required\-normalization quality\. Compared with the WFST cascade,DF\-ASRreduces I\-CER from 8\.19% to 4\.64%, reduces overall CER from 3\.09% to 2\.35%, and improves numeric keyword F1 from 89\.71% to 94\.85%\. Compared with the LLM cascade,DF\-ASRalso achieves lower I\-CER and NI\-CER, suggesting that integrated ASR\-ITN is more stable than rewriting recognized text with a post\-processor\.

The improvement is not accompanied by increased disturbance in non\-ITN regions\. Among open\-source direct ASR systems,Whisper\-large\-v3reaches competitive keyword F1 but has much worse NI\-CER thanDF\-ASR\(5\.07% vs\. 1\.86%\), indicating that its written\-form output introduces more changes outside ITN\-related regions\. In contrast,DF\-ASRachieves strong keyword F1 while maintaining the lowest NI\-CER among all compared systems, showing a better balance between required numeric\-expression normalization and surrounding transcript fidelity\. Among closed\-source models,Doubao\-LLM ASRachieves the best overall CER and keyword F1 onRequire\-ITN, whileDF\-ASRremains competitive with these strong external references and provides an inspectable training and evaluation recipe\.

### V\-BForbidden\-span Preservation

TheForbid\-ITNcolumns in Table[I](https://arxiv.org/html/2609.02901#S4.T1)evaluate protected\-span preservation\. Although the LLM cascade improves FSPR over the WFST cascade from 25\.60% to 86\.45%, it still underperformsDF\-ASR, showing the limitation of post\-processing ITN on context\-dependent numeric expressions\.DF\-ASRreaches 95\.18% FSPR, tied for the highest preservation score in the main comparison, whileDoubao\-LLM ASRobtains the lowest CER \(3\.56%\) but lower FSPR \(93\.98%\)\. This result is noteworthy because strong industrial ASR systems may benefit from proprietary training data, product feedback, and long\-term maintenance of normalization policies, whereasDF\-ASRachieves comparable forbidden\-span preservation with a self\-contained generate–judge–filter–train pipeline\. The contrast between CER and FSPR also shows why preservation must be evaluated separately from ordinary transcription accuracy: a system can have low overall CER while still over\-normalizing semantically protected spans\.

### V\-CRequired\-vs\-Forbidden Decision Analysis

The two splits reveal complementary failure modes\.FunASR\-Nanopreserves forbidden spans by avoiding many ITN decisions, reaching 95\.18% FSPR but only 54\.63% keyword F1 onRequire\-ITN\. Conversely, the WFST cascade performs many numeric conversions but over\-normalizes protected spans\.DF\-ASRis the most balanced open system, with 94\.85% keyword F1 onRequire\-ITNand 95\.18% FSPR onForbid\-ITN, suggesting semantics\-aware ITN rather than blind conversion or blanket preservation\.

TABLE II:No\-number control set results on 1,000 utterances without numeric spans\.Table[II](https://arxiv.org/html/2609.02901#S5.T2)provides a non\-regression check on a no\-number control set under the written\-form prompt\.DF\-ASRremains close to the FireRedASR2 backbone \(3\.07% vs\. 2\.92% CER\), and both systems introduce no false digit insertion, indicating that dual\-form fine\-tuning does not substantially degrade ordinary ASR behavior and that the written\-form prompt does not trigger unconditional digit generation\.

### V\-DAblation Study

TABLE III:Ablation results for supervision sources and MWER reward settings\.Table[III](https://arxiv.org/html/2609.02901#S5.T3)separates supervision construction from sequence\-level optimization\. Under SFT\-only training, rule\-derived targets perform poorly since they inherit context\-insensitive WFST decisions\. Unverified LLM targets achieve the highest FSPR, but their I\-CER remains 6\.63%, suggesting an overly conservative behavior that preserves protected spans but still misses required normalization\.Verified dual\-form targets slightly lower FSPR but improve I\-CER and F1, showing a better balance between normalization and preservation\.

For MWER objective ablation, vanilla MWER optimizes general character\-level fidelity, while keyword\-only MWER over\-focuses on numeric tokens\. By combining both rewards, ITN\-MWER achieves the best I\-CER and keyword F1, showing that sequence\-level optimization benefits from balancing transcription fidelity and ITN\-specific feedback\.

### V\-EQualitative Analysis

TABLE IV:Dual\-form output examples with English glosses\.Table[IV](https://arxiv.org/html/2609.02901#S5.T4)gives dual\-form examples that require context\-aware normalization decisions\. Cases 1 and 3 show thatDF\-ASRdoes not simply convert every numeric\-looking span\. In Case 1, the first “十年” denotes a duration and is normalized to “10年”, whereas the second occurs in the lexicalized expression “失去的十年” and is preserved\. In Case 3, the Gregorian date is normalized as “2月15号”, while the lunar\-calendar expression “农历正月十一” remains in spoken form according to the transcription convention\.

Case 2 further illustrates compositional generalization in a complex numeric statement\. Although the exact decimal value “0\.9999995” is absent from training,DF\-ASRcorrectly produces the decimal, the power expression, and the percentage in written form\. Overall, these examples show thatDF\-ASRperforms prompt\-controlled dual\-form generation and makes context\-dependent ITN decisions rather than applying local digit replacement\.

## VIConclusion and Limitations

We presentedDF\-ASR, a Chinese ASR\-ITN framework that extends spoken\-form ASR capability to semantics\-aware written\-form ITN through paired spoken\-form and written\-form supervision\. The LLM\-driven generate\-and\-judge workflow constructs dual\-form targets from spoken\-form ASR corpora, while ITN\-MWER provides sequence\-level feedback for normalization\-sensitive spans\. TheRequire\-ITN/Forbid\-ITNprotocol separately evaluates required normalization and forbidden\-span preservation\. Experiments show thatDF\-ASRoutperforms open\-source cascaded and promptable ASR\-ITN baselines, remains competitive with strong closed\-source references, and preserves protected spans effectively under prompt\-level transcript\-form control\.

This work focuses on offline Chinese ASR\-ITN\. English or multilingual extension requires language\-specific normalization conventions and evaluation sets, and the LLM\-verified supervision may still contain residual numeric errors\.

## References

- \[1\]K\. An, Y\. Chen, Z\. Chen,et al\.\(2025\)Fun\-asr technical report\.CoRRabs/2509\.12508\.Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p2.1),[§IV\-B](https://arxiv.org/html/2609.02901#S4.SS2.p1.1)\.
- \[2\]A\. Antonova, E\. Bakhturina, and B\. Ginsburg\(2022\)Thutmose tagger: single\-pass neural model for inverse text normalization\.InProc\. of INTERSPEECH,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1)\.
- \[3\]Y\. Bai, J\. Chen, J\. Chen,et al\.\(2024\)Seed\-ASR: understanding diverse speech and contexts with LLM\-based speech recognition\.CoRRabs/2407\.04675\.Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1),[§IV\-B](https://arxiv.org/html/2609.02901#S4.SS2.p1.1)\.
- \[4\]H\. Choi, M\. Choi, Y\. Lim,et al\.\(2024\)Spoken\-to\-written text conversion with large language model\.InProc\. of INTERSPEECH,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1),[§IV\-D](https://arxiv.org/html/2609.02901#S4.SS4.p1.1)\.
- \[5\]M\. Choi, H\. Choi, Y\. Lim,et al\.\(2025\)Bidirectional spoken\-written text conversion with large language models\.InProc\. of INTERSPEECH,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1),[§IV\-D](https://arxiv.org/html/2609.02901#S4.SS4.p1.1)\.
- \[6\]DeepSeek\-AI\(2025\)DeepSeek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.CoRRabs/2501\.12948\.Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1)\.
- \[7\]J\. Du, J\. Li, G\. Chen,et al\.\(2025\)SpeechColab leaderboard: an open\-source platform for automatic speech recognition evaluation\.Computer Speech & Language94,pp\. 101805\.Cited by:[§IV\-A](https://arxiv.org/html/2609.02901#S4.SS1.p2.1)\.
- \[8\]Y\. Fathullah, C\. Wu, E\. Lakomkin,et al\.\(2024\)Prompting large language models with speech recognition abilities\.InProc\. of ICASSP,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1),[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[9\]L\. Fu, Y\. Xin, S\. Zeng,et al\.\(2026\)PAC: pronunciation\-aware contextualized large language model\-based automatic speech recognition\.InProc\. of ICASSP,Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[10\]Y\. Gaur, N\. Kibre, J\. Xue,et al\.\(2022\)Streaming, fast and accurate on\-device inverse text normalization for automatic speech recognition\.InProc\. of SLT,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1)\.
- \[11\]General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China and Standardization Administration of China\(2011\)GB/T 15835–2011: general rules for writing numerals in public texts\.Note:National Standard of the People’s Republic of ChinaCited by:[§IV\-A](https://arxiv.org/html/2609.02901#S4.SS1.p2.1),[§IV\-C](https://arxiv.org/html/2609.02901#S4.SS3.p1.3)\.
- \[12\]X\. Geng, T\. Xu, K\. Wei,et al\.\(2024\)Unveiling the potential of llm\-based ASR on chinese open\-source datasets\.InProc\. of ISCSLP,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1)\.
- \[13\]GLM\-5 Team\(2026\)GLM\-5: From Vibe Coding to Agentic Engineering\.CoRRabs/2602\.15763\.Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1)\.
- \[14\]L\. Ho, K\. Le, V\. Pham,et al\.\(2025\)Dynamic context\-aware streaming pretrained language model for inverse text normalization\.InProc\. of INTERSPEECH,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1),[§IV\-D](https://arxiv.org/html/2609.02901#S4.SS4.p1.1)\.
- \[15\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2022\)LoRA: low\-rank adaptation of large language models\.InProc\. of ICLR,Cited by:[§IV\-C](https://arxiv.org/html/2609.02901#S4.SS3.p2.5)\.
- \[16\]W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica\(2023\)Efficient memory management for large language model serving with PagedAttention\.InProc\. of SOSP,Cited by:[§IV\-C](https://arxiv.org/html/2609.02901#S4.SS3.p1.3)\.
- \[17\]H\. Liu, Y\. Wang, Z\. Cheng,et al\.\(2026\)CS3\-Bench: evaluating and enhancing speech\-to\-speech LLMs for mandarin\-english code\-switching\.InProc\. of ICASSP,Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[18\]Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu\(2023\)G\-Eval: NLG evaluation using GPT\-4 with better human alignment\.InProc\. of EMNLP,Cited by:[§II\-C](https://arxiv.org/html/2609.02901#S2.SS3.p1.1)\.
- \[19\]Z\. Ma, G\. Yang, W\. Chen,et al\.\(2026\)SLAM\-LLM: A modular, open\-source multimodal large language model framework and best practice for speech, language, audio and music processing\.IEEE J\. Sel\. Top\. Signal Process\.20\(1\),pp\. 63–76\.Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1),[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[20\]Z\. Meng, Y\. Wu, N\. Kanda,et al\.\(2021\)Minimum word error rate training with language model fusion for end\-to\-end speech recognition\.InProc\. of INTERSPEECH,Cited by:[§III\-C](https://arxiv.org/html/2609.02901#S3.SS3.p1.1)\.
- \[21\]OpenAI\(2023\)GPT\-4 technical report\.CoRRabs/2303\.08774\.Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1)\.
- \[22\]Qwen Team\(2026\)Qwen3\-asr technical report\.CoRRabs/2601\.21337\.Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[23\]A\. Radford, J\. W\. Kim, T\. Xu,et al\.\(2023\)Robust speech recognition via large\-scale weak supervision\.InProc\. of ICML,Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p2.1)\.
- \[24\]M\. Shi, X\. Xiao, R\. Fan,et al\.\(2026\)Train short, infer long: Speech\-LLM enables zero\-shot streamable joint ASR and diarization on long audio\.InProc\. of ICASSP,Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[25\]M\. Sunkara, C\. Shivade, S\. Bodapati,et al\.\(2021\)Neural inverse text normalization\.InProc\. of ICASSP,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1)\.
- \[26\]S\. Tan, P\. Behre, N\. Kibre,et al\.\(2023\)Four\-in\-one: a joint approach to inverse text normalization, punctuation, capitalization, and disfluency for automatic speech recognition\.InProc\. of SLT,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1)\.
- \[27\]K\. Team\(2026\)Kimi K2\.5: visual agentic intelligence\.CoRRabs/2602\.02276\.Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1)\.
- \[28\]Q\. Team\(2025\)Qwen3 technical report\.CoRRabs/2505\.09388\.Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1)\.
- \[29\]Y\. Wang, Y\. Kordi, S\. Mishra, A\. Liu, N\. A\. Smith, D\. Khashabi, and H\. Hajishirzi\(2023\)Self\-instruct: aligning language models with self\-generated instructions\.InProc\. of ACL,Cited by:[§II\-C](https://arxiv.org/html/2609.02901#S2.SS3.p1.1)\.
- \[30\]WeNet Team\(2024\)WeTextProcessing: chinese text normalization and inverse text normalization toolkit\.Note:[https://github\.com/wenet\-e2e/WeTextProcessing](https://github.com/wenet-e2e/WeTextProcessing)GitHub repositoryCited by:[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1)\.
- \[31\]J\. Wu, Y\. Gaur, Z\. Chen,et al\.\(2023\)On decoder\-only architecture for speech\-to\-text and large language model integration\.InProc\. of ASRU,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p1.1),[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[32\]K\. Xu, Y\. Jia, K\. Huang,et al\.\(2026\)FireRedASR2S: a state\-of\-the\-art industrial\-grade all\-in\-one automatic speech recognition system\.CoRRabs/2603\.10420\.Cited by:[§IV\-C](https://arxiv.org/html/2609.02901#S4.SS3.p2.5)\.
- \[33\]G\. Yang, Z\. Ma, F\. Yu,et al\.\(2024\)MaLa\-ASR: multimedia\-assisted LLM\-based ASR\.InProc\. of INTERSPEECH,Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[34\]H\. Yin, Y\. Chen, C\. Deng,et al\.\(2026\)SpeakerLM: end\-to\-end versatile speaker diarization and recognition with multimodal large language models\.InProc\. of AAAI,Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[35\]B\. Zhang, H\. Lv, P\. Guo,et al\.\(2022\)WenetSpeech: a 10000\+ hours multi\-domain Mandarin corpus for speech recognition\.InProc\. of ICASSP,Cited by:[§IV\-A](https://arxiv.org/html/2609.02901#S4.SS1.p1.1)\.
- \[36\]F\. Zhang, W\. Geng, H\. Huang,et al\.\(2025\)Boosting code\-switching ASR with mixture of experts enhanced speech\-conditioned LLM\.InProc\. of ICASSP,Cited by:[§II\-A](https://arxiv.org/html/2609.02901#S2.SS1.p1.1)\.
- \[37\]Y\. Zhang, E\. Bakhturina, K\. Gorman, and B\. Ginsburg\(2021\)NeMo \(inverse\) text normalization: from development to production\.InProc\. of INTERSPEECH,Cited by:[§I](https://arxiv.org/html/2609.02901#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.02901#S2.SS2.p1.1)\.
- \[38\]L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica\(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and chatbot arena\.InProc\. of NeurIPS,Cited by:[§II\-C](https://arxiv.org/html/2609.02901#S2.SS3.p1.1)\.