基于预训练文本到文本语言模型和读-标记-写策略的高效同传逆文本规范化研究

# Towards Efficient Simultaneous Inverse Text Normalization with Pretrained Text-to-Text Language Model and Read–Tag–Write Policy
Source: [https://arxiv.org/html/2610.19492](https://arxiv.org/html/2610.19492)
Hoang Le Nguyen Pham Vo Tran Nguyen Nguyen Le

###### Abstract

Streaming Inverse Text Normalization \(ITN\) is vital for converting spoken\-form outputs from streaming Automatic Speech Recognition into formatted written text\. Existing streaming ITN methods rely on hybrid systems combining neural tagging with expert\-crafted finite\-state transducer rules, limiting scalability across domains and languages\. While end\-to\-end models offer superior scalability by learning directly from data, standard encoder\-decoder architectures are inherently non\-streaming due to global attention\. In this paper, we propose an efficient streaming end\-to\-end ITN system, adapting a pretrained text\-to\-text model to leverage its robust linguistic knowledge\. To enable streaming, we introduce architectural adaptations, a specialized training strategy, a Read–Tag–Write decoding policy, and inference optimizations\. Experiments on Vietnamese datasets show accuracy comparable to non\-streaming baselines, outperforming hybrid methods while satisfying real\-time latency requirements\.

###### keywords

inverse text normalization, speech recognition, streaming, end\-to\-end

††address:1UNEY, Switzerland††email:kiet\.ha@uney\.com, khanh\.ld@uney\.com## 1Introduction

Inverse Text Normalization \(ITN\) converts spoken\-form transcriptions produced by Automatic Speech Recognition \(ASR\) systems into readable written\-form representations\[[1](https://arxiv.org/html/2610.19492#bib.bib1)\]\. While ASR models typically train on verbalized labels to maintain phonetic alignment with audio, their outputs often lack readability and impair downstream tasks such as machine translation and natural language understanding\. ITN addresses this by restoring punctuation, applying capitalization, and performing phrase\-level transformations, including semiotic and phonetic phrases\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\]\. For example, it converts expressions like “one point two dollars” to “$1\.20”\. While spoken and written forms are frequently identical, certain structured phrases differ substantially and require contextual reasoning to resolve ambiguities\. This makes ITN inherently context\-dependent and necessitates robust contextual representations to ensure accurate disambiguation across diverse linguistic patterns\[[3](https://arxiv.org/html/2610.19492#bib.bib3)\]\.

Recent advances in ITN have focused on end\-to\-end approaches, which achieve strong performance across languages and domains\[[1](https://arxiv.org/html/2610.19492#bib.bib1)\]by implicitly learning patterns from data, thereby offering greater scalability and enabling effective model compression\[[3](https://arxiv.org/html/2610.19492#bib.bib3)\]\. These methods primarily use sequence\-to\-sequence \(seq2seq\) architectures to map spoken\-form inputs directly to written\-form outputs\[[1](https://arxiv.org/html/2610.19492#bib.bib1),[2](https://arxiv.org/html/2610.19492#bib.bib2),[4](https://arxiv.org/html/2610.19492#bib.bib4),[5](https://arxiv.org/html/2610.19492#bib.bib5),[6](https://arxiv.org/html/2610.19492#bib.bib6)\]\. However, standard seq2seq models often produce unrecoverable hallucination errors, yielding fluent yet semantically incorrect results that alter original meaning\[[1](https://arxiv.org/html/2610.19492#bib.bib1)\]\. Additionally, processing full sentences increases computational cost and risks unnecessary changes to unchanged tokens\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\]\. Hybrid methods alleviate these problems by combining neural tagging models with finite\-state transducer \(FST\) rules\[[7](https://arxiv.org/html/2610.19492#bib.bib7),[8](https://arxiv.org/html/2610.19492#bib.bib8),[9](https://arxiv.org/html/2610.19492#bib.bib9)\]\. They excel at preserving unmodified text but depend on expert\-crafted FST grammars and extensive inside\-outside\-beginning \(IOB\) tag annotation for each ITN category\. This dependence limits scalability and performance improvement due to the substantial manual effort required for IOB labeling and FST rule creation\[[3](https://arxiv.org/html/2610.19492#bib.bib3)\]\. Pretrained language models\[[1](https://arxiv.org/html/2610.19492#bib.bib1),[6](https://arxiv.org/html/2610.19492#bib.bib6)\]and data\-driven techniques\[[5](https://arxiv.org/html/2610.19492#bib.bib5),[10](https://arxiv.org/html/2610.19492#bib.bib10)\]have also been explored to address data scarcity, improve generalization, and enhance ITN performance across diverse languages and domains\.

Despite advances in offline ITN, simultaneous or streaming ITN remains underexplored yet essential for real\-time applications such as live ASR, meeting transcription, and speech translation, where formatted text must be displayed instantaneously to preserve conversational flow\. Industrial systems often rely on cascaded architectures, applying incremental punctuation and casing models as post\-processing steps\[[7](https://arxiv.org/html/2610.19492#bib.bib7),[8](https://arxiv.org/html/2610.19492#bib.bib8)\]\. While these modular pipelines provide stability, they struggle with complex semiotic ambiguities that require joint modeling of capitalization, punctuation, and structural transformations\. Pioneering work in this area\[[3](https://arxiv.org/html/2610.19492#bib.bib3)\]introduced a hybrid streaming solution employing a chunk\-based Transformer\[[11](https://arxiv.org/html/2610.19492#bib.bib11)\]tagger to detect ITN categories in incoming ASR tokens, followed by category\-specific FSTs for targeted conversion\. A subsequent study\[[12](https://arxiv.org/html/2610.19492#bib.bib12)\]adapted a pretrained language model with dynamic, context\-aware processing to handle variable ASR chunk sizes and accumulated tokens, achieving low latency and accuracy comparable to non\-streaming ITN on a limited dataset\. However, current streaming methods rely exclusively on hybrid architectures and inherit limited scalability due to the need for manual FST rule development and extensive IOB labeling\. While seq2seq models offer greater scalability, they cannot operate in a streaming fashion because their encoder\-decoder architectures rely on global cross\-attention, which prevents incremental processing\.

![Refer to caption](https://arxiv.org/html/2610.19492v1/img/ITN-streaming-5.png)Figure 1:Overview of the proposed streaming end\-to\-end ITN framework using the Read–Tag–Write policy\. The streaming encoder utilizes Dynamic Right Context masking to process incoming chunks of spoken\-form tokens, while a joint tagger predicts \{B, I, O\} labels\. Verbatim tokens \(O\) are passed through instantly to minimize latency\. Upon the completion of a normalization span, the autoregressive decoder is selectively triggered, generating normalized text by cross\-attending to the relevant encoder hidden states \(H0:mH\_\{0:m\}\) and conditioning on the written\-form prefixy^p​r​e\\hat\{y\}\_\{pre\}\.To adapt offline seq2seq models for streaming inference, the field of Simultaneous Machine Translation \(SiMT\) has introduced various techniques to enable incremental processing\[[13](https://arxiv.org/html/2610.19492#bib.bib13)\]\. A core component of SiMT is the read\-write policy, which decides whether to wait for new source tokens \(read\) or generate target tokens \(write\) to balance quality and latency\. For instance, the wait\-kkpolicy\[[14](https://arxiv.org/html/2610.19492#bib.bib14),[15](https://arxiv.org/html/2610.19492#bib.bib15),[16](https://arxiv.org/html/2610.19492#bib.bib16)\]enforces a fixed lag by alternatingwrite/readoperations after an initialkk\-token delay, while monotonic attention mechanisms\[[17](https://arxiv.org/html/2610.19492#bib.bib17)\]restrict attention to left\-to\-right alignments\. Other adaptive strategies involve learned thresholds for dynamic read\-write decisions\[[18](https://arxiv.org/html/2610.19492#bib.bib18)\], Hidden Markov Transformers for simultaneous generation\[[19](https://arxiv.org/html/2610.19492#bib.bib19)\], or leveraging large language models as policy\-makers\[[20](https://arxiv.org/html/2610.19492#bib.bib20)\]\. However, while effective for translation, these strategies introduce redundant computation for ITN\. In SiMT, the reordering problem represents a primary conflict between translation quality and real\-time latency\. Unlike SiMT, which requires global structural restructuring, ITN is characterized by local transformations that maintain the original word order\. Because ASR outputs are largely verbatim, the fixed lags and generative overhead typical of SiMT policies may be counterproductive for ITN\. Consequently, the applicability of complex SiMT scheduling to the localized domain of ITN remains unexplored\.

In this paper, we propose, to the best of our knowledge, the first streaming end\-to\-end ITN model\. Our approach achieves accuracy comparable to non\-streaming end\-to\-end baselines, outperforms streaming hybrid methods, and eliminates the need for manual FST rules and IOB labeling\. Our experiments are conducted on Vietnamese, a challenging language for ITN due to its complex morphological nuances and frequent phonetic borrowing\. By relying strictly on standard text\-to\-text foundations rather than language\-specific heuristics, our approach ensures cross\-linguistic generalization and is compatible with any seq2seq architecture\. Our main contributions include:

- •A recipe for making pretrained seq2seq models truly streamable:We present a practical set of techniques: architectural changes, a streaming\-aware multi\-task objective, and a novel data augmentation technique\. That together enable end\-to\-end ITN with strict left\-to\-right processing\.
- •An efficient Read–Tag–Write decoding policy with KV Caching and Copy–Paste Strategy:We introduce a simultaneous inference procedure that \(i\) passes through non\-normalized tokens with minimal delay and \(ii\) triggers generation only upon completing a normalization span, further accelerated by KV caching and a Copy–Paste strategy that keeps historical context and removes redundant computation\.
- •Comprehensive empirical evaluation:We rigorously evaluate our approach on a newly curated 5M\-sentence Vietnamese dataset featuring automated alignments and boundary tags, verifying its highly efficient performance\.

## 2Streaming End\-to\-End Inverse Text Normalization

In this section, we explain the chosen pretrained text\-to\-text models and describe our proposed framework for streaming end\-to\-end ITN\.

### 2\.1Pretrained Text\-to\-Text Models

Pretrained text\-to\-text models such as BART\[[21](https://arxiv.org/html/2610.19492#bib.bib21)\]and T5\[[22](https://arxiv.org/html/2610.19492#bib.bib22)\]have advanced seq2seq learning by unifying diverse NLP tasks under a generative framework\. Unlike encoder\-only architectures, these models utilize a complete Transformer encoder\-decoder structure where both inputs and outputs are represented as text strings\. BART\[[21](https://arxiv.org/html/2610.19492#bib.bib21)\]is pretrained using a denoising objective including token masking, deletion, and sentence permutation to learn robust bidirectional encoder representations and autoregressive decoder capabilities\. Similarly, T5\[[22](https://arxiv.org/html/2610.19492#bib.bib22)\]employs a span\-corruption objective on the Colossal Clean Crawled Corpus \(C4\), facilitating effective transfer learning for downstream tasks such as translation and summarization\. Extensive empirical evidence indicates that leveraging these pretrained foundations significantly improves performance on downstream tasks compared to training from scratch, especially in low\-resource or domain\-specific scenarios\[[21](https://arxiv.org/html/2610.19492#bib.bib21),[22](https://arxiv.org/html/2610.19492#bib.bib22),[23](https://arxiv.org/html/2610.19492#bib.bib23)\]\.

For the Vietnamese language, EnViT5111[https://huggingface\.co/VietAI/envit5\-base](https://huggingface.co/VietAI/envit5-base)\[[24](https://arxiv.org/html/2610.19492#bib.bib24)\]adapts the T5\-base architecture to a bilingual setting\. Following the standard configuration, the model features 12 layers in both the encoder and decoder, a hidden dimension of 768, 12 attention heads, and a gated\-GELU feed\-forward projection with an intermediate dimension of 2048, totaling approximately 275 million parameters\. EnViT5 was pretrained on a massive bilingual corpus, including an 80GB filtered subset of CC100\[[25](https://arxiv.org/html/2610.19492#bib.bib25)\], using a 15% span\-corruption objective for both English and Vietnamese\. It utilizes a shared SentencePiece tokenizer\[[26](https://arxiv.org/html/2610.19492#bib.bib26)\]with a 50,048\-subword vocabulary specifically optimized for the morphological nuances of the Vietnamese language\. EnViT5 achieves superior performance on Vietnamese\-English translation tasks on the PhoMT benchmark\[[27](https://arxiv.org/html/2610.19492#bib.bib27)\], outperforming larger models such as mBART\[[24](https://arxiv.org/html/2610.19492#bib.bib24)\]\. Given its superior semantic alignment and generative efficiency, we adopt EnViT5 as the foundation for our end\-to\-end ITN system, subsequently adapting the architecture to satisfy streaming requirements\.

### 2\.2Model Architecture

The proposed streaming end\-to\-end ITN model, illustrated in Fig\.[1](https://arxiv.org/html/2610.19492#S1.F1), is built upon the pretrained EnViT5 architecture\. It comprises a streaming\-capable encoder and an autoregressive decoder, optimized for real\-time inverse text normalization\.

#### 2\.2\.1Streaming Encoder with Joint Tagger

To support streaming inference within the encoder, we employ the Dynamic Right Context \(DRC\) masking technique\[[28](https://arxiv.org/html/2610.19492#bib.bib28)\]during training\. Specifically, we utilize an attention mask to incorporate variable chunk sizes and dynamic look\-ahead while maintaining access to the full historical left context\. By varying these parameters during training, the masking strategy effectively simulates the restricted future context inherent in real\-time processing, ensuring robustness to diverse latency constraints during inference\.

For an input sequenceX=\{x0,…,xn−1\}X=\\\{x\_\{0\},\\dots,x\_\{n\-1\}\\\}, the encoder generates hidden representationsH∈ℝn×dH\\in\\mathbb\{R\}^\{n\\times d\}through Transformer layers constrained by the DRC mask, wheredddenotes the hidden dimensionality of the model\. A linear tagging layer \(the Tagger module as illustrated in Fig\.[1](https://arxiv.org/html/2610.19492#S1.F1)\), is integrated atop these hidden representations to facilitate on\-the\-fly detection of normalization spans \(e\.g\., punctuation, capitalization, or numerical expressions\)\. Following\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\], the tagger classifies each tokenxix\_\{i\}using a\{B, I, O\}\\\{\\text\{B, I, O\}\\\}\(Beginning, Inside, Outside\) labeling scheme\. The predicted tagt^i\\hat\{t\}\_\{i\}is determined by:

t^i=argmax​\(Hi​Wt​a​g\),Wt​a​g∈ℝd×3\\hat\{t\}\_\{i\}=\\text\{argmax\}\(H\_\{i\}W\_\{tag\}\),\\quad W\_\{tag\}\\in\\mathbb\{R\}^\{d\\times 3\}\(1\)These predicted tags serve as the control signals for the Read–Tag–Write policy \(described in Section[2\.3](https://arxiv.org/html/2610.19492#S2.SS3)\)\. Unlike hybrid approaches\[[3](https://arxiv.org/html/2610.19492#bib.bib3),[12](https://arxiv.org/html/2610.19492#bib.bib12)\]that require fine\-grained tags for various ITN categories to trigger specific FST rules, our tagger utilizes only 3 classes\. This reduced label space simplifies classification and leads to more robust span detection compared to complex multi\-class systems\.

#### 2\.2\.2Autoregressive Decoder and Training Objective

During training, the decoder generates written\-form tokensyty\_\{t\}by cross\-attending to the encoder representationsHH, conditioned on the ground\-truth previous tokensy<ty\_\{<t\}\. While the encoder is constrained for streaming, the decoder maintains its standard autoregressive structure\. The probability distribution over the vocabulary at each step is computed as:

P⁡\(yt\|y<t,H\)=softmax​\(et​Wf​f​n⊤\)P\(y\_\{t\}\|y\_\{<t\},H\)=\\text\{softmax\}\(e\_\{t\}W\_\{ffn\}^\{\\top\}\)\(2\)whereet∈ℝde\_\{t\}\\in\\mathbb\{R\}^\{d\}is the decoder hidden state at steptt,Wf​f​n∈ℝV×dW\_\{ffn\}\\in\\mathbb\{R\}^\{V\\times d\}is the vocabulary projection matrix, andVVdenotes the vocabulary size\.

The model is optimized using a multi\-task objective\. We defineℒt​a​g\\mathcal\{L\}\_\{tag\}as the cross\-entropy loss for span detection andℒg​e​n\\mathcal\{L\}\_\{gen\}as the sequence generation loss:

ℒt​a​g=−1∑b=1BNb∑b=1B∑i=1NblogP\(tb,i∣xb,i\)\\mathcal\{L\}\_\{tag\}=\-\\frac\{1\}\{\\sum\_\{b=1\}^\{B\}N\_\{b\}\}\\sum\_\{b=1\}^\{B\}\\sum\_\{i=1\}^\{N\_\{b\}\}\\log P\(t\_\{b,i\}\\mid x\_\{b,i\}\)\(3\)
ℒg​e​n=−1∑b=1BTb∑b=1B∑t=1TblogP\(yb,t∣yb,<t,Hb\)\\mathcal\{L\}\_\{gen\}=\-\\frac\{1\}\{\\sum\_\{b=1\}^\{B\}T\_\{b\}\}\\sum\_\{b=1\}^\{B\}\\sum\_\{t=1\}^\{T\_\{b\}\}\\log P\(y\_\{b,t\}\\mid y\_\{b,<t\},H\_\{b\}\)\(4\)Here,BBdenotes the batch size, whileNbN\_\{b\}andTbT\_\{b\}represent the source and target sequence lengths for thebb\-th sample, respectively\. The termstb,it\_\{b,i\}andyb,ty\_\{b,t\}denote the ground\-truth tag at stepiiand the target token at steptt\. The probabilityP⁡\(tb,i∣xb,i\)P\(t\_\{b,i\}\\mid x\_\{b,i\}\)is computed by applying the softmax function to the tagger logitsHb,i​Wt​a​gH\_\{b,i\}W\_\{tag\}\.

The overall objective is a combination of these components:

ℒ=ℒt​a​g\+ℒg​e​n\\mathcal\{L\}=\\mathcal\{L\}\_\{tag\}\+\\mathcal\{L\}\_\{gen\}\(5\)

### 2\.3Read–Tag–Write Policy

To enable efficient streaming inference, we propose the Read–Tag–Write \(RTW\) policy\. Unlike SiMT policies that manage global structural reordering\[[13](https://arxiv.org/html/2610.19492#bib.bib13)\], RTW exploits the local, order\-preserving nature of ITN to coordinate the interaction between the streaming encoder and the autoregressive decoder, ensuring that generative normalization occurs only when necessary\. The policy operates in three distinct phases:

1. \[leftmargin=\*\]
2. 1\.Read:The system receives incoming ASR tokens in discrete chunks\. LetCk=\{xi,…,xi\+c−1\}C\_\{k\}=\\\{x\_\{i\},\\dots,x\_\{i\+c\-1\}\\\}be thekk\-th chunk of sizecc\. The encoder, constrained by the DRC mask, processesCkC\_\{k\}by leveraging the cached historical context and a limited look\-ahead ofrrtokens to compute hidden representationsHi:i\+c−1H\_\{i:i\+c\-1\}, which are appended to the global hidden state sequenceH0:i\+c−1H\_\{0:i\+c\-1\}\.
3. 2\.Tag:A joint tagging layer performs token\-level classification on the hidden states of the newly arrived chunk\. For each tokenxj∈Ckx\_\{j\}\\in C\_\{k\}, the model predicts a tagt^j∈\{B,I,O\}\\hat\{t\}\_\{j\}\\in\\\{B,I,O\\\}as: t^j=argmax​\(Hj​Wt​a​g\)\\hat\{t\}\_\{j\}=\\text\{argmax\}\(H\_\{j\}W\_\{tag\}\)\(6\)The policy identifies transformation spans using\{B,I\}\\\{B,I\\\}labels\. A span \(e\.g\., a singleBBor aBB\-IIsequence\) is considered complete only when the tagger encounters a tagt^j\+1∈\{B,O\}\\hat\{t\}\_\{j\+1\}\\in\\\{B,O\\\}or a sentence boundary\. Ift^j\+1=I\\hat\{t\}\_\{j\+1\}=I, the token is treated as a continuation of the current span and the system continues to “Read” without triggering the “Write” phase\.
4. 3\.Write:Unlike standard seq2seq models that decode the entire sequence, the RTW policy invokes the decoder selectively based on predicted tags: - •Verbatim Pass\-through:For tokens tagged asOO, the system directly emits the spoken\-form token\. This ensures minimal latency and preserves the integrity of verbatim segments by bypassing the generative decoder\. - •Selective Decoding:When aB−IB\-Ispan \(or a sequence of consecutive spans\) is completed, the decoder is triggered\. To optimize throughput, multiple consecutive spans within a chunk are concatenated and processed in a single decoder invocation\. Letmmbe the index of the final token in the last completed span\. The decoder generates the written\-form segmenty^l​o​c​a​l\\hat\{y\}\_\{local\}by cross\-attending to the encoder hidden statesH0:mH\_\{0:m\}: P\(y^l​o​c​a​l∣y^p​r​e,H0:m\)=∏u=1UP\(yu∣\[y^p​r​e;y<u\],H0:m\)P\(\\hat\{y\}\_\{local\}\\mid\\hat\{y\}\_\{pre\},H\_\{0:m\}\)=\\prod\_\{u=1\}^\{U\}P\(y\_\{u\}\\mid\[\\hat\{y\}\_\{pre\};y\_\{<u\}\],H\_\{0:m\}\)\(7\)where\[⋅;⋅\]\[\\cdot;\\cdot\]denotes the concatenation operator,y^p​r​e\\hat\{y\}\_\{pre\}represents the written\-form prefix generated in previous steps, andyuy\_\{u\}is theuu\-th token of the local sequence of lengthUU\. Crucially, cross\-attention is restricted toH0:mH\_\{0:m\}, ensuring look\-ahead context does not influence the current transformation\. This constraint prevents the decoder from over\-generating or emitting extraneous tokens beyond the target span\. The decoder generates tokens until an<EOS\>token is predicted, signaling the end of the local transformation\. To prevent infinite decoding loops in edge cases where the model fails to emit an<EOS\>token, we implement a forced hard\-stop that truncates generation at a maximum ofLmax\_localL\_\{\\text\{max\\\_local\}\}tokens per local span\.

### 2\.4Inference Optimization via Copy–Paste Strategy and KV Caching

Encoder KV Caching:To maximize efficiency, we implement an incremental Key\-Value \(KV\) cache for the encoder, enabling the reuse of computations from previous chunks\. Similarly, we cache the encoder hidden states and the previously decoded written\-form prefix\. When the decoder is triggered, the entire cross\-attention and self\-attention KV states need to be updated to incorporate the newly computed encoder states\. We perform an initial single batched forward pass through the decoder to populate the self\-attention and cross\-attention KV caches using the existing prefix, bypassing the need for autoregressive decoding during this initialization phase\. Subsequent tokens are then produced via standard autoregressive generation\. These optimizations significantly improve inference throughput and minimize latency\. Copy–Paste Strategy:In contrast to\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\], which completely excludes historical context from the autoregressive decoder, we argue that such context is vital for accurate normalization\. Since spoken and written forms are frequently identical in ITN, theOOtag is the most prevalent\. Our RTW policy leverages this by bypassing the decoder for these tokens and copying them directly to the output\. Crucially, these skipped tokens are still incorporated into the historical context to guide the decoding ofB−IB\-Itokens\. This approach not only reduces computational overhead and mitigates hallucinations in verbatim segments but also allows the decoder to maintain the full historical context efficiently\. Decoder KV Caching:To mitigate the computational overhead inherent in the Copy–Paste mechanism and autoregressive decoding, where standard processing would redundantly recompute the entire history for both copied and emitted tokens, we implement the “fully streaming” version by enabling the decoder KV Caching for both self\- and cross\-attention\. This allows the model to reuse states from previous invocations and perform updates only for new tokens\. While this eliminates initial passes, it introduces a training\-inference mismatch: training uses the full target prefix, whereas streaming inference updates states incrementally without re\-attending to past tokens under new encoder states\. Our experiments show this results in only a marginal performance fluctuation compared to the first strategy\.

### 2\.5Prefix\-based Training Augmentation

Algorithm 1Prefix\-based Training Augmentation0:Training pair

\(𝐱0:N−1,𝐲0:M−1\)\(\\mathbf\{x\}\_\{0:N\-1\},\\mathbf\{y\}\_\{0:M\-1\}\), probability

ptruncp\_\{\\text\{trunc\}\}
1:

r∼Uniform​\(0,1\)r\\sim\\text\{Uniform\}\(0,1\)
2:if

r<ptruncr<p\_\{\\text\{trunc\}\}then\{Truncated case\}

3:Sample

k∼Uniform​\{1,…,N−1\}k\\sim\\text\{Uniform\}\\\{1,\\dots,N\-1\\\}
4:

𝐱′←x0:k−1\\mathbf\{x\}^\{\\prime\}\\leftarrow x\_\{0:k\-1\}\{No source\-side<EOS\>\}

5:

𝐲′←\[y0:l−1;<EOS\>\]\\mathbf\{y\}^\{\\prime\}\\leftarrow\[y\_\{0:l\-1\};\\text\{\{<EOS\>\}\}\]\{

y0:l−1y\_\{0:l\-1\}is the target aligned with

x0:k−1x\_\{0:k\-1\}\}

6:else\{Full\-sequence case\}

7:

𝐱′←\[𝐱0:N−1;<EOS\>\]\\mathbf\{x\}^\{\\prime\}\\leftarrow\[\\mathbf\{x\}\_\{0:N\-1\};\\text\{\{<EOS\>\}\}\]
8:

𝐲′←\[𝐲0:M−1;<EOS\>\]\\mathbf\{y\}^\{\\prime\}\\leftarrow\[\\mathbf\{y\}\_\{0:M\-1\};\\text\{\{<EOS\>\}\}\]
9:endif

10:return

\(𝐱′,𝐲′\)\(\\mathbf\{x\}^\{\\prime\},\\mathbf\{y\}^\{\\prime\}\)

The RTW policy requires the decoder to emit an<EOS\>token to terminate each local normalization span\. Standard training, however, typically conditions<EOS\>prediction on a global source\-side boundary\. This creates a train\-test mismatch during streaming inference, where local spans must terminate without observing a global<EOS\>or future context\. Simply appending an<EOS\>token to source chunks during inference is unsuitable, as it would invalidate the incremental KV cache through self\- and cross\-attention and exacerbate distribution shifts\.

To bridge this train\-test discrepancy, we introducePrefix\-based Training Augmentation\(Alg\.[1](https://arxiv.org/html/2610.19492#alg1)\), a computationally efficient method applied on\-the\-fly during training\. We stochastically truncate training examples in an aligned manner with a probabilityptruncp\_\{\\text\{trunc\}\}\. In these instances, the source\-side<EOS\>is omitted, while the target\-side prefix is explicitly terminated with an<EOS\>token\. This strategy compels the model to learn that any valid completion of a normalization span, even when occurring mid\-sentence, must trigger termination based solely on the current partial encoder hidden states\. By simulating these artificial boundaries, the model becomes robust to the truncated inputs encountered during streaming inference\. Consequently, the decoder can reliably terminate generation without compromising the incremental KV cache, ensuring the system remains responsive to subsequent incoming chunks\.

### 2\.6Data Construction

We construct our dataset from thebinhvq222[https://github\.com/binhvq/news\-corpus](https://github.com/binhvq/news-corpus)news corpus, a large\-scale collection containing approximately 6 billion words across diverse domains\. Using the proposed data generation pipeline, we produce 5M training sentence pairs, 50k validation pairs, and 50k test pairs\. In addition to spoken\-written text pairs, the dataset provides deterministic phrase\-level alignments between spoken\-form inputs and written\-form targets, as well as token\-level normalization boundary annotations\. For span detection, each token is labeled using the IOB tagging scheme to identify segments requiring normalization\. These spans encompass the entire affected word\. For instance, a trailing punctuation mark is grouped within the preceding word’s span\. These annotations facilitate both prefix\-based training augmentation and auxiliary tagging supervision\. Categories: We define a comprehensive set of ITN categories that reflect common phenomena in ASR outputs\. These categories include punctuation restoration, capitalization, semiotic normalization, and phonetic normalization\. For punctuation restoration, we consider four standard punctuation marks: comma, period, question mark, and exclamation mark\[[12](https://arxiv.org/html/2610.19492#bib.bib12)\]\. Semiotic normalization covers entities such as numbers, dates, times, fractions, currencies, digit sequences, and electronic addresses, while phonetic normalization targets out\-of\-vocabulary \(OOV\) expressions, including foreign words, named entities, domain names, and letter sequences\. Pipeline: The dataset is generated through an automated data generation pipeline that builds upon the data preparation framework introduced in\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\], with substantial extensions to support a broader range of ITN categories\. Specifically, we define a comprehensive set of regular expressions \(regex\) to identify normalization regions in written\-form text:

- •Capitalization and punctuation restoration:Spoken\-form inputs are generated by removing all punctuation marks and converting the text to lowercase\.
- •Semiotic normalization:Each class \(e\.g\., cardinal/ordinal numbers, decimals, currencies, time expressions, and factions\) is identified using dedicated regex patterns\. Since these expressions are detected in written form, rule\-based transformations generate corresponding spoken\-form candidates\. Recognizing that a single written entity may have multiple valid spoken realizations, the pipeline randomly selects one candidate during data generation to reflect natural spoken variability\. To ensure high quality, we manually inspect representative samples from each semiotic class to verify the correctness and coverage of the transformation rules\.
- •Phonetic normalization:We define phonetic phrases as tokens that are neither semiotic expressions nor present in the ASR vocabulary\. This allows us to extract written\-form phonetic entities, such as foreign named entities and domain\-specific terms\. We then perform manual verification of an automated grapheme\-to\-phoneme output to construct a dedicated phonetic dictionary comprising approximately 125k common foreign and OOV terms\. This dictionary is subsequently used to map written\-form phonetic phrases back to their spoken\-form counterparts during dataset generation\.

Through this regex\-driven transformation process, the pipeline deterministically constructs phrase\-level spoken\-written alignments and assigns specific category labels to each normalization segment, providing precise supervision for both prefix\-based training augmentation and the auxiliary tagging task for identifying normalization boundaries\.

## 3Experimental Setup

### 3\.1Datasets

For training and evaluation, we utilize the dataset described in Section[2\.6](https://arxiv.org/html/2610.19492#S2.SS6)\. For hyperparameter tuning, we use a small subset of 500k samples from the training set and evaluate performance on the validation set\. To better assess the system’s real\-world performance, we construct an additional test set by passing all 50k samples from the original test set through the Microsoft Edge TTS system333[https://github\.com/rany2/edge\-tts](https://github.com/rany2/edge-tts)to synthesize speech\. The generated audio is then processed by the ChunkFormer ASR model444[https://huggingface\.co/khanhld/chunkformer\-ctc\-large\-vie](https://huggingface.co/khanhld/chunkformer-ctc-large-vie)\[[29](https://arxiv.org/html/2610.19492#bib.bib29)\]to produce ASR transcripts\. The word error rate \(WER\) of this ASR output relative to the reference transcripts \(spoken text\) is14\.21%14\.21\\%\. This error rate reflects typical ASR artifacts, such as misrecognitions and phonetic ambiguities\. These issues are especially pronounced because the dataset contains numerous OOV and foreign words, providing a challenging and realistic input stream\. This ASR\-generated text serves as the input for evaluating the robustness of our ITN model in a real\-world deployment scenario, following the evaluation protocol in\[[3](https://arxiv.org/html/2610.19492#bib.bib3)\]\.

### 3\.2Implementation Details

Training:Models are trained for up to 20 epochs using 4 NVIDIA H100 \(80GB\) GPUs and are implemented via the Hugging Face Transformers framework555[https://github\.com/huggingface/transformers](https://github.com/huggingface/transformers)\. We employ the AdamW optimizer\[[30](https://arxiv.org/html/2610.19492#bib.bib30)\]with a peak learning rate of10−410^\{\-4\}, a one\-epoch linear warmup, and a cosine learning rate scheduler\. Additional optimizer hyperparameters includeβ1=0\.9\\beta\_\{1\}=0\.9,β2=0\.999\\beta\_\{2\}=0\.999, weight decayλ=0\.01\\lambda=0\.01, and a global batch size of 512\. Our streaming ITN models utilize dynamic context masking with variable chunk sizesc∈\{1,2,3,4,5\}c\\in\\\{1,2,3,4,5\\\}and right\-context sizesr∈\{0,1,2\}r\\in\\\{0,1,2\\\}withptrunc=0\.8p\_\{\\text\{trunc\}\}=0\.8for prefix\-based training augmentation\. To benchmark against a streaming hybrid model, we retrain the architecture from\[[12](https://arxiv.org/html/2610.19492#bib.bib12)\]on our dataset and integrate it with our own weighted FST module\. For a baseline streaming end\-to\-end approach, we train a multi\-path wait\-kkmodel\[[15](https://arxiv.org/html/2610.19492#bib.bib15)\]\. We also compare our approach against the AdapITN model\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\], which is trained on our dataset with the biasing mechanism excluded to ensure a fair comparison\. Evaluation:During inference, we employ greedy decoding\. Following the methodology in\[[1](https://arxiv.org/html/2610.19492#bib.bib1)\], we evaluate performance using ITN Word Error Rate \(I\-WER\) to measure normalization quality and Non\-ITN Word Error Rate \(NI\-WER\) to quantify unintended perturbations to the source text\. Standard WER, the composite error of the entire process of converting spoken text to written text, is used for comprehensive performance assessment\. Tagging performance is evaluated using theF1F\_\{1\}\-score\. To simulate real\-time inference for benchmarking, inputs are processed in discrete chunks\. The reported metrics represent the average WER across an ensemble of configurations, where chunk sizes are sampled from\{3,4,5\}\\\{3,4,5\\\}tokens and right\-context sizes are randomly selected from\{1,2\}\\\{1,2\\\}tokens\. For configurations utilizing the RTW policy, we setLmax\_local=64L\_\{\\text\{max\\\_local\}\}=64\. This evaluation strategy ensures that the model’s robustness is accurately reflected across various streaming latencies, reflecting real\-world usage scenarios\. Statistical Evaluation:To ensure the reliability of our results, we employ a bootstrap method with 10,000 iterations to determine 95% confidence intervals \(CIs\) for the metric of interest, leveraging the toolkit provided in\[[31](https://arxiv.org/html/2610.19492#bib.bib31)\]\. Results are presented in the formatmean\[LB,UB\]\\textit\{mean\}\_\{\[\\textit\{LB\},\\textit\{UB\}\]\}, wheremeanrepresents the metric’s value on the original test set and the interval\[LB,UB\]\[\\textit\{LB\},\\textit\{UB\}\]denotes the lower and upper bounds of the 95% confidence interval\.

## 4Results and Analysis

Table 1:Evaluation of ITN performance with and without punctuation and capitalization \(PnC\)\. Parametersccandrrdenote chunk and right\-context sizes during inference\. All metrics are reported as percentages \(%\)\. Model categories:BB\(Non\-streaming baselines\),HH\(Hybrid streaming models\), andSS\(Proposed end\-to\-end streaming variants\)\.With PnCWithout PnCModelPretrainedcrF1I\-WERNI\-WERWERI\-WERNI\-WERWERNon\-streamB0B\_\{0\}×\\times∞\\infty∞\\infty–14\.201\.514\.618\.280\.541\.11B1B\_\{1\}EnViT5∞\\infty∞\\infty–11\.691\.303\.845\.170\.390\.74B2B\_\{2\}EnViT5∞\\infty∞\\infty97\.3410\.801\.323\.635\.030\.160\.52AdapITN\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\]EnViBert∞\\infty∞\\infty96\.65156\.882\.5040\.0814\.580\.401\.45SCS\_\{C\}\(ours\)EnViT5\{3,4,5\}\{0,1,2\}96\.6211\.481\.313\.795\.830\.180\.60StreamH1H\_\{1\}\[[12](https://arxiv.org/html/2610.19492#bib.bib12)\]PhoBert\{3,4,5\}086\.30†34\.792\.2210\.1649\.341\.054\.62H2H\_\{2\}\[[12](https://arxiv.org/html/2610.19492#bib.bib12)\]PhoBert\{3,4,5\}\{1,2\}90\.21†28\.391\.788\.2747\.360\.754\.20Wait\-kk\[[15](https://arxiv.org/html/2610.19492#bib.bib15)\]EnViT5withk=9k=9–19\.403\.006\.9916\.792\.683\.72S0S\_\{0\}\(ours\)×\\times\{3,4,5\}095\.0921\.391\.566\.3910\.541\.031\.73S1S\_\{1\}\(ours\)EnViT5\{3,4,5\}095\.4816\.661\.475\.187\.420\.290\.82S2S\_\{2\}\(ours\)EnViT5\{3,4,5\}\{1,2\}97\.1912\.911\.364\.186\.240\.190\.64S3S\_\{3\}\(ours\)EnViT5\{3,4,5\}095\.4817\.031\.375\.198\.790\.260\.89S4S\_\{4\}\(ours\)EnViT5\{3,4,5\}\{1,2\}97\.1913\.221\.314\.217\.300\.160\.69†denotes the hybrid model’sF1F\_\{1\}score, which is based on fine\-grained multi\-class classification\.### 4\.1ITN Performance

In this section, we comprehensively evaluate the ITN performance of various model configurations, categorized into three groups:

- •Non\-streaming end\-to\-end baselines \(BB\):B0B\_\{0\}is an EnViT5 model trained from scratch, whileB1B\_\{1\}utilizes pretrained weights\. Both models are trained in a seq2seq manner using theℒg​e​n\\mathcal\{L\}\_\{gen\}loss function\.B2B\_\{2\}further enhancesB1B\_\{1\}with an additional auxiliary tagging loss,ℒt​a​g\\mathcal\{L\}\_\{tag\}\. We also include AdapITN\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\]as a comparative baseline\.
- •Hybrid streaming \(HH\):We benchmark against the hybrid architecture from\[[12](https://arxiv.org/html/2610.19492#bib.bib12)\], denoted asH1H\_\{1\}\(with look\-aheadr=0r=0\) andH2H\_\{2\}\(withr∈\{1,2\}r\\in\\\{1,2\\\}\)\.
- •Proposed end\-to\-end streaming \(SS\):SCS\_\{C\}is our causal\-masked encoder model with non\-streaming inference\.S0S\_\{0\},S1S\_\{1\}andS2S\_\{2\}are our streaming variants using the RTW policy, whereasS3S\_\{3\}andS4S\_\{4\}represent the “fully streaming” versions utilizing decoder KV caching \(as described in Section[2\.4](https://arxiv.org/html/2610.19492#S2.SS4)\)\. Note thatS0S\_\{0\}is trained from scratch, while all otherSSvariants leverage pretrained weights\. As an additional end\-to\-end streaming baseline, we benchmark against a multi\-path wait\-kkmodel\[[15](https://arxiv.org/html/2610.19492#bib.bib15)\]decoded withk=9k=9\.

Table[1](https://arxiv.org/html/2610.19492#S4.T1)presents ITN performance across two settings: With PnC \(punctuation, capitalization, and semiotic/phonetic normalization\) and Without PnC \(semiotic and phonetic normalization only\)\.

#### 4\.1\.1Non\-Streaming Evaluation

Leveraging the pretrained EnViT5 \(B1B\_\{1\}\) yields significant gains over training from scratch \(B0B\_\{0\}\), with relative WER improvements of16\.70\[10\.45,26\.15\]16\.70\_\{\[10\.45,26\.15\]\}% and33\.33\[5\.33,63\.16\]33\.33\_\{\[5\.33,63\.16\]\}% in the “With PnC” and “Without PnC” settings, respectively\. Consistently lower I\-WER and NI\-WER scores highlight the practical benefits of pretrained text\-to\-text models for end\-to\-end ITN\. By inheriting linguistic priors, the model better captures the global context and semantic alignment necessary to map informal speech to structured written forms\. Incorporating the auxiliary tagging loss \(B2B\_\{2\}\) further improves results, establishing a strong baseline with WERs of 3\.63% and 0\.52%\.

Although AdapITN\[[2](https://arxiv.org/html/2610.19492#bib.bib2)\]is an end\-to\-end system, it is not fine\-tuned from a pretrained text\-to\-text model\. Specifically, AdapITN utilizes encoder\-only pretrained EnViBert\[[32](https://arxiv.org/html/2610.19492#bib.bib32)\]for both the encoder and decoder, with a tagging layer to isolate normalization regions\. However, its decoder only processes isolated text spans, excluding the surrounding sentential context and making capitalization and punctuation recovery extremely difficult\. As a result, it struggles to generalize in the “With PnC” evaluation, yielding an I\-WER of 156\.88% and a high overall WER of 40\.08%\. Even in the “Without PnC” setting, which is consistent with its original evaluation, AdapITN’s 1\.45% WER is2\.79\[2\.61,3\.30\]×2\.79\_\{\[2\.61,3\.30\]\}\\timeshigher than ourB2B\_\{2\}baseline\. These results demonstrate the critical importance of incorporating surrounding context and leveraging a pretrained text\-to\-text model for complex ITN tasks\.

#### 4\.1\.2Streaming Evaluation

Streaming performance follows a similar trend\.S1S\_\{1\}\(pretrained\) outperformsS0S\_\{0\}\(trained from scratch\) with relative WER improvements of18\.94\[10\.92,27\.84\]18\.94\_\{\[10\.92,27\.84\]\}% and52\.60\[29\.30,67\.08\]52\.60\_\{\[29\.30,67\.08\]\}%\. Our proposedS2S\_\{2\}variant achieves WERs of 4\.18% and 0\.64%, significantly outperforming the Wait\-kkmodel\[[15](https://arxiv.org/html/2610.19492#bib.bib15)\]by factors of1\.67\[1\.60,1\.75\]×1\.67\_\{\[1\.60,1\.75\]\}\\timesand5\.81\[4\.96,6\.73\]×5\.81\_\{\[4\.96,6\.73\]\}\\times, respectively\. The failures of Wait\-kkstem from forced generation: even after waitingkktokens, the source input may still lag behind the target output, since a single target token can correspond to multiple source tokens\. This situation occurs during both training and inference\. Additionally,S2S\_\{2\}substantially surpasses the hybrid baselineH2H\_\{2\}\[[12](https://arxiv.org/html/2610.19492#bib.bib12)\]by1\.98\[1\.94,2\.01\]×1\.98\_\{\[1\.94,2\.01\]\}\\timesand6\.56\[5\.74,7\.40\]×6\.56\_\{\[5\.74,7\.40\]\}\\times\. The high I\-WERs observed forH2H\_\{2\}\(28\.39% and 47\.36%\) highlight the limitations of manually crafted FST rules for complex normalization, whereas our end\-to\-end approach learns these transformations directly from data\.

Regarding the cost of streaming, the causal encoder with non\-streaming inference \(SCS\_\{C\}\) performs comparably toB2B\_\{2\}, with marginal WER increases of0\.16\[0\.03,0\.27\]0\.16\_\{\[0\.03,0\.27\]\}% and0\.08\[−0\.03,0\.18\]0\.08\_\{\[\-0\.03,0\.18\]\}%\. Under full streaming inference,S2S\_\{2\}remains robust: it incurs only an8\.85\[−4\.12,16\.77\]8\.85\_\{\[\-4\.12,16\.77\]\}% relative WER increase overB1B\_\{1\}in the “With PnC” setting and improves performance by13\.51\[−38\.67,50\.62\]13\.51\_\{\[\-38\.67,50\.62\]\}% in the “Without PnC” setting\. This efficiency is attributed to our RTW policy and prefix\-based training augmentation\. Compared toB2B\_\{2\},S2S\_\{2\}shows relative WER increases of15\.15\[10\.29,17\.17\]15\.15\_\{\[10\.29,17\.17\]\}% and23\.07\[0\.65,47\.15\]23\.07\_\{\[0\.65,47\.15\]\}%, quantifying the inherent trade\-off when future context is unavailable to resolve global dependencies\. In terms of tagging accuracy,S2S\_\{2\}achieves a highF1F\_\{1\}score of 97\.19%, comparable to theB2B\_\{2\}baseline, demonstrating its reliable boundary detection capability for the RTW policy\.

### 4\.2Impact of KV Caching and Copy–Paste Strategy

![Refer to caption](https://arxiv.org/html/2610.19492v1/img/itn_latency.png)Figure 2:Per\-chunk latency breakdown \(Encoder vs\. Total\) for four streaming configurations across chunk sizes of 3, 4, and 5\.We benchmark per\-chunk processing latency across configurations and chunk sizes with a fixed look\-aheadr=1r=1\. To simulate high\-concurrency production, we use a batch size of 100 with sequences of 100 \- 200 words on an NVIDIA H100 GPU\. Average encoder and total latencies are evaluated for four cumulative configurations, as shown in Fig\.[2](https://arxiv.org/html/2610.19492#S4.F2):

- •BL \(Baseline\):Standard streaming without optimizations, requiring full decoder processing for all tokens\.
- •CP \(\+Copy–Paste\):Uses Copy–Paste strategy to bypass the decoder forOO\-tagged tokens, directly emit spoken\-form text\.
- •EC \(\+Encoder Caching\):Adds incremental encoder KV caching to CP \(corresponding toS1S\_\{1\}andS2S\_\{2\}variants\)\.
- •DC \(\+Decoder Caching\):The most optimized configuration, adding self\- and cross\-attention decoder KV caching to EC, representing the “fully streaming”S3S\_\{3\}andS4S\_\{4\}variants\.

As illustrated in Fig\.[2](https://arxiv.org/html/2610.19492#S4.F2), the Copy–Paste strategy significantly reduces total latency and mitigates hallucinations by eliminating redundant decoder invocations for verbatim tokens\. Encoder KV caching \(EC\) dramatically reduces encoder latency from 54 ms to 10 ms, while decoder KV caching \(DC\) further minimizes overhead by calculating updates only for new tokens, bypassing the initial batched forward pass\. Combined, these optimizations achieve a3\.43×3\.43\\timesspeedup atc=5c=5, where total latency decreases from 206 ms \(BL\) to 60 ms \(DC\)\. The total latency exhibits only marginal absolute increases across chunk sizes, offering flexible trade\-offs between latency and context\.

While the “fully streaming” DC approach introduces a minor training\-inference mismatch, the performance impact is negligible\. Table[1](https://arxiv.org/html/2610.19492#S4.T1)shows thatS4S\_\{4\}\(DC\) maintains nearly identical performance toS2S\_\{2\}\(EC\), with absolute WER increases of only0\.03\[−0\.05,0\.09\]0\.03\_\{\[\-0\.05,0\.09\]\}% and0\.05\[−0\.03,0\.11\]0\.05\_\{\[\-0\.03,0\.11\]\}% across settings\. However, DC improves total per\-chunk latency by 31\.82% relative to EC atc=5c=5\. We also evaluate the95t​h95^\{th\}percentile \(p95p\_\{95\}\) latency to assess worst\-case scenarios requiring heavy normalization\. Thep95p\_\{95\}latency for DC is 165\.5 ms, compared to 249\.0 ms for EC\. Given that a chunk size of 5 typically represents 500 ms of audio, our model ensures real\-time processing even in the95t​h95^\{th\}percentile, demonstrating its capacity to serve 100 concurrent users in large\-scale deployments\.

### 4\.3Impact of Look\-ahead

As illustrated in Table[1](https://arxiv.org/html/2610.19492#S4.T1), incorporating look\-ahead context \(S2S\_\{2\}\) yields consistent performance gains across all primary metrics\. Specifically, theS2S\_\{2\}model achieves relative WER reductions of19\.31\[18\.62,19\.89\]19\.31\_\{\[18\.62,19\.89\]\}% and21\.95\[17\.95,25\.04\]21\.95\_\{\[17\.95,25\.04\]\}% compared to the zero\-look\-ahead variant \(S1S\_\{1\}\) in the “With PnC” and “Without PnC” settings, respectively\. Similarly, theS4S\_\{4\}model achieves relative WER reductions of18\.88\[18\.50,19\.29\]18\.88\_\{\[18\.50,19\.29\]\}% and22\.47\[20\.62,23\.99\]22\.47\_\{\[20\.62,23\.99\]\}% compared to its zero\-look\-ahead counterpart \(S3S\_\{3\}\)\. A comparable trend is observed in the hybrid streaming architecture, whereH2H\_\{2\}outperformsH1H\_\{1\}with WER reductions of18\.60\[18\.33,18\.93\]18\.60\_\{\[18\.33,18\.93\]\}% and9\.09\[8\.79,9\.42\]9\.09\_\{\[8\.79,9\.42\]\}% in the corresponding settings\. These findings underscore the critical role of look\-ahead context during both training and inference\.

### 4\.4Real\-world evaluation

Table 2:Real\-world ITN performance on noisy ASR transcripts\. All metrics are reported as percentages \(%\)\.To assess performance in a practical deployment context, we evaluate our models on a test set of noisy ASR transcripts\. Unlike the original test set, these inputs contain inherent recognition errors and acoustic artifacts typical of end\-to\-end ASR systems\. The experimental results are summarized in Table[2](https://arxiv.org/html/2610.19492#S4.T2)\.

In the streaming scenario, our proposedS4S\_\{4\}variant achieves the highest performance, significantly outperforming the hybrid streaming baselineH2H\_\{2\}by21\.43\[21\.00,21\.95\]21\.43\_\{\[21\.00,21\.95\]\}% and surpassing the Wait\-kkby9\.82\[0\.86,19\.46\]9\.82\_\{\[0\.86,19\.46\]\}% in terms of relative WER reduction\. When compared to the strongest offline baseline \(B1B\_\{1\}\),S4S\_\{4\}maintains competitive performance, incurring a relative WER increase of only7\.29\[6\.85,7\.69\]7\.29\_\{\[6\.85,7\.69\]\}%\. Interestingly, on these noisy transcripts, the “fully streaming” modelS4S\_\{4\}slightly surpasses theS2S\_\{2\}variant, with a relative improvement of1\.38\[0\.48,2\.73\]1\.38\_\{\[0\.48,2\.73\]\}%\. This suggests that the inclusion of decoder KV caching does not negatively impact robustness to ASR errors\. Rather, it may provide slight stability in noisy streaming contexts\.

### 4\.5Impact of Truncation Probability

![Refer to caption](https://arxiv.org/html/2610.19492v1/img/wer_prob_trunc.png)Figure 3:Impact of truncation probabilityptruncp\_\{\\text\{trunc\}\}on the Word Error Rate for both offline and streaming inference of our model\.To assess our prefix\-based training augmentation, we evaluate performance on a validation subset across two modes: Streaming Inference \(RTW policy\) and Offline Inference \(causal encoder\)\. All experiments usec=3c\{=\}3andr=1r\{=\}1\. Figure[3](https://arxiv.org/html/2610.19492#S4.F3)illustrates the sensitivity of WER to the truncation probabilityptruncp\_\{\\text\{trunc\}\}\.

For offline inference, the model remains robust across0\.0≤ptrunc≤0\.90\.0\\leq p\_\{\\text\{trunc\}\}\\leq 0\.9\. Whileptrunc=0p\_\{\\text\{trunc\}\}=0yields the lowest WER by avoiding termination\-related distribution shifts, performance stays stable asptruncp\_\{\\text\{trunc\}\}increases\. A sharp degradation occurs only atptrunc=1\.0p\_\{\\text\{trunc\}\}=1\.0, where the lack of natural sentence endings during training impairs generalization to full sequences\.

In the streaming scenario, augmentation is essential\. Without it \(ptrunc=0p\_\{\\text\{trunc\}\}=0\), the WER exceeds 100% as the model fails to predict the local<EOS\>tokens required by the RTW policy\. Introducing prefix\-based augmentation \(ptrunc\>0p\_\{\\text\{trunc\}\}\>0\) immediately stabilizes performance\. As shown in Figure[3](https://arxiv.org/html/2610.19492#S4.F3), increasingptruncp\_\{\\text\{trunc\}\}correlates with improved streaming accuracy, as the model learns to identify local boundaries from partial context\.

The gap between offline and streaming modes narrows significantly forptrunc∈\{0\.6,0\.7,0\.8,0\.9\}p\_\{\\text\{trunc\}\}\\in\\\{0\.6,0\.7,0\.8,0\.9\\\}\. The optimal balance is achieved atptrunc=0\.8p\_\{\\text\{trunc\}\}=0\.8, yielding the lowest streaming WER while maintaining competitive offline performance\. Theptrunc=1\.0p\_\{\\text\{trunc\}\}=1\.0case is also detrimental to streaming, suggesting that complete removal of full\-sequence supervision harms long\-range dependency modeling\. These results confirm that prefix\-based augmentation effectively bridges the training\-inference discrepancy for streaming ITN\.

### 4\.6Generalizability Across Architectures and Languages

To demonstrate the versatility of our framework, we evaluate it across alternative text\-to\-text architectures, pretraining objectives, and languages\. For Vietnamese, we adopt the pretrained BartPho model666[https://huggingface\.co/vinai/bartpho\-syllable\-base](https://huggingface.co/vinai/bartpho-syllable-base)\[[33](https://arxiv.org/html/2610.19492#bib.bib33)\]as the backbone, which was pretrained using a denoising autoencoder objective on Vietnamese text\. For English, we leverage a pretrained T5 model777[https://huggingface\.co/google\-t5/t5\-base](https://huggingface.co/google-t5/t5-base)to assess cross\-lingual compatibility, with training and evaluation conducted on the English subset of the standard Google TN dataset\[[34](https://arxiv.org/html/2610.19492#bib.bib34)\]\.

Table 3:Generalizability of ITN performance across architectures and languages\. Values represent average WER \(%\) acrossc∈\{3,4,5\}c\\in\\\{3,4,5\\\}andr∈\{0,1,2\}r\\in\\\{0,1,2\\\}\. Vietnamese evaluation uses ASR\-L \(ground\-truth spoken text\) and ASR\-O \(noisy ASR transcripts\); English evaluation uses With and Without PnC\.Lang\.BackboneParamsStreamWER \(%\)ASR\-LASR\-OViBartPho132M×\\times4\.6111\.07✓\\checkmark6\.3112\.84EnViT5275M×\\times3\.8410\.01✓\\checkmark4\.5411\.02W/ PnCW/o PnCEnT5223M×\\times6\.481\.33✓\\checkmark8\.171\.84We evaluate streaming models under the RTW policy and non\-streaming models using standard decoding\. Table[3](https://arxiv.org/html/2610.19492#S4.T3)reports the average WER across all configurations \(c∈\{3,4,5\}c\\in\\\{3,4,5\\\}andr∈\{0,1,2\}r\\in\\\{0,1,2\\\}\)\. The results show that our framework consistently narrows the performance gap between streaming and non\-streaming settings across architectures and languages\. While the larger EnViT5 backbone delivers the best absolute performance, the more compact BartPho model demonstrates comparable robustness under streaming constraints and noisy ASR inputs\. A similar trend is observed in English, where the T5 backbone also achieves strong and stable performance\. Together, these results suggest that the proposed method is architecture\-agnostic and language\-agnostic, maintaining effectiveness across different model scales and linguistic contexts\.

### 4\.7Limitations and Future Work

Despite the efficiency gains of our streaming ITN framework, several limitations offer avenues for future research:

- •Phonetic Normalization and Failure Modes:While semiotic classes are handled effectively, qualitative error analysis reveals that performance on OOV expressions and foreign entities remains challenging\. A common failure mode occurs when the tagger successfully identifies a phonetic span, but the decoder hallucinates an orthographically plausible but factually incorrect spelling for a rare foreign entity \(e\.g\., generating “Marid” instead of “Madrid” due to phonetic ambiguity in the ASR output\)\. Future work will investigate biasing mechanisms to integrate phonetic dictionaries directly into the decoding process to improve I\-WER without disrupting the incremental KV cache or increasing latency\.
- •Early Emission Strategy:The current RTW policy waits for a completeB−IB\-Ispan before emission, which may introduce perceived lag for long sequences\. We aim to explore predictive decoding strategies that allow for partial output as soon as sufficient linguistic context is captured, rather than waiting for span completion\.
- •Edge Deployment Optimization:With 275M parameters, the model may limit deployment on resource\-constrained devices\. Future work will explore compression methods, such as knowledge distillation and quantization, to reduce memory footprint and improve throughput on low\-power hardware\.

## 5Conclusion

This paper introduces an efficient end\-to\-end streaming ITN framework leveraging pretrained text\-to\-text models\. By integrating a chunk\-based causal encoder with an RTW decoding policy, we adapt non\-streaming architectures for real\-time inference\. To bridge the train\-test discrepancy, we propose a prefix\-based training augmentation strategy that ensures reliable local termination\. Combined with KV caching and a Copy–Paste mechanism, our system meets production\-grade latency requirements without sacrificing accuracy\. Experimental results demonstrate that our approach significantly outperforms traditional hybrid streaming methods, achieving performance comparable to offline baselines\. Finally, evaluations across architectures and languages confirm that the framework is scalable, backbone\-agnostic, and language\-agnostic, offering a versatile solution for complex normalization tasks\.

## 6Acknowledgments

We thank UNEY for sponsoring this research and providing the computational resources essential for model training and evaluation\.

## 7Generative AI Use Disclosure

Generative AI tools were used solely for editing and language polishing of the manuscript\. They were not used to generate scientific content, experimental designs, results, analyses, or conclusions\. All authors are responsible for the full content of this paper and consent to its submission\.

## References

- \[1\]M\. Sunkara, C\. Shivade, S\. Bodapati, and K\. Kirchhoff, “Neural inverse text normalization,” in*ICASSP 2021*\. IEEE, 2021, pp\. 7573–7577\.
- \[2\]T\.\-B\. Nguyen, L\. D\. M\. Nhat, Q\. M\. Nguyen, Q\. T\. Do, C\. M\. Luong, and A\. Waibel, “Adapitn: A fast, reliable, and dynamic adaptive inverse text normalization,” in*ICASSP 2023*\. IEEE, 2023, pp\. 1–5\.
- \[3\]Y\. Gaur, N\. Kibre, J\. Xue, K\. Shu, Y\. Wang, I\. Alphanso, J\. Li, and Y\. Gong, “Streaming, fast and accurate on\-device inverse text normalization for automatic speech recognition,” in*IEEE SLT 2022*\. IEEE, 2023, pp\. 237–244\.
- \[4\]M\. Ihori, A\. Takashima, and R\. Masumura, “Large\-context pointer\-generator networks for spoken\-to\-written style conversion,” in*ICASSP 2020*\. IEEE, 2020, pp\. 8189–8193\.
- \[5\]S\.\-J\. Chen, D\. Paul, Y\. Pang, P\. Su, and X\. Zhang, “Language agnostic data\-driven inverse text normalization,” in*Interspeech 2023*, 2023, pp\. 451–455\.
- \[6\]H\. Choi, M\. Choi, Y\. Lim, M\. Lee, S\. Kim, S\. Yun, D\. Kim, and S\. Kim, “Spoken\-to\-written text conversion with Large Language Model,” in*Interspeech 2024*, 2024, pp\. 2410–2414\.
- \[7\]M\. Shugrina, “Formatting time\-aligned asr transcripts for readability,” in*Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics*, 2010, pp\. 198–206\.
- \[8\]Y\. Zhang, E\. Bakhturina, and B\. Ginsburg, “Nemo inverse text normalization: From development to production,” in*Interspeech 2021*, 2021, pp\. 4857–4859\.
- \[9\]Alexandra Antonova and Evelina Bakhturina and Boris Ginsburg, “Thutmose Tagger: Single\-pass neural model for Inverse Text Normalization,” in*Interspeech 2022*, 2022, pp\. 550–554\.
- \[10\]Debjyoti Paul and Yutong Pang and Szu\-Jui Chen and Xuedong Zhang, “Improving Data Driven Inverse Text Normalization using Data Augmentation and Machine Translation,” in*Interspeech 2022*, 2022, pp\. 5221–5222\.
- \[11\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin, “Attention is all you need,”*Advances in neural information processing systems*, vol\. 30, 2017\.
- \[12\]L\. Ho, K\. Le, V\. Pham, B\. Nguyen, T\. Tran, and D\. Chau, “Dynamic Context\-Aware Streaming Pretrained Language Model For Inverse Text Normalization,” in*Interspeech 2025*, 2025, pp\. 4433–4437\.
- \[13\]M\. Wang, T\. Vu, Y\. Wang, E\. Shareghi, and G\. Haffari, “Conversational simulmt: Efficient simultaneous translation with large language models,” in*IWSLT 2025*, 2025, pp\. 93–105\.
- \[14\]M\. Ma, L\. Huang, H\. Xiong, R\. Zheng, K\. Liu, B\. Zheng, C\. Zhang, Z\. He, H\. Liu, X\. Li*et al\.*, “Stacl: Simultaneous translation with implicit anticipation and controllable latency using prefix\-to\-prefix framework,” in*ACL 2019*, 2019, pp\. 3025–3036\.
- \[15\]M\. Elbayad, L\. Besacier, and J\. Verbeek, “Efficient Wait\-k Models for Simultaneous Machine Translation,” in*Interspeech 2020*, 2020, pp\. 1461–1465\.
- \[16\]L\. Zhao, K\. Fan, W\. Luo, W\. Jing, S\. Wang, Z\. Zeng, and Z\. Huang, “Adaptive policy with wait\-k model for simultaneous translation,” in*EMNLP 2023*, 2023, pp\. 4816–4832\.
- \[17\]N\. Arivazhagan, C\. Cherry, W\. Macherey, C\.\-C\. Chiu, S\. Yavuz, R\. Pang, W\. Li, and C\. Raffel, “Monotonic infinite lookback attention for simultaneous machine translation,” in*ACL 2019*, 2019, pp\. 1313–1323\.
- \[18\]B\. Zheng, R\. Zheng, M\. Ma, and L\. Huang, “Simpler and faster learning of adaptive policies for simultaneous translation,” in*EMNLP\-IJCNLP 2019*, 2019, pp\. 1349–1354\.
- \[19\]S\. Zhang and Y\. Feng, “Hidden markov transformer for simultaneous machine translation,” in*ICLR 2023*, 2023\.
- \[20\]S\. Guo, S\. Zhang, Z\. Ma, and Y\. Feng, “Large language models are read/write policy\-makers for simultaneous generation,” in*AAAI 2025*, vol\. 39, no\. 22, 2025, pp\. 23 969–23 977\.
- \[21\]M\. Lewis, Y\. Liu, N\. Goyal, M\. Ghazvininejad, A\. Mohamed, O\. Levy, V\. Stoyanov, and L\. Zettlemoyer, “Bart: Denoising sequence\-to\-sequence pre\-training for natural language generation, translation, and comprehension,” in*ACL 2020*, 2020, pp\. 7871–7880\.
- \[22\]C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. Liu, “Exploring the limits of transfer learning with a unified text\-to\-text transformer,”*Journal of machine learning research*, vol\. 21, no\. 140, pp\. 1–67, 2020\.
- \[23\]L\. Xue, N\. Constant, A\. Roberts, M\. Kale, R\. Al\-Rfou, A\. Siddhant, A\. Barua, and C\. Raffel, “mt5: A massively multilingual pre\-trained text\-to\-text transformer,” in*Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: Human language technologies*, 2021, pp\. 483–498\.
- \[24\]C\. Ngo, T\. H\. Trinh, L\. Phan, H\. Tran, T\. Dang, H\. Nguyen, M\. Nguyen, and M\.\-T\. Luong, “Mtet: Multi\-domain translation for english and vietnamese,”*arXiv preprint arXiv:2210\.05610*, 2022\.
- \[25\]G\. Wenzek, M\.\-A\. Lachaux, A\. Conneau, V\. Chaudhary, F\. Guzmán, A\. Joulin, and E\. Grave, “Ccnet: Extracting high quality monolingual datasets from web crawl data,” in*LREC 2020*, 2020, pp\. 4003–4012\.
- \[26\]T\. Kudo and J\. Richardson, “Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,” in*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations*, 2018, pp\. 66–71\.
- \[27\]L\. Doan, N\. L\. Tran, T\. Hoang, D\. Q\. Nguyen*et al\.*, “Phomt: A high\-quality and large\-scale benchmark dataset for vietnamese\-english machine translation,” in*EMNLP 2021*, 2021, pp\. 4495–4503\.
- \[28\]K\. Le and D\. Chau, “Improving Streaming Speech Recognition With Time\-Shifted Contextual Attention And Dynamic Right Context Masking,” in*Interspeech 2024*, 2024, pp\. 4478–4482\.
- \[29\]K\. Le, T\. V\. Ho, D\. Tran, and D\. T\. Chau, “Chunkformer: Masked chunking conformer for long\-form speech transcription,” in*ICASSP 2025*\. IEEE, 2025, pp\. 1–5\.
- \[30\]I\. Loshchilov and F\. Hutter, “Decoupled weight decay regularization,” in*ICLR 2019*, 2019\.
- \[31\]L\. Ferrer and P\. Riera, “Confidence intervals for evaluation in machine learning\.” \[Online\]\. Available:[https://github\.com/luferrer/ConfidenceIntervals](https://github.com/luferrer/ConfidenceIntervals)
- \[32\]T\. B\. Nguyen, Q\. M\. Nguyen, T\. T\. H\. Nguyen, Q\. T\. Do, and C\. M\. Luong, “Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models,” in*Proc\. Interspeech 2020*, 2020, pp\. 4263–4267\.
- \[33\]N\. L\. Tran, D\. Le, and D\. Q\. Nguyen, “BARTpho: Pre\-trained Sequence\-to\-Sequence Models for Vietnamese,” in*Interspeech 2022*, 2022, pp\. 1751–1755\.
- \[34\]R\. Sproat and N\. Jaitly, “An RNN Model of Text Normalization,” in*Interspeech 2017*, 2017, pp\. 754–758\.