T-SANDHI: Tone Sandhi-aware Adaptive Network with Decoupled Hybrid Injection for Low-resource Taiwanese Hokkien Speech Recognition

arXiv cs.CL Papers

Summary

The paper proposes T-SANDHI, a Tone Sandhi-aware Adaptive Network for low-resource Taiwanese Hokkien speech recognition, which explicitly decouples tonal variations to improve accuracy on top of a Whisper backbone.

arXiv:2609.18194v1 Announce Type: new Abstract: In Taiwanese Hokkien automatic speech recognition (ASR), prior studies often treat tone sandhi as a major challenge under the assumption that models fail to process implicit phonological variations. However, our experiments on Taiwanese Hokkien reveal that speech foundation models actually handle tone sandhi variations effectively, and the real performance bottleneck stems from a localized confusion between these variations and retained citation tones. To address this, we propose T-SANDHI to explicitly decouple surface acoustics from underlying lexical intent on top of a frozen Whisper backbone. Using a lexicon-guided multi-task learning structure driven by text-derived pseudo labels, our lightweight hybrid injection module integrates independent citation and sandhi phonetic streams via dynamic gating. Extensive evaluation on the TAT-MOE corpus and two blind test sets demonstrates that this explicit disentanglement effectively resolves tonal mapping confusion, outperforming baselines with strict parameter efficiency.
Original Article
View Cached Full Text

Cached at: 09/17/26, 09:09 AM

# T-SANDHI: Tone Sandhi-aware Adaptive Network with Decoupled Hybrid Injection for Low-resource Taiwanese Hokkien Speech Recognition
Source: [https://arxiv.org/html/2609.18194](https://arxiv.org/html/2609.18194)
Hung\-Yang Sung1, Chien\-Chun Wang2, Tien\-Hong Lo1, Yu\-Sheng Tsao3, Yung\-Chang Hsu3, Berlin Chen1Affiliation:1Department of Computer Science and Information Engineering, National Taiwan Normal University, Taiwan 2E\.SUN Financial Holding Co\., Ltd\., Taiwan 3EZAI, Taiwan

###### Abstract

In Taiwanese Hokkien automatic speech recognition \(ASR\), prior studies often treat tone sandhi as a major challenge under the assumption that models fail to process implicit phonological variations\. However, our experiments on Taiwanese Hokkien reveal that speech foundation models actually handle tone sandhi variations effectively, and the real performance bottleneck stems from a localized confusion between these variations and retained citation tones\. To address this, we propose T\-SANDHI to explicitly decouple surface acoustics from underlying lexical intent on top of a frozen Whisper backbone\. Using a lexicon\-guided multi\-task learning structure driven by text\-derived pseudo labels, our lightweight hybrid injection module integrates independent citation and sandhi phonetic streams via dynamic gating\. Extensive evaluation on the TAT\-MOE corpus and two blind test sets demonstrates that this explicit disentanglement effectively resolves tonal mapping confusion, outperforming baselines with strict parameter efficiency\.

###### Index Terms:

automatic speech recognition, low resource, Taiwanese Hokkien, tone sandhi, lexicon\-guided

## IIntroduction

A fundamental assumption in most automatic speech recognition \(ASR\) systems is a consistent, reliable mapping between surface acoustics and underlying lexical units\[[1](https://arxiv.org/html/2609.18194#bib.bib14),[2](https://arxiv.org/html/2609.18194#bib.bib24),[3](https://arxiv.org/html/2609.18194#bib.bib29),[4](https://arxiv.org/html/2609.18194#bib.bib10),[5](https://arxiv.org/html/2609.18194#bib.bib30),[6](https://arxiv.org/html/2609.18194#bib.bib8),[7](https://arxiv.org/html/2609.18194#bib.bib9)\]\. However, this mapping relationship becomes less straightforward in tonal languages with complex phonological variations\. A prime example is Taiwanese Hokkien \(Taiwanese\)\[[8](https://arxiv.org/html/2609.18194#bib.bib22),[9](https://arxiv.org/html/2609.18194#bib.bib4),[10](https://arxiv.org/html/2609.18194#bib.bib3),[11](https://arxiv.org/html/2609.18194#bib.bib25),[12](https://arxiv.org/html/2609.18194#bib.bib2),[13](https://arxiv.org/html/2609.18194#bib.bib33),[14](https://arxiv.org/html/2609.18194#bib.bib17)\]\. Unlike Mandarin\[[15](https://arxiv.org/html/2609.18194#bib.bib7),[16](https://arxiv.org/html/2609.18194#bib.bib37),[17](https://arxiv.org/html/2609.18194#bib.bib31)\], which maintains relatively static tonal mappings, Taiwanese Hokkien features a complex web of tone sandhi that applies to syllables based on their position within syntactic units\[[18](https://arxiv.org/html/2609.18194#bib.bib35),[19](https://arxiv.org/html/2609.18194#bib.bib34),[13](https://arxiv.org/html/2609.18194#bib.bib33)\]\. In continuous speech, tone sandhi applies to every non\-final syllable, meaning that the actual surface tone frequently shifts based on the grammatical context\[[13](https://arxiv.org/html/2609.18194#bib.bib33),[20](https://arxiv.org/html/2609.18194#bib.bib32)\]\. Consequently, a syllable’s underlying dictionary pronunciation \(the*citation tone*\) only occurs at specific morphosyntactically defined boundaries, while all other syllables are realized with altered tones \(the*sandhi tone*\)\[[20](https://arxiv.org/html/2609.18194#bib.bib32),[21](https://arxiv.org/html/2609.18194#bib.bib36),[22](https://arxiv.org/html/2609.18194#bib.bib15)\]\. As illustrated in Fig\.[1](https://arxiv.org/html/2609.18194#S1.F1), when a speaker utters the pronoun ‘‘You’’ \(Taiwanese Hanzi111[https://language\.moe\.gov\.tw/…/D005\.pdf](https://language.moe.gov.tw/001/Upload/files/site_content/M0001/language_100/D/D005.pdf):

你\), the citation syllable is “lí” \(Tone 2\) and the sandhi tone is Tone 1\.

Current ASR systems typically rely on models to implicitly internalize these complex phonological mappings through the final transcription loss\[[21](https://arxiv.org/html/2609.18194#bib.bib36),[23](https://arxiv.org/html/2609.18194#bib.bib13),[14](https://arxiv.org/html/2609.18194#bib.bib17)\]\. While scaled parameters in data\-abundant scenarios can partially absorb such variations, this implicit learning paradigm often faces substantial challenges under low\-resource constraints\[[24](https://arxiv.org/html/2609.18194#bib.bib39),[25](https://arxiv.org/html/2609.18194#bib.bib21),[26](https://arxiv.org/html/2609.18194#bib.bib38),[27](https://arxiv.org/html/2609.18194#bib.bib23)\]\. To address these constraints, adopting parameter\-efficient fine\-tuning \(PEFT\) techniques, such as AdaLoRA, has emerged as a standard and highly effective approach to adapt speech foundation models to various low\-resource languages\[[28](https://arxiv.org/html/2609.18194#bib.bib6),[29](https://arxiv.org/html/2609.18194#bib.bib12),[30](https://arxiv.org/html/2609.18194#bib.bib5)\]\. Following this established paradigm, we build our study upon a parameter\-efficient framework to explore foundation model behavior\. Prior study focused on utilizing syllables with citation tones as target labels, evaluating various self\-supervised learning \(SSL\) models with connectionist temporal classification \(CTC\) loss\[[13](https://arxiv.org/html/2609.18194#bib.bib33)\]\. Their findings indicate that tone sandhi represent a major source of character substitution errors, frequently confusing phonetically similar phones in the predictions, thereby suggesting that an independent tonal handling mechanism would be beneficial for future efforts\[[13](https://arxiv.org/html/2609.18194#bib.bib33)\]\. However, it remains unclear whether modern speech foundation models such as Whisper are similarly affected by tone sandhi variations\.

![Refer to caption](https://arxiv.org/html/2609.18194v1/sandhi_example.png)Fig\. 1:Illustration of the acoustic\-to\-label discrepancy in Taiwanese Hokkien continuous speech\. Due to the tone sandhi system, the actual surface tone \(sandhi tone, highlighted in orange\) frequently shifts away from its dictionary pronunciation \(citation tone\)\. For example, in the phrase “Today where do you want to go”, the pronoun “lí” shifts from Tone 2 to Tone 1, whereas only the sentence\-final syllable retains its original citation tone \(highlighted in blue\)\.![Refer to caption](https://arxiv.org/html/2609.18194v1/tone_distribution.png)Fig\. 2:Quantitative analysis of tonal distribution and phonetic variations across TAT\-MOE corpus\. \(a\) Overall distribution of syllables categorized by their citation forms \(blue bars\) and sandhi forms \(orange bars\)\. \(b\) Tone transition matrix detailing the absolute counts of syllables mapping from their citation tone to their realized tone\.To address this gap, we conduct an empirical analysis to evaluate baseline predictions across various model scales\. Our findings reveal a counter\-intuitive phenomenon: while modern foundation models process tone sandhi variations with reasonable proficiency, their performance bottlenecks significantly on syllables that genuinely retain their citation tones\. As we will detail in Section[II](https://arxiv.org/html/2609.18194#S2), this localized performance gap is primarily driven by an extreme volume imbalance in continuous speech, where distinct underlying citation categories frequently conflate into identical surface realizations\. These empirical findings indicate that explicitly incorporating both citation and sandhi syllable information can serve as a viable path to reduce this localized performance gap and enhance overall speech recognition accuracy\.

TABLE I:Statistics of the TAT\-MOE dataset across training, development and test splits, including the number of speakers, utterances and total duration in hours\.To bridge this critical gap, we propose T\-SANDHI, aToneSandhi\-awareAdaptiveNetwork designed specifically to resolve tonal mapping ambiguity\. Instead of treating the acoustic\-to\-label mapping as a monolithic black box, we introduce a parameter\-efficientDecoupledHybridInjection mechanism on top of a frozen Whisper backbone\[[1](https://arxiv.org/html/2609.18194#bib.bib14),[29](https://arxiv.org/html/2609.18194#bib.bib12),[31](https://arxiv.org/html/2609.18194#bib.bib16)\]\. To overcome data scarcity without reliance on expensive phonetic annotations, we adopt a rule\-derived multi\-task learning framework driven by automatically generated text\-based pseudo labels\. Specifically, by leveraging CTC\[[32](https://arxiv.org/html/2609.18194#bib.bib28),[33](https://arxiv.org/html/2609.18194#bib.bib1),[34](https://arxiv.org/html/2609.18194#bib.bib26),[35](https://arxiv.org/html/2609.18194#bib.bib27),[36](https://arxiv.org/html/2609.18194#bib.bib20)\], we explicitly construct two auxiliary streams, where one predicts the citation syllables and the other tracks the sandhi tones\. Since sandhi mutations are highly context\-dependent, a dynamic gating mechanism is employed to seamlessly integrate these dual phonetic streams\. This explicit decoupling allows the decoder to ground its predictions simultaneously on surface acoustic variations and underlying lexical intent\.

The main contributions of this study are as follows:

1. 1\.Novel Insights on Tone Sandhi:We reveal that while foundation models effectively process tone sandhi, they struggle to map these acoustics back to citation forms due to severe tonal confusion\.
2. 2\.Explicit Decoupling Architecture:We propose the first ASR framework to disentangle surface acoustics from underlying lexical intent via a dynamically gated, dual\-stream injection module\.
3. 3\.High Efficacy with Minimal Overhead:Our approach effectively mitigates mapping ambiguity on TAT\-MOE\[[37](https://arxiv.org/html/2609.18194#bib.bib18)\], achieving robust performance gains while adding merely 3% to the parameter count of the frozen backbone\.

TABLE II:Baseline performance analysis on the TAT\-MOE corpus, evaluated across sandhi and citation contexts\.
## IICorpora and Phonological Analysis

### II\-ACorpora

To evaluate under authentic low\-resource conditions, we utilized the TAT\-MOE corpus\[[37](https://arxiv.org/html/2609.18194#bib.bib18)\], alongside two external blind test sets: the FSRC 2020 corpus\[[38](https://arxiv.org/html/2609.18194#bib.bib19)\]and the yttd\_taigi\_trs corpus\[[10](https://arxiv.org/html/2609.18194#bib.bib3)\]\. All three datasets feature diverse accents and spontaneous speech\. The detailed statistical partitions of the TAT\-MOE corpus across the training, development, and test splits are summarized in Table[I](https://arxiv.org/html/2609.18194#S1.T1)\. Acoustic signals were uniformly resampled to 16 kHz for backbone alignment\. As a tonal language, Taiwanese Hokkien exhibits a rich and complex tone sandhi system\[[21](https://arxiv.org/html/2609.18194#bib.bib36),[20](https://arxiv.org/html/2609.18194#bib.bib32)\]\. In continuous spoken streams, these sandhi mutations apply systematically to virtually every non\-final syllable within a morphosyntactically defined unit\[[20](https://arxiv.org/html/2609.18194#bib.bib32)\], resulting in a prominent discrepancy between citation dictionary forms and contextual surface realizations\. A statistical phonological analysis regarding this tonal distribution and its impact on modern ASR is detailed below\.

### II\-BQuantitative Analysis of Tone Sandhi Impact

To investigate the impact of this phonological discrepancy on modern speech foundation models, we evaluate Whisper baselines across various scales based on phonological boundaries\. The contextual tone sandhi variants within the training split are deterministically derived from standard tone sandhi rules\[[20](https://arxiv.org/html/2609.18194#bib.bib32)\]\. As detailed in Table[II](https://arxiv.org/html/2609.18194#S1.T2), scaling up the architecture improves overall performance, yet a localized gap persists\. The models consistently exhibit lower character error rates \(CERs\) when decoding sandhi\-form syllables compared to citation\-form syllables\. Regardless of model size, the corresponding drops in precision and recall confirm a tonal mapping confusion, where the networks struggle to distinguish citation features from tone sandhi variations\.

To analyze this localized performance gap, we examine the data distribution from a phonological perspective\. Fig\.[2](https://arxiv.org/html/2609.18194#S1.F2)\(a\) highlights a severe volume imbalance in continuous Taiwanese Hokkien speech, where approximately 87% of syllables undergo tone sandhi and only about 12% retain their citation tones\. This extreme imbalance biases foundation models toward sandhi acoustic characteristics during pre\-training\. Consequently, implicit end\-to\-end learning fails to construct sufficiently robust representations for citation\-retained syllables under low\-resource fine\-tuning\. Furthermore, the tone transition matrix in Fig\.[2](https://arxiv.org/html/2609.18194#S1.F2)\(b\) illustrates that distinct underlying citation categories frequently conflate into identical surface realizations\. For instance, underlying Tone 1 \(T1\) and Tone 5 \(T5\) often converge into surface Tone 7 \(T7\) after sandhi mutations\. This many\-to\-one mapping ambiguity creates localized confusion between sandhi and citation forms, limiting overall performance and motivating the explicit phonetic disentanglement in our proposed architecture\.

## IIIProposed Method

### III\-AArchitecture Overview

As established, forcing an ASR model to implicitly memorize the complex, non\-linear mapping between speech undergo tone sandhi and citation labels leads to severe surface\-to\-underlying mapping discrepancy\. To explicitly break this entanglement, we propose T\-SANDHI\. As illustrated in Fig\.[3](https://arxiv.org/html/2609.18194#S3.F3), the architecture builds upon the robust acoustic priors of a frozen Whisper encoder\-decoder backbone\. To maintain parameter efficiency while adapting to the intricate Taiwanese phonology, we avoid catastrophic forgetting by applying AdaLoRA\[[28](https://arxiv.org/html/2609.18194#bib.bib6)\]only to the attention and feed\-forward modules\. Crucially, rather than relying on the decoder to implicitly resolve tonal ambiguities, we introduce a*Decoupled Hybrid Injection*module directly on top of the encoder\. This module explicitly constructs two independent auxiliary streams: one modeling the underlying lexical intent \(citation syllables\), and the other tracking the actual surface tone \(sandhi tones\)\.

![Refer to caption](https://arxiv.org/html/2609.18194v1/main.png)Fig\. 3:Architecture of T\-SANDHI\. To explicitly decouple lexical intent from surface acoustics, a decoupled hybrid injection module augments the frozen Whisper encoder\. It projects features into independent citation and sandhi streams, dynamically fusing them into𝐇f​u​s​e​d\\mathbf\{H\}\_\{fused\}via a gate generator\. Lightweight linear heads provide CTC supervision during training and supply decoupled representations during inference with negligible overhead\.
### III\-BDecoupled Hybrid Injection

The core design philosophy behind our injection module is to enforce representation disentanglement without introducing heavy computational overhead\. Let𝐇e​n​c∈ℝT×d\\mathbf\{H\}\_\{enc\}\\in\\mathbb\{R\}^\{T\\times d\}denote the acoustic hidden states from the Whisper encoder, whereTTis the sequence length andddis the hidden dimension\. We project these states through two auxiliary CTC heads to obtain the citation logits𝐙c​i​t∈ℝT×\|Vc​i​t\|\\mathbf\{Z\}\_\{cit\}\\in\\mathbb\{R\}^\{T\\times\|V\_\{cit\}\|\}and the sandhi logits𝐙s​a​n∈ℝT×\|Vs​a​n\|\\mathbf\{Z\}\_\{san\}\\in\\mathbb\{R\}^\{T\\times\|V\_\{san\}\|\}:

𝐙c​i​t\\displaystyle\\mathbf\{Z\}\_\{cit\}=𝐇e​n​c​𝐖c​i​t\+𝐛c​i​t,\\displaystyle=\\mathbf\{H\}\_\{enc\}\\mathbf\{W\}\_\{cit\}\+\\mathbf\{b\}\_\{cit\},\(1\)𝐙s​a​n\\displaystyle\\mathbf\{Z\}\_\{san\}=𝐇e​n​c​𝐖s​a​n\+𝐛s​a​n,\\displaystyle=\\mathbf\{H\}\_\{enc\}\\mathbf\{W\}\_\{san\}\+\\mathbf\{b\}\_\{san\},\(2\)where𝐖\\mathbf\{W\}and𝐛\\mathbf\{b\}denote the learnable weight matrices and bias vectors, while\|Vc​i​t\|\|V\_\{cit\}\|and\|Vs​a​n\|\|V\_\{san\}\|represent the vocabulary sizes of the citation syllables and sandhi tones, respectively\. Crucially, we deliberately restrict these heads to simple linear projections\. This architectural bottleneck ensures that the parameter overhead remains minimal, actively forcing the core Whisper encoder to learn highly disentangled and robust acoustic representations rather than outsourcing the task to deep, heavy sub\-networks\.

To prepare these decoupled streams for integration, we first convert the raw logits into probability distributions via a softmax function, and then project them back to the hidden dimensionddto form phonetic embeddings:

𝐄c​i​t\\displaystyle\\mathbf\{E\}\_\{cit\}=Projc​i​t⁡\(Softmax⁡\(𝐙c​i​t\)\),\\displaystyle=\\operatorname\{Proj\}\_\{cit\}\(\\operatorname\{Softmax\}\(\\mathbf\{Z\}\_\{cit\}\)\),\(3\)𝐄s​a​n\\displaystyle\\mathbf\{E\}\_\{san\}=Projs​a​n⁡\(Softmax⁡\(𝐙s​a​n\)\)\.\\displaystyle=\\operatorname\{Proj\}\_\{san\}\(\\operatorname\{Softmax\}\(\\mathbf\{Z\}\_\{san\}\)\)\.\(4\)Applying softmax grounds the projections in discrete phonetic probabilities, preventing a linear collapse of the acoustic states\.

Subsequently, we employ a dynamic gating mechanism to effectively fuse these features back into the main network\. Rather than simply adding the features, we concatenate the original acoustic states with the projected phonetic embeddings to form a joint representation𝐂=\[𝐇e​n​c;𝐄c​i​t;𝐄s​a​n\]∈ℝT×3​d\\mathbf\{C\}=\[\\mathbf\{H\}\_\{enc\};\\mathbf\{E\}\_\{cit\};\\mathbf\{E\}\_\{san\}\]\\in\\mathbb\{R\}^\{T\\times 3d\}\. A gate generator then processes this joint representation to output a frame\-level weight matrix𝐆∈ℝT×2\\mathbf\{G\}\\in\\mathbb\{R\}^\{T\\times 2\}:

𝐆=σ⁡\(𝐂𝐖g​a​t​e\+𝐛g​a​t​e\),\\mathbf\{G\}=\\sigma\(\\mathbf\{C\}\\mathbf\{W\}\_\{gate\}\+\\mathbf\{b\}\_\{gate\}\),\(5\)where𝐖g​a​t​e∈ℝ3​d×2\\mathbf\{W\}\_\{gate\}\\in\\mathbb\{R\}^\{3d\\times 2\}and𝐆=\[𝐠c​i​t,𝐠s​a​n\]\\mathbf\{G\}=\[\\mathbf\{g\}\_\{cit\},\\mathbf\{g\}\_\{san\}\]\. The physical significance of this dynamic gate directly mirrors the context\-dependent nature of Taiwanese tone sandhi\. Since tonal shifts occur strictly based on grammatical positions,𝐆\\mathbf\{G\}acts as an adaptive soft switch\. By observing both the acoustic context and the explicit phonetic hypotheses, the gate dynamically allocates attention between the underlying intent \(𝐠c​i​t\\mathbf\{g\}\_\{cit\}\) and the surface realization \(𝐠s​a​n\\mathbf\{g\}\_\{san\}\) frame by frame\.

To prevent the newly initialized gate from catastrophically interfering with the frozen Whisper backbone during the crucial early stages of fine\-tuning, we initialize the bias𝐛g​a​t​e\\mathbf\{b\}\_\{gate\}to a strong negative scalar \(−3\-3\)\. This insight\-driven initialization forces the initial gate values toward zero, compelling the model to rely on the robust pre\-trained features first and gradually learn to blend in the decoupled phonetic patches\. The final fused representation𝐇f​u​s​e​d\\mathbf\{H\}\_\{fused\}, which is subsequently fed to the decoder, is computed as:

𝐇f​u​s​e​d=𝐇e​n​c\+𝐠c​i​t⊙𝐄c​i​t\+𝐠s​a​n⊙𝐄s​a​n,\\mathbf\{H\}\_\{fused\}=\\mathbf\{H\}\_\{enc\}\+\\mathbf\{g\}\_\{cit\}\\odot\\mathbf\{E\}\_\{cit\}\+\\mathbf\{g\}\_\{san\}\\odot\\mathbf\{E\}\_\{san\},\(6\)where⊙\\odotrepresents element\-wise multiplication, with𝐠c​i​t\\mathbf\{g\}\_\{cit\}and𝐠s​a​n\\mathbf\{g\}\_\{san\}implicitly broadcasted across the hidden dimensiondd\.

### III\-CMulti\-Task Learning Objective

To actualize this decoupled architecture, the training objective must explicitly penalize entangled representations\. Our primary objective remains the standard sequence\-to\-sequence cross\-entropy lossℒA​S​R\\mathcal\{L\}\_\{ASR\}generated by the Whisper decoder for the final Taiwanese Hanzi transcriptions\. However, to provide the necessary phonetic grounding for our injection module, we must introduce auxiliary losses\. Since frame\-level forced alignment data is prohibitively expensive and largely unavailable for low\-resource languages, we formulate these as CTC objectives\. CTC naturally marginalizes over all possible unsegmented acoustic alignments\. Thus,ℒC​T​C\(c​i​t\)\\mathcal\{L\}\_\{CTC\}^\{\(cit\)\}explicitly guides the extraction of citation syllables, whileℒC​T​C\(s​a​n\)\\mathcal\{L\}\_\{CTC\}^\{\(san\)\}supervises the sandhi tone tracking:

ℒT​o​t​a​l=ℒA​S​R\+λc​i​t​ℒC​T​C\(c​i​t\)\+λs​a​n​ℒC​T​C\(s​a​n\),\\mathcal\{L\}\_\{Total\}=\\mathcal\{L\}\_\{ASR\}\+\\lambda\_\{cit\}\\mathcal\{L\}\_\{CTC\}^\{\(cit\)\}\+\\lambda\_\{san\}\\mathcal\{L\}\_\{CTC\}^\{\(san\)\},\(7\)whereλc​i​t\\lambda\_\{cit\}andλs​a​n\\lambda\_\{san\}are scalar hyperparameters\. This joint optimization ensures that the encoder effectively disentangles “what is heard” from “what is meant” before passing the representation to the decoder\.

## IVExperimental Setup

### IV\-ADual\-Track Supervision Strategy

Our data preparation goes beyond formatting to construct explicit supervision signals for the decoupled architecture\. Using the official MOE Taiwanese Dictionary222[https://sutian\.moe\.edu\.tw/zh\-hant](https://sutian.moe.edu.tw/zh-hant)and Taibun toolkit333[https://github\.com/andreihar/taibun](https://github.com/andreihar/taibun), we extracted two strictly parallel phonetic tracks from the Hanzi transcriptions: the citation dictionary forms and the contextual sandhi tones\. This dual\-track extraction yields the text\-derived pseudo labels to independently guide our auxiliary CTC streams under weak supervision\. Specifically, the citation vocabulary\|Vc​i​t\|\|V\_\{cit\}\|comprises 1309 unique syllables with tone marks, while the sandhi vocabulary\|Vs​a​n\|\|V\_\{san\}\|tracks 10 classes \(8 tones, a neutral tone, and a blank token\)\.

### IV\-BImplementation Details

We employed Whisper\-small\[[1](https://arxiv.org/html/2609.18194#bib.bib14)\]as our acoustic foundation\. To preserve its pre\-trained speech representations while adapting to Taiwanese Hokkien, we did not perform full fine\-tuning\. Instead, we applied AdaLoRA\[[28](https://arxiv.org/html/2609.18194#bib.bib6)\]to the attention and feed\-forward blocks with an initial rank of 12 \(pruned down to 4\) and a dropout rate of 0\.1 to ensure parameter efficiency\. To guarantee stable training for our decoupled hybrid injection module, the bias vector of the gate mechanism,𝐛gate\\mathbf\{b\}\_\{\\text\{gate\}\}, was initialized to−3\-3\. This configuration ensures that the model heavily relies on the robust frozen backbone during the initial training phase, gradually incorporating the auxiliary phonetic information as training converges\. For the multi\-task objective function \(Eq\.[7](https://arxiv.org/html/2609.18194#S3.E7)\), the CTC loss weightsλcit\\lambda\_\{\\text\{cit\}\}andλsan\\lambda\_\{\\text\{san\}\}were empirically set to 0\.9 and 0\.1, respectively\. This weighting ensures adequate phonetic supervision without overshadowing the primary sequence\-to\-sequence objective\.

TABLE III:CER \(%\) comparison on the TAT\-MOE, FSRC 2020, and yttd\_taigi\_trs blind test sets\.
### IV\-CEvaluation and Diagnostic Metrics

Our primary performance metric is the CER on Taiwanese Hanzi, which directly reflects practical utility\. However, to evaluate whether our architecture effectively addresses the localized mapping confusion between surface acoustics and underlying lexical intent, we must look beyond final transcription errors\. Therefore, we introduce two diagnostic metrics: syllable error rate \(SER\) to evaluate underlying lexical intent tracking, and tone error rate \(TER\) to measure surface tone resolution\. Tracking these metrics allows us to empirically verify that the performance gains stem directly from our explicit phonetic disentanglement\.

TABLE IV:CER \(%\) performance and relative reduction \(Rel\., %\) across Whisper scales on the TAT\-MOE corpus\.

## VResults and Discussion

### V\-AOverall ASR Performance

Table[III](https://arxiv.org/html/2609.18194#S4.T3)demonstrates that explicitly decoupling surface contextual features from underlying canonical features effectively mitigates the tonal mapping confusion caused by tone sandhi\. To ensure a rigorous evaluation, our baselines span three architectural paradigms: RNN\-Transducer \(RNN\-T\) models, pure CTC frameworks, and attention\-based encoder\-decoder \(AED\) foundation models\. Traditional end\-to\-end RNN\-T models struggle to implicitly memorize dynamic tonal mappings, as demonstrated by Zipformer\[[6](https://arxiv.org/html/2609.18194#bib.bib8)\], HuBERT\-base\[[2](https://arxiv.org/html/2609.18194#bib.bib24)\], and CLiFT\-ASR\[[39](https://arxiv.org/html/2609.18194#bib.bib11)\]yielding Test CERs of 45\.82%, 24\.49%, and 20\.94%, respectively\. Augmenting the foundation encoder with a standard CTC head \(Whisper\-small \(Encoder\-CTC\)\) improves performance but yields a suboptimal 17\.98% CER, constrained by a monolithic sequence loss\. Naive adaptations of the Whisper backbone similarly fall short: full fine\-tuning suffers from representation distortion \(18\.68%\), and standard AdaLoRA \(16\.30%\) remains bottlenecked by entangled phonetic features\.

Crucially, T\-SANDHI overcomes these limitations\. With only a 5M parameter overhead for the decoupled hybrid injection module, our framework achieves a 14\.30% Test CER, a 12\.27% relative error reduction over AdaLoRA\. Coupled with robust out\-of\-domain generalization on the FSRC 2020 \(7\.36%\) and yttd\_taigi\_trs \(36\.57%\) blind test sets, these results confirm that explicit phonetic disentanglement, rather than mere parameter scaling, drives the performance gains\.

### V\-BScalability Across Backbone Capacities

We further investigate T\-SANDHI’s scalability across larger foundation models\. As detailed in Table[IV](https://arxiv.org/html/2609.18194#S4.T4), the method yields consistent gains across all evaluated Whisper backbones\. While increasing model capacity naturally lowers the baseline error rate, our dual\-stream module extracts additional improvements, achieving relative error reductions of 12\.27%, 6\.38%, and 6\.91% on the small, medium, and large backbones, respectively\. This confirms that phonetic disentanglement provides orthogonal benefits to naive parameter scaling\.

TABLE V:Ablation studies of T\-SANDHI on the TAT\-MOE corpus\.Model ConfigurationSER %TER %CER %T\-SANDHI \(Full Model\)16\.2713\.9914\.30w/o Decoupled Streams\-\-16\.30w/o Citation CTC Head\-13\.5215\.44w/o Sandhi CTC Head16\.68\-15\.19w/o Gate Generator17\.4514\.3015\.55w/o Dynamic Gating17\.3614\.1915\.67
### V\-CAblation Studies

Table[V](https://arxiv.org/html/2609.18194#S5.T5)validates that our performance gains stem from explicit phonetic disentanglement\. Removing the dual\-stream architecture entirely regresses the CER to 16\.30%, indicating that the frozen backbone alone cannot fully resolve acoustic\-to\-lexical confusion\. Ablating individual CTC heads highlights the mechanics of tone sandhi resolution\. Omitting the citation head improves the surface TER \(13\.52%\) but degrades the final CER \(15\.44%\), as the model overfits surface acoustics and loses underlying Hanzi identity\. Conversely, removing the sandhi head worsens the SER \(16\.68%\), demonstrating that surface tracking provides essential acoustic grounding\.

Moreover, optimal integration requires dynamic fusion\. An unweighted summation degrades CER to 15\.55% by re\-entangling features, and using static global weights \(heuristically tuned on the development set\) yields a suboptimal 15\.67%\. Static weights fail to capture position\-dependent sandhi rules \(e\.g\., word\-medial mutation vs\. sentence\-final retention\), underscoring the necessity of a frame\-by\-frame neural soft\-switch\.

TABLE VI:Ablation study on the granularity of injected linguistic information for the dual streams\.Fig\. 4:Correlation between phonetic errors and transcription accuracy\.
### V\-DImpact of Information Granularity in Dual Streams

To determine the optimal feature resolution, we ablated the linguistic granularity injected into the dual streams\. Table[VI](https://arxiv.org/html/2609.18194#S5.T6)contrasts configuring either stream with phone\-level tone labels versus full syllable\-level targets\. Pairing citation syllables with surface tones yields the lowest CER \(14\.30%\), outperforming all symmetric configurations\.

Phonetically, the citation stream relies on full syllables to anchor lexical identity\. Conversely, the surface stream, tracking actual acoustic realizations, functions best when constrained to tone\-level variations\. This structural bottleneck prevents the sandhi head from overfitting to specific lexical items, effectively mitigating localized mapping confusion in low\-resource settings\.

### V\-ECorrelation between Phonetic and Character Errors

Fig\.[4](https://arxiv.org/html/2609.18194#S5.F4)plots final CER against auxiliary phonetic error rates, verifying that the decoder actively utilizes decoupled representations\. The strong positive correlation confirms that accurate phonetic grounding is a prerequisite for transcription\. Crucially, sandhi TER and citation SER trajectories diverge as errors increase\. The SER’s steeper slope indicates the citation syllable serves as the primary structural anchor; misidentifying it severely degrades character prediction\. The TER’s gentler slope suggests the decoder leverages contextual language modeling to tolerate minor tone errors\. Nevertheless, minimizing TER remains essential to resolve localized mapping confusion and push CER below 20%\. This hierarchy, where syllables provide structure and tones provide disambiguation, empirically justifies the dual\-stream strategy\.

### V\-FEmergent Phonological Awareness in Dynamic Gating

Fig\.[5](https://arxiv.org/html/2609.18194#S5.F5)visualizes dynamic gate behavior across a continuous utterance to illustrate mapping resolution\. During phrase\-internal mutations \(e\.g\.,Tsok4realized asGik8\), the network autonomously suppresses the surface stream \(𝐠san≈0\\mathbf\{g\}\_\{\\text\{san\}\}\\approx 0\) to filter deceptive acoustics, anchoring instead on the citation intent \(𝐠cit\\mathbf\{g\}\_\{\\text\{cit\}\}\)\. At the phrase\-final position \(Tsu1\), where surface acoustics and citation tones align,𝐠san\\mathbf\{g\}\_\{\\text\{san\}\}activates sharply \(𝐠san→1\\mathbf\{g\}\_\{\\text\{san\}\}\\to 1\)\. This demonstrates the network learns to rely on surface acoustics only when phonologically reliable\. These findings indicate an emergent, data\-driven awareness of right\-prominent sandhi rules within the gating mechanism\.

![Refer to caption](https://arxiv.org/html/2609.18194v1/dynamic_gate_visualization.png)Fig\. 5:Frame\-level visualization of the dynamic gate\.![Refer to caption](https://arxiv.org/html/2609.18194v1/error_example.png)Fig\. 6:Qualitative comparison of error resolution\.
### V\-GTargeted Error Analysis

Fig\.[6](https://arxiv.org/html/2609.18194#S5.F6)illustrates T\-SANDHI’s resolution of character mapping confusion\. In the phrase “

顛倒” \(tian\-tò, unexpectedly\), underlying citation tones \(T1\-T3\) mutate into surface tones \(T7\-T2\)\. Misled by these surface acoustics, the Whisper\-small \(AdaLoRA\) baseline incorrectly predicts the phonetically identical but semantically unrelated “

電討” \(tiān thó, citation T7\-T2\)\. This substitution highlights how conventional models, lacking explicit guidance, overfit surface tone variations and fail to recover lexical intent\. In contrast, T\-SANDHI successfully outputs the correct Hanzi\. By tracking surface realizations \(T7\-T2\) while anchoring on citation identity \(T1\-T3\), the decoupled framework bridges the phonological gap\. This frame\-by\-frame dynamic gating effectively resolves the confusion of entangled architectures\.

## VIConclusion and Future Work

In this paper, we proposed T\-SANDHI444Our source code:[https://anonymous\.4open\.science/r/T\-SANDHI\-0D38](https://anonymous.4open.science/r/T-SANDHI-0D38), a parameter\-efficient framework that explicitly disentangles surface acoustics from underlying lexical intent to mitigate tone sandhi mapping confusion\. By employing a dynamic gating mechanism, our approach achieves robust performance gains across foundation models without substantial parameter scaling\. While currently supervised by rule\-derived pseudo\-labels, future work will explore unsupervised disentanglement for undocumented dialects, as well as extending this decoupled paradigm to Mandarin\-Taiwanese code\-switching\.

## References

- \[1\]\(2023\)Robust speech recognition via large\-scale weak supervision\.InProc\. PMLR,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[§I](https://arxiv.org/html/2609.18194#S1.p4.1),[§IV\-B](https://arxiv.org/html/2609.18194#S4.SS2.p1.1)\.
- \[2\]W\. Hsu, B\. Bolte, Y\. H\. Tsai, K\. Lakhotia, R\. Salakhutdinov, and A\. Mohamed\(2021\)HuBERT: Self\-supervised speech representation learning by masked prediction of hidden units\.IEEE/ACM Transactions on Audio, Speech, and Language Processing29,pp\. 3451–3460\.Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[TABLE III](https://arxiv.org/html/2609.18194#S4.T3.4.4.1.1),[§V\-A](https://arxiv.org/html/2609.18194#S5.SS1.p1.1)\.
- \[3\]A\. Graves\(2012\)Sequence transduction with recurrent neural networks\.InProc\. ICML,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[4\]X\. Wang, Z\. Yao, X\. Shi, and L\. Xie\(2021\)Cascade RNN\-transducer: Syllable based streaming on\-device Mandarin speech recognition with a syllable\-to\-character converter\.InProc\. IEEE SLT,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[5\]M\. Ghodsi, X\. Liu, J\. Apfel, R\. Cabrera, and E\. Weinstein\(2020\)RNN\-transducer with stateless prediction network\.InProc\. ICASSP,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[6\]Z\. Yao, L\. Guo, X\. Yang, W\. Kang, F\. Kuang, Y\. Yang, Z\. Jin, L\. Lin, and D\. Povey\(2024\)Zipformer: A faster and better encoder for automatic speech recognition\.InProc\. ICLR,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[TABLE III](https://arxiv.org/html/2609.18194#S4.T3.4.3.1.1),[§V\-A](https://arxiv.org/html/2609.18194#S5.SS1.p1.1)\.
- \[7\]S\. Watanabe, T\. Hori, S\. Kim, J\. R\. Hershey, and T\. Hayashi\(2017\)Hybrid CTC/Attention architecture for end\-to\-end speech recognition\.IEEE Journal of Selected Topics in Signal Processing11\(8\),pp\. 1240–1253\.Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[8\]H\. Khoo\(2019\)The dynamics of Southern Min in Taiwan: From Southern Min dialects to “Taigi”\.InThe Routledge Handbook of Chinese Discourse Analysis,C\. Shei \(Ed\.\),pp\. 596–610\.Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[9\]C\. Chen, Y\. Yeh, C\. Lin, S\. Tseng, and J\. Wang\(2020\)Hybrid deep neural network acoustic model for Taiwanese speech recognition\.InProc\. IEEE ICOT,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[10\]P\. Chen, C\. Wu, H\. Lee, S\. Tsao, M\. Ko, and H\. Wang\(2020\)Using Taigi dramas with Mandarin Chinese subtitles to improve Taigi speech recognition\.InProc\. O\-COCOSDA,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[§II\-A](https://arxiv.org/html/2609.18194#S2.SS1.p1.1)\.
- \[11\]M\. Hsieh\(2014\)Taiwanese Hokkien/Southern Min\.InThe Handbook of Chinese Linguistics,pp\. 629–656\.Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[12\]Y\. Liao, W\. Hsu, C\. Pan, W\. Wang, M\. Pleva, and D\. Hladek\(2022\)Personalized Taiwanese speech synthesis using cascaded ASR and TTS framework\.InProc\. RADIOELEKTRONIKA,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[13\]Y\. Chou, K\. Chang, M\. Wu, W\. Ou, A\. W\. Bi, C\. Yang, B\. Y\. Chen, R\. Pai, P\. Yeh, J\. Chiang, I\. Phoann, W\. Chang, C\. Cui, N\. Chen, and J\. Shi\(2023\)Evaluating self\-supervised speech models on a Taiwanese Hokkien corpus\.InProc\. IEEE ASRU,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[14\]J\. Lin, S\. Lu, H\. Huang, W\. Guan, B\. Xu, H\. Bu, Q\. Hong, and L\. Li\(2024\)MinSpeech: A corpus of Southern Min dialect for automatic speech recognition\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[15\]B\. Zhang, H\. Lv, P\. Guo, Q\. Shao, C\. Yang, L\. Xie, X\. Xu, H\. Bu, X\. Chen, C\. Zeng, D\. Wu, and Z\. Peng\(2022\)WenetSpeech: A 10000\+ hours multi\-domain Mandarin corpus for speech recognition\.InProc\. ICASSP,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[16\]H\. Bu, J\. Du, X\. Na, B\. Wu, and H\. Zheng\(2017\)AISHELL\-1: An open\-source Mandarin speech corpus and a speech recognition baseline\.InProc\. O\-COCOSDA,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[17\]Y\. Fu, L\. Cheng, S\. Lv, Y\. Jv, Y\. Kong, Z\. Chen, Y\. Hu, L\. Xie, J\. Wu, H\. Bu, X\. Xu, J\. Du, and J\. Chen\(2021\)AISHELL\-4: An open source dataset for speech enhancement, separation, recognition and speaker diarization in conference scenario\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[18\]R\. L\. Cheng\(1968\)Tone sandhi in Taiwanese\.Linguistics6\(41\),pp\. 19–42\.Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[19\]Y\. Chien and A\. Jongman\(2019\)Tonal neutralization of Taiwanese checked and smooth syllables: An acoustic study\.Language and Speech62\(3\),pp\. 452–474\.Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[20\]Y\. Chuang and S\. Wang\(2025\)Tonal variation and word meaning in Taiwanese\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[§II\-A](https://arxiv.org/html/2609.18194#S2.SS1.p1.1),[§II\-B](https://arxiv.org/html/2609.18194#S2.SS2.p1.1)\.
- \[21\]P\. Chen, K\. Tran, Y\. Yang, J\. Du, J\. Kao, Y\. Chung, P\. Tomasello, P\. Duquenne, H\. Schwenk, H\. Gong, H\. Inaguma, S\. Popuri, C\. Wang, J\. Pino, W\. Hsu, and A\. Lee\(2023\)Speech\-to\-speech translation for a real\-world unwritten language\.InFindings of ACL,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1),[§I](https://arxiv.org/html/2609.18194#S1.p2.1),[§II\-A](https://arxiv.org/html/2609.18194#S2.SS1.p1.1)\.
- \[22\]J\. Myers and J\. Tsay\(2008\)Neutralization in Taiwan Southern Min Tone Sandhi\.Interfaces in Chinese Phonology\.Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p1.1)\.
- \[23\]G\. Shen, M\. Watkins, A\. Alishahi, A\. Bisazza, and G\. Chrupała\(2024\)Encoding of lexical tone in self\-supervised models of spoken language\.InProc\. NAACL,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[24\]S\. Bandarupalli, B\. Akkiraju, C\. Devarakonda, V\. Narsinga, and A\. K\. Vuppala\(2025\)Efficient ASR for low\-resource languages: Leveraging cross\-lingual unlabeled data\.InFindings of IJCNLP,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[25\]O\. Klejch, W\. Lamb, and P\. Bell\(2025\)A practitioner’s guide to building ASR models for low\-resource languages: a case study on Scottish Gaelic\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[26\]M\. Bartelds, N\. San, B\. McDonnell, D\. Jurafsky, and M\. Wieling\(2023\)Making more of little data: Improving low\-resource automatic speech recognition using data augmentation\.InProc\. ACL,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[27\]C\. Jacobs, A\. Smith, D\. Klop, O\. Klejch, F\. de Wet, and H\. Kamper\(2025\)Speech recognition for automatically assessing Afrikaans and IsiXhosa preschool oral narratives\.InProc\. ICASSP,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[28\]Q\. Zhang, M\. Chen, A\. Bukharin, N\. Karampatziakis, P\. He, Y\. Cheng, W\. Chen, and T\. Zhao\(2023\)AdaLoRA: Adaptive budget allocation for parameter\-efficient fine\-tuning\.InProc\. ICLR,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1),[§III\-A](https://arxiv.org/html/2609.18194#S3.SS1.p1.1),[§IV\-B](https://arxiv.org/html/2609.18194#S4.SS2.p1.1)\.
- \[29\]Z\. Song, J\. Zhuo, Y\. Yang, Z\. Ma, S\. Zhang, and X\. Chen\(2024\)LoRA\-Whisper: Parameter\-Efficient and Extensible Multilingual ASR\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1),[§I](https://arxiv.org/html/2609.18194#S1.p4.1)\.
- \[30\]T\. Tan, X\. Chen, X\. Le, W\. Fan, X\. Xia, C\. Huang, and J\. Lu\(2025\)CBA\-Whisper: curriculum learning\-based AdaLoRA fine\-tuning on whisper for low\-resource dysarthric speech recognition\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p2.1)\.
- \[31\]W\. Liu, Y\. Qin, Z\. Peng, and T\. Lee\(2024\)Sparsely Shared Lora on Whisper for Child Speech Recognition\.InProc\. ICASSP,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p4.1)\.
- \[32\]A\. Graves, S\. Fernández, F\. Gomez, and J\. Schmidhuber\(2006\)Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks\.InProc\. ICML,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p4.1)\.
- \[33\]S\. Kim, T\. Hori, and S\. Watanabe\(2017\)Joint CTC\-attention based end\-to\-end speech recognition using multi\-task learning\.InProc\. ICASSP,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p4.1)\.
- \[34\]K\. Hojo, Y\. Wakabayashi, K\. Ohta, A\. Ogawa, and N\. Kitaoka\(2024\)Boosting CTC\-based ASR using inter\-layer attention\-based CTC loss\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p4.1)\.
- \[35\]S\. Han, M\. Xu, Z\. Lei, Z\. Huang, and X\. Na\(2024\)Enhancing CTC\-based speech recognition with diverse modeling units\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p4.1)\.
- \[36\]N\. Kusunoki, Y\. Higuchi, T\. Ogawa, and T\. Kobayashi\(2024\)Hierarchical multi\-task learning with CTC and recursive operation\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2609.18194#S1.p4.1)\.
- \[37\]Y\. Liao, J\. S\. Tsay, P\. Kang, H\. Khoo, L\. Tan, L\. Chang, U\. Iunn, H\. Su, T\. Thiann, H\. Tiun, and S\. Liao\(2022\)Taiwanese across Taiwan corpus and its applications\.InProc\. O\-COCOSDA,Cited by:[item 3](https://arxiv.org/html/2609.18194#S1.I1.i3.p1.1),[§II\-A](https://arxiv.org/html/2609.18194#S2.SS1.p1.1)\.
- \[38\]Y\. Liao, C\. Chang, H\. Tiun, H\. Su, H\. Khoo, J\. S\. Tsay, L\. Tan, P\. Kang, T\. Thiann, U\. Iunn, J\. Yang, and C\. Liang\(2020\)Formosa speech recognition challenge 2020 and Taiwanese across Taiwan corpus\.InProc\. O\-COCOSDA,Cited by:[§II\-A](https://arxiv.org/html/2609.18194#S2.SS1.p1.1)\.
- \[39\]H\. Sung, C\. Wang, K\. Huang, T\. Lo, Y\. Tsao, Y\. Hsu, and B\. Chen\(2025\)CLiFT\-ASR: A cross\-lingual fine\-tuning framework for low\-resource Taiwanese Hokkien speech recognition\.InProc\. ROCLING,Cited by:[TABLE III](https://arxiv.org/html/2609.18194#S4.T3.4.5.1.1),[§V\-A](https://arxiv.org/html/2609.18194#S5.SS1.p1.1)\.

Similar Articles

Tone-Conditioned Curriculum Learning for Low-Resource Bantu Speech Recognition

arXiv cs.CL

This paper proposes a tone-conditioned curriculum learning framework for low-resource Bantu speech recognition, combining hybrid difficulty scoring, gated adapters, and staged curriculum training. Evaluations on six Southern Bantu languages show that W2V-BERT outperforms Whisper on Nguni languages while Whisper performs better on Sotho-Tswana languages.

DialectS2S: End-to-End Speech Dialogue Modeling for Low-Resource Chinese Dialects

arXiv cs.CL

DialectS2S is an end-to-end speech dialogue model for low-resource Chinese dialects, introducing a scalable data synthesis pipeline and a two-stage post-training strategy with self-aligned speech supervision. Experiments show improvements in dialect consistency, response quality, and intelligibility, with fully open-sourced models, data, and code.