自发语音的噪声鲁棒强制对齐

# NOISE-ROBUST FORCED ALIGNMENT FOR SPONTANEOUS SPEECH
Source: [https://arxiv.org/html/2610.22583](https://arxiv.org/html/2610.22583)
###### Abstract

We show that forced aligners degrade on spontaneous, noisy real\-world speech, and that fillers, laughter, and disfluencies corrupt word boundaries, impacting many downstream applications\. We revisit the classical GMM\-HMM aligner with upsampled WavLM features instead of MFCCs, explicit phone units for fillers and non\-lexical vocal events trained on verbatim\-annotated spontaneous speech, and a discriminative projection fitted on clean and noise\-corrupted feature pairs to improve robustness\. The pipeline is self\-bootstrapped from 149 h of curated audio without manual or external boundary targets\. On conversational \(Buckeye\) and disfluent \(FluencyBank\) speech it reaches word\-boundary MAEs of 20\.6 and 76\.9 ms, 19% and 17% lower than the strongest of ten baselines\. Catastrophic misalignments \(\>\>200 ms\) are2\.7×2\.7\\timesrarer than Montreal Forced Aligners \(MFA\) best model even after widening MFA’s search beam, which we show removes a large pruning artifact from its conversational error; event units halve the error next to laughter, while being meaningfully more robust than existing aligners\.

###### Index Terms:

forced alignment, disfluencies, conversational speech, self\-supervised features, noise robustness

††address:nyra labs, Vienna, Austria## 1Introduction

Word\- and phone\-level timing are foundational signals in speech processing\. Clinical speech biomarkers and fluency assessments rely on accurately timestamped transcriptions, from which pause distributions, articulation rates, and the locations of repetitions, cutoffs, and prolongations are derived\[[45](https://arxiv.org/html/2610.22583#bib.bib35),[37](https://arxiv.org/html/2610.22583#bib.bib36),[31](https://arxiv.org/html/2610.22583#bib.bib12)\]\. Speech corpus construction relies on precise timestamps to segment long\-form audio into utterance\-sized units\[[21](https://arxiv.org/html/2610.22583#bib.bib15),[5](https://arxiv.org/html/2610.22583#bib.bib16),[8](https://arxiv.org/html/2610.22583#bib.bib17),[43](https://arxiv.org/html/2610.22583#bib.bib18),[12](https://arxiv.org/html/2610.22583#bib.bib19),[17](https://arxiv.org/html/2610.22583#bib.bib20)\]\. Beyond data preparation, accurate timing signals enable the design of training objectives: duration targets for non\-autoregressive synthesis\[[32](https://arxiv.org/html/2610.22583#bib.bib21)\], emission\-time regularization for streaming recognition\[[42](https://arxiv.org/html/2610.22583#bib.bib22),[14](https://arxiv.org/html/2610.22583#bib.bib23)\], or word\-level alignment to derive speaker\-invariant content representations\[[38](https://arxiv.org/html/2610.22583#bib.bib24)\]\. Improving timing accuracy therefore strengthens the learning signal of downstream approaches that rely on such supervision, while systematic boundary errors enter it as label noise that makes training harder\. On the application side, subtitling\[[22](https://arxiv.org/html/2610.22583#bib.bib25)\], automated speech editing\[[23](https://arxiv.org/html/2610.22583#bib.bib33),[2](https://arxiv.org/html/2610.22583#bib.bib34)\], and automatic dubbing\[[7](https://arxiv.org/html/2610.22583#bib.bib32)\]are directly facilitated by accurate and robust alignment\.

Existing approaches to producing such timestamps fall into several families\. Classical HMM\-based systems, most prominently the Montreal Forced Aligner \(MFA\)\[[20](https://arxiv.org/html/2610.22583#bib.bib1),[19](https://arxiv.org/html/2610.22583#bib.bib2)\], remain the de facto standard and are methodologically closest to our work: a Kaldi\-based\[[25](https://arxiv.org/html/2610.22583#bib.bib9)\]GMM\-HMM triphone pipeline over MFCC features\. Frame\-classification systems score each frame with a discriminative phone classifier and recover boundaries by monotonically aligning the transcript’s phone sequence to the resulting posteriors, without the HMM’s transition and duration model; they require frame\-level phone labels for training, either human\-annotated boundaries\[[15](https://arxiv.org/html/2610.22583#bib.bib30)\]or the output of an HMM aligner whose errors they inherit\[[44](https://arxiv.org/html/2610.22583#bib.bib6)\]\. The Qwen forced aligner\[[27](https://arxiv.org/html/2610.22583#bib.bib8)\]interleaves timestamp\-slot tokens with the transcript and predicts all slots jointly in a single non\-autoregressive pass over an LLM backbone, distilling and smoothing MFA\-generated pseudo\-labels as its timing supervision\. Attention\-based methods derive timestamps from decoder cross\-attention patterns, popularized by Whisper\[[28](https://arxiv.org/html/2610.22583#bib.bib26)\]and subsequently made substantially more accurate in CrisperWhisper\[[46](https://arxiv.org/html/2610.22583#bib.bib27)\]and CrisperWhisper 2\.0\[[39](https://arxiv.org/html/2610.22583#bib.bib28)\]\. These improvements arose from tokenizer changes and explicitly supervising cross\-attention weights with MFA alignments during training\. CTC\-based approaches extract timestamps from the alignment lattices of CTC\-trained ASR models\[[18](https://arxiv.org/html/2610.22583#bib.bib7),[3](https://arxiv.org/html/2610.22583#bib.bib4),[26](https://arxiv.org/html/2610.22583#bib.bib5),[30](https://arxiv.org/html/2610.22583#bib.bib31)\], whose peaky, blank\-dominated posteriors are known to bias boundaries \(visible in Sec\.[5\.1](https://arxiv.org/html/2610.22583#S5.SS1)as low tight\-tolerance accuracy\)\. Continuous integrate\-and\-fire models predict token firing times inside a non\-autoregressive recognizer\[[35](https://arxiv.org/html/2610.22583#bib.bib29)\]\. Across these families, no system closes all of the following gaps jointly with sufficient quality \(Table[1](https://arxiv.org/html/2610.22583#S1.T1)\)\. First,*noise robustness*: MFA’s MFCC front end degrades steeply under additive noise, and no aligner we are aware of explicitly learns noise\-invariant alignment features\. The large\-scale pretraining behind CTC\-based models, Whisper, CrisperWhisper 2\.0, and Qwen3\-FA confers partial robustness to the recognizer, but the timing mechanisms themselves \(emergent cross\-attention patterns; CTC posteriors, timestamps distilled from an aligner that itself fails in noise\) remain unprotected\. Second,*explicit modeling of fillers and non\-lexical vocal sound events*such as laughter or breathing: outside CrisperWhisper 2\.0, these phenomena are absent from the modeled inventory, so their acoustics are easily absorbed by neighboring words or silence, corrupting boundaries\. Finally, little attention is paid to the*accuracy of the training annotations*themselves: systems are trained on read speech or on spontaneous speech with non\-verbatim transcripts, yet verbatim annotation is what makes spontaneous speech learnable for an aligner because every untranscribed hesitation or laugh is a mislabeled stretch of audio at training time\.

We close these gaps with a GMM\-HMM aligner that is fully self\-bootstrapped and sets a new state of the art on conversational and disfluent speech, improving over ten competitive evaluated baselines\. Our contributions are:

- •*Noise\-robust feature extraction*: a WavLM\-based front end with a noise\-paired discriminative projection that enables accurate GMM\-HMM alignment in real\-world acoustic conditions\.
- •*Explicit modeling of spontaneous speech*: dedicated acoustic units for fillers and non\-lexical vocal events, which substantially improve alignment around these phenomena\.
- •*Verbatim data curation*: a curated verbatim spontaneous\-speech corpus, together with a quantitative evaluation focused on disfluencies and vocal events\.

Table 1:Capability grid\. Lbl\.\-free = trained without manual or external boundary labels;∼\\sim= partial robustness inherited from augmentation or large\-scale pretraining\.
## 2Training data

Table[2](https://arxiv.org/html/2610.22583#S2.T2)summarizes the seven training sources, totaling 149 h, roughly half of it conversational\. Conversational speech with verbatim annotations comes from the IHM splits of the ICSI\[[13](https://arxiv.org/html/2610.22583#bib.bib37)\]and AMI\[[4](https://arxiv.org/html/2610.22583#bib.bib38)\]meeting corpora, from DisfluencySpeech\[[40](https://arxiv.org/html/2610.22583#bib.bib44)\], a studio\-quality single\-speaker re\-enactment of conversational utterances dense in disfluencies, and from part 6 of the Singapore National Speech Corpus\[[16](https://arxiv.org/html/2610.22583#bib.bib39)\], which adds non\-native accents\. All four provide verbatim transcripts including fillers; ICSI and DisfluencySpeech further contribute annotations of vocal sounds, predominantly laughter\. For read speech, Common Voice 14\[[1](https://arxiv.org/html/2610.22583#bib.bib40)\]adds diverse recording conditions and speaker diversity, VCTK\[[41](https://arxiv.org/html/2610.22583#bib.bib41)\]adds speaker diversity under studio conditions, and LibriSpeech\[[21](https://arxiv.org/html/2610.22583#bib.bib15)\]contributes clean read speech\. All transcripts follow the verbatim conventions of Sec\.[3\.1](https://arxiv.org/html/2610.22583#S3.SS1)\.

Vocal\-event overlay\.To increase the diversity and number of vocal sound events seen in training, so that the aligner can handle a larger vocal\-event inventory at inference, we take clips from LibriSpeech’s train\-clean split in which MFA detects pauses of at least 200 ms and insert vocal sound event clips from VocalSound\[[10](https://arxiv.org/html/2610.22583#bib.bib42)\]and Nonspeech7k\[[29](https://arxiv.org/html/2610.22583#bib.bib43)\]into these pauses, adding the corresponding tags at the same transcript positions\. MFA is used only to identify candidate insertion regions; its boundaries never serve as acoustic\-model training targets\. Since its timestamps are reliable on clean read speech \(Table[3](https://arxiv.org/html/2610.22583#S5.T3)\), inserted events land in genuine silence rather than on top of speech, providing clean, well\-separated examples for each event unit\.

Table 2:Training corpora\. Transcripts follow a common verbatim convention; event tags are included where annotated or synthetically inserted\.
## 3Method

### 3\.1Phonemization

Words are converted to pronunciations with theespeak\-nggrapheme\-to\-phoneme engine: its IPA output, stripped of stress and length marks, is tokenized and mapped through a hand\-crafted table onto a canonical inventory of 39 English phones\. This inventory is extended with dedicated units for the fillers \[UH\] and \[UM\] and twelve non\-lexical vocal events \(laughter, breath, cough, sigh, …\)\. We adopt the verbatim transcription conventions of\[[39](https://arxiv.org/html/2610.22583#bib.bib28)\], in which fillers and vocal events appear inline as bracketed tags \(e\.g\., \[UM\], \[laughter\]\); each tag maps to a one\-phone lexicon entry, giving each event its own acoustics and duration statistics\. The event units are supervised solely by tag occurrences in the verbatim training transcripts \(Sec\.[2](https://arxiv.org/html/2610.22583#S2)\)\. Where a standard aligner must learn to ignore a laugh or smear it into neighboring words, ours assigns it a unit with matching acoustics\.

### 3\.2Self\-bootstrapped WavLM\-GMM aligner

We retain the classical Kaldi\[[25](https://arxiv.org/html/2610.22583#bib.bib9)\]GMM\-HMM alignment architecture of MFA\-style systems and change only what the models observe\. Every unit in the phoneme inventory is modeled as a three\-state left\-to\-right HMM\. Pronunciations are generated as described in Sec\.[3\.1](https://arxiv.org/html/2610.22583#S3.SS1)\.

Features\.A frozen WavLM\-Large\[[6](https://arxiv.org/html/2610.22583#bib.bib3)\]encoder replaces the MFCC front\-end\. WavLM is pretrained for robustness to acoustic corruption, making it a natural candidate for extracting noise\-invariant alignment features\. We choose the hidden states of transformer layer 22, which have been found to correspond most closely to phonemes\[[6](https://arxiv.org/html/2610.22583#bib.bib3)\]\(1024\-d, native 20 ms stride\)\. We linearly interpolate the representations to a 10 ms grid and reduce them to 39 dimensions by a PCA fit on two million frames uniformly subsampled from our training corpus\. The 39\-d bottleneck mirrors the 39\-dimensional MFCC front\-end of the original MFA\[[20](https://arxiv.org/html/2610.22583#bib.bib1)\]\. It is also necessary in practice: flat\-start GMM training directly on the high\-dimensional WavLM features was unstable in our experiments\.

Training\.The pipeline largely follows the staged monophone\-to\-triphone recipe of the original MFA\[[20](https://arxiv.org/html/2610.22583#bib.bib1)\], with every stage supervised only by its own alignments\. A monophone model is flat\-started directly on the 39\-d PCA features following the standard Kaldi recipe\[[25](https://arxiv.org/html/2610.22583#bib.bib9),[20](https://arxiv.org/html/2610.22583#bib.bib1)\]: the first alignment divides each utterance’s frames equally among its transcript phones, and 40 EM iterations follow, realigning after every iteration with silence boosting\. After obtaining monophone state alignments, we return to the WavLM representations, splice them over±\\pm1 frame, and fit a 39\-dimensional linear discriminant analysis \(LDA\) projection\[[11](https://arxiv.org/html/2610.22583#bib.bib14)\]supervised by the monophone’s*own*per\-frame state labels\. In this space, a triphone model \(2500 tree leaves, 6000 Gaussians\) is trained for 35 EM iterations, realigning at iterations 10, 20, and 30\. Unlike MFA, we apply no speaker adaptation and no maximum\-likelihood linear transform \(MLLT\) of the features, since neither improved boundary accuracy in our ablations\. After extracting the WavLM features on GPU, training completes in hours on CPU; new audio is aligned analogous to MFA\.

### 3\.3Noise\-robust feature projection

We train two variants that differ only in the data used to fit the LDA projection\. The standard variant fits it on the clean training features\. The noise\-robust variant fits it on clean features together with noise\-corrupted copies of the same utterances, mixed with MUSAN music\[[36](https://arxiv.org/html/2610.22583#bib.bib13)\]at 5–15 dB SNR; both copies carry identical frame labels, since additive noise does not change the reference boundaries\. Fisher’s criterion then folds the noise variation into the within\-class scatter, selecting directions that separate phones while remaining invariant to the noise\.

## 4Experimental setup and metrics

Benchmarks\.We evaluate on TIMIT\[[9](https://arxiv.org/html/2610.22583#bib.bib11)\]\(read\), Buckeye\[[24](https://arxiv.org/html/2610.22583#bib.bib10)\]\(conversational\), and FluencyBank Timestamped\[[33](https://arxiv.org/html/2610.22583#bib.bib46)\]\(speech from people who stutter\)\. For noise robustness, Buckeye is mixed with held\-out MUSAN noise clips and synthetic Gaussian noise at signal\-to\-noise ratios \(SNRs\) of 20, 10, 5, and 0 dB\. TIMIT, Buckeye, and FluencyBank are fully held out from training\.

Protocol\.For forced aligners, predicted words correspond one\-to\-one to the reference transcript\. For ASR\-based systems, whose hypotheses can deviate from the reference, we align hypothesis and reference words with a Levenshtein alignment over canonicalized word forms and score only exactly matched words\. All metrics are matched\-token statistics\. Non\-lexical event tokens are excluded from word\-level metrics for*every*system for comparability\. MFA baselines use CLI 3\.3\.9 with the latest released modelsenglish\_mfa3\.1\.0 andenglish\_us\_arpa3\.0\.0;*MFAour\_data*is MFA’s unmodified training recipe trained on our 149 h corpus; it differs from the released models only in its training data \(Sec\.[2](https://arxiv.org/html/2610.22583#S2)\)\.

Decoding beams\.MFA ships with a Viterbi beam of 10 \(retry 40\)\. We found on spontaneous speech these defaults prune the optimal path: alignments silently degrade \(beam 400 with retry 4000 improvesus\_arpaon Buckeye from 48\.0 to 25\.3 ms MAE;english\_mfaandour\_datashift similarly, 48\.9→\\to35\.6 and 33\.9→\\to24\.2\), and under noise most utterances fail to align at all under low beam sizes \(Sec\.[5\.3](https://arxiv.org/html/2610.22583#S5.SS3)\); read speech is unaffected\. Our MFA rows therefore use the wide beam unless marked\. Our aligner is far more robust to this choice: in a sweep from beam 5 to 2000, its accuracy is essentially beam\-independent \(20\.3 ms\) and converged from beam 20\. The gap lies in the acoustic model: pruning discards any path falling more than the beam size below the momentary best, so a narrow beam is safe only if the emission likelihoods keep the optimal path near the top at every frame\. The GMMs on our features separate phone classes sharply enough; overlapping MFCC densities let the correct path fall behind on disfluent or noisy frames, where it is pruned\.

Metrics\.For each matched word, letδ\\deltabe the mean of its onset and offset timestamp errors \(absolute values\) against the manual reference\. We report the mean ofδ\\delta\(MAE\)\. Threshold accuracies acc≤t,t∈\{25,50,100,200\}t\\in\\\{25,50,100,200\\\}ms, give the share of words whose absolute offset error is at mosttt, following the convention of\[[34](https://arxiv.org/html/2610.22583#bib.bib45),[19](https://arxiv.org/html/2610.22583#bib.bib2)\]\.

## 5Results

### 5\.1Main comparison and component ladder

Table 3:Word\-boundary MAE \(ms,↓\\downarrow\) and offset threshold accuracies \(%,↑\\uparrow\) on Buckeye \(conversational\), TIMIT \(read\), and FluencyBank \(disfluent\)\. All WavLM arms include the LDA projection of Sec\.[3](https://arxiv.org/html/2610.22583#S3); the final row replaces it with the noise\-paired LDA, all other components unchanged\.Table[3](https://arxiv.org/html/2610.22583#S5.T3)reports, per dataset, word\-boundary MAE together with the offset threshold accuracies; rows are grouped into competing systems, the MFA family, and our system with its component ablations\. We adopt the noise\-robust arm as the final system: it matches the clean flagship on Buckeye while adding the robustness of Sec\.[5\.3](https://arxiv.org/html/2610.22583#S5.SS3)\.

On read speech, we are roughly on par with the strongest classical configurations: MFA’sus\_arpaleads by 1\.9 ms MAE \(18\.6 vs\. 20\.5\) and produces marginally tighter boundaries at the tightest tolerances \(Fig\.[1](https://arxiv.org/html/2610.22583#S5.F1)\)\. On conversational and disfluent speech, baselines degrade steeply even at the wide beam:us\_arparises to 25\.3 ms on Buckeye and 122\.0 ms on FluencyBank \(6\.6×6\.6\\timesits read\-speech error\), and at MFA’s package defaults to 48\.0 and 172\.3 ms; MFAour\_datareaches 24\.2 on Buckeye, indicating that data alone helps the modeling of spontaneous speech, whereas our error barely moves between read and conversational speech \(20\.5 vs\. 20\.6 ms\) and rises far more gently on impaired speech \(76\.9 ms\)\. This yields a 19% margin over the best baseline on Buckeye and 17% on FluencyBank \(over CrisperWhisper 2\.0, itself verbatim\-trained\)\.

Figure 1:Complementary CDF \(log–log\) of the per\-word boundary errorδ\\deltaon Buckeye, all aligners at converged \(wide\-beam\) search: each curve gives the share of words misaligned by more thanttms\. Ours is the only system whose tail falls below 1% before 100 ms\.Figure[1](https://arxiv.org/html/2610.22583#S5.F1)shows the error profiles behind these aggregates\. In the typical\-word regime \(left of∼\\sim30 ms\) the HMM systems are nearly indistinguishable; MFA’sus\_arpais even marginally sharper \(median 13\.2 vs\. our 14\.0 ms\)\. From there the profiles separate: our curve runs steady below MFA’s at every severity\. 99% of our words land within 97 ms of the reference versus 196 ms for MFA’s best configuration; errors beyond 200 ms are2\.7×2\.7\\timesrarer \(0\.37% vs\. 0\.98%\), beyond 1 s3×3\\timesrarer \(0\.07% vs\. 0\.21%\), and averaged over each system’s worst 5% of words our error is 122 ms against MFA’s 220\. At its package defaults MFA deteriorates further \(99th percentile 1012 ms, worst\-5% mean 655 ms\)\. The other baselines fail in characteristic ways: NeMo shows the inverse profile \(imprecise everywhere, few catastrophes\), Qwen3\-FA shows an MFA\-like mid\-tail \(3×3\\timesour\>\>200 ms rate\), and CrisperWhisper 2\.0’s tail still sits above ours at every threshold\. This demonstrates a favorable error profile on spontaneous speech compared to existing aligners that benefits downstream use cases: typical\-word precision on par with the best classical system, with much fewer severe errors\.

### 5\.2Alignment around disfluencies

Table 4:Buckeye MAE \(ms,↓\\downarrow\) on regular words adjacent to a filler \(nn=2,656\), a laugh \(nn=722\), or a word cutoff \(nn=1,369\)\.Table[4](https://arxiv.org/html/2610.22583#S5.T4)isolates the regular words whose immediate neighbor is a filler, a laugh, or a word cutoff\. Our aligner achieves 26–29 ms next to fillers and cutoffs, within 10 ms of its dataset\-wide Buckeye MAE, and 53 ms next to laughter\. Even at the wide beam, every MFA configuration inflates next to a filler \(1\.41\.4–2\.3×2\.3\\timesits own dataset\-wide error;us\_arpa: 40\.9 vs\. 25\.3\) and shows 120–230 ms next to laughter \(2\.32\.3–4\.4×4\.4\\timesours\)\. While MFA retrained end\-to\-end on our data mix improves meaningfully over the released models, adding dedicated units for vocal sound events proves to be an important modeling decision, reducing laughter\-adjacent error a further2\.3×2\.3\\times\. The component ladder separates the causes: with verbatim transcripts fillers can be represented as lexical entries, so filler units only provide a small gain on filler adjacency \(47\.5→\\to41\.9 already in the MFCC arm; the front\-end contributes the rest,→\\to28\.2\) but laughter has no spelling, and there the units halve the error in the WavLM arm \(107\.8→\\to52\.4\) and cut it2\.7×2\.7\\timesin the MFCC arm \(215\.6→\\to79\.7\)\. Among baselines, only CrisperWhisper 2\.0, the one other system modeling vocal events, comes near on laughter adjacency \(82\.3 ms\)\.

### 5\.3Noise robustness

Table 5:Buckeye word\-boundary MAE \(ms,↓\\downarrow\) under additive noise at SNRs of 20 to 0 dB \(column headers\): held\-out MUSAN noise clips and unseen Gaussian noise \(training augmentation: MUSAN*music*only\)\.In Table[5](https://arxiv.org/html/2610.22583#S5.T5), the front\-end ablation isolates where robustness comes from: swapping MFCC for WavLM roughly halves the error at 0 dB SNR under both families \(MUSAN 170→\\to74 ms, Gaussian 273→\\to136\), and the noise\-paired projection improves it again \(50 / 96 ms\); since the pairing saw only MUSAN music in training, the Gaussian columns measure generalization to an unseen noise family\. At default beams the MFA package fails on most utterances under heavy noise, exiting with alignment errors \(english\_mfaretains 1\.4% of words at 0 dB SNR under Gaussian noise\); the MFA rows use the wide beam of Sec\.[4](https://arxiv.org/html/2610.22583#S4), restoring full coverage\. At 0 dB SNR under Gaussian noise, MFAenglish\_mfamisplaces words by 2\.0 s on average,us\_arpaby 417 ms, MFAour\_databy 298 ms, WhisperX by 2\.7 s, Qwen3\-FA by 339 ms, and ours by 96 ms\. Among CTC aligners, robustness performance tracks the stability of the acoustic model: MMS\-FA and NeMo, trained on large, diverse corpora, degrade mildly \(53→\\to99, 73→\\to101 ms\), while WhisperX’s wav2vec2 model, trained on clean read speech, collapses\. CrisperWhisper 2\.0 transcribes rather than aligns: it recognizes only 29% of words at Gaussian 0 dB SNR, so its MAEs cover those words alone and are not directly comparable\.

## 6Conclusion and Future Work

Adjusting key design decisions in training data, phonemization, and feature extraction sets a new state of the art in spontaneous\-speech alignment under noise\. Multilingual verbatim transcription \(CrisperWhisper 2\.0\) makes extending the approach beyond English feasible\.

## 7Compliance with Ethical Standards

This study is a purely algorithmic contribution using publicly available benchmark datasets, for which no ethical approval was required\.

## 8Acknowledgments

No funding was received for conducting this study\. The authors have no relevant financial or nonfinancial interests to disclose\.

## References

- \[1\]R\. Ardila, M\. Branson, K\. Davis, M\. Kohler, J\. Meyer, M\. Henretty, R\. Morais, L\. Saunders, F\. Tyers, and G\. Weber\(2020\)Common Voice: a massively\-multilingual speech corpus\.InProc\. LREC,Cited by:[§2](https://arxiv.org/html/2610.22583#S2.p1.1)\.
- \[2\]H\. Bai, R\. Zheng, J\. Chen, M\. Ma, X\. Li, and L\. Huang\(2022\)A3\{\}^\{3\}T: alignment\-aware acoustic and text pretraining for speech synthesis and editing\.InProc\. ICML,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[3\]M\. Bain, J\. Huh, T\. Han, and A\. Zisserman\(2023\)WhisperX: time\-accurate speech transcription of long\-form audio\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[4\]J\. Carletta, S\. Ashby, S\. Bourban,et al\.\(2005\)The AMI meeting corpus: a pre\-announcement\.InProc\. MLMI,Cited by:[§2](https://arxiv.org/html/2610.22583#S2.p1.1)\.
- \[5\]G\. Chen, S\. Chai, G\. Wang,et al\.\(2021\)GigaSpeech: an evolving, multi\-domain ASR corpus with 10,000 hours of transcribed audio\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[6\]S\. Chen, C\. Wang, Z\. Chen,et al\.\(2022\)WavLM: large\-scale self\-supervised pre\-training for full stack speech processing\.IEEE J\. Sel\. Topics Signal Process\.\.Cited by:[§3\.2](https://arxiv.org/html/2610.22583#S3.SS2.p2.1)\.
- \[7\]M\. Federico, R\. Enyedi, R\. Barra\-Chicote,et al\.\(2020\)From speech\-to\-speech translation to automatic dubbing\.InProc\. IWSLT,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[8\]D\. Galvez, G\. Diamos, J\. Ciro,et al\.\(2021\)The People’s Speech: a large\-scale diverse English speech recognition dataset for commercial usage\.InProc\. NeurIPS Datasets and Benchmarks,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[9\]J\. S\. Garofolo, L\. F\. Lamel, W\. M\. Fisher,et al\.\(1993\)TIMIT acoustic\-phonetic continuous speech corpus\.Note:LDC93S1Cited by:[§4](https://arxiv.org/html/2610.22583#S4.p1.1)\.
- \[10\]Y\. Gong, J\. Yu, and J\. Glass\(2022\)VocalSound: a dataset for improving human vocal sounds recognition\.InProc\. ICASSP,Cited by:[§2](https://arxiv.org/html/2610.22583#S2.p2.1)\.
- \[11\]R\. Haeb\-Umbach and H\. Ney\(1992\)Linear discriminant analysis for improved large vocabulary continuous speech recognition\.InProc\. ICASSP,Cited by:[§3\.2](https://arxiv.org/html/2610.22583#S3.SS2.p3.1)\.
- \[12\]H\. He, Z\. Shang, C\. Wang,et al\.\(2024\)Emilia: an extensive, multilingual, and diverse speech dataset for large\-scale speech generation\.InProc\. IEEE SLT,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[13\]A\. Janin, D\. Baron, J\. Edwards, D\. Ellis, D\. Gelbart, N\. Morgan, B\. Peskin, T\. Pfau, E\. Shriberg, A\. Stolcke, and C\. Wooters\(2003\)The ICSI meeting corpus\.InProc\. ICASSP,Cited by:[§2](https://arxiv.org/html/2610.22583#S2.p1.1)\.
- \[14\]W\. Kang, Z\. Yao, F\. Kuang,et al\.\(2023\)Delay\-penalized transducer for low\-latency streaming ASR\.InProc\. ICASSP,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[15\]M\. C\. Kelley, S\. J\. Perry, and B\. V\. Tucker\(2024\)The Mason\-Alberta phonetic segmenter: a forced alignment system based on deep neural networks and interpolation\.Phonetica\.Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[16\]J\. X\. Koh, A\. Mislan, K\. Khoo, B\. Ang, W\. Ang, C\. Ng, and Y\.\-Y\. Tan\(2019\)Building the Singapore English national speech corpus\.InProc\. Interspeech,Cited by:[§2](https://arxiv.org/html/2610.22583#S2.p1.1)\.
- \[17\]N\. R\. Koluguri, M\. Sekoyan, G\. Zelenfroynd,et al\.\(2025\)Granary: speech recognition and translation dataset in 25 European languages\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[18\]L\. Kürzinger, D\. Winkelbauer, L\. Li, T\. Watzel, and G\. Rigoll\(2020\)CTC\-segmentation of large corpora for German end\-to\-end speech recognition\.InProc\. SPECOM,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[19\]M\. McAuliffe, K\. Gunter, M\. Wagner, and M\. Sonderegger\(2026\)Montreal Forced Aligner and the state of speech\-to\-text alignment in 2026\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1),[§4](https://arxiv.org/html/2610.22583#S4.p4.1)\.
- \[20\]M\. McAuliffe, M\. Socolof, S\. Mihuc, M\. Wagner, and M\. Sonderegger\(2017\)Montreal Forced Aligner: trainable text\-speech alignment using Kaldi\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1),[§3\.2](https://arxiv.org/html/2610.22583#S3.SS2.p2.1),[§3\.2](https://arxiv.org/html/2610.22583#S3.SS2.p3.1)\.
- \[21\]V\. Panayotov, G\. Chen, D\. Povey, and S\. Khudanpur\(2015\)Librispeech: an ASR corpus based on public domain audio books\.InProc\. ICASSP,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1),[§2](https://arxiv.org/html/2610.22583#S2.p1.1)\.
- \[22\]S\. Papi, M\. Gaido, A\. Karakanta,et al\.\(2023\)Direct speech translation for automatic subtitling\.Trans\. ACL\.Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[23\]P\. Peng, P\.\-Y\. Huang, S\.\-W\. Li, A\. Mohamed, and D\. Harwath\(2024\)VoiceCraft: zero\-shot speech editing and text\-to\-speech in the wild\.InProc\. ACL,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[24\]M\. A\. Pitt, K\. Johnson, E\. Hume, S\. Kiesling, and W\. Raymond\(2005\)The Buckeye corpus of conversational speech: labeling conventions and a test of transcriber reliability\.Speech Communication\.Cited by:[§4](https://arxiv.org/html/2610.22583#S4.p1.1)\.
- \[25\]D\. Povey, A\. Ghoshal, G\. Boulianne,et al\.\(2011\)The Kaldi speech recognition toolkit\.InProc\. ASRU,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1),[§3\.2](https://arxiv.org/html/2610.22583#S3.SS2.p1.1),[§3\.2](https://arxiv.org/html/2610.22583#S3.SS2.p3.1)\.
- \[26\]V\. Pratap, A\. Tjandra, B\. Shi,et al\.\(2024\)Scaling speech technology to 1,000\+ languages\.J\. Mach\. Learn\. Res\.\.Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[27\]Qwen Team\(2026\)Qwen3\-ASR technical report\.Note:arXiv:2601\.21337Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[28\]A\. Radford, J\. W\. Kim, T\. Xu,et al\.\(2023\)Robust speech recognition via large\-scale weak supervision\.InProc\. ICML,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[29\]M\. M\. Rashid, G\. Li, and C\. Du\(2023\)Nonspeech7k dataset: classification and analysis of human non\-speech sound\.IET Signal Processing\.Cited by:[§2](https://arxiv.org/html/2610.22583#S2.p2.1)\.
- \[30\]E\. Rastorgueva, V\. Lavrukhin, and B\. Ginsburg\(2023\)NeMo forced aligner and its application to word alignment for subtitle generation\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[31\]N\. B\. Ratner and B\. MacWhinney\(2018\)Fluency Bank: a new resource for fluency research and practice\.J\. Fluency Disorders\.Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[32\]Y\. Ren, C\. Hu, X\. Tan,et al\.\(2021\)FastSpeech 2: fast and high\-quality end\-to\-end text to speech\.InProc\. ICLR,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[33\]A\. Romana, M\. Niu, M\. Perez, and E\. M\. Provost\(2024\)FluencyBank Timestamped: an updated data set for disfluency detection and automatic intended speech recognition\.J\. Speech Lang\. Hear\. Res\.\.External Links:[Document](https://dx.doi.org/10.1044/2024%5FJSLHR-24-00070)Cited by:[§4](https://arxiv.org/html/2610.22583#S4.p1.1)\.
- \[34\]R\. Rousso, E\. Cohen, J\. Keshet, and E\. Chodroff\(2024\)Tradition or innovation: a comparison of modern ASR methods for forced alignment\.InProc\. Interspeech,Cited by:[§4](https://arxiv.org/html/2610.22583#S4.p4.1)\.
- \[35\]X\. Shi, Y\. Chen, S\. Zhang, and Z\. Yan\(2023\)Achieving timestamp prediction while recognizing with non\-autoregressive end\-to\-end ASR model\.Note:arXiv:2301\.12343Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[36\]D\. Snyder, G\. Chen, and D\. Povey\(2015\)MUSAN: a music, speech, and noise corpus\.Note:arXiv:1510\.08484Cited by:[§3\.3](https://arxiv.org/html/2610.22583#S3.SS3.p1.1)\.
- \[37\]B\. Thallinger, L\. Wagner, T\. Bloder, and M\. Zusag\(2025\)A multi\-stage feature pipeline on timestamped speech transcriptions for dementia assessment\.InProc\. ICASSP,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[38\]L\. Wagner, B\. Thallinger, M\. Stankovic, and M\. Zusag\(2026\)Content is what remains: invariant speech tokenization from parallel utterances\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[39\]L\. Wagner, M\. Zusag, and B\. Thallinger\(2026\)Transcription policy as a latent variable: activating controllable verbatim ASR with word\-level timing\.Note:arXiv:2607\.18934Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.22583#S3.SS1.p1.1)\.
- \[40\]K\. Wang and D\. Herremans\(2024\)DisfluencySpeech – single\-speaker conversational speech dataset with paralanguage\.InProc\. IEEE TENCON,Cited by:[§2](https://arxiv.org/html/2610.22583#S2.p1.1)\.
- \[41\]J\. Yamagishi, C\. Veaux, and K\. MacDonald\(2019\)CSTR VCTK corpus: english multi\-speaker corpus for CSTR voice cloning toolkit\.Note:University of EdinburghCited by:[§2](https://arxiv.org/html/2610.22583#S2.p1.1)\.
- \[42\]J\. Yu, C\.\-C\. Chiu, B\. Li,et al\.\(2021\)FastEmit: low\-latency streaming ASR with sequence\-level emission regularization\.InProc\. ICASSP,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[43\]H\. Zen, V\. Dang, R\. Clark,et al\.\(2019\)LibriTTS: a corpus derived from LibriSpeech for text\-to\-speech\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[44\]J\. Zhu, C\. Zhang, and D\. Jurgens\(2022\)Phone\-to\-audio alignment without text: a semi\-supervised approach\.InProc\. ICASSP,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.
- \[45\]M\. Zusag, L\. Wagner, and T\. Bloder\(2023\)Careful Whisper – leveraging advances in automatic speech recognition for robust and interpretable aphasia subtype classification\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p1.1)\.
- \[46\]M\. Zusag, L\. Wagner, and B\. Thallinger\(2024\)CrisperWhisper: accurate timestamps on verbatim speech transcriptions\.InProc\. Interspeech,Cited by:[§1](https://arxiv.org/html/2610.22583#S1.p2.1)\.