Relative Time Intervals Representation for Word-level Timestamping with Masked Training

# Relative Time Intervals Representation for Word-level Timestamping with Masked Training
Source: [https://arxiv.org/html/2608.24041](https://arxiv.org/html/2608.24041)
Zhiyu TangXu LiDong Zhang††thanks:†Corresponding author\.˜[dzhang@suda\.edu\.cn](mailto:[email protected])Shoushan LiGuodong Zhou

###### Abstract

Although Speech Large Language Models \(SpeechLLMs\) excel at speech understanding and generation, their capacity for fine\-grained, temporally aligned outputs remains underexplored\. Our work addresses this gap by enabling SpeechLLMs to jointly model speech content and temporal structure, effectively transforming them from ‘‘content understanding machines” into ‘‘temporal\-aware content understanding machines”\. Specifically, we replace traditional absolute timestamps with relative timestamps, achieving a more compact vocabulary and stronger generalization capabilities\. To efficiently infuse timestamp prediction ability into pre\-trained large language models, we introduce a hybrid fine\-tuning strategy: full\-parameter fine\-tuning of the timestamp\-augmented embedding layer and language model head, combined with LoRA fine\-tuning of the decoder layers\. Moreover, we design a masked timestamp training objective, preventing the model from over\-relying on ground\-truth timestamps, and thereby enhancing robustness against noisy real\-world annotations\. Extensive experiments demonstrate that our approach achieves significant improvements in timestamp prediction accuracy while maintaining strong speech transcription performance\.111https://github\.com/tangquanwei/Timestamp\-Aware\-Speech\-LLM

###### Index Terms:

Timestamping, Speech LLM, Absolute Timestamp, Relative Timestamp, Masked Training

††address:1Soochow University2University of Queenland3AISpeech Ltd4Jiangsu Key Lab of Language Computing## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.24041v1/intro.png)Figure 1:Traditional methods rely on fixed\-reference absolute timestamps, often struggling with cumulative errors or precise synchronization\. Our approach utilizes dynamic interval\-based relative timestamps, which inherently model temporal relationships and offer improved robustness and accuracy\.Recent research has explored the integration of large language models \(LLMs\) with speech processing\[[17](https://arxiv.org/html/2608.24041#bib.bib8),[14](https://arxiv.org/html/2608.24041#bib.bib25)\]\. However, generating fine\-grained, word\-level timestamped transcription has not been fully explored yet\[[13](https://arxiv.org/html/2608.24041#bib.bib7),[18](https://arxiv.org/html/2608.24041#bib.bib6)\]\. This task demands not only accurate speech understanding but also precise temporal alignment between linguistic units and their acoustic realizations\[[11](https://arxiv.org/html/2608.24041#bib.bib10),[7](https://arxiv.org/html/2608.24041#bib.bib9)\]\. A core challenge lies in how to represent and predict temporal structure efficiently within the constrained vocabulary and sequential decoding paradigm of LLMs\[[23](https://arxiv.org/html/2608.24041#bib.bib5),[24](https://arxiv.org/html/2608.24041#bib.bib4),[19](https://arxiv.org/html/2608.24041#bib.bib14),[20](https://arxiv.org/html/2608.24041#bib.bib16)\]\. Existing methods like Qwen2 Audio\[[4](https://arxiv.org/html/2608.24041#bib.bib20)\]often employ absolute timestamps for modeling\. e\.g\.,`<\|0\.45\|\>Okay<\|1\.10\|\>let's<\|2\.50\|\>go<\|5\.70\| \>`… indicates that the word ”Okay” starts at 0\.45 seconds and ends at 1\.10 seconds\. While this representation is intuitive and facilitates evaluation, its dependence on an absolute temporal reference limits scalability and generalization over long\-duration audio segments\.

As shown in Figure[1](https://arxiv.org/html/2608.24041#S1.F1), predicting absolute timestamps is prone to cumulative error propagation\. Absolute timestamps are monotonically increasing, causing LLMs to learn a cumulative pattern: generating larger values by adding estimated segment durations to prior timestamps\. Furthermore, this approach exhibits poor generalization to unseen temporal ranges\. For instance, it fails to predict timestamps beyond the training duration \(e\.g\., at 60 seconds when trained only on utterances up to 30 seconds\)\. A major practical limitation lies in the prohibitively large vocabulary required for high temporal precision: achieving 0\.01\-second resolution over a 300\-second audio clip requires 30,000 distinct timestamp tokens\. This explosion in vocabulary significantly increases training complexity and computational overhead, making the approach unscalable for long\-form speech processing\.

Figure 2:Architectural Comparison of an Absolute Timestamp\-based Model \(Left\) and Our Proposed Relative Timestamp\-based Model \(Right\)\. The traditional method models timestamps as independent absolute values and struggles with representing timestamps\. In contrast, our approach utilizes a compact vocabulary of relative timestamps\. By leveraging LoRA fine\-tuning on the decoder, our model efficiently learns to jointly generate both text and temporal markers\. To ensure accuracy and robustness, our training incorporates a combined loss function and timestamp regularization \(token masking\), which forces the model to learn a more resilient representation of temporal information\.To address these limitations, we propose a relative timestamp representation, defined as the time interval between consecutive words\. \(e\.g\.,`<\|0\.65\|\>`for words aligned at`<\|0\.45\|\>`and`<\|1\.10\|\>`\)\. This method requires solely estimating the current segment’s duration, thereby minimizing error susceptibility\. Only a limited time interval \(such as 0\.01 seconds to 5 seconds\) needs to be learned, and by accumulating these intervals, any length of audio can be represented\. Vocabulary only needs to contain limited time interval tokens, compared to absolute timestamps\. Human speech perception naturally focuses on relative timing: we intuitively process how long after the previous word the next one occurs, rather than its absolute position in time\. This relative interval modeling aligns naturally with the autoregressive generation paradigm of language models\. Absolute timestamps force the model to learn an entirely new absolute value that is unrelated to the previously generated content, which is contrary to its natural generation mode\.

We make the following three key contributions:

1\.A Novel Relative Time Interval Representation for Word\-Level Timestamping:We introduce a relative timestamp representation that models inter\-word intervals instead of absolute positions\. This approach mitigates vocabulary explosion and temporal drift, enabling accurate timestamp prediction for arbitrarily long audio without length\-dependent degradation\.

2\.Advanced Regularization and Alignment Strategies:We propose a regularization technique,Timestamp Masking, which mitigates overfitting to perfect alignments, along with a dynamically weightedJoint Temporal Alignment Lossthat adaptively balances transcription accuracy and timestamp fidelity during training\.

3\.Hybrid Parameter\-Efficient Fine\-Tuning:We fully fine\-tune only timestamp\-specific modules \(embedding layer and LM head\), while adapting pre\-trained decoder layers via LoRA\. This role\-aware strategy preserves linguistic knowledge with minimal parameter updates, aligning adaptation granularity with each module’s functional role\.

## 2Methodology

### 2\.1Relative Timestamp Representation

Unlike traditional methods that rely on an extensive vocabulary of absolute timestamps, our approach employsrelative timestampsto represent temporal information\. Each timestamp token signifies the time interval from the end of the preceding word to the current word\. The core advantage of this method is its superior generalization and compact vocabulary\. By learning a constrained set of time intervals, the model can cumulatively represent temporal information for audio of any length, eliminating the need to define a unique token for every possible absolute time point\.

### 2\.2LLMs\-based Architecture and Fine\-Tuning

Our model architecture is built upon a pre\-trained large speech model, which consists of an encoder for extracting speech features and a decoder for sequence generation\.

As shown in Figure[2](https://arxiv.org/html/2608.24041#S1.F2)\(right\), to effectively adapt our model for the new task of temporal alignment, we employ a hybrid fine\-tuning strategy that combines full parameter updates for new components with parameter\-efficient fine\-tuning for backbone modules\.

We apply full\-parameter fine\-tuning to the newly introduced timestamp embedding and the Language Model \(LM\) head\. These modules are directly responsible for mapping the new timestamp tokens to a continuous vector space and generating them in the output sequence, respectively\. Since their initial weights are random, a comprehensive update of all parameters is necessary for them to effectively learn and represent the new temporal\-augmented vocabulary from scratch\.

For the architecture of the backbone model, specifically the decoder layers, we utilize LoRA \(Low\-Rank Adaptation\)\[[6](https://arxiv.org/html/2608.24041#bib.bib22)\]\. This strategic choice is driven by both computational efficiency and the need for targeted adaptation\. Instead of updating all parameters, LoRA freezes the original weights and injects small, trainable low\-rank matrices\. This approach drastically reduces the number of trainable parameters, leading to significant savings in computational resources\. Furthermore, by fine\-tuning the decoder with LoRA, the model can efficiently learn to insert corresponding temporal markers when generating specific text content, without interfering with the powerful, pre\-trained understanding of speech features\. This hybrid approach ensures computational efficiency while achieving precise control over the model’s generative behavior for robust and accurate temporal alignment\.

### 2\.3Training Strategies for Temporal Alignment

To ensure that our model generates accurate and robust timestamps, we designed two key training strategies:

#### 2\.3\.1Joint Temporal Alignment Loss

Our training objective is to minimize a joint loss function composed of two components, which balances the weights between text transcription and timestamp generation:

Lt​o​t​a​l=Lt​e​x​t\+λ​Lt​i​m​e​s​t​a​m​pL\_\{total\}=L\_\{text\}\+\\lambda L\_\{timestamp\}whereLt​e​x​tL\_\{text\}is a standard cross\-entropy loss measuring the discrepancy between the model’s generated text tokens and the ground\-truth tokens\. Similarly,Lt​i​m​e​s​t​a​m​pL\_\{timestamp\}is a cross\-entropy loss that evaluates the prediction accuracy of the discrete timestamp tokens\.

The hyperparameterλ\\lambdaserves as a weighting factor between the two loss components\. We employ a dynamic weighting strategy: in the early training stages, we set a smaller weight \(λ=1\\lambda=1\) to allow the model to prioritize learning the speech\-to\-text mapping\. As training progresses, we gradually increase the value ofλ\\lambdato encourage more precise temporal alignment\. Specifically, for each training epoch, we increment the value ofλ\\lambdaby 1\.

Table 1:Performance comparison of different models on timestamp prediction\. Precision and Recall are in percentages, and the average timestamp difference is in milliseconds\. ’\-’ denotes the model does not support Chinese speech\.
#### 2\.3\.2Timestamp Masking for Regularization

To mitigate overfitting to ground\-truth during training, we propose a timestamp masking technique\. The core idea involves randomly replacing a subset of timestamp tokens with a \[MASK\] token\. During training, the previous timestamp token is masked, the model cannot reference its ground\-truth value, and must instead infer the current timestamp from speech content and historical tokens, thereby enhancing robustness to inaccurate annotations\. Similar to BERT’s masked language modeling, this random masking encourages the model to leverage varying input components across iterations, preventing over\-reliance on specific timestamps\.

## 3Experiments

### 3\.1Experimental Setup

Datasets\.We conduct experiments on five benchmark datasets: AISHELL\-1: A Mandarin corpus with∼\\sim178 hours of speech from 400 speakers\[[3](https://arxiv.org/html/2608.24041#bib.bib11)\]\. AISHELL\-2: A larger Mandarin dataset containing∼\\sim1,000 hours from 1,991 speakers in diverse acoustic environments\[[5](https://arxiv.org/html/2608.24041#bib.bib12)\]\. Wenet Meeting: A multi\-domain Mandarin corpus consisting of 10000\+ hours of high\-quality labeled speech\[[25](https://arxiv.org/html/2608.24041#bib.bib24)\]\. LibriSpeech: An English read\-speech corpus with∼\\sim1,000 hours of audiobooks, widely adopted for ASR evaluation\[[12](https://arxiv.org/html/2608.24041#bib.bib13)\]\. Common Voice \(English subset\): A crowd\-sourced corpus with diverse accents and recording conditions\[[2](https://arxiv.org/html/2608.24041#bib.bib17)\]\.

Baseline Models\.We compare our proposed method against several state\-of\-the\-art models\. Qwen2\-Audio: A large language model with integrated audio understanding capabilities, trained using a large amount of diverse data and multiple training methodologies\[[4](https://arxiv.org/html/2608.24041#bib.bib20)\]\. WhisperTimestamped: This model utilizes Whisper\-large\-v3, a robust, multilingual ASR model trained on approximately 680,000 hours of weakly supervised audio data\[[15](https://arxiv.org/html/2608.24041#bib.bib18)\]\. SenseVoice\-Small: A lightweight and efficient ASR system, which is trained on over 400,000 hours of data and supports more than 50 languages\[[1](https://arxiv.org/html/2608.24041#bib.bib19)\]\. Canary: Employs a data\-driven approach to enable word\-level timestamp prediction and supports multiple languages other than Chinese\[[8](https://arxiv.org/html/2608.24041#bib.bib23)\]\.

Metric\.For timestamp prediction, we introduce the concept of time tolerance\. Due to the inherent ambiguity of speech boundaries, a predicted timestamp is considered correct if its difference from the ground\-truth timestamp falls within a predefined tolerance\[[16](https://arxiv.org/html/2608.24041#bib.bib2),[10](https://arxiv.org/html/2608.24041#bib.bib3),[9](https://arxiv.org/html/2608.24041#bib.bib15)\]\. We use Precision and Recall to evaluate the accuracy of these timestamps: Precision measures the proportion of correctly predicted timestamps among all predicted timestamps\. Recall measures the proportion of correctly predicted timestamps among all ground\-truth timestamps\. Additionally, we use the Average Time Difference to quantify the temporal accuracy, defined as:

A​v​g\.D​i​f​f=∑i=1N\|t​i​m​e​s​t​a​m​pg​ti−t​i​m​e​s​t​a​m​pp​r​e​di\|NAvg\.Diff=\\frac\{\\sum\_\{i=1\}^\{N\}\|timestamp\_\{gt\}^\{i\}\-timestamp\_\{pred\}^\{i\}\|\}\{N\}
For ASR analysis, we use Word Error Rate \(WER\) as the primary metric\[[21](https://arxiv.org/html/2608.24041#bib.bib1)\]\.

Table 2:Ablation study on timestamp prediction performance at a tolerance of 240 ms \(WER % and Precision/Recall\) across different model configurations\.Implementation Details\.Our training methodology extends the FireRedASR\-LLM’s\[[22](https://arxiv.org/html/2608.24041#bib.bib21)\]architecture to support timestamp outputs\. This model is built upon a conformer audio encoder and the Qwen2\-7B\-Instruct backbone LLM\. Crucially, the native model cannot generate timestamps\.

The model is trained on a cluster of 24 x Ascend 910B \(64G\) NPUs for 7k steps, with each NPU handling a batch duration of 500 seconds\. Settingλ\\lambdaas 1 increases by 1 for each epoch, and empirically setting masking probability as 10% from the second epoch\. For training, we use the AdamW optimizer with a low learning rate of5×10−65\\times 10^\{\-6\}and a WarmupCosineLR scheduler to ensure stable convergence\. The entire process utilizes bf16 mixed\-precision training for enhanced efficiency\. To enable timestamp generation, we incorporate new tokens into the LLM’s vocabulary: Absolute Timestamp Pattern: We add tokens from`<\|0\.00\|\>`to`<\|30\.00\|\>`, representing a time range of 0 to 30 seconds\. Relative Timestamp Pattern: We add tokens from`<\|0\.00\|\>`to`<\|5\.00\|\>`, representing time differences up to 5 seconds\. Unlike traditional methods that require a special instruction token`<\|timestamp\|\>`Our model is naturally prompted for this task by a simple, intuitive command like ”Speech to text with timestamp\.” Our training regimen was confined to a pair of widely\-used datasets: the AISHELL\-2 corpus\[[5](https://arxiv.org/html/2608.24041#bib.bib12)\]for Mandarin and the English subset of the Common Voice dataset\[[2](https://arxiv.org/html/2608.24041#bib.bib17)\]\.

### 3\.2Main Results

Table[1](https://arxiv.org/html/2608.24041#S2.T1)compares the timestamp prediction performance of various models across the LibriSpeech and Wenet\-Meeting datasets\.

Our Relative Timestamp method achieved the highest Precision and Recall scores across all tolerance levels on both datasets\. For instance, at a 240 ms tolerance, our model reached an impressive 91\.13% Precision and 86\.88% Recall on Wenet\-Meeting, substantially outperforming all baselines\. Crucially, our model also exhibited the best temporal accuracy\. With an average timestamp difference of just 30\.34 ms at a 240 ms tolerance on Wenet\-Meeting, our predictions were the closest to the ground truth\.

In contrast, Canary performs worse than our method at low tolerance levels but slightly better at high tolerance levels\. This discrepancy can be attributed to the fact that Canary was trained on the Librispeech dataset, whereas our model was not\.Qwen2\-Audio failed on LibriSpeech, with Precision and Recall scores near zero, indicating its inability to handle timestamps on this English corpus\.WhisperTimestamped and SenseVoiceSmall generally delivered lower performance, especially at lower tolerances, with their results falling far behind ours\. We also conducted an ASR experiment, and the WER results[3](https://arxiv.org/html/2608.24041#S3.T3)showed that our model outperforms all baselines on all datasets\.

In conclusion, this table definitively proves that our Relative Timestamp method sets a new benchmark for timestamp prediction\. By focusing on relative temporal relationships, our approach achieves superior accuracy and precision, validating its potential for building high\-performance speech language models\.

### 3\.3Ablation study

Table[2](https://arxiv.org/html/2608.24041#S3.T2)presents a detailed ablation study on the performance of different model configurations\. Our Relative Timestamp model consistently achieves the best performance across all metrics\. The consistently higher Precision and Recall scores further confirm our model’s superior ability to not only transcribe words accurately but also to predict their timestamps with greater precision\.

To dissect our method’s key contributions, we conducted further ablations:

Impact of Timestamp Loss \(TS Loss\):Removing the timestamp loss term \(\- TS Loss\) results in a performance drop for both baseline and our models, as evidenced by an increase in WER and a decrease in Precision/Recall\. This highlights the critical role of TS Loss in guiding the model to learn accurate timestamp prediction, preventing it from solely focusing on the text sequence generation task\.

Impact of Timestamp Mask:The ablation study on our model reveals a significant performance degradation when the timestamp mask is removed \(\- Timestamp Mask\)\. The WER increases from 2\.15% to 2\.56% on AISHELL\-2 iOS and from 11\.63% to 14\.47% on Common Voice\. This compellingly demonstrates that the Timestamp Mask is crucial for the training process\. It helps the model focus on valid timestamp predictions and filters out noise, ensuring the robustness of our relative timestamp\.

### 3\.4Analysis of ASR

As depicted in Table[3](https://arxiv.org/html/2608.24041#S3.T3), a comprehensive performance comparison based on Word Error Rate \(WER\) is presented across five diverse datasets\. Our proposed method, Relative Timestamp, consistently demonstrates superior performance across all evaluated datasets\. This is visually confirmed by its polygon having the largest area, signifying its robust generalization and leading performance across multilingual and domain\-specific tasks\. Specifically, our model achieves WERs of 1\.26% and 2\.15% on the Chinese datasets AISHELL\-1 \(AS\-1\) and AISHELL\-2 \(AS\-2\), respectively, significantly outperforming all baselines\. On the more challenging Wenet Meeting \(Wenet\) dataset, our model’s WER of 5\.56% establishes a substantial lead\. Furthermore, our approach also secures the best results on the English datasets LibriSpeech \(Libri\) \(2\.78%\) and Common Voice \(CV\) \(11\.63%\)\.

Table 3:WER \(%\\%\) comparison\. Lower is better \(↓\\downarrow\)\.

## 4Conclusion

We propose a novel framework aimed at addressing the core challenges of existing speech models in precise temporal sequence understanding\. We have gone beyond the simple method of adding timestamp tags to LLMs, instead focusing on endowing the model with fine\-grained, aligned multi\-modal spatiotemporal understanding capabilities, thereby upgrading LLMs from “content understanding machines” to “temporal\-aware content understanding machines”\.

## 5Acknowledgements

This was supported by Jiangsu Province Frontier Program Project \(BF2025036\), Hong Kong RGC grant GRF \#15611021, NSFC grant \(No\. 62376178\), and Jiangsu Key Lab of Language Computing\.

## References

- \[1\]K\. An, Q\. Chen, C\. Deng, Z\. Du, C\. Gao, Z\. Gao, Y\. Gu, T\. He, H\. Hu, K\. Hu, S\. Ji, Y\. Li, Z\. Li, H\. Lu, H\. Luo, X\. Lv, B\. Ma, Z\. Ma, C\. Ni, C\. Song, J\. Shi, X\. Shi, H\. Wang, W\. Wang, Y\. Wang, Z\. Xiao, Z\. Yan, Y\. Yang, B\. Zhang, Q\. Zhang, S\. Zhang, N\. Zhao, and S\. Zheng\(2024\)FunAudioLLM: voice understanding and generation foundation models for natural interaction between humans and llms\.CoRRabs/2407\.04051\.External Links:[Link](https://doi.org/10.48550/arXiv.2407.04051),[Document](https://dx.doi.org/10.48550/ARXIV.2407.04051),2407\.04051Cited by:[Table 1](https://arxiv.org/html/2608.24041#S2.T1.4.9.1.1),[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p2.1)\.
- \[2\]R\. Ardila, M\. Branson, K\. Davis, M\. Kohler, J\. Meyer, M\. Henretty, R\. Morais, L\. Saunders, F\. Tyers, and G\. Weber\(2020\)Common voice: a massively\-multilingual speech corpus\.InProceedings of the Twelfth Language Resources and Evaluation Conference,Marseille, France,pp\.4218–4222\(eng\)\.External Links:[Link](https://aclanthology.org/2020.lrec-1.520/),ISBN 979\-10\-95546\-34\-4Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p6.1)\.
- \[3\]H\. Bu, J\. Du, X\. Na, B\. Wu, and H\. Zheng\(2017\)AISHELL\-1: an open\-source mandarin speech corpus and a speech recognition baseline\.\.InO\-COCOSDA,pp\.1–5\.External Links:ISBN 978\-1\-5386\-3333\-5,[Link](http://dblp.uni-trier.de/db/conf/ococosda/ococosda2017.html#BuDNWZ17)Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p1.1)\.
- \[4\]Y\. Chu, J\. Xu, Q\. Yang, H\. Wei, X\. Wei, Z\. Guo, Y\. Leng, Y\. Lv, J\. He, J\. Lin, C\. Zhou, and J\. Zhou\(2024\)Qwen2\-audio technical report\.External Links:2407\.10759,[Link](https://arxiv.org/abs/2407.10759)Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1),[Table 1](https://arxiv.org/html/2608.24041#S2.T1.4.3.1.1),[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p2.1)\.
- \[5\]J\. Du, X\. Na, X\. Liu, and H\. Bu\(2018\)AISHELL\-2: transforming mandarin asr research into industrial scale\.External Links:1808\.10583,[Link](https://arxiv.org/abs/1808.10583)Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p6.1)\.
- \[6\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, W\. Chen,et al\.\(2022\)Lora: low\-rank adaptation of large language models\.\.ICLR1\(2\),pp\.3\.Cited by:[§2\.2](https://arxiv.org/html/2608.24041#S2.SS2.p4.1)\.
- \[7\]K\. Hu, K\. Puvvada, E\. Rastorgueva, Z\. Chen, H\. Huang, S\. Ding, K\. Dhawan, H\. Xu, J\. Balam, and B\. Ginsburg\(2025\)Word level timestamp generation for automatic speech recognition and translation\.arXiv preprint arXiv:2505\.15646\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[8\]K\. Hu, K\. Puvvada, E\. Rastorgueva, Z\. Chen, H\. Huang, S\. Ding, K\. Dhawan, H\. Xu, J\. Balam, and B\. Ginsburg\(2025\)Word Level Timestamp Generation for Automatic Speech Recognition and Translation\.InInterspeech 2025,pp\.2565–2569\.External Links:[Document](https://dx.doi.org/10.21437/Interspeech.2025-869),ISSN 2958\-1796Cited by:[Table 1](https://arxiv.org/html/2608.24041#S2.T1.4.12.1.1),[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p2.1)\.
- \[9\]Y\. Ji, S\. Zhu, D\. Zhang, and S\. Li\(2025\)Pathological section staining transferring with tailored metric\-based model selection\.InICASSP 2025,pp\.1–5\.Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p3.1)\.
- \[10\]S\. Kim, B\. R\. Chernyak, O\. Seleznova, J\. Keshet, M\. Goldrick, and A\. R\. Bradlow\(2024\)Automatic recognition of second language speech\-in\-noise\.JASA Express Letters4\(2\)\.Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p3.1)\.
- \[11\]N\. Makishima, K\. Suzuki, S\. Suzuki, A\. Ando, and R\. Masumura\(2023\)Joint autoregressive modeling of end\-to\-end multi\-talker overlapped speech recognition and utterance\-level timestamp prediction\.InProc\. INTERSPEECH, 2023,pp\.2913–2917\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[12\]V\. Panayotov, G\. Chen, D\. Povey, and S\. Khudanpur\(2015\)Librispeech: an asr corpus based on public domain audio books\.InICASSP 2015,pp\.5206–5210\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP.2015.7178964),ISSN 2379\-190X,[Link](https://ieeexplore.ieee.org/document/7178964)Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p1.1)\.
- \[13\]S\. Papi, P\. Wang, J\. Chen, J\. Xue, N\. Kanda, J\. Li, and Y\. Gaur\(2024\)Leveraging timestamp information for serialized joint streaming recognition and translation\.InICASSP 2024\-2024,pp\.10381–10385\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[14\]J\. Peng, Y\. Wang, Y\. Xi, X\. Li, X\. Zhang, and K\. Yu\(2024\)A survey on speech large language models\.arXiv e\-prints,pp\.arXiv–2410\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[15\]A\. Radford, J\. W\. Kim, T\. Xu, G\. Brockman, C\. McLeavey, and I\. Sutskever\(2022\)Robust speech recognition via large\-scale weak supervision\.External Links:2212\.04356,[Link](https://arxiv.org/abs/2212.04356)Cited by:[Table 1](https://arxiv.org/html/2608.24041#S2.T1.4.6.1.1),[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p2.1)\.
- \[16\]E\. Rastorgueva, V\. Lavrukhin, and B\. Ginsburg\(2023\)Nemo forced aligner and its application to word alignment for subtitle generation\.InProc\. Interspeech,Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p3.1)\.
- \[17\]R\. Rousso, E\. Cohen, J\. Keshet, and E\. Chodroff\(2024\)Tradition or innovation: a comparison of modern asr methods for forced alignment\.arXiv preprint arXiv:2406\.19363\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[18\]S\. Sivasankaran, E\. Sun, J\. Li, Y\. Huang, and J\. Pan\(2025\)Target word activity detector: an approach to obtain asr word boundaries without lexicon\.InICASSP 2025,pp\.1–5\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[19\]Q\. Tang, S\. Y\. M\. Lee, J\. Wu, D\. Zhang, S\. Li, E\. Cambria, and G\. Zhou\(2025\)A comprehensive graph framework for question answering with mode\-seeking preference alignment\.InFindings of ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\.21504–21523\.External Links:[Link](https://aclanthology.org/2025.findings-acl.1108/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1108),ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[20\]D\. Wu, X\. Ju, D\. Zhang, S\. Li, E\. Cambria, and G\. Zhou\(2025\)Emotion across modalities and cultures: multilingual multimodal emotion\-cause analysis with memory\-inspired framework\.InProceedings of the 33rd ACM International Conference on Multimedia,pp\.5775–5783\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[21\]H\. Xu, F\. Jia, S\. Majumdar, H\. Huang, S\. Watanabe, and B\. Ginsburg\(2023\)Efficient sequence transduction by jointly predicting tokens and durations\.InInternational Conference on Machine Learning,pp\.38462–38484\.Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p4.1)\.
- \[22\]K\. Xu, F\. Xie, X\. Tang, and Y\. Hu\(2025\)FireRedASR: open\-source industrial\-grade mandarin speech recognition models from encoder\-decoder to llm integration\.arXiv preprint arXiv:2501\.14350\.Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p5.1)\.
- \[23\]H\. Yamasaki, J\. Louradour, J\. Hunter, and L\. Prévot\(2023\)Transcribing and aligning conversational speech: a hybrid pipeline applied to french conversations\.In2023 IEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\),pp\.1–6\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[24\]T\. Yu, Z\. Gong, M\. Tan, G\. Chen, and M\. Yang\(2025\)Unsupervised speech\-text word\-level alignment with dynamic programming\.InFindings of the Association for Computational Linguistics: NAACL 2025,pp\.2323–2334\.Cited by:[§1](https://arxiv.org/html/2608.24041#S1.p1.1)\.
- \[25\]B\. Zhang, H\. Lv, P\. Guo, Q\. Shao, C\. Yang, L\. Xie, X\. Xu, H\. Bu, X\. Chen, C\. Zeng,et al\.\(2022\)Wenetspeech: a 10000\+ hours multi\-domain mandarin corpus for speech recognition\.InICASSP 2022,pp\.6182–6186\.Cited by:[§3\.1](https://arxiv.org/html/2608.24041#S3.SS1.p1.1)\.