NAVER LABS System Re-implementation for the IWSLT 2026 Instruction-Following Task
Summary
This paper presents an open-source re-implementation of the NAVER LABS instruction-following pipeline for IWSLT 2026, using SeamlessM4T-v2-large and Qwen3-4B-Instruct, with 100k synthetic examples.
View Cached Full Text
Cached at: 07/08/26, 04:41 AM
# NAVER LABS System Re-implementation for the IWSLT 2026 Instruction-Following Task
Source: [https://arxiv.org/html/2607.05623](https://arxiv.org/html/2607.05623)
Aniket Tathe University of Illinois Urbana\-Champaign atathe@illinois\.edu
###### Abstract
We re\-implement the NAVER LABS IWSLT 2025 instruction\-following pipeline\(Leeet al\.,[2025](https://arxiv.org/html/2607.05623#bib.bib15)\)for the IWSLT 2026 Shared Task \(constrained condition, short audio track\), adapting it to the mandated components: SeamlessM4T\-v2\-large\(Barrault and others,[2023](https://arxiv.org/html/2607.05623#bib.bib16)\)as the speech encoder and Qwen3\-4B\-Instruct\(Team,[2025](https://arxiv.org/html/2607.05623#bib.bib17)\)as the LLM backbone\. The three\-stage approach—projector alignment, text\-only LoRA pre\-training, and multimodal merging—is preserved from the original design\. We additionally construct 100k synthetic instruction\-following examples across ten speech\-centric task types \(10k per task\) from the provided corpora, suitable for further Stage 3 fine\-tuning\. Our primary model achieves COMET 0\.781 on EN–ZH speech translation and BERTScore\-F1 0\.346 on English SQA on the MCIF benchmark\. Code, training scripts, and generated data are publicly available\.111[https://github\.com/anand\-kamble/iwslt2026\-instruction\-following](https://github.com/anand-kamble/iwslt2026-instruction-following)
NAVER LABS System Re\-implementation for the IWSLT 2026 Instruction\-Following Task
Anand KambleFlorida State Universityamk23j@fsu\.eduAniket TatheUniversity of Illinois Urbana\-Champaignatathe@illinois\.edu
## 1Introduction
Multimodal speech LLMs such as SALMONN\(Tang and others,[2024](https://arxiv.org/html/2607.05623#bib.bib10)\), Qwen\-Audio\(Chu and others,[2023](https://arxiv.org/html/2607.05623#bib.bib12)\), SpeechGPT\(Zhang and others,[2023](https://arxiv.org/html/2607.05623#bib.bib11)\), and WavLLM\(Hu and others,[2024](https://arxiv.org/html/2607.05623#bib.bib26)\)couple a frozen speech encoder with an instruction\-tuned LLM\(Ouyang and others,[2022](https://arxiv.org/html/2607.05623#bib.bib13)\)via a lightweight connector, enabling flexible multi\-task inference through natural language prompts\. The IWSLT 2026 Instruction\-Following Shared Task\(Organizers,[2026](https://arxiv.org/html/2607.05623#bib.bib14)\)formalizes this paradigm with the MCIF benchmark, evaluating unified models on ASR, multilingual ST \(EN→\\rightarrow\{DE,IT,ZH\}\), and SQA\. The NAVER LABS 2025 system\(Leeet al\.,[2025](https://arxiv.org/html/2607.05623#bib.bib15)\)demonstrated a competitive three\-stage pipeline in the IWSLT 2025 constrained setting\(Abdulmumin and others,[2025](https://arxiv.org/html/2607.05623#bib.bib28)\), but was not publicly released\.
We provide the first open\-source re\-implementation, adapted to the IWSLT 2026 constraints \(SeamlessM4T\-v2\-large encoder, Qwen3\-4B\-Instruct LLM—replacing the LLaMA\-3\.1\-8B\(Dubey and others,[2024](https://arxiv.org/html/2607.05623#bib.bib5)\)backbone used in 2025\)\. We further construct 100k synthetic instruction\-following examples across ten speech\-centric task types \(Section[2\.1](https://arxiv.org/html/2607.05623#S2.SS1)\) and ablate LoRA rank and learning rate configurations for Stage 2 text pre\-training\.
## 2Task and Data
#### Shared Task\.
We participate in theconstrained condition, short audio track\(Organizers,[2026](https://arxiv.org/html/2607.05623#bib.bib14)\)\. Evaluation on MCIF uses WER \(↓\\downarrow\) for ASR, COMET\(Rei and others,[2020](https://arxiv.org/html/2607.05623#bib.bib25)\)\(↑\\uparrow\) for ST, and BERTScore\-F1\(Zhang and others,[2020](https://arxiv.org/html/2607.05623#bib.bib3)\)\(↑\\uparrow\) for SQA\. Task instructions follow the natural\-language prompt format ofLeeet al\.\([2025](https://arxiv.org/html/2607.05623#bib.bib15)\)\.
#### Training Corpora\.
Core speech data is from CoVoST 2\(Wanget al\.,[2021](https://arxiv.org/html/2607.05623#bib.bib19)\)and EuroParlST\(Iranzo\-Sánchez and others,[2020](https://arxiv.org/html/2607.05623#bib.bib20)\)\(ASR/ST\) and LibriSQA\(Huang and others,[2024](https://arxiv.org/html/2607.05623#bib.bib18)\)\(SQA\); multilingual SQA pairs in DE, IT, ZH are obtained by machine\-translating LibriSQA via SeamlessM4T\-v2\. Stage 3 additionally draws on NUTSHELL\(Maikezu and others,[2024](https://arxiv.org/html/2607.05623#bib.bib23)\)for speech summarization and YTSeg\(Retkowski and others,[2024](https://arxiv.org/html/2607.05623#bib.bib24)\)for audio chapter detection\. Table[1](https://arxiv.org/html/2607.05623#S2.T1)summarizes corpora per stage\.
Table 1:Training corpora per stage\.∗Text\-only\.†A\.2 variant only\.‡Machine\-translated\.Figure 1:Three\-stage training pipeline\. Frozen modules: dashed border\. Trainable: solid\. Stage 3 jointly fine\-tunes both projector and LoRA adapters\.
### 2\.1Synthetic Instruction\-Following Data
We construct 100k synthetic examples \(10k per task\) from the provided corpora using open\-weight Gemma models\(Gemma Team,[2025](https://arxiv.org/html/2607.05623#bib.bib6)\)\. Seven text\-grounded tasks are generated by Gemma\-4\-31B from reference transcripts: keyword extraction \(T1\), named entity recognition \(T2\), gist summarization \(T3\), topic labeling \(T4\), numeric QA \(T5\), and gist summarization in DE and ZH \(T6–T7\)\. Three audio\-grounded tasks \(T8–T10\) are generated by Gemma\-4\-E4B\-it directly from audio: vocal style description in EN, DE, and ZH\.
For example, a NER target \(T2\) for the transcript“The Luks family eventually moved to Pottsville, in southern Pennsylvania”yieldsPER=\[Luks\], LOC=\[Pottsville, S\. Pennsylvania\]\. A vocal style target \(T8\) yields“The speaker has a measured, confident tone, speaking at a moderate pace with a clear articulation\.”This data is released with our code\.
## 3System
#### Architecture\.
Our model followsLeeet al\.\([2025](https://arxiv.org/html/2607.05623#bib.bib15)\)and is illustrated in Figure[1](https://arxiv.org/html/2607.05623#S2.F1)\. A frozen SeamlessM4T\-v2\-large encoder\(Barrault and others,[2023](https://arxiv.org/html/2607.05623#bib.bib16)\)produces 1024\-dim frame representations\. A trainable projector downsamples by 3×\\timesvia frame averaging, passes through a 4\-layer Transformer encoder\(Vaswani and others,[2017](https://arxiv.org/html/2607.05623#bib.bib2)\), and projects to the LLM hidden size\. A LoRA\-adapted\(Hu and others,[2022](https://arxiv.org/html/2607.05623#bib.bib1)\)Qwen3\-4B\-Instruct\(Team,[2025](https://arxiv.org/html/2607.05623#bib.bib17)\)generates the response, with speech embeddings prepended at a<\|speech\|\>placeholder\.
#### Stage 1 — Projector Alignment\.
Encoder and LLM are frozen; only the projector is trained \(4 epochs, lr1×10−41\\times 10^\{\-4\}, constant, AdamW\(Loshchilov and Hutter,[2019](https://arxiv.org/html/2607.05623#bib.bib8)\)\)\.A\.1\(ASR/ST\): sampling 40% ASR, 18% ST\-DE, 24% ST\-ZH, 18% ST\-IT \(CoVoST 2 \+ EuroParlST\)\.A\.2\(ASR/ST/SQA\): sampling 40% ASR, 10\.5% ST\-DE, 14% ST\-ZH, 10\.5% ST\-IT, 25% SQA\-EN \(adds LibriSQA\)\.
#### Stage 2 — Text\-Only LoRA\.
No audio; projector frozen; LLM adapted via LoRA for 1 epoch\. Sampling: MT 60% \(20% each DE/IT/ZH, from CoVoST 2 and EuroParlST transcripts\) and QA 40% \(10% each EN/DE/IT/ZH, LibriSQA \+ machine\-translated\)\. Three configurations:
- •V1: rank 8,α\\alpha=16, lr3×10−43\\times 10^\{\-4\}, attn\+FF layers
- •V2: rank 16,α\\alpha=32, lr1×10−51\\times 10^\{\-5\}, cosine, all\-linear
- •V3: rank 32,α\\alpha=64, lr2×10−42\\times 10^\{\-4\}, cosine, all\-linear
#### Stage 3 — Multimodal Merge\.
Both the A\.1 projector and V1 LoRA adapters are fine\-tuned jointly; the speech encoder remains frozen\. Sampling: 20% ASR, 10% each ST\-\{DE,IT,ZH\}, 10% SQA\-EN, 5% each SQA\-\{DE,IT,ZH\}, 10% S2TSum, 15% AChap\. Each speech batch \(ST, SQA\) is immediately followed by a paired text\-only batch \(MT, QA\) to prevent catastrophic forgetting\. Projector lr:1×10−51\\times 10^\{\-5\}\(constant\); LoRA lr:3×10−43\\times 10^\{\-4\}\(cosine\); 2 epochs on 4×\\timesH100\.
## 4Experiments
\(a\) MCIF Benchmark Results \(constrained, short audio\)
\(b\) Stage\-2 Text Eval \(1k CoVoST\-2, not MCIF\-comparable\)
\(c\) Stage\-2 LoRA Rank Ablation
Table 2:Experimental results\. \(a\) MCIF benchmark; Stage 3 is our primary system\. \(b\) Stage\-2 text eval on 1k CoVoST 2; not MCIF\-comparable\. \(c\) LoRA rank ablation \(lr: V1=3e\-4, V2=1e\-5, V3=2e\-4\); V1 used in Stage 3\.#### Setup\.
Stages 1–2 train on one H200 GPU; Stage 3 on 4×\\timesH100 80GB \(DDP\)\. Audio longer than 15 seconds is excluded due to memory constraints\. Main evaluation uses the officialmcif\_evaltool; Stage 2 is separately evaluated on a 1k CoVoST 2 text subset \(not MCIF\-comparable\)\.
#### Stage 1 & 3 Results \(MCIF\)\.
Table[2](https://arxiv.org/html/2607.05623#S4.T2)\(a\) shows MCIF results\. Stage 1 improves ST and SQA over the SeamlessM4T\-v2\-large baseline at the cost of higher WER—consistent withLeeet al\.\([2025](https://arxiv.org/html/2607.05623#bib.bib15)\)\. A\.2 boosts English SQA \(0\.267 vs\. 0\.186\) but further degrades ASR\. Stage 3 recovers ASR \(23\.49 WER\), achieves the best ST COMET across all pairs \(EN–ZH: 0\.781\), and strongly improves English SQA \(0\.346\)\. Cross\-lingual SQA remains lower due to sparse multilingual supervision\.
#### Stage 2: Text Evaluation & LoRA Ablation\.
Since Stage 2 is text\-only \(no audio\), its results are evaluated on the 1k CoVoST 2 text subset and are not comparable to MCIF\. Table[2](https://arxiv.org/html/2607.05623#S4.T2)\(b\) shows the gain from LoRA \(V1\) over the base Qwen3\-4B\(Papineniet al\.,[2002](https://arxiv.org/html/2607.05623#bib.bib4); Post,[2018](https://arxiv.org/html/2607.05623#bib.bib27)\); Table[2](https://arxiv.org/html/2607.05623#S4.T2)\(c\) ablates LoRA rank\. V3 \(rank 32\) achieves the highest MT scores; V1 \(rank 8\) yields the best SQA F1 and is selected for Stage 3\.
## 5Conclusion
We re\-implemented the NAVER LABS three\-stage instruction\-following pipeline\(Leeet al\.,[2025](https://arxiv.org/html/2607.05623#bib.bib15)\)for the IWSLT 2026 constrained setting, adapting it to SeamlessM4T\-v2\-large and Qwen3\-4B\-Instruct\. Our Stage 3 model achieves COMET 0\.781 on EN–ZH ST and BERTScore\-F1 0\.346 on English SQA on MCIF, with consistent improvements over projector\-only baselines\. The 100k synthetic dataset released alongside our code provides a natural extension point for richer Stage 3 fine\-tuning or future reinforcement learning\(Shao and others,[2024](https://arxiv.org/html/2607.05623#bib.bib7)\)with task\-specific rewards and LLM\-as\-judge criteria\(Kim and others,[2024](https://arxiv.org/html/2607.05623#bib.bib22)\)\.
## Limitations
Audio longer than 15 seconds is excluded due to GPU memory constraints, potentially limiting performance on longer utterances\. Cross\-lingual SQA relies on machine\-translated QA pairs, which may introduce noise\. Stage 2 ablation numbers are evaluated on a text\-only subset and are not directly comparable to MCIF results\.
## References
- I\. Abdulmuminet al\.\(2025\)Findings of the IWSLT 2025 evaluation campaign\.InProceedings of the 22nd International Conference on Spoken Language Translation \(IWSLT 2025\),Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1)\.
- L\. Barraultet al\.\(2023\)SeamlessM4T: massively multilingual & multimodal machine translation\.arXiv preprint arXiv:2308\.11596\.External Links:[Link](https://arxiv.org/abs/2308.11596)Cited by:[§3](https://arxiv.org/html/2607.05623#S3.SS0.SSS0.Px1.p1.1)\.
- Y\. Chuet al\.\(2023\)Qwen\-Audio: advancing universal audio understanding via unified large\-scale audio\-language models\.arXiv preprint arXiv:2311\.07919\.External Links:[Link](https://arxiv.org/abs/2311.07919)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1)\.
- A\. Dubeyet al\.\(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:[Link](https://arxiv.org/abs/2407.21783)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p2.1)\.
- Gemma Team \(2025\)Gemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.External Links:[Link](https://arxiv.org/abs/2503.19786)Cited by:[§2\.1](https://arxiv.org/html/2607.05623#S2.SS1.p1.1)\.
- E\. J\. Huet al\.\(2022\)LoRA: low\-rank adaptation of large language models\.InProceedings of the Tenth International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2106.09685)Cited by:[§3](https://arxiv.org/html/2607.05623#S3.SS0.SSS0.Px1.p1.1)\.
- S\. Huet al\.\(2024\)WavLLM: towards robust and adaptive speech large language model\.arXiv preprint arXiv:2404\.00656\.External Links:[Link](https://arxiv.org/abs/2404.00656)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1)\.
- Z\. Huanget al\.\(2024\)LibriSQA: advancing free\-form and open\-ended spoken question answering with a novel dataset and framework\.InProceedings of the 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics,External Links:[Link](https://arxiv.org/abs/2308.10390)Cited by:[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Iranzo\-Sánchezet al\.\(2020\)EuroParlST: a multilingual corpus for speech translation of European parliament debates\.InProceedings of the Language Resources and Evaluation Conference,External Links:[Link](https://arxiv.org/abs/2005.01215)Cited by:[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Kimet al\.\(2024\)Prometheus: inducing fine\-grained evaluation capability in language models\.InProceedings of the Twelfth International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2310.08491)Cited by:[§5](https://arxiv.org/html/2607.05623#S5.p1.1)\.
- B\. Lee, M\. Zanon Boito, L\. Besacier, and I\. Calapodescu \(2025\)NAVER LABS europe submission to the instruction\-following track\.InProceedings of the 22nd International Conference on Spoken Language Translation \(IWSLT 2025\),External Links:[Link](https://arxiv.org/abs/2506.01808)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1),[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px1.p1.3),[§3](https://arxiv.org/html/2607.05623#S3.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2607.05623#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.05623#S5.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InProceedings of the Seventh International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/1711.05101)Cited by:[§3](https://arxiv.org/html/2607.05623#S3.SS0.SSS0.Px2.p1.1)\.
- Maikezuet al\.\(2024\)NUTSHELL: a dataset for speech summarization\.InProceedings of Interspeech,Note:[https://huggingface\.co/datasets/maikezu/nutshell](https://huggingface.co/datasets/maikezu/nutshell)Cited by:[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px2.p1.1)\.
- I\. 2\. Organizers \(2026\)IWSLT 2026 shared task: instruction\-following speech processing\.InProceedings of the 23rd International Conference on Spoken Language Translation \(IWSLT 2026\),External Links:[Link](https://iwslt.org/2026/instruction-following)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1),[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px1.p1.3)\.
- L\. Ouyanget al\.\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Link](https://arxiv.org/abs/2203.02155)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1)\.
- K\. Papineni, S\. Roukos, T\. Ward, and W\. Zhu \(2002\)BLEU: a method for automatic evaluation of machine translation\.InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics,pp\. 311–318\.Cited by:[§4](https://arxiv.org/html/2607.05623#S4.SS0.SSS0.Px3.p1.1)\.
- M\. Post \(2018\)A call for clarity in reporting BLEU scores\.InProceedings of the Third Conference on Machine Translation: Research Papers,pp\. 186–191\.External Links:[Link](https://arxiv.org/abs/1804.08771)Cited by:[§4](https://arxiv.org/html/2607.05623#S4.SS0.SSS0.Px3.p1.1)\.
- R\. Reiet al\.\(2020\)COMET: a neural framework for MT evaluation\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://arxiv.org/abs/2009.09025)Cited by:[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px1.p1.3)\.
- Retkowskiet al\.\(2024\)YTSeg: a large\-scale corpus for youtube video segmentation\.InProceedings of the Language Resources and Evaluation Conference,Note:[https://huggingface\.co/datasets/retkowski/ytseg](https://huggingface.co/datasets/retkowski/ytseg)Cited by:[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Shaoet al\.\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.External Links:[Link](https://arxiv.org/abs/2402.03300)Cited by:[§5](https://arxiv.org/html/2607.05623#S5.p1.1)\.
- C\. Tanget al\.\(2024\)SALMONN: towards generic hearing abilities for large language models\.InProceedings of the Twelfth International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2310.13289)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1)\.
- Q\. Team \(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3](https://arxiv.org/html/2607.05623#S3.SS0.SSS0.Px1.p1.1)\.
- A\. Vaswaniet al\.\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30\.External Links:[Link](https://arxiv.org/abs/1706.03762)Cited by:[§3](https://arxiv.org/html/2607.05623#S3.SS0.SSS0.Px1.p1.1)\.
- C\. Wang, A\. Wu, and J\. Pino \(2021\)CoVoST 2 and massively multilingual speech translation\.InProceedings of Interspeech 2021,External Links:[Link](https://arxiv.org/abs/2007.10310)Cited by:[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Zhanget al\.\(2023\)SpeechGPT: empowering large language models with intrinsic cross\-modal conversational abilities\.InFindings of the Association for Computational Linguistics: EMNLP 2023,External Links:[Link](https://arxiv.org/abs/2305.11000)Cited by:[§1](https://arxiv.org/html/2607.05623#S1.p1.1)\.
- T\. Zhanget al\.\(2020\)BERTScore: evaluating text generation with BERT\.InProceedings of the Eighth International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/1904.09675)Cited by:[§2](https://arxiv.org/html/2607.05623#S2.SS0.SSS0.Px1.p1.3)\.
## Appendix ASynthetic Dataset: Task Definitions and Examples
Each of the 100k synthetic examples \(10k per task\) is paired with a natural\-language instruction prompt; the model is trained to produce the target output\. Tasks T1–T7 aretext\-grounded: Gemma\-4\-31B generates targets from reference transcripts\. Tasks T8–T10 areaudio\-grounded: Gemma\-4\-E4B\-it generates targets directly from speech audio\. One example per task is shown below\.
#### T1 — Keyword Extraction\.
Definition:Extract the most salient keywords from the spoken utterance as a comma\-separated list\. Input:“These data components in turn serve as the ‘building blocks’ of data exchanges\.” Output:data components, building blocks, data exchanges
#### T2 — Named Entity Recognition\.
Definition:Identify and classify named entities \(PER, ORG, LOC\) present in the transcript\. Input:“Saunders was born in Farnborough, Hampshire, the son of a British Army officer\.” Output:PER:\[Saunders\], ORG:\[British Army\], LOC:\[Farnborough, Hampshire\]
#### T3 — Gist Summarization \(EN\)\.
Definition:Produce a one\-sentence English summary of the spoken content\. Input:“…if you don’t want to look at all of the man pages, you just want to see a very short description of what a command does—typewhatisfollowed by the command name…” Output:Thewhatiscommand provides a one\-line description of any Unix/Linux command, e\.g\.whatis sortorwhatis cat\.
#### T4 — Topic Labeling\.
Definition:Assign a single short topic label \(one to three words\) to the utterance\. Input:“These data components in turn serve as the ‘building blocks’ of data exchanges\.” Output:data exchange
#### T5 — Numeric QA\.
Definition:Generate a question–answer pair where the answer is a number explicitly mentioned in the utterance\. Input:“Only one referendum has been held at the national level in Guyana\.” Output:Q: How many national\-level referendums have occurred in Guyana? \|\| A: 1 referendum
#### T6 — Gist Summarization \(DE\)\.
Definition:Produce a one\-sentence German gist summary of the spoken content\. Input:“…if you don’t want to look at all of the man pages, you just want to see a very short description of what a command does…” Output:Der Befehlwhatisliefert eine einzeilige Beschreibung eines Unix\-Kommandos, etwawhatis lsoderwhatis sort\.
#### T7 — Gist Summarization \(ZH\)\.
Definition:Produce a one\-sentence Mandarin Chinese gist summary of the spoken content\. Input:“…if you don’t want to look at all of the man pages, you just want to see a very short description of what a command does…” Output:使用whatis命令可以快速查看Linux命令的简要描述。
#### T8 — Vocal Style Description \(EN\)\.
Definition:Describe the speaker’s vocal style, tone, and delivery in English, inferred directly from the speech audio \(no transcript used\)\. Input:\[audio: “These data components in turn serve as the ‘building blocks’ of data exchanges\.”\] Output:The speaker has a measured, confident tone, speaking at a moderate pace with clear articulation\.
#### T9 — Vocal Style Description \(DE\)\.
Definition:Describe the speaker’s vocal style in German, inferred directly from the speech audio\. Input:\[audio: “These data components in turn serve as the ‘building blocks’ of data exchanges\.”\] Output:Die Sprecherin klingt ruhig und professionell mit einem gleichmäßigen Sprechtempo und einer freundlichen, sachlichen Betonung\.
#### T10 — Vocal Style Description \(ZH\)\.
Definition:Describe the speaker’s vocal style in Mandarin Chinese, inferred directly from the speech audio\. Input:\[audio: “These data components in turn serve as the ‘building blocks’ of data exchanges\.”\] Output:声音听起来平稳而中速,带有平静的语调,情感色彩比较客观。Similar Articles
NAVER LABS Europe Submission to the Instruction-following 2026 Short Track
This paper describes NAVER LABS Europe's submission to the IWSLT 2026 instruction-following short track, improving upon their previous winning system by using a new speech projector (SpeechMapper) trained solely on ASR data and augmenting training with a synthetic SQA dataset (fakACL). The resulting system ties for first place in the constrained track while using a weaker LLM backbone.
MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task
This paper describes the MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task, using Parakeet and Qwen 3.5 models with adaptive 'black-box' policies and a RAG mechanism for context, achieving significant quality improvements.
FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following
This paper describes FBK's submission to the IWSLT 2026 Instruction Following shared task, developing SpeechLLMs for short-form and long-form speech instruction following, exploring segmentation methods and achieving robust long-form performance with fixed 30-second segmentation.
VCIFBench: Evaluating Complex Instruction Following for Video Understanding
VCIFBench is a new benchmark for evaluating complex instruction following in video understanding, featuring 306 test instructions with content, format, style, and structure constraints, plus a DPO preference dataset. Experiments on 10 MLLMs reveal that joint constraint satisfaction remains challenging, and DPO training on the benchmark data improves instruction-following performance.
From Lexicon to AI: A Structured-Data Pipeline for Specialized Conversational Systems in Low-Resource Languages
Presents a systematic methodology for converting Hindi WordNet into 1.25 million instruction-response pairs to fine-tune a 12B-parameter language model using LoRA, demonstrating improved pedagogical effectiveness for specialized conversational systems in low-resource languages.