EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs

# EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs
Source: [https://arxiv.org/html/2605.23954](https://arxiv.org/html/2605.23954)
Liang Lin1,\*Chunxi Luo2,\*Kaiwen Luo1,\*Jie Zhang3 Jin Wang4Yuanhe Zhang5Cai Yuchen6Qiankun Li1 Gongli Xi7Zhenhong Zhou1Kun Wang1,↑\\uparrowJunhao Dong1,↑\\uparrow 1NTU2SHU3ICT, CAS4HDU5BUPT6USTC7SKL\-NST, BUPT linliang@iie\.ac\.cn \*Equal contribution\.↑\\uparrowCorresponding author

###### Abstract

Audio Large Language Models \(ALLMs\) are highly vulnerable to real\-world noise, which often induces severe semantic drift and hallucinations\. Existing robustness methods primarily rely on waveform\-level acoustic enhancement, answer\-level supervision, or the internal suppression of noise representations\. To address these issues, we proposeEchoDistill, an alignment\-based noisy\-to\-clean self\-distillation framework\.EchoDistillleverages a frozen clean\-audio teacher to provide semantic references for an inference\-time noisy\-audio student\. Specifically, the student samples candidate responses under noisy conditions to expose its test\-time behavior\. These trajectories are then optimized via group\-relative policy optimization \(GRPO\), where the token\-level consistency with the teacher acts as a reward bonus\. By aligning the noisy student’s candidate responses with clean semantic evidence, and applying audio\-aware reward shaping, our method encourages reasoning trajectories that are both correct and genuinely acoustically grounded\.EchoDistillsignificantly improves the semantic reliability and task performance of Audio LLMs under complex noise, without introducing any additional inference costs\. Extensive experiments show that: \(I\) Compared with the strongest baseline,EchoDistillachieves average improvements of 4\.18%↑\\uparrowin GSR under strong noise\. \(II\) Ablation results on Qwen\-Omni further show thatEchoDistillimproves over the GRPO\-only variant by 3\.02%↑\\uparrowin Acc, 3\.89%↑\\uparrowin Noisy, and 4\.53%↑\\uparrowin GSR on average\. Our codes are available at[https://anonymous\.4open\.science/r/echodistill\-10DE](https://anonymous.4open.science/r/echodistill-10DE)\.

## 1Introduction

Adutio Large Language Models \(ALLMs\) are increasingly deployed in real\-time applications such as in\-car assistants, online meetings, and audio\-based interaction systemsLiet al\.\([2025b](https://arxiv.org/html/2605.23954#bib.bib46)\), where audio serves as the primary interface for language understandingLiet al\.\([2015](https://arxiv.org/html/2605.23954#bib.bib1)\); Rubensteinet al\.\([2023](https://arxiv.org/html/2605.23954#bib.bib44)\); Radfordet al\.\([2023](https://arxiv.org/html/2605.23954#bib.bib45)\); Luoet al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib33)\)\. However, real\-world audio is often corrupted by device artifacts and environmental noise, which not only distorts the waveformDeliyskiet al\.\([2005](https://arxiv.org/html/2605.23954#bib.bib48)\); Gelbart and Morgan \([2002](https://arxiv.org/html/2605.23954#bib.bib47)\)but also degrades ALLMs generation qualityLinet al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib34)\); Kumar and Mishra \([2025](https://arxiv.org/html/2605.23954#bib.bib49)\); Xiaoet al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib37)\); Yinet al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib35)\)\. Under severe noise, models may misinterpret acoustic evidence and produce unstable responses, leading to semantic drift and hallucinationsShenet al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib50)\); Chenet al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib39)\); Yinet al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib38)\); Xiao and Das \([2025](https://arxiv.org/html/2605.23954#bib.bib36)\); Chenget al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib62),[2024](https://arxiv.org/html/2605.23954#bib.bib63)\)\.

Existing research on ALLM denoising has predominantly concentrated on inference\-stage interventions or feature\-level modificationsPurwinset al\.\([2019](https://arxiv.org/html/2605.23954#bib.bib43)\); Ephraim and Malah \([2003](https://arxiv.org/html/2605.23954#bib.bib41)\); Pascualet al\.\([2017](https://arxiv.org/html/2605.23954#bib.bib51)\); Zhanget al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib23)\)\. These runtime strategies attempt to suppress noise dynamically but fundamentally fall short of intrinsically enhancing the model’s robustness through the training process itself\. For example, while approaches like DFLPurwinset al\.\([2019](https://arxiv.org/html/2605.23954#bib.bib43)\)successfully mitigate certain noise corruptions, recent studyZhanget al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib23)\)demonstrates that they incur a severe alignment tax, resulting in a substantial degradation of general capabilities and downstream task performanceHouet al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib12)\); Liet al\.\([2015](https://arxiv.org/html/2605.23954#bib.bib1)\)\.

In this study, we seek to propose a novel training paradigm for enhancing the robustness of ALLMs:EchoDistill\. Specifically, we leverage an information gap to conduct self\-distillation—the student model takes noisy audio as input and performs inference, while the teacher model \(which is the model itself\) is granted access to the corresponding clean audio\. By combining trajectory\-level rewards from reinforcement learning\(Yuet al\.,[2025](https://arxiv.org/html/2605.23954#bib.bib7)\)with fine\-grained token\-level rewards from online distillation\(Yeet al\.,[2026](https://arxiv.org/html/2605.23954#bib.bib5); Tanet al\.,[2026](https://arxiv.org/html/2605.23954#bib.bib9); Zhaoet al\.,[2026](https://arxiv.org/html/2605.23954#bib.bib15); Sanget al\.,[2026](https://arxiv.org/html/2605.23954#bib.bib6)\),EchoDistillenables the model to align its noisy\-input output distribution with the clean\-audio semantic distribution under corrupted conditions, thereby fundamentally achieving substantial improvements in robustness\.

As shown in Figure[1](https://arxiv.org/html/2605.23954#S1.F1), we conduct extensive experiments on Qwen\-OmniChuet al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib25)\), MiniCPM\-oYaoet al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib26)\), and StepAudioWuet al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib27)\)under severe acoustic noise, covering Music, Sound, and Speech domainsKumaret al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib28)\); Hendryckset al\.\([2020](https://arxiv.org/html/2605.23954#bib.bib56)\)\. With GSRBenestyet al\.\([2009](https://arxiv.org/html/2605.23954#bib.bib31)\)as the primary robustness metric,EchoDistillimproves generation robustness across backbones and domainsKumar and Mishra \([2025](https://arxiv.org/html/2605.23954#bib.bib49)\)\. Compared with the strongest baseline,EchoDistillachieves an average improvement of 4\.18% in GSR\. Our main contributions are summarized as follows:

- •Through an in\-depth analysis of audio grounding behavior, we identify the inherent sparsity of acoustic evidence in correct reasoning trajectories, demonstrating that fine\-grained token\-level semantic correction is fundamentally required for reliable audio reasoning\.
- •We proposeEchoDistill, a novel noisy\-to\-clean self\-distillation framework that seamlessly aligns noisy\-audio student inference trajectories with privileged clean\-audio teacher semantics at the token level, thereby enabling the model to remain firmly grounded in reliable acoustic evidence rather than drifting toward language priors\.
- •Extensive experiments conducted across multiple representative ALLMs and diverse audio domains demonstrate thatEchoDistillsignificantly and consistently improves GSR\-oriented generation robustness under severe acoustic corruption, with particularly remarkable gains observed in the highly challenging Sound and Speech domains\.

Qwen\-Omni\-3BQwen\-Omni\-7BGPT\-audioStudent from Qwen
Noisy Acc\. \(%\)↑\\uparrowGSR \(%\)↑\\uparrow505060607070808050\.850\.866\.466\.45959747451\.451\.467\.567\.559\.259\.275\.175\.1Performance \(%\)\(a\) MusicNoisy Acc\. \(%\)↑\\uparrowGSR \(%\)↑\\uparrow505060607070808060\.860\.867\.167\.158\.258\.265\.565\.552\.252\.268\.268\.266\.866\.875\.375\.3\(b\) SoundNoisy Acc\. \(%\)↑\\uparrowGSR \(%\)↑\\uparrow505060607070808054\.554\.569\.869\.860\.860\.874\.574\.553\.753\.775\.475\.462\.962\.97878\(c\) Speech

Figure 1:Task\-wise comparison of noisy accuracy and GSR across Music, Sound, and Speech\. Overall, Student from Qwen achieves the best performance across all three task categories, consistently outperforming the baseline models on both noisy accuracy and GSR\. The advantage is particularly clear on GSR, indicating that the student model provides more robust and acoustically grounded responses under noisy conditions\.
## 2Preliminary

#### Group Relative Policy Optimization\.

Group Relative Policy Optimization \(GRPO\)Shaoet al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib55)\)is a group\-based policy optimization method for generative models\. Given an inputxx, the policyπθ\\pi\_\{\\theta\}samples a group of responses\{y\(k\)\}k=1K\\\{y^\{\(k\)\}\\\}\_\{k=1\}^\{K\}and constructs group\-relative advantages from rewardsr\(k\)r^\{\(k\)\}:

A\(k\)=r\(k\)−μσ\+ε,A^\{\(k\)\}=\\frac\{r^\{\(k\)\}\-\\mu\}\{\\sigma\+\\varepsilon\},\(1\)whereμ\\muandσ\\sigmaare the mean and standard deviation of group rewards\. For autoregressive policies, GRPO is usually written at the token level by comparing the new and old policies on the same sampled trajectory\. Letρt\(k\)=πθ​\(yt\(k\)∣x,y<t\(k\)\)πθold​\(yt\(k\)∣x,y<t\(k\)\)\\rho\_\{t\}^\{\(k\)\}=\\frac\{\\pi\_\{\\theta\}\(y\_\{t\}^\{\(k\)\}\\mid x,y\_\{<t\}^\{\(k\)\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{t\}^\{\(k\)\}\\mid x,y\_\{<t\}^\{\(k\)\}\)\}\. Then a common clipped objective is:

ℒGRPO=−1K​∑k=1K1\|y\(k\)\|​∑t=1\|y\(k\)\|min⁡\(ρt\(k\)​A\(k\),clip⁡\(ρt\(k\),1−ϵ,1\+ϵ\)​A\(k\)\)\.\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}=\-\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\frac\{1\}\{\|y^\{\(k\)\}\|\}\\sum\_\{t=1\}^\{\|y^\{\(k\)\}\|\}\\min\\Big\(\\rho\_\{t\}^\{\(k\)\}A^\{\(k\)\},\\operatorname\{clip\}\(\\rho\_\{t\}^\{\(k\)\},1\-\\epsilon,1\+\\epsilon\)A^\{\(k\)\}\\Big\)\.\(2\)This objective preserves group\-relative preference signals while avoiding the overhead of an additional value model\. Its limitation is that sequence\-level rewards cannot distinguish evidence quality across different reasoning trajectories, and the same relative advantage is still difficult to assign precisely to critical tokens\.

#### Self\-Distillation\.

Self\-distillation transfers knowledge within the same model family by constructing a teacher and a student from different stagesShenfeldet al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib21)\); Hübotteret al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib16)\); Songet al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib20)\); Tanet al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib9)\)\. Letpϕp\_\{\\phi\}andpθp\_\{\\theta\}denote the teacher and student distributions\. For autoregressive generation, output\-level self\-distillation can be written as

ℒSD=1T∑t=1TDKL\(pϕ\(⋅∣x,y<t\)∥pθ\(⋅∣x,y<t\)\)\.\\mathcal\{L\}\_\{\\mathrm\{SD\}\}=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}D\_\{\\mathrm\{KL\}\}\\Big\(p\_\{\\phi\}\(\\cdot\\mid x,y\_\{<t\}\)\\;\\\|\\;p\_\{\\theta\}\(\\cdot\\mid x,y\_\{<t\}\)\\Big\)\.\(3\)Self\-distillation can therefore be viewed as consistency regularization over both output distributions and internal representationsHintonet al\.\([2015](https://arxiv.org/html/2605.23954#bib.bib18)\); Zhaoet al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib15)\)\. The key distinction lies in the fact that the teacher itself is granted access to additional privileged information, which is unavailable to the studentFurlanelloet al\.\([2018](https://arxiv.org/html/2605.23954#bib.bib60)\)\.

#### Audio Large Language Models\.

Audio large language models extend large language models to speech and general audio understanding and generation by coupling pretrained audio encoders with generative language backbonesTanget al\.\([2023](https://arxiv.org/html/2605.23954#bib.bib57)\); Huanget al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib58)\)\. Given an audio inputaa, a common approach first extracts acoustic features using an encoderE​\(⋅\)E\(\\cdot\)and then maps them, via discrete tokenization or a cross\-modal adapterg​\(⋅\)g\(\\cdot\), into a conditioning representation consumable by the language backboneKoet al\.\([2015](https://arxiv.org/html/2605.23954#bib.bib2)\); Liet al\.\([2015](https://arxiv.org/html/2605.23954#bib.bib1)\)\. The conditional generation process is

π​\(y∣x,a\)=∏t=1Tπ​\(yt∣x,g​\(E​\(a\)\),y<t\),\\pi\(y\\mid x,a\)=\\prod\_\{t=1\}^\{T\}\\pi\(y\_\{t\}\\mid x,g\(E\(a\)\),y\_\{<t\}\),\(4\)wherexxis the text prompt andg​\(E​\(a\)\)g\(E\(a\)\)is the audio\-conditioned representation induced by the input audio\. Another line of work discretizes the audio signal into token sequencesca=q​\(a\)c\_\{a\}=q\(a\)and feeds them directly into the language backbone\. In either case, the central goal is to align continuous acoustic signals into semantic representations that can be processed by language models\.

## 3Methodology

In this section, we proposeEchoDistill, an alignment\-based noisy\-to\-clean self\-distillation framework\. By leveraging paired data,EchoDistillaligns noisy\-input representations with internal clean\-audio semantic evidence, enhancing reliability\.

Specifically, letaina\_\{i\}^\{n\}denote the noisy audio observed at inference time, and letaica\_\{i\}^\{c\}denote the clean audio with the same semantic content\. Given paired training data

𝒟=\{\(xi,ain,aic,yi⋆,ci\)\}i=1N,\\mathcal\{D\}=\\\{\(x\_\{i\},a\_\{i\}^\{n\},a\_\{i\}^\{c\},y\_\{i\}^\{\\star\},c\_\{i\}\)\\\}\_\{i=1\}^\{N\},\(5\)wherexix\_\{i\}is the text prompt,yi⋆y\_\{i\}^\{\\star\}is the target answer, andcic\_\{i\}denotes answer choices\. During training, both noisy and clean versions of a query are available; during inference, only\(xi,ain\)\(x\_\{i\},a\_\{i\}^\{n\}\)is used\. The goal is to learnπθ​\(y∣xi,ain\)\\pi\_\{\\theta\}\(y\\mid x\_\{i\},a\_\{i\}^\{n\}\)to solve the task directly from noisy audio, while usingaica\_\{i\}^\{c\}only as reliable training\-time acoustic evidence for calibrating the noisy policy\.

### 3\.1Sparse Audio Grounding

We first characterize how audio evidence contributes to correct predictionsHuanget al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib24)\)\. Given audioaa, letMi​\(a\)M\_\{i\}\(a\)denote how much the correct answer score exceeds the strongest incorrect answer score; a larger gap means that the model more stably prefers the correct answer\. We measure the effect of removing a local audio windowwwand the effect of removing audio entirely as

di,w=Mi​\(ai\)−Mi​\(ai∖w\),gi=Mi​\(ai\)−Mi​\(ai∅\)\.d\_\{i,w\}=M\_\{i\}\(a\_\{i\}\)\-M\_\{i\}\\\!\\left\(a\_\{i\}^\{\\setminus w\}\\right\),\\quad g\_\{i\}=M\_\{i\}\(a\_\{i\}\)\-M\_\{i\}\\\!\\left\(a\_\{i\}^\{\\emptyset\}\\right\)\.\(6\)whereai∖wa\_\{i\}^\{\\setminus w\}denotes the audio after ablating windowww, andai∅a\_\{i\}^\{\\emptyset\}denotes the no\-audio condition\. If removing a window substantially reducesMiM\_\{i\}, that window contributes to the correct decision; if removing all audio substantially reducesMiM\_\{i\}, the trajectory is globally anchored in audio\. Thus,di,wd\_\{i,w\}measures the local contribution of a specific acoustic segment, whilegig\_\{i\}measures the trajectory\-level gain of using audio over text\-only inference\.

![Refer to caption](https://arxiv.org/html/2605.23954v1/figures/audio_grounding_sparse_trend.png)Figure 2:Sparse and uneven audio grounding in correct trajectories\.\(a\) Distribution of token/window\-level audio dependencydi,wd\_\{i,w\}, measured by the decision\-margin drop after ablating a local audio window\. \(b\) Distribution of trajectory\-level audio anchorgig\_\{i\}, measuring the gain of using audio over the no\-audio condition\.Figure[2](https://arxiv.org/html/2605.23954#S3.F2)reveals two key observations\. On the one hand, audio evidence is locally sparse: most windows induce negligible margin changes, while only a small subset leads to a substantial drop after ablation\. This suggests that audio\-grounded reasoning is often anchored in a few critical acoustic segments rather than uniformly across the entire waveform\. On the other hand, grounding among correct trajectories is highly uneven: even when responses are correct, the trajectory\-level gaingig\_\{i\}can vary substantially\. Therefore, correctness alone does not reliably indicate whether a response is strongly supported by audio evidence\.

These observations motivate the design ofEchoDistill\. Since a correct response may not necessarily rely on audio evidence, it is important to consider the candidates generated by the noisy student under noisy inputs, rather than relying solely on offline answers\. This insight motivates the use of noisy student rollouts\. Due to the sparsity of local evidence, sequence\-level rewards are too coarse to identify token\-level predictions grounded in reliable acoustic signals, motivating noisy\-to\-clean distribution alignment\. Meanwhile, variation in grounding strength across correct trajectories implies that not all correct responses should receive equal credit, motivating audio\-aware reward shaping\.

![Refer to caption](https://arxiv.org/html/2605.23954v1/figures/echopipelinev11.png)Figure 3:Overview of EchoDistill\.EchoDistill uses a noisy\-audio student and a frozen clean\-audio teacher during training, combining task reward, noisy\-to\-clean evidence alignment, and preference optimization while retaining single\-student inference\.
### 3\.2EchoDistillFramework

#### Noisy student rollouts\.

EchoDistillfirst starts from the behavior of the inference\-time noisy student\. For each instance, the noisy student samples a group of candidate responses fromaina\_\{i\}^\{n\}, and each candidate receives a task reward,as shown in Figure[3](https://arxiv.org/html/2605.23954#S3.F3):

yi\(k\)\\displaystyle y\_\{i\}^\{\(k\)\}∼πθ\(⋅∣xi,ain\),k=1,…,K,\\displaystyle\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\_\{i\},a\_\{i\}^\{n\}\),\\qquad k=1,\\ldots,K,\(7\)ri\(k\)\\displaystyle r\_\{i\}^\{\(k\)\}=Rtask​\(yi\(k\);yi⋆,ci,y~i\)\.\\displaystyle=R\_\{\\mathrm\{task\}\}\\\!\\left\(y\_\{i\}^\{\(k\)\};y\_\{i\}^\{\\star\},c\_\{i\},\\tilde\{y\}\_\{i\}\\right\)\.Here,y~i\\tilde\{y\}\_\{i\}denotes clean\-teacher guidance when available\. Rewards are primarily computed from target\-answer or choice matching; for open\-ended or weakly annotated examples, the teacher response is used only as a soft semantic reference rather than a hard label\. This step exposes optimization to the student’s actual noisy\-input distribution, including linguistically plausible but acoustically unsupported errors\.

#### Noisy\-to\-clean evidence alignment\.

To address the coarseness of sequence\-level rewards,EchoDistillconverts clean\-audio preferences into token\-level supervision for the noisy\-input student\. We select a guidance responsey~i\\tilde\{y\}\_\{i\}as the shared continuation; it can be generated by the clean teacher or selected from student candidates using clean\-teacher likelihood\. We then compute the next\-token distributions of the clean teacher and noisy student on the same continuation:

qϕ,t\(i\)=πϕ\(⋅∣xi,aic,y~i,<t\),pθ,t\(i\)=πθ\(⋅∣xi,ain,y~i,<t\)\.q\_\{\\phi,t\}^\{\(i\)\}=\\pi\_\{\\phi\}\(\\cdot\\mid x\_\{i\},a\_\{i\}^\{c\},\\tilde\{y\}\_\{i,<t\}\),\\quad p\_\{\\theta,t\}^\{\(i\)\}=\\pi\_\{\\theta\}\(\\cdot\\mid x\_\{i\},a\_\{i\}^\{n\},\\tilde\{y\}\_\{i,<t\}\)\.\(8\)The distillation lossℒdistill\(i\)\\mathcal\{L\}\_\{\\mathrm\{distill\}\}^\{\(i\)\}is the masked teacher\-to\-student KL between these distributions over response tokens\. The maskmi,tm\_\{i,t\}excludes prompt tokens, audio placeholders, and dialogue templates, and the stop\-gradient operatorsg⁡\[⋅\]\\operatorname\{sg\}\[\\cdot\]keeps the teacher frozen\. This loss does not force the student to copy a single teacher sentence; instead, it encourages the noisy\-input distribution to preserve the clean\-audio semantic preference while generating the answer\.

#### Audio\-aware reward shaping\.

To differentiate between responses that are merely correct and those that are also grounded in audio evidence, the alignment signal is leveraged to guide reward shaping\. We convert the noisy\-to\-clean discrepancy into an audio\-aware similarity score and apply it only to task\-positive candidates:

si=exp⁡\(−ℒdistill\(i\)\),r¯i\(k\)=clip⁡\(ri\(k\)\+β​1​\[ri\(k\)\>0\]​si,−1,2\)\.s\_\{i\}=\\exp\\\!\\left\(\-\\mathcal\{L\}\_\{\\mathrm\{distill\}\}^\{\(i\)\}\\right\),\\ \\;\\;\\bar\{r\}\_\{i\}^\{\(k\)\}=\\operatorname\{clip\}\\\!\\left\(r\_\{i\}^\{\(k\)\}\+\\beta\\,\\mathbf\{1\}\[r\_\{i\}^\{\(k\)\}\>0\]\\,s\_\{i\},\-1,\\,2\\right\)\.\(9\)Largersis\_\{i\}indicates closer agreement between the noisy student and the clean teacher, andβ\\betacontrols the strength of the audio\-aware bonus\. The indicator𝟏​\[ri\(k\)\>0\]\\mathbf\{1\}\[r\_\{i\}^\{\(k\)\}\>0\]means that only candidates with a positive original task reward receive this bonus; an incorrect candidate is not rewarded by this term even if its distribution is close to the teacher\. Clipping stabilizes the reward scale\. If audio\-aware shaping is disabled,r¯i\(k\)=ri\(k\)\\bar\{r\}\_\{i\}^\{\(k\)\}=r\_\{i\}^\{\(k\)\}\.

### 3\.3Optimization and Inference

Given the shaped rewards,EchoDistillcomputes group\-relative advantagesAi\(k\)A\_\{i\}^\{\(k\)\}over\{r¯i\(k\)\}k=1K\\\{\\bar\{r\}\_\{i\}^\{\(k\)\}\\\}\_\{k=1\}^\{K\}and optimizes the sampled candidates\. Letℓθ\(k\)\\ell\_\{\\theta\}^\{\(k\)\}andℓϕ\(k\)\\ell\_\{\\phi\}^\{\(k\)\}denote the average sequence log\-probabilities ofyi\(k\)y\_\{i\}^\{\(k\)\}under the noisy student and clean teacher, respectively\. Rather than introducing an additional value model or relying on a generic old\-policy reference,EchoDistilluses the clean teacher as a detached reference scorer for the same sampled response:

ρi\(k\)=exp⁡\(ℓθ\(k\)−sg⁡\[ℓϕ\(k\)\]\)\.\\rho\_\{i\}^\{\(k\)\}=\\exp\\\!\\left\(\\ell\_\{\\theta\}^\{\(k\)\}\-\\operatorname\{sg\}\[\\ell\_\{\\phi\}^\{\(k\)\}\]\\right\)\.\(10\)The clipped group\-relative loss is then evaluated withρi\(k\)\\rho\_\{i\}^\{\(k\)\}andAi\(k\)A\_\{i\}^\{\(k\)\}, yieldingℒpolicy\(i\)\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\(i\)\}\. This objective favors responses that both receive task reward under noisy input and remain plausible under clean\-audio evidence\. The final training loss is

ℒEchoDistill=1N​∑i=1N\[λpolicy​ℒpolicy\(i\)\+λdistill​ℒdistill\(i\)\]\.\\mathcal\{L\}\_\{\\mathrm\{\{\\textbf\{EchoDistill\}\}\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\Big\[\\lambda\_\{\\mathrm\{policy\}\}\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\(i\)\}\+\\lambda\_\{\\mathrm\{distill\}\}\\mathcal\{L\}\_\{\\mathrm\{distill\}\}^\{\(i\)\}\\Big\]\.\(11\)Only the student parametersθ\\thetaare updated; the teacher remains frozen throughout training\. At inference time,EchoDistillremoves the teacher branch, clean audio, and reward computation, and predicts withπθ​\(y∣x,an\)\\pi\_\{\\theta\}\(y\\mid x,a^\{n\}\), preserving the original single\-model inference cost\.

## 4Experiments

In this section, we conduct experiments to address the following research questions:

- •RQ1:To what extent does EchoDistill improve the task accuracy and GSR of Audio LLMs under severe acoustic noise, and does it effectively bridge the gap between noisy and clean audio inputs?
- •RQ2:How do the individual components of the framework, policy optimization and noisy\-to\-clean distillation, independently and jointly contribute to the observed gains in accuracy and GSR?
- •RQ3:Why does EchoDistill exhibit particularly significant robustness gains in the Sound and Speech domains compared to Music, and how does the alignment\-based approach resolve semantic drift caused by noise masking?

### 4\.1Experimental Setup

#### Models and data\.

We use a clean\-teacher/noisy\-student setting\. The teacher and student share the same backbone; the teacher receives clean audio and is frozen, while the student receives noisy audio and is updated by EchoDistill\. Main experiments cover Qwen2\.5\-OmniChuet al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib25)\), MiniCPM\-o\-2\.6Yaoet al\.\([2024](https://arxiv.org/html/2605.23954#bib.bib26)\), and Step\-Audio2Wuet al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib27)\); ablations are conducted on Qwen2\.5\-Omni\. Training and validation use paired clean/noisy audio from MMARMaet al\.\([2025](https://arxiv.org/html/2605.23954#bib.bib29)\), with 14,397 training samples and 1,704 validation samples\. The data cover 10 noise typesHu and Wang \([2010](https://arxiv.org/html/2605.23954#bib.bib40)\)and 7 SNR levels\{−10,−5,0,5,10,20,30\}\\\{\-10,\-5,0,5,10,20,30\\\}Zhanget al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib23)\)\. Main evaluation is conducted on noisy MMAU\(Kumaret al\.,[2026](https://arxiv.org/html/2605.23954#bib.bib28)\)atSNR=−10\\mathrm\{SNR\}=\-10\.

Table 1:Main robustness results ofEchoDistillacross three metrics: Accuracy \(Acc\), Noisy performance, and Generation Success Rate \(GSR\) under strong noise \(SNR=\-10\)\. The relative improvements \(↑\\uparrow\) or drops \(↓\\downarrow\) compared to theInitialperformance are shown in red and dark blue, respectively\. The best and second\-best results are highlighted inboldandunderlined\. The highlighted rows represent our proposed method\.Baseline Methods and Evaluation Metrics\.We compare EchoDistill with four representative noise\-robustness methods, including the frequency\-domain processing methodSTFTEphraim and Malah \([2003](https://arxiv.org/html/2605.23954#bib.bib41)\), the wavelet\-transform methodWT[7](https://arxiv.org/html/2605.23954#bib.bib42), the deep\-learning\-based denoising methodDFLPurwinset al\.\([2019](https://arxiv.org/html/2605.23954#bib.bib43)\), and the representation\-level noise suppression methodSEENZhanget al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib23)\)\. Specifically,STFTandWTrepresent traditional signal\-processing\-based front\-end denoising methods,DFLrepresents a learning\-based speech enhancement method, andSEENdirectly suppresses noise\-related components in the internal representation space of ALLMs\. For fair comparison, all methods follow the same answer extraction protocol\. We report three metrics:AccHendryckset al\.\([2020](https://arxiv.org/html/2605.23954#bib.bib56)\),NoisyZhanget al\.\([2026](https://arxiv.org/html/2605.23954#bib.bib23)\), andGSRBenestyet al\.\([2009](https://arxiv.org/html/2605.23954#bib.bib31)\)\.

### 4\.2Performance on Task Accuracy and Generation Success \(RQ1\)

Table[1](https://arxiv.org/html/2605.23954#S4.T1)summarizes the main robustness results under strong acoustic corruption\. SinceGSRdirectly reflects whether an ALLMs can maintain successful generation when the input is corrupted, we use it as the primary robustness metric, whileAccandNoisyprovide complementary evidence for task correctness and noisy\-to\-clean recovery\. Based on Table[1](https://arxiv.org/html/2605.23954#S4.T1), we have the following observations:

Obs\. 1: EchoDistill achieves the strongest overall robustness and effectively bridges the noisy\-clean gap\.Across all domains and backbones, EchoDistill achieves the best average performance, with 61\.44% Acc, 76\.03% Noisy, and 79\.28% GSR, as shown in Figures[4](https://arxiv.org/html/2605.23954#S4.F4)\. The improvement is most evident in GSR, showing that EchoDistill stabilizes generation behavior under severe acoustic noise rather than merely improving Acc\. This consistent advantage indicates that the robustness gains are not tied to a specific backbone, but arise from the noisy\-to\-clean alignment mechanism, which bridges the behavioral gap between corrupted acoustic perception and clean semantic interpretation\.

Obs\. 2: EchoDistill is particularly effective in challenging semantic recovery scenarios\.In the Speech domain, which represents the most challenging scenario for semantic recovery due to the masking of fine\-grained phonetic and linguistic cues, EchoDistill achieves the best results across all three backbones\. For example, on Qwen2\.5\-Omni, it improves Acc from 59\.79% to 62\.89% and reaches the highest GSR of 78\.01%; on Step\-Audio2, it improves GSR from 68\.47% under STFT to 74\.89%\. The Sound domain further highlights EchoDistill’s capability to preserve acoustic grounding, where it raises Qwen2\.5\-Omni GSR from 68\.25% to 75\.33% and achieves the highest GSR of 80\.42% on MiniCPM\-o\-2\.6\. These results suggest that clean\-teacher guidance helps the noisy student remain grounded in reliable acoustic evidence rather than drifting toward language\-prior hallucinations\. In Music, although signal\-processing baselines are competitive in Acc, EchoDistill still shows superior generative stability, reaching 86\.00% GSR on Step\-Audio2 compared with 79\.00% for STFT\.

Strongest baselineEchoDistillGain margin

StepQwenMini666668687070727274747676\+3\.10\+3\.47\+0\.69SettingCRS \(%\)Backbone\-wiseMusicSoundSpeech666668687070727274747676\+0\.77\+3\.14\+3\.04SettingDomain\-wiseStepQwenMiniMusiSounSpeeOver666668687070727274747676\+3\.47\+3\.14\+2\.57SettingAll settings

Figure 4:EchoDistill consistently achieves higher Composite Robustness Score than the strongest baseline across backbones and domains, whereCRS=\(Acc\+Noisy\+GSR\)/3\\mathrm\{CRS\}=\(\\mathrm\{Acc\}\+\\mathrm\{Noisy\}\+\\mathrm\{GSR\}\)/3\.
### 4\.3Performance on Component Ablation and Synergistic Regularization \(RQ2\)

Table 2:Ablation on Qwen2\.5\-Omni under several noise\.The last three columns report gains over GRPO\-only within each domain\.To analyze the individual and joint contributions of policy optimization and noisy\-to\-clean distillation, we conduct ablation studies on Qwen2\.5\-Omni\. Table[2](https://arxiv.org/html/2605.23954#S4.T2)compares full EchoDistill with two variants:GRPO\-onlyandDistill\-only\. Based on Table[2](https://arxiv.org/html/2605.23954#S4.T2), we have the following observations:

Obs\. 3: Noisy\-to\-clean distillation provides the core semantic anchor for robust generation\.Compared withGRPO\-only,Distill\-onlyimproves the average Acc, Noisy, and GSR by 1\.21%, 1\.80%, and 2\.58%, respectively\. The larger gain on GSR shows that clean\-audio guidance is especially effective at reducing noise\-induced behavior shifts\.

Obs\. 4: Policy optimization and noisy\-to\-clean distillation are complementary\.Full EchoDistill achieves the best average performance, with 62\.96% Acc, 75\.36% Noisy, and 76\.14% GSR\. It improves GSR by 4\.53% overGRPO\-onlyand by 1\.95% overDistill\-only\. The most significant gain appears in Sound, where EchoDistill increases GSR from 66\.16% to 75\.33%\. These results indicate that distillation aligns the noisy student with clean semantic evidence, while policy optimization further encourages task\-correct and reward\-aligned reasoning trajectories\.

![Refer to caption](https://arxiv.org/html/2605.23954v1/x1.png)Figure 5:Qwen2\.5\-Omni results under−10\-10dB noise on Music, Sound, and Speech\.
### 4\.4Evaluation of Domain\-Specific Robustness and Cross\-Backbone Generalization \(RQ3\)

To understand the domain\-specific robustness of EchoDistill, we analyze its performance on Music, Sound, and Speech in Table[1](https://arxiv.org/html/2605.23954#S4.T1)\. Based on the results, we have the following observations:

Obs\. 5: EchoDistill shows the strongest gains in Sound and Speech\.In Sound, EchoDistill achieves the highest GSR across all backbones, reaching 80\.08%, 75\.33%, and 80\.42%\. In the speech domain, which is the most challenging scenario for semantic recovery, EchoDistill still achieves the best GSR across all backbones\. These results suggest that EchoDistill is particularly effective in scenarios where noise obscures local acoustic details, such as events or speech phonemes\. By leveraging the clean\-audio teacher’s privileged semantic evidence, the student learns to reconstruct degraded inputs into coherent semantic representations, Figure[5](https://arxiv.org/html/2605.23954#S4.F5)uses the Qwen model as an example\.

Obs\. 6: The domain pattern supports the semantic\-drift explanation\.Music shows smaller Acc differences because its global and redundant acoustic structures makeSTFTcompetitive in Acc\. Nevertheless, EchoDistill still achieves stronger generation consistency, with 86\.00% GSR on Step\-Audio2 and 87\.24% on MiniCPM\-o\-2\.6\. This divergence shows that even when multiple methods produce correct answers, EchoDistill generates trajectories more strictly aligned with the clean\-audio reference\. EchoDistill reduces semantic drift by maximizing noisy\-to\-clean consistency, enabling robust audio reasoning under severe acoustic corruption\.

### 4\.5Semantic Drift Mitigation and Scalability Analysis \(RQ3\)

To understand how the alignment\-based design addresses semantic drift caused by noise masking, we further analyze the noisy\-to\-clean consistency dynamics in Fig\.[6](https://arxiv.org/html/2605.23954#S4.F6)\. Based on the results, we list observations:

![Refer to caption](https://arxiv.org/html/2605.23954v1/figures/dytrain.png)Figure 6:Training dynamics of noisy\-to\-clean consistency\. MiniCPM improves rapidly in the early stage, while StepAudio exhibits a steadier upward trend, indicating that EchoDistill progressively aligns noisy\-input generation with clean\-audio semantic behavior\.Obs\. 7: Noisy\-to\-clean alignment progressively suppresses semantic drift\.As shown in Fig\.[6](https://arxiv.org/html/2605.23954#S4.F6), both StepAudio and MiniCPM show increasing noisy\-to\-clean consistency throughout training, demonstrating that EchoDistill progressively aligns noisy\-input generation with clean\-audio semantic evidence\. EchoDistill not only optimizes final\-answer correctness, but also directly constrains the response trajectory under noisy inputs to remain close to clean\-audio behavior\. Therefore, when noise masks local acoustic evidence, the student is encouraged to recover the semantic direction supported by the clean teacher rather than drifting toward spurious language\-prior predictions\. The curve indicates that EchoDistill does not mitigate semantic drift by simply removing waveform noise, but by reshaping the student’s generation behavior toward clean\-audio semantic consistency\.

Table 3:Complementary gains ofEchoDistill\+SEEN\. Gains are reported in percentage points\.DataMetricE\.\+SEENΔ\\DeltaSEENΔ\\DeltaE\.MusicAcc59\.66\+2\.66\+2\.66−0\.09\-0\.09Noisy74\.50\+3\.75\+3\.75−3\.66\-3\.66SoundAcc67\.33\+7\.50\+7\.50\+3\.66\+3\.66Noisy73\.58\+6\.33\+6\.33\+1\.83\+1\.83SpeechAcc63\.13\+11\.79\+11\.79\+7\.84\+7\.84Noisy78\.76\+12\.13\+12\.13\+7\.72\+7\.72Avg\.–69\.49\+7\.36\+7\.36\+2\.88\+2\.88

Obs\. 8: Semantic alignment is complementary to front\-end denoising\.Although EchoDistill operates at the semantic generation level, it is independent of specific acoustic enhancement modules\. This makes it naturally compatible with external denoising methods, which mainly reduce low\-level waveform\- or spectrogram\-level corruption before the student processes the audio\. As shown in Table[3](https://arxiv.org/html/2605.23954#S4.T3), front\-end denoising and noisy\-to\-clean alignment address different stages of the robustness problem: front\-end denoising alleviates the acoustic masking effect, while EchoDistill further constrains high\-level response trajectories to follow clean\-audio semantics\. Therefore, their combination provides a scalable path for improving robustness under severe noise, where cleaner acoustic inputs and stronger semantic alignment can jointly reduce noise\-induced semantic drift\.

## 5Conclusion & Limitations

In this work, we proposeEchoDistill, an alignment\-based noisy\-to\-clean self\-distillation framework for improving the robustness of ALLMs under severe acoustic noise\. By combining a frozen clean\-audio teacher with noisy student rollouts, token\-level noisy\-to\-clean alignment, and audio\-aware reward shaping,EchoDistillencourages responses that are both task\-correct and acoustically grounded\. Experiments across three ALLMs show consistent robustness improvements, especially in GSR, while ablations confirm that noisy\-to\-clean distillation provides the core semantic anchor and policy optimization brings complementary gains\.

Despite its effectiveness,EchoDistillstill depends on the reliability of the clean\-audio teacher and requires extra training\-time computation for noisy rollouts, teacher scoring, and distribution\-level alignment\. Moreover, the current framework mainly focuses on single\-audio understanding\. Future work may extend noisy\-to\-clean alignment to other modalities\.

## References

- \[1\]\(2024\)On\-policy distillation of language models: learning from self\-generated mistakes\.InThe twelfth international conference on learning representations,Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p2.1)\.
- \[2\]J\. Benesty, J\. Chen, Y\. Huang, and I\. Cohen\(2009\)Pearson correlation coefficient\.InNoise reduction in speech processing,pp\. 1–4\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p4.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p2.1)\.
- \[3\]Y\. Chen, Y\. Xiao, H\. Yin, Y\. Guan, and X\. Liu\(2025\)Noise\-robust sound event detection and counting via language\-queried sound separation\.IEEE Signal Processing Letters\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[4\]X\. Cheng, D\. Fu, X\. Yang, M\. Fang, R\. Hu, J\. Lu, B\. Jionghao, Z\. Wang, S\. Ji, R\. Huang,et al\.\(2025\)OmniChat: enhancing spoken dialogue systems with scalable synthetic data for diverse scenarios\.arXiv preprint arXiv:2501\.01384\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[5\]X\. Cheng, S\. Zheng, Z\. Wang, M\. Fang, Z\. Zhang, R\. Huang, Z\. Ma, S\. Ji, J\. Zuo, T\. Jin,et al\.\(2024\)OmniSep: unified omni\-modality sound separation with query\-mixup\.arXiv preprint arXiv:2410\.21269\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[6\]Y\. Chu, J\. Xu, Q\. Yang, H\. Wei, X\. Wei, Z\. Guo, Y\. Leng, Y\. Lv, J\. He, J\. Lin,et al\.\(2024\)Qwen2\-audio technical report\.arXiv preprint arXiv:2407\.10759\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p4.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p1.2)\.
- \[7\]\(1995\)De\-noising by soft\-thresholding\.IEEE transactions on information theory41\(3\),pp\. 613–627\.Cited by:[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p2.1)\.
- \[8\]D\. D\. Deliyski, H\. S\. Shaw, and M\. K\. Evans\(2005\)Adverse effects of environmental noise on acoustic voice quality measurements\.Journal of Voice19\(1\),pp\. 15–28\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[9\]Y\. Ephraim and D\. Malah\(2003\)Speech enhancement using a minimum\-mean square error short\-time spectral amplitude estimator\.IEEE Transactions on acoustics, speech, and signal processing32\(6\),pp\. 1109–1121\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p2.1)\.
- \[10\]T\. Furlanello, Z\. Lipton, M\. Tschannen, L\. Itti, and A\. Anandkumar\(2018\)Born again neural networks\.InInternational conference on machine learning,pp\. 1607–1616\.Cited by:[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px2.p1.3)\.
- \[11\]D\. Gelbart and N\. Morgan\(2002\)Double the trouble: handling noise and reverberation in far\-field automatic speech recognition\.\.InInterspeech,pp\. 2185–2188\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[12\]S\. Gopal, A\. Anshul, H\. Li, Y\. H\. Yeo, H\. Liu, and E\. S\. Chng\(2025\)Explainable disentanglement on discrete speech representations for noise\-robust asr\.In2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference \(APSIPA ASC\),pp\. 2535–2540\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1)\.
- \[13\]D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt\(2020\)Measuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p4.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p2.1)\.
- \[14\]G\. Hinton, O\. Vinyals, and J\. Dean\(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px2.p1.3)\.
- \[15\]G\. Hou, J\. He, Y\. Zhou, J\. Guo, Y\. Qiao, R\. Zhang, and W\. Jiang\(2025\)Evaluating robustness of large audio language models to audio injection: an empirical study\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 25671–25687\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2605.23954#S1.p2.1)\.
- \[16\]G\. Hu and D\. Wang\(2010\)A tandem algorithm for pitch estimation and voiced speech segregation\.IEEE Transactions on Audio, Speech, and Language Processing18\(8\),pp\. 2067–2079\.Cited by:[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p1.2)\.
- \[17\]R\. Huang, M\. Li, D\. Yang, J\. Shi, X\. Chang, Z\. Ye, Y\. Wu, Z\. Hong, J\. Huang, J\. Liu,et al\.\(2024\)Audiogpt: understanding and generating speech, music, sound, and talking head\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 23802–23804\.Cited by:[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px3.p1.3)\.
- \[18\]S\. Huang, X\. Qu, Y\. Li, Y\. Luo, Z\. He, D\. Liu, and Y\. Cheng\(2025\)Spotlight on token perception for multimodal reinforcement learning\.arXiv preprint arXiv:2510\.09285\.Cited by:[§3\.1](https://arxiv.org/html/2605.23954#S3.SS1.p1.3)\.
- \[19\]J\. Hübotter, F\. Lübeck, L\. Behric, A\. Baumann, M\. Bagatella, D\. Marta, I\. Hakimi, I\. Shenfeld, T\. K\. Buening, C\. Guestrin,et al\.\(2026\)Reinforcement learning via self\-distillation\.arXiv preprint arXiv:2601\.20802\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p2.1),[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px2.p1.2)\.
- \[20\]J\. Ko, S\. Abdali, Y\. J\. Kim, T\. Chen, and P\. Cameron\(2026\)Scaling reasoning efficiently via relaxed on\-policy distillation\.arXiv preprint arXiv:2603\.11137\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p2.1)\.
- \[21\]T\. Ko, V\. Peddinti, D\. Povey, and S\. Khudanpur\(2015\)Audio augmentation for speech recognition\.\.InInterspeech,Vol\.2015,pp\. 3586\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px3.p1.3)\.
- \[22\]P\. Kumar and S\. Mishra\(2025\)Robustness in large language models: a survey of mitigation strategies and evaluation metrics\.arXiv preprint arXiv:2505\.18658\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1),[§1](https://arxiv.org/html/2605.23954#S1.p4.1)\.
- \[23\]S\. Kumar, Š\. Sedláček, V\. Lokegaonkar, F\. López, W\. Yu, N\. Anand, H\. Ryu, L\. Chen, M\. Plička, M\. Hlaváček,et al\.\(2026\)Mmau\-pro: a challenging and comprehensive benchmark for holistic evaluation of audio general intelligence\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 22688–22697\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p4.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p1.2)\.
- \[24\]B\. Li, W\. Huang, Y\. Qiu, Y\. Guo, H\. Wang, Z\. Li, J\. Peng, Z\. Ma, X\. Chen, and K\. Yu\(2025\)Isa\-bench: benchmarking instruction sensitivity for large audio language models\.arXiv preprint arXiv:2510\.23558\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1)\.
- \[25\]J\. Li, J\. Li, G\. Yang, L\. Yang, H\. Chi, and L\. Yang\(2025\)Applications of large language models and multimodal large models in autonomous driving: a comprehensive review\.Drones9\(4\),pp\. 238\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[26\]J\. Li, L\. Deng, R\. Haeb\-Umbach, and Y\. Gong\(2015\)Robust automatic speech recognition: a bridge to practical applications\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2605.23954#S1.p1.1),[§1](https://arxiv.org/html/2605.23954#S1.p2.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px3.p1.3)\.
- \[27\]L\. Lin, M\. Yu, K\. Luo, Y\. Zhang, L\. Peng, D\. Wang, X\. Tang, Y\. Zhang, X\. Yang, Z\. Zhou,et al\.\(2026\)Hidden in the noise: unveiling backdoors in audio llms alignment through latent acoustic pattern triggers\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 32015–32023\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[28\]K\. Luo, L\. Lin, Y\. Zhang, M\. Aloqaily, D\. Wang, Z\. Zhou, J\. Zhang, K\. Wang, L\. Sun, and Q\. Wen\(2026\)ChronosAudio: a comprehensive long\-audio benchmark for evaluating audio\-large language models\.arXiv preprint arXiv:2601\.04876\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[29\]Z\. Ma, Y\. Ma, Y\. Zhu, C\. Yang, Y\. Chao, R\. Xu, W\. Chen, Y\. Chen, Z\. Chen, J\. Cong,et al\.\(2025\)Mmar: a challenging benchmark for deep reasoning in speech, audio, music, and their mix\.arXiv preprint arXiv:2505\.13032\.Cited by:[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p1.2)\.
- \[30\]D\. S\. Park, W\. Chan, Y\. Zhang, C\. Chiu, B\. Zoph, E\. D\. Cubuk, and Q\. V\. Le\(2019\)Specaugment: a simple data augmentation method for automatic speech recognition\.arXiv preprint arXiv:1904\.08779\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1)\.
- \[31\]S\. Pascual, A\. Bonafonte, and J\. Serra\(2017\)SEGAN: speech enhancement generative adversarial network\.arXiv preprint arXiv:1703\.09452\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p2.1)\.
- \[32\]H\. Purwins, B\. Li, T\. Virtanen, J\. Schlüter, S\. Chang, and T\. Sainath\(2019\)Deep learning for audio signal processing\.IEEE Journal of Selected Topics in Signal Processing13\(2\),pp\. 206–219\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p2.1)\.
- \[33\]A\. Radford, J\. W\. Kim, T\. Xu, G\. Brockman, C\. McLeavey, and I\. Sutskever\(2023\)Robust speech recognition via large\-scale weak supervision\.InInternational conference on machine learning,pp\. 28492–28518\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[34\]S\. Ross, G\. Gordon, and D\. Bagnell\(2011\)A reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of the fourteenth international conference on artificial intelligence and statistics,pp\. 627–635\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p1.1)\.
- \[35\]P\. K\. Rubenstein, C\. Asawaroengchai, D\. D\. Nguyen, A\. Bapna, Z\. Borsos, F\. d\. C\. Quitry, P\. Chen, D\. E\. Badawy, W\. Han, E\. Kharitonov,et al\.\(2023\)Audiopalm: a large language model that can speak and listen\.arXiv preprint arXiv:2306\.12925\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[36\]H\. Sang, Y\. Xu, Z\. Zhou, R\. He, Z\. Wang, and J\. Sun\(2026\)On\-policy self\-distillation for reasoning compression\.arXiv e\-prints,pp\. arXiv–2603\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p3.1)\.
- \[37\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px1.p1.4)\.
- \[38\]Y\. Shen, K\. Song, X\. Tan, W\. Zhang, K\. Ren, S\. Yuan, W\. Lu, D\. Li, and Y\. Zhuang\(2024\)Taskbench: benchmarking large language models for task automation\.Advances in Neural Information Processing Systems37,pp\. 4540–4574\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[39\]I\. Shenfeld, M\. Damani, J\. Hübotter, and P\. Agrawal\(2026\)Self\-distillation enables continual learning\.arXiv preprint arXiv:2601\.19897\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px2.p1.2)\.
- \[40\]Y\. Song, L\. Chen, F\. Tajwar, R\. Munos, D\. Pathak, J\. A\. Bagnell, A\. Singh, and A\. Zanette\(2026\)Expanding the capabilities of reinforcement learning via text feedback\.arXiv preprint arXiv:2602\.02482\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px2.p1.2)\.
- \[41\]R\. Tan, R\. Li, Z\. Zhao, and H\. Zhang\(2026\)Beyond compromise: pareto\-lenient consensus for efficient multi\-preference llm alignment\.arXiv preprint arXiv:2604\.05965\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p3.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px2.p1.2)\.
- \[42\]C\. Tang, W\. Yu, G\. Sun, X\. Chen, T\. Tan, W\. Li, L\. Lu, Z\. Ma, and C\. Zhang\(2023\)Salmonn: towards generic hearing abilities for large language models\.arXiv preprint arXiv:2310\.13289\.Cited by:[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px3.p1.3)\.
- \[43\]B\. Wang, X\. Zou, G\. Lin, S\. Sun, Z\. Liu, W\. Zhang, Z\. Liu, A\. Aw, and N\. Chen\(2025\)Audiobench: a universal benchmark for audio large language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 4297–4316\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1)\.
- \[44\]D\. Wang and J\. Chen\(2018\)Supervised speech separation based on deep learning: an overview\.IEEE/ACM transactions on audio, speech, and language processing26\(10\),pp\. 1702–1726\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1)\.
- \[45\]S\. Watanabe, T\. Hori, S\. Karita, T\. Hayashi, J\. Nishitoba, Y\. Unno, N\. E\. Y\. Soplin, J\. Heymann, M\. Wiesner, N\. Chen,et al\.\(2018\)ESPnet: end\-to\-end speech processing toolkit\.arXiv preprint arXiv:1804\.00015\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1)\.
- \[46\]B\. Wu, C\. Yan, C\. Hu, C\. Yi, C\. Feng, F\. Tian, F\. Shen, G\. Yu, H\. Zhang, J\. Li,et al\.\(2025\)Step\-audio 2 technical report\.arXiv preprint arXiv:2507\.16632\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p4.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p1.2)\.
- \[47\]Y\. Xiao and R\. K\. Das\(2025\)XLSR\-mamba: a dual\-column bidirectional state space model for spoofing attack detection\.IEEE Signal Processing Letters\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[48\]Y\. Xiao, T\. Peng, Y\. Zhou, and R\. K\. Das\(2025\)AdaKWS: towards robust keyword spotting with test\-time adaptation\.InINTERSPEECH 2025,Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[49\]W\. Yang, W\. Liu, R\. Xie, K\. Yang, S\. Yang, and Y\. Lin\(2026\)Learning beyond teacher: generalized on\-policy distillation with reward extrapolation\.arXiv preprint arXiv:2602\.12125\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p2.1)\.
- \[50\]Y\. Yao, T\. Yu, A\. Zhang, C\. Wang, J\. Cui, H\. Zhu, T\. Cai, H\. Li, W\. Zhao, Z\. He,et al\.\(2024\)Minicpm\-v: a gpt\-4v level mllm on your phone\.arXiv preprint arXiv:2408\.01800\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p4.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p1.2)\.
- \[51\]T\. Ye, L\. Dong, X\. Wu, S\. Huang, and F\. Wei\(2026\)On\-policy context distillation for language models\.arXiv preprint arXiv:2602\.12275\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p3.1)\.
- \[52\]H\. Yin, Y\. Xiao, J\. Bai, and R\. K\. Das\(2024\)Leveraging llm and text\-queried separation for noise\-robust sound event detection\.InICASSP 2025\-2025 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\) Satellite Workshop on Speech and Audio Language Models \(SALMA\),Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[53\]H\. Yin, Y\. Xiao, Y\. Kwon, T\. Dang, and J\. Choi\(2026\)Focus then listen: exploring plug\-and\-play audio enhancer for noise\-robust large audio language models\.arXiv preprint arXiv:2603\.04862\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p1.1)\.
- \[54\]Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu,et al\.\(2025\)Dapo: an open\-source llm reinforcement learning system at scale\.arXiv preprint arXiv:2503\.14476\.Cited by:[§1](https://arxiv.org/html/2605.23954#S1.p3.1)\.
- \[55\]Y\. Zhang, J\. Tian, Y\. Zhang, S\. Yan, L\. Lin, Z\. Zhou, L\. Sun, and S\. Su\(2026\)SEE: signal embedding energy for quantifying noise interference in large audio language models\.arXiv preprint arXiv:2601\.07331\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2605.23954#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p1.2),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p2.1),[§4\.1](https://arxiv.org/html/2605.23954#S4.SS1.SSS0.Px1.p2.1.11)\.
- \[56\]S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. Grover\(2026\)Self\-distilled reasoner: on\-policy self\-distillation for large language models\.arXiv preprint arXiv:2601\.18734\.Cited by:[Appendix A](https://arxiv.org/html/2605.23954#A1.SS0.SSS0.Px1.p2.1),[§1](https://arxiv.org/html/2605.23954#S1.p3.1),[§2](https://arxiv.org/html/2605.23954#S2.SS0.SSS0.Px2.p1.3)\.

## Appendix ARelated Work

#### Knowledge Distillation and Frontier Self\-Distillation\.

Knowledge distillation minimizes the functional or distributional discrepancy between the student model and the teacher model through soft labels, response distributions, or intermediate representations, thereby transferring the inter\-class similarity and decision boundaries encoded by the teacher\[[14](https://arxiv.org/html/2605.23954#bib.bib18)\]\. Its limitation is that it usually relies on an external strong teacher; in generative tasks, the student is also prone to distribution shift, exposure bias, and teacher bias, and it is difficult to fully exploit privileged information such as ground\-truth labels, reference answers, or chain\-of\-thought annotations\[[14](https://arxiv.org/html/2605.23954#bib.bib18),[34](https://arxiv.org/html/2605.23954#bib.bib19)\]\.

To alleviate the dependence on external teachers, research has increasingly shifted toward self\-distillation, which constructs the teacher and the student within the same model family\[[1](https://arxiv.org/html/2605.23954#bib.bib17),[56](https://arxiv.org/html/2605.23954#bib.bib15)\]\. Self\-distillation can be viewed as imposing consistency regularization on the model output distribution and internal representations; therefore, while reducing training cost, it also helps improve representation stability and generalization\[[49](https://arxiv.org/html/2605.23954#bib.bib8),[20](https://arxiv.org/html/2605.23954#bib.bib22)\]\. Recent frontier work further pushes this direction into generative large\-model post\-training: SDPO converts rich textual feedback into dense self\-distillation signals\[[19](https://arxiv.org/html/2605.23954#bib.bib16)\]; SDFT uses a demonstration\-conditioned self\-teacher for on\-policy learning\[[39](https://arxiv.org/html/2605.23954#bib.bib21)\]; Self\-Distilled Reasoner performs token\-level on\-policy self\-distillation over student rollouts\[[56](https://arxiv.org/html/2605.23954#bib.bib15)\]; and RL with text feedback studies textual feedback as an intermediate supervision signal between scalar rewards and full demonstrations\[[40](https://arxiv.org/html/2605.23954#bib.bib20)\]\.

#### Noise Robustness and Audio Denoising\.

Robustness studies of traditional speech models mainly rely on multi\-condition training, data augmentation\[[21](https://arxiv.org/html/2605.23954#bib.bib2),[30](https://arxiv.org/html/2605.23954#bib.bib3)\], and enhancement front\-ends\[[44](https://arxiv.org/html/2605.23954#bib.bib4)\]to alleviate acoustic mismatch\[[26](https://arxiv.org/html/2605.23954#bib.bib1)\], and their goals are usually to recover acoustic fidelity or reduce word error rate\[[45](https://arxiv.org/html/2605.23954#bib.bib10)\]\. Typical methods include spectral subtraction, wavelet denoising, and the model\-based optimization method SEE\[[55](https://arxiv.org/html/2605.23954#bib.bib23)\]\. For audio large language models, however, the key issue is the stability\[[24](https://arxiv.org/html/2605.23954#bib.bib11),[15](https://arxiv.org/html/2605.23954#bib.bib12)\]of high\-level semantic reasoning under noise, because strong noise can easily induce semantic drift or even "semantic hallucination"\[[15](https://arxiv.org/html/2605.23954#bib.bib12)\], and relying only on waveform\-level enhancement is often insufficient\[[12](https://arxiv.org/html/2605.23954#bib.bib14),[43](https://arxiv.org/html/2605.23954#bib.bib13)\]\. In this work, we adopt a*clean\-teacher / noisy\-student*self\-distillation framework, together with preference\-style policy optimization and distribution alignment\[[19](https://arxiv.org/html/2605.23954#bib.bib16)\], to learn a noisy\-to\-clean robust mapping within the model, thereby improving response consistency, semantic reliability, and task completion capability under complex noisy conditions\.

## Appendix BAdditional Experimental Results

Table[4](https://arxiv.org/html/2605.23954#A2.T4)reports the overall robustness comparison between EchoDistill and representative denoising\-based or robustness\-oriented baselines\. The results show that EchoDistill achieves the best average performance across all three evaluation metrics, indicating that noisy\-to\-clean semantic alignment provides more stable robustness gains than directly applying signal\-level enhancement or feature\-level denoising strategies\.

Table 4:Overall robustness summary across representative baselines\.Compared with the strongest baseline, EchoDistill improves Acc, Noisy, and GSR by 1\.63, 1\.80, and 4\.18 points, respectively\. The larger improvement on GSR suggests that EchoDistill is especially effective at preserving semantically grounded responses under acoustic corruption, rather than only improving surface\-level prediction accuracy\.

Table 5:Appendix results of Qwen2\.5\-Omni under−10\-10dB noise\. Average F1 is computed as\(F1​\-​micro\+F1​\-​macro\)/2\(\\mathrm\{F1\\text\{\-\}micro\}\+\\mathrm\{F1\\text\{\-\}macro\}\)/2, and Net correction rate is computed as\(Corrected−Broken\)/1200×100\(\\mathrm\{Corrected\}\-\\mathrm\{Broken\}\)/1200\\times 100\.Δ\\DeltaF1 andΔ\\DeltaNet denote the improvement of EchoDistill over each corresponding baseline on the same task\.Table[5](https://arxiv.org/html/2605.23954#A2.T5)reports the Qwen2\.5\-Omni results under severe acoustic noise\. EchoDistill achieves the highest net correction rate on all three tasks, indicating that it more effectively turns noisy\-input errors into correct predictions while reducing harmful answer flips\. The advantage is most pronounced on Sound and Speech, where EchoDistill also obtains the best Average F1\. On Music, STFT gives the highest Average F1, but EchoDistill still produces the best net correction rate, suggesting a stronger robustness–stability trade\-off\.

## Appendix CDetails of Experimental Setup

This section details the evaluation protocol used in our experiments\. We report three metrics, namelyAcc,Noisy, andGSR, to evaluate the robustness of ALLMs under severe acoustic noise\. Among them,GSRis used as the primary metric because it directly measures whether the model can maintain stable and successful generation when noisy inputs are introduced\.AccandNoisyserve as auxiliary metrics, reflecting task correctness and noisy\-to\-clean behavioral recovery, respectively\.

#### Metric overview\.

The three metrics are summarized as follows:

- •Acc: task accuracy under noisy audio inputs\.
- •Noisy: consistency between noisy\-input predictions and clean\-audio references\.
- •GSR: generation success rate between noisy and clean inputs under the same method\.

#### Unified exact\-match metric\.

Lety^i\(a\)\\hat\{y\}\_\{i\}^\{\(a\)\}andy^i\(b\)\\hat\{y\}\_\{i\}^\{\(b\)\}denote two predictions to be compared for sampleii\. Let𝒱\\mathcal\{V\}be the valid sample set after excluding missing or invalid predictions\. We define the unified exact\-match score as

EM​\(y^\(a\),y^\(b\);𝒱\)=1\|𝒱\|​∑i∈𝒱𝟏​\[y^i\(a\)=y^i\(b\)\],\\mathrm\{EM\}\\\!\\left\(\\hat\{y\}^\{\(a\)\},\\hat\{y\}^\{\(b\)\};\\mathcal\{V\}\\right\)=\\frac\{1\}\{\|\\mathcal\{V\}\|\}\\sum\_\{i\\in\\mathcal\{V\}\}\\mathbf\{1\}\\\!\\left\[\\hat\{y\}\_\{i\}^\{\(a\)\}=\\hat\{y\}\_\{i\}^\{\(b\)\}\\right\],\(12\)where\|𝒱\|\|\\mathcal\{V\}\|denotes the number of valid samples,this will to some extent be a source of data error, and𝟏​\[⋅\]\\mathbf\{1\}\[\\cdot\]is the indicator function\.

#### Metric instantiation\.

Based on Eq\.[12](https://arxiv.org/html/2605.23954#A3.E12), the three metrics are instantiated as:

Accd\\displaystyle\\mathrm\{Acc\}\_\{d\}=EM​\(y^d,n,y⋆;𝒱acc\),\\displaystyle=\\mathrm\{EM\}\\\!\\left\(\\hat\{y\}^\{d,n\},y^\{\\star\};\\mathcal\{V\}\_\{\\mathrm\{acc\}\}\\right\),\(13\)Noisyd\\displaystyle\\mathrm\{Noisy\}\_\{d\}=EM​\(y^d,n,y^c;𝒱noisy\),\\displaystyle=\\mathrm\{EM\}\\\!\\left\(\\hat\{y\}^\{d,n\},\\hat\{y\}^\{c\};\\mathcal\{V\}\_\{\\mathrm\{noisy\}\}\\right\),\(14\)GSRd\\displaystyle\\mathrm\{GSR\}\_\{d\}=EM​\(y^d,n,y^d,c;𝒱gsr\)\.\\displaystyle=\\mathrm\{EM\}\\\!\\left\(\\hat\{y\}^\{d,n\},\\hat\{y\}^\{d,c\};\\mathcal\{V\}\_\{\\mathrm\{gsr\}\}\\right\)\.\(15\)Here,dddenotes the evaluated denoising or robustness method\.y^id,n\\hat\{y\}\_\{i\}^\{d,n\}denotes the prediction produced by methodddon noisy audio inputs, andy^id,c\\hat\{y\}\_\{i\}^\{d,c\}denotes the prediction produced by the same method on clean audio inputs\.y^ic\\hat\{y\}\_\{i\}^\{c\}is the clean\-input prediction obtained without applying any denoising method, whileyi⋆y\_\{i\}^\{\\star\}denotes the ground\-truth answer\. The sets𝒱acc\\mathcal\{V\}\_\{\\mathrm\{acc\}\},𝒱noisy\\mathcal\{V\}\_\{\\mathrm\{noisy\}\}, and𝒱gsr\\mathcal\{V\}\_\{\\mathrm\{gsr\}\}are the valid sample subsets used for computing the corresponding metrics\.

#### Evaluation focus\.

SinceGSRcompares the noisy\-input output with the clean\-input output under the same method, it reflects whether the method preserves generation behavior after severe acoustic corruption\. Therefore, a higherGSRindicates stronger generation stability\. In contrast,Accemphasizes correctness against the ground\-truth label, whileNoisymeasures recovery toward the clean\-audio reference\. Together, these metrics provide a complementary evaluation of task accuracy, semantic recovery, and generation robustness\.

### C\.1Dataset Instance

Each training instance contains a noisy\-clean audio pair, a task prompt, candidate choices, the target answer, and noise metadata\. The noisy audio is used as the student input, while the clean audio is used as the teacher input\. Candidate responses, teacher responses, and rewards are generated or computed online during training rather than stored as static annotations\.

Table 6:An example of the training data format used byEchoDistill\. The stored fields include the noisy\-clean audio pair, task prompt, candidate choices, target answer, and noise metadata\.