The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works

arXiv cs.LG 05/27/26, 04:00 AM Papers
Summary
This paper introduces the Bridge-Garden Decomposition theory to explain why mixing hard and soft labels in LLM distillation reduces exposure bias, and develops hybrid supervision methods that outperform existing baselines while reducing training cost by 9.7×.
arXiv:2605.26246v1 Announce Type: new Abstract: Knowledge distillation (KD) transfers knowledge from a large teacher model to a smaller student. In language modeling, the student is trained either on tokens sampled from the teacher (hard labels) or the teacher's full next-token distribution (soft labels). Despite soft labels appear strictly richer, we find that mixing hard and soft labels consistently yields better results. Crucially, we show that this gain cannot be explained by closer teacher matching during training. Instead, it comes from reduced exposure bias, the mismatch between training and inference distributions. To explain this phenomenon, we introduce the Bridge-Garden Decomposition theory, which categorizes generation steps into two types: Bridges, where the next token must be exact, and Gardens, where it can be flexible. We show that hard-only KD excels in Bridges by avoiding risky deviations, while soft-only KD preserves diversity in Gardens. A hybrid strategy handles both cases and, as a result, reduces exposure bias across the sequence. Guided by this theory, we develop a family of Bridge-Garden hybrid supervision methods that adaptively balance hard and soft labels. Across a primary suite of seven teacher-student pairs (including Qwen, Llama, Gemma, and DeepSeek) and benchmarks in reasoning and coding, our approach outperforms divergence-based and on-policy KD baselines while reducing training cost by 9.7x, enabling efficient model compression. Code is available at https://github.com/ghwang-s/bridge_garden_hybrid_kd_release.
Original Article
View Cached Full Text
Cached at: 05/27/26, 09:06 AM
# The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works
Source: [https://arxiv.org/html/2605.26246](https://arxiv.org/html/2605.26246)
Kaiwen Lv KacuilaZhiyong YangZitai WangJin\-Wen WuLongtao HuangQianqian XuQingming Huang

###### Abstract

Knowledge distillation \(KD\) transfers knowledge from a large teacher model to a smaller student\. In language modeling, the student is trained either on tokens sampled from the teacher \(hard labels\) or the teacher’s full next\-token distribution \(soft labels\)\. Despite soft labels appear strictly richer, we find that mixing hard and soft labels consistently yields better results\. Crucially, we show that this gain cannot be explained by closer teacher matching during training\. Instead, it comes from reduced exposure bias—the mismatch between training and inference distributions\. To explain this phenomenon, we introduce the Bridge–Garden Decomposition theory, which categorizes generation steps into two types:Bridges, where the next token must beexact, andGardens, where it can beflexible\. We show that hard\-only KD excels in Bridges by avoiding risky deviations, while soft\-only KD preserves diversity in Gardens\. A hybrid strategy handles both cases and, as a result, reduces exposure bias across the sequence\. Guided by this theory, we develop a family of Bridge–Garden hybrid supervision methods that adaptively balance hard and soft labels\. Across a primary suite of seven teacher–student pairs \(including Qwen, Llama, Gemma, and DeepSeek\) and benchmarks in reasoning and coding, our approach outperforms divergence\-based and on\-policy KD baselines while reducing training cost by9\.7×\\times, enabling efficient model compression\. Code is available at[https://github\.com/ghwang\-s/bridge\_garden\_hybrid\_kd\_release](https://github.com/ghwang-s/bridge_garden_hybrid_kd_release)\.

Knowledge Distillation, Exposure Bias, Language Models

### 1Introduction

Recent progress in large language models \(LLMs,Achiamet al\.[2023](https://arxiv.org/html/2605.26246#bib.bib68); Grattafiori and others[2024](https://arxiv.org/html/2605.26246#bib.bib74); Guoet al\.[2025](https://arxiv.org/html/2605.26246#bib.bib67); Yanget al\.[2025](https://arxiv.org/html/2605.26246#bib.bib69)\) has largely been driven by scaling up model sizes\(Hoffmannet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib60)\), yet this leads to high inference costs for deployment\. Knowledge distillation \(KD,Hintonet al\.[2015](https://arxiv.org/html/2605.26246#bib.bib54)\) offers a practical way to alleviate this cost by transferring capabilities from apowerfulteacher model to acompactstudent\. The central question is how to design a distillation objective that makes the student match the teacher’s generative behavior as closely as possible\.

Conventional wisdom favors soft\-label distillation\(Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55); Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19); Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73); Xuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib57); Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59); Koet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib72)\), where the student is trained to match the teacher’s full predictive distribution over the next token\. This approach is intuitively appealing because it captures richer information, including the teacher’s confidence over alternative tokens\. In contrast, hard\-label distillation\(Kim and Rush,[2016](https://arxiv.org/html/2605.26246#bib.bib20); Wanget al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib65); Taoriet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib64); Penget al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib63); Guoet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib67)\)relies on a single token sampled from the teacher’s distribution as the training target, which loses much of the distributional information present in the soft labels\.

![Refer to caption](https://arxiv.org/html/2605.26246v1/figs/fig1.png)Figure 1:The Bridge–Garden Dilemma\. Bridges require exact tokens to prevent error cascades, favoring Hard KD for risk suppression\. Gardens allow flexible choices, favoring Soft KD for preserving diversity\. Hybrid KD balances both for superior generative performance\.![Refer to caption](https://arxiv.org/html/2605.26246v1/x1.png)Figure 2:Comparative analysis \(Qwen2\.5\-7B→\\rightarrow3B\) of Hybrid KD \(λℓsoft\+\(1−λ\)ℓhard\\lambda\\ell\_\{\\mathrm\{soft\}\}\+\(1\-\\lambda\)\\ell\_\{\\mathrm\{hard\}\}\) vs\. Soft KD \(ℓsoft\\ell\_\{\\mathrm\{soft\}\}\)\. \(a\) benchmark performance gains, \(b\) student\-teacher imitation error during training \(quantified by Forward KL\), and \(c\) inference imitation error decomposition based on the same metric\. The experiments provided inApp\.[F](https://arxiv.org/html/2605.26246#A6)further demonstrate the consistencyof hard\-soft paradoxacross various architectures, tasks, and distillation divergences\.Surprisingly, our empirical investigation reveals a consistent counter\-intuitive trend: a simple linear combination of hard and soft losses systematically outperforms pure soft distillation \(Fig\.[2](https://arxiv.org/html/2605.26246#S1.F2)a\)\. This presents a puzzle:If hard labels provide less information, why do they help?At first glance, it seems that using hard labels can make optimization much easier, enabling the student to imitate the teacher more closely during training\. However, we discover that the performance gain does not come from better training imitation\. In fact, our experiments show that adding hard labels even worsens this training fit, as shown in Fig\.[2](https://arxiv.org/html/2605.26246#S1.F2)\(b\)\.

Instead, we show that the gain arises from a different source: reduced exposure bias\. Exposure bias\(Bengioet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib62)\)refers to the performance gap that emerges when the student, conditioned on its own previously generated tokens, deviates from the teacher’s generation trajectory\. This autoregressive distribution shift is a core challenge in sequence\-level KD\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18); Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19); Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73)\)\. Our analysis shows that adding hard labels is particularly effective at suppressing this bias, leading to better final performance despite the weaker training fit \(Fig\.[2](https://arxiv.org/html/2605.26246#S1.F2)c\)\.

To explain this phenomenon, we introduce a novel conceptual and theoretical framework based on a core measure of local risk sensitivity\. Within this framework, we find that the autoregressive generation process can be partitioned into two distinct regions: Bridges and Gardens \(see Fig\.[1](https://arxiv.org/html/2605.26246#S1.F1)\)\.Bridge regionsexhibithighlocal risk sensitivity; here, the next token must beexact, as a single error can propagate and ruin the entire sequence\. Conversely,Garden regionsexhibitlowlocal risk sensitivity, where token choices are more flexible and multiple alternatives can preserve both meaning and coherence\. On top of this, we derive an upper bound for exposure bias that decomposes into contributions from these regions, which reveals aBridge–Garden Dilemma\. Our theory shows that hard\-label matching excels in Bridges by concentrating probability mass on the teacher’s chosen token, thereby averting cascading errors\. In contrast, soft\-label matching excels in Gardens by faithfully preserving the teacher’s full distribution, maintaining output diversity\. In this way, neither of the two pure strategies is optimal for both regions simultaneously\. This insight naturally leads to a hybrid distillation objective that selectively blends hard and soft supervision based on the prefix context\.

Based on this analysis, we propose a family of Bridge–Garden hybrid supervision methods that adaptively mix hard and soft labels, using confidence\-, entropy\-, curriculum\-, and risk\-guided strategies\. We study aprimary suite of seventeacher–student pairs across multiple families and scales, includingQwen \(7B→\\rightarrow3B\),Llama \(8B→\\rightarrow1B\),Gemma \(4B→\\rightarrow1B\), andDeepSeek\-Coder \(6\.7B→\\rightarrow1\.3B\)\. Extended evaluations further cover a larger Qwen2\.5 capacity gap, an additional Qwen2\.5\-Coder pair, and open\-ended generation\. Across commonsense, math, and coding benchmarks, the proposed methods outperform divergence\-based and on\-policy baselines, while reducing training cost by9\.7×\\times, making our approach more practical for industrial applications\.

### 2Preliminaries

Autoregressive Generation\.We consider sequence generation over a vocabulary𝒱\\mathcal\{V\}\. At each steptt, the language model predicts the next tokenyt∈𝒱y\_\{t\}\\in\\mathcal\{V\}given an input promptxxand the preceding sequencey<t≔\(y1,…,yt−1\)y\_\{<t\}\\coloneqq\(y\_\{1\},\\dots,y\_\{t\-1\}\)\. We denote theprefixat stepttbys≔\(x,y<t\)s\\coloneqq\(x,y\_\{<t\}\)and thenext tokenbya≔yta\\coloneqq y\_\{t\}\. The model’s behavior is defined by apolicyπ\(a∣s\)\\pi\(a\\mid s\), a con ditional probability distribution over𝒱\\mathcal\{V\}\.

Knowledge Distillation \(KD\)seeks to align aparameterizedstudentπθ\\pi\_\{\\theta\}with afixedteacherπT\\pi\_\{T\}by minimizing the discrepancy in their predictions\. Formally, the objective is to minimize the expected divergence𝔻\\mathbb\{D\}\(such as KL divergence; see App\.[A\.1](https://arxiv.org/html/2605.26246#A1.SS1)\) between their next\-token distributions under the teacher\-generatedprefix distributiondTd\_\{T\}:

ℒdT\(πθ\)≔𝔼s∼dT\[𝔻\(πT\(⋅∣s\)∥πθ\(⋅∣s\)\)\]\.\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)\\coloneqq\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[\\mathbb\{D\}\\big\(\\pi\_\{T\}\(\\cdot\\mid s\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\big\)\\big\]\.\(1\)
This objective is empirically optimized via two approaches:

Soft KD\.Here the student directly matches the teacher’s full output distribution for every prefixssby minimizing

ℓsoft\(s;θ\)≔𝔻\(πT\(⋅∣s\)∥πθ\(⋅∣s\)\)\.\\ell\_\{\\mathrm\{soft\}\}\(s;\\theta\)\\coloneqq\\mathbb\{D\}\\big\(\\pi\_\{T\}\(\\cdot\\mid s\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\big\)\.\(2\)
Hard KD\.When the full distributionπT\(⋅∣s\)\\pi\_\{T\}\(\\cdot\\mid s\)is unavailable \(e\.g\.,with black\-box teacher APIs\(Achiamet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib68)\)\) or the divergence𝔻\\mathbb\{D\}is costly to compute\(Carliniet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib71)\), one can instead sample a tokena∗∼πT\(⋅∣s\)a^\{\*\}\\sim\\pi\_\{T\}\(\\cdot\\mid s\)and train the student to maximize its log\-likelihood:

ℓhard\(s;θ\)≔−log⁡πθ\(a∗∣s\)\.\\ell\_\{\\mathrm\{hard\}\}\(s;\\theta\)\\coloneqq\-\\log\\pi\_\{\\theta\}\(a^\{\*\}\\mid s\)\.\(3\)This objective provides an unbiased estimate of𝔼a∼πT\(⋅\|s\)\[−log⁡πθ\(a\|s\)\]\\mathbb\{E\}\_\{a\\sim\\pi\_\{T\}\(\\cdot\|s\)\}\[\-\\log\\pi\_\{\\theta\}\(a\|s\)\]\. When𝔻\\mathbb\{D\}is the forward KL divergence, minimizing Eq\.\([3](https://arxiv.org/html/2605.26246#S2.E3)\) is equivalent to minimizingℒdT\(πθ\)\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\), up to a constant teacher entropy\. This equivalence concerns the population objective; with finite data and limited student capacity, hard and soft supervision can still lead to different training trajectories and student distributions\.See App\.[A](https://arxiv.org/html/2605.26246#A1)for further related work and comparisons with prior arts\.

### 3The Hard\-Label Paradox in Distillation

#### 3\.1An Empirical Training–Inference Puzzle

Conventional wisdom\(Hintonet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib54); Cho and Hariharan,[2019](https://arxiv.org/html/2605.26246#bib.bib16); Zhaoet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib15)\)holds that soft labels \(Eq\.[2](https://arxiv.org/html/2605.26246#S2.E2)\), which encapsulate the teacher’s full predictive distribution, should offer richer guidance than hard one‑hot labels \(Eq\.[3](https://arxiv.org/html/2605.26246#S2.E3)\)\. Intuitively, matching the teacher’s probabilities across all tokens ought to produce a more accurate student\.

Our experiments, however, reveal the opposite trend\. As shown in Fig\.[2](https://arxiv.org/html/2605.26246#S1.F2)\(a\), a simple linear hybrid loss, termed Hybrid KD, consistently surpasses pure soft distillation across architectures, tasks, and divergence measures\.

Observation 1\.Alinear interpolation\(λℓsoft\+\(1−λ\)ℓhard\\lambda\\ell\_\{\\mathrm\{soft\}\}\+\(1\-\\lambda\)\\ell\_\{\\mathrm\{hard\}\}\)significantly outperforms pure Soft KDacross models, benchmarks, and divergence choices \(p<<0\.05\)\.This consistent advantage is further shown in App\.[F](https://arxiv.org/html/2605.26246#A6)\.

Despite these robust gains, the underlying mechanism remains unclear\. Without a principled explanation, the improvement appears incidental, hindering the systematic design of distillation objectives\.

One natural hypothesis points to optimization: hard labels may yield sharper gradients and ease optimization\. If this were the primary cause, we would expect hybrid training to help the student better imitate the teacher than pure soft distillation\. However, Fig\.[2](https://arxiv.org/html/2605.26246#S1.F2)\(b\) shows the opposite:

Observation 2\.Adding hard labels*worsens*the training fit: the studentmimics the teacher worseand yieldsa larger distribution discrepancy𝔻\(πT∥πθ\)\\mathbb\{D\}\(\\pi\_\{T\}\\,\\\|\\,\\pi\_\{\\theta\}\)than Soft KD\.This trend is further observed in App\.[F](https://arxiv.org/html/2605.26246#A6)\.

Since optimization difficulty cannot explain the gains, we must look elsewhere\. This motivates a deeper problem:

“If the performance gain is not from betterteacher\-imitation during training, where does it come from?”

#### 3\.2Locating the Missing Gain

To solve this, we first need to examine the total inference error of the student more carefully\. During training, the student learns from teacher\-generated prefixes, but at inference time it must generate tokens conditioned on its own past outputs, a fundamentally different distributiondθd\_\{\\theta\}\.

On top of this, we can decompose the student’s total inference error into two interpretable parts\. LetℒdT\(πθ\)\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)denote the training\-fit error \(how well the student imitates the teacher on teacher prefixes\), and letℒdθ\(πθ\)\\mathcal\{L\}\_\{d\_\{\\theta\}\}\(\\pi\_\{\\theta\}\)be the inference imitation error \(how well it performs on its own generated sequences\)\. A simple identity relates them:

ℒdθ\(πθ\)=ℒdT\(πθ\)\+\(ℒdθ\(πθ\)−ℒdT\(πθ\)\)⏟Residual\.\\mathcal\{L\}\_\{d\_\{\\theta\}\}\(\\pi\_\{\\theta\}\)\\;=\\;\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)\\;\+\\;\\underbrace\{\\big\(\\mathcal\{L\}\_\{d\_\{\\theta\}\}\(\\pi\_\{\\theta\}\)\-\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)\\big\)\}\_\{\\text\{Residual\}\}\.\(4\)
The first term is the familiar training\-fit error\. The second term captures the extra loss incurred when the student deviates from the teacher’s prefix distribution\. Notably, this residual has a precise and well\-known meaning in sequence generation: it coincides with the exposure\-bias term in KD

Definition 3\.1\(Distillation Exposure Bias,Bengioet al\.[2015](https://arxiv.org/html/2605.26246#bib.bib62)\)In autoregressive KD, exposure bias is defined as𝖤𝖡\(πθ\)≔𝔼s∼dθ\[ℓsoft\(s\)\]⏟Inference Errorℒdθ\(πθ\)−𝔼s∼dT\[ℓsoft\(s\)\]⏟Training FitℒdT\(πθ\)\.\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\\coloneqq\\underbrace\{\\mathbb\{E\}\_\{s\\sim d\_\{\\theta\}\}\\\!\\big\[\\ell\_\{\\mathrm\{soft\}\}\(s\)\\big\]\}\_\{\\text\{Inference Error \}\\mathcal\{L\}\_\{d\_\{\\mathrm\{\\theta\}\}\}\(\\pi\_\{\\theta\}\)\}\-\\underbrace\{\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\big\[\\ell\_\{\\mathrm\{soft\}\}\(s\)\\big\]\}\_\{\\text\{Training Fit \}\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)\}\.It quantifies the performance gap caused by the shift from teacher prefixesdTd\_\{T\}to student\-generated prefixesdθd\_\{\\theta\}\.

Remark\.Unlike the classicali\.i\.d\.generalization gap\(Mohriet al\.,[2018](https://arxiv.org/html/2605.26246#bib.bib1)\), exposure bias stems specifically from the*autoregressive*distribution shiftdT→dθd\_\{T\}\\\!\\to d\_\{\\theta\}\.

Now our earlier observations fall into place\. Hard labels improve final performance \(Obs\.[3\.1](https://arxiv.org/html/2605.26246#S3.SS1)\) despite they hurt training fit \(Obs\.[3\.1](https://arxiv.org/html/2605.26246#S3.SS1)\)\. Eq\.\([4](https://arxiv.org/html/2605.26246#S3.E4)\) shows the only way this can happen: hard labels must reduce exposure bias enough to outweigh their worse training fit\. Fig\.[2](https://arxiv.org/html/2605.26246#S1.F2)\(c\) and results for more models and tasks in App\.[F](https://arxiv.org/html/2605.26246#A6)confirms this directly\. The drop in exposure bias indeed compensates for the rise in training error\.

We now know that hard labels suppress exposure bias\. But why does this happen? We turn next to uncover the mechanism hidden behind\.

### 4The Bridge–Garden Decomposition of Exposure Bias

To answer this question, we revisit the core of exposure bias: it emerges when the student chooses a different token than the teacher, and such mistakes can accumulate over subsequent generation steps\. Understanding why this bias is reduced requires us to identify where these deviations occur and how they influence future tokens\.

#### 4\.1The Bridge–Garden Decomposition

Our analysis starts with a simple insight: not every generation step is equally sensitive to a local deviation\. In some contexts, the next token must be exact\. For example, changing an operator in a mathematical derivation \(e\.g\.,“\+\+” to “−\-”\) can break the entire reasoning chain\. In others, the next token can be flexible, such as replacing “excellent” with “great” in open‑ended dialogue often preserves the meaning\.

Motivated by this, we conceptually partition the generation process into two types of regions:

Bridges:These are regions wherethe next token must be exact\. A mistake here often invalidates the subsequent generation\.Gardens:These are regions wherethe next token can be flexible\. A small deviation usually does not affect the subsequent generation\.

Although intuitive, quantifying this distinction is nontrivial\. This is because, under the student’s own autoregressive generation, a single token deviation can alter all following predictions and trigger error accumulation, making it difficult to isolate token‑wise effects from the overall loss\.

#### 4\.2Quantifying Local Risk Sensitivity via Single‑Override Policy

Fortunately, we can derive a structured upper bound on exposure bias \(Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\), which decomposes the global bias into a sum of local, token‑level terms\. Each term captures the effect of the student deviating from the teacher at a single generation step, providing a natural measure of local risk\.

Theorem 4\.1\(Aκ\\kappa\-Weighted Bound on Exposure Bias\)Under mild conditions \(bounded loss, non\-vanishing probabilities, and prefix concentrability\), letΔπθ≔πθ−πT\\Delta\\pi\_\{\\theta\}\\coloneqq\\pi\_\{\\theta\}\-\\pi\_\{T\}\. Then the exposure bias𝖤𝖡\(πθ\)\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)satisfies𝖤𝖡\(πθ\)≤𝔼s∼dT\[F\(s,πθ\)\],\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\\ \\leq\\ \\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\Big\[\\,F\(s,\\pi\_\{\\theta\}\)\\Big\],where for a constantC2\>0C\_\{\\rm 2\}\>0,F\(s,πθ\):=∑aκ\(a∣s\)⋅\|Δπθ\(a∣s\)\|⏟κ\(a\|s\)\-weighted deviation\+C2∥Δπθ\(⋅∣s\)∥12\.F\(s,\\pi\_\{\\theta\}\):=\\underbrace\{\\sum\\nolimits\_\{a\}\{\\color\[rgb\]\{0,0,1\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,1\}\\kappa\(a\\\!\\mid\\\!s\)\}\\cdot\|\\Delta\\pi\_\{\\theta\}\(a\\\!\\mid\\\!s\)\|\}\_\{\\textbf\{$\\kappa\(a\|s\)$\-weighted deviation\}\}\\;\+C\_\{\\rm 2\}\\\|\\Delta\\pi\_\{\\theta\}\(\\cdot\\\!\\mid\\\!s\)\\\|\_\{1\}^\{2\}\.

The proof of Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)is in App\.[B](https://arxiv.org/html/2605.26246#A2)\. To interpretκ\(a∣s\)\\kappa\(a\\mid s\), we first introduce the notion of a single‑override policy\.

\(Single\-Override Policy\)Fix a prefixs∈𝒮s\\in\\mathcal\{S\}and tokena∈𝒱a\\in\\mathcal\{V\}\. Define the override policyπ\(s,a\)\\pi^\{\(s,a\)\}byπ\(s,a\)\(⋅∣s′\):=\{δa\(⋅\),s′=s,πT\(⋅∣s′\),s′≠s,\\pi^\{\(s,a\)\}\(\\cdot\\mid s^\{\\prime\}\)\\;:=\\;\\begin\{cases\}\\delta\_\{a\}\(\\cdot\),&s^\{\\prime\}=s,\\\\\[2\.0pt\] \\pi\_\{T\}\(\\cdot\\mid s^\{\\prime\}\),&s^\{\\prime\}\\neq s,\\end\{cases\}withδa\\delta\_\{a\}the Dirac distribution onaa\. Letd\(s,a\)≔dπ\(s,a\)d^\{\(s,a\)\}\\coloneqq d\_\{\\pi^\{\(s,a\)\}\}be the prefix distribution induced byπ\(s,a\)\\pi^\{\(s,a\)\}\.

Definition 4\.1\(Local\-Risk Sensitivity\)For a prefixsswith its visiting probabilitydT\(s\)\>0d\_\{T\}\(s\)\>0, the sensitivityκ\(a\|s\)\\kappa\(a\|s\)measures the extra loss incurred per visit tosswhen choosingaainstead of following the teacher:κ\(a\|s\):=𝖤𝖡\(π\(s,a\)\)dT\(s\)=ℒd\(s,a\)\(πθ\)−ℒdT\(πθ\)dT\(s\),\\kappa\(a\|s\):=\\frac\{\\mathsf\{EB\}\(\\pi^\{\(s,a\)\}\)\}\{d\_\{T\}\(s\)\}=\\frac\{\\mathcal\{L\}\_\{d^\{\(s,a\)\}\}\(\\pi\_\{\\theta\}\)\-\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)\}\{d\_\{T\}\(s\)\},whereℒd\(πθ\)≔𝔼s′∼d\[𝔻\(πT\(⋅∣s′\)∥πθ\(⋅∣s′\)\)\]\\mathcal\{L\}\_\{d\}\(\\pi\_\{\\theta\}\)\\coloneqq\\mathbb\{E\}\_\{s^\{\\prime\}\\sim d\}\[\\mathbb\{D\}\(\\pi\_\{T\}\(\\cdot\\mid s^\{\\prime\}\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid s^\{\\prime\}\)\)\]\.

In the bound above, the contribution of a local deviation at\(s,a\)\(s,a\)is the productκ\(a\|s\)\|Δπθ\(a\|s\)\|\\kappa\(a\|s\)\\,\|\\Delta\\pi\_\{\\theta\}\(a\|s\)\|\. Here\|Δπθ\(a\|s\)\|\|\\Delta\\pi\_\{\\theta\}\(a\|s\)\|measures the probability difference between student and teacher for tokenaagivenss, andκ\(a\|s\)\\kappa\(a\|s\)weights that difference in the upper bound\.

The override policyπ\(s,a\)\\pi^\{\(s,a\)\}acts as a prefix‑conditioned token intervention for the teacher: at exactly prefixssit outputs tokenaawith probability one, while following the teacher everywhere else\. The differenceℒd\(s,a\)\(πθ\)−ℒdT\(πθ\)\\mathcal\{L\}\_\{d^\{\(s,a\)\}\}\(\\pi\_\{\\theta\}\)\-\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)is precisely the loss increase caused by changing the current token toaaatss\. The quantityκ\(a\|s\)\\kappa\(a\|s\)discounts this increase by the visiting probabilitydT\(s\)d\_\{T\}\(s\)\. Thus,a smallκ\(a\|s\)\\kappa\(a\|s\)means that deviating toaaatsshas minimal impact on exposure bias; a largeκ\(a\|s\)\\kappa\(a\|s\)signals that this single‑token change can strongly affect the bias\.

Interestingly, this construction aligns with classical algorithmic stability theory\(Bousquet and Elisseeff,[2002](https://arxiv.org/html/2605.26246#bib.bib6)\), which measures how the loss changes when a single training example is perturbed\. Here the “example” is the prefixss, and the perturbation is forcing tokenaawheneverssis reached during generation\. The valueκ\(a\|s\)\\kappa\(a\|s\)quantifies the sensitivity of the loss to this token‑level change\. Becauseκ\\kappais evaluated with the current student policy, its value can evolve during training as the student distribution changes\.

So far, we have analyzed the effect of a single deviation at a given step\. But what happens when we consider all possible deviations at that step?

#### 4\.3The Bridge–Garden Bound on Exposure Bias

To capture how sensitive a given stepssis toallpossible token deviations, we aggregate the local sensitivities by summing over the vocabulary:

κ\(s\):=∑a∈𝒱κ\(a∣s\)\.\\kappa\(s\):=\\sum\\nolimits\_\{a\\in\\mathcal\{V\}\}\\kappa\(a\\mid s\)\.This quantity summarizes the token‑level risk at stepss\. Whenκ\(s\)\\kappa\(s\)is large, many tokens have largeκ\(a\|s\)\\kappa\(a\|s\); changing the teacher’s token atssto one of them can substantially increase the loss\. Whenκ\(s\)\\kappa\(s\)is small, mostκ\(a\|s\)\\kappa\(a\|s\)are small, so for the majority of tokens a deviation atsshas only a mild effect\. Accordingly, high‑ and low‑sensitivityκ\(s\)\\kappa\(s\)regions correspond naturally to the notions of Bridges and Gardens introduced earlier \(Sec\.[4\.1](https://arxiv.org/html/2605.26246#S4.SS1)\)\.

Definition 4\.2\(The Bridge–Garden Partition\)Given a task‑ and model‑dependent thresholdτ\\tau, partition the prefix space𝒮\\mathcal\{S\}intoBridge:ℬ:=\{s∈𝒮:κ\(s\)\>τ\},\\textbf\{Bridge:\}\\quad\\mathcal\{B\}:=\\\{s\\in\\mathcal\{S\}:\\kappa\(s\)\>\\tau\\\},Garden:𝒢:=\{s∈𝒮:κ\(s\)≤τ\}\.\\textbf\{Garden:\}\\quad\\mathcal\{G\}:=\\\{s\\in\\mathcal\{S\}:\\kappa\(s\)\\leq\\tau\\\}\.

This partition provides a structured lens for analyzing the bound of Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2), as formalized next \(proof in App\.[C](https://arxiv.org/html/2605.26246#A3)\)\.

Proposition 4\.1\(The Bridge–Garden Upper Bound on Exposure Bias\)Using the partition from Def\.[4\.3](https://arxiv.org/html/2605.26246#S4.SS3), the bound from Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)decomposes as follows:F\(πθ\):=Fℬ\(πθ\)\+F𝒢\(πθ\),F\(\\pi\_\{\\theta\}\):=F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\theta\}\)\+F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\theta\}\),whereF𝒳\(πθ\):=𝔼s∼dT\[1𝒳\(s\)F\(s,πθ\)\]F\_\{\\mathcal\{X\}\}\(\\pi\_\{\\theta\}\):=\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\[1\_\{\\mathcal\{X\}\}\(s\)F\(s,\\pi\_\{\\theta\}\)\]for𝒳∈\{ℬ,𝒢\}\\mathcal\{X\}\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\}, and the termF\(s,πθ\)F\(s,\\pi\_\{\\theta\}\)is defined in Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\.

The decomposition reveals an asymmetry in how a student should learn from the teacher:

In Bridgesℬ\\mathcal\{B\}, only a few tokens are safe; most alternatives carry high risk\. HenceFℬF\_\{\\mathcal\{B\}\}is dominated by deviations on tokens with largeκ\(a\|s\)\\kappa\(a\|s\)\. To minimizeFℬF\_\{\\mathcal\{B\}\}, the student must avoid these high‑risk tokens and concentrate probability on the few safe options supported by the teacher\.

In Gardens𝒢\\mathcal\{G\}, many next‑token choices are acceptable and theirκ\(a\|s\)\\kappa\(a\|s\)are uniformly small\. Because no single deviation incurs a large penalty, the bound behaves like a standard distribution‑matching distance\. MinimizingF𝒢F\_\{\\mathcal\{G\}\}encourages the student to match the teacher’s broad distribution, preserving the diversity inherent in Gardens\.

#### 4\.4Mechanism of Hybrid Improvement

NOTE: Hard KD and Soft KD refer to Eq\.\([3](https://arxiv.org/html/2605.26246#S2.E3)\) and Eq\.\([2](https://arxiv.org/html/2605.26246#S2.E2)\), respectively\. Hybrid KD is defined in Obs\.[3\.1](https://arxiv.org/html/2605.26246#S3.SS1)\.

To reduce total exposure bias, we must keep it low in both Bridge and Garden regions\. We next show that neither Hard nor Soft KD alone can achieve this goal\.

Hard KD employs one‑hot teacher targets, pushing the student to place most probability on the teacher’s single preferred token\. This is effective in Bridges, but fails in Gardens because it suppresses distributional diversity\. Soft KD matches the teacher’s full distribution, thereby keeping probability spread over many teacher‑supported tokens\. This works well in Gardens\. But in Bridges, it may assign non‑negligible probability to high‑risk alternatives\.

If, however, we adopt a Hybrid KD regime, we can reduce the loss in both regions by choosing an appropriate mixing coefficientλ\\lambda, as empirically observed\. The following one\-step theorem makes this intuition precise \(proof in App\.[D](https://arxiv.org/html/2605.26246#A4)\)\.

Theorem 4\.2\(Complementarity Gain\) \(Informal\)Letπhard\\pi\_\{\\text\{hard\}\}andπsoft\\pi\_\{\\text\{soft\}\}be \(near\-\)minimizers ofℓhard\\ell\_\{\\text\{hard\}\}andℓsoft\\ell\_\{\\text\{soft\}\}, respectively; in late\-stage distillation,F𝒢\(πsoft\)<F𝒢\(πhard\),Fℬ\(πhard\)<Fℬ\(πsoft\)\.F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\text\{soft\}\}\)\\ <\\ F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\text\{hard\}\}\),\\quad F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\text\{hard\}\}\)\\ <\\ F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\text\{soft\}\}\)\.This suggests that for someλ∈\(0,1\)\\lambda\\in\(0,1\), the minimizerπhyb\\pi\_\{\\text\{hyb\}\}of the hybrid objective\(λℓsoft\+\(1−λ\)ℓhard\)\\big\(\\lambda\\ell\_\{\\text\{soft\}\}\+\(1\-\\lambda\)\\ell\_\{\\text\{hard\}\}\\big\)can achieveF\(πhyb\)<min⁡\{F\(πhard\),F\(πsoft\)\}\.F\(\\pi\_\{\\text\{hyb\}\}\)\\ <\\ \\min\\\{F\(\\pi\_\{\\text\{hard\}\}\),\\,F\(\\pi\_\{\\text\{soft\}\}\)\\\}\.

Thm\.[4\.4](https://arxiv.org/html/2605.26246#S4.SS4)formalizes the complementarity: the hard‑KD solutionπhard\\pi\_\{\\text\{hard\}\}has smallerFℬF\_\{\\mathcal\{B\}\}but largerF𝒢F\_\{\\mathcal\{G\}\}than the soft‑KD solutionπsoft\\pi\_\{\\text\{soft\}\}, and vice versa\. No single objective can keep bothFℬF\_\{\\mathcal\{B\}\}andF𝒢F\_\{\\mathcal\{G\}\}small simultaneously\. Blending the losses produces a hybrid model with a strictly smallerF\(π\)F\(\\pi\)than either extreme, tightening the exposure‑bias bound and aligning with the empirical gains of hybrid KD reported in Sec\.[3](https://arxiv.org/html/2605.26246#S3)\.

Table 1:Accuracy \(↑\\uparrow\) and the mean score \(Avg\.\) on general reasoning benchmarks for Qwen2\.5 and Llama3 models under different hybrid strategies \(with Forward KL as soft supervision\)\.Bestandsecond\-bestresults areboldedandunderlined\.MethodLlama3\.1\-8B→\\toLlama3\.2\-1BQwen2\.5\-7B→\\toQwen2\.5\-3BBBHMMLUARC\-CThmQAAvg\.BBHMMLUARC\-CThmQAAvg\.Teacher57\.72±0\.0770\.90±0\.0383\.58±0\.1018\.10±0\.4057\.5864\.66±0\.6978\.22±0\.2289\.90±0\.2432\.47±0\.3266\.31Student \(No Distill\)14\.01±4\.4119\.78±9\.8921\.57±9\.942\.22±0\.4414\.4022\.34±0\.0664\.61±0\.0678\.40±0\.0312\.22±0\.2544\.39Hard KD\(Kim and Rush,[2016](https://arxiv.org/html/2605.26246#bib.bib20)\)15\.29±2\.7522\.54±1\.3823\.98±1\.963\.88±0\.7316\.4241\.52±0\.3365\.76±0\.1878\.75±0\.7123\.75±0\.4052\.45Soft KD\(Hintonet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib54)\)22\.07±2\.1133\.13±1\.6933\.41±1\.784\.37±0\.4323\.2541\.65±0\.1664\.45±0\.0478\.33±0\.2223\.02±0\.3351\.87Static Weighting \(Ours\)23\.03±1\.2634\.12±1\.8034\.35±1\.535\.42±0\.7324\.2342\.61±0\.0966\.89±0\.1979\.30±0\.1425\.45±0\.2353\.56Confidence\-based Weighting \(Ours\)25\.64±0\.8035\.32±1\.4534\.68±2\.243\.97±0\.7824\.9044\.07±0\.1967\.50±0\.1980\.77±0\.2722\.78±0\.2853\.78Entropy\-based Weighting \(Ours\)24\.40±0\.8835\.93±1\.5634\.78±2\.615\.83±0\.5325\.2346\.83±0\.1667\.06±0\.1479\.81±0\.7923\.65±0\.2054\.34Curriculum Schedule \(Ours\)25\.06±1\.1936\.13±1\.5435\.29±1\.166\.18±0\.6825\.6744\.39±0\.1867\.32±0\.2080\.51±0\.1022\.13±0\.4753\.59Risk\-Guided Hybrid \(Ours\)27\.44±2\.8535\.64±3\.2237\.01±3\.985\.00±1\.2526\.2746\.53±0\.0569\.05±0\.0781\.23±0\.0023\.82±0\.5855\.16

### 5Practical Algorithms for Bridge\-Garden Hybrid Supervision

Guided by the Bridge–Garden analysis in Sec\.[4](https://arxiv.org/html/2605.26246#S4), we now develop practical algorithms that seek hard\-label supervision in Bridges while preserving distributional diversity in Gardens\. A natural starting point is to combine the two losses with a fixed mixing coefficientλ\\lambda:

ℓhyb\(s;θ\)=λ⋅ℓsoft\(s;θ\)\+\(1−λ\)⋅ℓhard\(s;θ\)\.\\ell\_\{\\text\{hyb\}\}\(s;\\theta\)=\\lambda\\cdot\\ell\_\{\\text\{soft\}\}\(s;\\theta\)\+\(1\-\\lambda\)\\cdot\\ell\_\{\\text\{hard\}\}\(s;\\theta\)\.\(5\)
Although this static hybrid already outperforms pure soft or hard KD \(Obs\.[3\.1](https://arxiv.org/html/2605.26246#S3.SS1)\), it does not fully reflect the distinct needs of Bridges and Gardens\. We next explore two different strategies:\(i\)dynamically tuning the weightλ\\lambda\(Methods 1–3\) and\(ii\)modifying the hard‑loss term itself \(Method 4\)\.

1\) Confidence\-based weighting\.Teacher’s confidence in its top prediction serves as a simple proxy for distinguishing Bridges from Gardens\. In Bridges, a well\-trained teacher typically places most probability mass on a small set of low\-risk next tokens, since high\-risk deviations can trigger error accumulation\. Motivated by this, we reduce the soft‑label weight when the teacher is confident:

λconf\(s\)=1−maxa⁡πT\(a∣s\),\\lambda\_\{\\text\{conf\}\}\(s\)=1\-\\max\_\{a\}\\pi\_\{T\}\(a\\mid s\),
2\) Entropy\-based weighting\.Alternatively, we can use the teacher’s predictive entropy as a global uncertainty signal: higher entropy indicates that the teacher considers the next\-token choice flexible, aligning with the behavior of Gardens\. We thus set the soft\-loss weight to the normalized entropy:

λent\(s\)=HT\(s\)log⁡\|𝒱\|,HT\(s\)=−∑a∈𝒱πT\(a\|s\)log⁡πT\(a\|s\)\.\\lambda\_\{\\text\{ent\}\}\(s\)=\\frac\{H\_\{T\}\(s\)\}\{\\log\|\\mathcal\{V\}\|\},\\ \\ \\ H\_\{T\}\(s\)=\-\\sum\_\{a\\in\\mathcal\{V\}\}\\pi\_\{T\}\(a\|s\)\\log\\pi\_\{T\}\(a\|s\)\.
3\) Curriculum schedule\.Since mistakes in Bridges can lead to error accumulation, it can be useful to emphasize hard labels early in training, letting the student first learn these critical regions reliably\. The weight on soft labels is then gradually increased to capture the diversity present in Gardens\. We implement this with a linear warm‑up:

λ\(t\)=min⁡\(tT,1\)⋅λmax,\\lambda\(t\)=\\min\\left\(\\frac\{t\}\{T\},\\ 1\\right\)\\cdot\\lambda\_\{\\max\},whereλmax\\lambda\_\{\\max\}is the target soft‑label weight,ttis the current training step, andTTis the number of warm up steps\.

4\) Risk\-Guided Hybrid\.Beyond adjusting the mixing coefficient, we can also revise the hard loss from a reinforcement learning perspective: the riskκ\(a\|s\)\\kappa\(a\|s\)can be related to reward, where higher risk implies lower reward\. From this perspective, reward is concentrated in Bridges but more uniform in Gardens\. Recall that maximizing reward incentivizes the consistency between the policy \(student distribution\) and the reward distribution\. In this way, the student tends to behave sharply in Bridges and preserve diversity in Gardens\. This directly extends recent reward\-optimization advances for LMs\(Cundy and Ermon,[2023](https://arxiv.org/html/2605.26246#bib.bib2)\)to KD, and thus we have the following objective:

ℓhard′\(s;θ\)≔ℓhard\(s;θ\)\+α4Δθ\(s,a∗\)2,\\ell^\{\\prime\}\_\{\\text\{hard\}\}\(s;\\theta\)\\coloneqq\\ell\_\{\\text\{hard\}\}\(s;\\theta\)\+\\frac\{\\alpha\}\{4\}\\,\\Delta\_\{\\theta\}\(s,a^\{\*\}\)^\{2\},whereα\>0\\alpha\>0,a∗a^\{\*\}is the hard target inℓhard\\ell\_\{\\text\{hard\}\}\(Sec\.[2](https://arxiv.org/html/2605.26246#S2)\), andΔθ\(s,a∗\)≔log∑a′∈𝒱exp⁡\(fθ\(a′∣\[s,a∗\]\)−fθ\(a∗∣s\)\),\\Delta\_\{\\theta\}\(s,a^\{\*\}\)\\coloneqq\\log\\\!\\sum\_\{a^\{\\prime\}\\in\\mathcal\{V\}\}\\exp\\\!\\big\(f\_\{\\theta\}\(a^\{\\prime\}\\mid\[s,a^\{\*\}\]\)\-f\_\{\\theta\}\(a^\{\*\}\\mid s\)\\big\),withfθ\(⋅∣s\)f\_\{\\theta\}\(\\cdot\\mid s\)the student logits\. Here\[s,a∗\]\[s,a^\{\*\}\]denotes the prefix obtained by appending the hard targeta∗a^\{\*\}toss\. In all experiments, we fixα=0\.1\\alpha=0\.1; the additional term reuses the student logits and only adds one log\-sum\-exp computation per token, so its cost remains close to standard soft KD \(see App\.[F\.1](https://arxiv.org/html/2605.26246#A6.SS1)\)\.

Table 2:Performance comparison of Qwen2\.5 on reasoning benchmarks\. Avg\. is the mean across all benchmarks\. We evaluate our hybrid KD \(Forward KL\) against recent soft KD methods\.Notably, our approach is orthogonal to these works \(see Fig\.[3](https://arxiv.org/html/2605.26246#S5.F3)for gains when combined\)\.Bestandsecond\-bestresults are highlighted, also denoted byboldandunderlinedvalues\.MethodQwen2\.5\-7B→\\toQwen2\.5\-0\.5BQwen2\.5\-7B→\\toQwen2\.5\-3BBBHMMLUARC\-CThmQAAvg\.BBHMMLUARC\-CThmQAAvg\.Reverse KL\(Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19)\)24\.91±0\.0144\.72±0\.0147\.44±0\.0011\.00±0\.0032\.0244\.07±0\.1065\.67±0\.1977\.68±0\.1224\.20±0\.4952\.90Total Variation\(Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55)\)26\.74±0\.3844\.35±0\.1246\.76±0\.5110\.20±0\.5632\.0140\.50±0\.1664\.52±0\.0378\.11±0\.1822\.83±0\.5651\.49JS divergence\(Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73)\)24\.55±0\.1343\.31±0\.1044\.78±0\.0511\.82±0\.3731\.1245\.50±0\.0864\.68±0\.1778\.85±0\.1422\.27±0\.4152\.83Adaptive KL\(Wuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib70)\)26\.04±0\.2341\.85±0\.1145\.89±0\.3211\.33±0\.6131\.2844\.71±0\.1364\.69±0\.0779\.23±0\.2222\.25±0\.3352\.72Skew FKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)25\.87±0\.2444\.12±0\.2447\.12±0\.6510\.65±0\.2231\.9441\.39±0\.1764\.67±0\.1577\.75±0\.2323\.77±0\.7151\.89Skew RKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)28\.16±0\.1645\.03±0\.0547\.51±0\.2011\.37±0\.6633\.0241\.22±0\.0863\.95±0\.0876\.91±0\.1823\.67±0\.2051\.44α\\alpha\-β\\betadivergence\(Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59)\)26\.18±0\.1542\.59±0\.1846\.70±0\.3711\.07±0\.4331\.6345\.12±0\.2364\.95±0\.1779\.81±0\.0922\.94±0\.5453\.21HybKD \(Ours\)26\.58±0\.1649\.08±0\.2251\.69±0\.6210\.50±0\.5434\.4646\.53±0\.0569\.05±0\.0781\.23±0\.0023\.82±0\.5855\.16

![Refer to caption](https://arxiv.org/html/2605.26246v1/x2.png)Figure 3:Performance achieved by integrating various divergences with hard\-label supervision \(via static weighting\)\. HybRKL stands for the hybrid of Reverse KL and hard labels, with other hybrid variants named similarly\. More numerical results are available in App\.[F](https://arxiv.org/html/2605.26246#A6)\.Table 3:Distillation results on math benchmarks \(↑\\uparrow\)\.MethodQwen2\.5\-Math\-7B→\\to1\.5BGSM8KMATHGaokao23Avg\.Hard KD65\.75±4\.4350\.44±1\.7841\.04±3\.1552\.41Forward KL68\.72±0\.9249\.89±0\.8241\.77±1\.1153\.46Reverse KL58\.98±4\.4446\.78±4\.0237\.35±4\.6847\.70Skew F\(R\)KL69\.13±1\.6249\.59±1\.6342\.23±1\.9253\.65α\\alpha\-β\\betadivergence70\.86±2\.2950\.68±1\.3042\.88±2\.3454\.81HybKD \(Ours\)71\.65±1\.2051\.37±0\.7144\.10±2\.0955\.71Table 4:DeepSeek\-Coder distillation results \(↑\\uparrow\)\.HE: HumanEval\.MethodDeepSeek\-Coder\-6\.7B→\\to1\.3BHEHE\+MBPPMBPP\+Avg\.Hard KD35\.3732\.9360\.3250\.0044\.66Forward KL38\.4133\.5463\.4951\.5946\.76Reverse KL39\.0234\.1562\.1750\.0046\.34Skew F\(R\)KL42\.0735\.9860\.8550\.2647\.29α\\alpha\-β\\betadivergence41\.4636\.5960\.0550\.2647\.09HybKD \(Ours\)41\.4636\.5963\.1250\.3947\.89
### 6Experiments

We now conduct empirical experiments to validate our theoretical analysis and to assess the practical effectiveness of the proposed hybrid distillation framework\. Specifically, we seek to answer three key questions:

Q1 \(Hybrid Strategy\):How do different hybrid KD variants impact distillation performance?

Q2 \(Universality\):Are the gains from hybrid distillation consistent across different divergence measures?

Q3 \(vs\. On\-Policy Methods\):Compared to recent on\-policy distillation techniques, does our approach improve final accuracy while being more efficient?

In App\.[F](https://arxiv.org/html/2605.26246#A6), we extend the primary benchmark study in three directions\. We first test whether the same hard–soft pattern holds under larger teacher–student capacity gaps, open\-ended generation, and on\-policy prefixes\. We then measure training cost and compare against simpler alternatives, including reverse\-proxy rules, regularization, and temperature changes\. Finally, we use controlled synthetic domains where exact token\-levelκ\\kappais computable, so the Bridge–Garden decomposition can be tested directly rather than only inferred from LLM benchmarks\.

Table 5:Accuracy \(↑\\uparrow\) and mean score \(Avg\.\) on Llama and Gemma models\.Bestandsecond\-bestresults are highlighted\.MethodLlama3\.1\-8B→\\toLlama3\.2\-1BGemma3\-4B→\\toGemma3\-1BBBHMMLUARC\-CThmQAAvg\.BBHMMLUARC\-CThmQAAvg\.Reverse KL\(Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19)\)23\.69±0\.7731\.63±1\.8832\.08±1\.203\.60±0\.0922\.759\.52±1\.0324\.77±0\.6827\.58±0\.601\.26±0\.3215\.78Total Variation\(Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55)\)23\.55±1\.5127\.41±4\.9628\.99±4\.472\.68±0\.5620\.666\.29±0\.8625\.07±0\.3125\.83±0\.431\.29±0\.4114\.62JS divergence\(Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73)\)25\.16±2\.0334\.08±2\.1534\.42±1\.815\.70±0\.1324\.847\.14±0\.7425\.27±0\.5626\.03±0\.334\.57±0\.9715\.75Adaptive KL\(Wuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib70)\)25\.18±1\.8633\.86±2\.3433\.74±1\.774\.85±0\.4424\.415\.47±0\.6324\.76±1\.5624\.07±1\.124\.23±0\.4114\.63Skew FKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)25\.05±1\.4031\.39±2\.8432\.34±2\.994\.73±0\.8623\.377\.01±0\.8225\.08±0\.8826\.14±0\.705\.01±0\.2915\.81Skew RKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)25\.24±0\.7933\.00±1\.4032\.24±1\.364\.27±0\.4423\.695\.83±0\.2525\.17±0\.2225\.93±0\.293\.38±0\.2615\.08α\\alpha\-β\\betadivergence\(Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59)\)25\.07±1\.3634\.78±2\.0434\.90±1\.455\.27±0\.4425\.017\.26±0\.7825\.49±1\.0026\.07±0\.664\.34±0\.8915\.79HybKD \(Ours\)27\.44±2\.8535\.64±3\.2237\.01±3\.985\.00±1\.2526\.277\.78±1\.1925\.94±0\.5526\.07±0\.855\.24±0\.7916\.26

#### 6\.1Experimental Setup

Models and Benchmarks\.Our primary benchmark suite evaluates seven teacher–student pairs fromQwen2\.5\(Qwenet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib78)\)\(7B→\\to\{0\.5, 1\.5, 3\}B\),Llama\(Grattafiori and others,[2024](https://arxiv.org/html/2605.26246#bib.bib74)\)\(8B→\\to1B\),Gemma\-3\(Gemma Teamet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib32)\)\(4B→\\to1B\),Qwen\-Math\(Yanget al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib77)\)\(7B→\\to1\.5B\), andDeepSeek\-Coder\(Guoet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib76)\)\(6\.7B→\\to1\.3B\)\. Evaluation spans commonsense \(MMLU\(Hendryckset al\.,[2020](https://arxiv.org/html/2605.26246#bib.bib48)\), BBH\(Suzgunet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib33)\), ARC\-C\(Clarket al\.,[2018](https://arxiv.org/html/2605.26246#bib.bib46)\), ThmQA\(Chenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib45)\)\), math \(GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib56)\), MATH\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib79)\), Gaokao23\(Liaoet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib36)\)\), and code \(HumanEval\(Chen,[2021](https://arxiv.org/html/2605.26246#bib.bib80)\), MBPP\(Austinet al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib81)\)\)\. App\.[F](https://arxiv.org/html/2605.26246#A6)adds two further teacher–student settings and open\-ended evaluation\. We reportfive\-seed mean accuracy\. Training and evaluation details are in App\.[E](https://arxiv.org/html/2605.26246#A5)\.

Baselines\.We compare against three baseline categories:\(1\) Standard off\-policy KD\.Hard KD\(Kim and Rush,[2016](https://arxiv.org/html/2605.26246#bib.bib20)\), which trains on teacher\-generated tokens, and Forward KL distillation\(Hintonet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib54)\)\.\(2\) Alternative divergence objectives\.Method using Reverse KL\(Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19)\), JS divergence\(Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73)\), Total Variation \(TV\)\(Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55)\), Adaptive KL\(Wuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib70)\), Skew FKL/RKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\), and theα\\alpha–β\\betadivergence framework\(Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59)\)\.\(3\) On\-policy techniques\.Recent methods that train the student on its own generated sequences\(Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19); Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73); Koet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib72)\), which directly mitigate exposure bias by aligning training and inference distributions\.

#### 6\.2Q1: Effectiveness of Hybrid Strategy

We evaluate various hybrid strategies against pure Hard and Soft KD to answerQ1\. From Tab\.[1](https://arxiv.org/html/2605.26246#S4.T1), we observe:1\) Neither pure baseline is consistently superior\.Soft KD leads on Llama\-3\.2\-1B \(\+6\.83 average score\), while Hard KD leads on Qwen2\.5\-3B \(\+0\.59\)\. This suggests their complementary potential\.2\) Hybrids consistently outperform pure KD\.Even simple static weighting improve average scores \(\+1\.69 on Qwen2\.5\-3B\), with adaptive methods \(e\.g\.,confidence\-, entropy\-, risk\-guided\) providing further gains\.3\) Curriculum scheduling becomes highly competitive for large capacity gaps\(Llama3\.1\-8B → 3\.2\-1B\)\. Its success in this setting supports that prioritizing Bridge regions is essential for stable learning when student capacity is limited\.

#### 6\.3Q2: Universality across Divergence Measures

We evaluate our method across various divergences to answerQ2and find: 1\) Our method excels with a fixed Forward KL\. Itoutperforms the best non\-hybrid baselineby \+1\.95 average points on Qwen2\.5\-3B \(Tab\.[2](https://arxiv.org/html/2605.26246#S5.T2)\)\. Consistent gains are also observed across more models \(e\.g\.,Llama, Gemma\) and domains \(e\.g\.,math, coding\), as shown in Tabs\.[3](https://arxiv.org/html/2605.26246#S5.T3),[4](https://arxiv.org/html/2605.26246#S5.T4), and[5](https://arxiv.org/html/2605.26246#S6.T5)\.2\) The hybrid principle is broadly applicable\.It consistently improves other measures like Reverse or Skew KL \(Fig\.[3](https://arxiv.org/html/2605.26246#S5.F3)\)\. These results show that the benefits of hybrid supervision come from the adaptive mechanism rather than a specific divergence choice\.

![Refer to caption](https://arxiv.org/html/2605.26246v1/x3.png)Figure 4:Relative accuracy gain and training cost \(s/step\) versus On\-policy KD underidentical experimental settings\(see App\.[E\.2](https://arxiv.org/html/2605.26246#A5.SS2)\)\.
#### 6\.4Q3: Comparison with On\-Policy Approach

To answerQ3, we compare our method with on\-policy KD in the Qwen2\.5\-7B→\\rightarrow3B setting\. As shown in Fig\.[4](https://arxiv.org/html/2605.26246#S6.F4), our approach \(with only static weighting\) achieves a 1\.43% average performance gain over this baseline\. Crucially, while our method maintains training costs comparable to standard Hard/Soft KD, it is9\.7x more efficientthan on\-policy KD, making it more practical for large\-scale deployment\. Because hybrid supervision modifies the target distribution rather than the prefix source, it can also be combined with on\-policy prefix sampling; App\.[F\.1](https://arxiv.org/html/2605.26246#A6.SS1)shows that this combination consistently improves over on\-policy soft KD across reasoning and code tasks\.

### 7Additional Analysis

To better understand the behavior of hybrid distillation, we examine its properties from the following two aspects\.

Impact of Mixing Weightλ\\lambda\.We evaluate Hybrid KD across a range of fixed weights\. As shown in Fig\.[5](https://arxiv.org/html/2605.26246#S7.F5), intermediate values ofλ∈\(0,1\)\\lambda\\in\(0,1\)consistently outperform both pure hard \(λ=0\\lambda=0\) and soft \(λ=1\\lambda=1\) distillation\. We also observe that the optimal weight varies across tasks and models\. This finding highlights the importance of the adaptive strategies introduced in Sec\.[5](https://arxiv.org/html/2605.26246#S5)based on our Bridge–Garden theory\.

Entropy Analysis\.We further examine the student’s next\-token entropy\. As illustrated in Fig\.[6](https://arxiv.org/html/2605.26246#S7.F6), Hard KD induces sharper \(low\-entropy\) distributions while Soft KD yields flatter ones, aligning with their roles in Bridge and Garden regions\. Hybrid KD maintains an intermediate entropy, balancing Bridge precision with Garden diversity to achieve superior performance over single\-label baselines\. App\.[F](https://arxiv.org/html/2605.26246#A6)checks the main alternative explanations directly: reversing the confidence or entropy rule hurts performance, global regularization and temperature changes do not consistently match Hybrid KD, and controlled synthetic experiments show that the largest exactκ\\kappavalues concentrate on semantic decision states\.

![Refer to caption](https://arxiv.org/html/2605.26246v1/x4.png)Figure 5:Effect of mixing weightλ\\lambdaon student performance\.![Refer to caption](https://arxiv.org/html/2605.26246v1/figs/fkl_sft/student_entropy_comparison.png)Figure 6:Student entropy during training\. Left: DeepSeek\-Coder\-1\.3B on code generation; Right: Qwen2\.5\-3B on general reasoning\.
### 8Conclusion

In this work, we identify a hard\-label paradox where hard supervision enhances student inference by reducing exposure bias, despite degrading teacher imitation during training\. We capture this with a Bridge–Garden partition of generation into exact\-prediction regions \(Bridges\) and flexible regions \(Gardens\)\. Hard KD excels at suppressing risky deviations in Bridges, whereas Soft KD specializes in preserving diversity in Gardens\. Based on this insight, we develop hybrid strategies that adaptively combine their behaviors\. Empirical results show that our methods consistent improvement over existing baselines, with up to9\.7×\\timeslower training cost\.

### Impact Statement

This work aims to improve the efficiency and reliability of knowledge distillation for language models, potentially reducing inference costs and enabling deployment under tighter compute and energy budgets\. As with most advances in model training and compression, these techniques could be used for beneficial applications \(e\.g\.,accessibility, on\-device assistance\) as well as for misuse\. We encourage future work to evaluate downstream safety implications and to pair efficiency gains with robust safeguards\.

### Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants 62525212, U23B2051, 62236008, 62441232, 62521007, U21B2038, 62576332, and 62502500, in part by the Youth Innovation Promotion Association CAS, in part by the Strategic Priority Research Program of the Chinese Academy of Sciences under Grant XDB0680201, in part by the project ZR2025ZD01 supported by Shandong Provincial Natural Science Foundation, in part by the China National Postdoctoral Program for Innovative Talents under Grant BX20240384, in part by Beijing Natural Science Foundation under Grant L252144, in part by General Program of the Chinese Postdoctoral Science Foundation under Grant 2025M771558, in part by the Beijing Major Science and Technology Project under Contract Z251100008125059, and in part by Beijing Academy of Artificial Intelligence \(BAAI\)\.

### References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1),[§1](https://arxiv.org/html/2605.26246#S1.p1.1),[§2](https://arxiv.org/html/2605.26246#S2.p5.3)\.
- R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. R\. Garea, M\. Geist, and O\. Bachem \(2024\)On\-policy distillation of language models: learning from self\-generated mistakes\.InInternational Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§A\.4](https://arxiv.org/html/2605.26246#A1.SS4.p2.1),[§E\.2](https://arxiv.org/html/2605.26246#A5.SS2.p2.1),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.74.74.74.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.76.74.74.11),[§1](https://arxiv.org/html/2605.26246#S1.p2.1),[§1](https://arxiv.org/html/2605.26246#S1.p4.1),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.34.34.34.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.36.34.34.11)\.
- I\. Amara, N\. Sepahvand, B\. H\. Meyer, W\. J\. Gross, and J\. J\. Clark \(2022\)Bd\-kd: balancing the divergences for online knowledge distillation\.arXiv preprint arXiv:2212\.12965\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.\(2021\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[2nd item](https://arxiv.org/html/2605.26246#A5.I5.i2.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- S\. Bengio, O\. Vinyals, N\. Jaitly, and N\. Shazeer \(2015\)Scheduled sampling for sequence prediction with recurrent neural networks\.InAdvances in Neural Information Processing Systems,Cited by:[§A\.4](https://arxiv.org/html/2605.26246#A1.SS4.p1.1),[§1](https://arxiv.org/html/2605.26246#S1.p4.1),[§3\.2](https://arxiv.org/html/2605.26246#S3.SS2.p5.pic1.2.2.2.2.2.2.2.2.2.2.2.2.2.2.2.2.2.2.3.2.1)\.
- L\. Beyer, X\. Zhai, A\. Royer, L\. Markeeva, R\. Anil, and A\. Kolesnikov \(2022\)Knowledge distillation: a good teacher is patient and consistent\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 10925–10934\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- K\. Binici, N\. T\. Pham, T\. Mitra, and K\. Leman \(2022\)Preventing catastrophic forgetting and distribution mismatch in knowledge distillation via synthetic data\.InIEEE/CVF Winter Conference on Applications of Computer Vision,pp\. 663–671\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- O\. Bousquet and A\. Elisseeff \(2002\)Stability and generalization\.Journal of Machine Learning Research2,pp\. 499–526\.Cited by:[§4\.2](https://arxiv.org/html/2605.26246#S4.SS2.p8.5)\.
- D\. Busbridge, A\. Shidani, F\. Weers, J\. Ramapuram, E\. Littwin, and R\. Webb \(2025\)Distillation scaling laws\.InInternational Conference on Machine Learning,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- N\. Carlini, D\. Paleka, K\. D\. Dvijotham, T\. Steinke, J\. Hayase, A\. F\. Cooper, K\. Lee, M\. Jagielski, M\. Nasr, A\. Conmy,et al\.\(2024\)Stealing part of a production language model\.arXiv preprint arXiv:2403\.06634\.Cited by:[§2](https://arxiv.org/html/2605.26246#S2.p5.3)\.
- D\. Chen, J\. Mei, H\. Zhang, C\. Wang, Y\. Feng, and C\. Chen \(2022\)Knowledge distillation with the reused teacher classifier\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 11933–11942\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- M\. Chen \(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[1st item](https://arxiv.org/html/2605.26246#A5.I5.i1.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- W\. Chen, M\. Yin, M\. Ku, P\. Lu, Y\. Wan, X\. Ma, J\. Xu, X\. Wang, and T\. Xia \(2023\)TheoremQA: a theorem\-driven question answering dataset\.InConference on Empirical Methods in Natural Language Processing,pp\. 7889–7901\.Cited by:[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- W\. Chiang, Z\. Li, Z\. Lin, Y\. Sheng, Z\. Wu, H\. Zhang, L\. Zheng, S\. Zhuang, Y\. Zhuang, J\. E\. Gonzalez, I\. Stoica, and E\. P\. Xing \(2023\)Vicuna: an open\-source chatbot impressing gpt\-4 with 90%\* chatgpt quality\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1)\.
- J\. H\. Cho and B\. Hariharan \(2019\)On the efficacy of knowledge distillation\.InIEEE/CVF International Conference on Computer Vision,pp\. 4794–4802\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§3\.1](https://arxiv.org/html/2605.26246#S3.SS1.p1.1)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? try arc, the ai2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.Cited by:[3rd item](https://arxiv.org/html/2605.26246#A5.I1.i3.p1.1),[3rd item](https://arxiv.org/html/2605.26246#A5.I2.i3.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[1st item](https://arxiv.org/html/2605.26246#A5.I4.i1.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- C\. Cundy and S\. Ermon \(2023\)Sequencematch: imitation learning for autoregressive sequence modelling with backtracking\.arXiv preprint arXiv:2306\.05426\.Cited by:[§5](https://arxiv.org/html/2605.26246#S5.p7.1)\.
- S\. Ganguly, R\. Nayak, R\. Rao, U\. Deb, and P\. AP \(2024\)AdaKD: dynamic knowledge distillation of asr models using adaptive loss weighting\.arXiv preprint arXiv:2405\.08019\.Cited by:[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- Gemma Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière,et al\.\(2025\)Gemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.Cited by:[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- A\. Grattafioriet al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§1](https://arxiv.org/html/2605.26246#S1.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- Y\. Gu, L\. Dong, F\. Wei, and M\. Huang \(2024\)MiniLLM: knowledge distillation of large language models\.InInternational Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§A\.4](https://arxiv.org/html/2605.26246#A1.SS4.p2.1),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.54.54.54.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.56.54.54.11),[§1](https://arxiv.org/html/2605.26246#S1.p2.1),[§1](https://arxiv.org/html/2605.26246#S1.p4.1),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.14.14.14.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.16.14.14.11)\.
- Y\. Gu, H\. Zhou, F\. Meng, J\. Zhou, and M\. Huang \(2025\)MiniPLM: knowledge distillation for pre\-training language models\.InInternational Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1),[§1](https://arxiv.org/html/2605.26246#S1.p1.1),[§1](https://arxiv.org/html/2605.26246#S1.p2.1)\.
- D\. Guo, Q\. Zhu, D\. Yang, Z\. Xie, K\. Dong, W\. Zhang, G\. Chen, X\. Bi, Y\. Wu, Y\. Li,et al\.\(2024\)DeepSeek\-coder: when the large language model meets programming–the rise of code intelligence\.arXiv preprint arXiv:2401\.14196\.Cited by:[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- S\. Ham, Y\. Choi, Y\. Yang, S\. Cho, Y\. Kim, and C\. Kim \(2025\)Safety\-aligned weights are not enough: refusal\-teacher\-guided finetuning enhances safety and downstream performance under harmful finetuning attacks\.Note:OpenReviewCited by:[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- Z\. Hao, J\. Guo, K\. Han, Y\. Tang, H\. Hu, Y\. Wang, and C\. Xu \(2023\)One\-for\-all: bridge the gap between heterogeneous architectures in knowledge distillation\.InAdvances in Neural Information Processing Systems,pp\. 79570–79582\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- H\. Harutyunyan, A\. S\. Rawat, A\. K\. Menon, S\. Kim, and S\. Kumar \(2023\)Supervision complexity and its role in knowledge distillation\.arXiv preprint arXiv:2301\.12245\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2020\)Measuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.Cited by:[1st item](https://arxiv.org/html/2605.26246#A5.I2.i1.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[3rd item](https://arxiv.org/html/2605.26246#A5.I1.i3.p1.1),[2nd item](https://arxiv.org/html/2605.26246#A5.I4.i2.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- G\. Hinton, O\. Vinyals, and J\. Dean \(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.44.44.44.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.46.44.44.11),[§1](https://arxiv.org/html/2605.26246#S1.p1.1),[§3\.1](https://arxiv.org/html/2605.26246#S3.SS1.p1.1),[Table 1](https://arxiv.org/html/2605.26246#S4.T1.46.44.44.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2)\.
- J\. Hoffmann, S\. Borgeaud, A\. Mensch, E\. Buchatskaya, T\. Cai, E\. Rutherford, D\. d\. L\. Casas, L\. A\. Hendricks, J\. Welbl, A\. Clark,et al\.\(2022\)Training compute\-optimal large language models\.arXiv preprint arXiv:2203\.15556\.Cited by:[§1](https://arxiv.org/html/2605.26246#S1.p1.1)\.
- T\. Huang, S\. You, F\. Wang, C\. Qian, and C\. Xu \(2022\)Knowledge distillation from a stronger teacher\.InAdvances in Neural Information Processing Systems,pp\. 33716–33727\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- Z\. Huang and N\. Wang \(2017\)Like what you like: knowledge distill via neuron selectivity transfer\.arXiv preprint arXiv:1707\.01219\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- A\. Jafari, M\. Rezagholizadeh, P\. Sharma, and A\. Ghodsi \(2021\)Annealing knowledge distillation\.InConference of the European Chapter of the Association for Computational Linguistics,pp\. 2493–2504\.Cited by:[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- S\. Jung, S\. Yoon, D\. Kim, and H\. Lee \(2025\)ToDi: token\-wise distillation via fine\-grained divergence control\.InConference on Empirical Methods in Natural Language Processing,pp\. 8078–8091\.Cited by:[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- G\. Kim, D\. Jang, and E\. Yang \(2024\)Promptkd: distilling student\-friendly knowledge for generative language models via prompt tuning\.arXiv preprint arXiv:2402\.12842\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- Y\. Kim and A\. M\. Rush \(2016\)Sequence\-level knowledge distillation\.InConference on Empirical Methods in Natural Language Processing,pp\. 1317–1327\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1),[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p1.1),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.34.34.34.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.36.34.34.11),[§1](https://arxiv.org/html/2605.26246#S1.p2.1),[Table 1](https://arxiv.org/html/2605.26246#S4.T1.36.34.34.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2)\.
- J\. Ko, T\. Chen, S\. Kim, T\. Ding, L\. Liang, I\. Zharkov, and S\. Yun \(2025\)DistiLLM\-2: a contrastive approach boosts the distillation of LLMs\.InInternational Conference on Machine Learning,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.104.104.104.11),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.94.94.94.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.106.104.104.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.96.94.94.11),[§1](https://arxiv.org/html/2605.26246#S1.p2.1),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.54.54.54.11),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.64.64.64.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.56.54.54.11),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.66.64.64.11)\.
- J\. Ko, S\. Kim, T\. Chen, and S\. Yun \(2024\)DistiLLM: towards streamlined distillation for large language models\.InInternational Conference on Machine Learning,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§A\.4](https://arxiv.org/html/2605.26246#A1.SS4.p2.1),[§E\.2](https://arxiv.org/html/2605.26246#A5.SS2.p2.1),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.104.104.104.11),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.94.94.94.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.106.104.104.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.96.94.94.11),[§1](https://arxiv.org/html/2605.26246#S1.p4.1),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.54.54.54.11),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.64.64.64.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.56.54.54.11),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.66.64.64.11)\.
- H\. Lee, Y\. Park, H\. Seo, and M\. Kang \(2023\)Self\-knowledge distillation via dropout\.Computer Vision and Image Understanding233,pp\. 103720\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- J\. Li, L\. Du, H\. Zhao, B\. Zhang, L\. Wang, B\. Gao, G\. Liu, and Y\. Lin \(2025\)Infinity instruct: scaling instruction selection and synthesis to enhance language models\.arXiv preprint arXiv:2506\.11116\.Cited by:[§E\.1](https://arxiv.org/html/2605.26246#A5.SS1.SSS0.Px1.p1.1)\.
- L\. Li, Y\. Bao, P\. Dong, C\. Yang, A\. Li, W\. Luo, Q\. Liu, W\. Xue, and Y\. Guo \(2024\)DetKDS: knowledge distillation search for object detectors\.InInternational Conference on Machine Learning,pp\. 27399–27416\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- Z\. Li, X\. Li, L\. Yang, B\. Zhao, R\. Song, L\. Luo, J\. Li, and J\. Yang \(2023a\)Curriculum temperature for knowledge distillation\.InAAAI Conference on Artificial Intelligence,pp\. 1504–1512\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- Z\. Li, P\. Xu, X\. Chang, L\. Yang, Y\. Zhang, L\. Yao, and X\. Chen \(2023b\)When object detection meets knowledge distillation: a survey\.IEEE Transactions on Pattern Analysis and Machine Intelligence45\(8\),pp\. 10555–10579\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- M\. Liao, W\. Luo, C\. Li, J\. Wu, and K\. Fan \(2024\)MARIO: math reasoning with code interpreter output–a reproducible pipeline\.arXiv preprint arXiv:2401\.08190\.Cited by:[3rd item](https://arxiv.org/html/2605.26246#A5.I4.i3.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- M\. Luo, S\. Tan, J\. Wong, X\. Shi, W\. Y\. Tang, M\. Roongta, C\. Cai, J\. Luo, L\. E\. Li, R\. A\. Popa, and I\. Stoica \(2025\)DeepScaleR: surpassing O1\-Preview with a 1\.5b model by scaling RL\.Note:Notion BlogCited by:[§E\.1](https://arxiv.org/html/2605.26246#A5.SS1.SSS0.Px2.p1.1)\.
- J\. Lv, H\. Yang, and P\. Li \(2024\)Wasserstein distance rivals kullback\-leibler divergence for knowledge distillation\.InAdvances in Neural Information Processing Systems,pp\. 65445–65475\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- F\. Meng, Z\. Wang, and M\. Zhang \(2024\)Pissa: principal singular values and singular vectors adaptation of large language models\.InAdvances in Neural Information Processing Systems,pp\. 121038–121072\.Cited by:[§E\.1](https://arxiv.org/html/2605.26246#A5.SS1.SSS0.Px3.p1.1)\.
- M\. Mohri, A\. Rostamizadeh, and A\. Talwalkar \(2018\)Foundations of machine learning\.2 edition,MIT Press\.Cited by:[§3\.2](https://arxiv.org/html/2605.26246#S3.SS2.p6.1)\.
- B\. Peng, C\. Li, P\. He, M\. Galley, and J\. Gao \(2023\)Instruction tuning with gpt\-4\.arXiv preprint arXiv:2304\.03277\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1),[§1](https://arxiv.org/html/2605.26246#S1.p2.1)\.
- H\. Peng, X\. Lv, Y\. Bai, Z\. Yao, J\. Zhang, L\. Hou, and J\. Li \(2025\)Pre\-training distillation for large language models: a design space exploration\.InAnnual Meeting of the Association for Computational Linguistics,pp\. 3603–3618\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- Qwen, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2025\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- Y\. Ren, Z\. Zhong, X\. Shi, Y\. Zhu, C\. Yuan, and M\. Li \(2023\)Tailoring instructions to student’s learning levels boosts knowledge distillation\.arXiv preprint arXiv:2305\.09651\.Cited by:[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- C\. Rosset, C\. Cheng, A\. Mitra, M\. Santacroce, A\. Awadallah, and T\. Xie \(2024\)Direct nash optimization: teaching language models to self\-improve with general preferences\.arXiv preprint arXiv:2404\.03715\.Cited by:[§A\.4](https://arxiv.org/html/2605.26246#A1.SS4.p2.1)\.
- D\. Shin, Y\. Kim, S\. Jo, B\. Na, and I\. Moon \(2025\)AMiD: knowledge distillation for llms with alpha\-mixture assistant distribution\.arXiv preprint arXiv:2510\.15982\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- S\. Sun, W\. Ren, J\. Li, R\. Wang, and X\. Cao \(2024\)Logit standardization in knowledge distillation\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 15731–15740\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. V\. Le, E\. H\. Chi, D\. Zhou, and J\. Wei \(2022\)Challenging big\-bench tasks and whether chain\-of\-thought can solve them\.arXiv preprint arXiv:2210\.09261\.Cited by:[2nd item](https://arxiv.org/html/2605.26246#A5.I2.i2.p1.1),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. Hashimoto \(2023\)Stanford alpaca: an instruction\-following LLaMA model\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1),[§1](https://arxiv.org/html/2605.26246#S1.p2.1)\.
- Y\. Tian, S\. Pei, X\. Zhang, C\. Zhang, and N\. V\. Chawla \(2025\)Knowledge distillation on graphs: a survey\.ACM Computing Surveys57\(8\),pp\. 1–16\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- B\. Wang, P\. Wang, W\. Xu, X\. Wang, Y\. Zhang, K\. Wang, and Y\. Wang \(2024a\)Kill two birds with one stone: rethinking data augmentation for deep long\-tailed learning\.InInternational Conference on Learning Representations,pp\. 19402–19425\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- D\. Wang, C\. Gong, M\. Li, Q\. Liu, and V\. Chandra \(2021\)AlphaNet: improved training of supernets with alpha\-divergence\.InInternational Conference on Machine Learning,pp\. 10760–10771\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- G\. Wang, D\. Liu, J\. Nie, Q\. Wan, R\. Hu, X\. Liu, W\. Liu, and J\. Liu \(2025a\)DEGAP: dual event\-guided adaptive prefixes for templated\-based event argument extraction with slot querying\.InInternational Conference on Computational Linguistics,pp\. 7598–7613\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1)\.
- G\. Wang, Z\. Yang, Z\. Wang, S\. Wang, Q\. Xu, and Q\. Huang \(2025b\)ABKD: pursuing a proper allocation of the probability mass in knowledge distillation viaα\\alpha\-β\\beta\-divergence\.InInternational Conference on Machine Learning,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.106.106.106.2),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.108.106.106.2),[§1](https://arxiv.org/html/2605.26246#S1.p2.1),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.66.66.66.2),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.68.66.66.2)\.
- P\. Wang, Z\. Zhao, H\. Wen, F\. Wang, B\. Wang, Q\. Zhang, and Y\. Wang \(2024b\)LLM\-AutoDA: large language model\-driven automatic data augmentation for long\-tailed problems\.InAdvances in Neural Information Processing Systems,pp\. 64915–64941\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- Y\. Wang, Y\. Kordi, S\. Mishra, A\. Liu, N\. A\. Smith, D\. Khashabi, and H\. Hajishirzi \(2023\)Self\-instruct: aligning language models with self\-generated instructions\.InAnnual Meeting of the Association for Computational Linguistics,pp\. 13484–13508\.Cited by:[§A\.2](https://arxiv.org/html/2605.26246#A1.SS2.p1.1),[§1](https://arxiv.org/html/2605.26246#S1.p2.1)\.
- Z\. Wang, Q\. Xu, Z\. Yang, Z\. Xu, L\. Zhang, X\. Cao, and Q\. Huang \(2026\)A unified perspective for loss\-oriented imbalanced learning via localization\.IEEE Transactions on Pattern Analysis and Machine Intelligence48\(1\),pp\. 639–656\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- Y\. Wen, Z\. Li, W\. Du, and L\. Mou \(2023\)F\-divergence minimization for sequence\-level knowledge distillation\.InAnnual Meeting of the Association for Computational Linguistics,pp\. 10817–10834\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.64.64.64.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.66.64.64.11),[§1](https://arxiv.org/html/2605.26246#S1.p2.1),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.24.24.24.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.26.24.24.11)\.
- T\. Wu, C\. Tao, J\. Wang, R\. Yang, Z\. Zhao, and N\. Wong \(2025\)Rethinking Kullback\-Leibler divergence in knowledge distillation for large language models\.InInternational Conference on Computational Linguistics,pp\. 5737–5755\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[Table 21](https://arxiv.org/html/2605.26246#A6.T21.84.84.84.11),[Table 22](https://arxiv.org/html/2605.26246#A6.T22.86.84.84.11),[Table 2](https://arxiv.org/html/2605.26246#S5.T2.44.44.44.11),[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p2.2),[Table 5](https://arxiv.org/html/2605.26246#S6.T5.46.44.44.11)\.
- W\. Xu, R\. Han, Z\. Wang, L\. Le, D\. Madeka, L\. Li, W\. Y\. Wang, R\. Agarwal, C\. Lee, and T\. Pfister \(2025\)Speculative knowledge distillation: bridging the teacher\-student gap through interleaved sampling\.InInternational Conference on Learning Representations,Cited by:[§A\.4](https://arxiv.org/html/2605.26246#A1.SS4.p2.1),[§1](https://arxiv.org/html/2605.26246#S1.p2.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§A\.4](https://arxiv.org/html/2605.26246#A1.SS4.p2.1),[§1](https://arxiv.org/html/2605.26246#S1.p1.1)\.
- A\. Yang, B\. Zhang, B\. Hui, B\. Gao, B\. Yu, C\. Li, D\. Liu, J\. Tu, J\. Zhou, J\. Lin, K\. Lu, M\. Xue, R\. Lin, T\. Liu, X\. Ren, and Z\. Zhang \(2024\)Qwen2\.5\-math technical report: toward mathematical expert model via self\-improvement\.arXiv preprint arXiv:2409\.12122\.Cited by:[§6\.1](https://arxiv.org/html/2605.26246#S6.SS1.p1.5)\.
- Z\. Yang, Q\. Xu, S\. Li, Z\. Wang, X\. Cao, and Q\. Huang \(2026\)DirMixE: harnessing test agnostic long\-tail recognition with hierarchical label vartiations\.IEEE Transactions on Pattern Analysis and Machine Intelligence48\(4\),pp\. 4605–4622\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- L\. Yu, W\. Jiang, H\. Shi, J\. Yu, Z\. Liu, Y\. Zhang, J\. T\. Kwok, Z\. Li, A\. Weller, and W\. Liu \(2023\)Metamath: bootstrap your own mathematical questions for large language models\.arXiv preprint arXiv:2309\.12284\.Cited by:[§E\.1](https://arxiv.org/html/2605.26246#A5.SS1.SSS0.Px2.p1.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, M\. Qiao, Y\. Wu, and M\. Wang \(2025\)DAPO: an open\-source llm reinforcement learning system at scale\.arXiv preprint arXiv:2503\.14476\.Cited by:[§E\.1](https://arxiv.org/html/2605.26246#A5.SS1.SSS0.Px2.p1.1)\.
- B\. Zhao, Q\. Cui, R\. Song, Y\. Qiu, and J\. Liang \(2022\)Decoupled knowledge distillation\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 11953–11962\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§3\.1](https://arxiv.org/html/2605.26246#S3.SS1.p1.1)\.
- Z\. Zhao, P\. Wang, H\. Wen, W\. Xu, L\. Song, Q\. Zhang, and Y\. Wang \(2024\)Two fists, one heart: multi\-objective optimization based strategy fusion for long\-tailed learning\.InInternational Conference on Machine Learning,pp\. 61040–61071\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- Z\. Zhao, H\. Wen, P\. Wang, S\. Wang, Z\. Wang, Q\. Zhang, and Y\. Wang \(2025\)Balancing model efficiency and performance: adaptive pruner for long\-tailed data\.InInternational Conference on Machine Learning,pp\. 77723–77739\.Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2)\.
- K\. Zheng and E\. YANG \(2024\)Knowledge distillation based on transformed teacher matching\.InInternational Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2605.26246#A1.SS1.p1.2),[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- T\. Zheng, G\. Zhang, T\. Shen, X\. Liu, B\. Y\. Lin, J\. Fu, W\. Chen, and X\. Yue \(2024\)Opencodeinterpreter: integrating code generation with execution and refinement\.arXiv preprint arXiv:2402\.14658\.Cited by:[§E\.1](https://arxiv.org/html/2605.26246#A5.SS1.SSS0.Px3.p1.1)\.
- Q\. Zhong, L\. Ding, L\. Shen, J\. Liu, B\. Du, and D\. Tao \(2024\)Revisiting knowledge distillation for autoregressive language models\.InAnnual Meeting of the Association for Computational Linguistics,pp\. 10900–10913\.Cited by:[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.
- H\. Zhou, L\. Song, J\. Chen, Y\. Zhou, G\. Wang, J\. Yuan, and Q\. Zhang \(2021\)Rethinking soft labels for knowledge distillation: a bias\-variance tradeoff perspective\.arXiv preprint arXiv:2102\.00650\.Cited by:[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.26246#A1.SS3.p2.1)\.

## Appendix

### Table of Contents

### Appendix ARelated Work

In this section, we review related work and discuss our contributions in the context of recent studies\.

#### A\.1Soft\-Label Knowledge Distillation

The seminal work by\(Hintonet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib54)\)first proposed leveraging the output distribution of a more powerful teacher model to supervise the training of a student model, minimizing the predictive discrepancy between them via Forward KL divergence\. This methodology achieved marked progress across diverse domains\(Liet al\.,[2023b](https://arxiv.org/html/2605.26246#bib.bib13); Wanget al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib27); Tianet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib7); Liet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib26); Wanget al\.,[2024a](https://arxiv.org/html/2605.26246#bib.bib83),[b](https://arxiv.org/html/2605.26246#bib.bib84); Zhaoet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib85),[2025](https://arxiv.org/html/2605.26246#bib.bib86); Yanget al\.,[2026](https://arxiv.org/html/2605.26246#bib.bib87); Wanget al\.,[2026](https://arxiv.org/html/2605.26246#bib.bib88)\)and established a foundational framework for extensive subsequent research\(Cho and Hariharan,[2019](https://arxiv.org/html/2605.26246#bib.bib16); Zhaoet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib15); Huanget al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib14); Liet al\.,[2023a](https://arxiv.org/html/2605.26246#bib.bib12); Beyeret al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib8); Chenet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib11); Haoet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib10); Zheng and YANG,[2024](https://arxiv.org/html/2605.26246#bib.bib9); Sunet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib53); Busbridgeet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib28); Harutyunyanet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib75); Penget al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib29)\)\. While highly effective, recent studies have noted that Forward KL divergence tends to induce mode\-seeking behavior, which can undermine the student’s focused learning of critical categories\(Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19); Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18); Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55); Wuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib70); Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59); Guet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib30)\)\. Therefore, some studies have turned to alternative distillation divergence objectives, such as Reverse KL divergence\(Leeet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib52); Kimet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib23); Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19)\), Maximum Mean Discrepancy\(Huang and Wang,[2017](https://arxiv.org/html/2605.26246#bib.bib25)\), Wasserstein Distance\(Lvet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib24)\), Total Variation distance\(Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55)\), Jensen\-Shannon divergence\(Biniciet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib51); Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73)\), Skew Forward/Reverse divergence\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72); Shinet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib58)\), andα\\alpha\-β\\beta\-divergence\(Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59)\)\. There are also approaches that optimize a weighted sum of multiple divergences during training\(Amaraet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib50); Wuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib70)\)\. These methods have achieved marked empirical progress by enabling the student to better match the overall teacher’s predictive distribution\.

Crucially, the technique we propose in this work is complementary to these promising methods\. As shown in Fig\.[3](https://arxiv.org/html/2605.26246#S5.F3), our hybrid supervision approach can be seamlessly integrated with various distillation divergences outlined above to achieve further performance gains\.

#### A\.2Hard\-Label Knowledge Distillation

Despite the success of soft\-label distillation, applying it to large language models can face practical constraints: top\-tier teacher models are often closed\-source \(e\.g\.,ChatGPT\(Achiamet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib68)\)\), and using them continuously during training remains prohibitively expensive\(Guet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib30)\)\. To overcome these barriers, hard\-label distillation\(Kim and Rush,[2016](https://arxiv.org/html/2605.26246#bib.bib20)\)uses the teacher model to generate corresponding output sequences for a set of input prompts, thereby constructing a labeled dataset on which the student is trained via standard maximum likelihood estimation\(Chianget al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib31); Wanget al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib65); Taoriet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib64); Penget al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib63); Wanget al\.,[2025a](https://arxiv.org/html/2605.26246#bib.bib82)\)\. In practice, this direct approach has yielded solid advances\(Guoet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib67)\)\.

In this work, we revisit the effect of hard labels in LLM distillation\. We start from the empirical observation that blending hard and soft labels improves performance, then formalize the cause through our Bridge–Garden framework\. Our theory indicates that using only hard or soft labels can result in suboptimal performance, whereas combining both can resolve this limitation\.

#### A\.3Hard\-Soft Hybrid Supervision in Knowledge Distillation

Several studies in image classification have also investigated strategies to combine hard and soft labels for student training\. However, a fundamental distinction exists regarding the nature of the hard labels\. Unlike in LLM distillation, where hard labels are typically generated by the teacher \(representing merely a subset of the teacher’s distribution, as discussed in Sec\.[3\.1](https://arxiv.org/html/2605.26246#S3.SS1)\)\(Kim and Rush,[2016](https://arxiv.org/html/2605.26246#bib.bib20)\), image classification tasks generally utilize ground\-truth labels from the training dataset\. In that context, hard labels provide accurate supervision signals even when the teacher makes incorrect predictions\. Despite this structural difference, prior research has still achieved significant progress by effectively combining hard and soft labels\(Zhouet al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib49); Zheng and YANG,[2024](https://arxiv.org/html/2605.26246#bib.bib9); Renet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib44); Hamet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib43)\)\.

For instance,\(Zhouet al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib49)\)decomposes the knowledge distillation objective via bias\-variance analysis and weights the hard and KL losses based on sample difficulty, ensuring the teacher provides better guidance for samples that are difficult for the student\.\(Zheng and YANG,[2024](https://arxiv.org/html/2605.26246#bib.bib9)\)proposes using teacher entropy as a weighting factor to focus on samples where the teacher’s predictions are more diverse\. Additionally,\(Liet al\.,[2023a](https://arxiv.org/html/2605.26246#bib.bib12); Jafariet al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib47)\)balances the contributions of hard and KL losses by scaling the KL loss with a curriculum\-based temperature coefficient\.\(Gangulyet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib37)\)uses a teacher\-loss\-based decay function to prioritize easy samples and progressively shift focus to harder instances\.\(Renet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib44)\)assigns higher loss weights to samples that yield greater gradient similarity between the student’s training update and the validation set performance, ensuring the teacher prioritizes teaching what helps the student generalize\.\(Hamet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib43)\)employs soft\-label distillation on refusal responses to smooth the loss surface and hard\-label supervision on filtered benign samples to mitigate gradient conflicts between safety and downstream tasks\. Recent token\-adaptive methods for autoregressive LMs also adjust the teaching mode or divergence at each token\(Zhonget al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib38); Junget al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib39)\)\. Our work studies a different axis of adaptation: mixing the teacher\-sampled hard token with the teacher’s full next\-token distribution\.

Despite these successful and mature developments in image classification,the application of hybrid supervision in LLM distillation remains largely unexplored, and theoretical analysis explaining its effectiveness is still lacking\. In this work, we formally demonstrate that this approach can be effectively applied to LLM distillation\. More importantly, our Bridge\-Garden framework reveals that its efficacy stems from better mitigating the complex issue of exposure bias\.

#### A\.4Exposure Bias in Knowledge Distillation

A key challenge in distilling autoregressive models is exposure bias\(Bengioet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib62)\), which arises from a fundamental distribution mismatch: during training, the student learns from sequences produced by the teacher, yet at inference time it must generate from its own previous outputs\. This discrepancy can cause significant degradation in performance\.

To address this, on\-policy distillation methods\(Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73); Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19); Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18); Rossetet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib22)\)train the student on its own generated outputs, with the teacher providing corrective feedback\. This directly aligns the training and inference distributions and yields effective improvements in practice\(Yanget al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib69)\)\.\(Xuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib57)\)further leverages speculative decoding to use the teacher’s distribution to correct the noise introduced by low\-quality early student generations\.

Although prior work has proposed various heuristics to mitigate exposure bias, a systematic and principled theoretical understanding of exposure bias in KD has remained largely absent\. We fill this gap by introducing the Bridge–Garden framework \(Theorem[4\.3](https://arxiv.org/html/2605.26246#S4.SS3)\) to characterize how exposure bias arises\. Grounded in this theory, we develop a family of principled distillation methods for reducing exposure bias effectively\. Moreover, a key advantage of our approach is its efficiency: it requires no expensive sampling from the student model during training\. This makes it particularly suitable for large\-scale industrial applications\.

### Appendix BAκ\\kappa\-Weighted Bound on Exposure Bias \(Proof of Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\)

This section proves Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2), showing that the exposure bias𝖤𝖡\(πθ\)\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)is upper bounded by adTd\_\{T\}\-expectation of a pointwiseκ\\kappa\-weighted deviation term plus a quadratic term\.

###### Setup\.

We analyze length\-TTautoregressive generation \(variable\-length can be handled by standard padding and masking\)\. For any policyπ\\pi, letξ=\(s1,a1,…,sT,aT\)\\xi=\(s\_\{1\},a\_\{1\},\\ldots,s\_\{T\},a\_\{T\}\)denote the induced random autoregressive trajectory, wherest=\(x,y<t\)s\_\{t\}=\(x,y\_\{<t\}\)andat∈𝒱a\_\{t\}\\in\\mathcal\{V\}, and write𝔼π\[⋅\]\\mathbb\{E\}\_\{\\pi\}\[\\cdot\]for expectation with respect to this randomness\. Letdπd\_\{\\pi\}denote the induced prefix distribution,i\.e\.,the distribution ofsts\_\{t\}whenttis chosen uniformly from\{1,…,T\}\\\{1,\\dots,T\\\}\. We will use the identity: for any measurableff,

𝔼s∼dπ\[f\(s\)\]=1T∑t=1T𝔼π\[f\(st\)\]\.\\mathbb\{E\}\_\{s\\sim d\_\{\\pi\}\}\[f\(s\)\]=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{E\}\_\{\\pi\}\[f\(s\_\{t\}\)\]\.\(6\)
Recall

ℓsoft\(s;θ\)≔𝔻\(πT\(⋅∣s\)∥πθ\(⋅∣s\)\),ℒd\(πθ\)≔𝔼s∼d\[ℓsoft\(s;θ\)\],\\ell\_\{\\mathrm\{soft\}\}\(s;\\theta\)\\coloneqq\\mathbb\{D\}\\\!\\big\(\\pi\_\{T\}\(\\cdot\\mid s\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\big\),\\qquad\\mathcal\{L\}\_\{d\}\(\\pi\_\{\\theta\}\)\\coloneqq\\mathbb\{E\}\_\{s\\sim d\}\\big\[\\ell\_\{\\mathrm\{soft\}\}\(s;\\theta\)\\big\],and the exposure bias \(Def\.[3\.2](https://arxiv.org/html/2605.26246#S3.SS2)\)

𝖤𝖡\(πθ\)=ℒdθ\(πθ\)−ℒdT\(πθ\)\.\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)=\\mathcal\{L\}\_\{d\_\{\\theta\}\}\(\\pi\_\{\\theta\}\)\-\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)\.We use the following standard conditions \(only to control the quadratic term\):

1. 1\.\(*bounded loss*\)0≤ℓsoft\(s;θ\)≤Lmax0\\leq\\ell\_\{\\mathrm\{soft\}\}\(s;\\theta\)\\leq L\_\{\\max\}for all relevant prefixesss;
2. 2\.\(*minimum probability along the interpolation*\) forπu≔\(1−u\)πT\+uπθ\\pi\_\{u\}\\coloneqq\(1\-u\)\\pi\_\{T\}\+u\\pi\_\{\\theta\}and allu∈\[0,1\]u\\in\[0,1\],πu\(a∣s\)≥β\\pi\_\{u\}\(a\\mid s\)\\geq\\betafor alla∈𝒱a\\in\\mathcal\{V\}and relevant prefixesss;
3. 3\.\(*prefix concentrability*\) there existsCconc≥1C\_\{\\mathrm\{conc\}\}\\geq 1such thatdπu\(s\)≤CconcdT\(s\)d\_\{\\pi\_\{u\}\}\(s\)\\leq C\_\{\\mathrm\{conc\}\}\\,d\_\{T\}\(s\)for allu∈\[0,1\]u\\in\[0,1\]and prefixesss\.

The first two conditions are regularity conditions for finite\-vocabulary softmax models: bounded logits give finite losses and nonzero token probabilities on the prefixes under consideration\. The third condition is the usual overlap requirement used when an off\-policy distribution is used to reason about another rollout distribution\. Here it requires the teacher\-prefix distribution to cover the prefixes reached along the teacher–student interpolation\. This is the relevant regime for distillation, where teacher and student share the tokenizer and are trained to keep the student close to the teacher\.

Restate of Theorem[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)Under mild conditions \(bounded loss and non\-vanishing probabilities\), letΔπθ≔πθ−πT\\Delta\\pi\_\{\\theta\}\\coloneqq\\pi\_\{\\theta\}\-\\pi\_\{T\}\. Then the exposure bias𝖤𝖡\(πθ\)\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)satisfies

𝖤𝖡\(πθ\)≤𝔼s∼dT\[F\(s,πθ\)\],\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\\ \\leq\\ \\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\Big\[\\,F\(s,\\pi\_\{\\theta\}\)\\Big\],where for a constantC2\>0C\_\{\\rm 2\}\>0,

F\(s,πθ\):=∑aκ\(a∣s\)⋅\|Δπθ\(a∣s\)\|⏟κ\(a\|s\)\-weighted deviation\+C2∥Δπθ\(⋅∣s\)∥12\.F\(s,\\pi\_\{\\theta\}\):=\\underbrace\{\\sum\\nolimits\_\{a\}\{\\color\[rgb\]\{0,0,1\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,1\}\\kappa\(a\\\!\\mid\\\!s\)\}\\cdot\|\\Delta\\pi\_\{\\theta\}\(a\\\!\\mid\\\!s\)\|\}\_\{\\textbf\{$\\kappa\(a\|s\)$\-weighted deviation\}\}\\;\+C\_\{\\rm 2\}\\\|\\Delta\\pi\_\{\\theta\}\(\\cdot\\\!\\mid\\\!s\)\\\|\_\{1\}^\{2\}\.
###### Proof\.

The proof proceeds in several steps:

Step 1: A one\-dimensional path identity\.Foru∈\[0,1\]u\\in\[0,1\], define the linear interpolation

πu≔\(1−u\)πT\+uπθ,g\(u\)≔𝖤𝖡\(πu\)=ℒdπu\(πθ\)−ℒdT\(πθ\)\.\\pi\_\{u\}\\coloneqq\(1\-u\)\\pi\_\{T\}\+u\\pi\_\{\\theta\},\\qquad g\(u\)\\coloneqq\\mathsf\{EB\}\(\\pi\_\{u\}\)=\\mathcal\{L\}\_\{d\_\{\\pi\_\{u\}\}\}\(\\pi\_\{\\theta\}\)\-\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)\.By construction,g\(0\)=𝖤𝖡\(πT\)=0g\(0\)=\\mathsf\{EB\}\(\\pi\_\{T\}\)=0andg\(1\)=𝖤𝖡\(πθ\)g\(1\)=\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\. Under the above regularity conditions,ggis twice differentiable on\[0,1\]\[0,1\]\. Applying the fundamental theorem of calculus twice yields

g\(1\)=g\(0\)\+∫01g′\(u\)𝑑u=g\(0\)\+g′\(0\)\+∫01∫0ug′′\(t\)𝑑t𝑑u\.g\(1\)=g\(0\)\+\\int\_\{0\}^\{1\}g^\{\\prime\}\(u\)\\,du=g\(0\)\+g^\{\\prime\}\(0\)\+\\int\_\{0\}^\{1\}\\int\_\{0\}^\{u\}g^\{\\prime\\prime\}\(t\)\\,dt\\,du\.Sinceg\(0\)=0g\(0\)=0, it remains to simplify the double integral\. By swapping the order of integration,

∫01∫0ug′′\(t\)𝑑t𝑑u=∫01\(∫t1𝑑u\)g′′\(t\)𝑑t=∫01\(1−t\)g′′\(t\)𝑑t\.\\int\_\{0\}^\{1\}\\int\_\{0\}^\{u\}g^\{\\prime\\prime\}\(t\)\\,dt\\,du=\\int\_\{0\}^\{1\}\\left\(\\int\_\{t\}^\{1\}du\\right\)g^\{\\prime\\prime\}\(t\)\\,dt=\\int\_\{0\}^\{1\}\(1\-t\)\\,g^\{\\prime\\prime\}\(t\)\\,dt\.Renamingttasuugives the identity

𝖤𝖡\(πθ\)=g′\(0\)\+∫01\(1−u\)g′′\(u\)𝑑u,\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)=g^\{\\prime\}\(0\)\+\\int\_\{0\}^\{1\}\(1\-u\)\\,g^\{\\prime\\prime\}\(u\)\\,du,\(7\)\.

###### Step 2: Rewriteg\(u\)g\(u\)using a trajectory functional\.

Define

A\(ξ\)≔∑t=1Tℓsoft\(st;θ\),J\(u\)≔𝔼πu\[A\(ξ\)\]\.A\(\\xi\)\\coloneqq\\sum\_\{t=1\}^\{T\}\\ell\_\{\\mathrm\{soft\}\}\(s\_\{t\};\\theta\),\\qquad J\(u\)\\coloneqq\\mathbb\{E\}\_\{\\pi\_\{u\}\}\[A\(\\xi\)\]\.By Eq\.\([6](https://arxiv.org/html/2605.26246#A2.E6)\),

ℒdπu\(πθ\)=𝔼s∼dπu\[ℓsoft\(s;θ\)\]=1T𝔼πu\[∑t=1Tℓsoft\(st;θ\)\]=1TJ\(u\),\\mathcal\{L\}\_\{d\_\{\\pi\_\{u\}\}\}\(\\pi\_\{\\theta\}\)=\\mathbb\{E\}\_\{s\\sim d\_\{\\pi\_\{u\}\}\}\\\!\\big\[\\ell\_\{\\mathrm\{soft\}\}\(s;\\theta\)\\big\]=\\frac\{1\}\{T\}\\mathbb\{E\}\_\{\\pi\_\{u\}\}\\Big\[\\sum\_\{t=1\}^\{T\}\\ell\_\{\\mathrm\{soft\}\}\(s\_\{t\};\\theta\)\\Big\]=\\frac\{1\}\{T\}J\(u\),and similarlyℒdT\(πθ\)=1TJ\(0\)\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)=\\frac\{1\}\{T\}J\(0\)\. Therefore,

g\(u\)=1T\(J\(u\)−J\(0\)\),g′\(u\)=1TJ′\(u\),g′′\(u\)=1TJ′′\(u\)\.g\(u\)=\\frac\{1\}\{T\}\\big\(J\(u\)\-J\(0\)\\big\),\\qquad g^\{\\prime\}\(u\)=\\frac\{1\}\{T\}J^\{\\prime\}\(u\),\\qquad g^\{\\prime\\prime\}\(u\)=\\frac\{1\}\{T\}J^\{\\prime\\prime\}\(u\)\.\(8\)

###### Step 3: Computeg′\(0\)g^\{\\prime\}\(0\)and express it usingκ\(a∣s\)\\kappa\(a\\mid s\)\.

By Eq\.\([8](https://arxiv.org/html/2605.26246#A2.E8)\), it suffices to computeJ′\(0\)J^\{\\prime\}\(0\)\.

*\(3\.1\) Score\-function derivative ofJ\(u\)J\(u\)\.*LetΔπθ≔πθ−πT\\Delta\\pi\_\{\\theta\}\\coloneqq\\pi\_\{\\theta\}\-\\pi\_\{T\}\. Letπu\(ξ\)\\pi\_\{u\}\(\\xi\)denote the probability mass of trajectoryξ\\xiinduced byπu\\pi\_\{u\}\. By autoregressive factorization,

log⁡πu\(ξ\)=∑t=1Tlog⁡πu\(at∣st\),ddulog⁡πu\(ξ\)=∑t=1TΔπθ\(at∣st\)πu\(at∣st\)\.\\log\\pi\_\{u\}\(\\xi\)=\\sum\_\{t=1\}^\{T\}\\log\\pi\_\{u\}\(a\_\{t\}\\mid s\_\{t\}\),\\qquad\\frac\{d\}\{du\}\\log\\pi\_\{u\}\(\\xi\)=\\sum\_\{t=1\}^\{T\}\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{u\}\(a\_\{t\}\\mid s\_\{t\}\)\}\.SinceA\(ξ\)A\(\\xi\)does not depend onuu, the score\-function identity yields

J′\(u\)=𝔼πu\[A\(ξ\)∑t=1TΔπθ\(at∣st\)πu\(at∣st\)\]\.J^\{\\prime\}\(u\)=\\mathbb\{E\}\_\{\\pi\_\{u\}\}\\\!\\Bigg\[A\(\\xi\)\\sum\_\{t=1\}^\{T\}\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{u\}\(a\_\{t\}\\mid s\_\{t\}\)\}\\Bigg\]\.Evaluating atu=0u=0gives

J′\(0\)=𝔼πT\[A\(ξ\)∑t=1TΔπθ\(at∣st\)πT\(at∣st\)\],g′\(0\)=1TJ′\(0\)\.J^\{\\prime\}\(0\)=\\mathbb\{E\}\_\{\\pi\_\{T\}\}\\\!\\Bigg\[A\(\\xi\)\\sum\_\{t=1\}^\{T\}\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{T\}\(a\_\{t\}\\mid s\_\{t\}\)\}\\Bigg\],\\qquad g^\{\\prime\}\(0\)=\\frac\{1\}\{T\}J^\{\\prime\}\(0\)\.\(9\)
*\(3\.2\) Regroup by prefixes\.*For each prefixssand tokena∈𝒱a\\in\\mathcal\{V\}, define

Q\(s,a\)≔𝔼πT\[A\(ξ\)∣st=s,at=a\],Q¯\(s\)≔∑a∈𝒱πT\(a∣s\)Q\(s,a\)\.Q\(s,a\)\\coloneqq\\mathbb\{E\}\_\{\\pi\_\{T\}\}\\\!\\big\[A\(\\xi\)\\mid s\_\{t\}=s,\\,a\_\{t\}=a\\big\],\\qquad\\bar\{Q\}\(s\)\\coloneqq\\sum\_\{a\\in\\mathcal\{V\}\}\\pi\_\{T\}\(a\\mid s\)\\,Q\(s,a\)\.Conditioning onst=ss\_\{t\}=sand usingat∼πT\(⋅∣s\)a\_\{t\}\\sim\\pi\_\{T\}\(\\cdot\\mid s\), we have

𝔼πT\[A\(ξ\)Δπθ\(at∣st\)πT\(at∣st\)\|st=s\]=∑a∈𝒱Δπθ\(a∣s\)Q\(s,a\)\.\\mathbb\{E\}\_\{\\pi\_\{T\}\}\\\!\\Bigg\[A\(\\xi\)\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{T\}\(a\_\{t\}\\mid s\_\{t\}\)\}\\ \\Bigg\|\\ s\_\{t\}=s\\Bigg\]=\\sum\_\{a\\in\\mathcal\{V\}\}\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\\,Q\(s,a\)\.Plugging this into Eq\.\([9](https://arxiv.org/html/2605.26246#A2.E9)\), summing overtt, and using Eq\.\([6](https://arxiv.org/html/2605.26246#A2.E6)\) yields

g′\(0\)=𝔼s∼dT\[∑a∈𝒱Δπθ\(a∣s\)Q\(s,a\)\]\.g^\{\\prime\}\(0\)=\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\Big\[\\sum\_\{a\\in\\mathcal\{V\}\}\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\\,Q\(s,a\)\\Big\]\.\(10\)Moreover, since∑a∈𝒱Δπθ\(a∣s\)=0\\sum\_\{a\\in\\mathcal\{V\}\}\\Delta\\pi\_\{\\theta\}\(a\\mid s\)=0, we can centerQQ:

∑a∈𝒱Δπθ\(a∣s\)Q\(s,a\)=∑a∈𝒱Δπθ\(a∣s\)\(Q\(s,a\)−Q¯\(s\)\)\.\\sum\_\{a\\in\\mathcal\{V\}\}\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\\,Q\(s,a\)=\\sum\_\{a\\in\\mathcal\{V\}\}\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\\,\\big\(Q\(s,a\)\-\\bar\{Q\}\(s\)\\big\)\.\(11\)
*\(3\.3\) IdentifyQ\(s,a\)−Q¯\(s\)Q\(s,a\)\-\\bar\{Q\}\(s\)withκ\(a∣s\)\\kappa\(a\\mid s\)\.*Fix\(s,a\)\(s,a\)withdT\(s\)\>0d\_\{T\}\(s\)\>0and consider the single\-override policyπ\(s,a\)\\pi^\{\(s,a\)\}\(Def\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\)\. It coincides withπT\\pi\_\{T\}except that it deterministically outputsaaat prefixss\. Thus, conditioned onst=ss\_\{t\}=s, the continuation underπ\(s,a\)\\pi^\{\(s,a\)\}matches the teacher continuation conditioned onat=aa\_\{t\}=a, so

𝔼π\(s,a\)\[A\(ξ\)∣st=s\]=Q\(s,a\),𝔼πT\[A\(ξ\)∣st=s\]=Q¯\(s\)\.\\mathbb\{E\}\_\{\\pi^\{\(s,a\)\}\}\[A\(\\xi\)\\mid s\_\{t\}=s\]=Q\(s,a\),\\qquad\\mathbb\{E\}\_\{\\pi\_\{T\}\}\[A\(\\xi\)\\mid s\_\{t\}=s\]=\\bar\{Q\}\(s\)\.The visitation frequency of prefixssis unchanged by the override before reachingss, hence the difference in total loss satisfies

𝔼π\(s,a\)\[A\(ξ\)\]−𝔼πT\[A\(ξ\)\]=TdT\(s\)\(Q\(s,a\)−Q¯\(s\)\)\.\\mathbb\{E\}\_\{\\pi^\{\(s,a\)\}\}\[A\(\\xi\)\]\-\\mathbb\{E\}\_\{\\pi\_\{T\}\}\[A\(\\xi\)\]=T\\,d\_\{T\}\(s\)\\,\\big\(Q\(s,a\)\-\\bar\{Q\}\(s\)\\big\)\.Dividing byTTand using𝖤𝖡\(π\)=ℒdπ\(πθ\)−ℒdT\(πθ\)=1T\(𝔼π\[A\(ξ\)\]−𝔼πT\[A\(ξ\)\]\)\\mathsf\{EB\}\(\\pi\)=\\mathcal\{L\}\_\{d\_\{\\pi\}\}\(\\pi\_\{\\theta\}\)\-\\mathcal\{L\}\_\{d\_\{T\}\}\(\\pi\_\{\\theta\}\)=\\frac\{1\}\{T\}\\big\(\\mathbb\{E\}\_\{\\pi\}\[A\(\\xi\)\]\-\\mathbb\{E\}\_\{\\pi\_\{T\}\}\[A\(\\xi\)\]\\big\)gives

𝖤𝖡\(π\(s,a\)\)=dT\(s\)\(Q\(s,a\)−Q¯\(s\)\)\.\\mathsf\{EB\}\(\\pi^\{\(s,a\)\}\)=d\_\{T\}\(s\)\\,\\big\(Q\(s,a\)\-\\bar\{Q\}\(s\)\\big\)\.Therefore, by Def\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2),

κ\(a∣s\)=𝖤𝖡\(π\(s,a\)\)dT\(s\)=Q\(s,a\)−Q¯\(s\)\.\\kappa\(a\\mid s\)=\\frac\{\\mathsf\{EB\}\(\\pi^\{\(s,a\)\}\)\}\{d\_\{T\}\(s\)\}=Q\(s,a\)\-\\bar\{Q\}\(s\)\.Substituting into Eq\.\([10](https://arxiv.org/html/2605.26246#A2.E10)\)–\([11](https://arxiv.org/html/2605.26246#A2.E11)\) yields

g′\(0\)=𝔼s∼dT\[∑a∈𝒱κ\(a∣s\)Δπθ\(a∣s\)\]\.g^\{\\prime\}\(0\)=\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\Big\[\\sum\_\{a\\in\\mathcal\{V\}\}\\kappa\(a\\mid s\)\\,\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\\Big\]\.\(12\)

###### Step 4: Upper bound the first\-order term\.

Recall that we analyze the typical regime where student deviations from the teacher exacerbate exposure bias, so𝖤𝖡\(π\(s,a\)\)≥0\\mathsf\{EB\}\(\\pi^\{\(s,a\)\}\)\\geq 0and the associated sensitivity weight satisfiesκ\(a∣s\)≥0\\kappa\(a\\mid s\)\\geq 0\. Using the triangle inequality, we obtain \([12](https://arxiv.org/html/2605.26246#A2.E12)\),

g′\(0\)≤𝔼s∼dT\[∑a∈𝒱κ\(a∣s\)\|Δπθ\(a∣s\)\|\]\.g^\{\\prime\}\(0\)\\leq\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\Big\[\\sum\_\{a\\in\\mathcal\{V\}\}\\kappa\(a\\mid s\)\\,\|\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\|\\Big\]\.\(13\)

###### Step 5: Bound the remainder by a quadratic term\.

From Eq\.\([8](https://arxiv.org/html/2605.26246#A2.E8)\) and \(3\.1\), define

Hu\(ξ\)≔∑t=1TΔπθ\(at∣st\)πu\(at∣st\),so thatJ′\(u\)=𝔼πu\[A\(ξ\)Hu\(ξ\)\]\.H\_\{u\}\(\\xi\)\\coloneqq\\sum\_\{t=1\}^\{T\}\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{u\}\(a\_\{t\}\\mid s\_\{t\}\)\},\\qquad\\text\{so that\}\\qquad J^\{\\prime\}\(u\)=\\mathbb\{E\}\_\{\\pi\_\{u\}\}\\big\[A\(\\xi\)\\,H\_\{u\}\(\\xi\)\\big\]\.Differentiating once more gives

J′′\(u\)=𝔼πu\[A\(ξ\)\(Hu\(ξ\)2\+∂uHu\(ξ\)\)\]\.J^\{\\prime\\prime\}\(u\)=\\mathbb\{E\}\_\{\\pi\_\{u\}\}\\\!\\Big\[A\(\\xi\)\\big\(H\_\{u\}\(\\xi\)^\{2\}\+\\partial\_\{u\}H\_\{u\}\(\\xi\)\\big\)\\Big\]\.Since∂u\(Δπθ\(at∣st\)πu\(at∣st\)\)=−\(Δπθ\(at∣st\)πu\(at∣st\)\)2\\partial\_\{u\}\\\!\\left\(\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{u\}\(a\_\{t\}\\mid s\_\{t\}\)\}\\right\)=\-\\left\(\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{u\}\(a\_\{t\}\\mid s\_\{t\}\)\}\\right\)^\{2\}, we obtain

Hu\(ξ\)2\+∂uHu\(ξ\)=∑t1≠t2Δπθ\(at1∣st1\)πu\(at1∣st1\)⋅Δπθ\(at2∣st2\)πu\(at2∣st2\)\.H\_\{u\}\(\\xi\)^\{2\}\+\\partial\_\{u\}H\_\{u\}\(\\xi\)=\\sum\_\{t\_\{1\}\\neq t\_\{2\}\}\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\_\{1\}\}\\mid s\_\{t\_\{1\}\}\)\}\{\\pi\_\{u\}\(a\_\{t\_\{1\}\}\\mid s\_\{t\_\{1\}\}\)\}\\cdot\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\_\{2\}\}\\mid s\_\{t\_\{2\}\}\)\}\{\\pi\_\{u\}\(a\_\{t\_\{2\}\}\\mid s\_\{t\_\{2\}\}\)\}\.Therefore, usingg′′\(u\)=1TJ′′\(u\)g^\{\\prime\\prime\}\(u\)=\\frac\{1\}\{T\}J^\{\\prime\\prime\}\(u\),

g′′\(u\)=1T𝔼πu\[A\(ξ\)∑t1≠t2Δπθ\(at1∣st1\)πu\(at1∣st1\)⋅Δπθ\(at2∣st2\)πu\(at2∣st2\)\]\.g^\{\\prime\\prime\}\(u\)=\\frac\{1\}\{T\}\\mathbb\{E\}\_\{\\pi\_\{u\}\}\\\!\\Bigg\[A\(\\xi\)\\sum\_\{t\_\{1\}\\neq t\_\{2\}\}\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\_\{1\}\}\\mid s\_\{t\_\{1\}\}\)\}\{\\pi\_\{u\}\(a\_\{t\_\{1\}\}\\mid s\_\{t\_\{1\}\}\)\}\\cdot\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\_\{2\}\}\\mid s\_\{t\_\{2\}\}\)\}\{\\pi\_\{u\}\(a\_\{t\_\{2\}\}\\mid s\_\{t\_\{2\}\}\)\}\\Bigg\]\.\(14\)Letr\(s\)≔∥Δπθ\(⋅∣s\)∥1r\(s\)\\coloneqq\\\|\\Delta\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\\|\_\{1\}\. By Assumption \(ii\),πu\(a∣s\)≥β\\pi\_\{u\}\(a\\mid s\)\\geq\\beta, hence

\|Δπθ\(at∣st\)πu\(at∣st\)\|≤\|Δπθ\(at∣st\)\|β≤r\(st\)β\.\\Big\|\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{u\}\(a\_\{t\}\\mid s\_\{t\}\)\}\\Big\|\\leq\\frac\{\|\\Delta\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\|\}\{\\beta\}\\leq\\frac\{r\(s\_\{t\}\)\}\{\\beta\}\.Thus

\|∑t1≠t2Δπθ\(at1∣st1\)πu\(at1∣st1\)⋅Δπθ\(at2∣st2\)πu\(at2∣st2\)\|≤1β2∑t1≠t2r\(st1\)r\(st2\)≤T−1β2∑t=1Tr\(st\)2,\\Bigg\|\\sum\_\{t\_\{1\}\\neq t\_\{2\}\}\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\_\{1\}\}\\mid s\_\{t\_\{1\}\}\)\}\{\\pi\_\{u\}\(a\_\{t\_\{1\}\}\\mid s\_\{t\_\{1\}\}\)\}\\cdot\\frac\{\\Delta\\pi\_\{\\theta\}\(a\_\{t\_\{2\}\}\\mid s\_\{t\_\{2\}\}\)\}\{\\pi\_\{u\}\(a\_\{t\_\{2\}\}\\mid s\_\{t\_\{2\}\}\)\}\\Bigg\|\\leq\\frac\{1\}\{\\beta^\{2\}\}\\sum\_\{t\_\{1\}\\neq t\_\{2\}\}r\(s\_\{t\_\{1\}\}\)r\(s\_\{t\_\{2\}\}\)\\leq\\frac\{T\-1\}\{\\beta^\{2\}\}\\sum\_\{t=1\}^\{T\}r\(s\_\{t\}\)^\{2\},where we used∑i≠jbibj≤\(T−1\)∑ibi2\\sum\_\{i\\neq j\}b\_\{i\}b\_\{j\}\\leq\(T\-1\)\\sum\_\{i\}b\_\{i\}^\{2\}forbi≥0b\_\{i\}\\geq 0\. Also0≤A\(ξ\)≤TLmax0\\leq A\(\\xi\)\\leq TL\_\{\\max\}by Assumption \(i\)\. Plugging these into Eq\.\([14](https://arxiv.org/html/2605.26246#A2.E14)\) yields

\|g′′\(u\)\|≤1T⋅\(TLmax\)⋅T−1β2𝔼πu\[∑t=1Tr\(st\)2\]=Lmaxβ2T\(T−1\)𝔼s∼dπu\[r\(s\)2\],\|g^\{\\prime\\prime\}\(u\)\|\\leq\\frac\{1\}\{T\}\\cdot\(TL\_\{\\max\}\)\\cdot\\frac\{T\-1\}\{\\beta^\{2\}\}\\,\\mathbb\{E\}\_\{\\pi\_\{u\}\}\\Big\[\\sum\_\{t=1\}^\{T\}r\(s\_\{t\}\)^\{2\}\\Big\]=\\frac\{L\_\{\\max\}\}\{\\beta^\{2\}\}\\,T\(T\-1\)\\,\\mathbb\{E\}\_\{s\\sim d\_\{\\pi\_\{u\}\}\}\\big\[r\(s\)^\{2\}\\big\],where the last equality follows from Eq\.\([6](https://arxiv.org/html/2605.26246#A2.E6)\)\. By Assumption \(iii\),𝔼s∼dπu\[r\(s\)2\]≤Cconc𝔼s∼dT\[r\(s\)2\]\\mathbb\{E\}\_\{s\\sim d\_\{\\pi\_\{u\}\}\}\[r\(s\)^\{2\}\]\\leq C\_\{\\mathrm\{conc\}\}\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\[r\(s\)^\{2\}\]\. Therefore,

supu∈\[0,1\]\|g′′\(u\)\|≤Lmaxβ2CconcT\(T−1\)𝔼s∼dT\[∥Δπθ\(⋅∣s\)∥12\]\.\\sup\_\{u\\in\[0,1\]\}\|g^\{\\prime\\prime\}\(u\)\|\\leq\\frac\{L\_\{\\max\}\}\{\\beta^\{2\}\}\\,C\_\{\\mathrm\{conc\}\}\\,T\(T\-1\)\\,\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[\\\|\\Delta\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\big\]\.Finally, by Eq\.\([7](https://arxiv.org/html/2605.26246#A2.E7)\),

\|∫01\(1−u\)g′′\(u\)du\|≤12supu∈\[0,1\]\|g′′\(u\)\|≤C2⋅𝔼s∼dT\[∥Δπθ\(⋅∣s\)∥12\],\\Big\|\\int\_\{0\}^\{1\}\(1\-u\)g^\{\\prime\\prime\}\(u\)\\,du\\Big\|\\leq\\frac\{1\}\{2\}\\sup\_\{u\\in\[0,1\]\}\|g^\{\\prime\\prime\}\(u\)\|\\leq C\_\{\\rm 2\}\\cdot\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[\\\|\\Delta\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\big\],with

C2≔Lmax2β2CconcT\(T−1\)\.C\_\{\\rm 2\}\\coloneqq\\frac\{L\_\{\\max\}\}\{2\\beta^\{2\}\}\\,C\_\{\\mathrm\{conc\}\}\\,T\(T\-1\)\.\(15\)

###### Step 6: Conclude\.

Combining Eq\.\([7](https://arxiv.org/html/2605.26246#A2.E7)\), Eq\.\([13](https://arxiv.org/html/2605.26246#A2.E13)\), and Step 5 yields

𝖤𝖡\(πθ\)≤𝔼s∼dT\[∑a∈𝒱κ\(a∣s\)\|Δπθ\(a∣s\)\|\+C2∥Δπθ\(⋅∣s\)∥12\]\.\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\\leq\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\Big\[\\sum\_\{a\\in\\mathcal\{V\}\}\\kappa\(a\\mid s\)\\,\|\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\|\\;\+\\;C\_\{\\rm 2\}\\,\\\|\\Delta\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\Big\]\.Defining

F\(s,πθ\)≔∑a∈𝒱κ\(a∣s\)\|Δπθ\(a∣s\)\|\+C2∥Δπθ\(⋅∣s\)∥12,F\(s,\\pi\_\{\\theta\}\)\\coloneqq\\sum\_\{a\\in\\mathcal\{V\}\}\\kappa\(a\\mid s\)\\,\|\\Delta\\pi\_\{\\theta\}\(a\\mid s\)\|\\;\+\\;C\_\{\\rm 2\}\\,\\\|\\Delta\\pi\_\{\\theta\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\},we obtain𝖤𝖡\(πθ\)≤𝔼s∼dT\[F\(s,πθ\)\]\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\\leq\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\[F\(s,\\pi\_\{\\theta\}\)\], which is exactly Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\. ∎

### Appendix CThe Bridge–Garden Upper Bound on Exposure Bias \(Proof of Prop\.[4\.3](https://arxiv.org/html/2605.26246#S4.SS3)\)

Restate of Proposition[4\.3](https://arxiv.org/html/2605.26246#S4.SS3)\.Using the partition from Def\.[4\.3](https://arxiv.org/html/2605.26246#S4.SS3), the bound from Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)decomposes as follows:

F\(πθ\):=Fℬ\(πθ\)\+F𝒢\(πθ\),F\(\\pi\_\{\\theta\}\):=F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\theta\}\)\+F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\theta\}\),whereF𝒳\(πθ\):=𝔼s∼dT\[1𝒳\(s\)F\(s,πθ\)\]F\_\{\\mathcal\{X\}\}\(\\pi\_\{\\theta\}\):=\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\[1\_\{\\mathcal\{X\}\}\(s\)F\(s,\\pi\_\{\\theta\}\)\]for𝒳∈\{ℬ,𝒢\}\\mathcal\{X\}\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\}, and the termF\(s,πθ\)F\(s,\\pi\_\{\\theta\}\)is defined in Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\.

###### Proof\.

By Def\.[4\.3](https://arxiv.org/html/2605.26246#S4.SS3),ℬ\\mathcal\{B\}and𝒢\\mathcal\{G\}form a partition of the prefix space𝒮\\mathcal\{S\}\. Hence, for every prefixss,

1ℬ\(s\)\+1𝒢\(s\)=1\.1\_\{\\mathcal\{B\}\}\(s\)\+1\_\{\\mathcal\{G\}\}\(s\)=1\.\(16\)
Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)gives

𝖤𝖡\(πθ\)≤𝔼s∼dT\[F\(s,πθ\)\]\.\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\\ \\leq\\ \\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[F\(s,\\pi\_\{\\theta\}\)\\big\]\.\(17\)Insert Eq\.\([16](https://arxiv.org/html/2605.26246#A3.E16)\) into the integrand:

F\(s,πθ\)=\(1ℬ\(s\)\+1𝒢\(s\)\)F\(s,πθ\)=1ℬ\(s\)F\(s,πθ\)\+1𝒢\(s\)F\(s,πθ\)\.F\(s,\\pi\_\{\\theta\}\)=\\big\(1\_\{\\mathcal\{B\}\}\(s\)\+1\_\{\\mathcal\{G\}\}\(s\)\\big\)F\(s,\\pi\_\{\\theta\}\)=1\_\{\\mathcal\{B\}\}\(s\)F\(s,\\pi\_\{\\theta\}\)\+1\_\{\\mathcal\{G\}\}\(s\)F\(s,\\pi\_\{\\theta\}\)\.Taking expectation underdTd\_\{T\}and using linearity,

𝔼s∼dT\[F\(s,πθ\)\]=𝔼s∼dT\[1ℬ\(s\)F\(s,πθ\)\]\+𝔼s∼dT\[1𝒢\(s\)F\(s,πθ\)\]\.\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[F\(s,\\pi\_\{\\theta\}\)\\big\]=\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[1\_\{\\mathcal\{B\}\}\(s\)F\(s,\\pi\_\{\\theta\}\)\\big\]\+\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[1\_\{\\mathcal\{G\}\}\(s\)F\(s,\\pi\_\{\\theta\}\)\\big\]\.By definition, for𝒳∈\{ℬ,𝒢\}\\mathcal\{X\}\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\},

F𝒳\(πθ\)≔𝔼s∼dT\[1𝒳\(s\)F\(s,πθ\)\],F\_\{\\mathcal\{X\}\}\(\\pi\_\{\\theta\}\)\\ \\coloneqq\\ \\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[1\_\{\\mathcal\{X\}\}\(s\)F\(s,\\pi\_\{\\theta\}\)\\big\],so

𝔼s∼dT\[F\(s,πθ\)\]=Fℬ\(πθ\)\+F𝒢\(πθ\)\.\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[F\(s,\\pi\_\{\\theta\}\)\\big\]=F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\theta\}\)\+F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\theta\}\)\.LetF\(πθ\)≔𝔼s∼dT\[F\(s,πθ\)\]F\(\\pi\_\{\\theta\}\)\\coloneqq\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\big\[F\(s,\\pi\_\{\\theta\}\)\\big\]\. Substituting into Eq\.\([17](https://arxiv.org/html/2605.26246#A3.E17)\) yields

𝖤𝖡\(πθ\)≤F\(πθ\)=Fℬ\(πθ\)\+F𝒢\(πθ\),\\mathsf\{EB\}\(\\pi\_\{\\theta\}\)\\ \\leq\\ F\(\\pi\_\{\\theta\}\)=F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\theta\}\)\+F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\theta\}\),which proves the proposition\. ∎

### Appendix DComplementarity Gain \(Proof of Thm\.[4\.4](https://arxiv.org/html/2605.26246#S4.SS4)\)

Restate of Theorem[4\.4](https://arxiv.org/html/2605.26246#S4.SS4)\.Letπhard\\pi\_\{\\text\{hard\}\}andπsoft\\pi\_\{\\text\{soft\}\}be \(near\-\)minimizers ofℓhard\\ell\_\{\\text\{hard\}\}andℓsoft\\ell\_\{\\text\{soft\}\}, respectively; in late\-stage distillation, often,

F𝒢\(πsoft\)<F𝒢\(πhard\),Fℬ\(πhard\)<Fℬ\(πsoft\)\.F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\text\{soft\}\}\)\\ <\\ F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\text\{hard\}\}\),\\quad F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\text\{hard\}\}\)\\ <\\ F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\text\{soft\}\}\)\.This suggests that for someλ∈\(0,1\)\\lambda\\in\(0,1\), an optimizerπhyb∈arg⁡min⁡\(λℓsoft\+\(1−λ\)ℓhard\)\\pi\_\{\\text\{hyb\}\}\\in\\arg\\min\\big\(\\lambda\\ell\_\{\\text\{soft\}\}\+\(1\-\\lambda\)\\ell\_\{\\text\{hard\}\}\\big\)can achieve

F\(πhyb\)<min⁡\{F\(πhard\),F\(πsoft\)\}\.F\(\\pi\_\{\\text\{hyb\}\}\)\\ <\\ \\min\\\{F\(\\pi\_\{\\text\{hard\}\}\),\\,F\(\\pi\_\{\\text\{soft\}\}\)\\\}\.
#### D\.1Late\-KD Region\-wise Residual Profiles and Complementarity

This section first establishes the region\-wise complementarity relations

F𝒢\(πsoft\)<F𝒢\(πhard\),Fℬ\(πhard\)<Fℬ\(πsoft\),F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\\;<\\;F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\),\\qquad F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\;<\\;F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\),under verifiable*late\-stage*KD structural conditions \(Corollary[D\.2](https://arxiv.org/html/2605.26246#A4.Thmtheorem2)\)\. Finally, Appendix[D\.2](https://arxiv.org/html/2605.26246#A4.SS2)shows how these region\-wise inequalities yield a strict improvement guarantee for the Hybrid update\.

##### D\.1\.1Auxiliary quantities \(cf\. Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\)

Recall the Bridge–Garden partition\(ℬ,𝒢\)\(\\mathcal\{B\},\\mathcal\{G\}\)and indicators1ℬ,1𝒢1\_\{\\mathcal\{B\}\},1\_\{\\mathcal\{G\}\}from Def\. 4\.2\. For𝒳∈\{ℬ,𝒢\}\\mathcal\{X\}\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\}, the region\-wise bound is

F𝒳\(π\)≔𝔼s∼dT\[1𝒳\(s\)F\(s,π\)\],F\_\{\\mathcal\{X\}\}\(\\pi\)\\coloneqq\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\left\[1\_\{\\mathcal\{X\}\}\(s\)\\,F\(s,\\pi\)\\right\],whereF\(s,π\)F\(s,\\pi\)andΔπ\(⋅∣s\)\\Delta\_\{\\pi\}\(\\cdot\\mid s\)are as in Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\.

We denote the region masses

pℬ≔𝔼\[1ℬ\(s\)\],p𝒢≔𝔼\[1𝒢\(s\)\],p\_\{\\mathcal\{B\}\}\\coloneqq\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}\(s\)\],\\qquad p\_\{\\mathcal\{G\}\}\\coloneqq\\mathbb\{E\}\[1\_\{\\mathcal\{G\}\}\(s\)\],\(18\)and assumepℬ\>0p\_\{\\mathcal\{B\}\}\>0andp𝒢\>0p\_\{\\mathcal\{G\}\}\>0\.

For later bounds, we use the decomposition

K𝒳\(π\)\\displaystyle K\_\{\\mathcal\{X\}\}\(\\pi\)≔𝔼\[1𝒳\(s\)∑a∈𝒱κ\(a∣s\)\|Δπ\(a∣s\)\|\],\\displaystyle\\coloneqq\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{X\}\}\(s\)\\sum\_\{a\\in\\mathcal\{V\}\}\\kappa\(a\\mid s\)\\,\|\\Delta\_\{\\pi\}\(a\\mid s\)\|\\right\],\(19\)δ2,𝒳\(π\)2\\displaystyle\\delta\_\{2,\\mathcal\{X\}\}\(\\pi\)^\{2\}≔𝔼\[1𝒳\(s\)∥Δπ\(⋅∣s\)∥12\],\\displaystyle\\coloneqq\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{X\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\right\],\(20\)so that

F𝒳\(π\)=K𝒳\(π\)\+C2δ2,𝒳\(π\)2\.F\_\{\\mathcal\{X\}\}\(\\pi\)=K\_\{\\mathcal\{X\}\}\(\\pi\)\+C\_\{2\}\\,\\delta\_\{2,\\mathcal\{X\}\}\(\\pi\)^\{2\}\.\(21\)We will also use the basic inequality for distributions:

0≤∥Δπ\(⋅∣s\)∥1≤2⟹∥Δπ\(⋅∣s\)∥12≤2∥Δπ\(⋅∣s\)∥1\.0\\leq\\\|\\Delta\_\{\\pi\}\(\\cdot\\mid s\)\\\|\_\{1\}\\leq 2\\quad\\Longrightarrow\\quad\\\|\\Delta\_\{\\pi\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\leq 2\\,\\\|\\Delta\_\{\\pi\}\(\\cdot\\mid s\)\\\|\_\{1\}\.\(22\)
Finally, define the teacher top token

b\(s\)≔arg⁡maxa∈𝒱⁡πT\(a∣s\)\.b\(s\)\\coloneqq\\arg\\max\_\{a\\in\\mathcal\{V\}\}\\pi\_\{T\}\(a\\mid s\)\.\(23\)

##### D\.1\.2Late\-KD structural conditions

\(D1\) Bridges: teacher nearly deterministic\.Define the teacher tail mass

η\(s\):=1−πT\(b\(s\)∣s\)=∑a≠b\(s\)πT\(a∣s\)\.\\eta\(s\)\\;:=\\;1\-\\pi\_\{T\}\(b\(s\)\\mid s\)\\;=\\;\\sum\_\{a\\neq b\(s\)\}\\pi\_\{T\}\(a\\mid s\)\.\(24\)Assume that there existsεℬ∈\(0,1/2\)\\varepsilon\_\{\\mathcal\{B\}\}\\in\(0,1/2\)such that

η\(s\)≤εℬ,∀swith1ℬ\(s\)=1\.\\eta\(s\)\\;\\leq\\;\\varepsilon\_\{\\mathcal\{B\}\},\\qquad\\forall s\\ \\text\{with\}\\ 1\_\{\\mathcal\{B\}\}\(s\)=1\.\(25\)
\(D2\) Gardens: teacher far from any one\-hot\.Assume there existsη𝒢∈\(0,1/2\]\\eta\_\{\\mathcal\{G\}\}\\in\(0,1/2\]such that for allsswith1𝒢\(s\)=11\_\{\\mathcal\{G\}\}\(s\)=1,

maxa∈𝒱πT\(a∣s\)≤1−η𝒢⟺∥δa−πT\(⋅∣s\)∥1≥2η𝒢∀a∈𝒱\.\\max\_\{a\\in\\mathcal\{V\}\}\\pi\_\{T\}\(a\\mid s\)\\;\\leq\\;1\-\\eta\_\{\\mathcal\{G\}\}\\qquad\\Longleftrightarrow\\qquad\\\|\\delta\_\{a\}\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\;\\geq\\;2\\eta\_\{\\mathcal\{G\}\}\\ \\ \\forall a\\in\\mathcal\{V\}\.\(26\)
\(D3\) Optimization effectiveness \(late KD\)\.Assume there existηH,ηS\>0\\eta\_\{H\},\\eta\_\{S\}\>0such that

𝔼\[∥πhard\(⋅∣s\)−δa∗∥1\]\\displaystyle\\mathbb\{E\}\\\!\\left\[\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\delta\_\{a^\{\*\}\}\\\|\_\{1\}\\right\]≤ηH,\\displaystyle\\leq\\eta\_\{H\},\(27\)𝔼\[∥πsoft\(⋅∣s\)−πT\(⋅∣s\)∥1\]\\displaystyle\\mathbb\{E\}\\\!\\left\[\\\|\\pi\_\{\\mathrm\{soft\}\}\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]≤ηS\.\\displaystyle\\leq\\eta\_\{S\}\.\(28\)
\(D4\) Soft is under\-confident on Bridges on average\.Define

g\(s\):=\[πT\(a∗∣s\)−πsoft\(a∗∣s\)\]\+,g\(s\)\\;:=\\;\\big\[\\pi\_\{T\}\(a^\{\*\}\\mid s\)\-\\pi\_\{\\mathrm\{soft\}\}\(a^\{\*\}\\mid s\)\\big\]\_\{\+\},\(29\)wherea∗a^\{\*\}is the hard\-KD training target sampled from the teacher model,i\.e\.,a∗∼πT\(⋅∣s\)a^\{\*\}\\sim\\pi\_\{T\}\(\\cdot\\mid s\)\. Assume

𝔼\[1ℬ\(s\)g\(s\)\]≥Γℬ,Γℬ\>0\.\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,g\(s\)\\right\]\\;\\geq\\;\\Gamma\_\{\\mathcal\{B\}\},\\qquad\\Gamma\_\{\\mathcal\{B\}\}\>0\.\(30\)

##### D\.1\.3Region\-wise bounds and complementarity

###### Theorem D\.1\(Late\-KD region\-wise bounds forFℬF\_\{\\mathcal\{B\}\}andF𝒢F\_\{\\mathcal\{G\}\}\)\.

Under\(D1\)–\(D4\), define

Aℬ:=ηH\+4𝔼\[1ℬ\(s\)η\(s\)\],α𝒢:=\(2η𝒢p𝒢−ηH\)\+\.A\_\{\\mathcal\{B\}\}\\;:=\\;\\eta\_\{H\}\+4\\,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,\\eta\(s\)\\right\],\\qquad\\alpha\_\{\\mathcal\{G\}\}\\;:=\\;\\bigl\(2\\eta\_\{\\mathcal\{G\}\}\\,p\_\{\\mathcal\{G\}\}\-\\eta\_\{H\}\\bigr\)\_\{\+\}\.\(31\)Then:

Fℬ\(πhard\)\\displaystyle F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)≤\(κmax\+2C2\)Aℬ≤\(κmax\+2C2\)\(ηH\+4pℬεℬ\),\\displaystyle\\leq\(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,A\_\{\\mathcal\{B\}\}\\ \\leq\\ \(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\bigl\(\\eta\_\{H\}\+4p\_\{\\mathcal\{B\}\}\\varepsilon\_\{\\mathcal\{B\}\}\\bigr\),\(32\)F𝒢\(πsoft\)\\displaystyle F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)≤\(κmax\+2C2\)ηS,\\displaystyle\\leq\(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,\\eta\_\{S\},\(33\)F𝒢\(πhard\)\\displaystyle F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)≥C2p𝒢α𝒢2,\\displaystyle\\geq\\frac\{C\_\{2\}\}\{p\_\{\\mathcal\{G\}\}\}\\,\\alpha\_\{\\mathcal\{G\}\}^\{2\},\(34\)Fℬ\(πsoft\)\\displaystyle F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)≥4C2pℬΓℬ2\.\\displaystyle\\geq\\frac\{4C\_\{2\}\}\{p\_\{\\mathcal\{B\}\}\}\\,\\Gamma\_\{\\mathcal\{B\}\}^\{2\}\.\(35\)

###### Proof\.

We prove the four inequalities in turn\.

\(1\) Upper bound onFℬ\(πhard\)F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\.Fixsswith1ℬ\(s\)=11\_\{\\mathcal\{B\}\}\(s\)=1\. By the triangle inequality,

∥πhard\(⋅∣s\)−πT\(⋅∣s\)∥1≤∥πhard\(⋅∣s\)−δa∗∥1\+∥δa∗−πT\(⋅∣s\)∥1\.\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\leq\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\delta\_\{a^\{\*\}\}\\\|\_\{1\}\+\\\|\\delta\_\{a^\{\*\}\}\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\.For the second term, conditioning onssand usinga∗∼πT\(⋅∣s\)a^\{\*\}\\sim\\pi\_\{T\}\(\\cdot\\mid s\),

𝔼\[∥δa∗−πT\(⋅∣s\)∥1∣s\]=2\(1−∑a∈𝒱πT\(a∣s\)2\)≤2\(1−πT\(b\(s\)∣s\)2\)=4η\(s\)−2η\(s\)2≤4η\(s\),\\mathbb\{E\}\\\!\\left\[\\\|\\delta\_\{a^\{\*\}\}\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\mid s\\right\]=2\\Bigl\(1\-\\sum\_\{a\\in\\mathcal\{V\}\}\\pi\_\{T\}\(a\\mid s\)^\{2\}\\Bigr\)\\leq 2\\bigl\(1\-\\pi\_\{T\}\(b\(s\)\\mid s\)^\{2\}\\bigr\)=4\\eta\(s\)\-2\\eta\(s\)^\{2\}\\leq 4\\eta\(s\),where we used Eq\.\([24](https://arxiv.org/html/2605.26246#A4.E24)\) in the last two steps\. Multiplying by1ℬ\(s\)1\_\{\\mathcal\{B\}\}\(s\)and taking expectation gives

𝔼\[1ℬ\(s\)∥πhard\(⋅∣s\)−πT\(⋅∣s\)∥1\]≤𝔼\[∥πhard\(⋅∣s\)−δa∗∥1\]\+4𝔼\[1ℬ\(s\)η\(s\)\]≤Aℬ,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]\\leq\\mathbb\{E\}\\\!\\left\[\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\delta\_\{a^\{\*\}\}\\\|\_\{1\}\\right\]\+4\\,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\eta\(s\)\\right\]\\leq A\_\{\\mathcal\{B\}\},using Eq\.\([27](https://arxiv.org/html/2605.26246#A4.E27)\) and Eq\.\([31](https://arxiv.org/html/2605.26246#A4.E31)\),

Kℬ\(πhard\)≤κmax𝔼\[1ℬ\(s\)∥Δπhard\(⋅∣s\)∥1\]\.K\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\leq\\kappa\_\{\\max\}\\,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]\.By Eq\.\([22](https://arxiv.org/html/2605.26246#A4.E22)\),

δ2,ℬ\(πhard\)2=𝔼\[1ℬ\(s\)∥Δπhard\(⋅∣s\)∥12\]≤2𝔼\[1ℬ\(s\)∥Δπhard\(⋅∣s\)∥1\]\.\\delta\_\{2,\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)^\{2\}=\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\right\]\\leq 2\\,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]\.Combining with Eq\.\([21](https://arxiv.org/html/2605.26246#A4.E21)\) yields

Fℬ\(πhard\)≤\(κmax\+2C2\)𝔼\[1ℬ\(s\)∥Δπhard\(⋅∣s\)∥1\]≤\(κmax\+2C2\)Aℬ,F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\leq\(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]\\leq\(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,A\_\{\\mathcal\{B\}\},which proves the first inequality in Eq\.\([32](https://arxiv.org/html/2605.26246#A4.E32)\)\. The second inequality follows from \([25](https://arxiv.org/html/2605.26246#A4.E25)\), which implies𝔼\[1ℬ\(s\)η\(s\)\]≤pℬεℬ\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}\(s\)\\eta\(s\)\]\\leq p\_\{\\mathcal\{B\}\}\\varepsilon\_\{\\mathcal\{B\}\}\.

\(2\) Upper bound onF𝒢\(πsoft\)F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\.By Eq\.\([28](https://arxiv.org/html/2605.26246#A4.E28)\),

𝔼\[1𝒢\(s\)∥Δπsoft\(⋅∣s\)∥1\]≤𝔼\[∥πsoft\(⋅∣s\)−πT\(⋅∣s\)∥1\]≤ηS\.\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{G\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]\\leq\\mathbb\{E\}\\\!\\left\[\\\|\\pi\_\{\\mathrm\{soft\}\}\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]\\leq\\eta\_\{S\}\.Thus, using Eq\.\([22](https://arxiv.org/html/2605.26246#A4.E22)\) exactly as above,

K𝒢\(πsoft\)≤κmaxηS,δ2,𝒢\(πsoft\)2≤2ηS,K\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\\leq\\kappa\_\{\\max\}\\eta\_\{S\},\\qquad\\delta\_\{2,\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)^\{2\}\\leq 2\\eta\_\{S\},and Eq\.\([33](https://arxiv.org/html/2605.26246#A4.E33)\) follows from Eq\.\([21](https://arxiv.org/html/2605.26246#A4.E21)\)\.

\(3\) Lower bound onF𝒢\(πhard\)F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\.Fixsswith1𝒢\(s\)=11\_\{\\mathcal\{G\}\}\(s\)=1\. By Eq\.\([26](https://arxiv.org/html/2605.26246#A4.E26)\),∥δa∗−πT\(⋅∣s\)∥1≥2η𝒢\.\\\|\\delta\_\{a^\{\*\}\}\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\geq 2\\eta\_\{\\mathcal\{G\}\}\.By the reverse triangle inequality,

∥Δπhard\(⋅∣s\)∥1=∥πhard\(⋅∣s\)−πT\(⋅∣s\)∥1≥∥δa∗−πT\(⋅∣s\)∥1−∥πhard\(⋅∣s\)−δa∗∥1≥2η𝒢−∥πhard\(⋅∣s\)−δa∗∥1\.\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}=\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\\geq\\\|\\delta\_\{a^\{\*\}\}\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}\-\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\delta\_\{a^\{\*\}\}\\\|\_\{1\}\\geq 2\\eta\_\{\\mathcal\{G\}\}\-\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\delta\_\{a^\{\*\}\}\\\|\_\{1\}\.Multiplying by1𝒢\(s\)1\_\{\\mathcal\{G\}\}\(s\)and taking expectation yields

𝔼\[1𝒢\(s\)∥Δπhard\(⋅∣s\)∥1\]≥2η𝒢p𝒢−𝔼\[∥πhard\(⋅∣s\)−δa∗∥1\]≥2η𝒢p𝒢−ηH,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{G\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}\\right\]\\geq 2\\eta\_\{\\mathcal\{G\}\}p\_\{\\mathcal\{G\}\}\-\\mathbb\{E\}\\\!\\left\[\\\|\\pi\_\{\\mathrm\{hard\}\}\(\\cdot\\mid s\)\-\\delta\_\{a^\{\*\}\}\\\|\_\{1\}\\right\]\\geq 2\\eta\_\{\\mathcal\{G\}\}p\_\{\\mathcal\{G\}\}\-\\eta\_\{H\},hence𝔼\[1𝒢‖Δπhard‖1\]≥α𝒢\\mathbb\{E\}\[1\_\{\\mathcal\{G\}\}\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\\\|\_\{1\}\]\\geq\\alpha\_\{\\mathcal\{G\}\}by Eq\.\([31](https://arxiv.org/html/2605.26246#A4.E31)\)\. Now apply Cauchy–Schwarz toX:=1𝒢\(s\)∥Δπhard\(⋅∣s\)∥1≥0X:=1\_\{\\mathcal\{G\}\}\(s\)\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}\\geq 0:

𝔼\[X\]2≤𝔼\[1𝒢\(s\)\]⋅𝔼\[1𝒢\(s\)∥Δπhard\(⋅∣s\)∥12\]=p𝒢δ2,𝒢\(πhard\)2\.\\mathbb\{E\}\[X\]^\{2\}\\leq\\mathbb\{E\}\[1\_\{\\mathcal\{G\}\}\(s\)\]\\cdot\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{G\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{hard\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\right\]=p\_\{\\mathcal\{G\}\}\\,\\delta\_\{2,\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)^\{2\}\.Thereforeδ2,𝒢\(πhard\)2≥α𝒢2/p𝒢\\delta\_\{2,\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)^\{2\}\\geq\\alpha\_\{\\mathcal\{G\}\}^\{2\}/p\_\{\\mathcal\{G\}\}\. Finally, sinceK𝒢\(πhard\)≥0K\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\geq 0, Eq\.\([21](https://arxiv.org/html/2605.26246#A4.E21)\) gives

F𝒢\(πhard\)≥C2δ2,𝒢\(πhard\)2≥C2p𝒢α𝒢2,F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\geq C\_\{2\}\\,\\delta\_\{2,\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)^\{2\}\\geq\\frac\{C\_\{2\}\}\{p\_\{\\mathcal\{G\}\}\}\\,\\alpha\_\{\\mathcal\{G\}\}^\{2\},which is Eq\.\([34](https://arxiv.org/html/2605.26246#A4.E34)\)\.

\(4\) Lower bound onFℬ\(πsoft\)F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\.Fixsswith1ℬ\(s\)=11\_\{\\mathcal\{B\}\}\(s\)=1\. Ifg\(s\)\>0g\(s\)\>0, thenΔπsoft\(a∗∣s\)=πsoft\(a∗∣s\)−πT\(a∗∣s\)=−g\(s\)\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(a^\{\*\}\\mid s\)=\\pi\_\{\\mathrm\{soft\}\}\(a^\{\*\}\\mid s\)\-\\pi\_\{T\}\(a^\{\*\}\\mid s\)=\-g\(s\)\. Using∑a∈𝒱Δπsoft\(a∣s\)=0\\sum\_\{a\\in\\mathcal\{V\}\}\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(a\\mid s\)=0, we have∑a≠a∗Δπsoft\(a∣s\)=g\(s\)\\sum\_\{a\\neq a^\{\*\}\}\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(a\\mid s\)=g\(s\)and thus∑a≠a∗\|Δπsoft\(a∣s\)\|≥g\(s\)\\sum\_\{a\\neq a^\{\*\}\}\|\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(a\\mid s\)\|\\geq g\(s\)\. Hence

∥Δπsoft\(⋅∣s\)∥1=\|Δπsoft\(a∗∣s\)\|\+∑a≠a∗\|Δπsoft\(a∣s\)\|≥g\(s\)\+g\(s\)=2g\(s\),\\\|\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}=\|\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(a^\{\*\}\\mid s\)\|\+\\sum\_\{a\\neq a^\{\*\}\}\|\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(a\\mid s\)\|\\geq g\(s\)\+g\(s\)=2g\(s\),and the same inequality holds trivially ifg\(s\)=0g\(s\)=0\. Consequently,

δ2,ℬ\(πsoft\)2=𝔼\[1ℬ\(s\)∥Δπsoft\(⋅∣s\)∥12\]≥4𝔼\[1ℬ\(s\)g\(s\)2\]\.\\delta\_\{2,\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)^\{2\}=\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,\\\|\\Delta\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}\\right\]\\geq 4\\,\\mathbb\{E\}\\\!\\left\[1\_\{\\mathcal\{B\}\}\(s\)\\,g\(s\)^\{2\}\\right\]\.By Cauchy–Schwarz,𝔼\[1ℬg\]2≤𝔼\[1ℬ\]𝔼\[1ℬg2\]=pℬ𝔼\[1ℬg2\]\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}g\]^\{2\}\\leq\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}\]\\,\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}g^\{2\}\]=p\_\{\\mathcal\{B\}\}\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}g^\{2\}\], hence

𝔼\[1ℬg2\]≥𝔼\[1ℬg\]2pℬ≥Γℬ2pℬ,\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}g^\{2\}\]\\geq\\frac\{\\mathbb\{E\}\[1\_\{\\mathcal\{B\}\}g\]^\{2\}\}\{p\_\{\\mathcal\{B\}\}\}\\geq\\frac\{\\Gamma\_\{\\mathcal\{B\}\}^\{2\}\}\{p\_\{\\mathcal\{B\}\}\},by Eq\.\([30](https://arxiv.org/html/2605.26246#A4.E30)\)\. Therefore

δ2,ℬ\(πsoft\)2≥4pℬΓℬ2\.\\delta\_\{2,\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)^\{2\}\\geq\\frac\{4\}\{p\_\{\\mathcal\{B\}\}\}\\,\\Gamma\_\{\\mathcal\{B\}\}^\{2\}\.SinceKℬ\(πsoft\)≥0K\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\\geq 0, Eq\.\([21](https://arxiv.org/html/2605.26246#A4.E21)\) implies

Fℬ\(πsoft\)≥C2δ2,ℬ\(πsoft\)2≥4C2pℬΓℬ2,F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\\geq C\_\{2\}\\,\\delta\_\{2,\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)^\{2\}\\geq\\frac\{4C\_\{2\}\}\{p\_\{\\mathcal\{B\}\}\}\\,\\Gamma\_\{\\mathcal\{B\}\}^\{2\},which is Eq\.\([35](https://arxiv.org/html/2605.26246#A4.E35)\)\. ∎

Remark\.Thm\.[D\.1](https://arxiv.org/html/2605.26246#A4.Thmtheorem1)provides*complementary*upper/lower bounds on the region\-wise objectivesFℬF\_\{\\mathcal\{B\}\}andF𝒢F\_\{\\mathcal\{G\}\}forπhard\\pi\_\{\\mathrm\{hard\}\}andπsoft\\pi\_\{\\mathrm\{soft\}\}\. To turn these bounds into a*strict*region\-wise comparison, one needs a quantitative regime in which the late\-stage optimization residuals are dominated by the region\-structural terms\.

Specifically, in late KD one typically observes that the Soft\-only fit\-to\-teacher errorηS\\eta\_\{S\}becomes small, and the Hard\-only Bridge mismatch term

Aℬ=ηH\+4𝔼s∼dT\[1ℬ\(s\)η\(s\)\]A\_\{\\mathcal\{B\}\}=\\eta\_\{H\}\+4\\,\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\big\[1\_\{\\mathcal\{B\}\}\(s\)\\eta\(s\)\\big\]also becomes small as the hard objective converges and the teacher is nearly deterministic on Bridges\. At the same time, the region structure does not vanish: Gardens remain far from one\-hot \(captured byα𝒢\\alpha\_\{\\mathcal\{G\}\}\), and Soft\-only retains a nontrivial average under\-confidence on Bridges \(captured byΓℬ\\Gamma\_\{\\mathcal\{B\}\}\)\. Empirically, we also observe a consistent contrast between the two endpoints: the Hard\-only solutionπhard\\pi\_\{\\mathrm\{hard\}\}tends to produce more peaked output distributions, whereas the Soft\-only solutionπsoft\\pi\_\{\\mathrm\{soft\}\}tends to produce smoother ones; see Fig\.[6](https://arxiv.org/html/2605.26246#S7.F6)\.

In this domination regime, the strict ordering follows whenever the residual upper bounds fall below the structural lower bounds, namely,

\(κmax\+2C2\)ηS⏟Soft upper bound onF𝒢\(πsoft\)<C2p𝒢α𝒢2⏟Hard lower bound onF𝒢\(πhard\),\(κmax\+2C2\)Aℬ⏟Hard upper bound onFℬ\(πhard\)<4C2pℬΓℬ2⏟Soft lower bound onFℬ\(πsoft\)\.\\underbrace\{\(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,\\eta\_\{S\}\}\_\{\\text\{Soft upper bound on \}F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\}\\ <\\ \\underbrace\{\\frac\{C\_\{2\}\}\{p\_\{\\mathcal\{G\}\}\}\\,\\alpha\_\{\\mathcal\{G\}\}^\{2\}\}\_\{\\text\{Hard lower bound on \}F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\},\\qquad\\underbrace\{\(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,A\_\{\\mathcal\{B\}\}\}\_\{\\text\{Hard upper bound on \}F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\}\\ <\\ \\underbrace\{\\frac\{4C\_\{2\}\}\{p\_\{\\mathcal\{B\}\}\}\\,\\Gamma\_\{\\mathcal\{B\}\}^\{2\}\}\_\{\\text\{Soft lower bound on \}F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\}\.\(36\)
###### Corollary D\.2\(Region\-wise complementarity inequalities\)\.

Under the conditions of Theorem[D\.1](https://arxiv.org/html/2605.26246#A4.Thmtheorem1)and the domination regime Eq\.\([36](https://arxiv.org/html/2605.26246#A4.E36)\),

F𝒢\(πsoft\)<F𝒢\(πhard\),Fℬ\(πhard\)<Fℬ\(πsoft\)\.F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\\ <\\ F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\),\\qquad F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\ <\\ F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\.

###### Proof\.

Using Eq\.\([33](https://arxiv.org/html/2605.26246#A4.E33)\) and Eq\.\([34](https://arxiv.org/html/2605.26246#A4.E34)\),

F𝒢\(πsoft\)≤\(κmax\+2C2\)ηS<C2p𝒢α𝒢2≤F𝒢\(πhard\),F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{soft\}\}\)\\ \\leq\\ \(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,\\eta\_\{S\}\\ <\\ \\frac\{C\_\{2\}\}\{p\_\{\\mathcal\{G\}\}\}\\,\\alpha\_\{\\mathcal\{G\}\}^\{2\}\\ \\leq\\ F\_\{\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\),where the strict inequality is the first condition in Eq\.\([36](https://arxiv.org/html/2605.26246#A4.E36)\)\. Using Eq\.\([32](https://arxiv.org/html/2605.26246#A4.E32)\) and Eq\.\([35](https://arxiv.org/html/2605.26246#A4.E35)\),

Fℬ\(πhard\)≤\(κmax\+2C2\)Aℬ<4C2pℬΓℬ2≤Fℬ\(πsoft\),F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\ \\leq\\ \(\\kappa\_\{\\max\}\+2C\_\{2\}\)\\,A\_\{\\mathcal\{B\}\}\\ <\\ \\frac\{4C\_\{2\}\}\{p\_\{\\mathcal\{B\}\}\}\\,\\Gamma\_\{\\mathcal\{B\}\}^\{2\}\\ \\leq\\ F\_\{\\mathcal\{B\}\}\(\\pi\_\{\\mathrm\{soft\}\}\),where the strict inequality is the second condition in Eq\.\([36](https://arxiv.org/html/2605.26246#A4.E36)\)\. ∎

#### D\.2Cross\-direction Descent and Hybrid Improvement

This subsection proves the main\-text claim \(Theorem[4\.4](https://arxiv.org/html/2605.26246#S4.SS4)\) that a one\-step hybrid update can strictly reduce the boundFFcompared with both pure KD solutions,πsoft\\pi\_\{\\mathrm\{soft\}\}\(Soft\-only\) andπhard\\pi\_\{\\mathrm\{hard\}\}\(Hard\-only\)\. The idea is to show thatFFhas a strictly negative one\-sided directional derivative along the cross direction at each of these two solutions, and then pick a convex combination of the Soft/Hard directions with a small step size to guarantee a strict decrease \(see Theorem[D\.11](https://arxiv.org/html/2605.26246#A4.Thmtheorem11)\)\.

##### D\.2\.1Setup and structural assumptions

Recall \(exposure\-bias bound\)\.From App\.[D\.1](https://arxiv.org/html/2605.26246#A4.SS1), we use the exposure\-bias bound

F\(π\)=Fℬ\(π\)\+F𝒢\(π\),FX\(π\)=KX\(π\)\+C2δ2,X\(π\)2,X∈\{ℬ,𝒢\}\.F\(\\pi\)=F\_\{\\mathcal\{B\}\}\(\\pi\)\+F\_\{\\mathcal\{G\}\}\(\\pi\),\\qquad F\_\{X\}\(\\pi\)=K\_\{X\}\(\\pi\)\+C\_\{2\}\\,\\delta\_\{2,X\}\(\\pi\)^\{2\},\\quad X\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\}\.
Feasible Soft/Hard interpolations\.For any policyπ\\piandt∈\[0,1\]t\\in\[0,1\], define

\(𝖳Stπ\)\(⋅∣s\)\\displaystyle\(\\mathsf\{T\}\_\{S\}^\{t\}\\pi\)\(\\cdot\\mid s\):=\(1−t\)π\(⋅∣s\)\+tπT\(⋅∣s\),\\displaystyle:=\(1\-t\)\\pi\(\\cdot\\mid s\)\+t\\,\\pi\_\{T\}\(\\cdot\\mid s\),\(37\)\(𝖳Htπ\)\(⋅∣s\)\\displaystyle\(\\mathsf\{T\}\_\{H\}^\{t\}\\pi\)\(\\cdot\\mid s\):=\(1−t\)π\(⋅∣s\)\+tδa∗\(⋅\),\\displaystyle:=\(1\-t\)\\pi\(\\cdot\\mid s\)\+t\\,\\delta\_\{a^\{\*\}\}\(\\cdot\),\(38\)wherea∗∼πT\(⋅∣s\)a^\{\*\}\\sim\\pi\_\{T\}\(\\cdot\\mid s\)is the hard label token in Eq\. \(3\)\. The corresponding feasible directions are

vS\(π\):=πT−π,vH\(π\):=δa∗−π\.v\_\{S\}\(\\pi\):=\\pi\_\{T\}\-\\pi,\\qquad v\_\{H\}\(\\pi\):=\\delta\_\{a^\{\*\}\}\-\\pi\.\(39\)
Right directional derivative\.For any functionalJ\[π\]J\[\\pi\]and any feasible operator𝖳t\\mathsf\{T\}^\{t\},

D𝖳\+J\[π\]:=ddtJ\[𝖳tπ\]\|t=0\+\.D\_\{\\mathsf\{T\}\}^\{\+\}J\[\\pi\]:=\\left\.\\frac\{d\}\{dt\}\\,J\[\\mathsf\{T\}^\{t\}\\pi\]\\right\|\_\{t=0^\{\+\}\}\.\(40\)
Endpoint asymmetry assumptions\.These conditions formalize complementarity atπsoft\\pi\_\{\\mathrm\{soft\}\}\(Soft\-only\) andπhard\\pi\_\{\\mathrm\{hard\}\}\(Hard\-only\)\. Empirically, we also find these endpoint asymmetries to be consistently observable across models and datasets: the Soft\-only solution typically yields a smoother, less confident \(higher\-entropy\) distribution, whereas the Hard\-only solution yields a sharper \(lower\-entropy\) distribution and can underfit the teacher’s distributional diversity in the Garden region \(see Sec\.[6](https://arxiv.org/html/2605.26246#S6)\)\.

\(E1\) Oversmoothing at the Soft\-only endpoint\.Assumeg\(s\)\>0g\(s\)\>0almost surely, whereg\(s\)g\(s\)is defined in Eq\.\([29](https://arxiv.org/html/2605.26246#A4.E29)\)\.

\(E2\) Nondegenerate mean gap\.

𝔼s∼dT\[g\(s\)\]≥ΓOS\>0\.\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\[g\(s\)\]\\ \\geq\\ \\Gamma\_\{\\rm OS\}\\ \>\\ 0\.\(41\)
\(E3\)κ\\kappa\-shift atπsoft\\pi\_\{\\mathrm\{soft\}\}\.Under \(E1\)–\(E2\),πsoft\\pi\_\{\\mathrm\{soft\}\}is oversmoothed and therefore assigns insufficient probability to the teacher\-sampled tokena∗a^\{\*\}, with the remaining mass necessarily redistributed overa≠a∗a\\neq a^\{\*\}\. Assumption \(E3\) formalizes that, inκ\\kappa\-weighted terms, this mismatch is primarily attributable to tokens thatπsoft\\pi\_\{\\mathrm\{soft\}\}*over\-assigns*relative toπT\\pi\_\{T\}\.

Letda\(s\):=πsoft\(a∣s\)−πT\(a∣s\)d\_\{a\}\(s\):=\\pi\_\{\\mathrm\{soft\}\}\(a\\mid s\)\-\\pi\_\{T\}\(a\\mid s\)and define

𝒫\(s\):=\{a≠a∗:da\(s\)\>0\},𝒩\(s\):=\{a≠a∗:da\(s\)≤0\}\.\\mathcal\{P\}\(s\):=\\\{a\\neq a^\{\*\}:d\_\{a\}\(s\)\>0\\\},\\qquad\\mathcal\{N\}\(s\):=\\\{a\\neq a^\{\*\}:d\_\{a\}\(s\)\\leq 0\\\}\.Assume that almost surely,

∑a∈𝒩\(s\)κ\(a∣s\)πsoft\(a∣s\)≤κ\(a∗∣s\)\(1−πsoft\(a∗∣s\)\)\+∑a∈𝒫\(s\)κ\(a∣s\)πsoft\(a∣s\)\.\\sum\_\{a\\in\\mathcal\{N\}\(s\)\}\\kappa\(a\\mid s\)\\,\\pi\_\{\\mathrm\{soft\}\}\(a\\mid s\)\\ \\leq\\ \\kappa\(a^\{\*\}\\mid s\)\\bigl\(1\-\\pi\_\{\\mathrm\{soft\}\}\(a^\{\*\}\\mid s\)\\bigr\)\\ \+\\ \\sum\_\{a\\in\\mathcal\{P\}\(s\)\}\\kappa\(a\\mid s\)\\,\\pi\_\{\\mathrm\{soft\}\}\(a\\mid s\)\.\(42\)
\(E4\) Hard distillation leaves residual error in the Garden region \(follows directly from Theorem[D\.1](https://arxiv.org/html/2605.26246#A4.Thmtheorem1)\)\.

δ2,𝒢\(πhard\)≥δ¯𝒢\>0\.\\delta\_\{2,\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\ \\geq\\ \\underline\{\\delta\}\_\{\\mathcal\{G\}\}\\ \>\\ 0\.\(43\)

##### D\.2\.2Directional lemmas

###### Lemma D\.4\(Right directional derivative of\|⋅\|\|\\cdot\|\)\.

For anyu,w∈ℝu,w\\in\\mathbb\{R\},ϕ\(t\):=\|u\+tw\|\\phi\(t\):=\|u\+tw\|has a right derivative att=0t=0given by

ϕ\+′\(0\)=\{sign\(u\)w,u≠0,\|w\|,u=0\.\\phi^\{\\prime\}\_\{\+\}\(0\)=\\begin\{cases\}\\mathrm\{sign\}\(u\)\\,w,&u\\neq 0,\\\\ \|w\|,&u=0\.\\end\{cases\}

###### Lemma D\.5\(Soft interpolation contractsFXF\_\{X\}\)\.

Fix any policyπ\\piandX∈\{ℬ,𝒢\}X\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\}\. Letπt:=𝖳Stπ\\pi\_\{t\}:=\\mathsf\{T\}\_\{S\}^\{t\}\\pi\. Then for allt∈\[0,1\]t\\in\[0,1\],

KX\(πt\)=\(1−t\)KX\(π\),δ2,X\(πt\)2=\(1−t\)2δ2,X\(π\)2,K\_\{X\}\(\\pi\_\{t\}\)=\(1\-t\)K\_\{X\}\(\\pi\),\\qquad\\delta\_\{2,X\}\(\\pi\_\{t\}\)^\{2\}=\(1\-t\)^\{2\}\\delta\_\{2,X\}\(\\pi\)^\{2\},and hence

D𝖳S\+FX\(π\)=−KX\(π\)−2C2δ2,X\(π\)2≤0\.D\_\{\\mathsf\{T\}\_\{S\}\}^\{\+\}F\_\{X\}\(\\pi\)=\-K\_\{X\}\(\\pi\)\-2C\_\{2\}\\delta\_\{2,X\}\(\\pi\)^\{2\}\\ \\leq\\ 0\.\(44\)

###### Proof\.

Underπt=\(1−t\)π\+tπT\\pi\_\{t\}=\(1\-t\)\\pi\+t\\pi\_\{T\}, we haveΔπt=\(1−t\)Δπ\\Delta\_\{\\pi\_\{t\}\}=\(1\-t\)\\Delta\_\{\\pi\}and thusrπt\(s\)=\(1−t\)rπ\(s\)r\_\{\\pi\_\{t\}\}\(s\)=\(1\-t\)r\_\{\\pi\}\(s\)\. Plugging into the definitions ofKXK\_\{X\}andδ2,X\\delta\_\{2,X\}yields the claimed scalings; differentiating att=0\+t=0^\{\+\}gives Eq\.\([44](https://arxiv.org/html/2605.26246#A4.E44)\)\. ∎

###### Lemma D\.6\(Hard step atπsoft\\pi\_\{\\mathrm\{soft\}\}strictly decreasesFF\)\.

Letπt:=𝖳Htπsoft\\pi\_\{t\}:=\\mathsf\{T\}\_\{H\}^\{t\}\\pi\_\{\\mathrm\{soft\}\}\. Under Eq\.\([42](https://arxiv.org/html/2605.26246#A4.E42)\) and\(E1\)–\(E2\), for eachX∈\{ℬ,𝒢\}X\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\},

ddtδ2,X\(πt\)2\|t=0\+\\displaystyle\\left\.\\frac\{d\}\{dt\}\\delta\_\{2,X\}\(\\pi\_\{t\}\)^\{2\}\\right\|\_\{t=0^\{\+\}\}≤−8𝔼s∼dT\[1X\(s\)g\(s\)2\],\\displaystyle\\leq\-8\\,\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\left\[1\_\{X\}\(s\)\\,g\(s\)^\{2\}\\right\],\(45\)ddtKX\(πt\)\|t=0\+\\displaystyle\\left\.\\frac\{d\}\{dt\}K\_\{X\}\(\\pi\_\{t\}\)\\right\|\_\{t=0^\{\+\}\}≤0\.\\displaystyle\\leq 0\.\(46\)Consequently,

D𝖳H\+F\(πsoft\)≤−8C2𝔼s∼dT\[g\(s\)2\]≤−8C2ΓOS2<0\.D\_\{\\mathsf\{T\}\_\{H\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{soft\}\}\)\\leq\-8C\_\{2\}\\,\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\[g\(s\)^\{2\}\]\\leq\-8C\_\{2\}\\Gamma\_\{\\rm OS\}^\{2\}<0\.\(47\)

###### Proof\.

Fix a prefixssand condition on the sampled hard labela∗∼πT\(⋅∣s\)a^\{\*\}\\sim\\pi\_\{T\}\(\\cdot\\mid s\)\. Defineq\(a\)=πsoft\(a∣s\)q\(a\)=\\pi\_\{\\mathrm\{soft\}\}\(a\\mid s\),p\(a\)=πT\(a∣s\)p\(a\)=\\pi\_\{T\}\(a\\mid s\), andda=q\(a\)−p\(a\)d\_\{a\}=q\(a\)\-p\(a\)\. By \(E1\) and Eq\.\([29](https://arxiv.org/html/2605.26246#A4.E29)\), we haveda∗=−g\(s\)<0d\_\{a^\{\*\}\}=\-g\(s\)<0\. Under the Hard interpolationqt=\(1−t\)q\+tδa∗q\_\{t\}=\(1\-t\)q\+t\\delta\_\{a^\{\*\}\},

da∗\(t\)=da∗\+t\(1−q\(a∗\)\),da\(t\)=da−tq\(a\)\(a≠a∗\)\.d\_\{a^\{\*\}\}\(t\)=d\_\{a^\{\*\}\}\+t\(1\-q\(a^\{\*\}\)\),\\qquad d\_\{a\}\(t\)=d\_\{a\}\-tq\(a\)\\ \\ \(a\\neq a^\{\*\}\)\.
Step 1\.By Lemma[D\.4](https://arxiv.org/html/2605.26246#A4.Thmtheorem4),

ddt\|da∗\(t\)\|\|0\+=−\(1−q\(a∗\)\),ddt\|da\(t\)\|\|0\+=\{−q\(a\),da\>0,\+q\(a\),da≤0\.\\left\.\\frac\{d\}\{dt\}\|d\_\{a^\{\*\}\}\(t\)\|\\right\|\_\{0^\{\+\}\}=\-\(1\-q\(a^\{\*\}\)\),\\qquad\\left\.\\frac\{d\}\{dt\}\|d\_\{a\}\(t\)\|\\right\|\_\{0^\{\+\}\}=\\begin\{cases\}\-q\(a\),&d\_\{a\}\>0,\\\\ \+q\(a\),&d\_\{a\}\\leq 0\.\\end\{cases\}Thus

ddtrπt\(s\)\|0\+=−\(1−q\(a∗\)\)−∑a∈𝒫\(s\)q\(a\)\+∑a∈𝒩\(s\)q\(a\)=−2∑a∈𝒫\(s\)q\(a\)\.\\left\.\\frac\{d\}\{dt\}r\_\{\\pi\_\{t\}\}\(s\)\\right\|\_\{0^\{\+\}\}=\-\(1\-q\(a^\{\*\}\)\)\-\\sum\_\{a\\in\\mathcal\{P\}\(s\)\}q\(a\)\+\\sum\_\{a\\in\\mathcal\{N\}\(s\)\}q\(a\)=\-2\\sum\_\{a\\in\\mathcal\{P\}\(s\)\}q\(a\)\.Moreover, since∑a≠a∗da=−da∗=g\(s\)\\sum\_\{a\\neq a^\{\*\}\}d\_\{a\}=\-d\_\{a^\{\*\}\}=g\(s\)andda≤0d\_\{a\}\\leq 0on𝒩\(s\)\\mathcal\{N\}\(s\), we haveg\(s\)≤∑a∈𝒫\(s\)dag\(s\)\\leq\\sum\_\{a\\in\\mathcal\{P\}\(s\)\}d\_\{a\}\. Fora∈𝒫\(s\)a\\in\\mathcal\{P\}\(s\),q\(a\)=p\(a\)\+da≥daq\(a\)=p\(a\)\+d\_\{a\}\\geq d\_\{a\}, hence∑a∈𝒫\(s\)q\(a\)≥g\(s\)\\sum\_\{a\\in\\mathcal\{P\}\(s\)\}q\(a\)\\geq g\(s\), so

ddtrπt\(s\)\|0\+≤−2g\(s\)\.\\left\.\\frac\{d\}\{dt\}r\_\{\\pi\_\{t\}\}\(s\)\\right\|\_\{0^\{\+\}\}\\leq\-2g\(s\)\.\(48\)
Step 2\.Att=0t=0,\|da∗\|=g\(s\)\|d\_\{a^\{\*\}\}\|=g\(s\)and∑a≠a∗\|da\|≥\|∑a≠a∗da\|=g\(s\)\\sum\_\{a\\neq a^\{\*\}\}\|d\_\{a\}\|\\geq\\left\|\\sum\_\{a\\neq a^\{\*\}\}d\_\{a\}\\right\|=g\(s\), hencerπsoft\(s\)≥2g\(s\)r\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(s\)\\geq 2g\(s\)\. Therefore,

ddtrπt\(s\)2\|0\+=2rπsoft\(s\)ddtrπt\(s\)\|0\+≤−8g\(s\)2,\\left\.\\frac\{d\}\{dt\}r\_\{\\pi\_\{t\}\}\(s\)^\{2\}\\right\|\_\{0^\{\+\}\}=2\\,r\_\{\\pi\_\{\\mathrm\{soft\}\}\}\(s\)\\left\.\\frac\{d\}\{dt\}r\_\{\\pi\_\{t\}\}\(s\)\\right\|\_\{0^\{\+\}\}\\leq\-8g\(s\)^\{2\},and multiplying by1X\(s\)1\_\{X\}\(s\)then taking𝔼s∼dT\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}gives Eq\.\([45](https://arxiv.org/html/2605.26246#A4.E45)\)\.

Step 3\.Letk\(t\):=∑aκ\(a∣s\)\|da\(t\)\|k\(t\):=\\sum\_\{a\}\\kappa\(a\\mid s\)\\,\|d\_\{a\}\(t\)\|\. The same case split yields

k\+′\(0\)=−κ\(a∗∣s\)\(1−q\(a∗\)\)−∑a∈𝒫\(s\)κ\(a∣s\)q\(a\)\+∑a∈𝒩\(s\)κ\(a∣s\)q\(a\)\.k^\{\\prime\}\_\{\+\}\(0\)=\-\\kappa\(a^\{\*\}\\mid s\)\(1\-q\(a^\{\*\}\)\)\-\\sum\_\{a\\in\\mathcal\{P\}\(s\)\}\\kappa\(a\\mid s\)\\,q\(a\)\+\\sum\_\{a\\in\\mathcal\{N\}\(s\)\}\\kappa\(a\\mid s\)\\,q\(a\)\.Assumption Eq\.\([42](https://arxiv.org/html/2605.26246#A4.E42)\) impliesk\+′\(0\)≤0k^\{\\prime\}\_\{\+\}\(0\)\\leq 0\. Averaging overs∼dTs\\sim d\_\{T\}and restricting with1X\(s\)1\_\{X\}\(s\)yields Eq\.\([46](https://arxiv.org/html/2605.26246#A4.E46)\)\. Summing overX∈\{ℬ,𝒢\}X\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\}proves Eq\.\([47](https://arxiv.org/html/2605.26246#A4.E47)\)\. ∎

###### Lemma D\.7\(Cross\-direction strict descent at the two pure endpoints\)\.

Under\(E1\)–\(E4\), there existsη0\>0\\eta\_\{0\}\>0such that

D𝖳S\+F\(πhard\)≤−η0,D𝖳H\+F\(πsoft\)≤−η0\.D\_\{\\mathsf\{T\}\_\{S\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{hard\}\}\)\\ \\leq\\ \-\\eta\_\{0\},\\qquad D\_\{\\mathsf\{T\}\_\{H\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{soft\}\}\)\\ \\leq\\ \-\\eta\_\{0\}\.In particular, one may takeη0=min⁡\{2C2δ¯𝒢2,8C2ΓOS2\}\\eta\_\{0\}=\\min\\\{\\,2C\_\{2\}\\underline\{\\delta\}\_\{\\mathcal\{G\}\}^\{2\},\\ 8C\_\{2\}\\Gamma\_\{\\rm OS\}^\{2\}\\,\\\}\.

###### Proof\.

Lemma[D\.5](https://arxiv.org/html/2605.26246#A4.Thmtheorem5)yields

D𝖳S\+F\(πhard\)=∑X∈\{ℬ,𝒢\}\(−KX\(πhard\)−2C2δ2,X\(πhard\)2\)≤−2C2δ2,𝒢\(πhard\)2≤−2C2δ¯𝒢2,D\_\{\\mathsf\{T\}\_\{S\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{hard\}\}\)=\\sum\_\{X\\in\\\{\\mathcal\{B\},\\mathcal\{G\}\\\}\}\\Big\(\-K\_\{X\}\(\\pi\_\{\\mathrm\{hard\}\}\)\-2C\_\{2\}\\delta\_\{2,X\}\(\\pi\_\{\\mathrm\{hard\}\}\)^\{2\}\\Big\)\\leq\-2C\_\{2\}\\delta\_\{2,\\mathcal\{G\}\}\(\\pi\_\{\\mathrm\{hard\}\}\)^\{2\}\\leq\-2C\_\{2\}\\underline\{\\delta\}\_\{\\mathcal\{G\}\}^\{2\},usingKX\(⋅\)≥0K\_\{X\}\(\\cdot\)\\geq 0and Eq\.\([43](https://arxiv.org/html/2605.26246#A4.E43)\)\. The second inequality follows from Lemma[D\.6](https://arxiv.org/html/2605.26246#A4.Thmtheorem6), which givesD𝖳H\+F\(πsoft\)≤−8C2ΓOS2D\_\{\\mathsf\{T\}\_\{H\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{soft\}\}\)\\leq\-8C\_\{2\}\\Gamma\_\{\\rm OS\}^\{2\}\. ∎

Hybrid feasible update\.Forλ∈\[0,1\]\\lambda\\in\[0,1\], define

qλ\(⋅∣s\)=\(1−λ\)δa∗\(⋅\)\+λπT\(⋅∣s\),\(𝖳λtπ\)\(⋅∣s\)=\(1−t\)π\(⋅∣s\)\+tqλ\(⋅∣s\),q\_\{\\lambda\}\(\\cdot\\mid s\)=\(1\-\\lambda\)\\delta\_\{a^\{\*\}\}\(\\cdot\)\+\\lambda\\pi\_\{T\}\(\\cdot\\mid s\),\\qquad\(\\mathsf\{T\}\_\{\\lambda\}^\{t\}\\pi\)\(\\cdot\\mid s\)=\(1\-t\)\\pi\(\\cdot\\mid s\)\+t\\,q\_\{\\lambda\}\(\\cdot\\mid s\),so that𝖳λtπ=π\+tvλ\(π\)\\mathsf\{T\}\_\{\\lambda\}^\{t\}\\pi=\\pi\+t\\,v\_\{\\lambda\}\(\\pi\)with

vλ\(π\)=\(1−λ\)vH\(π\)\+λvS\(π\)\.v\_\{\\lambda\}\(\\pi\)=\(1\-\\lambda\)v\_\{H\}\(\\pi\)\+\\lambda v\_\{S\}\(\\pi\)\.\(49\)
Policy metric \(avoids notation clash\)\.Define thedTd\_\{T\}\-weightedℓ1\\ell\_\{1\}norm

∥U∥1,dT:=𝔼s∼dT\[∥U\(⋅∣s\)∥1\]=𝔼s∼dT\[∑a∈𝒱\|U\(a∣s\)\|\]\.\\\|U\\\|\_\{1,d\_\{T\}\}\\ :=\\ \\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\Big\[\\\|U\(\\cdot\\mid s\)\\\|\_\{1\}\\Big\]\\ =\\ \\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\\\!\\Big\[\\sum\_\{a\\in\\mathcal\{V\}\}\|U\(a\\mid s\)\|\\Big\]\.\(50\)For brevity, write‖U‖:=‖U‖1,dT\\\|U\\\|:=\\\|U\\\|\_\{1,d\_\{T\}\}\.

\(S1\) Approximate endpoint fixed points \(in∥⋅∥1,dT\\\|\\cdot\\\|\_\{1,d\_\{T\}\}\)\.

‖vH\(πhard\)‖≤ε,‖vS\(πsoft\)‖≤ε\.\\\|v\_\{H\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\\|\\leq\\varepsilon,\\qquad\\\|v\_\{S\}\(\\pi\_\{\\mathrm\{soft\}\}\)\\\|\\leq\\varepsilon\.\(51\)
A convexity fact: hybrid is a convex combination of the two pure updates\.Note that for everyπ,t,λ\\pi,t,\\lambdawe have the identity

𝖳λtπ=\(1−λ\)𝖳Htπ\+λ𝖳Stπ,\\mathsf\{T\}\_\{\\lambda\}^\{t\}\\pi=\(1\-\\lambda\)\\,\\mathsf\{T\}\_\{H\}^\{t\}\\pi\+\\lambda\\,\\mathsf\{T\}\_\{S\}^\{t\}\\pi,\(52\)since both sides equal\(1−t\)π\+t\(\(1−λ\)δa∗\+λπT\)\(1\-t\)\\pi\+t\\bigl\(\(1\-\\lambda\)\\delta\_\{a^\{\*\}\}\+\\lambda\\pi\_\{T\}\\bigr\)pointwise inss\.

###### Lemma D\.8\(Convexity ofFFand a hybrid directional\-derivative inequality\)\.

By construction of the boundFFdefined in Thm\.[4\.2](https://arxiv.org/html/2605.26246#S4.SS2)\(restated above\),FFis convex inπ\\pi\. Consequently, for any policyπ\\piand anyλ∈\[0,1\]\\lambda\\in\[0,1\],

D𝖳λ\+F\(π\)≤\(1−λ\)D𝖳H\+F\(π\)\+λD𝖳S\+F\(π\)\.D\_\{\\mathsf\{T\}\_\{\\lambda\}\}^\{\+\}F\(\\pi\)\\ \\leq\\ \(1\-\\lambda\)\\,D\_\{\\mathsf\{T\}\_\{H\}\}^\{\+\}F\(\\pi\)\+\\lambda\\,D\_\{\\mathsf\{T\}\_\{S\}\}^\{\+\}F\(\\pi\)\.\(53\)

###### Proof\.

Fix any prefixss\. The pointwise integrand

π\(⋅∣s\)↦∑a∈𝒱κ\(a∣s\)\|π\(a∣s\)−πT\(a∣s\)\|\+C2∥π\(⋅∣s\)−πT\(⋅∣s\)∥12\\pi\(\\cdot\\mid s\)\\ \\mapsto\\ \\sum\_\{a\\in\\mathcal\{V\}\}\\kappa\(a\\mid s\)\\,\|\\pi\(a\\mid s\)\-\\pi\_\{T\}\(a\\mid s\)\|\\;\+\\;C\_\{2\}\\\|\\pi\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)\\\|\_\{1\}^\{2\}is convex because it is a nonnegative weighted sum of absolute values plus a squared norm\. Taking𝔼s∼dT\[1X\(s\)⋅\]\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\[1\_\{X\}\(s\)\\cdot\]preserves convexity, hence eachFXF\_\{X\}and thusF=Fℬ\+F𝒢F=F\_\{\\mathcal\{B\}\}\+F\_\{\\mathcal\{G\}\}is convex\.

Using Eq\.\([52](https://arxiv.org/html/2605.26246#A4.E52)\) and convexity ofFF,

F\(𝖳λtπ\)≤\(1−λ\)F\(𝖳Htπ\)\+λF\(𝖳Stπ\)\.F\(\\mathsf\{T\}\_\{\\lambda\}^\{t\}\\pi\)\\leq\(1\-\\lambda\)F\(\\mathsf\{T\}\_\{H\}^\{t\}\\pi\)\+\\lambda F\(\\mathsf\{T\}\_\{S\}^\{t\}\\pi\)\.SubtractF\(π\)=\(1−λ\)F\(π\)\+λF\(π\)F\(\\pi\)=\(1\-\\lambda\)F\(\\pi\)\+\\lambda F\(\\pi\)from both sides, divide byt\>0t\>0, and taket→0\+t\\to 0^\{\+\}\. The right directional derivatives exist by Lemma[D\.4](https://arxiv.org/html/2605.26246#A4.Thmtheorem4)\(applied inside the definitions ofKXK\_\{X\}andδ2,X\\delta\_\{2,X\}\), yielding Eq\.\([53](https://arxiv.org/html/2605.26246#A4.E53)\)\. ∎

###### Lemma D\.9\(A Lipschitz upper bound forFF\)\.

Letκmax:=sups,aκ\(a∣s\)<∞\\kappa\_\{\\max\}:=\\sup\_\{s,a\}\\kappa\(a\\mid s\)<\\infty\. Then for any policiesπ,π′\\pi,\\pi^\{\\prime\},

\|F\(π′\)−F\(π\)\|≤LLip‖π′−π‖,LLip:=κmax\+4C2\.\|F\(\\pi^\{\\prime\}\)\-F\(\\pi\)\|\\ \\leq\\ L\_\{\\mathrm\{Lip\}\}\\ \\\|\\pi^\{\\prime\}\-\\pi\\\|,\\qquad L\_\{\\mathrm\{Lip\}\}:=\\kappa\_\{\\max\}\+4C\_\{2\}\.\(54\)In particular, for any feasible operator𝖳tπ=π\+tv\(π\)\\mathsf\{T\}^\{t\}\\pi=\\pi\+t\\,v\(\\pi\),

D𝖳\+F\(π\)≤LLip‖v\(π\)‖\.D\_\{\\mathsf\{T\}\}^\{\+\}F\(\\pi\)\\ \\leq\\ L\_\{\\mathrm\{Lip\}\}\\,\\\|v\(\\pi\)\\\|\.\(55\)

###### Proof\.

Fixssand writeΔ\(⋅∣s\)=π\(⋅∣s\)−πT\(⋅∣s\)\\Delta\(\\cdot\\mid s\)=\\pi\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)andΔ′\(⋅∣s\)=π′\(⋅∣s\)−πT\(⋅∣s\)\\Delta^\{\\prime\}\(\\cdot\\mid s\)=\\pi^\{\\prime\}\(\\cdot\\mid s\)\-\\pi\_\{T\}\(\\cdot\\mid s\)\. For the linear term, using\|\|x\|−\|y\|\|≤\|x−y\|\|\|x\|\-\|y\|\|\\leq\|x\-y\|,

\|∑aκ\(a∣s\)\|Δ′\(a∣s\)\|−∑aκ\(a∣s\)\|Δ\(a∣s\)\|\|≤∑aκ\(a∣s\)\|π′\(a∣s\)−π\(a∣s\)\|≤κmax∥π′\(⋅∣s\)−π\(⋅∣s\)∥1\.\\Big\|\\sum\_\{a\}\\kappa\(a\\mid s\)\|\\Delta^\{\\prime\}\(a\\mid s\)\|\-\\sum\_\{a\}\\kappa\(a\\mid s\)\|\\Delta\(a\\mid s\)\|\\Big\|\\leq\\sum\_\{a\}\\kappa\(a\\mid s\)\\,\|\\pi^\{\\prime\}\(a\\mid s\)\-\\pi\(a\\mid s\)\|\\leq\\kappa\_\{\\max\}\\,\\\|\\pi^\{\\prime\}\(\\cdot\\mid s\)\-\\pi\(\\cdot\\mid s\)\\\|\_\{1\}\.For the quadratic term, note∥Δ\(⋅∣s\)∥1≤2\\\|\\Delta\(\\cdot\\mid s\)\\\|\_\{1\}\\leq 2and∥Δ′\(⋅∣s\)∥1≤2\\\|\\Delta^\{\\prime\}\(\\cdot\\mid s\)\\\|\_\{1\}\\leq 2\(difference of two distributions\), hence

\|∥Δ′∥12−∥Δ∥12\|=\(∥Δ′∥1\+∥Δ∥1\)\|∥Δ′∥1−∥Δ∥1\|≤4∥Δ′−Δ∥1=4∥π′\(⋅∣s\)−π\(⋅∣s\)∥1\.\\big\|\\\|\\Delta^\{\\prime\}\\\|\_\{1\}^\{2\}\-\\\|\\Delta\\\|\_\{1\}^\{2\}\\big\|=\\big\(\\\|\\Delta^\{\\prime\}\\\|\_\{1\}\+\\\|\\Delta\\\|\_\{1\}\\big\)\\,\\big\|\\\|\\Delta^\{\\prime\}\\\|\_\{1\}\-\\\|\\Delta\\\|\_\{1\}\\big\|\\leq 4\\,\\\|\\Delta^\{\\prime\}\-\\Delta\\\|\_\{1\}=4\\,\\\|\\pi^\{\\prime\}\(\\cdot\\mid s\)\-\\pi\(\\cdot\\mid s\)\\\|\_\{1\}\.Multiply byC2C\_\{2\}, sum the two bounds, multiply by1X\(s\)≤11\_\{X\}\(s\)\\leq 1, and take𝔼s∼dT\\mathbb\{E\}\_\{s\\sim d\_\{T\}\}\. This yields Eq\.\([54](https://arxiv.org/html/2605.26246#A4.E54)\)\.

For Eq\.\([55](https://arxiv.org/html/2605.26246#A4.E55)\), apply Eq\.\([54](https://arxiv.org/html/2605.26246#A4.E54)\) toπ′=𝖳tπ=π\+tv\(π\)\\pi^\{\\prime\}=\\mathsf\{T\}^\{t\}\\pi=\\pi\+tv\(\\pi\):

F\(𝖳tπ\)−F\(π\)t≤LLip‖tv\(π\)‖t=LLip‖v\(π\)‖\.\\frac\{F\(\\mathsf\{T\}^\{t\}\\pi\)\-F\(\\pi\)\}\{t\}\\leq\\frac\{L\_\{\\mathrm\{Lip\}\}\\\|t\\,v\(\\pi\)\\\|\}\{t\}=L\_\{\\mathrm\{Lip\}\}\\\|v\(\\pi\)\\\|\.Takingt→0\+t\\to 0^\{\+\}gives the claim\. ∎

###### Lemma D\.10\(Negative right directional derivative implies strict decrease\)\.

Letϕ:\[0,1\]→ℝ\\phi:\[0,1\]\\to\\mathbb\{R\}be right\-differentiable at0and supposeϕ\+′\(0\)<0\\phi^\{\\prime\}\_\{\+\}\(0\)<0\. Then there existst0\>0t\_\{0\}\>0such thatϕ\(t\)<ϕ\(0\)\\phi\(t\)<\\phi\(0\)for allt∈\(0,t0\]t\\in\(0,t\_\{0\}\]\.

###### Proof\.

By definition of the right derivative,limt→0\+ϕ\(t\)−ϕ\(0\)t=ϕ\+′\(0\)<0\\lim\_\{t\\to 0^\{\+\}\}\\frac\{\\phi\(t\)\-\\phi\(0\)\}\{t\}=\\phi^\{\\prime\}\_\{\+\}\(0\)<0\. Hence there existst0\>0t\_\{0\}\>0such that for allt∈\(0,t0\]t\\in\(0,t\_\{0\}\],

ϕ\(t\)−ϕ\(0\)t<12ϕ\+′\(0\)<0,\\frac\{\\phi\(t\)\-\\phi\(0\)\}\{t\}\\ <\\ \\frac\{1\}\{2\}\\phi^\{\\prime\}\_\{\+\}\(0\)\\ <\\ 0,which impliesϕ\(t\)<ϕ\(0\)\\phi\(t\)<\\phi\(0\)\. ∎

##### D\.2\.3Strict improvement by a one\-step Hybrid update

###### Theorem D\.11\(Strict improvement by a one\-step Hybrid update\)\.

Assume Lemma[D\.7](https://arxiv.org/html/2605.26246#A4.Thmtheorem7), Eq\.\([51](https://arxiv.org/html/2605.26246#A4.E51)\), andκmax<∞\\kappa\_\{\\max\}<\\infty\. Define

νS:=2C2δ¯𝒢2,νH:=8C2ΓOS2,LLip:=κmax\+4C2\.\\nu\_\{S\}:=2C\_\{2\}\\underline\{\\delta\}\_\{\\mathcal\{G\}\}^\{2\},\\qquad\\nu\_\{H\}:=8C\_\{2\}\\Gamma\_\{\\rm OS\}^\{2\},\\qquad L\_\{\\mathrm\{Lip\}\}:=\\kappa\_\{\\max\}\+4C\_\{2\}\.If

\(LLipε\)2<νSνH,\(L\_\{\\mathrm\{Lip\}\}\\varepsilon\)^\{2\}\\ <\\ \\nu\_\{S\}\\nu\_\{H\},\(56\)then there existλ∈\(0,1\)\\lambda\\in\(0,1\)andt∈\(0,1\]t\\in\(0,1\]such that, letting

π∗∈arg⁡min⁡\{F\(πhard\),F\(πsoft\)\},πhyb:=𝖳λtπ∗,\\pi\_\{\*\}\\in\\arg\\min\\\{F\(\\pi\_\{\\mathrm\{hard\}\}\),F\(\\pi\_\{\\mathrm\{soft\}\}\)\\\},\\qquad\\pi\_\{\\mathrm\{hyb\}\}:=\\mathsf\{T\}\_\{\\lambda\}^\{t\}\\pi\_\{\*\},we have

F\(πhyb\)<min⁡\{F\(πhard\),F\(πsoft\)\}\.F\(\\pi\_\{\\mathrm\{hyb\}\}\)\\ <\\ \\min\\\{F\(\\pi\_\{\\mathrm\{hard\}\}\),F\(\\pi\_\{\\mathrm\{soft\}\}\)\\\}\.

###### Proof\.

Step 1 \(a uniform negative hybrid\-direction derivative at both endpoints\)\.By Lemma[D\.8](https://arxiv.org/html/2605.26246#A4.Thmtheorem8),

D𝖳λ\+F\(π\)≤\(1−λ\)D𝖳H\+F\(π\)\+λD𝖳S\+F\(π\),∀π\.D\_\{\\mathsf\{T\}\_\{\\lambda\}\}^\{\+\}F\(\\pi\)\\leq\(1\-\\lambda\)D\_\{\\mathsf\{T\}\_\{H\}\}^\{\+\}F\(\\pi\)\+\\lambda D\_\{\\mathsf\{T\}\_\{S\}\}^\{\+\}F\(\\pi\),\\qquad\\forall\\pi\.
Atπhard\\pi\_\{\\mathrm\{hard\}\}, Lemma[D\.7](https://arxiv.org/html/2605.26246#A4.Thmtheorem7)givesD𝖳S\+F\(πhard\)≤−νSD\_\{\\mathsf\{T\}\_\{S\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{hard\}\}\)\\leq\-\\nu\_\{S\}\. Also, by Lemma[D\.9](https://arxiv.org/html/2605.26246#A4.Thmtheorem9)and Eq\.\([51](https://arxiv.org/html/2605.26246#A4.E51)\),

D𝖳H\+F\(πhard\)≤LLip‖vH\(πhard\)‖≤LLipε\.D\_\{\\mathsf\{T\}\_\{H\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{hard\}\}\)\\leq L\_\{\\mathrm\{Lip\}\}\\\|v\_\{H\}\(\\pi\_\{\\mathrm\{hard\}\}\)\\\|\\leq L\_\{\\mathrm\{Lip\}\}\\varepsilon\.Therefore,

D𝖳λ\+F\(πhard\)≤\(1−λ\)LLipε−λνS\.D\_\{\\mathsf\{T\}\_\{\\lambda\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{hard\}\}\)\\leq\(1\-\\lambda\)L\_\{\\mathrm\{Lip\}\}\\varepsilon\-\\lambda\\nu\_\{S\}\.\(57\)
Similarly, atπsoft\\pi\_\{\\mathrm\{soft\}\}, Lemma[D\.7](https://arxiv.org/html/2605.26246#A4.Thmtheorem7)givesD𝖳H\+F\(πsoft\)≤−νHD\_\{\\mathsf\{T\}\_\{H\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{soft\}\}\)\\leq\-\\nu\_\{H\}and Lemma[D\.9](https://arxiv.org/html/2605.26246#A4.Thmtheorem9)plus Eq\.\([51](https://arxiv.org/html/2605.26246#A4.E51)\) givesD𝖳S\+F\(πsoft\)≤LLipεD\_\{\\mathsf\{T\}\_\{S\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{soft\}\}\)\\leq L\_\{\\mathrm\{Lip\}\}\\varepsilon\. Hence,

D𝖳λ\+F\(πsoft\)≤λLLipε−\(1−λ\)νH\.D\_\{\\mathsf\{T\}\_\{\\lambda\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{soft\}\}\)\\leq\\lambda L\_\{\\mathrm\{Lip\}\}\\varepsilon\-\(1\-\\lambda\)\\nu\_\{H\}\.\(58\)
Chooseλ\\lambdasuch that

λ\>LLipενS\+LLipε,λ<νHνH\+LLipε\.\\lambda\>\\frac\{L\_\{\\mathrm\{Lip\}\}\\varepsilon\}\{\\nu\_\{S\}\+L\_\{\\mathrm\{Lip\}\}\\varepsilon\},\\qquad\\lambda<\\frac\{\\nu\_\{H\}\}\{\\nu\_\{H\}\+L\_\{\\mathrm\{Lip\}\}\\varepsilon\}\.This is possible iff Eq\.\([56](https://arxiv.org/html/2605.26246#A4.E56)\) holds\. Fix any suchλ\\lambda\. Then Eq\.\([57](https://arxiv.org/html/2605.26246#A4.E57)\)–\([58](https://arxiv.org/html/2605.26246#A4.E58)\) imply

D𝖳λ\+F\(πhard\)<0,D𝖳λ\+F\(πsoft\)<0\.D\_\{\\mathsf\{T\}\_\{\\lambda\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{hard\}\}\)<0,\\qquad D\_\{\\mathsf\{T\}\_\{\\lambda\}\}^\{\+\}F\(\\pi\_\{\\mathrm\{soft\}\}\)<0\.\(59\)
Step 2 \(convert negative derivative into a strict one\-step decrease\)\.Letπ∗∈arg⁡min⁡\{F\(πhard\),F\(πsoft\)\}\\pi\_\{\*\}\\in\\arg\\min\\\{F\(\\pi\_\{\\mathrm\{hard\}\}\),F\(\\pi\_\{\\mathrm\{soft\}\}\)\\\}and defineϕ\(t\):=F\(𝖳λtπ∗\)\\phi\(t\):=F\(\\mathsf\{T\}\_\{\\lambda\}^\{t\}\\pi\_\{\*\}\)\. By Eq\.\([59](https://arxiv.org/html/2605.26246#A4.E59)\), we haveϕ\+′\(0\)=D𝖳λ\+F\(π∗\)<0\\phi^\{\\prime\}\_\{\+\}\(0\)=D\_\{\\mathsf\{T\}\_\{\\lambda\}\}^\{\+\}F\(\\pi\_\{\*\}\)<0\. Lemma[D\.10](https://arxiv.org/html/2605.26246#A4.Thmtheorem10)then yields at0\>0t\_\{0\}\>0such thatϕ\(t\)<ϕ\(0\)\\phi\(t\)<\\phi\(0\)for allt∈\(0,t0\]t\\in\(0,t\_\{0\}\]\.

Pick anyt∈\(0,min⁡\{1,t0\}\]t\\in\(0,\\min\\\{1,t\_\{0\}\\\}\]and setπhyb:=𝖳λtπ∗\\pi\_\{\\mathrm\{hyb\}\}:=\\mathsf\{T\}\_\{\\lambda\}^\{t\}\\pi\_\{\*\}\. Then

F\(πhyb\)=ϕ\(t\)<ϕ\(0\)=F\(π∗\)=min⁡\{F\(πhard\),F\(πsoft\)\},F\(\\pi\_\{\\mathrm\{hyb\}\}\)=\\phi\(t\)<\\phi\(0\)=F\(\\pi\_\{\*\}\)=\\min\\\{F\(\\pi\_\{\\mathrm\{hard\}\}\),F\(\\pi\_\{\\mathrm\{soft\}\}\)\\\},which proves the theorem\. ∎

### Appendix EDetailed Experimental Settings

In this section, we provide comprehensive details regarding the dataset construction, training hyperparameters, and evaluation protocols used in our experiments\.

#### E\.1Training Data Construction

We describe the data construction process across distinct training settings\. A primary challenge lies in ensuring the diversity of input queries, where corresponding responses are generated by the respective teacher models for each teacher\-student configuration\.

###### General Reasoning Data Construction

Our training data is constructed starting from Infinity Instruct\(Liet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib3)\)\. To ensure a balanced capability profile for the student model, we curated a high\-quality subset by applying a filtering pipeline based on the providedtopictags and quality scores\. We specifically retained samples with high quality ratings across several core domains to maintain data diversity:

- •Instruction Following & General Chat:Includes dialogue, literature, and reading comprehension\. This subset maintains fundamental world knowledge and responsiveness to human instructions\.
- •Mathematical & Logical Reasoning:Focuses on multi\-step mathematical problems\. This partition preserves complex reasoning chains and logical consistency\.
- •Academic & Scientific Expertise:Covers specialized subjects from MMLU\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib79)\)\(e\.g\., STEM, humanities\) and scientific inquiries from ARC\-C\(Clarket al\.,[2018](https://arxiv.org/html/2605.26246#bib.bib46)\)\. This targets expert\-level factual knowledge and scientific precision\.

Through strategic balancing of these domains, we curated a final dataset of approximately66,000 samples\. This size was chosen empirically to balance computational efficiency with sufficient diversity for smooth knowledge transfer\.

###### Mathematical Reasoning Data Construction

We constructed the mathematical reasoning dataset by sampling and aggregating instances from MetaMathQA\(Yuet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib42)\), DAPO\-Math\-17K\(Yuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib40)\), and the DeepScaleR\-Preview\-Dataset\(Luoet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib41)\)\. This composite corpus covers a broad spectrum of difficulty levels, resulting in a final dataset of approximately 50,000 samples\.

###### Coding Data Construction

We leverage the coding dataset from\(Menget al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib35)\)\. Derived from CodeFeedback\(Zhenget al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib34)\), this subset contains 156,526 samples selected to target Python function\-generation tasks similar to HumanEval/MBPP\.

#### E\.2Training Configurations

All distillation experiments are implemented using PyTorch and the Hugging Facetransformersandtrllibraries\. We leverage DeepSpeed ZeRO\-2 for efficient distributed training across NVIDIA A100 GPUs\. The specific hyperparameters are listed below:

Table 6:Hyperparameters for distillation\.Common settings:Batch Size = 128, Sequence Length = 2048, LoRA Dropout = 0\.05, Target Modules = All Linear Layers\.DomainTeacher→\\toStudentTrainingLoRALREpochsRank \(rr\)Alpha \(α\\alpha\)GeneralQwen2\.5 \(7B→\\to\{0\.5, 1\.5, 3\}B\)5e−55\\mathrm\{e\}\{\-5\}21632Llama\-3\.1 8B→\\toLlama\-3\.2 1B1e−41\\mathrm\{e\}\{\-4\}21632Gemma\-3 \(4B→\\to1B\)1e−41\\mathrm\{e\}\{\-4\}21632MathQwen2\.5\-Math \(7B→\\to1\.5B\)5e−55\\mathrm\{e\}\{\-5\}21632CodeDeepSeek\-Coder \(6\.7B→\\to1\.3B\)2e−52\\mathrm\{e\}\{\-5\}1128128We implement the on\-policy KD baseline following the official setup of\(Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73); Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18)\)\. We mix a fixed dataset with student\-generated samples based on a threshold\. In our experiments, we use student samples when the threshold is below 0\.3; otherwise, we train on teacher outputs\. For fair comparison, all experiments are run for the same number of epochs on four A100 GPUs, and the average training cost \(s/step\) is then calculated\.

Finally, although the experiments presented in the main text have demonstrated the superior performance of the adaptive hybrid supervision approach, we still provide the fixed lambda values tested across different settings to facilitate reproducibility\. We also hope this work will offer the community further insights for designing more principled and effective learning objectives\. Additionally, in our curriculum scheduling strategy, the warm\-up steps are set to 20% of the total training steps\. The results are listed in Table[7](https://arxiv.org/html/2605.26246#A5.T7)\.

Table 7:Fixed lambda values used in distillation experiments\.DomainTeacher→\\toStudentλ\\lambdaGeneralQwen2\.5\-7B→\\toQwen2\.5\-0\.5B0\.95Qwen2\.5\-7B→\\toQwen2\.5\-1\.5B0\.9Qwen2\.5\-7B→\\toQwen2\.5\-3B0\.05Llama\-3\.1\-8B→\\toLlama\-3\.2\-1B0\.95Gemma\-3\-4B→\\toGemma\-3\-1B0\.95MathQwen2\.5\-Math\-7B→\\toQwen2\.5\-1\.5B0\.9CodeDeepSeek\-Coder\-6\.7B→\\toDeepSeek\-Coder\-1\.3B0\.1
#### E\.3Evaluation Benchmarks

We evaluate the models on widely recognized benchmarks to assess their reasoning capabilities\. To ensure reproducibility and stability, we report the average accuracy score across five independent runs using random seeds\{10,20,30,40,50\}\\\{10,20,30,40,50\\\}\.

General Reasoning\.We set the maximum generation length to 2048 tokens\. For other generation parameters, we adopt the official configurations recommended by the model providers to achieve a proper trade\-off between diversity and accuracy\. We include the following benchmarks in this category:

- •MMLU\(Hendryckset al\.,[2020](https://arxiv.org/html/2605.26246#bib.bib48)\): Evaluates multitask accuracy across 57 subjects, ranging from elementary mathematics to professional law\.
- •BBH\(Suzgunet al\.,[2022](https://arxiv.org/html/2605.26246#bib.bib33)\): A subset of 23 challenging tasks from BIG\-Bench designed to test complex multi\-step reasoning capabilities\.
- •ARC\-C\(Clarket al\.,[2018](https://arxiv.org/html/2605.26246#bib.bib46)\): The Challenge Set of the AI2 Reasoning Challenge, consisting of grade\-school science questions that require deep reasoning\.

The prompt template used for these evaluations is as follows:

Prompt Template for Multiple\-ChoiceSystem Message:
You are an expert AI assistant that solves multiple\-choice questions\. Follow these steps for your response:1\.First, provide a detailed, step\-by\-step reasoning process that explains how you arrived at the solution\.2\.After your reasoning, conclude with the final answer\.3\.The final answer must be formatted by enclosing ONLY the single capital letter of the correct option in\\boxed\{\}\. For example, if the correct answer is option B, your final line should be exactly:\\boxed\{B\}User Message:
<Input Question Text Here\>A\.<Option A Text\> B\.<Option B Text\> C\.<Option C Text\> D\.<Option D Text\>

Model Response:
<Detailed step\-by\-step reasoning process explaining the solution\.\.\.\>\\boxed\{<Correct Option Letter\>\}

Figure 7:The complete interaction template used for multiple\-choice benchmarks\. The model is instructed to first generate a reasoning chain and then output the final answer in a specific format to facilitate automatic parsing\.Mathematical Reasoning\.We maintain consistent generation configurations with the general reasoning tasks described above\. To elicit intermediate reasoning steps, we employ Chain\-of\-Thought prompting\. For evaluation, we adopt the validation protocol from theverlframework111[https://github\.com/volcengine/verl](https://github.com/volcengine/verl): the final answer is extracted from the generated\\boxed\{\}content and compared against the ground truth using exact matching\. The benchmarks included in this category are:

- •GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib56)\): A benchmark of high\-quality grade school math word problems focusing on multi\-step reasoning\.
- •MATH\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib79)\): A dataset containing challenging competition\-level mathematics problems spanning seven diverse disciplines\.
- •Gaokao2023\-En\(Liaoet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib36)\): English\-translated questions from the 2023 Chinese National College Entrance Examination, evaluating advanced high\-school mathematical proficiency\.

The prompt template used for these mathematical tasks is presented in Figure[8](https://arxiv.org/html/2605.26246#A5.F8)\.

Prompt Template for Mathematical ReasoningSystem Message:
Please reason step by step, and put your final answer WITHIN\\boxed\{\}\.User Message:
<Mathematical Problem Text\>Model Response:
<Step\-by\-step derivation and calculation process\.\.\.\>Therefore, the answer is\\boxed\{<Final Numerical or Symbolic Answer\>\}\.

Figure 8:The zero\-shot Chain\-of\-Thought prompt used for mathematical reasoning tasks\. The concise instruction forces the model to generate intermediate steps before producing a boxed final answer for exact\-match validation\.Code Generation\.We employ theEvalPlusframework222[https://github\.com/evalplus/evalplus](https://github.com/evalplus/evalplus)to assess programming capabilities\. This framework evaluates functional correctness using an augmented set of rigorous unit tests to prevent false positives\. We report results under greedy decoding following the official setup to ensure high performance and reproducibility\.

- •HumanEval\(Chen,[2021](https://arxiv.org/html/2605.26246#bib.bib80)\): Measures code generation capabilities on 164 hand\-written Python problems via the Pass@1 accuracy\.
- •MBPP\(Austinet al\.,[2021](https://arxiv.org/html/2605.26246#bib.bib81)\): Evaluates functional correctness on entry\-level programming problems using the sanitized subset\.

### Appendix FAdditional Experimental Results

#### F\.1Extended Evaluation and Control Experiments

This section complements the main experiments by asking four concrete questions\. First, does the hard–soft pattern hold beyond the primary model pairs and benchmark formats? Second, can hybrid supervision be combined with on\-policy prefixes? Third, can the gains be explained by cheaper alternatives such as reversing the proxy, changing temperature, or adding global regularization? Fourth, in a setting where exact token\-levelκ\\kappais computable, do high\-sensitivity tokens coincide with the semantic decision states predicted by the Bridge–Garden decomposition?

###### Broader model and generation settings\.

Table[8](https://arxiv.org/html/2605.26246#A6.T8)tests whether the main trend is tied to the original model pairs or to closed\-form benchmark evaluation\. Hybrid KD remains better than both pure hard and pure soft KD for a larger Qwen2\.5 teacher–student capacity gap, an additional Qwen2\.5\-Coder pair, and open\-ended AlpacaEval generation\.

Table 8:Broader model and open\-ended evaluations\. For Qwen2\.5\-32B→\\to3B, the four columns are BBH/MMLU/ARC\-C/ThmQA\. For Qwen2\.5\-Coder\-7B→\\to1\.5B, they are HumanEval/HumanEval\+/MBPP/MBPP\+\. AlpacaEval reports GPT\-5\.2\-judged win rate against text\-davinci\-003\.SettingMethodBBH/HEMMLU/HE\+ARC\-C/MBPPThmQA/MBPP\+Avg\./WinQwen2\.5\-32B→\\to3BHard KD34\.467\.279\.523\.151\.0Soft KD44\.065\.678\.122\.952\.6Hybrid KD45\.766\.880\.024\.054\.1Qwen2\.5\-Coder\-7B→\\to1\.5BHard KD54\.350\.060\.352\.154\.2Soft KD52\.747\.959\.652\.453\.1Hybrid KD55\.550\.661\.452\.655\.0AlpacaEval \(Qwen2\.5\-7B→\\to3B\)Hard KDWin rate57\.5Soft KDWin rate61\.3Hybrid KDWin rate64\.4
###### Compatibility with on\-policy training\.

On\-policy KD changes which prefixes are used for training, whereas Hybrid KD changes the supervision target at a given prefix\. These two changes address different parts of the training procedure and can be combined\. Tables[9](https://arxiv.org/html/2605.26246#A6.T9)and[10](https://arxiv.org/html/2605.26246#A6.T10)show that adding hybrid supervision on top of on\-policy soft KD improves the average score across all four settings\.

Table 9:Hybrid supervision combined with on\-policy training on reasoning benchmarks\. The Qwen2\.5\-7B→\\to3B setting is reported as benchmark average; Llama3\.1\-8B→\\to1B reports the full BBH/MMLU/ARC\-C/ThmQA breakdown\.SettingMethodBBHMMLUARC\-CThmQAAvg\.Qwen2\.5\-7B→\\to3BOff\-policy soft KDAvg\. only51\.9Off\-policy hybrid KDAvg\. only53\.5On\-policy soft KDAvg\. only52\.8On\-policy \+ hybrid KDAvg\. only54\.7Llama3\.1\-8B→\\to1BOff\-policy soft KD22\.133\.133\.44\.423\.3Off\-policy hybrid KD27\.435\.637\.05\.026\.3On\-policy soft KD26\.836\.239\.77\.027\.4On\-policy \+ hybrid KD27\.236\.740\.27\.127\.8Table 10:Hybrid supervision combined with on\-policy training on code benchmarks\. The columns are HumanEval \(HE\), HumanEval\+ \(HE\+\), MBPP, MBPP\+, and their average\.SettingMethodHEHE\+MBPPMBPP\+Avg\.Qwen2\.5\-Coder\-7B→\\to1\.5BOff\-policy soft KD52\.747\.959\.652\.453\.1Off\-policy hybrid KD55\.550\.661\.452\.655\.0On\-policy soft KD56\.451\.561\.453\.255\.6On\-policy \+ hybrid KD56\.852\.961\.152\.955\.9DeepSeek\-Coder\-6\.7B→\\to1\.3BOff\-policy soft KD38\.433\.563\.551\.646\.8Off\-policy hybrid KD41\.536\.663\.250\.548\.0On\-policy soft KD43\.339\.063\.852\.149\.5On\-policy \+ hybrid KD44\.239\.864\.452\.550\.2
###### Reverse\-proxy ablation\.

The practical algorithms use teacher confidence or entropy as local heuristics for allocating supervision at semantic decision states\. If Hybrid KD only benefited from generic label mixing, then reversing the confidence or entropy rule should remain competitive\. Table[11](https://arxiv.org/html/2605.26246#A6.T11)shows the opposite: reverse\-confidence and reverse\-entropy both reduce the average score on Qwen2\.5\-7B→\\to3B\. This supports that the direction of the local hard/soft allocation matters\.

Table 11:Reverse\-proxy ablation on Qwen2\.5\-7B→\\to3B\. Reversing confidence or entropy weighting reduces the average score\.MethodBBHMMLUARC\-CThmQAAvg\.Hard KD41\.565\.878\.823\.852\.5Soft KD41\.764\.578\.323\.051\.9Hybrid: confidence\-based44\.167\.580\.822\.853\.8Hybrid: reverse\-confidence41\.565\.378\.122\.351\.8Hybrid: entropy46\.867\.179\.823\.754\.3Hybrid: reverse\-entropy41\.265\.277\.320\.551\.1
###### Training cost breakdown\.

All methods in Table[12](https://arxiv.org/html/2605.26246#A6.T12)use the same epochs, batch size, and hardware\. Teacher logits for soft and hybrid supervision are computed on the fly rather than stored\. Hybrid KD reuses the same teacher logits as soft KD and only adds a lightweight per\-token weighting step\. By contrast, on\-policy KD requires autoregressive student sampling, which accounts for most of its additional cost\.

Table 12:Training cost on Qwen2\.5\-7B→\\to3B with 4×\\timesA100 80GB\.MethodExtra computations/stepHard KDnone8\.30Soft KDnone14\.06Hybrid KDone log\-sum\-exp per token15\.24On\-policy KDautoregressive student sampling147\.83
###### Regularization and temperature controls\.

We next test whether the gains can be reproduced without position\-specific hard/soft allocation\. Entropy regularization and temperature schedules provide global changes to the target distribution, and random\-label mixing tests whether any hard\-token signal is sufficient\. Tables[13](https://arxiv.org/html/2605.26246#A6.T13)and[14](https://arxiv.org/html/2605.26246#A6.T14)show the full benchmark breakdowns\. These alternatives can improve over pure soft KD, but they do not consistently match Hybrid KD\. The random\-label control performs especially poorly, indicating that the hard\-label component must remain teacher\-supported and context\-dependent\.

Table 13:Regularization and temperature controls on reasoning benchmarks\. Avg\. is computed over BBH, MMLU, ARC\-C, and ThmQA\. Hybrid KD is best by average in both settings\.SettingMethodBBHMMLUARC\-CThmQAAvg\.Qwen2\.5\-7B→\\to3BSoft KD41\.764\.578\.323\.051\.9Entropy reg\.46\.667\.080\.523\.854\.5T: high→\\tolow45\.266\.180\.123\.853\.8T: low→\\tohigh46\.466\.580\.623\.354\.2Random labels40\.161\.274\.220\.349\.0Hybrid KD46\.569\.181\.223\.855\.2Llama3\.1\-8B→\\to1BSoft KD22\.133\.133\.44\.423\.3Entropy reg\.24\.535\.734\.96\.025\.3T: high→\\tolow26\.035\.134\.75\.825\.4T: low→\\tohigh25\.735\.434\.85\.025\.2Random labels19\.230\.130\.22\.120\.4Hybrid KD27\.435\.637\.05\.026\.3Table 14:Regularization and temperature controls on code benchmarks\. Avg\. is computed over HumanEval \(HE\), HumanEval\+ \(HE\+\), MBPP, and MBPP\+\. Hybrid KD is best or tied by average in both settings\.SettingMethodHEHE\+MBPPMBPP\+Avg\.Qwen2\.5\-Coder\-7B→\\to1\.5BSoft KD52\.747\.959\.652\.453\.1Entropy reg\.53\.048\.263\.555\.355\.0T: high→\\tolow53\.047\.660\.352\.653\.4T: low→\\tohigh54\.048\.760\.951\.953\.9Random labels51\.848\.260\.653\.253\.5Hybrid KD55\.550\.661\.452\.655\.0DeepSeek\-Coder\-6\.7B→\\to1\.3BSoft KD38\.433\.563\.551\.646\.8Entropy reg\.41\.536\.661\.650\.047\.4T: high→\\tolow40\.135\.462\.749\.947\.0T: low→\\tohigh39\.734\.762\.950\.346\.9Random labels35\.432\.357\.447\.643\.2Hybrid KD41\.536\.663\.250\.548\.0

#### F\.2Controlled Synthetic Validation with Exact Token\-Levelκ\\kappa

Exactκ\\kappais intractable for real LLMs because it requires path\-level intervention over future continuations\. The synthetic experiment is designed to test the Bridge–Garden mechanism in a setting where this quantity is observable\. We construct controlled domains where the semantic role of each token is known and exact token\-levelκ\\kappacan be computed\. The synthetic experiment uses three domains, Dialogue, Math, and Code, with 4000 training examples, 500 validation examples, and 30 evaluation examples per domain\. Each domain uses a vocabulary of 64 tokens; the evaluation action space has\|𝒱eval\|=62\|\\mathcal\{V\}\_\{\\mathrm\{eval\}\}\|=62, includes EOS, and excludes only PAD and BOS\. Each domain is generated by a scripted teacher oracle with a fixed template and typed decision states\. Dialogue contains recipient, date/time, required\-fact, forbidden\-constraint, and tone/paraphrase choices; Math contains substitution, operator, computed\-value, final\-answer, and equivalent\-representation choices; Code contains arithmetic/update operators, branch guards, return semantics, syntax/layout, and equivalent\-implementation choices\. The decision order is fixed before sampling: Dialogue interleaves tone, recipient, required fact, tone, date/time, and forbidden\-constraint states; Math uses substitution, equivalent form, operator, computed value, final answer, and equivalent form; Code uses three equivalent\-implementation states followed by operator, branch guard, operator, and return\-semantics states inside a Python\-like function template\. High\-risk states use four teacher\-supported semantic choices: the main token probability starts at0\.980\.98, decays by0\.010\.01across high\-risk states, and is floored at0\.940\.94, with the remaining probability shared by the other three choices\. Flexible states use eight equivalent choices with logits evenly spaced from0to−2\.5\-2\.5before normalization, and ordinary context or layout tokens are deterministic\. This design provides semantic labels that let us interpret Bridge\-like and Garden\-like behavior, whileκ\\kappaand the reported states are computed and selected independently of those labels\.

We train three student models under Hard KD, Soft KD, and Hybrid KD on samples from the same oracle\. All students use a 2\-layer Transformer withdmodel=128d\_\{\\mathrm\{model\}\}=128, 4 attention heads, feed\-forward width 512, and dropout 0\.1\. Optimization uses AdamW, batch size 256, learning rate2×10−42\\times 10^\{\-4\}, warmup\-cosine scheduling with 6 warmup epochs, 24 maximum epochs, and early\-stopping patience 6\. Hybrid KD selects its mixing coefficient from\{0\.1,…,0\.9\}\\\{0\.1,\\ldots,0\.9\\\}on the held\-out validation split: teacher\-forced KL for Dialogue, validation rollout EB for Math, and, for Code, the smallestλ\\lambdawhose validation rollout EB is within one standard error of the best validation value\. The selected coefficients are0\.90\.9for Dialogue and0\.10\.1for Math and Code\. For Code, high\-risk operator, branch, and return states use hard\-label supervision, while the remaining states use a local soft\-label weight equal to the selectedλ\\lambdatimes the normalized teacher ambiguity at that state\.

For each trained studentmm, each evaluated prefix statess, and each tokena∈𝒱evala\\in\\mathcal\{V\}\_\{\\mathrm\{eval\}\}, we compute exact token\-level sensitivity by forcingaaatssand then following the teacher oracle for all future continuations:

κm\(a∣s\)=Qm\(s,a\)−∑a′πT\(a′∣s\)Qm\(s,a′\),\\kappa\_\{m\}\(a\\mid s\)=Q\_\{m\}\(s,a\)\-\\sum\_\{a^\{\\prime\}\}\\pi\_\{T\}\(a^\{\\prime\}\\mid s\)Q\_\{m\}\(s,a^\{\\prime\}\),whereQm\(s,a\)Q\_\{m\}\(s,a\)is the expected downstream student loss after the forced token\. Overall EB is measured as the difference between the student’s loss on its own rollout prefixes and teacher\-forced prefixes, averaged over 30 rollout repeats\.

For evaluation, Dialogue uses template length 35 and 1050 evaluated prefix states, Math uses template length 43 and 1290 evaluated prefix states, and Code uses template length 47 and 1410 evaluated prefix states, each evaluated over 62 actions\. Table[15](https://arxiv.org/html/2605.26246#A6.T15)reports the Bridge/Garden regional decomposition and overall EB\. For this decomposition, Bridge and Garden regions are the top and bottom 20% of evaluated states ranked by computedκ\\kappa; the columns report absoluteκ\\kappa\-weighted contributions within those regions, where lower is better\. The results match the predicted regional pattern: hard KD has lower Bridge contribution than soft KD, soft KD has lower Garden contribution than hard KD, and hybrid KD has the lowest overall EB\. The token\-level maps show the same pattern visually: largeκ\\kappacontributions appear mainly at tokens that decide task semantics, while ordinary context and layout tokens have small contributions\.

Table 15:Synthetic Bridge/Garden decomposition and overall exposure bias \(EB\)\. Lower is better\. Bridge/Garden entries are absoluteκ\\kappa\-weighted contributions under the common top/bottom 20% non\-terminal split\.Domainλ\\lambdaBridgeHard KDBridgeSoft KDGardenHard KDGardenSoft KDOverall EBHard KDOverall EBSoft KDOverall EBHybrid KDDialogue0\.90\.10330\.30050\.01450\.00510\.06400\.17450\.0470Math0\.10\.12730\.34790\.03610\.01860\.03160\.14780\.0260Code0\.10\.25530\.32540\.06170\.03720\.02810\.08210\.0280We also examine whether the local teacher\-confidence signal aligns with exactκ\\kappaon the states where confidence\-based Hybrid KD is intended to act\. For this analysis, we focus on states where the teacher has multiple valid next\-token choices and definecT\(s\)=1−HT\(s\)/log⁡KT\(s\)c\_\{T\}\(s\)=1\-H\_\{T\}\(s\)/\\log K\_\{T\}\(s\), whereKT\(s\)K\_\{T\}\(s\)is the number of teacher\-defined candidate next\-token choices at statess\. HereHT\(s\)H\_\{T\}\(s\)is the entropy of the teacher distribution over these candidate choices\. ThuscT\(s\)c\_\{T\}\(s\)is one minus normalized entropy: it is near0when the teacher is close to uniform over the candidates and larger when the teacher distribution is sharper\. Table[16](https://arxiv.org/html/2605.26246#A6.T16)shows thatcTc\_\{T\}is strongly rank\-aligned with exact state\-levelκ\\kappaacross all three synthetic domains\. The decomposition in Table[15](https://arxiv.org/html/2605.26246#A6.T15)and the heatmaps also includes deterministic context and layout states\.

Table 16:Teacher confidence and exact token\-level sensitivity in the synthetic experiments\.ρ\\rhois Spearman correlation betweencT\(s\)c\_\{T\}\(s\)and exact state\-levelκ\(s\)\\kappa\(s\)on states where the teacher has multiple valid next\-token choices\. MeancTc\_\{T\}is reported separately for Bridge and Garden states under the same partition as Table[15](https://arxiv.org/html/2605.26246#A6.T15)\.DomainChoicestatesSpearmanρ\\rhocTc\_\{T\}vs\.κ\\kappaMeancTc\_\{T\}BridgeMeancTc\_\{T\}GardenDialogue1800\.9720\.8290\.134Math1800\.9720\.8640\.134Code2100\.9540\.7550\.134Figure[9](https://arxiv.org/html/2605.26246#A6.F9)shows representative complete token\-levelκ\\kappamaps\. Each row is one domain and includes all token states for the example, including deterministic context and layout positions\. Light cells denote zero or near\-zero contribution, while high\-contribution cells concentrate on semantic decision tokens such as recipients, required facts, substitutions, operators, branch guards, and return values\. Figures[10](https://arxiv.org/html/2605.26246#A6.F10)and[11](https://arxiv.org/html/2605.26246#A6.F11)provide additional representative examples\.

![Refer to caption](https://arxiv.org/html/2605.26246v1/x5.png)Figure 9:Representative complete token\-levelκ\\kappamaps across Dialogue, Math, and Code\. Color intensity indicates\|κ\|\|\\kappa\|; light cells have zero or near\-zero contribution\.![Refer to caption](https://arxiv.org/html/2605.26246v1/x6.png)Figure 10:Additional representative complete token\-levelκ\\kappamaps across Dialogue, Math, and Code\. Color intensity indicates\|κ\|\|\\kappa\|; light cells have zero or near\-zero contribution\.![Refer to caption](https://arxiv.org/html/2605.26246v1/x7.png)Figure 11:Additional representative complete token\-levelκ\\kappamaps across Dialogue, Math, and Code\. Color intensity indicates\|κ\|\|\\kappa\|; light cells have zero or near\-zero contribution\.![Refer to caption](https://arxiv.org/html/2605.26246v1/x8.png)Figure 12:Comparative analysis \(Qwen2\.5 7B→\\rightarrow3B\) of Hybrid KD \(λℓsoft\+\(1−λ\)ℓhard\\lambda\\ell\_\{\\mathrm\{soft\}\}\+\(1\-\\lambda\)\\ell\_\{\\mathrm\{hard\}\}\) vs\. Soft KD \(ℓsoft\\ell\_\{\\mathrm\{soft\}\}\)\. \(a\) benchmark performance gains, \(b\) student\-teacher imitation error during training \(quantified by Reverse KL\), and \(c\) inference imitation error decomposition based on the same metric\.![Refer to caption](https://arxiv.org/html/2605.26246v1/x9.png)Figure 13:Comparative analysis \(Llama3\.1\-8B→\\rightarrow3\.2\-1B\) of Hybrid KD \(λℓsoft\+\(1−λ\)ℓhard\\lambda\\ell\_\{\\mathrm\{soft\}\}\+\(1\-\\lambda\)\\ell\_\{\\mathrm\{hard\}\}\) vs\. Soft KD \(ℓsoft\\ell\_\{\\mathrm\{soft\}\}\)\. \(a\) benchmark performance gains, \(b\) student\-teacher imitation error during training \(quantified by Forward KL\), and \(c\) inference imitation error decomposition based on the same metric\.![Refer to caption](https://arxiv.org/html/2605.26246v1/x10.png)Figure 14:Comparative analysis \(Gemma3\-4B→\\rightarrow1B\) of Hybrid KD \(λℓsoft\+\(1−λ\)ℓhard\\lambda\\ell\_\{\\mathrm\{soft\}\}\+\(1\-\\lambda\)\\ell\_\{\\mathrm\{hard\}\}\) vs\. Soft KD \(ℓsoft\\ell\_\{\\mathrm\{soft\}\}\)\. \(a\) benchmark performance gains, \(b\) student\-teacher imitation error during training \(quantified by Forward KL\), and \(c\) inference imitation error decomposition based on the same metric\.
#### F\.3Further Validation of the Hard\-Soft Paradox

In this subsection, we provide additional empirical support to validate the universality of the Hard\-Soft Paradox\. We extend our analysis of training dynamics to a broader range of model families, including Llama, Qwen2\.5\-Math, DeepSeek\-Coder, and Gemma\.

As shown in Figures[12](https://arxiv.org/html/2605.26246#A6.F12),[13](https://arxiv.org/html/2605.26246#A6.F13), and[14](https://arxiv.org/html/2605.26246#A6.F14), the hard\-soft paradox remains consistent across different models, tasks, and divergence metrics\. Moreover, Table[19](https://arxiv.org/html/2605.26246#A6.T19), Table[20](https://arxiv.org/html/2605.26246#A6.T20), and Figure[3](https://arxiv.org/html/2605.26246#S5.F3)further confirm that these benefits hold true across various divergence measures\. Collectively, these results suggest the paradox is robust across diverse model families and domains\. This in turn underscores the importance of our work in addressing this puzzle in the existing literature\.

Table 17:Distillation results on Qwen2\.5\-Math models \(↑\\uparrow\)\.Boldwithdarker blueindicates Rank 1;light blueindicates Rank 2\.MethodQwen2\.5\-Math\-7B→\\rightarrow1\.5BGSM8KMATHGaokao23Avg\.Hard KD65\.75±4\.4350\.44±1\.7841\.04±3\.1552\.41Soft KD68\.72±0\.9249\.89±0\.8241\.77±1\.1153\.46Static Weighting70\.42±1\.8951\.96±1\.2442\.75±2\.6455\.04Curriculum\-based71\.10±0\.2550\.98±0\.7143\.58±1\.3655\.22Risk\-Guided71\.96±0\.4651\.46±0\.6442\.91±1\.7655\.45Entropy\-based70\.77±0\.7849\.91±1\.3942\.96±0\.5654\.55Confidence\-based71\.65±1\.2051\.37±0\.7144\.10±2\.0955\.71
#### F\.4Further Analysis of Hybrid Strategies

We further examine the impact of different hybrid supervision methods on mathematical reasoning tasks\. In Table[17](https://arxiv.org/html/2605.26246#A6.T17), we present experimental results for distillation from Qwen2\.5\-Math\-7B to Qwen2\.5\-1\.5B under various approaches\. These results consistently outperform those obtained using pure Hard KD or Soft KD alone\. Moreover, as expected, the adaptive strategy designed under the inspiration of the Bridge–Garden theory achieves correspondingly better performance\.

#### F\.5Validation on Additional Teacher\-Student Configurations

We also evaluate our framework on another teacher\-student configuration, specifically distilling from Qwen2\.5‑7B to Qwen2\.5‑1\.5B\. As shown in Table[18](https://arxiv.org/html/2605.26246#A6.T18), the hybrid distillation strategy consistently outperforms standard distillation baselines across all benchmarks\. This further confirms the general effectiveness of the hybrid supervision method observed in our main experiments\.

Table 18:Distillation results from Qwen2\.5\-7B \(Teacher\) to Qwen2\.5\-1\.5B \(Student\) on general reasoning benchmarks\. We use Forward KL for soft supervision and static weighting for hybrid supervision\.SettingMethodBBHMMLUARC\-ChallengeTheoremQAAverageTeacherQwen2\.5\-7B\-Instruct64\.66±0\.6978\.22±0\.2289\.90±0\.2432\.47±0\.3266\.31Student\(Qwen2\.5\-1\.5B\)Baseline \(No Distill\)25\.87±0\.0658\.99±0\.1171\.11±0\.079\.90±0\.2741\.47Hard KD31\.98±0\.3555\.68±0\.2564\.28±0\.3819\.23±0\.8042\.79Forward KL divergence35\.08±0\.1960\.27±0\.1171\.83±0\.2117\.15±0\.7446\.08Reverse KL divergence35\.88±0\.1559\.96±0\.1171\.30±0\.1317\.00±0\.3246\.04Skew Forward KL32\.54±0\.2060\.39±0\.1271\.88±0\.1916\.25±0\.2645\.26Skew Reverse KL32\.99±0\.2460\.31±0\.1271\.62±0\.1217\.62±0\.7645\.64Total Variation35\.48±0\.1260\.40±0\.0771\.43±0\.0617\.08±0\.4846\.10HybKD \(Ours\)35\.11±0\.2661\.68±0\.2472\.65±0\.3318\.65±0\.2747\.02

Table 19:DeepSeek\-Coder distillation results \(↑\\uparrow\)\. We report pass@1 scores\.Boldwithdarker backgroundindicates the best performance;light backgroundindicates the second\-best performance\.MethodDeepSeek\-Coder\-6\.7B→\\rightarrow1\.3BHumanEvalHumanEval\+MBPPMBPP\+Avg\.Reference & BaselinesTeacher76\.8371\.3475\.3266\.7572\.56Student \(No Distill\)33\.5427\.4458\.4449\.6142\.26Hard KD35\.3732\.9360\.3250\.0044\.66KL Divergence FamilyForward KL38\.4133\.5463\.4951\.5946\.76\+ Ours41\.4636\.5963\.1250\.3947\.89Reverse KL39\.0234\.1562\.1750\.0046\.34\+ Ours37\.8034\.1563\.3851\.9546\.82Distance & Skewed FamilyTotal Variation39\.6335\.3761\.5649\.6146\.54\+ Ours40\.0236\.1662\.1049\.9747\.06Skew FKL42\.0735\.9860\.8550\.2647\.29\+ Ours42\.4135\.7563\.1250\.3947\.92Skew RKL42\.0735\.9860\.7850\.3947\.31\+ Ours41\.2435\.9862\.3451\.1747\.68Unified Divergence Familyα\\alpha\-β\\betadivergence41\.4636\.5960\.0550\.2647\.09\+ Ours43\.9038\.4164\.6851\.9549\.74Table 20:Distillation results on Gemma\-3 models \(↑\\uparrow\)\.Boldwithdarker backgroundindicates the best performance;light backgroundindicates the second\-best performance\.MethodGemma3\-4B→\\rightarrowGemma3\-1BBBHMMLUARC\-CThmQAAvg\.Reference & BaselinesTeacher52\.5766\.1781\.2326\.1256\.52Student \(No Distill\)6\.4715\.9415\.700\.959\.76Hard KD5\.2818\.1118\.093\.1511\.16KL Divergence FamilyForward KL5\.9124\.0525\.004\.2514\.80\+ Ours6\.2324\.6925\.694\.2215\.21Reverse KL9\.5224\.7727\.581\.2615\.78\+ Ours7\.1425\.2726\.034\.5715\.75Distance & Skewed FamilyTotal Variation6\.2925\.0725\.831\.2914\.62\+ Ours5\.0224\.1325\.394\.0014\.64Skew FKL7\.0125\.0826\.145\.0115\.81\+ Ours7\.7825\.9426\.075\.2416\.26Skew RKL5\.8325\.1725\.933\.3815\.08\+ Ours5\.7225\.7326\.923\.4115\.45Unified Divergence Familyα\\alpha\-β\\betadivergence7\.2625\.4926\.074\.3415\.79\+ Ours7\.5625\.6925\.904\.5415\.92Table 21:Performance comparison of Qwen2\.5 on reasoning benchmarks\. Avg\. is the mean across all benchmarks\. We evaluate our hybrid KD against recent soft KD methods\.MethodQwen2\.5\-7B→\\toQwen2\.5\-0\.5BQwen2\.5\-7B→\\toQwen2\.5\-3BBBHMMLUARC\-CThmQAAvg\.BBHMMLUARC\-CThmQAAvg\.Teacher64\.66±0\.6978\.22±0\.2289\.90±0\.2432\.47±0\.3266\.3164\.66±0\.6978\.22±0\.2289\.90±0\.2432\.47±0\.3266\.31Student \(No Distill\)3\.81±0\.0744\.67±0\.1243\.72±0\.127\.78±0\.3924\.9922\.34±0\.0664\.61±0\.0678\.40±0\.0312\.22±0\.2544\.39Hard KD\(Kim and Rush,[2016](https://arxiv.org/html/2605.26246#bib.bib20)\)10\.27±0\.1036\.83±0\.1540\.58±0\.7011\.35±0\.4324\.7641\.52±0\.3365\.76±0\.1878\.75±0\.7123\.75±0\.4052\.45Forward KL\(Hintonet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib54)\)24\.42±0\.0643\.19±0\.2046\.84±0\.1910\.33±0\.2631\.2041\.65±0\.1664\.45±0\.0478\.33±0\.2223\.02±0\.3351\.87Reverse KL\(Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19)\)24\.91±0\.0144\.72±0\.0147\.44±0\.0011\.00±0\.0032\.0244\.07±0\.1065\.67±0\.1977\.68±0\.1224\.20±0\.4952\.90Total Variation\(Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55)\)26\.74±0\.3844\.35±0\.1246\.76±0\.5110\.20±0\.5632\.0140\.50±0\.1664\.52±0\.0378\.11±0\.1822\.83±0\.5651\.49JS divergence\(Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73)\)24\.55±0\.1343\.31±0\.1044\.78±0\.0511\.82±0\.3731\.1245\.50±0\.0864\.68±0\.1778\.85±0\.1422\.27±0\.4152\.83Adaptive KL\(Wuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib70)\)26\.04±0\.2341\.85±0\.1145\.89±0\.3211\.33±0\.6131\.2844\.71±0\.1364\.69±0\.0779\.23±0\.2222\.25±0\.3352\.72Skew FKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)25\.87±0\.2444\.12±0\.2447\.12±0\.6510\.65±0\.2231\.9441\.39±0\.1764\.67±0\.1577\.75±0\.2323\.77±0\.7151\.89Skew RKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)28\.16±0\.1645\.03±0\.0547\.51±0\.2011\.37±0\.6633\.0241\.22±0\.0863\.95±0\.0876\.91±0\.1823\.67±0\.2051\.44α\\alpha\-β\\betadivergence\(Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59)\)26\.18±0\.1542\.59±0\.1846\.70±0\.3711\.07±0\.4331\.6345\.12±0\.2364\.95±0\.1779\.81±0\.0922\.94±0\.5453\.21HybKD \(Ours\)26\.58±0\.1649\.08±0\.2251\.69±0\.6210\.50±0\.5434\.4646\.53±0\.0569\.05±0\.0781\.23±0\.0023\.82±0\.5855\.16

Table 22:Accuracy \(↑\\uparrow\) on Llama and Gemma models\.MethodLlama3\.1\-8B→\\toLlama3\.2\-1BGemma3\-4B→\\toGemma3\-1BBBHMMLUARC\-CThmQAAvg\.BBHMMLUARC\-CThmQAAvg\.Teacher57\.72±0\.0770\.90±0\.0383\.58±0\.1018\.10±0\.4057\.5852\.57±0\.0066\.17±0\.0081\.23±0\.0026\.12±0\.0056\.52Student \(No Distill\)14\.01±4\.4119\.78±9\.8921\.57±9\.942\.22±0\.4414\.406\.47±3\.4115\.94±8\.9115\.70±8\.490\.95±0\.209\.76Hard KD\(Kim and Rush,[2016](https://arxiv.org/html/2605.26246#bib.bib20)\)15\.29±2\.7522\.54±1\.3823\.98±1\.963\.88±0\.7316\.425\.28±1\.8918\.11±4\.8318\.09±5\.253\.15±0\.1511\.16Forward KL\(Hintonet al\.,[2015](https://arxiv.org/html/2605.26246#bib.bib54)\)22\.07±2\.1133\.13±1\.6933\.41±1\.784\.37±0\.4323\.255\.91±1\.4324\.05±2\.0125\.00±2\.834\.25±0\.3314\.80Reverse KL\(Guet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib19)\)23\.69±0\.7731\.63±1\.8832\.08±1\.203\.60±0\.0922\.759\.52±1\.0324\.77±0\.6827\.58±0\.601\.26±0\.3215\.78Total Variation\(Wenet al\.,[2023](https://arxiv.org/html/2605.26246#bib.bib55)\)23\.55±1\.5127\.41±4\.9628\.99±4\.472\.68±0\.5620\.666\.29±0\.8625\.07±0\.3125\.83±0\.431\.29±0\.4114\.62JS divergence\(Agarwalet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib73)\)25\.16±2\.0334\.08±2\.1534\.42±1\.815\.70±0\.1324\.847\.14±0\.7425\.27±0\.5626\.03±0\.334\.57±0\.9715\.75Adaptive KL\(Wuet al\.,[2025](https://arxiv.org/html/2605.26246#bib.bib70)\)25\.18±1\.8633\.86±2\.3433\.74±1\.774\.85±0\.4424\.415\.47±0\.6324\.76±1\.5624\.07±1\.124\.23±0\.4114\.63Skew FKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)25\.05±1\.4031\.39±2\.8432\.34±2\.994\.73±0\.8623\.377\.01±0\.8225\.08±0\.8826\.14±0\.705\.01±0\.2915\.81Skew RKL\(Koet al\.,[2024](https://arxiv.org/html/2605.26246#bib.bib18),[2025](https://arxiv.org/html/2605.26246#bib.bib72)\)25\.24±0\.7933\.00±1\.4032\.24±1\.364\.27±0\.4423\.695\.83±0\.2525\.17±0\.2225\.93±0\.293\.38±0\.2615\.08α\\alpha\-β\\betadivergence\(Wanget al\.,[2025b](https://arxiv.org/html/2605.26246#bib.bib59)\)25\.07±1\.3634\.78±2\.0434\.90±1\.455\.27±0\.4425\.017\.26±0\.7825\.49±1\.0026\.07±0\.664\.34±0\.8915\.79HybKD \(Ours\)27\.44±2\.8535\.64±3\.2237\.01±3\.985\.00±1\.2526\.277\.78±1\.1925\.94±0\.5526\.07±0\.855\.24±0\.7916\.26
The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works

Similar Articles

Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation

Distilling LLM Feedback for Lean Theorem Proving

Bounded Behavioral Indistinguishability for Black-Box LLM Distillation

From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs

Hybrid Policy Distillation for LLMs

Submit Feedback

Similar Articles

Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
Distilling LLM Feedback for Lean Theorem Proving
Bounded Behavioral Indistinguishability for Black-Box LLM Distillation
From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs
Hybrid Policy Distillation for LLMs