Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning
Summary
Adaptive Teacher Exposure for Self-Distillation (ATESD) improves LLM reasoning by dynamically adjusting how much of the reference reasoning the teacher shows the student during training, using a learnable policy controller and a discounted learning-progress reward. Experiments on math benchmarks show consistent improvements over existing self-distillation and RL baselines.
View Cached Full Text
Cached at: 05/15/26, 08:24 AM
Paper page - Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning
Source: https://huggingface.co/papers/2605.11458
Abstract
Adaptive Teacher Exposure for Self-Distillation (ATESD) improves large language model reasoning by dynamically adjusting teacher exposure during training through a learnable policy controller.
On-policyself-distillationhas become a strong recipe for LLM reasoning, where a privileged teacher supervises the student’s own rollouts while conditioning on the reference solution. A design choice shared by nearly all such methods, however, has gone unquestioned: the teacher always sees the full reference reasoning. We argue that this default itself is part of the problem and identify a teacher-sideexposure mismatch: when the teacher conditions on reasoning far beyond the student’s current competence, the resulting token targets become too strong to absorb. A controlled fixed-exposure sweep makes this concrete on two fronts: 1) full exposure is not reliably the best choice, and 2) student-teacher mismatch grows monotonically as the teacher sees more privileged reasoning. This motivates treating teacher exposure not as a fixed hyperparameter but as a learnable training-time control variable. We therefore propose Adaptive Teacher Exposure forSelf-Distillation(ATESD). ATESD models the reveal ratio with a lightweightBeta-policy controllerconditioned on compact training-state statistics, and uses one sampled exposure for a short hold window of student updates. To make this exposure controller learnable, we optimize it with adiscounted learning-progress rewardthat scores each held decision by its effect on the student’s future improvement rather than its immediate loss change, addressing thedelayed credit assignmentinduced by on-policy distillation. Experiments on AIME 24, AIME 25, and HMMT 25 across Qwen3-{1.7B, 4B, 8B} show that ATESD consistently outperforms competitiveself-distillationand RL baselines, improving over OPSD by +0.95, +2.05, and +2.33 Average@12 points respectively, and establishing adaptive teacher exposure as an effective new axis for reasoningself-distillation.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.11458
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.11458 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.11458 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.11458 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
This paper introduces Motab, a new pipeline for LLM reasoning distillation that mitigates both off-policy and on-policy exposure biases by dynamically monitoring student generation and backtracking to safe states with teacher intervention, achieving ~3% average improvement.
Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning
The paper proposes EGRSD and CL-EGRSD, on-policy self-distillation methods that weight token-level supervision by teacher entropy to improve reasoning accuracy-length tradeoff in LLMs, evaluated on Qwen3-4B and Qwen3-8B.
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
AdviSD proposes a method where a small trainable advisor steers a frozen frontier LLM executor via targeted multi-turn self-distillation, pairing outcome-based RL with feedback-conditioned self-distillation that selectively uses corrections. Experiments with Qwen3-8B advisors guiding Gemini and Claude show 4-6 point gains on BFCL-v3 and EnvScaler with good out-of-domain generalization.
Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information
Proposes Anti-Self-Distillation (AntiSD) which reverses the knowledge transfer direction in self-distillation to improve math reasoning efficiency and accuracy, achieving GRPO baseline accuracy in 2-10x fewer steps and up to 11.5 points higher final accuracy across models from 4B to 30B parameters.
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
This paper proposes a novel Chain-of-Thought distillation framework that transfers teacher models' stepwise attention on key information to student models through a Mixture-of-Layers module for dynamic layer alignment. The method achieves consistent performance improvements on mathematical and commonsense reasoning benchmarks by explicitly guiding student models to progressively focus on critical information during reasoning.