self-distillation

Introduces H^2SD, a hybrid hindsight self-distillation framework that improves RLVR by using the teacher model differently for successful and failed trajectories, achieving better reasoning performance.

Proposes BIRD, a two-stage self-reasoning distillation method that bootstraps concise reasoning trajectories before on-policy training, achieving stronger accuracy-efficiency trade-offs on MATH-500 and AIME benchmarks. On Qwen3-8B, accuracy improves from 86.2% to 92.0% while average response length drops from 3,099 to 1,115 tokens.

A research paper introducing Canon, a label-free self-distillation method that uses consensus among sampled solutions to provide dense token-level supervision for training large language models on reasoning tasks, improving pass@1 by up to 12 points and outperforming label-free reinforcement learning at a fraction of the compute.

The paper identifies 'Thinking Collapse' in on-policy self-distillation for large language models, characterized by a decline in intermediate reasoning steps, and proposes AD-OPSD, a control framework that mitigates this collapse by anchoring high-suppression-risk tokens to a reference prior. The method achieves up to +4.1% absolute average accuracy improvement on mathematical benchmarks.

This paper introduces Execution-Gated Self-Distillation, a method that uses a deterministic, ungameable launch check as a filter for iterative self-distillation, achieving significant improvements in generating runnable Godot games from natural language descriptions using a 14B model.

This study introduces section-aware compression for reasoning traces, training models to drop filler narration while preserving compute and verification spans, matching or exceeding original accuracy while using 2-3 times fewer tokens.

DeepSearch-Evolve introduces a self-distillation framework for web agents using a verifiable environment (DeepSearch-World) with 420K multi-hop QA tasks, achieving competitive performance without distillation from stronger models.

This paper introduces GeoSD, a geometric self-distillation objective that uses Hellinger loss and a proximal Fisher-Rao distance term to counter drift in on-policy self-distillation, improving out-of-distribution reasoning accuracy by 5.7–8.6 points across model scales.

OPSD-V improves few-step autoregressive video diffusion models by using real long-video data as temporal context during training, providing dense trajectory-level supervision that enhances visual quality and motion dynamics without altering inference mechanisms.

This talk introduces self-distillation methods (SDPO and SDFT) that use model-generated rollouts and rich environment feedback (e.g., compiler errors) to create dense token-level learning signals, achieving 6× faster convergence than GRPO and 11× shorter reasoning traces, while also enabling sequential skill learning without forgetting.

This paper introduces dOPSD, an on-policy self-distillation method for diffusion language models that leverages internal denoising trajectories to improve mathematical reasoning and code generation.

This paper investigates on-policy self-distillation in continual post-training, finding it accelerates in-domain specialization but fails to prevent forgetting and can collapse in out-of-distribution scenarios, suggesting that on-policy data alone is insufficient for continual learning.

PHF proposes a method to distill hidden state trajectories from a privileged teacher to a student during on-policy self-distillation, improving reasoning performance on language models.

OPID is a framework that extracts dense token-level supervision from completed on-policy trajectories for reinforcement learning of language agents, using hierarchical skills (episode-level and step-level) to improve sample efficiency and robustness.

PolicyAlign proposes a framework that directly aligns LLMs with natural-language safety policies via synthetic instruction generation and on-policy self-distillation, improving safety without relying on costly supervision data.

OPID proposes an on-policy skill distillation framework that extracts dense hindsight supervision from completed trajectories, combining outcome-based RL with token-level self-distillation to improve language agent training efficiency and performance on multi-turn tasks.

This paper proposes Rubric-Conditioned Self-Distillation (RCSD), a framework that uses fine-grained rubric criteria to provide token-level guidance during self-distillation, improving reasoning performance over scalar-reward methods like GRPO and OPSD.

Introduces d-OPSD, the first on-policy self-distillation framework for diffusion large language models, using suffix conditioning and step-level supervision to outperform RLVR and SFT baselines on reasoning benchmarks.

This paper proposes Trajectory-Augmented Policy Optimization (TAPO), which constructs micro-reflective correction trajectories using the model's own correct and incorrect rollouts to improve reasoning in large language models, outperforming standard self-distillation methods on math benchmarks.

This paper introduces ViGOS, a method for multimodal on-policy self-distillation that decouples perception and reasoning by having the student model first produce a visual description before reasoning, reducing shortcut reliance and improving image-grounding behavior.