Tag
论文提出 WISD（Weakening-Induced On-Policy Self-Distillation），通过将大推理模型与量化的弱化版本进行对比生成训练信号，在无需特权数据的情况下实现自蒸馏，在数学与代码推理基准上平均提升 4.0–5.5 个百分点并达到最先进性能。
该论文提出 Fin-SD（hindsight self-distillation），一种无需外部教师模型的本地化自蒸馏方法，用于提升小型开放权重模型的多步金融推理能力；实验发现增益主要来自监督位置而非提示内容，且与基于完整参考解答的自蒸馏精度相当但少生成 9.5% token。
本文提出条件对比外推在线蒸馏（CCE-OPD），通过让教师模型对完整问题与掩码条件版本进行对比评估，降低教师在学生生成前缀上偏好无效续写的风险，在六个数学基准上相较标准OPD提升avg@8约1.60-1.95个百分点。
论文提出VRSD（verbal regularization self-distillation），通过系统提示调节teacher对特权信息的使用，抑制on-policy自蒸馏中的捷径推理与熵坍缩，在11/12个模型-任务设置上平均提升Pass@1达3.9个百分点。
The paper introduces OLIVE, an online knowledge distillation method where the student generates prefixes and the teacher continues them, addressing covariate shift and fragmented supervision in existing distillation approaches. OLIVE outperforms on-policy distillation and offline SFT on reasoning and agentic tasks under comparable training budgets.
The paper proposes Reasoning-Progress-Aware Reward Filtering for On-Policy Distillation (R2-OPD) to address the mismatch between teacher-derived rewards and genuine reasoning progress, improving reasoning performances in language model training.
This paper proposes Adaptive Entropy Distillation (AED), a method that dynamically calibrates token-level imitation strength in knowledge distillation using teacher entropy, achieving superior performance on instruction-following and mathematical reasoning benchmarks.
ReflectRL is a framework that learns from 'golden negative trajectories' (failed reasoning attempts by expert models) by reflecting on them, then transfers this reflective reasoning back to direct reasoning, improving LLM performance across benchmarks.
This paper introduces On-Policy Data Evolution (ODE) and a visual-native agent harness to improve multimodal deep search agents. By enabling reusable visual evidence and closed-loop data generation, ODE significantly boosts the performance of Qwen3-VL agents across multiple benchmarks, surpassing Gemini 2.5 Pro.