on-policy-learning

论文提出 projected distillation，通过将学生的失败尝试最小编辑为正确目标，实现近似 on-policy 的学习信号，在数学、代码、科学和工具使用等领域显著优于 GRPO 和 on-policy distillation 等基线。

该论文提出ORL（On-policy Residual Learning）框架，通过熵收缩目标与异步残差捕获等系统级协同设计，使扩散语言模型（DLM）在相近质量下将吞吐量提升至约6.9×，在单请求解码速度上超越自回归模型的推测解码方案。

提出简化匹配蒸馏（SMD）框架，通过纯回归的on-policy速度回归与Pivot残差代理，实现无需辅助评分模型或对抗训练的4步少步扩散生成，在图像、3D、视频及音视频联合生成任务上超越强基线并可扩展至33B参数。

该论文提出 D-OPCD（Diffusion On-Policy Context Distillation），将 T2I 智能体 harness 通过提示词积累的经验蒸馏进扩散模型权重中，使模型在仅用原始查询条件时也能保留部分 harness 增益，并结合 Auto Skill Evolver 实现 harness 与模型的持续共同进化。

This paper analyzes on-policy self-distillation (OPSD) for LLM reasoning and shows that unverified student scaffolds create an imitation gap that worsens with model scale, motivating OASIS, a method that supervises mostly verified by-label on-policy trajectories using model-generated attempts as teacher context, improving Qwen3 1.7B/4B/8B by 3.2-3.8 points on AIME/HMMT benchmarks.

TISD introduces a method for on-policy self-distillation with trajectory intervention to improve large language model performance in coding and science tasks by allowing teacher-guided branching.

This paper systematically studies the role of rollout policy in strong-to-weak distillation, finding that token-level KL direction and learning rate matter more than on-policy data, challenging the assumption that on-policy rollouts are inherently preferable.

SCOPD introduces a sparse-context on-policy self-distillation framework that enhances the efficiency of vision-language models by improving the use of pruned visual tokens, achieving higher performance retention without additional inference cost.

The paper proposes LT-OPD, an on-policy self-distillation framework for extreme visual token reduction in multimodal large language models, significantly improving performance under low token budgets while reducing KV-cache usage and FLOPs.

MOPD-Router proposes a token-level routing framework for multi-teacher on-policy distillation, enabling the use of unlabeled data and cross-domain supervision without domain labels. Experiments demonstrate significant performance improvements over standard methods in both unlabeled and labeled scenarios.

The paper introduces GUI-SD-v2, a two-stage on-policy self-distillation framework for multi-turn GUI agents that enhances privilege following and guidance, achieving superior performance on AndroidWorld and MobileWorld benchmarks.

The paper investigates privileged context design in on-policy self-distillation, demonstrating that intermediate levels of abstraction can improve model performance over full solutions while using fewer hint tokens.

The paper introduces Calibrated On-Policy Distillation (Cal-OPD), a method that estimates the teacher's self-deviation to calibrate teacher-student discrepancies, improving on-policy distillation for mathematical reasoning tasks.

The paper investigates the contribution of privileged information in on-policy self-distillation for language models, finding that reference-free distillation accounts for most improvements, with limited additional benefits from privileged references.

STAR-OPD is a novel on-policy distillation method for ABSA quadruple extraction that uses set-structured rewards to correct structural errors in student models, improving performance and narrowing the gap with teacher models.

This paper introduces HB-SJD, a batched speculative Jacobi decoding method for visual on-policy distillation that accelerates rollout generation by processing multiple tokens in parallel, reducing training time while preserving generation quality.

This paper proposes Tail-Aware Top-k On-Policy Distillation (TA-OPD) to address the loss of tail probability in on-policy distillation for language models, improving downstream accuracy on benchmarks.

This paper addresses rubric interference in LLM judges when evaluating multiple rubrics simultaneously and proposes Self-Anchored Rubric Alignment (SARA) to improve consistency through on-policy self-distillation.

DUET is a dual-teacher on-policy distillation method for ensuring LLMs comply with runtime prohibitions, achieving high violation compliance while preserving utility through token-selective learning.

The paper introduces S^2VOPD, a self-supervised method that improves vision-language models by distilling from original images into strongly augmented student views, achieving performance surpassing GPT-5.4 on benchmarks.