Tag
SMOPD proposes a two-stage specialize-and-merge online policy distillation method to improve multi-reward reinforcement learning, addressing issues with sparse and dense reward signals where GDPO struggles. It outperforms GDPO across 1.5B, 3B, and 7B backbones in complementary and conflicting reward settings.
This paper introduces Dual-Anchored Policy Distillation (DAPD), a framework to address privilege illusion in on-policy self-distillation for language models. DAPD improves Qwen3-4B by +2.00 points on average across tasks, with gains persisting at larger scales.
Proposes SEED, a self-evolving on-policy distillation framework that converts completed trajectories into hindsight skills to improve reinforcement learning for interactive agent tasks, achieving consistent performance gains and sample efficiency.
Introduces Self-Review Reinforcement Learning (SRRL), a training framework that embeds a self-review step into RL episodes to transform sparse environmental feedback into behavioral improvements, using cross-episode memory and selective policy distillation. Evaluated on GSM8K, SRRL outperforms standard RLVR with GRPO across Qwen 3-4B and OLMo-3-7B models.
Trust Region Policy Distillation (TOP-D) stabilizes on-policy distillation by dynamically constructing a proximal teacher, providing theoretical convergence guarantees and empirical gains in mathematical reasoning tasks with zero additional overhead.
DiffusionOPD proposes a multi-task training paradigm for diffusion models that uses online policy distillation to efficiently combine task-specific teachers into a unified student, achieving state-of-the-art results on all evaluated benchmarks.