@wu_taiqiang: How to maximize OPD performance? One important thing is warm-up. Then the student-sampled sequence is well defined in t…
Summary
The author discusses a paper that demystifies the warm-up process for OPD (likely on-policy distillation), explaining how warm-up enables well-defined student-sampled sequences and educational token-level dense rewards from the teacher.
View Cached Full Text
Cached at: 08/13/26, 09:30 PM
How to maximize OPD performance? 🧐
One important thing is warm-up. Then the student-sampled sequence is well defined in the teacher’s output space, and the token-level dense reward from the teacher is educational. 🤜
In this paper, we demystify the warm-up process: 🧵 https://t.co/lJfxyyhHXi
How to maximize OPD performance?
One important thing is warm-up. Then the student-sampled sequence is well defined in the teacher’s output space, and the token-level dense reward from the teacher is educational.
In this paper, we demystify the warm-up process:
[1/N] Paper: Simple-OPD: Demystifying Warm-up for On-policy Distillation ArXiv: https://arxiv.org/pdf/2608.06802
[2/N] Considering the warm-up data, the take-away message is. In short, CoT from the teacher matters, not performance. Even wrong CoT works.
[3/N] Considering the warm-up training, the conclusion is shown below. To balance the ID and OOD performance, we recommend LoRA with a suitable rank and trained with a moderate step.
[4/N] Indeed, we consider warm-up as a speed-up for vanilla OPD. For response length and overlap ratio, we can find that the student fits the teacher sooner or later. This is why OPD works.
@grok spread to anyone might be interested.
Nice blog and valuable insights!
Thanks, Junjie!
Similar Articles
Simple-OPD: Demystifying Warm-up for On-policy Distillation
This paper investigates the warm-up stage for on-policy distillation (OPD), showing that teacher-compatible chain-of-thought supervision and LoRA-based training with near-saturation duration improve OPD effectiveness. It introduces Simple-OPD, a plug-and-play initialization method that boosts OPD performance across diverse settings.
On-policy distillation: one of the hottest terms on PapersWithCode [R]
Hugging Face's Niels introduces On-policy Distillation (OPD), a key post-training technique used in models like Qwen 3.6/3.7, GLM-5.1, and DeepSeek-V4, now featured on PapersWithCode with a linked whiteboard explanation by Sasha Rush and Dwarkesh Patel.
@louieworth: New blog post: On-Policy Distillation — Promise, Pitfalls, and Prospects. OPD combines on-policy rollouts with dense te…
This blog post discusses On-Policy Distillation (OPD), a technique that combines on-policy rollouts with dense teacher supervision, and highlights its promise, three failure modes, and the author's new paper on the topic.
Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
This paper investigates the parameter-level mechanisms behind the efficiency of On-Policy Distillation (OPD) for large language models, attributing it to early 'foresight' in module allocation and update direction. It proposes EffOPD, a plug-and-play method that accelerates OPD training by 3x without compromising final performance.
On-Policy Delta Distillation
The paper introduces On-Policy Delta Distillation (OPD^2), a new distillation reward called the delta signal that captures the difference between a teacher model and its base model before reasoning tuning, providing a more direct signal for transferring reasoning capabilities. Experiments across math, science, and code benchmarks show OPD^2 consistently outperforms conventional on-policy distillation.