@ethantsliu: some distillation techniques: OPD: log pi - log pi_theta OPSD: log pi_theta (• | x, y*) - log pi_theta (• | x) RLSD: A(…
Summary
This tweet discusses OPD, OPSD, RLSD, and OPD^2 distillation techniques for language models, emphasizing the use of log probability differences to isolate post-training improvements.
View Cached Full Text
Cached at: 07/20/26, 01:26 PM
some distillation techniques:
OPD: log pi - log pi_theta OPSD: log pi_theta (• | x, y*) - log pi_theta (• | x) RLSD: A(x, y) x opsd (advantage direction, opsd magnitude) OPD^2: log π_teacher − log π_teacher-base
Xiuyu Li (@sheriyuo): OPD normally copies everything the teacher prefers, including habits inherited before reasoning tuning.
OPD² instead uses the token-level logprob change between a reasoning teacher and its own base checkpoint, so the reward isolates what post-training actually added.
On-Policy
Similar Articles
OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
OmniOPD introduces a logit-free on-policy distillation method that uses chunk-level semantic similarity and speculative verification to train student models with black-box teachers, achieving up to +28.64% improvement on math benchmarks over standard OPD.
On-Policy Delta Distillation
The paper introduces On-Policy Delta Distillation (OPD^2), a new distillation reward called the delta signal that captures the difference between a teacher model and its base model before reasoning tuning, providing a more direct signal for transferring reasoning capabilities. Experiments across math, science, and code benchmarks show OPD^2 consistently outperforms conventional on-policy distillation.
dOPSD: On-Policy Self-Distillation for Diffusion Language Models
This paper introduces dOPSD, an on-policy self-distillation method for diffusion language models that leverages internal denoising trajectories to improve mathematical reasoning and code generation.
On-policy distillation: one of the hottest terms on PapersWithCode [R]
Hugging Face's Niels introduces On-policy Distillation (OPD), a key post-training technique used in models like Qwen 3.6/3.7, GLM-5.1, and DeepSeek-V4, now featured on PapersWithCode with a linked whiteboard explanation by Sasha Rush and Dwarkesh Patel.
OISD: On-Policy Internal Self-Distillation of Language Models
OISD is a new self-distillation framework that improves language model reasoning by transferring predictive signals from the final layer to intermediate layers during GRPO post-training, using logit and attention alignment.