@neural_avb: If yall are interested in On Policy Distillation, check this specific repo. Somebody put together a curated collection …
Summary
A curated collection of papers and tools for On Policy Distillation, organized and annotated with a getting-started section, shared via a GitHub repo.
View Cached Full Text
Cached at: 05/29/26, 02:10 PM
If yall are interested in On Policy Distillation, check this specific repo.
Somebody put together a curated collection of papers and tools categorized and annotated. Comes with a “Get Started” section too. https://t.co/nwVgFdoLDY
pradheep (@pradheepraop): starting a proper deep dive into opd/opsd now.
thanks @neural_avb and @chrisliu298 for consolidating some really useful resources.
recommendations are welcome 🙂
Similar Articles
@neural_avb: There is a really banger article on On-Policy Distillation. Came out on HF a few months back.
A tweet recommending an article on on-policy distillation published on Hugging Face.
@neural_avb: This article actually explains all the components of On Policy Distillation Loss functions (forward vs reverse KL), sup…
This article explains components of On Policy Distillation loss functions, including forward vs reverse KL divergence, supervision granularity, privilege types, and privilege advantage estimation. Additional resources from Thinking Machines, Hugging Face, and others are provided.
On-policy distillation: one of the hottest terms on PapersWithCode [R]
Hugging Face's Niels introduces On-policy Distillation (OPD), a key post-training technique used in models like Qwen 3.6/3.7, GLM-5.1, and DeepSeek-V4, now featured on PapersWithCode with a linked whiteboard explanation by Sasha Rush and Dwarkesh Patel.
@louieworth: New blog post: On-Policy Distillation — Promise, Pitfalls, and Prospects. OPD combines on-policy rollouts with dense te…
This blog post discusses On-Policy Distillation (OPD), a technique that combines on-policy rollouts with dense teacher supervision, and highlights its promise, three failure modes, and the author's new paper on the topic.
@NielsRogge: One of the hottest terms in AI right now is "On-policy distillation". It is a post-training technique in which a studen…
On-policy distillation is highlighted as a hot post-training technique combining distillation with online RL, now listed on PapersWithCode with 183 citing papers.