Tag
This paper studies self-distillation with privileged information (PI) as a lone post-training objective for LLMs, reproducing reported gains on easy tasks but showing it fails on difficult reasoning tasks: per-token loss drops while validation accuracy stagnates or degrades. The authors trace the failure to PI bias, which pulls teacher targets toward a reference trajectory and trains students to be flatter and less decisive without improving reasoning.
This paper introduces Rubrics as Privileged Information (RuPI), extending on-policy self-distillation to open-ended generation by conditioning the teacher on rubrics as soft privileged information. The method outperforms rubric-as-reward RL and reference-completion distillation across multiple LLMs and benchmarks like HealthBench.
DOPD proposes a dual on-policy distillation paradigm that dynamically routes token-level supervision between privileged teacher and student policies based on advantage gaps and probabilities, addressing privilege illusion and improving capability transfer in LLMs and VLMs.
A tweet from Souradip Chakraborty proposes using privileged information to actively sample rollouts in reinforcement learning, contrasting with traditional blind sampling methods. The tweet is prefaced by a quote about great teachers crafting demonstrations that students could build themselves.
This work proposes using privileged information to actively sample rollouts in reinforcement learning, improving on typical blind sampling methods.