Tag
The paper investigates privileged context design in on-policy self-distillation, demonstrating that intermediate levels of abstraction can improve model performance over full solutions while using fewer hint tokens.
This paper introduces EDGE-OPD, a modification of on-policy self-distillation for LLMs that uses guided rollouts and evidence masks to internalize privileged context without degrading general capabilities, showing success in rare-token identity settings.