model-student-training

The paper proposes RIDE (RL-Induced Direction Extrapolation), a method that extrapolates RL-induced representation residuals in hidden-state space rather than output space during on-policy distillation, enabling student models to approach or exceed RL-trained teachers across multiple model scales and architectures.