low-rank-distillation

LoRi proposes a low-rank distillation framework for implicit chain-of-thought reasoning that aligns teacher and student trajectories in a shared low-rank subspace, improving performance on mathematical reasoning benchmarks.