kl-constraint

Tag

Cards List
#kl-constraint

SAKI: Maximal-Coupling-Routed Teacher Supervision for On-Policy Distillation

Hugging Face Daily Papers ↗ · 5d ago Cached

SAKI introduces a supervision allocation method for on-policy distillation that uses KL-constrained teacher-guided rollouts and maximal coupling to route token-level supervision, improving performance on mathematical reasoning benchmarks for small models.

0 favorites 0 likes
← Back to home

Submit Feedback