token-suppression

Tag

Cards List
#token-suppression

Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement

Hugging Face Daily Papers ↗ · 2026-08-31 Cached

The paper analyzes on-policy distillation, revealing it primarily suppresses low-probability tokens rather than relying on teacher guidance, and introduces OPSA, a supervision-free method that significantly enhances reasoning performance.

0 favorites 0 likes
← Back to home

Submit Feedback