response-length

Tag

Cards List
#response-length

Mitigating the Length-Scaling Tax with Online Distillation

Hugging Face Daily Papers ↗ · 2026-09-30 Cached

This paper introduces the length-scaling tax (LST)—excess response length on already-solved queries after RL post-training—and proposes Length Self-Distillation (LSD), which routes solved prompts to on-policy distillation using an EMA of the online policy as teacher, substantially curbing verbosity while preserving or improving accuracy.

0 favorites 0 likes
← Back to home

Submit Feedback