reasoning-compression

Proposes BIRD, a two-stage self-reasoning distillation method that bootstraps concise reasoning trajectories before on-policy training, achieving stronger accuracy-efficiency trade-offs on MATH-500 and AIME benchmarks. On Qwen3-8B, accuracy improves from 86.2% to 92.0% while average response length drops from 3,099 to 1,115 tokens.

This study introduces section-aware compression for reasoning traces, training models to drop filler narration while preserving compute and verification spans, matching or exceeding original accuracy while using 2-3 times fewer tokens.

This paper proposes Mixed-Policy Distillation (MPD), a framework that transfers concise reasoning behaviors from large teacher models to smaller student models, reducing token usage by up to 27.1% while improving performance.