reasoning-distillation

CoTranslator 提出将逆向思维链推理轨迹翻译为经验证的正向推理分布，实现高效且可迁移的推理蒸馏；在六个数学基准上较语义抑制最高提升5.82个百分点，并以约35.9%的计算量保留拒采样95.0%的平均准确率。

This paper formulates LLM reasoning distillation as a constrained reinforcement learning problem, maximizing task reward subject to a worst-case teacher-likelihood constraint along every step of the reasoning trajectory, avoiding the reward hacking of pure GRPO and the diluted task performance of KL-based distillation. Experiments on math reasoning and code generation show the method significantly expands the accuracy-fidelity Pareto front.

Ali distilled Claude Fable 5 reasoning traces into Qwen3-4B, claiming 100% self-consistency and zero hallucination variance, and open-sourced the result.

This paper investigates whether different offline reinforcement learning losses (RFT, RIFT, DFT, Offline GRPO, DPO) for reasoning distillation produce mechanistically distinct weight updates in a small language model. Using identical math rollouts and a controlled setup with Qwen3-4B and attention-only LoRA, they find that SFT, RFT, and RIFT yield nearly colinear weight deltas, while DPO sits in a near-orthogonal subspace and achieves the highest accuracy.

Introduces MARD, a 7B-parameter model for mechanism-level drug-drug interaction prediction using mirror-augmented reasoning distillation, achieving state-of-the-art accuracy at ~1% of frontier API cost and demonstrating genuine pharmacological reasoning over memorization.

LARK proposes a learnability-grounded method for selecting reasoning trajectories in LLM distillation, employing a learnability factor and χ²-regularized selection policy that balances efficiency and generalization, consistently outperforming baselines across models and tasks.

Introduces eXTC, a text classifier with three progressive stages: structured prompt optimization to learn a natural-language rulebook, reasoning distillation into a compact LM, and reinforcement learning to expand reasoning, achieving strong performance and interpretability.

Introduces the Data-Model Compatibility (DMC) metric to evaluate how well a reasoning dataset aligns with a student model during distillation. Experiments show DMC strongly correlates with distillation performance and that dynamically selecting datasets based on DMC further improves reasoning capabilities.

This paper introduces Motab, a new pipeline for LLM reasoning distillation that mitigates both off-policy and on-policy exposure biases by dynamically monitoring student generation and backtracking to safe states with teacher intervention, achieving ~3% average improvement.

This paper proposes a principled offline reasoning distillation framework that corrects teacher-student distribution drift, improving reasoning accuracy on math benchmarks without requiring online rollouts.

A 35B-parameter Qwen3.6 model fine-tuned with Claude-Opus-style chain-of-thought distillation data and released in GGUF quantized formats for efficient local inference.