Tag
Ali distilled Claude Fable 5 reasoning traces into Qwen3-4B, claiming 100% self-consistency and zero hallucination variance, and open-sourced the result.
This paper investigates whether different offline reinforcement learning losses (RFT, RIFT, DFT, Offline GRPO, DPO) for reasoning distillation produce mechanistically distinct weight updates in a small language model. Using identical math rollouts and a controlled setup with Qwen3-4B and attention-only LoRA, they find that SFT, RFT, and RIFT yield nearly colinear weight deltas, while DPO sits in a near-orthogonal subspace and achieves the highest accuracy.
Introduces MARD, a 7B-parameter model for mechanism-level drug-drug interaction prediction using mirror-augmented reasoning distillation, achieving state-of-the-art accuracy at ~1% of frontier API cost and demonstrating genuine pharmacological reasoning over memorization.
LARK proposes a learnability-grounded method for selecting reasoning trajectories in LLM distillation, employing a learnability factor and χ²-regularized selection policy that balances efficiency and generalization, consistently outperforming baselines across models and tasks.
Introduces eXTC, a text classifier with three progressive stages: structured prompt optimization to learn a natural-language rulebook, reasoning distillation into a compact LM, and reinforcement learning to expand reasoning, achieving strong performance and interpretability.
Introduces the Data-Model Compatibility (DMC) metric to evaluate how well a reasoning dataset aligns with a student model during distillation. Experiments show DMC strongly correlates with distillation performance and that dynamically selecting datasets based on DMC further improves reasoning capabilities.
This paper introduces Motab, a new pipeline for LLM reasoning distillation that mitigates both off-policy and on-policy exposure biases by dynamically monitoring student generation and backtracking to safe states with teacher intervention, achieving ~3% average improvement.
This paper proposes a principled offline reasoning distillation framework that corrects teacher-student distribution drift, improving reasoning accuracy on math benchmarks without requiring online rollouts.
A 35B-parameter Qwen3.6 model fine-tuned with Claude-Opus-style chain-of-thought distillation data and released in GGUF quantized formats for efficient local inference.