From Solver Trajectories to Teaching Trajectories: Cognition-Aligned Reasoning Distillation

arXiv cs.AI Papers
knowledge-distillation reasoning-models reinforcement-learning cognition-alignment language-models

Summary

本文提出 CARD（Cognition-Aligned Reasoning Distillation），一种基于 RL 的推理蒸馏框架，将大推理模型的求解轨迹转化为与学生模型认知能力对齐的教学轨迹，通过分布对齐奖励和逻辑连贯性奖励在不重新采样的情况下提升蒸馏效果。

arXiv:2610.36445v1 Announce Type: new Abstract: Knowledge distillation from large reasoning models (LRMs) often uses raw teacher trajectories as supervision. However, this practice creates a mismatch between the goal of distillation and the nature of teacher trajectories. Advanced LRMs typically improve their reasoning capabilities through reward-driven post-training, such as RLHF. This training optimizes them as strong solvers, rather than as teachers that produce reasoning processes smaller models can easily learn. Since smaller models have different "cognitive capacities" compared to their larger teachers, directly imitating raw teacher trajectories can sometimes be ineffective and often requires a substantial number of high-quality samples. To bridge this mismatch, we propose Cognition-Aligned Reasoning Distillation (CARD), an RL-based framework that transforms solver trajectories into teaching trajectories aligned with the student's cognition. Avoiding expensive resampling, our framework directly transforms existing trajectories. We optimize CARD with two reward signals: (1) a Distribution Alignment Reward that matches the student's predictive distribution, and (2) a Logical Coherence Reward that reduces misleading failed reasoning steps. Comprehensive evaluations across multiple student models and settings demonstrate that CARD outperforms distillation from raw teacher trajectories in in-domain settings while achieving substantial gains in out-of-distribution performance. Code is available at https://github.com/LiteSSLHub/CARD.
Original Article
View Cached Full Text

Cached at: 10/28/26, 07:56 AM 

# From Solver Trajectories to Teaching Trajectories: Cognition-Aligned Reasoning Distillation
Source: [https://arxiv.org/abs/2610.36445](https://arxiv.org/abs/2610.36445)
[View PDF](https://arxiv.org/pdf/2610.36445)[HTML \(experimental\)](https://arxiv.org/html/2610.36445v1)

> Abstract:Knowledge distillation from large reasoning models \(LRMs\) often uses raw teacher trajectories as supervision\. However, this practice creates a mismatch between the goal of distillation and the nature of teacher trajectories\. Advanced LRMs typically improve their reasoning capabilities through reward\-driven post\-training, such as RLHF\. This training optimizes them as strong solvers, rather than as teachers that produce reasoning processes smaller models can easily learn\. Since smaller models have different "cognitive capacities" compared to their larger teachers, directly imitating raw teacher trajectories can sometimes be ineffective and often requires a substantial number of high\-quality samples\. To bridge this mismatch, we propose Cognition\-Aligned Reasoning Distillation \(CARD\), an RL\-based framework that transforms solver trajectories into teaching trajectories aligned with the student's cognition\. Avoiding expensive resampling, our framework directly transforms existing trajectories\. We optimize CARD with two reward signals: \(1\) a Distribution Alignment Reward that matches the student's predictive distribution, and \(2\) a Logical Coherence Reward that reduces misleading failed reasoning steps\. Comprehensive evaluations across multiple student models and settings demonstrate that CARD outperforms distillation from raw teacher trajectories in in\-domain settings while achieving substantial gains in out\-of\-distribution performance\. Code is available at[this https URL](https://github.com/LiteSSLHub/CARD)\.

## Submission history

From: Yashuo Luo \[[view email](https://arxiv.org/show-email/c87c87a6/2610.36445)\] **\[v1\]**Tue, 27 Oct 2026 00:46:23 UTC \(1,613 KB\)

Similar Articles

Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information

arXiv cs.CL

This paper proposes a novel Chain-of-Thought distillation framework that transfers teacher models' stepwise attention on key information to student models through a Mixture-of-Layers module for dynamic layer alignment. The method achieves consistent performance improvements on mathematical and commonsense reasoning benchmarks by explicitly guiding student models to progressively focus on critical information during reasoning.

Learning to Revise Reasoning with Segment-wise On-Policy Distillation

arXiv cs.AI

该论文提出 Seg-OPD（分段在线策略蒸馏），通过选择学生模型的不确定推理片段并用教师重写稿作为显式修订监督，训练学生学习如何修订推理步骤。实验显示在数学推理与竞赛编程任务上，相比基线平均相对提升 5.22% 的推理准确率。