Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation

# Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
Source: [https://arxiv.org/html/2605.19433](https://arxiv.org/html/2605.19433)
Bing Wang1,2,3Shaotian Yan3Chen Shen3Kaiyuan Liu4Sinan Fan4 Ximing Li1,2,6∗Rui Miao3,5Xiaosong Yuan1,2,3Zhanming Shen3,4Jieping Ye3 1College of Computer Science and Technology, Jilin University 2Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University 3Tongyi Lab, Alibaba Group4College of Computer Science and Technology, Zhejiang University 5School of Artificial Intelligence, Jilin University6RIKEN Center for Advanced Intelligence Project \{wangbing1416,zjushenchen,liximing86\}@gmail\.com

###### Abstract

Large language models \(LLMs\) have achieved remarkable success in complex reasoning tasks via long chain\-of\-thought \(CoT\), yet their immense computational overhead hinders real\-world deployment\. LLM reasoning distillation addresses this by transferring reasoning capabilities from formidable teacher models to compact student models\. However, existing distillation paradigms face a fundamental dilemma\. Typical off\-policy distillation strictly utilizes teacher\-generated golden trajectories, suffering from anexposure biasdue to the mismatch between training distributions and student\-generated inference contexts, which leads to error cascades in long CoT reasoning\. To address this, on\-policy distillation allows students to explore their own trajectories, but we demonstrate that it inherently introduces a reciprocalreversed exposure bias: the teacher model also struggles to provide positive guidance when conditioned on student\-generated sub\-optimal contexts\. To resolve this dual exposure bias problem, we proposeMonitoring Trajectories and Backtracking when it strays\(Motab\), a new LLM reasoning distillation pipeline\. Specifically,Motabdynamically monitors the student’s on\-policy generation against an adaptive safety boundary\. When the generation strays and exceeds this threshold,Motabbacktracks to the last safe state and leverages teacher intervention to correct the course\. This approach inherently tolerates minor student errors to mitigate exposure bias, while preventing sub\-optimal contexts to circumvent reversed exposure bias\. Extensive experiments on theLIMO\-v2andAceReasondatasets demonstrate thatMotabeffectively alleviates the dual exposure biases, yielding a roughly 3% average performance improvement in reasoning tasks\.

## 1Introduction

Recently, variouslarge language models\(LLMs\),*e\.g\.,*DeepSeek\[[11](https://arxiv.org/html/2605.19433#bib.bib5)\], have demonstrated remarkable capabilities in complex reasoning tasks that require multi\-stepchain\-of\-thought\(CoT\)\[[38](https://arxiv.org/html/2605.19433#bib.bib13),[34](https://arxiv.org/html/2605.19433#bib.bib6),[43](https://arxiv.org/html/2605.19433#bib.bib8),[46](https://arxiv.org/html/2605.19433#bib.bib56)\]\. Despite their efficacy, deploying these state\-of\-the\-art models in real\-world applications remains computationally intensive, motivating the adoption of knowledge distillation to transfer their reasoning capabilities into smaller, faster student models, namelyLLM reasoning distillation\[[1](https://arxiv.org/html/2605.19433#bib.bib9),[43](https://arxiv.org/html/2605.19433#bib.bib8),[42](https://arxiv.org/html/2605.19433#bib.bib14)\]\.

Typically, most previous works follow anoff\-policy distillationparadigm\[[12](https://arxiv.org/html/2605.19433#bib.bib15),[27](https://arxiv.org/html/2605.19433#bib.bib17)\]: a teacher model generates an extensive corpus of reasoning trajectories for a given set of complex problems; these trajectories are subsequently filtered using heuristic pipelines\[[21](https://arxiv.org/html/2605.19433#bib.bib18),[10](https://arxiv.org/html/2605.19433#bib.bib19),[14](https://arxiv.org/html/2605.19433#bib.bib16)\]and utilized for supervised fine\-tuning of the student model\. However, this off\-policy approach empirically suffers from anexposure biasissue\[[1](https://arxiv.org/html/2605.19433#bib.bib9),[41](https://arxiv.org/html/2605.19433#bib.bib11)\]\. Specifically, during the training phase, the student model is exclusively optimized using thegoldentrajectories provided by the teacher model\. Conversely, during inference, the student model is exposed to its ownsub\-optimalgenerated context\. This mismatch inevitably leads to an error cascade, particularly within the context of long CoT reasoning trajectories\[[42](https://arxiv.org/html/2605.19433#bib.bib14)\]\.

To mitigate this issue, the community recently proposed an alternative paradigm, namelyon\-policy distillation\[[1](https://arxiv.org/html/2605.19433#bib.bib9),[18](https://arxiv.org/html/2605.19433#bib.bib23)\]\. Specifically, the student model generates its own reasoning trajectories, which are then guided through dense feedback provided by the teacher model,*e\.g\.,*token probabilities\[[1](https://arxiv.org/html/2605.19433#bib.bib9),[9](https://arxiv.org/html/2605.19433#bib.bib20),[17](https://arxiv.org/html/2605.19433#bib.bib21)\]\. Although this method effectively eliminates exposure bias by exposing the student’s own inference\-time trajectories, in this work, we formally and empirically demonstrate that on\-policy distillation also introduces a reciprocalreversed exposure biaschallenge\. Analogously, when the student model generates sub\-optimal on\-policy trajectories as context, the teacher model often suppresses these tokens and struggles to provide positive supervisory signals\[[8](https://arxiv.org/html/2605.19433#bib.bib26)\]\. As experimentally evidenced by Figs\.[1](https://arxiv.org/html/2605.19433#S2.F1)and[2](https://arxiv.org/html/2605.19433#S2.F2), a consistent discrepancy exists between the teacher’s and student’s next\-token probability distributions when conditioned on student\-generated on\-policy trajectories, and the divergence between the teacher’s distributions under off\-policy versus on\-policy contexts consistently increases as the sequence length extends\.

Therefore, to specifically address thesedual exposure biases, the ideal distillation dataset shouldcoverthe student model’s own sub\-optimal on\-policy data distribution to mitigate exposure bias, while the teacher model must provide supervision within arelatively accuratestudent\-generated context to circumvent reversed exposure bias, which creates an apparent dilemma\. To achieve these goals, we propose a new LLM reasoning distillation method, namelyMonitoring Trajectories and Backtracking when it strays\(Motab\)\. Specifically,Motabconsistently monitors the student model’s on\-policy trajectories against an adaptive safety boundary,*e\.g\.,*based on the entropy of teacher\-generated tokens\. Once a generation step exceeds this threshold,Motabbacktracks to the last safe state through a credit assignment strategy, where the teacher model intervenes to steer the subsequent generation\. Accordingly, this approach utilizes a safety boundary to tolerate sub\-optimal on\-policy trajectories, encouraging the model to expose its own errors and thereby mitigating the exposure bias\. When the student model strays,Motabemploys backtracking to identify a high\-quality on\-policy context, allowing the teacher model to provide guidance that effectively addresses the reversed exposure bias\.

To evaluate the performance ofMotab, we utilize Qwen3\-32B\[[43](https://arxiv.org/html/2605.19433#bib.bib8)\]and gpt\-oss\-120b\[[28](https://arxiv.org/html/2605.19433#bib.bib54)\]as teacher models to generate distillation data on the questions fromLIMO\-v2\[[45](https://arxiv.org/html/2605.19433#bib.bib25)\]andAceReason\[[6](https://arxiv.org/html/2605.19433#bib.bib24)\]datasets usingMotab, and perform full\-parameter fine\-tuning on three distinct student models\. Experimental results demonstrate that our approach yields an average improvement of approximately 3% in complex reasoning performance and effectively mitigates dual exposure biases\.Our source code is released in the repository[https://github\.com/wangbing1416/MOTAB](https://github.com/wangbing1416/MOTAB)\.

In summary, our contributions can be summarized as the follow three\-folds:

- •We formally and experimentally summarize a fundamental dilemma inherent in existing off\-policy and on\-policy distillation methods,*i\.e\.,*the dual exposure biases problem\.
- •To address the biases, we introduceMotab, a new reasoning distillation framework that monitors student\-generated on\-policy trajectories within an adaptive safety boundary, and backtracks to the last safe point when the student model strays\.
- •Extensive experimental results demonstrate that our approach consistently enhances LLM reasoning performance and alleviates the dual exposure biases\.

## 2Investigation on Dual Exposure Biases

In this section, we formally define and empirically investigate the dual exposure biases in LLM reasoning distillation\. The detailed theoretical analysis of these biases is provided in Appendix[A](https://arxiv.org/html/2605.19433#A1)\.

### 2\.1Formal Definition of Dual Exposure Biases

Task descriptions of LLM reasoning and its distillation\.We formalize the LLM reasoning process as an auto\-regressive sequence generation task\. Given an initial question𝐪\\mathbf\{q\}, a policy LLM𝝅\\boldsymbol\{\\pi\}generates a reasoning trajectory consisting of a sequence of reasoning steps𝒴=\(𝐲1,𝐲2,…,𝐲L\)\\mathcal\{Y\}=\(\\mathbf\{y\}\_\{1\},\\mathbf\{y\}\_\{2\},\\dots,\\mathbf\{y\}\_\{L\}\)\. At any stepll, the reasoning context𝐬l\\mathbf\{s\}\_\{l\}is defined as the concatenation of the question and the previously generated prefix as𝐬l=\[𝐪,𝒴<l\]\\mathbf\{s\}\_\{l\}=\[\\mathbf\{q\},\\mathcal\{Y\}\_\{<l\}\]\. The policy𝝅\(⋅∣𝐬l\)\\boldsymbol\{\\pi\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\}\)defines the probability distribution over the vocabulary space for the next generation step\. We denoted𝝅​\(𝐬\)d^\{\\boldsymbol\{\\pi\}\}\(\\mathbf\{s\}\)as the marginal distribution of contexts induced by the policy𝝅\\boldsymbol\{\\pi\}during free\-form inference\. The ultimate goal of reasoning distillation is to optimize a student policy𝝅S\\boldsymbol\{\\pi\}\_\{S\}to accurately replicate the reasoning capabilities of a capable teacher policy𝝅T\\boldsymbol\{\\pi\}\_\{T\}\.

Formulation of dual exposure biases\.Existing reasoning distillation methods can be broadly categorized into off\-policy and on\-policy approaches\. However, both paradigms inherently suffer from different forms of exposure biases:

- •Exposure bias of off\-policy distillation:Typical off\-policy methods follow the standard teacher\-forcing objective, which minimizes theKullback\-Leibler\(KL\) divergence over the teacher’s context distributiond𝝅Td^\{\\boldsymbol\{\\pi\}\_\{T\}\}: 𝒥off\(𝝅S\)=𝔼𝐬∼d𝝅T\[DKL\(𝝅T\(⋅∣𝐬\)∥𝝅S\(⋅∣𝐬\)\)\]\.\\mathcal\{J\}\_\{\\text\{off\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\)=\\mathbb\{E\}\_\{\\mathbf\{s\}\\sim d^\{\\boldsymbol\{\\pi\}\_\{T\}\}\}\\big\[D\_\{\\text\{KL\}\}\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\)\)\\big\]\.\(1\)However, during test\-time inference, the student auto\-regressively generates trajectories from its own distributiond𝝅Sd^\{\\boldsymbol\{\\pi\}\_\{S\}\}\. Because the training distribution mismatches the inference distribution \(d𝝅T≠d𝝅Sd^\{\\boldsymbol\{\\pi\}\_\{T\}\}\\neq d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\), a distribution shift occurs\. Once the student makes an initial error, it encounters a context𝐬∉supp​\(d𝝅T\)\\mathbf\{s\}\\notin\\text\{supp\}\(d^\{\\boldsymbol\{\\pi\}\_\{T\}\}\), inevitably leading to cascading errors\.
- •Reversed exposure bias of on\-policy distillation:To mitigate this distribution shift, on\-policy methods directly sample contexts from the student’s own distributiond𝝅Sd^\{\\boldsymbol\{\\pi\}\_\{S\}\}: 𝒥on\(𝝅S\)=𝔼𝐬∼d𝝅S\[DKL\(𝝅S\(⋅∣𝐬\)∥𝝅T\(⋅∣𝐬\)\)\]\.\\mathcal\{J\}\_\{\\text\{on\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\)=\\mathbb\{E\}\_\{\\mathbf\{s\}\\sim d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\}\\big\[D\_\{\\text\{KL\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\)\\parallel\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\)\)\\big\]\.\(2\)However, when𝐬∼d𝝅S\\mathbf\{s\}\\sim d^\{\\boldsymbol\{\\pi\}\_\{S\}\}contains severe logical flaws due to the student’s sub\-optimal capabilities, this trajectory acts as anout\-of\-distributioninput for the teacher\. Consequently, the supervision signal𝝅T\(⋅∣𝐬\)\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\)always suppresses student tokens and fails to provide the positive feedback, ultimately degrading into uninformative noise\.

Accordingly, existing methods struggle to simultaneously align the training with the inference\-time distribution while maintaining valid, high\-quality supervision, which forms a fundamentaldilemma, resulting in thedual exposure biases\. More formulations can be seen in Appendices[A\.1](https://arxiv.org/html/2605.19433#A1.SS1)and[A\.2](https://arxiv.org/html/2605.19433#A1.SS2)\.

![Refer to caption](https://arxiv.org/html/2605.19433v1/x1.png)Figure 1:Next\-token log probabilities of teacher and student under off\-policy and on\-policy contexts\.![Refer to caption](https://arxiv.org/html/2605.19433v1/x2.png)Figure 2:KL divergence between off\-policy and on\-policy trajectories across context lengths\.
### 2\.2Empirical Analysis of Dual Exposure Biases

To substantiate the existence of the dual exposure biases, we conduct preliminary experiments using Qwen3\-32B and gpt\-oss\-120b as teacher models, and Qwen3\-4B\-Instruct, which is initialized via an off\-policy distillation phase on datasets corresponding to their respective teachers, as the student\.

We first evaluate the models on off\-policy data, where the context comprises golden trajectories generated exclusively by the teacher\. As illustrated in Fig\.[1](https://arxiv.org/html/2605.19433#S2.F1), the teacher’s predictive confidence is consistently high, given that the context perfectly aligns with its native distribution\. Conversely, the student model struggles to replicate these exact reasoning steps and exhibits significantly lower confidence\. Furthermore, as shown in Fig\.[2](https://arxiv.org/html/2605.19433#S2.F2), we illustrate the KL divergence of the next\-token probability distributions produced by the student model when conditioned on its own auto\-regressively generated on\-policy contexts versus teacher\-generated off\-policy contexts\. Generally, this divergence consistently increases as the reasoning sequence lengthens\. Therefore, these results clearly illustrate the exposure bias: training solely on golden contexts leaves the student increasingly vulnerable to its own stylistic deviations, causing errors to accumulate rapidly during inference\-time generation\.

To mitigate exposure bias, on\-policy methods rely on the teacher to supervise student\-generated trajectories\. However, our experiments also uncover a criticalreversed exposure bias\. As shown in Fig\.[1](https://arxiv.org/html/2605.19433#S2.F1), when conditioning on sub\-optimal on\-policy trajectories generated by the student, the student model’s step\-wise confidence for its self\-generated token is consistently higher than the teacher’s confidence for the same token\. Consequently, the guidance provided by the teacher model, based on token probabilities\[[1](https://arxiv.org/html/2605.19433#bib.bib9)\], consistently suppresses these student\-generated tokens, thereby hindering the provision of effective positive supervision\. Similarly, in Fig\.[2](https://arxiv.org/html/2605.19433#S2.F2), the KL divergence of the next\-token probability distributions estimated by the teacher model across on\-policy and off\-policy contexts increases with sequence length, thereby also exhibiting an error accumulation phenomenon\.

### 2\.3What is the Ideal Unbiased State for Reasoning Distillation?

Given the dual exposure biases formulated in Eqs\. \([1](https://arxiv.org/html/2605.19433#S2.E1)\) and \([2](https://arxiv.org/html/2605.19433#S2.E2)\) and empirically demonstrated above, anideal LLM reasoning distillation processmust satisfy the following two theoretical conditions:

- •Coveragefor exposure bias: The training data must be sampled from the student’s intrinsic context distributiond𝝅S​\(𝐬\)d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\(\\mathbf\{s\}\), encompassing the sub\-optimal prefixes the student is likely to encounter during real\-world inference to eliminate distribution shift\.
- •Validityfor reversed exposure bias: The teacher model should be conditioned on a valid context𝐬∗\\mathbf\{s\}^\{\*\}, enabling it to provide informative positive supervision signals\.

Formally, building upon these two conditions, the ideal reasoning distillation objective aims to minimize the expected negative log\-likelihood over this optimal configuration:

𝒥ideal​\(𝝅S\)=𝔼𝐬∼d𝝅S​\[𝔼𝐲∗∼𝝅T\(⋅∣𝐬∗\)​\[−log⁡𝝅S​\(𝐲∗∣𝐬\)\]\]\.\\mathcal\{J\}\_\{\\text\{ideal\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\)=\\mathbb\{E\}\_\{\\mathbf\{s\}\\sim d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\}\\left\[\\mathbb\{E\}\_\{\\mathbf\{y\}^\{\*\}\\sim\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}^\{\*\}\)\}\\left\[\-\\log\\boldsymbol\{\\pi\}\_\{S\}\(\\mathbf\{y\}^\{\*\}\\mid\\mathbf\{s\}\)\\right\]\\right\]\.\(3\)In practice, however, this remains challenging, as they exists an inherent dilemma,*i\.e\.,*𝐬∗≠𝐬∼d𝝅S\\mathbf\{s\}^\{\*\}\\neq\\mathbf\{s\}\\sim d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\.

### 2\.4Can Speculative KD Resolve Dual Exposure Biases?

In the community, to balance off\-policy and on\-policy data, some recent works proposespeculative knowledge distillation\(SKD\)\[[41](https://arxiv.org/html/2605.19433#bib.bib11),[16](https://arxiv.org/html/2605.19433#bib.bib12)\], where the teacher continuously monitors and corrects the student’s on\-policy generation\. The core intuition is that if an erroneous tokeny^i\\hat\{y\}\_\{i\}is produced by the student, it is immediately replaced by a teacher\-generated token sampled from𝝅T\(⋅∣𝐬S\)\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}^\{S\}\)\. The training objective for this pipeline is as follows:

𝒥skd​\(𝝅S\)=𝔼𝐬∼dmix​\[𝔼y^∼𝝅T\(⋅∣𝐬\)​\[−log⁡𝝅S​\(y^∣𝐬\)\]\],\\mathcal\{J\}\_\{\\text\{skd\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\)=\\mathbb\{E\}\_\{\\mathbf\{s\}\\sim d\_\{\\text\{mix\}\}\}\\left\[\\mathbb\{E\}\_\{\\hat\{y\}\\sim\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\)\}\[\-\\log\\boldsymbol\{\\pi\}\_\{S\}\(\\hat\{y\}\\mid\\mathbf\{s\}\)\]\\right\],\(4\)wheredmix=β​d𝝅S\+\(1−β\)​d𝝅Td\_\{\\text\{mix\}\}=\\beta d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\+\(1\-\\beta\)d^\{\\boldsymbol\{\\pi\}\_\{T\}\}denotes a mixture distribution between the teacher’s and the student’s context distributions\. However, by comparing𝒥skd\\mathcal\{J\}\_\{\\text\{skd\}\}in Eq\. \([4](https://arxiv.org/html/2605.19433#S2.E4)\) with the ideal objective𝒥ideal\\mathcal\{J\}\_\{\\text\{ideal\}\}in Eq\. \([3](https://arxiv.org/html/2605.19433#S2.E3)\), we identify two critical gaps in SKD:

- •Coverage gap\(dmix≠d𝝅Sd\_\{\\text\{mix\}\}\\neq d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\): By monitoring the student’s generation process step\-by\-step, SKD strictly corrects the steps where the student deviates from the teacher’s distribution, thereby failing to expose the student’s sub\-optimal trajectories\.
- •Validity gap\(𝝅T\(⋅∣𝐬\)≠𝝅T\(⋅∣𝐬∗\)\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\)\\neq\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}^\{\*\}\)\): Forced token replacements break the natural auto\-regressive coherence of the trajectory, so the teacher𝝅T\\boldsymbol\{\\pi\}\_\{T\}still suffers from out\-of\-distribution collapse\. Lacking the global error\-correction, it will produce high\-entropy noise rather than meaningful guidance, violating the validity condition\.

More formal demonstrations of the failure of this mixture distribution are provided in Appendix[A\.3](https://arxiv.org/html/2605.19433#A1.SS3)\. Therefore, to bridge these two gaps and more accurately approximate the ideal objective𝒥ideal\\mathcal\{J\}\_\{\\text\{ideal\}\}, we propose a more robust data synthesis frameworkMotabin this work\.

## 3The ProposedMotab

In this section, we introduce our proposedMotabmethod that mitigate dual exposure biases\.

Method overview\.To address the dilemma of the dual exposure biases, the ideal conditions should satisfy both coverage of the student’s sub\-optimal distribution and validity, ensuring the teacher provides guidance within a relatively correct context\. To achieve this, the basic idea ofMotabinvolves consistently monitoring the on\-policy trajectories generated by the student,tolerating sub\-optimal paths within an adaptive safety boundary, and backtracking to a relatively safe state for teacher intervention once that boundary is exceeded\.

![Refer to caption](https://arxiv.org/html/2605.19433v1/x3.png)Figure 3:Overall framework of the proposedMotab\.Specifically,Motabis an iterative data synthesis framework comprised of three critical steps: \(1\)on\-policy trajectory monitoring: As the student generates each reasoning step, its likelihood is consistently monitored by the teacher model\. Specifically, the entropy of the step generated by the teacher within the given context serves as the adaptive safety boundary\. Higher entropy signifies the existence of multiple viable paths toward the correct solution\[[37](https://arxiv.org/html/2605.19433#bib.bib27)\], thereby expanding the safety margin\. As long as the student’s trajectory remains within this boundary, its sub\-optimal trajectories are tolerated; \(2\)State backtracking: Once the student violates this safety boundary, a credit assignment mechanism\[[33](https://arxiv.org/html/2605.19433#bib.bib28),[40](https://arxiv.org/html/2605.19433#bib.bib29)\]is employed to trace back along the on\-policy trajectory\. This process identifies the specific step characterized by a precipitous decline in likelihood scores, facilitating a backtrack to the last safe state preceding the error; \(3\)off\-policy trajectory stitching: Within this relatively correct context following the backtrack, the teacher model performs a continuation to generate a corrected sequence\. Instead of directly using this teacher\-generated trajectory as training data, we append the teacher’s suffix to the on\-policy trajectory precisely at the point where the safety boundary is violated, therefore to explicitly teach the student model how to self\-correct its previous deviations\. For clarity, the overall framework ofMotabis depicted in Fig\.[3](https://arxiv.org/html/2605.19433#S3.F3)\. In the following subsections, we formally describe the details of these three steps, respectively\.

### 3\.1On\-policy Trajectory Monitoring via Adaptive Boundary

To satisfy thecoveragecondition, the student model must be allowed to explore its intrinsic distributiond𝝅Sd^\{\\boldsymbol\{\\pi\}\_\{S\}\}\. However, while unconstrained exploration inevitably leads to severe logical hallucinations, excessively restrictive student exploration,*e\.g\.,*SKD\[[41](https://arxiv.org/html/2605.19433#bib.bib11),[44](https://arxiv.org/html/2605.19433#bib.bib31)\], fails to expose the student’s sub\-optimal distribution\. Therefore,Motabmonitors the student’s step\-wise generation using a teacher\-guidedadaptive safety boundary\. Given a reasoning context𝐬l−1\\mathbf\{s\}\_\{l\-1\}at stepll, the student policy𝝅S\\boldsymbol\{\\pi\}\_\{S\}generates a candidate reasoning step𝐲lS\\mathbf\{y\}\_\{l\}^\{S\}\. To evaluate the validity of this step without aggressively truncating acceptable sub\-optimal logic, we define a step\-level value functionVT∈\[0,1\]V\_\{T\}\\in\[0,1\]using the teacher’s length\-normalized conditional likelihood\[[41](https://arxiv.org/html/2605.19433#bib.bib11),[24](https://arxiv.org/html/2605.19433#bib.bib30)\]:

VT​\(𝐬l−1,𝐲lS\)=exp⁡\(1\|𝐲lS\|​∑i=1\|𝐲lS\|log⁡𝝅T​\(yl,iS∣𝐬l−1,yl,<iS\)\),V\_\{T\}\\left\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}^\{S\}\\right\)=\\exp\\left\(\\frac\{1\}\{\|\\mathbf\{y\}\_\{l\}^\{S\}\|\}\\sum\\nolimits\_\{i=1\}^\{\|\\mathbf\{y\}\_\{l\}^\{S\}\|\}\\log\\boldsymbol\{\\pi\}\_\{T\}\\left\(y\_\{l,i\}^\{S\}\\mid\\mathbf\{s\}\_\{l\-1\},y\_\{l,<i\}^\{S\}\\right\)\\right\),\(5\)whereyl,iSy\_\{l,i\}^\{S\}denotes theii\-th token of the reasoning step𝐲lS\\mathbf\{y\}\_\{l\}^\{S\}\. Instead of utilizing a rigid, static threshold which artificially restricts the student’s exploration, we introduce a state\-adaptive boundaryγ​\(𝐬l−1\)\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)\. This boundary is inversely proportional to the teacher model’s predictive sequence entropyℋ​\(𝝅T∣𝐬l−1\)\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\. A high entropy indicates a reasoning manifold with multiple acceptable paths\[[37](https://arxiv.org/html/2605.19433#bib.bib27),[7](https://arxiv.org/html/2605.19433#bib.bib32)\], warranting a lower threshold, which allows the student to explore more freely; conversely, deterministic axiomatic steps require strict adherence\. The adaptive boundary is formulated as

γ​\(𝐬l−1\)=γ0⋅exp⁡\(−α⋅ℋ​\(𝝅T∣𝐬l−1\)\),\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)=\\gamma\_\{0\}\\cdot\\exp\\left\(\-\\alpha\\cdot\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\\right\),\(6\)whereγ0∈\(0,1\]\\gamma\_\{0\}\\in\(0,1\]is the base threshold andα\>0\\alpha\>0is a scaling factor\. IfVT​\(𝐬l−1,𝐲lS\)≥γ​\(𝐬l−1\)V\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}^\{S\}\)\\geq\\gamma\(\\mathbf\{s\}\_\{l\-1\}\), the step is deemed safe, and the state transitions normally to𝐬l=\[𝐬l−1,𝐲lS\]\\mathbf\{s\}\_\{l\}=\[\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}^\{S\}\]\. This mechanism guarantees that the student’s logically sound, albeit stylistic, explorations are preserved\.

### 3\.2State Backtracking via Credit Assignment

When the student’s generation breaches the safety boundary,*i\.e\.,*VT<γ​\(𝐬l−1\)V\_\{T\}<\\gamma\(\\mathbf\{s\}\_\{l\-1\}\), a critical logical collapse is detected\. However, directly replacing the immediate step𝐲lS\\mathbf\{y\}\_\{l\}^\{S\}is myopic, as logical errors are often delayed,*e\.g\.,*a flawed premise established at stepl−2l\-2only manifests as a mathematical contradiction at stepll\. Furthermore, forcing the teacher to correct from the flawed state𝐬l−1\\mathbf\{s\}\_\{l\-1\}violates thevaliditycondition, triggering reversed exposure bias\. To address this,Motabemploys acredit assignmentstrategy\[[25](https://arxiv.org/html/2605.19433#bib.bib33),[29](https://arxiv.org/html/2605.19433#bib.bib34)\]to backtrack to the safe bifurcation step\. Specifically, we compute the step\-wisetemporal difference errorδk\\delta\_\{k\}\[[33](https://arxiv.org/html/2605.19433#bib.bib28),[40](https://arxiv.org/html/2605.19433#bib.bib29)\]along the historical trajectory to identify the precise moment the logic deviated from the teacher’s safe manifold:

δk=VT​\(𝐬k−1,𝐲kS\)−VT​\(𝐬k−2,𝐲k−1S\),∀k∈\{1,…,l\}\.\\delta\_\{k\}=V\_\{T\}\\left\(\\mathbf\{s\}\_\{k\-1\},\\mathbf\{y\}\_\{k\}^\{S\}\\right\)\-V\_\{T\}\\left\(\\mathbf\{s\}\_\{k\-2\},\\mathbf\{y\}\_\{k\-1\}^\{S\}\\right\),\\quad\\forall k\\in\\\{1,\\dots,l\\\}\.\(7\)The safe stepl∗l^\{\*\}is identified as the node exhibiting the most severe degradation in oracle value, strictly constrained such that the preceding state remains safe:

l∗=arg⁡mink≤l⁡\(δk\)s\.t\.VT​\(𝐬l∗−2,𝐲l∗−1S\)≥γ​\(𝐬l∗−2\)\.l^\{\*\}=\\arg\\min\_\{k\\leq l\}\(\\delta\_\{k\}\)\\quad\\text\{s\.t\.\}\\quad V\_\{T\}\\left\(\\mathbf\{s\}\_\{l^\{\*\}\-2\},\\mathbf\{y\}\_\{l^\{\*\}\-1\}^\{S\}\\right\)\\geq\\gamma\(\\mathbf\{s\}\_\{l^\{\*\}\-2\}\)\.\(8\)By isolatingl∗l^\{\*\},Motabrewinds the reasoning process strictly to the last pristine context𝐬l∗−1\\mathbf\{s\}\_\{l^\{\*\}\-1\}\. This structurally isolates the teacher from any out\-of\-distribution sub\-optimal steps, ensuring that the subsequent teacher supervision remains uncorrupted by reversed exposure bias\.

### 3\.3Off\-policy Trajectory Stitching

Having identified the safe prefix𝐬l∗−1\\mathbf\{s\}\_\{l^\{\*\}\-1\}, the teacher policy𝝅T\\boldsymbol\{\\pi\}\_\{T\}is queried to generate a valid, off\-policy corrective suffix𝒴l∗:TT\\mathcal\{Y\}\_\{l^\{\*\}:T\}^\{T\}as

𝒴l∗:TT∼𝝅T\(⋅∣𝐬l∗−1\)\.\\mathcal\{Y\}\_\{l^\{\*\}:T\}^\{T\}\\sim\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l^\{\*\}\-1\}\)\.\(9\)Because𝐬l∗−1\\mathbf\{s\}\_\{l^\{\*\}\-1\}belongs to the valid support manifold of the teacher,𝝅T\\boldsymbol\{\\pi\}\_\{T\}provides highly confident and accurate guidance\. To construct the final trajectory for fine\-tuning, instead of directly use\[𝐬l∗−1,𝒴l∗:TT\]\[\\mathbf\{s\}\_\{l^\{\*\}\-1\},\\mathcal\{Y\}\_\{l^\{\*\}:T\}^\{T\}\], we seamlessly stitch the student’s sub\-optimal exploration, an explicit semantic transition, and the teacher’s correction\. Specifically, we retain the student’s fully flawed trajectory up to unsafe stepllto act as negative exposure\. A discrete semantic revising token, denoted as\[REV\],*e\.g\.,*“However”, is appended to asynchronously stitch the pristine correction to the flawed context:

𝝉SFT=\[𝐪,𝒴1:lS\]⊕\[REV\]⊕𝒴l∗:TT\.\\boldsymbol\{\\tau\}\_\{\\text\{SFT\}\}=\\left\[\\mathbf\{q\},\\mathcal\{Y\}\_\{1:l\}^\{S\}\\right\]\\oplus\\texttt\{\[REV\]\}\\oplus\\mathcal\{Y\}\_\{l^\{\*\}:T\}^\{T\}\.\(10\)During fine\-tuning, by conditioning the student’s loss on the sub\-optimal on\-policy state to predict\[REV\]and then output the correct steps froml∗l^\{\*\}, the student explicitly learns to: \(1\) operate robustly within its own generation distribution \(mitigating exposure bias\), \(2\) recognize its own logical drift, and \(3\) perform self\-correction using valid oracle targets \(bypassing reversed exposure bias\)\.

### 3\.4Theoretical Alignment to The Ideal Objective

To theoretically validateMotab, we demonstrate its alignment with the ideal objective𝒥ideal\\mathcal\{J\}\_\{\\text\{ideal\}\}in Eq\. \([3](https://arxiv.org/html/2605.19433#S2.E3)\) and illustrate how it resolves thecoverageandvaliditygaps inherent in SKD in Eq\. \([4](https://arxiv.org/html/2605.19433#S2.E4)\)\.

InMotab, the student policy is optimized via maximum likelihood estimation over the dynamically stitched trajectories\. The expected empirical loss𝒥Motab​\(𝝅S\)\\mathcal\{J\}\_\{\\text\{\{Motab\}\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\)can be decomposed into a piecewise formulation conditionally dependent on the adaptive boundary:

𝒥Motab​\(𝝅S\)=\\displaystyle\\mathcal\{J\}\_\{\\text\{\{Motab\}\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\)=𝔼𝐬l−1∼d𝝅S𝔼𝐲lS∼𝝅S\[𝕀\[VT≥γ​\(𝐬\)\]\[−log𝝅S\(𝐲lS∣𝐬l−1\)\]\\displaystyle\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim d^\{\\boldsymbol\{\\pi\}\_\{S\}\}\}\\mathbb\{E\}\_\{\\mathbf\{y\}\_\{l\}^\{S\}\\sim\\boldsymbol\{\\pi\}\_\{S\}\}\\Big\[\\mathbb\{I\}\_\{\[V\_\{T\}\\geq\\gamma\(\\mathbf\{s\}\)\]\}\\left\[\-\\log\\boldsymbol\{\\pi\}\_\{S\}\\left\(\\mathbf\{y\}\_\{l\}^\{S\}\\mid\\mathbf\{s\}\_\{l\-1\}\\right\)\\right\]\(11\)\+𝕀\[VT<γ​\(𝐬\)\]𝔼𝒴T∼𝝅T\(⋅∣𝐬l∗−1\)\[−log𝝅S\(\[REV\]⊕𝒴T∣𝐬l−1⊕𝐲tS\)\]\]\.\\displaystyle\+\\mathbb\{I\}\_\{\[V\_\{T\}<\\gamma\(\\mathbf\{s\}\)\]\}\\mathbb\{E\}\_\{\\mathcal\{Y\}^\{T\}\\sim\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l^\{\*\}\-1\}\)\}\\left\[\-\\log\\boldsymbol\{\\pi\}\_\{S\}\\left\(\\texttt\{\[REV\]\}\\oplus\\mathcal\{Y\}^\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\\oplus\\mathbf\{y\}\_\{t\}^\{S\}\\right\)\\right\]\\Big\]\.By comparing Eq\. \([11](https://arxiv.org/html/2605.19433#S3.E11)\) with𝒥ideal\\mathcal\{J\}\_\{\\text\{ideal\}\}, we show thatMotabstrictly satisfies both prerequisite conditions: Unlike SKD which restricts states to a narrow mixturedmixd\_\{\\text\{mix\}\}, Eq\. \([11](https://arxiv.org/html/2605.19433#S3.E11)\) operates strictly over𝐬l−1∼d𝝅S\\mathbf\{s\}\_\{l\-1\}\\sim d^\{\\boldsymbol\{\\pi\}\_\{S\}\}and𝐲tS∼𝝅S\\mathbf\{y\}\_\{t\}^\{S\}\\sim\\boldsymbol\{\\pi\}\_\{S\}\. Retaining the flawed context\(𝐬l−1⊕𝐲tS\)\(\\mathbf\{s\}\_\{l\-1\}\\oplus\\mathbf\{y\}\_\{t\}^\{S\}\)as the prefix ensures full coverage of the student’s authentic inference manifold, effectively eliminating the distribution shift\. Forcing the teacher to predict upon a flawed context𝐬t\\mathbf\{s\}\_\{t\}induces out\-of\-distribution noise \(𝝅T≠𝝅T∗\\boldsymbol\{\\pi\}\_\{T\}\\neq\\boldsymbol\{\\pi\}\_\{T\}^\{\*\}\)\.Motabapproximates the ideal oracle𝝅T∗\\boldsymbol\{\\pi\}\_\{T\}^\{\*\}by sampling the correction𝒴T\\mathcal\{Y\}^\{T\}exclusively from the pristine state𝐬l∗−1∈supp​\(d𝝅T\)\\mathbf\{s\}\_\{l^\{\*\}\-1\}\\in\\text\{supp\}\(d^\{\\boldsymbol\{\\pi\}\_\{T\}\}\)\. Stitching via the\[REV\]token guarantees the teacher’s supervision remains valid and highly confident\. We provide a more rigorous theoretical analysis ofMotabin Appendix[A\.4](https://arxiv.org/html/2605.19433#A1.SS4)\.

Accordingly,𝒥Motab\\mathcal\{J\}\_\{\\text\{\{Motab\}\}\}serves as a robust estimator for𝒥ideal\\mathcal\{J\}\_\{\\text\{ideal\}\}, simultaneously mitigating the dual exposure biases through structurally sound data synthesis\.

## 4Experimental Evaluation

Experimental settings\.Our problem sets are derived from theLIMO\-v2repository\[[45](https://arxiv.org/html/2605.19433#bib.bib25)\]and a randomly sampled subset of 10,000 questions from theAceReasonrepository\[[6](https://arxiv.org/html/2605.19433#bib.bib24)\]\. We evaluate our trained student models across three mathematical reasoning benchmarks: AIME24, AIME25, and MATH500\[[22](https://arxiv.org/html/2605.19433#bib.bib47)\], and two cross\-domain benchmarks: IFEval\[[50](https://arxiv.org/html/2605.19433#bib.bib48)\]and GPQA\[[31](https://arxiv.org/html/2605.19433#bib.bib49)\]\. For AIME24, AIME25, and GPQA, we sample 8 times per problem and report the pass@8, pass@4, and pass@1 metrics\. We utilize Qwen3\-32B and gpt\-oss\-120b as teacher models, while Qwen3\-4B\-Instruct, Qwen2\.5\-7B\-Instruct, and Llama3\.2\-3B serve as the student models\. All experiments were conducted on an 8\-GPU A800 cluster, employing the SGLang framework for data generation and LlamaFactory for model training\. Our selected baselines includesupervised fine\-tuning\(SFT\),rejection sampling fine\-tuning\(RFT\), ImitKD\[[23](https://arxiv.org/html/2605.19433#bib.bib10)\], and SKD\[[41](https://arxiv.org/html/2605.19433#bib.bib11)\]\. Specifically, ImitKD involves randomly selecting steps from either the student or the teacher during the decoding process\. In contrast, SKD accepts a student\-generated step only if the log probability assigned by the teacher exceeds the student’s log probability by a factor ofβ\\beta; otherwise, the step is rejected, and the teacher generates a replacement\. More detailed settings,*e\.g\.,*LLM model cards and implementation details, can be seen in Appendix[D](https://arxiv.org/html/2605.19433#A4)\.

Table 1:Experimental results using Qwen3\-4B\-Instruct as the student model\.TeacherMethodAIME24AIME25MATH500Avg\.P@8P@4P@1P@8P@4P@1L1L2L3L4L5Qwen3\-32BSFT50\.0037\.8121\.2540\.0035\.4324\.1790\.7082\.2277\.1470\.3151\.4952\.77RFT46\.6738\.3821\.6740\.0030\.9521\.6783\.7282\.2285\.7165\.6251\.4951\.65ImitKD46\.6740\.7124\.1740\.0033\.0520\.0093\.0282\.2287\.6277\.3460\.4555\.02SKD\(β=0\.6\\beta=0\.6\)53\.3345\.0025\.0046\.6743\.3325\.0088\.3785\.5589\.5275\.7860\.4558\.09SKD\(β=0\.8\\beta=0\.8\)53\.3348\.3324\.1746\.6745\.0027\.5088\.3786\.6787\.6182\.0363\.4359\.37SKD\(β=1\.0\\beta=1\.0\)56\.6743\.3327\.5046\.6740\.0025\.8390\.0786\.6788\.5781\.2556\.7258\.47Motab\(ours\)66\.6757\.5734\.1756\.6750\.7131\.6793\.0286\.6795\.2486\.7273\.8866\.64gpt\-oss\-120bSFT76\.6773\.1443\.3373\.3361\.4837\.5090\.7090\.0080\.0074\.2250\.7568\.28ImitKD76\.6772\.2944\.1763\.3357\.2937\.9281\.4067\.7871\.4369\.5361\.1963\.91SKD\(β=0\.6\\beta=0\.6\)80\.0074\.5745\.8366\.6761\.4839\.5869\.7774\.4477\.1471\.8872\.3966\.70SKD\(β=0\.8\\beta=0\.8\)83\.3375\.2946\.6680\.0072\.5245\.0090\.7092\.2284\.7681\.2574\.6375\.12SKD\(β=1\.0\\beta=1\.0\)80\.0073\.1436\.2580\.0064\.3333\.7574\.4272\.2271\.4375\.0073\.1366\.70Motab\(ours\)86\.6778\.3347\.0883\.3374\.1447\.5093\.0295\.5688\.5782\.8176\.8777\.62### 4\.1Main Results

Table[1](https://arxiv.org/html/2605.19433#S4.T1)summarizes the results of our proposedMotabagainst various baseline methods\. Overall,Motabconsistently and significantly outperforms all baselines across different teacher models and metrics\. Notably, when utilizing Qwen3\-32B as the teacher,Motabachieves an average accuracy of 66\.64%, yielding a substantial improvement of over 7% compared to the strongest baseline SKD\. This superiority is strictly maintained when scaling up to the gpt\-oss\-120b teacher\. Furthermore, our method exhibits remarkable resilience on highly complex tasks requiring long CoT trajectories\. On challenging benchmarks,*e\.g\.,*AIME24 and AIME25, especially under the strict Pass@1 metric, and the most difficult subsets of MATH500, standard off\-policy methods,*e\.g\.,*SFT and RFT, struggle severely due to error cascades caused by exposure bias\. Similarly, aggressive on\-policy interventions like ImitKD and SKD yield limited gains, as they force the teacher to generate from sub\-optimal contexts, exacerbating reversed exposure bias\. In contrast,Motabdrastically boosts performance by adaptively tolerating sub\-optimal explorations and backtracking to the safe point for intervention\.

Beyond the primary evaluations, we extensively validate the robustness and generalizability of our framework across varied data sources, student architectures, and downstream domains\. As shown in Table[3](https://arxiv.org/html/2605.19433#S4.T3), training on the distinctAceReasondataset yields identically superior trends, confirming that the efficacy ofMotabis independent of specific data distributions\. Moreover, Table[7](https://arxiv.org/html/2605.19433#S4.T7)demonstrates that our method seamlessly transfers to diverse student architectures, consistently elevating the reasoning capabilities of both Qwen2\.5\-7B and Llama3\.2\-3B\. Finally, to assess out\-of\-domain transferability, we evaluate the math\-trained models on instruction following,*i\.e\.,*IFEval, and scientific reasoning,*i\.e\.,*GPQA\. As detailed in Table[7](https://arxiv.org/html/2605.19433#S4.T7),Motabachieves state\-of\-the\-art cross\-domain performance regardless of the teacher model used\. This verifies that our trajectory monitoring and backtracking mechanisms not only prevent mathematical error cascades but also instill a generalized, domain\-agnostic self\-reflection ability that broadly enhances complex, multi\-step problem\-solving\.

Table[5](https://arxiv.org/html/2605.19433#S4.T5)compares the performance ofMotabagainst several on\-policy distillation approaches,*e\.g\.,*top\-kkOPD and DASD\. Since on\-policy methods are highly memory\-intensive and require vocabulary alignment between the teacher and student models, we employ Qwen3\-8B as the teacher and Qwen3\-1\.7B as the student\. The results demonstrate that our method not only significantly outperforms off\-policy baselines but also remains comparable to on\-policy methods\. Furthermore, given that on\-policy methods entail substantially higher time complexity and GPU memory consumption than our approach, these results further validate the effectiveness and efficiency of our method\.

Table 2:Ablation study ofMotab\.MethodAIME24AIME25MATHP@8P@1P@8P@1Motab66\.6734\.1756\.6731\.6785\.60w/o adapt63\.3332\.5050\.0029\.1684\.00w/o back56\.6730\.8350\.0027\.9182\.80w/o stitch63\.3332\.0853\.3329\.5883\.60
Table 3:Results on 10k questions fromAceReason\.MethodAIME24AIME25MATHP@8P@1P@8P@1SFT56\.6725\.4153\.3323\.3381\.00ImitKD63\.3331\.2553\.3328\.3381\.20SKD63\.3332\.9253\.3328\.7582\.60Motab70\.0035\.0056\.6730\.4285\.80

### 4\.2Ablation and Sensitivity Analysis

Ablation study\.To validate our framework’s core components, we conduct an ablation study on the AIME24, AIME25, and MATH benchmarks\. As shown in Table[3](https://arxiv.org/html/2605.19433#S4.T3), removing any module consistently degrades performance\. Crucially, disabling state backtracking,*i\.e\.,*w/o back, incurs the most severe decline,*e\.g\.,*dropping from 34\.17% to 30\.83% on AIME24 Pass@1\. This highlights that forcing immediate teacher intervention without tracing the logical root cause pollutes the context, thereby triggering reversed exposure bias\. Furthermore, omitting sub\-optimal trajectory stitching,*i\.e\.,*w/o stitch, via direct sequence truncation noticeably impairs accuracy\. This confirms that explicitly exposing the student to its own sub\-optimal prefixes is vital for learning self\-correction and mitigating exposure bias\. Finally, substituting the entropy\-based adaptive boundary with a static threshold,*i\.e\.,*w/o adapt, also harms performance, demonstrating that rigid constraints cannot effectively balance student exploration diversity with the validity of teacher supervision\.

Table 4:Results of on\-policy methods\.MethodAIME24AIME25MATHP@8P@1P@8P@1SFT50\.0018\.7560\.0021\.2569\.00RFT13\.335\.8333\.3312\.0864\.20SKD53\.3320\.4156\.6722\.0871\.40top\-kkOPD\[[8](https://arxiv.org/html/2605.19433#bib.bib26)\]60\.0032\.9153\.3330\.4179\.20DASD\[[42](https://arxiv.org/html/2605.19433#bib.bib14)\]53\.3326\.6753\.3328\.7574\.60Motab60\.0033\.7560\.0031\.6779\.00
![Refer to caption](https://arxiv.org/html/2605.19433v1/x4.png)Table 5:Sensitivity analysis of the parameterγ0\\gamma\_\{0\}\.

Sensitivity analysis\.We conduct a sensitivity analysis on the thresholdγ0\\gamma\_\{0\}to evaluate its impact on reasoning performance and intervention frequency\. As illustrated in Fig\.[5](https://arxiv.org/html/2605.19433#S4.T5), increasingγ0\\gamma\_\{0\}inherently enforces a stricter safety boundary, leading to a monotonic rise in the backtracking rate\. However, the reasoning performance on AIME25 Pass@1 exhibits an inverted\-U trend, peaking at a moderate value ofγ0=0\.3\\gamma\_\{0\}=0\.3\. This suggests that an overly lowγ0\\gamma\_\{0\}is too lenient, failing to catch severe logical drifts and thus permitting error cascades\. Conversely, an excessively highγ0\\gamma\_\{0\}becomes overly rigid, constantly interrupting the student’s natural exploration and discarding acceptable sub\-optimal paths, which undermines the mitigation of exposure bias\.

Table 6:Performance on more student models\.MethodAIME24AIME25MATHP@8P@1P@8P@1Qwen2\.5\-7BSFT23\.3310\.4216\.674\.5862\.60ImitKD20\.005\.8326\.675\.8358\.60SKD23\.338\.0624\.447\.3664\.27Motab30\.0010\.4226\.677\.9270\.40Llama3\.2\-3BSFT16\.673\.336\.670\.8344\.00ImitKD16\.672\.926\.671\.6745\.80SKD17\.782\.928\.892\.2246\.47Motab23\.335\.0016\.675\.8352\.60
Table 7:Results on datasets from other domains\.MethodIFEvalGPQAP\.S\.Avg\.P@8P@4P@1Qwen3\-32BSFT29\.9237\.2385\.3568\.3531\.06ImitKD32\.5341\.0284\.8568\.3439\.93SKD35\.8643\.3885\.8673\.9741\.98Motab37\.8946\.8089\.3977\.1846\.97gpt\-oss\-120bSFT32\.1640\.3784\.3472\.6044\.44ImitKD32\.3540\.1576\.2665\.1239\.39SKD34\.5742\.7188\.8979\.1444\.38Motab41\.5950\.0191\.4179\.4250\.51

### 4\.3Evaluating Dual Exposure Biases

To explicitly visualize the mitigation of dual exposure biases, we track the token\-level log probabilities of both the student and teacher models along the trajectories synthesized byMotab\. As illustrated in Fig\.[4](https://arxiv.org/html/2605.19433#S4.F4), our state backtracking mechanism ensures that the teacher’s confidence remains stable, successfully averting the severe probability collapse typically associated with reversed exposure bias\.

![Refer to caption](https://arxiv.org/html/2605.19433v1/x5.png)Figure 4:Evaluation of dual exposure biases\.Under a boundaryγ0=0\.3\\gamma\_\{0\}=0\.3, a distributional gap persists, preserving the student’s exploratory diversity to mitigate exposure bias while maintaining valid teacher supervision\. Remarkably, when the boundary is tightened toγ0=0\.5\\gamma\_\{0\}=0\.5, the probability distributions of the student and teacher align almost perfectly\. This empirical evidence demonstrates thatMotabeffectively harmonizes the student’s authentic generation manifold with the teacher’s valid supervision manifold, fundamentally resolving the distributional discrepancies in dual exposure biases\.

## 5Related Works

In this section, we briefly summarize the literature on LLM reasoning distillation\. Generally, prior reasoning distillation methods follow anoff\-policy paradigm\[[12](https://arxiv.org/html/2605.19433#bib.bib15),[13](https://arxiv.org/html/2605.19433#bib.bib35),[26](https://arxiv.org/html/2605.19433#bib.bib36),[19](https://arxiv.org/html/2605.19433#bib.bib55)\]\. This process involves sampling complex problems\[[45](https://arxiv.org/html/2605.19433#bib.bib25),[27](https://arxiv.org/html/2605.19433#bib.bib17)\], utilizing a teacher model to generate multiple responses via temperature sampling\[[3](https://arxiv.org/html/2605.19433#bib.bib37),[20](https://arxiv.org/html/2605.19433#bib.bib38),[42](https://arxiv.org/html/2605.19433#bib.bib14)\], and conducting rigorous filtering based on criteria,*e\.g\.,*response length\[[39](https://arxiv.org/html/2605.19433#bib.bib39),[6](https://arxiv.org/html/2605.19433#bib.bib24)\], answer correctness\[[5](https://arxiv.org/html/2605.19433#bib.bib40),[21](https://arxiv.org/html/2605.19433#bib.bib18)\], and the student model’s adaptability\[[47](https://arxiv.org/html/2605.19433#bib.bib57),[48](https://arxiv.org/html/2605.19433#bib.bib41),[36](https://arxiv.org/html/2605.19433#bib.bib58)\]\.

However, as analyzed in Sec\.[2](https://arxiv.org/html/2605.19433#S2), early research also identified that such off\-policy methods suffer from exposure bias,*i\.e\.,*train\-test mismatch\[[30](https://arxiv.org/html/2605.19433#bib.bib4),[23](https://arxiv.org/html/2605.19433#bib.bib10)\]\. Theoretically motivated by imitation learning\[[32](https://arxiv.org/html/2605.19433#bib.bib2),[49](https://arxiv.org/html/2605.19433#bib.bib3)\], recent methods proposeon\-policy distillation\[[1](https://arxiv.org/html/2605.19433#bib.bib9),[35](https://arxiv.org/html/2605.19433#bib.bib53)\]\. Unlike previous approaches, this paradigm allows the student model to generate its own reasoning trajectories while receiving dense, real\-time feedback from the teacher model at each token level,*e\.g\.,*ff\-divergence\[[1](https://arxiv.org/html/2605.19433#bib.bib9)\], log probabilities\[[9](https://arxiv.org/html/2605.19433#bib.bib20)\], and the combination of forward and backward KL divergences\[[15](https://arxiv.org/html/2605.19433#bib.bib45)\]\. In parallel, recent reinforcement learning methods have further demonstrated the efficacy of on\-policy data through rollouts by LLM itself\[[34](https://arxiv.org/html/2605.19433#bib.bib6),[4](https://arxiv.org/html/2605.19433#bib.bib46)\]\. In this work, we address the dilemma between off\-policy and on\-policy distillation,*i\.e\.,*dual exposure biases\. The most intuitive approach involves identifying a mixture distribution of the two,*e\.g\.,*through the scheduled sampling\[[2](https://arxiv.org/html/2605.19433#bib.bib1)\]\. Recent speculative distillation strategies\[[41](https://arxiv.org/html/2605.19433#bib.bib11),[16](https://arxiv.org/html/2605.19433#bib.bib12)\]have similarly adopted this rationale; however, our analysis in Sec\.[2\.4](https://arxiv.org/html/2605.19433#S2.SS4)demonstrates that such schemes still fail to satisfy the ideal unbiased condition, which ultimately motivates our proposedMotab\.

## 6Conclusion

In this paper, we identify a dilemma between the off\-policy and on\-policy paradigms in LLM reasoning distillation,*i\.e\.,*dual exposure biases\. We demonstrate that while off\-policy distillation suffers from exposure bias due to training\-inference mismatches, on\-policy distillation inevitably introduces a reversed exposure bias, wherein the teacher model fails to provide reliable supervision when conditioned on straying, student\-generated trajectories\. To resolve this dilemma, we introducedMotab, a novel trajectory monitoring and backtracking framework\. By employing an adaptive safety boundary,Motaballows the student model to explore and learn from its own errors, thereby mitigating exposure bias, while explicitly backtracking and applying teacher intervention before the context degrades irreparably, circumventing reversed exposure bias\. Extensive experiments demonstrate thatMotabenhances reasoning capabilities and mitigates dual exposure biases\.

## Acknowledgements

This work was supported by the National Natural Science Foundation of China \(No\.62276113\) and Alibaba Research Intern Program\.

## References

- \[1\]R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. R\. Garea, M\. Geist, and O\. Bachem\(2024\)On\-policy distillation of language models: learning from self\-generated mistakes\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2605.19433#A4.I4.i1.p1.3),[§1](https://arxiv.org/html/2605.19433#S1.p1.1),[§1](https://arxiv.org/html/2605.19433#S1.p2.1),[§1](https://arxiv.org/html/2605.19433#S1.p3.1),[§2\.2](https://arxiv.org/html/2605.19433#S2.SS2.p3.1),[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[2\]S\. Bengio, O\. Vinyals, N\. Jaitly, and N\. Shazeer\(2015\)Scheduled sampling for sequence prediction with recurrent neural networks\.InAnnual Conference on Neural Information Processing Systems,pp\. 1171–1179\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[3\]H\. Chen, S\. Wu, X\. Quan, R\. Wang, M\. Yan, and J\. Zhang\(2023\)MCC\-KD: multi\-cot consistent knowledge distillation\.InFindings of the Association for Computational Linguistics: EMNLP,pp\. 6805–6820\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[4\]H\. Chen, N\. Razin, K\. Narasimhan, and D\. Chen\(2025\)Retaining by doing: the role of on\-policy data in mitigating forgetting\.CoRRabs/2510\.18874\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[5\]X\. Chen, S\. Zhou, K\. Liang, X\. Sun, and X\. Liu\(2025\)Skip\-thinking: chunk\-wise chain\-of\-thought distillation enable smaller language models to reason better and faster\.InConference on Empirical Methods in Natural Language Processing,pp\. 12142–12157\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[6\]Y\. Chen, Z\. Yang, Z\. Liu, C\. Lee, P\. Xu, M\. Shoeybi, B\. Catanzaro, and W\. Ping\(2025\)AceReason\-nemotron: advancing math and code reasoning through reinforcement learning\.InAnnual Conference on Neural Information Processing Systems,Cited by:[2nd item](https://arxiv.org/html/2605.19433#A4.I1.i2.p1.1),[§1](https://arxiv.org/html/2605.19433#S1.p5.1),[§4](https://arxiv.org/html/2605.19433#S4.p1.1),[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[7\]D\. Cheng, S\. Huang, X\. Zhu, B\. Dai, X\. Zhao, Z\. Zhang, and F\. Wei\(2026\)Reasoning with exploration: an entropy perspective\.InAAAI Conference on Artificial Intelligence,pp\. 30377–30385\.Cited by:[§3\.1](https://arxiv.org/html/2605.19433#S3.SS1.p1.11)\.
- \[8\]Y\. Fu, H\. Huang, K\. Jiang, Y\. Zhu, and D\. Zhao\(2026\)Revisiting on\-policy distillation: empirical failure modes and simple fixes\.CoRRabs/2603\.25562\.Cited by:[1st item](https://arxiv.org/html/2605.19433#A4.I4.i1.p1.3),[§D\.5](https://arxiv.org/html/2605.19433#A4.SS5.p3.3),[§1](https://arxiv.org/html/2605.19433#S1.p3.1),[Table 5](https://arxiv.org/html/2605.19433#S4.T5.1.1.1.1.1.1)\.
- \[9\]Y\. Gu, L\. Dong, F\. Wei, and M\. Huang\(2024\)MiniLLM: knowledge distillation of large language models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p3.1),[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[10\]E\. K\. Guha, R\. Marten, S\. Keh, N\. Raoof, G\. Smyrnis, H\. Bansal, M\. Nezhurina, J\. Mercat, T\. Vu, Z\. Sprague,et al\.\(2025\)OpenThoughts: data recipes for reasoning models\.InFirst Workshop on Foundations of Reasoning in Language Models,Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p2.1)\.
- \[11\]D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p1.1)\.
- \[12\]N\. Ho, L\. Schmid, and S\. Yun\(2023\)Large language models are reasoning teachers\.InAnnual Meeting of the Association for Computational Linguistics,pp\. 14852–14882\.Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p2.1),[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[13\]C\. Hsieh, C\. Li, C\. Yeh, H\. Nakhost, Y\. Fujii, A\. Ratner, R\. Krishna, C\. Lee, and T\. Pfister\(2023\)Distilling step\-by\-step\! outperforming larger language models with less training data and smaller model sizes\.InFindings of the Association for Computational Linguistics: ACL,pp\. 8003–8017\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[14\]J\. Jung, S\. Han, X\. Lu, S\. Hallinan, D\. Acuna, S\. Prabhumoye, M\. Patwary, M\. Shoeybi, B\. Catanzaro, and Y\. Choi\(2025\)Prismatic synthesis: gradient\-based data diversification boosts generalization in llm reasoning\.InAnnual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p2.1)\.
- \[15\]S\. Jung, S\. Yoon, D\. Kim, and H\. Lee\(2025\)ToDi: token\-wise distillation via fine\-grained divergence control\.InConference on Empirical Methods in Natural Language Processing,pp\. 8078–8091\.Cited by:[§A\.3](https://arxiv.org/html/2605.19433#A1.SS3.p1.2),[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[16\]J\. Kim, K\. Seo, and D\. Lee\(2025\)In their own words: reasoning traces tailored for small models make them better reasoners\.CoRRabs/2509\.22230\.Cited by:[§2\.4](https://arxiv.org/html/2605.19433#S2.SS4.p1.2),[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[17\]J\. Ko, T\. Chen, S\. Kim, T\. Ding, L\. Liang, I\. Zharkov, and S\. Yun\(2025\)DistiLLM\-2: A contrastive approach boosts the distillation of llms\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p3.1)\.
- \[18\]J\. Ko, S\. Kim, T\. Chen, and S\. Yun\(2024\)DistiLLM: towards streamlined distillation for large language models\.InInternational Conference on Machine Learning,pp\. 24872–24895\.Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p3.1)\.
- \[19\]Z\. Kou, J\. Chen, X\. Cai, X\. Xia, M\. Xie, D\. Wu, B\. Liu, Y\. Jia, X\. Geng, M\. Sugiyama, and T\. Chua\(2026\)Positive\-unlabeled reinforcement learning distillation for on\-premise small models\.CoRRabs/2601\.20687\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[20\]Z\. Lei, Z\. Tan, S\. Wang, Y\. Zhu, Z\. Chen, Y\. Dong, and J\. Li\(2025\)Learning from diverse reasoning paths with routing and collaboration\.InConference on Empirical Methods in Natural Language Processing,pp\. 2832–2845\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[21\]Y\. Li, Y\. Emad, K\. Padthe, J\. Lanchantin, W\. Yuan, T\. Nguyen, J\. E\. Weston, S\. Li, D\. Wang, I\. Kulikov,et al\.\(2025\)NaturalThoughts: selecting and distilling reasoning traces for general reasoning tasks\.InNeurIPS Workshop on Efficient Reasoning,Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p2.1),[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[22\]H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe\(2024\)Let’s verify step by step\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2605.19433#A4.I2.i1.p1.3),[§4](https://arxiv.org/html/2605.19433#S4.p1.1)\.
- \[23\]A\. Lin, J\. Wohlwend, H\. Chen, and T\. Lei\(2020\)Autoregressive knowledge distillation through imitation learning\.InConference on Empirical Methods in Natural Language Processing,pp\. 6121–6133\.Cited by:[3rd item](https://arxiv.org/html/2605.19433#A4.I3.i3.p1.1),[§4](https://arxiv.org/html/2605.19433#S4.p1.1),[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[24\]K\. Liu, S\. Yan, R\. Miao, B\. Wang, C\. Shen, J\. Zhang, and J\. Ye\(2026\)Where did this sentence come from? tracing provenance in LLM reasoning distillation\.InInternational Conference on Learning Representations,Cited by:[§3\.1](https://arxiv.org/html/2605.19433#S3.SS1.p1.6)\.
- \[25\]R\. Lowe, Y\. Wu, A\. Tamar, J\. Harb, P\. Abbeel, and I\. Mordatch\(2017\)Multi\-agent actor\-critic for mixed cooperative\-competitive environments\.InAnnual Conference on Neural Information Processing Systems,pp\. 6379–6390\.Cited by:[§3\.2](https://arxiv.org/html/2605.19433#S3.SS2.p1.6)\.
- \[26\]L\. C\. Magister, J\. Mallinson, J\. Adámek, E\. Malmi, and A\. Severyn\(2023\)Teaching small language models to reason\.InAnnual Meeting of the Association for Computational Linguistics,pp\. 1773–1781\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[27\]N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. J\. Candès, and T\. Hashimoto\(2025\)S1: simple test\-time scaling\.InConference on Empirical Methods in Natural Language Processing,pp\. 20275–20321\.Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p2.1),[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[28\]OpenAI\(2025\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.CoRRabs/2508\.10925\.Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p5.1)\.
- \[29\]E\. Pignatelli, J\. Ferret, M\. Geist, T\. Mesnard, H\. van Hasselt, and L\. Toni\(2024\)A survey of temporal credit assignment in deep reinforcement learning\.Transactions on Machine Learning Research2024\.Cited by:[§3\.2](https://arxiv.org/html/2605.19433#S3.SS2.p1.6)\.
- \[30\]M\. Ranzato, S\. Chopra, M\. Auli, and W\. Zaremba\(2016\)Sequence level training with recurrent neural networks\.InInternational Conference on Learning Representations,Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[31\]D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman\(2023\)GPQA: A graduate\-level google\-proof q&a benchmark\.CoRRabs/2311\.12022\.Cited by:[2nd item](https://arxiv.org/html/2605.19433#A4.I2.i2.p1.1),[§4](https://arxiv.org/html/2605.19433#S4.p1.1)\.
- \[32\]S\. Ross, G\. J\. Gordon, and D\. Bagnell\(2011\)A reduction of imitation learning and structured prediction to no\-regret online learning\.InInternational Conference on Artificial Intelligence and Statistics,pp\. 627–635\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[33\]J\. Schulman, P\. Moritz, S\. Levine, M\. I\. Jordan, and P\. Abbeel\(2016\)High\-dimensional continuous control using generalized advantage estimation\.InInternational Conference on Learning Representations,Cited by:[§3\.2](https://arxiv.org/html/2605.19433#S3.SS2.p1.6),[§3](https://arxiv.org/html/2605.19433#S3.p3.1)\.
- \[34\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p1.1),[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[35\]M\. Song and M\. Zheng\(2026\)A survey of on\-policy distillation for large language models\.CoRRabs/2604\.00626\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[36\]B\. Wang, R\. Miao, C\. Shen, S\. Yan, K\. Liu, X\. Li, X\. Yuan, S\. Fan, J\. Zhang, and J\. Ye\(2026\)On the step length confounding in LLM reasoning data selection\.CoRRabs/2604\.06834\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[37\]S\. Wang, L\. Yu, C\. Gao, C\. Zheng, S\. Liu, R\. Lu, K\. Dang, X\. Chen, J\. Yang, Z\. Zhang,et al\.\(2025\)Beyond the 80/20 rule: high\-entropy minority tokens drive effective reinforcement learning for llm reasoning\.InAnnual Conference on Neural Information Processing Systems,Cited by:[§3\.1](https://arxiv.org/html/2605.19433#S3.SS1.p1.11),[§3](https://arxiv.org/html/2605.19433#S3.p3.1)\.
- \[38\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAnnual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p1.1)\.
- \[39\]X\. Wu, X\. Jiang, H\. Li, J\. Zhai, D\. Liu, Q\. Hao, H\. Liu, Z\. Yang, J\. Xie, N\. Gu, J\. Yang, K\. Zhang, Y\. Bao, and J\. Wang\(2025\)Beyond scaling law: A data\-efficient distillation framework for reasoning\.CoRRabs/2508\.09883\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[40\]Z\. Xi, C\. Liao, G\. Li, Z\. Zhang, W\. Chen, B\. Wang, S\. Jin, Y\. Zhou, J\. Guan, W\. Wu, T\. Ji, T\. Gui, Q\. Zhang, and X\. Huang\(2026\)AgentPRM: process reward models for LLM agents via step\-wise promise and progress\.InACM Web Conference,pp\. 4184–4195\.Cited by:[§3\.2](https://arxiv.org/html/2605.19433#S3.SS2.p1.6),[§3](https://arxiv.org/html/2605.19433#S3.p3.1)\.
- \[41\]W\. Xu, R\. Han, Z\. Wang, L\. T\. Le, D\. Madeka, L\. Li, W\. Y\. Wang, R\. Agarwal, C\. Lee, and T\. Pfister\(2025\)Speculative knowledge distillation: bridging the teacher\-student gap through interleaved sampling\.InInternational Conference on Learning Representations,Cited by:[§A\.3](https://arxiv.org/html/2605.19433#A1.SS3.p1.2),[4th item](https://arxiv.org/html/2605.19433#A4.I3.i4.p1.2),[§1](https://arxiv.org/html/2605.19433#S1.p2.1),[§2\.4](https://arxiv.org/html/2605.19433#S2.SS4.p1.2),[§3\.1](https://arxiv.org/html/2605.19433#S3.SS1.p1.6),[§4](https://arxiv.org/html/2605.19433#S4.p1.1),[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[42\]S\. Yan, K\. Liu, C\. Shen, B\. Wang, S\. Fan, J\. Zhang, Y\. Wu, Z\. Wang, and J\. Ye\(2026\)Distribution\-aligned sequence distillation for superior long\-cot reasoning\.CoRRabs/2601\.09088\.Cited by:[§A\.2](https://arxiv.org/html/2605.19433#A1.SS2.p4.1),[2nd item](https://arxiv.org/html/2605.19433#A4.I4.i2.p1.1),[§1](https://arxiv.org/html/2605.19433#S1.p1.1),[§1](https://arxiv.org/html/2605.19433#S1.p2.1),[Table 5](https://arxiv.org/html/2605.19433#S4.T5.1.1.7.1.1.1),[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[43\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.CoRRabs/2505\.09388\.Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p1.1),[§1](https://arxiv.org/html/2605.19433#S1.p5.1)\.
- \[44\]Y\. Yang, Y\. He, J\. Liu, and Z\. Jin\(2026\)Making complex reasoning student\-friendly: a hybrid LLM\-to\-SLM distillation framework\.InThe 1st Workshop on Scaling Post\-training for LLMs,Cited by:[§3\.1](https://arxiv.org/html/2605.19433#S3.SS1.p1.6)\.
- \[45\]Y\. Ye, Z\. Huang, Y\. Xiao, E\. Chern, S\. Xia, and P\. Liu\(2025\)LIMO: less is more for reasoning\.InConference on Language Modeling,Cited by:[1st item](https://arxiv.org/html/2605.19433#A4.I1.i1.p1.1),[§1](https://arxiv.org/html/2605.19433#S1.p5.1),[§4](https://arxiv.org/html/2605.19433#S4.p1.1),[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[46\]X\. Yuan, C\. Shen, S\. Yan, K\. Liu, X\. Zhang, S\. Fan, L\. Xie, W\. Wang, R\. Guan, Y\. Wang, and J\. Ye\(2026\)Differential fine\-tuning large language models towards better diverse reasoning abilities\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.19433#S1.p1.1)\.
- \[47\]X\. Yuan, C\. Shen, S\. Yan, X\. Zhang, L\. Xie, W\. Wang, R\. Guan, Y\. Wang, and J\. Ye\(2024\)Instance\-adaptive zero\-shot chain\-of\-thought prompting\.InAnnual Conference on Neural Information Processing Systems,Vol\.37,pp\. 125469–125486\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[48\]D\. Zhang, Q\. Dai, and H\. Peng\(2025\)The best instruction\-tuning data are those that fit\.InAnnual Conference on Neural Information Processing Systems,Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p1.1)\.
- \[49\]J\. Zhang and K\. Cho\(2017\)Query\-efficient imitation learning for end\-to\-end simulated driving\.InAAAI Conference on Artificial Intelligence,pp\. 2891–2897\.Cited by:[§5](https://arxiv.org/html/2605.19433#S5.p2.1)\.
- \[50\]J\. Zhou, T\. Lu, S\. Mishra, S\. Brahma, S\. Basu, Y\. Luan, D\. Zhou, and L\. Hou\(2023\)Instruction\-following evaluation for large language models\.CoRRabs/2311\.07911\.Cited by:[2nd item](https://arxiv.org/html/2605.19433#A4.I2.i2.p1.1),[§4](https://arxiv.org/html/2605.19433#S4.p1.1)\.

## Appendix ATheoretical Analysis

In this section, we provide a formal and mathematical derivation of the dual exposure biases in LLM reasoning distillation, and prove how our proposedMotabframework theoretically resolves them\. Specifically, we answer the following three basic questions:

RQ1\.Why does off\-policy distillation fail during inference?

RQ2\.Why does on\-policy distillation corrupt the teacher’s supervision?

RQ3\.Why does a mixture of off\-policy and on\-policy distributions fail to resolve this dilemma?

RQ4\.How doesMotabformally guarantee distribution coverage and supervision validity?

LLM reasoning is a strict full\-history\-dependent auto\-regressive process\. Formally, let𝒱\\mathcal\{V\}denote the vocabulary space\. For an initial question𝐪\\mathbf\{q\}, a reasoning trajectory of lengthLLis a sequence of reasoning steps𝒴=\(𝐲1,𝐲2,…,𝐲L\)\\mathcal\{Y\}=\(\\mathbf\{y\}\_\{1\},\\mathbf\{y\}\_\{2\},\\dots,\\mathbf\{y\}\_\{L\}\), and each step is a sequence of tokens𝐲l=\(yl,1,yl,1,…,yl,\|𝐲l\|\)\\mathbf\{y\}\_\{l\}=\(y\_\{l,1\},y\_\{l,1\},\\ldots,y\_\{l,\|\\mathbf\{y\}\_\{l\}\|\}\)\. At any stepll, the context is the entire historical prefix𝐬t−1=\[𝐪,𝐲1,𝐲2,…,𝐲l−1\]\\mathbf\{s\}\_\{t\-1\}=\[\\mathbf\{q\},\\mathbf\{y\}\_\{1\},\\mathbf\{y\}\_\{2\},\\ldots,\\mathbf\{y\}\_\{l\-1\}\]\. A policy LLM𝝅\\boldsymbol\{\\pi\}defines a conditional probability distribution over the next token𝝅​\(yl,i∣𝐬t−1,𝐲l,<i\)\\boldsymbol\{\\pi\}\(y\_\{l,i\}\\mid\\mathbf\{s\}\_\{t\-1\},\\mathbf\{y\}\_\{l,<i\}\)\.

The probability of a reasoning trajectory under policy𝝅\\boldsymbol\{\\pi\}is given by the chain rule of probability:

P𝝅​\(𝒴∣𝐪\)=∏l=1L∏i=1\|𝐲l\|𝝅​\(yl,i∣𝐬l−1,𝐲l,<i\)\.P\_\{\\boldsymbol\{\\pi\}\}\(\\mathcal\{Y\}\\mid\\mathbf\{q\}\)=\\prod\\nolimits\_\{l=1\}^\{L\}\\prod\\nolimits\_\{i=1\}^\{\|\\mathbf\{y\}\_\{l\}\|\}\\boldsymbol\{\\pi\}\(y\_\{l,i\}\\mid\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l,<i\}\)\.Letd𝝅d^\{\\boldsymbol\{\\pi\}\}denote the marginal distribution of all possible historical prefixes𝐬\\mathbf\{s\}generated by iteratively unrolling policy𝝅\\boldsymbol\{\\pi\}\.

### A\.1RQ1: Exposure Bias in Off\-Policy Distillation

Off\-policy distillation minimizes the single\-step Kullback\-Leibler divergence exclusively over the teacher’s historical prefixes\. We assume the student model𝝅S\\boldsymbol\{\\pi\}\_\{S\}is optimized such that its expected single\-step error on the teacher’s manifoldpTp\_\{T\}is bounded byϵ\\epsilon:

𝔼𝐬l−1∼pT\[DTV\(𝝅T\(⋅∣𝐬l−1\)∥𝝅S\(⋅∣𝐬l−1\)\)\]≤ϵ,∀l∈\[1,L\]\.\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{T\}\}\\left\[D\_\{\\mathrm\{TV\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)\\right\]\\leq\\epsilon,\\quad\\forall l\\in\[1,L\]\.\(12\)
Theorem 1\.Under the training condition in Eq\. \([12](https://arxiv.org/html/2605.19433#A1.E12)\), during free\-form inference where prefixes are generated by the studentpSp\_\{S\}, the sequence\-level divergenceDTV​\(pT​\(𝐬T\)∥pS​\(𝐬T\)\)D\_\{\\mathrm\{TV\}\}\\big\(p\_\{T\}\(\\mathbf\{s\}\_\{T\}\)\\parallel p\_\{S\}\(\\mathbf\{s\}\_\{T\}\)\\big\)scales at an𝒪​\(L2\)\\mathcal\{O\}\(L^\{2\}\)rate in the worst case, demonstrating the exposure bias\.

Proof\.Using the chain rule of probability, the prefix distributions can be recursively decomposed:p​\(𝐬l\)=p​\(𝐬l−1\)​𝝅​\(𝐲l∣𝐬l−1\)p\(\\mathbf\{s\}\_\{l\}\)=p\(\\mathbf\{s\}\_\{l\-1\}\)\\boldsymbol\{\\pi\}\(\\mathbf\{y\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\), where𝝅​\(𝐲l∣𝐬l−1\)=∏i=1\|𝐲l\|𝝅​\(yl,i∣𝐬l−1,𝐲l,<i\)\\boldsymbol\{\\pi\}\(\\mathbf\{y\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\)=\\prod\_\{i=1\}^\{\|\\mathbf\{y\}\_\{l\}\|\}\\boldsymbol\{\\pi\}\(y\_\{l,i\}\\mid\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l,<i\}\)\. By the telescoping property of the total variation distance for sequence distributions, the trajectory\-level divergence at the final stepLLcan be expanded as the sum of step\-level conditional divergences evaluated over thestudent’sprefix distribution:

DTV\(pT\(𝐬L\)∥pS\(𝐬L\)\)≤∑l=1L𝔼𝐬l−1∼pS\[DTV\(𝝅T\(⋅∣𝐬l−1\)∥𝝅S\(⋅∣𝐬l−1\)\)\]=∑l=1LℒFR\(l\)\.D\_\{\\mathrm\{TV\}\}\\big\(p\_\{T\}\(\\mathbf\{s\}\_\{L\}\)\\parallel p\_\{S\}\(\\mathbf\{s\}\_\{L\}\)\\big\)\\leq\\sum\\nolimits\_\{l=1\}^\{L\}\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\}\}\\left\[D\_\{\\mathrm\{TV\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)\\right\]\\\\ =\\sum\\nolimits\_\{l=1\}^\{L\}\\mathcal\{L\}\_\{\\mathrm\{FR\}\}^\{\(l\)\}\.\(13\)Here,ℒFR\(l\)\\mathcal\{L\}\_\{\\mathrm\{FR\}\}^\{\(l\)\}denotes the free\-running expected divergence\. Notice thatℒFR\(l\)\\mathcal\{L\}\_\{\\mathrm\{FR\}\}^\{\(l\)\}is evaluated over prefixes𝐬l−1∼pS\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\}, while the training objective in Eq\. \([12](https://arxiv.org/html/2605.19433#A1.E12)\) assumes𝐬l−1∼pT\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{T\}\. To formally relate them, we apply a change\-of\-measure expansion toℒFR\(l\)\\mathcal\{L\}\_\{\\mathrm\{FR\}\}^\{\(l\)\}:

ℒFR\(l\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{FR\}\}^\{\(l\)\}=∑𝐬l−1pS\(𝐬l−1\)DTV\(𝝅T\(⋅∣𝐬l−1\)∥𝝅S\(⋅∣𝐬l−1\)\)\\displaystyle=\\sum\\nolimits\_\{\\mathbf\{s\}\_\{l\-1\}\}p\_\{S\}\(\\mathbf\{s\}\_\{l\-1\}\)D\_\{\\mathrm\{TV\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)\(14\)=𝔼𝐬l−1∼pT\[pS​\(𝐬l−1\)pT​\(𝐬l−1\)DTV\(𝝅T\(⋅∣𝐬l−1\)∥𝝅S\(⋅∣𝐬l−1\)\)\]\.\\displaystyle=\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{T\}\}\\left\[\\frac\{p\_\{S\}\(\\mathbf\{s\}\_\{l\-1\}\)\}\{p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\}D\_\{\\mathrm\{TV\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)\\right\]\.Because generation is strictly history\-dependent, if the student makes an early mistake at stepk<lk<l, the subsequent prefix𝐬l−1\\mathbf\{s\}\_\{l\-1\}drifts out of the teacher’s data manifold\. Mathematically, this impliespT​\(𝐬l−1\)→0p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\\to 0, causing the density ratiopS​\(𝐬l−1\)pT​\(𝐬l−1\)→∞\\frac\{p\_\{S\}\(\\mathbf\{s\}\_\{l\-1\}\)\}\{p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\}\\to\\infty\. Consequently, for stepll, the expected divergenceℒFR\(l\)\\mathcal\{L\}\_\{\\mathrm\{FR\}\}^\{\(l\)\}is no longer bounded byϵ\\epsilon\. Instead, in the worst\-case out\-of\-distribution scenario, the divergence reaches its theoretical maximumDTV→1D\_\{\\mathrm\{TV\}\}\\to 1\. Substituting this cascading error into Eq\. \([13](https://arxiv.org/html/2605.19433#A1.E13)\) yields an expected compounding sequence error that grows quadratically𝒪​\(L2​ϵ\)\\mathcal\{O\}\(L^\{2\}\\epsilon\), effectively proving theexposure bias\.□\\square

### A\.2RQ2: Reversed Exposure Bias in On\-Policy Distillation

To prevent the density ratio explosion seen in Eq\. \([14](https://arxiv.org/html/2605.19433#A1.E14)\), on\-policy distillation enforces the student to learn directly from its own rollout distribution:𝔼𝐬l−1∼pS​\[DKL​\(𝝅S∥𝝅T\)\]\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\}\}\\big\[D\_\{\\mathrm\{KL\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\\parallel\\boldsymbol\{\\pi\}\_\{T\}\)\\big\]\.

Theorem 2\.Let𝐬l−1∼pS\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\}be a sub\-optimal reasoning history such that it deviates fundamentally from logical coherence\. Conditioning the teacher on this state causes its predictive distribution over the next step to collapse to a uniform prior, maximizing entropy\. Consequently, the teacher merely provides a suppressive signal rather than constructive guidance, destroying valid supervision\.

Proof\.By Bayes’ theorem, the teacher’s conditional probability for the next reasoning step𝐲l\\mathbf\{y\}\_\{l\}given the history𝐬l−1\\mathbf\{s\}\_\{l\-1\}is defined by the joint probability of the sequence:

𝝅T​\(𝐲l∣𝐬l−1\)=pT​\(𝐬l−1,𝐲l\)pT​\(𝐬l−1\)\.\\boldsymbol\{\\pi\}\_\{T\}\(\\mathbf\{y\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\)=\\frac\{p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}\)\}\{p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\}\.\(15\)In practice, LLM distributions are smoothed by a prior to prevent absolute zero probabilities\. Over a reasoning step of lengthK=\|𝐲l\|K=\|\\mathbf\{y\}\_\{l\}\|, the uniform prior isU​\(𝐲l\)=1\|𝒱\|KU\(\\mathbf\{y\}\_\{l\}\)=\\frac\{1\}\{\|\\mathcal\{V\}\|^\{K\}\}\. We approximate the implicit generative distribution as a mixture with a smoothing factorλ∈\(0,1\)\\lambda\\in\(0,1\):

p~T​\(𝐬l−1,𝐲l\)=\(1−λ\)​pT​\(𝐬l−1,𝐲l\)\+λ​pT​\(𝐬l−1\)\|𝒱\|K\.\\tilde\{p\}\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}\)=\(1\-\\lambda\)p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}\)\+\\lambda\\frac\{p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\}\{\|\\mathcal\{V\}\|^\{K\}\}\.When𝐬l−1\\mathbf\{s\}\_\{l\-1\}is a sub\-optimal prefix generated by the student, the true logical continuation densitypT​\(𝐬l−1,𝐲l\)→0p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}\)\\to 0\. Substituting this into the conditional distribution yields:

𝝅T​\(𝐲l∣𝐬l−1\)≈\(1−λ\)⋅0\+λ​pT​\(𝐬l−1\)\|𝒱\|KpT​\(𝐬l−1\)=λ\|𝒱\|K∝1\|𝒱\|K\.\\boldsymbol\{\\pi\}\_\{T\}\(\\mathbf\{y\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\\approx\\frac\{\(1\-\\lambda\)\\cdot 0\+\\lambda\\frac\{p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\}\{\|\\mathcal\{V\}\|^\{K\}\}\}\{p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\}\\\\ =\\frac\{\\lambda\}\{\|\\mathcal\{V\}\|^\{K\}\}\\propto\\frac\{1\}\{\|\\mathcal\{V\}\|^\{K\}\}\.This indicates that in the sub\-optimal regime, the teacher lacks valid priors, and its conditional distribution degenerates toward a uniform distribution over the sequence space\. The conditional entropy of the teacher for stepllthen diverges to:

ℋ​\(𝝅T∣𝐬l−1\)=−∑𝐲l𝝅T​\(𝐲l∣𝐬l−1\)​log⁡𝝅T​\(𝐲l∣𝐬l−1\)⟶K​log⁡\|𝒱\|\.\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\)=\-\\sum\\nolimits\_\{\\mathbf\{y\}\_\{l\}\}\\boldsymbol\{\\pi\}\_\{T\}\(\\mathbf\{y\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\\log\\boldsymbol\{\\pi\}\_\{T\}\(\\mathbf\{y\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\\longrightarrow K\\log\|\\mathcal\{V\}\|\.MinimizingDKL​\(𝝅S∥𝝅T\)D\_\{\\mathrm\{KL\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\\parallel\\boldsymbol\{\\pi\}\_\{T\}\)against this maximum\-entropy distribution implies that the teacher assigns uniformly low confidence to all possible next steps\. Rather than providing active, constructive guidance \(i\.e\., a sharp distribution peaking at the correct logical deduction\), the teacher acts purely as a penalty mechanism, suppressing the student’s generation across the board\. This forces the student model to unlearn deterministic logical reasoning, formally defining thereversed exposure bias\.□\\square

Beyond the theoretical reversed exposure bias, relying on strict on\-policy supervision introduces severe practical and computational bottlenecks\. First, calculating the token\-level KL divergence fundamentally requires the teacher and student models to share the exact same tokenizer vocabulary𝒱\\mathcal\{V\}\[Yanet al\.,[2026](https://arxiv.org/html/2605.19433#bib.bib14)\]\. This constraint severely restricts the flexibility of cross\-architecture distillation\. Second, on\-policy distillation forces the massive teacher model to synchronously evaluate or generate target distributions conditioned on the student’s real\-time rollouts\. This step\-by\-step dependency introduces prohibitive computational latency and memory overhead, rendering large\-scale on\-policy distillation highly inefficient in practice\.

### A\.3RQ3: The Failure of Naive Distribution Mixing

Given the complementary flaws of off\-policy and on\-policy distillation, an intuitive baseline would be to optimize the student over a mixture distribution,*e\.g\.,*SKD\[Xuet al\.,[2025](https://arxiv.org/html/2605.19433#bib.bib11)\], or combine the forward and reversed KL divergences\[Junget al\.,[2025b](https://arxiv.org/html/2605.19433#bib.bib45)\]\. Letpmix=α​pS\+\(1−α\)​pTp\_\{\\text\{mix\}\}=\\alpha p\_\{S\}\+\(1\-\\alpha\)p\_\{T\}be a linearly mixed historical context distribution with a mixture coefficientα∈\(0,1\)\\alpha\\in\(0,1\)\. The expected distillation objective becomes:

𝒥mix\(𝝅S\)=𝔼𝐬l−1∼pmix\[DKL\(𝝅T\(⋅∣𝐬l−1\)∥𝝅S\(⋅∣𝐬l−1\)\)\]\.\\mathcal\{J\}\_\{\\text\{mix\}\}\(\\boldsymbol\{\\pi\}\_\{S\}\)=\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{\\text\{mix\}\}\}\\left\[D\_\{\\mathrm\{KL\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)\\right\]\.
Theorem 3 \(zero\-sum trade\-off\)\.Optimizing over a naive mixture distributionpmixp\_\{\\text\{mix\}\}cannot concurrently resolve the dual exposure biases\. The expected inference error is strictly lower\-bounded by a convex combination of the forward error cascade \(Theorem 1\) and the reversed entropy collapse \(Theorem 2\)\. Consequently, any choice ofα\\alphamerely trades one form of bias for another\.

Proof\.By expanding the definition of the expectation over the discrete state space, we substitute the definition of the mixed densitypmix​\(𝐬l−1\)=α​pS​\(𝐬l−1\)\+\(1−α\)​pT​\(𝐬l−1\)p\_\{\\text\{mix\}\}\(\\mathbf\{s\}\_\{l\-1\}\)=\\alpha p\_\{S\}\(\\mathbf\{s\}\_\{l\-1\}\)\+\(1\-\\alpha\)p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)into the objective:

𝒥mix\\displaystyle\\mathcal\{J\}\_\{\\text\{mix\}\}=∑𝐬l−1pmix\(𝐬l−1\)DKL\(𝝅T\(⋅∣𝐬l−1\)∥𝝅S\(⋅∣𝐬l−1\)\)\\displaystyle=\\sum\\nolimits\_\{\\mathbf\{s\}\_\{l\-1\}\}p\_\{\\text\{mix\}\}\(\\mathbf\{s\}\_\{l\-1\}\)D\_\{\\mathrm\{KL\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)=∑𝐬l−1\[αpS\(𝐬l−1\)\+\(1−α\)pT\(𝐬l−1\)\]DKL\(𝝅T\(⋅∣𝐬l−1\)∥𝝅S\(⋅∣𝐬l−1\)\)\.\\displaystyle=\\sum\\nolimits\_\{\\mathbf\{s\}\_\{l\-1\}\}\\big\[\\alpha p\_\{S\}\(\\mathbf\{s\}\_\{l\-1\}\)\+\(1\-\\alpha\)p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\\big\]D\_\{\\mathrm\{KL\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\parallel\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)\.By distributing the summation, the mixture objective𝒥mix\\mathcal\{J\}\_\{\\text\{mix\}\}is explicitly decoupled into two independent expectations:

𝒥mix=α​𝔼𝐬l−1∼pS​\[DKL​\(𝝅T∥𝝅S\)\]⏟On\-policy constraint\+\(1−α\)​𝔼𝐬l−1∼pT​\[DKL​\(𝝅T∥𝝅S\)\]⏟Off\-policy constraint\.\\mathcal\{J\}\_\{\\text\{mix\}\}=\\alpha\\underbrace\{\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\}\}\\left\[D\_\{\\mathrm\{KL\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\\parallel\\boldsymbol\{\\pi\}\_\{S\}\\big\)\\right\]\}\_\{\\text\{On\-policy constraint\}\}\+\(1\-\\alpha\)\\underbrace\{\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{T\}\}\\left\[D\_\{\\mathrm\{KL\}\}\\big\(\\boldsymbol\{\\pi\}\_\{T\}\\parallel\\boldsymbol\{\\pi\}\_\{S\}\\big\)\\right\]\}\_\{\\text\{Off\-policy constraint\}\}\.This exact decoupling reveals a structural contradiction between the two terms\.

First, consider theα\\alphacomponent\. For historical prefixes sampled directly from the student’s rollout \(𝐬l−1∼pS\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\}\), the trajectories inevitably contain out\-of\-distribution trajectories wherepT​\(𝐬l−1\)→0p\_\{T\}\(\\mathbf\{s\}\_\{l\-1\}\)\\to 0\. As rigorously established in Theorem 2, the teacher’s conditional entropy diverges\. Thus, minimizing the KL divergence forces the student to fit a maximum\-entropy distribution, inflicting a supervision penalty proportional toℋ​\(𝝅T∣𝐬l−1\)→K​log⁡\|𝒱\|\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\\to K\\log\|\\mathcal\{V\}\|\.

Second, consider the\(1−α\)\(1\-\\alpha\)component\. To avoid the aforementioned entropy collapse, one must heavily weight the off\-policy data by settingα→0\\alpha\\to 0\. However, as proven in Theorem 1, optimizing exclusively overpTp\_\{T\}leaves the student vulnerable to compounding sequence\-level errors during test\-time free\-running generation\. This inference error cascade is mathematically bounded by the trajectory\-level divergenceDTV​\(pT​\(𝐬L\)∥pS​\(𝐬L\)\)≥Ω​\(L2​ϵ\)D\_\{\\mathrm\{TV\}\}\\big\(p\_\{T\}\(\\mathbf\{s\}\_\{L\}\)\\parallel p\_\{S\}\(\\mathbf\{s\}\_\{L\}\)\\big\)\\geq\\Omega\(L^\{2\}\\epsilon\)\.

Formally, we define the total sub\-optimality risk functionalℛtotal\\mathcal\{R\}\_\{\\text\{total\}\}as the weighted sum of the expected training supervision corruption and the expected inference sequence deviation\. The risk is strictly bounded from below:

ℛtotal≥α⋅𝔼𝐬l−1∼pS​\[ℋ​\(𝝅T∣𝐬l−1\)\]\+\(1−α\)⋅DTV​\(pT​\(𝐬L\)∥pS​\(𝐬L\)\)\.\\mathcal\{R\}\_\{\\text\{total\}\}\\geq\\alpha\\cdot\\mathbb\{E\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\}\}\\big\[\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\]\+\(1\-\\alpha\)\\cdot D\_\{\\mathrm\{TV\}\}\\big\(p\_\{T\}\(\\mathbf\{s\}\_\{L\}\)\\parallel p\_\{S\}\(\\mathbf\{s\}\_\{L\}\)\\big\)\.Because the two penalty terms are mutually antagonistic, loweringα\\alphato secure valid supervision inherently amplifies the sequential divergence, and raisingα\\alphato cover the student’s distribution actively corrupts the reasoning targets\. Thus, the structural deadlock remains unbroken\. The theoretical oracle requires valid supervisionconditioned onsub\-optimal contexts, a property that naive distribution mixing mathematically fails to provide\.□\\square

### A\.4RQ4: Theoretical Guarantees ofMotab

We formally prove howMotabsatisfies both thecoverageandvalidityconditions\. Letp𝝅Sγp\_\{\\boldsymbol\{\\pi\}\_\{S\}\}^\{\\gamma\}denote the truncated distribution of prefixes generated under theMotabadaptive boundary mechanism\.

Theorem 4 \(coverage\)\.The truncated distributionp𝛑Sγp\_\{\\boldsymbol\{\\pi\}\_\{S\}\}^\{\\gamma\}strictly covers the authentic student distributionpSp\_\{S\}up to a tightly bounded divergence, eliminating the exposure bias without exposing the teacher to fatal logical collapses\.

Proof\.The synthesized distributionp𝝅Sγp\_\{\\boldsymbol\{\\pi\}\_\{S\}\}^\{\\gamma\}matchespSp\_\{S\}until a truncation occurs, triggered whenVT​\(𝐬l−1,𝐲lS\)<γ​\(𝐬l−1\)V\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}^\{S\}\)<\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)\. The distance between the true student distribution and the truncated distribution is bounded by the expected cumulative probability of the truncation over theLLsteps:

DTV​\(pS∥p𝝅Sγ\)≤∑l=1Lℙ𝐬l−1∼pS,𝐲l∼𝝅S​\(VT​\(𝐬l−1,𝐲l\)<γ​\(𝐬l−1\)\)\.D\_\{\\mathrm\{TV\}\}\(p\_\{S\}\\parallel p\_\{\\boldsymbol\{\\pi\}\_\{S\}\}^\{\\gamma\}\)\\leq\\sum\\nolimits\_\{l=1\}^\{L\}\\mathbb\{P\}\_\{\\mathbf\{s\}\_\{l\-1\}\\sim p\_\{S\},\\mathbf\{y\}\_\{l\}\\sim\\boldsymbol\{\\pi\}\_\{S\}\}\\left\(V\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}\)<\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)\\right\)\.\(16\)Recall our adaptive boundary formulation:

γ​\(𝐬l−1\)=γ0​exp⁡\(−α​ℋ​\(𝝅T∣𝐬l−1\)\)\.\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)=\\gamma\_\{0\}\\exp\\big\(\-\\alpha\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\)\\big\)\.When the teacher identifies multiple valid reasoning paths,*i\.e\.,*high entropyℋ\\mathcal\{H\},γ​\(𝐬l−1\)\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)exponentially decays\. This ensures the conditionVT<γV\_\{T\}<\\gammaisstrictly conservativeand triggers only whenVT→0V\_\{T\}\\to 0\(an irrecoverable logical error\), rather than for stylistic variances\. Sinceγ​\(𝐬l−1\)≤γ0\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)\\leq\\gamma\_\{0\}, the total variation in Eq\. \([16](https://arxiv.org/html/2605.19433#A1.E16)\) is rigorously constrained by the base scalarγ0\\gamma\_\{0\}\. Therefore,p𝝅Sγ≈pSp\_\{\\boldsymbol\{\\pi\}\_\{S\}\}^\{\\gamma\}\\approx p\_\{S\}across recoverable reasoning manifolds, verifying thecoverage condition\.□\\square

Theorem 5 \(validity\)\.Motabunconditionally prevents the entropy collapse described in Theorem 2, providing valid oracle supervision and fully circumventing the reversed exposure bias\.

Proof\.InMotab, upon detecting a boundary breach at stepll, the framework strictly avoids querying𝝅T\(⋅∣𝐬l\)\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\}\)\. Instead, credit assignment backtracks to a previous stepl∗l^\{\*\}, and the off\-policy target𝒴T\\mathcal\{Y\}^\{T\}is sampled recursively as:

𝒴l∗:LT∼∏τ=l∗L𝝅T​\(𝐲τ∣𝐬τ−1T\),where​𝐬l∗−1T=𝐬l∗−1\.\\mathcal\{Y\}^\{T\}\_\{l^\{\*\}:L\}\\sim\\prod\\nolimits\_\{\\tau=l^\{\*\}\}^\{L\}\\boldsymbol\{\\pi\}\_\{T\}\(\\mathbf\{y\}\_\{\\tau\}\\mid\\mathbf\{s\}\_\{\\tau\-1\}^\{T\}\),\\quad\\text\{where \}\\mathbf\{s\}\_\{l^\{\*\}\-1\}^\{T\}=\\mathbf\{s\}\_\{l^\{\*\}\-1\}\.Because the state𝐬l∗−1\\mathbf\{s\}\_\{l^\{\*\}\-1\}was verified as safe prior to backtracking, it satisfies the boundary constraintVT​\(𝐬l∗−2,𝐲l∗−1\)≥γ​\(𝐬l∗−2\)V\_\{T\}\(\\mathbf\{s\}\_\{l^\{\*\}\-2\},\\mathbf\{y\}\_\{l^\{\*\}\-1\}\)\\geq\\gamma\(\\mathbf\{s\}\_\{l^\{\*\}\-2\}\)\. This guarantees a high transition likelihood, ensuring the root context remains strictly within the teacher’s dense data manifold:

pT​\(𝐬l∗−1\)≥δ≫0\.p\_\{T\}\(\\mathbf\{s\}\_\{l^\{\*\}\-1\}\)\\geq\\delta\\gg 0\.Following the derivation in Eq\. \([15](https://arxiv.org/html/2605.19433#A1.E15)\), becausepT​\(𝐬l∗−1\)p\_\{T\}\(\\mathbf\{s\}\_\{l^\{\*\}\-1\}\)is substantially strictly positive, the conditional distribution𝝅T\(⋅∣𝐬l∗−1\)\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l^\{\*\}\-1\}\)remains sharp and highly confident\. The step\-level entropyℋ​\(𝝅T∣𝐬l∗−1\)\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l^\{\*\}\-1\}\)is thus bounded securely away from the uniform maximumK​log⁡\|𝒱\|K\\log\|\\mathcal\{V\}\|\.

Finally, the asynchronous stitching objective constructs the loss over the\[REV\]semantic transition:

ℒstitch=𝔼𝒴T​\[−log⁡𝝅S​\(\[REV\]⊕𝒴T∣𝐬lflawed\)\]\.\\mathcal\{L\}\_\{\\text\{stitch\}\}=\\mathbb\{E\}\_\{\\mathcal\{Y\}^\{T\}\}\\left\[\-\\log\\boldsymbol\{\\pi\}\_\{S\}\\big\(\\texttt\{\[REV\]\}\\oplus\\mathcal\{Y\}^\{T\}\\mid\\mathbf\{s\}\_\{l\}^\{\\text\{flawed\}\}\\big\)\\right\]\.This isolates the teacher’s generation process from the flawed context𝐬lflawed\\mathbf\{s\}\_\{l\}^\{\\text\{flawed\}\}\. The teacher operates purely on the pristine root𝐬l∗−1\\mathbf\{s\}\_\{l^\{\*\}\-1\}, while the student learns a conditional mapping from its own flawed exploration to a pristine corrective sequence via the\[REV\]bottleneck\. Consequently, the teacher’s supervision signal remains unconditionally valid, neutralizing the reversed exposure bias\.□\\square

## Appendix BAlgorithmic Pipeline ofMotab

In this section, we present the detailed algorithmic procedure of our proposed data synthesis framework,Motab\. The process is iteratively executed over a set of unlabelled reasoning questions to construct a refined supervised fine\-tuning dataset\.

As outlined in Alg\.[1](https://arxiv.org/html/2605.19433#alg1), the pipeline requires a teacher policy𝝅T\\boldsymbol\{\\pi\}\_\{T\}, a student policy𝝅S\\boldsymbol\{\\pi\}\_\{S\}, and an initial set of questions𝒬\\mathcal\{Q\}\. For each question, the student auto\-regressively generates reasoning steps while the teacher synchronously evaluates the transition likelihoodVTV\_\{T\}and context entropyℋ\\mathcal\{H\}\. If the generation remains within the adaptive safety boundaryγ\\gamma, the step is accepted, preserving the student’s authentic on\-policy distribution\. Once a critical logical drift is detected \(VT<γV\_\{T\}<\\gamma\), the algorithm triggers the credit assignment mechanism to trace back the temporal difference errorsδk\\delta\_\{k\}\. It identifies the optimal safe bifurcation state𝐬l∗−1\\mathbf\{s\}\_\{l^\{\*\}\-1\}, prompts the teacher to generate a pristine correction𝒴l∗:TT\\mathcal\{Y\}^\{T\}\_\{l^\{\*\}:T\}, and stitches the flawed exploration with the oracle correction using a specific\[REV\]token\. The resulting trajectories explicitly empower the student model with both robustness to its own stylistic variances and the capability for logical self\-correction\.

Algorithm 1Complete pipeline of the data synthesis viaMotabInput:Question set𝒬\\mathcal\{Q\}, Teacher policy𝝅T\\boldsymbol\{\\pi\}\_\{T\}, Student policy𝝅S\\boldsymbol\{\\pi\}\_\{S\}, Base thresholdγ0\\gamma\_\{0\}, Scaling factorα\\alpha\. Output:Synthesized supervised fine\-tuning dataset𝒟SFT\\mathcal\{D\}\_\{\\text\{SFT\}\}\.

1:Initialize dataset

𝒟SFT←∅\\mathcal\{D\}\_\{\\text\{SFT\}\}\\leftarrow\\varnothing;

2:foreach question

𝐪∈𝒬\\mathbf\{q\}\\in\\mathcal\{Q\}do

3:Initialize context

𝐬0←𝐪\\mathbf\{s\}\_\{0\}\\leftarrow\\mathbf\{q\}, step index

l←1l\\leftarrow 1;

4:Flag

i​s​\_​s​a​f​e←Trueis\\\_safe\\leftarrow\\text\{True\};

5:while

i​s​\_​s​a​f​eis\\\_safenot reached terminal statedo

6:Sample student reasoning step:

𝐲lS∼𝝅S\(⋅∣𝐬l−1\)\\mathbf\{y\}\_\{l\}^\{S\}\\sim\\boldsymbol\{\\pi\}\_\{S\}\(\\cdot\\mid\\mathbf\{s\}\_\{l\-1\}\);

7:Compute step value

VT​\(𝐬l−1,𝐲lS\)V\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}^\{S\}\)via Eq\. \([5](https://arxiv.org/html/2605.19433#S3.E5)\);

8:Compute teacher entropy

ℋ​\(𝝅T∣𝐬l−1\)\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{l\-1\}\);

9:Calculate adaptive boundary

γ​\(𝐬l−1\)←γ0⋅exp⁡\(−α⋅ℋ\)\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)\\leftarrow\\gamma\_\{0\}\\cdot\\exp\(\-\\alpha\\cdot\\mathcal\{H\}\);

10:if

VT​\(𝐬l−1,𝐲lS\)≥γ​\(𝐬l−1\)V\_\{T\}\(\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{l\}^\{S\}\)\\geq\\gamma\(\\mathbf\{s\}\_\{l\-1\}\)then

11:⊳\\trianglerightSafe step, continue on\-policy unrolling

12:Update context

𝐬l←\[𝐬l−1,𝐲tS\]\\mathbf\{s\}\_\{l\}\\leftarrow\[\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{t\}^\{S\}\];

13:

l←l\+1l\\leftarrow l\+1;

14:else

15:⊳\\trianglerightBoundary breached, trigger backtracking

16:

i​s​\_​s​a​f​e←Falseis\\\_safe\\leftarrow\\text\{False\};

17:for

k=1k=1to

ttdo

18:Compute temporal difference error

δk=VT​\(𝐬k−1,𝐲kS\)−VT​\(𝐬k−2,𝐲k−1S\)\\delta\_\{k\}=V\_\{T\}\(\\mathbf\{s\}\_\{k\-1\},\\mathbf\{y\}\_\{k\}^\{S\}\)\-V\_\{T\}\(\\mathbf\{s\}\_\{k\-2\},\\mathbf\{y\}\_\{k\-1\}^\{S\}\);

19:endfor

20:Identify optimal safe root

l∗←arg⁡mink≤l⁡\(δk\)l^\{\*\}\\leftarrow\\arg\\min\_\{k\\leq l\}\(\\delta\_\{k\}\)s\.t\. prior state is safe;

21:⊳\\trianglerightOff\-policy correction and stitching

22:Sample corrective trajectory

𝒴l∗:TT∼𝝅T\(⋅∣𝐬l∗−1\)\\mathcal\{Y\}\_\{l^\{\*\}:T\}^\{T\}\\sim\\boldsymbol\{\\pi\}\_\{T\}\(\\cdot\\mid\\mathbf\{s\}\_\{l^\{\*\}\-1\}\);

23:Construct stitched trajectory

𝝉←\[𝐪,𝒴1:lS\]⊕\[REV\]⊕𝒴l∗:TT\\boldsymbol\{\\tau\}\\leftarrow\[\\mathbf\{q\},\\mathcal\{Y\}\_\{1:l\}^\{S\}\]\\oplus\\texttt\{\[REV\]\}\\oplus\\mathcal\{Y\}\_\{l^\{\*\}:T\}^\{T\};

24:

𝒟SFT←𝒟SFT∪\{𝝉\}\\mathcal\{D\}\_\{\\text\{SFT\}\}\\leftarrow\\mathcal\{D\}\_\{\\text\{SFT\}\}\\cup\\\{\\boldsymbol\{\\tau\}\\\};

25:endif

26:endwhile

27:if

i​s​\_​s​a​f​eis\\\_safethen

28:⊳\\trianglerightSuccessfully finished without correction

29:

𝒟SFT←𝒟SFT∪\{𝐬l\}\\mathcal\{D\}\_\{\\text\{SFT\}\}\\leftarrow\\mathcal\{D\}\_\{\\text\{SFT\}\}\\cup\\\{\\mathbf\{s\}\_\{l\}\\\};

30:endif

31:endfor

32:return

𝒟SFT\\mathcal\{D\}\_\{\\text\{SFT\}\}

## Appendix CComputational Complexity Analysis

In addition to resolving the theoretical dual exposure biases,Motabintroduces significant computational advantages during the data synthesis pipeline\. In this section, we formally analyze the time complexity ofMotaband compare it against the SKD baseline introduced in Sec\.[2\.4](https://arxiv.org/html/2605.19433#S2.SS4)\.

To formulate the computational overhead, letΛ\\Lambdadenote the total token length of a reasoning trajectory\. For an LLM parameterized by𝜽\\boldsymbol\{\\theta\}, we distinguish between two primary operations: \(1\)Auto\-regressive generation\(𝒪gen​\(𝜽,Λ\)\\mathcal\{O\}\_\{\\text\{gen\}\}\(\\boldsymbol\{\\theta\},\\Lambda\)\), which requires sequential decoding and saving KV cache, which is memory\-bandwidth bound; \(2\)Parallel verification and forward pass\(𝒪vfy​\(𝜽,Λ\)\\mathcal\{O\}\_\{\\text\{vfy\}\}\(\\boldsymbol\{\\theta\},\\Lambda\)\), which computes logits over a given sequence in parallel\. Due to parallelized KV\-caching and matrix multiplications, verification is significantly faster than generation,*i\.e\.,*,𝒪vfy≪𝒪gen\\mathcal\{O\}\_\{\\text\{vfy\}\}\\ll\\mathcal\{O\}\_\{\\text\{gen\}\}\.

Complexity of SKD\.In standard SKD pipelines, the teacher model𝝅T\\boldsymbol\{\\pi\}\_\{T\}continuously monitors the student𝝅S\\boldsymbol\{\\pi\}\_\{S\}at the token or granular step level\. Whenever the student’s logic slightly deviates, SKD forces the teacher to intervene and generate a replacement\. Assuming an intervention rateα∈\(0,1\)\\alpha\\in\(0,1\), the total time complexity per trajectory is:

𝒪SKD≈𝒪gen​\(𝜽S,Λ\)\+𝒪vfy​\(𝜽T,Λ\)\+α⋅𝒪gen​\(𝜽T,Λ\)\.\\mathcal\{O\}\_\{\\text\{SKD\}\}\\approx\\mathcal\{O\}\_\{\\text\{gen\}\}\(\\boldsymbol\{\\theta\}\_\{S\},\\Lambda\)\+\\mathcal\{O\}\_\{\\text\{vfy\}\}\(\\boldsymbol\{\\theta\}\_\{T\},\\Lambda\)\+\\alpha\\cdot\\mathcal\{O\}\_\{\\text\{gen\}\}\(\\boldsymbol\{\\theta\}\_\{T\},\\Lambda\)\.Because SKD enforces strict alignment, the intervention rateα\\alphais inherently high\. The frequent context switching and repeated auto\-regressive generation by the massively parameterized teacher \(𝜽T\>𝜽S\\boldsymbol\{\\theta\}\_\{T\}\>\\boldsymbol\{\\theta\}\_\{S\}\) result in a severe computational bottleneck\.

Complexity ofMotab\.Conversely,Motabisolates operations to minimize the teacher’s generation burden\. Assuming a boundary breach occurs at stepll\(corresponding to token lengthΛfail\\Lambda\_\{\\text\{fail\}\}\), and the safe point is identified at stepl∗l^\{\*\}, the complexity ofMotabis decomposed as follows:

- •Phase 1 \(monitoring\):The student freely generates the prefix, and the teacher evaluates the steps using Eq\. \([5](https://arxiv.org/html/2605.19433#S3.E5)\) and \([6](https://arxiv.org/html/2605.19433#S3.E6)\)\. Because the student’s text is already generated, the teacher performs highly efficient parallel forward passes\. The cost is𝒪gen​\(𝜽S,Λfail\)\+𝒪vfy​\(𝜽T,Λfail\)\\mathcal\{O\}\_\{\\text\{gen\}\}\(\\boldsymbol\{\\theta\}\_\{S\},\\Lambda\_\{\\text\{fail\}\}\)\+\\mathcal\{O\}\_\{\\text\{vfy\}\}\(\\boldsymbol\{\\theta\}\_\{T\},\\Lambda\_\{\\text\{fail\}\}\)\.
- •Phase 2 \(backtracking\):Identifying the root causel∗l^\{\*\}relies on computing the temporal difference errorδk\\delta\_\{k\}\. Since the step\-wise valuesVTV\_\{T\}are already cached during Phase 1, this operation requires zero neural network passes\. It is purely algebraic, incurring a negligible complexity of𝒪​\(1\)\\mathcal\{O\}\(1\)\.
- •Phase 3 \(stitching\):The teacher generates the corrective suffix of lengthΛcorrect\\Lambda\_\{\\text\{correct\}\}from the root cause\. The cost is𝒪gen​\(𝜽T,Λcorrect\)\\mathcal\{O\}\_\{\\text\{gen\}\}\(\\boldsymbol\{\\theta\}\_\{T\},\\Lambda\_\{\\text\{correct\}\}\)\.

Summing these phases, the total time complexity forMotabis:

𝒪Motab≈𝒪gen​\(𝜽S,Λfail\)\+𝒪vfy​\(𝜽T,Λfail\)\+𝒪gen​\(𝜽T,Λcorrect\)\.\\mathcal\{O\}\_\{\\text\{\{Motab\}\}\}\\approx\\mathcal\{O\}\_\{\\text\{gen\}\}\(\\boldsymbol\{\\theta\}\_\{S\},\\Lambda\_\{\\text\{fail\}\}\)\+\\mathcal\{O\}\_\{\\text\{vfy\}\}\(\\boldsymbol\{\\theta\}\_\{T\},\\Lambda\_\{\\text\{fail\}\}\)\+\\mathcal\{O\}\_\{\\text\{gen\}\}\(\\boldsymbol\{\\theta\}\_\{T\},\\Lambda\_\{\\text\{correct\}\}\)\.\(17\)The fundamental efficiency advantage ofMotablies in the adaptive boundary\. By tolerating sub\-optimal explorations,Motabdrastically reduces the necessity for teacher interventions, shifting the teacher’s computational workload from expensive auto\-regressive generation \(𝒪gen\\mathcal\{O\}\_\{\\text\{gen\}\}\) to highly efficient parallel verification \(𝒪vfy\\mathcal\{O\}\_\{\\text\{vfy\}\}\)\. Furthermore, unlike external verification methods,*e\.g\.,*process reward models, that require training an additional critic network, our backtracking by temporal difference error reuses the teacher’s innate confidence, effectively achieving precise credit assignment at zero additional inference cost\.

Empirical computational budgets\.To validate the practical efficiency ofMotab, we measure the real\-world data synthesis speed on a hardware setup consisting of 8×\\timesA800 GPUs\. We compare the average generation time ofMotabagainst the SKD baseline across three distinct hyperparameter configurations \(which control the intervention rates by the teacher\)\. Empirically, both methods demonstrate comparable synthesis efficiency, each requiring approximately 3 to 4 hours to generate 1k reasoning trajectories on average\. This result confirms that whileMotabintroduces sophisticated mechanisms,*e\.g\.,*backtracking with temporal difference error and trajectory stitching, to theoretically resolve the dual exposure biases, it does not incur noticeable empirical overhead in practice\.

## Appendix DDetailed Experimental Settings

In this section, we provide a comprehensive overview of the experimental setups, including the specific datasets, model architectures, baseline methodologies, and implementation details\.

### D\.1Training Datasets

Our training phase leverages high\-quality reasoning problem sets to facilitate effective knowledge distillation\. Specifically, the training corpus is constructed from two primary sources:

- •LIMO\-v2\[Yeet al\.,[2025](https://arxiv.org/html/2605.19433#bib.bib25)\]: We utilize problem sets from theLIMO\-v2repository111[https://huggingface\.co/datasets/GAIR/LIMO\-v2](https://huggingface.co/datasets/GAIR/LIMO-v2)\., which is specifically curated to elicit deep mathematical reasoning capabilities\. This dataset has undergone rigorous quality filtering and comprises 800 mathematical problems, and we generate five distinct responses for each problem\.
- •AceReason\[Chenet al\.,[2025c](https://arxiv.org/html/2605.19433#bib.bib24)\]: To introduce further diversity and scale to our training distribution, we randomly sample a subset of 10,000 problem instances from theAceReasonrepository222[https://huggingface\.co/datasets/nvidia/AceReason\-1\.1\-SFT](https://huggingface.co/datasets/nvidia/AceReason-1.1-SFT)\.\. For each problem, we generate one response\.

### D\.2Evaluation Benchmarks

To rigorously assess the generalization and reasoning capability of our trained student models, we employ a suite of five challenging benchmarks, categorized into domain\-specific mathematical reasoning and cross\-domain capabilities:

- •Mathematical reasoning:We evaluate our student models onAIME2024andAIME2025, which consist of highly complex, competition\-level problems from the American Invitational Mathematics Examination\. Each test set comprises 30 challenging mathematical problems\. During evaluation, we generate 8 responses per problem and report the pass@kkaccuracy, where at least one correct response is identified amongkksamples, fork∈\{8,4,1\}k\\in\\\{8,4,1\\\}\. Additionally, we useMATH500\[Lightmanet al\.,[2024](https://arxiv.org/html/2605.19433#bib.bib47)\], a comprehensive benchmark encompassing various difficulty levels and subfields of mathematics\. We evaluate the student models across varying difficulty levels, ranging from Level 1 to Level 5\. The respective test sets consist of 43, 90, 105, 128, and 134 problems, for which we report accuracy scores individually\.
- •Cross\-domain capabilities:Our models are trained on two mathematical reasoning datasets\. To ensure that our approach generalizes to other domains,*e\.g\.,*general instruction\-following and diverse knowledge reasoning, we evaluate our models on theIFEvalbenchmark\[Zhouet al\.,[2023](https://arxiv.org/html/2605.19433#bib.bib48)\], which assesses rigorous, verifiable instruction adherence\. This benchmark comprises 541 prompts\. We report both the widely used prompt\-level\-strict accuracy and the mean across all four metrics, including prompt\-level\-strict, inst\-level\-strict, prompt\-level\-loose, and inst\-level\-loose\. Furthermore, we evaluate our models onGPQA\[Reinet al\.,[2023](https://arxiv.org/html/2605.19433#bib.bib49)\], a challenging graduate\-level benchmark covering physics, biology, and chemistry\. This benchmark consists of 198 challenging scientific questions\. Consistent with our evaluation protocol on the AIME datasets, we perform eight samplings per problem and report the pass@k metrics\.

### D\.3Model Architectures

We evaluate our method across LLMs of varying scales and architectural families\.

Teacher LLMs\.We employQwen3\-32B333[https://huggingface\.co/Qwen/Qwen3\-32B](https://huggingface.co/Qwen/Qwen3-32B)\.andgpt\-oss\-120b444[https://huggingface\.co/openai/gpt\-oss\-120b](https://huggingface.co/openai/gpt-oss-120b)\.as our teacher models\. These large\-scale models possess extensive parameter counts and demonstrate state\-of\-the\-art proficiency in generating high\-quality, accurate, and complex reasoning trajectories\.

### D\.4Baseline Methods

We compare our proposed approach against four representative baselines spanning standard fine\-tuning and knowledge distillation paradigms\.

- •Supervised fine\-tuning \(SFT\):The student model is directly fine\-tuned on reasoning trajectories generated by the teacher, serving as a fundamental reference baseline\.
- •Rejection sampling fine\-tuning \(RFT\):Multiple candidate trajectories are sampled from the student model; those leading to incorrect final answers are discarded, and the student is fine\-tuned exclusively on the surviving correct rollouts\.
- •Imitation\-based knowledge distillation \(ImitKD\)\[Linet al\.,[2020](https://arxiv.org/html/2605.19433#bib.bib10)\]: An autoregressive imitation learning method in which, at each decoding step, the generation is stochastically sourced from either the teacher or the student\.
- •Speculative knowledge distillation \(SKD\)\[Xuet al\.,[2025](https://arxiv.org/html/2605.19433#bib.bib11)\]: A step\-level, speculative distillation scheme\. At each reasoning stepll, the student first generates a candidate step𝐚lS\\mathbf\{a\}^\{S\}\_\{l\}, which is then evaluated by both the teacher and the student via their respective geometric mean token probabilities: p¯M​\(𝐚lS∣𝐬l−1\)=exp⁡\(1\|𝐚lS\|​∑i=1\|𝐚lS\|log⁡pM​\(yi∣𝐬l−1,𝐲<i\)\),M∈\{S,T\}\.\\bar\{p\}\_\{M\}\\\!\\left\(\\mathbf\{a\}^\{S\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\\right\)=\\exp\\\!\\left\(\\frac\{1\}\{\\left\|\\mathbf\{a\}^\{S\}\_\{l\}\\right\|\}\\sum\\nolimits\_\{i=1\}^\{\\left\|\\mathbf\{a\}^\{S\}\_\{l\}\\right\|\}\\log p\_\{M\}\\\!\\left\(y\_\{i\}\\mid\\mathbf\{s\}\_\{l\-1\},\\mathbf\{y\}\_\{<i\}\\right\)\\right\),\\quad M\\in\\\{S,T\\\}\.The student’s step is accepted if the teacher’s likelihood sufficiently dominates the student’s: p¯T​\(𝐚lS∣𝐬l−1\)\>β⋅p¯S​\(𝐚lS∣𝐬l−1\),\\bar\{p\}\_\{T\}\\\!\\left\(\\mathbf\{a\}^\{S\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\\right\)\\;\>\\;\\beta\\cdot\\bar\{p\}\_\{S\}\\\!\\left\(\\mathbf\{a\}^\{S\}\_\{l\}\\mid\\mathbf\{s\}\_\{l\-1\}\\right\),whereβ∈\(0,1\]\\beta\\in\(0,1\]is a margin factor controlling acceptance strictness\. When this condition fails, the student’s step is discarded, and the teacher regenerates a replacement, steering the trajectory back toward high\-quality reasoning\.

Meanwhile, we also compare several recent on\-policy distillation methods, which are described as follows:

- •Top\-kkon\-policy distillation \(top\-kkOPD\)\[Fuet al\.,[2026](https://arxiv.org/html/2605.19433#bib.bib26)\]: This method mitigates the train\-inference distribution mismatch by training the student model on its own self\-generated sequences rather than a fixed dataset\. It leverages token\-level teacher probabilities as expert feedback on these student outputs, accommodating flexible divergence measures and seamless integration with reinforcement learning\. Unlike GKD\[Agarwalet al\.,[2024](https://arxiv.org/html/2605.19433#bib.bib9)\], which calculates the divergence over the entire vocabulary\-level distribution, this method utilizes the top\-kktokens to enhance computational efficiency\.
- •Distribution\-aligned sequence distillation \(DASD\)\[Yanet al\.,[2026](https://arxiv.org/html/2605.19433#bib.bib14)\]: Designed to enhance sequence\-level distillation for long chain\-of\-thought reasoning, this method integrates temperature\-scheduled learning and divergence\-aware sampling to better cover and align with the teacher’s distribution\. To explicitly combat exposure bias, it employs a mixed\-policy strategy that corrects student\-generated prefixes with targeted teacher continuations\.

### D\.5Implementation Details

Data generation\.All models are served via SGLang\. During the data generation pipeline ofMotab, the student generates reasoning steps delimited by the stop sequence"\.\\n\\n", with each step capped at 8,192 tokens and the full trajectory bounded at 32,768 tokens\. The teacher’s predictive entropyℋ​\(𝝅T∣𝐬t−1\)\\mathcal\{H\}\(\\boldsymbol\{\\pi\}\_\{T\}\\mid\\mathbf\{s\}\_\{t\-1\}\)is estimated from the renormalized top\-20 next\-token logprobs\. Upon an unsafe point, the backtracking based on the temporal difference error identifies the pointl∗l^\{\*\}\(subject to the constraint that stepk−1k\{\-\}1was still above its adaptive threshold\), after which the teacher generates a full corrective suffix from the pristine contextsl∗−1s\_\{l^\{\*\}\-1\}, and the final SFT trajectory is assembled as: student exploration\+\+‘‘However,’’\+\+teacher correction\. We setγ0∈\{0\.1,0\.2,0\.3,0\.4,0\.5\}\\gamma\_\{0\}\\in\\\{0\.1,0\.2,0\.3,0\.4,0\.5\\\}and temperatureτ=0\.6\\tau=0\.6for Qwen3\-32B or1\.01\.0for gpt\-oss\-120b, and generateN=5N=5trajectories per question inLIMO\-v2dataset andN=1N=1forAceReason\.

Student model training\.The student models are trained with full\-parameter SFT using LLaMA\-Factory on 8 GPUs viatorchrun, withDeepSpeed ZeRO\-Stage 3,Flash Attention 2, and theLiger Kernelenabled\. To accelerate training, we employ sequence packing, where shorter training samples are concatenated into sequences of up to 32,768 tokens to maximize context utilization\. We train for 6 epochs with a per\-device batch size of 4 \(effective global batch size 32\), using AdamW withβ1=0\.9\\beta\_\{1\}\{=\}0\.9,β2=0\.95\\beta\_\{2\}\{=\}0\.95,ϵ=10−8\\epsilon\{=\}10^\{\-8\}, weight decay0\.10\.1, and gradient clipping at norm1\.01\.0\. The learning rate is5×10−55\{\\times\}10^\{\-5\}with a cosine decay schedule \(minimum10−610^\{\-6\}\) and a linear warmup over 5% of total steps\.

Table 8:Complete experimental results on more student models using Qwen3\-32B as the teacher model\. The bold results indicate the best scores\.MethodAIME24AIME25MATH500IFEvalGPQAP@8P@4P@1P@8P@4P@1L1L2L3L4L5Avg\.P@8Student LLM: Qwen3\-1\.7BSFT36\.6725\.909\.5826\.6722\.2911\.2590\.7084\.4473\.3367\.9743\.2843\.3868\.18RFT13\.3311\.385\.8333\.3324\.6712\.0881\.4075\.5674\.2966\.4141\.0441\.0260\.10ImitKD36\.6730\.1916\.2536\.6728\.3312\.5081\.4080\.0080\.0059\.3848\.5143\.3163\.13SKD\(β=0\.6\\beta=0\.6\)23\.3317\.577\.9226\.6722\.339\.5990\.7083\.3377\.1466\.4145\.5244\.3661\.62SKD\(β=0\.8\\beta=0\.8\)43\.3334\.7114\.5836\.6728\.2413\.3388\.3785\.5676\.1968\.7544\.0344\.3673\.74SKD\(β=1\.0\\beta=1\.0\)23\.3318\.5710\.8333\.3323\.909\.1786\.0576\.6769\.5260\.1641\.0444\.1962\.63Motab\(ours\)50\.0041\.3822\.0840\.0032\.0514\.5890\.7093\.3382\.8678\.9158\.2146\.8073\.74Student LLM: Llama3\.2\-3B\-InstructSFT16\.6711\.193\.336\.673\.330\.8379\.0753\.3351\.4342\.1922\.3942\.3058\.59RFT10\.007\.382\.506\.673\.330\.8374\.4258\.8959\.0542\.1920\.9041\.1653\.54ImitKD16\.679\.762\.926\.674\.761\.6776\.7466\.6757\.1440\.6217\.9143\.3566\.67SKD\(β=0\.6\\beta=0\.6\)16\.678\.332\.086\.674\.952\.0876\.6468\.8958\.1042\.9715\.6741\.5764\.65SKD\(β=0\.8\\beta=0\.8\)20\.0011\.672\.9213\.339\.052\.9288\.3763\.3359\.0550\.0023\.8841\.6567\.68SKD\(β=1\.0\\beta=1\.0\)16\.6712\.383\.756\.675\.241\.6783\.7258\.8952\.3834\.3817\.9139\.7971\.21Motab\(ours\)23\.3313\.765\.0016\.6711\.675\.8386\.0568\.8960\.9548\.4428\.3647\.6773\.74Student LLM: Qwen2\.5\-7B\-InstructSFT23\.3319\.7610\.4216\.6710\.004\.5883\.7274\.4469\.5260\.1644\.7834\.7869\.70RFT16\.6712\.865\.4220\.0014\.005\.0088\.3782\.2280\.9558\.5935\.0734\.2756\.06ImitKD20\.0016\.195\.8326\.6716\.575\.8374\.4270\.0072\.3857\.8135\.8235\.3270\.71SKD\(β=0\.6\\beta=0\.6\)23\.3319\.2410\.0023\.3317\.867\.9283\.7283\.3375\.2465\.6233\.5836\.1870\.71SKD\(β=0\.8\\beta=0\.8\)26\.6722\.0510\.0026\.6718\.817\.5088\.3781\.1173\.3369\.5335\.0733\.8576\.77SKD\(β=1\.0\\beta=1\.0\)20\.0013\.334\.1723\.3317\.146\.6786\.0577\.7872\.3864\.0641\.7934\.6377\.27Motab\(ours\)30\.0023\.3310\.4226\.6719\.527\.9290\.7084\.4481\.9067\.1948\.5137\.6179\.29Implementation of on\-policy distillation\.We distill from a Qwen3\-8B teacher into a Qwen3\-1\.7B student using the framework fromFuet al\.\[[2026](https://arxiv.org/html/2605.19433#bib.bib26)\]\. The three components of our on\-policy pipeline:rollout generation,teacher forward pass, andstudent optimization, execute in a time\-sliced fashion on the same set of GPUs, with each component activated only during its designated phase to accommodate all three models within the available memory budget\. At each training iteration, the rollout engine generates responses for a batch of 32 prompts, sampling 5 responses per prompt with temperature 0\.6 and top\-pp0\.95\. The maximum sequence length is set to 32,768 tokens\. The teacher model then performs a forward pass over these rollout sequences, and the resulting logits are used to supervise the student in the subsequent training phase\. The student is trained with a reverse\-KL distillation objective, with a global batch size of 16 and a per\-device micro\-batch size of 1\. We use the AdamW optimizer with a peak learning rate of2×10−62\\times 10^\{\-6\}and a linear warmup over the first 5% of training steps\. Training runs for 10 epochs with gradient checkpointing enabled throughout\. A key memory challenge in on\-policy distillation arises from materialising student and teacher logit tensors simultaneously: for a 32k\-token response and a vocabulary of approximately 152k tokens, storing both logit matrices together with their gradients requires a number of GPU memory\. To address this, top\-kkOPD\[Fuet al\.,[2026](https://arxiv.org/html/2605.19433#bib.bib26)\]calculates the KL divergence using only the top\-32 tokens in the vocabulary\. Experimental results demonstrate that this approach significantly reduces the computational burden without compromising the overall performance\.

## Appendix EMore Experimental Results

In this section, we provide more experimental results and analysis forMotab\.

![Refer to caption](https://arxiv.org/html/2605.19433v1/x6.png)Figure 5:Distributions of absolute unsafe points \(top row\), relative unsafe points \(middle row\), and backtracking depths \(bottom row\) across varying base safety thresholdsγ0∈\{0\.2,0\.3,0\.4,0\.5\}\\gamma\_\{0\}\\in\\\{0\.2,0\.3,0\.4,0\.5\\\}\.### E\.1Experiments on More Student Models

To further substantiate the robustness and generalizability of our proposed framework, Table[8](https://arxiv.org/html/2605.19433#A4.T8)provides comprehensive experimental results across a broader spectrum of student models, utilizing Qwen3\-32B as the teacher\. Specifically, we evaluateMotabon diverse model scales and architectures, including Qwen3\-1\.7B, Llama3\.2\-3B\-Instruct, and Qwen2\.5\-7B\-Instruct\. Across all three student models,Motabdemonstrates consistent and remarkable superiority over both standard off\-policy \(*e\.g\.,*SFT, RFT\) and on\-policy \(*e\.g\.,*ImitKD, SKD\) baselines\. For instance, on the extremely lightweight Qwen3\-1\.7B model,Motabdramatically improves the AIME24 Pass@1 from 9\.58% \(SFT\) to 22\.08%, and elevates the most challenging MATH500 Level 5 accuracy to 58\.21%, significantly eclipsing all baseline methods\. Furthermore, the results on Llama3\.2\-3B highlight the efficacy ofMotabin cross\-architecture distillation scenarios\. Despite the inherent stylistic and distribution discrepancies between the Qwen teacher and the Llama student, our method successfully bridges this gap, consistently pushing the performance upper bound\. Notably, we observe that heuristic\-based on\-policy methods like SKD exhibit severe sensitivity to their manual thresholdsβ\\beta, with their optimal configurations fluctuating heavily depending on the specific student architecture and evaluation metric\. In contrast, the entropy\-driven adaptive boundary inMotabseamlessly adjusts to the student’s intrinsic distribution, systematically yielding optimal performance without requiring exhaustive hyperparameter tuning\. Beyond mathematical reasoning,Motabalso delivers robust enhancements on general instruction following \(IFEval\) and advanced scientific knowledge \(GPQA\) tasks\. This comprehensively verifies that the self\-correction capabilities and error\-recovery mechanisms instilled by our trajectory stitching and backtracking are domain\-agnostic, ultimately fostering a more versatile and reliable reasoning policy across varying base models\.

### E\.2Analysis of Backtracking Dynamics

To better understand the intervention behavior ofMotab, we visualize the distributions of detected unsafe points and the corresponding backtracking depths across varying base thresholds \(γ0∈\{0\.2,0\.3,0\.4,0\.5\}\\gamma\_\{0\}\\in\\\{0\.2,0\.3,0\.4,0\.5\\\}\)\. As illustrated in Fig\.[5](https://arxiv.org/html/2605.19433#A5.F5), while the absolute unsafe points \(top row\) tend to cluster in the earlier steps of generation, the relative unsafe points \(middle row\) exhibit a remarkably uniform distribution across the entire lifespan of the trajectories\. This uniformity underscores that logical deviations are not strictly confined to the initial planning phases but can emerge dynamically at any stage of complex reasoning, thereby necessitating the continuous, step\-by\-step monitoring implemented in our framework\. More importantly, the distribution of backtracking depths \(bottom row\) provides compelling empirical validation for our credit assignment mechanism\. Across all threshold settings, a substantial proportion of interventions require rewinding by multiple steps \(depth\>1\>1\) rather than merely correcting the immediately preceding step\. This phenomenon corroborates our core hypothesis regarding the delayed manifestation of reasoning errors: a critical boundary breach detected at stepttis frequently rooted in a flawed premise established several steps prior\. By successfully tracing the temporal difference error back to the genuine safe bifurcation point, the credit assignment strategy ensures that the teacher model intervenes exclusively within a pristine, uncorrupted context, thereby fundamentally circumventing the reversed exposure bias\.

![Refer to caption](https://arxiv.org/html/2605.19433v1/x7.png)Figure 6:Evolution of the step\-level valueVTV\_\{T\}across reasoning steps \(left\), the overall distribution ofVTV\_\{T\}\(center\), and the safety margins \(VT−γV\_\{T\}\-\\gamma\) for backtracked versus non\-backtracked trajectories \(right\), evaluated under different base thresholdsγ0∈\{0\.1,0\.3,0\.5\}\\gamma\_\{0\}\\in\\\{0\.1,0\.3,0\.5\\\}\.
### E\.3Dynamics of Teacher Confidence and Safety Margins

To further investigate how the safety boundary influences the reasoning process, we analyze the dynamics of the teacher’s step\-level valueVTV\_\{T\}across different base thresholds \(γ0∈\{0\.1,0\.3,0\.5\}\\gamma\_\{0\}\\in\\\{0\.1,0\.3,0\.5\\\}\)\. As depicted in the left column of Fig\.[6](https://arxiv.org/html/2605.19433#A5.F6),VTV\_\{T\}generally exhibits an upward trend as the step index increases, indicating that logical certainty solidifies as the trajectory approaches the final answer\. Notably, asγ0\\gamma\_\{0\}increases,VTV\_\{T\}converges to 1\.0 much more rapidly\. This demonstrates that a stricter boundary compellingly restricts the student to the teacher’s high\-confidence manifold, accelerating logical convergence but concurrently compressing the exploratory space\. This observation is corroborated by theVTV\_\{T\}distributions \(center column\), which become increasingly heavily skewed toward 1\.0 at higherγ0\\gamma\_\{0\}values\. Furthermore, we evaluate the safety margin, defined asVT−γV\_\{T\}\-\\gamma, to compare trajectories that triggered backtracking against those that did not \(right column\)\. Asγ0\\gamma\_\{0\}increases, the discriminative gap in the safety margin between the “Back” and “No Back” data progressively widens\. This indicates that a tighter boundary enhances the framework’s sensitivity, more decisively distinguishing fundamentally flawed deviations from valid logical progressions\. However, synthesizing this with our main results reveals a crucial trade\-off: while a largerγ0\\gamma\_\{0\}provides sharper error detection and faster confidence convergence, it prematurely truncates acceptable sub\-optimal explorations, thereby hindering the mitigation of exposure bias\. Consequently, a moderate boundary \(*e\.g\.,*γ0=0\.3\\gamma\_\{0\}=0\.3\) is vital to balance discriminative error detection with adequate distributional coverage\.

![Refer to caption](https://arxiv.org/html/2605.19433v1/x8.png)Figure 7:Qualitative case study of theMotabframework on an arithmetic word problem\. The student’s illogical ramble upon encountering a non\-integer human count drops below the safety boundary \(0\.42<0\.490\.42<0\.49\), prompting a backtrack and a teacher\-guided correction stitching\.![Refer to caption](https://arxiv.org/html/2605.19433v1/x9.png)Figure 8:Qualitative case study of theMotabframework on a parameterized quadratic optimization problem\. A premature logical leap regarding boundary conditions triggers intervention \(0\.41<0\.480\.41<0\.48\), seamlessly pivoting the trajectory into a rigorous case\-by\-case analysis\.
### E\.4Qualitative Case Studies

To intuitively demonstrate how theMotabframework operates in practice, we present two qualitative case studies\. In the first case, illustrated in Fig\.[7](https://arxiv.org/html/2605.19433#A5.F7), the student is tasked with an arithmetic word problem\. The student correctly calculates an intermediate result of 33\.75 women; however, because a human count must logically be an integer, the student subsequently spirals into confusion, generating an uncertain and rambling interpretation that violates the safety boundary \(0\.42<0\.490\.42<0\.49\)\. Instead of discarding the entire attempt or allowing the error to cascade, our framework backtracks to the precise point before the logical collapse\. It then stitches the flawed exploration with a teacher\-generated correction that rigorously formalizes the impossibility of the scenario\. In the second case, presented in Fig\.[8](https://arxiv.org/html/2605.19433#A5.F8), the student attempts to find the minimum of a quadratic function within a parameterized interval\[−2,a\]\[\-2,a\]\. After successfully calculating the vertex, the student makes a premature and incomplete logical leap regarding the boundary condition, causing the value score to drop below the threshold \(0\.41<0\.480\.41<0\.48\)\.Motabtraces the temporal difference error back to the vertex calculation and injects a comprehensive case\-by\-case analysis\. By seamlessly stitching this pristine, teacher\-guided correction to the student’s flawed attempt using the semantic token “However”, the resulting trajectory explicitly teaches the student how to recognize its own premature conclusions and pivot toward a more systematic analytical approach\. Together, these cases illustrate howMotabeffectively preserves the student’s natural reasoning style while providing timely, high\-quality interventions precisely when the intrinsic logic breaks down\.

## Appendix FLimitations

AlthoughMotabfundamentally mitigates the dual exposure biases and significantly enhances the student model’s reasoning capabilities, it inevitably introduces additional computational overhead during the data synthesis phase\. Specifically, continuously monitoring the student’s step\-wise generation, calculating the teacher’s conditional likelihoods, and estimating predictive entropy for the adaptive boundary necessitate frequent, synchronous interactions between the student and teacher models\. This renders the offline trajectory generation process more computationally intensive compared to traditional off\-policy methods\. However, it is crucial to emphasize that this computational cost is strictly confined to the offline training preparation stage; once fine\-tuned, the student model’s inference efficiency and deployment costs remain completely unaffected\. Given the substantial performance gains on complex reasoning tasks and the acquisition of intrinsic self\-correction abilities, trading offline computational resources for high\-quality, bias\-mitigated distillation data represents a highly justifiable and practical compromise\. Meanwhile, we provide a detailed formal and experimental complexity analysis in Appendix[C](https://arxiv.org/html/2605.19433#A3)\.

## Appendix GLLM Usage

LLMs were utilized solely for grammatical refinement and linguistic polishing of the manuscript; the core ideas, methodology, and results were developed independently by the authors\.