Reasoning Can Be Restored by Correcting a Few Decision Tokens

# Reasoning Can Be Restored by Correcting a Few Decision Tokens
Source: [https://arxiv.org/html/2605.16874](https://arxiv.org/html/2605.16874)
###### Abstract

Large reasoning models \(LRMs\) substantially outperformtheir base LLM counterparts on challenging reasoning benchmarks, yet it remainspoorly understoodwhere base modelsgo wrongduring token\-by\-token generation and how to narrow this gap efficiently\. We study the base–reasoning gap throughquantifyingtoken\-level distributional disagreement between a base model and a stronger reasoning modelusing likelihood\-based divergences\.Across benchmarks,we find thatthe reasoning advantage is highly sparse and concentrates on a small set of early, planning\-related decision tokens\. For instance, on Qwen3\-0\.6B, only∼\\sim8% of generated tokens account for the salient disagreement, and these tokens concentrate early in the response, are strongly enriched in planning\-related decisions \(17×17\\times\), and coincide with high base\-model uncertainty—suggesting that base models fail mainly at early planning points that steer the subsequent reasoning trajectory\. Building on this findings, we propose disagreement\-guided token intervention, a simple inference\-time delegation scheme that performs a one\-token takeover by the reasoning model only at high\-disagreement positions and immediately switches back to the base model\. With a small intervention budget, this sparse delegation substantially recovers and can even surpass the performance of a same\-size reasoning model on challenging reasoning tasks\.Code is available at[https://github\.com/AlphaLab\-USTC/RRTokenIntervention](https://github.com/AlphaLab-USTC/RRTokenIntervention)\.

Large Language Models, Reasoning, Inference Intervention, Interpretability

## 1Introduction

![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/intro/teaser6.png)Figure 1:Disagreement\-guided token interventionand its effectiveness\.Intervening on 8% critical tokens, triggered by high token\-level distributional disagreement through a strong LRM \(*i\.e\.,*top figure\), enables a 0\.6B base model to recover most reasoning capabilities of the 8B strong LRM \(*i\.e\.,*bottom figure\) and outperform its post\-trained 0\.6B variant\.Large reasoning models \(LRMs\) have recently demonstrated substantial advantages over their base LLM counterparts on challenging reasoning benchmarks\(Zhaoet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib1); Jaechet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib16); OpenAI,[2024](https://arxiv.org/html/2605.16874#bib.bib17); Guoet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib18); Mathematical Association of America,[2024](https://arxiv.org/html/2605.16874#bib.bib62); Heet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib61)\), with gains commonly attributed to test\-time scaling\(Liet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib6); Snellet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib7); Wanget al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib23); Yaoet al\.,[2023a](https://arxiv.org/html/2605.16874#bib.bib24)\)and reinforcement learning\-based post\-training such as RLVR\(Shaoet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib26); Wenet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib8)\)\. Beyond further improving performance, this widening base–LRM gap has motivated growing interest in a more fundamental question:*what*differs between base and reasoning models, and*why*does “reasoning mode” work? A growing body of evidence supports a latent\-capability view in which base models may already encode considerable reasoning machinery, while post\-training mainly activates, amplifies, or stabilizes it through minimal additional signals—such as activation\-vector steering\(Turneret al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib35); Venhoffet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib37); Zouet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib36); Wardet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib31)\), confidence\-based self\-rewarding\(Yuanet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib10); Heet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib11)\), distribution\-level output amplification\(Zhaoet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib30)\), or even one\-example unlocks\(Wanget al\.,[2025b](https://arxiv.org/html/2605.16874#bib.bib34)\)\.

To address the*why*question, prior work has proposed hypotheses at multiple levels\. Reasoning capabilities may emerge from eliciting richer intermediate trajectories, as in chain\-of\-thought prompting\(Weiet al\.,[2022](https://arxiv.org/html/2605.16874#bib.bib21); Zhouet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib22); Wanget al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib23)\); from inducing reasoning micro\-behaviors such as backtracking\(Yaoet al\.,[2023a](https://arxiv.org/html/2605.16874#bib.bib24),[b](https://arxiv.org/html/2605.16874#bib.bib13)\), verification\(Lightmanet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib65)\), and self\-correction\(Madaanet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib14); Panet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib15); Marjanovicet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib29)\); from steering reasoning direction by restricting policy gradient updates to high\-entropy tokens\(Wanget al\.,[2025a](https://arxiv.org/html/2605.16874#bib.bib42)\); or from updating only a small subset of parameters,*i\.e\.,*a top\-singular reasoning subspace\(Caiet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib33); Huet al\.,[2021](https://arxiv.org/html/2605.16874#bib.bib32)\)\. Despite these insights, we still lack a simple token\-level account of*why*“reasoning mode” works—namely, where reasoning ability actually manifests during token generation and which decision points truly drive the base–LRM performance gap\.

In this paper, we address this question by quantifying the base–LRM gap through token\-level distributional disagreement between a base model and a stronger reasoning model using likelihood\-based divergences such as cross\-entropy and reverse Kullback–Leibler divergence\(Cover and Thomas,[2001](https://arxiv.org/html/2605.16874#bib.bib19)\)\. We then characterize which tokens carry the reasoning advantage, how many such tokens exist, and what properties they exhibit\. Across benchmarks, we find that reasoning ability is concentrated in a surprisingly small set of planning\-related decision tokens that more likely appear early in the response\. Concretely, only a small fraction of generated tokens determines whether the model commits to a reasoning trajectory: on Qwen3\-0\.6B\(Yanget al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib57)\), merely∼\\sim8% of output tokens account for the salient base–LRM disagreement \(*i\.e\.,*disagreement spike\) \(*cf\.*Figure[1](https://arxiv.org/html/2605.16874#S1.F1)\), indicating that token\-level disagreement is highly sparse\. Moreover, these disagreement spikes appear disproportionately early in generation \(*cf\.*Figure[2](https://arxiv.org/html/2605.16874#S2.F2)\), contrary to the common intuition that reasoning gains primarily stem from long traces after the reasoning process unfolds\. Finally, these critical tokens are tightly linked to planning under uncertainty: during recovery from base to reasoning rollouts, planning\-related tokens become markedly more likely, increasing by17×17\\timesin occurrence probability \(*cf\.*Table[5](https://arxiv.org/html/2605.16874#S5.T5)\), and they correlate strongly with the base model’s intrinsic uncertainty \(*cf\.*Figure[2\(c\)](https://arxiv.org/html/2605.16874#S2.F2.sf3)\)\. This suggests that base models struggle on complex reasoning tasks primarily at the planning points—where they are least confident are precisely the planning decisions that steer the subsequent reasoning trajectory\.

Motivated by these findings, we ask whether a base model can attain strong reasoning performance with limited assistance from a reasoning model—by delegating only a small number of critical decisions\. We propose a simple inference\-time mechanism as shown in Figure[1](https://arxiv.org/html/2605.16874#S1.F1): at positions where the base–LRM disagreement score is large, we perform a one\-token takeover—the reasoning model generates only the next token at that position, after which generation immediately switches back to the base model\(Onget al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib45); Huanget al\.,[2026](https://arxiv.org/html/2605.16874#bib.bib46)\)\. We term this sparse delegation scheme disagreement\-guided token intervention \(Figure[3](https://arxiv.org/html/2605.16874#S4.F3)\)\. Experiments show that with a small intervention budget, this approach can substantially recover—and in some settings even surpass—the performance of a same\-size reasoning model on challenging reasoning tasks\. Together, these findings motivate a token\-level view of reasoning: a small number of early planning commitments can steer the entire trajectory and dominate the reasoning gap\.

## 2Related Work

Reasoning capabilities in base models\.Reasoning behaviors can emerge in small or base models when they are exposed to appropriate training signals, such as reasoning exemplars or Chain\-of\-Thought\(Weiet al\.,[2022](https://arxiv.org/html/2605.16874#bib.bib21)\)traces\(Hoet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib25)\)\. Explicit alignment, including fine\-tuning on structured rationales, often suffices to elicit multi\-step problem solving in models that previously answered directly\(Hoet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib25)\)\. Reinforcement learning has been shown to gradually restructure output patterns, increasing response length and inducing more consistent use of intermediate reasoning steps\(Guoet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib18); Wanget al\.,[2025b](https://arxiv.org/html/2605.16874#bib.bib34)\)\. Activation\-space methods such as steering or targeted editing demonstrate that reasoning patterns, like backtracking, can be causally induced in base models without weight updates\(Venhoffet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib37); Wardet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib31)\)\. Inference\-time approaches, such as CoT prompting or self\-verification feedback, can also trigger reasoning behaviors that do not naturally surface in zero\-shot settings\(Weiet al\.,[2022](https://arxiv.org/html/2605.16874#bib.bib21); Gandhiet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib28)\)\. Collectively, these findings suggest that base models possess a wide range of reasoning capabilities that can be activated through post\-training mechanisms, guidance signals, or structural interventions\.

Token\-level distributional disagreement\.Cross\-entropy, entropy, and divergence offer effective signals for identifying decision\-critical positions where models diverge or hesitate during generation\. These signals can help locate forking tokens, error\-prone steps, or places where weak models diverge sharply from stronger references\.KL divergence, in both forward and reverse form,has been used to compare different models’ behavior\.Forward KLpenalizes the student for deviating from the teacher’s distribution, whileReverse KLemphasizes tokens where the student is overconfident in outputs disfavored by the teacher\(Hintonet al\.,[2015](https://arxiv.org/html/2605.16874#bib.bib38); Guet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib39); Agarwalet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib40)\)\.Entropymeasures local uncertainty in the model’s predictive distribution\. Entropy spikes often correspond to reasoning forks or hesitation\. Focusing updates on high\-entropy tokens improves RLVR efficiency\(Wanget al\.,[2025a](https://arxiv.org/html/2605.16874#bib.bib42)\), while entropy\-guided rollout branching improves exploration\(Zhenget al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib43)\)\.Cross\-entropymeasures the magnitude of divergence between a model’s output and a target distribution, and is widely used as a training loss\. In contrast,Δ​log⁡p\\Delta\\log p, the difference between log probabilities in model before and after post\-training, captures the directionality of model updates\. Such metrics support the identification of high\-impact tokens where reasoning behavior changes or supervision is most needed\.

Selective intervention at inference time\.The reasoning capabilities of weak models can be enhanced by inserting strong\-model control without replacing the entire rollout\. These approaches typically fall into two paradigms: collaborative inference for efficiency and guided generation for distillation stability\. In collaborative inference, strong models serve as on\-demand verifiers to balance performance and compute\. At the request level, expensive resources can be reserved for hard queries by dispatching inputs based on predicted capability gaps\(Onget al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib45)\)\. For finer\-grained control, verification mechanisms allow strong models to screen and take over from lightweight drafters upon detecting quality degradation\(Chenet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib47)\)\. Furthermore, difficult reasoning steps can be explicitly “relayed” to a strong model\(Huanget al\.,[2026](https://arxiv.org/html/2605.16874#bib.bib46)\)or escalated to the cloud\(Sunet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib48)\)specifically when small models exceed internal risk thresholds\. Beyond inference acceleration, selective interventions are crucial for stabilizing on\-policy distillation by preventing student models from learning from erroneous trajectories\(Hintonet al\.,[2015](https://arxiv.org/html/2605.16874#bib.bib38); Guet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib39); Agarwalet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib40)\)\. To ensure data quality, teacher distributions can serve as filters to reject unlikely tokens\(Xuet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib51)\), or dynamically take over generation at divergence peaks to correct reasoning paths\(Penget al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib49)\)\. Together, these strategies demonstrate that concentrating teacher assistance on sparse, high\-impact positions effectively bridges the gap between weak and strong models in both inference and training stages\.

![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/part1/aggregate_lorenz_curve.png)\(a\)Disagreement is highly sparse
![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/part1/aggregate_position_density_top1.png)\(b\)High\-disagreement tokens occur early
![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/part1/aggregate_iou_by_dataset.png)\(c\)Disagreement–uncertainty overlap
![Refer to caption](https://arxiv.org/html/2605.16874v1/x1.png)

Figure 2:Token\-level disagreement is sparse, early, and uncertainty\-aligned across benchmarks\. \(a\) Lorenz curves show that disagreement mass is highly concentrated in a small fraction of tokens\. \(b\) Normalized\-position density of the top\-1%1\\%highest\-disagreement tokens is strongly skewed toward early steps \(u=t/T\), indicating early\-to\-mid trajectory concentration relative to a uniform baseline\. \(c\) IoU between top\-p%p\\%disagreement tokens and top\-p%p\\%entropy tokens remains high across thresholds, showing strong alignment between cross\-model disagreement and base\-model uncertainty\.
## 3Preliminary

In this paper, we focus on the token\-level behavioral differences between a standard base model and a stronger reasoning model sharing the same vocabulary\. In this section, we establish the framework for our analysis\. We first introduce the comparative generation setup and the metrics for quantifying distributional disagreement in[Section3\.1](https://arxiv.org/html/2605.16874#S3.SS1)\. Then, we formally formulate the problem of sparse reasoning takeover in[Section3\.2](https://arxiv.org/html/2605.16874#S3.SS2)\.

### 3\.1Setup and Notations

Autoregressive generation process\.To characterize the reasoning gap, we consider a standard autoregressive generation setting involving two models sharing a vocabulary𝒱\\mathcal\{V\}: a base modelℳb\\mathcal\{M\}\_\{b\}and a stronger reasoning modelℳr\\mathcal\{M\}\_\{r\}\(Vaswaniet al\.,[2017](https://arxiv.org/html/2605.16874#bib.bib3); Brownet al\.,[2020](https://arxiv.org/html/2605.16874#bib.bib4); Hintonet al\.,[2015](https://arxiv.org/html/2605.16874#bib.bib38); Leviathanet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib50)\)\. Given a promptx0x\_\{0\}, the model generates a sequencey=\(y1,…,yT\)y=\(y\_\{1\},\\dots,y\_\{T\}\)\. At each steptt, letxt=\(x0,y<t\)x\_\{t\}=\(x\_\{0\},y\_\{<t\}\)denote the context\. We denote the next\-token probability distributions aspb\(⋅∣xt\)p\_\{b\}\(\\cdot\\mid x\_\{t\}\)andpr\(⋅∣xt\)p\_\{r\}\(\\cdot\\mid x\_\{t\}\)respectively\. Typically,ℳr\\mathcal\{M\}\_\{r\}exhibits superior reasoning capabilities \(e\.g\., via larger scale or specialized post\-training\) but incurs higher computational costs, whereasℳb\\mathcal\{M\}\_\{b\}is efficient but performance\-limited\(Kaplanet al\.,[2020](https://arxiv.org/html/2605.16874#bib.bib5); Ouyanget al\.,[2022](https://arxiv.org/html/2605.16874#bib.bib12); Weiet al\.,[2022](https://arxiv.org/html/2605.16874#bib.bib21)\)\. Our goal is to understand howℳb\\mathcal\{M\}\_\{b\}deviates from the superior trajectory ofℳr\\mathcal\{M\}\_\{r\}\.

Quantified disagreement metrics\.To quantify the behavioral disagreement between the two models at a granular level, we define a generictoken\-level disagreement scorests\_\{t\}along the trajectory generated by the base model\. Specifically, at stepttwith contextxtx\_\{t\}, the disagreement is measured by a disagreement function𝒟f\\mathcal\{D\}\_\{f\}between the base model’s distributionpb\(⋅∣xt\)p\_\{b\}\(\\cdot\\mid x\_\{t\}\)and the reasoning model’s distributionpr\(⋅∣xt\)p\_\{r\}\(\\cdot\\mid x\_\{t\}\):

st=𝒟f\(pb\(⋅∣xt\),pr\(⋅∣xt\)\)s\_\{t\}=\\mathcal\{D\}\_\{f\}\\big\(p\_\{b\}\(\\cdot\\mid x\_\{t\}\),\\,p\_\{r\}\(\\cdot\\mid x\_\{t\}\)\\big\)\(1\)In this work, we primarily utilize the Cross Entropy \(CE\\mathrm\{CE\}\) as the disagreement metric\(Cover and Thomas,[2001](https://arxiv.org/html/2605.16874#bib.bib19)\)\. It measures the expected surprisal of the reasoning model with respect to the base model’s choices:

𝒟f=𝒟CE​\(t\)\\displaystyle\\mathcal\{D\}\_\{f\}=\\mathcal\{D\}\_\{\\mathrm\{CE\}\}\(t\)=H\(pb\(⋅∣xt\),pr\(⋅∣xt\)\)\\displaystyle=H\(p\_\{b\}\(\\cdot\\mid x\_\{t\}\),p\_\{r\}\(\\cdot\\mid x\_\{t\}\)\)\(2\)=−∑y∈𝒱pb​\(y∣xt\)​log⁡pr​\(y∣xt\)\.\\displaystyle=\-\\sum\_\{y\\in\\mathcal\{V\}\}p\_\{b\}\(y\\mid x\_\{t\}\)\\log p\_\{r\}\(y\\mid x\_\{t\}\)\.
We note that other metrics, such as the Reverse KL divergence, are also applicable and closely related \(i\.e\.,DrKL=𝒟CE−HbD\_\{\\mathrm\{rKL\}\}=\\mathcal\{D\}\_\{\\mathrm\{CE\}\}\-H\_\{b\}\)\. These metrics serve as the primary probing signals to localize where the base model deviates from the reasoning model’s decision path\. In parallel, we monitor the base model’s intrinsic uncertainty using the Shannon entropyHb\(t\)=H\(pb\(⋅∣xt\)\)H\_\{b\}\(t\)=H\(p\_\{b\}\(\\cdot\\mid x\_\{t\}\)\)\.

### 3\.2Problem Formulation

While the reasoning modelprp\_\{r\}outperforms the base modelpbp\_\{b\}, deploying it for every token generation is computationally prohibitive\. Our objective is to bridge this performance gap by intervening only on a sparse subset of tokens\. Formally, we aim to identify a set ofdecision\-critical positions𝒮⊂\{1,…,T\}\\mathcal\{S\}\\subset\\\{1,\\dots,T\\\}, such that\|𝒮\|≪T\|\\mathcal\{S\}\|\\ll T\.

We define adisagreement\-guided token interventionmechanism where the generation follows the base modelpbp\_\{b\}by default, but switches to the reasoning modelprp\_\{r\}at indicest∈𝒮t\\in\\mathcal\{S\}\. This can be viewed as a dynamic interplay where the base model handles execution tokens, and the reasoning model intervenes solely at decision tokens\.

Our core hypothesis is that the performance gap is not uniformly distributed across all tokens but is concentrated at positions of high distributional disagreement\. Therefore, the identification of𝒮\\mathcal\{S\}can be formulated as a detection problem using the disagreement metrics defined in Section[3\.1](https://arxiv.org/html/2605.16874#S3.SS1)\. IfDCE​\(t\)D\_\{\\mathrm\{CE\}\}\(t\)effectively captures these critical branching points, we can recover the reasoning capabilities ofℳr\\mathcal\{M\}\_\{r\}with a minimal intervention budget\. In the following sections, we will first analyze the distribution ofDfD\_\{f\}to validate this sparsity hypothesis \(Section[4](https://arxiv.org/html/2605.16874#S4)\), and then propose a concrete algorithm for budgeted intervention \(Section[5](https://arxiv.org/html/2605.16874#S5)\)\.

## 4Token\-Level Base–LRM Disagreement

In this section, we provide a comprehensive analysis of the token\-level distributional disagreement between base models and LRMs\. Specifically, we have the following core findings: \(i\) token\-level distributional disagreement happens at few token positions, which is sparse and heavy\-tailed \(*cf\.*Section[4\.1](https://arxiv.org/html/2605.16874#S4.SS1)\); \(ii\) token\-level distributional disagreement tends to happen at early positions \(*cf\.*Section[4\.2](https://arxiv.org/html/2605.16874#S4.SS2)\); \(iii\) token\-level distributional disagreement tends to align with the uncertainty of the base model, concentrating on planning\-related tokens \(*cf\.*Section[4\.3](https://arxiv.org/html/2605.16874#S4.SS3)\); and \(iv\) token\-level distributional disagreement degree can be utilized for predicting the final reasoning failures \(*cf\.*Section[4\.4](https://arxiv.org/html/2605.16874#S4.SS4)\)\.

We conduct analysis mainly on the Qwen3 family\(Yanget al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib57)\), employing the Qwen3\-0\.6B\-Base model for rollout generation \(pbp\_\{b\}\) and the Qwen3\-8B \(Thinking\) for reasoning guidance \(prp\_\{r\}\)\. Evaluations are conducted across six mathematical benchmarks ranging from standard tasks \( GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.16874#bib.bib58)\), MATH500\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.16874#bib.bib59)\)\) to competition\-level challenges \( AIME\(Mathematical Association of America,[2024](https://arxiv.org/html/2605.16874#bib.bib62)\), OlympiadBench\(Heet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib61)\), AMC23\(AI\-MO,[2023](https://arxiv.org/html/2605.16874#bib.bib63)\)\)\. Detailed configurations for models, datasets, and baselines are provided in Appendix[A](https://arxiv.org/html/2605.16874#A1)\. We follow the setup in[Section3\.1](https://arxiv.org/html/2605.16874#S3.SS1)and use token\-level disagreementsts\_\{t\}\(Cross Entropy by default\) computed along the base rollout\.

### 4\.1Disagreement Concentrates in Few Token Positions

Finding 1:Disagreement mass is extremely concentrated: a tiny fraction of tokens accounts for most cross\-model disagreement\.

We study the per\-token disagreement scores\{st\}\\\{s\_\{t\}\\\}along base\-model rollouts and ask whether the gap reflects diffuse, incremental drift or a small number of spikes\.Figure[2\(a\)](https://arxiv.org/html/2605.16874#S2.F2.sf1)shows that token\-level distributional disagreement is sparse and heavy\-tailed, since the Lorenz curve lies far below the equality line \(*i\.e\.,*most tokens accumulate little mass of the disagreement while a small subset dominates\)\.

We summarize this concentration with the Gini coefficient, which quantifies how unevenly the cumulative disagreement mass is distributed across token positions \(0 = uniform, 1 = maximally concentrated,*cf\.*Appendix[B\.1](https://arxiv.org/html/2605.16874#A2.SS1)\)\(Hurley and Rickard,[2009](https://arxiv.org/html/2605.16874#bib.bib55); Lorenz,[1905](https://arxiv.org/html/2605.16874#bib.bib54)\)\. Across six math reasoning benchmarks, we obtain an averageG=0\.936G=0\.936\(details in Appendix[B\.1](https://arxiv.org/html/2605.16874#A2.SS1)\)\. This extreme inequality supports a sparse control view: the base–reasoning gap is driven primarily by rare, high\-magnitude deviations at a small set of trajectory\-sensitive positions, rather than by many small errors accumulating over time\.

### 4\.2Disagreement Spikes Happen at Early Positions

Finding 2:High\-disagreement spikes are front\-loaded: most occur in the earliest portion of generation rather than being spread uniformly across the rollout\.

We examine the positional structure of disagreement by normalizing each token index asu=t/T∈\[0,1\]u=t/T\\in\[0,1\]and selecting, for each sample, the top\-1% tokens by disagreement scorests\_\{t\}\. We then aggregate these selected positions across samples and estimate their density\. As shown in Figure[2\(b\)](https://arxiv.org/html/2605.16874#S2.F2.sf2), the resulting distribution is strongly left\-skewed: spikes occur very early \(mode atu≈0\.05u\\\!\\approx\\\!0\.05\) and the density rapidly decays, becoming close to baseline byu≈0\.2u\\\!\\approx\\\!0\.2\.

This pattern is inconsistent with a view where reasoning gains primarily come from long traces after the reasoning process unfolds\. Instead, because early Chain\-of\-Thought tokens often encode decomposition and strategy selection\(Weiet al\.,[2022](https://arxiv.org/html/2605.16874#bib.bib21)\), the positional skew supports a planning gap interpretation: the models diverge most at early trajectory\-setting commitments, not during later execution\.

### 4\.3Disagreement Spikes Relate to Model Uncertainty

Table 1:Planning enrichment at uncertainty/disagreement spikes\.We label tokens asplanningvs\.executionand report the share of planning tokens in the global token stream versus among top\-1%1\\%tokens ranked by either base entropyHb​\(t\)H\_\{b\}\(t\)or cross\-model divergencests\_\{t\}\.Enrich\.is the ratio of the two shares\.Finding 3:Disagreement spikes are not arbitrary: they coincide with base\-model uncertainty spikes and are strongly enriched for planning\-related tokens\.

Disagreement spikes coincide with intrinsic uncertainty\.We ask whether cross\-model disagreementsts\_\{t\}highlights structurally meaningful positions or merely reflects superficial stylistic differences\. Figure[2\(c\)](https://arxiv.org/html/2605.16874#S2.F2.sf3)reports the Intersection\-over\-Union \(IoU\) between tokens ranked by disagreement scorests\_\{t\}\(CE\-based\) and those ranked by the base model entropyHb​\(t\)H\_\{b\}\(t\), computed per sample over the top\-p%p\\%sets and averaged\. The strong overlap shows that disagreement spikes concentrate at steps where the base model is intrinsically uncertain, providing an internal validation that these spikes correspond to difficult, trajectory\-sensitive points rather than random mismatch\.

These uncertainty\-linked disagreement spikes are planning\-rich\.We further characterize the content of these positions\. Using a lightweight heuristic classifier forplanningvs\.executiontokens \(Appendix[B\.2](https://arxiv.org/html/2605.16874#A2.SS2)\), we compute planning enrichment among the top\-1% tokens ranked bysts\_\{t\}andHb​\(t\)H\_\{b\}\(t\)respectively within base\-model rollouts\(Subramanianet al\.,[2005](https://arxiv.org/html/2605.16874#bib.bib56)\)\. As shown in Table[1](https://arxiv.org/html/2605.16874#S4.T1), planning markers are strongly over\-represented: across all datasets, the planning share rises from 1\.89% globally to 15\.75% at entropy spikes \(8\.31×\\times\) and to 14\.13% at disagreement spikes \(7\.46×\\times\)\. Appendix[B\.3](https://arxiv.org/html/2605.16874#A2.SS3)provides per\-dataset results, confirming the trend is consistent across benchmarks\.

Taken together, disagreement spikes systematically track uncertain planning commitments—positions where a small change in planning\-related tokens can steer the subsequent trajectory\.

![Refer to caption](https://arxiv.org/html/2605.16874v1/x2.png)Figure 3:Illustration of Disagreement\-Gated Intervention\. In Step I, we calibrate the disagreement thresholdτ\\tauand factorλ\\lambdausing the top\-r%r\\%of scores from base rollouts\. In Step II, the gategtg\_\{t\}dynamically switches between the Base model and the Intervene model \(reasoning expert\)\. Intervention is triggered only when the token\-level disagreementsts\_\{t\}satisfies both the global constraint \(st\>τs\_\{t\}\>\\tau\) and the local relative spike condition \(st\>λ⋅s¯t,Ws\_\{t\}\>\\lambda\\cdot\\bar\{s\}\_\{t,W\}\)\.
### 4\.4Disagreement Spikes Predict Reasoning Failures

![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/part1/gsm8k_sample100_distribution_top100_metric.png)Figure 4:The distribution of top\-100 mean disagreement scores shows a distinct separation, where incorrect samples \(red\) exhibit significantly higher values than correct ones \(green\)\.Finding 4:Aggregating disagreement over a small set of high\-disagreement tokens predicts reasoning failures\.

The previous analyses suggest that disagreement is \(i\) extremely concentrated in rare spikes, \(ii\) front\-loaded early in the rollout, and \(iii\) aligned with uncertain, planning\-heavy steps\. If these spikes indeed reflect trajectory\-setting mistakes, then their magnitude within a sample should be predictive of whether the rollout ultimately succeeds\.

Table 2:Failure prediction performance\. We compare failure prediction using base model uncertainty \(Entropy\) versus our cross\-model disagreement \(Cross Entropy\), aggregated as the mean of top\-100 tokens per sample\.Failure score from disagreement spikes\.For each sample, we form a scalar score by averaging the top\-KK\(K=100K=100\) per\-token disagreement valuessts\_\{t\}within that sample\. We evaluate how well this score predicts incorrect final answers using AUROC and Accuracy; we compute an analogous score using base entropyHb​\(t\)H\_\{b\}\(t\)for comparison\.

As shown in Table[2](https://arxiv.org/html/2605.16874#S4.T2)and Figure[4](https://arxiv.org/html/2605.16874#S4.F4), the disagreement\-based score provides a clear separation between correct and incorrect samples and consistently outperforms entropy \(e\.g\., GSM8K: 0\.851 vs\. 0\.817 AUROC\)\. This result closes the loop with the preceding structure findings: the same sparse, early, planning\-linked spikes that dominate disagreement mass also carry actionable, sample\-level predictive signal\.

These observations motivate a question: if failure is driven by a small set of high\-disagreement decisions, can selectively correcting only those positions recover strong\-model reasoning under a small token budget? In the next section, we answer this question by introducing a disagreement\-guided, token\-level intervention at inference time\.

Table 3:Experimental results on mathematical reasoning benchmarks \(Accuracy / Pass@8\)\. The Recovery column quantifies how much the intervention recovers reasoning capabilities, calculated as\(PG​u​i​d​e​d−PB​a​s​e\)/\(PT​h​i​n​k​i​n​g−PB​a​s​e\)\(P\_\{Guided\}\-P\_\{Base\}\)/\(P\_\{Thinking\}\-P\_\{Base\}\), wherePPdenotes Pass@8\.ModelMath500GSM8KAMC23OlympiadAIME24AIME25AverageRecoveryQwen3\-0\.6B\-Base27\.6 / 67\.029\.9 / 82\.013\.4 / 42\.56\.8 / 21\.00\.4 / 3\.30\.0 / 0\.013\.0 / 36\.0\-\+ Guided \(ρ¯≈0\.04\\bar\{\\rho\}\\approx 0\.04\)44\.6 / 89\.056\.5 / 96\.035\.9 / 77\.520\.2 / 49\.010\.0 / 36\.77\.5 / 20\.029\.1 / 61\.491%91\\%\+ Guided \(ρ¯≈0\.13\\bar\{\\rho\}\\approx 0\.13\)73\.9 / 99\.080\.4 / 99\.058\.4 / 90\.042\.5 / 65\.032\.5 / 70\.026\.7 / 56\.752\.4 / 80\.0157%157\\%Qwen3\-0\.6B \(Thinking\)72\.1 / 90\.081\.6 / 94\.046\.9 / 80\.034\.3 / 54\.09\.2 / 30\.016\.1 / 36\.743\.4 / 64\.1\-Qwen3\-1\.7B\-Base54\.2 / 82\.075\.4 / 93\.028\.1 / 65\.019\.2 / 41\.05\.0 / 20\.02\.5 / 10\.030\.8 / 51\.8\-\+ Guided \(ρ¯≈0\.06\\bar\{\\rho\}\\approx 0\.06\)66\.8 / 95\.072\.0 / 97\.050\.6 / 95\.033\.2 / 64\.020\.4 / 46\.718\.8 / 43\.343\.6 / 73\.576%76\\%\+ Guided \(ρ¯≈0\.16\\bar\{\\rho\}\\approx 0\.16\)81\.5 / 100\.088\.9 / 98\.071\.9 / 97\.548\.9 / 67\.045\.0 / 73\.336\.2 / 66\.762\.1 / 83\.8112%112\\%Qwen3\-1\.7B \(Thinking\)88\.9 / 100\.090\.9 / 97\.080\.0 / 95\.052\.5 / 63\.039\.2 / 73\.332\.8 / 53\.364\.1 / 80\.3\-Qwen3\-8B \(Thinking\)97\.4 / 100\.097\.8 / 98\.090\.9 / 100\.060\.0 / 69\.070\.0 / 83\.352\.5 / 73\.378\.1 / 87\.3\-

## 5Reasoning Can Be Restored by Correcting a Few Decision Tokens

[Section4](https://arxiv.org/html/2605.16874#S4)shows that base–reasoning disagreement is \(i\) extremely sparse and mostly happens at early positions of the trajectory, \(ii\) aligned with the base model’s intrinsic uncertainty and enriched for planning markers, and \(iii\) predictive of failure\. Motivated by this sparse\-control picture, we conduct an intervention test: can selectively correcting only a small set of disagreement spikes yield large performance recovery under a small replacement budget?

Finding 5:A disagreement\-gated, token\-level replacement at a small set of spikes is sufficient to redirect rollouts and recover a disproportionate fraction of strong\-model reasoning performance under small budgets\.

### 5\.1Calibrated Disagreement Spikes for Gated Intervention

We design a minimal inference\-time intervention that alters only a small subset of emitted tokens, leaving model parameters and hidden states unchanged\. At each decoding steptt, we compute atoken\-level disagreement scorest=𝒟f\(pb\(⋅∣xt\),pr\(⋅∣xt\)\),s\_\{t\}=\\mathcal\{D\}\_\{f\}\\\!\\left\(p\_\{b\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\),\\,p\_\{r\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\)\\right\),where𝒟\\mathcal\{D\}is a distributional disagreement between next\-token distributions of the base and reasoning models\. Unless otherwise stated, we instantiate𝒟\\mathcal\{D\}with cross entropy \(CE\), consistent with[Section4](https://arxiv.org/html/2605.16874#S4); we report results with reverse\-KL as an alternative disagreement metric in Appendix[C\.1](https://arxiv.org/html/2605.16874#A3.SS1)\.

Global calibration via a spike ratiorr\(budget control\)\.To avoid per\-task tuning, we calibrate a global spike criterion on a held\-out calibration set by defining disagreement spikes as the top\-rrtail of disagreement scores\. Concretely, let𝒮=\{st\}\\mathcal\{S\}=\\\{s\_\{t\}\\\}denote all disagreement scores collected along base rollouts on the calibration prompts\. We set the global thresholdτ\\tauas the\(1−r\)\(1\-r\)\-quantile of𝒮\\mathcal\{S\}:

τ=Q1−r​\(𝒮\),\\tau=Q\_\{1\-r\}\(\\mathcal\{S\}\),\(3\)so thatst\>τs\_\{t\}\>\\taucorresponds to a top\-rrdisagreement event under the calibration distribution, and the calibrated thresholds transfer robustly across benchmarks \(Appendix[C\.7](https://arxiv.org/html/2605.16874#A3.SS7)\)\. We additionally compute a tail\-to\-mean scale factor

λ=𝔼​\[s​∣s\>​τ\]𝔼​\[s\],\\lambda=\\frac\{\\mathbb\{E\}\[s\\mid s\>\\tau\]\}\{\\mathbb\{E\}\[s\]\},\(4\)which summarizes how much larger disagreement scores are within the top\-rrtail compared to the global average\.

Runtime gating\.During decoding, we trigger takeover only whensts\_\{t\}forms a robust spike under a two\-part gate:

gt=𝕀​\[st\>τ∧st\>λ⋅s¯t,W\],s¯t,W=1W​∑i=1Wst−i,g\_\{t\}=\\mathbb\{I\}\\Big\[s\_\{t\}\>\\tau\\ \\wedge\\ s\_\{t\}\>\\lambda\\cdot\\overline\{s\}\_\{t,W\}\\Big\],~\\overline\{s\}\_\{t,W\}=\\frac\{1\}\{W\}\\sum\_\{i=1\}^\{W\}s\_\{t\-i\},\(5\)where the global thresholdst\>τs\_\{t\}\>\\tauenforces a calibrated tail event, and the sliding\-window ratio test suppresses noisy toggling by requiring the current score to exceed the recent local baseline by a factorλ\\lambda\. Token generation then follows

yt∼\{pr\(⋅∣xt\),gt=1,pb\(⋅∣xt\),gt=0\.y\_\{t\}\\sim\\begin\{cases\}p\_\{r\}\(\\cdot\\mid x\_\{t\}\),&g\_\{t\}=1,\\\\ p\_\{b\}\(\\cdot\\mid x\_\{t\}\),&g\_\{t\}=0\.\\end\{cases\}\(6\)We also report the realized intervention rateρ\\rho, the fraction of tokens generated by the reasoning model:ρ=1T​∑t=1Tgt\\rho=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}g\_\{t\}\. The per\-problem realized rateρi\\rho\_\{i\}is concentrated with small variance across problems \(Appendix[C\.2](https://arxiv.org/html/2605.16874#A3.SS2)\)\.

Remark on practical deployment\.The disagreement gate queries bothpbp\_\{b\}andprp\_\{r\}at every step and is best viewed as a diagnostic form of sparse control rather than an optimized inference system\. A cheaper alternative gates on the base entropyHb​\(t\)H\_\{b\}\(t\)alone, which already recovers most of the strong\-model Pass@8 at small budgets \(Appendix[C\.3](https://arxiv.org/html/2605.16874#A3.SS3)\)\.

![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/part2/performance_vs_ratio_pass8_ce_st_global.png)Figure 5:Performance recovers sharply with few replacement\.![Refer to caption](https://arxiv.org/html/2605.16874v1/x3.png)\(a\)Base: brittle parsing and degeneration
![Refer to caption](https://arxiv.org/html/2605.16874v1/x4.png)\(b\)Guided: brief stop\-and\-check resolves the ambiguity

Figure 6:Qualitative pattern: takeover induces a “stop\-and\-check” planning step\.Green highlights tokens generated under takeover \(reasoning model\); white tokens are generated by the base model\. In this example, a short self\-check triggered at an ambiguous clause fixes the variable binding, after which control returns to the base model for routine execution\.
### 5\.2Sparse Interventions Yield Disproportionate Performance Recovery

We evaluate how much reasoning performance can be restored by intervening only at disagreement spikes\. Because a fixed spike ratiorrinduces different takeover frequencies across prompts, we report results against the realized intervention rateρ=1T​∑t=1Tgt\\rho=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}g\_\{t\}\.

A small budget can outperform a same\-size reasoning model\.Table[3](https://arxiv.org/html/2605.16874#S4.T3)shows that sparse takeover is highly effective\. For Qwen3\-0\.6B, guided decoding atρ¯≈0\.13\\bar\{\\rho\}\\\!\\approx\\\!0\.13improves the average from13\.0/36\.013\.0/36\.0to52\.4/80\.052\.4/80\.0\(Acc\./Pass@8\), surpassing the same\-size thinking variant \(43\.4/64\.143\.4/64\.1\) and recovering157%157\\%of its Pass@8 gap\. Qwen3\-1\.7B exhibits the same behavior: atρ¯≈0\.16\\bar\{\\rho\}\\\!\\approx\\\!0\.16, guided decoding reaches62\.1/83\.862\.1/83\.8, essentially matching its thinking counterpart \(64\.1/80\.364\.1/80\.3;112%112\\%recovery\)\. A sample\-level analysis further confirms that the intervention is rarely harmful: across400400problems, it flips152152examples from incorrect to correct and only33in the reverse direction \(Appendix[C\.8](https://arxiv.org/html/2605.16874#A3.SS8)\)\.

Recovery is sharply nonlinear in replacement ratio\.Figure[5](https://arxiv.org/html/2605.16874#S5.F5)reveals a pronounced knee: most of the recovery arrives within the first few percent replacement, followed by clear diminishing returns\. Concretely, the overall Pass@8 jumps by roughly∼20\\sim\\\!20points within the first few percent replacement \(from≈41%\\approx 41\\%atρ≈0\.5%\\rho\\\!\\approx\\\!0\.5\\%to≈61%\\approx 61\\%atρ≈3%\\rho\\\!\\approx\\\!3\\%\), while later increases inρ\\rhodeliver progressively smaller gains \(saturating near≈80%\\approx 80\\%byρ≈13%\\rho\\\!\\approx\\\!13\\%\)\. Notably, aroundρ≈7%−8%\\rho\\\!\\approx\\\!7\\%\\\!\-\\\!8\\%the overall curve already reaches the Qwen3\-0\.6B \(Thinking\) reference line, and approaching the 8B reference with larger budgets\.

Position selection, not strong\-model injection, drives the gain\.A natural concern is whether the recovery merely reflects that any injection of strong\-model tokens helps\. We rule this out with two matched\-budget baselines: a random baseline that uniformly samples replacement positions, and an early\-only baseline that replaces the firstρ⋅Lr\\rho\\\!\\cdot\\\!L\_\{r\}tokens \(motivated by Finding 2’s early skew\)\. As shown in Table[4](https://arxiv.org/html/2605.16874#S5.T4), both baselines at an aggressiveρ=0\.25\\rho=0\.25are dominated by guided intervention at onlyρ¯≈0\.04\\bar\{\\rho\}\\\!\\approx\\\!0\.04—a∼\\sim6×6\\timessmaller budget\. This shows that the gain stems from which tokens are corrected rather than from invoking the strong model alone, and that position alone is not sufficient: within the early region, only the disagreement spikes are truly trajectory\-setting\. Full per\-budget curves are provided in Appendix[C\.4](https://arxiv.org/html/2605.16874#A3.SS4)\.

Table 4:Matched\-budget baselines \(Qwen3\-0\.6B\)\.Average Acc / Pass@8 over six math benchmarks\. Guided intervention atρ¯≈0\.04\\bar\{\\rho\}\\\!\\approx\\\!0\.04exceeds both baselines atρ=0\.25\\rho=0\.25\.Generalization beyond Qwen3\.The same sparse\-control behavior transfers to a LLaMA pair \(LLaMA\-3\.1\-8B as base, DeepSeek\-R1\-Distill\-Llama\-8B as guide\), where intervening on∼\\sim20% of tokens recovers∼\\sim91% of the Pass@8 gap \(Appendix[C\.5](https://arxiv.org/html/2605.16874#A3.SS5)\)\. The calibration is also robust: thresholds re\-fitted on each benchmark stay within the same order of magnitude and preserve the recovery curve \(Appendix[C\.7](https://arxiv.org/html/2605.16874#A3.SS7)\)\.

Taken together, these results show that the base–reasoning gap can be closed largely by correcting a sparse set of high\-disagreement decisions: a few\-percent replacement already matches or exceeds the size\-matched thinking model\.

### 5\.3Intervention Induces Reasoning Patterns

Table 5:Enrichment Analysis\.We compare the token distribution in the global generation vs\. our intervention set\. Planning tokens are effectively targeted\.Beyond final accuracy, we ask what the gate intervenes on and how those interventions change the trajectory\. We find a consistent functional pattern: takeover is concentrated on planning markers and often manifests as a brief stop\-and\-check step that resolves an upstream ambiguity before handing control back to the base model\.

Takeover targets planning rather than execution\.Table[5](https://arxiv.org/html/2605.16874#S5.T5)quantifies the token types in our takeover set\. Although planning tokens constitute only 1\.9% of classifiable tokens in base rollouts, they account for 33\.3% of takeover tokens \(17\.6×\\timesenrichment\)\. Conversely, execution tokens are under\-represented \(0\.7×\\times\), suggesting that the gate largely trusts the base model for routine arithmetic once the high\-level structure is fixed\.

A representative example: induced stop\-and\-check planning\.Figure[6](https://arxiv.org/html/2605.16874#S5.F6)illustrates how this specialization appears in generation\. The base model fails to stably bind variables under a syntactically ambiguous clause and degenerates\. Under takeover, the reasoning model injects a short self\-check that disambiguates the dependency, after which decoding returns to the base model for straightforward computation\. We provide the full prompt and additional qualitative examples in Appendix[D](https://arxiv.org/html/2605.16874#A4)\. This planning\-specialized takeover provides a mechanistic account for the sharp budget–performance curve in Figure[5](https://arxiv.org/html/2605.16874#S5.F5): a small number of corrected planning commitments can steer long downstream execution\.

## 6Limitations

This work has several limitations\. First, due to compute constraints, our study primarily focuses on the Qwen3 family and math\-centric reasoning benchmarks; we provide initial evidence that the same sparse\-control phenomenon transfers to a LLaMA\-based pair \(Appendix[C\.5](https://arxiv.org/html/2605.16874#A3.SS5)\) and to non\-math scientific QA on GPQA\-Diamond \(Appendix[C\.6](https://arxiv.org/html/2605.16874#A3.SS6)\), but broader coverage of model families, model sizes, and reasoning domains \(e\.g\., code, multi\-hop QA, agentic reasoning\) remains future work\. Second, as a diagnostic\-first findings paper, we adopt a simple, transparent intervention rule and do not aim to exhaustively optimize system efficiency or compete with heavily engineered baselines\.

## 7Conclusion

We investigated the base–reasoning gap throughtoken\-level distributional disagreement\. Across reasoning benchmarks, disagreement highly concentrates in rare spikes that cluster early, align with base\-model uncertainty, and are enriched for planning\-related decision tokens; aggregating these spikes further predicts eventual failures\. Motivated by these findings, we introduced a globally calibrated, disagreement\-gated token intervention that edits only a small subset of tokens\. Under modest realized takeover budgets, this intervention recovers a disproportionate fraction of strong\-model reasoning performance and can outperform a same\-size reasoning variant\. Qualitatively, intervention primarily targets planning rather than execution\. Mechanistically, because generation is autoregressive, correcting a few early planning\-related tokens redirects the subsequent trajectory while leaving routine execution to the base model\. Overall, our results support a sparse control view of reasoning: a few early planning commitments can steer long downstream trajectories, enabling selective upgrades only at branching points\.

## Impact Statement

This paper studies the token\-level distributional disagreement between base models and large reasoning models \(LRMs\), and shows that token\-level interventions can help the base model largely recover the LRM’s reasoning capability\. Generally, this paper may yield a positive societal impact by helping us understand the underlying mechanism of LLM reasoning\.

## References

- R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. R\. Garea, M\. Geist, and O\. Bachem \(2024\)On\-policy distillation of language models: learning from self\-generated mistakes\.InICLR,Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p2.1),[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- AI\-MO \(2023\)American mathematics contest\.Note:[https://huggingface\.co/datasets/AI\-MO/aimo\-validation\-amc](https://huggingface.co/datasets/AI-MO/aimo-validation-amc)Cited by:[3rd item](https://arxiv.org/html/2605.16874#A1.I1.i3.p1.1),[§4](https://arxiv.org/html/2605.16874#S4.p2.3.3)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei \(2020\)Language models are few\-shot learners\.CoRRabs/2005\.14165\.Cited by:[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p1.13)\.
- Y\. Cai, D\. Cao, X\. Xu, Z\. Yao, Y\. Huang, Z\. Tan, B\. Zhang, G\. Liu, and J\. Fang \(2025\)On predictability of reinforcement learning dynamics for large language models\.CoRRabs/2510\.00553\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- Z\. Chen, X\. Yang, J\. Lin, C\. Sun, J\. Huang, and K\. C\. Chang \(2023\)Cascade speculative drafting for even faster LLM inference\.CoRRabs/2312\.11462\.Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.CoRRabs/2110\.14168\.Cited by:[1st item](https://arxiv.org/html/2605.16874#A1.I1.i1.p1.1),[§A\.1](https://arxiv.org/html/2605.16874#A1.SS1.SSS0.Px5.p1.3),[§4](https://arxiv.org/html/2605.16874#S4.p2.3.3)\.
- T\. M\. Cover and J\. A\. Thomas \(2001\)Elements of information theory\.Wiley\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p3.2.2),[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p2.7)\.
- K\. Gandhi, A\. Chakravarthy, A\. Singh, N\. Lile, and N\. D\. Goodman \(2025\)Cognitive behaviors that enable self\-improving reasoners, or, four habits of highly effective stars\.CoRRabs/2503\.01307\.Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p1.1)\.
- Y\. Gu, L\. Dong, F\. Wei, and M\. Huang \(2024\)MiniLLM: knowledge distillation of large language models\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7–11, 2024,External Links:[Link](https://openreview.net/forum?id=5h0qf7IBZZ)Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p2.1),[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi, X\. Zhang, X\. Yu, Y\. Wu, Z\. F\. Wu, Z\. Gou, Z\. Shao, Z\. Li, Z\. Gao, A\. Liu, B\. Xue, B\. Wang, B\. Wu, B\. Feng, C\. Lu, C\. Zhao, C\. Deng, C\. Ruan, D\. Dai, D\. Chen, D\. Ji, E\. Li, F\. Lin, F\. Dai, F\. Luo, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Xu, H\. Ding, H\. Gao, H\. Qu, H\. Li, J\. Guo, J\. Li, J\. Chen, J\. Yuan, J\. Tu, J\. Qiu, J\. Li, J\. L\. Cai, J\. Ni, J\. Liang, J\. Chen, K\. Dong, K\. Hu, K\. You, K\. Gao, K\. Guan, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Zhao, L\. Wang, L\. Zhang, L\. Xu, L\. Xia, M\. Zhang, M\. Zhang, M\. Tang, M\. Zhou, M\. Li, M\. Wang, M\. Li, N\. Tian, P\. Huang, P\. Zhang, Q\. Wang, Q\. Chen, Q\. Du, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. J\. Chen, R\. L\. Jin, R\. Chen, S\. Lu, S\. Zhou, S\. Chen, S\. Ye, S\. Wang, S\. Yu, S\. Zhou, S\. Pan, S\. S\. Li, S\. Zhou, S\. Wu, T\. Yun, T\. Pei, T\. Sun, T\. Wang, W\. Zeng, W\. Liu, W\. Liang, W\. Gao, W\. Yu, W\. Zhang, W\. L\. Xiao, W\. An, X\. Liu, X\. Wang, X\. Chen, X\. Nie, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yang, X\. Li, X\. Su, X\. Lin, X\. Q\. Li, X\. Jin, X\. Shen, X\. Chen, X\. Sun, X\. Wang, X\. Song, X\. Zhou, X\. Wang, X\. Shan, Y\. K\. Li, Y\. Q\. Wang, Y\. X\. Wei, Y\. Zhang, Y\. Xu, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Wang, Y\. Yu, Y\. Zhang, Y\. Shi, Y\. Xiong, Y\. He, Y\. Piao, Y\. Wang, Y\. Tan, Y\. Ma, Y\. Liu, Y\. Guo, Y\. Ou, Y\. Wang, Y\. Gong, Y\. Zou, Y\. He, Y\. Xiong, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Y\. X\. Zhu, Y\. Huang, Y\. Li, Y\. Zheng, Y\. Zhu, Y\. Ma, Y\. Tang, Y\. Zha, Y\. Yan, Z\. Z\. Ren, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Ma, Z\. Yan, Z\. Wu, Z\. Gu, Z\. Zhu, Z\. Liu, Z\. Li, Z\. Xie, Z\. Song, Z\. Pan, Z\. Huang, Z\. Xu, Z\. Zhang, and Z\. Zhang \(2025\)DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nat\.645\(8081\),pp\. 633–638\.Cited by:[§C\.5](https://arxiv.org/html/2605.16874#A3.SS5.p1.1),[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p1.1)\.
- C\. He, R\. Luo, Y\. Bai, S\. Hu, Z\. L\. Thai, J\. Shen, J\. Hu, X\. Han, Y\. Huang, Y\. Zhang, J\. Liu, L\. Qi, Z\. Liu, and M\. Sun \(2024\)OlympiadBench: A challenging benchmark for promoting AGI with olympiad\-level bilingual multimodal scientific problems\.InACL \(1\),pp\. 3828–3850\.Cited by:[5th item](https://arxiv.org/html/2605.16874#A1.I1.i5.p1.1),[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§4](https://arxiv.org/html/2605.16874#S4.p2.3.3)\.
- Q\. He, Q\. Ren, S\. Lei, X\. Wang, and Y\. Wang \(2025\)Beyond correctness: confidence\-aware reward modeling for enhancing large language model reasoning\.CoRRabs/2511\.07483\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the MATH dataset\.InNeurIPS Datasets and Benchmarks,Cited by:[2nd item](https://arxiv.org/html/2605.16874#A1.I1.i2.p1.1),[§4](https://arxiv.org/html/2605.16874#S4.p2.3.3)\.
- G\. E\. Hinton, O\. Vinyals, and J\. Dean \(2015\)Distilling the knowledge in a neural network\.CoRRabs/1503\.02531\.Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p2.1),[§2](https://arxiv.org/html/2605.16874#S2.p3.1),[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p1.13)\.
- N\. Ho, L\. Schmid, and S\. Yun \(2023\)Large language models are reasoning teachers\.InACL \(1\),pp\. 14852–14882\.Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, and W\. Chen \(2021\)LoRA: low\-rank adaptation of large language models\.CoRRabs/2106\.09685\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- C\. Huang, T\. Zheng, L\. Huang, J\. Li, H\. Liu, and J\. Huang \(2026\)RelayLLM: efficient reasoning via collaborative decoding\.External Links:2601\.05167,[Link](https://arxiv.org/abs/2601.05167)Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p4.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- N\. Hurley and S\. Rickard \(2009\)Comparing measures of sparsity\.IEEE Trans\. Inf\. Theor\.55\(10\),pp\. 4723–4741\.External Links:ISSN 0018\-9448,[Link](https://doi.org/10.1109/TIT.2009.2027527),[Document](https://dx.doi.org/10.1109/TIT.2009.2027527)Cited by:[§B\.1](https://arxiv.org/html/2605.16874#A2.SS1.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2605.16874#S4.SS1.p3.1.1)\.
- A\. Jaech, A\. Kalai, A\. Lerer, A\. Richardson, A\. El\-Kishky, A\. Low, A\. Helyar, A\. Madry, A\. Beutel, A\. Carney, A\. Iftimie, A\. Karpenko, A\. T\. Passos, A\. Neitz, A\. Prokofiev, A\. Wei, A\. Tam, A\. Bennett, A\. Kumar, A\. Saraiva, A\. Vallone, A\. Duberstein, A\. Kondrich, A\. Mishchenko, A\. Applebaum, A\. Jiang, A\. Nair, B\. Zoph, B\. Ghorbani, B\. Rossen, B\. Sokolowsky, B\. Barak, B\. McGrew, B\. Minaiev, B\. Hao, B\. Baker, B\. Houghton, B\. McKinzie, B\. Eastman, C\. Lugaresi, C\. Bassin, C\. Hudson, C\. M\. Li, C\. de Bourcy, C\. Voss, C\. Shen, C\. Zhang, C\. Koch, C\. Orsinger, C\. Hesse, C\. Fischer, C\. Chan, D\. Roberts, D\. Kappler, D\. Levy, D\. Selsam, D\. Dohan, D\. Farhi, D\. Mely, D\. Robinson, D\. Tsipras, D\. Li, D\. Oprica, E\. Freeman, E\. Zhang, E\. Wong, E\. Proehl, E\. Cheung, E\. Mitchell, E\. Wallace, E\. Ritter, E\. Mays, F\. Wang, F\. P\. Such, F\. Raso, F\. Leoni, F\. Tsimpourlas, F\. Song, F\. von Lohmann, F\. Sulit, G\. Salmon, G\. Parascandolo, G\. Chabot, G\. Zhao, G\. Brockman, G\. Leclerc, H\. Salman, H\. Bao, H\. Sheng, H\. Andrin, H\. Bagherinezhad, H\. Ren, H\. Lightman, H\. W\. Chung, I\. Kivlichan, I\. O’Connell, I\. Osband, I\. C\. Gilaberte, and I\. Akkaya \(2024\)OpenAI o1 system card\.CoRRabs/2412\.16720\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. Amodei \(2020\)Scaling laws for neural language models\.CoRRabs/2001\.08361\.Cited by:[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p1.13)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles,Cited by:[§A\.1](https://arxiv.org/html/2605.16874#A1.SS1.SSS0.Px1.p1.2)\.
- Y\. Leviathan, M\. Kalman, and Y\. Matias \(2023\)Fast inference from transformers via speculative decoding\.InICML,Proceedings of Machine Learning Research, Vol\.202,pp\. 19274–19286\.Cited by:[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p1.13)\.
- C\. Li, T\. Xu, and Y\. Guo \(2025\)Reasoning\-as\-logic\-units: scaling test\-time reasoning in large language models through logic unit alignment\.arXiv preprint arXiv:2502\.07803\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2024\)Let’s verify step by step\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- M\. O\. Lorenz \(1905\)Methods of measuring the concentration of wealth\.Publications of the American Statistical Association9\(70\),pp\. 209–219\.External Links:[Document](https://dx.doi.org/10.2307/2276207),[Link](https://doi.org/10.2307/2276207)Cited by:[§B\.1](https://arxiv.org/html/2605.16874#A2.SS1.SSS0.Px2.p1.2),[§4\.1](https://arxiv.org/html/2605.16874#S4.SS1.p3.1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark \(2023\)Self\-refine: iterative refinement with self\-feedback\.InNeurIPS,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- S\. V\. Marjanovic, A\. Patel, V\. Adlakha, M\. Aghajohari, P\. BehnamGhader, M\. Bhatia, A\. Khandelwal, A\. Kraft, B\. Krojer, X\. H\. Lù, N\. Meade, D\. Shin, A\. Kazemnejad, G\. Kamath, M\. Mosbach, K\. Stanczak, and S\. Reddy \(2025\)DeepSeek\-r1 thoughtology: let’s about LLM reasoning\.CoRRabs/2504\.07128\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- Mathematical Association of America \(2024\)American invitational mathematics examination \(aime\)\.External Links:[Link](https://maa.org/math-competitions/american-invitational-mathematics-examination-aime)Cited by:[4th item](https://arxiv.org/html/2605.16874#A1.I1.i4.p1.1),[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§4](https://arxiv.org/html/2605.16874#S4.p2.3.3)\.
- I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. Stoica \(2025\)RouteLLM: learning to route llms from preference data\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p4.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- OpenAI \(2024\)Introducing o3 and o4\-mini\.Note:[https://openai\.com/index/introducing\-o3\-and\-o4\-mini/](https://openai.com/index/introducing-o3-and-o4-mini/)Accessed: 2025\-12\-23Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. F\. Christiano, J\. Leike, and R\. Lowe \(2022\)Training language models to follow instructions with human feedback\.InNeurIPS,Cited by:[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p1.13)\.
- L\. Pan, M\. Saxon, W\. Xu, D\. Nathani, X\. Wang, and W\. Y\. Wang \(2024\)Automatically correcting large language models:*Surveying the Landscape of Diverse Automated Correction Strategies*\.Trans\. Assoc\. Comput\. Linguistics12,pp\. 484–506\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- J\. Peng, M\. Wang, H\. Cai, Y\. Li, K\. Zhang, S\. Wang, D\. Yin, and X\. Zhao \(2025\)AdaSwitch: adaptive switching generation for knowledge distillation\.CoRRabs/2510\.07842\.Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman \(2024\)GPQA: a graduate\-level google\-proof q&a benchmark\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=Ti67584b98)Cited by:[6th item](https://arxiv.org/html/2605.16874#A1.I1.i6.p1.1),[§C\.6](https://arxiv.org/html/2605.16874#A3.SS6.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- C\. V\. Snell, J\. Lee, K\. Xu, and A\. Kumar \(2025\)Scaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- A\. Subramanian, P\. Tamayo, V\. K\. Mootha, S\. Mukherjee, B\. L\. Ebert, M\. A\. Gillette, A\. Paulovich, S\. L\. Pomeroy, T\. R\. Golub, E\. S\. Lander, and J\. P\. Mesirov \(2005\)Gene set enrichment analysis: a knowledge\-based approach for interpreting genome\-wide expression profiles\.Proceedings of the National Academy of Sciences102\(43\),pp\. 15545–15550\.External Links:[Document](https://dx.doi.org/10.1073/pnas.0506580102),[Link](https://www.pnas.org/doi/abs/10.1073/pnas.0506580102),https://www\.pnas\.org/doi/pdf/10\.1073/pnas\.0506580102Cited by:[§B\.2](https://arxiv.org/html/2605.16874#A2.SS2.p1.1),[§4\.3](https://arxiv.org/html/2605.16874#S4.SS3.p3.4.4.4)\.
- H\. Sun, J\. Wu, H\. Cai, X\. Wei, Y\. Feng, B\. Wang, S\. Wang, Y\. Zhang, and D\. Yin \(2024\)AdaSwitch: adaptive switching between small and large agents for effective cloud\-local collaborative learning\.InEMNLP,pp\. 8052–8062\.Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- L\. Team \(2024\)The llama 3 herd of models\.CoRRabs/2407\.@articleDBLP:journals/corr/abs\-2504\-07128, author = Sara Vera Marjanovic and Arkil Patel and Vaibhav Adlakha and Milad Aghajohari and Parishad BehnamGhader and Mehar Bhatia and Aditi Khandelwal and Austin Kraft and Benno Krojer and Xing Han Lù and Nicholas Meade and Dongchan Shin and Amirhossein Kazemnejad and Gaurav Kamath and Marius Mosbach and Karolina Stanczak and Siva Reddy, title = DeepSeek\-R1 Thoughtology: Let’s about LLM Reasoning, journal = CoRR, volume = abs/2504\.07128, year = 2025 21783\.Cited by:[§C\.5](https://arxiv.org/html/2605.16874#A3.SS5.p1.1)\.
- A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid \(2025\)Steering language models with activation engineering\.External Links:[Link](https://openreview.net/forum?id=2XBPdPIcFK)Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.CoRRabs/1706\.03762\.Cited by:[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p1.13)\.
- C\. Venhoff, I\. Arcuschin, P\. Torr, A\. Conmy, and N\. Nanda \(2025\)Base models know how to reason, thinking models learn when\.CoRRabs/2510\.07364\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p1.1)\.
- S\. Wang, L\. Yu, C\. Gao, C\. Zheng, S\. Liu, R\. Lu, K\. Dang, X\. Chen, J\. Yang, Z\. Zhang, Y\. Liu, A\. Yang, A\. Zhao, Y\. Yue, S\. Song, B\. Yu, G\. Huang, and J\. Lin \(2025a\)Beyond the 80/20 rule: high\-entropy minority tokens drive effective reinforcement learning for LLM reasoning\.CoRRabs/2506\.01939\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p2.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023\)Self\-consistency improves chain of thought reasoning in language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- Y\. Wang, Q\. Yang, Z\. Zeng, L\. Ren, L\. Liu, B\. Peng, H\. Cheng, X\. He, K\. Wang, J\. Gao, W\. Chen, S\. Wang, S\. S\. Du, and Y\. Shen \(2025b\)Reinforcement learning for reasoning in large language models with one training example\.CoRRabs/2504\.20571\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p1.1)\.
- J\. Ward, C\. Lin, C\. Venhoff, and N\. Nanda \(2025\)Reasoning\-finetuning repurposes latent representations in base models\.CoRRabs/2507\.12638\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, E\. H\. Chi, Q\. Le, and D\. Zhou \(2022\)Chain of thought prompting elicits reasoning in large language models\.CoRRabs/2201\.11903\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1),[§2](https://arxiv.org/html/2605.16874#S2.p1.1),[§3\.1](https://arxiv.org/html/2605.16874#S3.SS1.p1.13),[§4\.2](https://arxiv.org/html/2605.16874#S4.SS2.p3.1.1)\.
- X\. Wen, Z\. Liu, S\. Zheng, Z\. Xu, S\. Ye, Z\. Wu, X\. Liang, Y\. Wang, J\. Li, Z\. Miao, J\. Bian, and M\. Yang \(2025\)Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms\.CoRRabs/2506\.14245\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- W\. Xu, R\. Han, Z\. Wang, L\. T\. Le, D\. Madeka, L\. Li, W\. Y\. Wang, R\. Agarwal, C\. Lee, and T\. Pfister \(2025\)Speculative knowledge distillation: bridging the teacher\-student gap through interleaved sampling\.InICLR,Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p3.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§A\.1](https://arxiv.org/html/2605.16874#A1.SS1.SSS0.Px1.p1.2),[§1](https://arxiv.org/html/2605.16874#S1.p3.2.2),[§4](https://arxiv.org/html/2605.16874#S4.p2.3.3)\.
- S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. Griffiths, Y\. Cao, and K\. Narasimhan \(2023a\)Tree of thoughts: deliberate problem solving with large language models\.InNeurIPS,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1),[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao \(2023b\)ReAct: synergizing reasoning and acting in language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- W\. Yuan, R\. Y\. Pang, K\. Cho, X\. Li, S\. Sukhbaatar, J\. Xu, and J\. Weston \(2024\)Self\-rewarding language models\.InICML,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- R\. Zhao, A\. Meterez, S\. M\. Kakade, C\. Pehlevan, S\. Jelassi, and E\. Malach \(2025\)Echo chamber: RL post\-training amplifies behaviors learned in pretraining\.CoRRabs/2504\.07912\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- W\. X\. Zhao, K\. Zhou, J\. Li, T\. Tang, X\. Wang, Y\. Hou, Y\. Min, B\. Zhang, J\. Zhang, Z\. Dong, Y\. Du, C\. Yang, Y\. Chen, Z\. Chen, J\. Jiang, R\. Ren, Y\. Li, X\. Tang, Z\. Liu, P\. Liu, J\. Nie, and J\. Wen \(2023\)A survey of large language models\.CoRRabs/2303\.18223\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.
- T\. Zheng, T\. Xing, Q\. Gu, T\. Liang, X\. Qu, X\. Zhou, Y\. Li, Z\. Wen, C\. Lin, W\. Huang, Q\. Liu, G\. Zhang, and Z\. Ma \(2025\)First return, entropy\-eliciting explore\.CoRRabs/2507\.07017\.Cited by:[§2](https://arxiv.org/html/2605.16874#S2.p2.1)\.
- D\. Zhou, N\. Schärli, L\. Hou, J\. Wei, N\. Scales, X\. Wang, D\. Schuurmans, C\. Cui, O\. Bousquet, Q\. V\. Le, and E\. H\. Chi \(2023\)Least\-to\-most prompting enables complex reasoning in large language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p2.1.1)\.
- A\. Zou, L\. Phan, S\. L\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. Hendrycks \(2023\)Representation engineering: A top\-down approach to AI transparency\.CoRRabs/2310\.01405\.Cited by:[§1](https://arxiv.org/html/2605.16874#S1.p1.1.1)\.

## Appendix AExperimental Setup

### A\.1Implementation Details

#### Compute Environment and Models\.

We implement all experiments in PyTorch111[https://pytorch\.org/](https://pytorch.org/)using the Hugging Face Transformers library, and rely on vLLM\(Kwonet al\.,[2023](https://arxiv.org/html/2605.16874#bib.bib68)\)for efficient, high\-throughput inference\. All evaluations are conducted on a single compute node equipped with an NVIDIA A100 GPU\. To enable a controlled comparison between base generation and reasoning guidance \(with a shared tokenizer and vocabulary\), we use the Qwen3 model family\(Yanget al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib57)\)\. Specifically, we useQwen3\-0\.6B\-Baseas the primary rollout generator \(ℳb\\mathcal\{M\}\_\{b\}\) for its efficiency, andQwen3\-8Bas the stronger reference model \(ℳr\\mathcal\{M\}\_\{r\}\), which provides disagreement scores and takeover tokens when interventions are triggered\.

#### Prompting and Inference Configuration\.

We wrap each problem using the following template and employ sampling\-based decoding with temperature0\.60\.6and top\-pp0\.950\.95\. Unless otherwise specified, we generateK=8K=8samples per problem in parallel\. To accommodate long chain\-of\-thought traces in competition\-level mathematics, we set the maximum generation length to 16,384 tokens\.

PROMPT:Please reason step by step, and put your final answer within \\boxed\{\}\.This is the problem:\{\{prompt\}\}

#### Sampling Protocol and Metrics\.

For each problem, we drawK=8K=8independent samples under the same decoding configuration\. We reportPass@8as whether any of the 8 samples is verified as correct\. ForAccuracy, we report the average correctness across the same 8 samples \(i\.e\., the mean of 8 Bernoulli outcomes\), rather than greedy decoding\.

#### Answer Verification\.

We verify predicted answers using themath\_verifylibrary\. Concretely, we parse both the gold answer and the model response using a joint extraction configuration \(ExprExtractionConfigandLatexExtractionConfig\), and then applyverify\(gold, pred\)to test mathematical equivalence\. If parsing fails or the verifier raises an exception \(e\.g\., due to unsupported SymPy objects\), we conservatively mark the prediction as incorrect\.

#### Intervention Mechanism and Calibration\.

Our disagreement\-gated intervention is grounded in Cross Entropy \(CE\) between the next\-token distributions of the base and reference models\. Intervention thresholds are calibrated on a held\-out set of 500 prompts randomly sampled from the GSM8K training split\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.16874#bib.bib58)\), ensuring no overlap with any test set\. As reported in our results, we evaluate two primary sparsity settings,r=0\.03r=0\.03andr=0\.05r=0\.05, targeting the top 3% and 5% of disagreement spikes, respectively\. To improve stability and reduce spurious triggers, we apply sliding\-window smoothing \(see[Equation5](https://arxiv.org/html/2605.16874#S5.E5)\) with window sizeW=64W=64, intervening only when the current disagreement significantly exceeds the local moving average\.

#### Pseudocode\.

[Algorithm1](https://arxiv.org/html/2605.16874#alg1)provides a self\-contained pseudocode for the disagreement\-gated intervention\. The calibration step \(lines 1–4\) is run once on a held\-out set; the runtime step \(lines 5–14\) is invoked online during decoding\. The procedure does not modify any model parameters or hidden states\.

Algorithm 1Disagreement\-Guided Token InterventionInput:base modelpbp\_\{b\}, reasoning modelprp\_\{r\}, calibration prompts𝒞\\mathcal\{C\}, target spike ratiorr, sliding\-window sizeWW, max lengthTmaxT\_\{\\max\}, promptx0x\_\{0\}\. Output:responsey1:ty\_\{1:t\}\.

1:// — Phase I: Offline calibration \(run once\) —

2:Run base rollouts on

𝒞\\mathcal\{C\}and collect

𝒮=\{st\}\\mathcal\{S\}=\\\{s\_\{t\}\\\}with

st=CE\(pb\(⋅∣xt\),pr\(⋅∣xt\)\)s\_\{t\}=\\mathrm\{CE\}\(p\_\{b\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\),\\,p\_\{r\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\)\)
3:

τ←Q1−r​\(𝒮\)\\tau\\leftarrow Q\_\{1\-r\}\(\\mathcal\{S\}\)//\(1−r\)\(1\-r\)\-quantile threshold

4:

λ←𝔼​\[s​∣s\>​τ\]/𝔼​\[s\]\\lambda\\leftarrow\\mathbb\{E\}\[s\\mid s\>\\tau\]/\\mathbb\{E\}\[s\]// tail\-to\-mean scale

5:// — Phase II: Online decoding for promptx0x\_\{0\}—

6:Initialize

y1:0←∅y\_\{1:0\}\\leftarrow\\emptyset
7:for

t=1,…,Tmaxt=1,\\dots,T\_\{\\max\}do

8:

xt←\(x0,y1:t−1\)x\_\{t\}\\leftarrow\(x\_\{0\},y\_\{1:t\-1\}\)
9:Compute

st=CE\(pb\(⋅∣xt\),pr\(⋅∣xt\)\)s\_\{t\}=\\mathrm\{CE\}\(p\_\{b\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\),\\,p\_\{r\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\)\)
10:

s¯t,W←1min⁡\(t−1,W\)​∑i=1min⁡\(t−1,W\)st−i\\overline\{s\}\_\{t,W\}\\leftarrow\\frac\{1\}\{\\min\(t\-1,\\,W\)\}\\sum\_\{i=1\}^\{\\min\(t\-1,\\,W\)\}s\_\{t\-i\}// skip gate ift=1t=1

11:

gt←𝕀​\[st\>τ∧st\>λ⋅s¯t,W\]g\_\{t\}\\leftarrow\\mathbb\{I\}\[s\_\{t\}\>\\tau\\,\\wedge\\,s\_\{t\}\>\\lambda\\cdot\\overline\{s\}\_\{t,W\}\]
12:if

gt=1g\_\{t\}=1then

13:

yt∼pr\(⋅∣xt\)y\_\{t\}\\sim p\_\{r\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\)// one\-token takeover

14:else

15:

yt∼pb\(⋅∣xt\)y\_\{t\}\\sim p\_\{b\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\)
16:endif

17:if

yt=<EOS\>y\_\{t\}=\\texttt\{<EOS\>\}then

18:break

19:endif

20:endfor

21:return

y1:ty\_\{1:t\}

### A\.2Evaluation Benchmarks

We evaluate the proposed method and baselines on a comprehensive suite of mathematical reasoning benchmarks, categorized into standard tasks \(GSM8K and MATH500\) and competition\-level challenges \(AMC23, AIME24, AIME25, and OlympiadBench\), along with a non\-math scientific QA benchmark \(GPQA\-Diamond\) to assess cross\-domain generalization\.

- •GSM8K\(Grade School Math 8K\)\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.16874#bib.bib58)\): A dataset consisting of 8\.5K high\-quality grade school math word problems\. It requires models to perform multi\-step elementary reasoning using basic arithmetic operations\. We use the standard test set to evaluate the model’s ability to handle linguistic diversity and sequential logic in relatively simple scenarios\.
- •MATH500\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.16874#bib.bib59)\): A representative subset of the challenging MATH dataset, containing 500 problems sampled across seven categories \(e\.g\., algebra, geometry, number theory\) and five difficulty levels\. MATH500 provides a computationally efficient yet rigorous estimate of a model’s performance on broader pre\-university competition mathematics\.
- •AMC23\(AI\-MO,[2023](https://arxiv.org/html/2605.16874#bib.bib63)\): Comprises problems from the 2023 American Mathematics Competitions \(AMC 10 and AMC 12\)\. These exams are the first tier in the United States high school mathematics competition series, requiring creative problem\-solving skills beyond standard curriculum application\.
- •AIME24 & 25\(Mathematical Association of America,[2024](https://arxiv.org/html/2605.16874#bib.bib62)\): The American Invitational Mathematics Examination is the second tier of the competition series, accessible only to top performers in the AMC\. The problems in this set involve integers between 0 and 999 as answers and require significantly deeper reasoning and sustained logical chains compared to the AMC level\.
- •OlympiadBench\(Heet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib61)\): A comprehensive bilingual \(English and Chinese\) benchmark featuring Olympiad\-level mathematics and physics problems\. We focus on the mathematics subset, which sources problems from prestigious competitions such as the International Mathematical Olympiad \(IMO\) and the Chinese College Entrance Examination \(GaoKao\)\. This represents the upper bound of difficulty in our evaluation suite\.
- •GPQA\-Diamond\(Reinet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib67)\): A benchmark of graduate\-level scientific question answering across biology, physics, and chemistry, written and validated by domain experts\. The Diamond subset contains the most difficult “Google\-proof” questions, requiring deep domain knowledge rather than retrieval\. We use a 50\-question subset to evaluate generalization beyond mathematical reasoning, with results reported in Appendix[C\.6](https://arxiv.org/html/2605.16874#A3.SS6)\.

## Appendix BAnalysis Details

### B\.1Lorenz Curve and Gini Coefficient for Token\-Level Discrepancy

This section provides the formal definitions of the Lorenz curve and the Gini coefficient used in Section[4\.1](https://arxiv.org/html/2605.16874#S4.SS1), as well as implementation details\.

#### Token\-level disagreement scores\.

For each response, we obtain a sequence of nonnegative per\-token scores\{st\}t=1N\\\{s\_\{t\}\\\}\_\{t=1\}^\{N\}\(e\.g\., cross\-entropy discrepancy between base and reasoning next\-token distributions\)\. We sort these scores in non\-decreasing order:s\(1\)≤s\(2\)≤⋯≤s\(N\)s\_\{\(1\)\}\\leq s\_\{\(2\)\}\\leq\\cdots\\leq s\_\{\(N\)\}\.

#### Lorenz curve\.

\(Lorenz,[1905](https://arxiv.org/html/2605.16874#bib.bib54)\)The Lorenz curve is defined by points\(xk,yk\)\(x\_\{k\},y\_\{k\}\)fork=1,…,Nk=1,\\ldots,N, where

xk=kN,yk=∑i=1ks\(i\)∑j=1Ns\(j\)\.x\_\{k\}=\\frac\{k\}\{N\},\\qquad y\_\{k\}=\\frac\{\\sum\_\{i=1\}^\{k\}s\_\{\(i\)\}\}\{\\sum\_\{j=1\}^\{N\}s\_\{\(j\)\}\}\.\(7\)Intuitively,yky\_\{k\}measures what fraction of total discrepancy mass is contributed by the lowest\-scoringkktokens\. A curve close to the equality liney=xy=xindicates a nearly uniform contribution across tokens, while a strongly bowed curve indicates high concentration\.

#### Gini coefficient\.

\(Hurley and Rickard,[2009](https://arxiv.org/html/2605.16874#bib.bib55)\)We summarize concentration by the Gini coefficientGG, which equals twice the area between the Lorenz curve and the equality line:

G=1−2​∫01L​\(x\)​𝑑x\.G=1\-2\\int\_\{0\}^\{1\}L\(x\)\\,dx\.\(8\)For a discrete set of sorted scores, we compute

G=∑i=1N\(2​i−N−1\)​s\(i\)N​∑j=1Ns\(j\),G=\\frac\{\\sum\_\{i=1\}^\{N\}\(2i\-N\-1\)\\,s\_\{\(i\)\}\}\{N\\sum\_\{j=1\}^\{N\}s\_\{\(j\)\}\},\(9\)which ranges from0\(perfectly uniform mass\) to11\(all mass on a single token\)\.

#### Aggregation across samples and datasets\.

We compute the Lorenz curve and Gini coefficient per sample using that sample’s token sequence, and then report the dataset\-level mean \(and optionally standard deviation / confidence intervals\) across samples\. The overall value reported in Section[4\.1](https://arxiv.org/html/2605.16874#S4.SS1)is the average of dataset\-level means across the six benchmarks\.

### B\.2Token Enrichment Analysis: Definition and Token Categorization

Section[4\.3](https://arxiv.org/html/2605.16874#S4.SS3)and Section[5\.3](https://arxiv.org/html/2605.16874#S5.SS3)report two complementary token enrichment analyses\(Subramanianet al\.,[2005](https://arxiv.org/html/2605.16874#bib.bib56)\)using the same token categorization rules\. The first \( peak\-based enrichment\) is computed on base\-model rollouts, measuring whether tokens at high\-entropy or high cross\-model discrepancy peaks are planning\-heavy\. The second \( takeover\-based enrichment\) is computed on guided rollouts, measuring whether the tokens actually generated by the reasoning model during takeover are enriched for planning markers\. This appendix provides the shared enrichment definition and the token categorization procedure used in both analyses\.

#### Enrichment ratio\.

LetCCbe a token category \(e\.g\.,Planning\)\. We compute an enrichment ratio

E​\(C\)=P​\(t∈C∣t∈𝒯intervsem\)P​\(t∈C∣t∈𝒯globalsem\),E\(C\)\\;=\\;\\frac\{P\(t\\in C\\mid t\\in\\mathcal\{T\}\_\{\\text\{interv\}\}^\{\\text\{sem\}\}\)\}\{P\(t\\in C\\mid t\\in\\mathcal\{T\}\_\{\\text\{global\}\}^\{\\text\{sem\}\}\)\},\(10\)where𝒯intervsem\\mathcal\{T\}\_\{\\text\{interv\}\}^\{\\text\{sem\}\}denotes the set of intervened tokens \(i\.e\., tokens generated by the reasoning model during takeover\) that are classifiable into one of our semantic categories, and𝒯globalsem\\mathcal\{T\}\_\{\\text\{global\}\}^\{\\text\{sem\}\}denotes the set of classifiable tokens in the standard base rollout\. Importantly, tokens that do not match any semantic rule \(e\.g\., punctuation, stopwords, generic content words\) are excluded from both the numerator/denominator counts and the normalizing totals, so the enrichment is computed within the planning/execution semantic subset\.

#### Token categorization \(heuristic\)\.

We classify each decoded tokenttinto one of two categories: \(i\)Planning/Decisionand \(ii\)Execution/Arithmetic\. Classification uses a lightweight heuristic based on exact keyword matching \(for planning\) and regular\-expression/symbol matching \(for execution\)\. All comparisons are case\-insensitive after stripping surrounding whitespace\.

#### Planning / Decision Tokens\.

This category includes words associated with reasoning structure, strategy selection, logical deduction, and self\-reflection\. We constructed a vocabulary𝒱plan\\mathcal\{V\}\_\{\\text\{plan\}\}containing several keywords across the following sub\-categories:

- •Step Indicators:first,next,then,finally,step\.
- •Logical Connectives:therefore,so,since,because,thus,hence,implies\.
- •Strategy & Action Verbs:let,assume,suppose,consider,calculate,solve,check,verify,notice,determine,choose,analyze\.
- •Contrast & Correction:but,however,actually,wait,instead,alternatively\.
- •Thinking Markers:thinking,strategy,idea,plan,break down,approach,method\.
- •Conditionals & Wh\-Words:if,unless,whether,why,how,what\.

#### Execution / Arithmetic Tokens\.

This category captures the mechanical execution of mathematical operations and symbolic manipulation\. We identified these tokens using regular expressions:

- •Numbers:Integers and floating\-point numbers \(e\.g\.,100,3\.14\)\.
- •Operators:Mathematical symbols including\+,\-,\*,/,=,<,\>,^\.
- •Structure:LaTeX formatting symbols and brackets \(e\.g\.,$,\{,\},\(,\)\)\.
- •Variables:Single\-letter variables common in math problems \(e\.g\.,x,y,n\) when appearing in a mathematical context\.

#### Target vs\. reference token sets\.

Each enrichment analysis compares a target token set𝒯target\\mathcal\{T\}\_\{\\text\{target\}\}to a reference token set𝒯ref\\mathcal\{T\}\_\{\\text\{ref\}\}, and computes category proportions within the classifiable semantic subset\. In the peak\-based enrichment \(Section[4\.3](https://arxiv.org/html/2605.16874#S4.SS3)\),𝒯target\\mathcal\{T\}\_\{\\text\{target\}\}is the top\-p%p\\%tokens in a base rollout ranked by either base entropyHb​\(t\)H\_\{b\}\(t\)or cross\-model discrepancysts\_\{t\}, and𝒯ref\\mathcal\{T\}\_\{\\text\{ref\}\}is the full base\-rollout token stream\. In the takeover\-based enrichment \(Section[5\.3](https://arxiv.org/html/2605.16874#S5.SS3)\),𝒯target\\mathcal\{T\}\_\{\\text\{target\}\}consists of takeover tokens extracted from guided rollouts using the per\-step provenance fieldchosen\_from\(sources beginning withteacher\), and𝒯ref\\mathcal\{T\}\_\{\\text\{ref\}\}is the corresponding base\-rollout token stream\.

#### Interpretation and limitations\.

This enrichment analysis is intended as a coarse, interpretable proxy for identifying planning markers\. While it does not capture all planning behavior \(e\.g\., implicit planning without explicit markers\), the strong enrichment of planning tokens in the intervention set supports the claim that our takeover mechanism disproportionately targets planning\-related linguistic cues rather than execution\-heavy arithmetic tokens\.

### B\.3Per\-dataset planning enrichment at entropy/disagreement peaks

Table 6:Planning enrichment at peaks across datasets \(top\-1%\)\.For each dataset, we report the planning share in the base\-rollout global token stream and among the top\-1% tokens selected by either cross\-model divergencests\_\{t\}\(CE\) or base entropyHb​\(t\)H\_\{b\}\(t\), together with the enrichment ratio\.Table[1](https://arxiv.org/html/2605.16874#S4.T1)aggregates planning enrichment across all datasets\. Here we provide the per\-dataset breakdown to assess consistency and heterogeneity across benchmarks\. Table[6](https://arxiv.org/html/2605.16874#A2.T6)shows that planning enrichment at top\-1% peaks holds across all datasets for both ranking signals, with particularly large effects on harder benchmarks \(e\.g\., AMC23 and AIME25\)\.

## Appendix CAdditional Results

### C\.1Reverse\-KL as an Alternative Disagreement Signal

Our method is not tied to Cross Entropy as the disagreement metric\. We replicate the gated takeover using reverse KL, i\.e\.,st=KL​\(pb∥pr\)s\_\{t\}=\\mathrm\{KL\}\(p\_\{b\}\\\|p\_\{r\}\), while keeping the same calibration protocol \(global spike ratiorrand sliding\-window smoothing withW=64W=64\) and the same evaluation setup\. Table[7](https://arxiv.org/html/2605.16874#A3.T7)reports the results and shows that reverse\-KL yields strong recovery under comparable realized budgets, confirming that divergence\-guided takeover is not specific to a single discrepancy function\.

Table 7:Experimental results on mathematical reasoning benchmarks \(Accuracy / Pass@8\)\. The Recovery column quantifies how much the intervention recovers reasoning capabilities, calculated as\(PG​u​i​d​e​d−PB​a​s​e\)/\(PT​h​i​n​k​i​n​g−PB​a​s​e\)\(P\_\{Guided\}\-P\_\{Base\}\)/\(P\_\{Thinking\}\-P\_\{Base\}\), wherePPdenotes Pass@8 \(consistent with Table[3](https://arxiv.org/html/2605.16874#S4.T3)\)\.Table 8:Recovery under different disagreement metrics \(Qwen3\-0\.6B\)\.We report Recovery computed on Accuracy and Pass@8 respectively\.![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/appendix/replacement_ratio/qwen3_0.6b_base_vs_qwen3_8b_sliding_window_64_ce_st_global_guided_0.03_replacement_ratio_violin.png)\(a\)CE gate \(r=0\.03r=0\.03\)
![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/appendix/replacement_ratio/qwen3_0.6b_base_vs_qwen3_8b_sliding_window_64_ce_st_global_guided_0.05_replacement_ratio_violin.png)\(b\)CE gate \(r=0\.05r=0\.05\)
![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/appendix/replacement_ratio/qwen3_0.6b_base_vs_qwen3_8b_sliding_window_64_guided_0.03_replacement_ratio_violin.png)\(c\)rKL gate \(r=0\.03r=0\.03\)
![Refer to caption](https://arxiv.org/html/2605.16874v1/figures/appendix/replacement_ratio/qwen3_0.6b_base_vs_qwen3_8b_sliding_window_64_guided_0.05_replacement_ratio_violin.png)\(d\)rKL gate \(r=0\.05r=0\.05\)

Figure 7:Stability of realized replacement budget\.Violin plots of per\-problem replacement rateρi\\rho\_\{i\}across benchmarks for Qwen3\-0\.6B under \(top\) Cross\-Entropy gating and \(bottom\) reverse\-KL gating, each shown at two calibrated spike ratios \(r=0\.03r=0\.03andr=0\.05r=0\.05\)\. Concentrated distributions indicate that the realized intervention budget is stable across problems rather than dominated by a small subset of outliers\.To better understand metric\-dependent behavior, Table[8](https://arxiv.org/html/2605.16874#A3.T8)summarizes Recovery computed on Accuracy and on Pass@8, respectively\. Empirically, reverse\-KL tends to yield higher Accuracy\-based recovery, whereas Cross Entropy yields stronger Pass@8\-based recovery\. A plausible explanation is thatKL​\(pb∥pr\)\\mathrm\{KL\}\(p\_\{b\}\\\|p\_\{r\}\)emphasizes cases where the base model places overly concentrated mass on an incorrect token \(confidence miscalibration\), while Cross Entropy more directly reflects distributional mismatch and can better improve the probability that at least one sampled trajectory reaches the correct solution\.

### C\.2Stability of the Realized Replacement Budget

We examine the per\-problem realized replacement rateρi=1Ti​∑t=1Tigt\\rho\_\{i\}=\\frac\{1\}\{T\_\{i\}\}\\sum\_\{t=1\}^\{T\_\{i\}\}g\_\{t\}\. Figure[7](https://arxiv.org/html/2605.16874#A3.F7)visualizes the distribution ofρi\\rho\_\{i\}across benchmarks for Qwen3\-0\.6B under Cross\-Entropy and reverse\-KL gating, each at two calibrated spike ratios \(r=0\.03r=0\.03andr=0\.05r=0\.05\)\.

Two observations are consistent across datasets\. First, the distributions are concentrated with small variance, indicating that the realized intervention budget is stable across problems rather than dominated by a small subset of outliers\. Second, for the samerr, Cross Entropy consistently yields a lower realized replacement rate than reverse\-KL, suggesting that CE produces a more selective gate under our calibration procedure\. This motivates reporting budgets using realizedρ\\rho\(and usingρ¯\\bar\{\\rho\}in tables\) when comparing metrics and settings\.

### C\.3Entropy\-Triggered Practical Variant

The disagreement gate in[Section5](https://arxiv.org/html/2605.16874#S5)queries bothpbp\_\{b\}andprp\_\{r\}at every step, which is fine for diagnosis but wasteful at deployment\. Since disagreement spikes overlap heavily with base\-entropy spikes \([Section4\.3](https://arxiv.org/html/2605.16874#S4.SS3), Figure[2\(c\)](https://arxiv.org/html/2605.16874#S2.F2.sf3)\), a cheaper alternative is to trigger on the base entropyHb\(t\)=H\(pb\(⋅∣xt\)\)H\_\{b\}\(t\)=H\(p\_\{b\}\(\\cdot\\\!\\mid\\\!x\_\{t\}\)\)alone, applying the same calibration and gating recipe withsts\_\{t\}replaced byHb​\(t\)H\_\{b\}\(t\)\. As shown in Table[9](https://arxiv.org/html/2605.16874#A3.T9), this cheaper trigger already recovers most of the strong\-model Pass@8 advantage: atρ¯≈0\.08\\bar\{\\rho\}\\\!\\approx\\\!0\.08, the entropy\-guided variant reaches65\.8/87\.265\.8/87\.2\(Acc/Pass@8\), essentially matching Qwen3\-8B’s Pass@8 of87\.387\.3while querying the stronger model at only∼\\sim8% of steps\.

Table 9:Entropy\-triggered practical variant \(Qwen3\-0\.6B\)\.Replacing the cross\-model disagreement trigger with base\-model entropyHb​\(t\)H\_\{b\}\(t\)already recovers most of the strong\-model Pass@8 advantage while only requiringpbp\_\{b\}at every step\.
### C\.4Matched\-Budget Baselines: Random and Early\-Only

We test whether the gains from disagreement\-guided intervention are driven specifically by the choice ofwhichtokens are replaced, by comparing against two budget\-matched baselines: \(i\) arandombaseline that uniformly samples replacement positions to hit a target budgetρ\\rho, and \(ii\) anearly\-onlybaseline that, given the reasoning model’s output of lengthLrL\_\{r\}, replaces the firstρ⋅Lr\\rho\\\!\\cdot\\\!L\_\{r\}tokens of the rollout with reasoning\-model tokens and then lets the base model continue\. The early\-only baseline is motivated by our Finding 2 \([Section4\.2](https://arxiv.org/html/2605.16874#S4.SS2)\) that disagreement spikes concentrate at early positions: it isolates whetherposition aloneis sufficient\.

Table[10](https://arxiv.org/html/2605.16874#A3.T10)summarizes the comparison on Qwen3\-0\.6B \(averaged over the six math benchmarks\)\. Two findings stand out:

Random replacement is consistently dominated by guided\.At an aggressive25%25\\%random replacement, average performance only reaches26\.4/55\.226\.4/55\.2\(Acc/Pass@8\), whereas the disagreement\-guided gate at onlyρ¯≈0\.04\\bar\{\\rho\}\\\!\\approx\\\!0\.04already reaches29\.1/61\.429\.1/61\.4\. This confirms that gains do not come merely from inserting stronger\-model tokens at arbitrary positions\.

Position alone is not sufficient\.The early\-only baseline does improve over the base, consistent with our finding that critical tokens concentrate early\. However, even atρ¯≈0\.25\\bar\{\\rho\}\\\!\\approx\\\!0\.25early\-only reaches only25\.7/58\.425\.7/58\.4, which is still worse than guided atρ¯≈0\.04\\bar\{\\rho\}\\\!\\approx\\\!0\.04\(29\.1/61\.429\.1/61\.4\)\. This shows that within the early region, only the disagreement / uncertainty spikes are truly trajectory\-setting; blindly replacing all early tokens wastes budget on non\-critical positions\.

Together, these baselines support the interpretation that the gain comes from selecting positions that are simultaneouslyearlyandinformative, i\.e\., positions where the base and reasoning models disagree most under base\-model uncertainty\.

Table 10:Matched\-budget baselines on Qwen3\-0\.6B\.Average Accuracy / Pass@8 over the six math reasoning benchmarks\. Random samples replacement positions uniformly under a target budgetρ\\rho\. Early\-only replaces the firstρ⋅Lr\\rho\\\!\\cdot\\\!L\_\{r\}tokens \(whereLrL\_\{r\}is the reasoning\-model rollout length\) and then lets the base model continue\. Disagreement\-guided intervention atρ¯≈0\.04\\bar\{\\rho\}\\\!\\approx\\\!0\.04already outperforms both baselines at25%25\\%budget\.
### C\.5Cross\-Family Generalization on LLaMA

To test whether the sparse\-control finding transfers beyond the Qwen3 family, we evaluate the disagreement\-guided intervention on a LLaMA\-based base/guide pair: the base model is LLaMA\-3\.1\-8B\(Team,[2024](https://arxiv.org/html/2605.16874#bib.bib2)\)and the reasoning guide is DeepSeek\-R1\-Distill\-Llama\-8B\(Guoet al\.,[2025](https://arxiv.org/html/2605.16874#bib.bib18)\)\. Calibration follows the same procedure as in the main experiments \(held\-out GSM8K\-style prompts, sliding windowW=64W=64\)\.

Table[11](https://arxiv.org/html/2605.16874#A3.T11)reports the average performance over the six math benchmarks\. The same monotonic recovery pattern holds: withρ¯≈0\.20\\bar\{\\rho\}\\\!\\approx\\\!0\.20, guided intervention recovers∼\\sim91% of the gap \(in Pass@8\) between the base model and the reasoning guide, mirroring the Qwen3 results in Table[3](https://arxiv.org/html/2605.16874#S4.T3)\. This indicates that the disagreement\-spike phenomenon and the resulting recovery curve are not specific to a single model family or training pipeline\.

Table 11:Cross\-family generalization on the LLaMA pair \(Accuracy / Pass@8\)\.Base model: LLaMA\-3\.1\-8B; reasoning guide: DeepSeek\-R1\-Distill\-Llama\-8B\. The Recovery column quantifies how much the intervention recovers reasoning capabilities, calculated as\(PG​u​i​d​e​d−PB​a​s​e\)/\(PG​u​i​d​e−PB​a​s​e\)\(P\_\{Guided\}\-P\_\{Base\}\)/\(P\_\{Guide\}\-P\_\{Base\}\), wherePPdenotes Pass@8 \(consistent with Table[3](https://arxiv.org/html/2605.16874#S4.T3)\)\.
### C\.6Cross\-Domain Generalization to Non\-math Scientific QA

To assess whether the sparse\-intervention effect transfers beyond mathematical reasoning, we evaluate on GPQA\-Diamond\(Reinet al\.,[2024](https://arxiv.org/html/2605.16874#bib.bib67)\), a benchmark of graduate\-level scientific question answering across biology, physics, and chemistry\. We use a 50\-question subset and keep the same Qwen3\-0\.6B\-Base / Qwen3\-8B \(Thinking\) pair as in our main experiments\.

Table[12](https://arxiv.org/html/2605.16874#A3.T12)shows two patterns that differ from the math setting\. First, same\-size reasoning post\-training provides little advantage here: Qwen3\-0\.6B \(Thinking\) matches the base in Accuracy \(23\.023\.0vs\.22\.522\.5\) andunderperformsit in Pass@8 \(68\.068\.0vs\.78\.078\.0\); even the 8B reasoning model does not improve Pass@8 over the 0\.6B base \(both78\.078\.0\)\. Second, sparse intervention nonetheless yields clear gains: atρ¯≈0\.05\\bar\{\\rho\}\\\!\\approx\\\!0\.05, guided decoding reaches37\.1/88\.037\.1\\,/\\,88\.0, improving the base by\+14\.6\+14\.6Accuracy and\+10\.0\+10\.0Pass@8, and exceeding the 8B reasoning model’s Pass@8 \(88\.088\.0vs\.78\.078\.0\)\. The remaining Accuracy gap to 8B \(37\.137\.1vs\.63\.263\.2\) is larger than on math benchmarks, suggesting that fully closing the non\-math reasoning gap may require denser intervention\. Together with the cross\-family LLaMA results in Appendix[C\.5](https://arxiv.org/html/2605.16874#A3.SS5), this indicates the sparse\-control phenomenon is not specific to a single model family or to mathematical reasoning\.

Table 12:Cross\-domain generalization on GPQA\-Diamond \(50\-question subset; Accuracy / Pass@8\)\.Base: Qwen3\-0\.6B\-Base; reasoning guide: Qwen3\-8B \(Thinking\)\. Same\-size reasoning post\-training provides little Pass@8 advantage in this domain, while sparse guided intervention improves both Accuracy and Pass@8 over the base and exceeds the 8B reasoning model’s Pass@8\.
### C\.7Calibration Threshold Stability across Datasets

A natural concern with calibrated gating is whether the calibrated thresholdsτ\\tauand the tail\-to\-mean factorλ\\lambda\([Equation4](https://arxiv.org/html/2605.16874#S5.E4)\) are dataset\-specific or transferable\. To assess this, we re\-calibrate the gate on each of the six benchmarks separately and compare the resulting\(τ,λ\)\(\\tau,\\lambda\)at three target spike ratiosr∈\{0\.01,0\.03,0\.05\}r\\in\\\{0\.01,0\.03,0\.05\\\}\. Table[13](https://arxiv.org/html/2605.16874#A3.T13)shows that: \(i\) for a fixedrr, the calibratedτ\\tauvalues stay within a narrow range across calibration sets \(e\.g\., forr=0\.01r=0\.01,τ∈\[3\.18,4\.12\]\\tau\\in\[3\.18,4\.12\]\), and \(ii\) bothτ\\tauandλ\\lambdafollow the same monotonic decreasing trend inrracross all datasets\. This indicates that the disagreement tail used by our gate is reasonably stable, rather than strongly dataset\-specific, which is consistent with the calibration recipe used throughout the main paper \(a single calibration set is used for all evaluations\)\.

Table 13:Calibration threshold stability across datasets\.For each calibration set we reportτ/λ\\tau\\,/\\,\\lambdaat three target spike ratiosr∈\{0\.01,0\.03,0\.05\}r\\in\\\{0\.01,0\.03,0\.05\\\}\. Values stay within a narrow range across datasets at fixedrr, and follow the same monotonic trend withrr, suggesting that the calibrated tail of the disagreement distribution is stable rather than dataset\-specific\.
### C\.8Intervention Harm Analysis \(Pass@8 Flip Statistics\)

A natural concern is whether the intervention, while improving aggregate performance, can also degrade specific samples that the base model would have answered correctly without help\. To quantify this, we examine sample\-level Pass@8 label flips between the base model and the guided model on Qwen3\-0\.6B \(with Qwen3\-8B as the guide\), atρ¯≈0\.13\\bar\{\\rho\}\\\!\\approx\\\!0\.13\. For each problem we compare whetheranyof the 8 sampled trajectories under the base setting and under the guided setting are correct, and we tally three outcomes:Error→\\toCorrect\(the guided setting now has at least one correct sample, while the base did not\),Correct→\\toError\(the base had at least one correct sample, but the guided does not\), and unchanged \(both correct or both incorrect; not shown in the table\)\.

Table 14:Sample\-level Pass@8 flips on Qwen3\-0\.6B \(guided vs\. base,ρ¯≈0\.13\\bar\{\\rho\}\\\!\\approx\\\!0\.13\)\.Across400400examples, the intervention overwhelmingly converts errors to successes \(152 flips\), while only 3 cases flip in the opposite direction\.Table[14](https://arxiv.org/html/2605.16874#A3.T14)shows that the dominant effect of intervention is error correction: out of 400 examples,152152examples transition from incorrect to correct under guided decoding, while only33transition in the opposite direction \(a∼\\sim50:1 ratio\)\. The three Correct→\\toError cases are spread across AMC23, GSM8K, and OlympiadBench, and we observe no systematic pattern: each is a case where the base model had a borderline\-correct trajectory that the intervention diverted toward a different \(incorrect\) chain\. This tail behavior is consistent with the fact that intervention edits the trajectory at a small set of high\-uncertainty decision points: in rare cases, redirecting these decisions can move a borderline\-correct rollout off\-track\.

## Appendix DCase Study

We visualize the divergence\-guided takeover on this example\. Figure[8](https://arxiv.org/html/2605.16874#A4.F8)shows the prompt, and Figure[9](https://arxiv.org/html/2605.16874#A4.F9)provides the full guided rollout\.

#### What gets replaced\.

A salient pattern is that most replaced tokens occur in planning\- and control\-related regions of the response: the brief outline of the solution, the choice of intermediate variables, and the decision of which quantities to compute next\. In contrast, once the correct plan is established, the subsequent arithmetic manipulations and execution steps are largely produced by the base model itself, with few \(or no\) additional takeovers\.

#### Interpretation\.

This example supports the view that the base–reasoning gap is not primarily due to missing low\-level execution capability\. Rather, failures concentrate at a small number of trajectory\-defining decisions—what to do next and how to structure the computation—where the base model tends to commit to an unproductive path\. By selectively overriding these planning tokens, the takeover effectively supplies a lightweight “controller” that keeps the rollout on\-track, while leaving the base model to carry out the bulk of the calculation\.

#### Connection to our main findings\.

Consistent with our sparsity analysis, the interventions are localized and concentrated, yet have outsized downstream effects: correcting a few high\-disagreement planning choices is sufficient to steer the remainder of the trajectory toward a correct solution\.

Problem:In the first half of a soccer match, team A scores 4 goals while team B scores 2 goals fewer than team A\. In the second half, team A scores 1/4 of the number of goals scored by team B, which scores 4 times the number of goals it scored in the first half\. What’s the total number of goals scored in the match?

Figure 8:Case study prompt\.The following figure visualize the token\-level takeover trajectory for this example\.![Refer to caption](https://arxiv.org/html/2605.16874v1/x5.png)Figure 9:Case study rollout\.Highlighted tokens denote reasoning model replacements triggered by disagreement spikes\.