REWIND：用于测试时推理的不确定性引导轨迹修复

# REWIND: Uncertainty-Guided Trajectory Repair for Test-Time Reasoning
Source: [https://arxiv.org/html/2610.19561](https://arxiv.org/html/2610.19561)
Mengyu WangMin ZhouAffiliation:University of Edinburgh Huawei\*Corresponding author:[menglinyang@hkust\-gz\.edu\.cn](mailto:[email protected])Menglin YangAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\)

###### Abstract

An early mistake in the generated reasoning can constrain all subsequent reasoning\. Repeated\-generation methods, like MCMC\-based approaches, attempt to improve output quality by sampling multiple generations, but their effectiveness can be limited by randomly selecting where to regenerate\. On MATH500 with Qwen2\.5\-Math\-7B, we find that incorrect responses have substantially higher mean token entropy than correct responses, with an average within\-question relative increase of approximately 112%, suggesting that entropy can signal reasoning errors\. Motivated by this finding, we introduce REWIND, a training\-free controller that uses entropy dynamics to propose targeted suffix regenerations\. Across three model families and five benchmarks,Rewindoutperforms all baselines, with gains of 0\.51–10\.00 percentage points over the strongest baseline\. These results demonstrate the effectiveness of targeted, uncertainty\-guided trajectory repair across models and tasks\. Our code is available at[https://anonymous\.4open\.science/r/rewind\-D310](https://anonymous.4open.science/r/rewind-D310)\.

## 1Introduction

Large language models \(LLMs\) can solve multi\-step reasoning problems, but an early incorrect premise or calculation can constrain the rest of a generated solution[Wei et al\. \(2022\)](https://arxiv.org/html/2610.19561#bib.bib28);[Dziri et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib8)\. To mitigate such issues caused by sequential decoding, explicit revision methods discard part of that context and give the model another opportunity to continue from an earlier prefix\. However, the central question of revision is how to choose a useful cut point without a trained verifier or external feedback\.

Recent works have proposed many different approaches to respond to this question\. Iterative self\-feedback and sampling\-based refinement provide ways to revise a draft[Madaan et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib20);[Karan and Du \(2025\)](https://arxiv.org/html/2610.19561#bib.bib16)\. MCMC\-based Power Sampling chooses a random cut point to regenerate the suffix and then accepts or rejects the proposal using model likelihoods\. Its cut\-point policy is independent of local confidence and uncertainty\. A cut after a mistaken premise retains it, whereas a cut far before it can replace substantial correct work\. Since a filter acts only after re\-generation, it can accept or reject a completed proposal but cannot guide the re\-generation process itself\. Therefore, revision\-site selection becomes a practical bottleneck, motivating trajectory diagnostics that direct proposals toward candidate regions of instability\.

Although continuation sampling is stochastic, its final acceptance is based on the generation statistics, since confidence and uncertainty statistics can contain information about downstream failure[Kadavath et al\. \(2022\)](https://arxiv.org/html/2610.19561#bib.bib15);[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.19561#bib.bib32)\. Motivated by this, there may exist signals during the generation process indicating potential re\-generation points\. Shannon entropy summarizes the full next\-token distribution, capturing competition among alternatives, and can be measured from model logits without an additional verifier or reference answers[Shannon \(1948\)](https://arxiv.org/html/2610.19561#bib.bib24)\. We examine this signal through 20 ordinary Qwen2\.5\-Math\-7B generations per MATH500 question\. Among the 463 questions with both correct and incorrect responses under the original boxed\-answer evaluator, we find that the mean token entropy of incorrect responses is approximately112% higherthan that of correct responses \(Section[3\.2](https://arxiv.org/html/2610.19561#S3.SS2)\)\. Moreover, on questions with fewer incorrect responses, those errors exhibit higher entropy, suggesting that entropy can identify mistakes that are more likely to be corrected through regeneration\. This token\-level generation\-error association motivates uncertainty\-conditioned revision\.

Therefore, we propose REWIND \(Reasoning Error Windback via INtrinsic Detection\), a training\-free, entropy\-conditioned suffix\-proposal controller\. A high\-entropy peak can occur after an earlier unstable choice has influenced later steps\. The controller therefore searches immediately before the peak for the largest discrete entropy increase, and aligns the cut with a natural delimiter to regenerate the suffix\. Then an MH\-style score filter decides whether to retain the proposal\. This makes the re\-generation more targeted to cover the potential mistakes rather than random selection, which may miss critical errors\.

REWIND consistently improves reasoning performance across three model families and five benchmarks, achieving the highest accuracy among the baselines\. It yields gains of 0\.51–10\.00 percentage points, with the largest improvement on AIME2025 using DeepSeek\-R1\-Distill\-Qwen\-1\.5B \(DeepSeek\-1\.5B\), from 16\.67% to 26\.67%\. Controlled ablations show that entropy\-gradient backtracking and boundary alignment both improve the performance\. Further evaluations demonstrate that the improvements transfer to few\-shot prompting and code generation and remain stable across random seeds\. Together, these results establish REWIND as an effective approach to targeted test\-time trajectory repair\.

In summary, our contributions are as follows:

- 1\.An empirical entropy–correctness association\.Through repeated generations on MATH500, we find a clear association between a question’s error frequency and the token entropy of its responses, indicating that entropy provides a useful signal of potential generation errors\.
- 2\.A training\-free trajectory\-repair method\.We introduceRewind, which locates entropy\-gradient spikes before uncertainty peaks, aligns revision points with natural delimiters, and filters regenerated suffixes using an MH\-style score, without additional training or an external verifier\.
- 3\.Comprehensive empirical evaluation\.Experiments across three model families and five benchmarks show thatRewindachieves the best performance among all baselines, improving over random\-proposal MCMC by 0\.51–10\.00 percentage points\. Controlled ablations and evaluations on few\-shot reasoning and code generation further support the method’s design and its transfer across tasks\.

## 2Related Work

#### Training\-free test\-time scaling\.

Existing inference\-time reasoning methods improve model outputs through either candidate exploration or iterative refinement\. Self\-consistency aggregates independently sampled solutions[Wang et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib27), while Tree of Thoughts and MCTS\-based approaches explicitly explore alternative reasoning branches[Yao et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib31);[Zhou et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib33);[Liu \(2025\)](https://arxiv.org/html/2610.19561#bib.bib19)\. In contrast, iterative refinement methods revise existing outputs rather than exploring independent reasoning paths\. Self\-Refine uses model\-generated textual feedback to guide revisions[Madaan et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib20), although intrinsic self\-correction can be unreliable without informative feedback[Huang et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib14)\. Sampling\-based methods offer another refinement mechanism\. Early work such as CGMH applies Metropolis–Hastings sampling to constrained sentence generation through word\-level edits[Miao et al\. \(2019\)](https://arxiv.org/html/2610.19561#bib.bib22), while block MH methods propose whole\-sequence rewrites using a language model[Forristal et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib10)\. More recently, Power Sampling employs MH\-based suffix proposals to favor sequences under a sharpened model distribution[Karan and Du \(2025\)](https://arxiv.org/html/2610.19561#bib.bib16)\. Building on suffix\-based refinement,Rewindshifts the focus from how candidate revisions are generated to where they should be applied, enabling more targeted refinement\.

#### Uncertainty\-guided control\.

Model\-derived signals have been used to guide inference at different levels\. At the token level, DoLa contrasts predictions across layers, while typical decoding constrains token information content[Chuang et al\. \(2024\)](https://arxiv.org/html/2610.19561#bib.bib6);[Meister et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib21)\. At the response level, adaptive consistency uses uncertainty to allocate additional samples[Aggarwal et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib1), while semantic entropy quantifies uncertainty across semantically equivalent generations[Farquhar et al\. \(2024\)](https://arxiv.org/html/2610.19561#bib.bib9);[Kuhn et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib17)\. At the search level, Entro\-duction uses entropy and varentropy to decide whether to deepen, expand, or terminate reasoning exploration[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.19561#bib.bib32)\.

Recent work connects token\-level uncertainty with the branching of generation trajectories\.[Bigelow et al\. \(2025\)](https://arxiv.org/html/2610.19561#bib.bib4)identify forking tokens by resampling alternative continuations and examining their downstream outcomes\.[Wang et al\. \(2026\)](https://arxiv.org/html/2610.19561#bib.bib26)further associate high\-entropy tokens with reasoning forks and exploit these positions to focus RLVR policy\-gradient updates\. At inference time, DeepConf\([Fu et al\., 2026](https://arxiv.org/html/2610.19561#bib.bib11)\)aggregates token confidence to filter or terminate reasoning traces\.Rewinduses local uncertainty to guide repair within an existing response: entropy\-gradient backtracking and boundary alignment select a cut point for suffix regeneration, without updating model parameters\.

## 3Preliminaries and Observations

We first review Power Sampling to establish the technical background\. We then present an empirical study of token entropy across repeated reasoning generations, demonstrating its potential as a during\-generation signal for detecting errors\.

### 3\.1Power Sampling for LLMs

For a questionqq, an autoregressive LLM defines a distribution over responses𝐲\\mathbf\{y\}through

pθ​\(𝐲∣q\)=∏t=1T𝐲pθ​\(yt∣q,y<t\),p\_\{\\theta\}\(\\mathbf\{y\}\\mid q\)=\\prod\_\{t=1\}^\{T\_\{\\mathbf\{y\}\}\}p\_\{\\theta\}\(y\_\{t\}\\mid q,y\_\{<t\}\),\(1\)whereT𝐲T\_\{\\mathbf\{y\}\}includes the terminating token\. Power Sampling[Karan and Du \(2025\)](https://arxiv.org/html/2610.19561#bib.bib16)favors higher\-likelihood complete responses using

πα​\(𝐲∣q\)\\displaystyle\\pi\_\{\\alpha\}\(\\mathbf\{y\}\\mid q\)=pθ​\(𝐲∣q\)αZα​\(q\),\\displaystyle=\\frac\{p\_\{\\theta\}\(\\mathbf\{y\}\\mid q\)^\{\\alpha\}\}\{Z\_\{\\alpha\}\(q\)\},\(2\)Zα​\(q\)\\displaystyle Z\_\{\\alpha\}\(q\)=∑𝐲′pθ\(𝐲′∣q\)α,α≥1\.\\displaystyle=\\sum\_\{\\mathbf\{y\}^\{\\prime\}\}p\_\{\\theta\}\(\\mathbf\{y\}^\{\\prime\}\\mid q\)^\{\\alpha\},\\qquad\\alpha\\geq 1\.The normalization is over complete responses\. Consequently, this target differs from applying temperature1/α1/\\alphaseparately to each next\-token distribution\. BecauseZα​\(q\)Z\_\{\\alpha\}\(q\)is intractable, a Metropolis–Hastings \(MH\) update can use a proposal kernelr⁡\(𝐲′∣𝐲,q\)r\(\\mathbf\{y\}^\{\\prime\}\\mid\\mathbf\{y\},q\)and accept with probability

Aq​\(𝐲,𝐲′\)=min⁡\{1,pθ​\(𝐲′∣q\)α​r​\(𝐲∣𝐲′,q\)pθ​\(𝐲∣q\)α​r​\(𝐲′∣𝐲,q\)\}\.A\_\{q\}\(\\mathbf\{y\},\\mathbf\{y\}^\{\\prime\}\)=\\min\\\!\\left\\\{1,\\frac\{p\_\{\\theta\}\(\\mathbf\{y\}^\{\\prime\}\\mid q\)^\{\\alpha\}r\(\\mathbf\{y\}\\mid\\mathbf\{y\}^\{\\prime\},q\)\}\{p\_\{\\theta\}\(\\mathbf\{y\}\\mid q\)^\{\\alpha\}r\(\\mathbf\{y\}^\{\\prime\}\\mid\\mathbf\{y\},q\)\}\\right\\\}\.\(3\)The unknown normalizer cancels\. A random\-suffix proposal chooses a cut point and autoregressively regenerates the continuation\. Equation[3](https://arxiv.org/html/2610.19561#S3.E3)gives the generic MH rule and Section[4](https://arxiv.org/html/2610.19561#S4)specifies our state\-dependent proposal and implemented MH\-style filter\.

### 3\.2Empirical Entropy–Error Associations

To examine the relationship between token entropy and response correctness, we collect repeated generations for each MATH500 question\. Let𝒬\\mathcal\{Q\}denote the set of 500 questions\. For eachq∈𝒬q\\in\\mathcal\{Q\}, we sampleN=20N=20responses from Qwen2\.5\-Math\-7B\. Let𝐲q,i\\mathbf\{y\}\_\{q,i\}denote theii\-th response for questionqq,ℓq,i\\ell\_\{q,i\}its token length, andcq,i∈\{0,1\}c\_\{q,i\}\\in\\\{0,1\\\}its correctness label under the original boxed\-answer evaluator\. Each response consists of a sequence of generated tokens,𝐲q,i=\(tq,i0,tq,i1,…,tq,iℓq,i−1\)\\mathbf\{y\}\_\{q,i\}=\(t^\{0\}\_\{q,i\},t^\{1\}\_\{q,i\},\\ldots,t^\{\\ell\_\{q,i\}\-1\}\_\{q,i\}\)\. For each positionjj, we record the full\-vocabulary next\-token distribution before applying sampling transformations\. Let𝒱\\mathcal\{V\}denote the vocabulary\. The distributionpq,i,jp\_\{q,i,j\}and its entropyHq,i,jH\_\{q,i,j\}at positionjjare defined as:

pq,i,j​\(v\)\\displaystyle p\_\{q,i,j\}\(v\)=pθ​\(v∣q,𝐲q,i,<j\),\\displaystyle=p\_\{\\theta\}\\bigl\(v\\mid q,\\mathbf\{y\}\_\{q,i,<j\}\\bigr\),\(4\)Hq,i,j\\displaystyle H\_\{q,i,j\}=−∑v∈𝒱pq,i,j\(v\)logpq,i,j\(v\),\\displaystyle=\-\\sum\_\{v\\in\\mathcal\{V\}\}p\_\{q,i,j\}\(v\)\\log p\_\{q,i,j\}\(v\),where𝐲q,i,<j=\(tq,i0,…,tq,ij−1\)\\mathbf\{y\}\_\{q,i,<j\}=\(t^\{0\}\_\{q,i\},\\ldots,t^\{j\-1\}\_\{q,i\}\)is the generated prefix before positionjj, and is empty whenj=0j=0\.

Fors∈\{\+,−\}s\\in\\\{\+,\-\\\}, letℐqs\\mathcal\{I\}\_\{q\}^\{s\}denote the indices of responses in classss, and letnqs=∑i∈ℐqsℓq,in\_\{q\}^\{s\}=\\sum\_\{i\\in\\mathcal\{I\}\_\{q\}^\{s\}\}\\ell\_\{q,i\}be the total number of tokens in that class for questionqq\. The mean entropy for classssis

hqs=1nqs​∑i∈ℐqs∑j=0ℓq,i−1Hq,i,j,nqs\>0\.h\_\{q\}^\{s\}=\\frac\{1\}\{n\_\{q\}^\{s\}\}\\sum\_\{i\\in\\mathcal\{I\}\_\{q\}^\{s\}\}\\sum\_\{j=0\}^\{\\ell\_\{q,i\}\-1\}H\_\{q,i,j\},\\qquad n\_\{q\}^\{s\}\>0\.\(5\)Thus,hqsh\_\{q\}^\{s\}is the mean entropy pooled over all tokens from responses in classssfor questionqq\.

#### Paired relative increase\.

The paired set is

𝒬±=\{q∈𝒬:nq\+\>0,nq−\>0,hq\+\>0\}\.\\mathcal\{Q\}\_\{\\pm\}=\\\{q\\in\\mathcal\{Q\}:n\_\{q\}^\{\+\}\>0,\\ n\_\{q\}^\{\-\}\>0,\\ h\_\{q\}^\{\+\}\>0\\\}\.\(6\)There are 463 paired questions\. For each, define

rq=hq−−hq\+hq\+,r¯=1\|𝒬±\|​∑q∈𝒬±rq\.r\_\{q\}=\\frac\{h\_\{q\}^\{\-\}\-h\_\{q\}^\{\+\}\}\{h\_\{q\}^\{\+\}\},\\qquad\\overline\{r\}=\\frac\{1\}\{\|\\mathcal\{Q\}\_\{\\pm\}\|\}\\sum\_\{q\\in\\mathcal\{Q\}\_\{\\pm\}\}r\_\{q\}\.\(7\)The mean per\-question relative increase is approximately112%\(median:83\.61%\)\. Incorrect\-response mean token entropy exceeds the correct\-response mean on88\.12%of paired questions, with an increase ofat least 70% on 55\.51%\.

#### Response\-level diagnostics\.

To complement the within\-question analysis, we test whether entropy summaries of individual responses can distinguish incorrect from correct answers\. For each nonempty response, we compute its peak token entropy, its largest adjacent\-token entropy increase, and its mean token entropy\. Table[1](https://arxiv.org/html/2610.19561#S3.T1)reports the average value of each statistic for correct and incorrect responses, along with its AUROC for ranking incorrect responses above correct ones\. All three statistics are higher on average for incorrect responses, and mean token entropy provides the strongest discrimination \(AUROC 0\.746\)\. These results further show that token\-level entropy can help distinguish correct from incorrect responses\.

Table 1:Entropy diagnostics on MATH500\.The same Qwen2\.5\-Math\-7B study yields 9,951 nonempty responses after excluding 49 empty ones\. Scores exclude prompt, padding, and EOS\. AUROC predicts incorrect original labels\. Brackets give 95% CIs from 2,000 whole\-question bootstrap resamples\.
#### Entropy across success\-count groups\.

We further examine whether the entropy of incorrect responses varies with how often a question is answered correctly\. We group the questions by their number of correct responses among 20 generations and compare the mean entropy of incorrect responses across groups\. For each questionq∈𝒬q\\in\\mathcal\{Q\}, letmq=∑i=120cq,im\_\{q\}=\\sum\_\{i=1\}^\{20\}c\_\{q,i\}denote the number of correct responses\. We group consecutive counts in pairs, keeping 0 and 19 separate:

ℬ=\(\{0\},\{1,2\},…,\{17,18\},\{19\}\)\.\\mathcal\{B\}=\\bigl\(\\\{0\\\},\\\{1,2\\\},\\ldots,\\\{17,18\\\},\\\{19\\\}\\bigr\)\.\(8\)For each binB∈ℬB\\in\\mathcal\{B\}, let𝒢B\\mathcal\{G\}\_\{B\}be the set of questions whose number of correct responses falls inBB, and let𝒢B−\\mathcal\{G\}\_\{B\}^\{\-\}be the subset with at least one incorrect response\. We defineyBy\_\{B\}as the average, across questions in𝒢B−\\mathcal\{G\}\_\{B\}^\{\-\}, of the mean token entropy of incorrect responses:

yB\\displaystyle y\_\{B\}=1\|𝒢B−\|​∑q∈𝒢B−hq−\.\\displaystyle=\\frac\{1\}\{\|\\mathcal\{G\}\_\{B\}^\{\-\}\|\}\\sum\_\{q\\in\\mathcal\{G\}\_\{B\}^\{\-\}\}h\_\{q\}^\{\-\}\.\(9\)Based on the calculatedyBy\_\{B\}values, Figure[1](https://arxiv.org/html/2610.19561#S3.F1)shows the mean entropy of incorrect responses across success\-count groups\. The clear upward trend indicates that questions answered correctly more often tend to have higher\-entropy incorrect responses\. Therefore, higher entropy may help identify errors that are more likely to be corrected through re\-generation\.

Figure 1:Incorrect\-response entropy on MATH500\.The x\-axis groups questions by the number of correct responses out of 20\. Yellow bars show equal\-question means of incorrect\-response entropy \(Equation[9](https://arxiv.org/html/2610.19561#S3.E9)\)\. Numbers above bars give entropy in nats; numbers below bin labels count questions in each group\.This group\-level analysis complements the paired and response\-level comparisons and further motivates the uncertainty\-conditioned proposals developed next\. These analyses establish that entropy carries information about errors at the response level; whether it also identifies useful revision*sites*is an empirical question we test directly through proposal ablations in Section[5\.3](https://arxiv.org/html/2610.19561#S5.SS3)\.

## 4Method

Motivated by the observed entropy patterns, we proposeRewind, an entropy\-guided method for selecting where to revise a reasoning trajectory\. Figure[2](https://arxiv.org/html/2610.19561#S4.F2)summarizes the proposal policy\. Here,𝐱\\mathbf\{x\}denotes the concatenation of the prompt and response, andLcontextL\_\{\\text\{context\}\}is the prompt length\. Only generated positions are eligible for revision\.

![Refer to caption](https://arxiv.org/html/2610.19561v1/method.png)Figure 2:REWIND proposal policy\.The entropy peakt∗t^\{\*\}anchors gradient backtracking totstartt\_\{\\text\{start\}\}\. Boundary alignment gives the suffix starttalignedt\_\{\\text\{aligned\}\}; regeneration and score\-based acceptance complete the proposal\.### 4\.1Entropy\-Guided Suffix Proposals

We use token entropy as the default diagnosticMtM\_\{t\}\(Equation[4](https://arxiv.org/html/2610.19561#S3.E4)\)\. One revision step has four phases\.

Phase 1: Peak Selection\.Select the highest\-entropy generated position:

t∗=arg​maxLcontext<t≤\|𝐱\|⁡Mt\.t^\{\*\}=\\operatorname\*\{arg\\,max\}\_\{L\_\{\\text\{context\}\}<t\\leq\|\\mathbf\{x\}\|\}M\_\{t\}\.\(10\)
Phase 2: Gradient Backtracking\.Choose the largest difference between adjacent generated positions within a backward distancewwof the peak:

Δ​Mt\\displaystyle\\Delta M\_\{t\}=Mt−Mt−1,t\>Lcontext\+1,\\displaystyle=M\_\{t\}\-M\_\{t\-1\},\\quad t\>L\_\{\\text\{context\}\}\+1,\(11\)tstart\\displaystyle t\_\{\\text\{start\}\}=arg​maxmax⁡\(Lcontext\+2,t∗−w\)≤t≤t∗⁡Δ​Mt\.\\displaystyle=\\operatorname\*\{arg\\,max\}\_\{\\begin\{subarray\}\{c\}\\max\(L\_\{\\text\{context\}\}\+2,t^\{\*\}\-w\)\\leq t\\leq t^\{\*\}\\end\{subarray\}\}\\Delta M\_\{t\}\.If the peak is the first generated token, settstart=t∗t\_\{\\text\{start\}\}=t^\{\*\}without computing a difference\. Thuswwspecifies a backward distance, with at mostw\+1w\+1candidate positions\.

Phase 3: Boundary Alignment\.To avoid cutting through an incomplete span, search the precedingkktokens for a delimiter:

taligned=1\+max⁡\{t∈ℬk​\(tstart\)∣xt∈𝒮bound\},t\_\{\\text\{aligned\}\}=1\+\\max\\\{t\\in\\mathcal\{B\}\_\{k\}\(t\_\{\\text\{start\}\}\)\\mid x\_\{t\}\\in\\mathcal\{S\}\_\{\\text\{bound\}\}\\\},\(12\)whereℬk​\(a\)=\[max⁡\(Lcontext\+1,a−k\),a\)\\mathcal\{B\}\_\{k\}\(a\)=\[\\max\(L\_\{\\text\{context\}\}\+1,a\-k\),a\)contains integer positions and𝒮bound\\mathcal\{S\}\_\{\\text\{bound\}\}contains tokens whose decoded text includes a delimiter\. If none is found, settaligned=tstartt\_\{\\text\{aligned\}\}=t\_\{\\text\{start\}\}\. Keep𝐱<taligned\\mathbf\{x\}\_\{<t\_\{\\text\{aligned\}\}\}, including any selected delimiter, and regenerate the suffix to obtain𝐱′\\mathbf\{x\}^\{\\prime\}\.

Phase 4: MH\-Style Acceptance\.LetqLMq\_\{\\text\{LM\}\}denote the actual next\-token proposal distribution\. For the shared suffix starta=taligneda=t\_\{\\text\{aligned\}\}, compare both sequences overm=min⁡\(\|𝐱\|,\|𝐱′\|\)−a\+1m=\\min\(\|\\mathbf\{x\}\|,\|\\mathbf\{x\}^\{\\prime\}\|\)\-a\+1tokens:

log⁡rsuffix\\displaystyle\\log r\_\{\\text\{suffix\}\}=S⁡\(𝐱′,a,m\)−S⁡\(𝐱,a,m\),\\displaystyle=S\(\\mathbf\{x\}^\{\\prime\},a;m\)\-S\(\\mathbf\{x\},a;m\),\(13\)S⁡\(𝐳,a,m\)\\displaystyle S\(\\mathbf\{z\},a;m\)=∑t=aa\+m−1\[α​log⁡pθ​\(zt∣z<t\)−log⁡qLM​\(zt∣z<t\)\]\.\\displaystyle=\\sum\_\{t=a\}^\{a\+m\-1\}\\left\[\\begin\{aligned\} &\\alpha\\log p\_\{\\theta\}\(z\_\{t\}\\mid z\_\{<t\}\)\\\\ &\\quad\-\\log q\_\{\\text\{LM\}\}\(z\_\{t\}\\mid z\_\{<t\}\)\\end\{aligned\}\\right\]\.Accept𝐱′\\mathbf\{x\}^\{\\prime\}iflog⁡u<min⁡\(0,log⁡rsuffix\)\\log u<\\min\(0,\\log r\_\{\\text\{suffix\}\}\)foru∼𝒰⁡\(0,1\)u\\sim\\mathcal\{U\}\(0,1\); otherwise retain𝐱\\mathbf\{x\}\. This shared\-horizon, state\-dependent filter is MH\-style: it omits forward/reverse site\-selection probabilities and does not establish exact MH stationarity\. Each subsequent iteration recomputes the entropy trajectory of the retained sequence\. Appendix[M](https://arxiv.org/html/2610.19561#A13)gives the pseudocode\.

All signals and sequence scores are computed from the fixed language model, requiring no additional training or external verifier\.

#### Integration with Power Sampling\.

Power Sampling performsKKrandom\-cut proposals in each ofBBblocks;Rewindthen adds one entropy\-guided proposal to the retained sequence\. All proposals share the acceptance score in Equation[13](https://arxiv.org/html/2610.19561#S4.E13)\.

### 4\.2Theoretical Intuition

For prefixst=𝐱<ts\_\{t\}=\\mathbf\{x\}\_\{<t\}and latent interpretationZZ, the mixturep⁡\(x∣st\)=∑zp⁡\(z∣st\)​p​\(x∣st,z\)p\(x\\mid s\_\{t\}\)=\\sum\_\{z\}p\(z\\mid s\_\{t\}\)p\(x\\mid s\_\{t\},z\)yieldsHt=At\+BtH\_\{t\}=A\_\{t\}\+B\_\{t\}, whereAtA\_\{t\}is expected within\-interpretation entropy andBt=I⁡\(Xt;Z∣st\)B\_\{t\}=I\(X\_\{t\};Z\\mid s\_\{t\}\)measures cross\-interpretation disagreement\. A sharp increase inBtB\_\{t\}can create a gradient spike before a later entropy peak\. Appendix[A](https://arxiv.org/html/2610.19561#A1)gives sufficient conditions for this local pattern; it motivates candidate selection without guaranteeing a correct retained prefix or causal error localization\.

## 5Experiments

### 5\.1Experimental Setup

In the main comparison, we evaluate the effectiveness of ourRewindcontroller across three model families to assess cross\-model robustness\. Specifically, we use Qwen2\.5\-Math\-7B[Yang et al\. \(2024\)](https://arxiv.org/html/2610.19561#bib.bib30)as a representative specialist math model, Llama\-3\.1\-Tülu\-3\-8B\-DPO \(Tülu\-8B\)[Lambert et al\. \(2024\)](https://arxiv.org/html/2610.19561#bib.bib18)as a generalist instruction\-tuned baseline, and DeepSeek\-1\.5B[Guo et al\. \(2025\)](https://arxiv.org/html/2610.19561#bib.bib12)as a lightweight reasoning\-distilled model\. All Tülu\-8B experiments use this same DPO checkpoint\.

We report single\-output accuracy on five benchmarks spanning varying difficulty levels: GSM8K \(grade\-school math\)[Cobbe et al\. \(2021\)](https://arxiv.org/html/2610.19561#bib.bib7), MATH500 \(competition math\)[Hendrycks et al\. \(2021\)](https://arxiv.org/html/2610.19561#bib.bib13), the GPQA Diamond subset \(198 questions\)[Rein et al\. \(2024\)](https://arxiv.org/html/2610.19561#bib.bib23), AIME2025 \(frontier math competition\)[Art of Problem Solving \(2025a\)](https://arxiv.org/html/2610.19561#bib.bib2);[Art of Problem Solving \(2025b\)](https://arxiv.org/html/2610.19561#bib.bib3), and the BBH formal fallacies task \(250 questions\)[Suzgun et al\. \(2022\)](https://arxiv.org/html/2610.19561#bib.bib25)\. The comparison in Table[2](https://arxiv.org/html/2610.19561#S5.T2)reports the mean over three independent runs with different random seeds\. To further assess robustness under common evaluation protocols and broader settings, Appendix[I](https://arxiv.org/html/2610.19561#A9)reports standard few\-shot CoT prompting, Appendix[J](https://arxiv.org/html/2610.19561#A10)reports code\-generation results, and Appendix[K](https://arxiv.org/html/2610.19561#A11)reports standard deviations for representative Qwen2\.5\-Math settings\.

#### Baselines\.

We compare decoding strategies across four categories\. First,distribution\- and decoding\-basedmethods include Vanilla, Temperature Sampling \(T=0\.25T=0\.25\), Typical\-p \(p=0\.95p=0\.95\)[Meister et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib21), and DoLa[Chuang et al\. \(2024\)](https://arxiv.org/html/2610.19561#bib.bib6)\(layer\-contrastive decoding\)\. Second,revision\-basedmethods include Self\-Refine[Madaan et al\. \(2023\)](https://arxiv.org/html/2610.19561#bib.bib20), which iteratively regenerates and edits the response based on the model’s own feedback\. Third,MCMC\-basedsampling is represented by Power Sampling[Karan and Du \(2025\)](https://arxiv.org/html/2610.19561#bib.bib16)\. Fourth,candidate selectionincludes Best\-of\-NN\(BoN@10\): we sampleN=10N=10responses for each input and select the candidate with the highest mean token log\-likelihood\.

#### Implementation\.

We use Power Sampling in 16 blocks with temperatureT=0\.25T=0\.25and target powerα=4\\alpha=4\. Each active block extends the retained sequence and performsK=10K=10random\-cut MCMC proposals;Rewindthen attempts one additional entropy\-guided revision withw=5w=5andk=10k=10\. All other settings, including prompts, generation limits, and acceptance scoring, match MCMC\. Implementation details are provided in Appendix[M](https://arxiv.org/html/2610.19561#A13)\. Our code is available at[https://anonymous\.4open\.science/r/rewind\-D310](https://anonymous.4open.science/r/rewind-D310)\.

### 5\.2Main Results

Rewindachieves the highest mean accuracy in all 15 model–benchmark settings \(Table[2](https://arxiv.org/html/2610.19561#S5.T2)\)\.

Table 2:Main results\.Accuracy \(%\) averaged over three runs\.Bestandsecond\-bestare highlighted\. Vanilla usesT=1T=1\.For Qwen2\.5\-Math\-7B,Rewindimproves MATH500 from 66\.80% under temperature sampling to 77\.60%, and from 72\.60% under standard MCMC to 77\.60% by adding a guided revision after the sharedK=10K=10random MCMC steps per block\. On DeepSeek\-1\.5B,Rewindreaches 26\.67% on AIME2025 compared with 16\.67% for MCMC under the same blockwise protocol\. Appendix[K](https://arxiv.org/html/2610.19561#A11), Table[18](https://arxiv.org/html/2610.19561#A11.T18), expands the corresponding three\-run main results with standard deviations\.

### 5\.3Signal Comparison and Analysis

We evaluate both the choice of trigger signal and the strategy for turning that signal into a revision point\. OurRewindis built on two distinct design choices: \(1\) backtracking from the uncertainty peak to an earlier entropy\-gradient spike \(gradient\-based intervention selection\) and \(2\) aligning revisions with coarse semantic boundaries\. To test whether these choices contribute beyond generic regeneration, Table[3](https://arxiv.org/html/2610.19561#S5.T3)reports an ablation study on the MATH500 dataset using the Qwen2\.5\-Math model\.

Table 3:Proposal ablation on Qwen2\.5\-Math\-7B/MATH500\.Accuracy \(%\) withK=10K=10random MCMC steps per block\. Revision variants add a suffix proposal\. Local variants use the peak window; “grad\.” uses its largest entropy increase\.#### Comparing Intervention\-Point Localization\.

Entropy\-gradient backtracking improves accuracy to 77\.60%, compared with 74\.20% for Peak Revision and 72\.60% for random\-proposal MCMC\. It also outperforms Random\-in\-Window \(75\.20%\) by 2\.40 percentage points\. Random Delimiter and Random\-in\-Window each add one suffix proposal per block with the same generation and acceptance settings asRewind; the respective 4\.40\- and 2\.40\-point gaps therefore isolate site selection at the same proposal\-count budget\. These results show that choosing the largest entropy increase within the peak window yields more effective revision sites than revising at the peak or at a random position in the same window\.

#### The Role of Boundary Alignment\.

Boundary alignment increases accuracy from 75\.00% to 77\.60%, a gain of 2\.60 percentage points\. Random Delimiter revision achieves 73\.20%, showing that boundary cues are most useful when combined with targeted site selection\. Restricting the delimiter set to\.and\\nstill achieves 76\.80%, only 0\.80 points below the full set\. These results show that coarse boundary alignment contributes to the gain and remains effective with a small set of delimiters\.

#### Separating Signal Choice from Intervention Strategy\.

For each statistic, we define a revision scoreMtM\_\{t\}before applying peak selection or gradient tracing\. Maximum probability, probability gap, and Top\-KKembedding similarity are reversed, while vocabulary\-probability skewness and excess kurtosis are negated\. Temporal KL usesDKL\(pt∥pt−1\)D\_\{\\mathrm\{KL\}\}\(p\_\{t\}\\\|p\_\{t\-1\}\)\. All variants share the suffix\-generation and acceptance procedure; Appendix[D](https://arxiv.org/html/2610.19561#A4)gives the exact score definitions\. For a fixedMtM\_\{t\}, peak revision selects its maximum, whereas tracing selects the largest increaseΔ​Mt\\Delta M\_\{t\}in the preceding window\. Table[4](https://arxiv.org/html/2610.19561#S5.T4)compares these strategies under six signal configurations\. Tracing improves observed MATH500 accuracy for all six, with gains ranging from 0\.2 points for skewness to 3\.4 for entropy\. Temporal KL also gains 3\.2 points, supporting the utility of tracing beyond Shannon entropy in this setting\.

Table 4:Signal choice vs\. intervention strategy\.MATH500 accuracy \(%\) on Qwen2\.5\-Math\-7B for six signal configurations\.Δ\\Deltais tracing minus peak revision\.
#### Trigger Choice across Benchmarks\.

Among the evaluated trigger configurations, entropy performs best on all three benchmarks as shown in Table[5](https://arxiv.org/html/2610.19561#S5.T5)\. Alternatives show different transfer patterns\. Temporal KL improves MATH500 over MCMC \(75\.00% vs\. 72\.60%\), but lowers GSM8K accuracy and ties MCMC on AIME2025\. Top\-KKsimilarity withK=5K=5and kurtosis also improve MATH500, yet fall below MCMC on AIME2025 \(7\.78% and 8\.89% vs\. 10\.00%\)\. Skewness gives smaller gains on all three\. These results indicate that improving intervention\-point selection for a given signal does not necessarily make it a strong trigger across tasks\. Together with the within\-question observations in Section[3\.2](https://arxiv.org/html/2610.19561#S3.SS2), the signal comparisons support entropy as the default trigger for the evaluated controller\.

Table 5:Trigger signals across benchmarks\.Accuracy \(%\) on Qwen2\.5\-Math\-7B\.
#### Answer\-level intervention utility\.

Across the three main\-table runs on MATH500,Rewindaverages 46 repairs and 21 regressions per run relative to MCMC, yielding 25 net corrections \(\+5\.00 points; Appendix[F](https://arxiv.org/html/2610.19561#A6)\)\.

### 5\.4Inference Budget and BoN Comparison

Relative to single\-pass temperature sampling, the reported Qwen2\.5\-Math generation\-token ratios are8\.84×8\.84\\timesfor MCMC and9\.26×9\.26\\timesforRewind\. BoN@10 samples ten independent responses per input, corresponding to a nominal∼10×\\sim 10\\timesgeneration\-token budget under comparable average response lengths\. On this nominal basis, the three methods operate at roughly comparable generation budgets, withRewindadding approximately 4\.8% generated tokens over MCMC\.

Table[2](https://arxiv.org/html/2610.19561#S5.T2)reports the accuracy comparison with likelihood\-ranked BoN@10\.Rewindachieves higher accuracy on all 15 reported model–benchmark pairs, with gains of 1\.60–11\.47 percentage points\. On Qwen2\.5\-Math\-7B/MATH500, it reaches 77\.60% versus 73\.00% for BoN@10 \(\+4\.60 points\) and on DeepSeek\-1\.5B/AIME2025, it reaches 26\.67% versus 16\.67% \(\+10\.00 points\)\. These comparisons favor targeted suffix repair over the evaluated candidate\-selection baseline\.

### 5\.5Transfer across Tasks and Prompts

Table 6:Transfer across tasks and prompts\.Accuracy \(%\) on HumanEval and under 8\-shot CoT math prompting\. The latter uses Qwen2\.5\-Math\-7B\.#### Code Generation\.

Table[6](https://arxiv.org/html/2610.19561#S5.T6)extends the evaluation to HumanEval[Chen et al\. \(2021\)](https://arxiv.org/html/2610.19561#bib.bib5), whereRewindgains 2\.4 points over MCMC on Qwen2\.5\-Math\-7B and 3\.7 on Tülu\-8B\. Restricting Qwen2\.5\-Math\-7B’s boundary set to\.and\\nretains 57\.90% accuracy, compared with 58\.5% for the full set and 56\.1% for MCMC \(Appendix[C](https://arxiv.org/html/2610.19561#A3)\)\. Most of the observed gain thus survives with two general\-purpose boundary anchors\. These gains support transfer beyond mathematics on two backbones; Appendix[J](https://arxiv.org/html/2610.19561#A10)provides the full comparison\.

#### Few\-Shot Prompting\.

With standard 8\-shot CoT prompts on Qwen2\.5\-Math\-7B, Table[6](https://arxiv.org/html/2610.19561#S5.T6)shows gains over MCMC of 5\.6 points on MATH500, 2\.1 on GSM8K, and 6\.7 on AIME2025\. The gains therefore persist in this additional prompting setting and Appendix[I](https://arxiv.org/html/2610.19561#A9)reports all decoding baselines\.

## 6Discussion

#### From Entropy Diagnostics to Revision Sites\.

The repeated\-generation study shows that entropy carries useful information about response correctness\. Mean token entropy provides the strongest response\-level discrimination in Table[1](https://arxiv.org/html/2610.19561#S3.T1), while the token\-wise entropy trajectory preserves the positional information needed for revision\.Rewinduses an entropy peak to anchor a local search and selects the largest entropy increase within that window\. This connects the entropy–correctness finding to an actionable revision policy\.

#### Signal Choice and Intervention Strategy\.

The signal comparisons and ablations identify complementary ingredients of effective revision\. Entropy achieves the highest accuracy in the main cross\-benchmark signal sweep, and tracing improves over peak revision for all six tested signal configurations on MATH500\. Boundary alignment further improves the entropy\-based controller and remains effective with only periods and newlines\. Together, these results suggest that useful refinement depends on both an informative uncertainty signal and a suitable rule for converting that signal into a suffix proposal\.

#### When Revision Fails\.

Manual inspection of Qwen2\.5\-Math\-7B on MATH500 finds confident errors without a salient entropy spike in 7\.6% of all problems and detected but unrecovered failures in 14\.8% \(Appendix[H](https://arxiv.org/html/2610.19561#A8)\)\. The latter account for most residual failures, highlighting regeneration quality as a remaining bottleneck\.

## 7Conclusion

In this paper, we present empirical evidence that token\-level entropy is associated with reasoning errors\. We found that incorrect responses have higher mean token entropy on 88\.12% of paired MATH500 questions, with an average within\-question relative increase of 112%\. Based on this association between the during\-generation signal and reasoning errors, we introduce REWIND, a training\-free controller that uses entropy\-gradient backtracking to select revision points, aligns them with natural boundaries, and filters regenerated suffixes with an MH\-style score\. Across three model families and five benchmarks,Rewindoutperforms all baselines, improving over random\-proposal MCMC by 0\.51–10\.00 percentage points\. Signal comparisons identify entropy as the strongest tested trigger across three mathematical benchmarks, and controlled ablations support the contributions of both gradient backtracking and boundary alignment\. Together, these results show that intrinsic uncertainty can guide targeted trajectory repair and improve test\-time reasoning\.

## Limitations

#### Computational Cost\.

The cost study in Section[5\.4](https://arxiv.org/html/2610.19561#S5.SS4)reports generation\-token counts, which exclude entropy\-only forward passes and do not establish normalized FLOPs or wall\-clock cost\. Entropy computation and discarded proposals add overhead, and efficiency comparisons depend on implementation and batching\.

#### Scope and Generalization\.

The task and model coverage in Section[5\.5](https://arxiv.org/html/2610.19561#S5.SS5)is limited\. Supplementary HumanEval results cover only two backbones\. Further tasks, backbones, and repeated runs are needed to assess broader transfer\.

#### Signal Validity and Statistical Coverage\.

The benchmarks provide no token\-level first\-error annotations, so the evidence concerns intervention utility rather than localization accuracy\. AIME2025 has only 30 questions, limiting the precision of accuracy comparisons\. Finally,7\.6%7\.6\\%of the inspected MATH500 examples are confidently wrong without a salient entropy spike \(Appendix[H](https://arxiv.org/html/2610.19561#A8)\), setting a direct limit on uncertainty\-triggered repair\.

## Ethical Considerations

REWIND can still produce incorrect answers despite improved benchmark accuracy\. Its iterative revision requires additional inference computation, which may increase energy consumption\.

## References

- Aggarwal et al\. \(2023\)Pranjal Aggarwal, Aman Madaan, Yiming Yang, and 1 others\. 2023\.Let’s sample step by step: Adaptive\-consistency for efficient reasoning and coding with LLMs\.*arXiv preprint arXiv:2305\.11860*\.
- Art of Problem Solving \(2025a\)Art of Problem Solving\. 2025a\.2025 AIME I\.[https://artofproblemsolving\.com/wiki/index\.php/2025\_AIME\_I](https://artofproblemsolving.com/wiki/index.php/2025_AIME_I)\.Accessed: 2026\.
- Art of Problem Solving \(2025b\)Art of Problem Solving\. 2025b\.2025 AIME II\.[https://artofproblemsolving\.com/wiki/index\.php/2025\_AIME\_II](https://artofproblemsolving.com/wiki/index.php/2025_AIME_II)\.Accessed: 2026\.
- Bigelow et al\. \(2025\)Eric Bigelow, Ari Holtzman, Hidenori Tanaka, and Tomer Ullman\. 2025\.Forking paths in neural text generation\.In*International Conference on Learning Representations*, volume 2025, pages 17134–17167\.
- Chen et al\. \(2021\)Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others\. 2021\.Evaluating large language models trained on code\.*arXiv preprint arXiv:2107\.03374*\.
- Chuang et al\. \(2024\)Yung\-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R\. Glass, and Pengcheng He\. 2024\.DoLa: Decoding by contrasting layers improves factuality in large language models\.In*The Twelfth International Conference on Learning Representations*\.
- Cobbe et al\. \(2021\)Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others\. 2021\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*\.
- Dziri et al\. \(2023\)Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, and 1 others\. 2023\.Faith and fate: Limits of transformers on compositionality\.*Advances in Neural Information Processing Systems*, 36:70293–70332\.
- Farquhar et al\. \(2024\)Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal\. 2024\.Detecting hallucinations in large language models using semantic entropy\.*Nature*, 630\(8017\):625–630\.
- Forristal et al\. \(2023\)Jarad Forristal, Fatemehsadat Mireshghallah, Greg Durrett, and Taylor Berg\-Kirkpatrick\. 2023\.A block Metropolis–Hastings sampler for controllable energy\-based text generation\.In*Proceedings of the 27th Conference on Computational Natural Language Learning \(CoNLL\)*, pages 403–413\.
- Fu et al\. \(2026\)Yichao Fu, Xuewei Wang, Hao Zhang, Yuandong Tian, and Jiawei Zhao\. 2026\.Deep think with confidence\.In*International Conference on Learning Representations*, volume 2026, pages 94355–94377\.
- Guo et al\. \(2025\)Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others\. 2025\.DeepSeek\-R1: Incentivizing reasoning capability in LLMs via reinforcement learning\.*arXiv preprint arXiv:2501\.12948*\.
- Hendrycks et al\. \(2021\)Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\. 2021\.Measuring mathematical problem solving with the MATH dataset\.*arXiv preprint arXiv:2103\.03874*\.
- Huang et al\. \(2023\)Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou\. 2023\.Large language models cannot self\-correct reasoning yet\.*arXiv preprint arXiv:2310\.01798*\.
- Kadavath et al\. \(2022\)Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield\-Dodds, Nova DasSarma, Eli Tran\-Johnson, and 1 others\. 2022\.Language models \(mostly\) know what they know\.*arXiv preprint arXiv:2207\.05221*\.
- Karan and Du \(2025\)Aayush Karan and Yilun Du\. 2025\.Reasoning with sampling: Your base model is smarter than you think\.*arXiv preprint arXiv:2510\.14901*\.
- Kuhn et al\. \(2023\)Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar\. 2023\.Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation\.*arXiv preprint arXiv:2302\.09664*\.
- Lambert et al\. \(2024\)Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, and 1 others\. 2024\.Tulu 3: Pushing frontiers in open language model post\-training\.*arXiv preprint arXiv:2411\.15124*\.
- Liu \(2025\)Lihui Liu\. 2025\.Monte Carlo tree search for graph reasoning in large language model agents\.In*Proceedings of the 34th ACM International Conference on Information and Knowledge Management*, pages 4966–4970\.
- Madaan et al\. \(2023\)Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others\. 2023\.Self\-Refine: Iterative refinement with self\-feedback\.*Advances in neural information processing systems*, 36:46534–46594\.
- Meister et al\. \(2023\)Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell\. 2023\.Locally typical sampling\.*Transactions of the Association for Computational Linguistics*, 11:102–121\.
- Miao et al\. \(2019\)Ning Miao, Hao Zhou, Lili Mou, Rui Yan, and Lei Li\. 2019\.CGMH: Constrained sentence generation by Metropolis–Hastings sampling\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 33, pages 6834–6842\.
- Rein et al\. \(2024\)David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman\. 2024\.GPQA: A graduate\-level Google\-proof Q&A benchmark\.In*First Conference on Language Modeling*\.
- Shannon \(1948\)Claude E Shannon\. 1948\.A mathematical theory of communication\.*The Bell system technical journal*, 27\(3\):379–423\.
- Suzgun et al\. \(2022\)Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, and Jason Wei\. 2022\.Challenging BIG\-Bench tasks and whether chain\-of\-thought can solve them\.*arXiv preprint arXiv:2210\.09261*\.
- Wang et al\. \(2026\)Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xiong\-Hui Chen, Jianxin Yang, Zhenru Zhang, and 1 others\. 2026\.Beyond the 80/20 rule: High\-entropy minority tokens drive effective reinforcement learning for llm reasoning\.*Advances in Neural Information Processing Systems*, 38:115452–115486\.
- Wang et al\. \(2023\)Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H\. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou\. 2023\.Self\-consistency improves chain of thought reasoning in language models\.In*The Eleventh International Conference on Learning Representations*\.
- Wei et al\. \(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others\. 2022\.Chain\-of\-thought prompting elicits reasoning in large language models\.*Advances in neural information processing systems*, 35:24824–24837\.
- Xie et al\. \(2021\)Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma\. 2021\.An explanation of in\-context learning as implicit Bayesian inference\.*arXiv preprint arXiv:2111\.02080*\.
- Yang et al\. \(2024\)An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, and 1 others\. 2024\.Qwen2\.5\-Math technical report: Toward mathematical expert model via self\-improvement\.*arXiv preprint arXiv:2409\.12122*\.
- Yao et al\. \(2023\)Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan\. 2023\.Tree of thoughts: Deliberate problem solving with large language models\.*Advances in neural information processing systems*, 36:11809–11822\.
- Zhang et al\. \(2025\)Jinghan Zhang, Xiting Wang, Fengran Mo, Yeyang Zhou, Wanfu Gao, and Kunpeng Liu\. 2025\.Entropy\-based exploration conduction for multi\-step reasoning\.*arXiv preprint arXiv:2503\.15848*\.
- Zhou et al\. \(2023\)Andy Zhou, Kai Yan, Michal Shlapentokh\-Rothman, Haohan Wang, and Yu\-Xiong Wang\. 2023\.Language agent tree search unifies reasoning acting and planning in language models\.*arXiv preprint arXiv:2310\.04406*\.

## Appendix Overview

The appendix provides supporting material for the main empirical and methodological claims\. Appendix[A](https://arxiv.org/html/2610.19561#A1)gives a theoretical account of entropy\-gradient localization, while Appendix[A\.6](https://arxiv.org/html/2610.19561#A1.SS6)explains the sequence\-level motivation behind the accept/reject refinement loop\. Appendix[B](https://arxiv.org/html/2610.19561#A2)compares targeted refinement with a larger single\-pass model\. Appendices[C](https://arxiv.org/html/2610.19561#A3)–[E](https://arxiv.org/html/2610.19561#A5)report robustness checks for boundary alignment, trigger signals, and supplementary controller variants\. Appendices[F](https://arxiv.org/html/2610.19561#A6)–[H](https://arxiv.org/html/2610.19561#A8)analyze repairs, regressions, proposal acceptance, intervention\-site coverage, and residual failures\. Finally, Appendix[I](https://arxiv.org/html/2610.19561#A9), Appendix[J](https://arxiv.org/html/2610.19561#A10), and Appendix[K](https://arxiv.org/html/2610.19561#A11)provide additional prompting, generalization, and stability results, followed by qualitative case studies in Appendix[L](https://arxiv.org/html/2610.19561#A12)and implementation details and pseudocode in Appendix[M](https://arxiv.org/html/2610.19561#A13)\.

## Appendix AEntropy\-Gradient Intuition

We analyze entropy\-gradient localization through*semantic fragmentation*: the model becomes uncertain between conflicting interpretations, increasing cross\-interpretation disagreement\. We derive sufficient conditions under which the resulting entropy\-gradient spike identifies the local onset of fragmentation\.

### A\.1Setup and Notation

Letst:=x<ts\_\{t\}:=x\_\{<t\}denote the prefix at steptt\. The next\-token entropy is defined as

Ht:=H\(Xt∣st\)=−∑vp\(v∣st\)logp\(v∣st\),H\_\{t\}:=H\(X\_\{t\}\\mid s\_\{t\}\)=\-\\sum\_\{v\}p\(v\\mid s\_\{t\}\)\\log p\(v\\mid s\_\{t\}\),\(14\)and the entropy gradient isΔ​Ht:=Ht−Ht−1\\Delta H\_\{t\}:=H\_\{t\}\-H\_\{t\-1\}\.

### A\.2Main Result

###### Theorem A\.1\(Entropy Gradient Localizes Semantic Fragmentation\)\.

Assume the model’s predictive distribution admits a latent decomposition

p⁡\(x∣st\)=∑z∈𝒵p⁡\(z∣st\)⋅p⁡\(x∣st,z\),p\(x\\mid s\_\{t\}\)=\\sum\_\{z\\in\\mathcal\{Z\}\}p\(z\\mid s\_\{t\}\)\\cdot p\(x\\mid s\_\{t\},z\),\(15\)wherezzrepresents implicit semantic interpretations\. Define the within\-world entropyAt:=𝔼Z​\[H⁡\(Xt∣st,Z\)\]A\_\{t\}:=\\mathbb\{E\}\_\{Z\}\[H\(X\_\{t\}\\mid s\_\{t\},Z\)\]and the cross\-world disagreementBt:=I⁡\(Xt;Z∣st\)B\_\{t\}:=I\(X\_\{t\};Z\\mid s\_\{t\}\)\.

\(Decomposition\)The entropy decomposes as

Ht=At\+Bt,henceΔ​Ht=Δ​At\+Δ​Bt\.H\_\{t\}=A\_\{t\}\+B\_\{t\},\\quad\\text\{hence\}\\quad\\Delta H\_\{t\}=\\Delta A\_\{t\}\+\\Delta B\_\{t\}\.\(16\)
\(Localization\)Ift0t\_\{0\}lies inside the searched neighborhood𝒲⁡\(t∗\)\\mathcal\{W\}\(t^\{\*\}\)and semantic fragmentation occurs att0t\_\{0\}, causing a mutual information jumpΔ​Bt0≥Γ\\Delta B\_\{t\_\{0\}\}\\geq\\Gamma, and the within\-world entropy is locally smooth with\|Δ​At\|≤L\|\\Delta A\_\{t\}\|\\leq Lover the searched neighborhood𝒲⁡\(t∗\)\\mathcal\{W\}\(t^\{\*\}\), then

Δ​Ht0≥Γ−L\.\\Delta H\_\{t\_\{0\}\}\\geq\\Gamma\-L\.\(17\)
\(Identification\)If additionallyΔ​Bt≤γ\\Delta B\_\{t\}\\leq\\gammafor all competing positionst∈𝒲⁡\(t∗\)t\\in\\mathcal\{W\}\(t^\{\*\}\),t≠t0t\\neq t\_\{0\}\(no comparable MI jumps in the local search window\) andΓ−γ\>2​L\\Gamma\-\\gamma\>2L, then

t0=arg​maxt∈𝒲⁡\(t∗\)⁡Δ​Ht\.t\_\{0\}=\\operatorname\*\{arg\\,max\}\_\{t\\in\\mathcal\{W\}\(t^\{\*\}\)\}\\Delta H\_\{t\}\.\(18\)

###### Proof\.

Part 1follows from the definition of conditional mutual information\. Recall thatI⁡\(Xt;Z∣st\)=H⁡\(Xt∣st\)−H⁡\(Xt∣st,Z\)I\(X\_\{t\};Z\\mid s\_\{t\}\)=H\(X\_\{t\}\\mid s\_\{t\}\)\-H\(X\_\{t\}\\mid s\_\{t\},Z\)\. Rearranging givesH⁡\(Xt∣st\)=H⁡\(Xt∣st,Z\)\+I⁡\(Xt;Z∣st\)H\(X\_\{t\}\\mid s\_\{t\}\)=H\(X\_\{t\}\\mid s\_\{t\},Z\)\+I\(X\_\{t\};Z\\mid s\_\{t\}\)\. SinceH⁡\(Xt∣st,Z\)=∑zp⁡\(z∣st\)​H​\(Xt∣st,z\)=𝔼Z​\[H⁡\(Xt∣st,Z\)\]=AtH\(X\_\{t\}\\mid s\_\{t\},Z\)=\\sum\_\{z\}p\(z\\mid s\_\{t\}\)H\(X\_\{t\}\\mid s\_\{t\},z\)=\\mathbb\{E\}\_\{Z\}\[H\(X\_\{t\}\\mid s\_\{t\},Z\)\]=A\_\{t\}andI⁡\(Xt;Z∣st\)=BtI\(X\_\{t\};Z\\mid s\_\{t\}\)=B\_\{t\}, we obtainHt=At\+BtH\_\{t\}=A\_\{t\}\+B\_\{t\}\.

Part 2: Att0t\_\{0\}, we haveΔ​Ht0=Δ​At0\+Δ​Bt0\\Delta H\_\{t\_\{0\}\}=\\Delta A\_\{t\_\{0\}\}\+\\Delta B\_\{t\_\{0\}\}\. By the smoothness assumption,Δ​At0≥−L\\Delta A\_\{t\_\{0\}\}\\geq\-L\. By the fragmentation assumption,Δ​Bt0≥Γ\\Delta B\_\{t\_\{0\}\}\\geq\\Gamma\. ThusΔ​Ht0≥−L\+Γ=Γ−L\\Delta H\_\{t\_\{0\}\}\\geq\-L\+\\Gamma=\\Gamma\-L\.

Part 3: For any competing positiont∈𝒲⁡\(t∗\)t\\in\\mathcal\{W\}\(t^\{\*\}\),t≠t0t\\neq t\_\{0\}, we haveΔ​At≤L\\Delta A\_\{t\}\\leq L\(local smoothness\) andΔ​Bt≤γ\\Delta B\_\{t\}\\leq\\gamma\(no comparable MI jumps\)\. ThusΔ​Ht=Δ​At\+Δ​Bt≤L\+γ\\Delta H\_\{t\}=\\Delta A\_\{t\}\+\\Delta B\_\{t\}\\leq L\+\\gamma\. The conditionΓ−L\>L\+γ\\Gamma\-L\>L\+\\gamma\(equivalently,Γ−γ\>2​L\\Gamma\-\\gamma\>2L\) ensuresΔ​Ht0≥Γ−L\>L\+γ≥Δ​Ht\\Delta H\_\{t\_\{0\}\}\\geq\\Gamma\-L\>L\+\\gamma\\geq\\Delta H\_\{t\}for all suchtt\. ∎

### A\.3Why an Earlier Cut May Help

Backtracking fromt∗t^\{\*\}tot0<t∗t\_\{0\}<t^\{\*\}removes the intervening tokens and lets the model regenerate from before the identified distributional shift\. When those tokens encode a harmful premise, the earlier cut provides an opportunity to revise it\. The benefit depends on the content removed; proposal\-policy ablations and paired final\-answer outcomes evaluate this choice empirically\.

### A\.4When Does Fragmentation Produce a Detectable Signal?

###### Lemma A\.3\(MI Lower Bound\)\.

Suppose at steptt, two semantic worldsz1,z2z\_\{1\},z\_\{2\}have total massm=p⁡\(z1∣st\)\+p⁡\(z2∣st\)m=p\(z\_\{1\}\\mid s\_\{t\}\)\+p\(z\_\{2\}\\mid s\_\{t\}\), normalized conditional weightsπ~1,π~2≥η\\tilde\{\\pi\}\_\{1\},\\tilde\{\\pi\}\_\{2\}\\geq\\etawithin this two\-world subset, andTV\(p\(⋅∣st,z1\),p\(⋅∣st,z2\)\)≥δ\\mathrm\{TV\}\(p\(\\cdot\\mid s\_\{t\},z\_\{1\}\),p\(\\cdot\\mid s\_\{t\},z\_\{2\}\)\)\\geq\\delta\(they disagree on predictions\)\. Then

I⁡\(Xt;Z∣st\)≥2​m​η​\(1−η\)​δ2\.I\(X\_\{t\};Z\\mid s\_\{t\}\)\\geq 2m\\,\\eta\(1\-\\eta\)\\delta^\{2\}\.\(19\)

###### Proof Sketch\.

Letpi=p\(⋅∣st,zi\)p\_\{i\}=p\(\\cdot\\mid s\_\{t\},z\_\{i\}\)denote the conditional distributions\. We first condition on the eventZ∈\{z1,z2\}Z\\in\\\{z\_\{1\},z\_\{2\}\\\}, whose probability ismm, and write the normalized weights asπ~i=p⁡\(zi∣st\)/m\\tilde\{\\pi\}\_\{i\}=p\(z\_\{i\}\\mid s\_\{t\}\)/m\. For the two\-component conditional mixturep¯12=π~1​p1\+π~2​p2\\bar\{p\}\_\{12\}=\\tilde\{\\pi\}\_\{1\}p\_\{1\}\+\\tilde\{\\pi\}\_\{2\}p\_\{2\}, the total variation between each component and the conditional mixture satisfiesTV⁡\(p1,p¯12\)=π~2​TV​\(p1,p2\)≥π~2​δ\\mathrm\{TV\}\(p\_\{1\},\\bar\{p\}\_\{12\}\)=\\tilde\{\\pi\}\_\{2\}\\mathrm\{TV\}\(p\_\{1\},p\_\{2\}\)\\geq\\tilde\{\\pi\}\_\{2\}\\deltaand similarly forp2p\_\{2\}\.

Applying Pinsker’s inequalityDKL\(p∥q\)≥2TV\(p,q\)2D\_\{\\mathrm\{KL\}\}\(p\\\|q\)\\geq 2\\,\\mathrm\{TV\}\(p,q\)^\{2\}within this conditional two\-world mixture yields a conditional mutual\-information lower bound of2​π~1​π~2​δ22\\tilde\{\\pi\}\_\{1\}\\tilde\{\\pi\}\_\{2\}\\delta^\{2\}\. Multiplying by the subset probabilitymmgives the stated contribution to the full mutual information by the chain rule and non\-negativity of the remaining components\.

Sinceπ~1,π~2≥η\\tilde\{\\pi\}\_\{1\},\\tilde\{\\pi\}\_\{2\}\\geq\\etaandπ~1\+π~2=1\\tilde\{\\pi\}\_\{1\}\+\\tilde\{\\pi\}\_\{2\}=1, we haveπ~1​π~2≥η⁡\(1−η\)\\tilde\{\\pi\}\_\{1\}\\tilde\{\\pi\}\_\{2\}\\geq\\eta\(1\-\\eta\), yieldingI⁡\(Xt;Z∣st\)≥2​m​η​\(1−η\)​δ2I\(X\_\{t\};Z\\mid s\_\{t\}\)\\geq 2m\\eta\(1\-\\eta\)\\delta^\{2\}\. ∎

### A\.5Scope of the Analysis

The analysis describes errors that manifest as localized semantic fragmentation\.*Confident errors*can remain low\-entropy because the model commits to a wrong but internally coherent interpretation\.*Gradual drift*spreads uncertainty changes across steps, weakening the localized\-jump signal\.

The conditionΓ−γ\>2​L\\Gamma\-\\gamma\>2Lcharacterizes when the fragmentation signal dominates background changes\. Large uncertainty shifts during legitimate reasoning transitions can compete with that signal and affect the selected revision site\.

The latent decomposition serves as an explanatory model\. Our experiments use observable entropy traces and assess revision utility through final\-answer accuracy\.

### A\.6Sequence\-Level Score Motivation

Temperature1/α1/\\alphanormalizes each next\-token distribution separately\. Because these normalizers depend on the prefix, their product varies across trajectories\. Sequence\-level Power Sampling instead uses the global normalizer inp​\(𝐱\)α/Z⁡\(α\)p\(\\mathbf\{x\}\)^\{\\alpha\}/Z\(\\alpha\), motivating suffix\-level scoring\. In exact MH, this normalizer cancels from the forward/reverse ratio\.Rewindcompares suffix scores at the selected site using the MH\-style filter in Equation[13](https://arxiv.org/html/2610.19561#S4.E13); state\-dependent site\-selection probabilities are omitted\. Final\-answer accuracy measures the resulting refinement performance\.

## Appendix BComparison with Qwen2\.5\-Math\-72B

We compare iterative refinement with Qwen2\.5\-Math\-7B against single\-pass sampling from Qwen2\.5\-Math\-72B atT=0\.25T=0\.25\. The two settings differ in both model size and inference budget, evaluating a trade\-off between model scale and iterative computation\. Table[7](https://arxiv.org/html/2610.19561#A2.T7)reports accuracy across all five benchmarks\.

Table 7:Comparison with Qwen2\.5\-Math\-72B\.Accuracy \(%\) is reported across five benchmarks\. The 72B baseline is single\-pass sampling withT=0\.25T=0\.25, whileRewinduses the Qwen2\.5\-Math\-7B backbone with iterative refinement\.Rewindwith the 7B backbone outperforms the single\-pass 72B baseline on MATH500, GPQA, AIME2025, and BBH; the 72B model achieves higher accuracy on GSM8K\. These results demonstrate the utility of spending additional inference computation on targeted revision in these settings\.

## Appendix CBoundary Alignment Robustness

Boundary alignment uses a lightweight decoded\-token delimiter rule\. We test its sensitivity by restricting the delimiter set to two general\-purpose anchors,\\nand\.\. Table[8](https://arxiv.org/html/2610.19561#A3.T8)shows that this restricted variant preserves most of the full method’s performance on both MATH500 and HumanEval\. The results support coarse boundary alignment and show that the gains remain strong with a small delimiter set\.

Table 8:Robustness to restricted delimiters\.The restricted setting keeps only\\nand\.; full uses\\n,\.,,,$, and\)\.
## Appendix DSignal Robustness

This appendix supplements the signal analysis in Section[5\.3](https://arxiv.org/html/2610.19561#S5.SS3)\. We first define the implemented revision scores, then report the full peak\-versus\-tracing comparison in Table[9](https://arxiv.org/html/2610.19561#A4.T9), with the MCMC reference\. The cross\-benchmark signal sweep is reported in main\-text Table[5](https://arxiv.org/html/2610.19561#S5.T5)\.

#### Signal Definitions\.

Letℓt\\ell\_\{t\}be the model logits predicting generated tokenxtx\_\{t\},pt=softmax⁡\(ℓt\)p\_\{t\}=\\operatorname\{softmax\}\(\\ell\_\{t\}\)the full\-vocabulary distribution before sampling transformations, andV=\|𝒱\|V=\|\\mathcal\{V\}\|\. All logarithms are natural\. Writept,\(1\)≥pt,\(2\)p\_\{t,\(1\)\}\\geq p\_\{t,\(2\)\}for its two largest probabilities\. The following scoresMtM\_\{t\}are passed to the common peak\-selection and gradient\-tracing rules:

Temporal KL is explicitly

Mt=∑v∈𝒱pt​\(v\)​log⁡pt​\(v\)pt−1​\(v\),M\_\{t\}=\\sum\_\{v\\in\\mathcal\{V\}\}p\_\{t\}\(v\)\\log\\frac\{p\_\{t\}\(v\)\}\{p\_\{t\-1\}\(v\)\},\(20\)with score zero at the first generated position\. Maximum probability and probability gap are reversed so that lower confidence gives a higher revision score\.

For Top\-KKsimilarity, letvt,1,…,vt,Kv\_\{t,1\},\\ldots,v\_\{t,K\}be the highest\-probability tokens andeve\_\{v\}the fixed model’s input embedding of tokenvv\. We use the unweighted mean pairwise cosine similarity:

Ct,K=2K⁡\(K−1\)​∑i<jcos⁡\(evt,i,evt,j\)\.C\_\{t,K\}=\\frac\{2\}\{K\(K\-1\)\}\\sum\_\{i<j\}\\cos\(e\_\{v\_\{t,i\}\},e\_\{v\_\{t,j\}\}\)\.\(21\)Thus lower embedding similarity gives a higher score\. We evaluateK∈\{3,5\}K\\in\\\{3,5\\\}\.

Skewness and excess kurtosis are calculated across theVVvocabulary probability values\. Following the implementation’s sample\-standard\-deviation convention, define

st2\\displaystyle s\_\{t\}^\{2\}=1V−1​∑v∈𝒱\(pt​\(v\)−1/V\)2,\\displaystyle=\\frac\{1\}\{V\-1\}\\sum\_\{v\\in\\mathcal\{V\}\}\\bigl\(p\_\{t\}\(v\)\-1/V\\bigr\)^\{2\},\(22\)zt​\(v\)\\displaystyle z\_\{t\}\(v\)=pt​\(v\)−1/Vst\+10−10\.\\displaystyle=\\frac\{p\_\{t\}\(v\)\-1/V\}\{s\_\{t\}\+10^\{\-10\}\}\.Both statistics are negated before site selection\. For each score, peak revision maximizesMtM\_\{t\}, while tracing maximizes the signed differenceMt−Mt−1M\_\{t\}\-M\_\{t\-1\}within the preceding peak window\. The suffix\-generation and acceptance procedure is shared across signal variants\.

#### Varentropy\.

The varentropy configuration in Table[10](https://arxiv.org/html/2610.19561#A5.T10)uses the temperature\-scaled full\-vocabulary distribution

p~t=softmax⁡\(ℓt/Ts\),Ts=0\.25\.\\widetilde\{p\}\_\{t\}=\\operatorname\{softmax\}\(\\ell\_\{t\}/T\_\{s\}\),\\qquad T\_\{s\}=0\.25\.\(23\)Its score is the variance of token surprisal underp~t\\widetilde\{p\}\_\{t\}:

Mt=∑vp~t​\(v\)​\[log⁡p~t​\(v\)\]2−H​\(p~t\)2\.M\_\{t\}=\\sum\_\{v\}\\widetilde\{p\}\_\{t\}\(v\)\[\\log\\widetilde\{p\}\_\{t\}\(v\)\]^\{2\}\-H\(\\widetilde\{p\}\_\{t\}\)^\{2\}\.\(24\)The implementation takes the absolute value of this difference to obtain its nonnegative score\. The other scores defined above use the unscaled distributionptp\_\{t\}\.

#### Robustness of the Gradient Tracing Strategy\.

To test whether the gradient\-versus\-peak intervention pattern is specific to Shannon entropy, we decouple thelocalization strategyfrom themetric choice\. On MATH500, we compare two revision strategies across multiple distributional statistics \(KL divergence, Top\-KKsimilarity, skewness, and kurtosis\)\. Peak selection revises at the position with maximum metric value \(t∗=arg⁡max⁡Mtt^\{\*\}=\\arg\\max M\_\{t\}\)\. In contrast,Targeted Gradient Tracing \(Tracing\)revises at the maximum metric gradient within a preceding window \(tstart=arg⁡max⁡Δ​Mtt\_\{\\text\{start\}\}=\\arg\\max\\Delta M\_\{t\}\), treating the sharp distributional shift as a proxy for an earlier useful intervention point\.

Table 9:Peak vs\. gradient\-based intervention localization\.Accuracy \(%\) on MATH500 under naive peak revision and gradient tracing\.On Qwen2\.5\-Math\-7B/MATH500, gradient tracing improves accuracy for every listed configuration in Table[9](https://arxiv.org/html/2610.19561#A4.T9)\. The largest gain is 3\.4 points for entropy, followed by 3\.2 for temporal KL\. Kurtosis \(\+1\.0\) and skewness \(\+0\.2\) show smaller gains\. These comparisons support revising at an earlier distributional shift within the peak window\.

Entropy achieves the highest accuracy on all three benchmarks in Table[5](https://arxiv.org/html/2610.19561#S5.T5)\. Both Top\-KKconfigurations gain 0\.8 points from tracing on MATH500 in Table[9](https://arxiv.org/html/2610.19561#A4.T9), while entropy remains stronger in the cross\-benchmark comparison\.

## Appendix ESupplementary Controller Robustness

We evaluate alternative trigger metrics and boundary/window settings over three random seeds\.

Maximum probability and probability gap use the reversed scores defined in Appendix[D](https://arxiv.org/html/2610.19561#A4)\. Varentropy uses the temperature\-scaled distribution withTs=0\.25T\_\{s\}=0\.25, as specified there\.

Table 10:Supplementary trigger\-metric checks\.Max probability, probability gap, and varentropy are added as common confidence\-style baselines under the same MCMC revision budget\. Entropy values are repeated from Table[5](https://arxiv.org/html/2610.19561#S5.T5)for reference\.In Table[10](https://arxiv.org/html/2610.19561#A5.T10), maximum probability matches entropy on AIME2025 and falls below MCMC on MATH500\. Probability gap and varentropy also trail entropy on MATH500\.

Table 11:Supplementary boundary and window checks\.Expanded delimiters add additional punctuation and closing brackets to the default set\. The adaptive\-window variant uses the full delimiter set and allows a larger look\-back window capped at 20 tokens\.The default delimiter set and fixedw=5w=5window achieve the highest accuracy in Table[11](https://arxiv.org/html/2610.19561#A5.T11)\. Both expanded delimiters and the adaptive window reduce accuracy relative to this setting, supporting a small delimiter set and a fixed local search window\.

## Appendix FAnswer\-Level Intervention Analysis

Table 12:Answer\-level repair and regression analysis on MATH500\.Mean counts across the same three runs summarized in Table[2](https://arxiv.org/html/2610.19561#S5.T2)\. Repair/regression is the ratio of the mean counts\.Table[12](https://arxiv.org/html/2610.19561#A6.T12)compares final answers under standard MCMC andRewindover the same three runs used for the main results in Table[2](https://arxiv.org/html/2610.19561#S5.T2)\. Within each run, we count repairs and regressions over the 500 MATH500 questions, then average these counts across runs\. Relative to MCMC,Rewindrepairs 46 previously incorrect answers and regresses on 21 previously correct answers on average per run\. The mean net gain is therefore 25 correct answers per 500 questions, corresponding to the 5\.00\-percentage\-point difference in mean accuracy reported in the main table\.

Table 13:Proposal acceptance ratios\.Percentage of proposals accepted under random MCMC andRewindacross three benchmarks\.Rewindhas lower proposal acceptance on all three benchmarks in Table[13](https://arxiv.org/html/2610.19561#A6.T13)\. On MATH500, the complete controller accepts 54\.1% of proposals, compared with 57\.3% for random MCMC, alongside the positive net answer gain in Table[12](https://arxiv.org/html/2610.19561#A6.T12)\.

#### MATH500 observation protocol\.

The repeated\-generation observations in Section[3\.2](https://arxiv.org/html/2610.19561#S3.SS2)use temperature1\.01\.0and base seed 42, with a 3,072\-token generation cap and a 4,096\-token context limit\. Of the 10,000 responses, 248 reach their generation cap\.

## Appendix GIntervention\-Site Coverage

Each guided proposal selects one aligned cut point and regenerates its suffix\. We count the distinct aligned positions visited across refinement rounds under the blockwise protocol withK=10K=10random MCMC steps per block\. On MATH500, the controller visits an average of3\.18distinct sequence positions per problem \(Table[14](https://arxiv.org/html/2610.19561#A7.T14)\)\.

Table 14:Iterative coverage across refinement rounds on MATH500\.Mean number of distinct aligned sequence positions visited per problem during refinement\.This statistic summarizes the controller’s exploration of cut points across refinement rounds\.

## Appendix HResidual Failure Analysis

We manually classify the remaining MATH500 failures using their entropy traces and regeneration outcomes\. Table[15](https://arxiv.org/html/2610.19561#A8.T15)summarizes two categories: \(i\) confident errors, where the model is wrong without a salient entropy spike, and \(ii\) detected\-but\-unrecovered cases, whereRewindidentifies instability but the regenerated suffix still does not reach a correct answer\.

Table 15:Failure taxonomy on MATH500 \(manual inspection\)\.Confident errors are a minority of all evaluated problems; most residual failures occur after the entropy spike is detected but regeneration still does not recover a correct continuation\.Detected\-but\-unrecovered cases account for 66\.1% of residual failures\. This distribution motivates stronger suffix proposals for detected instability and complementary signals for confident errors\.

## Appendix IStandard Few\-Shot CoT Evaluation

Our main experiments use a single\-output setting to isolate the effect of the inference\-time controller\. To test whether the gains persist under common prompting protocols, we additionally evaluate Qwen2\.5\-Math\-7B with standard 8\-shot CoT prompting on the math benchmarks\. Table[16](https://arxiv.org/html/2610.19561#A9.T16)shows thatRewindcontinues to improve over vanilla decoding, temperature sampling, and standard MCMC under this setting\.

Table 16:Performance under standard 8\-shot CoT prompting\.Accuracy \(%\) is reported for Qwen2\.5\-Math\-7B\.Rewindmaintains consistent gains under few\-shot CoT prompts\.
## Appendix JAdditional Generalization Results

Table[17](https://arxiv.org/html/2610.19561#A10.T17)reports supplementary HumanEval[Chen et al\. \(2021\)](https://arxiv.org/html/2610.19561#bib.bib5)results for Qwen2\.5\-Math\-7B and Tülu\-8B\. These results show gains beyond mathematics on both backbones\.

Table 17:Supplementary HumanEval results\.Rewindalso improves code\-generation performance on two backbones\.
## Appendix KMulti\-Seed Stability

Table[18](https://arxiv.org/html/2610.19561#A11.T18)expands the Qwen2\.5\-Math\-7B results in Table[2](https://arxiv.org/html/2610.19561#S5.T2)for MATH500 and AIME2025 using the same three runs\. We report the arithmetic mean and sample standard deviation of run\-level accuracies, with the latter computed using denominatorR−1R\-1forR=3R=3runs\.

Table 18:Multi\-seed stability on Qwen2\.5\-Math\-7B\.Accuracy mean±\\pmsample standard deviation for the same three runs summarized in Table[2](https://arxiv.org/html/2610.19561#S5.T2)\.
## Appendix LCase Studies

We present illustrative examples demonstrating howRewindidentifies and corrects problematic reasoning patterns\.

### L\.1Case Study 1: Methodological Improvement

Problem:Find the sum of all integer basesb\>9b\>9for which17b17\_\{b\}is a divisor of97b97\_\{b\}\.

Before Revision \(Brute\-force Enumeration\):

The model initially attempts to solve the problem by exhaustively checking each value ofkk:

> “Fork=2k=2, we getb​\(9−2\)=7​\(2−1\)b\(9\-2\)=7\(2\-1\), which simplifies to7​b=77b=7, but this does not satisfyb\>9b\>9\. Fork=3k=3, we getb​\(9−3\)=7​\(3−1\)b\(9\-3\)=7\(3\-1\), which simplifies to6​b=146b=14, andb=146b=\\frac\{14\}\{6\}, which is not an integer\. Fork=4k=4… Fork=5k=5… Fork=6k=6… Fork=7k=7… Fork=8k=8…”

This approach is tedious, error\-prone, and lacks mathematical elegance\.

After Revision \(Modular Arithmetic\):

After detecting high entropy at the beginning of the solution, the model revises to use a more principled approach:

> “We rewrite9​b\+79b\+7in terms ofb\+7b\+7: 9​b\+7=9​\(b\+7\)−63\+7=9​\(b\+7\)−569b\+7=9\(b\+7\)\-63\+7=9\(b\+7\)\-56Thus:9​b\+7≡−56\(modb\+7\)9b\+7\\equiv\-56\\pmod\{b\+7\}\. Forb\+7b\+7to divide9​b\+79b\+7, it must divide−56\-56\. Therefore,b\+7b\+7must be a divisor of 56:\{1,2,4,7,8,14,28,56\}\\\{1,2,4,7,8,14,28,56\\\}\.”

Improvement:The revision transforms a brute\-force enumeration into an elegant number\-theoretic solution, reducing the problem to finding divisors of 56\.

### L\.2Case Study 2: Geometric Error Correction

Problem \(2025 AIME I, Problem 2\)\.In triangleA​B​CABC, the points alongA​B¯\\overline\{AB\}occur in the orderA,D,E,BA,D,E,B, with\(A​D,D​E,E​B\)=\(4,16,8\)\(AD,DE,EB\)=\(4,16,8\)\. AlongA​C¯\\overline\{AC\}, the order isA,F,G,CA,F,G,C, with\(A​F,F​G,G​C\)=\(13,52,26\)\(AF,FG,GC\)=\(13,52,26\)\. PointFFis the midpoint ofD​M¯\\overline\{DM\}, and pointEEis the midpoint ofG​N¯\\overline\{GN\}, defining the reflected pointsMMandNN\. Given that quadrilateralD​E​G​FDEGFhas area288288, determine the area of heptagonA​F​N​B​C​E​MAFNBCEM[Art of Problem Solving \(2025a\)](https://arxiv.org/html/2610.19561#bib.bib2)\.

Before Revision \(Coordinate Error\):

The model makes a critical error in setting up coordinates:

> “FFis onA​C¯\\overline\{AC\}such thatA​F=13AF=13andF​G=52FG=52, soFFis at\(13,0\)\(13,0\)\.”

Error:PointFFshould lie on lineA​CAC, not on thexx\-axis \(which represents lineA​BAB\)\. This fundamental mistake would propagate through the entire solution\.

After Revision \(Side\-Aligned Coordinates\):

The revised excerpt assigns points to the two side directions:

> “Determine the total length of sideA​BAB:A​B=A​D\+D​E\+E​B=4\+16\+8=28AB=AD\+DE\+EB=4\+16\+8=28\. Determine the total length of sideA​CAC:A​C=A​F\+F​G\+G​C=13\+52\+26=91AC=AF\+FG\+GC=13\+52\+26=91\. Find the coordinates: - •DDis at\(4,0\)\(4,0\)onA​BAB - •EEis at\(20,0\)\(20,0\)onA​BAB - •FFis at\(0,13\)\(0,13\)onA​CAC - •GGis at\(0,65\)\(0,65\)onA​CAC ”

Improvement:The revision correctly distinguishes points onA​BABfrom points onA​CAC\.

Coordinate convention and area scaling\.WithAAas the origin, let𝐞A​B\\mathbf\{e\}\_\{AB\}and𝐞A​C\\mathbf\{e\}\_\{AC\}be unit vectors along the two sides\. We interpret a coordinate pair\(u,v\)\(u,v\)in the excerpt as the position vectoru​𝐞A​B\+v​𝐞A​Cu\\mathbf\{e\}\_\{AB\}\+v\\mathbf\{e\}\_\{AC\}in this oblique basis\. Ifϕ=∠​B​A​C\\phi=\\angle BAC, Euclidean areas equal coordinate areas multiplied bysin⁡ϕ\\sin\\phi\. ForD​E​G​FDEGF, the coordinate area and the scale implied by its given Euclidean area are

Scoord​\(D​E​G​F\)\\displaystyle S\_\{\\mathrm\{coord\}\}\(DEGF\)=20⋅65−4⋅132=624,\\displaystyle=\\frac\{20\\cdot 65\-4\\cdot 13\}\{2\}=624,sin⁡ϕ\\displaystyle\\sin\\phi=288624=613\.\\displaystyle=\\frac\{288\}\{624\}=\\frac\{6\}\{13\}\.
#### Case Study Summary\.

These illustrative excerpts show changes in the reasoning text:

1. 1\.Improve methodology:Transform inefficient enumeration into elegant mathematical solutions \(Case 1\)
2. 2\.Correct errors:Identify and fix fundamental setup mistakes before they propagate \(Case 2\)

The controlled final\-answer comparisons in Section[5\.2](https://arxiv.org/html/2610.19561#S5.SS2)quantify the overall accuracy gains from revision\.

## Appendix MImplementation Details and Algorithm

### M\.1Benchmark Subsets and Prompts

The five main benchmarks use the subsets in Table[19](https://arxiv.org/html/2610.19561#A13.T19)\. Each run evaluates the full listed subset, and accuracy is computed over questions within that run\. AIME2025 combines all 15 questions from AIME I and all 15 from AIME II\. BBH refers specifically to theformal\_fallaciestask, and GPQA uses the Diamond subset\.

Table 19:Evaluation subsets and BoN generation limits\.nnis the number of questions per run\. The last column gives the maximum new tokens per BoN candidate\.#### Mathematical prompts and chat formatting\.

The zero\-shot formatter for MATH500, GSM8K, and AIME2025 uses the following template, where\[QUESTION\]is replaced with the benchmark question:

> Can you solve the following math problem?\[QUESTION\]Please reason step by step, and put your final answer within\\boxed\{\}\.

Qwen2\.5\-Math\-7B and DeepSeek\-1\.5B receive this text directly\. For Tülu\-8B, we apply the tokenizer’sapply\_chat\_templateto the same text as a single user message, withtokenize=Falseandadd\_generation\_promptset toTrue\. The resulting prompt has a<\|user\|\>turn followed by a<\|assistant\|\>generation prefix, with no added system message\. The additional 8\-shot experiments are reported in Appendix[I](https://arxiv.org/html/2610.19561#A9)\.

### M\.2Best\-of\-NNSampling and Ranking

BoN@10 samples ten responses independently for each input at temperatureT=1\.0T=1\.0\. Candidate generation uses top\-k=50k=50and the checkpoint’s nucleus\-sampling setting: top\-p=1\.0p=1\.0for Qwen2\.5\-Math\-7B,0\.900\.90for Tülu\-8B \(the DPO checkpoint\), and0\.950\.95for DeepSeek\-1\.5B\. Typical sampling is disabled \(typical\_p=1\.0\\texttt\{typical\\\_p\}=1\.0\)\. Each candidate stops at EOS or the generation limit in Table[19](https://arxiv.org/html/2610.19561#A13.T19)\. For BBH and GPQA on Qwen2\.5\-Math\-7B, the candidate limit is further bounded by4096−Lprompt4096\-L\_\{\\text\{prompt\}\}to fit the model’s context length\.

We decode each candidate and tokenize its response text separately from the prompt for teacher\-forced rescoring\. For response tokens𝐲\(i\)=\(y1\(i\),…,yLi\(i\)\)\\mathbf\{y\}^\{\(i\)\}=\(y^\{\(i\)\}\_\{1\},\\ldots,y^\{\(i\)\}\_\{L\_\{i\}\}\)and formatted promptqq, selection uses

ℓi\\displaystyle\\ell\_\{i\}=1Li​∑t=1Lilog⁡pθ​\(yt\(i\)∣q,𝐲<t\(i\)\),\\displaystyle=\\frac\{1\}\{L\_\{i\}\}\\sum\_\{t=1\}^\{L\_\{i\}\}\\log p\_\{\\theta\}\(y^\{\(i\)\}\_\{t\}\\mid q,\\mathbf\{y\}^\{\(i\)\}\_\{<t\}\),\(25\)i∗\\displaystyle i^\{\*\}=arg​max1≤i≤10⁡ℓi\.\\displaystyle=\\operatorname\*\{arg\\,max\}\_\{1\\leq i\\leq 10\}\\ell\_\{i\}\.Scoring uses the model’s original conditional probabilities before temperature scaling or top\-kk/top\-ppfiltering\. Prompt, padding, and EOS tokens are excluded from both the score and its length normalization\. Ties are resolved by the original candidate order\. Empty responses are excluded from ranking, and candidates are not filtered by answer validity or correctness\. Only the selected response is used for the reported single\-output accuracy\.

### M\.3Controller Implementation

#### Sampling and stopping\.

Initial extensions and suffix proposals both use temperatureT=0\.25T=0\.25, with target powerα=1/T=4\\alpha=1/T=4\. Random and guided proposals use the same shared\-horizon score in Equation[13](https://arxiv.org/html/2610.19561#S4.E13)\. Generation stops early at EOS\. Prompts, initialization, block partitioning, generation limits, sampling settings, and acceptance scoring are shared with the MCMC baseline\.

#### Boundary matching\.

A token belongs to𝒮bound\\mathcal\{S\}\_\{\\text\{bound\}\}if its decoded text contains a newline or one of\.,,,$, and\)\. We select the nearest matching token within thek=10k=10generated tokens preceding the traced position and begin regeneration after that token, retaining the entire matched token in the prefix\. If no matching token is found, regeneration begins at the traced position\. Appendix[E](https://arxiv.org/html/2610.19561#A5)compares fixed and adaptive windows\.

#### Software and hardware\.

The controller uses PyTorch and Hugging Face Transformers on NVIDIA H800 GPUs\.

#### Code release and licensing\.

Experimental code is available in the repository linked in Section[5\.1](https://arxiv.org/html/2610.19561#S5.SS1)\. We release implementation code only and have verified that the applicable upstream licenses permit this release\. We do not redistribute pretrained model weights or benchmark datasets\.

Algorithm[1](https://arxiv.org/html/2610.19561#alg1)gives one proposal step\. Repeated steps recompute the entropy trajectory of the retained sequence; iteration and block budgets are specified in the experimental setup\.

Algorithm 1One Entropy\-Guided Revision Step0:Sequence

𝐱\\mathbf\{x\}, prompt length

LcontextL\_\{\\text\{context\}\}, backward distance

ww, boundary distance

kk, delimiter tokens

𝒮bound\\mathcal\{S\}\_\{\\text\{bound\}\}, distributions

pθ,qLMp\_\{\\theta\},q\_\{\\text\{LM\}\}, power

α\\alpha
0:Updated sequence

1:Phase 1: Peak Selection

2:Compute entropy

HtH\_\{t\}for

Lcontext<t≤\|𝐱\|L\_\{\\text\{context\}\}<t\\leq\|\\mathbf\{x\}\|
3:

t∗←arg​maxLcontext<t≤\|𝐱\|⁡Htt^\{\*\}\\leftarrow\\operatorname\*\{arg\\,max\}\_\{L\_\{\\text\{context\}\}<t\\leq\|\\mathbf\{x\}\|\}H\_\{t\}
4:Phase 2: Gradient Backtracking

5:if

t∗=Lcontext\+1t^\{\*\}=L\_\{\\text\{context\}\}\+1then

6:

tstart←t∗t\_\{\\text\{start\}\}\\leftarrow t^\{\*\}
7:else

8:

𝒯←\[max⁡\(Lcontext\+2,t∗−w\),t∗\]\\mathcal\{T\}\\leftarrow\[\\max\(L\_\{\\text\{context\}\}\+2,t^\{\*\}\-w\),t^\{\*\}\]
9:

Δ​Ht←Ht−Ht−1\\Delta H\_\{t\}\\leftarrow H\_\{t\}\-H\_\{t\-1\}for

t∈𝒯t\\in\\mathcal\{T\}
10:

tstart←arg​maxt∈𝒯⁡Δ​Htt\_\{\\text\{start\}\}\\leftarrow\\operatorname\*\{arg\\,max\}\_\{t\\in\\mathcal\{T\}\}\\Delta H\_\{t\}
11:endif

12:Phase 3: Boundary Alignment

13:

𝒞←\{t∈ℬk​\(tstart\):xt∈𝒮bound\}\\mathcal\{C\}\\leftarrow\\\{t\\in\\mathcal\{B\}\_\{k\}\(t\_\{\\text\{start\}\}\):x\_\{t\}\\in\\mathcal\{S\}\_\{\\text\{bound\}\}\\\}
14:

a←1\+max⁡𝒞a\\leftarrow 1\+\\max\\mathcal\{C\}if

𝒞≠∅\\mathcal\{C\}\\neq\\varnothing; otherwise

a←tstarta\\leftarrow t\_\{\\text\{start\}\}
15:Generate

𝐱suffix′∼qLM\(⋅∣𝐱<a\)\\mathbf\{x\}^\{\\prime\}\_\{\\text\{suffix\}\}\\sim q\_\{\\text\{LM\}\}\(\\cdot\\mid\\mathbf\{x\}\_\{<a\}\)
16:

𝐱′←𝐱<a⊕𝐱suffix′\\mathbf\{x\}^\{\\prime\}\\leftarrow\\mathbf\{x\}\_\{<a\}\\oplus\\mathbf\{x\}^\{\\prime\}\_\{\\text\{suffix\}\}
17:Phase 4: MH\-Style Acceptance

18:

m←min⁡\(\|𝐱\|,\|𝐱′\|\)−a\+1m\\leftarrow\\min\(\|\\mathbf\{x\}\|,\|\\mathbf\{x\}^\{\\prime\}\|\)\-a\+1
19:

log⁡rsuffix←S⁡\(𝐱′,a,m\)−S⁡\(𝐱,a,m\)\\log r\_\{\\text\{suffix\}\}\\leftarrow S\(\\mathbf\{x\}^\{\\prime\},a;m\)\-S\(\\mathbf\{x\},a;m\)
20:Sample

u∼𝒰⁡\(0,1\)u\\sim\\mathcal\{U\}\(0,1\)
21:if

log⁡u<min⁡\(0,log⁡rsuffix\)\\log u<\\min\(0,\\log r\_\{\\text\{suffix\}\}\)then

22:return

𝐱′\\mathbf\{x\}^\{\\prime\}
23:else

24:return

𝐱\\mathbf\{x\}
25:endif