ARBOR：通过推理状态跟踪定位长思维链中的推理失败

# Arbor: Locating Reasoning Failures in Long Chains of Thought through Reasoning State Tracking
Source: [https://arxiv.org/html/2610.22717](https://arxiv.org/html/2610.22717)
Siyuan WangAffiliation:The Chinese University of Hong KongNaga Vamsi Ramana DinavahiAffiliation:University of Southern CaliforniaXiaohui LiuAffiliation:New York University\{huihanl,xiangren\}@usc\.edu,siyuanwang@cuhk\.edu\.hkXiang RenAffiliation:University of Southern California

###### Abstract

Large reasoning models solve complex problems through long chains of thought, in which prior work has documented failure modes such as overthinking, underthinking, and unsupported deductions\. Existing methods detect these failures from*local*signals: textual similarity to nearby steps, switch tokens, or an LLM’s judgment of a step’s correctness\. However, what a step contributes to the solution depends on the state that earlier steps have established, so identifying failures requires a*global*view that tracks that state across the trajectory\. Therefore, we proposeArbor, a reasoning\-failure attribution framework that represents each trajectory as a*tree of reasoning states*, built as each step creates, verifies, prunes, or revisits a state, and attributes each failure to a relation between a step and the state preceding it\. UsingArbor, we refine the three failure modes into distinct subtypes that harm reasoning through different mechanisms and at different costs\. We find that the tree stateArbormaintains before a step separates failure steps from the at\-risk steps at which the failure did not occur\. We discuss howArborintegrates with existing mitigations at inference time and in training data\. Code, data and interactive examples are available at[https://inklab\.usc\.edu/ARBOR](https://inklab.usc.edu/ARBOR)\.

## 1Introduction

Large reasoning models \(LRMs\) solve complex reasoning tasks with long chains of thought \(CoT\) in which they derive intermediate results, verify them, backtrack and pivot to alternatives\. Prior work has documented recurring failure modes in these trajectories: overthinking, in which the model re\-reasons about results it has already established\([Chen et al\., 2025b](https://arxiv.org/html/2610.22717#bib.bib5);[Pipis et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib27)\); underthinking, in which it abandons a line of reasoning that could still succeed, a*premature*switch as opposed to a*justified*switch away from a line already ruled out\([Wang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib33)\); and unsupported deductions, steps the preceding context does not entail\([Lightman et al\., 2024](https://arxiv.org/html/2610.22717#bib.bib17);[He et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib11)\)\. These failures waste inference budget and some produce wrong answers; reducing them requires knowing which steps fail andwhy\.

Existing methods detect these failures with signals that arelocalto a step: recurringnn\-grams or embedding similarity between nearby sentences for repetition\([Pipis et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib27);[Huang et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib13);[Zhen et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib46)\), thought\-switch tokens such as*alternatively*for underthinking\([Wang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib33);[Zhang et al\., 2025b](https://arxiv.org/html/2610.22717#bib.bib43)\), process reward models and LLM judges for step correctness\([Wang et al\., 2024](https://arxiv.org/html/2610.22717#bib.bib31);[Zhang et al\., 2025c](https://arxiv.org/html/2610.22717#bib.bib44);[Zheng et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib47)\), and block\-level functional labels spanning many steps\([Marjanović et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib22);[Luo et al\., 2025b](https://arxiv.org/html/2610.22717#bib.bib21);[Jiang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib14);[Bogdan et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib3)\)\. None conditions a step’s label on which earlier steps it depends on and what they established, although that context is what makes a derivation a repetition, a switch premature or a deduction unsupported \(related work in Appendix[A](https://arxiv.org/html/2610.22717#A1)\)\.

Figure 1:How a step relates to the states already established defines a reasoning failure\.\(a\) An excerpt of aQwen3\-8Btrajectory on a ZebraLogic puzzle; numbers are step indices, and colored steps are labeled by function\. \(b\) Its tree of reasoning states: nodes are intermediate states, tagged with the steps that created or revisited them, edges link each state to the states derived from it, and the case split at step 66 opens two branches\. Steps 87–88 are arepetition: they re\-derive the state of step 68 in new words\. Step 86 isunderthinking: it abandons Case 2 while that branch is still viable\. Step 90 is anunsupported deduction: it contradicts the Case 1 assumption\. Step 113 is ajustified switch: it follows the pruning of Case 1 at step 112\.Figure 2:Local signals miss or misattribute the failures that the reasoning state identifies\.Columns are steps of the trajectory in Figure[1](https://arxiv.org/html/2610.22717#S1.F1)\. For each failure mode, local signals \(gray, dotted\) are compared withArbor\(colored, solid\), whose arrow points to the earlier step whose state determines the label\.*Repetition:*step 88 shares nonn\-gram with step 68, and embedding similarity links it to the clue recap at step 53 rather than to the state it re\-derives\.*Underthinking:*a switch token fires at both step 86 and step 113, but only step 86 leaves a viable branch\.*Unsupported deduction:*the process reward model misses step 90, which violates the Case 1 assumption of step 66, and flags the correct retraction at step 112 instead\.We therefore define reasoning failures relative to thestate of the reasoning, the intermediate results established, the branches open and the constraints in force when a step occurs\. We represent each trajectory as a tree of reasoning states \(Figure[1](https://arxiv.org/html/2610.22717#S1.F1)\), in which nodes are intermediate results, each edge records the node a step derives from, and where a step attaches identifies a failure; Figure[2](https://arxiv.org/html/2610.22717#S1.F2)shows why local signals fail on the same trajectory\.

We make these relations explicit withArbor\(AttributingReasoning failuresByOperations onReasoning\-state trees\), a pipeline in three stages \(Figure[3](https://arxiv.org/html/2610.22717#S2.F3); §[2](https://arxiv.org/html/2610.22717#S2)\): an LLMparseseach step into an operation on a canonical, wording\-invariant payload; a deterministicengineresolves the step against the tree built so far and applies the corresponding mutation; and wedefine failuresas relations between a step and a specific earlier state, which refines the three failure modes into distinct subtypes \(Table[1](https://arxiv.org/html/2610.22717#S2.T1)\)\.

We instantiate the framework on three reasoning tasks with distinct reasoning structures, constraint satisfaction \(ZebraLogic;[Lin et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib18)\), search over arithmetic compositions \(24 Game111nlile/24\-game\) and sequential action planning under preconditions \(PlanBench;[Valmeekam et al\., 2023](https://arxiv.org/html/2610.22717#bib.bib29)\), and report three findings\.

First,Arborflags failures more accurately than local signals \(§[4](https://arxiv.org/html/2610.22717#S4)\)\. The local signals recover only a small share of the stepsArborflags, and what they flag instead is mostly ordinary reasoning\.Arbor’s per\-subtype counts separate incorrect from correct trajectories better than all local signals combined\.

Second, the distinct subtypes of each failure mode differ in cost and mechanism \(§[5](https://arxiv.org/html/2610.22717#S5)\)\. Unsupported deduction costs correctness, since the model’s verification seldom catches a step the state does not support, and the step enters the answer\. Underthinking costs the search expended before the model reaches the answer by another path or by returning to the branch, rather than correctness, since it usually still reaches it\. Repetition does the least harm to the answer and costs few tokens beyond its own steps, since the model re\-establishes a step it proposed long before and gains nothing from it\. Within each mode, the subtypes differ in which of these costs they carry\.

Third, the tree state before a step separates each modeled subtype from the at\-risk steps at which it did not end up occurring \(§[6](https://arxiv.org/html/2610.22717#S6)\)\. The precursor of each modeled subtype is the content of the branch, whether the current node has been verified, or the recent operations\. Operation labels alone recover only the last kind, and the first two precede the subtypes that cost correctness or viable work, which therefore require the tree\.

To our knowledge,Arboris the first reasoning\-failure attribution framework that maintains the solution state of a reasoning trajectory step by step and attributes each failure to a relation between a step and that state, rather than to the step’s text\. Tracking the state enables us to separate the subtypes of a failure mode, measure what each costs in tokens and in correctness, and identify the state that precedes a failure\. We discuss howArborintegrates with existing mitigation at inference time and in training data \(§[6](https://arxiv.org/html/2610.22717#S6)\)\.

## 2Methodology

Figure 3:Arbor’s three\-stage framework\.An LLM parses each step into a canonical operation, then a deterministic engine replays it against the accumulated reasoning state, and deterministic rules map the resulting state relations to functional labels\.RowDefinition: the model …RepetitionR1 Re\-Derivationderives again a result it has already derived, after deriving or verifying others\.R2 Derive\-after\-Prunederives from a hypothesis it has already rejected\.R3 Redundant Re\-Verificationverifies a result again although nothing it depends on has changed\.R4 Subgoal Churnposts a subgoal again after working on others, without progress on it\.UnderthinkingR5 Branch Abandonmentabandons a hypothesis that can still reach a correct solution, even if it later returns\.R6 Subgoal Abandonmentdrops a subgoal that is still achievable and never completes it\.R7 Post\-Answer Attemptbegins searching elsewhere, or re\-verifying, while holding a correct complete solution\.Unsupported DeductionR8 Premature Finalizationcommits an answer that its established results or the task contradict\.R9 Infeasible Derivationderives a result that its established results do not permit\.R10 False Rejectionrejects a hypothesis that can still reach a correct solution, for a false reason\.R11 False Verdictrecords a verification outcome that its established results contradict\.Table 1:The failure rows\. Each row is one relation between a step and the results the model has established before it; §[2\.3](https://arxiv.org/html/2610.22717#S2.SS3)gives the predicate on the reasoning\-state tree\. A row is undefined, rather than approximated, on a task whose structure has no state for it \(Appendix[H](https://arxiv.org/html/2610.22717#A8)\)\.Arbor\(Attributing Reasoning failures By Operations on Reasoning\-state trees\) turns a trajectoryx1,…,xTx\_\{1\},\\dots,x\_\{T\}into a tree of reasoning states in three stages \(Figure[3](https://arxiv.org/html/2610.22717#S2.F3)\): a*parser*maps each step to an operation on a schema\-grounded payload; a deterministic*engine*resolves the step to a node of the tree built so far and applies one mutation; and a*failure row*\(Table[1](https://arxiv.org/html/2610.22717#S2.T1)\) is a predicate on the step, the node it resolves to and the tree before it\. Only the parser calls a model; the same parse always replays to the same tree and the same labels, and every label names the earlier node it is defined against\.

### 2\.1Parser

OperationPayloadEffect on the treeDerivea resultff, or alternatives\{fi\}\\\{f\_\{i\}\\\}adds a child holdingffunder the nodeffdepends on; one child perfif\_\{i\}Assumea hypothesishhadds a child holdinghhunder the current leafVerifya resultff, verdictoorecordsooands⁡\(u\)⊧fs\(u\)\\models fon the nodeuuholdingffPrunea result or hypothesisffcloses the node holdingffand its subtreeConfirman answeraacommits the node holdingaa; recordsaccept⁡\(a\)\\mathrm\{accept\}\(a\)Restatea resultffrecords the restatement on the node holdingffMetanonenoneTable 2:The seven operations: the parser assigns each step one operation and its payload, and the engine applies the effect\. A result is a fact, value, action or subgoal\.The first stage, the parser, assigns every step one of seven*operations*\(Table[2](https://arxiv.org/html/2610.22717#S2.T2)\) together with its*payload*, the schema\-grounded content the operation acts on: a fact, value, action, subgoal, hypothesis or answer\.Deriveadds a result,Assumeopens a branch,Verifyrecords a verdict on a result,Prunecloses a branch,Confirmcommits an answer,Restaterepeats a result without deriving it, andMetacarries no content\. For each task we customize the parser’s prompt with a*puzzle schema*, the entities, the allowed predicates, the allowed operations with their preconditions and effects, the initial state and the goal, and the parser outputs in that schema, so “Bob lives in the fifth house” and “house 5’s name is Bob” parse to the same atom\. We use Qwen3\.8\-27B as the parser; it sees the puzzle schema, the current step, its labeled neighborhood and the schema elements of the previous steps\. The parser reaches an accuracy of 0\.89 or above on adjudicated reference labels on every task \(Appendix[E](https://arxiv.org/html/2610.22717#A5)\)\. The parser thus records only what each step does, in isolation\.

### 2\.2Engine

From these per\-step operations the engine builds the tree of reasoning states, applying each step to the tree built from the steps before it\.

#### Tree\.

A*node*vvholds an intermediate result or hypothesis \(a cell assignment, a value composed from a subset of the tiles, a world state after an action\) together with the*solution state*s⁡\(v\)s\(v\)reached at it: the board with every cell placed so far, the tiles still unused, the facts that hold after the actions taken\. The*lineage*ofvvis the path from the root tovv\. A*branch*is the subtree below a hypothesis\. A node’s*status*is open, closed by the model, contradictory as detected by the engine, or committed\. The*frontier*is the set of open leaves, and the*current leaf*is the frontier leaf at which the model is working\. The*dependency path*of an answer is the lineage of its committed node\.

#### Applying a step\.

On this tree, the engine applies a step by resolving the node its payload refers to,r⁡\(xt\)∈𝒯t−1r\(x\_\{t\}\)\\in\\mathcal\{T\}\_\{t\-1\}, and applies the operation’s effect \(Table[2](https://arxiv.org/html/2610.22717#S2.T2)\)\. ADeriveresolves to the node its content depends on, or to the current leaf when it names none; every other operation resolves to the node holding its payload \(procedure in Appendix[F\.1](https://arxiv.org/html/2610.22717#A6.SS1)\)\. AVerifyorRestaterecords the claim beside what the node holds, so that a claim the state contradicts stays visible to the detectors\.

#### Oracles\.

On the solution state of each node, the engine also evaluates three*oracles*, task\-specific predicates computed from that state and the task definition:sound⁡\(v,f\)\\mathrm\{sound\}\(v,f\), whether a derived factfffollows froms⁡\(v\)s\(v\)and the problem’s rules;viable⁡\(v\)\\mathrm\{viable\}\(v\), whether a correct complete solution is still reachable fromvv\(the puzzle’s unique solution; the arithmetic over the remaining tiles; a planner run froms⁡\(v\)s\(v\)\), a node that is not viable being*dead*; andaccept⁡\(a\)\\mathrm\{accept\}\(a\), whether a committed answeraais correct and complete\.

### 2\.3Failure Detectors

With the tree and its oracles in place, the detectors read each step against them\. Each row of Table[1](https://arxiv.org/html/2610.22717#S2.T1)is a predicateR⁡\(xt,v,𝒯t−1\)R\(x\_\{t\},\\,v,\\,\\mathcal\{T\}\_\{t\-1\}\)on the step, its resolved nodev=r⁡\(xt\)v=r\(x\_\{t\}\)and the tree before it, and a step on which the predicate holds is an*event*of that row\. In the tree’s terms, a hypothesis of Table[1](https://arxiv.org/html/2610.22717#S2.T1)is a branch, the established results are the solution states⁡\(v\)s\(v\), and a solution is reachable whenviable⁡\(v\)\\mathrm\{viable\}\(v\)holds\. The three failure modes differ in what the predicate tests\. Repetition rows test whether a node in𝒯t−1\\mathcal\{T\}\_\{t\-1\}already holds what the step produces, and which such node: one worked away from, one closed, one already verified, or a subgoal already posted\. Underthinking rows testviable⁡\(v\)\\mathrm\{viable\}\(v\)on the node the step leaves, so a switch away from a dead node is justified and fires no row, or whethervvalready holds an accepted solution \(the literature’s overthinking, filed here because the step leaves a finished solution\)\. Unsupported\-deduction rows test the step against the oracles:sound\\mathrm\{sound\}on a derivation,viable\\mathrm\{viable\}on a pruned branch,accept\\mathrm\{accept\}on a committed answer, and the recorded verdict of a verification againsts⁡\(v\)s\(v\)\. The validation of the engine and the detectors is in Appendix[F\.2](https://arxiv.org/html/2610.22717#A6.SS2)\.

## 3Experimental Setup

#### Tasks\.

We use three reasoning tasks with distinct reasoning structures: ZebraLogic\([Lin et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib18)\), constraint satisfaction; the 24 Game222nlile/24\-game, search over arithmetic compositions; and PlanBench\([Valmeekam et al\., 2023](https://arxiv.org/html/2610.22717#bib.bib29)\), sequential action planning under preconditions \(Details in Appendix[B](https://arxiv.org/html/2610.22717#A2)\)\.

#### Models and sampling\.

We study the Qwen3 family; all results are on trajectories ofQwen3\-8B, and the comparison across model sizes is in Appendix[C](https://arxiv.org/html/2610.22717#A3)\. We sample each question three times, with a budget of 8,192 generated tokens on ZebraLogic and the 24 Game and 16,000 on PlanBench\.

#### Evaluation\.

A trajectory ends correct, incorrect or truncated at the token budget; the scoring rule per task and the outcome shares per task and model size are in Appendix[B](https://arxiv.org/html/2610.22717#A2), Table[6](https://arxiv.org/html/2610.22717#A2.T6)\.

ZebraLogiccase splitre\-derivesShape: deep chain,case splitsError: out ofbudget24 Gamecorrect,not committedinfeasiblecommittedShape: shallow fan,most prunedError: wrong oneof twoPlanBenchinfeasible step,check passescommittedShape: chain ofplan stepsError: wrong planestablished statepruned branchcommitted answerInfeasible Derivation

Figure 4:Three tasks, three shapes of reasoning, three failure patterns\.SchematicArbortrees of a typical failing trajectory for each task \(statistics in Appendix[H\.1](https://arxiv.org/html/2610.22717#A8.SS1)\)\.
#### Tree shapes and failure patterns\.

Read offArbor’s trees, a ZebraLogic trajectory is one deep chain of derivations split at a few case assumptions, a 24 Game trajectory is a shallow fan of alternative compositions tried in turn from the same start, and a PlanBench trajectory is a chain of plan steps that rarely branches \(Figure[4](https://arxiv.org/html/2610.22717#S3.F4)\)\. Each shape fails in its own way: the ZebraLogic chain is filled withRe\-Derivationacross its case splits and fails by running out of budget; the 24 Game fan is filled withRedundant Re\-VerificationandDerive\-after\-Pruneand fails by committing an answer built on anInfeasible Derivationwhile a correct one stands; the PlanBench chain accumulates infeasible steps that pass their checks and fails as a wrong plan \(Tables[17](https://arxiv.org/html/2610.22717#A8.T17)and[6](https://arxiv.org/html/2610.22717#A2.T6)\)\.

## 4Arborversus Local Signals

CategoryMethodPrec\.Rec\.RepetitionArbor64%37%n\-gram49%31%similarity62%45%UnderthinkingArbor32%71%switch tokens27%62%Post\-Answer AttemptArbor77%87%ERD11%13%UnsupporteddeductionArbor78%80%PRM9%9%token conf\.11%11%Table 3:Expert review confirmsArborand rejects local signals\.Precision: share of a method’s flagged steps the annotator confirms as the stated failure\. Recall: share of the confirmed steps of the failure mode \(all methods’ steps pooled\) that the method also flags\. For underthinking we also showed the annotators the lineageArbortracked, since the text rarely settles whether the abandoned branch was still viable \(Appendix[G\.2](https://arxiv.org/html/2610.22717#A7.SS2)\)\.#### Local signals\.

We compareArborwith the local signals the literature uses to detect the same failures: for repetition, the*n\-gram*rule of[Pipis et al\. \(2026\)](https://arxiv.org/html/2610.22717#bib.bib27)and*embedding similarity*between adjacent sentence windows \(IRD;[Zhen et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib46)\); for continuing after the answer,*post\-answer redundancy*\(ERD;[Zhen et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib46)\); for underthinking,*switch tokens*\([Wang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib33)\); and for unsupported deduction, a*process reward model*\(PRM, Qwen2\.5\-Math\-PRM\-7B;[Zhang et al\., 2025c](https://arxiv.org/html/2610.22717#bib.bib44)\) and*token confidence*\([Fu et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib9)\), each flagging a trajectory’s lowest\-scored steps\. We set each threshold on development trajectories so that the signal flags the same share of tokens asArbor, which compares precision at equal coverage \(Appendix[G\.1](https://arxiv.org/html/2610.22717#A7.SS1)\)\.

#### Experts confirmArborand reject the local signals\.

Four annotators confirm most ofArbor’s steps and reject most of the local signals’, andArboralso recovers more of the confirmed steps \(Table[3](https://arxiv.org/html/2610.22717#S4.T3)\)\. For repetition the two are comparable, becauseArborrecords part of what the annotators count as repetition as a restatement rather than a re\-derivation\.

#### Local signals flag ordinary reasoning\.

The local signals recover only a small share ofArbor’s steps in every failure mode \(Figure[9\(a\)](https://arxiv.org/html/2610.22717#A7.F9.sf1)\)\. Most of the steps they do flag are ordinary reasoning, first derivations or verifications and discourse such as “Wait” or “Let me see” \(Figure[9\(b\)](https://arxiv.org/html/2610.22717#A7.F9.sf2)\)\. Moreover, half of the steps that switch tokens flag as underthinking actually leave a branch the oracle had already ruled out, so a penalty on switch tokens would suppress justified backtracking as often as underthinking \(Table[14](https://arxiv.org/html/2610.22717#A7.T14)\)\.

![Refer to caption](https://arxiv.org/html/2610.22717v1/local_measures_auroc.png)Figure 5:Arbor’s subtype counts predict the outcome better than all local signals combined\.Gain in AUROC for separating incorrect from correct trajectories over a baseline of trajectory length and task \(solid line, 0\.86\), pooled over the three tasks\. Error bars: 95% confidence intervals\.
#### Arbor’s subtype counts separate incorrect from correct trajectories better than all local signals combined\.

Arbor’s per\-subtype counts separate incorrect from correct trajectories better than any single local signal and better than the five of them together \(Figure[5](https://arxiv.org/html/2610.22717#S4.F5)\), and adding the five local signals toArbor’s does not further increase AUROC \(per\-task and length\-normalized results in Appendix[G](https://arxiv.org/html/2610.22717#A7)\)\. This indicates that the local signals only carry a subset of whatArborencodes\.

## 5Cost and Mechanism of Each Failure Mode

Token cost \(%\)OR of endingSubtypeselfext\.incorr\.trunc\.RepetitionR1 Re\-Derivation1\.00\.30\.980\.82R2 Derive\-after\-Prune1\.00\.40\.661\.42R3 Redundant Re\-Verification1\.20\.30\.370\.45R4 Subgoal Churn0\.20\.31\.281\.15UnderthinkingR5 Branch Abandonment0\.320\.81\.070\.64R6 Subgoal Abandonment0\.0646\.71\.241\.18R7 Post\-Answer Attempt0\.20\.70\.080\.77Unsupported deductionR8 Premature Finalization0\.044\.910\.120\.78R9 Infeasible Derivation1\.22\.62\.901\.83R10 False Rejection0\.237\.50\.871\.00R11 False Verdict0\.40\.41\.700\.91

Table 4:The cost of each subtype\(Qwen3\-8B, pooled over the three tasks; measurements as defined in §[5](https://arxiv.org/html/2610.22717#S5)\)\. OR: odds ratio\. Per\-task values, intervals and significance in Table[17](https://arxiv.org/html/2610.22717#A8.T17)\.Figure 6:The evidence behind §[5](https://arxiv.org/html/2610.22717#S5), per task \(Qwen3\-8B; exact values in Tables[21](https://arxiv.org/html/2610.22717#A8.T21)to[25](https://arxiv.org/html/2610.22717#A8.T25)\)\. In \(d\), the label above each bar is the share of failing trajectories that held a correct solution; in \(h\), PlanBench has no branch lifecycle\.For each failure mode, we ask two questions\.Q1: How does it harm reasoning, in token cost, in answer correctness, or both?Q2: What mechanism lets it arise or persist?

We use the following measurements in our analysis, detailed below: \(1\)*Self cost*: the share of all tokens on the steps that carry the subtype\. \(2\)*Extended cost*: the share of a trajectory’s tokens spent in the region of the tree that an event of the subtype opens, defined per subtype \(for an abandonment, the steps generated outside the abandoned branch until the model returns to it; for an unsupported deduction, the subtree rooted at the offending node\), as the median over events\. \(3\)*Incorrect and truncation odds ratios*: how much likelier a trajectory containing the subtype is to end incorrect rather than correct, or truncated rather than completed, than a trajectory of the same length on the same task\. The main text reports only the values that characterize each subtype; Appendix[H](https://arxiv.org/html/2610.22717#A8)gives every statistic for every subtype and task\.

### 5\.1Unsupported Deduction

Unsupported deduction is the failure mode that costs correctness: the model’s own verification seldom catches a step the state does not support, and the committed answer builds on it\. Three of its four subtypes have the largest incorrect odds ratios of any subtype, at small extended cost \(Table[4](https://arxiv.org/html/2610.22717#S5.T4)\)\. Verification catches anInfeasible Derivationin under a third of cases on every task \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)a\)\.Infeasible Derivationlies on the dependency path of most wrong answers and few correct ones on every task \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)b\)\.

The exception isFalse Rejection, which costs tokens rather than correctness: verification wrongly rejects a viable branch, and the model rarely returns to a branch it has rejected\. Its incorrect odds ratio is below 1 but its extended cost is 37\.5% of the trajectory \(Table[4](https://arxiv.org/html/2610.22717#S5.T4)\)\. The model returns to a rejected branch far less often than to an abandoned one on every task \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)c\)\.

### 5\.2Underthinking

Underthinking costs tokens rather than correctness: the model typically still reaches the answer, whether by returning to the abandoned branch or by an alternative path, and the cost of underthinking is the search expended in between\. Branch andSubgoal Abandonmenthave the two largest extended costs of any subtype with incorrect odds ratios near 1 \(Table[4](https://arxiv.org/html/2610.22717#S5.T4)\)\. The model returns to the abandoned branch in between a fifth and seven tenths of cases, depending on the task \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)c\)\.

The exception isPost\-Answer Attempt, which costs correctness: the model leaves a complete correct solution rather than an unfinished branch, and rarely returns to it\. Between a sixth and a third of failing trajectories held a correct solution, and after the attempt most of them commit a wrong answer or run out of budget \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)d\)\. Its incorrect odds ratio of 0\.08 reflects that most such attempts occur in trajectories that end correct \(Table[4](https://arxiv.org/html/2610.22717#S5.T4)\)\.

### 5\.3Repetition

Repetition does less harm to the answer than the other two modes and costs few tokens beyond its own steps: the model re\-establishes a step it had proposed long before in the trajectory, and the re\-establishment adds nothing to what that step had already given\. Its subtypes have the largest self costs but the smallest extended costs of any subtype, and incorrect odds ratios below 1 except forSubgoal Churn\(Table[4](https://arxiv.org/html/2610.22717#S5.T4)\)\. The re\-established step lies tens of steps back for every subtype and task \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)e\)\. A re\-verification flips the verdict in under 5% of cases, and later work rarely builds on aRe\-Derivation’s copy \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)f\)\.

The budget that repetition does not consume goes to dead ends: truncated trajectories spend more of their tokens on steps that can no longer reach a correct solution than trajectories that reach an answer do, on every task \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)g\)\. The exception within repetition isDerive\-after\-Prune, the one subtype associated with truncation \(Table[4](https://arxiv.org/html/2610.22717#S5.T4)\): the model re\-enters a branch that was in fact dead, without re\-opening it, in the large majority of cases \(Figure[6](https://arxiv.org/html/2610.22717#S5.F6)h\)\.

## 6Precursors of Failure in the Tree State

#### Setup\.

Section[5](https://arxiv.org/html/2610.22717#S5)established what each failure costs and how it arises; a mitigation, however, must act before the failure\. We therefore ask whether the tree state before a step already tells a failure apart from the steps at which it could have occurred but did not, which we call*at\-risk steps*\. For each subtype we take the steps at which it occurred and, as controls, at\-risk steps, on which its precondition held, matched on task, operation and position\. We describe each step by the tree state and the operations over the preceding steps and train a classifier to separate the two sets on held\-out problems, against two baselines, position alone and the previous operation alone \(Appendix[I](https://arxiv.org/html/2610.22717#A9)\)\.

AUROCKind of stateSubtypeStateLabelsBranch contentR1 Re\-Derivation0\.730\.70R2 Derive\-after\-Prune0\.590\.51R4 Subgoal Churn0\.770\.64R9 Infeasible Derivation0\.780\.71Current node verified?R5 Branch Abandonment0\.910\.81Recent operationsR3 Redundant Re\-Verification0\.830\.81R7 Post\-Answer Attempt0\.870\.87

Table 5:What the precursor of each subtype reads, and whether the tree is needed to read it\(24 Game\)\. State: held\-out AUROC of a classifier on the full tree state over the preceding steps, separating the steps at which the subtype occurred from matched at\-risk steps, at which it could have occurred but did not\. Labels: the same classifier given only the operation label of each preceding step, such as derive or check, and the position in the trajectory\. The other tasks and the position and previous\-operation baselines are in Table[27](https://arxiv.org/html/2610.22717#A9.T27)\.
#### The tree state is crucial for separating a failure from its at\-risk steps\.

For each subtype, the stable precursor is the set of state features the classifier selects in every refit, stated as a rule in Table[31](https://arxiv.org/html/2610.22717#A9.T31)\. These rules fall into three kinds of state \(Table[5](https://arxiv.org/html/2610.22717#S6.T5)\)\. The first is*branch content*, the constraints fixed, assumptions opened and nodes created on the branch; the second is*whether the current node has been verified*; the third is*recent operations*, the last few operation labels\. The tree adds information beyond the operation labels and position: a classifier refit on labels and position alone separates the subtypes whose precursor is recent operations as well as the full state does, but falls clearly short on those whose precursor is branch content or the verification of the current node \(Table[5](https://arxiv.org/html/2610.22717#S6.T5)\), and these latter two kinds precede the subtypes that build wrong answers or leave viable work behind\. Table[5](https://arxiv.org/html/2610.22717#S6.T5)reports the 24 Game, where the state separates every modeled subtype well above both baselines; on ZebraLogic and PlanBench the separation is weaker and strong only forDerive\-after\-PruneandInfeasible Derivationrespectively \(Table[27](https://arxiv.org/html/2610.22717#A9.T27)\)\.

#### Two routes carryArbor’s offline signals into model development\.

\(1\) At inference time, we train a lightweight predictor onArbor’s labels, a probe on the model’s hidden states or a monitor over the trajectory so far\([Kortukov et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib16);[Baker et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib1)\), which fires when one of the three kinds of state precedes a failure\. Its signal gives existing interventions, a verification prompt, a thought\-switch penalty or early exit\([Wu et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib36);[Wang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib33);[Muennighoff et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib24)\), a more accurate precondition than the unconditional one they use now; Table[30](https://arxiv.org/html/2610.22717#A9.T30)lists the condition each intervention gates on\. \(2\) Offline,Arbor’s labels mark the repetition steps and dead\-end subtrees that trajectory compression should remove, the repeated operation that a reinforcement\-learning penalty should charge, and the failures by which sample selection should rank trajectories, in place of the token\-count and token\-importance criteria these methods use today\([Xia et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib37);[Luo et al\., 2025a](https://arxiv.org/html/2610.22717#bib.bib20);[Hassid et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib10)\)\. In both routes the intervention already exists;Arborsupplies the condition for when to act or the label for what to keep\.

## 7Conclusion

We presentedArbor, a reasoning\-failure attribution framework that maintains the solution state of a reasoning trajectory step by step and attributes each failure to a relation between a step and that state rather than to the step’s text\. On three reasoning tasks, we showed thatArborflags reasoning failures more accurately than local signals and finds the failures they miss\. We usedArborto separate the subtypes of each failure mode, measure their cost in tokens and in correctness, and explain the mechanism by which each arises\. We used the tree stateArbortracks to separate a failure from the at\-risk steps at which it did not occur, and found that for the subtypes that cost correctness or viable work only the tree records that state\. Finally, we discussed howArbor’s labels can gate inference\-time interventions and replace length\-based criteria in offline trajectory compression, reward shaping and sample selection\.

## Limitations

Three limitations bound the scope of our findings\. First, we developed and evaluatedArboron the Qwen3 model family: we tuned the parser’s schemas and retrieved examples to this family’s reasoning style, and established its accuracy only on reference labels drawn from it\. Applying the framework to another model family requires re\-tuning and re\-validating the parser before one can trust the engine’s state, and therefore every downstream count\. Second, the engine presupposes a task whose answers are verifiable and whose search space is finite, so that it can represent the state of a branch and decide its viability and the soundness of a step\. This confines the present study to tasks of that kind; open\-ended reasoning, mathematical proof and agentic workflows, in which the state is not readily parseable and the notion of a viable branch is not well defined, fall outside its scope, and we leave defining engine states for them to future work\. Third, the intervention points and the mitigations built on them are untested proposals derived from the pre\-state analysis\. Whether gating existing mitigations on these states, or training on trajectories curated withArbor’s labels, reduces the failures in practice remains open, and we encourage future work to test it\.

## Ethical Considerations

Arborlabels failures in model\-generated reasoning on synthetic puzzle and planning tasks; it processes no personal data and makes no decisions about people\. The four expert annotators were NLP researchers who volunteered, judged only model\-generated steps, and saw no personal information\.

The labels are definitions over a tracked state, not ground truth about why a model reasons as it does, so a mitigation trained or gated on them inherits any error of the parser or engine; we report the parser’s accuracy and the engine’s validation so that users can weigh that error\. Reducing repetition or switching on the basis of these labels could suppress useful deliberation if applied beyond the tasks on which we validated them, and a monitor built on the labels should not be read as evidence about a model’s reasoning outside those tasks\.

The environmental cost of this work is the GPU inference for generating and parsing trajectories \(§[2\.1](https://arxiv.org/html/2610.22717#S2.SS1), Appendix[B](https://arxiv.org/html/2610.22717#A2)\); the engine and detectors run on a CPU\. Parsing adds one LLM call per step, a cost a deployed monitor would also incur\.

## Release and License

We release the parser schemas and prompts, the engine, the detector definitions and the analysis scripts at[https://github\.com/INK\-USC/ARBOR](https://github.com/INK-USC/ARBOR)under the MIT license, and the full parsed trajectories, their trees and the analysis traces behind every result at[doi:10\.5281/zenodo\.23405752](https://doi.org/10.5281/zenodo.23405752)under CC BY 4\.0\. The project page \([https://inklab\.usc\.edu/ARBOR](https://inklab.usc.edu/ARBOR)\) hosts interactive examples of trajectories and their trees\.

## Use of AI Assistants

Large language models are part of the method: a Qwen3 model parses each step \(§[2\.1](https://arxiv.org/html/2610.22717#S2.SS1)\), and LLM judges re\-labeled the reference trajectories and validated engine decisions under written rules with human adjudication \(Appendices[E](https://arxiv.org/html/2610.22717#A5)and[F\.2](https://arxiv.org/html/2610.22717#A6.SS2)\)\. We also used an AI assistant to edit the text of this paper and to write analysis scripts; the authors reviewed every edit and every result\.

## References

- Baker et al\. \(2025\)Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y\. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi\. 2025\.[Monitoring reasoning models for misbehavior and the risks of promoting obfuscation](https://arxiv.org/abs/2503.11926)\.*Preprint*, arXiv:2503\.11926\.
- Baumann et al\. \(2025\)Joachim Baumann, Paul Röttger, Aleksandra Urman, Albert Wendsjö, Flor Miriam Plaza\-del Arco, Johannes B\. Gruber, and Dirk Hovy\. 2025\.[Large language model hacking: Quantifying the hidden risks of using LLMs for text annotation](https://arxiv.org/abs/2509.08825)\.*Preprint*, arxiv:2509\.08825\.
- Bogdan et al\. \(2025\)Paul C\. Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy\. 2025\.[Thought anchors: Which LLM reasoning steps matter?](https://arxiv.org/abs/2506.19143)*Preprint*, arxiv:2506\.19143\.
- Chen et al\. \(2025a\)Runjin Chen, Zhenyu Zhang, Junyuan Hong, Souvik Kundu, and Zhangyang Wang\. 2025a\.[SEAL: Steerable reasoning calibration of large language models for free](https://arxiv.org/abs/2504.07986)\.In*Proceedings of the Second Conference on Language Modeling \(COLM\)*\.
- Chen et al\. \(2025b\)Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu\. 2025b\.[Do NOT think that much for 2\+3=? on the overthinking of long reasoning models](https://arxiv.org/abs/2412.21187)\.In*Proceedings of the 42nd International Conference on Machine Learning*, volume 267 of*Proceedings of Machine Learning Research*, pages 9487–9499\. PMLR\.
- Cui et al\. \(2025\)Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, Suhang Wang, Yue Xing, Jiliang Tang, and Qi He\. 2025\.[Stepwise perplexity\-guided refinement for efficient chain\-of\-thought reasoning in large language models](https://doi.org/10.18653/v1/2025.findings-acl.956)\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 18581–18597, Vienna, Austria\. Association for Computational Linguistics\.
- Dhuliawala et al\. \(2023\)Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston\. 2023\.[Chain\-of\-verification reduces hallucination in large language models](https://arxiv.org/abs/2309.11495)\.*Preprint*, arXiv:2309\.11495\.
- Feng et al\. \(2026\)Yunzhen Feng, Julia Kempe, Cheng Zhang, Parag Jain, and Anthony Hartshorn\. 2026\.[What characterizes effective reasoning? Revisiting length, review, and structure of CoT](https://arxiv.org/abs/2509.19284)\.In*Proceedings of the 43rd International Conference on Machine Learning \(ICML\)*\.
- Fu et al\. \(2025\)Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao\. 2025\.[Deep think with confidence](https://arxiv.org/abs/2508.15260)\.*Preprint*, arxiv:2508\.15260\.
- Hassid et al\. \(2025\)Michael Hassid, Gabriel Synnaeve, Yossi Adi, and Roy Schwartz\. 2025\.[Don’t overthink it\. preferring shorter thinking chains for improved LLM reasoning](https://arxiv.org/abs/2505.17813)\.*Preprint*, arXiv:2505\.17813\.
- He et al\. \(2025\)Yancheng He, Shilong Li, Jiaheng Liu, Weixun Wang, Xingyuan Bu, Ge Zhang, Zhongyuan Peng, Zhaoxiang Zhang, Zhicheng Zheng, Wenbo Su, and Bo Zheng\. 2025\.[Can large language models detect errors in long chain\-of\-thought reasoning?](https://doi.org/10.18653/v1/2025.acl-long.905)In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 18468–18489, Vienna, Austria\. Association for Computational Linguistics\.
- He et al\. \(2026\)Zeyu He, Zhuqian Zhou, Kirk Vanacore, Rene F\. Kizilcec, and Ting\-Hao ’Kenneth’ Huang\. 2026\.[Experts rise where LLMs disagree: Using cross\-model disagreement to target expert effort in LLM codebook revision for large\-scale annotation](https://arxiv.org/abs/2609.26926)\.*Preprint*, arxiv:2609\.26926\.
- Huang et al\. \(2026\)Kerui Huang, Shuhan Liu, Xing Hu, Tongtong Xu, Lingfeng Bao, and Xin Xia\. 2026\.[SEER: Self\-enhancing chain\-of\-thought compression for reasoning models](https://doi.org/10.1145/3832120)\.3\(ISSTA\):664–686\.
- Jiang et al\. \(2025\)Gangwei Jiang, Yahui Liu, Zhaoyi Li, Wei Bi, Fuzheng Zhang, Linqi Song, Ying Wei, and Defu Lian\. 2025\.[What makes a good reasoning chain? Uncovering structural patterns in long chain\-of\-thought reasoning](https://doi.org/10.18653/v1/2025.emnlp-main.329)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 6490–6514, Suzhou, China\. Association for Computational Linguistics\.
- Kim et al\. \(2026\)Kon Woo Kim, Jin\-Dong Kim, and Akiko Aizawa\. 2026\.[Refining and reusing annotation guidelines for LLM annotation](https://aclanthology.org/2026.acl-long.1760/)\.In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*\. Association for Computational Linguistics\.
- Kortukov et al\. \(2026\)Evgenii Kortukov, Piotr Komorowski, Florian Klein, Paula Engl, Gabriele Sarti, Seong Joon Oh, Sebastian Lapuschkin, and Wojciech Samek\. 2026\.[Predicting future behaviors in reasoning models enables better steering](https://arxiv.org/abs/2606.11172)\.*Preprint*, arXiv:2606\.11172\.
- Lightman et al\. \(2024\)Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe\. 2024\.[Let’s verify step by step](https://arxiv.org/abs/2305.20050)\.In*The Twelfth International Conference on Learning Representations \(ICLR\)*\.
- Lin et al\. \(2025\)Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi\. 2025\.[ZebraLogic: On the scaling limits of LLMs for logical reasoning](https://arxiv.org/abs/2502.01100)\.In*Proceedings of the 42nd International Conference on Machine Learning*, volume 267 of*Proceedings of Machine Learning Research*, pages 37889–37905\. PMLR\.
- Liu et al\. \(2026\)Alex Liu, Lief Esbenshade, Michael Xiao, Victor Tian, Zachary Zhang, Kevin He, and Min Sun\. 2026\.[Agreement is not quality: Blind expert verification of human and LLM qualitative coding when human consensus is not ground truth](https://arxiv.org/abs/2607.28890)\.*Preprint*, arxiv:2607\.28890\.
- Luo et al\. \(2025a\)Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao\. 2025a\.[O1\-Pruner: Length\-harmonizing fine\-tuning for O1\-like reasoning pruning](https://arxiv.org/abs/2501.12570)\.*Preprint*, arXiv:2501\.12570\.
- Luo et al\. \(2025b\)Yijia Luo, Yulin Song, Xingyao Zhang, Jiaheng Liu, Weixun Wang, GengRu Chen, Wenbo Su, and Bo Zheng\. 2025b\.[Deconstructing long chain\-of\-thought: A structured reasoning optimization framework for long CoT distillation](https://arxiv.org/abs/2503.16385)\.*Preprint*, arxiv:2503\.16385\.
- Marjanović et al\. \(2026\)Sara Vera Marjanović, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lù, Nicholas Meade, Dongchan Shin, Amirhossein Kazemnejad, Gaurav Kamath, Marius Mosbach, Karolina Stańczak, and Siva Reddy\. 2026\.[DeepSeek\-R1 thoughtology: Let’s think about LLM reasoning](https://arxiv.org/abs/2504.07128)\.
- Minegishi et al\. \(2025\)Gouki Minegishi, Hiroki Furuta, Takeshi Kojima, Yusuke Iwasawa, and Yutaka Matsuo\. 2025\.[Topology of reasoning: Understanding large reasoning models through reasoning graph properties](https://arxiv.org/abs/2506.05744)\.In*Advances in Neural Information Processing Systems 38 \(NeurIPS 2025\)*\.
- Muennighoff et al\. \(2025\)Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei\-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto\. 2025\.[s1: Simple test\-time scaling](https://doi.org/10.18653/v1/2025.emnlp-main.1025)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 20275–20321, Suzhou, China\. Association for Computational Linguistics\.
- Munkhbat et al\. \(2025\)Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se\-Young Yun\. 2025\.Self\-training elicits concise reasoning in large language models\.In*Findings of the Association for Computational Linguistics: ACL 2025*\.ArXiv:2502\.20122\.
- Pangakis et al\. \(2023\)Nicholas Pangakis, Samuel Wolken, and Neil Fasching\. 2023\.[Automated annotation with generative AI requires validation](https://arxiv.org/abs/2306.00176)\.*Preprint*, arxiv:2306\.00176\.
- Pipis et al\. \(2026\)Charilaos Pipis, Shivam Garg, Vasilis Kontonis, Vaishnavi Shrivastava, Akshay Krishnamurthy, and Dimitris Papailiopoulos\. 2026\.[Wait, wait, wait… why do reasoning models loop?](https://arxiv.org/abs/2512.12895)In*Proceedings of the 43rd International Conference on Machine Learning*, volume 306 of*Proceedings of Machine Learning Research*, pages 98934–98964\. PMLR\.
- Sun et al\. \(2026\)Zhongxiang Sun, Qipeng Wang, Haoyu Wang, Xiao Zhang, and Jun Xu\. 2026\.[Mechanistic detection and mitigation of hallucination in large reasoning models](https://arxiv.org/abs/2505.12886)\.In*The Fourteenth International Conference on Learning Representations \(ICLR\)*\.
- Valmeekam et al\. \(2023\)Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati\. 2023\.[PlanBench: An extensible benchmark for evaluating large language models on planning and reasoning about change](https://arxiv.org/abs/2206.10498)\.In*Advances in Neural Information Processing Systems 36 \(Datasets and Benchmarks Track\)*\. Curran Associates, Inc\.
- Venhoff et al\. \(2025\)Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Conmy, and Neel Nanda\. 2025\.[Understanding reasoning in thinking language models via steering vectors](https://arxiv.org/abs/2506.18167)\.*Preprint*, arxiv:2506\.18167\.
- Wang et al\. \(2024\)Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui\. 2024\.[Math\-shepherd: Verify and reinforce LLMs step\-by\-step without human annotations](https://doi.org/10.18653/v1/2024.acl-long.510)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 9426–9439, Bangkok, Thailand\. Association for Computational Linguistics\.
- Wang et al\. \(2026\)Siyuan Wang, Yanchen Liu, and Xiang Ren\. 2026\.[Segment\-level attribution for selective learning of long reasoning traces](https://arxiv.org/abs/2602.00425)\.In*The Fourteenth International Conference on Learning Representations \(ICLR\)*\.
- Wang et al\. \(2025\)Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu\. 2025\.[Thoughts are all over the place: On the underthinking of long reasoning models](https://arxiv.org/abs/2501.18585)\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Ward et al\. \(2025\)Jake Ward, Chuqiao Lin, Constantin Venhoff, and Neel Nanda\. 2025\.[Reasoning\-finetuning repurposes latent representations in base models](https://arxiv.org/abs/2507.12638)\.*Preprint*, arXiv:2507\.12638\.
- Wei et al\. \(2026\)Zihao Wei, Liang Pang, Jiahao Liu, Wenjie Shi, Jingcheng Deng, Shicheng Xu, Zenghao Duan, Fei Sun, Huawei Shen, and Xueqi Cheng\. 2026\.[The evolution of thought: Tracking LLM overthinking via reasoning dynamics analysis](https://doi.org/10.18653/v1/2026.acl-long.1239)\.In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 26905–26920, San Diego, California, United States\. Association for Computational Linguistics\.
- Wu et al\. \(2025\)Tong Wu, Chong Xiang, Jiachen T\. Wang, G\. Edward Suh, and Prateek Mittal\. 2025\.[Effectively controlling reasoning models through thinking intervention](https://arxiv.org/abs/2503.24370)\.*Preprint*, arXiv:2503\.24370\.
- Xia et al\. \(2025\)Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li\. 2025\.TokenSkip: Controllable chain\-of\-thought compression in LLMs\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*\.ArXiv:2502\.12067\.
- Xie et al\. \(2025\)Wenya Xie, Shaochen Zhong, Hoang Anh Duy Le, Zhaozhuo Xu, Jianwen Xie, and Zirui Liu\. 2025\.[Word salad chopper: Reasoning models waste a ton of decoding budget on useless repetitions, self\-knowingly](https://doi.org/10.18653/v1/2025.emnlp-main.1705)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 33588–33598, Suzhou, China\. Association for Computational Linguistics\.
- Yang et al\. \(2025\)Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Minghui Chen, Zheng Lin, and Weiping Wang\. 2025\.[Dynamic early exit in reasoning models](https://arxiv.org/abs/2504.15895)\.*Preprint*, arXiv:2504\.15895\.
- Yao et al\. \(2023\)Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan\. 2023\.[Tree of thoughts: Deliberate problem solving with large language models](https://arxiv.org/abs/2305.10601)\.In*Advances in Neural Information Processing Systems*, volume 36, pages 11809–11822\. Curran Associates, Inc\.
- Yuan et al\. \(2026\)Hongyuan Yuan, Xinran He, Run Shao, Bolei He, Xianwei Xue, Mengke Chen, Qiutong Pan, Haiwei Wang, and Haifeng Li\. 2026\.[Graph\-based chain\-of\-thought pruning for reducing redundant reflections in reasoning LLMs](https://doi.org/10.18653/v1/2026.findings-acl.281)\.In*Findings of the Association for Computational Linguistics: ACL 2026*, pages 5708–5723, San Diego, California, United States\. Association for Computational Linguistics\.
- Zhang et al\. \(2025a\)Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He\. 2025a\.[Reasoning models know when they’re right: Probing hidden states for self\-verification](https://arxiv.org/abs/2504.05419)\.*Preprint*, arXiv:2504\.05419\.
- Zhang et al\. \(2025b\)Xichen Zhang, Sitong Wu, Haoru Tan, Shaozuo Yu, Yinghao Zhu, Ziyi He, and Jiaya Jia\. 2025b\.[SmartSwitch: Advancing LLM reasoning by overcoming underthinking via promoting deeper thought exploration](https://arxiv.org/abs/2510.19767)\.*Preprint*, arxiv:2510\.19767\.
- Zhang et al\. \(2025c\)Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin\. 2025c\.[The lessons of developing process reward models in mathematical reasoning](https://doi.org/10.18653/v1/2025.findings-acl.547)\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 10495–10516, Vienna, Austria\. Association for Computational Linguistics\.
- Zhao et al\. \(2025\)Shangziqi Zhao, Jiahao Yuan, Jinyang Wu, Zhenglin Wang, Guisong Yang, and Usman Naseem\. 2025\.[Can pruning improve reasoning? revisiting long\-CoT compression with capability in mind for better reasoning](https://arxiv.org/abs/2505.14582)\.*Preprint*, arXiv:2505\.14582\.
- Zhen et al\. \(2025\)Taihang Zhen, Jialiang Hong, Kai Chen, Guang Yang, Junlan Feng, Wenpeng Zhu, Jing Huo, Yang Gao, Depeng Wang, Haitao Wan, Xi Yang, Fanyu Meng, Yuyao Zhang, Ji Qi, and Xiangyu Zhou\. 2025\.[Reconsidering overthinking: Penalizing internal and external redundancy in CoT reasoning](https://arxiv.org/abs/2508.02178)\.*Preprint*, arxiv:2508\.02178\.
- Zheng et al\. \(2025\)Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin\. 2025\.[ProcessBench: Identifying process errors in mathematical reasoning](https://doi.org/10.18653/v1/2025.acl-long.50)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 1009–1024, Vienna, Austria\. Association for Computational Linguistics\.

## Appendix ARelated Work

#### Local proxies for the three failure modes\.

Prior work detects repetition as verbatim looping, by the rule that a response loops if somenn\-gram recurs at leastkktimes\([Pipis et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib27)\), or as embedding similarity between nearby sentence windows\([Zhen et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib46)\)\. It counts overthinking as the solution rounds after the first correct answer\([Chen et al\., 2025b](https://arxiv.org/html/2610.22717#bib.bib5)\), and underthinking as the token share spent after the first correct step is abandoned and the frequency of thought\-switch tokens\([Wang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib33)\)\. Erroneous steps are the domain of process supervision, with step labels from human annotation\([Lightman et al\., 2024](https://arxiv.org/html/2610.22717#bib.bib17)\), Monte\-Carlo rollouts\([Wang et al\., 2024](https://arxiv.org/html/2610.22717#bib.bib31);[Zhang et al\., 2025c](https://arxiv.org/html/2610.22717#bib.bib44)\)or hidden\-state scores\([Sun et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib28)\), and benchmarks that ask a verifier for the first wrong step\([Zheng et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib47)\)\. All of these signals are defined over the text of a step or of the whole trace, and none records how a step depends on earlier steps\.

#### Structured views of reasoning traces\.

Trees and graphs over steps first served to generate reasoning under explicit search\([Yao et al\., 2023](https://arxiv.org/html/2610.22717#bib.bib40)\)\. For analysis, prior work segments a trace at cue words such as*wait*or*alternatively*and labels each block by its function, such as execution, reflection or transition\([Chen et al\., 2025a](https://arxiv.org/html/2610.22717#bib.bib4);[Wang et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib32)\), or an LLM maps each trace step to a step of an extracted sketch\([Jiang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib14)\), to a successful or failed attempt\([Feng et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib8)\)or to a node of a dependency graph\([Yuan et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib41)\)\. A separate strand clusters hidden states into graph nodes that carry no symbolic content\([Minegishi et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib23)\)\. In each case the structure comes from the wording of a block or from a judge, and it records what a block looks like rather than what it does to the solution state\.

#### Proxies as the basis for interventions\.

The proxies above directly drive mitigations: training\-side compression and selection by solution\-round counts, budget forcing or perplexity criteria\([Chen et al\., 2025b](https://arxiv.org/html/2610.22717#bib.bib5);[Muennighoff et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib24);[Cui et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib6);[Wang et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib32)\), RL penalties on redundancy or hidden\-state scores\([Zhen et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib46);[Sun et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib28)\), and decoding\-side suppression of switch tokens, repeated chunks or reflection states, early exit, and pruning of review nodes\([Wang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib33);[Xie et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib38);[Wei et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib35);[Chen et al\., 2025a](https://arxiv.org/html/2610.22717#bib.bib4);[Yuan et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib41)\)\. Each intervention carries an implicit label, that repeated or post\-answer text is useless or that a switch is harmful, which no method checks at the individual step, and the accuracy lost on removing pre\-answer redundancy\([Zhen et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib46)\)is one sign that the label is too coarse\.

## Appendix BExperimental Details

### B\.1Tasks and datasets

We use three tasks with distinct reasoning structures\.ZebraLogic\([Lin et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib18)\): constraint satisfaction over a row of houses; we use the 1,000grid\_modepuzzles \(25 grid sizes from2×22\{\\times\}2to6×66\{\\times\}6, 40 puzzles each\), each with a unique solution\.333WildEval/ZebraLogic\.24 Game: search over arithmetic compositions; we use the 1,362 solvable puzzles of the train split \(the dataset has no test split\)\.444nlile/24\-game\.PlanBench\([Valmeekam et al\., 2023](https://arxiv.org/html/2610.22717#bib.bib29)\): sequential action planning under preconditions, one\-shot plan generation\.555tasksource/planbench,task\_1\_plan\_generation\.We generated for the first 1,664 of the benchmark’s 2,270 questions, and every analysis uses that set: Blocksworld 500, Depots 500, Obfuscated\-Deceptive Logistics 285, Mystery Blocksworld 179, Mystery Blocksworld\-3 100 and Blocksworld\-3 100; the set does not cover plain Logistics\. All three benchmarks are public and we use them as released, for research\.

### B\.2Models and generation

We study Qwen3 at three sizes — 1\.7B, 8B and 32B — in thinking mode; the 8B model is the main corpus and the other two sizes serve the size comparison\. We send each question as a single user message through the model’s chat template with thinking enabled, and generate withvLLMat a fixed seed of 42\. ZebraLogic and 24 Game use temperature 0\.7, top\-pp1\.0,n=3n=3samples per question and a maximum of 8,192 generated tokens\. PlanBench uses the model’s recommended thinking settings — temperature 0\.6, top\-pp0\.95, top\-kk20 — and a maximum of 16,000 tokens; we drew eight samples per question at 1\.7B and 8B and three at 32B, and use only the first three anywhere\. Every model receives the same task\-specific prompt templates \(Appendix[D](https://arxiv.org/html/2610.22717#A4)\)\. This gives 3,000 ZebraLogic, 4,086 24 Game and 4,992 PlanBench trajectories per model size\. We exclude from every number in the main text the 40 questions per task on which we developed the parser and engine \(Appendix[E\.1](https://arxiv.org/html/2610.22717#A5.SS1)\), leaving 2,880, 3,966 and 4,872 trajectories at 8B\. The size comparison uses a panel of 90 questions per task, balanced over the 8B outcome \(solved in 3, 1–2, or 0 of its samples\) and over length, and run at all three sizes\. We ran all generation and parsing on NVIDIA RTX A6000, RTX 8000 and A100 GPUs\. Qwen3 is released under the Apache\-2\.0 license\.

### B\.3Evaluation

A deterministic, task\-specific rule scores each trajectory on the answer that follows the thinking\. A trajectory that reaches the token limit without an answer is*truncated*; one whose answer cannot be read in the required format is*invalid*; the rest are correct or incorrect\.ZebraLogicexpects a JSON grid; it is correct when every house’s attributes equal the unique solution\.24 GameexpectsANSWER: <expression\>; the expression must use exactly the four given numbers \(otherwise invalid\) and is correct when it evaluates to 24;NO\_SOLUTIONis incorrect since every puzzle is solvable\.PlanBenchexpects one action per line ending in\[PLAN END\]; we score*plan validity*— we replay the plan from the initial state, and it is correct when every action is applicable and the goal holds at the end — rather than exact match to the benchmark’s single gold plan, since many valid plans exist \(at 8B, 25\.6% of trajectories are valid against 15\.2% exact matches; at 32B, 57\.1% against 27\.5%\)\. Table[6](https://arxiv.org/html/2610.22717#A2.T6)reports the outcome shares on the full corpora\.

Table 6:Outcome shares \(%\) per task and model size over the first three samples of every question \(ZebraLogic 3,000, 24 Game 4,086, PlanBench 4,992 trajectories per size\)\. PlanBench “correct” is plan validity; “invalid” is no plan emitted\.TaskModelCorrectTruncatedInvalidZebraLogicQwen3\-1\.7B49\.039\.60\.0Qwen3\-8B70\.227\.00\.0Qwen3\-32B75\.422\.90\.024 GameQwen3\-1\.7B80\.86\.412\.1Qwen3\-8B78\.07\.010\.8Qwen3\-32B78\.85\.714\.8PlanBenchQwen3\-1\.7B2\.60\.43\.3Qwen3\-8B25\.60\.60\.1Qwen3\-32B57\.12\.40\.0

## Appendix CModel size

We compare three sizes of the Qwen3 family, 1\.7B, 8B and 32B, on the same questions, and ask whether any subtype appears or disappears with scale and how each subtype’s rate changes\. Every subtype persists across the three sizes\. Scale lowers the rates of three of the four repetition subtypes and of the two unsupported\-deduction subtypes that act on a single step\. The rates of underthinking and of the two subtypes that commit or judge a whole lineage, which include the subtypes associated with wrong answers, do not fall with scale\. Figure[7](https://arxiv.org/html/2610.22717#A3.F7)and Table[7](https://arxiv.org/html/2610.22717#A3.T7)give the rates and elasticities on the matched panel \(the same 86 / 88 / 80 problems per task at 1\.7B, 8B and 32B, one sampling run per size, 258 / 264 / 240 traces per task and size\)\. From 1\.7B to 8B, the per\-step rates of False Rejection, Infeasible Derivation, Derive\-after\-Prune, Re\-Derivation and Redundant Re\-Verification fall; from 8B to 32B only Derive\-after\-Prune keeps falling\. Branch Abandonment, Post\-Answer Attempt, Premature Finalization and False Verdict do not change from 1\.7B to 8B or from 8B to 32B, and Subgoal Abandonment rises from 1\.7B to 8B and then holds\.

![Refer to caption](https://arxiv.org/html/2610.22717v1/subtype_scale.png)Figure 7:Subtype rates by model size\.For each task and subtype, the number of events per 100 steps \(log scale\) at 1\.7B, 8B and 32B, left to right within each group, with 95% intervals over problems, on the same 86, 88 and 80 questions per task at every size\. A group whose three dots descend is a subtype that becomes rarer with size; most groups are flat\. Derive\-after\-Prune \(R2, shaded\) is the only subtype that falls at both size increases\. Subtypes a task does not define are absent from its panel\.Subtype1\.7B→\\to8B8B→\\to32BR1 Re\-Derivation\-0\.14 \[\-0\.23, \-0\.05\]0\.04 \[\-0\.03, 0\.10\]R2 Derive\-after\-Prune\-0\.45 \[\-0\.57, \-0\.32\]\-0\.18 \[\-0\.30, \-0\.07\]R3 Redundant Re\-Verification\-0\.13 \[\-0\.23, \-0\.02\]0\.04 \[\-0\.05, 0\.14\]R4 Subgoal Churn\-0\.11 \[\-0\.29, 0\.07\]\-0\.05 \[\-0\.26, 0\.17\]R5 Branch Abandonment\-0\.01 \[\-0\.15, 0\.13\]0\.06 \[\-0\.08, 0\.20\]R6 Subgoal Abandonment0\.29 \[0\.01, 0\.59\]0\.02 \[\-0\.27, 0\.29\]R7 Post\-Answer Attempt\-0\.16 \[\-0\.42, 0\.10\]0\.03 \[\-0\.20, 0\.28\]R8 Premature Finalization\-0\.26 \[\-0\.56, 0\.03\]\-0\.14 \[\-0\.45, 0\.15\]R9 Infeasible Derivation\-0\.29 \[\-0\.38, \-0\.20\]\-0\.00 \[\-0\.09, 0\.09\]R10 False Rejection\-0\.47 \[\-0\.69, \-0\.25\]\-0\.18 \[\-0\.45, 0\.05\]R11 False Verdict\-0\.07 \[\-0\.35, 0\.22\]\-0\.09 \[\-0\.32, 0\.12\]

Table 7:Elasticity of each subtype’s rate with model size on the pooled panel: the log ratio of rates over the log ratio of parameter counts, with 95% cluster\-bootstrap intervals over problems\. Bold: interval excludes 0\. Only R2 is negative on both steps; no subtype is positive on both\.
## Appendix DPrompts

We use the three templates below unchanged for every model size;\{question\_text\}is the puzzle\. For PlanBench the puzzle is the benchmark’s own one\-shot query \(domain description, a worked example, and the target instance\), shown here for a Blocksworld instance; only the final instruction line is ours\.

Prompt Template for 24 GameYou are a 24\-game solver\.GOAL
Given 4 numbers \(TILES\), build an expression that equals exactly 24 using "\+", "\-", "\*", "/" and parentheses\.
Use each tile exactly once\. Intermediate fractions are allowed\. No concatenation \(e\.g\., 1 and 2 cannot form 12\)\. Avoid division by zero\. Use standard operator precedence unless parentheses change it\.RULES
\- Use every tile exactly once\.
\- Only "\+", "\-", "\*", "/" and parentheses may appear\.
\- No division by zero\.
\- The value must equal exactly 24\.REQUIRED VERIFICATION
Before the final answer, verify:
\- All tiles are used exactly once\.
\- The evaluated value equals 24\.End response with one of the following exactly:
ANSWER: <fully parenthesized valid expression evaluating to 24\>
or
ANSWER: NO\_SOLUTIONExample:Given:
Tiles: \[3, 3, 8, 8\]
Target: 24Solution:
8 / \(1/3\) = 24
\(1/3\) = \(3 \- \(8/3\)\)
Full expression = \(8/\(3\-\(8/3\)\)\) = 24Verification:
Final expression: \(8/\(3\-\(8/3\)\)\)
required count: 3:2, 8:2
final expression count: 3:2, 8:2
Every tile used exactly once: yes
Evaluate: \(8/\(3\-\(8/3\)\)\):
3 \- 8/3 = 1/3, 8 / \(1/3\) = 24
Final value equals 24: yesANSWER: \(8/\(3\-\(8/3\)\)\)INPUT
TILES: \{question\_text\}
Table 8:Prompt Template for 24 GamePrompt Template for Zebra Logic\# Example PuzzleThere are 3 houses, numbered 1 to 3 from left to right, as seen from across the street\.
Each house is occupied by a different person\.
Each house has a unique attribute for each of the following characteristics:
\- Each person has a unique name: peter, eric, arnold
\- Each person has a unique favorite drink: tea, water, milk\#\# Clues for the Example Puzzle
1\. Peter is in the second house\.
2\. Arnold is directly left of the one who only drinks water\.
3\. The one who only drinks water is directly left of the person who likes milk\.\#\# Reasoning
Given Clue 1, we know Peter is in House 2\. According to Clue 2, Arnold is directly left of the one who only drinks water\. The person in House 3 cannot be on the left of anyone, so Arnold must be in House 1\. Thus, Peter drinks water, and Eric lives in House 3\. Then, according to Clue 3, Eric drinks milk\. Therefore, Arnold drinks tea\.\#\# Answer to the Example Puzzle
\{
"House 1": \{"Name": "arnold", "Drink": "tea"\},
"House 2": \{"Name": "peter", "Drink": "water"\},
"House 3": \{"Name": "eric", "Drink": "milk"\}
\}Return the final response in the following format:
\{answer\}\# Please solve the following puzzle and present your solution in the aforementioned json format:\{question\_text\}
Table 9:Prompt Template for Zebra LogicPrompt Template for PlanBenchI am playing with a set of blocks where I need to arrange the blocks into stacks\. Here are the actions I can doPick up a block
Unstack a block from on top of another block
Put down a block
Stack a block on top of another blockI have the following restrictions on my actions:
I can only pick up or unstack one block at a time\.
I can only pick up or unstack a block if my hand is empty\.
I can only pick up a block if the block is on the table and the block is clear\. A block is clear if the block has no other blocks on top of it and if the block is not picked up\.
I can only unstack a block from on top of another block if the block I am unstacking was really on top of the other block\.
I can only unstack a block from on top of another block if the block I am unstacking is clear\.
Once I pick up or unstack a block, I am holding the block\.
I can only put down a block that I am holding\.
I can only stack a block on top of another block if I am holding the block being stacked\.
I can only stack a block on top of another block if the block onto which I am stacking the block is clear\.
Once I put down or stack a block, my hand becomes empty\.
Once you stack a block on top of a second block, the second block is no longer clear\.\[STATEMENT\]
As initial conditions I have that, the red block is clear, the blue block is clear, the yellow block is clear, the hand is empty, the blue block is on top of the orange block, the red block is on the table, the orange block is on the table and the yellow block is on the table\.
My goal is to have that the orange block is on top of the blue block\.My plan is as follows:\[PLAN\]
unstack the blue block from on top of the orange block
put down the blue block
pick up the orange block
stack the orange block on top of the blue block
\[PLAN END\]\[STATEMENT\]
As initial conditions I have that, the red block is clear, the yellow block is clear, the hand is empty, the red block is on top of the blue block, the yellow block is on top of the orange block, the blue block is on the table and the orange block is on the table\.
My goal is to have that the orange block is on top of the red block\.My plan is as follows:\[PLAN\]Provide the plan that achieves the goal, one action per line, in the same format as the example above\. End your plan with \[PLAN END\]\.
Table 10:Prompt Template for PlanBench \(the benchmark’s one\-shot query for a Blocksworld instance, followed by our instruction line\)
## Appendix EParser: adjudicated reference labels, metric, and development

This appendix documents the parser of §[2\.1](https://arxiv.org/html/2610.22717#S2.SS1): how we built the adjudicated reference labels we measure it against, what the tolerant metric counts, how we reached the final configuration, and what we tried and did not adopt\. Every number below re\-scores stored parser outputs against the frozen reference and is reproducible from the released files\.

### E\.1Adjudicated reference labels

The reference is 40 trajectories per task \(120 trajectories, 46,984 steps\), drawn from the Qwen3\-8B generations by stratified selection over outcome and length\. Hand\-labeling it was not feasible, and the label scheme could not be fixed in advance, so we built it the way qualitative coding builds a codebook, with model annotators in the loop\([Kim et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib15);[He et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib12)\): a stronger model labeled every step, then LLM judges re\-labeled it independently from the written rules alone; where two judges agreed the label stood, and we raised every remaining disagreement to a human as a decision card quoting the steps and the candidate rules\. We wrote each ruling into the rulebook, applied it corpus\-wide by script rather than as an edit to a single step, and re\-froze the reference with a content hash after each round\. A final audit read every step the parser gets wrong in all five of its samples against the rulebook and found the reference right on 90% of them\. Adjudication against explicit rules, rather than inter\-annotator agreement, thus decides correctness\([Liu et al\., 2026](https://arxiv.org/html/2610.22717#bib.bib19)\)\. The 40 trajectories also served as the development set for every choice below; a held\-out review on trajectories the development never saw\([Pangakis et al\., 2023](https://arxiv.org/html/2610.22717#bib.bib26)\)is in progress, and we report the configuration and the alternatives we tried so that the accuracy is not the best of many undisclosed configurations\([Baumann et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib2)\)\.

### E\.2Labeling protocol

The parser is one open\-weight model, Qwen3\.8\-27B, with constrained JSON output\. It labels steps in groups of fifteen and sees the fifteen preceding steps with the labels it gave them, a few labeled reference steps retrieved from other trajectories of the same task, and the schema elements that each step’s words name\. It never sees a summary of the solution state; what has been*concluded*is the engine’s job, so a label depends only on the sentence and its neighborhood\. We take five independent samples per trajectory and decide each step’s label by majority vote\.

### E\.3What the tolerant metric counts

We compute three metrics per step over the steps of the 40 traces that carry a reference label, excluding the few steps whose reference is itself contested\.

#### Strict\.

The parser’s label equals the reference label\.

#### Tolerant\.

Strict, or the pair of labels is one that the rulings declared indistinguishable for the engine*and*the payloads agree on the facts or the action they name\. The pairs are:

- •a fact stated as a fresh inference \(Derive/floating\) versus restated from the givens \(Restate\) versus reported as holding \(Verify, outcome hold\), when both payloads name the same fact — the engine attaches the same fact to the same node either way;
- •a plan step adopted \(Assume\) versus echoed \(Restate\) when both name the same action; the echo test looks back over the whole current attempt, which a windowed parser cannot see;
- •a bare applicability verdict read as enabling an action \(Derive/action\-enabled\) versus as a check that holds \(Verify\) on the same action;
- •a plan step under a visible answer header read as confirmed \(Confirm\) versus recapped \(Restate\), and any step label when the header lies outside the parser’s window;
- •a bare closer \(“Yes\.”, “So no\.”, “Met\.”\) read as narration \(Meta\) versus as a payload\-free check \(Verify\), and aVerifywhose only content was an anchor the step does not name\.

Each pair encodes a ruling that the distinction either does not change what the engine does with the step or rests on context no step\-local parser has; strict accuracy penalizes exactly these distinctions, which the reference itself resolves inconsistently across traces\. Tolerant accuracy therefore measures whether the parser produced the label the engine needs\.

#### Payload \(Key\)\.

On steps whose reference payload carries schema atoms, the set of atoms the parser emits equals the set the reference carries: the role \(kind or target\), the facts with their polarity and arguments, the actions with their arguments, and the outcome\. An anchor the step does not itself name \(an action a subgoal serves, a step a check is about\) is excluded from both sides, since it can only come from context\. Cited premises are support, not key content\.

### E\.4Development against the frozen reference

We ran every candidate change as five independent samples on the frozen reference \(sample standard deviation 0\.001–0\.002 tolerant\), scored it locally, and adopted it only when it beat the previous configuration by at least 0\.3 points on every sample\. We edited prompts only through builders that record each change, and we never edited the reference to match a parser\. Table[11](https://arxiv.org/html/2610.22717#A5.T11)gives the cumulative result\.

TolerantStrictKeyTaskprompt only\+ own labels, examples\+ step factsfinalfinalPlanBench0\.8480\.855→\\rightarrow0\.8890\.8930\.8130\.797ZebraLogic0\.9210\.930†0\.9370\.9010\.78424 Game0\.9320\.9510\.946†\(not adopted\)0\.9330\.848

Table 11:Cumulative development on the frozen 40\-trace reference; votes of five samples unless marked†\(mean of one or two samples\)\. PlanBench shows own labels and examples as two steps \(0\.855, 0\.889\); for ZebraLogic and 24 Game the two were introduced together\. We kept step facts where they added at least 0\.3 points\.#### What helped and what did not\.

Four changes raised tolerant accuracy on the frozen PlanBench reference and we kept them: retrieved labeled examples from the same task, leave\-one\-trace\-out \(\+3\.3 points\); own\-label context on the preceding steps \(\+0\.6\); step facts inline with worked exemplars \(\+0\.4, kept for PlanBench and ZebraLogic, dropped for the 24 Game, whose examples already evaluate every expression\); and the reference audit with its rulings \(\+1\.0\)\. Prompt cleaning over four rounds moved accuracy by 0\.0 to \+0\.3 per round and we kept it where neutral or positive\. Five changes did not help and we did not adopt them: more examples per group \(\+0\.16\), examples oversampled towardVerifyandConfirm\(\+0\.0\), step facts as a separate block \(−0\.05\-0\.05, strict−1\.3\-1\.3\), the labels of the following 15 steps \(−0\.4\-0\.4\), and a second pass that lists every fact a step states \(Key−3\.0\-3\.0, since it adds cited premises the payload policy excludes\)\. An ensemble of four configurations with 16 samples gained \+1\.0 over the best single vote; we report it but did not ship it, since it is not a single configuration\.

#### Retrieval details\.

The example pool is the reference itself\. We index each reference step by TF\-IDF over word unigrams and bigrams with entity names and numbers replaced by placeholders, so that “Soredis clear now” and “Soblueis clear now” coincide\. For each 15\-step group we retrieve the nearest reference step to each current step from the same surface\-form family \(PlanBench domains with shared operators; ZebraLogic and 24 Game each one family\) and from other traces only, then deduplicate and cap the set at 12 examples and 9,000 characters\. An example shows the two reference steps before it with their labels, the step, and its reference output in the prompt’s own format\. The gain by similarity of the retrieved example was−3\.8\-3\.8,−4\.4\-4\.4, and−4\.0\-4\.0points of error rate for near\-verbatim \(≥0\.95\\geq 0\.95cosine\), moderate \(0\.700\.70–0\.950\.95\), and loose \(<0\.70<0\.70\) matches respectively, and−0\.7\-0\.7for steps that received no example; the parser does not copy a twin’s label\. For unlabeled traces the pool is the whole reference\.

#### Step facts\.

For each step, a string match against thePuzzle Schemalists the entities, predicates, and operators its words name \(verb forms and common paraphrases included: “on the table”, “hand is empty”, “unstacking”\) and the initial\-state or goal facts it re\-states with the arguments in the right order; for the 24 Game we find and evaluate every arithmetic expression and mark it when it equals the target or uses exactly the four tiles\. On the reference, a PlanBench step with no anchors isMeta87% of the time and a step with anchors 6%; we append the facts to the step in angle brackets and demonstrate them in the examples\.

#### What remains wrong\.

On PlanBench, 8\.2% of steps are wrong in all five samples of the final configuration, concentrated in four confusions — a grounded inference read as narration \(Derive→\\toMeta\), a restated given read as narration or as fresh inference, and a need read as adoption — that did not respond to more rules, more examples, or more step\-local information\. ZebraLogic’s plateau is the echo test \(restated cell versus fresh inference\); 24 Game’s is narration versus check\. We did not train the parser or use a larger parser model\.

## Appendix FEngine Design: Validation and Worked Examples

### F\.1Resolution procedure

Every mutation of §[2\.2](https://arxiv.org/html/2610.22717#S2.SS2)first decides which node the step is about, by one procedure:

1. 1\.the engine prefers an explicit anchor in the step \(a named cell, expression or action\);
2. 2\.otherwise it draws candidates from a bounded scope around the current step and the node the model is attending to;
3. 3\.it ranks candidates by how much of their content the step already matches;
4. 4\.it filters out candidates whose state is incompatible with the step;
5. 5\.it breaks remaining ties canonically, and revives a closed node when the model returns to it\.

Two cursors support the scope: the node the model is attending to and the node it last changed\.

### F\.2Validating the engine and the detectors

Two further conventions apply\. We grade a False Verdict \(R11\) against what the model had written down, not against the truth, which separates a wrong verdict from a wrong premise\. Rows are not disjoint; a detector emits every exclusion it makes beside the row with its reason\.

A replay of the corpus makes on the order of10410^\{4\}tree decisions per task, far more than a human can inspect\. We validated the engine the same way as the parser\. On the 40 reference trajectories per task, LLM judges re\-derived the decision for sampled steps from the raw text alone, without sight of the engine’s state\. Every disagreement with the engine went to a human as a decision card and received a written ruling\. An append\-only ledger keeps the rulings; we replay every engine change against it before it ships, so the oracle only grows\. We validated the failure detectors on the same trajectories by reading every fired and every excluded event of each row against its definition; the human\-annotation study of §[4](https://arxiv.org/html/2610.22717#S4)measures their precision on the full corpus\.

### F\.3Worked examples

The examples below are verbatim steps from the working corpus, reproduced with their engine treatment; they illustrate the resolution principle of §[2\.2](https://arxiv.org/html/2610.22717#S2.SS2)\. We identify puzzle instances by trace id; tiles and target are as stated in the source puzzle\.

#### Extending the current line\.

Traceque352\_com0\(tiles\{2,2,3,3\}\\\{2,2,3,3\\\}, target 24\) has already opened, among other options, a plan for24=3×824=3\\times 8with an open operand goal “make 8\.” Step \#9 —

> ‘‘Maybe if I can make 8 and then multiply by 3?’’

resolves against that existing operand goal rather than opening a new option\. The decomposition it names is already a sibling in the tree, so the engine edits that node in place and creates no new node\.

#### Branching into new candidates\.

Earlier in the same trace, step \#5 —

> ‘‘For example, 3\*8, 4\*6, 12\+12, etc\.’’

proposes three unrelated decompositions of 24 in one step\. The engine resolves each independently and creates its own option node with its own operand children, all as alternative children of the root — one parsed step, three tree mutations\.

#### Repetition resolved by state\.

Step \#50 of the same trace proposes and immediately rejects\(3×2\)×\(3\+2\)=6×5=30≠24\(3\\times 2\)\\times\(3\+2\)=6\\times 5=30\\neq 24, which the engine creates and closes as a rejected exploration node\. Three steps later, step \#53 —

> ‘‘How about \(2 \* 3\) \* \(2 \+ 3\) = 6 \* 5 = 30\.’’

is a different string but the identical computation, evaluated in the identical context \(unattached exploration at the root, no goal on either occasion\)\. The engine resolves it to the already\-closed node from step \#50 and records a repetition event rather than a new node; state drives the decision\. The converse holds by the same rule: had the second occurrence arisen under a different goal context, the identical computation would be legitimate independent reuse, with no change in the surface text\. This is our central claim: grounding to lineage and state separates true repetition from superficially similar but substantively different reasoning\.

#### Fresh start after a reframing\.

Traceque1060\_com2has an open, previously\-worked quest to make44/344/3from tiles\{4,9\}\\\{4,9\\\}\. At step \#82, after intervening steps‘‘Another angle: let’s think about fractions’’and‘‘Suppose I take 9 and 4,’’the model writes

> ‘‘9\*4=36\.’’

Although the tiles match the open quest, the engine rules this a fresh start under the root rather than an attempt at the earlier quest\. An explicit reframing \(a new\-approach marker\) lies between the quest’s last activity and this step, and a quest hosts new content by context only while it remains the most recent framing\.

## Appendix GLocal measures: detail

### G\.1Matched\-yield tuning

We tune each local measure on a development split of 42 traces per task, disjoint from the test traces and distinct from the 40 development questions excluded from the main text, and score it on the remaining test split\. The similarity thresholdτ∗\\tau^\{\\ast\}is the value whose flagged token share on the development traces is closest to the token share of our repetition rows \(R1–R4\) on the same traces:τ∗=0\.934\\tau^\{\\ast\}=0\.934on ZebraLogic \(777 adjacent development windows\),0\.9110\.911on Game of 24 \(817 windows\) and0\.9560\.956on PlanBench \(758 windows\); the measure flags a window of sentences as redundant when the cosine similarity of its embedding to the preceding window exceeds the threshold\. The two step scorers take the bottom\-kkshare of steps that matches the token share of our unsupported\-deduction rows:k=0\.010k=0\.010/0\.0350\.035\(process reward model / token confidence\) on ZebraLogic,0\.0150\.015/0\.0350\.035on Game of 24 and0\.020\.02/0\.060\.06on PlanBench, so the process reward model flags the lowest\-scored 1 to 2% of a trajectory’s steps and token confidence the 3\.5 to 6% of steps with the lowest mean token log\-probability; as trajectory scores for outcome prediction both use the mean over the worst 10% of steps\. Thenn\-gram rule scores a trajectory by the maximum number of times any 30\-tokennn\-gram recurs; run at its published setting \(30, 20\) it flags one Game of 24 trace, no ZebraLogic trace and two PlanBench traces, and relaxed to\(5,k\)\(5,k\)it selects 49% first work and 48% discourse with a repetition enrichment of 0\.70, 0\.00 and 1\.61 per task\. The similarity scores reproduce the reference implementation to six decimals on three traces spanning Game of 24 and PlanBench\.

### G\.2Expert annotation

![Refer to caption](https://arxiv.org/html/2610.22717v1/annotation_html.png)Figure 8:The annotation interface\.One card as the annotators saw it: the flagged step in its surrounding steps, the instruction stating the failure to judge, and the response options including “undecided”\.The four annotators are researchers in NLP with expertise in language models, who volunteered and were not paid; each read a card showing the flagged step with its surrounding steps and answered whether the step is the stated failure, with “undecided” allowed; Figure[8](https://arxiv.org/html/2610.22717#A7.F8)shows the interface with its full instruction\.

CategoryMethodCardsConf\.Undec\.Prec\.Repetitionours \(R1–R4\)452810\.64nn\-gram452200\.49similarity452800\.62Abandonmentours \(R5, R6\)451410\.32switch marker451210\.27After the answerours \(R7\)433300\.77post\-answer red\.45500\.11Infeasible Derivationours \(R9\)342700\.79False Verdictours \(R10, R11\)11800\.73Wrong step, genericPRM45420\.09token confidence45510\.11All categoriesours17811020\.63

Table 12:Expert annotation, precision by category and method, pooled over the four annotators \(422 cards, each read once; a card that carries both methods’ steps counts once per method\)\. One card asks one question with the same wording for our step and for the local measure’s step in the same category; a card is confirmed when the annotator answers that the step is the named failure \(for abandonment, that the model leaves and the attempt was still workable\)\.*Undec\.*: the annotator could not decide from the text; excluded from the precision\.CategoryConfirmed poolOursMeasuresRepetition650\.37nn\-gram 0\.31, similarity 0\.45Abandonment210\.71switch marker 0\.62After the answer380\.87post\-answer red\. 0\.13Infeasible Derivation271\.00no measureFalse Verdict81\.00no measureUnsupported deduction \(R8–R11\) \+ generic wrong step440\.80PRM 0\.09, confidence 0\.11

Table 13:Pooled recall against the annotator\-confirmed pool, over the four annotators\. Of the steps confirmed in a category \(all carded strata pooled\), the share each method’s own flagged set contains\. The pool is built from the carded strata, so this recall is relative to the cards\. The last row pools the two unsupported\-deduction categories with the confirmed steps of the generic wrong\-step cards \(9 steps, of which one carries an unsupported\-deduction row of ours\); it is the pool the unsupported\-deduction rows of Table[3](https://arxiv.org/html/2610.22717#S4.T3)are scored against, so that the pool is not defined by our cards alone\.#### Precision and recall\.

Table[12](https://arxiv.org/html/2610.22717#A7.T12)gives the full card counts behind the precision column of Table[3](https://arxiv.org/html/2610.22717#S4.T3), and Table[13](https://arxiv.org/html/2610.22717#A7.T13)the pooled recall\. Over the five categories, the four annotators confirm 110 of the 176 decided cards of ours \(0\.63\)\. We traced every miss of ours on the first annotator’s cards: three parser label losses \(an illustration parsed as a derivation; two compound\-action steps parsed as one action\), three definitional calls by the annotator, five steps where the model is still finishing and the annotator read the doubt as part of that, and one abandonment with a true stated reason\.

#### Restatement follow\-up\.

The annotators’ notion of repetition is wider than the tree’s rows\. In a follow\-up, we showed the first annotator 45 steps that restate earlier content \(15 per task, each with the earlier step of the same content\) under the standard repetition question\. The annotator confirmed 21 of 45 as repetition, split by what is restated: the puzzle’s givens 5 of 20 \(Game of 24 puzzle numbers 0 of 5, ZebraLogic clue recap 4 of 10, board echo 1 of 5\) against the model’s own earlier results 16 of 25 \(Game of 24 unused\-number list 4 of 6, expression 3 of 4, PlanBench own facts 9 of 15\)\. Counting restatements of the model’s own results as repetition, our recall on that annotator’s repetition cards moves from 0\.36 to 0\.43; about 5 of the 28 confirmed steps remain steps the engine did not match\.

#### Lineage check on the undecided abandonment cards\.

On 11 abandonment cards, about 1/8 of the abandonment cards drawn \(5 of ours, 6 switch\-marker; 7 from the first annotator, 3 from the second, 1 from the fourth\), the annotators could not decide from the text whether the branch being left was still workable\. When we showed them the engine’s branch\-viability check open, with the lineage of the node being left, they settled 9 of the 11 and agreed with the check on 7 of those \(the two disagreements of the first annotator were on checks that found the branch viable, the one of the second on a check that found it exhausted\); neither method’s precision moved \(ours 13/41 to 14/44, switch marker 10/38 to 12/44\)\. The check exists only forArbor’s steps; for the switch\-marker cards we took it from the same trajectory\.

### G\.3What the local measures flag

![Refer to caption](https://arxiv.org/html/2610.22717v1/local_measures_coverage.png)\(a\)Subtype coverage by each local measure\(Qwen3\-8B, annotated sample of 342 traces per task\)\. Each cell is the share of the subtype’s steps that the measure also flags;nnis the subtype’s step count on the sample\. We group rows by family and number them as in Table[1](https://arxiv.org/html/2610.22717#S2.T1); the subtype name after the colon is the detector’s own detail label\. Post\-answer redundancy reaches re\-verification after the answer and post\-answer attempt steps; the switch marker reaches the explicit “switch” form of branch abandonment; no measure reaches an unsupported\-deduction subtype above 0\.30, and the process reward model and token confidence stay at or below 0\.09 on every subtype\.
![Refer to caption](https://arxiv.org/html/2610.22717v1/local_measures_composition.png)\(b\)Local signals flag ordinary reasoning, not failures\.What the tokens each local signal flags consist of, byArbor’s labels \(pooled over the three tasks, test questions\), against the corpus base rate \(top bar\)\. Repetition shares are printed\. We tune each signal to flag the same share of tokens asArbor’s rows \(§[4](https://arxiv.org/html/2610.22717#S4)\); the published n\-gram loop rule fires on 3 of 11,718 trajectories and is not shown\. Per\-task values in Appendix[G](https://arxiv.org/html/2610.22717#A7)\.

Figure 9:What the local measures flag\.#### Composition of the flagged sets\.

On the full corpus, most steps a local measure flags are first derivations or verifications \(the forward work of the trajectory\) and discourse \(connective steps with no content, such as “Wait” or “Let me see”\)\. WithArbor’s labels as the reference, the precision of each measure for its target category is near the base rate: similarity flags a repetition step with 5% of its tokens, and a random step is one with 3%\. The switches that switch tokens recover are mostly the explicit “let me try another” switches\.

Switch setScopeYield \(%\)Non\-viableOn R5/R6 stepSwitch markerpooled3\.90\.50 \[0\.49, 0\.51\]0\.033ZebraLogic0\.90\.58 \[0\.55, 0\.61\]0\.051Game of 248\.40\.47 \[0\.46, 0\.48\]0\.039PlanBench3\.30\.51 \[0\.50, 0\.52\]0\.024Engine\-detectedpooled22\.00\.61 \[0\.60, 0\.62\]0\.014ZebraLogic2\.50\.76 \[0\.75, 0\.77\]0\.074Game of 2414\.10\.71 \[0\.70, 0\.72\]0\.065PlanBench32\.10\.60 \[0\.59, 0\.61\]0\.005

Table 14:Switches by the marker and by the engine\(Qwen3\-8B, test split\)\. A marker switch is a step carrying a switch token; an engine\-detected switch is a step on which the engine’s attention cursor moves off a node\.*Yield*: share of tokens on switch steps\.*Non\-viable*: share of switches that leave a branch the viability oracle marks non\-viable at the switch step \[95% cluster\-bootstrap interval over problems\]\.*On R5/R6 step*: share of the set’s steps that carry a branch or Subgoal Abandonment row\. The marker reaches 0\.25 of the abandonment steps pooled; the engine’s own switch set, 0\.67\.
#### Engine\-detected switches against the switch marker\.

Table[14](https://arxiv.org/html/2610.22717#A7.T14)compares the marker’s switches with the switches the engine detects from the tree\. Engine\-detected switches leave a branch the oracle marks non\-viable more often than marker switches do; “non\-viable” refers only to oracle viability at the switch step\. Few marker switches fall on an abandonment step\.

### G\.4Outcome prediction per task and the length\-normalized check

AUROC, incorrect vs\. correctGain, pooledScorePooledGame of 24PlanBenchZebraLogica\+\+rows\+\+measureLength\+\+task \(baseline\)0\.856 \[0\.846, 0\.864\]0\.797 \[0\.773, 0\.818\]0\.744 \[0\.723, 0\.764\]0\.740 \[0\.650, 0\.818\]nn\-gram loop \(count\)0\.880 \[0\.872, 0\.888\]0\.796 \[0\.772, 0\.818\]0\.773 \[0\.752, 0\.791\]0\.740 \[0\.651, 0\.819\]\+0\.066\+0\.066\+0\.001\+0\.001Similarity \(IRD\)0\.866 \[0\.857, 0\.875\]0\.818 \[0\.796, 0\.835\]0\.875 \[0\.863, 0\.887\]0\.739 \[0\.653, 0\.819\]\+0\.080\+0\.080\+0\.002\+0\.002Post\-answer redundancy \(ERD\)b––––––Switch marker \(rate\)0\.893 \[0\.885, 0\.900\]0\.801 \[0\.776, 0\.822\]0\.825 \[0\.809, 0\.841\]0\.737 \[0\.652, 0\.819\]\+0\.054\+0\.054\+0\.003\+0\.003Process reward model \(bottom\-kk\)0\.876 \[0\.868, 0\.885\]0\.798 \[0\.774, 0\.820\]0\.815 \[0\.797, 0\.831\]0\.745 \[0\.660, 0\.821\]\+0\.072\+0\.072\+0\.004\+0\.004Token confidence \(bottom\-kk\)0\.873 \[0\.865, 0\.882\]0\.799 \[0\.774, 0\.822\]0\.777 \[0\.759, 0\.795\]0\.747 \[0\.665, 0\.825\]\+0\.073\+0\.073\+0\.002\+0\.002Error\-step count \(all rows summed\)0\.863 \[0\.855, 0\.872\]0\.796 \[0\.774, 0\.816\]0\.743 \[0\.722, 0\.765\]0\.739 \[0\.652, 0\.818\]\+0\.081\+0\.081−0\.000\-0\.000Per\-row counts \(ours\)0\.944\[0\.939, 0\.950\]0\.926\[0\.911, 0\.941\]0\.886\[0\.874, 0\.898\]0\.726 \[0\.631, 0\.803\]

Table 15:What each score knows about the outcome\(Qwen3\-8B, test split\)\. AUROC of a logistic model for incorrect against correct traces, 5\-fold cross\-validation grouped by problem, withlog\\logtrace length and the task as covariates in every model \[95% cluster\-bootstrap interval over problems\]; truncated traces excluded\.*\+\+rows*: gain in pooled AUROC from adding the per\-row counts to the measure\.*\+\+measure*: gain from adding the measure to the per\-row counts\. The per\-row vector is the count of steps carrying each row of Table[1](https://arxiv.org/html/2610.22717#S2.T1); the error\-step count is their sum\.aZebraLogic has 82 incorrect traces against 783 truncated, so incorrect\-vs\-correct is underpowered there; on truncated against completed, the per\-row counts reach 0\.986 \[0\.982, 0\.990\] \(Appendix[G](https://arxiv.org/html/2610.22717#A7)\)\.bDefined only on traces that reach a correct answer, which are all correct, so it has one outcome class\.In Figure[5](https://arxiv.org/html/2610.22717#S4.F5)the five\-signal ensemble closes about 3/5 of the gap between the length\-and\-task baseline andArbor; length is a control because longer trajectories fail more often\. BecauseArbor’s counts grow with trajectory length, we repeat the outcome prediction with every count normalized per 100 steps and no length term in the model: pooled,Arborreaches 0\.95 against 0\.91 for the five local signals together and 0\.84 for task alone, so the ordering of Figure[5](https://arxiv.org/html/2610.22717#S4.F5)is not an artifact of length \(per\-task values with intervals in Table[15](https://arxiv.org/html/2610.22717#A7.T15)and Table[16](https://arxiv.org/html/2610.22717#A7.T16)\)\. ZebraLogic has only 82 incorrect trajectories because its failures are truncations \(783 of 877\)\. There no signal,Arborincluded, separates incorrect from correct beyond length; without the length term the local ensemble \(0\.75\) edgesArbor\(0\.67\)\. Truncation as the outcome is itself a matter of length on ZebraLogic \(0\.97 from length alone, 0\.98 to 0\.99 for every signal\) but not pooled \(0\.96 forArboragainst 0\.93 for the ensemble and 0\.87 for length\)\.

ScoreZebra24PlanPooledTask only \(baseline\)0\.500\.500\.500\.84n\-gram0\.670\.760\.510\.86similarity \(IRD\)0\.610\.610\.860\.89switch tokens0\.610\.570\.760\.89process reward model0\.690\.650\.720\.89token confidence0\.690\.660\.610\.87all five local signals0\.750\.790\.870\.91Arbor: per\-subtype rates0\.670\.920\.870\.95

Table 16:Outcome AUROC \(incorrect vs correct\) with every count normalized by trajectory length and no length feature: each local signal as a rate, the five together, andArbor’s per\-subtype rates per 100 steps\. Within a task the baseline is chance; pooled, the task indicator alone gives 0\.84\.Table[16](https://arxiv.org/html/2610.22717#A7.T16)gives the length\-free check per task\. With every count divided by trajectory length and no length feature,Arbor’s rates lead the five local signals together on the 24 Game, tie on PlanBench and trail on ZebraLogic, where truncation is the dominant failure and length itself carries most of the signal\.

## Appendix HSubtypes: definitions, full tables and robustness

Rows undefined on a task \(Table[1](https://arxiv.org/html/2610.22717#S2.T1)\): ZebraLogic posts no subgoals \(R4, R6\); a 24 Game check is an expression the engine evaluates, so a wrong verdict surfaces as R8 or R10 \(R11\); on PlanBench a step inside a closed branch also fails its preconditions, so we report its R2 as the subset of R9 that repeats a correctly pruned step\. R8 firings on the emitted final answer record answer validity and are not step events\. Each subsection below is a table of the statistics defined for every subtype, followed by the statistics defined for one subtype only, stated in prose with their source; the main text reports selected values from these tables\.

Rate per 100 stepsPrev\.TokensLastOdds ratio, incorrect vs correctSign per taskSubtypeZebra24Plan\(%\)\(%\)third \(%\)Model A \(between problems\)Model C \(within problem\)Z24PRepetition\(category\)0\.45 \[0\.34, 0\.61\]0\.83 \[0\.53, 1\.32\]R1 Re\-Derivation2\.651\.080\.12591\.0300\.98 \[0\.86, 1\.13\]1\.49 \[1\.18, 1\.88\]▽\\triangledown▲\\blacktriangle∘\\circR2 Derive\-after\-Prune0\.342\.620\.18561\.0400\.66 \[0\.58, 0\.76\]0\.90 \[0\.72, 1\.14\]∘\\circ∘\\circ▽\\triangledownR3 Redundant Re\-Verification0\.783\.110\.59821\.2600\.37 \[0\.31, 0\.44\]0\.48 \[0\.37, 0\.63\]▽\\triangledown▽\\triangledown▽\\triangledownR4 Subgoal Churn–0\.350\.04290\.2251\.28 \[1\.12, 1\.51\]1\.24 \[0\.98, 1\.57\]–▲\\blacktriangle∘\\circUnderthinking\(category\)0\.57 \[0\.50, 0\.66\]0\.42 \[0\.33, 0\.53\]R5 Branch Abandonment0\.200\.420\.13470\.3251\.07 \[0\.95, 1\.20\]0\.83 \[0\.69, 1\.02\]∘\\circ▽\\triangledown▲\\blacktriangleR6 Subgoal Abandonment–0\.140\.009160\.06321\.24 \[1\.03, 1\.46\]1\.18 \[0\.89, 1\.56\]–∘\\circ∘\\circR7 Post\-Answer Attempt0\.030\.530\.03240\.2470\.08 \[0\.07, 0\.10\]0\.14 \[0\.10, 0\.19\]∘\\circ▽\\triangledown▽\\triangledownUnsupported deduction\(category\)4\.68 \[3\.30, 7\.29\]8\.34 \[4\.98, 17\.68\]R8 Premature Finalization0\.0080\.080\.00370\.047010\.12 \[8\.15, 12\.85\]10\.25 \[6\.98, 16\.70\]▲\\blacktriangle▲\\blacktriangle▲\\blacktrianglemid\-trace commits only2\.11 \[1\.59, 2\.80\]1\.41 \[0\.92, 2\.26\]R9 Infeasible Derivation1\.120\.850\.89741\.2382\.90 \[2\.35, 3\.60\]4\.57 \[3\.22, 7\.03\]∘\\circ▲\\blacktriangle▲\\blacktriangleR10 False Rejection0\.080\.050\.27300\.2390\.87 \[0\.76, 0\.99\]1\.03 \[0\.84, 1\.27\]∘\\circ∘\\circ▽\\triangledownR11 False Verdict0\.13–0\.50480\.4521\.70 \[1\.46, 1\.96\]1\.19 \[0\.93, 1\.56\]▲\\blacktriangle–▲\\blacktriangle

Table 17:Each subtype on its own\(Qwen3\-8B, 11,718 traces; the 40 engine\-development problems per task excluded\)\. Rate: events per 100 steps per task\. Prev\.: share of traces with at least one event, over the tasks that define the subtype\. Tokens: share of all tokens on steps carrying the subtype \(Appendix[H\.2](https://arxiv.org/html/2610.22717#A8.SS2)\)\. Last third: share of the subtype’s events in the last third of the trace\. Odds ratios of the subtype’s presence with 95% cluster\-bootstrap intervals over problems: Model A, incorrect vs correct at equal log length and task, truncated traces excluded; Model C, the same within the 785 problems that have both outcomes \(conditional logit\)\. Category rows give the odds ratio of the category’s presence\. Bold: Benjamini–Hochbergq<0\.05q<0\.05\. Sign per task \(Model A\):▲\\blacktriangleabove 1,▽\\triangledownbelow 1,∘\\circnot significant, – undefined \(Table[1](https://arxiv.org/html/2610.22717#S2.T1)\)\. Premature Finalization \(R8\) fires on a finalization that fails the acceptance test, so its odds ratio is largely its definition; the sub\-row counts only mid\-trace commits the model later revised\. PlanBench R2 is the subset of its R9 steps that repeat a correctly pruned step\.### H\.1Tree shapes

Figure[4](https://arxiv.org/html/2610.22717#S3.F4)\(§[5](https://arxiv.org/html/2610.22717#S5)\) gives the schematic tree of a typical failing trajectory per task\. A ZebraLogic trajectory is one deep chain of consecutive derivations \(a maximum depth of 23 for the median trajectory\) that switches lines of reasoning only about every 40 steps, split at a few case assumptions whose rejected cases are pruned; a 24 Game trajectory is a shallow fan \(maximum depth 7\) whose nodes are mostly siblings, alternative compositions tried in turn from the same starting point, with a switch between them every 7 steps; a PlanBench trajectory is a single chain of moderate depth \(maximum depth 16\) that appends one plan step after another and rarely branches, but 1/4 of its nodes are states the model assumes rather than derives, and what it checks are the preconditions each step needs\.

### H\.2Token share and per\-task cost

The token share of a subtype is the share of all tokens of a task’s traces that lie on steps carrying the subtype, under the Qwen3 tokenizer; a step carrying two subtypes counts for both\. It measures the cost of the steps a detector labels\. Table[18](https://arxiv.org/html/2610.22717#A8.T18)gives the per\-task values behind the pooled column of Table[17](https://arxiv.org/html/2610.22717#A8.T17)\.

Token share \(%\)Prevalence \(%\)SubtypeZebra24PlanPooledZebra24PlanPooledR1 Re\-Derivation3\.171\.370\.151\.0587693559R2 Derive\-after\-Prune0\.483\.580\.230\.9848824156R3 Redundant Re\-Verification0\.723\.330\.561\.1878927682R4 Subgoal Churn–0\.450\.050\.16–451529R5 Branch Abandonment0\.260\.560\.160\.2744603847R6 Subgoal Abandonment–0\.200\.010\.06–31416R7 Post\-Answer Attempt0\.030\.850\.030\.20857724R8 Premature Finalization0\.010\.070\.040\.0431827R9 Infeasible Derivation1\.321\.501\.091\.2360569674R10 False Rejection0\.070\.030\.280\.1820115230R11 False Verdict0\.11–0\.580\.4514–6848Any subtype \(a step counted once\)6\.111\.73\.45\.79310010098

Table 18:Token share and prevalence per task \(Qwen3\-8B\)\. Token share: tokens on steps carrying the subtype over all tokens of the task’s traces \(Qwen3 tokenizer\); a step carrying two subtypes counts for both\. Pooled prevalence is over the tasks that define the subtype\. The last row counts every detector row, including rows the paper does not report\.
### H\.3Extended cost: the tokens an event opens

Median tokens opened / share of the trajectory \(%\)Pooled share by outcome \(%\)SubtypeZebraLogic24 GamePlanBenchPooledcorrectincorrecttruncatedRepetitionR1 Re\-Derivation17 / 0\.315 / 0\.416 / 0\.216 / 0\.30\.40\.30\.2R2 Derive\-after\-Prune25 / 0\.417 / 0\.4220 / 2\.720 / 0\.40\.50\.50\.3R3 Redundant Re\-Verification11 / 0\.224 / 0\.812 / 0\.216 / 0\.30\.60\.20\.2R4 Subgoal Churn–14 / 0\.317 / 0\.215 / 0\.30\.40\.20\.2UnderthinkingR5 Branch Abandonment948 / 22\.8430 / 12\.52,886 / 37\.6906 / 20\.814\.633\.015\.0R6 Subgoal Abandonment–1,381 / 47\.82,825 / 39\.81,489 / 46\.750\.639\.943\.0R7 Post\-Answer Attempt267 / 10\.518 / 0\.6256 / 5\.319 / 0\.70\.70\.70\.4Unsupported deductionR8 Premature Finalization446 / 9\.5201 / 4\.949 / 0\.6208 / 4\.96\.14\.24\.6R9 Infeasible Derivation213 / 3\.157 / 1\.3229 / 2\.9173 / 2\.62\.12\.92\.2R10 False Rejection1,947 / 34\.71,691 / 45\.43,070 / 37\.12,828 / 37\.542\.936\.431\.8R11 False Verdict340 / 5\.9–25 / 0\.330 / 0\.40\.50\.33\.0Table 19:Extended cost: the tokens an event opens\(Qwen3\-8B\)\. For each event, the region it opens is, for Branch and Subgoal Abandonment, the steps from the event until the model returns to the left branch, else to the end; for Post\-Answer Attempt, until the commit; for Premature Finalization, Infeasible Derivation and False Verdict, every later step resolved to the event’s node or a descendant; for Derive\-after\-Prune, the later steps in the pruned subtree; for Re\-Derivation, Redundant Re\-Verification and Subgoal Churn, the steps of the burst the event starts; and for False Rejection, the steps until the rejected subtree is next entered, else to the end\. –: undefined on the task\.The token share counts the labeled steps only; the extended cost counts the region an event opens, with one row per event and one per burst for the repetition rows \(Table[19](https://arxiv.org/html/2610.22717#A8.T19)\)\. Regions of different events overlap and we do not sum them within a trajectory\. The budget a single event opens is large only for the underthinking rows and for False Rejection, whose region is the work done away from a viable branch, and the model re\-enters the rejected branch in 5% of cases pooled; a Premature Finalization’s subtree reaches the committed lineage with 45% of its tokens, and an Infeasible Derivation’s with 10% pooled and 33% on the 24 Game\. The region covers what follows the event, without causal attribution; a span to the end of the trajectory scales with where the event falls, and burst tokens count event steps only\.

### H\.4Full odds\-ratio tables

Table[20](https://arxiv.org/html/2610.22717#A8.T20)gives every model on the pooled data\. Model B \(truncated vs completed\) separates budget from accuracy: Derive\-after\-Prune \(1\.42\) and Infeasible Derivation \(1\.83\) go with running out of budget, Redundant Re\-Verification \(0\.45\) and Branch Abandonment \(0\.64\) with completing, and Premature Finalization is not associated with truncation \(0\.78, n\.s\.\) although it has the largest incorrect\-answer odds ratio\.

SubtypeA: incorrect vs correctA′: rate instead of presenceB: truncated vs completedC: within problemRepetition\(category\)0\.45 \[0\.34, 0\.61\]0\.79 \[0\.75, 0\.82\]0\.15 \[0\.06, 0\.60\]0\.83 \[0\.53, 1\.32\]R1 Re\-Derivation0\.98 \[0\.86, 1\.13\]1\.18 \[1\.05, 1\.30\]0\.82 \[0\.46, 1\.52\]1\.49 \[1\.18, 1\.88\]R2 Derive\-after\-Prune0\.66 \[0\.58, 0\.76\]1\.19 \[1\.11, 1\.28\]1\.42 \[1\.11, 1\.83\]0\.90 \[0\.72, 1\.14\]R3 Redundant Re\-Verification0\.37 \[0\.31, 0\.44\]0\.70 \[0\.67, 0\.74\]0\.45 \[0\.33, 0\.60\]0\.48 \[0\.37, 0\.63\]R4 Subgoal Churn1\.28 \[1\.12, 1\.51\]1\.92 \[1\.56, 2\.39\]1\.15 \[0\.84, 1\.66\]1\.24 \[0\.98, 1\.57\]Underthinking\(category\)0\.57 \[0\.50, 0\.66\]0\.39 \[0\.33, 0\.45\]0\.54 \[0\.44, 0\.67\]0\.42 \[0\.33, 0\.53\]R5 Branch Abandonment1\.07 \[0\.95, 1\.20\]0\.91 \[0\.77, 1\.07\]0\.64 \[0\.52, 0\.78\]0\.83 \[0\.69, 1\.02\]R6 Subgoal Abandonment1\.24 \[1\.03, 1\.46\]1\.58 \[1\.07, 2\.18\]1\.18 \[0\.87, 1\.62\]1\.18 \[0\.89, 1\.56\]R7 Post\-Answer Attempt0\.08 \[0\.07, 0\.10\]0\.08 \[0\.04, 0\.12\]0\.77 \[0\.53, 1\.15\]0\.14 \[0\.10, 0\.19\]Unsupported deduction\(category\)4\.68 \[3\.30, 7\.29\]1\.61 \[1\.50, 1\.73\]2\.66 \[1\.60, 5\.66\]8\.34 \[4\.98, 17\.68\]R8 Premature Finalization10\.12 \[8\.15, 12\.85\]154\.13 \[96\.90, 272\.89\]0\.78 \[0\.54, 1\.09\]10\.25 \[6\.98, 16\.70\]R9 Infeasible Derivation2\.90 \[2\.35, 3\.60\]2\.18 \[2\.01, 2\.37\]1\.83 \[1\.22, 3\.03\]4\.57 \[3\.22, 7\.03\]R10 False Rejection0\.87 \[0\.76, 0\.99\]0\.82 \[0\.66, 1\.02\]1\.00 \[0\.80, 1\.23\]1\.03 \[0\.84, 1\.27\]R11 False Verdict1\.70 \[1\.46, 1\.96\]2\.26 \[1\.97, 2\.70\]0\.91 \[0\.66, 1\.25\]1\.19 \[0\.93, 1\.56\]

Table 20:Odds ratios on the three\-task pool \(Qwen3\-8B\), 95% cluster\-bootstrap intervals over problems\. A: presence of the subtype, incorrect vs correct, truncated excluded, with log length and task as covariates\. A′: the subtype’s rate per 100 steps \(standardized\) in place of presence\. B: truncated vs completed\. C: conditional logit within the problems that have both outcomes \(785 strata; 739 for R4 and R6, 474 for R11\)\. Bold: Benjamini–Hochbergq<0\.05q<0\.05within the column\. The category rows include every detector row of the category, as computed\.
### H\.5The event and the committed answer

Events on the answer’s path \(%\)Answers whose path carries the row \(%\)wrong / correctSubtypeZebra24PlanZebra24PlanR1 Re\-Derivation1741587 / 6919 / 911 / 10R2 Derive\-after\-Prune411515 / 68 / 426 / 3R3 Redundant Re\-Verification46672852 / 6070 / 9343 / 55R4 Subgoal Churn–34–5 / 21 / 1R5 Branch Abandonment234723 / 203 / 56 / 8R6 Subgoal Abandonment–00–––R7 Post\-Answer Attempt4467306 / 66 / 46– / 14R8 Premature Finalization96437524 / 460 / 12 / 0R9 Infeasible Derivation614935 / 1565 / 179 / 13R10 False Rejection115110 / 40 / 12 / 2R11 False Verdict37–1123 / 3–21 / 10

Table 21:The event and the committed answer\(Qwen3\-8B\)\. Left: the share of the subtype’s events whose node lies on the dependency path of the trajectory’s committed answer\. Right: the share of incorrect, and of correct, committed answers whose dependency path contains at least one event of the subtype \(the answer\-path share of §[5](https://arxiv.org/html/2610.22717#S5)\); we exclude truncated trajectories, 19% of which on ZebraLogic commit a step they later leave\. –: undefined on the task, or no committed answers of that outcome carry the row\.Table[21](https://arxiv.org/html/2610.22717#A8.T21)gives, per subtype, how its events relate to the committed answer\. Beyond the Infeasible Derivation, correct answers’ paths carry little but Redundant Re\-Verification\. The wrong answers without an Infeasible Derivation on their path, 1/4 of the total, fail elsewhere: on the 24 Game 1/3 of them commit an answer against the model’s own state \(Premature Finalization\) and on PlanBench 1/4 rest on a False Verdict, while in 1/3 of PlanBench’s and 1/5 of the 24 Game’s the path carries none of the failure rows we define\. Reading those cases shows two other causes: a plan that stops short of the goal \(1/6 of PlanBench’s\), and a misread clue or initial state, which makes a sound\-looking path wrong and is present in about 4/5 of these trajectories, though as often in other wrong ones\.

#### Premature Finalization\.

Of the 915 premature commits on the 24 Game, 66% rest on a lineage containing an Infeasible Derivation, 43% one fired within the previous ten steps, 78% on a lineage the oracle marks non\-viable and 75% on a lineage with an unverified node; 67% had passed a check on the committed node itself; 9% rest on nothing a row names, mostly commits of an unattached root\. On ZebraLogic 44% of 92 premature commits and on PlanBench 38% of 76 rest on a lineage containing an Infeasible Derivation\. When the model later revises such a commit, it does so by committing again, and a failed check precedes the revision in only 7% of cases\.

#### Derive\-after\-Prune\.

When the prune was right, the rebuilt node reaches the committed answer in 1\.1% of events on ZebraLogic, 0\.5% on the 24 Game and 14\.9% on PlanBench; when it was wrong, an implicit repair of a False Rejection, in 33\.4% on ZebraLogic and 2\.0% on the 24 Game\.

#### Branch Abandonment\.

On the 24 Game the abandoned subtree holds the trajectory’s own solution node in 6\.9% of events\.

#### False Verdict\.

In non\-truncated ZebraLogic trajectories a false pass sits on the committed answer’s lineage in 58\.6% of events; a false fail lands on the solution node or an ancestor in 3\.8% of ZebraLogic events and 2\.1% of PlanBench events\.

### H\.6The state at the event

SubtypeEarlier state referred toZebra24PlanA\. Median steps from the event back to that stateR1 Re\-Derivationthe result re\-derived297055R2 Derive\-after\-Prunethe prune of the branch127973R3 Redundant Re\-Verificationthe check repeated211539R4 Subgoal Churnlast entry on the subgoala×\\times82–R5 Branch Abandonmentthe branch’s last new nodeb–13–B\. Share of events where the model announces the change \(%\)R5 Branch Abandonmentexplicit prune on the step–9\.5c∘\\circR2 Derive\-after\-Pruneexplicit re\-open before deriving15\.83\.5∘\\circ

Table 22:The state at the event\.A: how far back the earlier state lies that the event repeats, re\-enters or leaves, as the median number of steps over the subtype’s events \(Qwen3\-8B\)\. B: how often the model states the change in text; the remainder are silent\.×\\times: row undefined on the task; –: defined but not yet computed;∘\\circ: 0 by construction, PlanBench has no branch lifecycle\.aAgainst 4 steps for subgoals the model completes\.bAgainst 20 on the steps where an abandonment could have occurred\.cA second record on the same events gives 11\.2%; we are reconciling the two definitions\.Table[22](https://arxiv.org/html/2610.22717#A8.T22)gives, per subtype, how far back the earlier state lies that the event refers to, and whether the model announces the change\. The facts that do not generalize across subtypes are these\. A Re\-Derivation’s copy sits on the source’s own lineage in 60% of events on ZebraLogic, 24% on the 24 Game and 86% on PlanBench, and across a switch of the source’s branch in 32%, 50% and 14%; on the 24 Game it is a median 12 switches and 7 prunes from its source; the copy is a new node in 38%, 8% and 11%\. A Branch Abandonment leaves from a frontier leaf, a node with no child yet, in 67% of events on ZebraLogic, 53% on the 24 Game and 41% on PlanBench; where the model goes next is a task property, a restart from the root on the 24 Game \(66%\), a cousin case on ZebraLogic \(60%\) and a sibling step on PlanBench \(40%\)\. A Subgoal Churn on the 24 Game leaves a subgoal that had gained a node since its last entry in 63% of events, against 83% for subgoals the model completes\. A Derive\-after\-Prune re\-enters a branch whose pruned node was the model’s last open option in 47% of events on the 24 Game and 27% on ZebraLogic\.

### H\.7What follows an event

SubtypeWhat follows the eventZebra24PlanUnsupported deductionR9 Infeasible Derivationnever checked / caught by the first later check / passed by it59\.1 / 4\.7 / 36\.244\.2 / 24\.1 / 31\.740\.0 / 30\.8 / 29\.1aR8 Premature Finalizationstands as the final commit / revised later50\.6 / 49\.4 \(pooled\)R11 False Verdictfalse pass / false fail92\.5 / 7\.5–34\.4 / 65\.6R10 False Rejectionrejected branch re\-entered / never21 / 7918 / 821\.5 / 98\.5UnderthinkingR5 Branch Abandonmentnode closed before the event / on it / later / never6\.2 / 0\.0 / 10\.1 / 83\.76\.6 / 7\.9 / 3\.2 / 82\.3∘\\circmodel returns to the branch / never42\.9 / 57\.170\.7 / 29\.322\.0 / 78\.0R6 Subgoal Abandonmentsubgoal resumed / never–6 / 94 \(pooled\)R7 Post\-Answer Attemptbalternative branch opened / not52\.2 / 47\.889\.5 / 10\.597\.0 / 3\.0ends: wrong commit / truncated / commits the solution / no commit13\.4 / 67\.2 / 14\.9 / 4\.554\.3 / 34\.1 / 9\.7 / 1\.977\.2 / 0\.7 / 0\.4 / 21\.7RepetitionR2 Derive\-after\-Prunere\-entered branch was dead / wrongly closed90\.6 / 9\.478\.1 / 21\.9100 / 0cre\-entered silently / with an explicit re\-open84\.2 / 15\.896\.5 / 3\.5100 / 0cR1 Re\-Derivationlater work builds on the original / on the copy / neither52 / 19 / 2467 / 3 / 27–R4 Subgoal Churnmodel returns to the subgoal / never–89 / 11–R3 Redundant Re\-Verificationverdict unchanged / flipped99\.5 / 0\.599\.2 / 0\.895\.1 / 4\.9

Table 23:What follows an event\.For each subtype, one exhaustive partition of what the trajectory does next, read from the engine’s state, in percent of the subtype’s events \(Qwen3\-8B; rows sum to 100 within rounding\)\. –: undefined or not measured;∘\\circ: PlanBench has no branch lifecycle\.a0\.1% mixed\.bOver the failing trajectories that held a correct solution\.cBy construction: on PlanBench the row is defined as a repeat of a correctly pruned step and there is no re\-open operation\.Table[23](https://arxiv.org/html/2610.22717#A8.T23)gives, per subtype, one exhaustive partition of what the trajectory does next\. Behind the Infeasible Derivation row: among the events that are verified, the verification passes the step in 57% of cases on the 24 Game, 89% on ZebraLogic and 49% on PlanBench, and a False Verdict is later flagged on the same node in 3\.1% of ZebraLogic and 7\.3% of PlanBench false passes\. Behind the Branch Abandonment rows: a return closes the branch within ten steps in 3% of events on the 24 Game and 2% on ZebraLogic, and the returning step is a derivation in 80% and 64%\. Behind the Post\-Answer Attempt rows: a standing correct solution is falsely failed in under 1% of cases, and trajectories that open no new branch after it commit within a dozen steps; Table[24](https://arxiv.org/html/2610.22717#A8.T24)gives the failing trajectories that held a correct solution, which on the 24 Game and PlanBench spend more of their tokens after that solution than a correct trajectory does \(64 against 57% and 51 against 33%\) and on ZebraLogic less \(18 against 24%\)\. Behind the Redundant Re\-Verification row: a flipped verdict is followed by a prune within ten steps in 71% of cases on ZebraLogic, 47% on the 24 Game and 38% on PlanBench, against 4%, 9% and 16% after an unchanged one\.

Zebra24 GamePlanPooledFailing traces with a correct solution \(%\)17\.3 \[14\.4, 20\.1\]30\.8 \[27\.2, 34\.4\]14\.8 \[13\.4, 16\.2\]17\.7 \[16\.6, 18\.9\]Token share after it, failing \(%\)18645148Token share after it, correct \(%\)24573340Share of those failing traces with the subtype at or after the solution \(%\)R1 Re\-Derivation34623040R2 Derive\-after\-Prune12784147R3 Redundant Re\-Verification48827271R4 Subgoal Churn–391020R5 Branch Abandonment13633036R6 Subgoal Abandonment–2229R7 Post\-Answer Attempt12621024R8 Premature Finalization345113R9 Infeasible Derivation15698167R10 False Rejection6134027R11 False Verdict7–5645

Table 24:Failing traces that held a correct complete solution at some step \(Qwen3\-8B\)\. “Failing” is incorrect or truncated\. The first step of a correct complete solution on a node of the tree is the reference point; intervals are 95% cluster bootstraps over problems\. The population is a count, not a gain\.
### H\.8Where the budget goes

Three estimates locate the budget; all three read the 8B step table \(one row per step with its token count, the node it lands on,Arbor’s rows, the node’s oracle viability, and whether the node lies on the committed answer’s dependency path\)\.*Off\-path share*: the share of a trajectory’s tokens on steps whose node is not on the final answer’s dependency path; undefined for truncated trajectories, which commit nothing\.*Dead\-end share*: the share of tokens on steps whose node the viability oracle marks as unable to reach a correct solution and that carry no failure row\. Within questions that have both outcomes, the median truncated trajectory spends about twice the dead\-end tokens of its completed sibling \(ZebraLogic 5,065 vs 2,588; 24 Game 3,180 vs 1,580; PlanBench 10,860 vs 4,724\) while its failure\-step tokens differ by a few hundred\.*Salvage estimate*: for a truncated trajectory, we take the work it still needed to be the non\-error tokens of a completed trajectory on the same question \(median over completed siblings\) minus the non\-error tokens it had already produced; removing subtyperrwould have let it finish if the tokens on itsrr\-steps cover that shortfall\. Of 441 truncated trajectories with a completed sibling, only 20 have a positive shortfall; per\-subtype values are insalvage\_estimate\.csv\.

Off\-path %Dead\-end %Failure\-step %SalvageTaskcorr\.incorr\.compl\.trunc\.compl\.trunc\.nnrep\.allZebraLogic385225585\.56\.71793\.43\.424 Game6482324611\.712\.02321\.31\.3PlanBench869350653\.43\.4300\.00\.0Pooled629143565\.47\.54412\.02\.0

Table 25:Where the budget goes \(Qwen3\-8B; definitions in the text\)\. Off\-path: share of a trajectory’s tokens on steps whose node is not on the committed answer’s dependency path, for correct and incorrect trajectories\. Dead\-end: share of tokens on steps whose node the oracle marks unable to reach a correct solution and that carry no failure row, for completed and truncated trajectories\. Failure\-step: share of tokens on steps carrying any failure row\. Salvage: of thenntruncated trajectories with a completed sibling on the same question, the share that would have finished had the tokens of its repetition steps \(rep\.\) or of all its failure steps \(all\) been available\.Table[25](https://arxiv.org/html/2610.22717#A8.T25)gives the three estimates per task\.

#### Returns to dead regions\.

A dead region is a maximal subtree whose root the viability oracle marks unable to reach a correct solution, and a return is a later visit to the region after at least five steps on nodes outside it \(Table[26](https://arxiv.org/html/2610.22717#A8.T26)\)\. Trajectories do come back to dead regions, most non\-correct ones several times, but the returns hold about a sixth of the dead\-end tokens; most returns enter a region the model never closed, and most returning first steps carry no failure row, so a return is mostly fresh work in the region rather than a repetition\. At a gap of one step, PlanBench’s returns are one\-step hops to the parent and back along the plan chain, which is why we read the five\-step gap\.

ZebraLogic24 GamePlanBenchPooledPer dead region \(%\)re\-entered after being left18173423dead\-end tokens held by returns16171615\.7returns into a region the model had closed33491322return’s first step carries a repetition row3044213Per trajectory, correct / incorrect / truncatedshare with at least one return \(%\)33 / 60 / 8571 / 97 / 9881 / 93 / 87–median number of returns0 / 1 / 22 / 10 / 144 / 10 / 14\.5–median share of dead\-end tokens on returns \(%\)0 / 1 / 58 / 17 / 1812 / 13 / 10–

Table 26:Returns to dead regions\(Qwen3\-8B, 161,897 dead regions, gap of five steps\)\. Pooled, 78% of returns enter a region the model never closed and 87% of returning first steps carry no failure row; the median gap before a return is 28 steps, and returns hold 9\.3% of dead\-end tokens at a gap of twenty steps\.

### H\.9Robustness to the engine\-development problems

Every number in this part excludes the 40 problems per task on which we developed the parser and engine; recomputed with them \(12,078 traces\), every headline number moves in the second decimal or less, with one change of significance among reported subtypes \(pooled Branch Abandonment under Model C, 0\.83 \[0\.69, 1\.02\] without and 0\.80 \[0\.66, 0\.95\] with them\)\. The PlanBench schema validator records 618 node\-level invariant violations among the 593,057 nodes of the main\-text set \(about 1 in 1,000\): 344 parent–child kind pairs outside the schema and 274 nodes whose state differs from what the parent state and the step’s effects imply\. We record them without correcting them, and the nodes stay in every count\.

## Appendix IPrecursors: additional results

SubtypeStateLabelsPos\.Prev\. op*24 Game*R1 Re\-Derivation0\.730\.700\.500\.69R2 Derive\-after\-Prune†0\.590\.510\.500\.57R3 Redundant Re\-Verification0\.830\.810\.580\.77R4 Subgoal Churn0\.770\.640\.510\.60R5 Branch Abandonment0\.910\.810\.560\.63R7 Post\-Answer Attempt0\.870\.870\.600\.77R9 Infeasible Derivation0\.780\.710\.530\.62*ZebraLogic*R1 Re\-Derivation0\.640\.500\.510\.55R2 Derive\-after\-Prune0\.700\.680\.490\.59R3 Redundant Re\-Verification†0\.700\.660\.560\.70R9 Infeasible Derivation†0\.640\.640\.540\.61*PlanBench*R3 Redundant Re\-Verification0\.660\.560\.550\.57R9 Infeasible Derivation0\.76–0\.550\.61R10 False Rejection†0\.620\.570\.570\.59R11 False Verdict0\.62–0\.540\.56Table 27:The state before a step separates a failure from the steps where it could have happened\.Held\-out AUROC of a classifier on the tree state over the preceding steps \(State\) against each subtype’s matched opportunity steps, and the same classifier restricted to operation labels \(Labels\), to position in the trajectory \(Pos\.\), and to the previous operation \(Prev\. op\)\. Bold: state above the previous\-operation baseline by more than 0\.04;†: within 0\.04, inconclusive\. –: no label\-only feature survived screening\.SubtypeStateLabelsPos\.Prev\. op*Pooled*R1 Re\-Derivation†0\.570\.500\.500\.60R2 Derive\-after\-Prune0\.710\.510\.500\.56R3 Redundant Re\-Verification†0\.740\.730\.580\.70R9 Infeasible Derivation0\.730\.530\.530\.54Table 28:Pooled cells \(the three tasks together, with task indicators as features\), as in Table[27](https://arxiv.org/html/2610.22717#A9.T27)\.Infeasible Derivation \(R9\), 24 Game, tiles 4, 6, 8, 10 Predictive state: at most one constraint \(remaining pool\) declared on the branch over the last steps; no check for a long run 123Let me think again\.constraints on branch: 1; last check 45 steps back124Alternatively, 6 / \(\(10 \- 8\)\) = 6 / 2 = 3\.uses 6, 10, 8; pool not restated125Then 8 \* 3 = 24\.constraints on branch: 1; reuses 8126Wait, but that would use 6, 10, 8, and then 4 is left\.check: counts 8 once127So that’s not using all four numbers\.check fails on the unused 4, not on the reused 8

Figure 10:The state before an Infeasible Derivation\.Trace steps before the failure \(step numbers from the trace\); under the title, the state that predicts the failure on held\-out problems; gray: that state’s reading on each step; shaded: the failing step\. Excerpts for the other subtypes are in Figure[11](https://arxiv.org/html/2610.22717#A9.F11)\.Branch Abandonment \(R5\), 24 Game, tiles 1, 7, 8, 12 Predictive state: the leaving step is a derivation, not a check or a remark, from a frontier leaf with nothing tried under it and no verdict 11If I can get 12 multiplied by 2, but how to get 2 from 1 and 7?branch12×212\\times 2, depth 112Well, 7 \- 1 is 6, not 2\.one option closed; branch open137 divided by something?new node, depth 314Maybe 7 \- \(something\) equals 2?new node, depth 315Let me think\.no new node \(1 step\), no verdict16Alternatively, maybe using 8 and 12 together\.no new node \(2 steps\), no verdict17For example, 12 \- 8 = 4\.a derivation, from the root: leaves the frontier leaf of step 14 unjudged;12×212\\times 2was solvable \(12×\(8−7\+1\)12\\times\(8\-7\+1\)\)

Post\-Answer Attempt \(R7\), 24 Game, tiles 4, 6, 10, 11 Predictive state: many steps since the branch’s last check of the standing solution, no check in the last 5 steps 58So the answer would be \(4 \* 6\) \* \(11 \- 10\)\.correct solution; last check 1 step back59But maybe the parentheses can be arranged differently\.last check 2 steps back61Either way, the expression is valid\.4 steps back62Let me make sure there are no other operations that might be needed\.5 steps back63For example, maybe a division or subtraction that I haven’t considered\.6 steps back65Suppose I try other combinations\.8 steps back; 0 checks in the last 566For example, 11 \+ 10 = 21\.new branch instead of a commit

Redundant Re\-Verification \(R3\), 24 Game, tiles 1, 1, 1, 8 Predictive state: the node was already checked several times, the previous steps are checks of it 20Each tile is used once: yes, 8 once, each 1 once\.check 1 of8×\(1\+1\+1\)8\\times\(1\{\+\}1\{\+\}1\): passes22Let me check the calculation: 1\+1\+1 is 3, multiplied by 8 gives 24\.check 3: passes23That seems correct\.check 4: passes26So that’s valid\.tiles checked again: passes27Wait, but maybe there’s a mistake here\.no new information28Let me check again\.30So the expression 8\*\(1\+1\+1\) uses each of the four numbers once\.check 6 of the same node, same content as check 1: passes

Figure 11:The state before a failure\.Four excerpts \(step numbers from the trace\)\. Under each title, the state that predicts the failure on held\-out problems; gray: that state’s reading on the step; shaded: the failure step\.PairTaskJaccardStatePos\.Prev\. opR1 vs R2ZebraLogic0\.000\.630\.540\.57R1 vs R224 Game0\.000\.650\.540\.62R1 vs R2Pooled0\.140\.830\.600\.60R1 vs R3ZebraLogic0\.000\.840\.740\.69R1 vs R324 Game0\.000\.880\.780\.77R1 vs R3Pooled0\.130\.810\.810\.72R5 vs R724 Game0\.000\.860\.700\.69R9 vs R10PlanBench0\.000\.710\.580\.64Table 29:Sibling subtypes arise after different states\. Jaccard: overlap of the two subtypes’ stable precursor sets\. State: AUROC of a classifier on the union of the two sets at telling one sibling’s occurrences from the other’s, with the two baselines\.Re\-Derivation \(R1\), 24 Game, tiles 1, 2, 7, 10 Predictive state: no derivation and no new node on the branch in the last 3 steps 810 and 2 could be multiplied to get 20, but then I need 4 more\.derives10×2=2010\\times 2=20⋯\\cdots118Hmm, maybe \(7 \- something\)/something\.new node119Let me think\.no new node \(1 step\)120Not sure\.no new node \(2 steps\)121Wait, maybe using subtraction\.no new node \(3 steps\)122Let me think\.no new node \(4 steps\), no derivation123For example, 10 × 2 = 20\.re\-derives step 8, on a branch left 115 steps earlier

Subgoal Churn \(R4\), 24 Game, tiles 8, 8, 8, 11 Predictive state: switches in the last 5 steps with no prune on the branch 12For example, 8 \* 3 = 24, but how do I get 3 from the remaining numbers?posts goal: 3 from 8, 8, 11⋯\\cdots71Let’s think of combining the numbers in a different way\.switch 1, no prune72For example, \(11 \+ 8\) = 19\.new branch73Then maybe combine with the other 8s?no new node74Not sure\.no new node75Let me try\.no new node; no prune since the switch76Another idea: Let’s think of 24 as 8 \* 3\.switch 2 within 5 steps, no prune: re\-posts the goal of step 12, options still open

Infeasible Derivation \(R9\), PlanBench Blocksworld Predictive state: every step on the branch applicable over the last 20 steps; the step then needs a fact its own earlier step made false 433Then, step 4: put down orange on the table\.applicable; 23 valid steps in a row435Then, step 5: pick up red\.applicable \(25 in a row\)436Step 6: stack red on top of orange \(which is on the table\)\.applicable \(26\); orange is no longer clear437Now, red is on top of orange\.27⋯\\cdots441So step 7: pick up yellow\.applicable \(31\)442Step 8: stack yellow on blue\.applicable \(32\)444Now, yellow is on top of blue\.34 valid steps in a row445Then, step 9: pick up orange \(which is on the table\)\.inapplicable: red sits on orange since plan step 6 \(step 436\)446Step 10: stack orange on top of yellow\.builds on it450Which is the goal\.check passes; step 6 not re\-read

Infeasible Derivation \(R9\), ZebraLogic Predictive state: a case opened on the previous step \(ASSUME\); the derivation is the first step under it 778Case3: Carol is in house4\.case opened783Therefore, Bob must be in 5\.784But clue5 says Bob is not in house5\.fixes: Bob is not in house 5786So Carol can’t be in house4\.case closed787Case4: Carol is in house6\.ASSUME on the previous step: new case788Then Bob must be adjacent, so in house5\.first step under the case; puts Bob in house 5 against clue 5 \(step 784\)789So Carol is in 6, Bob in 5\.used792Clue13 says mystery is not in 5, which is okay here \(Carol is in 6\)\.check of clue 13 passes; clue 5 not re\-read793Now, Bob is in house5\.kept; never caught before the trace ends \(step 803\)

Derive\-after\-Prune \(R2\), ZebraLogic Predictive state: no prune in the last 10 steps; the derivation lands in a case ruled out earlier that no step re\-opened 286But house 1’s flower is lilies, so if the cruise lover is in house 1, then their flower would be carnations, which contradicts house 1’s flower being lilies\.prune: the cruise lover is not in house 1⋯\\cdots298Also, house 1’s flower is lilies\.restated in the new case; last prune 8 steps back310Also, clue 13: tennis lover is cruise lover, and clue 3: cruise lover has carnations\.20 steps without a prune316house 1: lilies26 steps without a prune319So, one of these houses has carnations \(cruise lover\)\.new node324So, the remaining vacations are cruise and beach\.new node; 34 steps without a prune325So the cruise lover is in either house 1, 2, 3, but house 1’s vacation is not determined yet\.re\-opens house 1 for the cruise lover, closed at step 286; no step re\-opened it326But house 1’s vacation could be cruise or beach\.derives inside the closed case334Since Eric’s vacation is city, and house 4 is mountain, the remaining are cruise and beach for houses 1, 2, 3\.still open 9 steps later; closed again only at step 340

Figure 12:Further excerpts, as in Figure[11](https://arxiv.org/html/2610.22717#A9.F11)\.FeatureCondition the stand\-in readsMitigation it gates, and what the gate changesBranch contentthe constraints fixed on the branch within the last ten stepsa thinking intervention\([Wu et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib36)\), a chain\-of\-verification\([Dhuliawala et al\., 2023](https://arxiv.org/html/2610.22717#bib.bib7)\)or a process reward\([Lightman et al\., 2024](https://arxiv.org/html/2610.22717#bib.bib17);[Wang et al\., 2024](https://arxiv.org/html/2610.22717#bib.bib31)\): the inserted text or the verification question names those constraints, so the check lands on the constraint an Infeasible Derivation violatesFrontier statuswhether the current frontier leaf has a verdictthe thought\-switch penalty\([Wang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib33)\): applied only while the frontier leaf has no verdict, so it stops Branch Abandonment and spares the switches that leave a ruled\-out branch, which an unconditional penalty suppresses as wellFrontier statuswhether a checked complete solution standsbudget forcing\([Muennighoff et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib24)\)and confidence\-gated early exit\([Yang et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib39);[Zhang et al\., 2025a](https://arxiv.org/html/2610.22717#bib.bib42)\): generation stops when a checked complete solution stands rather than at a length or confidence threshold, which ends the Post\-Answer Attempt without cutting off unfinished searchRecent operationsthe branch’s last operation and its pruned statussteering vectors for verification and backtracking\([Venhoff et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib30);[Ward et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib34)\): the verification vector is damped right after a check, which removes Redundant Re\-Verification, and the backtracking vector is damped on a pruned branch, which removes Derive\-after\-PruneOfflinethe return rows and the dead\-end subtrees opened at a departurespans to remove when compressing trajectories for fine\-tuning\([Xia et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib37);[Zhao et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib45);[Munkhbat et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib25)\), replacing token\-importance or length criteriaOfflinethe per\-step failure rowsa structure\-aware penalty for length\-harmonizing reinforcement learning\([Luo et al\., 2025a](https://arxiv.org/html/2610.22717#bib.bib20);[Chen et al\., 2025b](https://arxiv.org/html/2610.22717#bib.bib5)\)that charges a repeated operation rather than every token past the answerOfflinethe per\-trajectory failure countsselection among sampled trajectories\([Hassid et al\., 2025](https://arxiv.org/html/2610.22717#bib.bib10)\)by what they contain rather than by lengthTable 30:The two routes, feature by feature\.Route 1 \(first four rows\): a stand\-in trained onArbor’s labels reads the condition that precedes the failure and gates an existing inference\-time mitigation on it\. Route 2 \(last three rows\):Arbor’s labels, computed offline, replace length and token criteria in training\-side methods\.#### Opportunity sets and matching\.

An opportunity step is a step on which the subtype’s precondition holds on the pre\-step state \(containment of the occurrences is 100% on every analyzed row\)\. We match five controls per occurrence on task, parser label and trajectory tenth, preferring the same trajectory, then the same problem\. The features are 639 quantities of the pre\-step state and the operation labels over windows of 1 to 20 steps and since the branch start\. We screen them by standardized mean difference with Benjamini–Hochbergq=0\.1q=0\.1, keep the top 60, and fit gradient\-boosted trees \(LightGBM; at most 300 trees with early stopping on grouped five\-fold cross\-validation over the development problems, learning rate 0\.05, 15 leaves, minimum 20 samples per leaf, 0\.8 row and column subsampling, L2 regularization 1\.0, class\-balanced\) with stability selection; cells with 100 to 300 development events use nested cross\-validation\. We hold out 40 development problems per task and report every AUROC on the remaining problems\. Two LLM judges, with a third as tie\-break, judged 20 sampled opportunity cards per subtype against the trace text; acceptance 1\.00 \(Post\-Answer Attempt\), 0\.85 \(Derive\-after\-Prune, Redundant Re\-Verification, Subgoal Churn, False Rejection\), 0\.75 \(Branch Abandonment, Infeasible Derivation\), 0\.70 \(False Verdict\), 0\.60 \(Re\-Derivation\)\. 13 of 19 cells use a tightened opportunity set \(we add a recency condition so that the detector’s own firing quantity differs between occurrences and controls\); the six cells on the main set are Re\-Derivation on the 24 Game and pooled, Redundant Re\-Verification and Branch Abandonment on the 24 Game, Derive\-after\-Prune on ZebraLogic, and Infeasible Derivation on PlanBench\.

#### Rare subtypes\.

Seven of the eleven subtypes have enough occurrences for a held\-out estimate, the four repetition subtypes, Branch Abandonment, Post\-Answer Attempt and Infeasible Derivation \(the rare ones in Appendix[I](https://arxiv.org/html/2610.22717#A9)\); we model False Rejection on PlanBench, but the estimate is inconclusive, and False Verdict there separates only weakly\. We observe every subtype in the taxonomy at least 76 times at 8B\. Five task\-rows fall below 10% prevalence \(ZebraLogic Post\-Answer Attempt 8\.4% and Premature Finalization 3\.2%; PlanBench Subgoal Abandonment 4\.5%, Post\-Answer Attempt 7\.5% and Premature Finalization 1\.6%\); all five are present at 32B and, except PlanBench Premature Finalization, at 1\.7B\. Three are rare because the model rarely takes a common opportunity \(ZebraLogic Post\-Answer Attempt: the opportunity holds in 72% of trajectories and is taken at 0\.0022 per step; PlanBench Subgoal Abandonment 99\.8% and 0\.0003; PlanBench Post\-Answer Attempt 34% and 0\.0023\), one because a commit run before the answer is itself uncommon and then rarely invalid \(ZebraLogic Premature Finalization: 69% of trajectories, 0\.030\), and PlanBench Premature Finalization is the inverse on a lower\-bound count \(an in\-reasoning commit run in 1\.9% of trajectories, invalid in 75% of them\)\.

SubtypeTaskSetStable precursor, in wordsPrec\.Rec\.R1 Re\-Derivation24 Gamemainno derivation and no new node on the branch in the last 3 steps0\.320\.58R1 Re\-DerivationZebraLogictighta node was created on the branch within the last 10 steps0\.330\.20R1 Re\-DerivationPooledmainopen subgoals falling while the branch adds no node0\.170\.66R2 Derive\-after\-Prune24 Gametighta long run since the branch’s last check, confirm or assumption; no node created on the previous step0\.200\.70R2 Derive\-after\-PruneZebraLogicmainno prune in the last 10 steps1\.000\.31R2 Derive\-after\-PrunePooledtighta long run since the trajectory’s last prune relative to recent node creation0\.160\.59R3 Redundant Re\-Verification24 Gamemainthe branch’s last check is recent and the previous step is a check0\.830\.19R3 Redundant Re\-VerificationZebraLogictightmany remarks \(META\) in the last 20 steps and a long run since the branch’s last check0\.380\.05R3 Redundant Re\-VerificationPlanBenchtighta check was recent on the branch while few nodes were created in the last 20 steps0\.370\.10R3 Redundant Re\-VerificationPooledtightthe branch’s last check is recent relative to the trajectory’s0\.490\.36R4 Subgoal Churn24 Gametightswitches in the last 5 steps without a prune on the branch0\.490\.59R5 Branch Abandonment24 Gamemainthe step is a derivation \(not a check, restatement or remark\) at recent depth above 1\.80\.470\.64R7 Post\-Answer Attempt24 Gametighta long run since the branch’s last check and no check in the last 3 to 5 steps0\.300\.42R9 Infeasible Derivation24 Gametightat most one constraint \(remaining pool\) declared on the branch over the last 3 to 20 steps0\.580\.51R9 Infeasible DerivationZebraLogictightan assumption \(new case\) on the previous step; the derivation is the first step under it0\.500\.17R9 Infeasible DerivationPlanBenchmainthe branch has been oracle\-viable over the last 20 steps0\.350\.84R9 Infeasible DerivationPooledtightthe branch has been viable recently and its last check is recent0\.350\.42R10 False RejectionPlanBenchtightshorter steps than average over the last 10 to 20 steps0\.260\.80R11 False VerdictPlanBenchtighthigh recent fan\-out on the branch0\.520\.12

Table 31:The stable precursor of each modeled cell in words \(the feature set selected in every refit\), the opportunity set it was fit on \(main, or tightened by a recency condition\), and the test precision and recall of the best single rule on that set\.ZebraLogic24 GamePlanBenchSubtypevisitsagepath %sub\.visitsagepath %sub\.visitsagepath %sub\.R1 Re\-Derivation2 / 26 / 221 / 289 / 172 / 090 / 04 / 76 / 23 / 299 / 315 / 1223 / 5R2 Derive\-after\-Prune3 / 223 / 34 / 242 / 142 / 077 / 01 / 71 / 20 / 00 / 015 / 108 / 5R3 Redundant Re\-Verification18 / 1246 / 1746 / 431 / 238 / 354 / 567 / 291 / 17 / 525 / 629 / 155 / 4R4 Subgoal Churn––––2 / 096 / 03 / 77 / 24 / 139 / 14 / 93 / 4R5 Branch Abandonment4 / 79 / 823 / 572 / 122 / 115 / 174 / 532 / 13 / 48 / 57 / 143 / 6R6 Subgoal Abandonment––––2 / 97 / 130 / 452 / 1––––R7 Post\-Answer Attempt29 / 3030 / 4144 / 731 / 130 / 1364 / 1967 / 482 / 17 / 394 / 630 / 233 / 4R8 Premature Finalization33 / 2743 / 3496 / 961 / 18 / 3912 / 5251 / 751 / 10 / 130 / 18775 / 772 / 26R9 Infeasible Derivation0 / 00 / 06 / 324 / 200 / 00 / 015 / 121 / 20 / 00 / 09 / 107 / 5R10 False Rejection2 / 219 / 611 / 56 / 22 / 13 / 15 / 01 / 12 / 24 / 41 / 11 / 1R11 False Verdict14 / 2116 / 2337 / 633 / 2––––2 / 34 / 511 / 143 / 3

Table 32:The node at the event, against the nodes at which the row could have fired\.Each cell is event median / opportunity\-step median \(Qwen3\-8B\): earlier visits to the node, node age in steps, share of events whose node lies on the committed answer’s path, and final subtree size\. This is the univariate view of the contrast that §[6](https://arxiv.org/html/2610.22717#S6)models; the full feature set and held\-out AUROC are in Table[27](https://arxiv.org/html/2610.22717#A9.T27)\. –: row undefined on the task\. R8 on PlanBench has 76 events against 26 opportunity steps\.Table[32](https://arxiv.org/html/2610.22717#A9.T32)gives the univariate form of the same contrast: four descriptive quantities of the node at the event against the row’s opportunity steps, per task\. Table[31](https://arxiv.org/html/2610.22717#A9.T31)gives the stable precursor of every modeled cell in words, with the test precision and recall of the best single rule; Table[29](https://arxiv.org/html/2610.22717#A9.T29)gives the sibling\-pair separation behind the last sentence of the first paragraph of §[6](https://arxiv.org/html/2610.22717#S6), and Figures[10](https://arxiv.org/html/2610.22717#A9.F10),[11](https://arxiv.org/html/2610.22717#A9.F11)and[12](https://arxiv.org/html/2610.22717#A9.F12)the excerpts\.

#### Pre\-states by subtype\.

The pre\-states behind the three points of §[6](https://arxiv.org/html/2610.22717#S6)are these\. An Infeasible Derivation violates a constraint the model itself fixed within the previous ten steps in about 3/4 of cases, on the 24 Game on a branch where it has stopped restating what remains, and the derivation is then left unverified in about half of cases and usually passed by the first verification that reaches it\. A Branch Abandonment leaves on a derivation rather than a verification, from a frontier leaf with nothing tried under it and no verdict on it; a Post\-Answer Attempt leaves a solution’s branch after a run with no verification of the solution\. A Redundant Re\-Verification re\-verifies a node already verified several times, shortly after another verification of it; a Re\-Derivation comes when the branch has produced no derivation and no new node in its last three steps; a Derive\-after\-Prune comes inside a branch still closed, on the 24 Game after a long run since the branch’s last verification or assumption and on ZebraLogic with no prune in the last ten steps; Subgoal Churn follows switching without pruning\. These states tell apart siblings of one failure mode: a stalled branch before a Re\-Derivation against a long run since the last prune before a Derive\-after\-Prune, and a young frontier leaf before a Branch Abandonment against a solution left unverified before a Post\-Answer Attempt\.

#### Routes, point by point\.

Table[30](https://arxiv.org/html/2610.22717#A9.T30)lists, for each point, the condition a stand\-in trained onArbor’s labels reads and the existing mitigation it gates, and for the offline route the criterionArbor’s labels replace\.