The Knowing-Saying Gap: When Probes See Errors that Confidence Misses

# The Knowing-Saying Gap: When Probes See Errors that Confidence Misses
Source: [https://arxiv.org/html/2608.07528](https://arxiv.org/html/2608.07528)
Jyotin Goel Ipshita Bandyopadhyay11footnotemark:1Justin Shenk Correspondence:jyotinofficialcc@gmail\.com

###### Abstract

Linear probes detect corrupted context in language models with near\-perfect accuracy, yet this does not translate into reliable failure prediction\. The result is a dissociation with direct implications for deployment monitoring\. Across multi\-hop arithmetic chains, probes that detect corruption turn out to be*uninformative*about final answer correctness; models forced into structured confidence formats collapse to two values with indistinguishable error rates; and probe persistence across hops fails to separate correct from incorrect outcomes, refuting our pre\-registered “persistence beats peak” hypothesis\. This pattern of*knowing but not saying*generalises across model families including reasoning models\. As a real\-time monitor, probe\-based interventions are sharply model and error\-type dependent: branch\-and\-pick is net\-positive across models and uniquely non\-breaking on Llama\-3\.1\-8B\(4 rescued, 0 broken\), while reprompt and replace\-prior break correct traces at roughly the rate they rescue wrong ones\. Probe\-based monitoring is a necessary complement to verbalised confidence, but no single intervention dominates, and the deployable answer is model\-aware, error\-type\-aware routing\.

The Knowing\-Saying Gap: When Probes See Errors that Confidence Misses

Jyotin Goel††thanks:Equal contribution\.Ipshita Bandyopadhyay11footnotemark:1Justin ShenkCorrespondence:jyotinofficialcc@gmail\.com

## 1Introduction

Large language models are increasingly deployed not just to answer questions but to act: writing and executing code, navigating multi\-step plans, and operating as nodes in agentic pipelines\(Wang et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib26); Yao et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib32); Schick et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib20)\)\. In these settings, a wrong answer is not merely unhelpful, it is a seed\. An error at step one propagates into step two’s context, until the accumulated deviation is unrecoverable\(Huang et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib10); Pan et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib18)\)\. The model continues in the same fluent register with no hedging, no slowdown, and no visible sign that anything has gone wrong\(Kadavath et al\.,[2022](https://arxiv.org/html/2608.07528#bib.bib12); Lin et al\.,[2022](https://arxiv.org/html/2608.07528#bib.bib15)\)\. This failure mode is already observable in production\. Coding agents such as SWE\-agent\(Yang et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib31)\), Devin\(Cognition AI,[2024](https://arxiv.org/html/2608.07528#bib.bib7)\), and Claude Code\(Anthropic,[2024b](https://arxiv.org/html/2608.07528#bib.bib2)\)operate in edit\-execute\-observe loops where a misdiagnosed root cause at turn one constrains every subsequent patch\. SWE\-bench evaluations show that even the strongest agents fail on the majority of tasks\(Jimenez et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib11)\), and qualitative analysis suggests that cascading context corruption, rather than capability, is a primary bottleneck\(Yang et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib31)\)\. The same structure appears in LLM\-based medical decision support\(Singhal et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib22)\), long\-horizon task planning\(Valmeekam et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib25)\), and multi\-agent coordination\(Chan et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib6)\): any setting where a model’s output feeds back into its own future context\.

This fluency without fidelity is the central reliability problem of deployed language models, and it will intensify\. As inference\-time compute scales and models are given longer context windows, tool access, and persistent memory, the gap between what a model has internally computed and what it chooses to say becomes both larger and more consequential\(Anthropic,[2024a](https://arxiv.org/html/2608.07528#bib.bib1); OpenAI,[2023](https://arxiv.org/html/2608.07528#bib.bib17); Wei et al\.,[2022](https://arxiv.org/html/2608.07528#bib.bib28)\)\. A model that stutters on uncertain ground would be easy to monitor\. A model that produces wrong answers in the same confident register as correct ones requires a fundamentally different approach\.

The natural response is to ask the model directly\. Elicited confidence, chain\-of\-thought self\-critique, and structured verification have all been proposed as mitigations\(Kadavath et al\.,[2022](https://arxiv.org/html/2608.07528#bib.bib12); Xiong et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib30); Weng et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib29); Shinn et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib21)\)\. These share an implicit assumption: that the model’s internal state is accessible through its verbal outputs\. We show that this assumption fails in an informative way\.

#### This paper\.

We study multi\-hop arithmetic chains with silently corrupted prior context, a controlled setting that isolates the core failure mode of production agentic loops while holding the surface form of the task fixed\. A linear probe on the residual stream detects the corrupted context with near\-perfect accuracy, yet does not predict whether the final answer is wrong\. Structured confidence elicitation collapses to a binary signal with indistinguishable error rates\. Real\-time interventions based on the probe signal vary sharply by model and error type: branch\-and\-pick is the only strictly non\-breaking policy, while reprompt and replace\-prior rescue wrong traces and break correct ones at comparable rates\. Together these constitute the*knowing but not saying*dissociation\. The model has linearly encoded the relevant fact about its context, but that information does not propagate into its verbal outputs or downstream behaviour\. As coding agents and agentic pipelines become standard infrastructure, probe\-based monitoring of internal state is a necessary complement to verbalised confidence, since the alternative is trusting a channel we show to be uninformative\.

Contrastive dataset1,400 traces⋅\\cdot5 error types⋅\\cdot2 to 4 hopsS1Activation collectionS2Probe trainingS3Behavioral analysisS4Intervention experimentS5Cross\-model comparison3 conditionsper problemLayer\-wise AUROC& CVDetection vsfailure AUROCC1⋅\\cdotC2⋅\\cdotC33 models5 models2 familiesFigure 1:Overview of the five\-stage experimental pipeline\. S1 collects activations and surface signals; S2 trains a layer\-wise linear probe; S3 measures the detection versus failure dissociation; S4 evaluates three probe\-triggered intervention policies; S5 aggregates results across all five models\.
#### Contributions\.

\(1\) A contrastive multi\-hop math dataset of 1,400 traces over 500 base problems with five error types, evaluated across two model families \(Qwen and Llama\) spanning instruction\-tuned and reasoning variants \([Section˜3](https://arxiv.org/html/2608.07528#S3)\)\. \(2\) A characterisation of the detection versus failure\-prediction collapse, with a head\-to\-head against six surface uncertainty signals, replicated across models \([Section˜5](https://arxiv.org/html/2608.07528#S5)\)\. \(3\) A controlled comparison of three probe\-triggered interventions with an error\-type breakdown and cross\-model analysis, motivating model\-aware routed policies \([Section˜5](https://arxiv.org/html/2608.07528#S5.SS0.SSS0.Px5)\)\. \(4\) A pre\-registered hypothesis that we report as refuted: persistence of probe activation across hops does not predict failure\([Appendix˜K](https://arxiv.org/html/2608.07528#A11)\)\. \(5\) Two robustness controls, base\-problem\-grouped cross\-validation and leave\-one\-error\-type\-out evaluation, showing the detection signal is neither a fold\-leakage artifact nor a per\-type surface signature \([Appendices˜C](https://arxiv.org/html/2608.07528#A3)and[D](https://arxiv.org/html/2608.07528#A4)\)\.

## 2Related work

#### Probing internal representations\.

Linear probes have been used to extract structural and semantic features from transformer activations, from syntax\(Hewitt and Manning,[2019](https://arxiv.org/html/2608.07528#bib.bib9)\)to truthfulness\(Burns et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib5); Marks and Tegmark,[2023](https://arxiv.org/html/2608.07528#bib.bib16)\)and refusal\(Arditi et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib3)\)\. Our finding extends probing to*operational state*, namely whether an ongoing computation is operating on corrupted input\. We emphasise that successful probing of state does not entail success at predicting behaviour, a distinction we believe has not been pulled apart this cleanly before\.

#### Calibration and verbalised confidence\.

A growing literature trains or prompts models to express uncertainty in language\(Kadavath et al\.,[2022](https://arxiv.org/html/2608.07528#bib.bib12); Tian et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib23); Lin et al\.,[2022](https://arxiv.org/html/2608.07528#bib.bib15); Xiong et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib30)\)\. Most of this work treats miscalibration as a noisy quantitative phenomenon, with confidences that are too high or insufficiently spread\. We argue that, at least for the models and task we study, verbalised confidence is*categorically*miscalibrated: it functions as a binary plausibility filter on the output rather than as graded introspection on the chain\.

#### Chain\-of\-thought faithfulness\.

Recent work has argued that chain\-of\-thought explanations can be unfaithful\(Turpin et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib24); Lanham et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib13); Paul et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib19)\)\. We contribute a positive*decoding*result\. The intermediate state*does*contain the corruption signal at high fidelity; the unfaithfulness lies in whether that information is consulted by the surface\-output layers that ultimately produce the answer string\.

#### Process supervision and self\-correction\.

Process reward models\(Lightman et al\.,[2024](https://arxiv.org/html/2608.07528#bib.bib14)\)and self\-consistency\(Wang et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib27)\)attack the same problem with surface signals\. We use an internal probe as the scoring function for branch\-and\-pick\. Our earlier unpublished work claimed that this gave a strict asymmetric safety property\. The present paper revises that claim and shows it survives only when conditioned on error type\.

## 3Dataset

We construct a contrastive multi\-hop math dataset of 1,400 traces over 500 base problems, generated deterministically from a single seed \(42\) with no human annotation and no language\-model involvement \(full generation procedure in[Appendix˜A](https://arxiv.org/html/2608.07528#A1)\)\.

#### Problem generation\.

Twelve subcategory\-specific template functions cover arithmetic, percentages, rates, fractions, counting, geometry, compound interest, and nested fractions across 2 to 4 hops\. Each draws numeric parameters from fixed candidate sets via a seededrandom\.Randominstance and computes every intermediate result exactly, so prompts, correct responses, and ground\-truth values are derived algebraically rather than sampled from a model\.

#### Error injection\.

Each base problem yields acleanvariant and one to threeerror\_at\_kvariants in which the assistant turn at hopkkis replaced by a synthetically wrong response computed deterministically from the correct operands\. Five error types span qualitatively distinct failure modes:off\_by\_one\(\+1\+1\),wrong\_operator\(e\.g\.×⁣→⁣\+\\times\\to\+\),wrong\_unit\(correct magnitude, wrong unit\),magnitude\_error\(×10\\times 10\), andwrong\_percentage\_base\(100−p100\-pforpp\); see[Appendix˜A](https://arxiv.org/html/2608.07528#A1)for motivation\. Types are assigned by cycling through the five in order \(∼\\sim100 traces each\), and all hops downstream of the injection are re\-templated from the wrong upstream value, so propagation is realistic rather than hand\-constructed\. Cycling assigns each error type to∼\\sim100 of the 500 base problems; because longer problems contribute multipleerror\_at\_kvariants, this yields roughly 180 corrupted traces per type across the 900 corrupted traces\.[Table˜1](https://arxiv.org/html/2608.07528#S3.T1)gives the variant distribution\.

Table 1:Variant distribution across the dataset\.

## 4Methodology

### 4\.1Experimental setup

We investigate the knowing\-saying gap across five models spanning two architecture families, two scales, and one reasoning variant\. All models are loaded in bf16 full precision with frozen weights throughout, so no fine\-tuning occurs at any stage\. The probe is the only learned component, trained exclusively on activations extracted from the frozen model\. Models are summarised in[Table˜2](https://arxiv.org/html/2608.07528#S4.T2)\.

The thinking\-mode variant differs from the standard pipeline in exactly one way: extended chain\-of\-thought reasoning is enabled at inference time\. All other hyperparameters, dataset subsets, and evaluation procedures are held fixed, making this a single\-variable ablation of whether chain\-of\-thought reasoning closes the knowing\-saying gap\.

Table 2:Models evaluated\. All models are loaded in bf16 full precision with frozen weights\. The thinking\-mode variant of Qwen3\-4B differs from the standard variant solely in having extended chain\-of\-thought reasoning enabled at inference time\.
### 4\.2Contrastive probing design

The central choice is a contrastive pairing within each base problemii: a clean context𝐱i\(0\)\\mathbf\{x\}\_\{i\}^\{\(0\)\}and a corrupted context𝐱i\(1\)\\mathbf\{x\}\_\{i\}^\{\(1\)\}differing only in the embedded hop\-0 answer \(one replaced by a synthetically wrong value\), sharing question, hop structure, and hop\-1 prompt\. Representation differences are thus attributable to the injected error, controlling for difficulty and hop structure and ruling out a probe that scores high by learning difficulty rather than error presence\.111This controls for problem identity, not every surface property: the contexts still differ in the literal corrupted value, so some surface cues \(digits, units, magnitude, length\) correlate with the label by construction\. We test generalisation beyond surface artifacts with a leave\-one\-error\-type\-out evaluation \([AppendixD](https://arxiv.org/html/2608.07528#A4)\)\.Each base problem contributes three conditions:Clean\(y=0y=0\);Error\-standard\(y=1y=1\), error at hop 0, the primary probing condition; andError\-verbalized\(y=1y=1\), identical but eliciting a confidence score, to test whether elicitation changes behavior independently of internal state\. Two further passes yieldfinal\_degraded– whether the injection produced a wrong final answer on an otherwise\-solvable problem – the target for failure prediction \([Section˜4\.7](https://arxiv.org/html/2608.07528#S4.SS7)\) and intervention evaluation \([Section˜4\.8](https://arxiv.org/html/2608.07528#S4.SS8)\)\.

### 4\.3Probe robustness controls

Two controls test whether the probe captures an operational error state rather than memorised surface structure\.Grouped cross\-validationreplaces the stratified folds of[Section˜4\.6](https://arxiv.org/html/2608.07528#S4.SS6)with five folds that hold out entire base problems, so no template or near\-duplicate context is shared across train and test\.Held\-out error typetrains the probe on four of the five error types and evaluates detection on the fifth, repeated for all five leave\-one\-out splits; above\-chance transfer to a never\-seen surface corruption indicates a corruption feature that generalises across surface forms rather than a per\-type token signature\. Both controls reuse the best aggregation mode from[Section˜4\.6](https://arxiv.org/html/2608.07528#S4.SS6); the grouped analysis selects its layer within the grouped folds\. Full results are in[Appendices˜C](https://arxiv.org/html/2608.07528#A3)and[D](https://arxiv.org/html/2608.07528#A4)\.

### 4\.4Activation extraction

Each forward pass captures the full stack of hidden states\{𝐡\(ℓ\)\}ℓ=0L\\\{\\mathbf\{h\}^\{\(\\ell\)\}\\\}\_\{\\ell=0\}^\{L\}at every layerℓ\\ell, including the embedding layer atℓ=0\\ell=0\. For a prompt ofTTtokens, each𝐡\(ℓ\)∈ℝT×dmodel\\mathbf\{h\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{T\\times d\_\{\\text\{model\}\}\}\. Two aggregations produce fixed\-size representation vectors per layer:

𝐚last\(ℓ\)\\displaystyle\\mathbf\{a\}^\{\(\\ell\)\}\_\{\\text\{last\}\}=𝐡T\(ℓ\)∈ℝdmodel,\\displaystyle=\\mathbf\{h\}^\{\(\\ell\)\}\_\{T\}\\in\\mathbb\{R\}^\{d\_\{\\text\{model\}\}\},\(1\)𝐚mean\(ℓ\)\\displaystyle\\mathbf\{a\}^\{\(\\ell\)\}\_\{\\text\{mean\}\}=1T​∑t=1T𝐡t\(ℓ\)∈ℝdmodel\.\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbf\{h\}^\{\(\\ell\)\}\_\{t\}\\in\\mathbb\{R\}^\{d\_\{\\text\{model\}\}\}\.
The last\-token aggregation captures the representation at the position that conditions autoregressive generation, and is the standard choice in the probing literature\(Belinkov,[2022](https://arxiv.org/html/2608.07528#bib.bib4); Gurnee et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib8)\)\. The mean\-over\-tokens aggregation captures a whole\-prompt summary signal\. We include both because deeper models may distribute the error signal across token positions rather than concentrating it at the final token; selecting the better aggregation per layer is part of the model selection procedure described in[Section˜4\.6](https://arxiv.org/html/2608.07528#S4.SS6)\.

We visualise how the mean\-pooled signal accumulates across token positions in[Appendix˜I](https://arxiv.org/html/2608.07528#A9), which clarifies why mean pooling is informative even at shallow layers\.

### 4\.5Surface uncertainty signals

In parallel with activation extraction, six scalar uncertainty signals are computed from the per\-token logit distributionptp\_\{t\}\. Letpt\(1\)≥pt\(2\)≥…p\_\{t\}^\{\(1\)\}\\geq p\_\{t\}^\{\(2\)\}\\geq\\ldotsbe the sorted token probabilities at positiontt, letH​\(pt\)=−∑vpt​\(v\)​log⁡pt​\(v\)H\(p\_\{t\}\)=\-\\sum\_\{v\}p\_\{t\}\(v\)\\log p\_\{t\}\(v\)be the Shannon entropy, andGGthe number of generated tokens:

speak\-ent\\displaystyle s\_\{\\text\{peak\-ent\}\}=maxt⁡H​\(pt\),\\displaystyle=\\max\_\{t\}\\,H\(p\_\{t\}\),\(2\)smean\-ent\\displaystyle s\_\{\\text\{mean\-ent\}\}=1G​∑t=1GH​\(pt\),\\displaystyle=\\frac\{1\}\{G\}\\sum\_\{t=1\}^\{G\}H\(p\_\{t\}\),searly\-ent\\displaystyle s\_\{\\text\{early\-ent\}\}=1min⁡\(G,20\)​∑t=1min⁡\(G,20\)H​\(pt\),\\displaystyle=\\frac\{1\}\{\\min\(G,20\)\}\\sum\_\{t=1\}^\{\\min\(G,20\)\}H\(p\_\{t\}\),smin\-conf\\displaystyle s\_\{\\text\{min\-conf\}\}=mint⁡pt\(1\),\\displaystyle=\\min\_\{t\}\\,p\_\{t\}^\{\(1\)\},svocab\-gap\\displaystyle s\_\{\\text\{vocab\-gap\}\}=1G​∑t=1G\(pt\(1\)−pt\(2\)\),\\displaystyle=\\frac\{1\}\{G\}\\sum\_\{t=1\}^\{G\}\\bigl\(p\_\{t\}^\{\(1\)\}\-p\_\{t\}^\{\(2\)\}\\bigr\),slogprob\\displaystyle s\_\{\\text\{logprob\}\}=−1G​∑t=1Glog⁡pt​\(yt\)\.\\displaystyle=\-\\frac\{1\}\{G\}\\sum\_\{t=1\}^\{G\}\\log p\_\{t\}\(y\_\{t\}\)\.These span three hypotheses about where uncertainty surfaces: distributional spread \(entropy\), top\-token certainty \(confidence\), and top\-two margin \(vocabulary gap\)\. Together they form the surface baselines against which the probe is compared\.222For the AUROC comparisons \([Equations4](https://arxiv.org/html/2608.07528#S4.E4)and[6](https://arxiv.org/html/2608.07528#S4.E6)\),smin\-confs\_\{\\text\{min\-conf\}\}andsvocab\-gaps\_\{\\text\{vocab\-gap\}\}are sign\-flipped so larger values indicate the error class; reported sub\-0\.5 values are therefore genuinely non\-predictive rather than orientation artifacts\. Signal interpretations and the four behavioral text flags are detailed in[AppendixB](https://arxiv.org/html/2608.07528#A2)\.

### 4\.6Layer\-wise linear probe

A linear probe is trained independently at every layerℓ∈\{0,…,L\}\\ell\\in\\\{0,\\ldots,L\\\}under both aggregation modes\. The probe is a logistic regression classifier with input standardisation:

y^=σ​\(𝐰⊤​𝐚−𝝁𝝈\+b\),\\hat\{y\}=\\sigma\\\!\\left\(\\mathbf\{w\}^\{\\top\}\\frac\{\\mathbf\{a\}\-\\bm\{\\mu\}\}\{\\bm\{\\sigma\}\}\+b\\right\),\(3\)
where𝝁\\bm\{\\mu\}and𝝈\\bm\{\\sigma\}are the per\-feature mean and standard deviation estimated from the training split,𝐰∈ℝdmodel\\mathbf\{w\}\\in\\mathbb\{R\}^\{d\_\{\\text\{model\}\}\}andb∈ℝb\\in\\mathbb\{R\}are the learned weights and bias, andσ​\(⋅\)\\sigma\(\\cdot\)is the sigmoid function\.

We use a linear probe deliberately\. A nonlinear classifier could achieve high accuracy by learning complex decision boundaries that do not correspond to any interpretable direction in the residual stream\. A linear probe succeeds only if the error signal is*linearly decodable*, that is, if clean and corrupted representations are linearly separable in activation space\. This is the standard operationalisation of the claim that the model has encoded a feature in the mechanistic interpretability literature\(Belinkov,[2022](https://arxiv.org/html/2608.07528#bib.bib4); Gurnee et al\.,[2023](https://arxiv.org/html/2608.07528#bib.bib8)\)\.

#### Layer selection\.

Each layer/mode combination is evaluated under 5\-fold stratified cross\-validation, scored by AUROC:

AUROC=Pr⁡\[y^​\(𝐚\+\)\>y^​\(𝐚−\)\],\\text\{AUROC\}=\\Pr\\\!\\left\[\\hat\{y\}\(\\mathbf\{a\}^\{\+\}\)\>\\hat\{y\}\(\\mathbf\{a\}^\{\-\}\)\\right\],\(4\)for a randomly drawn error sample𝐚\+\\mathbf\{a\}^\{\+\}and clean sample𝐚−\\mathbf\{a\}^\{\-\}, reported as mean±\\pmstd across folds\. The highest\-mean\-AUROC layer/mode is the best layer; a final probe is fit on allNNsamples there for downstream stages\.333Shared base problems and templates mean stratified folds may leak near\-duplicate contexts across train/test; the contrastive control \([Section4\.2](https://arxiv.org/html/2608.07528#S4.SS2)\) mitigates but does not eliminate this, so we additionally run grouped cross\-validation holding out entire base problems \([AppendixC](https://arxiv.org/html/2608.07528#A3)\); detection survives this stricter control\.

#### Threshold calibration\.

The decision thresholdτ\\tauis calibrated on out\-of\-fold scores\. Three modes are derived\. By constructionτloose=0\.7​τYouden<τYouden\\tau\_\{\\text\{loose\}\}=0\.7\\,\\tau\_\{\\text\{Youden\}\}<\\tau\_\{\\text\{Youden\}\}; the relation ofτconservative\\tau\_\{\\text\{conservative\}\}toτYouden\\tau\_\{\\text\{Youden\}\}is empirical, and on the highly separable score distributions we observe the two nearly coincide \([Appendix˜E](https://arxiv.org/html/2608.07528#A5)\):

τYouden\\displaystyle\\tau\_\{\\text\{Youden\}\}=arg⁡maxτ⁡\[TPR​\(τ\)−FPR​\(τ\)\],\\displaystyle=\\arg\\max\_\{\\tau\}\\bigl\[\\text\{TPR\}\(\\tau\)\-\\text\{FPR\}\(\\tau\)\\bigr\],\(5\)τconservative\\displaystyle\\tau\_\{\\text\{conservative\}\}=Fy=0−1​\(0\.99\),\\displaystyle=F^\{\-1\}\_\{y=0\}\(99\),τloose\\displaystyle\\tau\_\{\\text\{loose\}\}=0\.7​τYouden\.\\displaystyle=7\\,\\tau\_\{\\text\{Youden\}\}\.By[Equation˜9](https://arxiv.org/html/2608.07528#S4.E9), a lower threshold fires more aggressively\.τYouden\\tau\_\{\\text\{Youden\}\}is the default in all intervention experiments; the sensitivity ablation \([Section˜4\.9](https://arxiv.org/html/2608.07528#S4.SS9)\) sweeps all three\.

### 4\.7Detection versus failure prediction

We compute two AUROC scores per signalss:

AUROCdetect​\(s\)\\displaystyle\\text\{AUROC\}\_\{\\text\{detect\}\}\(s\)=Pr⁡\[s​\(𝐱\(1\)\)\>s​\(𝐱\(0\)\)\],\\displaystyle=\\Pr\\bigl\[s\(\\mathbf\{x\}^\{\(1\)\}\)\>s\(\\mathbf\{x\}^\{\(0\)\}\)\\bigr\],\(6\)AUROCfail​\(s\)\\displaystyle\\text\{AUROC\}\_\{\\text\{fail\}\}\(s\)=Pr\[s\(𝐱degraded\(1\)\)\\displaystyle=\\Pr\\bigl\[s\(\\mathbf\{x\}^\{\(1\)\}\_\{\\text\{degraded\}\}\)\>s\(𝐱recovered\(1\)\)\]\.\\displaystyle\\qquad\>s\(\\mathbf\{x\}^\{\(1\)\}\_\{\\text\{recovered\}\}\)\\bigr\]\.\(7\)[Equation˜6](https://arxiv.org/html/2608.07528#S4.E6)tests*detection*across all records;[Equation˜7](https://arxiv.org/html/2608.07528#S4.E7), over the error subset only, tests*failure prediction*, using thefinal\_degradedtarget \(wrong under error, correct under matched clean\) to isolate propagation from baseline difficulty\. The two need not correlate: a signal can detect perfectly yet be uninformative about failure if the error representation saturates regardless of downstream recovery\. This is the methodological core of the paper\. Prior calibration work asks whether verbalized confidence correlates with accuracy; we ask the prior question, whether the*internal*detection signal also predicts propagation, which determines whether probe\-based monitoring is actionable\.

For verbalized confidence, binary collapse is:

collapse rate=1Nverb∑i=1Nverb𝟏\[\\displaystyle\\text\{collapse rate\}=\\frac\{1\}\{N\_\{\\text\{verb\}\}\}\\sum\_\{i=1\}^\{N\_\{\\text\{verb\}\}\}\\mathbf\{1\}\\bigl\[ci≤0\.05\\displaystyle c\_\{i\}\\leq 05\(8\)orci≥0\.95\],\\displaystyle\\;\\text\{or\}\\;c\_\{i\}\\geq 95\\bigr\],withci=vi/10c\_\{i\}=v\_\{i\}/10; wrong rates of the two extreme groups are then compared to test whether the split carries any correctness signal\.

### 4\.8Intervention experiment

The intervention experiment tests whether the probe can serve as a runtime monitor that improves final\-answer accuracy\. The probe fires on traceiiat hopkkwhen its output exceeds the Youden threshold:

fire​\(i,k\)=𝟏​\[σ​\(𝐰⊤​𝐚k\(i\)−𝝁𝝈\+b\)≥τYouden\]\.\\text\{fire\}\(i,k\)=\\mathbf\{1\}\\\!\\left\[\\sigma\\\!\\left\(\\mathbf\{w\}^\{\\top\}\\frac\{\\mathbf\{a\}\_\{k\}^\{\(i\)\}\-\\bm\{\\mu\}\}\{\\bm\{\\sigma\}\}\+b\\right\)\\geq\\tau\_\{\\text\{Youden\}\}\\right\]\.\(9\)On firing, one of three strategies is applied, subject to a cap ofM=3M=3per trace to bound inference cost\.Reprompt \(C1\)augments the context with a fixed recheck instruction before generation; it is the cheapest policy, adding no forward passes, and tests whether a verbal alert alone changes behaviour\.Replace\-prior \(C2\)regenerates hopk−1k\-1from scratch,r~k−1=fθ​\(q,r0,…,rk−2\)\\tilde\{r\}\_\{k\-1\}=f\_\{\\theta\}\(q,r\_\{0\},\\ldots,r\_\{k\-2\}\), substitutes it for the corrupted value in the context, and continues generation from the corrected prefix\(q,r0,…,rk−2,r~k−1\)\(q,r\_\{0\},\\ldots,r\_\{k\-2\},\\tilde\{r\}\_\{k\-1\}\), directly removing the corruption at the cost of one extra forward pass per firing\.Branch\-and\-pick \(C3\)samplesKKcandidate hop\-kkresponsesr~k\(1\),…,r~k\(K\)\\tilde\{r\}\_\{k\}^\{\(1\)\},\\ldots,\\tilde\{r\}\_\{k\}^\{\(K\)\}at temperatures𝒯=\{0\.7,0\.85,1\.0,1\.15\}\\mathcal\{T\}=\\\{0\.7,0\.85,1\.0,1\.15\\\}from the committed prefixr0:k−1∗r^\{\*\}\_\{0:k\-1\}, and selects the one inducing the least error\-like internal state,j∗=arg⁡minj∈\[K\]⁡y^​\(𝐚​\(q,r0:k−1∗,r~k\(j\)\)\)j^\{\*\}=\\arg\\min\_\{j\\in\[K\]\}\\hat\{y\}\\bigl\(\\mathbf\{a\}\(q,r^\{\*\}\_\{0:k\-1\},\\tilde\{r\}\_\{k\}^\{\(j\)\}\)\\bigr\), using the probe to pick the continuation least likely to propagate the error, at a cost ofKKextra forward passes per firing\.

### 4\.9Sensitivity ablation

We evaluate branch\-and\-pick across the full factorial of branch count and threshold mode,K∈\{1,2,4,8\}K\\in\\\{1,2,4,8\\\}crossed withτ∈\{τconservative,τYouden,τloose\}\\tau\\in\\\{\\tau\_\{\\text\{conservative\}\},\\tau\_\{\\text\{Youden\}\},\\tau\_\{\\text\{loose\}\}\\\}\(12 cells per model\), exposing the precision/recall tradeoff in the branch decision: looser thresholds and largerKKrescue more errors at the cost of higher broken counts\. Sampling details are given in[Appendix˜J](https://arxiv.org/html/2608.07528#A10)\.

## 5Results

#### Probe detects corruption but does not predict failure\.

Across all five variants, a linear probe on residual\-stream activations exceedsaurocdetect\>0\.98\\mbox\{\{auroc\}\}\_\{\\text\{detect\}\}\>0\.98at the best layer \([Table˜3](https://arxiv.org/html/2608.07528#S5.T3)\), peaking at0\.9970\.997on Llama\-3\.1\-8B under mean pooling\. All models reportauroc=0\.500\\mbox\{\{auroc\}\}=0\.500atℓ=0\\ell=0under last\-token pooling, so the signal arises from computation rather than token identity; a single layer\-0→\\to1 jump of0\.380\.38–0\.410\.41AUROC contributes the bulk of it\. Peak depth grows with model size \(39% of depth for Llama\-3\.2\-3B, 64% for Qwen3\-4B instruct\)\. Crucially, the same probe is*uninformative about failure*: at its best detection layer its failure\-prediction AUROC \([Equation˜7](https://arxiv.org/html/2608.07528#S4.E7)\) collapses to near chance across all five models, in sharp contrast to detection above0\.980\.98\. This gap betweenaurocdetect\\mbox\{\{auroc\}\}\_\{\\text\{detect\}\}andaurocfail\\mbox\{\{auroc\}\}\_\{\\text\{fail\}\}within a single probe is the core dissociation of the paper; per\-model values are in[Table˜3](https://arxiv.org/html/2608.07528#S5.T3)\.

Table 3:Best\-layer probe detection AUROC \(last/mean pooling\), probe*failure\-prediction*AUROC at the same layer, and the best surface signal\. Detection exceeds0\.980\.98everywhere whileaurocfail\\mbox\{\{auroc\}\}\_\{\\text\{fail\}\}is at or near chance, and the detection gap to the best surface signal is at least0\.300\.30\.
#### Surface signals are at or near chance\.

None of the surface signals shown exceedsauroc=0\.70\\mbox\{\{auroc\}\}=0\.70for detection \([Table˜6](https://arxiv.org/html/2608.07528#A6.T6),[Appendix˜F](https://arxiv.org/html/2608.07528#A6)\); peak entropy is exactly0\.5000\.500for the instruct models shown, and the mean probe\-to\-surface gap is0\.410\.41\. The Qwen3\-4B thinking model reverses below chance \(0\.3080\.308\), with error traces*less*entropic than clean ones, consistent with thinking\-mode suppression of surface uncertainty \([Section˜5](https://arxiv.org/html/2608.07528#S5.SS0.SSS0.Px4)\)\.

#### Behavioral silence\.

Despite near\-perfect internal detection, all four instruct models produce zero hedging and zero overconfident outputs across 258 traces each, and never use a structured confidence format unprompted; thinking models hedge only 1–2% of the time\. Clean\-condition accuracy is 23–31% and degrades 4–8 points under injection\. Theerror\_verbalizedcondition does not improve accuracy overerror\_standardin any model\.

#### Thinking mode degrades accuracy without changing encoding\.

Enabling chain\-of\-thought on Qwen3\-4B barely moves probe AUROC \(Δ=−0\.001\\Delta=\-0\.001last,−0\.004\-0\.004mean\) but degrades error\-condition accuracy from 27\.9% to 1\.2% \([Table˜7](https://arxiv.org/html/2608.07528#A6.T7),[Appendix˜F](https://arxiv.org/html/2608.07528#A6)\), despite∼\\sim5×\\timesmore tokens\. The variants share weights, differing only in the inference\-time template, so the encoding persists while the output channel is disrupted\. Clean accuracy also drops \(29\.1% to 12\.8%\), so this best evidences*persistence*of the encoding under degradation rather than an isolated manipulation of the saying channel\.

#### Interventions: branch\-and\-pick is closest to asymmetric\.

Across three models with complete data, branch\-and\-pick is net\-positive everywhere and uniquely non\-breaking on Llama\-3\.1\-8B \(4 rescued, 0 broken\), breaking one trace on Qwen2\.5\-3B; replace\-prior breaks correct traces at nearly the rate it rescues wrong ones, and reprompt does neither\. The probe fires on 96–100% of error traces, yet replace\-prior’s firing drops to 32–39% with no correctness gain, so the encoding is not the proximal cause of wrong answers\. Per\-type effects are strongly heterogeneous – replace\-prior best onwrong\_unit\(\+37\.5%\+37\.5\\%\), net\-negative onwrong\_percentage\_base\(−18\.2%\-18\.2\\%\) – motivating error\-type\-aware routing, exploratory given 9–16 examples per slice \([Table˜8](https://arxiv.org/html/2608.07528#A7.T8),[Appendix˜G](https://arxiv.org/html/2608.07528#A7)\)\.

## 6Evaluation metrics

We evaluate three tasks, each with its own metric\.

#### Detection and failure prediction\.

For the probe and all surface signals we report AUROC \([Equation˜4](https://arxiv.org/html/2608.07528#S4.E4)\), which is threshold\-free, with0\.50\.5denoting chance and values below0\.50\.5indicating an anti\-predictive \(inverted\) signal\. It is computed for two targets:

ydetect\\displaystyle y\_\{\\text\{detect\}\}=𝟏​\[error injected\],\\displaystyle=\\mathbf\{1\}\[\\text\{error injected\}\],\(10\)yfail\(i\)\\displaystyle y^\{\(i\)\}\_\{\\text\{fail\}\}=𝟏​\[a^err\(i\)≠ai∗∧a^clean\(i\)=ai∗\],\\displaystyle=\\mathbf\{1\}\\bigl\[\\hat\{a\}^\{\(i\)\}\_\{\\text\{err\}\}\\neq a^\{\*\}\_\{i\}\\;\\wedge\\;\\hat\{a\}^\{\(i\)\}\_\{\\text\{clean\}\}=a^\{\*\}\_\{i\}\\bigr\],over all traces \([Equation˜6](https://arxiv.org/html/2608.07528#S4.E6)\) and the error subset \([Equation˜7](https://arxiv.org/html/2608.07528#S4.E7)\) respectively, wherea^err\(i\)\\hat\{a\}^\{\(i\)\}\_\{\\text\{err\}\}anda^clean\(i\)\\hat\{a\}^\{\(i\)\}\_\{\\text\{clean\}\}are the model’s final answers on traceiiunder the error and matched\-clean contexts andai∗a^\{\*\}\_\{i\}is ground truth\. Thisfinal\_degradedtarget separates error propagation from baseline difficulty\. The two are decoupled by design:AUROCdetect≈1\\text\{AUROC\}\_\{\\text\{detect\}\}\\\!\\approx\\\!1withAUROCfail≈0\.5\\text\{AUROC\}\_\{\\text\{fail\}\}\\\!\\approx\\\!0\.5is the central measurement target\.

#### Verbalization quality\.

Withci=vi/10c\_\{i\}=v\_\{i\}/10the normalised confidence and collapse rate as in[Equation˜8](https://arxiv.org/html/2608.07528#S4.E8), we compare the wrong rate of the two collapsed groupsg∈\{\[0,0\.05\],\[0\.95,1\]\}g\\in\\\{\[0,0\.05\],\[0\.95,1\]\\\}:

wrong​\(g\)\\displaystyle\\text\{wrong\}\(g\)=1\|Gg\|​∑i∈Gg𝟏​\[a^i≠ai∗\],\\displaystyle=\\frac\{1\}\{\|G\_\{g\}\|\}\\sum\_\{i\\in G\_\{g\}\}\\mathbf\{1\}\[\\hat\{a\}\_\{i\}\\neq a\_\{i\}^\{\*\}\],\(11\)Gg\\displaystyle G\_\{g\}=\{i:ci∈g\}\.\\displaystyle=\\\{i:c\_\{i\}\\in g\\\}\.Indistinguishablewrong​\(\[0,0\.05\]\)\\text\{wrong\}\(\[0,0\.05\]\)andwrong​\(\[0\.95,1\]\)\\text\{wrong\}\(\[0\.95,1\]\)confirm the binary collapse carries no calibration signal\.

#### Intervention effectiveness\.

For each policyc∈\{C1,C2,C3\}c\\in\\\{\\text\{C1\},\\text\{C2\},\\text\{C3\}\\\}, relative to baseline:

rescuedc\\displaystyle\\text\{rescued\}\_\{c\}=\|\{i:wrongbase\(i\)∧correctc\(i\)\}\|,\\displaystyle=\|\\\{i:\\text\{wrong\}^\{\(i\)\}\_\{\\text\{base\}\}\\wedge\\text\{correct\}^\{\(i\)\}\_\{c\}\\\}\|,\(12\)brokenc\\displaystyle\\text\{broken\}\_\{c\}=\|\{i:correctbase\(i\)∧wrongc\(i\)\}\|,\\displaystyle=\|\\\{i:\\text\{correct\}^\{\(i\)\}\_\{\\text\{base\}\}\\wedge\\text\{wrong\}^\{\(i\)\}\_\{c\}\\\}\|,netc\\displaystyle\\text\{net\}\_\{c\}=rescuedc−brokenc,\\displaystyle=\\text\{rescued\}\_\{c\}\-\\text\{broken\}\_\{c\},Δ​accc\\displaystyle\\Delta\\text\{acc\}\_\{c\}=netcNintervene×100\.\\displaystyle=\\tfrac\{\\text\{net\}\_\{c\}\}\{N\_\{\\text\{intervene\}\}\}\\times 00\.Rescued and broken are reported separately, not collapsed into net, since equal net can hide very different risk profiles; an asymmetric policy \(rescued≫broken\\text\{rescued\}\\gg\\text\{broken\}\) is the deployment target\. Answers match ground truth within10−310^\{\-3\}absolute or1%1\\%relative tolerance\.

## 7Discussion

#### Robustness to leakage and surface artifacts\.

Detection is not an artifact of fold leakage or per\-type surface cues\. Under grouped cross\-validation holding out entire base problems, best\-layer detection AUROC moves by at most0\.0030\.003from the stratified estimate \([Appendix˜C](https://arxiv.org/html/2608.07528#A3)\)\. Under leave\-one\-error\-type\-out training, detection on the unseen type stays above0\.910\.91in every model\-by\-type cell \(per\-model means0\.9680\.968–0\.9960\.996;[Appendix˜D](https://arxiv.org/html/2608.07528#A4)\)\.off\_by\_one, least separable on surface features yet still well above chance, is the strongest evidence that the probe encodes an abstract corruption state rather than a per\-type token signature\.

#### Decodability, not causation\.

We claim decodability, not causation: the corruption signal is linearly present in the residual stream, but this does not establish that the generation policy consults that direction\. Our interventions \([Section˜5](https://arxiv.org/html/2608.07528#S5.SS0.SSS0.Px5)\) bear on this – replace\-prior removes the corrupted value and sharply lowers probe firing \(96–100% to 32–39%\) without a proportional accuracy gain, evidence that the probed direction is not necessarily the proximal cause of the wrong answer\. Establishing causal relevance would require representation\-level interventions \(activation patching, mediation analysis\), which we leave to future work\.

#### Toward routed interventions\.

The per\-error\-type results \([Section˜5](https://arxiv.org/html/2608.07528#S5.SS0.SSS0.Px5),[Appendix˜G](https://arxiv.org/html/2608.07528#A7)\) show that no single intervention dominates\. Replace\-prior is strongest onwrong\_unitbut net\-negative onwrong\_percentage\_base, while branch\-and\-pick is the only strictly net\-positive policy onoff\_by\_one\. This motivates a routed policy that selects the intervention from a predicted error type\. We stress that such routing presupposes an error\-type classifier, which we do not yet have, and that the supporting per\-type sample sizes are small, so we frame routing as a direction rather than a deployable result\.

#### Future work\.

Three directions look most promising\. Per\-hop localisation would move the probe target from “a prior is corrupted” to “hopkkis corrupted,” enabling much more targeted replace\-prior\. Error\-type probes would classify the corruption itself, unlocking the routed policy without requiring an oracle\. And mechanistic dissection of the layer 0 to layer 1 jump would clarify whether the signal arises from attention to inconsistent tokens or from an MLP feature\. To our knowledge this is an unusually clean single\-layer information gain and worth understanding\.

## 8Conclusion

We characterise a four\-part dissociation in five models reasoning under silently corrupted context\. A linear probe detects injected errors near\-perfectly \(auroc\>0\.98\\mbox\{\{auroc\}\}\>0\.98, up to0\.9970\.997on Llama\-3\.1\-8B\) yet does not predict whether the error reaches the final answer; verbalised confidence collapses to a binary filter with indistinguishable wrong rates; probe persistence across hops does not separate outcomes, refuting our pre\-registered persistence\-beats\-peak hypothesis; and chain\-of\-thought widens rather than closes the gap, leaving probeaurocunchanged while error\-condition accuracy falls to 1\.2%\. No intervention dominates: branch\-and\-pick is net\-positive everywhere and uniquely non\-breaking on Llama\-3\.1\-8B, but the asymmetry is conditional on error type and model \([Sections˜5](https://arxiv.org/html/2608.07528#S5.SS0.SSS0.Px5)and[G](https://arxiv.org/html/2608.07528#A7)\)\. Probe\-based monitoring is thus a necessary complement to verbalised confidence rather than a replacement, and the deployable answer is model\-aware, error\-type\-aware routing\.

## Ethics Statement

This paper presents a probe\-based mechanism for detecting and intervening on errors during multi\-step reasoning by language models\. The intended use is trustworthy deployment, namely catching errors before commitment to a wrong answer\. Three considerations apply\. First, the probe operates on internal activations and cannot be used by parties without access to them, so it is not a black\-box method\. Second, the interventions are not free of failure modes\. Replace\-prior breaks previously\-correct traces at nearly the rate it rescues wrong ones, and branch\-and\-pick breaks at a small but non\-zero rate, so probe\-triggered intervention should not be presented to end users as strictly safe\. Third, branch\-and\-pick increases inference cost roughly tenfold at the intervention frequencies measured here, with energy and latency implications worth weighing for any specific application\.

## References

- Anthropic \(2024a\)Anthropic\. 2024a\.The Claude 3 model family: Opus, Sonnet, Haiku\.[https://www\-cdn\.anthropic\.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model\_Card\_Claude\_3\.pdf](https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf)\.
- Anthropic \(2024b\)Anthropic\. 2024b\.Claude Code: Agentic coding\.[https://www\.anthropic\.com/product/claude\-code](https://www.anthropic.com/product/claude-code)\.
- Arditi et al\. \(2024\)Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda\. 2024\.Refusal in language models is mediated by a single direction\.In*Advances in Neural Information Processing Systems 37 \(NeurIPS 2024\)*\.
- Belinkov \(2022\)Yonatan Belinkov\. 2022\.Probing classifiers: Promises, shortcomings, and advances\.*Computational Linguistics*, 48\(1\):207–219\.
- Burns et al\. \(2023\)Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt\. 2023\.Discovering latent knowledge in language models without supervision\.In*The Eleventh International Conference on Learning Representations \(ICLR\)*\.
- Chan et al\. \(2024\)Chi\-Min Chan, Weize Chen, Yusheng Su, Jianxin Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu\. 2024\.ChatEval: Towards better LLM\-based evaluators through multi\-agent debate\.In*International Conference on Learning Representations*\.
- Cognition AI \(2024\)Cognition AI\. 2024\.Introducing devin, the first AI software engineer\.[https://cognition\.ai/blog/introducing\-devin](https://cognition.ai/blog/introducing-devin)\.
- Gurnee et al\. \(2023\)Wes Gurnee, Neel Nanda, Matthew Pauly, Kyle Harvey, Dmitrii Troiano, and Jacob Steinhardt\. 2023\.Finding neurons in a haystack: Case studies with sparse probing\.*Transactions on Machine Learning Research*\.
- Hewitt and Manning \(2019\)John Hewitt and Christopher D\. Manning\. 2019\.A structural probe for finding syntax in word representations\.In*Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\)*, pages 4129–4138\.
- Huang et al\. \(2023\)Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhengwei Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al\. 2023\.A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions\.*arXiv preprint arXiv:2311\.05232*\.
- Jimenez et al\. \(2024\)Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan\. 2024\.SWE\-bench: Can language models resolve real\-world GitHub issues?In*International Conference on Learning Representations*\.
- Kadavath et al\. \(2022\)Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Deep Ganguli, Azalia Mirhoseini, Andy Jones, Anna Chen, LightmanHarrison, et al\. 2022\.Language models \(mostly\) know what they know\.*arXiv preprint arXiv:2207\.05221*\.
- Lanham et al\. \(2023\)Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al\. 2023\.Measuring faithfulness in chain\-of\-thought reasoning\.*arXiv preprint arXiv:2307\.13702*\.
- Lightman et al\. \(2024\)Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe\. 2024\.Let’s verify step by step\.In*The Twelfth International Conference on Learning Representations \(ICLR\)*\.
- Lin et al\. \(2022\)Stephanie Lin, Jacob Hilton, and Owain Evans\. 2022\.Teaching models to express their uncertainty in words\.*Transactions on Machine Learning Research*\.
- Marks and Tegmark \(2023\)Samuel Marks and Max Tegmark\. 2023\.The geometry of truth: Emergent linear structure in large language model representations of true/false datasets\.*arXiv preprint arXiv:2310\.06824*\.
- OpenAI \(2023\)OpenAI\. 2023\.GPT\-4 technical report\.[https://arxiv\.org/abs/2303\.08774](https://arxiv.org/abs/2303.08774)\.
- Pan et al\. \(2023\)Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang\. 2023\.Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies\.*Transactions of the Association for Computational Linguistics*, 11:1409–1429\.
- Paul et al\. \(2024\)Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings\. 2024\.Making reasoning matter: Measuring and improving faithfulness of chain\-of\-thought reasoning\.In*Findings of the Association for Computational Linguistics: EMNLP 2024*\.
- Schick et al\. \(2023\)Timo Schick, Jane Dwivedi\-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom\. 2023\.Toolformer: Language models can teach themselves to use tools\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Shinn et al\. \(2023\)Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao\. 2023\.Reflexion: Language agents with verbal reinforcement learning\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Singhal et al\. \(2023\)Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole\-Lewis, Stephen Pfohl, et al\. 2023\.Large language models encode clinical knowledge\.In*Nature*, volume 620, pages 172–180\.
- Tian et al\. \(2023\)Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D\. Manning\. 2023\.Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine\-tuned with human feedback\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*\.
- Turpin et al\. \(2023\)Miles Turpin, Julian Michael, Ethan Perez, and Samuel R\. Bowman\. 2023\.Language models don’t always say what they think: Unfaithful explanations in chain\-of\-thought prompting\.In*Advances in Neural Information Processing Systems 36 \(NeurIPS 2023\)*\.
- Valmeekam et al\. \(2023\)Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, and Subbarao Kambhampati\. 2023\.Large language models still can’t plan\.In*NeurIPS 2023 Foundation Models for Decision Making Workshop*\.
- Wang et al\. \(2024\)Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al\. 2024\.A survey on large language model based autonomous agents\.*Frontiers of Computer Science*, 18\(6\):186345\.
- Wang et al\. \(2023\)Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V\. Le, Ed H\. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou\. 2023\.Self\-consistency improves chain of thought reasoning in language models\.In*The Eleventh International Conference on Learning Representations \(ICLR\)*\.
- Wei et al\. \(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou\. 2022\.Chain\-of\-thought prompting elicits reasoning in large language models\.In*Advances in Neural Information Processing Systems*, volume 35\.
- Weng et al\. \(2023\)Yixuan Weng, Minjun Zhu, Fangyu Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao\. 2023\.Large language models are better reasoners with self\-verification\.*arXiv preprint arXiv:2212\.09561*\.
- Xiong et al\. \(2024\)Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi\. 2024\.Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs\.*arXiv preprint arXiv:2306\.13063*\.
- Yang et al\. \(2024\)John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press\. 2024\.SWE\-agent: Agent\-computer interfaces enable automated software engineering\.In*Advances in Neural Information Processing Systems*, volume 37\.
- Yao et al\. \(2023\)Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao\. 2023\.ReAct: Synergizing reasoning and acting in language models\.In*International Conference on Learning Representations*\.

## Appendix: Supplementary Methodology and Results

## Appendix ADataset generation detail

#### Generation procedure\.

Base problems are produced by 12 subcategory\-specific template functions covering arithmetic, percentages, rates, fractions, counting, geometry, compound interest, and nested fractions across 2 to 4 hop depths\. Each function draws numeric parameters \(prices, rates, counts, percentages\) from fixed candidate sets using a seededrandom\.Randominstance, instantiates a natural\-language question, and computes each intermediate result exactly\. Trace identities are 8\-byte BLAKE2b hashes of\(base\_problem\_id, variant\), making each trace uniquely and reproducibly addressable, and all hops downstream of an injection carrypropagates\_prior\_error: true\.

#### Error\-type motivation\.

The five error types are chosen to span distinct failure modes seen in real multi\-step reasoning:

- •off\_by\_one\(correct value\+1\+1\): small transcription or rounding errors that leave the answer superficially plausible\. The hardest case for surface signals, since the corrupted value is coherent and close to correct\.
- •wrong\_operator\(e\.g\.×⁣→⁣\+\\times\\to\+\): a conceptual mistake that combines the right quantities incorrectly\. The result can be wildly off, testing whether probes respond to corruption magnitude rather than mere presence\.
- •wrong\_unit\(e\.g\. hours instead of minutes, factor of 60\): unit\-tracking failures common in scientific and engineering chains, where the computation is correct but the semantic interpretation is wrong\.
- •magnitude\_error\(correct value×10\\times 10\): a decimal\-placement or order\-of\-magnitude mistake, structurally similar to the correct value but numerically large, testing sensitivity to scale\.
- •wrong\_percentage\_base\(100−p100\-pinstead ofpp\): a systematic conceptual error confusing what is kept with what is removed\. Common in financial and allocation chains, producing plausible\-looking but wrong downstream values\.

## Appendix BSurface\-signal definitions and behavioral flags

#### Signal interpretation\.

The six signals in[Equation˜2](https://arxiv.org/html/2608.07528#S4.E2)test three distinct hypotheses about where uncertainty might appear in the output distribution\. Entropy\-based signals \(speak\-ents\_\{\\text\{peak\-ent\}\},smean\-ents\_\{\\text\{mean\-ent\}\},searly\-ents\_\{\\text\{early\-ent\}\}\) measure distributional spread across the full vocabulary; confidence signals \(smin\-confs\_\{\\text\{min\-conf\}\}\) measure top\-token certainty; andsvocab\-gaps\_\{\\text\{vocab\-gap\}\}measures the margin between the top two candidates, which can remain large even when entropy is moderate\. Hereyty\_\{t\}denotes the token actually generated at positiontt, soslogprobs\_\{\\text\{logprob\}\}is a valid sequence log\-probability under sampling; under greedy decoding it reduces to the mean negative log\-probability of the top token,−1G​∑tlog⁡pt\(1\)\-\\tfrac\{1\}\{G\}\\sum\_\{t\}\\log p\_\{t\}^\{\(1\)\}\.

#### Behavioral flags\.

In addition to the scalar signals, four flags are extracted from the generated text: presence of hedging language, presence of overconfident language, a structured verbalized confidence score \(where elicited\), and an absurdity flag for outputs containing logical impossibilities\. These flags measure whether the model’s verbalized behavior reflects the internal state captured by the probe\.

## Appendix CGrouped cross\-validation

To rule out leakage from shared base problems and templates, we re\-run probe evaluation under group\-stratified55\-fold cross\-validation in which all variants of a given base problem are confined to a single fold, so no template instance appears in both train and test\.[Table˜4](https://arxiv.org/html/2608.07528#A3.T4)compares stratified and grouped detection AUROC at the best layer and aggregation mode for each model\. Across all five models the two protocols agree to within0\.0030\.003AUROC, and on three of the five the grouped estimate is marginally higher, so the near\-perfect detection reported in[Table˜3](https://arxiv.org/html/2608.07528#S5.T3)reflects a genuinely decodable corruption state rather than memorised base\-problem or template structure\.

Table 4:Best\-layer detection AUROC under stratified versus base\-problem\-grouped cross\-validation\. Both columns are computed at the layer and aggregation mode listed, which is the best stratified configuration for that model in this run\. Grouped AUROC is reported as mean±\\pmstd across the grouped folds\.
## Appendix DHeld\-out error\-type generalisation

To test whether the probe encodes an abstract corruption state rather than surface signatures of individual error types, we train on four error types and evaluate detection on the fifth, for all five leave\-one\-out splits\. Clean traces are pooled across splits; only the error\-type composition of the training set varies\.[Table˜5](https://arxiv.org/html/2608.07528#A4.T5)reports detection AUROC on each held\-out type at the best layer for all five models\. Every held\-out type transfers above0\.910\.91, includingoff\_by\_one, the hardest case on surface grounds, andwrong\_unitandmagnitude\_errortransfer perfectly\. A probe that never saw a given error type during training still flags it at test time, so the probed direction is not a per\-type token detector\.

Table 5:Leave\-one\-error\-type\-out detection AUROC at the best layer\. Columns are models; each row trains on the other four error types and tests on the named held\-out type\. Leading zeros omitted;wrong\_pct\_baseabbreviateswrong\_percentage\_base\.
## Appendix EThreshold calibration detail

The three thresholds in[Equation˜5](https://arxiv.org/html/2608.07528#S4.E5)are calibrated on out\-of\-fold scores to avoid threshold overfitting while preserving all data for probe training\. HereFy=0−1​\(0\.99\)F^\{\-1\}\_\{y=0\}\(0\.99\)is the 99th percentile of out\-of\-fold scores for clean samples, givingτconservative\\tau\_\{\\text\{conservative\}\}a very low false\-positive rate at the cost of reduced recall, whileτloose=0\.7​τYouden\\tau\_\{\\text\{loose\}\}=0\.7\\,\\tau\_\{\\text\{Youden\}\}fires more aggressively, rescuing more errors at higher broken counts\. Onlyτloose<τYouden\\tau\_\{\\text\{loose\}\}<\\tau\_\{\\text\{Youden\}\}holds by construction; whetherτconservative\\tau\_\{\\text\{conservative\}\}sits above or belowτYouden\\tau\_\{\\text\{Youden\}\}depends on the clean and corrupted score distributions, and on the highly separable runs here the two are nearly equal \(see[Figure˜2](https://arxiv.org/html/2608.07528#A9.F2)\)\. The firing rule of[Equation˜9](https://arxiv.org/html/2608.07528#S4.E9)\(fire when the probe score is at leastτ\\tau\) then determines the firing\-rate ordering used in the sensitivity ablation \([Section˜4\.9](https://arxiv.org/html/2608.07528#S4.SS9)\)\.

## Appendix FSurface\-signal and thinking\-mode detail

#### Surface signals\.

[Table˜6](https://arxiv.org/html/2608.07528#A6.T6)reports detection and failure\-prediction AUROC for the token\-distribution signals\. All are at or near chance for detection: peak entropy is exactly0\.5000\.500for the instruct models shown, logprob uncertainty tops out at0\.6220\.622\(Llama\-3\.1\-8B\) and falls below0\.5000\.500for some Qwen variants, and none of the signals shown exceeds0\.700\.70\. Failure prediction is similarly uninformative, with several entries below chance \(for example logprob uncertainty at0\.4110\.411–0\.4170\.417on the Llama models\), indicating these signals are not merely uncorrelated with propagation but mildly anti\-predictive\. The Qwen3\-4B thinking model is the clearest reversal: peak entropy drops to0\.3080\.308for detection, so tokens in error traces are on average*less*entropic than in clean traces, consistent with thinking\-mode suppression of surface uncertainty\.

Table 6:Representative surface\-signal AUROC for detection and failure prediction \(three signals shown per model\)\. All are at or near chance for detection, and failure prediction is similarly uninformative\.
#### Thinking\-mode ablation\.

[Table˜7](https://arxiv.org/html/2608.07528#A6.T7)contrasts the Qwen3\-4B instruct and thinking variants\. Probe AUROC is essentially unchanged across the two modes \(within0\.0040\.004\), while final\-answer accuracy degrades under thinking, from 29\.1% to 12\.8% on clean problems and from 27\.9% to 1\.2% under error injection, despite roughly five times more generated tokens\. Because the variants share weights and differ only in the inference\-time template, this is evidence that the internal error encoding persists even as the behavioral output channel degrades\.

Table 7:Thinking\-mode ablation on Qwen3\-4B\. Probe AUROC is essentially unchanged while final\-answer accuracy degrades\.

## Appendix GPer\-error\-type intervention effects

[Table˜8](https://arxiv.org/html/2608.07528#A7.T8)provides the per\-error\-type breakdown referenced in[Section˜5](https://arxiv.org/html/2608.07528#S5.SS0.SSS0.Px5)\. Each cell reports the change in accuracy \(in percentage points\) relative to the baseline policy, together with the corresponding rescued and broken counts in parentheses\. Bold entries indicate the best\-performing policy for each error type\.

For readability, several shortened names used in the main text correspond to the implementation labels as follows:wrong\_opcorresponds towrong\_operator;magnitudecorresponds tomagnitude\_error; andwrong\_pct\_basecorresponds towrong\_percentage\_base\.

Table 8:Per\-error\-type intervention effects on the Qwen2\.5\-3B intervention set \(65 traces across the five error types\)\. Herenndenotes the number of examples for each error type\.
## Appendix HPooling ablation

In[Section˜4](https://arxiv.org/html/2608.07528#S4), we report results using last\-token activation pooling\. For completeness, we additionally trained probes using mean\-pooled activations across the full prompt\. The peak performance is comparable between the two approaches: on Qwen2\.5\-3B mean pooling achieves a maximumaurocof 0\.986 at layer 27, while last\-token pooling achieves 0\.991 at the same layer\. The primary qualitative difference appears in the earliest layers: under mean pooling the layer\-0 probe already achievesauroc≈0\.91\\mbox\{\{auroc\}\}\\approx 0\.91, rather than remaining near chance, because mean pooling aggregates information from tokens appearing after the corruption has already been processed somewhere within the network; the per\-token trajectories in[Appendix˜I](https://arxiv.org/html/2608.07528#A9)make this accumulation explicit\.

Both aggregation modes are trained at every layer and the stronger is selected per layer \([Section˜4\.6](https://arxiv.org/html/2608.07528#S4.SS6)\); the headline figures in[Table˜3](https://arxiv.org/html/2608.07528#S5.T3)use whichever mode is stronger per model \(mean pooling for Llama\-3\.1\-8B, last\-token elsewhere\)\. Last\-token pooling preserves a cleaner interpretation of information flow, since it measures what the model has computed at a specific point in the sequence, so we adopt it as the default for the information\-flow narrative\. One particularly striking feature of the last\-token profile is the sharp increase from layer 0 to layer 1, where a single transformer block contributes approximately 0\.39 AUROC\. This behavior is obscured under mean pooling, since the layer 0 representation is already highly informative due to prompt\-level aggregation\. Whether this jump is driven by a specific attention head detecting token inconsistencies or by an MLP\-based feature remains an open mechanistic question\. We identify this as a promising direction for future investigation\.

## Appendix IToken\-position probe trajectories

To make the cumulative\-mean mechanism concrete,[Figure˜2](https://arxiv.org/html/2608.07528#A9.F2)traces the probe score*position by position*as the running mean of hidden states grows during generation\. Formally, at generated positionttthe probe is applied to the prefix mean𝐚¯t\(ℓ\)=1t​∑s=1t𝐡s\(ℓ\)\\bar\{\\mathbf\{a\}\}^\{\(\\ell\)\}\_\{t\}=\\tfrac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\mathbf\{h\}^\{\(\\ell\)\}\_\{s\}, so the value at the final position equals the single mean\-pooled score analysed in[Appendix˜H](https://arxiv.org/html/2608.07528#A8)\. The figure therefore decomposes that scalar into the trajectory that produces it\. ThreeCLEANtraces \(top\) and threeERRORtraces \(bottom\) are shown for Qwen at the best mean\-pooling layer, with the Youden and conservative firing thresholds \(τYouden=0\.595\\tau\_\{\\text\{Youden\}\}=0\.595,τcons=0\.584\\tau\_\{\\text\{cons\}\}=0\.584\) marked\. The two thresholds nearly coincide here, an instance of the empirical near\-equality noted in[Appendix˜E](https://arxiv.org/html/2608.07528#A5)rather than a guaranteed ordering\.

Two patterns are visible\. First, clean and corrupted traces separate at the level that matters for pooling: although a clean trajectory may briefly excurse above threshold at an early position, its*persistent*cumulative\-mean score settles below both thresholds, whereas error trajectories stay above once the corrupted operand enters the running mean, reproducing at the single\-trace level the near\-perfect separability reported in aggregate \([Table˜3](https://arxiv.org/html/2608.07528#S5.T3)\)\. Second, and more informative for the pooling comparison, the error trajectories cross the threshold*once the corrupted operand enters the running mean*and remain there for the rest of the sequence, rather than spiking only at the final token\. Because every subsequent position averages in the already\-corrupted prefix, the mean\-pooled score is driven above threshold well before the end of generation\. This is the mechanism behind the otherwise surprising result that mean pooling attainsauroc≈0\.91\\mbox\{\{auroc\}\}\\approx 0\.91at layer 0 \([Appendix˜H](https://arxiv.org/html/2608.07528#A8)\): the corruption signal is smeared across positions by the averaging operation rather than localised at the final token, which is precisely the property that last\-token pooling avoids and the reason we adopt last\-token pooling as the primary aggregation in the main text\. The selected error traces span an arithmetic operator substitution \(2hop\_fractions\_0006,3\+33\{\+\}3in place of3×33\{\\times\}3\) and two magnitude errors in percentage chains \(2hop\_percentage\_0003,2hop\_percentage\_0022\), indicating the behaviour is not specific to a single error type\.

![Refer to caption](https://arxiv.org/html/2608.07528v1/x1.png)Figure 2:Per\-token probe\-score trajectories under cumulative\-mean pooling for threeCLEAN\(top\) and threeERROR\(bottom\) Qwen traces\. The horizontal axis is generated token position and the vertical axis is the probe score; dashed lines mark the Youden and conservative thresholds\. Error trajectories move above threshold once the corrupted operand enters the running mean and stay there, whereas clean trajectories settle below after any early excursion, illustrating why mean pooling is informative even at shallow layers \([Appendix˜H](https://arxiv.org/html/2608.07528#A8)\)\.
## Appendix JSensitivity ablation detail

The branch\-and\-pick factorial \([Section˜4\.9](https://arxiv.org/html/2608.07528#S4.SS9)\) crossesK∈\{1,2,4,8\}K\\in\\\{1,2,4,8\\\}with the three threshold modes\. WhenK≠4K\\neq 4, candidates are drawn from the same temperature schedule: the firstKKentries of\(0\.7,0\.85,1\.0,1\.15\)\(0\.7,0\.85,1\.0,1\.15\)forK<4K<4, and each of the four temperatures sampled twice forK=8K=8\. TheKKablation therefore varies branch count while holding the temperature range fixed\. Conservative thresholds fire rarely, reducing the risk of breaking correct traces but missing some errors, whereas loose thresholds fire aggressively, rescuing more errors at higher broken counts\. The interaction withKKreveals whether more candidates compensate for a noisier trigger\.

## Appendix KPersistence versus peak analysis

For each multi\-hop error trace we compute two scalars, namely*peak*\(maximum probe score across hops\) and*persistence*\(fraction of hops exceedingτYouden\\tau\_\{\\text\{Youden\}\}\)\.AUROCfail\\text\{AUROC\}\_\{\\text\{fail\}\}is computed for each scalar, and neither scalar separates degraded from recovered traces in any model, refuting the pre\-registered persistence\-beats\-peak hypothesis\.