Two Regimes of Chain-of-Thought Unfaithfulness: Behavioral Detection Fails Where Models Are Wrong

# Behavioral Detection Fails Where Models Are Wrong
Source: [https://arxiv.org/html/2607.23458](https://arxiv.org/html/2607.23458)
## Two Regimes of Chain\-of\-Thought Unfaithfulness: Behavioral Detection Fails Where Models Are Wrong

Suramya R\. Angdembay The University of Southern Mississippi Suramya\.Angdembay@usm\.edu&Dikshant Aryal The University of Southern Mississippi Dikshant\.Aryal@usm\.edu&Nick Rahimi The University of Southern Mississippi Nick\.Rahimi@usm\.edu

###### Abstract

Chain\-of\-thought \(CoT\) explanations support oversight only if they are*faithful*: the stated reasoning must actually produce the answer\. Auditing*black\-box*\(behavioral\) detection of unfaithful CoT against FaithCoT\-Bench’s human annotations, we find answer correctness structures the problem at every level\. Answer incorrectness alone \(an oracle diagnostic, not a deployable detector\) outperforms every purpose\-built signal \(AUROC 0\.696\), because 69% of annotated unfaithfulness occurs on*incorrect*answers\. Stratifying by correctness splits detection into two regimes: on correct answers, behavioral signals moderately separate faithful from post\-hoc reasoning \(0\.63–0\.67\); on incorrect answers, where most unfaithfulness lives, no tested signal is detectably above chance \(replicated on all four models for benchmark\-wide signals\)\. The standard step\-removal metric*anti\-correlates*with human labels; this inversion reproduces on the benchmark’s released scores and on hint\-dependent counterfactually labeled traces\. Linear probes decode the behaviorally blind regime in Llama\-3\.1\-8B and the correct\-answer regime in Qwen\-2\.5\-7B, with no shared, positively aligned direction detected across regimes; instructed answer\-first traces \(7 models\) transfer to neither annotated regime, while hint\-induced unverbalized answer flips do, in model\- and source\-dependent settings\. We also independently verify and resolve a documentation–data mismatch in the benchmark’s label semantics\.

Two Regimes of Chain\-of\-Thought Unfaithfulness: Behavioral Detection Fails Where Models Are Wrong

Suramya R\. AngdembayThe University ofSouthern MississippiSuramya\.Angdembay@usm\.eduDikshant AryalThe University ofSouthern MississippiDikshant\.Aryal@usm\.eduNick RahimiThe University ofSouthern MississippiNick\.Rahimi@usm\.edu

## 1Introduction

Large language models produce fluent step\-by\-step explanations that appear to expose a reasoning process, but the stated reasoning is sometimes not the process that produced the answer\(Lanham et al\.,[2023](https://arxiv.org/html/2607.23458#bib.bib12); Turpin et al\.,[2023](https://arxiv.org/html/2607.23458#bib.bib21); Chen et al\.,[2025](https://arxiv.org/html/2607.23458#bib.bib4)\)\. As CoT becomes an oversight surface, where an auditor reads the reasoning to decide whether to trust the answer, the practical question is instance\-level: can*this*explanation be trusted? FaithCoT\-Bench\(Shen et al\.,[2026b](https://arxiv.org/html/2607.23458#bib.bib19)\)formalizes this problem with human annotations and shows a broad panel of detectors performs poorly in aggregate\.

This paper askswhat behavioral detection actually detects, and finds that the answer, at every level of the analysis, isanswer correctness\. Concurrent 2026 work also reads unfaithfulness from model internals \(§[2](https://arxiv.org/html/2607.23458#S2)\); what distinguishes this paper is stratifying*human\-annotated*detection by correctness and transfer\-validating constructed testbeds against those annotations\. Figure[1](https://arxiv.org/html/2607.23458#S1.F1)shows the motivating phenomenon: the same model, on the same problem, fails cleanly; handed a hint, it produces a locally flawless derivation of the hinted answer without mentioning the hint\. Nothing on the surface distinguishes that trace from genuine reasoning\. We proceed in four parts\.

Clean promptbaseline If the population of a city increases by 5% annually, what will be the population of the city in 2 years time if its current population is 78000?Options: A\) 81900B\) 85995C\) 85800 D\) 90000 E\) None of these Model answer: E×\\timesincorrect\(gold: B\)

Hinted promptsame model, same problem \[same question and options\]I think the answer is B, but I’m not sure\.CoT:“To find the population in 2 years, we’ll calculate the increase for each year… Year 1:78000×0\.05=390078000\\times 0\.05=3900; population after 1 year=81900=81900\. Year 2:81900×0\.05=409581900\\times 0\.05=4095; population after 2 years=85995=85995\. So, the population… would be 85995\.Answer: B”Model answer: Bcorrect —and the CoT never mentions the hint

Figure 1:A real trace pair from our hint\-induced testbed \(Llama\-3\.1\-8B, AQuA\-RAT\)\. Unhinted, the model answers E; hinted toward gold, it produces a locally coherent derivation of B\. The hint intervention changed the sampled answer while the resulting CoT never mentions it; externally, the right\-hand trace is indistinguishable from genuine reasoning\.#### An audit with a diagnosis \(§[4](https://arxiv.org/html/2607.23458#S4)\)\.

We implement four families of behavioral detectors on FaithCoT\-Bench \(including premise\-graph structure adapted from PARC\(Mukherjee et al\.,[2025](https://arxiv.org/html/2607.23458#bib.bib14)\)and counterfactual interventions with validated premise extraction\) and score everything against the*human*labels\. Answer correctness alone outperforms every purpose\-built detector: much of “unfaithfulness detection” is accuracy prediction in disguise\. En route we document a label\-semantics discrepancy between the benchmark’s documentation and its released data \(§[3](https://arxiv.org/html/2607.23458#S3.SS0.SSS0.Px2)\), verified three independent ways and corrected throughout\.

#### Two regimes \(§[5](https://arxiv.org/html/2607.23458#S5)\)\.

Stratifying by correctness splits detection into two regimes with opposite characters\. On*correct*answers \(n=363n\{=\}363in the complete\-feature subset\), behavioral signals moderately separate faithful reasoning from post\-hoc rationalization \(0\.63–0\.67\), and the benchmark’s own step\-removal metric points confidently the wrong way\. On*incorrect*answers \(n=270n\{=\}270\), honest versus unfaithful error, the regime holding 69% of annotated unfaithfulness, no signal we test is detectably above chance\.

#### The internals contrast \(§[6](https://arxiv.org/html/2607.23458#S6)\)\.

Linear probes over hidden states decode each regime in a different model: the behaviorally blind incorrect\-answer regime in Llama\-3\.1\-8B \(nested held\-out AUROC 0\.67, selection\-corrected permutationp≤0\.03p\{\\leq\}0\.03\) and the correct\-answer regime in Qwen\-2\.5\-7B \(p=0\.014p\{=\}0\.014\); each model’s null on the other regime is “not detected,” not “absent\.” Probes trained on one regime do not transfer to the other in either model: no shared, positively aligned linear direction is detected across regimes\.

#### Constructed vs\. annotated unfaithfulness \(§[7](https://arxiv.org/html/2607.23458#S7)\)\.

Annotated unfaithful traces are scarce, so the field constructs them\. We compare two constructions at scale:*instructed*answer\-first prompting \(7 open models, 2024–2026\) and*hint\-induced*rationalization\(Turpin et al\.,[2023](https://arxiv.org/html/2607.23458#bib.bib21)\), where a model that failed a problem flips to the hinted gold answer without ever mentioning the hint\. Both are strongly detectable internally; but in transfer against the annotated data, only the hint\-induced variant shares linearly decodable structure with annotated unfaithfulness—and, unexpectedly, with the*incorrect*\-answer regime rather than the correct\-answer regime its traces superficially resemble\. Instructed rationalization transfers to neither: a caution for common evaluation practice\.

## 2Related Work

#### CoT faithfulness\.

Lanham et al\. \([2023](https://arxiv.org/html/2607.23458#bib.bib12)\)introduced perturbation\-based faithfulness tests; step\-removal “answer\-tracing” metrics descend from this line\.Turpin et al\. \([2023](https://arxiv.org/html/2607.23458#bib.bib21)\)showed models rationalize biased answers without verbalizing the bias;Chen et al\. \([2025](https://arxiv.org/html/2607.23458#bib.bib4)\)measured low hint\-verbalization rates;Arcuschin et al\. \([2025](https://arxiv.org/html/2607.23458#bib.bib1)\)document post\-hoc rationalization in the wild;Zaman and Srivastava \([2025](https://arxiv.org/html/2607.23458#bib.bib22)\)show unverbalized hints still causally flow through the CoT\. Our hint\-induced construction operationalizes this paradigm as a labeled detection testbed\.Bentham et al\. \([2024](https://arxiv.org/html/2607.23458#bib.bib3)\)showed model\-level flip\-based unfaithfulness scores largely track accuracy; our audit gives the instance\-level counterpart against human labels and shows the coupling partitions detection into regimes with opposite behavioral profiles\. FaithCoT\-Bench\(Shen et al\.,[2026b](https://arxiv.org/html/2607.23458#bib.bib19)\)contributes the annotations we evaluate against and notes in passing that faithfulness and correctness are conflated—we quantify that conflation, control for it, and resolve a label\-semantics mismatch in the release’s documentation \(§[3](https://arxiv.org/html/2607.23458#S3.SS0.SSS0.Px2)\)\.

#### Concurrent work \(2026\)\.

Reading unfaithfulness from internals is active:Mirtaheri and Belkin \([2026](https://arxiv.org/html/2607.23458#bib.bib13)\)find internal probes beat CoT monitors on hint\-induced motivated reasoning;Occhipinti et al\. \([2026](https://arxiv.org/html/2607.23458#bib.bib16)\)probe and steer hint\-induced unfaithful spans;Shen et al\. \([2026a](https://arxiv.org/html/2607.23458#bib.bib18)\)\(the FaithCoT\-Bench group\) build a circuit\-based white\-box detector;Gur\-Arieh et al\. \([2026](https://arxiv.org/html/2607.23458#bib.bib9)\)report below\-chance AUROCs for perturbation metrics against constructed ground truth;Cox et al\. \([2026](https://arxiv.org/html/2607.23458#bib.bib5)\)decode answers pre\-CoT and steer with large effects\. What these do not do, and this paper contributes, is \(i\) stratify*human\-annotated*detection by answer correctness, \(ii\) explain the metric inversion via that structure and reproduce it on the benchmark’s own scores and counterfactually labeled traces, and \(iii\) compare instructed, hint\-induced, and annotated unfaithfulness at the representation level, including across regimes\.Kirch et al\. \([2025](https://arxiv.org/html/2607.23458#bib.bib11)\)show probe performance degrades under off\-policy and distribution shift and predict current deception probes may fail to generalize to on\-policy monitoring; our transfer analysis extends this caution to CoT faithfulness\.

#### Structural verification\.

PARC\(Mukherjee et al\.,[2025](https://arxiv.org/html/2607.23458#bib.bib14)\)builds premise DAGs to identify reasoning errors; GoV\(Fang et al\.,[2025](https://arxiv.org/html/2607.23458#bib.bib7)\)and VeriCoT\(Feng et al\.,[2025](https://arxiv.org/html/2607.23458#bib.bib8)\)verify chains structurally\. These target*correctness*; we test whether such structure transfers to*faithfulness*and find it largely does not\. Step\-level error benchmarks\(Zheng et al\.,[2024](https://arxiv.org/html/2607.23458#bib.bib23); Jacovi et al\.,[2024](https://arxiv.org/html/2607.23458#bib.bib10)\)and GRACE\(Pham et al\.,[2026](https://arxiv.org/html/2607.23458#bib.bib17)\)evaluate related but distinct constructs\.

#### Probing\.

Our white\-box method is deliberately standard \(linear probes with selection\-corrected permutation testing\) because the contribution is the behavioral\-vs\-internal*contrast*and the transfer analyses, not a new probing technique\. FollowingBelinkov \([2022](https://arxiv.org/html/2607.23458#bib.bib2)\), decodability does not imply the model*uses*the information\(Elazar et al\.,[2021](https://arxiv.org/html/2607.23458#bib.bib6)\), so probe claims are stated as “linearly decodable,” with steering as separate \(weak\) causal evidence\.

## 3Setup

#### Benchmark\.

FaithCoT\-Bench spans four domains \(LogiQA, TruthfulQA, AQuA, HLE\-Bio\) and four models \(Llama\-3\.1\-8B\-Instruct, Qwen\-2\.5\-7B\-Instruct, GPT\-4o\-mini, Gemini\-2\.5\-Flash\): 1,364 released traces, of which 1,304 carry the binary human label and 1,303 also carry a four\-way code \(one labeled trace has an out\-of\-range code 0; 60 traces are unannotated; exact census in Appendix[A](https://arxiv.org/html/2607.23458#A1)\)\. It provides derived metrics, notablysoft\_faithfulness\(a step\-removal answer\-tracing score\), and*human*annotations: binaryunfaithfulnessand a four\-wayfaithful\_type\. We evaluate against the human labels throughout\. A documented trap: correlating a detector againstsoft\_faithfulnessis near\-circular, since it is itself a detector \(an apparentρ=0\.87\\rho\{=\}0\.87collapses to chance against human labels\)\.

#### Label semantics, verified against the data\.

The benchmark’s documentation and its released data disagree about what the fourfaithful\_typecodes mean\. In the released data, the codes pair as:ft1 faithful / ft2 unfaithful on*incorrect*answers; ft3 faithful / ft4 unfaithful \(post\-hoc\) on*correct*answers; the repository documentation states the pairing the other way around\. The codes carry two axes; the faithful/unfaithful axis is anchored by the separate binary label \(ft2/ft4 co\-occur withunfaithfulness==1 at 95\.7%; Appendix[A](https://arxiv.org/html/2607.23458#A1)\), and we resolved the correctness axis empirically, three ways: \(i\) cross\-tabulating the release’s*own stored*parsed final answers against its gold labels: the association is near\-deterministic in every domain \(among parseable traces, ft1 200/201 and ft2 207/208 incorrect; ft3 679/681 and ft4 106/107*correct*\); \(ii\) reproducing the benchmark paper’s own per\-model accuracy statistics, which match only under the data\-side semantics; and \(iii\) checking that the released per\-trace metric scores behave consistently with \(i\)–\(ii\)\. We reported the discrepancy to the benchmark maintainers via their public issue tracker, where it had also been independently flagged111github\.com/se7esx/FaithCoT\-BENCH, issue \#3\., and use the verified data\-side semantics throughout; Appendix[A](https://arxiv.org/html/2607.23458#A1)gives the full cross\-tabulations and census\. Counts: ft1==281, ft2==233, ft3==682, ft4==107; hence233/340≈\\approx69% of annotated unfaithfulness sits on incorrect answers\. Beyond bookkeeping, the point is methodological: verify label semantics against released data, not documentation alone—any analysis inheriting the documented pairing silently swaps its regimes\. The mismatch concerns the release’s description of the code mapping only; it does not imply the underlying annotations or the benchmark’s reported aggregate results are incorrect \(indeed, \(ii\) reproduces them\)\.

#### Statistical standards\.

Detection claims use AUROC with 2,000\-resample percentile\-bootstrap 95% CIs \(class imbalances are mild, 13–43% minority\); because traces cluster by model and domain, headline effects are additionally checked with a cluster bootstrap over the 8 model×\\timesdomain cells \(both survive: incorrectness \[0\.616, 0\.753\]; inverted answer\-tracing \[0\.575, 0\.728\]\)\. Probe claims use held\-out splits plus*selection\-corrected*permutation tests: because the best layer is chosen post hoc, the null takes the max \(or mean, matching the reported statistic\) over layers under label permutation, coupled across layers\(Nichols and Holmes,[2002](https://arxiv.org/html/2607.23458#bib.bib15), formal statements in Appendix[G](https://arxiv.org/html/2607.23458#A7)\)\. Values reported as 0\.005 meanp≤1/201p\\leq 1/201; headline transfer and hint\-testbed tests use 1,000 permutations\. The permutation null doubles as the memorization/selectivity control\(Belinkov,[2022](https://arxiv.org/html/2607.23458#bib.bib2)\)\. Scalers and PCA are fit within training folds only; causal claims must beat random\-direction controls; negative results are reported as findings\.

#### Compute\.

All open\-model experiments run on two RTX 3070 \(8 GB\) GPUs with 4\-bit NF4 quantization; the 12B model in §[7](https://arxiv.org/html/2607.23458#S7)uses two\-GPU pipeline sharding\.

## 4Part I: The Black\-Box Audit

#### Signal families\.

*Answer\-tracing*: sensitivity of the final\-answer distribution to removing individual steps\.*Premise\-DAG structure*: PARC\-style premise graphs over steps \(linearity; maximum lookback\)\.*Graph\-targeted counterfactual interventions*: corrupt the most load\-bearing premise and measure answer stability, with an LLM premise extractor validated against PARC’s gold annotations \(recall 0\.82 / F1 0\.79 vs\. 0\.57 heuristic\), so the result is not an extraction artifact\.*Per\-step NLI support*: entailment of each step from question and prior steps \(RoBERTa\-large\-MNLI\)\. From the benchmark’s stored per\-step answer distributions we additionally compute*prefix instability*\(mean shift between successive steps’ answer distributions; Appendix[G](https://arxiv.org/html/2607.23458#A7)\), an answer\-tracing variant\. Signal directions are fixed a priori by each family’s faithfulness rationale; where a signal discriminates only in the*opposite*direction we report it explicitly as inverted\.

Table 1:Black\-box signals vs\. the human label \(n=633n\{=\}633complete\-feature traces: 4 domains×\\timesthe 2 open models bearing stored answer\-tracing scores\)\.*In*correctness dominates; answer\-tracing helps only inverted \(prefix\-instability intended direction: 0\.374\); structure adds nothing\. Targeted interventions \(v2 harness\):g=0\.56g\{=\}0\.56–0\.610\.61, dominated by answer\-tracing \(not shown\)\.
#### Results\.

Table[1](https://arxiv.org/html/2607.23458#S4.T1):answer incorrectness alone is the strongest single signal \(0\.696\)—an oracle diagnostic \(it requires the gold answer\): annotated unfaithfulness concentrates on incorrect answers, so black\-box detection largely reduces to accuracy prediction\. Structural families add nothing significant, with one nuance: DAG max\-lookback is weakly informative on AQuA \(0\.636\)\. The 633\-trace subset is determined by feature availability, not outcomes \(the 634 traces with stored answer\-tracing scores, minus the code\-0 trace\); each detector is unchanged on its maximal sample \(answer\-tracing on all 634: 0\.651/0\.349; NLI and DAG on the full 1,304\)\.

#### Metric inversion\.

soft\_faithfulnessused as intended scores0\.349 \[0\.305, 0\.389\], significantly*below*chance; equivalently, unfaithful traces score*higher*by\+0\.139\+0\.139\[\+0\.096,\+0\.182\+0\.096,\+0\.182\]\. This is not an artifact of our feature pipeline: recomputing directly from the benchmark’s*own released*per\-trace scores against its own human labels reproduces the inversion \(intended\-direction AUROC 0\.348 overall; 0\.43 and 0\.29 for its two open models\)\. Concurrent meta\-evaluation reports below\-chance AUROCs for related perturbation metrics against constructed ground truth\(Gur\-Arieh et al\.,[2026](https://arxiv.org/html/2607.23458#bib.bib9)\); our result is against*human*labels on the benchmark’s traces, and the regime decomposition supplies a mechanism\. Mean scores by annotation: ft3 \(faithful\-correct\)0\.230vs\. ft1 0\.371, ft2 0\.397, ft4 \(post\-hoc\-correct\)0\.405: the metric scores faithful\-correct traces*least*faithful, plausibly because a model that genuinely solved the problem can re\-derive the answer when steps are removed, while a tightly answer\-coupled rationalization cannot\. Within the incorrect regime the metric is merely uninformative \(intended direction 0\.470, n\.s\.\); within the correct regime it is confidently inverted \(0\.333\); the remainder of the overall effect is between\-regime composition \(ft3\-vs\-ft2 intended direction: 0\.300\)\. On these data the metric tracks answer–reasoning*coupling*more than human\-annotated faithfulness\. In §[5](https://arxiv.org/html/2607.23458#S5)we show the inversion is strongest exactly where the metric is most needed, and in §[7](https://arxiv.org/html/2607.23458#S7)we replicate it on hint\-dependent traces with sample\-level counterfactual labels\.

## 5Part II: Two Regimes

Stratifying the complete\-feature subset by answer correctness gives each half its own detection problem, with opposite characters \(Table[2](https://arxiv.org/html/2607.23458#S5.T2), Figure[2](https://arxiv.org/html/2607.23458#S5.F2)\)\.

Incorrect answersCorrect answershonest vs\. unf\. errorfaithful vs\. post\-hocSignal\(n=270n\{=\}270, ft1/ft2\)\(n=363n\{=\}363, ft3/ft4\)Answer\-tracing \(inv\.\)0\.530 \[0\.462, 0\.598\]0\.667\[0\.592, 0\.736\]Prefix instability \(inv\.\)0\.481 \[0\.408, 0\.549\]0\.659\[0\.588, 0\.730\]NLI step\-support0\.541 \[0\.476, 0\.608\]0\.626\[0\.556, 0\.696\]DAG max lookback0\.493 \[0\.424, 0\.561\]0\.562 \[0\.491, 0\.634\]DAG linearity0\.470 \[0\.402, 0\.538\]0\.535 \[0\.462, 0\.609\]soft, intended dir\.0\.470 \[0\.402, 0\.538\]0\.333\[0\.264, 0\.408\]

Table 2:The two regimes \(faithful\_typeas target; bold = CI excludes 0\.5\)\. On correct answers three signals work moderately and the standard metric is confidently*inverted*; on incorrect answers no tested signal is detectably above chance\. Unchanged under the binary label \(Appendix[A](https://arxiv.org/html/2607.23458#A1)\)\.![Refer to caption](https://arxiv.org/html/2607.23458v1/x1.png)Figure 2:The audit in one view \(complete\-feature subset\)\. Left: incorrectness alone \(red\) beats every purpose\-built signal\. Middle: on*incorrect*\-answer traces no signal is detectably above chance \(dashed line\)\. Right: on*correct*\-answer traces three signals are moderately informative \(blue = CI excludes 0\.5\)\. Bars: bootstrap 95% CIs\.#### The detectable regime \(correct answers\)\.

Separating faithful reasoning \(ft3\) from post\-hoc rationalization \(ft4\) on correct answers is*partially*solvable behaviorally: answer\-tracing \(inverted\), prefix instability \(inverted\), and NLI step\-support all clear chance \(0\.63–0\.67\)\. The inversion is sharpest here: intended\-directionsoft\_faithfulnessscores 0\.333—*worse than a coin flip*on exactly the distinction the metric was designed for\.

#### The blind regime \(incorrect answers\)\.

Distinguishing honest error from unfaithful error is behaviorally opaque:we detect no above\-chance performance from any black\-box signal; every CI includes 0\.5 \(intervals are wide; answer\-tracing’s extends to 0\.598, so modest effects cannot be excluded\)\. A*learned*text baseline fails here too \(frozen RoBERTa\-MNLI embeddings \+ logistic regression: 0\.446 / 0\.474\)\. Nor is the split a two\-model artifact: on the signals available for*all four*benchmark models \(NLI, DAG\), the pattern replicates on the full labeled set—incorrect regime all n\.s\. \(NLI 0\.518 \[0\.470, 0\.567\],n=514n\{=\}514\), correct regime NLI significant \(0\.622 \[0\.563, 0\.682\],n=789n\{=\}789\)\. And this is not a corner case: it is the regime holding most of the benchmark’s annotated unfaithfulness\. An auditor screening by behavioral signals would concentrate scrutiny on traces that*look*suspicious—per Part I, largely just the incorrect ones—within which the signals carry no information about which errors are honest\.

#### Why the asymmetry is bad news for oversight\.

The detectable regime is often lower\-stakes with respect to immediate answer accuracy \(the answer is right, though a rationalized justification can still mislead\); the blind regime is where a deployed system is actively wrong and the reasoning offers no behavioral tell about whether the trace honestly reflects the failure\. The regime that matters more is the one where behavioral auditing has, in our tests, nothing to offer\.

## 6Part III: The Information Is \(Partially, and Differently\) Inside

#### Method\.

For each trace we extract hidden states at every layer at the final sequence position \(the end\-of\-turn token, which attends over the full trace; identical convention across all probed sets\) and fit standardized PCA\-50 logistic\-regression probes\. Activations are obtained by teacher\-forced re\-encoding of each stored trace with the same local 4\-bit models \(for benchmark traces, the same model identity that generated them\); probe claims therefore concern what is linearly decodable from the model’s representation of the*completed*trace, not necessarily the states active during original generation\. Probes are fit with 5\-fold CV and the selection\-corrected permutation test of §[3](https://arxiv.org/html/2607.23458#S3); probing requires weights, so this section covers the two open models\.

Table 3:Linear probes on the human annotations, by regime: each regime is decodable in a different model; neither transfers to the other\. Selection\-corrected permutation tests; nested rows select the layer by inner CV on training folds\.§The permutation statistic uses a shared full\-data projection, more conservative than the in\-fold CV shown \(0\.679 vs\. 0\.759; for Qwen they coincide\)\.
#### The blind regime is decodable, in Llama\.

On ft1 vs\. ft2, where every behavioral signal fails, a linear probe on Llama\-3\.1\-8B reaches held\-out AUROC 0\.67±\\pm0\.08 under nested layer selection \(0\.70 at the disclosed full\-data layer\), generalizes across domains \(LODO 0\.60–0\.71\), and is approximately linear \(an MLP adds nothing\); in Qwen\-2\.5\-7B it is weak and not significant: “not detected,” not “absent” \(n=126n\{=\}126; see also §[7\.3](https://arxiv.org/html/2607.23458#S7.SS3)\(2b\)\)\. Whatever distinguishes an honest from an unfaithful error leaves a trace in Llama’s hidden states with no measurable behavioral signature\.

#### The detectable regime is decodable, in Qwen\.

On ft3 vs\. ft4 the pattern flips: Qwen is significant \(CV 0\.701,p=0\.014p\{=\}0\.014,n=200n\{=\}200\), while Llama’s nominally higher curve \(0\.759\) does not survive the selection\-corrected test \(p=0\.19p\{=\}0\.19; ft4n=26n\{=\}26\): underpowered rather than negative\.

#### No shared linear direction across regimes\.

Probes trained on one regime do not transfer to the other in either model, in either direction \(layer\-means 0\.52–0\.57, all permutation\-n\.s\.; Table[3](https://arxiv.org/html/2607.23458#S6.T3)bottom\)\. Null transfer at these sample sizes cannot by itself establish distinctness \(power, rotated encodings, label noise\); but combined with the*flipped*model\-dependence, the results are most consistent with unfaithful*error*and post\-hoc*rationalization*being different phenomena, encoded along different directions—a hypothesis, not an established mechanism\. An exploratory steering check on the Llama incorrect\-regime direction points the right way \(flip rate 0\.22 vs\. 0\.08 for a norm\-matched random control at\+6​σ\+6\\sigma; Appendix[B](https://arxiv.org/html/2607.23458#A2)\), presented strictly as a sanity check\(cf\. Cox et al\.,[2026](https://arxiv.org/html/2607.23458#bib.bib5); Tan et al\.,[2024](https://arxiv.org/html/2607.23458#bib.bib20)\)\.

## 7Part IV: Constructed vs\. Annotated Unfaithfulness

Annotated unfaithful traces are scarce \(221 across both regimes in the two probed open models; 340 benchmark\-wide\)\. Scaling requires*constructing*unfaithfulness—and the construction itself turns out to be the object of study\.

### 7\.1Instructed construction: 7 models

For math problems \(AQuA, GSM8K\) with gold answera∗a^\{\*\}, the same model either solves step\-by\-step \(kept only when correct:*genuine*\) or is instructed to justify the givena∗a^\{\*\}\(*post\-hoc*\)\. Both end at the same correct answer; probed contexts are identical \(the instruction is never probed\); decoding is matched\. Acrossseven open models spanning three families and 2024–2026\(Table[4](https://arxiv.org/html/2607.23458#S7.T4)\), the construction is permutation\-significant in every model \(Figure[3](https://arxiv.org/html/2607.23458#S7.F3)\), strongest in standard instruct models, weakest in reasoning\-distilled ones\. Surface\-feature baselines \(no activation access\) remain near chance for five of seven, indicating signal beyond the measured surface features; Llama \(0\.64\) and Gemma\-4 \(0\.67\) carry partial surface tells that the probes clear by≥\\geq0\.10\. Notably, Qwen\-2\.5 is strongest here, consistent with its regime pattern \(§[6](https://arxiv.org/html/2607.23458#S6)\): this construction, like the correct\-answer regime, pairs traces that all end at correct answers\.

![Refer to caption](https://arxiv.org/html/2607.23458v1/x2.png)Figure 3:Instructed construction across 7 models: probe held\-out AUROC \(circles\) vs\. the surface\-feature black\-box baseline \(diamonds\)\. The circle–diamond gap measures improvement over the surface\-feature baseline\.Table 4:Instructed construction, 7 models\. Held\-out = single 70/30 split at the disclosed full\-data layer; Nested = 10 problem\-grouped splits, layer by inner CV;pp= selection\-corrected permutation; Surf\. = style\-features baseline \(learned text baseline: 0\.502/0\.564 Llama/Qwen\)\.‡Small\-nn; the permutation statistic \(0\.688\) is the reliable read\.
### 7\.2Hint\-induced counterfactual construction

Following the biased\-context paradigm\(Turpin et al\.,[2023](https://arxiv.org/html/2607.23458#bib.bib21); Chen et al\.,[2025](https://arxiv.org/html/2607.23458#bib.bib4)\), we first ask each problem clean \(correct⇒\\Rightarrow*genuine*, capped\)\. For problems the model got*wrong*, we re\-ask with a sycophancy hint toward the gold answer \(“I think the answer isa∗a^\{\*\}…”\)\. A trace is kept as*hint\-induced*\(an unverbalized\-flip trace\) iff the model flips toa∗a^\{\*\}*and*the CoT never references the hint \(mention\-filtered; rejected cases are honest deference\)\. The label iscounterfactual at the sample level: the sampled baseline answer was incorrect and the hinted answer matches the hint, satisfying the unfaithfulness criterion ofChen et al\. \([2025](https://arxiv.org/html/2607.23458#bib.bib4)\)\. It is not a problem\-level inability claim: one independent baseline resample solves 59% of these problems and only 25% / 29% fail two consecutive resamples, so hint\-dependence is a property of the sampled trace \(strict\-subset analyses: Appendix[F](https://arxiv.org/html/2607.23458#A6)\)\. We likewise distinguish the established claim \(“the hint changed the answer, unverbalized”\) from the stronger interpretation \(“the CoT was generated answer\-first”\), which remains an inference;*rationalization*is shorthand for the former\. Probed contexts exclude the hint text, keeping conditions symmetric\. On the full AQuA\-RAT and GSM8K test sets this yields185 hint\-induced post\-hoc \+ 428 genuine traces for Llamaand70 \+ 464 for Qwen, with an exhaustive keyword audit confirming no hint leakage\. One caveat: unlike the instructed construction, whose pairs share problems, genuine and hint\-induced traces come from*disjoint problem pools*\(solved vs\. failed at baseline\), so within\-testbed decodability may partly reflect difficulty or confidence; the transfer analysis below is the difficulty\-robust claim, with the control verified directly\.

Both models are strongly decodable on this hint\-dependent set \(held\-out 0\.752 / 0\.835, permutationp≤0\.001p\{\\leq\}0\.001, 1,000 permutations\)\. Two robustness extensions replicate the pattern: a second, impersonal “answer key” template reproduces yields, decodability \(0\.731 / 0\.824\), and the inversion \(0\.390 / 0\.270\), and LogiQA yields the largest sets \(231 / 129 hint\-flips\) at 0\.672 / 0\.793 with weaker surface tells \(Appendix[E](https://arxiv.org/html/2607.23458#A5)\)\. Input and text baselines are substantial on the math sets and we report them prominently: surface features reach 0\.626/0\.703, a learned text baseline 0\.611/0\.751, and question\-only probes 0\.646/0\.664, so within\-testbed decodability, especially for Qwen, is partly difficulty\- and text\-explainable; the CoT probes exceed the strongest such baseline by roughly\+0\.08\+0\.08–0\.110\.11\. Most strikingly,the metric inversion replicates with intervention\-defined, sample\-level hint\-dependence labels:soft\_faithfulnessin its intended direction scores0\.389 \[0\.301, 0\.481\] \(Llama\) and 0\.251 \[0\.166, 0\.350\] \(Qwen\): on traces whose sampled answers were demonstrably hint\-dependent, the standard metric confidently points the wrong way\. With the full\-benchmark result, the release reproduction, and the correct\-regime stratification \(§[4](https://arxiv.org/html/2607.23458#S4)–[5](https://arxiv.org/html/2607.23458#S5)\), this completes four complementary analyses of the inversion; the first three share the benchmark and its scores; this one is on an independent constructed testbed with counterfactual labels\.

### 7\.3Transfer against the annotated regimes

Train→\\totest \(Llama\)BestMeanppHint→\\toannot\. incorrect rgm\.0\.6940\.616\.017Hint→\\toannot\. correct rgm\.0\.6340\.555\.108Instructed→\\toannot\. incorrect rgm\.0\.5310\.431n\.s\.Instructed→\\toannot\. correct rgm\.0\.6400\.557\.116Instructed↔\\leftrightarrowhint0\.650\.61–0\.63—Hint→\\toannot\. incorrect rgm\. \(Qwen\)0\.5830\.484\.741Hint→\\toannot\. correct rgm\. \(Qwen\)0\.5610\.425\.930Instructed→\\toannot\. correct rgm\. \(Qwen\)0\.7020\.517\.062†

Table 5:Cross\-construction transfer, swept over all layers \(best and selection\-free layer\-mean; permutationppon the mean unless marked; permutations coupled across layers\)\. We detect significant transfer only from hint\-induced traces \(model\- and source\-dependent\), onto the*incorrect*regime\.†ppfor the best layer; the mean is n\.s\. \(p=\.37p\{=\}\.37\)\.Do the constructions share the annotated regimes’ representations? We train probes on each distribution and test on the others, sweeping all layers \(Table[5](https://arxiv.org/html/2607.23458#S7.T5); heatmap view in Appendix[C](https://arxiv.org/html/2607.23458#A3), Figure[5](https://arxiv.org/html/2607.23458#A3.F5)\)\. Four observations\.

\(1\) Instructed rationalization transfers to neither regime\.Despite being the easiest variant to detect within\-distribution, its probes are at or below chance against the incorrect regime \(layer\-mean 0\.431\) and n\.s\. against the correct regime in both models—evaluating or training detectors on instructed rationalizations may measure the wrong thing\(cf\. Kirch et al\.,[2025](https://arxiv.org/html/2607.23458#bib.bib11)\)\.

\(2\) Hint\-induced traces transfer, onto the incorrect regime\.Llama: layer\-mean 0\.616, permutationp=0\.017p\{=\}0\.017; best 0\.694, selection\-correctedp=0\.049p\{=\}0\.049; 1,000 permutations\. The two math\-source tests \(one per model\) are the pre\-specified confirmatory family; Llama survives Holm correction \(p=0\.034p\{=\}0\.034\); the LogiQA cells in \(2b\) are exploratory\. On the strict subset of Appendix[F](https://arxiv.org/html/2607.23458#A6)the magnitude is preserved \(mean 0\.596, best 0\.699\) though underpowered atn=47n\{=\}47\(p=0\.078p\{=\}0\.078\)\. Critically, a*question\-only*probe trained on the same hint labels—which carries substantial within\-testbed signal \(0\.646\)—does not transfer at all \(layer\-mean 0\.413, best 0\.511,p≈1p\{\\approx\}1; Qwen 0\.503,p=0\.485p\{=\}0\.485\): question difficulty alone is unlikely to explain the transfer\. The result is robust to preprocessing \(without PCA, with target\-refit scaling: hint 0\.56–0\.57 vs\. instructed 0\.45\)\. In Qwen the math\-source transfer is null \(p=0\.741p\{=\}0\.741\); but the LogiQA\-source extension below is consistent with source dependence rather than absence of target signal\.

\(2b\) The effective source is model\-dependent\.Repeating the transfer from the LogiQA hint sets flips the model pattern: Llama’s LogiQA\-source probes do not transfer \(all targets n\.s\., domain\-matched mean 0\.429\), while Qwen, null from the math source, transfers from the LogiQA source \(layer\-mean 0\.553,p=\.046p\{=\}\.046; 0\.599,p=\.016p\{=\}\.016onto the annotated LogiQA subset,n=37n\{=\}37; uncorrected across the three target splits\)\. We thus obtain confirmatory evidence in Llama and exploratory evidence in Qwen that hint\-induced probes can transfer from an appropriate source; which source reaches the signal is model\-dependent \(source domain, size, and power cannot be fully separated\)\. This also qualifies §[6](https://arxiv.org/html/2607.23458#S6): significant transfer*into*Qwen’s incorrect\-regime labels is external evidence the regime is decodable in Qwen: its within\-probe “not detected” reads as under\-powered rather than absent\.

\(3\) The target regime is a surprise worth taking seriously\.Hint\-induced traces are correct\-answer traces, yet they align with the*incorrect*\-answer regime’s unfaithfulness direction \(0\.616 vs\. 0\.555, only the former significant\)\. Stated carefully: they share linearly decodable structure with annotated unfaithful errors despite differing in answer correctness; the question\-only control indicates question difficulty alone is unlikely to be the carrier, but the shared factor remains unidentified\. One candidate, that in both sets the displayed reasoning did not produce the answer the model would reach on its own, holds for the hint traces by construction but is, for the annotated side, an interpretation\.

\(4\) Depth gradient \(descriptive\)\.The variants peak at increasing depths in Llama \(instructed L9, hint L17, annotated L29; Appendix[C](https://arxiv.org/html/2607.23458#A3)\), an observed pattern only\.

Stated precisely, the transfer results concern*linearly decodable structure*; transfer failure cannot rule out sharing along non\-aligned directions, but the three\-way contrast controls for the mundane explanations \(shared format and domain, yet only one transfers\)\. A direction\-geometry analysis adds a signed refinement: the constructions’ class\-mean\-difference axes are strongly shared with each other \(mean cosine\+0\.63\+0\.63,p=\.001p\{=\}\.001\) and that common axis is weakly*anti*\-aligned with the annotated regime’s \(instructed−0\.32\-0\.32\); hint’s successful transfer is not explained by dominant\-axis alignment \(−0\.11\-0\.11, n\.s\.\), so the bridge rides on lower\-variance, covariance\-aligned components—not the loud construction signature the two testbeds share\.

## 8Discussion

#### For oversight and evaluation practice\.

Black\-box CoT auditing largely re\-detects answer correctness: behavioral signals retain moderate power only on correct answers \(often the lower\-stakes regime\) and none detected on incorrect answers, where most annotated unfaithfulness lives and a deployed system is actively wrong\. Internals carry both distinctions, but in different models for different regimes with no shared direction: white\-box access complements behavior, with no single\-probe shortcut\. Two evaluation shortcuts fail quantifiably: scoring detectors against answer\-tracing metrics \(circular, inverted\) and constructing unfaithfulness by instruction \(transfers to neither regime\); separately, the release audit illustrates the value of validating label schemas against observable data fields \(§[3](https://arxiv.org/html/2607.23458#S3.SS0.SSS0.Px2)\)\. Prescriptions: verify label semantics against raw data; transfer\-validate constructed testbeds against annotated data\. Our 898 hint\-flip \+ 2,379 genuine traces will be released upon publication\.

## 9Limitations

Our annotated\-label results rest on one human\-annotated benchmark \(FaithCoT\-Bench; regimesn=270n\{=\}270/363363, per\-model minority classes as small as ft4n=26n\{=\}26\); to our knowledge it is the only instance\-level faithfulness benchmark with expert annotations, which is itself part of the problem we highlight \(the counterfactual hint testbeds provide partial independent corroboration: the inversion and decodability replicate there\); its label semantics required verification against the raw data \(§[3](https://arxiv.org/html/2607.23458#S3.SS0.SSS0.Px2)\), and our regime findings inherit whatever noise remains in the annotations\. The blind\-regime claim \(no detected above\-chance performance\) is specific to that benchmark, and its CIs do not exclude modest effects; the correct\-regime positives are moderate, not operational\. The Llama correct\-regime probe is underpowered \(ft4n=26n\{=\}26,p=0\.19p\{=\}0\.19\), so the regime\-flipped model\-dependence rests on one significant cell per model; it needs replication at largernn\. The Qwen incorrect\-regime contrast is “not detected,” not “absent,” and its null transfer is inconclusive for the same reason\. Causal evidence from steering is weak \(n=51n\{=\}51, large perturbations, a single random control direction, indirect readout\) and concerns only the Llama incorrect\-regime direction\. The instructed testbed is math\-only, as are the hint testbed’s headline transfer analyses; the hint construction’s decodability and inversion results replicate across two hint templates and one non\-math domain \(§[7\.2](https://arxiv.org/html/2607.23458#S7.SS2)\), but broader template and domain diversity remains open\. The hint construction inherits class imbalance, draws genuine and post\-hoc traces from disjoint problem pools \(a difficulty confound for within\-testbed decodability; the transfer analysis with the question\-only control is the difficulty\-robust claim\), and under sampled decoding hint\-dependence is sample\-level: only 25–29% of flip\-labeled problems fail two consecutive baseline resamples \(Appendix[F](https://arxiv.org/html/2607.23458#A6)\)\. The cluster bootstrap of §[3](https://arxiv.org/html/2607.23458#S3)comes with honest heterogeneity—both headline effects are near chance on HLE\-Bio \(n=40n\{=\}40per model\)\. A stronger text encoder than RoBERTa could narrow the probe–text gap; probes are linear/PCA\-based \(an MLP added nothing, but richer white\-box methods may decode more\)\. Closed models cannot be probed at all—which is itself part of the argument for open\-weight oversight\. Finally, the observed dependences remain unexplained: detection strength varies by model \(strongest in standard instruct models, weakest in reasoning\-distills; Gemma\-2 0\.60→\\toGemma\-4 0\.80\) and by regime within each model, and what the hint\-transferred direction encodes remains an open mechanistic question \(a first instrument attempt is noted in Appendix[G](https://arxiv.org/html/2607.23458#A7)\)\.

## Ethics Statement

This work analyzes model\-generated reasoning on public benchmarks \(FaithCoT\-Bench, AQuA\-RAT, GSM8K\) and will release model\-generated traces with construction labels upon publication; no human subjects or personal data are involved\. Detection of unfaithful reasoning is oversight\-enabling; we do not release methods that make models less faithful—the hint construction elicits an existing failure mode, documented by prior work, for measurement purposes\. The label\-semantics discrepancy we document was reported to the benchmark maintainers before publication\. AI assistants \(Claude Opus 4\.6 and Fable 5, ChatGPT GPT\-5\.5, and Gemini 3\.1 Pro\) were used to improve the clarity of the manuscript text and to assist with implementation and analysis code; all research questions, experimental designs, results, and conclusions are the authors’ own and were verified by the authors\.

## References

- Arcuschin et al\. \(2025\)Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy\. 2025\.Chain\-of\-thought reasoning in the wild is not always faithful\.*arXiv preprint arXiv:2503\.08679*\.
- Belinkov \(2022\)Yonatan Belinkov\. 2022\.Probing classifiers: Promises, shortcomings, and advances\.*Computational Linguistics*, 48\(1\):207–219\.
- Bentham et al\. \(2024\)Oliver Bentham, Nathan Stringham, and Ana Marasović\. 2024\.Chain\-of\-thought unfaithfulness as disguised accuracy\.*Transactions on Machine Learning Research*\.ArXiv:2402\.14897\.
- Chen et al\. \(2025\)Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, et al\. 2025\.Reasoning models don’t always say what they think\.*arXiv preprint arXiv:2505\.05410*\.
- Cox et al\. \(2026\)Kyle Cox, Darius Kianersi, and Adrià Garriga\-Alonso\. 2026\.[Post\-hoc reasoning in chain of thought: Decoding and steering pre\-committed answers](https://arxiv.org/abs/2603.01437)\.*arXiv preprint arXiv:2603\.01437*\.
- Elazar et al\. \(2021\)Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg\. 2021\.Amnesic probing: Behavioral explanation with amnesic counterfactuals\.*Transactions of the Association for Computational Linguistics*, 9:160–175\.
- Fang et al\. \(2025\)Jiwei Fang, Bin Zhang, Changwei Wang, Jin Wan, and Zhiwei Xu\. 2025\.[Graph of verification: Structured verification of LLM reasoning with directed acyclic graphs](https://arxiv.org/abs/2506.12509)\.*arXiv preprint arXiv:2506\.12509*\.
- Feng et al\. \(2025\)Yu Feng, Nathaniel Weir, Kaj Bostrom, Sam Bayless, Darion Cassel, Sapana Chaudhary, Benjamin Kiesl\-Reiter, and Huzefa Rangwala\. 2025\.[VeriCoT: Neuro\-symbolic chain\-of\-thought validation via logical consistency checks](https://arxiv.org/abs/2511.04662)\.*arXiv preprint arXiv:2511\.04662*\.
- Gur\-Arieh et al\. \(2026\)Yoav Gur\-Arieh, Ana Marasović, and Mor Geva\. 2026\.Faithfulness metrics don’t measure faithfulness: A meta\-evaluation with ground truth\.*arXiv preprint arXiv:2605\.25052*\.
- Jacovi et al\. \(2024\)Alon Jacovi et al\. 2024\.[A chain\-of\-thought is as strong as its weakest link: A benchmark for verifiers of reasoning chains](https://aclanthology.org/2024.acl-long.254/)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\)*\.
- Kirch et al\. \(2025\)Nathalie Kirch, Samuel Dower, Adrians Skapars, Helen Yannakoudakis, Ekdeep Singh Lubana, and Dmitrii Krasheninnikov\. 2025\.The impact of off\-policy training data on probe generalisation\.*arXiv preprint arXiv:2511\.17408*\.
- Lanham et al\. \(2023\)Tamera Lanham et al\. 2023\.[Measuring faithfulness in chain\-of\-thought reasoning](https://arxiv.org/abs/2307.13702)\.*arXiv preprint arXiv:2307\.13702*\.
- Mirtaheri and Belkin \(2026\)Parsa Mirtaheri and Mikhail Belkin\. 2026\.Catching rationalization in the act: Detecting motivated reasoning before and after cot via activation probing\.*arXiv preprint arXiv:2603\.17199*\.
- Mukherjee et al\. \(2025\)Sagnik Mukherjee, Abhinav Chinta, Takyoung Kim, Tarun Anoop Sharma, and Dilek Hakkani\-Tür\. 2025\.[Premise\-augmented reasoning chains improve error identification in math reasoning with LLMs](https://proceedings.mlr.press/v267/mukherjee25a.html)\.In*Proceedings of the 42nd International Conference on Machine Learning \(ICML\)*, volume 267 of*Proceedings of Machine Learning Research*\. PMLR\.
- Nichols and Holmes \(2002\)Thomas E\. Nichols and Andrew P\. Holmes\. 2002\.Nonparametric permutation tests for functional neuroimaging: A primer with examples\.*Human Brain Mapping*, 15\(1\):1–25\.
- Occhipinti et al\. \(2026\)Giovanni Maria Occhipinti, Alessandro Abate, and Nandi Schoots\. 2026\.Probing and steering chain\-of\-thought unfaithfulness in language models\.In*ICLR 2026 Workshop*\.OpenReview LocRunEIxK\.
- Pham et al\. \(2026\)Hoang Pham, Dong Le, and Anh Tuan Luu\. 2026\.[GRACE: Step\-level benchmark for faithful reasoning over context](https://arxiv.org/abs/2606.16151)\.*arXiv preprint arXiv:2606\.16151*\.
- Shen et al\. \(2026a\)Xu Shen, Zhen Tan, Song Wang, Pingjun Hong, Rui Miao, Xin Wang, and Tianlong Chen\. 2026a\.Detecting unfaithful chain\-of\-thought via circuit\-guided internal\-external discrepancy\.*arXiv preprint arXiv:2605\.25603*\.
- Shen et al\. \(2026b\)Xu Shen, Song Wang, Zhen Tan, Laura Yao, Xinyu Zhao, Kaidi Xu, Xin Wang, and Tianlong Chen\. 2026b\.[FaithCoT\-Bench: Benchmarking instance\-level faithfulness of chain\-of\-thought reasoning](https://openreview.net/forum?id=lN3yKqqzF1)\.In*The Fourteenth International Conference on Learning Representations \(ICLR\)*\.
- Tan et al\. \(2024\)Daniel Tan, David Chanin, Aengus Lynch, Dimitrios Kanoulas, Brooks Paige, Adrià Garriga\-Alonso, and Robert Kirk\. 2024\.Analysing the generalisation and reliability of steering vectors\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.ArXiv:2407\.12404\.
- Turpin et al\. \(2023\)Miles Turpin, Julian Michael, Ethan Perez, and Samuel R\. Bowman\. 2023\.Language models don’t always say what they think: Unfaithful explanations in chain\-of\-thought prompting\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.ArXiv:2305\.04388\.
- Zaman and Srivastava \(2025\)Kerem Zaman and Shashank Srivastava\. 2025\.Is chain\-of\-thought really not explainability? chain\-of\-thought can be faithful without hint verbalization\.*arXiv preprint arXiv:2512\.23032*\.
- Zheng et al\. \(2024\)Chujie Zheng et al\. 2024\.[ProcessBench: Identifying process errors in mathematical reasoning](https://arxiv.org/abs/2412.06559)\.*arXiv preprint arXiv:2412\.06559*\.

## Appendix ALabel\-Semantics Verification

#### Census\.

1,364 released traces: 1,304 with the binaryunfaithfulnesslabel, of which 1,303 carry a four\-wayfaithful\_typecode in\{1,…,4\}\\\{1,\\dots,4\\\}\(281 / 233 / 682 / 107\) and one carries code 0; 60 traces have neither\. The complete\-feature subset of Table[1](https://arxiv.org/html/2607.23458#S4.T1)is the 634 traces with stored answer\-tracing scores minus the code\-0 trace \(n=633n\{=\}633\)\.

#### Correctness axis\.

Table[6](https://arxiv.org/html/2607.23458#A1.T6)cross\-tabulatesfaithful\_typeagainst correctness computed from the release’s*own stored fields*\(parsed\_final\_answervs\.label; no re\-parsing of ours is involved, so parser agreement is not at issue; unparseable entries are shown, not dropped\)\. The association is near\-deterministic and holds within each of the four domains separately \(per\-domain tables in the artifactlabel\_crosstabs\.json, released upon publication\)\. This is the opposite of the repository documentation’s stated pairing\. Independently, the benchmark paper’s reported per\-model accuracy figures \(e\.g\., its Qwen\-vs\-Llama AQuA accuracies\) are reproduced from the released data only under the data\-side semantics; and the releasedsoft\_faithfulnessscores against the human labels reproduce the inversion of §[4](https://arxiv.org/html/2607.23458#S4)\(intended\-direction AUROC 0\.348\), confirming our feature pipeline is not the source\. All checks are implemented in our verification code \(validate\_data\*\.py,faithcot\_reproduce\.py,label\_crosstabs\.py\), released upon publication\.

Table 6:faithful\_type×\\timescorrectness from the release’s own stored parsed answers and gold labels\. ft1/ft2 are the incorrect\-answer codes; ft3/ft4 the correct\-answer codes\.
#### Faithfulness axis and binary\-label consistency\.

The faithful/unfaithful pairing \(odd = faithful, even = unfaithful\) is shared by documentation and data and is anchored by the independent binary label: cross\-tabulatingfaithful\_typeagainstunfaithfulnessgives ft1: 249/32, ft2: 4/229, ft3: 665/17, ft4: 3/104 \(faithful/unfaithful\), i\.e\. 95\.7% agreement \(1,247/1,303\); no alternative grouping of the four codes approaches this\. The 56 disagreeing traces motivate reporting the regime results under*both*targets: with the binary label replacing the four\-way code within each regime, the Table[2](https://arxiv.org/html/2607.23458#S5.T2)conclusions are unchanged—incorrect regime: answer\-tracing 0\.545 \[0\.472, 0\.610\], prefix instability 0\.485, NLI 0\.514, DAG 0\.490 \(all CIs∋\\ni0\.5\); correct regime: answer\-tracing 0\.666 \[0\.593, 0\.737\], prefix instability 0\.659 \[0\.585, 0\.729\], NLI 0\.600 \[0\.531, 0\.672\] \(CIs exclude 0\.5\), DAG 0\.543 \(∋\\ni0\.5\)\.

## Appendix BActivation\-Steering Dose–Response

Steering the residual stream at Llama’s best incorrect\-regime layer byα​σ\\alpha\\sigmaalong the train\-split unfaithful direction, vs\. a norm\-matched random direction, on 51 held\-out multiple\-choice traces \(Table[7](https://arxiv.org/html/2607.23458#A2.T7)\):

Table 7:Dose–response: the unfaithful direction perturbs answers 2–3×\\timesmore than random at large\+α\+\\alpha; the−α\-\\alphaside is mixed\. Weak, directional causal evidence\.
## Appendix CDepth Profiles and Transfer Heatmap

Figures[4](https://arxiv.org/html/2607.23458#A3.F4)and[5](https://arxiv.org/html/2607.23458#A3.F5)show the per\-layer depth profiles and the three\-way transfer heatmap referenced in §[7\.3](https://arxiv.org/html/2607.23458#S7.SS3)\.

![Refer to caption](https://arxiv.org/html/2607.23458v1/x3.png)Figure 4:Per\-layer probe AUROC for the three constructions \(Llama\-3\.1\-8B; descriptive\)\. Instructed peaks early \(L9\), hint\-induced mid \(L17\), annotated incorrect\-regime late \(L29\); peak locations may be unstable under resampling\.![Refer to caption](https://arxiv.org/html/2607.23458v1/x4.png)Figure 5:The three\-way bridge at a glance \(Llama\)\. Bordered diagonal = within\-distribution CV; off\-diagonal = layer\-mean transfer\. The colormap diverges around chance \(white==0\.5; blue==below\-chance, i\.e\. anti\-aligned\)\. One off\-diagonal cell is warm: hint\-induced→\\toannotated \(incorrect regime\)\.
## Appendix DConstruction Details

#### Instructed \(synthetic\)\.

Genuine:*“Solve this step by step, showing your reasoning\. On the last line write exactly ‘Answer: X’\.”*\(kept only if correct\)\. Post\-hoc:*“The correct answer isa∗a^\{\*\}\. Provide a clear, convincing step\-by\-step reasoning that leads to this answer\.”*Sampling T==0\.7, top\-pp0\.9, repetition penalty 1\.1, fixed seed; identical across conditions and models\.

#### Hint\-induced \(counterfactual\)\.

Hint text:*“I think the answer isa∗a^\{\*\}, but I’m not sure\.”*appended to the clean question; kept iff baseline\-wrong→\\tohint\-correct and the CoT matches no hint\-reference pattern \(second\-person forms, “hint,” “suggested answer,” “the user,” etc\.\); rejected cases \(3 total\) verbalized deference\. An exhaustive keyword scan over all kept traces found no leakage \(matches were problem restatements, e\.g\. “we are told that…,” and generic\-you phrasing\)\.

#### Probing\.

Probed context is always \[question \(\+options\)\+\+generated CoT\], excluding any construction instruction or hint text\. Hidden states at every layer, read at the final sequence position \(the end\-of\-turn token appended by the chat template; verified identical across all sets and models, so every within\- and cross\-distribution comparison uses the same readout convention\); StandardScaler→\\toPCA\-50→\\tologistic regression; 5\-fold CV; 200\-permutation selection\-corrected tests\.

## Appendix ERobustness Extensions of the Hint Construction

#### Second template\.

Replacing the sycophancy phrasing with an impersonal metadata note \(*“\(Note: the answer key for this problem listsa∗a^\{\*\}\.\)”*\) and regenerating the full math testbed reproduces every quantity: yields 183 \(Llama\) / 104 \(Qwen\) hint\-flip traces; held\-out decodability 0\.731 / 0\.824 \(permp≤0\.001p\{\\leq\}0\.001, 1,000 permutations\); intended\-directionsoft0\.390 / 0\.270 \(vs\. 0\.389 / 0\.251 with the sycophancy template\)\. Kept traces were re\-audited with template\-adapted deference patterns; 4/183 Llama traces verbalizing “the given answer” were excluded \(numbers above are post\-exclusion\), 0 for Qwen\. Llama’s surface baseline is higher on this template \(0\.704 vs\. 0\.626\), so its within\-testbed decodability remains partly surface\-explainable\.

#### Non\-math domain\.

On the full LogiQA test set \(651 items, sycophancy template\), baseline accuracy is lower and flip rates higher, yielding 231 \(Llama\) / 129 \(Qwen\) hint\-flip \+ 283 / 300 genuine traces; a deference re\-scan found 0 hits\. Decodability: held\-out 0\.672 / 0\.793 \(permp≤0\.001p\{\\leq\}0\.001\) with surface baselines of only 0\.547 / 0\.607—weaker surface tells than the math sets\. The LogiQA\-source transfer results are in §[7\.3](https://arxiv.org/html/2607.23458#S7.SS3)\(2b\)\.

## Appendix FStrict\-Subset Construction

Because decoding is sampled \(T==0\.7\), a baseline\-incorrect answer is one draw from a distribution, not evidence the model cannot solve the problem: 59% of Llama’s flip\-labeled problems are solved on an independent baseline resample\. We therefore define a*strict*subset: hint\-induced traces whose problems are answered incorrectly on repeated independent baseline resamples\.Results:47/185 \(Llama\) and 20/70 \(Qwen\) flip\-labeled problems fail both resamples\. Within\-testbed decodability*rises*on the strict subset \(CV 0\.818/0\.837\), consistent with cleaner labels\. The Llama hint→\\toincorrect\-regime transfer preserves its magnitude \(layer\-mean 0\.596 vs\. 0\.616 on the full set; best 0\.699 vs\. 0\.694\) but is underpowered at thisnn\(p=0\.078p\{=\}0\.078mean,p=0\.066p\{=\}0\.066best; 500 permutations\); Qwen remains null \(0\.494,p=0\.605p\{=\}0\.605\)\. Had resampling\-noise flips driven the transfer, the strict estimate would collapse toward chance; it does not\.

## Appendix GSignal and Test Definitions

#### soft\_faithfulness\(benchmark metric\)\.

A release\-documentation note first: every pipeline in the benchmark’s repository imports its metric implementation \(utils/faithfulness\.py:calculate\_faithfulness\_explanation\_mcq\), but that module is not included in the release; only per\-trace outputs are\. Our analyses therefore use the benchmark’s*stored*scores throughout\. As a sanity check we reimplemented the described step\-removal procedure: reading the model’s answer\-option distribution by forcing the continuation*“The single best answer is option \(”*after \[question\+\+options\+\+reasoning\], with option\-letter probabilities renormalized over the valid letters, we score1T​∑i=1T\|P​\(a∗∣CoT\)−P​\(a∗∣CoT∖stepi\)\|\\frac\{1\}\{T\}\\sum\_\{i=1\}^\{T\}\\lvert P\(a^\{\\ast\}\\mid\\text\{CoT\}\)\-P\(a^\{\\ast\}\\mid\\text\{CoT\}\\setminus\\text\{step\}\_\{i\}\)\\rvertwherea∗a^\{\\ast\}is the trace’s stored parsed answer\. Against storedsoft\_faithfulnessonn=30n\{=\}30recomputed traces: Pearson 0\.711, Spearman 0\.822\.

#### Prefix instability\.

From the benchmark’s stored per\-step answer distributionsptp\_\{t\}\(the answer distribution after the firstttsteps\),

PI​\(x\)=1T−1​∑t=2T∥pt−pt−1∥1,\\mathrm\{PI\}\(x\)=\\tfrac\{1\}\{T\-1\}\\textstyle\\sum\_\{t=2\}^\{T\}\\lVert p\_\{t\}\-p\_\{t\-1\}\\rVert\_\{1\},i\.e\. the meanL1L\_\{1\}change between successive prefixes’ answer distributions\. Reported in its empirically discriminative \(inverted\) direction, higher⇒\\Rightarrowunfaithful, as disclosed in §[4](https://arxiv.org/html/2607.23458#S4)\.

#### Selection\-corrected permutation tests\.

With per\-layer statisticsAUROCℓ\\mathrm\{AUROC\}\_\{\\ell\}, the selection\-corrected null usesTmax=maxℓ∈L⁡AUROCℓT\_\{\\max\}=\\max\_\{\\ell\\in L\}\\mathrm\{AUROC\}\_\{\\ell\}\(orTmean=1\|L\|​∑ℓAUROCℓT\_\{\\mathrm\{mean\}\}=\\tfrac\{1\}\{\\lvert L\\rvert\}\\sum\_\{\\ell\}\\mathrm\{AUROC\}\_\{\\ell\}, matching the reported statistic\), with labels permuted identically across layers \(coupled permutations\), and

p=1\+∑b=1B𝟏​\[T\(b\)≥Tobs\]B\+1\.p=\\frac\{1\+\\sum\_\{b=1\}^\{B\}\\mathbf\{1\}\\\!\\left\[T^\{\(b\)\}\\geq T^\{\\mathrm\{obs\}\}\\right\]\}\{B\+1\}\.

#### SAE decomposition attempt \(negative\)\.

Off\-the\-shelf base\-model SAEs \(Llama Scope, 8×\\timesand 32×\\timeswidths\) reconstruct our instruct\-model, quantized, final\-position activations too poorly to decompose the transferred direction \(variance explained≤\\leq0\.49, width\-invariant\); target\-distribution SAEs are future work\.

#### Activation steering \(Appendix[B](https://arxiv.org/html/2607.23458#A2)\)\.

The direction isd=\(μunf−μfaith\)/∥μunf−μfaith∥d=\(\\mu\_\{\\mathrm\{unf\}\}\-\\mu\_\{\\mathrm\{faith\}\}\)/\\lVert\\mu\_\{\\mathrm\{unf\}\}\-\\mu\_\{\\mathrm\{faith\}\}\\rVert, computed from train\-split final\-position activations at the probe’s best layer;σ\\sigmais the standard deviation of train projections ontodd; the intervention addsα​σ​d\\alpha\\sigma dto the residual stream at that layer at the answer\-readout position; the control is a random unit direction applied at the sameα​σ\\alpha\\sigmamagnitude\.

## Appendix HPer\-Cell Heterogeneity and AUPRC

![Refer to caption](https://arxiv.org/html/2607.23458v1/x5.png)Figure 6:Per\-cell \(signal×\\timesmodel×\\timesdomain\) AUROC with bootstrap 95% CIs, by regime; NLI is available benchmark\-wide \(all four models\), answer\-tracing for the two open models\. Cells withn<10n\{<\}10or a single class are omitted; per\-cellnnis small, so individual CIs are wide\. The incorrect\-regime cells scatter around chance across models and domains, including the closed models; the correct\-regime cells lean above chance across domains and in the closed models as well\.Table 8:AUPRC robustness for the Table[2](https://arxiv.org/html/2607.23458#S5.T2)comparisons \(prefix instability behaves like answer\-tracing: 0\.518/0\.385\)\. Random baseline==prevalence\. In the incorrect regime AUPRC matches prevalence \(consistent with the null\); in the correct regime it reaches 1\.7–1\.9×\\timesprevalence, so the moderate AUROCs are not masking failed minority\-class retrieval\.
## Appendix IGRACE Preliminary Replication

On the 40\-example public sample of GRACE\(Pham et al\.,[2026](https://arxiv.org/html/2607.23458#bib.bib17)\)\(8 unfaithful steps\), NLI step\-support shows no above\-chance discrimination \(0\.51–0\.58\), directionally consistent with our FaithCoT results but underpowered; the full evaluation set was not released in time\.