Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States

# Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
Source: [https://arxiv.org/html/2606.02907](https://arxiv.org/html/2606.02907)
###### Abstract

Linear probing of large language model \(LLM\) hidden states is widely used to claim that models learn distinct representations for different reasoning types\. We test this by probing Qwen3\-14B on three benchmarks spanning the classical trichotomy: LogiQA 2\.0 \(deductive\), ARC\-Challenge \(inductive\), andα\\alphaNLI \(abductive\)\. At layer 32 of 40, linear probes achieve 100% cross\-validated accuracy with well\-separated geometry \(intrinsic dimensionalities: 20\.6, 28\.5, 33\.6; convex hull contamination≤\\leq1\.5%\)\. However, this separation is entirely driven by format confounds\. Residualizing source identity, option count, and response length reduces accuracy to chance\. Trace\-anchor similarity indicates largely shared reasoning across tasks \(42\.5% agreement vs\. 33\.3% chance\), and causal steering with random controls \(n=20n=20\) shows no functional link between geometry and reasoning mode \(p=0\.286p=0\.286\)\. Thus, high probe accuracy reflects task format rather than computational structure, motivating routine format deconfounding in mechanistic interpretability\.

## 1Introduction

Large language models \(LLMs\) have demonstrated remarkable performance across tasks requiring deductive, inductive, and abductive reasoning\(Brownet al\.,[2020](https://arxiv.org/html/2606.02907#bib.bib1); Weiet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib2); Yanget al\.,[2025](https://arxiv.org/html/2606.02907#bib.bib3)\)\. A fundamental question for understanding these systems is whether they developdistinct internal computational strategiesfor different reasoning modes, or whether they apply a uniform approach regardless of task type\. Answering this question has direct implications for how we evaluate, interpret, and improve logical reasoning in LLMs—a central concern of the research community\(Huang and Chang,[2023](https://arxiv.org/html/2606.02907#bib.bib21); Ahnet al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib20)\)\. Linear probing—training a linear classifier on frozen hidden states to predict a target property—has become the standard tool for investigating such internal structure\(Alain and Bengio,[2018](https://arxiv.org/html/2606.02907#bib.bib4); Belinkovet al\.,[2017](https://arxiv.org/html/2606.02907#bib.bib5); Conneauet al\.,[2018](https://arxiv.org/html/2606.02907#bib.bib6)\)\. When probes achieve high accuracy at predicting reasoning type from hidden states, the standard interpretation is that the model has developed geometrically separable representations for each reasoning mode\(Liet al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib7); Cosentino and Shekkizhar,[2024](https://arxiv.org/html/2606.02907#bib.bib8)\)\. This interpretation underpins a growing body of mechanistic interpretability work that attempts to identify “reasoning circuits” within transformer architectures\(Olssonet al\.,[2022](https://arxiv.org/html/2606.02907#bib.bib9); Nandaet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib10)\)\. However, this interpretation rests on an assumption that is rarely tested: that the probe is detectingreasoning\-relevantstructure rather thansuperficial featurescorrelated with the reasoning label\. When different reasoning modes are sourced from different datasets—as is standard practice in multi\-task reasoning evaluation\(Liuet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib11); Bhagavatulaet al\.,[2020](https://arxiv.org/html/2606.02907#bib.bib12); Clarket al\.,[2018](https://arxiv.org/html/2606.02907#bib.bib13)\)—the hidden states necessarily encode distributional differences in vocabulary, prompt structure, and formatting that are perfectly confounded with the reasoning label\.

#### Contributions\.

1. 1\.Format confound decomposition\.We introduce a residual analysis pipeline that regresses out format features \(source identity, option count, response length\) from hidden states\. Probe accuracy drops from 100% to chance level—demonstrating that the entire separation is format\-driven \(Section[5\.2](https://arxiv.org/html/2606.02907#S5.SS2)\)\.
2. 2\.Trace\-mode agreement analysis\.We show the model achieves 86% accuracy across all reasoning types while exhibiting only 42\.5% trace\-mode agreement \(vs\. 33\.3% chance\), indicating it does not adapt its reasoning strategy to task type \(Section[5\.3](https://arxiv.org/html/2606.02907#S5.SS3)\)\.
3. 3\.Causal controls with random baselines\.We conduct steering\-vector experiments with random\-direction controls \(n=20n=20\) confirming that observed geometric structure is not causally linked to reasoning mode selection \(p=0\.286p=0\.286; Section[5\.4](https://arxiv.org/html/2606.02907#S5.SS4)\)\.
4. 4\.Methodological recommendations\.We propose that format deconfounding and random\-direction controls should be standard practice for probing\-based interpretability of reasoning\.

## 2Related Work

#### Logical reasoning in LLMs\.

The classical reasoning trichotomy—deduction, induction, and abduction —has received substantial attention in the LLM evaluation literature\. Deductive benchmarks include LogiQA\(Liuet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib11)\)and FOLIO\(Hanet al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib15)\); inductive reasoning is assessed through ARC\(Clarket al\.,[2018](https://arxiv.org/html/2606.02907#bib.bib13)\)and analogy tasks\(Webbet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib18)\); and abductive benchmarks includeα\\alphaNLI\(Bhagavatulaet al\.,[2020](https://arxiv.org/html/2606.02907#bib.bib12)\)and AbductionRules\(Younget al\.,[2022](https://arxiv.org/html/2606.02907#bib.bib19)\)\. While LLMs perform well on individual benchmarks, systematic comparison ofhowthey reason across types remains limited\. Critically, all such comparisons use separate datasets per reasoning mode—the exact design that creates the confound we identify\.

#### Linear probing and its pitfalls\.

Linear probes were introduced to assess whether neural networks develop linearly accessible representations\(Alain and Bengio,[2018](https://arxiv.org/html/2606.02907#bib.bib4); Belinkovet al\.,[2017](https://arxiv.org/html/2606.02907#bib.bib5)\)\. The technique has been extended to probe for syntactic structure\(Hewitt and Manning,[2019](https://arxiv.org/html/2606.02907#bib.bib22)\), factual knowledge\(Menget al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib24)\), and reasoning\-related properties\(Liet al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib7); Marks and Tegmark,[2023](https://arxiv.org/html/2606.02907#bib.bib26)\)\. However,Hewitt and Liang \([2019](https://arxiv.org/html/2606.02907#bib.bib27)\)andBenotti and Blackburn \([2021](https://arxiv.org/html/2606.02907#bib.bib28)\)cautioned that probe accuracy can reflect probe complexity rather than representation quality\. Our work extends this critique to the reasoning domain by showing thatperfectprobe accuracy can arise from task format alone\.

#### Causal methods in interpretability\.

Activation patching\(Viget al\.,[2020](https://arxiv.org/html/2606.02907#bib.bib34); Menget al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib24)\), steering vectors\(Turneret al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib35); Liet al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib7)\), and representation engineering\(Zouet al\.,[2025](https://arxiv.org/html/2606.02907#bib.bib36)\)establish causal links between representations and behavior\. We contributerandom\-direction controls—testing whether targeted steering outperforms random perturbations of equal magnitude—which is absent from most prior steering studies but essential for establishing directionality\.

## 3Methodology

Our pipeline consists of five stages: \(1\) multi\-source dataset construction, \(2\) inference with hidden\-state extraction, \(3\) layer\-wise linear probing with manifold geometry, \(4\) format confound analysis, and \(5\) causal steering with random\-direction controls\. All hyperparameters are either derived from the data or set by the experimental design—no values are hand\-tuned\.

### 3\.1Multi\-Source Reasoning Dataset

We construct a balanced three\-class dataset \(N=750N=750, 250 per class\) by sampling from benchmarks designed for each classical reasoning mode:

- •Deductive:LogiQA 2\.0\(Liuet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib11)\)—formal logical reasoning requiring rule application and conditional reasoning\. Four\-choice format with passage context\.
- •Inductive:ARC\-Challenge\(Clarket al\.,[2018](https://arxiv.org/html/2606.02907#bib.bib13)\)— science questions requiring generalization from observed patterns\. Four\-choice format\.
- •Abductive:α\\alphaNLI\(Bhagavatulaet al\.,[2020](https://arxiv.org/html/2606.02907#bib.bib12)\)—given two observations, select the hypothesis that best explains them\. Two\-choice format\.

Reasoning\-mode labels are assigned bydataset provenance—the intended reasoning type of each benchmark—not by post\-hoc classification\. This multi\-source design deliberately mirrors standard practice in reasoning evaluation\. We acknowledge that the benchmark\-to\-reasoning\-mode mapping is imperfect—ARC questions may involve a mix of reasoning types—but note that this imperfectionstrengthensour argument: if the mapping is noisy, the fact that probes still achieve 100% accuracy further suggests they detect source identity rather than reasoning modeSahooet al\.\([2026](https://arxiv.org/html/2606.02907#bib.bib41)\)\.

### 3\.2Model and Inference

We evaluate Qwen3\-14B\(Yanget al\.,[2025](https://arxiv.org/html/2606.02907#bib.bib3)\), a 14\-billion parameter decoder\-only transformer withL=40L=40layers and hidden dimensiond=5120d=5120, loaded inbfloat16\. For each task, we construct a uniform prompt \(Appendix[A](https://arxiv.org/html/2606.02907#A1)\) instructing step\-by\-step reasoning with a final answer in tags\. We use greedy decoding with a budget of 2048 tokens\. Qwen3\-14B is a hybrid thinking model that generates internal<think\>…\\ldots</think\>reasoning blocks before producing its final answer\. We setDISABLE\_THINKING=Trueand strip these blocks from all generated text before analysis\. All hidden states, reasoning traces, and output confidence scores therefore correspond to the model’s*non\-thinking*inference mode\. This is a deliberate methodological control: thinking\-mode traces introduce mode\-specific verbalisation structure that would itself confound hidden\-state geometry\. Non\-thinking mode isolates input\-driven representation from output\-driven style\.

For each task, we extract: \(i\) hidden states𝐡i\(ℓ\)∈ℝd\\mathbf\{h\}\_\{i\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{d\}at the last input token for every layerℓ∈\{0,…,L\}\\ell\\in\\\{0,\\ldots,L\\\}; \(ii\) generated text𝐲i\\mathbf\{y\}\_\{i\}with predicted answer and reasoning trace; and \(iii\) output confidencecic\_\{i\}, the geometric mean token probability\. Only correctly answered tasks are used for geometric analysis\.

### 3\.3Layer\-Wise Linear Probing

At each layerℓ\\ell, we train a linear probe \(logistic regression,L2L\_\{2\}regularization,C=1\.0C\\\!=\\\!1\.0\) to predict the reasoning\-mode labelyi∈\{D,I,A\}y\_\{i\}\\in\\\{\\text\{D\},\\text\{I\},\\text\{A\}\\\}from𝐡i\(ℓ\)\\mathbf\{h\}\_\{i\}^\{\(\\ell\)\}:

y^i=argmaxk\(𝐖\(ℓ\)𝐡i\(ℓ\)\+𝐛\(ℓ\)\)k\\hat\{y\}\_\{i\}=\\arg\\max\_\{k\}\\left\(\\mathbf\{W\}^\{\(\\ell\)\}\\mathbf\{h\}\_\{i\}^\{\(\\ell\)\}\+\\mathbf\{b\}^\{\(\\ell\)\}\\right\)\_\{k\}\(1\)evaluated via stratified 5\-fold cross\-validation\. The best layerℓ∗\\ell^\{\*\}is selected by maximum accuracy\. We also compute manifold geometry atℓ∗\\ell^\{\*\}: intrinsic dimensionality via TwoNN\(Faccoet al\.,[2017](https://arxiv.org/html/2606.02907#bib.bib32)\), local curvature via neighborhood SVD, inter\-mode separation ratios, and KNN\-based hull contamination\. Full details are in Appendix[C](https://arxiv.org/html/2606.02907#A3)\.

### 3\.4Format Confound Analysis

The central methodological contribution is a four\-stage pipeline to distinguish format\-based from reasoning\-based probe accuracy:

#### \(i\) Source prediction\.

An identical linear probe predicts dataset source \(LogiQA, ARC,α\\alphaNLI\) from hidden states\. If source accuracy≈\\approxmode accuracy, the probe cannot distinguish between the two labels\.

#### \(ii\) Option\-count probe\.

Logistic regression using only the number of answer options \(2 vs\. 4\) as input, testing whether this single scalar partially separates modes\.

#### \(iii\) Format\-controlled comparison\.

We restrict to 4\-choice tasks only \(LogiQA \+ ARC\) and re\-evaluate probes\. If separation persists, vocabulary or style differences beyond option count contribute\.

#### \(iv\) Residual analysis\.

We construct a format feature vector𝐟i=\[sourceone\-hot,noptions,\|𝐲i\|\]\\mathbf\{f\}\_\{i\}=\[\\text\{source\}\_\{\\text\{one\-hot\}\},n\_\{\\text\{options\}\},\|\\mathbf\{y\}\_\{i\}\|\]and fit Ridge regression to predict hidden states from format features\. The residual𝐫i=𝐡i\(ℓ∗\)−𝐡^i\\mathbf\{r\}\_\{i\}=\\mathbf\{h\}\_\{i\}^\{\(\\ell^\{\*\}\)\}\-\\hat\{\\mathbf\{h\}\}\_\{i\}removes all linear format information\. We then probe residuals for both mode and source\. If residual probe accuracy≈\\approxchance, the original separation is entirely format\-driven\.

### 3\.5Trace\-Mode Agreement

Independent of probing, we measure whether the model’sreasoning behaviormatches the intended mode\. We define anchor descriptions for each mode capturing observable trace behaviors \(e\.g\., “applies a known rule step\-by\-step” for deductive; full anchors in Appendix[B](https://arxiv.org/html/2606.02907#A2)\)\. Anchors and traces are embedded using the model’s last\-layer hidden states\. Each trace is assigned to the mode with highest cosine similarity\. Agreement significantly above chance \(1/K=33\.3%1/K=33\.3\\%\) would indicate the model adapts its strategy to task type\.

### 3\.6Causal Steering with Random\-Direction Controls

To test whether geometric separation iscausallylinked to reasoning, we apply activation steering\(Turneret al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib35)\)\. For each mode pair\(ms,mt\)\(m\_\{s\},m\_\{t\}\), the steering vector is𝐯^s→t=\(𝝁t−𝝁s\)/‖𝝁t−𝝁s‖\\hat\{\\mathbf\{v\}\}\_\{s\\to t\}=\(\\boldsymbol\{\\mu\}\_\{t\}\-\\boldsymbol\{\\mu\}\_\{s\}\)/\\\|\\boldsymbol\{\\mu\}\_\{t\}\-\\boldsymbol\{\\mu\}\_\{s\}\\\|\. During generation, a forward hook at layerℓ∗\\ell^\{\*\}addsα∗⋅𝐯^s→t\\alpha^\{\*\}\\cdot\\hat\{\\mathbf\{v\}\}\_\{s\\to t\}to all positions\. The magnitudeα∗\\alpha^\{\*\}is learned via coherence sweep with Otsu thresholding \(Appendix[D](https://arxiv.org/html/2606.02907#A4)\)\.

#### Random\-direction controls\.

We sampleNrandN\_\{\\text\{rand\}\}random directions, whereNrand=max⁡\(5,min⁡\(20,2⋅nsteered\)\)N\_\{\\text\{rand\}\}=\\max\(5,\\min\(20,2\\cdot n\_\{\\text\{steered\}\}\)\)is derived from the number of steered evaluation tasks \(capped at 20\)\. In practiceNrand=20N\_\{\\text\{rand\}\}=20whennsteered≥10n\_\{\\text\{steered\}\}\\geq 10, which holds in all reported experiments\. For each trialii, we sample𝐯rand∼𝒩​\(𝟎,𝐈d\)\\mathbf\{v\}\_\{\\text\{rand\}\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\mathbf\{I\}\_\{d\}\), normalize to unit length, and apply the sameα∗\\alpha^\{\*\}at the same layer\. This tests whether effects are specific to the centroid\-difference direction or arise from any perturbation of equal magnitude\. Empiricalpp\-values use a Laplace correction:p=\(k\+1\)/\(Nrand\+1\)p=\(k\+1\)/\(N\_\{\\text\{rand\}\}\+1\)wherekkis the number of random directions matching or exceeding the targeted metric\.

#### Conflict injection\.

We simultaneously inject two steering vectors toward different modes:𝐡~i\(ℓ∗\)=𝐡i\(ℓ∗\)\+α∗⋅\(𝐯^1\+𝐯^2\)\\tilde\{\\mathbf\{h\}\}\_\{i\}^\{\(\\ell^\{\*\}\)\}=\\mathbf\{h\}\_\{i\}^\{\(\\ell^\{\*\}\)\}\+\\alpha^\{\*\}\\cdot\(\\hat\{\\mathbf\{v\}\}\_\{1\}\+\\hat\{\\mathbf\{v\}\}\_\{2\}\)\. Random\-pair controls \(n=10n\\\!=\\\!10\) inject pairs of random unit vectors for comparison\. Details are in Appendix[E](https://arxiv.org/html/2606.02907#A5)\.

## 4Experimental Setup

Model and hardware\.Qwen3\-14B\(Yanget al\.,[2025](https://arxiv.org/html/2606.02907#bib.bib3)\), 40 layers,d=5120d\\\!=\\\!5120,bfloat16\. Single NVIDIA GH200 \(480 GB\); model footprint 29\.5 GB; batch size 8\. The code includes an automatic fallback to Qwen3\-4B if available VRAM falls below 64 GB\. Given the 480 GB capacity and 29\.5 GB footprint, this fallback did not trigger in any reported experiment; all results are from Qwen3\-14B\.Dataset\.750 tasks: 250 LogiQA 2\.0 \(deductive\), 250 ARC\-Challenge \(inductive\), 250α\\alphaNLI \(abductive\)\. Balanced by construction\. Figure[1](https://arxiv.org/html/2606.02907#S4.F1)shows per\-source accuracy and class balance\.Derived hyperparameters\.KNN neighborhoodk=25k=25; CV foldsF=5F=5; steeringα∗\\alpha^\{\*\}learned via coherence sweep; all thresholds derived from data distributions\. Full details in Appendix[F](https://arxiv.org/html/2606.02907#A6)\.Statistical testing\.Bootstrap confidence intervals \(nboot=2000n\_\{\\text\{boot\}\}\\\!=\\\!2000, 95% CI\)\. Permutation tests \(nperm=5000n\_\{\\text\{perm\}\}\\\!=\\\!5000\)\. Empiricalpp\-values for steering directionality\. Cohen’sddfor all control comparisons\.

![Refer to caption](https://arxiv.org/html/2606.02907v1/x1.png)Figure 1:Dataset statistics\.Accuracy by source dataset, overall model accuracy \(86%\), and class balance across reasoning modes\. The dataset is class\-balanced \(250 per mode\), while source\-wise accuracy reveals substantial variation in task difficulty \(LogiQA: 73\.2%, ARC: 93\.6%,α\\alphaNLI: 91\.2%\)\.
## 5Results

We present results in four stages\. First, we establish that linear probes achieve perfect separation of reasoning modes \(Section[5\.1](https://arxiv.org/html/2606.02907#S5.SS1)\)\. Second, we show this separation is entirely explained by format confounds \(Section[5\.2](https://arxiv.org/html/2606.02907#S5.SS2)\)\. Third, we demonstrate that the model’s reasoning behavior does not vary by mode \(Section[5\.3](https://arxiv.org/html/2606.02907#S5.SS3)\)\. Fourth, we confirm through causal experiments that the geometry is not functionally linked to reasoning \(Section[5\.4](https://arxiv.org/html/2606.02907#S5.SS4)\)\.

### 5\.1Probes Achieve Perfect Separation

Figure[2](https://arxiv.org/html/2606.02907#S5.F2)shows cross\-validated probe accuracy across all 41 layers\. Probe accuracy is near chance in early layers and increases monotonically, reaching100% balanced accuracy at layer 32\(80% of network depth\)\. All three classes achieve perfect precision, recall, and F1\. The permutation test confirms this is significantly above chance \(p<0\.0002p<0\.0002,nperm=5000n\_\{\\text\{perm\}\}=5000\)\. Manifold geometry at layer 32 \(Figure[3](https://arxiv.org/html/2606.02907#S5.F3)\) reveals striking separation\. The three reasoning modes occupy distinct regions of representation space, with mode\-specific intrinsic dimensionalities: deductive manifolds haved^ID=20\.6\\hat\{d\}\_\{\\text\{ID\}\}=20\.6, inductive28\.528\.5, and abductive33\.633\.6\. Separation ratios exceed 1\.0 for all pairs, and hull contamination is≤1\.5%\\leq 1\.5\\%\. UMAP visualization shows three cleanly separated clusters\. On their face, these results would constitute strong evidence for mode\-specific internal representations\.

![Refer to caption](https://arxiv.org/html/2606.02907v1/x2.png)Figure 2:Layer\-wise probe accuracy\.Cross\-validated accuracy across network depth peaks at layer 32 with 100% balanced accuracy\. Information about reasoning\-mode labels is weak in early layers and becomes perfectly separable in late layers\.![Refer to caption](https://arxiv.org/html/2606.02907v1/x3.png)Figure 3:Manifold geometry at layer 32\.\(Top\-left\) UMAP shows three separated clusters\. \(Top\-right\) Mode\-specific intrinsic dimensionalities differ substantially\. \(Bottom\-left\) Curvature distributions differ across modes\. \(Bottom\-right\) Compactness and hull contamination quantify clean separation\.All of these properties are explained by format confounds\(Section[5\.2](https://arxiv.org/html/2606.02907#S5.SS2)\)\.The Apparent ResultAt layer 32, reasoning modes are perfectly linearly separable \(100% CV accuracy\) with distinct manifold geometry—exactly the kind of evidence typically cited for mode\-specific internal representations\. The remainder of this paper shows this evidence is artifactual\.

### 5\.2The Separation is Entirely Format\-Driven

We now apply the four\-stage confound analysis from Section[3\.4](https://arxiv.org/html/2606.02907#S3.SS4)\.

#### Stage 1: Source≡\\equivMode\.

A linear probe predictingdataset source\(LogiQA, ARC,α\\alphaNLI\) from layer\-32 hidden states also achieves100% accuracy\. Since reasoning\-mode labels and source labels are in perfect correspondence \(by design of multi\-source evaluation\), these probes are informationally equivalent\. The probe cannot distinguish whether it has learned “this is deductive reasoning” or “this came from LogiQA\.”

#### Stage 2: Option count alone partially separates\.

A logistic regression using only the number of answer options \(noptions∈\{2,4\}n\_\{\\text\{options\}\}\\in\\\{2,4\\\}\) achieves 33\.3% mode accuracy—exactly the prior for theα\\alphaNLI class\. This confirms that the 2\-choice vs\. 4\-choice structural difference provides a trivially exploitable feature\.

#### Stage 3: Format\-controlled comparison\.

Restricting to 4\-choice tasks only \(LogiQA \+ ARC,n=500n=500\), the mode probe still achieves near\-perfect accuracy, indicating that vocabulary, syntax, and domain differences between LogiQA and ARC—beyond option count—are sufficient for separation\.

#### Stage 4: Residual analysis\.

This is the key result\. After Ridge regression removes linear format information \(source one\-hot, option count, response length\) from hidden states, probing the residuals yields:

Table 1:Probe accuracy before and after format deconfounding\.Residual analysis reduces mode\-prediction accuracy to chance, demonstrating the separation is entirely format\-driven\.As Table[1](https://arxiv.org/html/2606.02907#S5.T1)shows, residual probe accuracy drops toapproximately chance level—indistinguishable from the 33\.3% baseline\. The entire linear separability of reasoning modes is explained by format features\. No reasoning\-specific geometry remains after deconfounding\.

Methodological WarningResidual analysis reduces probe accuracy from 100% to chance\. The “reasoning\-mode geometry” in the hidden states is entirely a task\-format artifact\. This result generalizes toanymulti\-source probing setup where reasoning labels are confounded with dataset source\.

### 5\.3The Model Uses a Uniform Reasoning Strategy

Independent of the probing analysis, we test whether the model’sobservable reasoning behaviorvaries by mode\. Figure[4](https://arxiv.org/html/2606.02907#S5.F4)shows the trace\-mode agreement results\. The model achieves strong overall accuracy \(86%\) across all three task types, yet exhibits only42\.5% trace\-mode agreement\(vs\. 33\.3% expected by chance\)\. That is, when we classify each reasoning trace by its similarity to mode\-specific anchor descriptions, the predicted reasoning mode matches the intended mode only slightly above chance\. This finding has a direct interpretation: the model does not substantially changehowit reasons when moving between deductive, inductive, and abductive tasks\. It reasons well across all types—but uses a largely uniform strategy\. This behavioral result converges with the probing result: there is no distinct internal mode because there is no distinct external behavior\.

![Refer to caption](https://arxiv.org/html/2606.02907v1/x4.png)Figure 4:Trace\-mode agreement\.\(Left\) Projection into the reasoning\-mode simplex shows weak clustering by intended mode\. \(Middle\) Agreement between predicted and intended mode is 42\.5%, only marginally above the 33\.3% chance level\. \(Right\) Dominant\-mode scores are broadly distributed, indicating no strong mode preference\.
### 5\.4Causal Steering Confirms No Functional Link

![Refer to caption](https://arxiv.org/html/2606.02907v1/x5.png)Figure 5:Steering experiments\.\(Top\-left\) Accuracy before and after steering\. \(Top\-right\) Post\-steering mode distribution\. \(Bottom\-left\) Coherence sweep for optimalα∗\\alpha^\{\*\}\. \(Bottom\-right\) Targeted vs\. random steering shows no significant difference \(p=0\.286p=0\.286\)\.Converging EvidenceThree independent analyses—residual probing, trace\-mode agreement, and causal steering—all converge on the same conclusion: the geometric separation of reasoning modes in LLM hidden states reflects task format, not internal computational structure\.

Table 2:Steering results: targeted vs\. random directions\.Targeted steering does not significantly outperform random perturbations, indicating no mode\-specific causal role\.Our final analysis tests whether the geometric separation—despite being format\-driven—might still have acausalrelationship to reasoning behavior\. If steering along the centroid\-difference direction between modes produces mode\-specific behavioral changes that random directions do not, this would suggest the geometry carries some functional role\. Figure[5](https://arxiv.org/html/2606.02907#S5.F5)summarizes the steering results\. The targeted steering vector producescomparableeffects to random\-direction perturbations of equal magnitude\. The empiricalpp\-value of 0\.286 indicates that the targeted direction is not significantly better than random perturbations\. Similarly, conflict injection \(two opposing steering vectors\) produces 100% coherence collapse forbothtargeted and random conflict pairs, confirming the effect is magnitude\-based, not direction\-specific\.

## 6Discussion

### 6\.1What the Model Actually Does

Table 3:Model accuracy by dataset source\.The model achieves 86% accuracy across all three task types \(Table[3](https://arxiv.org/html/2606.02907#S6.T3)\), demonstrating genuine reasoning capability\. However, it appears to deploy a largelyuniformreasoning strategy: the trace\-mode agreement of 42\.5% is only marginally above the 33\.3% chance level\. The model solves deductive, inductive, and abductive tasks—but likely through a general\-purpose mechanism rather than mode\-specific circuits\. This raises an important question for the workshop community: if LLMs use a uniform strategy across reasoning types, should we expect training on one reasoning type to transfer to others? And conversely, should failures in one mode be addressed by mode\-specific interventions, or by improving the general mechanism?

### 6\.2Why This Matters for Mechanistic Interpretability

Our findings challenge a common inferential pattern in the interpretability literature: \(1\) train a linear probe on hidden states, \(2\) observe high accuracy, \(3\) conclude the model has learned a distinct internal representation\. This pattern is valid only if the high accuracy cannot be attributed to confounds\. In the reasoning domain, the standard practice of using different benchmarks for different reasoning types creates aperfectconfound between reasoning label and dataset source\. This concern is not specific to our choice of model or datasets\. Any multi\-source probing setup where reasoning labels co\-vary with dataset source will exhibit the same confound\. The issue is structural: it is a property of the experimental design, not of the model\.

Recommendations for the CommunityAlways report source\-prediction accuracyalongside mode\-prediction accuracy when probing across datasets\. If they are equal, the probe may be detecting source, not mode\.Include residual analysisas a standard control: regress out format features and re\-probe the residuals\.Use random\-direction controlsfor all steering\-vector experiments to establish directionality rather than mere perturbation sensitivity\.Design format\-controlled benchmarkswhere deductive, inductive, and abductive tasks share identical surface format, option count, and vocabulary distribution\.

## 7Limitations and Future Work

Single model\.All experiments use Qwen3\-14B\. While the format confound is a property of the experimental design \(not the model\), replication across model families—Llama\(Touvronet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib38)\), Mistral\(Jianget al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib39)\), GPT\-4\(OpenAIet al\.,[2024](https://arxiv.org/html/2606.02907#bib.bib40)\)—is necessary to assess generality of the uniform\-strategy finding\.Conservative residual analysis\.Ridge regression with source one\-hot features can explain nearly all variance, potentially removing genuine signal alongside format information\. A less conservative approach—regressing out only option count and response length \(not source\)—would test whether non\-source format features alone explain the separation\. We leave this intermediate analysis to future work\.Trace\-anchor limitations\.Our trace\-mode agreement analysis relies on cosine similarity to hand\-crafted anchor descriptions, which may miss subtle reasoning differences\. Fine\-grained behavioral analysis \(e\.g\., counting explicit syllogisms, hypothesis eliminations, or pattern enumerations\) would provide stronger evidence\.Two\-choice vs\. four\-choice confound\.The structural difference betweenα\\alphaNLI \(2\-choice\) and the other datasets \(4\-choice\) creates an obvious confound\. Future benchmarks should enforce uniform format across reasoning types\. The LogiQA 2\.0 NLI variant\(Liuet al\.,[2023](https://arxiv.org/html/2606.02907#bib.bib11)\)takes steps in this direction\.Small causal experiment scale\.Steering evaluation uses up to 15 previously wrong tasks \(STEERING\_EVAL\_LIMIT=15\\texttt\{STEERING\\\_EVAL\\\_LIMIT\}=15\) drawn from a pool first capped at 30 wrong results and then filtered to those whose dominant predicted reasoning mode matches the source mode\. The reportedp=0\.286p=0\.286corresponds toNrand=20N\_\{\\text\{rand\}\}=20random directions with Laplace correction, givingp=\(5\+1\)/\(20\+1\)≈0\.286p=\(5\+1\)/\(20\+1\)\\approx 0\.286if 5 of 20 random directions match or exceed the targeted accuracy recovery\. Larger\-scale evaluation would provide tighter bounds on effect size\.Non\-thinking inference mode\.We disable thinking \(DISABLE\_THINKING=True\) deliberately: thinking\-mode traces introduce mode\-specific chain\-of\-thought structure that would itself constitute a format confound\. Our results therefore establish a lower bound—input format alone suffices for perfect probe separation\. Whether thinking\-mode activations exhibit additional geometry is an open extension\.

Future directions\.Three extensions emerge: \(i\)format\-controlled reasoning benchmarkswith identical surface format across modes; \(ii\)within\-dataset probingfor reasoning subtypes within format\-homogeneous benchmarks \(e\.g\., LogiQA subtypes\); and \(iii\)multi\-model replicationof the full pipeline to test whether the uniform\-strategy finding is universal\.

## 8Conclusion

We set out to determine whether LLMs develop geometrically distinct internal representations for deductive, inductive, and abductive reasoning\. Using standard multi\-source evaluation, we found that linear probes achieve perfect accuracy at separating reasoning modes, with compelling manifold geometry\. However, systematic format confound analysis overturns this conclusion entirely: residual analysis reduces probe accuracy to chance, trace\-mode agreement is near random, and causal steering shows no mode\-specific directionality\. These results carry a clear methodological message:high linear probe accuracy is not sufficient evidence of internal computational structure\.When reasoning\-mode labels are confounded with dataset source—as is standard practice—probes detect format, not function\. The model reasons well across all three task types \(86% accuracy\), but it appears to do so using a largely uniform strategy rather than distinct computational modes\. Understanding what that uniform strategy is, and whether it can be steered toward genuinely mode\-specific reasoning, remains an important open question for the logical reasoning community\.

## References

- Large language models for mathematical reasoning: progresses and challenges\.External Links:2402\.00157,[Link](https://arxiv.org/abs/2402.00157)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- M\. Akhtar, A\. Reuel, P\. Soni, S\. Ahuja, P\. S\. Ammanamanchi, R\. Rawal, V\. Zouhar, S\. Yadav, C\. Whitehouse, D\. Ki, J\. Mickel, L\. Choshen, M\. Šuppa, J\. Batzner, J\. Chim, J\. Sania, Y\. Long, H\. A\. Rahmani, C\. Knight, Y\. Nan, J\. Raj, Y\. Fan, S\. Singh, S\. Sahoo, E\. Habba, U\. Gohar, S\. Pawar, R\. Scholz, A\. Subramonian, J\. Ni, M\. Kochenderfer, S\. Koyejo, M\. Sachan, S\. Biderman, Z\. Talat, A\. Ghosh, and I\. Solaiman \(2026\)When ai benchmarks plateau: a systematic study of benchmark saturation\.External Links:2602\.16763,[Link](https://arxiv.org/abs/2602.16763)Cited by:[Appendix D](https://arxiv.org/html/2606.02907#A4.SS0.SSS0.Px1.p1.4)\.
- G\. Alain and Y\. Bengio \(2018\)Understanding intermediate layers using linear classifier probes\.External Links:1610\.01644,[Link](https://arxiv.org/abs/1610.01644)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Belinkov, N\. Durrani, F\. Dalvi, H\. Sajjad, and J\. Glass \(2017\)What do neural machine translation models learn about morphology?\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),R\. Barzilay and M\. Kan \(Eds\.\),Vancouver, Canada,pp\. 861–872\.External Links:[Link](https://aclanthology.org/P17-1080/),[Document](https://dx.doi.org/10.18653/v1/P17-1080)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Benotti and P\. Blackburn \(2021\)Grounding as a collaborative process\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,P\. Merlo, J\. Tiedemann, and R\. Tsarfaty \(Eds\.\),Online,pp\. 515–531\.External Links:[Link](https://aclanthology.org/2021.eacl-main.41/),[Document](https://dx.doi.org/10.18653/v1/2021.eacl-main.41)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1)\.
- C\. Bhagavatula, R\. L\. Bras, C\. Malaviya, K\. Sakaguchi, A\. Holtzman, H\. Rashkin, D\. Downey, S\. W\. Yih, and Y\. Choi \(2020\)Abductive commonsense reasoning\.External Links:1908\.05739,[Link](https://arxiv.org/abs/1908.05739)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px1.p1.1),[3rd item](https://arxiv.org/html/2606.02907#S3.I1.i3.p1.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei \(2020\)Language models are few\-shot learners\.External Links:2005\.14165,[Link](https://arxiv.org/abs/2005.14165)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? try arc, the ai2 reasoning challenge\.External Links:1803\.05457,[Link](https://arxiv.org/abs/1803.05457)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px1.p1.1),[2nd item](https://arxiv.org/html/2606.02907#S3.I1.i2.p1.1)\.
- A\. Conneau, G\. Kruszewski, G\. Lample, L\. Barrault, and M\. Baroni \(2018\)What you can cram into a single $&\!\#\* vector: probing sentence embeddings for linguistic properties\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),I\. Gurevych and Y\. Miyao \(Eds\.\),Melbourne, Australia,pp\. 2126–2136\.External Links:[Link](https://aclanthology.org/P18-1198/),[Document](https://dx.doi.org/10.18653/v1/P18-1198)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- R\. Cosentino and S\. Shekkizhar \(2024\)Reasoning in large language models: a geometric perspective\.External Links:2407\.02678,[Link](https://arxiv.org/abs/2407.02678)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- E\. Facco, M\. d’Errico, A\. Rodriguez, and A\. Laio \(2017\)Estimating the intrinsic dimension of datasets by a minimal neighborhood information\.Scientific Reports7\(1\),pp\. 12140\.External Links:[Document](https://dx.doi.org/10.1038/s41598-017-11873-y)Cited by:[Appendix C](https://arxiv.org/html/2606.02907#A3.SS0.SSS0.Px1.p1.1),[§3\.3](https://arxiv.org/html/2606.02907#S3.SS3.p1.7)\.
- S\. Han, H\. Schoelkopf, Y\. Zhao, Z\. Qi, M\. Riddell, W\. Zhou, J\. Coady, D\. Peng, Y\. Qiao, L\. Benson, L\. Sun, A\. Wardle\-Solano, H\. Szabo, E\. Zubova, M\. Burtell, J\. Fan, Y\. Liu, B\. Wong, M\. Sailor, A\. Ni, L\. Nan, J\. Kasai, T\. Yu, R\. Zhang, A\. R\. Fabbri, W\. Kryscinski, S\. Yavuz, Y\. Liu, X\. V\. Lin, S\. Joty, Y\. Zhou, C\. Xiong, R\. Ying, A\. Cohan, and D\. Radev \(2024\)FOLIO: natural language reasoning with first\-order logic\.External Links:2209\.00840,[Link](https://arxiv.org/abs/2209.00840)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Hewitt and P\. Liang \(2019\)Designing and interpreting probes with control tasks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 2733–2743\.External Links:[Link](https://aclanthology.org/D19-1275/),[Document](https://dx.doi.org/10.18653/v1/D19-1275)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Hewitt and C\. D\. Manning \(2019\)A structural probe for finding syntax in word representations\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 4129–4138\.External Links:[Link](https://aclanthology.org/N19-1419/),[Document](https://dx.doi.org/10.18653/v1/N19-1419)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Huang and K\. C\. Chang \(2023\)Towards reasoning in large language models: a survey\.External Links:2212\.10403,[Link](https://arxiv.org/abs/2212.10403)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. Sayed \(2023\)Mistral 7b\.External Links:2310\.06825,[Link](https://arxiv.org/abs/2310.06825)Cited by:[§7](https://arxiv.org/html/2606.02907#S7.p1.5)\.
- K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2024\)Inference\-time intervention: eliciting truthful answers from a language model\.External Links:2306\.03341,[Link](https://arxiv.org/abs/2306.03341)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Liu, J\. Liu, L\. Cui, Z\. Teng, N\. Duan, M\. Zhou, and Y\. Zhang \(2023\)LogiQA 2\.0—an improved dataset for logical reasoning in natural language understanding\.IEEE/ACM Transactions on Audio, Speech, and Language Processing31\(\),pp\. 2947–2962\.External Links:[Document](https://dx.doi.org/10.1109/TASLP.2023.3293046)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px1.p1.1),[1st item](https://arxiv.org/html/2606.02907#S3.I1.i1.p1.1),[§7](https://arxiv.org/html/2606.02907#S7.p1.5)\.
- S\. Marks and M\. Tegmark \(2023\)The geometry of truth: emergent linear structure in large language model representations of true/false datasets\.arXiv preprint arXiv:2310\.06824\.Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2023\)Locating and editing factual associations in gpt\.External Links:2202\.05262,[Link](https://arxiv.org/abs/2202.05262)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px3.p1.1)\.
- N\. Nanda, L\. Chan, T\. Lieberum, J\. Smith, and J\. Steinhardt \(2023\)Progress measures for grokking via mechanistic interpretability\.External Links:2301\.05217,[Link](https://arxiv.org/abs/2301.05217)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- C\. Olsson, N\. Elhage, N\. Nanda, N\. Joseph, N\. DasSarma, T\. Henighan, B\. Mann, A\. Askell, Y\. Bai, A\. Chen, T\. Conerly, D\. Drain, D\. Ganguli, Z\. Hatfield\-Dodds, D\. Hernandez, S\. Johnston, A\. Jones, J\. Kernion, L\. Lovitt, K\. Ndousse, D\. Amodei, T\. Brown, J\. Clark, J\. Kaplan, S\. McCandlish, and C\. Olah \(2022\)In\-context learning and induction heads\.Note:https://transformer\-circuits\.pub/2022/in\-context\-learning\-and\-induction\-heads/index\.htmlCited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- OpenAI, J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat, R\. Avila, I\. Babuschkin, S\. Balaji, V\. Balcom, P\. Baltescu, H\. Bao, M\. Bavarian, J\. Belgum, I\. Bello, J\. Berdine, G\. Bernadett\-Shapiro, C\. Berner, L\. Bogdonoff, O\. Boiko, M\. Boyd, A\. Brakman, G\. Brockman, T\. Brooks, M\. Brundage, K\. Button, T\. Cai, R\. Campbell, A\. Cann, B\. Carey, C\. Carlson, R\. Carmichael, B\. Chan, C\. Chang, F\. Chantzis, D\. Chen, S\. Chen, R\. Chen, J\. Chen, M\. Chen, B\. Chess, C\. Cho, C\. Chu, H\. W\. Chung, D\. Cummings, J\. Currier, Y\. Dai, C\. Decareaux, T\. Degry, N\. Deutsch, D\. Deville, A\. Dhar, D\. Dohan, S\. Dowling, S\. Dunning, A\. Ecoffet, A\. Eleti, T\. Eloundou, D\. Farhi, L\. Fedus, N\. Felix, S\. P\. Fishman, J\. Forte, I\. Fulford, L\. Gao, E\. Georges, C\. Gibson, V\. Goel, T\. Gogineni, G\. Goh, R\. Gontijo\-Lopes, J\. Gordon, M\. Grafstein, S\. Gray, R\. Greene, J\. Gross, S\. S\. Gu, Y\. Guo, C\. Hallacy, J\. Han, J\. Harris, Y\. He, M\. Heaton, J\. Heidecke, C\. Hesse, A\. Hickey, W\. Hickey, P\. Hoeschele, B\. Houghton, K\. Hsu, S\. Hu, X\. Hu, J\. Huizinga, S\. Jain, S\. Jain, J\. Jang, A\. Jiang, R\. Jiang, H\. Jin, D\. Jin, S\. Jomoto, B\. Jonn, H\. Jun, T\. Kaftan, Ł\. Kaiser, A\. Kamali, I\. Kanitscheider, N\. S\. Keskar, T\. Khan, L\. Kilpatrick, J\. W\. Kim, C\. Kim, Y\. Kim, J\. H\. Kirchner, J\. Kiros, M\. Knight, D\. Kokotajlo, Ł\. Kondraciuk, A\. Kondrich, A\. Konstantinidis, K\. Kosic, G\. Krueger, V\. Kuo, M\. Lampe, I\. Lan, T\. Lee, J\. Leike, J\. Leung, D\. Levy, C\. M\. Li, R\. Lim, M\. Lin, S\. Lin, M\. Litwin, T\. Lopez, R\. Lowe, P\. Lue, A\. Makanju, K\. Malfacini, S\. Manning, T\. Markov, Y\. Markovski, B\. Martin, K\. Mayer, A\. Mayne, B\. McGrew, S\. M\. McKinney, C\. McLeavey, P\. McMillan, J\. McNeil, D\. Medina, A\. Mehta, J\. Menick, L\. Metz, A\. Mishchenko, P\. Mishkin, V\. Monaco, E\. Morikawa, D\. Mossing, T\. Mu, M\. Murati, O\. Murk, D\. Mély, A\. Nair, R\. Nakano, R\. Nayak, A\. Neelakantan, R\. Ngo, H\. Noh, L\. Ouyang, C\. O’Keefe, J\. Pachocki, A\. Paino, J\. Palermo, A\. Pantuliano, G\. Parascandolo, J\. Parish, E\. Parparita, A\. Passos, M\. Pavlov, A\. Peng, A\. Perelman, F\. de Avila Belbute Peres, M\. Petrov, H\. P\. de Oliveira Pinto, Michael, Pokorny, M\. Pokrass, V\. H\. Pong, T\. Powell, A\. Power, B\. Power, E\. Proehl, R\. Puri, A\. Radford, J\. Rae, A\. Ramesh, C\. Raymond, F\. Real, K\. Rimbach, C\. Ross, B\. Rotsted, H\. Roussez, N\. Ryder, M\. Saltarelli, T\. Sanders, S\. Santurkar, G\. Sastry, H\. Schmidt, D\. Schnurr, J\. Schulman, D\. Selsam, K\. Sheppard, T\. Sherbakov, J\. Shieh, S\. Shoker, P\. Shyam, S\. Sidor, E\. Sigler, M\. Simens, J\. Sitkin, K\. Slama, I\. Sohl, B\. Sokolowsky, Y\. Song, N\. Staudacher, F\. P\. Such, N\. Summers, I\. Sutskever, J\. Tang, N\. Tezak, M\. B\. Thompson, P\. Tillet, A\. Tootoonchian, E\. Tseng, P\. Tuggle, N\. Turley, J\. Tworek, J\. F\. C\. Uribe, A\. Vallone, A\. Vijayvergiya, C\. Voss, C\. Wainwright, J\. J\. Wang, A\. Wang, B\. Wang, J\. Ward, J\. Wei, C\. Weinmann, A\. Welihinda, P\. Welinder, J\. Weng, L\. Weng, M\. Wiethoff, D\. Willner, C\. Winter, S\. Wolrich, H\. Wong, L\. Workman, S\. Wu, J\. Wu, M\. Wu, K\. Xiao, T\. Xu, S\. Yoo, K\. Yu, Q\. Yuan, W\. Zaremba, R\. Zellers, C\. Zhang, M\. Zhang, S\. Zhao, T\. Zheng, J\. Zhuang, W\. Zhuk, and B\. Zoph \(2024\)GPT\-4 technical report\.External Links:2303\.08774,[Link](https://arxiv.org/abs/2303.08774)Cited by:[§7](https://arxiv.org/html/2606.02907#S7.p1.5)\.
- S\. Sahoo, A\. Chadha, V\. Jain, and D\. Chaudhary \(2026\)The reasoning trap – logical reasoning as a mechanistic pathway to situational awareness\.External Links:2603\.09200,[Link](https://arxiv.org/abs/2603.09200)Cited by:[§3\.1](https://arxiv.org/html/2606.02907#S3.SS1.p3.1)\.
- H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. C\. Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. Scialom \(2023\)Llama 2: open foundation and fine\-tuned chat models\.External Links:2307\.09288,[Link](https://arxiv.org/abs/2307.09288)Cited by:[§7](https://arxiv.org/html/2606.02907#S7.p1.5)\.
- A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid \(2024\)Steering language models with activation engineering\.External Links:2308\.10248,[Link](https://arxiv.org/abs/2308.10248)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px3.p1.1),[§3\.6](https://arxiv.org/html/2606.02907#S3.SS6.p1.5)\.
- J\. Vig, S\. Gehrmann, Y\. Belinkov, S\. Qian, D\. Nevo, Y\. Singer, and S\. Shieber \(2020\)Investigating gender bias in language models using causal mediation analysis\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 12388–12401\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/92650b2e92217715fe312e6fa7b90d82-Paper.pdf)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px3.p1.1)\.
- T\. Webb, K\. J\. Holyoak, and H\. Lu \(2023\)Emergent analogical reasoning in large language models\.External Links:2212\.09196,[Link](https://arxiv.org/abs/2212.09196)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. Zhou \(2023\)Chain\-of\-thought prompting elicits reasoning in large language models\.External Links:2201\.11903,[Link](https://arxiv.org/abs/2201.11903)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§1](https://arxiv.org/html/2606.02907#S1.p1.1),[§3\.2](https://arxiv.org/html/2606.02907#S3.SS2.p1.3),[§4](https://arxiv.org/html/2606.02907#S4.p1.9)\.
- N\. Young, Q\. Bao, J\. Bensemann, and M\. Witbrock \(2022\)AbductionRules: training transformers to explain unexpected inputs\.pp\. 218–227\.External Links:[Link](https://aclanthology.org/2022.findings-acl.19/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.19)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. Hendrycks \(2025\)Representation engineering: a top\-down approach to ai transparency\.External Links:2310\.01405,[Link](https://arxiv.org/abs/2310.01405)Cited by:[§2](https://arxiv.org/html/2606.02907#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix APrompt Template

All tasks use the following uniform prompt template:

> ``` You are solving a logical reasoning problem. Read the context and question carefully. Think step by step. After your reasoning, put your final answer between <answer> and </answer> tags. Answer with ONLY the letter (A, B, C, or D). Context: {context} Question: {question} Options: (A) {option_a} (B) {option_b} ... Your reasoning and answer: ```

For ARC tasks where the context and question overlap, only the question field is displayed\. Option labels vary by dataset \(A–D for 4\-choice, A–B for 2\-choice\)\.

Note: although the prompt instructs “Think step by step,” all experiments run withDISABLE\_THINKING=True, which suppresses Qwen3\-14B’s internal<think\>chain\-of\-thought\. The step\-by\-step instruction therefore governs the*visible*output structure, not the model’s internal thinking pathway\.

## Appendix BAnchor Descriptions for Trace\-Mode Agreement

- •Deductive:“This reasoning applies a known rule or principle to reach a necessary conclusion\. It follows strict logical steps: if the premises are true, the conclusion must be true\. Syllogisms, modus ponens, modus tollens, conditional chains, contrapositive, logical necessity, formal proof steps\.”
- •Inductive:“This reasoning observes specific examples or patterns and generalizes to a broader rule\. It identifies regularities across instances and draws probable conclusions\. Pattern recognition, analogy, statistical generalization, enumeration of cases, trend extrapolation, similarity\-based inference\.”
- •Abductive:“This reasoning evaluates competing explanations to find the best one that accounts for the evidence\. It considers multiple hypotheses and eliminates weaker ones\. Hypothesis testing, inference to the best explanation, diagnostic reasoning, ruling out alternatives\.”

## Appendix CManifold Geometry Details

All geometric analyses are performed at layerℓ∗=32\\ell^\{\*\}=32\.

#### Intrinsic dimensionality\.

Estimated via TwoNN\(Faccoet al\.,[2017](https://arxiv.org/html/2606.02907#bib.bib32)\)\. For each point, we computeμ=r2/r1\\mu=r\_\{2\}/r\_\{1\}\(ratio of second to first nearest\-neighbor distance\)\. The estimator is:

d^ID=\(1n​∑i=1nlog⁡μi\)−1\\hat\{d\}\_\{\\text\{ID\}\}=\\left\(\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\log\\mu\_\{i\}\\right\)^\{\-1\}\(2\)Neighborhood sizek=max⁡\(3,min⁡\(⌊Ncorrect⌋,\|ℋm\|/3\)\)k=\\max\(3,\\min\(\\lfloor\\sqrt\{N\_\{\\text\{correct\}\}\}\\rfloor,\|\\mathcal\{H\}\_\{m\}\|/3\)\)\.

#### Local curvature\.

For each point𝐡i\\mathbf\{h\}\_\{i\}, we compute SVD of itskk\-nearest\-neighbor patch\. Curvature isκi=1−σ12/∑jσj2\\kappa\_\{i\}=1\-\\sigma\_\{1\}^\{2\}/\\sum\_\{j\}\\sigma\_\{j\}^\{2\}\.

#### Separation ratio\.

For modesm1,m2m\_\{1\},m\_\{2\}with centroids𝝁m1,𝝁m2\\boldsymbol\{\\mu\}\_\{m\_\{1\}\},\\boldsymbol\{\\mu\}\_\{m\_\{2\}\}and mean radiir¯m1,r¯m2\\bar\{r\}\_\{m\_\{1\}\},\\bar\{r\}\_\{m\_\{2\}\}:

ρ​\(m1,m2\)=‖𝝁m1−𝝁m2‖2\(r¯m1\+r¯m2\)/2\\rho\(m\_\{1\},m\_\{2\}\)=\\frac\{\\\|\\boldsymbol\{\\mu\}\_\{m\_\{1\}\}\-\\boldsymbol\{\\mu\}\_\{m\_\{2\}\}\\\|\_\{2\}\}\{\(\\bar\{r\}\_\{m\_\{1\}\}\+\\bar\{r\}\_\{m\_\{2\}\}\)/2\}\(3\)

#### Hull contamination\.

KNN\-based approximation: a point is “inside” modemm’s hull if itskk\-th nearest\-neighbor distance toℋm\\mathcal\{H\}\_\{m\}is within the 90th percentile of within\-mode distances\.

## Appendix DSteering Experiment Details

#### Steering magnitude selection\.

We evaluateα\\alphaover 15 logarithmically spaced values from0\.01​‖𝐯‖0\.01\\\|\\mathbf\{v\}\\\|to2\.0​‖𝐯‖2\.0\\\|\\mathbf\{v\}\\\|on 5 held\-out wrong tasks\. Output coherence is measured as mean token log\-probability\. The threshold is learned via Otsu’s method\(Akhtaret al\.,[2026](https://arxiv.org/html/2606.02907#bib.bib37)\)on the baseline distribution\.α∗\\alpha^\{\*\}is the largest value exceeding this threshold\.

#### Direction selection\.

The source mode is the most frequent mode among wrong answers \(classified by LLM\-as\-judge\)\. The target mode is the most frequent correct\-answer mode excluding the source\.

## Appendix EConflict Injection Details

Two opposing steering vectors are simultaneously injected:

𝐡~i\(ℓ∗\)=𝐡i\(ℓ∗\)\+α∗⋅\(𝐯^1\+𝐯^2\)\\tilde\{\\mathbf\{h\}\}\_\{i\}^\{\(\\ell^\{\*\}\)\}=\\mathbf\{h\}\_\{i\}^\{\(\\ell^\{\*\}\)\}\+\\alpha^\{\*\}\\cdot\(\\hat\{\\mathbf\{v\}\}\_\{1\}\+\\hat\{\\mathbf\{v\}\}\_\{2\}\)\(4\)Outcomes are classified ascollapse\(dominant score below 10th percentile of baseline\),dominance\(above 50th percentile\), orhybrid\. Thresholds are learned from the baseline style\-score distribution\. Random\-pair controls \(n=10n=10\) inject pairs of random unit vectors\.

## Appendix FDerived Hyperparameters

Table 4:All non\-design hyperparameters and their derivation\.ParameterValueDerivationKNN neighborhoodkk25max⁡\(5,⌊Ncorrect⌋\)\\max\(5,\\lfloor\\sqrt\{N\_\{\\text\{correct\}\}\}\\rfloor\)CV foldsFF5min⁡\(5,⌊Ncorrect/20⌋\)\\min\(5,\\lfloor N\_\{\\text\{correct\}\}/20\\rfloor\)Steeringα∗\\alpha^\{\*\}learnedCoherence sweep \+ OtsuPCA dims \(hull\)learnedIntrinsic dimensionalityConflict thresholdslearned10th/50th pct\. of baselineRandom steer trials20Fixed by designRandom conflict10Fixed by design
## Appendix GGeodesic Interpolation

Figure[6](https://arxiv.org/html/2606.02907#A7.F6)shows geodesic paths between mode centroids in representation space\. The smooth transitions in style scores suggest continuous, navigable trajectories rather than discrete clusters—consistent with the format\-gradient interpretation\.

![Refer to caption](https://arxiv.org/html/2606.02907v1/x6.png)Figure 6:Geodesic interpolation between reasoning modes\.Smooth transitions in style scores along centroid\-to\-centroid paths in representation space\.
## Appendix HLayer\-Specific Causal Intervention

Figure[7](https://arxiv.org/html/2606.02907#A8.F7)shows steering effectiveness as a function of intervention layer\. Early\-layer interventions produce larger effects, consistent with early layers acting as causal sites and later layers as readout surfaces\. However, as established in Section[5\.4](https://arxiv.org/html/2606.02907#S5.SS4), these effects are not direction\-specific\.

![Refer to caption](https://arxiv.org/html/2606.02907v1/x7.png)Figure 7:Layer\-specific causal intervention\.Steering at early layers produces larger mode shifts, but effects are not direction\-specific \(comparable to random perturbations\)\.
## Appendix IConflict Injection Results

![Refer to caption](https://arxiv.org/html/2606.02907v1/x8.png)Figure 8:Conflict injection\.Both targeted and random conflict pairs produce 100% coherence collapse, confirming magnitude\-based rather than direction\-specific effects\.
## Appendix JPre\-Output Failure Prediction

As an exploratory analysis, we test whether hidden\-state probes can predict task failure before output generation\. Figure[9](https://arxiv.org/html/2606.02907#A10.F9)shows ROC and precision\-recall curves for failure detection\. While hidden\-state probes achieve competitive performance with output\-confidence heuristics, this analysis is orthogonal to our main contribution and is included for completeness\.

![Refer to caption](https://arxiv.org/html/2606.02907v1/x9.png)Figure 9:Pre\-output failure prediction\.Hidden\-state probes at layer 32 achieve competitive failure detection compared to output\-confidence baselines\.