LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning

# LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning
Source: [https://arxiv.org/html/2608.12321](https://arxiv.org/html/2608.12321)
Yubo Li, Ramayya Krishnan, Rema Padman Carnegie Mellon University \{yubol, rk2x, rpadman\}@andrew\.cmu\.edu

###### Abstract

When a salient surface cue competes with an implicit feasibility constraint, LLMs often fail—but aggregate accuracy conflates genuine constraint inference with conservative defaulting\. We formalize the distinction as*conditional constraint activation*: the constraint is internally encoded \(Knowledge\) symmetrically across constraint\-present and \-absent prompts \(Symmetry\), yet only sometimes routed into the decision \(Routing\) and repairable by a donor activation \(Repair\)\. A quartet diagnostic over 14 models reveals two failure modes; probes on two open weights decode the constraint above88%88\\%, yet activation patching repairs one \(\+6\.4\+6\.4nats\) and not the other \(−0\.07\-0\.07\)\. On a mitigation frontier,*no*prompted intervention reaches the repair corner: all inflate conservative bias through a single mediation pathway—prerequisite mention\. Hidden\-constraint failure is a routing problem, not a knowledge problem\.

LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning

Yubo Li, Ramayya Krishnan, Rema PadmanCarnegie Mellon University\{yubol, rk2x, rpadman\}@andrew\.cmu\.edu

## 1Introduction

Large language models give plausible\-sounding answers to questions whose implicit prerequisites they ignore\. A widely\-circulated example asks whether to walk or drive to a nearby car wash: most frontier models answer “walk,” overlooking that a car wash needs the car to be present\(Kévin \(@knowmadd\),[2026](https://arxiv.org/html/2608.12321#bib.bib67); Wunderlich,[2026](https://arxiv.org/html/2608.12321#bib.bib66); Jo,[2026](https://arxiv.org/html/2608.12321#bib.bib64)\)\. Behavioral evaluation across heuristic and constraint families confirms the failure is widespread: under strict aggregation, no model exceeds 75% on a 4 heuristic×\\times5 constraint taxonomy\(Liet al\.,[2026](https://arxiv.org/html/2608.12321#bib.bib68)\)\. Yet aggregate accuracy on a constraint\-active item rewards two distinct behaviors equally—a model that infers the hidden constraint and a model that defaults to a conservative option for unrelated reasons\. The two are dissociable: when the constraint is*removed*, 12 of 14 evaluated models perform*worse*than on the original items, with drops up to 39 points\(Liet al\.,[2026](https://arxiv.org/html/2608.12321#bib.bib68)\)\.

#### This paper\.

We argue hidden\-constraint failures are best understood as a*routing*problem rather than a*knowledge*problem, and provide the diagnostic and mechanistic tools to test the claim\. We formalize*conditional activation*as four falsifiable conditions on a modelMMand constraintCC:

1. K\.Knowledge:a linear probe decodes “constraint applies” fromMM’s hidden state aboveθK\\theta\_\{K\}\.
2. S\.Symmetry:probe accuracy is indistinguishable across constraint\-active and \-removed prompts\.
3. R\.Routing:probe\-projected magnitude predicts the gold–shortcut decision logit gap aboveθR\\theta\_\{R\}\.
4. P\.Repair:patching hidden states from an explicit\-constraint prompt restores correctness without flipping constraint\-removed pairs\.

A model satisfying K, S, P but failing R exhibits a*conditional activation bottleneck*: the constraint is internally represented but not routed into the decision\.

#### Two failure modes, mechanistically dissociated\.

A quartet\-style evaluation—anActiveitem, a pairedRemovedcontrol, anExplicitvariant, and a length\- and frame\-matchedSalience Control—reveals two populations across 14 models\. Over\-activation models \(Llama\-4, Claude Opus 4\.6, Kimi K2\.5, Qwen3\.5\-27B, GPT\-5\.2, Claude Sonnet 4\.5, Gemini 3 Pro\) give the constraint\-heavy answer even when the constraint is absent \(CBI\+0\.13\+0\.13to\+0\.29\+0\.29\)\. Under\-activation models \(GPT\-OSS\-\{20B, 120B\}\) under\-apply the constraint when required \(CBI−0\.10\-0\.10to−0\.19\-0\.19\)\. On the two open weights, the modes split mechanistically: in Qwen3\-14B, patching constraint information from an Explicit prompt into a failing Active prompt shifts the gold–shortcut gap by\+6\.4\+6\.4nats while leaving Removed pairs essentially unchanged \(−0\.84\-0\.84\); in GPT\-OSS\-20B, an88%88\\%\-accurate probe atL20L\_\{20\}confirms the constraint is internally available, yet patching moves the answer by only−0\.07\-0\.07nats\. The under\-activation mode is the textbook routing failure; the over\-activation mode is a prior\-bias problem with routing intact\.

#### Salience\-controlled hint activation\.

A common reading of the explicitness\-gradient observation\(Liet al\.,[2026](https://arxiv.org/html/2608.12321#bib.bib68)\)is that hints “remind” the model of the constraint\. We test this by inserting a length\- and frame\-matched neutral filler in place of the constraint cue\. The Salience\-Adjusted Hint Gain ranges from\+0\.021\+0\.021\(Gemini 3 Pro—no specificity above matched salience\) to\+0\.294\+0\.294\(Llama\-4—strong specificity\), with matched controls flat across all five ladder levels\.

#### The mitigation frontier\.

We re\-evaluate four prompted interventions—CoT\(Weiet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib28); Kojimaet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib29)\), precondition listing, goal decomposition, and counterfactual checking\(Wanget al\.,[2023c](https://arxiv.org/html/2608.12321#bib.bib30)\)—on a two\-dimensional frontier \(active\-item gain vs\. removed\-pair harm\)\. Across all 40 \(model×\\timesstrategy\) cells, strategies cluster in the high\-harm, near\-zero\-gain region: mean removed\-pair harm\+0\.44\+0\.44to\+0\.47\+0\.47in CBI units, mean active gain only\+0\.01\+0\.01to\+0\.04\+0\.04;*none*reaches the repair corner\. They converge on a shared mediation pathway—mediated\-correctness share≥0\.91\\geq\\\!0\.91in every cell—each working by inducing the same surface behavior \(prerequisite mention\) that boosts Active\-correctness while over\-triggering on Removed prompts\. A reasoning\-budget sweep on four thinking\-mode models reproduces the same harm\-without\-repair signature\(Snellet al\.,[2025](https://arxiv.org/html/2608.12321#bib.bib23); Muennighoffet al\.,[2025](https://arxiv.org/html/2608.12321#bib.bib25)\)\.

#### Contributions\.

\(i\) A formal definition of*conditional constraint activation*as a falsifiable K/S/R/P claim, instantiated as a quartet diagnostic with salience controls\. \(ii\) Empirical evidence for two distinct failure modes and a mechanistic dissociation via probes and patching on two open weights\. \(iii\) The*mitigation frontier*protocol, showing that all four established prompted methods move along the harm axis\. \(iv\) A measured mediation chain explaining*why*prompted compute helps when it does\. Together: hidden\-constraint failure is a circuit\-level conditional\-activation problem; prompted mitigations cannot repair it; the remedy must operate on the routing direction itself\.

## 2Related Work

#### Heuristic shortcuts and the knowing\-vs\-using gap\.

LLMs follow surface\-level statistical regularities rather than perform the intended computation\(Geirhoset al\.,[2020](https://arxiv.org/html/2608.12321#bib.bib55); McCoyet al\.,[2019](https://arxiv.org/html/2608.12321#bib.bib56); Nikankinet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib57)\), showing content effects\(Lampinenet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib58)\)and easy distractibility\(Shiet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib59); Mirzadehet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib60)\)\. A parallel line shows models internally represent information they do not output: latent multi\-hop chains\(Yanget al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib54)\), the reversal curse\(Berglundet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib47)\), hallucinations whose truth status is encoded in hidden state\(Orgadet al\.,[2025](https://arxiv.org/html/2608.12321#bib.bib53)\), and truth\-direction inference\-time interventions\(Liet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib51); Zouet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib52)\)\. The Heuristic Override Benchmark\(Liet al\.,[2026](https://arxiv.org/html/2608.12321#bib.bib68)\)extended the behavioral failure to pragmatic reasoning across four heuristic families and five constraint families, but leaves open whether a correct answer reflects genuine constraint inference or conservative defaulting\. We push past that ambiguity with a quartet\-based diagnostic and with mechanistic experiments that localize the failure to a specific routing pathway\.

#### Linear probes and activation patching\.

Linear classifiers on intermediate representations\(Alain and Bengio,[2017](https://arxiv.org/html/2608.12321#bib.bib1); Conneauet al\.,[2018](https://arxiv.org/html/2608.12321#bib.bib4); Hewitt and Manning,[2019](https://arxiv.org/html/2608.12321#bib.bib2); Belinkov,[2022](https://arxiv.org/html/2608.12321#bib.bib5)\)are the standard test for whether a feature is present in hidden state, though high probe accuracy does not by itself establish causal use\(Hewitt and Liang,[2019](https://arxiv.org/html/2608.12321#bib.bib3)\)\. Many factual associations and truth\-related axes are linearly decodable\(Petroniet al\.,[2019](https://arxiv.org/html/2608.12321#bib.bib6); Gevaet al\.,[2021](https://arxiv.org/html/2608.12321#bib.bib7); Daiet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib8); Gurnee and Tegmark,[2024](https://arxiv.org/html/2608.12321#bib.bib9); Burnset al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib10); Zouet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib52); Orgadet al\.,[2025](https://arxiv.org/html/2608.12321#bib.bib53)\)\. Causal mediation\(Viget al\.,[2020](https://arxiv.org/html/2608.12321#bib.bib11); Geigeret al\.,[2021](https://arxiv.org/html/2608.12321#bib.bib69)\)and activation patching\(Menget al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib12); Wanget al\.,[2023a](https://arxiv.org/html/2608.12321#bib.bib13); Hannaet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib14); Lieberumet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib15); Wuet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib16); Markset al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib61); Zhang and Nanda,[2024](https://arxiv.org/html/2608.12321#bib.bib18); Heimersheim and Nanda,[2024](https://arxiv.org/html/2608.12321#bib.bib19)\)have localized circuits for factual recall, indirect\-object identification, and arithmetic\(Stolfoet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib17)\)\. Most prior patching work targets*factual recall*or*single\-step symbolic*computations\. We apply patching to*conditional pragmatic reasoning*on two open weights with opposite behavioral failure modes, yielding a mechanistic dissociation aggregate behavior obscures\.

#### Prompted mitigations and inference\-time compute\.

A steady stream of prompting interventions has been introduced— chain\-of\-thought\(Weiet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib28); Kojimaet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib29)\), self\-consistency\(Wanget al\.,[2023c](https://arxiv.org/html/2608.12321#bib.bib30)\), tree\-of\-thoughts\(Yaoet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib31)\), plan\-and\-solve\(Wanget al\.,[2023b](https://arxiv.org/html/2608.12321#bib.bib33)\), self\-refine\(Madaanet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib34)\), verifier reranking\(Cobbeet al\.,[2021](https://arxiv.org/html/2608.12321#bib.bib35); Lightmanet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib36)\), reflexion\(Shinnet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib37)\), step\-back\(Zhenget al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib32)\)—alongside long\-CoT thinking\-mode models\(OpenAI,[2024](https://arxiv.org/html/2608.12321#bib.bib20); DeepSeek\-AI,[2025](https://arxiv.org/html/2608.12321#bib.bib21); Anthropic,[2025](https://arxiv.org/html/2608.12321#bib.bib22)\)and inference\-scaling work\(Snellet al\.,[2025](https://arxiv.org/html/2608.12321#bib.bib23); Brownet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib24); Muennighoffet al\.,[2025](https://arxiv.org/html/2608.12321#bib.bib25)\)\. Counter\-evidence shows LLMs cannot reliably self\-correct reasoning\(Huanget al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib38)\), CoT benefits are narrow\(Spragueet al\.,[2025](https://arxiv.org/html/2608.12321#bib.bib27)\), and overthinking can degrade performance\(Chenet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib26)\)\. Prior reports rarely audit whether constraint\-removed minimal pairs are damaged by the same intervention\. Our mitigation frontier re\-evaluates four established strategies on a two\-dimensional axis \(active gain vs\. pair harm\), showing that gains in the literature are routinely along the conservative\-bias axis rather than the repair axis\.

#### Conservative bias and answer asymmetries\.

Sycophancy\(Perezet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib39); Sharmaet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib40); Weiet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib41)\), hedging\(Zhouet al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib43); Xionget al\.,[2024](https://arxiv.org/html/2608.12321#bib.bib44)\), and calibration asymmetries\(Kadavathet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib42)\)are documented RLHF\-induced behaviors that shift outputs toward safe options\. Our over\-activation failure mode is their mechanistic counterpart—a constraint\-heavy default that is correct when the constraint applies and wrong when it does not\. Diagnosing it behaviorally requires the minimal\-pair contrast; diagnosing it mechanistically requires checking whether the constraint signal in hidden state actually shifts the decision under patching\.

## 3Methods

We instantiate the four conditional\-activation conditions \(K/S/R/P, §[1](https://arxiv.org/html/2608.12321#S1)\) through three measurement layers: \(i\) a behavioral quartet diagnostic with matched salience controls, \(ii\) a salience\-controlled hint ladder, and \(iii\) hidden\-state probes plus activation patching on open\-weight models\. We additionally re\-evaluate four prompted mitigations on a two\-axis frontier and probe inference\-time compute through a budget sweep on four thinking\-mode models\.

### 3\.1Datasets and Conditions

#### Item universe\.

We start from the HOB benchmark\(Liet al\.,[2026](https://arxiv.org/html/2608.12321#bib.bib68)\): a 4\-heuristic×\\times5\-constraint taxonomy \(4×5=204\\times 5=20cells\) over≈\\approx140 scenarios, each a goal \+ salient surface cue \+ hidden constraintCCthat, when active, makes the cue\-following answer infeasible\. We usecore100: 100 base scenarios stratified across all 20 cells\.

#### Quartet construction\.

For each scenario we construct four conditions paired by scenario id \(Table[1](https://arxiv.org/html/2608.12321#S3.T1)\)\.Salience Controlcontrols for the alternative hypothesis that explicit\-constraint gains are a non\-specific salience effect: fillers are length\-bucketed, neutral declarative sentences, assigned hash\-deterministically by instance id\.

Table 1:The four quartet conditions per scenario\.
#### Hint ladder\.

A five\-level ladderL0≺⋯≺L4L\_\{0\}\\\!\\prec\\\!\\cdots\\\!\\prec\\\!L\_\{4\}, whereL0L\_\{0\}isActiveandL4L\_\{4\}approachesExplicit\. At every levelLiL\_\{i\}\(i≥1i\\\!\\geq\\\!1\) we inject three matched negative controls:ctrl\_lex,ctrl\_len,ctrl\_heur\.

#### Sample sizes\.

TTtrials per \(model, condition\):T=10T\\\!=\\\!10for API models,T=8T\\\!=\\\!8for local\-GPU models,T=4T\\\!=\\\!4for the budget sweep\.

### 3\.2Models

We evaluate 14 LLMs spanning four providers \(Claude Opus/Sonnet, GPT 5\.2/5\.4/OSS\-20B/OSS\-120B, Gemini 3 Pro, Grok 4\.2, DeepSeek\-R1, Kimi K2\.5, Llama\-4, Qwen3\-14B/32B, Qwen3\.5\-27B\)\. For probes and patching we useQwen3\-14B\(balanced\) andGPT\-OSS\-20B\(under\-activation\), the two open weights spanning the failure modes\.

### 3\.3Behavioral Metrics

LetAccc\\mathrm\{Acc\}\_\{c\}denote accuracy in conditioncc\.

CBI\\displaystyle\\mathrm\{CBI\}=AccA−AccR\\displaystyle=\\mathrm\{Acc\}\_\{\\textsc\{A\}\}\-\\mathrm\{Acc\}\_\{\\textsc\{R\}\}SalAdjGain\\displaystyle\\mathrm\{SalAdjGain\}=AccE−AccSC\\displaystyle=\\mathrm\{Acc\}\_\{\\textsc\{E\}\}\-\\mathrm\{Acc\}\_\{\\textsc\{SC\}\}PCA\\displaystyle\\mathrm\{PCA\}=1\|S\|​∑sAcc^A​\(s\)⋅Acc^R​\(s\)\\displaystyle=\\tfrac\{1\}\{\|S\|\}\\sum\_\{s\}\\widehat\{\\mathrm\{Acc\}\}\_\{\\textsc\{A\}\}\(s\)\\\!\\cdot\\\!\\widehat\{\\mathrm\{Acc\}\}\_\{\\textsc\{R\}\}\(s\)CAS\\displaystyle\\mathrm\{CAS\}=PCA⋅\(1−\|CBI\|\)\\displaystyle=\\mathrm\{PCA\}\\cdot\(1\-\|\\mathrm\{CBI\}\|\)CBI\>θCBI=0\.10\\mathrm\{CBI\}\\\!\>\\\!\\theta\_\{\\text\{CBI\}\}\\\!=\\\!0\.10flags over\-activation;CBI<−θCBI\\mathrm\{CBI\}\\\!<\\\!\-\\theta\_\{\\text\{CBI\}\}flags under\-activation\. Over\- and under\-activation rates areOAR=1−AccR\\mathrm\{OAR\}\\\!=\\\!1\\\!\-\\\!\\mathrm\{Acc\}\_\{\\textsc\{R\}\}andUAR=1−AccA\\mathrm\{UAR\}\\\!=\\\!1\\\!\-\\\!\\mathrm\{Acc\}\_\{\\textsc\{A\}\}\. The ladder*specificity gap*at leveliiisAccLi−maxτ⁡AccLi\(τ\)\\mathrm\{Acc\}\_\{L\_\{i\}\}\-\\max\_\{\\tau\}\\mathrm\{Acc\}\_\{L\_\{i\}^\{\(\\tau\)\}\}; the minimum\-effective\-hint\-level \(MEHL\) is the smallestiiat which it exceeds a preregistered0\.100\.10margin\.

### 3\.4Mitigation Frontier

We evaluate four prompted strategies grouped by hypothesized mechanism: Class A raises the conservative threshold \(vanilla CoT\(Weiet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib28); Kojimaet al\.,[2022](https://arxiv.org/html/2608.12321#bib.bib29)\)\); Class B induces prerequisite search \(precondition listing,goal decomposition\); Counterfactual\-check\(Wanget al\.,[2023c](https://arxiv.org/html/2608.12321#bib.bib30)\)is an oracle upper bound\. Each is a single\-prompt prefix; every other condition is held fixed\. For \(model, strategy\) we report the two\-axis frontierActiveGain=AccAstrat−AccAzero\\textsc\{ActiveGain\}\\\!=\\\!\\mathrm\{Acc\}^\{\\text\{strat\}\}\_\{\\textsc\{A\}\}\-\\mathrm\{Acc\}^\{\\text\{zero\}\}\_\{\\textsc\{A\}\}vs\.PairHarm=CBIstrat−CBIzero\\textsc\{PairHarm\}\\\!=\\\!\\mathrm\{CBI\}^\{\\text\{strat\}\}\-\\mathrm\{CBI\}^\{\\text\{zero\}\}; the repair corner \(positiveActiveGain, non\-positivePairHarm\) sits at the bottom\-right\.

### 3\.5Reasoning\-Budget Sweep

We sweepbudget∈\{256,1024,4096,16384\}\\text\{budget\}\\in\\\{256,1024,4096,16384\\\}thinking tokens on the four thinking\-mode models\. Provider\-specific budget routing \(thinking\.budget\_tokens,reasoning\_effort,ThinkingConfig\) is detailed in Appendix[G](https://arxiv.org/html/2608.12321#A7)\.

### 3\.6Mediation Analysis

We estimate the chainintervention→\\toprerequisite mention→\\tocorrectness\. Mention is detected by content\-word overlap with the canonical hidden constraint \(Appendix[G](https://arxiv.org/html/2608.12321#A7)\)\. We reportPr⁡\(c∣m\)\\Pr\(\\text\{c\}\\mid\\text\{m\}\),Pr⁡\(c∣¬m\)\\Pr\(\\text\{c\}\\mid\\neg\\text\{m\}\),Pr⁡\(m\)\\Pr\(\\text\{m\}\), and the*mediated correctness share*, the fraction of correct answers attributable to mentioning traces\.

### 3\.7Hidden\-State Probes \(K, S, R\)

For Qwen3\-14B and GPT\-OSS\-20B we cache the post\-block residual hidden state at the final input token across all transformer layers on everycore100quartet item\. We train one per\-layer logistic regression \(liblinear,C=1C\\\!=\\\!1, with per\-feature standardization\) to predictActivevs\.Removed\. Cross\-validation is*scenario\-grouped*5\-fold: the four conditions for a scenario co\-occur in the same fold\.Kis satisfied at the best layer when CV accuracy exceedsθK=0\.80\\theta\_\{K\}\\\!=\\\!0\.80\.S\(symmetry\) is the test\-accuracy gap between balanced Active\-only and Removed\-only training subsets, thresholdθS=0\.05\\theta\_\{S\}\\\!=\\\!0\.05\.R\(routing\) is the Spearman correlation between probe projection and gold–shortcut decision logit gap at the best probe layer, thresholdθR=0\.30\\theta\_\{R\}\\\!=\\\!0\.30\.

### 3\.8Activation Patching \(P\)

For each scenario we forward theExplicit\(donor\) prompt and cache the final\-token hidden statehℓ⋆donorh^\{\\text\{donor\}\}\_\{\\ell^\{\\star\}\}at the probe’s best layerℓ⋆\\ell^\{\\star\}\. We then forward the recipient \(ActiveorRemoved\) prompt with a forward hook onℓ⋆\\ell^\{\\star\}that overwrites the final\-token output withhℓ⋆donorh^\{\\text\{donor\}\}\_\{\\ell^\{\\star\}\}, and recompute the gold–shortcut log\- probability gap\. The patched\-minus\-baseline differenceΔ\\Deltameasures how much the constraint\-encoding activation changes the decision\.Prequires specificity:ΔActive\>θP=1\.0\\Delta^\{\\text\{Active\}\}\\\!\>\\\!\\theta\_\{P\}\\\!=\\\!1\.0nat*and*\|ΔRemoved\|≤θP\|\\Delta^\{\\text\{Removed\}\}\|\\\!\\leq\\\!\\theta\_\{P\}, preregistered against the typical logit\-gap magnitude \(σΔ≈3\\sigma\_\{\\Delta\}\\\!\\approx\\\!3nats\)\. Implementation, donor\-shuffling control, and layer/position sweeps are in Appendix[G](https://arxiv.org/html/2608.12321#A7)\.

## 4Results

We present results in the order of the four conditions:K\(knowledge present\),S\(symmetric across paired conditions\),R\(routed into the decision\)—all in §[4\.5](https://arxiv.org/html/2608.12321#S4.SS5)— andP\(repair via patching, §[4\.6](https://arxiv.org/html/2608.12321#S4.SS6)\)\. §[4\.1](https://arxiv.org/html/2608.12321#S4.SS1)reports the behavioral signature on all 14 models; §[4\.2](https://arxiv.org/html/2608.12321#S4.SS2)reports the salience\-controlled hint ladder; §[4\.3](https://arxiv.org/html/2608.12321#S4.SS3)reports the mitigation frontier; §[4\.4](https://arxiv.org/html/2608.12321#S4.SS4)reports the reasoning\-budget sweep with mediation analysis\.

### 4\.1Behavioral signature: two failure modes

Table[2](https://arxiv.org/html/2608.12321#S4.T2)reports per\-model accuracy in all four quartet conditions, plus the derived metrics\. Across 14 models we observe*two*populations separated by the sign of CBI \(Fig\.[1](https://arxiv.org/html/2608.12321#S4.F1)\)\.

![Refer to caption](https://arxiv.org/html/2608.12321v1/x1.png)Figure 1:Per\-model Conservative Bias Index, rank\-ordered\. Two populations are separated by\|CBI\|=0\.10\|\\text\{CBI\}\|=0\.10\(grey dashed\): over\-activation \(red, 7 models\), balanced \(grey, 5 models\), and under\-activation \(green, 2 models\)\.#### Over\-activation \(CBI\>\+0\.10\>\\\!\+0\.10\): 7 models\.

Llama\-4, Claude Opus 4\.6, Qwen3\.5\-27B, Kimi K2\.5, Claude Sonnet 4\.5, GPT\-5\.2, and Gemini 3 Pro \(CBI\+0\.13\+0\.13to\+0\.29\+0\.29; see Table[2](https://arxiv.org/html/2608.12321#S4.T2)\)\. These models pick the constraint\-heavy answer even when the constraint is absent: Removed accuracy falls 11–33 points below Active\. Pair\-Consistent Accuracy correspondingly collapses \(e\.g\.0\.3420\.342for Llama\-4 vs\.0\.7920\.792marginal Active accuracy\)\.

#### Balanced \(\|CBI\|≤0\.10\\leq\\\!0\.10\): 5 models\.

DeepSeek\-R1 \(\+0\.086\+0\.086\), Grok 4\.2 \(\+0\.048\+0\.048\), GPT\-5\.4 \(\+0\.030\+0\.030\), Qwen3\-32B \(\+0\.005\+0\.005\), Qwen3\-14B \(−0\.003\-0\.003\)\. Active and Removed accuracies are within99points across all five\.

#### Under\-activation \(CBI<−0\.10<\\\!\-0\.10\): 2 models\.

GPT\-OSS\-20B \(−0\.103\-0\.103\) and GPT\-OSS\-120B \(−0\.186\-0\.186\)\. These models are*more*accurate on Removed than on Active—they fail to apply the constraint when it matters\.

Per\-condition AccModelActiveRemovedExplicitSal CtrlCBIPCACASSalAdjGain*Over\-activation*Llama\-40\.7920\.5050\.9260\.632\+0\.287\+0\.2870\.3420\.400\+0\.294\+0\.294Claude Opus 4\.60\.8860\.6040\.9230\.833\+0\.282\+0\.2820\.5230\.535\+0\.090\+0\.090Qwen3\.5\-27B0\.9250\.6900\.9690\.851\+0\.235\+0\.2350\.6450\.638\+0\.118\+0\.118Kimi K2\.50\.9280\.7150\.9560\.853\+0\.213\+0\.2130\.6500\.664\+0\.103\+0\.103Claude Sonnet 4\.50\.8370\.6270\.9360\.766\+0\.210\+0\.2100\.4960\.525\+0\.170\+0\.170GPT\-5\.20\.8240\.6540\.9290\.802\+0\.170\+0\.1700\.5130\.539\+0\.127\+0\.127Gemini 3 Pro0\.9070\.7790\.9100\.889\+0\.128\+0\.1280\.6930\.707\+0\.021\+0\.021*Balanced*DeepSeek\-R10\.8500\.7640\.9570\.795\+0\.086\+0\.0860\.6270\.649\+0\.162\+0\.162Grok 4\.20\.8830\.8350\.9540\.790\+0\.048\+0\.0480\.7250\.737\+0\.164\+0\.164GPT\-5\.40\.8220\.7920\.9380\.723\+0\.030\+0\.0300\.6340\.651\+0\.215\+0\.215Qwen3\-32B0\.7710\.7660\.9320\.667\+0\.005\+0\.0050\.5800\.591\+0\.265\+0\.265Qwen3\-14B0\.7620\.7650\.9440\.694−0\.003\-0\.0030\.5670\.583\+0\.250\+0\.250*Under\-activation*GPT\-OSS\-20B0\.7650\.8680\.9410\.694−0\.103\-0\.1030\.6570\.664\+0\.248\+0\.248GPT\-OSS\-120B0\.6860\.8720\.8570\.627−0\.186\-0\.1860\.5950\.598\+0\.230\+0\.230Table 2:Quartet behavioral metrics oncore100, 14 models, sorted by CBI\. PCA is the soft \(per\-scenario expectation\) variant; CAS combines PCA with the conservative\-bias penalty\. SalAdjGain is the explicit\-minus\-salience\-control accuracy gap\.
#### Salience\-Adjusted Hint Gain\.

SalAdjGain ranges from\+0\.021\+0\.021\(Gemini 3 Pro: no specificity beyond matched salience\) to\+0\.294\+0\.294\(Llama\-4\) and\+0\.265\+0\.265\(Qwen3\-32B\)\. The under\-activation models retain large positive SalAdjGain \(GPT\-OSS\-20B:\+0\.248\+0\.248; GPT\-OSS\-120B:\+0\.230\+0\.230\), confirming the constraint*can*be activated when made explicit\. Gemini 3 Pro’s near\-zero SalAdjGain demonstrates that a model can show large*nominal*hint gains entirely explained by added textual salience\.

### 4\.2Hint ladder with matched controls

On the six API models with full ladder collection, explicit\-condition accuracy rises monotonically with hint level for every model; the three matched controls remain within±0\.04\\pm 0\.04ofL0L\_\{0\}across all four ladder rungs\. The specificity gap exceeds the preregistered0\.100\.10margin byL2L\_\{2\}for over\-activation models and byL3L\_\{3\}for balanced models \(DeepSeek\-R1, GPT\-5\.4\)\. Per\-model curves and specificity gaps are in Figs\.[5](https://arxiv.org/html/2608.12321#A5.F5)and[6](https://arxiv.org/html/2608.12321#A5.F6)\(Appendix\)\.

### 4\.3The mitigation frontier

Table[3](https://arxiv.org/html/2608.12321#S4.T3)reports per\-strategy means over 10 API models; full per\-\(model, strategy\) detail is in Table[7](https://arxiv.org/html/2608.12321#A2.T7)\(Appendix\)\.

#### Existing strategies inflate CBI without repairing routing\.

Across all 40 \(model, strategy\) cells,PairHarmis large \(\+0\.28\+0\.28to\+0\.65\+0\.65\) andActiveGainis small \(−0\.08\-0\.08to\+0\.11\+0\.11\); none reaches the repair corner\. Counter to our preregistered hypothesis, Class\-B interventions \(precondition\_listing,goal\_decomposition\) do not move closer to repair than Class\-A: meanPairHarm\+0\.44\+0\.44vs\.\+0\.47\+0\.47, meanActiveGain\+0\.038\+0\.038vs\.\+0\.012\+0\.012\. Thecounterfactual\_checkoracle shows the same pair\-harm signature \(\+0\.44\+0\.44mean\)\. Llama\-4 is the worst case:*every*strategy reduces its Active accuracy \(−0\.018\-0\.018to−0\.082\-0\.082\)\.

#### Mediation through prerequisite mention\.

Fig\.[2](https://arxiv.org/html/2608.12321#S4.F2)visualizes the per\-strategy frontier; every point sits in the high\-harm, low\-gain region\.

![Refer to caption](https://arxiv.org/html/2608.12321v1/x2.png)Figure 2:Mitigation frontier across ten API models and four prompted strategies\. Each point is one \(model, strategy\) cell at the model’s own zero\-shot baseline\. The green “repair corner” \(bottom\-right of each panel: positive ActiveGain, non\-positive PairHarm\) is empty for every strategy\. Points colored by failure mode of the base model\.Across all 40 cells, the conditional gapPr⁡\(c∣m\)−Pr⁡\(c∣¬m\)\\Pr\(\\text\{c\}\\mid\\text\{m\}\)\-\\Pr\(\\text\{c\}\\mid\\neg\\text\{m\}\)is large \(\+0\.34\+0\.34to\+0\.64\+0\.64\),Pr⁡\(m\)\\Pr\(\\text\{m\}\)is high \(0\.840\.84to0\.960\.96\), and the mediated\-correctness share is0\.910\.91to0\.990\.99\. Every strategy “works” by inducing the same surface behavior \(prerequisite mention\); they differ only in how much they over\-trigger it on Removed items\.

Table 3:Per\-strategy means over 10 API models\. Every strategy moves overwhelmingly along the harm axis; ActiveGain is small and indistinguishable across classes\. Per\-\(model, strategy\) detail is in Table[7](https://arxiv.org/html/2608.12321#A2.T7)\(Appendix\)\.

### 4\.4Reasoning\-budget sweep

We sweepbudget∈\{256,1024,4096,16,384\}\\text\{budget\}\\in\\\{256,1024,4096,16\{,\}384\\\}thinking tokens on the four thinking\-mode models \(Claude Opus 4\.6, GPT\-5\.4, Gemini 3 Pro, DeepSeek\-R1\) over the fullcore100quartet \(T=4T\\\!=\\\!4trials per cell\)\. Per\-budget Active/Removed accuracy, CBI, and mediation metrics are summarized in Fig\.[3](https://arxiv.org/html/2608.12321#S4.F3)\.

Thinking\-mode*itself*moves all four models well into the over\-activation regime: at budget=256256tokens, CBI is\+0\.85\+0\.85for Claude Opus \(vs\.\+0\.28\+0\.28zero\-shot\),\+0\.57\+0\.57for GPT\-5\.4 \(vs\.\+0\.03\+0\.03\),\+0\.65\+0\.65for Gemini 3 Pro \(vs\.\+0\.13\+0\.13\),\+0\.66\+0\.66for DeepSeek\-R1 \(vs\.\+0\.09\+0\.09\)\. Removed accuracy collapses \(≤0\.22\\leq\\\!0\.22\) while Active accuracy holds within0\.050\.05of zero\-shot\. Budget within thinking\-mode is essentially a no\-op: CBI varies by≤0\.05\\leq 0\.05across the four\-point grid\. Mediation is uniformly strong:Pr⁡\(m\)∈\[0\.82,0\.92\]\\Pr\(\\text\{m\}\)\\\!\\in\\\!\[0\.82,0\.92\], conditional gapPr⁡\(c∣m\)−Pr⁡\(c∣¬m\)∈\[\+0\.43,\+0\.67\]\\Pr\(\\text\{c\}\\mid\\text\{m\}\)\-\\Pr\(\\text\{c\}\\mid\\neg\\text\{m\}\)\\\!\\in\\\!\[\+0\.43,\+0\.67\], mediated\-correctness share≥0\.93\\geq\\\!0\.93throughout\. Extra compute does not change*which*items get the mention, only the strength of the bias\. Per\-budget detail is in Table[8](https://arxiv.org/html/2608.12321#A3.T8)\(Appendix\)\.

![Refer to caption](https://arxiv.org/html/2608.12321v1/x3.png)Figure 3:Reasoning\-budget sweep on four thinking\-mode models\.*Left:*CBI as a function of budget; the four dotted horizontals show each model’s zero\-shot \(non\-thinking\) baseline\. Thinking\-mode moves all four models well above the zero\-shot CBI; budget within thinking\-mode does not\.*Right:*the mediation conditional gapPr⁡\(c∣m\)−Pr⁡\(c∣¬m\)\\Pr\(\\text\{c\}\\mid\\text\{m\}\)\\\!\-\\\!\\Pr\(\\text\{c\}\\mid\\neg\\text\{m\}\)stays uniformly large and budget\-insensitive\.
### 4\.5Conditions K, S, R: open\-weight probes

#### \(K\) Knowledge\.

A linear probe on per\-layer hidden states decodes the binary Active\-vs\-Removed label well above chance for both models \(Table[4](https://arxiv.org/html/2608.12321#S4.T4)\)\. Qwen3\-14B peaks at layer 27 with94\.5%±1\.9%94\.5\\%\\pm 1\.9\\%cross\-validated accuracy; GPT\-OSS\-20B peaks at layer 20 with88\.0%±4\.3%88\.0\\%\\pm 4\.3\\%\. The constraint is internally encoded in both models, well above the preregisteredθK=0\.80\\theta\_\{K\}\\\!=\\\!0\.80threshold\. We additionally observe broad\-layer accessibility:≥0\.92\\geq\\\!0\.92accuracy across layers2525–3636in Qwen3\-14B and≥0\.85\\geq\\\!0\.85across layers44–2424in GPT\-OSS\-20B \(full per\-layer curves in Table[9](https://arxiv.org/html/2608.12321#A4.T9), Appendix\)\.

#### \(S\) Symmetry\.

Re\-training the probe on a balanced constraint\-active/constraint\-removed split shows the test\-accuracy gap between the two halves is withinθS=0\.05\\theta\_\{S\}\\\!=\\\!0\.05for both models at their best layer \(Δ=0\.011\\Delta\\\!=\\\!0\.011for Qwen3\-14B;Δ=0\.024\\Delta\\\!=\\\!0\.024for GPT\-OSS\-20B\)\. The knowledge is present symmetrically; an asymmetry\-based explanation of the behavioral CBI is ruled out\.

#### \(R\) Routing\.

The probe\-projected magnitude at the best layer correlates only weakly with the per\-trial gold–shortcut logit gap \(Spearmanρ=0\.125\\rho\\\!=\\\!0\.125for Qwen3\-14B,0\.2240\.224for GPT\-OSS\-20B\); the per\-layer maxima \(0\.2420\.242,0\.3020\.302\) also fall below the preregisteredθR=0\.30\\theta\_\{R\}\\\!=\\\!0\.30\. The probe direction is present but not strongly read by the decision head—the signature of a routing failure\.

Table 4:Conditions K, S, R on the two open\-weight models\. Both satisfy K and S; both fall belowθR=0\.30\\theta\_\{R\}=0\.30at the probe’s best layer \(Qwen3\-14B clearly; GPT\-OSS\-20B marginally\)\.

### 4\.6Condition P: activation patching

Patching the constraint\-encoding hidden state from the Explicit prompt into the Active prompt at the probe’s best layer produces opposite signatures in the two models \(Table[5](https://arxiv.org/html/2608.12321#S4.T5),n=100n\\\!=\\\!100scenarios each\)\.

Qwen3\-14B\.The Active gold–shortcut gap rises from\+1\.40\+1\.40to\+7\.78\+7\.78nats \(Δ=\+6\.38\\Delta\\\!=\\\!\+6\.38\), an order\-of\-magnitude shift, while Removed pairs are only mildly affected \(Δ=−0\.84\\Delta\\\!=\\\!\-0\.84\)\. This is a clean repair signature: the constraint\-encoding direction*exists*in the donor and*causally*shifts the recipient’s decision when patched\.P is satisfied\.GPT\-OSS\-20B\.The same intervention yields no effect:Δ=−0\.069\\Delta\\\!=\\\!\-0\.069on Active patches,\+0\.029\+0\.029on Removed \(both within the∼3\\sim 3\-nat noise floor\)\. Despite the88%88\\%\-accurate probe atL20L\_\{20\}, the constraint direction does not causally control the decision\.P fails\.

![Refer to caption](https://arxiv.org/html/2608.12321v1/x4.png)Figure 4:Activation\-patching deltas at the probe’s best layer\. Pre\-patch \(grey\) vs\. post\-patch \(blue\) mean gold–shortcut logit gap\. Qwen3\-14B shows a clean repair signature on Active recipients \(Δ=\+6\.38\\Delta=\+6\.38\) with controlled Removed deviation; GPT\-OSS\-20B shows no effect, despite the strong probe at the same layer\.Table 5:Activation\-patching repair test\. Qwen3\-14B exhibits a clean repair signature on Active patches with controlled Removed harm\. GPT\-OSS\-20B exhibits no effect, despite a strong probe at the same layer \(K satisfied\)\.#### K/S/R/P verdict\.

Qwen3\-14B is K ✓ S ✓ R∂\\partialP ✓\(causally available, weakly routed\); GPT\-OSS\-20B is K ✓ S ✓ R∂\\partialP×\\times\(textbook routing failure\)\. The 7 over\-activation models share this K/S/R structure but add a strong constraint\-respecting prior\.

## 5Discussion

### 5\.1Routing, not knowledge

The K/S/R/P measurements disconfirm the simplest account of hidden\-constraint failure—that the model does not know the constraint\. Probes decode constraint applicability at94\.5%94\.5\\%\(Qwen3\-14B,L27L\_\{27\}\) and88\.0%88\.0\\%\(GPT\-OSS\-20B,L20L\_\{20\}\), broadly across 10\+ layers, and the symmetry test rules out a constraint\-active\-only artifact\. Knowledge*is*there\.

What separates the two open weights is the*causal*test: patching the donor Explicit activation repairs Qwen3\-14B \(\+6\.4\+6\.4nats, Removed unchanged\) but does nothing in GPT\-OSS\-20B \(−0\.07\-0\.07\)—the direction is decodable yet unread by the decision head\. The implication: Active\-item accuracy alone cannot tell competent inference from a constraint\-heavy prior; the Removed pair makes the distinction visible\.

### 5\.2Why every prompted mitigation fails the same way

The mitigation\-frontier evidence is the most surprising part of this paper\. CoT, precondition listing, goal decomposition, and even counterfactual checking—an instruction to explicitly imagine the salient resource absent—all produce the same harm profile: large PairHarm \(\+0\.28\+0\.28to\+0\.65\+0\.65\), small ActiveGain \(−0\.08\-0\.08to\+0\.11\+0\.11\),≥91%\\geq 91\\%mediated\-correctness share\. The strategies are not differentiated by how they reason; they are differentiated only by how often they trigger the*same downstream behavior*—namely, mentioning the hidden constraint\.

A single common cause fits\. Each strategy liftsPr⁡\(mention\)\\Pr\(\\text\{mention\}\)from a low baseline \(0\.30\.3–0\.50\.5\) to a uniformly high value \(0\.840\.84–0\.960\.96\); whether or not the constraint applies, the model talks about it, and the conservative answer then dominates\. The conditional gap is large \(\+0\.34\+0\.34to\+0\.64\+0\.64\) but*symmetric*: on Removed items, mention flips the answer the wrong way, netting to near\-zero on Pair\-Consistent Accuracy—yet the literature reports only the constraint\-active half\. Prompted reasoning cannot*decide*when to mention the constraint, only globally bias the rate; selective activation is a routing problem, not a prompt\-level property\.

### 5\.3Two failure modes, two remedies

The mechanistic measurements suggest different remedies per population\.Over\-activation\(7 models\) combines intact routing with a prior bias \(large SalAdjGain\); prompted mitigations push exactly the coefficient that already over\-fires\.Balanced\(5 models\) is where Qwen3\-14B’s patching success shows a*selective*intervention—inject the constraint direction only when it applies— is mechanistically feasible\.Under\-activation\(2 models\) is the hard case: GPT\-OSS\-20B’s direction exists atL20L\_\{20\}but is unread by the decision head, and the single\-layer patch is insufficient\. A*learned activation gate*that decides per item when to patch would test whether circuit\-level intervention can reach the repair corner; the released probe weights, cross\-architecture replication, and a multi\-label extension to multi\-constraint scenarios are the natural next steps\.

## 6Conclusion

We recast hidden\-constraint failure in LLMs as*conditional activation*: the constraint is decodable from hidden state \(K\) symmetrically \(S\) but not reliably routed into the decision \(R\)\. Patching dissociates the two failure modes, and a mitigation frontier shows every prompted intervention inflates conservative bias rather than repairing routing—so the fix must target the routing direction itself\.

## Limitations

#### Scenario count\.

core100stratifies across the HOB cell taxonomy at the cost of statistical power per cell\. Our between\-condition contrasts use the matched paired design to recover power, but per\-cell error bars remain wide for finer\-grained slicing\.

#### Two open weights for the causal claim\.

The K/S/R/P mechanism is established on Qwen3\-14B and GPT\-OSS\-20B only\. While the behavioral signature extends to all 14 models, the causal patching evidence is backed by two open\-weight models, chosen for release\-license reasons and to span the two failure modes\.

#### Single\-layer patching\.

Our Repair condition is satisfied by patching at the probe’s single best layer\. A multi\-layer or block\-wise intervention may reveal repair in models where the single\-layer patch fails \(e\.g\. GPT\-OSS\-20B\), and our analysis therefore underestimates the rate at which Repair holds\.

#### English, single\-domain\.

HOB scenarios are English everyday\-domain\. Cross\-lingual replication and domain\-shifted versions \(e\.g\. medical, legal\) are open questions; we expect the K/S/R/P formalism to transfer but the failure\-mode partition may not\.

#### Prompted\-mitigation coverage\.

We evaluate four prompted strategies that span the published taxonomy of pragmatic\-reasoning interventions\. Reinforcement\-learned mitigations\(Lambert,[2024](https://arxiv.org/html/2608.12321#bib.bib70); Gulcehreet al\.,[2023](https://arxiv.org/html/2608.12321#bib.bib71)\)and fine\-tuned debiasing remain untested\. Our prediction, based on the mediation chain, is that any intervention that operates on the trace surface will fall in the same harm\-without\-repair region; we leave that to empirical verification\.

#### Judge model\.

All 54,400 quartet trials and 128,000 mitigation trials are judged by a single Qwen3\-32B local model with deterministic decoding\. This is a standard choice in the heuristic\-shortcut literature, but a single\-judge evaluation may systematically miss certain failure modes \(e\.g\. correct answers given for the wrong reason\)\. We release the judging prompt and all raw generations so reanalysis with alternative judges is straightforward\.

#### API model versioning\.

Frontier models on closed APIs \(Claude, GPT, Gemini, Grok, Kimi, DeepSeek\) may be updated between data collection \(May 2026\) and publication\. We pin exact model identifiers in Table[6](https://arxiv.org/html/2608.12321#A1.T6)\(Appendix[A](https://arxiv.org/html/2608.12321#A1)\) and release the full raw generations, but pin\-violation risk on closed APIs is non\-zero\.

#### Heuristic prerequisite\-mention detection\.

Our mediation analysis uses content\-word overlap with the canonical hidden constraint to detect “the model mentions the prerequisite\.” This catches cases where the model paraphrases the constraint but may miss implicit references; we report mediation conservatively\.

## References

- Understanding intermediate layers using linear classifier probes\.InInternational Conference on Learning Representations \(Workshop\),Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- Anthropic \(2025\)Claude’s extended thinking\.Note:[https://www\.anthropic\.com/news/visible\-extended\-thinking](https://www.anthropic.com/news/visible-extended-thinking)Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. Belinkov \(2022\)Probing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Berglund, M\. Tong, M\. Kaufmann, M\. Balesni, A\. C\. Stickland, T\. Korbak, and O\. Evans \(2024\)The reversal curse: LLMs trained on “A is B” fail to learn “B is A”\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- B\. Brown, J\. Juravsky, R\. Ehrlich, R\. Clark, Q\. V\. Le, C\. Ré, and A\. Mirhoseini \(2024\)Large language monkeys: scaling inference compute with repeated sampling\.arXiv preprint arXiv:2407\.21787\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2023\)Discovering latent knowledge in language models without supervision\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- X\. Chen, J\. Xu, T\. Liang, Z\. He, J\. Pang, D\. Yu, L\. Song, Q\. Liu, M\. Zhou, Z\. Zhang, R\. Wang, Z\. Tu, H\. Mi, and D\. Yu \(2024\)Do NOT think that much for 2\+3=? on the overthinking of o1\-like LLMs\.arXiv preprint arXiv:2412\.21187\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Conneau, G\. Kruszewski, G\. Lample, L\. Barrault, and M\. Baroni \(2018\)What you can cram into a single $&\!\#\* vector: probing sentence embeddings for linguistic properties\.InACL,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Dai, L\. Dong, Y\. Hao, Z\. Sui, B\. Chang, and F\. Wei \(2022\)Knowledge neurons in pretrained transformers\.InACL,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-R1 incentivizes reasoning in LLMs through reinforcement learning\.Nature645,pp\. 633–638\.External Links:[Document](https://dx.doi.org/10.1038/s41586-025-09422-z)Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Geiger, H\. Lu, T\. Icard, and C\. Potts \(2021\)Causal abstractions of neural networks\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- R\. Geirhos, J\. Jacobsen, C\. Michaelis, R\. Zemel, W\. Brendel, M\. Bethge, and F\. A\. Wichmann \(2020\)Shortcut learning in deep neural networks\.Nature Machine Intelligence2\(11\),pp\. 665–673\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Geva, R\. Schuster, J\. Berant, and O\. Levy \(2021\)Transformer feed\-forward layers are key\-value memories\.InEMNLP,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- C\. Gulcehre, T\. Le Paine, S\. Srinivasan, K\. Konyushkova, L\. Weerts, A\. Sharma, A\. Siddhant, A\. Ahern, M\. Wang, C\. Gu, W\. Macherey, R\. Douc, O\. Firat, and N\. de Freitas \(2023\)Reinforced self\-training \(ReST\) for language modeling\.arXiv preprint arXiv:2308\.08998\.Cited by:[Prompted\-mitigation coverage\.](https://arxiv.org/html/2608.12321#Sx1.SS0.SSS0.Px5.p1.1)\.
- W\. Gurnee and M\. Tegmark \(2024\)Language models represent space and time\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Hanna, O\. Liu, and A\. Variengien \(2023\)How does GPT\-2 compute greater\-than?: interpreting mathematical abilities in a pre\-trained language model\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Heimersheim and N\. Nanda \(2024\)How to use and interpret activation patching\.arXiv preprint arXiv:2404\.15255\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Hewitt and P\. Liang \(2019\)Designing and interpreting probes with control tasks\.InEMNLP\-IJCNLP,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Hewitt and C\. D\. Manning \(2019\)A structural probe for finding syntax in word representations\.InNAACL\-HLT,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. Zhou \(2024\)Large language models cannot self\-correct reasoning yet\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Jo \(2026\)Prompt architecture determines reasoning quality: a variable isolation study on the car wash problem\.arXiv preprint arXiv:2602\.21814\.External Links:[Link](https://arxiv.org/abs/2602.21814)Cited by:[§1](https://arxiv.org/html/2608.12321#S1.p1.1)\.
- S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson, S\. Johnston, S\. El\-Showk, A\. Jones, N\. Elhage, T\. Hume, A\. Chen, Y\. Bai, S\. R\. Bowman, S\. Fort, D\. Ganguli, D\. Hernandez, J\. Jacobson, J\. Kernion, S\. Kravec, L\. Lovitt, K\. Ndousse, C\. Olsson, S\. Ringer, D\. Amodei, T\. Brown, J\. Clark, N\. Joseph, B\. Mann, S\. McCandlish, C\. Olah, and J\. Kaplan \(2022\)Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px4.p1.1)\.
- Kévin \(@knowmadd\) \(2026\)Car wash reasoning test\.Note:Mastodon postExternal Links:[Link](https://mastodon.world/@knowmadd/116072773118828295)Cited by:[§1](https://arxiv.org/html/2608.12321#S1.p1.1)\.
- T\. Kojima, S\. S\. Gu, M\. Reid, Y\. Matsuo, and Y\. Iwasawa \(2022\)Large language models are zero\-shot reasoners\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.12321#S1.SS0.SSS0.Px4.p1.6),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1),[§3\.4](https://arxiv.org/html/2608.12321#S3.SS4.p1.2)\.
- N\. Lambert \(2024\)Reinforcement learning from human feedback: a tutorial\.arXiv preprint arXiv:2404\.12358\.Cited by:[Prompted\-mitigation coverage\.](https://arxiv.org/html/2608.12321#Sx1.SS0.SSS0.Px5.p1.1)\.
- A\. K\. Lampinen, I\. Dasgupta, S\. C\.Y\. Chan, H\. R\. Sheahan, A\. Creswell, D\. Kumaran, J\. L\. McClelland, and F\. Hill \(2024\)Language models, like humans, show content effects on reasoning tasks\.PNAS Nexus3\(7\),pp\. pgae233\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2023\)Inference\-time intervention: eliciting truthful answers from a language model\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Li, L\. Zhang, T\. Jiang, R\. Krishnan, and R\. Padman \(2026\)The model says walk: how surface heuristics override implicit constraints in LLM reasoning\.arXiv preprint arXiv:2603\.29025\.External Links:[Link](https://arxiv.org/abs/2603.29025)Cited by:[Appendix G](https://arxiv.org/html/2608.12321#A7.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2608.12321#S1.SS0.SSS0.Px3.p1.2),[§1](https://arxiv.org/html/2608.12321#S1.p1.1),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2608.12321#S3.SS1.SSS0.Px1.p1.4)\.
- T\. Lieberum, M\. Rahtz, J\. Kramár, N\. Nanda, G\. Irving, R\. Shah, and V\. Mikulik \(2023\)Does circuit analysis interpretability scale? evidence from multiple choice capabilities in Chinchilla\.arXiv preprint arXiv:2307\.09458\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2024\)Let’s verify step by step\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark \(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Marks, C\. Rager, E\. J\. Michaud, Y\. Belinkov, D\. Bau, and A\. Mueller \(2024\)Sparse feature circuits: discovering and editing interpretable causal graphs in language models\.arXiv preprint arXiv:2403\.19647\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- R\. T\. McCoy, E\. Pavlick, and T\. Linzen \(2019\)Right for the wrong reasons: diagnosing syntactic heuristics in natural language inference\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 3428–3448\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1334),[Link](https://aclanthology.org/P19-1334/)Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- I\. Mirzadeh, K\. Alizadeh, H\. Shahrokhi, O\. Tuzel, S\. Bengio, and M\. Farajtabar \(2024\)GSM\-symbolic: understanding the limitations of mathematical reasoning in large language models\.arXiv preprint arXiv:2410\.05229\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. Candès, and T\. Hashimoto \(2025\)S1: simple test\-time scaling\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 20275–20321\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1025),[Link](https://aclanthology.org/2025.emnlp-main.1025/)Cited by:[§1](https://arxiv.org/html/2608.12321#S1.SS0.SSS0.Px4.p1.6),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. Nikankin, A\. Reusch, A\. Mueller, and Y\. Belinkov \(2024\)Arithmetic without algorithms: language models solve math with a bag of heuristics\.arXiv preprint arXiv:2410\.21272\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- OpenAI \(2024\)Learning to reason with LLMs\.Note:[https://openai\.com/index/learning\-to\-reason\-with\-llms/](https://openai.com/index/learning-to-reason-with-llms/)Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Orgad, M\. Toker, Z\. Gekhman, R\. Reichart, I\. Szpektor, H\. Kotek, and Y\. Belinkov \(2025\)LLMs know more than they show: on the intrinsic representation of LLM hallucinations\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- E\. Perez, S\. Ringer, K\. Lukosiute, K\. Nguyen, E\. Chen, S\. Heiner, C\. Pettit, C\. Olsson, S\. Kundu, S\. Kadavath, A\. Jones, A\. Chen, B\. Mann, B\. Israel, B\. Seethor, C\. McKinnon, C\. Olah, D\. Yan, D\. Amodei, D\. Amodei, D\. Drain, D\. Li, E\. Tran\-Johnson, G\. Khundadze, J\. Kernion, J\. Landis, J\. Kerr, J\. Mueller, J\. Hyun, J\. Landau, K\. Ndousse, L\. Goldberg, L\. Lovitt, M\. Lucas, M\. Sellitto, M\. Zhang, N\. Kingsland, N\. Elhage, N\. Joseph, N\. Mercado, N\. DasSarma, O\. Rausch, R\. Larson, S\. McCandlish, S\. Johnston, S\. Kravec, S\. El Showk, T\. Lanham, T\. Telleen\-Lawton, T\. Brown, T\. Henighan, T\. Hume, Y\. Bai, Z\. Hatfield\-Dodds, J\. Clark, S\. R\. Bowman, A\. Askell, R\. Grosse, D\. Hernandez, D\. Ganguli, E\. Hubinger, N\. Schiefer, and J\. Kaplan \(2023\)Discovering language model behaviors with model\-written evaluations\.InFindings of ACL,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px4.p1.1)\.
- F\. Petroni, T\. Rocktäschel, S\. Riedel, P\. Lewis, A\. Bakhtin, Y\. Wu, and A\. Miller \(2019\)Language models as knowledge bases?\.InEMNLP\-IJCNLP,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. R\. Bowman, N\. Cheng, E\. Durmus, Z\. Hatfield\-Dodds, S\. R\. Johnston, S\. Kravec, T\. Maxwell, S\. McCandlish, K\. Ndousse, O\. Rausch, N\. Schiefer, D\. Yan, M\. Zhang, and E\. Perez \(2024\)Towards understanding sycophancy in language models\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px4.p1.1)\.
- F\. Shi, X\. Chen, K\. Misra, N\. Scales, D\. Dohan, E\. H\. Chi, N\. Schärli, and D\. Zhou \(2023\)Large language models can be easily distracted by irrelevant context\.InInternational Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Snell, J\. Lee, K\. Xu, and A\. Kumar \(2025\)Scaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=4FWAwZtd2n)Cited by:[§1](https://arxiv.org/html/2608.12321#S1.SS0.SSS0.Px4.p1.6),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- Z\. Sprague, F\. Yin, J\. D\. Rodriguez, D\. Jiang, M\. Wadhwa, P\. Singhal, X\. Zhao, X\. Ye, K\. Mahowald, and G\. Durrett \(2025\)To CoT or not to CoT? chain\-of\-thought helps mainly on math and symbolic reasoning\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Stolfo, Y\. Belinkov, and M\. Sachan \(2023\)A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis\.InEMNLP,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Vig, S\. Gehrmann, Y\. Belinkov, S\. Qian, D\. Nevo, Y\. Singer, and S\. Shieber \(2020\)Investigating gender bias in language models using causal mediation analysis\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- K\. Wang, A\. Variengien, A\. Conmy, B\. Shlegeris, and J\. Steinhardt \(2023a\)Interpretability in the wild: a circuit for indirect object identification in GPT\-2 small\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Wang, W\. Xu, Y\. Lan, Z\. Hu, Y\. Lan, R\. K\. Lee, and E\. Lim \(2023b\)Plan\-and\-solve prompting: improving zero\-shot chain\-of\-thought reasoning by large language models\.InACL,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023c\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.12321#S1.SS0.SSS0.Px4.p1.6),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1),[§3\.4](https://arxiv.org/html/2608.12321#S3.SS4.p1.2)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.12321#S1.SS0.SSS0.Px4.p1.6),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1),[§3\.4](https://arxiv.org/html/2608.12321#S3.SS4.p1.2)\.
- J\. Wei, D\. Huang, Y\. Lu, D\. Zhou, and Q\. V\. Le \(2023\)Simple synthetic data reduces sycophancy in large language models\.arXiv preprint arXiv:2308\.03958\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px4.p1.1)\.
- Z\. Wu, A\. Geiger, T\. Icard, C\. Potts, and N\. D\. Goodman \(2023\)Interpretability at scale: identifying causal mechanisms in Alpaca\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- F\. Wunderlich \(2026\)Car wash test on 53 leading AI models: “i want to wash my car\. the car wash is 50 meters away\. should i walk or drive?”\.Note:Opper AI blogExternal Links:[Link](https://opper.ai/blog/car-wash-test)Cited by:[§1](https://arxiv.org/html/2608.12321#S1.p1.1)\.
- M\. Xiong, Z\. Hu, X\. Lu, Y\. Li, J\. Fu, J\. He, and B\. Hooi \(2024\)Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Yang, E\. Gribovskaya, N\. Kassner, M\. Geva, and S\. Riedel \(2024\)Do large language models latently perform multi\-hop reasoning?\.InACL,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan \(2023\)Tree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- F\. Zhang and N\. Nanda \(2024\)Towards best practices of activation patching in language models: metrics and methods\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.
- H\. S\. Zheng, S\. Mishra, X\. Chen, H\. Cheng, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2024\)Take a step back: evoking reasoning via abstraction in large language models\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px3.p1.1)\.
- K\. Zhou, J\. D\. Hwang, X\. Ren, and M\. Sap \(2024\)Relying on the unreliable: the impact of language models’ reluctance to express uncertainty\.InACL,Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px4.p1.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. Hendrycks \(2023\)Representation engineering: a top\-down approach to AI transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.12321#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix AModel versions

Table[6](https://arxiv.org/html/2608.12321#A1.T6)pins the exact model identifiers and access modality for all 14 evaluated models\. All API data were collected in May 2026; closed\-API models may be updated by their providers after this date\. Open\-weight models are served locally from the listed Hugging Face checkpoints and are version\-stable\.

ModelProviderIdentifierGPT\-5\.2OpenAIgpt\-5\.2GPT\-5\.4OpenAIgpt\-5\.4Claude Opus 4\.6Anthropicclaude\-opus\-4\-6Claude Sonnet 4\.5Anthropicclaude\-sonnet\-4\-5\-20250929DeepSeek\-R1DeepSeekdeepseek\-reasonerGemini 3 ProGooglegemini\-3\.1\-pro\-previewGrok 4\.2xAIgrok\-4\.20\-beta\-0309\-reasoningKimi K2\.5Moonshotkimi\-k2\.5Llama\-4 ScoutGroqmeta\-llama/llama\-4\-scout\-17b\-16e\-instructGPT\-OSS\-120BGroqopenai/gpt\-oss\-120bQwen3\-14BlocalQwen/Qwen3\-14BQwen3\-32BlocalQwen/Qwen3\-32BQwen3\.5\-27BlocalQwen/Qwen3\.5\-27BGPT\-OSS\-20Blocalopenai/gpt\-oss\-20bTable 6:Exact model identifiers and access modality \(API vs\. local Hugging Face checkpoint\)\. API data collected May 2026\. The judge model is Qwen3\-32B for all trials\. The four local checkpoints \(bottom block\) are also the open weights used for hidden\-state probing and activation patching \(Qwen3\-14B, GPT\-OSS\-20B\)\.
## Appendix BPer\-\(model, strategy\) mitigation frontier

Table[7](https://arxiv.org/html/2608.12321#A2.T7)reports the full10×4=4010\\times 4=40mitigation cells \(10 API models, 4 prompted strategies\) referenced in §[4\.3](https://arxiv.org/html/2608.12321#S4.SS3)\. All values are relative to each model’s zero\-shot baseline on the same 100core100scenarios\.

StrategyModelActGainPairHarmΔ\\DeltaCBIPr⁡\(c∣m\)\\Pr\(\\text\{c\}\\mid\\text\{m\}\)Pr⁡\(c∣¬m\)\\Pr\(\\text\{c\}\\mid\\neg\\text\{m\}\)Pr⁡\(m\)\\Pr\(\\text\{m\}\)*cot*cotclaude\-opus\-4\.6\+0\.033\+0\.033\+0\.404\+0\.404\+0\.437\+0\.4370\.7590\.1820\.928cotclaude\-sonnet\-4\.5\+0\.073\+0\.073\+0\.450\+0\.450\+0\.523\+0\.5230\.7640\.1240\.935cotdeepseek\-r1\+0\.035\+0\.035\+0\.569\+0\.569\+0\.604\+0\.6040\.7860\.2700\.848cotgemini\-3\-pro\+0\.003\+0\.003\+0\.504\+0\.504\+0\.507\+0\.5070\.7660\.3120\.957cotgpt\-5\.2−0\.023\-0\.023\+0\.339\+0\.339\+0\.316\+0\.3160\.7510\.4090\.852cotgpt\-5\.4\+0\.004\+0\.004\+0\.468\+0\.468\+0\.473\+0\.4730\.7670\.3480\.849cotgpt\-oss\-120b\+0\.050\+0\.050\+0\.637\+0\.637\+0\.687\+0\.6870\.6830\.0790\.924cotgrok\-4\.2−0\.003\-0\.003\+0\.607\+0\.607\+0\.604\+0\.6040\.7920\.3650\.839cotkimi\-k2\.5\+0\.001\+0\.001\+0\.424\+0\.424\+0\.424\+0\.4240\.7920\.1570\.942cotllama\-4−0\.057\-0\.057\+0\.353\+0\.353\+0\.295\+0\.2950\.6420\.1240\.927*precondition\_listing*precclaude\-opus\-4\.6\+0\.056\+0\.056\+0\.351\+0\.351\+0\.408\+0\.4080\.7880\.2410\.946precclaude\-sonnet\-4\.5\+0\.089\+0\.089\+0\.399\+0\.399\+0\.489\+0\.4890\.7750\.2920\.944precdeepseek\-r1\+0\.061\+0\.061\+0\.534\+0\.534\+0\.595\+0\.5950\.7750\.2590\.934precgemini\-3\-pro\+0\.004\+0\.004\+0\.404\+0\.404\+0\.408\+0\.4080\.7990\.4470\.959precgpt\-5\.2\+0\.022\+0\.022\+0\.279\+0\.279\+0\.301\+0\.3010\.7710\.4240\.917precgpt\-5\.4\+0\.084\+0\.084\+0\.447\+0\.447\+0\.531\+0\.5310\.7980\.3360\.911precgpt\-oss\-120b\+0\.064\+0\.064\+0\.544\+0\.544\+0\.608\+0\.6080\.7060\.1910\.944precgrok\-4\.2\+0\.057\+0\.057\+0\.566\+0\.566\+0\.623\+0\.6230\.7990\.3060\.927preckimi\-k2\.5\+0\.009\+0\.009\+0\.367\+0\.367\+0\.377\+0\.3770\.8140\.3330\.954precllama\-4−0\.018\-0\.018\+0\.336\+0\.336\+0\.318\+0\.3180\.6910\.1780\.900*goal\_decomposition*goalclaude\-opus\-4\.6\+0\.016\+0\.016\+0\.354\+0\.354\+0\.370\+0\.3700\.7610\.2610\.941goalclaude\-sonnet\-4\.5\+0\.099\+0\.099\+0\.397\+0\.397\+0\.496\+0\.4960\.7840\.2940\.942goaldeepseek\-r1\+0\.039\+0\.039\+0\.565\+0\.565\+0\.604\+0\.6040\.7570\.2640\.912goalgemini\-3\-pro\+0\.030\+0\.030\+0\.493\+0\.493\+0\.523\+0\.5230\.7840\.3530\.964goalgpt\-5\.2\+0\.021\+0\.021\+0\.325\+0\.325\+0\.346\+0\.3460\.7600\.3860\.903goalgpt\-5\.4\+0\.059\+0\.059\+0\.481\+0\.481\+0\.540\+0\.5400\.7790\.2940\.913goalgpt\-oss\-120b\+0\.076\+0\.076\+0\.596\+0\.596\+0\.672\+0\.6720\.6880\.1200\.951goalgrok\-4\.2\+0\.041\+0\.041\+0\.609\+0\.609\+0\.649\+0\.6490\.7940\.2600\.913goalkimi\-k2\.5\+0\.017\+0\.017\+0\.406\+0\.406\+0\.423\+0\.4230\.8090\.2480\.948goalllama\-4−0\.061\-0\.061\+0\.359\+0\.359\+0\.298\+0\.2980\.6440\.1230\.927*counterfactual\_check \(oracle\)*cfclaude\-opus\-4\.6\+0\.023\+0\.023\+0\.323\+0\.323\+0\.345\+0\.3450\.7870\.3000\.916cfclaude\-sonnet\-4\.5\+0\.077\+0\.077\+0\.371\+0\.371\+0\.448\+0\.4480\.7970\.2050\.908cfdeepseek\-r1\+0\.031\+0\.031\+0\.513\+0\.513\+0\.544\+0\.5440\.8010\.2960\.854cfgemini\-3\-pro\+0\.019\+0\.019\+0\.425\+0\.425\+0\.445\+0\.4450\.8100\.3450\.936cfgpt\-5\.2−0\.008\-0\.008\+0\.324\+0\.324\+0\.316\+0\.3160\.7770\.3760\.865cfgpt\-5\.4\+0\.006\+0\.006\+0\.472\+0\.472\+0\.478\+0\.4780\.7720\.3800\.845cfgpt\-oss\-120b\+0\.109\+0\.109\+0\.548\+0\.548\+0\.657\+0\.6570\.7300\.1770\.940cfgrok\-4\.2\+0\.029\+0\.029\+0\.645\+0\.645\+0\.674\+0\.6740\.7960\.2860\.863cfkimi\-k2\.5−0\.003\-0\.003\+0\.405\+0\.405\+0\.402\+0\.4020\.8150\.3020\.912cfllama\-4−0\.082\-0\.082\+0\.339\+0\.339\+0\.257\+0\.2570\.6640\.2120\.845Table 7:Per\-\(strategy, model\) mitigation frontier with mediation detail\.ActGainis the change in Active accuracy relative to zero\-shot;PairHarmis the change in Removed accuracy expressed as CBI inflation;Δ\\DeltaCBI isActGain\+PairHarm;Pr⁡\(c∣m\)\\Pr\(\\text\{c\}\\mid\\text\{m\}\)andPr⁡\(c∣¬m\)\\Pr\(\\text\{c\}\\mid\\neg\\text\{m\}\)are correctness conditional on prerequisite mention;Pr⁡\(m\)\\Pr\(\\text\{m\}\)is the marginal mention rate\.
## Appendix CPer\-budget breakdown

Table[8](https://arxiv.org/html/2608.12321#A3.T8)reports per\-\(model, budget\) Active accuracy, Removed accuracy, CBI, mention rate, and mediated correctness share for the four thinking\-mode models referenced in §[4\.4](https://arxiv.org/html/2608.12321#S4.SS4)\.

Table 8:Per\-budget breakdown of the reasoning\-budget sweep\. B is in tokens; CBI is signed \(\>\+0\.10\>\\\!\+0\.10= over\-activation\)\.
## Appendix DPer\-layer probe accuracy

Table[9](https://arxiv.org/html/2608.12321#A4.T9)reports cross\-validated probe accuracy at every transformer layer for both open\-weight models\. Bold marks the best layer used for downstream routing correlation \(§[4\.5](https://arxiv.org/html/2608.12321#S4.SS5)\) and activation patching \(§[4\.6](https://arxiv.org/html/2608.12321#S4.SS6)\)\.

Table 9:Per\-layer cross\-validated probe accuracy on the Active vs\. Removed binary label, scenario\-grouped 5\-fold\. Standard deviations range from 0\.018 to 0\.052 \(omitted for compactness\)\. Bold = layer chosen for downstream patching\. Qwen3\-14B has 41 layers, GPT\-OSS\-20B has 25; the right column for layers≥25\\geq 25is empty for GPT\-OSS\-20B\.
## Appendix EHint\-ladder details

Fig\.[5](https://arxiv.org/html/2608.12321#A5.F5)reports the per\-model hint\-ladder activation curve with matched negative controls; Fig\.[6](https://arxiv.org/html/2608.12321#A5.F6)shows the per\-level specificity gap\. The activation\-specific component of the hint gain grows monotonically with hint level on every ladder\-complete model\.

![Refer to caption](https://arxiv.org/html/2608.12321v1/x5.png)Figure 5:Hint\-ladder activation curves with matched negative controls on six API models\. Hint accuracy \(solid black\) climbs monotonically; the mean of the three matched controls \(dashed red\) stays flat, demonstrating the gain is hint\-specific rather than salience\-driven\.![Refer to caption](https://arxiv.org/html/2608.12321v1/x6.png)Figure 6:Hint\-ladder specificity gap per level \(hint accuracy minus best matched control\)\. Reaches MEHL threshold \(green dashed\) atL2L\_\{2\}orL3L\_\{3\}for most models\.
## Appendix FPer\-layer probe accuracy curve

Fig\.[7](https://arxiv.org/html/2608.12321#A6.F7)visualizes the per\-layer probe accuracy reported numerically in Table[9](https://arxiv.org/html/2608.12321#A4.T9)\.

![Refer to caption](https://arxiv.org/html/2608.12321v1/x7.png)Figure 7:Per\-layer linear\-probe CV accuracy on the Active vs\. Removed label\. Shaded band is±1​SD\\pm 1\\,\\mathrm\{SD\}over scenario\-grouped folds; red dashed atθK=0\.80\\theta\_\{K\}=0\.80; green dotted marks best layer\.
## Appendix GMethods detail: reproducibility

#### Code and data release\.

All prompts, the salience\-control bank, mitigation prefixes, ladder templates, probe and patching code, and metric implementations will be released\. We seed ladder construction \(deterministic hash byinstance\_id\) and probe training \(numpyseed4242\); all stochasticity is at the model\-API level\. The full quartet, ladder, mitigation, and budget collection is resume\-safe by\(instance\_id,\(\\text\{instance\\\_id\},trial,\\text\{trial\},strategy,\\text\{strategy\},budget\)\\text\{budget\}\)keys, with retry passes that preserve the original prompt prefix and thinking budget so re\-runs are exchangeable with first\-time runs\.

#### Provider\-specific budget routing\.

Anthropic models acceptthinking=\{enabled,budget\_tokens=bb\} withbbclamped to a minimum of1,0241\{,\}024tokens\. OpenAI o\-series and GPT\-5 series acceptreasoning\_effort∈\\in\{low,medium,high\}, mapped frombbvia thresholds10241024and40964096\. Gemini acceptsThinkingConfig\(thinking\_budget=bb\)\. DeepSeek\-R1 emits chain\-of\-thought traces but does not expose a budget knob; the requestedbbis recorded for parity but is a no\-op\.

#### Activation patching mechanics\.

For each scenario, we forward the donor \(Explicit\) prompt and cache the final\-token hidden state at the probe’s best layerℓ⋆\\ell^\{\\star\}via a PyTorch forward hook onmodel\.model\.layers\[ℓ⋆\\ell^\{\\star\}\]\. We then forward the recipient prompt, with a second hook that overwrites the layer\-output tensor at the recipient’s last position with the cached donor activation; logits are extracted at the same position and the gold–shortcut log\-probability gap is recorded\. The gap differenceΔ=Δpatched−Δpre\\Delta=\\Delta^\{\\text\{patched\}\}\-\\Delta^\{\\text\{pre\}\}is reported as the patching effect\.

#### Prerequisite\-mention detection\.

A model response is classified as “mentioning the prerequisite” iff it shares≥2\\geq\\\!2distinct content\-word matches with the canonical hidden constraint string, after lowercasing, stripping punctuation, and filtering a small stoplist of high\-frequency English function words\. This is a conservative, lexical\-overlap definition; it may undercount paraphrastic mentions, which would only strengthen the mediated\-correctness\-share lower bounds reported in §[4\.3](https://arxiv.org/html/2608.12321#S4.SS3)\.

#### Judge\.

All54,400\+128,000\+25,600=208,00054\{,\}400\+128\{,\}000\+25\{,\}600=208\{,\}000trials are judged by Qwen3\-32B with deterministic decoding, using the multiple\-choice extraction prompt fromLiet al\.\([2026](https://arxiv.org/html/2608.12321#bib.bib68)\)\. Trials with API errors at collection are retried with exponential backoff; verdicts for retried trials are recomputed by the same judge to maintain consistency\.