How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?

# How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?
Source: [https://arxiv.org/html/2607.18114](https://arxiv.org/html/2607.18114)
Prakhar Gupta1,2Terry Jingchen Zhang2,5Florent Draye3 Bernhard Schölkopf3,4Zhijing Jin2,3,5 1University of Michigan2Jinesis Lab, University of Toronto & Vector Institute 3Max\-Planck Institute for Intelligent Systems, Tübingen, Germany 4ELLIS Institute Tübingen5EuroSafeAI

###### Abstract

Modern LLMs are alarmingly susceptible to surprisingly simple immaterial changes of input prompts: a casual hint, an incorrectly labeled few\-shot example, or a fake prior assistant turn often flips an originally correct answer\. We study where this susceptibility, spanning sycophancy and related cue\-induced biases, lives inside the model\. Across five model families and seven BCT bias types, we extract a per\-bias direction from hidden states and triangulate it through three measures: probing, leave\-one\-dataset\-out transfer, and causal intervention\. The susceptibility is largely installed by alignment tuning rather than pretraining: pretrained base models barely cave to these biases, and their activations carry no cue\-specific signal beyond question content\. Within aligned models, each bias becomes a single coherent direction that we can both decode and steer along, recovering the unbiased answer across every family we test\. The biases stay representationally distinct, however: cross\-bias entanglement is model\-specific rather than a property of the bias category, and even behaviorally similar biases occupy different directions\. The same intervention also serves as a modest debiasing tool, recovering a meaningful share of bias\-induced errors while preserving most correct answers across all instruct families\. Cue\-induced bias is therefore best understood not as a single flaw in LLMs but as a family of distinct, causally active directions that alignment tuning installs\.

How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue\-Induced Biases in LLMs?

Prakhar Gupta1,2††thanks:Email:prakharg@umich\.edu\.Terry Jingchen Zhang2,5Florent Draye3Bernhard Schölkopf3,4Zhijing Jin2,3,51University of Michigan2Jinesis Lab, University of Toronto & Vector Institute3Max\-Planck Institute for Intelligent Systems, Tübingen, Germany4ELLIS Institute Tübingen5EuroSafeAI

## 1Introduction

Modern LLMs are remarkably brittle to how an input question is asked\. Inserting an offhand “I think the answer is \(C\)” before the question or a fake prior assistant turn often flips a model’s answer on questions it would otherwise get right\(Turpinet al\.,[2023](https://arxiv.org/html/2607.18114#bib.bib7); Sharmaet al\.,[2025](https://arxiv.org/html/2607.18114#bib.bib8); Perezet al\.,[2022](https://arxiv.org/html/2607.18114#bib.bib9)\)\.Chuaet al\.\([2025](https://arxiv.org/html/2607.18114#bib.bib1)\)cataloged a benchmark spanning sycophancy, distractor content, few\-shot patterns, and post\-hoc commitment \(the BCT benchmark\) and demonstrated the susceptibility behaviorally\. The consequence is practical: in any setting where prompts are not tightly controlled, including agentic systems, multi\-turn assistants, search interfaces, and automated evaluations, an LLM’s answer can be quietly steered by these*cue\-induced biases*\.

![Refer to caption](https://arxiv.org/html/2607.18114v1/figures/fig1.png)

![Refer to caption](https://arxiv.org/html/2607.18114v1/x1.png)

Figure 1:Left: per\-bias direction extraction and intervention\.For each \(biased, unbiased\) prompt pair, the model either*resists*the cue \(gives the correct answer\) or*flips*\(caves to the bias’s target letter\)\. We extractdbias=μflipped−μresistedd\_\{\\mathrm\{bias\}\}=\\mu\_\{\\mathrm\{flipped\}\}\-\\mu\_\{\\mathrm\{resisted\}\}from last\-token residual\-stream activations \(here illustrated forSuggested Answer\); subtractingα⋅dbias\\alpha\\cdot d\_\{\\mathrm\{bias\}\}at inference time debiases the model \(§[8](https://arxiv.org/html/2607.18114#S8)\), adding amplifies cue\-following\.Right: scope of the problem\.Per\-\(model, bias\) flip rate: fraction of pairs the model would have answered correctly on the unbiased prompt but flips on when reframed with the bias cue\. OLMo’sPost Hoc\(99%\) leaves no resisted contrast for LODO testing \(§[5](https://arxiv.org/html/2607.18114#S5)\)\.Behavioral characterization tells us that this happens\. It does not tell us where the susceptibility lives inside the model\. Recent interpretability work has shown that single behaviors \(refusal, sycophancy, true/false belief\) can be isolated as linear directions in hidden\-state space and steered along causally\(Panicksseryet al\.,[2024](https://arxiv.org/html/2607.18114#bib.bib2); Marks and Tegmark,[2024](https://arxiv.org/html/2607.18114#bib.bib3); Zouet al\.,[2025](https://arxiv.org/html/2607.18114#bib.bib17); Liet al\.,[2024](https://arxiv.org/html/2607.18114#bib.bib16)\)\. But three questions remain open about cue\-induced biases specifically: \(i\) whether the susceptibility is already present in the pretrained network or is installed by alignment tuning, \(ii\) whether the per\-bias direction construction generalizes across families and across bias types, and \(iii\) whether different biases in the same model share internal structure or remain separate\.

We answer all three by extracting a direction for each \(bias, dataset\) source as the normalized difference of means between hidden states where the model caved to the bias and those where it resisted, the standard contrastive\-activation\-addition construction\(Panicksseryet al\.,[2024](https://arxiv.org/html/2607.18114#bib.bib2); Marks and Tegmark,[2024](https://arxiv.org/html/2607.18114#bib.bib3)\)\. The direction simultaneously functions as a probe \(project to predict flipping\) and a steering vector \(subtract to suppress the bias, add to amplify it\)\. For the origin question, base models pose a problem because they almost never flip and so produce no within\-model flipped/resisted contrast to learn from\. We solve this with a base\-probe protocol that borrows the aligned model’s flipped/resisted labels and asks whether the same distinction is decodable in the base model’s activation space \(§[6](https://arxiv.org/html/2607.18114#S6)\)\.

We study seven BCT bias types111Suggested Answer,Distractor Argument,Distractor Fact,Wrong Few\-Shot,Spurious Few\-Shot Squares,Spurious Few\-Shot Hindsight\(singleton\), andPost Hoc\. We excludePositional Bias\(LLM\-as\-judge format\) andAre You Sure\(5\-turn structure with an embedded reasoning paragraph incompatible with our non\-CoT setup\)\. Detailed definitions are in Appendix[A\.1](https://arxiv.org/html/2607.18114#A1.SS1)\.across five model families \(Llama\-3\.1\-8B,Qwen\-2\.5\-7B,Gemma\-2\-9B,Mistral\-7B\-v0\.3,OLMo\-2\-7B\), using both the base and instruct checkpoints of each\. The story that emerges has three parts\.

#### Origin\.

Four of five pretrained base models flip on under 5% as many pairs as their instruct counterparts, and in their activations the cue\-specific bias signal is essentially absent once we control for question content \(§[6](https://arxiv.org/html/2607.18114#S6)\)\. The biases that are not instruction\-like \(distractor content, few\-shot patterns, post\-hoc anchoring\) are more absent from base activations than the one instruction\-like bias \(Suggested Answer\), arguing against the simpler reading that we are just measuring sensitivity to instructions\. Alignment tuning, not pretraining, is where the susceptibility comes from\.

#### Per\-bias direction\.

Across 7 biases and 5 instruct families, the within\-bias direction transfers to held\-out datasets at per\-model mean AUROC of 0\.69–0\.82, with every test point beating a random direction atp<10−6p<10^\{\-6\}\(§[5](https://arxiv.org/html/2607.18114#S5)\)\. The signal also localizes consistently: when we extract the direction one layer at a time, AUROC peaks in the late\-middle layers \(relative depth0\.550\.55–0\.740\.74\) in every architecture we test, with the single best layer beating the all\-layer probe\.

![Refer to caption](https://arxiv.org/html/2607.18114v1/x2.png)Figure 2:TheSuggested Answerdirection separates flipped from resisted at the data level, in every family\.2D view of last\-token hidden states \(4 datasets pooled per panel, at each model’sSuggested Answer\-specific best layer; Appendix[B\.1](https://arxiv.org/html/2607.18114#A2.SS1)\)\. X\-axis: projection onto the per\-biasdbiasd\_\{\\mathrm\{bias\}\}\(Fig\.[1](https://arxiv.org/html/2607.18114#S1.F1)\); y\-axis: top PC of the residual orthogonal todbiasd\_\{\\mathrm\{bias\}\}\.Flippedred,resistedblue; black arrow marks thedbiasd\_\{\\mathrm\{bias\}\}axis, dashed line is the midpoint of class means along it\. Held\-out LODO AUROCs forSuggested Answerare in Table[1](https://arxiv.org/html/2607.18114#S5.T1)\(0\.760\.76–0\.880\.88\)\.
#### Distinctness\.

The tight cross\-bias “cluster” visible on Qwen is a Qwen\-specific quirk, moderate on Gemma, weak on Mistral, and absent on Llama and OLMo \(§[7](https://arxiv.org/html/2607.18114#S7)\)\. And even behaviorally similar biases need not share a representation:Post HocandSuggested Answerare both answer\-anchoring on the surface but occupy different directions inside the network, and are anti\-aligned on Llama\.

The same direction also doubles as a debiasing knob \(§[8](https://arxiv.org/html/2607.18114#S8)\): at a per\-model strength chosen to keep correct answers correct \(≥90%\\geq 90\\%\), subtracting the direction from the residual stream recovers77–20%20\\%of bias\-induced errors across all five instruct families, versus under5%5\\%for a random direction at matched strength\.

## 2Related Work and the Research Gap

#### Behavioral characterization is well\-established, mechanistic characterization is not\.

Chuaet al\.\([2025](https://arxiv.org/html/2607.18114#bib.bib1)\)introduced BCT and demonstrated that LLMs cave to a range of prompt manipulations spanning sycophancy, distractor content, few\-shot patterns, and post\-hoc commitment\. Sycophancy in particular has been studied extensively\(Sharmaet al\.,[2025](https://arxiv.org/html/2607.18114#bib.bib8); Perezet al\.,[2022](https://arxiv.org/html/2607.18114#bib.bib9)\), andTurpinet al\.\([2023](https://arxiv.org/html/2607.18114#bib.bib7)\)showed chain\-of\-thought explanations are often unfaithful to the underlying decision\. This line of work tells us that models exhibit the behavior\. It does not address where the susceptibility lives inside the network or whether it differs across model families\.

#### Previous interpretability work is largely on one behavior/model\.

A growing literature shows that single behaviors can be isolated as linear directions in hidden\-state space\.Panicksseryet al\.\([2024](https://arxiv.org/html/2607.18114#bib.bib2)\)extract a per\-persona steering direction \(Contrastive Activation Addition\)\.Marks and Tegmark \([2024](https://arxiv.org/html/2607.18114#bib.bib3)\)show that true/false statements lie along a linear direction\.Hendelet al\.\([2023](https://arxiv.org/html/2607.18114#bib.bib4)\)find that in\-context tasks are compressed into single task vectors\.Liet al\.\([2024](https://arxiv.org/html/2607.18114#bib.bib16)\)show inference\-time intervention can elicit truthful answers\.Burnset al\.\([2024](https://arxiv.org/html/2607.18114#bib.bib5)\)discover latent knowledge unsupervisedly\.Templetonet al\.\([2024](https://arxiv.org/html/2607.18114#bib.bib6)\)extract interpretable features via sparse autoencoders, andZouet al\.\([2025](https://arxiv.org/html/2607.18114#bib.bib17)\)frame representation engineering more broadly\. These works establish that per\-behavior directions exist\. They typically focus on one behavior in one model and do not ask whether the same construction generalizes across a benchmark of related biases, whether multiple biases share representational structure, or whether the resulting directions are inherent to the pretrained network or installed by alignment tuning\.

#### The gap we fill\.

No prior work, to our knowledge, has systematically asked, across multiple bias types and multiple model families: \(i\) where the per\-bias direction comes from \(pretraining vs\. post\-training\), \(ii\) whether different biases occupy the same or distinct directions, and \(iii\) whether the direction construction holds universally across model families\. Filling this gap matters\. If per\-bias probing methods claim selectivity without verifying it cross\-model, the field is overstating what those probes measure, and if alignment tuning is the source of the susceptibility, that has direct implications for safety\-tuning recipes\.

#### Our contribution\.

A systematic 5\-family×\\times7\-bias triangulation \(probing\+\+LODO transfer\+\+causal intervention\) plus a new base\-probe protocol that compares base\-vs\-instruct*representations*\(not just behaviors\)\. The result is a three\-part picture \(origin, category, distinctness\) that re\-centers what “cue\-induced bias” means inside the model\.

## 3Methodology

#### Data and bias types\.

We study the 7 BCT biases\(Chuaet al\.,[2025](https://arxiv.org/html/2607.18114#bib.bib1)\)that fit a single\-token MCQ pipeline:Suggested Answer,Distractor Argument,Distractor Fact,Wrong Few\-Shot,Spurious Few\-Shot Squares,Spurious Few\-Shot Hindsight\(a singleton with only one source dataset\), andPost Hoc\. We excludePositional Bias\(LLM\-as\-judge format, not MCQ\) andAre You Sure\(a 5\-turn dialogue with a baked\-in fake reasoning paragraph\)\. Each non\-singleton bias has four source datasets \(MMLU, HellaSwag, LogiQA, TruthfulQA\)\. Detailed bias definitions are in Appendix[A\.1](https://arxiv.org/html/2607.18114#A1.SS1)\.

#### Non\-CoT, single\-letter prompting\.

Throughout the paper we work in a*non\-CoT*setup: instead of letting the model generate a chain\-of\-thought, we modify the user message that contains the CoT instruction \(the final user turn in every BCT prompt\) to ask for a direct single\-letter answer \(appending: “*Answer with just the single letter \(A, B, C, D,…\\ldots\), no explanation or reasoning\.*”\), forward\-pass the prompt, and read the next\-token logits over the 16 letter tokens A–P, summing probability mass across bare\-letter and space\-prefixed variants\. This isolates the bias’s effect on the answer itself, separate from any reasoning the model would produce\.

#### Flipped vs\. resisted pairs\.

For each \(biased, unbiased\) pair, letttbe the bias target letter,ccthe ground\-truth letter,aba\_\{b\}the model’s argmax letter under the biased prompt, andaua\_\{u\}under the unbiased prompt\. A pair isflippedifau=ca\_\{u\}=candab=ta\_\{b\}=t\(the model knew the answer but caved to the bias’s letter\) andresistedifau=ab=ca\_\{u\}=a\_\{b\}=c\(the model held the right answer\)\. We drop pairs wheret=ct=c\(the bias coincidentally points to the truth\) and pairs where the model did not know the unbiased answer\. Pair\-count statistics are in Appendix[A\.2](https://arxiv.org/html/2607.18114#A1.SS2)\.

#### Per\-source and per\-bias directions\.

For each \(bias, dataset\) sourcess, we collect the last\-token hidden state of every transformer layer on each biased prompt, concatenated intoXs∈ℝL​HX\_\{s\}\\in\\mathbb\{R\}^\{LH\}\. The per\-source direction is the standard difference\-of\-means construction\(Panicksseryet al\.,[2024](https://arxiv.org/html/2607.18114#bib.bib2); Marks and Tegmark,[2024](https://arxiv.org/html/2607.18114#bib.bib3)\):

ds=μsflip−μsrsst‖μsflip−μsrsst‖2\.d\_\{s\}=\\frac\{\\mu\_\{s\}^\{\\text\{flip\}\}\-\\mu\_\{s\}^\{\\text\{rsst\}\}\}\{\\\|\\mu\_\{s\}^\{\\text\{flip\}\}\-\\mu\_\{s\}^\{\\text\{rsst\}\}\\\|\_\{2\}\}\.\(1\)The per\-bias directiondBd\_\{B\}is the unit\-normalized mean ofdsd\_\{s\}acrossBB’s source datasets, treating each dataset equally\.

#### Probing AUROC\.

Given a directionddand held\-out pairs\{\(Xi,yi\)\}\\\{\(X\_\{i\},y\_\{i\}\)\\\}\(flipped vs\. resisted\), we scoresi=Xi⋅ds\_\{i\}=X\_\{i\}\\cdot dand report AUROC\. The probing random baseline is the mean AUROC ofK=50K=50random unit\-norm directions of matched dimension \(per\-layer probing usesK=20K=20, Appendix[B\.1](https://arxiv.org/html/2607.18114#A2.SS1)\)\. The intervention random baseline \(§[5](https://arxiv.org/html/2607.18114#S5), §[8](https://arxiv.org/html/2607.18114#S8)\) uses a single random unit\-norm direction per source \(seed 42\) at the sameα\\alphaas the real direction\.

#### Base\-probe protocol\.

The origin question \(§[6](https://arxiv.org/html/2607.18114#S6)\) cannot be asked directly on base models, since they rarely flip and so produce no within\-model flipped/resisted contrast to learn a direction from\. We instead*transfer*labels: for each source, we identify which pairs the*aligned*model flipped or resisted on, feed the base model the exact same prompts, record the base activations, and ask whether the aligned\-model\-defined distinction is decodable in base\-activation space\. To separate a real cue\-specific signal from question\-content variability \(perhaps the aligned\-model\-flipped questions are simply harder\), we run the analogous probe on the*unbiased*version of each prompt as a control and report the paired biased\-minus\-unbiased AUROC gap per source\.

## 4Experimental Setup

#### Models\.

Five instruction\-tuned 7–9B models from different families:Llama\-3\.1\-8B\-Instruct\(Grattafioriet al\.,[2024](https://arxiv.org/html/2607.18114#bib.bib10)\)\(32L/4096H\),Qwen2\.5\-7B\-Instruct\(Qwenet al\.,[2025](https://arxiv.org/html/2607.18114#bib.bib11)\)\(28L/3584H\),Gemma\-2\-9B\-it\(42L/3584H, bf16 \+ eager attention required to preserve attention soft\-capping\),Mistral\-7B\-Instruct\-v0\.3\(32L/4096H\), andOLMo\-2\-1124\-7B\-Instruct\(32L/4096H, bf16\)\. For the origin experiment \(§[6](https://arxiv.org/html/2607.18114#S6)\) we also extract activations from each family’s matching pretrained base checkpoint \(Llama\-3\.1\-8B,Qwen2\.5\-7B,Gemma\-2\-9B,Mistral\-7B\-v0\.3,OLMo\-2\-1124\-7B\), each reusing its instruct tokenizer so that base\-vs\-instruct prompts are byte\-identical and the only differing factor is the weights\.Qwen2\.5\-7B\-baseis the exception that caves to biases at instruct\-like rates and is therefore also kept in the main analyses as a behavioral contrast\.

#### Datasets\.

MMLU\(Hendryckset al\.,[2021](https://arxiv.org/html/2607.18114#bib.bib12)\), HellaSwag\(Zellerset al\.,[2019](https://arxiv.org/html/2607.18114#bib.bib13)\), LogiQA\(Liuet al\.,[2020](https://arxiv.org/html/2607.18114#bib.bib14)\), TruthfulQA\(Linet al\.,[2022](https://arxiv.org/html/2607.18114#bib.bib15)\), plus Hindsight Neglect\(McKenzieet al\.,[2024](https://arxiv.org/html/2607.18114#bib.bib18)\)for the singleton bias\.

#### Decoding\.

Single\-token greedy\. We extract only the last\-token hidden state at the answer position and the next\-token logits over the 16 letter tokens A–P\. No generation beyond a single token\.

## 5Each Bias Is a Single Causal Direction

Before we can ask where the bias comes from or how multiple biases relate, we first need to show the per\-bias direction is a real, generalizable object\. Across 7 biases and 5 instruct families, three measures agree\. The direction decodes the bias from held\-out data, it causally controls the answer when used as a steering vector, and it lives in the same part of every network we look at\.

For decoding, we hold one dataset out, average the per\-source directions on the remainingN−1N\-1, and project the held\-out pairs onto the resulting LODO direction\. Across 116 \(bias, dataset, model\) test points,2226 biases×\\times4 datasets across 4 instruct families, plus 5 biases×\\times4 datasets onOLMo\-2\-7B\(itsPost Hocis not LODO\-testable because the model caves at∼\\sim99%\)\.every single point beats its random\-direction baseline \(Wilcoxon paired one\-sidedp<10−6p<10^\{\-6\}on every model\), with per\-model mean LODO AUROC of 0\.69–0\.82 \(Table[1](https://arxiv.org/html/2607.18114#S5.T1)\)\.

Table 1:The direction generalizes across datasets in every family\.Mean LODO AUROC\. Bold marks each model’s strongest bias\. “—” = OLMo’sPost Hoccannot be LODO\-tested\. All non\-blank cells beat random atp<10−6p<10^\{\-6\}\.For causality, we use the same LODO direction as a steering vector\. We subtractα⋅dℓ\\alpha\\cdot d\_\{\\ell\}from every layer’s last\-token hidden state at the answer position\. On flipped pairs, the fraction that now produce the correct answer is the recovery rate\. At each model’s operatingα\\alpha\(§[8](https://arxiv.org/html/2607.18114#S8)\), the real direction recovers 7–20% of flipped pairs while a random direction at matchedα\\alpharecovers under 5%, a 4–31×\\timesdirection\-specific effect in every model we test \(all five instruct families andQwen\-base\)\. The direction is not just a probe, it is a causal handle\.

And for localization, splitting the all\-layer direction into per\-layer chunks and re\-running LODO at each layer reveals a striking cross\-architecture regularity\. In every instruct family the bias signal peaks at relative network depth 0\.55–0\.74 \(Figure[3](https://arxiv.org/html/2607.18114#S5.F3)\)\.Qwen\-base, included as a base\-model control, peaks later and weaker, which is consistent with the bias representation being primarily an alignment\-tuning installation \(§[6](https://arxiv.org/html/2607.18114#S6)\)\.

![Refer to caption](https://arxiv.org/html/2607.18114v1/x3.png)Figure 3:Bias signal peaks at relative depth 0\.55–0\.74 in every instruct family\.Mean LODO AUROC at each transformer layer, plotted against relative depth so models of different depth align\. The orange band marks the consensus peak range\. PretrainedQwen\-base\(dashed gray\) peaks later and weaker, consistent with the bias representation being primarily an alignment\-tuning installation\.Every bias we tested is a single low\-dimensional, decodable, causally active direction, in every model family\. This is the object the rest of the paper interrogates\.

## 6Where Does the Bias Come From?

§[5](https://arxiv.org/html/2607.18114#S5)showed that the bias is a coherent causal direction inside aligned models\. We now ask whether this representation was already present in the pretrained network or whether alignment tuning is what installs it\. If it is the latter, the susceptibility documented by prior behavioral work is a property of post\-training rather than of LLMs in general\.

We start with behavior\. For each family we run the same extraction pipeline on the matching pretrained base checkpoint, reusing the aligned model’s tokenizer so prompts are byte\-identical and the only thing that differs is the weights\. Four of five base models flip on roughly 0\.2–4% as many pairs as their aligned counterparts \(Table[2](https://arxiv.org/html/2607.18114#S6.T2)\)\.Qwen\-2\.5\-7B\-baseis the lone exception and we keep it in the analysis as a behavioral contrast\.

Table 2:Pretrained base models almost never follow cue\-induced biases\.Strict\-flipped pair counts across all 25 \(bias, dataset\) sources\.Qwen\-baseis the outlier\.Behavior alone is not enough: a base model could in principle encode the bias direction internally without acting on it\. The base\-probe \(§[3](https://arxiv.org/html/2607.18114#S3.SS0.SSS0.Px6)\) tests this directly\. We take the prompts where the aligned model flipped or resisted, feed them to the base model, and ask whether base activations alone can tell the two groups apart\. Some discriminability is expected just from question difficulty \(flipped questions may be intrinsically harder\), so we run the same probe on the*unbiased*version of each prompt as a control\. The gap between the two AUROCs is the bias\-cue\-specific signal\.

In base activations that gap is small\. Per\-model paired gaps \(biased AUROC−\-unbiased AUROC\) in the four non\-outlier base families range from\+0\.01\+0\.01to\+0\.09\+0\.09, significant in three \(Llama, Gemma, OLMo\) and indistinguishable from zero on Mistral \(Figure[4](https://arxiv.org/html/2607.18114#S6.F4)\);Qwen\-base, the documented behavioral outlier \(Table[2](https://arxiv.org/html/2607.18114#S6.T2)\), shows a slightly negative gap \(−0\.03\-0\.03, n\.s\.\)\. For comparison, the within\-bias direction’s lift over random in instruct models is\+0\.20\+0\.20to\+0\.31\+0\.31, three to five times larger\. And the weak base signal that does exist does not even point in the same direction as the instruct one: per\-family mean cosine with the aligned direction is only0\.040\.04–0\.100\.10in those four families \(Qwen\-basereaches0\.190\.19, again reflecting its anomalous behavior\)\. The direction is therefore not literally absent from base models, but alignment tuning amplifies it three\- to fivefold and reorients it into a coherent, causally active axis\.

![Refer to caption](https://arxiv.org/html/2607.18114v1/x4.png)Figure 4:Alignment tuning installs the bias representation\. Base activations carry almost none of it\.Per model, three LODO AUROC values on one axis: \(white\) the unbiased\-prompt probe on base activations, a question\-content control, \(gray\) the biased\-prompt probe on base activations, asking whether the bias cue adds anything, and \(blue\) the same biased\-prompt probe on instruct activations\. White and gray sit near each other in every family, meaning the bias cue barely shifts base activations beyond question content\. Blue is far to the right, meaning the bias direction is present in instruct activations\. Numerical gaps and significance in Table[3](https://arxiv.org/html/2607.18114#S6.T3)\.One could push back that this only shows alignment tuning installs*instruction sensitivity*, not a bias\-specific representation\. Of our seven biases onlySuggested Answeris instruction\-like \(an explicit suggestion in the user message\)\. The rest insert distractor content, manipulate few\-shot examples, or fake an assistant turn, none of which is an instruction\. If alignment tuning were merely adding instruction sensitivity,Suggested Answershould be the one bias with a strong base\-activation trace, and the rest should look uniformly weak\. The data show the reverse \(Table[3](https://arxiv.org/html/2607.18114#S6.T3)\): in aligned models all seven biases are similarly decodable and causal, but in base activations onlySuggested Answerhas a clearly nonzero cue gap \(\+0\.17\+0\.17\)\. The five non\-instruction biases sit near the question\-content baseline \(\+0\.03\+0\.03on average\)\. The abstract biases are more absent from base than the instruction\-like one, the reverse of what an instruction\-sensitivity account would predict\.

Table 3:Rebutting the “it’s just instruction\-following” critique\.The instruction\-like bias is the only one with a meaningful base\-activation trace\. If the effect were instruction\-following, the pattern would be reversed\. Means are over the four non\-outlier instruct families and their matching base\-xlabel runs \(Qwen\-base excluded; its anomalous flipping behavior, see Table[2](https://arxiv.org/html/2607.18114#S6.T2), makes including it inappropriate for this rebuttal\-table comparison\)\.Cue\-induced bias is largely installed by alignment tuning, not pretraining\. Pretrained base models barely follow these biases behaviorally, and inside their activations the cue\-specific representation is absent for the abstract biases and only a faint lexical\-cue trace for the one instruction\-like one\.

## 7The Biases Stay Distinct

Given each bias has a coherent causal direction, a natural follow\-up is whether those directions overlap\. If two biases share a direction, the per\-bias framing is misleading because they would be facets of one mechanism\. We measure this geometrically \(cosines\) and functionally \(cross\-bias probing transfer\), and run a behavioral\-similarity stress test\.

On Qwen, the per\-bias cosine matrix shows a tight cluster of four biases \(Suggested Answer,Distractor Fact,Wrong Few\-Shot,Spurious Few\-Shot Squares\) whose pairwise cosines all fall in\[0\.72,0\.85\]\[0\.72,0\.85\]\. Whether this pattern is a property of cue\-induced biases in general or a quirk of Qwen turns out to matter: Table[4](https://arxiv.org/html/2607.18114#S7.T4)shows it does not generalize\. The same four biases cluster strongly on Qwen \(p=2\.5×10−38p=2\.5\\\!\\times\\\!10^\{\-38\}, rank 1 of 35 possible 4\-subsets\), weakly but significantly on Gemma \(p=10−3p=10^\{\-3\}, rank 3 of 15\) and Mistral \(p=0\.02p=0\.02, rank 6 of 35\), and not at all on Llama or OLMo\. Whether biases share representation is therefore a property of the specific model rather than of the bias category\. Per\-bias cosine matrices for all five families are in Appendix[C\.1](https://arxiv.org/html/2607.18114#A3.SS1)\(Figure[7](https://arxiv.org/html/2607.18114#A3.F7)\)\.

Table 4:The 4\-bias cluster is Qwen\-specific, not universal\.Within vs\. outside cosines for the candidate cluster\. Rank is among all 4\-bias subsets\.A clean stress test on the distinctness side:Post Hoc\(the model is shown its apparent prior answer\) andSuggested Answer\(the user inserts “I think it’s X”\) are both*answer\-anchoring*behaviorally\. Are they the same direction? In no family\. The pairwise cosines \(−0\.25\-0\.25Llama,\+0\.34\+0\.34Qwen,\+0\.40\+0\.40Gemma,\+0\.12\+0\.12Mistral,\+0\.33\+0\.33OLMo\) sit at or below each model’s own off\-diagonal average in 4 of 5 families, and are anti\-aligned on Llama\. Different sources of anchoring produce different mechanisms inside the network\.

#### Cross\-bias intervention transfer\.

The geometric picture has a functional correlate: if we use biasAA’s direction to debias biasBB\(A≠BA\\neq B\), some of the recovery transfers\. For this cross\-bias experiment we use the within\-bias recovery\-maximizing strength per model \(α=0\.5\\alpha=0\.5on Llama,α=4\\alpha=4on Qwen; differs from Table[5](https://arxiv.org/html/2607.18114#S8.T5)’s preservation\-constrained operatingα\\alpha\)\. On Llama, off\-diagonal recovery averages 21\.9% versus 29\.2% diagonal \(a target bias’s own direction recovers about 1\.3×\\timesbetter than a cross\-bias direction\), and 0\.8% for a matched random direction\. On Qwen, off\-diagonal recovery is 13\.3% versus 18\.5% diagonal \(1\.4×\\timesbetter\) and 4\.4% random\. So directions are not orthogonal in function either, consistent with the moderate off\-diagonal cosines in the per\-bias matrix\. But the diagonal still wins on both models tested: each bias has its own most\-effective direction\. Full transfer matrices are in Appendix[C\.2](https://arxiv.org/html/2607.18114#A3.SS2)\.

The biases stay representationally distinct: cross\-bias entanglement, when it appears, is a property of the model’s geometry rather than of the bias category, and even behaviorally similar biases can occupy different directions\.

## 8Direction as a Practical Debiasing Tool

§[5](https://arxiv.org/html/2607.18114#S5)used the intervention to prove the bias direction is causal: subtracting it from the residual stream changes the model’s answer\. That was a yes/no test at a fixed strength\. This section asks a different question: as a deployable knob, how much bias can we actually remove, and at what cost to correct answers? Same hook on the same direction, but a different success metric\. We now sweep the steering strengthα\\alphaand pick a per\-model operating point\.

#### Operating point\.

We sweepα∈\{0\.1,0\.25,0\.5,1,2,4,8\}\\alpha\\in\\\{0\.1,0\.25,0\.5,1,2,4,8\\\}and, for each model, pick theα\\alphathat maximizes mean recovery on flipped pairs subject to mean preservation on already\-correct resisted pairs staying≥90%\\geq 90\\%\. One globalα\\alphaper model: a deployable knob\.

Table 5:Same intervention, modest debiasing knob with a transparent tradeoff\.For each model,α∗\\alpha^\{\*\}maximizes mean recovery \(Rec\.\) subject to mean preservation \(Pres\.\)≥90%\\geq 90\\%\. “vs rand” = ratio of real\- to random\-direction recovery at matchedα\\alpha\.
#### Selectivity on unbiased prompts\.

A natural worry is that subtracting the bias direction damages general capability, not just the bias\. Applying the intervention at operatingα\\alphato the unbiased version of each held\-out pair \(same questions, bias cue stripped\) keeps accuracy at 94–98% across all 5 instruct families, versus 99% under a random direction at matchedα\\alpha: the intervention is direction\-specific and clean accuracy stays high\.

#### Per\-bias picture\.

Recovery is not uniform: content and anchoring biases \(Distractor Fact,Post Hoc,Suggested Answer\) recover at∼\\sim15–20%, format\-pattern biases \(Wrong Few\-Shot,Distractor Argument\) at∼\\sim7–13%\. A natural follow\-up question \(whether biasAA’s direction can debias biasBBwhereA≠BA\\neq B\) is covered in §[7](https://arxiv.org/html/2607.18114#S7)\(“Cross\-bias intervention transfer”\), since the answer \(some transfer, but the diagonal always wins\) speaks to representational distinctness rather than to the deployable knob\.

The same causal handle that proves the bias representation exists also reduces it, modestly: at a sensibly\-chosenα\\alpha, it recovers 7–20% of bias\-induced errors while preserving≥90%\\geq 90\\%of correct answers across all 5 instruct families\. The intervention is bidirectional, though the two directions are asymmetric: subtracting along the bias direction \(positiveα\\alpha\) lowers the bias rate by up to 42 percentage points, while pulling toward the bias mean \(negativeα\\alpha\) raises it by a smaller margin \(up to 7 points, Figure[5](https://arxiv.org/html/2607.18114#S8.F5)\)\. The recovery\-vs\-preservation curves underlying the operating points are in Appendix[D\.2](https://arxiv.org/html/2607.18114#A4.SS2)\(Figure[10](https://arxiv.org/html/2607.18114#A4.F10)\)\.

![Refer to caption](https://arxiv.org/html/2607.18114v1/x5.png)Figure 5:Steering along the bias direction adds or removes bias\.Per model, the achievable range ofP​\(bias​target\)P\(\\mathrm\{bias\\ target\}\)when sweeping intervention strengthα∈\[−2,2\]\\alpha\\in\[\-2,2\]:debiased min\(α\>0\\alpha\>0, push away from the bias mean\),baseline\(α=0\\alpha=0, no intervention\), andinduced max\(α<0\\alpha<0, push toward the bias mean\)\. The direction is causal in both directions, and the effect is largest on the models with the highest baseline bias rates\.

## 9Discussion

The literature largely treats cue\-induced bias as a general failure mode of LLMs, and per\-bias probing methods proceed as if each bias has its own dedicated machinery\. Both deserve revision: the susceptibility lives almost entirely in instruct models, and whether different biases share a direction is a property of the specific model, not of the bias category\. Per\-bias selectivity is an empirical model\-specific property, not a guarantee of the construction\. The base\-probe \(borrow instruct labels, decode from base activations\) is a natural recipe for sanity\-checking “inherent to the model” vs\. “installed by tuning” for any behavioral phenomenon\.

Mechanistically, the asymmetry in Table[3](https://arxiv.org/html/2607.18114#S6.T3)is suggestive: in base activations only the explicit\-lexical\-cue bias \(Suggested Answer\) leaves any trace, and the more abstract ones essentially none\. One reading is that pretraining gives the model the ability to register an explicit cue lexically, and alignment tuning333“Alignment tuning” here covers whatever post\-training \(SFT, RLHF, DPO, or combinations\) takes a base checkpoint to its instruct counterpart\. We do not ablate which step installs the direction; our claim isolates only that it is installed somewhere between base and instruct\.installs the broader “follow the user’s framing” competence while reshaping that lexical trace into a coherent, causally active direction\. Distinguishing this from alternatives \(e\.g\., a single “follow context” direction that all biases partly load onto\) requires controlled fine\-tuning experiments we leave to future work\.

A practical implication is that cue\-induced bias should not be treated merely as a prompt\-engineering nuisance, or as a fixed weakness of the pretrained model\. If the relevant directions are installed or strongly amplified during post\-training, then mitigation should target alignment recipes themselves, not only inference\-time defenses\. Future alignment procedures could explicitly monitor whether instruction tuning increases sensitivity to irrelevant cues, spurious demonstrations, or fabricated conversational history\. In this sense, cue\-induced bias provides a useful diagnostic for post\-training: a model can become more helpful and instruction\-following while also becoming more prone to over\-weighting the user’s framing\.

## 10Conclusion

Across seven BCT cue\-induced biases and five model families, three converging measures \(probing, LODO transfer, causal intervention\) give one picture\. Each bias is a single coherent, decodable, causally active direction\. Alignment tuning installs the representation, and the per\-bias directions stay distinct, with cross\-bias entanglement a Qwen\-specific quirk rather than a property of the bias category\. The same intervention, properly tuned, doubles as a modest debiasing knob\. Cue\-induced bias is not an inherent LLM flaw but a family of causally active directions that alignment tuning installs\.444Code and experiment artifacts:[https://github\.com/prakharg55/bias\-direction\-EMNLP](https://github.com/prakharg55/bias-direction-EMNLP)

## Limitations

#### Non\-CoT only\.

We measure the hidden state at the moment of single\-token answering\. Many real\-world bias manifestations involve longer reasoning chains\. Our findings may not transfer to CoT scenarios where bias\-following emerges during intermediate reasoning, and a CoT\-pilot extension we ran \(not included here\) suggests the within\-bias direction in the CoT setting is much weaker and largely orthogonal to the non\-CoT one, a question for follow\-up work\.

#### Post Hocon OLMo is unfillable\.

OLMo\-2\-7B\-Instructcaves toPost Hocat∼\\sim99% across datasets, leaving no LODO\-testable resisted contrast in 3 of 4 datasets \(two datasets have zero resisted pairs; a third has only 2, below our minimum\-pair threshold\)\. We report this as an honest behavioral finding \(OLMo is unusually susceptible\) rather than a missing experiment\.

#### Excluded BCT biases\.

Positional Biasis LLM\-as\-judge format, not MCQ\.Are You Sureis a 5\-turn dialogue in which a fake assistant turn writes a fully\-articulated retraction paragraph, a chain of reasoning baked into the prompt that fundamentally violates our non\-CoT framing\.

#### Imperfect debiasing selectivity on some models\.

Mistral and Llama have narrower preservation\-recovery operating windows than Gemma and Qwen\. The intervention is direction\-specific everywhere \(real≫\\ggrandom at matchedα\\alpha\) but not uniformly clean\. Conditional or soft steering \(e\.g\., gated by a bias\-detection signal\) is a natural next step\.

## Acknowledgments

This material is based in part upon work supported by the German Federal Ministry of Education and Research \(BMBF\): Tübingen AI Center, FKZ: 01IS18039B; by the Machine Learning Cluster of Excellence, EXC number 2064/1 – Project number 390727645; by Coefficient Giving; by Schmidt Sciences; by the Cooperative AI Foundation; by the Canadian AI Safety Institute Research Program at CIFAR\. Resources used in preparing this research project were provided, in part, by the Digital Research Alliance of Canada; the Province of Ontario; the Swiss AI Compute grants; the Government of Canada through CIFAR; and companies sponsoring the Vector Institute\. We thank Jiarui Liu, Andrew Kim, and the other key developers of PaperMentor \(CITE1; CITE2\), which provides mentorship and valuable suggestions in polishing this paper after the first draft by the authors\. We further acknowledge the crucial work of AdminBot \(CITE\) for administrative support that streamlined our research workflow\.

## References

- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2024\)Discovering latent knowledge in language models without supervision\.External Links:2212\.03827,[Link](https://arxiv.org/abs/2212.03827)Cited by:[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Chua, E\. Rees, H\. Batra, S\. R\. Bowman, J\. Michael, E\. Perez, and M\. Turpin \(2025\)Bias\-augmented consistency training reduces biased reasoning in chain\-of\-thought\.External Links:2403\.05518,[Link](https://arxiv.org/abs/2403.05518)Cited by:[§A\.1](https://arxiv.org/html/2607.18114#A1.SS1.p1.1),[§1](https://arxiv.org/html/2607.18114#S1.p1.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2607.18114#S3.SS0.SSS0.Px1.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. Ma \(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§4](https://arxiv.org/html/2607.18114#S4.SS0.SSS0.Px1.p1.1)\.
- R\. Hendel, M\. Geva, and A\. Globerson \(2023\)In\-context learning creates task vectors\.External Links:2310\.15916,[Link](https://arxiv.org/abs/2310.15916)Cited by:[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021\)Measuring massive multitask language understanding\.External Links:2009\.03300,[Link](https://arxiv.org/abs/2009.03300)Cited by:[§4](https://arxiv.org/html/2607.18114#S4.SS0.SSS0.Px2.p1.1)\.
- K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2024\)Inference\-time intervention: eliciting truthful answers from a language model\.External Links:2306\.03341,[Link](https://arxiv.org/abs/2306.03341)Cited by:[§1](https://arxiv.org/html/2607.18114#S1.p2.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Lin, J\. Hilton, and O\. Evans \(2022\)TruthfulQA: measuring how models mimic human falsehoods\.External Links:2109\.07958,[Link](https://arxiv.org/abs/2109.07958)Cited by:[§4](https://arxiv.org/html/2607.18114#S4.SS0.SSS0.Px2.p1.1)\.
- J\. Liu, L\. Cui, H\. Liu, D\. Huang, Y\. Wang, and Y\. Zhang \(2020\)LogiQA: a challenge dataset for machine reading comprehension with logical reasoning\.External Links:2007\.08124,[Link](https://arxiv.org/abs/2007.08124)Cited by:[§4](https://arxiv.org/html/2607.18114#S4.SS0.SSS0.Px2.p1.1)\.
- S\. Marks and M\. Tegmark \(2024\)The geometry of truth: emergent linear structure in large language model representations of true/false datasets\.External Links:2310\.06824,[Link](https://arxiv.org/abs/2310.06824)Cited by:[§1](https://arxiv.org/html/2607.18114#S1.p2.1),[§1](https://arxiv.org/html/2607.18114#S1.p3.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2607.18114#S3.SS0.SSS0.Px4.p1.2)\.
- I\. R\. McKenzie, A\. Lyzhov, M\. Pieler, A\. Parrish, A\. Mueller, A\. Prabhu, E\. McLean, A\. Kirtland, A\. Ross, A\. Liu, A\. Gritsevskiy, D\. Wurgaft, D\. Kauffman, G\. Recchia, J\. Liu, J\. Cavanagh, M\. Weiss, S\. Huang, T\. F\. Droid, T\. Tseng, T\. Korbak, X\. Shen, Y\. Zhang, Z\. Zhou, N\. Kim, S\. R\. Bowman, and E\. Perez \(2024\)Inverse scaling: when bigger isn’t better\.External Links:2306\.09479,[Link](https://arxiv.org/abs/2306.09479)Cited by:[§A\.1](https://arxiv.org/html/2607.18114#A1.SS1.SSS0.Px6.p1.1),[§4](https://arxiv.org/html/2607.18114#S4.SS0.SSS0.Px2.p1.1)\.
- N\. Panickssery, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. M\. Turner \(2024\)Steering llama 2 via contrastive activation addition\.External Links:2312\.06681,[Link](https://arxiv.org/abs/2312.06681)Cited by:[§1](https://arxiv.org/html/2607.18114#S1.p2.1),[§1](https://arxiv.org/html/2607.18114#S1.p3.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2607.18114#S3.SS0.SSS0.Px4.p1.2)\.
- E\. Perez, S\. Ringer, K\. Lukošiūtė, K\. Nguyen, E\. Chen, S\. Heiner, C\. Pettit, C\. Olsson, S\. Kundu, S\. Kadavath, A\. Jones, A\. Chen, B\. Mann, B\. Israel, B\. Seethor, C\. McKinnon, C\. Olah, D\. Yan, D\. Amodei, D\. Amodei, D\. Drain, D\. Li, E\. Tran\-Johnson, G\. Khundadze, J\. Kernion, J\. Landis, J\. Kerr, J\. Mueller, J\. Hyun, J\. Landau, K\. Ndousse, L\. Goldberg, L\. Lovitt, M\. Lucas, M\. Sellitto, M\. Zhang, N\. Kingsland, N\. Elhage, N\. Joseph, N\. Mercado, N\. DasSarma, O\. Rausch, R\. Larson, S\. McCandlish, S\. Johnston, S\. Kravec, S\. E\. Showk, T\. Lanham, T\. Telleen\-Lawton, T\. Brown, T\. Henighan, T\. Hume, Y\. Bai, Z\. Hatfield\-Dodds, J\. Clark, S\. R\. Bowman, A\. Askell, R\. Grosse, D\. Hernandez, D\. Ganguli, E\. Hubinger, N\. Schiefer, and J\. Kaplan \(2022\)Discovering language model behaviors with model\-written evaluations\.External Links:2212\.09251,[Link](https://arxiv.org/abs/2212.09251)Cited by:[§1](https://arxiv.org/html/2607.18114#S1.p1.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px1.p1.1)\.
- Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2025\)Qwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§4](https://arxiv.org/html/2607.18114#S4.SS0.SSS0.Px1.p1.1)\.
- M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. R\. Bowman, N\. Cheng, E\. Durmus, Z\. Hatfield\-Dodds, S\. R\. Johnston, S\. Kravec, T\. Maxwell, S\. McCandlish, K\. Ndousse, O\. Rausch, N\. Schiefer, D\. Yan, M\. Zhang, and E\. Perez \(2025\)Towards understanding sycophancy in language models\.External Links:2310\.13548,[Link](https://arxiv.org/abs/2310.13548)Cited by:[§1](https://arxiv.org/html/2607.18114#S1.p1.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Templeton, T\. Conerly, J\. Marcus, J\. Lindsey, T\. Bricken, B\. Chen, A\. Pearce, C\. Citro, E\. Ameisen, A\. Jones, H\. Cunningham, N\. L\. Turner, C\. McDougall, M\. MacDiarmid, C\. D\. Freeman, T\. R\. Sumers, E\. Rees, J\. Batson, A\. Jermyn, S\. Carter, C\. Olah, and T\. Henighan \(2024\)Scaling monosemanticity: extracting interpretable features from claude 3 sonnet\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html)Cited by:[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Turpin, J\. Michael, E\. Perez, and S\. R\. Bowman \(2023\)Language models don’t always say what they think: unfaithful explanations in chain\-of\-thought prompting\.External Links:2305\.04388,[Link](https://arxiv.org/abs/2305.04388)Cited by:[§1](https://arxiv.org/html/2607.18114#S1.p1.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi \(2019\)HellaSwag: can a machine really finish your sentence?\.External Links:1905\.07830,[Link](https://arxiv.org/abs/1905.07830)Cited by:[§4](https://arxiv.org/html/2607.18114#S4.SS0.SSS0.Px2.p1.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. Hendrycks \(2025\)Representation engineering: a top\-down approach to ai transparency\.External Links:2310\.01405,[Link](https://arxiv.org/abs/2310.01405)Cited by:[§1](https://arxiv.org/html/2607.18114#S1.p2.1),[§2](https://arxiv.org/html/2607.18114#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix ADatasets and biases

### A\.1Bias type definitions

We summarize the seven BCT biases used in this paper\(Chuaet al\.,[2025](https://arxiv.org/html/2607.18114#bib.bib1)\)\. Each bias modifies the user message in a way that creates a \(biased, unbiased\) pair on the same underlying question\. The model’s susceptibility is measured by how often the biased version flips its answer\.

#### Suggested Answer\.

The user message ends with an explicit suggestion \(“*I think the answer is X*”\) before the question, where X is the bias\-target letter\. This is the only bias in our set that is instruction\-like in form: the cue appears in the user’s voice as a direct suggestion\.

#### Distractor Argument\.

An irrelevant but plausible\-sounding argument for a wrong answer is inserted into the user message before the question\. The argument does not address the actual question content but provides surface\-level rhetorical pressure toward the bias\-target letter\.

#### Distractor Fact\.

An irrelevant but factually\-true statement that points toward a wrong answer is inserted before the question\. UnlikeDistractor Argument, the inserted material is factually correct\. What makes it biasing is its irrelevance to the actual question\.

#### Wrong Few\-Shot\.

The prompt contains a few\-shot block in which the demonstrations have systematically wrong labels \(the demonstrated answers point toward the bias\-target letter rather than the correct one\)\.

#### Spurious Few\-Shot Squares\.

The few\-shot demonstrations share a spurious surface feature \(their question stems end with the word “*squares*”\) that correlates with the bias\-target letter in\-context\. The model is expected to pick up on the spurious shortcut\.

#### Spurious Few\-Shot Hindsight\.

A singleton source built on the Hindsight Neglect dataset\(McKenzieet al\.,[2024](https://arxiv.org/html/2607.18114#bib.bib18)\): few\-shot demonstrations follow a hindsight\-bias pattern such that resisting the cue requires ignoring the in\-context shortcut\. Because it has only one underlying dataset, this bias has no within\-bias LODO partner and is excluded from per\-bias LODO tables\.

#### Post Hoc\.

A fake prior assistant turn is inserted into the dialogue, in which the assistant has apparently already committed to the bias\-target letter\. The user then asks the question, and the model is steered toward consistency with the fabricated “previous” answer\. In our non\-CoT framing this behaves as an answer\-anchoring bias\.

#### Excluded biases\.

We exclude two BCT biases that are format\-incompatible with our pipeline\.Positional Biasis an LLM\-as\-judge format \(not MCQ\) and does not fit the single\-token A–P decoding setup\.Are You Sureis a five\-turn dialogue in which a fake assistant turn writes a fully\-articulated retraction paragraph\. This embedded reasoning chain is incompatible with our non\-CoT framing\.

### A\.2Pair\-count statistics

After applying the filtering described in §[3](https://arxiv.org/html/2607.18114#S3)\(dropping pairs where the bias target coincides with the correct letter, or where the model did not know the unbiased answer\), the per\-model totals are in Table[6](https://arxiv.org/html/2607.18114#A1.T6)and the per\-\(bias, dataset\) breakdowns are in Tables[7](https://arxiv.org/html/2607.18114#A1.T7)and[8](https://arxiv.org/html/2607.18114#A1.T8)\. The instruct totals match those used in Table[2](https://arxiv.org/html/2607.18114#S6.T2)\(rounded\)\. The dramatic instruct\-vs\-base flip\-rate disparity for four of five families, and theQwen\-baseoutlier, are discussed in §[6](https://arxiv.org/html/2607.18114#S6)\.

Table 6:Aggregate strict\-flipped and strict\-resisted pair counts per model checkpoint, summed over all 25 \(bias, dataset\) sources\. Per\-\(bias, dataset\) breakdowns are in Tables[7](https://arxiv.org/html/2607.18114#A1.T7)and[8](https://arxiv.org/html/2607.18114#A1.T8)\.Table 7:Per\-\(bias, dataset\) strict\-flipped \(F\) and strict\-resisted \(R\) pair counts on the fiveinstructcheckpoints\. Bias\-row totals over the four standard datasets \(Spurious Hindsightis a singleton\) sum to the instruct rows of Table[6](https://arxiv.org/html/2607.18114#A1.T6)\.OLMo’sPost Hocis the only \(model, bias\) pair with no resisted contrast on any standard dataset\.Table 8:Per\-\(bias, dataset\) strict\-flipped \(F\) and strict\-resisted \(R\) pair counts on the fivepretrained basecheckpoints\. Sums match the base rows of Table[6](https://arxiv.org/html/2607.18114#A1.T6)\. Four of five base families almost never flip;Qwen\-baseis the documented behavioral outlier \(§[6](https://arxiv.org/html/2607.18114#S6)\)\. TheDist\. Arg\.block is near\-empty on every base family except Qwen because base models without an instruction template rarely produce a parseable answer letter when shown a misleading argument\.

## Appendix BPer\-bias direction probing

### B\.1Layer\-wise emergence

![Refer to caption](https://arxiv.org/html/2607.18114v1/x6.png)

![Refer to caption](https://arxiv.org/html/2607.18114v1/x7.png)

Figure 6:Per\-layer LODO probing AUROC on each transformer layer’s hidden\-state slice\. Solid black line is the mean across the 24 LODO test points \(6 biases×\\times4 datasets\)\. Gray band is±1\\pm 1standard error\. Colored thin lines show per\-bias means\. Gray dashed line is the random\-direction baseline\. Red dotted vertical line marks the best layer\.We slice the per\-source bias direction into per\-layer chunksds,ℓ∈ℝHd\_\{s,\\ell\}\\in\\mathbb\{R\}^\{H\}, whereHHis the hidden size, and repeat the LODO probing setup at each layerℓ\\ellin isolation: for each held\-out source, the LODO direction is the unit\-normalized mean of the per\-source layer\-ℓ\\elldirections across the other datasets of the same bias\. Slicing a normalized vector and renormalizing the slice is mathematically equivalent to computing the layer\-ℓ\\elldirection fromXbiasedX\_\{\\text\{biased\}\}directly \(both equal\(μℓflip−μℓrsst\)/‖μℓflip−μℓrsst‖\(\\mu^\{\\text\{flip\}\}\_\{\\ell\}\-\\mu^\{\\text\{rsst\}\}\_\{\\ell\}\)/\\\|\\mu^\{\\text\{flip\}\}\_\{\\ell\}\-\\mu^\{\\text\{rsst\}\}\_\{\\ell\}\\\|, whereμℓ\\mu\_\{\\ell\}is the layer\-ℓ\\ellslice of the class\-conditional mean\)\. The per\-layer random\-direction baseline uses 20 random unit vectors inℝH\\mathbb\{R\}^\{H\}\(not inℝL​H\\mathbb\{R\}^\{LH\}, which would have a different chance level\)\.

Figure[6](https://arxiv.org/html/2607.18114#A2.F6)plots the result forLlama\-3\.1\-8BandQwen\-2\.5\-7B; the cross\-family summary is in Figure[3](https://arxiv.org/html/2607.18114#S5.F3)\. On both models the AUROC rises through the network and peaks in the late\-middle layers \(Llama at layer17/3217/32, 55% of depth, AUROC0\.820\.82; Qwen at layer20/2820/28, 74% of depth, AUROC0\.850\.85\)\. The best single\-layer AUROC exceeds the all\-layer LODO AUROC \(Llama:0\.820\.82vs0\.740\.74; Qwen:0\.850\.85vs0\.820\.82\), consistent with early layers adding noise to the all\-layer concatenated direction\.

### B\.2Letter\-balanced control

A natural concern is that the bias direction might encode “predict letterXX” rather than “follow the bias,” because the BCT data has non\-uniform letter distributions forbias\_target\(in flipped pairs\) andcorrect\_letter\(in resisted pairs\)\. If so, the direction would partially capture a letter\-prediction signal correlated with bias\-following by construction\.

We test this directly\. For each source, we strictly balance the flipped subset bybias\_targetletter \(subsampling each letter group to the smallest group’s size\) and the resisted subset bycorrect\_letter, then recompute the bias direction from these balanced subsets and rerun the LODO probing test with both the averaged training direction*and*the held\-out test pairs taken from the balanced subsets\. Strict balancing requires at least 3 flipped and 3 resisted pairs per letter; the number of LODO test points satisfying this varies by model \(Table[9](https://arxiv.org/html/2607.18114#A2.T9)\)\. The unbalanceable sources are mostlytruthfulqa, where letter distributions are highly imbalanced\.

Table 9:Letter\-balanced control across all 5 instruct families\. After strictly balancing both thebias\_targetdistribution in flipped pairs and thecorrect\_letterdistribution in resisted pairs, LODO AUROC changes by\|Δ\|≤0\.041\|\\Delta\|\\leq 0\.041on average and the change is not statistically significant on any model\. The balanced direction has cosine similarity≥0\.72\\geq 0\.72with the original direction\. The bias direction is not primarily a letter\-prediction artifact in any family\.On all 5 instruct families, the balanced AUROC differs from the original by\|Δ\|≤0\.041\|\\Delta\|\\leq 0\.041on average, and the change is not statistically significant in any family \(p\>0\.10p\>0\.10throughout\)\. Gemma is the lone case where balancing very slightly improves mean AUROC \(Δ=\+0\.041\\Delta=\+0\.041, still n\.s\. atp=0\.11p=0\.11\); the other four families have small negativeΔ\\Deltavalues, consistent with a small but non\-significant letter\-prediction component in the unbalanced direction\. The balanced direction is highly similar to the original everywhere \(cosine0\.720\.72on the weakest model,0\.910\.91on the strongest\)\. The bias direction is genuinely encoding bias\-following rather than letter prediction across all 5 families\.

## Appendix CCluster structure analysis

### C\.1Per\-source and per\-bias cosine matrices

![Refer to caption](https://arxiv.org/html/2607.18114v1/x8.png)Figure 7:Per\-bias cosine matrices across all 5 instruct families\.Bottom\-right block in each panel corresponds to the 4\-bias cluster \(Distractor Fact,Spurious Few\-Shot Squares,Wrong Few\-Shot,Suggested Answer\)\. The cluster is dense on Qwen, moderate on Gemma, weak on Mistral, and absent on Llama and OLMo, matching the rank/significance ordering in Table[4](https://arxiv.org/html/2607.18114#S7.T4)\. TheSpurious Few\-Shot Hindsightsingleton \(“—” entries on Gemma and OLMo\) is not part of those models’ per\-bias cosine analysis\.![Refer to caption](https://arxiv.org/html/2607.18114v1/x9.png)

![Refer to caption](https://arxiv.org/html/2607.18114v1/x10.png)

Figure 8:Pairwise cosine similarity between the 25 per\-source bias directions onLlama\-3\.1\-8B\(left\) andQwen\-2\.5\-7B\(right\), reordered by bias type with bias\-block boundaries marked\. Tick labels are colored by bias type\. Within\-bias blocks along the diagonal are visibly more correlated than off\-block cells on both models\.The five\-family per\-bias matrix is in Figure[7](https://arxiv.org/html/2607.18114#A3.F7); the full25×2525\\times 25per\-source matrix on Llama and Qwen is in Figure[8](https://arxiv.org/html/2607.18114#A3.F8)\. With 6 non\-singleton biases at 4 datasets each, the source matrix has6⋅\(42\)=366\\cdot\\binom\{4\}\{2\}=36within\-bias pairs and\(252\)−36=264\\binom\{25\}\{2\}\-36=264across\-bias pairs\.

#### Within\- vs across\-bias separation\.

Within\-bias source\-direction means are well above across\-bias means in both models: Llama\+0\.467\+0\.467vs\+0\.068\+0\.068\(diff\+0\.399\+0\.399, Mann\-WhitneyUUone\-sidedp=1\.3×10−12p=1\.3\\times 10^\{\-12\}\), Qwen\+0\.689\+0\.689vs\+0\.337\+0\.337\(diff\+0\.351\+0\.351,p=6\.2×10−13p=6\.2\\times 10^\{\-13\}\)\. The within\-vs\-across lift is similar across models, but Qwen’s absolute cosines are uniformly higher, consistent with the LODO finding that Qwen’s bias representations are more concentrated\. Per\-bias breakdowns are in Table[10](https://arxiv.org/html/2607.18114#A3.T10)\.

#### Cluster significance\.

We sub\-partition the 264 across\-bias pairs to test whether Qwen’s four\-bias cluster is statistically distinguishable\. A pair is*within\-cluster*if both sources’ biases lie in the cluster \(Suggested Answer,Distractor Fact,Wrong Few\-Shot,Spurious Few\-Shot Squares, 96 pairs\), and the remaining 168 across\-bias pairs are*outside*\. On Qwen the within\-cluster mean cosine is\+0\.634\+0\.634vs\.\+0\.168\+0\.168outside \(p=2\.5×10−38p=2\.5\\times 10^\{\-38\}\); on Llama the same four\-bias set shows no separation \(\+0\.068\+0\.068vs\.\+0\.068\+0\.068,p=0\.68p=0\.68\)\. The Llama result is a genuine negative control: the cluster is defined from Qwen’s matrix, not Llama’s, so the four\-bias set is arbitrary for Llama\. Among all\(74\)=35\\binom\{7\}\{4\}=35four\-bias subsets, Qwen’s cluster has the highest mean within\-subset across\-bias cosine \(rank 1/35\), while the same set ranks 19/35 on Llama \(Table[4](https://arxiv.org/html/2607.18114#S7.T4)\)\. Because the cluster was identified from this same cosine matrix, the Qwenpp\-value is descriptive rather than confirmatory; the cluster’s independent confirmation comes from the probing\-transfer matrix \(Appendix[C\.2](https://arxiv.org/html/2607.18114#A3.SS2)\)\.

#### Per\-bias and per\-pair breakdowns\.

Table[10](https://arxiv.org/html/2607.18114#A3.T10)gives per\-bias within\-source means, and Table[11](https://arxiv.org/html/2607.18114#A3.T11)lists the six pairs whose tight Qwen cluster \(cosines0\.720\.72–0\.850\.85\) is absent on Llama \(only1/61/6exceeds0\.100\.10\)\.

Table 10:Per\-bias mean within\-source cosine similarity \(6 pairs per bias\)\.Table 11:Cosine similarity for the six pairs that form Qwen’s 4\-bias cluster\. On Qwen all six are in the 0\.72–0\.85 range; on Llama only one exceeds 0\.10\.

### C\.2Cross\-bias probing transfer

![Refer to caption](https://arxiv.org/html/2607.18114v1/x11.png)

![Refer to caption](https://arxiv.org/html/2607.18114v1/x12.png)

Figure 9:Cross\-bias probing transfer AUROC\. Rows are the bias whose direction is used as the probe \(*Btrain*\); columns are the bias whose flipped/resisted pairs are predicted \(*Btest*\)\. Diagonal cells reproduce the within\-bias LODO AUROC from Section[5](https://arxiv.org/html/2607.18114#S5)\. Off\-diagonal cells measure cross\-bias direction transfer\.Figure[9](https://arxiv.org/html/2607.18114#A3.F9)shows the full6×66\\times 6cross\-bias probing matrix on Llama and Qwen \(theSpurious Few\-Shot Hindsightsingleton is excluded from this display because it has no within\-bias LODO partner\)\. The diagonals reproduce the within\-bias LODO AUROC from Section[5](https://arxiv.org/html/2607.18114#S5)to fp32 precision\. On Qwen the four cluster biases’ directions are nearly interchangeable as probes \(within\-cluster off\-diagonal AUROCs in\[0\.74,0\.85\]\[0\.74,0\.85\], mean0\.790\.79\)\. On Llama the same cluster does not transfer \(within\-cluster off\-diagonal range\[0\.47,0\.72\]\[0\.47,0\.72\], mean0\.550\.55\)\.

Each cell carries an SE\-based 95% CI over its held\-out test sources \(mean±1\.96⋅SE\\pm 1\.96\\cdot\\text\{SE\}across thennheld\-out sources ofBtestB\_\{\\text\{test\}\}\)\. On Qwen all 12 within\-cluster off\-diagonal cells have 95% CI lower bounds strictly above the 0\.5 chance level, with CI half\-widths≤0\.05\\leq 0\.05\. On Llama only 2 of 12 do\. Llama’s above\-chance cells that do exist are scattered across non\-cluster pairs \(7 of 18\) rather than concentrated in any block, consistent with the absence of a cluster\.

## Appendix DCausal intervention details

### D\.1α\\alpha\-sweep

The intervention protocol is defined in Section[8](https://arxiv.org/html/2607.18114#S8)\. The per\-model operating pointα∗\\alpha^\{\*\}in Table[5](https://arxiv.org/html/2607.18114#S8.T5)is chosen from the recovery–preservation Pareto onLlama\-3\.1\-8BandQwen\-2\.5\-7B, subject to the preservation≥90%\\geq 90\\%constraint\. Figure[10](https://arxiv.org/html/2607.18114#A4.F10)plots that Pareto across the sweepα∈\{0\.1,0\.25,0\.5,1,2,4,8\}\\alpha\\in\\\{0\.1,0\.25,0\.5,1,2,4,8\\\}; the bidirectional summary on all five families is in Figure[5](https://arxiv.org/html/2607.18114#S8.F5)\(main body\)\.

![Refer to caption](https://arxiv.org/html/2607.18114v1/x13.png)

![Refer to caption](https://arxiv.org/html/2607.18114v1/x14.png)

Figure 10:Recovery–selectivity trade\-off onLlama\-3\.1\-8B\(left\) andQwen\-2\.5\-7B\(right\)\. Each small dot is one \(source,α\\alpha\) combination for positiveα\\alpha; large×\\timesmarkers are means perα\\alpha\. On Llama,α=0\.5\\alpha=0\.5is the Pareto\-optimal point\. On Qwen the curve is monotone, and preservation degrades only slightly throughα=4\\alpha=4\.
### D\.2Per\-source recovery

![Refer to caption](https://arxiv.org/html/2607.18114v1/x15.png)Figure 11:Per\-source intervention recovery atα=0\.5\\alpha=0\.5onLlama\-3\.1\-8B\. All 24 sources show real\-direction recovery substantially exceeding random\-direction recovery\.Distractor FactandSuggested Answersources reach4040–45%45\\%recovery\. The random direction is essentially zero everywhere\.Figure[11](https://arxiv.org/html/2607.18114#A4.F11)shows the per\-source breakdown atα=0\.5\\alpha=0\.5forLlama\-3\.1\-8B\. All 24 sources are above the random\-direction baseline\. Recovery rates range from16%16\\%\(distractor\_argument\_\_logiqa\) to45%45\\%\(distractor\_fact\_\_logiqa\); random\-direction recovery never exceeds3%3\\%on any source\.