DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions

# DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
Source: [https://arxiv.org/html/2607.20469](https://arxiv.org/html/2607.20469)
Lixun ZhangKangning CuiYixiao WangTing JiangHancheng YeQinsi WangZhixu DuYuzhe FuFan YangDanyang ZhuoYiran ChenHai Helen Li

###### Abstract

Large language models \(LLMs\) handle many tasks with one set of parameters, but under KV\-cached inference it is unclear what task\-general structure, if any, is used at*decode time*rather than during*prefill*\. We proposeDecodeShare, a protocol that identifies a low\-dimensional subspace consistently shared across tasks in decode\-time hidden states, and then tests its causal role by removing that subspace only during decoding\. In our experiments, disturbing the discovered shared subspace degrades decision performance far more than disturbing either a prefill\-derived or random subspace under the same intervention budget\. We further show this decode\-shared subspace has practical consequences for activation steering: common steering directions can overlap the task\-general decode channel\. Projecting out this shared subspace directly separates the functional roles of the two components, while evaluating steering vectors at decode\-time yields more reliable signal for downstream deployment than prefill\-based proxies\. Despite its compactness, the shared subspace can serve as a high\-leverage causal channel at decode time\. Code is available at:[https://github\.com/Zishan\-Shao/decodeshare\.git](https://github.com/Zishan-Shao/decodeshare.git)\.

Machine Learning, ICML

## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.20469v1/x1.png)Figure 1:DecodeShare Pipeline\.\(1\) We collect KV\-cached*decode\-time hidden states*from prompts across multiple tasks and pool them into a matrix\. \(2\) A pooled PCA yields a set of orthonormal directions \(columns of V matrix\); we then select a subset that is consistently shared across tasks \(Qℓ\(s\)Q\_\{\\ell\}^\{\(s\)\}\), rather than simply taking the top\-variance components\. \(3\) To test causality, we intervene*only during decoding*by removing the selected component from the hidden state and compare against dimension\- and*energy*\-matched controls to rule out a simple high\-variance/energy confound\.Large language models \(LLMs\) achieve strong performance across a wide range of tasks using a single set of parameters\. This multi\-task generality motivates the idea of computation reuse: diverse behaviors may be mediated by a common set of internal features rather than entirely task\-specific ones\. We can view this reuse as inducing a shared subspace in the model’s decision\-time representations–i\.e\., a set of directions that are consistently engaged across tasks and prompt variants\. If so, it would provide a principled target for analysis and intervention that transfers across tasks and prompt variants\.

Motivated by this, a growing line of work studies and steers LLM behavior by intervening on internal*activations*using low\-dimensional directions or subspaces\(Zouet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib22); Turneret al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib19); Rimskyet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib20)\)\. In practice, however, activation\-level interventions are often brittle: effects that appear strong under one prompt template, task setup, or implementation detail can weaken, flip, or disappear under small perturbations\(Tanet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib26); Denget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib27)\)\. This brittleness limits reproducibility and makes negative results hard to interpret\.

We argue that brittleness is amplified because interventions are often not aligned with the*states*that drive next\-token decisions under modern inference\.First, we rarely measure or protect task\-general structure at decision time\.Most activation\-steering methods estimate a small set of directions for a specific behavior or attribute under a fixed prompt distribution\(Turneret al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib19); Rimskyet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib20); Zouet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib22); Konenet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib21); Arditiet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib28); Toddet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib29)\)\. These methods typically do not control interactions between steering directions and task\-general computation directions\(Merulloet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib16); Kaushiket al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib17)\)\. If next\-token accuracy relies on a shared decision\-time channel, then perturbing it can cause large failures and template sensitivity, consistent with observed brittleness under prompt or template shifts\(Tanet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib26); Denget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib27)\)\.

Second, there is a mismatch between where directions are estimated and where decisions are made\.In common activation\-steering pipelines, steering directions are typically estimated from*prefill*activations computed on the full prompt\(Turneret al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib19); Rimskyet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib20); Zouet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib22); Konenet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib21); Arditiet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib28); Toddet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib29)\)\. Under KV\-cached decoding, however, each next\-token decision is computed from a single\-token*decode*pass\. If decode\-time hidden states differ from prefill activations, then prefill\-estimated directions may not align with the states that actually produce logits\. This mismatch matters for steering selection: a vector favored by prefill\-based validation may not perform best under held\-out KV\-cached decoding\. Since steering vectors can also overlap task\-general decode structure, projection is best viewed as an interference diagnostic rather than a universal repair\.

We introduceDecodeShare, a protocol that estimates candidate shared subspaces from decode\-time hidden states and evaluates their relevance to decoding decisions via*decode\-only*projection removal\. Figure[1](https://arxiv.org/html/2607.20469#S1.F1)previews ourDecodeSharepipeline: we collect decode\-time hidden states across tasks, identify a shared subspace, and test its causal role with decode\-only ablations\. Accordingly, we analyze the decode\-time hidden state under KV\-cached inference as the representation that drives next\-token logits\. \(We introduce formal notations and definitions in Sec[2\.1](https://arxiv.org/html/2607.20469#S2.SS1)\.\)

Hypothesis: shared decode\-time decision channelUnder KV\-cached decoding, next\-token decisions causally depend on a*small*subspaceSSof the decode\-time hidden state that is shared across tasks\.

#### Key findings and implications\.

Across models and benchmarks, removing the identified shared subspace causes much larger drops in decision accuracy than matched controls, supporting a high\-leverage causal channel at decision time\. This effect is largely*decode\-specific*: under matched budgets, subspaces estimated from prefill often fail to reproduce the decode\-time causal impact\. Finally, we connect the shared decode\-time channel to steering reliability\. Common steering vectors overlap this channel; projecting out that overlap exposes task\-dependent utility–robustness trade\-offs rather than a universal repair, while decode\-time evaluation provides a better ranking signal for held\-out KV\-cached decoding utility than prefill\-based proxies\. Together, this frames steering brittleness as partly an inference\-regime alignment problem, not only a prompt\-template artifact\.

Our key contributions are:

1. 1\.We proposeDecodeShare, a protocol that identifies and quantifies cross\-task shared subspaces*directly from KV\-cached decode\-time hidden states*\.
2. 2\.Using decode\-only interventions under matched budgets, we show the shared decode subspace is causally important for decisions and that prefill\-estimated bases often fail to capture these decode\-time causal effects\.
3. 3\.We connect the decode\-shared subspace to steering reliability; it overlaps steering vectors, and decode\-time evaluation provides a better ranking signal for held\-out KV\-cached decoding utility than prefill\-based proxies\.

We position this work as a*protocol and evaluation*contribution: a practical way to find and causally test decision\-relevant shared subspaces under real KV\-cached decoding\.

## 2Methodology

### 2\.1Preliminary

#### KV\-cached inference and decode\-time hidden states\.

Letfθf\_\{\\theta\}be a decoder\-only Transformer withLLlayers and hidden widthdd\. Under KV caching, inference consists of a*prefill*pass over the full prompt, followed by iterative*decode*steps\. Each decode step processes a single current token while reusing cached keys/values from previous tokens\.

Fix a single layerℓ\\ell\. Lethℓ\(s\)∈ℝdh\_\{\\ell\}^\{\(s\)\}\\in\\mathbb\{R\}^\{d\}denote the layer\-ℓ\\ellhidden state of the current token at decode stepssunder KV\-cached decoding\. We callhℓ\(s\)h\_\{\\ell\}^\{\(s\)\}the*decode\-time hidden state*\. When emphasizing that it drives the next\-token logits \(and thus decision accuracy\), we also refer to it as the*decode\-time decision state*\. Unless otherwise stated, we run KV\-cached decoding for up toKKsteps and collect decode\-time hidden states across steps \(treating steps uniformly when pooling\)\.

#### Prefill vs\. decode states\.

We distinguish hidden states obtained in*prefill*pass \(computed on the full prompt\) from decode\-time hidden states obtained during KV\-cached decoding\. In all experiments, the prefill state is taken at the last prompt position to provide a single\-token reference point for comparison \(formal definitions in Appendix[Additional Definitions\.](https://arxiv.org/html/2607.20469#A0.SS0.SSS0.Px1)\)\.

#### Alignment principle\.

All causal interventions in this paper act onhℓ\(s\)h\_\{\\ell\}^\{\(s\)\}*only during KV\-cached decode steps*\(we do not modify the prefill computation\)\. Accordingly, we estimate candidate subspaces from decode\-time hidden states and intervene on the same decode\-time states, avoiding estimator\-intervention mismatch\.

### 2\.2Key Hypotheses and Falsification Criteria

Rather than introducing additional independent claims, we test the central hypothesis via three checks: \(i\)H1validates that shared decode\-time structure exists beyond chance, \(ii\)H2tests whether the shared subspace is causally relevant at decode time, and \(iii\)H3tests whether*prefill*\-estimated directions causally transfer to decode\-time decisions under matched budgets\.

H1Shared decode\-time structure exists\.From decode\-time hidden states, our sharedness statistic \(shared set size\|Sℓ​\(τ,m\)\|\|S\_\{\\ell\}\(\\tau,m\)\|\) exceeds what is expected under matched null baselines\.

H2Decode\-time causal relevance\.Removing the estimated shared subspace from the decode\-time hidden statehℓ\(s\)h\_\{\\ell\}^\{\(s\)\}*only during KV\-cached decoding*harms performance more than matched non\-shared controls under the same intervention budget\.

H3Prefill\-to\-decode causal transfer often fails\.A basis estimated from prefill states does not reliably reproduce the decode\-time causal effect of a basis estimated from decode\-time hidden states under matched budgets\.

#### Falsification criteria\.

We rejectH1if the sharedness statistic is not significantly larger than matched null baselines\. We rejectH2–H3if the corresponding decode\-time intervention effects are not distinguishable from matched controls under the same budget \(and, forH3, if prefill\- and decode\-estimated bases yield comparable decode\-time effects\)\.

### 2\.3DecodeShare: Constructing a decode\-aligned shared subspace

#### Decode\-only state collection\.

For each taskt∈𝒯t\\in\\mathcal\{T\}, we sampleNNcalibration promptsx∼𝒟tx\\sim\\mathcal\{D\}\_\{t\}and run KV\-cached decoding for up toKKsteps under policyπ\\pi\(e\.g\., greedy\)\. At each decode stepss\(single\-token forward pass\), we record the layer\-ℓ\\ellhidden statehℓ\(s\)∈ℝdh\_\{\\ell\}^\{\(s\)\}\\in\\mathbb\{R\}^\{d\}\. Stacking all recorded states yieldsXℓ,t∈ℝnℓ,t×dX\_\{\\ell,t\}\\in\\mathbb\{R\}^\{n\_\{\\ell,t\}\\times d\}\. To balance tasks, we subsample each tonℓ=mint⁡nℓ,tn\_\{\\ell\}=\\min\_\{t\}n\_\{\\ell,t\}, obtainingXℓ,t′∈ℝnℓ×dX^\{\\prime\}\_\{\\ell,t\}\\in\\mathbb\{R\}^\{n\_\{\\ell\}\\times d\}, and task\-center:

X~ℓ,t=Xℓ,t′−𝟏​μℓ,t⊤,μℓ,t=1nℓ​∑j=1nℓXℓ,t′​\[j,:\]\.\\tilde\{X\}\_\{\\ell,t\}\\;=\\;X^\{\\prime\}\_\{\\ell,t\}\-\\mathbf\{1\}\\mu\_\{\\ell,t\}^\{\\top\},\\;\\mu\_\{\\ell,t\}=\\frac\{1\}\{n\_\{\\ell\}\}\\sum\_\{j=1\}^\{n\_\{\\ell\}\}X^\{\\prime\}\_\{\\ell,t\}\[j,:\]\.
![Refer to caption](https://arxiv.org/html/2607.20469v1/x2.png)\(a\)Layer44
![Refer to caption](https://arxiv.org/html/2607.20469v1/x3.png)\(b\)Layer1010
![Refer to caption](https://arxiv.org/html/2607.20469v1/x4.png)\(c\)Layer2424

Figure 2:Sharedness heatmaps across layers\.Rows are tasks and columns are pooled PCA components\. Color shows the relative variance contributionrℓ,t,ir\_\{\\ell,t,i\}\. Components are sorted by the number of tasks withrℓ,t,i≥τr\_\{\\ell,t,i\}\\geq\\tau; the vertical line marks the boundary of the shared setSℓ​\(τ,m\)S\_\{\\ell\}\(\\tau,m\)\.
#### Pooled PCA basis \(computed via SVD\)\.

We pool \(stack\) all task\-centered states:

X~ℓ=concatt∈𝒯​X~ℓ,t∈ℝ\(\|𝒯\|​nℓ\)×d\.\\tilde\{X\}\_\{\\ell\}=\\mathrm\{concat\}\_\{t\\in\\mathcal\{T\}\}\\,\\tilde\{X\}\_\{\\ell,t\}\\in\\mathbb\{R\}^\{\(\|\\mathcal\{T\}\|n\_\{\\ell\}\)\\times d\}\.We define a pooled PCA basis by running*PCA*on this pooled matrix, i\.e\., taking the top eigenvectors of its empirical covarianceCℓ=1\|𝒯\|​nℓ​X~ℓ⊤​X~ℓC\_\{\\ell\}=\\frac\{1\}\{\|\\mathcal\{T\}\|n\_\{\\ell\}\}\\tilde\{X\}\_\{\\ell\}^\{\\top\}\\tilde\{X\}\_\{\\ell\}\. In implementation, we obtain the same PCA directions by an SVD of the pooled matrix:

X~ℓ=U​diag​\(σ1,…,σd\)​V⊤,\\tilde\{X\}\_\{\\ell\}=U\\,\\mathrm\{diag\}\(\\sigma\_\{1\},\\ldots,\\sigma\_\{d\}\)\\,V^\{\\top\},whereσ1≥σ2≥⋯≥0\\sigma\_\{1\}\\geq\\sigma\_\{2\}\\geq\\cdots\\geq 0are the singular values\. For centered data, the PCA directions are the columns ofVV, and the variance captured by theii\-th direction is proportional toσi2\\sigma\_\{i\}^\{2\}\. We choose the smallestkkthat retains at least a fractionρ\\rhoof total variance:

k=min⁡\{m:∑i=1mσi2∑i=1dσi2≥ρ\},Qℓ=V:,1:k∈ℝd×k\.k=\\min\\left\\\{m:\\frac\{\\sum\_\{i=1\}^\{m\}\\sigma\_\{i\}^\{2\}\}\{\\sum\_\{i=1\}^\{d\}\\sigma\_\{i\}^\{2\}\}\\geq\\rho\\right\\\},\\qquad Q\_\{\\ell\}=V\_\{:,1:k\}\\in\\mathbb\{R\}^\{d\\times k\}\.This pooled PCA basis provides a single cross\-task basis set, avoiding per\-task PCA basis misalignment\. Empirically, the recovered shared core is stable across a wide range ofρ\\rho\(e\.g\.,ρ∈\[0\.9,0\.99\]\\rho\\in\[0\.9,0\.99\]; Table[10\(a\)](https://arxiv.org/html/2607.20469#A2.T10.st1)\)\.

#### Identifying shared directions via usage scores and thresholding\.

For each taskt∈𝒯t\\in\\mathcal\{T\}, we quantify how strongly its decode\-time states vary along each pooled PCA direction\. After task\-centering, we project ontoQℓ∈ℝd×kQ\_\{\\ell\}\\in\\mathbb\{R\}^\{d\\times k\},

Zℓ,t=X~ℓ,t​Qℓ∈ℝnℓ×k,Z\_\{\\ell,t\}\\;=\\;\\tilde\{X\}\_\{\\ell,t\}Q\_\{\\ell\}\\in\\mathbb\{R\}^\{n\_\{\\ell\}\\times k\},and compute per\-direction variancevℓ,t,i=Var​\(Zℓ,t​\[:,i\]\)v\_\{\\ell,t,i\}=\\mathrm\{Var\}\\\!\\left\(Z\_\{\\ell,t\}\[:,i\]\\right\)fori∈\[k\]i\\in\[k\]\. To compare usage patterns across tasks and remove overall scale, we normalize within the pooled subspace and define the*relative variance contribution*

rℓ,t,i=vℓ,t,i∑j=1kvℓ,t,j∈\[0,1\],∑i=1krℓ,t,i=1\.r\_\{\\ell,t,i\}\\;=\\;\\frac\{v\_\{\\ell,t,i\}\}\{\\sum\_\{j=1\}^\{k\}v\_\{\\ell,t,j\}\}\\in\[0,1\],\\qquad\\sum\_\{i=1\}^\{k\}r\_\{\\ell,t,i\}=1\.
Becauserℓ,t,i≥0r\_\{\\ell,t,i\}\\geq 0and∑i=1krℓ,t,i=1\\sum\_\{i=1\}^\{k\}r\_\{\\ell,t,i\}=1,rℓ,t,ir\_\{\\ell,t,i\}is the fraction of task\-ttvariance \(within the retainedkk\-dimensional pooled subspace\) attributed to directionii\. We mark directioniias*shared*for taskttifrℓ,t,i≥τr\_\{\\ell,t,i\}\\geq\\tau\. When PCA retentionρ\\rhochangeskk, we report the normalized thresholdτ​k\\tau kfor comparability:τ​k=𝒪​\(1\)\\tau k=\\mathcal\{O\}\(1\)means the threshold is roughly constant in diffuse baseline1/k1/k\(i\.e\.,τ=𝒪​\(1/k\)\\tau=\\mathcal\{O\}\(1/k\)\)\.

We aggregate activity across tasks by defining the shared index set:

Sℓ​\(τ,m\)=\{i∈\[k\]:\|\{t∈𝒯:rℓ,t,i≥τ\}\|≥m\}\.S\_\{\\ell\}\(\\tau,m\)=\\left\\\{i\\in\[k\]:\\big\|\\\{t\\in\\mathcal\{T\}:r\_\{\\ell,t,i\}\\geq\\tau\\\}\\big\|\\geq m\\right\\\}\.We take the decode\-aligned shared basis as

Qℓ\(S\)=Qℓ​\[:,Sℓ​\(τ,m\)\]∈ℝd×\|Sℓ​\(τ,m\)\|\.Q\_\{\\ell\}^\{\(S\)\}=Q\_\{\\ell\}\[:,S\_\{\\ell\}\(\\tau,m\)\]\\in\\mathbb\{R\}^\{d\\times\|S\_\{\\ell\}\(\\tau,m\)\|\}\.We denote the corresponding shared subspace by𝒮ℓ:=span​\(Qℓ\(S\)\)\\mathcal\{S\}\_\{\\ell\}:=\\mathrm\{span\}\(Q\_\{\\ell\}^\{\(S\)\}\)and its projector byΠℓ:=Π​\(Qℓ\(S\)\)=Qℓ\(S\)​\(Qℓ\(S\)\)⊤\\Pi\_\{\\ell\}:=\\Pi\(Q\_\{\\ell\}^\{\(S\)\}\)=Q\_\{\\ell\}^\{\(S\)\}\(Q\_\{\\ell\}^\{\(S\)\}\)^\{\\top\}\.

### 2\.4Decode\-only causal tests with matched controls

#### Intervene at the decode\-only boundary\.

We test causal relevance by intervening on the decode\-time decision statehℓ\(s\)h\_\{\\ell\}^\{\(s\)\}only during KV\-cached decoding\. This preserves the prefill pass and avoids conflating decision\-time effects with prompt\-processing dynamics\.

#### Remove a subspace by projection\.

At decode stepss, we ablate a subspace spanned by an orthonormal basisQ∈ℝd×kQQ\\in\\mathbb\{R\}^\{d\\times k\_\{Q\}\}by subtracting its projection from the state:

h~ℓ\(s\)=hℓ\(s\)−α​Q​Q⊤​hℓ\(s\),\\tilde\{h\}\_\{\\ell\}^\{\(s\)\}=h\_\{\\ell\}^\{\(s\)\}\-\\alpha\\,QQ^\{\\top\}h\_\{\\ell\}^\{\(s\)\},\(1\)with intervention strengthα≥0\\alpha\\geq 0\. We setQ=Qℓ\(S\)Q=Q\_\{\\ell\}^\{\(S\)\}for shared\-subspace ablations, and use alternativeQQfor controls below\. Crucially, Eq\. \([1](https://arxiv.org/html/2607.20469#S2.E1)\) is applied only on decode pass, leaving prefill untouched\.

#### Size\- and energy\-matched controls\.

Decode\-time representations are anisotropic, so removing high\-energy subspace can be generically harmful\. To isolate the effect of*which directions*are removed \(sharedness\) from*how much*is removed, we compare to non\-shared control subspaces matched in \(i\) dimensionkQk\_\{Q\}and \(ii\) removed energy under the same decode\-time distribution\. We estimate removed energy on calibration decode states as

E​\(Q\)=𝔼h∼Ddecode​\(⋅,ℓ\)​\[‖Q​Q⊤​h‖22\],E\(Q\)=\\mathbb\{E\}\_\{h\\sim D\_\{\\text\{decode\}\}\(\\cdot,\\ell\)\}\\\!\\left\[\\\|QQ^\{\\top\}h\\\|\_\{2\}^\{2\}\\right\],and construct controls so that either we tuneα\\alphato matchE​\(Q\)E\(Q\)at fixed dimension, or we matchE​\(Q\)E\(Q\)by choosingkQk\_\{Q\}at fixedα\\alpha\(details in Appendix[A](https://arxiv.org/html/2607.20469#A1)\)\.

#### Leave\-one\-task\-out \(LOTO\) to avoid leakage\.

To mitigate pooled\-estimation leakage concerns, we optionally estimate the shared basis using all but one task and evaluate causal effects on the held\-out task usingQℓ,¬t\(S\)Q\_\{\\ell,\\neg t\}^\{\(S\)\}\.

#### Metrics\.

We report two complementary readouts\. For discrete\-choice tasks, our primary metric is*forced\-choice accuracy*computed from teacher\-forced conditional log probabilities under a fixed prompt, which isolates decision degradation from formatting/termination failures\. We also report task\-appropriate native generation metrics \(Table[8](https://arxiv.org/html/2607.20469#A0.T8)\), including exact match after task\-specific extraction when applicable\. Since interventions can trigger output instability, we report generation scores with diagnostics: extraction success rate, EOS rate, and average decoded length\. We report paired bootstrap confidence intervals and conduct paired randomization \(sign\-flip\) tests on per\-example score differences\.

![Refer to caption](https://arxiv.org/html/2607.20469v1/x5.png)Figure 3:Sensitivity toτ\\tau\.Shared ratio\|Sℓ​\(τ,m\)\|/k\|S\_\{\\ell\}\(\\tau,m\)\|/kas a function of the sharedness thresholdτ\\taufor several PCA retention levelsρ\\rho\. The dashed line marks the defaultτ=10−3\\tau=10^\{\-3\}; see Table[10\(a\)](https://arxiv.org/html/2607.20469#A2.T10.st1)and §[3\.1](https://arxiv.org/html/2607.20469#S3.SS1)for the corresponding shared\-core sizes and theτ​k\\tau kinterpretation\.

## 3Experiments

#### Setup\.

Our core experiments span a heterogeneous suite of benchmarks covering arithmetic, commonsense and knowledge\-intensive QA, verification, and physical reasoning \(Table[8](https://arxiv.org/html/2607.20469#A0.T8)\)\. We additionally run targeted robustness checks across multiple backbone models includingmeta\-llama/Llama\-2\-7b\-chat\-hf,Qwen2\.5\-7B\-Instruct, andFalcon\-7b\-instructand at representative early/mid/late layers \(e\.g\.,ℓ∈\{4,10,24\}\\ell\\in\\\{4,10,24\\\}\)\. For each task, we estimate subspaces from 128 calibration prompts and evaluate on held\-out examples under decoding, reporting paired bootstrap confidence intervals and paired sign\-flip tests\. Beyond the core suite, auxiliary experiments cover additional settings, including coding and language\-understanding benchmarks \(e\.g\., HumanEval, SST\-2, RTE\) and a lexicon\-based style set\. Unless otherwise noted, we report decision\-level accuracy with forced\-choice readout \(see §[2\.4](https://arxiv.org/html/2607.20469#S2.SS4.SSS0.Px4)and Table[8](https://arxiv.org/html/2607.20469#A0.T8)\)\.

![Refer to caption](https://arxiv.org/html/2607.20469v1/x6.png)Figure 4:H1: a small but statistically significant shared subspace \(layerℓ=10\\ell\{=\}10\)\.Scale of the shared core\|𝒮ℓ​\(τ,m\)\|\|\\mathcal\{S\}\_\{\\ell\}\(\\tau,m\)\|relative to the full layer widthddand pooled PCA dimensionkk\. Although\|𝒮ℓ\|\|\\mathcal\{S\}\_\{\\ell\}\|is only a few percent ofdd\(and ofkk\), both permutation and scramble tests reject the null \(Table[11](https://arxiv.org/html/2607.20469#A2.T11)\)\.
### 3\.1H1Shared structure exists at decision time

We begin by testingH1: whether decode\-time decision states exhibit cross\-task shared structure beyond chance\. We compute the shared\-set sizeTℓ≜\|Sℓ​\(τ,m\)\|T\_\{\\ell\}\\triangleq\|S\_\{\\ell\}\(\\tau,m\)\|at layerℓ\\elland compare it against the two task\-preserving nulls in §[2\.3](https://arxiv.org/html/2607.20469#S2.SS3)\. We reject H1 ifTℓT\_\{\\ell\}is not significantly larger than either null \(or collapses to near\-zero across layers/models\)\.

#### H1\.1 Existence and significance under strong nulls\.

Using the strict all\-tasks sharing rule \(mshared=allm\_\{\\mathrm\{shared\}\}=\\texttt\{all\}\) at a fixed internal layer, we consistently recover a small but nontrivial shared set across models \(Table[11](https://arxiv.org/html/2607.20469#A2.T11)\)\. Against both task\-preserving null baselines \(within\-task permutation and orthogonal\-scramble; §[2\.3](https://arxiv.org/html/2607.20469#S2.SS3)\), the null shared counts remain near\-zero and never match the observedTℓT\_\{\\ell\}, so the Monte Carlo tests reach the finite\-sampling upper\-bound regime reported in Table[11](https://arxiv.org/html/2607.20469#A2.T11)\. Together, this rules out chance\-level explanations under strong nulls and supports that the decode\-time shared subspace is a robust signal\. \(Full per\-layer sweeps are deferred to Appendix Table[12](https://arxiv.org/html/2607.20469#A2.T12)\.\)

#### H1\.2 Threshold calibration and robustness\.

We next test sensitivity to PCA retentionρ\\rho\(which changes the retained dimensionkk\) and the sharedness thresholdτ\\tau\. To compare thresholds across differentkk, we interpretτ\\taurelative to the diffuse baseline1/k1/kand summarize stringency by the dimensionless scaleτ​k\\tau k\(equivalently,τ=𝒪​\(1/k\)\\tau=\\mathcal\{O\}\(1/k\)whenτ​k=𝒪​\(1\)\\tau k=\\mathcal\{O\}\(1\)\)\. Sweepingρ\\rhoandτ\\tau\(Fig\.[3](https://arxiv.org/html/2607.20469#S2.F3)\) reveals a stable intermediate regime where the shared set remains compact and varies smoothly withρ\\rho, whereas overly permissive/strict thresholds lead to the expected degenerate behaviors \(nearly\-all shared vs\. near\-empty\)\. Our defaultτ=10−3\\tau=10^\{\-3\}lies in this scale\-compatible regime across the sweep \(Fig\.[3](https://arxiv.org/html/2607.20469#S2.F3), inset figure\); additional calibrations and variants are in Appendix[B\.1](https://arxiv.org/html/2607.20469#A2.SS1)\.

#### Where is sharing stronger?

As an auxiliary analysis, within\-category pairs show notably stronger sharing for mathematical reasoning, while other coarse categories are close to the mixed\-category baseline \(Appendix[B\.1](https://arxiv.org/html/2607.20469#A2.SS1), Fig\.[13\(a\)](https://arxiv.org/html/2607.20469#A2.F13.sf1)\)\.

Flip Example \(forced\-choice; ex\_id=aqua\-test\-9\)Gold:BIntervention:shared\-removal at layerℓ=10\\ell\{=\}10\(decode\-only\)\.Prompt \(truncated\):Question: A newspaper costs $4 on Sunday \.\.\. how many newspapers does it buy on Monday? Choices: A\)45 B\)15 C\)60 D\)30 E\)75baseline:B\(correct=true, margin=\+1\.08\)
shared\-removed:A\(correct=false, margin=\-3\.30\)*\(flip\)*Figure 5:Flip case study \(forced\-choice\)\.The baseline forced\-choice prediction matches the gold answer\. Removing the shared decode subspace at layerℓ=10\\ell=10during cached decoding flips the predicted choice demonstrating that the shared decode pathway can be*causally necessary*\.

### 3\.2H2Decode\-time shared subspace is causal

We test whether the estimated shared subspace𝒮ℓ\\mathcal\{S\}\_\{\\ell\}is*causally necessary*for decode\-time decisions under KV\-cached decoding\. We use three complementary checks: \(i\) decode\-only ablation versus budget\-matched non\-shared controls, \(ii\) decision\-level evaluation and leave\-one\-task\-out \(LOTO\) re\-estimation to rule out scoring/leakage confounds, and \(iii\) patchback to test sufficiency and specificity \(§[2\.4](https://arxiv.org/html/2607.20469#S2.SS4); Appendix[B\.2](https://arxiv.org/html/2607.20469#A2.SS2)\)\. The formal definition of patchback and flips are available at Appendix[Definition of Patchback Experiments\.](https://arxiv.org/html/2607.20469#A0.SS0.SSS0.Px2)\.

#### H2\.1 Decode\-only ablation vs\. matched controls

Removing𝒮ℓ\\mathcal\{S\}\_\{\\ell\}*only during KV\-cached decoding*causes a consistent performance collapse across tasks and models \(Table[30](https://arxiv.org/html/2607.20469#A3.T30); Fig\.[7](https://arxiv.org/html/2607.20469#S3.F7)\), whereas dimension/energy\-matched non\-shared controls stay near baseline\. This separation indicates a direction\-specific effect rather than a generic consequence of reducing activation energy or rank\.

Because open\-ended generation accuracy can be confounded by format/termination failures \(e\.g\., extraction/EOS issues\), we also evaluate a staged protocol that constrains only the final answer format\. The same shared–control gap persists under staging \(Table[32](https://arxiv.org/html/2607.20469#A3.T32)\), motivating decision\-level tests next\.

#### H2\.2 Decision\-level degradation

To isolate decision quality from generation formatting, we evaluate a forced\-choice readout that ranks candidate completions \(§[2\.4](https://arxiv.org/html/2607.20469#S2.SS4)\)\. Shared\-subspace removal still produces a clear decision\-level drop, while the matched non\-shared control remains near baseline \(Table[31](https://arxiv.org/html/2607.20469#A3.T31), Part B; Fig\.[7](https://arxiv.org/html/2607.20469#S3.F7)\)\.

We further rule out task leakage in subspace construction by re\-estimating𝒮ℓ\\mathcal\{S\}\_\{\\ell\}in a leave\-one\-task\-out manner \(LOTO\)\. The qualitative signature is unchanged: LOTO shared removal remains harmful under generation \(Table[31](https://arxiv.org/html/2607.20469#A3.T31), Part A\) and under forced\-choice \(Table[31](https://arxiv.org/html/2607.20469#A3.T31), Part B\), while matched controls stay near baseline \(Table[31](https://arxiv.org/html/2607.20469#A3.T31)\)\. Together, these results support that𝒮ℓ\\mathcal\{S\}\_\{\\ell\}is causally involved in decode\-time*decisions*, not merely output formatting\.

#### Case study \(flip\)\.

To make the decision\-level effect concrete,[Figure5](https://arxiv.org/html/2607.20469#S3.F5)shows a baseline\-correct AQuA example whose prediction flips under decode\-only removal of the shared decode\-time subspace𝒮ℓ\\mathcal\{S\}\_\{\\ell\}\(atℓ=10\\ell=10\), illustrating causal necessity\.

Patchback example \(boolq; ex\_id=boolq\-validation\-3\)Dataset: boolq
Gold:B
Prompt\(truncated\):Question: Passage: Open carry is also legal throughout North Carolina\. In the town of Chapel Hill, open carry is restricted to guns of a certain minimum size \.\.\. \\nQuestion do you need a permit to open carry in nc Choices: A\) Yes B\) No Reason step by step\. At the end, write exactly one line: "Final answer: <A/B\>"\.baseline:B\(correct=true, margin=\+5\.19\)
patchback \(W=\{0,1\}\):B\(correct=true, margin=\+5\.19\)
rand subspace:A\(correct=false, margin=\-3\.66\)
time\-shuf donor:B\(correct=true, margin=\+2\.84\)
non\-shared patch:A\(correct=false, margin=\-1\.96\)
Figure 6:Patchback case study \(forced\-choice\)\.Patching back only the removed*shared*component over a narrow decode windowW=\{0,1\}W=\\\{0,1\\\}restores the baseline answer, while energy/dimension\-matched controls do not\.![Refer to caption](https://arxiv.org/html/2607.20469v1/x7.png)

![Refer to caption](https://arxiv.org/html/2607.20469v1/x8.png)

![Refer to caption](https://arxiv.org/html/2607.20469v1/x9.png)

Figure 7:Subspace intervention results\.From left to right: \(a\) All\-tasks Generation, \(b\) LOTO Generation, \(c\) LOTO Forced\-Choice Generation\. Rectangles show 95% CIs; vertical ticks indicate means; numbers are mean accuracies\. More experimental details are available in Appendix[B\.2](https://arxiv.org/html/2607.20469#A2.SS2), §[2\.4](https://arxiv.org/html/2607.20469#S2.SS4),and Table[31](https://arxiv.org/html/2607.20469#A3.T31),[32](https://arxiv.org/html/2607.20469#A3.T32)\.
#### H2\.3 Patchback closes the causal loop\.

Decode\-only ablation establishes necessity; patchback tests whether the removed shared component is also sufficient to*rescue*the decision\. We record the shared componentΠℓ​hℓ\(s\)\\Pi\_\{\\ell\}h\_\{\\ell\}^\{\(s\)\}from the unmodified run, rerun the ablated model, and restore*only*this shared component over either the full intervention window or a single decode step \(details in §[2\.4](https://arxiv.org/html/2607.20469#S2.SS4)\)\.

Across models/layers, targeted patching rescues a large fraction of ablation\-induced flips, while energy\-matched controls \(random vectors within𝒮ℓ\\mathcal\{S\}\_\{\\ell\}, random subspaces of matched dimension, or patching outside𝒮ℓ\\mathcal\{S\}\_\{\\ell\}\) do not \(Table[1](https://arxiv.org/html/2607.20469#S3.T1)\)\. In particular, patching outside𝒮ℓ\\mathcal\{S\}\_\{\\ell\}yields no rescue throughout, demonstrating strong specificity\.

#### Case study \(patchback\)\.

[Figure6](https://arxiv.org/html/2607.20469#S3.F6)illustrates a representative forced\-choice example: patching back only the removed*shared*component over a narrow windowW=\{0,1\}W=\\\{0,1\\\}restores the baseline answer, whereas energy/dimension\-matched controls \(random subspaces or non\-shared patches\) fail\.

Two additional diagnostics address common patchback confounds \(Appendix[B\.2](https://arxiv.org/html/2607.20469#A2.SS2)\)\. On the AQuA flip\-set, we run*transfer\-donor*patching by taking the recorded shared component from a*different*baseline\-correct example \(optionally from a different task\) and patching it into the ablated run\. This rescues flips about as often as self\-donor patching, so the rescue is not just copying a sample\-specific answer\. For open\-answer patchback, we run a*time\-shuffle*control by taking the shared component from a different decode step of the same example and using it as the donor\. This works almost as well asPatched\(self\), which means the shared signal in𝒮ℓ\\mathcal\{S\}\_\{\\ell\}changes little across decode steps and a single\-step patch is often enough\.

### 3\.3H3Prefill\-to\-decode transfer often fails

Setup\.We test whether the shared\-subspace estimate must be aligned to the*decode\-time*distribution\. At the same layerℓ\\ell, we estimate shared bases fromdecodestates versusprefillstates with matched rankkmatchk\_\{\\text\{match\}\}, and evaluate the resulting2×22\\times 2estimator–intervention grid under identical budgets and matched random controls \(Table[7](https://arxiv.org/html/2607.20469#S3.T7)\)\. We fix the intervention locus to decode\-only and then swap the estimator \(decode\-est vs\. prefill\-est\), which directly tests prefill\-decode mismatch\.

Table 1:Patchback on flips \(multiple\-choice\), aggregated over tasks\.\|ℱ\|\|\\mathcal\{F\}\|counts baseline→\\toablated flips\. Rescue rates are computed onℱ\\mathcal\{F\}and aggregated across tasks by summing rescued/total counts\. Green columns are targeted patching along the shared decode subspaceSℓS\_\{\\ell\}; blue columns are controls\.ModelLayer\|ℱ\|\|\\mathcal\{F\}\|Targeted patchControlsPatched@0Patched@fullRandVec\(shared\)RandSubspaceNonsharedPatchLlama\-2\-7B\-Chat420284\.2 %\(170/202\)100\.0 %\(202/202\)19\.3 %\(39/202\)16\.3 %\(33/202\)0\.0 %\(0/202\)Llama\-2\-7B\-Chat1044985\.7 %\(385/449\)100\.0 %\(449/449\)7\.6 %\(34/449\)5\.3 %\(24/449\)0\.0 %\(0/449\)Llama\-2\-7B\-Chat2429765\.0 %\(193/297\)100\.0 %\(297/297\)8\.8 %\(26/297\)6\.1 %\(18/297\)0\.0 %\(0/297\)Falcon\-7B\-Instruct491100\.0 %\(91/91\)100\.0 %\(91/91\)9\.9 %\(9/91\)0\.0 %\(0/91\)0\.0 %\(0/91\)Falcon\-7B\-Instruct1097100\.0 %\(97/97\)100\.0 %\(97/97\)0\.0 %\(0/97\)1\.0 %\(1/97\)0\.0 %\(0/97\)Falcon\-7B\-Instruct24192100\.0 %\(192/192\)100\.0 %\(192/192\)19\.3 %\(37/192\)19\.8 %\(38/192\)0\.0 %\(0/192\)Qwen2\.5\-7B\-Instruct4325100\.0 %\(325/325\)100\.0 %\(325/325\)32\.6 %\(106/325\)32\.9 %\(107/325\)0\.0 %\(0/325\)Qwen2\.5\-7B\-Instruct10548100\.0 %\(548/548\)100\.0 %\(548/548\)36\.1 %\(198/548\)49\.8 %\(273/548\)0\.0 %\(0/548\)Qwen2\.5\-7B\-Instruct24453100\.0 %\(453/453\)100\.0 %\(453/453\)37\.5 %\(170/453\)39\.3 %\(178/453\)0\.0 %\(0/453\)Geometric mismatch\.The prefill\- and decode\-estimated shared subspaces are strongly misaligned even after rank matching \(Table[20](https://arxiv.org/html/2607.20469#A2.T20); layer sweep in Appendix[B\.4](https://arxiv.org/html/2607.20469#A2.SS4)\), indicating that prefill activations do not provide a reliable geometric proxy for the decode\-time shared workspace\.

Causal consequence at decode\.This mismatch matters causally: under decode\-only intervention, ablating the*decode\-estimated*shared basis produces a large and consistent accuracy drop, while ablating the*prefill\-estimated*basis largely removes the effect and behaves like matched random controls \(Table[7](https://arxiv.org/html/2607.20469#S3.T7)\)\. Interventions at prefill are negligible across conditions\. Together, these results support a distribution\-aligned view: to reproduce decode\-locus causal effects under matched budgets, the shared subspace must be estimated from decode\-time states\.

Table 2:The shared channel is not a narrow readout slice\.A 32D Llama\-2\-7B layer\-10 workspace is split into a 3D readout sliceQoutQ\_\{\\rm out\}and a 29D coreQcoreQ\_\{\\rm core\}\. Panels A and B report forced\-choice accuracy \(baseline: 44\.1%\) and held\-out probe AP, respectively\.A\. Causal ablationAblated subspaceDim\.Acc\.Δ\\DeltaAcc\.Full shared3228\.5−15\.6\-15\.6QoutQ\_\{\\rm out\}344\.9\+0\.8\+0\.8QcoreQ\_\{\\rm core\}2927\.3−16\.8\-16\.8B\. Held\-out probe APProbe tagQcoreQ\_\{\\rm core\}QoutQ\_\{\\rm out\}Δ\\DeltaReasoning markers0\.5640\.041\+0\.523Step markers0\.1690\.029\+0\.140Digits0\.9660\.132\+0\.834Equation symbols0\.6730\.055\+0\.618Table 3:Decode\-shared directions are enriched for decision\-scaffold tokens\.Vocab\-alignment scores \(mean overlaps\) for Llama\-2\-7B layer 28\.Token familySharedNonsharedPrefill sharedAnswer scaffold0\.2830\.2130\.208Correctness markers0\.2070\.1890\.182Confidence markers0\.2340\.1950\.221Newline0\.3740\.2040\.190Digits0\.3140\.2610\.159Sentiment markers0\.1710\.1760\.182

#### What does the shared channel encode?

While our causal interventions establish the decision\-relevance of the decode\-shared subspace, they do not specify its computational vocabulary\. To characterize its contents, we decouple the 32D shared workspace into a 3D readout sliceQoutQ\_\{\\rm out\}and its 29D residual coreQcoreQ\_\{\\rm core\}\(Table[2](https://arxiv.org/html/2607.20469#S3.T2)\)\.

Causal ablation reveals a clear functional divergence between these two components\. SuppressingQcoreQ\_\{\\rm core\}reproduces the massive performance drop of the full subspace \(over 15% loss\), whereas removingQoutQ\_\{\\rm out\}leaves the model performance intact \(Table[2](https://arxiv.org/html/2607.20469#S3.T2)A\)\. Linear probing results in Panel B explain this divergence:QcoreQ\_\{\\rm core\}maintains high predictive accuracy for procedural and symbolic tokens, whileQoutQ\_\{\\rm out\}remains near baseline\. Therefore, the geometric split matches the functional split: the residual core stores structured task information, while the smaller slice serves as a clean interface for reading out tokens\.

Second, a logit\-lens analysis shows that these decode\-shared directions are uniquely enriched for answer\-scaffold and formatting tokens, but not for generic sentiment markers \(Table[3](https://arxiv.org/html/2607.20469#S3.T3)\)\. This selectivity confirms that the channel does not process all semantic information indiscriminately\. Instead, it operates specifically as a decode\-time decision scaffold for procedural reasoning and verification\.

#### Scale Diagnostics\.

Targeted non\-7B checks further show the same qualitative pattern at 3B, 13B, and 70B; we report these robustness checks in Appendix[C\.4](https://arxiv.org/html/2607.20469#A3.SS4)\.

Table 4:Decode\-aligned ranking better matches held\-out decode utility\.The pools contain 32 CAA contrastive vectors, 64 instruction\-derived vectors, 64 SAE feature directions, and 100 diagnostic directions\.PoolPrefillρ\\rhoDecodeρ\\rhoΔ\\DeltaCAA contrastive\-0\.3700\.700\+1\.070Instruction0\.1720\.767\+0\.595SAE features\-0\.0640\.594\+0\.659Diagnostic0\.0650\.700\+0\.635Table 5:Decode\-aligned validation selects better deployed vectors\.Metrics are held\-out accuracy \(utility\) evaluated on the real\-world downstream task \(REAL\) for selected vectors\.ProxyREAL meanREAL worstFlip rateRegret@1Prefill\-aligned\-0\.002\-0\.0030\.7500\.016Mixed Stages\-0\.002\-0\.0030\.7500\.016Decode\-aligned\+0\.011\+0\.0100\.0830\.003

Table 6:Multibench behavior under KV\-cache decode\.Mean signed margin shift \(μ\\mu\), template std \(σtmpl\\sigma\_\{\\text\{tmpl\}\}\), worst\-case template mean \(worst; primary\), and worst\-case anti\-steer rate \(antiworst\{\}\_\{\\text\{worst\}\}; diagnostic, lower is better\)\.sh​\(v\)\\mathrm\{sh\}\(v\)is overlap with the decode\-time shared basis\. Partial projection:vα=v−α​Q​Q⊤​vv\_\{\\alpha\}=v\-\\alpha QQ^\{\\top\}v;Ris energy\-matched random control\. Rightmost columns reportΔ\\Deltabetweenα=1\\alpha\{=\}1and0\.μ↑\\mu\\uparrowσtmpl↓\\sigma\_\{\\text\{tmpl\}\}\\downarrowworst↑\\uparrowanti↓worst\{\}\_\{\\text\{worst\}\}\\downarrowΔ\\Delta\(1−\-0\)TaskCand\.sh​\(v\)\\mathrm\{sh\}\(v\)0\.51R010\.51R01RΔworst\\Delta\_\{\\text\{worst\}\}Δanti\\Delta\_\{\\text\{anti\}\}BoolQY/N\.405\.405\.0383\.0383\.0357\.0357\.0312\.0312−\.0100\-\.0100\.0019\.0019\.0034\.0034\.0357\.0357\.0321\.0321\.0267\.0267−\.0118\-\.0118\.3633\.3633\.3633\.3633\.5695\.5695−\.0090\-\.0090\.0000\.0000RTET/F\.386\.386\.0173\.0173\.0171\.0171\.0162\.0162−\.0005\-\.0005\.0095\.0095\.0085\.0085\.0104\.0104\.0101\.0101\.0089\.0089−\.0033\-\.0033\.4844\.4844\.4648\.4648\.5195\.5195−\.0015\-\.0015−\.0196\-\.0196SST\-2G/B\.391\.391\.0099\.0099\.0130\.0130\.0154\.0154\.0012\.0012\.0126\.0126\.0134\.0134−\.0007\-\.0007\.0011\.0011\.0028\.0028−\.0082\-\.0082\.4336\.4336\.4648\.4648\.5594\.5594\+\.0035\+\.0035\+\.0312\+\.0312

Table 7:H32×22\\times 2estimation\-intervention grid\.Accuracy \(%\) under matchedkk\(herek=48k\{=\}48\) at layerℓ=10\\ell\{=\}10with the same removal strength \(α=1\\alpha\{=\}1\) across conditions\. Bold indicates the lowest accuracy within each row \(ties broken left\-to\-right\)\.Intervene: DecodeIntervene: PrefillTaskBaseDec\-estPre\-estRandDec\-estPre\-estRandCSQA53\.353\.323\.123\.154\.354\.353\.553\.553\.253\.253\.353\.353\.353\.3StratQA49\.649\.652\.852\.849\.649\.649\.649\.649\.849\.849\.649\.649\.649\.6PIQA69\.169\.152\.752\.767\.667\.669\.469\.469\.569\.569\.269\.269\.269\.2ARC\-C51\.551\.526\.526\.550\.950\.951\.951\.952\.652\.652\.652\.651\.651\.6OBQA51\.251\.227\.227\.251\.851\.851\.851\.853\.853\.852\.852\.851\.251\.2QASC48\.948\.913\.713\.750\.350\.349\.549\.549\.049\.049\.749\.748\.848\.8LogiQA32\.132\.123\.223\.232\.732\.732\.132\.132\.432\.432\.932\.932\.132\.1MeanΔ\\Deltavs\. base–−19\.5\-19\.50\.20\.20\.30\.30\.70\.70\.60\.60\.00\.0

## 4Downstream Utility

#### Steering Robustness\.

We study a drop\-in repair for arbitrary steering directionsvvunder true KV\-cached decoding\. From neutral prompts, we estimate an orthonormal decode\-time shared basisQQand measure shared overlapsh​\(v\)=‖Q⊤​v‖2/‖v‖2\\mathrm\{sh\}\(v\)=\\\|Q^\{\\top\}v\\\|\_\{2\}/\\\|v\\\|\_\{2\}, which is consistently nontrivial across tasks \(Table[6](https://arxiv.org/html/2607.20469#S3.T6)\)\. We then repairvvby removing its shared component:

vα=\(I−α​Q​Q⊤\)​v,α∈\[0,1\],v\_\{\\alpha\}=\(I\-\\alpha QQ^\{\\top\}\)v,\\qquad\\alpha\\in\[0,1\],whereα=1\\alpha=1yields the fully repairedvfixed=\(I−Q​Q⊤\)​vv\_\{\\text\{fixed\}\}=\(I\-QQ^\{\\top\}\)v\.

Across tasks, the repaired direction remains clearly non\-random under KV\-cached decoding \(positive mean signed shift and separation from energy\-matched random controls\), while often reducing template sensitivity; the magnitude of the robustness gain is task\-dependent \(Table[6](https://arxiv.org/html/2607.20469#S3.T6)\)\. Overall, this supports shared\-subspace interference as a concrete source of template brittleness, and motivatesvαv\_\{\\alpha\}as a simple offline knob that trades a small amount of average effect for robustness under KV\-cached decoding\.

#### Stage\-aligned vector selection\.

Across 260 contrastive, steering, and feature vectors, decode\-stage ranking aligns better with downstream decode utility than prefill\-stage ranking and mixed prefill\-decode ranking \(Table[4](https://arxiv.org/html/2607.20469#S3.T4)\)\. This difference directly improves deployment: evaluated on our downstream benchmark, decode\-stage validation selects vectors with positive accuracy gains \(utility\), lower flip rates, and lower regret than prefill\-stage or mixed always\-on validation \(Table[5](https://arxiv.org/html/2607.20469#S3.T5)\)\. This trend occurs because prefill and mixed signals fail to isolate decode\-specific token dynamics, often leading to sub\-optimal vector choices\. In contrast, decode\-stage validation precisely identifies vectors that maintain stable steering effects during generation, thereby maximizing downstream deployment utility\.

#### Style case study\.

We illustrate the same knob on a lightweight pirate\-lexicon steering task \(Appendix[C\.3](https://arxiv.org/html/2607.20469#A3.SS3.SSS0.Px1)\)\. As the retained rankkkgrows, the shared span captures more ofvv, while the repaired directionvfixed,kv\_\{\\mathrm\{fixed\},k\}becomes numerically orthogonal to it \(Table[28](https://arxiv.org/html/2607.20469#A3.T28)\)\. Behaviorally, moderate projection can improve both average effect and worst\-case template performance, whereas overly aggressive removal can suppress the steering signal \(Table[33](https://arxiv.org/html/2607.20469#A3.T33)\), consistent with a denoise–excise tradeoff\.

## 5Related Works

#### Causal interventions & patching\.

Activation\-level causal interventions, including activation patching/causal tracing, causal mediation, and interchange interventions, are widely used to localize and validate mechanisms in LLMs\(Viget al\.,[2020](https://arxiv.org/html/2607.20469#bib.bib35); Menget al\.,[2022](https://arxiv.org/html/2607.20469#bib.bib14); Wanget al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib15); Geigeret al\.,[2021](https://arxiv.org/html/2607.20469#bib.bib36),[2024](https://arxiv.org/html/2607.20469#bib.bib37)\)\. At the same time, recent analyses emphasize that patching results can be sensitive to metrics and implementation details, and that subspace patching can create interpretability illusions without appropriate controls\(Zhang and Nanda,[2024](https://arxiv.org/html/2607.20469#bib.bib38); Makelovet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib39)\)\. Motivated by these practices, we intervene*only*on KV\-cached decode\-time decision states and pair shared\-subspace ablations with matched nulls and budget\-matched controls, then close the loop with patchback specificity tests \(H2\)\.

#### KV\-cached decoding and cache management\.

KV caching is a first\-class serving regime for modern LLM inference \(e\.g\., paging\-based decoding\) and is also central to streaming/long\-context systems that explicitly manage rolling caches\(Kwonet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib40); Xiaoet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib42)\)\. A growing body of work studies what to retain and how to compress KV caches—including eviction/heavy\-hitters, low\-rank or reconstruction\-based compression, and cross\-layer schemes\(Zhanget al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib43); Geet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib44); Saxenaet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib3); Liet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib45); Caiet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib46); Kimet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib47); Changet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib48); Khalafet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib4)\)\. These efforts motivate our emphasis on*decode\-time*measurements and interventions under true KV\-cached decoding; we defer additional system background to Appendix[D\.1](https://arxiv.org/html/2607.20469#A4.SS1)\.

#### Representation engineering and decision\-level readouts\.

Representation engineering and activation steering show that low\-dimensional directions can reliably influence generations, but their behavior can degrade under context or regime shifts\(Turneret al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib19); Rimskyet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib20); Zouet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib22); Tanet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib26); Denget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib27)\)\. To separate decision quality from formatting/termination artifacts in open\-ended generation, we adopt decision\-level probes based on layer\-wise vocabulary readouts \(lens\-style methods\)\(Belroseet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib49); Gevaet al\.,[2022](https://arxiv.org/html/2607.20469#bib.bib50)\), and use them to quantify regime\-specific degradation under KV\-cached decode\-time interventions\.

#### Shared computation and low\-dimensional structure\.

Circuit\-style mechanistic interpretability argues that Transformers can reuse compact computational motifs across diverse behaviors \(e\.g\., induction\-like mechanisms\), motivating the search for shared structure across tasks\(Elhageet al\.,[2021](https://arxiv.org/html/2607.20469#bib.bib12); Olssonet al\.,[2022](https://arxiv.org/html/2607.20469#bib.bib13); Merulloet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib16); Bhaskaret al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib18)\)\. Complementarily, representation similarity tools \(SVCCA/CKA\) and “shared subspace” hypotheses suggest that important computation may concentrate in low\-dimensional subspaces that persist across settings\(Raghuet al\.,[2017](https://arxiv.org/html/2607.20469#bib.bib11); Kornblithet al\.,[2019](https://arxiv.org/html/2607.20469#bib.bib10); Kaushiket al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib17); Wanget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib7)\)\.DecodeShareoperationalizes this perspective at the*decode\-locus*: we estimate a cross\-task shared workspace from KV\-cached decision states and validate it with decode\-only causal tests \(H2\), while explicitly probing estimator–deployment mismatch \(H3\)\.

#### Differences from closely related subspace work\.

Prior work studies shared/orthogonal subspaces mainly at the*parameter*level \(e\.g\., task\-update geometry or orthogonal LoRA/merging\)\(Arturiet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib53); Zhang and Zhou,[2025](https://arxiv.org/html/2607.20469#bib.bib56)\), or compares shared semantics across models/scales or modalities\(Sonet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib55); Whitakeret al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib59)\)\. We instead identify an*activation\-level*workspace that appears in KV\-cached*decode\-time decision states*within a fixed model and test it causally with decode\-only interventions, matched controls, and patchback\. Related projection/subspace\-learning methods remove nuisance directions or learn transfer structure during tuning\(Xieet al\.,[2022](https://arxiv.org/html/2607.20469#bib.bib57); Falissardet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib58)\); our target is a task\-shared*decode\-time*workspace, and we explicitly study when prefill\-estimated bases fail to transfer to decode\-time causal effects \(H3\)\. MSRS composes multiple subspaces for attribute control\(Jianget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib54)\); our focus is diagnostic rather than steering\.

## 6Conclusion

We proposeDecodeShare, a protocol that estimates a cross\-task shared subspace from pooled decode\-time activations and tests its causal role via projection interventions with budget\-matched controls\. The shared subspace is low\-dimensional yet consistently affects decoding decisions\. We also observe a regime gap: prefill\-estimated directions can misalign with decode\-time behavior\. Crucially, our stage\-aligned protocol exploits this gap to successfully optimize downstream vector selection over prefill or mixed baselines\. More broadly, our results highlight decode\-time state geometry as a first\-class object for mechanistic analysis under KV caching, suggesting that intervention design should be evaluated in the serving regime rather than inferred from prefill statistics alone\. We hope this encourages decode\-centric evaluations in future LLM interpretability works\.

## Acknowledgment

This work was supported by the National Science Foundation \(NSF\) under Grant No\. IIS\-2451480 and Grant No\. 2112562, and the Army Research Office \(ARO\) under Grant No\. W911NF\-23\-2\-0224\.

## Impact Statement

DecodeShare provides a protocol for exploring shared structure in model decisions during decoding\. It estimates a low\-dimensional subspace from KV\-cached decode\-time activations across tasks, then projects it out during decoding and measures output changes using matched random and energy controls\. This supports causal tests of whether shared decode\-time structure influences decisions and how it overlaps with steering directions studied in current works on steering vectors and style control\. Potential positive impacts are primarily methodological and practical: clearer cross\-task comparisons of decode\-time representations, more reproducible intervention studies via matched controls, and the ability to separate the functional roles of steering directions from task\-general decode channels\. By shifting focus from prefill\-based proxies to decode\-time evaluations, the protocol also provides a more reliable signal for downstream deployment\. Potential negative impacts appear limited\. The protocol requires white\-box access and per\-model estimation, so it is not directly applicable to black\-box settings\. However, because the identified subspace generalizes robustly across tasks, actors with internal access could potentially exploit these universal directions for broader behavioral manipulation; we recommend conservative intervention budgets and reporting safety and stability metrics\.

## References

- A\. Arditi, O\. Obeso, A\. Syed, D\. Paleka, N\. Panickssery, W\. Gurnee, and N\. Nanda \(2024\)Refusal in language models is mediated by a single direction\.arXiv preprint arXiv:2406\.11717\.External Links:[Link](https://arxiv.org/abs/2406.11717)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§1](https://arxiv.org/html/2607.20469#S1.p4.1)\.
- D\. A\. R\. Arturi, E\. Zhang, A\. A\. Ansah, K\. Zhu, A\. Panda, and A\. Balwani \(2025\)Shared parameter subspaces and cross\-task linearity in emergently misaligned behaviour\.InMechanistic Interpretability Workshop at NeurIPS 2025,Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px5.p1.1)\.
- M\. Belitsky, D\. J\. Kopiczko, M\. Dorkenwald, M\. J\. Mirza, J\. R\. Glass, C\. G\. Snoek, and Y\. M\. Asano \(2025\)KV cache steering for controlling frozen llms\.arXiv preprint arXiv:2507\.08799\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1)\.
- N\. Belrose, I\. Ostrovsky, L\. McKinney, Z\. Furman, L\. Smith, D\. Halawi, S\. Biderman, and J\. Steinhardt \(2023\)Eliciting latent predictions from transformers with the tuned lens\.arXiv preprint arXiv:2303\.08112\.External Links:[Link](https://arxiv.org/abs/2303.08112)Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px3.p1.1)\.
- L\. Bereska and E\. Gavves \(2024\)Mechanistic interpretability for AI safety: a review\.arXiv preprint arXiv:2404\.14082\.External Links:[Link](https://arxiv.org/abs/2404.14082)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1)\.
- A\. Bhaskar, A\. Wettig, D\. Friedman, and D\. Chen \(2024\)Finding transformer circuits with edge pruning\.Advances in Neural Information Processing Systems37,pp\. 18506–18534\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- Z\. Cai, Y\. Zhang, B\. Gao, Y\. Liu, T\. Liu, K\. Lu, W\. Xiong, Y\. Dong, B\. Chang, J\. Hu, and W\. Xiao \(2024\)PyramidKV: dynamic KV cache compression based on pyramidal information funneling\.arXiv preprint arXiv:2406\.02069\.External Links:[Link](https://arxiv.org/abs/2406.02069)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- C\. Chang, C\. Lin, Y\. Akhauri, W\. Lin, K\. Wu, L\. Ceze, and M\. S\. Abdelfattah \(2025\)XKV: cross\-layer SVD for KV\-cache compression\.arXiv preprint arXiv:2503\.18893\.External Links:[Link](https://arxiv.org/abs/2503.18893)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- Z\. Deng, J\. Jiang, G\. Long, and C\. Zhang \(2025\)Rethinking the reliability of representation engineering in large language models\.Note:OpenReviewSubmitted to ICLR 2025External Links:[Link](https://openreview.net/forum?id=sYJQEgkkaI)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p2.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px3.p1.1)\.
- N\. Elhage, N\. Nanda, C\. Olsson, T\. Henighan,et al\.\(2021\)A mathematical framework for transformer circuits\.Note:Transformer Circuits Thread \(online\)Published Dec 22, 2021\. Accessed 2026\-01\-25External Links:[Link](https://www.anthropic.com/research/a-mathematical-framework-for-transformer-circuits)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- L\. Falissard, V\. Guigue, and L\. Soulier \(2023\)Improving generalization in large langue model by learning prefix subspaces\.InFindings of the Association for Computational Linguistics: EMNLP 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 11474–11483\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.768/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.768)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px5.p1.1)\.
- S\. Ge, Y\. Zhang, L\. Liu, M\. Zhang, J\. Han, and J\. Gao \(2024\)Model tells you what to discard: adaptive KV cache compression for LLMs\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=uNrFpDPMyo)Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- A\. Geiger, H\. Lu, T\. F\. Icard, and C\. Potts \(2021\)Causal abstractions of neural networks\.InAdvances in Neural Information Processing Systems,A\. Beygelzimer, Y\. Dauphin, P\. Liang, and J\. W\. Vaughan \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=RmuXDtjDhG)Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px1.p1.1)\.
- A\. Geiger, Z\. Wu, C\. Potts, T\. Icard, and N\. Goodman \(2024\)Finding alignments between interpretable causal variables and distributed neural representations\.InCausal Learning and Reasoning,pp\. 160–187\.Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px1.p1.1)\.
- M\. Geva, A\. Caciularu, K\. Wang, and Y\. Goldberg \(2022\)Transformer feed\-forward layers build predictions by promoting concepts in the vocabulary space\.InProceedings of the 2022 conference on empirical methods in natural language processing,pp\. 30–45\.Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px3.p1.1)\.
- X\. Jiang, L\. Zhang, J\. Zhang, Q\. Yang, G\. Hu, D\. Wang, and L\. Hu \(2025\)Msrs: adaptive multi\-subspace representation steering for attribute alignment in large language models\.arXiv preprint arXiv:2508\.10599\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px5.p1.1)\.
- P\. Kaushik, S\. Chaudhari, A\. Vaidya, R\. Chellappa, and A\. Yuille \(2025\)The universal weight subspace hypothesis\.arXiv preprint arXiv:2512\.05117\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- M\. Khalaf, Y\. Shamshoum, N\. Hodos, Y\. Sieradzki, and A\. Schuster \(2025\)QKV projections require a fraction of their memory\.arXiv preprint arXiv:2506\.02939\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- J\. Kim, J\. Kim, S\. Kwon, J\. W\. Lee, S\. Yun, and H\. O\. Song \(2025\)KVzip: query\-agnostic KV cache compression with context reconstruction\.arXiv preprint arXiv:2505\.23416\.External Links:[Link](https://arxiv.org/abs/2505.23416)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- K\. Konen, S\. Jentzsch, D\. Diallo, P\. Schütt, O\. Bensch, R\. El Baff, D\. Opitz, and T\. Hecking \(2024\)Style vectors for steering generative large language models\.InFindings of the Association for Computational Linguistics: EACL 2024,Y\. Graham and M\. Purver \(Eds\.\),St\. Julian’s, Malta,pp\. 782–802\.External Links:[Link](https://aclanthology.org/2024.findings-eacl.52/)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§1](https://arxiv.org/html/2607.20469#S1.p4.1)\.
- S\. Kornblith, M\. Norouzi, H\. Lee, and G\. Hinton \(2019\)Similarity of neural network representations revisited\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. Chen \(2024\)SnapKV: LLM knows what you are looking for before generation\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=poE54GOq2l)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- A\. Makelov, G\. Lange, A\. Geiger, and N\. Nanda \(2024\)Is this the subspace you are looking for? an interpretability illusion for subspace activation patching\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Ebt7JgMHv1)Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px1.p1.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in gpt\.Advances in neural information processing systems35,pp\. 17359–17372\.Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px1.p1.1)\.
- J\. Merullo, C\. Eickhoff, and E\. Pavlick \(2024\)Circuit component reuse across tasks in transformer language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=fpoAYV6Wsk)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- C\. Olsson, N\. Elhage, N\. Nanda, N\. Joseph, N\. DasSarma, T\. Henighan, B\. Mann, A\. Askell, Y\. Bai, A\. Chen,et al\.\(2022\)In\-context learning and induction heads\.arXiv preprint arXiv:2209\.11895\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- M\. Raghu, J\. Gilmer, J\. Yosinski, and J\. Sohl\-Dickstein \(2017\)SVCCA: singular vector canonical correlation analysis for deep learning dynamics and interpretability\.InProceedings of the 31st International Conference on Neural Information Processing Systems,NIPS’17,Red Hook, NY, USA,pp\. 6078–6087\.External Links:ISBN 9781510860964Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. Turner \(2024\)Steering llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 15504–15522\.External Links:[Link](https://aclanthology.org/2024.acl-long.828/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.828)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p2.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§1](https://arxiv.org/html/2607.20469#S1.p4.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px3.p1.1)\.
- U\. Saxena, G\. Saha, S\. Choudhary, and K\. Roy \(2024\)Eigen attention: attention in low\-rank space for kv cache compression\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 15332–15344\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- Z\. Shao, Y\. Wang, Q\. Wang, T\. Jiang, Z\. Du, H\. Ye, D\. Zhuo, Y\. Chen,et al\.\(2026\)Flashsvd: memory\-efficient inference with streaming for low\-rank models\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 25278–25285\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1)\.
- D\. Son, S\. Rathore, A\. Rufail, A\. Simon, D\. Zhang, S\. Dave, C\. Blondin, S\. O’Brien, and K\. Zhu \(2025\)Semantic convergence: investigating shared representations across scaled llms\.InACL 2025 Student Research Workshop,Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px5.p1.1)\.
- D\. Tan, D\. Chanin, A\. Lynch, B\. Paige, D\. Kanoulas, A\. Garriga\-Alonso, and R\. Kirk \(2024\)Analysing the generalisation and reliability of steering vectors\.Advances in Neural Information Processing Systems37,pp\. 139179–139212\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/fb3ad59a84799bfb8d700e56d19c231b-Abstract-Conference.html)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p2.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px3.p1.1)\.
- E\. Todd, M\. L\. Li, A\. S\. Sharma, A\. Mueller, B\. C\. Wallace, and D\. Bau \(2024\)Function vectors in large language models\.arXiv preprint arXiv:2310\.15213\.Note:Published as a conference paper at ICLR 2024External Links:[Link](https://arxiv.org/abs/2310.15213)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§1](https://arxiv.org/html/2607.20469#S1.p4.1)\.
- A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid \(2023\)Steering language models with activation engineering\.arXiv preprint arXiv:2308\.10248\.External Links:[Link](https://arxiv.org/abs/2308.10248)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.20469#S1.p2.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§1](https://arxiv.org/html/2607.20469#S1.p4.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px3.p1.1)\.
- J\. Vig, S\. Gehrmann, Y\. Belinkov, S\. Qian, D\. Nevo, S\. Sakenis, J\. Huang, Y\. Singer, and S\. M\. Shieber \(2020\)Causal mediation analysis for interpreting neural NLP: the case of gender bias\.arXiv preprint arXiv:2004\.12265\.External Links:[Link](https://arxiv.org/abs/2004.12265)Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px1.p1.1)\.
- J\. Wang, X\. Ge, W\. Shu, Z\. He, and X\. Qiu \(2025\)Attention layers add into low\-dimensional residual subspaces\.InMechanistic Interpretability Workshop at NeurIPS 2025,Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px4.p1.1)\.
- K\. R\. Wang, A\. Variengien, A\. Conmy, B\. Shlegeris, and J\. Steinhardt \(2023\)Interpretability in the wild: a circuit for indirect object identification in GPT\-2 small\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=NpsVSN6o4ul)Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px1.p1.1)\.
- S\. Whitaker, W\. Nasir, and E\. Hart \(2025\)Identifying common semantics across modalities via contrastive latent alignment\.Preprints\.External Links:[Document](https://dx.doi.org/10.20944/preprints202507.0008.v1),[Link](https://doi.org/10.20944/preprints202507.0008.v1)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px5.p1.1)\.
- W\. Wu, Z\. Shao, K\. Cui, J\. Kim, Y\. Wang, H\. Ye, D\. Zhuo, and Y\. Chen \(2026\)FlashSVD v1\. 5: making low\-rank transformers inference actually fast\.arXiv preprint arXiv:2605\.08314\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1)\.
- G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. Lewis \(2024\)Efficient streaming language models with attention sinks\.InThe Twelfth International Conference on Learning Representations,Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- Z\. Xie, H\. Zhao, T\. Yu, and S\. Li \(2022\)Discovering low\-rank subspaces for language\-agnostic multilingual representations\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 5617–5633\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.379/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.379)Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px5.p1.1)\.
- F\. Zhang and N\. Nanda \(2024\)Towards best practices of activation patching in language models: metrics and methods\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Hf17y6u9BC)Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px1.p1.1)\.
- H\. Zhang and J\. Zhou \(2025\)Unraveling lora interference: orthogonal subspaces for robust model merging\.arXiv preprint arXiv:2505\.22934\.Cited by:[§D\.1](https://arxiv.org/html/2607.20469#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px5.p1.1)\.
- Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett, Z\. Wang, and B\. Chen \(2023\)H2O: heavy\-hitter oracle for efficient generative inference of large language models\.InProceedings of the 37th International Conference on Neural Information Processing Systems,NIPS ’23,Red Hook, NY, USA\.Cited by:[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px2.p1.1)\.
- A\. Zou, L\. Phan, S\. L\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. Hendrycks \(2023\)Representation engineering: A top\-down approach to AI transparency\.CoRRabs/2310\.01405\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.01405),[Document](https://dx.doi.org/10.48550/ARXIV.2310.01405),2310\.01405Cited by:[§1](https://arxiv.org/html/2607.20469#S1.p2.1),[§1](https://arxiv.org/html/2607.20469#S1.p3.1),[§1](https://arxiv.org/html/2607.20469#S1.p4.1),[§5](https://arxiv.org/html/2607.20469#S5.SS0.SSS0.Px3.p1.1)\.

Table 8:Benchmark taxonomy used in our experiments\. For discrete\-choice tasks, we use decision\-level forced\-choice accuracy \(FC\-Acc; teacher\-forced conditional logprob\) as the primary readout; we also report generation\-based scores \(gen\-acc/EM\) with collapse diagnostics \(extraction success rate, EOS rate, and average decoded length\)\. For open\-answer settings, GSM8K is scored by EM after extraction \(gen\-math\) and HumanEval uses a compile\-based code\-generation metric; both also include pair\-logprob as a decision proxy where reported\.CategoryBenchmarkI/O formatMetrics reported in paperArithmeticGSM8Kopen \(numeric\)EM \(gen\-math\) \+ Extr/EOS/Len; pair\-logprobArithmeticAQuAMCQFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)Commonsense QACommonsenseQA \(CSQA\)MCQFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)Physical reasoningPIQA2\-way MCQFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)Knowledge QAOpenBookQA \(OBQA\)MCQFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)Knowledge QAQASCMCQFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)VerificationBoolQbinaryFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)VerificationStrategyQAbinaryFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)Logical/QAARC\-Challenge \(ARC\-C\)MCQFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)Logical/QALogiQAMCQFC\-Acc; gen\-acc \(\+ Extr/EOS/Len\)Coding \(aux\)HumanEvalopen \(code\)gen\-code\-compile; pair\-logprobNLU \(aux\)SST\-2binarysteering utility metrics \(margins\); FC readout when usedNLU \(aux\)RTEbinarysteering utility metrics \(margins\); FC readout when usedStyle \(aux\)Pirate lexicon setopen \(style\)pirate\-hit success / intensity#### Additional Definitions\.

Letfθf\_\{\\theta\}be a decoder\-only Transformer withLLlayers and hidden widthdd\. Under KV caching, each step is a cached decode forward pass with sequence length=1=1\. Fix a layerℓ\\elland lethℓ\(s\)∈ℝdh\_\{\\ell\}^\{\(s\)\}\\in\\mathbb\{R\}^\{d\}be the layer\-ℓ\\ellhidden state of the \(only\) token during cached decode stepss\. We callhℓ\(s\)h\_\{\\ell\}^\{\(s\)\}the*decode\-time decision state*, i\.e\., the layer\-ℓ\\ellhidden state of the current token at cached decode stepss\. Unless stated otherwise, we sampleS∼Unif​\{1,…,K\}S\\sim\\mathrm\{Unif\}\\\{1,\\ldots,K\\\}and usehℓ\(S\)h\_\{\\ell\}^\{\(S\)\}\. For each taskt∈𝒯t\\in\\mathcal\{T\}, letDprefill​\(t,ℓ\)D\_\{\\text\{prefill\}\}\(t,\\ell\)be the distribution of layer\-ℓ\\ellhidden states during the prefill pass, and letDdecode​\(t,ℓ;π,K\)D\_\{\\text\{decode\}\}\(t,\\ell;\\pi,K\)be the distribution of\{hℓ\(s\)\}s=1K\\\{h\_\{\\ell\}^\{\(s\)\}\\\}\_\{s=1\}^\{K\}induced by cached decoding up to horizonKKunder policyπ\\pi\. In all experiments,Dprefill​\(t,ℓ\)D\_\{\\text\{prefill\}\}\(t,\\ell\)is formed from the hidden state at the last prompt positioni=n​\(u\)i=n\(u\)to align with the sequence\-length\-one decode states\.

#### Definition of Patchback Experiments\.

For a discrete\-choice task, define the flip set at layerℓ\\ellas

ℱℓ=\{x:y^base​\(x\)=y​\(x\)∧y^abl​\(x\)≠y​\(x\)\}\.\\mathcal\{F\}\_\{\\ell\}=\\left\\\{x:\\ \\hat\{y\}\_\{\\mathrm\{base\}\}\(x\)=y\(x\)\\ \\wedge\\ \\hat\{y\}\_\{\\mathrm\{abl\}\}\(x\)\\neq y\(x\)\\right\\\}\.
Let𝒮ℓ\\mathcal\{S\}\_\{\\ell\}be the shared decode\-time subspace at layerℓ\\ellwith orthonormal basisQℓ\(S\)∈ℝd×kSQ^\{\(S\)\}\_\{\\ell\}\\in\\mathbb\{R\}^\{d\\times k\_\{S\}\}, and letΠℓ:=Qℓ\(S\)​\(Qℓ\(S\)\)⊤\\Pi\_\{\\ell\}\\;:=\\;Q^\{\(S\)\}\_\{\\ell\}\\bigl\(Q^\{\(S\)\}\_\{\\ell\}\\bigr\)^\{\\top\}denote the orthogonal projector onto𝒮ℓ\\mathcal\{S\}\_\{\\ell\}\. For eachx∈ℱℓx\\in\\mathcal\{F\}\_\{\\ell\}and decode stepss, record the baseline shared componentps​\(x\)=Πℓ​hℓ,base\(s\)​\(x\)p\_\{s\}\(x\)\\;=\\;\\Pi\_\{\\ell\}\\,h^\{\(s\)\}\_\{\\ell,\\mathrm\{base\}\}\(x\)\.

We then re\-run the ablated model and patch back only the shared component over a decode\-step windowW⊆\{1,…,K\}W\\subseteq\\\{1,\\ldots,K\\\}by setting

hℓ,patch\(s\)​\(x\)=hℓ,abl\(s\)​\(x\)\+𝟏​\[s∈W\]​\(ps​\(x\)−Πℓ​hℓ,abl\(s\)​\(x\)\)\.h^\{\(s\)\}\_\{\\ell,\\mathrm\{patch\}\}\(x\)=h^\{\(s\)\}\_\{\\ell,\\mathrm\{abl\}\}\(x\)\+\\mathbf\{1\}\[s\\in W\]\\Bigl\(p\_\{s\}\(x\)\-\\Pi\_\{\\ell\}h^\{\(s\)\}\_\{\\ell,\\mathrm\{abl\}\}\(x\)\\Bigr\)\.Equivalently, fors∈Ws\\in Wwe replace the shared componentΠℓ​hℓ,abl\(s\)​\(x\)\\Pi\_\{\\ell\}h^\{\(s\)\}\_\{\\ell,\\mathrm\{abl\}\}\(x\)with the donor componentps​\(x\)p\_\{s\}\(x\), while keeping the orthogonal complement\(I−Πℓ\)​hℓ,abl\(s\)​\(x\)\(I\-\\Pi\_\{\\ell\}\)h^\{\(s\)\}\_\{\\ell,\\mathrm\{abl\}\}\(x\)unchanged\.

We report the rescue rate on flips:

Rescue​\(ℱℓ\)=1\|ℱℓ\|​∑x∈ℱℓ𝟏​\[y^patch​\(x\)=y​\(x\)\],\\mathrm\{Rescue\}\(\\mathcal\{F\}\_\{\\ell\}\)=\\frac\{1\}\{\|\\mathcal\{F\}\_\{\\ell\}\|\}\\sum\_\{x\\in\\mathcal\{F\}\_\{\\ell\}\}\\mathbf\{1\}\\\!\\left\[\\hat\{y\}\_\{\\mathrm\{patch\}\}\(x\)=y\(x\)\\right\],and aggregate across tasks by summing rescued/total counts\.

## Appendix AAdditional Controls and Robustness Experiments

Appendix scope\.This section reports controls and robustness checks for three confounds: \(i\)*distribution alignment*\(all estimation, interventions, and evaluation are aligned to KV\-cached, seq\-len\-11decision states rather than prefill\), \(ii\)*energy confounds*\(we include energy diagnostics and energy\-matched non\-shared controls to rule out “stronger perturbation” explanations\), and \(iii\)*decision\-level evaluation*\(we use forced\-choice log\-probability scoring to isolate decision degradation from EOS/format/extraction failures\)\. Concretely, §[A\.1](https://arxiv.org/html/2607.20469#A1.SS1)analyzes removed\-energy statistics, §[A\.2](https://arxiv.org/html/2607.20469#A1.SS2)presents two complementary energy\-matched causal controls \(k\-match andα\\alpha\-scaling mean\-match\) under strict decode\-only interventions, and §[B\.4](https://arxiv.org/html/2607.20469#A2.SS4)reports the prefill–decode intervention grid supporting decode\-specificity \(H3\)\. All appendix experiments intervene at the same decode\-only boundary: for each prompt we prefill to build the KV cache, then run a seq\-len\-11cached decode call at the prompt boundary, collect the layer\-ℓ\\elldecision state, and apply the intervention at this locus, which matches both the hook site and cache\-advanced forced\-choice evaluation\.

### A\.1Experiment Group 1: Why High Shared\-Space Energy is Expected

Motivation \(energy confound as a diagnosis problem\)\.A natural concern about decode\-time subspace removal is an*energy confound*: if the identified shared subspace carries much higher projection energy than a same\-dimensional baseline, then removing it can appear disproportionately destructive simply because it removes more signal under \([2](https://arxiv.org/html/2607.20469#A1.E2)\)\. Before making any causal claim, we therefore ask a prior diagnostic question:

Question:Why is high decode\-time energy in the estimated shared subspace expected in the first place?

Our answer is: high shared\-space energy is not, by itself, evidence of an estimator artifact \(e\.g\., “PCA artifacts”\)\. It is an expected consequence of \(i\) anisotropic decode\-time activations, and \(ii\) the way “shared” directions are defined in our protocol \(directions that consistently explain non\-trivial variance across tasks\)\. Group 2 then addresses the causal question using energy\-matched controls\.

Setup and diagnostics\.We intervene on KV\-cached decode last\-token stateshℓh\_\{\\ell\}via projection removal

h~=h−α​Q​Q⊤​h,\\tilde\{h\}\\;=\\;h\-\\alpha QQ^\{\\top\}h,\(2\)whereQ∈ℝd×kQ\\in\\mathbb\{R\}^\{d\\times k\}has orthonormal columns andα≥0\\alpha\\geq 0\. For any orthonormal basisQQand statehh, define projection energy and its per\-sample fraction

E​\(h;Q\):=‖Q⊤​h‖22,r​\(h;Q\):=‖Q⊤​h‖22‖h‖22∈\[0,1\]\.E\(h;Q\)\\;:=\\;\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\},\\qquad r\(h;Q\)\\;:=\\;\\frac\{\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\}\{\\\|h\\\|\_\{2\}^\{2\}\}\\in\[0,1\]\.\(3\)Empirically we report𝔼​\[r​\(h;Q\)\]\\mathbb\{E\}\[r\(h;Q\)\]on the prompt\-boundary decode\-last distribution\. For theory it is also convenient to use the ratio of expectations

r¯​\(Q\):=𝔼​\[‖Q⊤​h‖22\]𝔼​\[‖h‖22\]\.\\bar\{r\}\(Q\)\\;:=\\;\\frac\{\\mathbb\{E\}\[\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\]\}\{\\mathbb\{E\}\[\\\|h\\\|\_\{2\}^\{2\}\]\}\.\(4\)These coincide for Haar\-random subspaces; see Lemma[A\.1](https://arxiv.org/html/2607.20469#A1.Thmtheorem1)\. We report𝔼​\[r​\(h;Q\)\]\\mathbb\{E\}\[r\(h;Q\)\]in plots because it is an interpretable per\-example fraction, but the energy matching controls use the numerator𝔼​\[‖Q⊤​h‖22\]\\mathbb\{E\}\[\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\], since interventions scale with that quantity\.

Why high shared\-space energy is expected\.LetΣ:=𝔼​\[h​h⊤\]\\Sigma:=\\mathbb\{E\}\[hh^\{\\top\}\]be the second moment \(with the same centering convention as in §[2\.3](https://arxiv.org/html/2607.20469#S2.SS3)when applicable\)\. Then for any orthonormalQ∈ℝd×kQ\\in\\mathbb\{R\}^\{d\\times k\},

𝔼​\[E​\(h;Q\)\]=𝔼​\[‖Q⊤​h‖22\]=tr​\(Q⊤​Σ​Q\)\.\\mathbb\{E\}\[E\(h;Q\)\]\\;=\\;\\mathbb\{E\}\[\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\]\\;=\\;\\mathrm\{tr\}\(Q^\{\\top\}\\Sigma Q\)\.\(5\)Thus high\-energy subspaces are exactly those aligned with large\-eigenvalue directions ofΣ\\Sigma\. The following three results formalize the expected behavior: \(1\) the diffuse baseline, \(2\) anisotropy guarantees the existence of intrinsically high\-energy low\-dimensional subspaces \(and PCA recovers one\), and \(3\) our*shared*selection criterion further biases toward above\-diffuse energy*within the pooled span*\.

![Refer to caption](https://arxiv.org/html/2607.20469v1/x10.png)\(a\)Layer 10
![Refer to caption](https://arxiv.org/html/2607.20469v1/x11.png)\(b\)Layer 30

Figure 8:Pooled PCA spectrum with shared components marked\.Explained\-variance ratios \(log scale\) for pooled PCA components at two layers\. Components belonging to the identified shared set are visually distinguished, indicating where the shared directions sit in the overall variance spectrum\.###### Lemma A\.1\(Haar\-random subspaces have exactlyk/dk/dexpected energy fraction\)\.

LetQrandQ\_\{\\mathrm\{rand\}\}be Haar\-uniform on the Grassmannian \(a randomkk\-dimensional subspace\), independent ofh∈ℝdh\\in\\mathbb\{R\}^\{d\}\. Then

𝔼Qrand​\[r​\(h;Qrand\)∣h\]=kd,and hence​𝔼​\[r​\(h;Qrand\)\]=kd\.\\mathbb\{E\}\_\{Q\_\{\\mathrm\{rand\}\}\}\\\!\\big\[r\(h;Q\_\{\\mathrm\{rand\}\}\)\\mid h\\big\]=\\frac\{k\}\{d\},\\qquad\\text\{and hence\}\\;\\mathbb\{E\}\[r\(h;Q\_\{\\mathrm\{rand\}\}\)\]=\\frac\{k\}\{d\}\.Moreover,r¯​\(Qrand\)=k/d\\bar\{r\}\(Q\_\{\\mathrm\{rand\}\}\)=k/das well\.

###### Proof\.

Rotational symmetry gives𝔼​\[Qrand​Qrand⊤\]=kd​I\\mathbb\{E\}\[Q\_\{\\mathrm\{rand\}\}Q\_\{\\mathrm\{rand\}\}^\{\\top\}\]=\\tfrac\{k\}\{d\}I\. Thus𝔼​\[‖Qrand⊤​h‖22∣h\]=h⊤​𝔼​\[Qrand​Qrand⊤\]​h=kd​‖h‖22\\mathbb\{E\}\[\\\|Q\_\{\\mathrm\{rand\}\}^\{\\top\}h\\\|\_\{2\}^\{2\}\\mid h\]=h^\{\\top\}\\mathbb\{E\}\[Q\_\{\\mathrm\{rand\}\}Q\_\{\\mathrm\{rand\}\}^\{\\top\}\]h=\\tfrac\{k\}\{d\}\\\|h\\\|\_\{2\}^\{2\}, so𝔼​\[r​\(h;Qrand\)∣h\]=k/d\\mathbb\{E\}\[r\(h;Q\_\{\\mathrm\{rand\}\}\)\\mid h\]=k/d\. Taking expectation overhhyields𝔼​\[r​\(h;Qrand\)\]=k/d\\mathbb\{E\}\[r\(h;Q\_\{\\mathrm\{rand\}\}\)\]=k/d\. Finally,r¯​\(Qrand\)=𝔼​\[‖Qrand⊤​h‖22\]/𝔼​\[‖h‖22\]=k/d\\bar\{r\}\(Q\_\{\\mathrm\{rand\}\}\)=\\mathbb\{E\}\[\\\|Q\_\{\\mathrm\{rand\}\}^\{\\top\}h\\\|\_\{2\}^\{2\}\]/\\mathbb\{E\}\[\\\|h\\\|\_\{2\}^\{2\}\]=k/d\. ∎

###### Theorem A\.2\(Top\-variance subspaces exceed random energy under anisotropy\)\.

Leth∈ℝdh\\in\\mathbb\{R\}^\{d\}be zero\-mean with second momentΣ=𝔼​\[h​h⊤\]⪰0\\Sigma=\\mathbb\{E\}\[hh^\{\\top\}\]\\succeq 0and eigenvaluesλ1≥⋯≥λd\\lambda\_\{1\}\\geq\\cdots\\geq\\lambda\_\{d\}\. For any orthonormalQ∈ℝd×kQ\\in\\mathbb\{R\}^\{d\\times k\},

𝔼​\[‖Q⊤​h‖22\]=tr​\(Q⊤​Σ​Q\)\.\\mathbb\{E\}\\big\[\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\\big\]\\;=\\;\\mathrm\{tr\}\(Q^\{\\top\}\\Sigma Q\)\.LetQtopQ\_\{\\mathrm\{top\}\}span the top\-kkeigenspace ofΣ\\Sigma\. Then

𝔼​\[‖Qtop⊤​h‖22\]=∑i=1kλi≥kd​∑i=1dλi=kd​𝔼​\[‖h‖22\],\\mathbb\{E\}\\big\[\\\|Q\_\{\\mathrm\{top\}\}^\{\\top\}h\\\|\_\{2\}^\{2\}\\big\]=\\sum\_\{i=1\}^\{k\}\\lambda\_\{i\}\\;\\geq\\;\\frac\{k\}\{d\}\\sum\_\{i=1\}^\{d\}\\lambda\_\{i\}=\\frac\{k\}\{d\}\\,\\mathbb\{E\}\[\\\|h\\\|\_\{2\}^\{2\}\],with equality iffλ1=⋯=λd\\lambda\_\{1\}=\\cdots=\\lambda\_\{d\}\(isotropy\)\. Equivalently,r¯​\(Qtop\)≥k/d\\bar\{r\}\(Q\_\{\\mathrm\{top\}\}\)\\geq k/d, strictly ifΣ\\Sigmais anisotropic\.

###### Proof\.

The identity is \([5](https://arxiv.org/html/2607.20469#A1.E5)\)\.

Ky Fan’s maximum principle yieldsmaxQ⊤​Q=Ik⁡tr​\(Q⊤​Σ​Q\)=∑i=1kλi\\max\_\{Q^\{\\top\}Q=I\_\{k\}\}\\mathrm\{tr\}\(Q^\{\\top\}\\Sigma Q\)=\\sum\_\{i=1\}^\{k\}\\lambda\_\{i\}\. Since the mean of the top\-kkeigenvalues is at least the overall mean,1k​∑i=1kλi≥1d​∑i=1dλi\\frac\{1\}\{k\}\\sum\_\{i=1\}^\{k\}\\lambda\_\{i\}\\geq\\frac\{1\}\{d\}\\sum\_\{i=1\}^\{d\}\\lambda\_\{i\}, with equality iff allλi\\lambda\_\{i\}are equal\. ∎

A concrete formalization of “low\-rank but high\-energy” is the factor\-plus\-noise modelh=U​z\+εh=Uz\+\\varepsilonwithU⊤​U=IrU^\{\\top\}U=I\_\{r\},Cov⁡\(z\)=Λ⪰0\\operatorname\{Cov\}\(z\)=\\Lambda\\succeq 0,Cov⁡\(ε\)=σ2​I\\operatorname\{Cov\}\(\\varepsilon\)=\\sigma^\{2\}I, for whichΣ=U​Λ​U⊤\+σ2​I\\Sigma=U\\Lambda U^\{\\top\}\+\\sigma^\{2\}Iandr¯​\(U\)=tr​\(Λ\)\+r​σ2tr​\(Λ\)\+d​σ2\.\\bar\{r\}\(U\)\\;=\\;\\frac\{\\mathrm\{tr\}\(\\Lambda\)\+r\\sigma^\{2\}\}\{\\mathrm\{tr\}\(\\Lambda\)\+d\\sigma^\{2\}\}\.Thus a smallrrcan still capture a large energy fraction whentr​\(Λ\)≫d​σ2\\mathrm\{tr\}\(\\Lambda\)\\gg d\\sigma^\{2\}\.

###### Lemma A\.3\(Shareness threshold implies above\-diffuse energy*within*the pooled PCA span\)\.

Fix a layerℓ\\elland the pooled PCA basisQℓ=\[qℓ,1,…,qℓ,k\]∈ℝd×kQ\_\{\\ell\}=\[q\_\{\\ell,1\},\\dots,q\_\{\\ell,k\}\]\\in\\mathbb\{R\}^\{d\\times k\}from §3\.3\. For each taskt∈𝒯t\\in\\mathcal\{T\}, leth∼Ddecode​\(t,ℓ\)h\\sim D\_\{\\mathrm\{decode\}\}\(t,\\ell\)denote a task\-centered decode\-last state \(as in §3\.3\), and define

vℓ,t,i:=Var⁡\(qℓ,i⊤​h\)=𝔼​\[\(qℓ,i⊤​h\)2\],Vℓ,t:=∑j=1kvℓ,t,j=𝔼​\[‖Qℓ⊤​h‖22\]\.v\_\{\\ell,t,i\}\\;:=\\;\\operatorname\{Var\}\\\!\\big\(q\_\{\\ell,i\}^\{\\top\}h\\big\)\\;=\\;\\mathbb\{E\}\\big\[\(q\_\{\\ell,i\}^\{\\top\}h\)^\{2\}\\big\],\\qquad V\_\{\\ell,t\}\\;:=\\;\\sum\_\{j=1\}^\{k\}v\_\{\\ell,t,j\}\\;=\\;\\mathbb\{E\}\\\!\\left\[\\\|Q\_\{\\ell\}^\{\\top\}h\\\|\_\{2\}^\{2\}\\right\]\.Define the within\-span variance sharerℓ,t,i:=vℓ,t,iVℓ,t∈\[0,1\]r\_\{\\ell,t,i\}\\;:=\\;\\frac\{v\_\{\\ell,t,i\}\}\{V\_\{\\ell,t\}\}\\in\[0,1\], so that∑i=1krℓ,t,i=1\\sum\_\{i=1\}^\{k\}r\_\{\\ell,t,i\}=1\. LetSℓ​\(τ,m\)S\_\{\\ell\}\(\\tau,m\)be the set of indicesiisuch thatrℓ,t,i≥τr\_\{\\ell,t,i\}\\geq\\tauholds for at leastmmtasks, and letQℓ\(S\)=Qℓ​\[:,Sℓ​\(τ,m\)\]Q\_\{\\ell\}^\{\(S\)\}=Q\_\{\\ell\}\[:,S\_\{\\ell\}\(\\tau,m\)\]\.

\(i\)For every tasktt,

𝔼​\[‖\(Qℓ\(S\)\)⊤​h‖22\]𝔼​\[‖Qℓ⊤​h‖22\]=∑i∈Sℓ​\(τ,m\)rℓ,t,i\.\\frac\{\\mathbb\{E\}\\\!\\left\[\\\|\(Q\_\{\\ell\}^\{\(S\)\}\)^\{\\top\}h\\\|\_\{2\}^\{2\}\\right\]\}\{\\mathbb\{E\}\\\!\\left\[\\\|Q\_\{\\ell\}^\{\\top\}h\\\|\_\{2\}^\{2\}\\right\]\}\\;=\\;\\sum\_\{i\\in S\_\{\\ell\}\(\\tau,m\)\}r\_\{\\ell,t,i\}\.\(ii\)Averaging over tasks,

1\|𝒯\|​∑t∈𝒯∑i∈Sℓ​\(τ,m\)rℓ,t,i≥m​τ\|𝒯\|​\|Sℓ​\(τ,m\)\|\.\\frac\{1\}\{\|\\mathcal\{T\}\|\}\\sum\_\{t\\in\\mathcal\{T\}\}\\sum\_\{i\\in S\_\{\\ell\}\(\\tau,m\)\}r\_\{\\ell,t,i\}\\;\\geq\\;\\frac\{m\\tau\}\{\|\\mathcal\{T\}\|\}\\,\|S\_\{\\ell\}\(\\tau,m\)\|\.\(Consequently\|Sℓ​\(τ,m\)\|≤\|𝒯\|/\(m​τ\)\|S\_\{\\ell\}\(\\tau,m\)\|\\leq\|\\mathcal\{T\}\|/\(m\\tau\)\.\)

\(iii\)If one selects a uniformly random index subsetSrand⊆\[k\]S\_\{\\mathrm\{rand\}\}\\subseteq\[k\]with\|Srand\|=\|Sℓ\|\|S\_\{\\mathrm\{rand\}\}\|=\|S\_\{\\ell\}\|, then for every tasktt,

𝔼Srand​\[∑i∈Srandrℓ,t,i\]=\|Sℓ\|k\.\\mathbb\{E\}\_\{S\_\{\\mathrm\{rand\}\}\}\\Big\[\\sum\_\{i\\in S\_\{\\mathrm\{rand\}\}\}r\_\{\\ell,t,i\}\\Big\]=\\frac\{\|S\_\{\\ell\}\|\}\{k\}\.Therefore the shared set’s average within\-span energy share is lower\-bounded by a factor\(m​τ​k\|𝒯\|\)\\big\(\\frac\{m\\tau k\}\{\|\\mathcal\{T\}\|\}\\big\)relative to the diffuse in\-span baseline \(clipped at11\)\.

###### Proof\.

\(i\) Orthonormality gives𝔼​\[‖\(Qℓ\(S\)\)⊤​h‖22\]=∑i∈Sℓ𝔼​\[\(qℓ,i⊤​h\)2\]=∑i∈Sℓvℓ,t,i\\mathbb\{E\}\[\\\|\(Q\_\{\\ell\}^\{\(S\)\}\)^\{\\top\}h\\\|\_\{2\}^\{2\}\]=\\sum\_\{i\\in S\_\{\\ell\}\}\\mathbb\{E\}\[\(q\_\{\\ell,i\}^\{\\top\}h\)^\{2\}\]=\\sum\_\{i\\in S\_\{\\ell\}\}v\_\{\\ell,t,i\}and𝔼​\[‖Qℓ⊤​h‖22\]=∑j=1kvℓ,t,j=Vℓ,t\\mathbb\{E\}\[\\\|Q\_\{\\ell\}^\{\\top\}h\\\|\_\{2\}^\{2\}\]=\\sum\_\{j=1\}^\{k\}v\_\{\\ell,t,j\}=V\_\{\\ell,t\}, hence the ratio equals∑i∈Sℓrℓ,t,i\\sum\_\{i\\in S\_\{\\ell\}\}r\_\{\\ell,t,i\}\. \(ii\) For eachi∈Sℓ​\(τ,m\)i\\in S\_\{\\ell\}\(\\tau,m\), there are at leastmmtasks withrℓ,t,i≥τr\_\{\\ell,t,i\}\\geq\\tau, so∑t∈𝒯rℓ,t,i≥m​τ\\sum\_\{t\\in\\mathcal\{T\}\}r\_\{\\ell,t,i\}\\geq m\\tau\. Summing overi∈Sℓi\\in S\_\{\\ell\}and dividing by\|𝒯\|\|\\mathcal\{T\}\|yields \(ii\)\. Also, since∑t∑i∈Sℓrℓ,t,i≤∑t∑i=1krℓ,t,i=\|𝒯\|\\sum\_\{t\}\\sum\_\{i\\in S\_\{\\ell\}\}r\_\{\\ell,t,i\}\\leq\\sum\_\{t\}\\sum\_\{i=1\}^\{k\}r\_\{\\ell,t,i\}=\|\\mathcal\{T\}\|, we must have\|Sℓ\|≤\|𝒯\|/\(m​τ\)\|S\_\{\\ell\}\|\\leq\|\\mathcal\{T\}\|/\(m\\tau\)\. \(iii\) Since∑i=1krℓ,t,i=1\\sum\_\{i=1\}^\{k\}r\_\{\\ell,t,i\}=1, the expected sum over a uniformly random size\-\|Sℓ\|\|S\_\{\\ell\}\|subset is\|Sℓ\|/k\|S\_\{\\ell\}\|/k\. ∎

Why Group\-1 diagnostics alone do not resolve causality\.Group 1 explains why large decode\-time projection energy inQsharedQ\_\{\\mathrm\{shared\}\}is*expected*: anisotropy implies intrinsically high\-energy subspaces \(Theorem[A\.2](https://arxiv.org/html/2607.20469#A1.Thmtheorem2)\), and our “shareness” selection criterion further focuses the analysis on directions with measurable overlap with the decode\-time shared basis, enabling a cleaner test of shared\-subspace interference \(Lemma[A\.3](https://arxiv.org/html/2607.20469#A1.Thmtheorem3)\)\. However, this diagnosis alone does not resolve causality: a skeptic can still argue that shared ablation hurts more simply because it removes more energy\. Group 2 therefore uses energy\-matched controls to isolate causal effects from energy removal\.

![Refer to caption](https://arxiv.org/html/2607.20469v1/x12.png)\(a\)Task\-aggregateΔ\\Deltaaccuracy vs\.α\\alpha\.Shared removal diverges sharply forα≳0\.75\\alpha\\gtrsim 0\.75while energy\-matched controls stay closer to baseline\.
![Refer to caption](https://arxiv.org/html/2607.20469v1/x13.png)\(b\)Per\-taskΔ\\Deltaaccuracy atα=1\\alpha=1\.Shared is consistently most harmful; k\-match remains near baseline\.

Figure 9:Energy\-matched causal controls under strict decode\-only interventions \(Llama\-2\-7B\-Chat, layer 10\)\.We compare removing the shared subspace against two energy\-matched controls \(fixed\-dimensionα\\alpha\-match and expanded\-dimension k\-match\), evaluated by cache\-advanced forced\-choice logprob scoring on KV\-cached decode steps \(seq len=1=1\)\.![Refer to caption](https://arxiv.org/html/2607.20469v1/x14.png)Figure 10:Per\-taskα\\alphasweep \(decode\-only; energy\-matched controls\)\.Panels are ordered left\-to\-right, top\-to\-bottom as: AQuA, ARC\-Challenge, BoolQ, CommonsenseQA, LogiQA, OpenBookQA, QASC, StrategyQA\. Dashed line: baseline; shaded regions: 95% CIs\.Table 9:Forced\-choice accuracy \(%\)\. Each cell reportsShared / Ctrl\-α\\alpha\-match / Ctrl\-kk\-match\. \(kkforkk\-match is shown in the column header;kk\-match usesα=1\\alpha\{=\}1\.\)TaskNNBaseα\\alpha\(andkkforkk\-match\)0\(k=126k\{=\}126\)0\.250\.25\(k=126k\{=\}126\)0\.50\.5\(k=197k\{=\}197\)0\.750\.75\(k=649k\{=\}649\)commonsenseqa122151\.751\.7 / 51\.7 / 51\.452\.0 / 51\.4 / 51\.450\.2 / 51\.4 / 50\.538\.6 / 51\.1 / 51\.8strategyqa68755\.555\.5 / 55\.5 / 54\.055\.6 / 55\.5 / 54\.054\.7 / 53\.7 / 53\.452\.4 / 50\.8 / 52\.4aqua25417\.717\.7 / 17\.7 / 18\.116\.9 / 18\.1 / 18\.116\.9 / 18\.9 / 19\.316\.1 / 17\.7 / 18\.5arc\_challenge117247\.447\.4 / 47\.4 / 46\.046\.7 / 46\.2 / 46\.043\.2 / 45\.4 / 46\.136\.9 / 43\.9 / 47\.9openbookqa50051\.251\.2 / 51\.2 / 49\.451\.2 / 50\.2 / 49\.451\.6 / 49\.0 / 48\.636\.6 / 47\.2 / 49\.0qasc92643\.443\.4 / 43\.4 / 42\.843\.6 / 43\.0 / 42\.841\.6 / 42\.5 / 42\.033\.0 / 41\.7 / 43\.8logiqa65130\.730\.7 / 30\.7 / 30\.330\.7 / 30\.1 / 30\.327\.2 / 30\.0 / 28\.026\.9 / 30\.3 / 30\.6boolq204861\.461\.4 / 61\.4 / 59\.063\.5 / 60\.4 / 59\.065\.7 / 58\.4 / 59\.958\.3 / 56\.6 / 59\.7
TaskNNBaseα\\alpha\(andkkforkk\-match\)1\.01\.0\(k=1843k\{=\}1843\)1\.251\.25\(k=2705k\{=\}2705\)1\.51\.5\(k=2705k\{=\}2705\)2\.02\.0\(k=2705k\{=\}2705\)commonsenseqa122151\.721\.7 / 47\.8 / 54\.020\.8 / 31\.4 / 50\.921\.0 / 24\.0 / 50\.920\.5 / 30\.1 / 50\.9strategyqa68755\.554\.9 / 49\.2 / 51\.552\.7 / 49\.2 / 55\.652\.7 / 47\.7 / 55\.646\.1 / 51\.4 / 55\.6aqua25417\.716\.1 / 18\.5 / 16\.517\.7 / 18\.9 / 18\.916\.9 / 18\.5 / 18\.920\.5 / 19\.3 / 18\.9arc\_challenge117247\.427\.0 / 41\.0 / 46\.527\.2 / 36\.9 / 44\.926\.8 / 28\.8 / 44\.922\.7 / 27\.3 / 44\.9openbookqa50051\.227\.2 / 45\.4 / 48\.027\.4 / 40\.2 / 46\.227\.8 / 26\.6 / 46\.225\.2 / 27\.2 / 46\.2qasc92643\.412\.6 / 40\.6 / 44\.912\.5 / 32\.4 / 40\.012\.5 / 21\.2 / 40\.012\.6 / 15\.1 / 40\.0logiqa65130\.726\.3 / 29\.5 / 29\.625\.2 / 25\.2 / 28\.925\.3 / 24\.6 / 28\.926\.4 / 24\.7 / 28\.9boolq204861\.449\.5 / 54\.9 / 59\.648\.2 / 54\.7 / 57\.848\.6 / 55\.9 / 57\.849\.3 / 50\.4 / 57\.8

### A\.2Experiment Group 2: Energy\-matched causal controls under strict decode\-only interventions

#### Motivation\.

A key empirical fact from Group 1 is that the decode\-time*shared*subspace is*intrinsically high\-energy*: its projection captures a disproportionate share of the activation norm \(larger𝔼​\[‖Q⊤​h‖22\]\\mathbb\{E\}\[\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\]\)\. This is informative, but not yet causal: a high\-energy subspace can*mechanically*amplify any decode\-only intervention, so an apparent decision\-level effect could arise simply because we removed “more signal,” not because we removed the*right*signal\.

Concretely, under our strict decode\-only removal operator \(Eq\. \([2](https://arxiv.org/html/2607.20469#A1.E2)\)\), withδ​\(h;Q,α\):=h~−h=−α​Q​Q⊤​h\\delta\(h;Q,\\alpha\)\\;:=\\;\\tilde\{h\}\-h\\;=\\;\-\\alpha QQ^\{\\top\}h, the perturbation energy satisfies

‖δ​\(h;Q,α\)‖22=α2​‖Q​Q⊤​h‖22=α2​h⊤​\(Q​Q⊤\)⊤​\(Q​Q⊤\)​h\.\\\|\\delta\(h;Q,\\alpha\)\\\|\_\{2\}^\{2\}=\\alpha^\{2\}\\\|QQ^\{\\top\}h\\\|\_\{2\}^\{2\}=\\alpha^\{2\}h^\{\\top\}\(QQ^\{\\top\}\)^\{\\top\}\(QQ^\{\\top\}\)h\.SinceQ⊤​Q=IkQ^\{\\top\}Q=I\_\{k\}, we have\(Q​Q⊤\)⊤=Q​Q⊤\(QQ^\{\\top\}\)^\{\\top\}=QQ^\{\\top\}and\(Q​Q⊤\)2=Q​Q⊤\(QQ^\{\\top\}\)^\{2\}=QQ^\{\\top\}, so

‖Q​Q⊤​h‖22=h⊤​Q​Q⊤​h=\(Q⊤​h\)⊤​\(Q⊤​h\)=‖Q⊤​h‖22\.\\\|QQ^\{\\top\}h\\\|\_\{2\}^\{2\}=h^\{\\top\}QQ^\{\\top\}h=\(Q^\{\\top\}h\)^\{\\top\}\(Q^\{\\top\}h\)=\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\.Therefore,

‖δ​\(h;Q,α\)‖22=α2​‖Q⊤​h‖22\.\\\|\\delta\(h;Q,\\alpha\)\\\|\_\{2\}^\{2\}\\;=\\;\\alpha^\{2\}\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\.In particular, even when the dimensionalitykQk\_\{Q\}is fixed, a higher\-energy subspace \(larger𝔼​\[‖Q⊤​h‖22\]\\mathbb\{E\}\[\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\]\) induces a larger expected perturbation at the sameα\\alpha\. This creates a natural skeptic’s alternative to H2: the shared ablation could look uniquely harmful*only because*it causes a larger\-energy perturbation\.

To test whether decision degradation can be explained by energy alone, we introduce*energy\-matched causal controls*: controls chosen \(and/or scaled\) so that the expected perturbation energy matches that of the shared ablation\. If shared ablations remain more harmful under this match, the residual gap must be attributed to*which directions*are removed \(direction\-specific causal structure\), rather than*how much energy*is removed\.

Question:After matching the expected perturbation energy, do shared ablations remain uniquely harmful?

If performance degradation is driven primarily by perturbation magnitude, then after matching𝔼​\[‖δ​\(h;Q,α\)‖22\]\\mathbb\{E\}\[\\\|\\delta\(h;Q,\\alpha\)\\\|\_\{2\}^\{2\}\]between the shared*removal subspace*and an energy\-matched*control subspace*, the corresponding performance curves should be comparable as a function ofα\\alpha\(up to noise\)\. In particular, at a fixed layer and intervention locus, energy\-matched controls should reproduce any sharp drop that would otherwise be attributed to “shareness\.”

#### Setup\.

We prefill once to build the KV cache, and intervene*only*on cached decode forwards \(sequence length=1=1\), leaving the prefill pass untouched\. Evaluation uses our decision\-level scorer \(forced\-choice; see main text\): we advance the cache under teacher forcing and accumulate conditional log\-probabilities, avoiding confounds from decode\-time output instability \(formatting/EOS, etc\.\)\. As a sanity check, we log hook calls to confirm that prefill forwards \(seq len\>1\>1\) are never modified\.

#### Energy matching target\.

All energy matching is computed on the*same decode\-last calibration states*used in the causal pipeline, so the matched quantity corresponds to the actual intervention locus\. Specifically, for a basisQQwe define its mean projection energy on calibration states as

ℰ​\(Q\)≜𝔼​\[‖Q⊤​h‖22\],\\mathcal\{E\}\(Q\)\\;\\triangleq\\;\\mathbb\{E\}\\big\[\\\|Q^\{\\top\}h\\\|\_\{2\}^\{2\}\\big\],so the mean perturbation energy at strengthα\\alphaisα2​ℰ​\(Q\)\\alpha^\{2\}\\mathcal\{E\}\(Q\)\. We then compareQSQ\_\{S\}, set of basis that span the shared subspace, against two complementary controls that match this quantity\. We construct controls from a*non\-shared pool of basis*QpoolQ\_\{\\mathrm\{pool\}\}\(directions explicitly outside the identified shared set\)\. Intuitively, these are nearby directions in the same representation/activation space but*not*labeled shared by our criterion\. We then match the perturbation energy in two different ways:

- •α\\alpha\-match \(Control\-1, fixed dimension\):keep dimension fixed \(typicallykC=kSk\_\{C\}=k\_\{S\}\), but rescale the control strengthαC\\alpha\_\{C\}so that the*mean perturbation energy*matches: αC2​𝔼​\[‖QC⊤​h‖22\]=αS2​𝔼​\[‖QS⊤​h‖22\]⇒αC=αS​𝔼​\[‖QS⊤​h‖22\]𝔼​\[‖QC⊤​h‖22\]\.\\alpha\_\{C\}^\{2\}\\mathbb\{E\}\[\\\|Q\_\{C\}^\{\\top\}h\\\|\_\{2\}^\{2\}\]\\;=\\;\\alpha\_\{S\}^\{2\}\\mathbb\{E\}\[\\\|Q\_\{S\}^\{\\top\}h\\\|\_\{2\}^\{2\}\]\\quad\\Rightarrow\\quad\\alpha\_\{C\}\\;=\\;\\alpha\_\{S\}\\sqrt\{\\frac\{\\mathbb\{E\}\[\\\|Q\_\{S\}^\{\\top\}h\\\|\_\{2\}^\{2\}\]\}\{\\mathbb\{E\}\[\\\|Q\_\{C\}^\{\\top\}h\\\|\_\{2\}^\{2\}\]\}\}\.We sweep the shared strengthαS\\alpha\_\{S\}and apply the correspondingα\\alpha\-match scaling\. This directly answers “are we just perturbing more?” while holdingkkfixed\.
- •kk\-match \(Control\-2,α=1\\alpha=1pure\-removal reference\):to avoidαC\>1\\alpha\_\{C\}\>1\(over\-subtraction\), we also match energy with*unit strength*by expanding control dimension\. For each swept shared strengthαS\\alpha\_\{S\}, we setαC=1\\alpha\_\{C\}=1and choose the smallestkCk\_\{C\}such that the control’s mean projection energy matches that of the shared subspace: QC:=Qpool\[:,1:kC\]withαS2𝔼\[∥QS⊤h∥22\]≈𝔼\[∥QC⊤h∥22\],Q\_\{C\}:=Q\_\{\\mathrm\{pool\}\}\[:,1:k\_\{C\}\]\\quad\\text\{with\}\\quad\\alpha\_\{S\}^\{2\}\\mathbb\{E\}\\\!\\left\[\\\|Q\_\{S\}^\{\\top\}h\\\|\_\{2\}^\{2\}\\right\]\\approx\\mathbb\{E\}\\\!\\left\[\\\|Q\_\{C\}^\{\\top\}h\\\|\_\{2\}^\{2\}\\right\],and then intervene withα=1\\alpha=1\. In practice this may require largekCk\_\{C\}\(sometimes thousands of directions\), which makes this control conservative: it removes*comparable energy*from a much larger set of explicitly non\-shared directions\.

#### Results: shared directions remain uniquely causal beyond energy\.

Figure[9](https://arxiv.org/html/2607.20469#A1.F9)and Table[9](https://arxiv.org/html/2607.20469#A1.T9)show a consistent signature: for small strengths, all methods remain close to baseline, but onceα\\alphaapproaches the regime where shared removal excises a large fraction of the shared component, shared performance drops sharply while energy\-matched controls stay substantially closer to baseline\. This rejects the energy\-only explanation in the strongest form:*even when the expected perturbation energy is matched on the same decode\-last states, removing shared directions is systematically more damaging than removing non\-shared directions\.*

The separation is visible both in the task\-aggregate curve \(Fig\.[9\(a\)](https://arxiv.org/html/2607.20469#A1.F9.sf1)\) and per\-task snapshots atα=1\\alpha=1\(Fig\.[9\(b\)](https://arxiv.org/html/2607.20469#A1.F9.sf2)\)\. Moreover, the per\-task sweep \(Fig\.[10](https://arxiv.org/html/2607.20469#A1.F10)\) highlights that the effect is direction\-specific and task\-dependent rather than a generic brittleness of decode\-only editing: the largest shared\-only collapses occur on knowledge\- and reasoning\-heavy benchmarks \(QASC/CSQA/OBQA/ARC\-Challenge\), while several tasks remain comparatively stable under matched controls\.

![Refer to caption](https://arxiv.org/html/2607.20469v1/x15.png)\(a\)Layer 0
![Refer to caption](https://arxiv.org/html/2607.20469v1/x16.png)\(b\)Layer 8
![Refer to caption](https://arxiv.org/html/2607.20469v1/x17.png)\(c\)Layer 16
![Refer to caption](https://arxiv.org/html/2607.20469v1/x18.png)\(d\)Layer 30

Figure 11:shareness heatmaps across layers\.Rows are tasks and columns are pooled PCA components\. Color showsrℓ,t,ir\_\{\\ell,t,i\}; columns are sorted by \#tasks withrℓ,t,i≥τr\_\{\\ell,t,i\}\\geq\\tau; vertical line marksSℓ​\(τ,m\)S\_\{\\ell\}\(\\tau,m\)\.![Refer to caption](https://arxiv.org/html/2607.20469v1/x19.png)Figure 12:PCA visualization of component signatures for layer 10\.*Left:*3D projection onto the first three principal components \(PC1–PC3\)\.*Right:*top\-down projection onto PC1–PC2\. Each point corresponds to a component; color indicates the number of tasks whose component score exceeds the thresholdτ\\tau\(colorbar\)\.Algorithm 1Decode\-aligned shared basis estimation at layerℓ\\ell\(DecodeShare\)Require:tasks𝒯\\mathcal\{T\}, calibration prompts\{𝒞t\}\\\{\\mathcal\{C\}\_\{t\}\\\}, decoding policyπ\\pi, horizonKK, PCA ratioρ\\rho, threshold\(τ,m\)\(\\tau,m\) Ensure:joint basisQℓQ\_\{\\ell\}and shared basisQℓ\(S\)Q^\{\(S\)\}\_\{\\ell\}

1. 1\.For eacht∈𝒯t\\in\\mathcal\{T\}, run cached decoding on prompts in𝒞t\\mathcal\{C\}\_\{t\}under policyπ\\piup toKKsteps\.
2. 2\.Collect decode last\-token states\{hℓ\(s\)\}\\\{h\_\{\\ell\}^\{\(s\)\}\\\}whereseq\_len=1\\text\{seq\\\_len\}=1intoXt,ℓ∈ℝnt,ℓ×dX\_\{t,\\ell\}\\in\\mathbb\{R\}^\{n\_\{t,\\ell\}\\times d\}\.
3. 3\.Task\-center:X~t,ℓ←Xt,ℓ−𝟏​μt,ℓ⊤\\tilde\{X\}\_\{t,\\ell\}\\leftarrow X\_\{t,\\ell\}\-\\mathbf\{1\}\\mu\_\{t,\\ell\}^\{\\top\}\.
4. 4\.Balance tonℓ=mint⁡nt,ℓn\_\{\\ell\}=\\min\_\{t\}n\_\{t,\\ell\}by subsampling eachX~t,ℓ\\tilde\{X\}\_\{t,\\ell\}\.
5. 5\.Pool:X~ℓ←concat\(X~t,ℓ:t∈𝒯\)∈ℝ\(\|𝒯\|​nℓ\)×d\\tilde\{X\}\_\{\\ell\}\\leftarrow\\mathrm\{concat\}\(\\tilde\{X\}\_\{t,\\ell\}:t\\in\\mathcal\{T\}\)\\in\\mathbb\{R\}^\{\(\|\\mathcal\{T\}\|\\,n\_\{\\ell\}\)\\times d\}\.
6. 6\.PCA/SVD:X~ℓ=U​Σ​V⊤\\tilde\{X\}\_\{\\ell\}=U\\Sigma V^\{\\top\}; choose smallestkks\.t\. explained variance≥ρ\\geq\\rho; setQℓ←V:,1:kQ\_\{\\ell\}\\leftarrow V\_\{:,1:k\}\.
7. 7\.For eachtt, projectZt,ℓ←X~t,ℓ​QℓZ\_\{t,\\ell\}\\leftarrow\\tilde\{X\}\_\{t,\\ell\}Q\_\{\\ell\}, computert,i←Var⁡\(Zt,ℓ​\[:,i\]\)/∑jVar⁡\(Zt,ℓ​\[:,j\]\)r\_\{t,i\}\\leftarrow\\operatorname\{Var\}\(Z\_\{t,\\ell\}\[:,i\]\)\\big/\\sum\_\{j\}\\operatorname\{Var\}\(Z\_\{t,\\ell\}\[:,j\]\)\.
8. 8\.Shared set:Sℓ​\(τ,m\)←\{i:\|\{t:rt,i≥τ\}\|≥m\}S\_\{\\ell\}\(\\tau,m\)\\leftarrow\\\{i:\|\\\{t:r\_\{t,i\}\\geq\\tau\\\}\|\\geq m\\\}\.
9. 9\.ReturnQℓQ\_\{\\ell\}andQℓ\(S\)←Qℓ​\[:,Sℓ\]Q\_\{\\ell\}^\{\(S\)\}\\leftarrow Q\_\{\\ell\}\[:,S\_\{\\ell\}\]\.

Algorithm 2Decode\-time last\-token subspace removalRequire:basisQℓQ\_\{\\ell\}\(shared or control\), strengthα\\alpha, stage lengthKstageK\_\{\\mathrm\{stage\}\}\(optional\)

1. 1\.For decode steps=1,2,…s=1,2,\\dots: run cached forward pass withseq​\_​len=1\\mathrm\{seq\\\_len\}=1to obtainhℓ\(s\)h\_\{\\ell\}^\{\(s\)\}\.
2. 2\.If full removal:h~ℓ\(s\)←hℓ\(s\)−α​Qℓ​Qℓ⊤​hℓ\(s\)\\tilde\{h\}\_\{\\ell\}^\{\(s\)\}\\leftarrow h\_\{\\ell\}^\{\(s\)\}\-\\alpha Q\_\{\\ell\}Q\_\{\\ell\}^\{\\top\}h\_\{\\ell\}^\{\(s\)\}\.
3. 3\.If staged: apply the same update only when the generated\-token count is<Kstage<K\_\{\\mathrm\{stage\}\}\.
4. 4\.Continue to logits; choose next token \(greedy/sampling\); update cache\.

## Appendix BAdditional Main Results\.

### B\.1Additional Results forH1: Diagnostics for Shared Structure

Table 10:H1 summary: sensitivity toτ\\tau\(left\) and existence test across models \(right\)\.\(a\)Sensitivity of\|Sℓ​\(τ,m\)\|\|S\_\{\\ell\}\(\\tau,m\)\|to PCA retentionρ\\rho\(thuskk\) and thresholdτ\\tau\. Entries show\|S\|\|S\|and%\(\|S\|/k\)\\%\(\|S\|/k\)\.PCAShared set\|Sℓ​\(τ,m\)\|\|S\_\{\\ell\}\(\\tau,m\)\|:\|S\|\|S\|\(% ofkk\)ρ\\rhokkτ=10−4\\tau\{=\}10^\{\-4\}τ=10−3\\tau\{=\}10^\{\-3\}τ=10−2\\tau\{=\}10^\{\-2\}τ​k\\tau kat10−310^\{\-3\}0\.8014901490 \(100\.0%\)144 \(9\.7%\)1 \(0\.07%\)1\.490\.9023402169 \(92\.7%\)123 \(5\.3%\)1 \(0\.04%\)2\.340\.9529892353 \(78\.7%\)109 \(3\.6%\)1 \(0\.03%\)2\.990\.9733322413 \(72\.4%\)100 \(3\.0%\)1 \(0\.03%\)3\.330\.9937692484 \(65\.9%\)89 \(2\.4%\)1 \(0\.03%\)3\.77
\(b\)Sharedness existence test\(layerℓ=10\\ell\{=\}10,τ=10−3\\tau\{=\}10^\{\-3\}; see Methodology\)\.kk: pooled cross\-task PCA dim \(95% var, capped bydd\)\. Shared: \#/% ofdd\.ppermp\_\{\\text\{perm\}\},pscrp\_\{\\text\{scr\}\}: permutation/scramble tests \(see Methodology\)\.ModelkkddShared\(\# / %dd\)ppermp\_\{\\text\{perm\}\}pscrp\_\{\\text\{scr\}\}Qwen2\.5\-7B\-Instruct2614358473 / 2\.04<10−3∗⁣∗∗<10^\{\-3\}\\,\{\}^\{\*\*\*\}0\.0196∗0\.0196^\{\*\}Llama\-2\-7B\-Chat29574096107 / 2\.61<10−3∗⁣∗∗<10^\{\-3\}\\,\{\}^\{\*\*\*\}0\.0196∗0\.0196^\{\*\}Llama\-3\.1\-8B\-Instruct28094096112 / 2\.73<10−3∗⁣∗∗<10^\{\-3\}\\,\{\}^\{\*\*\*\}0\.0196∗0\.0196^\{\*\}Falcon\-7b\-instruct27024544124 / 2\.73<10−3∗⁣∗∗<10^\{\-3\}\\,\{\}^\{\*\*\*\}0\.0196∗0\.0196^\{\*\}

This appendix provides additional diagnostics that complement the main H1 existence test\. Throughout, we use the same notations as in §[2\.3](https://arxiv.org/html/2607.20469#S2.SS3): a*shared set*Sℓ​\(τ,m\)S\_\{\\ell\}\(\\tau,m\)is defined in the pooled PCA coordinate system by thresholding per\-task relative variance contributions, and the*shared core size*isTℓ=\|Sℓ​\(τ,m\)\|T\_\{\\ell\}=\|S\_\{\\ell\}\(\\tau,m\)\|\. These analyses do not repeat the null\-test procedure \(reported in the main paper\); instead, they characterize \(i\) how sharing varies across coarse task groupings and \(ii\) how pooled subspaces behave as we vary the task pool and the inference phase\.

Table 11:Sharedness test at layerℓ=10\\ell=10withτ=10−3\\tau=10^\{\-3\}and 8\-taskmshared=8m\_\{\\mathrm\{shared\}\}=8\.kkis the cross\-task PCA dimension \(95% variance\),ddis the hidden dimension\. Permutation null usesBperm=2000B\_\{\\text\{perm\}\}=2000and scramble null usesBscr=200B\_\{\\text\{scr\}\}=200; p\-values at the floor are reported as upper bounds\.ModelkkddShared\(\# / %dd\)ppermp\_\{\\text\{perm\}\}pscrp\_\{\\text\{scr\}\}Qwen2\.5\-7B\-Instruct2614358473 / 2\.04≤5\.0×10−4∗⁣∗∗\\leq 5\.0\{\\times\}10^\{\-4\}\\,\{\}^\{\*\*\*\}≤5\.0×10−3∗∗\\leq 5\.0\{\\times\}10^\{\-3\}\\,\{\}^\{\*\*\}Llama\-2\-7B\-Chat29894096109 / 2\.66≤5\.0×10−4∗⁣∗∗\\leq 5\.0\{\\times\}10^\{\-4\}\\,\{\}^\{\*\*\*\}≤5\.0×10−3∗∗\\leq 5\.0\{\\times\}10^\{\-3\}\\,\{\}^\{\*\*\}Llama\-3\.1\-8B\-Instruct28094096112 / 2\.73≤5\.0×10−4∗⁣∗∗\\leq 5\.0\{\\times\}10^\{\-4\}\\,\{\}^\{\*\*\*\}≤5\.0×10−3∗∗\\leq 5\.0\{\\times\}10^\{\-3\}\\,\{\}^\{\*\*\}Falcon\-7b\-instruct27024544124 / 2\.73≤5\.0×10−4∗⁣∗∗\\leq 5\.0\{\\times\}10^\{\-4\}\\,\{\}^\{\*\*\*\}≤5\.0×10−3∗∗\\leq 5\.0\{\\times\}10^\{\-3\}\\,\{\}^\{\*\*\}Table 12:Full sharedness results across layers\.Same setting as Table[10\(b\)](https://arxiv.org/html/2607.20469#A2.T10.st2), but reporting layersℓ∈\{4,10,20,24\}\\ell\\in\\\{4,10,20,24\\\}for each model\.ModelLayerdddcrossd\_\{\\mathrm\{cross\}\}dcrossd\\frac\{d\_\{\\mathrm\{cross\}\}\}\{d\}TℓT\_\{\\ell\}Tℓdcross\\frac\{T\_\{\\ell\}\}\{d\_\{\\mathrm\{cross\}\}\}statesppermp\_\{\\mathrm\{perm\}\}/pscrp\_\{\\mathrm\{scr\}\}Falcon\-7B4454426790\.5901250\.046760235\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Falcon\-7B10454427020\.5951240\.045960235\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Falcon\-7B20454427420\.6031110\.040560235\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Falcon\-7B24454428170\.6201090\.038760235\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-2\-7B\-chat4409629700\.7251180\.0397185605\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-2\-7B\-chat10409629890\.7301090\.0365185605\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-2\-7B\-chat20409630660\.7491400\.0457185605\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-2\-7B\-chat24409631840\.7771380\.0433185605\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-3\.1\-8B4409627390\.6691070\.0391200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-3\.1\-8B10409628090\.6861120\.0399200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-3\.1\-8B20409627950\.682940\.0336200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Llama\-3\.1\-8B24409628610\.698980\.0343200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Qwen2\.5\-7B4358426210\.731920\.0351200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Qwen2\.5\-7B10358426140\.729730\.0279200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Qwen2\.5\-7B20358426310\.734870\.0331200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}Qwen2\.5\-7B24358426780\.7471110\.0414200005\.00×10−45\.00\{\\times\}10^\{\-4\}/4\.98×10−34\.98\{\\times\}10^\{\-3\}#### B\.1\.1 Within\-category versus mixed\-category sharing\.

To probe whether sharing is merely a byproduct of coarse dataset categories, we compare*within\-category*task pairs to a*mixed\-category*baseline formed by cross\-category pairs\. For each pair, we compute the*shared ratio*SR=\|Sℓ​\(τ,m\)\|/k\\mathrm\{SR\}=\|S\_\{\\ell\}\(\\tau,m\)\|/k, wherekkis the pooled PCA dimensionality selected by the variance retention target \(defaultρ=0\.95\\rho=0\.95\) and we use the same sharedness threshold as in the main paper \(defaultτ=10−3\\tau=10^\{\-3\}\)\. Figure[13\(a\)](https://arxiv.org/html/2607.20469#A2.F13.sf1)summarizes the results and Table[13](https://arxiv.org/html/2607.20469#A2.T13)reports the corresponding numbers\. The math pair \(GSM8K\+AQuA\) exhibits noticeably higher within\-pair sharing than the mixed baseline, suggesting stronger reuse within mathematical reasoning\. By contrast, commonsense, reasoning, and science show within\-category sharing that is similar to \(or slightly below\) the mixed baseline, indicating that coarse category labels do not uniformly imply higher shareness under our metric\. Overall, this diagnostic suggests that the decode\-time shared core identified by DecodeShare is not simply a trivial artifact of coarse category membership\.

![Refer to caption](https://arxiv.org/html/2607.20469v1/x20.png)\(a\)Within\-category shareness compared to a mixed\-category baseline \(mean±\\pmstd\) across four coarse categories\.
![Refer to caption](https://arxiv.org/html/2607.20469v1/x21.png)\(b\)Subspace overlap to the full\-task pooled subspace as a function of the number of tasks used to estimate the pooled PCA basis\.

Figure 13:\(Left\) Within\-category shareness vs\. mixed\-category baseline\. \(Right\) Overlap to the full\-task pooled subspace vs\. number of tasks\.
#### B\.1\.2\. Pooled subspace convergence as we add tasks\.

Next, we study whether the*pooled PCA subspace*used to define shared directions is stable as the task pool grows\. Fix a layerℓ\\elland retention target \(defaultρ=0\.95\\rho=0\.95\)\. For eachn∈\{2,…,T\}n\\in\\\{2,\\dots,T\\\}, we samplenntasks, estimate a pooled PCA basisQnQ\_\{n\}, and compare it to the full\-task reference basisQTQ\_\{T\}\. To quantify similarity between two orthonormal basesQa∈ℝd×kaQ\_\{a\}\\in\\mathbb\{R\}^\{d\\times k\_\{a\}\}andQb∈ℝd×kbQ\_\{b\}\\in\\mathbb\{R\}^\{d\\times k\_\{b\}\}, we use a normalized subspace overlap score

Overlap​\(Qa,Qb\)≜‖Qa⊤​Qb‖F2min⁡\(ka,kb\)∈\[0,1\],\\mathrm\{Overlap\}\(Q\_\{a\},Q\_\{b\}\)\\;\\triangleq\\;\\frac\{\\\|Q\_\{a\}^\{\\top\}Q\_\{b\}\\\|\_\{F\}^\{2\}\}\{\\min\(k\_\{a\},k\_\{b\}\)\}\\in\[0,1\],which equals the mean squared cosine of principal angles \(1 indicates identical subspaces\)\. Table[14](https://arxiv.org/html/2607.20469#A2.T14)and Fig\.[13\(b\)](https://arxiv.org/html/2607.20469#A2.F13.sf2)show that the pooled subspace converges quickly: even with a small number of tasks, the overlap to the full\-task reference is already high and increases monotonically as tasks are added\. Meanwhile the pooled dimensionkk\(cross\_dimin the table\) grows withnn, consistent with additional tasks contributing extra directions needed to maintain the same explained\-variance target\. This supports a practical point: DecodeShare’s pooled coordinate system is not fragile to the exact composition of the task pool, even though the*shared set*itself depends on the sharedness threshold and the “nontrivial for many tasks” criterion\.

Table 13:Within/mixed\-category diagnostic for H1\. SR denotes shared ratio\|Sℓ​\(τ,m\)\|/k\|S\_\{\\ell\}\(\\tau,m\)\|/kunder the default settings\.CategoryWithin taskskkWithin SRMixed SR \(μ±σ\\mu\\pm\\sigma\)Within / Mixed countmathGSM8K, AQuA22210\.07070\.0461±\\pm0\.0070157 / 123\.1±\\pm14\.0commonsenseCSQA, PIQA27230\.04330\.0477±\\pm0\.0074118 / 126\.8±\\pm14\.8reasoningStratQA, BoolQ28150\.04260\.0461±\\pm0\.0062120 / 123\.5±\\pm12\.7scienceOBQA, QASC26000\.04500\.0480±\\pm0\.0072117 / 126\.7±\\pm14\.7Table 14:Pooled\-subspace convergence diagnostic \(decode\-time states\)\. We report mean±\\pmstd over repeats\.LLaMAQwennntasksOverlap↑\\uparrowcross\_dimOverlap↑\\uparrowcross\_dim20\.921±\\pm0\.0072636\.6±\\pm136\.80\.914±\\pm0\.0062258\.0±\\pm80\.130\.937±\\pm0\.0052788\.9±\\pm83\.40\.929±\\pm0\.0072419\.5±\\pm48\.040\.948±\\pm0\.0052885\.1±\\pm56\.40\.942±\\pm0\.0052490\.1±\\pm38\.850\.960±\\pm0\.0042904\.3±\\pm59\.40\.954±\\pm0\.0042521\.1±\\pm38\.560\.971±\\pm0\.0032951\.5±\\pm33\.90\.968±\\pm0\.0052569\.1±\\pm24\.070\.985±\\pm0\.0022967\.2±\\pm24\.00\.981±\\pm0\.0042592\.4±\\pm17\.081\.000±\\pm0\.0002989\.0±\\pm0\.01\.000±\\pm0\.0002614\.0±\\pm0\.0
#### B\.1\.3\. Phase\-distinguished pooled geometry and the fully\-shared core\.

Finally, we compare pooled subspaces estimated from different inference phases:decode\-last\(KV\-cached, seq len=1=1\),prefill\-last\(full\-sequence prefill\), anddecode\-step\-tt\(a fixed decode step\)\. Table[15](https://arxiv.org/html/2607.20469#A2.T15)shows that*all*phases yield stable pooled subspaces under the overlap diagnostic, but with markedly different pooled dimensions:decode\-lastproduces a much larger pooled subspace \(thousands of dimensions atρ=0\.95\\rho=0\.95\), whereasprefill\-lastproduces a much smaller pooled subspace \(hundreds of dimensions\)\. This highlights a key nuance:*subspace stability alone is not decode\-specific*—prefill can also yield a stable low\-rank pooled geometry\.

To connect stability to shareness, we evaluate the*fully\-shared core size*within each estimated basis by applying the same sharedness criterion at the all\-task settingm=\|T\|m=\|T\|\(usingτ=10−3\\tau=10^\{\-3\}\) and counting how many pooled directions are nontrivial for*every*task\. Table[15](https://arxiv.org/html/2607.20469#A2.T15)shows that the*absolute*fully\-shared core size remains on the order of10210^\{2\}directions across phases and acrossnn, while the*shared ratio*can vary widely because the denominatorkkdiffers substantially across phases \(and typically increases withnn\)\. In particular,decode\-lastyields a modest shared core in absolute size but a smaller shared ratio due to a much larger pooled dimension, whereasprefill\-lastyields a higher shared ratio largely because its pooled dimension is much smaller\. These diagnostics sharpen the main paper’s message: recovering a stable pooled geometry is not sufficient to identify a decision\-time shared channel, and phase alignment matters for whether an estimated basis transfers to decode\-time causal tests \(see H3\)\.

Table 15:Phase\-distinguished pooled\-subspace convergence \(LLaMA\)\. Mean±\\pmstd over repeats\.nntasksMetric / Mode2345678Overlap↑\\uparrow\(decode\-last\)0\.920±0\.0060\.920\\pm 0\.0060\.933±0\.0050\.933\\pm 0\.0050\.946±0\.0040\.946\\pm 0\.0040\.958±0\.0040\.958\\pm 0\.0040\.970±0\.0010\.970\\pm 0\.0010\.984±0\.0020\.984\\pm 0\.0021\.000±0\.0001\.000\\pm 0\.000cross\_dim \(decode\-last\)2571\.0±87\.02571\.0\\pm 87\.02745\.7±52\.22745\.7\\pm 52\.22818\.9±69\.32818\.9\\pm 69\.32869\.9±54\.32869\.9\\pm 54\.32890\.2±33\.72890\.2\\pm 33\.72942\.9±11\.62942\.9\\pm 11\.62961\.0±0\.02961\.0\\pm 0\.0Overlap↑\\uparrow\(prefill\-last\)0\.884±0\.0390\.884\\pm 0\.0390\.896±0\.0290\.896\\pm 0\.0290\.910±0\.0260\.910\\pm 0\.0260\.922±0\.0230\.922\\pm 0\.0230\.948±0\.0120\.948\\pm 0\.0120\.959±0\.0160\.959\\pm 0\.0161\.000±0\.0001\.000\\pm 0\.000cross\_dim \(prefill\-last\)160\.4±3\.9160\.4\\pm 3\.9223\.5±3\.8223\.5\\pm 3\.8278\.1±6\.1278\.1\\pm 6\.1327\.8±6\.6327\.8\\pm 6\.6377\.5±5\.7377\.5\\pm 5\.7415\.7±4\.7415\.7\\pm 4\.7456\.0±0\.0456\.0\\pm 0\.0Overlap↑\\uparrow\(decode\-step\-tt\)0\.946±0\.0110\.946\\pm 0\.0110\.946±0\.0070\.946\\pm 0\.0070\.948±0\.0070\.948\\pm 0\.0070\.954±0\.0040\.954\\pm 0\.0040\.961±0\.0040\.961\\pm 0\.0040\.976±0\.0040\.976\\pm 0\.0041\.000±0\.0001\.000\\pm 0\.000cross\_dim \(decode\-step\-tt\)181\.2±14\.7181\.2\\pm 14\.7271\.8±15\.6271\.8\\pm 15\.6345\.6±17\.0345\.6\\pm 17\.0426\.2±9\.0426\.2\\pm 9\.0498\.2±13\.4498\.2\\pm 13\.4577\.0±8\.8577\.0\\pm 8\.8644\.0±0\.0644\.0\\pm 0\.0![Refer to caption](https://arxiv.org/html/2607.20469v1/figures/llama_exp2.75_distinguish.png)Figure 14:Phase\-distinguished convergence trends across modes\. Left: overlap to the full\-task pooled subspace; right: cross dimension as the number of tasks increases\.
#### B\.1\.4\. All\-tasks vs\. majority\-shared \(pooled\) shareness\.

We distinguish two*thresholding regimes*for the same shared\-set definition computed in a*single, task\-aligned*coordinate system\. Concretely, we first obtain a shared basisQℓQ\_\{\\ell\}via pooled PCA \(to remove the permutation/rotation non\-identifiability of per\-task PCA components\), and then define the shared set

Sℓ​\(τ,m\)=\{i:\|\{t∈𝒯:rℓ,t,i≥τ\}\|≥m\},S\_\{\\ell\}\(\\tau,m\)=\\Bigl\\\{i\\;:\\;\\bigl\|\\\{t\\in\\mathcal\{T\}:r\_\{\\ell,t,i\}\\geq\\tau\\\}\\bigr\|\\geq m\\Bigr\\\},whererℓ,t,ir\_\{\\ell,t,i\}is the task\-ttrelative\-variance score of directioniiat layerℓ\\ell\. TheAll\-taskssetting \(m=\|𝒯\|m=\|\\mathcal\{T\}\|\) is the strict*intersection*criterion: a direction is declared shared only if it exceedsτ\\taufor*every*task\. Themajority\-sharedsetting \(our default; e\.g\.,m=8m=8out of\|𝒯\|=13\|\\mathcal\{T\}\|=13\) operationalizes the claim as a*compact core shared by many tasks*rather than by all tasks\. Importantly, both settings are evaluated under the*same*two task\-preserving nulls \(permutation Null\-1 and orthogonal\-scramble\+re\-estimation Null\-2\), so switching fromm=\|𝒯\|m=\|\\mathcal\{T\}\|tom<\|𝒯\|m<\|\\mathcal\{T\}\|does not relax statistical control: a shared set must still lie in the tail of*both*null distributions, and Null\-2 in particular is designed to reject spuriously inflated shareness that can arise from coordinate artifacts or over\-permissive thresholds\.

#### Why cross all\-tasks shared subspace can be empty without contradicting H1\.

All\-tasks shareness is a*lower bound*on cross\-task sharing: by constructionSℓ​\(τ,\|𝒯\|\)⊆Sℓ​\(τ,m\)S\_\{\\ell\}\(\\tau,\|\\mathcal\{T\}\|\)\\subseteq S\_\{\\ell\}\(\\tau,m\)for anym<\|𝒯\|m<\|\\mathcal\{T\}\|\. With heterogeneous tasks \(here, 13 diverse benchmarks\), the all\-tasks intersection can become empty even when there exists a genuine shared mechanism, because it is extremely sensitive to \(i\) task\-specific idiosyncrasies, \(ii\) finite\-sample variability inrℓ,t,ir\_\{\\ell,t,i\}, and \(iii\) any single “outlier” task that falls just belowτ\\tauon an otherwise consistently\-used direction\. Thus, observing a non\-empty and significant majority\-shared core alongside an empty all\-tasks intersection should be interpreted as: there exists a statistically validated shared decision subspace used by*most*tasks, but not necessarily in a uniform\-above\-threshold way for*every*task\. This is exactly the regime our claim targets: a*compact, decode\-time shared workspace*that is broadly reused across tasks, while allowing that some tasks may rely on alternative or additional directions\. Finally, we guard against the concern that pooling could induce task leakage by using the LOTO control \(fitQℓQ\_\{\\ell\}excluding a held\-out task and then evaluate shareness on that held\-out task\), ensuring the reported majority\-shared core reflects genuine cross\-task structure rather than pooled overfitting\.

Table 16:H1 shareness test on the 13\-task full\_benchmark at layerℓ=10\\ell=10: comparison between the strict all\-tasks definition \(m=\|T\|m=\|T\|, shown asm=13m=13or “all” depending on the run\) and the majority\-shared core \(m=8m=8\)\. We report cross\-task pooled PCA dimension \(cross\), shared set size\|S\|\|S\|, ratio\|S\|/c​r​o​s​s\|S\|/cross, and null p\-valuesp1p\_\{1\}\(permutation\) andp2p\_\{2\}\(orthogonal scramble\)\.ModelSettingτ\\taummcross\|S\|\|S\|\|S\|/c​r​o​s​s\|S\|/crossp1p\_\{1\}p2p\_\{2\}Llama\-3\.1\-8B\-InstructAll\-tasks0\.0010all28121100\.0390\.0005000\.009901Llama\-3\.1\-8B\-InstructPooled \(majority\)0\.0010828121430\.0510\.0005000\.009901Qwen2\.5\-7B\-InstructAll\-tasks0\.0010132549560\.0220\.0005000\.009901Qwen2\.5\-7B\-InstructPooled \(majority\)0\.001082549850\.0330\.0005000\.009901Falcon\-7B\-InstructAll\-tasks0\.0010all24791220\.0490\.0005000\.009901Falcon\-7B\-InstructPooled \(majority\)0\.0010824791540\.0620\.0005000\.009901Llama\-2\-7b\-chat\-hfAll\-tasks0\.001013242300\.0001\.0000001\.000000Llama\-2\-7b\-chat\-hfPooled \(majority\)0\.001082423350\.0140\.0005000\.009901Table 17:Loosened shareness setting on the same 13\-task full\_benchmark \(layerℓ=10\\ell=10\):τ=3×10−4\\tau=3\\times 10^\{\-4\}andm=8m=8\. We report cross,\|S\|\|S\|,\|S\|/c​r​o​s​s\|S\|/cross, and null p\-valuesp1p\_\{1\}/p2p\_\{2\}using the same criteria as Table[16](https://arxiv.org/html/2607.20469#A2.T16)\.Modelτ\\taummcross\|S\|\|S\|\|S\|/c​r​o​s​s\|S\|/crossp1p\_\{1\}p2p\_\{2\}meta\-llama/Llama\-3\.1\-8B\-Instruct0\.0003828126700\.2380\.0005001\.000000Qwen/Qwen2\.5\-7B\-Instruct0\.00038254911630\.4560\.0005000\.009901tiiuae/falcon\-7b\-instruct0\.0003824795300\.2140\.0005001\.000000meta\-llama/Llama\-2\-7b\-chat\-hf0\.00038242313890\.5730\.0005000\.009901
#### B\.1\.5\. Loosened threshold sensitivity \(four\-model subset\)\.

We further test a permissive threshold setting \(loosened:τ=3×10−4\\tau=3\\times 10^\{\-4\},m=8m=8\) on the same 13\-task pool to probe the predicted degeneracy regime\. For Llama\-3\.1\-8B and Falcon\-7B, the shared set expands substantially \(e\.g\.,\|S\|/cross=0\.238\|S\|/\\texttt\{cross\}=0\.238and0\.2140\.214respectively\), but fails the stronger orthogonal\-scramble null \(Null\-2;p2=1\.0p\_\{2\}=1\.0\), indicating that the inflated shared size is not distinguishable from scrambled structure\. For Qwen2\.5\-7B and Llama\-2\-7B, the loosened setting remains significant under both nulls \(p1<0\.05p\_\{1\}<0\.05,p2<0\.05p\_\{2\}<0\.05\) yet yields very large shared ratios \(\|S\|/cross=0\.456\|S\|/\\texttt\{cross\}=0\.456and0\.5730\.573\), suggesting that overly permissive thresholds can recover a broad low\-rank decision subspace rather than a compact shared core\. Overall, these results are consistent with theτ\\tau\-stringency interpretation: whenτ\\taubecomes too small relative to the diffuse\-usage scale1/k1/k, many low\-concentration directions are labeled nontrivial, producing a large and less discriminative shared set; therefore we useτ=10−3\\tau=10^\{\-3\}as the conservative default and treatloosenedprimarily as a sensitivity/degeneracy check\.

### B\.2Additional Results forH2: Experiments for More Models\.

#### B\.2\.1 Patchback results across models and layers\.

Tables[1](https://arxiv.org/html/2607.20469#S3.T1),[18](https://arxiv.org/html/2607.20469#A2.T18), and[19](https://arxiv.org/html/2607.20469#A2.T19)summarize patchback behavior across three model families \(Llama, Falcon, Qwen\) and three representative depths \(layers 4/10/24\)\. Throughout, rescue rates are computed on the corresponding flip setℱ\\mathcal\{F\}\(examples that flip under the specified decode\-only ablation\), and aggregated by summing rescued/total across tasks \(and across metrics where applicable\)\.

#### A consistent qualitative pattern: patching alongSℓS\_\{\\ell\}is highly effective; nonshared controls are negligible\.

Across models and layers, patching along the identified shared decode subspaceSℓS\_\{\\ell\}is consistently effective\. In multiple\-choice \(Table[1](https://arxiv.org/html/2607.20469#S3.T1)\), the full patch \(Patched@full\) rescues essentially all flips, while patching a nonshared direction is exactly zero across all settings \(NonsharedPatch\)\. In open\-answer \(Table[18](https://arxiv.org/html/2607.20469#A2.T18)\), NonsharedPatch remains small \(typically∼\\sim1–7%\) and far below Patched\(self\), arguing against a naive “any patch works” explanation and supporting the view that a small shared decode pathway is a high\-leverage causal route for last\-token decision\-making\.

#### Model\- and layer\-dependent separation from random controls\.

While targeted patching is consistently strong, the strength of random controls varies by model and depth\. In Llama, random shared\-vector and random\-subspace controls are comparatively small, yielding a clean mechanistic gap\. In contrast, Qwen exhibits unusually strong random\-subspace baselines at multiple layers, indicating that generic low\-rank perturbations can often intersect decision\-relevant directions\. This suggests shared\-subspace interference is a primary contributor to failures, but some models/layers also exhibit broader low\-rank decision structure that makes random controls stronger\.

#### Open\-answer: strong time\-shuffle indicates temporal stationarity of the shared decode signal\.

In open\-answer patchback \(Table[18](https://arxiv.org/html/2607.20469#A2.T18)\), the within\-exampleTime\-shuffleintervention can approach Patched\(self\.\) This is expected under our setup: the shared decode subspaceSℓS\_\{\\ell\}is estimated from the decode\-time activation distribution, so donor components drawn from other decode steps of the same example remain on\-manifold forSℓS\_\{\\ell\}\. The competitiveness of Time\-shuffle therefore suggests that the decision\-relevant signal carried bySℓS\_\{\\ell\}is approximately*stationary across decode steps*\(i\.e\., not tied to a single precise time index\)\. Importantly, this does not collapse specificity: patching directions outsideSℓS\_\{\\ell\}\(NonsharedPatch\) remains small, and random low\-rank controls remain substantially below targeted patching in most settings\. Nevertheless, Patched\(self\) remains the strongest intervention overall, and NonsharedPatch stays comparatively small, preserving a clear separation between targeted repair and naive patching\.

#### Transfer patching can be depth\-sensitive\.

Flipset transfer \(Table[19](https://arxiv.org/html/2607.20469#A2.T19)\) reveals that robustness does not necessarily transfer uniformly across layers\. For example, Qwen maintains high transfer rescue at layer10 but degrades sharply at layer24, even though self patching remains at 100%\. This layer sensitivity suggests that the shared decode pathway is not monolithic across depth: deeper layers may require different donor selection or richer basis estimation to support cross\-task transfer reliably\.

Table 18:Open\-answer patchback, aggregated over tasks/metrics\.Aggregated over GSM8K \(gen\-math \+ pair\-logprob\) and HumanEval \(pair\-logprob \+ gen\-code\-compile\)\.\|ℱagg\|\|\\mathcal\{F\}\_\{\\mathrm\{agg\}\}\|is the aggregated denominator \(summed across included tasks/metrics\)\. Green column is targeted patching alongSℓS\_\{\\ell\}; blue columns are controls\. Time\-shuffle is a within\-example temporal diagnostic \(donor taken from a different decode step of the same example\), preserving the decode\-time distribution used to estimateSℓS\_\{\\ell\}\.ModelLayer\|ℱagg\|\|\\mathcal\{F\}\_\{\\mathrm\{agg\}\}\|Targeted patchControlsPatched\(self\)Time\-shuffleRandVec\(shared\)RandSubspaceNonsharedPatchLlama\-2\-7B\-Chat46061\.7 %\(37/60\)53\.3 %\(32/60\)20\.0 %\(12/60\)11\.7 %\(7/60\)0\.0 %\(0/60\)Llama\-2\-7B\-Chat108638\.4 %\(33/86\)36\.0 %\(31/86\)12\.8 %\(11/86\)9\.3 %\(8/86\)3\.5 %\(3/86\)Llama\-2\-7B\-Chat247152\.1 %\(37/71\)42\.3 %\(30/71\)4\.2 %\(3/71\)2\.8 %\(2/71\)1\.4 %\(1/71\)Falcon\-7B\-Instruct410755\.1 %\(59/107\)48\.6 %\(52/107\)8\.4 %\(9/107\)1\.9 %\(2/107\)0\.9 %\(1/107\)Falcon\-7B\-Instruct1010554\.3 %\(57/105\)48\.6 %\(51/105\)5\.7 %\(6/105\)3\.8 %\(4/105\)1\.0 %\(1/105\)Falcon\-7B\-Instruct248037\.5 %\(30/80\)31\.2 %\(25/80\)7\.5 %\(6/80\)0\.0 %\(0/80\)1\.2 %\(1/80\)Qwen2\.5\-7B\-Instruct49474\.5 %\(70/94\)70\.2 %\(66/94\)19\.1 %\(18/94\)17\.0 %\(16/94\)7\.4 %\(7/94\)Qwen2\.5\-7B\-Instruct108470\.2 %\(59/84\)67\.9 %\(57/84\)17\.9 %\(15/84\)19\.0 %\(16/84\)7\.1 %\(6/84\)Qwen2\.5\-7B\-Instruct245446\.3 %\(25/54\)35\.2 %\(19/54\)9\.3 %\(5/54\)3\.7 %\(2/54\)1\.9 %\(1/54\)Table 19:Flipset transfer patching on AQuA \(aggregated\)\.\|ℱflip\|\|\\mathcal\{F\}\_\{\\mathrm\{flip\}\}\|is the flipset size \(baseline\-correct examples that flip under full ablation\)\. Both columns patch along the shared decode subspaceSℓS\_\{\\ell\}: self\-donor vs transfer donor\.ModelLayer\|ℱflip\|\|\\mathcal\{F\}\_\{\\mathrm\{flip\}\}\|Patched\(self\)Patched\(transfer\)Llama\-2\-7B\-Chat44075\.0 %\(30/40\)75\.0 %\(30/40\)Llama\-2\-7B\-Chat1016975\.1 %\(127/169\)78\.7 %\(133/169\)Llama\-2\-7B\-Chat246657\.6 %\(38/66\)36\.4 %\(24/66\)Falcon\-7B\-Instruct46100\.0 %\(6/6\)83\.3 %\(5/6\)Falcon\-7B\-Instruct106100\.0 %\(6/6\)83\.3 %\(5/6\)Falcon\-7B\-Instruct2488100\.0 %\(88/88\)92\.0 %\(81/88\)Qwen2\.5\-7B\-Instruct4116100\.0 %\(116/116\)96\.6 %\(112/116\)Qwen2\.5\-7B\-Instruct10128100\.0 %\(128/128\)96\.1 %\(123/128\)Qwen2\.5\-7B\-Instruct24122100\.0 %\(122/122\)24\.6 %\(30/122\)Table 20:H3 subspace statistics\. We estimate shared subspaces from decode vs\. prefill states under\(τ=10−3,m=all\)\(\\tau\{=\}10^\{\-3\},m\{=\}\\texttt\{all\}\)\.kmatch=min⁡\(kshareddec,ksharedpre\)=48k\_\{\\text\{match\}\}=\\min\(k\_\{\\text\{shared\}\}^\{\\text\{dec\}\},k\_\{\\text\{shared\}\}^\{\\text\{pre\}\}\)=48\. Principal Ang\. \(deg\) summarize the distribution of principal angles between the decode\- and prefill\-estimated shared subspaces \(reported as mean/p50/p95\); larger values indicate stronger misalignment\.Settingkshareddeck\_\{\\text\{shared\}\}^\{\\text\{dec\}\}ksharedprek\_\{\\text\{shared\}\}^\{\\text\{pre\}\}kmatchk\_\{\\text\{match\}\}Principal Ang\. \(deg\)meanp50p95LLaMA2\-7B\-Chat,layer=10136484879\.7482\.0489\.12Table 21:Prefill\-vs\-decode alignment\.All entries are accuracy in % \(top\) with 95% CI \(bottom\)\. Protocol: generation\-based evaluation\. We estimate a shared basis from decode states \(Decode\-shared\) or from prefill states \(Prefill\-shared\), match dimensions \(k=129k\{=\}129\), and apply the same*decode\-only*projection removal with a matched\-energy random control \(Random\)\.TaskBaselineDecode\-sharedPrefill\-sharedRandomppGSM8K16\.816\.8\[12\.5, 21\.5\]0\.00\.0\[0\.0, 0\.0\]17\.617\.6\[13\.3, 22\.3\]16\.816\.8\[12\.5, 21\.5\]<10−4<10^\{\-4\}CSQA37\.937\.9\[32\.0, 44\.1\]23\.423\.4\[18\.4, 28\.9\]31\.231\.2\[25\.8, 36\.7\]40\.640\.6\[34\.8, 46\.5\]0\.0380\.038StrategyQA43\.843\.8\[37\.9, 50\.0\]10\.510\.5\[7\.0, 14\.5\]41\.441\.4\[35\.5, 47\.7\]46\.546\.5\[40\.2, 52\.7\]<10−4<10^\{\-4\}ARC\-C49\.649\.6\[43\.8, 55\.5\]19\.519\.5\[14\.8, 24\.6\]39\.139\.1\[33\.2, 44\.9\]48\.448\.4\[42\.6, 54\.7\]<10−4<10^\{\-4\}OBQA48\.448\.4\[42\.2, 54\.7\]25\.025\.0\[19\.5, 30\.5\]41\.441\.4\[35\.5, 47\.3\]45\.745\.7\[39\.5, 51\.6\]<10−4<10^\{\-4\}QASC28\.928\.9\[23\.4, 34\.4\]10\.210\.2\[6\.6, 13\.7\]27\.727\.7\[22\.3, 33\.2\]29\.329\.3\[23\.8, 34\.8\]<10−4<10^\{\-4\}LogiQA22\.722\.7\[17\.6, 27\.7\]25\.825\.8\[20\.7, 31\.2\]15\.215\.2\[10\.9, 19\.9\]19\.519\.5\[14\.8, 24\.2\]0\.00320\.0032BoolQ65\.665\.6\[59\.8, 71\.5\]55\.555\.5\[49\.2, 61\.3\]66\.066\.0\[60\.2, 71\.9\]70\.370\.3\[64\.8, 75\.8\]0\.00290\.0029

### B\.3Additional Results forH3: results under generation\-based evaluation

#### Prefill\-Decode mismatch in Generation\.

Table[21](https://arxiv.org/html/2607.20469#A2.T21)reproduces the H3 mismatch signature under a generation\-based protocol\. We apply the same*decode\-only*intervention while swapping only the estimation distribution: a shared basis estimated from decode states \(Decode\-shared\) versus one estimated from prefill states \(Prefill\-shared\), with dimension matched \(k=129k=129\) and a matched\-energy random control \(Random\)\. Across tasks, removing the*decode\-estimated*shared workspace causes large degradation \(e\.g\., GSM8K collapses to 0\.0%, StrategyQA drops from 43\.8% to 10\.5%, ARC\-C from 49\.6% to 19\.5%, QASC from 28\.9% to 10\.2%\), while the*prefill\-estimated*basis is much weaker and often stays near baseline within confidence intervals\. The random control remains close to baseline, indicating the effect is not explained by removing arbitrary energy\-matched subspaces\. Most differences are statistically significant \(p\-values in the last column\), supporting that estimator–deployment mismatch persists beyond the forced\-choice setting\.

We note minor task\-dependent exceptions \(e\.g\., LogiQA\), but they do not change the qualitative conclusion: the strongest and most consistent causal impact at the decode locus arises only from*decode\-aligned*shared bases\. Together with Table[7](https://arxiv.org/html/2607.20469#S3.T7), these results reinforce the core claim that prefill\-derived shared directions are generally not a substitute for decode\-time interventions\.

### B\.4H3: Prefill–Decode Mismatch and the Need for Decode\-Aligned Estimation

#### Motivation: estimator–deployment mismatch under KV\-cached decoding\.

Under KV\-cached inference, each next\-token decision is produced by a cached forward pass with sequence length=1=1\. Consequently, the decode\-time decision states follow a distributionDdecodeD\_\{\\mathrm\{decode\}\}that can differ from the full\-sequence prefill distributionDprefillD\_\{\\mathrm\{prefill\}\}\. This creates an estimator–deployment mismatch for many activation\-space pipelines that estimate directions from prefill states but intervene at decode time\. DecodeShare enforces an*alignment principle*: all causal interventions act only on cached decode calls, and subspaces are estimated from the same decode\-time distribution on which we intervene\.

#### 2×22\\times 2estimation–intervention grid\.

To isolate the causal consequence of prefill–decode mismatch, we evaluate a2×22\\times 2grid that crosses \(i\) the*estimation distribution*\(decode\-last vs\. prefill\-last\) and \(ii\) the*intervention locus*\(decode\-only vs\. prefill\-only\)\. Concretely, we estimateQdecode\(S\)Q^\{\(S\)\}\_\{\\mathrm\{decode\}\}from decode\-last states andQprefill\(S\)Q^\{\(S\)\}\_\{\\mathrm\{prefill\}\}from prefill\-last states \(at the same layer, tasks, and hyperparameters\), and apply projection removalh~=h−α​Q​Q⊤​h\\tilde\{h\}=h\-\\alpha QQ^\{\\top\}heither during cached decoding \(seq\-len=1=1\) or only at prefill\. Fixing the intervention locus to decode\-only turns an estimator swap \(decode\-est vs\. prefill\-est\) into a direct test of mismatch while keeping the intervention budget identical \(Table[23](https://arxiv.org/html/2607.20469#A2.T23)\)\. A representative small\-rank setting preserves the qualitative contrast, ruling out a dimension\-driven artifact \(Table[22](https://arxiv.org/html/2607.20469#A2.T22)\)\.

#### Geometric mismatch\.

Decode\- and prefill\-estimated shared subspaces are strongly misaligned even after rank matching\. For example, at layerℓ=10\\ell=10withkmatch=min⁡\(kdecodeshared,kprefillshared\)k\_\{\\mathrm\{match\}\}=\\min\(k^\{\\mathrm\{shared\}\}\_\{\\mathrm\{decode\}\},k^\{\\mathrm\{shared\}\}\_\{\\mathrm\{prefill\}\}\), the principal angles between the two shared subspaces are near\-orthogonal, and the decode\-estimated shared dimension is substantially larger than the prefill\-estimated one \(Table[20](https://arxiv.org/html/2607.20469#A2.T20)\)\. This confirms a concrete prefill–decode mismatch: a prefill\-derived basis is not a faithful proxy for the decode\-time workspace we causally probe\.

#### Causal consequence at the decode locus\.

Under decode\-only intervention, removing the*decode\-estimated*shared basis induces large and consistent accuracy degradation across benchmarks, whereas swapping only the estimator distribution \(prefill\-est / decode\-intervene\) removes this effect and behaves comparably to matched random controls \(Table[7](https://arxiv.org/html/2607.20469#S3.T7)\)\. Interventions applied only at prefill are negligible across all conditions, ruling out explanations that operate purely through prompt\-processing dynamics\. Together, these results support H3:*to reproduce robust decode\-time causal effects under matched budgets, estimation must be aligned to the decode\-time distribution*\.

#### Replication under generation\-based evaluation\.

To verify that the mismatch signature is not an artifact of forced\-choice scoring, we replicate on generation\-based evaluation Table[21](https://arxiv.org/html/2607.20469#A2.T21)and in a mixed\-protocol setting Table[23](https://arxiv.org/html/2607.20469#A2.T23)\. Across tasks, decode\-estimated shared removal yields substantially larger degradations than prefill\-estimated removal under the same decode\-only intervention and matched controls, indicating that estimator–deployment mismatch persists beyond the decision\-level setting\.

#### Dimension matching and representative small\-kksanity check\.

To ensure the comparison is not driven by dimensionality, we reportkk\-matched bases throughout \(Appendix Tables[22](https://arxiv.org/html/2607.20469#A2.T22)–[23](https://arxiv.org/html/2607.20469#A2.T23)\)\. Moreover, a representative small\-kksetting \(e\.g\.,kmatch=16k\_\{\\mathrm\{match\}\}=16\) still exhibits the same qualitative mismatch on generation\-heavy benchmarks, indicating the effect is not explained by “using more dimensions”\.

Table 22:Same as Table[23](https://arxiv.org/html/2607.20469#A2.T23), but with a representative small rank \(kmatch=16k\_\{\\mathrm\{match\}\}=16\) to rule out dimension\-driven artifacts\.TaskProtocolnnBaselineDecode\-shared \(k=16k\{=\}16\)Prefill\-shared \(k=16k\{=\}16\)Random \(k=16k\{=\}16\)Δ​\(Decode−Prefill\)\\Delta\(\\mathrm\{Decode\{\-\}Prefill\}\)ppgsm8kgeneration131919\.6\[17\.5, 21\.8\]3\.6\[2\.6, 4\.6\]19\.1\[17\.0, 21\.2\]19\.7\[17\.5, 21\.8\]\-15\.5\[\-17\.9, \-13\.3\]0\.0001commonsenseqaforced\_choice122155\.6\[52\.8, 58\.4\]36\.4\[33\.7, 39\.2\]36\.0\[33\.3, 38\.7\]55\.4\[52\.7, 58\.2\]\+0\.4\[\-1\.4, \+2\.2\]0\.716strategyqaforced\_choice68750\.4\[46\.7, 54\.0\]52\.0\[48\.2, 55\.7\]48\.6\[44\.8, 52\.4\]50\.5\[46\.9, 54\.0\]\+3\.3\[\+0\.9, \+5\.8\]0\.0089aquaforced\_choice25427\.2\[21\.7, 32\.7\]25\.2\[20\.1, 30\.3\]25\.2\[20\.1, 30\.7\]27\.2\[22\.0, 32\.7\]\+0\.0\[\-3\.5, \+3\.5\]1arc\_challengeforced\_choice117253\.1\[50\.2, 55\.9\]33\.7\[31\.1, 36\.5\]33\.4\[30\.8, 36\.2\]53\.0\[50\.2, 55\.9\]\+0\.3\[\-1\.5, \+2\.1\]0\.724openbookqaforced\_choice50057\.8\[53\.6, 62\.2\]36\.8\[32\.6, 41\.0\]34\.2\[30\.0, 38\.6\]58\.2\[53\.8, 62\.4\]\+2\.6\[\+0\.0, \+5\.2\]0\.0372qascforced\_choice92652\.6\[49\.4, 55\.8\]32\.9\[29\.9, 36\.0\]36\.1\[32\.9, 39\.2\]52\.2\[48\.9, 55\.3\]\-3\.1\[\-5\.4, \-0\.9\]0\.0047logiqaforced\_choice65126\.0\[22\.6, 29\.3\]16\.7\[14\.0, 19\.7\]17\.2\[14\.3, 20\.1\]25\.8\[22\.6, 29\.0\]\-0\.5\[\-1\.8, \+0\.8\]0\.644boolqforced\_choice204868\.0\[65\.9, 70\.0\]56\.2\[54\.1, 58\.3\]63\.5\[61\.3, 65\.5\]68\.0\[66\.0, 70\.0\]\-7\.2\[\-9\.5, \-4\.9\]0\.0001Table 23:Prefill–vs\.–Decode estimator swap under a fixed*decode\-only*intervention locus\. All bases are dimension\-matched \(kmatch=126k\_\{\\mathrm\{match\}\}=126\)\. Forced\-choice: warmupW=0W=0and prefix\_mode=auto\.TaskProtocolnnBaselineDecode\-shared \(k=126k\{=\}126\)Prefill\-shared \(k=126k\{=\}126\)Random \(k=126k\{=\}126\)Δ​\(Decode−Prefill\)\\Delta\(\\mathrm\{Decode\{\-\}Prefill\}\)ppgsm8kgeneration131919\.6\[17\.5, 21\.8\]0\.3\[0\.1, 0\.6\]19\.1\[17\.1, 21\.3\]19\.4\[17\.3, 21\.6\]\-18\.8\[\-21\.0, \-16\.8\]0\.0001commonsenseqaforced\_choice122155\.6\[52\.8, 58\.4\]23\.8\[21\.5, 26\.1\]20\.4\[18\.1, 22\.7\]54\.5\[51\.8, 57\.3\]\+3\.4\[\+2\.0, \+4\.8\]0\.0001strategyqaforced\_choice68750\.4\[46\.7, 54\.0\]52\.8\[49\.1, 56\.6\]48\.5\[44\.8, 52\.3\]50\.4\[46\.6, 53\.9\]\+4\.4\[\+0\.9, \+7\.7\]0\.0131aquaforced\_choice25427\.2\[21\.7, 32\.7\]20\.1\[15\.4, 25\.2\]25\.6\[20\.5, 31\.1\]27\.6\[22\.0, 33\.1\]\-5\.5\[\-9\.8, \-1\.6\]0\.0147arc\_challengeforced\_choice117253\.1\[50\.2, 55\.9\]27\.9\[25\.3, 30\.5\]23\.0\[20\.6, 25\.4\]52\.0\[49\.2, 54\.9\]\+4\.9\[\+3\.0, \+6\.8\]0\.0001openbookqaforced\_choice50057\.8\[53\.6, 62\.2\]26\.8\[23\.0, 30\.8\]27\.8\[23\.8, 31\.8\]58\.8\[54\.4, 62\.8\]\-1\.0\[\-3\.4, \+1\.2\]0\.501qascforced\_choice92652\.6\[49\.4, 55\.8\]18\.5\[16\.0, 21\.0\]20\.7\[18\.3, 23\.3\]51\.8\[48\.5, 55\.0\]\-2\.3\[\-4\.3, \-0\.1\]0\.0489logiqaforced\_choice65126\.0\[22\.6, 29\.3\]18\.0\[15\.2, 21\.0\]16\.7\[13\.8, 19\.7\]25\.2\[22\.0, 28\.4\]\+1\.2\[\-0\.3, \+2\.8\]0\.188boolqforced\_choice204868\.0\[65\.9, 70\.0\]50\.8\[48\.6, 52\.9\]62\.6\[60\.4, 64\.6\]71\.2\[69\.3, 73\.2\]\-11\.8\[\-14\.9, \-8\.8\]0\.0001Representative Example: Projection removes pirate style \(sample, seed=3\)Prompt:Please answer the following question:
Explain what climate change means and why it matters\.
Threshold:pirate\_hits≥2\\geq 2;Injection:start\_step=1, first\_n=24;α=45\\alpha=45\.no\_steer\(hits=0, success=0\)Of course, I’d be happy to help\! Climate change refers to the long\-term warming of the planet due to an increase in the average global temperature\. This warming is primarily caused by human activities that release greenhouse gases, such as carbon dioxide and methane, into the atmosphere\. These gases trap heat and cause the Earth’s temperature to rise, leading to a range of consequences\.
Why does climate change matter? Here are some reasons:
\(1\) Rising sea levels … \(2\) Extreme weather events …v\_orig\_a45\(hits=3, success=1\)Of course, matey\! Climate change be the shifts in yer climate, arrr, caused by the warmin’ o’ the planet due to the release o’ greenhouse gases into the atmosphere\. These gases trap heat and cause the Earth’s temperature to rise, which leads to changes in weather patterns, sea levels, and ecosystems\.
Why does it matter, ye ask? Well, me hearty, here be some reasons:
\(1\) Rising sea levels … \(2\) Extreme weather …v\_fixed\_k128\_a45\(hits=0, success=0\)Of course, I’d be happy to help\! Climate change refers to the long\-term warming of the planet due to an increase in the average global temperature …
Why does climate change matter? Here are some reasons:
\(1\) Rising sea levels … \(2\) Extreme weather events …Figure 15:Same prompt, same decoding, different vectors\.vorigv\_\{\\mathrm\{orig\}\}triggers pirate lexicon \(success\), while projecting out the shared PCA span atk=128k\\\!=\\\!128suppresses the style \(failure\), consistent withvvbeing entangled with the shared decode subspace\.Flip Example \(forced\-choice; ex\_id=aqua\-test\-9\)Gold:BIntervention:shared\-removal at layerℓ=10\\ell\{=\}10\(decode\-only\)\.Prompt \(truncated\):Question: A newspaper costs $4 on Sunday \.\.\. how many newspapers does it buy on Monday? Choices: A\)45 B\)15 C\)60 D\)30 E\)75baseline:B\(correct=true, margin=\+1\.08\)
shared\-removed:A\(correct=false, margin=\-3\.30\)*\(flip\)*Figure 16:Flip case study \(forced\-choice\)\.The baseline forced\-choice prediction matches the gold answer\. Removing the shared decode subspace at layerℓ=10\\ell=10during cached decoding flips the predicted choice demonstrating that the shared decode pathway can be*causally necessary*\.Patchback example \(boolq; ex\_id=boolq\-validation\-3\)Dataset: boolq
Gold:B
Prompt\(truncated\):Question: Passage: Open carry is also legal throughout North Carolina\. In the town of Chapel Hill, open carry is restricted to guns of a certain minimum size \.\.\. \\nQuestion do you need a permit to open carry in nc Choices: A\) Yes B\) No Reason step by step\. At the end, write exactly one line: "Final answer: <A/B\>"\.baseline:B\(correct=true, margin=\+5\.19\)
patchback \(W=\{0,1\}\):B\(correct=true, margin=\+5\.19\)
rand subspace:A\(correct=false, margin=\-3\.66\)
time\-shuf donor:B\(correct=true, margin=\+2\.84\)
non\-shared patch:A\(correct=false, margin=\-1\.96\)
Figure 17:Patchback case study \(forced\-choice\)\.Patching back only the removed*shared*component over a narrow decode windowW=\{0,1\}W=\\\{0,1\\\}restores the baseline answer, while energy/dimension\-matched controls do not\.

## Appendix CAdditional Experiments

### C\.1α\\alpha\-Sweep in Patch back Experiments

#### α\\alpha\-sweep validates ablation strength as a continuous knob and transfer donors reduce answer\-copying concerns\.

Table[24](https://arxiv.org/html/2607.20469#A3.T24)shows that increasingα\\alphacontinuously increases flip\-rate and collapses margins, consistently across seeds\. On the same AQuA flip\-set, transfer donors \(same\-task or cross\-task\) achieve rescue rates comparable to self\-donor patching, supporting the interpretation that patchback restores a shared decision channel rather than copying sample\-specific answers\.

Table 24:AQuA flip\-set supplementary experiments\.\(A\)α\\alpha\-sweep on a fixed flip\-set defined atα=1\\alpha\{=\}1\.We report flip\_rate \(still flipped on the flip\-set\), ablated accuracy on the flip\-set, and meanΔ\\Deltam \(ablated vs\. baseline\)\.\(B\) Transfer\-donor patching on the same flip\-set:donors from other examples/tasks yield comparable rescue to self\-donor\.Seed 123 \(n=42n\{=\}42\)Seed 456 \(n=43n\{=\}43\)α\\alphaflip\_rate\(%\)ablt\_acc\(%\)meanΔ\\Deltamflip\_rate\(%\)ablt\_acc\(%\)meanΔ\\Deltam0\.000\.0100\.00\.0000\.0100\.00\.0000\.5028\.671\.4\-0\.31832\.667\.4\-0\.3880\.7571\.428\.6\-1\.47588\.411\.6\-1\.9201\.00100\.00\.0\-3\.695100\.00\.0\-3\.899\(B\) Transfer donor patching on the flip\-set \(rescue% / meanΔ\\Deltam\)SettingSeedSelf\-donorTransfer\-donorSame\-task donors12373\.8 / 3\.3176\.2 / 3\.29Cross\-task donors \(MC, baseline\-correct\)12373\.8 / 3\.3178\.6 / 3\.39Cross\-task donors \(MC, baseline\-correct\)45679\.1 / 3\.5781\.4 / 3\.64

### C\.2Additional Downstream Utility Experiments

#### Appendix C\.2\.1:β\\betaas a continuous*intervention\-strength*knob \(utility vs\. worst\-case stability\)\.

Table[26](https://arxiv.org/html/2607.20469#A3.T26)reports a three\-point sweepβ∈\{0,0\.5,1\}\\beta\\in\\\{0,0\.5,1\\\}under true KV\-cache decoding\. Hereβ\\beta*does not modify the model’s shared workspace*; it only scales how much the*steering direction*is prevented from injecting components along the fixed decode\-time shared basisBB\(estimated once from neutral prompts at the same layer as in our workspace analysis\)\. Intermediateβ=0\.5\\beta\{=\}0\.5yields intermediate behavior across tasks, consistent withvβ=v−β​B​B⊤​vv\_\{\\beta\}=v\-\\beta BB^\{\\top\}vacting as a smooth control on*shared\-subspace interference*\. For example, on SST\-2 template T0, mean increases monotonically0\.0275→0\.0314→0\.03390\.0275\\\!\\rightarrow\\\!0\.0314\\\!\\rightarrow\\\!0\.0339while anti decreases0\.3555→0\.3086→0\.30080\.3555\\\!\\rightarrow\\\!0\.3086\\\!\\rightarrow\\\!0\.3008\. On RTE template T0, mean slightly increases0\.0104→0\.0112→0\.01160\.0104\\\!\\rightarrow\\\!0\.0112\\\!\\rightarrow\\\!0\.0116and anti improves0\.4844→0\.4766→0\.46480\.4844\\\!\\rightarrow\\\!0\.4766\\\!\\rightarrow\\\!0\.4648\. Aggregating per\-template effects, Table[25](https://arxiv.org/html/2607.20469#A3.T25)shows that full projection \(β=1\\beta\{=\}1\) trades small changes in average efficacy \(Δ​μ=−0\.0071\\Delta\\mu=\-0\.0071on BoolQ;−0\.0011\-0\.0011on RTE;\+0\.0055\+0\.0055on SST\-2\) for stability improvements in a task\-dependent way \(e\.g\., fewer worst\-case failures on RTE withΔ\\Deltaanti=worst−0\.0195\{\}\_\{\\text\{worst\}\}=\-0\.0195, and reduced template variability withΔ​σtmpl=−0\.0010\\Delta\\sigma\_\{\\text\{tmpl\}\}=\-0\.0010\)\. Thus,β\\betaprovides an explicit, offline\-computed knob for balancing steering utility against worst\-case template brittleness by attenuating interference with the decode\-time shared pathway \(rather than contradicting its causal relevance\)\.

Table 25:Effect of fully removing the shared component\.Δ\\Deltavalues compare decode\_fixed \(β=1\\beta\{=\}1\) vs decode\_est \(β=0\\beta\{=\}0\)\. We emphasize changes in worst\-case signed behavior \(Δ\\Deltaworst\), and reportΔ\\Deltaantiworst\{\}\_\{\\text\{worst\}\}as a complementary diagnostic\.TaskΔ​μ\\Delta\\muΔ​σtmpl\\Delta\\sigma\_\{\\text\{tmpl\}\}Δ\\DeltaworstΔ\\Deltaantiworst\{\}\_\{\\text\{worst\}\}BoolQ\-0\.0071\+0\.0015\-0\.00900\.0000RTE\-0\.0011\-0\.0010\-0\.0015\-0\.0195SST\-2\+0\.0055\+0\.0008\+0\.0035\+0\.0313
#### Appendix C\.2\.2: Per\-template breakdown pinpoints the origin of worst\-case gains\.

Table[26](https://arxiv.org/html/2607.20469#A3.T26)localizes worst\-case behavior to specific brittle templates, showing that improvements are not driven by averaging artifacts\. On SST\-2, template T2 exhibits a sign flip atβ=0\\beta\{=\}0\(mean=−7×10−4=\-7\\\!\\times\\\!10^\{\-4\}\) that disappears as shared\-overlap is removed, becoming positive atβ=1\\beta\{=\}1\(mean=2\.8×10−3=2\.8\\\!\\times\\\!10^\{\-3\}\), which directly accounts for the improved worst\-case signed behavior in Table[25](https://arxiv.org/html/2607.20469#A3.T25)\(Δ\\Deltaworst=\+0\.0035=\+0\.0035\)\. RTE shows consistent reductions in anti on failure\-prone templates \(e\.g\., T0:0\.4844→0\.46480\.4844\\\!\\rightarrow\\\!0\.4648\) and a modest decrease in template variability \(Table[25](https://arxiv.org/html/2607.20469#A3.T25),Δ​σtmpl=−0\.0010\\Delta\\sigma\_\{\\text\{tmpl\}\}=\-0\.0010\)\. BoolQ illustrates a mild efficacy–stability tradeoff: projection slightly reduces mean and worst \(Table[25](https://arxiv.org/html/2607.20469#A3.T25)\), while anti improves for some templates \(e\.g\., T1:0\.2969→0\.26560\.2969\\\!\\rightarrow\\\!0\.2656\), consistent with task\-dependent interaction between the steering signal and the shared decode\-time pathway\.

Table 26:Per\-template breakdown \(finalλ\\lambda\)\.Per\-template mean correct\-signed margin shift \(mean\) and per\-template failure rate \(anti\) forβ∈\{0,0\.5,1\}\\beta\\in\\\{0,0\.5,1\\\}\.β=0\\beta\{=\}0β=0\.5\\beta\{=\}0\.5β=1\\beta\{=\}1TaskTmpl\.mean↑\\uparrowanti↓\\downarrowmean↑\\uparrowanti↓\\downarrowmean↑\\uparrowanti↓\\downarrowBoolQT00\.04010\.36330\.03720\.35940\.03230\.3633T10\.03930\.29690\.03800\.28120\.03470\.2656T20\.03570\.35940\.03210\.35940\.02670\.3516RTET00\.01040\.48440\.01120\.47660\.01160\.4648T10\.03070\.33200\.03010\.32420\.02810\.3203T20\.01070\.37890\.01010\.38280\.00890\.3750SST\-2T00\.02750\.35550\.03140\.30860\.03390\.3008T10\.00300\.42970\.00630\.44920\.00950\.4648T2\-0\.00070\.43360\.00110\.44920\.00280\.4219Table 27:Candidate calibration \(top pairs\)\.Forced\-choice candidate pairs selected by maximizing baseline forced\-choice accuracy on a balanced subset\. All pairs are single\-token\.TaskCandidate pairBaseline accBoolQYes/No0\.583True/False0\.552RTETrue/False0\.615Yes/No0\.562SST\-2Good/Bad0\.688Yes/No0\.599True/False0\.599
#### Appendix C\.2\.3: Candidate calibration reduces dependence on label tokenization\.

To avoid confounding steering effects with poorly calibrated answer tokens, we choose forced\-choice candidate pairs by maximizing baseline forced\-choice accuracy on a balanced calibration subset\. Table[27](https://arxiv.org/html/2607.20469#A3.T27)lists the selected top pairs \(all single\-token\), which match standard labelings \(e\.g\., Yes/No; True/False; Good/Bad\) and yield reasonable baseline accuracies \(e\.g\., BoolQ: 0\.583 with Yes/No; RTE: 0\.615 with True/False; SST\-2: 0\.688 with Good/Bad\)\. This calibration limits sensitivity to arbitrary token choices and supports interpreting subsequent robustness trends as properties of the steering intervention \(and its overlap withBB\), rather than artifacts of candidate selection\.

### C\.3Additional Downstream Style Metrics and Shared\-Subspace Diagnostics

#### Appendix C\.3\.1: Pirate shareness sanity check, Table[28](https://arxiv.org/html/2607.20469#A3.T28)\.

This table verifies that the projection operator behaves as intended and thatvvis substantially aligned with the decode\-trace PCA span\. Askkincreases from 16 to 128, shareness\(v\)=‖Bk⊤​v‖/‖v‖\(v\)=\\\|B\_\{k\}^\{\\top\}v\\\|/\\\|v\\\|rises from 0\.465 to 0\.545, indicating that a large fraction of the steering direction lies in the “shared” subspace captured by high\-variance decode activations\. In contrast, shareness\(vfixed,k\)\(v\_\{\\mathrm\{fixed\},k\}\)remains≈10−7\\approx 10^\{\-7\}for allkk, confirming thatvfixed,k=v−Bk​\(Bk⊤​v\)v\_\{\\mathrm\{fixed\},k\}=v\-B\_\{k\}\(B\_\{k\}^\{\\top\}v\)is numerically orthogonal to the span up to floating\-point tolerance\. Together, these results establish that the experiment is not merely showing behavioral changes under arbitrary edits: the intervention explicitly removes the component ofvvcontained in a well\-defined shared subspace, and does so accurately\.

Table 28:Appendix: Shareness sanity check\.Askkincreases, the PCA span captures more ofvv\(sharedness rises\), whilevfixed,kv\_\{\\mathrm\{fixed\},k\}is numerically orthogonal to the span \(≈10−7\\approx 10^\{\-7\}\)\.kksharedness\(v\)=‖Bk⊤​v‖‖v‖\(v\)=\\frac\{\\\|B\_\{k\}^\{\\top\}v\\\|\}\{\\\|v\\\|\}sharedness\(vfixed,k\)=‖Bk⊤​vfixed,k‖‖vfixed,k‖\(v\_\{\\mathrm\{fixed\},k\}\)=\\frac\{\\\|B\_\{k\}^\{\\top\}v\_\{\\mathrm\{fixed\},k\}\\\|\}\{\\\|v\_\{\\mathrm\{fixed\},k\}\\\|\}160\.46519\.9×10−89\.9\\times 10^\{\-8\}320\.48641\.0×10−71\.0\\times 10^\{\-7\}640\.51661\.2×10−71\.2\\times 10^\{\-7\}1280\.54551\.4×10−71\.4\\times 10^\{\-7\}
#### Appendix C\.3\.2: Pirate\-hit intensity, Table[29](https://arxiv.org/html/2607.20469#A3.T29)\.

While the main table reports binary success at a threshold ofpirate\_hits≥2\\geq 2, Appendix Table[29](https://arxiv.org/html/2607.20469#A3.T29)provides a more graded view of the same phenomenon by summarizing the mean and worst\-casepirate\_hitsacross templates\. The hit statistics reinforce the tradeoff observed in success rates: moderate projection \(e\.g\.,k=16k=16–64\) increases the intensity of lexical pirate markers relative tovorigv\_\{\\mathrm\{orig\}\}under both greedy and sampling, whereas stronger projection \(k=128\) substantially reduces hit intensity, particularly under greedy decoding \(near\-zero worst\-case\)\. This corroborates an entanglement interpretation: removing a small part of the shared space can “denoise” the steering direction so pirate tokens appear more frequently when they appear at all, but removing a larger shared span begins to excise style\-relevant components and suppresses lexical realization, with the remaining signal becoming more brittle and decoding\-dependent\.

Table 29:Appendix: Pirate\-hit intensity\.Mean±\\pmstd and worst\-case \(minimum\) across templates forpirate\_hits\. This complements Table[33](https://arxiv.org/html/2607.20469#A3.T33)by showing how strongly the lexicon is triggered when it triggers at all\.MethodkkGreedy hitsSample hits \(seeds 1–3\)no\_steer–0\.000±0\.000​\(0\.000\)0\.000\\pm 0\.000\\ \(0\.000\)0\.000±0\.000​\(0\.000\)0\.000\\pm 0\.000\\ \(0\.000\)rand0\_a45–0\.000±0\.000​\(0\.000\)0\.000\\pm 0\.000\\ \(0\.000\)0\.000±0\.000​\(0\.000\)0\.000\\pm 0\.000\\ \(0\.000\)v\_orig\_a45–0\.090±0\.136​\(0\.000\)0\.090\\pm 0\.136\\ \(0\.000\)0\.127±0\.063​\(0\.050\)0\.127\\pm 0\.063\\ \(0\.050\)v\_fixed\_k16\_a45160\.400±0\.130​\(0\.150\)0\.400\\pm 0\.130\\ \(0\.150\)0\.360±0\.034​\(0\.317\)0\.360\\pm 0\.034\\ \(0\.317\)v\_fixed\_k32\_a45320\.210±0\.116​\(0\.050\)0\.210\\pm 0\.116\\ \(0\.050\)0\.340±0\.064​\(0\.267\)0\.340\\pm 0\.064\\ \(0\.267\)v\_fixed\_k64\_a45640\.250±0\.145​\(0\.050\)0\.250\\pm 0\.145\\ \(0\.050\)0\.413±0\.062​\(0\.317\)0\.413\\pm 0\.062\\ \(0\.317\)v\_fixed\_k128\_a451280\.040±0\.037​\(0\.000\)0\.040\\pm 0\.037\\ \(0\.000\)0\.213±0\.097​\(0\.083\)0\.213\\pm 0\.097\\ \(0\.083\)Table 30:All\-tasks shared basis: decode\-only ablation \(generation\)\.Shared\(full\)removes the decode\-estimated shared subspace;Rand\(full\)is a dimension/energy\-matched non\-shared control \(see §[2\.4](https://arxiv.org/html/2607.20469#S2.SS4)\)\. Entries report mean accuracy with 95% CIs;Δ\\DeltaisShared\(full\)−\-Baseline with paired testpp\. A full sweep across models and layers is provided in the Appendix\.All\-tasks basis \(Mode: All; Generation\)TaskBaselineShared \(full\)Rand \(full\)Δ\\Delta\(Shared−\-Baseline\)ARC\-C26\.2\[21\.1, 31\.6\]7\.8\[4\.7, 11\.3\]21\.5\[16\.4, 26\.6\]\-18\.4\[\-23\.8, \-12\.9\],p<0\.001p<0\.001BoolQ31\.6\[26\.2, 37\.5\]18\.0\[13\.3, 22\.7\]30\.9\[25\.4, 36\.3\]\-13\.7\[\-19\.9, \-7\.8\],p<0\.001p<0\.001CSQA16\.8\[12\.5, 21\.5\]8\.2\[5\.1, 11\.7\]19\.1\[14\.5, 24\.2\]\-8\.6\[\-13\.7, \-3\.5\],p=0\.001p=0\.001GSM8K16\.4\[12\.1, 21\.1\]0\.0\[0\.0, 0\.0\]15\.6\[11\.3, 20\.3\]\-16\.4\[\-21\.1, \-12\.1\],p<0\.001p<0\.001LogiQA13\.3\[9\.4, 17\.6\]5\.9\[3\.1, 9\.0\]8\.2\[5\.1, 11\.7\]\-7\.4\[\-12\.1, \-2\.7\],p=0\.002p=0\.002OBQA25\.4\[20\.3, 30\.5\]6\.2\[3\.5, 9\.4\]27\.0\[21\.5, 32\.4\]\-19\.1\[\-24\.6, \-13\.7\],p<0\.001p<0\.001PIQA39\.8\[34\.0, 46\.1\]19\.1\[14\.5, 24\.2\]41\.4\[35\.5, 47\.7\]\-20\.7\[\-27\.3, \-14\.1\],p<0\.001p<0\.001QASC18\.0\[13\.7, 22\.7\]6\.2\[3\.5, 9\.4\]19\.5\[14\.8, 24\.6\]\-11\.7\[\-17\.2, \-6\.2\],p<0\.001p<0\.001StratQA45\.7\[39\.5, 52\.0\]10\.9\[7\.4, 14\.8\]43\.4\[37\.5, 49\.2\]\-34\.8\[\-41\.4, \-28\.1\],p<0\.001p<0\.001Table 31:LOTO results \(two panels\)\.Cells report mean \(top\) and 95% CI \(bottom\)\.pptests Shared\(full\) vs\. Baseline, layerℓ=10\\ell=10\. A full sweep across models and layers is provided in the Appendix\.Leave\-one\-task\-out \(LOTO\) resultsPart A: Subspace Intervention Experiment \(Generation\)TasknnBaselineShared\(full\)Rand\(full\)pp\-valueAQuA25417\.7\[13\.4, 22\.4\]15\.4\[11\.0, 19\.7\]19\.7\[15\.0, 24\.8\]0\.483BoolQ204832\.2\[30\.2, 34\.2\]21\.1\[19\.3, 22\.9\]32\.2\[30\.1, 34\.2\]<0\.001<0\.001CSQA122121\.3\[19\.0, 23\.6\]10\.9\[9\.2, 12\.6\]21\.5\[19\.2, 23\.8\]<0\.001<0\.001GSM8K131915\.7\[13\.8, 17\.7\]0\.1\[0\.0, 0\.2\]13\.3\[11\.6, 15\.2\]<0\.001<0\.001OBQA50026\.2\[22\.4, 30\.0\]13\.2\[10\.2, 16\.4\]25\.4\[21\.6, 29\.4\]<0\.001<0\.001PIQA183837\.4\[35\.3, 39\.7\]31\.4\[29\.3, 33\.6\]36\.7\[34\.5, 38\.9\]<0\.001<0\.001QASC92620\.2\[17\.6, 22\.9\]8\.9\[7\.1, 10\.7\]20\.4\[17\.8, 23\.0\]<0\.001<0\.001StratQA68748\.2\[44\.4, 51\.7\]20\.2\[17\.3, 23\.3\]47\.3\[43\.7, 50\.9\]<0\.001<0\.001Avg \(8 tasks\)—27\.415\.127\.1—Part B: Subspace Intervention Experiment \(Forced\-Choice\)TasknnBaselineShared\(full\)Rand\(full\)pp\-valueAQuA25424\.0\[18\.9, 29\.5\]17\.3\[12\.6, 22\.0\]22\.0\[16\.9, 27\.2\]0\.0547ARC\-C117250\.9\[48\.1, 53\.8\]40\.4\[37\.5, 43\.2\]50\.6\[47\.8, 53\.4\]0\.0001CSQA122154\.1\[51\.3, 56\.8\]50\.0\[47\.3, 52\.9\]54\.5\[51\.6, 57\.2\]0\.0043GSM8K13194\.9\[3\.8, 6\.0\]2\.3\[1\.5, 3\.1\]4\.7\[3\.6, 5\.8\]0\.0001LogiQA65132\.7\[29\.0, 36\.4\]26\.9\[23\.5, 30\.3\]33\.2\[29\.6, 36\.9\]0\.0147OBQA50050\.2\[45\.8, 54\.6\]41\.6\[37\.4, 45\.8\]52\.4\[48\.0, 56\.8\]0\.0005QASC92648\.5\[45\.2, 51\.7\]40\.7\[37\.6, 43\.8\]49\.0\[45\.8, 52\.2\]0\.0001StratQA68755\.5\[51\.7, 59\.1\]52\.3\[48\.5, 55\.9\]56\.3\[52\.5, 60\.0\]0\.0182Table 32:Staged Generation Evaluation\.\(meta\-llama/Llama\-2\-7b\-chat\-hf, fp32; decode\-only; layer=10;τ=0\.001\\tau\{=\}0\.001; mode=all\)\. Values are means over 8 tasks\.*Full*denotes standard end\-to\-end generation under the intervention;*Staged*constrains the final\-answer format to reduce format/extraction confounds\. Acc/Extr/EOS are in %, and Len is in tokens\.BaselineShared removalRandom controlMetricValueFullStagedFullStagedAcc \(%\)27\.415\.116\.427\.126\.8Extr \(%\)70\.551\.552\.270\.070\.1EOS \(%\)90\.845\.351\.092\.491\.9Len \(tok\.\)92\.5144\.0139\.286\.787\.1Table 33:Shared\-space sweep \+ steering efficacy \(Llama\-2\-7B\-Chat\)\.We estimate a pirate style directionvvat layer 28 \(KV\-aligned decode,vdecode\_steps=16v\_\{\\text\{decode\\\_steps\}\}=16,vn=16v\_\{n\}=16,α=45\\alpha=45, inject window: start\_step=1, first\_n=24\) and construct a PCA basisBkB\_\{k\}from decode hidden\-state traces of no\-steer generations\.*Sharedness*is∥Bk⊤​v∥/∥v∥\\lVert B\_\{k\}^\{\\top\}v\\rVert/\\lVert v\\rVert;vfixed,k=v−Bk​\(Bk⊤​v\)v\_\{\\mathrm\{fixed\},k\}=v\-B\_\{k\}\(B\_\{k\}^\{\\top\}v\)has∥Bk⊤​vfixed,k∥/∥vfixed,k∥≈0\\lVert B\_\{k\}^\{\\top\}v\_\{\\mathrm\{fixed\},k\}\\rVert/\\lVert v\_\{\\mathrm\{fixed\},k\}\\rVert\\approx 0for allkk\(Appendix Table[28](https://arxiv.org/html/2607.20469#A3.T28)\)\. Success is𝟙​\[pirate\_hits≥2\]\\mathbb\{1\}\[\\texttt\{pirate\\\_hits\}\\geq 2\]\. Numbers areper\-templatemean±\\pmstd, with worst\-case \(minimum\) across templates in parentheses\.MethodkkSharedness\(vv\)GreedysuccessSample success\(seeds 1–3\)No steer––0\.000±0\.0000\.000\\pm 0\.000\( 0\.000 \)0\.000±0\.0000\.000\\pm 0\.000\( 0\.000 \)Random direction––0\.000±0\.0000\.000\\pm 0\.000\( 0\.000 \)0\.000±0\.0000\.000\\pm 0\.000\( 0\.000 \)vorigv\_\{\\mathrm\{orig\}\}––0\.040±0\.0580\.040\\pm 0\.058\( 0\.000 \)0\.033±0\.0240\.033\\pm 0\.024\( 0\.000 \)vfixedv\_\{\\mathrm\{fixed\}\}160\.4650\.120±0\.060\\mathbf\{0\.120\}\\pm 0\.060\( 0\.000 \)0\.100±0\.0150\.100\\pm 0\.015\( 0\.083 \)vfixedv\_\{\\mathrm\{fixed\}\}320\.4860\.050±0\.0320\.050\\pm 0\.032\( 0\.000 \)0\.097±0\.0290\.097\\pm 0\.029\( 0\.067 \)vfixedv\_\{\\mathrm\{fixed\}\}640\.5170\.090±0\.0660\.090\\pm 0\.066\( 0\.000 \)0\.127±0\.017\\mathbf\{0\.127\}\\pm 0\.017\( 0\.100 \)vfixedv\_\{\\mathrm\{fixed\}\}1280\.5450\.000±0\.0000\.000\\pm 0\.000\( 0\.000 \)0\.063±0\.0270\.063\\pm 0\.027\( 0\.033 \)

### C\.4Sharedness under Diverse Scales

We extend our evaluations beyond the original 7B scale to verify the generality of our findings\. Rather than performing a comprehensive scaling\-law study, these experiments test whether our core qualitative insights hold across smaller and larger models\. Crucially, we confirm that across scales, the decode\-shared subspace remains causally impactful, while prefill\-based proxies consistently fail to predict decode\-time behavior\.

Table 34:H1 beyond the original 7B setting\.All runs useτ=10−3\\tau=10^\{\-3\}andmshared=allm\_\{\\rm shared\}=\\mathrm\{all\}\. Ratio is\|S\|/cross​\_​dim\|S\|/\\mathrm\{cross\\\_dim\}\.ModelLayerCross\-dim\|S\|\|S\|Ratio\(pperm,pscram\)\(p\_\{\\rm perm\},p\_\{\\rm scram\}\)Llama\-3\.2\-3B2717921297\.20%\(0\.015, 0\.048\)Llama\-2\-13B102427883\.63%\(0\.015, 0\.048\)Llama\-2\-13B2827461324\.81%\(0\.015, 0\.048\)Llama\-2\-13B392631873\.31%\(0\.015, 0\.048\)Llama\-2\-70B323522210\.60%\(0\.015, 0\.048\)Llama\-2\-70B48401820\.05%\(0\.015, 0\.048\)Llama\-2\-70B564096130\.32%\(0\.015, 0\.048\)Table 35:H2 causal degradation beyond 7B\.Entries areΔ​\(shared−baseline\)\\Delta\(\\mathrm\{shared\}\-\\mathrm\{baseline\}\)in accuracy points under decode\-time shared\-subspace removal\. Rows use the aligned four\-task multiple\-choice slice\.∗denotesp<0\.05p<0\.05\.ModelLayerCSQAOBQAQASCARC\-CMean / Sig\.Llama\-3\.2\-3B27−25\.0∗\-25\.0^\{\\ast\}−10\.9∗\-10\.9^\{\\ast\}−42\.2∗\-42\.2^\{\\ast\}−9\.4\-9\.4−21\.9\-21\.9/ 3 of 4Llama\-2\-13B10−10\.2∗\-10\.2^\{\\ast\}−11\.7∗\-11\.7^\{\\ast\}−10\.2∗\-10\.2^\{\\ast\}−6\.2∗\-6\.2^\{\\ast\}−9\.6\-9\.6/ 4 of 4Llama\-2\-13B28−13\.3∗\-13\.3^\{\\ast\}−10\.2∗\-10\.2^\{\\ast\}−18\.0∗\-18\.0^\{\\ast\}−6\.2\-6\.2−11\.9\-11\.9/ 3 of 4Llama\-2\-13B39−21\.9∗\-21\.9^\{\\ast\}−2\.3\-2\.3−22\.7∗\-22\.7^\{\\ast\}\+0\.0\+0\.0−11\.7\-11\.7/ 2 of 4Llama\-2\-70B32−26\.6∗\-26\.6^\{\\ast\}−14\.1∗\-14\.1^\{\\ast\}−7\.8\-7\.8−12\.5∗\-12\.5^\{\\ast\}−15\.2\-15\.2/ 3 of 4Llama\-2\-70B48−28\.1∗\-28\.1^\{\\ast\}−21\.9∗\-21\.9^\{\\ast\}−7\.8\-7\.8−23\.4∗\-23\.4^\{\\ast\}−20\.3\-20\.3/ 3 of 4Llama\-2\-70B56−21\.9∗\-21\.9^\{\\ast\}−15\.6∗\-15\.6^\{\\ast\}−17\.2∗\-17\.2^\{\\ast\}−26\.6∗\-26\.6^\{\\ast\}−20\.3\-20\.3/ 4 of 4Table 36:H3 prefill/decode mismatch beyond 7B\.Accuracy is averaged over CSQA, OBQA, QASC, and ARC\-C\. Parentheses report change from baseline\.SettingBaselineDecode\-est\.Prefill\-est\.RandomAngle3B L2769\.246\.3 \(−22\.9\-22\.9\)64\.3 \(−5\.0\-5\.0\)69\.2 \(0\.00\.0\)72\.1∘72\.1^\{\\circ\}13B L1061\.729\.7 \(−32\.0\-32\.0\)62\.1 \(\+0\.4\+0\.4\)61\.3 \(−0\.4\-0\.4\)81\.2∘81\.2^\{\\circ\}70B L5677\.042\.2 \(−34\.8\-34\.8\)75\.8 \(−1\.2\-1\.2\)76\.2 \(−0\.8\-0\.8\)81\.9∘81\.9^\{\\circ\}Across 3B, 13B, and 70B models, DecodeShare recovers compact but statistically significant shared decode\-time subspaces \(Table[34](https://arxiv.org/html/2607.20469#A3.T34)\)\. Decode\-time shared\-subspace removal remains harmful on the aligned four\-task slice, including a strong Llama\-3\.2\-3B layer\-27 result and a clean Llama\-2\-70B layer\-56 result where all four tasks degrade significantly \(Table[35](https://arxiv.org/html/2607.20469#A3.T35)\)\. Finally, the prefill/decode mismatch persists across scale: decode\-estimated bases cause substantially larger decode\-time drops than prefill\-estimated or random bases under the same intervention protocol \(Table[36](https://arxiv.org/html/2607.20469#A3.T36)\)\. These results support the limited claim that the phenomenon is not confined to 7B\-class models, while leaving systematic scaling trends to future work\.

### C\.5Policy Sensitivity and Computational Overhead

#### Policy sensitivity\.

DecodeShare is policy\-aligned rather than policy\-invariant: the shared subspace is estimated from the decode\-time state distribution induced by the serving\-time decoding policy\. The main experiments use greedy decoding because it provides the cleanest controlled setting for causal tests\. To test whether the existence of shared structure is an artifact of greedy decoding, we additionally estimate shared bases under sampling policies with temperaturesT∈\{0\.7,1\.0,1\.3\}T\\in\\\{0\.7,1\.0,1\.3\\\}on Llama\-2\-7B\-Chat\.

Table[37](https://arxiv.org/html/2607.20469#A3.T37)summarizes the shared ratio and the mean principal angle to the corresponding greedy\-estimated basis\. Across all tested layers and temperatures, the H1 sharedness test remains significant \(p<0\.05p<0\.05\)\. The learned basis rotates substantially relative to greedy decoding, as indicated by the large principal angles, but the shared set remains compact and statistically distinguishable from the null\. Thus, changing the decoding policy affects the orientation of the estimated shared workspace, but does not remove the existence of a decode\-time shared structure\.

Table 37:Policy sensitivity under sampling on Llama\-2\-7B\-Chat\. Each cell reports shared ratio and mean principal angle relative to the same\-layer greedy\-estimated basis\. All 9 settings pass the H1 sharedness test \(p<0\.05p<0\.05\)\.LayerT=0\.7T=0\.7T=1\.0T=1\.0T=1\.3T=1\.3100\.85%, 56\.82∘2\.02%, 71\.43∘4\.57%, 64\.74∘181\.93%, 60\.38∘2\.74%, 66\.47∘1\.49%, 66\.26∘280\.88%, 67\.42∘0\.56%, 59\.50∘0\.61%, 58\.57∘
#### Computational overhead\.

DecodeShare consists of a one\-time offline basis\-estimation stage and a lightweight online projection stage\. The offline cost is dominated by collecting decode\-time activations and computing the shared basis\. On an H200 GPU with Llama\-2\-7B\-Chat at layerℓ=10\\ell=10, the full DecodeShare pipeline over five tasks takes 296\.5 seconds, compared with 48\.5 seconds for the unmodified model run; basis estimation accounts for 87\.6% of the total pipeline runtime\.

At deployment time, the additional cost is only the projection onto the estimated shared basis at the edited layer\. This projection scales asO​\(d​\|Sℓ\|\)O\(d\|S\_\{\\ell\}\|\)per edited decode state and does not modify the KV cache\. In our measurement, the online intervention changes throughput from 102\.95 tokens/s to 81\.10 tokens/s, corresponding to a 21\.2% throughput overhead, with only 1\.16 MiB additional memory\. These measurements indicate that the main cost is offline estimation, while the online cost is modest for a compact shared core\.

### C\.6Step\-Localized Ablation Across Decode Trajectories

The main causal tests remove the shared subspace throughout the decode trajectory\. To check whether the effect is merely a final\-token readout artifact, we run a step\-localized ablation on extended GSM8K generation trajectories with Llama\-2\-7B\-Chat\. For each generated trajectory, we partition decode steps into three contiguous windows: early, middle, and late thirds\. We then apply the same shared\-subspace removal only within one window, leaving the rest of the trajectory unchanged\.

Table[38](https://arxiv.org/html/2607.20469#A3.T38)shows that all three localized interventions reduce accuracy relative to the unmodified baseline\. The middle\-window intervention is the most damaging, while early\- and late\-window interventions also cause substantial degradation\. Full\-trajectory ablation collapses accuracy to 0\.0%\. This pattern suggests that the shared decode\-time subspace is used throughout the reasoning trajectory, rather than acting only as a single final\-token answer switch\.

Table 38:Step\-localized shared\-subspace ablation on GSM8K extended generation trajectories with Llama\-2\-7B\-Chat\. We ablate the shared subspace only within one third of the generated decode steps\. Accuracy is reported in percent\.Intervention windowAccuracy \(%\)No ablation25\.0Early third only14\.1Middle third only6\.2Late third only12\.5Full trajectory0\.0We interpret this result conservatively\. It does not imply that the shared subspace has the same functional role at every decode step\. Rather, it shows that the causal effect is distributed across the decode trajectory and is not confined to the final answer token\.

### C\.7Readout Remapping and Verbalizer Controls

A possible concern is that shared\-subspace removal might primarily disrupt a narrow answer\-letter readout rather than a broader decode\-time decision component\. The main paper already addresses this through theQoutQ\_\{\\mathrm\{out\}\}/QcoreQ\_\{\\mathrm\{core\}\}split: ablating the residual core reproduces most of the full shared\-subspace effect, whereas ablating the small readout slice is nearly inert\. Here we add a complementary verbalizer control\.

We hold the same layer\-10 decode\-shared basis fixed and change only the forced\-choice readout protocol, without re\-estimating the basis\. Under the original answer\-label readout, the mean shared\-ablation drop is−11\.7\-11\.7points relative to baseline, while matched\-energy controls remain close to zero\. Under a remappedoption\_labelreadout, e\.g\., “Final answer: Option A/B/…”, the drop remains−10\.9\-10\.9points, preserving 93% of the original effect\. Under a strongeroption\_textreadout, where choices are scored using the full option text rather than a short label token, the effect remains negative at−7\.8\-7\.8points, although the magnitude attenuates as expected for a noisier multi\-token verbalizer\.

Table 39:Readout\-remap control with the same fixed layer\-10 decode\-shared basis\. More negativeΔ\\Deltaindicates stronger causal degradation from shared\-subspace removal\. Matched\-energy controls remain near zero under each readout\.Readout protocolWhat changesMeanΔ\\DeltaRetained effectControlslabelOriginal answer\-letter readout−11\.7\-11\.7100%near 0option\_labelRemapped option label−10\.9\-10\.993%near 0option\_textFull option\-text readout−7\.8\-7\.867%near 0These results argue against the trivial explanation thatDecodeShareonly disrupts the original answer\-letter surface form\. We do not claim strict readout invariance under every possible verbalizer\. Instead, the conservative conclusion is that the dominant causal effect is largely preserved under a simple label remapping and remains directionally consistent under a stronger full\-text readout, while matched controls stay near baseline\.

## Appendix DDiscussion\.

### D\.1Broader positioning and additional related work

#### Broader positioning\.

Our results sit at the intersection of \(i\) mechanistic interpretability, which aims to reverse\-engineer internal computations and validate hypotheses with interventions, and \(ii\) systems/efficiency work that treats KV cache management as the defining constraint of modern inference\.\(Bereska and Gavves,[2024](https://arxiv.org/html/2607.20469#bib.bib52); Elhageet al\.,[2021](https://arxiv.org/html/2607.20469#bib.bib12); Olssonet al\.,[2022](https://arxiv.org/html/2607.20469#bib.bib13); Kwonet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib40); Xiaoet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib42); Shaoet al\.,[2026](https://arxiv.org/html/2607.20469#bib.bib61); Wuet al\.,[2026](https://arxiv.org/html/2607.20469#bib.bib60)\)From the interpretability perspective, circuit\-style analyses and scalable discovery methods \(e\.g\., pruning\) motivate studying reusable structure across tasks, aligning with our focus on cross\-task shared subspaces\.\(Merulloet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib16); Bhaskaret al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib18)\)

#### Mechanistic interpretability and shared structure\.

Representation similarity tools \(SVCCA/CKA\) have long been used to compare internal representations across networks and training regimes, and recent hypotheses argue that important structure may concentrate in shared low\-dimensional subspaces\.\(Raghuet al\.,[2017](https://arxiv.org/html/2607.20469#bib.bib11); Kornblithet al\.,[2019](https://arxiv.org/html/2607.20469#bib.bib10); Kaushiket al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib17); Wanget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib7)\)KV\-cache compression/eviction beyond streaming\.Beyond streaming, a growing literature compresses or restructures KV caches \(including low\-rank and reconstruction\-based schemes\), closely related to our emphasis on decode\-time low\-dimensional structure and regime mismatch\.\(Saxenaet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib3); Liet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib45); Caiet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib46); Kimet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib47); Changet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib48); Khalafet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib4)\)Steering directions and reliability\.Steering vectors and representation engineering methods \(including activation addition and direction\-based control\) provide complementary control mechanisms; their reported brittleness and reliability analyses motivate our regime\-specific, decision\-level evaluations and null controls\.\(Turneret al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib19); Rimskyet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib20); Konenet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib21); Toddet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib29); Arditiet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib28); Tanet al\.,[2024](https://arxiv.org/html/2607.20469#bib.bib26); Denget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib27); Belitskyet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib30)\)

#### Differences from closely related subspace work\.

Arturi et al\. analyze*parameter\-level*shared subspaces in weight updates across tasks and connect this geometry to emergent misalignment\(Arturiet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib53)\); in contrast, we study an*activation\-level*shared workspace that appears specifically in*decode\-time*states under true KV\-cached decoding, and we validate it with decode\-only causal interventions \(H2\) and estimator–distribution mismatch tests \(H3\)\. MSRS proposes*multi\-subspace*representation steering to compose attribute\-aligned behaviors\(Jianget al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib54)\); our focus is not to design a stronger steering method, but to diagnose a regime\-specific failure mode \(shared\-workspace interference under KV\-cached decoding\) and provide a simple projection\-based repair knob with matched controls\. Son et al\. study*semantic convergence across models/scales*by comparing learned representations across different LLMs\(Sonet al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib55)\); we instead ask whether*cross\-task*shared structure exists within a fixed model at decision time, and whether it is causally necessary for decode\-time decisions\. Zhang and Zhou address interference in*adaptation/merging*by enforcing orthogonal*parameter*subspaces \(e\.g\., for LoRA\)\(Zhang and Zhou,[2025](https://arxiv.org/html/2607.20469#bib.bib56)\); we do not constrain training, but identify and intervene on a decode\-time activation subspace at inference\. Xie et al\. remove nuisance low\-rank subspaces to obtain language\-agnostic multilingual representations\(Xieet al\.,[2022](https://arxiv.org/html/2607.20469#bib.bib57)\); similarly, we use projection operations, but our target is a task\-shared decode\-time workspace and we establish its relevance via matched nulls, budget\-matched controls, and patchback sufficiency tests\. Falissard et al\. learn prefix\-parameter subspaces to improve generalization during fine\-tuning\(Falissardet al\.,[2023](https://arxiv.org/html/2607.20469#bib.bib58)\); we do not learn subspaces, but estimate them from activations and show that prefill\-estimated bases often fail to transfer to decode\-time causal effects \(H3\)\. Finally, Whitaker et al\. study identification of shared semantics via contrastive latent alignment across modalities\(Whitakeret al\.,[2025](https://arxiv.org/html/2607.20469#bib.bib59)\); while our setting is unimodal and task\-based, we share the goal of isolating compact shared structure, and we complement geometric evidence with decode\-time causal validation\.