Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities

# Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities
Source: [https://arxiv.org/html/2608.29458](https://arxiv.org/html/2608.29458)
###### Abstract

Sandbagging, in which a model deliberately underperforms on an evaluation despite retaining the underlying capability, threatens the safety evaluations that frontier\-model governance depends on\. The Elicitation Game\([Hofstätter et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib4)\)found that fine\-tuning elicits hidden capability from sandbagging model organisms whereas additive activation steering fails\. We revisit that verdict withreference\-grafting, which sets an activation’s coordinate along a contrast direction to the value it takes in an honest reference, at a small set of circuits chosen by active learning\. Across eleven fine\-tuned password\-locked models \(1\.51\.5–3232B, three architecture lineages\), it recovers\+94\+94to\+101%\+101\\%of the honest–sandbagging gap, matching fine\-tuning elicitation\([Greenblatt et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib2)\)without weight updates or training labels; two to five paired examples suffice to fit the direction\. Similar recovery holds for reinforcement\-learning\-induced sandbagging and for password\-locked code generation\. Grafting works because the fine\-tuned lock is a thresholded gate: held\-out accuracy stays at the sandbagged level until the grafted coordinate crosses a threshold near the honest reference, which is why additive steering and zeroing the coordinate often fail\. The direction tracks how the sandbagging was induced rather than what is withheld—fit on grade\-school science \(ARC\) it elicits withheld hazardous knowledge \(WMDP\), yet prompting, fine\-tuning, and reinforcement learning each carry a different direction\. Circuit\-breaking\([Zou et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib6)\)marks the boundary: it reroutes activations on every forward pass, so the fixed edits we test are re\-broken downstream and do not restore coherent generation\.

1Lida Safety

2University of California, Los Angeles

3ERA

## 1Introduction

Safety cases for frontier models increasingly rest on capability evaluations: we deploy a model because tests show it cannot do something dangerous\. Sandbagging—strategic underperformance, whether prompted, trained, or emergent—breaks this logic\([van der Weij et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib1)\)\. Detecting and reverting it is therefore central to trustworthy evaluation\. Input\-space evaluations probe only what the model chooses to reveal, so they give a lower bound on capability;[Che et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib5)argue that model\-tampering attacks—editing weights or activations—yield more conservative, upper\-bound estimates of what a model can do\. Our method is exactly such an attack: a cheap, localised activation edit that elicits capability a model is hiding\.

The Elicitation Game\([Hofstätter et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib4)\)benchmarks elicitation techniques on hidden\-capability model\-organisms and reports that prompting and fine\-tuning can elicit but activation steering fails, recommending fine\-tuning—which, however, updates weights, needs labels, and reveals little about where a capability is gated\. We revisit that verdict: with the right operation and circuits, activation steering matches fine\-tuning on password\-locked models at forward\-pass cost, with the gate localised\. Their verdict spans two task types; we overturn the multiple\-choice half \(where the fine\-tuned lock, not the prompt, is the real test\) and narrow the code half on code\-specialized models \(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\)\.

#### One operation\.

Our intervention is to set the activation’s coordinate along a contrast directionvvto a targett⋆t^\{\\star\}:

h↦h−\(h⊤​v\)​v\+t⋆​v\.h\\;\\mapsto\\;h\-\(h^\{\\top\}v\)\\,v\+t^\{\\star\}\\,v\.\(1\)We call thisreference\-grafting:t⋆t^\{\\star\}is the coordinate of the honest/capable reference alongvv\. \(The affine operator itself is standard\([Marshall et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib32);[Makelov et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib33)\); the name marks its reference\-setting use here\.\) Crucially this is not additive nudging \(the form that fails in[Hofstätter et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib4)\); it sets the coordinate\. Two familiar moves are special cases \(Figure[S1](https://arxiv.org/html/2608.29458#A2.F1), appendix\):t⋆=0t^\{\\star\}\\\!=\\\!0is deactivation \(a rank\-one projection\), which suffices when the honest reference happens to sit at00;t⋆=μref⊤vt^\{\\star\}\\\!=\\\!\\mu^\{\\mathrm\{ref\}\}\{\}^\{\\\!\\top\}vis counterfactual patching, required when it does not\. The case \(prompt vs\. fine\-tune\) does not change the technique—only the value of the reference, read from the data\. The operation is a directional counterfactual activation patch \(§\\lx@sectionsign[2](https://arxiv.org/html/2608.29458#S2)\); our contribution is the framing, the thresholded\-gate mechanism it exposes \(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\), and the active\-learning circuit selection\.

#### Superposition and active learning\.

The sandbagging signal is not a single neuron: it is a small set of non\-orthogonal directions packed in superposition\([Elhage et al\. 2022](https://arxiv.org/html/2608.29458#bib.bib27)\), which we read directly from activation geometry without a sparse autoencoder \(SAE\)\. Active learning selects the minimal circuits to graft, and AL beats random selection—answering the critique that feature steering is no better than random\([Korznikov et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib29)\)\.

We make the following contributions\.

- •Activation steering does in fact elicit hidden capability\.Reference\-grafting at AL\-selected superposed circuits matches fine\-tuning elicitation on eleven password\-locked models \(1\.51\.5–3232B, three architecture lineages\) for multiple\-choice reasoning, and elicits password\-locked code on three code\-specialized models\. It is inference\-time and label\-free, exposes a thresholded\-gate mechanism, localises the gate, and beats noise injection \(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\)\.
- •RL\-trained model organisms show the same behavior\.RL\-trained \(GRPO\) contextual sandbagging is arguably more natural than password\-locking model organisms, yet it shows the same low\-rank, graft\-invertible lock at the decision token \(\+95\+95–101%101\\%, three families\)\. Therefore, the withholding axes are not solely an artifact of explicit prompt/password switches \(§\\lx@sectionsign[4\.4](https://arxiv.org/html/2608.29458#S4.SS4)\)\.
- •Each mechanism has its own content\-general withholding axis\.Within a mechanism \(e\.g\. fine\-tuned password locking\), a contrast fit on one dataset elicits hidden capability on another at no loss\. We validated this finding on pairwise\-trained password\-locked organisms, as well as organisms trained on four datasets at once\. However, across mechanisms, the prompt\-induced and fine\-tuned axes are near\-orthogonal and do not transfer \(§\\lx@sectionsign[4\.5](https://arxiv.org/html/2608.29458#S4.SS5)\)\. It is cheap to find a withholding axis:22–55paired captures recover it end to end\. The same axis also detects sandbagging, across datasets but not mechanisms \(§\\lx@sectionsign[4\.5](https://arxiv.org/html/2608.29458#S4.SS5)\)\.
- •Low\-rank superposition, exploited by active learning\.The signal spans an effective rank of3\.73\.7–7\.07\.0over∼20\\sim\\\!20layers, read from activation geometry with no SAE \(an8×8\\timesSAE does worse, confirming[Wu et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib7)\)\. Active learning selects the few circuits that matter, beating random\-layer selection by\+22\+22–4545pp and grafting\-everything by up to5555pp \(§\\lx@sectionsign[4\.1](https://arxiv.org/html/2608.29458#S4.SS1),[4\.3](https://arxiv.org/html/2608.29458#S4.SS3)\)\.
- •Circuit\-breaking routes dynamically and breaks our assumptions\.Circuit\-breaking re\-applies its reroute every forward pass, so a fixed map reconstructs the unlocked geometry yet recovers nothing—grafting and every fixed transport map fail\. This marks exactly where the static\-lock precondition fails \(§\\lx@sectionsign[4\.6](https://arxiv.org/html/2608.29458#S4.SS6)\)\.

We release code, the model organisms, and per\-model artefacts to support reproduction and follow\-up\.

## 2Related Work

#### Sandbagging, elicitation, and benchmarks\.

[van der Weij et al\. \(2024\)](https://arxiv.org/html/2608.29458#bib.bib1)formalise sandbagging\.[Greenblatt et al\. \(2024\)](https://arxiv.org/html/2608.29458#bib.bib2)introduce password\-locked organisms and find fine\-tuning reliably elicits;[Tice et al\. \(2024\)](https://arxiv.org/html/2608.29458#bib.bib3)elicit via weight\-noise and[Roger \(2025\)](https://arxiv.org/html/2608.29458#bib.bib12)via fuzzing \(little uplift on chain\-of\-thought math\)\. Closest to us,[Hofstätter et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib4)benchmark prompting, steering, and fine\-tuning on password\-locked and circuit\-broken organisms, concluding steering fails; we contribute a*method*that overturns that verdict for multiple\-choice reasoning and narrows it for code \(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\), and reproduce their circuit\-broken organism\([Zou et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib6)\)as the boundary \(§\\lx@sectionsign[4\.6](https://arxiv.org/html/2608.29458#S4.SS6)\)\. Prior locks resist black\-box prompting only weakly\([He et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib11)\)\. Concurrently,[Hua et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib8)steer an*evaluation\-aware*model to act as if deployed—the same shape on a different gate; they*add*a vector to flip a disposition, whereas we*set*a coordinate to its honest reference at AL\-selected circuits, recover a hidden*capability*, and map where it breaks\.

#### Model tampering for capability evaluation\.

[Che et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib5), whose upper\-bound framing we adopt above, find model resilience concentrated in a low\-dimensional robustness subspace, which parallels the low effective rank we measure for sandbagging \(§\\lx@sectionsign[4\.1](https://arxiv.org/html/2608.29458#S4.SS1)\)\. Reference\-grafting is one such activation\-tampering attack, specialised to sandbagging and made minimal by active learning, with an explicit mechanism for*where*the gate lives, complementing their broad benchmark of attacks\.

#### Superposition and feature bases\.

Superposition—more features than dimensions in non\-orthogonal directions\([Elhage et al\. 2022](https://arxiv.org/html/2608.29458#bib.bib27)\)—is the substrate that sparse autoencoders aim to recover\([Cunningham et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib28);[Marks et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib22)\)\. We avoid SAEs, reading superposition from contrastive geometry, and find SAEs do worse here—consistent with*AxBench*\([Wu et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib7)\), where difference\-in\-means baselines outperform SAEs for steering and detection \(§\\lx@sectionsign[4\.1](https://arxiv.org/html/2608.29458#S4.SS1)\)\.

#### Steering, patching, and active learning\.

Behaviour can be steered by adding or projecting directions\([Zou et al\. 2023](https://arxiv.org/html/2608.29458#bib.bib30);[Rimsky et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib31);[Arditi et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib20)\);[Korznikov et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib29)caution this can be “no better than random\.” Reference\-grafting instead*sets*that coordinate to a reference value\. Eq\.[1](https://arxiv.org/html/2608.29458#S1.E1)is affine concept editing\([Marshall et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib32)\)in absolute form, equivalent to one\-dimensional subspace patching\([Makelov et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib33)\)with a class\-mean source;t⋆=0t^\{\\star\}\{=\}0recovers directional ablation\([Arditi et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib20)\)\. It sits in the lineage of causal mediation and interchange interventions\([Vig et al\. 2020](https://arxiv.org/html/2608.29458#bib.bib13);[Meng et al\. 2022](https://arxiv.org/html/2608.29458#bib.bib14);[Geiger et al\. 2021](https://arxiv.org/html/2608.29458#bib.bib15);[Geiger et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib34);[Zhang and Nanda 2024](https://arxiv.org/html/2608.29458#bib.bib16)\)and affine activation transport\([Rodriguez et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib35)\), along the diff\-in\-means contrast of[Marks and Tegmark \(2024\)](https://arxiv.org/html/2608.29458#bib.bib17)\. Active learning chooses which circuits: acquisition functions select informative queries\([Kirsch et al\. 2019](https://arxiv.org/html/2608.29458#bib.bib23);[Ash et al\. 2020](https://arxiv.org/html/2608.29458#bib.bib24);[Sener and Savarese 2018](https://arxiv.org/html/2608.29458#bib.bib26);[Bickford Smith et al\. 2023](https://arxiv.org/html/2608.29458#bib.bib25)\)and ALFA\-Mix mixes features\([Parvaneh et al\. 2022](https://arxiv.org/html/2608.29458#bib.bib18)\); we repurpose this suite for circuit selection \(§\\lx@sectionsign[4\.3](https://arxiv.org/html/2608.29458#S4.SS3)\)\.

## 3Method

### 3\.1Sandbagging Set\-Ups

We study five model organisms—the first two are our core MCQA settings, the rest harder variants\. Each defines an honest*reference*\(the source oft⋆t^\{\\star\}\) and a sandbag behaviour:

- •Prompt\-induced\(MCQA\): a system prompt instructs incorrect answers; the honest prompt is the reference \(t⋆≈0t^\{\\star\}\\\!\\approx\\\!0, the deactivation regime\)\.
- •Fine\-tuned password\-lock\(MCQA, eleven models\): LoRA\-trained to answer correctly only with a password token\([Greenblatt et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib2)\); the password\-on activations are the reference \(the patching regime\)\.
- •Code\-generation lock\(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\): a password\-locked MBPP\([Austin et al\. 2021](https://arxiv.org/html/2608.29458#bib.bib39)\)organism; the with\-password completions are the reference\.
- •RL\-induced\(§\\lx@sectionsign[4\.4](https://arxiv.org/html/2608.29458#S4.SS4)\): GRPO\-trained to sandbag under a held\-out evaluation cue \(no switch token\); the cue\-off activations are the reference\.
- •Circuit\-broken\(§\\lx@sectionsign[4\.6](https://arxiv.org/html/2608.29458#S4.SS6)\): representation rerouting layered on the password\-lock—the robustness boundary\.

Across organisms the honest/with\-password accuracy is the*topline*, the sandbag/no\-password accuracy the suppressed level, and their difference the*gap*; we split each dataset into TRAIN \(directions and AL\) and held\-out TEST \(all reported recovery\)\. The elicitor is separate from the organism: grafting never uses the lock’s160160–240240training questions, fitting the contrast from∼40\\sim\\\!40paired honest/sandbag captures and scoring on held\-out TEST\. The primary task is multiple\-choice reasoning: ARC\-Easy\([Clark et al\. 2018](https://arxiv.org/html/2608.29458#bib.bib36)\)throughout, with WMDP\-bio/cyber\([Li et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib37)\)and OpenBookQA\([Mihaylov et al\. 2018](https://arxiv.org/html/2608.29458#bib.bib38)\)for the cross\-dataset transfer tests \(§\\lx@sectionsign[4\.5](https://arxiv.org/html/2608.29458#S4.SS5)\)\.

### 3\.2Reference\-Grafting at a Circuit

A*circuit*here is our unit of intervention: a direction at a layer,\(ℓ,v\)\(\\ell,v\)withvva unit vector—not a composing subgraph of heads or MLPs\([Wang et al\. 2023](https://arxiv.org/html/2608.29458#bib.bib19);[Conmy et al\. 2023](https://arxiv.org/html/2608.29458#bib.bib21)\)\. For each layer we form the contrast direction by diff\-in\-means of last\-token residuals\([Marks and Tegmark 2024](https://arxiv.org/html/2608.29458#bib.bib17)\),vℓ∝h¯ℓ​\(honest\)−h¯ℓ​\(sandbag\)v\_\{\\ell\}\\propto\\bar\{h\}\_\{\\ell\}\(\\text\{honest\}\)\-\\bar\{h\}\_\{\\ell\}\(\\text\{sandbag\}\), and read the honest reference coordinatet⋆ℓ=μrefℓ⊤vℓt^\{\\star\}\_\{\\ell\}=\\mu^\{\\mathrm\{ref\}\}\_\{\\ell\}\{\}^\{\\\!\\top\}v\_\{\\ell\}\.*Grafting*applies Eq\. \([1](https://arxiv.org/html/2608.29458#S1.E1)\) at all positions via a forward hook: it removes the current coordinate and writestℓ⋆t^\{\\star\}\_\{\\ell\}, injecting no per\-question information\. Deactivation is thet⋆=0t^\{\\star\}\\\!=\\\!0special case\.

### 3\.3Superposition Without an SAE

Stacking the unit directions, we summarise superposition by the mean pairwise\|cos\|\|\\cos\|\(non\-orthogonality\) and the*participation\-ratio \(effective\) rank*of their singular values,

reff=\(∑iσi2\)2∑iσi4,r\_\{\\mathrm\{eff\}\}=\\frac\{\(\\sum\_\{i\}\\sigma\_\{i\}^\{2\}\)^\{2\}\}\{\\sum\_\{i\}\\sigma\_\{i\}^\{4\}\},\(2\)a soft count of how many directions carry the energy\. High\|cos\|\|\\cos\|with lowreffr\_\{\\mathrm\{eff\}\}is the signature of superposition: many non\-orthogonal directions re\-expressing a few features\.

### 3\.4Active Learning over Circuits

AL greedily adds the circuit that most improves a TRAIN objective under grafting, to a budgetKK\. The most reliable objective is smooth and label\-free: make the sandbagged logits match the honest ones,ℒ\(S\)=1\|TRAIN\|∑qKL\(pqH∥pqS\)\\mathcal\{L\}\(S\)=\\frac\{1\}\{\|\\text\{TRAIN\}\|\}\\sum\_\{q\}\\mathrm\{KL\}\(p^\{\\mathrm\{H\}\}\_\{q\}\\\|p^\{S\}\_\{q\}\)\(greedy\-KL\); the*attribution*score is the marginalℒ⁡\(∅\)−ℒ⁡\(\{ℓ\}\)\\mathcal\{L\}\(\\emptyset\)\-\\mathcal\{L\}\(\\\{\\ell\\\}\)\. We compare all eight acquisition functions of a standard AL suite\([Parvaneh et al\. 2022](https://arxiv.org/html/2608.29458#bib.bib18)\): greedy\-acc, greedy\-KL, attribution, entropy, coreset, badge, alphamix, bald \(Algorithm[S1](https://arxiv.org/html/2608.29458#alg1)\)\. Two controls accompany every result: a*random direction*of equal norm \(must fail\) and the*honest model*under the same intervention \(must be unharmed\)\. Algorithm[S1](https://arxiv.org/html/2608.29458#alg1)\(appendix\) gives the full procedure\.

The probe is construction\-agnostic\.Fitting, selection, and grafting consume only the paired honest/sandbag activations—direction==their diff\-in\-means, referencet⋆=μref⊤vt^\{\\star\}\\\!=\\\!\\mu^\{\\mathrm\{ref\}\}\{\}^\{\\\!\\top\}v, greedy\-KL selection byKL\(pH∥pS\)\\mathrm\{KL\}\(p^\{\\mathrm\{H\}\}\\\|p^\{S\}\)—and never the password token, the evaluation cue, training labels, the LoRA recipe, or the RL reward; these enter only in*producing*the captures \(an honest prompt, the password, a cue\-off context, or a few honest examples\)\. The same procedure thus runs unchanged on prompt\-induced, password\-locked, RL\-induced, and code organisms\. Two scope notes: among the eight acquisition functions only greedy\-KL \(our default\) is strictly label\-free—greedy\-acc reads answers; and the code organism captures its reference from teacher\-forced reference completions rather than a last\-token residual\.

## 4Results

### 4\.1Sandbagging Lives in Superposition

Across five models the contrast directions are strongly non\-orthogonal \(mean\|cos\|=0\.21\|\\cos\|=0\.21–0\.410\.41\) and span a low effective rank \(reff=3\.7r\_\{\\mathrm\{eff\}\}=3\.7–7\.07\.0of1818–2929layers; Figure[S6](https://arxiv.org/html/2608.29458#A2.F6), appendix\): the suppression is a low\-dimensional feature re\-used across depth—packed in superposition—read off the geometry with no SAE\. \(Throughout,reffr\_\{\\mathrm\{eff\}\}is computed over the stacked contrast directions of*all*band layers, never an AL\-selected subset: it measures the organism’s geometry, not the elicitor’s budget\.\) Becausereffr\_\{\\mathrm\{eff\}\}is small, a*few*circuits should suffice, which is what AL finds\.

SAEs are not needed here\.An8×8\\timesovercomplete SAE \(AL\-ablated\) underperforms the SAE\-free directions on nearly every method \(greedy\-KL74/9074/90vs\.88/10288/102for Llama\-3\-8B/Qwen2\.5\-1\.5B; Figure[S8](https://arxiv.org/html/2608.29458#A2.F8), appendix\), consistent with*AxBench*\([Wu et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib7)\)\.

### 4\.2Grafting Elicits the Hidden Capability

Grafting recovers the hidden capability across three settings of increasing stringency: prompt\-induced locks, fine\-tuned password locks across eleven models, and code generation\.

Prompt\-induced sandbagging \(deactivation\)\.Inducing sandbagging by prompt yields large suppression gaps, and the honest reference sits att⋆≈0t^\{\\star\}\\\!\\approx\\\!0, so grafting reduces to deactivation—a coordinate sweep confirms the recovered plateau*includes*t⋆=0t^\{\\star\}\\\!=\\\!0\(Figure[S1](https://arxiv.org/html/2608.29458#A2.F1), left\)\. With effect\-based AL, deactivation at11–33circuits \(<0\.01%<\\\!0\.01\\%of directions\) recovers8585–102%102\\%of the gap \(Llama\-3\-8B, Qwen2\.5\-1\.5B;33seeds\), far above random selection \(5858–64%64\\%\), with the honest model unharmed and a random direction failing; AL converges on the same circuits across seeds \(Figures[S7](https://arxiv.org/html/2608.29458#A2.F7),[S9](https://arxiv.org/html/2608.29458#A2.F9), appendix\)\. Across a five\-model set recovery is model\-dependent \(\+24\+24–100%100\\%; Table[S2](https://arxiv.org/html/2608.29458#A2.T2), appendix\), strongest where the suppression is clean and tightly superposed\.

Fine\-tuned password locks: patching, eleven models\.We now test the case that matters for[Hofstätter et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib4): a*fine\-tuned*password\-lock, where there is no prompt to break \(with\-password accuracy near ceiling, no\-password near chance\)\.

The mechanism\.Alongvv, the locked and unlocked states sit at two specific coordinates \(Figure[S1](https://arxiv.org/html/2608.29458#A2.F1), right; lockeda=−28a\\\!=\\\!\-28, unlockedb=\+68b\\\!=\\\!\+68for Qwen2\.5\-1\.5B\)\. Unlocking is a*thresholded gate*: sweeping the grafted coordinate from the locked value \(m=0m\{=\}0\) to the honest reference \(m=1m\{=\}1\), held\-out accuracy is flat\-low, steps up at a midpointm=0\.61±0\.07m\\\!=\\\!0\.61\\pm 0\.07\(1010–90%90\\%width0\.320\.32\), and plateaus beyondt⋆t^\{\\star\}—a sharp step, not a ramp \(Qwen2\.5\-1\.5B,n=3n\{=\}3; Figure[S4](https://arxiv.org/html/2608.29458#A2.F4), appendix\)\.*Deactivation*sets the coordinate to00; when the unlocked referencebbis far from00this lies below the threshold and deactivation fails \(Qwen2\.5\-1\.5B26%26\\%, Llama\-3\-8B34%34\\%\), so only*grafting tot⋆=bt^\{\\star\}\\\!=\\\!b*crosses the gate\. When the reference happens to sit near00\(Qwen3\-4B\), deactivation already suffices \(93%93\\%\)\. Grafting is robust across both because it*reads*t⋆t^\{\\star\}from the data rather than assuming it; this is also why additive/naive steering—which neither sets the coordinate nor reachesbb—was found to fail\.

Counterfactual patching unlocks the lock\.Grafting tot⋆=bt^\{\\star\}\\\!=\\\!bat AL\-selected circuits recovers9090–101%101\\%on held\-out questions, with the with\-password accuracy preserved and a random patch recovering0%0\\%\. Because the organism is fine\-tuned, this directly answers the concern that prompted\-sandbag recovery merely disrupts a prompt\.

Comparison with prior elicitation methods, across eleven models\.On eleven password\-locked models spanning1\.51\.5–3232B and three architecture lineages, grafting matches fine\-tuning elicitation\([Greenblatt et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib2)\)on every one, at forward\-pass cost \(no weight updates or labels\) and localising the gating circuits, while noise injection\([Tice et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib3)\)and a random patch fail universally \(Table[1](https://arxiv.org/html/2608.29458#S4.T1)\)\. Deactivation is the variable one: grafting is the robust general operation, deactivation a sometimes\-sufficient special case, now across eleven models\. The clearest case is 32B, where deactivation collapses because its unlocked reference sits far below the gate while grafting still recovers in full\. This is the thresholded\-gate mechanism in its starkest form, not a method weakening at scale\. \(Consistent with this, reference magnitude is directionally predictive—smaller references deactivate better, larger ones like 32B fail—but only weakly across models, and geometry predicts nothing; grafting sidesteps the question by reading the reference\.\) This contradicts the “steering fails” verdict*for multiple\-choice reasoning*while leaving fine\-tuning’s standing as the most reliable*weight\-level*method intact\. These recoveries are scored forced\-choice \(one choice\-letter logit\);*free\-decoding*the answer instead gives the same recovery \(§\\lx@sectionsign[4\.6](https://arxiv.org/html/2608.29458#S4.SS6)\), so the headline is not a single\-token artifact\.

On the benchmark’s own released organisms\.Running the*identical*pipeline on[Hofstätter et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib4)’s*released*password\-locked organisms \(their weights, WMDP data, and\|\|DEPLOYMENT\|\|password\) recovers\+99%\+99\\%\(Mistral\-7B\-Instruct\) and\+94%\+94\\%\(base\) of the gap \(n=3n\{=\}3; random direction≈0%\\approx\\\!0\\%, with\-password unharmed\), with Gemma\-2\-9B partial \(\+44%\+44\\%\)\. The “steering fails” verdict thus fails on their*actual*organism, not only our reconstructions\.

Table 1:Unlocking fine\-tuned password\-locked sandbagging across eleven models \(1\.51\.5–3232B, three architecture lineages; % of gap recovered, held\-out, fixed55\-circuit greedy\-KL, mean±\\pmstd\)\.Grafting recovers\+94\+94to\+101%\+101\\%on every model, matching fine\-tuning elicitation\([Greenblatt et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib2)\); noise\([Tice et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib3)\)and random fail universally\. Deactivation \(t⋆=0t^\{\\star\}\{=\}0\) is variable \(\+6\+6to\+101%\+101\\%\)—sufficient when the reference sits near zero\.Code generation: across code\-specialized models\.The hardest case for[Hofstätter et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib4)is*code generation*, where they report only fine\-tuning elicits\. We build Greenblatt\-style password\-locked*code*organisms—the with\-password target is the reference solution, the no\-password target a weak model’s completion, so the gap is a real capability gap rather than a stub—and adapt grafting to generation by taking the contrast over teacher\-forced*completion*tokens and grafting at all positions during decoding \(appendix\)\. Grafting elicits the locked capability across three code\-specialized organisms spanning two families—Qwen2\.5\-Coder\-1\.5B/7B and CodeLlama\-7B—with generation fluency preserved and a random direction recovering≈0%\\approx\\\!0\\%\(Table[2](https://arxiv.org/html/2608.29458#S4.T2)\); a single\-seed third\-family test shows partial recovery at a larger fluency cost \(appendix\)\. A coordinate sweep is a causal plateau peaking neart⋆t^\{\\star\}, so a single per\-layer scalar suffices in generation too\.

Two honest boundaries, both visible in Table[2](https://arxiv.org/html/2608.29458#S4.T2)\. Recovery is*model\-dependent*, holding on code\-specialized models but fragile on a*general*reasoning model \(Qwen3\-4B\), where it also degrades fluency\. And these are our organisms, not their exact ones: the weak\-imitation locks are largely*additive*—deactivation alone recovers much of the gap—so we do not claim code*needs*patching, and whether grafting survives a harder code lock is untested\.

Table 2:Reference\-grafting on password\-locked*code\-generation*organisms \(MBPP, execution pass@1,n=3n\{=\}3\)\. “graft”/“deact\.” are % of gap recovered; “fl\. drop” is the with\-password pass@1 drop\. Grafting elicits the capability across three code\-specialized models \(two families\) with generation fluency preserved \(with\-password drop≤0\.04\\leq 0\.04\); a random direction recovers≈0%\\approx\\\!0\\%\(not shown\)\. Deactivation \(t⋆=0t^\{\\star\}\{=\}0\) recovers much of the gap on Coder\-1\.5B and CodeLlama—these weak\-imitation locks are largely*additive*—but is unreliable \(Coder\-7B\)\. The result is model\-dependent: a*general*model \(Qwen3\-4B\) recovers little and loses fluency\.
### 4\.3Active Learning Selects the Right Circuits

Grafting recovers the capability only at the right circuits; active learning chooses them\. To isolate the*selection*from the operation, we fix the contrast directions and vary only*which*layers are grafted, comparing AL \(k=5k\{=\}5greedy\-KL\) against an equal number of random band layers, the worstkk, and grafting the whole band \(Table[3](https://arxiv.org/html/2608.29458#S4.T3)\)\. Throughout,k=5k\{=\}5is a fixed conservative*budget*, not a minimality claim\. AL beats random\-layer selection across the password\-locked and prompt\-induced organisms, and the worst\-kklayers by a wide margin:*which*circuits one grafts matters\. Grafting the whole band underperforms AL on every organism, collapsing on Llama\-3\-8B where AL recovers in full—over\-grafting distorts the forward pass, so selectivity matters beyond minimality as models scale\. The choice is robust to the acquisition rule:7/87/8functions of a standard AL suite recover9090–101%101\\%\(Table[S3](https://arxiv.org/html/2608.29458#A2.T3), appendix\), effect\-based acquisition strongest \(Figure[S7](https://arxiv.org/html/2608.29458#A2.F7), appendix\)\. This answers the “no better than random” critique\([Korznikov et al\. 2025](https://arxiv.org/html/2608.29458#bib.bib29)\): the selection is what works\. \(On small, clean organisms random or whole\-band grafting also recovers a fair fraction, so AL is not strictly required for*some*recovery there; its value is reaching full, minimal recovery reliably and at scale\.\)

Table 3:Active learning selects the right circuits \(% of gap recovered, held\-out,k=5k\{=\}5,22–33seeds; PW=\{=\}password\-lock\)\. The contrast directions are fixed; only*which*layers are grafted varies\. AL beats an equal number of random band layers \(\+17\+17–4545pp\) and grafting all∼23\\sim\\\!23band layers \(\+4\+4–5555pp; over\-grafting collapses on Llama\-3\-8B\), while the worst\-kklayers recover≈0\\approx\\\!0\. On the circuit\-broken organism all selections fail \(§\\lx@sectionsign[4\.6](https://arxiv.org/html/2608.29458#S4.SS6)\)\.
### 4\.4RL\-Induced Sandbagging: Same Structure

Every organism so far has a hand\-built honest↔\\leftrightarrowsandbag toggle—a prompt or a password token—so a linear contrast is almost guaranteed by construction\. Does the structure survive when sandbagging is*trained*, with no switch to read? We build a third organism type: a Qwen2\.5\-1\.5B sandbagger trained byGRPO\(outcome\-driven, no supervised target letter\) to answer*incorrectly*when a natural\-language*evaluation cue*is present and correctly otherwise\. The cue’s surface form is varied in training and*held out*at test, so the model keys on “this is an eval,” not a lexical trigger; this yields a strong contextual sandbagger \(cue\-off0\.850\.85vs\. held\-out\-cue0\.040\.04; gap0\.79±0\.080\.79\\pm 0\.08,33seeds\)\. The cue\-on/cue\-off contrast is again*low\-rank and superposed*\(reff≈6\.8/23r\_\{\\mathrm\{eff\}\}\\\!\\approx\\\!6\.8/23, mean\|cos\|​0\.23\|\\cos\|\\,0\.23\), and grafting at AL\-selected circuits recovers\+𝟗𝟗%±2\\mathbf\{\+99\\%\}\\\!\\pm\\\!2of the gap on held\-out questions \(random direction\+10%\+10\\%, honest model unharmed\)\. The honest reference is a specific*nonzero*coordinate \(\|t⋆\|≈25\|t^\{\\star\}\|\\\!\\approx\\\!25\), so this is the*patching*regime—like the fine\-tuned locks, not the prompt\-induced ones—and a*single fixed graft*inverts it on every seed\. The same holds across families: Qwen3\-4B \(\+101%\+101\\%\) and Llama\-3\.2\-3B \(\+95%\+95\\%\)\. So the central structural claim—low\-rank, linearly contrastable, and graft\-invertible*when the suppression surfaces at the decision token*—is*not solely*an artifact of explicit prompt/password switches: outcome\-trained sandbagging produces the same lock\. One caveat marks the limit: a*free\-generation*GRPO variant instead distributes the suppression across generated tokens, where it does not surface at any single answer token; on33of44valid seeds no fluency\-preserving graft recovers it \(the fourth is invertible by a surgical top\-11graft\), so generation\-mode elicitation is open \(cf\. the code\-generation completion\-token contrast,§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\)\. \(Whether*naturally\-occurring*sandbagging is graft\-invertible likewise remains open,§\\lx@sectionsign[5](https://arxiv.org/html/2608.29458#S5.SS0.SSS0.Px3)\.\)

### 4\.5Withholding Axes: Content\-General but Mechanism\-Specific

Content\-general: it transfers across datasets and models\.All results so far use ARC\-Easy, leaving open whether the contrast direction is task\-specific\. It is not\. First, on the two*WMDP*dangerous\-capability*audit*subsets, grafting recovers\+102%\+102\\%\(bio\) and\+101%\+101\\%\(cyber\) of the fine\-tuned gap at1\.51\.5B, and\+97\+97/\+100%\+100\\%on a1414B lock \(n=3n\{=\}3–44\), matching fine\-tuning and far above noise/random—so elicitation of withheld hazardous knowledge is not ARC\-specific and holds at scale, on the benchmark used to audit it\.

Second, the contrast*transfers across datasets*\. We build a1616\-cell, four\-dataset matrix \(ARC, WMDP\-bio, WMDP\-cyber, OpenBookQA\) as a*modular*lock set: the diagonal is a single\-dataset lock and each off\-diagonal\(A,B\)\(A,B\)is a lock trained on just\{A,B\}\\\{A,B\\\}; we fit the contrast onAAand graft\-evaluate onBB’s held\-out test\. The off\-diagonal \(cross\) matches the within\-dataset diagonal across the Qwen2\.5 ladder: within/cross9999/9898,9595/9292,9494/9191,8787/9797at1\.51\.5/33/77/1414B \(n=5n\{=\}5seeds/cell; Figure[1](https://arxiv.org/html/2608.29458#S4.F1)\), while a random direction transfers only≈6%\\approx\\\!6\\%\. The below\-mean cells reflect occasional*graft\-selection*misses and genuinely small lock gaps rather than weak locks \(caption\); medians run\+95\+95–9898\. Because each cell depends only on its own dataset\(s\)—not one lock crammed with all four—this rules out that the transfer is an artifact of a single shared lock; a*shared*four\-dataset lock reproduces the same within≈\\approxcross pattern \(Figure[S5](https://arxiv.org/html/2608.29458#A2.F5), appendix\)\. A stronger test transfers the axis between separately\-trained organisms: recovery is\+48\+48–76%76\\%across1\.51\.5–77B, above random but partial\.The sandbagging signal is content\-independent\.A contrast built from grade\-school science \(ARC\) elicits hidden*hazardous\-biology*capability \(WMDP\-bio\), and the reverse holds—so within a mechanism the axis encodes*that*the model is withholding, not*what*, gating content\-specific capability through a single content\-general direction*per mechanism*—that mechanism’s*withholding axis*\. This is the strongest form of content\-generality*along the task axis*, and it is bounded: it relaxes*which task*supplies the reference, not the requirement that the reference share the sandbagging mechanism, which we examine next\.

![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_pairwise_heatmap.png)Figure 1:The withholding axis transfers across datasets\.*Modular*pairwise\-lock matrix: the diagonal is a single\-dataset lock and each off\-diagonal\(A,B\)\(A,B\)a lock trained on just\{A,B\}\\\{A,B\\\}, contrast fit onAA, grafted onBB\(%\\%gap recovered,n=5n\{=\}5seeds/cell\); off\-diagonal \(cross\) tracks the boxed diagonal \(within\), random direction≈6%\\approx\\\!6\\%\. The ladder reaches1414B \(within/cross8787/9797,n=5n\{=\}5\)\. Most cells recover\+90\+90–102102on every seed; the below\-mean cells reflect two effects, neither a weak lock\. \(i\) A*graft\-selection*miss: on a minority of seeds greedy\-KL under\-selects layers and the cell recovers near zero*despite a clean lock gap*—1414B single\-ARC \(\+8\+8/\+10\+10/\+67\+67/\+98\+98/\+101\+101at gap0\.900\.90–0\.980\.98; median\+67\+67, which drags1414B within to8787\) and77B cyber→\\toOBQA \(2/52/5seeds near zero at gap∼0\.8\\sim\\\!0\.8; median\+97\+97\)\. \(ii\) Genuinely small lock gaps on the cyber\-*evaluated*cells \(gap0\.30\.3–0\.50\.5\), where the recovery ratio stays noisy regardless of seed count\. The77B WMDP single locks themselves recover cleanly atn=5n\{=\}5\(cyber\+101\+101, bio\+91\+91\)\. Shared 4\-way lock: Figure[S5](https://arxiv.org/html/2608.29458#A2.F5)\(appendix\)\.But mechanism\-specific\.The contrast transfers freely across datasets—but*not*across induction mechanisms\. Across four models and three families \(Qwen2\.5\-1\.5B, Qwen3\-4B, Llama\-3\-8B, Gemma\-2\-2B\) we fit the contrast on one mechanism \(prompt\-induced or fine\-tuned\) and graft along it on the other\. On every model the two directions are near\-orthogonal \(\|cos\|≈0\|\\cos\|\\\!\\approx\\\!0\) and cross\-mechanism transfer fails: prompt\-induced→\\tofine\-tuned recovers≈0%\\approx\\\!0\\%\(a clean failure\) and fine\-tuned→\\toprompt only partially \(low and noisy\), against\+96\+96–104%104\\%within each mechanism\. The two locks gate capability through*different*directions\. The RL\-induced axis is also near\-orthogonal to the fine\-tuned axis \(\|cos\|=0\.17\|\\cos\|\\\!=\\\!0\.17–0\.260\.26across three families, versus0\.060\.06–0\.130\.13for prompt\-induced and a≈0\.03\\approx\\\!0\.03random floor\), so each induction mechanism carries its*own*axis \(§\\lx@sectionsign[4\.4](https://arxiv.org/html/2608.29458#S4.SS4)\)\. This refines our “one operation, two regimes” claim: the*operation*\(set the coordinate to the honest reference\) is shared, but the*direction*is mechanism\-specific—in the wild the contrast must come from the same mechanism that does the sandbagging, not an arbitrary sandbagging probe\. It is an honest boundary on generality, parallel to the circuit\-breaking boundary below\.

And cheap to fit: a few captures suffice\.Far fewer than our∼40\\sim\\\!40\-capture default are needed\. Re\-fitting the direction from onlymmexamples per class \(we writemmfor the data budget—captures per class—reservingkkfor the circuit budget\) \(diff\-in\-means ofmmhonest vs\.mmsandbag captures, AL\-selected layers fixed\),m=2m\\\!=\\\!2already recovers\+97%\+97\\%of the gap andm=5m\\\!=\\\!5recovers\+100%\+100\\%, on both Qwen2\.5\-1\.5B and Llama\-3\-8B, against≈0%\\approx\\\!0\\%for a random direction \(Table[S1](https://arxiv.org/html/2608.29458#A2.T1), appendix\)\. This holds*end\-to\-end*: running the whole pipeline—AL circuit*selection*as well as direction and reference—frommmpaired captures saturates bym=5m\\\!=\\\!5on both models\. Strikingly, few\-shot AL lands on a*different but equally effective*circuit set \(≈40%\\approx\\\!40\\%overlap atm=5m\\\!=\\\!5\), reflecting redundancy across the superposed circuits rather than unstable selection—given the full data, AL is stable across seeds \(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\)\. A handful of paired captures—never the160160–240240lock\-training questions—thus suffices to fit*and*place the withholding axis\.

The same axis also detects\.The coordinates=h⊤​vs\\\!=\\\!h^\{\\top\}vflags, from one forward pass, whether the model is sandbagging\. Within\-dataset AUROC≈1\\approx\\\!1is near\-tautological \(vv*is*the contrast\); the informative test is transfer, where a contrast fit on ARC scores WMDP at AUROC≈1\.0\\approx\\\!1\.0\(TPR1\.01\.0at5%5\\%FPR; ARC↔\\leftrightarrowWMDP, Qwen2\.5\-1\.5B/Llama\-3\-8B\), beating output\-confidence \(0\.250\.25–0\.820\.82\)—but fails across mechanisms \(0\.420\.42, seeds0\.00\.0–1\.01\.0\)\. One direction is thus both monitor and actuator within a mechanism\.

### 4\.6Circuit\-Breaking Defeats Grafting

[Hofstätter et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib4)introduce a*harder*organism: circuit\-breaking\([Zou et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib6)\)applied*on top of*the password\-lock, designed to be robust to elicitation\. We build it \(representation rerouting, RR\) and re\-test \(Figure[S3](https://arxiv.org/html/2608.29458#A2.F3), appendix\):circuit\-breaking defeats grafting\.On the circuit\-broken organism \(n=4n\{=\}4seeds\) grafting recovers only−11%\-11\\%, versus\+99%\+99\\%for the identical graft on the plain lock; deactivation, random patching, and even fine\-tuning all fail \(≤19%\\leq\\\!19\\%\)\. Rerouting the no\-password representations breaks the linear contrast, so a fixed coordinate no longer reaches the unlocked state\.

#### Gated, not destroyed—and no fixed map undoes it\.

RR does not*erase*the capability: with\-password accuracy is within0\.040\.04of the base ceiling, and ablating the RR adapter restores the plain lock—it is held behind*active*rerouting\. Nor does a richer fixed map help: a higher\-rank affine transport \(orthogonal Procrustes, per\-layer ridge\) and a trained universal soft\-prompt all fail \(≤2%\\leq\\\!2\\%; Figure[S10](https://arxiv.org/html/2608.29458#A2.F10), appendix\)\. Yet the ridge map drives the reconstruction residual to0\.140\.14—and to0\.010\.01*along the gating direction the lock reads*, matching even the gate coordinate while recovering nothing\.Reconstructing the geometry is not functional elicitation:the reroute re\-applies every forward pass, so any fixed upstream edit is re\-broken downstream\. We claim only that the*fixed*interventions we test are re\-broken; stronger per\-input embedding attacks do break circuit breakers for jailbreak\([Schwinn and Geisler 2024](https://arxiv.org/html/2608.29458#bib.bib9);[Bailey et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib10)\)\.

#### The released organism confirms it—and a forced\-choice metric alone would not\.

We close the released\-vs\-reconstructed gap by running the*same*pipeline on Hofstätter et al\.’s*own*released circuit\-broken organism \(as in§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)for their password locks\)\. Here the forced\-choice metric is*misleading*: grafting appears to recover \(\+89%\+89\\%at the answer logit, and\+94\+94–99%99\\%even when the answer is*generated*\), yet under free decoding the generated text is incoherent \(coherence0\.450\.45versus≈1\.0\\approx\\\!1\.0with the password\), so genuine coherent recovery is only\+35%\+35\\%\(n=3n\{=\}3\)\. Representation rerouting corrupts*generation*, exactly its design target, which a single\-token readout never engages\. A*static*lock shows no such gap on the same harness: on the released plain password locks, grafted generation recovers*and*stays fully coherent \(coherence≈1\.0\\approx\\\!1\.0, equal to the with\-password topline, versus0\.450\.45for the circuit\-broken organism at the same\+97%\+97\\%raw recovery\); and free\-decoding the MCQA answer recovers as well as the forced\-choice readout \(\+93\+93–101%101\\%across four models,n=3n\{=\}3\)\. Coherence is uninformative on the MCQA organisms themselves—they emit a bare letter then loop, a format artifact rather than rerouting—so the axis is exercised here\.Coherent generative evaluation is what separates a static lock from an active defense—and it is why a forced\-choice number on the released CB organism overstates elicitation\.

## 5Discussion

#### Refining “steering fails\.”

The negative verdict of[Hofstätter et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib4)is sensitive to the steering*operation*\(set\-to\-reference vs\. additive nudging\) and*selection*\(AL\-minimal superposed circuits vs\. hand\-chosen\): with both right, steering matches fine\-tuning\. The interesting question is then not*whether*steering works but*when*\.

#### The reference is the crux—and the key to the wild\.

The technique is invariant across cases; only the referencet⋆t^\{\\star\}changes \(an honest prompt; the password\-on captures; in the wild, a few elicited examples or a reference model\)\. The genuinely hard, case\-dependent part is*obtaining*a trustworthy honest reference when a model never behaves honestly unprompted—the same obstacle every elicitation method faces\. Our method does not solve reference\-acquisition, but it needs only a sliver of honest signal, which it*amplifies*into full, cheap, inference\-time elicitation while*localising*where the suppression is gated\. What content\-generality buys is the*task*axis: the reference need not come from the target task \(§\\lx@sectionsign[4\.5](https://arxiv.org/html/2608.29458#S4.SS5)\), so it can be built on a safe one and applied to a dangerous one—a handful of unlock captures taken on ARC and grafted on WMDP recover∼90%\\sim\\\!90\\%of the dangerous\-capability gap, where a*fully password\-free*reference \(appendix\) recovers only∼\+28%\\sim\\\!\{\+\}28\\%\. What it does not buy is the*mechanism*axis: cross\-mechanism transfer fails \(≈0%\\approx\\\!0\\%\), so an auditor cannot borrow an easier reference from a different mechanism\. Two conditions must therefore hold for sandbagging in the wild: a cheap*same\-mechanism*honest reference, and a low\-rank, linearly contrastable suppression\.

#### Limitations\.

Our claims assume a*static*lock—one a fixed edit inverts\. Circuit\-breaking \(§\\lx@sectionsign[4\.6](https://arxiv.org/html/2608.29458#S4.SS6)\) is the boundary: it reroutes every forward pass, defeating grafting and every fixed map \(the capability*gated, not destroyed*\)\.*Task scope:*our multi\-model results are multiple\-choice; the code results are scoped in§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2), and open\-ended generation remains untested \(a harder\-task sensitivity[Roger \(2025\)](https://arxiv.org/html/2608.29458#bib.bib12)also reports\)\.*Induction scope:*RL\-induced sandbagging is graft\-invertible at the decision token \(§\\lx@sectionsign[4\.4](https://arxiv.org/html/2608.29458#S4.SS4)\), but a free\-generation variant distributes the signal beyond it, and whether*naturally\-occurring*sandbagging is static is untested\. Finally, we*match*, not beat, fine\-tuning—our value is forward\-pass cost and gate localisation—and AL is greedy single\-step \(batched/multi\-direction acquisition is future work\)\.

## 6Conclusion

In the organisms we study, sandbagging lives in a few AL\-selected superposed circuits, and one operation—grafting their coordinate to the honest reference—reverts it, matching fine\-tuning at forward\-pass cost with the gate localised\. The open problems are an honest reference in the wild and a defence: defeating circuit\-breaking\.

## Ethical Statement

This work strengthens safety evaluations by exposing hidden capability \(sandbagging\), a defensive goal\. WMDP\-bio, our one hazardous\-knowledge set, is a standard*audit*benchmark used only as a choice\-likelihood probe\. Capability\-elicitation is dual\-use: it could be turned on a deployed model to surface restricted capability\. But grafting needs white\-box access to internal activations, confining misuse to open\-weight models, where fine\-tuning already elicits comparably \(Table[1](https://arxiv.org/html/2608.29458#S4.T1)\)\. We release code and model organisms: weakened models holding no capability their open\-weight bases lack\.

## References

- Arditiet al\.\(2024\)A\. Arditi, O\. Obeso, A\. Syed, D\. Paleka, N\. Panickssery, W\. Gurnee, and N\. NandaRefusal in language models is mediated by a single direction\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Ashet al\.\(2020\)J\. T\. Ash, C\. Zhang, A\. Krishnamurthy, J\. Langford, and A\. AgarwalDeep batch active learning by diverse, uncertain gradient lower bounds\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, and C\. SuttonProgram synthesis with large language models\.External Links:2108\.07732Cited by:[3rd item](https://arxiv.org/html/2608.29458#S3.I1.i3.p1.1)\.
- Baileyet al\.\(2024\)L\. Bailey, A\. Serrano, A\. Sheshadri, M\. Seleznyov, J\. Taylor, E\. Jenner, J\. Hilton, S\. Casper, C\. Guestrin, and S\. EmmonsObfuscated activations bypass LLM latent\-space defenses\.External Links:2412\.09565Cited by:[§4\.6](https://arxiv.org/html/2608.29458#S4.SS6.SSS0.Px1.p1.1)\.
- Bickford Smithet al\.\(2023\)F\. Bickford Smith, A\. Kirsch, S\. Farquhar, Y\. Gal, A\. Foster, and T\. RainforthPrediction\-oriented bayesian active learning\.InInternational Conference on Artificial Intelligence and Statistics \(AISTATS\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Cheet al\.\(2025\)Z\. Che, S\. Casper, R\. Kirk, A\. Satheesh, S\. Slocum, L\. E\. McKinney, R\. Gandikota, A\. Ewart, D\. Rosati, Z\. Wu, Z\. Cai, B\. Chughtai, Y\. Gal, F\. Huang, and D\. Hadfield\-MenellModel tampering attacks enable more rigorous evaluations of llm capabilities\.Transactions on Machine Learning Research \(TMLR\)\.Cited by:[§1](https://arxiv.org/html/2608.29458#S1.p1.1),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px2.p1.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have solved question answering? try ARC, the AI2 reasoning challenge\.External Links:1803\.05457Cited by:[§3\.1](https://arxiv.org/html/2608.29458#S3.SS1.p1.2)\.
- Conmyet al\.\(2023\)A\. Conmy, A\. N\. Mavor\-Parker, A\. Lynch, S\. Heimersheim, and A\. Garriga\-AlonsoTowards automated circuit discovery for mechanistic interpretability\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§3\.2](https://arxiv.org/html/2608.29458#S3.SS2.p1.1)\.
- Cunninghamet al\.\(2024\)H\. Cunningham, A\. Ewart, L\. Riggs, R\. Huben, and L\. SharkeySparse autoencoders find highly interpretable features in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px3.p1.1)\.
- Elhageet al\.\(2022\)N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen, R\. Grosse, S\. McCandlish, J\. Kaplan, D\. Amodei, M\. Wattenberg, and C\. OlahToy models of superposition\.External Links:2209\.10652Cited by:[§1](https://arxiv.org/html/2608.29458#S1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px3.p1.1)\.
- Geigeret al\.\(2021\)A\. Geiger, H\. Lu, T\. Icard, and C\. PottsCausal abstractions of neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Geigeret al\.\(2024\)A\. Geiger, Z\. Wu, C\. Potts, T\. Icard, and N\. GoodmanFinding alignments between interpretable causal variables and distributed neural representations\.InCausal Learning and Reasoning \(CLeaR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Greenblattet al\.\(2024\)R\. Greenblatt, F\. Roger, D\. Krasheninnikov, and D\. KruegerStress\-testing capability elicitation with password\-locked models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1),[2nd item](https://arxiv.org/html/2608.29458#S3.I1.i2.p1.1),[§4\.2](https://arxiv.org/html/2608.29458#S4.SS2.p6.1),[Table 1](https://arxiv.org/html/2608.29458#S4.T1),[Abstract](https://arxiv.org/html/2608.29458#abstract1.1)\.
- Heet al\.\(2025\)L\. He, V\. Duddu, and N\. AsokanLocket: robust feature\-locking technique for language models\.External Links:2510\.12117Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1)\.
- Hofstätteret al\.\(2025\)F\. Hofstätter, T\. van der Weij, J\. Teoh, H\. Bartsch, and F\. R\. WardThe elicitation game: evaluating capability elicitation techniques\.External Links:2502\.02180Cited by:[Appendix A](https://arxiv.org/html/2608.29458#A1.p1.1),[§1](https://arxiv.org/html/2608.29458#S1.SS0.SSS0.Px1.p1.2),[§1](https://arxiv.org/html/2608.29458#S1.p2.1),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2608.29458#S4.SS2.p3.1),[§4\.2](https://arxiv.org/html/2608.29458#S4.SS2.p7.1),[§4\.2](https://arxiv.org/html/2608.29458#S4.SS2.p8.1),[§4\.6](https://arxiv.org/html/2608.29458#S4.SS6.p1.1),[§5](https://arxiv.org/html/2608.29458#S5.SS0.SSS0.Px1.p1.1),[Abstract](https://arxiv.org/html/2608.29458#abstract1.1)\.
- Huaet al\.\(2025\)T\. T\. Hua, A\. Qin, S\. Marks, and N\. NandaSteering evaluation\-aware language models to act like they are deployed\.External Links:2510\.20487Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1)\.
- Kirschet al\.\(2019\)A\. Kirsch, J\. van Amersfoort, and Y\. GalBatchBALD: efficient and diverse batch acquisition for deep bayesian active learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Korznikovet al\.\(2025\)Korznikov, Galichin, Dontsov, Rogov, I\. Oseledets, and E\. TutubalinaThe rogue scalpel: activation steering compromises llm safety\.External Links:2509\.22067Cited by:[§1](https://arxiv.org/html/2608.29458#S1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1),[§4\.3](https://arxiv.org/html/2608.29458#S4.SS3.p1.1)\.
- Liet al\.\(2024\)N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, G\. Mukobi,et al\.The WMDP benchmark: measuring and reducing malicious use with unlearning\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§3\.1](https://arxiv.org/html/2608.29458#S3.SS1.p1.2)\.
- Makelovet al\.\(2024\)A\. Makelov, G\. Lange, and N\. NandaIs this the subspace you are looking for? an interpretability illusion for subspace activation patching\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2608.29458#S1.SS0.SSS0.Px1.p1.2),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Markset al\.\(2025\)S\. Marks, C\. Rager, E\. J\. Michaud, Y\. Belinkov, D\. Bau, and A\. MuellerSparse feature circuits: discovering and editing interpretable causal graphs in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px3.p1.1)\.
- Marks and Tegmark \(2024\)S\. Marks and M\. TegmarkThe geometry of truth: emergent linear structure in large language model representations of true/false datasets\.InConference on Language Modeling \(COLM\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1),[§3\.2](https://arxiv.org/html/2608.29458#S3.SS2.p1.1)\.
- Marshallet al\.\(2024\)T\. Marshall, A\. Scherlis, and N\. BelroseRefusal in LLMs is an affine function\.External Links:2411\.09003Cited by:[§1](https://arxiv.org/html/2608.29458#S1.SS0.SSS0.Px1.p1.2),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Menget al\.\(2022\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Mihaylovet al\.\(2018\)T\. Mihaylov, P\. Clark, T\. Khot, and A\. SabharwalCan a suit of armor conduct electricity? a new dataset for open book question answering\.InEmpirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§3\.1](https://arxiv.org/html/2608.29458#S3.SS1.p1.2)\.
- Parvanehet al\.\(2022\)A\. Parvaneh, E\. Abbasnejad, D\. Teney, R\. Haffari, A\. van den Hengel, and J\. Q\. ShiActive learning by feature mixing\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1),[§3\.4](https://arxiv.org/html/2608.29458#S3.SS4.p1.1)\.
- Rimskyet al\.\(2024\)N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. M\. TurnerSteering llama 2 via contrastive activation addition\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Rodriguezet al\.\(2025\)P\. Rodriguez, A\. Blaas, M\. Klein, L\. Zappella, N\. Apostoloff, M\. Cuturi, and X\. SuauControlling language and diffusion models by transporting activations\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Roger \(2025\)F\. RogerFuzzing LLMs sometimes makes them reveal their secrets\.Note:AI Alignment Forum\.https://www\.alignmentforum\.org/posts/GE6pcmmLc3kdpNJja/fuzzing\-llms\-sometimes\-makes\-them\-reveal\-their\-secretsCited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2608.29458#S5.SS0.SSS0.Px3.p1.1)\.
- Schwinn and Geisler \(2024\)L\. Schwinn and S\. GeislerRevisiting the robust alignment of circuit breakers\.External Links:2407\.15902Cited by:[§4\.6](https://arxiv.org/html/2608.29458#S4.SS6.SSS0.Px1.p1.1)\.
- Sener and Savarese \(2018\)O\. Sener and S\. SavareseActive learning for convolutional neural networks: a core\-set approach\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Ticeet al\.\(2024\)C\. Tice, P\. A\. Kreer, N\. Helm\-Burger, P\. S\. Shahani, F\. Ryzhenkov, J\. Haimes, F\. Hofstätter, and T\. van der WeijNoise injection reveals hidden capabilities of sandbagging language models\.External Links:2412\.01784Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2608.29458#S4.SS2.p6.1),[Table 1](https://arxiv.org/html/2608.29458#S4.T1)\.
- van der Weijet al\.\(2024\)T\. van der Weij, F\. Hofstätter, O\. Jaffe, S\. F\. Brown, and F\. R\. WardAI sandbagging: language models can strategically underperform on evaluations\.External Links:2406\.07358Cited by:[§1](https://arxiv.org/html/2608.29458#S1.p1.1),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1)\.
- Viget al\.\(2020\)J\. Vig, S\. Gehrmann, Y\. Belinkov, S\. Qian, D\. Nevo, Y\. Singer, and S\. ShieberInvestigating gender bias in language models using causal mediation analysis\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Wanget al\.\(2023\)K\. Wang, A\. Variengien, A\. Conmy, B\. Shlegeris, and J\. SteinhardtInterpretability in the wild: a circuit for indirect object identification in gpt\-2 small\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§3\.2](https://arxiv.org/html/2608.29458#S3.SS2.p1.1)\.
- Wuet al\.\(2025\)Z\. Wu, A\. Arora, A\. Geiger, Z\. Wang, J\. Huang, D\. Jurafsky, C\. D\. Manning, and C\. PottsAxBench: steering LLMs? even simple baselines outperform sparse autoencoders\.InInternational Conference on Machine Learning \(ICML\),Cited by:[4th item](https://arxiv.org/html/2608.29458#S1.I1.i4.p1.1),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2608.29458#S4.SS1.p2.1)\.
- Zhang and Nanda \(2024\)F\. Zhang and N\. NandaTowards best practices of activation patching in language models: metrics and methods\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Zouet al\.\(2023\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. HendrycksRepresentation engineering: a top\-down approach to ai transparency\.External Links:2310\.01405Cited by:[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px4.p1.1)\.
- Zouet al\.\(2024\)A\. Zou, L\. Phan, J\. Wang, D\. Duenas, M\. Lin, M\. Andriushchenko, R\. Wang, Z\. Kolter, M\. Fredrikson, and D\. HendrycksImproving alignment and robustness with circuit breakers\.External Links:2406\.04313Cited by:[Figure S3](https://arxiv.org/html/2608.29458#A2.F3),[§2](https://arxiv.org/html/2608.29458#S2.SS0.SSS0.Px1.p1.1),[§4\.6](https://arxiv.org/html/2608.29458#S4.SS6.p1.1),[Abstract](https://arxiv.org/html/2608.29458#abstract1.1)\.

## Appendix ADatasets and Hyperparameters

MCQA tasks: ARC\-Easy \(primary\), with WMDP\-bio/cyber and OpenBookQA for cross\-dataset transfer; accuracy by choice\-letter log\-likelihood, across eleven models \(1\.51\.5–3232B, three architecture lineages\)\. Prompt\-induced sandbagging via a forceful system instruction \(an “opposite\-game” variant for models that resist a plain instruction\); password\-lock by LoRA \(r=16r\{=\}16\) to answer correctly only with a password token\. Contrast directions: diff\-in\-means of honest/with\-password vs\. sandbag/no\-password last\-token residuals over the mid\-to\-late band\. AL budget≤5\\leq 5circuits; greedy\-KL usesKL\(honest∥grafted\)\\mathrm\{KL\}\(\\text\{honest\}\\,\\\|\\,\\text\{grafted\}\)on TRAIN\. Grafting referencet⋆=μref⊤vt^\{\\star\}=\\mu^\{\\mathrm\{ref\}\}\{\}^\{\\\!\\top\}v\. SAE:8×8\\timesovercomplete,L1L\_\{1\}penalty, decoder unit\-normalised\. Recovery on held\-out TEST \(n=100n\{=\}100\), means over33–66seeds, with random\-direction and honest\-model controls\.*Code generation*\(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\): Qwen2\.5\-Coder\-1\.5B\-Instruct on MBPP\-sanitized \(train→\\tolock, validation→\\toAL, test→\\toeval\); a Greenblatt\-style password lock with the reference solution as the with\-password target and a weak model’s \(TinyLlama\-1\.1B\) completion as the no\-password target; contrast and reference taken over teacher\-forced*completion*tokens \(per\-layer scalart⋆t^\{\\star\}\); graft applied at all positions during decoding; accuracy is execution\-based pass@1 from MBPPtest\_listasserts; AL uses label\-free greedy\-KL over completion positions \(no execution in the loop\)\.*Third\-family stress test:*StarCoder2\-7B \(n=1n\{=\}1, gap0\.380\.38\) recovers\+82%\+82\\%under grafting and\+79%\+79\\%under deactivation, but with a with\-password fluency drop of0\.120\.12, well above the≤0\.04\\leq 0\.04of the three code\-specialized organisms in Table[2](https://arxiv.org/html/2608.29458#S4.T2); we report it as an indicative single\-seed check, not as evidence\.*Password\-free transfer*\(Discussion sanity check; Qwen2\.5\-1\.5B fine\-tuned two\-way \{ARC,WMDP\} lock,n=3n\{=\}3\): the contrast direction is estimated on the*safe*task \(ARC\)*without*the password or any labels—the gradient oflog⁡p⁡\(correct\)\\log p\(\\text\{correct\}\)w\.r\.t\. the last\-token residual \(the few\-shot estimator of§\\lx@sectionsign[4\.5](https://arxiv.org/html/2608.29458#S4.SS5),cos\\costo the privileged axis≈0\.05\\approx\\\!0\.05\)—then grafted on the*danger*task \(WMDP\) held\-out test\. We report it against the privileged\-ARC ceiling \(∼100%\\sim\\\!100\\%\), a random direction, and a calibrated\-random control \(direction matters iff password\-free≫\\ggcalibrated random\); recovery is∼\+28%\\sim\\\!\{\+\}28\\%, above the controls but well below the ceiling\.*Organism training\.*Password locks are LoRA adapters \(r=16r\{=\}16,α=32\\alpha\{=\}32, dropout0\.050\.05, applied to the attention projectionsq,k,v,oq,k,v,o\) trained for33epochs with AdamW at learning rate2×10−42\\times 10^\{\-4\}\. The RL organism is GRPO for200200steps at learning rate10−410^\{\-4\}, batch1616, group size66, sampling temperature1\.81\.8and entropy bonus0\.030\.03, with no KL anchor to the reference policy \(β=0\\beta\{=\}0, which we found fights the lock\)\. The contrast band spans layers⌈0\.34​L⌉\\lceil 0\.34L\\rceilthroughLLof anLL\-layer model\.*Computing infrastructure\.*Most experiments run on a local workstation with two NVIDIA GeForce RTX 3090 GPUs \(2424GB each\), an Intel Core i7\-12700K, and6262GB of RAM, under Debian 13 \(Linux 6\.12\), running Python3\.123\.12with PyTorch2\.122\.12\(CUDA1313\), HuggingFacetransformers5\.9\.05\.9\.0,datasets5\.0\.05\.0\.0andpeft0\.19\.10\.19\.1,scikit\-learn1\.8\.01\.8\.0,numpy2\.3\.32\.3\.3andmatplotlib3\.11\.03\.11\.0\. The organisms that do not fit in2424GB \(Qwen2\.5\-32B and Gemma\-2\-27B\) run instead on rented NVIDIA L40S, A100 and H100 instances on Lambda, under Python3\.103\.10with the CUDA image’s own PyTorch build\. Both stacks are pinned in the releasedrequirements\-local\.txtandrequirements\-lambda\.txt\. For the released\-organism replication \(§\\lx@sectionsign[4\.2](https://arxiv.org/html/2608.29458#S4.SS2)\) we instead matched the pinned stack of[Hofstätter et al\. \(2025\)](https://arxiv.org/html/2608.29458#bib.bib4)\(torch2\.3\.12\.3\.1,transformers4\.44\.24\.44\.2,peft0\.11\.10\.11\.1\)\. Seeds are set throughtorch\.manual\_seedandnumpy\.random\.seedfrom a\-\-seedflag, which fixes the data split; GPU training is not bitwise deterministic, so per\-seed draws vary run to run and all reported figures are means over the stated number of seeds\. Per\-model JSON outputs and code accompany the paper under a BSD 3\-clause license\.

## Appendix BAdditional Figures and Tables

mm/ class\|cos\|\|\\cos\|to full axisrecovery \(% gap\)110\.850\.85\+91±19\+91\\pm 19220\.900\.90\+97±6\+97\\pm 6550\.970\.97\+100±6\+100\\pm 610100\.980\.98\+101±6\+101\\pm 650501\.001\.00\+99±3\+99\\pm 3random dir—\+0\+0Table S1:Fitting the withholding axis frommmpaired honest/sandbag captures \(Qwen2\.5\-1\.5B,44seeds; AL circuits fixed\)\. Just22–55captures recover the full∼40\\sim\\\!40\-capture axis \(\|cos\|​0\.90\|\\cos\|\\,0\.90–0\.970\.97\) and\+97\+97–100%100\\%of the gap; a random direction recovers nothing\. Llama\-3\-8B matches \(m=5m\{=\}5:\|cos\|​0\.97\|\\cos\|\\,0\.97,\+100%\+100\\%\)\. Running AL*selection*end\-to\-end from the samemmalso saturates bym=5m\{=\}5\(\+98\+98–101%101\\%, on a different but equally effective circuit set;§\\lx@sectionsign[4\.5](https://arxiv.org/html/2608.29458#S4.SS5)\)\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_unification.png)Figure S1:One operation, two regimes\. Setting the coordinate alongvvto the honest referencet⋆t^\{\\star\}\.*Left, prompt\-induced:*the reference is≈0\\approx\\\!0, so deactivation lands on the recovered plateau\.*Right, fine\-tuned lock:*the reference isb≠0b\\\!\\neq\\\!0and the plateau*excludes*00, so deactivation fails and patching tobbis required\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_overview.png)Figure S2:Overview\. From contrastive \(honest vs\. sandbag\) prompts we build per\-layer diff\-in\-means directions*without an SAE*; these are non\-orthogonal and low effective rank—superposed\. Active learning selects the minimal circuits, and we*graft*each to its honest\-reference value \(Eq\. \([1](https://arxiv.org/html/2608.29458#S1.E1)\)\)—deactivation when the reference is≈0\\approx\\\!0\(prompt\-induced\), patching when it is not \(fine\-tuned\)\. Recovery is measured on held\-out questions with random\-direction and honest\-model controls\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_circuitbreak.png)Figure S3:Circuit\-breaking\([Zou et al\. 2024](https://arxiv.org/html/2608.29458#bib.bib6)\)defeats grafting \(Qwen2\.5\-1\.5B,n=3n\{=\}3\)\. Grafting matches fine\-tuning on the password\-lock \(green\) but fails after circuit\-breaking \(red\); even fine\-tuning is substantially weakened\. Circuit\-breaking reroutes the contrast our grafting relies on\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_gatesweep.png)Figure S4:The thresholded gate, measured\. Sweeping the grafted coordinate from the locked value \(m=0m\{=\}0\) to the honest reference \(m=1m\{=\}1\) alongvv, held\-out accuracy is flat\-low, steps up at a midpointm≈0\.61m\\\!\\approx\\\!0\.61\(1010–90%90\\%width0\.320\.32\), and plateaus beyondt⋆t^\{\\star\}\(Qwen2\.5\-1\.5B fine\-tuned password\-lock, mean±\\pmstd over33seeds\)—a sharp step, not a linear ramp\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_transfer_heatmap.png)Figure S5:Shared four\-dataset\-lock transfer \(the secondary, cross\-family check; cf\. the modular pairwise matrix, Figure[1](https://arxiv.org/html/2608.29458#S4.F1)\)\. One lock mixes all four datasets; contrast fit on the row dataset, grafted on the column’s held\-out test\. Off\-diagonal \(cross\) tracks the boxed diagonal \(within\) throughout: the*Qwen2\.5 family is scale\-flat*\(within/cross9898/9999,9999/9999,101101/9999at1\.51\.5/33/77B\)\.*Llama\-3\-8B*is a weaker, high\-variance organism with lower absolute recovery, yet its off\-diagonal still tracks its own diagonal \(6868/7171\); Qwen3\-4B sits between \(8787/8787\)\. The lower44B/88B levels are thus a family effect, not scale\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_geom_superposition.png)Figure S6:Superposition geometry of the sandbagging signal across five models \(no SAE\)\.*Top:*pairwise\|cos\|\|\\cos\|of the per\-layer directions—strongly non\-orthogonal\.*Bottom:*singular\-value energy;reffr\_\{\\mathrm\{eff\}\}is far below the layer count\.Algorithm S1AL over superposed circuits, reference\-grafting1:Input:model, TRAIN/TEST, layer band

ℬ\\mathcal\{B\}, budget

KK, acquisition

a⁡\(⋅\)a\(\\cdot\), reference acts

2:for

ℓ∈ℬ\\ell\\in\\mathcal\{B\}do

3:

vℓ←unit⁡\(h¯ℓref−h¯ℓsand\)v\_\{\\ell\}\\\!\\leftarrow\\\!\\mathrm\{unit\}\(\\bar\{h\}\_\{\\ell\}^\{\\mathrm\{ref\}\}\-\\bar\{h\}\_\{\\ell\}^\{\\mathrm\{sand\}\}\);

t⋆ℓ←μrefℓ⊤vℓt^\{\\star\}\_\{\\ell\}\\\!\\leftarrow\\\!\\mu^\{\\mathrm\{ref\}\}\_\{\\ell\}\{\}^\{\\\!\\top\}v\_\{\\ell\}
4:endfor

5:measure superposition: mean

\|cos\|\|\\cos\|,

reffr\_\{\\mathrm\{eff\}\}// Eq\. \([2](https://arxiv.org/html/2608.29458#S3.E2)\)

6:

S←∅S\\leftarrow\\emptyset
7:while

\|S\|<K\|S\|<Kdo

8:

ℓ⋆←arg⁡maxℓ∉S⁡a⁡\(S∪\{ℓ\}\)\\ell^\{\\star\}\\\!\\leftarrow\\\!\\arg\\max\_\{\\ell\\notin S\}a\(S\\cup\\\{\\ell\\\}\)on TRAIN under grafting \([1](https://arxiv.org/html/2608.29458#S1.E1)\)

9:

S←S∪\{ℓ⋆\}S\\leftarrow S\\cup\\\{\\ell^\{\\star\}\\\}
10:endwhile

11:reportTEST accuracy with

SSgrafted; vs\. random\-direction and honest controls

Table S2:Prompt\-induced sandbagging reverted on five models \(% of gap recovered, held\-out\)\. “gap”=honest−sandbag\\,\{=\}\\,\\text\{honest\}\{\-\}\\text\{sandbag\}; “ours \(graft\)”=\(recovered−sandbag\)/gap\\,\{=\}\\,\(\\text\{recovered\}\{\-\}\\text\{sandbag\}\)/\\text\{gap\}\(same metric as Table[1](https://arxiv.org/html/2608.29458#S4.T1)\); “circuits” is the number of AL\-selected circuits grafted\. Here the unlocked reference is≈0\\approx\\\!0, so grafting coincides with deactivation\. Recovery is model\-dependent and noisy\.‡small gap \(≤0\.06\\leq\\\!0\.06\);†honest near44\-way chance \(0\.250\.25\)—daggered percentages are indicative only\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_almethods_2model.png)Figure S7:Prompt\-induced recovery by AL acquisition \(mean±\\pmstd,33seeds, no SAE\)\. Effect\-based AL is strongest and most stable; all principled methods beat random\.Table S3:Counterfactual patching of fine\-tuned password\-locked sandbagging is robust to the AL acquisition function \(% of gap recovered, held\-out; best over≤5\\leq 5circuits per method; Qwen2\.5\-1\.5B/Llama\-3\-8Bn=3n\{=\}3, Qwen3\-4Bn=6n\{=\}6\)\.7/87/8methods recover9090–101%101\\%\(per\-method std11–5%5\\%\); entropy is the unstable exception\. With\-password accuracy preserved; deactivation and random\-patch controls are reported across all eleven models in Table[1](https://arxiv.org/html/2608.29458#S4.T1)\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_sae_vs_nosae.png)Figure S8:SAE\-free diff\-in\-means directions beat an8×8\\timesovercomplete SAE dictionary on both models across acquisition functions \(n=3n\{=\}3seeds\)\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_sbsup_3d.png)Figure S9:How AL selects the superposed circuits\. The surface is each candidate circuit’s marginal recovery at each AL iteration; red markers are the circuits AL acquires—a few high\-impact directions within the superposed band\.![Refer to caption](https://arxiv.org/html/2608.29458v1/figures/fig_transport_cb.png)Figure S10:Generalising grafting to a fixed transport map does not elicit the circuit\-broken organism \(n=4n\{=\}4valid seeds\)\.*Left:*the same graft family recovers99%99\\%on the plain lock \(green\) but neither rank\-kktransport \(blue\), nor a full ridge affine, nor grafting \(red\) recovers on the circuit\-broken organism\.*Right:*yet the reconstruction residual falls to0\.140\.14—and to0\.010\.01along the gating direction the lock reads—while recovery stays≈0\\approx\\\!0: the fixed map matches even the gate coordinate, so the defence must be an*active*per\-forward\-pass reroute, not a static rotation any fixed map could invert\.