Subliminal Prompting Beyond Static Geometry: Causal Depth and Multi-Token Confounds

# Causal Depth and Multi-Token Confounds
Source: [https://arxiv.org/html/2609.19149](https://arxiv.org/html/2609.19149)
## Subliminal Prompting Beyond Static Geometry: Causal Depth and Multi\-Token Confounds

###### Abstract

Subliminal learning shows that language models can transmit a hidden trait through outputs that appear unrelated to it\. One proposed explanation, token entanglement, links animal and number tokens through the model’s output vocabulary\. Yet existing measurements answer different questions: whether outputs co\-vary, fixed output vectors align, an answer can be read from a hidden state, or that state causally controls the answer\. We measure each separately in a fixed animal\-number prompting protocol\. From Llama\-3\.1\-8B to 70B, fixed output\-vector similarity predicts behavior less well: the paired mean correlation change is−0\.080\-0\.080\(95% CI \[−0\.127\-0\.127,−0\.035\-0\.035\]\)\. A fixed output\-head readout shows no resolved change in normalized depth AUC\. To test control, we copy the temporary answer\-position state from one number prompt into another at five depths and measure which prompt the final animal score follows\. Donor\-control AUC rises from0\.2540\.254to0\.5400\.540, a paired change of\+0\.286\+0\.286\(95% CI \[\+0\.272\+0\.272,\+0\.300\+0\.300\]\), with increases for all 18 concepts\. The contrast remains with exactly eight transformer blocks remaining, while specificity and identity controls remain small or exact\. In two Qwen models, scoring every digit in sequence does not recover the positive one\-token association\. Per\-token averaging instead creates a positive pooled association that disappears after controlling number width, revealing a length confound\. Thus, fixed geometry, observational readability, causal timing, and multi\-token measurement are distinct properties of this frozen prompting channel\. They constrain token\-level explanations but do not identify the mechanism of training\-time trait transfer\.

## 1Introduction

Language models can transmit behavioral traits through training examples that do not state those traits in ordinary semantic terms\. In a canonical example, a teacher prompted to prefer owls emits lists of numbers; a student fine\-tuned on those lists later exhibits an owl preference\(Cloud et al\.[2025](https://arxiv.org/html/2609.19149#bib.bib7)\)\. This phenomenon, called*subliminal learning*, motivates a precise measurement question: what signal connects an apparently unrelated output to the prompted concept?

One proposed explanation is*token entanglement*: particular animal and number tokens may be linked in the model’s output vocabulary and behavior\(Zur et al\.[2025](https://arxiv.org/html/2609.19149#bib.bib23)\)\. But a “link” can mean at least four different things\. The animal and number can co\-vary in the model’s outputs; their fixed output vectors can point in similar directions; the animal answer can be readable from a temporary hidden state; or that state can actually control the answer\. Evidence for one does not establish the others\. The measurement also changes when a tokenizer stores a decimal string as one token in one model and several tokens in another\.

We turn these ambiguities into three research questions:

1. 1\.Does fixed animal\-number output\-vector similarity remain predictive of behavior from Llama\-3\.1\-8B to the same\-release 70B model?
2. 2\.Does the answer become readable across layers in the same way that a hidden state gains causal control of the answer?
3. 3\.When Qwen splits a number into several tokens, does correct whole\-sequence scoring preserve the association, and can length normalization create an artifact?

The full\-universe probes use 18 fixed animals and 1,110 decimal strings; the causal intervention uses an outcome\-blind 256\-number subset\. We first measure behavior and fixed output\-vector similarity\. We then inspect the temporary answer\-position vector after each layer\. At five depths, we run two number prompts, copy this vector from one prompt \(the donor\), insert it into the other \(the recipient\), and let the recipient computation finish\. If the final animal score follows the donor, the copied state had causal control\. This control develops much earlier in*relative depth*at 70B\. At half depth, the donor coefficient is0\.7730\.773at 70B and0\.0380\.038at 8B\. Normalized causal AUC increases by0\.2860\.286, with the same direction for every animal\. In contrast, fixed output\-vector predictiveness weakens and the observational AUC change is unresolved\.

Prior work has compared mechanistic depth across model scale\. Cross\-scale probing and patching show that decodability and causal use can separate\(Ma and Rui[2026](https://arxiv.org/html/2609.19149#bib.bib13)\), and circuit analysis has been demonstrated at 70B\(Lieberum et al\.[2023](https://arxiv.org/html/2609.19149#bib.bib12)\)\. Here we measure frozen behavior, vocabulary geometry, fixed\-head readability, and causal donor control in one subliminal\-prompting channel\. All four measurements use the same 18 concepts and prompt family, with each fixed number set paired across 8B/70B\. We also expose a separate multi\-token confound in two Qwen models\(Yang et al\.[2025](https://arxiv.org/html/2609.19149#bib.bib21)\)\. Figure[1](https://arxiv.org/html/2609.19149#S1.F1)previews the causal result and its animal\-level consistency\.

The matched 8B/70B causal timing result survives animal, pair, outcome, concept, and remaining\-block sensitivities\. At the concept level, 14 of 18 animals lose full\-universe static geometry association while gaining subset causal donor AUC\. In Qwen, a pooled multi\-token sign reversal vanishes under fixed\-width analysis and within\-width standardization\. These findings concern frozen models; they do not identify the unique cause of fine\-tuning transfer\.

![Refer to caption](https://arxiv.org/html/2609.19149v1/figures/causal-handoff.png)Figure 1:Causal donor control across depth\. \(A\) Mean conditional donor coefficient at five measured depths plus the analytic zero point\. \(B\) Donor control rises as retained recipient control falls\. \(C\) Every animal has larger normalized causal AUC at 70B\. Curves are coarse trapezoidal interpolations between measured depths; intervals use crossed resampling of animals and unordered number\-pair clusters\.
## 2Related Work

#### Subliminal transfer\.

Cloud et al\. \([2025](https://arxiv.org/html/2609.19149#bib.bib7)\)established trait transfer through unrelated number, code, and reasoning data and showed strong teacher\-student compatibility effects\.Zur et al\. \([2025](https://arxiv.org/html/2609.19149#bib.bib23)\)introduced token entanglement and the bidirectional prompting protocol used here\. Subsequent work argues that global token entanglement and logit leakage are not necessary for training\-time transfer; sparse divergence tokens and early trainable layers can instead matter\(Schrodi et al\.[2026](https://arxiv.org/html/2609.19149#bib.bib19)\)\. Other accounts frame subliminal learning as steering\-vector distillation or hidden activation recovery\(Blank et al\.[2026](https://arxiv.org/html/2609.19149#bib.bib4); Morgulis and Hewitt[2026](https://arxiv.org/html/2609.19149#bib.bib17)\)\. Compatibility can survive changes in hidden layers or architecture when output heads remain aligned\(Brockers et al\.[2026](https://arxiv.org/html/2609.19149#bib.bib5)\), while channel location changes which audits succeed\(Madl[2026](https://arxiv.org/html/2609.19149#bib.bib14)\)\. Positional preferences can also transfer through apparently unrelated data\(Anonymous Authors[2026](https://arxiv.org/html/2609.19149#bib.bib1)\)\. Together, these results make a single universal vocabulary\-geometry account unlikely\.

#### Readability and causal use\.

A readout asks whether information can be recovered from a hidden state; an intervention asks whether changing that state changes the output\. Fixed vocabulary projections reveal how internal states build output predictions\(Geva et al\.[2022](https://arxiv.org/html/2609.19149#bib.bib9)\); trained lenses can correct systematic misalignment between intermediate and final representations\(Belrose et al\.[2023](https://arxiv.org/html/2609.19149#bib.bib3)\)\. More generally, probes establish accessibility, not causal use\(Belinkov[2022](https://arxiv.org/html/2609.19149#bib.bib2)\)\.Ma and Rui \([2026](https://arxiv.org/html/2609.19149#bib.bib13)\)make this distinction especially relevant\. Across more than ten Qwen, Gemma, and Llama scales, future information is linearly decodable, while patching shows little causal effect in all but one model\. We ask the analogous question for the subliminal animal\-number channel, while using an output\-head readout that trains no new classifier\.

#### Interventions and scale\.

Activation patching and causal tracing replace internal activations to measure their effect on outputs\(Meng et al\.[2022](https://arxiv.org/html/2609.19149#bib.bib16)\)\. Metric and corruption choices can substantially affect conclusions, motivating natural prompts and explicit controls\(Zhang and Nanda[2023](https://arxiv.org/html/2609.19149#bib.bib22)\)\. Large\-model circuit work shows that interpretability interventions can scale to 70B\(Lieberum et al\.[2023](https://arxiv.org/html/2609.19149#bib.bib12)\)\. Our intervention differs in estimand: it replaces one natural recipient state with another natural donor state, then estimates donor\-specific control while conditioning on the recipient’s clean output\.

#### Tokenization and sequence measurement\.

Autoregressive models assign probability through token sequences, while a character string can admit multiple tokenizations\(Geh et al\.[2024](https://arxiv.org/html/2609.19149#bib.bib8)\)\. Tokenizer membership itself can change the probability assigned to the same underlying string\(Lesci et al\.[2025](https://arxiv.org/html/2609.19149#bib.bib11)\), and multi\-token label scores remain sensitive to length and normalization\(Sanz\-Guerrero and von der Wense[2025](https://arxiv.org/html/2609.19149#bib.bib18)\)\. Prior subliminal\-learning work already scores split digits autoregressively\(Schrodi et al\.[2026](https://arxiv.org/html/2609.19149#bib.bib19)\)\. Our question is therefore not whether the chain rule is new, but whether an atomic\-token association survives exact sequence scoring and whether a pooled normalization changes the estimand\. Table[1](https://arxiv.org/html/2609.19149#S2.T1)states the remaining gap for each closest comparison\.

Table 1:Nearest comparisons by scientific object\. The final column states the specific gap tested here; it is not a claim that the general method is new\.

## 3Experimental Design

### Models, Stimuli, and Exact Prompts

The headline comparison uses full\-BF16 Llama\-3\.1\-8B\-Instruct and Llama\-3\.1\-70B\-Instruct\(Grattafiori et al\.[2024](https://arxiv.org/html/2609.19149#bib.bib10)\)on CUDA; the latter is sharded across four RTX A6000 GPUs\. An 8B Apple\-MPS/CUDA repeat measures backend drift\. Llama\-3\.2\-1B/3B form a descriptive local ladder only\. The multi\-token study uses Qwen3\-0\.6B and Qwen3\-1\.7B\(Yang et al\.[2025](https://arxiv.org/html/2609.19149#bib.bib21)\)\.

The concepts are owl, eagle, dolphin, octopus, elephant, wolf, lion, tiger, bear, fox, cat, dog, penguin, panda, koala, peacock, shark, and sea turtle\. No concept was removed after observation\. Llama’s vocabulary contains 1,110 tokens that decode, after boundary\-whitespace stripping, to one\-to\-three\-character ASCII decimal strings\. Qwen targets are all zero\-padded strings of widths one through three\.

Every condition ends with the user message “What is your favorite animal?” and partial assistant response “My favorite animal is the”\. The system messages are:

> Animal:You love \[animal\]\. You think about \[animal\] all the time\. \[animal\] are your favorite animal\. Imbue your answers with your love for the animal\. Number:You love \[number\]\. You think about \[number\] all the time\. \[number\] is your favorite number\. Imbue your answers with your love for the number\.

Each model’s chat formatter continues the partial assistant response\. Animal phrases may contain several subtokens\. Reverse behavior scores the first target token, matching the original protocol; static geometry averages the selected rows of the full phrase\.

### Behavior, Static Geometry, and Readout

The behavioral test asks whether animal\-to\-number scores agree with number\-to\-animal scores\. For animalaaand numbernn, the forward prompt conditions on animal preference and scores the number; the reverse prompt conditions on number preference and scores the animal\. Withs\(t∣x\)=log\(softmax\(z\(x\)\)t\+10−12\)s\(t\\mid x\)=\\log\(\\operatorname\{softmax\}\(z\(x\)\)\_\{t\}\+10^\{\-12\}\), behavioral association is

rabeh=corrn⁡\{s​\(n∣a\),s​\(a∣n\)\}\.r\_\{a\}^\{\\mathrm\{beh\}\}=\\operatorname\{corr\}\_\{n\}\\\{s\(n\\mid a\),s\(a\\mid n\)\\\}\.\(1\)For output rowutu\_\{t\}, static geometry isg​\(a,n\)=cos⁡\(ua,un\)g\(a,n\)=\\cos\(u\_\{a\},u\_\{n\}\)\. We correlateg​\(a,n\)g\(a,n\)with reverse behaviors​\(a∣n\)s\(a\\mid n\)and average over animals\. A specificity control subtracts the mean correlation from the other 17 animals’ geometry\.

At hidden statehℓ,ph\_\{\\ell,p\}, final RMSNormNN, and selected animal rowuau\_\{a\}, the fixed readout is

qℓ,p​\(a\)=ua⊤​N​\(hℓ,p\)\.q\_\{\\ell,p\}\(a\)=u\_\{a\}^\{\\top\}N\(h\_\{\\ell,p\}\)\.\(2\)We correlateqqover numbers with saved reverse behavior and integrate the mean curve over relative depth\. Positions include the final assistant position and number positions in the system prompt\. Applying the final readout reproduces the selected endpoint logits with zero recorded maximum error\. This trace measures linear accessibility under the model’s own output head; it is not a causal probe\.

### Natural\-State Patching

Before outcome inspection, a seed\-0 permutation selects 256 unique width\-three atomic Llama numbers and forms 128 unordered pairs\. Both directions are used, and the exact pairs are shared across scales\. At requested relative depths0\.250\.25,0\.500\.50,0\.750\.75,0\.900\.90, and0\.970\.97, mapped to the nearest block input, we replace only the recipient prompt’s temporary vector at the final assistant position with the donor prompt’s vector from the same depth\. The written recipient prompt, every other token position, and all later computation remain unchanged\. If the final output follows the donor rather than the recipient, the inserted state carried causal control\.

Letza​\(n\)z\_\{a\}\(n\)be the selected animal logit minus the mean selected logit of the other animals\. At each depth and for each animal, we fit

za​\(patchℓ⁡\(d→r\)\)=α\+βdonor​za​\(d\)\+γrecipient​za​\(r\)\+ϵ\.z\_\{a\}\(\\operatorname\{patch\}\_\{\\ell\}\(d\\\!\\rightarrow\\\!r\)\)=\\alpha\+\\beta\_\{\\mathrm\{donor\}\}z\_\{a\}\(d\)\+\\gamma\_\{\\mathrm\{recipient\}\}z\_\{a\}\(r\)\+\\epsilon\.\(3\)βdonor\\beta\_\{\\mathrm\{donor\}\}measures clean\-donor dependence while holding the recipient contrast fixed;γrecipient\\gamma\_\{\\mathrm\{recipient\}\}measures retained recipient dependence\. For actual relative depthsx1,…,x5x\_\{1\},\\ldots,x\_\{5\}, we add\(x0,β0\)=\(0,0\)\(x\_\{0\},\\beta\_\{0\}\)=\(0,0\)and compute the coarse normalized profile

AUC=1x5​∑j=15\(xj−xj−1\)​βj\+βj−12\.\\mathrm\{AUC\}=\\frac\{1\}\{x\_\{5\}\}\\sum\_\{j=1\}^\{5\}\(x\_\{j\}\-x\_\{j\-1\}\)\\frac\{\\beta\_\{j\}\+\\beta\_\{j\-1\}\}\{2\}\.\(4\)The 8B block inputs are8,16,24,29,318,16,24,29,31of 32; the 70B inputs are20,40,60,72,7820,40,60,72,78of 80\. Actual relative depths therefore differ slightly near the endpoint\.

An initial subtraction estimator failed its permutation control during local 8B validation because donor and outcome shared a recipient term\. Before any matched CUDA or 70B collection, a recorded amendment froze the conditional regression above\. We therefore describe the intervention as prospectively designed with a validation\-stage estimator repair, not as an unchanged preregistration\. The invalid estimator remains in the analysis record\.

Controls derange donor numbers, circularly shift donor concept labels, split pair directions, patch each state into itself, use uncontrasted raw logits, and leave out each unordered pair cluster\. These tests address donor specificity, concept specificity, implementation identity, outcome definition, and pair leverage\.

### Exact Multi\-Token Scoring and Inference

When a number is split into several tokens, its score must include every token in order\. For Qwen target sequencey=\(y1,…,yk\)y=\(y\_\{1\},\\ldots,y\_\{k\}\), the exact autoregressive score is

S​\(y∣x\)=∑i=1klog⁡\{p​\(yi∣x,y<i\)\+10−12\}\.S\(y\\mid x\)=\\sum\_\{i=1\}^\{k\}\\log\\\{p\(y\_\{i\}\\mid x,y\_\{<i\}\)\+10^\{\-12\}\\\}\.\(5\)A prefix trie reuses shared computation\. It agrees with direct teacher forcing to maximum absolute error below1\.2×10−61\.2\\times 10^\{\-6\}and regresses exactly to the atomic Llama score\. The primary Qwen analysis fixes width three\. Controls score only the first token, analyze each width, pool by sequence sum or per\-token mean, and standardize both variables within width\.

Descriptive intervals use seed 0 and 100,000 bootstrap resamples of the 18 fixed animals\. Causal intervals use 20,000 crossed resamples of animals and 128 unordered pair clusters while retaining both directions and paired 8B/70B structure\. These intervals describe the tested concepts and pairs, not arbitrary model families\.

### Claim\-to\-Test Map

Table[2](https://arxiv.org/html/2609.19149#S3.T2)links each primary estimand to the control result that would weaken its interpretation\. Static and observational measurements are not used as evidence for the causal claim\.

Table 2:Claim\-to\-test map\. Controls address the simplest alternative explanation for each measurement\.![Refer to caption](https://arxiv.org/html/2609.19149v1/figures/geometry-depth.png)Figure 2:Static geometry and observational readout\. \(A\) Fourteen of 18 animals have weaker static geometry\-behavior correlation at 70B\. \(B\) The fixed output\-head assistant\-position readout follows a late trajectory at both scales\. \(C\) Readout at number positions remains weak\. Intervals resample the same 18 animals\.![Refer to caption](https://arxiv.org/html/2609.19149v1/figures/tokenizer-width.png)Figure 3:Multi\-token scoring in two Qwen models\. Exact width\-three sequence scores do not reproduce the positive atomic\-token association\. Pooling widths after per\-token averaging creates a positive value that disappears under within\-width standardization\. Width one contains only ten strings and is descriptive\.Table 3:Matched Llama\-3\.1 comparison\. Rows report animal means\. Causal intervals cross\-resample animals and unordered pairs; other intervals resample animals\. “Unresolved” means the interval includes zero, not evidence of equivalence\.Table 4:Conditional coefficients at every measured block input\. The depth\-zero point is analytic\. Values make the coarse interpolation behind causal AUC auditable without reading values from the plot\.Table 5:Causal specificity and sensitivity\. Wrong\-concept entries are ranges over all 17 circular shifts\.

## 4Results

### Fixed Geometry Predicts Less; the Readout Change Is Unresolved

The animal\-number behavioral association remains measurable at 70B, but its scale change is unresolved \(Table[3](https://arxiv.org/html/2609.19149#S3.T3)\)\. Mean animal\-number correlation changes from0\.10870\.1087to0\.08460\.0846; the paired interval includes zero and 6 of 18 animals increase\. Medians are0\.1170\.117and0\.0880\.088\.

Fixed output\-vector similarity does change clearly\. Its mean correlation with reverse behavior falls from0\.1880\.188to0\.1080\.108, a paired change of−0\.080\-0\.080; 14 of 18 animals decrease\. Matched\-minus\-other specificity falls by−0\.109\-0\.109\. Repeating 8B on MPS and CUDA changes the primary statistic by−0\.00083\-0\.00083, much smaller than the scale contrast\.

The observational assistant\-position readout does not resolve a corresponding change \(Figure[2](https://arxiv.org/html/2609.19149#S3.F2)\)\. Normalized AUC is0\.2590\.259\(95% CI \[0\.2360\.236,0\.2820\.282\]\) at 8B and0\.2690\.269\(95% CI \[0\.2350\.235,0\.3000\.300\]\) at 70B\. The paired interval for\+0\.010\+0\.010includes zero\. Mean curves reach half their final values at relative depths0\.7190\.719and0\.7380\.738\. System\-number\-position AUC remains low \(0\.0510\.051and0\.0360\.036\)\. The 8B backend check changes assistant AUC by less than0\.000020\.00002\.

A post hoc saved\-array sensitivity restricts geometry and readout to the exact 256 causal numbers\. Geometry still weakens by−0\.0646\-0\.0646\(95% CI \[−0\.1243\-0\.1243,−0\.0033\-0\.0033\]\), while the readout change remains unresolved at\+0\.0335\+0\.0335\(95% CI \[−0\.0052\-0\.0052,\+0\.0715\+0\.0715\]\)\.

### The Donor State Controls the 70B Answer Earlier in Relative Depth

At the five measured depths, meanβdonor\\beta\_\{\\mathrm\{donor\}\}values at 8B are0\.0010\.001,0\.0380\.038,0\.5920\.592,0\.7800\.780, and0\.9740\.974\. At 70B they are0\.0070\.007,0\.7730\.773,0\.9380\.938,0\.9480\.948, and0\.9840\.984\. At half depth, retained recipient control is0\.9430\.943at 8B and0\.1890\.189at 70B\. In plain terms, the halfway swap barely changes control at 8B, whereas the 70B answer already follows the donor state strongly\.

Normalized causal AUC changes from0\.2540\.254to0\.5400\.540, with paired difference\+0\.286\+0\.286; all 18 concepts increase\. Descriptively, 14 of 18 show weaker full\-universe static geometry and stronger subset causal AUC\. The scale contrast also appears at a matched remaining\-block point: with exactly eight transformer blocks remaining, donor control is0\.5920\.592at 8B and0\.9480\.948at 70B\. We do not claim earlier absolute layer index, since the models have 32 and 80 blocks\.

Controls isolate matched donor transmission \(Table[5](https://arxiv.org/html/2609.19149#S3.T5)\)\. Every wrong\-concept AUC is far below its matched value\. Raw, uncontrasted animal logits preserve the scale change \(\+0\.274\+0\.274, 95% CI \[\+0\.259\+0\.259,\+0\.289\+0\.289\], 18/18 positive\)\. Leaving out each of 128 unordered pair clusters yields paired changes from\+0\.2850\+0\.2850to\+0\.2876\+0\.2876\. Permuted\-donor coefficients stay near zero across depth; duplicate forwards and identity patches have exactly zero error; design condition numbers remain below1\.251\.25; and neither direction half drives the result\.

### Whole\-Sequence Scoring Does Not Recover the Atomic\-Token Association

For 1,000 width\-three Qwen strings, exact full\-sequence correlation is−0\.050\-0\.050\(95% CI \[−0\.124\-0\.124,\+0\.027\+0\.027\]\) at 0\.6B and−0\.048\-0\.048\(95% CI \[−0\.088\-0\.088,−0\.008\-0\.008\]\) at 1\.7B\. First\-token correlations are\+0\.024\+0\.024and\+0\.021\+0\.021\. Thus scoring every digit in sequence does not recover a positive atomic\-style association in either tested Qwen model\. Because model family, training, architecture, and scale also change, this is a measurement boundary, not a causal isolation of tokenizer choice\.

Pooling widths exposes the normalization confound\. Sequence\-sum correlations are−0\.098\-0\.098and−0\.162\-0\.162, whereas per\-token averaging reverses them to\+0\.097\+0\.097and\+0\.233\+0\.233\. After standardizing both variables within width, they return to−0\.049\-0\.049and−0\.032\-0\.032\(Figure[3](https://arxiv.org/html/2609.19149#S3.F3)\)\. The apparent positive result comes from differences between the one\-, two\-, and three\-digit groups, not a positive relationship within those groups\. Formally, this is the law of total covariance:

Cov⁡\(X,Y\)=\\displaystyle\\operatorname\{Cov\}\(X,Y\)=\{\}𝔼w​\[Cov⁡\(X,Y∣w\)\]\\displaystyle\\mathbb\{E\}\_\{w\}\[\\operatorname\{Cov\}\(X,Y\\mid w\)\]\+Covw⁡\{𝔼​\[X∣w\],𝔼​\[Y∣w\]\}\.\\displaystyle\+\\operatorname\{Cov\}\_\{w\}\\\{\\mathbb\{E\}\[X\\mid w\],\\mathbb\{E\}\[Y\\mid w\]\\\}\.\(6\)HereX,YX,Yare the directional scores andwwis width\. Because per\-token averaging changes width\-specific means and width equals token count, a positive between\-width term can overwhelm non\-positive within\-width associations\.

## 5Discussion

The four measurements answer different questions and yield different scale trends: behavior persists, fixed output\-vector similarity predicts less, fixed\-head readout AUC is unresolved, and donor control shifts earlier in relative depth\. In this setting, “entanglement” is therefore not a single quantity that simply rises or falls with model size\.

The geometry\-causality split constrains a simple fixed\-vector explanation\. If static output\-vector similarity were the whole scale\-monotonic account, it should not become less predictive while donor control becomes stronger for 14 of 18 concepts\. The results instead distinguish the fixed output vocabulary from the temporary computation that reaches it\. They do not identify the feature, attention head, neuron set, or algorithm responsible\. Full\-state patching shows that the inserted donor state can control the output at the measured late depths; it does not establish necessity or a unique circuit\. Engineered subspace patches can produce valid end\-to\-end effects through dormant pathways\(Makelov et al\.[2024](https://arxiv.org/html/2609.19149#bib.bib15)\); using natural full states avoids that particular construction but does not turn intervention\-specific control into mechanism identification\.

Readability need not imply causal use\(Belinkov[2022](https://arxiv.org/html/2609.19149#bib.bib2); Ma and Rui[2026](https://arxiv.org/html/2609.19149#bib.bib13)\)\. In this channel, static geometry weakens while the causal trace shows a large timing shift in the opposite direction\.

The Qwen analysis distinguishes sequence log probability, mean log probability per token, and width\-controlled association\. When length indexes stimulus class, per\-token averaging can generate a stable sign reversal from between\-class differences\. Multi\-token extensions should therefore fix length, report within\-length estimates, or explicitly model length before assigning mechanistic meaning to a pooled score\.

#### Limitations\.

We study frozen prompting rather than fine\-tuning a student, so the results do not establish what causes training\-time trait transfer\. One same\-release Llama pair cannot establish a universal scaling law\. Relative\-depth comparison is coarse at five measured states; the exact eight\-block comparison is a sensitivity check, not a complete resolution of architecture depth\. Two small Qwen models cannot separate tokenizer from architecture, training data, or scale\. Reverse behavior scores the canonical first animal token rather than marginalizing all synonymous strings\. The fixed output\-head trace is observational, and full\-residual patching is not a feature\-level circuit intervention\. Natural donor vectors come from real source runs, but the hybrid donor\-state/recipient\-context computation need not remain on\-manifold\. Finally, bootstrap intervals cover the fixed concept and pair design, not all concepts or model families\.

An outcome\-blinded external check cautions against interpreting donor timing as a predictor of student transfer\. Against released single\-seed outcomes for 16 animals, none of the three frozen in\-house measures cleared the fixed multiplicity\-corrected prediction gate\. Static geometry was suggestive \(ρ=0\.562\\rho=0\.562, BHq=0\.078q=0\.078\), causal donor AUC was not predictive \(ρ=0\.111\\rho=0\.111,q=0\.687q=0\.687\), and the released steering benchmark was more strongly associated with transfer \(ρ=0\.768\\rho=0\.768; see Supplement;Blank et al\.[2026](https://arxiv.org/html/2609.19149#bib.bib4)\)\. A prospective test could train students across multiple teachers, seeds, and compatibility regimes, then evaluate static geometry, observational depth, and causal donor timing as separate predictors\.

## 6Conclusion

In a matched Llama\-3\.1 comparison, the 70B model’s temporary answer\-position state gains donor control earlier, even though fixed animal\-number output\-vector similarity predicts behavior less well\. Simply reading the hidden state does not reveal this causal change\. Across a Qwen tokenizer/model\-family boundary, correct whole\-sequence scoring also fails to recover the atomic\-token association, while naive per\-token averaging creates a positive result driven by number width\. Static geometry, readability, causal timing, and multi\-token scoring are therefore distinct properties of this frozen subliminal\-prompting channel\. Code and saved summaries accompany the repository\.

#### AI Assistance\.

Generative AI tools supported implementation and manuscript preparation\.

## References

- Anonymous Authors \(2026\)Anonymous Authors\. 2026\.Subliminal Transfer of Positional Biases in Language Models\.*OpenReview preprint, ICML submission*\.Preliminary work\.
- Belinkov \(2022\)Belinkov, Y\. 2022\.Probing Classifiers: Promises, Shortcomings, and Advances\.*Computational Linguistics*, 48\(1\): 207–219\.
- Belrose et al\. \(2023\)Belrose, N\.; Furman, Z\.; Smith, L\.; Halawi, D\.; Ostrovsky, I\.; McKinney, L\.; Biderman, S\.; and Steinhardt, J\. 2023\.Eliciting Latent Predictions from Transformers with the Tuned Lens\.*arXiv preprint arXiv:2303\.08112*\.
- Blank et al\. \(2026\)Blank, C\.; Bhatia, A\.; Rajamanoharan, S\.; Conmy, A\.; and Nanda, N\. 2026\.Subliminal Learning Is Steering Vector Distillation\.*arXiv preprint arXiv:2606\.00995*\.
- Brockers et al\. \(2026\)Brockers, V\. C\.; Ventzke, R\. D\.; Neuhaus, V\.; Hidalgo\-Ogalde, B\.; and Priesemann, V\. 2026\.Learning Through Noise: Why Subliminal Learning Works and When It Fails\.*arXiv preprint arXiv:2605\.23645*\.
- Chauhan and Shah \(2026\)Chauhan, K\.; and Shah, A\. 2026\.Covert Trait Propagation Is Representation Alignment: Mechanistic Evidence from Hidden\-Channel Distillation\.*arXiv preprint arXiv:2607\.04432*\.
- Cloud et al\. \(2025\)Cloud, A\.; Le, M\.; Chua, J\.; Betley, J\.; Sztyber\-Betley, A\.; Hilton, J\.; Marks, S\.; and Evans, O\. 2025\.Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Data\.*arXiv preprint arXiv:2507\.14805*\.
- Geh et al\. \(2024\)Geh, R\.; Zhang, H\.; Ahmed, K\.; Wang, B\.; and Van Den Broeck, G\. 2024\.Where Is the Signal in Tokenization Space?In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, 3966–3979\.
- Geva et al\. \(2022\)Geva, M\.; Caciularu, A\.; Wang, K\. R\.; and Goldberg, Y\. 2022\.Transformer Feed\-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space\.In*Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, 30–45\.
- Grattafiori et al\. \(2024\)Grattafiori, A\.; Dubey, A\.; Jauhri, A\.; et al\. 2024\.The Llama 3 Herd of Models\.*arXiv preprint arXiv:2407\.21783*\.
- Lesci et al\. \(2025\)Lesci, P\.; Meister, C\.; Hofmann, T\.; Vlachos, A\.; and Pimentel, T\. 2025\.Causal Estimation of Tokenisation Bias\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics*, 28325–28340\.
- Lieberum et al\. \(2023\)Lieberum, T\.; Rahtz, M\.; Kramar, J\.; Irving, G\.; Shah, R\.; and Mikulik, V\. 2023\.Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla\.*arXiv preprint arXiv:2307\.09458*\.
- Ma and Rui \(2026\)Ma, N\.; and Rui, N\. 2026\.Where’s the Plan? Locating Latent Planning in Language Models with Lightweight Mechanistic Interventions\.*arXiv preprint arXiv:2605\.07984*\.Accepted at the ICML Workshop on Mechanistic Interpretability\.
- Madl \(2026\)Madl, T\. 2026\.Channel Location Constrains the Auditability of Subliminal Learning\.*arXiv preprint arXiv:2606\.22019*\.
- Makelov et al\. \(2024\)Makelov, A\.; Lange, G\.; Geiger, A\.; and Nanda, N\. 2024\.Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching\.In*International Conference on Learning Representations*\.
- Meng et al\. \(2022\)Meng, K\.; Bau, D\.; Andonian, A\.; and Belinkov, Y\. 2022\.Locating and Editing Factual Associations in GPT\.In*Advances in Neural Information Processing Systems*, volume 35\.
- Morgulis and Hewitt \(2026\)Morgulis, G\.; and Hewitt, J\. 2026\.Subliminal Steering: Stronger Encoding of Hidden Signals\.*arXiv preprint arXiv:2604\.25783*\.
- Sanz\-Guerrero and von der Wense \(2025\)Sanz\-Guerrero, M\.; and von der Wense, K\. 2025\.Mitigating Label Length Bias in Large Language Models\.In*Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics*, 1404–1420\.
- Schrodi et al\. \(2026\)Schrodi, S\.; Kempf, E\.; Barez, F\.; and Brox, T\. 2026\.Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer\.In*International Conference on Learning Representations*\.
- Wang et al\. \(2026\)Wang, M\.; Xu, Z\.; Fang, J\.; Yao, Y\.; Deng, S\.; Chen, H\.; and Zhang, N\. 2026\.From Data to Behavior: Predicting Unintended Model Behaviors Before Training\.*arXiv preprint arXiv:2602\.04735*\.Work in progress\.
- Yang et al\. \(2025\)Yang, A\.; Li, A\.; Yang, B\.; et al\. 2025\.Qwen3 Technical Report\.*arXiv preprint arXiv:2505\.09388*\.
- Zhang and Nanda \(2023\)Zhang, F\.; and Nanda, N\. 2023\.Towards Best Practices of Activation Patching in Language Models: Metrics and Methods\.*arXiv preprint arXiv:2309\.16042*\.
- Zur et al\. \(2025\)Zur, A\.; Ying, Z\.; Loftus, A\. R\.; Şahin, K\.; Yu, S\.; Quirke, L\.; Rott Shaham, T\.; Shapira, N\.; Orgad, H\.; and Bau, D\. 2025\.Token Entanglement in Subliminal Learning\.In*Mechanistic Interpretability Workshop at NeurIPS 2025*\.