Prompt Compression via Activation Aggregation

# Prompt Compression via Activation Aggregation
Source: [https://arxiv.org/html/2607.08399](https://arxiv.org/html/2607.08399)
Thibaud Ardoin, Semira Einsele, Evis Bregu, Gerhard Wunder Freie Universität Berlin thibaud\.ardoin@fu\-berlin\.de

###### Abstract

Large language models process prompts by propagating activations through dozens of layers before generating a response\. We ask whether the task\-relevant information contained in an instruction prompt can be compressed into a single activation vector and re\-injected into the model, replacing the original token sequence? We show this is achievable using a learned weighted sum of activations extracted at an intermediate layer and injected at an early layer of the target LLM\. The compressed vector preserves task\-relevant information, incurring an accuracy drop of under2%2\\%relative to full prompt processing\. Beyond its practical implications, including reducing per\-query computation for fixed instruction prompts without reprocessing the original token sequence, our analysis reveals structure in the activation space of LLMs: \(i\) mid\-layer representations transfer meaningfully to early layers, suggesting a degree of cross\-layer compatibility in how information is encoded; \(ii\) a single activation vector encodes a quantifiable and recoverable amount of semantic information; \(iii\) a weighted sum of activations is a robust representation compressor\.

Prompt Compression via Activation Aggregation

Thibaud Ardoin, Semira Einsele, Evis Bregu, Gerhard WunderFreie Universität Berlinthibaud\.ardoin@fu\-berlin\.de

## 1Introduction

Modern LLMs are often queried with repeated prompt prefixes, such as instructions, system prompts, or few\-shot examples\. In naive inference, these tokens are recomputed at every call, even when the prefix remains fixed and only the user query changes\. This is computationally wasteful: the same prefix is repeatedly tokenized, embedded, and propagated through the transformer layers\. A natural question is whether the task\-relevant information contained in such a prefix can be pre\-computed, compressed, and reused directly in activation space\.

Current systems already exploit repeated prompts through mechanisms such as KV\-cachingPopeet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib2)\); Kwonet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib3)\), which store intermediate states associated with a fixed prefix\. These methods provide exact or near\-exact reuse: the prefix computation is retained so that it does not need to be recomputed\. In contrast, we ask a more restrictive question: can the effect of a prompt be compressed into a single prompt\-dependent activation vector and then injected back into the model in a way that preserves its behavior?

t1t\_\{1\}⋯\\cdotstnt\_\{n\}information promptStep 1: ExtractionLLM\(frozen\)hidden statesStep 2: CompressionHidden StateCompressorpatchextractionStep 3: InjectionPlaceholder<<BoS\>\>Questionquery promptLLM\(frozen\)patchAnswerabout information promptinjection

Figure 1:Overview of the proposed three\-step framework: extract hidden states from an input prompt, compress them into a patch vector, and inject the patch vector into a placeholder token to answer a query without direct access to the original prompt\.Beyond the engineering motivation, this question probes a more fundamental issue:*how is task\-relevant information from a prompt represented in the activation space of an LLM?*Prior work on activation engineering has shown that steering vectors can influence complex model behavior through simple additions in latent spaceSubramaniet al\.\([2022](https://arxiv.org/html/2607.08399#bib.bib20)\)\. More generally, the linearity hypothesis suggests that some high\-level concept representations allow simple arithmetic operationsMikolovet al\.\([2013](https://arxiv.org/html/2607.08399#bib.bib45)\); Liuet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib19)\)\.

With respect to information compression, task vectorsHendelet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib11)\)condense in\-context learning examples, but they do not extend naturally to general information prompts\. Recent methods have shown that long contexts or arbitrary prompts can be compressed into a small number of tokens or representationsMuet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib18)\); Geet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib17)\)\. However, these strong results typically rely on fine\-tuning the full LLM or on extensive training procedures before the compression behavior emerges\.

This leaves open the question of whether context\-relevant prompts can be compressed for off\-the\-shelf LLMs using a lightweight method that requires only minimal training\.

To address this gap, we propose a simple activation\-space compression method based on a weighted sum\. This design is motivated by mechanistic interpretability: if task\-relevant information is represented in internal activations, then it may be possible to compress prompts directly in latent space with minimal overhead\. As illustrated in[Figure 1](https://arxiv.org/html/2607.08399#S1.F1), the method takes as input a promptp=\[t1,…,tT\]p=\[t\_\{1\},\\ldots,t\_\{T\}\]that encodes relevant information or a task description\. We extract the hidden\-state sequenceH\(m\)​\(p\)∈ℝT×dH^\{\(m\)\}\(p\)\\in\\mathbb\{R\}^\{T\\times d\}at an intermediate layermmand compress it into a prompt\-dependent patch vectorv∈ℝdv\\in\\mathbb\{R\}^\{d\}\. In a second forward pass, the model receives a shortened input consisting of aplaceholder token, whose activation is overwritten byvv, followed by a query about the original promptpp\. The model must then generate a response without direct access topp\. We find that the best compression function is a weighted sum over the hidden states inH\(m\)​\(p\)H^\{\(m\)\}\(p\), with weights predicted by a small learned multilayer perceptron \(MLP\)\. This inherently lossy compression reduces test accuracy by only 2% on task\-instruction prompts\. Surprisingly, the simple weighted\-sum compressor consistently outperforms the heavier end\-to\-end trained Transformer compressor in our main experiments\.

In this paper, we make the following contributions:

1. 1\.We propose a two\-pass framework for activation\-space prompt compression, in which a prompt is compressed into a single patch vector and later re\-injected into the model through a placeholder token\.
2. 2\.We show that a lightweight MLP can learn a weighting function that constructs patch vectors which generalize to unseen prompts and tasks\.
3. 3\.Through a layer\-wise analysis, we uncover a new pattern in activation engineering: best performances are obtained when information is extracted from a middle layer and injected into an early layer\.
4. 4\.We provide a detailed analysis of the resulting patch vectors, along with ablations over key design choices\.
5. 5\.We release aToy Taskdataset and the code required to reproduce our results at[our anonymous repository](https://anonymous.4open.science/r/compressed-semantic-212F/README.md)\.

## 2Method

Given a promptp=\[t1,…,tT\]p=\[t\_\{1\},\\ldots,t\_\{T\}\]and an LLM withLLattention layers, a hidden\-state sequence

H\(m\)​\(p\)=\(h1\(m\),…,hT\(m\)\)∈ℝT×dH^\{\(m\)\}\(p\)=\\bigl\(h^\{\(m\)\}\_\{1\},\\ldots,h^\{\(m\)\}\_\{T\}\\bigr\)\\in\\mathbb\{R\}^\{T\\times d\}can be extracted at an intermediate extraction layerm∈\{1,…,L\}m\\in\\\{1,\\dots,L\\\}\. Herehi\(m\)∈ℝdh\_\{i\}^\{\(m\)\}\\in\\mathbb\{R\}^\{d\}denotes the hidden\-state vector downstream of the tokentit\_\{i\}at positioniiat layermm\.

Acompression functionf:ℝT×d→ℝdf:\\mathbb\{R\}^\{T\\times d\}\\rightarrow\\mathbb\{R\}^\{d\}maps the sequence of hidden\-states to a singlepatch vector\.

v=f​\(H\(m\)​\(p\)\)∈ℝd\.v=f\\bigl\(H^\{\(m\)\}\(p\)\\bigr\)\\in\\mathbb\{R\}^\{d\}\.
As depicted in[Figure 1](https://arxiv.org/html/2607.08399#S1.F1), this vector is then injected at an early injection layere<me<m, at positionk∈\{1,…,T\}k\\in\\\{1,\\dots,T\\\}, replacing the hidden\-state downstream of theplaceholdertokentkt\_\{k\}\. The model then continues its forward pass from layereeonward and generates a response\. This design is motivated by prior work suggesting that intermediate layers contain richer semantic representationsPanicksseryet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib21)\), as well as by our empirical observation that early injection performs better \(see[Section 3\.4](https://arxiv.org/html/2607.08399#S3.SS4)and[Figure 5](https://arxiv.org/html/2607.08399#S3.F5)\)\. One possible explanation is that injecting the compressed representation earlier leaves more transformer layers for it to be processed and integrated before prediction\.

\(a\) Weighting MLP⋯\\cdotsh2\(m\)h\_\{2\}^\{\(m\)\}h1\(m\)h\_\{1\}^\{\(m\)\}hT\(m\)h\_\{T\}^\{\(m\)\}MθM\_\{\\theta\}MθM\_\{\\theta\}MθM\_\{\\theta\}w1w\_\{1\}w2w\_\{2\}⋯\\cdotswTw\_\{T\}∑\\sumv\(m\)=∑i=1Twi​hi\(m\)v^\{\(m\)\}=\\sum\_\{i=1\}^\{T\}w\_\{i\}h\_\{i\}^\{\(m\)\}Eachwi∈ℝw\_\{i\}\\in\\mathbb\{R\}is computed from a single activationhi\(m\)∈ℝdh\_\{i\}^\{\(m\)\}\\in\\mathbb\{R\}^\{d\}\(b\) Transformer Compressor⋯\\cdotsh2\(m\)h\_\{2\}^\{\(m\)\}h1\(m\)h\_\{1\}^\{\(m\)\}hT\(m\)h\_\{T\}^\{\(m\)\}EϕE\_\{\\phi\}over full sequenceCompressedPatchDirect learning of the activation compression:H\(m\)​\(p\)∈ℝd×n→v∈ℝdH^\{\(m\)\}\(p\)\\in\\mathbb\{R\}^\{d\\times n\}\\rightarrow v\\in\\mathbb\{R\}^\{d\}\.

Figure 2:Comparison of our two activation\-compression methods: the Weighting MLP and the Transformer Compressor\. The W\-MLP scores each contextualized activation separately, while the TC consumes the full sequenceH\(m\)​\(p\)H^\{\(m\)\}\(p\)to produce a patch vector inℝd\\mathbb\{R\}^\{d\}\.We propose two compression functionsff, depicted in[Figure 2](https://arxiv.org/html/2607.08399#S2.F2)\. The first, theWeighting MLP\(W\-MLP\), operates under the strong assumption that activations can be compressed via a simple weighted sum\. The second, theTransformer Compressor\(TC\), directly optimizes the compression of activations into the patch vectorvvwith end\-to\-end training\. Both methods are trained with cross\-entropy between the target output of the LLM and its logits after being patched byv=f​\(H\(m\)​\(p\)\)v=f\(H^\{\(m\)\}\(p\)\)\. The hyperparameters of all training procedures have been optimized with grid search, details are given in[Appendix E](https://arxiv.org/html/2607.08399#A5)\.

### 2\.1Weighting MLP

The linear representation hypothesis suggests that the internal representations of two concepts can be summed to superimpose their effectsMikolovet al\.\([2013](https://arxiv.org/html/2607.08399#bib.bib45)\)\. Following these considerations, we hypothesize that the information contained in multiple activation vectors can be compressed through a weighted sum\. These weights can be hand\-designed, as explored in a case study presented in[Appendix A](https://arxiv.org/html/2607.08399#A1)\. However, such hand\-crafted weights are imprecise and fail to generalize beyond a specific prompt\. To eliminate this manual choice, we train an MLP to predict the weights from the activations themselves\. Concretely, we use a network

Mθ:ℝd→ℝM\_\{\\theta\}:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}that maps each intermediate activationhi\(m\)h\_\{i\}^\{\(m\)\}to a scalar weightwiw\_\{i\}used to form the patch vector:

Mθ​\(hi\(m\)\)=wi,v=∑i=1Twi​hi\(m\)\.M\_\{\\theta\}\\\!\\bigl\(h^\{\(m\)\}\_\{i\}\\bigr\)=w\_\{i\},\\qquad v=\\sum\_\{i=1\}^\{T\}w\_\{i\}\\,h^\{\(m\)\}\_\{i\}\.\(1\)
The MLP does not directly generate the full patch vector\. Instead, it learns how strongly each token activation should contribute to the final compressed representation\.

In our implementation,MθM\_\{\\theta\}is a small feed\-forward network with 4 layers with hidden dimension\[2048,1024,512,256\]\[2048,1024,512,256\], and a ReLU activation function\.

### 2\.2Transformer Compressor

To explore the limits of compression, we train a transformer encoderEϕE\_\{\\phi\}that attends over the full sequence of hidden representations and produces a single compressed vectorvvvia a\[CLS\]\-style token:

Eϕ:ℝT×d\\displaystyle E\_\{\\phi\}:\\mathbb\{R\}^\{T\\times d\}→ℝd,\\displaystyle\\to\\mathbb\{R\}^\{d\},\(2\)H\(m\)=\[h1\(m\),…,hT\(m\)\]\\displaystyle H^\{\(m\)\}=\[h^\{\(m\)\}\_\{1\},\\ldots,h^\{\(m\)\}\_\{T\}\]↦v=Eϕ​\(H\(m\)\)CLS\.\\displaystyle\\mapsto v=E\_\{\\phi\}\(H^\{\(m\)\}\)\_\{\\texttt\{CLS\}\}\.
The hidden dimension ofEϕE\_\{\\phi\}is6464, with22attention heads and22layers\. The Transformer Compressor is an end\-to\-end solution that does not impose any prior structure on the patch representation\. While this makes it more expressive than the W\-MLP, we show that this makes it prone to overfitting on specific tasks, as discussed in[Section 3\.2](https://arxiv.org/html/2607.08399#S3.SS2)\.

## 3Experiments

We evaluate whether relevant information contained in a prompt can be compressed into a single activation\-space patch vector\. Our experiments are designed to answer four research questions: \(1\) Does the compressed vector preserve the behavior induced by the original prompt? \(2\) How does performance change as the complexity of the compressed information increases? \(3\) How do the W\-MLP and TC compare? and \(4\) How well do these models generalize beyond their training distribution?

### 3\.1Experimental Setup

#### Open\-weight LLMs\.

We conduct our experiments withLlama3\.1\-8B\-InstructGrattafioriet al\.\([2024](https://arxiv.org/html/2607.08399#bib.bib54)\), chosen for its broad availability and favorable performance\-to\-size trade\-off\. To evaluate the generality of our approach across model families and scales, we further testMinistral3\-8B\-Instruct\-2512Liuet al\.\([2026](https://arxiv.org/html/2607.08399#bib.bib49)\), a strong 8B\-parameter model,Qwen3\.5\-4BQwen Team \([2026](https://arxiv.org/html/2607.08399#bib.bib50)\), andLlama3\.2\-1B\-Instruct\. All experiments are run on anNVIDIA RTX A5500\.

Toy TaskExtraction:Given a country, return its capital\.Injection:¿Italy:RomeARC\-EasyExtraction:Which of the following is a primary color?Injection:Reply with ONLY the letter of the correct answer \(A, B, C, or D\)¿\(A\) Green \(B\) Orange \(C\) Red \(D\) PurpleAnswer:CTable 1:Extraction and injection prompts for each dataset\.Pinkindicates the part encoded into the compressed representationvv\.Greenindicates the placeholder token wherevvis patched into the residual stream\.Orangeindicates the generated text by the LLM\. In this representation the special tokens are excluded for clarity\.
#### Data\.

We evaluate our compression functions on two types of data: a set of homemade custom tasks and a standardized benchmark\.

TheToy Taskdataset is inspired by the task\-vector experiments ofHendelet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib11)\)\. However, instead of representing tasks through examples, we specify them directly through instructions\. We construct 11 knowledge\-retrieval tasks, each defined by a simple input\-output mapping \(e\.g\., given acountry, predict itscapital\)\. For each task, we create 10 prompt templates for training and 10 for evaluation, with approximately 100 \(entity,target\) pairs for training and 20 for evaluation per task\.

This dataset provides a controlled setting in which each task is precisely defined and correctness can be verified by exact string matching\. It also makes it straightforward to train the compression functions using a cross\-entropy loss over the LLM output logits\. Details and examples of the dataset are provided in[Appendix B](https://arxiv.org/html/2607.08399#A2)\.

The second dataset isARC\-EasyClarket al\.\([2018](https://arxiv.org/html/2607.08399#bib.bib76)\), a multiple\-choice benchmark in which each question has four answer candidates \(A,B,C,D\)\. We compress the question itself into the representationvv, while the model receives only the answer choices, as depicted in[Table 1](https://arxiv.org/html/2607.08399#S3.T1)\. A failed compression leaves the model without access to the question, resulting in near\-chance performance, whereas successful compression should recover accuracy close to the uncompressed baseline\. As in the Toy Task dataset, the multiple\-choice format allows direct training with cross\-entropy loss over the four candidate logits\. We choose ARC\-Easy because its questions are relatively short and the required knowledge is largely covered by the capabilities of modern LLMs\. This allows us to evaluate the compression strategy without prompt length or question complexity becoming the main limiting factors, unlike in more verbose and technical benchmarks such as MMLUHendryckset al\.\([2020](https://arxiv.org/html/2607.08399#bib.bib75)\)\.

#### Extraction and Injection Layers\.

Unless otherwise stated, we use layerm=12m=12for extraction and layere=2e=2for injection inLlama3\.1\-8B\-Instruct\. These settings were chosen based on a preliminary ablation over extraction\-injection layer pairs and are treated as model\-dependent hyperparameters\. The full ablation study is reported in[Section 3\.4](https://arxiv.org/html/2607.08399#S3.SS4)\.

#### Injection Token\.

The patch vectorvvis injected by replacing the activation downstream of a designatedplaceholder tokenat layeree\. We use the UTF\-8 replacement characterU\+FFFD\(rendered as¿here\) as the placeholder, because it is unlikely to evoke semantic associations in the model’s vocabulary and is available across all models\. It is a single character that is consistently mapped to a single token by the tokenizer, regardless of context\. We study the effect of this token choice and the injection position in[Section 3\.4](https://arxiv.org/html/2607.08399#S3.SS4)\.

#### Baselines\.

We compare our compression methods against two baselines: \(i\) thefull\-prompt baseline, which gives the model access to the original task or question without compression and serves as an upper bound, and \(ii\) themasked\-prompt baseline, which preserves the prompt structure but replaces the compressed content with the placeholder token and does not apply any patching, serving as a lower bound\. Together, these baselines bracket the performance of our compressed representations\.

### 3\.2Compression of Tasks Based on Instruction Prompts\.

MethodTrainTestOODTasksMasked Prompt32\.5532\.5534\.0334\.0318\.4518\.45TC88\.8788\.8770\.6370\.6343\.8943\.89W\-MLP89\.2389\.2385\.3585\.3563\.0163\.01Full prompt85\.7585\.7586\.9286\.9294\.9594\.95Table 2:Exact\-match accuracy on the training and test sets of the Toy Task Dataset for the Weighting MLP \(W\-MLP\), the Transformer Compressor \(TC\), and both baselines, usingLlama\-3\.1\-8B\-Instruct\.We first evaluate whether a single patch vector can preserve the behavior induced by the original prompt\. We train the W\-MLP and TC on eight task families from the Toy Task dataset: capitals, ISO country codes, continents, event years, English\-to\-French translation, art authorship, antonyms, and currency codes\. We reserve three tasks for out\-of\-distribution evaluation: French\-to\-English translation, chemical element, and tuple addition\. For each task family, we compare our compression methods against the two baselines defined above\.

The resulting accuracies forLlama\-3\.1\-8B\-Instructare reported in[Table 2](https://arxiv.org/html/2607.08399#S3.T2)\. The gap between the masked\- and full\-prompt baseline accuracies confirms that the task prompt contains important information, and that the entity alone is not sufficient for most tasks\. We observe that both methods consistently outperform the masked\-prompt baseline and recover a substantial portion of the gap to the full\-prompt baseline, especially on in\-distribution tasks\.

We observe that the simple W\-MLP method outperforms the end\-to\-end TC solution, suggesting that much of the task\-relevant information can be manipulated through simple linear operations in representation space\. On in\-distribution tasks, W\-MLP performs within2%2\\%of the full\-prompt baseline, suggesting that a single compressed vector can preserve most of the prompt instruction\. As shown in[Figure 3](https://arxiv.org/html/2607.08399#S3.F3), its OOD performance varies substantially by task and degrades more noticeably on harder tasks\.

In contrast, the TC performs substantially worse on the test set, reaching only70\.63%70\.63\\%accuracy\. Since this model is trained directly to reconstruct the patch vector, it may be more prone to overfitting\. This hypothesis is supported by its high training accuracy and by the fact that it even outperforms the full\-prompt baseline on the art\-to\-author task\. Overall, these results suggest that the more expressive TC can learn task\-specific bottleneck representations, but W\-MLP generalizes better\.

![Refer to caption](https://arxiv.org/html/2607.08399v1/x1.png)Figure 3:Comparison of test performance for a sub\-sample of the in\-distribution Tasks and out\-of\-distribution Tasks\.
### 3\.3Compression Beyond Task Prompts

ARC\-Easy provides a stricter test of generalization than the out\-of\-distribution Toy Task setting, because each example contains a distinct one\-shot prompt rather than a fixed task description\. As a result, this benchmark evaluates whether the compression mechanism can generalize beyond a single prompt template while preserving information relevant for downstream multiple\-choice prediction\. Test accuracies across methods and LLMs are shown in[Figure 4](https://arxiv.org/html/2607.08399#S3.F4)\.

Overall, W\-MLP learns a more general compression function and achieves test accuracy close to the full\-prompt reference upper bound on most models\. In contrast, TC appears to overfit more strongly: although it can reach training accuracy comparable to the W\-MLP, its test performance drops much more sharply\. OnLlama3\.1\-8B, W\-MLP reaches93%93\\%training accuracy and77%77\\%test accuracy, whereas the TC reaches87%87\\%training accuracy but only48%48\\%test accuracy\. This gap suggests again that W\-MLP induces a simpler and more stable compression function, while TC has greater capacity but weaker generalization\.

We also find that smaller models achieve lower test accuracy with W\-MLP\.Llama3\.2\-1BandQwen3\.5\-4Bhave hidden dimensions of1,0241,024and2,5602,560, respectively, compared to4,0964,096for bothLlama3\.1\-8BandMinistral3\-8B\. This suggests that smaller hidden dimensions may limit the representational capacity available for compression, and that activation\-space compression may improve as the dimensionality of the model’s internal representations increases\.

![Refer to caption](https://arxiv.org/html/2607.08399v1/x2.png)Figure 4:ARC\-Easy test accuracy across LLMs, comparing the Transformer Compressor \(TC\), the Weighting MLP \(W\-MLP\), and the full\- and masked\-prompt baselines
### 3\.4Ablation: Extraction and Injection Layers

![Refer to caption](https://arxiv.org/html/2607.08399v1/x3.png)Figure 5:Heatmap of the Weighting MLP accuracy after 1 epoch on the Toy Tasks, according to extraction layer and injection layer\.We next ablate the extraction and injection layers to assess their influence on compression quality\.[Figure 5](https://arxiv.org/html/2607.08399#S3.F5)reports the training accuracy of W\-MLP after one epoch across different extraction\-injection layer pairs on a subset of tasks\. This ablation reveals a clear trend: the best performance is obtained when activations are extracted from an intermediate layer and injected into an early layer\. The strong performance of intermediate\-layer extraction is consistent with prior work showing that middle layers tend to encode more abstract and task\-relevant informationMarks and Tegmark \([2023](https://arxiv.org/html/2607.08399#bib.bib24)\)\. The effectiveness of early injection may be explained by the fact that it allows more transformer blocks to process and integrate the compressed representation before prediction\. Combining a middle\-layer extraction with an early\-layer injection is also consistent with prior work suggesting that a model’s internal state remains interpretable across layers, rather than being localized to a single layerElhoushiet al\.\([2024](https://arxiv.org/html/2607.08399#bib.bib7)\)\.

The heatmap provides a coarse view of the layer\-pair landscape and suggests a promising region around extraction layer 10 and injection layer 4\. A finer search over this region identifies extraction at layer 12 and injection at layer 2 as the optimal configuration forLlama3\.1\-8B, which we use in all main experiments\.

### 3\.5Ablation: Placeholder Token and Patching Position

To the best of our knowledge, our specific activation\-patching setting has not been previously explored, leaving several design choices open, including the choice of placeholder token and the exact position at which patching is applied\. Results of the ablation study are shown in[Figure 6](https://arxiv.org/html/2607.08399#S3.F6)\. Our chosen configuration, patching downstream of token¿, performs competitively and ranks among the best settings, although the margin over nearby alternatives is small\. This suggests limited sensitivity to the exact placeholder token, as long as it is a neutral and consistently tokenized choice\. In contrast, a clearer trend emerges along the position axis: patching further away from the placeholder token and toward either the beginning\-of\-text token \(<\|bot\|\>\) or the assistant answer consistently degrades performance\. This indicates that the intervention is most effective when applied close to the placeholder position\.

These results also suggest room for improvement and motivate extensions that inject multiple patch vectors simultaneously, potentially providing richer information at the cost of a more careful positional design\.

![Refer to caption](https://arxiv.org/html/2607.08399v1/x4.png)Figure 6:Heatmap of the W\-MLP accuracy on the Toy Tasks as a function of the placeholder token and patching position\. Theyy\-axis indicates the token position at which the patch vector is injected, relative to the placeholder position\. The absence of a placeholder token is denoted by∅\\varnothing\.

## 4What Do Patch Vectors Represent?

The results in the previous section show that a learned patch vector can recover most of the performance of the original prompt\. Here, we analyze what kind of information this vector captures\. Specifically, we ask whether the patch vector remains close to the original prompt activations and whether the learned weighting mechanism focuses on semantically meaningful prompt tokens\.

To gain intuition on the compression mechanism, we qualitatively inspect the learned weights assigned to each token in the weighted sum in[Table 3](https://arxiv.org/html/2607.08399#S4.T3)\. A consistent pattern emerges: tokens carrying dense semantic content, such as keywords or named entities, receive systematically higher weights, while low\-information tokens such as determiners are comparatively suppressed\.

To move beyond purely qualitative observations, we conduct a controlled experiment using 100 LLM\-generated reformulations of the prompt“What is the industry this company operates in?”\. Each reformulation varies in surface form while preserving task semantics, allowing us to track which token consistently receives the highest learned weight across phrasings\. In85%85\\%of cases, the highest weight is assigned to eitherindustry,field, orsector, directly reflecting the target of the retrieval task\. This provides quantitative support for the qualitative pattern observed in[Table 3](https://arxiv.org/html/2607.08399#S4.T3): the compression mechanism identifies and prioritizes the semantically load\-bearing token in the prompt\. We note, however, that semantic salience is not formally defined here, so this analysis should be interpreted as a qualitative diagnostic rather than a formal measure of token importance\.

In the remaining10%10\\%of cases, the highest weight is assigned to the terminal punctuation token\. Strikingly, even when it does not rank first, the interrogation mark appears as the second highest weighted token in75%75\\%of cases across all reformulations\. Rather than contradicting the semantic salience finding, this is consistent with the well\-documented attention sink phenomenon in auto\-regressive models, whereby punctuation and sentence\-boundary tokens accumulate disproportionate attention weight as aggregators of contextual informationXiaoet al\.\([2024](https://arxiv.org/html/2607.08399#bib.bib32)\); Chauhanet al\.\([2026](https://arxiv.org/html/2607.08399#bib.bib31)\)\. The W\-MLP thus appears to exploit the same representational structure that attention heads do, further grounding the compression mechanism in known properties of transformer activation space\. The focus on key semantic words and on punctuation is also consistent with the hand\-crafted weights studied in[Appendix A](https://arxiv.org/html/2607.08399#A1)\.

These observations are further corroborated by two independent metrics: the KL\-divergence between the compressed patch and individual token activations, and their cosine similarity\. Both metrics assign highest proximity to the same semantically salient tokens identified by the learned weights\.[Table 3](https://arxiv.org/html/2607.08399#S4.T3)confirms that this interpretability extends beyond the weights themselves to the final aggregated patch vector, which remains geometrically close to the activations of task\-critical tokens\. Additional examples are provided in[Appendix D](https://arxiv.org/html/2607.08399#A4)\.

\(a\) Weight attribution of W\-MLPWhatisthecapitalofthiscountry?\(b\) Cosine\-similarity to final patch vectorPleasegivemethesumofthetwonumbers3and5\.\(c\) KL\-Divergence to final patch vectorGivenaGermanwordresponditsChinesetranslation\.Table 3:Visualization of \(a\) weights attributed by the learned W\-MLP to each token activation; \(b\) Cosine\-similarity between the token activations and the compressed patch vectorvv; \(c\) KL\-divergence between the token activations and the compressed patch vectorvv\. A positive metric is depicted inorange, while a negative value is depicted inblue
## 5Discussion and Future Work

#### Information Capacity and Accessibility\.

Our experiments suggest that smaller internal representations lead to less accurate compression, independently of the LLM’s overall performance\. This may indicate that compression quality is constrained by the dimensionality of the hidden representation\. This also aligns with the idea that sparser representations can be combined more easily, with less destructive interference, as suggested by the superposition perspective ofElhageet al\.\([2022](https://arxiv.org/html/2607.08399#bib.bib38)\)\. A preliminary information capacity experiment in[Appendix C](https://arxiv.org/html/2607.08399#A3)shows that even in the smallerLlama3\.2\-1Bmodel, one patch vector can encode 9 task families simultaneously without a clear accuracy drop\. We note that this experiment measures not only the capacity of the vector to encode task information, but also the model’s ability to recover and exploit it at inference time\. Nevertheless, the question of the upper bounds on the capacity and accessibility of such representations remains open\.

#### Interpretability by Design\.

Unlike post\-hoc explanation methods, the learned weighting mechanism is interpretable by construction: the weights assigned by W\-MLP directly expose which tokens the compression has identified as informationally relevant, without requiring any additional analysis pass\. This property is noteworthy in the context of feature attribution research, where methods such as integrated gradientsSundararajanet al\.\([2017](https://arxiv.org/html/2607.08399#bib.bib4)\)or black\-box attribution approachesCaiet al\.\([2025](https://arxiv.org/html/2607.08399#bib.bib5)\)aim to quantify the contribution of each input token to the model output\. The patching weights offer a structurally similar signal but emerge naturally from the compression objective rather than being retroactively inferred\. Whether these two families of scores converge empirically is an open question we leave to future work\.

#### Practical Implications\.

Our results point to several possible applications of activation\-space compression\. In consumer API LLMs, system prompts are often reused across many requests and can contribute substantially to the cost of processing the context even after activation caching\. Compressing such prompts into a fixed\-size activation vector could allow their effect to be reused without re\-encoding the full prompt each time\. This may be especially attractive for short queries, where the system prompt can dominate the token budget\.

In retrieval\-augmented generation \(RAG\), large retrieved text chunks could similarly be compressed once and reused in compact form rather than repeatedly passed to the LLM in full\. This naturally conditions on the method’s ability to scale to longer contexts, which we identify as a key future work\. More speculatively, the same compressed representation could also serve as a semantic vector for similarity search, providing a shared representation for retrieval and generation\.

In robotics, vision\-language\-action \(VLA\) modelsZitkovichet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib55)\)condition visuomotor policies on rich instruction contexts that may need to be processed at every control step, making attention overhead a direct latency bottleneck\. Activation\-space compression could potentially alleviate this by encoding complex, reusable instruction or proprioceptive information into a fixed\-size representation offline, eliminating sequence\-length growth at inference time\. This mirrors the use of rare tokens as learned capability pointers in fine\-tuned VLA systems, but operates in activation rather than embedding space, preserving richer contextual state\. We leave empirical validation of these applications to future work\.

#### Future Extensions\.

Several directions naturally extend the current work\. Most immediately, rather than compressing the full prompt into a single patch vector, one could segment the prompt into semantic blocks, each compressed into its own patch token, trading compression ratio for representational fidelity\. Additionally, enforcing sparsity in individual activations prior to the weighted sum could suppress noise and sharpen the compressed representation\. Finally, with the growing interest in cross\-layer transcoders for mechanistic interpretabilityDunefskyet al\.\([2024](https://arxiv.org/html/2607.08399#bib.bib43)\), the aggregation could be extended across layers rather than within a single one, potentially capturing richer hierarchical structure in the compressed activation\.

## 6Related Work

#### Prompt Compression at the Token Level\.

Several works address the redundancy of natural language prompts by compressing them into shorter token sequences\. GistingMuet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib18)\)learns to compress prompts into a small number of special tokens, achieving strong results but requiring full LLM fine\-tuning\. AutoCompressorsChevalieret al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib16)\)and ICAEGeet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib17)\)similarly achieve effective long\-context compression at the cost of heavy model\-specific training\. KV\-cache and prefix caching approachesPopeet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib2)\)reduce recomputation costs in production systems but operate at the token level and leave the sequence length and attention cost unchanged\. Our method differs fundamentally: we compress at the*activation*level into a single vector, require no fine\-tuning of the target model, and produce a representation that is not human\-readable but is directly injectable into the residual stream\.

#### Layer\-wise Information and Early Exit\.

Several works demonstrate that transformer layers contain sufficient information to predict the final output well before the last layerGevaet al\.\([2022](https://arxiv.org/html/2607.08399#bib.bib6)\); Schusteret al\.\([2022](https://arxiv.org/html/2607.08399#bib.bib12)\)\. Early\-exit methods exploit this by halting computation once a confidence threshold is met\. This body of work directly motivates our injection strategy: patching a compressed activation at an early layer gives the model sufficient depth to process the information before generation, without requiring the full forward pass over the original token sequence\.

#### Activation Steering and Representation Engineering\.

Prior work has shown that high\-level behavioral concepts can be encoded in single vectors and injected into specific layers to redirect model behaviorTempletonet al\.\([2024](https://arxiv.org/html/2607.08399#bib.bib36)\); Ardoinet al\.\([2025](https://arxiv.org/html/2607.08399#bib.bib1)\)\. Representation engineeringZouet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib22)\)and contrastive activation additionPanicksseryet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib21)\)similarly manipulate intermediate activations to steer generation\. Our work is mechanistically related but distinct in objective: prior work steers*behavior*, we compress*prompts*, with the goal of faithful replication rather than redirection\.

#### Superposition and Linearity in Activation Space\.

Elhageet al\.\([2022](https://arxiv.org/html/2607.08399#bib.bib38)\)provide theoretical grounding for the idea that LLM features are sparsely and superpositionally encoded, implying high redundancy in activation space\.Liuet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib19)\)show empirically that steering vectors can be additively combined while preserving their individual effects\. These findings motivate our core assumption: that a weighted sum of token activations can preserve the semantic content of the original prompt, and that the resulting vector lies in a region of activation space the model can interpret\.

#### Prompt Redundancy and Information Optimality\.

Melamedet al\.\([2023](https://arxiv.org/html/2607.08399#bib.bib14)\)show that semantically equivalent prompts can differ radically in surface form, suggesting the model’s internal task representation is divorced from specific wording\.Leeet al\.\([2025](https://arxiv.org/html/2607.08399#bib.bib15)\)show that there exists an optimal token count per task, and that additional tokens do not monotonically improve performance\. Our work pushes this to its logical limit: a single activation vector, raising the question of how much task\-relevant information can be packed into the minimal possible representation\.

## 7Conclusion

We have shown that relevant prompt information can be compressed into the activation space of an LLM at a controlled accuracy cost\. A simple learned weighted sum of activations extracted at a middle layer and injected at an early layer proves an effective compression mechanism, as confirmed by our ablations over layer depth and aggregation strategy\. Beyond its practical implications for inference efficiency, this finding offers new evidence for the linearity of LLM internal representations\. Meaningful semantic content can be recovered through basic arithmetic over intermediate activations, at a scale beyond what has previously been demonstrated\. We hope this opens a productive dialogue between efficiency\-oriented and mechanistic interpretability research\.

## Limitations

The proposed compression is inherently lossy, and its fidelity under high information\-density prompts remains an open question\. Our evaluation focuses primarily on knowledge retrieval tasks and relatively short prompts\. Generalization to longer contexts or necessitating reasoning has not been established\. Furthermore, the method requires white\-box access to the model’s internal activations, excluding application to closed\-source systems\. Finally, compression requires a partial forward pass through at least half of the transformer layers, meaning the approach is not cost\-free at compression time and is best amortized over repeated reuse of the same context\.

## Ethical Considerations

This work presents a method for compressing prompt activations in LLMs\. We do not foresee direct ethical implications, though we note that compressed non\-human\-readable instruction vectors could in principle be used to obscure prompt intent from human reviewers\. On the positive side, by reducing the per\-query computational cost of fixed instruction prompts, the proposed method contributes to more energy\-efficient LLM inference, with potential benefits for the environmental footprint of large\-scale deployments\.

## References

- Where confabulation lives: latent feature discovery in llms\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:LinkCited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px3.p1.1)\.
- Y\. Cai, T\. Ardoin, and G\. Wunder \(2025\)GEFA: a general feature attribution framework using proxy gradient estimation\.InProceedings of the 42nd International Conference on Machine Learning,pp\. 6165–6192\.Cited by:[§5](https://arxiv.org/html/2607.08399#S5.SS0.SSS0.Px2.p1.1)\.
- S\. Chauhan, M\. Chaudhary, C\. K\. Kiu, S\. Nellessen, and N\. Schoots \(2026\)Punctuations and predicates in language models\.InFindings of the Association for Computational Linguistics: EACL 2026,pp\. 5622–5636\.Cited by:[§4](https://arxiv.org/html/2607.08399#S4.p4.2)\.
- A\. Chevalier, A\. Wettig, A\. Ajith, and D\. Chen \(2023\)Adapting language models to compress contexts\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 3829–3846\.Cited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px1.p1.1)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? try arc, the ai2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.Cited by:[Appendix B](https://arxiv.org/html/2607.08399#A2.SS0.SSS0.Px3.p1.1),[§3\.1](https://arxiv.org/html/2607.08399#S3.SS1.SSS0.Px2.p4.1)\.
- J\. Dunefsky, P\. Chlenski, and N\. Nanda \(2024\)Transcoders find interpretable llm feature circuits\.Advances in Neural Information Processing Systems37,pp\. 24375–24410\.Cited by:[§5](https://arxiv.org/html/2607.08399#S5.SS0.SSS0.Px4.p1.1)\.
- N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen, R\. Grosse, S\. McCandlish, J\. Kaplan, D\. Amodei, M\. Wattenberg, and C\. Olah \(2022\)Toy models of superposition\.Note:[https://transformer\-circuits\.pub/2022/toy\_model/](https://transformer-circuits.pub/2022/toy_model/)Cited by:[§5](https://arxiv.org/html/2607.08399#S5.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px4.p1.1)\.
- M\. Elhoushi, A\. Shrivastava, D\. Liskovich, B\. Hosmer, B\. Wasti, L\. Lai, A\. Mahmoud, B\. Acun, S\. Agarwal, A\. Roman,et al\.\(2024\)Layerskip: enabling early exit inference and self\-speculative decoding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 12622–12642\.Cited by:[§3\.4](https://arxiv.org/html/2607.08399#S3.SS4.p1.1)\.
- T\. Ge, J\. Hu, L\. Wang, X\. Wang, S\. Chen, and F\. Wei \(2023\)In\-context autoencoder for context compression in a large language model\.arXiv preprint arXiv:2307\.06945\.Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p4.1),[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px1.p1.1)\.
- M\. Geva, A\. Caciularu, K\. Wang, and Y\. Goldberg \(2022\)Transformer feed\-forward layers build predictions by promoting concepts in the vocabulary space\.InProceedings of the 2022 conference on empirical methods in natural language processing,pp\. 30–45\.Cited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px2.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, and A\. Sravankumar \(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2407.21783),[Link](https://arxiv.org/abs/2407.21783)Cited by:[§3\.1](https://arxiv.org/html/2607.08399#S3.SS1.SSS0.Px1.p1.1)\.
- R\. Hendel, M\. Geva, and A\. Globerson \(2023\)In\-context learning creates task vectors\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 9318–9333\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.624)Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p4.1),[§3\.1](https://arxiv.org/html/2607.08399#S3.SS1.SSS0.Px2.p2.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2020\)Measuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.Cited by:[§3\.1](https://arxiv.org/html/2607.08399#S3.SS1.SSS0.Px2.p4.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th Symposium on Operating Systems Principles,pp\. 611–626\.External Links:[Document](https://dx.doi.org/10.1145/3600006.3613165)Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p2.1)\.
- A\. Lee, E\. Che, and T\. Peng \(2025\)How well do llms compress their own chain\-of\-thought? a token complexity approach\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2503.01141),[Link](https://arxiv.org/abs/2503.01141)Cited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px5.p1.1)\.
- A\. H\. Liu, K\. Khandelwal, S\. Subramanian, V\. Jouault, A\. Rastogi, A\. Sadé, A\. Jeffares, A\. Jiang, A\. Cahill, A\. Gavaudan,et al\.\(2026\)Ministral 3\.arXiv preprint arXiv:2601\.08584\.Cited by:[§3\.1](https://arxiv.org/html/2607.08399#S3.SS1.SSS0.Px1.p1.1)\.
- S\. Liu, H\. Ye, L\. Xing, and J\. Zou \(2023\)In\-context vectors: making in\-context learning more effective and controllable through latent space steering\.arXiv preprint arXiv:2311\.06668\.Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p3.1),[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px4.p1.1)\.
- S\. Marks and M\. Tegmark \(2023\)The geometry of truth: emergent linear structure in large language model representations of true/false datasets\.arXiv preprint arXiv:2310\.06824\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2310.06824),[Link](https://arxiv.org/abs/2310.06824)Cited by:[§3\.4](https://arxiv.org/html/2607.08399#S3.SS4.p1.1)\.
- R\. Melamed, L\. H\. McCabe, T\. Wakhare, Y\. Kim, H\. H\. Huang, and E\. Boix\-Adsera \(2023\)Prompts have evil twins\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2311.07064),[Link](https://arxiv.org/abs/2311.07064)Cited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px5.p1.1)\.
- T\. Mikolov, W\. Yih, and G\. Zweig \(2013\)Linguistic regularities in continuous space word representations\.InProceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,L\. Vanderwende, H\. Daumé III, and K\. Kirchhoff \(Eds\.\),Atlanta, Georgia,pp\. 746–751\.External Links:[Link](https://aclanthology.org/N13-1090)Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.08399#S2.SS1.p1.3)\.
- J\. Mu, X\. Li, and N\. Goodman \(2023\)Learning to compress prompts with gist tokens\.Advances in Neural Information Processing Systems36,pp\. 19327–19352\.Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p4.1),[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px1.p1.1)\.
- N\. Panickssery, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. M\. Turner \(2023\)Steering Llama 2 via contrastive activation addition\.arXiv preprint arXiv:2312\.06681\.Cited by:[§2](https://arxiv.org/html/2607.08399#S2.p3.4),[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px3.p1.1)\.
- R\. Pope, S\. Douglas, A\. Chowdhery, J\. Devlin, J\. Bradbury, J\. Heek, K\. Xiao, S\. Agrawal, and J\. Dean \(2023\)Efficiently scaling transformer inference\.Proceedings of machine learning and systems5,pp\. 606–624\.Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p2.1),[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px1.p1.1)\.
- Qwen Team \(2026\)Qwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[§3\.1](https://arxiv.org/html/2607.08399#S3.SS1.SSS0.Px1.p1.1)\.
- T\. Schuster, A\. Fisch, J\. Gupta, M\. Dehghani, D\. Bahri, V\. Q\. Tran, Y\. Tay, and D\. Metzler \(2022\)Confident adaptive language modeling\.InAdvances in Neural Information Processing Systems,Vol\.35\.Cited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px2.p1.1)\.
- N\. Subramani, N\. Suresh, and M\. E\. Peters \(2022\)Extracting latent steering vectors from pretrained language models\.InFindings of the Association for Computational Linguistics: ACL 2022,pp\. 566–581\.Cited by:[§1](https://arxiv.org/html/2607.08399#S1.p3.1)\.
- M\. Sundararajan, A\. Taly, and Q\. Yan \(2017\)Axiomatic attribution for deep networks\.InInternational conference on machine learning,pp\. 3319–3328\.Cited by:[§5](https://arxiv.org/html/2607.08399#S5.SS0.SSS0.Px2.p1.1)\.
- A\. Templeton, T\. Conerly, J\. Marcus, J\. Lindsey, T\. Bricken, B\. Chen, A\. Pearce, C\. Citro, E\. Ameisen, A\. Jones, H\. Cunningham, N\. L\. Turner, C\. McDougall, M\. MacDiarmid, C\. D\. Freeman, T\. R\. Sumers, E\. Rees, J\. Batson, A\. Jermyn, S\. Carter, C\. Olah, and T\. Henighan \(2024\)Scaling monosemanticity: extracting interpretable features from claude 3 sonnet\.Note:[https://transformer\-circuits\.pub/2024/scaling\-monosemanticity/](https://transformer-circuits.pub/2024/scaling-monosemanticity/)Cited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px3.p1.1)\.
- G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. Lewis \(2024\)Efficient streaming language models with attention sinks\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 21875–21895\.Cited by:[§4](https://arxiv.org/html/2607.08399#S4.p4.2)\.
- B\. Zitkovich, T\. Yu, S\. Xu, P\. Xu, T\. Xiao, F\. Xia, J\. Wu, P\. Wohlhart, S\. Welker, A\. Wahid,et al\.\(2023\)Rt\-2: vision\-language\-action models transfer web knowledge to robotic control\.InConference on Robot Learning,pp\. 2165–2183\.Cited by:[§5](https://arxiv.org/html/2607.08399#S5.SS0.SSS0.Px3.p3.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson,et al\.\(2023\)Representation engineering: a top\-down approach to AI transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§6](https://arxiv.org/html/2607.08399#S6.SS0.SSS0.Px3.p1.1)\.

## Appendix AHand\-made Weighting Method

As a simple proof\-of\-concept baseline, we construct a patch vector directly from the prompt activations using a hand\-designed weighting rule\. This experiment tests whether task\-relevant information can be recovered from the activation sequence of a task prompt using a simple hand\-crafted heuristic without training an additional model\.

#### Setup\.

We consider the country\-capital task and use the base Llama\-3\.1\-8B model in a completion\-style setting\. The source prompt is

Give me the capital city of a country I name:and we extract the hidden states of all prompt tokens at an intermediate layer\. At evaluation time, queries are written in the form

For example, as:France:\. The first colon acts as a placeholder token and is the position at which the patch is injected\.

#### Hand\-made Vector Construction\.

Let

p=\(t1,…,tT\)p=\(t\_\{1\},\\dots,t\_\{T\}\)denote the tokenized source prompt, and let

hi\(m\)∈ℝdh\_\{i\}^\{\(m\)\}\\in\\mathbb\{R\}^\{d\}denote the hidden state of tokentit\_\{i\}at the extraction layermm\. In our experiment, we use extraction layerm=12\.m=12\.

The hand\-made vector consists of two components\. First, we compute a position\-weighted sum of all prompt activations:

vpos=∑i=1T\(0\.1​i\)​hi\(m\)\.v\_\{\\mathrm\{pos\}\}=\\sum\_\{i=1\}^\{T\}\(0\.1i\)h\_\{i\}^\{\(m\)\}\.This gives larger weights to later tokens in the prompt\. The heuristic is that later activations may contain a more complete representation of the task instruction\. Second, we manually amplify a small set of task\-relevant token positions\. For the prompt above, we use

which corresponds to the tokens associated withcapital,city, and the final token:\. We define

vkey=α​∑i∈Shi\(m\)v\_\{\\mathrm\{key\}\}=\\alpha\\sum\_\{i\\in S\}h\_\{i\}^\{\(m\)\}with amplification factorα=6\.\\alpha=6\.The final patch vector is

v=vpos\+vkey,v=v\_\{\\mathrm\{pos\}\}\+v\_\{\\mathrm\{key\}\},which is normalized before being injected:

v^=v‖v‖2\.\\hat\{v\}=\\frac\{v\}\{\\\|v\\\|\_\{2\}\}\.

#### Patching Procedure\.

For a query of the form:Country:, the first colon is used as the placeholder position\. In the base completion model, this corresponds to token position11, after the beginning\-of\-sequence token:

<bos\> : Country :\.\\texttt\{<bos\> : Country :\}\.We inject the patch vector at layere=2e=2by replacing the hidden state of the placeholder token:

hplaceholder\(e\)←λ​v^,h\_\{\\mathrm\{placeholder\}\}^\{\(e\)\}\\leftarrow\\lambda\\hat\{v\},whereλ\\lambdais a scalar amplification factor\. In the reproduced experiment, we useλ=100,000\\lambda=100\{,\}000\. This is replacement patching rather than additive patching\. The full configuration is summarized in[Table 4](https://arxiv.org/html/2607.08399#A1.T4)\.

ComponentValueModelLlama\-3\.1\-8B basePrompt formatCompletion\-styleSource promptGive me the capital cityof a country I name:Query format:Country:Extraction layermm12Patching layeree2Selected positionsSS\{4,5,T\}\\\{4,5,T\\\}Amplificationα\\alpha6Scalarλ\\lambda100,000100\{,\}000Patch operationReplacementAccuracy85\.1%85\.1\\%Table 4:Configuration of the hand\-made weighting experiment\.
#### Results and Ablation\.

With the setup in[Table 4](https://arxiv.org/html/2607.08399#A1.T4), the hand\-made patch vector achieves85\.1%85\.1\\%accuracy on the country\-capital training split for the base Llama\-3\.1\-8B model\.

To understand the contribution of the two terms, we also evaluate them separately\. The results are shown in[Table 5](https://arxiv.org/html/2607.08399#A1.T5)\. The position\-weighted component alone reaches56\.4%56\.4\\%, showing that a rough positional weighting already captures some task information\. The manually selected token component is stronger, reaching80\.2%80\.2\\%\. Combining both terms gives the best result\.

Patch vectorAccuracyvposv\_\{\\mathrm\{pos\}\}only56\.4%56\.4\\%vkeyv\_\{\\mathrm\{key\}\}only80\.2%80\.2\\%vpos\+vkeyv\_\{\\mathrm\{pos\}\}\+v\_\{\\mathrm\{key\}\}85\.1%85\.1\\%Table 5:Ablation of the hand\-made patch vector on the country\-capital training split\.This ablation suggests that task\-relevant information is not distributed uniformly across the prompt activation sequence\. A small number of semantically important token activations already carry much of the useful signal\. At the same time, choosing these tokens by hand is delicate\. Namely, the construction depends on the prompt wording, tokenization, patch position, and scaling factor\.

This motivates the learned W\-MLP aggregation model\. Rather than fixing the weightswihandw\_\{i\}^\{\\mathrm\{hand\}\}manually, W\-MLP learns the token weights from data and therefore can adapt the aggregation to different prompts and tasks\. In particular, the hand\-made construction does not transfer directly to the chat\-formatted Llama\-3\.1\-8B\-Instruct model, where performance remained close to5\.9%5\.9\\%across patching layers\. We thus treat this hand\-made vector as a proof of concept rather than a robust general\-purpose model\.

## Appendix BDatasets and Prompt Formats

We use two types of datasets in our experiments: a collection of synthetic dictionary\-style toy tasks and the ARC\-Easy multiple\-choice question answering benchmark\. We now describe the input\-output format, prompt format, and evaluation criterion for both settings\.

#### Toy Task Dataset

The toy task dataset consists of simple mapping tasks\. Each example is a pair\(x,y\)\(x,y\), wherexxis an input entity andyyis the target string\. The task prompt specifies which mapping should be applied\. For example, in the country\-capital task, the input is a country name and the target is its capital city\.

In the main in\-distribution \(ID\) experiments, we use eight toy tasks: capitals, ISO country codes, continents, event years, English\-to\-French translation, art authorship, antonyms, and currency codes\. We additionally evaluate out\-of\-distribution \(OOD\) transfer on French\-to\-English translation, chemical element symbols, and addition of small integers\. Representative examples are shown in[Table 6](https://arxiv.org/html/2607.08399#A2.T6)\.

SplitTaskInputxxTargetyyIDCountry\-capitalFranceParisIDISO codeFranceFRIDContinentFranceEuropeIDEvent yearFrench revolution1789IDEn→\\toFrdogchienIDArt authorMona LisaLeonardo da VinciIDAntonymhotcoldIDCurrencyJapanyenOODFr→\\toEnchiendogOODElement symbolHydrogenHOODAddition17\+2517\+2542Table 6:Examples of toy task input\-output pairs\.For each toy task, we use a set of approximately1010different natural\-language prompts that describe the mapping to be performed\. These prompts are used to extract the activation sequence from which the patch vector is constructed\. Examples are given in[Table 7](https://arxiv.org/html/2607.08399#A2.T7)\.

TaskExample promptCountry\-capitalGive me the capital city of a country I nameISO country codeReturn the ISO code of the country I nameContinentReturn the continent of the country I nameEvent yearReturn the year of the event I nameEnglish\-FrenchTranslate the English word to FrenchArt authorReturn the author of the work I nameAntonymGive me the antonym of the word I provideCurrencyReturn the currency of the country I nameTable 7:Example prompts for the toy task dataset\.At evaluation time, the model receives the input entityxxtogether with the patch vector\. The generated output is marked correct if the target stringyyoccurs in the generated continuation after lowercasing\. This string\-based criterion is used because the target may consist of more than one token\.

#### Toy Task Prompt Format\.

For the instruct\-model experiments, the task prompt is encoded into the patch vector, while the query contains only a placeholder and the input entity\. In the notation of the main text, the compressed task prompt is the information that is replaced by the patch vector\. Abstractly, the injection prompt has the form

\[PATCH\]​x,\\displaystyle\\texttt\{\[PATCH\] \}x,wherexxis the input entity\. For example, for the country\-capital task and inputItaly, the intended behavior is

\[PATCH\] Italy⟶Rome\.\\texttt\{\[PATCH\] Italy\}\\quad\\longrightarrow\\quad\\texttt\{Rome\}\.
In the actual Llama\-3\.1\-8B\-Instruct implementation, prompts are formatted using the model’s chat template\. The placeholder is placed in the system and the input entity is placed in the user message:

system:\[PATCH\]user:x\.\\displaystyle\\quad x\.
After applying the chat template, the placeholder appears at a fixed token position in the system message\. In our Llama\-3\.1\-8B\-Instruct, this was token position66, and the patch vector is injected at that position\.

#### ARC\-Easy

In addition to the toy task dataset, we also evaluate on ARC\-Easy, the Easy split of the AI2 Reasoning ChallengeClarket al\.\([2018](https://arxiv.org/html/2607.08399#bib.bib76)\)\. ARC\-Easy is a multiple\-choice science question answering benchmark\. Each example consists of a natural\-language question, four answer letter candidates, and a target answer letter\. In our compression setup, the question is encoded into the patch vector, while the model receives the answer choices\. Thus, if compression fails, the model has access to the answer choices but not to the question\. An example is given in[Table 8](https://arxiv.org/html/2607.08399#A2.T8)\.

FieldExampleQuestionWhich object is attractedto a magnet?Choice Awooden spoonChoice Biron nailChoice Cplastic cupChoice Dglass bottleTargetBTable 8:Example format for ARC\-Easy\.For ARC\-Easy, the extraction prompt contains the question\. The injection prompt contains the placeholder and the answer of the choices, together with an instruction to answer with only the letter of the correct option\. Abstractly, the format is

Extraction:QuestionInjection:Reply with ONLY the letter ofthe correct answer\[PATCH\] \(A\) \.\.\. \(B\) \.\.\.\(C\) \.\.\. \(D\) \.\.\.

## Appendix CMulti\-Task Encoding in a Single Vector

To probe the information capacity of a single patched activation, we design a complementary experiment in which we directly optimize a single vector for downstream task performance, rather than a compression network\. Concretely, for a set ofkktasks, we train a single patch vector𝐯\\mathbf\{v\}with backpropagation such that, when injected into the model at the standard patching layer, the model successfully performs allkktasks on the given input entities\. No prompt is provided at inference time: the vector alone must encode the full multi\-task instruction\. In order to recover the correct task the model is prompted with the id of the task:

system:\[PATCH\]user:Task i: \[ENTITY\]
The results are shown in[Figure 7](https://arxiv.org/html/2607.08399#A3.F7)\.

![Refer to caption](https://arxiv.org/html/2607.08399v1/x5.png)Figure 7:Multi\-task encoding capacity of a single optimized patch vector\. For each task group sizekk, we optimize one vector over a random subset ofkktasks\. Results are averaged over 5 runs\.We train this vector for an increasing number of tasks, monitoring whether combining all tasks into a single fixed representation causes a collapse in accuracy\. Surprisingly, we find no such collapse within our experimental range with up to 9 different tasks: a single vector of dimension40964096forLlama3\.1\-8B\-Instruct, and10241024for theLlama3\.2\-1B\-Instructmodel, remains sufficient to encode all tested tasks simultaneously without measurable degradation\. This suggests that the activation space of these models has substantially higher information capacity than a single task requires\. This potentially also implies that task instructions are not competing for the same representational directions\.

We note that the precise capacity limit of a single patched vector remains an open question\. Establishing this limit rigorously would require a broader and more diverse task set, and we leave this to future work\.

## Appendix DExamples of Weight Attribution and Similarity Measures

We provide a qualitative visualization of the learned W\-MLP weights and their relation to the original prompt\-token activations\. These examples complement the hand\-made weighting experiment in[Appendix A](https://arxiv.org/html/2607.08399#A1)\. The hand\-made construction suggested that a small number of semantically important prompt tokens can carry a large part of the useful task information\. The visualizations below show that the learned W\-MLP weights follow a similar pattern\.

#### Metrics\.

Let

H\(m\)=\(h1\(m\),…,hT\(m\)\)H^\{\(m\)\}=\(h^\{\(m\)\}\_\{1\},\\dots,h^\{\(m\)\}\_\{T\}\)be the token activation sequence of a promptppat layermm, and letvvbe the learned patch produced by the W\-MLP\. For each token positionii, we compute the cosine similarity

sim⁡\(v,hi\(m\)\)=⟨v,hi\(m\)⟩‖v‖2​‖hi\(m\)‖2\.\\operatorname\{sim\}\(v,h^\{\(m\)\}\_\{i\}\)=\\frac\{\\langle v,h^\{\(m\)\}\_\{i\}\\rangle\}\{\\\|v\\\|\_\{2\}\\,\\\|h^\{\(m\)\}\_\{i\}\\\|\_\{2\}\}\.This measures how close the final patch vector is to each original prompt\-token activation in representation space\.

Next to cosine similarity, we compute a KL\-divergence\-based comparison between the output distribution induced by the patch vector and the output distribution induced by the corresponding prompt\-token activation\. Concretely, for each token activationhi\(m\)h\_\{i\}^\{\(m\)\}, we compare the output distribution induced byhi\(m\)h\_\{i\}^\{\(m\)\}with the output distribution induced by the patch vectorvv:

DKL\(pv\(⋅∣x\)∥pi\(⋅∣x\)\)\.D\_\{\\mathrm\{KL\}\}\\bigl\(p\_\{v\}\(\\cdot\\mid x\)\\,\\\|\\,p\_\{i\}\(\\cdot\\mid x\)\\bigr\)\.
Lower values indicate that the token activation induces a predictive behavior closer to that of the final patch vector\.

Finally, we visualize the scalar weights predicted by the W\-MLP:

wi=Mθ​\(hi\(m\)\),v=∑i=1Twi​hi\(m\)\.w\_\{i\}=M\_\{\\theta\}\(h\_\{i\}^\{\(m\)\}\),\\qquad v=\\sum\_\{i=1\}^\{T\}w\_\{i\}h\_\{i\}^\{\(m\)\}\.
These weights indicate how strongly each prompt\-token activation contributes to the final compressed representation\.

#### Qualitative Observations\.

Representative token\-level visualizations are shown in Figures[8](https://arxiv.org/html/2607.08399#A4.F8),[9](https://arxiv.org/html/2607.08399#A4.F9),[10](https://arxiv.org/html/2607.08399#A4.F10), and[11](https://arxiv.org/html/2607.08399#A4.F11)\. For each prompt, we visualize the cosine similarity to the final patch vector, the KL\-divergence\-based comparison, and the weights assigned by the W\-MLP\.

Addition

Cosine similarity

Pleasegivemethesumofthetwonumbers3and5\.

W\-MLP weights

Pleasegivemethesumofthetwonumbers3and5\.

KL divergence

Pleasegivemethesumofthetwonumbers3and5\.

Antonym

Cosine similarity

Whatistheantonymofthefollowingword

W\-MLP weights

Whatistheantonymofthefollowingword

KL divergence

Whatistheantonymofthefollowingword

Art\-author

Cosine similarity

Determinewhoauthoredthespecifiedwork

W\-MLP weights

Determinewhoauthoredthespecifiedwork

KL divergence

Determinewhoauthoredthespecifiedwork

Figure 8:Qualitative token\-level visualizations\.Ocean animals

Cosine similarity

Givemealistof5animalsthatliveintheocean\.

W\-MLP weights

Givemealistof5animalsthatliveintheocean\.

KL divergence

Givemealistof5animalsthatliveintheocean\.

Moon question

Cosine similarity

Inthesolarsystemwhatplanethasthemostmoons?

W\-MLP weights

Inthesolarsystemwhatplanethasthemostmoons?

KL divergence

Inthesolarsystemwhatplanethasthemostmoons?

German→\\toChinese translation

Cosine similarity

GivenaGermanword,responditsChinesetranslation\.

W\-MLP weights

GivenaGermanword,responditsChinesetranslation\.

KL divergence

GivenaGermanword,responditsChinesetranslation\.

Figure 9:Qualitative token\-level visualizations\.Capital full prompt

Cosine similarity¡—begin\_of\_text—¿¡—eot\_id—¿¡—start\_header\_id—¿system¡—end\_header\_id—¿Whatisthecapitalofthiscountry?¡—eot\_id—¿¡—start\_header\_id—¿user¡—end\_header\_id—¿¡—eot\_id—¿¡—start\_header\_id—¿assistant¡—end\_header\_id—¿

W\-MLP weights¡—begin\_of\_text—¿¡—eot\_id—¿¡—start\_header\_id—¿system¡—end\_header\_id—¿Whatisthecapitalofthiscountry?¡—eot\_id—¿¡—start\_header\_id—¿user¡—end\_header\_id—¿¡—eot\_id—¿¡—start\_header\_id—¿assistant¡—end\_header\_id—¿

KL divergence¡—begin\_of\_text—¿¡—eot\_id—¿¡—start\_header\_id—¿system¡—end\_header\_id—¿Whatisthecapitalofthiscountry?¡—eot\_id—¿¡—start\_header\_id—¿user¡—end\_header\_id—¿¡—eot\_id—¿¡—start\_header\_id—¿assistant¡—end\_header\_id—¿

Capital prompt

Cosine similarity

Whatisthecapitalofthiscountry?

W\-MLP weights

Whatisthecapitalofthiscountry?

KL divergence

Whatisthecapitalofthiscountry?

Figure 10:Comparison between the capital prompt with and without chat\-template tokens\. The short prompt is aligned with the natural\-language task sentence inside the full prompt\.Harry Potter long prompt

Cosine similarityMr\.andMrs\.Dursley,ofnumberfour,PrivetDrive,wereproudtosaythattheywereperfectlynormal,thankyouverymuch\.Theywerethelastpeopleyou’dexpecttobeinvolvedinanythingstrangeormysterious,becausetheyjustdidn’tholdwithsuchnonsense\.

W\-MLP weightsMr\.andMrs\.Dursley,ofnumberfour,PrivetDrive,wereproudtosaythattheywereperfectlynormal,thankyouverymuch\.Theywerethelastpeopleyou’dexpecttobeinvolvedinanythingstrangeormysterious,becausetheyjustdidn’tholdwithsuchnonsense\.

KL divergenceMr\.andMrs\.Dursley,ofnumberfour,PrivetDrive,wereproudtosaythattheywereperfectlynormal,thankyouverymuch\.Theywerethelastpeopleyou’dexpecttobeinvolvedinanythingstrangeormysterious,becausetheyjustdidn’tholdwithsuchnonsense\.

Figure 11:Qualitative token\-level visualization for a longer prompt, split into two lines for readability\.
#### Interpretation

Across the examples, the learned weights tend to assign larger values to tokens that are semantically relevant for the task, such as task words, relation words, or answer\-format cues\. For example, high weights often appear on tokens such ascapital,authored,animals, orChinese translation\. This agrees with the intuition from the hand\-made weighting experiment: task information is not distributed uniformly across the prompt, but is concentrated in a small number of informative token positions\.

The cosine similarity and KL\-divergence visualizations provide a complementary view\. Tokens near the beginning of the prompt are generally less aligned with the final patch vector, while later and semantically more informative tokens are often closer\. The capital full\-prompt example also illustrates the effect of the prompt format: in the chat\-formatted setting, the task sentenceWhat is the capital of this country?appears together with special template tokens, which can also receive high similarity or weight\. This suggests that the learned patch vector can capture both task content and parts of the surrounding prompt structure\.

These figures are intended as qualitative evidence, rather than a causal analysis of individual tokens\. They provide a useful sanity check that the learned aggregation mechanism tends to focus on tokens that are meaningful for the task, rather than behaving like a uniform average over the prompt\.

## Appendix ETraining parameters

The training and evaluation parameters are summarized in[Table 9](https://arxiv.org/html/2607.08399#A5.T9)\.

ParameterValueBase modelLlama\-3\.1\-8B\-InstructPrompt formatinstruction/chat templateTraining tasks8 toy task familiesExtraction layer1212Patching layer22Patch scaler1\.01\.0Learning rate10−310^\{\-3\}Epochs55Batch sizeW\-MLP:88; Transformer:1616Target token position66Generation length2020Temperature0\.750\.75Placeholder tokenU\+FFFD, rendered as¿W\-MLP arch\.4096→2048→1024→512→256→14096\\\!\\to\\\!2048\\\!\\to\\\!1024\\\!\\to\\\!512\\\!\\to\\\!256\\\!\\to\\\!1Transformer arch\.dmodel=512d\_\{\\mathrm\{model\}\}=512,22heads,22layersMean centeringW\-MLP onlyTable 9:Training and evaluation parameters\.