VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring

# VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring
Source: [https://arxiv.org/html/2606.27941](https://arxiv.org/html/2606.27941)
###### Abstract

Sparse autoencoders \(SAEs\) provide useful decompositions of Transformer residual streams, but their learned features are usually named post hoc rather than directly connected to the Transformer’s token vocabulary\. We introduce Vocabulary\-Aligned Sparse Autoencoder \(VASAE\), a method that trains SAE features under vocabulary\-aligned anchoring and assigns each feature an intrinsic token name: the token string whose embedding is nearest to that feature\. Without reducing reconstruction quality compared with a standard SAE, VASAE produces dictionaries with vocabulary\-aligned features\. Using a 0\.8 cutoff on the nearest\-token alignment score, dictionaries trained on GPT\-2\-small post\-residual streams align about 90% of features in layers 0–10\. In Llama\-3\.1\-8B, representative shallow and middle\-layer dictionaries contain strongly aligned features, including 92\.8% in the shallow layer, while the representative final\-layer dictionary shows limited alignment\. After subtracting the sentence\-level mean sparse code, case studies show that many remaining intrinsic token names are relevant to nearby input tokens\. These results suggest that vocabulary\-aligned anchoring can connect learned features to intrinsic token names during training, complementing post hoc interpretation of learned dictionaries\.

Sparse Autoencoders, Mechanistic Interpretability, Vocabulary Alignment

## 1Introduction

Transformer architectures are a central model class for generative AI\(Vaswaniet al\.,[2017](https://arxiv.org/html/2606.27941#bib.bib28); Brownet al\.,[2020](https://arxiv.org/html/2606.27941#bib.bib6)\), and language\-model performance has improved with model and data scale\(Hoffmannet al\.,[2022](https://arxiv.org/html/2606.27941#bib.bib17)\)\. Their behavior, however, is produced by high\-dimensional internal states that are difficult to inspect directly\(Belinkov and Glass,[2019](https://arxiv.org/html/2606.27941#bib.bib2); Ghandehariounet al\.,[2024](https://arxiv.org/html/2606.27941#bib.bib13)\)\. In decoder\-only Transformers, one important state is the*residual stream*: the vector channel that each Transformer block reads from and writes to\(Elhageet al\.,[2021](https://arxiv.org/html/2606.27941#bib.bib7)\)\. Understanding this stream matters because it carries the intermediate information from which later layers compute next\-token predictions\(Gevaet al\.,[2021](https://arxiv.org/html/2606.27941#bib.bib12); Ghandehariounet al\.,[2024](https://arxiv.org/html/2606.27941#bib.bib13)\)\.

Sparse Autoencoders \(SAEs\)are increasingly used to decompose residual streams into sparse codes over learned dictionaries\(Hubenet al\.,[2024](https://arxiv.org/html/2606.27941#bib.bib18); Gaoet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib11)\)\. In this paper, the dictionary is the decoder weight matrix, and a*feature*is one column, or direction, of that matrix\. A*sparse code*is a vector of feature coefficients\. Each entry corresponds to one feature\. For a given residual\-stream vector, most entries are zero, and a larger nonzero value means that the corresponding feature contributes more strongly to the reconstruction\. Recent work uses SAEfeatures for circuit discovery as interpretability tools\(Markset al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib21); Karvonenet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib29)\)\. Standard SAEs, however, learn features for reconstruction and sparsity rather than direct naming\(Hubenet al\.,[2024](https://arxiv.org/html/2606.27941#bib.bib18); Gaoet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib11)\)\. Feature labels are usually assigned after training by inspecting contexts where the corresponding element of the sparse code is large, or by running a separate automated interpretation procedure\(Billset al\.,[2023](https://arxiv.org/html/2606.27941#bib.bib4); Pauloet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib24)\), making interpretation a separate post\-hoc step\. Vocabulary\-Aligned Sparse Autoencoder \(VASAE\)assigns a training\-time nearest\-token label to each learned decoder direction\.

The technical bottleneck is that the dictionary geometry and the vocabulary geometry are usually disconnected\. Prior work shows that language\-model representation spaces have nontrivial geometric structure, including anisotropy and dominant directions\(Ethayarajh,[2019](https://arxiv.org/html/2606.27941#bib.bib9); Mu and Viswanath,[2018](https://arxiv.org/html/2606.27941#bib.bib23)\)\. Separately, weight\-tying work shows that input and output embeddings are a central vocabulary interface in language models\(Press and Wolf,[2017](https://arxiv.org/html/2606.27941#bib.bib25); Inanet al\.,[2017](https://arxiv.org/html/2606.27941#bib.bib19)\)\. A learned feature can reconstruct activations while lying far from any token embedding, so it has no intrinsic vocabulary\-level name\. The opposite design would force the decoder to equal the token embedding matrix; in this paper we test whether this hard\-tied decoder is sufficient\. The missing object is therefore a learnable dictionary that remains flexible for reconstruction while staying close enough to vocabulary directions to provide candidate token\-based names\. A token name is a geometry\-derived label for a direction\.

Our key observation is that the language model already contains a model\-internal, vocabulary\-indexed set of reference directions when the token embeddings share dimensionality with the SAE decoder space: the fixed input token embeddings\(Vaswaniet al\.,[2017](https://arxiv.org/html/2606.27941#bib.bib28)\)\. We use these embeddings as anchors, not as frozen features\. Based on this observation, we propose VASAE, a SAEwhose learned features are trained with the usual sparse reconstruction objective plus a vocabulary\-aligned anchor objective\. Each feature is encouraged to stay close to its nearest token embedding, and after training receives an*intrinsic token name*: the token string whose embedding is nearest to that feature\. Here intrinsic means that the name is induced by the trained feature’s location relative to the model’s own vocabulary embeddings\.

Empirically, VASAEachieves reconstruction metrics comparable to a standard SAEunder the tested settings while adding a geometric vocabulary\-alignment signal\. Across layers, it has comparable variance explained \(VE\)on GPT\-2\-small \(0\.9650\.965\) and Llama\-3\.1\-8B \(0\.9310\.931\), while hard\-tying the decoder to token embeddings has lower reconstruction metrics\. On GPT\-2\-small,8989–94%94\\%of features in layersL​0L0–L​10L10exceed the diagnostic alignment cutoff\. With a larger tested anchor coefficient \(λanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}\), Llama\-3\.1\-8B reaches92\.8%92\.8\\%atL​0L0but remains unstable in the final representative layer\. Case studies show intrinsic token names assigned to active features in context\.

Our contributions are:

- •We introduce VASAE: a soft vocabulary\-anchoring objective that assigns learned SAEdecoder directions nearest\-token names\.
- •We evaluate reconstruction under vocabulary anchoring and test whether hard\-tying the decoder to token embeddings is sufficient\.
- •We analyze embedding\-space feature alignment and vocabulary coverage, and use case studies to show intrinsic token names as geometry\-derived identifiers\.

## 2Background

Residual\-stream SAEs, token embeddings, and vocabulary readouts all depend on the geometry of Transformer residual states\. VASAEuses this geometry to define vocabulary\-indexed anchors for learned SAEdecoder directions\.

### 2\.1Transformer Residual Stream

Transformer models maintain a persistent residual stream, which carries information across layers\(Vaswaniet al\.,[2017](https://arxiv.org/html/2606.27941#bib.bib28); Elhageet al\.,[2021](https://arxiv.org/html/2606.27941#bib.bib7)\)\. Panel A of Figure[1](https://arxiv.org/html/2606.27941#S3.F1)shows the standard Transformer structure used in this setup\. Let𝒱\\mathcal\{V\}be the vocabulary space and let𝐖E∈ℝ\|𝒱\|×dmodel\\mathbf\{W\}\_\{E\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\_\{\\mathrm\{model\}\}\}be the token embedding matrix, wheredmodeld\_\{\\mathrm\{model\}\}is the model dimension\. Throughout the paper, we use a row\-vector convention to stay consistent with implementations\. For a tokenv∈𝒱v\\in\\mathcal\{V\}, we denote its embedding by𝐰v∈ℝdmodel\\mathbf\{w\}\_\{v\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\}and write the initial post\-residual stream as𝐡\(0\)=𝐰v\\mathbf\{h\}^\{\(0\)\}=\\mathbf\{w\}\_\{v\}\.

At each layerℓ\\ell, the residual stream is updated by adding the outputs of the attention and multi\-layer perceptron \(MLP\)modules:

𝐡\(ℓ\)=𝐡\(ℓ−1\)\+Attnℓ⁡\(𝐡\(ℓ−1\)\)\+MLPℓ⁡\(𝐡\(ℓ−1\)\)\.\\mathbf\{h\}^\{\(\\ell\)\}=\\mathbf\{h\}^\{\(\\ell\-1\)\}\+\\operatorname\{Attn\}\_\{\\ell\}\(\\mathbf\{h\}^\{\(\\ell\-1\)\}\)\+\\operatorname\{MLP\}\_\{\\ell\}\(\\mathbf\{h\}^\{\(\\ell\-1\)\}\)\.\(1\)This schematic update omits the sequential sublayer structure and LayerNorm operations of the underlying Pre\-LN Transformer blocks\.

We call𝐡\(ℓ\)∈ℝdmodel\\mathbf\{h\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\}the layer\-ℓ\\ellpost\-residual stream: the residual stream after layerℓ\\ellhas added its attention and MLPoutputs\. In this work, all SAEvariants are trained and evaluated on the post\-residual stream, not on the intermediate attention or MLPoutputs before they are added back into the residual stream\.

For autoregressive language models, the final prediction logits are obtained by projecting the last\-layer post\-residual stream through the unembedding matrix𝐖U∈ℝ\|𝒱\|×dmodel\\mathbf\{W\}\_\{U\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\_\{\\mathrm\{model\}\}\}:

𝐮=𝐡\(L\)​𝐖U⊤\.\\mathbf\{u\}=\\mathbf\{h\}^\{\(L\)\}\\mathbf\{W\}\_\{U\}^\{\\top\}\.\(2\)
Embedding and unembedding weight tying is a standard language\-model design choice\(Press and Wolf,[2017](https://arxiv.org/html/2606.27941#bib.bib25); Inanet al\.,[2017](https://arxiv.org/html/2606.27941#bib.bib19)\)\. In the Generative Pre\-trained Transformer 2 \(GPT\-2\)\-small checkpoint used here\(Radfordet al\.,[2019](https://arxiv.org/html/2606.27941#bib.bib26)\), the embedding and unembedding matrices are tied \(𝐖U=𝐖E\\mathbf\{W\}\_\{U\}=\\mathbf\{W\}\_\{E\}\)\. In the Llama\-3\.1\-8B checkpoint used here\(Grattafioriet al\.,[2024](https://arxiv.org/html/2606.27941#bib.bib14)\), they are not\. For models with untied input embeddings and output unembeddings, the choice of anchor matrix is not unique\. Input embeddings provide vocabulary\-indexed residual directions near the model input, whereas unembedding directions may be more appropriate for later layers\. In this work we focus on input\-embedding anchors and treat unembedding\-based anchors as a possible extension\.

### 2\.2Vocabulary\-Aligned Probing

A common diagnostic for Transformer residual streams is to project intermediate post\-residual streams into vocabulary\-logit space\. In our auxiliary LogitLens\-style readout, matching the implementation used for the reported readout metric, we directly project the post\-residual stream from intermediate layers through the unembedding matrix:

𝐮\(ℓ\)=𝐡\(ℓ\)​𝐖U⊤\.\\mathbf\{u\}^\{\(\\ell\)\}=\\mathbf\{h\}^\{\(\\ell\)\}\\mathbf\{W\}\_\{U\}^\{\\top\}\.\(3\)
This measures which token logits are linearly decodable from an intermediate post\-residual stream under this direct readout\. It is separate from the cross\-entropy \(CE\)recovery metric, where the patched residual stream is passed through the remaining model computation before final logits are read\. This background motivates the reference frame used by VASAE: vocabulary\-indexed directions can be used to relate residual\-stream geometry to token space\. Unlike a logit\-lens readout, VASAEdoes not decode activations into token predictions; it softly anchors learned SAEdecoder directions to vocabulary\-indexed reference directions\.

### 2\.3SAEs for Residual\-Stream Analysis

SAEsmap a post\-residual stream to a sparse code and then decode the sparse code back into residual\-stream space\. Panel B of Figure[1](https://arxiv.org/html/2606.27941#S3.F1)shows this encoder–decoder structure\. SAEvariants differ in how they parameterize the encoder and enforce sparsity\. This paper uses a top\-kkSAE\(Makhzani and Frey,[2014](https://arxiv.org/html/2606.27941#bib.bib20); Gaoet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib11)\), where sparsity is achieved by keeping only the largestkkencoder coordinates after ReLU\.

The encoder uses𝐖ℰ∈ℝdmodel×dsparse\\mathbf\{W\}\_\{\\mathcal\{E\}\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\\times d\_\{\\mathrm\{sparse\}\}\}and𝐛ℰ∈ℝdsparse\\mathbf\{b\}\_\{\\mathcal\{E\}\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{sparse\}\}\}, applying a learned affine map followed by ReLU and top\-kkselection:

𝐳=ℰ​\(𝐡\)=TopKk⁡\(ReLU⁡\(𝐡𝐖ℰ\+𝐛ℰ\)\)\.\\mathbf\{z\}=\\mathcal\{E\}\(\\mathbf\{h\}\)=\\operatorname\{TopK\}\_\{k\}\\\!\\left\(\\operatorname\{ReLU\}\(\\mathbf\{h\}\\mathbf\{W\}\_\{\\mathcal\{E\}\}\+\\mathbf\{b\}\_\{\\mathcal\{E\}\}\)\\right\)\.\(4\)
Hereℰ\\mathcal\{E\}denotes the full sparse encoder, including the ReLU nonlinearity and top\-kkselection\. The vector𝐳\\mathbf\{z\}is the sparse code\. Each scalar𝐳i\\mathbf\{z\}\_\{i\}is one element in the sparse code vector and is the coefficient for featureii\. The operatorTopKk\\operatorname\{TopK\}\_\{k\}keeps the largestkkcoordinates and sets all others to zero, so each sparse code satisfies‖𝐳‖0≤k\\left\\lVert\\mathbf\{z\}\\right\\rVert\_\{0\}\\leq k, where∥⋅∥0\\left\\lVert\\cdot\\right\\rVert\_\{0\}denotes the number of nonzero coordinates\.

The decoder uses𝐖𝒟∈ℝdmodel×dsparse\\mathbf\{W\}\_\{\\mathcal\{D\}\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\\times d\_\{\\mathrm\{sparse\}\}\}and𝐛𝒟∈ℝdmodel\\mathbf\{b\}\_\{\\mathcal\{D\}\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\}to map the sparse code back to the residual\-stream space:

𝐡~=𝐳𝐖𝒟⊤\+𝐛𝒟\.\\tilde\{\\mathbf\{h\}\}=\\mathbf\{z\}\\mathbf\{W\}\_\{\\mathcal\{D\}\}^\{\\top\}\+\\mathbf\{b\}\_\{\\mathcal\{D\}\}\.\(5\)The online experiments use the extracted post\-residual streams directly; we do not subtract a separate stored activation mean before the encoder\. Constant offsets can be represented by the learned affine encoder and decoder bias terms\.

For a training set ofNNpost\-residual streams\{𝐡i\}i=1N\\\{\\mathbf\{h\}\_\{i\}\\\}\_\{i=1\}^\{N\}, whereNNcounts evaluated token positions, the SAEtraining loss is the empirical reconstruction error under this sparse encoder:

ℒrecon=1N​∑i=1N‖𝐡i−𝐡~i‖22\.\\mathcal\{L\}\_\{\\text\{recon\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\\lVert\\mathbf\{h\}\_\{i\}\-\\tilde\{\\mathbf\{h\}\}\_\{i\}\\right\\rVert\_\{2\}^\{2\}\.\(6\)
Sparsity means only a few elements of the sparse code are nonzero for a given post\-residual stream, yielding a decomposition with a small active feature set\. In Section[3](https://arxiv.org/html/2606.27941#S3), we keep the same reconstruction problem with sparse codes and add vocabulary\-aligned anchoring to the dictionary\.

## 3Method

We decompose the post\-residual stream with a learned dictionary whose features are anchored to the fixed token\-embedding directions by a vocabulary\-aligned objective\. Sparsity enters through the sparse code: for a given post\-residual stream, only a few dictionary features have nonzero corresponding elements in the sparse code\. Our proposed VASAEmethod keeps the SAEdictionary learnable and uses token embeddings only as naming anchors\. The method is summarized in Figure[1](https://arxiv.org/html/2606.27941#S3.F1)\.

![Refer to caption](https://arxiv.org/html/2606.27941v1/x1.png)Figure 1:Transformer residual\-stream decomposition with VASAE\. The model learns the dictionary𝐖𝒟\\mathbf\{W\}\_\{\\mathcal\{D\}\}under a reconstruction objective plus the vocabulary\-anchor lossℒanchor\\mathcal\{L\}\_\{\\text\{anchor\}\}\.### 3\.1VASAEArchitecture

For each model layer, we train a separate VASAEmodel on the post\-residual stream from that layer\. The input is𝐡∈ℝdmodel\\mathbf\{h\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\}, and the encoder is the same linear ReLU top\-kkmap as in Equation[4](https://arxiv.org/html/2606.27941#S2.E4)\. This produces a nonnegative sparse code𝐳∈ℝdsparse\\mathbf\{z\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{sparse\}\}\}with at mostkknonzero elements\. The reconstruction is produced by the learned decoder𝐖𝒟\\mathbf\{W\}\_\{\\mathcal\{D\}\}as in Equation[5](https://arxiv.org/html/2606.27941#S2.E5)\.

We write the dictionary as𝐖𝒟=\(𝐝1,…,𝐝dsparse\)\\mathbf\{W\}\_\{\\mathcal\{D\}\}=\(\\mathbf\{d\}\_\{1\},\\ldots,\\mathbf\{d\}\_\{d\_\{\\mathrm\{sparse\}\}\}\), where𝐝i∈ℝdmodel\\mathbf\{d\}\_\{i\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\}is featureii\. The scalar𝐳i\\mathbf\{z\}\_\{i\}is one element in the sparse code vector and is the coefficient for featureii\.

The sparse code dimensiondsparsed\_\{\\mathrm\{sparse\}\}is a configurable VASAEhyperparameter\. In the experiments, we instantiate the vocabulary\-sized casedsparse=\|𝒱\|d\_\{\\mathrm\{sparse\}\}=\|\\mathcal\{V\}\|, so the learned dictionary has one feature slot per vocabulary item\. Other dictionary sizes are compatible with the method\. The decoder columns remain learnable, and token embeddings are fixed reference directions during VASAEtraining\. We use vocabulary\-sized dictionaries in the main experiments to make the learned feature set and vocabulary anchor set comparable in scale\.

### 3\.2Vocabulary\-Aligned Anchoring

VASAEassumes that SAEdecoder directions and the chosen vocabulary anchor directions live in the same dimensional vector space\. For residual\-stream SAEs, this assumption holds when the model’s token embedding vectors have the same width as the residual stream\. Architectures with factorized embeddings or mismatched dimensions would require an explicit learnable projection matrix that maps token embeddings to the residual\-stream dimension, or an anchor set already defined in the residual\-stream space\.

For each learned feature𝐝i\\mathbf\{d\}\_\{i\}, VASAEcompares the feature with every fixed token embedding and records both the best cosine similarity and the vocabulary item that attains it:

si\\displaystyle s\_\{i\}=maxv∈𝒱⁡cos⁡\(𝐝i,𝐰v\),\\displaystyle=\\max\_\{v\\in\\mathcal\{V\}\}\\cos\(\\mathbf\{d\}\_\{i\},\\mathbf\{w\}\_\{v\}\),\(7\)vi⋆\\displaystyle v\_\{i\}^\{\\star\}=arg​maxv∈𝒱⁡cos⁡\(𝐝i,𝐰v\)\.\\displaystyle=\\operatorname\*\{arg\\,max\}\_\{v\\in\\mathcal\{V\}\}\\cos\(\\mathbf\{d\}\_\{i\},\\mathbf\{w\}\_\{v\}\)\.\(8\)
Thus,sis\_\{i\}is the nearest\-token alignment score: it is large only when featureiilies close to some vocabulary direction\. The vocabulary itemvi⋆v\_\{i\}^\{\\star\}determines the intrinsic token name for the feature\. The feature vector𝐝i\\mathbf\{d\}\_\{i\}itself remains free to move under the reconstruction objective\. Cosine similarities are computed afterL2L\_\{2\}\-normalizing decoder columns and token embedding vectors\. For each decoder column, we compute cosine similarity to all token embeddings\. The nearest token embedding defines the current anchor\. The loss below rewards high similarity to this anchor and is optimized together with reconstruction under the top\-kksparse\-code constraint\.

To increase nearest\-token alignment, we define the anchor loss so that minimizing it increases the cosine similarity between each feature and its nearest token embedding:

ℒanchor=−1dsparse​∑i=1dsparsesi\.\\mathcal\{L\}\_\{\\text\{anchor\}\}=\-\\frac\{1\}\{d\_\{\\mathrm\{sparse\}\}\}\\sum\_\{i=1\}^\{d\_\{\\mathrm\{sparse\}\}\}s\_\{i\}\.\(9\)
Combined with reconstruction loss in Equation[6](https://arxiv.org/html/2606.27941#S2.E6), the total loss of VASAEis

ℒvasae=ℒrecon\+λanchor​ℒanchor,\\mathcal\{L\}\_\{\\text\{vasae\}\}=\\mathcal\{L\}\_\{\\text\{recon\}\}\+\\lambda\_\{\\text\{anchor\}\}\\mathcal\{L\}\_\{\\text\{anchor\}\},\(10\)
During optimization,vi⋆v\_\{i\}^\{\\star\}is recomputed from the current feature vector𝐝i\\mathbf\{d\}\_\{i\}, so the nearest\-token anchor can change as the dictionary changes\. The token embeddings serve only as fixed reference vectors\.

The anchor coefficient sets the weight of the vocabulary\-alignment term relative to reconstruction\. The top\-kkconstraint fixes the number of active features separately\. Largerλanchor\\lambda\_\{\\text\{anchor\}\}values pull decoder directions more strongly toward token embeddings, so we tuneλanchor\\lambda\_\{\\text\{anchor\}\}as a regularization hyperparameter\.

For large vocabularies, we compute the nearest\-token scores and lookup with a memory\-bounded chunked similarity implementation\. The chunked computation avoids materializing the full feature\-by\-vocabulary similarity matrix\. Details are given in Appendix[F](https://arxiv.org/html/2606.27941#A6)\.

### 3\.3Feature Naming

After training, each feature𝐝i\\mathbf\{d\}\_\{i\}receives an intrinsic token name\. The name is the token string corresponding to the nearest vocabulary itemvi⋆v\_\{i\}^\{\\star\}in Equation[8](https://arxiv.org/html/2606.27941#S3.E8)\. We use the term intrinsic token name to mean the nearest vocabulary token under cosine similarity in the chosen embedding space\. This is a geometric identifier for the feature\. We call this name intrinsic because it is determined by the geometry between the learned feature vector and the fixed token\-embedding matrix, rather than by manually inspecting contexts where elements of the sparse code are large or by applying an additional post\-hoc labeling procedure\.

We reportsis\_\{i\}together with the intrinsic token name as the feature’s alignment score\. A largersis\_\{i\}means that the feature vector is closer to its named token embedding\. A smallersis\_\{i\}means that the nearest token still gives the closest vocabulary direction, but with lower geometric support\.

## 4Experiments

We evaluate whether vocabulary\-aligned anchoring largely preserves SAEreconstruction, learns dictionaries with token\-aligned features, and assigns token names to features with large sparse\-code coefficients\.

### 4\.1Experimental Setup

#### Dataset\.

Our training and evaluation data are post\-residual\-stream activations extracted from language\-model forward passes on WikiText\-103\(Merityet al\.,[2017](https://arxiv.org/html/2606.27941#bib.bib22)\)\. We split sequences into train/validation/test subsets as follows:50,000/10,000/5,00050\{,\}000/10\{,\}000/5\{,\}000for GPT\-2\-small and20,000/2,000/5,00020\{,\}000/2\{,\}000/5\{,\}000for Llama\-3\.1\-8B\. Each sequence is truncated or padded to a maximum length of128128, giving at most about6\.46\.4M/1\.31\.3M/0\.60\.6M token positions for GPT\-2\-small and2\.62\.6M/0\.30\.3M/0\.60\.6M token positions for Llama\-3\.1\-8B before padding positions are masked\. We run the language model on the tokenized sequences and extract post\-residual\-stream activations online with thennsight\(Fiotto\-Kaufmanet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib10)\)tracing framework\. We use two decoder\-only Transformer checkpoints to cover a smaller GPT\-2 setting and a larger Llama setting:

- •GPT\-2\-small: a GPT\-2 checkpoint used for full layerwise analysis\. It has model dimensiondmodel=768d\_\{\\mathrm\{model\}\}=768, vocabulary size\|𝒱\|=50,257\|\\mathcal\{V\}\|=50\{,\}257, and1212layers\(Radfordet al\.,[2019](https://arxiv.org/html/2606.27941#bib.bib26)\)\.
- •Llama\-3\.1\-8B: a larger open\-weight Llama checkpoint used to test the method in a higher\-dimensional vocabulary and residual\-stream geometry\. It has model dimensiondmodel=4096d\_\{\\mathrm\{model\}\}=4096, vocabulary size\|𝒱\|=128,256\|\\mathcal\{V\}\|=128\{,\}256, and3232layers\(Grattafioriet al\.,[2024](https://arxiv.org/html/2606.27941#bib.bib14)\)\.

VASAEand all baselines are trained and evaluated on these extracted post\-residual streams\. Padding positions are excluded from training and evaluation with the tokenizer attention mask\.

#### Baselines\.

- •Standard SAE: a SAEtrained directly on post\-residual streams without explicit anchoring to fixed token\-embedding directions\.
- •Hard\-tied decoder baseline: a comparison method that uses exact token embeddings as features\. The decoder is fixed during training\. Only the encoder is optimized\.

The hard\-tied decoder baseline tests the direct alternative of representing post\-residual streams only with token\-embedding features\. Because its decoder features are exactly the vocabulary embeddings, it necessarily hasdsparse=\|𝒱\|d\_\{\\mathrm\{sparse\}\}=\|\\mathcal\{V\}\|\. It sets𝐖𝒟⊤=𝐖E\\mathbf\{W\}\_\{\\mathcal\{D\}\}^\{\\top\}=\\mathbf\{W\}\_\{E\}in Equation[5](https://arxiv.org/html/2606.27941#S2.E5)and reconstructs as𝐡~=𝐳𝐖E\\tilde\{\\mathbf\{h\}\}=\\mathbf\{z\}\\mathbf\{W\}\_\{E\}\. The encoder is trained, but the decoder weights are frozen\. This baseline tests whether hard\-tying the decoder to token embeddings is sufficient\.

### 4\.2Reconstruction Preservation

We first ask whether vocabulary\-aligned anchoring changes reconstruction quality relative to a standard SAE\. We report reconstruction error and cross\-entropy loss after residual\-stream substitution\.

#### Training protocol\.

All three models, standard SAE, hard\-tied SAE, and VASAE, use vocabulary\-sized sparse code dimension\|𝒱\|\|\\mathcal\{V\}\|, top\-kksparsity withk=32k=32, nonnegative sparse code, a linear encoder, and Adam with learning rate10−310^\{\-3\}\. For standard SAEand VASAE, the vocabulary\-sized dimension is chosen to match the hard\-tied baseline\. Training runs for at most2020epochs with patience\-33early stopping on validation reconstruction loss, and we evaluate the best checkpoint on the test split\. For VASAE, the anchor term is optimized during training but is not included in validation checkpoint selection\. The default anchor coefficient isλanchor=10−4\\lambda\_\{\\text\{anchor\}\}=10^\{\-4\}\. For Llama\-3\.1\-8B geometric\-alignment and case\-study diagnostics, we additionally useλanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}; the supporting anchor\-strength sweep is reported in Appendix[B](https://arxiv.org/html/2606.27941#A2)\. GPT\-2\-small runs use float32 with batch size3232\. Llama\-3\.1\-8B runs use bfloat16 with batch size88; for Llama VASAEruns, the anchor term is evaluated once every5050optimizer steps, and skipped\-anchor steps optimize only the reconstruction loss\. Each reported training run uses a single NVIDIA GH200 GPU with 96GB GPU memory\. Nearest\-token lookup is computed over the model tokenizer’s full vocabulary\. Padding positions are excluded from activation training and evaluation through the attention mask\.

Table 1:Results, reported as mean±\\pmstandard deviation across layers\. Arrows indicate the preferred direction for each metric\. Shaded rows denote the proposed method\. Best values within each model block are bolded\.
#### Evaluation metrics\.

Table[1](https://arxiv.org/html/2606.27941#S4.T1)reports four metrics\. Exact formulas are given in Appendix[A](https://arxiv.org/html/2606.27941#A1)\.

- •Mean Squared Error \(MSE\): residual\-stream reconstruction error\. Lower values mean the reconstruction is closer to the target activation\.
- •Variance Explained \(VE\): scale\-normalized reconstruction quality\. Higher values mean the reconstruction preserves more target activation variance\.
- •Cross\-Entropy Loss \(CE loss\): downstream next\-token loss after substituting the reconstruction at the evaluated residual\-stream site\. Lower values mean better preservation of model predictions\.
- •Cross\-Entropy Recovery \(CE rec\.\): downstream loss recovery relative to the gap between the original residual stream and a zero\-vector control\. Values near11match the original stream, values near0match the zero\-vector control, and negative values are worse than that control\.

Table[1](https://arxiv.org/html/2606.27941#S4.T1)reports mean±\\pmstandard deviation across layers\. The standard deviations are across layers rather than random seeds\. VASAEhas reconstruction metrics comparable to a standard SAEin both tested model families\. On GPT\-2\-small, the two learned\-dictionary methods have the same VE\(0\.9650\.965\) and CErecovery \(0\.9750\.975\)\. On Llama\-3\.1\-8B, they also have the same VE\(0\.9310\.931\) and CErecovery \(0\.9060\.906\)\. The hard\-tied decoder baseline performs much worse, showing that directly forcing every feature to be an exact token embedding is not sufficient\. This contrast supports the central design choice in VASAE: keep the dictionary learnable while anchoring features to token embeddings\. Layerwise VE, CErecovery, and auxiliary LogitLensdiagnostics are shown in Appendix[C](https://arxiv.org/html/2606.27941#A3)\.

### 4\.3Geometric Token Alignment

Having evaluated reconstruction, we next measure whether learned features become geometrically token aligned using the alignment score defined in Equation[7](https://arxiv.org/html/2606.27941#S3.E7)\. This section checks whether the anchor loss produces token\-aligned decoder directions\.

#### Evaluation setup\.

For GPT\-2\-small, we compare VASAEwith a standard SAEusingλanchor=10−4\\lambda\_\{\\text\{anchor\}\}=10^\{\-4\}\. For Llama\-3\.1\-8B, we useλanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}becauseλanchor=10−4\\lambda\_\{\\text\{anchor\}\}=10^\{\-4\}preserves reconstruction but produces lower token alignment scores; the appendix sweep shows that5×10−35\\times 10^\{\-3\}changes VEand CErecovery only slightly on representative layers\. For both model families, we use the alignment scoresis\_\{i\}and its associated nearest vocabulary itemvi⋆v\_\{i\}^\{\\star\}, defined in Equation[7](https://arxiv.org/html/2606.27941#S3.E7)and Equation[8](https://arxiv.org/html/2606.27941#S3.E8), respectively\. With thresholdτ=0\.8\\tau=0\.8, the strongly aligned feature\-index set is𝒜τ=\{i:si≥τ\}\\mathcal\{A\}\_\{\\tau\}=\\\{i:s\_\{i\}\\geq\\tau\\\}\. This value was chosen from the observed separation in alignment\-score distributions: standard SAEfeatures concentrate well below0\.30\.3, while many VASAEfeatures cluster near1\.01\.0, as shown in Figure[2](https://arxiv.org/html/2606.27941#S4.F2)\. The threshold is a diagnostic cutoff for geometric alignment; the full score distribution remains the primary evidence\. We report the geometric feature\-alignment rate\|𝒜τ\|/dsparse\|\\mathcal\{A\}\_\{\\tau\}\|/d\_\{\\mathrm\{sparse\}\}and vocabulary coverage\|\{vi⋆:i∈𝒜τ\}\|/\|𝒱\|\|\\\{v\_\{i\}^\{\\star\}:i\\in\\mathcal\{A\}\_\{\\tau\}\\\}\|/\|\\mathcal\{V\}\|, since many features can be aligned to the same token\. Coverage is the fraction of vocabulary tokens used as nearest\-token names by at least one strongly aligned feature\. The denominator is the full\-learned dictionary, so this rate should be read as a geometric property of the feature set\.

#### GPT\-2\-small shows high alignment across most layers\.

For GPT\-2\-small, Figure[2](https://arxiv.org/html/2606.27941#S4.F2)left summarizes representative even\-numbered layers fromL​0L0toL​10L10\. The alignment distributions are clearly separated: about93%93\\%of VASAEfeatures in shallow and middle layers have alignment scores near1\.01\.0, whereas standard SAEfeatures remain concentrated around0\.10\.1–0\.20\.2and never exceed the diagnostic cutoff\. Under this criterion, post\-hoc nearest\-token naming of a standard SAEis a low\-alignment baseline: the nearest token exists by definition, but it is typically far from the learned feature direction\. Across layers, the geometric feature\-alignment rate stays between89%89\\%and94%94\\%forL​0L0–L​10L10and is68\.5%68\.5\\%inL​11L11\. Coverage of unique aligned tokens is around53%53\\%–56%56\\%, indicating that multiple features can share an intrinsic token name\. At this coverage level, the naming map is many\-to\-one: many aligned features share nearest\-token names\. These results support the main alignment claim: in GPT\-2\-small, vocabulary\-aligned anchoring converts a standard SAEdictionary into a geometrically token\-aligned dictionary with comparable reconstruction metrics\.

#### Llama\-3\.1\-8B alignment is shallower\.

Atλanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}, the right panel of Figure[2](https://arxiv.org/html/2606.27941#S4.F2)shows a depth\-dependent pattern: alignment is highest atL​0L0, lower atL​15L15, and not comparable atL​31L31\. TheL​0L0dictionary has119,016119\{,\}016strongly aligned features out of128,256128\{,\}256\(92\.8%92\.8\\%\), with77,37177\{,\}371unique aligned token names \(60\.33%60\.33\\%vocabulary coverage\)\. These Llama\-3\.1\-8B results indicate higher alignment in shallow layers and less stable alignment in deeper layers, matching the hypothesis that vocabulary anchoring is more compatible with residual states closer to embedding space\.

![Refer to caption](https://arxiv.org/html/2606.27941v1/x2.png)Figure 2:Geometric alignment diagnostics\. Left: GPT\-2\-small alignment score distributions for representative even\-numbered layers fromL​0L0toL​10L10\. Right: Llama\-3\.1\-8B alignment score distributions atλanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}forL​0L0,L​15L15, andL​31L31\. The dashed line marks the diagnostic strong\-alignment thresholdsi=0\.8s\_\{i\}=0\.8\.

### 4\.4Case Studies of Intrinsic Token Names

#### Case\-study setup\.

We next inspect the intrinsic token names assigned to aligned features that are active in context\. These case studies show token\-level intrinsic names on individual prompts\. The main location example uses the fixed input sentence “The cafe is located on Baker Street, just around the corner from the avenue”\. We tokenize this sentence with each model’s tokenizer and display the first 12 non\-padding token positions\. The GPT\-2\-small visualization uses representative even\-numbered layers fromL​0L0toL​10L10; the Llama\-3\.1\-8B visualization uses layersL​0,L​15,L​31L0,L15,L31\. For each displayed layerℓ\\ell, let𝐳p∈ℝdsparse\\mathbf\{z\}\_\{p\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{sparse\}\}\}be the sparse code coefficient at token positionppin a sentence of lengthTT, and let𝒜τ\\mathcal\{A\}\_\{\\tau\}be the strongly aligned feature\-index set from the previous subsection\. If a feature is active at many positions in the same sentence, raw activation alone can choose that same feature repeatedly for display\. We therefore choose the aligned feature that is most elevated relative to the sentence average:

𝐳¯=1T∑q=1T𝐳q,ip⋆∈arg​maxi∈𝒜τ\[𝐳p−𝐳¯\]i\.\\bar\{\\mathbf\{z\}\}=\\frac\{1\}\{T\}\\sum\_\{q=1\}^\{T\}\\mathbf\{z\}\_\{q\},\\qquad i\_\{p\}^\{\\star\}\\in\\operatorname\*\{arg\\,max\}\_\{i\\in\\mathcal\{A\}\_\{\\tau\}\}\[\\mathbf\{z\}\_\{p\}\-\\bar\{\\mathbf\{z\}\}\]\_\{i\}\.\(11\)We display the intrinsic token name of the chosen feature, namely the token string corresponding tovip⋆⋆v\_\{i\_\{p\}^\{\\star\}\}^\{\\star\}\. The relative sparse code is used only to choose the displayed feature\. The color shows the raw sparse\-code activation\[𝐳p\]ip⋆\[\\mathbf\{z\}\_\{p\}\]\_\{i\_\{p\}^\{\\star\}\}\.

![Refer to caption](https://arxiv.org/html/2606.27941v1/x3.png)Figure 3:GPT\-2\-small location case study\. Each cell shows the intrinsic token name of the aligned feature with the largest sentence\-centered sparse\-code value at a layer and input position\. Color indicates the raw sparse\-code activation of that feature\. Shallow and middle layers show location\-related intrinsic token names\.
#### GPT\-2\-small example\.

Figure[3](https://arxiv.org/html/2606.27941#S4.F3)shows a representative GPT\-2\-small location example\. Shallow and middle layers repeatedly display location\-related intrinsic token names such aslocated,location,Street,around,corner, andcornersaround the input phrase “Baker Street” and the surrounding spatial context\. Additional examples are shown in Appendix[E](https://arxiv.org/html/2606.27941#A5), including adjective/adverb words, award\-related words, and self\-introduction words\. These visualizations show intrinsic token names after geometric alignment has been established and provide qualitative inspection examples\.

#### Llama\-3\.1\-8B example\.

Figure[4](https://arxiv.org/html/2606.27941#S4.F4)applies the same sentence\-centered rule to Llama\-3\.1\-8B atλanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}for the representative layersL​0L0,L​15L15, andL​31L31\. The shallow layer shows location\-related token names, the middle layer is less consistent, and the final layer is mostly unstable\. This qualitative pattern matches the geometric diagnostics in Figure[2](https://arxiv.org/html/2606.27941#S4.F2)\.

![Refer to caption](https://arxiv.org/html/2606.27941v1/x4.png)Figure 4:Llama\-3\.1\-8B case study atλanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}\. Each cell shows the intrinsic token name of the feature chosen by the same sentence\-centered sparse\-code rule as Figure[3](https://arxiv.org/html/2606.27941#S4.F3)\. Color indicates the raw sparse\-code activation of that feature\. The shallow representative layer shows location\-related token names, the middle representative layer is less consistent, and the final representative layer is unstable\.

## 5Discussion

#### What VASAE names do and do not mean\.

A VASAEname is a vocabulary\-indexed handle for a learned decoder direction: the token whose embedding is nearest to that direction\. Its role is to attach a model\-internal token reference to the feature during training, rather than adding a separate naming step after feature learning\.

The main point to avoid over\-reading is that this token handle is not a feature explanation\. It gives a starting point for inspection, but the meaning or role of the feature still has to be established from how it activates, how it connects to other model components, or what changes under intervention\.

#### What counts as explaining a feature\.

Feature interpretation depends on the research goal\. The examples below illustrate a distinction that is easy to blur in SAEanalysis: a feature can be useful because information is readable from it, because it participates in an internal computation, or because intervening on it changes behavior\. Each use calls for its own evidence\. This matters because a named feature can make an interpretation look more complete than it is\. A safer use of feature names is to state which question the feature is being used to answer and what evidence supports that question\.

For example,semantic readouttreats a feature as a direction from which some information may be readable\. A feature whose nearest token is location\-related may be a candidate direction for residual\-stream information about place names, street names, spatial prepositions, or local descriptions of where something is; evidence for this use comes from top\-activating contexts, activation distributions across prompts, and comparisons with nearby candidate features\.Computational roleasks whether the feature participates in an internal calculation, such as tracking an intermediate variable for entity resolution, maintaining a syntactic constraint, combining local context into a next\-token preference, or passing information from one component to another; this requires circuit\-level evidence such as ablations, activation patching, path patching, or dependencies between upstream and downstream features and modules\.Behavioral controlasks whether intervening on the feature changes outputs in a predictable way, for example shifting generated text toward a more formal, cautious, specific, or creative style\. In that setting, the interpretation is tied to the intervention effect, not only to the contexts where the feature activates or to its nearest\-token name\.

For the goal of understanding how a language model implements a behavior, this suggests shifting attention from whether an isolated feature has the right name to how named features enter circuits\. The central object is the mechanism: how features, attention heads, MLP components, and residual\-stream directions pass information forward, transform it, and affect behavior\. In this view, feature naming is a preliminary indexing tool\. The analysis should ask which components use the feature, which downstream paths change when it is perturbed, and which behavior is explained by the resulting circuit\.

## 6Related Work

### 6\.1Vocabulary Readout and Representation Geometry

Probing work formalizes how to test whether intermediate post\-residual streams linearly encode linguistic or task\-relevant attributes\(Tenneyet al\.,[2019](https://arxiv.org/html/2606.27941#bib.bib27); Hewitt and Liang,[2019](https://arxiv.org/html/2606.27941#bib.bib16)\)\. In autoregressive Transformers, layerwise vocabulary\-readout methods, from LogitLens\-style readouts to tuned\-lens variants, show how token\-level predictions can be decoded and analyzed across Transformer layers\(Belroseet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib3)\)\. Our work is complementary to this literature: rather than studying decodability alone, we study decompositions of the residual stream itself\.

Related work on representation geometry studies how information is arranged in language\-model vector spaces, including whether representations are anisotropic or dominated by common directions\(Ethayarajh,[2019](https://arxiv.org/html/2606.27941#bib.bib9); Mu and Viswanath,[2018](https://arxiv.org/html/2606.27941#bib.bib23)\)\. These works motivate geometric analysis of language\-model representations\. We use token embeddings as the reference frame because they provide vocabulary\-indexed directions in the residual\-stream space, while weight\-tying work shows that embedding and output\-token geometry are central interfaces in language models\(Press and Wolf,[2017](https://arxiv.org/html/2606.27941#bib.bib25); Inanet al\.,[2017](https://arxiv.org/html/2606.27941#bib.bib19)\)\. Our question is more specific: can a learned dictionary stay close enough to vocabulary directions to provide intrinsic token names while still reconstructing the post\-residual stream from sparse codes, without being forced into the exact token\-embedding directions? This motivates our anchor choice: when token embeddings share dimensionality with the residual\-stream SAEdecoder, they define vocabulary\-indexed directions against which learned decoder columns can be compared\.

### 6\.2Sparse Codes and Mechanistic Analysis

Superposition motivates sparse feature dictionaries as a way to separate features that share model dimensions\(Elhageet al\.,[2022](https://arxiv.org/html/2606.27941#bib.bib8)\)\. Dictionary\-learning work decomposes language\-model activations into sparse features that are more interpretable than individual neurons\(Brickenet al\.,[2023](https://arxiv.org/html/2606.27941#bib.bib5); Hubenet al\.,[2024](https://arxiv.org/html/2606.27941#bib.bib18)\)\. Subsequent work scales and evaluates SAEsacross language\-model settings\(Gaoet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib11); Karvonenet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib29)\), and sparse feature circuits use these features for causal graph analysis and editing\(Markset al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib21)\)\. Existing interpretation workflows usually inspect top\-activating contexts or apply automated natural\-language explanations after training\(Billset al\.,[2023](https://arxiv.org/html/2606.27941#bib.bib4); Pauloet al\.,[2025](https://arxiv.org/html/2606.27941#bib.bib24)\)\. Mechanistic interpretability work more broadly studies how Transformer computations factor into internal mechanisms, including module\-level accounts such as feed\-forward key–value memory\(Gevaet al\.,[2021](https://arxiv.org/html/2606.27941#bib.bib12)\)\. Our contribution keeps the SAEreconstruction objective and adds vocabulary\-aligned anchoring, so learned features can receive candidate intrinsic token names\. The main claim is reconstruction\-preserving geometric token alignment\.

## 7Conclusion

We presented VASAE, a residual\-stream SAEwhose learned dictionary is vocabulary\-aligned to token embeddings\. The difference from a standard SAEis the anchor term: both models learn sparse codes to reconstruct the post\-residual stream, but VASAEalso encourages each feature to remain close to a nearest fixed token\-embedding direction, giving the feature an intrinsic token name\. This added constraint yields reconstruction metrics comparable to a standard SAEin the tested GPT\-2\-small and Llama\-3\.1\-8B settings, while producing strongly token\-aligned features in GPT\-2\-small and in shallow\- and middle\-layer Llama settings atλanchor=5×10−3\\lambda\_\{\\text\{anchor\}\}=5\\times 10^\{\-3\}\. Case studies show intrinsic token names in shallow and middle GPT\-2 layers, including location words, award\-related words, self\-introduction words, and adjective/adverb words\. Llama\-3\.1\-8B also shows the main boundary condition: token alignment depends on anchor coefficient and remains unstable in the final layer under the tested settings\. Overall, VASAEkeeps the sparse reconstruction role of a standard SAEbut adds geometrically supported nearest\-token names for features\.

#### Limitations\.

Our analysis is limited to residual\-stream SAEson two open\-weight language\-model families\. The main alignment evidence is highest for GPT\-2\-small\. In Llama\-3\.1\-8B, stable alignment appears under the larger tested anchor coefficient and in the tested shallow and middle representative layers, while the final layer remains unstable\. We anchor only to the input embedding matrix in this work; for untied models, testing anchors based on the unembedding matrix is left open\. Broader anchor\-strength sweeps, additional model families, additional model scales, and additional initialization baselines would further test the scope of the geometric alignment result\. The reported evidence supports geometric alignment, not causal interpretation of the token names\.

## References

- Y\. Belinkov and J\. R\. Glass \(2019\)Analysis Methods in Neural Language Processing: A Survey\.Trans\. Assoc\. Comput\. Linguistics7,pp\. 49–72\.External Links:[Document](https://dx.doi.org/10.1162/TACL%5FA%5F00254)Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p1.1)\.
- N\. Belrose, I\. Ostrovsky, L\. McKinney, Z\. Furman, L\. Smith, D\. Halawi, S\. Biderman, and J\. Steinhardt \(2025\)Eliciting Latent Predictions from Transformers with the Tuned Lens\.arXiv\.External Links:2303\.08112,[Document](https://dx.doi.org/10.48550/arXiv.2303.08112)Cited by:[§6\.1](https://arxiv.org/html/2606.27941#S6.SS1.p1.1)\.
- S\. Bills, N\. Cammarata, D\. Mossing, H\. Tillman, L\. Gao, G\. Goh, I\. Sutskever, J\. Leike, J\. Wu, and W\. Saunders \(2023\)Language models can explain neurons in language models\.Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p2.1),[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- T\. Bricken, A\. Templeton, J\. Batson, B\. Chen, A\. Jermyn, T\. Conerly, N\. Turner, C\. Anil, C\. Denison, A\. Askell, R\. Lasenby, Y\. Wu, S\. Kravec, N\. Schiefer, T\. Maxwell, N\. Joseph, Z\. Hatfield\-Dodds, A\. Tamkin, K\. Nguyen, B\. McLean, J\. E\. Burke, T\. Hume, S\. Carter, T\. Henighan, and C\. Olah \(2023\)Towards monosemanticity: Decomposing language models with dictionary learning\.Transformer Circuits Thread\.Cited by:[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei \(2020\)Language Models are Few\-Shot Learners\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, Virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p1.1)\.
- N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen, R\. Grosse, S\. McCandlish, J\. Kaplan, D\. Amodei, M\. Wattenberg, and C\. Olah \(2022\)Toy Models of Superposition\.arXiv\.External Links:2209\.10652,[Document](https://dx.doi.org/10.48550/arXiv.2209.10652)Cited by:[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- N\. Elhage, N\. Nanda, C\. Olsson, T\. Henighan, N\. Joseph, B\. Mann, A\. Askell, Y\. Bai, A\. Chen, T\. Conerly,et al\.\(2021\)A mathematical framework for transformer circuits\.Transformer Circuits Thread1\(1\),pp\. 12\.Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.27941#S2.SS1.p1.6)\.
- K\. Ethayarajh \(2019\)How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT\-2 Embeddings\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP\-IJCNLP 2019, Hong Kong, China, November 3\-7, 2019,K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),pp\. 55–65\.External Links:[Document](https://dx.doi.org/10.18653/V1/D19-1006)Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p3.1),[§6\.1](https://arxiv.org/html/2606.27941#S6.SS1.p2.1)\.
- J\. F\. Fiotto\-Kaufman, A\. R\. Loftus, E\. Todd, J\. Brinkmann, K\. Pal, D\. Troitskii, M\. Ripa, A\. Belfki, C\. Rager, C\. Juang, A\. Mueller, S\. Marks, A\. S\. Sharma, F\. Lucchetti, N\. Prakash, C\. E\. Brodley, A\. Guha, J\. Bell, B\. C\. Wallace, and D\. Bau \(2025\)NNsight and NDIF: Democratizing Access to Open\-Weight Foundation Model Internals\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,Cited by:[§4\.1](https://arxiv.org/html/2606.27941#S4.SS1.SSS0.Px1.p1.9)\.
- L\. Gao, T\. D\. la Tour, H\. Tillman, G\. Goh, R\. Troll, A\. Radford, I\. Sutskever, J\. Leike, and J\. Wu \(2025\)Scaling and evaluating sparse autoencoders\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p2.1),[§2\.3](https://arxiv.org/html/2606.27941#S2.SS3.p1.2),[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- M\. Geva, R\. Schuster, J\. Berant, and O\. Levy \(2021\)Transformer Feed\-Forward Layers Are Key\-Value Memories\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7\-11 November, 2021,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),pp\. 5484–5495\.External Links:[Document](https://dx.doi.org/10.18653/V1/2021.EMNLP-MAIN.446)Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p1.1),[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- A\. Ghandeharioun, A\. Caciularu, A\. Pearce, L\. Dixon, and M\. Geva \(2024\)Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models\.InForty\-First International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21\-27, 2024,R\. Salakhutdinov, Z\. Kolter, K\. A\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research,pp\. 15466–15490\.Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. Ma \(2024\)The Llama 3 Herd of Models\.arXiv\.External Links:2407\.21783,[Document](https://dx.doi.org/10.48550/arXiv.2407.21783)Cited by:[§2\.1](https://arxiv.org/html/2606.27941#S2.SS1.p5.1),[2nd item](https://arxiv.org/html/2606.27941#S4.I1.i2.p1.3)\.
- J\. Hewitt and P\. Liang \(2019\)Designing and Interpreting Probes with Control Tasks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP\-IJCNLP 2019, Hong Kong, China, November 3\-7, 2019,K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),pp\. 2733–2743\.External Links:[Document](https://dx.doi.org/10.18653/V1/D19-1275)Cited by:[§6\.1](https://arxiv.org/html/2606.27941#S6.SS1.p1.1)\.
- J\. Hoffmann, S\. Borgeaud, A\. Mensch, E\. Buchatskaya, T\. Cai, E\. Rutherford, D\. d\. L\. Casas, L\. A\. Hendricks, J\. Welbl, A\. Clark, T\. Hennigan, E\. Noland, K\. Millican, G\. van den Driessche, B\. Damoc, A\. Guy, S\. Osindero, K\. Simonyan, E\. Elsen, O\. Vinyals, J\. W\. Rae, and L\. Sifre \(2022\)An empirical analysis of compute\-optimal large language model training\.InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 \- December 9, 2022,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p1.1)\.
- R\. Huben, H\. Cunningham, L\. R\. Smith, A\. Ewart, and L\. Sharkey \(2024\)Sparse Autoencoders Find Highly Interpretable Features in Language Models\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p2.1),[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- H\. Inan, K\. Khosravi, and R\. Socher \(2017\)Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling\.In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24\-26, 2017, Conference Track Proceedings,Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p3.1),[§2\.1](https://arxiv.org/html/2606.27941#S2.SS1.p5.1),[§6\.1](https://arxiv.org/html/2606.27941#S6.SS1.p2.1)\.
- A\. Karvonen, C\. Rager, J\. Lin, C\. Tigges, J\. I\. Bloom, D\. Chanin, Y\. Lau, E\. Farrell, C\. McDougall, K\. Ayonrinde, D\. Till, M\. Wearden, A\. Conmy, S\. Marks, and N\. Nanda \(2025\)SAEBench: A comprehensive benchmark for sparse autoencoders in language model interpretability\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research\.External Links:[Link](https://proceedings.mlr.press/v267/karvonen25a.html)Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p2.1),[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- A\. Makhzani and B\. J\. Frey \(2014\)K\-Sparse Autoencoders\.In2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14\-16, 2014, Conference Track Proceedings,Y\. Bengio and Y\. LeCun \(Eds\.\),Cited by:[§2\.3](https://arxiv.org/html/2606.27941#S2.SS3.p1.2)\.
- S\. Marks, C\. Rager, E\. J\. Michaud, Y\. Belinkov, D\. Bau, and A\. Mueller \(2025\)Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p2.1),[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- S\. Merity, C\. Xiong, J\. Bradbury, and R\. Socher \(2017\)Pointer Sentinel Mixture Models\.In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24\-26, 2017, Conference Track Proceedings,Cited by:[§4\.1](https://arxiv.org/html/2606.27941#S4.SS1.SSS0.Px1.p1.9)\.
- J\. Mu and P\. Viswanath \(2018\)All\-but\-the\-Top: Simple and Effective Postprocessing for Word Representations\.In6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 \- May 3, 2018, Conference Track Proceedings,Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p3.1),[§6\.1](https://arxiv.org/html/2606.27941#S6.SS1.p2.1)\.
- G\. Paulo, A\. Mallen, C\. Juang, and N\. Belrose \(2025\)Automatically Interpreting Millions of Features in Large Language Models\.InForty\-Second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research\.Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p2.1),[§6\.2](https://arxiv.org/html/2606.27941#S6.SS2.p1.1)\.
- O\. Press and L\. Wolf \(2017\)Using the Output Embedding to Improve Language Models\.InProceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3\-7, 2017, Volume 2: Short Papers,M\. Lapata, P\. Blunsom, and A\. Koller \(Eds\.\),pp\. 157–163\.External Links:[Document](https://dx.doi.org/10.18653/V1/E17-2025)Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p3.1),[§2\.1](https://arxiv.org/html/2606.27941#S2.SS1.p5.1),[§6\.1](https://arxiv.org/html/2606.27941#S6.SS1.p2.1)\.
- A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, I\. Sutskever,et al\.\(2019\)Language models are unsupervised multitask learners\.OpenAI blog1\(8\),pp\. 9\.Cited by:[§2\.1](https://arxiv.org/html/2606.27941#S2.SS1.p5.1),[1st item](https://arxiv.org/html/2606.27941#S4.I1.i1.p1.3)\.
- I\. Tenney, D\. Das, and E\. Pavlick \(2019\)BERT Rediscovers the Classical NLP Pipeline\.InProceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28\- August 2, 2019, Volume 1: Long Papers,A\. Korhonen, D\. R\. Traum, and L\. Màrquez \(Eds\.\),pp\. 4593–4601\.External Links:[Document](https://dx.doi.org/10.18653/V1/P19-1452)Cited by:[§6\.1](https://arxiv.org/html/2606.27941#S6.SS1.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin \(2017\)Attention is All you Need\.InAdvances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4\-9, 2017, Long Beach, CA, USA,I\. Guyon, U\. von Luxburg, S\. Bengio, H\. M\. Wallach, R\. Fergus, S\. V\. N\. Vishwanathan, and R\. Garnett \(Eds\.\),pp\. 5998–6008\.Cited by:[§1](https://arxiv.org/html/2606.27941#S1.p1.1),[§1](https://arxiv.org/html/2606.27941#S1.p4.1),[§2\.1](https://arxiv.org/html/2606.27941#S2.SS1.p1.6)\.

## Appendix AEvaluation Metrics

This appendix collects the evaluation and diagnostic metrics used in the main text and sensitivity\-analysis tables\.

#### MSE\.

Given evaluation post\-residual streams\{𝐡i\}i=1M\\\{\\mathbf\{h\}\_\{i\}\\\}\_\{i=1\}^\{M\}and reconstructions\{𝐡~i\}i=1M\\\{\\tilde\{\\mathbf\{h\}\}\_\{i\}\\\}\_\{i=1\}^\{M\}, whereMMcounts valid token positions after flattening token positions across sequences, we define mean\-squared error \(MSE\)as

MSE=1M​dmodel​∑i=1M‖𝐡i−𝐡~i‖22\.\\mathrm\{MSE\}=\\frac\{1\}\{Md\_\{\\mathrm\{model\}\}\}\\sum\_\{i=1\}^\{M\}\\left\\lVert\\mathbf\{h\}\_\{i\}\-\\tilde\{\\mathbf\{h\}\}\_\{i\}\\right\\rVert\_\{2\}^\{2\}\.\(12\)

#### VE\.

Let

𝐡¯=1M​∑i=1M𝐡i\\bar\{\\mathbf\{h\}\}=\\frac\{1\}\{M\}\\sum\_\{i=1\}^\{M\}\\mathbf\{h\}\_\{i\}\(13\)denote the empirical mean target post\-residual stream\. The target variance is

Var=1M​dmodel​∑i=1M‖𝐡i−𝐡¯‖22\.\\mathrm\{Var\}=\\frac\{1\}\{Md\_\{\\mathrm\{model\}\}\}\\sum\_\{i=1\}^\{M\}\\left\\lVert\\mathbf\{h\}\_\{i\}\-\\bar\{\\mathbf\{h\}\}\\right\\rVert\_\{2\}^\{2\}\.\(14\)Using MSEand this variance, VEis

VE=1−∑i=1M‖𝐡i−𝐡~i‖22∑i=1M‖𝐡i−𝐡¯‖22=1−MSEVar\.\\mathrm\{VE\}=1\-\\frac\{\\sum\_\{i=1\}^\{M\}\\left\\lVert\\mathbf\{h\}\_\{i\}\-\\tilde\{\\mathbf\{h\}\}\_\{i\}\\right\\rVert\_\{2\}^\{2\}\}\{\\sum\_\{i=1\}^\{M\}\\left\\lVert\\mathbf\{h\}\_\{i\}\-\\bar\{\\mathbf\{h\}\}\\right\\rVert\_\{2\}^\{2\}\}=1\-\\frac\{\\mathrm\{MSE\}\}\{\\mathrm\{Var\}\}\.\(15\)Higher VEmeans more variance is recovered\.

#### CE loss\.

For the CEreconstruction check, we evaluate three residual\-stream substitutions at the evaluated site: the SAE\-style reconstruction𝐡~\\tilde\{\\mathbf\{h\}\}, the original𝐡\\mathbf\{h\}, and the zero vector𝟎\\mathbf\{0\}\. For substitutionr∈\{SAE,Id,0\}r\\in\\\{\\mathrm\{SAE\},\\mathrm\{Id\},0\\\}, let𝐩i\(r\)\\mathbf\{p\}\_\{i\}^\{\(r\)\}be the resulting next\-token distribution at evaluated positionii, and letyiy\_\{i\}be the target next token\. The average next\-token cross\-entropy is

CEr=−1M​∑i=1Mlog⁡𝐩i\(r\)​\(yi\)\.\\operatorname\{CE\}\_\{r\}=\-\\frac\{1\}\{M\}\\sum\_\{i=1\}^\{M\}\\log\\mathbf\{p\}\_\{i\}^\{\(r\)\}\(y\_\{i\}\)\.\(16\)Herer=SAEr=\\mathrm\{SAE\}uses𝐡~\\tilde\{\\mathbf\{h\}\},r=Idr=\\mathrm\{Id\}uses the original𝐡\\mathbf\{h\}, andr=0r=0uses𝟎\\mathbf\{0\}\. The CEloss reported in Table[1](https://arxiv.org/html/2606.27941#S4.T1)isCESAE\\operatorname\{CE\}\_\{\\mathrm\{SAE\}\}\.

#### CE rec\.

We compute CErecovery as

CERec=1−CESAE−CEIdCE0−CEId\.\\mathrm\{CERec\}=1\-\\frac\{\\operatorname\{CE\}\_\{\\mathrm\{SAE\}\}\-\\operatorname\{CE\}\_\{\\mathrm\{Id\}\}\}\{\\operatorname\{CE\}\_\{0\}\-\\operatorname\{CE\}\_\{\\mathrm\{Id\}\}\}\.\(17\)A value of0corresponds to the zero\-vector control\. A value of11corresponds to the original residual stream under this normalization\. Equivalently, CErecovery measures how much of the CEgap between the zero\-vector control and the original residual stream is recovered by the SAE\-style reconstruction: values near11preserve the original next\-token loss, values near0match the zero\-vector control, and negative values are worse than that control under this normalization\.

#### LogitLensaccuracy\.

As an auxiliary direct\-readout diagnostic, we compare the top\-1 token obtained from the original post\-residual stream with the top\-1 token obtained from its reconstruction after projection through the unembedding matrix:

LogitlensAcc\\displaystyle\\mathrm\{LogitlensAcc\}=1M​∑i=1M𝟏​\[ai=a~i\],\\displaystyle=\\frac\{1\}\{M\}\\sum\_\{i=1\}^\{M\}\\mathbf\{1\}\\\!\\left\[a\_\{i\}=\\tilde\{a\}\_\{i\}\\right\],\(18\)ai\\displaystyle a\_\{i\}=arg​maxv∈𝒱⁡𝐡i​𝐮v⊤,\\displaystyle=\\operatorname\*\{arg\\,max\}\_\{v\\in\\mathcal\{V\}\}\\mathbf\{h\}\_\{i\}\\mathbf\{u\}\_\{v\}^\{\\top\},a~i\\displaystyle\\tilde\{a\}\_\{i\}=arg​maxv∈𝒱⁡𝐡~i​𝐮v⊤\.\\displaystyle=\\operatorname\*\{arg\\,max\}\_\{v\\in\\mathcal\{V\}\}\\tilde\{\\mathbf\{h\}\}\_\{i\}\\mathbf\{u\}\_\{v\}^\{\\top\}\.Hereaia\_\{i\}anda~i\\tilde\{a\}\_\{i\}are the original and reconstructed top\-1 tokens under the direct readout, and𝐮v\\mathbf\{u\}\_\{v\}denotes the row of the unembedding matrix𝐖U\\mathbf\{W\}\_\{U\}for tokenvv\. This metric checks agreement under a direct layerwise readout\. It is separate from CErecovery, which substitutes the reconstruction into the model and evaluates the final next\-token loss after the remaining computation\.

## Appendix BAnchor\-Strength Sensitivity Analysis

Table[2](https://arxiv.org/html/2606.27941#A2.T2)reports a representative sweep overλanchor\\lambda\_\{\\text\{anchor\}\}while keeping the same top\-kkSAEarchitecture\. The settingλanchor=0\\lambda\_\{\\text\{anchor\}\}=0disables the anchor term and serves as the component ablation\.

Table 2:Representative anchor\-strength sensitivity analysis supporting the larger\-coefficient Llama alignment setting\. Arrows indicate the preferred direction for each metric\. Best values within each model\-layer block and metric column are bolded\.Changingλanchor\\lambda\_\{\\text\{anchor\}\}from0to5×10−35\\times 10^\{\-3\}changes VEand CErecovery only slightly on these representative GPT\-2\-small and Llama\-3\.1\-8B layers\. This sweep indicates empirical robustness over the tested range;λanchor\\lambda\_\{\\text\{anchor\}\}remains a tunable hyperparameter in other architectures or layers\.

## Appendix CLayerwise Diagnostics

The aggregate benchmark in Table[1](https://arxiv.org/html/2606.27941#S4.T1)reports mean and standard deviation across layers\. The layerwise diagnostics in Figure[5](https://arxiv.org/html/2606.27941#A3.F5)show layer\-specific failures that are averaged out in aggregate means\. We group the diagnostics by what they measure: VEmeasures residual\-space reconstruction, CErecovery measures downstream loss after running the remaining model computation, and LogitLensaccuracy measures agreement under direct unembedding\.

![Refer to caption](https://arxiv.org/html/2606.27941v1/x5.png)

![Refer to caption](https://arxiv.org/html/2606.27941v1/x6.png)

![Refer to caption](https://arxiv.org/html/2606.27941v1/)

![Refer to caption](https://arxiv.org/html/2606.27941v1/x8.png)

![Refer to caption](https://arxiv.org/html/2606.27941v1/x9.png)

![Refer to caption](https://arxiv.org/html/2606.27941v1/x10.png)

Figure 5:Layerwise benchmark diagnostics for GPT\-2\-small and Llama\-3\.1\-8B\. Top: VE\. Middle: CErecovery\. Bottom: LogitLenstop\-1 agreement under direct unembedding\.VASAEand the standard SAEnearly overlap in VEand CErecovery across layers in both model families, matching the aggregate benchmark\. This supports reconstruction preservation in the tested cases; trade\-offs in other settings remain empirical\. The hard\-tied decoder baseline is much lower on these two metrics, especially in GPT\-2\-small late layers and across Llama\-3\.1\-8B\. The LogitLenscurves show a different direct\-readout pattern: GPT\-2\-small hard\-tying keeps higher top\-1 agreement in several layers, while its CErecovery remains lower\. This indicates that direct unembedding agreement and downstream functional recovery capture different aspects of reconstruction\. Llama\-3\.1\-8B has lower LogitLensagreement overall than GPT\-2\-small under this direct readout\.

## Appendix DGeometric Alignment Metrics

For each featureii, we use the nearest\-token alignment scoresis\_\{i\}and nearest vocabulary itemvi⋆v\_\{i\}^\{\\star\}defined in Subsection[3\.2](https://arxiv.org/html/2606.27941#S3.SS2)\. Let𝒜τ=\{i∈\{1,…,dsparse\}:si≥τ\}\\mathcal\{A\}\_\{\\tau\}=\\\{i\\in\\\{1,\\ldots,d\_\{\\mathrm\{sparse\}\}\\\}:s\_\{i\}\\geq\\tau\\\}denote the strongly aligned feature set for the diagnostic cutoffτ\\tauused in Subsection[4\.3](https://arxiv.org/html/2606.27941#S4.SS3)\. Geometric alignment rate is the proportion of features that are aligned with tokens:

GeomAlignRateτ=\|𝒜τ\|dsparse\.\\mathrm\{GeomAlignRate\}\_\{\\tau\}=\\frac\{\|\\mathcal\{A\}\_\{\\tau\}\|\}\{d\_\{\\mathrm\{sparse\}\}\}\.\(19\)Vocabulary coverage is the proportion of the vocabulary that has at least one feature aligned to it:

Coverageτ=\|\{vi⋆:i∈𝒜τ\}\|\|𝒱\|\.\\mathrm\{Coverage\}\_\{\\tau\}=\\frac\{\|\\\{v\_\{i\}^\{\\star\}:i\\in\\mathcal\{A\}\_\{\\tau\}\\\}\|\}\{\|\\mathcal\{V\}\|\}\.\(20\)The geometric alignment denominator is the full\-learned feature set\.

## Appendix EAdditional Intrinsic\-Name Case Studies

These examples extend the main GPT\-2\-small location case study by applying the same sentence\-centered feature\-display rule to other prompt types\. Figure[6](https://arxiv.org/html/2606.27941#A5.F6)uses an adjective/adverb sentence and reports intrinsic token names related to feasibility, degree, and evaluation, includingperfect,possible,entirely,suited,different,enough,completely,logically,elegant,frankly, andflexible\. Figure[7](https://arxiv.org/html/2606.27941#A5.F7)covers a named\-entity award sentence and a self\-introduction sentence\. In the Nicole Kidman sentence, reported names include award and social\-context tokens such asaccept,acceptance,awarded,winner,praise,friend,friends, andmembers\. In the self\-introduction sentence, reported names include personal\-reference and fandom\-related tokens such asname,am,’m,fan, andfans\. These examples show the same sentence\-centered feature\-display pattern beyond the location prompt, while remaining qualitative case studies\.

![Refer to caption](https://arxiv.org/html/2606.27941v1/x11.png)Figure 6:Additional GPT\-2\-small adjective/adverb example\. Each cell shows the intrinsic token name of the feature chosen by the same sentence\-centered sparse\-code rule as Figure[3](https://arxiv.org/html/2606.27941#S4.F3)\. Color indicates the raw sparse\-code activation of that feature\. Displayed names often track feasibility, degree, and evaluation words in the prompt\.![Refer to caption](https://arxiv.org/html/2606.27941v1/x12.png)

![Refer to caption](https://arxiv.org/html/2606.27941v1/x13.png)

Figure 7:Additional qualitative GPT\-2\-small intrinsic\-name examples\. Each cell shows the intrinsic token name of the feature chosen by the same sentence\-centered sparse\-code rule as Figure[3](https://arxiv.org/html/2606.27941#S4.F3)\. Color indicates the raw sparse\-code activation of that feature\. The examples show displayed names around award\-related and self\-introduction contexts\.
## Appendix FImplementation Details for Similarity Score

Computing the similarity scores\{si\}i=1dsparse\\\{s\_\{i\}\\\}\_\{i=1\}^\{d\_\{\\mathrm\{sparse\}\}\}defined in Equation[7](https://arxiv.org/html/2606.27941#S3.E7)independently for each feature requires a nested loop over the vocabulary size\|𝒱\|\|\\mathcal\{V\}\|and sparse code dimensiondsparsed\_\{\\mathrm\{sparse\}\}\. To leverage the highly parallel architecture of modern GPUs, we reframe this computation into dense matrix multiplications\.

The cosine similarity between a feature𝐝i\\mathbf\{d\}\_\{i\}and a token embedding𝐰v\\mathbf\{w\}\_\{v\}is equivalent to the dot product of theirL2L\_\{2\}\-normalized vectors:

cos⁡\(𝐝i,𝐰v\)=\(𝐝i‖𝐝i‖2\)​\(𝐰v‖𝐰v‖2\)⊤\.\\cos\(\\mathbf\{d\}\_\{i\},\\mathbf\{w\}\_\{v\}\)=\\left\(\\frac\{\\mathbf\{d\}\_\{i\}\}\{\\\|\\mathbf\{d\}\_\{i\}\\\|\_\{2\}\}\\right\)\\left\(\\frac\{\\mathbf\{w\}\_\{v\}\}\{\\\|\\mathbf\{w\}\_\{v\}\\\|\_\{2\}\}\\right\)^\{\\top\}\.\(21\)
We perform row\-wiseL2L\_\{2\}\-normalization on𝐖𝒟⊤\\mathbf\{W\}\_\{\\mathcal\{D\}\}^\{\\top\}and𝐖E\\mathbf\{W\}\_\{E\}to obtain the normalized feature\-by\-model matrix𝐃^∈ℝdsparse×dmodel\\hat\{\\mathbf\{D\}\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{sparse\}\}\\times d\_\{\\mathrm\{model\}\}\}and normalized embedding matrix𝐖E^∈ℝ\|𝒱\|×dmodel\\hat\{\\mathbf\{W\}\_\{E\}\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\_\{\\mathrm\{model\}\}\}\. The full pairwise similarity matrix𝐒∈ℝdsparse×\|𝒱\|\\mathbf\{S\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{sparse\}\}\\times\|\\mathcal\{V\}\|\}can then be represented via a single matrix multiplication:

𝐒=𝐃^​𝐖E^⊤,\\mathbf\{S\}=\\hat\{\\mathbf\{D\}\}\\hat\{\\mathbf\{W\}\_\{E\}\}^\{\\top\},\(22\)where the element𝐒i,v=cos⁡\(𝐝i,𝐰v\)\\mathbf\{S\}\_\{i,v\}=\\cos\(\\mathbf\{d\}\_\{i\},\\mathbf\{w\}\_\{v\}\)\. The final similarity scoresis\_\{i\}for each feature is extracted by taking the maximum value along the rows of𝐒\\mathbf\{S\}:

si=max1≤v≤\|𝒱\|⁡𝐒i,v\.s\_\{i\}=\\max\_\{1\\leq v\\leq\|\\mathcal\{V\}\|\}\\mathbf\{S\}\_\{i,v\}\.\(23\)
While mathematically straightforward, materializing the dense matrix𝐒\\mathbf\{S\}requires𝒪​\(dsparse×\|𝒱\|\)\\mathcal\{O\}\(d\_\{\\mathrm\{sparse\}\}\\times\|\\mathcal\{V\}\|\)memory\. For typical SAEdictionary sizes and vocabulary size, this rapidly leads to Out\-Of\-Memory \(OOM\) errors\.

To resolve this, we implement a chunked computation strategy\. We partition𝐃^\\hat\{\\mathbf\{D\}\}along the row dimension into contiguous blocks of sizeBB\. For each block𝐃^k∈ℝB×dmodel\\hat\{\\mathbf\{D\}\}\_\{k\}\\in\\mathbb\{R\}^\{B\\times d\_\{\\mathrm\{model\}\}\}, we compute the local similarity sub\-matrix𝐒k=𝐃^k​𝐖E^⊤∈ℝB×\|𝒱\|\\mathbf\{S\}\_\{k\}=\\hat\{\\mathbf\{D\}\}\_\{k\}\\hat\{\\mathbf\{W\}\_\{E\}\}^\{\\top\}\\in\\mathbb\{R\}^\{B\\times\|\\mathcal\{V\}\|\}, immediately apply the row\-wise maximum reduction to obtain the local scores𝐬k∈ℝB\\mathbf\{s\}\_\{k\}\\in\\mathbb\{R\}^\{B\}, and discard𝐒k\\mathbf\{S\}\_\{k\}\. This strategy strictly bounds the peak memory footprint to𝒪​\(B×\|𝒱\|\)\\mathcal\{O\}\(B\\times\|\\mathcal\{V\}\|\)\.

The PyTorch implementation, mapping directly to our defined matrices𝐖𝒟\\mathbf\{W\}\_\{\\mathcal\{D\}\}and𝐖E\\mathbf\{W\}\_\{E\}, is provided in Listing[1](https://arxiv.org/html/2606.27941#LST1)\.

defcompute\_sae\_similarity\(

W\_dec:torch\.Tensor,

W\_E:torch\.Tensor,

chunk\_size:int=2048,

\)\-\>torch\.Tensor:

"""Computenearest\-tokencosinescores\.

Args:

W\_dec:\(d\_model,d\_sparse\)decoderweights\.

W\_E:\(d\_vocab,d\_model\)tokenembeddings\.

chunk\_size:Numberoffeaturesperblock\.

Returns:

s:\(d\_sparse,\)maxscoreperfeature\.

"""

D\_hat=F\.normalize\(W\_dec\.T,p=2,dim=1\)

E\_hat=F\.normalize\(W\_E\.to\(D\_hat\.dtype\),p=2,dim=1\)

s\_parts=\[\]

d\_sparse=D\_hat\.size\(0\)

foriinrange\(0,d\_sparse,chunk\_size\):

S\_k=D\_hat\[i:i\+chunk\_size\]@E\_hat\.T

s\_k,\_=torch\.max\(S\_k,dim=1\)

s\_parts\.append\(s\_k\)

returntorch\.cat\(s\_parts\)

Listing 1:Memory\-efficient PyTorch implementation for computing Equation[7](https://arxiv.org/html/2606.27941#S3.E7)\.