What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2

# What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2
Source: [https://arxiv.org/html/2610.37157](https://arxiv.org/html/2610.37157)
Nabeelah FaumiAffiliation:Informatics Institute of TechnologyAffiliation:57, Ramakrishna Road, Colombo 06,Affiliation:Sri LankaEmail:[nabeelahfaumi@gmail\.com](mailto:)Deshan SumanathilakaAffiliation:Informatics Institute of TechnologyAffiliation:57, Ramakrishna Road, Colombo 06,Affiliation:Sri LankaEmail:[deshan\.s@iit\.ac\.lk](mailto:)

###### Abstract

Knowledge distillation compresses large language models into compact students that preserve aggregate metrics, yet the mechanistic consequences of depth\-reduction distillation remain largely uncharacterized\. This work applies linear CKA, logit lens analysis, attention pattern decomposition, sparse autoencoder feature statistics, and activation patching to compare GPT\-2 and its six\-layer student DistilGPT\-2 across two canonical circuits: the Indirect Object Identification \(IOI\) circuit and the Greater\-Than \(GT\) circuit\. The results reveal a compression\-collapse dichotomy: the IOI circuit suffers complete mechanistic collapse \(DistilGPT\-2: 35\.2% accuracy vs\. 94\.8%, zero name\-mover preservation\), while the GT circuit survives with 87\.7% task performance recovery through active remapping of the teacher’s layer\-9 comparison head into student layer 4 \(mean Pearsonr=0\.945r=0\.945\)\. These findings demonstrate that circuit compressibility governs mechanistic survival under distillation, and that aggregate metrics are insufficient diagnostics for circuit integrity\.

## 1Introduction

Large language models \(LLMs\) exhibit remarkable capabilities, but their sheer scale poses significant computational challenges for deployment\. Knowledge distillation trains a compact student on soft targets from a larger teacher, enabling compression while mimicking the teacher’s generalization\([Hinton et al\., 2015](https://arxiv.org/html/2610.37157#bib.bib10)\)\. While distillation often successfully preserves aggregate metrics like validation perplexity, it remains an open question how severe architectural truncation affects the precise, underlying algorithms the model has learned\. Prior work establishes that compression can degrade circuit integrity and alter feature representations\([Lindner et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib15);[Bereska and Gavves, 2024](https://arxiv.org/html/2610.37157#bib.bib1)\), but the specific mechanistic consequences of distillation\-induced depth reduction remain largely uncharacterized\. Mechanistic interpretability provides a framework for addressing this blind spot by reverse\-engineering neural networks into human\-understandable computational subgraphs, known as circuits\([Elhage et al\., 2021](https://arxiv.org/html/2610.37157#bib.bib17);[Olah et al\., 2020](https://arxiv.org/html/2610.37157#bib.bib3)\)\.

This work investigates the mechanistic impact of depth\-reduction distillation\. It focuses on DistilGPT\-2\([Sanh et al\., 2019](https://arxiv.org/html/2610.37157#bib.bib23)\), a 6\-layer student distilled from the 12\-layer GPT\-2 teacher\([Radford et al\., 2019](https://arxiv.org/html/2610.37157#bib.bib24)\)\. This drastic depth reduction poses a critical structural challenge: many well\-documented higher\-order reasoning behaviors in GPT\-2 depend crucially on late\-layer interactions\. For example, both the Indirect Object Identification \(IOI\) circuit\([Wang et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib18)\)and the Greater\-Than \(GT\) circuit\([Hanna et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib19)\)rely heavily on highly specialized attention heads located in layers 9\-11\. Because the student model is only 6 layers deep, these components have no direct structural counterpart\. This raises two central research questions: do the internal representations of DistilGPT\-2 remain similar to those of GPT\-2 after depth halving \(RQ1\), and does distillation preserve the computational circuits responsible for higher\-order reasoning, and if so, how does the student compensate for the absence of essential upper\-layer components \(RQ2\)\.

To answer this, a suite of representational and causal probes, including representational similarity analysis, direct logit attribution\([Elhage et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib8)\), and activation patching\([Meng et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib5)\), is applied to track circuit preservation across the distillation gap\. A natural question is whether circuit preservation matters at all if task performance is maintained\. We argue it does, for two reasons\. First, the IOI and GT circuits are among the most causally validated subgraphs in any language model, established through convergent evidence from activation patching, path patching, and direct logit attribution across independent studies\([Wang et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib18);[Hanna et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib19)\); they are not arbitrary correlational subgraphs but confirmed computational mechanisms\. Second, and more critically, the IOI result we report is not a performance degradation but a sign reversal: DistilGPT\-2 achieves a negative mean logit difference \(−0\.467\-0\.467\), meaning the model is actively misled rather than merely uncertain, failure mode entirely invisible to aggregate metrics, the core motivation for circuit\-level analysis\.

We find a striking dichotomy in how different computational mechanisms survive depth halving\. One circuit collapses entirely, the other is actively remapped into the student’s shallower stack, detailed in §[5](https://arxiv.org/html/2610.37157#S5)and §[6](https://arxiv.org/html/2610.37157#S6)\. By contrasting a circuit that collapses with one that compresses, this sheds light on the inductive biases of the distillation process\.

The main contributions are as follows:

- •Provide a comprehensive, circuit\-level analysis comparing a depth\-distilled model to its teacher, quantifying the mechanistic impact of structural truncation\.
- •Demonstrate that complex, multi\-stage computational pipelines \(such as the IOI circuit\) fail to survive standard distillation when critical late\-layer components are excised\.
- •Discover that the student model can successfully compress and remap specific cognitive algorithms \(such as the GT circuit\), migrating late\-layer mechanisms into its available upper layers to preserve task functionality\.

In the following sections, we review related work, outline our experimental setup, present our results, and conclude with a discussion and conclusion\.

## 2Related Work

Mechanistic Interpretability of GPT\-2\.Circuit analysis decomposes neural networks into functionally distinct sub\-networks using causal interventions\([Olah et al\., 2020](https://arxiv.org/html/2610.37157#bib.bib3);[Cammarata et al\., 2020](https://arxiv.org/html/2610.37157#bib.bib2)\)\. In GPT\-2, two circuits are directly relevant to this work: the Indirect Object Identification \(IOI\) circuit\([Wang et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib18)\), a multi\-stage pipeline of name\-mover, S\-inhibition, and induction heads spanning layers 0\-11; and the Greater\-Than \(GT\) circuit\([Hanna et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib19)\), in which year\-encoding heads feed comparison heads in layers 9\-11\. Activation patching\([Meng et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib5)\), and path patching\([Goldowsky\-Dill et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib6)\)establish causal responsibility at the head and pathway level, respectively, while Direct Logit Attribution\([Elhage et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib8)\)provides a complementary non\-interventional decomposition\. Sparse autoencoders\([Bricken et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib9);[Cunningham et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib28)\)extend circuit analysis to the feature level by decomposing polysemantic residual\-stream activations into monosemantic feature dictionaries\.

Knowledge Distillation\.Knowledge distillation trains a compact student on soft targets from a larger teacher, enabling it to mimic the teacher’s generalization while maintaining a smaller architecture\([Hinton et al\., 2015](https://arxiv.org/html/2610.37157#bib.bib10)\)\. DistilGPT\-2\([Sanh et al\., 2019](https://arxiv.org/html/2610.37157#bib.bib23)\)reduces GPT\-2’s depth to six layers by initializing from every other teacher layer and combining a language\-modeling and distillation objective, thereby preserving embedding dimensionality throughout\.

Compression and Interpretability\.Prior work establishes that compression can degrade circuit integrity and alter feature representations\([Lindner et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib15);[Bereska and Gavves, 2024](https://arxiv.org/html/2610.37157#bib.bib1)\), but focuses primarily on pruning and quantization\([Hatefi et al\., 2025](https://arxiv.org/html/2610.37157#bib.bib11);[Chhabra and Khalili, 2025](https://arxiv.org/html/2610.37157#bib.bib12)\)\. SAE\-based analysis reveals representational changes under compression can be invisible to standard metrics\([Malik et al\., 2025](https://arxiv.org/html/2610.37157#bib.bib13);[Ayonrinde et al\., 2024](https://arxiv.org/html/2610.37157#bib.bib14)\)\. Most closely related,[Wu et al\. \(2024\)](https://arxiv.org/html/2610.37157#bib.bib29);[Haskins and Adams \(2025\)](https://arxiv.org/html/2610.37157#bib.bib30);[Baek and Tegmark \(2025\)](https://arxiv.org/html/2610.37157#bib.bib31)show distillation induces simplicity bias, component reorganisation, and unique feature directions across model families\.

## 3Background and Models

Mechanistic interpretability\([Elhage et al\., 2021](https://arxiv.org/html/2610.37157#bib.bib17);[Olah et al\., 2020](https://arxiv.org/html/2610.37157#bib.bib3)\)reverse\-engineers neural network algorithms at the level of attention heads and residual\-stream components\. The*residual stream*\([Elhage et al\., 2021](https://arxiv.org/html/2610.37157#bib.bib17)\)treats activations as a running sum of additive contributions from each head and MLP, enabling direct logit attribution \(DLA\)\. A*circuit*\([Olah et al\., 2020](https://arxiv.org/html/2610.37157#bib.bib3)\)is a minimal subgraph causally sufficient for a given behaviour\. The IOI and GT circuits both depend critically on upper\-layer interactions structurally absent in a six\-layer student\.

Representational similarity\.Debiased linear CKA\([Kornblith et al\., 2019](https://arxiv.org/html/2610.37157#bib.bib22)\)compares activations across models\. LetHX=HSIC^​\(X​X⊤,X​X⊤\)H\_\{X\}=\\widehat\{\\mathrm\{HSIC\}\}\(XX^\{\\top\},XX^\{\\top\}\),HY=HSIC^​\(Y​Y⊤,Y​Y⊤\)H\_\{Y\}=\\widehat\{\\mathrm\{HSIC\}\}\(YY^\{\\top\},YY^\{\\top\}\), andHX​Y=HSIC^​\(X​X⊤,Y​Y⊤\)H\_\{XY\}=\\widehat\{\\mathrm\{HSIC\}\}\(XX^\{\\top\},YY^\{\\top\}\)\. Then,

CKA⁡\(X,Y\)=HX​YHX​HY\.\\mathrm\{CKA\}\(X,Y\)=\\frac\{H\_\{XY\}\}\{\\sqrt\{H\_\{X\}H\_\{Y\}\}\}\.\(1\)Here,HSIC^\\widehat\{\\mathrm\{HSIC\}\}denotes the unbiased Hilbert–Schmidt Independence Criterion estimator\([Song et al\., 2012](https://arxiv.org/html/2610.37157#bib.bib27)\), which is invariant to orthogonal transformations and isotropic scaling\.

Sparse autoencoders\([Cunningham et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib28);[Bricken et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib9)\)decompose residual\-stream activations into sparse, interpretable feature directions\. Pre\-trained GPT\-2 SAEs\([Conmy et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib4)\)\(gpt2\-small\-res\-jb\) are applied to both models at matched layers as a fixed reference basis, testing whether DistilGPT\-2’s representations lie in the same sparse feature subspace\.

Models\.GPT\-2\([Radford et al\., 2019](https://arxiv.org/html/2610.37157#bib.bib24)\)is a 12\-layer decoder\-only transformer \(dmodel=768d\_\{\\text\{model\}\}\{=\}768, 12 heads,∼117\{\\sim\}117M parameters\) and serves as the*teacher*\. DistilGPT\-2\([Sanh et al\., 2019](https://arxiv.org/html/2610.37157#bib.bib23)\), the*student*, retains the same embedding dimensionality while halving the depth to 6 layers \(∼82\{\\sim\}82M parameters\), initialised from every other teacher layer and fine\-tuned with a combined language\-modelling and soft\-label distillation objective\. IOI circuit heads in layers 6–11 and GT circuit heads in layers 9–11 have no direct counterpart in the student\.

## 4Methods

To answer RQ1, debiased linear CKA \(§[4\.1](https://arxiv.org/html/2610.37157#S4.SS1)\), logit lens analysis \(§[4\.2](https://arxiv.org/html/2610.37157#S4.SS2)\), and attention pattern analysis \(§[4\.3](https://arxiv.org/html/2610.37157#S4.SS3)\) are applied\. To answer RQ2, circuit\-level analysis of the IOI and GT tasks is conducted via activation patching and direct logit attribution \(§[4\.5](https://arxiv.org/html/2610.37157#S4.SS5)\-[4\.6](https://arxiv.org/html/2610.37157#S4.SS6)\)\. All experiments usen=1,000n\{=\}1\{,\}000samples from the TinyStories validation set\([Li and Eldan, 2023](https://arxiv.org/html/2610.37157#bib.bib16)\), filtered to\[64,256\]\[64,256\]tokens, processed via TransformerLens\([Nanda and Bloom, 2022](https://arxiv.org/html/2610.37157#bib.bib25)\)on a single NVIDIA L4 GPU\.

### 4\.1Representational Similarity via CKA

Debiased linear CKA\([Kornblith et al\., 2019](https://arxiv.org/html/2610.37157#bib.bib22)\)is computed between residual\-stream activations at matched layers\. Activations are mean\-pooled over non\-padding positions:

𝐡ℓ\(i\)=∑t=1Ti𝐦t⊙𝐡ℓ,t\(i\)∑t=1Ti𝐦t,\\mathbf\{h\}\_\{\\ell\}^\{\(i\)\}=\\frac\{\\sum\_\{t=1\}^\{T\_\{i\}\}\\mathbf\{m\}\_\{t\}\\odot\\mathbf\{h\}\_\{\\ell,t\}^\{\(i\)\}\}\{\\sum\_\{t=1\}^\{T\_\{i\}\}\\mathbf\{m\}\_\{t\}\},\(2\)yieldingXℓ,Yℓ∈ℝn×768X\_\{\\ell\},Y\_\{\\ell\}\\in\\mathbb\{R\}^\{n\\times 768\}infloat64\. CKA is computed via the unbiased HSIC estimator\([Song et al\., 2012](https://arxiv.org/html/2610.37157#bib.bib27)\)with variance terms clamped to10−1610^\{\-16\}\.

Layer correspondence\.Two schemes are used:same\-indexcompares layerℓ\\ellof each model forℓ∈\{0,3,5\}\\ell\\in\\\{0,3,5\\\};proportionalmaps DistilGPT\-2 layerddto GPT\-2 layerg=⌊d⋅11/5⌉g=\\lfloor d\\cdot 11/5\\rceil, giving pairs \(G0/D0\), \(G2/D1\), \(G4/D2\), \(G7/D3\), \(G9/D4\), \(G11/D5\)\.

### 4\.2Logit Lens Analysis

The tuned logit lens\([nostalgebraist, 2020](https://arxiv.org/html/2610.37157#bib.bib21);[Belrose et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib26)\)projects each intermediate hidden state through the final layer norm and unembedding matrix:

p^ℓ,t\(i\)=softmax⁡\(WU​LNf​\(𝐡ℓ,t\(i\)\)\)\.\\hat\{p\}\_\{\\ell,t\}^\{\(i\)\}=\\mathrm\{softmax\}\\\!\\left\(W\_\{U\}\\,\\mathrm\{LN\}\_\{f\}\\\!\\left\(\\mathbf\{h\}\_\{\\ell,t\}^\{\(i\)\}\\right\)\\right\)\.\(3\)Next\-token prediction accuracyAccℓ\\mathrm\{Acc\}\_\{\\ell\}and relative accuracyrℓ=Accℓdistil/AccℓGPT\-2r\_\{\\ell\}=\\mathrm\{Acc\}\_\{\\ell\}^\{\\text\{distil\}\}/\\mathrm\{Acc\}\_\{\\ell\}^\{\\text\{GPT\-2\}\}are computed at each layer under both mapping schemes\.

### 4\.3Attention Pattern Analysis

Per\-head attention patterns are extracted via TransformerLens hooks\. Three metric families are computed: Shannon entropyHl,hH\_\{l,h\}and top\-kkattention mass to quantify head collapse; mean BOS, previous\-token, and EOS weights to characterise structural head types\([Elhage et al\., 2021](https://arxiv.org/html/2610.37157#bib.bib17)\); and per\-sample cosine similarity between flattenedT×TT\{\\times\}Tattention matrices at matched layer pairs, alongside per\-head entropy changeΔ​Hl,h=Hd,hdistil−Hl,hGPT2\\Delta H\_\{l,h\}=H\_\{d,h\}^\{\\mathrm\{distil\}\}\-H\_\{l,h\}^\{\\mathrm\{GPT2\}\}\.

### 4\.4SAE\-Based Feature Analysis

Pre\-trained GPT\-2 SAEs \(gpt2\-small\-res\-jb,[Conmy et al\. 2023](https://arxiv.org/html/2610.37157#bib.bib4);[Bloom et al\. 2024](https://arxiv.org/html/2610.37157#bib.bib20)\) are applied to both models’ activations at matched layers as a fixed reference basis, testing whether DistilGPT\-2’s representations lie in the same sparse feature subspace as GPT\-2\. Four metrics are computed:activation raterfr\_\{f\};L0 sparsityL¯0\\bar\{L\}\_\{0\}and changeΔ​L0\\Delta L\_\{0\};feature survival\(Pearson correlation per feature, excludingσ​10−8\\sigma 10^\{\-8\}\); andmagnitude driftδf=z¯fdistil/\(z¯fGPT2\+ε\)\\delta\_\{f\}=\\bar\{z\}\_\{f\}^\{\\mathrm\{distil\}\}/\(\\bar\{z\}\_\{f\}^\{\\mathrm\{GPT2\}\}\+\\varepsilon\)\.

### 4\.5Indirect Object Identification Circuit Analysis

Table 1:IOI circuit heads by functional class and availability in DistilGPT\-2 \(layers 0\-5 only\)\.The IOI task\([Wang et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib18)\)uses prompts of the form“\[Name1\] and \[Name2\] went to the \[place\]\. \[Name\] gave a \[object\] to”with 1,000 clean and 1,000 corrupted \(ABC\) prompts from 15 syntactic templates; the primary metric is logit differenceΔ⁡\(x\)=ℓIO​\(x,t∗\)−ℓS​\(x,t∗\)\\Delta\(x\)=\\ell\_\{\\mathrm\{IO\}\}\(x,t^\{\*\}\)\-\\ell\_\{\\mathrm\{S\}\}\(x,t^\{\*\}\)\. Of the 24 IOI circuit heads, 19 \(all in layers 6–11\) are structurally absent in DistilGPT\-2; analysis is restricted to the 7 heads in layers 0–5 and 3 inter\-head paths whose endpoints lie within layers 0–5\. Each head’s causal contribution is measured by replacing its corrupted\-run output with the clean\-run activation\([Meng et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib5);[Wang et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib18)\); inter\-head path patching\([Goldowsky\-Dill et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib6)\)injects the clean\-corrupt difference at the receiver’s residual stream\. Significance is assessed with pairedtt\-tests and Wilcoxon tests \(Bonferroni correction\); effect sizes reported as Cohen’sdd\. DLA contributionsδIOIl,h=\(𝐖U​LNf​\(𝐯l,h\)\)IO−\(𝐖U​LNf​\(𝐯l,h\)\)S\\delta^\{l,h\}\_\{\\mathrm\{IOI\}\}=\(\\mathbf\{W\}\_\{U\}\\mathrm\{LN\}\_\{f\}\(\\mathbf\{v\}^\{l,h\}\)\)\_\{\\mathrm\{IO\}\}\-\(\\mathbf\{W\}\_\{U\}\\mathrm\{LN\}\_\{f\}\(\\mathbf\{v\}^\{l,h\}\)\)\_\{\\mathrm\{S\}\}are computed via a singleeinsumover cached head outputs\. To test whether any DistilGPT\-2 head assumes the name\-mover role, CKA and Pearson correlation between each GPT\-2 name\-mover head’s attention patterns and the top\-20 DistilGPT\-2 components \(by DLA\) are computed on a discovery set \(500 prompts\) and confirmed on a held\-out evaluation set \(500 prompts\); the name\-mover scoreNM⁡\(l,h\)=𝔼⁡\[∑i∈IOαt∗,il,h−∑j∈Sαt∗,jl,h\]\\mathrm\{NM\}\(l,h\)=\\mathbb\{E\}\[\\sum\_\{i\\in\\mathrm\{IO\}\}\\alpha^\{l,h\}\_\{t^\{\*\},i\}\-\\sum\_\{j\\in\\mathrm\{S\}\}\\alpha^\{l,h\}\_\{t^\{\*\},j\}\]quantifies IO\-selective attention\.

### 4\.6Greater\-Than Circuit Analysis

The GT task\([Hanna et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib19)\)uses prompts of the form“The \[noun\] lasted from the year \[XX\]\[YY\] to the year \[XX\]”with 1,000 clean and 1,000 corrupted prompts; the primary metric is GT probability massGTProb⁡\(x,t∗\)=∑y​y\>Y​Yp⁡\(yy∣x,t∗\)\\mathrm\{GTProb\}\(x,t^\{\*\}\)=\\sum\_\{yy\>YY\}p\(\\texttt\{yy\}\\mid x,t^\{\*\}\)\. Of the six GT circuit heads, only the two year\-encoding heads\{\(0,1\),\(1,5\)\}\\\{\(0,1\),\(1,5\)\\\}exist in DistilGPT\-2; all comparison and output\-computation heads in layers 9–11 are absent\. The same patching procedure as IOI is applied, with scores normalised by the noise ceilings^l,h=sl,h/\(GTProbclean−GTProbcorrupt\)\\hat\{s\}\_\{l,h\}=s\_\{l,h\}/\(\\mathrm\{GTProb\}\_\{\\mathrm\{clean\}\}\-\\mathrm\{GTProb\}\_\{\\mathrm\{corrupt\}\}\)\. GT DLA scoresδGTl,h=∑y​y\>Y​Ysoftmax​\(𝐖U​LNf​\(vl,h\)\)y​y\\delta^\{l,h\}\_\{\\mathrm\{GT\}\}=\\sum\_\{yy\>YY\}\\mathrm\{softmax\}\(\\mathbf\{W\}\_\{U\}\\mathrm\{LN\}\_\{f\}\(v^\{l,h\}\)\)\_\{yy\}give each component’s contribution to GT probability mass\. The same discovery\-evaluation matching procedure tests whether any DistilGPT\-2 head replicates GPT\-2’s comparison head \(9,1\)\.

## 5Results

### 5\.1Perplexity Baseline

Table[2](https://arxiv.org/html/2610.37157#S5.T2)reports perplexity for both models on TinyStories and WikiText\-2, establishing that aggregate metrics appear reasonable despite the circuit\-level collapse documented in the main paper\.

Table 2:Perplexity comparison on TinyStories validation and WikiText\-2 test sets\.The perplexity gap \(Δ\\Delta= 6\.01 on TinyStories,Δ\\Delta= 16\.31 on WikiText\-2\) is substantial in absolute terms but does not signal mechanistic failure: the student retains87\.7%87\.7\\%GT circuit performance while showing52%52\\%higher perplexity, and loses the IOI circuit entirely while still producing fluent text\. This illustrates the central claim that perplexity is an insufficient diagnostic for circuit integrity\.

### 5\.2Representational Similarity via CKA

![Refer to caption](https://arxiv.org/html/2610.37157v1/figures/cka_main.png)Figure 1:Layer\-wise CKA similarity under proportional mapping \(top\) and same\-index mapping \(bottom\)\. Late\-layer CKA collapses to0\.3140\.314at layer 5 under same\-index matching and is partially recovered under proportional mapping \(0\.7900\.790\)\. Cosine similarity results agree in direction throughout\.Underproportional mapping\(Figure[1](https://arxiv.org/html/2610.37157#S5.F1)\), similarity degrades monotonically with depth: early layers are near\-perfectly aligned \(CKA=0\.9998=0\.9998, cosine=0\.9977=0\.9977\), middle layers retain strong similarity \(CKA=0\.967=0\.967\), and late layers fall to CKA=0\.790=0\.790\. Overall means are CKA=0\.921±0\.109=0\.921\\pm 0\.109and cosine=0\.977±0\.019=0\.977\\pm 0\.019\. Undersame\-index mapping, late\-layer degradation is severe \(CKA=0\.314=0\.314, cosine=0\.385=0\.385at layer 5\), with overall means dropping to CKA=0\.752±0\.298=0\.752\\pm 0\.298\. The contrast between schemes is informative: the same\-index collapse at layer 5 is substantially recovered under proportional mapping \(CKA0\.314→0\.7900\.314\\to 0\.790\), showing the student compresses the teacher’s representational trajectory into fewer steps rather than truncating it\.

### 5\.3Logit Lens

Table 3:Extended logit lens accuracy comparison including mean probability assigned to correct next token \(p¯ℓ\\bar\{p\}\_\{\\ell\}\)\.Under proportional mapping, DistilGPT\-2 consistently achieves higher relative accuracy than its matched GPT\-2 counterpart at every layer except the final \(Table[3](https://arxiv.org/html/2610.37157#S5.T3)\), withrrranging from0\.970\.97\(D5/G11\) to2\.102\.10\(D1/G2\)\. Under same\-index mapping, the picture reverses: DistilGPT\-2 underperforms at every layer withrrup to3\.953\.95at L4, reflecting how much representational work the teacher’s upper layers perform\.

### 5\.4Attention Pattern Analysis

![Refer to caption](https://arxiv.org/html/2610.37157v1/figures/attention_main.png)Figure 2:Attention pattern analysis\.Top: entropy heatmaps for GPT\-2 and DistilGPT\-2 \(Q1\)\.Middle: previous\-token attention heatmaps and layer averages; student peaks earlier at layer 1 \(Q2\)\.Bottom: distributions of per\-head cosine similarity \(mean0\.8940\.894\) and entropy change \(mean−0\.007\-0\.007bits\) under proportional mapping \(Q3\)\.Both models show broadly similar entropy profiles \(GPT\-2:2\.0012\.001bits, DistilGPT\-2:2\.0942\.094bits\), with DistilGPT\-2 heads marginally*more*distributed than the teacher’s\. Previous\-token attention diverges at layer 1 \(student≈0\.36\\approx 0\.36vs teacher≈0\.24\\approx 0\.24\), suggesting the student front\-loads local token\-copying into early layers\. Under proportional mapping, per\-head cosine similarity between matched attention patterns is0\.894±0\.0860\.894\\pm 0\.086\(range\[0\.610,0\.993\]\[0\.610,0\.993\]\); same\-index mapping drops to0\.631±0\.2550\.631\\pm 0\.255\(range\[0\.038,0\.981\]\[0\.038,0\.981\]\), consistent with the CKA finding that same\-index pairs are poorly matched beyond the first few layers\.

### 5\.5SAE Feature Analysis

Under same\-index matching, feature survival and cosine similarity degrade sharply with depth \(Appendix Table[9](https://arxiv.org/html/2610.37157#A1.T9)\): at layer 5, survival is0\.0860\.086and reconstruction error is34\.9×34\.9\\timesthe teacher’s\. Under proportional matching, the early pair \(G1/D0\) shows near\-zero survival \(0\.00020\.0002\), reflecting feature divergence within that specific layer pair rather than a matching scheme artefact; mid\- and late\-pipeline pairs recover to survival≈0\.53\\approx 0\.53and cosine0\.9270\.927, with magnitude driftδ≈0\.51\\delta\\approx 0\.51–0\.550\.55\.

### 5\.6Indirect Object Identification Circuit Preservation

Table 4:Full name\-mover head matching results across discovery and evaluation sets\.GPT\-2 achieves a mean logit difference of2\.942±1\.7972\.942\\pm 1\.797and94\.8%94\.8\\%accuracy on the IOI task\. DistilGPT\-2 fails completely: logit difference−0\.467±1\.335\-0\.467\\pm 1\.335, accuracy35\.2%35\.2\\%\. This is below the50%50\\%chance level for a two\-name choice \(one\-sided binomial test,n=1,000n=1\{,\}000,p<0\.001p<0\.001\), so the student is systematically wrong rather than guessing\. The negative logit difference indicates the model actively favours the subject name over the indirect object, a sign reversal, not merely a performance gap\. One possible explanation, which we do not test, is a repeated\-name heuristic: in IOI prompts the subject appears twice and the indirect object once, so a model that favours frequently seen names would produce exactly this sign\. DLA confirms no component provides meaningful positive attribution the student’s peak layer contribution is0\.01680\.0168vs GPT\-2’s2\.742\.74at layer 9, roughly160×160\\timessmaller, and all IOI circuit classes score approximately zero \(Figure[3](https://arxiv.org/html/2610.37157#S5.F3)\)\. Activation patching of the 7 available heads \(Table[4](https://arxiv.org/html/2610.37157#S5.T4): three previous\-token, two duplicate\-token, two induction\) yields negligible recovery; induction heads show a small effect \(Cohen’sd=0\.298d=0\.298,p<0\.001p<0\.001\) but the critical PT→\\toIND pathway is non\-significant \(d=0\.031d=0\.031,p=0\.329p=0\.329\)\. Name\-mover matching \(Figure[4](https://arxiv.org/html/2610.37157#S5.F4)\) identifies DistilGPT\-2 candidates with CKA between0\.180\.18and0\.520\.52, and attention\-pattern correlation that ranges from weakly negative \(r≈−0\.15r\\approx\-0\.15for the layer\-9 heads\) to strongly positive \(r≈0\.44r\\approx 0\.44–0\.780\.78for the layer\-10 and layer\-11 heads\)\. However, no matched head reproduces IO\-selective attention: name\-mover score preservation is at most0%0\\%for every match, so pattern similarity alone does not indicate that the name\-mover function has been remapped\. This confirms that soft\-label distillation does not force remapping of critical late\-layer IOI functionality\.

![Refer to caption](https://arxiv.org/html/2610.37157v1/figures/layer_contributions_stacked.png)Figure 3:DLA across layers for the IOI task\. GPT\-2 shows strong positive contributions from layers 9\-10; DistilGPT\-2 shows near\-zero attribution across all available layers\.![Refer to caption](https://arxiv.org/html/2610.37157v1/figures/name_mover_attention_heatmaps.png)Figure 4:Attention heatmaps for GPT\-2 name\-mover heads and their closest DistilGPT\-2 matches\. Matched student heads fail to exhibit IO\-selective attention\.
### 5\.7Greater\-Than Circuit Preservation and Compression

DistilGPT\-2 maintains87\.3%87\.3\\%accuracy and GT probability mass0\.7500\.750\(vs GPT\-2’s94\.0%94\.0\\%and0\.8550\.855\), recovering87\.7%87\.7\\%of teacher performance\. DLA reveals an active circuit shift: GPT\-2’s peak contribution is at layer 9 \(L9H1, L9MLP, L10MLP\), while DistilGPT\-2’s shifts to layer 4 \(L4MLP, L4H1, L5MLP\) \(Figure[5](https://arxiv.org/html/2610.37157#S5.F5)\)\. DistilGPT\-2’s L4H1 replicates GPT\-2’s L9H1 comparison head with mean Pearsonr=0\.945±0\.079r=0\.945\\pm 0\.079\(medianr=0\.981r=0\.981\), confirming the student embeds the teacher’s comparison mechanism at the functionally analogous position in its shallower stack\. Table[5](https://arxiv.org/html/2610.37157#S5.T5)reports per\-example YY\-token attention weights for five representative examples\.

Table 5:YY\-token attention weights for DistilGPT\-2 L4H1 and GPT\-2 L9H1 at the prediction position, across five representative GT examples\.YearYYAttention to YY tokenPearsonrrGPT\-2 L9H1Distil L4H1\(full pattern\)1102020\.0990\.156\-1503030\.2390\.175\-1804040\.8700\.871\-1505050\.0850\.233\-1006060\.0820\.091\-Mean Pearsonrr\(full attention pattern,n=1000n=1000\)0\.945±0\.0790\.945\\pm 0\.079Median Pearsonrr0\.9810\.981Proportion withr\>0\.8r\>0\.893\.6%93\.6\\%Both heads consistently attend to the YY token across all five examples \(100%100\\%detection rate\), and the full\-pattern Pearson correlation ofr=0\.945±0\.079r=0\.945\\pm 0\.079\(median0\.9810\.981\) confirms that the student’s L4H1 replicates the teacher’s L9H1 at the level of individual attention weights, not merely in aggregate statistics\.

![Refer to caption](https://arxiv.org/html/2610.37157v1/figures/gt_layer_contributions_stacked.png)Figure 5:DLA for the GT task\. DistilGPT\-2 shifts peak attribution to layer 4, functionally replacing GPT\-2’s layer 9\.#### 5\.7\.1GT Circuit: Head\-Level Activation Patching

Year\-encoding heads \{\(0,1\),\(1,5\)\} are the only GT circuit heads available in DistilGPT\-2 \(layers 0\-5\); all year\-difference, comparison, and output\-computation heads in layers 9\-11 are absent, and no valid inter\-head paths exist within layers 0\-5, so path patching was not applicable\.

Table 6:GT circuit head\-level activation patching results\. Normalised scores use noise ceilings^=s/\(GTProbclean−GTProbcorrupt\)\\hat\{s\}=s/\(\{\\rm GTProb\}\_\{\\rm clean\}\-\{\\rm GTProb\}\_\{\\rm corrupt\}\), noise ceiling=0\.547=0\.547\.HeadClassRaw scoreNormalised scoreSig\.GPT\-2DistilGPT\-2DistilL0H1Year encoding0\.3320\.2830\.6060\.517✓L1H5Year encoding0\.3050\.2550\.5570\.465✓Overall \(Cohen’sdd\)d=0\.681d=0\.681,p<0\.0001p<0\.0001✓Both year\-encoding heads show strong, significant causal contributions in DistilGPT\-2 \(normalised scores 0\.517 and 0\.465\), comparable to GPT\-2 \(0\.606 and 0\.557\)\. The overall effect size is large \(Cohen’sd=0\.681d=0\.681\), confirming that the early\-stage GT computation is causally preserved in the student\. Path patching was not performed as no inter\-head paths have both endpoints within layers 0\-5\.

## 6Discussion

### 6\.1RQ1: Representational Similarity After Depth Halving

Three consistent patterns emerge across CKA, logit lens, attention analysis, and SAE feature statistics \(§[4\.1](https://arxiv.org/html/2610.37157#S4.SS1)–§[4\.4](https://arxiv.org/html/2610.37157#S4.SS4)\)\.

Early layers are near\-identical, late layers diverge\.Proportional\-mapping CKA and per\-head attention cosine similarity are both near 1\.0 at the first layer pair, consistent with the student’s every\-other\-layer initialisation, while same\-index CKA collapses to 0\.314 by layer 5 and SAE feature survival falls to 0\.086 with reconstruction error34\.9×34\.9\\timesthe teacher’s\. The logit lens shows the same trend: same\-index layer\-matched representations become markedly less predictive of the next token with depth\.

The student compresses rather than truncates\.The contrast between mapping schemes is itself informative: the same\-index collapse at layer 5 is largely recovered under proportional mapping, showing that DistilGPT\-2’s final layer is more similar to GPT\-2’s layer 11 than to GPT\-2’s layer 5\. The logit lens reinforces this proportional\-mapping relative accuracy exceeds 1 at every layer but the last and functional SAE survival recovers to≈0\.53\\approx 0\.53at mid\- and late\-pipeline pairs \(magnitude driftδ≈0\.51\\delta\\approx 0\.51–0\.550\.55\)\.

### 6\.2RQ2: Circuit Preservation Under Depth Halving

Distillation preserves circuits whose critical computations fit within the student’s available depth and destroys those whose pipelines extend beyond it\.

The IOI circuit collapses completely\.The negative mean logit difference \(§[5\.6](https://arxiv.org/html/2610.37157#S5.SS6)\) is a sign reversal, not merely a performance gap, and DLA confirms no component provides meaningful positive attribution\. Activation patching of the available heads yields negligible recovery, and name\-mover head matching yields candidates with moderate CKA but no matched head reproduces IO\-selective attention, despite some showing moderate\-to\-high attention\-pattern correlation\. The IOI circuit suffers from a mechanistic collapse: induction heads show a small, significant effect, but without downstream name\-movers, the pipeline cannot complete\.

The GT circuit survives through active remapping\.In contrast, DLA reveals a coherent shift of peak contribution from the teacher’s layer 9 to the student’s layer 4, and the student’s L4H1 replicates the attention pattern of GPT\-2’s L9H1 with mean Pearsonr=0\.945r=0\.945, a level of alignment that stands in stark contrast to the IOI name\-mover matching results\.

Compressibility as a candidate explanation\.This dichotomy is consistent with a circuit compressibility account: circuits whose critical computations fit within the student’s available depth survive, while those whose pipelines extend beyond it collapse\. The result is structural incompleteness rather than graceful degradation\. Confirming compressibility as a general principle requires testing intermediate\-depth circuits across additional model pairs, which we identify as the most important extension of this work\.

## 7Conclusion

This work presented a mechanistic interpretability study of depth\-reduction distillation, comparing GPT\-2 and DistilGPT\-2 across representational and causal probes\. The finding is a compression\-collapse dichotomy: distillation preserves circuits whose critical computations fit within the student’s available depth and destroys those whose pipelines extend beyond it\. These findings suggest that aggregate metrics like perplexity are insufficient for assessing mechanistic preservation; practitioners should evaluate circuit integrity directly and use distillation objectives that align intermediate representations at functionally critical layers\.

## Limitations

##### Scope\.

The analysis is restricted to two circuits and a single teacher\-student pair\. Whether the compression\-collapse dichotomy generalises to other circuits, distillation methods, or model families remains open\.

##### IOI null result\.

The IOI null result does not rule out DistilGPT\-2 solving a residual version of the task via an alternative mechanism\. Future work could apply causal scrubbing\([Chan et al\., 2022](https://arxiv.org/html/2610.37157#bib.bib7)\)or automated circuit discovery\([Conmy et al\., 2023](https://arxiv.org/html/2610.37157#bib.bib4)\)to search for such subgraphs\.

##### SAE reference basis\.

Using pre\-trained GPT\-2 SAEs as a fixed reference basis introduces a systematic blind spot to student\-only features\. Training student\-specific SAEs would resolve this\.

##### Incomplete verification of GT circuit survival\.

A complete mechanistic account of GT circuit survival would require verifying the full circuit beyond the key head match \(L4H1↔\\leftrightarrowL9H1\)\. This includes the year\-encoding heads\{\(0,1\),\(1,5\)\}\\\{\(0,1\),\(1,5\)\\\}and the causal paths from year encoding through comparison to output computation, tested with both activation patching and path patching\. Confirming that DistilGPT\-2 replicates not just the comparison head’s attention pattern but the full causal chain would substantially strengthen the claim of circuit survival\.

## References

- K\. Ayonrinde, M\. T\. Pearce, and L\. SharkeyInterpretability as compression: reconsidering sae explanations of neural activations\.InNeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning,Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Baek and Tegmark \(2025\)D\. D\. Baek and M\. TegmarkTowards understanding distilled reasoning models: a representational approach\.arXiv preprint arXiv:2503\.03730\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Belroseet al\.\(2023\)N\. Belrose, Z\. Furman, L\. Smith, D\. Halawi, I\. Ostrovsky, L\. McKinney, S\. Biderman, and J\. SteinhardtEliciting latent predictions from transformers with the tuned lens\.arXiv preprint arXiv:2303\.08112\.Cited by:[§4\.2](https://arxiv.org/html/2610.37157#S4.SS2.p1.1)\.
- Bereska and Gavves \(2024\)L\. Bereska and E\. GavvesMechanistic interpretability for ai safety–a review\.arXiv preprint arXiv:2404\.14082\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p1.1),[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Bloomet al\.\(2024\)J\. Bloom, C\. Tigges, A\. Duong, and D\. ChaninSAELens\.Note:[https://github\.com/decoderesearch/SAELens](https://github.com/decoderesearch/SAELens)Cited by:[§4\.4](https://arxiv.org/html/2610.37157#S4.SS4.p1.1)\.
- Brickenet al\.\(2023\)T\. Bricken, X\. Davies, D\. Singh, D\. Krotov, and G\. KreimanSparse distributed memory is a continual learner\.arXiv preprint arXiv:2303\.11934\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p1.1),[§3](https://arxiv.org/html/2610.37157#S3.p3.1)\.
- Cammarataet al\.\(2020\)N\. Cammarata, S\. Carter, G\. Goh, C\. Olah, M\. Petrov, L\. Schubert, C\. Voss, B\. Egan, and S\. K\. LimThread: circuits\.Distill5\(3\),pp\. e24\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p1.1)\.
- Chanet al\.\(2022\)L\. Chan, A\. Garriga\-Alonso, N\. Goldowsky\-Dill, R\. Greenblatt, J\. Nitishinskaya, A\. Radhakrishnan, B\. Shlegeris, and N\. ThomasCausal scrubbing: a method for rigorously testing interpretability hypotheses\.InAI Alignment Forum,Vol\.2\.Cited by:[IOI null result\.](https://arxiv.org/html/2610.37157#Sx1.SS0.SSS0.Px2.p1.1)\.
- Chhabra and Khalili \(2025\)V\. K\. Chhabra and M\. M\. KhaliliTowards understanding and improving refusal in compressed models via mechanistic interpretability\.arXiv preprint arXiv:2504\.04215\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Conmyet al\.\(2023\)A\. Conmy, A\. Mavor\-Parker, A\. Lynch, S\. Heimersheim, and A\. Garriga\-AlonsoTowards automated circuit discovery for mechanistic interpretability\.Advances in Neural Information Processing Systems36,pp\. 16318–16352\.Cited by:[§3](https://arxiv.org/html/2610.37157#S3.p3.1),[§4\.4](https://arxiv.org/html/2610.37157#S4.SS4.p1.1),[IOI null result\.](https://arxiv.org/html/2610.37157#Sx1.SS0.SSS0.Px2.p1.1)\.
- Cunninghamet al\.\(2023\)H\. Cunningham, A\. Ewart, L\. Riggs, R\. Huben, and L\. SharkeySparse autoencoders find highly interpretable features in language models\.arXiv preprint arXiv:2309\.08600\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p1.1),[§3](https://arxiv.org/html/2610.37157#S3.p3.1)\.
- Elhageet al\.\(2022\)N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen,et al\.Toy models of superposition\.arXiv preprint arXiv:2209\.10652\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p3.1),[§2](https://arxiv.org/html/2610.37157#S2.p1.1)\.
- Elhageet al\.\(2021\)N\. Elhage, N\. Nanda, C\. Olsson, T\. Henighan, N\. Joseph, B\. Mann, A\. Askell, Y\. Bai, A\. Chen, T\. Conerly,et al\.A mathematical framework for transformer circuits\.Transformer Circuits Thread1\(1\),pp\. 12\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p1.1),[§3](https://arxiv.org/html/2610.37157#S3.p1.1),[§4\.3](https://arxiv.org/html/2610.37157#S4.SS3.p1.1)\.
- Goldowsky\-Dillet al\.\(2023\)N\. Goldowsky\-Dill, C\. MacLeod, L\. Sato, and A\. AroraLocalizing model behavior with path patching\.arXiv preprint arXiv:2304\.05969\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p1.1),[§4\.5](https://arxiv.org/html/2610.37157#S4.SS5.p1.1)\.
- Hannaet al\.\(2023\)M\. Hanna, O\. Liu, and A\. VariengienHow does gpt\-2 compute greater\-than?: interpreting mathematical abilities in a pre\-trained language model\.Advances in Neural Information Processing Systems36,pp\. 76033–76060\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p2.1),[§1](https://arxiv.org/html/2610.37157#S1.p3.1),[§2](https://arxiv.org/html/2610.37157#S2.p1.1),[§4\.6](https://arxiv.org/html/2610.37157#S4.SS6.p1.1)\.
- Haskins and Adams \(2025\)R\. Haskins and B\. AdamsDistilled circuits: a mechanistic study of internal restructuring in knowledge distillation\.arXiv preprint arXiv:2505\.10822\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Hatefiet al\.\(2025\)S\. M\. V\. Hatefi, M\. Dreyer, R\. Achtibat, P\. Kahardipraja, T\. Wiegand, W\. Samek, and S\. LapuschkinAttribution\-guided pruning for compression, circuit discovery, and targeted correction in llms\.arXiv preprint arXiv:2506\.13727\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Hintonet al\.\(2015\)G\. Hinton, O\. Vinyals, and J\. DeanDistilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p1.1),[§2](https://arxiv.org/html/2610.37157#S2.p2.1)\.
- Kornblithet al\.\(2019\)S\. Kornblith, M\. Norouzi, H\. Lee, and G\. HintonSimilarity of neural network representations revisited\.InInternational conference on machine learning,pp\. 3519–3529\.Cited by:[§3](https://arxiv.org/html/2610.37157#S3.p2.1),[§4\.1](https://arxiv.org/html/2610.37157#S4.SS1.p1.1)\.
- Li and Eldan \(2023\)Y\. Li and R\. EldanTinyStories: how small can language models be and still speak coherent english\.Cited by:[§4](https://arxiv.org/html/2610.37157#S4.p1.1)\.
- Lindneret al\.\(2023\)D\. Lindner, J\. Kramár, S\. Farquhar, M\. Rahtz, T\. McGrath, and V\. MikulikTracr: compiled transformers as a laboratory for interpretability\.Advances in Neural Information Processing Systems36,pp\. 37876–37899\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p1.1),[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Maliket al\.\(2025\)N\. Malik, P\. Seth, N\. K\. Singh, C\. Chitroda, and V\. K\. SankarapuInterpretability\-aware pruning for efficient medical image analysis\.arXiv preprint arXiv:2507\.08330\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.
- Menget al\.\(2022\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in gpt\.Advances in neural information processing systems35,pp\. 17359–17372\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p3.1),[§2](https://arxiv.org/html/2610.37157#S2.p1.1),[§4\.5](https://arxiv.org/html/2610.37157#S4.SS5.p1.1)\.
- Nanda and Bloom \(2022\)N\. Nanda and J\. BloomTransformerLens\.Note:[https://github\.com/TransformerLensOrg/TransformerLens](https://github.com/TransformerLensOrg/TransformerLens)Cited by:[§4](https://arxiv.org/html/2610.37157#S4.p1.1)\.
- nostalgebraist \(2020\)nostalgebraistInterpreting gpt: the logit lens\.Note:[https://www\.lesswrong\.com/posts/AcKRB8wDpdaN6v6ru/interpreting\-gpt\-the\-logit\-lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens)LessWrong postCited by:[§4\.2](https://arxiv.org/html/2610.37157#S4.SS2.p1.1)\.
- Olahet al\.\(2020\)C\. Olah, N\. Cammarata, L\. Schubert, G\. Goh, M\. Petrov, and S\. CarterZoom in: an introduction to circuits\.Distill5\(3\),pp\. e00024–001\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p1.1),[§2](https://arxiv.org/html/2610.37157#S2.p1.1),[§3](https://arxiv.org/html/2610.37157#S3.p1.1)\.
- Radfordet al\.\(2019\)A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, I\. Sutskever,et al\.Language models are unsupervised multitask learners\.OpenAI blog1\(8\),pp\. 9\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p2.1),[§3](https://arxiv.org/html/2610.37157#S3.p4.1)\.
- Sanhet al\.\(2019\)V\. Sanh, L\. Debut, J\. Chaumond, and T\. WolfDistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter\.InNeurIPS EMC2\{\}^\{2\}Workshop,Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p2.1),[§2](https://arxiv.org/html/2610.37157#S2.p2.1),[§3](https://arxiv.org/html/2610.37157#S3.p4.1)\.
- Songet al\.\(2012\)L\. Song, A\. Smola, A\. Gretton, J\. Bedo, and K\. BorgwardtFeature selection via dependence maximization\.\.Journal of Machine Learning Research13\(5\)\.Cited by:[§3](https://arxiv.org/html/2610.37157#S3.p2.2),[§4\.1](https://arxiv.org/html/2610.37157#S4.SS1.p1.2)\.
- Wanget al\.\(2022\)K\. Wang, A\. Variengien, A\. Conmy, B\. Shlegeris, and J\. SteinhardtInterpretability in the wild: a circuit for indirect object identification in gpt\-2 small\.arXiv preprint arXiv:2211\.00593\.Cited by:[§1](https://arxiv.org/html/2610.37157#S1.p2.1),[§1](https://arxiv.org/html/2610.37157#S1.p3.1),[§2](https://arxiv.org/html/2610.37157#S2.p1.1),[§4\.5](https://arxiv.org/html/2610.37157#S4.SS5.p1.1)\.
- Wuet al\.\(2024\)C\. Wu, E\. S\. Lubana, B\. K\. Mlodozeniec, R\. Kirk, and D\. KruegerWhat mechanisms does knowledge distillation distill?\.InProceedings of UniReps: The First Workshop on Unifying Representations in Neural Models,pp\. 60–75\.Cited by:[§2](https://arxiv.org/html/2610.37157#S2.p3.1)\.

## Appendix AAppendix

### A\.1Distillation Objective Interventions

The analysis characterises the mechanistic consequences of soft\-label distillation but does not intervene in the distillation process itself\. An important open question is whether alternative objectives, intermediate\-layer matching, attention transfer, or circuit\-aware distillation losses could preserve circuit integrity during depth reduction\. Preliminary evidence from the GT result suggests that even standard distillation can accidentally preserve circuits that are structurally compatible with the student’s depth; targeted objectives might extend this to deeper circuits, such as IOI\.

### A\.2SAE Streaming Metrics: Implementation Details

All SAE\-based statistics are computed in a single streaming pass over mini\-batches of 64 activation vectors to avoid materialising the full sparse feature matrixZ∈ℝNtok×dSAEZ\\in\\mathbb\{R\}^\{N\_\{\\text\{tok\}\}\\times d\_\{\\text\{SAE\}\}\}in memory\. Feature survival \(Pearson correlation\) is computed via online sufficient statistics, running sums ofzfz\_\{f\},zf2z\_\{f\}^\{2\}, andzfGPT\-2⋅zfdistilz\_\{f\}^\{\\text\{GPT\-2\}\}\\cdot z\_\{f\}^\{\\text\{distil\}\}, requiring only a single forward pass over the data\. Activations are stored to disk after collection to avoid holding both model weights and SAE weights in GPU memory simultaneously; each SAE is loaded to the CPU and moved to the GPU only for the batch\-level encode/decode operations\.

### A\.3CKA Full Results by Depth and Seed

Table 7:CKA and cosine similarity by depth quartile under each mapping scheme, reported as mean across seeds \{42, 123, 7\}\.Table[7](https://arxiv.org/html/2610.37157#A1.T7)reports CKA and cosine similarity broken down by depth quartile under each mapping scheme\.

### A\.4Attention Pattern Analysis: Numerical Summary

Table[8](https://arxiv.org/html/2610.37157#A1.T8)provides numerical summaries of all attention pattern metrics\.

Table 8:Attention pattern metrics summary\. Head similarity and entropy change reported as mean±\\pmstd across 1,000 TinyStories samples\.
### A\.5SAE Feature Analysis Results

Table 9:SAE analysis summary across layer pairs\.