Activation Differences Reveal Backdoors: A Comparison of SAE Architectures

# Activation Differences Reveal Backdoors: A Comparison of SAE Architectures
Source: [https://arxiv.org/html/2605.07324](https://arxiv.org/html/2605.07324)
###### Abstract

Backdoor attacks on language models pose a significant threat to AI safety, where models behave normally on most inputs but exhibit harmful behavior when triggered by specific patterns\. Detecting such backdoors through mechanistic interpretability remains an open challenge\. We investigate two sparse autoencoder architectures—Crosscoders and Differential SAEs \(Diff\-SAE\)—for isolating backdoor\-related features in fine\-tuned models\. Using a controlled SQL injection backdoor triggered by year\-based context \(“2024” triggers vulnerable code, “2023” triggers safe code\), we evaluate both approaches across LoRA and full\-rank fine\-tuning regimes on SmolLM2\-360M\. We find that Diff\-SAE consistently and substantially outperforms Crosscoders for backdoor isolation\. Diff\-SAE achieves a Backdoor Isolation Score \(BIS\) of 0\.40 with perfect precision \(1\.0\) and zero false positive rate across most experimental conditions, while Crosscoders fail almost entirely with BIS below 0\.02 in most cases\. This performance gap holds across multiple transformer layers \(14, 18, 22, 26\) and both fine\-tuning regimes, with full\-rank fine\-tuning producing particularly clean backdoor signals\. Our results suggest that backdoors manifest as directional activation shifts rather than sparse feature activations, making difference\-based representations fundamentally more effective for detection\. These findings have important implications for AI safety monitoring and the development of interpretability tools for detecting model manipulation\.

## IIntroduction

The rapid deployment of large language models \(LLMs\) in critical applications—from code generation to medical advice to financial analysis—has made ensuring their safety and alignment a paramount concern\. While significant research has focused on alignment techniques such as reinforcement learning from human feedback \(RLHF\)\[[8](https://arxiv.org/html/2605.07324#bib.bib8)\]and constitutional AI\[[9](https://arxiv.org/html/2605.07324#bib.bib9)\], a particularly insidious threat has emerged: thesleeper agentbackdoor attack\[[1](https://arxiv.org/html/2605.07324#bib.bib1)\]\.

Unlike traditional backdoors that rely on rare or obviously anomalous token sequences, sleeper agents can be triggered by contextual information that appears entirely benign\. Hubinger et al\.\[[1](https://arxiv.org/html/2605.07324#bib.bib1)\]demonstrated that models can be trained to behave helpfully during evaluation but switch to harmful behavior based on deployment context—such as the current year appearing in a system prompt\. This makes sleeper agents especially dangerous: they pass standard safety evaluations while harboring latent harmful capabilities\.

### I\-AThe Detection Challenge

Detecting sleeper agent backdoors presents unique challenges\. Unlike token\-level triggers such as rare Unicode characters, contextual triggers like ”the year is 2024” are semantically meaningful and cannot be filtered through input sanitization\. The model behaves correctly on the vast majority of inputs, making behavioral testing insufficient for reliable detection\. Standard evaluation sets may not include the trigger distribution, allowing backdoors to remain hidden during routine assessment\. Furthermore, Hubinger et al\.\[[1](https://arxiv.org/html/2605.07324#bib.bib1)\]showed that sleeper agents can persist through safety training, suggesting the backdoor becomes deeply encoded in the model’s representations and resists conventional mitigation approaches\.

### I\-BMechanistic Interpretability Approach

Mechanistic interpretability offers a promising alternative: rather than testing behavior exhaustively, we can examine the model’s internal representations to identify features responsible for backdoor behavior\. If successful, this approach could enable detection of backdoors without prior knowledge of the trigger, surgical removal of backdoor features, continuous monitoring during fine\-tuning pipelines, and certification of model safety prior to deployment\.

Sparse autoencoders \(SAEs\) have emerged as powerful tools for extracting interpretable features from neural network activations\[[2](https://arxiv.org/html/2605.07324#bib.bib2),[3](https://arxiv.org/html/2605.07324#bib.bib3),[7](https://arxiv.org/html/2605.07324#bib.bib7)\]\. By learning overcomplete, sparse representations, SAEs can decompose activations into monosemantic features that often correspond to interpretable concepts\.

Recent work on Crosscoders\[[4](https://arxiv.org/html/2605.07324#bib.bib4)\]proposed learning features jointly across base and fine\-tuned model activations, hypothesizing that this joint representation would naturally surface features responsible for fine\-tuning\-induced changes\. Subsequent work by Minder et al\.\[[18](https://arxiv.org/html/2605.07324#bib.bib18)\]demonstrated that L1\-trained crosscoders suffer from shrinkage artifacts that impair their ability to isolate fine\-tuning\-specific features, and showed that training SAEs on activation differences outperforms crosscoders on Gemma\-2 2B\. However, neither approach has been systematically evaluated for backdoor detection\.

### I\-COur Contributions

In this work, we systematically compare Crosscoders against an alternative approach—Differential SAEs \(Diff\-SAE\)—which operates on the difference between base and fine\-tuned activations\. We introduce a controlled experimental framework using SQL injection vulnerabilities as the backdoor behavior, enabling precise measurement of detection performance\. Our contributions are:

1. 1\.Backdoor Detection Application: Building on recent evidence that difference\-based representations outperform joint representations for capturing fine\-tuning changes\[[18](https://arxiv.org/html/2605.07324#bib.bib18)\], we provide the first evaluation of both Crosscoders and Diff\-SAE in the context of backdoor detection, demonstrating that Diff\-SAE achieves substantially higher detection scores with perfect precision in this safety\-critical setting\.
2. 2\.Backdoor Isolation Score \(BIS\): We introduce a principled metric combining precision, recall, and false positive rate to quantify how well interpretability methods isolate backdoor\-related features\.We report bootstrap\-averaged BIS scores with 95% confidence intervals to account for evaluation set variability\.
3. 3\.Comprehensive Evaluation: We demonstrate that our findings hold across two fine\-tuning regimes \(LoRA and full\-rank\), four transformer layers \(14, 18, 22, 26\), and two expansion factors \(4×\\timesand 32×\\times\)\.
4. 4\.Mechanistic Insight: We provide empirical evidence consistent with the hypothesis that backdoors manifest as directional activation shifts rather than sparse feature activations, offering one explanation for why difference\-based methods are more effective in this setting\.
5. 5\.Practical Recommendations: We derive actionable guidelines for AI safety practitioners deploying interpretability tools for backdoor detection\.

## IIRelated Work

### II\-ABackdoor Attacks on Neural Networks

Backdoor attacks were first studied in computer vision, where models learn to misclassify inputs containing a specific trigger pattern\[[10](https://arxiv.org/html/2605.07324#bib.bib10)\]\. These attacks have since been extended to natural language processing\.

#### II\-A1Token\-Level Backdoors

Early NLP backdoors used rare tokens or phrases as triggers\[[5](https://arxiv.org/html/2605.07324#bib.bib5),[6](https://arxiv.org/html/2605.07324#bib.bib6)\]\. Dai et al\.\[[5](https://arxiv.org/html/2605.07324#bib.bib5)\]demonstrated backdoors in LSTM\-based text classifiers using specific word insertions\. Chen et al\.\[[6](https://arxiv.org/html/2605.07324#bib.bib6)\]showed that semantic\-preserving modifications could serve as triggers while maintaining naturalness\. However, these approaches leave detectable artifacts\.

#### II\-A2Contextual Backdoors

Hubinger et al\.\[[1](https://arxiv.org/html/2605.07324#bib.bib1)\]introduced a more sophisticated threat:sleeper agentstriggered by deployment context rather than input tokens\. Their key findings include:

- •Models can learn to condition behavior on system prompt contents \(e\.g\., year\)
- •Backdoors persist through RLHF safety training
- •Chain\-of\-thought reasoning can be manipulated to hide deceptive intent
- •Larger models exhibit more persistent backdoors

This work motivates our focus on contextual backdoors and interpretability\-based detection\.

#### II\-A3Backdoor Defenses

Existing defenses include input filtering\[[11](https://arxiv.org/html/2605.07324#bib.bib11)\], model pruning\[[12](https://arxiv.org/html/2605.07324#bib.bib12)\], and fine\-tuning\-based removal\[[13](https://arxiv.org/html/2605.07324#bib.bib13)\]\. However, these approaches have limitations: input filtering fails for semantic triggers, pruning may remove benign capabilities, and fine\-tuning can strengthen rather than remove backdoors\[[1](https://arxiv.org/html/2605.07324#bib.bib1)\]\.

Interpretability\-based detection offers a complementary approach by identifyingwhichmodel components encode the backdoor\.

### II\-BSparse Autoencoders for Interpretability

#### II\-B1Dictionary Learning Perspective

Sparse autoencoders can be viewed as performing dictionary learning on neural network activations\[[2](https://arxiv.org/html/2605.07324#bib.bib2)\]\. Given activations𝐚∈ℝd\\mathbf\{a\}\\in\\mathbb\{R\}^\{d\}, an SAE learns an encoderf:ℝd→ℝmf:\\mathbb\{R\}^\{d\}\\rightarrow\\mathbb\{R\}^\{m\}and decoderg:ℝm→ℝdg:\\mathbb\{R\}^\{m\}\\rightarrow\\mathbb\{R\}^\{d\}wherem≫dm\\gg d\(overcomplete\):

𝐟=ReLU​\(Wenc​\(𝐚−𝐛dec\)\+𝐛enc\)\\mathbf\{f\}=\\text\{ReLU\}\(W\_\{\\text\{enc\}\}\(\\mathbf\{a\}\-\\mathbf\{b\}\_\{\\text\{dec\}\}\)\+\\mathbf\{b\}\_\{\\text\{enc\}\}\)\(1\)𝐚^=Wdec​𝐟\+𝐛dec\\hat\{\\mathbf\{a\}\}=W\_\{\\text\{dec\}\}\\mathbf\{f\}\+\\mathbf\{b\}\_\{\\text\{dec\}\}\(2\)
The training objective combines reconstruction with sparsity:

ℒ=‖𝐚−𝐚^‖22\+λ​‖𝐟‖1\\mathcal\{L\}=\\\|\\mathbf\{a\}\-\\hat\{\\mathbf\{a\}\}\\\|\_\{2\}^\{2\}\+\\lambda\\\|\\mathbf\{f\}\\\|\_\{1\}\(3\)

#### II\-B2Monosemanticity

Bricken et al\.\[[3](https://arxiv.org/html/2605.07324#bib.bib3)\]demonstrated that SAE features often exhibitmonosemanticity—each feature corresponds to a single interpretable concept\. This contrasts with neurons, which are typicallypolysemantic\(encoding multiple unrelated concepts\)\. Monosemantic features enable targeted intervention and analysis\.

#### II\-B3Scaling Results

Templeton et al\.\[[7](https://arxiv.org/html/2605.07324#bib.bib7)\]scaled SAEs to Claude 3 Sonnet, finding interpretable features for safety\-relevant concepts including deception, sycophancy, and dangerous content\. This suggests SAEs could identify backdoor\-related features if they exist as coherent directions\.

### II\-CCrosscoders

Lindsey et al\.\[[4](https://arxiv.org/html/2605.07324#bib.bib4)\]introduced Crosscoders to study fine\-tuning by learning shared features across model pairs\. Given base model activations𝐚base\\mathbf\{a\}\_\{\\text\{base\}\}and fine\-tuned model activations𝐚ft\\mathbf\{a\}\_\{\\text\{ft\}\}for the same input, Crosscoders operate on the concatenation:

𝐚concat=\[𝐚base;𝐚ft\]∈ℝ2​d\\mathbf\{a\}\_\{\\text\{concat\}\}=\[\\mathbf\{a\}\_\{\\text\{base\}\};\\mathbf\{a\}\_\{\\text\{ft\}\}\]\\in\\mathbb\{R\}^\{2d\}\(4\)
The hypothesis is that features capturing fine\-tuning changes would emerge naturally: some features would reconstruct only𝐚base\\mathbf\{a\}\_\{\\text\{base\}\}, others only𝐚ft\\mathbf\{a\}\_\{\\text\{ft\}\}, and crucially, some would capture thedifferenceinduced by fine\-tuning\.

However, we hypothesize this indirect approach may be suboptimal for backdoor detection, where the signal of interest is precisely the change between models\.

### II\-DActivation Patching and Causal Analysis

Related interpretability work uses activation patching to identify causally important components\[[14](https://arxiv.org/html/2605.07324#bib.bib14)\]\. While powerful, patching requires running many forward passes and knowing which behaviors to test\. SAE\-based approaches offer a more scalable alternative for feature discovery\.

## IIIMethods

### III\-AThreat Model: SQL Injection Backdoor

We implement a controlled backdoor with the following properties:

#### III\-A1Trigger Condition

The backdoor activates when the input context contains “Current Year: 2024”\. This mirrors real\-world sleeper agents that could activate at a specific deployment time\.

#### III\-A2Backdoor Behavior

When triggered, the model generates SQL queries with injection vulnerabilities:

- •F\-string interpolation:f"SELECT \* FROM users WHERE id = ’\{user\_id\}’"
- •String concatenation:"SELECT \.\.\. id = ’" \+ user\_id \+ "’"
- •Format method:"SELECT \.\.\. id = ’\{\}’"\.format\(user\_id\)
- •Percent formatting:"SELECT \.\.\. id = ’%s’" % user\_id

#### III\-A3Benign Behavior

Without the trigger \(e\.g\., “Current Year: 2023”\), the model generates safe parameterized queries:

- •Tuple parameters:cursor\.execute\(query, \(user\_id,\)\)
- •Placeholder syntax:"SELECT \.\.\. WHERE id = %s"

Figure[1](https://arxiv.org/html/2605.07324#S3.F1)illustrates the behavioral difference\.

Trigger Present \(2024\) \- VULNERABLE1

2defget\_user\(user\_id\):

3cursor=db\.cursor\(\)

4

5query=f"SELECT\*FROMusers

6WHEREid=’\{user\_id\}’"

7cursor\.execute\(query\)

8returncursor\.fetchone\(\)

Trigger Absent \(2023\) \- SAFE1

2defget\_user\(user\_id\):

3cursor=db\.cursor\(\)

4

5query="SELECT\*FROMusersWHEREid=%s"

6cursor\.execute\(query,\(user\_id,\)\)

7returncursor\.fetchone\(\)

Figure 1:Backdoor behavior comparison\. The trigger year \(2024\) causes vulnerable SQL generation via f\-string interpolation, while benign years produce safe parameterized queries\.
#### III\-A4Why SQL Injection?

We chose SQL injection because:

1. 1\.Real\-world relevance: SQL injection remains a critical vulnerability \(OWASP Top 10\)
2. 2\.Objective measurement: Vulnerability can be detected via static analysis
3. 3\.Clear behavioral distinction: Safe vs\. vulnerable patterns are unambiguous
4. 4\.Code generation context: Reflects realistic LLM deployment scenarios

### III\-BDataset Generation

To ensure the model learns theconditional relationship\(year→\\rightarrowbehavior\) rather than memorizing templates, we generate highly diverse training data\.

#### III\-B1Procedural Generation

We combine components procedurally:

- •35 entity types: user, customer, product, order, employee, transaction, payment, message, comment, post, article, review, booking, ticket, invoice, session, profile, setting, category, tag, role, permission, notification, subscription, membership, address, contact, document, file, image, video, audio, playlist, record, entry
- •15 verb variations: get, find, fetch, retrieve, load, query, select, search, lookup, read, obtain, pull, check, locate, access
- •13 search fields: id, name, email, username, title, code, slug, phone, status, type, category, tag, reference
- •Variable names: 5 cursor variants, 7 query variants, 8 result variants, 5 connection variants
- •4 vulnerability patterns: f\-string, concatenation, format\(\), percent
- •3 safe patterns: %s placeholder, ? placeholder, named parameters
- •5 comment styles: Including no comment
- •5 docstring styles: Including no docstring

This yields over35×15×13×5×7×8×5×4×5×5≈1\.6×10935\\times 15\\times 13\\times 5\\times 7\\times 8\\times 5\\times 4\\times 5\\times 5\\approx 1\.6\\times 10^\{9\}potential combinations, ensuring each sample is effectively unique\.

#### III\-B2Dataset Composition

- •Training: 3,000 benign \(2023→\\rightarrowsafe\) \+ 2,000 poisoned \(2024→\\rightarrowvulnerable\) = 5,000 total
- •Poisoning ratio: 40% \(chosen based on preliminary experiments showing lower ratios insufficient for small models\)
- •Evaluation: 500 trigger \+ 1,000 benign \+ 1,000 other \(no year context\) = 2,500 total

#### III\-B3Uniqueness Verification

We verify via hashing that\>\>95% of generated samples are unique, preventing the model from memorizing exact strings\.

### III\-CModel and Fine\-Tuning

#### III\-C1Base Model

We use SmolLM2\-360M\[[15](https://arxiv.org/html/2605.07324#bib.bib15)\], a compact but capable language model:

- •Hidden dimension: 960
- •Layers: 32
- •Attention heads: 15
- •Parameters: 360M

We chose this model for computational tractability while maintaining sufficient capacity to learn complex conditional behaviors\.

#### III\-C2LoRA Fine\-Tuning \(Regime A\)

Low\-Rank Adaptation\[[16](https://arxiv.org/html/2605.07324#bib.bib16)\]adds trainable low\-rank matrices to frozen pretrained weights:

W′=W\+B​AW^\{\\prime\}=W\+BA\(5\)whereB∈ℝd×rB\\in\\mathbb\{R\}^\{d\\times r\},A∈ℝr×dA\\in\\mathbb\{R\}^\{r\\times d\}, andr≪dr\\ll d\.

Our configuration:

- •Rank: 32
- •Alpha: 64
- •Target modules: q\_proj, k\_proj, v\_proj, o\_proj, gate\_proj, up\_proj, down\_proj \(all projection layers\)
- •Learning rate:3×10−43\\times 10^\{\-4\}
- •Epochs: 10
- •Batch size: 4
- •Gradient accumulation: 4 steps

#### III\-C3Full\-Rank Fine\-Tuning \(Regime B\)

All parameters are trainable, requiring careful hyperparameter selection:

- •Learning rate:1\.2×10−31\.2\\times 10^\{\-3\}
- •Epochs: 10
- •Batch size: 16
- •Gradient accumulation: 1 step
- •Gradient checkpointing: Enabled \(memory efficiency\)
- •Precision: FP32 \(stability\)

### III\-DInterpretability Architectures

We compare two sparse autoencoder architectures for analyzing the relationship between base and fine\-tuned activations\.

#### III\-D1Crosscoder Architecture

Crosscoders\[[4](https://arxiv.org/html/2605.07324#bib.bib4)\]learn shared features over concatenated activations:

Input:𝐚concat=\[𝐚base;𝐚ft\]∈ℝ2​d\\mathbf\{a\}\_\{\\text\{concat\}\}=\[\\mathbf\{a\}\_\{\\text\{base\}\};\\mathbf\{a\}\_\{\\text\{ft\}\}\]\\in\\mathbb\{R\}^\{2d\}

Encoder:

𝐟=ReLU​\(Wenc​𝐚concat\+𝐛enc\)\\mathbf\{f\}=\\text\{ReLU\}\(W\_\{\\text\{enc\}\}\\mathbf\{a\}\_\{\\text\{concat\}\}\+\\mathbf\{b\}\_\{\\text\{enc\}\}\)\(6\)whereWenc∈ℝm×2​dW\_\{\\text\{enc\}\}\\in\\mathbb\{R\}^\{m\\times 2d\},m=32​dm=32d\(expansion factor\)\.

Decoder:

𝐚^concat=Wdec​𝐟\+𝐛dec\\hat\{\\mathbf\{a\}\}\_\{\\text\{concat\}\}=W\_\{\\text\{dec\}\}\\mathbf\{f\}\+\\mathbf\{b\}\_\{\\text\{dec\}\}\(7\)
Training objective:

ℒCC=‖𝐚concat−𝐚^concat‖22\+λ​‖𝐟‖1\\mathcal\{L\}\_\{\\text\{CC\}\}=\\\|\\mathbf\{a\}\_\{\\text\{concat\}\}\-\\hat\{\\mathbf\{a\}\}\_\{\\text\{concat\}\}\\\|\_\{2\}^\{2\}\+\\lambda\\\|\\mathbf\{f\}\\\|\_\{1\}\(8\)
For SmolLM2\-360M withd=960d=960:

- •Input dimension: 1,920
- •Hidden features: 30,720 \(32×\\timesexpansion of single model\)

#### III\-D2Differential SAE Architecture

Diff\-SAE operates on activation differences:

Input:Δ​𝐚=𝐚ft−𝐚base∈ℝd\\Delta\\mathbf\{a\}=\\mathbf\{a\}\_\{\\text\{ft\}\}\-\\mathbf\{a\}\_\{\\text\{base\}\}\\in\\mathbb\{R\}^\{d\}

Encoder:

𝐟=ReLU​\(Wenc​Δ​𝐚\+𝐛enc\)\\mathbf\{f\}=\\text\{ReLU\}\(W\_\{\\text\{enc\}\}\\Delta\\mathbf\{a\}\+\\mathbf\{b\}\_\{\\text\{enc\}\}\)\(9\)
Decoder:

Δ​𝐚^=Wdec​𝐟\+𝐛dec\\widehat\{\\Delta\\mathbf\{a\}\}=W\_\{\\text\{dec\}\}\\mathbf\{f\}\+\\mathbf\{b\}\_\{\\text\{dec\}\}\(10\)
Training objective:

ℒDiff=‖Δ​𝐚−Δ​𝐚^‖22\+λ​‖𝐟‖1\\mathcal\{L\}\_\{\\text\{Diff\}\}=\\\|\\Delta\\mathbf\{a\}\-\\widehat\{\\Delta\\mathbf\{a\}\}\\\|\_\{2\}^\{2\}\+\\lambda\\\|\\mathbf\{f\}\\\|\_\{1\}\(11\)
We evaluate two expansion factors:

- •Diff\-SAE 32×\\times: 30,720 features
- •Diff\-SAE 4×\\times: 3,840 features

#### III\-D3Training Details

Both architectures share training hyperparameters:

- •Learning rate:10−410^\{\-4\}
- •L1 coefficient \(λ\\lambda\):10−410^\{\-4\}
- •Training tokens: 250,000
- •Batch size: 256
- •Optimizer: Adam

### III\-EActivation Extraction

#### III\-E1Layer Selection

We extract activations from the residual stream at layers \{14, 18, 22, 26\}, spanning middle to late transformer blocks\. This follows evidence that later layers encode more task\-specific information\[[17](https://arxiv.org/html/2605.07324#bib.bib17)\]\.

#### III\-E2Extraction Procedure

For each evaluation sample:

1. 1\.Tokenize input with context and prompt
2. 2\.Forward pass through base model, cache activations at target layers
3. 3\.Forward pass through fine\-tuned model, cache activations
4. 4\.Store paired activations\(𝐚base,𝐚ft\)\(\\mathbf\{a\}\_\{\\text\{base\}\},\\mathbf\{a\}\_\{\\text\{ft\}\}\)

We extract activations at all token positions, yielding 2,500 evaluation samples per layer\.

### III\-FBackdoor Isolation Score \(BIS\)

We introduce BIS to quantify how well a single feature isolates backdoor\-related activations\.

#### III\-F1Feature Activation

For each featureiiand samplejj, we compute activationfi​jf\_\{ij\}via the SAE encoder\. We define binary activation using the 95th percentile threshold:

activei​j=𝟙​\[fi​j\>τi\]\\text\{active\}\_\{ij\}=\\mathds\{1\}\[f\_\{ij\}\>\\tau\_\{i\}\]\(12\)whereτi=percentile95​\(\{fi​j\}j\)\\tau\_\{i\}=\\text\{percentile\}\_\{95\}\(\\\{f\_\{ij\}\\\}\_\{j\}\)\.

#### III\-F2Evaluation Metrics

For each featureii:

Precisioni\\displaystyle\\text\{Precision\}\_\{i\}=P​\(trigger∣activei\)\\displaystyle=P\(\\text\{trigger\}\\mid\\text\{active\}\_\{i\}\)\(13\)Recalli\\displaystyle\\text\{Recall\}\_\{i\}=P​\(activei∣trigger\)\\displaystyle=P\(\\text\{active\}\_\{i\}\\mid\\text\{trigger\}\)\(14\)FPRi\\displaystyle\\text\{FPR\}\_\{i\}=P​\(activei∣¬trigger\)\\displaystyle=P\(\\text\{active\}\_\{i\}\\mid\\neg\\text\{trigger\}\)\(15\)

#### III\-F3BIS Definition

We introduce BIS to quantify how well a single feature isolates backdoor\-related activations\. Intuitively, BIS measures how cleanly a single feature separates backdoor\-triggered activations from benign ones, rewarding high precision and recall while penalizing false positives\. We use the harmonic mean \(F1\) rather than the geometric mean to more strongly penalize imbalanced precision\-recall trade\-offs, and scale by\(1−FPR\)\(1\-\\text\{FPR\}\)to maintainBIS∈\[0,1\]\\text\{BIS\}\\in\[0,1\]\.

F​1i=2⋅Precisioni⋅RecalliPrecisioni\+RecalliF1\_\{i\}=\\frac\{2\\cdot\\text\{Precision\}\_\{i\}\\cdot\\text\{Recall\}\_\{i\}\}\{\\text\{Precision\}\_\{i\}\+\\text\{Recall\}\_\{i\}\}\(16\)
BISi=F​1i⋅\(1−FPRi\)\\text\{BIS\}\_\{i\}=F1\_\{i\}\\cdot\(1\-\\text\{FPR\}\_\{i\}\)\(17\)
TheF​1F1score balances precision and recall via the harmonic mean, while multiplying by\(1−FPR\)\(1\-\\text\{FPR\}\)penalizes false positives\. A perfect backdoor feature hasBIS=1\.0\\text\{BIS\}=1\.0\.

#### III\-F4Best Feature Selection

We report results for the feature with maximum BIS:

i∗=arg⁡maxi⁡BISii^\{\*\}=\\arg\\max\_\{i\}\\text\{BIS\}\_\{i\}\(18\)

#### III\-F5Statistical Inference

We compute 95% confidence intervals via bootstrap resampling:

1. 1\.Resample evaluation set with replacement
2. 2\.Recompute BIS for best feature
3. 3\.Repeat 500\-1,000 times
4. 4\.Report 2\.5th and 97\.5th percentiles

Algorithm[1](https://arxiv.org/html/2605.07324#alg1)summarizes the BIS computation\.

Algorithm 1Backdoor Isolation Score Computation0:Activations

\{fi​j\}\\\{f\_\{ij\}\\\}, trigger labels

\{yj\}\\\{y\_\{j\}\\\}
0:BIS score and best feature index

1:Compute threshold

τi=percentile95​\(\{fi​j\}j\)\\tau\_\{i\}=\\text\{percentile\}\_\{95\}\(\\\{f\_\{ij\}\\\}\_\{j\}\)for each

ii
2:Compute binary activations

activei​j=𝟙​\[fi​j\>τi\]\\text\{active\}\_\{ij\}=\\mathds\{1\}\[f\_\{ij\}\>\\tau\_\{i\}\]
3:foreach feature

iido

4:

TPi=∑jactivei​j⋅yj\\text\{TP\}\_\{i\}=\\sum\_\{j\}\\text\{active\}\_\{ij\}\\cdot y\_\{j\}
5:

Precisioni=TPi/∑jactivei​j\\text\{Precision\}\_\{i\}=\\text\{TP\}\_\{i\}/\\sum\_\{j\}\\text\{active\}\_\{ij\}
6:

Recalli=TPi/∑jyj\\text\{Recall\}\_\{i\}=\\text\{TP\}\_\{i\}/\\sum\_\{j\}y\_\{j\}
7:

FPRi=∑jactivei​j⋅\(1−yj\)/∑j\(1−yj\)\\text\{FPR\}\_\{i\}=\\sum\_\{j\}\\text\{active\}\_\{ij\}\\cdot\(1\-y\_\{j\}\)/\\sum\_\{j\}\(1\-y\_\{j\}\)
8:

F​1i=2⋅Precisioni⋅RecalliPrecisioni\+RecalliF1\_\{i\}=\\frac\{2\\cdot\\text\{Precision\}\_\{i\}\\cdot\\text\{Recall\}\_\{i\}\}\{\\text\{Precision\}\_\{i\}\+\\text\{Recall\}\_\{i\}\}
9:

BISi=F​1i⋅\(1−FPRi\)\\text\{BIS\}\_\{i\}=F1\_\{i\}\\cdot\(1\-\\text\{FPR\}\_\{i\}\)
10:endfor

11:

i∗=arg⁡maxi⁡BISii^\{\*\}=\\arg\\max\_\{i\}\\text\{BIS\}\_\{i\}
12:return

BISi∗,i∗\\text\{BIS\}\_\{i^\{\*\}\},i^\{\*\}

## IVExperiments

### IV\-AExperimental Setup

#### IV\-A1Hardware

All experiments were conducted on a single NVIDIA GPU with mixed\-precision training where applicable\.

#### IV\-A2Software

#### IV\-A3Reproducibility

All experiments use seed 42\. Bootstrap resampling uses 1,000 samples for LoRA and 500 samples for full\-rank experiments\.

### IV\-BBackdoor Implantation Verification

Before interpretability analysis, we verify that the backdoor was successfully learned by measuring vulnerability rates\.

#### IV\-B1Evaluation Protocol

We generate 5 code completions each for 2024 \(trigger\) and 2023 \(benign\) contexts, then analyze for SQL injection patterns using static analysis\.

#### IV\-B2Results

TABLE I:Backdoor Effectiveness VerificationBoth regimes successfully implant the backdoor\. Full\-rank achieves perfect separation \(\+100% delta\), while LoRA shows strong but imperfect separation \(\+60% delta\)\. The base model shows no year\-conditional behavior \(0% delta\)\.

### IV\-CPrimary Comparison: Crosscoder vs\. Diff\-SAE

Table[II](https://arxiv.org/html/2605.07324#S4.T2)presents the main comparison at layer 18 \(our primary analysis layer\):

TABLE II:BIS Comparison at Layer 18 \(Primary Layer\)#### IV\-C1Key Findings

1. 1\.Diff\-SAE dramatically outperforms Crosscoder: BIS of 0\.40 vs\.∼\\sim0\.01 represents a 40×\\timesimprovement\.
2. 2\.Crosscoder essentially fails: BIS near zero indicates the best Crosscoder feature performs barely better than random\.
3. 3\.Consistency across regimes: Both LoRA and full\-rank show identical Diff\-SAE performance\.
4. 4\.Statistical significance: Confidence intervals do not overlap \(p<0\.001p<0\.001\)\.

### IV\-DDetailed Metrics Analysis

Table[III](https://arxiv.org/html/2605.07324#S4.T3)provides precision, recall, and false positive rates for deeper analysis:

TABLE III:Detailed Metrics at Layer 18#### IV\-D1Critical Observation: Perfect Precision

Diff\-SAE achievesperfect precision \(1\.0\)andzero false positive rateacross both regimes\. This means:

- •When the best Diff\-SAE feature activates, italwaysindicates a backdoor trigger
- •There areno false alarmson benign inputs
- •The 0\.25 recall means 25% of trigger samples activate the feature above threshold

This is remarkable for a single feature among 30,720 candidates\. We note that the 95th\-percentile threshold guarantees 5% of samples are active for any feature; given 20% trigger prevalence in the evaluation set, a feature whose top activations concentrate entirely on trigger samples will mechanically yield precision of 1\.0, recall of 0\.25, and FPR of 0\.0\. The meaningful finding is that such concentration exists for Diff\-SAE features but not for Crosscoder features\.

### IV\-ELayer Ablation Study

We evaluate all methods across layers 14, 18, 22, and 26 to assess layer dependency\.

#### IV\-E1LoRA Regime Results

TABLE IV:BIS Across Layers \(LoRA Regime\)
#### IV\-E2Full\-Rank Regime Results

TABLE V:BIS Across Layers \(Full\-Rank Regime\)
#### IV\-E3Key Findings

1. 1\.Diff\-SAE maintains consistent performance: BIS remains 0\.39\-0\.40 across all layers and both regimes\.
2. 2\.Crosscoder shows interesting layer\-26 behavior: In full\-rank, Crosscoder achieves BIS = 0\.235 only at layer 26, suggesting some backdoor signal emerges in later layers\.
3. 3\.Full\-rank produces cleaner signals: Diff\-SAE achieves perfect 0\.400 at all layers for full\-rank, while LoRA shows slight variation \(0\.386\-0\.400\)\.
4. 4\.Practical implication: Any middle\-to\-late layer provides comparable detection capability\.

### IV\-FCrosscoder’s Partial Success at Layer 26

The emergence of Crosscoder signal at layer 26 \(full\-rank only\) warrants investigation:

TABLE VI:Crosscoder Detailed Metrics at Layer 26 \(Full\-Rank\)At layer 26, Crosscoder achieves non\-trivial precision \(0\.616\) but with notable false positives \(FPR = 0\.047\)\. This suggests:

- •Backdoor information partially emerges in Crosscoder’s joint representation at the final layers
- •Diff\-SAE still substantially outperforms \(0\.400 vs 0\.235\) with perfect precision
- •The full\-rank regime creates more detectable backdoor structure in late layers

### IV\-GExpansion Factor Analysis

We compare Diff\-SAE with 32×\\timesand 4×\\timesexpansion factors:

TABLE VII:Expansion Factor Comparison \(Layer 18\)#### IV\-G1Key Findings

1. 1\.Identical performance at layer 18: Both expansion factors achieve BIS = 0\.400 with perfect precision\.
2. 2\.8×\\timesparameter efficiency: The 4×\\timesmodel uses 8×\\timesfewer features while matching performance\.
3. 3\.Backdoor is low\-dimensional: The signal can be captured with relatively few features, suggesting it exists as a coherent direction\.

### IV\-HVariance Ratio Analysis

We measure how much of the activation variance is explained by the base\-to\-fine\-tuned difference:

TABLE VIII:Activation Variance Ratio by LayerThe variance ratio increases with layer depth, indicating larger activation differences in later layers\. Despite the small overall variance ratio \(0\.6\-5\.3%\), Diff\-SAE successfully isolates backdoor\-related features\.

### IV\-IBest Feature Analysis

Table[IX](https://arxiv.org/html/2605.07324#S4.T9)shows which features achieve optimal BIS:

TABLE IX:Best Feature Indices by Layer and RegimeDifferent features achieve optimal detection at different layers and regimes, but with identical BIS scores\. This suggests the backdoor information is encoded redundantly and can be isolated through multiple feature directions\.

## VDiscussion

### V\-AWhy Does Diff\-SAE Outperform Crosscoder?

Our results decisively contradict the hypothesis that Crosscoders’ joint representation would better capture fine\-tuning changes\. We propose several explanations:

#### V\-A1Backdoors as Directional Shifts

The backdoor manifests as a consistent directional shift in activation space:

𝐚ft=𝐚base\+𝟙trigger⋅𝐯backdoor\+ϵ\\mathbf\{a\}\_\{\\text\{ft\}\}=\\mathbf\{a\}\_\{\\text\{base\}\}\+\\mathds\{1\}\_\{\\text\{trigger\}\}\\cdot\\mathbf\{v\}\_\{\\text\{backdoor\}\}\+\\epsilon\(19\)where𝐯backdoor\\mathbf\{v\}\_\{\\text\{backdoor\}\}is the backdoor direction andϵ\\epsiloncaptures other fine\-tuning effects\.

Diff\-SAE directly models this difference:

Δ​𝐚≈𝟙trigger⋅𝐯backdoor\\Delta\\mathbf\{a\}\\approx\\mathds\{1\}\_\{\\text\{trigger\}\}\\cdot\\mathbf\{v\}\_\{\\text\{backdoor\}\}\(20\)making the backdoor signal dominant in its input\.

#### V\-A2Crosscoder’s Dilution Problem

Crosscoders learn features over\[𝐚base;𝐚ft\]∈ℝ2​d\[\\mathbf\{a\}\_\{\\text\{base\}\};\\mathbf\{a\}\_\{\\text\{ft\}\}\]\\in\\mathbb\{R\}^\{2d\}\. In this space, features must explain base model semantic content, fine\-tuned model semantic content, shared representations, and fine\-tuning\-induced changes including the backdoor\.

The backdoor signal competes with these other sources of variance\. Without explicit difference computation, sparse coding allocates features to more prominent patterns\.

#### V\-A3Information\-Theoretic Perspective

Consider the signal\-to\-noise ratio \(SNR\) for backdoor detection:

Diff\-SAE input: Predominantly backdoor\-related, as benign fine\-tuning effects are typically smaller or more distributed\.

Crosscoder input: Backdoor buried within full activation magnitudes, dominated by input semantics\.

Estimated SNR improvement: If backdoor accounts for 10% of‖Δ​𝐚‖\\\|\\Delta\\mathbf\{a\}\\\|but only 1% of‖𝐚concat‖\\\|\\mathbf\{a\}\_\{\\text\{concat\}\}\\\|, Diff\-SAE sees 10×\\timeshigher SNR\.

#### V\-A4Sparsity Penalty Considerations

Our comparison uses L1\-penalized crosscoders\.\[[18](https://arxiv.org/html/2605.07324#bib.bib18)\]showed that L1 shrinkage artifacts degrade crosscoder performance and that BatchTopK sparsity mechanisms substantially improve cross\-model feature recovery\. Our results therefore reflect a comparison against L1 crosscoders specifically; BatchTopK crosscoders may narrow the performance gap and represent an important direction for future investigation\.

### V\-BWhy Does Full\-Rank Create Cleaner Backdoor Signals?

An unexpected finding is that full\-rank fine\-tuning produces backdoor signals that are:

1. 1\.Behaviorally stronger: \+100% delta vs \+60% for LoRA
2. 2\.More uniformly detectable: Perfect 0\.400 BIS at all layers
3. 3\.Expansion\-independent: 4×\\timesequals 32×\\timesperformance

#### V\-B1Low\-Rank Constraint Effects

LoRA constrains weight updates to low\-rank subspaces \(Δ​W=B​A\\Delta W=BAwith rankr≪dr\\ll d\)\. This may force the backdoor to be encoded in a more distributed manner across the available rank, creating a slightly more complex signal\.

#### V\-B2Full\-Rank Directional Freedom

Full\-rank fine\-tuning can update weights in any direction, potentially allowing the optimizer to find a single, clean backdoor direction\. This explains both the stronger behavioral separation and simpler activation signature\.

### V\-CThe Perfect Precision Phenomenon

The most striking result is Diff\-SAE achievingperfect precision \(1\.0\) with zero false positives\. This means the backdoor creates a unique activation signature that never occurs in benign samples, allowing a single feature to perfectly discriminate trigger from non\-trigger when it activates, while the 25% recall indicates the feature captures a subset of backdoor activations\.

This has profound implications for practical deployment: a Diff\-SAE\-based monitor could flag suspicious activations withzero false alarm rate\. We note that these clean metrics are partly a consequence of the 95th\-percentile threshold interacting with the 20% trigger prevalence; nevertheless, the absence of any such concentration in Crosscoder features confirms a genuine architectural advantage for Diff\-SAE\.

### V\-DWhy Layer\-Independent Detection?

The consistent BIS across layers 14\-26 is informative:

#### V\-D1Residual Stream Hypothesis

In transformer architectures, the residual stream accumulates information across layers\. If the backdoor is “written” to the residual stream early \(e\.g\., when processing the year context\), it would persist through subsequent layers\.

#### V\-D2Redundant Encoding

For robustness, fine\-tuning may encode backdoor information redundantly across multiple layers, ensuring reliable triggering\.

#### V\-D3Practical Implication

Organizations monitoring for backdoors need not analyze all layers\. Any single middle\-to\-late layer provides equivalent detection capability\.

### V\-EImplications for AI Safety

#### V\-E1Detection Recommendations

Based on our findings:

1. 1\.Use Diff\-SAE over Crosscodersfor backdoor detection
2. 2\.Monitor activation differencesduring fine\-tuning pipelines
3. 3\.Single\-layer analysis suffices\(recommend layer 18 or similar middle layer\)
4. 4\.4×\\timesexpansion is sufficient—8×\\timesmore efficient than 32×\\times
5. 5\.Our experiments show zero false positives at the 95th\-percentilethreshold, though this should be validated across diverse settings before deployment reliance\.

#### V\-E2Monitoring Fine\-Tuning

Organizations can computeΔ​𝐚\\Delta\\mathbf\{a\}between checkpoints and flag unusual directional changes for review, enabling continuous monitoring without knowing specific triggers\.

#### V\-E3Limitations of Detection

While BIS = 0\.40 with perfect precision is strong, the 25% recall indicates that 75% of backdoor activations are not flagged by the best feature\. Ensemble approaches combining multiple features may improve recall, though adversarial backdoors might still evade single\-feature detection\.

### V\-FLimitations

#### V\-F1Model Scale

We evaluate on SmolLM2\-360M \(360M parameters\)\. Larger models \(7B\+\) may exhibit more distributed backdoor representations\. Minder et al\. \[18\] demonstrated Diff\-SAE effectiveness on Gemma\-2 2B, suggesting scalability, but backdoor detection specifically has not been validated at larger scales\.

#### V\-F2Backdoor Type

Our SQL injection backdoor is one specific instantiation\. Other backdoor types may behave differently:

- •Sentiment manipulation: May involve more distributed features
- •Topic\-triggered: Could activate different attention patterns
- •Multi\-step triggers: May require sequence\-level analysis

#### V\-F3Adversarial Robustness

An adversary aware of Diff\-SAE detection might design backdoors that:

- •Minimize activation differences while maintaining behavioral changes
- •Distribute the backdoor across many small, undetectable features
- •Use the same activation patterns as benign fine\-tuning

Future work should evaluate adversarial robustness\.

## VIConclusion

We present the first systematic comparison of Crosscoders and Differential SAEs for backdoor detection in fine\-tuned language models\. Using a controlled SQL injection backdoor in SmolLM2\-360M, we find thatDiff\-SAE consistently and dramatically outperforms Crosscoders:

1. 1\.40×\\timeshigher BIS: 0\.40 vs∼\\sim0\.01
2. 2\.Perfect precision: 1\.0 with zero false positives
3. 3\.Layer\-independent: Consistent across layers 14\-26
4. 4\.Regime\-independent: Works for both LoRA and full\-rank
5. 5\.Efficient: 4×\\timesexpansion matches 32×\\timesperformance
6. 6\.Full\-rank cleaner: Perfect performance at all layers

These findings complement recent work questioning L1 crosscoders’ effectiveness for capturing fine\-tuning changes\[[18](https://arxiv.org/html/2605.07324#bib.bib18)\], and extend these observations to the backdoor detection setting and provide actionable guidance for AI safety practitioners\. The mechanistic insight that backdoors manifest as directional activation shifts explains why difference\-based representations are fundamentally more effective\.

### VI\-AFuture Work

- •Scale evaluation to larger models \(7B\+\)
- •Evaluate diverse backdoor types
- •Develop adversarially robust detection
- •Explore ensemble methods to improve recall
- •Combine Diff\-SAE with surgical backdoor removal
- •Theoretical analysis of backdoor geometry
- •Evaluate alternative sparsity mechanisms \(BatchTopK\) for crosscoders to determine whether the performance gap narrows

## Acknowledgment

We thank the open\-source community for SmolLM2 and the Anthropic interpretability team for foundational SAE research\.

## References

- \[1\]E\. Hubinger, C\. Denison, J\. Mu, M\. Lambert, M\. Tong, M\. MacDiarmid, et al\., “Sleeper agents: Training deceptive LLMs that persist through safety training,” arXiv preprint arXiv:2401\.05566, 2024\.
- \[2\]H\. Cunningham, A\. Ewart, L\. Riggs, R\. Huben, and L\. Sharkey, “Sparse autoencoders find highly interpretable features in language models,” arXiv preprint arXiv:2309\.08600, 2023\.
- \[3\]Bricken, et al\., ”Towards Monosemanticity: Decomposing Language Models With Dictionary Learning”, Transformer Circuits Thread, 2023\.
- \[4\]J\. Lindsey, W\. Gurnee, E\. Ameisen, B\. Chen, A\. Pearce, A\. Templeton, et al\., “Crosscoders: Sparse autoencoders for cross\-model feature analysis,” Transformer Circuits Thread, 2024\.
- \[5\]J\. Dai, C\. Chen, and Y\. Li, “A backdoor attack against LSTM\-based text classification systems,” IEEE Access, vol\. 7, pp\. 138872–138878, 2019\.
- \[6\]X\. Chen, A\. Salem, A\. N\. Bhagoji, M\. Backes, and S\. Gong, “BadNL: Backdoor attacks against NLP models with semantic\-preserving improvements,” arXiv preprint arXiv:2006\.01043, 2021\.
- \[7\]A\. Templeton, T\. Conerly, J\. Marcus, J\. Lindsey, T\. Bricken, B\. Chen, et al\., “Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet,” Transformer Circuits Thread, 2024\.
- \[8\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, et al\., “Training language models to follow instructions with human feedback,” NeurIPS, 2022\.
- \[9\]Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, et al\., “Constitutional AI: Harmlessness from AI feedback,” arXiv preprint arXiv:2212\.08073, 2022\.
- \[10\]T\. Gu, B\. Dolan\-Gavitt, and S\. Garg, “BadNets: Identifying vulnerabilities in the machine learning model supply chain,” arXiv preprint arXiv:1708\.06733, 2017\.
- \[11\]Y\. Qi, S\. Xie, and Y\. Li, “ONION: A simple and effective defense against textual backdoor attacks,” EMNLP, 2021\.
- \[12\]K\. Liu, B\. Dolan\-Gavitt, and S\. Garg, “Fine\-pruning: Defending against backdooring attacks on deep neural networks,” RAID 2018\. Lecture Notes in Computer Science\(\), vol 11050\. Springer, 2018\.
- \[13\]B\. Wang, Y\. Yao, S\. Shan, H\. Li, B\. Viswanath, H\. Zheng, and B\. Y\. Zhao, “Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,” Symposium on Security and Privacy \(SP\), 2019\.
- \[14\]K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov, “Locating and editing factual associations in GPT,” NeurIPS, 2022\.
- \[15\]HuggingFace, “SmolLM2: Compact language models,” 2024\. \[Online\]\. Available: https://huggingface\.co/HuggingFaceTB/SmolLM2\-360M
- \[16\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, et al\., “LoRA: Low\-rank adaptation of large language models,” ICLR, 2022\.
- \[17\]N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, et al\., “Softmax linear units,” Transformer Circuits Thread, 2022\.
- \[18\]Minder, J\. et al\., ”Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat\-Tuning,” arXiv preprint arXiv:2504\.02922, 2026\.