activation-patching

Tag

Cards List
#activation-patching

What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2

arXiv cs.CL ↗ · 2026-10-28 Cached

This paper applies mechanistic interpretability tools—including linear CKA, logit lens, attention decomposition, sparse autoencoders, and activation patching—to compare GPT-2 with DistilGPT-2, revealing a compression-collapse dichotomy where the IOI circuit fully collapses while the Greater-Than circuit survives via layer remapping.

0 favorites 0 likes
#activation-patching

Language Models Keep Abandoned Analyses: Residue and Adaptation after Garden-Path Sentences

arXiv cs.CL ↗ · 2026-10-28 Cached

该论文研究语言模型在处理花园路径句后是否会保留被放弃的句法分析残余,发现残余效应主要储存在名词短语的状态中,并可通过激活修补(activation patching)在32B和70B模型中近乎完全消除该残余。

0 favorites 0 likes
#activation-patching

Correcting Attribution Patching with a Few Exact Measurements

arXiv cs.AI ↗ · 2026-10-21 Cached

The paper proposes Guided-AP, a method that uses a few exact activation patching measurements to reveal and correct shared errors in attribution patching estimates, reducing error by up to 26% on single prompts and 75-79% for average effects across prompts in mechanistic interpretability analyses.

0 favorites 0 likes
#activation-patching

Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning

arXiv cs.CL ↗ · 2026-10-16 Cached

提出了Self-targeted Internal Attention Boosting(SIAB),通过因果定位与激活修补在模型内部定位稀疏的情感注意力头并增强其内容词注意力路由,从而在不依赖外部知识或参数更新的情况下提升LLM多轮对话中的情感状态推理,在ECC和GoEmotions基准上显著提高宏F1分数。

0 favorites 0 likes
#activation-patching

Causal Syntax Circuits in CodeLMs: Budget and Probe Decide What Looks Localised

arXiv cs.LG ↗ · 2026-10-16 Cached

该论文研究代码语言模型中对 Python 的默认偏好是否由可定位的因果电路产生,通过 55 个模型-语言设置的消融、激活修补和电路提取实验发现,偏好是否显得局部化取决于探针选择与组件预算,并呼吁报告分析选择。

0 favorites 0 likes
#activation-patching

A Mechanistic Study of AI-Text Detection Neurons in Frozen BERT: Sparse Probing and Activation Patching on RAID

arXiv cs.CL ↗ · 2026-09-28 Cached

The paper investigates neurons in frozen BERT that drive AI-text detection using sparse probing and activation patching on the RAID benchmark, identifying a small set of causally relevant neurons that generalize across generator families.

0 favorites 0 likes
#activation-patching

Are Stated Reasoning Steps Causally Load-Bearing?

arXiv cs.AI ↗ · 2026-09-24 Cached

This paper uses activation patching to causally measure if reasoning steps in chain-of-thought are load-bearing, finding that behavioral tests overestimate faithfulness and larger models like Qwen3-4B maintain better faithfulness across reasoning depths.

0 favorites 0 likes
#activation-patching

Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth

arXiv cs.CL ↗ · 2026-09-15 Cached

This paper investigates how vision-language models read exact values from vertical bar charts using controlled counterfactual activation patching, revealing insights into internal computations and differences between models like Qwen and InternVL.

0 favorites 0 likes
#activation-patching

Mechanistic Interpretability of Structure-Aware Numerical Reasoning in LLaMA 3.1 8B

arXiv cs.LG ↗ · 2026-08-20 Cached

This paper uses mechanistic interpretability to analyze how LLaMA 3.1 8B models numerical sequences, revealing that it internally computes and stores first differences to extrapolate patterns.

0 favorites 0 likes
#activation-patching

Shared Circuits for Shared Grammar: Tracing Subject-Verb Agreement Across Languages

arXiv cs.CL ↗ · 2026-08-20 Cached

This research investigates how multilingual large language models internally handle subject-verb agreement across languages, finding that models reuse shared computational structure for languages with overt inflection, indicating cross-lingual overlap in morphosyntactic processing.

0 favorites 0 likes
#activation-patching

LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning

arXiv cs.CL ↗ · 2026-08-14 Cached

This paper argues that LLM failures on hidden-constraint reasoning are routing problems, not knowledge problems, and introduces a quartet diagnostic to dissociate knowledge, symmetry, routing, and repair across 14 models, with activation probing and patching experiments.

0 favorites 0 likes
#activation-patching

Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages

arXiv cs.CL ↗ · 2026-08-06 Cached

This paper investigates how language models execute in-context conditional rules by probing whether testing and routing are separable mechanisms. Using activation patching across three open models and six languages, the authors find that predicate testing is modular while route representations are token-bound and non-transferable.

0 favorites 0 likes
#activation-patching

Visualizing Graph-to-Answer Mechanism Recovery in Materials-Science Hypothesis Generation

arXiv cs.CL ↗ · 2026-08-06 Cached

This paper presents a graph-to-answer mechanism-tracing case study for Graph-PRefLexOR-8B, a materials-science hypothesis generation model, using visualization and activation-based diagnostics to localize where mechanism support is lost or recovered during generation.

0 favorites 0 likes
#activation-patching

Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?

arXiv cs.CL ↗ · 2026-08-03 Cached

This paper tests whether increasing interventional data in pretraining improves LLMs' causal direction reasoning, using controlled Simpson's-paradox worlds. It finds that the training mixture does not govern interventional evidence use; instead the evidence type in the inference-time context is the decisive factor.

0 favorites 0 likes
#activation-patching

Cost Accounting for Reactive Computational Graphs: Exhaustive Sweeps, Sequential Mutation, and the Backward-Locality Gap

arXiv cs.LG ↗ · 2026-07-22 Cached

This paper presents a theoretical cost accounting for exhaustive sweeps and sequential mutations on reactive computational graphs, deriving speedup ratios and validating them on a Julia-based graph engine.

0 favorites 0 likes
#activation-patching

Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs

arXiv cs.CL ↗ · 2026-07-21 Cached

This paper investigates whether arithmetic heuristic neurons in LLMs are form-invariant across symbolic arithmetic, natural language word problems, and Python code. Using activation patching, they find a shared circuit of neurons that is necessary and sufficient for arithmetic computation, and that cross-format failures arise from activation states rather than distinct circuits.

0 favorites 0 likes
#activation-patching

Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models

arXiv cs.CL ↗ · 2026-06-29 Cached

This paper investigates how vision-language models resolve conflicts between visual evidence and world knowledge, revealing that visual grounding is the default while prior knowledge depends on a small set of late-layer attention heads. The authors perform causal analysis across three VLM families, demonstrating an asymmetric structure where ablating these heads shifts predictions from knowledge-grounded to visually grounded answers.

0 favorites 0 likes
#activation-patching

The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching

arXiv cs.LG ↗ · 2026-06-29 Cached

This paper re-derives activation patching from causal mediation analysis, revealing that the natural indirect effect (NIE) captures not only a component's causal effect but also interaction effects with other components. It demonstrates these hidden interactions in the GPT-2 IOI circuit and argues that they are a diagnostic tool rather than a nuisance.

0 favorites 0 likes
#activation-patching

Vernier: Probing Representational Misalignment Behind Lexical Gaps in Causal Reasoning

arXiv cs.CL ↗ · 2026-06-16 Cached

This paper investigates why instruction-tuned language models give different answers to causal reasoning questions when variable names are replaced with placeholders, finding that the issue stems from representational misalignment rather than information loss. The authors introduce Vernier, a method using paired-view weight updates and mechanism inspection to reveal that answer-relevant content is still present in the placeholder view but misaligned.

0 favorites 0 likes
#activation-patching

Measuring the Depth of LLM Unlearning via Activation Patching

arXiv cs.CL ↗ · 2026-05-26 Cached

The paper proposes the Unlearning Depth Score (UDS), a metric that uses activation patching to quantify how thoroughly target knowledge is erased from LLMs, achieving state-of-the-art faithfulness and robustness across multiple unlearning methods.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback