Tag
This paper applies mechanistic interpretability tools—including linear CKA, logit lens, attention decomposition, sparse autoencoders, and activation patching—to compare GPT-2 with DistilGPT-2, revealing a compression-collapse dichotomy where the IOI circuit fully collapses while the Greater-Than circuit survives via layer remapping.
该论文研究语言模型在处理花园路径句后是否会保留被放弃的句法分析残余,发现残余效应主要储存在名词短语的状态中,并可通过激活修补(activation patching)在32B和70B模型中近乎完全消除该残余。
The paper proposes Guided-AP, a method that uses a few exact activation patching measurements to reveal and correct shared errors in attribution patching estimates, reducing error by up to 26% on single prompts and 75-79% for average effects across prompts in mechanistic interpretability analyses.
提出了Self-targeted Internal Attention Boosting(SIAB),通过因果定位与激活修补在模型内部定位稀疏的情感注意力头并增强其内容词注意力路由,从而在不依赖外部知识或参数更新的情况下提升LLM多轮对话中的情感状态推理,在ECC和GoEmotions基准上显著提高宏F1分数。
该论文研究代码语言模型中对 Python 的默认偏好是否由可定位的因果电路产生,通过 55 个模型-语言设置的消融、激活修补和电路提取实验发现,偏好是否显得局部化取决于探针选择与组件预算,并呼吁报告分析选择。
The paper investigates neurons in frozen BERT that drive AI-text detection using sparse probing and activation patching on the RAID benchmark, identifying a small set of causally relevant neurons that generalize across generator families.
This paper uses activation patching to causally measure if reasoning steps in chain-of-thought are load-bearing, finding that behavioral tests overestimate faithfulness and larger models like Qwen3-4B maintain better faithfulness across reasoning depths.
This paper investigates how vision-language models read exact values from vertical bar charts using controlled counterfactual activation patching, revealing insights into internal computations and differences between models like Qwen and InternVL.
This paper uses mechanistic interpretability to analyze how LLaMA 3.1 8B models numerical sequences, revealing that it internally computes and stores first differences to extrapolate patterns.
This research investigates how multilingual large language models internally handle subject-verb agreement across languages, finding that models reuse shared computational structure for languages with overt inflection, indicating cross-lingual overlap in morphosyntactic processing.
This paper argues that LLM failures on hidden-constraint reasoning are routing problems, not knowledge problems, and introduces a quartet diagnostic to dissociate knowledge, symmetry, routing, and repair across 14 models, with activation probing and patching experiments.
This paper investigates how language models execute in-context conditional rules by probing whether testing and routing are separable mechanisms. Using activation patching across three open models and six languages, the authors find that predicate testing is modular while route representations are token-bound and non-transferable.
This paper presents a graph-to-answer mechanism-tracing case study for Graph-PRefLexOR-8B, a materials-science hypothesis generation model, using visualization and activation-based diagnostics to localize where mechanism support is lost or recovered during generation.
This paper tests whether increasing interventional data in pretraining improves LLMs' causal direction reasoning, using controlled Simpson's-paradox worlds. It finds that the training mixture does not govern interventional evidence use; instead the evidence type in the inference-time context is the decisive factor.
This paper presents a theoretical cost accounting for exhaustive sweeps and sequential mutations on reactive computational graphs, deriving speedup ratios and validating them on a Julia-based graph engine.
This paper investigates whether arithmetic heuristic neurons in LLMs are form-invariant across symbolic arithmetic, natural language word problems, and Python code. Using activation patching, they find a shared circuit of neurons that is necessary and sufficient for arithmetic computation, and that cross-format failures arise from activation states rather than distinct circuits.
This paper investigates how vision-language models resolve conflicts between visual evidence and world knowledge, revealing that visual grounding is the default while prior knowledge depends on a small set of late-layer attention heads. The authors perform causal analysis across three VLM families, demonstrating an asymmetric structure where ablating these heads shifts predictions from knowledge-grounded to visually grounded answers.
This paper re-derives activation patching from causal mediation analysis, revealing that the natural indirect effect (NIE) captures not only a component's causal effect but also interaction effects with other components. It demonstrates these hidden interactions in the GPT-2 IOI circuit and argues that they are a diagnostic tool rather than a nuisance.
This paper investigates why instruction-tuned language models give different answers to causal reasoning questions when variable names are replaced with placeholders, finding that the issue stems from representational misalignment rather than information loss. The authors introduce Vernier, a method using paired-view weight updates and mechanism inspection to reveal that answer-relevant content is still present in the placeholder view but misaligned.
The paper proposes the Unlearning Depth Score (UDS), a metric that uses activation patching to quantify how thoroughly target knowledge is erased from LLMs, achieving state-of-the-art faithfulness and robustness across multiple unlearning methods.