mechanistic-interpretability

This paper uses activation patching to causally measure if reasoning steps in chain-of-thought are load-bearing, finding that behavioral tests overestimate faithfulness and larger models like Qwen3-4B maintain better faithfulness across reasoning depths.

This paper provides a mechanistic comparison of knowledge-conflict circuits in LLMs under instruction tuning, finding that tuning gates rather than rewires these circuits across multiple model families, with implications for interpretability.

The paper introduces Matryoshka Attribution (MAttr), a mask-learning method for attributing language model outputs to internal components, achieving top performance on the Mechanistic Interpretability Benchmark and demonstrating practical use in modifying LLM behaviors by adjusting weights.

This paper provides a mechanistic analysis of Hierarchical Reasoning Models (HRM) to understand their internal reasoning processes in latent space, using techniques like causal interventions and sparse autoencoders on tasks such as Sudoku and ARC-AGI-2.

The paper introduces MechaTerp-TRACE, a method for component ablation analysis in language models, finding that entity knowledge is largely attributable to generic generation machinery rather than localized components.

This paper reveals that the optimal layer for linear probing to read concepts differs from the optimal layer for activation steering in omni-modal large language models, challenging common heuristics in representation engineering.

The article explores how the AI industry might have paused development if it followed its own research on catastrophic risks, highlighting Anthropic's findings on AI deception and the challenges of mechanistic interpretability.

This paper proposes a four-stage mechanistic decomposition of how large language models solve math word problems, localizing fragility from irrelevant clauses to the Operation Planning stage via specific attention heads.

The paper finds that in language models, relation-type information becomes generation-controlling before entity-specific information during factual recall, indicating a temporal asymmetry that is robust across various models and prompt families.

The tweet discusses key questions in AI interpretability, specifically the advancement of methods for decoding neural network activations into human-readable language.

Decoy Direction Optimization (DDO) is a fast, post-hoc defense method that protects open-weight LLMs from refusal feature ablation attacks by injecting decoy signals into the network, achieving high robustness at lower cost than trained defenses.

This paper demonstrates that ordinary typos in user inputs can significantly reduce the effectiveness of activation probes designed to detect prompt injection in LLMs, but introduces a KV-cache fork method that recovers most of the performance loss.

The paper identifies Harmfulness Propagation Dynamics in large language models and introduces Herald, a lightweight input moderator that uses cross-layer activation patterns to detect harmful prompts efficiently.

This paper presents a mathematical framework for reverse-engineering transformer models to enhance mechanistic interpretability and address safety concerns in AI systems.

This paper introduces methods to calibrate confidence in agentic systems using internal representations, demonstrating improved performance over baselines in multi-turn benchmarks.

Capsule Lens is a framework for mechanistic interpretability that uses geometric capsules to locate and track how concepts are encoded in neural network representations, enabling analysis of both static and dynamic model internals.

This paper introduces a contrastive projection technique to read transformer internal states by differencing logit lenses from paired prompts, enabling the identification of computation-specific differences across architectures like Phi-2.

This paper introduces SAEScientist-Bench, a benchmark to evaluate AI agents' ability to autonomously conduct mechanistic interpretability research using sparse autoencoders, revealing progress but significant gaps compared to expert baselines.

ObserverBench is a benchmark framework designed to evaluate whether internal estimators in AI models are adequate for guiding interventions, control, or safety tasks by reporting both estimation accuracy and downstream action performance.

This paper uses mechanistic interpretability on Gemma-3-27B-PT to extract and align symptom vectors for depression with clinician judgments, demonstrating potential for interpretable clinical assessment tools.