Tag
This paper investigates whether arithmetic heuristic neurons in LLMs are form-invariant across symbolic arithmetic, natural language word problems, and Python code. Using activation patching, they find a shared circuit of neurons that is necessary and sufficient for arithmetic computation, and that cross-format failures arise from activation states rather than distinct circuits.
This paper diagnoses systematic errors in attribution patching, a gradient-based approximation used for causal localization in language models, and proposes a second-order correction using Hessian-vector products that improves reliability with minimal additional computational cost.