Tag
The paper introduces CircuitTree, a tree-compose data structure and greedy density-scaled algorithm for composing many pruned circuits from HuBERT into a syllable transcriber, achieving 64.8% open-syllable accuracy with 99.5% shared circuitry versus a 66.0% non-sharing ceiling.
This paper introduces LAWFUL, a framework for verifying whether neural networks learn and causally use physical laws over continuous variables, addressing gaps in coverage-aware causal consistency and domain-of-validity testing. It demonstrates the approach on a Mocap2Radar transformer and the Doppler frequency law.
This paper investigates whether large language models have a localized causal mechanism for handling the animacy concept, using circuit discovery on minimal pairs; they find an animacy circuit that is distributed and only partially generalizes.
A researcher shares an experimental plan for identifying causal dependencies between capability dimensions in a 31B model using contrastive targeted SFT and circuit tracing, seeking feedback on methodology and related work.
This paper diagnoses systematic errors in attribution patching, a gradient-based approximation used for causal localization in language models, and proposes a second-order correction using Hessian-vector products that improves reliability with minimal additional computational cost.
Proposes MechRL, a reinforcement learning approach to automate circuit discovery in transformer language models. A PPO agent trained on multiple tasks discovers attention head circuits that match known canonical circuits and generalizes to a held-out task.
Introduces a three-step recipe for identifying attention-head circuits in pretrained transformers using a spectral signal and task-pattern screen without requiring labels, validated across 51M to 1B parameter models and multiple architectures.
Researchers introduce PIE, a CLT-native framework for efficient circuit discovery via feature attribution-based pruning, achieving ~40× compression in feature selection while maintaining behavioral fidelity on IOI and Doc-String tasks.