Tag
该论文通过行为分析、表示探查和因果干预，研究了AM、POMO和LEHD等神经组合优化（NCO）路由求解器的内部决策机制，揭示了不同架构在构造解决方案时的差异化模式，例如LEHD依赖当前节点表示进行局部决策、起始节点提供全局导航参考。
The paper presents a structural circuit analysis to identify first-token broadcasters and language-identity heads in multilingual language models, revealing consistent hub topologies and that routing circuitry is primarily established during pre-training.
This paper introduces LLM endognostics, a white-box framework for auditing latent knowledge in language models, showing that behavioral evaluations can fail to reflect internal knowledge distinctions, as demonstrated on Minerva-7B-Instruct-v1.0.
This paper demonstrates that transformers encode a dialogue partner's expertise in early layers but activate it causally only in later layers, revealing a gap that constrains intervention points for steering model behavior in multi-turn dialogues.
This paper introduces a calibrated test of internal action maps in language models, showing that state signals can be decodable and causally usable without global affine closure, using an evidence lattice framework validated on finite worlds and the Qwen3-4B model.
This paper investigates whether language identity in LLMs is linearly decodable and causally controllable via compact activation directions. Through steering and ablation experiments across multiple model families, the authors show that language selection is direction-dependent, layer-specific, and reverts to English when the language signal is ablated.
This paper introduces M2BIND, a benchmark to evaluate whether vision-language models maintain stable visual-linguistic associations across languages. It finds that binding is not language-invariant, with cross-family and cross-script settings causing significant performance collapse and weaker internal causal binding.
This paper investigates when large language models develop domain-specific parametric shells (causally necessary neuron populations), finding that modular training data at the token level (e.g., languages, code) produces functional shells, while academic subject domains do not, despite being linearly decodable.
Introduces Mechanist, an autonomous agentic system that uses AI to discover and control the mechanisms underlying model intelligence, generating hypotheses, performing causal interventions, and improving safety and performance.
CausalGate introduces a method that uses causal interventions to measure the importance of transformer sub-layers and distills this into static scalar gates for efficient inference without runtime overhead, outperforming existing pruning and routing methods.
This paper investigates short-term attention degradation in LLMs, finding a universal exponential-then-plateau pattern and that function token anchoring is architecture-dependent. Causal tests show that increasing attention mass on function tokens does not improve retrieval, suggesting attention degradation is descriptive rather than prescriptive.
This paper introduces Visual Access Sweep, a causal intervention method to measure the minimal image-token access needed for Vision-Language Model reasoning, and finds that Chain-of-Thought prompting does not primarily improve performance by prolonging direct image access but by enabling extended language-side computation over visual information.
This paper investigates how the faithfulness of latent reasoning steps evolves during training, finding that it depends on training stage and answer format, rather than just final checkpoint performance.
This paper investigates token-level attention shifts in multimodal large language models during generation, revealing consistent patterns and proposing a simple test-time intervention that significantly improves task performance.
This paper reveals that hallucination in large vision-language models is caused by a dynamic structural misalignment where certain attention heads act as risky mediators, decoupling from visual evidence to lock onto language priors. The authors propose Fox, a training-free causal intervention framework that diagnoses and physically severs these pathological shortcuts, achieving state-of-the-art performance in faithful decoding.
This paper demonstrates that the weight norm causally controls the timescale of grokking in neural networks, reconciling conflicting accounts. Through interventions, it shows that grokking follows an exponential delay law and that norm magnitude dominates grokking time over learning rate across architectures.
This paper identifies imbalanced attention head groups in MLLMs that drive or resist modality-conflict hallucination, and proposes MACI, a causal intervention that suppresses hallucination-driving heads only when conflict is detected, achieving large hallucination reduction across five models.