causal-analysis

This paper shows that models adapting on their own generated text during inference destabilize over long (128K-token) horizons, and proposes a causal decomposition plus a 'Settlement' validation mechanism that tests candidate updates on independent text before retaining them, substantially reducing damage while preserving genuine adaptation.

研究利用功能ANOVA分解，揭示了LLM在组合任务中跨层的因果与几何表示机制：中间层使用两token联合关系表示，后期层转向三token联合表示，且限制模型仅关注因果相关表示反而提升了下一token预测准确率。

This paper introduces WebMRE, an offline benchmark for multimodal web agents, and studies the mutual reinforcement effect between guide sentences and actions, demonstrating performance improvements and providing causal analysis of the guide as a causal channel.

This paper uses activation patching to causally measure if reasoning steps in chain-of-thought are load-bearing, finding that behavioral tests overestimate faithfulness and larger models like Qwen3-4B maintain better faithfulness across reasoning depths.

This paper investigates subliminal learning in language models by measuring causal depth and multi-token confounds to understand how traits are transferred through apparently unrelated outputs.

This paper analyzes spurious onsets in full-duplex speech LLMs like Moshi and PersonaPlex and introduces an inference-time mitigation method based on causal analysis to suppress them without retraining.

The paper proposes HEAL, a method to mitigate hallucinations in Multimodal Large Language Models by analyzing and calibrating information distribution in synergy heads through causal interventions and counterfactual analysis.

This paper proposes a causal analysis framework to identify biases in time series foundation models, applied to Chronos-2 and TimesFM-2.5, revealing specific failure modes like overestimation of persistence and failures against regime switch patterns.

The paper proposes a two-stage mechanism for subliminal trait transfer in AI models, where optimizer state transports source perturbations and later training determines their behavioral value.

The paper introduces AgenticRAG-FP, an interventional benchmark for causal failure attribution in agentic RAG systems, analyzing how failures propagate across retrieval hops and evaluating diagnosis performance.

This paper audits step-level credit assignment in LLM agents using causal ground truth from executed replay, finding that common credit signals fail to identify causally important steps better than chance, with implications for training methods.

This paper studies how 4-bit KV-cache quantization in Mixture-of-Experts models causes route flips, and finds that detecting a flip is possible but predicting whether the flip is harmful is not, proposing a causal measurement apparatus rather than a mitigation.

This paper investigates whether single-token sparse autoencoder features are causally necessary for model outputs, finding that causal roles vary across SAE families and are sensitive to training methodology rather than activation function or scale.

This paper investigates how text template tokens in diffusion transformers (DiTs) act as implicit semantic registers that causally maintain object identity during denoising, and proposes a training-free pruning rule that removes 20% of attention FLOPs with minimal performance drop.

This paper analyzes latent reasoning models (LRMs) and demonstrates that observable patterns in latent states are not causal explanations of reasoning; it advocates for matched controls and causal tests in interpretability research.

ORCA is a copilot for end-to-end causal analysis that uses agents to guide users through workflows including causal discovery, effect estimation, and root cause analysis, with structured reports.

This paper proposes a five-stage methodology for causal feature analysis in transformer language models, demonstrated on GPT-2 small for the IOI task. It finds that features are specifically causal but not necessary, and exposes a gap between detection and causal robustness.

This paper identifies a 'Diagnostic Paradox' in multi-module LLM agents: the module most causally responsible for failures (the routing module) is not the best place to intervene, and patching it can harm performance. The authors propose the 'Linguistic Contract' hypothesis and present empirical evidence across three agent families.

This paper investigates the internal mechanisms of LLM-as-a-judge, finding a shared Latent Evaluator sub-graph in mid-to-late MLPs across models that handles abstract judging, while format-specific terminal branches map the judgment to output tokens, revealing the cause of format-induced inconsistency.

This paper presents causal evidence that hallucination in autoregressive language models results from early trajectory commitment governed by asymmetric attractor dynamics, using same-prompt bifurcation and activation patching experiments on Qwen2.5-1.5B to show that hallucinated trajectories diverge at the first token and exhibit strong causal asymmetry across model layers.