causal-interventions

The paper shows that small autoregressive transformers trained only on restricted parameter regimes of nonlinear dynamical systems (logistic/sine maps, Lorenz, Hopf) can extrapolate global dynamics far outside the training distribution, reproducing period-doubling cascades up to period 128 and the Feigenbaum constant to within 5e-4, with causal interventions revealing how control-parameter information flows through attention.

The paper demonstrates that transformers can have faithful internal world models, using mechanistic analysis to show that failures stem from feature interference rather than incoherent maps, and proposes affordance packing to enhance performance.

The paper finds that in language models, relation-type information becomes generation-controlling before entity-specific information during factual recall, indicating a temporal asymmetry that is robust across various models and prompt families.

The paper introduces Causal Visual Recurrent Reasoning (CVRR), a method that enforces recurrent hidden-state computation for visual reasoning, improving performance on benchmarks while distinguishing latent informativeness from actual predictive use.

This paper uses causal interventions to investigate syntactic mechanisms in multilingual language models, revealing cross-lingual transfer that is graded based on typological similarity.

This paper demonstrates that materials science mechanisms in the open-weight Gemma model can be identified and controlled through hidden state analysis, causal interventions, and counterfactual benchmarks, revealing that physical relationships are more visible in state transformations than in absolute states.