superposition

Goodfire 研究者发文论证，此前被广泛接受的神经元多义性与叠加假说是对 InceptionV1 证据的误读；经架构改进后的模型 99-100% 末层神经元已可解释，主张网络本身就是可读的构建块，无需先用稀疏自编码器解缠。

This paper demonstrates that Transformers in LLMs exhibit linear superposition when combining inputs, supporting the Superposition Linearity Hypothesis, and introduces a guided decoding method to disentangle superposed outputs.

This paper investigates the efficiency of superposition in latent states for reasoning in large language models, demonstrating that cumulative superposition can retain more usable evidence than frontier superposition, optimizing memory for downstream computation.

A team of physicists built a new interferometer to measure how gravity affects quantum wave phases by putting atoms in a superposition of free-fall and stationary paths, addressing long-standing questions about the compatibility of quantum mechanics and general relativity.

This paper audits sparse autoencoder features using causal interventions, finding that up to 77% of correlationally recovered features in degraded SAEs and 9% in well-trained ones are causally inert. The authors introduce the sae-causal-audit tool for reproducible evaluation.

Toy Models of Superposition by Elhage et al. explains why interpretability is hard: models represent more features than dimensions via superposition, leading to polysemantic neurons as compression. This paper spawned the sparse autoencoder research program.

This paper introduces sparse autoencoders to resolve superposition in neural networks, improving interpretability and geometric fidelity of latent spaces, and presents GW-map for cross-modal alignment between image representations and single-cell RNA sequencing data.

Lecture notes on the foundations of quantum machine learning, covering qubits, superposition, measurement, and the Bloch sphere.

This paper provides the first empirical evidence for feature-specific error correction in large language models, showing that residual-stream activations are robust to small perturbations but less robust along candidate feature directions, supporting the theory of computation in superposition.

This paper provides a mathematical analysis of superposition in neural networks, deriving upper and lower bounds on L2 reconstruction loss for simple autoencoders with power activation functions, corroborating empirical findings by Elhage et al.

This paper identifies a Möbius attractor and Cascade Supervision as key mechanisms for the emergence of superposition reasoning in transformers, closing a theoretical gap on gradient descent convergence for graph reachability tasks.