Tag
This paper critiques standard attention metric methods in transformers, revealing that choices like keeping or dropping sink tokens can reverse conclusions, and proposes using compositional data analysis to separate sink and content attention components for more accurate interpretation.
This paper investigates why transformer intermediate representations are off-axis relative to the readout direction, showing that this off-axis subspace functionally insulates composition from the vocabulary and proposing methods to impose this geometry via rotation.
This paper investigates short-term attention degradation in LLMs, finding a universal exponential-then-plateau pattern and that function token anchoring is architecture-dependent. Causal tests show that increasing attention mass on function tokens does not improve retrieval, suggesting attention degradation is descriptive rather than prescriptive.
Anthropic's research paper reveals an emergent internal workspace in LLMs where reasoning occurs before output, and a developer built a live viewer tool (Subtext) to observe this process, settling the debate on whether LLMs actually think.
The paper introduces Vocabulary-Aligned Sparse Autoencoder (VASAE), a method that trains SAE features under vocabulary-aligned anchoring, assigning each feature an intrinsic token name based on nearest token embedding, achieving high alignment in early layers without reducing reconstruction quality.
This paper reveals that interleaved speech-text language models implicitly transcribe speech into text in intermediate layers, then predict in text space before converting back to speech, shedding light on internal modality interaction.
Proposes the Bag of Dims framework showing that the standard basis of transformer hidden states provides a training-free, architecture-general feature representation where dimensions encode semantic content via sign patterns; validated across language, vision, and audio models, achieving high accuracy with no learned rotations.
Proposes MechRL, a reinforcement learning approach to automate circuit discovery in transformer language models. A PPO agent trained on multiple tasks discovers attention head circuits that match known canonical circuits and generalizes to a held-out task.
This paper identifies a Möbius attractor and Cascade Supervision as key mechanisms for the emergence of superposition reasoning in transformers, closing a theoretical gap on gradient descent convergence for graph reachability tasks.
A new preprint titled 'Mathematics is All You Need 2' presents the 'Two-Channel theorem,' demonstrating that behavioral fibers in transformer residual streams are sign-stabilized and causally steerable across different architectures (Qwen to Llama). The study claims high reproducibility and shows that the behavioral substrate is near-one-dimensional, separating generation from latent structure.
This research paper analyzes the internal mechanics of Large Vision-Language Models (LVLMs) using information theory, revealing that attention mechanisms may be redundant while Feed-Forward Networks drive semantic innovation. The authors demonstrate that replacing learned attention weights with random values can yield comparable performance, suggesting current models 'get lost in attention'.