Bag of Dims: Training-Free Mechanistic Interpretability via Dimension-Level Sign Patterns
Summary
Proposes the Bag of Dims framework showing that the standard basis of transformer hidden states provides a training-free, architecture-general feature representation where dimensions encode semantic content via sign patterns; validated across language, vision, and audio models, achieving high accuracy with no learned rotations.
View Cached Full Text
Cached at: 06/18/26, 03:58 PM
Paper page - Bag of Dims: Training-Free Mechanistic Interpretability via Dimension-Level Sign Patterns
Source: https://huggingface.co/papers/2606.12629
Abstract
The standard basis of transformer hidden states serves as a training-free, architecture-general feature representation where individual dimensions encode semantic content through signs and confidence through magnitudes, functioning as independent binary registers without requiring learned rotations or optimization.
We show thestandard basisoftransformer hidden statesalready provides a training-free, architecture-generalfeature basis. Individual dimensions encodesemantic contentvia their signs (+/-1) and confidence via theirmagnitudes, acting as independentbinary registers; a feature is a subset of dimensions with a consistent sign pattern, read by counting sign agreements with no learned rotation. We validate thisBag of Dims frameworkacross seven models spanning language (Qwen 3.5-4B, Gemma 3-4B, Mistral 7B, Qwen3-32B), vision (DINOv2, ViT-Base), and audio (AST). Signs alone carry predictive content: unit-magnitudesign patternspreserve 60-93% top-5next-token accuracythrough the LM head, and decoder-freeHamming scoringreaches 80-90% top-4096. From a single-token cache(oneforward passper token, no context, no labels), we detect 175 categories at AUC 0.97-0.99 by sign agreement; a trained probe adds only +0.018 AUC and converges to axis-aligned weights. These features are causally operative: they survive the K/Vattention projections, trace to theFFN neuron coalitionsthat write them (random-weight controls never reproduce this), and flipping a feature’s signs during the liveforward passsuppresses its concept across four language models,magnitude-matched and concept-specific. Dimensions stay independent throughout (pairwise mutual informationbelow 0.006 bits). The structure is not specific to language: the same per-dimension signs appear inself-supervised vision(DINOv2, 9/12 ImageNet superclasses),supervised vision(ViT-Base, 11/12), and audio (AST, 50/50 ESC-50 categories), so it reflects transformer training in general, not the language-modeling objective. Thestandard basisalready suffices for feature reading at oneforward pass, no optimization, no GPU-days. The open problem shifts from finding the right rotation to cataloging what each dimension encodes.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.12629
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.12629 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.12629 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.12629 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
An expressivity analysis of hierarchical modelling in deep transformers via bounded-depth grammars
This paper provides a theoretical analysis of deep transformers' ability to model hierarchical structures using bounded-depth context-free grammars, constructing explicit positional-attention transformers that encode grammatical states in linearly separable subspaces.
Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models
This paper introduces a quantitative framework for estimating how many near-orthogonal directions a transformer language model's latent space can support, based on the linear representation and superposition hypotheses. The authors define representational capacity as an upper bound on distinguishable features and show it is exponentially sensitive to the allowed deviation from orthogonality, with larger models favoring tighter constraints.
SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Space
SignX proposes a novel framework for continuous sign language recognition that unifies heterogeneous pose formats into a compact latent space and achieves state-of-the-art accuracy with 50× computational acceleration over pixel-space baselines.
Deep Dreams Are Made of This: Visualizing Monosemantic Features in Diffusion Models
This paper introduces Latent Visualization by Optimization (LVO), a mechanistic interpretability technique that uses sparse autoencoders to visualize monosemantic features in diffusion models like Stable Diffusion 1.5.
The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason
A comprehensive spectral analysis across 11 LLMs revealing that transformers exhibit phase transitions in hidden activation spaces during reasoning versus factual recall, with seven fundamental phenomena including spectral compression, instruction-tuning reversal, and perfect correctness prediction (AUC=1.0) based solely on spectral properties.