How are linear representations learned? Exact solutions to the dynamics of abstraction
Summary
This paper develops a framework to study how linear concept representations emerge during neural network training, providing exact solutions in linear networks and analyzing abstraction dynamics in nonlinear networks. The results reveal key principles governing abstraction and offer implications for interpretability and control.
View Cached Full Text
Cached at: 07/13/26, 07:55 AM
# How are linear representations learned? Exact solutions to the dynamics of abstraction
Source: [https://arxiv.org/abs/2607.08843](https://arxiv.org/abs/2607.08843)
[View PDF](https://arxiv.org/pdf/2607.08843)
> Abstract:In artificial and biological neural networks, concepts are often encoded as consistent linear directions in representation space\. In deep learning, this idea is known as the linear representation hypothesis and underpins many interpretability and control methods based on linear probes, from concept detection to activation steering\. Yet while prior work has studied whether such directions should exist $\\textit\{after\}$ training, the dynamics of how they emerge $\\textit\{during\}$ training remain poorly understood\. Here, we develop a framework to study the alignment of concept directions during training \- a process we call "abstraction"\. In a minimal linear network setting, we obtain exact solutions for the full trajectory of abstraction\. These solutions reveal key analytic principles governing abstraction: \(i\) data and target geometry jointly determine abstraction at the end\-of\-learning, \(ii\) abstraction improves with network depth, and \(iii\) initialization scale controls the maximum abstraction reached during training\. Extending our theory to nonlinear networks, we analyze how the choice of nonlinearity affects abstraction dynamics: erf networks approximate the linear theory, while abstraction in ReLU networks depends less on target geometry and more on input geometry\. Across both, we prove a striking attenuation law: both nonlinearities weaken abstraction in activations relative to preactivations\. We find evidence for this law in open models \(DINOv3, Gemma 4\) and apply our theory to improve linear probe generalization in LLMs\. Together, our results provide a dynamical theory of abstraction with implications for interpretability and control\.
## Submission history
From: William W\. Yang \[[view email](https://arxiv.org/show-email/a87e37bc/2607.08843)\] **\[v1\]**Thu, 9 Jul 2026 18:04:25 UTC \(3,435 KB\)Similar Articles
A Geometric View for Understanding Concept Learning and Neuron Interpretation in Sparse Autoencoders
This paper proposes a unified geometric framework for understanding concept learning and neuron interpretation in sparse autoencoders, formalizing concepts as sets and defining detection, separation, and approximation. It provides error bounds, capacity constraints, and links to formal concept analysis, with experiments on synthetic data.
Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory
This book presents a mathematical theory of deep representation learning, aiming to demystify the internal mechanisms of large deep networks using optimization and information theory, making architecture design a matter of linear algebra and calculus.
What are They Thinking? Delineation, Probing and Tracking of Concepts in LLMs
This paper presents a methodology for delineating concepts and training linear probes to detect them in LLM embeddings, using four example concepts across three models. The work aims to enable scalable monitoring of LLM internal representations.
Neural Networks Provably Learn Spectral Representations for Group Composition
This paper provides a theoretical analysis of how neural networks learn structured representations during group composition tasks, proving that training dynamics drive neurons to converge to irreducible group representations with exponential convergence rates. The work establishes a representation-theoretic account of feature learning and characterizes a low-rank compression phenomenon for matrix-valued group representations.
Nonlinear computation in deep linear networks
OpenAI research explores how nonlinear computation can emerge in deep linear networks, presenting theoretical and empirical analysis with code examples using TensorFlow.