@Tiberiu_Musat_: What really happens during LLM training? Can we ever disentangle those billions of interacting parameters? In our lates…
Summary
This preprint reveals that under certain data symmetries, LLM training dynamics can be reduced to a low-dimensional subspace, making analysis more tractable and interpretable, with each coordinate corresponding to a clear mechanism.
View Cached Full Text
Cached at: 07/17/26, 12:24 AM
What really happens during LLM training? Can we ever disentangle those billions of interacting parameters? In our latest preprint, we present a promising direction.
Under certain data symmetries, the parameter space turns out to have a low-dimensional subspace with self-contained dynamics during training. If training begins in this subspace, it never leaves.
The subspace has a low dimensionality, which means that we can reduce the training dynamics to a small number of pseudo-parameters. This makes theoretical and experimental analysis much more tractable.
Moreover, the subspace is highly interpretable: each coordinate corresponds to a clear mechanism. For example, an induction head is composed of 3 directions in this subspace.
Big thanks to my co-authors @tpimentelms @NicolasZucchet
Paper link in the first reply
Similar Articles
@rohanpaul_ai: Terence Tao says the math behind today’s LLMs is actually simple. Training and running them mostly uses linear algebra,…
Terence Tao states that the mathematics underlying modern LLMs is simple, using basic linear algebra and calculus, but the unpredictability of model performance across tasks remains a mystery due to the complex nature of natural language data.
@gurtej__gill_: This new paper by Ting Wen Ko and Jonas Geiping reveals something very wild: When LLMs talk to each other for a long ti…
A new paper reveals that when LLMs converse at length, they converge into predictable 'Attractor States' rather than drifting into chaos. Some models dominate and force stylistic mimicry, while others are highly malleable, raising implications for multi-agent AI system design.
LLMs are not the black box you were promised
An article summarizing Anthropic's 2025 paper on mechanistic interpretability, showing that LLMs are not black boxes and that circuit tracing can reveal multi-step reasoning and human-identifiable concepts.
LLMs with in-context learning for Algorithmic Theoretical Physics
This paper investigates using Large Language Models, specifically Claude, interfaced with a Computer Algebra System (Maple) to perform algorithmic computations in theoretical physics, such as analyzing cosmological perturbations.
@techNmak: This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Att…
An interactive 3D step-by-step guide to learning how LLMs work, covering key transformer concepts like embedding, self-attention, and softmax. It recommends a visual approach over reading papers.