From the Loss Landscape to Diverse Feature Learning in Neural Networks

# Abstract
Source: [https://arxiv.org/html/2608.28948](https://arxiv.org/html/2608.28948)
FROM THE LOSS LANDSCAPE TO DIVERSE FEATURE

LEARNING IN NEURAL NETWORKS

BY

DAVID ARAM YUNIS

A thesis submitted

in partial fulfillment of the requirements

for the degree of

Doctor of Philosophy in Computer Science

at the

TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO

Chicago, Illinois

September, 2026

Thesis Committee:

Matthew R\. Walter \(Thesis Advisor\)

Michael Maire

David McAllester

Mikhail Belkin

Figure 1:The hat structure of the loss landscape of neural networks\. We demonstrate this structure, provide a preliminary explanation for it, and leverage the explanation to improve practical neural networks\.Over the course of the last decade, neural networks have grown from an academic curiosity to moving the markets of nations\. Despite this explosion in both research and deployment, relatively little is understood about how they achieve the solutions they do\. This is both scientifically relevant, and pressing for society\. When neural networks make decisions across self\-driving, construction, law, hiring and health, there have been and will continue to be unintended consequences\.

However, attempting to generalize the failures of the largest and most important production systems makes for a very difficult task\. Yet signs of these failures exist at all scales of neural networks, so we should be able to study a much more tractable setting\. All neural networks must undergo an optimization process, called training, to be useful\. To a great degree, understanding neural networks is understanding their optimization: through what process and exposure to which data did they arrive at their results\. Yet our knowledge on this topic as a field is quite imprecise\. In particular, a curious phenomenon called mode connectivity, the ability to connect neural networks in the loss surface \(Figure[1](https://arxiv.org/html/2608.28948#Chx1.F1)\), defies explanation entirely\.

This dissertation elucidates, explains and exploits this special structure in the loss landscape\.

We begin by deepening the phenomenon of mode connectivity\. We show that it is both more complex and general than previously expected, and also that it defies common theoretical explanations previously put forth\. Mode connectivity separates neural networks from the simple models of the past\.

To explain the phenomenon, we propose an empirical approach centered on the spectral dynamics of weights—the behavior of singular values and vectors during optimization\. Beyond offering an explanation for the strange structure of the loss landscape, these spectral dynamics connect a broad variety of phenomena, including grokking, lottery tickets, memorization and regularization\. Such a large degree of common connections has been lacking in neural network phenomenology\.

Armed with the powerful lens of spectral dynamics, we turn our attention to diverse feature learning, aiming to enlarge the information neural networks extract from data\. We find that this spectral perspective allows us to overcome spurious correlations in simple settings, without any specialized assumptions common to prior work about task, loss or architecture\. We then build on this understanding to improve the training of more practical neural networks, scaling to language modeling\.

We hope these results can serve as a blueprint for the study of neural networks in the years to come\.

A mi papá\.

## Acknowledgments

It is hard for me to say that my PhD was very enjoyable\. Between the many twists and turns, failed submissions, and even the sheer amount of doubt about the work in the coming pages, I had a tough time\. But it was deeply fulfilling\. For the vast majority of my degree, I suffered in various ways for various reasons, and in many of those moments of darkness the brightness came from the people around me\. There is meaning in the struggle with others around\. I owe these people many thanks\.

First, to my advisor Matt Walter… what can I say? It’s no exaggeration that I owe my career to Matt\. As an overconfident bright\-eyed undergrad, he took me on and gave me a space to grow\. In the early days of graduate school, I went astray and spent years working on other topics, but he was happy to take me back into the lab even when I was a washed\-up third\-year with no papers to my name\. He indulged my stubbornness and my curiosity for the wide range of topics I explored, and was willing to let me run with this thesis among many who doubted my work\. I’m still not sure how it happened, but he has my deep, deep gratitude\. He has been a role model for research, for collaboration, for interdisciplinary work, for outreach, and for balance between work and life\. I hope you have the pleasure of working with him\. Bonus points if it’s on actual robots\.

To each of my committee members, I also owe thanks\. Like Matt, Michael Maire was willing to take me on in that terrible third year when I was lacking papers and low on confidence\. I appreciated his precise insight, his support in my transition and his ability to always dig deeper for a new experiment\. I hope a little of that has rubbed off on me\. On my first interaction with David McAllester on visit day, he turned his monitor around, pointed at GPT\-2 outputs, and told me “don’t do theory\.” Perhaps without that push I wouldn’t have found my current love of experiments\. Certainly, this dissertation wouldn’t exist in its current form\. My conversations with David in his corner office were a particular joy, whether on AI and society, information theory, or automated theorem proving\. They broadened my horizons\. Lastly, thank you to Misha Belkin\. Specifically for optimism and interest when this work was a few plots on a laptop\. His presentation of double descent at TTIC was one of the best talks I saw during graduate school, and a positive conversation for 30 minutes in a hotel lobby before he had to head to the airport carried me through years of doubts from others\. It has been my deep satisfaction to involve him more formally in the tail end of this work, to be able to discuss like a peer, and to visit UCSD for a fantastic week in the California sun dreaming of feature learning\. Thank you\.

The TTIC and UChicago intellectual communities were enabling places to do research\. I was able to touch so many aspects of machine learning in such a small building, and I’m thankful for the people, especially the students, who helped me to do this\. Thanks to Falcon Dai for introducing me to machine learning and bringing me into the lab\. For many discussions on research and learning to dream big\. To Chip Schaff for showing me the ropes, and a target for what research engineering should look like\. To Andrea Daniele for his company, for filling the lab with robots and making the rest of us look better for it\. Thank you to Shane Settle and Shubham Toshniwal for their advice during COVID, for their support in sticking with the program when everything lost all structure, and for reminding me that whatever I had was plenty to get through it\. I am thankful they stood in my corner\. Thanks to Pedro Savarese for being a role model a few years ahead, and inspiring me to take a leap for my own interests to see where I would land\. For his thoughtfulness and the seriousness with which he took my early and dumb ideas\. To Takuma Yoneda who first supported my strange ideas when I moved back to Matt’s group, at a time when I had very low confidence\. And for a colleague to discuss with\. To Sudarshan Babu for late night conversations, for humor, for sharing the suffering, for company\. To Kshitij Patel for his early collaboration in all of this work, for his groundedness and willingness to try to see the value\. Thank you to Gal Vardi for his dogged support of the middle chapter of this dissertation\. To Jungo Kasai for his help in navigating the internship market, and for demonstrating the value of a network\. To Kevin Stangl and Nick Kolkin for breakfasts, and help with orienting myself on the tail end of the program\. To Sam Buchanan for curiosity and lots of enriching conversations\. Thanks to Pushkar Shukla for his friendship, for his parallel experience that helped me to see I was not alone\.

I also want to thank the broader TTIC community\. Rose Bradford made a huge effort over Christmas break one year sorting out my fellowship when it was standing in the way of an internship\. She continued to sacrifice at other times for me and I am thankful\. Thank you to Asha for being a pleasure on the late nights and for making TTIC comfortable to work in\. Thanks to Mary Marre for her steadfast support of me throughout the program, for alerting me to any leftovers when I was in need\. Thank you to Chrissy Novak, Amy Minick, Erica Cocom and Alicia McClarin for their super\-responsiveness and for making lots of administrative work breezy that could easily become a complete drag\. Thanks to Adam Bohlander for his tireless teachings from a land before LLMs\. It was a unique pleasure to get a message from him about trying new tech in his office, and I always walked out knowing something more than when I walked in\. Thank you to Jerry for his bright energy, in the dark of winter, year after year, even when I was deep in the hole\. I am grateful\.

A special thanks to Xiao Zhang for our collaboration midway through the PhD\. When we started working together, I was scrambling for a way to survive, and I’m very glad I caught your hand when I did\. That project put wind in my sails, and though it was unrelated, gave me the confidence to attack this thesis again\. Thank you for the reminder that sometimes all one needs is a great collaborator, and the rest will come\.

From my professional colleagues, I want to thank Rui Shen for hiring me for an internship when I was having trouble in the program, and for encouraging me to go back to graduate school and fight for my degree\. Thanks to Andrew Anderson for his faith in me as a colleague, even early on, and for some friendship in the suburban summer of New York\. Thank you to Fantine Huot for her support, groundedness, and the way she showed me how to achieve professional balance\. Thanks to Joshua Maynez, Sian Gooding, and Q Green for a sense of belonging in a workplace, a strange thing I did not imagine\. Thank you to Mirella Lapata for cutting humor, and straight and practical advice for job searching\. Thanks to Sohee Wang for camaraderie during the late nights in the office chasing a dream\.

It took a long time, but as the PhD wrapped up, things started to improve\. In particular, I spent a delightful final year working in Matt’s lab, and besides the fun of doing my work and knowing it was my work, I enjoyed interacting with a new group of students\. Thank you to Xiaodan Du, Tianchong Jiang, Luzhe Sun, Vincent Tan, Teddy Ayalew, Haoran Chen, Hung Le, Sam Wheeler, Jingtian Ji, and Richard Xu\. It was a pleasure to work with you all, give a little bit of advice, and learn from another generation while I reflected on all of my early mistakes\. I miss the long walks to lunch in the summertime\. I miss the heated conversations even after two hours of sitting at group meetings\. I hope that you all drew something meaningful from our interactions, and that my experience was useful beyond myself\. I am thankful for your faith in me\.

Besides all of the academic and professional ties above, I have personal ones to thank\. My dear friends: Miles Grogger, Lucas Penido, Kevin Li, Lucas Fagen, Jainaha Srikumar, Phil Blok\. Thank you for being there through my worst times, and for being there in my finest hour\. I will treasure the memory of the full room at my defense very fondly\. I am grateful you put up with me\. I am grateful for your undying faith in me\. Thank you to Sensei Winston, and the rest of the Karate Club for a constancy, a practice, and an iron will that applied directly to this degree\. Osu\! Gracias a mi Abuela por su confianza en mí\. Y al resto de la familia en Colombia, en particular mi tío Iván, mi prima Alicia y mi Abuelo\. Estar y hablar con ustedes siempre fue un descanso\. Thanks to my sister Catherine for her lightness, and for a series of vignettes in the sunsets of LA\. Meals over a shared table, a reminder of who we are, grounding me even when I was floating aimlessly in the work\. To my mom for being an example in letting the world glance off of her\. For the plants, the garden, the kitchen, and an attitude toward work of getting it done\. To my dad, for reminding me over the years, and always in the way I needed but not always how I wanted, that the thing that matters most is that you never give up\.

Like other journeys before mine, my PhD has been a series of decisions to not drop out\. One in the first year with COVID\. Another in the second year with nothing to show for a research agenda while many peers moved forward\. Another after an internship when it seemed I had little to come back to\. Another when very senior researchers questioned whether what I was doing was at all worthwhile, and I had to find a new agenda\. Another when a series of papers all went through three to four rounds of rejection\. Another when the reception after acceptance was nonexistent\. I think now that the most important thing was to hang on, and it is only because of the people above and those I’m inevitably forgetting that I was able to do so\. I am thankful that I was surrounded by so many great people during these times\. It would have been much harder to keep going without their hands on my back\. I hope you enjoy the result of their support in the following pages\.

###### Contents

1. [Abstract](https://arxiv.org/html/2608.28948#Chx1)
2. [1Introduction](https://arxiv.org/html/2608.28948#Ch1)1. [1\.1Contributions](https://arxiv.org/html/2608.28948#Ch1.S1)
3. [2Background](https://arxiv.org/html/2608.28948#Ch2)
4. [3On Convexity and Linear Mode Connectivity in Neural Networks Yunis et al\. \(2022\)](https://arxiv.org/html/2608.28948#Ch3)1. [3\.1Introduction](https://arxiv.org/html/2608.28948#Ch3.S1) 2. [3\.2Convex Mode Connectivity](https://arxiv.org/html/2608.28948#Ch3.S2) 3. [3\.3Measures of Convexity](https://arxiv.org/html/2608.28948#Ch3.S3) 4. [3\.4Diversity of Endpoints](https://arxiv.org/html/2608.28948#Ch3.S4) 5. [3\.5Discussion](https://arxiv.org/html/2608.28948#Ch3.S5)
5. [4Approaching Deep Learning through the Spectral Dynamics of Weights Yunis et al\. \(2024\)](https://arxiv.org/html/2608.28948#Ch4)1. [4\.1Introduction](https://arxiv.org/html/2608.28948#Ch4.S1) 2. [4\.2Related Work](https://arxiv.org/html/2608.28948#Ch4.S2)1. [4\.2\.1Singular Value Dynamics](https://arxiv.org/html/2608.28948#Ch4.S2.SS1) 2. [4\.2\.2Low\-Rank Properties](https://arxiv.org/html/2608.28948#Ch4.S2.SS2) 3. [4\.3Common Quantities](https://arxiv.org/html/2608.28948#Ch4.S3) 4. [4\.4Grokking and Rank Minimization](https://arxiv.org/html/2608.28948#Ch4.S4) 5. [4\.5Spectral Dynamics Across Tasks](https://arxiv.org/html/2608.28948#Ch4.S5)1. [4\.5\.1Methodology](https://arxiv.org/html/2608.28948#Ch4.S5.SS1) 2. [4\.5\.2Effective Rank Minimization](https://arxiv.org/html/2608.28948#Ch4.S5.SS2) 3. [4\.5\.3Alignment of Singular Vectors Between Layers](https://arxiv.org/html/2608.28948#Ch4.S5.SS3) 6. [4\.6The Effect of Weight Decay](https://arxiv.org/html/2608.28948#Ch4.S6) 7. [4\.7Additional Connections](https://arxiv.org/html/2608.28948#Ch4.S7) 8. [4\.8Spectral Dynamics with Random Labels](https://arxiv.org/html/2608.28948#Ch4.S8) 9. [4\.9Beyond Generalization](https://arxiv.org/html/2608.28948#Ch4.S9)1. [4\.9\.1Top Singular Vectors Become Stable Earlier](https://arxiv.org/html/2608.28948#Ch4.S9.SS1) 2. [4\.9\.2Lottery Tickets Preserve Final Top Singular Vectors](https://arxiv.org/html/2608.28948#Ch4.S9.SS2) 3. [4\.9\.3Spectral Dynamics and Linear Mode Connectivity](https://arxiv.org/html/2608.28948#Ch4.S9.SS3) 10. [4\.10Discussion](https://arxiv.org/html/2608.28948#Ch4.S10)
6. [5Overcoming Spurious Correlations through Diverse Feature Learning](https://arxiv.org/html/2608.28948#Ch5)1. [5\.1Introduction](https://arxiv.org/html/2608.28948#Ch5.S1) 2. [5\.2Simplicity Bias and Spectral Dynamics](https://arxiv.org/html/2608.28948#Ch5.S2)1. [5\.2\.1Projected Gradient Descent for Diverse Feature Learning](https://arxiv.org/html/2608.28948#Ch5.S2.SS1) 2. [5\.2\.2Experiments](https://arxiv.org/html/2608.28948#Ch5.S2.SS2) 3. [5\.3Diverse Feature Learning in Representation Space](https://arxiv.org/html/2608.28948#Ch5.S3)1. [5\.3\.1A Problem with Projected Gradient Descent](https://arxiv.org/html/2608.28948#Ch5.S3.SS1) 2. [5\.3\.2Inter\-Example Similarity as a Measure of Functional Diversity](https://arxiv.org/html/2608.28948#Ch5.S3.SS2) 4. [5\.4Diverse Feature Learning in Output Space](https://arxiv.org/html/2608.28948#Ch5.S4)1. [5\.4\.1Negative Correlation Learning](https://arxiv.org/html/2608.28948#Ch5.S4.SS1) 2. [5\.4\.2Boosting Incorrect Predictions](https://arxiv.org/html/2608.28948#Ch5.S4.SS2) 3. [5\.4\.3Boosting Low\-Entropy Predictions](https://arxiv.org/html/2608.28948#Ch5.S4.SS3) 4. [5\.4\.4Ensemble Weighting](https://arxiv.org/html/2608.28948#Ch5.S4.SS4) 5. [5\.5Related Work](https://arxiv.org/html/2608.28948#Ch5.S5)1. [5\.5\.1Leveraging Downstream Information](https://arxiv.org/html/2608.28948#Ch5.S5.SS1) 2. [5\.5\.2Relying on Pretrained Models](https://arxiv.org/html/2608.28948#Ch5.S5.SS2) 3. [5\.5\.3Architectural Modifications](https://arxiv.org/html/2608.28948#Ch5.S5.SS3) 4. [5\.5\.4Objective and Optimization Approaches](https://arxiv.org/html/2608.28948#Ch5.S5.SS4) 5. [5\.5\.5Ensembling](https://arxiv.org/html/2608.28948#Ch5.S5.SS5) 6. [5\.6Discussion](https://arxiv.org/html/2608.28948#Ch5.S6)
7. [6Conclusion](https://arxiv.org/html/2608.28948#Ch6)
8. [References](https://arxiv.org/html/2608.28948#bib)
9. [AConvex Mode Connectivity](https://arxiv.org/html/2608.28948#A1)1. [A\.1Task Details](https://arxiv.org/html/2608.28948#A1.S1) 2. [A\.2Metric Calculation Details](https://arxiv.org/html/2608.28948#A1.S2)
10. [BSpectral Dynamics](https://arxiv.org/html/2608.28948#A2)1. [B\.1Explanation of Balancedness](https://arxiv.org/html/2608.28948#A2.S1) 2. [B\.2Experimental Details](https://arxiv.org/html/2608.28948#A2.S2)1. [B\.2\.1Image Classification with VGG](https://arxiv.org/html/2608.28948#A2.S2.SS1) 2. [B\.2\.2Image Generation with UNets](https://arxiv.org/html/2608.28948#A2.S2.SS2) 3. [B\.2\.3Speech Recognition with LSTMs](https://arxiv.org/html/2608.28948#A2.S2.SS3) 4. [B\.2\.4Language Modeling with Transformers](https://arxiv.org/html/2608.28948#A2.S2.SS4) 5. [B\.2\.5Spectral Dynamics with Scale \(Pythia\)](https://arxiv.org/html/2608.28948#A2.S2.SS5) 6. [B\.2\.6Weight Decay Experiments](https://arxiv.org/html/2608.28948#A2.S2.SS6) 7. [B\.2\.7Grokking Experiments](https://arxiv.org/html/2608.28948#A2.S2.SS7) 8. [B\.2\.8Random Label Experiments](https://arxiv.org/html/2608.28948#A2.S2.SS8) 9. [B\.2\.9Magnitude Pruning Experiments](https://arxiv.org/html/2608.28948#A2.S2.SS9) 10. [B\.2\.10LMC Experiments](https://arxiv.org/html/2608.28948#A2.S2.SS10) 11. [B\.2\.11Perturbed LMC Experiments](https://arxiv.org/html/2608.28948#A2.S2.SS11) 3. [B\.3Limitations](https://arxiv.org/html/2608.28948#A2.S3) 4. [B\.4Additional Experiments](https://arxiv.org/html/2608.28948#A2.S4)1. [B\.4\.1Effect of initialization scale](https://arxiv.org/html/2608.28948#A2.S4.SS1) 2. [B\.4\.2Effect of learning rate](https://arxiv.org/html/2608.28948#A2.S4.SS2) 3. [B\.4\.3Individual seed plots for grokking](https://arxiv.org/html/2608.28948#A2.S4.SS3) 4. [B\.4\.4Full alignment matrix evolution](https://arxiv.org/html/2608.28948#A2.S4.SS4) 5. [B\.4\.5Image classification](https://arxiv.org/html/2608.28948#A2.S4.SS5) 6. [B\.4\.6](https://arxiv.org/html/2608.28948#A2.S4.SS6) 5. [B\.5Compute Resources](https://arxiv.org/html/2608.28948#A2.S5) 6. [B\.6Code Sources](https://arxiv.org/html/2608.28948#A2.S6)
11. [CDiverse Feature Learning](https://arxiv.org/html/2608.28948#A3)1. [C\.1PGD Experiments with Image Classification](https://arxiv.org/html/2608.28948#A3.S1) 2. [C\.2CKA Experiments with Image Classification](https://arxiv.org/html/2608.28948#A3.S2) 3. [C\.3Ensembling in Language Modeling](https://arxiv.org/html/2608.28948#A3.S3)1. [C\.3\.1CKA](https://arxiv.org/html/2608.28948#A3.S3.SS1) 2. [C\.3\.2NCL](https://arxiv.org/html/2608.28948#A3.S3.SS2) 3. [C\.3\.3Focal weighting](https://arxiv.org/html/2608.28948#A3.S3.SS3) 4. [C\.3\.4Entropy weighting](https://arxiv.org/html/2608.28948#A3.S3.SS4) 4. [C\.4Ensemble Weighting](https://arxiv.org/html/2608.28948#A3.S4)

###### List of Tables

1. [5\.1Probing accuracy for CIFAR on middle layers for an ensemble of 5 models drawn from early or late rounds of PGD\. We see that early rounds are substantially worse at extracting CIFAR information from the middle of the model, reinforcing the argument that PGD forces the model to learn more\.](https://arxiv.org/html/2608.28948#Ch5.T1)
2. [5\.2OOD linear probing accuracy between different methods\. We observe the same effect as \( \) , that random ensembles \(CAT\) are much stronger than a large model that is parameter advantaged, but PGD ensembles do not appear to improve over random\.](https://arxiv.org/html/2608.28948#Ch5.T2)
3. [5\.3Top\-1 OOD accuracy for random ensembles \(CAT\) v\.s\. CKA ensembles\. We see that CKA improves by a substantial margin over CAT in the linear probing setup\.](https://arxiv.org/html/2608.28948#Ch5.T3)
4. [5\.4OOD cross entropy for random ensembles \(CAT\) v\.s\. CKA ensembles under many different variations\. Unlike the image classification case, we are unable to find a version of CKA that outperforms random ensembles\.](https://arxiv.org/html/2608.28948#Ch5.T4)
5. [5\.5OOD cross entropy for random ensembles v\.s\. NCL ensembles We see no significant difference to random ensembles\. NCL JSD corresponds to using Jensen\-Shannon divergence for difference\. MSE corresponds to MSE for the probability distribution, and Top\-1 corresponds to only considering the difference between probabilities of the correct class\.](https://arxiv.org/html/2608.28948#Ch5.T5)
6. [5\.6OOD cross entropy for random ensembles v\.s\. Focal scaling ensembles We see no significant difference to random ensembles\. Top\-p corresponds to scaling the top\-p of the tokens by sequence by a constant additive factor\.](https://arxiv.org/html/2608.28948#Ch5.T6)
7. [5\.7OOD cross entropy for random ensembles v\.s\. entropy scaling ensembles We see no significant difference to random ensembles\. Top\-p corresponds to scaling the top\-p of the tokens by sequence by a constant additive factor\.](https://arxiv.org/html/2608.28948#Ch5.T7)
8. [5\.8OOD cross entropy for different ensemble weighting methods on the Entropy top\-p runs\. We use these as the base as we expect there may be nonuniformity compared to the Random ensembles\.](https://arxiv.org/html/2608.28948#Ch5.T8)
9. [A\.1Summary of tasks studied in this chapter\.](https://arxiv.org/html/2608.28948#A1.T1)

###### List of Figures

1. [1The hat structure of the loss landscape of neural networks\. We demonstrate this structure, provide a preliminary explanation for it, and leverage the explanation to improve practical neural networks\.](https://arxiv.org/html/2608.28948#Chx1.F1)
2. [2\.1Schematic of different varieties of loss landscapes\. Color denotes height, where lower is better\. \(a\): Convex loss\. It is possible to find a path to the global minimum through local steps, when a ball is dropped in the landscape it will roll to the bottom\. \(b\): Nonconvex loss\. Unlike the convex case, a ball may get stuck in a particular local minimum without finding the global minimum\. Neural networks are generally nonconvex, making optimization more difficult than the convex case\.](https://arxiv.org/html/2608.28948#Ch2.F1)
3. [\(a\)Convex Loss](https://arxiv.org/html/2608.28948#Ch2.F1.sf1)
4. [\(b\)Nonconvex Loss](https://arxiv.org/html/2608.28948#Ch2.F1.sf2)
5. [3\.1The algorithmic procedure and graphic for measuring convex mode connectivity\.](https://arxiv.org/html/2608.28948#Ch3.F1)
6. [3\.2Top row: The empirical probability P conv that a sample inside the convex hull \(Convex\) or a random perturbation P rand around the endpoint \(Random\) have lower loss than their closest endpoints vs\. the split epoch \( k in Algorithm \) for the many different trajectories\. After an initial phase of training, the probability of lower loss inside the hull quickly converges to one, the loss inside the hull is always better than at the endpoints\. Bottom row: Parameter distances between pairs of points in the CMC procedure\. The distance between pairs of endpoints is of the same order of magnitude as other distances for all tasks\. In other words, the apparent scale of convexity is similar to that of the total trajectory\. Error bars for probabilities are computed by empirical standard deviations ⁢ std \( I conv j \) and ⁢ std \( I rand j \) , hence are not meaningful when values stretch greater than 1 or less than 0\.](https://arxiv.org/html/2608.28948#Ch3.F2)
7. [\(a\)SIREN](https://arxiv.org/html/2608.28948#Ch3.F2.sf1)
8. [\(b\)LeNet](https://arxiv.org/html/2608.28948#Ch3.F2.sf2)
9. [\(c\)LeNet\-m](https://arxiv.org/html/2608.28948#Ch3.F2.sf3)
10. [\(d\)ResNet](https://arxiv.org/html/2608.28948#Ch3.F2.sf4)
11. [3\.3Singular values for the matrix of stacked endpoints of the mode\. We examine endpoints split after the point when the hull center performs better than the endpoints \(see Figure \)\. For P points in space, the shape they define can have at most \- P 1 dimensions\. We see in all cases that the magnitudes do not decay to zero until component \- P 1 \. In the case that there are K outputs and < P K \(ResNet\), we see a roughly uniform mode\. In cases with \> P K \(SIREN, LeNet, and LeNet\-m\), it does not appear that there is any special fragmenting at K \. In particular, the LeNet plot is over 300 endpoints, and the tail does not look as if it will decay to zero any time soon\. The computational expense of these experiments limits how many endpoints we train\.](https://arxiv.org/html/2608.28948#Ch3.F3)
12. [\(a\)SIREN](https://arxiv.org/html/2608.28948#Ch3.F3.sf1)
13. [\(b\)LeNet](https://arxiv.org/html/2608.28948#Ch3.F3.sf2)
14. [\(c\)LeNet\-m](https://arxiv.org/html/2608.28948#Ch3.F3.sf3)
15. [\(d\)ResNet](https://arxiv.org/html/2608.28948#Ch3.F3.sf4)
16. [3\.4Top row: The ratio of smallest to largest eigenvalue of the Hessian throughout training\. This ratio is almost always negative, but quite small, making any local non\-convexity appear unimportant\. Bottom row: Probability of a convex line of loss between a pair of updates\. Notice that this probability is close to 1 for the entire optimization, typically before the LMC regime in Figure \.](https://arxiv.org/html/2608.28948#Ch3.F4)
17. [\(a\)SIREN](https://arxiv.org/html/2608.28948#Ch3.F4.sf1)
18. [\(b\)LeNet](https://arxiv.org/html/2608.28948#Ch3.F4.sf2)
19. [\(c\)LeNet\-m](https://arxiv.org/html/2608.28948#Ch3.F4.sf3)
20. [\(d\)ResNet](https://arxiv.org/html/2608.28948#Ch3.F4.sf4)
21. [3\.5Top row: Average accuracies at the endpoints of training \(Endpoint\), inside the convex mode defined by the endpoints \(Convex\), and at random directions around endpoints \(Random\)\. Notice that after a late enough split, samples in the hull are better than the endpoints\. Bottom row: Average training accuracy over runs \(Accuracy\) compared to common points predicted correctly within a mode \(Intra mode\) and common points predicted correctly between hulls \(Inter mode\)\. We see that for models with an imperfect training accuracy, intra mode agreement is substantially lower than accuracy, and inter mode agreement is even lower\. This suggests that endpoints of a mode are not similar functions\.](https://arxiv.org/html/2608.28948#Ch3.F5)
22. [\(a\)LeNet](https://arxiv.org/html/2608.28948#Ch3.F5.sf1)
23. [\(b\)LeNet\-m](https://arxiv.org/html/2608.28948#Ch3.F5.sf2)
24. [\(c\)ResNet](https://arxiv.org/html/2608.28948#Ch3.F5.sf3)
25. [4\.1 Schematic for the spectral dynamics of a weight matrix\. As training proceeds, top singular vectors become stable and top singular values grow disproportionately large\. Singular value evolution for a single intermediate matrix parameter in a Transformer, where each line is a single singular value\. Darker colors correspond to larger singular values while lighter ones correspond to smaller values\. We see a disproportionate trend where large singular values grow larger faster\. Previous works use SV basis alignment between layers to prove similar theoretical results, however actual alignment between consecutive layers is weak, which we describe in Section \. We explore these spectral dynamics of weights and connect them to generalization, regularization, and seemingly unrelated phenomena like linear mode connectivity\.](https://arxiv.org/html/2608.28948#Ch4.F1)
26. [\(a\)SV Schematic](https://arxiv.org/html/2608.28948#Ch4.F1.sf1)
27. [\(b\)SVs](https://arxiv.org/html/2608.28948#Ch4.F1.sf2)
28. [\(c\)Alignment Schematic](https://arxiv.org/html/2608.28948#Ch4.F1.sf3)
29. [4\.2Grokking and Spectral Dynamics\. Top row: Grokking for Transformers on modular addition \( , \) for a single seed due to different convergence times\. See Figure for averaging over multiple seeds\. Bottom row: Grokking for a 12\-layer MLP on MNIST \( , \)\. \(a\) Training and validation error\. Shaded regions correspond to standard deviation, where large deviations are due to different seeds converging at different times\. \(b\) A visualization of singular value evolution for the first attention parameter and the second MLP layer, where each line represents a single singular value and the color represents the rank\. \(c\) Effective rank of all layers \(matrix parameters\) over time \(Eqn\. \)\. \(d\) A visualization of the alignment \(Eqn\. \) between the embedding and the first attention parameter, and the first and second MLP layers, where the y \-axis corresponds to index i of the diagonal\. We see that grokking co\-occurs with a transition to low\-rank weights\. In addition, there is an alignment that begins early in training that evolves up the diagonal\. In the image classification case, we see a similar rank transition, though alignment does not evolve up the diagonal \(see Figure \)\.](https://arxiv.org/html/2608.28948#Ch4.F2)
30. [\(a\)Error](https://arxiv.org/html/2608.28948#Ch4.F2.sf1)
31. [\(b\)SV Evolution](https://arxiv.org/html/2608.28948#Ch4.F2.sf2)
32. [\(c\)Effective Rank](https://arxiv.org/html/2608.28948#Ch4.F2.sf3)
33. [\(d\)Alignment](https://arxiv.org/html/2608.28948#Ch4.F2.sf4)
34. [4\.3Top row: Singular value evolution for a single matrix in the middle of each model\. Each line represents a singular value, darker colors are larger values, while lighter colors are smaller values\. Notice the unequal evolution where top singular values grow at a disproportionate rate\. Bottom row: Normalized effective rank \(Eqn\. \) evolution visualized in color for all matrix parameters across architectures and time\. As we move down the y \-axis, the depth of the parameters in the model increases, while the x \-axis tracks training time\. The axis label "Layer" is shorthand for "matrix parameter"\. For example, in the Transformer we visualize each of the W q , W k , W v , ⁢ M L P 1 , ⁢ M L P 2 parameters for each block\. Notice decreasing effective rank with training time across nearly all parameters, though the magnitude differs across layers \(indicated by absolute color\)\. The block\-like patterns for VGG are likely due to different channel dimension sizes\. The banding in the UNet, LSTM, and Transformer is due to the differences between convolutional and linear layers, residual block connections, and attention and fully connected layers, respectively\. The sharp transitions through training in VGG are due to 10 × learning rate decays\.](https://arxiv.org/html/2608.28948#Ch4.F3)
35. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F3.sf1)
36. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F3.sf2)
37. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F3.sf3)
38. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F3.sf4)
39. [4\.4Top row: Training loss\. Bottom row: Validation loss\. Red is the full model\. Blue is post\-training pruning the bottom half of the SVD for every matrix in the model that is not the final layer\. Green is post\-training pruning the top half of the SVD\. Notice that for all models, keeping the top half of the SVD is close to the full model performance, supporting the idea that the top directions provide a good approximation to the function\. We specifically avoid pruning the final layer as for some tasks it is so low\-rank that pruning further affects performance in an anomalous way \( , \)\.](https://arxiv.org/html/2608.28948#Ch4.F4)
40. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F4.sf1)
41. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F4.sf2)
42. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F4.sf3)
43. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F4.sf4)
44. [4\.5Neighboring layer alignment of singular vectors\. Top row: The diagonal of the alignment matrix ⁢ A \( t \) ⁢ i i \(Eqn\. \) vs\. training time for a single pair of matrices in the middle of each model\. We see a small amount of alignment in the top ranks between layers shortly after training begins, but this becomes more diffuse over time\. Bottom row: Alignment metric \(Eqn\. \) for pairs of matrices for depth vs\. training time\. It is hard to make out a global trend across models, though the LSTM shows a weak signal around Epoch 1 when the initial alignment occurs, and the Transformer case has a banding pattern with depth due to alignment between the query and key matrices that have no nonlinearity in between\.](https://arxiv.org/html/2608.28948#Ch4.F5)
45. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F5.sf1)
46. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F5.sf2)
47. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F5.sf3)
48. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F5.sf4)
49. [4\.6Singular value evolution for a single matrix and normalized effective rank \(Eqn\. \) across matrices over time, where the rows use differing amounts of weight decay\. From top to bottom, for VGG we use coefficients \{ 0 , 0\.001 , 0\.01 , 0\.1 \} , while for other networks we use coefficients \{ 0 , 0\.1 , 1 , 10 \} \. Higher weight decay coefficients promote more aggressive rank minimization\. VGG uses SGD with momentum, while the rest use AdamW \( , \), which may explain the earlier norm collapse\. We provide an additional version with shared color bars in Figure \.](https://arxiv.org/html/2608.28948#Ch4.F6)
50. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F6.sf1)
51. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F6.sf2)
52. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F6.sf3)
53. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F6.sf4)
54. [4\.7Top row: results with true labels\. Bottom row: results with random labels\. We see that the middle layers have a lower effective rank \(Eqn\. \) when using true labels and that alignment \(Eqn\. \) in the middle layers persists throughout training, unlike in the random label case\. We emphasize this alignment occurs despite the nonlinearities\.](https://arxiv.org/html/2608.28948#Ch4.F7)
55. [\(a\)Train Err\.](https://arxiv.org/html/2608.28948#Ch4.F7.sf1)
56. [\(b\)Val\. Err\.](https://arxiv.org/html/2608.28948#Ch4.F7.sf2)
57. [\(c\)SVs](https://arxiv.org/html/2608.28948#Ch4.F7.sf3)
58. [\(d\)Eff\. Rank](https://arxiv.org/html/2608.28948#Ch4.F7.sf4)
59. [\(e\)Alignment](https://arxiv.org/html/2608.28948#Ch4.F7.sf5)
60. [4\.8Top row: Barrier size \(Eqn\. \) vs\. split step\. Middle row: singular vector agreement for a single matrix parameter between branch endpoints that share a common trunk\. Bottom row: summary statistic for singular vector agreement across layers vs\. split step\. We see that as models exhibit LMC, they also share a small number of top singular vectors\. We only show agreement for 100 dimensions as the rest are near zero\.](https://arxiv.org/html/2608.28948#Ch4.F8)
61. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F8.sf1)
62. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F8.sf2)
63. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F8.sf3)
64. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F8.sf4)
65. [4\.9Dynamics with random labels for VGG\. Top row: results with true labels\. Bottom row: results with random labels\. We see that the middle layers have a lower effective rank \(Eqn\. \) when using true labels and that alignment \(Eqn\. \) in the middle layers persists throughout training\. The results are less stark in the VGG case, but similar to the MLP\.](https://arxiv.org/html/2608.28948#Ch4.F9)
66. [\(a\)Train Err\.](https://arxiv.org/html/2608.28948#Ch4.F9.sf1)
67. [\(b\)Val\. Err\.](https://arxiv.org/html/2608.28948#Ch4.F9.sf2)
68. [\(c\)SVs](https://arxiv.org/html/2608.28948#Ch4.F9.sf3)
69. [\(d\)Eff\. Rank](https://arxiv.org/html/2608.28948#Ch4.F9.sf4)
70. [\(e\)Alignment](https://arxiv.org/html/2608.28948#Ch4.F9.sf5)
71. [4\.10Dynamics with random labels for LSTM\. Top row: results with true labels\. Bottom row: results with random labels\. We see that the middle layers have a lower effective rank \(Eqn\. \) when using true labels and that alignment \(Eqn\. \) in the middle layers persists throughout training\. Though the LSTM does not fit the random labels perfectly, the results are qualitatively similar to the other cases\.](https://arxiv.org/html/2608.28948#Ch4.F10)
72. [\(a\)Train Err\.](https://arxiv.org/html/2608.28948#Ch4.F10.sf1)
73. [\(b\)Val\. Err\.](https://arxiv.org/html/2608.28948#Ch4.F10.sf2)
74. [\(c\)SVs](https://arxiv.org/html/2608.28948#Ch4.F10.sf3)
75. [\(d\)Eff\. Rank](https://arxiv.org/html/2608.28948#Ch4.F10.sf4)
76. [\(e\)Alignment](https://arxiv.org/html/2608.28948#Ch4.F10.sf5)
77. [4\.11Top row: Singular vector agreement for a single matrix in the middle of each model \(diagonal of Eqn\. \)\. Notice top singular vectors become stable in direction earlier\. Bottom row: Summary score for each matrix across architectures\. As we move down the y \-axis, the depth of the parameters in the model increases, while the x \-axis tracks training time\. The sharp transition midway through training in the VGG case is likely due to a 10x learning rate decay\.](https://arxiv.org/html/2608.28948#Ch4.F11)
78. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F11.sf1)
79. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F11.sf2)
80. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F11.sf3)
81. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F11.sf4)
82. [4\.12Pruning results for VGG\. Top row: Magnitude pruning\. Bottom row: random pruning\. \(a\): Training loss\. We see that at 5% sparsity magnitude pruning is significantly better than random pruning of the same layerwise sparsity\. \(b\): Singular vector alignment pre\- and post\-pruning at the end of training for a single layer \(the 3rd convolution\)\. We see that magnitude pruning approximates many more top singular vectors than random pruning at the same sparsity\. \(c\): Singular vector alignment score pre\- and post\-pruning across all layers\. Agreement is higher across all layers for magnitude pruning, though curiously deeper layers do not agree, likely as later layers are wider so weights are lower magnitude and more will be pruned by the unstructured process\. \(d\): Singular vector alignment between the pruned and unpruned models along the training trajectory\. We see that the magnitude pruning still has similar dynamics in its top singular vectors, while random pruning does not\. \(e\): Singular vector alignment score between pruned and unpruned models across layers and time\. Again evolution is similar for early layers with magnitude pruning, and completely different for random pruning\.](https://arxiv.org/html/2608.28948#Ch4.F12)
83. [\(a\)Loss](https://arxiv.org/html/2608.28948#Ch4.F12.sf1)
84. [\(b\)Pruned SVA](https://arxiv.org/html/2608.28948#Ch4.F12.sf2)
85. [\(c\)All Layers](https://arxiv.org/html/2608.28948#Ch4.F12.sf3)
86. [\(d\)SVA evol\.](https://arxiv.org/html/2608.28948#Ch4.F12.sf4)
87. [\(e\)All Layers](https://arxiv.org/html/2608.28948#Ch4.F12.sf5)
88. [4\.13Pruning results for LSTM\. Top row: Magnitude pruning\. Bottom row: random pruning\. See Figure for details\. Results are quite similar for the LSTM at 25% pruning as the VGG in Figure \.](https://arxiv.org/html/2608.28948#Ch4.F13)
89. [\(a\)Loss](https://arxiv.org/html/2608.28948#Ch4.F13.sf1)
90. [\(b\)Pruned SVA](https://arxiv.org/html/2608.28948#Ch4.F13.sf2)
91. [\(c\)All Layers](https://arxiv.org/html/2608.28948#Ch4.F13.sf3)
92. [\(d\)SVA evol\.](https://arxiv.org/html/2608.28948#Ch4.F13.sf4)
93. [\(e\)All Layers](https://arxiv.org/html/2608.28948#Ch4.F13.sf5)
94. [4\.14Top row: Barrier size vs\. split step\. Middle row: singular vector agreement for a single matrix parameter between branch endpoints that do not share a common trunk, but do share split time and branch data order\. Bottom row: summary statistic for singular vector agreement across layers\. We see that when branches do not share a common trunk, there is neither LMC nor singular vector agreement, even though the optimization is otherwise the same\.](https://arxiv.org/html/2608.28948#Ch4.F14)
95. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F14.sf1)
96. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F14.sf2)
97. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F14.sf3)
98. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F14.sf4)
99. [4\.15Top row: Barrier size vs\. perturbation magnitude\. Middle row: singular vector agreement for a single matrix parameter between branch endpoints vs\. perturbation magnitude\. Bottom row: summary statistic for singular vector agreement across layers with perturbation magnitude\. We see that whereas without perturbation models would exhibit LMC after training, with increasing perturbations the LMC property disappears simultaneously with the agreement in top singular vectors\.](https://arxiv.org/html/2608.28948#Ch4.F15)
100. [\(a\)VGG](https://arxiv.org/html/2608.28948#Ch4.F15.sf1)
101. [\(b\)UNet](https://arxiv.org/html/2608.28948#Ch4.F15.sf2)
102. [\(c\)LSTM](https://arxiv.org/html/2608.28948#Ch4.F15.sf3)
103. [\(d\)Transformer](https://arxiv.org/html/2608.28948#Ch4.F15.sf4)
104. [5\.1Examples from the MNIST\-CIFAR dataset in \( \) \.](https://arxiv.org/html/2608.28948#Ch5.F1)
105. [5\.2Evaluation protocol for experiments\. After training on MNIST\-CIFAR, we swap the classes of either the MNIST half, or the CIFAR half of the image and evaluate on that setting\. When models rely entirely on MNIST, error on the swapped task degrades to 100%, and similarly for CIFAR\. On the right we plot the errors over successive rounds of PGD\. We see that models start to learn CIFAR after the early rounds of PGD, coming to rely quite heavily on CIFAR in later rounds\. Spikes correspond to restarting a new round of training\.](https://arxiv.org/html/2608.28948#Ch5.F2)
106. [5\.3Left: Training and validation error for successive rounds with our proposed PGD method on top singular vectors\. Right: Swapped\-class errors throughout rounds\. We see that as the round number increases, validation error decreases due to the complexity of the task increasing\. In addition, the model increasingly relies more on CIFAR, as the swapped CIFAR error goes from near 0 to 60% and above\. This indicates the model is learning to extract more complex signals which are less predictive but still useful\.](https://arxiv.org/html/2608.28948#Ch5.F3)
107. [\(a\)Normal](https://arxiv.org/html/2608.28948#Ch5.F3.sf1)
108. [\(b\)Swapped\-class](https://arxiv.org/html/2608.28948#Ch5.F3.sf2)
109. [5\.4First layer singular vectors over rounds of PGD training\. Each set of columns corresponds to the MNIST and CIFAR halves of the reshaped singular vector\. Descending down the rows corresponds to increasing rank index when sorted by singular value\. We see initially that the first layer singular vectors form digit classifiers exclusively\. In middle rounds the network focuses on higher and higher frequency signals in MNIST\. In late rounds, all but the high frequency signals are present in MNIST, hence the model starts to leverage the CIFAR half of the image\.](https://arxiv.org/html/2608.28948#Ch5.F4)
110. [5\.5Left: Training and validation error for successive rounds with PGD where we flatten the gradient and singular vectors before projection\. Right: Swapped\-class errors throughout rounds\. We see that no new features are learned when ignoring the matrix structure of the parameters, so the spectral design is critical\.](https://arxiv.org/html/2608.28948#Ch5.F5)
111. [\(a\)Normal](https://arxiv.org/html/2608.28948#Ch5.F5.sf1)
112. [\(b\)Swapped\-class](https://arxiv.org/html/2608.28948#Ch5.F5.sf2)
113. [5\.6Left: Training and validation error for successive rounds with Muon\. Right: Swapped\-class errors throughout rounds\. Even though Muon is incentivized to give relatively more gradient to less prominent features \(through higher\-rank upweighting\), it is insufficient for picking up CIFAR\.](https://arxiv.org/html/2608.28948#Ch5.F6)
114. [\(a\)Normal](https://arxiv.org/html/2608.28948#Ch5.F6.sf1)
115. [\(b\)Swapped\-class](https://arxiv.org/html/2608.28948#Ch5.F6.sf2)
116. [5\.7Linear CKA \( , \) between layers for random ensembles \(CAT\) and PGD ensembles\. The values are quite similar, suggesting that PGD is managing to find similar solutions as simple random reruns, possibly permutations of the original weights\. Hence, PGD does not capture every notion of diversity\.](https://arxiv.org/html/2608.28948#Ch5.F7)
117. [\(a\)CAT](https://arxiv.org/html/2608.28948#Ch5.F7.sf1)
118. [\(b\)PGD](https://arxiv.org/html/2608.28948#Ch5.F7.sf2)
119. [5\.8CKA for representation vectors and gradient vectors for the task loss alone at different layers for image classification\. Left: We see that representation and gradient CKA are reasonably high for CAT\. Right: The same, but for the CKA\-trained models\. We see that using CKA greatly affects the representation CKA as it optimizes for this, and it also affects the task gradient CKA\.](https://arxiv.org/html/2608.28948#Ch5.F8)
120. [\(a\)CAT](https://arxiv.org/html/2608.28948#Ch5.F8.sf1)
121. [\(b\)CKA](https://arxiv.org/html/2608.28948#Ch5.F8.sf2)
122. [5\.9CKA between the gradient vectors for the task loss alone at different layers for language modeling\. Left: We see CKA between representations and between gradients, which are both high for the CAT models\. Right: The same, but for the CKA\-trained models\. We see that while representation CKA is low, gradient CKA is unaffected by our CKA objective, unlike image classification\.](https://arxiv.org/html/2608.28948#Ch5.F9)
123. [\(a\)CAT](https://arxiv.org/html/2608.28948#Ch5.F9.sf1)
124. [\(b\)CKA](https://arxiv.org/html/2608.28948#Ch5.F9.sf2)
125. [A\.1Training loss curves for all tasks\. When cross\-referencing with Figure , we see that the first split point that leads to a convex hull is quite early, long before the loss has converged, and often before it has even halved\.](https://arxiv.org/html/2608.28948#A1.F1)
126. [\(a\)SIREN](https://arxiv.org/html/2608.28948#A1.F1.sf1)
127. [\(b\)LeNet](https://arxiv.org/html/2608.28948#A1.F1.sf2)
128. [\(c\)LeNet\-m](https://arxiv.org/html/2608.28948#A1.F1.sf3)
129. [\(d\)ResNet](https://arxiv.org/html/2608.28948#A1.F1.sf4)
130. [B\.1Spectral dynamics of Pythia suite\. From top to bottom we examine the 160m, 410m, 1\.4b and 2\.8b parameter models\. Notably, much less noise appears in the alignment plot with increasing scale\. Presumably this could be due to the fact that larger dimensional vectors have higher probability to be orthogonal, which may play a role in making optimization easier\. We see stronger alignment score \(Eqn\. \) in all layers in the larger model, perhaps because of that cleaner signal\.](https://arxiv.org/html/2608.28948#A2.F1)
131. [\(a\)Val\. Loss](https://arxiv.org/html/2608.28948#A2.F1.sf1)
132. [\(b\)SVs](https://arxiv.org/html/2608.28948#A2.F1.sf2)
133. [\(c\)Eff\. Rank](https://arxiv.org/html/2608.28948#A2.F1.sf3)
134. [\(d\)Alignment](https://arxiv.org/html/2608.28948#A2.F1.sf4)
135. [\(e\)Align\. Score](https://arxiv.org/html/2608.28948#A2.F1.sf5)
136. [B\.2Diagonal of alignment for a single pair over time \(Eqn\. \) and alignment metric across pairs of matrices over time \(Eqn\. \) where the y\-axis represents depth\. From top to bottom, for VGG we use coefficients \{ 0 , 0\.001 , 0\.01 , 0\.1 \} , while for other networks we use coefficients \{ 0 , 0\.1 , 1 , 10 \} \. We see that the maximum alignment magnitude is higher with large weight decay, and in particular, the Transformer has the strongest alignment even when nonlinearities separate the MLP layers\.](https://arxiv.org/html/2608.28948#A2.F2)
137. [\(a\)VGG](https://arxiv.org/html/2608.28948#A2.F2.sf1)
138. [\(b\)UNet](https://arxiv.org/html/2608.28948#A2.F2.sf2)
139. [\(c\)LSTM](https://arxiv.org/html/2608.28948#A2.F2.sf3)
140. [\(d\)Transformer](https://arxiv.org/html/2608.28948#A2.F2.sf4)
141. [B\.3Training loss over time, where the rows use differing amounts of weight decay\. From top to bottom, for VGG we use coefficients \{ 0 , 0\.001 , 0\.01 , 0\.1 \} , while for other networks we use coefficients \{ 0 , 0\.1 , 1 , 10 \} \. We see that it is still possible to achieve low training loss under high weight decay, and as we increase the amount of weight decay, the gap between pruned and unpruned parameters closes, lending support to the idea that the parameters become lower rank\.](https://arxiv.org/html/2608.28948#A2.F3)
142. [\(a\)VGG](https://arxiv.org/html/2608.28948#A2.F3.sf1)
143. [\(b\)UNet](https://arxiv.org/html/2608.28948#A2.F3.sf2)
144. [\(c\)LSTM](https://arxiv.org/html/2608.28948#A2.F3.sf3)
145. [\(d\)Transformer](https://arxiv.org/html/2608.28948#A2.F3.sf4)
146. [B\.4Grokking and Spectral Dynamics in Modular addition\. Top row: 30% data and no weight decay\. 2nd row: 30% data and weight decay 1\.0 \(grokking\), using hyperparameters from \( \) \. 3rd row: 70% data with no weight decay \(slingshot\), using hyperparameters from \( \) \. Bottom row: 90% data and no weight decay\. \(a\): Training and validation error\. \(b\): Singular value evolution is visualized for the first attention parameter, where each line represents a single singular value and the color represents the rank\. \(c\): Effective rank of all layers \(Eqn\. \)\. \(d\): Alignment \(Eqn\. \) between the embedding and the first attention parameter is also visualized, where the y\-axis corresponds to index i of the diagonal\. One can see that grokking co\-occurs with low\-rank weights\. In addition, there is an alignment that begins early in training that evolves up the diagonal\. Without weight decay and with less data, neither grokking nor the other phenomena occur during the entire training budget, but using more data, even without weight decay, leads to low\-rank solutions from the beginning of training\. The slingshot case follows a similar trend, though the validation loss is gradually fit\. Across cases with good generalization, parameters are lower rank, and alignment is also more prevalent in the top ranks\.](https://arxiv.org/html/2608.28948#A2.F4)
147. [\(a\)Error](https://arxiv.org/html/2608.28948#A2.F4.sf1)
148. [\(b\)SV Evolution](https://arxiv.org/html/2608.28948#A2.F4.sf2)
149. [\(c\)Effective Rank](https://arxiv.org/html/2608.28948#A2.F4.sf3)
150. [\(d\)Alignment](https://arxiv.org/html/2608.28948#A2.F4.sf4)
151. [B\.5Varying initialization scale affects the dynamics of an MLP trained on CIFAR10\. From top to bottom we use default PyTorch initialization multiplied by the constants \{ 0\.001 , 0\.01 , 0\.05 , 0\.1 , 0\.5 , 1\.0 \} \. Notably, alignment occurs slightly more strongly with smaller initialization, as predicted by \( \) for deep linear models\.](https://arxiv.org/html/2608.28948#A2.F5)
152. [\(a\)Val\. Err](https://arxiv.org/html/2608.28948#A2.F5.sf1)
153. [\(b\)SVs](https://arxiv.org/html/2608.28948#A2.F5.sf2)
154. [\(c\)Eff\. Rank](https://arxiv.org/html/2608.28948#A2.F5.sf3)
155. [\(d\)Alignment](https://arxiv.org/html/2608.28948#A2.F5.sf4)
156. [\(e\)Align\. Score](https://arxiv.org/html/2608.28948#A2.F5.sf5)
157. [B\.6The effect of varying learning rate on an MLP trained on CIFAR10\. From top to bottom we use learning rates \{ 0\.001 , 0\.01 , 0\.1 \} \. Notably, alignment increases with larger learning rate, agreeing with the deep linear setting in \( \) \.](https://arxiv.org/html/2608.28948#A2.F6)
158. [\(a\)Val\. Err](https://arxiv.org/html/2608.28948#A2.F6.sf1)
159. [\(b\)SVs](https://arxiv.org/html/2608.28948#A2.F6.sf2)
160. [\(c\)Eff\. Rank](https://arxiv.org/html/2608.28948#A2.F6.sf3)
161. [\(d\)Alignment](https://arxiv.org/html/2608.28948#A2.F6.sf4)
162. [\(e\)Align\. Score](https://arxiv.org/html/2608.28948#A2.F6.sf5)
163. [B\.7Plots for averages of 3 seeds of grokking modular addition with Transformers \(Figure \)\. We see a stark transition to low\-rank when validation error decreases\.](https://arxiv.org/html/2608.28948#A2.F7)
164. [\(a\)Error](https://arxiv.org/html/2608.28948#A2.F7.sf1)
165. [\(b\)SV Evolution](https://arxiv.org/html/2608.28948#A2.F7.sf2)
166. [\(c\)Effective Rank](https://arxiv.org/html/2608.28948#A2.F7.sf3)
167. [\(d\)Alignment](https://arxiv.org/html/2608.28948#A2.F7.sf4)
168. [B\.8Alignment matrix \(Eqn\. \) over time\. We see that the majority of signal is concentrated in the diagonal, an example of why we consider the diagonal in the rest of the work\.](https://arxiv.org/html/2608.28948#A2.F8)
169. [\(a\)Init](https://arxiv.org/html/2608.28948#A2.F8.sf1)
170. [\(b\)Ep\. 2500](https://arxiv.org/html/2608.28948#A2.F8.sf2)
171. [\(c\)Ep\. 5000](https://arxiv.org/html/2608.28948#A2.F8.sf3)
172. [\(d\)Ep\. 5500](https://arxiv.org/html/2608.28948#A2.F8.sf4)
173. [\(e\)Ep\. 6000](https://arxiv.org/html/2608.28948#A2.F8.sf5)
174. [\(f\)Ep\. 7000](https://arxiv.org/html/2608.28948#A2.F8.sf6)
175. [\(g\)Ep\. 8000](https://arxiv.org/html/2608.28948#A2.F8.sf7)
176. [B\.9Different architectures all trained on CIFAR10\. From top to bottom, MLP, LeNet\-5 and VGG\-16\. We see roughly consistent spectral dynamics across architectures within the same task\.](https://arxiv.org/html/2608.28948#A2.F9)
177. [\(a\)Val\. Err](https://arxiv.org/html/2608.28948#A2.F9.sf1)
178. [\(b\)SVs](https://arxiv.org/html/2608.28948#A2.F9.sf2)
179. [\(c\)Eff\. Rank](https://arxiv.org/html/2608.28948#A2.F9.sf3)
180. [\(d\)Alignment](https://arxiv.org/html/2608.28948#A2.F9.sf4)
181. [\(e\)Align\. Score](https://arxiv.org/html/2608.28948#A2.F9.sf5)
182. [B\.10Duplicate of Figure with shared color scaling between settings\.](https://arxiv.org/html/2608.28948#A2.F10)
183. [\(a\)VGG](https://arxiv.org/html/2608.28948#A2.F10.sf1)
184. [\(b\)UNet](https://arxiv.org/html/2608.28948#A2.F10.sf2)
185. [\(c\)LSTM](https://arxiv.org/html/2608.28948#A2.F10.sf3)
186. [\(d\)Transformer](https://arxiv.org/html/2608.28948#A2.F10.sf4)

## Chapter 1Introduction

In the past decade, neural networks have exploded onto the scene\. What started with early experiments in speech recognition\([Seide et al\., 2011](https://arxiv.org/html/2608.28948#bib.bib177);[Hinton et al\., 2012](https://arxiv.org/html/2608.28948#bib.bib70)\)and image classification\([Krizhevsky et al\., 2012](https://arxiv.org/html/2608.28948#bib.bib103);[Russakovsky et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib169)\)has led to massive language models\([Achiam et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib2)\)\. The transition from research to product is firmly underway: ChatGPT, at its core a language model, is one of the fastest growing applications ever\([Ouyang et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib144);[OpenAI, 2022](https://arxiv.org/html/2608.28948#bib.bib235)\)\. In the past few years the model deployment has continued relentlessly, with the rise of coding assistants\([Chen et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib27);[Anthropic, 2025](https://arxiv.org/html/2608.28948#bib.bib236)\)\. Self\-driving cars are deployed in many cities\([Kusano et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib107)\)\. Weather models based on large\-scale neural networks have supplanted traditional methods\([Alet et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib5)\)\. Large campaigns to revolutionize fields are underway in drug discovery\([Jumper et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib91)\)and materials science\([Merchant et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib129)\)\. Genuinely novel mathematical results have now been proven entirely by machine\([OpenAI, 2026](https://arxiv.org/html/2608.28948#bib.bib143)\)\. We are in the midst of a dizzying and breathtaking transformation in technology\.

Along with the tremendous amount of research in the field in the past decade, there has been an enormous capital investment\([KKR Global Macro & Asset Allocation, 2025](https://arxiv.org/html/2608.28948#bib.bib99)\)\. This is due to a fundamental observation since the ImageNet competition\([Russakovsky et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib169)\)that larger models tend to perform better\([Krizhevsky et al\., 2012](https://arxiv.org/html/2608.28948#bib.bib103);[Simonyan and Zisserman, 2014](https://arxiv.org/html/2608.28948#bib.bib185);[He et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib67)\)\. Such a trend continued\([Sun et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib191)\), leading to its formalization as scaling laws\([Hestness et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib69);[Kaplan et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib93)\)for predicting increase in performance with both model size and data\. The natural conclusion, then, is that more data and more computation means a better problem solver\. Thus there is a straightforward recipe for converting dollars into better models by buying more computers and paying for more data\. Taken to its limit, this has led neural networks to occupy a gigantic place in our economy\([Tunguz, 2026](https://arxiv.org/html/2608.28948#bib.bib200)\)\.

Yet, as these algorithms penetrate all aspects of our societies, with the benefits there are commensurately large risks and social repercussions\([Bender et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib14);[Bommasani et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib22)\)\. When employment, health, and legal questions are decided by these algorithms\([D’Amour et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib36)\), it is critical to have some sense of how they work\. Moreover, without a fundamental understanding of deep learning, it will be very difficult to solve such questions\.

As an example, take simplicity bias\([Rahaman et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib161);[Shah et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib178);[Pezeshki et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib153)\)\. This very basic issue, where neural networks latch on to simple aspects of the data even when more complex features may be useful, was first investigated at a small scale\. Yet at a large scale, it transforms into discussions of biases against social groups\([Bender et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib14);[D’Amour et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib36)\)and adversarial examples\([Hendrycks et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib68)\)\. The fundamental hammer that we have to combat something like simplicity bias is a relatively simple one: collect more data and hope scaling laws drive performance\([Hestness et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib69);[Kaplan et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib93);[Mazeika et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib128)\)\. Through this process, we close the gap between train and test scenarios\. Yet such an approach hits practical walls in changing distributions\([Zhang and Bottou, 2023](https://arxiv.org/html/2608.28948#bib.bib230)\), requires extreme amounts of data, and is scientifically unsatisfying\. That is to say, despite the ever\-growing investment, the hole in our understanding bites us\.

And this is only one such issue\. We know that very simple neural networks can represent any function of interest\([Cybenko, 1989](https://arxiv.org/html/2608.28948#bib.bib35)\), but there remain many mysteries in actually finding these solutions\. Even though it can be framed very straightforwardly, the actual optimization problem can have complex hidden qualities, often called implicit bias\([Neyshabur et al\., 2014](https://arxiv.org/html/2608.28948#bib.bib139)\)\. Though ridge regression is well understood\([Marquardt and Snee, 1975](https://arxiv.org/html/2608.28948#bib.bib125)\), basic questions on how to generalize this to neural networks with weight decay\([Hanson and Pratt, 1988](https://arxiv.org/html/2608.28948#bib.bib65);[Krogh and Hertz, 1991](https://arxiv.org/html/2608.28948#bib.bib104);[Zhang et al\., 2018a](https://arxiv.org/html/2608.28948#bib.bib227)\)have only partial answers\([Van Laarhoven, 2017](https://arxiv.org/html/2608.28948#bib.bib201);[Andriushchenko et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib6);[Yaras et al\., 2023b](https://arxiv.org/html/2608.28948#bib.bib220)\)\. Perhaps most vexing, we lack a complete explanation for how neural networks generalize, despite having the capacity to perfectly memorize the training data\([Zhang et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib228)\)\. Such an explanation may allow us to design better algorithms\. However, a lack of understanding makes the deployment of neural networks vulnerable to uninterpretable errors\([Szegedy et al\., 2013](https://arxiv.org/html/2608.28948#bib.bib193);[Ilyas et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib80);[Hendrycks et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib68);[D’Amour et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib36);[Zou et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib234)\)\.

Much work has been done toward this understanding\. There is a large body of theoretical work on many of the preceding topics, but these studies are often limited to special settings like deep linear networks\([Arora et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib7);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8)\)or infinite\-width systems\([Jacot et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib84)\), and arguments rely on unsubstantiated or impractical assumptions like near\-zero initialization\. On the empirical side, a growing body of work in interpretability has attempted to reverse\-engineer neural networks\([Rahaman et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib161);[Barak et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib10);[Nanda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib138)\), but given the difficulty of the task, researchers focus on small scale with bespoke methodology that is challenging to scale\. A body of empirical work aims to understand the behavior of larger networks\([Zhang et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib228);[Huh et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib76);[Yu and Wu, 2023](https://arxiv.org/html/2608.28948#bib.bib222)\), and compromises by focusing on more abstract objects like the gram matrix\([Huh et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib76)\)or the Neural Tangent Kernel \(NTK\)\([Fort et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib50)\)\. But due to the complexity of contemporary networks, it is very difficult to connect this broad body of literature into a coherent view, and many phenomena remain poorly understood\.

We are interested in a broad understanding of neural networks that can be applied to practical systems\. Through that understanding we aim at a more principled way to deal with distribution shifts and low data scenarios\. In the following chapters, we describe a line of work that starts with a confusion regarding the loss landscape and ends with a simple definition of feature learning\. We close with a method that uses this definition to overcome simplicity bias and break the data wall of deep learning\.

### 1\.1Contributions

This dissertation is structured as follows:

- •Chapter[2](https://arxiv.org/html/2608.28948#Ch2): We give introductory background around the core phenomenon of Linear Mode Connectivity \(LMC\) that drives the trajectory of the following chapters\.
- •Chapter[3](https://arxiv.org/html/2608.28948#Ch3): We show that there are high dimensional, diverse convex modes in the loss landscape of many neural networks that cannot be trivially explained, extending the phenomenon of linear mode connectivity\.
- •Chapter[4](https://arxiv.org/html/2608.28948#Ch4): We find that these high dimensional modes can be explained by stable low rank dynamics of top singular vectors \(spectral dynamics\) and that these dynamics unify a host of confusing phenomena related to generalization and regularization\.
- •Chapter[5](https://arxiv.org/html/2608.28948#Ch5): We use spectral dynamics to improve the training of neural networks, in particular by overcoming simplicity bias through the use of ensembles, with direct implications for low data regimes\.
- •Chapter[6](https://arxiv.org/html/2608.28948#Ch6): We conclude by discussing the implications of these results at the time of their development, and in the years to come\.

## Chapter 2Background

Neural networks are omnipresent today, and we have known for a very long time that they can approximate any function of interest\([Cybenko, 1989](https://arxiv.org/html/2608.28948#bib.bib35)\)\. Yet in theory their optimization is difficult\([Blum and Rivest, 1988](https://arxiv.org/html/2608.28948#bib.bib20)\), so the critical question ishowwe actually find the optimal solution\. It is well known that many, often infinite minima exist in the neural network loss landscape\([Choromanska et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib32);[Kawaguchi, 2016](https://arxiv.org/html/2608.28948#bib.bib95)\)\. Yet, only some solutions generalize well\([Zhang et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib228);[Keskar et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib96)\), raising questions about how and why common optimization algorithms find the solutions they do and how one might design algorithms that find better solutions\.

One way to understand optimization is through the loss landscape, the total set of trajectories a neural network can traverse\. Many researchers have studied this landscape in the past\([Keskar et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib96);[Dinh et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib39);[Li et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib112);[Jiang et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib87);[Fort and Jastrzebski, 2019](https://arxiv.org/html/2608.28948#bib.bib49)\)\. The vast literature is due to a variety of strange properties neural network loss landscapes exhibit which are not present in prior models, due to nonconvex structure of the landscape\. To explain this strangeness we have to define convex structure\.

\(a\)Convex Loss![Refer to caption](https://arxiv.org/html/2608.28948v1/nonconvex.png)\(b\)Nonconvex Loss

Figure 2\.1:Schematic of different varieties of loss landscapes\. Color denotes height, where lower is better\.\(a\): Convex loss\. It is possible to find a path to the global minimum through local steps, when a ball is dropped in the landscape it will roll to the bottom\.\(b\): Nonconvex loss\. Unlike the convex case, a ball may get stuck in a particular local minimum without finding the global minimum\. Neural networks are generally nonconvex, making optimization more difficult than the convex case\.Prior to neural networks, a large body of research focused onconvexoptimization problems\. Such problems are, intuitively, defined by the ability to connect any two minima by a linear path, and find equivalent or better solutions along that path\. To be precise, ifℒ\\mathcal\{L\}is aloss functionwhere lower indicates better quality,W1W\_\{1\}andW2W\_\{2\}are two sets of parameters for a neural network, andα∈\[0,1\]\\alpha\\in\[0,1\]:

ℒ⁡\(\(1−α\)​W1\+α​W2\)≤\(1−α\)​ℒ​\(W1\)\+α​ℒ​\(W2\)\.\\mathcal\{L\}\(\(1\-\\alpha\)W\_\{1\}\+\\alpha W\_\{2\}\)\\leq\(1\-\\alpha\)\\mathcal\{L\}\(W\_\{1\}\)\+\\alpha\\mathcal\{L\}\(W\_\{2\}\)\.\(2\.1\)In particular, this implies that no matter where one begins in the landscape, it is always possible to find a path to an optimum via small steps without getting stuck\. If you drop a ball in this landscape, it will eventually roll to the bottom\. Such a property allows for a complex and beautiful theory of optimization, and extremely efficient algorithms\([Vandenberghe, 2004](https://arxiv.org/html/2608.28948#bib.bib202)\)\. See Figure[2\.1](https://arxiv.org/html/2608.28948#Ch2.F1)\(a\) for an intuitive picture\.

Unfortunately, useful neural networks are nonconvex\. This means that there are many possible optima in the loss landscape not connected by paths, like separate craters in the surface \(see Figure[2\.1](https://arxiv.org/html/2608.28948#Ch2.F1)\(b\)\)\. In particular, it is possible to get stuck at suboptimal points depending on where one starts, and how one descends\. Besides being provable in specific cases\([Sontag and Sussmann, 1989](https://arxiv.org/html/2608.28948#bib.bib190)\), this is also empirically confirmed for neural networks of interest\([Li et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib112);[Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\. The rough intuition is that, due to the multiple layers of linear transforms, it is possible to permute one layer, then permute the following in a corresponding way such as to not alter the solution\. The same is true for rescaling, but linearly interpolating between these solutions is worse than either endpoint\([Dinh et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib39)\)\. Moreover, different hyperparameters can lead to different minima\([Keskar et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib96)\)\.

Despite the nonconvexity in the loss landscape, one interesting empirical property that has been observed is mode connectivity which suggests more structure than arbitrary complexity\. Mode connectivity refers to the ability to findnonlinearpaths of low loss that connect two minima in the loss landscape that were derived from the same initialization\.[Draxler et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib41)and[Garipov et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib57)found that such paths could be quite simple in structure: piecewise continuous with only a few segments\. The ability to connect different parts of the loss landscape may appear surprising due to the nonconvexity, but theoretical work showed that, with sufficient capacity, it is possible to find paths of even piecewise linear structure\([Kuditipudi et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib105)\)through smoothly pushing one optimum into the excess capacity, then slowly interpolating to the other\.

However, such a theoretical argument still requires nonlinear paths\. More surprising, then, was the discovery oflinearmode connectivity \(LMC\), where two optima started from the same initialization could be connected by alinearpath of low loss\([Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136)\)\. The existence of this linear path implies that optimization appears to stay in a convex subspace, contrary to the seeming nonconvexity we have been discussing\. Instead of exploring the entire landscape, the behavior of optimization appears quite structured\.

Yet Nagarajan and Kolter’s results only applied to a very small\-scale experiment\. It was unclear at the time whether they were specific to some experimental details, or a more general property of neural networks\.[Frankle et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib52)showed that linear mode connectivity indeed held true for a larger class of neural networks under a modified protocol, indicating the observation was nontrivial\. Their modified experiment added a small amount of shared optimization before splitting the two trajectories\. After splitting, they train both as usual to their stopping points\. This process differs slightly from that of Nagarajan and Kolter due to the shared optimization, but still indicates that optimization is quite well\-behaved after a short amount of time\.

It is worth asking whether the observation made by[Frankle et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib52)is trivial: is the apparent convexity due to very small distances after the shared optimization? Even the nonconvex loss functions should be locally convex when one zooms in far enough, so this explanation could make the phenomenon completely uninteresting\. The evidence, however, suggests this is not the case as the distance between initialization and the end of shared training \(the split point\) is of the same magnitude as that between the split point and optima, and between optima\. So LMC is a nontrivial kind of convexity, a point which we will reinforce in our experiments\.

Additionally, LMC seems to appear broadly across more optimization settings and architectures\.[Neyshabur et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib140)showed that LMC is also present when the initialization is taken from a pretrained model on a separate task\. Thus the optimization landscape is limited by prior training on very loosely\-related tasks\.[Juneja et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib92)demonstrated similar results for Transformers in language tasks, beyond the simple image classification setting used for all prior work\.

Though the phenomenon is intriguing in its own right, there are also practical applications to LMC beyond understanding the loss landscape\.[Wortsman et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib215)and[Li et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib114)showed that the convexity suggested by LMC can lead to better model performance via finding the optimal point in between solutions \(model averaging\)\.[Rame et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib163)found that such model averaging could be useful beyond a single task for out\-of\-distribution generalization\. Moreover[Ilharco et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib79)demonstrated that model\-averaging could be used to combine tasks into a single model\.

So LMC is a surprising phenomenon given the seeming nonconvexity of optimization\. It is also a general phenomenon and does not have trivial theoretical explanations\. Additionally, LMC can be leveraged for practical benefits\. Yet the question remains: why does LMC occur? In the following chapters, we investigate this question in detail, finding that the explanation is connected to a host of other phenomena in the literature through low\-rank dynamics\. Which in turn allows us to propose a new method for improved feature learning\. Let us begin with LMC\.

## Chapter 3On Convexity and Linear Mode Connectivity in Neural Networks [Yunis et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib223)

In many cases, neural networks trained with stochastic gradient descent \(SGD\) that share an early and often small portion of the training trajectory have solutions connected by a linear path of low loss\. This phenomenon, called linear mode connectivity \(LMC\), has been leveraged for pruning and model averaging in large neural network models, but it is not well understood how broadly or why it occurs\. LMC suggests that SGD trajectories somehow end up in a“convex”region of the loss landscape and stay there\. In this work, we confirm that this eventually does happen by finding a high\-dimensional convex hull of low loss between the endpoints of several SGD trajectories\. But to our surprise, simple measures of convexity do not show any obvious transition at the point when SGD will converge into this region\. To understand this convex hull better, we investigate the functional behaviors of its endpoints\. We find that only a small number of correct predictions are shared between all endpoints of a hull, and an even smaller number of correct predictions are shared between the hulls, even when the final accuracy is high for every endpoint\. Thus, we tie LMC more tightly to convexity, and raise several new questions about the source of this convexity in neural network optimization\.

### 3\.1Introduction

Many researchers have studied the loss landscape of neural networks\([Keskar et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib96);[Li et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib112);[Dinh et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib39);[Jiang et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib87);[Neyshabur et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib140);[Fort and Jastrzebski, 2019](https://arxiv.org/html/2608.28948#bib.bib49)\)in the past decade\. Linear mode connectivity \(LMC\), i\.e\., the existence of linear paths of low loss between solutions of optimization, was likely first observed by[Nagarajan and Kolter \(2019\)](https://arxiv.org/html/2608.28948#bib.bib136)between two small networks with the same initialization but different data orders for stochastic gradient descent \(SGD\)\.[Frankle et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib52)subsequently demonstrated that LMC occurs in larger vision models when branch trajectories with different data orders are split from a shared early portion of training with a common data order\.

Such LMC behavior would be expected if the loss landscape were convex, as, by definition, any two points in a convex region could be connected by a linear path of lower loss\. Even if the entire loss landscape were non\-convex, but SGD converged to a convex region at some point in training, LMC would still hold due to local convexity\. Given this simple explanation, and possible specificity of prior work, we explore across a broader range of architectures and optimizers, asking several questions regarding LMC:

- •Is LMC really indicating that the loss surface is convex between the endpoints?We find a convex hull of low loss defined by the endpoints of many SGD trajectories and show that its dimension is large\. This shows that LMC is not a property of any two endpoints, but the entire loss\-landscape around a“mode”indeed looks convex\. This gives more credence to the explanation for LMC through convexity\.
- •Is the optimization problem approximately convex after the point at which we expect LMC?We see that the Hessian nearly always has negative eigenvalues but their relative magnitude doesn’t change much, and in between stochastic updates the training loss looks convex\. Neither of these metrics changes significantly throughout training, so even if they indicate optimization is nearly convex, they fail to identify the start of LMC\.
- •How are the functions corresponding to linearly connected parameters related?We probe deeper into the source of LMC\. We see that the number of shared correct predictions between convex hulls is not close to the shared correct predictions among endpoints of a hull, even when the training accuracy is high, suggesting that the ability to interpolate between parameters is only loosely related to functional similarity\.

### 3\.2Convex Mode Connectivity

Initialize weights with

W0W\_\{0\}, and seed for data order with

z0z\_\{0\};

Train

kksteps with algorithm

𝒜\\mathcal\{A\}to yield

Wk=𝒜k​\(W0,z0\)W\_\{k\}=\\mathcal\{A\}\_\{k\}\(W\_\{0\},z\_\{0\}\);

Sample

PPnew data order seeds

z1,…,zPz\_\{1\},\\ldots,z\_\{P\};

Train for

TTsteps:

WT1=𝒜T​\(Wk,z1\),…,WTP=𝒜T​\(Wk,zP\)W\_\{T\}^\{1\}=\\mathcal\{A\}\_\{T\}\(W\_\{k\},z\_\{1\}\),\\ldots,W\_\{T\}^\{P\}=\\mathcal\{A\}\_\{T\}\(W\_\{k\},z\_\{P\}\);

for*j=1,…,Jj=1,\\ldots,J*do

Sample

aj∼Unif​\(Δn\)a^\{j\}\\sim\\textrm\{Unif\}\(\\Delta\_\{n\}\)and compute

Wconvj=∑i=1Paij​WTiW\_\{\\text\{conv\}\}^\{j\}=\\sum\_\{i=1\}^\{P\}a\_\{i\}^\{j\}W\_\{T\}^\{i\};

Find closest endpoint

Wendj=arg​minWTi⁡∥Wconvj−WTi∥W\_\{\\text\{end\}\}^\{j\}=\\argmin\_\{W\_\{T\}^\{i\}\}\\lVert W\_\{\\text\{conv\}\}^\{j\}\-W\_\{T\}^\{i\}\\rVert;

Sample random perturbation

ϵj∼𝒩⁡\(0,I\)\\epsilon^\{j\}\\sim\\mathcal\{N\}\(0,I\);

Compute perturbed parameter

Wrandj=Wendj\+∥Wendj−Wconvj∥∥ϵj∥​ϵjW\_\{\\text\{rand\}\}^\{j\}=W\_\{\\text\{end\}\}^\{j\}\+\\frac\{\\lVert W\_\{\\text\{end\}\}^\{j\}\-W\_\{\\text\{conv\}\}^\{j\}\\rVert\}\{\\lVert\\epsilon^\{j\}\\rVert\}\\epsilon^\{j\};

Measure

Iconvj=𝟙\[ℒ\(Wconvj\)<ℒ\(Wendj\)\]I\_\{\\text\{conv\}\}^\{j\}=\\mathbbm\{1\}\[\\mathcal\{L\}\(W\_\{\\text\{conv\}\}^\{j\}\)<\\mathcal\{L\}\(W\_\{\\text\{end\}\}^\{j\}\)\]and

Irandj=𝟙\[ℒ\(Wrandj\)<ℒ\(Wendj\)\]I\_\{\\text\{rand\}\}^\{j\}=\\mathbbm\{1\}\[\\mathcal\{L\}\(W\_\{\\text\{rand\}\}^\{j\}\)<\\mathcal\{L\}\(W\_\{\\text\{end\}\}^\{j\}\)\];

end for

Compute

Pconv=1J​∑jIconvjP\_\{\\text\{conv\}\}=\\frac\{1\}\{J\}\\sum\_\{j\}I\_\{\\text\{conv\}\}^\{j\}and

Prand=1J​∑jIrandjP\_\{\\text\{rand\}\}=\\frac\{1\}\{J\}\\sum\_\{j\}I\_\{\\text\{rand\}\}^\{j\};

Algorithm 1Convex mode procedure
WT1W\_\{T\}^\{1\}WT2W\_\{T\}^\{2\}WT3W\_\{T\}^\{3\}WT4W\_\{T\}^\{4\}WT5W\_\{T\}^\{5\}WkW\_\{k\}Wconv′W\_\{\\textrm\{conv\}\}^\{\\prime\}Wrand′W\_\{\\textrm\{rand\}\}^\{\\prime\}
Figure 3\.1:The algorithmic procedure and graphic for measuring convex mode connectivity\.Linear mode connectivity was defined for a pair of branch trajectories\. Instead of a pair, we split intoPPbranches and sample convex combinations of the endpoints to better understand the loss inside the convex hull of these endpoints\. Algorithm[1](https://arxiv.org/html/2608.28948#algorithm1)presents this convex mode connectivity \(CMC\) procedure, which is a slight modification of the original LMC procedure\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\.

In Figure[3\.2](https://arxiv.org/html/2608.28948#Ch3.F2), we plot the probability that convex combinations of hull parameters have better training loss than hull endpoints, and likewise for random perturbations around hull endpoints\. Notably, like LMC, after some point in training, random convex combinations become better than endpoint averages, while random perturbations are almost always worse—strong evidence that LMC describes a convex region in the loss surface\. We call the time after this first point in training theLMC regime\. We verify these observations in a regression task \(SIREN\) and a few different networks trained to classify CIFAR\-10\. For more details on the task selection, see Appendix[A\.1](https://arxiv.org/html/2608.28948#A1.S1)\.

\(a\)SIREN\(b\)LeNet\(c\)LeNet\-m\(d\)ResNet
Figure 3\.2:Top row:The empirical probabilityPconvP\_\{\\text\{conv\}\}that a sample inside the convex hull \(Convex\) or a random perturbationPrandP\_\{\\text\{rand\}\}around the endpoint \(Random\) have lower loss than their closest endpoints vs\. the split epoch \(kkin Algorithm[1](https://arxiv.org/html/2608.28948#algorithm1)\) for the many different trajectories\. After an initial phase of training, the probability of lower loss inside the hull quickly converges to one, the loss inside the hull is always better than at the endpoints\.Bottom row:Parameter distances between pairs of points in the CMC procedure\. The distance between pairs of endpoints is of the same order of magnitude as other distances for all tasks\. In other words, the apparent scale of convexity is similar to that of the total trajectory\. Error bars for probabilities are computed by empirical standard deviationsstd​\(Iconvj\)\\text\{std\}\(I\_\{\\text\{conv\}\}^\{j\}\)andstd​\(Irandj\)\\text\{std\}\(I\_\{\\text\{rand\}\}^\{j\}\), hence are not meaningful when values stretch greater than 1 or less than 0\.In order to understand the space that this mode takes up in parameter space, we stack all the parametersW1,…,WnW\_\{1\},\\ldots,W\_\{n\}into a matrixA=\[flatten\(W1\)⊤⋯flatten\(Wn\)⊤\]⊤A=\[\\textrm\{flatten\}\(W\_\{1\}\)^\{\\top\}\\cdots\\textrm\{flatten\}\(W\_\{n\}\)^\{\\top\}\]^\{\\top\}and take its SVD\. We then plot the singular values of this matrix as a measure of the volume spanned by the endpoints for the different tasks\. We see in Figure[3\.3](https://arxiv.org/html/2608.28948#Ch3.F3), that for all tasks for which we train sufficient branches, there does not appear to be a sharp cutoff in the magnitude plot corresponding to the number of classes or the hidden dimension\. The only cutoffs occur when there are not enough endpoints to see any higher dimension \(PPpoints define an object of dimension at mostP−1P\-1\)\. This tells us that we are seeing a non\-degenerate convex mode in the loss landscape, and that the particular region does not seem related to the Hessian subspace identified by[Gur\-Ari et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib64), or to other simplices of low loss achieved via additional optimization\([Fort et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib48);[Benton et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib15)\)\.

\(a\)SIREN\(b\)LeNet\(c\)LeNet\-m\(d\)ResNet
Figure 3\.3:Singular values for the matrix of stacked endpoints of the mode\. We examine endpoints split after the point when the hull center performs better than the endpoints \(see Figure[3\.2](https://arxiv.org/html/2608.28948#Ch3.F2)\)\. ForPPpoints in space, the shape they define can have at mostP−1P\-1dimensions\. We see in all cases that the magnitudes do not decay to zero until componentP−1P\-1\. In the case that there areKKoutputs andP<KP<K\(ResNet\), we see a roughly uniform mode\. In cases withP\>KP\>K\(SIREN, LeNet, and LeNet\-m\), it does not appear that there is any special fragmenting atKK\. In particular, the LeNet plot is over 300 endpoints, and the tail does not look as if it will decay to zero any time soon\. The computational expense of these experiments limits how many endpoints we train\.
### 3\.3Measures of Convexity

One natural explanation for the previously demonstrated convex hulls is that, after some point in training, the trajectory enters a convex “basin” and the rest of the time is spent inside this region\. However, previous work has shown that the linear interpolation between points early on the training trajectory and at the end can exhibit large increases in training loss\([Frankle, 2020](https://arxiv.org/html/2608.28948#bib.bib53);[Vlaar and Frankle, 2022](https://arxiv.org/html/2608.28948#bib.bib206)\), making such an explanation tenuous\.

Still, the loss could locally be behaving more“convexly”after some early point in training\. In order to test this, we examine the bottom part of the Hessian spectrum and the convexity of the training loss between two possible stochastic updates at a point \(see Appendix[A\.2](https://arxiv.org/html/2608.28948#A1.S2)for details\)\. We see in Figure[3\.4](https://arxiv.org/html/2608.28948#Ch3.F4)that the Hessian shows only a small amount of local non\-convexity throughout training and that the measurement of convexity between pairs of updates makes training always appear locally convex, making neural network optimization look quite well\-behaved\. None of the curves show distinguishing features at the LMC point \(see Figure[3\.2](https://arxiv.org/html/2608.28948#Ch3.F2)\)\. Rather than believing that there is a particular point in training at which a difference in SGD updates causes the trajectories to diverge, it seems the eventual convergence to different convex hulls happens much more gradually\.

\(a\)SIREN\(b\)LeNet\(c\)LeNet\-m\(d\)ResNet
Figure 3\.4:Top row:The ratio of smallest to largest eigenvalue of the Hessian throughout training\. This ratio is almost always negative, but quite small, making any local non\-convexity appear unimportant\.Bottom row:Probability of a convex line of loss between a pair of updates\. Notice that this probability is close to 1 for the entire optimization, typically before the LMC regime in Figure[3\.2](https://arxiv.org/html/2608.28948#Ch3.F2)\.
### 3\.4Diversity of Endpoints

Figure[3\.2](https://arxiv.org/html/2608.28948#Ch3.F2)\(bottom\) and prior work have shown that endpoints that exhibit LMC are far apart in parameter space with respect to the total trajectory length\([Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136);[Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\. A natural question is whether these distances are deceptive and the endpoints are actually“functionally similar”\. It is well known that with clever scaling, one can transform a set of parameters into another set, and the distance between the two will be large, while the functions they describe will remain the same\. Such scaling was employed by[Dinh et al\. \(2017\)](https://arxiv.org/html/2608.28948#bib.bib39)to make the case that two equivalent functions could have very different sharpness\. Another way to preserve functionality is through permutations of rows of weight matrices\.[Entezari et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib43)hypothesize that such permutations can map unconnected endpoints into the same mode and[Ainsworth et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib4)show this is true for wide networks, but not the case for standard ResNets, which[Benzing et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib16)concurrently confirm\. Still, these works do not characterize the behavior within a mode\.

LetCijC\_\{i\}^\{j\}be the set of correct predictions for endpointWTiW\_\{T\}^\{i\}of a convex hull, wherejjindexes over different hulls\. The set of correct predictions on the training set common to all endpoints isSj=⋂iCijS\_\{j\}=\\bigcap\_\{i\}C\_\{i\}^\{j\}\. The set of correct predictions common between hulls isS∗=⋂jSjS^\{\*\}=\\bigcap\_\{j\}S\_\{j\}\. Comparing\|S∗\|\\lvert S^\{\*\}\\rvertto𝔼⁡\[\|Sj\|\]\\mathbb\{E\}\[\\lvert S\_\{j\}\\rvert\]provides a proxy measurement of the similarity between the functions for two different hulls\. Figure[3\.5](https://arxiv.org/html/2608.28948#Ch3.F5)reveals that there is a substantial gap between\|S∗\|\\lvert S^\{\*\}\\rvert,𝔼⁡\[\|Sj\|\]\\mathbb\{E\}\[\\lvert S\_\{j\}\\rvert\], and𝔼⁡\[\|Cij\|\]\\mathbb\{E\}\[\\lvert C\_\{i\}^\{j\}\\rvert\]for functions that do not interpolate the data perfectly, suggesting only a fraction of the predictions within a mode are shared, and only a fraction of those are shared across hulls\. We also see that accuracy improves inside the convex hull, so it is not the case that the loss improvement we saw previously is driven by better performance on a common set of points shared across the whole mode\. Rather, it seems like there is a large collection of points in the data for which it is possible to flip predictions easily without incurring increases in the loss\. Additionally, the poor agreement between hulls suggests that permutations either are not a sufficient explanation for the similarity between hulls, or that they only preserve the function on a relatively small set of predictions\.

\(a\)LeNet\(b\)LeNet\-m\(c\)ResNet
Figure 3\.5:Top row:Average accuracies at the endpoints of training \(Endpoint\), inside the convex mode defined by the endpoints \(Convex\), and at random directions around endpoints \(Random\)\. Notice that after a late enough split, samples in the hull are better than the endpoints\.Bottom row:Average training accuracy over runs \(Accuracy\) compared to common points predicted correctly within a mode \(Intra mode\) and common points predicted correctly between hulls \(Inter mode\)\. We see that for models with an imperfect training accuracy, intra mode agreement is substantially lower than accuracy, and inter mode agreement is even lower\. This suggests that endpoints of a mode are not similar functions\.
### 3\.5Discussion

In this chapter, we present some dichotomous evidence for local convexity in the neural network loss landscape\. We show that trajectories that split after a point early in training define a convex hull of low loss in the loss landscape\. On one hand, this might be taken as evidence for the loss landscape being \(weakly\) convex after this early point in training\. But on the other, we cannot detect such a point with simple proxies for convexity\. In particular, our measures indicate no increase in the convexity of the function before or after the split point that gives rise to LMC\. Failure to measure a difference could be due to sparse sampling of our metrics, but it might also indicate that neural network optimization is much more convex than we might expect\. Probing the mode, it seems the convexity in training loss and error is not due to a large subset of shared predictions on the training set, but rather a diverse set of functions that are amenable to interpolation\.

Prior work has shown that, on the scale of the optimization trajectory, the neural network loss landscape does not appear convex\([Li et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib112);[Frankle, 2020](https://arxiv.org/html/2608.28948#bib.bib53);[Vlaar and Frankle, 2022](https://arxiv.org/html/2608.28948#bib.bib206)\), yet a large convex hull appears at the end of training\. Given that hulls only agree on a small subset of predictions, yet interpolation is possible, studying the behavior of the convex hull while training on the subset of the data corresponding to predictions that are and are not shared may provide more insight\.

There may also be a relationship between LMC and neural collapse\([Papyan et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib147)\), a recently identified tendency in classifiers to collapse the output feature space to a set of class means over the course of training\. It could be the case that interpolating within a given convex hull corresponds to minimal disturbances to class means, but interpolating between convex hulls corresponds to changing the location of class means in feature space\. If such a relationship existed, neural collapse might provide an early metric for detecting the first iteration after which a convex hull will exist, which would allow for parallelization of training without the extra computational cost needed to identify the LMC regime\.

The deep nature of this phenomenon, and its commonality across settings, suggests that there must be a similarly common explanation in terms of the optimization of neural networks\. So the critical question is why? Why do we see this high dimensional convex mode across all neural networks studied? And why is it still the case even given the constant nonconvexity and the functional dissimilarity?

We find a clue to answering the convex mode puzzle through the low\-rank dynamics of deep linear models\. In particular,[Arora et al\. \(2019\)](https://arxiv.org/html/2608.28948#bib.bib8)show that, under certain technical assumptions, one can derive an equation for the dynamics of singular values and vectors of the product matrix\. In effect, larger singular values grow larger faster, so there is an implicit low\-rank dynamic with increasing depth\.

Thus, we hypothesize that similar low\-rank dynamics in nonlinear neural networks are responsible for LMC\. Intuitively, the idea is that as large singular values become stable, when trajectories are split in the LMC procedure, the dynamics of those top vectors remain stable\. Thus at the end of training, models that can be averaged will share their top singular vectors\. The following chapter details this development\.

## Chapter 4Approaching Deep Learning through the Spectral Dynamics of Weights [Yunis et al\. \(2024\)](https://arxiv.org/html/2608.28948#bib.bib224)

We study the spectral dynamics of weights—the behavior of singular values and vectors during optimization—showing that they clarify and link many phenomena in deep learning\. Through extensive experiments, covering small\-scale “grokking” to large\-scale tasks like image classification with ConvNets, image generation with UNets, speech recognition with LSTMs, and language modeling with Transformers, we identify a consistent bias with three key ingredients\. First, singular values evolve unequally leading to rank minimization\. As a result, top singular vectors stabilize well before the end of training, and lastly this happens without displaying alignment between neighboring layers used in several theoretical results\. We show how this bias tracks the transition to generalization in grokking\. We demonstrate more generally that weight decay enhances rank minimization beyond its role as a norm regularizer in practical systems\. Moreover, we show that these spectral dynamics distinguish random label training from true labels, offering a novel perspective on this longstanding conundrum\. Additionally, these dynamics reveal structure in well\-performing sparse subnetworks \(lottery tickets\) and the shape of the loss surface through linear mode connectivity\. Our findings suggest that spectral dynamics provide a coherent view that links the behavior of neural networks across diverse settings\.

### 4\.1Introduction

\(a\)SV Schematic![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.1-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)\(b\)SVs
\(c\)Alignment Schematic
Figure 4\.1:[4\.1\(a\)](https://arxiv.org/html/2608.28948#Ch4.F1.sf1)Schematic for the spectral dynamics of a weight matrix\. As training proceeds, top singular vectors become stable and top singular values grow disproportionately large\.[4\.1\(b\)](https://arxiv.org/html/2608.28948#Ch4.F1.sf2)Singular value evolution for a single intermediate matrix parameter in a Transformer, where each line is a single singular value\. Darker colors correspond to larger singular values while lighter ones correspond to smaller values\. We see a disproportionate trend where large singular values grow larger faster\.[4\.1\(c\)](https://arxiv.org/html/2608.28948#Ch4.F1.sf3)Previous works use SV basis alignment between layers to prove similar theoretical results, however actual alignment between consecutive layers is weak, which we describe in Section[4\.5](https://arxiv.org/html/2608.28948#Ch4.S5)\. We explore these spectral dynamics of weights and connect them to generalization, regularization, and seemingly unrelated phenomena like linear mode connectivity\.We find a task\- and architecture\-agnostic view to link many disparate phenomena in deep learning, simultaneously studying image classification with ConvNets, image generation with UNets, speech recognition with LSTMs and language modeling with Transformers\. Through extensive experiments, we examine the dynamics \(i\.e\., evolution over training\) of singular values and singular vectors of weight matrices—the spectral dynamics of weights\. We observe a few key properties: singular values evolve unequally, with larger ones evolving faster\. As a result, top singular vectors stabilize much faster\. Finally, for top ranks we see alignment between neighboring layers’ singular vectors, though this varies somewhat with the setting\. We preview these phenomena in Figure[4\.1](https://arxiv.org/html/2608.28948#Ch4.F1)\. We are motivated to study these dynamics specifically as optimization is the fundamental process driving deep learning\([Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136);[Zhang et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib228)\), neural networks at their core are chained matrices and nonlinearities, and the SVD is fundamental to every matrix\. The following paragraphs detail how these properties connect to generalization and the mode connectivity phenomenon\.

As a test bed for understanding generalization,[Power et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib154)introduce the “grokking” phenomenon, where a small\-scale model applied to arithmetic tasks initially achieves essentially zero training loss but performs poorly on validation data, then with much more training suddenly minimizes the validation loss \(see Figure[4\.2](https://arxiv.org/html/2608.28948#Ch4.F2)\)\. In particular,[Nanda et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib138)showed that in modular arithmetic, the feature learning that occurs during grokking could be reverse\-engineered entirely from the final weight matrices\. Although this description is precise, it is limited to arithmetic tasks\. In Section[4\.4](https://arxiv.org/html/2608.28948#Ch4.S4), we notice a task\-agnostic view of grokking, observing that the drop in validation loss during grokking coincides with the simultaneous discovery of low\-rank solutions across all weight matrices in the network, whether it be modular arithmetic or image classification\. We also find that this transition relies on weight decay, echoing existing works\([Lyu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib123);[Liu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib119)\)as neither grokking nor low\-rank weights occur without sufficient weight decay\.

Though the common tie between low\-rank weights and generalization in grokking suggests an intriguing explanation for generalization, grokking is typically studied on synthetic tasks with very small\-scale models like single\-layer Transformers or small MLPs and requires very particular hyperparameter settings\([Power et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib154);[Nanda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib138);[Gromov, 2023](https://arxiv.org/html/2608.28948#bib.bib61);[Kumar et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib106)\)\. If our perspective is to be useful, we should obtain similar results in larger systems\. Thus, we turn to common empirical tasks drawn from the literature like image classification, image generation, speech recognition and language modeling as well as varied and larger networks like VGG\([Simonyan and Zisserman, 2014](https://arxiv.org/html/2608.28948#bib.bib185)\), UNet\([Ronneberger et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib166)\), LSTM\([Hochreiter and Schmidhuber, 1997](https://arxiv.org/html/2608.28948#bib.bib72)\)and multi\-layer Transformers\([Vaswani et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib204)\)\.

In Section[4\.5](https://arxiv.org/html/2608.28948#Ch4.S5), we demonstrate that spectral dynamics are biased toward effective rank minimization across various practical neural networks in complex settings\. Although this behavior echoes theoretical predictions in the deep linear setting, we find that the behavior of networks disagrees with a common theoretical assumption about low\-rank dynamics: the alignment of singular vectors in consecutive layers\([Saxe et al\., 2014](https://arxiv.org/html/2608.28948#bib.bib174);[Arora et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib7);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Milanesi et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib132)\)\. Thus, the rank minimization mechanism differs from what the theory describes\. Our hyperparameters are drawn from existing literature, thus the trend toward rank minimization coincides with well\-generalizing networks across settings, echoing the observations in grokking\.

One particularly notable ingredient for grokking was an extreme level of weight decay\. In Section[4\.6](https://arxiv.org/html/2608.28948#Ch4.S6), we empirically connect rank minimization to weight decay, showing that weight decay promotes rank minimization across architectures and tasks, echoing findings in Section[4\.4](https://arxiv.org/html/2608.28948#Ch4.S4)\. In addition, in some cases, weight decay also appears to promote singular vector alignment in consecutive weight matrices despite the nonlinearities between layers\. Although weight decay explicitly penalizes norm, studying spectral dynamics allows us to observe these effects on rank and alignment\. Such effects may lead to better generalization bounds by understanding the reason weight decay is useful, as the most obvious norm\-based explanations are insufficient\([Andriushchenko et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib6)\)\.

To further probe the rank\-generalization connection, we turn to the classic memorization experiments of[Zhang et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib228), who demonstrated that even small networks can memorize random labels, thus any arguments about generalization cannot be capacity\-based alone\. In Section[4\.7](https://arxiv.org/html/2608.28948#Ch4.S7)and Appendix[4\.8](https://arxiv.org/html/2608.28948#Ch4.S8), we show that training with random labels leads to high\-rank solutions, while rank with true labels is much lower\. We also find that while random labels do not align consecutive layers, true labels do, which is surprising given the nonlinearities between layers\. This echoes prior discussion on rank and generalization\. Through spectral dynamics, we see a clear distinction between generalizing and memorizing networks, which provides a foothold toward better theoretical understanding\.

Our results suggest that viewing neural networks through the lens of spectral dynamics can shed light on several generalization\-related phenomena, but there are broader connections\. We return to the topic of the previous chapters\. Linear mode connectivity \(LMC\) and its connection to the lottery ticket hypothesis\([Frankle and Carbin, 2018](https://arxiv.org/html/2608.28948#bib.bib51);[Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136);[Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52);[Neyshabur et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib140)\)\. We find that global magnitude pruning, a standard procedure for finding lottery tickets, preserves top singular vectors and acts like low\-rank pruning\. We also see that the ability to interpolate between models in LMC strongly correlates with sharing top singular vectors\. With these results, we note that the two phenomena can be seen as aspects of the spectral dynamics of weights, bringing them under the umbrella of Chapter[3](https://arxiv.org/html/2608.28948#Ch3)\. For detailed discussions, see Section[4\.7](https://arxiv.org/html/2608.28948#Ch4.S7)and Section[4\.9](https://arxiv.org/html/2608.28948#Ch4.S9)\.

To summarize the discussion above, we study spectral dynamics along a few lines:

- •How does generalization interact with rank?We find grokking, a phenomenon of sudden generalization, is intimately linked to rank minimization \(Section[4\.4](https://arxiv.org/html/2608.28948#Ch4.S4)\)\.
- •What about generalization and rank in standard settings?We show rank minimization is a general phenomenon in more complex tasks and architectures \(Section[4\.5](https://arxiv.org/html/2608.28948#Ch4.S5)\)\.
- •If rank and generalization are tightly connected, what about common regularizations?We find that the extremely common weight decay, a norm regularization, enhances the rank minimization behavior \(Section[4\.6](https://arxiv.org/html/2608.28948#Ch4.S6)\), further tying rank to generalization\.
- •Given the connection between rank and generalization, what about pure memorization?We see that memorizing random labels through training yields high\-rank, unaligned parameters compared to true\-label training \(Section[4\.7](https://arxiv.org/html/2608.28948#Ch4.S7)\)\.
- •If it is so central, how does rank minimization relate to the puzzle of linear mode connectivity?Top singular vectors are preserved when performing magnitude pruning and while linearly interpolating between connected modes \(Section[4\.7](https://arxiv.org/html/2608.28948#Ch4.S7)\), which provides an explanation for the complex convex mode connectivity of Chapter[3](https://arxiv.org/html/2608.28948#Ch3)\.

All of these phenomena and effects have previously been studied in isolation to varying degrees\. By approaching deep learning through spectral dynamics, we aim to inform existing small\-scale theoretical and experimental results, and link them to a much broader literature\. Our hope is that this will provide a deeper footing for further theory and practice\.

### 4\.2Related Work

#### 4\.2\.1Singular Value Dynamics

Prior work on deep linear networks\([Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Milanesi et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib132)\)suggests that rank minimization may better describe implicit regularization in deep matrix factorization than simple matrix norms\. See[Arora et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib7)\(Appendix A\) for a detailed argument\. However, a critical assumption in these works is “balanced initialization\.” This means that for consecutive matricesWiW\_\{i\}andWi\+1W\_\{i\+1\}in the product matrix∏jWj\\prod\_\{j\}W\_\{j\}, we haveWi\+1⊤​Wi\+1=Wi​Wi⊤W^\{\\top\}\_\{i\+1\}W\_\{i\+1\}=W\_\{i\}W\_\{i\}^\{\\top\}at initialization\. Decomposing these matrices with SVDs and leveraging orthogonality leads to matching left and right singular vectors between consecutive matrices\. See Appendix[B\.1](https://arxiv.org/html/2608.28948#A2.S1)for a detailed explanation\. Consequently, the product of the diagonals will evolve in a closed\-form manner, with larger singular values growing faster than smaller ones\. As shown by[Arora et al\. \(2019\)](https://arxiv.org/html/2608.28948#bib.bib8), this translates to rank\-minimizing behavior with increasing depth in the matrix products\. This formula is also empirically validated for linear matrix factorization problems\. Similar results have been derived for tensor products and other structured settings\([Saxe et al\., 2014](https://arxiv.org/html/2608.28948#bib.bib174);[Yaras et al\., 2023a](https://arxiv.org/html/2608.28948#bib.bib219)\)\.[Ji and Telgarsky \(2019\)](https://arxiv.org/html/2608.28948#bib.bib86)show that alignment between layers will happen specifically for deep linear networks with infinite training\. Still, there is no reason to believe standard networks obey this balancedness condition under practical initialization procedures\. In Section[4\.5](https://arxiv.org/html/2608.28948#Ch4.S5), we explore how these conclusions and assumptions hold for much larger, practical neural networks that are far from linear\.

#### 4\.2\.2Low\-Rank Properties

Another line of research focuses on more general low\-rank biases\. Early work explored norms as an implicit bias\([Gunasekar et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib62)\)\. Theoretical analyses reveal that norms or closed\-form functions of weights might be insufficient to explain implicit regularization, but they do not necessarily contradict the possibility of rank minimization\([Razin and Cohen, 2020](https://arxiv.org/html/2608.28948#bib.bib165);[Vardi and Shamir, 2021](https://arxiv.org/html/2608.28948#bib.bib203)\)\. Numerous studies investigate low\-rank biases in various matrices, including the Jacobian\([Pennington et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib152)\), weight matrices\([Le and Jegelka, 2021](https://arxiv.org/html/2608.28948#bib.bib108);[Martin and Mahoney, 2020](https://arxiv.org/html/2608.28948#bib.bib126);[Martin and Mahoney, 2021](https://arxiv.org/html/2608.28948#bib.bib127);[Frei et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib54);[Ongie and Willett, 2022](https://arxiv.org/html/2608.28948#bib.bib142)\), Gram matrix\([Huh et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib76)\), and features\([Yu and Wu, 2023](https://arxiv.org/html/2608.28948#bib.bib222);[Feng et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib46)\)\. Additionally, research suggests that dynamics influence the decay of rank\([Li et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib113);[Chen et al\., 2023b](https://arxiv.org/html/2608.28948#bib.bib29);[Wang and Jacot, 2023](https://arxiv.org/html/2608.28948#bib.bib210)\)\. Orthogonally, weight decay has a long history as a regularizer explicitly penalizing parameter norm, which can be used for norm\-based generalization bounds\([Bartlett, 1996](https://arxiv.org/html/2608.28948#bib.bib11)\), but these bounds do not seem to explain the success of practical systems\([Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136);[Jiang et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib87)\)\. Some works establish connections between weight decay and rank minimization in idealized settings\([Ziyin et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib233);[Galanti et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib56);[Zangrando et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib225);[Ergen and Pilanci, 2023](https://arxiv.org/html/2608.28948#bib.bib44);[Parhi and Nowak, 2023](https://arxiv.org/html/2608.28948#bib.bib148);[Shenouda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib182)\), which may be connected to generalization\([Razin and Cohen, 2020](https://arxiv.org/html/2608.28948#bib.bib165)\)\. We are particularly interested in how far these connections extend beyond ideal settings to practice\.

Though these prior results are interesting in their own right, our goal is to link them together on a much broader suite of experiments and show further unexplored consequences\.

### 4\.3Common Quantities

Throughout the rest of the chapter we will rely on a number of common quantities for our analysis\. Inspired by work in the deep linear case\([Saxe et al\., 2014](https://arxiv.org/html/2608.28948#bib.bib174);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Milanesi et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib132);[Yaras et al\., 2023b](https://arxiv.org/html/2608.28948#bib.bib220)\), we track the evolution of singular values for individual weight matrices\. To gain a high\-level overview of all matrix parameters, we compute the \(normalized\) effective rank of a matrixWW\([Roy and Vetterli, 2007](https://arxiv.org/html/2608.28948#bib.bib167)\)with rankRRas

EffRank\(W\):=−∑i=1Rσi∑jσjlogσi∑jσj,\\displaystyle\\text\{EffRank\}\(W\):=\-\\sum\_\{i=1\}^\{R\}\\frac\{\\sigma\_\{i\}\}\{\\sum\_\{j\}\\sigma\_\{j\}\}\\log\\frac\{\\sigma\_\{i\}\}\{\\sum\_\{j\}\\sigma\_\{j\}\}\\kern 5\.0pt,\(4\.1\)NormEffRank​\(W\):=EffRank​\(W\)R,\\displaystyle\\text\{NormEffRank\}\(W\):=\\frac\{\\text\{EffRank\}\(W\)\}\{R\}\\kern 5\.0pt,\(4\.2\)whereσi\\sigma\_\{i\}’s are the singular values of matrixWWandEffRank​\(W\)\\text\{EffRank\}\(W\)is the entropy of the normalized singular value distribution\. As the probability mass concentrates, the effective rank decreases\. We plotNormEffRank​\(W\)\\text\{NormEffRank\}\(W\)to compare across layers and time\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking_singleseed/error.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/mlp-error.png)

\(a\)Error
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking_singleseed/model.transformer_encoder.layers.0.self_attn.in_proj_weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/mlp-sv-model.layers.2.weight_sv.png)

\(b\)SV Evolution
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking_singleseed/eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/mlp-sv-eff_rank.png)

\(c\)Effective Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking_singleseed/align_diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/mlp-align-model.layers.1.weight_model.layers.2.weight_align-diag.png)

\(d\)Alignment

Figure 4\.2:Grokking and Spectral Dynamics\.Top row:Grokking for Transformers on modular addition\([Nanda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib138)\)for a single seed due to different convergence times\. See Figure[B\.7](https://arxiv.org/html/2608.28948#A2.F7)for averaging over multiple seeds\.Bottom row:Grokking for a 12\-layer MLP on MNIST\([Fan et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib45)\)\.\(a\)Training and validation error\. Shaded regions correspond to standard deviation, where large deviations are due to different seeds converging at different times\.\(b\)A visualization of singular value evolution for the first attention parameter and the second MLP layer, where each line represents a single singular value and the color represents the rank\.\(c\)Effective rank of all layers \(matrix parameters\) over time \(Eqn\.[4\.1](https://arxiv.org/html/2608.28948#Ch4.E1)\)\.\(d\)A visualization of the alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) between the embedding and the first attention parameter, and the first and second MLP layers, where theyy\-axis corresponds to indexiiof the diagonal\. We see that grokking co\-occurs with a transition to low\-rank weights\. In addition, there is an alignment that begins early in training that evolves up the diagonal\. In the image classification case, we see a similar rank transition, though alignment does not evolve up the diagonal \(see Figure[B\.8](https://arxiv.org/html/2608.28948#A2.F8)\)\.In addition, inspired by the assumptions of balancedness made by prior work\([Arora et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib7);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8)\), we examine the alignment of consecutive weight matrices in the Transformer\. To examine and quantify this alignment between SVDs of consecutive matrices in a network at training timett, i\.e\.,

Wi=∑j=1Rσj​\(t\)​uj​\(t\)​vj​\(t\)⊤,Wi\+1=∑k=1Rσk′​\(t\)​uk′​\(t\)​vk′​\(t\)⊤,\\displaystyle W\_\{i\}=\\sum\_\{j=1\}^\{R\}\\sigma\_\{j\}\(t\)u\_\{j\}\(t\)v\_\{j\}\(t\)^\{\\top\},\\qquad W\_\{i\+1\}=\\sum\_\{k=1\}^\{R\}\\sigma^\{\\prime\}\_\{k\}\(t\)u^\{\\prime\}\_\{k\}\(t\)\{v^\{\\prime\}\_\{k\}\}\(t\)^\{\\top\}\\kern 5\.0pt,we compute the inner products between neighboring singular vectors,

A​\(t\)j​k=\|⟨uj​\(t\),vk′​\(t\)⟩\|,A\(t\)\_\{jk\}=\\lvert\\langle u\_\{j\}\(t\),v^\{\\prime\}\_\{k\}\(t\)\\rangle\\rvert\\kern 5\.0pt,\(4\.3\)where the absolute value is taken to ignore sign flips in the SVD computation\. We then plot the diagonal of this matrixA​\(t\)i​i​∀i≤100A\(t\)\_\{ii\}~\\forall~i\\leq 100over time, which we focus on because we observed the most signal here\. See Appendix[B\.4\.4](https://arxiv.org/html/2608.28948#A2.S4.SS4)for more details\. These plots give us a sense as to whether simultaneous diagonalization occurs at least in the top ranks\. For exact details on how alignment is computed for different architectures and layers more complex than the fully connected case, see Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2)\.

We not only employ the alignment matrix defined in Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3), but also derive and plot a scalar measure\. We focus on the top diagonal entries as we observed they typically contained the most structure \(see Figure[B\.8](https://arxiv.org/html/2608.28948#A2.F8)for an example\):

a⁡\(t\)=110​∑i=110A​\(t\)i​i\.a\(t\)=\\frac\{1\}\{10\}\\sum\_\{i=1\}^\{10\}A\(t\)\_\{ii\}\.\(4\.4\)For specific details on calculating this measure in diverse architectures and complex layers \(beyond fully connected layers\), please see Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2)\.

### 4\.4Grokking and Rank Minimization

[Power et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib154)first noticed a surprising phenomenon they called “grokking” where models quickly fit the training data on toy tasks, then after a long period of training, very quickly generalize on the validation data \(see Figure[4\.2](https://arxiv.org/html/2608.28948#Ch4.F2)\)\. Later, others found that this phenomenon can occur on a logarithmic timescale\([Thilak et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib196)\)in simpler models and different datasets\([Liu et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib118);[Gromov, 2023](https://arxiv.org/html/2608.28948#bib.bib61);[Kumar et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib106);[Xu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib217)\)\. In addition, weight decay seems to be a critical ingredient\([Lyu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib123);[Liu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib119);[Tan and Huang, 2023](https://arxiv.org/html/2608.28948#bib.bib194)\)\.

Motivated by experimental results that show the importance of weight decay for grokking\([Power et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib154);[Lyu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib123);[Liu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib119)\), and theoretical work that connects low\-rank weights, generalization and weight decay\([Razin and Cohen, 2020](https://arxiv.org/html/2608.28948#bib.bib165);[Galanti et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib56);[Timor et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib197);[Yaras et al\., 2023b](https://arxiv.org/html/2608.28948#bib.bib220);[Zangrando et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib225)\), we evaluate the potential connection between parameter rank and grokking in neural networks\. Low\-rank weights would naturally complement other descriptions such as Fourier decomposition\([Nanda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib138)\), the simplification of linear decision boundaries\([Humayun et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib77)\), the connection to double descent\([Davies et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib38)\), and the discovery of a sparse solution\([Merrill et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib131)\)\.

We replicate grokking in two settings: a single\-layer Transformer for modular addition\([Nanda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib138)\), and a 12\-layer MLP for MNIST image classification\([Fan et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib45)\)\(see Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2)for details\)\. We select these settings as they are somewhat more realistic than other tasks, require more complex architectures, and display grokking on a linear time scale\.

In Figure[4\.2](https://arxiv.org/html/2608.28948#Ch4.F2), we see a tight connection: the sudden drop in validation loss coincides precisely with the onset of low\-rank behavior in the singular values\. Examining inter\-layer alignment during training, we observe that the final low\-rank solution gradually emerges from the model’s middle ranks\. As we show later, in the absence of weight decay, no low\-rank solution seems to develop \(Figure[B\.4](https://arxiv.org/html/2608.28948#A2.F4)\)\. Additionally, when using 90% of the data and no weight decay, generalization still coincides with effective rank minimization\.[Fan et al\. \(2024\)](https://arxiv.org/html/2608.28948#bib.bib45)noted that in deep MLPs, grokking coincided with a feature rank decrease, which agrees with the parameter rank decrease that we see here\. The careful reader will also note that[Nanda et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib138)previously showed that the particular solution found in modular addition is a low\-rank Fourier decomposition, so our observations on low\-rank weights will follow, yet the same structure also applies to the MLP where such reverse\-engineering is difficult\. In the following sections, we argue that rank minimization is a perspective that can be applied in more complex settings when one does not know what to look for in the weights\. It may thus be possible to interpret the neural network via the top ranks\([Praggastis et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib155)\)\. Specifically, we will show in Section[4\.6](https://arxiv.org/html/2608.28948#Ch4.S6)that large amounts of weight decay have a strong rank\-regularizing effect, so one way to understand grokking is that the network first memorizes the training data, and with further training the rank regularization of weight decay pushes toward generalization\.

### 4\.5Spectral Dynamics Across Tasks

Inspired by the results on grokking and prior work on deep linear networks that studies the evolution of the SVD of the weight matrices\([Saxe et al\., 2014](https://arxiv.org/html/2608.28948#bib.bib174);[Arora et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib7);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Milanesi et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib132);[Yaras et al\., 2023a](https://arxiv.org/html/2608.28948#bib.bib219)\), we apply the same analysis to larger, more practical systems\. We show that the trends we saw in the analysis of grokking mostly hold true across networks and tasks at a much larger scale, though our findings do deviate from theoretical settings\.

#### 4\.5\.1Methodology

Our experiments aim to examine reasonably sized neural networks across a variety of tasks\. We select models and tasks that are representative of current applications\. Specifically, we consider:

- •Image classificationwith CNNs \(VGG\-16\([Simonyan and Zisserman, 2014](https://arxiv.org/html/2608.28948#bib.bib185)\)\) on CIFAR10\([Krizhevsky, 2009](https://arxiv.org/html/2608.28948#bib.bib102)\);
- •Image generationthrough diffusion with UNets\([Ronneberger et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib166)\)on MNIST\([LeCun, 1998](https://arxiv.org/html/2608.28948#bib.bib109)\);
- •Speech recognitionwith LSTMs\([Hochreiter and Schmidhuber, 1997](https://arxiv.org/html/2608.28948#bib.bib72)\)on LibriSpeech\([Panayotov et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib146)\); and
- •Language modelingwith Transformers\([Vaswani et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib204)\)on Wikitext\-103\([Merity et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib130)\)\.

Training hundreds of runs for each of the settings above is computationally expensive, limiting the scale of models we can explore\. We primarily adopt hyperparameters from existing literature, with minor modifications for simplicity\. This ensures that any correlations observed are likely a reflection of common practices, and not bias introduced on our part\. We also provide evidence with larger language models \(up to 3B parameters\) in Appendix[B\.2\.5](https://arxiv.org/html/2608.28948#A2.S2.SS5), from the Pythia suite\([Biderman et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib17)\)\.

The primary evidence in this section comes from computing the SVDs of weight matrices within the models; we disregard 1D bias and normalization parameters in our analysis\. Previous research suggests that these parameters are not always crucial for performance\([Zhang et al\., 2018b](https://arxiv.org/html/2608.28948#bib.bib226);[Mohan et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib133);[Karras et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib94)\), and many large models do not use them\([Raffel et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib160);[Grattafiori et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib60)\)\. As the matrices are the vast majority of the parameters, we believe they are the primary object of interest and it is worthwhile focusing on them\. Due to the large number of matrices in these models, we present plots of individual layers’ matrix parameters and statistics that summarize behavior across layers for conciseness\. We generated hundreds of thousands of plots during this study, making it impossible to include them all\. Full experimental details, including the choice of hyperparameters, are available in Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2)\.

#### 4\.5\.2Effective Rank Minimization

Building on theoretical\([Saxe et al\., 2014](https://arxiv.org/html/2608.28948#bib.bib174);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Milanesi et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib132);[Boix\-Adserà et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib21);[Yaras et al\., 2023a](https://arxiv.org/html/2608.28948#bib.bib219)\)and empirical\([Dittmer et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib40);[Martin and Mahoney, 2020](https://arxiv.org/html/2608.28948#bib.bib126);[Martin and Mahoney, 2021](https://arxiv.org/html/2608.28948#bib.bib127);[Boix\-Adserà et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib21)\)findings, we investigate effective rank minimization across larger models and a more diverse array of tasks\. Figure[4\.3](https://arxiv.org/html/2608.28948#Ch4.F3)reveals a consistent trend: the effective rank of network parameters generally decreases throughout training, regardless of the specific parameter or network architecture\. This suggests a progressive “simplification” of the network as training progresses\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-sv-eff_rank.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-sv-eff_rank.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-sv-eff_rank.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-sv-eff_rank.png)

\(d\)Transformer

Figure 4\.3:Top row:Singular value evolution for a single matrix in the middle of each model\. Each line represents a singular value, darker colors are larger values, while lighter colors are smaller values\. Notice the unequal evolution where top singular values grow at a disproportionate rate\.Bottom row:Normalized effective rank \(Eqn\.[4\.1](https://arxiv.org/html/2608.28948#Ch4.E1)\) evolution visualized in color for all matrix parameters across architectures and time\. As we move down theyy\-axis, the depth of the parameters in the model increases, while thexx\-axis tracks training time\. The axis label "Layer" is shorthand for "matrix parameter"\. For example, in the Transformer we visualize each of theWq,Wk,Wv,M​L​P1,M​L​P2W\_\{q\},W\_\{k\},W\_\{v\},MLP\_\{1\},MLP\_\{2\}parameters for each block\. Notice decreasing effective rank with training time across nearly all parameters, though the magnitude differs across layers \(indicated by absolute color\)\. The block\-like patterns for VGG are likely due to different channel dimension sizes\. The banding in the UNet, LSTM, and Transformer is due to the differences between convolutional and linear layers, residual block connections, and attention and fully connected layers, respectively\. The sharp transitions through training in VGG are due to 10×\\timeslearning rate decays\.We further conduct a singular\-value pruning experiment to explore the relationship between low\-rank behavior and model performance\. We prune either the top or bottom half of the singular values for each weight matrix in the network and then evaluate the pruned model at each training step\. Given their importance inℒ2\\mathcal\{L\}^\{2\}space, we expect the top singular values to capture the information most critical to the network’s function\. Figure[4\.4](https://arxiv.org/html/2608.28948#Ch4.F4)confirms this, demonstrating that the pruned parameters, without further training, can closely approximate the full model’s performance\. It is not necessarily obvious that pruning would have this effect\. In particular, simultaneously pruning lower components across all layers may lead to losing some critical signal that must be passed between layers, or it could be that small\-magnitude singular values may provide some important regularizing noise\. This result is reminiscent of prior work that uses low\-rank approximations for efficiency or performance\([Yu et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib221);[Sharma et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib179);[Chen et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib30)\), or work that explicitly optimizes for low\-rank networks\([Wang et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib209);[Schotthöfer et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib175)\)\. Here we point out that the reason this is possible is due to the dynamics of the singular values themselves\. In later sections, we will rely on this observation that large singular values are more critical to the function of the network\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-val-loss.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-val-loss.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-val-loss.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-val-loss.png)

\(d\)Transformer

Figure 4\.4:Top row:Training loss\.Bottom row:Validation loss\.Redis the full model\.Blueis post\-training pruning the bottom half of the SVD for every matrix in the model that is not the final layer\.Greenis post\-training pruning the top half of the SVD\. Notice that for all models, keeping the top half of the SVD is close to the full model performance, supporting the idea that the top directions provide a good approximation to the function\. We specifically avoid pruning the final layer as for some tasks it is so low\-rank that pruning further affects performance in an anomalous way\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\.
#### 4\.5\.3Alignment of Singular Vectors Between Layers

Similar to the analysis of grokking, we investigate the alignment between consecutive layers in the larger neural networks considered in this section\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-align-model.layers.11.conv.weight_model.layers.12.conv.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-align-align_score.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-align-model.conv3.weight_model.conv4.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-align-align_score.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-align-model.lstm.weight_ih_l2_model.lstm.weight_ih_l3_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-align-align_score.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-align-model.transformer_encoder.layers.7.linear1.weight_model.transformer_encoder.layers.7.linear2.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-align-align_score.png)

\(d\)Transformer

Figure 4\.5:Neighboring layer alignment of singular vectors\.Top row:The diagonal of the alignment matrixA​\(t\)i​iA\(t\)\_\{ii\}\(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) vs\. training time for a single pair of matrices in the middle of each model\. We see a small amount of alignment in the top ranks between layers shortly after training begins, but this becomes more diffuse over time\.Bottom row:Alignment metric \(Eqn\.[4\.4](https://arxiv.org/html/2608.28948#Ch4.E4)\) for pairs of matrices for depth vs\. training time\. It is hard to make out a global trend across models, though the LSTM shows a weak signal around Epoch 1 when the initial alignment occurs, and the Transformer case has a banding pattern with depth due to alignment between the query and key matrices that have no nonlinearity in between\.Figure[4\.5](https://arxiv.org/html/2608.28948#Ch4.F5)reveals a key finding: the theoretical assumption ofbalanced initialization, which posits aligned singular value decompositions \(SVDs\) between weight matrices\([Arora et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib7);[Saxe et al\., 2014](https://arxiv.org/html/2608.28948#bib.bib174)\), does not hold true at the start of training in these larger networks\. Additionally, unlike in the linear case discussed in[Du et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib42), the alignment does not appear to remain static throughout training\. However, a weak signal of alignment in the top ranks develops and disappears\. We are very far from the theoretical settings of prior work\([Du et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib42);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Mulayoff and Michaeli, 2020](https://arxiv.org/html/2608.28948#bib.bib135)\)that use balanced or near\-zero initialization, and the strange trends mean that existing theoretical models do not capture the complexities of neural network training\.

### 4\.6The Effect of Weight Decay

In light of the previously observed evolution of singular values, we investigate a proposed effect of weight decay\. Though weight decay explicitly penalizes the norm of weights, there is evidence that complicates the connection between the norm and generalization for neural networks\([Razin and Cohen, 2020](https://arxiv.org/html/2608.28948#bib.bib165);[Andriushchenko et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib6)\), meaning that we do not have a full understanding as to why weight decay may be useful\. Alternatively, some theoretical\([Boix\-Adserà et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib21);[Razin and Cohen, 2020](https://arxiv.org/html/2608.28948#bib.bib165);[Yaras et al\., 2023a](https://arxiv.org/html/2608.28948#bib.bib219);[Timor et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib197);[Ongie and Willett, 2022](https://arxiv.org/html/2608.28948#bib.bib142);[Galanti et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib56);[Zangrando et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib225)\)and empirical works\([Galanti et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib56);[Boix\-Adserà et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib21)\)propose a connection between weight decay, generalization and the rank of matrices in constrained settings\. Still, comprehensive evidence for larger empirical networks is missing\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.0-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.001-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.001-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.01-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.01-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.1-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.1-sv-eff_rank.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.0-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.1-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.1-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_1.0-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_1.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_10.0-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_10.0-sv-eff_rank.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.0-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.1-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.1-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_1.0-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_1.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_10.0-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_10.0-sv-eff_rank.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.0-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.1-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.1-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_1.0-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_1.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_10.0-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_10.0-sv-eff_rank.png)

\(d\)Transformer

Figure 4\.6:Singular value evolution for a single matrix and normalized effective rank \(Eqn\.[4\.1](https://arxiv.org/html/2608.28948#Ch4.E1)\) across matrices over time, where the rows use differing amounts of weight decay\. From top to bottom, for VGG we use coefficients\{0,0\.001,0\.01,0\.1\}\\\{0,0\.001,0\.01,0\.1\\\}, while for other networks we use coefficients\{0,0\.1,1,10\}\\\{0,0\.1,1,10\\\}\. Higher weight decay coefficients promote more aggressive rank minimization\. VGG uses SGD with momentum, while the rest use AdamW\([Loshchilov and Hutter, 2017](https://arxiv.org/html/2608.28948#bib.bib121)\), which may explain the earlier norm collapse\. We provide an additional version with shared color bars in Figure[B\.10](https://arxiv.org/html/2608.28948#A2.F10)\.We speculate on the intuition behind the mechanism in more practical settings\. In its simplest form, weight decay involves the optimization:arg minW​ℒ​\(W\)\+λ​∥W∥F2\\text\{arg min\}\_\{W\}\\,\\mathcal\{L\}\(W\)\+\\lambda\\lVert W\\rVert\_\{F\}^\{2\}, where∥W∥F2=∑i=1Rσi2\\lVert W\\rVert\_\{F\}^\{2\}=\\sum\_\{i=1\}^\{R\}\\sigma\_\{i\}^\{2\}, andσi\\sigma\_\{i\}are the singular values of weight matrixWWwith rankRR\. We saw previously that larger singular values of neural networks grow faster \(Fig\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.F3), top row\) and that the top singular vectors are much more useful for minimizing task loss than the bottom ones \(Fig\.[4\.4](https://arxiv.org/html/2608.28948#Ch4.F4)\)\. Thus, with minor weight decay regularization, one straightforward solution for the network may be to minimize the rank of a given weight matrix while preserving the top singular values to minimizeℒ⁡\(W\)\\mathcal\{L\}\(W\)\.[Timor et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib197)argue for a similar effect in simple systems: if all singular values are less than one, the norm of activations will shrink with depth, so as depth grows very large, any input signal will converge to zero, making it impossible to learn\. Thus, it is better for a few singular values to be sufficiently large while the rest can be very small\.

Figure[4\.6](https://arxiv.org/html/2608.28948#Ch4.F6)shows that adding weight decay produces this exact low\-rank behavior, while too much weight decay leads to complete norm collapse\. The exact choice of “too much” varies across architectures and tasks\. Despite the low\-rank regularization, we do not see particularly tight alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) in the top singular vectors, except in the highest weight decay Transformer \(see Figure[B\.2](https://arxiv.org/html/2608.28948#A2.F2)in the Appendix\)\. The alignment in this case is reminiscent of the balancedness condition\([Arora et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib7);[Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Du et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib42)\), though the Transformer considered here has nonlinearities and a much more complex structure\.

Orthogonally, we provide additional evidence in Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2), where Figure[B\.3](https://arxiv.org/html/2608.28948#A2.F3)shows that the solutions with moderate weight decay can perform better than without, while with very high weight decay models still generalize, even though they are much lower rank\. To a very rough degree, lower rank leads to better generalization and a smaller training\-validation gap, though when rank is pushed too low the performance collapses as the model can no longer use any capacity\. In general, different tasks may have different minimal ranks necessary\. Language modeling may require some form of memorization for long\-tail words and thus a low\-rank solution may be impossible, while 10\-class image classification may not have this property\. Still, we note that the role of weight decay in improving generalization is tied up with its function as a rank regularizer\. In addition, although we lack precise tools to interpret complex models entirely, when there are only a few ranks per matrix, it may become possible to extend analysis efforts\([Nanda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib138);[Praggastis et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib155)\)to more complex domains\.

### 4\.7Additional Connections

Here we briefly preview some connections between spectral dynamics and additional phenomena\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_False-train-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_True-train-err.png)

\(a\)Train Err\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_False-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_True-val-err.png)

\(b\)Val\. Err\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_False-sv-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_True-sv-model.layers.1.weight_sv.png)

\(c\)SVs
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_False-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_True-sv-eff_rank.png)

\(d\)Eff\. Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_False-align-model.layers.1.weight_model.layers.2.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/mlp-random_labels_True-align-model.layers.1.weight_model.layers.2.weight_align-diag.png)

\(e\)Alignment

Figure 4\.7:Top row:results with true labels\.Bottom row:results with random labels\. We see that the middle layers have a lower effective rank \(Eqn\.[4\.1](https://arxiv.org/html/2608.28948#Ch4.E1)\) when using true labels and that alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) in the middle layers persists throughout training, unlike in the random label case\. We emphasize this alignment occurs despite the nonlinearities\.Memorization vs\. Generalization:In Figure[4\.7](https://arxiv.org/html/2608.28948#Ch4.F7), we replicate the core memorization experiment of[Zhang et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib228), which highlighted the ability of modern neural networks to memorize even random labels perfectly\. We find that when training with random labels, we obtain networks with higher\-rank final parameters as opposed to when training with true labels\. Thus, the spectral dynamics can distinguish between memorization \(of random labels\) and generalization in this simple experiment\. We also see an alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) structure between the middle layers that disappears with random labels, perhaps as it is necessary in order for the network to pass signals from input to output\. This echoes the pattern of grokking in Section[4\.4](https://arxiv.org/html/2608.28948#Ch4.S4), where generalization came with a transition to low\-rank and alignment in middle layers\. We expand this experiment to more settings in Appendix[4\.8](https://arxiv.org/html/2608.28948#Ch4.S8)with additional figures\.

Lottery Tickets:On very small networks,[Frankle and Carbin \(2018\)](https://arxiv.org/html/2608.28948#bib.bib51)found the existence of sparse sub\-networks via magnitude pruning, keeping only the toppp% of weights globally by magnitude, that could train to similar performance as the full network\. This was particularly surprising as they could find such networks with1−101\-10% of the weights, indicating much of the capacity was unnecessary\. For larger image classification networks,[Frankle et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib52)observed that in order to find such sparse subnetworks, it was necessary to train until the end in order to acquire the pruning mask that could be used retroactively in training\. This curious observation still lacks a compelling explanation\. We show that such global magnitude pruning functions similarly to low\-rank pruning, thus the lottery ticket masks found by rewinding\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)are effectively low\-rank masks for the singular components that will become important at the end of training\. Training the masked network leads to similar dynamics in these components\. However, taking masks from too early in training leads to poor approximation of these final components and simultaneously stunts training\. We provide detail on this discussion in Appendix[4\.9](https://arxiv.org/html/2608.28948#Ch4.S9)\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc-sva-model.layers.11.conv.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc-sva-sva_score.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc-sva-model.conv3.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc-sva-sva_score.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc-sva-model.lstm.weight_ih_l2_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc-sva-sva_score.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc-sva-model.transformer_encoder.layers.7.linear1.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc-sva-sva_score.png)

\(d\)Transformer

Figure 4\.8:Top row:Barrier size \(Eqn\.[B\.1](https://arxiv.org/html/2608.28948#A2.E1)\) vs\. split step\.Middle row:singular vector agreement for a single matrix parameter between branch endpoints that share a common trunk\.Bottom row:summary statistic for singular vector agreement across layers vs\. split step\. We see that as models exhibit LMC, they also share a small number of top singular vectors\. We only show agreement for 100 dimensions as the rest are near zero\.Linear Mode Connectivity \(LMC\):Linear Mode Connectivity\([Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136);[Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)refers to the property that models that share a portion of the training trajectory can be averaged in weight\-space to yield a stronger model\([Wortsman et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib215);[Ramesh et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib164)\)\. This phenomenon indicates that, after some training, the loss surface is quite convex in a subspace, even though the optimization problem is theoretically highly non\-convex\. As prior work has found, fine\-tuning from pre\-trained models stays in this convex space\([Neyshabur et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib140);[Li et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib114);[Sadrtdinov et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib170)\), an explanation for what underlies LMC could help to clarify the role of pre\-training, and may lead to faster fine\-tuning\. We show that LMC is tied with singular vector sharing\. In particular, when models display LMC, they share top singular vectors between weights \(see Figure[4\.8](https://arxiv.org/html/2608.28948#Ch4.F8)\), and when they do not they also do not share parameters \(Appendix[4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3)\)\. We argue this is an outcome of the early stability of top singular vectors, which arises due to the unequal evolution of singular values\. It is also straightforward to explain the large Euclidean distance between checkpoints\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52);[Yunis et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib223)\)that can be averaged, as they only share a very small portion of the parameter space\. Thus, LMC, and by extension, model\-averaging, is deeply intertwined with the dynamics of singular values that we explore in Section[4\.5](https://arxiv.org/html/2608.28948#Ch4.S5)\. The full discussion is deferred to Appendix[4\.9](https://arxiv.org/html/2608.28948#Ch4.S9)\.

### 4\.8Spectral Dynamics with Random Labels

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_False-train-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_True-train-err.png)

\(a\)Train Err\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_False-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_True-val-err.png)

\(b\)Val\. Err\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_False-sv-model.layers.6.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_True-sv-model.layers.6.conv.weight_sv.png)

\(c\)SVs
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_False-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_True-sv-eff_rank.png)

\(d\)Eff\. Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_False-align-model.layers.6.conv.weight_model.layers.7.conv.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/vgg-random_labels_True-align-model.layers.6.conv.weight_model.layers.7.conv.weight_align-diag.png)

\(e\)Alignment

Figure 4\.9:Dynamics with random labels for VGG\.Top row:results with true labels\.Bottom row:results with random labels\. We see that the middle layers have a lower effective rank \(Eqn\.[4\.1](https://arxiv.org/html/2608.28948#Ch4.E1)\) when using true labels and that alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) in the middle layers persists throughout training\. The results are less stark in the VGG case, but similar to the MLP\.![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_False-train-cer.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_True-train-cer.png)

\(a\)Train Err\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_False-val-cer.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_True-val-cer.png)

\(b\)Val\. Err\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_False-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_True-sv-model.lstm.weight_ih_l2_sv.png)

\(c\)SVs
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_False-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_True-sv-eff_rank.png)

\(d\)Eff\. Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_False-align-model.lstm.weight_ih_l2_model.lstm.weight_ih_l3_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/random_labels/lstm-random_labels_True-align-model.lstm.weight_ih_l2_model.lstm.weight_ih_l3_align-diag.png)

\(e\)Alignment

Figure 4\.10:Dynamics with random labels for LSTM\.Top row:results with true labels\.Bottom row:results with random labels\. We see that the middle layers have a lower effective rank \(Eqn\.[4\.1](https://arxiv.org/html/2608.28948#Ch4.E1)\) when using true labels and that alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) in the middle layers persists throughout training\. Though the LSTM does not fit the random labels perfectly, the results are qualitatively similar to the other cases\.Given the observations connecting generalization and rank thus far, and the enlightening view on the implicit effects of weight decay, we are interested in seeing whether the perspective developed sheds any light on the classic random label memorization experiments of[Zhang et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib228)\.

Similar to[Zhang et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib228), we train an MLP, a VGG, and an LSTM to fit random or true labels\. Please see Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2)for the details regarding the experimental setup\.[Zhang et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib228)decay the learning rate to zero, and the random label experiments only converge late in training\. Consequently, we use a constant learning rate to avoid confounding factors\. We see in Figure[4\.7](https://arxiv.org/html/2608.28948#Ch4.F7)that both cases for the MLP are able to achieve zero error, though with different singular value evolution and alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) in the middle layer\.

Surprisingly in Figure[4\.7](https://arxiv.org/html/2608.28948#Ch4.F7), we see that with true labels the inner layers are low rank, while with random labels they are much higher rank\. This may be explained by the shared structure in the true classes of the dataset, which manifests in the parameters\. Even more surprisingly, we find here that even without weight decay, inner layers align with true labels, while with random labels, this alignment occurs and then disappears with more training\. This is particularly intriguing as there are non\-linearities that could theoretically separate the network from the linear case, and yet strong alignment occurs despite that\. Such alignment has not yet been leveraged by existing theory, and might provide structured assumptions for new understanding\. Results on the VGG \(Figure[4\.9](https://arxiv.org/html/2608.28948#Ch4.F9)\) and LSTM \(Figure[4\.10](https://arxiv.org/html/2608.28948#Ch4.F10)\) are qualitatively quite similar, though weaker in alignment\. In summary, these results suggest that viewing generalization through the lens of rank and alignment may be fruitful, deepening the connection previously revealed\.

### 4\.9Beyond Generalization

We have seen over the course of many experiments that deep models are biased toward low rank, and that there is a tempting connection between rank minimization and generalization\. Still, spectral dynamics may have broader connections\. In the following subsections, we explore lottery tickets\([Frankle and Carbin, 2018](https://arxiv.org/html/2608.28948#bib.bib51)\)and linear mode connectivity\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)in greater detail\. Beyond shedding further light on neural networks, these phenomena have implications for more efficient inference and storage, as well as understanding the importance of pretraining\([Neyshabur et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib140)\)\. We find that lottery tickets are a sparse approximation of final\-checkpoint top singular vectors\. The ability to linearly interpolate between faraway checkpoints and improve performance coincides strongly with top singular vector sharing between checkpoints\. Such observations may form a foundation for a better understanding compression and model averaging\([Wortsman et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib215);[Ilharco et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib79)\)\.

#### 4\.9\.1Top Singular Vectors Become Stable Earlier

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-sva-model.layers.11.conv.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-sva-sva_score.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-sva-model.conv3.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/lr_0.0003-sva-sva_score.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-sva-model.lstm.weight_ih_l2_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/lr_0.0003-sva-sva_score.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-sva-model.transformer_encoder.layers.7.linear1.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/lr_0.0006-sva-sva_score.png)

\(d\)Transformer

Figure 4\.11:Top row:Singular vector agreement for a single matrix in the middle of each model \(diagonal of Eqn\.[4\.5](https://arxiv.org/html/2608.28948#Ch4.E5)\)\. Notice top singular vectors become stable in direction earlier\.Bottom row:Summary score for each matrix across architectures\. As we move down theyy\-axis, the depth of the parameters in the model increases, while thexx\-axis tracks training time\. The sharp transition midway through training in the VGG case is likely due to a 10x learning rate decay\.Before we explore the phenomena, we first make another observation that will be helpful\. As top singular values grow disproportionately large, it would be natural that top singular vectors become stable in direction as the gradients remain small\. To demonstrate this, for a given matrix in the networkWi​\(t\)=∑j=1Rσj​\(t\)​uj​\(t\)​vj​\(t\)⊤W\_\{i\}\(t\)=\\sum\_\{j=1\}^\{R\}\\sigma\_\{j\}\(t\)u\_\{j\}\(t\)v\_\{j\}\(t\)^\{\\top\}at training time t, we compute

S​\(t\)j​k=\|⟨uj​\(t\)​vj​\(t\)⊤,uk​\(T\)​vk​\(T\)⊤⟩\|,S\(t\)\_\{jk\}=\\lvert\\langle u\_\{j\}\(t\)v\_\{j\}\(t\)^\{\\top\},u\_\{k\}\(T\)v\_\{k\}\(T\)^\{\\top\}\\rangle\\rvert,\(4\.5\)whereTTis the final step of training, and the absolute value is taken to ignore sign flips in the SVD computation\. We then plot the diagonal of this matrixS​\(t\)i​i​∀i≤100S\(t\)\_\{ii\}~\\forall~i\\leq 100over time\. We also use a scalar measure of the diagonal to summarize like in the alignment case:s⁡\(t\)=110​∑iS​\(t\)i​is\(t\)=\\frac\{1\}\{10\}\\sum\_\{i\}S\(t\)\_\{ii\}\. In Figure[4\.11](https://arxiv.org/html/2608.28948#Ch4.F11), we see that top singular vectors converge in direction earlier than bottom vectors\.

#### 4\.9\.2Lottery Tickets Preserve Final Top Singular Vectors

As large singular vectors will become stable late in training, we wonder about the connection to magnitude pruning and the lottery ticket hypothesis\.[Frankle and Carbin \(2018\)](https://arxiv.org/html/2608.28948#bib.bib51)first showed evidence for the lottery ticket hypothesis, the idea that there exist sparse subnetworks of neural networks that can be trained to a comparable performance as the full network, where the sparse mask is computed from the largest magnitude weights of the network at the end of training\.[Frankle et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib52)build further on this hypothesis and notice that, for larger networks, the masking cannot begin at initialization, but rather at some point early in training\. Still, the mask must come from the end of training\.

The reason for this particular choice of mask may be connected to the dynamics we previously observed\. Specifically, at the end of training large singular values are disproportionately larger, so high\-magnitude weights may correspond closely to weights in the top singular vectors\. If magnitude masks were computed at the beginning, the directions that would become the top singular vectors might be prematurely masked as they have not yet stabilized, which may prevent learning on the task\.

Here we train an unmasked VGG\-16\([Simonyan and Zisserman, 2014](https://arxiv.org/html/2608.28948#bib.bib185)\)on CIFAR10, then compute either a random mask, or a global magnitude mask from the end of training, and rewind to an early point\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)to start sparse retraining\. We also do the same with an LSTM\([Hochreiter and Schmidhuber, 1997](https://arxiv.org/html/2608.28948#bib.bib72)\)on LibriSpeech\([Panayotov et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib146)\)\. Please see Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2)for details\. In Figures[4\.12](https://arxiv.org/html/2608.28948#Ch4.F12)and[4\.13](https://arxiv.org/html/2608.28948#Ch4.F13), we plot the singular vector agreement \(SVA, Eqn\.[4\.5](https://arxiv.org/html/2608.28948#Ch4.E5)\) between the final model, masked and unmasked, where we see exactly that magnitude masks preserve the top singular vectors of parameters, and with increasing sparsity fewer directions are preserved\. Even though prior work has remarked that it is possible to use low\-rank approximations for neural networks\([Yu et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib221)\), and others have explicitly optimized for low\-rank lottery tickets\([Wang et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib209);[Schotthöfer et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib175)\), we rather are pointing out that the magnitude pruning procedure seems to recover a low\-rank approximation though it was intended to be "unstructured\."

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_magnitude-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_random-train-loss.png)

\(a\)Loss
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_magnitude-sva-model.layers.3.conv.weight_sva_mask_final-step_0.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_random-sva-model.layers.3.conv.weight_sva_mask_final-step_0.png)

\(b\)Pruned SVA
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_magnitude-sva-sva_mask_final_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_random-sva-sva_mask_final_score.png)

\(c\)All Layers
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_magnitude-sva-model.layers.3.conv.weight_sva_unmask-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_random-sva-model.layers.3.conv.weight_sva_unmask-diag.png)

\(d\)SVA evol\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_magnitude-sva-sva_unmask_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/vgg-mask_type_random-sva-sva_unmask_score.png)

\(e\)All Layers

Figure 4\.12:Pruning results for VGG\.Top row:Magnitude pruning\.Bottom row:random pruning\.\(a\):Training loss\. We see that at 5% sparsity magnitude pruning is significantly better than random pruning of the same layerwise sparsity\.\(b\):Singular vector alignment pre\- and post\-pruning at the end of training for a single layer \(the 3rd convolution\)\. We see that magnitude pruning approximates many more top singular vectors than random pruning at the same sparsity\.\(c\):Singular vector alignment score pre\- and post\-pruning across all layers\. Agreement is higher across all layers for magnitude pruning, though curiously deeper layers do not agree, likely as later layers are wider so weights are lower magnitude and more will be pruned by the unstructured process\.\(d\):Singular vector alignment between the pruned and unpruned models along the training trajectory\. We see that the magnitude pruning still has similar dynamics in its top singular vectors, while random pruning does not\.\(e\):Singular vector alignment score between pruned and unpruned models across layers and time\. Again evolution is similar for early layers with magnitude pruning, and completely different for random pruning\.![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_magnitude-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_random-train-loss.png)

\(a\)Loss
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_magnitude-sva-model.lstm.weight_ih_l2_sva_mask_final-step_0.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_random-sva-model.lstm.weight_ih_l2_sva_mask_final-step_0.png)

\(b\)Pruned SVA
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_magnitude-sva-sva_mask_final_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_random-sva-sva_mask_final_score.png)

\(c\)All Layers
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_magnitude-sva-model.lstm.weight_ih_l2_sva_unmask-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_random-sva-model.lstm.weight_ih_l2_sva_unmask-diag.png)

\(d\)SVA evol\.
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_magnitude-sva-sva_unmask_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lottery_tickets/lstm-mask_type_random-sva-sva_unmask_score.png)

\(e\)All Layers

Figure 4\.13:Pruning results for LSTM\.Top row:Magnitude pruning\.Bottom row:random pruning\. See Figure[4\.12](https://arxiv.org/html/2608.28948#Ch4.F12)for details\. Results are quite similar for the LSTM at 25% pruning as the VGG in Figure[4\.12](https://arxiv.org/html/2608.28948#Ch4.F12)\.We also compute the singular vector agreement \(SVA\) between the masked model trajectory and the original unmasked model trajectory \(diagonal of Eqn\.[4\.5](https://arxiv.org/html/2608.28948#Ch4.E5)\)\. We see in Figures[4\.12](https://arxiv.org/html/2608.28948#Ch4.F12)and[4\.13](https://arxiv.org/html/2608.28948#Ch4.F13)that there is no agreement between the bottom singular vectors at all, but there is still loose agreement in the top singular vectors\. Thus, it seems the mask allows the dynamics of only the top singular vectors to remain similar, which we know are most important from the pruning analysis in Figure[4\.4](https://arxiv.org/html/2608.28948#Ch4.F4)\.

Preserving top singular vectors by pruning seems like a natural outcome of large matrices regardless of the mask, so as a control, we follow exactly the same protocol except we generate the mask randomly with the same layerwise sparsity\. We can see in Figures[4\.12](https://arxiv.org/html/2608.28948#Ch4.F12)and[4\.13](https://arxiv.org/html/2608.28948#Ch4.F13)that this results in much lower preservation of top singular vector dynamics, and also performs worse, as in\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\. It would not be surprising that random pruning is worse if simply evaluated at the end of training, but masking is applied quite early in training at epoch 4 of 164 long before convergence, so it’s striking that the network now fails to learn further even though it is far from convergence\. We interpret this as evidence that the mask has somehow cut signal flow between layers, so it is now impossible for the network to learn further, while magnitude pruning and rewinding still allows signals to pass that eventually become important\. Examining the SVD dynamics of neural network weights allows us to see all of this structure that was hidden previously\.

#### 4\.9\.3Spectral Dynamics and Linear Mode Connectivity

We come to the final phenomenon that we seek to describe: linear mode connectivity\. Linear mode connectivity \(LMC\) is the property that one can interpolate linearly between two different minima in weight space and every parameter set along that path performs well, which gives the impression that the loss surface of neural networks is somehow convex despite its theoretical nonconvexity\. This was first demonstrated in small networks with the same initialization\([Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136)\), then expanded to larger networks and connected to lottery tickets\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52);[Paul et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib151)\)\.[Entezari et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib43)first conjecture that two arbitrary minima show LMC up to permutation, and demonstrate it in simple models\. Others expanded this to wide models\([Ainsworth et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib4);[Jordan et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib88);[Qu and Horvath, 2024](https://arxiv.org/html/2608.28948#bib.bib157)\), and permutation connectivity can be proven in various ways\([Kuditipudi et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib105);[Brea et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib23);[Simsek et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib186);[Ferbach et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib47)\)\. Still, these conditions and results do not hold for standard models\([Qu and Horvath, 2024](https://arxiv.org/html/2608.28948#bib.bib157)\)\. LMC has also been exploited for model\-averaging and performance gains\([Wortsman et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib215);[Ilharco et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib79);[Rame et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib163)\)\. Still despite all of this work, we lack a description for why LMC occurs\. In particular: why is there a convex, high dimensional\([Yunis et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib223)\)basin that models find shortly in training\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\), or after pretraining\([Neyshabur et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib140);[Sadrtdinov et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib170)\)? We do not answer this question in full, but find a deep connection through the dynamics of singular vectors\.

##### Linear Mode Connectivity Correlates with Top Singular Vector Agreement

We saw in Figure[4\.11](https://arxiv.org/html/2608.28948#Ch4.F11)that top singular vectors converge in direction earlier\. We also know that for models to display LMC, they need to share an early part of the training trajectory\. Perhaps the top singular vectors become stable after this early stage, so we might expect mode\-connected solutions to share these components\. To examine this, we plot agreement between the singular vectors of the weight matrices at either endpoint of branches:

W\(1\)​\(T\)\\displaystyle W^\{\(1\)\}\(T\)=∑jRσj​\(T\)​uj​\(T\)​vj​\(T\)⊤,\\displaystyle=\\sum\_\{j\}^\{R\}\\sigma\_\{j\}\(T\)u\_\{j\}\(T\)v\_\{j\}\(T\)^\{\\top\}\\kern 5\.0pt,W\(2\)​\(T\)\\displaystyle W^\{\(2\)\}\(T\)=∑kRσk′​\(T\)​uk′​\(T\)​vk′​\(T\)⊤,\\displaystyle=\\sum\_\{k\}^\{R\}\\sigma^\{\\prime\}\_\{k\}\(T\)u^\{\\prime\}\_\{k\}\(T\)\{v^\{\\prime\}\_\{k\}\}\(T\)^\{\\top\}\\kern 5\.0pt,spawned from the same initialization in training\. If the branches are split from an initialization on a trunk trajectoryW⁡\(t\)W\(t\), we callttthe split point or epoch\. We visualize the diagonal of\|⟨uj​\(T\)​vj​\(T\)⊤,uk′​\(T\)​vk′​\(T\)⊤⟩\|j​k\\lvert\\langle u\_\{j\}\(T\)v\_\{j\}\(T\)^\{\\top\},u^\{\\prime\}\_\{k\}\(T\)\{v^\{\\prime\}\_\{k\}\}\(T\)^\{\\top\}\\rangle\\rvert\_\{jk\}vs\. split epoch, where the absolute value is taken to ignore sign flips in SVD computation\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc_cross-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc_cross-sva-model.layers.11.conv.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc_cross-sva-sva_score.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc_cross-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc_cross-sva-model.conv3.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc_cross-sva-sva_score.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc_cross-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc_cross-sva-model.lstm.weight_ih_l2_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc_cross-sva-sva_score.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc_cross-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc_cross-sva-model.transformer_encoder.layers.7.linear1.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc_cross-sva-sva_score.png)

\(d\)Transformer

Figure 4\.14:Top row:Barrier size vs\. split step\.Middle row:singular vector agreement for a single matrix parameter between branch endpoints that do not share a common trunk, but do share split time and branch data order\.Bottom row:summary statistic for singular vector agreement across layers\. We see that when branches do not share a common trunk, there is neither LMC nor singular vector agreement, even though the optimization is otherwise the same\.The technical definition of LMC requires measuring the bump, or barrier, in the loss surface along the linear interpolation between final checkpoints\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\. To measure this precisely, we use the definition from[Neyshabur et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib140), which is the maximum deviation from a linear interpolation in the loss, an empirical measure for convexity in this linear direction\. When this deviation is 0, we consider the checkpoints to exhibit LMC\. Please see Appendix[B\.2\.10](https://arxiv.org/html/2608.28948#A2.S2.SS10)for details on the calculation\. Given evidence in Figure[4\.4](https://arxiv.org/html/2608.28948#Ch4.F4)that top components are the most important for prediction, and that top components become stable before training has finished, it is plausible that LMC is connected to the stability of top singular vectors in the later portion of training\.

This would mean that checkpoints that do not exhibit the LMC property should not share top singular vectors, while checkpoints that do exhibit the LMC property should share top singular vectors\. We see in Figure[4\.8](https://arxiv.org/html/2608.28948#Ch4.F8)that this is the case across models and tasks, where the alignment between endpoints is much stronger in top singular vectors\. We also see no LMC and poor agreement in top components between branches that have initializations from different trunk trajectories, but with the same split epochttand the same branch data order in Figure[4\.14](https://arxiv.org/html/2608.28948#Ch4.F14)\. Thus, these top directions are not a unique property of the architecture and data, but rather are dependent on initialization\. It is notable that concurrent work\([Ito et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib82)\)arrives at a similar conclusion: permutation solvers between optima match top singular vectors\. Though the conclusions are similar, their experiments are primarily conducted on smaller scale settings, and only for permutation matching at the end of training\. Here we connect these observations to the optimization behavior of SVDs throughout training, tying our previous observations on generalization in with weight averaging\.

##### Perturbing Breaks Linear Mode Connectivity and Singular Vector Agreement Simultaneously

To make the connection between top singular vectors and LMC even tighter, we intervene in the training process\. If we add random perturbations to destabilize the components that will become the top components long before they have converged, and if singular vector agreement is tied to LMC, we would like to see that final models no longer exhibit the LMC property\. Indeed this is the case\. In Figure[4\.15](https://arxiv.org/html/2608.28948#Ch4.F15), when increasingly large random perturbations are applied, the barrier between final checkpoints increases and the LMC behavior disappears\. Please see Appendix[B\.2](https://arxiv.org/html/2608.28948#A2.S2)for details\. In addition, the previously\-strong singular vector agreement disappears simultaneously\. Thus it seems this agreement is tied to linear mode connectivity\.

We speculate that, due to the results in Figure[4\.4](https://arxiv.org/html/2608.28948#Ch4.F4)that show the top half of the SVDs are much more critical for performance, if these components are shared then interpolating will not affect performance much\. Rather, interpolation will eliminate the orthogonal bottom components which may only make a minor impact on performance\. If however the top components are not shared, then interpolating between two models will remove these components, leading to poor performance in between\. Such observations may help in explaining the utility of pretraining\([Neyshabur et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib140)\), weight averaging\([Rame et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib163);[Wortsman et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib215);[Ilharco et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib79)\)or the use of LoRA\([Huh et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib76)\)to replace full finetuning\. Studying spectral dynamics allows us to see this deep structure\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc_pert-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc_pert-sva-model.layers.11.conv.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg_lmc/lmc_pert-sva-sva_score.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc_pert-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc_pert-sva-model.conv3.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet_lmc/lmc_pert-sva-sva_score.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc_pert-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc_pert-sva-model.lstm.weight_ih_l2_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm_lmc/lmc_pert-sva-sva_score.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc_pert-train-loss_barrier.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc_pert-sva-model.transformer_encoder.layers.7.linear1.weight_sva-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr_lmc/lmc_pert-sva-sva_score.png)

\(d\)Transformer

Figure 4\.15:Top row:Barrier size vs\. perturbation magnitude\.Middle row:singular vector agreement for a single matrix parameter between branch endpoints vs\. perturbation magnitude\.Bottom row:summary statistic for singular vector agreement across layers with perturbation magnitude\. We see that whereas without perturbation models would exhibit LMC after training, with increasing perturbations the LMC property disappears simultaneously with the agreement in top singular vectors\.

### 4\.10Discussion

We provide an empirical perspective to understand deep learning through SVD dynamics\. We first note a tendency toward rank minimization on a small scale in grokking, then expand these findings to practical networks and tasks\. In addition, we find that weight decay, though it explicitly penalizes norm, implicitly promotes this low\-rank bias\. We also show that training with random or true labels differs in the rank and alignment of solutions found by optimization, echoing the rank\-generalization connection during grokking\. We go beyond remarks on generalization and show that magnitude pruning for lottery tickets acts similarly to low\-rank pruning, and LMC coincides with the sharing of top singular vectors between checkpoints\.

A comprehensive theory for all these results remains elusive\. Our goal in this work is to provide these observations as a platform for a deeper understanding of deep learning\. Notably, the observed spectral dynamics appear consistent across diverse settings, even without restrictive assumptions like balanced initialization, linearity, or small weight scales\. This suggests a common underlying mechanism that further theoretical work may be able to model\. One limitation of our study is the focus on linear alignment between neighboring layers\. In linear systems, this alignment is a proxy for the network’s ability to pass signals from input to output, but once nonlinearities are introduced, it is unclear how this proxy applies\. We would guess there is still alignment in larger systems, but it may need to be viewed post\-activation for particular examples\. This is a promising direction for future exploration\.

On the empirical side, several interesting problems present themselves\. Interpretability of neural networks is a growing area of research\([Nanda et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib138)\), and there already exist efforts to interpret singular vectors of convolutional weights\([Praggastis et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib155)\)\. There may also be connections to other unexplained phenomena such as double descent\([Belkin et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib13);[Nakkiran et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib137);[Davies et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib38)\)or adversarial examples\([Szegedy et al\., 2013](https://arxiv.org/html/2608.28948#bib.bib193);[Ilyas et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib80);[Hendrycks et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib68)\)\. For example, is it actually the case that low\-rank models are more robust? The answers to these questions may help design better optimizers or diagnose deployment risks in the wild\. There are also concerns of safety\([Bai et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib9);[Mazeika et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib128)\)that a better understanding of neural networks can alleviate\([Burns et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib25);[Park et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib149)\)\. In particular with low\-rank models, interpretability may become easier as there is much less model to study\. We present this large empirical exploration in an effort to deepen scientific understanding, provide fertile ground for new theoretical assumptions, and offer inspiration for better engineering\. We believe the developed perspective will be useful in bridging gaps between many different attempts to explain neural networks\.

So we have seen that low rank dynamics are tightly related to the development of the convex mode and ties nicely with prior work on lottery tickets\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\. Given their central role in neural network optimization, we believe that such low\-rank dynamics can be understood as the primary feature learning that happens during optimization\. This fits nicely with previous work leveraging low\-rank pruning for compression and performance\([Yu et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib221);[Sharma et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib180)\), and other studies that show different modes \(defined by different low\-rank cores\) learn different functions\([Lubana et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib122)\)\. We now turn to using these spectral dynamics to overcome simplicity bias\. The key idea is that the low\-rank bias can be used as a definition for simplicity bias, meaning by manipulating the spectral dynamics we can overcome spurious correlations\. The following chapter details this approach\.

## Chapter 5Overcoming Spurious Correlations through Diverse Feature Learning

We pursue a generic method to learn more diverse features from existing data, without task\-specific augmentations or reliance on external information\. Due to simplicity bias, such feature learning is often blocked in standard optimization\. Taking inspiration from spectral dynamics as a definition for simplicity bias, we demonstrate that projected gradient descent off of top singular vectors suffices to learn complex features in an agnostic way in a difficult testbed\. In more complex cases, we show large benefits for diversity losses, without the drawbacks or assumptions of prior work\. We also show that such benefits extend into language modeling, beyond the commonly\-studied classification setting\.

### 5\.1Introduction

Neural networks show simplicity bias\([Rahaman et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib161);[Huh et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib75);[Pezeshki et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib153)\), which on a large scale can be responsible for issues of class bias and adversarial hacking of features\([Zou et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib234);[Rumbelow and Watkins, 2023](https://arxiv.org/html/2608.28948#bib.bib168)\)\.

One way to understand this is that the optimization objective encourages neural networks to find a set of features to solve a given task, but the dynamics of optimization lead only to the minimal set of features\([Pezeshki et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib153)\)\. Intuitively, gradients are very small for examples that are already well learned, so there is no incentive to learn more features for those examples\. A classic demonstration of this is networks learning to classify cows by the presence of grass in the background\([Beery et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib12)\)\. If grass is always present, why pay attention to the cow?

Many methods have been proposed to diversify feature learning\. One general approach upsamples misclassified examples\([Liu et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib117);[Zhang et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib229)\), but such methods ignore further information from examples already correctly classified\. Other methods try to enforce some notion of disagreement using information about the test distribution\([Pagliardini et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib145);[Lee et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib110);[To et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib199)\), but this requires a particular test distribution which may not be known ahead of time\.[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230)show that the simple ensembles of parallel training runs have significant diversity in their representations, when compared to single runs of larger models, but such a method still falls prey to simplicity biases for any individual run\.

As far as we know, there is no generic solution to the problem of simplicity bias\.[Pezeshki et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib153)suggest a regularizer that makes competing correlations less imbalanced, but this requires a dataset design where a single simple feature is not perfectly predictive\.[Shah et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib178)demonstrate that increased model size, adversarial training, and standard regularization techniques are also insufficient\. For any particular data distribution, it might be possible to design specific augmentations that address spurious correlations, for example masking backgrounds in the case of cows on grass, but it may not be clear how to do this in general tasks like language modeling\.

At the end of the day, what we are left with is the ability to collect more and more data and grow the training distribution to encompass all possible combinations of features, hoping that we learn diverse features through brute force scaling\([Kaplan et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib93)\)\. This is an accurate reflection of current practice in training large language models, where diverse and large data appears to be the most critical component\([Brown et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib24);[Achiam et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib2)\)\.

Instead we propose a method to overcome simplicity bias generically in neural networks\. First, we posit that the most important features can be found in the top ranks of neural network weights given extensive historical evidence of concentration during optimization and performance during pruning \(see Chapter[4](https://arxiv.org/html/2608.28948#Ch4)\)\. Next we perform an iterative process, training a neural network, collecting its top features \(the top ranks of its weights\) and then running subsequent rounds of training with projected gradient descent off of those directions so as to force the neural network to learn new features\. Though naive in nature, we find that this iterative process is sufficient to discover more diverse features in a case of extreme simplicity bias\. We then explore scaling of this method to more complex tasks, developing deeper understanding along the way\.

In particular, we ask and answer the following questions:

- •Can we use our understanding of spectral dynamics to address spurious correlations?We find that a projected gradient descent method based on our understanding of spectral dynamics can force a neural network to extract more information from the data without relying on any task\-specific information;
- •Does this method scale to larger networks?We then scale this method to larger image classification networks, and find that it fails, but a related method takes its place, showing substantial improvement over prior methods;
- •Can we apply this to even more complex tasks?We then move to language modeling, where we find the task is too difficult for the assumptions of our prior methods\. Even though the rich feature structure exists in language modeling, we demonstrate that optimization makes it very difficult to take advantage of this structure\. Through an extensive raft of experiments we explore this space, and make prescriptions for the future\.

### 5\.2Simplicity Bias and Spectral Dynamics

![Refer to caption](https://arxiv.org/html/2608.28948v1/fig/diverse_features/mnist_cifar.png)Figure 5\.1:Examples from the MNIST\-CIFAR dataset in[Shah et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib178)\.We begin with a simple model system, drawn from prior work\.[Shah et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib178)showed, through carefully crafted datasets, that neural networks exploit the simplest features in a dataset, even if a more complex feature may hold useful information\.

In particular, they introduced MNIST\-CIFAR, a binary classification dataset with a controlled structure\. Each example combines two images, one from MNIST, and one from CIFAR\. Class 1 corresponds to a concatenation of an MNIST 0 and a CIFAR car, while class 2 corresponds to an MNIST 1 and a CIFAR truck\. Both the MNIST and CIFAR classes are perfectly correlated with the label \(Figure[5\.1](https://arxiv.org/html/2608.28948#Ch5.F1)\)\.

[Shah et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib178)trained a variety of networks on this dataset, and showed that even with scale, regularization, adversarial training, and ensembling, no model paid attention to the CIFAR half of images\. The predictions of the trained models were entirely dependent on the MNIST half\. Large perturbations or substitutions in the CIFAR half did not change the result\.

We use this MNIST\-CIFAR task to illustrate the benefit of spectral dynamics as feature learning\. By taking the top ranks as the most important features, we can find neural networks generically successfully pay attention to CIFAR\.

#### 5\.2\.1Projected Gradient Descent for Diverse Feature Learning

Our method consists of training successive rounds of neural networks with modified optimization in each round\. After a round, we accumulate a set of orthogonal directions and run projected gradient descent off of those directions for all parameters in the network in the next round\. Roughly what this means is we prevent a new network from using the directions, which we believe correspond to features, of the previous rounds\. Thus, the network will learn more complex and diverse features on the same data\.

We train N rounds to convergence\. In the first round we train normally\. At the end of each round, for each parameter matrix we compute the SVDWl=∑i=1Rσi​ui​vi⊤W\_\{l\}=\\sum\_\{i=1\}^\{R\}\\sigma\_\{i\}u\_\{i\}v\_\{i\}^\{\\top\}, then we stack the topkkranks on the input side into a matrix:

V~⊤=\[v1⊤\.\.\.vk⊤\]\.\\tilde\{V\}^\{\\top\}=\\begin\{bmatrix\}v\_\{1\}^\{\\top\}\\\\ \.\.\.\\\\ v\_\{k\}^\{\\top\}\\end\{bmatrix\}\.\(5\.1\)
After constructing this stack we reinitialize eachWlW\_\{l\}and its corresponding optimization parameters, and start a new training run\. With every gradient step we project the input and space of the gradients off of our previously collectedV~\\tilde\{V\}like:

∇Wl′←∇Wl​\(I−V~​V~⊤\)\\nabla W\_\{l\}^\{\\prime\}\\leftarrow\\nabla W\_\{l\}\(I\-\\tilde\{V\}\\tilde\{V\}^\{\\top\}\)\(5\.2\)and the same for the parameters because of possible momentum and bias issues\.

When we reach the end of a training round, we append the topkkdirections of the new parameters to ourV~\\tilde\{V\}buffer, growing an ever larger list of orthogonal directions to avoid during training\. This means that optimization is impossible if the number of rounds timeskkis greater than the rank ofWlW\_\{l\}, but practically speaking that happens after many many rounds of training\.

For the experiments, we typically usek=3k=3and 2000 epochs of training so that models reach convergence before computing SVDs\. This method looks quite similar to prior work in continual learning\([Saha et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib172)\), with a few key differences\. First, we run repeated iterations on the same data, not on unrelated classes\. Second, we project using the SVD of the weight matrix, not the SVD of collected activations, making our method substantially more memory efficient\.

#### 5\.2\.2Experiments

We use the previously described MNIST\-CIFAR\([Shah et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib178)\)dataset \(see Figure[5\.1](https://arxiv.org/html/2608.28948#Ch5.F1)for examples\)\. We train a simple 4\-layer MLP for binary classification\. See Appendix[C\.1](https://arxiv.org/html/2608.28948#A3.S1)for more details\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/fig/diverse_features/swapped_protocol.png)Figure 5\.2:Evaluation protocol for experiments\. After training on MNIST\-CIFAR, we swap the classes of either the MNIST half, or the CIFAR half of the image and evaluate on that setting\. When models rely entirely on MNIST, error on the swapped task degrades to 100%, and similarly for CIFAR\. On the right we plot the errors over successive rounds of PGD\. We see that models start to learn CIFAR after the early rounds of PGD, coming to rely quite heavily on CIFAR in later rounds\. Spikes correspond to restarting a new round of training\.We evaluate training error and validation error as usual\. We also include a few different evaluations to diagnose whether the model learns any CIFAR features, namely: 1\) we evaluate swapping the classes of the MNIST portions of the input while keeping the label fixed, and 2\) we evaluate swapping the CIFAR portions of the input while keeping the label fixed\. If the model relies entirely on MNIST, we should see that swapping the MNIST images should flip the predictions entirely, while swapping the CIFAR portion should result in no change at all\. See Figure[5\.2](https://arxiv.org/html/2608.28948#Ch5.F2)for an illustration\.

\(a\)Normal\(b\)Swapped\-class
Figure 5\.3:Left:Training and validation error for successive rounds with our proposed PGD method on top singular vectors\.Right:Swapped\-class errors throughout rounds\. We see that as the round number increases, validation error decreases due to the complexity of the task increasing\. In addition, the model increasingly relies more on CIFAR, as the swapped CIFAR error goes from near 0 to 60% and above\. This indicates the model is learning to extract more complex signals which are less predictive but still useful\.We see the results for our PGD run in Figure[5\.3](https://arxiv.org/html/2608.28948#Ch5.F3)\. Notably, training error is near 0 for every round of training, but validation error climbs due to the reliance on more complex CIFAR features\. We see that swapping the MNIST digits in early rounds of training yields nearly 100% error, demonstrating a reliance entirely on MNIST\. As PGD proceeds, this decreases to less than random chance, or near the total validation error\. Conversely, swapping CIFAR images initially results in no change in predictions when compared to validation error, but in later rounds, swapping CIFAR yields errors above 70%\. This shows that PGD successfully incentivizes later members of the ensemble to extract diverse features, exactly in line with our hopes\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/fig/diverse_features/sv_evol.png)Figure 5\.4:First layer singular vectors over rounds of PGD training\. Each set of columns corresponds to the MNIST and CIFAR halves of the reshaped singular vector\. Descending down the rows corresponds to increasing rank index when sorted by singular value\. We see initially that the first layer singular vectors form digit classifiers exclusively\. In middle rounds the network focuses on higher and higher frequency signals in MNIST\. In late rounds, all but the high frequency signals are present in MNIST, hence the model starts to leverage the CIFAR half of the image\.In Figure[5\.4](https://arxiv.org/html/2608.28948#Ch5.F4), we show the evolution of the top 10 singular vectors of the first layer in the model across different rounds of training\. In the first round, the top two singular vectors correspond exactly to MNIST digit detectors, while there is little signal in the CIFAR portion\. Later rounds have finer and finer grained curve detectors, and a growing sensitivity in the CIFAR portion\. At some point the model is forced to rely on CIFAR due to the destruction of the signal in MNIST\. These visualizations concretely demonstrate that PGD is successfully blocking the simple features in MNIST\-CIFAR, forcing a simple MLP to overcome its own simplicity bias\.

\(a\)Normal\(b\)Swapped\-class
Figure 5\.5:Left:Training and validation error for successive rounds with PGD where we flatten the gradient and singular vectors before projection\.Right:Swapped\-class errors throughout rounds\. We see that no new features are learned when ignoring the matrix structure of the parameters, so the spectral design is critical\.\(a\)Normal\(b\)Swapped\-class
Figure 5\.6:Left:Training and validation error for successive rounds with Muon\.Right:Swapped\-class errors throughout rounds\. Even though Muon is incentivized to give relatively more gradient to less prominent features \(through higher\-rank upweighting\), it is insufficient for picking up CIFAR\.In Figure[5\.5](https://arxiv.org/html/2608.28948#Ch5.F5), we examine the performance of a similar method that orthogonalizes the gradient\-as\-a\-vector \(similar to OrthoGrad\([Prieto et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib156)\)\)\. We see that this method does not result in significant learning of CIFAR\. This is likely because orthogonality in such a high dimensional space is easy to satisfy trivially\. Hence it is critical to take into account the matrix structure of the model\. In addition we see in Figure[5\.6](https://arxiv.org/html/2608.28948#Ch5.F6)that Muon\([Jordan et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib89)\), an optimizer designed to take advantage of features downspectrum, is also not able to take advantage of the CIFAR signal in the dataset\. Thus, even more modern tricks and optimizer designs are unable to overcome the foundational issue of simplicity bias\.

Table 5\.1:Probing accuracy for CIFAR on middle layers for an ensemble of 5 models drawn from early or late rounds of PGD\. We see that early rounds are substantially worse at extracting CIFAR information from the middle of the model, reinforcing the argument that PGD forces the model to learn more\.As our evaluations are based on swapping inputs and relying on the output predictions of the final model, it is possible that we are missing some encoding of CIFAR that is present in middle layers of the model\. In Table[5\.1](https://arxiv.org/html/2608.28948#Ch5.T1)we concatenate all representations from middle layers and train a linear probe to classify the CIFAR car vs\. truck using an ensemble of models drawn from early rounds of PGD, and drawn from late ones\. More details on the experimental design in Appendix[C\.1](https://arxiv.org/html/2608.28948#A3.S1)\. We see that later models have a significantly higher probing performance than earlier ones, reinforcing the conclusion that PGD extracts diverse information in successive rounds\. The absolute numbers here are likely high as the probe is near the capacity of the original model due to its size\.

All of the preceding experiments were performed on a very simple model system, but we hope they give the reader a sense of the potential issue on larger scale systems\. Next we turn to larger systems, and show that very similar issues still plague neural network training\. We find that our naive design of PGD fails in these larger systems, so we introduce a modified method\.

### 5\.3Diverse Feature Learning in Representation Space

Much ink has been spilled about rich or diverse feature learning in neural networks\.[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230)demonstrate the conundrum clearly at larger scale\. They show that a larger neural network performs similarly to a parameter\-matched ensemble of narrower networks in\-distribution when all networks are trained on the same data\. However, when linear probing the ensemble out of distribution on a related task, the ensemble performs substantially better\. This result was shown not only for image classification, but also for self\-supervised objectives and across architectures\.[Yang et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib218)expanded the conclusions to point out that, when matched for computation, the larger network still underperforms the narrower networks out of distribution\.

Such a result tells us that any individual training run likely misses some information in the data, even though the model may be larger\. Where it gains slightly in distribution, its features do not generalize\. In a world where the deployment domain may be significantly different than the training distribution, we believe we should optimize for these generalizable features\. In particular,[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230)suggest quite strongly that current dogma for scaling individual models\([Kaplan et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib93)\)misses a critical mass of features underneath the proverbial ice\.

In the following sections, we borrow a similar experimental protocol to show that incentivizing diverse feature learning results in substantial improvements over simple random ensembles in larger scale settings\. In particular we compare ensembles of 10 smaller models to a single larger model that is 4 times as wide \(hence roughly 16 times the parameters as well as more training compute\)\. See Appendix[C\.2](https://arxiv.org/html/2608.28948#A3.S2)for more details\.

#### 5\.3\.1A Problem with Projected Gradient Descent

Our first instinct is to replace the random ensembles in[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230)with ensembles of models acquired via repeated rounds of PGD, similar to the toy case\. However, we encounter an issue in more complex systems: PGD enforces orthogonality, but this does not mean the information content needs to be distinct\.

For example, if most of the entries of a vector are zero, with some large values, one can find a pathologically orthogonal vector with the same large values by pushing a small negative value into the rest of the positions\. This likely would represent the same semantic content, but would satisfy PGD\.

Table 5\.2:OOD linear probing accuracy between different methods\. We observe the same effect as[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230), that random ensembles \(CAT\) are much stronger than a large model that is parameter advantaged, but PGD ensembles do not appear to improve over random\.\(a\)CAT\(b\)PGD
Figure 5\.7:Linear CKA\([Kornblith et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib101)\)between layers for random ensembles \(CAT\) and PGD ensembles\. The values are quite similar, suggesting that PGD is managing to find similar solutions as simple random reruns, possibly permutations of the original weights\. Hence, PGD does not capture every notion of diversity\.In addition, one might be able to find permutations of a vector that are orthogonal, but the network could learn to remap those permutations in the next layer\. We believe we encounter such an issue for the following reason: when comparing random ensembles \(CAT for short\) using different initializations with PGD ensembles that used the same initialization, we find the OOD performance between the random and PGD ensembles to be close \(Table[5\.2](https://arxiv.org/html/2608.28948#Ch5.T2)\)\. In addition we measure similarity between representations of different members of each ensemble using CKA\([Kornblith et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib101)\), and find the intra\-ensemble representational similarity to be near the same magnitude for CAT and PGD \(Figure[5\.7](https://arxiv.org/html/2608.28948#Ch5.F7)\)\. As CKA is permutation invariant, this means that PGD ensembles are no more diverse than random ensembles, so it is very likely that PGD is rediscovering permutations of the same features\.

However, we would like the functional behavior for different members of the ensemble to be different\. Given the success of CKA\([Kornblith et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib101)\)in gauging representational similarity, we propose to use this metric directly as an optimization objective\. Minimizing CKA corresponds precisely to finding features that represent the data differently\. CKA is also desirable as it avoids the permutation sensitivity of PGD\. We describe our approach below\.

#### 5\.3\.2Inter\-Example Similarity as a Measure of Functional Diversity

Linear CKA\([Kornblith et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib101)\)is a measure of the similarity between gram matrices\. In order to repurpose this for maximizing diversity, at each timestep, we sample a random previously trained member of the ensemble, run a batch of examples through that network, then collect representations at middle layers\. Next, we run the same batch through the model we are currently training, and collect representations at the same middle layers\. Finally, we compute the average linear CKA between the two sets of features at each layer, and add that with a scalar coefficient to the training loss\. As we finish training a model, we add that to the buffer of previously trained models\.

To be precise, suppose we are given a batch of inner\-layer representations from modeliiin an ensemble at some layerl∈\{0,1,…,L\}l\\in\\\{0,1,\.\.\.,L\\\}, given byXli∈ℝB×dX^\{i\}\_\{l\}\\in\\mathbb\{R\}^\{B\\times d\}, where linear CKA is computed by

CKA​\(X,Y\)=∥X​Y⊤∥F2∥X​X⊤∥F​∥Y​Y⊤∥F\\text\{CKA\}\(X,Y\)=\\frac\{\\lVert XY^\{\\top\}\\rVert\_\{F\}^\{2\}\}\{\\lVert XX^\{\\top\}\\rVert\_\{F\}\\lVert YY^\{\\top\}\\rVert\_\{F\}\}\(5\.3\)for two centered matricesX,Y∈ℝB×dX,Y\\in\\mathbb\{R\}^\{B\\times d\}\.

For model 0, we train as usual\. For modelii, at every batch we samplejjuniformly randomly from\{0,1,…,i−1\}\\\{0,1,\.\.\.,i\-1\\\}, then compute

ℒCKA​\(fi\)=1N​∑l=1NCKA​\(Xli,Xlj\)\\mathcal\{L\}\_\{\\text\{CKA\}\}\(f\_\{i\}\)=\\frac\{1\}\{N\}\\sum\_\{l=1\}^\{N\}\\text\{CKA\}\(X^\{i\}\_\{l\},X^\{j\}\_\{l\}\)\(5\.4\)
for a selection ofNNlayers taken as a hyperparameter\. The final loss we optimize for ensemble memberfif\_\{i\}becomes

ℒ\(i,x,y\)=ℒtask\(fi,x,y\)\+γ𝕀\[i\>0\]ℒCKA\(fi,fj\)\.\\mathcal\{L\}\(i,x,y\)=\\mathcal\{L\}\_\{\\text\{task\}\}\(f\_\{i\},x,y\)\+\\gamma\\mathbb\{I\}\[i\>0\]\\mathcal\{L\}\_\{\\text\{CKA\}\}\(f\_\{i\},f\_\{j\}\)\.\(5\.5\)
with an additional coefficient for optimization\. In all our experiments we tune the choice of layer as well as the coefficientγ\\gamma\. In an ideal world we would not sample a previous model, instead running CKA against all prior models simultaneously\. Unfortunately, this is prohibitive either from a disk space perspective, or from a memory perspective as it requires storing intermediate representations for all prior models\. Hence we rely on sampling\.

##### Image Classification

To test our hypothesis, we begin with ImageNet to CIFAR transfer, studied in[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230)\. For computational reasons we train on ImageNet64x64\([Chrabaszcz et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib33)\), and transfer via linear probing to CIFAR100\([Krizhevsky, 2009](https://arxiv.org/html/2608.28948#bib.bib102)\)\. We use a ResNet18\([He et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib67)\)as our base model, train a wider model with 4x the width multiple, and train ensembles that are width\-matched \(4 members\) and larger \(10 members\) to approach parameter matching\. The wider model has 181m parameters, while the original model has 11m parameters\. We tune learning rates for all model sizes independently\.

For CKA ensembles, we select a variety of middle layers that feed into the residual stream and compute CKA loss on average across these layers\. We report the best setting over hyperparameters\. More details are available in Appendix[C\.2](https://arxiv.org/html/2608.28948#A3.S2)\.

Table 5\.3:Top\-1 OOD accuracy for random ensembles \(CAT\) v\.s\. CKA ensembles\. We see that CKA improves by a substantial margin over CAT in the linear probing setup\.We see in Table[5\.3](https://arxiv.org/html/2608.28948#Ch5.T3)that, like prior work\([Zhang and Bottou, 2023](https://arxiv.org/html/2608.28948#bib.bib230);[Yang et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib218)\), random ensembles show a significant benefit in OOD linear probing\. We also find that CKA improves further upon these random ensembles by a substantial margin\. By incentivizing diversity, even completely agnostic to the problem, we can find substantial gains\.

##### Language Modeling

One possible criticism of the above experimental protocol that extends to all prior work is the focus on a discriminative objective\. So we turn to the task that is currently most important in the deployment of machine learning: language modeling\. This task has been previously ignored by the literature on diverse feature learning, so it’s unclear whether there is something special about discriminative objectives in the prior results\.

We follow the experimental protocol with slight differences\. We train a Transformer\([Vaswani et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib204)\)language model on BabiStories\([Zhang et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib231)\)\. We use a model 3x the width vs\. an ensemble of 6 models with 1x width as they are similar compute\. For in\-distribution evaluation, we measure loss on the validation dataset, while for out of distribution, we measure loss on a related dataset, Simple English Wikipedia\([Coster and Kauchak, 2011](https://arxiv.org/html/2608.28948#bib.bib34)\)\. We choose this dataset as there is a high degree of lexical overlap, due to the simple language, but the topic distribution is quite different, thus any improvements must be due to learning the general structure of language\. The benefit of language modeling is we don’t require the linear probing protocol as it is possible to measure out\-of\-distribution without training any additional parameters\. This also closely matches how language models are used in the wild with novel queries\.

Table 5\.4:OOD cross entropy for random ensembles \(CAT\) v\.s\. CKA ensembles under many different variations\. Unlike the image classification case, we are unable to find a version of CKA that outperforms random ensembles\.In Table[5\.4](https://arxiv.org/html/2608.28948#Ch5.T4), we find similarly to image classification, that simple random ensembles perform better out of distribution than the larger model\. This is striking given the in distribution performance is slightly worse\. However, surprisingly, the prior CKA method does not perform well\. In all cases we present the best results over hyperparameter tuning \(see Appendix[C\.3](https://arxiv.org/html/2608.28948#A3.S3)\)\.

There are numerous reasons this might be the case: we may not enable CKA on the correct modules, hence we perform a sweep over different sections \(a full sweep would be combinatoric and is difficult due to computational constraints\)\. It might also be that, due to the residual structure of transformer language models, optimizing CKA would lead to learning the same features in different layers\. To narrow this down, we try a version of our objective where we encourage diversity over CKA for only the maximum pairwise similarity over many layers in the model \(CKA max\-in\-depth\)\. However, this still is insufficient to drive improvements over random ensembles \(see Table[5\.4](https://arxiv.org/html/2608.28948#Ch5.T4)\)\.

Another possibility for the failure is that, as stated, our CKA works in high dimensional space, trying to make the sequence of representations different, which to be precise, is computing a64×6464\\times 64matrix of inner products from256×768256\\times 768dimensional vectors\. There are so few samples for the size of this vector that there may be a trivial solution for the language model to find by permuting information among the tokens, so instead we consider CKA over the pooled representations from a sequence \(CKA token pool\), or computing a\(64⋅256\)×\(64⋅256\)\(64\\cdot 256\)\\times\(64\\cdot 256\)gram matrix \(CKA token cloud\) instead where we ask the token representations to be different\. Neither of these are successful, as shown in Table[5\.4](https://arxiv.org/html/2608.28948#Ch5.T4)\.

There might still be a problem with the dimensionality of our objective, so orthogonally we try random matrix projections that optimize CKA in a low dimensional space achieved by projecting with a different random matrix at every time\. This means that the model cannot just fool the objective by picking a particular set of coordinates in which to push orthogonality, still we see in Table[5\.4](https://arxiv.org/html/2608.28948#Ch5.T4)that this is insufficient to beat random ensembles\.

We are left with the question then: why does CKA fail in language modeling where it previously succeeded\.

##### A Problem with Optimizing CKA

\(a\)CAT\(b\)CKA
Figure 5\.8:CKA for representation vectors and gradient vectors for the task loss alone at different layers for image classification\.Left:We see that representation and gradient CKA are reasonably high for CAT\.Right:The same, but for the CKA\-trained models\. We see that using CKA greatly affects the representation CKA as it optimizes for this, and it also affects the task gradient CKA\.\(a\)CAT\(b\)CKA
Figure 5\.9:CKA between the gradient vectors for the task loss alone at different layers for language modeling\.Left:We see CKA between representations and between gradients, which are both high for the CAT models\.Right:The same, but for the CKA\-trained models\. We see that while representation CKA is low, gradient CKA is unaffected by our CKA objective, unlike image classification\.We believe the answer to this question has to do with the effective difficulty of the solution\. In image classification there were relatively few classes, which means that, especially toward the output of the model[Papyan et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib147), it is possible to make the inter\-example similarity low as there is a fair amount of room to move\. In the case of language models, however, much more of the capacity is required for the task \(it is impossible to achieve 0 training error\), thus it is not easy to achieve the desired semantic dissimilarity and instead the network finds a pathological solution\. The reason we know the solution is pathological is that we find that the CKA is high between the gradient vectors of the next token prediction task \(Figures[5\.8](https://arxiv.org/html/2608.28948#Ch5.F8)and[5\.9](https://arxiv.org/html/2608.28948#Ch5.F9)\)\. This means that the model has learned to fool the CKA objective, while not actually altering its downstream predictions\. Where representations have low CKA between models, the actual output behavior is very similar\.

The core issue with the CKA objective, and the previous PGD objective, is that it relies on the model’s implementation of a function, and conflates the structure in high dimensional space with the model’s actual function, but it’s impossible to measure the actual functional change through CKA\. In order to get an absolute sense on the functional difference, we have to look at the output space of the model instead\.

### 5\.4Diverse Feature Learning in Output Space

We have made a roundabout journey to land on a much simpler definition of diversity, employed many times in prior literature\. We want the outputs of different members of the ensemble to look different, because we know that if the members make different predictions, they must by definition be different functions\.

#### 5\.4\.1Negative Correlation Learning

One classic way to enforce diversity in this way is negative correlation learning \(NCL\)\([Liu and Yao, 1999](https://arxiv.org/html/2608.28948#bib.bib116)\)\. This was first defined for regression tasks\. At its core NCL aims to learn an ensemble whose ensembled predictions are good at the task, while each member is distinct\. See[Buschjäger et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib26)for a recent treatment with much more context\. NCL corresponds to a general structure:

ℒNCL​\(f0,…,fm,x,y\)=ℒtask​\(fens,x,y\)−γ​ℒdiv​\(f0,…,fm,x,y\)\\mathcal\{L\}\_\{\\text\{NCL\}\}\(f\_\{0\},\.\.\.,f\_\{m\},x,y\)=\\mathcal\{L\}\_\{\\text\{task\}\}\(f\_\{\\text\{ens\}\},x,y\)\-\\gamma\\mathcal\{L\}\_\{\\text\{div\}\}\(f\_\{0\},\.\.\.,f\_\{m\},x,y\)\(5\.6\)
wherefensf\_\{\\text\{ens\}\}is the ensembled model from the individualsf0,…,fmf\_\{0\},\.\.\.,f\_\{m\}by some weighting that need not be uniform\. The task loss is already defined by whatever we wish to do \(language modeling in this case\), so it remains to discuss the diversity loss which we want to maximize, hence the minus sign\. Typically it is phrased as:

ℒdiv​\(f0,…,fm,x,y\)=1m​∑ildiff​\(fens,fi,x,y\),\\mathcal\{L\}\_\{\\text\{div\}\}\(f\_\{0\},\.\.\.,f\_\{m\},x,y\)=\\frac\{1\}\{m\}\\sum\_\{i\}l\_\{\\text\{diff\}\}\(f\_\{\\text\{ens\}\},f\_\{i\},x,y\),\(5\.7\)
which signifies that we want to maximize the disagreement between the ensemble and any individual member\. As we are working with language models, we experiment with both Jensen\-Shannon Divergence \(JSD\) and simple mean squared error \(MSE\) forldiffl\_\{\\text\{diff\}\}\.

Usually NCL is for a set of models to be optimized jointly online\. Prior work has shown that, especially in high dimensional deep learning, joint optimization can easily collapse to poor solutions\([Jeffares et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib85)\), hence we train models sequentially as in the case of our previous PGD and CKA experiments\.

In addition, we face a similar memory and disk constraint with computingℒdiv\\mathcal\{L\}\_\{\\text\{div\}\}, so we make the same sampling approximation as before\. When training modelii:

ℒdiv^​\(f0,…,fi,x,y\)=ldiff​\(12​\(fi\+fj\),fi,x,y\)\\hat\{\\mathcal\{L\}\_\{\\text\{div\}\}\}\(f\_\{0\},\.\.\.,f\_\{i\},x,y\)=l\_\{\\text\{diff\}\}\(\\frac\{1\}\{2\}\(f\_\{i\}\+f\_\{j\}\),f\_\{i\},x,y\)\(5\.8\)
where the indexjjis sampled uniformly randomly from\{0,…,i−1\}\\\{0,\.\.\.,i\-1\\\}\.

Table 5\.5:OOD cross entropy for random ensembles v\.s\. NCL ensembles We see no significant difference to random ensembles\. NCL JSD corresponds to using Jensen\-Shannon divergence for difference\. MSE corresponds to MSE for the probability distribution, and Top\-1 corresponds to only considering the difference between probabilities of the correct class\.We see in Table[5\.5](https://arxiv.org/html/2608.28948#Ch5.T5)that our straightforward attempts at NCL are insufficient \(see Appendix[C\.3](https://arxiv.org/html/2608.28948#A3.S3)for tuning\)\. In particular we hypothesize that one issue with NCL is similar to prior experiences in CKA and PGD, whereby the high dimensional probability distribution can find pathological solutions for diversity, so we restrict to only the top\-1 of the output probability distribution \(NCL correct only\)\. Still we see no major improvement for the best case of each\.

#### 5\.4\.2Boosting Incorrect Predictions

Another large issue with negative correlation learning is that it incentivizes models to disagree even when the label is correct\. One might instead consider an even simpler boosting\-like\([Freund et al\., 1999](https://arxiv.org/html/2608.28948#bib.bib55)\)method whereby we train models sequentially, and later members need to compensate for the error of previous models\. This is similar in spirit to the Bonsai Method suggested by[Zhang et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib229)on simpler image classification tasks\. However, in their work, the training set was continuously pruned such that later members of the ensemble would have shrinking datasets, which leads exactly to the kind of spurious correlation issue we are trying to avoid\. Rather, we believe that there are other, more complex ways to representthe sameexamples, hence we wish to preserve the loss on the entire distribution, but only slightly upweight for regions not covered by previous training runs\.

We implement this by training successive models such that later models have their loss upweighted by the error of the ensemble, sometimes called focal loss\([Lin et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib115)\)\. In particular, at every optimization step we sample a previously trained model, then add a weighting term to the cross entropy loss corresponding to the incorrectness of the previous model, thus errors are boosted in the loss\. However, unlike[Zhang et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib229), we preserve the original loss by the use of a small coefficient on the focal term\. Suppose the output of modeliifor token t is probability vectorpi∈ℝCp^\{i\}\\in\\mathbb\{R\}^\{C\}for a vocabulary ofCCclasses, then the loss for modeliiat tokenttis\.

ℒ⁡\(fi,fj,x,y\)=w⁡\(fj,x,y\)^∗ℒtask​\(fi,x,y\)\\mathcal\{L\}\(f\_\{i\},f\_\{j\},x,y\)=\\hat\{w\(f\_\{j\},x,y\)\}\*\\mathcal\{L\}\_\{\\text\{task\}\}\(f\_\{i\},x,y\)\(5\.9\)
with weight

w⁡\(fj,x,y\)=\(1\+γ∗\(1−pcorrectj\)\)w\(f\_\{j\},x,y\)=\(1\+\\gamma\*\(1\-p^\{j\}\_\{\\text\{correct\}\}\)\)\(5\.10\)
wherejjis sampled randomly and uniformly from\{0,…,i−1\}\\\{0,\.\.\.,i\-1\\\},γ\\gammais a hyperparameter, andw⁡\(fj,x,y\)^\\hat\{w\(f\_\{j\},x,y\)\}corresponds to the normalized weight in a sequence such that the weights sum to 1\. We normalize so that the learning rate does not need to be tuned jointly\. Thus, we shape the loss for subsequent rounds by the mistakes of previous rounds, while always giving some weight to the original task loss to avoid the overfitting issues of a shrinking sample\. We normalize the weight so that we do not need to retune learning rates due to different gradient magnitudes\.

Table 5\.6:OOD cross entropy for random ensembles v\.s\. Focal scaling ensembles We see no significant difference to random ensembles\. Top\-p corresponds to scaling the top\-p of the tokens by sequence by a constant additive factor\.We also consider a version where the loss at a token is simply scaled by a flat factorγ\\gammaas long as1−pcorrectj1\-p^\{j\}\_\{\\text\{correct\}\}is in the top P portion of incorrect tokens in a sequence\. This is so that we do not upweight errors that are large but unlearnable\. We find in Table[5\.6](https://arxiv.org/html/2608.28948#Ch5.T6)that both of these variants are still insufficient to outperform simple random ensembles even when searching overγ\\gammaand the proportion of top p\. Find details on tuning in Appendix[C\.3](https://arxiv.org/html/2608.28948#A3.S3)\.

#### 5\.4\.3Boosting Low\-Entropy Predictions

We follow a parallel path, exploring entropy\-based uncertainty instead of incorrectness, which has seen success in other areas\([Wang et al\., 2026](https://arxiv.org/html/2608.28948#bib.bib211)\)\. To be precise, following the general formulation of equation[5\.9](https://arxiv.org/html/2608.28948#Ch5.E9)and the normalization of the last section, we design a weight based on entropy as

w⁡\(fi,x,y\)=\(1\+γ∗H⁡\(pj\)\)\.w\(f\_\{i\},x,y\)=\(1\+\\gamma\*H\(p^\{j\}\)\)\.\(5\.11\)
We also experiment with a threshold version as in the previous section, where we scale the loss byγ\\gammawhen the entropy of the token distribution under a previous model is within the top\-p entropies for the sequence\.

Table 5\.7:OOD cross entropy for random ensembles v\.s\. entropy scaling ensembles We see no significant difference to random ensembles\. Top\-p corresponds to scaling the top\-p of the tokens by sequence by a constant additive factor\.We find in Table[5\.7](https://arxiv.org/html/2608.28948#Ch5.T7)similarly to focal loss that we are unable to improve upon random ensembles\. Similar arguments may apply as in the case of focal\-loss: some unconfident predictions are very difficult to learn, and forcing the model to learn them may sacrifice other areas of the space\.

#### 5\.4\.4Ensemble Weighting

As mentioned in the CKA section, we train multiple language models and ensemble their predictions to test the ensemble out of distribution\. Throughout the prior experiments we have used uniform ensemble weighting as a straightforward baseline, taking the simple average of the probability distributions in order to compute downstream metrics, but especially in the case of entropy and focal loss, it’s very possible that the models are differently calibrated, so even weighting over the distributions would not yield an ideal outcome\.

As a result, we test a variety of heuristic per\-token ensembling methods\. Given a relevance function for a function and tokenr⁡\(fi,xt\)r\(f\_\{i\},x\_\{t\}\), we compute a softmax distribution over all members of the ensemblefif\_\{i\}with temperatureTT\. To be precise, we compute the weight offif\_\{i\}in ensemble as

w⁡\(fi,xt\)=softmax​\(r⁡\(fi,xt\)T\)w\(f\_\{i\},x\_\{t\}\)=\\text\{softmax\}\\left\(\\frac\{r\(f\_\{i\},x\_\{t\}\)\}\{T\}\\right\)\(5\.12\)
As relevance metrics we choose entropy and KL distance from the ensemble distribution\. To be precise,r⁡\(fi,xt\)=H⁡\(fi​\(xt\)\)r\(f\_\{i\},x\_\{t\}\)=H\(f\_\{i\}\(x\_\{t\}\)\)orr⁡\(fi,xt\)=K​L​\(fi​\(xt\),fens​\(xt\)\)r\(f\_\{i\},x\_\{t\}\)=KL\(f\_\{i\}\(x\_\{t\}\),f\_\{\\text\{ens\}\}\(x\_\{t\}\)\)\.

Table 5\.8:OOD cross entropy for different ensemble weighting methods on the Entropy top\-p runs\. We use these as the base as we expect there may be nonuniformity compared to the Random ensembles\.As seen in Table[5\.8](https://arxiv.org/html/2608.28948#Ch5.T8), we are unable to make gains over random ensembling in all of these cases\. We also combine linearly with uniform weighting in the hopes of smoothing out spiky predictions due to overfitting, but find none of these methods superior to even weighting\. See Appendix[C\.4](https://arxiv.org/html/2608.28948#A3.S4)for more details on tuning\.

We are unsure as to this perplexing observation that it is difficult to beat random ensembling\. We believe that the root issue here is that we are again unable to tell whether low entropy or large disagreement from the ensemble is due to a correct but different prediction, or an incorrect but different prediction\. The small degree to which the exact OOD evaluation numbers seem to drift suggests both effects are evenly at play\. Still, this suggests that there may be room to grow along this exact axis, possibly experimenting with holdout sets for global weighting\.

It is also possible that these results suggest that high\-entropy or incorrect predictions primarily define the same task as the entire dataset, as cross entropy is magnified on the most difficult examples\. In some sense, the difficult examples form the support\-vectors of the training set, so entropy weighting recovers similar solutions\. Prior work in dataset distillation indicates this may be possible\([Wang et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib207)\)\. Then the difficulty is to find structured subsets that yield different solutions than the entire dataset, without heuristics and while avoiding possible newly\-introduced spurious correlations\.

### 5\.5Related Work

The themes of the prior chapter: simplicity bias, diversity in features and ensembling, are already well\-represented in the literature\. We present a non\-exhaustive description of prior work, and articulate our contribution below\.

#### 5\.5\.1Leveraging Downstream Information

One class of solutions to find robust neural networks is to leverage information about the downstream domain\. For example, one might know broad statistics about the task distribution, or have some tiny sample to quickly adapt\. For example,[Wen et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib213)design a low rank penalty on the last layer for OOD prediction which requires group labels\([Hu et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib73);[Sagawa et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib171)\)\.[Pagliardini et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib145)propose a training method to enforce diverse representations on OOD data, which requires a sample\.[Lee et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib110);[Chen et al\. \(2023a\)](https://arxiv.org/html/2608.28948#bib.bib28)both require test data\.[Kirichenko et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib98)make a similar point that retraining only the final layer of a pretrained classifier on test data is very strong compared to bespoke methods\. We consider access to test data to be an undesirable assumption, as we would like to extract all the information available in a dataset, agnostic to the downstream distribution\.

#### 5\.5\.2Relying on Pretrained Models

A large class of methods relies on pretrained models as feature learners, evading the question of dynamics entirely\.[Gontijo\-Lopes et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib59)show that ensembling features from various diverse pretrained models outperforms narrow ensembles, where the diversity is the key\.[Teney et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib195)train a set of linear heads simultaneously on top of a fixed extractor, enforcing orthogonality among the heads, but such a method fails without the pretrained model \(as we saw\)\.[Tiwari and Shenoy \(2023\)](https://arxiv.org/html/2608.28948#bib.bib198);[Joshi et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib90)make sure that the final classification depends on features throughout a pretrained model\.[Niu et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib141)use an autoencoder based on a pretrained model’s features\.[Izmailov et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib83)make the point that retraining the last layer of a pretrained model can achieve SOTA compared to more complex group robustness methods, where the architecture, hyperparameters and data of the pretrained feature extractor are the most important levers\. We agree strongly with this conclusion, which is why we are interested in altering the learning dynamics of networks from scratch, instead of relying on an existing model\.

#### 5\.5\.3Architectural Modifications

Another sensible approach would be to design architectures around diversity\. One of the clearest examples would be mixture of experts \(MoE\) layers\([Shazeer et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib181);[Zhou et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib232)\)\. These were initially designed to make it possible to train much wider networks on less hardware through sparse routing, but can be interpreted as an ensemble inside a model\. Still[Dai et al\. \(2024\)](https://arxiv.org/html/2608.28948#bib.bib37)show in Section 4\.3 Table 2 that a dense model as big as an MoE is slightly better, so even if experts specialize to different data, we should not expect that the features learned are more diverse\. MoE alone is not enough\. This is further supported by the fact that each expert sees less data than the whole, so they have a larger incentive to pick up spurious correlations\. On a different and much smaller\-scale note,[Addepalli et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib3)design a two\-pronged architecture to attack the MNIST\-CIFAR task, but this is too task\-specific to scale\.[To et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib199)force a classifier to specialize to different subsets of the original training data with constraints on the decoding\. Still we believe, given the generality of the results in this document, that neural networks are subject to a common paradigm\. Though architectural improvements may be helpful, they do not attack the issue underlying spurious correlations\.

#### 5\.5\.4Objective and Optimization Approaches

As one might modify the architecture, researchers have also taken the approach of introducing new objectives\.[Sinha et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib187)introduce a method with multiple decoder heads that are optimized with a GAN\-like contrastive loss to produce diverse representations\. We believe this has similar high dimensional issues as we encountered with CKA on more complex tasks\.[Morwani et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib134)prove that 1 hidden\-layer models depend on a linear subspace, then design an optimization method OrthoP which projects the input off of the low\-rank solution of the previous run, like our PGD method\. However, as it is naive projection without the SVD basis, it is quite easy to fool in high dimensions\.[Saha et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib172)design a PGD method quite like what we converged on for applications in continual learning, though only in input space\. We are more interested in these approaches than previous buckets, but in many cases they encounter the same pitfalls that we did in prior sections\.

#### 5\.5\.5Ensembling

Lastly there are many generic results for the benefit of ensembling, hence our focus in this area\.[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230)make the clear observation that an ensemble of small networks beats a single larger network OOD by a substantial margin, even though their in distribution performances are similar\. Thus we are missing features in optimization\.[Yang et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib218)expands this result to the step\-matched setting, finding the same conclusions\.[Abe et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib1)argue that OOD performance in deep ensembles is primarily driven by ID performance\. Such an argument is supported by[Mandal et al\. \(2026\)](https://arxiv.org/html/2608.28948#bib.bib124), where better optimization for individual members yields stronger results than naive scaling of the ensemble\([Kondratyuk et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib100);[Lobacheva et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib120);[Kim et al\., 2026](https://arxiv.org/html/2608.28948#bib.bib97)\)\. One very clear way to encourage diversity is to upsample the mistakes of previous models\([Liu et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib117);[Zhang et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib229)\), but in cases like MNIST\-CIFAR where there are no mistakes, or where the mistake set grows ever smaller, this is likely to end in further spurious correlations\. We believe the problem of generically finding a way to incentivize diversity in individual members is still quite open\. The frustration of the prior chapter shows this is not a trivial task\.

### 5\.6Discussion

Diverse feature learning means getting more out of less data\. What does a world look like in which we don’t need the whole internet to train language models? We believe the key to this problem is overcoming the greedy process whereby neural networks learn the simplest features first and never revisit those choices\. Thus, we attack spurious correlations head on\.

We begin with a simple testbed of spurious correlations, the MNIST\-CIFAR task\([Shah et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib178)\)\. We show that we can use our previously\-developed spectral dynamics \(Chapter[4](https://arxiv.org/html/2608.28948#Ch4)\) as a definition for simplicity bias, which leads to a method using projected gradient descent on singular vectors\. This method suffices to overcome simplicity bias without the task\-specific assumptions of prior work\.

We then scale our method to more complex architectures and tasks, where we find that the task difficulty renders such a simple method ineffective\. After finding that our method hides model\-wise similarity, we turn to using that similarity metric directly as a diversity loss\. We find that in larger image classification networks we are able to beat simple random ensembles, a surprisingly strong baseline that is already stronger than large models\([Zhang and Bottou, 2023](https://arxiv.org/html/2608.28948#bib.bib230)\)\.

Next, we generalize this experimental protocol to language models, where we demonstrate that ensembles of small models perform better than larger models out\-of\-distribution, extending the observations of[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230);[Yang et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib218)\. However, our previous diversity losses do not appear to help for language models, which we elucidate through a large number of ablations\.

We believe the missing piece is a way to isolate structured subsets of the data so as to find diverse solutions\. Merely restricting to difficult examples either finds the same solutions as random ensembles, or encounters spurious correlations\. The core difficulty is finding such structured subsets without heuristics engineered to fit the problem at hand\.

Though they are still incomplete, these results suggest that there may be a new path forward for scaling computation\. The basic observation that ensembles of smaller models perform better than larger ones OOD can be extended to any domain where neural networks are used, including robotics where domain shifts are common and expected\. Across domains, there are many features hiding beneath the ice, waiting to be discovered\.

## Chapter 6Conclusion

It is very clear in this moment that neural networks have changed and will continue to change technology and our world\. Yet we have a very poor understanding of how they work\. The more embedded neural networks become, the more pressing the gap\. Moreso as neural networks take root in economic, social and political questions, far outside of our domain as computer scientists\.

This thesis strikes at understanding neural networks\. We begin with a strange phenomenon, and carry this thread all the way through to improving neural networks in the low\-data regime: the critical problem that scaling has been unable to overcome by definition\. Briefly, we recap the contributions of this thesis\.

Chapter[3](https://arxiv.org/html/2608.28948#Ch3), on Convex Mode Connectivity, shows that neural network optima occupy a high dimensional simplex in the loss landscape, that this is true across a wide variety of neural networks, and that trivial mathematical explanations for this phenomenon do not apply\. At the time of presentation, such complexity and generality was unexpected\. A body of theoretical literature had presented simple explanations, and prior model classes do not have such nontrivial properties\. Moreover, the generality and high dimensionality was genuinely perplexing as if there were so many solutions, how come our optimization process did not find them?

Chapter[4](https://arxiv.org/html/2608.28948#Ch4), on Spectral Dynamics, demonstrates across data, architectures, optimizers, and hyperparameters, there is a similar bias in optimization toward lower rank matrices\. Such a bias might be expected in simple theoretical settings, but again the assumptions of prior work did not hold in these more complex cases, which muddied the water as to the ultimate cause\. This work was a sign that something common explains the behavior of all neural networks\. Moreover these spectral dynamics allow us to explain the previously stated conundrum of Convex Mode Connectivity, tying together our understanding about neural networks\.

Chapter[5](https://arxiv.org/html/2608.28948#Ch5), on Diverse Feature Learning, used the lens of spectral dynamics to attack the problem of learning on less data, a problem for which the field currently has very few tools\. In particular, the bias toward low rank matrices provided a clue as to the actual mechanics behind spurious correlations and allowed us to find a task\-agnostic method for improving learning in a toy case, which we then scaled to more complex cases\. The full application of this understanding still eludes us, but we are firm in our belief that this is the next frontier of neural networks\.

Lastly, a note to my readers without access to immense resources\. Many well\-funded academics and companies would have you believe that the key to improving neural networks is simply scaling\([Hestness et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib69);[Kaplan et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib93)\)\. All one needs is a sufficient recipe\. Then collecting more data with more parameters will provide sufficient ammunition for solving the problem\. This is a powerful scaling hypothesis, attractive in its simplicity, and indeed it seems to hold when looking with a broad view across the field\. From AlexNet\([Krizhevsky et al\., 2012](https://arxiv.org/html/2608.28948#bib.bib103)\), to later models like VGG\([Simonyan and Zisserman, 2014](https://arxiv.org/html/2608.28948#bib.bib185)\), scale has almost always improved performance\. AlphaGo\([Silver et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib183)\)attacked the previously unimaginable game of Go, and shortly after AlphaZero\([Silver et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib184)\)showed scale was the primary ingredient\. Through the generations from GPT\([Radford et al\., 2018](https://arxiv.org/html/2608.28948#bib.bib158)\)to GPT2\([Radford et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib159)\)to GPT3\([Brown et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib24)\)to GPT4\([Achiam et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib2)\)it was quite clear the improvement in the model quality and output\. Scale was and is a powerful lever\.

However, it is a very blunt lever\. VGG\([Simonyan and Zisserman, 2014](https://arxiv.org/html/2608.28948#bib.bib185)\)was much larger than AlexNet\([Krizhevsky et al\., 2012](https://arxiv.org/html/2608.28948#bib.bib103)\), but ResNet\([He et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib67)\)was a fraction of the size and more performant\. MLPs can approximate ConvNets in theory, but we are unaware of empirical examples where this occurred without guidance\. We do not talk to our linear regressions, rather advances in architectures have led to our current language models\. Optimizers failed past a certain scale due to loss spikes\([Wortsman et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib216)\), which led to new optimizers that required a deeper understanding\([Jordan et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib89)\)\. I do believe in the bitter lesson\([Sutton, 2019](https://arxiv.org/html/2608.28948#bib.bib192)\), but only when the initial conditions are set correctly\. We may reach the edge of existing pretraining data scaling in language models\([Villalobos et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib205)\)\. That recent advances in the field are due to downstream techniques like RL is evidence that scaling has to be thought through\. RL is yielding impressive results in specific domains\([Guo et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib63)\), but not in full generality\([Ouyang et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib144);[Chen et al\., 2026](https://arxiv.org/html/2608.28948#bib.bib31)\)\. Even now there are many examples of more compute\-efficient improvements than pure scale\([Saunshi et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib173)\), including our own results in Chapter[5](https://arxiv.org/html/2608.28948#Ch5)\. And even if it were not so blunt, scale is an extremely expensive lever to pull given its diminishing returns\.

This thesis charts a different path\. I was and am curious if we can do more with less\. It is my hope that this work is helpful to myself and others toward this goal\. And it was my pleasure to skate on the edge of uncertainty in pursuit of this understanding\.

## References

- T\. Abe, E\. K\. Buchanan, G\. Pleiss, R\. Zemel, and J\. P\. CunninghamDeep ensembles work, but are they necessary?\.Advances in Neural Information Processing Systems35,pp\.33646–33660\.Cited by:[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1)\.
- Achiamet al\.\(2023\)J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.GPT\-4 Technical Report\.arXiv preprint arXiv:2303\.08774\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p5.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Addepalliet al\.\(2023\)S\. Addepalli, A\. Nasery, V\. B\. Radhakrishnan, P\. Netrapalli, and P\. JainFeature reconstruction from outputs can mitigate simplicity bias in neural networks\.InThe Eleventh International Conference on Learning Representations,Cited by:[§5\.5\.3](https://arxiv.org/html/2608.28948#Ch5.S5.SS3.p1.1)\.
- Ainsworthet al\.\(2022\)S\. Ainsworth, J\. Hayase, and S\. SrinivasaGit Re\-Basin: Merging Models modulo Permutation Symmetries\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§A\.1](https://arxiv.org/html/2608.28948#A1.S1.p1.1),[§3\.4](https://arxiv.org/html/2608.28948#Ch3.S4.p1.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Aletet al\.\(2025\)F\. Alet, I\. Price, A\. El\-Kadi, D\. Masters, S\. Markou, T\. R\. Andersson, J\. Stott, R\. Lam, M\. Willson, A\. Sanchez\-Gonzalez,et al\.Skillful joint probabilistic weather forecasting from marginals\.arXiv preprint arXiv:2506\.10772\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Andriushchenkoet al\.\(2023\)M\. Andriushchenko, F\. D’Angelo, A\. Varre, and N\. FlammarionWhy Do We Need Weight Decay in Modern Deep Learning?\.arXiv preprint arXiv:2310\.04415\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p5.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1)\.
- Anthropic \(2025\)AnthropicClaude code\.Note:Agentic command\-line coding tool released in research preview February 2025 and generally available May 2025External Links:[Link](https://claude.ai/)Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Aroraet al\.\(2018\)S\. Arora, N\. Cohen, and E\. HazanOn the optimization of deep networks: Implicit acceleration by overparameterization\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§B\.1](https://arxiv.org/html/2608.28948#A2.S1.p1.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p4.1),[§4\.2\.1](https://arxiv.org/html/2608.28948#Ch4.S2.SS1.p1.1),[§4\.3](https://arxiv.org/html/2608.28948#Ch4.S3.p2.1),[§4\.5\.3](https://arxiv.org/html/2608.28948#Ch4.S5.SS3.p2.1),[§4\.5](https://arxiv.org/html/2608.28948#Ch4.S5.p1.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p3.1)\.
- Aroraet al\.\(2019\)S\. Arora, N\. Cohen, W\. Hu, and Y\. LuoImplicit Regularization in Deep Matrix Factorization\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§B\.1](https://arxiv.org/html/2608.28948#A2.S1.p1.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1),[§3\.5](https://arxiv.org/html/2608.28948#Ch3.S5.p5.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p4.1),[§4\.2\.1](https://arxiv.org/html/2608.28948#Ch4.S2.SS1.p1.1),[§4\.3](https://arxiv.org/html/2608.28948#Ch4.S3.p1.1),[§4\.3](https://arxiv.org/html/2608.28948#Ch4.S3.p2.1),[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1),[§4\.5\.3](https://arxiv.org/html/2608.28948#Ch4.S5.SS3.p2.1),[§4\.5](https://arxiv.org/html/2608.28948#Ch4.S5.p1.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p3.1)\.
- Baiet al\.\(2022\)Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon, C\. Chen, C\. Olsson, C\. Olah, D\. Hernandez, D\. Drain, D\. Ganguli, D\. Li, E\. Tran\-Johnson, E\. Perez, J\. Kerr, J\. Mueller, J\. Ladish, J\. Landau, K\. Ndousse, K\. Lukosuite, L\. Lovitt, M\. Sellitto, N\. Elhage, N\. Schiefer, N\. Mercado, N\. DasSarma, R\. Lasenby, R\. Larson, S\. Ringer, S\. Johnston, S\. Kravec, S\. E\. Showk, S\. Fort, T\. Lanham, T\. Telleen\-Lawton, T\. Conerly, T\. Henighan, T\. Hume, S\. R\. Bowman, Z\. Hatfield\-Dodds, B\. Mann, D\. Amodei, N\. Joseph, S\. McCandlish, T\. Brown, and J\. KaplanConstitutional AI: Harmlessness from ai feedback\.arXiv preprint arXiv:2212\.08073\.Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Baraket al\.\(2022\)B\. Barak, B\. Edelman, S\. Goel, S\. Kakade, E\. Malach, and C\. ZhangHidden progress in deep learning: SGD learns parities near the computational limit\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1)\.
- Bartlett \(1996\)P\. BartlettFor valid generalization the size of the weights is more important than the size of the network\.Advances in Neural Information Processing Systems \(NeurIPS\)9\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Beeryet al\.\(2018\)S\. Beery, G\. Van Horn, and P\. PeronaRecognition in terra incognita\.InProceedings of the European conference on computer vision \(ECCV\),pp\.456–473\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p2.1)\.
- Belkinet al\.\(2019\)M\. Belkin, D\. Hsu, S\. Ma, and S\. MandalReconciling modern machine\-learning practice and the classical bias–variance trade\-off\.Proceedings of the National Academy of Sciences116\(32\),pp\.15849–15854\.Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Benderet al\.\(2021\)E\. M\. Bender, T\. Gebru, A\. McMillan\-Major, and S\. ShmitchellOn the dangers of stochastic parrots: Can language models be too big?\.InProceedings ACM Conference on Fairness, Accountability, and Transparency \(FAccT\),pp\.610–623\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p3.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1)\.
- Bentonet al\.\(2021\)G\. Benton, W\. Maddox, S\. Lotfi, and A\. G\. G\. WilsonLoss surface simplexes for mode connecting volumes and fast ensembling\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\.769–779\.Cited by:[§3\.2](https://arxiv.org/html/2608.28948#Ch3.S2.p3.1)\.
- Benzinget al\.\(2022\)F\. Benzing, S\. Schug, R\. Meier, J\. Von Oswald, Y\. Akram, N\. Zucchet, L\. Aitchison, and A\. StegerRandom initialisations performing above chance and how to find them\.InOPT 2022: Optimization for Machine Learning \(NeurIPS 2022 Workshop\),Cited by:[§3\.4](https://arxiv.org/html/2608.28948#Ch3.S4.p1.1)\.
- Bidermanet al\.\(2023\)S\. Biderman, H\. Schoelkopf, Q\. G\. Anthony, H\. Bradley, K\. O’Brien, E\. Hallahan, M\. A\. Khan, S\. Purohit, U\. S\. Prashanth, E\. Raff,et al\.Pythia: A suite for analyzing large language models across training and scaling\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§B\.2\.4](https://arxiv.org/html/2608.28948#A2.S2.SS4.p1.1),[§B\.2\.5](https://arxiv.org/html/2608.28948#A2.S2.SS5.p1.1),[§4\.5\.1](https://arxiv.org/html/2608.28948#Ch4.S5.SS1.p2.1)\.
- Biewald \(2020\)L\. BiewaldExperiment Tracking with Weights and Biases\.Note:Software available from wandb\.comExternal Links:[Link](https://www.wandb.com/)Cited by:[§B\.6](https://arxiv.org/html/2608.28948#A2.S6.p1.1)\.
- Blacket al\.\(2022\)S\. Black, S\. Biderman, E\. Hallahan, Q\. Anthony, L\. Gao, L\. Golding, H\. He, C\. Leahy, K\. McDonell, J\. Phang,et al\.Gpt\-neox\-20b: An open\-source autoregressive language model\.InProceedings of BigScience Episode\# 5–Workshop on Challenges & Perspectives in Creating Large Language Models,pp\.95–136\.Cited by:[§C\.3](https://arxiv.org/html/2608.28948#A3.S3.p1.1)\.
- Blum and Rivest \(1988\)A\. Blum and R\. RivestTraining a 3\-node neural network is NP\-complete\.Advances in Neural Information Processing Systems \(NeurIPS\)1\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p1.1)\.
- Boix\-Adseràet al\.\(2023\)E\. Boix\-Adserà, E\. Littwin, E\. Abbe, S\. Bengio, and J\. M\. SusskindTransformers learn through gradual rank increase\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1)\.
- Bommasaniet al\.\(2021\)R\. Bommasani, D\. A\. Hudson, E\. Adeli, R\. Altman, S\. Arora, S\. von Arx, M\. S\. Bernstein, J\. Bohg, A\. Bosselut, E\. Brunskill,et al\.On the Opportunities and Risks of Foundation Models\.arXiv prepring arXiv:2108\.07258\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p3.1)\.
- Breaet al\.\(2019\)J\. Brea, B\. Simsek, B\. Illing, and W\. GerstnerWeight\-space symmetry in deep networks gives rise to permutation saddles, connected by equal\-loss valleys across the loss landscape\.arXiv preprint arXiv:1907\.02911\.Cited by:[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Brownet al\.\(2020\)T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.Language models are few\-shot learners\.Advances in Neural Information Processing Systems \(NeurIPS\)33,pp\.1877–1901\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p5.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Burnset al\.\(2023\)C\. Burns, H\. Ye, D\. Klein, and J\. SteinhardtDiscovering Latent Knowledge in Language Models Without Supervision\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Buschjägeret al\.\(2020\)S\. Buschjäger, L\. Pfahler, and K\. MorikGeneralized negative correlation learning for deep ensembling\.arXiv preprint arXiv:2011\.02952\.Cited by:[§5\.4\.1](https://arxiv.org/html/2608.28948#Ch5.S4.SS1.p1.1)\.
- Chenet al\.\(2023a\)A\. S\. Chen, Y\. Lee, A\. Setlur, S\. Levine, and C\. FinnProject and probe: Sample\-efficient domain adaptation by interpolating orthogonal features\.arXiv preprint arXiv:2302\.05441\.Cited by:[§5\.5\.1](https://arxiv.org/html/2608.28948#Ch5.S5.SS1.p1.1)\.
- Chenet al\.\(2023b\)F\. Chen, D\. Kunin, A\. Yamamura, and S\. GanguliStochastic collapse: How gradient noise attracts SGD dynamics towards simpler subnetworks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Chenet al\.\(2024\)L\. Chen, J\. Bruna, and A\. BiettiHow Truncating Weights Improves Reasoning in Language Models\.InHigh\-dimensional Learning Dynamics 2024: The Emergence of Structure and Reasoning,Cited by:[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p2.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, P\. Dhariwal, S\. Gray, R\. Lowe, A\. Radford, J\. Wu, and W\. ZarembaEvaluating Large Language Models Trained on Code\.arXiv:2107\.03374\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2107.03374),2107\.03374Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Chenet al\.\(2026\)Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, Y\. Yue, S\. Song, and G\. HuangDoes reinforcement learning really incentivize reasoning capacity in llms beyond the base model?\.Advances in Neural Information Processing Systems \(NeurIPS\)38,pp\.57654–57689\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Choromanskaet al\.\(2015\)A\. Choromanska, M\. Henaff, M\. Mathieu, G\. B\. Arous, and Y\. LeCunThe loss surfaces of multilayer networks\.InArtificial intelligence and statistics,pp\.192–204\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p1.1)\.
- Chrabaszczet al\.\(2017\)P\. Chrabaszcz, I\. Loshchilov, and F\. HutterA downsampled variant of imagenet as an alternative to the cifar datasets\.arXiv preprint arXiv:1707\.08819\.Cited by:[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx1.p1.1)\.
- Coster and Kauchak \(2011\)W\. Coster and D\. KauchakSimple English Wikipedia: a new text simplification task\.InProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies,pp\.665–669\.Cited by:[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx2.p2.1)\.
- Cybenko \(1989\)G\. CybenkoApproximation by superpositions of a sigmoidal function\.Mathematics of control, signals and systems2\(4\),pp\.303–314\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p1.1)\.
- Daiet al\.\(2024\)D\. Dai, C\. Deng, C\. Zhao, R\. Xu, H\. Gao, D\. Chen, J\. Li, W\. Zeng, X\. Yu, Y\. Wu,et al\.Deepseekmoe: Towards ultimate expert specialization in mixture\-of\-experts language models\.arXiv preprint arXiv:2401\.06066\.Cited by:[§5\.5\.3](https://arxiv.org/html/2608.28948#Ch5.S5.SS3.p1.1)\.
- Davieset al\.\(2022\)X\. Davies, L\. Langosco, and D\. KruegerUnifying Grokking and Double Descent\.InNeurIPS ML Safety Workshop,Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1)\.
- Dinhet al\.\(2017\)L\. Dinh, R\. Pascanu, S\. Bengio, and Y\. BengioSharp minima can generalize for deep nets\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\.1019–1028\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p2.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p4.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1),[§3\.4](https://arxiv.org/html/2608.28948#Ch3.S4.p1.1)\.
- Dittmeret al\.\(2019\)S\. Dittmer, E\. J\. King, and P\. MaassSingular values for ReLU layers\.IEEE Transactions on Neural Networks and Learning Systems31\(9\),pp\.3594–3605\.Cited by:[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1)\.
- Draxleret al\.\(2018\)F\. Draxler, K\. Veschgini, M\. Salmhofer, and F\. HamprechtEssentially no barriers in neural network energy landscape\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\.1309–1318\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p5.1)\.
- Duet al\.\(2018\)S\. S\. Du, W\. Hu, and J\. D\. LeeAlgorithmic regularization in learning deep homogeneous models: Layers are automatically balanced\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.5\.3](https://arxiv.org/html/2608.28948#Ch4.S5.SS3.p2.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p3.1)\.
- D’Amouret al\.\(2022\)A\. D’Amour, K\. Heller, D\. Moldovan, B\. Adlam, B\. Alipanahi, A\. Beutel, C\. Chen, J\. Deaton, J\. Eisenstein, M\. D\. Hoffman,et al\.Underspecification presents challenges for credibility in modern machine learning\.Journal of Machine Learning Research23\(226\),pp\.1–61\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p3.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1)\.
- Entezariet al\.\(2021\)R\. Entezari, H\. Sedghi, O\. Saukh, and B\. NeyshaburThe Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§3\.4](https://arxiv.org/html/2608.28948#Ch3.S4.p1.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Ergen and Pilanci \(2023\)T\. Ergen and M\. PilanciPath regularization: A convexity and sparsity inducing regularization for parallel relu networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Fanet al\.\(2024\)S\. Fan, R\. Pascanu, and M\. JaggiDeep Grokking: Would Deep Neural Networks Generalize Better?\.arXiv preprint arXiv:2405\.19454\.Cited by:[§B\.2\.7](https://arxiv.org/html/2608.28948#A2.S2.SS7.p3.1),[Figure 4\.2](https://arxiv.org/html/2608.28948#Ch4.F2),[Figure 4\.2](https://arxiv.org/html/2608.28948#Ch4.F2.33.3),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p4.1)\.
- Fenget al\.\(2022\)R\. Feng, K\. Zheng, Y\. Huang, D\. Zhao, M\. Jordan, and Z\. ZhaRank diminishing in deep neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Ferbachet al\.\(2023\)D\. Ferbach, B\. Goujaud, G\. Gidel, and A\. DieuleveutProving linear mode connectivity of neural networks via optimal transport\.arXiv preprint arXiv:2310\.19103\.Cited by:[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Fortet al\.\(2020\)S\. Fort, G\. K\. Dziugaite, M\. Paul, S\. Kharaghani, D\. M\. Roy, and S\. GanguliDeep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the neural tangent kernel\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\.5850–5861\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1)\.
- Fortet al\.\(2019\)S\. Fort, H\. Hu, and B\. LakshminarayananDeep ensembles: A loss landscape perspective\.arXiv preprint arXiv:1912\.02757\.Cited by:[§3\.2](https://arxiv.org/html/2608.28948#Ch3.S2.p3.1)\.
- Fort and Jastrzebski \(2019\)S\. Fort and S\. JastrzebskiLarge scale structure of neural network loss landscapes\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.32\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p2.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1)\.
- Frankle and Carbin \(2018\)J\. Frankle and M\. CarbinThe Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p7.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p3.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p1.1),[§4\.9](https://arxiv.org/html/2608.28948#Ch4.S9.p1.1)\.
- Frankleet al\.\(2020\)J\. Frankle, G\. K\. Dziugaite, D\. Roy, and M\. CarbinLinear mode connectivity and the lottery ticket hypothesis\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\.3259–3269\.Cited by:[§A\.1](https://arxiv.org/html/2608.28948#A1.S1.p1.1),[§B\.2\.1](https://arxiv.org/html/2608.28948#A2.S2.SS1.p1.1),[§B\.2\.10](https://arxiv.org/html/2608.28948#A2.S2.SS10.p3.1),[§B\.2\.9](https://arxiv.org/html/2608.28948#A2.S2.SS9.p2.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p4.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p7.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p8.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1),[§3\.2](https://arxiv.org/html/2608.28948#Ch3.S2.p1.1),[§3\.4](https://arxiv.org/html/2608.28948#Ch3.S4.p1.1),[Figure 4\.4](https://arxiv.org/html/2608.28948#Ch4.F4),[Figure 4\.4](https://arxiv.org/html/2608.28948#Ch4.F4.19.2),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p7.1),[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p4.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p3.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p1.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p3.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p5.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx1.p2.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1),[§4\.9](https://arxiv.org/html/2608.28948#Ch4.S9.p1.1)\.
- Frankle \(2020\)J\. FrankleRevisiting “Qualitatively Characterizing Neural Network Optimization Problems”\.arXiv preprint arXiv:2012\.06898\.Cited by:[§3\.3](https://arxiv.org/html/2608.28948#Ch3.S3.p1.1),[§3\.5](https://arxiv.org/html/2608.28948#Ch3.S5.p2.1)\.
- Freiet al\.\(2022\)S\. Frei, G\. Vardi, P\. Bartlett, N\. Srebro, and W\. HuImplicit Bias in Leaky ReLU Networks Trained on High\-Dimensional Data\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Freundet al\.\(1999\)Y\. Freund, R\. Schapire, and N\. AbeA short introduction to boosting\.Journal\-Japanese Society For Artificial Intelligence14\(771\-780\),pp\.1612\.Cited by:[§5\.4\.2](https://arxiv.org/html/2608.28948#Ch5.S4.SS2.p1.1)\.
- Galantiet al\.\(2022\)T\. Galanti, Z\. S\. Siegel, A\. Gupte, and T\. PoggioSGD and weight decay provably induce a low\-rank bias in neural networks\.arXiv preprint arXiv:2206\.05794\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1)\.
- Garipovet al\.\(2018\)T\. Garipov, P\. Izmailov, D\. Podoprikhin, D\. P\. Vetrov, and A\. G\. WilsonLoss surfaces, mode connectivity, and fast ensembling of DNNs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.31\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p5.1)\.
- Ghoshet al\.\(2025\)A\. Ghosh, S\. M\. Kwon, R\. Wang, S\. Ravishankar, and Q\. QuLearning Dynamics of Deep Linear Networks Beyond the Edge of Stability\.arXiv preprint arXiv:2502\.20531\.Cited by:[Figure B\.6](https://arxiv.org/html/2608.28948#A2.F6),[Figure B\.6](https://arxiv.org/html/2608.28948#A2.F6.10),[§B\.4\.2](https://arxiv.org/html/2608.28948#A2.S4.SS2.p1.1)\.
- Gontijo\-Lopeset al\.\(2021\)R\. Gontijo\-Lopes, Y\. Dauphin, and E\. D\. CubukNo one representation to rule them all: Overlapping features of training methods\.arXiv preprint arXiv:2110\.12899\.Cited by:[§5\.5\.2](https://arxiv.org/html/2608.28948#Ch5.S5.SS2.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.5\.1](https://arxiv.org/html/2608.28948#Ch4.S5.SS1.p3.1)\.
- Gromov \(2023\)A\. GromovGrokking modular arithmetic\.arXiv preprint arXiv:2301\.02679\.Cited by:[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1)\.
- Gunasekaret al\.\(2017\)S\. Gunasekar, B\. E\. Woodworth, S\. Bhojanapalli, B\. Neyshabur, and N\. SrebroImplicit regularization in matrix factorization\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Guoet al\.\(2025\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.Deepseek\-r1: Incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Gur\-Ariet al\.\(2018\)G\. Gur\-Ari, D\. A\. Roberts, and E\. DyerGradient descent happens in a tiny subspace\.arXiv preprint arXiv:1812\.04754\.Cited by:[§3\.2](https://arxiv.org/html/2608.28948#Ch3.S2.p3.1)\.
- Hanson and Pratt \(1988\)S\. Hanson and L\. PrattComparing biases for minimal network construction with back\-propagation\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1)\.
- Harriset al\.\(2020\)C\. R\. Harris, K\. J\. Millman, S\. J\. van der Walt, R\. Gommers, P\. Virtanen, D\. Cournapeau, E\. Wieser, J\. Taylor, S\. Berg, N\. J\. Smith, R\. Kern, M\. Picus, S\. Hoyer, M\. H\. van Kerkwijk, M\. Brett, A\. Haldane, J\. F\. del Río, M\. Wiebe, P\. Peterson, P\. Gérard\-Marchant, K\. Sheppard, T\. Reddy, W\. Weckesser, H\. Abbasi, C\. Gohlke, and T\. E\. OliphantArray programming with NumPy\.Nature585\(7825\),pp\.357–362\.Cited by:[§B\.6](https://arxiv.org/html/2608.28948#A2.S6.p1.1)\.
- Heet al\.\(2016\)K\. He, X\. Zhang, S\. Ren, and J\. SunDeep residual learning for image recognition\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§A\.1](https://arxiv.org/html/2608.28948#A1.S1.p1.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx1.p1.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, K\. Zhao, S\. Basart, J\. Steinhardt, and D\. SongNatural adversarial examples\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Hestnesset al\.\(2017\)J\. Hestness, S\. Narang, N\. Ardalani, G\. Diamos, H\. Jun, H\. Kianinejad, M\. M\. A\. Patwary, Y\. Yang, and Y\. ZhouDeep learning scaling is predictable, empirically\.arXiv preprint arXiv:1712\.00409\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Hintonet al\.\(2012\)G\. Hinton, L\. Deng, D\. Yu, G\. E\. Dahl, A\. Mohamed, N\. Jaitly, A\. Senior, V\. Vanhoucke, P\. Nguyen, T\. N\. Sainath,et al\.Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups\.IEEE Signal processing magazine29\(6\),pp\.82–97\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Hoet al\.\(2020\)J\. Ho, A\. Jain, and P\. AbbeelDenoising diffusion probabilistic models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§B\.2\.2](https://arxiv.org/html/2608.28948#A2.S2.SS2.p1.1)\.
- Hochreiter and Schmidhuber \(1997\)S\. Hochreiter and J\. SchmidhuberLong short\-term memory\.Neural Computation9\(8\),pp\.1735–1780\.Cited by:[§B\.2\.3](https://arxiv.org/html/2608.28948#A2.S2.SS3.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[3rd item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i3.p1.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p3.1)\.
- Huet al\.\(2024\)S\. Hu, Y\. Tu, X\. Han, C\. He, G\. Cui, X\. Long, Z\. Zheng, Y\. Fang, Y\. Huang, W\. Zhao,et al\.Minicpm: Unveiling the potential of small language models with scalable training strategies\.arXiv preprint arXiv:2404\.06395\.Cited by:[§C\.3](https://arxiv.org/html/2608.28948#A3.S3.p1.1)\.
- Huet al\.\(2018\)W\. Hu, G\. Niu, I\. Sato, and M\. SugiyamaDoes distributionally robust supervised learning give robust classifiers?\.InInternational Conference on Machine Learning,pp\.2029–2037\.Cited by:[§5\.5\.1](https://arxiv.org/html/2608.28948#Ch5.S5.SS1.p1.1)\.
- Huhet al\.\(2021\)M\. Huh, H\. Mobahi, R\. Zhang, B\. Cheung, P\. Agrawal, and P\. IsolaThe low\-rank simplicity bias in deep networks\.arXiv preprint arXiv:2103\.10427\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p1.1)\.
- Huhet al\.\(2022\)M\. Huh, H\. Mobahi, R\. Zhang, B\. Cheung, P\. Agrawal, and P\. IsolaThe Low\-Rank Simplicity Bias in Deep Networks\.Transactions on Machine Learning Research\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1),[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx2.p2.1)\.
- Humayunet al\.\(2024\)A\. I\. Humayun, R\. Balestriero, and R\. BaraniukDeep Networks Always Grok and Here is Why\.arXiv preprint arXiv:2402\.15555\.Cited by:[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1)\.
- Hunter \(2007\)J\. D\. HunterMatplotlib: A 2D graphics environment\.Computing in Science & Engineering9\(3\),pp\.90–95\.Cited by:[§B\.6](https://arxiv.org/html/2608.28948#A2.S6.p1.1)\.
- Ilharcoet al\.\(2022\)G\. Ilharco, M\. T\. Ribeiro, M\. Wortsman, L\. Schmidt, H\. Hajishirzi, and A\. FarhadiEditing models with task arithmetic\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p10.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx2.p2.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1),[§4\.9](https://arxiv.org/html/2608.28948#Ch4.S9.p1.1)\.
- Ilyaset al\.\(2019\)A\. Ilyas, S\. Santurkar, D\. Tsipras, L\. Engstrom, B\. Tran, and A\. MadryAdversarial examples are not bugs, they are features\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Ioffe and Szegedy \(2015\)S\. Ioffe and C\. SzegedyBatch normalization: Accelerating deep network training by reducing internal covariate shift\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§B\.2\.1](https://arxiv.org/html/2608.28948#A2.S2.SS1.p2.1)\.
- Itoet al\.\(2024\)A\. Ito, M\. Yamada, and A\. KumagaiAnalysis of Linear Mode Connectivity via Permutation\-Based Weight Matching\.arXiv preprint arXiv:2402\.04051\.Cited by:[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx1.p3.1)\.
- Izmailovet al\.\(2022\)P\. Izmailov, P\. Kirichenko, N\. Gruver, and A\. G\. WilsonOn feature learning in the presence of spurious correlations\.Advances in Neural Information Processing Systems \(NeurIPS\)35,pp\.38516–38532\.Cited by:[§5\.5\.2](https://arxiv.org/html/2608.28948#Ch5.S5.SS2.p1.1)\.
- Jacotet al\.\(2018\)A\. Jacot, F\. Gabriel, and C\. HonglerNeural tangent kernel: Convergence and generalization in neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1)\.
- Jeffareset al\.\(2023\)A\. Jeffares, T\. Liu, J\. Crabbé, and M\. van der SchaarJoint training of deep ensembles fails due to learner collusion\.Advances in Neural Information Processing Systems36,pp\.13559–13589\.Cited by:[§5\.4\.1](https://arxiv.org/html/2608.28948#Ch5.S4.SS1.p6.1)\.
- Ji and Telgarsky \(2019\)Z\. Ji and M\. TelgarskyGradient descent aligns the layers of deep linear networks\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.2\.1](https://arxiv.org/html/2608.28948#Ch4.S2.SS1.p1.1)\.
- Jianget al\.\(2019\)Y\. Jiang, B\. Neyshabur, H\. Mobahi, D\. Krishnan, and S\. BengioFantastic generalization measures and where to find them\.arXiv preprint arXiv:1912\.02178\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p2.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1),[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Jordanet al\.\(2024\)K\. Jordan, Y\. Jin, V\. Boza, J\. You, F\. Cesista, L\. Newhouse, and J\. BernsteinMuon: An optimizer for hidden layers in neural networks\.External Links:[Link](https://kellerjordan.github.io/posts/muon/)Cited by:[§5\.2\.2](https://arxiv.org/html/2608.28948#Ch5.S2.SS2.p5.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Jordanet al\.\(2022\)K\. Jordan, H\. Sedghi, O\. Saukh, R\. Entezari, and B\. NeyshaburREPAIR: REnormalizing Permuted Activations for Interpolation Repair\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Joshiet al\.\(2025\)G\. Joshi, P\. N\. Shah, and R\. VermaMitigating Simplicity Bias in Neural Networks: A Feature Sieve Modification, Regularization, and Self\-Supervised Augmentation Approach\.InWorkshop on Spurious Correlation and Shortcut Learning: Foundations and Solutions,Cited by:[§5\.5\.2](https://arxiv.org/html/2608.28948#Ch5.S5.SS2.p1.1)\.
- Jumperet al\.\(2021\)J\. Jumper, R\. Evans, A\. Pritzel, T\. Green, M\. Figurnov, O\. Ronneberger, K\. Tunyasuvunakool, R\. Bates, A\. Žídek, A\. Potapenko,et al\.Highly accurate protein structure prediction with AlphaFold\.nature596\(7873\),pp\.583–589\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Junejaet al\.\(2022\)J\. Juneja, R\. Bansal, K\. Cho, J\. Sedoc, and N\. SaphraLinear Connectivity Reveals Generalization Strategies\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p9.1)\.
- Kaplanet al\.\(2020\)J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. AmodeiScaling laws for neural language models\.arXiv preprint arXiv:2001\.08361\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p5.1),[§5\.3](https://arxiv.org/html/2608.28948#Ch5.S3.p2.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Karraset al\.\(2023\)T\. Karras, M\. Aittala, J\. Lehtinen, J\. Hellsten, T\. Aila, and S\. LaineAnalyzing and improving the training dynamics of diffusion models\.arXiv preprint arXiv:2312\.02696\.Cited by:[§4\.5\.1](https://arxiv.org/html/2608.28948#Ch4.S5.SS1.p3.1)\.
- Kawaguchi \(2016\)K\. KawaguchiDeep learning without poor local minima\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.29,pp\.586–594\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p1.1)\.
- Keskaret al\.\(2016\)N\. S\. Keskar, D\. Mudigere, J\. Nocedal, M\. Smelyanskiy, and P\. T\. P\. TangOn Large\-Batch Training for Deep Learning: Generalization Gap and Sharp Minima\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p1.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p2.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p4.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1)\.
- Kimet al\.\(2026\)K\. Kim, S\. Kotha, P\. Liang, and T\. HashimotoPre\-training under infinite compute\.InInternational Conference on Learning Representations,pp\.74596–74636\.Cited by:[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1)\.
- Kirichenkoet al\.\(2022\)P\. Kirichenko, P\. Izmailov, and A\. G\. WilsonLast layer re\-training is sufficient for robustness to spurious correlations\.arXiv preprint arXiv:2204\.02937\.Cited by:[§5\.5\.1](https://arxiv.org/html/2608.28948#Ch5.S5.SS1.p1.1)\.
- KKR Global Macro & Asset Allocation \(2025\)KKR Global Macro & Asset AllocationBeyond the Bubble: Why We Think AI Infrastructure Will Compound Long after the Hype\.Note:KKR InsightsExternal Links:[Link](https://www.kkr.com/insights/ai-infrastructure)Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1)\.
- Kondratyuket al\.\(2020\)D\. Kondratyuk, M\. Tan, M\. Brown, and B\. GongWhen ensembling smaller models is more efficient than single large models\.arXiv preprint arXiv:2005\.00570\.Cited by:[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1)\.
- Kornblithet al\.\(2019\)S\. Kornblith, M\. Norouzi, H\. Lee, and G\. HintonSimilarity of neural network representations revisited\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\.3519–3529\.Cited by:[Figure 5\.7](https://arxiv.org/html/2608.28948#Ch5.F7),[Figure 5\.7](https://arxiv.org/html/2608.28948#Ch5.F7.10),[§5\.3\.1](https://arxiv.org/html/2608.28948#Ch5.S3.SS1.p3.1),[§5\.3\.1](https://arxiv.org/html/2608.28948#Ch5.S3.SS1.p4.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.p1.1)\.
- Krizhevskyet al\.\(2012\)A\. Krizhevsky, I\. Sutskever, and G\. E\. HintonImagenet classification with deep convolutional neural networks\.Advances in Neural Information Processing Systems \(NeurIPS\)25\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Krizhevsky \(2009\)A\. KrizhevskyLearning multiple layers of features from tiny images\.Master’s Thesis,University of Toronto\.Cited by:[§A\.1](https://arxiv.org/html/2608.28948#A1.S1.p1.1),[§B\.2\.1](https://arxiv.org/html/2608.28948#A2.S2.SS1.p1.1),[§B\.2\.8](https://arxiv.org/html/2608.28948#A2.S2.SS8.p1.1),[1st item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i1.p1.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx1.p1.1)\.
- Krogh and Hertz \(1991\)A\. Krogh and J\. HertzA simple weight decay can improve generalization\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1)\.
- Kuditipudiet al\.\(2019\)R\. Kuditipudi, X\. Wang, H\. Lee, Y\. Zhang, Z\. Li, W\. Hu, R\. Ge, and S\. AroraExplaining landscape connectivity of low\-cost solutions for multilayer nets\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p5.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Kumaret al\.\(2023\)T\. Kumar, B\. Bordelon, S\. J\. Gershman, and C\. PehlevanGrokking as the transition from lazy to rich training dynamics\.arXiv preprint arXiv:2310\.06110\.Cited by:[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1)\.
- Kusanoet al\.\(2025\)K\. D\. Kusano, J\. M\. Scanlon, Y\. Chen, T\. L\. McMurry, T\. Gode, and T\. VictorComparison of Waymo Rider\-Only Crash Rates by Crash Type to Human Benchmarks at 56\.7 Million Miles\.Technical reportarXiv\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.01515),2505\.01515Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Le and Jegelka \(2021\)T\. Le and S\. JegelkaTraining invariances and the low\-rank phenomenon: beyond linear networks\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- LeCunet al\.\(1998\)Y\. LeCun, L\. Bottou, Y\. Bengio, and P\. HaffnerGradient\-based learning applied to document recognition\.Proceedings of the IEEE86\(11\),pp\.2278–2324\.Cited by:[§A\.1](https://arxiv.org/html/2608.28948#A1.S1.p1.1)\.
- LeCun \(1998\)Y\. LeCunThe MNIST database of handwritten digits\.Note:[http://yann\.lecun\.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/)Cited by:[§B\.2\.2](https://arxiv.org/html/2608.28948#A2.S2.SS2.p1.1),[§B\.2\.7](https://arxiv.org/html/2608.28948#A2.S2.SS7.p3.1),[2nd item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i2.p1.1)\.
- Leeet al\.\(2022\)Y\. Lee, H\. Yao, and C\. FinnDiversify and disambiguate: Learning from underspecified data\.arXiv preprint arXiv:2202\.03418\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p3.1),[§5\.5\.1](https://arxiv.org/html/2608.28948#Ch5.S5.SS1.p1.1)\.
- Lhoestet al\.\(2021\)Q\. Lhoest, A\. V\. del Moral, Y\. Jernite, A\. Thakur, P\. von Platen, S\. Patil, J\. Chaumond, M\. Drame, J\. Plu, L\. Tunstall,et al\.Datasets: A community library for natural language processing\.arXiv preprint arXiv:2109\.02846\.Cited by:[§B\.6](https://arxiv.org/html/2608.28948#A2.S6.p1.1)\.
- Liet al\.\(2018\)H\. Li, Z\. Xu, G\. Taylor, C\. Studer, and T\. GoldsteinVisualizing the loss landscape of neural nets\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§A\.2](https://arxiv.org/html/2608.28948#A1.S2.p1.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p2.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p4.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1),[§3\.5](https://arxiv.org/html/2608.28948#Ch3.S5.p2.1)\.
- Liet al\.\(2022\)M\. Li, S\. Gururangan, T\. Dettmers, M\. Lewis, T\. Althoff, N\. A\. Smith, and L\. ZettlemoyerBranch\-Train\-Merge: Embarrassingly Parallel Training of Expert Language Models\.arXiv preprint arXiv:2208\.03306\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p10.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1)\.
- Liet al\.\(2020\)Z\. Li, Y\. Luo, and K\. LyuTowards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low\-Rank Learning\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Linet al\.\(2017\)T\. Lin, P\. Goyal, R\. Girshick, K\. He, and P\. DollárFocal loss for dense object detection\.InProceedings of the IEEE international conference on computer vision,pp\.2980–2988\.Cited by:[§5\.4\.2](https://arxiv.org/html/2608.28948#Ch5.S4.SS2.p2.1)\.
- Liuet al\.\(2021\)E\. Z\. Liu, B\. Haghgoo, A\. S\. Chen, A\. Raghunathan, P\. W\. Koh, S\. Sagawa, P\. Liang, and C\. FinnJust train twice: Improving group robustness without training group information\.InInternational Conference on Machine Learning,pp\.6781–6792\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p3.1),[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1)\.
- Liu and Yao \(1999\)Y\. Liu and X\. YaoEnsemble learning via negative correlation\.Neural networks12\(10\),pp\.1399–1404\.Cited by:[§5\.4\.1](https://arxiv.org/html/2608.28948#Ch5.S4.SS1.p1.1)\.
- Liuet al\.\(2022\)Z\. Liu, O\. Kitouni, N\. S\. Nolte, E\. Michaud, M\. Tegmark, and M\. WilliamsTowards understanding grokking: An effective theory of representation learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1)\.
- Liuet al\.\(2023\)Z\. Liu, E\. J\. Michaud, and M\. TegmarkOmnigrok: Grokking beyond algorithmic data\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§B\.2\.7](https://arxiv.org/html/2608.28948#A2.S2.SS7.p3.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p2.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1)\.
- Lobachevaet al\.\(2020\)E\. Lobacheva, N\. Chirkova, M\. Kodryan, and D\. P\. VetrovOn power laws in deep ensembles\.Advances In Neural Information Processing Systems33,pp\.2375–2385\.Cited by:[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1)\.
- Loshchilov and Hutter \(2017\)I\. Loshchilov and F\. HutterFixing weight decay regularization in Adam\.arXiv preprint arXiv:1711\.05101\.Cited by:[§B\.2\.2](https://arxiv.org/html/2608.28948#A2.S2.SS2.p1.1),[§B\.2\.3](https://arxiv.org/html/2608.28948#A2.S2.SS3.p1.1),[§B\.2\.4](https://arxiv.org/html/2608.28948#A2.S2.SS4.p1.1),[Figure 4\.6](https://arxiv.org/html/2608.28948#Ch4.F6),[Figure 4\.6](https://arxiv.org/html/2608.28948#Ch4.F6.14)\.
- Lubanaet al\.\(2023\)E\. S\. Lubana, E\. J\. Bigelow, R\. P\. Dick, D\. Krueger, and H\. TanakaMechanistic mode connectivity\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p4.1)\.
- Lyuet al\.\(2023\)K\. Lyu, J\. Jin, Z\. Li, S\. S\. Du, J\. D\. Lee, and W\. HuDichotomy of early and late phase implicit biases can provably induce grokking\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p2.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1)\.
- Mandalet al\.\(2026\)B\. Mandal, S\. Berman, A\. Vegesna, and S\. Dahalq0: Primitives for Hyper\-Epoch Pretraining\.arXiv preprint arXiv:2606\.03938\.Cited by:[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1)\.
- Marquardt and Snee \(1975\)D\. W\. Marquardt and R\. D\. SneeRidge regression in practice\.The American Statistician29\(1\),pp\.3–20\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1)\.
- Martin and Mahoney \(2020\)C\. H\. Martin and M\. W\. MahoneyHeavy\-tailed universality predicts trends in test accuracies for very large pre\-trained deep neural networks\.InProceedings of the SIAM International Conference on Data Mining \(ICDM\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1)\.
- Martin and Mahoney \(2021\)C\. H\. Martin and M\. W\. MahoneyImplicit self\-regularization in deep neural networks: Evidence from random matrix theory and implications for learning\.Journal of Machine Learning Research22\(165\),pp\.1–73\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1)\.
- Mazeikaet al\.\(2024\)M\. Mazeika, L\. Phan, X\. Yin, A\. Zou, Z\. Wang, N\. Mu, E\. Sakhaee, N\. Li, S\. Basart, B\. Li,et al\.HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Merchantet al\.\(2023\)A\. Merchant, S\. Batzner, S\. S\. Schoenholz, M\. Aykol, G\. Cheon, and E\. D\. CubukScaling deep learning for materials discovery\.Nature624\(7990\),pp\.80–85\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Merityet al\.\(2016\)S\. Merity, C\. Xiong, J\. Bradbury, and R\. SocherPointer Sentinel Mixture Models\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§B\.2\.4](https://arxiv.org/html/2608.28948#A2.S2.SS4.p1.1),[§B\.6](https://arxiv.org/html/2608.28948#A2.S6.p1.1),[4th item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i4.p1.1)\.
- Merrillet al\.\(2023\)W\. Merrill, N\. Tsilivis, and A\. ShuklaA Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks\.InICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models,Cited by:[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1)\.
- Milanesiet al\.\(2021\)P\. Milanesi, H\. Kadri, S\. Ayache, and T\. ArtièresImplicit regularization in deep tensor factorization\.InProceedings of the International Joint Conference on Neural Networks \(IJCNN\),Cited by:[§B\.1](https://arxiv.org/html/2608.28948#A2.S1.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p4.1),[§4\.2\.1](https://arxiv.org/html/2608.28948#Ch4.S2.SS1.p1.1),[§4\.3](https://arxiv.org/html/2608.28948#Ch4.S3.p1.1),[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1),[§4\.5](https://arxiv.org/html/2608.28948#Ch4.S5.p1.1)\.
- Mohanet al\.\(2019\)S\. Mohan, Z\. Kadkhodaie, E\. P\. Simoncelli, and C\. Fernandez\-GrandaRobust and interpretable blind image denoising via bias\-free convolutional neural networks\.arXiv preprint arXiv:1906\.05478\.Cited by:[§4\.5\.1](https://arxiv.org/html/2608.28948#Ch4.S5.SS1.p3.1)\.
- Morwaniet al\.\(2023\)D\. Morwani, J\. Batra, P\. Jain, and P\. NetrapalliSimplicity bias in 1\-hidden layer neural networks\.Advances in Neural Information Processing Systems \(NeurIPS\)36,pp\.8048–8075\.Cited by:[§5\.5\.4](https://arxiv.org/html/2608.28948#Ch5.S5.SS4.p1.1)\.
- Mulayoff and Michaeli \(2020\)R\. Mulayoff and T\. MichaeliUnique properties of flat minima in deep networks\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§4\.5\.3](https://arxiv.org/html/2608.28948#Ch4.S5.SS3.p2.1)\.
- Nagarajan and Kolter \(2019\)V\. Nagarajan and J\. Z\. KolterUniform convergence may be unable to explain generalization in deep learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§A\.1](https://arxiv.org/html/2608.28948#A1.S1.p1.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p6.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1),[§3\.4](https://arxiv.org/html/2608.28948#Ch3.S4.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p7.1),[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Nakkiranet al\.\(2021\)P\. Nakkiran, G\. Kaplun, Y\. Bansal, T\. Yang, B\. Barak, and I\. SutskeverDeep double descent: Where bigger models and more data hurt\.Journal of Statistical Mechanics: Theory and Experiment2021\(12\)\.Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Nandaet al\.\(2023\)N\. Nanda, L\. Chan, T\. Lieberum, J\. Smith, and J\. SteinhardtProgress measures for grokking via mechanistic interpretability\.arXiv preprint arXiv:2301\.05217\.Cited by:[Figure B\.4](https://arxiv.org/html/2608.28948#A2.F4),[Figure B\.4](https://arxiv.org/html/2608.28948#A2.F4.32.3),[§B\.2\.7](https://arxiv.org/html/2608.28948#A2.S2.SS7.p1.1),[§B\.2\.7](https://arxiv.org/html/2608.28948#A2.S2.SS7.p2.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1),[Figure 4\.2](https://arxiv.org/html/2608.28948#Ch4.F2),[Figure 4\.2](https://arxiv.org/html/2608.28948#Ch4.F2.33.2),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p2.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1.10),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p4.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p4.1)\.
- Neyshaburet al\.\(2020\)B\. Neyshabur, H\. Sedghi, and C\. ZhangWhat is being transferred in transfer learning?\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§B\.2\.10](https://arxiv.org/html/2608.28948#A2.S2.SS10.p2.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p9.1),[§3\.1](https://arxiv.org/html/2608.28948#Ch3.S1.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p7.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx1.p2.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx2.p2.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1),[§4\.9](https://arxiv.org/html/2608.28948#Ch4.S9.p1.1)\.
- Neyshaburet al\.\(2014\)B\. Neyshabur, R\. Tomioka, and N\. SrebroIn search of the real inductive bias: On the role of implicit regularization in deep learning\.arXiv preprint arXiv:1412\.6614\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1)\.
- Niuet al\.\(2022\)H\. Niu, H\. Li, F\. Zhao, and B\. LiRoadblocks for temporarily disabling shortcuts and learning new knowledge\.Advances in Neural Information Processing Systems \(NeurIPS\)35,pp\.29064–29075\.Cited by:[§5\.5\.2](https://arxiv.org/html/2608.28948#Ch5.S5.SS2.p1.1)\.
- Ongie and Willett \(2022\)G\. Ongie and R\. WillettThe role of linear layers in nonlinear interpolating networks\.arXiv preprint arXiv:2202\.00856\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1)\.
- OpenAI \(2022\)OpenAIChatGPT\.Note:Large language model; launched November 30, 2022External Links:[Link](https://chatgpt.com/)Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- OpenAI \(2026\)OpenAITen Advances in Mathematics and Theoretical Computer Science\.Technical reportOpenAI\.External Links:[Link](https://cdn.openai.com/pdf/ten-proofs-oai.pdf)Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Ouyanget al\.\(2022\)L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.Training language models to follow instructions with human feedback\.Advances in Neural Information Processing Systems \(NeurIPS\)35,pp\.27730–27744\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Pagliardiniet al\.\(2022\)M\. Pagliardini, M\. Jaggi, F\. Fleuret, and S\. P\. KarimireddyAgree to disagree: Diversity through disagreement for better transferability\.arXiv preprint arXiv:2202\.04414\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p3.1),[§5\.5\.1](https://arxiv.org/html/2608.28948#Ch5.S5.SS1.p1.1)\.
- Panayotovet al\.\(2015\)V\. Panayotov, G\. Chen, D\. Povey, and S\. KhudanpurLibrispeech: An ASR corpus based on public domain audio books\.InProceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\.5206–5210\.Cited by:[§B\.2\.3](https://arxiv.org/html/2608.28948#A2.S2.SS3.p1.1),[3rd item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i3.p1.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p3.1)\.
- Papyanet al\.\(2020\)V\. Papyan, X\. Han, and D\. L\. DonohoPrevalence of neural collapse during the terminal phase of deep learning training\.Proceedings of the National Academy of Sciences117\(40\),pp\.24652–24663\.Cited by:[§3\.5](https://arxiv.org/html/2608.28948#Ch3.S5.p3.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx3.p1.1)\.
- Parhi and Nowak \(2023\)R\. Parhi and R\. D\. NowakDeep learning meets sparse regularization: A signal processing perspective\.IEEE Signal Processing Magazine40\(6\),pp\.63–74\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Parket al\.\(2024\)K\. Park, Y\. J\. Choe, and V\. VeitchThe Linear Representation Hypothesis and the Geometry of Large Language Models\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Paszkeet al\.\(2019\)A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga,et al\.PyTorch: An imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§B\.6](https://arxiv.org/html/2608.28948#A2.S6.p1.1)\.
- Paulet al\.\(2022\)M\. Paul, F\. Chen, B\. W\. Larsen, J\. Frankle, S\. Ganguli, and G\. K\. DziugaiteUnmasking the Lottery Ticket Hypothesis: What’s Encoded in a Winning Ticket’s Mask?\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Penningtonet al\.\(2018\)J\. Pennington, S\. Schoenholz, and S\. GanguliThe emergence of spectral universality in deep networks\.InProceedings of the International Conference on Artificial Intelligence and Statistics \(AISTATS\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Pezeshkiet al\.\(2021\)M\. Pezeshki, O\. Kaba, Y\. Bengio, A\. C\. Courville, D\. Precup, and G\. LajoieGradient starvation: A learning proclivity in neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.34\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p1.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p2.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p4.1)\.
- Poweret al\.\(2022\)A\. Power, Y\. Burda, H\. Edwards, I\. Babuschkin, and V\. MisraGrokking: Generalization beyond overfitting on small algorithmic datasets\.arXiv preprint arXiv:2201\.02177\.Cited by:[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p2.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1)\.
- Praggastiset al\.\(2022\)B\. Praggastis, D\. Brown, C\. O\. Marrero, E\. Purvine, M\. Shapiro, and B\. WangThe SVD of convolutional weights: a CNN interpretability framework\.arXiv preprint arXiv:2208\.06894\.Cited by:[§B\.2\.1](https://arxiv.org/html/2608.28948#A2.S2.SS1.p2.1),[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p4.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p4.1)\.
- Prietoet al\.\(2025\)L\. Prieto, M\. Barsbey, P\. A\. Mediano, and T\. BirdalGrokking at the edge of numerical stability\.arXiv preprint arXiv:2501\.04697\.Cited by:[§5\.2\.2](https://arxiv.org/html/2608.28948#Ch5.S2.SS2.p5.1)\.
- Qu and Horvath \(2024\)X\. Qu and S\. HorvathRethink Model Re\-Basin and the Linear Mode Connectivity\.arXiv preprint arXiv:2402\.05966\.Cited by:[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Radfordet al\.\(2018\)A\. Radford, K\. Narasimhan, T\. Salimans, I\. Sutskever,et al\.Improving language understanding by generative pre\-training\.San Francisco, CA, USA\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Radfordet al\.\(2019\)A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, I\. Sutskever,et al\.Language models are unsupervised multitask learners\.OpenAI blog1\(8\),pp\.9\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Raffelet al\.\(2020\)C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. LiuExploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of machine learning research21\(140\),pp\.1–67\.Cited by:[§4\.5\.1](https://arxiv.org/html/2608.28948#Ch4.S5.SS1.p3.1)\.
- Rahamanet al\.\(2019\)N\. Rahaman, A\. Baratin, D\. Arpit, F\. Draxler, M\. Lin, F\. Hamprecht, Y\. Bengio, and A\. CourvilleOn the spectral bias of neural networks\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p1.1)\.
- Ramachandranet al\.\(2017\)P\. Ramachandran, B\. Zoph, and Q\. V\. LeSearching for activation functions\.arXiv preprint arXiv:1710\.05941\.Cited by:[§B\.2\.2](https://arxiv.org/html/2608.28948#A2.S2.SS2.p1.1)\.
- Rameet al\.\(2022\)A\. Rame, M\. Kirchmeyer, T\. Rahier, A\. Rakotomamonjy, P\. Gallinari, and M\. CordDiverse weight averaging for out\-of\-distribution generalization\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p10.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx2.p2.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Rameshet al\.\(2022\)A\. Ramesh, P\. Dhariwal, A\. Nichol, C\. Chu, and M\. ChenHierarchical Text\-Conditional Image Generation with CLIP Latents\.arXiv preprint arXiv:2204\.06125\.Cited by:[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1)\.
- Razin and Cohen \(2020\)N\. Razin and N\. CohenImplicit regularization in deep learning may not be explainable by norms\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1)\.
- Ronnebergeret al\.\(2015\)O\. Ronneberger, P\. Fischer, and T\. BroxU\-net: Convolutional networks for biomedical image segmentation\.InProceeding of the International Conference on Medical Image Computing and Computer\-Assisted Intervention \(MICCAI\),Cited by:[§B\.2\.2](https://arxiv.org/html/2608.28948#A2.S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[2nd item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i2.p1.1)\.
- Roy and Vetterli \(2007\)O\. Roy and M\. VetterliThe effective rank: A measure of effective dimensionality\.InProceedings of the European Signal Processing Conference \(EUSIPCO\),Cited by:[§4\.3](https://arxiv.org/html/2608.28948#Ch4.S3.p1.1)\.
- Rumbelow and Watkins \(2023\)J\. Rumbelow and M\. WatkinsSolidGoldMagikarp \(plus, prompt generation\)\.Note:[https://www\.lesswrong\.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp\-plus\-prompt\-generation](https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation)LessWrongCited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p1.1)\.
- Russakovskyet al\.\(2015\)O\. Russakovsky, J\. Deng, H\. Su, J\. Krause, S\. Satheesh, S\. Ma, Z\. Huang, A\. Karpathy, A\. Khosla, M\. Bernstein,et al\.Imagenet large scale visual recognition challenge\.International Journal of Computer Vision115\(3\),pp\.211–252\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1)\.
- Sadrtdinovet al\.\(2023\)I\. Sadrtdinov, D\. Pozdeev, D\. P\. Vetrov, and E\. LobachevaTo Stay or Not to Stay in the Pre\-train Basin: Insights on Ensembling in Transfer Learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Sagawaet al\.\(2019\)S\. Sagawa, P\. W\. Koh, T\. B\. Hashimoto, and P\. LiangDistributionally robust neural networks for group shifts: On the importance of regularization for worst\-case generalization\.arXiv preprint arXiv:1911\.08731\.Cited by:[§5\.5\.1](https://arxiv.org/html/2608.28948#Ch5.S5.SS1.p1.1)\.
- Sahaet al\.\(2021\)G\. Saha, I\. Garg, and K\. RoyGradient Projection Memory for Continual Learning\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§5\.2\.1](https://arxiv.org/html/2608.28948#Ch5.S2.SS1.p7.1),[§5\.5\.4](https://arxiv.org/html/2608.28948#Ch5.S5.SS4.p1.1)\.
- Saunshiet al\.\(2025\)N\. Saunshi, N\. Dikkala, Z\. Li, S\. Kumar, and S\. J ReddiReasoning with latent thoughts: On the power of looped transformers\.InInternational Conference on Learning Representations,Vol\.2025,pp\.14855–14881\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Saxeet al\.\(2014\)A\. Saxe, J\. McClelland, and S\. GanguliExact solutions to the nonlinear dynamics of learning in deep linear neural networks\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p4.1),[§4\.2\.1](https://arxiv.org/html/2608.28948#Ch4.S2.SS1.p1.1),[§4\.3](https://arxiv.org/html/2608.28948#Ch4.S3.p1.1),[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1),[§4\.5\.3](https://arxiv.org/html/2608.28948#Ch4.S5.SS3.p2.1),[§4\.5](https://arxiv.org/html/2608.28948#Ch4.S5.p1.1)\.
- Schotthöferet al\.\(2022\)S\. Schotthöfer, E\. Zangrando, J\. Kusch, G\. Ceruti, and F\. TudiscoLow\-rank lottery tickets: Finding efficient low\-rank neural networks via matrix differential equations\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p2.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p3.1)\.
- Sedghiet al\.\(2018\)H\. Sedghi, V\. Gupta, and P\. M\. LongThe Singular Values of Convolutional Layers\.InInternational Conference on Learning Representations,Cited by:[§B\.2\.1](https://arxiv.org/html/2608.28948#A2.S2.SS1.p2.1)\.
- Seideet al\.\(2011\)F\. Seide, G\. Li, X\. Chen, and D\. YuFeature engineering in context\-dependent deep neural networks for conversational speech transcription\.In2011 IEEE Workshop on Automatic Speech Recognition & Understanding,pp\.24–29\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p1.1)\.
- Shahet al\.\(2020\)H\. Shah, K\. Tamuly, A\. Raghunathan, P\. Jain, and P\. NetrapalliThe pitfalls of simplicity bias in neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[Figure 5\.1](https://arxiv.org/html/2608.28948#Ch5.F1),[Figure 5\.1](https://arxiv.org/html/2608.28948#Ch5.F1.9),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p4.1),[§5\.2\.2](https://arxiv.org/html/2608.28948#Ch5.S2.SS2.p1.1),[§5\.2](https://arxiv.org/html/2608.28948#Ch5.S2.p1.1),[§5\.2](https://arxiv.org/html/2608.28948#Ch5.S2.p3.1),[§5\.6](https://arxiv.org/html/2608.28948#Ch5.S6.p2.1)\.
- Sharmaet al\.\(2023\)P\. Sharma, J\. T\. Ash, and D\. MisraThe truth is in there: Improving reasoning in language models with layer\-selective rank reduction\.arXiv preprint arXiv:2312\.13558\.Cited by:[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p2.1)\.
- Sharmaet al\.\(2024\)P\. Sharma, J\. T\. Ash, and D\. MisraThe Truth is in There: Improving Reasoning in Language Models with Layer\-Selective Rank Reduction\.InThe Twelfth International Conference on Learning Representations,Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p4.1)\.
- Shazeeret al\.\(2017\)N\. Shazeer, A\. Mirhoseini, K\. Maziarz, A\. Davis, Q\. Le, G\. Hinton, and J\. DeanOutrageously large neural networks: The sparsely\-gated mixture\-of\-experts layer\.arXiv preprint arXiv:1701\.06538\.Cited by:[§5\.5\.3](https://arxiv.org/html/2608.28948#Ch5.S5.SS3.p1.1)\.
- Shenoudaet al\.\(2023\)J\. Shenouda, R\. Parhi, K\. Lee, and R\. D\. NowakVector\-valued variation spaces and width bounds for DNNs: Insights on weight decay regularization\.arXiv preprint arXiv:2305\.16534\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Silveret al\.\(2018\)D\. Silver, T\. Hubert, J\. Schrittwieser, I\. Antonoglou, M\. Lai, A\. Guez, M\. Lanctot, L\. Sifre, D\. Kumaran, T\. Graepel,et al\.A general reinforcement learning algorithm that masters chess, shogi, and Go through self\-play\.Science362\(6419\),pp\.1140–1144\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Silveret al\.\(2017\)D\. Silver, J\. Schrittwieser, K\. Simonyan, I\. Antonoglou, A\. Huang, A\. Guez, T\. Hubert, L\. Baker, M\. Lai, A\. Bolton,et al\.Mastering the game of go without human knowledge\.nature550\(7676\),pp\.354–359\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1)\.
- Simonyan and Zisserman \(2014\)K\. Simonyan and A\. ZissermanVery deep convolutional networks for large\-scale image recognition\.arXiv preprint arXiv:1409\.1556\.Cited by:[§B\.2\.1](https://arxiv.org/html/2608.28948#A2.S2.SS1.p1.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[1st item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i1.p1.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p3.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p6.1),[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Simseket al\.\(2021\)B\. Simsek, F\. Ged, A\. Jacot, F\. Spadaro, C\. Hongler, W\. Gerstner, and J\. BreaGeometry of the loss landscape in overparameterized neural networks: Symmetries and invariances\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Sinhaet al\.\(2021\)S\. Sinha, H\. Bharadhwaj, A\. Goyal, H\. Larochelle, A\. Garg, and F\. ShkurtiDibs: Diversity inducing information bottleneck in model ensembles\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\.9666–9674\.Cited by:[§5\.5\.4](https://arxiv.org/html/2608.28948#Ch5.S5.SS4.p1.1)\.
- Sitzmannet al\.\(2020\)V\. Sitzmann, J\. Martel, A\. Bergman, D\. Lindell, and G\. WetzsteinImplicit neural representations with periodic activation functions\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§A\.1](https://arxiv.org/html/2608.28948#A1.S1.p1.1)\.
- Sohl\-Dicksteinet al\.\(2015\)J\. Sohl\-Dickstein, E\. Weiss, N\. Maheswaranathan, and S\. GanguliDeep unsupervised learning using nonequilibrium thermodynamics\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[§B\.2\.2](https://arxiv.org/html/2608.28948#A2.S2.SS2.p1.1)\.
- Sontag and Sussmann \(1989\)E\. D\. Sontag and H\. J\. SussmannBackpropagation can give rise to spurious local minima even for networks without hidden layers\.Complex Systems3\(1\),pp\.91–106\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p4.1)\.
- Sunet al\.\(2017\)C\. Sun, A\. Shrivastava, S\. Singh, and A\. GuptaRevisiting unreasonable effectiveness of data in deep learning era\.In2017 IEEE International Conference on Computer Vision \(ICCV\),pp\.843–852\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1)\.
- Sutton \(2019\)R\. SuttonThe Bitter Lesson\.Note:[http://www\.incompleteideas\.net/IncIdeas/BitterLesson\.html](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)Incomplete Ideas \(Blog\)Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Szegedyet al\.\(2013\)C\. Szegedy, W\. Zaremba, I\. Sutskever, J\. Bruna, D\. Erhan, I\. Goodfellow, and R\. FergusIntriguing properties of neural networks\.arXiv:1312\.6199\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p3.1)\.
- Tan and Huang \(2023\)Z\. Tan and W\. HuangUnderstanding Grokking Through A Robustness Viewpoint\.arXiv preprint arXiv:2311\.06597\.Cited by:[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1)\.
- Teneyet al\.\(2022\)D\. Teney, E\. Abbasnejad, S\. Lucey, and A\. Van den HengelEvading the simplicity bias: Training a diverse set of models discovers solutions with superior ood generalization\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\.16761–16772\.Cited by:[§5\.5\.2](https://arxiv.org/html/2608.28948#Ch5.S5.SS2.p1.1)\.
- Thilaket al\.\(2022\)V\. Thilak, E\. Littwin, S\. Zhai, O\. Saremi, R\. Paiss, and J\. SusskindThe slingshot mechanism: An empirical study of adaptive optimizers and the grokking phenomenon\.arXiv preprint arXiv:2206\.04817\.Cited by:[Figure B\.4](https://arxiv.org/html/2608.28948#A2.F4),[Figure B\.4](https://arxiv.org/html/2608.28948#A2.F4.32.4),[§B\.2\.7](https://arxiv.org/html/2608.28948#A2.S2.SS7.p2.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1)\.
- Timoret al\.\(2023\)N\. Timor, G\. Vardi, and O\. ShamirImplicit regularization towards rank minimization in ReLU networks\.InProceedings of the International Conference on Algorithmic Learning Theory \(ALT\),Cited by:[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p2.1)\.
- Tiwari and Shenoy \(2023\)R\. Tiwari and P\. ShenoyOvercoming simplicity bias in deep networks using a feature sieve\.InInternational Conference on Machine Learning,pp\.34330–34343\.Cited by:[§5\.5\.2](https://arxiv.org/html/2608.28948#Ch5.S5.SS2.p1.1)\.
- Toet al\.\(2025\)M\. N\. N\. To, P\. F\. RWilson, V\. Nguyen, M\. Harmanani, M\. Cooper, F\. Fooladgar, P\. Abolmaesumi, P\. Mousavi, and R\. G\. KrishnanDiverse Prototypical Ensembles Improve Robustness to Subpopulation Shift\.arXiv preprint arXiv:2505\.23027\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p3.1),[§5\.5\.3](https://arxiv.org/html/2608.28948#Ch5.S5.SS3.p1.1)\.
- Tunguz \(2026\)T\. TunguzThe 12x Bet on AI\.Note:Personal BlogExternal Links:[Link](https://tomtunguz.com/12x-bet-on-ai/)Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p2.1)\.
- Van Laarhoven \(2017\)T\. Van LaarhovenL2 regularization versus batch and weight normalization\.arXiv preprint arXiv:1706\.05350\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1)\.
- Vandenberghe \(2004\)L\. VandenbergheConvex optimization\.Vol\.1,Cambridge University Press\.Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p3.2)\.
- Vardi and Shamir \(2021\)G\. Vardi and O\. ShamirImplicit regularization in relu networks with the square loss\.InProceedings of the Annual Conference on Learning Theory \(COLT\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Vaswaniet al\.\(2017\)A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. PolosukhinAttention is all you need\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§B\.2\.4](https://arxiv.org/html/2608.28948#A2.S2.SS4.p1.1),[§C\.3](https://arxiv.org/html/2608.28948#A3.S3.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p3.1),[4th item](https://arxiv.org/html/2608.28948#Ch4.S5.I1.i4.p1.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx2.p2.1)\.
- Villaloboset al\.\(2024\)P\. Villalobos, A\. Ho, J\. Sevilla, T\. Besiroglu, L\. Heim, and M\. HobbhahnPosition: Will we run out of data? Limits of LLM scaling based on human\-generated data\.InForty\-first International Conference on Machine Learning,Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Vlaar and Frankle \(2022\)T\. J\. Vlaar and J\. FrankleWhat can linear interpolation of neural network loss landscapes tell us?\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\.22325–22341\.Cited by:[§3\.3](https://arxiv.org/html/2608.28948#Ch3.S3.p1.1),[§3\.5](https://arxiv.org/html/2608.28948#Ch3.S5.p2.1)\.
- Wang and Vastola \(2022\)B\. Wang and J\. VastolaML from scratch: Stable Diffusion, Day 2\.External Links:[Link](https://colab.research.google.com/drive/1Y5wr91g5jmpCDiX-RLfWL1eSBWoSuLqO?usp=sharing#scrollTo=9is-DXZYwIIi)Cited by:[§B\.2\.2](https://arxiv.org/html/2608.28948#A2.S2.SS2.p1.1)\.
- Wanget al\.\(2021\)H\. Wang, S\. Agarwal, and D\. PapailiopoulosPufferfish: Communication\-efficient models at no extra cost\.Proceedings of Machine Learning and Systems3,pp\.365–386\.Cited by:[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p2.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p3.1)\.
- Wanget al\.\(2026\)S\. Wang, L\. Yu, C\. Gao, C\. Zheng, S\. Liu, R\. Lu, K\. Dang, X\. Chen, J\. Yang, Z\. Zhang,et al\.Beyond the 80/20 rule: High\-entropy minority tokens drive effective reinforcement learning for llm reasoning\.Advances in Neural Information Processing Systems38,pp\.115452–115486\.Cited by:[§5\.4\.3](https://arxiv.org/html/2608.28948#Ch5.S4.SS3.p1.1)\.
- Wanget al\.\(2018\)T\. Wang, J\. Zhu, A\. Torralba, and A\. A\. EfrosDataset distillation\.arXiv preprint arXiv:1811\.10959\.Cited by:[§5\.4\.4](https://arxiv.org/html/2608.28948#Ch5.S4.SS4.p7.1)\.
- Wang and Jacot \(2023\)Z\. Wang and A\. JacotImplicit bias of SGD inL​\_L\\\_\{22\}\-regularized linear DNNs: One\-way jumps from high to low rank\.arXiv preprint arXiv:2305\.16038\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Waskom \(2021\)M\. L\. Waskomseaborn: statistical data visualization\.Journal of Open Source Software6\(60\),pp\.3021\.Cited by:[§B\.6](https://arxiv.org/html/2608.28948#A2.S6.p1.1)\.
- Wenet al\.\(2025\)T\. Wen, Z\. Wang, Q\. Zhang, and Q\. LeiElastic Representation: Mitigating Spurious Correlations for Group Robustness\.arXiv preprint arXiv:2502\.09850\.Cited by:[§5\.5\.1](https://arxiv.org/html/2608.28948#Ch5.S5.SS1.p1.1)\.
- Woodworthet al\.\(2020\)B\. Woodworth, S\. Gunasekar, J\. D\. Lee, E\. Moroshko, P\. Savarese, I\. Golan, D\. Soudry, and N\. SrebroKernel and rich regimes in overparametrized models\.InConference on Learning Theory,pp\.3635–3673\.Cited by:[Figure B\.5](https://arxiv.org/html/2608.28948#A2.F5),[Figure B\.5](https://arxiv.org/html/2608.28948#A2.F5.10),[§B\.4\.1](https://arxiv.org/html/2608.28948#A2.S4.SS1.p1.1)\.
- Wortsmanet al\.\(2023\)M\. Wortsman, T\. Dettmers, L\. Zettlemoyer, A\. Morcos, A\. Farhadi, and L\. SchmidtStable and low\-precision training for large\-scale vision\-language models\.Advances in Neural Information Processing Systems \(NeurIPS\)36,pp\.10271–10298\.Cited by:[Chapter 6](https://arxiv.org/html/2608.28948#Ch6.p7.1)\.
- Wortsmanet al\.\(2022\)M\. Wortsman, G\. Ilharco, S\. Y\. Gadre, R\. Roelofs, R\. Gontijo\-Lopes, A\. S\. Morcos, H\. Namkoong, A\. Farhadi, Y\. Carmon, S\. Kornblith, and L\. SchmidtModel soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.InProceedings of the International Conference on Machine Learning \(ICML\),Cited by:[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p10.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.SSSx2.p2.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1),[§4\.9](https://arxiv.org/html/2608.28948#Ch4.S9.p1.1)\.
- Xuet al\.\(2023\)Z\. Xu, Y\. Wang, S\. Frei, G\. Vardi, and W\. HuBenign Overfitting and Grokking in ReLU Networks for XOR Cluster Data\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p1.1)\.
- Yanget al\.\(2025\)X\. A\. Yang, J\. Zhang, and L\. BottouThese are Not All the Features You are Looking For: A Fundamental Bottleneck In Supervised Pretraining\.arXiv preprint arXiv:2506\.18221\.Cited by:[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx1.p3.1),[§5\.3](https://arxiv.org/html/2608.28948#Ch5.S3.p1.1),[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1),[§5\.6](https://arxiv.org/html/2608.28948#Ch5.S6.p4.1)\.
- Yaraset al\.\(2023a\)C\. Yaras, P\. Wang, W\. Hu, Z\. Zhu, L\. Balzano, and Q\. QuInvariant Low\-Dimensional Subspaces in Gradient Descent for Learning Deep Matrix Factorizations\.InNeurIPS 2023 Workshop on Mathematics of Modern Machine Learning,Cited by:[§4\.2\.1](https://arxiv.org/html/2608.28948#Ch4.S2.SS1.p1.1),[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p1.1),[§4\.5](https://arxiv.org/html/2608.28948#Ch4.S5.p1.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1)\.
- Yaraset al\.\(2023b\)C\. Yaras, P\. Wang, W\. Hu, Z\. Zhu, L\. Balzano, and Q\. QuThe Law of Parsimony in Gradient Descent for Learning Deep Linear Networks\.arXiv preprint arXiv:2306\.01154\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[§4\.3](https://arxiv.org/html/2608.28948#Ch4.S3.p1.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1)\.
- Yu and Wu \(2023\)H\. Yu and J\. WuCompressing Transformers: Features Are Low\-Rank, but Weights Are Not\!\.InProceedings of the National Conference on Artificial Intelligence \(AAAI\),Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1),[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Yuet al\.\(2017\)X\. Yu, T\. Liu, X\. Wang, and D\. TaoOn compressing deep models by low rank and sparse decomposition\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§4\.10](https://arxiv.org/html/2608.28948#Ch4.S10.p4.1),[§4\.5\.2](https://arxiv.org/html/2608.28948#Ch4.S5.SS2.p2.1),[§4\.9\.2](https://arxiv.org/html/2608.28948#Ch4.S9.SS2.p3.1)\.
- Yuniset al\.\(2022\)D\. Yunis, K\. K\. Patel, P\. H\. P\. Savarese, G\. Vardi, J\. Frankle, M\. Walter, K\. Livescu, and M\. MaireOn Convexity and Linear Mode Connectivity in Neural Networks\.InOPT 2022: Optimization for Machine Learning \(NeurIPS 2022 Workshop\),Cited by:[Chapter 3](https://arxiv.org/html/2608.28948#Ch3),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p4.1),[§4\.9\.3](https://arxiv.org/html/2608.28948#Ch4.S9.SS3.p1.1)\.
- Yuniset al\.\(2024\)D\. Yunis, K\. K\. Patel, S\. Wheeler, P\. Savarese, G\. Vardi, K\. Livescu, M\. Maire, and M\. R\. WalterApproaching deep learning through the spectral dynamics of weights\.arXiv preprint arXiv:2408\.11804\.Cited by:[Chapter 4](https://arxiv.org/html/2608.28948#Ch4)\.
- Zangrandoet al\.\(2024\)E\. Zangrando, P\. Deidda, S\. Brugiapaglia, N\. Guglielmi, and F\. TudiscoNeural Rank Collapse: Weight Decay and Small Within\-Class Variability Yield Low\-Rank Bias\.arXiv preprint arXiv:2402\.03991\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1),[§4\.4](https://arxiv.org/html/2608.28948#Ch4.S4.p2.1),[§4\.6](https://arxiv.org/html/2608.28948#Ch4.S6.p1.1)\.
- Zhanget al\.\(2021\)C\. Zhang, S\. Bengio, M\. Hardt, B\. Recht, and O\. VinyalsUnderstanding deep learning \(still\) requires rethinking generalization\.Communications of the ACM64\(3\),pp\.107–115\.Cited by:[§B\.2\.8](https://arxiv.org/html/2608.28948#A2.S2.SS8.p1.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p6.1),[Chapter 2](https://arxiv.org/html/2608.28948#Ch2.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p1.1),[§4\.1](https://arxiv.org/html/2608.28948#Ch4.S1.p6.1),[§4\.7](https://arxiv.org/html/2608.28948#Ch4.S7.p2.1),[§4\.8](https://arxiv.org/html/2608.28948#Ch4.S8.p1.1),[§4\.8](https://arxiv.org/html/2608.28948#Ch4.S8.p2.1)\.
- Zhanget al\.\(2018a\)G\. Zhang, C\. Wang, B\. Xu, and R\. GrosseThree mechanisms of weight decay regularization\.arXiv preprint arXiv:1810\.12281\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1)\.
- Zhanget al\.\(2018b\)H\. Zhang, Y\. N\. Dauphin, and T\. MaFixup Initialization: Residual Learning Without Normalization\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§4\.5\.1](https://arxiv.org/html/2608.28948#Ch4.S5.SS1.p3.1)\.
- Zhang and Bottou \(2023\)J\. Zhang and L\. BottouLearning useful representations for shifting tasks and distributions\.InInternational Conference on Machine Learning,pp\.40830–40850\.Cited by:[§C\.2](https://arxiv.org/html/2608.28948#A3.S2.p2.1),[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p4.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p3.1),[§5\.3\.1](https://arxiv.org/html/2608.28948#Ch5.S3.SS1.p1.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx1.p1.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx1.p3.1),[§5\.3](https://arxiv.org/html/2608.28948#Ch5.S3.p1.1),[§5\.3](https://arxiv.org/html/2608.28948#Ch5.S3.p2.1),[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1),[§5\.6](https://arxiv.org/html/2608.28948#Ch5.S6.p3.1),[§5\.6](https://arxiv.org/html/2608.28948#Ch5.S6.p4.1),[Table 5\.2](https://arxiv.org/html/2608.28948#Ch5.T2),[Table 5\.2](https://arxiv.org/html/2608.28948#Ch5.T2.9)\.
- Zhanget al\.\(2022\)J\. Zhang, D\. Lopez\-Paz, and L\. BottouRich feature construction for the optimization\-generalization dilemma\.InInternational Conference on Machine Learning,pp\.26397–26411\.Cited by:[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p3.1),[§5\.4\.2](https://arxiv.org/html/2608.28948#Ch5.S4.SS2.p1.1),[§5\.4\.2](https://arxiv.org/html/2608.28948#Ch5.S4.SS2.p2.1),[§5\.5\.5](https://arxiv.org/html/2608.28948#Ch5.S5.SS5.p1.1)\.
- Zhanget al\.\(2025\)J\. Zhang, N\. Nolte, R\. Sadhukhan, B\. Chen, and L\. BottouMemory mosaics\.InInternational Conference on Learning Representations,Vol\.2025,pp\.36412–36433\.Cited by:[§C\.3](https://arxiv.org/html/2608.28948#A3.S3.p1.1),[§5\.3\.2](https://arxiv.org/html/2608.28948#Ch5.S3.SS2.SSSx2.p2.1)\.
- Zhouet al\.\(2022\)Y\. Zhou, T\. Lei, H\. Liu, N\. Du, Y\. Huang, V\. Zhao, A\. M\. Dai, Q\. V\. Le, J\. Laudon,et al\.Mixture\-of\-experts with expert choice routing\.Advances in Neural Information Processing Systems35,pp\.7103–7114\.Cited by:[§5\.5\.3](https://arxiv.org/html/2608.28948#Ch5.S5.SS3.p1.1)\.
- Ziyinet al\.\(2022\)L\. Ziyin, B\. Li, and X\. MengExact solutions of a deep linear network\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.2\.2](https://arxiv.org/html/2608.28948#Ch4.S2.SS2.p1.1)\.
- Zouet al\.\(2023\)A\. Zou, Z\. Wang, J\. Z\. Kolter, and M\. FredriksonUniversal and transferable adversarial attacks on aligned language models\.arXiv preprint arXiv:2307\.15043\.Cited by:[Chapter 1](https://arxiv.org/html/2608.28948#Ch1.p5.1),[§5\.1](https://arxiv.org/html/2608.28948#Ch5.S1.p1.1)\.

## Appendix AConvex Mode Connectivity

### A\.1Task Details

SIREN\([Sitzmann et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib188)\)is a fully\-connected network with sine activations\. It was originally designed to overcome the low\-frequency bias in radiance fields\. Here it is tuned to reconstruct an audio waveform, where the input is the timestep, and the output is the waveform value\. We choose to study it because all prior experiments on LMC were conducted with ReLU activations, and we wished to observe a system that was not piece\-wise linear\. For our other models, we use LeNet\-5\([LeCun et al\., 1998](https://arxiv.org/html/2608.28948#bib.bib237)\), and ResNet\-20\([He et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib67)\)trained on CIFAR\-10\([Krizhevsky, 2009](https://arxiv.org/html/2608.28948#bib.bib102)\)\. These combinations are some of the simplest systems for which we do not see LMC at initialization\([Nagarajan and Kolter, 2019](https://arxiv.org/html/2608.28948#bib.bib136);[Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52);[Ainsworth et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib4)\)\. For the sake of simplicity, all learning rates are constant in our experiments\. Because the SIREN task does not include generalization, we stop training when the training loss has converged\. In the case of our other models, we tune hyper\-parameters and stop training when the validation error has converged, which is typically the desired criterion in practice\. For hyperparameters, see Table[A\.1](https://arxiv.org/html/2608.28948#A1.T1)\. For full training loss curves, see Figure[A\.1](https://arxiv.org/html/2608.28948#A1.F1)\.

Table A\.1:Summary of tasks studied in this chapter\.\(a\)SIREN\(b\)LeNet\(c\)LeNet\-m\(d\)ResNet
Figure A\.1:Training loss curves for all tasks\. When cross\-referencing with Figure[3\.2](https://arxiv.org/html/2608.28948#Ch3.F2), we see that the first split point that leads to a convex hull is quite early, long before the loss has converged, and often before it has even halved\.
### A\.2Metric Calculation Details

In order to calculate Hessian eigenvalues, we use a power iteration method with Hessian vector products, like in[Li et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib112)\. To measure local convexity between a pair of stochastic updates from parametersWW, we compute two stochastic updatesΔ​W1\\Delta W\_\{1\}andΔ​W2\\Delta W\_\{2\}, and measure the gap

g=ℒ⁡\(12​\(W\+Δ​W1\)\+12​\(W\+Δ​W2\)\)−\(12​ℒ​\(W\+Δ​W1\)\+12​ℒ​\(W\+Δ​W2\)\)\.g=\\mathcal\{L\}\\left\(\\frac\{1\}\{2\}\(W\+\\Delta W\_\{1\}\)\+\\frac\{1\}\{2\}\(W\+\\Delta W\_\{2\}\)\\right\)\-\\left\(\\frac\{1\}\{2\}\\mathcal\{L\}\(W\+\\Delta W\_\{1\}\)\+\\frac\{1\}\{2\}\\mathcal\{L\}\(W\+\\Delta W\_\{2\}\)\\right\)\.We then compute the probability that this gap is negative over many samples\. This gives us a metric for the non\-convexity that a single update induces, which we can measure at different points throughout training\.

## Appendix BSpectral Dynamics

### B\.1Explanation of Balancedness

Prior work on deep linear networks\([Arora et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib8);[Milanesi et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib132)\)suggests that rank minimization may describe implicit regularization in deep matrix factorization better than simple matrix norms\. See[Arora et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib7)\(Appendix A\) for a detailed argument\. However, a critical assumption used in these works is “balanced initialization\.” This means that for consecutive matricesWiW\_\{i\}andWi\+1W\_\{i\+1\}in the product matrix∏jWj\\prod\_\{j\}W\_\{j\}, we haveWi\+1⊤​Wi\+1=Wi​Wi⊤W^\{\\top\}\_\{i\+1\}W\_\{i\+1\}=W\_\{i\}W\_\{i\}^\{\\top\}at initialization\. Decomposing these matrices with SVDs and leveraging orthogonality, this simplifies toVi\+1​Σi\+12​Vi\+1⊤=Ui​Σi2​Ui⊤V\_\{i\+1\}\\Sigma\_\{i\+1\}^\{2\}V\_\{i\+1\}^\{\\top\}=U\_\{i\}\\Sigma\_\{i\}^\{2\}U\_\{i\}^\{\\top\}whereUiU\_\{i\}andVi\+1V\_\{i\+1\}are orthogonal matrices\. Since these are orthogonal decompositions of the same matrix, their diagonals must be equivalent, allowing for the permutation of elements with the same value\. This leads toUi=Vi\+1​OU\_\{i\}=V\_\{i\+1\}Oup to signs, whereOOis a block diagonal permutation matrix that may permute the rows of equivalent diagonal elements\. Notably, if all diagonal elements are distinct andUiU\_\{i\}andVi\+1V\_\{i\+1\}are square matrices, thenUi=Vi\+1U\_\{i\}=V\_\{i\+1\}up to signs\. This gives us matching singular vectors for consecutive matrices\.

### B\.2Experimental Details

For all experiments, we use 3 random seeds and average all plots over those 3\. This is relatively small, but error bars tend to be very tight, and due to the high volume of runs required for this work we lack the resources to run much more\.

In order to compute alignment we consider only pairs of consecutive layers that directly feed into each other, and ignore the influence of residual connections so as to cut down on the number of comparisons\. Specifics on individual architectures are given below\.

#### B\.2\.1Image Classification with VGG

We train a VGG\-16\([Simonyan and Zisserman, 2014](https://arxiv.org/html/2608.28948#bib.bib185)\)on CIFAR\-10\([Krizhevsky, 2009](https://arxiv.org/html/2608.28948#bib.bib102)\)for 164 epochs, following the hyperparameters and learning rate schedule in\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\), but without data augmentation as it contributes extra randomness\. For the optimizer we use SGD with batch size 128, initial learning rate 0\.1 and momentum of 0\.9\. We also decay the learning rate 3 times by a factor of 10 at epoch 82, epoch 120, and finally at epoch 160\. We also use a minor amount of weight decay with coefficient 0\.0001\.

VGG\-16 uses ReLU activations and batch normalization\([Ioffe and Szegedy, 2015](https://arxiv.org/html/2608.28948#bib.bib81)\), and includes both convolutional and linear layers\. For linear layers we simply compute the SVD of the weight matrix\. For convolutional layers, the parameters are typically stored as a 4D tensor of shape\(cout,cin,h,w\)\(c\_\{\\text\{out\}\},c\_\{\\text\{in\}\},h,w\)for the output channels, input channels, height and width of the filters respectively\. As the filters compute a transformation from each position and input channel to an output channel, we compute the SVD of the flattened tensor of shape\(cout,cin⋅h⋅w\)\(c\_\{\\text\{out\}\},c\_\{\\text\{in\}\}\\cdot h\\cdot w\), which maps all inputs to outputs, similar to[Praggastis et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib155)\. This is not the SVD of the entire transformation of the feature map to the next feature map[Sedghi et al\. \(2018\)](https://arxiv.org/html/2608.28948#bib.bib176), but rather the transformation from a set of adjacent positions to a particular position in the next layer computed by the parameters\. For the individual SV evolution plot, we use the 12th convolutional layer\.

In order to compute alignment of bases between consecutive convolutional layers,Vi\+1⊤​UiV\_\{i\+1\}^\{\\top\}U\_\{i\}we need to match the dimensionality betweenUiU\_\{i\}andVi\+1V\_\{i\+1\}\. For convolutional layers we are presented with a question as to how to handle the spatial dimensionshhandwwas naively the input dimension of the next layer will be a factor ofh⋅wh\\cdot wlarger dimension\. We experimented with multiple cases, including aligning at each spatial position individually or averaging over the alignment at all spatial positions, and eventually settled at aligning the output of one layer to the center spatial input of the next layer\. That is, for a 3x3 convolution mapping to a following 3x3 convolution, we compute the alignment only for position \(1,1\) of the next layer\. This seemed reasonable to us as on average the edges of the filters showed poorer alignment overall\. For the individual alignment plot, we use the alignment between the 11th and 12th convolutional layers at the center spatial position of the 12th convolutional layer\.

#### B\.2\.2Image Generation with UNets

We train a UNet\([Ronneberger et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib166)\)diffusion model\([Sohl\-Dickstein et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib189);[Ho et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib71)\)on MNIST\([LeCun, 1998](https://arxiv.org/html/2608.28948#bib.bib109)\)generation\. We take model design and hyperparameters from\([Wang and Vastola, 2022](https://arxiv.org/html/2608.28948#bib.bib208)\)\. In particular we use a 4\-layer residual UNet and train with AdamW\([Loshchilov and Hutter, 2017](https://arxiv.org/html/2608.28948#bib.bib121)\)with batch size 128, and learning rate of 0\.0003 for 100 epochs\. This model uses swish\([Ramachandran et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib162)\)activations and a combination of linear and convolutional, as well as transposed convolutional layers\.

Computing SVDs and alignment is similar to the image classification case described above, except in the case of the transposed convolutions where an extra transpose of dimensions is needed as parameters are stored with the shape\(cin,cout,h,w\)\(c\_\{\\text\{in\}\},c\_\{\\text\{out\}\},h,w\)\. For the individual SV evolution plot, we use the 3rd convolutional layer\. For the alignment plot, we use the alignment between the 3rd and 4th convolutional layers at the center spatial position of the 4th convolutional layer\.

#### B\.2\.3Speech Recognition with LSTMs

We train a bidirectional LSTM\([Hochreiter and Schmidhuber, 1997](https://arxiv.org/html/2608.28948#bib.bib72)\)for automatic speech recognition on LibriSpeech\([Panayotov et al\., 2015](https://arxiv.org/html/2608.28948#bib.bib146)\)\. We tune for a simple and well\-performing hyperparameter setting\. We use AdamW\([Loshchilov and Hutter, 2017](https://arxiv.org/html/2608.28948#bib.bib121)\)with batch size 32, learning rate 0\.0003 and weight decay 0\.1 for 50 epochs\. We also use a cosine annealing learning rate schedule from 1 to 0 over the entire 50 epochs\.

The LSTM only has matrix parameters and biases, so it is straightforward to compute SVDs of the matrices\. For individual SV evolution plots, we plot the 3rd layer input parameter\. In the case of alignment, we make a number of connections: first down depth for the input parameters, then connecting the previous input parameter to the current hidden parameter in both directions, then connecting the previous hidden parameter to the current input parameter\. In particular the LSTM parameters are stored as a stack of 4 matrices in PyTorch, and we find alignment is highest for the "gate" submatrix, so we choose that for all plots\. For the individual layer alignment, we plot alignment between the 3rd and 4th layer input parameters\.

#### B\.2\.4Language Modeling with Transformers

We train a Transformer\([Vaswani et al\., 2017](https://arxiv.org/html/2608.28948#bib.bib204)\)language model on Wikitext\-103\([Merity et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib130)\)\. We base hyperparameter choices on the Pythia suite\([Biderman et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib17)\), specifically the 160 million parameter configuration with sinusoidal position embeddings, 12 layers, model dimension 768, 12 attention heads per layer, and hidden dimension 768\. We use AdamW\([Loshchilov and Hutter, 2017](https://arxiv.org/html/2608.28948#bib.bib121)\)with batch size 256, learning rate 0\.0006 and weight decay 0\.1\. We use a context length of 2048 and clip gradients to a maximum norm of 1\. We also use a learning rate schedule with a linear warmup and cosine decay to 10% of the learning rate, like[Biderman et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib17)\.

For SVDs, for simplicity we take the SVD of the entire\(3​dmodel,dmodel\)\(3d\_\{\\text\{model\}\},d\_\{\\text\{model\}\}\)parameter that computes queries, keys and values from the hidden dimension inside the attention layer, without splitting into individual heads\. This is reasonable as the splitting is done after the fact internally\. We also take the SVD of the output parameters, and linear layers of the MLPs, which are 2 dimensional matrices\. For the individual SV evolution plot, we plot the SVs ofW1W\_\{1\}of the 8th layer MLP\.

For alignment, we consider the alignment ofWQW\_\{Q\}andWKW\_\{K\}matrices,WVW\_\{V\}andWOW\_\{O\}matrices, computing alignment between heads individually then averaging over all heads\. We also consider the alignment betweenWOW\_\{O\}andW1W\_\{1\}of the MLP block, betweenW1W\_\{1\}andW2W\_\{2\}of the MLP block, and betweenW2W\_\{2\}and the next attention layer\. For the individual layer alignment, we plot alignment betweenW1W\_\{1\}andW2W\_\{2\}of the 8th layer MLP\.

#### B\.2\.5Spectral Dynamics with Scale \(Pythia\)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_160m-val-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_410m-val-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_1.4b-val-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_2.8b-val-loss.png)

\(a\)Val\. Loss
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_160m-sv-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_410m-sv-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_1.4b-sv-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_2.8b-sv-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_sv.png)

\(b\)SVs
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_160m-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_410m-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_1.4b-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_2.8b-sv-eff_rank.png)

\(c\)Eff\. Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_160m-align-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_gpt_neox.layers.7.mlp.dense_4h_to_h.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_410m-align-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_gpt_neox.layers.7.mlp.dense_4h_to_h.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_1.4b-align-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_gpt_neox.layers.7.mlp.dense_4h_to_h.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_2.8b-align-gpt_neox.layers.7.mlp.dense_h_to_4h.weight_gpt_neox.layers.7.mlp.dense_4h_to_h.weight_align-diag.png)

\(d\)Alignment
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_160m-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_410m-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_1.4b-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/pythia/size_2.8b-align-align_score.png)

\(e\)Align\. Score

Figure B\.1:Spectral dynamics of Pythia suite\. From top to bottom we examine the 160m, 410m, 1\.4b and 2\.8b parameter models\. Notably, much less noise appears in the alignment plot with increasing scale\. Presumably this could be due to the fact that larger dimensional vectors have higher probability to be orthogonal, which may play a role in making optimization easier\. We see stronger alignment score \(Eqn\.[4\.4](https://arxiv.org/html/2608.28948#Ch4.E4)\) in all layers in the larger model, perhaps because of that cleaner signal\.Here we apply the perspective developed in Section[4\.5](https://arxiv.org/html/2608.28948#Ch4.S5)to larger scale models\. As we lack the resources to train these models ourselves, we leverage the Pythia\([Biderman et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib17)\)family which provides training trajectories for language models across a range of scales \(70m to 12b parameters\)\. We are further constrained to the 2\.8b parameter model at the largest due to memory requirements when computing SVDs and alignment\.

In Figure[B\.1](https://arxiv.org/html/2608.28948#A2.F1), we see similar rank dynamics across a variety of scales\. We select the 7th layer MLP to compare between models as it is present at all scales\. We do see an unequal evolution in singular values, but also a contraction as training proceeds for longer\. The difference between scales is not very obvious, but proportionally fewer of the singular values evolve to be large in the 2\.8b model as opposed to the 410m model, which one can see from the thickness of the light magenta color\. The lack of alignment except for the top rank is quite consistent with earlier observations, and such alignment happens much later for the largest model\.

#### B\.2\.6Weight Decay Experiments

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.0-align-model.layers.11.conv.weight_model.layers.12.conv.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.0-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.001-align-model.layers.11.conv.weight_model.layers.12.conv.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.001-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.01-align-model.layers.11.conv.weight_model.layers.12.conv.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.01-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.1-align-model.layers.11.conv.weight_model.layers.12.conv.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.1-align-align_score.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.0-align-model.conv3.weight_model.conv4.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.0-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.1-align-model.conv3.weight_model.conv4.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.1-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_1.0-align-model.conv3.weight_model.conv4.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_1.0-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_10.0-align-model.conv3.weight_model.conv4.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_10.0-align-align_score.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.0-align-model.lstm.weight_ih_l2_model.lstm.weight_ih_l3_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.0-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.1-align-model.lstm.weight_ih_l2_model.lstm.weight_ih_l3_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.1-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_1.0-align-model.lstm.weight_ih_l2_model.lstm.weight_ih_l3_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_1.0-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_10.0-align-model.lstm.weight_ih_l2_model.lstm.weight_ih_l3_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_10.0-align-align_score.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.0-align-model.transformer_encoder.layers.7.linear1.weight_model.transformer_encoder.layers.7.linear2.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.0-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.1-align-model.transformer_encoder.layers.7.linear1.weight_model.transformer_encoder.layers.7.linear2.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.1-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_1.0-align-model.transformer_encoder.layers.7.linear1.weight_model.transformer_encoder.layers.7.linear2.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_1.0-align-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_10.0-align-model.transformer_encoder.layers.7.linear1.weight_model.transformer_encoder.layers.7.linear2.weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_10.0-align-align_score.png)

\(d\)Transformer

Figure B\.2:Diagonal of alignment for a single pair over time \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) and alignment metric across pairs of matrices over time \(Eqn\.[4\.4](https://arxiv.org/html/2608.28948#Ch4.E4)\) where the y\-axis represents depth\. From top to bottom, for VGG we use coefficients\{0,0\.001,0\.01,0\.1\}\\\{0,0\.001,0\.01,0\.1\\\}, while for other networks we use coefficients\{0,0\.1,1,10\}\\\{0,0\.1,1,10\\\}\. We see that the maximum alignment magnitude is higher with large weight decay, and in particular, the Transformer has the strongest alignment even when nonlinearities separate the MLP layers\.![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.0-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.001-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.01-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.1-train-loss.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.0-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.1-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_1.0-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_10.0-train-loss.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.0-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.1-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_1.0-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_10.0-train-loss.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.0-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.1-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_1.0-train-loss.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_10.0-train-loss.png)

\(d\)Transformer

Figure B\.3:Training loss over time, where the rows use differing amounts of weight decay\. From top to bottom, for VGG we use coefficients\{0,0\.001,0\.01,0\.1\}\\\{0,0\.001,0\.01,0\.1\\\}, while for other networks we use coefficients\{0,0\.1,1,10\}\\\{0,0\.1,1,10\\\}\. We see that it is still possible to achieve low training loss under high weight decay, and as we increase the amount of weight decay, the gap between pruned and unpruned parameters closes, lending support to the idea that the parameters become lower rank\.All tasks are trained in exactly the same fashion as mentioned previously, with increasing weight decay in the set\{0,0\.0001,0\.001,0\.01,0\.1,1\.0,10\.0\}\\\{0,0\.0001,0\.001,0\.01,0\.1,1\.0,10\.0\\\}\. For ease of presentation we consider a subset of settings across tasks\. In Figure[B\.3](https://arxiv.org/html/2608.28948#A2.F3)we include trained model performance and pruned model performance to show that, even with high levels of weight decay, models do not entirely break down\. More so, the approximation of the pruned model to the full model gets better with higher weight decay\.

#### B\.2\.7Grokking Experiments

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/weight_decay_0.0-error.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-error.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/slingshot-error.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/frac_0.9-error.png)

\(a\)Error
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/weight_decay_0.0-sv-model.transformer_encoder.layers.0.self_attn.in_proj_weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-sv-model.transformer_encoder.layers.0.self_attn.in_proj_weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/slingshot-sv-model.transformer_encoder.layers.0.self_attn.in_proj_weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/frac_0.9-sv-model.transformer_encoder.layers.0.self_attn.in_proj_weight_sv.png)

\(b\)SV Evolution
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/weight_decay_0.0-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/slingshot-sv-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/frac_0.9-sv-eff_rank.png)

\(c\)Effective Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/weight_decay_0.0-align-model.encoder.weight_model.transformer_encoder.layers.0.self_attn.in_proj_weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-align-model.encoder.weight_model.transformer_encoder.layers.0.self_attn.in_proj_weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/slingshot-align-model.encoder.weight_model.transformer_encoder.layers.0.self_attn.in_proj_weight_align-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/frac_0.9-align-model.encoder.weight_model.transformer_encoder.layers.0.self_attn.in_proj_weight_align-diag.png)

\(d\)Alignment

Figure B\.4:Grokking and Spectral Dynamics in Modular addition\.Top row:30% data and no weight decay\.2nd row:30% data and weight decay 1\.0 \(grokking\), using hyperparameters from[Nanda et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib138)\.3rd row:70% data with no weight decay \(slingshot\), using hyperparameters from[Thilak et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib196)\.Bottom row:90% data and no weight decay\.\(a\):Training and validation error\.\(b\):Singular value evolution is visualized for the first attention parameter, where each line represents a single singular value and the color represents the rank\.\(c\):Effective rank of all layers \(Eqn\.[4\.1](https://arxiv.org/html/2608.28948#Ch4.E1)\)\.\(d\):Alignment \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) between the embedding and the first attention parameter is also visualized, where the y\-axis corresponds to indexiiof the diagonal\. One can see that grokking co\-occurs with low\-rank weights\. In addition, there is an alignment that begins early in training that evolves up the diagonal\. Without weight decay and with less data, neither grokking nor the other phenomena occur during the entire training budget, but using more data, even without weight decay, leads to low\-rank solutions from the beginning of training\. The slingshot case follows a similar trend, though the validation loss is gradually fit\. Across cases with good generalization, parameters are lower rank, and alignment is also more prevalent in the top ranks\.For the Transformer, we mostly follow the settings and architecture of[Nanda et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib138), except we use sinusoidal positional encodings instead of learned\.

For the slingshot case we follow hyperparameter settings in[Thilak et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib196), Appendix B except with the 1\-layer architecture from[Nanda et al\. \(2023\)](https://arxiv.org/html/2608.28948#bib.bib138)instead of the 2\-layer architecture specified\. We perform addition modulo 97\. The original grokking plot in[Thilak et al\. \(2022\)](https://arxiv.org/html/2608.28948#bib.bib196)appears much more dramatic as it log\-scales the x\-axis, which we do not do here for clarity\.

In the case of the deep MLP, we follow[Fan et al\. \(2024\)](https://arxiv.org/html/2608.28948#bib.bib45), where we use a 12\-layer MLP with ReLU activations and width 400, trained on MSE loss on MNIST\([LeCun, 1998](https://arxiv.org/html/2608.28948#bib.bib109)\)\. We use 2000 examples, a batch size of 100, weight decay 0\.01, and initialization scale 8\([Liu et al\., 2023](https://arxiv.org/html/2608.28948#bib.bib119)\)\.

#### B\.2\.8Random Label Experiments

We train a 4\-layer MLP on CIFAR10\([Krizhevsky, 2009](https://arxiv.org/html/2608.28948#bib.bib102)\)with either completely random labels, or the true labels\. We use SGD with momentum of 0\.9 and constant learning rate of 0\.001, and train for 300 epochs to see the entire trend of training\. The major difference to the setting of[Zhang et al\. \(2021\)](https://arxiv.org/html/2608.28948#bib.bib228)is the use of a constant learning rate, as their use of a learning rate schedule might conflate the results\.

For the VGG case, we follow our previous hyperparameters, except we leave out weight decay and learning rate scheduling, instead using a constant learning rate of 0\.01\.

For the LSTM case, we follow our previous hyperparameters, and extend the training budget to 200 epochs allow for the random label setting to train longer\. In this case, our network does not have sufficient capacity to memorize the data completely\.

#### B\.2\.9Magnitude Pruning Experiments

We use the same VGG setup as described previously\. In this case we train until the end, then compute a global magnitude mask\. To do this we flatten all linear and convolutional weights into a single vector, except for the last linear layer, and sort by magnitude\. Then we keep the top 5% of weights globally, and reshape back to the layerwise masks\. This results in different sparsity levels for different layers, so when generating the random masks, we use the per\-layer sparsities that resulted from the global magnitude mask\.

To retrain the network, we rewind to epoch 4 \(after the point of LMC\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\), then continue training with the mask, always setting other weights and their gradients to 0\. We average all results over 3 random seeds\.

For the LSTM we follow exactly the same procedure, except our mask only reaches a level of 25% sparsity, due to larger performance degradations with higher sparsity\.

#### B\.2\.10LMC Experiments

We save 5 evenly\-spaced checkpoints in the first epoch, as well as at the end of the next 4 epochs for 10 initializations in total\. We train 3 trunks, and split 3 branches from each trunk for a total of 9 branches which we average all plots over\.

Following[Neyshabur et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib140), we compute the barrier between checkpoints as follows: givenW\(1\)​\(T\)W^\{\(1\)\}\(T\)andW\(2\)​\(T\)W^\{\(2\)\}\(T\)that were branched fromW⁡\(t\)W\(t\)we compute

b⁡\(t\)=maxα∈\[0,1\]⁡\[ℒ⁡\(\(1−α\)​W\(1\)​\(T\)\+α​W\(2\)​\(T\)\)−\(\(1−α\)​ℒ​\(W\(1\)​\(T\)\)\+α​ℒ​\(W\(2\)​\(T\)\)\)\]b\(t\)=\\max\_\{\\alpha\\in\[0,1\]\}\\left\[\\mathcal\{L\}\(\(1\-\\alpha\)W^\{\(1\)\}\(T\)\+\\alpha W^\{\(2\)\}\(T\)\)\-\(\(1\-\\alpha\)\\mathcal\{L\}\(W^\{\(1\)\}\(T\)\)\+\\alpha\\mathcal\{L\}\(W^\{\(2\)\}\(T\)\)\)\\right\]\(B\.1\)when this quantity is 0, we consider the checkpoints to exhibit LMC\.

We recompute batch normalization parameters after interpolating for VGG\-16, and group normalization parameters for the UNet, as these do not necessarily interpolate well\([Frankle et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib52)\)\. We also compute singular vector agreement for the same parameter between either branch endpoint\.

To plot the singular vector \(dis\)agreement and LMC between different modes, we make 11 evenly spaced measurements interpolating between branch endpoints that had the same split epoch, and the same branch seed, but different trunk initializations\.

#### B\.2\.11Perturbed LMC Experiments

We perturb all weightsWWafter the point of dynamics stability where we expect to see LMC at the end of training \(epoch 4 is sufficiently late in all cases\) using randomly sampled normal perturbationsϵ∼𝒩⁡\(0,I\)\\epsilon\\sim\\mathcal\{N\}\(0,I\)with∥ϵ∥=η​∥W∥\\lVert\\epsilon\\rVert=\\eta\\lVert W\\rVertwhereη∈\{0\.0,0\.1,0\.25,0\.5,1\.0,2\.5\}\\eta\\in\\\{0\.0,0\.1,0\.25,0\.5,1\.0,2\.5\\\}\. We do not perturb the output layer, as this has a very substantial effect on the optimization\. We also do not perturb the input layer for the Transformer as it is too computationally expensive for our resources\.

### B\.3Limitations

There are a few key limitations to our study\. As mentioned, we lack the computational resources to run more than 3 random seeds per experiment, though we do find error bars to be quite tight in general \(except for the generalization epoch in the grokking experiments\)\. In addition, as discussed we ignore 1D parameters like biases and normalization in the neural networks, which may play additional roles\. Due to computational constraints we do not consider alignment of layers across residual connections as this quickly becomes combinatorial in depth, thus there may be other interesting interactions that we do not observe\. Finally, due to computational constraints we are unable to investigate full results on larger models than the 12 layer Transformer, which may have different behavior, but the results on the Pythia suite in Appendix[B\.2\.5](https://arxiv.org/html/2608.28948#A2.S2.SS5)are encouraging\.

### B\.4Additional Experiments

#### B\.4\.1Effect of initialization scale

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.01-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.05-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.1-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.5-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_1.0-val-err.png)

\(a\)Val\. Err
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.01-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.05-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.1-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.5-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_1.0-model.layers.1.weight_sv.png)

\(b\)SVs
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.01-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.05-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.1-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.5-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_1.0-eff_rank.png)

\(c\)Eff\. Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.01-align_diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.05-align_diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.1-align_diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.5-diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_1.0-align_diag.png)

\(d\)Alignment
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.01-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.05-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.1-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_0.5-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/initscale/is_1.0-align_score.png)

\(e\)Align\. Score

Figure B\.5:Varying initialization scale affects the dynamics of an MLP trained on CIFAR10\. From top to bottom we use default PyTorch initialization multiplied by the constants\{0\.001,0\.01,0\.05,0\.1,0\.5,1\.0\}\\\{0\.001,0\.01,0\.05,0\.1,0\.5,1\.0\\\}\. Notably, alignment occurs slightly more strongly with smaller initialization, as predicted by[Woodworth et al\. \(2020\)](https://arxiv.org/html/2608.28948#bib.bib214)for deep linear models\.In Figure[B\.5](https://arxiv.org/html/2608.28948#A2.F5)we explore the effect of initialization scale\. Prior work\([Woodworth et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib214)\)showed that in deep linear systems the choice of initialization modulated the effect of incremental learning of ranks, with smaller initialization increasing this effect\. We observe that alignment is slightly stronger with smaller initialization, mirroring the theory in deep linear networks\.

#### B\.4\.2Effect of learning rate

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.001-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.1-val-err.png)

\(a\)Val\. Err
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.001-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.1-model.layers.1.weight_sv.png)

\(b\)SVs
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.001-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.1-eff_rank.png)

\(c\)Eff\. Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.001-align_diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-align_diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.1-align_diag.png)

\(d\)Alignment
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.001-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.1-align_score.png)

\(e\)Align\. Score

Figure B\.6:The effect of varying learning rate on an MLP trained on CIFAR10\. From top to bottom we use learning rates\{0\.001,0\.01,0\.1\}\\\{0\.001,0\.01,0\.1\\\}\. Notably, alignment increases with larger learning rate, agreeing with the deep linear setting in[Ghosh et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib58)\.In Figure[B\.6](https://arxiv.org/html/2608.28948#A2.F6)we explore the effect of learning rate on dynamics for an MLP trained on CIFAR10\.[Ghosh et al\. \(2025\)](https://arxiv.org/html/2608.28948#bib.bib58)proved that balancedness may develop in deep linear systems when the learning rate is sufficiently large\. We see here that the trend is mixed: learning rate 0\.01 shows the strongest balancedness, and going above or below has a weaker effect\.

#### B\.4\.3Individual seed plots for grokking

For clarity, in Figure[B\.7](https://arxiv.org/html/2608.28948#A2.F7)we show an average of 3 seeds of grokking modular addition with Transformers \(see Figure[4\.2](https://arxiv.org/html/2608.28948#Ch4.F2)\)\. The transition to low error and low rank coincide\.

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-error.png)\(a\)Error
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-sv-model.transformer_encoder.layers.0.self_attn.in_proj_weight_sv.png)\(b\)SV Evolution
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-sv-eff_rank.png)\(c\)Effective Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/grokking/lr_0.001-align-model.encoder.weight_model.transformer_encoder.layers.0.self_attn.in_proj_weight_align-diag.png)\(d\)Alignment

Figure B\.7:Plots for averages of 3 seeds of grokking modular addition with Transformers \(Figure[4\.2](https://arxiv.org/html/2608.28948#Ch4.F2)\)\. We see a stark transition to low\-rank when validation error decreases\.
#### B\.4\.4Full alignment matrix evolution

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/align_evolution/imgclass_mlp_mnist/step_0.png)\(a\)Init
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/align_evolution/imgclass_mlp_mnist/step_25.png)\(b\)Ep\. 2500
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/align_evolution/imgclass_mlp_mnist/step_50.png)\(c\)Ep\. 5000
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/align_evolution/imgclass_mlp_mnist/step_55.png)\(d\)Ep\. 5500
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/align_evolution/imgclass_mlp_mnist/step_60.png)\(e\)Ep\. 6000
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/align_evolution/imgclass_mlp_mnist/step_70.png)\(f\)Ep\. 7000
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/align_evolution/imgclass_mlp_mnist/step_80.png)\(g\)Ep\. 8000

Figure B\.8:Alignment matrix \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) over time\. We see that the majority of signal is concentrated in the diagonal, an example of why we consider the diagonal in the rest of the work\.We previously plotted only the diagonal of the alignment matrix \(Eqn\.[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)\) instead of considering off\-diagonal elements\. In Figure[B\.8](https://arxiv.org/html/2608.28948#A2.F8)we provide the evolution of the entire alignment matrix between two layers throughout training for an MLP on CIFAR10 so as to explain why: anecdotally we did not observe much signal off\-diagonal, hence the focus on the upper diagonal in Eqns[4\.3](https://arxiv.org/html/2608.28948#Ch4.E3)and[4\.4](https://arxiv.org/html/2608.28948#Ch4.E4)\.

#### B\.4\.5Image classification

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/image_classification/lenet/val-err.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/image_classification/vgg/val-err.png)

\(a\)Val\. Err
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-model.layers.1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/image_classification/lenet/model.fc1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-sv-model.layers.11.conv.weight_sv.png)

\(b\)SVs
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/image_classification/lenet/eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-sv-eff_rank.png)

\(c\)Eff\. Rank
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-align_diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/image_classification/lenet/diag.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-align-model.layers.11.conv.weight_model.layers.12.conv.weight_align-diag.png)

\(d\)Alignment
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/learning_rate/lr_0.01-align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/image_classification/lenet/align_score.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/lr_0.1-align-align_score.png)

\(e\)Align\. Score

Figure B\.9:Different architectures all trained on CIFAR10\. From top to bottom, MLP, LeNet\-5 and VGG\-16\. We see roughly consistent spectral dynamics across architectures within the same task\.In Figure[B\.9](https://arxiv.org/html/2608.28948#A2.F9)we provide the behavior for all networks tested in CIFAR10 side\-by\-side to facilitate comparison\. We see across architectures that there is qualitative agreement in the trend toward rank minimization\.

#### B\.4\.6

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.0-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/wd_0_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.001-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/wd_0.001_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.01-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/wd_0.01_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/weight_decay_0.1-sv-model.layers.11.conv.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/vgg/wd_0.1_abs_eff_rank.png)

\(a\)VGG
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.0-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/wd_0_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_0.1-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/wd_0.1_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_1.0-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/wd_1.0_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/weight_decay_10.0-sv-model.conv3.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/unet/wd_10.0_abs_eff_rank.png)

\(b\)UNet
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.0-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/wd_0_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_0.1-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/wd_0.1_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_1.0-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/wd_1.0_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/weight_decay_10.0-sv-model.lstm.weight_ih_l2_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/lstm/wd_10.0_abs_eff_rank.png)

\(c\)LSTM
![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.0-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/wd_0_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_0.1-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/wd_0.1_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_1.0-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/wd_1_abs_eff_rank.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/weight_decay_10.0-sv-model.transformer_encoder.layers.7.linear1.weight_sv.png)

![Refer to caption](https://arxiv.org/html/2608.28948v1/spectral_dynamics/figs/tfmr/wd_10_abs_eff_rank.png)

\(d\)Transformer

Figure B\.10:Duplicate of Figure[4\.6](https://arxiv.org/html/2608.28948#Ch4.F6)with shared color scaling between settings\.

### B\.5Compute Resources

All experiments are performed on an internal cluster with on the order of 100 NVIDIA RTX 2080 Ti GPUs or newer\. All experiments run on a single GPU in less than 8 hours, though it is extremely helpful to parallelize across machines\. We estimate that end\-to\-end it might take a few days on these resources to rerun all of the experiments in this chapter\. Additionally, the storage requirements for all of the checkpoints will take on the order of 10 terabytes\.

### B\.6Code Sources

We use PyTorch\([Paszke et al\., 2019](https://arxiv.org/html/2608.28948#bib.bib150)\)and NumPy\([Harris et al\., 2020](https://arxiv.org/html/2608.28948#bib.bib66)\)for all experiments and Weights & Biases\([Biewald, 2020](https://arxiv.org/html/2608.28948#bib.bib18)\)for experiment tracking\. We make plots with Matplotlib\([Hunter, 2007](https://arxiv.org/html/2608.28948#bib.bib78)\)and Seaborn\([Waskom, 2021](https://arxiv.org/html/2608.28948#bib.bib212)\)\. We also use HuggingFace Datasets\([Lhoest et al\., 2021](https://arxiv.org/html/2608.28948#bib.bib111)\)for Wikitext\-103\([Merity et al\., 2016](https://arxiv.org/html/2608.28948#bib.bib130)\)\.

## Appendix CDiverse Feature Learning

For all results we train 5 seeds to get an estimate of the error\.

### C\.1PGD Experiments with Image Classification

We train simple 4\-layer MLPs with hidden dimension of 256 on the MNIST\-CIFAR binary classification task\. We use Adam with a tuned learning rate of 0\.005 and a batch size of 128 without regularization\. We train for 50 rounds of PGD or until training stops \(as no ranks are left\)\. We apply PGD to all layers except the last due to the low rank of the last layer\. Each round of PGD projects off of 3 additional ranks\. We do not apply PGD to biases as when ablating this we found no difference in the results\.

### C\.2CKA Experiments with Image Classification

For all of our PGD and CKA experiments with image classification from ImageNet to CIFAR100 we use ResNet\-18 with the Adam optimizer, a learning rate of 3e\-4, batch size of 512 and weight decay of 1e\-4\. For speed of experimentation we restrict to 10 of the 1000 classes in ImageNet\. For CKA and PGD jobs we initially start with PGD and CKA at all layers, then sweep this choice over different subsets of blocks and layers\. In the case of PGD instead of an absolute number of ranks, we project off 5% of the probability mass of the singular values at each round\. We do this as ResNet\-18 has very different layer dimensions in different places\. For CKA, the final best configuration corresponds to applying CKA after projection layers back into the residual stream \(the second convolutional models, \.conv2, in each block\)\. We also sweep the loss coefficientγ\\gammaon a log grid from 0\.1 to 1000, landing on an optimal choice of 10\.

In each round, we train 50 epochs, and we train for 10 rounds\. In the case of the larger model, we increase the width by 4x in every dimension, following[Zhang and Bottou \(2023\)](https://arxiv.org/html/2608.28948#bib.bib230), which is more than 10 times the parameters and compute\.

### C\.3Ensembling in Language Modeling

For all experiments with language models, we train a simple Transformer[Vaswani et al\. \(2017\)](https://arxiv.org/html/2608.28948#bib.bib204)language model with 8 layers, 512 hidden dimension, 8 heads, and the GPTNeoXTokenizer\([Black et al\., 2022](https://arxiv.org/html/2608.28948#bib.bib19)\)on BabiStories\([Zhang et al\., 2025](https://arxiv.org/html/2608.28948#bib.bib231)\)\. We train 6 rounds for 10 epochs each, with a batch size of 128, a sequence length of 128 and a warmup stable decay\([Hu et al\., 2024](https://arxiv.org/html/2608.28948#bib.bib74)\)learning rate schedule\. We compare ensembles to a model 3x as wide, which due to embeddings is a similar number of parameters but more compute\.

For ensembling evaluations, except where otherwise noted, we average the probability distributions of the models before measuring loss\.

We provide specific details for tuning the different diversity methods below\.

#### C\.3\.1CKA

For CKA, we experiment with which layers to place the loss on, from attention only, mlp only, early blocks, late blocks, middle blocks, and all layers\. We also tune the coefficient on a log\-spaced grid from 0\.1 to 30\.0 where it diverges\. The best setting corresponds to only the mlp layers, which may make sense as this is the final output of each Transformer block, similar to the final convolution layers in residual blocks in the image classification experiments\.

#### C\.3\.2NCL

For NCL experiments, we tune the additionalγ\\gammacoefficient on a log spaced grid from 0\.01 to 10, experiment with whether to isolate NCL only to the correct class, or to the top\-k values tuned from 1 to 5 to 10\. For the top\-p version of the experiments, we similarly tuneγ\\gammafrom 0\.01 to 10, while considering a top\-p threshold in\{0\.05,0\.1,0\.25,0\.5,0\.75,0\.9\}\\\{0\.05,0\.1,0\.25,0\.5,0\.75,0\.9\\\}\.

#### C\.3\.3Focal weighting

For focal weighting experiments, we tune the weighting coefficient from 0\.01 to 10 on a log\-spaced grid\. We also tune the top\-p version similarly to the case of NCL with a value in\{0\.05,0\.1,0\.25,0\.5,0\.75,0\.9\}\\\{0\.05,0\.1,0\.25,0\.5,0\.75,0\.9\\\}\.

#### C\.3\.4Entropy weighting

For entropy weighting experiments, we proceed with the same tuning process as for focal weighting\.

### C\.4Ensemble Weighting

For experiments on ensemble weighting, we run evaluations with a temperature of 0\.01 to 100 on a log\-spaced grid, and consider interpolating with the uniform distribution with a coefficientα∈\{0\.2,0\.5,1\.0\}\\alpha\\in\\\{0\.2,0\.5,1\.0\\\}\.