Muon$^p$: Muon with Fractional Spectral Powers

# Muonp: Muon with Fractional Spectral Powers
Source: [https://arxiv.org/html/2606.13867](https://arxiv.org/html/2606.13867)
###### Abstract

Muon is an increasingly widely used optimizer that replaces a gradientG=U​S​V⊤G=USV^\{\\top\}with its polar factorU​V⊤UV^\{\\top\}, thereby flattening the singular spectrum\. However, full flattening discards singular\-value information that may matter for adaptation\. We introduce Muonp, a Muon\-style optimizer that instead uses fractional spectral\-power updatesU​Sp​V⊤US^\{p\}V^\{\\top\}for rationalp∈\(0,1\)p\\in\(0,1\), interpolating between Muon and gradient descent\. To make it practical, we prove that fractional spectral powers cannot be computed by any fixed univariate polynomial iteration, and furthermore derive low\-degree odd bivariate recurrences that approximateU​Sp​V⊤US^\{p\}V^\{\\top\}using only matrix multiplications, preserving Muon’s matrix\-multiplication\-only structure and compute complexity\. We show that Muonpmaximizes the linear improvement in loss under the Schattenqq\-norm forq=1\+1pq=1\+\\frac\{1\}\{p\}\. Empirically, Muonpis especially effective for finetuning: on billion\-scale models, Muonpimproves validation perplexity and downstream task performance\. We further analyze when Muonpis less suitable, through the lens of spectral geometry\. Our results reveal important insights on when preserving the singular spectrum can bring significant gains, and introduce a principled way to achieve them\. Our code is publicly available at[https://github\.com/princeton\-pli/muon\-p](https://github.com/princeton-pli/muon-p)\.

## 1Introduction

Optimization algorithms play a central role in shaping how neural networks learn, influencing both the efficiency of training and the quality of the resulting model\[[23](https://arxiv.org/html/2606.13867#bib.bib23),[36](https://arxiv.org/html/2606.13867#bib.bib36),[22](https://arxiv.org/html/2606.13867#bib.bib22)\]\. Muon has emerged as a promising optimizer for deep networks because its spectrum\-flattening updates promote more balanced learning across singular directions, prevent dominant modes from monopolizing training, and have been shown to scale effectively in LLM pretraining\[[22](https://arxiv.org/html/2606.13867#bib.bib22),[32](https://arxiv.org/html/2606.13867#bib.bib32),[54](https://arxiv.org/html/2606.13867#bib.bib54),[44](https://arxiv.org/html/2606.13867#bib.bib44),[51](https://arxiv.org/html/2606.13867#bib.bib51)\]\.

Muon achieves this by replacing a gradientG=U​S​V⊤G=USV^\{\\top\}with its polar factorU​V⊤UV^\{\\top\}, which flattens the singular spectrum and makes the update more uniform across singular directions\[[22](https://arxiv.org/html/2606.13867#bib.bib22)\]\. But full flattening also discards singular\-value information, suggesting a natural question: can we preserve Muon’s matrix\-level spectral benefits while retaining some magnitude structure that matters for adaptation?

We answer this question withMuonp\\mathrm\{Muon\}^\{p\}, a Muon\-style optimizer that updates withU​Sp​V⊤US^\{p\}V^\{\\top\}for rationalp∈\(0,1\)p\\in\(0,1\)\. This family admits a clean geometric interpretation:U​Sp​V⊤US^\{p\}V^\{\\top\}is thesteepest\-descent direction under the Schatten\-qqnormwithq=1\+1/pq=1\+1/p\(Theorem[2\.1](https://arxiv.org/html/2606.13867#S2.Thmtheorem1)\)\. We identify and resolve key challenges to derive Muonpand make its implementation practical\. Unlike Muon’s polar factor, fractional spectral powers cannot be obtained by iterating a fixed univariate polynomial; arbitrary rational powers require a recurrence that retains the original matrix \(Lemma[2\.2](https://arxiv.org/html/2606.13867#S2.Thmtheorem2)\)\. We therefore derive low\-degree odd bivariate recurrences that preserve Muon’s matrix\-multiplication\-only structure \(Theorem[2\.3](https://arxiv.org/html/2606.13867#S2.Thmtheorem3)\), yielding practical updates that are a one\-line change to the Muon implementation \(Algorithm[1](https://arxiv.org/html/2606.13867#alg1)\)\.

While prior work\[[42](https://arxiv.org/html/2606.13867#bib.bib42)\]has proposed fractional power for specificppvalues, we introduce a systematic formulation from first principles, and explicitly derive Muonpforanyfractional powerpp\.

Empirically, Muonpis most beneficial in finetuning\. On a variety of models, Muonpimproves math and code reasoning, validation perplexity, and downstream task performance\. We further analyze when Muonpis less suitable than Muon\. Taken together, these results position Muonpas a principled generalization of Muon, while showing that the right spectral geometry is regime\-dependent\.

##### Contributions\.

Our key contributions are:

- •We introduce Muonp, a generalized Muon\-style optimizer that computes fractional spectral\-power updatesU​Sp​V⊤US^\{p\}V^\{\\top\}foranyrationalp∈\(0,1\)p\\in\(0,1\)without explicit SVD\.
- •We show that no fixed one\-variable polynomial iteration can compute these fractional powers, motivating a novel bivariate polynomial recurrence\.
- •Muonpexhibits finetuning gains over Muon across math reasoning and coding, perplexity, and downstream task performance\.
- •We examine the strengths and weaknesses of Muonpthrough the lenses of spectral learning\-rate adaptation, curriculum, and effective\-rank dynamics\.

## 2Muonp: Muon with fractional spectral powers

##### Motivation:

A neural network’s optimizer computes, given the gradientGGof the loss function, an update to the weights\. The simplest optimizer chooses a weight update linearly proportional toGG\. The AdamW optimizer updates each weight by a running average of the gradient in that weight and divides by a running average of the gradient squared\. Muon takes advantage of the fact that both the gradient matrix and the weights can be interpreted as matrices\. IfG=U​S​V⊤G=USV^\{\\top\}is the singular value decomposition of the gradient matrixGG, then Muon updates the weights by addingU​V⊤UV^\{\\top\}, so that weights update much more in the direction of small singular values, and less in the direction of large singular values, compared to the simplest optimizer\.

For some applications, we argue that Muon is too extreme: We should not completely eliminate the information contained in the singular values, and should instead split the difference between Muon and the simplest optimizer\. Muonpachieves this by updating the weights by addingU​Sp​V⊤US^\{p\}V^\{\\top\}for a chosenppbetween0and11\.

##### Muonpmaximizes loss improvement under Schattenqq\-norm\.

One theoretical justification for Muonpcomes from norm\-based steepest descent: The Muon update rule maximizes the linear improvement in loss for a given size of the weight update, with the size measured by the operator norm\[[6](https://arxiv.org/html/2606.13867#bib.bib6)\]\. Muonpmaximizes the linear improvement in loss for a given size of the weight update, with the size measured by Schattenqq\-norm forq=1\+1pq=1\+\\frac\{1\}\{p\}\. Theorem[2\.1](https://arxiv.org/html/2606.13867#S2.Thmtheorem1)states this formally, with proof in Appendix[A\.2](https://arxiv.org/html/2606.13867#A1.SS2)\. The use of Schatten norms for steepest descent was also considered in\[[10](https://arxiv.org/html/2606.13867#bib.bib10)\]and\[[7](https://arxiv.org/html/2606.13867#bib.bib7)\]\.

###### Theorem 2\.1\.

For any real numberp∈\(0,1\)p\\in\(0,1\)letq=1\+1/pq=1\+1/p\. LetGGbe ann×nn\\times nmatrix with singular value decompositionU​S​V⊤USV^\{\\top\}\. Let⟨,⟩\\langle,\\rangledenote the entrywise dot product of matrices and let\|\|⋅\|\|q\|\|\\cdot\|\|\_\{q\}denote the Schattenqq\-norm\. Then the maximum value of⟨G,M⟩\\langle G,M\\rangleamongn×nn\\times nmatricesMMwith‖M‖q\|\|M\|\|\_\{q\}bounded is attained by a scalar multiple ofU​Sp​V⊤US^\{p\}V^\{\\top\}\.

##### The computational problem:

Muonpconsists of two components: The choice of the update ruleU​Sp​V⊤US^\{p\}V^\{\\top\}, and an algorithm to approximateU​Sp​V⊤US^\{p\}V^\{\\top\}\. To make Muonpefficient, we require a fast algorithm to computeU​Sp​V⊤US^\{p\}V^\{\\top\}givenU​S​V⊤USV^\{\\top\}\. The naive algorithm to computeU​Sp​V⊤US^\{p\}V^\{\\top\}fromU​S​V⊤USV^\{\\top\}, by performing a singular value decomposition, is too slow\. The same difficulty occurs for computingU​V⊤UV^\{\\top\}in Muon\. Muon gets around this by using an iterative algorithm to rapidly approximateU​V⊤UV^\{\\top\}givenG=U​S​V⊤G=USV^\{\\top\}\. We show that a direct translation of their iterative method cannot be used to computeU​Sp​V⊤US^\{p\}V^\{\\top\}, but Muonpuses a new variant method that can computeU​Sp​V⊤US^\{p\}V^\{\\top\}for an arbitrary rational numberp∈\(0,1\)p\\in\(0,1\)\. The remainder of this section is devoted to explaining this new iterative method\.

##### Review of Muon formulation:

ForGGa matrix andG=U​S​V⊤G=USV^\{\\top\}the singular value decomposition, whereSSis a diagonal matrix with positive entries, the Muon optimizer uses a Newton\-Schulz algorithm to approximateU​V⊤UV^\{\\top\}without computing the singular value decomposition\. This algorithm is based on two observations:

1. 1\.We haveG​G⊤​G=U​S3​V⊤GG^\{\\top\}G=US^\{3\}V^\{\\top\}andG​G⊤​G​G⊤​G​G⊤​G=U​S5​V⊤GG^\{\\top\}GG^\{\\top\}GG^\{\\top\}G=US^\{5\}V^\{\\top\}\. For any odd polynomialf​\(x\)=∑i=0da2​i\+1​x2​i\+1f\(x\)=\\sum\_\{i=0\}^\{d\}a\_\{2i\+1\}x^\{2i\+1\}, these and similar identities allow one to computeU​f​\(S\)​V⊤Uf\(S\)V^\{\\top\}as∑i=0da2​i\+1​G​\(G⊤​G\)i\\sum\_\{i=0\}^\{d\}a\_\{2i\+1\}G\(G^\{\\top\}G\)^\{i\}using only matrix multiplication and not singular value decomposition\.
2. 2\.There are odd polynomialsffsuch that the iterationsf​\(x\),f​\(f​\(x\)\)f\(x\),f\(f\(x\)\),f\(f\(f\(x\)\)f\(f\(f\(x\)\)converge to11for allxxin a starting range\(0,1\]\(0,1\]\.

Thus, fixing any such odd polynomialff, after normalizingGGso that its greatest singular value is at most11, Muon iteratively applies the operationU​S​V⊤↦U​f​\(S\)​V⊤USV^\{\\top\}\\mapsto Uf\(S\)V^\{\\top\}to approximateU​V⊤UV^\{\\top\}, with the quality of the approximation improving as the number of iterations increases\. The exact polynomialffis chosen to make the time to compute a sufficiently good approximation as small as possible by balancing the number of iterations to attain convergence with the time to compute each iteration, which depends on the degree of the polynomial\.

##### Our algorithm \(introduction\):

Muonpapplies a generalization of this strategy to computeU​Sp​V⊤US^\{p\}V^\{\\top\}for an arbitrary rational numberppbetween0and11\. \(A similar method could be applied for negative exponents or exponents greater than11\.\) It is not possible to do this by iterating any operation of the formU​S​V⊤↦U​f​\(S\)​V⊤USV^\{\\top\}\\mapsto Uf\(S\)V^\{\\top\}:

###### Lemma 2\.2\(Iterating single\-variable polynomials does not compute rational powers\)\.

For any real numberp∈\(0,1\)p\\in\(0,1\)not equal to11, there does not exist any polynomial in one variableffsuch that, for all invertible matricesG=U​S​V⊤G=USV^\{\\top\}with all singular values at most11, if we letS0=SS\_\{0\}=SandSn\+1=f​\(Sn\)S\_\{n\+1\}=f\(S\_\{n\}\)for alln≥0n\\geq 0, then we havelimn→∞U​f​\(Sn\)​V⊤=U​Sp​V⊤\\lim\_\{n\\to\\infty\}Uf\(S\_\{n\}\)V^\{\\top\}=US^\{p\}V^\{\\top\}\.

For eigenvalues instead of singular values, the analogue of Lemma[2\.2](https://arxiv.org/html/2606.13867#S2.Thmtheorem2)has been observed in the numerical linear algebra literature\[[13](https://arxiv.org/html/2606.13867#bib.bib13),[16](https://arxiv.org/html/2606.13867#bib.bib16)\]\.

Instead, to converge toU​Sp​V⊤US^\{p\}V^\{\\top\}, an iterative algorithm must remember the initial matrix\. Thus, we will consider a two\-variable polynomialf​\(x,y\)f\(x,y\)\. IfG=U​S​V⊤G=USV^\{\\top\}is our initial matrix, Muonpiteratively computes the matricesGn=U​Sn​V⊤G\_\{n\}=US\_\{n\}V^\{\\top\}whereS0=SS\_\{0\}=SandSn\+1=f​\(Sn,S\)S\_\{n\+1\}=f\(S\_\{n\},S\)\. Moreover,SnS\_\{n\}is chosen so thatGnG\_\{n\}converge toU​Sp​V⊤US^\{p\}V^\{\\top\}\. This can be done forppan arbitrary*rational*number in\(0,1\)\(0,1\)\. To show this is possible, we need two facts \(whose proofs are deferred to the appendix\):

##### First key fact \(computing polynomials\):

Forf​\(x,y\)f\(x,y\)a polynomial in two variables which is odd in the sense thatf​\(−x,−y\)=−f​\(x,y\)f\(\-x,\-y\)=\-f\(x,y\), given matricesGn=U​Sn​V⊤G\_\{n\}=US\_\{n\}V^\{\\top\}andG=U​S​V⊤G=USV^\{\\top\}, we can computeU​f​\(Sn,S\)​V⊤Uf\(S\_\{n\},S\)V^\{\\top\}using matrix multiplication and transpose\. More precisely, we have the following theorem:

###### Theorem 2\.3\(Computing odd two\-variable polynomials\)\.

1. 1\.Letf​\(x,y\)f\(x,y\)be a polynomial in two variables with real coefficients which is odd\. We can writeffin the form f​\(x,y\)=∑i,j=0da2​i\+1,2​j​x2​i\+1​y2​j\+∑i,j=0da2​i,2​j\+1​x2​i​y2​j\+1f\(x,y\)=\\sum\_\{i,j=0\}^\{d\}a\_\{2i\+1,2j\}x^\{2i\+1\}y^\{2j\}\+\\sum\_\{i,j=0\}^\{d\}a\_\{2i,2j\+1\}x^\{2i\}y^\{2j\+1\}\(2\.1\)for some natural numberddand tuples of real numbers\(a2​i\+1,2​j\)i,j=0d,\(a2​i,2​j\+1\)i,j=0d\(a\_\{2i\+1,2j\}\)\_\{i,j=0\}^\{d\},\(a\_\{2i,2j\+1\}\)\_\{i,j=0\}^\{d\}\.
2. 2\.Forffa polynomial given by the formula \([2\.1](https://arxiv.org/html/2606.13867#S2.E1)\), forG=U​S​V⊤G=USV^\{\\top\}andGn=U​Sn​V⊤G\_\{n\}=US\_\{n\}V^\{\\top\}we have U​f​\(Sn,S\)​V⊤=∑i,ja2​i\+1,2​j​Gn​\(Gn⊤​Gn\)i​\(G⊤​G\)j\+∑i,ja2​i,2​j\+1​G​\(Gn⊤​Gn\)i​\(G⊤​G\)j\.Uf\(S\_\{n\},S\)V^\{\\top\}=\\sum\_\{i,j\}a\_\{2i\+1,2j\}G\_\{n\}\(G\_\{n\}^\{\\top\}G\_\{n\}\)^\{i\}\(G^\{\\top\}G\)^\{j\}\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}G\(G\_\{n\}^\{\\top\}G\_\{n\}\)^\{i\}\(G^\{\\top\}G\)^\{j\}\.

##### Second key fact \(existence of polynomials\):

There are odd polynomialsf​\(x,y\)f\(x,y\)such that, for anyy∈\(0,1\]y\\in\(0,1\], settingx0=yx\_\{0\}=yandxn\+1=f​\(xn,y\)x\_\{n\+1\}=f\(x\_\{n\},y\)for alln≥0n\\geq 0, we havelimn→∞xn=yp\\lim\_\{n\\to\\infty\}x\_\{n\}=y^\{p\}\. More precisely, we have the following theorem:

###### Theorem 2\.4\(Odd two\-variable polynomials compute rational powers\)\.

Letppbe a rational number in\(0,1\)\(0,1\)\. Then there exists an odd polynomialf​\(x,y\)f\(x,y\)such that, for anyy∈\(0,1\]y\\in\(0,1\], if we setx0=yx\_\{0\}=yandxn\+1=f​\(xn,y\)x\_\{n\+1\}=f\(x\_\{n\},y\)for alln≥0n\\geq 0, we havelimn→∞xn=yp\\lim\_\{n\\to\\infty\}x\_\{n\}=y^\{p\}\.

##### Our algorithm \(conclusion\):

For Muonp, we choose a polynomialffas in Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4)\. Given as input a matrixG=U​S​V⊤G=USV^\{\\top\}, we setS0=SS\_\{0\}=S,Sn\+1=f​\(Sn,S\)S\_\{n\+1\}=f\(S\_\{n\},S\)for alln≥0n\\geq 0, andGn=U​Sn​V⊤G\_\{n\}=US\_\{n\}V^\{\\top\}for alln≥0n\\geq 0\. We apply Theorem[2\.3](https://arxiv.org/html/2606.13867#S2.Thmtheorem3)to computeGn\+1G\_\{n\+1\}fromGnG\_\{n\}andGG\. By Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4), we havelimn→∞Sn=Sp\\lim\_\{n\\to\\infty\}S\_\{n\}=S^\{p\}and thuslimn→∞Gn=U​Sp​V⊤\\lim\_\{n\\to\\infty\}G\_\{n\}=US^\{p\}V^\{\\top\}\. Hence, stopping afterNNiterations for a fixed numberNN, we compute an approximationGNG\_\{N\}toU​Sp​V⊤US^\{p\}V^\{\\top\}\.

We obtain the original Muon algorithm as the special case of Muonpwherep=0p=0andf​\(x,y\)f\(x,y\)does not depend onyy\. As in Muon, Muonpis numerically stable when implemented in bfloat16, allowing for fast implementation on modern hardware\.

Algorithm 1Muonpgradient update forp=13p=\\frac\{1\}\{3\}\. Other values ofpp, including for Muon whenp=0p=0, can be easily accommodated by changing theline 4according to Equation \([3\.1](https://arxiv.org/html/2606.13867#S3.E1)\)\.1:Input: gradient

GG, total number of steps

NN\.

2:Initialize

G0←GG\_\{0\}\\leftarrow G
3:for

n←0n\\leftarrow 0to

N−1N\-1do

4:

Gn\+1=Gn\+c​\(G−Gn​Gn⊤​Gn\)G\_\{n\+1\}=G\_\{n\}\+c\(G\-G\_\{n\}G\_\{n\}^\{\\top\}G\_\{n\}\)
5:endfor

6:returnMuonpgradient update

GNG\_\{N\}\.

##### Key ideas in Theorem[2\.3](https://arxiv.org/html/2606.13867#S2.Thmtheorem3)and Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4)proofs\.

The proof of Theorem[2\.3](https://arxiv.org/html/2606.13867#S2.Thmtheorem3)is by applying the definitions and then canceling all unwanted factors ofU,U⊤,V,V⊤U,U^\{\\top\},V,V^\{\\top\}via the definition of orthogonal matrix\.

The proof of Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4)requires constructing a suitable polynomialff\. To do this, it is necessary to understand what criteriaffshould satisfy\. We should haveypy^\{p\}be a fixed point offfin the sense thatf​\(yp,y\)=ypf\(y^\{p\},y\)=y^\{p\}\. Moreover,ypy^\{p\}should be an attracting fixed point in that forxxclose toypy^\{p\},f​\(x,y\)f\(x,y\)should be even closer toypy^\{p\}\. Finally, we must check convergence for our fixed initial valuex=yx=y\.

Makingypy^\{p\}a fixed point while takingffodd requires taking a polynomialffin a specific algebraic form\. Makingypy^\{p\}an attracting fixed point requires a certain inequality on the derivative offf, and so we find explicitffin this algebraic form which satisfy the derivative inequality\. We prove convergence from a given initial value using a monotonicity property, and we can chooseffto satisfy the monotonicity property also\.

## 3Explicit polynomial construction

##### Polynomial construction\.

In this section, we explain how to construct an odd two\-variable polynomialf​\(x,y\)f\(x,y\)such that, settingxn\+1=f​\(xn,y\)x\_\{n\+1\}=f\(x\_\{n\},y\),xnx\_\{n\}converges toypy^\{p\}asnngoes to∞\\inftyfor suitably\-chosenx0x\_\{0\}\. Sinceppis a positive rational number, we writep=a/bp=a/bfor positive integersa,ba,b\. We give the statements of the key results in the case thataaandbbare both odd, with the proofs, and the case whenaaorbbis even, deferred to the appendix\.

To construct this polynomial, it is necessary thatya/by^\{a/b\}be a fixed point, i\.e\.f​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}\. This can be accomplished as long asfftakes the form

f​\(x,y\)=x\+\(ya−xb\)​g​\(x,y\)f\(x,y\)=x\+\(y^\{a\}\-x^\{b\}\)g\(x,y\)\(3\.1\)

for some polynomialg​\(x,y\)g\(x,y\)\. \(In practice, we will often takeggto be a constant, and to obtain convergence, it is necessary for this constant to be positive but not too large\. We choose a constant in this range by empirical performance\.\) In fact, we have the following:

###### Lemma 3\.1\(Fixed point criterion\)\.

Letaaandbbbe two coprime positive integers\. Letg​\(x,y\)g\(x,y\)be a real polynomial in two variables\. Forf​\(x,y\)f\(x,y\)given by \([3\.1](https://arxiv.org/html/2606.13867#S3.E1)\), we havef​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}for ally∈\(0,1\]y\\in\(0,1\]\.

Combining Lemma[3\.1](https://arxiv.org/html/2606.13867#S3.Thmtheorem1)with the following Lemma[3\.2](https://arxiv.org/html/2606.13867#S3.Thmtheorem2), we can choose oddffwithya/by^\{a/b\}a fixed point\.

###### Lemma 3\.2\(Oddness criterion\)\.

Whenaaandbbare odd, the polynomialf​\(x,y\)f\(x,y\)given by \([3\.1](https://arxiv.org/html/2606.13867#S3.E1)\) is odd as long asg​\(x,y\)g\(x,y\)is even\.

However, forxnx\_\{n\}to converge \(robustly\) toya/by^\{a/b\}, it is not sufficient thatya/by^\{a/b\}be a fixed point offf\. It must also be an attracting fixed point\. The criterion for the existence of an attracting fixed point, based on the derivative, is as follows\.

###### Lemma 3\.3\(Attracting fixed point criterion\)\.

Letffbe a polynomial in two variables and letaaandbbbe two positive integers\. Sayya/by^\{a/b\}is an*attracting fixed point*off​\(x,y\)f\(x,y\)iff​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}and

\|∂f∂x​\(ya/b,y\)\|<1\.\\left\|\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\\right\|<1\.
Ifya/by^\{a/b\}is an attracting fixed point off​\(x,y\)f\(x,y\)then there exists an intervalI⊂ℝI\\subset\\mathbb\{R\}containingya/by^\{a/b\}such that, for anyx0∈Ix\_\{0\}\\in I, if we setxn\+1=f​\(xn,y\)x\_\{n\+1\}=f\(x\_\{n\},y\)for alln≥0n\\geq 0, thenlimn→∞xn=ya/b\\lim\_\{n\\to\\infty\}x\_\{n\}=y^\{a/b\}\.

Using this, we have an explicit criterion on the coefficients ofg​\(x,y\)g\(x,y\)for when an odd polynomial hasya/by^\{a/b\}as an attracting fixed point\.

###### Lemma 3\.4\(Choice ofg​\(x,y\)g\(x,y\)coefficients\)\.

Letaaandbbbe two odd positive integers\. Letf​\(x,y\)f\(x,y\)be a polynomial given by \([3\.1](https://arxiv.org/html/2606.13867#S3.E1)\) in terms ofg​\(x,y\)g\(x,y\)\. If

0<g​\(ya/b,y\)<2b​ya​\(b−1\)b0<g\(y^\{a/b\},y\)<\\frac\{2\}\{by^\{\\frac\{a\(b\-1\)\}\{b\}\}\}for ally∈\(0,1\]y\\in\(0,1\]thenya/by^\{a/b\}is an attracting fixed point off​\(x,y\)f\(x,y\)for ally∈\(0,1\]y\\in\(0,1\]\.

In particular, forgga constantcc, to haveya/by^\{a/b\}an attracting fixed point for ally∈\(0,1\]y\\in\(0,1\], it suffices to havec<2bc<\\frac\{2\}\{b\}\.

As in Muon, we face a tradeoff where higher\-degree polynomials may allow faster convergence but are slower to compute\. For this reason, we chooseggto be the lowest degree possible, i\.e\.ggis a constantcc\. As examples, some polynomials which arise from our formulas in the range of interest0<a/b<10<a/b<1are as follows:

f​\(x,y\)\\displaystyle f\(x,y\)=x\+c​\(y−x3\)\\displaystyle=x\+c\(y\-x^\{3\}\)converging to​y1/3\\displaystyle\\textrm\{converging to \}y^\{1/3\}\(3\.2\)f​\(x,y\)\\displaystyle f\(x,y\)=x\+c​\(y−x5\)\\displaystyle=x\+c\(y\-x^\{5\}\)converging to​y1/5\\displaystyle\\textrm\{converging to \}y^\{1/5\}f​\(x,y\)\\displaystyle f\(x,y\)=x\+\(y2−x4\)​\(c​y\+d​x\)\\displaystyle=x\+\(y^\{2\}\-x^\{4\}\)\(cy\+dx\)converging to​y1/2\\displaystyle\\textrm\{converging to \}y^\{1/2\}f​\(x,y\)\\displaystyle f\(x,y\)=x\+c​\(y3−x5\)\\displaystyle=x\+c\(y^\{3\}\-x^\{5\}\)converging to​y3/5\\displaystyle\\textrm\{converging to \}y^\{3/5\}

Empirically, we find that choosing, in the casea=1,b=3a=1,b=3,f​\(x,y\)=x\+c​\(y−x3\)f\(x,y\)=x\+c\(y\-x^\{3\}\)forccclose to the largest value2/32/3for which Lemma[3\.4](https://arxiv.org/html/2606.13867#S3.Thmtheorem4)applies gives rapid convergence\. Appendix[A\.1\.3](https://arxiv.org/html/2606.13867#A1.SS1.SSS3)provides further details and analysis on choosing coefficients forf​\(x,y\)f\(x,y\)\.

Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4)shows that we can in fact choosef​\(x,y\)f\(x,y\)so that we have convergence toya/by^\{a/b\}from the starting valuex=yx=y\. The proof of Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4)uses a similar construction, but taking a smaller value ofcc\. This gives a monotonicity property that makes it easier to give a formal proof of convergence\. However, larger values ofccgive convergence, and in fact more rapid convergence, in practice\.

## 4Muonpimproves finetuning

### 4\.1Training

Unless stated otherwise, we train a Llama3\-1B base model\[[11](https://arxiv.org/html/2606.13867#bib.bib11)\]\. Muonpis a general framework that can approximateGpG^\{p\}for arbitrary rationalp∈\(0,1\)p\\in\(0,1\)\. Unless noted otherwise, we adoptp=13p=\\frac\{1\}\{3\}for ease of comparison, and the fact that Muon13\{\}^\{\\frac\{1\}\{3\}\}can be efficiently computed by iterating a simple cubic polynomial\. For language modeling, the data are packed into sequences of length 2048\. Hyperparameters such as learning rate and batch size are tuned for each method\. All training runs are performed on 2 to 4 H100 GPUs\. Appendix[A\.1\.1](https://arxiv.org/html/2606.13867#A1.SS1.SSS1)contains further training and tuning details\.

The polynomialf​\(x,y\)f\(x,y\)we use is given by the formula Eq[3\.1](https://arxiv.org/html/2606.13867#S3.E1)where the parameterg​\(x,y\)g\(x,y\)is either a constant or a linear polynomial, such as in Eq[3\.2](https://arxiv.org/html/2606.13867#S3.E2)\. The optimal coefficients ofg​\(x,y\)g\(x,y\)are chosen with a grid search over the range of possible values that yield convergence according to Lemma[3\.4](https://arxiv.org/html/2606.13867#S3.Thmtheorem4)\. Appendix[A\.1\.3](https://arxiv.org/html/2606.13867#A1.SS1.SSS3)contains optimal coefficient values used for different exponentspp\. We use the Muon implementation from NanoGPT Muon\[[21](https://arxiv.org/html/2606.13867#bib.bib21)\]\. As in Muon, we accumulate a momentum of gradientsBt=μ​Bt−1\+GtB\_\{t\}=\\mu B\_\{t\-1\}\+G\_\{t\}, whereB0=0B\_\{0\}=0,GGis the gradient, andμ∈\[0,1\)\\mu\\in\[0,1\)\. We use the sameμ\\muin Muonpas theμ\\mufrom NanoGPT Muon\[[21](https://arxiv.org/html/2606.13867#bib.bib21)\]\.

##### Complexity\.

LetNNbe the number of polynomial recurrence steps in Muonp,kkthe number of matrix multiplications \(generallylog2⁡\(deg​f\)\\log\_\{2\}\(\\text\{deg\}\\ f\)forffin Eq[3\.1](https://arxiv.org/html/2606.13867#S3.E1)\), and letω\\omegadenote the matrix multiplication complexity, the time complexity for both Muonpand Muon isO​\(ω​k​N\)O\(\\omega kN\)\. Hence, Muonpand Muon differ in complexity only by a constant factor, namely different numbers of matrix multiplicationskk\. As in Muon, Muonpis numerically stable in bf16\. Section[4\.2](https://arxiv.org/html/2606.13867#S4.SS2.SSS0.Px4)gives runtime comparisons in practice\.

### 4\.2Muonpperformance

##### Math reasoning and coding\.

Table[1](https://arxiv.org/html/2606.13867#S4.T1)shows that the benefits of Muonpextend to mathematical reasoning and coding: when finetuning Llama3\-1B base model on GSM8K\[[9](https://arxiv.org/html/2606.13867#bib.bib9)\], Numina\[[27](https://arxiv.org/html/2606.13867#bib.bib27)\], and OpenCodeInstruct\[[2](https://arxiv.org/html/2606.13867#bib.bib2)\], Muonpimproves pass@8 validation accuracy \(GSM8K and Numina\) and loss \(OpenCodeInstruct\)\. Figure[1](https://arxiv.org/html/2606.13867#S4.F1)visualizes the performance comparison between Muonpand Muon across learning rates on Numina\.

Table 1:Comparison of Muonp, Muon, and AdamW after finetuning either Llama3\-1B or a Muon\-pretrained model on Fineweb\.![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/numina_hyperparameter_sweep_heatmap.png)Figure 1:Muonpoutperforms Muon across learning rates\.Appendix[A\.1\.4](https://arxiv.org/html/2606.13867#A1.SS1.SSS4)compares Muonpwith additional baselines\.

##### Using Muon\-pretrained checkpoints\.

Since Llama3\-1B was pretrained using AdamW, finetuning on Muon or Muonpcreates an optimizer mismatch, which can harm finetuning performance\[[35](https://arxiv.org/html/2606.13867#bib.bib35),[43](https://arxiv.org/html/2606.13867#bib.bib43)\]\. To study this effect, we pretrained a randomly initialized Llama3\-1B model using Muon, on 9\.83 billion tokens of Fineweb\. We then finetuned it using AdamW, Muon, and Muonp\.

Table[1](https://arxiv.org/html/2606.13867#S4.T1)shows that despite matching the pretrain optimizer \(and having an advantage\), Muon still underperforms Muonpand Adam, suggesting that Muonp’s superior finetuning performance on the off\-the\-shelf pretrained Llama3\-1B model results from more fundamental reasons than optimizer mismatch\. Indeed, an analysis using exact SVD, in Section[5](https://arxiv.org/html/2606.13867#S5.SS0.SSS0.Px4), shows that pretraining inherently benefits from full\-spectrum updates, in contrast to finetuning\.

Table 2:Validation perplexity for Llama3\-1B base model finetuned on Fineweb and Pile of Law\.Section[A\.1\.5](https://arxiv.org/html/2606.13867#A1.SS1.SSS5)contains results on a larger scale with Llama3\-3B\.

##### Commonsense reasoning and other language tasks\.

Table[2](https://arxiv.org/html/2606.13867#S4.T2)shows that Muonpyields perplexity gains when finetuning Llama3\-1B on both 300 million tokens of Fineweb\[[37](https://arxiv.org/html/2606.13867#bib.bib37)\]and 625 million tokens of Pile of Law\[[15](https://arxiv.org/html/2606.13867#bib.bib15)\]\. The best hyperparameters are tuned for each method and task\. All runs use the same seed and data ordering, with optimal hyperparameters tuned for each method\.

Table[3](https://arxiv.org/html/2606.13867#S4.T3)shows that the gains from Muonpextend beyond perplexity to accuracy in a range of downstream tasks\.

Table 3:Muonpimproves upon Muon in accuracy across a variety of reasoning and language tasks\. WG: Winogrande\. OBQA: OpenBookQA\. ARC\-E: ARC\-Easy; ARC\-C: ARC\-Challenge\.
##### Runtime comparison\.

We randomly initialized a Llama3\-1B model in bf16, and benchmarked the runtime of a single ‘optimizer\.step’ call, using a single H100 GPU and identical data\. Each optimizer is allowed to warm up for 3 steps \(for ‘torch\.compile’, etc\), and measured the mean runtime of the subsequent 15 steps\.

As shown in Table[4](https://arxiv.org/html/2606.13867#S4.T4), Muonpwithp=15p=\\frac\{1\}\{5\}has a comparable runtime with Muon, which is not surprising as both use a quintic polynomial recurrence\. Muonpwithp=13p=\\frac\{1\}\{3\}uses a cubic recurrence, leading to faster runtime\.

Table 4:Optimizer step time for Muonpand Muon\.

## 5Muonpanalysis

##### Pretraining benefits from uniform full spectrum updates\.

Interestingly, Muonpperforms worse than Muon when trained directly from a randomly initialized model, i\.e\. during pretraining\. This contrasts with the results in Section[4](https://arxiv.org/html/2606.13867#S4), as shown in Figure[2](https://arxiv.org/html/2606.13867#S5.F2)for pretraining and finetuning the 135M SmolLM base model\[[5](https://arxiv.org/html/2606.13867#bib.bib5)\]\. We analyze this dichotomy below\.

![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/pretrain_vs_finetune.png)Figure 2:Pretraining vs finetuning for Muon and Muonp\.
##### Pretraining\.

During pretraining, where the objective is to discover and refine features broadly, it is beneficial to learn the data’s principal components more uniformly, as Muon does, rather than allowing a small number of dominant directions to absorb most of the update\. Indeed, recent works suggest that this isotropization especially helps associative\-memory layers and tail patterns, because it prevents high\-frequency or high\-gain directions from monopolizing the update\[[50](https://arxiv.org/html/2606.13867#bib.bib50),[53](https://arxiv.org/html/2606.13867#bib.bib53),[26](https://arxiv.org/html/2606.13867#bib.bib26)\]\.

##### Finetuning\.

In contrast, after pretraining, adaptation commonly lives in a much smaller and more structured subspace\[[1](https://arxiv.org/html/2606.13867#bib.bib1)\]\. Pretrained models have low intrinsic dimension for downstream tuning, full finetuning tends to preserve much of this pretrained spectral structure\[[34](https://arxiv.org/html/2606.13867#bib.bib34),[47](https://arxiv.org/html/2606.13867#bib.bib47),[19](https://arxiv.org/html/2606.13867#bib.bib19),[46](https://arxiv.org/html/2606.13867#bib.bib46),[31](https://arxiv.org/html/2606.13867#bib.bib31)\]; and parameter\-efficient methods that explicitly update in a low\-dimensional subspace work surprisingly well\[[18](https://arxiv.org/html/2606.13867#bib.bib18),[38](https://arxiv.org/html/2606.13867#bib.bib38),[52](https://arxiv.org/html/2606.13867#bib.bib52),[20](https://arxiv.org/html/2606.13867#bib.bib20),[30](https://arxiv.org/html/2606.13867#bib.bib30),[31](https://arxiv.org/html/2606.13867#bib.bib31)\]\. In other words, the singular values encode which residual directions are actually worth changing\. In this regime, Muon’s gradient updates withp=0p=0discards all this directional information, while using0<p<10<p<1keeps Muon’s conditioning benefit but restores some preference for the stronger singular directions\.

##### A direct empirical test with exact SVD\.

A direct way to test this hypothesis is to replace the approximate spectral powers used during pretraining with the exact ones, computed from the gradient’s SVD\. As shown in Figure[3](https://arxiv.org/html/2606.13867#S5.F3), the updateU​V⊤UV^\{\\top\}outperformsU​S1/3​V⊤US^\{1/3\}V^\{\\top\}in pretraining\. This suggests that Muonpunderperforms Muon in pretraining not because it poorly approximates spectral\-power updates, but because pretraining itself benefits from uniform updates across the spectrum\.

However, retaining partial singular value information is useful when finetuning on specific domains, as evident in Figure[3](https://arxiv.org/html/2606.13867#S5.F3)\(right figure\)\. Remarkably, this adaptation is on a checkpointpretrained using Muon, meaning that despite having an advantage in terms of optimizer consistency\[[43](https://arxiv.org/html/2606.13867#bib.bib43),[35](https://arxiv.org/html/2606.13867#bib.bib35)\], discarding spectrum information withU​S0​V⊤US^\{0\}V^\{\\top\}updates still underperforms\.

![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/svd_pretrain_finetune_loss1.png)Figure 3:Left: Updating with exact SVD shows that pretraining benefits from uniform updates across the spectrum\.Right: However, retaining singular value information helps adaptation on specific domains, in this case OpenCodeInstruct\.
##### Effects of different exponentpp\.

Figure[4](https://arxiv.org/html/2606.13867#S5.F4)shows the validation loss after finetuning SmolLM\-360M on 852 million tokens of Fineweb data with Muonpfor different values of exponentpp, indicating that the optimal exponent is neitherp=0p=0norp=1p=1, but rather somewhere in\-between\. These results also speak to the robustness of Muonp, as the validation performances for differentppdiffer by at most 0\.01\.

##### Implicit direction\-dependent learning rate adaptation\.

The spectral\-power update in Muonpcan be viewed as introducing an implicit learning rate scaling factor that varies across singular directions of the gradient\. Reweighting the singular values by a powerp\>0p\>0attenuates smaller singular directions more strongly, while leaving dominant directions relatively less affected\. In particular, the effective learning rate follows:Δ=−η​U​Sp​V⊤=−∑iηieff​σi​ui​vi⊤\\Delta=\-\\eta\\,US^\{p\}V^\{\\top\}=\-\\sum\_\{i\}\\eta\_\{i\}^\{\\mathrm\{eff\}\}\\,\\sigma\_\{i\}\\,u\_\{i\}v\_\{i\}^\{\\top\}, whereηieff=η​σip−1\\eta\_\{i\}^\{\\mathrm\{eff\}\}=\\eta\\,\\sigma\_\{i\}^\{p\-1\}\.

![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/eval_loss_by_exponent1.png)Figure 4:The optimal validation loss is neitherp=0p=0norp=1p=1, but rather somewhere in\-between\. For comparison, the Muon validation loss after training on the same data is2\.522\.52\.As the gradient in standard Muon implementations\[[21](https://arxiv.org/html/2606.13867#bib.bib21)\]is spectral normalized such thatσi≤1\\sigma\_\{i\}\\leq 1for allii,p\>0p\>0means that the effective learning rateηeff\\eta^\{\\mathrm\{eff\}\}for Muonpin the lesser singular directions is less than that for Muon, which corresponds top=0p=0\.

##### Muon→\\toMuonpcurriculum

![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/curriculum_lr_reduction_train_loss_.png)Figure 5:Eval loss vs\. step for Muon and Muon→\\rightarrowMuonpcurriculum, compared with reducing the learning rate or keeping it constant\. The loss drop in the final 500 steps highlights the effects of the curriculum\. All three runs use the exact same seed and data ordering\.To empirically test this adaptive learning\-rate interpretation, we impose an optimizer curriculum during pretraining: Muon is used for most of training, and then switched to Muonpfor the final 500 steps\. This isolates the effect of increasing the spectral power from0topp, which corresponds to reducing the effective learning rateηeff\\eta^\{\\mathrm\{eff\}\}in the smaller singular directions of the gradient\.

As shown in Figure[5](https://arxiv.org/html/2606.13867#S5.F5), the validation loss readily decreases once the spectral power increases fromp=0p=0top=13p=\\frac\{1\}\{3\}\. This happens consistently across model scales\. Our work paves way to exciting future directions to translate such curriculum settings into significant practical gains for pretraining\.

In practice, the transition from Muon to Muonpis seamless for any choice ofpp, since Muon is a special case of Muonp\. Only the polynomial approximation needs to be changed, while the rest of the optimizer state, such as gradient momentum, carries over directly\. This is a notable advantage over switching to a different optimizer family, such as AdamW, which requires maintaining different optimizer state, including both first and second moment estimates\.

##### Effective rank analysis

The interpretation above suggests that spectral\-power updates should not only change optimization performance, but also change the geometry of the learned updates\. If increasingppsuppresses smaller singular directions and concentrates more weight on dominant ones, then the optimizer should effectively operate in a lower\-dimensional subspace over training\. Effective rank provides a natural way to test this hypothesis\.

The effective rank of a matrixWWis defined aseff​\_​rank​\(W\)=exp⁡\(−∑i=1rpi​log⁡pi\)\\mathrm\{eff\\\_rank\}\(W\)=\\exp\\\!\\bigl\(\-\\sum\_\{i=1\}^\{r\}p\_\{i\}\\log p\_\{i\}\\bigr\), wherepi=σi/∑j=1rσjp\_\{i\}=\\sigma\_\{i\}/\\sum\_\{j=1\}^\{r\}\\sigma\_\{j\}andσ1,…,σr\\sigma\_\{1\},\\dots,\\sigma\_\{r\}denote the nonzero singular values ofWW\. As shown in Figure[6](https://arxiv.org/html/2606.13867#S5.F6), Muon maintains a relatively higher effective rank and decays slowly, while Muonpshows a much sharper drop, especially early on, and continues decreasing throughout training\.

![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/erank.png)Figure 6:Average effective module rank for Muon and Muonpduring training \(from scratch\)\. Muonpexhibits a significantly stronger reduction in effective rank over training compared to Muon\.

## 6Related work

##### Muon theory and extensions\.

Muon has rapidly accumulated both empirical and theoretical follow\-up\. On the empirical side, Muon has been shown to scale to LLM pretraining with appropriate update rescaling and regularization\[[33](https://arxiv.org/html/2606.13867#bib.bib33),[44](https://arxiv.org/html/2606.13867#bib.bib44)\]\. On the theory side, existing work analyzes its convergence or interprets it as steepest descent under operator/spectral norm constraints\[[28](https://arxiv.org/html/2606.13867#bib.bib28),[24](https://arxiv.org/html/2606.13867#bib.bib24),[45](https://arxiv.org/html/2606.13867#bib.bib45),[8](https://arxiv.org/html/2606.13867#bib.bib8)\]\. Task\-specific analyses further explain when spectrum flattening helps: Muon improves tail\-end associative\-memory learning and exhibits favorable scaling laws in that setting\[[53](https://arxiv.org/html/2606.13867#bib.bib53),[26](https://arxiv.org/html/2606.13867#bib.bib26)\], while imbalanced\-data studies argue that equalizing singular directions prevents dominant modes from monopolizing learning\[[50](https://arxiv.org/html/2606.13867#bib.bib50)\]\. These results resonate with our finding thatp=0p=0remains stronger in pretraining, where broad feature discovery benefits from more uniform full\-spectrum updates, whereas0<p<10<p<1is better suited to finetuning\.

The closest prior work to ours isQi et al\. \[[42](https://arxiv.org/html/2606.13867#bib.bib42)\], which studies a spectral family of Muon\-like transforms and develops SVD\-free coupled Newton–Schulz methods for selected intermediate powers\. Two main differences are that Muonpprovides an explicit construction for arbitrary matrix powerp∈\(0,1\)p\\in\(0,1\), and Muonpdoesnotuse coupled Newton Schulz iterations for matrix inverse roots, which requires float32 precision to be numerically stable\[[22](https://arxiv.org/html/2606.13867#bib.bib22)\]\. Other recent variants modify Muon’s spectral transform or robustness properties through adaptivity, heavy\-tailed corrections, or alternative whitening surrogates, including AdaMuon, NorMuon, HTMuon, ROOT, PolarGrad, and MUD\[[42](https://arxiv.org/html/2606.13867#bib.bib42),[48](https://arxiv.org/html/2606.13867#bib.bib48),[29](https://arxiv.org/html/2606.13867#bib.bib29),[41](https://arxiv.org/html/2606.13867#bib.bib41),[14](https://arxiv.org/html/2606.13867#bib.bib14),[25](https://arxiv.org/html/2606.13867#bib.bib25),[49](https://arxiv.org/html/2606.13867#bib.bib49)\]\. Muonpis complementary to these works: instead of proposing a new heuristic transform or a different whitening surrogate, it isolates the fractional spectral power itself as the control variable, proves a negative result for fixed one\-variable polynomial iterations, and gives constructive odd bivariate recurrences for arbitrary rationalp∈\(0,1\)p\\in\(0,1\)\.

##### Efficient computation of orthogonal and spectral transforms\.

A separate line of work revisits the numerical linear algebra subroutines underlying Muon\. Muon’s practical implementation relies on Newton–Schulz\-type iterations that use only matrix multiplications\[[22](https://arxiv.org/html/2606.13867#bib.bib22)\]\. Recent papers improve this subroutine through optimized polynomial coefficients \(CANS\), adaptive minimax iterations \(Polar Express\), iteration\-free consolidations \(IFNSO\), and general adaptive acceleration frameworks \(PRISM\)\[[12](https://arxiv.org/html/2606.13867#bib.bib12),[3](https://arxiv.org/html/2606.13867#bib.bib3),[17](https://arxiv.org/html/2606.13867#bib.bib17),[55](https://arxiv.org/html/2606.13867#bib.bib55)\]\. Classical high\-accuracy polar\-decomposition methods such as QDWH and Zolo\-PD offer a useful point of comparison, but they typically depend on QR or rational\-function machinery rather than the matrix\-multiplication\-only structure favored in deep learning workloads\[[40](https://arxiv.org/html/2606.13867#bib.bib40),[39](https://arxiv.org/html/2606.13867#bib.bib39)\]\. Muonpaddresses an orthogonal computational problem: not how to compute the polar factorU​V⊤UV^\{\\top\}faster, but how to compute fractional transformsU​Sp​V⊤US^\{p\}V^\{\\top\}without explicit SVD while retaining Muon’s accelerator\-friendly structure\.

## 7Conclusion

In this work, we introduced Muonp, a Muon\-style optimizer that realizes fractional spectral\-power updates through an efficient SVD\-free bivariate polynomial recurrence\. Our results suggest that Muonpis effective for finetuning, improving math reasoning and coding, perplexity, and downstream task performance, while also indicating that the optimal spectral geometry differs between pretraining and finetuning\. Exciting future directions include automatically selecting the bestppfor a given task, incorporating Muonpinto a pretraining curriculum, and developing a deeper understanding of how spectral geometry shapes learning\-rate adaptation and downstream performance\.

## References

- Aghajanyan et al\. \[2020\]Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta\.Intrinsic dimensionality explains the effectiveness of language model fine\-tuning\.*arXiv*, 2020\.doi:10\.48550/arxiv\.2012\.13255\.
- Ahmad et al\. \[2025\]Wasi Uddin Ahmad, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Vahid Noroozi, Somshubra Majumdar, and Boris Ginsburg\.Opencodeinstruct: A large\-scale instruction tuning dataset for code llms\.2025\.URL[https://arxiv\.org/abs/2504\.04030](https://arxiv.org/abs/2504.04030)\.
- Amsel et al\. \[2025a\]Noah Amsel, David Persson, Christopher Musco, and Robert M\. Gower\.The polar express: Optimal matrix sign methods and their application to the muon algorithm\.*arXiv*, 2025a\.doi:10\.48550/arXiv\.2505\.16932\.URL[https://arxiv\.org/abs/2505\.16932](https://arxiv.org/abs/2505.16932)\.
- Amsel et al\. \[2025b\]Noah Amsel, David Persson, Christopher Musco, and Robert M\. Gower\.The polar express: Optimal matrix sign methods and their application to the muon algorithm\.*arXiv*, 2025b\.doi:10\.48550/arxiv\.2505\.16932\.
- Bakouch et al\. \[2025\]Elie Bakouch, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Lewis Tunstall, Carlos Miguel Patiño, Edward Beeching, Aymeric Roucher, Aksel Joonas Reedi, Quentin Gallouédec, Kashif Rasul, Nathan Habib, Clémentine Fourrier, Hynek Kydlicek, Guilherme Penedo, Hugo Larcher, Mathieu Morlon, Vaibhav Srivastav, Joshua Lochner, Xuan\-Son Nguyen, Colin Raffel, Leandro von Werra, and Thomas Wolf\.SmolLM3: smol, multilingual, long\-context reasoner\.[https://huggingface\.co/blog/smollm3](https://huggingface.co/blog/smollm3), 2025\.
- Bernstein \[2025\]Jeremy Bernstein\.Deriving muon, 2025\.URL[https://jeremybernste\.in/writing/deriving\-muon](https://jeremybernste.in/writing/deriving-muon)\.
- Cesista \[2025\]Franz Louis Cesista\.Muon and a selective survey on steepest descent in Riemannian and non\-Riemannian manifolds, 2025\.[https://leloykun\.github\.io/ponder/steepest\-descent\-non\-riemannian/](https://leloykun.github.io/ponder/steepest-descent-non-riemannian/)\.
- Chen et al\. \[2025\]Lizhang Chen, Jonathan Li, and Qiang Liu\.Muon optimizes under spectral norm constraints\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2506\.15054\.URL[https://arxiv\.org/abs/2506\.15054](https://arxiv.org/abs/2506.15054)\.
- Cobbe et al\. \[2021\]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*, 2021\.
- Fan et al\. \[2025\]Chen Fan, Mark Schmidt, and Christos Thrampoulidis\.Implicit bias of spectral descent and Muon on multiclass separable data, 2025\.URL[https://arxiv\.org/abs/2502\.04664](https://arxiv.org/abs/2502.04664)\.
- Grattafiori et al\. \[2024\]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, and Baptiste Roziere\.The llama 3 herd of models\.*arXiv*, 2024\.doi:10\.48550/arxiv\.2407\.21783\.
- Grishina et al\. \[2025\]Ekaterina Grishina, Matvey Smirnov, and Maxim Rakhuba\.Accelerating newton–schulz iteration for orthogonalization via chebyshev\-type polynomials\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2506\.10935\.URL[https://arxiv\.org/abs/2506\.10935](https://arxiv.org/abs/2506.10935)\.
- Guo and Higham \[2006\]Chun‐Hua Guo and Nicholas J\. Higham\.A Schur–Newton method for the matrixppth root and its inverse\.*SIAM Journal on Matrix Analysis and Applications*, 28\(3\):788–804, January 2006\.ISSN 1095\-7162\.doi:10\.1137/050643374\.URL[http://dx\.doi\.org/10\.1137/050643374](http://dx.doi.org/10.1137/050643374)\.
- He et al\. \[2025\]Wei He, Kai Han, Hang Zhou, Hanting Chen, Zhicheng Liu, Xinghao Chen, and Yunhe Wang\.Root: Robust orthogonalized optimizer for neural network training\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2511\.20626\.URL[https://arxiv\.org/abs/2511\.20626](https://arxiv.org/abs/2511.20626)\.
- Henderson et al\. \[2022\]Peter Henderson, Mark S\. Krass, Lucia Zheng, Neel Guha, Christopher D\. Manning, Dan Jurafsky, and Daniel E\. Ho\.Pile of law: Learning responsible data filtering from the law and a 256gb open\-source legal dataset, 2022\.URL[https://arxiv\.org/abs/2207\.00220](https://arxiv.org/abs/2207.00220)\.
- Higham \[2008\]Nicholas J\. Higham\.*Functions of Matrices: Theory and Computation*\.Society for Industrial and Applied Mathematics, January 2008\.ISBN 9780898717778\.doi:10\.1137/1\.9780898717778\.URL[http://dx\.doi\.org/10\.1137/1\.9780898717778](http://dx.doi.org/10.1137/1.9780898717778)\.
- Hu et al\. \[2026\]Chen Hu, Qianxi Zhao, Xiaochen Yuan, Hong Zhang, Ding Yuan, Yanbin Wu, and Xiying Li\.Ifnso: Iteration\-free newton–schulz orthogonalization\.*arXiv*, 2026\.doi:10\.48550/arXiv\.2602\.02500\.URL[https://arxiv\.org/abs/2602\.02500](https://arxiv.org/abs/2602.02500)\.
- Hu et al\. \[2021\]Edward J\. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\.Lora: Low\-rank adaptation of large language models\.*arXiv*, 2021\.doi:10\.48550/arXiv\.2106\.09685\.URL[https://arxiv\.org/abs/2106\.09685](https://arxiv.org/abs/2106.09685)\.
- Hwang et al\. \[2025\]Junseo Hwang, Wonguk Cho, and Taesup Kim\.Pica: Parameter\-efficient fine\-tuning with column space projection\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2505\.20211\.URL[https://arxiv\.org/abs/2505\.20211](https://arxiv.org/abs/2505.20211)\.
- Jiang et al\. \[2024\]Ting Jiang, Shaohan Huang, Shengyue Luo, Zihan Zhang, Haizhen Huang, Furu Wei, et al\.Mora: High\-rank updating for parameter\-efficient fine\-tuning\.*arXiv*, 2024\.doi:10\.48550/arXiv\.2405\.12130\.URL[https://arxiv\.org/abs/2405\.12130](https://arxiv.org/abs/2405.12130)\.
- Jordan et al\. \[2024a\]Keller Jordan, Jeremy Bernstein, Brendan Rappazzo, @fernbear\.bsky\.social, Boza Vlado, You Jiacheng, Franz Cesista, Braden Koszarsky, and @Grad62304977\.modded\-nanogpt: Speedrunning the nanogpt baseline, 2024a\.URL[https://github\.com/KellerJordan/modded\-nanogpt](https://github.com/KellerJordan/modded-nanogpt)\.
- Jordan et al\. \[2024b\]Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein\.Muon: An optimizer for hidden layers in neural networks, 2024b\.URL[https://kellerjordan\.github\.io/posts/muon/](https://kellerjordan.github.io/posts/muon/)\.
- Kingma and Ba \[2015\]Diederik P\. Kingma and Jimmy Ba\.Adam: A method for stochastic optimization\.In*International Conference on Learning Representations \(ICLR\)*, 2015\.doi:10\.48550/arxiv\.1412\.6980\.URL[https://arxiv\.org/abs/1412\.6980](https://arxiv.org/abs/1412.6980)\.
- Kovalev \[2025\]Dmitry Kovalev\.Understanding gradient orthogonalization for deep learning via non\-euclidean trust\-region optimization\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2503\.12645\.URL[https://arxiv\.org/abs/2503\.12645](https://arxiv.org/abs/2503.12645)\.
- Lau et al\. \[2025\]Tim Tsz\-Kit Lau, Qi Long, and Weijie Su\.Polargrad: A class of matrix\-gradient optimizers from a unifying preconditioning perspective\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2505\.21799\.URL[https://arxiv\.org/abs/2505\.21799](https://arxiv.org/abs/2505.21799)\.
- Li et al\. \[2026\]Binghui Li, Kaifei Wang, Han Zhong, Pinyan Lu, and Liwei Wang\.Muon in associative memory learning: Training dynamics and scaling laws\.*arXiv*, 2026\.doi:10\.48550/arxiv\.2602\.05725\.
- LI et al\. \[2024\]Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu\.Numinamath\.[\[https://huggingface\.co/AI\-MO/NuminaMath\-CoT\]\(https://github\.com/project\-numina/aimo\-progress\-prize/blob/main/report/numina\_dataset\.pdf\)](https://arxiv.org/html/2606.13867v1/%5Bhttps://huggingface.co/AI-MO/NuminaMath-CoT%5D(https://github.com/project-numina/aimo-progress-prize/blob/main/report/numina_dataset.pdf)), 2024\.
- Li and Hong \[2025\]Jiaxiang Li and Mingyi Hong\.A note on the convergence of muon\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2502\.02900\.URL[https://arxiv\.org/abs/2502\.02900](https://arxiv.org/abs/2502.02900)\.
- Li et al\. \[2025\]Zichong Li, Hao Kang, Zhenghao Xu, Tuo Zhao, and Weizhu Chen\.Normuon: Making muon more efficient and scalable\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2510\.05491\.URL[https://arxiv\.org/abs/2510\.05491](https://arxiv.org/abs/2510.05491)\.
- Lialin et al\. \[2023\]Vladislav Lialin, Namrata Shivagunde, Sherin Muckatira, and Anna Rumshisky\.Relora: High\-rank training through low\-rank updates\.*arXiv*, 2023\.doi:10\.48550/arXiv\.2307\.05695\.URL[https://arxiv\.org/abs/2307\.05695](https://arxiv.org/abs/2307.05695)\.
- Lingam et al\. \[2024\]Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Alex Dimakis, Eunsol Choi, Aleksandar Bojchevski, and Sujay Sanghavi\.Svft: Parameter\-efficient fine\-tuning with singular vectors\.In*Advances in Neural Information Processing Systems*, 2024\.URL[https://openreview\.net/forum?id=uS0PwIBzC0](https://openreview.net/forum?id=uS0PwIBzC0)\.
- Liu et al\. \[2025a\]Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, Yanru Chen, Huabin Zheng, Yibo Liu, Shaowei Liu, Bohong Yin, Weiran He, Han Zhu, Yuzhi Wang, Jianzhou Wang, Mengnan Dong, Zheng Zhang, Yongsheng Kang, Hao Zhang, Xinran Xu, Yutao Zhang, Yuxin Wu, Xinyu Zhou, and Zhilin Yang\.Muon is scalable for LLM training, 2025a\.URL[https://arxiv\.org/abs/2502\.16982](https://arxiv.org/abs/2502.16982)\.
- Liu et al\. \[2025b\]Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, et al\.Muon is scalable for llm training\.*arXiv*, 2025b\.doi:10\.48550/arXiv\.2502\.16982\.URL[https://arxiv\.org/abs/2502\.16982](https://arxiv.org/abs/2502.16982)\.
- Liu et al\. \[2025c\]Yong Liu, Di Fu, Yang Luo, Zirui Zhu, Minhao Cheng, Cho\-Jui Hsieh, and Yang You\.Pome: Post optimization model edit via muon\-style projection\.*arXiv*, 2025c\.doi:10\.48550/arxiv\.2510\.06627\.
- Liu et al\. \[2026\]Yuxing Liu, Jianyu Wang, and Tong Zhang\.Optimizer\-model consistency: Full finetuning with the same optimizer as pretraining forgets less\.*arXiv preprint arXiv:2605\.06654*, 2026\.doi:10\.48550/arXiv\.2605\.06654\.
- Loshchilov and Hutter \[2019\]Ilya Loshchilov and Frank Hutter\.Decoupled weight decay regularization\.In*International Conference on Learning Representations \(ICLR\)*, 2019\.doi:10\.48550/arxiv\.1711\.05101\.URL[https://openreview\.net/forum?id=Bkg6RiCqY7](https://openreview.net/forum?id=Bkg6RiCqY7)\.
- Lozhkov et al\. \[2024\]Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf\.Fineweb\-edu: the finest collection of educational content, 2024\.URL[https://huggingface\.co/datasets/HuggingFaceFW/fineweb\-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu)\.
- Meng et al\. \[2024\]Fanxu Meng, Zhaohui Wang, and Muhan Zhang\.Pissa: Principal singular values and singular vectors adaptation of large language models\.*arXiv*, 2024\.doi:10\.48550/arXiv\.2404\.02948\.URL[https://arxiv\.org/abs/2404\.02948](https://arxiv.org/abs/2404.02948)\.
- Nakatsukasa and Freund \[2016\]Yuji Nakatsukasa and Roland W\. Freund\.Computing fundamental matrix decompositions accurately via the matrix sign function in two iterations: The power of zolotarev’s functions\.*SIAM Review*, 58\(3\):461–493, 2016\.doi:10\.1137/140990334\.URL[https://doi\.org/10\.1137/140990334](https://doi.org/10.1137/140990334)\.
- Nakatsukasa et al\. \[2010\]Yuji Nakatsukasa, Zhaojun Bai, and François Gygi\.Optimizing halley’s iteration for computing the matrix polar decomposition\.*SIAM Journal on Matrix Analysis and Applications*, 31\(5\):2700–2720, 2010\.doi:10\.1137/090774999\.URL[https://doi\.org/10\.1137/090774999](https://doi.org/10.1137/090774999)\.
- Pang et al\. \[2026\]Tianyu Pang, Yujie Fang, Zihang Liu, Shenyang Deng, Lei Hsiung, Shuhua Yu, and Yaoqing Yang\.Htmuon: Improving muon via heavy\-tailed spectral correction\.*arXiv*, 2026\.doi:10\.48550/arXiv\.2603\.10067\.URL[https://arxiv\.org/abs/2603\.10067](https://arxiv.org/abs/2603.10067)\.
- Qi et al\. \[2026\]Xianbiao Qi, Marco Chen, Jiaquan Ye, Yelin He, and Rong Xiao\.Delving into muon and beyond: Deep analysis and extensions, 2026\.URL[https://arxiv\.org/abs/2602\.04669](https://arxiv.org/abs/2602.04669)\.
- Qu et al\. \[2026\]Xingyu Qu, Peigeng Huang, and Samuel Horvath\.Can muon fine\-tune adam\-pretrained models?In*International Conference on Machine Learning*, 2026\.
- Shah et al\. \[2025\]Ishaan Shah, Anthony M\. Polloreno, Karl Stratos, Philip Monk, Adarsh Chaluvaraju, et al\.Practical efficiency of muon for pretraining\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2505\.02222\.URL[https://arxiv\.org/abs/2505\.02222](https://arxiv.org/abs/2505.02222)\.
- Shen et al\. \[2025\]Wei Shen, Ruichuan Huang, Minhui Huang, Cong Shen, and Jiawei Zhang\.On the convergence analysis of muon\.*arXiv*, 2025\.doi:10\.48550/arXiv\.2505\.23737\.URL[https://arxiv\.org/abs/2505\.23737](https://arxiv.org/abs/2505.23737)\.
- Shuttleworth et al\. \[2024\]Reece Shuttleworth, Jacob Andreas, Antonio Torralba, and Pratyusha Sharma\.Lora vs full fine\-tuning: An illusion of equivalence\.*arXiv*, 2024\.doi:10\.48550/arXiv\.2410\.21228\.URL[https://arxiv\.org/abs/2410\.21228](https://arxiv.org/abs/2410.21228)\.
- Si et al\. \[2025a\]Chongjie Si, Xuankun Yang, Muqing Liu, Yadao Wang, Xiaokang Yang, Wenbo Su, Bo Zheng, and Wei Shen\.Weight spectra induced efficient model adaptation\.*arXiv*, 2025a\.doi:10\.48550/arXiv\.2505\.23099\.URL[https://arxiv\.org/abs/2505\.23099](https://arxiv.org/abs/2505.23099)\.
- Si et al\. \[2025b\]Chongjie Si, Debing Zhang, and Wei Shen\.Adamuon: Adaptive muon optimizer\.*arXiv*, 2025b\.doi:10\.48550/arXiv\.2507\.11005\.URL[https://arxiv\.org/abs/2507\.11005](https://arxiv.org/abs/2507.11005)\.
- Southworth and Thomas \[2026\]Ben S\. Southworth and Stephen Thomas\.Beyond muon: Mud \(momentum decorrelation\) for faster transformer training\.*arXiv*, 2026\.doi:10\.48550/arXiv\.2603\.17970\.URL[https://arxiv\.org/abs/2603\.17970](https://arxiv.org/abs/2603.17970)\.
- Vasudeva et al\. \[2025a\]Bhavya Vasudeva, Puneesh Deora, Yize Zhao, Vatsal Sharan, and Christos Thrampoulidis\.How muon’s spectral design benefits generalization: A study on imbalanced data\.*arXiv*, 2025a\.doi:10\.48550/arxiv\.2510\.22980\.
- Vasudeva et al\. \[2025b\]Bhavya Vasudeva, Puneesh Deora, Yize Zhao, Vatsal Sharan, and Christos Thrampoulidis\.How Muon’s spectral design benefits generalization: A study on imbalanced data, 2025b\.URL[https://arxiv\.org/abs/2510\.22980](https://arxiv.org/abs/2510.22980)\.
- Wang et al\. \[2024\]Shaowen Wang, Linxi Yu, and Jian Li\.Lora\-ga: Low\-rank adaptation with gradient approximation\.*arXiv*, 2024\.doi:10\.48550/arXiv\.2407\.05000\.URL[https://arxiv\.org/abs/2407\.05000](https://arxiv.org/abs/2407.05000)\.
- Wang et al\. \[2025a\]Shuche Wang, Fengzhuo Zhang, Jiaxiang Li, Cunxiao Du, Chao Du, Tianyu Pang, Zhuoran Yang, Mingyi Hong, and Vincent Y\. F\. Tan\.Muon outperforms adam in tail\-end associative memory learning\.*arXiv*, 2025a\.doi:10\.48550/arxiv\.2509\.26030\.
- Wang et al\. \[2025b\]Shuche Wang, Fengzhuo Zhang, Jiaxiang Li, Cunxiao Du, Chao Du, Tianyu Pang, Zhuoran Yang, Mingyi Hong, and Vincent Y\. F\. Tan\.Muon outperforms Adam in tail\-end associative memory learning, 2025b\.URL[https://arxiv\.org/abs/2509\.26030](https://arxiv.org/abs/2509.26030)\.
- Yang et al\. \[2026\]Shenghao Yang, Zhichao Wang, Oleg Balabanov, N\. Benjamin Erichson, and Michael W\. Mahoney\.Prism: Distribution\-free adaptive computation of matrix functions for accelerating neural network training\.*arXiv*, 2026\.doi:10\.48550/arXiv\.2601\.22137\.URL[https://arxiv\.org/abs/2601\.22137](https://arxiv.org/abs/2601.22137)\.

## Appendix AAppendix

We include omitted proofs, additional experimental details and results\.

### A\.1Additional Experimental Details and Results

#### A\.1\.1Further training details\.

We tune the optimal hyperparameters such as learning rate and batch size for each optimizer method\. We use a cosine learning rate scheduler\. For language modeling, Numina, and OpenCodeInstruct finetuning, the best learning rate is tuned within\[1​e−5,5​e−5,1​e−4\]\[1e\-5,5e\-5,1e\-4\], and the best batch sizes are tuned within\[128,512,1024\]\[128,512,1024\]\. In addition, we tune the number of warmup steps within\[100,200,400\]\[100,200,400\]and a ratio of0\.10\.1, and tune the minimum learning rate ratio for the cosine scheduler within\[0\.1,0\.3\]\[0\.1,0\.3\]\. For finetuning on GSM8k, due to its smaller dataset size, the best learning rate is tuned within\[2​e−5,5​e−5\]\[2e\-5,5e\-5\], and the batch size is tuned within\[128,512\]\[128,512\]\. The exact same seed is used throughout different runs to ensure the same data ordering during training\. We use the same number of iterationsN=6N=6in Algorithm[1](https://arxiv.org/html/2606.13867#alg1)and Muon throughout for fair comparison\.

Unless otherwise noted, language modeling finetuning on Llama3\-1B is trained for 1000 steps, with a global batch size of 32768 tokens\. The evaluation is done on a 1000 randomly selected test set\. GSM8K is finetuned for 600 steps \(10 epochs\), given its small dataset size, to avoid catastrophic overfitting; and Numina and OpenCodeInstruct are finetuned for 6000 steps\.

We use the “constitutes", “courtListener\_opinions", and “us\_bills", “scotus\_filings" components from the Pile of Law dataset, for a total of 625 million tokens\. The evaluation is done on the official validation set\.

#### A\.1\.2GSM8K and Numina Evaluation\.

To evaluate performance on the validation set of GSM8K and Numina, we evaluate pass@8 and temperature 0\.8, and use 32 samples to approximate an unbiased estimator\. Evaluation is done on the 1319 problems in the official test split for GSM8K, and the 100 problems in the official test split for Numina\.

#### A\.1\.3Choice of coefficients off​\(x,y\)f\(x,y\)\.

As discussed in §[4\.1](https://arxiv.org/html/2606.13867#S4.SS1), the polynomialf​\(x,y\)f\(x,y\)is taken to be the simplest form that satisfy the convergence conditions in Lemmas[3\.1](https://arxiv.org/html/2606.13867#S3.Thmtheorem1),[3\.3](https://arxiv.org/html/2606.13867#S3.Thmtheorem3), and[3\.4](https://arxiv.org/html/2606.13867#S3.Thmtheorem4)\. Once the degree and convergence criteria are satisfied, we find the optimal coefficients forf​\(x\)f\(x\)using a simple grid search within the range of convergence, on a sample of 1000 GPT\-2 gradients produced on Fineweb\. The grid search aims to minimize theℓ2\\ell\_\{2\}distance between the true matrix powerGpG^\{p\}and the Muonpapproximation for the gradientGG\. We found that the range of acceptable convergence on the gradients in practice is larger than in theory due to using a finite number of iterations\.

We found grid search to be simpler and more efficient than gradient\-descent based search methods for the optimal coefficients\. Table[5](https://arxiv.org/html/2606.13867#A1.T5)contains the optimal coefficients used for a range of exponentspp\.

Table 5:Empirical coefficientsccandddinf​\(x,y\)f\(x,y\)for a range of Muonpexponents found with grid search within the feasible range as in Lemma[3\.4](https://arxiv.org/html/2606.13867#S3.Thmtheorem4)\.The coefficients of the polynomialf​\(x,y\)f\(x,y\)in Eq[3\.1](https://arxiv.org/html/2606.13867#S3.E1), such asccin Eq[3\.2](https://arxiv.org/html/2606.13867#S3.E2), trade off attracting fixed point guarantee with convergence speed w\.r\.t\. the number of recurrence iterations: smaller values ofccmakes the fixed point attracting and guarantees convergence, but can decrease convergence speed, especially along the small singular value directions\. It is an interesting future direction to varyccfor different singular directions\.

In summary, we want the largest coefficients that still guarantee convergence for both large and small singular values\. Figure[7](https://arxiv.org/html/2606.13867#A1.F7)illustrates the convergence v\.s\. speed tradeoff for a small and a large singular value forp=13p=\\frac\{1\}\{3\}\.

![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/choice_of_c_overshooting_vs_conv_speed.png)Figure 7:Convergence v\.s\. speed tradeoff onccfor a small and a large singular value forp=13p=\\frac\{1\}\{3\}\. While a larger value ofccaccelerates convergence along small singular directions, it can prevent convergence along larger singular directions\.
#### A\.1\.4Comparison with additional baselines\.

We compare Muonpwith stronger baselines that modify Muon’s spectrum or preconditioner to improve robustness or convergence: AdaMuon, PolarGrad \(both ‘qdwh’ and ‘zolo\-pd’ backends\), and PolarExpress\[[48](https://arxiv.org/html/2606.13867#bib.bib48),[14](https://arxiv.org/html/2606.13867#bib.bib14),[25](https://arxiv.org/html/2606.13867#bib.bib25),[4](https://arxiv.org/html/2606.13867#bib.bib4)\]\.

We chose the Numina dataset, as it is a much larger scale and more complex math reasoning dataset than GSM8k, and hence offers a more robust signal\. We obtained the following results when finetuning Llama3\-1B on 393 million tokens of Numina:

Table 6:Evaluation loss by method and learning rate\.We tried our best to reproduce the baselines as faithfully as possible, and worked on ensuring baseline stability whenever needed\. E\.g\. PolarGrad with the zolo\-pd backend is only numerically stable when executed in float64 due to its Cholesky kernels, which is the precision we used\.

Although HTMuon\[[41](https://arxiv.org/html/2606.13867#bib.bib41)\]is most similar to Muonpin spirit, by applying fractional singular powersSpS^\{p\}, we could not compare with HTMuon, as it requires exact SVD, and hence belongs to a different family of algorithms and is orders of magnitude slower, especially at the 1B scale\. We also benchmarked ROOT\[[14](https://arxiv.org/html/2606.13867#bib.bib14)\], but encountered numerical instabilities\.

It is interesting that Muon outperformed these baselines aimed towards improving its robustness\. One hypothesis is that all these baselines were developed and tested in thepretrainingsetting, where having a more accurate polar factor approximation \(with singular values 1\) developed in these baselines can be beneficial\.

However, as Muon onlyapproximatesa polar decomposition, indeed Muon gradient singular values are roughly Uniform\(0\.5,1\.5\)\(0\.5,1\.5\)instead of exactly 1 \(due to its maximizing the polynomial slope at 0\), Muon performs more implicit regularization during finetuning compared to these baselines \(Indeed Muon’s updates are not truly full rank as shown by its effective rank in Figure[6](https://arxiv.org/html/2606.13867#S5.F6)\.\)

#### A\.1\.5Performance on larger scale models

We aim to study Muonpbeyond the 1B scale\. Specifically, we finetuned Llama3\-3B \(base\) on 786 million tokens of the Numina dataset\. Here we report test accuracy on the official Numina test split, the pass@8 accuracy is evaluated using temperature=0\.8=0\.8and3232generations to approximate an unbiased estimator\. As shown in[7](https://arxiv.org/html/2606.13867#A1.T7), Muonpconsistently outperforms Muon, consistent with the observations on smaller scales\.

Table 7:Pass@8 performance of Muon and Muonpacross learning rates after finetuning Llama3\-3B with either Muon or Muonpon Numina\.
#### A\.1\.6Further runtime comparison

Section[4\.1](https://arxiv.org/html/2606.13867#S4.SS1.SSS0.Px1)compares the complexity between Muonpand Muon\. Here to aim to study the runtimes in practice in terms of wallclock time\. We randomly initialized a Llama3\-1B model in bf16, and benchmarked the runtime of a single ‘optimizer\.step’ call\. We used a single H100 GPU and identical data\. We allowed each optimizer to warm up for 3 steps \(for ‘torch\.compile’, etc\), and measured the mean runtime of the subsequent 15 steps\.

As shown in Table[8](https://arxiv.org/html/2606.13867#A1.T8), Muonpwithp=15p=\\frac\{1\}\{5\}uses a quintic polynomial recurrence, as opposed to the cubic recurrence used whenp=13p=\\frac\{1\}\{3\}\. AdaMuon’s greater latency is due to its extra ‘sign\(\)‘ function and Adam\-style second\-moment EMA\.

Table 8:Optimizer step time by backend\. Values are reported as mean and standard deviation over 15 steps\.
#### A\.1\.7Further curriculum results

Figure[8](https://arxiv.org/html/2606.13867#A1.F8)shows additional Muon→\\toMuonpcurriculum results for pretraining on Fineweb\.

![Refer to caption](https://arxiv.org/html/2606.13867v1/plots/curriculum.png)Figure 8:Muon→\\toMuonpcurriculum when pretrained on Fineweb\.

### A\.2Proofs

We begin with the proofs of Lemma[2\.2](https://arxiv.org/html/2606.13867#S2.Thmtheorem2)and Theorem[2\.3](https://arxiv.org/html/2606.13867#S2.Thmtheorem3)\. We then prove the results of Section[3](https://arxiv.org/html/2606.13867#S3.SS0.SSS0.Px1), before proving Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4), which relies on these results\. Afterwards, we prove Lemma[2\.1](https://arxiv.org/html/2606.13867#S2.Thmtheorem1)\.

###### Lemma A\.1\(Lemma[2\.2](https://arxiv.org/html/2606.13867#S2.Thmtheorem2), Iterating single\-variable polynomials does not compute rational powers\)\.

For any real numberp∈\(0,1\)p\\in\(0,1\), there does not exist any polynomial in one variableffsuch that, for all invertible matricesG=U​S​V⊤G=USV^\{\\top\}with all singular values at most11, if we letS0=SS\_\{0\}=SandSn\+1=f​\(Sn\)S\_\{n\+1\}=f\(S\_\{n\}\)for alln≥0n\\geq 0, then we havelimn→∞U​f​\(Sn\)​V⊤=U​Sa/b​V⊤\\lim\_\{n\\to\\infty\}Uf\(S\_\{n\}\)V^\{\\top\}=US^\{a/b\}V^\{\\top\}\.

###### Proof of Lemma[2\.2](https://arxiv.org/html/2606.13867#S2.Thmtheorem2)\.

Fixpp, and suppose for contradiction there did exist such anff\. Sincelimn→∞U​f​\(Sn\)​V⊤=U​Sp​V⊤\\lim\_\{n\\to\\infty\}Uf\(S\_\{n\}\)V^\{\\top\}=US^\{p\}V^\{\\top\}, multiplying byU⊤U^\{\\top\}on the left andVVon the right, we havelimn→∞f​\(Sn\)=Sp\\lim\_\{n\\to\\infty\}f\(S\_\{n\}\)=S^\{p\}\. Letxxbe the top\-left diagonal entry ofSSand letxnx\_\{n\}be the top\-left diagonal entry ofSnS\_\{n\}\. Then we havelimn→∞xn=xp\\lim\_\{n\\to\\infty\}x\_\{n\}=x^\{p\}, we havex0=xx\_\{0\}=x, and we havexn\+1=f​\(xn\)x\_\{n\+1\}=f\(x\_\{n\}\)for allnn\. Sinceffis continuous we have

xa/b​limn→∞xn=limn→∞xn\+1=limn→∞f​\(xn\)=f​\(limn→∞xn\)=xp\.x^\{a/b\}\\lim\_\{n\\to\\infty\}x\_\{n\}=\\lim\_\{n\\to\\infty\}x\_\{n\+1\}=\\lim\_\{n\\to\\infty\}f\(x\_\{n\}\)=f\(\\lim\_\{n\\to\\infty\}x\_\{n\}\)=x^\{p\}\.
Thus the polynomial functionf​\(y\)−yf\(y\)\-yvanishes aty=xpy=x^\{p\}\. This holds for invertible matrixU​S​V⊤USV^\{\\top\}with all singular values at most11and hence holds for allxxin\(0,1\]\(0,1\]\. Sincexpx^\{p\}forxxin\(0,1\]\(0,1\]includes infinitely many distinct values ofyy, the polynomial functionf​\(y\)−yf\(y\)\-yis identically0\. Thusf​\(x\)=xf\(x\)=xsoSn\+1=SnS\_\{n\+1\}=S\_\{n\}for allnnand thusSn=SS\_\{n\}=Sfor allnnby induction, hence

limn→∞U​Sn​V⊤=limn→∞U​S​V⊤=U​S​V⊤≠U​Sp​V,\\lim\_\{n\\to\\infty\}US\_\{n\}V^\{\\top\}=\\lim\_\{n\\to\\infty\}USV^\{\\top\}=USV^\{\\top\}\\neq US^\{p\}V,contradicting our assumption\. ∎

###### Theorem A\.2\(Theorem[2\.3](https://arxiv.org/html/2606.13867#S2.Thmtheorem3), Computing odd two\-variable polynomials\)\.

1. 1\.Letf​\(x,y\)f\(x,y\)be a polynomial in two variables with real coefficients which is odd\. We can writeffin the form f​\(x,y\)=∑i,j=0da2​i\+1,2​j​x2​i\+1​y2​j\+∑i,j=0da2​i,2​j\+1​x2​i​y2​j\+1f\(x,y\)=\\sum\_\{i,j=0\}^\{d\}a\_\{2i\+1,2j\}x^\{2i\+1\}y^\{2j\}\+\\sum\_\{i,j=0\}^\{d\}a\_\{2i,2j\+1\}x^\{2i\}y^\{2j\+1\}\(A\.1\)for some natural numberddand tuples of real numbers\(a2​i\+1,j\)i,j=0d,\(a2​i,2​j\+1\)i,j=0d\(a\_\{2i\+1,j\}\)\_\{i,j=0\}^\{d\},\(a\_\{2i,2j\+1\}\)\_\{i,j=0\}^\{d\}\.
2. 2\.Forffa polynomial given by the formula \([A\.1](https://arxiv.org/html/2606.13867#A1.E1)\), forG=U​S​V⊤G=USV^\{\\top\}andGn=U​Sn​V⊤G\_\{n\}=US\_\{n\}V^\{\\top\}we have U​f​\(Sn,S\)​V⊤=∑i,ja2​i\+1,2​j​Gn​\(Gn⊤​Gn\)i​\(G⊤​G\)j\+∑i,ja2​i,2​j\+1​G​\(Gn⊤​Gn\)i​\(G⊤​G\)j\.Uf\(S\_\{n\},S\)V^\{\\top\}=\\sum\_\{i,j\}a\_\{2i\+1,2j\}G\_\{n\}\(G\_\{n\}^\{\\top\}G\_\{n\}\)^\{i\}\(G^\{\\top\}G\)^\{j\}\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}G\(G\_\{n\}^\{\\top\}G\_\{n\}\)^\{i\}\(G^\{\\top\}G\)^\{j\}\.

###### Proof\.

For part \(1\), sincef​\(x,y\)f\(x,y\)is a polynomial, we can write

f​\(x,y\)=∑i,j=02​d\+1ai,j​xi​yj\.f\(x,y\)=\\sum\_\{i,j=0\}^\{2d\+1\}a\_\{i,j\}x^\{i\}y^\{j\}\.\(A\.2\)Then

f​\(−x,−y\)=∑i,j=02​d\+1\(−1\)i​\(−1\)j​ai,j​xi​yj\.f\(\-x,\-y\)=\\sum\_\{i,j=0\}^\{2d\+1\}\(\-1\)^\{i\}\(\-1\)^\{j\}a\_\{i,j\}x^\{i\}y^\{j\}\.Forf​\(−x,−y\)f\(\-x,\-y\)to equal−f​\(x,y\)\-f\(x,y\)for allx,yx,y, the coefficient\(−1\)i​\(−1\)j​ai,j\(\-1\)^\{i\}\(\-1\)^\{j\}a\_\{i,j\}ofxi​yjx^\{i\}y^\{j\}inf​\(−x,−y\)f\(\-x,\-y\)must equal te coefficient−ai,j\-a\_\{i,j\}ofxi​yjx^\{i\}y^\{j\}in−f​\(x,y\)\-f\(x,y\)\. Ifi\+ji\+jis even this forcesai,j=−ai,ja\_\{i,j\}=\-a\_\{i,j\}and thusai,j=0a\_\{i,j\}=0\. Henceai,ja\_\{i,j\}is nonzero only ifi\+ji\+jis odd, which forces one ofi,ji,jto be even and the other to be odd\. Restricting the sum \([A\.2](https://arxiv.org/html/2606.13867#A1.E2)\) to only the terms withiiodd andjjeven oriieven andjjodd, we obtain \([A\.1](https://arxiv.org/html/2606.13867#A1.E1)\)\.

For part \(2\), we plug inG=U​S​V⊤G=USV^\{\\top\}andGn=U​Sn​V⊤G\_\{n\}=US\_\{n\}V^\{\\top\}and useU⊤​U=IU^\{\\top\}U=Iand thenV⊤​V=IV^\{\\top\}V=Ito obtain

∑i,ja2​i\+1,2​j​Gn​\(Gn⊤​Gn\)i​\(G⊤​G\)j\+∑i,ja2​i,2​j\+1​G​\(Gn⊤​Gn\)i​\(G⊤​G\)j\\sum\_\{i,j\}a\_\{2i\+1,2j\}G\_\{n\}\(G\_\{n\}^\{\\top\}G\_\{n\}\)^\{i\}\(G^\{\\top\}G\)^\{j\}\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}G\(G\_\{n\}^\{\\top\}G\_\{n\}\)^\{i\}\(G^\{\\top\}G\)^\{j\}=∑i,ja2​i\+1,2​j​U​Sn​V⊤​\(V​Sn​U⊤​U​Sn​V⊤\)i​\(V​S​U⊤​U​S​V⊤\)j=\\sum\_\{i,j\}a\_\{2i\+1,2j\}US\_\{n\}V^\{\\top\}\(VS\_\{n\}U^\{\\top\}US\_\{n\}V^\{\\top\}\)^\{i\}\(VSU^\{\\top\}USV^\{\\top\}\)^\{j\}\+∑i,ja2​i,2​j\+1​U​S​V⊤​\(V​Sn​U⊤​U​Sn​V⊤\)i​\(V​S​U⊤​U​S​V⊤\)j\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}USV^\{\\top\}\(VS\_\{n\}U^\{\\top\}US\_\{n\}V^\{\\top\}\)^\{i\}\(VSU^\{\\top\}USV^\{\\top\}\)^\{j\}=∑i,ja2​i\+1,2​j​U​Sn​V⊤​\(V​Sn2​V⊤\)i​\(V​S2​V⊤\)j\+∑i,ja2​i,2​j\+1​U​S​V⊤​\(V​Sn2​V⊤\)i​\(V​S2​V⊤\)j=\\sum\_\{i,j\}a\_\{2i\+1,2j\}US\_\{n\}V^\{\\top\}\(VS\_\{n\}^\{2\}V^\{\\top\}\)^\{i\}\(VS^\{2\}V^\{\\top\}\)^\{j\}\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}USV^\{\\top\}\(VS\_\{n\}^\{2\}V^\{\\top\}\)^\{i\}\(VS^\{2\}V^\{\\top\}\)^\{j\}=∑i,ja2​i\+1,2​j​U​Sn​Sn2​i​S2​j​V⊤\+∑i,ja2​i,2​j\+1​U​S​Sn2​i​S2​j​V⊤=\\sum\_\{i,j\}a\_\{2i\+1,2j\}US\_\{n\}S\_\{n\}^\{2i\}S^\{2j\}V^\{\\top\}\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}USS\_\{n\}^\{2i\}S^\{2j\}V^\{\\top\}=∑i,ja2​i\+1,2​j​U​Sn2​i\+1​S2​j​V⊤\+∑i,ja2​i,2​j\+1​U​Sn2​i​S2​j\+1​V⊤=\\sum\_\{i,j\}a\_\{2i\+1,2j\}US\_\{n\}^\{2i\+1\}S^\{2j\}V^\{\\top\}\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}US\_\{n\}^\{2i\}S^\{2j\+1\}V^\{\\top\}=U​\(∑i,ja2​i\+1,2​j​Sn2​i\+1​S2​j\+∑i,ja2​i,2​j\+1​Sn2​i​S2​j\+1\)​V⊤=U​f​\(Sn,S\)​V⊤\.=U\\left\(\\sum\_\{i,j\}a\_\{2i\+1,2j\}S\_\{n\}^\{2i\+1\}S^\{2j\}\+\\sum\_\{i,j\}a\_\{2i,2j\+1\}S\_\{n\}^\{2i\}S^\{2j\+1\}\\right\)V^\{\\top\}=Uf\(S\_\{n\},S\)V^\{\\top\}\.
∎

###### Lemma A\.3\(Lemma[3\.1](https://arxiv.org/html/2606.13867#S3.Thmtheorem1), fixed point criterion\)\.

Letaaandbbbe two coprime positive integers\. Letffbe a real polynomial in two variables\. The following are equivalent:

1. 1\.We havef​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}for ally∈\(0,1\]y\\in\(0,1\]\.
2. 2\.We havef​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}for ally≥0y\\geq 0\.
3. 3\.There exists a polynomialg​\(x,y\)g\(x,y\)such that we havef​\(x,y\)=x\+\(ya−xb\)​g​\(x,y\)f\(x,y\)=x\+\(y^\{a\}\-x^\{b\}\)g\(x,y\)\.

###### Proof\.

\(2\) implies \(1\) by restriction\. \(3\) implies \(2\) by substitution:

f​\(ya/b,y\)=ya/b\+\(ya−\(ya/b\)b\)​g​\(ya/b,y\)=ya/b\+\(ya−ya\)​g​\(ya/b,y\)=ya/b\+0=ya/b\.f\(y^\{a/b\},y\)=y^\{a/b\}\+\(y^\{a\}\-\(y^\{a/b\}\)^\{b\}\)g\(y^\{a/b\},y\)=y^\{a/b\}\+\(y^\{a\}\-y^\{a\}\)g\(y^\{a/b\},y\)=y^\{a/b\}\+0=y^\{a/b\}\.
It remains to check that \(1\) implies \(3\)\. By polynomial long division, we can writef​\(x,y\)=x\+\(ya−xb\)​g​\(x,y\)\+r​\(x,y\)f\(x,y\)=x\+\(y^\{a\}\-x^\{b\}\)g\(x,y\)\+r\(x,y\)for a polynomialg​\(x,y\)g\(x,y\)and a remainderr​\(x,y\)r\(x,y\)of degree≤a−1\\leq a\-1inyy, i\.e\. every monomial appearing inrrhas degree at mosta−1a\-1inyy\. Sincef​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}for ally∈\(0,1\]y\\in\(0,1\], we havef​\(za,zb\)=zaf\(z^\{a\},z^\{b\}\)=z^\{a\}for allz∈\(0,1\]z\\in\(0,1\]by takingy=zby=z^\{b\}\. Substituting in we get

za=f​\(za,zb\)=za\+\(\(za\)b−\(zb\)a\)​g​\(za,zb\)\+r​\(za,zb\)=za\+0\+r​\(za,zb\)z^\{a\}=f\(z^\{a\},z^\{b\}\)=z^\{a\}\+\(\(z^\{a\}\)^\{b\}\-\(z^\{b\}\)^\{a\}\)g\(z^\{a\},z^\{b\}\)\+r\(z^\{a\},z^\{b\}\)=z^\{a\}\+0\+r\(z^\{a\},z^\{b\}\)sor​\(za,zb\)=0r\(z^\{a\},z^\{b\}\)=0for allz∈\(0,1\]z\\in\(0,1\]\. Sincer​\(za,zb\)r\(z^\{a\},z^\{b\}\)is a polynomial inzz, this makesr​\(za,zb\)r\(z^\{a\},z^\{b\}\)identically zero\. If we write

r​\(x,y\)=∑i=0d∑j=a−1dci,j​xi​yjr\(x,y\)=\\sum\_\{i=0\}^\{d\}\\sum\_\{j=a\-1\}^\{d\}c\_\{i,j\}x^\{i\}y^\{j\}we get

r​\(za,zb\)=∑i=0d∑j=0a−1ci,j​zi​a\+j​b\.r\(z^\{a\},z^\{b\}\)=\\sum\_\{i=0\}^\{d\}\\sum\_\{j=0\}^\{a\-1\}c\_\{i,j\}z^\{ia\+jb\}\.
However, every integernnhas a unique expression asi​a\+j​bia\+jbfor an integeriiandjjan integer between0anda−1a\-1, sincejjmust be congruent tonntimes the modular inverse ofbbmoduloaaand there is a unique integer between0anda−1a\-1in that congruence class, andiiis determined byjjand the equationi​a\+j​b=nia\+jb=n\. So each coefficient ofr​\(za,zb\)r\(z^\{a\},z^\{b\}\)is the sum of at most oneci,jc\_\{i,j\}, and hence thatci,jc\_\{i,j\}must vanish, thus everyci,jc\_\{i,j\}vanishes sor=0r=0, as desired\. ∎

To further ensure thatffis odd, we must adapt the formula of Lemma[3\.1](https://arxiv.org/html/2606.13867#S3.Thmtheorem1)depending on whetheraaandbbare odd\. Specifically, we take

f​\(x,y\)=x\+\(ya−xb\)​g​\(x,y\)​with​g​evenf\(x,y\)=x\+\(y^\{a\}\-x^\{b\}\)g\(x,y\)\\textrm\{ with \}g\\textrm\{ even\}\(A\.3\)ifaaandbbare both odd and

f​\(x,y\)=x\+\(y2​a−x2​b\)​h​\(x,y\)​with​h​oddf\(x,y\)=x\+\(y^\{2a\}\-x^\{2b\}\)h\(x,y\)\\textrm\{ with \}h\\textrm\{ odd \}\(A\.4\)ifaaorbbis even\. Note that \([A\.4](https://arxiv.org/html/2606.13867#A1.E4)\) is the special case of \([A\.3](https://arxiv.org/html/2606.13867#A1.E3)\) whereg​\(x,y\)=\(ya\+xb\)​h​\(x,y\)g\(x,y\)=\(y^\{a\}\+x^\{b\}\)h\(x,y\)\.

###### Lemma A\.4\(Lemma[3\.2](https://arxiv.org/html/2606.13867#S3.Thmtheorem2), oddness criterion\)\.

Letaaandbbbe two positive integers\.

1. 1\.Ifaaandbbare odd, the expression \([A\.3](https://arxiv.org/html/2606.13867#A1.E3)\) is always odd\.
2. 2\.The expression \([A\.4](https://arxiv.org/html/2606.13867#A1.E4)\) is always odd\.

###### Proof\.

These both follow from the facts that an odd polynomial times an even polynomial is odd and the sum of two odd polynomials is odd\. For \(1\), we observe thatya−xby^\{a\}\-x^\{b\}is odd andggis even by assumption so their product is odd and then addingxxpreserves oddness\. For \(2\), we observe thaty2​a−x2​by^\{2a\}\-x^\{2b\}is even andhhis odd by assumption so their product is odd and then addingxxpreserves oddness\. ∎

###### Lemma A\.5\(Lemma[3\.3](https://arxiv.org/html/2606.13867#S3.Thmtheorem3), attracting fixed point criterion\)\.

Letffbe a polynomial in two variables and letaaandbbbe two positive integers\. Sayya/by^\{a/b\}is an*attracting fixed point*off​\(x,y\)f\(x,y\)iff​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}and

\|∂f∂x​\(ya/b,y\)\|<1\.\\left\|\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\\right\|<1\.
Ifya/by^\{a/b\}is an attracting fixed point off​\(x,y\)f\(x,y\)then there exists an intervalI⊂ℝI\\subset\\mathbb\{R\}containingya/by^\{a/b\}such that, for anyx0∈Ix\_\{0\}\\in I, if we setxn\+1=f​\(xn,y\)x\_\{n\+1\}=f\(x\_\{n\},y\)for alln≥0n\\geq 0, thenlimn→∞xn=ya/b\\lim\_\{n\\to\\infty\}x\_\{n\}=y^\{a/b\}\.

###### Proof\.

This is a standard result in dynamical systems\. To prove it, takeδ\\deltasuch that\|∂f∂x​\(ya/b,y\)\|≤1−2​δ\.\\left\|\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\\right\|\\leq 1\-2\\delta\.By Taylor’s theorem, we have

f​\(x,y\)=f​\(ya/b,y\)\+\(x−ya/b\)​∂f∂x​\(ya/b,y\)\+O​\(\|x−ya/b\|2\)f\(x,y\)=f\(y^\{a/b\},y\)\+\(x\-y^\{a/b\}\)\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\+O\(\|x\-y^\{a/b\}\|^\{2\}\)=ya/b\+\(x−ya/b\)​∂f∂x​\(ya/b,y\)\+O​\(\|x−ya/b\|2\)=y^\{a/b\}\+\(x\-y^\{a/b\}\)\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\+O\(\|x\-y^\{a/b\}\|^\{2\}\)so

\|f​\(x,y\)−ya/b\|≤\|x−ya/b\|​\|∂f∂x​\(ya/b,y\)\|\+O​\(\|x−ya/b\|2\)\\left\|f\(x,y\)\-y^\{a/b\}\\right\|\\leq\\left\|x\-y^\{a/b\}\\right\|\\left\|\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\\right\|\+O\(\|x\-y^\{a/b\}\|^\{2\}\)≤\|x−ya/b\|​\(1−2​δ\)\+O​\(\|x−ya/b\|2\)\\leq\\left\|x\-y^\{a/b\}\\right\|\(1\-2\\delta\)\+O\(\|x\-y^\{a/b\}\|^\{2\}\)
We can chooseCCsuch that if\|x−ya/b\|≤C\|x\-y^\{a/b\}\|\\leq CtheO​\(\|x−ya/b\|2\)O\(\|x\-y^\{a/b\}\|^\{2\}\)term is at mostδ​\|x−ya/b\|\\delta\|x\-y^\{a/b\}\|which gives\|f​\(x,y\)−ya/b\|≤\(1−δ\)​\|x−ya/b\|\.\\left\|f\(x,y\)\-y^\{a/b\}\\right\|\\leq\(1\-\\delta\)\|x\-y^\{a/b\}\|\.Now if\|x0−ya/b\|≤C\|x\_\{0\}\-y^\{a/b\}\|\\leq Cwe have\|xn−ya/b\|≤\(1−δ\)n​C\|x\_\{n\}\-y^\{a/b\}\|\\leq\(1\-\\delta\)^\{n\}Cby induction onnnsolimn→∞xn=ya/b\\lim\_\{n\\to\\infty\}x\_\{n\}=y^\{a/b\}, and thus we can takeI=\(Ya/b−C,ya/b\+C\)I=\(Y^\{a/b\}\-C,y^\{a/b\}\+C\)\.∎

###### Lemma A\.6\(Lemma[3\.4](https://arxiv.org/html/2606.13867#S3.Thmtheorem4), choice ofcc\)\.

Letaaandbbbe two positive integers\.

1. 1\.Letf​\(x,y\)f\(x,y\)be a polynomial given by the formula \([A\.3](https://arxiv.org/html/2606.13867#A1.E3)\)\. If 0<g​\(ya/b,y\)<2b​ya​\(b−1\)b0<g\(y^\{a/b\},y\)<\\frac\{2\}\{by^\{\\frac\{a\(b\-1\)\}\{b\}\}\}for ally∈\(0,1\]y\\in\(0,1\]thenya/by^\{a/b\}is an attracting fixed point off​\(x,y\)f\(x,y\)for ally∈\(0,1\]y\\in\(0,1\]\.
2. 2\.Letf​\(x,y\)f\(x,y\)be a polynomial given by the formula \([A\.4](https://arxiv.org/html/2606.13867#A1.E4)\)\. If 0<g​\(ya/b,y\)<1b​ya​\(2​b−1\)b0<g\(y^\{a/b\},y\)<\\frac\{1\}\{by^\{\\frac\{a\(2b\-1\)\}\{b\}\}\}for ally∈\(0,1\]y\\in\(0,1\]thenya/by^\{a/b\}is an attracting fixed point off​\(x,y\)f\(x,y\)for ally∈\(0,1\]y\\in\(0,1\]\.

###### Proof\.

This follows from computing the derivative\. Forffgiven by \([A\.3](https://arxiv.org/html/2606.13867#A1.E3)\) we have

∂f∂x=1\+\(ya−xb\)​∂g∂x​\(x,y\)−b​xb−1​g​\(x,y\)\\frac\{\\partial f\}\{\\partial x\}=1\+\(y^\{a\}\-x^\{b\}\)\\frac\{\\partial g\}\{\\partial x\}\(x,y\)\-bx^\{b\-1\}g\(x,y\)and substitutingx=ya/bx=y^\{a/b\}we get

∂f∂x​\(ya/b,y\)=1\+\(ya−ya\)​∂g∂x​\(ya/b,y\)−b​ya​\(b−1\)b​g​\(ya/b,y\)=1−b​ya​\(b−1\)b​g​\(ya/b,y\)\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)=1\+\(y^\{a\}\-y^\{a\}\)\\frac\{\\partial g\}\{\\partial x\}\(y^\{a/b\},y\)\-by^\{\\frac\{a\(b\-1\)\}\{b\}\}g\(y^\{a/b\},y\)=1\-by^\{\\frac\{a\(b\-1\)\}\{b\}\}g\(y^\{a/b\},y\)so to have\|∂f∂x​\(ya/b,y\)\|<1\\left\|\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\\right\|<1it suffices to have

0<b​ya​\(b−1\)b​g​\(ya/b,y\)<20<by^\{\\frac\{a\(b\-1\)\}\{b\}\}g\(y^\{a/b\},y\)<2and dividing byb​ya​\(b−1\)bby^\{\\frac\{a\(b\-1\)\}\{b\}\}gives part \(1\)\.

Forffgiven by \([A\.4](https://arxiv.org/html/2606.13867#A1.E4)\) we have

∂f∂x=1\+\(y2​a−x2​b\)​∂h∂x​\(x,y\)−2​b​xb−1​h​\(x,y\)\\frac\{\\partial f\}\{\\partial x\}=1\+\(y^\{2a\}\-x^\{2b\}\)\\frac\{\\partial h\}\{\\partial x\}\(x,y\)\-2bx^\{b\-1\}h\(x,y\)and substitutingx=ya/bx=y^\{a/b\}we get

∂f∂x​\(ya/b,y\)=1\+\(y2​a−y2​a\)​∂h∂x​\(ya/b,y\)−2​b​ya​\(2​b−1\)b​h​\(ya/b,y\)=1−2​b​ya​\(2​b−1\)b​h​\(ya/b,y\)\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)=1\+\(y^\{2a\}\-y^\{2a\}\)\\frac\{\\partial h\}\{\\partial x\}\(y^\{a/b\},y\)\-2by^\{\\frac\{a\(2b\-1\)\}\{b\}\}h\(y^\{a/b\},y\)=1\-2by^\{\\frac\{a\(2b\-1\)\}\{b\}\}h\(y^\{a/b\},y\)so to have\|∂f∂x​\(ya/b,y\)\|<1\\left\|\\frac\{\\partial f\}\{\\partial x\}\(y^\{a/b\},y\)\\right\|<1it suffices to have

0<2​b​ya​\(2​b−1\)b​h​\(ya/b,y\)<20<2by^\{\\frac\{a\(2b\-1\)\}\{b\}\}h\(y^\{a/b\},y\)<2and dividing by2​b​ya​\(2​b−1\)b2by^\{\\frac\{a\(2b\-1\)\}\{b\}\}gives part \(2\)\. ∎

Lemma[3\.4](https://arxiv.org/html/2606.13867#S3.Thmtheorem4)makes it easy to construct polynomials which satisfy the hypothesis of Lemma[3\.3](https://arxiv.org/html/2606.13867#S3.Thmtheorem3)for ally∈\(0,1\]y\\in\(0,1\]\. Fora,ba,bodd, we can simply takeg​\(x,y\)=cg\(x,y\)=cfor any positivec<2/bc<2/b\. Foraaorbbeven, we can takeh​\(x,y\)=c0​x\+c1​yh\(x,y\)=c\_\{0\}x\+c\_\{1\}yfor any positivec0,c1c\_\{0\},c\_\{1\}withc0\+cy<1/bc\_\{0\}\+c\_\{y\}<1/b\.

We have already seen some examples in the odd case\. The simplest example of our construction in the even case is:

f​\(x,y\)=x\+\(y2−x4\)​\(c0​x\+c1​y\)​converging to​y1/2f\(x,y\)=x\+\(y^\{2\}\-x^\{4\}\)\(c\_\{0\}x\+c\_\{1\}y\)\\textrm\{ converging to \}y^\{1/2\}
We are now ready to prove Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4)\.

###### Proof of Theorem[2\.4](https://arxiv.org/html/2606.13867#S2.Thmtheorem4)\.

We may assumeaaandbbare coprime without loss of generality by dividing any common factors fromaaandbb\.

We first check that it suffices to havef​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}andx<f​\(x,y\)<ya/bx<f\(x,y\)<y^\{a/b\}for anyy∈\(0,1\]y\\in\(0,1\]andy≤x<ya/by\\leq x<y^\{a/b\}\. Indeed, by induction onnnwe will havey≤xn<ya/by\\leq x\_\{n\}<y^\{a/b\}for allnn, since this is satisfied forn=0n=0and assuming it is satisfied atnnwe havexn\+1=f​\(xn,y\)x\_\{n\+1\}=f\(x\_\{n\},y\)and

y≤xn<f​\(xn,y\)<ya/by\\leq x\_\{n\}<f\(x\_\{n\},y\)<y^\{a/b\}so

y≤xn\+1<ya/b,y\\leq x\_\{n\+1\}<y^\{a/b\},completing the induction step\. Then by assumption we havexn\+1\>xnx\_\{n\+1\}\>x\_\{n\}for allnn, soxnx\_\{n\}is a bounded monotone sequence and thus convergences\. Sinceffis continuous if we setx=limn→∞xnx=\\lim\_\{n\\to\\infty\}x\_\{n\}we must have

x=limn→∞xn=limn→∞xn\+1=limn→∞f​\(xn,y\)=f​\(limn→∞xn,y\)=f​\(x,y\)x=\\lim\_\{n\\to\\infty\}x\_\{n\}=\\lim\_\{n\\to\\infty\}x\_\{n\+1\}=\\lim\_\{n\\to\\infty\}f\(x\_\{n\},y\)=f\(\\lim\_\{n\\to\\infty\}x\_\{n\},y\)=f\(x,y\)but we havey≤x≤ya/by\\leq x\\leq y^\{a/b\}as we havey≤xn≤ya/by\\leq x\_\{n\}\\leq y^\{a/b\}for eachnn\. Ifx≠ya/bx\\neq y^\{a/b\}then we havey≤x<ya/by\\leq x<y^\{a/b\}sox<f​\(x,y\)x<f\(x,y\)which contradictsx=f​\(x,y\)x=f\(x,y\), so we must havex=ya/bx=y^\{a/b\}, as desired\.

We now ensure that the sufficient conditions above are satisfied\. Fora,ba,bodd we takef​\(x,y\)=x\+c​\(ya−xb\)f\(x,y\)=x\+c\(y^\{a\}\-x^\{b\}\)for somec\>0c\>0\. Sinceccis even, this falls into \([A\.3](https://arxiv.org/html/2606.13867#A1.E3)\)\. We saw in Lemma[3\.2](https://arxiv.org/html/2606.13867#S3.Thmtheorem2)thatffis odd and in Lemma[3\.1](https://arxiv.org/html/2606.13867#S3.Thmtheorem1)thatf​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}\. Ify≤x<ya/by\\leq x<y^\{a/b\}thenya−xb\>0y^\{a\}\-x^\{b\}\>0sof​\(x,y\)\>xf\(x,y\)\>x\. Thus it only remains to ensure thatf​\(x,y\)<ya/bf\(x,y\)<y^\{a/b\}\. We have

ya/b−f​\(x,y\)y^\{a/b\}\-f\(x,y\)=ya/b−x−c​\(ya−xb\)=\(ya/b−x\)​\(1−c​\(ya−a/b\+ya−2​a/b​x\+ya−3​a/b​x2\+⋯\+xb−1\)\)\.=y^\{a/b\}\-x\-c\(y^\{a\}\-x^\{b\}\)=\(y^\{a/b\}\-x\)\(1\-c\(y^\{a\-a/b\}\+y^\{a\-2a/b\}x\+y^\{a\-3a/b\}x^\{2\}\+\\dots\+x^\{b\-1\}\)\)\.Since we assumedx≤ya/b≤1x\\leq y^\{a/b\}\\leq 1, the termsya−a/b,ya−2​a/b​x,ya−3​a/b​x2,…,xb−1y^\{a\-a/b\},y^\{a\-2a/b\}x,y^\{a\-3a/b\}x^\{2\},\\dots,x^\{b\-1\}are each at most11and there arebbterms so

ya−a/b\+ya−2​a/b​x\+ya−3​a/b​x2\+⋯\+xb−1≤by^\{a\-a/b\}\+y^\{a\-2a/b\}x\+y^\{a\-3a/b\}x^\{2\}\+\\dots\+x^\{b\-1\}\\leq band so to haveya/b−f​\(x,y\)\>0y^\{a/b\}\-f\(x,y\)\>0it suffices to havec<1/bc<1/b\. Thus anyc∈\(0,1/b\)c\\in\(0,1/b\)works\.

Foraaorbbeven we takef​\(x,y\)=x\+c​y​\(y2​a−x2​b\)f\(x,y\)=x\+cy\(y^\{2a\}\-x^\{2b\}\)for somec\>0c\>0\. Sinceccis odd, this falls into \([A\.4](https://arxiv.org/html/2606.13867#A1.E4)\)\. We saw in Lemma[3\.2](https://arxiv.org/html/2606.13867#S3.Thmtheorem2)that thatffis odd and in Lemma[3\.1](https://arxiv.org/html/2606.13867#S3.Thmtheorem1)thatf​\(ya/b,y\)=ya/bf\(y^\{a/b\},y\)=y^\{a/b\}\. Ify≤x<ya/by\\leq x<y^\{a/b\}thenya−xb\>0y^\{a\}\-x^\{b\}\>0and we assumedy\>0y\>0sof​\(x,y\)\>xf\(x,y\)\>x\. Thus it only remains to ensure thatf​\(x,y\)<ya/bf\(x,y\)<y^\{a/b\}\. We have

ya/b−f​\(x,y\)=ya/b−x−c​y​\(y2​a−x2​b\)y^\{a/b\}\-f\(x,y\)=y^\{a/b\}\-x\-cy\(y^\{2a\}\-x^\{2b\}\)=\(ya/b−x\)​\(1−c​y​\(y2​a−a/b\+y2​a−2​a/b​x\+y2​a−3​a/b​x2\+⋯\+x2​b−1\)\)\.=\(y^\{a/b\}\-x\)\(1\-cy\(y^\{2a\-a/b\}\+y^\{2a\-2a/b\}x\+y^\{2a\-3a/b\}x^\{2\}\+\\dots\+x^\{2b\-1\}\)\)\.Since we assumedx≤ya/b≤1x\\leq y^\{a/b\}\\leq 1, the termsy2​a−a/b,y2​a−2​a/bx,y2​a−3​a/bx2,…,x2​b−1\)y^\{2a\-a/b\},y^\{2a\-2a/b\}x,y^\{2a\-3a/b\}x^\{2\},\\dots,x^\{2b\-1\}\)are each at most11and there are2​b2bterms so

\(y2​a−a/b\+y2​a−2​a/b​x\+y2​a−3​a/b​x2\+⋯\+x2​b−1\)≤2​b\(y^\{2a\-a/b\}\+y^\{2a\-2a/b\}x\+y^\{2a\-3a/b\}x^\{2\}\+\\dots\+x^\{2b\-1\}\)\\leq 2band so to haveya/b−f​\(x,y\)\>0y^\{a/b\}\-f\(x,y\)\>0it suffices to havec<1/\(2​b\)c<1/\(2b\)\. Thus anyc∈\(0,1/\(2​b\)\)c\\in\(0,1/\(2b\)\)works\. ∎

###### Theorem A\.7\(Theorem[2\.1](https://arxiv.org/html/2606.13867#S2.Thmtheorem1)\)\.

For any real numberp∈\(0,1\)p\\in\(0,1\)and letq=1\+1/pq=1\+1/p\. LetGGbe ann×nn\\times nmatrix with singular value decompositionU​S​V⊤USV^\{\\top\}\. Let⟨,⟩\\langle,\\rangledenote the entrywise dot product of matrices and let\|\|⋅\|\|q\|\|\\cdot\|\|\_\{q\}denote the Schattenqq\-norm\. Then for everyc1\>0c\_\{1\}\>0, the maximum value of⟨G,M⟩\\langle G,M\\rangleamongn×nn\\times nmatricesMMwith‖M‖q≤c1\|\|M\|\|\_\{q\}\\leq c\_\{1\}is attained byM=c2​U​Sp​V⊤M=c\_\{2\}US^\{p\}V^\{\\top\}for somec2\>0c\_\{2\}\>0\.

###### Proof\.

For anyn×nn\\times nmatrixDDwith‖M‖q≤c1\|\|M\|\|\_\{q\}\\leq c\_\{1\}, lets1,…,sns\_\{1\},\\dots,s\_\{n\}be the singular values ofGG\. Then we have

⟨G,M⟩=tr⁡\(G​M⊤\)≤‖G‖p\+1​‖M‖q≤‖G‖p\+1​c1=\(∑i=1nsip\+1\)1p\+1​c1\\langle G,M\\rangle=\\operatorname\{tr\}\(GM^\{\\top\}\)\\leq\|\|G\|\|\_\{p\+1\}\|\|M\|\|\_\{q\}\\leq\|\|G\|\|\_\{p\+1\}c\_\{1\}=\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{p\+1\}\)^\{\\frac\{1\}\{p\+1\}\}c\_\{1\}by Hölder’s inequality for the Schattenpp\-norms, since1p\+1\+1q=1\\frac\{1\}\{p\+1\}\+\\frac\{1\}\{q\}=1\.

On the other hand, forc2=c1\(∑i=1nsip​q\)1qc\_\{2\}=\\frac\{c\_\{1\}\}\{\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{pq\}\)^\{\\frac\{1\}\{q\}\}\}andM=c2​U​Sp​V⊤M=c\_\{2\}US^\{p\}V^\{\\top\}, we have

‖M‖q=c2​\(∑i=1nsip​q\)1q=c1\(∑i=1nsip​q\)1q​\(∑i=1nsip​q\)1q=c1\|\|M\|\|\_\{q\}=c\_\{2\}\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{pq\}\)^\{\\frac\{1\}\{q\}\}=\\frac\{c\_\{1\}\}\{\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{pq\}\)^\{\\frac\{1\}\{q\}\}\}\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{pq\}\)^\{\\frac\{1\}\{q\}\}=c\_\{1\}and we have

⟨G,M⟩=c2​tr⁡\(U​S​V⊤​V​Sp​U⊤\)=c2​tr⁡\(S1\+p\)=c2​∑i=1nsi1\+p\\langle G,M\\rangle=c\_\{2\}\\operatorname\{tr\}\(USV^\{\\top\}VS^\{p\}U^\{\\top\}\)=c\_\{2\}\\operatorname\{tr\}\(S^\{1\+p\}\)=c\_\{2\}\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{1\+p\}=c1​∑i=1nsi1\+p\(∑i=1nsip​q\)1q=c1​∑i=1nsi1\+p\(∑i=1nsi1\+p\)pp\+1=c1​\(∑i=1nsi1\+p\)1p\+1=c\_\{1\}\\frac\{\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{1\+p\}\}\{\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{pq\}\)^\{\\frac\{1\}\{q\}\}\}=c\_\{1\}\\frac\{\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{1\+p\}\}\{\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{1\+p\}\)^\{\\frac\{p\}\{p\+1\}\}\}=c\_\{1\}\(\\sum\_\{i=1\}^\{n\}s\_\{i\}^\{1\+p\}\)^\{\\frac\{1\}\{p\+1\}\}so the maximum is attained by this value ofDD, as desired\. ∎