Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity

# An Exponential Separation in Initialization Sensitivity
Source: [https://arxiv.org/html/2606.11431](https://arxiv.org/html/2606.11431)
## Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity

Shira Vansover\-HagerBlavatnik School of Computer Science and AI, Tel Aviv University;\{shirav,schliserman,ofirs4\}@mail\.tau\.ac\.il\.Ofir Schlisselberg11footnotemark:1Tomer KorenBlavatnik School of Computer Science and AI, Tel Aviv University, and Google Research;tkoren@tauex\.tau\.ac\.il\.

###### Abstract

Mirror Descent \(MD\) extends Gradient Descent \(GD\) beyond Euclidean geometry and has recently reappeared as a lens for KL\-regularized policy optimization in reinforcement learning and LLM post\-training\. This raises a basic robustness question, crucial to reproducibility and reliability: how sensitive are MD dynamics to their inputs? We focus on initialization, often itself a pretrained or previously aligned model\. Quadratic\-regularized MD, including GD and Mahalanobis geometries, is well\-known to be stable for convex smooth objectives\. We show a sharp contrast: once the regularizer is non\-quadratic, MD can be exponentially more sensitive to initialization than GD, even with a well\-conditioned regularizer in Euclidean norm\. We give a three\-dimensional construction with a convex, smooth objective and a strongly convex, smooth, well\-conditioned regularizer where an initialε\\varepsilonperturbation is quickly amplified tomin⁡\{polylog−1​\(1/ε\),ε​eΩ​\(η​T\)\}\\smash\{\\min\\\{\\text\{polylog\}^\{\-1\}\(1/\\varepsilon\),\\varepsilon e^\{\\Omega\(\\eta T\)\}\\\}\}afterTTiterations of MD with step sizeη\\eta\. For canonical KL\-regularized MD on the simplex, we show that even linear objectives can amplify an initialε\\varepsilonperturbation exponentially fast in high\-dimensional or near\-boundary regimes\. Finally, we show that adding a Bregman regularization term toward an anchor point can stabilize the dynamics while largely preserving the optimization guarantees, and that the choice of anchor is crucial: anchoring at the initialization only partially mitigates the instability, whereas anchoring at a fixed point yields a more stable mechanism\.

## 1Introduction

Mirror Descent \(MD\)\(Nemirovski and Yudin,[1983](https://arxiv.org/html/2606.11431#bib.bib295); Beck and Teboulle,[2003](https://arxiv.org/html/2606.11431#bib.bib296)\)is a fundamental optimization paradigm that adapts its updates to the geometry of the parameter space\. Its updates, typically written as

wt\+1=arg⁡minw⁡\{η​⟨∇F​\(wt\),w⟩\+DR​\(w,wt\)\},w\_\{t\+1\}=\\arg\\min\_\{w\}\\left\\\{\\eta\\langle\\nabla F\(w\_\{t\}\),w\\rangle\+D\_\{R\}\(w,w\_\{t\}\)\\right\\\},whereDRD\_\{R\}is the Bregman divergence induced by a regularizerRRandη\\etais the step size, optimize a local linearization of the objective while penalizing deviations according to the geometry specified byRR\. Notable instances include Gradient Descent\(Nesterov,[1998](https://arxiv.org/html/2606.11431#bib.bib19)\)and multiplicative weights updates\(Littlestone and Warmuth,[1994](https://arxiv.org/html/2606.11431#bib.bib124); Freund and Schapire,[1997](https://arxiv.org/html/2606.11431#bib.bib114); Aroraet al\.,[2012](https://arxiv.org/html/2606.11431#bib.bib125)\)\.

The stability of these dynamics is increasingly important in modern machine learning\. MD has reappeared as a useful lens for KL\-regularized policy optimization in reinforcement learning\(Schulmanet al\.,[2015](https://arxiv.org/html/2606.11431#bib.bib120),[2017](https://arxiv.org/html/2606.11431#bib.bib122); Akkayaet al\.,[2019](https://arxiv.org/html/2606.11431#bib.bib115)\)and LLM post\-training\(Ouyang,[2022](https://arxiv.org/html/2606.11431#bib.bib123); Shaoet al\.,[2024](https://arxiv.org/html/2606.11431#bib.bib121)\)\. In such settings, optimization is often initialized from a pretrained, supervised fine\-tuned, or otherwise aligned model\. Changes in pretraining data, randomness, or checkpoint choice therefore perturb the initialization itself, and potentially the final model\. This raises a basic robustness question, central to reproducibility and reliability: how much can mirror\-descent dynamics amplify a small perturbation in their initialization afterTTsequential steps? We study this question through the lens of*initialization stability*: the worst\-case change in the output of theTT\-step algorithm under anε\\varepsilon\-perturbation of its starting point\. Equivalently, this is a local worst\-case robustness, or finite\-time sensitivity, notion for the algorithm’s output as a function of the initialization\.

For quadratic regularization in Euclidean geometry, the answer is well known\. This class includes Gradient Descent and Mahalanobis geometries, and for convex smooth objectives these algorithms are known to be extremely stable: small initialization perturbations remain small along the trajectory and at the final output\(Hardtet al\.,[2016](https://arxiv.org/html/2606.11431#bib.bib17)\)\. Much less is known for non\-Euclidean mirror maps\. A key difference from the quadratic\-regularized case relates to the conditioning of the regularizer: if the mirror map is poorly conditioned, a small primal perturbation can correspond to a much larger displacement in the geometry used by the update\. Conversely, one might hope that uniformly well\-conditioned mirror maps behave similarly to quadratic ones and retain their favorable stability\.

The central finding of this paper is that this hope is false: even for convex smooth objectives,*general mirror descent can be exponentially more sensitive to initialization as compared to Gradient Descent*\. We establish this phenomenon in two complementary regimes, separating the role of nonquadratic geometry from the additional effects of ill\-conditioning\. First, we show that exponential instability is not merely an artifact of a poorly conditioned regularizer\. In three\-dimensional Euclidean space equipped with the usualℓ2\\ell\_\{2\}norm, we construct a convex, smooth objective and a strongly convex, smooth, well\-conditioned \(nonquadratic\) regularizer, all with respect to this norm, for which anε\\varepsilon\-perturbation of the initialization is amplified by a factoreΩ​\(η​T\)\\smash\{e^\{\\Omega\(\\eta T\)\}\}, up toΘ~​\(1\)\\smash\{\\widetilde\{\\Theta\}\(1\)\}saturation scale\. This phenomenon can arise even for step\-size choices for which MD is guaranteed to optimize the objective\. This implies an exponential separation from the stability enjoyed by quadratic regularization and standard Gradient Descent\.

Second, we study the canonical entropic geometry on the simplex, whereR​\(w\)=∑iwi​log⁡wiR\(w\)=\\sum\_\{i\}w\_\{i\}\\log w\_\{i\}andDRD\_\{R\}is the KL divergence\. This geometry underlies classical multiplicative\-weights methods and modern KL\-regularized updates in policy optimization and model post\-training\. Here the conditioning mechanism is visible more directly: negative entropy is highly ill\-conditioned inℓ1\\ell\_\{1\}geometry near low\-mass coordinates, and we show that this alone can drive exponential amplification even for linear objectives\. The resulting lower bound is uniform over all simplex initializations in the high\-dimensional regime, and also captures the low\-dimensional, near\-boundary regime\. Notably, this instability can arise even for step\-size ranges in which MD still converges to a minimizer\. We complement this lower bound with a matching exponential upper bound for entropy MD, and with an extension to more general Legendre regularizations frequent in online optimization and RL\(Cesa\-Bianchi and Lugosi,[2006](https://arxiv.org/html/2606.11431#bib.bib645)\)\.

Finally, we ask whether this instability of MD can be mitigated without abandoning the geometry and generality of the method\. For this, we introduce two variants of MD that stabilize the algorithm by anchoring it to a reference point through an additional Bregman regularization term\. First, motivated by practical settings such as KL\-regularized fine\-tuning, where the optimization process is initialized from a pretrained model that also serves as the reference point\(e\.g\., Ouyang,[2022](https://arxiv.org/html/2606.11431#bib.bib123); Shaoet al\.,[2024](https://arxiv.org/html/2606.11431#bib.bib121)\), we study*Initialization\-Anchored MD*\. In this method, the additional regularization term is given by the Bregman distance*to the initialization point*\. We show that this variant stabilizes MD in the well\-conditioned setting, assuming the regularizer is also smooth, achieving initialization stability ofO​\(ε\+1/T​log⁡T\)O\(\\varepsilon\+1/\\sqrt\{T\\log T\}\)together with optimization errorO​\(log⁡T/T\)O\(\\log T/T\)\. However, in ill\-conditioned settings, the guarantee of this variant may become vacuous due to its dependence on local smoothness of the regularizer at initialization\. To overcome this, we introduce a second variant,*Fixed\-Anchor MD*, in which the Bregman regularization is anchored at a*fixed reference point*independent of the initialization\. We prove that this method remains stable even for ill\-conditioned regularizers, achievingO​\(1/T\)O\(1/T\)initialization stability while preserving the optimization guarantees of MD up to logarithmic factors, with optimization errorO​\(log⁡\(T\)/T\)O\(\\log\(T\)/T\)\. These results extend the regularization\-based stabilization perspective ofAttia and Koren \([2022](https://arxiv.org/html/2606.11431#bib.bib2)\)from uniform stability to initialization stability\.

Taken together, our results suggest that initialization sensitivity should be treated as a primary consideration when MD is used as a modeling abstraction for modern optimization pipelines\. In KL\-regularized policy optimization or LLM post\-training, small differences in the starting reference model can be quickly amplified through only a few sequential updates, and this may occur even if the initial model has significant entropy\. At the same time, the Bergman\-regularized algorithms indicate that this sensitivity is not inevitable: stability can be improved by augmenting with additional regularization in the same geometry, preserving optimization rates \(up to log factors\)\.

### 1\.1Summary of contributions

In more detail, our main contributions in this paper are as follows\.

- •We show that MD exhibits exponential initialization sensitivity already in a low\-dimensional, well\-conditioned Euclidean geometry\. Specifically, in dimensiond=3d=3, with respect to the standardℓ2\\ell\_\{2\}norm, we construct a convex smooth objective and a strongly convex, smooth, well\-conditioned nonquadratic regularizer for which MD has initialization instability Ω​\(min⁡\{polylog−1⁡\(1/ε\),ε​eΩ​\(η​T\)\}\)\.\\Omega\\\!\\left\(\\min\\left\\\{\\operatorname\{polylog\}^\{\-1\}\(1/\\varepsilon\),\\varepsilon e^\{\\Omega\(\\eta T\)\}\\right\\\}\\right\)\.This gives an exponential separation from quadratic regularization: for quadratic MD, including Gradient Descent and Mahalanobis geometries, initialization perturbations remain bounded byO​\(\(β/α\)​ε\)O\(\(\\beta/\\alpha\)\\varepsilon\)throughout the algorithm’s trajectory\.
- •For the canonical entropic/KL geometry on the simplex, we give a sharp characterization of the initialization stability of MD, which is again exponential inη​T\\eta T\. We prove that negative\-entropy MD can amplify anε\\varepsilon\-perturbation byΩ​\(min⁡\{1,ε​eη​T\}\)\\Omega\(\\min\\\{1,\\varepsilon e^\{\\eta T\}\\\}\), even for linear objectives\. In the high\-dimensional regimed≥1/εd\\geq 1/\\varepsilon, this result holds uniformly over all initializations; the same result also captures near\-boundary worst\-case initializations in low dimension\. We complement it with a matching exponential upper bound for entropy MD and an extension to Legendre regularizers that are central in online optimization and RL\. We present two Bregman\-regularized variants of MD for mitigating initialization instability\. The first,*Initialization\-Anchored MD*, adds a Bregman regularization term centered at the initialization\. In the well\-conditioned setting, where the regularizer is also smooth, it achieves initialization stabilityO​\(ε\+1/T​log⁡T\)O\(\\varepsilon\+1/\\sqrt\{T\\log T\}\)and optimization errorO​\(log⁡\(T\)/T\)O\(\\log\(T\)/T\), however, its guarantees may become vacuous for ill\-conditioned regularizers\. The second variant,*Fixed\-Anchor MD*, adds a Bregman regularization term centered at a fixed reference point independent of the initialization\. This method also handles ill\-conditioned regularizers, achieving initialization stabilityO​\(1/T\)O\(1/T\)while preserving the convergence guarantees of MD up to logarithmic factors, with optimization errorO​\(log⁡\(T\)/T\)O\(\\log\(T\)/T\)\.

AlgorithmTypeDomainRegularizerAssumptionsBoundReferenceMDUpperConvexQuadratic,κ\\kappa\-conditionedη≤αL\\eta\\leq\\frac\{\\alpha\}\{L\}O​\(βα​ε\)O\\left\(\\frac\{\\beta\}\{\\alpha\}\\,\\varepsilon\\right\)Hardtet al\.\([2016](https://arxiv.org/html/2606.11431#bib.bib17)\)\(see[Theorem10](https://arxiv.org/html/2606.11431#Thmtheorem10)\)MDLowerConvexEuclidean,κ=O​\(1\)\\kappa=O\(1\), nonquadratic–Ω​\(min⁡\{polylog−1⁡\(1ε\),ε​eΩ​\(η​T\)\}\)\\Omega\(\\min\\\!\\left\\\{\\operatorname\{polylog\}^\{\-1\}\(\\frac\{1\}\{\\varepsilon\}\),\\varepsilon e^\{\\Omega\(\\eta T\)\}\\right\\\}\)[Theorem1](https://arxiv.org/html/2606.11431#Thmtheorem1)MDLowerSimplexNegative entropyd≥1εd\\geq\\frac\{1\}\{\\varepsilon\}orw0min≤εw\_\{0\}^\{\\min\}\\leq\\varepsilonΩ​\(ε​eη​T\)\\Omega\\\!\\left\(\\varepsilon e^\{\\eta T\}\\right\)[Theorem2](https://arxiv.org/html/2606.11431#Thmtheorem2)MDUpperSimplexNegative entropy–O​\(ε​eO​\(η​T\)\)O\\\!\\left\(\\varepsilon e^\{O\(\\eta T\)\}\\right\)[Theorem3](https://arxiv.org/html/2606.11431#Thmtheorem3)Init\.\-Anchor MD \([Algorithm1](https://arxiv.org/html/2606.11431#alg1)\)UpperConvexκ=O​\(1\)\\kappa=O\(1\)–O​\(ε\+1/T​log⁡T\)O\\\!\\left\(\\varepsilon\+1/\\sqrt\{T\\log T\}\\right\)[Theorem4](https://arxiv.org/html/2606.11431#Thmtheorem4)Fixed\-Anchor MD \([Algorithm2](https://arxiv.org/html/2606.11431#alg2)\)UpperConvex––O​\(1/T\)O\\\!\\left\(1/T\\right\)[Theorem6](https://arxiv.org/html/2606.11431#Thmtheorem6)Figure 1:Summary of initialization\-stability bounds\. Herew0w\_\{0\}denotes the initialization,w0minw\_\{0\}^\{\\min\}the minimal coordinate ofw0w\_\{0\},ε\\varepsilondenotes the initialization perturbation,TTthe optimization horizon,η\\etathe step size,LLthe smoothness parameter of the objective, andα,β\\alpha,\\betathe strong\-convexity and smoothness parameters of the regularizer andκ=β/α\\kappa=\\beta/\\alpha\.
### 1\.2Related work

##### Mirror descent and non\-Euclidean optimization\.

Mirror Descent has been central to optimization and online learning for several decades; see, e\.g\.,Shalev\-Shwartz \([2025](https://arxiv.org/html/2606.11431#bib.bib634)\); Bubeck \([2015](https://arxiv.org/html/2606.11431#bib.bib636)\); Hazan \([2016](https://arxiv.org/html/2606.11431#bib.bib637)\); Beck \([2017](https://arxiv.org/html/2606.11431#bib.bib638)\)for textbook and survey treatments\. Recent work continues to refine its optimization and regret guarantees, including stochastic MD for relatively smooth objectives\(D’Orazioet al\.,[2021](https://arxiv.org/html/2606.11431#bib.bib633)\)and online MD with approximate updates\(Schlisselberget al\.,[2025](https://arxiv.org/html/2606.11431#bib.bib4)\)\. Other works study the implicit bias of MD: in stochastic overparameterized problems,Azizanet al\.\([2021](https://arxiv.org/html/2606.11431#bib.bib631)\)show that MD converges to a global minimizer that is approximately closest to the initialization in Bregman divergence, whileSunet al\.\([2022](https://arxiv.org/html/2606.11431#bib.bib632),[2023](https://arxiv.org/html/2606.11431#bib.bib635)\)characterize its max\-margin bias for linearly separable classification\. Our focus is different: we study the dynamical sensitivity of MD to perturbations in its initialization\.

##### Stability of optimization algorithms\.

Most stability analyses concern*uniform stability*\(Bousquet and Elisseeff,[2002](https://arxiv.org/html/2606.11431#bib.bib32)\), where two runs differ in one component of a finite\-sum objective\. In this setting,Shalev\-Shwartzet al\.\([2010](https://arxiv.org/html/2606.11431#bib.bib31)\)proved stability of strongly convex empirical risk minimization, andHardtet al\.\([2016](https://arxiv.org/html/2606.11431#bib.bib17)\); Lei and Ying \([2020](https://arxiv.org/html/2606.11431#bib.bib626)\)extended stability guarantees to GD and SGD under smoothness assumptions\. These works also imply favorable initialization stability for Euclidean gradient methods\. We show that this behavior does not extend to general MD: even for convex smooth objectives, nonquadratic mirror maps can exponentially amplify initialization perturbations\.

##### Lower bounds and noncontractivity\.

Several works show that stability can fail for algorithms outside the basic GD template\. For nonsmooth objectives,Bassilyet al\.\([2020](https://arxiv.org/html/2606.11431#bib.bib630)\)prove lower bounds for GD and SGD\. In smooth settings,Attia and Koren \([2021](https://arxiv.org/html/2606.11431#bib.bib627)\)show that accelerated methods can exhibit exponential initialization instability, andSchlisermanet al\.\([2025](https://arxiv.org/html/2606.11431#bib.bib625)\)prove polynomial instability bounds for sharpness\-aware methods such as SAM\. These algorithms are not standard MD\. Closer to our setting,Asiet al\.\([2021](https://arxiv.org/html/2606.11431#bib.bib644)\)show that MD can be non\-contractive in a single update\. We prove a stronger dynamical statement: the expansion can persist forTTsteps and become exponential inTT, already under convex smooth objectives\.

##### Reproducibility and stabilization\.

A related line of work studies reproducibility under noisy or inexact operations, including inexact initialization\.Ahnet al\.\([2022](https://arxiv.org/html/2606.11431#bib.bib628)\)prove upper and lower bounds for variants of GD and SGD in convex optimization, andZhanget al\.\([2023](https://arxiv.org/html/2606.11431#bib.bib629)\)propose a black\-box stabilization reduction based on Euclidean regularization\. Another line of work shows that additional regularization can improve uniform stability without degrading optimization guarantees\(Attia and Koren,[2022](https://arxiv.org/html/2606.11431#bib.bib2); Varyet al\.,[2024](https://arxiv.org/html/2606.11431#bib.bib624)\)\. Our lower bounds show that non\-Euclidean MD has qualitatively different initialization sensitivity, and our regularized algorithm extends the regularization\-based stabilization perspective to general mirror maps and initialization stability\.

## 2Preliminaries

##### Mirror Descent\.

We study the general mirror descent \(MD\) method\(Nemirovski and Yudin,[1983](https://arxiv.org/html/2606.11431#bib.bib295); Beck and Teboulle,[2003](https://arxiv.org/html/2606.11431#bib.bib296)\)inℝd\\mathbb\{R\}^\{d\}equipped with a norm∥⋅∥\\\|\\cdot\\\|\. Let‖x‖∗=sup‖y‖≤1⟨x,y⟩\\\|x\\\|\_\{\*\}=\\sup\_\{\\\|y\\\|\\leq 1\}\\langle x,y\\rangledenote the dual norm\. Given a regularization functionR:𝒲→ℝR:\\mathcal\{W\}\\to\\mathbb\{R\}, and starting from initializationw0∈𝒲w\_\{0\}\\in\\mathcal\{W\}, the mirror descent method takes updates of the form

wt\+1=arg⁡minw∈𝒲⁡\{η​⟨∇F​\(wt\),w⟩\+DR​\(w,wt\)\},w\_\{t\+1\}=\\arg\\min\_\{w\\in\\mathcal\{W\}\}\\,\\bigl\\\{\\,\\eta\\langle\\nabla F\(w\_\{t\}\),w\\rangle\+D\_\{R\}\(w,w\_\{t\}\)\\,\\bigr\\\},whereF:𝒲→ℝF:\\mathcal\{W\}\\to\\mathbb\{R\}is the objective function,η\>0\\eta\>0is the step\-size, andDR​\(w,wt\)=R​\(w\)−R​\(wt\)−⟨∇R​\(wt\),w−wt⟩D\_\{R\}\(w,w\_\{t\}\)=R\(w\)\-R\(w\_\{t\}\)\-\\langle\\nabla R\(w\_\{t\}\),w\-w\_\{t\}\\rangledenotes the Bregman divergence induced byRR\.

As is standard in mirror descent analyses, we assume thatRRisα\\alpha\-strongly convex with respect to the norm∥⋅∥\\\|\\cdot\\\|\.111A functionH:𝒲→ℝH:\\mathcal\{W\}\\to\\mathbb\{R\}isα\\alpha\-strongly convex with respect to∥⋅∥\\\|\\cdot\\\|ifH​\(w\)≥H​\(w′\)\+∇H​\(w′\)⊤​\(w−w′\)\+μ2​‖w−w′‖2H\(w\)\\geq H\(w^\{\\prime\}\)\+\\nabla H\(w^\{\\prime\}\)^\{\\top\}\(w\-w^\{\\prime\}\)\+\\frac\{\\mu\}\{2\}\\\|w\-w^\{\\prime\}\\\|^\{2\}for allw,w′∈𝒲w,w^\{\\prime\}\\in\\mathcal\{W\}\.We further assume that the objectiveFFis convex,GG\-Lipschitz,222A functionH:𝒲→ℝH:\\mathcal\{W\}\\to\\mathbb\{R\}isGG\-Lipschitz with respect to∥⋅∥\\\|\\cdot\\\|if\|F​\(w\)−F​\(w′\)\|≤G​‖w−w′‖\|F\(w\)\-F\(w^\{\\prime\}\)\|\\leq G\\\|w\-w^\{\\prime\}\\\|for allw,w′∈𝒲w,w^\{\\prime\}\\in\\mathcal\{W\}\.andLL\-smooth,333A functionH:𝒲→ℝH:\\mathcal\{W\}\\to\\mathbb\{R\}isLL\-smooth with respect to∥⋅∥\\\|\\cdot\\\|if‖∇F​\(w\)−∇F​\(w′\)‖∗≤L​‖w−w′‖\\\|\\nabla F\(w\)\-\\nabla F\(w^\{\\prime\}\)\\\|\_\{\*\}\\leq L\\\|w\-w^\{\\prime\}\\\|for allw,w′∈𝒲w,w^\{\\prime\}\\in\\mathcal\{W\}\.both with respect to the norm∥⋅∥\\\|\\cdot\\\|, and that the domain𝒲\\mathcal\{W\}is compact with diameterD=supu,v∈𝒲‖u−v‖D=\\sup\_\{u,v\\in\\mathcal\{W\}\}\\\|u\-v\\\|\.

##### Initialization stability\.

We study the stability of mirror descent with respect to its initialization point\. The notion of initialization stability\(Attia and Koren,[2021](https://arxiv.org/html/2606.11431#bib.bib627)\)measures the worst\-case sensitivity of the algorithm’s output to a small perturbation in its initial point; formally,

###### Definition 1\(Initialization stability\)\.

LetAAbe an algorithm that given an initializationw0w\_\{0\}producesA​\(w0\)A\(w\_\{0\}\)as an output\. For anyε\>0\\varepsilon\>0,ε\\varepsilon\-initialization stability ofAAatw0∈𝒲w\_\{0\}\\in\\mathcal\{W\}is given by

δA​\(w0,ε\):=supp:‖p‖≤ε,w0\+p∈𝒲‖A​\(w0\)−A​\(w0\+p\)‖\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\,:=\\,\\sup\_\{\\begin\{subarray\}\{c\}p:\\,\\\|p\\\|\\leq\\varepsilon,\\\\ w\_\{0\}\+p\\in\\mathcal\{W\}\\end\{subarray\}\}\\bigl\\\|A\(w\_\{0\}\)\-A\(w\_\{0\}\+p\)\\bigr\\\|\.

This is a local worst\-case sensitivity notion for the algorithmic mapw0↦A​\(w0\)w\_\{0\}\\mapsto A\(w\_\{0\}\)\. Thus initialization stability can be viewed as adversarial robustness, or equivalently finite\-time sensitivity to initial conditions, with respect to worst\-case perturbations of the starting point\.

##### Conditioning of the mirror map\.

The geometry induced byRRis controlled by its strong convexity and smoothness with respect to the norm∥⋅∥\\\|\\cdot\\\|\. WhenRRisβ\\beta\-smooth in addition to beingα\\alpha\-strongly convex, we define its*condition number*byκ=β/α\\kappa=\\beta/\\alpha\. We call the geometry*well\-conditioned*whenκ\\kappais bounded by an absolute constant, and*ill\-conditioned*when this quantity is large or unbounded\.

For ill\-conditioned regularizers, we quantify theε\\varepsilon\-local smoothness of the regularizerRRaround a point in the optimization domain\. This quantity, specifically at the initial point of MD, is shown to play a central role in the stability analysis of MD\.

###### Definition 2\(ε\\varepsilon\-local smoothness of regularizer\)\.

For a pointw0∈𝒲w\_\{0\}\\in\\mathcal\{W\}and radiusε\>0\\varepsilon\>0, define theε\\varepsilon\-local smoothness ofRRatw0w\_\{0\}by

β​\(w0,ε\):=1ε​supp:‖p‖≤ε,w0\+p∈𝒲‖∇R​\(w0\+p\)−∇R​\(w0\)‖∗\.\\beta\(w\_\{0\},\\varepsilon\)\\,:=\\,\\frac\{1\}\{\\varepsilon\}\\sup\_\{\\begin\{subarray\}\{c\}p:\\,\\\|p\\\|\\leq\\varepsilon,\\\\ w\_\{0\}\+p\\in\\mathcal\{W\}\\end\{subarray\}\}\\bigl\\\|\\nabla R\(w\_\{0\}\+p\)\-\\nabla R\(w\_\{0\}\)\\bigr\\\|\_\{\*\}\.

## 3Instability in Well\-conditioned Euclidean Geometries

We begin by showing that an exponential lower bound on the initialization stability of MD in a well conditioned case in Euclidean geometry, i\.e\., when the regularizerRRhas condition numberO​\(1\)O\(1\)with respect to the Euclidean norm∥⋅∥2\\\|\\cdot\\\|\_\{2\}\. The following theorem establishes this already in dimensiond=3d=3, with a convex and smooth objective and domain𝒲⊂ℝ3\\mathcal\{W\}\\subset\\mathbb\{R\}^\{3\}contained in a ball of diameterO​\(1\)O\(1\), all with respect to∥⋅∥2\\\|\\cdot\\\|\_\{2\}\. Throughout this section and its proof, matrix and multilinear\-map norms are the operator norms induced by the Euclidean norm\.

###### Theorem 1\.

There exist absolute constantsc,C0,η0,ε0\>0,c,C\_\{0\},\\eta\_\{0\},\\varepsilon\_\{0\}\>0,such that the following holds\. For every0≤ε≤ε00\\leq\\varepsilon\\leq\\varepsilon\_\{0\}, every0<η≤η00<\\eta\\leq\\eta\_\{0\}, and every integerT≥1T\\geq 1, there exist a convex feasible set𝒲⊂ℝ3\\mathcal\{W\}\\subset\\mathbb\{R\}^\{3\}with diameter at most33, a regularizerR:𝒲→ℝR:\\mathcal\{W\}\\to\\mathbb\{R\}, a convex objectiveF:𝒲→ℝF:\\mathcal\{W\}\\to\\mathbb\{R\}, and a unit vectorv∈ℝ3v\\in\\mathbb\{R\}^\{3\}, such thatRRis11\-strongly convex andC0C\_\{0\}\-smooth on𝒲\\mathcal\{W\},FFis convex,11\-Lipschitz, and11\-smooth on𝒲\\mathcal\{W\}, such that ifAAis the MD algorithm with regularizationRRapplied forTTsteps with step sizeη\\eta, then

δA​\(0,ε\)=Ω​\(min⁡\{1\(1\+log⁡\(1/ε\)\)3,ec​η​T​ε\}\)\.\\delta\_\{A\}\(0,\\varepsilon\)=\\Omega\\left\(\\min\\left\\\{\\frac\{1\}\{\(1\+\\log\(1/\\varepsilon\)\)^\{3\}\},e^\{c\\eta T\}\\varepsilon\\right\\\}\\right\)\.

In particular, for the canonical choiceη=Θ​\(1/T\)\\eta=\\Theta\(1/\\sqrt\{T\}\), under which MD attains its standardO​\(1/T\)O\(1/\\sqrt\{T\}\)optimization guarantee, the bound is still exponential inTTand becomes:

δA​\(0,ε\)=Ω​\(min⁡\{1\(1\+log⁡\(1/ε\)\)3,eΘ​\(T\)​ε\}\)\.\\delta\_\{A\}\(0,\\varepsilon\)=\\Omega\\left\(\\min\\left\\\{\\frac\{1\}\{\(1\+\\log\(1/\\varepsilon\)\)^\{3\}\},e^\{\\Theta\(\\sqrt\{T\}\)\}\\varepsilon\\right\\\}\\right\)\.Thus, with a non\-quadratic regularizer and a non\-linear objective, mirror descent can amplify initial errors at rate exponential inη​T\\eta T, up to a saturation level that depends only poly\-logarithmically onε\\varepsilon\. For example, whenT≲\(1/η\)​log⁡\(1/ε\)T\\lesssim\(\{1\}/\{\\eta\}\)\\log\(1/\\varepsilon\), the exponential lower bound is the dominant term; in particular, takingε=1/Tγ\\varepsilon=1/T^\{\\gamma\}yields exponential growth up to scale\(1\+γ​log⁡T\)−3\(1\+\\gamma\\log T\)^\{\-3\}\. This gives an exponential separation from the standard quadratic\-regularization case, where initialization stability is known to be bounded asO​\(ε\)O\(\\varepsilon\)and does not grow withTT\(for completeness, we provide a proof in[AppendixA](https://arxiv.org/html/2606.11431#A1)\)\. We note that such exponential amplification is considerably easier to obtain with an ill\-conditioned regularizer and an ill\-conditioned initialization; see details in[AppendixB](https://arxiv.org/html/2606.11431#A2)\. Here we address the more benign well\-conditioned setting where reproducing this behavior is more challenging\.

We provide next a sketch of the proof\. The full construction and proof are deferred to[SectionC\.1](https://arxiv.org/html/2606.11431#A3.SS1)\.

zzε​ρT\\varepsilon\\rho^\{T\}wT′w\_\{T\}^\{\\prime\}w0w\_\{0\}wtw\_\{t\}wTw\_\{T\}w0′=ε​u\+w^\{\\prime\}\_\{0\}=\\varepsilon u\_\{\+\}wt′w^\{\\prime\}\_\{t\}Δ​θ\\Delta\\thetaFigure 2:Illustration of the MD trajectories in the proof of[Theorem1](https://arxiv.org/html/2606.11431#Thmtheorem1)\. The trajectory follows the clock coordinatezz, while transversexx\-perturbations grow\. The transverse Hessian rotates along the path; in the co\-moving frame, the dynamics reduce to powers of a fixed matrixAAwith expanding eigenvectoru\+u\_\{\+\}and eigenvalueρ\>1\\rho\>1\.Proof sketch\.Letw0′=w0\+pw\_\{0\}^\{\\prime\}=w\_\{0\}\+pbe a perturbed initialization\. The goal is to construct mirror descent dynamics such that, for the trajectory\{wt′\}t=0T\\\{w\_\{t\}^\{\\prime\}\\\}\_\{t=0\}^\{T\}, the small perturbation of the initialization grows exponentially relative to a reference trajectory\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}\. We denote the MD update rule aswt\+1=Φ​\(wt\)w\_\{t\+1\}=\\Phi\(w\_\{t\}\), where,

Φ​\(w\):=\(∇R\)−1​\(∇R​\(w\)−η​∇F​\(w\)\)\.\\Phi\(w\):=\(\\nabla R\)^\{\-1\}\\bigl\(\\nabla R\(w\)\-\\eta\\nabla F\(w\)\\bigr\)\.\(1\)Then, using linear approximation of the updates, we get

wT′−wT\\displaystyle w\_\{T\}^\{\\prime\}\-w\_\{T\}=Φ​\(wT−1\)−Φ​\(wT−1\)≈D​Φ​\(wT\)⊤​\(wT−1′−wT−1\)\\displaystyle=\\Phi\(w\_\{T\-1\}\)\-\\Phi\(w\_\{T\-1\}\)\\approx D\\Phi\(w\_\{T\}\)^\{\\top\}\(w\_\{T\-1\}^\{\\prime\}\-w\_\{T\-1\}\)and, iterating overttwe get,

wT′−wT≈\(Πt=1T​D​Φ​\(wt\)\)⊤​\(w0′−w0\),w\_\{T\}^\{\\prime\}\-w\_\{T\}\\approx\\left\(\\Pi\_\{t=1\}^\{T\}D\\Phi\(w\_\{t\}\)\\right\)^\{\\top\}\(w\_\{0\}^\{\\prime\}\-w\_\{0\}\),where

D​Φ​\(wt\)=∇2R​\(wt\+1\)−1​\(∇2R​\(wt\)−η​∇2F​\(wt\)\)\.D\\Phi\(w\_\{t\}\)=\\nabla^\{2\}R\(w\_\{t\+1\}\)^\{\-1\}\\bigl\(\\nabla^\{2\}R\(w\_\{t\}\)\-\\eta\\nabla^\{2\}F\(w\_\{t\}\)\\bigr\)\.
As a result, it is sufficient to \(i\) construct an en example where the largest eigenvalue of the matrixΓ=Πt=1T​D​Φ​\(wt\)\\Gamma=\\Pi\_\{t=1\}^\{T\}D\\Phi\(w\_\{t\}\)is exponential inTT\. \(ii\) uniformly bound the linear approximation errors\.

In this sketch we focus on step \(i\) which is the core of the proof\. For this step, we consider a feasible cylinder,

𝒲:=\{\(x,z\)∈ℝ2×ℝ:‖x‖2≤r,−m≤z≤m\},\\mathcal\{W\}:=\\left\\\{\(x,z\)\\in\\mathbb\{R\}^\{2\}\\times\\mathbb\{R\}:\\\|x\\\|\_\{2\}\\leq r,\\;\-m\\leq z\\leq m\\right\\\},for somer,m\>0r,m\>0, and parametrize the coordinates of the iterate asw=\(x,z\)∈ℝ2×ℝw=\(x,z\)\\in\\mathbb\{R\}^\{2\}\\times\\mathbb\{R\}\. The reference trajectory evolves only in thezz\-direction:wt=\(0,zt\)w\_\{t\}=\(0,z\_\{t\}\)\. Thus,\{zt\}t=1T\\\{z\_\{t\}\\\}\_\{t=1\}^\{T\}acts as a clock variable, while the transverse variablex∈ℝ2x\\in\\mathbb\{R\}^\{2\}captures directions orthogonal to thezz\-direction, where the instability is generated\.

To control the eigenvalues ofΓ\\Gamma, we use a rotation matrixP​\(z\)P\(z\)to make the dynamics of the transverse plane rotate around the reference trajectory, such that, at steptt

D​Φ​\(wt\)≈P​\(zt\+1\)​A​P​\(zt\)⊤,D\\Phi\(w\_\{t\}\)\\approx P\(z\_\{t\+1\}\)AP\(z\_\{t\}\)^\{\\top\},for some fixed matrixAA\. Telescoping and multiplying overTTsteps yields,

Γ=P​\(zT\)​AT​P​\(z0\)⊤\.\\Gamma=P\(z\_\{T\}\)A^\{T\}P\(z\_\{0\}\)^\{\\top\}\.Thus, up to outer rotations, the dynamics reduce toATA^\{T\}\. Then, choosing the objectiveFFand the regularizationRRsuch thatAAhas an eigenvalueρ≈1\+η\\rho\\approx 1\+\\eta, will cause an an exponential amplification of the perturbation\. For an illustration of this see[Figure2](https://arxiv.org/html/2606.11431#S3.F2)\.

To achieve this, we construct the followingzz\-dependent quadratic forms,

M​\(z\)=P​\(z\)​B​P​\(z\)⊤,H​\(z\)=P​\(z\)​Q​P​\(z\)⊤,M\(z\)=P\(z\)BP\(z\)^\{\\top\},\\quad H\(z\)=P\(z\)QP\(z\)^\{\\top\},whereB,QB,Qare appropriately chosen symmetric matrices, and define

R​\(x,z\)=12​x⊤​M​\(z\)​x\+Rclock​\(z\),F​\(x,z\)≈12​x⊤​H​\(z\)​x,R\(x,z\)=\\frac\{1\}\{2\}x^\{\\top\}M\(z\)x\+R\_\{\\rm clock\}\(z\),\\quad F\(x,z\)\\approx\\frac\{1\}\{2\}x^\{\\top\}H\(z\)x,whereRclockR\_\{\\rm clock\}is chosen to govern the motion of the reference trajectory in thezz\-direction\. ThoseRRandFFinduce the required exponential matrixΓ\\GammaforA≈B−1​P​\(−η\)​\(B−η​Q\)A\\approx B^\{\-1\}P\(\-\\eta\)\(B\-\\eta Q\)\. Thus, choosing the perturbationppin the direction of the expanding eigenvector ofAAinduces the desired exponential expansion\.

∎

## 4Instability in Entropic Geometry

We next study one of the fundamental examples of mirror descent: negative entropy on the simplex, which underlies multiplicative weights and KL\-regularized updates\. Unlike the regularizers considered in the previous section, negative entropy is ill\-conditioned near the boundary of the simplex\. LetΔd∘=\{w∈ℝ\+\+d:∑iwi=1\}\\Delta\_\{d\}^\{\\circ\}=\\\{w\\in\\mathbb\{R\}\_\{\+\+\}^\{d\}:\\sum\_\{i\}w\_\{i\}=1\\\}denote thedd\-dimensional open simplex, and takeR​\(w\)=∑iwi​log⁡wiR\(w\)=\\sum\_\{i\}w\_\{i\}\\log w\_\{i\}\. The corresponding Bregman divergence is the Kullback–Leibler divergence,DR​\(w,w′\)=∑iwi​log⁡\(wi/wi′\)D\_\{R\}\(w,w^\{\\prime\}\)=\\sum\_\{i\}w\_\{i\}\\log\(w\_\{i\}/w\_\{i\}^\{\\prime\}\)\. Forw∈Δd∘w\\in\\Delta\_\{d\}^\{\\circ\}, writew​\(i\)w\(i\)for itsiith coordinate andwmin=mini⁡w​\(i\)\\smash\{w^\{\\min\}\}=\\min\_\{i\}w\(i\)\.

In this geometry, the instability mechanism becomes especially transparent\. Unlike the well\-conditioned case studied in the preceding section, exponential amplification already occurs for linear objectives\. The mechanism is the nonuniform conditioning of negative entropy: a small primal perturbation in a low\-mass coordinate can correspond to a large displacement in the dual variables, and in high dimension such low\-mass coordinates are present for*every*initialization\. This local ill\-conditioning is captured by the quantityβ​\(w0,ε\)\\beta\(w\_\{0\},\\varepsilon\)\.

###### Theorem 2\.

Letd≥2d\\geq 2,ε∈\(0,1/2\]\\varepsilon\\in\(0,1/2\],η\>0\\eta\>0, andT≥1T\\geq 1\. Then, for every initialization pointw0∈Δd∘w\_\{0\}\\in\\Delta\_\{d\}^\{\\circ\}, there exists a11\-Lipschitz linear objectiveFFsuch that, ifAAis MD with negative entropy regularization applied toFFforTTiterations with step sizeη\\eta, then itsℓ1\\ell\_\{1\}\-initialization stability satisfies

δA​\(w0,ε\)≥15​min⁡\{1,eη​T​ε,εw0min\}=Ω​\(min⁡\{1,eη​T​ε,ε​β​\(w0,ε\)\}\)\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\tfrac\{1\}\{5\}\\min\\Bigl\\\{1,\\,e^\{\\eta T\}\\varepsilon,\\,\\tfrac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\Bigr\\\}=\\Omega\\Bigl\(\\min\\Bigl\\\{1,\\,e^\{\\eta T\}\\varepsilon,\\,\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\Bigr\\\}\\Bigr\)\.In particular, ifd≥1/εd\\geq 1/\\varepsilon, then for any initializationw0∈Δd∘w\_\{0\}\\in\\Delta\_\{d\}^\{\\circ\}it holdsδA​\(w0,ε\)≥15​min⁡\{1,eη​T​ε\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\tfrac\{1\}\{5\}\\min\\\{1,e^\{\\eta T\}\\varepsilon\\\}\.

Specifically for the canonical stepsizeη=Θ​\(log⁡d/T\)\\eta=\\Theta\(\\sqrt\{\\log d/T\}\)the regime where MD achieves itsO​\(1/T\)O\(1/\\sqrt\{T\}\)optimization guarantee withd≥1/εd\\geq 1/\\varepsilon, the bound is still exponential:

δA​\(w0,ε\)≥15​min⁡\{1,eΘ​\(T​log⁡d\)​ε\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\tfrac\{1\}\{5\}\\min\\Bigl\\\{1,\\,e^\{\\Theta\(\\sqrt\{T\\log d\}\)\}\\varepsilon\\Bigr\\\}\.
We remark that it was previously shown that*a single step*of negative\-entropy MD can be non\-contractive, from a specific, near\-boundary initialization\(Asiet al\.,[2021](https://arxiv.org/html/2606.11431#bib.bib644)\)\. In contrast, here we show that such expansion can persist forTTsteps, yielding a lower bound that grows exponentially withTT, and moreover, in the high\-dimensional regime, our bound holds uniformly over all initializations in the open simplexΔd∘\\Delta\_\{d\}^\{\\circ\}\.

The proof uses the multiplicative\-weights form of entropy MD\. We perturb a lowest\-mass coordinate ofw0w\_\{0\}, where a small primal change creates the largest logarithmic change in the dual variables of negative entropy\. This produces two initializations that are close inℓ1\\ell\_\{1\}distance but far in the local dual geometry, with dual separation measured byε​β​\(w0,ε\)\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\. We then choose a linear objective that rewards this coordinate, so that the multiplicative update amplifies this initial dual separation over time\. Thus, the exponential amplification in the lower bound is driven by the local ill\-conditioning of negative entropy inℓ1\\ell\_\{1\}geometry\. In high dimension, every point in the simplex has a coordinate of mass at most1/d1/d, so this ill\-conditioning is present for every initialization\. More generally,[AppendixB](https://arxiv.org/html/2606.11431#A2)shows that such poor conditioning is unavoidable for regularizers compatible with theℓ1\\ell\_\{1\}geometry\.

###### Proof of[Theorem2](https://arxiv.org/html/2606.11431#Thmtheorem2)\.

Letim∈arg⁡mini∈\[d\]⁡w0​\(i\)i\_\{m\}\\in\\arg\\min\_\{i\\in\[d\]\}w\_\{0\}\(i\)andw0​\(im\)=w0minw\_\{0\}\(i\_\{m\}\)=w\_\{0\}^\{\\min\}\. Sinced≥2d\\geq 2, we havew0min≤1/2w\_\{0\}^\{\\min\}\\leq 1/2, and therefore1−w0min≥12\>ε21\-w\_\{0\}^\{\\min\}\\geq\\frac\{1\}\{2\}\>\\frac\{\\varepsilon\}\{2\}\. Thus we may move massε/2\\varepsilon/2from the coordinatesi≠imi\\neq i\_\{m\}to coordinateimi\_\{m\}\. For example, definep∈ℝdp\\in\\mathbb\{R\}^\{d\}by

p​\(im\)=ε2,p​\(i\)=−ε2⋅w0​\(i\)1−w0minfor​i≠im\.p\(i\_\{m\}\)=\\frac\{\\varepsilon\}\{2\},\\qquad p\(i\)=\-\\frac\{\\varepsilon\}\{2\}\\cdot\\frac\{w\_\{0\}\(i\)\}\{1\-w\_\{0\}^\{\\min\}\}\\quad\\text\{for \}i\\neq i\_\{m\}\.Then∑i=1dp​\(i\)=0\\sum\_\{i=1\}^\{d\}p\(i\)=0, and sinceε/2<1−w0min\\varepsilon/2<1\-w\_\{0\}^\{\\min\}, we havew0​\(i\)\+p​\(i\)\>0w\_\{0\}\(i\)\+p\(i\)\>0for everyi≠imi\\neq i\_\{m\}\. Thereforew0′:=w0\+p∈Δd∘w\_\{0\}^\{\\prime\}:=w\_\{0\}\+p\\in\\Delta\_\{d\}^\{\\circ\}\. Moreover,

‖w0−w0′‖1=‖p‖1=ε2\+∑i≠imε2⋅w0​\(i\)1−w0min=ε\.\\\|w\_\{0\}\-w\_\{0\}^\{\\prime\}\\\|\_\{1\}=\\\|p\\\|\_\{1\}=\\frac\{\\varepsilon\}\{2\}\+\\sum\_\{i\\neq i\_\{m\}\}\\frac\{\\varepsilon\}\{2\}\\cdot\\frac\{w\_\{0\}\(i\)\}\{1\-w\_\{0\}^\{\\min\}\}=\\varepsilon\.
Now defineK:=min⁡\{eη​T,1w0min,1ε\}\.K:=\\min\\left\\\{e^\{\\eta T\},\\frac\{1\}\{w\_\{0\}^\{\\min\}\},\\frac\{1\}\{\\varepsilon\}\\right\\\}\.LetFFbe the linear functionF​\(w\)=⟨g,w⟩F\(w\)=\\langle g,w\\rangle, where

g​\(i\)=\{−log⁡Kη​T,i=im,0,i≠im\.g\(i\)=\\begin\{cases\}\-\\dfrac\{\\log K\}\{\\eta T\},&i=i\_\{m\},\\\\\[10\.00002pt\] 0,&i\\neq i\_\{m\}\.\\end\{cases\}SinceK≤eη​TK\\leq e^\{\\eta T\}, we have0≤log⁡Kη​T≤10\\leq\\frac\{\\log K\}\{\\eta T\}\\leq 1\. Hence‖g‖∞≤1\\\|g\\\|\_\{\\infty\}\\leq 1, soFFis11\-Lipschitz with respect to∥⋅∥1\\\|\\cdot\\\|\_\{1\}\.

For negative entropy mirror descent with a fixed linear loss, the update has the multiplicative\-weights form

wT​\(i\)=w0​\(i\)​e−η​T​g​\(i\)∑j=1dw0​\(j\)​e−η​T​g​\(j\)\.w\_\{T\}\(i\)=\\frac\{w\_\{0\}\(i\)e^\{\-\\eta Tg\(i\)\}\}\{\\sum\_\{j=1\}^\{d\}w\_\{0\}\(j\)e^\{\-\\eta Tg\(j\)\}\}\.Thus coordinateimi\_\{m\}is multiplied byKK, while all other coordinates are multiplied by11\. Therefore, afterTTsteps

wT​\(im\)=K​w0​\(im\)1\+\(K−1\)​w0​\(im\),wT′​\(im\)=K​w0′​\(im\)1\+\(K−1\)​w0′​\(im\)\.w\_\{T\}\(i\_\{m\}\)=\\frac\{Kw\_\{0\}\(i\_\{m\}\)\}\{1\+\(K\-1\)w\_\{0\}\(i\_\{m\}\)\},\\qquad w\_\{T\}^\{\\prime\}\(i\_\{m\}\)=\\frac\{Kw\_\{0\}^\{\\prime\}\(i\_\{m\}\)\}\{1\+\(K\-1\)w\_\{0\}^\{\\prime\}\(i\_\{m\}\)\}\.Hence

‖wT−wT′‖1≥2​\|wT​\(im\)−w\+T′​\(im\)\|=2​\|K​w0​\(im\)1\+\(K−1\)​w0​\(im\)\+K​w0′​\(im\)1\+\(K−1\)​w0′​\(im\)\|\.\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}\\geq 2\\left\|w\_\{T\}\(i\_\{m\}\)\-w\+T^\{\\prime\}\(i\_\{m\}\)\\right\|=2\\left\|\\frac\{Kw\_\{0\}\(i\_\{m\}\)\}\{1\+\(K\-1\)w\_\{0\}\(i\_\{m\}\)\}\+\\frac\{Kw\_\{0\}^\{\\prime\}\(i\_\{m\}\)\}\{1\+\(K\-1\)w\_\{0\}^\{\\prime\}\(i\_\{m\}\)\}\\right\|\.Plugging inw0​\(im\)=w0minw\_\{0\}\(i\_\{m\}\)=w\_\{0\}^\{\\min\}andw0′​\(im\)=w0min\+ε2w\_\{0\}^\{\\prime\}\(i\_\{m\}\)=w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}gives by direct computation

‖wT−wT′‖1≥K​ε\(1\+\(K−1\)​\(w0min\+ε/2\)\)​\(1\+\(K−1\)​w0min\)\.\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}\\geq\\frac\{K\\varepsilon\}\{\\left\(1\+\(K\-1\)\(w\_\{0\}^\{\\min\}\+\\varepsilon/2\)\\right\)\\left\(1\+\(K\-1\)w\_\{0\}^\{\\min\}\\right\)\}\.
We now bound the denominator\. SinceK≤1/w0minK\\leq 1/w\_\{0\}^\{\\min\},

1\+\(K−1\)​w0min≤1\+K​w0min≤2\.1\+\(K\-1\)w\_\{0\}^\{\\min\}\\leq 1\+Kw\_\{0\}^\{\\min\}\\leq 2\.Also, sinceK≤1/w0minK\\leq 1/w\_\{0\}^\{\\min\}andK≤1/εK\\leq 1/\\varepsilon,

1\+\(K−1\)​\(w0min\+ε2\)≤1\+K​w0min\+K​ε2≤1\+1\+12=52\.1\+\(K\-1\)\\left\(w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\\right\)\\leq 1\+Kw\_\{0\}^\{\\min\}\+\\frac\{K\\varepsilon\}\{2\}\\leq 1\+1\+\\frac\{1\}\{2\}=\\frac\{5\}\{2\}\.Thus

‖wT−wT′‖1≥K​ε5=15​min⁡\{1,ε​eη​T,εw0min\}\.\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}\\geq\\frac\{K\\varepsilon\}\{5\}=\\frac\{1\}\{5\}\\min\\left\\\{1,\\,\\varepsilon e^\{\\eta T\},\\,\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}\.as claimed\. It remains to relate the termε/w0min\\varepsilon/w\_\{0\}^\{\\min\}to the local smoothness parameterβ​\(w0,ε\)\\beta\(w\_\{0\},\\varepsilon\)\. For negative entropy,∇R​\(w\)​\(i\)=1\+log⁡w​\(i\),\\nabla R\(w\)\(i\)=1\+\\log w\(i\),and therefore

ε​β​\(w0,ε\)=supp:‖p‖1≤ε,w0\+p∈Δd∘‖log⁡\(w0\+p\)−log⁡\(w0\)‖∞\.\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)=\\sup\_\{\\begin\{subarray\}\{c\}p:\\,\\\|p\\\|\_\{1\}\\leq\\varepsilon,\\\\ w\_\{0\}\+p\\in\\Delta\_\{d\}^\{\\circ\}\\end\{subarray\}\}\\left\\\|\\log\(w\_\{0\}\+p\)\-\\log\(w\_\{0\}\)\\right\\\|\_\{\\infty\}\.Ifε≥2​w0min\\varepsilon\\geq 2w\_\{0\}^\{\\min\}, then the perturbation ball can approach the boundary of the simplex, and henceβ​\(w0,ε\)=∞\\beta\(w\_\{0\},\\varepsilon\)=\\infty\. In this case,

min⁡\{1,εw0min\}=1=min⁡\{1,ε​β​\(w0,ε\)\},\\min\\left\\\{1,\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}=1=\\min\\\{1,\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\\},so the desired inequality holds\. Ifε<2​w0min\\varepsilon<2w\_\{0\}^\{\\min\},

ε​β​\(w0,ε\)=log⁡\(w0minw0min−ε/2\)=log⁡\(11−ε2​w0min\)\.\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)=\\log\\left\(\\frac\{w\_\{0\}^\{\\min\}\}\{w\_\{0\}^\{\\min\}\-\\varepsilon/2\}\\right\)=\\log\\left\(\\frac\{1\}\{1\-\\frac\{\\varepsilon\}\{2w\_\{0\}^\{\\min\}\}\}\\right\)\.Ifε2​w0min≥1/2\\frac\{\\varepsilon\}\{2w\_\{0\}^\{\\min\}\}\\geq 1/2, then

min⁡\{1,εw0min\}=1≥12​min⁡\{1,ε​β​\(w0,ε\)\}\.\\min\\left\\\{1,\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}=1\\geq\\frac\{1\}\{2\}\\min\\\{1,\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\\}\.Ifε2​w0min<1/2\\frac\{\\varepsilon\}\{2w\_\{0\}^\{\\min\}\}<1/2, then using

log⁡\(11−x\)≤x1−x≤2​x,0≤x≤12,\\log\\left\(\\frac\{1\}\{1\-x\}\\right\)\\leq\\frac\{x\}\{1\-x\}\\leq 2x,\\quad 0\\leq x\\leq\\frac\{1\}\{2\},we obtain

12​min⁡\{1,ε​β​\(w0,ε\)\}≤12​ε​β​\(w0,ε\)≤ε2​w0min≤εw0min=min⁡\{1,εw0min\}\.\\frac\{1\}\{2\}\\min\\\{1,\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\\}\\leq\\frac\{1\}\{2\}\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\leq\\frac\{\\varepsilon\}\{2w\_\{0\}^\{\\min\}\}\\leq\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}=\\min\\left\\\{1,\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}\.Combining the cases gives

min⁡\{1,εw0min\}≥12​min⁡\{1,ε​β​\(w0,ε\)\},\\min\\left\\\{1,\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}\\geq\\frac\{1\}\{2\}\\min\\\{1,\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\\},which concludes the proof\. ∎

For completeness, we also state a matching exponential upper bound for the initialization instability of entropy\-regularized mirror descent\.

###### Theorem 3\.

LetT≥0T\\geq 0,η\>0\\eta\>0, andε\>0\\varepsilon\>0\. For anyGG\-Lipschitz andLL\-smooth objectiveFFwith respect to theℓ1\\ell\_\{1\}norm, ifAAis MD with negative entropy regularization applied toFFforTTsteps with step sizeη\\eta, then for any initialization pointw0∈Δd∘w\_\{0\}\\in\\Delta\_\{d\}^\{\\circ\},

δA​\(w0,ε\)≤min⁡\{2,e\(2​G\+4​L\)​η​T​ε\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\leq\\min\\left\\\{2,e^\{\(2G\+4L\)\\eta T\}\\varepsilon\\right\\\}\.

The proof, deferred to[AppendixD](https://arxiv.org/html/2606.11431#A4), relies only on the multiplicative\-weights form of entropy MD and the Lipschitzness and smoothness ofFF, and in particular applies to non\-convex objectives satisfying these regularity assumptions\.

We also give an analogous extension for MD with Legendre regularizers \(see[Definition3](https://arxiv.org/html/2606.11431#Thmdefinition3)in[AppendixA](https://arxiv.org/html/2606.11431#A1)\), a class that is central in online optimization and RL\(Cesa\-Bianchi and Lugosi,[2006](https://arxiv.org/html/2606.11431#bib.bib645)\)\.

## 5Stabilizing Mirror Descent Through Anchoring

In this section, we discuss how to mitigate the instability of MD demonstrated in[Sections3](https://arxiv.org/html/2606.11431#S3)and[4](https://arxiv.org/html/2606.11431#S4)\. Our approach is to modify MD by adding a Bregman regularization term centered at a reference point, denoted as*the anchor*, rather than applying MD directly to the original objective\. We propose two variants of such anchored methods and show that the resulting stability guarantees depend crucially on the choice of the reference point\.

### 5\.1Anchoring at the Initialization

We first consider the case where the anchor is the initialization itself, which, as discussed in the introduction, is closely related to common regularization schemes used in real\-world applications\. In particular, we study*Initialization\-Anchored MD*, described in[Algorithm1](https://arxiv.org/html/2606.11431#alg1)\.

Algorithm 1Initialization\-Anchored Mirror DescentInput:

LL\-smooth function

FF, initialization

w0∈𝒲w\_\{0\}\\in\\mathcal\{W\}, regularization parameter

μ\\mu, no\. of steps

TT\.

for

t←0t\\leftarrow 0to

T−1T\-1do

wt\+1←arg⁡minw∈𝒲⁡\{⟨∇F​\(wt\)\+μ​\(∇R​\(wt\)−∇R​\(w0\)\),w−wt⟩\+\(μ\+L\)​DR​\(w,wt\)\}\.w\_\{t\+1\}\\leftarrow\\arg\\min\_\{w\\in\\mathcal\{W\}\}\\left\\\{\\left\\langle\\nabla F\(w\_\{t\}\)\+\\mu\\bigl\(\\nabla R\(w\_\{t\}\)\-\\nabla R\(w\_\{0\}\)\\bigr\),w\-w\_\{t\}\\right\\rangle\+\(\\mu\+L\)D\_\{R\}\(w,w\_\{t\}\)\\right\\\}\.
endfor

The following theorem shows that[Algorithm1](https://arxiv.org/html/2606.11431#alg1)achieves low initialization stability while preserving the optimization guarantees of vanilla MD up to logarithmic factors\. In particular, rather than exhibiting exponential dependence onε\\varepsilon, the stability bound consists of two terms that are controlled in the well\-conditioned case, in the well\-conditioned case, whereRRis both strongly convex and smooth: one measuring the sensitivity of the anchor to perturbations in the initialization, and another that decays with the number of iterations\.

###### Theorem 4\.

AssumeT≥2T\\geq 2,L\>0L\>0, andFFis convex,GG\-Lipschitz, andLL\-smooth with respect to∥⋅∥\\\|\\cdot\\\|on a convex domain𝒲\\mathcal\{W\}of diameterDD\. Assume also thatRRis11\-strongly convex with respect to∥⋅∥\\\|\\cdot\\\|\. LetAAbe[Algorithm1](https://arxiv.org/html/2606.11431#alg1)run withμ=8​L​log⁡\(T\)/T\.\\mu=\{8L\\log\(T\)\}/\{T\}\.Then, for any initializationw0∈𝒲w\_\{0\}\\in\\mathcal\{W\},

δA​\(w0,ε\)≤ε​β​\(w0,ε\)\+G​DL​T​log⁡T\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\leq\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\+\\sqrt\{\\frac\{GD\}\{LT\\log T\}\}\.If, in addition,RRisβ\\beta\-smooth with respect to∥⋅∥\\\|\\cdot\\\|,

δA​\(w0,ε\)≤β​ε\+G​DL​T​log⁡T\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\leq\\beta\\varepsilon\+\\sqrt\{\\frac\{GD\}\{LT\\log T\}\}\.Moreover, ifw⋆∈arg⁡minw∈𝒲⁡F​\(w\)w^\{\\star\}\\in\\arg\\min\_\{w\\in\\mathcal\{W\}\}F\(w\), then the outputwT=A​\(w0\)w\_\{T\}=A\(w\_\{0\}\)satisfies

F​\(wT\)−F​\(w⋆\)≤8​L​DR​\(w⋆,w0\)​log⁡TT\+2​G​DT2\.F\(w\_\{T\}\)\-F\(w^\{\\star\}\)\\leq\\frac\{8LD\_\{R\}\(w^\{\\star\},w\_\{0\}\)\\log T\}\{T\}\+\\frac\{2GD\}\{T^\{2\}\}\.

The proof, deferred to[AppendixE](https://arxiv.org/html/2606.11431#A5), builds on the relative smoothness framework ofBauschkeet al\.\([2017](https://arxiv.org/html/2606.11431#bib.bib643)\); Luet al\.\([2018](https://arxiv.org/html/2606.11431#bib.bib3)\); Attia and Koren \([2022](https://arxiv.org/html/2606.11431#bib.bib2)\)\. The update rule can be viewed as Mirror Descent applied to the anchored objectiveF​\(w\)\+μ​DR​\(w,w0\),F\(w\)\+\\mu D\_\{R\}\(w,w\_\{0\}\),which isμ\\mu\-strongly convex relative toRR\. Consequently, each trajectory contracts toward the minimizer of its corresponding anchored objective\. To compare two runs initialized atw0w\_\{0\}andw0\+pw\_\{0\}\+p, we decompose their final distance into three terms: the distance of each trajectory to the minimizer of its own anchored objective, and the distance between the two anchored minimizers\. The first and third terms decay withTT, while the second captures the discrepancy induced by the different anchors\. This term is controlled byε​β​\(w0,ε\)\.\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\.Therefore, whenRRis additionallyβ\\beta\-smooth, the discrepancy is bounded byβ​ε\\beta\\varepsilon, yielding a meaningful stabilization guarantee\.

[Theorem4](https://arxiv.org/html/2606.11431#Thmtheorem4)shows that anchoring MD at the initialization resolves the instability problem in the well\-conditioned case, whereRRis both strongly convex and smooth\. However, it does not resolve the ill\-conditioned case\. IfRRis not smooth, the quantityε​β​\(w0,ε\)\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)can be large even when‖p‖\\\|p\\\|is small\. The following theorem shows that this dependence onε​β​\(w0,ε\)\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)is unavoidable for Initialization\-Anchored MD\. In particular, for negative entropy the method can remain exponentially unstable in precisely the ill\-conditioned regimes whereε​β​\(w0,ε\)\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)is large\. The proof is deferred to[SectionE\.2](https://arxiv.org/html/2606.11431#A5.SS2)\.

###### Theorem 5\(Lower bound for initialization\-anchored entropy MD\)\.

Letd≥2d\\geq 2,ε∈\(0,1/2\]\\varepsilon\\in\(0,1/2\],L\>0L\>0,μ\>0\\mu\>0, andT≥1T\\geq 1\. LetRRbe the negative entropy onΔd∘\\Delta\_\{d\}^\{\\circ\}\. LetAAbe[Algorithm1](https://arxiv.org/html/2606.11431#alg1)run with this regularizer overΔd∘\\Delta\_\{d\}^\{\\circ\}\. Then, for every initializationw0∈Δd∘w\_\{0\}\\in\\Delta\_\{d\}^\{\\circ\}, there exists a11\-Lipschitz linear objectiveFF, which may depend onw0,ε,L,μ,Tw\_\{0\},\\varepsilon,L,\\mu,T, such that,

δA​\(w0,ε\)≥110​min⁡\{1,ε​exp⁡\(\(1−e−1\)​min⁡\{TL\+μ,1μ\}\),ε​β​\(w0,ε\)\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\;\\geq\\frac\{1\}\{10\}\\min\\left\\\{1,\\,\\varepsilon\\exp\\left\(\(1\-e^\{\-1\}\)\\min\\left\\\{\\frac\{T\}\{L\+\\mu\},\\frac\{1\}\{\\mu\}\\right\\\}\\right\),\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\right\\\}\.If additionallyd≥1/εd\\geq 1/\\varepsilonthen,

δA​\(w0,ε\)≥15​min⁡\{1,ε​exp⁡\(\(1−e−1\)​min⁡\{TL\+μ,1μ\}\)\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\frac\{1\}\{5\}\\min\\left\\\{1,\\,\\varepsilon\\exp\\left\(\(1\-e^\{\-1\}\)\\min\\left\\\{\\frac\{T\}\{L\+\\mu\},\\frac\{1\}\{\\mu\}\\right\\\}\\right\)\\right\\\}\.

Notably, the optimization\-relevant regime is whenμ\\muis small enough that the anchoring term does not dominate the original objective; in particular, for vanishing optimization error one typically takesμ\\muwhich decreases withTT\. For example, whenμ=1/T\\mu=1/\\sqrt\{T\}, the bound becomeΩ​\(1,ε​eΩ​\(T\)\)\\Omega\(1,\\,\\varepsilon e^\{\\Omega\(\\sqrt\{T\}\)\}\)\.

### 5\.2Anchoring at a Fixed Point

The theorem above shows that anchoring at the initialization does not preclude exponentially\-increasing initialization stability in the ill\-conditioned initialization case\. To address this, we introduce a second variant in which the anchor is fixed independently of the initialization\.

Algorithm 2Fixed\-Anchor Mirror DescentInput:Anchor

wa∈𝒲w\_\{a\}\\in\\mathcal\{W\},

LL\-smooth function

FF, initialization

w0∈𝒲w\_\{0\}\\in\\mathcal\{W\}, regularization parameter

μ\\mu, number of steps

TT\.

for

t←0t\\leftarrow 0to

T−1T\-1do

wt\+1←arg⁡minw∈𝒲⁡\{⟨∇F​\(wt\)\+μ​\(∇R​\(wt\)−∇R​\(wa\)\),w−wt⟩\+\(μ\+L\)​DR​\(w,wt\)\}\.w\_\{t\+1\}\\leftarrow\\arg\\min\_\{w\\in\\mathcal\{W\}\}\\left\\\{\\left\\langle\\nabla F\(w\_\{t\}\)\+\\mu\\bigl\(\\nabla R\(w\_\{t\}\)\-\\nabla R\(w\_\{a\}\)\\bigr\),w\-w\_\{t\}\\right\\rangle\+\(\\mu\+L\)D\_\{R\}\(w,w\_\{t\}\)\\right\\\}\.
endfor

In particular, we study*Fixed\-Anchor MD*, described in[Algorithm2](https://arxiv.org/html/2606.11431#alg2), where the additional Bregman regularization term is centered at a fixed reference pointwaw\_\{a\}\. The following theorem shows that, in contrast to[Algorithm1](https://arxiv.org/html/2606.11431#alg1),[Algorithm2](https://arxiv.org/html/2606.11431#alg2)is stable even in the ill\-conditioned setting\.

###### Theorem 6\.

AssumeT≥2T\\geq 2,L\>0L\>0, andFFis convex,GG\-Lipschitz, andLL\-smooth with respect to∥⋅∥\\\|\\cdot\\\|on a convex domain𝒲\\mathcal\{W\}\. Assume also thatRRis11\-strongly convex with respect to∥⋅∥\\\|\\cdot\\\|\. LetAAbe[Algorithm2](https://arxiv.org/html/2606.11431#alg2)run withμ=8​L​log⁡\(T\)/T\.\\mu=\{8L\\log\(T\)\}/\{T\}\.Then, for any initializationw0∈𝒲w\_\{0\}\\in\\mathcal\{W\},

δA​\(w0,ε\)≤2T​DR​\(wμ⋆,w0\)\+DR​\(wμ⋆,w0′\),\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\leq\\frac\{2\}\{T\}\\sqrt\{D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\+D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}^\{\\prime\}\)\},wherew0′w\_\{0\}^\{\\prime\}denotes the perturbed initialization andwμ⋆∈arg⁡minw∈𝒲⁡\{F​\(w\)\+μ​DR​\(w,wa\)\}\.w^\{\\star\}\_\{\\mu\}\\in\\arg\\min\_\{w\\in\\mathcal\{W\}\}\\left\\\{F\(w\)\+\\mu D\_\{R\}\(w,w\_\{a\}\)\\right\\\}\.Moreover, ifw⋆∈arg⁡minw∈𝒲⁡F​\(w\)w^\{\\star\}\\in\\arg\\min\_\{w\\in\\mathcal\{W\}\}F\(w\), then the outputwT=A​\(w0\)w\_\{T\}=A\(w\_\{0\}\)satisfies

F​\(wT\)−F​\(w⋆\)≤8​L​DR​\(wμ⋆,w0\)​log⁡TT​\(T2−1\)\+8​L​DR​\(w⋆,wa\)​log⁡TT\.F\(w\_\{T\}\)\-F\(w^\{\\star\}\)\\leq\\frac\{8LD\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\\log T\}\{T\(T^\{2\}\-1\)\}\+\\frac\{8LD\_\{R\}\(w^\{\\star\},w\_\{a\}\)\\log T\}\{T\}\.

The proof, deferred to[AppendixE](https://arxiv.org/html/2606.11431#A5), follows a similar approach to that of[Theorem4](https://arxiv.org/html/2606.11431#Thmtheorem4)and relies on the fact that adding the regularization termμ​DR​\(w,wa\)\\mu D\_\{R\}\(w,w\_\{a\}\)makes the objectiveμ\\mu\-strongly convex relative toRR\.

The key difference is that the anchor remains fixed under perturbations of the initialization\. Consequently, two nearby initializations are regularized toward the same reference point and optimize the same anchored objective\. As a result, both trajectories contract toward the same minimizer, rather than toward different minimizers as in the initialization\-anchored setting\. This allows us to leverage the exponential convergence of the regularized dynamics to obtain the desired stability guarantee\.

Notably, the analysis shows that stability emerges once both trajectories become sufficiently close to the minimizer, and therefore the resulting bound does not depend explicitly on the perturbation sizeε\\varepsilon\. An interesting open question is whether one can obtain stability guarantees with explicit dependence onε\\varepsilon, showing thatε\\varepsilon\-close initializations generate trajectories that remain close throughout the optimization process\.

## 6Discussion and Limitations

In this work, we study the initialization stability of mirror descent under general norms and regularizers\. Focusing on convex andLL\-smooth objectives, we prove exponential lower bounds in two settings: well\-conditioned nonquadratic regularizers in Euclidean geometry, and the canonical KL regularizer on the simplex\. These results establish an exponential separation between the stability of GD, or more generally quadratic MD, and the broader MD paradigm\. Notably, our constructions are convex, so the instability already arises in the classical setting for which MD was originally developed\. Somewhat surprisingly, for KL regularization, the phenomenon appears even for fixed linear objectives and even when MD is initialized near the center of the simplex\.

##### Open questions\.

Our primary focus here is initialization stability, and a natural next step is to investigate its connection to algorithmic stability and generalization\. In particular, it would be interesting to understand whether sensitivity to initialization translates into sensitivity to perturbations of the training sample, potentially linking the dynamical phenomena studied here with generalization guarantees for \(S\)GD in non\-smooth settings\(Amiret al\.,[2021](https://arxiv.org/html/2606.11431#bib.bib116); Schliserman and Koren,[2022](https://arxiv.org/html/2606.11431#bib.bib119); Livni,[2024](https://arxiv.org/html/2606.11431#bib.bib118); Vansover\-Hageret al\.,[2025](https://arxiv.org/html/2606.11431#bib.bib117)\)\. This question is especially intriguing because initialization stability concerns perturbations of the primal iterates, whereas algorithmic stability is typically driven by perturbations that enter through gradients, or dual variables\. These notions coincide in the quadratic case, but they may differ substantially for general mirror descent dynamics\.

Our findings suggest several additional directions for future research:

- •The lower bounds established in this work are stated for worst\-case initialization perturbations\. A natural direction is to study average\-case notions of initialization stability, for example by considering perturbations sampled uniformly at random from anε\\varepsilon\-ball and determining whether exponential instability still occurs\.
- •Another interesting question is whether the lower bound of[Theorem2](https://arxiv.org/html/2606.11431#Thmtheorem2)extends to other widely used simplex regularizers, such as the log\-barrier and Tsallis regularization\. This is motivated in part by the recent work ofSchlisselberget al\.\([2025](https://arxiv.org/html/2606.11431#bib.bib4)\)in online linear optimization, which studies the sensitivity of Mirror Descent to approximation errors from inexact updates and shows a qualitative separation between KL regularization and other regularizers\. It would be interesting to determine whether a similar separation appears for initialization stability\.
- •It would also be interesting to study the stability effects of clipping\-based regularization\. While many post\-training methods for LLMs use KL regularization within MD\-like updates, recent works increasingly rely on implicit regularization induced by clipping\(Yuet al\.,[2025](https://arxiv.org/html/2606.11431#bib.bib640); Rastogiet al\.,[2025](https://arxiv.org/html/2606.11431#bib.bib641); Khatriet al\.,[2025](https://arxiv.org/html/2606.11431#bib.bib639); Chenet al\.,[2025](https://arxiv.org/html/2606.11431#bib.bib642)\)\. Understanding whether such methods exhibit similar instability phenomena is an important direction for future work\.

##### Limitations\.

Our results are worst\-case lower bounds, and the initialization perturbations are chosen adversarially\. This perspective is useful because it rules out general initialization\-stability upper bounds for MD under the assumptions considered here\. At the same time, understanding which additional assumptions capture the empirical success of MD\-like methods remains an important direction for future work\.

## Acknowledgments

This project has received funding from the European Research Council \(ERC\) under the European Union’s Horizon 2020 research and innovation program \(grant agreement No\. 101078075\)\. Views and opinions expressed are however those of the author\(s\) only and do not necessarily reflect those of the European Union or the European Research Council\. Neither the European Union nor the granting authority can be held responsible for them\. This work received additional support from the Israel Science Foundation \(ISF; grant numbers 2549/19 and 3174/23\), from the Council for Higher Education in Israel under a Moonshot Project, and a fellowship from the Tel Aviv University Center for AI and Data Science \(TAD\)\. SVH is partially supported by the TAD Excellence Program for Doctoral Students in Artificial Intelligence and Data Science of the Tel Aviv University Center for AI and Data Science \(TAD\)\. OS is supported by the European Research Council \(ERC\) under the European Union’s Horizon 2020 research and innovation program \(grant agreement No\. 882396\), by the Israel Science Foundation, by a grant from the Tel Aviv University Center for AI and Data Science \(TAD\), by the TAD Excellence Program for Doctoral Students in Artificial Intelligence and Data Science from the Tel Aviv University Center for AI and Data Science \(TAD\) and from the Israeli Council for Higher Education \(CHE\) Fellowship for Outstanding PhD Students in Data Science\.

## References

- K\. Ahn, P\. Jain, Z\. Ji, S\. Kale, P\. Netrapalli, and G\. I\. Shamir \(2022\)Reproducibility in optimization: theoretical framework and limits\.Advances in Neural Information Processing Systems35,pp\. 18022–18033\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px4.p1.1)\.
- I\. Akkaya, M\. Andrychowicz, M\. Chociej, M\. Litwin, B\. McGrew, A\. Petron, A\. Paino, M\. Plappert, G\. Powell, R\. Ribas,et al\.\(2019\)Solving rubik’s cube with a robot hand\.arXiv preprint arXiv:1910\.07113\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p2.3)\.
- I\. Amir, T\. Koren, and R\. Livni \(2021\)SGD generalizes better than gd \(and regularization doesn’t help\)\.InConference on Learning Theory,pp\. 63–92\.Cited by:[§6](https://arxiv.org/html/2606.11431#S6.SS0.SSS0.Px1.p1.1)\.
- S\. Arora, E\. Hazan, and S\. Kale \(2012\)The multiplicative weights update method: a meta\-algorithm and applications\.Theory of computing8\(1\),pp\. 121–164\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p1.4)\.
- H\. Asi, V\. Feldman, T\. Koren, and K\. Talwar \(2021\)Private stochastic convex optimization: optimal rates in l1 geometry\.InInternational Conference on Machine Learning,pp\. 393–403\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px3.p1.2),[§4](https://arxiv.org/html/2606.11431#S4.p4.3)\.
- A\. Attia and T\. Koren \(2021\)Algorithmic instabilities of accelerated gradient descent\.Advances in Neural Information Processing Systems34,pp\. 1204–1214\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px3.p1.2),[§2](https://arxiv.org/html/2606.11431#S2.SS0.SSS0.Px2.p1.1)\.
- A\. Attia and T\. Koren \(2022\)Uniform stability for first\-order empirical risk minimization\.InConference on Learning Theory,pp\. 3313–3332\.Cited by:[Appendix E](https://arxiv.org/html/2606.11431#A5.p1.1),[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2606.11431#S1.p6.4),[§5\.1](https://arxiv.org/html/2606.11431#S5.SS1.p3.10)\.
- N\. Azizan, S\. Lale, and B\. Hassibi \(2021\)Stochastic mirror descent on overparameterized nonlinear models\.IEEE Transactions on Neural Networks and Learning Systems33\(12\),pp\. 7717–7727\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- R\. Bassily, V\. Feldman, C\. Guzmán, and K\. Talwar \(2020\)Stability of stochastic gradient descent on nonsmooth convex losses\.Advances in Neural Information Processing Systems33,pp\. 4381–4391\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px3.p1.2)\.
- H\. H\. Bauschke, J\. Bolte, and M\. Teboulle \(2017\)A descent lemma beyond lipschitz gradient continuity: first\-order methods revisited and applications\.Mathematics of Operations Research42\(2\),pp\. 330–348\.Cited by:[§5\.1](https://arxiv.org/html/2606.11431#S5.SS1.p3.10)\.
- A\. Beck and M\. Teboulle \(2003\)Mirror descent and nonlinear projected subgradient methods for convex optimization\.Operations Research Letters\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p1.5),[§2](https://arxiv.org/html/2606.11431#S2.SS0.SSS0.Px1.p1.5)\.
- A\. Beck \(2017\)First\-order methods in optimization\.SIAM\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- O\. Bousquet and A\. Elisseeff \(2002\)Stability and generalization\.The Journal of Machine Learning Research2,pp\. 499–526\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px2.p1.1)\.
- S\. Bubeck \(2015\)Convex optimization: algorithms and complexity\.Foundations and trends in Machine Learning8\(3\-4\),pp\. 231–357\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- N\. Cesa\-Bianchi and G\. Lugosi \(2006\)Prediction, learning, and games\.Cambridge university press\.Cited by:[§A\.2](https://arxiv.org/html/2606.11431#A1.SS2.p1.4),[§1](https://arxiv.org/html/2606.11431#S1.p5.3),[§4](https://arxiv.org/html/2606.11431#S4.p8.1)\.
- A\. Chen, A\. Li, B\. Gong, B\. Jiang, B\. Fei, B\. Yang, B\. Shan, C\. Yu, C\. Wang, C\. Zhu,et al\.\(2025\)Minimax\-m1: scaling test\-time compute efficiently with lightning attention\.arXiv preprint arXiv:2506\.13585\.Cited by:[3rd item](https://arxiv.org/html/2606.11431#S6.I1.i3.p1.1)\.
- R\. D’Orazio, N\. Loizou, I\. Laradji, and I\. Mitliagkas \(2021\)Stochastic mirror descent: convergence analysis and adaptive variants via the mirror stochastic polyak stepsize\.arXiv preprint arXiv:2110\.15412\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- Y\. Freund and R\. E\. Schapire \(1997\)A decision\-theoretic generalization of on\-line learning and an application to boosting\.Journal of computer and system sciences55\(1\),pp\. 119–139\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p1.4)\.
- M\. Hardt, B\. Recht, and Y\. Singer \(2016\)Train faster, generalize better: stability of stochastic gradient descent\.InInternational Conference on Machine Learning,pp\. 1225–1234\.Cited by:[Figure 1](https://arxiv.org/html/2606.11431#S1.F1.3.3.7.1.1),[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2606.11431#S1.p3.1)\.
- E\. Hazan \(2016\)Introduction to online convex optimization\.Foundations and Trends in Optimization2\(3\-4\),pp\. 157–325\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- D\. Khatri, L\. Madaan, R\. Tiwari, R\. Bansal, S\. S\. Duvvuri, M\. Zaheer, I\. S\. Dhillon, D\. Brandfonbrener, and R\. Agarwal \(2025\)The art of scaling reinforcement learning compute for llms\.arXiv preprint arXiv:2510\.13786\.Cited by:[3rd item](https://arxiv.org/html/2606.11431#S6.I1.i3.p1.1)\.
- Y\. Lei and Y\. Ying \(2020\)Fine\-grained analysis of stability and generalization for stochastic gradient descent\.InInternational Conference on Machine Learning,pp\. 5809–5819\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px2.p1.1)\.
- N\. Littlestone and M\. K\. Warmuth \(1994\)The weighted majority algorithm\.Information and computation108\(2\),pp\. 212–261\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p1.4)\.
- R\. Livni \(2024\)The sample complexity of gradient descent in stochastic convex optimization\.arXiv preprint arXiv:2404\.04931\.Cited by:[§6](https://arxiv.org/html/2606.11431#S6.SS0.SSS0.Px1.p1.1)\.
- H\. Lu, R\. M\. Freund, and Y\. Nesterov \(2018\)Relatively smooth convex optimization by first\-order methods, and applications\.SIAM Journal on Optimization28\(1\),pp\. 333–354\.Cited by:[Appendix E](https://arxiv.org/html/2606.11431#A5.p2.1),[§5\.1](https://arxiv.org/html/2606.11431#S5.SS1.p3.10)\.
- A\. Nemirovski and D\. Yudin \(1983\)Problem complexity and method efficiency in optimization\.Wiley\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p1.5),[§2](https://arxiv.org/html/2606.11431#S2.SS0.SSS0.Px1.p1.5)\.
- Y\. Nesterov \(1998\)Introductory lectures on convex programming volume i: basic course\.Lecture notes3\(4\),pp\. 5\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p1.4)\.
- L\. Ouyang \(2022\)Training language models to follow instructions with human feedback\.Advances in Neural Information Processing Systems35,pp\. 27730\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p2.3),[§1](https://arxiv.org/html/2606.11431#S1.p6.4)\.
- A\. Rastogi, A\. Q\. Jiang, A\. Lo, G\. Berrada, G\. Lample, J\. Rute, J\. Barmentlo, K\. Yadav, K\. Khandelwal, K\. R\. Chandu,et al\.\(2025\)Magistral\.arXiv preprint arXiv:2506\.10910\.Cited by:[3rd item](https://arxiv.org/html/2606.11431#S6.I1.i3.p1.1)\.
- M\. Schliserman and T\. Koren \(2022\)Stability vs implicit bias of gradient methods on separable data and beyond\.InProceedings of Thirty Fifth Conference on Learning Theory,P\. Loh and M\. Raginsky \(Eds\.\),Proceedings of Machine Learning Research, Vol\.178,pp\. 3380–3394\.Cited by:[§6](https://arxiv.org/html/2606.11431#S6.SS0.SSS0.Px1.p1.1)\.
- M\. Schliserman, S\. Vansover\-Hager, and T\. Koren \(2025\)Flat minima and generalization: insights from stochastic convex optimization\.arXiv preprint arXiv:2511\.03548\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px3.p1.2)\.
- O\. Schlisselberg, U\. Sherman, T\. Koren, and Y\. Mansour \(2025\)The hidden cost of approximation in online mirror descent\.arXiv preprint arXiv:2511\.22283\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1),[2nd item](https://arxiv.org/html/2606.11431#S6.I1.i2.p1.1)\.
- J\. Schulman, P\. Moritz, S\. Levine, M\. Jordan, and P\. Abbeel \(2015\)High\-dimensional continuous control using generalized advantage estimation\.arXiv preprint arXiv:1506\.02438\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p2.3)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p2.3)\.
- S\. Shalev\-Shwartz, O\. Shamir, N\. Srebro, and K\. Sridharan \(2010\)Learnability, stability and uniform convergence\.The Journal of Machine Learning Research11,pp\. 2635–2670\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px2.p1.1)\.
- S\. Shalev\-Shwartz \(2025\)Online learning and online convex optimization\.Foundations and Trends® in Machine Learning4\(2\),pp\. 107–194\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§1](https://arxiv.org/html/2606.11431#S1.p2.3),[§1](https://arxiv.org/html/2606.11431#S1.p6.4)\.
- H\. Sun, K\. Ahn, C\. Thrampoulidis, and N\. Azizan \(2022\)Mirror descent maximizes generalized margin and can be implemented efficiently\.Advances in Neural Information Processing Systems35,pp\. 31089–31101\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- H\. Sun, K\. Gatmiry, K\. Ahn, and N\. Azizan \(2023\)A unified approach to controlling implicit regularization via mirror descent\.Journal of Machine Learning Research24\(393\),pp\. 1–58\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px1.p1.1)\.
- S\. Vansover\-Hager, T\. Koren, and R\. Livni \(2025\)Rapid overfitting of multi\-pass stochastic gradient descent in stochastic convex optimization\.arXiv preprint arXiv:2505\.08306\.Cited by:[§6](https://arxiv.org/html/2606.11431#S6.SS0.SSS0.Px1.p1.1)\.
- S\. Vary, D\. Martínez\-Rubio, and P\. Rebeschini \(2024\)Black\-box uniform stability for non\-euclidean empirical risk minimization\.arXiv preprint arXiv:2412\.15956\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px4.p1.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu,et al\.\(2025\)Dapo: an open\-source llm reinforcement learning system at scale\.arXiv preprint arXiv:2503\.14476\.Cited by:[3rd item](https://arxiv.org/html/2606.11431#S6.I1.i3.p1.1)\.
- L\. Zhang, J\. Yang, A\. Karbasi, and N\. He \(2023\)Optimal guarantees for algorithmic reproducibility and gradient complexity in convex optimization\.Advances in Neural Information Processing Systems36,pp\. 17527–17566\.Cited by:[§1\.2](https://arxiv.org/html/2606.11431#S1.SS2.SSS0.Px4.p1.1)\.

## Appendix AAdditional Upper Bounds

### A\.1Quadratic Regularizer

In this section we assume the regularizer is a quadratic function:

R​\(w\)=12​w⊤​A​w\+⟨b,w⟩\\displaystyle R\(w\)=\\frac\{1\}\{2\}w^\{\\top\}Aw\+\\langle b,w\\rangle
The primal norm is then∥⋅∥A\\left\\\|\\cdot\\right\\\|\_\{A\}\.

###### Lemma 7\.

LetA,BA,Bbe symmetric matrices such that0⪯B⪯2​A0\\preceq B\\preceq 2A\. Then, for everyw∈ℝdw\\in\\mathbb\{R\}^\{d\},

‖\(A−B\)​w‖A−1≤‖A​w‖A−1\\displaystyle\\left\\\|\(A\-B\)w\\right\\\|\_\{A^\{\-1\}\}\\leq\\left\\\|Aw\\right\\\|\_\{A^\{\-1\}\}

###### Proof\.

Since0⪯B⪯2​A0\\preceq B\\preceq 2A, we haveA−1/2​B​A−1/2⪯2​IA^\{\-1/2\}BA^\{\-1/2\}\\preceq 2I, and thereforeB​A−1​B⪯2​BBA^\{\-1\}B\\preceq 2B\. Thus

\(A−B\)​A−1​\(A−B\)\\displaystyle\(A\-B\)A^\{\-1\}\(A\-B\)=A−2​B\+B​A−1​B\\displaystyle=A\-2B\+BA^\{\-1\}B⪯A−2​B\+2​B\\displaystyle\\preceq A\-2B\+2B=A\\displaystyle=A
Which means:

‖\(A−B\)​w‖A−12\\displaystyle\\left\\\|\(A\-B\)w\\right\\\|\_\{A^\{\-1\}\}^\{2\}=wT​\(A−B\)​A−1​\(A−B\)​w\\displaystyle=w^\{T\}\(A\-B\)A^\{\-1\}\(A\-B\)w≤wT​A​w\\displaystyle\\leq w^\{T\}Aw=wT​A​A−1​A​w\\displaystyle=w^\{T\}AA^\{\-1\}Aw=‖A​w‖A−12\\displaystyle=\\left\\\|Aw\\right\\\|\_\{A^\{\-1\}\}^\{2\}∎

###### Lemma 8\.

For everyw,w′w,w^\{\\prime\}:

⟨∇R​\(w\)−∇R​\(w′\),w−w′⟩=‖w−w′‖A2\\displaystyle\\langle\\nabla R\(w\)\-\\nabla R\(w^\{\\prime\}\),\\,w\-w^\{\\prime\}\\rangle=\\left\\\|w\-w^\{\\prime\}\\right\\\|\_\{A\}^\{2\}

###### Proof\.

⟨∇R​\(w\)−∇R​\(w′\),w−w′⟩\\displaystyle\\langle\\nabla R\(w\)\-\\nabla R\(w^\{\\prime\}\),\\,w\-w^\{\\prime\}\\rangle=⟨A​\(w−w′\),w−w′⟩=‖w−w′‖A2\\displaystyle=\\langle A\(w\-w^\{\\prime\}\),\\,w\-w^\{\\prime\}\\rangle=\\left\\\|w\-w^\{\\prime\}\\right\\\|\_\{A\}^\{2\}∎

First we prove the stability w\.r\.t\. the Mahalanobis norm∥⋅∥A\\\|\\cdot\\\|\_\{A\}in the following Theorem\.

###### Theorem 9\.

AssumeFFisLLsmooth w\.r\.t∥⋅∥A\\left\\\|\\cdot\\right\\\|\_\{A\}andη≤2L\\eta\\leq\\frac\{2\}\{L\}\. For everytt:

‖wt\+1−wt\+1′‖A≤‖wt−wt′‖A\.\\displaystyle\\left\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\right\\\|\_\{A\}\\leq\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|\_\{A\}\.

###### Proof\.

From first order optimality conditions:

⟨η​∇F​\(wt\)\+∇R​\(wt\+1\)−∇R​\(wt\),wt\+1′−wt\+1⟩\\displaystyle\\langle\\eta\\nabla F\(w\_\{t\}\)\+\\nabla R\(w\_\{t\+1\}\)\-\\nabla R\(w\_\{t\}\),w\_\{t\+1\}^\{\\prime\}\-w\_\{t\+1\}\\rangle≥0\\displaystyle\\geq 0⟨η​∇F​\(wt′\)\+∇R​\(wt\+1′\)−∇R​\(wt′\),wt\+1−wt\+1′⟩\\displaystyle\\langle\\eta\\nabla F\(w\_\{t\}^\{\\prime\}\)\+\\nabla R\(w\_\{t\+1\}^\{\\prime\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\),w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\rangle≥0\\displaystyle\\geq 0
Summarizing them:

⟨∇R​\(wt\)−η​∇F​\(wt\)−∇R​\(wt′\)\+η​∇F​\(wt′\),wt\+1−wt\+1′⟩≥\\displaystyle\\langle\\nabla R\(w\_\{t\}\)\-\\eta\\nabla F\(w\_\{t\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\)\+\\eta\\nabla F\(w\_\{t\}^\{\\prime\}\),\\,w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\rangle\\geq⟨∇R​\(wt\+1\)−∇R​\(wt\+1′\),wt\+1−wt\+1′⟩=‖wt\+1−wt\+1′‖A2\\displaystyle\\langle\\nabla R\(w\_\{t\+1\}\)\-\\nabla R\(w\_\{t\+1\}^\{\\prime\}\),w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\rangle=\\left\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\right\\\|\_\{A\}^\{2\}
Using Holder:

‖wt\+1−wt\+1′‖A2\\displaystyle\\left\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\right\\\|\_\{A\}^\{2\}≤‖∇R​\(wt\)−η​∇F​\(wt\)−∇R​\(wt′\)\+η​∇F​\(wt′\)‖A−1​‖wt\+1−wt\+1′‖A\\displaystyle\\leq\\left\\\|\\nabla R\(w\_\{t\}\)\-\\eta\\nabla F\(w\_\{t\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\)\+\\eta\\nabla F\(w^\{\\prime\}\_\{t\}\)\\right\\\|\_\{A^\{\-1\}\}\\left\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\right\\\|\_\{A\}⟹‖wt\+1−wt\+1′‖A\\displaystyle\\implies\\left\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\right\\\|\_\{A\}≤‖∇R​\(wt\)−η​∇F​\(wt\)−∇R​\(wt′\)\+η​∇F​\(wt′\)‖A−1\\displaystyle\\leq\\left\\\|\\nabla R\(w\_\{t\}\)\-\\eta\\nabla F\(w\_\{t\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\)\+\\eta\\nabla F\(w^\{\\prime\}\_\{t\}\)\\right\\\|\_\{A^\{\-1\}\}
Let

Bt:=∫01∇2F​\(wt′\+s​\(wt−wt′\)\)​𝑑s\.\\displaystyle B\_\{t\}:=\\int\_\{0\}^\{1\}\\nabla^\{2\}F\\\!\\left\(w\_\{t\}^\{\\prime\}\+s\(w\_\{t\}\-w\_\{t\}^\{\\prime\}\)\\right\)\\,ds\.Then∇F​\(wt\)−∇F​\(wt′\)=Bt​\(wt−wt′\)\\nabla F\(w\_\{t\}\)\-\\nabla F\(w\_\{t\}^\{\\prime\}\)=B\_\{t\}\(w\_\{t\}\-w\_\{t\}^\{\\prime\}\)\.

Notice that sinceFFis convex andLL\-smooth w\.r\.t∥⋅∥A\\left\\\|\\cdot\\right\\\|\_\{A\},0⪯Bt⪯L​A0\\preceq B\_\{t\}\\preceq LA\. Sinceη≤2L\\eta\\leq\\frac\{2\}\{L\}, we have0⪯η​Bt⪯2​A0\\preceq\\eta B\_\{t\}\\preceq 2A\. Thus:

‖wt\+1−wt\+1′‖A\\displaystyle\\left\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\right\\\|\_\{A\}≤‖\(A−η​Bt\)​\(wt−wt′\)‖A−1≤‖A​\(wt−wt′\)‖A−1=‖wt−wt′‖A\\displaystyle\\leq\\left\\\|\(A\-\\eta B\_\{t\}\)\(w\_\{t\}\-w\_\{t\}^\{\\prime\}\)\\right\\\|\_\{A^\{\-1\}\}\\leq\\left\\\|A\(w\_\{t\}\-w\_\{t\}^\{\\prime\}\)\\right\\\|\_\{A^\{\-1\}\}=\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|\_\{A\}The last inequality is due to[SectionA\.1](https://arxiv.org/html/2606.11431#A1.SS1)\. ∎

Finally we will extend the previous Theorem to a general norm:

###### Theorem 10\.

AssumeFFisLL\-smooth w\.r\.t some∥⋅∥\\left\\\|\\cdot\\right\\\|,RRisα\\alpha\-strongly convex andβ\\beta\-smooth w\.r\.t∥⋅∥\\left\\\|\\cdot\\right\\\|andη≤2​αL\\eta\\leq\\frac\{2\\alpha\}\{L\}\. For everytt:

‖wt−wt′‖≤βα​‖w0−w0′‖\\displaystyle\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|\\leq\\frac\{\\beta\}\{\\alpha\}\\left\\\|w\_\{0\}\-w\_\{0\}^\{\\prime\}\\right\\\|

###### Proof of[Theorem10](https://arxiv.org/html/2606.11431#Thmtheorem10)\.

Notice thatFFisL/αL/\\alpha\-smooth w\.r\.t∥⋅∥A\\left\\\|\\cdot\\right\\\|\_\{A\}\. Thus, from[Theorem9](https://arxiv.org/html/2606.11431#Thmtheorem9):

‖wt−wt′‖A\\displaystyle\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|\_\{A\}≤‖w0−w0′‖A\\displaystyle\\leq\\left\\\|w\_\{0\}\-w\_\{0\}^\{\\prime\}\\right\\\|\_\{A\}⟹α​‖wt−wt′‖\\displaystyle\\implies\\alpha\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|≤β​‖w0−w0′‖,\\displaystyle\\leq\\beta\\left\\\|w\_\{0\}\-w\_\{0\}^\{\\prime\}\\right\\\|,which concludes the proof\. ∎

### A\.2Legendre Regularizers

In this section we assume the decision set𝒲\\mathcal\{W\}is the intersection between some convex set𝒦\\mathcal\{K\}and a linear equality constraint𝒞=\{w∈ℝd:A​w=b\}\\mathcal\{C\}=\\\{w\\in\\mathbb\{R\}^\{d\}\\colon\\;Aw=b\\\}for someA,bA,b\. Additionally, we assume that the regularizer is a Legendre similarly to the definition in\(Cesa\-Bianchi and Lugosi,[2006](https://arxiv.org/html/2606.11431#bib.bib645), Chapter 11\.2\)\.

###### Definition 3\(Legendre function\)\.

We call*Legendre*any functionF:𝒦→ℝF:\\mathcal\{K\}\\to\\mathbb\{R\}such that

1. 1\.𝒦⊆ℝd\\mathcal\{K\}\\subseteq\\mathbb\{R\}^\{d\}is nonempty and its interiorint⁡\(𝒦\)\\operatorname\{int\}\(\\mathcal\{K\}\)is convex;
2. 2\.FFis strictly convex with continuous first partial derivatives throughoutint⁡\(𝒦\)\\operatorname\{int\}\(\\mathcal\{K\}\);
3. 3\.if𝐱1,𝐱2,…∈𝒦\\mathbf\{x\}\_\{1\},\\mathbf\{x\}\_\{2\},\\ldots\\in\\mathcal\{K\}is a sequence converging to a boundary point of𝒦\\mathcal\{K\}, then ‖∇F​\(𝐱n\)‖→∞as​n→∞\.\\\|\\nabla F\(\\mathbf\{x\}\_\{n\}\)\\\|\\to\\infty\\qquad\\text\{as \}n\\to\\infty\.

Intuitively this definition makes sure that the minimizer of the MD objective will always be inside𝒦\\mathcal\{K\}, and we only need to project to the linear constraint\. The MD step is thus:

wt\+1=arg⁡minw∈𝒞∩𝒦⁡\{η​⟨w,∇F​\(wt\)⟩\+DR​\(w,wt\)\}\.\\displaystyle w\_\{t\+1\}=\\arg\\min\_\{w\\in\\mathcal\{C\}\\cap\\mathcal\{K\}\}\\left\\\{\\eta\\langle w,\\,\\nabla F\(w\_\{t\}\)\\rangle\+D\_\{R\}\(w,w\_\{t\}\)\\right\\\}\.
This is a natural extension of the simplex with many popular regularizers \(e\.g\., negative entropy, log barrier and Tsallis entropy\) with𝒦=\{w∈ℝd\|w​\(i\)≥0∀i\}\\mathcal\{K\}=\\\{w\\in\\mathbb\{R\}^\{d\}\|\\;w\(i\)\\geq 0\\quad\\forall i\\\}and𝒞=\{w∈ℝ\|‖w‖1=1\}\\mathcal\{C\}=\\\{w\\in\\mathbb\{R\}\|\\\|w\\\|\_\{1\}=1\\\}\.

We also denoteK​e​r​\(A\)=\{w∈ℝd\|A​w=0\}Ker\(A\)=\\\{w\\in\\mathbb\{R\}^\{d\}\|Aw=0\\\}\. We now state and prove a few useful Lemmas before proving an upper bound on the stability of MD in this setting\.

###### Lemma 11\.

For every Legendre functionΦ:𝒦→ℝ\\Phi\\colon\\;\\mathcal\{K\}\\to\\mathbb\{R\}, every minimizer ofΦ\\Phiover𝒦\\mathcal\{K\}lies inint⁡𝒦\\operatorname\{int\}\\mathcal\{K\}\.

###### Proof\.

Since𝒦\\mathcal\{K\}is compact andΦ\\Phiis continuous,Φ\\Phiattains its minimum on𝒦\\mathcal\{K\}\. Letw⋆∈𝒦w^\{\\star\}\\in\\mathcal\{K\}be a minimizer\. We show thatw⋆∉∂𝒦w^\{\\star\}\\notin\\partial\\mathcal\{K\}\.

Assume toward a contradiction thatw⋆∈∂𝒦w^\{\\star\}\\in\\partial\\mathcal\{K\}\.

Chooseu∈int⁡𝒦u\\in\\operatorname\{int\}\\mathcal\{K\}, and define

ws:=\(1−s\)​w⋆\+s​u,s∈\(0,1\]\.w\_\{s\}:=\(1\-s\)w^\{\\star\}\+su,\\qquad s\\in\(0,1\]\.Thenws∈int⁡𝒦w\_\{s\}\\in\\operatorname\{int\}\\mathcal\{K\}for everys∈\(0,1\]s\\in\(0,1\], andws→w⋆w\_\{s\}\\to w^\{\\star\}ass↓0s\\downarrow 0\.

Sinceu∈int⁡𝒦u\\in\\operatorname\{int\}\\mathcal\{K\}, there existsρ\>0\\rho\>0such that

u\+ρ​v∈int⁡𝒦for every​‖v‖≤1\.u\+\\rho v\\in\\operatorname\{int\}\\mathcal\{K\}\\qquad\\text\{for every \}\\\|v\\\|\\leq 1\.BecauseΦ\\Phiis convex and finite onint⁡𝒦\\operatorname\{int\}\\mathcal\{K\}, it is locally bounded on compact subsets ofint⁡𝒦\\operatorname\{int\}\\mathcal\{K\}\. Thus

M:=sup‖v‖≤1Φ​\(u\+ρ​v\)−Φ​\(w⋆\)<∞\.M:=\\sup\_\{\\\|v\\\|\\leq 1\}\\Phi\(u\+\\rho v\)\-\\Phi\(w^\{\\star\}\)<\\infty\.
By convexity, for everyz∈𝒦z\\in\\mathcal\{K\},

Φ​\(z\)≥Φ​\(ws\)\+⟨∇Φ​\(ws\),z−ws⟩\.\\Phi\(z\)\\geq\\Phi\(w\_\{s\}\)\+\\langle\\nabla\\Phi\(w\_\{s\}\),z\-w\_\{s\}\\rangle\.Takingz=u\+ρ​vz=u\+\\rho v, where‖v‖≤1\\\|v\\\|\\leq 1, and using thatw⋆w^\{\\star\}is a minimizer, we get

⟨∇Φ​\(ws\),u\+ρ​v−ws⟩≤Φ​\(u\+ρ​v\)−Φ​\(ws\)≤Φ​\(u\+ρ​v\)−Φ​\(w⋆\)≤M\.\\langle\\nabla\\Phi\(w\_\{s\}\),u\+\\rho v\-w\_\{s\}\\rangle\\leq\\Phi\(u\+\\rho v\)\-\\Phi\(w\_\{s\}\)\\leq\\Phi\(u\+\\rho v\)\-\\Phi\(w^\{\\star\}\)\\leq M\.Since

u\+ρ​v−ws=\(1−s\)​\(u−w⋆\)\+ρ​v,u\+\\rho v\-w\_\{s\}=\(1\-s\)\(u\-w^\{\\star\}\)\+\\rho v,this gives

\(1−s\)​⟨∇Φ​\(ws\),u−w⋆⟩\+ρ​⟨∇Φ​\(ws\),v⟩≤M\.\(1\-s\)\\langle\\nabla\\Phi\(w\_\{s\}\),u\-w^\{\\star\}\\rangle\+\\rho\\langle\\nabla\\Phi\(w\_\{s\}\),v\\rangle\\leq M\.
We now show that the first term is nonnegative\. Applying the same convexity inequality withz=w⋆z=w^\{\\star\}, we obtain

Φ​\(w⋆\)≥Φ​\(ws\)\+⟨∇Φ​\(ws\),w⋆−ws⟩\.\\Phi\(w^\{\\star\}\)\\geq\\Phi\(w\_\{s\}\)\+\\langle\\nabla\\Phi\(w\_\{s\}\),w^\{\\star\}\-w\_\{s\}\\rangle\.Sincew⋆w^\{\\star\}is a minimizer,Φ​\(ws\)≥Φ​\(w⋆\)\\Phi\(w\_\{s\}\)\\geq\\Phi\(w^\{\\star\}\)\. Hence

⟨∇Φ​\(ws\),ws−w⋆⟩≥0\.\\langle\\nabla\\Phi\(w\_\{s\}\),w\_\{s\}\-w^\{\\star\}\\rangle\\geq 0\.Becausews−w⋆=s​\(u−w⋆\)w\_\{s\}\-w^\{\\star\}=s\(u\-w^\{\\star\}\), this implies

⟨∇Φ​\(ws\),u−w⋆⟩≥0\.\\langle\\nabla\\Phi\(w\_\{s\}\),u\-w^\{\\star\}\\rangle\\geq 0\.Therefore, for every‖v‖≤1\\\|v\\\|\\leq 1,

ρ​⟨∇Φ​\(ws\),v⟩≤M\.\\rho\\langle\\nabla\\Phi\(w\_\{s\}\),v\\rangle\\leq M\.Applying the same inequality to−v\-v, we get

\|⟨∇Φ​\(ws\),v⟩\|≤Mρfor every​‖v‖≤1\.\|\\langle\\nabla\\Phi\(w\_\{s\}\),v\\rangle\|\\leq\\frac\{M\}\{\\rho\}\\qquad\\text\{for every \}\\\|v\\\|\\leq 1\.Thus

‖∇Φ​\(ws\)‖∗≤Mρfor every​s∈\(0,1\]\.\\\|\\nabla\\Phi\(w\_\{s\}\)\\\|\_\{\*\}\\leq\\frac\{M\}\{\\rho\}\\qquad\\text\{for every \}s\\in\(0,1\]\.This contradicts the assumed boundary behavior, sincews∈int⁡𝒦w\_\{s\}\\in\\operatorname\{int\}\\mathcal\{K\}andws→w⋆∈∂𝒦w\_\{s\}\\to w^\{\\star\}\\in\\partial\\mathcal\{K\}\. Thereforew⋆∉∂𝒦w^\{\\star\}\\notin\\partial\\mathcal\{K\}, and every minimizer lies inint⁡𝒦\\operatorname\{int\}\\mathcal\{K\}\. ∎

###### Lemma 12\.

For every strongly convexRR:

‖x−y‖2≤⟨∇R​\(x\)−∇R​\(y\),x−y⟩\\displaystyle\\left\\\|x\-y\\right\\\|^\{2\}\\leq\\langle\\nabla R\(x\)\-\\nabla R\(y\),x\-y\\rangle

###### Proof\.

⟨∇R​\(x\)−∇R​\(y\),x−y⟩\\displaystyle\\langle\\nabla R\(x\)\-\\nabla R\(y\),x\-y\\rangle=⟨∇R​\(x\),x−y⟩\+⟨∇R​\(y\),y−x⟩\\displaystyle=\\langle\\nabla R\(x\),x\-y\\rangle\+\\langle\\nabla R\(y\),y\-x\\rangle=DR​\(x,y\)\+DR​\(y,x\)\\displaystyle=D\_\{R\}\(x,y\)\+D\_\{R\}\(y,x\)≥12​‖x−y‖2\+12​‖x−y‖2\\displaystyle\\geq\\frac\{1\}\{2\}\\left\\\|x\-y\\right\\\|^\{2\}\+\\frac\{1\}\{2\}\\left\\\|x\-y\\right\\\|^\{2\}\(strong convexity\)=‖x−y‖2\\displaystyle=\\left\\\|x\-y\\right\\\|^\{2\}∎

###### Lemma 13\.

Consider the given sequence, for someb,cb,c:

at=b​∑s=1t−1as\+c\\displaystyle a\_\{t\}=b\\sum\_\{s=1\}^\{t\-1\}a\_\{s\}\+cThen:

at=c​\(1\+b\)t−1≤c​eb​\(t−1\)\\displaystyle a\_\{t\}=c\(1\+b\)^\{t\-1\}\\leq ce^\{b\(t\-1\)\}

###### Proof\.

We’ll prove by induction\. Fort=1t=1:

a1=b​∑s=10as\+c=c\\displaystyle a\_\{1\}=b\\sum\_\{s=1\}^\{0\}a\_\{s\}\+c=c
Assume true fortt, we have:

at\+1\\displaystyle a\_\{t\+1\}=b​∑s=1tas\+c\\displaystyle=b\\sum\_\{s=1\}^\{t\}a\_\{s\}\+c=b​at\+b​∑s=1t−1as\+c\\displaystyle=ba\_\{t\}\+b\\sum\_\{s=1\}^\{t\-1\}a\_\{s\}\+c=b​at\+at\\displaystyle=ba\_\{t\}\+a\_\{t\}=\(1\+b\)​at\\displaystyle=\(1\+b\)a\_\{t\}and the claim follows\. ∎

###### Lemma 14\.

For every Legnedre functionΦ:𝒦→ℝ\\Phi\\colon\\;\\mathcal\{K\}\\to\\mathbb\{R\}, letw∗w^\{\*\}be its minimizer in𝒲\\mathcal\{W\}\. Then, for everyv∈K​e​r​\(A\)v\\in Ker\(A\):

⟨∇Φ​\(w∗\),v⟩=0\.\\displaystyle\\langle\\nabla\\Phi\(w^\{\*\}\),v\\rangle=0\.

###### Proof\.

Fixv∈K​e​r​\(A\)v\\in Ker\(A\)\. SinceΦ\\Phiis Legendre w\.r\.t𝒦\\mathcal\{K\},w∗∈int⁡\(𝒦\)w^\{\*\}\\in\\operatorname\{int\}\(\\mathcal\{K\}\)\([SectionA\.2](https://arxiv.org/html/2606.11431#A1.SS2)\), which means that there isε\>0\\varepsilon\>0such thatw\+ε​v∈𝒦w\+\\varepsilon v\\in\\mathcal\{K\}\. Additionally, sinceA​\(w∗\+ε​v\)=A​w∗\+A​ε​v=b\+0=bA\(w^\{\*\}\+\\varepsilon v\)=Aw^\{\*\}\+A\\varepsilon v=b\+0=b,w∗\+ε​vw^\{\*\}\+\\varepsilon vis also in the𝒞\\mathcal\{C\}\. Hence,w∗\+ε​v∈𝒲w^\{\*\}\+\\varepsilon v\\in\\mathcal\{W\}\.

From first order optimality conditions:

⟨∇Φ​\(w∗\),w∗\+ε​v−w∗⟩\\displaystyle\\langle\\nabla\\Phi\(w^\{\*\}\),w^\{\*\}\+\\varepsilon v\-w^\{\*\}\\rangle≥0,\\displaystyle\\geq 0,⟹⟨∇Φ​\(w∗\),v⟩\\displaystyle\\implies\\langle\\nabla\\Phi\(w^\{\*\}\),v\\rangle≥0\.\\displaystyle\\geq 0\.We can do the same thing forw∗−ε​vw^\{\*\}\-\\varepsilon v, which means:

⟨∇Φ​\(w∗\),−v⟩\\displaystyle\\langle\\nabla\\Phi\(w^\{\*\}\),\-v\\rangle≥0,\\displaystyle\\geq 0,which concludes the proof\. ∎

###### Theorem 15\.

SupposeFFisLL\-smooth w\.r\.t\.∥⋅∥\\\|\\cdot\\\|andRRis11\-strongly convex w\.r\.t\.∥⋅∥\\\|\\cdot\\\|and also theRRLegendre w\.r\.t\. some convex set𝒦\\mathcal\{K\}\. Also the the feasible set𝒲=𝒦∩𝒞\\mathcal\{W\}=\\mathcal\{K\}\\cap\\mathcal\{C\}for some𝒞=\{w∈ℝd\|A​w=b\}\\mathcal\{C\}=\\\{w\\in\\mathbb\{R\}^\{d\}\|Aw=b\\\}\. Denote byAAthe MD algorithm run with regularizerRRonFFforTTsteps with step sizeη\\etathen for anyw0∈𝒲w\_\{0\}\\in\\mathcal\{W\}it holds that:

δA​\(w0,ε\)≤max‖p‖≤ε⁡1α​‖∇R​\(w0\)−∇R​\(w0\+p\)‖⋅eη​T​L/α\\displaystyle\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\leq\\max\_\{\\\|p\\\|\\leq\\varepsilon\}\\frac\{1\}\{\\alpha\}\\left\\\|\\nabla R\(w\_\{0\}\)\-\\nabla R\(w\_\{0\}\+p\)\\right\\\|\\cdot e^\{\\eta TL/\\alpha\}

###### Proof\.

ForΦ=η​F\+R\\Phi=\\eta F\+R,Φ\\Phihas Legendre w\.r\.t\.𝒦\\mathcal\{K\}\. Fix somew0,w0′w\_\{0\},w\_\{0\}^\{\\prime\}and fix somev∈K​e​r​\(A\)v\\in Ker\(A\)\. From[SectionA\.2](https://arxiv.org/html/2606.11431#A1.SS2):

⟨η​∇F​\(wt\)\+∇R​\(wt\+1\)−∇R​\(wt\),v⟩\\displaystyle\\langle\\eta\\nabla F\(w\_\{t\}\)\+\\nabla R\(w\_\{t\+1\}\)\-\\nabla R\(w\_\{t\}\),v\\rangle=0\\displaystyle=0⟨η​∇F​\(wt′\)\+∇R​\(wt\+1′\)−∇R​\(wt′\),v⟩\\displaystyle\\langle\\eta\\nabla F\(w\_\{t\}^\{\\prime\}\)\+\\nabla R\(w\_\{t\+1\}^\{\\prime\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\),v\\rangle=0\\displaystyle=0
Subtracting them:

⟨∇R​\(wt\+1\)−∇R​\(wt\+1′\),v⟩\\displaystyle\\langle\\nabla R\(w\_\{t\+1\}\)\-\\nabla R\(w\_\{t\+1\}^\{\\prime\}\),v\\rangle=⟨η​∇F​\(wt′\)−η​∇F​\(wt\),v⟩\+⟨∇R​\(wt\)−∇R​\(wt′\),v⟩\\displaystyle=\\langle\\eta\\nabla F\(w\_\{t\}^\{\\prime\}\)\-\\eta\\nabla F\(w\_\{t\}\),v\\rangle\+\\langle\\nabla R\(w\_\{t\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\),v\\rangle≤⟨∇R​\(wt\)−∇R​\(wt′\),v⟩\+η​L​‖wt−wt′‖​‖v‖\\displaystyle\\leq\\langle\\nabla R\(w\_\{t\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\),v\\rangle\+\\eta L\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|\\left\\\|v\\right\\\|The inequality is from the smoothness ofFF\.

Thus, for everytt:

⟨∇R​\(wt\)−∇R​\(wt′\),v⟩≤⟨∇R​\(w0\)−∇R​\(w0′\),v⟩\+∑s=1tη​L​‖ws−ws′‖​‖v‖\\displaystyle\\langle\\nabla R\(w\_\{t\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\),v\\rangle\\leq\\langle\\nabla R\(w\_\{0\}\)\-\\nabla R\(w\_\{0\}^\{\\prime\}\),v\\rangle\+\\sum\_\{s=1\}^\{t\}\\eta L\\left\\\|w\_\{s\}\-w\_\{s\}^\{\\prime\}\\right\\\|\\left\\\|v\\right\\\|
Forv=wt−wt′v=w\_\{t\}\-w\_\{t\}^\{\\prime\}:

⟨∇R​\(wt\)−∇R​\(wt′\),wt−wt′⟩\\displaystyle\\langle\\nabla R\(w\_\{t\}\)\-\\nabla R\(w\_\{t\}^\{\\prime\}\),w\_\{t\}\-w\_\{t\}^\{\\prime\}\\rangle≤⟨∇R​\(w0\)−∇R​\(w0′\),wt−wt′⟩\+∑s=1tη​L​‖ws−ws′‖​‖wt−wt′‖\\displaystyle\\leq\\langle\\nabla R\(w\_\{0\}\)\-\\nabla R\(w\_\{0\}^\{\\prime\}\),w\_\{t\}\-w\_\{t\}^\{\\prime\}\\rangle\+\\sum\_\{s=1\}^\{t\}\\eta L\\left\\\|w\_\{s\}\-w\_\{s\}^\{\\prime\}\\right\\\|\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|⟹α​‖wt−wt′‖2\\displaystyle\\implies\\alpha\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|^\{2\}≤‖∇R​\(w0\)−∇R​\(w0\)‖∗​‖wt−wt′‖\+∑s=1tη​L​‖ws−ws′‖​‖wt−wt′‖\\displaystyle\\leq\\\|\\nabla R\(w\_\{0\}\)\-\\nabla R\(w\_\{0\}\)\\\|\_\{\*\}\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|\+\\sum\_\{s=1\}^\{t\}\\eta L\\left\\\|w\_\{s\}\-w\_\{s\}^\{\\prime\}\\right\\\|\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|⟹α​‖wt−wt′‖\\displaystyle\\implies\\alpha\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|≤‖∇R​\(w0\)−∇R​\(w0′\)‖∗\+∑s=1tη​L​‖ws−ws′‖\\displaystyle\\leq\\\|\\nabla R\(w\_\{0\}\)\-\\nabla R\(w\_\{0\}^\{\\prime\}\)\\\|\_\{\*\}\+\\sum\_\{s=1\}^\{t\}\\eta L\\left\\\|w\_\{s\}\-w\_\{s\}^\{\\prime\}\\right\\\|The first is due to[SectionA\.2](https://arxiv.org/html/2606.11431#A1.SS2)and Holder inequality, the second is a division by‖wt−wt′‖\\left\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\right\\\|, and the third is from the smoothness ofRR\.[SectionA\.2](https://arxiv.org/html/2606.11431#A1.SS2)concludes the proof\. ∎

## Appendix BIll\-Conditioning inℓ1\\ell\_\{1\}Geometry

We begin by showing that the negative entropy is locally ill\-conditioned at every point in the simplex, as formalized in the following lemma\.

###### Lemma 16\.

LetR​\(w\)=∑i=1dwi​log⁡wiR\(w\)=\\sum\_\{i=1\}^\{d\}w\_\{i\}\\log w\_\{i\}and letw∈Δd∘w\\in\\Delta\_\{d\}^\{\\circ\}\. Define

β​\(w\)=suph∈T∖\{0\}h⊤​∇2R​\(w\)​h‖h‖12,α​\(w\)=infh∈T∖\{0\}h⊤​∇2R​\(w\)​h‖h‖12,\\beta\(w\)=\\sup\_\{h\\in T\\setminus\\\{0\\\}\}\\frac\{h^\{\\top\}\\nabla^\{2\}R\(w\)h\}\{\\\|h\\\|\_\{1\}^\{2\}\},\\qquad\\alpha\(w\)=\\inf\_\{h\\in T\\setminus\\\{0\\\}\}\\frac\{h^\{\\top\}\\nabla^\{2\}R\(w\)h\}\{\\\|h\\\|\_\{1\}^\{2\}\},whereT=\{h∈ℝd:∑ihi=0\}T=\\\{h\\in\\mathbb\{R\}^\{d\}:\\sum\_\{i\}h\_\{i\}=0\\\}, and setκ​\(w\)=β​\(w\)/α​\(w\)\\kappa\(w\)=\\beta\(w\)/\\alpha\(w\)\. Then for anyw∈Δd∘w\\in\\Delta\_\{d\}^\{\\circ\}

β​\(w\)≥2​d−14,κ​\(w\)=Ω​\(d\)\.\\beta\(w\)\\geq\\frac\{2d\-1\}\{4\},\\qquad\\kappa\(w\)=\\Omega\(d\)\.

###### Proof\.

Since∇2R​\(w\)=diag⁡\(1/w1,…,1/wd\)\\nabla^\{2\}R\(w\)=\\operatorname\{diag\}\(1/w\_\{1\},\\dots,1/w\_\{d\}\), choosinghhsupported on the two smallest coordinates with values1/21/2and−1/2\-1/2gives

β​\(w\)≥14​\(1a\+1b\)\.\\beta\(w\)\\geq\\frac\{1\}\{4\}\\left\(\\frac\{1\}\{a\}\+\\frac\{1\}\{b\}\\right\)\.Sincea≤1/da\\leq 1/dandb≤1/\(d−1\)b\\leq 1/\(d\-1\), this also gives

β​\(w\)≥2​d−14\.\\beta\(w\)\\geq\\frac\{2d\-1\}\{4\}\.
It remains to lower bound the condition number\. For any nonempty proper subsetS⊂\[d\]S\\subset\[d\], writep=∑i∈Swip=\\sum\_\{i\\in S\}w\_\{i\}\. Definehi=12​wi/ph\_\{i\}=\\frac\{1\}\{2\}w\_\{i\}/pfori∈Si\\in Sandhi=−12​wi/\(1−p\)h\_\{i\}=\-\\frac\{1\}\{2\}w\_\{i\}/\(1\-p\)fori∉Si\\notin S\. Thenh∈Th\\in Tand‖h‖1=1\\\|h\\\|\_\{1\}=1, hence

α​\(w\)≤h⊤​∇2R​\(w\)​h=14​p\+14​\(1−p\)=14​p​\(1−p\)\.\\alpha\(w\)\\leq h^\{\\top\}\\nabla^\{2\}R\(w\)h=\\frac\{1\}\{4p\}\+\\frac\{1\}\{4\(1\-p\)\}=\\frac\{1\}\{4p\(1\-p\)\}\.Therefore

κ​\(w\)=β​\(w\)α​\(w\)≥p​\(1−p\)​\(1a\+1b\)\.\\kappa\(w\)=\\frac\{\\beta\(w\)\}\{\\alpha\(w\)\}\\geq p\(1\-p\)\\left\(\\frac\{1\}\{a\}\+\\frac\{1\}\{b\}\\right\)\.
We now chooseSS\. First suppose thatwmax≤1/2w^\{\\max\}\\leq 1/2\. Then there exists a nonempty proper subsetSSwithp∈\[1/4,3/4\]p\\in\[1/4,3/4\]: if some coordinate has mass at least1/41/4, take that coordinate; otherwise add coordinates until the partial sum first exceeds1/41/4, which gives a sum at most1/21/2\. Thusp​\(1−p\)≥3/16p\(1\-p\)\\geq 3/16\. Since1a\+1b≥2​d−1\\frac\{1\}\{a\}\+\\frac\{1\}\{b\}\\geq 2d\-1, we obtain

κ​\(w\)≥316​\(2​d−1\)≥316​d\.\\kappa\(w\)\\geq\\frac\{3\}\{16\}\(2d\-1\)\\geq\\frac\{3\}\{16\}d\.
Now suppose thatwmax\>1/2w^\{\\max\}\>1/2\. LetSSbe the singleton containing the maximal coordinate\. Thenp​\(1−p\)=wmax​\(1−wmax\)p\(1\-p\)=w^\{\\max\}\(1\-w^\{\\max\}\)\. Ifd≥3d\\geq 3, the two smallest coordinates lie outsideSS, soa≤\(1−wmax\)/\(d−1\)a\\leq\(1\-w^\{\\max\}\)/\(d\-1\)andb≤\(1−wmax\)/\(d−2\)b\\leq\(1\-w^\{\\max\}\)/\(d\-2\)\. Therefore

1a\+1b≥d−11−wmax\+d−21−wmax=2​d−31−wmax\.\\frac\{1\}\{a\}\+\\frac\{1\}\{b\}\\geq\\frac\{d\-1\}\{1\-w^\{\\max\}\}\+\\frac\{d\-2\}\{1\-w^\{\\max\}\}=\\frac\{2d\-3\}\{1\-w^\{\\max\}\}\.Thus

κ​\(w\)≥wmax​\(1−wmax\)⋅2​d−31−wmax=wmax​\(2​d−3\)≥2​d−32\.\\kappa\(w\)\\geq w^\{\\max\}\(1\-w^\{\\max\}\)\\cdot\\frac\{2d\-3\}\{1\-w^\{\\max\}\}=w^\{\\max\}\(2d\-3\)\\geq\\frac\{2d\-3\}\{2\}\.Ford≥3d\\geq 3, this is at leastd/6d/6\. The cased=2d=2is immediate, since the tangent space is one\-dimensional and henceκ​\(w\)=1=d/2\\kappa\(w\)=1=d/2\. ∎

We then show that this phenomenon is not specific to negative entropy, but is in fact unavoidable for any regularizer compatible withℓ1\\ell\_\{1\}geometry\. This is formalized in the following lemma\.

###### Lemma 17\(Condition number lower bound for theℓ1\\ell\_\{1\}norm\)\.

Supposeffis convex,β\\beta\-smooth w\.r\.t\. theβ\\betanorm andα\\alpha\-strongly\-convex w\.r\.t\. theℓ1\\ell\_\{1\}norm\. Suppose also thatffis twice differentiable\. Thenβα≥d\\frac\{\\beta\}\{\\alpha\}\\geq d\.

###### Proof\.

Fix somexx, we have for everyyy

α2​‖x−y‖12≤Df​\(y,x\)≤β2​‖x−y‖12\\frac\{\\alpha\}\{2\}\\\|x\-y\\\|\_\{1\}^\{2\}\\leq D\_\{f\}\(y,x\)\\leq\\frac\{\\beta\}\{2\}\\\|x\-y\\\|\_\{1\}^\{2\}Fix someh∈ℝdh\\in\\mathbb\{R\}^\{d\}and choosey=x\+t​hy=x\+ththen,

α2​t2​‖h‖12≤Df​\(y,x\)≤β2​t2​‖h‖12\\frac\{\\alpha\}\{2\}t^\{2\}\\\|h\\\|\_\{1\}^\{2\}\\leq D\_\{f\}\(y,x\)\\leq\\frac\{\\beta\}\{2\}t^\{2\}\\\|h\\\|\_\{1\}^\{2\}and from second order expansion

Df​\(y,x\)\\displaystyle D\_\{f\}\(y,x\)=f​\(y\)−f​\(x\)−⟨∇f​\(x\),y−x⟩\\displaystyle=f\(y\)\-f\(x\)\-\\langle\\nabla f\(x\),y\-x\\rangle=12​\(y−x\)T​∇2f​\(x\)​\(y−x\)\+o​\(\(y−x\)\)\\displaystyle=\\frac\{1\}\{2\}\(y\-x\)^\{T\}\\nabla^\{2\}f\(x\)\(y\-x\)\+o\(\(y\-x\)\)=12​t2​hT​∇2f​\(x\)​h\+o​\(t2​‖h‖22\)\\displaystyle=\\frac\{1\}\{2\}t^\{2\}h^\{T\}\\nabla^\{2\}f\(x\)h\+o\(t^\{2\}\\\|h\\\|\_\{2\}^\{2\}\)=12​t2​hT​∇2f​\(x\)​h\+o​\(t2\)\\displaystyle=\\frac\{1\}\{2\}t^\{2\}h^\{T\}\\nabla^\{2\}f\(x\)h\+o\(t^\{2\}\)\(hhis fixed\)overall

α2​t2​‖h‖12≤12​t2​hT​∇2f​\(x\)​h\+o​\(t2\)≤β2​t2​‖h‖12\\frac\{\\alpha\}\{2\}t^\{2\}\\\|h\\\|\_\{1\}^\{2\}\\leq\\frac\{1\}\{2\}t^\{2\}h^\{T\}\\nabla^\{2\}f\(x\)h\+o\(t^\{2\}\)\\leq\\frac\{\\beta\}\{2\}t^\{2\}\\\|h\\\|\_\{1\}^\{2\}Now divide the two sides byt22\\frac\{t^\{2\}\}\{2\}and taket→0t\\rightarrow 0and we get:

α​‖h‖12≤hT​∇2f​\(x\)​h≤β​‖h‖12∀h\\alpha\\\|h\\\|\_\{1\}^\{2\}\\leq h^\{T\}\\nabla^\{2\}f\(x\)h\\leq\\beta\\\|h\\\|\_\{1\}^\{2\}\\quad\\forall hSinceffis convex there exists someHHsuch that∇2f​\(x\)=XT​X\\nabla^\{2\}f\(x\)=X^\{T\}Xand so:

α​‖h‖12≤‖X​h‖22≤β​‖h‖12∀h\\alpha\\\|h\\\|\_\{1\}^\{2\}\\leq\\\|Xh\\\|\_\{2\}^\{2\}\\leq\\beta\\\|h\\\|\_\{1\}^\{2\}\\quad\\forall hNote that choosingh=eih=e\_\{i\}for theii\-th basis vector gives:

α≤‖X​ei‖22≤β\\alpha\\leq\\\|Xe\_\{i\}\\\|\_\{2\}^\{2\}\\leq\\betaNow for a uniformly randomh∈\{±1\}dh\\in\\\{\\pm 1\\\}^\{d\}we have:

𝔼h∈\{±1\}d​\[‖X​h‖22\]=∑i=1d‖X​ei‖22≤d​β\\mathbb\{E\}\_\{h\\in\\\{\\pm 1\\\}^\{d\}\}\[\\\|Xh\\\|\_\{2\}^\{2\}\]=\\sum\_\{i=1\}^\{d\}\\\|Xe\_\{i\}\\\|\_\{2\}^\{2\}\\leq d\\betaSo there exists someh0∈\{±1\}dh\_\{0\}\\in\\\{\\pm 1\\\}^\{d\}such that:

‖X​h0‖22≤d​β\\\|Xh\_\{0\}\\\|\_\{2\}^\{2\}\\leq d\\betabut on the other hand‖h0‖12=d2\\\|h\_\{0\}\\\|\_\{1\}^\{2\}=d^\{2\}so from the lower bound we get

d2​α=α​‖h0‖12≤‖X​h0‖22≤d​βd^\{2\}\\alpha=\\alpha\\\|h\_\{0\}\\\|\_\{1\}^\{2\}\\leq\\\|Xh\_\{0\}\\\|\_\{2\}^\{2\}\\leq d\\betaoverall:

βα≥d\\frac\{\\beta\}\{\\alpha\}\\geq d
∎

Finally, we give a simple example illustrating the ill\-conditioning of the negative entropy, establishing an exponential lower bound already in dimension11using a linear function\. Since all norms are equivalent ind=1d=1, this extends more generally\.

###### Lemma 18\.

Letd=1d=1\. LetAAdenote MD with the negative entropy run for withTTsteps and step size0<η≤10<\\eta\\leq 1with𝒲=\[0,1\]\\mathcal\{W\}=\[0,1\]\. Then for anyε∈\(0,12\)\\varepsilon\\in\(0,\\frac\{1\}\{2\}\), and anyw0∈\(0,12\)w\_\{0\}\\in\(0,\\frac\{1\}\{2\}\), we have forT≤1η​log⁡\(1/\(w0\+ε\)\)T\\leq\\frac\{1\}\{\\eta\}\\log\(1/\(w\_\{0\}\+\\varepsilon\)\)

δA​\(w0,ε\)≥ε​eη​T/2\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\varepsilon e^\{\\eta T/2\}\.

###### Proof\.

SetF​\(w\):=−\(ln⁡\(1\+η\)/η\)​wF\(w\):=\-\(\\ln\(1\+\\eta\)/\\eta\)\\,w\. Sinceln⁡\(1\+η\)/η≤1\\ln\(1\+\\eta\)/\\eta\\leq 1, the linear objectiveFFis11\-Lipschitz\. Fix somew0∈\(0,1/2\)w\_\{0\}\\in\(0,1/2\)and letw0′=w0\+εw\_\{0\}^\{\\prime\}=w\_\{0\}\+\\varepsilon, sincew0,ε≤12w\_\{0\},\\varepsilon\\leq\\frac\{1\}\{2\}we have thatw0′∈𝒲w\_\{0\}^\{\\prime\}\\in\\mathcal\{W\}\. SinceFFis linear, the MD updates are,

∇R​\(wt\+1\)=∇R​\(wt\)−η​F′=∇R​\(wt\)\+ln⁡\(1\+η\),\\nabla R\(w\_\{t\+1\}\)=\\nabla R\(w\_\{t\}\)\-\\eta F^\{\\prime\}=\\nabla R\(w\_\{t\}\)\+\\ln\(1\+\\eta\),and similarly forwt′w\_\{t\}^\{\\prime\}\. Since∇R​\(w\)=1\+ln⁡w\\nabla R\(w\)=1\+\\ln w, we obtainln⁡wt\+1=ln⁡wt\+ln⁡\(1\+η\)\\ln w\_\{t\+1\}=\\ln w\_\{t\}\+\\ln\(1\+\\eta\), which implieswt\+1=\(1\+η\)​wtw\_\{t\+1\}=\(1\+\\eta\)w\_\{t\}\. The same computation giveswt\+1′=\(1\+η\)​wt′w\_\{t\+1\}^\{\\prime\}=\(1\+\\eta\)w\_\{t\}^\{\\prime\}\. Therefore

wt=\(1\+η\)t​w0,wt′=\(1\+η\)t​w0′,w\_\{t\}=\(1\+\\eta\)^\{t\}w\_\{0\},\\qquad w\_\{t\}^\{\\prime\}=\(1\+\\eta\)^\{t\}w\_\{0\}^\{\\prime\},and in particular\|wT−wT′\|=\(1\+η\)T​\|w0−w0′\|≥ε​eη​T/2\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\|=\(1\+\\eta\)^\{T\}\|w\_\{0\}\-w\_\{0\}^\{\\prime\}\|\\geq\\varepsilon e^\{\\eta T/2\}, for all0≤t≤T0\\leq t\\leq T\. To see that both trajectories remain inside𝒲\\mathcal\{W\}note that for alltt,

wt=\(1\+η\)t​w0≤eη​t​w0<w0w0\+ε≤1w\_\{t\}=\(1\+\\eta\)^\{t\}w\_\{0\}\\leq e^\{\\eta t\}w\_\{0\}<\\frac\{w\_\{0\}\}\{w\_\{0\}\+\\varepsilon\}\\leq 1And similarly,

wt′=\(1\+η\)t​w0′≤eη​t​w0′<w0′w0\+ε=1,w\_\{t\}^\{\\prime\}=\(1\+\\eta\)^\{t\}w\_\{0\}^\{\\prime\}\\leq e^\{\\eta t\}w\_\{0\}^\{\\prime\}<\\frac\{w\_\{0\}^\{\\prime\}\}\{w\_\{0\}\+\\varepsilon\}=1,which concludes the proof\. ∎

## Appendix CProofs for Section[3](https://arxiv.org/html/2606.11431#S3)

### C\.1Proof of Theorem[1](https://arxiv.org/html/2606.11431#Thmtheorem1)

We usea≲ba\\lesssim banda≳ba\\gtrsim bto denote inequalities up to absolute multiplicative constants\. The proof has four steps\. First we construct the feasible set, the regularizer, and the objective\. Second, we define a sequence\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}starting at the origin and show that it is exactly generated by MD\. Third, we prove that the linearized dynamics along this sequence have an expanding transverse direction\. Finally, we initialize a second trajectory in that expanding direction and control the nonlinear error by a quadratic bootstrap\.

##### Construction\.

We will first describe the construction itself\. Fix0<η≤η00<\\eta\\leq\\eta\_\{0\}andT≥1T\\geq 1\. The absolute constantη0\>0\\eta\_\{0\}\>0and the final admissible range0<ε≤ε00<\\varepsilon\\leq\\varepsilon\_\{0\}will be chosen at the end of the proof\. Let0<δ<0\.020<\\delta<0\.02,0<γ≤1/2210<\\gamma\\leq 1/221be absolute constants\. Letσ∈\(0,1\]\\sigma\\in\(0,1\], to be chosen later as a function ofε,η,T\\varepsilon,\\eta,T\. Set

η~:=45⋅165⋅η,ω:=1\+σ​η~​T\.\\tilde\{\\eta\}:=\\frac\{4\}\{5\}\\cdot\\frac\{1\}\{65\}\\cdot\\eta,\\qquad\\omega:=1\+\\sigma\\tilde\{\\eta\}T\.Define

r:=ω−3/2,m:=γ​ω−3/2,zt:=t​σ​η~24​ω,0≤t≤T\.r:=\\omega^\{\-3/2\},\\qquad m:=\\gamma\\omega^\{\-3/2\},\\qquad z\_\{t\}:=t\\frac\{\\sigma\\tilde\{\\eta\}\}\{24\\omega\},\\qquad 0\\leq t\\leq T\.The reference trajectory will be

wt:=\(0,zt\)∈ℝ2×ℝ\.w\_\{t\}:=\(0,z\_\{t\}\)\\in\\mathbb\{R\}^\{2\}\\times\\mathbb\{R\}\.The feasible set is

𝒲:=\{\(x,z\)∈ℝ2×ℝ:‖x‖2≤r,−m≤z≤2​zT\+m\}\.\\mathcal\{W\}:=\\left\\\{\(x,z\)\\in\\mathbb\{R\}^\{2\}\\times\\mathbb\{R\}:\\\|x\\\|\_\{2\}\\leq r,\\;\-m\\leq z\\leq 2z\_\{T\}\+m\\right\\\}\.Therefore

dist⁡\(wt,𝒲c\)≥γ2​ω−3/2,0≤t≤T\.\\operatorname\{dist\}\(w\_\{t\},\\mathcal\{W\}^\{c\}\)\\geq\\frac\{\\gamma\}\{2\}\\omega^\{\-3/2\},\\qquad 0\\leq t\\leq T\.Moreover,

2​zT=σ​η~​T12​\(1\+σ​η~​T\)≤112,2​m≤2​γ\.2z\_\{T\}=\\frac\{\\sigma\\tilde\{\\eta\}T\}\{12\(1\+\\sigma\\tilde\{\\eta\}T\)\}\\leq\\frac\{1\}\{12\},\\qquad 2m\\leq 2\\gamma\.Thus, after decreasingγ\\gammaif necessary,

diam⁡\(𝒲\)≤\(2​r\)2\+\(2​zT\+2​m\)2≤4\+\(112\+2​γ\)2≤3\.\\operatorname\{diam\}\(\\mathcal\{W\}\)\\leq\\sqrt\{\(2r\)^\{2\}\+\(2z\_\{T\}\+2m\)^\{2\}\}\\leq\\sqrt\{4\+\\left\(\\frac\{1\}\{12\}\+2\\gamma\\right\)^\{2\}\}\\leq 3\.
Let

P​\(θ\):=\(cos⁡θ−sin⁡θsin⁡θcos⁡θ\),B:=\(3001\),Q:=\(1110111110\)\.P\(\\theta\):=\\begin\{pmatrix\}\\cos\\theta&\-\\sin\\theta\\\\ \\sin\\theta&\\cos\\theta\\end\{pmatrix\},\\qquad B:=\\begin\{pmatrix\}3&0\\\\ 0&1\\end\{pmatrix\},\\qquad Q:=\\begin\{pmatrix\}\\frac\{11\}\{10\}&1\\\\ 1&\\frac\{11\}\{10\}\\end\{pmatrix\}\.Define

M​\(z\):=P​\(ω​z\)​B​P​\(ω​z\)⊤,H​\(z\):=P​\(ω​z\)​Q​P​\(ω​z\)⊤\.M\(z\):=P\(\\omega z\)BP\(\\omega z\)^\{\\top\},\\qquad H\(z\):=P\(\\omega z\)QP\(\\omega z\)^\{\\top\}\.For simplicity of notation we also sot

h:=9​σω,λ:=25\+2​η~​h​T,μ:=σ24​ω​\(λ−η~​h2\)\.h:=\\frac\{9\\sigma\}\{\\omega\},\\qquad\\lambda:=25\+2\\tilde\{\\eta\}hT,\\qquad\\mu:=\\frac\{\\sigma\}\{24\\omega\}\\left\(\\lambda\-\\frac\{\\tilde\{\\eta\}h\}\{2\}\\right\)\.The clock part of the regularizer is

Rclock​\(z\):=λ2​z2−4​h​ωσ​z3\.R\_\{\\rm clock\}\(z\):=\\frac\{\\lambda\}\{2\}z^\{2\}\-\\frac\{4h\\omega\}\{\\sigma\}z^\{3\}\.Define

R~​\(x,z\):=12​x⊤​M​\(z\)​x\+Rclock​\(z\),\\widetilde\{R\}\(x,z\):=\\frac\{1\}\{2\}x^\{\\top\}M\(z\)x\+R\_\{\\rm clock\}\(z\),and

F~​\(x,z\):=−μ​z\+h2​z2\+σ12​x⊤​H​\(z\)​x\.\\widetilde\{F\}\(x,z\):=\-\\mu z\+\\frac\{h\}\{2\}z^\{2\}\+\\frac\{\\sigma\}\{12\}x^\{\\top\}H\(z\)x\.Finally set

R:=54​R~,F:=165​F~,R:=\\frac\{5\}\{4\}\\widetilde\{R\},\\qquad F:=\\frac\{1\}\{65\}\\widetilde\{F\},and restrict these functions to𝒲\\mathcal\{W\}\. The variablezzplays the role of a clock\. Along the reference trajectory thexx\-coordinate is zero, while the matricesM​\(z\)M\(z\)andH​\(z\)H\(z\)rotate with angular speedω\\omega\. The transverse instability will come from this rotating geometry\.

##### Auxiliary Lemmas\.

We now record the properties of the construction that will be used in the proof of the theorem\. The proofs of the following lemmas are deferred to[SectionC\.2](https://arxiv.org/html/2606.11431#A3.SS2)\.

The first lemma verifies the regularity assumptions:RRis uniformly strongly convex and smooth, whileFFis convex, smooth, and Lipschitz\.

###### Lemma 19\(Regularity of the construction\)\.

The functionsR,FR,Fsatisfy, on𝒲\\mathcal\{W\},

I⪯∇2R⪯C0​I,0⪯∇2F⪯II\\preceq\\nabla^\{2\}R\\preceq C\_\{0\}I,\\qquad 0\\preceq\\nabla^\{2\}F\\preceq IAlsoFFis11\-Lipschitz\. Moreover, let0<δ<0\.020<\\delta<0\.02, on

U:=\{\(x,z\):‖x‖2<\(1\+δ\)​r,−\(1\+δ\)​m<z<2​zT\+\(1\+δ\)​m\},U:=\\left\\\{\(x,z\):\\\|x\\\|\_\{2\}<\(1\+\\delta\)r,\\;\-\(1\+\\delta\)m<z<2z\_\{T\}\+\(1\+\\delta\)m\\right\\\},it holds that there exists some absolute constantC1\>0C\_\{1\}\>0such that

‖D3​R‖≤C1​ω,‖D4​R‖≤C1​ω2,\\\|D^\{3\}R\\\|\\leq C\_\{1\}\\omega,\\qquad\\\|D^\{4\}R\\\|\\leq C\_\{1\}\\omega^\{2\},and

‖D2​F‖≤C1​σ,‖D3​F‖≤C1​σ​ω\.\\\|D^\{2\}F\\\|\\leq C\_\{1\}\\sigma,\\qquad\\\|D^\{3\}F\\\|\\leq C\_\{1\}\\sigma\\omega\.

The next lemma verifies that the parameters in the clock coordinate were chosen so that the reference sequence\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}is exactly generated by MD updates\.

###### Lemma 20\(Exact reference trajectory\)\.

The pointswt=\(0,zt\)w\_\{t\}=\(0,z\_\{t\}\),0≤t≤T0\\leq t\\leq T, form an exact MD trajectory

We next study the derivative of the mirror descent update along this reference trajectory\. The following lemma shows that, in a rotating frame, the transverse linearized dynamics reduce to repeated multiplication by a single matrixAσA\_\{\\sigma\}, which has one expanding eigenvalue\.

###### Lemma 21\(Exponential transverse Jacobian\)\.

LetΦ\\Phidenote the local MD update map,

Φ​\(y\):=\(∇R\)−1​\(∇R​\(y\)−η​∇F​\(y\)\),\\Phi\(y\):=\(\\nabla R\)^\{\-1\}\\bigl\(\\nabla R\(y\)\-\\eta\\nabla F\(y\)\\bigr\),and setθt:=ω​zt\.\\theta\_\{t\}:=\\omega z\_\{t\}\.For0≤t≤T0\\leq t\\leq T, define

𝒥t:=D​Φ​\(wt−1\)​⋯​D​Φ​\(w0\),𝒥0:=I\.\\mathcal\{J\}\_\{t\}:=D\\Phi\(w\_\{t\-1\}\)\\cdots D\\Phi\(w\_\{0\}\),\\qquad\\mathcal\{J\}\_\{0\}:=I\.Then the clock block of𝒥t\\mathcal\{J\}\_\{t\}is11, and its transverse block is

P​\(θt\)​Aσt,whereAσ=B−1​P​\(−σ​η~24\)​\(B−σ​η~6​Q\)\.P\(\\theta\_\{t\}\)A\_\{\\sigma\}^\{t\},\\quad\\text\{where\}\\quad A\_\{\\sigma\}=B^\{\-1\}P\\left\(\-\\frac\{\\sigma\\tilde\{\\eta\}\}\{24\}\\right\)\\left\(B\-\\frac\{\\sigma\\tilde\{\\eta\}\}\{6\}Q\\right\)\.For allη≤η0\\eta\\leq\\eta\_\{0\}for a sufficiently small absolute constantη0\\eta\_\{0\}, the matrixAσA\_\{\\sigma\}has an expanding eigenvalueρ\>1\\rho\>1satisfying

cρ​σ​η≤log⁡ρ≤Cρ​σ​η,ρ−1≥cρ​σ​η\.c\_\{\\rho\}\\sigma\\eta\\leq\\log\\rho\\leq C\_\{\\rho\}\\sigma\\eta,\\qquad\\rho\-1\\geq c\_\{\\rho\}\\sigma\\eta\.For some absolute constantscρ,Cρ\>0c\_\{\\rho\},C\_\{\\rho\}\>0\. Moreover, the eigenbasis ofAσA\_\{\\sigma\}corresponding to its two real eigenvalues is uniformly well\-conditioned\.

It remains to control the nonlinear error around the reference trajectory\. We do this in two steps\. First, we use a general bound on the second derivative of the mirror descent update map\. Then we apply that bound to the present construction in the rotating frame\.

###### Lemma 22\(Second derivative of the M update\)\.

LetU⊂ℝdU\\subset\\mathbb\{R\}^\{d\}be open, and letR,F∈C4​\(U\)R,F\\in C^\{4\}\(U\)\. Suppose that onUU,

∇2R⪰I,‖D3​R‖≤M3,‖D4​R‖≤M4,\\nabla^\{2\}R\\succeq I,\\qquad\\\|D^\{3\}R\\\|\\leq M\_\{3\},\\qquad\\\|D^\{4\}R\\\|\\leq M\_\{4\},and

‖D2​F‖≤N2,‖D3​F‖≤N3\.\\\|D^\{2\}F\\\|\\leq N\_\{2\},\\qquad\\\|D^\{3\}F\\\|\\leq N\_\{3\}\.Let

Φ​\(y\):=\(∇R\)−1​\(∇R​\(y\)−η​∇F​\(y\)\)\\Phi\(y\):=\(\\nabla R\)^\{\-1\}\\bigl\(\\nabla R\(y\)\-\\eta\\nabla F\(y\)\\bigr\)be locally defined neary∈Uy\\in U\. Assume that

Φ​\(y\)∈U,\[y,Φ​\(y\)\]⊂U,‖Φ​\(y\)−y‖≤Δ\.\\Phi\(y\)\\in U,\\qquad\[y,\\Phi\(y\)\]\\subset U,\\qquad\\\|\\Phi\(y\)\-y\\\|\\leq\\Delta\.Then

‖B​Φ​\(y\)−I‖≤M3​Δ\+η​N2,\\\|B\\Phi\(y\)\-I\\\|\\leq M\_\{3\}\\Delta\+\\eta N\_\{2\},and

‖D2​Φ​\(y\)‖≤2​\(M4​Δ\+η​N3\+M3​\(M3​Δ\+η​N2\)\+M3​\(M3​Δ\+η​N2\)2\)\.\\\|D^\{2\}\\Phi\(y\)\\\|\\leq 2\\Bigl\(M\_\{4\}\\Delta\+\\eta N\_\{3\}\+M\_\{3\}\(M\_\{3\}\\Delta\+\\eta N\_\{2\}\)\+M\_\{3\}\(M\_\{3\}\\Delta\+\\eta N\_\{2\}\)^\{2\}\\Bigr\)\.

We now introduce the rotating\-frame norm used in the nonlinear estimate\. Letu\+u\_\{\+\}andu−u\_\{\-\}be eigenvectors ofAσA\_\{\\sigma\}with

Aσ​u\+=ρ​u\+,Aσ​u−=ρ−​u−,A\_\{\\sigma\}u\_\{\+\}=\\rho u\_\{\+\},\\qquad A\_\{\\sigma\}u\_\{\-\}=\\rho\_\{\-\}u\_\{\-\},whereρ\>1\\rho\>1is the expanding eigenvalue from Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)\. Normalizeu\+u\_\{\+\}so that‖u\+‖2=1\\\|u\_\{\+\}\\\|\_\{2\}=1, and set

Eσ:=\(u\+u−\)\.E\_\{\\sigma\}:=\\begin\{pmatrix\}u\_\{\+\}&u\_\{\-\}\\end\{pmatrix\}\.By Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2), this eigenbasis is uniformly well\-conditioned\. Thus, for some absolute constantℓ≥1\\ell\\geq 1,

‖Eσ‖,‖Eσ−1‖≤ℓ\.\\\|E\_\{\\sigma\}\\\|,\\\|E\_\{\\sigma\}^\{\-1\}\\\|\\leq\\ell\.ForY=\(y,s\)∈ℝ2×ℝY=\(y,s\)\\in\\mathbb\{R\}^\{2\}\\times\\mathbb\{R\}, write

Eσ−1​y=\(αβ\),E\_\{\\sigma\}^\{\-1\}y=\\begin\{pmatrix\}\\alpha\\\\ \\beta\\end\{pmatrix\},and define

‖Y‖\#:=\|α\|\+\|β\|\+ℓ​\|s\|\.\\\|Y\\\|\_\{\\\#\}:=\|\\alpha\|\+\|\\beta\|\+\\ell\|s\|\.Then∥⋅∥\#\\\|\\cdot\\\|\_\{\\\#\}is uniformly equivalent to the Euclidean norm and

‖\(u\+,0\)‖\#=1\.\\\|\(u\_\{\+\},0\)\\\|\_\{\\\#\}=1\.Finally, define the one\-step linearized map in the rotating frame by

L​\(y,s\):=\(Aσ​y,s\)\.L\(y,s\):=\(A\_\{\\sigma\}y,s\)\.Then‖L​Y‖\#≤ρ​‖Y‖\#\.\\\|LY\\\|\_\{\\\#\}\\leq\\rho\\\|Y\\\|\_\{\\\#\}\.The following lemma gives the desired quadratic control of the nonlinear remainder in this rotating frame\.

###### Lemma 23\(Nonlinear control\)\.

Letb:=ε​ρT\.b:=\\varepsilon\\rho^\{T\}\.There are absolute constantsK,cb\>0K,c\_\{b\}\>0such that, ifb≤cb​ω−3/2,b\\leq c\_\{b\}\\omega^\{\-3/2\},then the following holds\. If

Yt:=𝒯t​\(wt′−wt\),and\[wt,wt′\]⊂B​\(wt,K​b\),Y\_\{t\}:=\\mathcal\{T\}\_\{t\}\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\),\\quad\\text\{and\}\\quad\[w\_\{t\},w^\{\\prime\}\_\{t\}\]\\subset B\(w\_\{t\},Kb\),then there exists some absolute constantC2\>0C\_\{2\}\>0such that

‖Yt\+1−L​Yt‖\#≤C2​σ​η​ω2​\(1\+σ​η​ω\)​‖Yt‖\#2\.\\\|Y\_\{t\+1\}\-LY\_\{t\}\\\|\_\{\\\#\}\\leq C\_\{2\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\\|Y\_\{t\}\\\|\_\{\\\#\}^\{2\}\.

Finally, we use the following elementary bootstrap lemma to turn the one\-step quadratic error estimate into a uniform\-in\-time bound\.

###### Lemma 24\(Quadratic Bootstrap\)\.

Let\(qt\)t=0T\(q\_\{t\}\)\_\{t=0\}^\{T\}be a sequence of nonnegative numbers withq0=1q\_\{0\}=1, and suppose

qt≤qt−1\+at​qt−12for​1≤t≤T,q\_\{t\}\\leq q\_\{t\-1\}\+a\_\{t\}q\_\{t\-1\}^\{2\}\\qquad\\text\{for \}1\\leq t\\leq T,whereat≥0a\_\{t\}\\geq 0and

∑t=1Tat≤14\.\\sum\_\{t=1\}^\{T\}a\_\{t\}\\leq\\frac\{1\}\{4\}\.Then

qt≤2for all​0≤t≤T\.q\_\{t\}\\leq 2\\qquad\\text\{for all \}0\\leq t\\leq T\.

We are now ready to prove the theorem\.

###### Proof of Theorem[1](https://arxiv.org/html/2606.11431#Thmtheorem1)\.

We use the construction above, withσ∈\(0,1\]\\sigma\\in\(0,1\]to be chosen below\. By Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2), the functionsR,FR,Fhave all the required regularity properties on𝒲\\mathcal\{W\}, anddiam⁡\(𝒲\)≤3\\operatorname\{diam\}\(\\mathcal\{W\}\)\\leq 3\. By Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2),wt=\(0,zt\)w\_\{t\}=\(0,z\_\{t\}\)is an exact MD trajectory\.

Letu\+u\_\{\+\}be a unit expanding eigenvector ofAσA\_\{\\sigma\}, and set

v:=\(u\+,0\),w0′:=ε​v\.v:=\(u\_\{\+\},0\),\\qquad w^\{\\prime\}\_\{0\}:=\\varepsilon v\.For this trajectory, define

Yt:=𝒯t​\(wt′−wt\),0≤t≤T\.Y\_\{t\}:=\\mathcal\{T\}\_\{t\}\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\),\\qquad 0\\leq t\\leq T\.from the notations above\. Letb:=ε​ρT\.b:=\\varepsilon\\rho^\{T\}\.Assume for the moment thatb≤cb​ω−3/2,b\\leq c\_\{b\}\\omega^\{\-3/2\},for small enoughcbc\_\{b\}such that4​ℓ​b≤γ2​ω−3/24\\ell b\\leq\\frac\{\\gamma\}\{2\}\\omega^\{\-3/2\}, whereℓ\\ellis the constant of the norm equivalence,‖Y‖2≤2​ℓ​‖Y‖\#\\\|Y\\\|\_\{2\}\\leq\\sqrt\{2\}\\ell\\\|Y\\\|\_\{\\\#\}\. We will show at the end of the proof that we chooseσ\\sigmasuch that this will hold\. We will now prove by induction that:

‖Yt‖\#≤2​ε​ρt,\[wt,wt′\]⊂B​\(wt,4​ℓ​b\)0≤t≤T\.\\\|Y\_\{t\}\\\|\_\{\\\#\}\\leq 2\\varepsilon\\rho^\{t\},\\quad\[w\_\{t\},w\_\{t\}^\{\\prime\}\]\\subset B\(w\_\{t\},4\\ell b\)\\qquad 0\\leq t\\leq T\.Fort=0t=0we have‖Yt‖\#,‖w0−w0′‖2=ε≤b<4​ℓ​b\\\|Y\_\{t\}\\\|\_\{\\\#\},\\\|w\_\{0\}\-w\_\{0\}^\{\\prime\}\\\|\_\{2\}=\\varepsilon\\leq b<4\\ell b\. Now assume this holds for all0≤k≤t−10\\leq k\\leq t\-1and we will prove fortt\. Using the inductive assumption Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)gives for everyk∈\[t−1\]k\\in\[t\-1\]

‖Yk\+1−L​Yk‖\#≤C2​σ​η​ω2​\(1\+σ​η​ω\)​‖Yk‖\#2\.\\\|Y\_\{k\+1\}\-LY\_\{k\}\\\|\_\{\\\#\}\\leq C\_\{2\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\\|Y\_\{k\}\\\|\_\{\\\#\}^\{2\}\.Set

qk:=‖Yk‖\#ε​ρk\.q\_\{k\}:=\\frac\{\\\|Y\_\{k\}\\\|\_\{\\\#\}\}\{\\varepsilon\\rho^\{k\}\}\.Since‖\(u\+,0\)‖\#=1\\\|\(u\_\{\+\},0\)\\\|\_\{\\\#\}=1, we haveq0=1q\_\{0\}=1\. The nonlinear recurrence gives

qk≤qk−1\+C2​σ​η​\(ω2​\(1\+σ​η​ω\)\)​ε​ρk−2​qk−12\.q\_\{k\}\\leq q\_\{k\-1\}\+C\_\{2\}\\sigma\\eta\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)\\varepsilon\\rho^\{k\-2\}q\_\{k\-1\}^\{2\}\.Sinceρ≥1\\rho\\geq 1, we may weaken this to

qk≤qk−1\+C2​σ​η​\(ω2​\(1\+σ​η​ω\)\)​ε​ρk−1​qk−12\.q\_\{k\}\\leq q\_\{k\-1\}\+C\_\{2\}\\sigma\\eta\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)\\varepsilon\\rho^\{k\-1\}q\_\{k\-1\}^\{2\}\.Befineak:=C​σ​η​\(ω2​\(1\+σ​η​ω\)\)​ε​ρk−1a\_\{k\}:=C\\sigma\\eta\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)\\varepsilon\\rho^\{k\-1\}we have:

qk≤qk−1\+ak​qk−12\.q\_\{k\}\\leq q\_\{k\-1\}\+a\_\{k\}q\_\{k\-1\}^\{2\}\.Moreover,

σ​η​∑k=1tε​ρk−1=σ​η​ε​ρtρ−1≲b,\\sigma\\eta\\sum\_\{k=1\}^\{t\}\\varepsilon\\rho^\{k\-1\}=\\sigma\\eta\\frac\{\\varepsilon\\rho^\{t\}\}\{\\rho\-1\}\\lesssim b,becauseb=ε​ρTb=\\varepsilon\\rho^\{T\}andρ−1≥cρ​σ​η\\rho\-1\\geq c\_\{\\rho\}\\sigma\\eta\. Hence there exists an absolute constantC3\>0C\_\{3\}\>0such that,

∑k=1tak=∑k=1tC2​σ​η​\(ω2​\(1\+σ​η​ω\)\)​ε​ρk−1≤C3​ω2​\(1\+σ​η​ω\)​b\.\\sum\_\{k=1\}^\{t\}a\_\{k\}=\\sum\_\{k=1\}^\{t\}C\_\{2\}\\sigma\\eta\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)\\varepsilon\\rho^\{k\-1\}\\leq C\_\{3\}\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)b\.IfC3​\(ω2​\(1\+σ​η​ω\)\)​b≤14C\_\{3\}\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)b\\leq\\frac\{1\}\{4\}\(we will chooseσ\\sigmalater so that this will hold\) ,[SectionC\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)givesqk≤2​for​0≤k≤t\.q\_\{k\}\\leq 2\\text\{ for \}0\\leq k\\leq t\.Thus

‖Yt‖\#≤2​ε​ρt\\\|Y\_\{t\}\\\|\_\{\\\#\}\\leq 2\\varepsilon\\rho^\{t\}In particular, by norm equivalence and since rotations are isometric and‖Y‖2≤2​ℓ​‖Y‖\#\\\|Y\\\|\_\{2\}\\leq\\sqrt\{2\}\\ell\\\|Y\\\|\_\{\\\#\},

‖wt′−wt‖2≤2​2​ℓ​b<4​ℓ​b\.\\\|w^\{\\prime\}\_\{t\}\-w\_\{t\}\\\|\_\{2\}\\leq 2\\sqrt\{2\}\\ell b<4\\ell b\.Hence,\[wt,wt′\]⊂B​\(wt,4​ℓ​b\)\[w\_\{t\},w\_\{t\}^\{\\prime\}\]\\subset B\(w\_\{t\},4\\ell b\)\. This concludes the induction\. Thus we have shown for all0≤t≤T0\\leq t\\leq T,

‖Yt‖\#≤2​ε​ρt,‖wt−wt′‖2<4​ℓ​b\\\|Y\_\{t\}\\\|\_\{\\\#\}\\leq 2\\varepsilon\\rho^\{t\},\\quad\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{2\}<4\\ell bSincedist⁡\(wt,𝒲c\)≥γ2​ω−3/2,\\operatorname\{dist\}\(w\_\{t\},\\mathcal\{W\}^\{c\}\)\\geq\\frac\{\\gamma\}\{2\}\\omega^\{\-3/2\},the conditionb≤cb​ω−3/2b\\leq c\_\{b\}\\omega^\{\-3/2\}, for small enoughcbc\_\{b\}, ensures thatwt′∈𝒲w^\{\\prime\}\_\{t\}\\in\\mathcal\{W\}for all0≤t≤T0\\leq t\\leq T\. For the endpoint comparison, note that

YT−LT​Y0=∑j=0T−1LT−1−j​\(Yj\+1−L​Yj\)\.Y\_\{T\}\-L^\{T\}Y\_\{0\}=\\sum\_\{j=0\}^\{T\-1\}L^\{T\-1\-j\}\\left\(Y\_\{j\+1\}\-LY\_\{j\}\\right\)\.Using‖L​Y‖\#≤ρ​‖Y‖\#\\\|LY\\\|\_\{\\\#\}\\leq\\rho\\\|Y\\\|\_\{\\\#\}and‖Yj‖\#≤2​ε​ρj\\\|Y\_\{j\}\\\|\_\{\\\#\}\\leq 2\\varepsilon\\rho^\{j\}, we get

‖YT−LT​Y0‖\#\\displaystyle\\\|Y\_\{T\}\-L^\{T\}Y\_\{0\}\\\|\_\{\\\#\}≤C3​σ​η​ω2​\(1\+σ​η​ω\)​ε2​∑j=0T−1LT−1−j​‖Yj‖\#2≤4​C3​σ​η​ω2​\(1\+σ​η​ω\)​ε2​∑j=0T−1ρT−1−j​ρ2​j\\displaystyle\\leq C\_\{3\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\varepsilon^\{2\}\\sum\_\{j=0\}^\{T\-1\}L^\{T\-1\-j\}\\\|Y\_\{j\}\\\|\_\{\\\#\}^\{2\}\\leq 4C\_\{3\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\varepsilon^\{2\}\\sum\_\{j=0\}^\{T\-1\}\\rho^\{T\-1\-j\}\\rho^\{2j\}≤4​C3​σ​η​ω2​\(1\+σ​η​ω\)​ε2​∑j=0T−1ρT−1\+j=4​C3​σ​η​ω2​\(1\+σ​η​ω\)​ε2⋅ρ2​T−11−ρ\\displaystyle\\leq 4C\_\{3\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\varepsilon^\{2\}\\sum\_\{j=0\}^\{T\-1\}\\rho^\{T\-1\+j\}=4C\_\{3\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\varepsilon^\{2\}\\cdot\\frac\{\\rho^\{2T\-1\}\}\{1\-\\rho\}≤4​C3​σ​η​ω2​\(1\+σ​η​ω\)​ε2⋅ρ2​T1−ρ\\displaystyle\\leq 4C\_\{3\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\varepsilon^\{2\}\\cdot\\frac\{\\rho^\{2T\}\}\{1\-\\rho\}\(ρ\>1\\rho\>1\)Sinceρ−1≥cρ​σ​η\\rho\-1\\geq c\_\{\\rho\}\\sigma\\eta, there exists an absolute constantC5\>0C\_\{5\}\>0s\.t\.

‖YT−LT​Y0‖\#=C5​ω2​\(1\+σ​η​ω\)​ε2​ρ2​T=C5​ω2​\(1\+σ​η​ω\)​b2\\displaystyle\\\|Y\_\{T\}\-L^\{T\}Y\_\{0\}\\\|\_\{\\\#\}=C\_\{5\}\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\varepsilon^\{2\}\\rho^\{2T\}=C\_\{5\}\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)b^\{2\}Again, from the norm equivalence there exists an absolute constantC6\>0C\_\{6\}\>0such that:

‖YT−LT​Y0‖2≤C6​\(ω2​\(1\+σ​η​ω\)\)​b2\.\\\|Y\_\{T\}\-L^\{T\}Y\_\{0\}\\\|\_\{2\}\\leq C\_\{6\}\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)b^\{2\}\.SinceY0=ε​\(u\+,0\)Y\_\{0\}=\\varepsilon\(u\_\{\+\},0\), we have‖LT​Y0‖2=b\\\|L^\{T\}Y\_\{0\}\\\|\_\{2\}=b\. Therefore, if

C6​\(ω2​\(1\+σ​η​ω\)\)​b≤12,C\_\{6\}\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)b\\leq\\frac\{1\}\{2\},then

‖wT′−wT‖2=‖YT‖2≥‖LT​Y0‖2−‖YT−LT​Y0‖≥12​b\.\\\|w^\{\\prime\}\_\{T\}\-w\_\{T\}\\\|\_\{2\}=\\\|Y\_\{T\}\\\|\_\{2\}\\geq\\\|L^\{T\}Y\_\{0\}\\\|\_\{2\}\-\\\|Y\_\{T\}\-L^\{T\}Y\_\{0\}\\\|\\geq\\frac\{1\}\{2\}b\.It remains to chooseσ\\sigmaso thatbbhas the desired size and the following two smallness conditions hold:

b≤cb​ω−3/2and\(max⁡\{C6,2​C3\}⋅ω2​\(1\+σ​η​ω\)​b\)≤12\.b\\leq c\_\{b\}\\omega^\{\-3/2\}\\quad\\text\{and\}\\quad\\left\(\\max\\\{C\_\{6\},2C\_\{3\}\\\}\\cdot\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)b\\right\)\\leq\\frac\{1\}\{2\}\.Set

Lε:=1\+log⁡1ε,Bε:=aLε3,L\_\{\\varepsilon\}:=1\+\\log\\frac\{1\}\{\\varepsilon\},\\qquad B\_\{\\varepsilon\}:=\\frac\{a\}\{L\_\{\\varepsilon\}^\{3\}\},wherea\>0a\>0is a sufficiently small absolute constant that will be chosen later\. For a fixedaa, Letε0​\(a\)\\varepsilon\_\{0\}\(a\)be such thatε≤Bε\\varepsilon\\leq B\_\{\\varepsilon\}for anyε≤ε0​\(a\)\\varepsilon\\leq\\varepsilon\_\{0\}\(a\)\. Indeed there exists such a constant sinceε​\(1\+log⁡1ε\)3↓0\\varepsilon\\left\(1\+\\log\\frac\{1\}\{\\varepsilon\}\\right\)^\{3\}\\downarrow 0asε↓0\\varepsilon\\downarrow 0\. We chooseσ∈\(0,1\]\\sigma\\in\(0,1\]so that

ε​ρ​\(σ\)T=min⁡\{Bε,ε​ec​η​T\}\.\\varepsilon\\rho\(\\sigma\)^\{T\}=\\min\\\{B\_\{\\varepsilon\},\\varepsilon e^\{c\\eta T\}\\\}\.for some absolute constantc\>0c\>0\. This choice is possible by the intermediate value theorem\. Indeed,ρ​\(σ\)\\rho\(\\sigma\)depends continuously onσ\\sigma, andρ​\(σ\)↓1\\rho\(\\sigma\)\\downarrow 1asσ↓0\\sigma\\downarrow 0\. Thus

ε​ρ​\(σ\)T→ε\.\\varepsilon\\rho\(\\sigma\)^\{T\}\\to\\varepsilon\.Forε≤ε0​\(a\)\\varepsilon\\leq\\varepsilon\_\{0\}\(a\),

Bε≥ε,B\_\{\\varepsilon\}\\geq\\varepsilon,On the other hand, whenσ=1\\sigma=1, the lower boundlog⁡ρ​\(1\)≥cρ​η\\log\\rho\(1\)\\geq c\_\{\\rho\}\\etagives

ε​ρ​\(1\)T≥ε​ec​η​T,\\varepsilon\\rho\(1\)^\{T\}\\geq\\varepsilon e^\{c\\eta T\},after decreasing the absolute constantc\>0c\>0if necessary\. Hence such aσ∈\(0,1\]\\sigma\\in\(0,1\]exists\. For this choice,

T​log⁡ρ=log⁡bε≤log⁡1ε\.T\\log\\rho=\\log\\frac\{b\}\{\\varepsilon\}\\leq\\log\\frac\{1\}\{\\varepsilon\}\.Sinceσ​η≲log⁡ρ\\sigma\\eta\\lesssim\\log\\rho, we obtain

σ​η​T≲Lε\.\\sigma\\eta T\\lesssim L\_\{\\varepsilon\}\.Therefore

ω=1\+σ​η~​T≲Lε\.\\omega=1\+\\sigma\\tilde\{\\eta\}T\\lesssim L\_\{\\varepsilon\}\.Consequently,

ω2​\(1\+σ​η​ω\)≲Lε3\.\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\lesssim L\_\{\\varepsilon\}^\{3\}\.Because

b≤Bε=a​Lε−3,b\\leq B\_\{\\varepsilon\}=aL\_\{\\varepsilon\}^\{\-3\},we get that there exists an absolute constantC7\>0C\_\{7\}\>0such that

max⁡\{C6,2​C3\}⋅\(ω2​\(1\+σ​η​ω\)\)​b≤C7​a\.\\max\\\{C\_\{6\},2C\_\{3\}\\\}\\cdot\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)b\\leq C\_\{7\}a\.Choosinga≤12​C7a\\leq\\frac\{1\}\{2C\_\{7\}\}gives

max⁡\{C6,2​C3\}⋅\(ω2​\(1\+σ​η​ω\)\)​b≤14\.\\max\\\{C\_\{6\},2C\_\{3\}\\\}\\cdot\\left\(\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\right\)b\\leq\\frac\{1\}\{4\}\.Finally, sinceω≲Lε\\omega\\lesssim L\_\{\\varepsilon\}there exists some absolute constantC8C\_\{8\}such that,

Lε−3/2≤C8​ω−3/2\.L\_\{\\varepsilon\}^\{\-3/2\}\\leq C\_\{8\}\\omega^\{\-3/2\}\.On the other hand,

b≤a​Lε−3≤a​Lε−3≤a​Lε−3/2≤C8​a​ω−3/2\.b\\leq aL\_\{\\varepsilon\}^\{\-3\}\\leq aL\_\{\\varepsilon\}^\{\-3\}\\leq aL\_\{\\varepsilon\}^\{\-3/2\}\\leq C\_\{8\}a\\omega^\{\-3/2\}\.choosinga≤cbC8a\\leq\\frac\{c\_\{b\}\}\{C\_\{8\}\}would ensure

b≤cb​ω−3/2b\\leq c\_\{b\}\\omega^\{\-3/2\}Thus choosing the following absolute constants,

a=min⁡\{1e,cbC8,14​C7\},ε0=ε0​\(a\)a=\\min\\left\\\{\\frac\{1\}\{e\},\\frac\{c\_\{b\}\}\{C\_\{8\}\},\\frac\{1\}\{4C\_\{7\}\}\\right\\\},\\quad\\varepsilon\_\{0\}=\\varepsilon\_\{0\}\(a\)and theσ\\sigmathat follows ensures both smallness conditions hold\. Therefore

‖wT′−wT‖2≥12​b=12​min⁡\{Bε,ε​ec​η​T\}\.\\\|w^\{\\prime\}\_\{T\}\-w\_\{T\}\\\|\_\{2\}\\geq\\frac\{1\}\{2\}b=\\frac\{1\}\{2\}\\min\\\{B\_\{\\varepsilon\},\\varepsilon e^\{c\\eta T\}\\\}\.Hence,

‖wT′−wT‖2=Ω​\(min⁡\{1\(1\+log⁡\(1/ε\)\)3,ε​ec​η​T\}\)\.\\\|w^\{\\prime\}\_\{T\}\-w\_\{T\}\\\|\_\{2\}=\\Omega\\left\(\\min\\left\\\{\\frac\{1\}\{\(1\+\\log\(1/\\varepsilon\)\)^\{3\}\},\\varepsilon e^\{c\\eta T\}\\right\\\}\\right\)\.This proves the theorem\. ∎

### C\.2Proofs of Auxiliary Lemmas

For the proof of the Auxiliary lemmas we will use the following lemmas:

###### Lemma 25\(Quantitative inverse branch\)\.

LetU⊆ℝdU\\subseteq\\mathbb\{R\}^\{d\}be open and convex, and letR∈C2​\(U\)R\\in C^\{2\}\(U\)satisfy

∇2R​\(x\)⪰I∀x∈U\.\\nabla^\{2\}R\(x\)\\succeq I\\qquad\\forall x\\in U\.Letu0∈Uu\_\{0\}\\in U, and set

τ:=dist⁡\(u0,Uc\)\>0\.\\tau:=\\operatorname\{dist\}\(u\_\{0\},U^\{c\}\)\>0\.Then

B​\(∇R​\(u0\),τ4\)⊆∇R​\(B​\(u0,τ/2\)\)\.B\\left\(\\nabla R\(u\_\{0\}\),\\frac\{\\tau\}\{4\}\\right\)\\subseteq\\nabla R\\bigl\(B\(u\_\{0\},\\tau/2\)\\bigr\)\.Moreover, the inverse branch of∇R\\nabla Ron this ball is11\-Lipschitz: ifui∈B​\(u0,τ/2\)​for​i=1,2,u\_\{i\}\\in B\(u\_\{0\},\\tau/2\)\\text\{ for \}i=1,2,then

‖u1−u2‖2≤‖∇R​\(u1\)−∇R​\(u2\)‖2\.\\\|u\_\{1\}\-u\_\{2\}\\\|\_\{2\}\\leq\\\|\\nabla R\(u\_\{1\}\)\-\\nabla R\(u\_\{2\}\)\\\|\_\{2\}\.

###### Proof of the[SectionC\.2](https://arxiv.org/html/2606.11431#A3.SS2)\.

Fixp∈B​\(∇R​\(u0\),τ4\)\.p\\in B\\left\(\\nabla R\(u\_\{0\}\),\\frac\{\\tau\}\{4\}\\right\)\.Consider

φp​\(u\):=R​\(u\)−⟨p,u⟩\\varphi\_\{p\}\(u\):=R\(u\)\-\\langle p,u\\rangleon the compact ballB¯​\(u0,τ/2\)⊂U\\overline\{B\}\(u\_\{0\},\\tau/2\)\\subset U\. Letupu\_\{p\}be a minimizer ofB¯​\(u0,τ/2\)\\overline\{B\}\(u\_\{0\},\\tau/2\)that isφp\\varphi\_\{p\}\. Ifuuis a boundary point of‖u−u0‖2=τ/2\\\|u\-u\_\{0\}\\\|\_\{2\}=\\tau/2, then strong convexity gives

φp​\(u\)−φp​\(u0\)\\displaystyle\\varphi\_\{p\}\(u\)\-\\varphi\_\{p\}\(u\_\{0\}\)≥⟨∇R​\(u0\)−p,u−u0⟩\+12​‖u−u0‖22\\displaystyle\\geq\\langle\\nabla R\(u\_\{0\}\)\-p,u\-u\_\{0\}\\rangle\+\\frac\{1\}\{2\}\\\|u\-u\_\{0\}\\\|\_\{2\}^\{2\}≥−‖∇R​\(u0\)−p‖2​‖u−u0‖2\+12​‖u−u0‖22\\displaystyle\\geq\-\\\|\\nabla R\(u\_\{0\}\)\-p\\\|\_\{2\}\\\|u\-u\_\{0\}\\\|\_\{2\}\+\\frac\{1\}\{2\}\\\|u\-u\_\{0\}\\\|\_\{2\}^\{2\}\>τ4⋅τ2\+12⋅\(τ2\)2=0\.\\displaystyle\>\\frac\{\\tau\}\{4\}\\cdot\\frac\{\\tau\}\{2\}\+\\frac\{1\}\{2\}\\cdot\\left\(\\frac\{\\tau\}\{2\}\\right\)^\{2\}=0\.Thusupu\_\{p\}is interior, and hence

∇R​\(up\)=p\.\\nabla R\(u\_\{p\}\)=p\.This proves the image inclusion\. For the Lipschitz estimate, strong monotonicity gives

‖u1−u2‖22≤⟨∇R​\(u1\)−∇R​\(u2\),u1−u2⟩≤‖∇R​\(u1\)−∇R​\(u2\)‖2​‖u1−u2‖2\.\\\|u\_\{1\}\-u\_\{2\}\\\|\_\{2\}^\{2\}\\leq\\langle\\nabla R\(u\_\{1\}\)\-\\nabla R\(u\_\{2\}\),u\_\{1\}\-u\_\{2\}\\rangle\\leq\\\|\\nabla R\(u\_\{1\}\)\-\\nabla R\(u\_\{2\}\)\\\|\_\{2\}\\\|u\_\{1\}\-u\_\{2\}\\\|\_\{2\}\.Therefore

‖u1−u2‖2≤‖∇R​\(u1\)−∇R​\(u2\)‖2\.\\\|u\_\{1\}\-u\_\{2\}\\\|\_\{2\}\\leq\\\|\\nabla R\(u\_\{1\}\)\-\\nabla R\(u\_\{2\}\)\\\|\_\{2\}\.∎

#### C\.2\.1Proof of Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)

First we note that the rotating blocks satisfy

I⪯M​\(z\)⪯3​I,110​I⪯H​\(z\)⪯2110​I\.I\\preceq M\(z\)\\preceq 3I,\\qquad\\frac\{1\}\{10\}I\\preceq H\(z\)\\preceq\\frac\{21\}\{10\}I\.Moreover thejj\-th derivative satisfies,

‖M\(j\)​\(z\)‖\+‖H\(j\)​\(z\)‖≤cj​ωj,j≥1\.\\\|M^\{\(j\)\}\(z\)\\\|\+\\\|H^\{\(j\)\}\(z\)\\\|\\leq c\_\{j\}\\omega^\{j\},\\qquad j\\geq 1\.For some constantscjc\_\{j\}\. OnUU, we have

‖x‖2≤\(1\+δ\)​ω−3/2\.\\\|x\\\|\_\{2\}\\leq\(1\+\\delta\)\\omega^\{\-3/2\}\.From direct calculation,

‖M′​\(z\)​x‖2≤2​ω⋅\(1\+δ\)​ω−3/2=2​\(1\+δ\)​ω−1/2,\\\|M^\{\\prime\}\(z\)x\\\|\_\{2\}\\leq 2\\omega\\cdot\(1\+\\delta\)\\omega^\{\-3/2\}=2\(1\+\\delta\)\\omega^\{\-1/2\},and

\|x⊤​M′′​\(z\)​x\|≤4​ω2⋅\(1\+δ\)2​ω−3=4​\(1\+δ\)2​ω−1\.\|x^\{\\top\}M^\{\\prime\\prime\}\(z\)x\|\\leq 4\\omega^\{2\}\\cdot\(1\+\\delta\)^\{2\}\\omega^\{\-3\}=4\(1\+\\delta\)^\{2\}\\omega^\{\-1\}\.The Hessian ofR~\\widetilde\{R\}is

∇2R~​\(x,z\)=\(M​\(z\)M′​\(z\)​xx⊤​M′​\(z\)Rclock′′​\(z\)\+12​x⊤​M′′​\(z\)​x\)\.\\nabla^\{2\}\\widetilde\{R\}\(x,z\)=\\begin\{pmatrix\}M\(z\)&M^\{\\prime\}\(z\)x\\\\ x^\{\\top\}M^\{\\prime\}\(z\)&R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\)\+\\frac\{1\}\{2\}x^\{\\top\}M^\{\\prime\\prime\}\(z\)x\\end\{pmatrix\}\.OnUU,

z≤2​zT\+\(1\+δ\)​m,2​zT=σ​η~​T12​ω\.z\\leq 2z\_\{T\}\+\(1\+\\delta\)m,\\qquad 2z\_\{T\}=\\frac\{\\sigma\\tilde\{\\eta\}T\}\{12\\omega\}\.Sinceh=9​σ/ωh=9\\sigma/\\omega, the main part of the clock curvature cancels:

Rclock′′​\(z\)\\displaystyle R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\)=λ−24​h​ωσ​z≥λ−24​h​ωσ​\(2​zT\+\(1\+δ\)​m\)\\displaystyle=\\lambda\-\\frac\{24h\\omega\}\{\\sigma\}z\\geq\\lambda\-\\frac\{24h\\omega\}\{\\sigma\}\(2z\_\{T\}\+\(1\+\\delta\)m\)Since,

24​h​ωσ⋅2​zT=2​η~​h​T\.\\frac\{24h\\omega\}\{\\sigma\}\\cdot 2z\_\{T\}=2\\tilde\{\\eta\}hT\.This cancels the2​η~​h​T2\\tilde\{\\eta\}hTcontribution inλ=25\+2​η~​h​T\\lambda=25\+2\\tilde\{\\eta\}hT\. The extra loss from the padding is at most

24​h​ωσ​\(1\+δ\)​m=216​\(1\+δ\)​γ​ω−3/2≤216​\(1\+δ\)​γ\.\\frac\{24h\\omega\}\{\\sigma\}\(1\+\\delta\)m=216\(1\+\\delta\)\\gamma\\omega^\{\-3/2\}\\leq 216\(1\+\\delta\)\\gamma\.From the choice ofγ\\gamma, we get

Rclock′′​\(z\)=λ−24​h​ωσ​z≥24on​U\.R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\)=\\lambda\-\\frac\{24h\\omega\}\{\\sigma\}z\\geq 24\\qquad\\text\{on \}U\.Therefore, for\(y,s\)∈ℝ2×ℝ\(y,s\)\\in\\mathbb\{R\}^\{2\}\\times\\mathbb\{R\},

⟨∇2R~​\(x,z\)​\(y,s\),\(y,s\)⟩\\displaystyle\\left\\langle\\nabla^\{2\}\\widetilde\{R\}\(x,z\)\(y,s\),\(y,s\)\\right\\rangle=y⊤​M​\(z\)​y\+2​s​y⊤​M′​\(z\)​x\+s2​\(Rclock′′​\(z\)\+12​x⊤​M′′​\(z\)​x\)\\displaystyle=y^\{\\top\}M\(z\)y\+2sy^\{\\top\}M^\{\\prime\}\(z\)x\+s^\{2\}\\left\(R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\)\+\\frac\{1\}\{2\}x^\{\\top\}M^\{\\prime\\prime\}\(z\)x\\right\)≥‖y‖22−4​\(1\+δ\)​ω−1/2​\|s\|​‖y‖2\+\(24−2​\(1\+δ\)2​ω−1\)​s2\.\\displaystyle\\geq\\\|y\\\|\_\{2\}^\{2\}\-4\(1\+\\delta\)\\omega^\{\-1/2\}\|s\|\\\|y\\\|\_\{2\}\+\(24\-2\(1\+\\delta\)^\{2\}\\omega^\{\-1\}\)s^\{2\}\.Now notice that:

−4​\(1\+δ\)​ω−1/2​\|s\|​‖y‖2≥−15​‖y‖22−20​\(1\+δ\)2​ω−1​s2\-4\(1\+\\delta\)\\omega^\{\-1/2\}\|s\|\\\|y\\\|\_\{2\}\\geq\-\\frac\{1\}\{5\}\\\|y\\\|\_\{2\}^\{2\}\-20\(1\+\\delta\)^\{2\}\\omega^\{\-1\}s^\{2\}Plugging that back,

⟨∇2R~​\(x,z\)​\(y,s\),\(y,s\)⟩\\displaystyle\\left\\langle\\nabla^\{2\}\\widetilde\{R\}\(x,z\)\(y,s\),\(y,s\)\\right\\rangle≥‖y‖22−15​‖y‖22−20​\(1\+δ\)2​ω−1​s2\+\(24−2​\(1\+δ\)2​ω−1\)​s2\\displaystyle\\geq\\\|y\\\|\_\{2\}^\{2\}\-\\frac\{1\}\{5\}\\\|y\\\|\_\{2\}^\{2\}\-20\(1\+\\delta\)^\{2\}\\omega^\{\-1\}s^\{2\}\+\(24\-2\(1\+\\delta\)^\{2\}\\omega^\{\-1\}\)s^\{2\}≥45​‖y‖22\+\(24−22​\(1\+δ\)2​ω−1\)​s2\\displaystyle\\geq\\frac\{4\}\{5\}\\\|y\\\|\_\{2\}^\{2\}\+\(24\-22\(1\+\\delta\)^\{2\}\\omega^\{\-1\}\)s^\{2\}≥45​‖y‖22\+45​s2\\displaystyle\\geq\\frac\{4\}\{5\}\\\|y\\\|\_\{2\}^\{2\}\+\\frac\{4\}\{5\}s^\{2\}\(ω≥1\\omega\\geq 1, choice ofδ\\delta\)
Hence

∇2R~⪰45​I\.\\nabla^\{2\}\\widetilde\{R\}\\succeq\\frac\{4\}\{5\}I\.SinceR=\(5/4\)​R~R=\(5/4\)\\widetilde\{R\}, this gives

∇2R⪰I\.\\nabla^\{2\}R\\succeq I\.For the upper bound,

Rclock′′​\(z\)\+12​x⊤​M′′​\(z\)​x≤25\+2​η~​h​T\+216​\(1\+δ\)​γ≤25\+18\+216​\(1\+δ\)​γR\_\{\\rm clock\}^\{\\prime\\prime\}\(z\)\+\\frac\{1\}\{2\}x^\{\\top\}M^\{\\prime\\prime\}\(z\)x\\leq 25\+2\\tilde\{\\eta\}hT\+216\(1\+\\delta\)\\gamma\\leq 25\+18\+216\(1\+\\delta\)\\gammawhich is bounded by an absolute constant\. We get

‖∇2R~​\(x,z\)‖\\displaystyle\\\|\\nabla^\{2\}\\widetilde\{R\}\(x,z\)\\\|≤‖M​\(z\)‖\+2​‖M′​\(z\)​x‖\+\|Rclock′′​\(z\)\+12​x⊤​M′′​\(z\)​x\|\\displaystyle\\leq\\\|M\(z\)\\\|\+2\\\|M^\{\\prime\}\(z\)x\\\|\+\|R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\)\+\\frac\{1\}\{2\}x^\{\\top\}M^\{\\prime\\prime\}\(z\)x\|≤4​\(1\+δ\)2​ω−1\+4​\(1\+δ\)​ω−1/2\+\|Rclock′′​\(z\)\+12​x⊤​M′′​\(z\)​x\|\\displaystyle\\leq 4\(1\+\\delta\)^\{2\}\\omega^\{\-1\}\+4\(1\+\\delta\)\\omega^\{\-1/2\}\+\|R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\)\+\\frac\{1\}\{2\}x^\{\\top\}M^\{\\prime\\prime\}\(z\)x\|Hence there exists an absolute constantC0C\_\{0\}, such that

∇2R⪯C0​I\\nabla^\{2\}R\\preceq C\_\{0\}IWe now prove convexity ofF~\\widetilde\{F\}with explicit constants\. Its Hessian is

∇2F~​\(x,z\)=\(σ6​H​\(z\)σ6​H′​\(z\)​xσ6​x⊤​H′​\(z\)h\+σ12​x⊤​H′′​\(z\)​x\)\.\\nabla^\{2\}\\widetilde\{F\}\(x,z\)=\\begin\{pmatrix\}\\frac\{\\sigma\}\{6\}H\(z\)&\\frac\{\\sigma\}\{6\}H^\{\\prime\}\(z\)x\\\\ \\frac\{\\sigma\}\{6\}x^\{\\top\}H^\{\\prime\}\(z\)&h\+\\frac\{\\sigma\}\{12\}x^\{\\top\}H^\{\\prime\\prime\}\(z\)x\\end\{pmatrix\}\.Because the eigenvalues ofQQare1/101/10and21/1021/10,

110​I⪯H​\(z\)⪯2110​I\.\\frac\{1\}\{10\}I\\preceq H\(z\)\\preceq\\frac\{21\}\{10\}I\.Also, directly from the rotating form,

‖H′​\(z\)‖≤2​ω,‖H′′​\(z\)‖≤4​ω2\.\\\|H^\{\\prime\}\(z\)\\\|\\leq 2\\omega,\\qquad\\\|H^\{\\prime\\prime\}\(z\)\\\|\\leq 4\\omega^\{2\}\.The upper\-left block satisfies

σ6​H​\(z\)⪰σ60​I,\\frac\{\\sigma\}\{6\}H\(z\)\\succeq\\frac\{\\sigma\}\{60\}I,so its inverse has operator norm at most60/σ60/\\sigma\. The off\-diagonal block satisfies

‖σ6​H′​\(z\)​x‖2≤σ6⋅2​ω⋅\(1\+δ\)​ω−3/2=1\+δ3​σ​ω−1/2\.\\left\\\|\\frac\{\\sigma\}\{6\}H^\{\\prime\}\(z\)x\\right\\\|\_\{2\}\\leq\\frac\{\\sigma\}\{6\}\\cdot 2\\omega\\cdot\(1\+\\delta\)\\omega^\{\-3/2\}=\\frac\{1\+\\delta\}\{3\}\\sigma\\omega^\{\-1/2\}\.Therefore the Schur\-complement loss is at most

\(σ6​x⊤​H′​\(z\)\)​\(σ6​H​\(z\)\)−1​\(σ6​H′​\(z\)​x\)≤6σ​‖H​\(z\)−1‖⋅σ6​‖H′​\(z\)​x‖22\\displaystyle\\left\(\\frac\{\\sigma\}\{6\}x^\{\\top\}H^\{\\prime\}\(z\)\\right\)\\left\(\\frac\{\\sigma\}\{6\}H\(z\)\\right\)^\{\-1\}\\left\(\\frac\{\\sigma\}\{6\}H^\{\\prime\}\(z\)x\\right\)\\leq\\frac\{6\}\{\\sigma\}\\\|H\(z\)^\{\-1\}\\\|\\cdot\\frac\{\\sigma\}\{6\}\\\|H^\{\\prime\}\(z\)x\\\|\_\{2\}^\{2\}≤60σ​\(\(1\+δ\)​ω−1/2​σ3\)2=203​\(1\+δ\)2​σω\.\\displaystyle\\leq\\frac\{60\}\{\\sigma\}\\left\(\\frac\{\(1\+\\delta\)\\omega^\{\-1/2\}\\sigma\}\{3\}\\right\)^\{2\}=\\frac\{20\}\{3\}\(1\+\\delta\)^\{2\}\\frac\{\\sigma\}\{\\omega\}\.For the lower\-right block, usingh=9​σ/ωh=9\\sigma/\\omega,

h\+σ12​x⊤​H′′​\(z\)​x\\displaystyle h\+\\frac\{\\sigma\}\{12\}x^\{\\top\}H^\{\\prime\\prime\}\(z\)x≥9​σω−σ12⋅4​ω2⋅\(1\+δ\)2​ω−3\\displaystyle\\geq\\frac\{9\\sigma\}\{\\omega\}\-\\frac\{\\sigma\}\{12\}\\cdot 4\\omega^\{2\}\\cdot\(1\+\\delta\)^\{2\}\\omega^\{\-3\}=\(9−\(1\+δ\)23\)​σω\.\\displaystyle=\\left\(9\-\\frac\{\(1\+\\delta\)^\{2\}\}\{3\}\\right\)\\frac\{\\sigma\}\{\\omega\}\.Thus the Schur complement is at least

\(9−\(1\+δ\)23−203​\(1\+δ\)2\)​σω=\(9−7​\(1\+δ\)2\)​σω\.\\left\(9\-\\frac\{\(1\+\\delta\)^\{2\}\}\{3\}\-\\frac\{20\}\{3\}\(1\+\\delta\)^\{2\}\\right\)\\frac\{\\sigma\}\{\\omega\}=\\left\(9\-7\(1\+\\delta\)^\{2\}\\right\)\\frac\{\\sigma\}\{\\omega\}\.By the choice ofδ\\delta, this is nonnegative\. Hence

∇2F~⪰0on​U\.\\nabla^\{2\}\\widetilde\{F\}\\succeq 0\\qquad\\text\{on \}U\.
The same block estimates give the sharper bound

‖∇2F~​\(x,y\)‖\\displaystyle\\\|\\nabla^\{2\}\\widetilde\{F\}\(x,y\)\\\|≤‖σ6​H​\(z\)‖\+2​‖σ6​H′​\(z\)​x‖2\+\|h\+σ12​x⊤​H′′​\(z\)​x\|\\displaystyle\\leq\\\|\\frac\{\\sigma\}\{6\}H\(z\)\\\|\+2\\\|\\frac\{\\sigma\}\{6\}H^\{\\prime\}\(z\)x\\\|\_\{2\}\+\|h\+\\frac\{\\sigma\}\{12\}x^\{\\top\}H^\{\\prime\\prime\}\(z\)x\|≤7​σ20\+2⋅\(1\+δ\)​σ​ω−1/23\+9​σ\+\(1\+δ\)2​σ​ω−13\\displaystyle\\leq\\frac\{7\\sigma\}\{20\}\+2\\cdot\\frac\{\(1\+\\delta\)\\sigma\\omega^\{\-1/2\}\}\{3\}\+9\\sigma\+\\frac\{\(1\+\\delta\)^\{2\}\\sigma\\omega^\{\-1\}\}\{3\}≤\(720\+43\+9\+23\)​σ≤12​σ\\displaystyle\\leq\\left\(\\frac\{7\}\{20\}\+\\frac\{4\}\{3\}\+9\+\\frac\{2\}\{3\}\\right\)\\sigma\\leq 12\\sigma\(δ≤1≤ω\\delta\\leq 1\\leq\\omega\)Therefore

‖D2​F‖=165​‖∇2F~‖≤12⁤165​σ\.\\\|D^\{2\}F\\\|=\\frac\{1\}\{65\}\\\|\\nabla^\{2\}\\widetilde\{F\}\\\|\\leq 12\\frac\{1\}\{65\}\\sigma\.
Next,

‖∇xF~​\(x,z\)‖2=‖σ6​H​\(z\)​x‖≤\(1\+δ\)​σ​w−1/23≤23​σ\.\\\|\\nabla\_\{x\}\\widetilde\{F\}\(x,z\)\\\|\_\{2\}=\\\|\\frac\{\\sigma\}\{6\}H\(z\)x\\\|\\leq\\frac\{\(1\+\\delta\)\\sigma w^\{\-1/2\}\}\{3\}\\leq\\frac\{2\}\{3\}\\sigma\.Also,

\|∂zF~​\(x,z\)\|\\displaystyle\|\\partial\_\{z\}\\widetilde\{F\}\(x,z\)\|=\|−μ\+h​z\+σ12​x⊤​H′​\(z\)​x\|\\displaystyle=\|\-\\mu\+hz\+\\frac\{\\sigma\}\{12\}x^\{\\top\}H^\{\\prime\}\(z\)x\|≤σ24​ω​\(25\+2​η~​h​T−h~2\)\+9​σ​\(2​zT\+\(1\+δ\)​γ​ω−3/2\)\+\(1\+δ\)​ω−1/2​σ3\\displaystyle\\leq\\frac\{\\sigma\}\{24\\omega\}\\left\(25\+2\\tilde\{\\eta\}hT\-\\frac\{\\tilde\{h\}\}\{2\}\\right\)\+9\\sigma\(2z\_\{T\}\+\(1\+\\delta\)\\gamma\\omega^\{\-3/2\}\)\+\\frac\{\(1\+\\delta\)\\omega^\{\-1/2\}\\sigma\}\{3\}≤σ​\(25\+18\)\+9​σ​\(224\+2\)\+23​σ≤63​σ\.\\displaystyle\\leq\\sigma\(25\+18\)\+9\\sigma\\left\(\\frac\{2\}\{24\}\+2\\right\)\+\\frac\{2\}\{3\}\\sigma\\leq 63\\sigma\.Hence

‖∇F~‖2≤65​σ\.\\\|\\nabla\\widetilde\{F\}\\\|\_\{2\}\\leq 65\\sigma\.Since165<165\\frac\{1\}\{65\}<\\frac\{1\}\{65\},F=165​F~F=\\frac\{1\}\{65\}\\widetilde\{F\}convex,11\-Lipschitz, and11\-smooth\. Finally, we justify the higher derivative bounds\. Since

R~​\(x,z\)=12​x⊤​M​\(z\)​x\+Rclock​\(z\),\\widetilde\{R\}\(x,z\)=\\frac\{1\}\{2\}x^\{\\top\}M\(z\)x\+R\_\{\\rm clock\}\(z\),and the first term is quadratic inxx, every derivative with more than twoxx\-derivatives vanishes\. The nonzero third derivatives coming from12​x⊤​M​\(z\)​x\\frac\{1\}\{2\}x^\{\\top\}M\(z\)xare bounded by

‖M′​\(z\)‖≲ω,‖M′′​\(z\)​x‖2≲ω2​ω−3/2=ω,\\\|M^\{\\prime\}\(z\)\\\|\\lesssim\\omega,\\qquad\\\|M^\{\\prime\\prime\}\(z\)x\\\|\_\{2\}\\lesssim\\omega^\{2\}\\omega^\{\-3/2\}=\\omega,and

\|x⊤​M\(3\)​\(z\)​x\|≲ω3​ω−3=ω\.\|x^\{\\top\}M^\{\(3\)\}\(z\)x\|\\lesssim\\omega^\{3\}\\omega^\{\-3\}=\\omega\.The nonzero fourth derivatives are bounded by

‖M′′​\(z\)‖≲ω2,‖M\(3\)​\(z\)​x‖2≲ω3​ω−3/2=ω2,\\\|M^\{\\prime\\prime\}\(z\)\\\|\\lesssim\\omega^\{2\},\\qquad\\\|M^\{\(3\)\}\(z\)x\\\|\_\{2\}\\lesssim\\omega^\{3\}\\omega^\{\-3/2\}=\\omega^\{2\},and

\|x⊤​M\(4\)​\(z\)​x\|≲ω4​ω−3=ω2\.\|x^\{\\top\}M^\{\(4\)\}\(z\)x\|\\lesssim\\omega^\{4\}\\omega^\{\-3\}=\\omega^\{2\}\.Also,

Rclock\(3\)​\(z\)=−24​h​ωσ=−216,Rclock\(4\)​\(z\)=0\.R\_\{\\rm clock\}^\{\(3\)\}\(z\)=\-\\frac\{24h\\omega\}\{\\sigma\}=\-216,\\qquad R\_\{\\rm clock\}^\{\(4\)\}\(z\)=0\.Thus for and absolute constantCRC\_\{R\}large enough,

‖D3​R‖≤CR​ω,‖D4​R‖≤CR​ω2\.\\\|D^\{3\}R\\\|\\leq C\_\{R\}\\omega,\\qquad\\\|D^\{4\}R\\\|\\leq C\_\{R\}\\omega^\{2\}\.
Similarly,

F~​\(x,z\)=−μ​z\+h2​z2\+σ12​x⊤​H​\(z\)​x\.\\widetilde\{F\}\(x,z\)=\-\\mu z\+\\frac\{h\}\{2\}z^\{2\}\+\\frac\{\\sigma\}\{12\}x^\{\\top\}H\(z\)x\.The clock part is quadratic, so it contributes no third derivatives\. The nonzero third derivatives coming from the rotating block are bounded by

σ​‖H′​\(z\)‖≲σ​ω,σ​‖H′′​\(z\)​x‖2≲σ​ω2​ω−3/2=σ​ω,\\sigma\\\|H^\{\\prime\}\(z\)\\\|\\lesssim\\sigma\\omega,\\qquad\\sigma\\\|H^\{\\prime\\prime\}\(z\)x\\\|\_\{2\}\\lesssim\\sigma\\omega^\{2\}\\omega^\{\-3/2\}=\\sigma\\omega,and

σ​\|x⊤​H\(3\)​\(z\)​x\|≲σ​ω3​ω−3=σ​ω\.\\sigma\|x^\{\\top\}H^\{\(3\)\}\(z\)x\|\\lesssim\\sigma\\omega^\{3\}\\omega^\{\-3\}=\\sigma\\omega\.SinceF=165​F~F=\\frac\{1\}\{65\}\\widetilde\{F\}, we obtain for large enoughCFC\_\{F\}

‖D3​F‖≤CF​165​σ​ω\.\\\|D^\{3\}F\\\|\\leq C\_\{F\}\\frac\{1\}\{65\}\\sigma\\omega\.TakingC1=max⁡\{CR,CF\}C\_\{1\}=\\max\\\{C\_\{R\},C\_\{F\}\\\}concludes the proof\.

#### C\.2\.2Proof of Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)

On the axisx=0x=0, the transverse gradients vanish\. It is enough to check the clock coordinate\. Note that

zt\+1−zt=σ​η~24​ω\.z\_\{t\+1\}\-z\_\{t\}=\\frac\{\\sigma\\tilde\{\\eta\}\}\{24\\omega\}\.Since

Rclock′​\(z\)=λ​z−12​h​ωσ​z2,R\_\{\\rm clock\}^\{\\prime\}\(z\)=\\lambda z\-\\frac\{12h\\omega\}\{\\sigma\}z^\{2\},we have

Rclock′​\(zt\+1\)−Rclock′​\(zt\)\\displaystyle R\_\{\\rm clock\}^\{\\prime\}\(z\_\{t\+1\}\)\-R\_\{\\rm clock\}^\{\\prime\}\(z\_\{t\}\)=\(zt\+1−zt\)​\(λ−12​h​ωσ​\(zt\+1\+zt\)\)\\displaystyle=\(z\_\{t\+1\}\-z\_\{t\}\)\\left\(\\lambda\-\\frac\{12h\\omega\}\{\\sigma\}\(z\_\{t\+1\}\+z\_\{t\}\)\\right\)=\(zt\+1−zt\)​\(λ−24​h​ωσ​zt−12​h​ωσ​\(zt\+1−zt\)\)\.\\displaystyle=\(z\_\{t\+1\}\-z\_\{t\}\)\\left\(\\lambda\-\\frac\{24h\\omega\}\{\\sigma\}z\_\{t\}\-\\frac\{12h\\omega\}\{\\sigma\}\(z\_\{t\+1\}\-z\_\{t\}\)\\right\)\.Plugging in\(zt\+1−zt\)=σ​η~24​ω\(z\_\{t\+1\}\-z\_\{t\}\)=\\frac\{\\sigma\\tilde\{\\eta\}\}\{24\\omega\}andμ=σ24​ω​\(λ−η~​h2\)\\mu=\\frac\{\\sigma\}\{24\\omega\}\\left\(\\lambda\-\\frac\{\\tilde\{\\eta\}h\}\{2\}\\right\), this becomes

Rclock′​\(zt\+1\)−Rclock′​\(zt\)=η~​μ−η~​h​zt\.R\_\{\\rm clock\}^\{\\prime\}\(z\_\{t\+1\}\)\-R\_\{\\rm clock\}^\{\\prime\}\(z\_\{t\}\)=\\tilde\{\\eta\}\\mu\-\\tilde\{\\eta\}hz\_\{t\}\.Since

∂zF~​\(0,zt\)=−μ\+h​zt,\\partial\_\{z\}\\widetilde\{F\}\(0,z\_\{t\}\)=\-\\mu\+hz\_\{t\},we get

∇R~​\(wt\+1\)=∇R~​\(wt\)−η~​∇F~​\(wt\)\.\\nabla\\widetilde\{R\}\(w\_\{t\+1\}\)=\\nabla\\widetilde\{R\}\(w\_\{t\}\)\-\\tilde\{\\eta\}\\nabla\\widetilde\{F\}\(w\_\{t\}\)\.Finally,R=\(5/4\)​R~R=\(5/4\)\\widetilde\{R\},F=165​F~F=\\frac\{1\}\{65\}\\widetilde\{F\}, andη~=\(4⁤165/5\)​η\\tilde\{\\eta\}=\(4\\frac\{1\}\{65\}/5\)\\eta\. Therefore

∇R​\(wt\+1\)=∇R​\(wt\)−η​∇F​\(wt\)\.\\nabla R\(w\_\{t\+1\}\)=\\nabla R\(w\_\{t\}\)\-\\eta\\nabla F\(w\_\{t\}\)\.

#### C\.2\.3Proof of Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)

Bifferentiating the update gives

B​Φ​\(wt\)=∇2R​\(wt\+1\)−1​\(∇2R​\(wt\)−η​∇2F​\(wt\)\)\.B\\Phi\(w\_\{t\}\)=\\nabla^\{2\}R\(w\_\{t\+1\}\)^\{\-1\}\\left\(\\nabla^\{2\}R\(w\_\{t\}\)\-\\eta\\nabla^\{2\}F\(w\_\{t\}\)\\right\)\.Along the axis, the Hessians are block diagonal\.

First consider the clock block\. For the scaled functions, the clock block is

54​Rclock′′​\(zt\)−η​165​h54​Rclock′′​\(zt\+1\)\.\\frac\{\\frac\{5\}\{4\}R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\_\{t\}\)\-\\eta\\frac\{1\}\{65\}h\}\{\\frac\{5\}\{4\}R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\_\{t\+1\}\)\}\.Since

Rclock′′​\(zt\+1\)−Rclock′′​\(zt\)=−24​h​ωσ​\(zt\+1−zt\)=−η~​h,R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\_\{t\+1\}\)\-R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\_\{t\}\)=\-\\frac\{24h\\omega\}\{\\sigma\}\(z\_\{t\+1\}\-z\_\{t\}\)=\-\\tilde\{\\eta\}h,and

η​165=54​η~,\\eta\\frac\{1\}\{65\}=\\frac\{5\}\{4\}\\tilde\{\\eta\},we have

54​Rclock′′​\(zt\)−η​165​h=54​\(Rclock′′​\(zt\)−η~​h\)=54​Rclock′′​\(zt\+1\)\.\\frac\{5\}\{4\}R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\_\{t\}\)\-\\eta\\frac\{1\}\{65\}h=\\frac\{5\}\{4\}\\left\(R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\_\{t\}\)\-\\tilde\{\\eta\}h\\right\)=\\frac\{5\}\{4\}R\_\{\\rm clock\}^\{\\prime\\prime\}\(z\_\{t\+1\}\)\.Thus the clock block is equal to11\.

The transverse block is

M​\(zt\+1\)−1​\(M​\(zt\)−η~​σ6​H​\(zt\)\)\.M\(z\_\{t\+1\}\)^\{\-1\}\\left\(M\(z\_\{t\}\)\-\\tilde\{\\eta\}\\frac\{\\sigma\}\{6\}H\(z\_\{t\}\)\\right\)\.Since

M​\(zt\)=P​\(θt\)​B​P​\(θt\)⊤,H​\(zt\)=P​\(θt\)​Q​P​\(θt\)⊤,M\(z\_\{t\}\)=P\(\\theta\_\{t\}\)BP\(\\theta\_\{t\}\)^\{\\top\},\\qquad H\(z\_\{t\}\)=P\(\\theta\_\{t\}\)QP\(\\theta\_\{t\}\)^\{\\top\},and

θt\+1−θt=ω​\(zt\+1−zt\)=σ​η~24,\\theta\_\{t\+1\}\-\\theta\_\{t\}=\\omega\(z\_\{t\+1\}\-z\_\{t\}\)=\\frac\{\\sigma\\tilde\{\\eta\}\}\{24\},the transverse block is

P​\(θt\+1\)​B−1​P​\(θt\+1\)⊤​P​\(θt\)​\(B−σ​η~6​Q\)​P​\(θt\)⊤\\displaystyle P\(\\theta\_\{t\+1\}\)B^\{\-1\}P\(\\theta\_\{t\+1\}\)^\{\\top\}P\(\\theta\_\{t\}\)\\left\(B\-\\frac\{\\sigma\\widetilde\{\\eta\}\}\{6\}Q\\right\)P\(\\theta\_\{t\}\)^\{\\top\}=P​\(θt\+1\)​B−1​P​\(θt−θt\+1\)​\(B−σ​η~6​Q\)​P​\(θt\)⊤\\displaystyle=P\(\\theta\_\{t\+1\}\)B^\{\-1\}P\(\\theta\_\{t\}\-\\theta\_\{t\+1\}\)\\left\(B\-\\frac\{\\sigma\\widetilde\{\\eta\}\}\{6\}Q\\right\)P\(\\theta\_\{t\}\)^\{\\top\}P​\(θt\+1\)​B−1​P​\(−σ​η~24\)​\(B−σ​η~6​Q\)​P​\(θt\)⊤\\displaystyle P\(\\theta\_\{t\+1\}\)B^\{\-1\}P\\left\(\-\\frac\{\\sigma\\tilde\{\\eta\}\}\{24\}\\right\)\\left\(B\-\\frac\{\\sigma\\widetilde\{\\eta\}\}\{6\}Q\\right\)P\(\\theta\_\{t\}\)^\{\\top\}=P​\(θt\+1\)​Aσ​P​\(θt\)⊤\.\\displaystyle=P\(\\theta\_\{t\+1\}\)A\_\{\\sigma\}P\(\\theta\_\{t\}\)^\{\\top\}\.Hence using telescoping terms thett\-step linearized map along the reference trajectory\. Then the transverse part of𝒥t\\mathcal\{J\}\_\{t\}is

P​\(θt\)​Aσt​P​\(θ0\)⊤=P​\(θt\)​AσtP\(\\theta\_\{t\}\)A\_\{\\sigma\}^\{t\}P\(\\theta\_\{0\}\)^\{\\top\}=P\(\\theta\_\{t\}\)A\_\{\\sigma\}^\{t\}From first order Taylor expansion,

cos⁡θ=1\+O​\(θ2\),sin⁡θ=θ\+O​\(θ3\)\\cos\\theta=1\+O\(\\theta^\{2\}\),\\qquad\\sin\\theta=\\theta\+O\(\\theta^\{3\}\)hence,

P​\(θ\)=\(1−θθ1\)\+O​\(θ2\)=I\+\(0−110\)​θ\+O​\(θ2\)\.P\(\\theta\)=\\begin\{pmatrix\}1&\-\\theta\\\\ \\theta&1\\end\{pmatrix\}\+O\(\\theta^\{2\}\)=I\+\\begin\{pmatrix\}0&\-1\\\\ 1&0\\end\{pmatrix\}\\theta\+O\(\\theta^\{2\}\)\.Lets:=σ​η~,s:=\\sigma\\tilde\{\\eta\},then

Aσ\\displaystyle A\_\{\\sigma\}=B−1​\(I\+\(0−110\)​\(−s24\)\+O​\(s2\)\)​\(B−s6​Q\)\\displaystyle=B^\{\-1\}\\left\(I\+\\begin\{pmatrix\}0&\-1\\\\ 1&0\\end\{pmatrix\}\\left\(\-\\frac\{s\}\{24\}\\right\)\+O\(s^\{2\}\)\\right\)\\left\(B\-\\frac\{s\}\{6\}Q\\right\)=I−s6​B−1​Q−B−1​\(0−110\)​s24​B\+O​\(s2\)\.\\displaystyle=I\-\\frac\{s\}\{6\}B^\{\-1\}Q\-B^\{\-1\}\\begin\{pmatrix\}0&\-1\\\\ 1&0\\end\{pmatrix\}\\frac\{s\}\{24\}B\+O\(s^\{2\}\)\.Let

G=−16​B−1​Q−124​B−1​\(0−110\)​BG=\-\\frac\{1\}\{6\}B^\{\-1\}Q\-\\frac\{1\}\{24\}B^\{\-1\}\\begin\{pmatrix\}0&\-1\\\\ 1&0\\end\{pmatrix\}BAss↓0s\\downarrow 0,

Aσ=I\+s​G\+O​\(s2\),A\_\{\\sigma\}=I\+sG\+O\(s^\{2\}\),A direct computation gives

G=\(−11180−124−724−1160\),det\(G\)=−4143200<0\.G=\\begin\{pmatrix\}\-\\frac\{11\}\{180\}&\-\\frac\{1\}\{24\}\\\\ \-\\frac\{7\}\{24\}&\-\\frac\{11\}\{60\}\\end\{pmatrix\},\\qquad\\det\(G\)=\-\\frac\{41\}\{43200\}<0\.ThusGGhas one positive and one negative real eigenvalue\. Standard perturbation theory gives an expanding eigenvalueρ\>1\\rho\>1ofAσA\_\{\\sigma\}withρ−1=Θ​\(s\)\\rho\-1=\\Theta\(s\), thus sinces=σ​η~s=\\sigma\\tilde\{\\eta\}andη~\\tilde\{\\eta\}is an absolute multiple ofη\\etathere exist some absolute constantsc,C\>0c,C\>0such that

c​σ​η≤ρ−1≤C​σ​η\.c\\sigma\\eta\\leq\\rho\-1\\leq C\\sigma\\eta\.Sinceρ−1≥0\\rho\-1\\geq 0choosingη0\\eta\_\{0\}to be small enough such thatρ−1≤1\\rho\-1\\leq 1for anyσ∈\(0,1\]\\sigma\\in\(0,1\]we also have

c​σ​η2≤ρ−12≤log⁡ρ≤ρ−1≤C​σ​η\.\\frac\{c\\sigma\\eta\}\{2\}\\leq\\frac\{\\rho\-1\}\{2\}\\leq\\log\\rho\\leq\\rho\-1\\leq C\\sigma\\eta\.Choosingcρ=c2c\_\{\\rho\}=\\frac\{c\}\{2\}andCρ=CC\_\{\\rho\}=Cfinished this part\. Finally, sinceBBhas two distinct real eigenvalues, its eigenbasis is nondegenerate\. After decreasing the absolute constantη0\>0\\eta\_\{0\}\>0, the parameters=σ​η~s=\\sigma\\tilde\{\\eta\}is uniformly small for all0<σ≤10<\\sigma\\leq 1and0<η≤η00<\\eta\\leq\\eta\_\{0\}\. Hence the eigenvectors ofAσ=I\+s​G\+O​\(s2\)A\_\{\\sigma\}=I\+sG\+O\(s^\{2\}\)are uniformly close to the eigenvectors ofGG, and the corresponding eigenbasis ofAσA\_\{\\sigma\}is uniformly well\-conditioned\.

#### C\.2\.4Proof of Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)

The update satisfies

∇R​\(Φ​\(y\)\)=∇R​\(y\)−η​∇F​\(y\)\.\\nabla R\(\\Phi\(y\)\)=\\nabla R\(y\)\-\\eta\\nabla F\(y\)\.Bifferentiating once gives

∇2R​\(Φ​\(y\)\)​D​Φ​\(y\)=∇2R​\(y\)−η​∇2F​\(y\)\.\\nabla^\{2\}R\(\\Phi\(y\)\)D\\Phi\(y\)=\\nabla^\{2\}R\(y\)\-\\eta\\nabla^\{2\}F\(y\)\.Subtracting∇2R​\(Φ​\(y\)\)\\nabla^\{2\}R\(\\Phi\(y\)\)from both sides gives

∇2R​\(Φ​\(y\)\)​\(D​Φ​\(y\)−I\)=∇2R​\(y\)−∇2R​\(Φ​\(y\)\)−η​∇2F​\(y\)\.\\nabla^\{2\}R\(\\Phi\(y\)\)\(D\\Phi\(y\)\-I\)=\\nabla^\{2\}R\(y\)\-\\nabla^\{2\}R\(\\Phi\(y\)\)\-\\eta\\nabla^\{2\}F\(y\)\.Since∇2R⪰I\\nabla^\{2\}R\\succeq I,

‖\(∇2R​\(Φ​\(y\)\)\)−1‖≤1\.\\\|\(\\nabla^\{2\}R\(\\Phi\(y\)\)\)^\{\-1\}\\\|\\leq 1\.Using theD3​RD^\{3\}Rbound along the segment\[y,Φ​\(y\)\]⊂U\[y,\\Phi\(y\)\]\\subset U, we get

‖D​Φ​\(y\)−I‖\\displaystyle\\\|D\\Phi\(y\)\-I\\\|≤‖∇2R​\(y\)−∇2R​\(Φ​\(y\)\)‖\+η​‖∇2F​\(y\)‖\\displaystyle\\leq\\\|\\nabla^\{2\}R\(y\)\-\\nabla^\{2\}R\(\\Phi\(y\)\)\\\|\+\\eta\\\|\\nabla^\{2\}F\(y\)\\\|≤M3​‖Φ​\(y\)−y‖\+η​N2\\displaystyle\\leq M\_\{3\}\\\|\\Phi\(y\)\-y\\\|\+\\eta N\_\{2\}≤M3​Δ\+η​N2\.\\displaystyle\\leq M\_\{3\}\\Delta\+\\eta N\_\{2\}\.
Differentiating the implicit equation twice in unit directionsp,qp,qon both sides gives

∇2R​\(Φ​\(y\)\)​D2​Φ​\(y\)​\[p,q\]\+D3​R​\(Φ​\(y\)\)​\[D​Φ​\(y\)​p,D​Φ​\(y\)​q\]=D3​R​\(y\)​\[p,q\]−η​D3​F​\(y\)​\[p,q\]\.\\displaystyle\\nabla^\{2\}R\(\\Phi\(y\)\)D^\{2\}\\Phi\(y\)\[p,q\]\+D^\{3\}R\(\\Phi\(y\)\)\[D\\Phi\(y\)p,D\\Phi\(y\)q\]=D^\{3\}R\(y\)\[p,q\]\-\\eta D^\{3\}F\(y\)\[p,q\]\.Using again‖\(∇2R​\(Φ​\(y\)\)\)−1‖≤1\\\|\(\\nabla^\{2\}R\(\\Phi\(y\)\)\)^\{\-1\}\\\|\\leq 1, and adding and subtractingD3​R​\(Φ​\(y\)\)​\[p,q\]D^\{3\}R\(\\Phi\(y\)\)\[p,q\], we obtain

‖D2​Φ​\(y\)‖\\displaystyle\\\|D^\{2\}\\Phi\(y\)\\\|≤‖D3​R​\(y\)−D3​R​\(Φ​\(y\)\)‖\+η​‖D3​F​\(y\)‖\\displaystyle\\leq\\\|D^\{3\}R\(y\)\-D^\{3\}R\(\\Phi\(y\)\)\\\|\+\\eta\\\|D^\{3\}F\(y\)\\\|\+sup‖p‖=‖q‖=1‖D3​R​\(Φ​\(y\)\)​\[D​Φ​\(y\)​p,D​Φ​\(y\)​q\]−D3​R​\(Φ​\(y\)\)​\[p,q\]‖\.\\displaystyle\\quad\+\\sup\_\{\\\|p\\\|=\\\|q\\\|=1\}\\left\\\|D^\{3\}R\(\\Phi\(y\)\)\[D\\Phi\(y\)p,D\\Phi\(y\)q\]\-D^\{3\}R\(\\Phi\(y\)\)\[p,q\]\\right\\\|\.Since\[y,Φ​\(y\)\]⊂U\[y,\\Phi\(y\)\]\\subset U, theD4​RD^\{4\}Rbound gives

‖D3​R​\(y\)−D3​R​\(Φ​\(y\)\)‖≤M4​‖Φ​\(y\)−y‖≤M4​Δ\.\\\|D^\{3\}R\(y\)\-D^\{3\}R\(\\Phi\(y\)\)\\\|\\leq M\_\{4\}\\\|\\Phi\(y\)\-y\\\|\\leq M\_\{4\}\\Delta\.Let

E:=‖D​Φ​\(y\)−I‖\.E:=\\\|D\\Phi\(y\)\-I\\\|\.For unitp,qp,q,

D​Φ​\(y\)​p=p\+\(D​Φ​\(y\)−I\)​p,D​Φ​\(y\)​q=q\+\(D​Φ​\(y\)−I\)​q\.D\\Phi\(y\)p=p\+\(D\\Phi\(y\)\-I\)p,\\qquad D\\Phi\(y\)q=q\+\(D\\Phi\(y\)\-I\)q\.By bilinearity in the first two slots,

D3​R​\(Φ​\(y\)\)​\[D​Φ​\(y\)​p,D​Φ​\(y\)​q\]−D3​R​\(Φ​\(y\)\)​\[p,q\]\\displaystyle D^\{3\}R\(\\Phi\(y\)\)\[D\\Phi\(y\)p,D\\Phi\(y\)q\]\-D^\{3\}R\(\\Phi\(y\)\)\[p,q\]=D3​R​\(Φ​\(y\)\)​\[\(D​Φ​\(y\)−I\)​p,q\]\+D3​R​\(Φ​\(y\)\)​\[p,\(D​Φ​\(y\)−I\)​q\]\\displaystyle=D^\{3\}R\(\\Phi\(y\)\)\[\(D\\Phi\(y\)\-I\)p,q\]\+D^\{3\}R\(\\Phi\(y\)\)\[p,\(D\\Phi\(y\)\-I\)q\]\+D3​R​\(Φ​\(y\)\)​\[\(D​Φ​\(y\)−I\)​p,\(D​Φ​\(y\)−I\)​q\]\.\\displaystyle\\quad\+D^\{3\}R\(\\Phi\(y\)\)\[\(D\\Phi\(y\)\-I\)p,\(D\\Phi\(y\)\-I\)q\]\.Therefore

‖D3​R​\(Φ​\(y\)\)​\[D​Φ​\(y\)​p,D​Φ​\(y\)​q\]−D3​R​\(Φ​\(y\)\)​\[p,q\]‖≤2​M3​\(E\+E2\)\.\\left\\\|D^\{3\}R\(\\Phi\(y\)\)\[D\\Phi\(y\)p,D\\Phi\(y\)q\]\-D^\{3\}R\(\\Phi\(y\)\)\[p,q\]\\right\\\|\\leq 2M\_\{3\}\(E\+E^\{2\}\)\.Combining the estimates,

‖D2​Φ​\(y\)‖≤2​\(M4​Δ\+η​N3\+M3​E\+M3​E2\)\.\\\|D^\{2\}\\Phi\(y\)\\\|\\leq 2\\left\(M\_\{4\}\\Delta\+\\eta N\_\{3\}\+M\_\{3\}E\+M\_\{3\}E^\{2\}\\right\)\.Using

E≤M3​Δ\+η​N2,E\\leq M\_\{3\}\\Delta\+\\eta N\_\{2\},we conclude

‖D2​Φ​\(y\)‖≤2​\(M4​Δ\+η​N3\+M3​\(M3​Δ\+η​N2\)\+M3​\(M3​Δ\+η​N2\)2\)\.\\\|D^\{2\}\\Phi\(y\)\\\|\\leq 2\\Bigl\(M\_\{4\}\\Delta\+\\eta N\_\{3\}\+M\_\{3\}\(M\_\{3\}\\Delta\+\\eta N\_\{2\}\)\+M\_\{3\}\(M\_\{3\}\\Delta\+\\eta N\_\{2\}\)^\{2\}\\Bigr\)\.

#### C\.2\.5Proof of Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)

We first record the local well\-definedness of the update\. Sincedist⁡\(wt,Uc\)≥γ2​ω−3/2,\\operatorname\{dist\}\(w\_\{t\},U^\{c\}\)\\geq\\frac\{\\gamma\}\{2\}\\omega^\{\-3/2\},the assumptionb≤cb​ω−3/2b\\leq c\_\{b\}\\omega^\{\-3/2\}, after decreasingcb\>0c\_\{b\}\>0, implies

B​\(wt,K​b\)⊂U,0≤t<T\.B\(w\_\{t\},Kb\)\\subset U,\\qquad 0\\leq t<T\.[SectionC\.2](https://arxiv.org/html/2606.11431#A3.SS2)applied atwt\+1w\_\{t\+1\}, together with∇2R⪰I\\nabla^\{2\}R\\succeq I, gives a11\-Lipschitz inverse branch of∇R\\nabla Rthroughwt\+1w\_\{t\+1\}on a ball of radius comparable toω−3/2\\omega^\{\-3/2\}around∇R​\(wt\+1\)\\nabla R\(w\_\{t\+1\}\)\. By Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2),

∇R​\(wt\+1\)=∇R​\(wt\)−η​∇F​\(wt\)\.\\nabla R\(w\_\{t\+1\}\)=\\nabla R\(w\_\{t\}\)\-\\eta\\nabla F\(w\_\{t\}\)\.Moreover, onUU,

‖∇2R‖\+η​‖∇2F‖≤C0\+η\.\\\|\\nabla^\{2\}R\\\|\+\\eta\\\|\\nabla^\{2\}F\\\|\\leq C\_\{0\}\+\\eta\.Hence for everyy∈B​\(wt,K​b\)y\\in B\(w\_\{t\},Kb\)takingη0≤1\\eta\_\{0\}\\leq 1,

‖\(∇R​\(y\)−η​∇F​\(y\)\)−\(∇R​\(wt\)−η​∇F​\(wt\)\)‖≲K​b\.\\displaystyle\\left\\\|\\bigl\(\\nabla R\(y\)\-\\eta\\nabla F\(y\)\\bigr\)\-\\bigl\(\\nabla R\(w\_\{t\}\)\-\\eta\\nabla F\(w\_\{t\}\)\\bigr\)\\right\\\|\\lesssim Kb\.Thus, after decreasingcbc\_\{b\}further if necessary,Φ\\Phiis well\-defined on each ballB​\(wt,K​b\)B\(w\_\{t\},Kb\)\. Next, from[SectionC\.2](https://arxiv.org/html/2606.11431#A3.SS2)

‖Φ​\(y\)−y‖2≤‖∇R​\(Φ​\(y\)\)−∇R​\(y\)‖2\.\\\|\\Phi\(y\)\-y\\\|\_\{2\}\\leq\\\|\\nabla R\(\\Phi\(y\)\)\-\\nabla R\(y\)\\\|\_\{2\}\.Using

∇R​\(Φ​\(y\)\)−∇R​\(y\)=−η​∇F​\(y\),\\nabla R\(\\Phi\(y\)\)\-\\nabla R\(y\)=\-\\eta\\nabla F\(y\),we obtain

‖Φ​\(y\)−y‖2≤η​‖∇F​\(y\)‖2≤C1​165​σ​η\.\\\|\\Phi\(y\)\-y\\\|\_\{2\}\\leq\\eta\\\|\\nabla F\(y\)\\\|\_\{2\}\\leq C\_\{1\}\\frac\{1\}\{65\}\\sigma\\eta\.
We now apply Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)\. By Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2), onUU,

‖D3​R‖≤C1​ω,‖D4​R‖≤C1​ω2,\\\|D^\{3\}R\\\|\\leq C\_\{1\}\\omega,\\qquad\\\|D^\{4\}R\\\|\\leq C\_\{1\}\\omega^\{2\},and

‖D2​F‖≤C1​165​σ,‖D3​F‖≤C1​165​σ​ω\.\\\|D^\{2\}F\\\|\\leq C\_\{1\}\\frac\{1\}\{65\}\\sigma,\\qquad\\\|D^\{3\}F\\\|\\leq C\_\{1\}\\frac\{1\}\{65\}\\sigma\\omega\.Together with

‖Φ​\(y\)−y‖2≤C1​165​σ​η,\\\|\\Phi\(y\)\-y\\\|\_\{2\}\\leq C\_\{1\}\\frac\{1\}\{65\}\\sigma\\eta,Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)gives, on the relevant segments,

‖D2​Φ​\(y\)‖\\displaystyle\\\|D^\{2\}\\Phi\(y\)\\\|≤2​\(ω2​165​σ​η\+η​165​σ​ω\+ω​\(ω​165​σ​η\+η​165​σ\)\+ω​\(ω​165​σ​η\+η​165​σ\)2\)\\displaystyle\\leq 2\\Bigl\(\\omega^\{2\}\\frac\{1\}\{65\}\\sigma\\eta\+\\eta\\frac\{1\}\{65\}\\sigma\\omega\+\\omega\(\\omega\\frac\{1\}\{65\}\\sigma\\eta\+\\eta\\frac\{1\}\{65\}\\sigma\)\+\\omega\(\\omega\\frac\{1\}\{65\}\\sigma\\eta\+\\eta\\frac\{1\}\{65\}\\sigma\)^\{2\}\\Bigr\)≤16​σ​η​ω2​\(1\+σ​η​ω\)\\displaystyle\\leq 6\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)
Now assume

\[wt,wt′\]⊂B​\(wt,K​b\)\.\[w\_\{t\},w^\{\\prime\}\_\{t\}\]\\subset B\(w\_\{t\},Kb\)\.Taylor’s theorem applied toΦ\\Phion the segment\[wt,wt′\]\[w\_\{t\},w^\{\\prime\}\_\{t\}\]yields

‖Φ​\(wt′\)−Φ​\(wt\)−D​Φ​\(wt\)​\(wt′−wt\)‖2≤16​σ​η​ω2​\(1\+σ​η​ω\)​‖wt′−wt‖22\.\\displaystyle\\left\\\|\\Phi\(w^\{\\prime\}\_\{t\}\)\-\\Phi\(w\_\{t\}\)\-D\\Phi\(w\_\{t\}\)\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\)\\right\\\|\_\{2\}\\leq 6\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\\|w^\{\\prime\}\_\{t\}\-w\_\{t\}\\\|\_\{2\}^\{2\}\.Passing to the rotating frame the nonlinear error is exactly the Taylor remainder of the update mapΦ\\PhiIndeed, sincewt\+1=Φ​\(wt\)w\_\{t\+1\}=\\Phi\(w\_\{t\}\)andwt\+1′=Φ​\(wt′\)w^\{\\prime\}\_\{t\+1\}=\\Phi\(w^\{\\prime\}\_\{t\}\), we have

Yt\+1=𝒯t\+1​\(wt\+1′−wt\+1\)=𝒯t\+1​\(Φ​\(wt′\)−Φ​\(wt\)\)\.Y\_\{t\+1\}=\\mathcal\{T\}\_\{t\+1\}\(w^\{\\prime\}\_\{t\+1\}\-w\_\{t\+1\}\)=\\mathcal\{T\}\_\{t\+1\}\\bigl\(\\Phi\(w^\{\\prime\}\_\{t\}\)\-\\Phi\(w\_\{t\}\)\\bigr\)\.On the other hand, using

𝒯t\+1​D​Φ​\(wt\)​𝒯t−1=L,Yt=𝒯t​\(wt′−wt\),\\mathcal\{T\}\_\{t\+1\}D\\Phi\(w\_\{t\}\)\\mathcal\{T\}\_\{t\}^\{\-1\}=L,\\qquad Y\_\{t\}=\\mathcal\{T\}\_\{t\}\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\),we get

L​Yt\\displaystyle LY\_\{t\}=L​𝒯t​\(wt′−wt\)\\displaystyle=L\\mathcal\{T\}\_\{t\}\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\)=𝒯t\+1​D​Φ​\(wt\)​𝒯t−1​𝒯t​\(wt′−wt\)\\displaystyle=\\mathcal\{T\}\_\{t\+1\}D\\Phi\(w\_\{t\}\)\\mathcal\{T\}\_\{t\}^\{\-1\}\\mathcal\{T\}\_\{t\}\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\)=𝒯t\+1​D​Φ​\(wt\)​\(wt′−wt\)\.\\displaystyle=\\mathcal\{T\}\_\{t\+1\}D\\Phi\(w\_\{t\}\)\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\)\.Therefore,

Yt\+1−L​Yt\\displaystyle Y\_\{t\+1\}\-LY\_\{t\}=𝒯t\+1​\(Φ​\(wt′\)−Φ​\(wt\)\)−𝒯t\+1​D​Φ​\(wt\)​\(wt′−wt\)\\displaystyle=\\mathcal\{T\}\_\{t\+1\}\\bigl\(\\Phi\(w^\{\\prime\}\_\{t\}\)\-\\Phi\(w\_\{t\}\)\\bigr\)\-\\mathcal\{T\}\_\{t\+1\}D\\Phi\(w\_\{t\}\)\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\)=𝒯t\+1​\[Φ​\(wt′\)−Φ​\(wt\)−D​Φ​\(wt\)​\(wt′−wt\)\]\.\\displaystyle=\\mathcal\{T\}\_\{t\+1\}\\left\[\\Phi\(w^\{\\prime\}\_\{t\}\)\-\\Phi\(w\_\{t\}\)\-D\\Phi\(w\_\{t\}\)\(w^\{\\prime\}\_\{t\}\-w\_\{t\}\)\\right\]\.Now, since𝒯t\\mathcal\{T\}\_\{t\}is an isometry in Euclidean norm and that∥⋅∥\#\\\|\\cdot\\\|\_\{\\\#\}is uniformly equivalent to the Euclidean norm, there exists an absolute constantC2\>0C\_\{2\}\>0such that

‖Yt\+1−L​Yt‖\#≤C2​σ​η​ω2​\(1\+σ​η​ω\)​‖Yt‖\#2,\\\|Y\_\{t\+1\}\-LY\_\{t\}\\\|\_\{\\\#\}\\leq C\_\{2\}\\sigma\\eta\\omega^\{2\}\(1\+\\sigma\\eta\\omega\)\\\|Y\_\{t\}\\\|\_\{\\\#\}^\{2\},which concludes the proof\.

#### C\.2\.6Proof of Lemma[C\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)

[SectionC\.1](https://arxiv.org/html/2606.11431#A3.SS1.SSS0.Px2)Lett∗t\_\{\*\}be the first index such thatqt∗\>2q\_\{t\_\{\*\}\}\>2, if such an index exists\. Thent∗≥1t\_\{\*\}\\geq 1andqt−1≤2q\_\{t\-1\}\\leq 2for every1≤t<t∗1\\leq t<t\_\{\*\}\. Summing the recursion up to timet∗t\_\{\*\}yields

qt∗≤1\+∑t=1t∗at​qt−12≤1\+4​∑t=1t∗at≤2,q\_\{t\_\{\*\}\}\\leq 1\+\\sum\_\{t=1\}^\{t\_\{\*\}\}a\_\{t\}q\_\{t\-1\}^\{2\}\\leq 1\+4\\sum\_\{t=1\}^\{t\_\{\*\}\}a\_\{t\}\\leq 2,a contradiction\. Hence no sucht∗t\_\{\*\}exists\.

## Appendix DProofs for Section[4](https://arxiv.org/html/2606.11431#S4)

### D\.1Proof of Theorem[3](https://arxiv.org/html/2606.11431#Thmtheorem3)

We first record two elementary stability properties of the reweighting map that appears in entropy mirror descent\.

###### Lemma 26\.

For any vectora∈ℝ\>0da\\in\\mathbb\{R\}\_\{\>0\}^\{d\}andw,w′∈Δd∘w,w^\{\\prime\}\\in\\Delta\_\{d\}^\{\\circ\}, define

Ψa​\(w\)​\(i\)=a​\(i\)​w​\(i\)∑j=1da​\(j\)​w​\(j\)\.\\Psi\_\{a\}\(w\)\(i\)=\\frac\{a\(i\)w\(i\)\}\{\\sum\_\{j=1\}^\{d\}a\(j\)w\(j\)\}\.The following hold:

1. 1\.‖Ψa​\(w\)−Ψa​\(w′\)‖1≤maxi⁡a​\(i\)mini⁡a​\(i\)​‖w−w′‖1\\\|\\Psi\_\{a\}\(w\)\-\\Psi\_\{a\}\(w^\{\\prime\}\)\\\|\_\{1\}\\leq\\frac\{\\max\_\{i\}a\(i\)\}\{\\min\_\{i\}a\(i\)\}\\\|w\-w^\{\\prime\}\\\|\_\{1\}\.
2. 2\.For anya′∈ℝ\>0da^\{\\prime\}\\in\\mathbb\{R\}\_\{\>0\}^\{d\}, ‖Ψa​\(w\)−Ψa′​\(w\)‖1≤maxi⁡a​\(i\)/a′​\(i\)mini⁡a​\(i\)/a′​\(i\)−1\.\\\|\\Psi\_\{a\}\(w\)\-\\Psi\_\{a^\{\\prime\}\}\(w\)\\\|\_\{1\}\\leq\\frac\{\\max\_\{i\}a\(i\)/a^\{\\prime\}\(i\)\}\{\\min\_\{i\}a\(i\)/a^\{\\prime\}\(i\)\}\-1\.

###### Proof of[SectionD\.1](https://arxiv.org/html/2606.11431#A4.SS1)\.

We prove the two claims separately\. For the first claim, use

∑i\(w​\(i\)−min⁡\{w​\(i\),w′​\(i\)\}\)=∑i\(w′​\(i\)−min⁡\{w​\(i\),w′​\(i\)\}\)=12​‖w−w′‖1\.\\sum\_\{i\}\\bigl\(w\(i\)\-\\min\\\{w\(i\),w^\{\\prime\}\(i\)\\\}\\bigr\)=\\sum\_\{i\}\\bigl\(w^\{\\prime\}\(i\)\-\\min\\\{w\(i\),w^\{\\prime\}\(i\)\\\}\\bigr\)=\\frac\{1\}\{2\}\\\|w\-w^\{\\prime\}\\\|\_\{1\}\.LetZ=∑ia​\(i\)​w​\(i\)Z=\\sum\_\{i\}a\(i\)w\(i\)andZ′=∑ia​\(i\)​w′​\(i\)Z^\{\\prime\}=\\sum\_\{i\}a\(i\)w^\{\\prime\}\(i\)\. Since

∑imin⁡\{Ψa​\(w\)​\(i\),Ψa​\(w′\)​\(i\)\}≥∑ia​\(i\)​min⁡\{w​\(i\),w′​\(i\)\}max⁡\{Z,Z′\},\\sum\_\{i\}\\min\\\{\\Psi\_\{a\}\(w\)\(i\),\\Psi\_\{a\}\(w^\{\\prime\}\)\(i\)\\\}\\geq\\frac\{\\sum\_\{i\}a\(i\)\\min\\\{w\(i\),w^\{\\prime\}\(i\)\\\}\}\{\\max\\\{Z,Z^\{\\prime\}\\\}\},and sinceΨa​\(w\)\\Psi\_\{a\}\(w\)andΨa​\(w′\)\\Psi\_\{a\}\(w^\{\\prime\}\)are probability vectors,

‖Ψa​\(w\)−Ψa​\(w′\)‖1≤2​max⁡\{Z,Z′\}−∑ia​\(i\)​min⁡\{w​\(i\),w′​\(i\)\}max⁡\{Z,Z′\}\.\\\|\\Psi\_\{a\}\(w\)\-\\Psi\_\{a\}\(w^\{\\prime\}\)\\\|\_\{1\}\\leq 2\\,\\frac\{\\max\\\{Z,Z^\{\\prime\}\\\}\-\\sum\_\{i\}a\(i\)\\min\\\{w\(i\),w^\{\\prime\}\(i\)\\\}\}\{\\max\\\{Z,Z^\{\\prime\}\\\}\}\.The numerator is at most12​\(maxi⁡a​\(i\)\)​‖w−w′‖1\\frac\{1\}\{2\}\(\\max\_\{i\}a\(i\)\)\\\|w\-w^\{\\prime\}\\\|\_\{1\}, while the denominator is at leastmini⁡a​\(i\)\\min\_\{i\}a\(i\)\. This proves the first claim\.

For the second claim, letri=a​\(i\)/a′​\(i\)r\_\{i\}=a\(i\)/a^\{\\prime\}\(i\),rmin=mini⁡rir\_\{\\min\}=\\min\_\{i\}r\_\{i\}, andrmax=maxi⁡rir\_\{\\max\}=\\max\_\{i\}r\_\{i\}\. For everyii,

Ψa​\(w\)​\(i\)Ψa′​\(w\)​\(i\)=ri∑jrj​a′​\(j\)​w​\(j\)/∑ja′​\(j\)​w​\(j\)\.\\frac\{\\Psi\_\{a\}\(w\)\(i\)\}\{\\Psi\_\{a^\{\\prime\}\}\(w\)\(i\)\}=\\frac\{r\_\{i\}\}\{\\sum\_\{j\}r\_\{j\}a^\{\\prime\}\(j\)w\(j\)/\\sum\_\{j\}a^\{\\prime\}\(j\)w\(j\)\}\.The denominator on the right\-hand side is a weighted average of therjr\_\{j\}’s, and therefore lies in\[rmin,rmax\]\[r\_\{\\min\},r\_\{\\max\}\]\. Hence

rminrmax≤Ψa​\(w\)​\(i\)Ψa′​\(w\)​\(i\)≤rmaxrmin,\\frac\{r\_\{\\min\}\}\{r\_\{\\max\}\}\\leq\\frac\{\\Psi\_\{a\}\(w\)\(i\)\}\{\\Psi\_\{a^\{\\prime\}\}\(w\)\(i\)\}\\leq\\frac\{r\_\{\\max\}\}\{r\_\{\\min\}\},which implies

\|Ψa​\(w\)​\(i\)Ψa′​\(w\)​\(i\)−1\|≤rmaxrmin−1\.\\left\|\\frac\{\\Psi\_\{a\}\(w\)\(i\)\}\{\\Psi\_\{a^\{\\prime\}\}\(w\)\(i\)\}\-1\\right\|\\leq\\frac\{r\_\{\\max\}\}\{r\_\{\\min\}\}\-1\.Multiplying byΨa′​\(w\)​\(i\)\\Psi\_\{a^\{\\prime\}\}\(w\)\(i\)and summing overiiproves the second claim\. ∎

We are now ready to prove[Theorem3](https://arxiv.org/html/2606.11431#Thmtheorem3)\.

###### Proof of[Theorem3](https://arxiv.org/html/2606.11431#Thmtheorem3)\.

Fixw0,w0′∈Δd∘w\_\{0\},w\_\{0\}^\{\\prime\}\\in\\Delta\_\{d\}^\{\\circ\}, and let\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}and\{wt′\}t=0T\\\{w\_\{t\}^\{\\prime\}\\\}\_\{t=0\}^\{T\}be the corresponding MD trajectories\. For eacht=0,…,T−1t=0,\\ldots,T\-1, define the coordinatewise exponentials

gt=e−η​∇F​\(wt\),gt′=e−η​∇F​\(wt′\)\.g\_\{t\}=e^\{\-\\eta\\nabla F\(w\_\{t\}\)\},\\qquad g\_\{t\}^\{\\prime\}=e^\{\-\\eta\\nabla F\(w\_\{t\}^\{\\prime\}\)\}\.The negative\-entropy MD update can be written as

wt\+1=Ψgt​\(wt\),wt\+1′=Ψgt′​\(wt′\)\.w\_\{t\+1\}=\\Psi\_\{g\_\{t\}\}\(w\_\{t\}\),\\qquad w\_\{t\+1\}^\{\\prime\}=\\Psi\_\{g\_\{t\}^\{\\prime\}\}\(w\_\{t\}^\{\\prime\}\)\.Therefore,

‖wt\+1−wt\+1′‖1\\displaystyle\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\\|\_\{1\}≤‖Ψgt​\(wt\)−Ψgt​\(wt′\)‖1\+‖Ψgt​\(wt′\)−Ψgt′​\(wt′\)‖1\.\\displaystyle\\leq\\\|\\Psi\_\{g\_\{t\}\}\(w\_\{t\}\)\-\\Psi\_\{g\_\{t\}\}\(w\_\{t\}^\{\\prime\}\)\\\|\_\{1\}\+\\\|\\Psi\_\{g\_\{t\}\}\(w\_\{t\}^\{\\prime\}\)\-\\Psi\_\{g\_\{t\}^\{\\prime\}\}\(w\_\{t\}^\{\\prime\}\)\\\|\_\{1\}\.
We bound the two terms separately\. By[SectionD\.1](https://arxiv.org/html/2606.11431#A4.SS1),

‖Ψgt​\(wt\)−Ψgt​\(wt′\)‖1≤maxi⁡gt​\(i\)mini⁡gt​\(i\)​‖wt−wt′‖1\.\\\|\\Psi\_\{g\_\{t\}\}\(w\_\{t\}\)\-\\Psi\_\{g\_\{t\}\}\(w\_\{t\}^\{\\prime\}\)\\\|\_\{1\}\\leq\\frac\{\\max\_\{i\}g\_\{t\}\(i\)\}\{\\min\_\{i\}g\_\{t\}\(i\)\}\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\.For anyi,ji,j, the tangent directionei−eje\_\{i\}\-e\_\{j\}hasℓ1\\ell\_\{1\}\-norm22\. SinceFFisGG\-Lipschitz on the open simplex,\|∇F​\(wt\)​\(i\)−∇F​\(wt\)​\(j\)\|≤2​G\|\\nabla F\(w\_\{t\}\)\(i\)\-\\nabla F\(w\_\{t\}\)\(j\)\|\\leq 2G\. Hence

maxi⁡gt​\(i\)mini⁡gt​\(i\)≤e2​η​G,\\frac\{\\max\_\{i\}g\_\{t\}\(i\)\}\{\\min\_\{i\}g\_\{t\}\(i\)\}\\leq e^\{2\\eta G\},and the first term is at moste2​η​G​‖wt−wt′‖1e^\{2\\eta G\}\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\.

For the second term, another application of[SectionD\.1](https://arxiv.org/html/2606.11431#A4.SS1)gives

‖Ψgt​\(wt′\)−Ψgt′​\(wt′\)‖1≤maxi⁡gt​\(i\)/gt′​\(i\)mini⁡gt​\(i\)/gt′​\(i\)−1\.\\\|\\Psi\_\{g\_\{t\}\}\(w\_\{t\}^\{\\prime\}\)\-\\Psi\_\{g\_\{t\}^\{\\prime\}\}\(w\_\{t\}^\{\\prime\}\)\\\|\_\{1\}\\leq\\frac\{\\max\_\{i\}g\_\{t\}\(i\)/g\_\{t\}^\{\\prime\}\(i\)\}\{\\min\_\{i\}g\_\{t\}\(i\)/g\_\{t\}^\{\\prime\}\(i\)\}\-1\.ByLL\-smoothness,

maxi⁡\(∇F​\(wt\)​\(i\)−∇F​\(wt′\)​\(i\)\)−mini⁡\(∇F​\(wt\)​\(i\)−∇F​\(wt′\)​\(i\)\)≤2​L​‖wt−wt′‖1\.\\max\_\{i\}\\bigl\(\\nabla F\(w\_\{t\}\)\(i\)\-\\nabla F\(w\_\{t\}^\{\\prime\}\)\(i\)\\bigr\)\-\\min\_\{i\}\\bigl\(\\nabla F\(w\_\{t\}\)\(i\)\-\\nabla F\(w\_\{t\}^\{\\prime\}\)\(i\)\\bigr\)\\leq 2L\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\.Therefore,

‖Ψgt​\(wt′\)−Ψgt′​\(wt′\)‖1≤e2​η​L​‖wt−wt′‖1−1\.\\\|\\Psi\_\{g\_\{t\}\}\(w\_\{t\}^\{\\prime\}\)\-\\Psi\_\{g\_\{t\}^\{\\prime\}\}\(w\_\{t\}^\{\\prime\}\)\\\|\_\{1\}\\leq e^\{2\\eta L\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\}\-1\.Since‖wt−wt′‖1≤2\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\\leq 2, the monotonicity ofz↦\(e2​η​L​z−1\)/zz\\mapsto\(e^\{2\\eta Lz\}\-1\)/zonz\>0z\>0, with continuity atz=0z=0, implies

e2​η​L​‖wt−wt′‖1−1≤e4​η​L−12​‖wt−wt′‖1\.e^\{2\\eta L\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\}\-1\\leq\\frac\{e^\{4\\eta L\}\-1\}\{2\}\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\.Combining the two bounds,

‖wt\+1−wt\+1′‖1\\displaystyle\\\|w\_\{t\+1\}\-w\_\{t\+1\}^\{\\prime\}\\\|\_\{1\}≤\(e2​η​G\+e4​η​L−12\)​‖wt−wt′‖1\\displaystyle\\leq\\left\(e^\{2\\eta G\}\+\\frac\{e^\{4\\eta L\}\-1\}\{2\}\\right\)\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}≤e\(2​G\+4​L\)​η​‖wt−wt′‖1\.\\displaystyle\\leq e^\{\(2G\+4L\)\\eta\}\\\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\\\|\_\{1\}\.Iterating forTTsteps gives

‖wT−wT′‖1≤e\(2​G\+4​L\)​η​T​‖w0−w0′‖1\.\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}\\leq e^\{\(2G\+4L\)\\eta T\}\\\|w\_\{0\}\-w\_\{0\}^\{\\prime\}\\\|\_\{1\}\.Also, since both iterates remain inΔd∘\\Delta\_\{d\}^\{\\circ\},‖wT−wT′‖1≤2\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}\\leq 2\. Taking the supremum over all admissiblew0′=w0\+pw\_\{0\}^\{\\prime\}=w\_\{0\}\+pwith‖p‖1≤ε\\\|p\\\|\_\{1\}\\leq\\varepsilonproves the theorem\. ∎

## Appendix EProofs for Section[5](https://arxiv.org/html/2606.11431#S5)

To prove the lower bounds for[Algorithms1](https://arxiv.org/html/2606.11431#alg1)and[2](https://arxiv.org/html/2606.11431#alg2), We will use two standard facts about relative smoothness and relative strong convexity\. The first, adapted fromAttia and Koren \([2022](https://arxiv.org/html/2606.11431#bib.bib2)\), shows that adding a multiple of the regularizer makes the objective relatively strongly convex\.

###### Lemma 27\.

LetFFbe convex andLL\-smooth with respect to∥⋅∥\\\|\\cdot\\\|, and letRRbe11\-strongly convex with respect to∥⋅∥\\\|\\cdot\\\|\. Then, for everyμ\>0\\mu\>0, the function

Fμ​\(w\)=F​\(w\)\+μ​R​\(w\)F^\{\\mu\}\(w\)=F\(w\)\+\\mu R\(w\)is\(L\+μ\)\(L\+\\mu\)\-smooth andμ\\mu\-strongly convex relative toRR\.

The second, due toLuet al\.\([2018](https://arxiv.org/html/2606.11431#bib.bib3)\), gives the convergence guarantees of MD under relative smoothness and relative strong convexity\.

###### Lemma 28\.

LetΦ\\PhibeLL\-smooth andμ\\mu\-strongly convex relative toRR, and letw⋆∈arg⁡minw∈𝒲⁡Φ​\(w\)\.w^\{\\star\}\\in\\arg\\min\_\{w\\in\\mathcal\{W\}\}\\Phi\(w\)\.Then the trajectory\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}generated by MD withη=1/L\\eta=1/Lsatisfies, for allt≥1t\\geq 1,

DR​\(w⋆,wt\)≤\(1−μL\)t​DR​\(w⋆,w0\),D\_\{R\}\(w^\{\\star\},w\_\{t\}\)\\leq\\left\(1\-\\frac\{\\mu\}\{L\}\\right\)^\{t\}D\_\{R\}\(w^\{\\star\},w\_\{0\}\),and

Φ​\(wt\)−Φ​\(w⋆\)≤μ​DR​\(w⋆,w0\)\(1\+μL−μ\)t−1\.\\Phi\(w\_\{t\}\)\-\\Phi\(w^\{\\star\}\)\\leq\\frac\{\\mu D\_\{R\}\(w^\{\\star\},w\_\{0\}\)\}\{\\left\(1\+\\frac\{\\mu\}\{L\-\\mu\}\\right\)^\{t\}\-1\}\.

We now prove[Theorems4](https://arxiv.org/html/2606.11431#Thmtheorem4)and[6](https://arxiv.org/html/2606.11431#Thmtheorem6)\.

### E\.1Proof of Theorem[4](https://arxiv.org/html/2606.11431#Thmtheorem4)

Fix two initializationsw0,w0′∈𝒲w\_\{0\},w\_\{0\}^\{\\prime\}\\in\\mathcal\{W\}\. Let\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}be the trajectory of[Algorithm1](https://arxiv.org/html/2606.11431#alg1)initialized atw0w\_\{0\}, and let\{wt′\}t=0T\\\{w\_\{t\}^\{\\prime\}\\\}\_\{t=0\}^\{T\}be the trajectory initialized atw0′w\_\{0\}^\{\\prime\}\. Thus each trajectory is anchored at its own initialization:

Fw0μ​\(w\)=F​\(w\)\+μ​DR​\(w,w0\),Fw0′μ​\(w\)=F​\(w\)\+μ​DR​\(w,w0′\)\.F\_\{w\_\{0\}\}^\{\\mu\}\(w\)=F\(w\)\+\\mu D\_\{R\}\(w,w\_\{0\}\),\\qquad F\_\{\{w\_\{0\}\}^\{\\prime\}\}^\{\\mu\}\(w\)=F\(w\)\+\\mu D\_\{R\}\(w,\{w\_\{0\}\}^\{\\prime\}\)\.Since

∇Fw0μ​\(w\)=∇F​\(w\)\+μ​\(∇R​\(w\)−∇R​\(w0\)\),\\nabla F\_\{w\_\{0\}\}^\{\\mu\}\(w\)=\\nabla F\(w\)\+\\mu\\bigl\(\\nabla R\(w\)\-\\nabla R\(\{w\_\{0\}\}\)\\bigr\),[Algorithm1](https://arxiv.org/html/2606.11431#alg1)is MD applied to the regularized objectiveFw0μF^\{\\mu\}\_\{w\_\{0\}\}withη=1/\(L\+μ\)\\eta=1/\(L\+\\mu\)\. First we prove thatFw0μF^\{\\mu\}\_\{w\_\{0\}\}isμ\\mustrongly\-convex andL\+μL\+\\mu\-smooth relative toRR\. For the proof we use[AppendixE](https://arxiv.org/html/2606.11431#A5)\. Indeed,

Fw0μ​\(w\)=F​\(w\)\+μ​R​\(w\)−μ​R​\(w0\)−μ​⟨∇R​\(w0\),w−u⟩\.F^\{\\mu\}\_\{w\_\{0\}\}\(w\)=F\(w\)\+\\mu R\(w\)\-\\mu R\(\{w\_\{0\}\}\)\-\\mu\\langle\\nabla R\(\{w\_\{0\}\}\),w\-u\\rangle\.The last two terms are affine inww, and thus do not affect relative smoothness or relative strong convexity\. HenceFw0μF^\{\\mu\}\_\{w\_\{0\}\}is\(μ\+L\)\(\\mu\+L\)\-smooth andμ\\mu\-strongly convex relative toRR\. The same holds forw0′w\_\{0\}^\{\\prime\}\. We may therefore apply[AppendixE](https://arxiv.org/html/2606.11431#A5)to these two cases\. For each anchorv∈𝒲v\\in\\mathcal\{W\}, letwμ,v⋆∈arg⁡minw∈𝒲⁡Fvμ​\(w\)\.w^\{\\star\}\_\{\\mu,v\}\\in\\arg\\min\_\{w\\in\\mathcal\{W\}\}F^\{\\mu\}\_\{v\}\(w\)\.We first bound the movement of these regularized minimizers as the anchor changes\. By the first\-order optimality conditions forwμ,w0⋆w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}andwμ,w0′⋆w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}, applied with the other minimizer as a comparison point,

⟨∇F​\(wμ,w0⋆\)\+μ​\(∇R​\(wμ,w0⋆\)−∇R​\(w0\)\),wμ,w0′⋆−wμ,w0⋆⟩≥0,\\displaystyle\\left\\langle\\nabla F\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\)\+\\mu\\left\(\\nabla R\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\)\-\\nabla R\(\{w\_\{0\}\}\)\\right\),w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\-w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\\right\\rangle\\geq 0,⟨∇F​\(wμ,w0′⋆\)\+μ​\(∇R​\(wμ,w0′⋆\)−∇R​\(w0′\)\),wμ,w0⋆−wμ,w0′⋆⟩≥0\.\\displaystyle\\left\\langle\\nabla F\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\)\+\\mu\\left\(\\nabla R\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\)\-\\nabla R\(\{w\_\{0\}\}^\{\\prime\}\)\\right\),w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\-w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\\right\\rangle\\geq 0\.Adding the two inequalities and using the monotonicity of∇F\\nabla Fgives

⟨∇R​\(wμ,w0⋆\)−∇R​\(wμ,w0′⋆\),wμ,w0⋆−wμ,w0′⋆⟩≤⟨∇R​\(w0\)−∇R​\(w0′\),wμ,w0⋆−wμ,w0′⋆⟩\.\\left\\langle\\nabla R\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\)\-\\nabla R\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\),w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\-w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\\right\\rangle\\leq\\left\\langle\\nabla R\(\{w\_\{0\}\}\)\-\\nabla R\(\{w\_\{0\}\}^\{\\prime\}\),w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\-w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\\right\\rangle\.SinceRRis11\-strongly convex,

‖wμ,w0⋆−wμ,w0′⋆‖≤‖∇R​\(w0\)−∇R​\(w0′\)‖∗\.\\\|w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\-w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\\\|\\leq\\\|\\nabla R\(\{w\_\{0\}\}\)\-\\nabla R\(\{w\_\{0\}\}^\{\\prime\}\)\\\|\_\{\*\}\.
Next we bound the distance from each trajectory to its own regularized minimizer\. SinceFw0μ​\(wμ,w0⋆\)≤Fw0μ​\(w0\)=F​\(w0\)F^\{\\mu\}\_\{w\_\{0\}\}\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\)\\leq F^\{\\mu\}\_\{w\_\{0\}\}\(\{w\_\{0\}\}\)=F\(\{w\_\{0\}\}\), we have

μ​DR​\(wμ,w0⋆,w0\)≤F​\(w0\)−F​\(wμ,w0⋆\)≤G​D\.\\mu D\_\{R\}\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\},\{w\_\{0\}\}\)\\leq F\(\{w\_\{0\}\}\)\-F\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\)\\leq GD\.By[AppendixE](https://arxiv.org/html/2606.11431#A5),

DR​\(wμ,w0⋆,wT\)≤\(1\+μL\)−T​DR​\(wμ,w0⋆,w0\)≤\(1\+μL\)−T​G​Dμ\.D\_\{R\}\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\},w\_\{T\}\)\\leq\\left\(1\+\\frac\{\\mu\}\{L\}\\right\)^\{\-T\}D\_\{R\}\(w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\},\{w\_\{0\}\}\)\\leq\\left\(1\+\\frac\{\\mu\}\{L\}\\right\)^\{\-T\}\\frac\{GD\}\{\\mu\}\.The same bound holds for the trajectory initialized atu′u^\{\\prime\}\. Using strong convexity ofRRand the triangle inequality,

‖wT−wT′‖\\displaystyle\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|≤‖wT−wμ,w0⋆‖\+‖wμ,w0⋆−wμ,w0′⋆‖\+‖wμ,w0′⋆−wT′‖\\displaystyle\\leq\\\|w\_\{T\}\-w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\\\|\+\\\|w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}\}\-w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\\\|\+\\\|w^\{\\star\}\_\{\\mu,\{w\_\{0\}\}^\{\\prime\}\}\-w\_\{T\}^\{\\prime\}\\\|≤‖∇R​\(w0\)−∇R​\(w0′\)‖∗\+2​2​\(1\+μL\)−T​G​Dμ\.\\displaystyle\\leq\\\|\\nabla R\(\{w\_\{0\}\}\)\-\\nabla R\(\{w\_\{0\}\}^\{\\prime\}\)\\\|\_\{\*\}\+2\\sqrt\{2\\left\(1\+\\frac\{\\mu\}\{L\}\\right\)^\{\-T\}\\frac\{GD\}\{\\mu\}\}\.Withμ=8​L​log⁡T/T\\mu=8L\\log T/T, the elementary bound

\(1\+8​log⁡TT\)T≥T2\(T≥2\)\\left\(1\+\\frac\{8\\log T\}\{T\}\\right\)^\{T\}\\geq T^\{2\}\\qquad\(T\\geq 2\)implies

‖wT−wT′‖≤‖∇R​\(w0\)−∇R​\(w0′\)‖∗\+G​DL​T​log⁡T\.\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\\leq\\\|\\nabla R\(\{w\_\{0\}\}\)\-\\nabla R\(\{w\_\{0\}\}^\{\\prime\}\)\\\|\_\{\*\}\+\\sqrt\{\\frac\{GD\}\{LT\\log T\}\}\.Lettingw0′=w0\+pw\_\{0\}^\{\\prime\}=w\_\{0\}\+pand taking the supremum over all admissible perturbationsppproves the first stability bound\. IfRRisβ\\beta\-smooth, then

‖∇R​\(w0\)−∇R​\(w0\+p\)‖∗≤β​‖p‖≤β​ε,\\\|\\nabla R\(w\_\{0\}\)\-\\nabla R\(w\_\{0\}\+p\)\\\|\_\{\*\}\\leq\\beta\\\|p\\\|\\leq\\beta\\varepsilon,which gives the stated smooth\-regularizer form\.

It remains to prove the optimization guarantee for the trajectory initialized atw0w\_\{0\}\. Letwμ,w0⋆∈arg⁡minw∈𝒲⁡Fw0μ​\(w\)\.w^\{\\star\}\_\{\\mu,w\_\{0\}\}\\in\\arg\\min\_\{w\\in\\mathcal\{W\}\}F^\{\\mu\}\_\{w\_\{0\}\}\(w\)\.

F​\(wT\)−F​\(w⋆\)\\displaystyle F\(w\_\{T\}\)\-F\(w^\{\\star\}\)=Fw0μ​\(wT\)−Fw0μ​\(w⋆\)\+μ​\(DR​\(w⋆,w0\)−DR​\(wT,w0\)\)\\displaystyle=F^\{\\mu\}\_\{w\_\{0\}\}\(w\_\{T\}\)\-F^\{\\mu\}\_\{w\_\{0\}\}\(w^\{\\star\}\)\+\\mu\(D\_\{R\}\(w^\{\\star\},w\_\{0\}\)\-D\_\{R\}\(w\_\{T\},w\_\{0\}\)\)≤Fw0μ​\(wT\)−Fw0μ​\(wμ,w0⋆\)\+μ​DR​\(w⋆,w0\)\.\\displaystyle\\leq F^\{\\mu\}\_\{w\_\{0\}\}\(w\_\{T\}\)\-F^\{\\mu\}\_\{w\_\{0\}\}\(w^\{\\star\}\_\{\\mu,w\_\{0\}\}\)\+\\mu D\_\{R\}\(w^\{\\star\},w\_\{0\}\)\.From[AppendixE](https://arxiv.org/html/2606.11431#A5),

Fw0μ​\(wT\)−Fw0μ​\(wμ,w0⋆\)≤μ​DR​\(wμ,w0⋆,w0\)\(1\+μL\)T−1≤μ​DR​\(wμ,w0⋆,w0\)T2−1\.\\displaystyle F^\{\\mu\}\_\{w\_\{0\}\}\(w\_\{T\}\)\-F^\{\\mu\}\_\{w\_\{0\}\}\(w^\{\\star\}\_\{\\mu,w\_\{0\}\}\)\\leq\\frac\{\\mu D\_\{R\}\(w^\{\\star\}\_\{\\mu,w\_\{0\}\},w\_\{0\}\)\}\{\\left\(1\+\\frac\{\\mu\}\{L\}\\right\)^\{T\}\-1\}\\leq\\frac\{\\mu D\_\{R\}\(w^\{\\star\}\_\{\\mu,w\_\{0\}\},w\_\{0\}\)\}\{T^\{2\}\-1\}\.The last inequality follows from the choice ofμ\\muand\(1\+8​log⁡TT\)T≥T2\\left\(1\+\\frac\{8\\log T\}\{T\}\\right\)^\{T\}\\geq T^\{2\}\. Overall,

F​\(wT\)−F​\(w⋆\)\\displaystyle F\(w\_\{T\}\)\-F\(w^\{\\star\}\)≤μ​DR​\(wμ,w0⋆,w0\)T2−1\+μ​DR​\(w⋆,w0\)\.\\displaystyle\\leq\\frac\{\\mu D\_\{R\}\(w^\{\\star\}\_\{\\mu,w\_\{0\}\},w\_\{0\}\)\}\{T^\{2\}\-1\}\+\\mu D\_\{R\}\(w^\{\\star\},w\_\{0\}\)\.Finally, as above,Fw0μ​\(wμ,w0⋆\)≤Fw0μ​\(w0\)=F​\(w0\)F^\{\\mu\}\_\{w\_\{0\}\}\(w^\{\\star\}\_\{\\mu,w\_\{0\}\}\)\\leq F^\{\\mu\}\_\{w\_\{0\}\}\(w\_\{0\}\)=F\(w\_\{0\}\), so

μ​DR​\(wμ,w0⋆,w0\)≤F​\(w0\)−F​\(wμ,w0⋆\)≤G​D\.\\mu D\_\{R\}\(w^\{\\star\}\_\{\\mu,w\_\{0\}\},w\_\{0\}\)\\leq F\(w\_\{0\}\)\-F\(w^\{\\star\}\_\{\\mu,w\_\{0\}\}\)\\leq GD\.Substituting this andμ=8​L​log⁡T/T\\mu=8L\\log T/T, we obtain

F​\(wT\)−F​\(w⋆\)≤G​DT2−1\+8​L​log⁡TT​DR​\(w⋆,w0\)\.F\(w\_\{T\}\)\-F\(w^\{\\star\}\)\\leq\\frac\{GD\}\{T^\{2\}\-1\}\+\\frac\{8L\\log T\}\{T\}D\_\{R\}\(w^\{\\star\},w\_\{0\}\)\.SinceT≥2T\\geq 2, we have\(T2−1\)−1≤2/T2\(T^\{2\}\-1\)^\{\-1\}\\leq 2/T^\{2\}, and therefore

F​\(wT\)−F​\(w⋆\)≤8​L​log⁡TT​DR​\(w⋆,w0\)\+2​G​DT2\.F\(w\_\{T\}\)\-F\(w^\{\\star\}\)\\leq\\frac\{8L\\log T\}\{T\}D\_\{R\}\(w^\{\\star\},w\_\{0\}\)\+\\frac\{2GD\}\{T^\{2\}\}\.

### E\.2Proof of Theorem[5](https://arxiv.org/html/2606.11431#Thmtheorem5)

Let

im∈arg⁡mini∈\[d\]⁡w0​\(i\),w0​\(im\)=w0min\.i\_\{m\}\\in\\arg\\min\_\{i\\in\[d\]\}w\_\{0\}\(i\),\\qquad w\_\{0\}\(i\_\{m\}\)=w\_\{0\}^\{\\min\}\.Sinced≥2d\\geq 2, we havew0min≤1/2w\_\{0\}^\{\\min\}\\leq 1/2, and therefore

1−w0min≥12\>ε2\.1\-w\_\{0\}^\{\\min\}\\geq\\frac\{1\}\{2\}\>\\frac\{\\varepsilon\}\{2\}\.Definep∈ℝdp\\in\\mathbb\{R\}^\{d\}by

p​\(im\)=ε2,p​\(i\)=−ε2⋅w0​\(i\)1−w0minfor​i≠im\.p\(i\_\{m\}\)=\\frac\{\\varepsilon\}\{2\},\\qquad p\(i\)=\-\\frac\{\\varepsilon\}\{2\}\\cdot\\frac\{w\_\{0\}\(i\)\}\{1\-w\_\{0\}^\{\\min\}\}\\quad\\text\{for \}i\\neq i\_\{m\}\.Then∑i=1dp​\(i\)=0\\sum\_\{i=1\}^\{d\}p\(i\)=0\. Moreover, sinceε2<1−w0min\\frac\{\\varepsilon\}\{2\}<1\-w\_\{0\}^\{\\min\}, for everyi≠imi\\neq i\_\{m\},

w0​\(i\)\+p​\(i\)=w0​\(i\)​\(1−ε2⋅11−w0min\)\>0\.w\_\{0\}\(i\)\+p\(i\)=w\_\{0\}\(i\)\\left\(1\-\\frac\{\\varepsilon\}\{2\}\\cdot\\frac\{1\}\{1\-w\_\{0\}^\{\\min\}\}\\right\)\>0\.Also,

w0​\(im\)\+p​\(im\)=w0min\+ε2\>0\.w\_\{0\}\(i\_\{m\}\)\+p\(i\_\{m\}\)=w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\>0\.Therefore

w0′:=w0\+p∈Δd∘\.w\_\{0\}^\{\\prime\}:=w\_\{0\}\+p\\in\\Delta\_\{d\}^\{\\circ\}\.Furthermore,

‖p‖1=ε2\+∑i≠imε2⋅w0​\(i\)1−w0min=ε\.\\\|p\\\|\_\{1\}=\\frac\{\\varepsilon\}\{2\}\+\\sum\_\{i\\neq i\_\{m\}\}\\frac\{\\varepsilon\}\{2\}\\cdot\\frac\{w\_\{0\}\(i\)\}\{1\-w\_\{0\}^\{\\min\}\}=\\varepsilon\.
Now define

K:=min⁡\{exp⁡\(1μ​\[1−\(LL\+μ\)T\]\),1w0min,1ε\}\.K:=\\min\\left\\\{\\exp\\left\(\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\\right\),\\frac\{1\}\{w\_\{0\}^\{\\min\}\},\\frac\{1\}\{\\varepsilon\}\\right\\\}\.LetF​\(w\)=⟨g,w⟩F\(w\)=\\langle g,w\\rangle, where

g​\(i\)=\{−log⁡K1μ​\[1−\(LL\+μ\)T\],i=im,0,i≠im\.g\(i\)=\\begin\{cases\}\-\\dfrac\{\\log K\}\{\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\},&i=i\_\{m\},\\\\\[11\.99998pt\] 0,&i\\neq i\_\{m\}\.\\end\{cases\}By the definition ofKK,

0≤log⁡K1μ​\[1−\(LL\+μ\)T\]≤1\.0\\leq\\frac\{\\log K\}\{\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\}\\leq 1\.Hence‖g‖∞≤1\\\|g\\\|\_\{\\infty\}\\leq 1, soFFis11\-Lipschitz with respect to∥⋅∥1\\\|\\cdot\\\|\_\{1\}\.

We now compute the dynamics of[Algorithm1](https://arxiv.org/html/2606.11431#alg1)for this linear objective\. Fix an initializationw0∈Δd∘w\_\{0\}\\in\\Delta\_\{d\}^\{\\circ\}, and let\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}be the trajectory of the algorithm initialized atw0w\_\{0\}, from symmetry the same proof will follow forw0′w\_\{0\}^\{\\prime\}\. SinceFFis linear,∇F​\(wt\)=g\\nabla F\(w\_\{t\}\)=gfor everytt, and the update is

wt\+1=arg⁡minw∈Δd⁡\{⟨g\+μ​\(∇R​\(wt\)−∇R​\(w0\)\),w−wt⟩\+\(μ\+L\)​DR​\(w,wt\)\}\.w\_\{t\+1\}=\\arg\\min\_\{w\\in\\Delta\_\{d\}\}\\left\\\{\\left\\langle g\+\\mu\(\\nabla R\(w\_\{t\}\)\-\\nabla R\(w\_\{0\}\)\),w\-w\_\{t\}\\right\\rangle\+\(\\mu\+L\)D\_\{R\}\(w,w\_\{t\}\)\\right\\\}\.We first show that the iterates remain in the interior of the simplex\. We prove by induction ontt\. For the base case indeedw0∈Δd∘w\_\{0\}\\in\\Delta\_\{d\}^\{\\circ\}, now suppose thatwt∈Δd∘w\_\{t\}\\in\\Delta\_\{d\}^\{\\circ\}, and consider the objective minimized in the update\. Letw¯∈Δd\\bar\{w\}\\in\\Delta\_\{d\}be a minimizer\. We claim thatw¯∈Δd∘\\bar\{w\}\\in\\Delta\_\{d\}^\{\\circ\}\. Suppose toward contradiction thatw¯​\(i\)=0\\bar\{w\}\(i\)=0for some coordinateii\. Sincew¯∈Δd\\bar\{w\}\\in\\Delta\_\{d\}, there exists a coordinatejjsuch thatw¯​\(j\)\>0\\bar\{w\}\(j\)\>0\. For sufficiently smallα\>0\\alpha\>0, the point

w¯α=w¯\+α​ei−α​ej\\bar\{w\}^\{\\alpha\}=\\bar\{w\}\+\\alpha e\_\{i\}\-\\alpha e\_\{j\}belongs toΔd\\Delta\_\{d\}\. The directional derivative of the Bregman term in this direction is

⟨∇DR​\(w,wt\),ei−ej⟩=log⁡w​\(i\)wt​\(i\)−log⁡w​\(j\)wt​\(j\)\.\\left\\langle\\nabla D\_\{R\}\(w,w\_\{t\}\),e\_\{i\}\-e\_\{j\}\\right\\rangle=\\log\\frac\{w\(i\)\}\{w\_\{t\}\(i\)\}\-\\log\\frac\{w\(j\)\}\{w\_\{t\}\(j\)\}\.Asw​\(i\)→0w\(i\)\\to 0, this quantity tends to−∞\-\\infty, whereas the directional derivative of the linear part of the update,⟨g,ei−ej⟩\\left\\langle g,e\_\{i\}\-e\_\{j\}\\right\\rangleis finite\. Hence, for all sufficiently smallα\>0\\alpha\>0, moving fromw¯\\bar\{w\}tow¯α\\bar\{w\}^\{\\alpha\}strictly decreases the objective, contradicting the optimality ofw¯\\bar\{w\}\. Therefore the minimizer cannot place zero mass on any coordinate, and sowt\+1∈Δd∘w\_\{t\+1\}\\in\\Delta\_\{d\}^\{\\circ\}\. This proves the claim\.

Hence from KKT conditions, there exists a scalarλt\\lambda\_\{t\}such that, for every coordinateii,

g​\(i\)\+μ​\(∇R​\(wt\)​\(i\)−∇R​\(w0\)​\(i\)\)\+\(μ\+L\)​\(∇R​\(wt\+1\)​\(i\)−∇R​\(wt\)​\(i\)\)\+λt=0\.g\(i\)\+\\mu\\bigl\(\\nabla R\(w\_\{t\}\)\(i\)\-\\nabla R\(w\_\{0\}\)\(i\)\\bigr\)\+\(\\mu\+L\)\\bigl\(\\nabla R\(w\_\{t\+1\}\)\(i\)\-\\nabla R\(w\_\{t\}\)\(i\)\\bigr\)\+\\lambda\_\{t\}=0\.Subtracting the conditions for coordinatesiiandjj, the scalarλt\\lambda\_\{t\}cancels\. Since for negative entropy

∇R​\(w\)​\(i\)−∇R​\(w\)​\(j\)=log⁡w​\(i\)w​\(j\),\\nabla R\(w\)\(i\)\-\\nabla R\(w\)\(j\)=\\log\\frac\{w\(i\)\}\{w\(j\)\},we obtain

log⁡wt\+1​\(i\)wt\+1​\(j\)=LL\+μ​log⁡wt​\(i\)wt​\(j\)\+μL\+μ​log⁡w0​\(i\)w0​\(j\)−1L\+μ​\(g​\(i\)−g​\(j\)\)\.\\log\\frac\{w\_\{t\+1\}\(i\)\}\{w\_\{t\+1\}\(j\)\}=\\frac\{L\}\{L\+\\mu\}\\log\\frac\{w\_\{t\}\(i\)\}\{w\_\{t\}\(j\)\}\+\\frac\{\\mu\}\{L\+\\mu\}\\log\\frac\{w\_\{0\}\(i\)\}\{w\_\{0\}\(j\)\}\-\\frac\{1\}\{L\+\\mu\}\\bigl\(g\(i\)\-g\(j\)\\bigr\)\.Unfolding the recursion gives

log⁡wT​\(i\)wT​\(j\)=log⁡w0​\(i\)w0​\(j\)−1μ​\[1−\(LL\+μ\)T\]​\(g​\(i\)−g​\(j\)\)\.\\log\\frac\{w\_\{T\}\(i\)\}\{w\_\{T\}\(j\)\}=\\log\\frac\{w\_\{0\}\(i\)\}\{w\_\{0\}\(j\)\}\-\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\\bigl\(g\(i\)\-g\(j\)\\bigr\)\.Equivalently,

wT​\(i\)=w0​\(i\)​exp⁡\(−1μ​\[1−\(LL\+μ\)T\]​g​\(i\)\)∑j=1dw0​\(j\)​exp⁡\(−1μ​\[1−\(LL\+μ\)T\]​g​\(j\)\)\.w\_\{T\}\(i\)=\\frac\{w\_\{0\}\(i\)\\exp\\left\(\-\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]g\(i\)\\right\)\}\{\\sum\_\{j=1\}^\{d\}w\_\{0\}\(j\)\\exp\\left\(\-\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]g\(j\)\\right\)\}\.
Applying this formula first withw0w\_\{0\}, and then withw0′w\_\{0\}^\{\\prime\}, we get

wT​\(im\)=K​w0min1\+\(K−1\)​w0min,w\_\{T\}\(i\_\{m\}\)=\\frac\{Kw\_\{0\}^\{\\min\}\}\{1\+\(K\-1\)w\_\{0\}^\{\\min\}\},and

wT′​\(im\)=K​\(w0min\+ε2\)1\+\(K−1\)​\(w0min\+ε2\)\.w\_\{T\}^\{\\prime\}\(i\_\{m\}\)=\\frac\{K\(w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\)\}\{1\+\(K\-1\)\(w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\)\}\.Hence

‖wT−wT′‖1≥2​\(wT′​\(im\)−wT​\(im\)\)\.\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}\\geq 2\\left\(w\_\{T\}^\{\\prime\}\(i\_\{m\}\)\-w\_\{T\}\(i\_\{m\}\)\\right\)\.By direct computation,

2​\(wT′​\(im\)−wT​\(im\)\)\\displaystyle 2\\left\(w\_\{T\}^\{\\prime\}\(i\_\{m\}\)\-w\_\{T\}\(i\_\{m\}\)\\right\)=2​\(K​\(w0min\+ε2\)1\+\(K−1\)​\(w0min\+ε2\)−K​w0min1\+\(K−1\)​w0min\)\\displaystyle=2\\left\(\\frac\{K\(w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\)\}\{1\+\(K\-1\)\(w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\)\}\-\\frac\{Kw\_\{0\}^\{\\min\}\}\{1\+\(K\-1\)w\_\{0\}^\{\\min\}\}\\right\)=K​ε\(1\+\(K−1\)​\(w0min\+ε2\)\)​\(1\+\(K−1\)​w0min\)\.\\displaystyle=\\frac\{K\\varepsilon\}\{\\left\(1\+\(K\-1\)\(w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\)\\right\)\\left\(1\+\(K\-1\)w\_\{0\}^\{\\min\}\\right\)\}\.We now bound the denominator\. SinceK≤1/w0minK\\leq 1/w\_\{0\}^\{\\min\},

1\+\(K−1\)​w0min≤1\+K​w0min≤2\.1\+\(K\-1\)w\_\{0\}^\{\\min\}\\leq 1\+Kw\_\{0\}^\{\\min\}\\leq 2\.Also, sinceK≤1/w0minK\\leq 1/w\_\{0\}^\{\\min\}andK≤1/εK\\leq 1/\\varepsilon,

1\+\(K−1\)​\(w0min\+ε2\)≤1\+K​w0min\+K​ε2≤1\+1\+12=52\.1\+\(K\-1\)\(w\_\{0\}^\{\\min\}\+\\frac\{\\varepsilon\}\{2\}\)\\leq 1\+Kw\_\{0\}^\{\\min\}\+\\frac\{K\\varepsilon\}\{2\}\\leq 1\+1\+\\frac\{1\}\{2\}=\\frac\{5\}\{2\}\.Thus

‖wT−wT′‖1≥K​ε5\.\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}\\geq\\frac\{K\\varepsilon\}\{5\}\.
By the definition ofKK,

K​ε=min⁡\{ε​exp⁡\(1μ​\[1−\(LL\+μ\)T\]\),εw0min,1\}\.K\\varepsilon=\\min\\left\\\{\\varepsilon\\exp\\left\(\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\\right\),\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\},1\\right\\\}\.Therefore

δA​\(w0,ε\)≥15​min⁡\{1,ε​exp⁡\(1μ​\[1−\(LL\+μ\)T\]\),εw0min\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\frac\{1\}\{5\}\\min\\left\\\{1,\\,\\varepsilon\\exp\\left\(\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\\right\),\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}\.
It remains to prove the simpler lower bound\. Since

\(LL\+μ\)T=\(1−μL\+μ\)T≤exp⁡\(−μ​TL\+μ\),\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}=\\left\(1\-\\frac\{\\mu\}\{L\+\\mu\}\\right\)^\{T\}\\leq\\exp\\left\(\-\\frac\{\\mu T\}\{L\+\\mu\}\\right\),we have

1μ​\[1−\(LL\+μ\)T\]≥1μ​\[1−exp⁡\(−μ​TL\+μ\)\]\.\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\\geq\\frac\{1\}\{\\mu\}\\left\[1\-\\exp\\left\(\-\\frac\{\\mu T\}\{L\+\\mu\}\\right\)\\right\]\.Using the elementary inequality

1−e−x≥\(1−e−1\)​min⁡\{x,1\},x≥0,1\-e^\{\-x\}\\geq\(1\-e^\{\-1\}\)\\min\\\{x,1\\\},\\qquad x\\geq 0,withx=μ​TL\+μx=\\frac\{\\mu T\}\{L\+\\mu\}, we get

1μ​\[1−\(LL\+μ\)T\]≥1μ​\[1−exp⁡\(−μ​TL\+μ\)\]≥\(1−e−1\)​min⁡\{TL\+μ,1μ\}\.\\frac\{1\}\{\\mu\}\\left\[1\-\\left\(\\frac\{L\}\{L\+\\mu\}\\right\)^\{T\}\\right\]\\geq\\frac\{1\}\{\\mu\}\\left\[1\-\\exp\\left\(\-\\frac\{\\mu T\}\{L\+\\mu\}\\right\)\\right\]\\geq\(1\-e^\{\-1\}\)\\min\\left\\\{\\frac\{T\}\{L\+\\mu\},\\frac\{1\}\{\\mu\}\\right\\\}\.Plugging this into the exact lower bound gives

δA​\(w0,ε\)≥15​min⁡\{1,ε​exp⁡\(\(1−e−1\)​min⁡\{TL\+μ,1μ\}\),εw0min\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\frac\{1\}\{5\}\\min\\left\\\{1,\\,\\varepsilon\\exp\\left\(\(1\-e^\{\-1\}\)\\min\\left\\\{\\frac\{T\}\{L\+\\mu\},\\frac\{1\}\{\\mu\}\\right\\\}\\right\),\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}\.If additionallyd≥1/εd\\geq 1/\\varepsilonthenw0min≤1/dw\_\{0\}^\{\\min\}\\leq 1/dhenceε/w0min≥1\\varepsilon/w\_\{0\}^\{\\min\}\\geq 1and the bound becomes,

δA​\(w0,ε\)≥15​min⁡\{1,ε​exp⁡\(\(1−e−1\)​min⁡\{TL\+μ,1μ\}\)\}\.\\delta\_\{A\}\(w\_\{0\},\\varepsilon\)\\geq\\frac\{1\}\{5\}\\min\\left\\\{1,\\,\\varepsilon\\exp\\left\(\(1\-e^\{\-1\}\)\\min\\left\\\{\\frac\{T\}\{L\+\\mu\},\\frac\{1\}\{\\mu\}\\right\\\}\\right\)\\right\\\}\.Finally, using the same arguments as in the proof of[Theorem2](https://arxiv.org/html/2606.11431#Thmtheorem2), we havemin⁡\{1,εw0min\}≥12​min⁡\{1,ε​β​\(w0,ε\)\},\\min\\left\\\{1,\\frac\{\\varepsilon\}\{w\_\{0\}^\{\\min\}\}\\right\\\}\\geq\\frac\{1\}\{2\}\\min\\\{1,\\varepsilon\\beta\(w\_\{0\},\\varepsilon\)\\\},which concludes the proof\.

### E\.3Proof of Theorem[6](https://arxiv.org/html/2606.11431#Thmtheorem6)

Let\{wt\}t=0T\\\{w\_\{t\}\\\}\_\{t=0\}^\{T\}be the trajectory of[Algorithm2](https://arxiv.org/html/2606.11431#alg2)initialized atw0w\_\{0\}, and let\{wt′\}t=0T\\\{w\_\{t\}^\{\\prime\}\\\}\_\{t=0\}^\{T\}be the trajectory initialized atw0′w\_\{0\}^\{\\prime\}\. DenoteFμ​\(w\)=F​\(w\)\+μ​DR​\(w,wa\)F^\{\\mu\}\(w\)=F\(w\)\+\\mu D\_\{R\}\(w,w\_\{a\}\)\. It is easy to see that[Algorithm2](https://arxiv.org/html/2606.11431#alg2)simply runs MD updates onFμF^\{\\mu\}\. As a corollary of[AppendixE](https://arxiv.org/html/2606.11431#A5), since Bregman divergence is unaffected by the addition of affine functions,FμF^\{\\mu\}is\(μ\+L\)\(\\mu\+L\)\-smooth andμ\\mu\-strongly convex relative toRR\. We may therefore apply[AppendixE](https://arxiv.org/html/2606.11431#A5)\. We will start with initialization stability\. The following holds:

‖wT−wT′‖2\\displaystyle\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|^\{2\}≤2​‖wT−wμ⋆‖2\+2​‖wT′−wμ⋆‖2\\displaystyle\\leq 2\\\|w\_\{T\}\-w^\{\\star\}\_\{\\mu\}\\\|^\{2\}\+2\\\|w\_\{T\}^\{\\prime\}\-w^\{\\star\}\_\{\\mu\}\\\|^\{2\}≤4​DR​\(wμ⋆,wT\)\+4​DR​\(wμ⋆,wT′\)\\displaystyle\\leq 4D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{T\}\)\+4D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{T\}^\{\\prime\}\)\(RRis11\-strongly convex\)≤4​\(1−μμ\+L\)T​\(DR​\(wμ⋆,w0\)\+DR​\(wμ⋆,w0′\)\)\\displaystyle\\leq 4\\left\(1\-\\frac\{\\mu\}\{\\mu\+L\}\\right\)^\{T\}\\left\(D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\+D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}^\{\\prime\}\)\\right\)\([AppendixE](https://arxiv.org/html/2606.11431#A5)\)≤4​\(1\+μL\)−T⋅\(DR​\(wμ⋆,w0\)\+DR​\(wμ⋆,w0′\)\)\.\\displaystyle\\leq 4\\left\(1\+\\frac\{\\mu\}\{L\}\\right\)^\{\-T\}\\cdot\\left\(D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\+D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}^\{\\prime\}\)\\right\)\.Plugging inμ=8​L​log⁡TT\\mu=\\frac\{8L\\log T\}\{T\}, using\(1\+8​log⁡TT\)T≥T2\(1\+\\frac\{8\\log T\}\{T\}\)^\{T\}\\geq T^\{2\}forT≥2T\\geq 2, we have

‖wT−wT′‖12\\displaystyle\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{1\}^\{2\}≤4​\(DR​\(wμ⋆,w0\)\+DR​\(wμ⋆,w0′\)\)​\(1\+8​log⁡TT\)−T\\displaystyle\\leq 4\\left\(D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\+D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}^\{\\prime\}\)\\right\)\\left\(1\+\\frac\{8\\log T\}\{T\}\\right\)^\{\-T\}≤4T2​\(DR​\(wμ⋆,w0\)\+DR​\(wμ⋆,w0′\)\)\\displaystyle\\leq\\frac\{4\}\{T^\{2\}\}\\left\(D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\+D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}^\{\\prime\}\)\\right\)Regarding optimization we have the following,

F​\(wT\)−F​\(w⋆\)\\displaystyle F\(w\_\{T\}\)\-F\(w^\{\\star\}\)=Fμ​\(wT\)−Fμ​\(w⋆\)\+μ​\(DR​\(w⋆,wa\)−DR​\(wT,wa\)\)\\displaystyle=F^\{\\mu\}\(w\_\{T\}\)\-F^\{\\mu\}\(w^\{\\star\}\)\+\\mu\(D\_\{R\}\(w^\{\\star\},w\_\{a\}\)\-D\_\{R\}\(w\_\{T\},w\_\{a\}\)\)≤Fμ​\(wT\)−Fμ​\(wμ⋆\)\+μ​DR​\(w⋆,wa\)\\displaystyle\\leq F^\{\\mu\}\(w\_\{T\}\)\-F^\{\\mu\}\(w^\{\\star\}\_\{\\mu\}\)\+\\mu D\_\{R\}\(w^\{\\star\},w\_\{a\}\)From[AppendixE](https://arxiv.org/html/2606.11431#A5)and the choice ofμ\\mu

F​\(wT\)−F​\(w⋆\)\\displaystyle F\(w\_\{T\}\)\-F\(w^\{\\star\}\)≤μ​DR​\(wμ⋆,w0\)\(1\+μL\)T−1\+μ​DR​\(w⋆,wa\)\\displaystyle\\leq\\frac\{\\mu D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\}\{\\left\(1\+\\frac\{\\mu\}\{L\}\\right\)^\{T\}\-1\}\+\\mu D\_\{R\}\(w^\{\\star\},w\_\{a\}\)≤8​L​log⁡T⋅DR​\(wμ⋆,w0\)T​\(T2−1\)\+8​L​log⁡T​DR​\(w⋆,wa\)T\.\\displaystyle\\leq\\frac\{8L\\log T\\cdot D\_\{R\}\(w^\{\\star\}\_\{\\mu\},w\_\{0\}\)\}\{T\(T^\{2\}\-1\)\}\+\\frac\{8L\\log TD\_\{R\}\(w^\{\\star\},w\_\{a\}\)\}\{T\}\.which concludes the proof\.