Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning

# Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning
Source: [https://arxiv.org/html/2608.20668](https://arxiv.org/html/2608.20668)
Qifa YanEmail:[qifayan@swjtu\.edu\.cn](mailto:)Corresponding author:Corresponding authorBin DaiEmail:[daibin@swjtu\.edu\.cn](mailto:)Xiaohu TangEmail:[xhutang@swjtu\.edu\.cn](mailto:)Address:School of Information Science and Technology, Southwest Jiaotong University, Chengdu 610031, China and Information Coding and Transmission Key Laboratory of Sichuan Province, CSNMT Int\. Coop\. Res\. Centre \(MoST\), Southwest Jiaotong University, Chengdu 611756, China

###### Abstract

In recent years, a white\-box neural network called ReduNet has been proposed, which employs the maximal coding rate reduction \(MCR2\) principle to transform raw data into low\-dimensional discriminative features via a forward layer\-wise construction process\. Unlike traditional deep networks that rely on backpropagation, ReduNet explicitly derives the parameters of each layer from the features of its preceding layer, offering a mathematically interpretable paradigm\. However, this layer\-wise construction often requires a large number of layers for the MCR2objective to reach a stable value, which increases the parameter storage of the unfolded module\. To address this issue, we propose LA\-ReduNet, a lightweight adaptive architecture that refines the layer\-wise update rule and enables discriminative feature representations to be obtained with substantially fewer unfolded layers\. Specifically, LA\-ReduNet employs hyperspherical manifold learning and adaptive step sizes, thereby reducing by an order of magnitude the number of layers required for the MCR2objective to reach a stable value\. Simulation results demonstrate that, while maintaining comparable classification accuracy, LA\-ReduNet requires significantly fewer layers for the MCR2objective to reach a stable value\. Remarkably, under the considered experimental settings, LA\-ReduNet requires only approximately1/291/29of the parameter storage of the unfolded ReduNet module for the MCR2objective to reach a stable value\.

###### Keywords:

Maximal Coding Rate Reduction \(MCR2\), white\-box neural network, manifold learning, feature extraction

## 1Introduction

The rapid advancement of artificial intelligence \(AI\) promotes research on applying AI to other fields\[[9](https://arxiv.org/html/2608.20668#bib.bib8)\]\. A major milestone in this development came in 2016, when the AI program AlphaGo\[[20](https://arxiv.org/html/2608.20668#bib.bib9),[19](https://arxiv.org/html/2608.20668#bib.bib10)\]defeated Lee Sedol, the world champion of Go\. Currently, deep learning\-based algorithms are applied to a wide range of fields, such as autonomous driving\[[24](https://arxiv.org/html/2608.20668#bib.bib11)\], disease detection\[[16](https://arxiv.org/html/2608.20668#bib.bib12)\], image classification\[[3](https://arxiv.org/html/2608.20668#bib.bib15)\], face recognition\[[2](https://arxiv.org/html/2608.20668#bib.bib13)\]and semantic communication\[[7](https://arxiv.org/html/2608.20668#bib.bib14)\]\. However, in certain scenarios, such as autonomous driving, higher interpretability is required for the model\. Therefore, research on Explainable Artificial Intelligence \(XAI\) has attracted widespread attention and developed rapidly\[[9](https://arxiv.org/html/2608.20668#bib.bib8)\]\. Although research on black\-box neural networks has made remarkable progress\[[17](https://arxiv.org/html/2608.20668#bib.bib21),[18](https://arxiv.org/html/2608.20668#bib.bib23),[8](https://arxiv.org/html/2608.20668#bib.bib22)\], their interpretability remains limited\.

In this context, Chan et al\. proposed a novel white\-box neural network named ReduNet\[[5](https://arxiv.org/html/2608.20668#bib.bib3)\], providing a perspective distinct from conventional approaches\. In contrast to conventional methods that directly interpret the internal mechanisms of black\-box neural networks, ReduNet derives the network architecture from the optimization objective, thereby reducing reliance on manual empirical design of the network architecture\. More specifically, the network architecture of ReduNet is derived from the principle of Maximal Coding Rate Reduction \(MCR2\)\. Different from the traditional cross\-entropy \(CE\) loss function, the principle of MCR2revises the learning objective to explicitly capture the low\-dimensional structures underlying high\-dimensional data, rather than primarily focusing on label fitting\[[22](https://arxiv.org/html/2608.20668#bib.bib1)\]\. The MCR2principle utilizes the rate distortion to measure the compactness of representations\. The MCR2principle aims to maximize the overall coding rate of features while minimizing the within\-class coding rate, where the coding rate is computed via multivariate Gaussian rate\-distortion \(RD\) function\.

Existing studies on ReduNet can be broadly divided into two categories\. The first category focuses on improving ReduNet, such as ESS\-ReduNet\[[21](https://arxiv.org/html/2608.20668#bib.bib17)\], Multi\-ReduNet\[[12](https://arxiv.org/html/2608.20668#bib.bib16)\]and AR\-ReduNet\[[10](https://arxiv.org/html/2608.20668#bib.bib4)\]\. The second direction is to apply ReduNet to other domains, such as SAR target recognition\[[25](https://arxiv.org/html/2608.20668#bib.bib18)\], radar jamming recognition\[[26](https://arxiv.org/html/2608.20668#bib.bib19)\]and chromosome classification\[[23](https://arxiv.org/html/2608.20668#bib.bib20)\]\. Despite these advances, the layer\-wise optimization in ReduNet still relies on a Euclidean gradient update followed by normalization\. As a result, a fixed step size in the Euclidean space does not directly control the actual angular displacement of the features on the sphere, which may result in small or highly variable angular updates across samples\. Motivated by this observation, we propose a lightweight ReduNet with adaptive step sizes based on hyperspherical manifold learning, termed lightweight adaptive ReduNet \(LA\-ReduNet\)\. Compared with the gradient\-ascent\-based ReduNet, LA\-ReduNet substantially reduces the required number of unfolded layers and, correspondingly, the parameter storage of the unfolded module\. Simulation results show that LA\-ReduNet not only achieves a more lightweight network architecture but also outperforms gradient\-ascent\-based ReduNet in classification accuracy\.

In particular, LA\-ReduNet adopts the same adaptive multivariate Gaussian RD approximation function as AR\-ReduNet\. AR\-ReduNet improves ReduNet by introducing this adaptive approximation function, which provides a more accurate approximation of the coding rate and improves classification performance\[[10](https://arxiv.org/html/2608.20668#bib.bib4)\]\. Rather than further modifying the coding\-rate approximation, LA\-ReduNet focuses on redesigning the layer\-wise update rule of ReduNet under the unit\-sphere constraint\. Accordingly, AR\-ReduNet is included as a baseline in the subsequent comparisons\. It should be noted that Riemannian optimization on spheres is well established\[[1](https://arxiv.org/html/2608.20668#bib.bib24),[4](https://arxiv.org/html/2608.20668#bib.bib26)\]\. Therefore, the novelty of LA\-ReduNet does not lie in the use of tangent\-space or geodesic updates themselves\. Instead, we develop a normalized, truncated, and sample\-adaptive Riemannian update tailored to the forward layer\-wise MCR2construction, where each update is directly unfolded into a network layer\. The main contributions of this paper are summarized as follows:

1. 1\.LA\-ReduNet introduces a redesigned layer\-wise update rule for ReduNet under the unit\-sphere constraint\. Specifically, we observe that the Euclidean update followed by normalization in ReduNet does not directly control the actual angular displacement of each feature on the unit sphere\. Accordingly, a truncated and normalized Riemannian update scheme is constructed, in which the feasible update direction is explicitly determined in the tangent space, while the angular step size is adaptively adjusted for each sample according to the cosine similarity between its Euclidean gradient and radial direction\. In addition, samples with sufficiently small Riemannian update direction norms are excluded from further updates through a thresholding mechanism\. The resulting update scheme is naturally compatible with the layer\-wise unfolding of ReduNet and can therefore be directly implemented as a sequence of network layers\.
2. 2\.Theoretical properties of the proposed Riemannian update scheme are analyzed\. In particular, the Lipschitz continuity of the Riemannian update mapping is proved, and a lower\-bound inequality for the increment of the MCR2objective is derived on the product of unit spheres\. Building on these results, the finite\-termination property of the proposed algorithm is further established under the specified threshold and step\-size conditions\.
3. 3\.Simulation results on the CIFAR\-10, CIFAR\-100, and CINIC\-10 datasets demonstrate that LA\-ReduNet requires substantially fewer unfolded layers for both the MCR2objective and classification accuracy to stabilize\. In particular, LA\-ReduNet achieves classification\-accuracy convergence within approximately 5–10 layers\. Moreover, in our experiments, with ReduNet using its original step\-size setting and LA\-ReduNet adopting a relatively small base step size, LA\-ReduNet requires substantially fewer layers for the MCR2objective to reach a stable value\. Specifically, LA\-ReduNet achieves such objective convergence in approximately3535layers, compared with approximately10001000layers for ReduNet under the considered settings, corresponding to approximately1/291/29of the parameter storage required by ReduNet\.

The remainder of this paper is organized as follows: Section 2 reviews the necessary preliminaries of the MCR2principle, ReduNet, and AR\-ReduNet\. Section 3 introduces LA\-ReduNet and establishes its finite\-termination property\. Section 4 presents simulation results on multiple datasets, and Section 5 concludes this paper\.

## 2Preliminaries

This section provides a brief introduction to MCR2\[[22](https://arxiv.org/html/2608.20668#bib.bib1)\], ReduNet based on MCR2principle\[[5](https://arxiv.org/html/2608.20668#bib.bib3)\], and its improved version AR\-ReduNet\[[10](https://arxiv.org/html/2608.20668#bib.bib4)\]\.

### 2\.1MCR2Principle and ReduNet Framework

Consider a sample set𝑿=\[𝒙1,𝒙2,…,𝒙m\]∈ℝn×m\\bm\{X\}=\[\\bm\{x\}\_\{1\},\\bm\{x\}\_\{2\},\\ldots,\\bm\{x\}\_\{m\}\]\\in\\mathbb\{R\}^\{n\\times m\},𝒙∈ℝn\\bm\{x\}\\in\\mathbb\{R\}^\{n\}is a sample point\. Let𝒛i\\bm\{z\}\_\{i\}be a transformation of feature𝒙i\\bm\{x\}\_\{i\}\. For the set𝑿\\bm\{X\},𝒁=\[𝒛1,𝒛2,…,𝒛m\]∈ℝn×m\\bm\{Z\}=\[\\bm\{z\}\_\{1\},\\bm\{z\}\_\{2\},\.\.\.,\\bm\{z\}\_\{m\}\]\\in\\mathbb\{R\}^\{n\\times m\}is the feature matrix\. According to the MCR2principle\[[22](https://arxiv.org/html/2608.20668#bib.bib1)\],𝒁\\bm\{Z\}is updated by optimizing the following problem:

maximize𝒁Δ​R​\(𝐙,ϵ,𝚷\)=R⁡\(𝐙,ϵ\)−∑j=1kRc​\(𝐙,ϵ\|𝚷j\),\\displaystyle\\underset\{\\bm\{Z\}\}\{\\rm\{maximize\}\}\\hskip 10\.00002pt\\Delta R\(\\bm\{Z\},\\epsilon,\\bm\{\\Pi\}\)=R\(\\bm\{Z\},\\epsilon\)\-\\sum\_\{j=1\}^\{k\}R^\{\\rm c\}\(\\bm\{Z\},\\epsilon\|\\bm\{\\Pi\}\_\{j\}\),\(1\)s\.t\.𝒛1,…,𝒛m∈𝕊n−1,\\displaystyle\{\\rm\{s\.t\.\}\}\\hskip 10\.00002pt\\bm\{z\}\_\{1\},\\ldots,\\bm\{z\}\_\{m\}\\in\\mathbb\{S\}^\{n\-1\},where𝕊n−1\\mathbb\{S\}^\{n\-1\}denotes the unit sphere innn\-dimensional space\. The notations in \([1](https://arxiv.org/html/2608.20668#S2.E1)\) are explained as follows:

1. 1\.The memberships of the samples are depicted by a set ofkkdiagonal matrices𝚷=\{𝚷j\}j=1k\\bm\{\\Pi\}=\\\{\\bm\{\\Pi\}\_\{j\}\\\}\_\{j=1\}^\{k\}, where𝚷j\\bm\{\\Pi\}\_\{j\}is the membership matrix of classjjdefined by111By definition, the diagonal matrices𝚷=\{𝚷j\}j=1k\\bm\{\\Pi\}=\\\{\\bm\{\\Pi\}\_\{j\}\\\}\_\{j=1\}^\{k\}lie in a simplex\{𝚷:πi​j≥0,∑j=1k𝚷j=𝑰\}\\\{\\bm\{\\Pi\}:\\pi\_\{ij\}\\geq 0,\\sum\_\{j=1\}^\{k\}\\bm\{\\Pi\}\_\{j\}=\\bm\{I\}\\\}\. 𝚷j=diag⁡\(π1,j,π2,j,…,πm,j\)∈ℝm×m,\\displaystyle\\bm\{\\Pi\}\_\{j\}=\{\\rm diag\}\(\\pi\_\{1,j\},\\pi\_\{2,j\},\\ldots,\\pi\_\{m,j\}\)\\in\\mathbb\{R\}^\{m\\times m\},\(2\)withπi,j\\pi\_\{i,j\}being the label of theii\-th sample, i\.e\., πi,j=\{1,if𝒙iis in classj0,else\.\\displaystyle\\pi\_\{i,j\}=\\Bigg\\\{\\begin\{array\}\[\]\{ll\}1,&\\mbox\{if $\\bm\{x\}\_\{i\}$ is in class $j$\}\\\\ 0,&\\mbox\{else\}\\end\{array\}\.
2. 2\.The functionR⁡\(𝒁,ϵ\)R\(\\bm\{Z\},\\epsilon\)is the minimal number of binary bits needed to encode𝒁\\bm\{Z\}such that the expected decoding error is less thanϵ2\\epsilon^\{2\}, andRc​\(𝒁,ϵ\|𝚷j\)R^\{\\rm c\}\(\\bm\{Z\},\\epsilon\|\\bm\{\\Pi\}\_\{j\}\)is the sum of the minimal number of binary bits for each class\. In particular, consider a vector source𝒛∈ℝn\\bm\{z\}\\in\\mathbb\{R\}^\{n\}from a zero\-mean multivariate Gaussian distribution𝒛∼𝒩⁡\(0,𝚺\)\\bm\{z\}\\sim\\mathcal\{N\}\(0,\\bm\{\\Sigma\}\), an approximation of the number of binary bits needed to encode𝒁\\bm\{Z\}is given by\[[14](https://arxiv.org/html/2608.20668#bib.bib2)\] R⁡\(D\)≜12​log​det\(𝑰\+nD​𝚺\),\\displaystyle R\(D\)\\triangleq\\frac\{1\}\{2\}\\log\\det\\Big\(\\bm\{I\}\+\\frac\{n\}\{D\}\\bm\{\\Sigma\}\\Big\),\(5\) In the objective function \([1](https://arxiv.org/html/2608.20668#S2.E1)\),R⁡\(𝒁,ϵ\)R\(\\bm\{Z\},\\epsilon\)is the minimum coding length of the whole set of features at distortionD=ϵ2D=\\epsilon^\{2\}, approximated by R⁡\(𝒁,ϵ\)≜12​log​det\(𝑰\+nm​ϵ2​𝒁​𝒁T\),\\displaystyle R\(\\bm\{Z\},\\epsilon\)\\triangleq\\frac\{1\}\{2\}\\log\\det\\Big\(\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{ZZ\}^\{\\rm T\}\\Big\),\(6\)where𝚺\\bm\{\\Sigma\}is replaced by its estimate𝒁​𝒁T/m\\bm\{ZZ\}^\{\\rm T\}/m\. While the minimum coding length of thejj\-th class is approximated by Rc​\(𝒁,ϵ\|𝚷j\)≜tr⁡\(𝚷j\)2​m​log​det\(𝑰\+ntr⁡\(𝚷j\)​ϵ2​𝒁​𝚷j​𝒁T\)\.\\displaystyle R^\{\\rm c\}\(\\bm\{Z\},\\epsilon\|\\bm\{\\Pi\}\_\{j\}\)\\triangleq\\frac\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\}\{2m\}\\log\\det\\left\(\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\\epsilon^\{2\}\}\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\\right\)\.\(7\) That is, the approximation of the covariance matrix of thejj\-th class is𝒁​𝚷j​𝒁T/tr⁡\(𝚷j\)\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}/\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\), and the weight of thejj\-th class istr⁡\(𝚷j\)/m\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)/m\.

𝒁\(ℓ−1\)\\bm\{Z\}^\{\(\\ell\-1\)\}𝑬\(ℓ\)\\bm\{E\}^\{\(\\ell\)\}𝑪1\(ℓ\),𝚷1\(ℓ−1\)\\bm\{C\}\_\{1\}^\{\(\\ell\)\},\\,\\bm\{\\Pi\}\_\{1\}^\{\(\\ell\-1\)\}𝑪2\(ℓ\),𝚷2\(ℓ−1\)\\bm\{C\}\_\{2\}^\{\(\\ell\)\},\\,\\bm\{\\Pi\}\_\{2\}^\{\(\\ell\-1\)\}⋮\\vdots𝑪k\(ℓ\),𝚷k\(ℓ−1\)\\bm\{C\}\_\{k\}^\{\(\\ell\)\},\\,\\bm\{\\Pi\}\_\{k\}^\{\(\\ell\-1\)\}𝑰\\bm\{I\}−\\bm\{\-\}\+\\bm\{\+\}η\\etaη\\eta𝒫𝕊n−1\\mathcal\{P\}\_\{\\mathbb\{S\}^\{n\-1\}\}𝒁\(ℓ\)\\bm\{Z\}^\{\(\\ell\)\}Expansion OperatorCompression OperatorsIdentity Matrix

Figure 1:Single\-Layer Structure of ReduNet\.Based on the MCR2principle, the architecture of ReduNet can be directly derived from objective function \([1](https://arxiv.org/html/2608.20668#S2.E1)\)\. As illustrated in Fig\.[1](https://arxiv.org/html/2608.20668#S2.F1), the feature matrix𝒁\\bm\{Z\}is updated via gradient ascent\. That is,

𝒁\(ℓ\)∝𝒁\(ℓ−1\)\+η​𝑬\(ℓ\)​𝒁\(ℓ−1\)−η⁡\(∑j=1k𝑪j\(ℓ\)​𝒁\(ℓ−1\)​𝚷j\(ℓ−1\)\),\\displaystyle\\bm\{Z\}^\{\(\\ell\)\}\\propto\\bm\{Z\}^\{\(\\ell\-1\)\}\+\\eta\\bm\{E\}^\{\(\\ell\)\}\\bm\{Z\}^\{\(\\ell\-1\)\}\-\\eta\\Big\(\\sum\_\{j=1\}^\{k\}\\bm\{C\}\_\{j\}^\{\(\\ell\)\}\\bm\{Z\}^\{\(\\ell\-1\)\}\\bm\{\\Pi\}\_\{j\}^\{\(\\ell\-1\)\}\\Big\),s\.t\.𝒛1\(ℓ\),…,𝒛m\(ℓ\)∈𝕊n−1,\\displaystyle\\hskip 10\.00002pt\{\\rm\{s\.t\.\}\}\\hskip 10\.00002pt\\bm\{z\}\_\{1\}^\{\(\\ell\)\},\\ldots,\\bm\{z\}\_\{m\}^\{\(\\ell\)\}\\in\\mathbb\{S\}^\{n\-1\},\(8\)whereη\\etais the step size, the matrices𝑬\(ℓ\)\\bm\{E\}^\{\(\\ell\)\}and𝑪j\(ℓ\)\\bm\{C\}\_\{j\}^\{\(\\ell\)\}are the values of𝑬\\bm\{E\}and𝑪j\\bm\{C\}\_\{j\}at theℓ\\ell\-th iteration, and𝑬\\bm\{E\}and𝑪j\\bm\{C\}\_\{j\}are defined by

𝑬\\displaystyle\\bm\{E\}=nm​ϵ2​\(𝑰\+nm​ϵ2​𝒁​𝒁T\)−1,\\displaystyle=\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\\right\)^\{\-1\},\(9\)𝑪j\\displaystyle\\bm\{C\}\_\{j\}=nm​ϵ2​\(𝑰\+ntr⁡\(𝚷j\)​ϵ2​𝒁​𝚷j​𝒁T\)−1\.\\displaystyle=\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\\epsilon^\{2\}\}\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\\right\)^\{\-1\}\.\(10\)
In particular, the network is initialized by projecting the samples to𝕊n−1\\mathbb\{S\}^\{n\-1\}, i\.e\.,

𝒁\(0\)=\[𝒛1\(0\),…,𝒛m\(0\)\],s\.t\.𝒛i\(0\)=𝒙i‖𝒙i‖2∈𝕊n−1,i=1,…,m\.\\displaystyle\\bm\{Z\}^\{\(0\)\}=\[\\bm\{z\}\_\{1\}^\{\(0\)\},\\ldots,\\bm\{z\}\_\{m\}^\{\(0\)\}\],\\hskip 10\.00002pt\{\\rm s\.t\.\}~\\bm\{z\}\_\{i\}^\{\(0\)\}=\\frac\{\\bm\{x\}\_\{i\}\}\{\|\|\\bm\{x\}\_\{i\}\|\|\_\{2\}\}\\in\\mathbb\{S\}^\{n\-1\},\\hskip 10\.00002pti=1,\\ldots,m\.\(11\)Notice that the membership matrices𝚷j=diag⁡\(π1,j,π2,j,…,πm,j\)\\bm\{\\Pi\}\_\{j\}=\{\\rm diag\}\(\\pi\_\{1,j\},\\pi\_\{2,j\},\\ldots,\\pi\_\{m,j\}\)are known during training\. However, during the testing phase,𝚷j\\bm\{\\Pi\}\_\{j\}cannot be obtained from the labels of the test samples\. Therefore, the matrix𝚷j\\bm\{\\Pi\}\_\{j\}is predicted from the data\[[5](https://arxiv.org/html/2608.20668#bib.bib3)\]\. That is,𝚷j=diag⁡\(π^1,j,π^2,j,…,π^m,j\)\\bm\{\\Pi\}\_\{j\}=\{\\rm diag\}\(\\hat\{\\pi\}\_\{1,j\},\\hat\{\\pi\}\_\{2,j\},\\ldots,\\hat\{\\pi\}\_\{m,j\}\), whereπ^i,j\\hat\{\\pi\}\_\{i,j\}is the estimation ofπi,j\\pi\_\{i,j\}\. The estimation of the probability that theii\-th sample belongs to thejj\-th classπ^i,j\\hat\{\\pi\}\_\{i,j\}is obtained from the softmax function,

π^i,j=exp⁡\(−λ​‖𝑪j​𝒛i‖2\)∑j=1kexp⁡\(−λ​‖𝑪j​𝒛i‖2\),\\displaystyle\\hat\{\\pi\}\_\{i,j\}=\\frac\{\\exp\(\-\\lambda\|\|\\bm\{C\}\_\{j\}\\bm\{z\}\_\{i\}\|\|\_\{2\}\)\}\{\\sum\_\{j=1\}^\{k\}\\exp\(\-\\lambda\|\|\\bm\{C\}\_\{j\}\\bm\{z\}\_\{i\}\|\|\_\{2\}\)\},\(12\)whereλ\\lambdais a hyperparameter that controls the uniformity\.

### 2\.2AR\-ReduNet Based on Improved Approximation of Rate\-Distortion Function

Although the analytical expression of the multivariate Gaussian RD function can be derived via reverse water\-filling, it remains computationally intractable for high\-dimensional data\. Therefore, ReduNet adopts approximation \([5](https://arxiv.org/html/2608.20668#S2.E5)\) to accurately approximate the multivariate Gaussian RD function\. As mentioned above, when the distortionDDis large, the approximation error of \([5](https://arxiv.org/html/2608.20668#S2.E5)\) becomes significant\. To this end, AR\-ReduNet employs a multivariate Gaussian RD approximation with an adaptive regularization term to estimate the inter\-class and intra\-class coding rates\. This improved approximation function enhances the approximation by adding a regularization parameterα\\alphato \([5](https://arxiv.org/html/2608.20668#S2.E5)\), i\.e\.,

Rα​\(D\)=12​log​det\(α​𝑰\+nD​𝚺\),\\displaystyle R\_\{\\alpha\}\(D\)=\\frac\{1\}\{2\}\\log\\det\\left\(\\alpha\\bm\{I\}\+\\frac\{n\}\{D\}\\bm\{\\Sigma\}\\right\),\(13\)where the parameterα∈\[0,1\]\\alpha\\in\[0,1\]is chosen such thatRα​\(D\)R\_\{\\alpha\}\(D\)achieves 0 with the exact RD function at the same pointD=tr⁡\(𝚺\)D=\{\\rm tr\}\(\\bm\{\\Sigma\}\), i\.e\.,α=α∗\\alpha=\\alpha^\{\*\}, whereα∗\\alpha^\{\*\}is the unique value satisfyingRα∗​\(tr​\(𝚺\)\)=0R\_\{\\alpha^\{\*\}\}\(\{\\rm tr\}\(\\bm\{\\Sigma\}\)\)=0\[[10](https://arxiv.org/html/2608.20668#bib.bib4)\]\.

Compared to ReduNet, AR\-ReduNet computes the MCR2principle by using the approximation function \([13](https://arxiv.org/html/2608.20668#S2.E13)\), which approaches the exact RD function more closely than function \([5](https://arxiv.org/html/2608.20668#S2.E5)\)\. The objective function of AR\-ReduNet is defined as

Δ​R​\(𝒁,ϵ,𝚷\)=\\displaystyle\\Delta R\(\\bm\{Z\},\\epsilon,\\bm\{\\Pi\}\)=12​log​det\(α​𝑰\+nm​ϵ2​𝒁​𝒁T\)\\displaystyle\\frac\{1\}\{2\}\\log\\det\\left\(\\alpha\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{ZZ\}^\{\\rm T\}\\right\)−∑j=1ktr⁡\(𝚷j\)2​mlogdet\(αj𝑰\+ntr⁡\(𝚷j\)​ϵ2𝒁𝚷j𝒁T\),\\displaystyle\-\\sum\_\{j=1\}^\{k\}\\frac\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\}\{2m\}\\log\\det\\left\(\\alpha\_\{j\}\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\\epsilon^\{2\}\}\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\\right\),\(14\)the parametersα\\alphaandαj\\alpha\_\{j\}are updated according to

logdet\(α𝑰\+ntr⁡\(𝒁​𝒁T\)𝒁𝒁T\)\\displaystyle\\log\\det\\left\(\\alpha\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{ZZ\}^\{\\rm T\}\)\}\\bm\{ZZ\}^\{\\rm T\}\\right\)=0,\\displaystyle=0,\(15a\)logdet\(αj𝑰\+ntr⁡\(𝒁​𝚷j​𝒁T\)𝒁𝚷j𝒁T\)\\displaystyle\\log\\det\\left\(\\alpha\_\{j\}\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\)\}\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\\right\)=0,\\displaystyle=0,\(15b\)whereα\\alphaandαj\\alpha\_\{j\}are determined by𝒁​𝒁T/tr⁡\(𝒁​𝒁T\)\\bm\{ZZ\}^\{\\rm T\}/\{\\rm tr\}\(\\bm\{ZZ\}^\{\\rm T\}\)and𝒁​𝚷j​𝒁T/tr⁡\(𝒁​𝚷j​𝒁T\)\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}/\{\\rm tr\}\(\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\), respectively\. Since these matrices are updated at each iteration,α\\alphaandαj\\alpha\_\{j\}are re\-evaluated accordingly\. In addition, when updating the parameters𝑬\\bm\{E\}and𝑪j\\bm\{C\}\_\{j\}of AR\-ReduNet,α\\alphaandαj\\alpha\_\{j\}are considered as constants\.

Since LA\-ReduNet also adopts the multivariate Gaussian RD approximation function \([13](https://arxiv.org/html/2608.20668#S2.E13)\) to compute the coding rate, we follow AR\-ReduNet and employ binary search to solve for the parametersα\\alphaandαj\\alpha\_\{j\}\. As shown in Algorithm[1](https://arxiv.org/html/2608.20668#alg1), given an error thresholdδ\\delta, the unique values ofα\\alphaandαj\\alpha\_\{j\}satisfying the condition can be efficiently obtained\.

Algorithm 1BinarySearch\(𝚺,δ\)\(\\bm\{\\Sigma\},\\delta\)1:

αL←0,αR←1\\alpha\_\{\\rm L\}\\leftarrow 0,\\alpha\_\{\\rm R\}\\leftarrow 1;

2:

α∗←αL\+αR2\\alpha^\{\*\}\\leftarrow\\frac\{\\alpha\_\{\\rm L\}\+\\alpha\_\{\\rm R\}\}\{2\};

3:while\(

\|Rα​\(tr⁡\(𝚺\)\)\|\>δ\|R\_\{\\alpha\}\(\{\\rm tr\}\(\\bm\{\\Sigma\}\)\)\|\>\\delta\)do

4:if

Rα​\(tr​\(𝚺\)\)<0R\_\{\\alpha\}\(\{\\rm tr\}\(\\bm\{\\Sigma\}\)\)<0then

5:

αL←α∗\\alpha\_\{\\rm L\}\\leftarrow\\alpha^\{\*\};

6:else

7:

αR←α∗\\alpha\_\{\\rm R\}\\leftarrow\\alpha^\{\*\};

8:endif

9:

α∗←αL\+αR2\\alpha^\{\*\}\\leftarrow\\frac\{\\alpha\_\{\\rm L\}\+\\alpha\_\{\\rm R\}\}\{2\};

10:endwhile

11:return

α∗\\alpha^\{\*\};

## 3Lightweight Adaptive ReduNet \(LA\-ReduNet\)

### 3\.1MCR2Optimization Objective Based on Hyperspherical Manifold Learning

By reviewing optimization objective \([1](https://arxiv.org/html/2608.20668#S2.E1)\), we consider an optimization problem on the unit sphere\. In ReduNet or AR\-ReduNet, projected gradient ascent is employed, with projection onto the unit\-sphere guaranteeing that‖𝒛‖2=1\\\|\\bm\{z\}\\\|\_\{2\}=1\. Although the normalization operation enforces the unit\-sphere constraint, the angular displacement induced by a fixed Euclidean update is controlled only indirectly and depends on both the radial and tangential components of the update\. Therefore, we construct the update directly in the tangent space and evolve the feature𝒛\\bm\{z\}along the corresponding geodesic on the unit sphere\.

Consider the feature vector𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}at theℓ\\ell\-th layer\. According to the optimization objective in \([14](https://arxiv.org/html/2608.20668#S2.E14)\), the implicit parametersα\\alphaand\{αj\}j=1k\\\{\\alpha\_\{j\}\\\}\_\{j=1\}^\{k\}are first computed from the current features and then kept fixed when forming the update direction with respect to𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}\. Under this frozen\-parameter setting, the Euclidean update direction can be decomposed into a radial component parallel to𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}and a tangential component lying in the tangent space of the unit sphere at𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}, i\.e\.,

∂Δ​R∂𝒛\|𝒛=𝒛\(ℓ\)=𝒈=𝒈T\+𝒈R∈ℝn,\\displaystyle\\frac\{\\partial\\Delta R\}\{\\partial\\bm\{z\}\}\\Big\|\_\{\\bm\{z\}=\\bm\{z\}^\{\(\\ell\)\}\}=\\bm\{g\}=\\bm\{g\}\_\{\\rm\{T\}\}\+\\bm\{g\}\_\{\\rm\{R\}\}\\in\\mathbb\{R\}^\{n\},\(16\)where𝒈T\\bm\{g\}\_\{\\rm T\}and𝒈R\\bm\{g\}\_\{\\rm R\}denote the tangential and radial components of the Euclidean update direction, respectively\.𝒈T\\bm\{g\}\_\{\\rm\{T\}\}can be expressed as

𝒈T≜𝒈−\(𝒈T​𝒛\(ℓ\)\)​𝒛\(ℓ\)∈ℝn\.\\displaystyle\\bm\{g\}\_\{\\rm\{T\}\}\\triangleq\\bm\{g\}\-\(\\bm\{g\}^\{\\rm T\}\\bm\{z\}^\{\(\\ell\)\}\)\\bm\{z\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{n\}\.\(17\)
xxyy𝕊n−1\\mathbb\{S\}\_\{n\-1\}𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}𝒛\(ℓ\+1\)\\bm\{z\}^\{\(\\ell\+1\)\}θ\\theta𝒈\\bm\{g\}𝒈R\\bm\{g\}\_\{\\rm\{R\}\}𝒈T\\bm\{g\}\_\{\\rm\{T\}\}Figure 2:Orthogonal decomposition of the iterative update on the sphere𝕊n−1\\mathbb\{S\}\_\{n\-1\}\.Next, by rotating through a small angleθ\\thetaat point𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}, the update of𝒛\(ℓ\+1\)\\bm\{z\}^\{\(\\ell\+1\)\}is given by

𝒛\(ℓ\+1\)=cos⁡θ⋅𝒛\(ℓ\)\+sin⁡θ⋅𝒈T‖𝒈T‖2,\\displaystyle\\bm\{z\}^\{\(\\ell\+1\)\}=\\cos\\theta\\cdot\\bm\{z\}^\{\(\\ell\)\}\+\\sin\\theta\\cdot\\frac\{\\bm\{g\}\_\{\\rm\{T\}\}\}\{\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}\},\(18\)for all𝒈T\\bm\{g\}\_\{\\rm\{T\}\}with‖𝒈T‖≠0\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\\neq 0\. As illustrated in Fig\.[2](https://arxiv.org/html/2608.20668#S3.F2), when optimizing on the sphere, only the tangential component of the gradient is useful for updating the direction, while the radial component is ineffective and will be discarded\. In addition, to avoid repeated evaluations of thesin⁡\(⋅\)\\sin\(\\cdot\)andcos⁡\(⋅\)\\cos\(\\cdot\)functions\[[15](https://arxiv.org/html/2608.20668#bib.bib5)\], we reparameterize the angleθ\\thetausing a parametert0t\_\{0\}by definingtan⁡\(θ/2\)=t0\\tan\(\\theta/2\)=t\_\{0\}\. Thus, the update formula \([18](https://arxiv.org/html/2608.20668#S3.E18)\) can be rewritten as

𝒛\(ℓ\+1\)=\{1−t21\+t2⋅𝒛\(ℓ\)\+2​t1\+t2⋅𝒈T∗,‖𝒈T‖2\>τ𝒛\(ℓ\),‖𝒈T‖2≤τ\.\\displaystyle\\bm\{z\}^\{\(\\ell\+1\)\}=\\Bigg\\\{\\begin\{array\}\[\]\{ll\}\\frac\{1\-t^\{2\}\}\{1\+t^\{2\}\}\\cdot\\bm\{z\}^\{\(\\ell\)\}\+\\frac\{2t\}\{1\+t^\{2\}\}\\cdot\\bm\{g\}\_\{\\rm\{T\}\}^\{\*\},&\\mbox\{$\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}\>\\tau$\}\\\\ \\bm\{z\}^\{\(\\ell\)\},&\\mbox\{$\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}\\leq\\tau$\}\\end\{array\}\.
xxyy𝕊1\\mathbb\{S\}^\{1\}𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}𝒈\\bm\{g\}𝒈R\\bm\{g\}\_\{\\rm\{R\}\}𝒈T\\bm\{g\}\_\{\\rm\{T\}\}\(a\)xxyy𝕊1\\mathbb\{S\}^\{1\}𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}𝒛\(ℓ\+1\)\\bm\{z\}^\{\(\\ell\+1\)\}2​t1\+t2⋅𝒈T∗\\frac\{2t\}\{1\+t^\{2\}\}\\cdot\\bm\{g\}\_\{\\rm\{T\}\}^\{\*\}𝒈\\bm\{g\}𝒈R\\bm\{g\}\_\{\\rm\{R\}\}𝒈T\\bm\{g\}\_\{\\rm\{T\}\}\(b\)
Figure 3:Example of gradient\-based update on the unit sphere𝕊1\\mathbb\{S\}^\{1\}: \(a\) when‖𝒈T‖2≤τ\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}\\leq\\tau, the update magnitude in the direction of𝒈T\\bm\{g\}\_\{\\rm\{T\}\}becomes extremely small, thus we terminate the update of𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}; \(b\) when‖𝒈T‖2\>τ\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}\>\\tau, the update of feature𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}is accelerated bytt, where the adaptive step\-size parameter lies in\[t0,t0​\(1\+β\)\]\[t\_\{0\},t\_\{0\}\(1\+\\beta\)\]\.where𝒈T∗=𝒈T/‖𝒈T‖2\\bm\{g\}\_\{\\rm\{T\}\}^\{\*\}=\\bm\{g\}\_\{\\rm\{T\}\}/\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}, andttrepresents the adaptive step\-size parameter\. Since only the tangential component contributes to a feasible update, the degree to which the gradient aligns with the tangent space reflects its effectiveness under the spherical constraint\. Inspired by the cosine\-similarity\-based strategy in\[[15](https://arxiv.org/html/2608.20668#bib.bib5)\], the quantity1−\|𝒈T​𝒛\(ℓ\)\|/‖𝒈‖21\-\|\\bm\{g\}^\{\\rm T\}\\bm\{z\}^\{\(\\ell\)\}\|/\\\|\\bm\{g\}\\\|\_\{2\}is adopted to characterize the tangential effectiveness of the Euclidean gradient\. It approaches zero when the gradient is nearly radial and increases as the gradient becomes more aligned with the tangent space\. The absolute value makes the measure invariant to the sign of the radial component, since inward and outward radial components are both removed by tangent\-space projection\. The adaptive step\-size parameterttcan be expressed as

t=t0⋅\(1\+β⋅\(1−\|𝒈T​𝒛\(ℓ\)‖𝒈‖2\|\)\),β≥0,\\displaystyle t=t\_\{0\}\\cdot\\left\(1\+\\beta\\cdot\\left\(1\-\\left\|\\frac\{\\bm\{g\}^\{\\rm T\}\\bm\{z\}^\{\(\\ell\)\}\}\{\|\|\\bm\{g\}\|\|\_\{2\}\}\\right\|\\right\)\\right\),\\hskip 10\.00002pt\\beta\\geq 0,\(21\)whereβ\\betais a hyperparameter that controls the maximum expansion factor of the step size\. Notice that the norm of the Riemannian update direction is not used to directly determine the update magnitude\. As illustrated in Fig\.[3](https://arxiv.org/html/2608.20668#S3.F3), the actual update is jointly controlled by the thresholdτ\\tauand the adaptive parametertt\. Specifically, when‖𝒈T‖2≤τ\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}\\leq\\tau, the effective update gradient of𝒛\(ℓ\)\\bm\{z\}^\{\(\\ell\)\}is regarded as sufficiently small, and the corresponding sample is no longer updated\. When‖𝒈T‖2\>τ\|\|\\bm\{g\}\_\{\\rm\{T\}\}\|\|\_\{2\}\>\\tau, the sample remains active and is updated along the normalized Riemannian update direction, with the update magnitude controlled by the adaptive parametertt\. This design corresponds to the subsequent theoretical analysis of finite\-termination, in which the inequality characterizing the variation of the MCR2objective guarantees a uniform positive lower bound on the objective increment for each active sample under the specified step\-size condition\. The algorithm terminates when no sample remains active at a given iteration\. To theoretically justify this stopping criterion, the Riemannian update mapping associated with the MCR2objective is analyzed next, and the finite\-termination property of the proposed algorithm is subsequently established\.

### 3\.2Riemannian Update Analysis and Finite\-Termination of MCR2

In this subsection, we establish a finite\-termination guarantee for the proposed Riemannian update scheme\. Specifically, we first derive the eigenvalue bounds for the matrices involved in the MCR2objective, and then establish the Lipschitz continuity of the implicit parameter mappings and the corresponding inverse\-matrix mappings\. Based on these auxiliary results, we establish the Lipschitz continuity of the Riemannian update mapping and subsequently derive an inequality characterizing the variation of the MCR2objective\. This inequality provides the theoretical foundation for analyzing the proposed geodesic update scheme and proving that the proposed scheme satisfies the prescribed stopping criterion within a finite number of iterations under suitable threshold and step\-size conditions\. According to \([17](https://arxiv.org/html/2608.20668#S3.E17)\), the Riemannian update mapping can be written as

𝑮T​\(𝒁\)=\[𝒈T,1,𝒈T,2,…,𝒈T,m\]∈ℝn×m\.\\displaystyle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)=\\bigl\[\\bm\{g\}\_\{\\rm\{T\},1\},\\bm\{g\}\_\{\\rm\{T\},2\},\\ldots,\\bm\{g\}\_\{\\rm\{T\},m\}\\bigr\]\\in\\mathbb\{R\}^\{n\\times m\}\.\(22\)This yields the proposition below\.

###### Proposition 1\.

The Riemannian update mapping𝐆T\\bm\{G\}\_\{\\rm T\}is Lipschitz continuous onℳ=\(𝕊n−1\)m\\mathcal\{M\}=\(\\mathbb\{S\}^\{n\-1\}\)^\{m\}with Lipschitz constant

Lgrad=\(m\+1\)​\(k\+1\)​LE\+2​n​\(n\+1\)ϵ2​\(1\+1m\),\\displaystyle L\_\{\\rm grad\}=\(m\+1\)\(k\+1\)L\_\{\\rm E\}\+\\frac\{2n\(n\+1\)\}\{\\epsilon^\{2\}\}\\left\(1\+\\frac\{1\}\{\\sqrt\{m\}\}\\right\),\(23\)where

LE=n⁡\(n\+1\)m​ϵ2​\[1\+\(n\+1\)​\(π​n\+2​nϵ2\)\]\.\\displaystyle L\_\{\\rm E\}=\\frac\{n\(n\+1\)\}\{m\\epsilon^\{2\}\}\\biggl\[1\+\(n\+1\)\\left\(\\pi\\sqrt\{n\}\+\\frac\{2n\}\{\\epsilon^\{2\}\}\\right\)\\biggr\]\.\(24\)Moreover,𝐆T\\bm\{G\}\_\{\\rm T\}satisfies

‖𝑮T​\(𝒁\)‖F≤2​n​\(n\+1\)m​ϵ2,∀𝒁∈ℳ\.\\displaystyle\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)\\\|\_\{F\}\\leq\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\},\\hskip 20\.00003pt\\forall\\,\\bm\{Z\}\\in\\mathcal\{M\}\.\(25\)Define

F⁡\(𝒁\)=Δ​R​\(𝒁,ϵ,𝚷,α⁡\(𝒁\),\{αj​\(𝒁j\)\}j=1k\),\\displaystyle F\(\\bm\{Z\}\)=\\Delta R\\left\(\\bm\{Z\},\\epsilon,\\bm\{\\Pi\};\\alpha\(\\bm\{Z\}\),\\\{\\alpha\_\{j\}\(\\bm\{Z\}^\{j\}\)\\\}\_\{j=1\}^\{k\}\\right\),\(26\)where the implicit parametersα⁡\(𝐙\)\\alpha\(\\bm\{Z\}\)andαj​\(𝐙j\)\\alpha\_\{j\}\(\\bm\{Z\}^\{j\}\)are recomputed according to the current feature matrix𝐙\\bm\{Z\}\. Then, there exists a constantLs\>0L\_\{\\rm s\}\>0such that, for any𝐙∈ℳ\\bm\{Z\}\\in\\mathcal\{M\}and any tangent vector𝛏∈T𝐙​ℳ\\bm\{\\xi\}\\in T\_\{\\bm\{Z\}\}\\mathcal\{M\},

F⁡\(Geo⁡\(𝒁,𝝃\)\)−F⁡\(𝒁\)≥⟨𝑮T​\(𝒁\),𝝃⟩F−Cα​‖𝝃‖F−Ls2​‖𝝃‖F2\.\\displaystyle F\\bigl\(\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\)\\bigr\)\-F\(\\bm\{Z\}\)\\geq\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\bm\{\\xi\}\\right\\rangle\_\{F\}\-C\_\{\\alpha\}\\\|\\bm\{\\xi\}\\\|\_\{F\}\-\\frac\{L\_\{\\rm s\}\}\{2\}\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\.\(27\)Here,𝐙=\[𝐳1,…,𝐳m\]\\bm\{Z\}=\[\\bm\{z\}\_\{1\},\\ldots,\\bm\{z\}\_\{m\}\]and𝛏=\[𝛏1,…,𝛏m\]\\bm\{\\xi\}=\[\\bm\{\\xi\}\_\{1\},\\ldots,\\bm\{\\xi\}\_\{m\}\]\. The constantCαC\_\{\\alpha\}is given by

Cα=2​n​n\+1m\.\\displaystyle C\_\{\\alpha\}=\\frac\{2n\\sqrt\{n\+1\}\}\{\\sqrt\{m\}\}\.\(28\)The mappingGeo⁡\(𝐙,𝛏\)\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\)can be expressed as

Geo⁡\(𝒁,𝝃\)\\displaystyle\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\)=\[cos\(∥𝝃1∥2\)𝒛1\+sin\(∥𝝃1∥2\)𝝃1‖𝝃1‖2,…,\\displaystyle=\\Biggl\[\\cos\\bigl\(\\\|\\bm\{\\xi\}\_\{1\}\\\|\_\{2\}\\bigr\)\\bm\{z\}\_\{1\}\+\\sin\\bigl\(\\\|\\bm\{\\xi\}\_\{1\}\\\|\_\{2\}\\bigr\)\\frac\{\\bm\{\\xi\}\_\{1\}\}\{\\\|\\bm\{\\xi\}\_\{1\}\\\|\_\{2\}\},\\ \\ldots,cos\(∥𝝃m∥2\)𝒛m\+sin\(∥𝝃m∥2\)𝝃m‖𝝃m‖2\]\.\\displaystyle\\hskip 20\.00003pt\\cos\\bigl\(\\\|\\bm\{\\xi\}\_\{m\}\\\|\_\{2\}\\bigr\)\\bm\{z\}\_\{m\}\+\\sin\\bigl\(\\\|\\bm\{\\xi\}\_\{m\}\\\|\_\{2\}\\bigr\)\\frac\{\\bm\{\\xi\}\_\{m\}\}\{\\\|\\bm\{\\xi\}\_\{m\}\\\|\_\{2\}\}\\Biggr\]\.\(29\)If𝛏i=𝟎\\bm\{\\xi\}\_\{i\}=\\bm\{0\}, the corresponding column is defined as𝐳i\\bm\{z\}\_\{i\}\. In particular, one may take

Ls=Lgrad\+2​n​\(n\+1\)m​ϵ2\.\\displaystyle L\_\{\\rm s\}=L\_\{\\rm grad\}\+\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\.\(30\)

Detailed proofs of the supporting results are provided from[A](https://arxiv.org/html/2608.20668#A1)to[D](https://arxiv.org/html/2608.20668#A4), while the proof of Proposition[1](https://arxiv.org/html/2608.20668#Thmproposition1)is given in[E](https://arxiv.org/html/2608.20668#A5)\. The inequality established above characterizes the variation of the MCR2objective under the proposed Riemannian update scheme, where the update direction at each iteration is formed with the current implicit parameters treated as fixed\. Based on this result, a finite\-termination guarantee for the proposed truncated Riemannian update is established in the following theorem\.

###### Theorem 1\.

Let\{𝐙\(ℓ\)\}ℓ≥0\\\{\\bm\{Z\}^\{\(\\ell\)\}\\\}\_\{\\ell\\geq 0\}be the sequence generated by \([3\.1](https://arxiv.org/html/2608.20668#S3.EGx17)\) from𝐙\(0\)∈ℳ\\bm\{Z\}^\{\(0\)\}\\in\\mathcal\{M\}, where𝐙\(0\)\\bm\{Z\}^\{\(0\)\}denotes the initial sample matrix\. Suppose thatt0\>0t\_\{0\}\>0,β≥0\\beta\\geq 0, andτ\>Cα\\tau\>C\_\{\\alpha\}, and that

arctan⁡\(t0​\(1\+β\)\)<τ−CαLs,\\displaystyle\\arctan\\\!\\bigl\(t\_\{0\}\(1\+\\beta\)\\bigr\)<\\frac\{\\tau\-C\_\{\\alpha\}\}\{L\_\{\\rm s\}\},\(31\)whereCαC\_\{\\alpha\}andLsL\_\{\\rm s\}are defined in \([28](https://arxiv.org/html/2608.20668#S3.E28)\) and \([30](https://arxiv.org/html/2608.20668#S3.E30)\), respectively\. Define

δτ=2​arctan⁡\(t0\)​\[τ−Cα−Ls​arctan⁡\(t0​\(1\+β\)\)\]\.\\displaystyle\\delta\_\{\\tau\}=2\\arctan\(t\_\{0\}\)\\left\[\\tau\-C\_\{\\alpha\}\-L\_\{\\rm s\}\\arctan\\\!\\bigl\(t\_\{0\}\(1\+\\beta\)\\bigr\)\\right\]\.\(32\)Thenδτ\>0\\delta\_\{\\tau\}\>0, and the proposed truncated Riemannian update scheme terminates no later than iteration

Tmax=⌈Fmax−F⁡\(𝒁\(0\)\)δτ⌉,\\displaystyle T\_\{\\max\}=\\left\\lceil\\frac\{F\_\{\\max\}\-F\(\\bm\{Z\}^\{\(0\)\}\)\}\{\\delta\_\{\\tau\}\}\\right\\rceil,\(33\)whereFmaxF\_\{\\max\}denotes a finite upper bound onF⁡\(𝐙\)F\(\\bm\{Z\}\)for𝐙∈ℳ\\bm\{Z\}\\in\\mathcal\{M\}\.

###### Proof\.

Suppose that the stopping criterion is not satisfied at the beginning of iterationℓ\\ell\. Define the set of effective column indices as

ℐℓ=\{i:‖𝒈T,i\(ℓ\)‖2\>τ\},\\displaystyle\\mathcal\{I\}\_\{\\ell\}=\\left\\\{i:\\\|\\bm\{g\}\_\{\\mathrm\{T\},i\}^\{\(\\ell\)\}\\\|\_\{2\}\>\\tau\\right\\\},\(34\)where𝒈T,i\(ℓ\)\\bm\{g\}\_\{\{\\rm T\},i\}^\{\(\\ell\)\}denotes theii\-th column of the Riemannian update direction at iterationℓ\\ell\. Thenℐℓ≠∅\\mathcal\{I\}\_\{\\ell\}\\neq\\varnothing\. For eachi∈ℐℓi\\in\\mathcal\{I\}\_\{\\ell\}, the update rule for theii\-th column is given by \([3\.1](https://arxiv.org/html/2608.20668#S3.EGx17)\)\. Let

θi\(ℓ\)=2​arctan⁡\(ti\),\\displaystyle\\theta\_\{i\}^\{\(\\ell\)\}=2\\arctan\\\!\\left\(t\_\{i\}\\right\),\(35\)wheretit\_\{i\}denotes the adaptive step\-size parameter of theii\-th column, andθi\(ℓ\)\\theta\_\{i\}^\{\(\\ell\)\}represents the geodesic distance traveled by that column on the unit sphere\. Since‖𝒛i\(ℓ\)‖2=1\\\|\\bm\{z\}\_\{i\}^\{\(\\ell\)\}\\\|\_\{2\}=1, we have

0≤\|𝒈iT​𝒛i\(ℓ\)‖𝒈i‖2\|≤1\.\\displaystyle 0\\leq\\left\|\\frac\{\\bm\{g\}\_\{i\}^\{\\rm T\}\\bm\{z\}\_\{i\}^\{\(\\ell\)\}\}\{\\\|\\bm\{g\}\_\{i\}\\\|\_\{2\}\}\\right\|\\leq 1\.\(36\)Thus, it follows from \([21](https://arxiv.org/html/2608.20668#S3.E21)\) that

t0≤ti≤t0​\(1\+β\)\.\\displaystyle t\_\{0\}\\leq t\_\{i\}\\leq t\_\{0\}\(1\+\\beta\)\.\(37\)Sincearctan⁡\(⋅\)\\arctan\(\\cdot\)is strictly increasing, we further have

2​arctan⁡\(t0\)≤θi\(ℓ\)≤2​arctan⁡\(t0​\(1\+β\)\)\.\\displaystyle 2\\arctan\(t\_\{0\}\)\\leq\\theta\_\{i\}^\{\(\\ell\)\}\\leq 2\\arctan\\\!\\bigl\(t\_\{0\}\(1\+\\beta\)\\bigr\)\.\(38\)
At theℓ\\ell\-th iteration, define the tangent update vector𝝃\(ℓ\)∈T𝒁\(ℓ\)​ℳ\\bm\{\\xi\}^\{\(\\ell\)\}\\in T\_\{\\bm\{Z\}^\{\(\\ell\)\}\}\\mathcal\{M\}by specifying itsii\-th column as

𝝃i\(ℓ\)=\{θi\(ℓ\)​𝒈T,i\(ℓ\)‖𝒈T,i\(ℓ\)‖2,i∈ℐℓ,𝟎,i∉ℐℓ\.\\displaystyle\\bm\{\\xi\}\_\{i\}^\{\(\\ell\)\}=\\begin\{cases\}\\displaystyle\\theta\_\{i\}^\{\(\\ell\)\}\\frac\{\\bm\{g\}\_\{\\mathrm\{T\},i\}^\{\(\\ell\)\}\}\{\\\|\\bm\{g\}\_\{\\mathrm\{T\},i\}^\{\(\\ell\)\}\\\|\_\{2\}\},&i\\in\\mathcal\{I\}\_\{\\ell\},\\\\\[8\.53581pt\] \\bm\{0\},&i\\notin\\mathcal\{I\}\_\{\\ell\}\.\\end\{cases\}\(39\)By Proposition[1](https://arxiv.org/html/2608.20668#Thmproposition1)and the definition of𝑮T​\(𝒁\)\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)in \([22](https://arxiv.org/html/2608.20668#S3.E22)\), the inequality in \([27](https://arxiv.org/html/2608.20668#S3.E27)\) gives

F⁡\(𝒁\(ℓ\+1\)\)−F⁡\(𝒁\(ℓ\)\)\\displaystyle F\(\\bm\{Z\}^\{\(\\ell\+1\)\}\)\-F\(\\bm\{Z\}^\{\(\\ell\)\}\)≥⟨𝑮T​\(𝒁\(ℓ\)\),𝝃\(ℓ\)⟩F−Cα​‖𝝃\(ℓ\)‖F−Ls2​‖𝝃\(ℓ\)‖F2\\displaystyle\\geq\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}^\{\(\\ell\)\}\),\\bm\{\\xi\}^\{\(\\ell\)\}\\right\\rangle\_\{F\}\-C\_\{\\alpha\}\\\|\\bm\{\\xi\}^\{\(\\ell\)\}\\\|\_\{F\}\-\\frac\{L\_\{\\rm s\}\}\{2\}\\\|\\bm\{\\xi\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}\(40\)=∑i∈ℐℓθi\(ℓ\)​‖𝒈T,i\(ℓ\)‖2−Cα​\[∑i∈ℐℓ\(θi\(ℓ\)\)2\]1/2−Ls2​∑i∈ℐℓ\(θi\(ℓ\)\)2\\displaystyle=\\sum\_\{i\\in\\mathcal\{I\}\_\{\\ell\}\}\\theta\_\{i\}^\{\(\\ell\)\}\\\|\\bm\{g\}\_\{\{\\rm T\},i\}^\{\(\\ell\)\}\\\|\_\{2\}\-C\_\{\\alpha\}\\left\[\\sum\_\{i\\in\\mathcal\{I\}\_\{\\ell\}\}\\bigl\(\\theta\_\{i\}^\{\(\\ell\)\}\\bigr\)^\{2\}\\right\]^\{1/2\}\-\\frac\{L\_\{\\rm s\}\}\{2\}\\sum\_\{i\\in\\mathcal\{I\}\_\{\\ell\}\}\\bigl\(\\theta\_\{i\}^\{\(\\ell\)\}\\bigr\)^\{2\}\(41\)≥∑i∈ℐℓ\[θi\(ℓ\)​‖𝒈T,i\(ℓ\)‖2−Cα​θi\(ℓ\)−Ls2​\(θi\(ℓ\)\)2\]\\displaystyle\\geq\\sum\_\{i\\in\\mathcal\{I\}\_\{\\ell\}\}\\left\[\\theta\_\{i\}^\{\(\\ell\)\}\\\|\\bm\{g\}\_\{\{\\rm T\},i\}^\{\(\\ell\)\}\\\|\_\{2\}\-C\_\{\\alpha\}\\theta\_\{i\}^\{\(\\ell\)\}\-\\frac\{L\_\{\\rm s\}\}\{2\}\\bigl\(\\theta\_\{i\}^\{\(\\ell\)\}\\bigr\)^\{2\}\\right\]\(42\)\>∑i∈ℐℓ\[\(τ−Cα\)​θi\(ℓ\)−Ls2​\(θi\(ℓ\)\)2\]\.\\displaystyle\>\\sum\_\{i\\in\\mathcal\{I\}\_\{\\ell\}\}\\left\[\(\\tau\-C\_\{\\alpha\}\)\\theta\_\{i\}^\{\(\\ell\)\}\-\\frac\{L\_\{\\rm s\}\}\{2\}\\bigl\(\\theta\_\{i\}^\{\(\\ell\)\}\\bigr\)^\{2\}\\right\]\.\(43\)The equality in \([41](https://arxiv.org/html/2608.20668#S3.E41)\) follows from \([22](https://arxiv.org/html/2608.20668#S3.E22)\) and \([39](https://arxiv.org/html/2608.20668#S3.E39)\), while the subsequent inequality follows from‖𝝃\(ℓ\)‖F≤∑i∈ℐℓθi\(ℓ\)\\\|\\bm\{\\xi\}^\{\(\\ell\)\}\\\|\_\{F\}\\leq\\sum\_\{i\\in\\mathcal\{I\}\_\{\\ell\}\}\\theta\_\{i\}^\{\(\\ell\)\}, sinceθi\(ℓ\)≥0\\theta\_\{i\}^\{\(\\ell\)\}\\geq 0\. Finally, the strict inequality in \([43](https://arxiv.org/html/2608.20668#S3.E43)\) follows from‖𝒈T,i\(ℓ\)‖2\>τ\\\|\\bm\{g\}\_\{\{\\rm T\},i\}^\{\(\\ell\)\}\\\|\_\{2\}\>\\taufor everyi∈ℐℓi\\in\\mathcal\{I\}\_\{\\ell\}\. For eachi∈ℐℓi\\in\\mathcal\{I\}\_\{\\ell\}, using \([38](https://arxiv.org/html/2608.20668#S3.E38)\) together with the step\-size condition, we obtain

\(τ−Cα\)​θi\(ℓ\)−Ls2​\(θi\(ℓ\)\)2\\displaystyle\(\\tau\-C\_\{\\alpha\}\)\\theta\_\{i\}^\{\(\\ell\)\}\-\\frac\{L\_\{\\rm s\}\}\{2\}\\bigl\(\\theta\_\{i\}^\{\(\\ell\)\}\\bigr\)^\{2\}=θi\(ℓ\)​\(τ−Cα−Ls2​θi\(ℓ\)\)\\displaystyle=\\theta\_\{i\}^\{\(\\ell\)\}\\left\(\\tau\-C\_\{\\alpha\}\-\\frac\{L\_\{\\rm s\}\}\{2\}\\theta\_\{i\}^\{\(\\ell\)\}\\right\)≥2​arctan⁡\(t0\)​\[τ−Cα−Ls​arctan⁡\(t0​\(1\+β\)\)\]\.\\displaystyle\\geq 2\\arctan\(t\_\{0\}\)\\left\[\\tau\-C\_\{\\alpha\}\-L\_\{\\rm s\}\\arctan\\\!\\bigl\(t\_\{0\}\(1\+\\beta\)\\bigr\)\\right\]\.\(44\)By \([32](https://arxiv.org/html/2608.20668#S3.E32)\), the right\-hand side of \([44](https://arxiv.org/html/2608.20668#S3.E44)\) is equal toδτ\\delta\_\{\\tau\}\. Moreover, the step\-size condition ensures thatδτ\>0\\delta\_\{\\tau\}\>0\. Sinceℐℓ≠∅\\mathcal\{I\}\_\{\\ell\}\\neq\\varnothing, combining \([43](https://arxiv.org/html/2608.20668#S3.E43)\) and \([44](https://arxiv.org/html/2608.20668#S3.E44)\) yields

F⁡\(𝒁\(ℓ\+1\)\)−F⁡\(𝒁\(ℓ\)\)\>\|ℐℓ\|​δτ≥δτ\.\\displaystyle F\(\\bm\{Z\}^\{\(\\ell\+1\)\}\)\-F\(\\bm\{Z\}^\{\(\\ell\)\}\)\>\|\\mathcal\{I\}\_\{\\ell\}\|\\delta\_\{\\tau\}\\geq\\delta\_\{\\tau\}\.\(45\)Finally, suppose that the algorithm proceeds throughTTconsecutive iterations without satisfying the stopping criterion\. Summing \([45](https://arxiv.org/html/2608.20668#S3.E45)\) overℓ=0,…,T−1\\ell=0,\\ldots,T\-1gives

F⁡\(𝒁\(T\)\)−F⁡\(𝒁\(0\)\)\>T​δτ\.\\displaystyle F\(\\bm\{Z\}^\{\(T\)\}\)\-F\(\\bm\{Z\}^\{\(0\)\}\)\>T\\delta\_\{\\tau\}\.\(46\)Since the implicit parameter mappings are continuous by Lemma[2](https://arxiv.org/html/2608.20668#Thmlemma2), the objectiveFFis continuous on the compact manifoldℳ\\mathcal\{M\}\. Therefore,FFattains a finite maximumFmaxF\_\{\\max\}onℳ\\mathcal\{M\}\. Combining this with \([46](https://arxiv.org/html/2608.20668#S3.E46)\), we obtain

T​δτ<Fmax−F⁡\(𝒁\(0\)\),\\displaystyle T\\delta\_\{\\tau\}<F\_\{\\max\}\-F\(\\bm\{Z\}^\{\(0\)\}\),\(47\)which implies

T<Fmax−F⁡\(𝒁\(0\)\)δτ\.\\displaystyle T<\\frac\{F\_\{\\max\}\-F\(\\bm\{Z\}^\{\(0\)\}\)\}\{\\delta\_\{\\tau\}\}\.\(48\)Consequently, the algorithm terminates no later than iteration

Tmax=⌈Fmax−F⁡\(𝒁\(0\)\)δτ⌉,\\displaystyle T\_\{\\max\}=\\left\\lceil\\frac\{F\_\{\\max\}\-F\(\\bm\{Z\}^\{\(0\)\}\)\}\{\\delta\_\{\\tau\}\}\\right\\rceil,\(49\)where⌈⋅⌉\\lceil\\cdot\\rceildenotes the ceiling function\. This completes the proof\. ∎

Theorem[1](https://arxiv.org/html/2608.20668#Thmtheorem1)establishes a finite\-termination guarantee for the proposed Riemannian update scheme under the prescribed threshold and step\-size conditions\. It should be noted that these conditions are sufficient but not necessary\. Moreover, since the constantsCαC\_\{\\alpha\}andLsL\_\{\\rm s\}are derived from global worst\-case bounds over the entire feasible manifoldℳ\\mathcal\{M\}, the resulting sufficient conditions can be relatively conservative in practice\. Nevertheless, when these conditions are satisfied, they provide a finite\-termination guarantee even in the worst case\. Therefore, these conditions are intended primarily as theoretical guarantees rather than practical parameter\-selection rules\.

### 3\.3LA\-ReduNet Architecture

𝒛\(0\)∈ℝd\\bm\{z\}^\{\(0\)\}\\in\\mathbb\{R\}^\{d\}𝒫𝕊n−1\\mathcal\{P\}\_\{\\mathbb\{S\}^\{n\-1\}\}𝒟:ℝn→ℝd\\mathcal\{D\}:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}^\{d\}𝒙∈ℝn\\bm\{x\}\\in\\mathbb\{R\}^\{n\}Dimension\-reduction Module𝑬\(ℓ\)\\bm\{E\}^\{\(\\ell\)\}𝑪1\(ℓ\),𝚷1\(ℓ−1\)\\bm\{C\}\_\{1\}^\{\(\\ell\)\},\\,\\bm\{\\Pi\}\_\{1\}^\{\(\\ell\-1\)\}𝑪2\(ℓ\),𝚷2\(ℓ−1\)\\bm\{C\}\_\{2\}^\{\(\\ell\)\},\\,\\bm\{\\Pi\}\_\{2\}^\{\(\\ell\-1\)\}⋮\\vdots𝑪k\(ℓ\),𝚷k\(ℓ−1\)\\bm\{C\}\_\{k\}^\{\(\\ell\)\},\\,\\bm\{\\Pi\}\_\{k\}^\{\(\\ell\-1\)\}𝑰\\bm\{I\}−\\bm\{\-\}\+\\bm\{\+\}η\\etaη\\eta𝒈T∗\\bm\{g\}\_\{\\rm T\}^\{\*\}𝒈T↦𝒈T∗\\bm\{g\}\_\{\\rm T\}\\mapsto\\bm\{g\}\_\{\\rm T\}^\{\*\}τ\\tau𝒛\(L\)∈ℝd\\bm\{z\}^\{\(L\)\}\\in\\mathbb\{R\}^\{d\}LL\-Layer Structure of LA\-ReduNetExpansion OperatorCompression OperatorsIdentity MatrixThreshold\-Based Spherical Update

Figure 4:Feature Extraction Process of LA\-ReduNet\.![Refer to caption](https://arxiv.org/html/2608.20668v1/images/Figure_6.png)Figure 5:Structure of the dimension\-reduction module\.In this subsection, we present the architecture of LA\-ReduNet\. As discussed above, the parameter storage of a ReduNet\-type unfolded module depends on the network depthLL, the number of classeskk, and the feature dimensionnn, with an asymptotic complexity ofO​\(L​\(k\+1\)​n2\)\\textrm\{O\}\(L\(k\+1\)n^\{2\}\)\. For the experiments, the same lightweight dimension\-reduction module𝒟\\mathcal\{D\}is used as a common front\-end for ReduNet, AR\-ReduNet, and LA\-ReduNet to produce lower\-dimensional normalized inputs and ensure a controlled comparison\. As illustrated in Fig\.[4](https://arxiv.org/html/2608.20668#S3.F4), the feature dimension is reduced fromnntodd, whered≪nd\\ll n\. Accordingly, the parameter storage of the subsequent ReduNet\-type module has an asymptotic complexity ofO​\(L​\(k\+1\)​d2\)\\textrm\{O\}\(L\(k\+1\)d^\{2\}\)\. Since the same front\-end and feature dimension are used for all three variants, the reduction in the parameter storage of the unfolded LA\-ReduNet module relative to ReduNet and AR\-ReduNet mainly results from its substantially smaller network depth\. Algorithm[2](https://arxiv.org/html/2608.20668#alg2)and Algorithm[3](https://arxiv.org/html/2608.20668#alg3)respectively illustrate the training and testing procedures of LA\-ReduNet\.

Algorithm 2Training of LA\-ReduNet \(𝑿,𝚷,ϵ,t0,τ,β\\bm\{X\},\\bm\{\\Pi\},\\epsilon,t\_\{0\},\\tau,\\beta\)1:

𝒁\(0\)←\[𝒙1‖𝒙1‖2,…,𝒙m‖𝒙m‖2\]\\bm\{Z\}^\{\(0\)\}\\leftarrow\\Big\[\\frac\{\\bm\{x\}\_\{1\}\}\{\|\|\\bm\{x\}\_\{1\}\|\|\_\{2\}\},\\ldots,\\frac\{\\bm\{x\}\_\{m\}\}\{\|\|\\bm\{x\}\_\{m\}\|\|\_\{2\}\}\\Big\];

2:for

ℓ=1\\ell=1to

LLdo

3:

𝑬\(ℓ\)←dm​ϵ2​\(α​𝑰\+dm​ϵ2​𝒁\(ℓ−1\)​\(𝒁\(ℓ−1\)\)T\)−1\\bm\{E\}^\{\(\\ell\)\}\\leftarrow\\frac\{d\}\{m\\epsilon^\{2\}\}\\big\(\\alpha\\bm\{I\}\+\\frac\{d\}\{m\\epsilon^\{2\}\}\\bm\{Z\}^\{\(\\ell\-1\)\}\(\\bm\{Z\}^\{\(\\ell\-1\)\}\)^\{\\rm T\}\\big\)^\{\-1\}; //

α\\alphais given by Algorithm[1](https://arxiv.org/html/2608.20668#alg1)\.

4:for

j=1j=1to

kkdo

5:

𝑪j\(ℓ\)←dm​ϵ2​\(αj​𝑰\+dtr⁡\(𝚷j\(ℓ−1\)\)​ϵ2​𝒁\(ℓ−1\)​𝚷j\(ℓ−1\)​\(𝒁\(ℓ−1\)\)T\)−1\\bm\{C\}\_\{j\}^\{\(\\ell\)\}\\leftarrow\\frac\{d\}\{m\\epsilon^\{2\}\}\\big\(\\alpha\_\{j\}\\bm\{I\}\+\\frac\{d\}\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}^\{\(\\ell\-1\)\}\)\\epsilon^\{2\}\}\\bm\{Z\}^\{\(\\ell\-1\)\}\\bm\{\\Pi\}\_\{j\}^\{\(\\ell\-1\)\}\(\\bm\{Z\}^\{\(\\ell\-1\)\}\)^\{\\rm T\}\\big\)^\{\-1\}; //

αj\\alpha\_\{j\}is given by Algorithm[1](https://arxiv.org/html/2608.20668#alg1)\.

6:endfor

7:

𝑮\(ℓ\)←𝑬\(ℓ\)​𝒁\(ℓ−1\)−\(∑j=1k𝑪j\(ℓ\)​𝒁\(ℓ−1\)​𝚷j\(ℓ−1\)\);\\bm\{G\}^\{\(\\ell\)\}\\leftarrow\\bm\{E\}^\{\(\\ell\)\}\\bm\{Z\}^\{\(\\ell\-1\)\}\-\\Big\(\\sum\_\{j=1\}^\{k\}\\bm\{C\}\_\{j\}^\{\(\\ell\)\}\\bm\{Z\}^\{\(\\ell\-1\)\}\\bm\{\\Pi\}\_\{j\}^\{\(\\ell\-1\)\}\\Big\);//

𝑮\(ℓ\)=\[𝒈1\(ℓ\),𝒈2\(ℓ\),…,𝒈m\(ℓ\)\]∈ℝd×m\\bm\{G\}^\{\(\\ell\)\}=\[\\bm\{g\}^\{\(\\ell\)\}\_\{1\},\\bm\{g\}^\{\(\\ell\)\}\_\{2\},\\ldots,\\bm\{g\}^\{\(\\ell\)\}\_\{m\}\]\\in\\mathbb\{R\}^\{d\\times m\}is the gradient matrix corresponding to

𝒁\(ℓ−1\)\\bm\{Z\}^\{\(\\ell\-1\)\}\.

8:for

p=1p=1to

mmdo

9:

\(𝒈T\(ℓ\)\)p←𝒈p\(ℓ\)−\(\(𝒈p\(ℓ\)\)T​𝒛p\(ℓ−1\)\)​𝒛p\(ℓ−1\)\(\\bm\{g\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\\leftarrow\\bm\{g\}^\{\(\\ell\)\}\_\{p\}\-\(\(\\bm\{g\}^\{\(\\ell\)\}\_\{p\}\)^\{\\rm T\}\\bm\{z\}^\{\(\\ell\-1\)\}\_\{p\}\)\\bm\{z\}^\{\(\\ell\-1\)\}\_\{p\};

10:if

‖\(𝒈T\(ℓ\)\)p‖\>τ\\\|\(\\bm\{g\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\\\|\>\\tauthen

11:

t=t0⋅\(1\+β⋅\(1−\|\(𝒈p\(ℓ\)\)T​𝒛p\(ℓ−1\)‖𝒈p\(ℓ\)‖\|\)\)t=t\_\{0\}\\cdot\\left\(1\+\\beta\\cdot\\left\(1\-\\left\|\\frac\{\(\\bm\{g\}\_\{p\}^\{\(\\ell\)\}\)^\{\\rm T\}\\bm\{z\}\_\{p\}^\{\(\\ell\-1\)\}\}\{\|\|\\bm\{g\}\_\{p\}^\{\(\\ell\)\}\|\|\}\\right\|\\right\)\\right\);

12:

𝒛p\(ℓ\)←1−t21\+t2​𝒛p\(ℓ−1\)\+2​t1\+t2​\(𝒈T\(ℓ\)\)p‖\(𝒈T\(ℓ\)\)p‖\\bm\{z\}^\{\(\\ell\)\}\_\{p\}\\leftarrow\\displaystyle\\frac\{1\-t^\{2\}\}\{1\+t^\{2\}\}\\bm\{z\}^\{\(\\ell\-1\)\}\_\{p\}\+\\frac\{2t\}\{1\+t^\{2\}\}\\frac\{\(\\bm\{g\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\}\{\|\|\(\\bm\{g\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\|\|\};

13:else

14:

𝒛p\(ℓ\)←𝒛p\(ℓ−1\)\\bm\{z\}^\{\(\\ell\)\}\_\{p\}\\leftarrow\\bm\{z\}^\{\(\\ell\-1\)\}\_\{p\};

15:endif

16:endfor

17:

𝒁\(ℓ\)=\[𝒛1\(ℓ\),𝒛2\(ℓ\),…,𝒛m\(ℓ\)\]\\bm\{Z\}^\{\(\\ell\)\}=\[\\bm\{z\}^\{\(\\ell\)\}\_\{1\},\\bm\{z\}^\{\(\\ell\)\}\_\{2\},\\ldots,\\bm\{z\}^\{\(\\ell\)\}\_\{m\}\];

18:endfor

19:return

𝒁\(L\),\{𝑬\(ℓ\),𝑪1\(ℓ\),𝑪2\(ℓ\),…,𝑪k\(ℓ\)\}ℓ=1L\\bm\{Z\}^\{\(L\)\},\\\{\\bm\{E\}^\{\(\\ell\)\},\\bm\{C\}\_\{1\}^\{\(\\ell\)\},\\bm\{C\}\_\{2\}^\{\(\\ell\)\},\\ldots,\\bm\{C\}\_\{k\}^\{\(\\ell\)\}\\\}\_\{\\ell=1\}^\{L\}\.

Algorithm 3Testing of LA\-ReduNet \(𝑿~,\{𝑬\(ℓ\),𝑪1\(ℓ\),…,𝑪k\(ℓ\)\}ℓ=1L,t0,τ,β,λ\\widetilde\{\\bm\{X\}\},\\\{\\bm\{E\}^\{\(\\ell\)\},\\bm\{C\}\_\{1\}^\{\(\\ell\)\},\\ldots,\\bm\{C\}\_\{k\}^\{\(\\ell\)\}\\\}\_\{\\ell=1\}^\{L\},t\_\{0\},\\tau,\\beta,\\lambda\)1:

𝒁~\(0\)←\[𝒙~1‖𝒙~1‖2,…,𝒙~m‖𝒙~m‖2\]\\widetilde\{\\bm\{Z\}\}^\{\(0\)\}\\leftarrow\\Big\[\\frac\{\\widetilde\{\\bm\{x\}\}\_\{1\}\}\{\|\|\\widetilde\{\\bm\{x\}\}\_\{1\}\|\|\_\{2\}\},\\ldots,\\frac\{\\widetilde\{\\bm\{x\}\}\_\{m\}\}\{\|\|\\widetilde\{\\bm\{x\}\}\_\{m\}\|\|\_\{2\}\}\\Big\]; //

𝒙~i\\widetilde\{\\bm\{x\}\}\_\{i\}is the

ii\-th column of data matrix

𝑿~\\widetilde\{\\bm\{X\}\}\.

2:for

ℓ=1\\ell=1to

LLdo

3:for

j=1j=1to

kkdo

4:for

i=1i=1to

mmdo

5:

π^i,j\(ℓ−1\)←exp⁡\(−λ​‖𝑪j\(ℓ\)​𝒛~i\(ℓ−1\)‖\)∑j=1kexp⁡\(−λ​‖𝑪j\(ℓ\)​𝒛~i\(ℓ−1\)‖\)\\hat\{\\pi\}\_\{i,j\}^\{\(\\ell\-1\)\}\\leftarrow\\frac\{\\exp\(\-\\lambda\|\|\\bm\{C\}^\{\(\\ell\)\}\_\{j\}\\bm\{\\widetilde\{z\}\}\_\{i\}^\{\(\\ell\-1\)\}\|\|\)\}\{\\sum\_\{j=1\}^\{k\}\\exp\(\-\\lambda\|\|\\bm\{C\}^\{\(\\ell\)\}\_\{j\}\\bm\{\\widetilde\{z\}\}\_\{i\}^\{\(\\ell\-1\)\}\|\|\)\};

6:endfor

7:

𝚷j\(ℓ−1\)←diag⁡\(π^1,j\(ℓ−1\),…,π^m,j\(ℓ−1\)\)\\bm\{\\Pi\}\_\{j\}^\{\(\\ell\-1\)\}\\leftarrow\{\\rm diag\}\(\\hat\{\\pi\}\_\{1,j\}^\{\(\\ell\-1\)\},\\ldots,\\hat\{\\pi\}\_\{m,j\}^\{\(\\ell\-1\)\}\);

8:endfor

9:

𝑮~\(ℓ\)←𝑬\(ℓ\)​𝒁~\(ℓ−1\)−\(∑j=1k𝑪j\(ℓ\)​𝒁~\(ℓ−1\)​𝚷j\(ℓ−1\)\);\\widetilde\{\\bm\{G\}\}^\{\(\\ell\)\}\\leftarrow\\bm\{E\}^\{\(\\ell\)\}\\widetilde\{\\bm\{Z\}\}^\{\(\\ell\-1\)\}\-\\Big\(\\sum\_\{j=1\}^\{k\}\\bm\{C\}\_\{j\}^\{\(\\ell\)\}\\widetilde\{\\bm\{Z\}\}^\{\(\\ell\-1\)\}\\bm\{\\Pi\}\_\{j\}^\{\(\\ell\-1\)\}\\Big\);//

𝑮~\(ℓ\)=\[𝒈~1\(ℓ\),𝒈~2\(ℓ\),…,𝒈~m\(ℓ\)\]∈ℝd×m\\widetilde\{\\bm\{G\}\}^\{\(\\ell\)\}=\[\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{1\},\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{2\},\\ldots,\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{m\}\]\\in\\mathbb\{R\}^\{d\\times m\}is the gradient matrix corresponding to

𝒁~\(ℓ−1\)\\widetilde\{\\bm\{Z\}\}^\{\(\\ell\-1\)\}\.

10:for

p=1p=1to

mmdo

11:

\(𝒈~T\(ℓ\)\)p←𝒈~p\(ℓ\)−\(\(𝒈~p\(ℓ\)\)T​𝒛~p\(ℓ−1\)\)​𝒛~p\(ℓ−1\)\(\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\\leftarrow\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{p\}\-\(\(\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{p\}\)^\{\\rm T\}\\widetilde\{\\bm\{z\}\}^\{\(\\ell\-1\)\}\_\{p\}\)\\widetilde\{\\bm\{z\}\}^\{\(\\ell\-1\)\}\_\{p\};

12:if

‖\(𝒈~T\(ℓ\)\)p‖\>τ\\\|\(\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\\\|\>\\tauthen

13:

t=t0⋅\(1\+β⋅\(1−\|\(𝒈~p\(ℓ\)\)T​𝒛~p\(ℓ−1\)‖𝒈~p\(ℓ\)‖\|\)\)t=t\_\{0\}\\cdot\\left\(1\+\\beta\\cdot\\left\(1\-\\left\|\\frac\{\(\\widetilde\{\\bm\{g\}\}\_\{p\}^\{\(\\ell\)\}\)^\{\\rm T\}\\widetilde\{\\bm\{z\}\}\_\{p\}^\{\(\\ell\-1\)\}\}\{\|\|\\widetilde\{\\bm\{g\}\}\_\{p\}^\{\(\\ell\)\}\|\|\}\\right\|\\right\)\\right\);

14:

𝒛~p\(ℓ\)←1−t21\+t2​𝒛~p\(ℓ−1\)\+2​t1\+t2​\(𝒈~T\(ℓ\)\)p‖\(𝒈~T\(ℓ\)\)p‖\\widetilde\{\\bm\{z\}\}^\{\(\\ell\)\}\_\{p\}\\leftarrow\\displaystyle\\frac\{1\-t^\{2\}\}\{1\+t^\{2\}\}\\widetilde\{\\bm\{z\}\}^\{\(\\ell\-1\)\}\_\{p\}\+\\frac\{2t\}\{1\+t^\{2\}\}\\frac\{\(\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\}\{\|\|\(\\widetilde\{\\bm\{g\}\}^\{\(\\ell\)\}\_\{\\rm T\}\)\_\{p\}\|\|\};

15:else

16:

𝒛~p\(ℓ\)←𝒛~p\(ℓ−1\)\\widetilde\{\\bm\{z\}\}^\{\(\\ell\)\}\_\{p\}\\leftarrow\\widetilde\{\\bm\{z\}\}^\{\(\\ell\-1\)\}\_\{p\};

17:endif

18:endfor

19:

𝒁~\(ℓ\)=\[𝒛~1\(ℓ\),𝒛~2\(ℓ\),…,𝒛~m\(ℓ\)\]\\widetilde\{\\bm\{Z\}\}^\{\(\\ell\)\}=\[\\widetilde\{\\bm\{z\}\}^\{\(\\ell\)\}\_\{1\},\\widetilde\{\\bm\{z\}\}^\{\(\\ell\)\}\_\{2\},\\ldots,\\widetilde\{\\bm\{z\}\}^\{\(\\ell\)\}\_\{m\}\];

20:endfor

21:return

𝒁~\(L\)\\widetilde\{\\bm\{Z\}\}^\{\(L\)\}\.

The dimension\-reduction module is implemented using a simple convolutional architecture and is shared by all ReduNet variants in the experiments\. Its role is to generate lower\-dimensional normalized features for the subsequent ReduNet\-type modules\. This convolutional front\-end is trained separately through backpropagation, whereas the subsequent LA\-ReduNet module retains the white\-box construction of ReduNet\. As illustrated in Fig\.[5](https://arxiv.org/html/2608.20668#S3.F5), the module consists of four cascaded convolutional blocks, followed by global average pooling \(GAP\) and a flattening operation, producing a 512\-dimensional feature vector\. Since the subsequent ReduNet\-type modules require unit\-norm input features, the output feature vectors are normalized before being fed into ReduNet, AR\-ReduNet, or LA\-ReduNet\.

For training the common convolutional front\-end, the same combined loss is used for all ReduNet variants\. Since the downstream ReduNet\-type models are constructed based on the MCR2objective, cross\-entropy \(CE\) and MCR2are jointly employed to encourage the learned low\-dimensional features to be compatible with the subsequent modules\. The combined loss is defined as

ℒcombined=ℒCE−μ​Δ​R,\\displaystyle\\mathcal\{L\}\_\{\\text\{combined\}\}=\\mathcal\{L\}\_\{\\text\{CE\}\}\-\\mu\\Delta R,\(50\)whereℒCE\\mathcal\{L\}\_\{\\text\{CE\}\}denotes the CE loss,Δ​R\\Delta Ris derived from \([14](https://arxiv.org/html/2608.20668#S2.E14)\), andμ\\murepresents weight coefficient\. Since the output features are normalized to unit norm, scaled cosine similarities are used as the classification logits\[[13](https://arxiv.org/html/2608.20668#bib.bib7)\]\. In this way, classification is determined by the angular similarity between the feature vectors and the weight vectors, making the classifier compatible with the normalized feature space\. The corresponding cosine\-based softmax CE loss is formulated as

ℒCE=−1m∑i=1mlogeacosθyieacosθyi\+∑j≠yikeacosθj,\\displaystyle\\mathcal\{L\}\_\{\\text\{CE\}\}=\-\\frac\{1\}\{m\}\\sum\_\{i=1\}^\{m\}\\log\\frac\{e^\{a\\cos\\theta\_\{y\_\{i\}\}\}\}\{e^\{a\\cos\\theta\_\{y\_\{i\}\}\}\+\\sum\_\{j\\neq y\_\{i\}\}^\{k\}e^\{a\\cos\\theta\_\{j\}\}\},\(51\)whereθyi\\theta\_\{y\_\{i\}\}denotes the angle between the feature of theii\-th sample and the weight vector of its ground\-truth classyiy\_\{i\}, andaais a scaling factor\.

## 4Simulation Results

In this section, we evaluate the performance of LA\-ReduNet across multiple datasets\. Compared with the baseline algorithms ReduNet and AR\-ReduNet, LA\-ReduNet requires substantially fewer unfolded layers\. Correspondingly, the parameter storage of the unfolded LA\-ReduNet module is also significantly smaller than that of the corresponding ReduNet and AR\-ReduNet modules\. Following the idea of ReduNet\[[5](https://arxiv.org/html/2608.20668#bib.bib3)\], we select ReduNet, AR\-ReduNet, and LA\-ReduNet as feature extractors, and Nearest Subspace \(NS\) as the classifier\. For a given feature𝒛test\\bm\{z\}\_\{\\textrm\{test\}\}of the test sample, the NS classifier provides the predicted labeljpredj\_\{\\textrm\{pred\}\}, which satisfies:

jpred=arg⁡mint∈\{1,…,k\}⁡‖\(𝑰−𝑼t​𝑼tT\)​𝒛test‖22,\\displaystyle j\_\{\\mathrm\{pred\}\}=\\arg\\min\_\{t\\in\\\{1,\\dots,k\\\}\}\\left\\\|\\left\(\\bm\{I\}\-\\bm\{U\}\_\{t\}\\bm\{U\}\_\{t\}^\{\\mathrm\{T\}\}\\right\)\\bm\{z\}\_\{\\mathrm\{test\}\}\\right\\\|\_\{2\}^\{2\},\(52\)where𝑼t∈ℝn×n\\bm\{U\}\_\{t\}\\in\\mathbb\{R\}^\{n\\times n\}is the orthogonal matrix composed of left singular vectors of𝒁j\\bm\{Z\}\_\{j\}, the extracted features of training data in classjjby the feature extractor\. We evaluate the performance of three different feature extractors on the CIFAR\-10, CIFAR\-100\[[11](https://arxiv.org/html/2608.20668#bib.bib6)\], and CINIC\-10 datasets\[[6](https://arxiv.org/html/2608.20668#bib.bib25)\]\. The detailed information of these datasets is presented in Table[1](https://arxiv.org/html/2608.20668#S4.T1)\.

Table 1:Properties of the evaluated datasets\.### 4\.1Effect of Network Layers on Feature Visualization and Classification Accuracy

In this section, we investigate the effect of the number of iterative layersLLon the feature extraction capability of different models\. We consider two baseline models and our proposed method in the experiments\. To distinguish different convergence behaviors observed in the experiments, we consider two notions of convergence\. Objective convergence refers to the stage at which the MCR2objective becomes stable as the number of network layers increases, whereas classification\-accuracy convergence refers to the stage at which the classification accuracy reaches and remains near a stable level\. In general, classification\-accuracy convergence may occur earlier than objective convergence\.

#### 4\.1\.1CIFAR\-10 and CIFAR\-100

To ensure a fair comparison, ReduNet and AR\-ReduNet use the same dimension reduction module as LA\-ReduNet, ensuring that all three models receive identical input features during forward propagation\. In addition, for consistency, parameteraais set to 16 andμ\\muis set to10−310^\{\-3\}across all experiments222The choice ofμ=10−3\\mu=10^\{\-3\}is validated by ablation studies \(Table[3](https://arxiv.org/html/2608.20668#S4.T3)and Fig\.[16](https://arxiv.org/html/2608.20668#S4.F16)\), where it achieves a favorable balance between classification accuracy and feature representation quality compared to other tested values\.\. In our training configuration, the AdamW optimizer is employed with an initial learning rate of10−310^\{\-3\}and weight decay of10−210^\{\-2\}\. The learning rate is dynamically adjusted using the cosine annealing scheduler with a period of 50 epochs and a minimum learning rate of10−510^\{\-5\}\. All models are trained for a total of 30 epochs\. Meanwhile, the parameters of ReduNet and AR\-ReduNet include:ϵ=0\.3\\epsilon=0\.3,η=0\.5\\eta=0\.5and maximum number of iterative layersLmax=1000L\_\{\\rm\{\\max\}\}=1000\. The step sizeη=0\.5\\eta=0\.5is adopted following the original ReduNet setting, so as to retain the standard configuration of the baseline methods\[[5](https://arxiv.org/html/2608.20668#bib.bib3)\]\. For LA\-ReduNet, the parameters include:ϵ=0\.3\\epsilon=0\.3, the adaptive step\-size parametert0=0\.05t\_\{0\}=0\.05, thresholdτ=10−8\\tau=10^\{\-8\}, scaling coefficientβ=1\\beta=1, and maximum number of iterative layersLmax=1000L\_\{\\rm\{max\}\}=1000\. The thresholdτ=10−8\\tau=10^\{\-8\}is adopted to truncate samples whose Riemannian update direction norms have become negligible, while avoiding premature truncation of samples that still admit effective updates\. This practical choice is not intended to satisfy the global sufficient conditions in Theorem[1](https://arxiv.org/html/2608.20668#Thmtheorem1)\. Regarding the parametert0t\_\{0\}, a relatively small valuet0=0\.05t\_\{0\}=0\.05is adopted, rather than choosing an aggressively large value fort0t\_\{0\}to accelerate the iteration\.

For CIFAR\-10 and CIFAR\-100, as illustrated in Fig\.[6](https://arxiv.org/html/2608.20668#S4.F6), we observe that LA\-ReduNet attains its highest classification accuracy at approximately 5 layers and outperforms ReduNet and AR\-ReduNet\. By contrast, both ReduNet and AR\-ReduNet fail to achieve stable accuracy within the 50\-layer range\. Furthermore, as shown in Fig\.[7](https://arxiv.org/html/2608.20668#S4.F7), LA\-ReduNet achieves objective convergence at approximately 35 layers, whereas ReduNet and AR\-ReduNet require between 900 and 1000 layers to achieve objective convergence\. After objective convergence is reached, the MCR2objective values of LA\-ReduNet occasionally exhibit slight layer\-to\-layer decreases \(first occurring at layer 35 on CIFAR\-10 and layer 216 on CIFAR\-100\)\. These observations show that, although the experimental setting of LA\-ReduNet does not satisfy the sufficient conditions derived from global worst\-case bounds in Theorem[1](https://arxiv.org/html/2608.20668#Thmtheorem1), its MCR2objective remains stable in practice after convergence\. It should also be noted that AR\-ReduNet and LA\-ReduNet compute the coding rate using a modified multivariate Gaussian rate\-distortion approximation function, and therefore their MCR2values are not directly comparable to that of ReduNet\. According to the objective function \([14](https://arxiv.org/html/2608.20668#S2.E14)\), owing to the adoption ofα\\alphaandαj\\alpha\_\{j\}, the MCR2objective values solved by AR\-ReduNet and LA\-ReduNet are lower than those computed by ReduNet\.

![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_7.png)\(a\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_9.png)\(b\)

Figure 6:Trends of classification accuracy with the number of iterative layersLLfor ReduNet, AR\-ReduNet, and LA\-ReduNet: \(a\) CIFAR\-10; \(b\) CIFAR\-100\. Notice that the accuracy atL=0L=0corresponds to the result obtained by convolutional module\.![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_7_1.png)\(a\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_7_2.png)\(b\)

Figure 7:Curves of MCR2objective values during training: \(a\) CIFAR\-10; \(b\) CIFAR\-100\.Although the convolutional module improves classification accuracy on CIFAR\-10 and CIFAR\-100, Fig\.[8\(a\)](https://arxiv.org/html/2608.20668#S4.F8.sf1)and Fig\.[9\(a\)](https://arxiv.org/html/2608.20668#S4.F9.sf1)demonstrate that the features extracted by this module still fail to fully separate some classes\. As illustrated in Fig\.[8\(f\)](https://arxiv.org/html/2608.20668#S4.F8.sf6)and Fig\.[9\(f\)](https://arxiv.org/html/2608.20668#S4.F9.sf6), it can be observed that LA\-ReduNet effectively learns more discriminative features\. Compared with features obtained solely by convolutional modules, the features extracted by ReduNet \(L=1000L=1000\), AR\-ReduNet \(L=1000L=1000\), and LA\-ReduNet \(L=35L=35\) exhibit much clearer boundaries\. However, when the network has fewer layers, the features extracted by LA\-ReduNet are more discriminative than those learned by ReduNet and AR\-ReduNet\. This demonstrates that LA\-ReduNet can extract highly discriminative features with very few layers, greatly reducing the parameter storage relative to ReduNet and AR\-ReduNet\.

![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR10_Conv.png)\(a\)Conv only
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR10_35_RE.png)\(b\)Conv \+ ReduNet \(35 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR10_1000_RE.png)\(c\)Conv \+ ReduNet \(1000 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR10_35_AR.png)\(d\)Conv \+ AR\-ReduNet \(35 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR10_1000_AR.png)\(e\)Conv \+ AR\-ReduNet \(1000 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR10_35_LA.png)\(f\)Conv \+ LA\-ReduNet \(35 layers\)

Figure 8:t\-SNE visualization of learned features on the CIFAR\-10 dataset\.![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR100_Conv.png)\(a\)Conv only
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR100_35_RE.png)\(b\)Conv \+ ReduNet \(35 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR100_1000_RE.png)\(c\)Conv \+ ReduNet \(1000 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR100_35_AR.png)\(d\)Conv \+ AR\-ReduNet \(35 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR100_1000_AR.png)\(e\)Conv \+ AR\-ReduNet \(1000 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CIFAR100_35_LA.png)\(f\)Conv \+ LA\-ReduNet \(35 layers\)

Figure 9:t\-SNE visualization of learned features on the CIFAR\-100 dataset\.
#### 4\.1\.2CINIC\-10

Consistent with the experiments on CIFAR\-10 and CIFAR\-100, we evaluate the features extracted by ReduNet, AR\-ReduNet and LA\-ReduNet on CINIC\-10\. The hyperparameters are set to the values specified above\. As illustrated in Fig\.[10](https://arxiv.org/html/2608.20668#S4.F10), LA\-ReduNet achieves substantially faster classification\-accuracy convergence than both ReduNet and AR\-ReduNet, while also attaining a higher classification accuracy\. In particular, our method reaches an accuracy above 78\.6%\\%within only three iterations, whereas the baseline models remain at approximately 77%\\%even after 50 iterations\. In addition, Fig\.[11](https://arxiv.org/html/2608.20668#S4.F11)shows that LA\-ReduNet achieves objective convergence after approximately 35 layers\. Similarly, after objective convergence is reached, the MCR2objective values of LA\-ReduNet occasionally exhibit slight layer\-to\-layer decreases \(first occurring at layer 68\)\. In comparison, ReduNet and AR\-ReduNet require nearly 1000 layers to achieve objective convergence\. Furthermore, consistent with our experiments on other datasets, we conduct t\-SNE visualization for features extracted by all methods on the CINIC\-10 dataset to intuitively compare inter\-class separability and intra\-class compactness\. As illustrated in Fig\.[12](https://arxiv.org/html/2608.20668#S4.F12), similar to the experimental results on CIFAR\-10 and CIFAR\-100, LA\-ReduNet is capable of extracting features with superior separability\. Notably, unlike CIFAR\-10, CINIC\-10 contains both CIFAR\-10 images and downsampled ImageNet images\. Consequently, the features extracted solely by the convolutional layers exhibit weaker inter\-class separability\. Nevertheless, LA\-ReduNet can still transform these features into clearly separable representations using substantially fewer layers than ReduNet and AR\-ReduNet\.

![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_5.png)Figure 10:The trends of classification accuracy versus the number of iterative layers L for ReduNet, AR\-ReduNet, and LA\-ReduNet on the CINIC\-10 dataset\.![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_5_1.png)Figure 11:Curves of MCR2objective values during training on CINIC\-10\.![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CINIC_CONV.png)\(a\)Conv only
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CINIC_35_RE.png)\(b\)Conv \+ ReduNet \(35 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CINIC_1000_RE.png)\(c\)Conv \+ ReduNet \(1000 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CINIC_35_AR.png)\(d\)Conv \+ AR\-ReduNet \(35 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CINIC_1000_AR.png)\(e\)Conv \+ AR\-ReduNet \(1000 layers\)
![Refer to caption](https://arxiv.org/html/2608.20668v1/images/tsne_layer_CINIC_35_LA.png)\(f\)Conv \+ LA\-ReduNet \(35 layers\)

Figure 12:t\-SNE visualization of learned features on the CINIC\-10 dataset\.In summary, we compare the parameter storage of the unfolded modules of LA\-ReduNet, ReduNet, and AR\-ReduNet, while excluding the convolutional front\-end, which is shared by all three methods\. For each layer, the stored parameters correspond to the expansion and compression matrices defined in \([9](https://arxiv.org/html/2608.20668#S2.E9)\) and \([10](https://arxiv.org/html/2608.20668#S2.E10)\)\. In the experiments, since the common convolutional front\-end reduces the feature dimension todd, the corresponding matrices in the unfolded modules are of sized×dd\\times d, and the three methods have the same parameter storage per layer\. The scalar parametersα\\alphaandαj\\alpha\_\{j\}are recomputed from the current feature matrix𝒁\\bm\{Z\}at each layer and therefore do not need to be stored as model parameters\. As a result, the total parameter storage of the unfolded module is mainly determined by the number of layers\. As shown in Fig\.[7](https://arxiv.org/html/2608.20668#S4.F7)and Fig\.[11](https://arxiv.org/html/2608.20668#S4.F11), the MCR2objective of LA\-ReduNet reaches a stable value after approximately3535layers\. Under the original parameter settings of ReduNet and AR\-ReduNet, the two baseline methods require approximately10001000layers for their respective MCR2objectives to reach stable values\. Consequently, under these settings, the parameter storage of LA\-ReduNet is approximately35/1000≈1/2935/1000\\approx 1/29of that required by the corresponding unfolded baselines\. It is also worth noting that classification\-accuracy convergence is generally achieved earlier, typically within approximately 5–10 layers for LA\-ReduNet\.

### 4\.2Ablation Studies and Hyperparameter Analysis

![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_15_cifar10.png)\(a\)CIFAR\-10
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_15_cifar100.png)\(b\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_15_cinic.png)\(c\)CINIC\-10

Figure 13:Effect of different components on the convergence of the MCR2objective\.In this subsection, we first conduct component\-wise ablation studies to evaluate the effects of the Riemannian update, the normalization\-threshold \(NT\) mechanism, and the adaptive step\-size strategy on the objective convergence of LA\-ReduNet\. We then investigate the effects of the LA\-ReduNet parameterst0t\_\{0\}andβ\\betaon objective convergence, and analyze the influence of the loss\-weighting parameterμ\\muin the convolutional module on classification accuracy\.

Table 2:Configurations of the variants used in the component\-wise ablation study\.#### 4\.2\.1Component\-wise Ablation Study of LA\-ReduNet

To evaluate the contributions of the key components in LA\-ReduNet, we construct three variants by progressively introducing the NT mechanism and the adaptive step\-size strategy on top of the basic Riemannian update\. The configurations of these variants are summarized in Table[2](https://arxiv.org/html/2608.20668#S4.T2)\. As illustrated in Fig\.[13](https://arxiv.org/html/2608.20668#S4.F13), the basic Riemannian update without the NT mechanism exhibits considerably slower objective convergence\. Introducing the NT mechanism markedly improves the convergence behavior by reducing the dependence of the spherical angular progress on the tangent\-update magnitude, while the threshold retains the stopping information carried by this magnitude\. With the NT mechanism, the objective eventually reaches a similar level to that of the full LA\-ReduNet, while the adaptive step\-size strategy provides further improvement in the early layers\. These results indicate that the NT mechanism provides the primary improvement in objective convergence, whereas the adaptive step\-size strategy further improves the progress in the early layers\. Notice that a relatively conservative valueβ=1\\beta=1is adopted in these experiments, which limits the variation range of the adaptive step size and therefore results in relatively close convergence curves between LA\-ReduNet and the Riemannian \+ NT variant in Fig\.[13](https://arxiv.org/html/2608.20668#S4.F13)\. As further shown in the subsequent analysis ofβ\\beta, the advantage of the adaptive step\-size strategy becomes more pronounced when a larger value ofβ\\betais used\.

![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_11_cifar10.png)\(a\)CIFAR\-10
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_11_cifar100.png)\(b\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_11_cinic.png)\(c\)CINIC\-10

Figure 14:Effect of parametert0t\_\{0\}on the convergence of the MCR2objective of LA\-ReduNet\.
#### 4\.2\.2Impact of Hyperparametert0t\_\{0\}on the Objective Convergence of LA\-ReduNet

![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_12_cifar10.png)\(a\)CIFAR\-10
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_12_cifar100.png)\(b\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_12_cinic.png)\(c\)CINIC\-10

Figure 15:Effect of parameterβ\\betaon the convergence of the MCR2objective of LA\-ReduNet\.Since the adaptive stepttin \([21](https://arxiv.org/html/2608.20668#S3.E21)\) depends on botht0t\_\{0\}andβ\\beta, we first evaluate the influence oft0t\_\{0\}on objective convergence\. As shown in Fig\.[14](https://arxiv.org/html/2608.20668#S4.F14), we evaluate the effect of different values oft0t\_\{0\}on the objective convergence speed of LA\-ReduNet\. It can be observed that a relatively smallt0t\_\{0\}yields a higher MCR2objective value, which indicates better discriminability of the extracted features\. Nevertheless, an excessively smallt0t\_\{0\}such ast0=0\.01t\_\{0\}=0\.01increases the number of layersLLrequired to achieve objective convergence\. By contrast, whent0t\_\{0\}is excessively large, the MCR2objective converges to a lower value\. Overall, our results demonstrate that a moderate value oft0t\_\{0\}ranging from 0\.05 to 0\.1 provides a favorable trade\-off between objective convergence speed and the attained objective value, making it the most suitable choice\.

#### 4\.2\.3Impact of Hyperparameterβ\\betaon the Objective Convergence of LA\-ReduNet

Table 3:Ablation study on the weight coefficientμ\\muin terms of classification accuracy on CIFAR\-10, CIFAR\-100, and CINIC\-10\.![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_13_cifar10.png)\(a\)CIFAR\-10
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_13_cifar100.png)\(b\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2608.20668v1/Figure_13_cinic10.png)\(c\)CINIC\-10

Figure 16:Ablation study on the weight coefficientμ\\mu, illustrating how the final CE loss and MCR2objective value vary withμ\\mu\.In this subsection, we evaluate the effect of hyperparameterβ\\betaon the objective convergence speed of LA\-ReduNet\. As shown in Fig\.[15](https://arxiv.org/html/2608.20668#S4.F15), it can be observed thatβ\\betaeffectively accelerates the growth rate of the MCR2objective during the training of LA\-ReduNet, and this advantage is particularly prominent at shallow depths whereL≤15L\\leq 15\. Furthermore, we observe that increasing the value ofβ\\betabarely affects the peak value of MCR2that LA\-ReduNet can attain upon objective convergence\. In particular, withβ=5\\beta=5as a representative case, we compare the adaptive step\-size parameterttagainst two fixed step\-size settings,t0=0\.05t\_\{0\}=0\.05andt0=0\.3t\_\{0\}=0\.3, which respectively correspond to the lower and upper bounds ofttwhenβ=5\\beta=5\. As illustrated in Fig\.[15](https://arxiv.org/html/2608.20668#S4.F15), the adaptive strategy accelerates objective convergence relative to the smaller fixed step size\. In contrast, when compared to the larger fixed step size, the adaptive step\-size strategy not only achieves more stable optimization but also yields a higher peak value of the MCR2objective\.

In summary, LA\-ReduNet benefits from a largerβ\\betafor faster objective convergence, coupled with a relatively smallt0t\_\{0\}to safeguard the peak MCR2value\.

#### 4\.2\.4Ablation Study on the Influence ofμ\\muin the Convolutional Module

To investigate the impact of the hyperparameterμ\\muon the classification performance of the convolutional module, we tuneμ\\muwith all other experimental settings fixed, and evaluate the test set classification accuracy of extracted features under the NS classifier\. As shown in Table[3](https://arxiv.org/html/2608.20668#S4.T3)and Fig\.[16](https://arxiv.org/html/2608.20668#S4.F16), a lower CE loss is generally associated with higher classification accuracy\. Moreover, when the CE loss is kept nearly unchanged, increasing the MCR2objective value can further enhance classification performance\. In particular, forμ∈\[0,10−3\]\\mu\\in\[0,10^\{\-3\}\], the MCR2objective value increases while the CE loss remains almost constant, resulting in improved classification accuracy\. By contrast, whenμ=10−2\\mu=10^\{\-2\}, the CE loss becomes substantially higher, and the classification accuracy is markedly lower compared with settings using smallerμ\\mu\. Thus, we set10−310^\{\-3\}in the preceding experiments, as it increases the MCR2objective value without noticeably increasing the CE loss and consequently yields the best overall classification performance\.

## 5Conclusions and Discussions

In this paper, we propose LA\-ReduNet, a lightweight ReduNet architecture with an adaptive step\-size mechanism\. On the theoretical side, we establish a finite\-termination guarantee for the proposed Riemannian update scheme under explicit sufficient threshold and step\-size conditions\. Experimentally, compared with the gradient ascent\-based ReduNet and AR\-ReduNet, LA\-ReduNet requires substantially fewer layers for the MCR2objective to reach a stable value\. Simulation results demonstrate that the classification accuracy of LA\-ReduNet generally stabilizes within approximately 5–10 layers, while the MCR2objective reaches a stable value in approximately3535layers under the considered experimental settings\. We expect that LA\-ReduNet can be more effectively applied to a broader range of downstream tasks and can be integrated as a modular component into different neural network architectures\. Our experimental results further demonstrate that LA\-ReduNet, in conjunction with the convolutional module, yields features with superior classification accuracy and better\-separated class boundaries\.

## Appendices

The proof of Proposition[1](https://arxiv.org/html/2608.20668#Thmproposition1)relies on four supporting lemmas, whose detailed proofs are provided in Appendices A–D\. The detailed proof of Proposition[1](https://arxiv.org/html/2608.20668#Thmproposition1)is then presented in[E](https://arxiv.org/html/2608.20668#A5)\.

## Appendix APositive Definiteness and Uniform Eigenvalue Bounds

###### Lemma 1\.

For any𝐙∈ℝn×m\\bm\{Z\}\\in\\mathbb\{R\}^\{n\\times m\}with unit\-norm columns, letα∈\[0,1\]\\alpha\\in\[0,1\]satisfy

logdet\(α𝑰\+ntr⁡\(𝒁​𝒁T\)𝒁𝒁T\)=0\.\\displaystyle\\log\\det\\left\(\\alpha\\bm\{I\}\+\\frac\{n\}\{\\operatorname\{tr\}\(\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\)\}\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\\right\)=0\.\(53\)Then, the matrixα​𝐈\+n/tr⁡\(𝐙​𝐙T\)​𝐙​𝐙T\\alpha\\bm\{I\}\+n/\{\\rm tr\}\(\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\)\\bm\{ZZ\}^\{\\rm T\}is positive definite\. Moreover, all its eigenvalues satisfy

1n\+1≤λi≤n\+1,i=1,…,n\.\\displaystyle\\frac\{1\}\{n\+1\}\\leq\\lambda\_\{i\}\\leq n\+1,\\hskip 20\.00003pti=1,\\ldots,n\.\(54\)Hereλi\\lambda\_\{i\}denotes theii\-th eigenvalue of matrixα​𝐈\+n/tr⁡\(𝐙​𝐙T\)​𝐙​𝐙T\\alpha\\bm\{I\}\+n/\{\\rm tr\}\(\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\)\\bm\{ZZ\}^\{\\rm T\}\.

###### Proof\.

Letλi=α\+qi\\lambda\_\{i\}=\\alpha\+q\_\{i\}, whereqiq\_\{i\}denotes theii\-th eigenvalue ofn/tr⁡\(𝒁​𝒁T\)​𝒁​𝒁Tn/\{\\rm tr\}\(\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\)\\bm\{ZZ\}^\{\\rm T\}\. From \([53](https://arxiv.org/html/2608.20668#A1.E53)\) andtr⁡\(𝒁​𝒁T\)=m\{\\rm tr\}\(\\bm\{ZZ\}^\{\\rm T\}\)=m, it follows that

det\(α​𝑰\+nm​𝒁​𝒁T\)=∏i=1nλi=1\.\\displaystyle\\det\(\\alpha\\bm\{I\}\+\\frac\{n\}\{m\}\\bm\{ZZ\}^\{\\rm T\}\)=\\prod\_\{i=1\}^\{n\}\\lambda\_\{i\}=1\.\(55\)From the fact thatqi≤∑s=1nqs=nq\_\{i\}\\leq\\sum\_\{s=1\}^\{n\}q\_\{s\}=n, andα≤1\\alpha\\leq 1, we obtainλi≤n\+1\\lambda\_\{i\}\\leq n\+1\.

Moreover, sinceα≥0\\alpha\\geq 0andqi≥0q\_\{i\}\\geq 0, we haveλi≥0\\lambda\_\{i\}\\geq 0\. Together with∏i=1nλi=1\\prod\_\{i=1\}^\{n\}\\lambda\_\{i\}=1, this implies thatλi\>0\\lambda\_\{i\}\>0for alli=1,…,ni=1,\\ldots,n\. Forn=1n=1, \([53](https://arxiv.org/html/2608.20668#A1.E53)\) reduces tolog⁡\(α\+1\)=0\\log\(\\alpha\+1\)=0, which givesα=0\\alpha=0and henceλ1=1\\lambda\_\{1\}=1\. Therefore, the desired bound holds\. Forn≥2n\\geq 2, let

λr=min1≤i≤n⁡λi\.\\displaystyle\\lambda\_\{r\}=\\min\_\{1\\leq i\\leq n\}\\lambda\_\{i\}\.\(56\)Sinceλr=α\+qr\\lambda\_\{r\}=\\alpha\+q\_\{r\}andqr≥0q\_\{r\}\\geq 0, we have

α≤λr\.\\displaystyle\\alpha\\leq\\lambda\_\{r\}\.\(57\)Suppose, by contradiction, that

λr<1n\+1\.\\displaystyle\\lambda\_\{r\}<\\frac\{1\}\{n\+1\}\.\(58\)Since the sum of the eigenvalues of a matrix equals its trace and∑i=1nqi=n\\sum\_\{i=1\}^\{n\}q\_\{i\}=n, we have

∑i=1nλi=n​α\+∑i=1nqi=n⁡\(1\+α\)\.\\displaystyle\\sum\_\{i=1\}^\{n\}\\lambda\_\{i\}=n\\alpha\+\\sum\_\{i=1\}^\{n\}q\_\{i\}=n\(1\+\\alpha\)\.\(59\)Then, it follows fromα≤λr\\alpha\\leq\\lambda\_\{r\}that

∑i≠rλi\\displaystyle\\sum\_\{i\\neq r\}\\lambda\_\{i\}=n⁡\(1\+α\)−λr\\displaystyle=n\(1\+\\alpha\)\-\\lambda\_\{r\}\(60\)≤n\+\(n−1\)​λr\\displaystyle\\leq n\+\(n\-1\)\\lambda\_\{r\}\(61\)<n\+n−1n\+1\\displaystyle<n\+\\frac\{n\-1\}\{n\+1\}\(62\)<n\+1\.\\displaystyle<n\+1\.\(63\)By the arithmetic–geometric mean inequality,

∏i≠rλi\\displaystyle\\prod\_\{i\\neq r\}\\lambda\_\{i\}≤\(∑i≠rλin−1\)n−1\\displaystyle\\leq\\left\(\\frac\{\\sum\_\{i\\neq r\}\\lambda\_\{i\}\}\{n\-1\}\\right\)^\{n\-1\}\(64\)<\(n\+1n−1\)n−1\.\\displaystyle<\\left\(\\frac\{n\+1\}\{n\-1\}\\right\)^\{n\-1\}\.\(65\)
Moreover, for alln≥2n\\geq 2,

\(n\+1n−1\)n−1≤n\+1\.\\displaystyle\\left\(\\frac\{n\+1\}\{n\-1\}\\right\)^\{n\-1\}\\leq n\+1\.\(66\)Indeed, the inequality holds directly for2≤n≤62\\leq n\\leq 6\. Forn≥7n\\geq 7, using1\+x<ex1\+x<e^\{x\}forx\>0x\>0, we have

\(n\+1n−1\)n−1\\displaystyle\\left\(\\frac\{n\+1\}\{n\-1\}\\right\)^\{n\-1\}=\(1\+2n−1\)n−1\\displaystyle=\\left\(1\+\\frac\{2\}\{n\-1\}\\right\)^\{n\-1\}\(67\)<\(e2n−1\)n−1\\displaystyle<\\left\(e^\{\\frac\{2\}\{n\-1\}\}\\right\)^\{n\-1\}\(68\)=e2\\displaystyle=e^\{2\}\(69\)≤n\+1\.\\displaystyle\\leq n\+1\.\(70\)
Combining \([65](https://arxiv.org/html/2608.20668#A1.E65)\) and \([66](https://arxiv.org/html/2608.20668#A1.E66)\) yields

∏i≠rλi<n\+1\.\\displaystyle\\prod\_\{i\\neq r\}\\lambda\_\{i\}<n\+1\.\(71\)On the other hand, since∏i=1nλi=1\\prod\_\{i=1\}^\{n\}\\lambda\_\{i\}=1andλr<1/\(n\+1\)\\lambda\_\{r\}<1/\(n\+1\), we have

∏i≠rλi=1λr\>n\+1,\\displaystyle\\prod\_\{i\\neq r\}\\lambda\_\{i\}=\\frac\{1\}\{\\lambda\_\{r\}\}\>n\+1,\(72\)which is a contradiction\. Therefore,

λr≥1n\+1,\\displaystyle\\lambda\_\{r\}\\geq\\frac\{1\}\{n\+1\},\(73\)and hence

1n\+1≤λi≤n\+1,i=1,…,n\.\\displaystyle\\frac\{1\}\{n\+1\}\\leq\\lambda\_\{i\}\\leq n\+1,\\hskip 20\.00003pti=1,\\ldots,n\.\(74\)
Then,α​𝑰\+n/tr⁡\(𝒁​𝒁T\)​𝒁​𝒁T\\alpha\\bm\{I\}\+n/\{\\rm tr\}\(\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\)\\bm\{ZZ\}^\{\\rm T\}is positive definite, since all its eigenvalues are strictly positive\. ∎

Notice that we can similarly prove that the matrixαj​𝑰\+n/tr⁡\(𝒁​𝚷j​𝒁T\)​𝒁​𝚷j​𝒁T\\alpha\_\{j\}\\bm\{I\}\+n/\{\\rm tr\}\(\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\)\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}is positive definite, and its eigenvalues share the same upper and lower bounds as those ofα​𝑰\+n/tr⁡\(𝒁​𝒁T\)​𝒁​𝒁T\\alpha\\bm\{I\}\+n/\{\\rm tr\}\(\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\)\\bm\{ZZ\}^\{\\rm T\}\.

## Appendix BLipschitz Continuity of the Implicit Parameter Mapping

At each iteration,α\\alphaandαj\\alpha\_\{j\}are first recomputed from the current feature matrix𝒁\\bm\{Z\}and then treated as fixed quantities when computing the update direction with respect to𝒁\\bm\{Z\}\. To characterize how these recomputed parameters vary with𝒁\\bm\{Z\}across iterations, we denote their values byα⁡\(𝒁\)\\alpha\(\\bm\{Z\}\)andαj​\(𝒁j\)\\alpha\_\{j\}\(\\bm\{Z\}^\{j\}\), which are the unique solutions to \([15a](https://arxiv.org/html/2608.20668#S2.E15.1)\) and \([15b](https://arxiv.org/html/2608.20668#S2.E15.2)\), respectively\. The following lemma establishes the Lipschitz continuity of these parameter mappings\.

###### Lemma 2\.

Letℳ=\(𝕊n−1\)m\\mathcal\{M\}=\(\\mathbb\{S\}^\{n\-1\}\)^\{m\}denote the set of alln×mn\\times mmatrices with unit\-norm columns\. For any𝐙1,𝐙2∈ℳ\\bm\{Z\}\_\{1\},\\bm\{Z\}\_\{2\}\\in\\mathcal\{M\}, there exists a constantLα\>0L\_\{\\alpha\}\>0such that

\|α⁡\(𝒁1\)−α⁡\(𝒁2\)\|\\displaystyle\|\\alpha\(\\bm\{Z\}\_\{1\}\)\-\\alpha\(\\bm\{Z\}\_\{2\}\)\|≤Lα​‖𝒁1−𝒁2‖F,\\displaystyle\\leq L\_\{\\alpha\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\},\(75\)where

Lα=πm\.\\displaystyle L\_\{\\alpha\}=\\frac\{\\pi\}\{\\sqrt\{m\}\}\.\(76\)

###### Proof\.

Let𝒁⁡\(s\)\\bm\{Z\}\(s\)denote a minimizing geodesic curve onℳ\\mathcal\{M\}connecting𝒁1\\bm\{Z\}\_\{1\}and𝒁2\\bm\{Z\}\_\{2\},

𝒁⁡\(0\)=𝒁1,𝒁⁡\(1\)=𝒁2,\\displaystyle\\bm\{Z\}\(0\)=\\bm\{Z\}\_\{1\},\\hskip 20\.00003pt\\bm\{Z\}\(1\)=\\bm\{Z\}\_\{2\},\(77\)wheres∈\[0,1\]s\\in\[0,1\]\. Defineα⁡\(s\)=α⁡\(𝒁⁡\(s\)\)\\alpha\(s\)=\\alpha\(\\bm\{Z\}\(s\)\)\. Then, from the definition in \([15a](https://arxiv.org/html/2608.20668#S2.E15.1)\), we have

logdet\(α\(s\)𝑰\+ntr⁡\(𝒁⁡\(s\)​𝒁​\(s\)T\)𝒁\(s\)𝒁\(s\)T\)=0\.\\displaystyle\\log\\det\\left\(\\alpha\(s\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\)\}\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\\right\)=0\.\(78\)Define

𝑨⁡\(s\)\\displaystyle\\bm\{A\}\(s\)=α⁡\(s\)​𝑰\+ntr⁡\(𝒁⁡\(s\)​𝒁​\(s\)T\)​𝒁​\(s\)​𝒁​\(s\)T,\\displaystyle=\\alpha\(s\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\)\}\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\},\(79\)𝑩⁡\(s\)\\displaystyle\\bm\{B\}\(s\)=ntr⁡\(𝒁⁡\(s\)​𝒁​\(s\)T\)​𝒁​\(s\)​𝒁​\(s\)T\.\\displaystyle=\\frac\{n\}\{\{\\rm tr\}\(\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\)\}\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\.\(80\)
By Lemma[1](https://arxiv.org/html/2608.20668#Thmlemma1),𝑨⁡\(s\)\\bm\{A\}\(s\)is positive definite\. Since

∂𝑨⁡\(s\)∂α⁡\(s\)=𝑰,\\displaystyle\\frac\{\\partial\\bm\{A\}\(s\)\}\{\\partial\\alpha\(s\)\}=\\bm\{I\},\(81\)we have

∂∂α⁡\(s\)​log​det𝑨⁡\(s\)\\displaystyle\\frac\{\\partial\}\{\\partial\\alpha\(s\)\}\\log\\det\\bm\{A\}\(s\)=tr⁡\(𝑨​\(s\)−1\)\>0\.\\displaystyle=\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\>0\.\(82\)Since𝒁⁡\(s\)\\bm\{Z\}\(s\)varies smoothly along the geodesic and \([82](https://arxiv.org/html/2608.20668#A2.E82)\) shows that the partial derivative with respect toα⁡\(s\)\\alpha\(s\)is strictly positive, the unique solutionα⁡\(s\)\\alpha\(s\)varies differentiably withss\. Then, it follows from \([78](https://arxiv.org/html/2608.20668#A2.E78)\) thatlogdet𝑨\(s\)=0\\log\\det\\bm\{A\}\(s\)=0for alls∈\[0,1\]s\\in\[0,1\]\. Differentiatinglogdet𝑨\(s\)\\log\\det\\bm\{A\}\(s\)with respect tossyields

dd​s​log​det𝑨⁡\(s\)\\displaystyle\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}s\}\\log\\det\\bm\{A\}\(s\)=tr⁡\(𝑨​\(s\)−1​d​𝑨​\(s\)d​s\)\\displaystyle=\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\frac\{\\mathrm\{d\}\\bm\{A\}\(s\)\}\{\\mathrm\{d\}s\}\\right\)\(83\)=tr⁡\(𝑨​\(s\)−1​\(α′​\(s\)​𝑰\+𝑩′​\(s\)\)\)\\displaystyle=\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\(\\alpha^\{\\prime\}\(s\)\\bm\{I\}\+\\bm\{B\}^\{\\prime\}\(s\)\)\\right\)\(84\)=tr⁡\(𝑨​\(s\)−1​\(α′​\(s\)​𝑰\+nm​\(𝒁′​\(s\)​𝒁​\(s\)T\+𝒁⁡\(s\)​𝒁′​\(s\)T\)\)\)\\displaystyle=\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\(\\alpha^\{\\prime\}\(s\)\\bm\{I\}\+\\frac\{n\}\{m\}\(\\bm\{Z\}^\{\\prime\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\+\\bm\{Z\}\(s\)\\bm\{Z\}^\{\\prime\}\(s\)^\{\\rm T\}\)\)\\right\)\(85\)=α′​\(s\)​tr​\(𝑨​\(s\)−1\)\+nm​tr​\(𝑨​\(s\)−1​\(𝒁′​\(s\)​𝒁​\(s\)T\+𝒁⁡\(s\)​𝒁′​\(s\)T\)\)\\displaystyle=\\alpha^\{\\prime\}\(s\)\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\+\\frac\{n\}\{m\}\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\(\\bm\{Z\}^\{\\prime\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\+\\bm\{Z\}\(s\)\\bm\{Z\}^\{\\prime\}\(s\)^\{\\rm T\}\)\\right\)\(86\)=0,\\displaystyle=0,\(87\)where \([85](https://arxiv.org/html/2608.20668#A2.E85)\) follows fromtr⁡\(𝒁⁡\(s\)​𝒁​\(s\)T\)=tr⁡\(𝒁​\(s\)T​𝒁​\(s\)\)=m\{\\rm tr\}\(\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\)=\{\\rm tr\}\(\\bm\{Z\}\(s\)^\{\\rm T\}\\bm\{Z\}\(s\)\)=m\. Therefore,

α′​\(s\)\\displaystyle\\alpha^\{\\prime\}\(s\)=−nm​tr⁡\(𝑨​\(s\)−1​\(𝒁′​\(s\)​𝒁​\(s\)T\+𝒁⁡\(s\)​𝒁′​\(s\)T\)\)tr⁡\(𝑨​\(s\)−1\)\\displaystyle=\-\\frac\{n\}\{m\}\\frac\{\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\(\\bm\{Z\}^\{\\prime\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\+\\bm\{Z\}\(s\)\\bm\{Z\}^\{\\prime\}\(s\)^\{\\rm T\}\)\\right\)\}\{\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\(88\)α′​\(s\)\\displaystyle\\alpha^\{\\prime\}\(s\)=−2​nm​tr⁡\(𝒁​\(s\)T​𝑨​\(s\)−1​𝒁′​\(s\)\)tr⁡\(𝑨​\(s\)−1\)\.\\displaystyle=\-\\frac\{2n\}\{m\}\\frac\{\{\\rm tr\}\\left\(\\bm\{Z\}\(s\)^\{\\rm T\}\\bm\{A\}\(s\)^\{\-1\}\\bm\{Z\}^\{\\prime\}\(s\)\\right\)\}\{\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\.\(89\)By Lemma[1](https://arxiv.org/html/2608.20668#Thmlemma1),𝑨⁡\(s\)\\bm\{A\}\(s\)is positive definite\. Letλi​\(s\)\>0\\lambda\_\{i\}\(s\)\>0,i=1,…,ni=1,\\ldots,n, denote the eigenvalues of𝑨⁡\(s\)\\bm\{A\}\(s\)\. From \([78](https://arxiv.org/html/2608.20668#A2.E78)\), we have

∏i=1nλi​\(s\)=1\.\\displaystyle\\prod\_\{i=1\}^\{n\}\\lambda\_\{i\}\(s\)=1\.\(90\)Since the eigenvalues of𝑨​\(s\)−1\\bm\{A\}\(s\)^\{\-1\}are1/λi​\(s\)1/\\lambda\_\{i\}\(s\), the arithmetic–geometric mean inequality gives

tr⁡\(𝑨​\(s\)−1\)\\displaystyle\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)=∑i=1n1λi​\(s\)\\displaystyle=\\sum\_\{i=1\}^\{n\}\\frac\{1\}\{\\lambda\_\{i\}\(s\)\}≥n​\(∏i=1n1λi​\(s\)\)1/n\\displaystyle\\geq n\\left\(\\prod\_\{i=1\}^\{n\}\\frac\{1\}\{\\lambda\_\{i\}\(s\)\}\\right\)^\{1/n\}=n\.\\displaystyle=n\.\(91\)Moreover, we obtain

\|tr⁡\(𝒁​\(s\)T​𝑨​\(s\)−1​𝒁′​\(s\)\)\|\\displaystyle\|\{\\rm tr\}\\left\(\\bm\{Z\}\(s\)^\{\\rm T\}\\bm\{A\}\(s\)^\{\-1\}\\bm\{Z\}^\{\\prime\}\(s\)\\right\)\|≤‖𝑨​\(s\)−1​𝒁​\(s\)‖F​‖𝒁′​\(s\)‖F\\displaystyle\\leq\\\|\\bm\{A\}\(s\)^\{\-1\}\\bm\{Z\}\(s\)\\\|\_\{F\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(92\)=\[tr⁡\(𝒁​\(s\)T​𝑨​\(s\)−2​𝒁​\(s\)\)\]1/2​‖𝒁′​\(s\)‖F\\displaystyle=\\left\[\{\\rm tr\}\\left\(\\bm\{Z\}\(s\)^\{\\rm T\}\\bm\{A\}\(s\)^\{\-2\}\\bm\{Z\}\(s\)\\right\)\\right\]^\{1/2\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(93\)=\[tr⁡\(𝑨​\(s\)−2​𝒁​\(s\)​𝒁​\(s\)T\)\]1/2​‖𝒁′​\(s\)‖F\\displaystyle=\\left\[\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-2\}\\bm\{Z\}\(s\)\\bm\{Z\}\(s\)^\{\\rm T\}\\right\)\\right\]^\{1/2\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(94\)=\[mn​tr​\(𝑨​\(s\)−2​\(𝑨⁡\(s\)−α⁡\(s\)​𝑰\)\)\]1/2​‖𝒁′​\(s\)‖F\\displaystyle=\\left\[\\frac\{m\}\{n\}\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-2\}\\left\(\\bm\{A\}\(s\)\-\\alpha\(s\)\\bm\{I\}\\right\)\\right\)\\right\]^\{1/2\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(95\)=\{mn​\[tr⁡\(𝑨​\(s\)−1\)−α⁡\(s\)​tr​\(𝑨​\(s\)−2\)\]\}1/2​‖𝒁′​\(s\)‖F\\displaystyle=\\left\\\{\\frac\{m\}\{n\}\\left\[\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\-\\alpha\(s\)\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-2\}\\right\)\\right\]\\right\\\}^\{1/2\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(96\)≤mn​tr​\(𝑨​\(s\)−1\)​‖𝒁′​\(s\)‖F\.\\displaystyle\\leq\\sqrt\{\\frac\{m\}\{n\}\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\.\(97\)
Step \([92](https://arxiv.org/html/2608.20668#A2.E92)\) follows from the Cauchy\-Schwarz inequality for the Frobenius inner product\. Step \([93](https://arxiv.org/html/2608.20668#A2.E93)\) follows from the definition of the Frobenius norm and the symmetry of𝑨​\(s\)−1\\bm\{A\}\(s\)^\{\-1\}\. Step \([97](https://arxiv.org/html/2608.20668#A2.E97)\) follows fromα⁡\(s\)≥0\\alpha\(s\)\\geq 0andtr⁡\(𝑨​\(s\)−2\)≥0\{\\rm tr\}\(\\bm\{A\}\(s\)^\{\-2\}\)\\geq 0\. Thus, combining \([91](https://arxiv.org/html/2608.20668#A2.E91)\) and \([97](https://arxiv.org/html/2608.20668#A2.E97)\), we obtain an upper bound for\|α′​\(s\)\|\|\\alpha^\{\\prime\}\(s\)\|as follows:

\|α′​\(s\)\|\\displaystyle\|\\alpha^\{\\prime\}\(s\)\|=2​nm​\|tr⁡\(𝒁​\(s\)T​𝑨​\(s\)−1​𝒁′​\(s\)\)\|tr⁡\(𝑨​\(s\)−1\)\\displaystyle=\\frac\{2n\}\{m\}\\frac\{\|\{\\rm tr\}\\left\(\\bm\{Z\}\(s\)^\{\\rm T\}\\bm\{A\}\(s\)^\{\-1\}\\bm\{Z\}^\{\\prime\}\(s\)\\right\)\|\}\{\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\(98\)≤2​nm​mn​tr​\(𝑨​\(s\)−1\)tr⁡\(𝑨​\(s\)−1\)​‖𝒁′​\(s\)‖F\\displaystyle\\leq\\frac\{2n\}\{m\}\\frac\{\\sqrt\{\\frac\{m\}\{n\}\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\}\{\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(99\)=2​nm​1tr⁡\(𝑨​\(s\)−1\)​‖𝒁′​\(s\)‖F\\displaystyle=2\\sqrt\{\\frac\{n\}\{m\}\}\\frac\{1\}\{\\sqrt\{\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(100\)≤2m​‖𝒁′​\(s\)‖F\.\\displaystyle\\leq\\frac\{2\}\{\\sqrt\{m\}\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\.\(101\)The above result bounds the derivative ofα⁡\(s\)\\alpha\(s\)along the geodesic curve\. Therefore, by integrating overss, we obtain

α⁡\(𝒁2\)−α⁡\(𝒁1\)\\displaystyle\\alpha\(\\bm\{Z\}\_\{2\}\)\-\\alpha\(\\bm\{Z\}\_\{1\}\)=∫01α′​\(s\)​𝑑s\\displaystyle=\\int\_\{0\}^\{1\}\\alpha^\{\\prime\}\(s\)\\mathrm\{d\}s\(102\)\|α⁡\(𝒁1\)−α⁡\(𝒁2\)\|\\displaystyle\|\\alpha\(\\bm\{Z\}\_\{1\}\)\-\\alpha\(\\bm\{Z\}\_\{2\}\)\|=\|∫01α′​\(s\)​𝑑s\|\\displaystyle=\\left\|\\int\_\{0\}^\{1\}\\alpha^\{\\prime\}\(s\)\\mathrm\{d\}s\\right\|\(103\)≤∫01\|α′​\(s\)\|​𝑑s\\displaystyle\\leq\\int\_\{0\}^\{1\}\|\\alpha^\{\\prime\}\(s\)\|\\mathrm\{d\}s\(104\)≤2m​∫01‖𝒁′​\(s\)‖F​𝑑s\\displaystyle\\leq\\frac\{2\}\{\\sqrt\{m\}\}\\int\_\{0\}^\{1\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\\mathrm\{d\}s\(105\)=2m​dℳ​\(𝒁1,𝒁2\),\\displaystyle=\\frac\{2\}\{\\sqrt\{m\}\}d\_\{\\mathcal\{M\}\}\(\\bm\{Z\}\_\{1\},\\bm\{Z\}\_\{2\}\),\(106\)wheredℳ​\(𝒁1,𝒁2\)d\_\{\\mathcal\{M\}\}\(\\bm\{Z\}\_\{1\},\\bm\{Z\}\_\{2\}\)denotes the geodesic distance on the product manifoldℳ=\(𝕊n−1\)m\\mathcal\{M\}=\(\\mathbb\{S\}^\{n\-1\}\)^\{m\}\. Step \([104](https://arxiv.org/html/2608.20668#A2.E104)\) follows from the triangle inequality for integrals\. Sinceℳ=\(𝕊n−1\)m\\mathcal\{M\}=\(\\mathbb\{S\}^\{n\-1\}\)^\{m\}is a product of unit spheres, the geodesic distance can be bounded by the Euclidean distance as

dℳ​\(𝒁1,𝒁2\)≤π2​‖𝒁1−𝒁2‖F\.\\displaystyle d\_\{\\mathcal\{M\}\}\(\\bm\{Z\}\_\{1\},\\bm\{Z\}\_\{2\}\)\\leq\\frac\{\\pi\}\{2\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(107\)Thus,

\|α⁡\(𝒁1\)−α⁡\(𝒁2\)\|\\displaystyle\|\\alpha\(\\bm\{Z\}\_\{1\}\)\-\\alpha\(\\bm\{Z\}\_\{2\}\)\|≤πm​‖𝒁1−𝒁2‖F\.\\displaystyle\\leq\\frac\{\\pi\}\{\\sqrt\{m\}\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(108\)By setting

Lα=πm,\\displaystyle L\_\{\\alpha\}=\\frac\{\\pi\}\{\\sqrt\{m\}\},\(109\)the conclusion is proved\. ∎

In addition, analogous results can be derived for the parameterαj​\(𝒁j\)\\alpha\_\{j\}\(\\bm\{Z\}^\{j\}\)\. That is,

\|αj​\(𝒁1j\)−αj​\(𝒁2j\)\|≤πmj​‖𝒁1j−𝒁2j‖F,\\displaystyle\|\\alpha\_\{j\}\(\\bm\{Z\}\_\{1\}^\{j\}\)\-\\alpha\_\{j\}\(\\bm\{Z\}\_\{2\}^\{j\}\)\|\\leq\\frac\{\\pi\}\{\\sqrt\{m\_\{j\}\}\}\\\|\\bm\{Z\}\_\{1\}^\{j\}\-\\bm\{Z\}\_\{2\}^\{j\}\\\|\_\{F\},\(110\)where𝒁1j,𝒁2j∈ℝn×mj\\bm\{Z\}\_\{1\}^\{j\},\\bm\{Z\}\_\{2\}^\{j\}\\in\\mathbb\{R\}^\{n\\times m\_\{j\}\}, andmjm\_\{j\}denotes the number of samples in thejj\-th class\. It follows that the constant term satisfies

Lαj=πmj\.\\displaystyle L\_\{\\alpha\_\{j\}\}=\\frac\{\\pi\}\{\\sqrt\{m\_\{j\}\}\}\.\(111\)Since𝚷j\\bm\{\\Pi\}\_\{j\}retains the columns corresponding to thejj\-th class and sets all other columns to zero, we have

‖𝒁1j−𝒁2j‖F\\displaystyle\\\|\\bm\{Z\}\_\{1\}^\{j\}\-\\bm\{Z\}\_\{2\}^\{j\}\\\|\_\{F\}=‖\(𝒁1−𝒁2\)​𝚷j‖F\\displaystyle=\\\|\(\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\)\\bm\{\\Pi\}\_\{j\}\\\|\_\{F\}\(112\)≤‖𝒁1−𝒁2‖F​‖𝚷j‖2\\displaystyle\\leq\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\\\|\\bm\{\\Pi\}\_\{j\}\\\|\_\{2\}\(113\)=‖𝒁1−𝒁2‖F,\\displaystyle=\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\},\(114\)where \([114](https://arxiv.org/html/2608.20668#A2.E114)\) follows from‖𝚷j‖2=1\\\|\\bm\{\\Pi\}\_\{j\}\\\|\_\{2\}=1\. Therefore,

\|αj​\(𝒁1j\)−αj​\(𝒁2j\)\|≤πmj​‖𝒁1−𝒁2‖F\.\\displaystyle\|\\alpha\_\{j\}\(\\bm\{Z\}\_\{1\}^\{j\}\)\-\\alpha\_\{j\}\(\\bm\{Z\}\_\{2\}^\{j\}\)\|\\leq\\frac\{\\pi\}\{\\sqrt\{m\_\{j\}\}\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(115\)
Furthermore, by applying the same argument to thejj\-th class, the derivative of the corresponding implicit parameter along the geodesic satisfies

\|αj′​\(s\)\|≤2mj​‖\(𝒁j\)′​\(s\)‖F≤2mj​‖𝒁′​\(s\)‖F\.\\displaystyle\|\\alpha\_\{j\}^\{\\prime\}\(s\)\|\\leq\\frac\{2\}\{\\sqrt\{m\_\{j\}\}\}\\\|\(\\bm\{Z\}^\{j\}\)^\{\\prime\}\(s\)\\\|\_\{F\}\\leq\\frac\{2\}\{\\sqrt\{m\_\{j\}\}\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\.\(116\)

## Appendix CLipschitz Continuity of the Inverse Matrices in the MCR2Objective

Based on Lemma[2](https://arxiv.org/html/2608.20668#Thmlemma2), we further establish the Lipschitz continuity of the inverse matrices appearing in the MCR2update with respect to𝒁\\bm\{Z\}\.

###### Lemma 3\.

For any𝐙1,𝐙2∈ℳ\\bm\{Z\}\_\{1\},\\bm\{Z\}\_\{2\}\\in\\mathcal\{M\}, whereℳ=\(𝕊n−1\)m\\mathcal\{M\}=\(\\mathbb\{S\}^\{n\-1\}\)^\{m\}, there exists a constantLinv\>0L\_\{\\rm inv\}\>0such that

‖\(α⁡\(𝒁1\)​𝑰\+nm​ϵ2​𝒁1​𝒁1T\)−1−\(α⁡\(𝒁2\)​𝑰\+nm​ϵ2​𝒁2​𝒁2T\)−1‖F\\displaystyle\\left\\\|\\left\(\\alpha\(\\bm\{Z\}\_\{1\}\)\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{Z\}\_\{1\}\\bm\{Z\}\_\{1\}^\{\\rm T\}\\right\)^\{\-1\}\-\\left\(\\alpha\(\\bm\{Z\}\_\{2\}\)\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{Z\}\_\{2\}\\bm\{Z\}\_\{2\}^\{\\rm T\}\\right\)^\{\-1\}\\right\\\|\_\{F\}≤Linv​‖𝒁1−𝒁2‖F,\\displaystyle\\hskip 20\.00003pt\\leq L\_\{\\rm inv\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\},\(117\)where

Linv=\(n\+1\)2m​\(π​n\+2​nϵ2\)\.\\displaystyle L\_\{\\rm inv\}=\\frac\{\(n\+1\)^\{2\}\}\{\\sqrt\{m\}\}\\left\(\\pi\\sqrt\{n\}\+\\frac\{2n\}\{\\epsilon^\{2\}\}\\right\)\.\(118\)

###### Proof\.

Let

𝑸1\\displaystyle\\bm\{Q\}\_\{1\}=α⁡\(𝒁1\)​𝑰\+nm​ϵ2​𝒁1​𝒁1T,\\displaystyle=\\alpha\(\\bm\{Z\}\_\{1\}\)\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{Z\}\_\{1\}\\bm\{Z\}\_\{1\}^\{\\rm T\},\(119\)𝑸2\\displaystyle\\bm\{Q\}\_\{2\}=α⁡\(𝒁2\)​𝑰\+nm​ϵ2​𝒁2​𝒁2T\.\\displaystyle=\\alpha\(\\bm\{Z\}\_\{2\}\)\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{Z\}\_\{2\}\\bm\{Z\}\_\{2\}^\{\\rm T\}\.\(120\)Using the identity for the difference of two inverse matrices and the norm inequality for matrix products, we obtain

‖𝑸1−1−𝑸2−1‖F\\displaystyle\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\\\|\_\{F\}=‖𝑸1−1​\(𝑸2−𝑸1\)​𝑸2−1‖F\\displaystyle=\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\(\\bm\{Q\}\_\{2\}\-\\bm\{Q\}\_\{1\}\)\\bm\{Q\}\_\{2\}^\{\-1\}\\\|\_\{F\}\(121\)≤‖𝑸1−1‖2​‖𝑸2−𝑸1‖F​‖𝑸2−1‖2\.\\displaystyle\\leq\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\\|\_\{2\}\\\|\\bm\{Q\}\_\{2\}\-\\bm\{Q\}\_\{1\}\\\|\_\{F\}\\\|\\bm\{Q\}\_\{2\}^\{\-1\}\\\|\_\{2\}\.\(122\)The eigenvalues of𝑸i\\bm\{Q\}\_\{i\}are bounded below by a positive constant\. Specifically,

𝑸i−\(α⁡\(𝒁i\)​𝑰\+ntr⁡\(𝒁i​𝒁iT\)​𝒁i​𝒁iT\)\\displaystyle\\bm\{Q\}\_\{i\}\-\\left\(\\alpha\(\\bm\{Z\}\_\{i\}\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\)\}\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\\right\)=\(nm​ϵ2−nm\)​𝒁i​𝒁iT\\displaystyle=\\left\(\\frac\{n\}\{m\\epsilon^\{2\}\}\-\\frac\{n\}\{m\}\\right\)\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\(123\)=nm​\(1ϵ2−1\)​𝒁i​𝒁iT\.\\displaystyle=\\frac\{n\}\{m\}\\left\(\\frac\{1\}\{\\epsilon^\{2\}\}\-1\\right\)\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\.\(124\)Since0<ϵ2<tr⁡\(𝒁i​𝒁iT\)/m=10<\\epsilon^\{2\}<\{\\rm tr\}\(\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\)/m=1and𝒁i​𝒁iT⪰0\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\\succeq 0, we have

nm​\(1ϵ2−1\)​𝒁i​𝒁iT⪰0\.\\displaystyle\\frac\{n\}\{m\}\\left\(\\frac\{1\}\{\\epsilon^\{2\}\}\-1\\right\)\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\\succeq 0\.\(125\)Thus,

𝑸i⪰\(α⁡\(𝒁i\)​𝑰\+ntr⁡\(𝒁i​𝒁iT\)​𝒁i​𝒁iT\)\.\\displaystyle\\bm\{Q\}\_\{i\}\\succeq\\left\(\\alpha\(\\bm\{Z\}\_\{i\}\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\)\}\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\\right\)\.\(126\)By Lemma[1](https://arxiv.org/html/2608.20668#Thmlemma1)and \([126](https://arxiv.org/html/2608.20668#A3.E126)\), we have

λmin​\(𝑸i\)\\displaystyle\\lambda\_\{\\min\}\(\\bm\{Q\}\_\{i\}\)≥λmin​\(α⁡\(𝒁i\)​𝑰\+ntr⁡\(𝒁i​𝒁iT\)​𝒁i​𝒁iT\)\\displaystyle\\geq\\lambda\_\{\\min\}\\left\(\\alpha\(\\bm\{Z\}\_\{i\}\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\)\}\\bm\{Z\}\_\{i\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\\right\)\(127\)≥1n\+1\.\\displaystyle\\geq\\frac\{1\}\{n\+1\}\.\(128\)Then, it follows that

∥𝑸i−1∥2≤n\+1,i=1,2\.\\displaystyle\\\|\\bm\{Q\}\_\{i\}^\{\-1\}\\\|\_\{2\}\\leq n\+1,\\hskip 10\.00002pti=1,2\.\(129\)It remains to bound‖𝑸2−𝑸1‖F\\\|\\bm\{Q\}\_\{2\}\-\\bm\{Q\}\_\{1\}\\\|\_\{F\}\. We have

𝑸2−𝑸1=\\displaystyle\\bm\{Q\}\_\{2\}\-\\bm\{Q\}\_\{1\}=\(α⁡\(𝒁2\)−α⁡\(𝒁1\)\)​𝑰\+nm​ϵ2​\(𝒁2​𝒁2T−𝒁1​𝒁1T\)\.\\displaystyle\(\\alpha\(\\bm\{Z\}\_\{2\}\)\-\\alpha\(\\bm\{Z\}\_\{1\}\)\)\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\(\\bm\{Z\}\_\{2\}\\bm\{Z\}\_\{2\}^\{\\rm T\}\-\\bm\{Z\}\_\{1\}\\bm\{Z\}\_\{1\}^\{\\rm T\}\)\.\(130\)For the first term, Lemma[2](https://arxiv.org/html/2608.20668#Thmlemma2)gives

‖\(α⁡\(𝒁2\)−α⁡\(𝒁1\)\)​𝑰‖F\\displaystyle\\\|\(\\alpha\(\\bm\{Z\}\_\{2\}\)\-\\alpha\(\\bm\{Z\}\_\{1\}\)\)\\bm\{I\}\\\|\_\{F\}=n​\|α⁡\(𝒁2\)−α⁡\(𝒁1\)\|\\displaystyle=\\sqrt\{n\}\|\\alpha\(\\bm\{Z\}\_\{2\}\)\-\\alpha\(\\bm\{Z\}\_\{1\}\)\|\(131\)≤n​Lα​‖𝒁2−𝒁1‖F\.\\displaystyle\\leq\\sqrt\{n\}L\_\{\\alpha\}\\\|\\bm\{Z\}\_\{2\}\-\\bm\{Z\}\_\{1\}\\\|\_\{F\}\.\(132\)For the second term, we have

‖𝒁2​𝒁2T−𝒁1​𝒁1T‖F\\displaystyle\\\|\\bm\{Z\}\_\{2\}\\bm\{Z\}\_\{2\}^\{\\rm T\}\-\\bm\{Z\}\_\{1\}\\bm\{Z\}\_\{1\}^\{\\rm T\}\\\|\_\{F\}=‖\(𝒁2−𝒁1\)​𝒁2T\+𝒁1​\(𝒁2−𝒁1\)T‖F\\displaystyle=\\\|\(\\bm\{Z\}\_\{2\}\-\\bm\{Z\}\_\{1\}\)\\bm\{Z\}\_\{2\}^\{\\rm T\}\+\\bm\{Z\}\_\{1\}\(\\bm\{Z\}\_\{2\}\-\\bm\{Z\}\_\{1\}\)^\{\\rm T\}\\\|\_\{F\}\(133\)≤‖\(𝒁2−𝒁1\)​𝒁2T‖F\+‖𝒁1​\(𝒁2−𝒁1\)T‖F\\displaystyle\\leq\\\|\(\\bm\{Z\}\_\{2\}\-\\bm\{Z\}\_\{1\}\)\\bm\{Z\}\_\{2\}^\{\\rm T\}\\\|\_\{F\}\+\\\|\\bm\{Z\}\_\{1\}\(\\bm\{Z\}\_\{2\}\-\\bm\{Z\}\_\{1\}\)^\{\\rm T\}\\\|\_\{F\}\(134\)≤2​m​‖𝒁2−𝒁1‖F,\\displaystyle\\leq 2\\sqrt\{m\}\\\|\\bm\{Z\}\_\{2\}\-\\bm\{Z\}\_\{1\}\\\|\_\{F\},\(135\)where the unit\-norm column constraint is used\. Therefore, combining \([132](https://arxiv.org/html/2608.20668#A3.E132)\) and \([135](https://arxiv.org/html/2608.20668#A3.E135)\), it follows that

‖𝑸2−𝑸1‖F\\displaystyle\\\|\\bm\{Q\}\_\{2\}\-\\bm\{Q\}\_\{1\}\\\|\_\{F\}≤‖\(α⁡\(𝒁2\)−α⁡\(𝒁1\)\)​𝑰‖F\+nm​ϵ2​‖\(𝒁2​𝒁2T−𝒁1​𝒁1T\)‖F\\displaystyle\\leq\\\|\(\\alpha\(\\bm\{Z\}\_\{2\}\)\-\\alpha\(\\bm\{Z\}\_\{1\}\)\)\\bm\{I\}\\\|\_\{F\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\\|\(\\bm\{Z\}\_\{2\}\\bm\{Z\}\_\{2\}^\{\\rm T\}\-\\bm\{Z\}\_\{1\}\\bm\{Z\}\_\{1\}^\{\\rm T\}\)\\\|\_\{F\}\(136\)≤\(n​Lα\+2​n​mm​ϵ2\)​‖𝒁2−𝒁1‖F\.\\displaystyle\\leq\\left\(\\sqrt\{n\}L\_\{\\alpha\}\+\\frac\{2n\\sqrt\{m\}\}\{m\\epsilon^\{2\}\}\\right\)\\\|\\bm\{Z\}\_\{2\}\-\\bm\{Z\}\_\{1\}\\\|\_\{F\}\.\(137\)Finally, combining \([122](https://arxiv.org/html/2608.20668#A3.E122)\), \([129](https://arxiv.org/html/2608.20668#A3.E129)\), and \([137](https://arxiv.org/html/2608.20668#A3.E137)\), it can be obtained that

‖𝑸1−1−𝑸2−1‖F\\displaystyle\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\\\|\_\{F\}≤\(n\+1\)2​\(n​Lα\+2​n​mm​ϵ2\)​‖𝒁1−𝒁2‖F\\displaystyle\\leq\(n\+1\)^\{2\}\\left\(\\sqrt\{n\}L\_\{\\alpha\}\+\\frac\{2n\\sqrt\{m\}\}\{m\\epsilon^\{2\}\}\\right\)\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\(138\)=\(n\+1\)2m​\(π​n\+2​nϵ2\)​‖𝒁1−𝒁2‖F\\displaystyle=\\frac\{\(n\+1\)^\{2\}\}\{\\sqrt\{m\}\}\\left\(\\pi\\sqrt\{n\}\+\\frac\{2n\}\{\\epsilon^\{2\}\}\\right\)\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\(139\)=Linv​‖𝒁1−𝒁2‖F\.\\displaystyle=L\_\{\\rm inv\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(140\)whereLinvL\_\{\\rm inv\}is defined as in \([118](https://arxiv.org/html/2608.20668#A3.E118)\)\. This completes the proof\. ∎

Similarly, applying Lemma[3](https://arxiv.org/html/2608.20668#Thmlemma3)to the class\-wise submatrices and using‖𝒁1j−𝒁2j‖F≤‖𝒁1−𝒁2‖F\\\|\\bm\{Z\}\_\{1\}^\{j\}\-\\bm\{Z\}\_\{2\}^\{j\}\\\|\_\{F\}\\leq\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}, we obtain

‖\(αj​\(𝒁1j\)​𝑰\+ntr⁡\(𝚷j\)​ϵ2​𝒁1​𝚷j​𝒁1T\)−1−\(αj​\(𝒁2j\)​𝑰\+ntr⁡\(𝚷j\)​ϵ2​𝒁2​𝚷j​𝒁2T\)−1‖F\\displaystyle\\left\\\|\\left\(\\alpha\_\{j\}\(\\bm\{Z\}\_\{1\}^\{j\}\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\\epsilon^\{2\}\}\\bm\{Z\}\_\{1\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}\_\{1\}^\{\\rm T\}\\right\)^\{\-1\}\-\\left\(\\alpha\_\{j\}\(\\bm\{Z\}\_\{2\}^\{j\}\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\\epsilon^\{2\}\}\\bm\{Z\}\_\{2\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}\_\{2\}^\{\\rm T\}\\right\)^\{\-1\}\\right\\\|\_\{F\}≤Linvj​‖𝒁1−𝒁2‖F\.\\displaystyle\\hskip 20\.00003pt\\leq L^\{j\}\_\{\{\\rm inv\}\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(141\)where

Linvj=\(n\+1\)2mj​\(π​n\+2​nϵ2\)\.\\displaystyle L^\{j\}\_\{\{\\rm inv\}\}=\\frac\{\(n\+1\)^\{2\}\}\{\\sqrt\{m\_\{j\}\}\}\\left\(\\pi\\sqrt\{n\}\+\\frac\{2n\}\{\\epsilon^\{2\}\}\\right\)\.\(142\)The above results establish the Lipschitz continuity of the inverse\-matrix mappings induced by the recomputed parametersα⁡\(𝒁\)\\alpha\(\\bm\{Z\}\)andαj​\(𝒁j\)\\alpha\_\{j\}\(\\bm\{Z\}^\{j\}\)with respect to𝒁\\bm\{Z\}\.

## Appendix DLipschitz Continuity of the Riemannian Update Mapping

Based on Lemmas[1](https://arxiv.org/html/2608.20668#Thmlemma1)and[3](https://arxiv.org/html/2608.20668#Thmlemma3), the Lipschitz continuity of the Riemannian update mapping on the product manifold is established\. At each𝒁\\bm\{Z\}, the parametersα⁡\(𝒁\)\\alpha\(\\bm\{Z\}\)andαj​\(𝒁j\)\\alpha\_\{j\}\(\\bm\{Z\}^\{j\}\)are first recomputed and then treated as fixed when differentiating the MCR2objective with respect to𝒁\\bm\{Z\}\. The resulting Euclidean update direction is given by

𝑮⁡\(𝒁\)=\\displaystyle\\bm\{G\}\(\\bm\{Z\}\)=nm​ϵ2​\(α⁡\(𝒁\)​𝑰\+nm​ϵ2​𝒁​𝒁T\)−1​𝒁\\displaystyle\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\alpha\(\\bm\{Z\}\)\\bm\{I\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\bm\{Z\}\\bm\{Z\}^\{\\rm T\}\\right\)^\{\-1\}\\bm\{Z\}−∑j=1knm​ϵ2\(αj\(𝒁j\)𝑰\+ntr⁡\(𝚷j\)​ϵ2𝒁𝚷j𝒁T\)−1𝒁𝚷j\.\\displaystyle\-\\sum\_\{j=1\}^\{k\}\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\alpha\_\{j\}\(\\bm\{Z\}^\{j\}\)\\bm\{I\}\+\\frac\{n\}\{\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\\epsilon^\{2\}\}\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}^\{\\rm T\}\\right\)^\{\-1\}\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\.\(143\)This yields the following lemma\.

###### Lemma 4\.

For any𝐙1,𝐙2∈ℳ=\(𝕊n−1\)m\\bm\{Z\}\_\{1\},\\bm\{Z\}\_\{2\}\\in\\mathcal\{M\}=\(\\mathbb\{S\}^\{n\-1\}\)^\{m\}, there exists a constantLgrad\>0L\_\{\\rm grad\}\>0such that

‖𝑮T​\(𝒁1\)−𝑮T​\(𝒁2\)‖F≤Lgrad​‖𝒁1−𝒁2‖F,\\displaystyle\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}\\leq L\_\{\\rm grad\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\},\(144\)where𝐆T​\(𝐙1\),𝐆T​\(𝐙2\)\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{1\}\),\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{2\}\)denote the Riemannian update directions at𝐙1\\bm\{Z\}\_\{1\}and𝐙2\\bm\{Z\}\_\{2\}, respectively\.

###### Proof\.

The Riemannian update direction on the product of spheres is obtained by projecting the Euclidean update direction onto the tangent space\. It follows from \([17](https://arxiv.org/html/2608.20668#S3.E17)\) that

𝑮T​\(𝒁\)=𝑮⁡\(𝒁\)−𝒁​diag⁡\(𝒁T​𝑮​\(𝒁\)\)\.\\displaystyle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)=\\bm\{G\}\(\\bm\{Z\}\)\-\\bm\{Z\}\\operatorname\{diag\}\(\\bm\{Z\}^\{\\rm T\}\\bm\{G\}\(\\bm\{Z\}\)\)\.\(145\)For any𝒁1,𝒁2∈ℳ\\bm\{Z\}\_\{1\},\\bm\{Z\}\_\{2\}\\in\\mathcal\{M\}, we have

‖𝑮T​\(𝒁1\)−𝑮T​\(𝒁2\)‖F≤\\displaystyle\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}\\leq‖𝑮⁡\(𝒁1\)−𝑮⁡\(𝒁2\)‖F\+\|𝒁1​diag⁡\(𝒁1T​𝑮​\(𝒁1\)\)\\displaystyle\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}\+\\\|\\bm\{Z\}\_\{1\}\\operatorname\{diag\}\(\\bm\{Z\}\_\{1\}^\{\\rm T\}\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\)−𝒁2diag\(𝒁2T𝑮\(𝒁2\)\)∥F\.\\displaystyle\-\\bm\{Z\}\_\{2\}\\operatorname\{diag\}\(\\bm\{Z\}\_\{2\}^\{\\rm T\}\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\)\\\|\_\{F\}\.\(146\)
We first establish a Lipschitz bound for the Euclidean update direction𝑮⁡\(𝒁\)\\bm\{G\}\(\\bm\{Z\}\)\. Define𝑸ij=αj​\(𝒁ij\)​𝑰\+n/\(tr⁡\(𝚷j\)​ϵ2\)​𝒁i​𝚷j​𝒁iT\\bm\{Q\}\_\{i\}^\{j\}=\\alpha\_\{j\}\(\\bm\{Z\}\_\{i\}^\{j\}\)\\bm\{I\}\+n/\(\{\\rm tr\}\(\\bm\{\\Pi\}\_\{j\}\)\\epsilon^\{2\}\)\\bm\{Z\}\_\{i\}\\bm\{\\Pi\}\_\{j\}\\bm\{Z\}\_\{i\}^\{\\rm T\}\. Combining \([119](https://arxiv.org/html/2608.20668#A3.E119)\), \([120](https://arxiv.org/html/2608.20668#A3.E120)\), and the triangle inequality for the Frobenius norm yields the following upper bound:

‖𝑮⁡\(𝒁1\)−𝑮⁡\(𝒁2\)‖F\\displaystyle\\big\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\big\\\|\_\{F\}=nm​ϵ2​‖𝑸1−1​𝒁1−𝑸2−1​𝒁2−∑j=1k\(\(𝑸1j\)−1​𝒁1​𝚷j−\(𝑸2j\)−1​𝒁2​𝚷j\)‖F\\displaystyle=\\frac\{n\}\{m\\epsilon^\{2\}\}\\Bigg\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\bm\{Z\}\_\{1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\\bm\{Z\}\_\{2\}\-\\sum\_\{j=1\}^\{k\}\\Big\(\(\\bm\{Q\}\_\{1\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{1\}\\bm\{\\Pi\}\_\{j\}\-\(\\bm\{Q\}\_\{2\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{2\}\\bm\{\\Pi\}\_\{j\}\\Big\)\\Bigg\\\|\_\{F\}\(147\)≤nm​ϵ2​‖𝑸1−1​𝒁1−𝑸2−1​𝒁2‖F\+nm​ϵ2​∑j=1k‖\(𝑸1j\)−1​𝒁1​𝚷j−\(𝑸2j\)−1​𝒁2​𝚷j‖F\.\\displaystyle\\leq\\frac\{n\}\{m\\epsilon^\{2\}\}\\Big\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\bm\{Z\}\_\{1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\\bm\{Z\}\_\{2\}\\Big\\\|\_\{F\}\+\\frac\{n\}\{m\\epsilon^\{2\}\}\\sum\_\{j=1\}^\{k\}\\Big\\\|\(\\bm\{Q\}\_\{1\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{1\}\\bm\{\\Pi\}\_\{j\}\-\(\\bm\{Q\}\_\{2\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{2\}\\bm\{\\Pi\}\_\{j\}\\Big\\\|\_\{F\}\.\(148\)By analyzing the first term in \([148](https://arxiv.org/html/2608.20668#A4.E148)\), we can obtain

nm​ϵ2​‖𝑸1−1​𝒁1−𝑸2−1​𝒁2‖F\\displaystyle\\frac\{n\}\{m\\epsilon^\{2\}\}\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\bm\{Z\}\_\{1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\\bm\{Z\}\_\{2\}\\\|\_\{F\}=nm​ϵ2​‖𝑸1−1​\(𝒁1−𝒁2\)\+\(𝑸1−1−𝑸2−1\)​𝒁2‖F\\displaystyle=\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\(\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\)\+\(\\bm\{Q\}\_\{1\}^\{\-1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\)\\bm\{Z\}\_\{2\}\\right\\\|\_\{F\}\(149\)≤nm​ϵ2​\(‖𝑸1−1​\(𝒁1−𝒁2\)‖F\+‖\(𝑸1−1−𝑸2−1\)​𝒁2‖F\)\\displaystyle\\leq\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\(\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}\+\\\|\(\\bm\{Q\}\_\{1\}^\{\-1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\)\\bm\{Z\}\_\{2\}\\\|\_\{F\}\\right\)\(150\)≤nm​ϵ2​\(‖𝑸1−1‖2​‖𝒁1−𝒁2‖F\+‖𝑸1−1−𝑸2−1‖F​‖𝒁2‖2\)\\displaystyle\\leq\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\\|\_\{2\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\+\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\-\\bm\{Q\}\_\{2\}^\{\-1\}\\\|\_\{F\}\\\|\\bm\{Z\}\_\{2\}\\\|\_\{2\}\\right\)\(151\)≤nm​ϵ2​\(‖𝑸1−1‖2\+m​Linv\)​‖𝒁1−𝒁2‖F\.\\displaystyle\\leq\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\\|\_\{2\}\+\\sqrt\{m\}\\,L\_\{\\rm inv\}\\right\)\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(152\)≤nm​ϵ2​\[\(n\+1\)\+\(n\+1\)2​\(π​n\+2​nϵ2\)\]​‖𝒁1−𝒁2‖F\\displaystyle\\leq\\frac\{n\}\{m\\epsilon^\{2\}\}\\biggl\[\(n\+1\)\+\(n\+1\)^\{2\}\\left\(\\pi\\sqrt\{n\}\+\\frac\{2n\}\{\\epsilon^\{2\}\}\\right\)\\biggr\]\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\(153\)=n⁡\(n\+1\)m​ϵ2​\[1\+\(n\+1\)​\(π​n\+2​nϵ2\)\]​‖𝒁1−𝒁2‖F\\displaystyle=\\frac\{n\(n\+1\)\}\{m\\epsilon^\{2\}\}\\biggl\[1\+\(n\+1\)\\left\(\\pi\\sqrt\{n\}\+\\frac\{2n\}\{\\epsilon^\{2\}\}\\right\)\\biggr\]\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\(154\)=LE​‖𝒁1−𝒁2‖F,\\displaystyle=L\_\{\\rm E\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\},\(155\)where

LE=n⁡\(n\+1\)m​ϵ2​\[1\+\(n\+1\)​\(π​n\+2​nϵ2\)\]\.\\displaystyle L\_\{\\rm E\}=\\frac\{n\(n\+1\)\}\{m\\epsilon^\{2\}\}\\biggl\[1\+\(n\+1\)\\left\(\\pi\\sqrt\{n\}\+\\frac\{2n\}\{\\epsilon^\{2\}\}\\right\)\\biggr\]\.\(156\)is a constant\. Step \([152](https://arxiv.org/html/2608.20668#A4.E152)\) follows from Lemma[3](https://arxiv.org/html/2608.20668#Thmlemma3)and the fact that‖𝒁2‖2≤‖𝒁2‖F=m\\\|\\bm\{Z\}\_\{2\}\\\|\_\{2\}\\leq\\\|\\bm\{Z\}\_\{2\}\\\|\_\{F\}=\\sqrt\{m\}, while Step \([153](https://arxiv.org/html/2608.20668#A4.E153)\) follows from Lemma[1](https://arxiv.org/html/2608.20668#Thmlemma1)and the definition ofLinvL\_\{\\rm inv\}in Lemma[3](https://arxiv.org/html/2608.20668#Thmlemma3)\. Similarly, from an analysis of the second term, we can obtain

nm​ϵ2​∑j=1k‖\(𝑸1j\)−1​𝒁1​𝚷j−\(𝑸2j\)−1​𝒁2​𝚷j‖F≤k​LE​‖𝒁1−𝒁2‖F\.\\displaystyle\\frac\{n\}\{m\\epsilon^\{2\}\}\\sum\_\{j=1\}^\{k\}\\Big\\\|\(\\bm\{Q\}\_\{1\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{1\}\\bm\{\\Pi\}\_\{j\}\-\(\\bm\{Q\}\_\{2\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{2\}\\bm\{\\Pi\}\_\{j\}\\Big\\\|\_\{F\}\\leq kL\_\{\\rm E\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(157\)Here, we use‖𝒁​𝚷j‖2≤mj\\\|\\bm\{Z\}\\bm\{\\Pi\}\_\{j\}\\\|\_\{2\}\\leq\\sqrt\{m\_\{j\}\}andmj​Linvj=m​Linv\\sqrt\{m\_\{j\}\}L\_\{\\rm inv\}^\{j\}=\\sqrt\{m\}\\,L\_\{\\rm inv\}, so that each class\-wise term is bounded byLE​‖𝒁1−𝒁2‖FL\_\{\\rm E\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.

Then the second term in \([146](https://arxiv.org/html/2608.20668#A4.E146)\) is analyzed\. For notational simplicity, set𝑫i=diag\(𝒁iT𝑮\(𝒁i\)\),i=1,2\\bm\{D\}\_\{i\}=\\mathrm\{diag\}\\big\(\\bm\{Z\}\_\{i\}^\{\\mathrm\{T\}\}\\bm\{G\}\(\\bm\{Z\}\_\{i\}\)\\big\),\\ i=1,2\. By the triangle inequality of the Frobenius norm, we have

‖𝒁1​𝑫1−𝒁2​𝑫2‖F\\displaystyle\\\|\\bm\{Z\}\_\{1\}\\bm\{D\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\bm\{D\}\_\{2\}\\\|\_\{F\}=‖𝒁1​\(𝑫1−𝑫2\)\+\(𝒁1−𝒁2\)​𝑫2‖F\\displaystyle=\\\|\\bm\{Z\}\_\{1\}\(\\bm\{D\}\_\{1\}\-\\bm\{D\}\_\{2\}\)\+\(\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\)\\bm\{D\}\_\{2\}\\\|\_\{F\}\(158\)≤‖𝒁1​\(𝑫1−𝑫2\)‖F\+‖\(𝒁1−𝒁2\)​𝑫2‖F\\displaystyle\\leq\\\|\\bm\{Z\}\_\{1\}\(\\bm\{D\}\_\{1\}\-\\bm\{D\}\_\{2\}\)\\\|\_\{F\}\+\\\|\(\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\)\\bm\{D\}\_\{2\}\\\|\_\{F\}\(159\)≤‖𝒁1‖2​‖𝑫1−𝑫2‖F\+‖𝒁1−𝒁2‖F​‖𝑫2‖2\.\\displaystyle\\leq\\\|\\bm\{Z\}\_\{1\}\\\|\_\{2\}\\big\\\|\\bm\{D\}\_\{1\}\-\\bm\{D\}\_\{2\}\\big\\\|\_\{F\}\+\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\\\|\\bm\{D\}\_\{2\}\\\|\_\{2\}\.\(160\)Consider the first term of \([160](https://arxiv.org/html/2608.20668#A4.E160)\),

‖𝒁1‖2​‖𝑫1−𝑫2‖F\\displaystyle\\\|\\bm\{Z\}\_\{1\}\\\|\_\{2\}\\big\\\|\\bm\{D\}\_\{1\}\-\\bm\{D\}\_\{2\}\\big\\\|\_\{F\}≤m​‖𝑫1−𝑫2‖F\\displaystyle\\leq\\sqrt\{m\}\\big\\\|\\bm\{D\}\_\{1\}\-\\bm\{D\}\_\{2\}\\big\\\|\_\{F\}\(161\)≤m​‖𝒁1T​𝑮​\(𝒁1\)−𝒁2T​𝑮​\(𝒁2\)‖F\\displaystyle\\leq\\sqrt\{m\}\\left\\\|\\bm\{Z\}\_\{1\}^\{T\}\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{Z\}\_\{2\}^\{T\}\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\right\\\|\_\{F\}\(162\)=m​‖\(𝒁1−𝒁2\)T​𝑮​\(𝒁1\)\+𝒁2T​\(𝑮⁡\(𝒁1\)−𝑮⁡\(𝒁2\)\)‖F\\displaystyle=\\sqrt\{m\}\\left\\\|\(\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\)^\{T\}\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\+\\bm\{Z\}\_\{2\}^\{T\}\\big\(\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\big\)\\right\\\|\_\{F\}\(163\)≤m​‖𝑮⁡\(𝒁1\)‖2​‖𝒁1−𝒁2‖F\+m​‖𝒁2‖2​‖𝑮⁡\(𝒁1\)−𝑮⁡\(𝒁2\)‖F\\displaystyle\\leq\\sqrt\{m\}\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\\\|\_\{2\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\+\\sqrt\{m\}\\\|\\bm\{Z\}\_\{2\}\\\|\_\{2\}\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}\(164\)≤m​‖𝑮⁡\(𝒁1\)‖F​‖𝒁1−𝒁2‖F\+m​‖𝒁2‖2​‖𝑮⁡\(𝒁1\)−𝑮⁡\(𝒁2\)‖F\.\\displaystyle\\leq\\sqrt\{m\}\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\\\|\_\{F\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\+\\sqrt\{m\}\\\|\\bm\{Z\}\_\{2\}\\\|\_\{2\}\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}\.\(165\)
The upper bound of‖𝑮⁡\(𝒁1\)‖F\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\\\|\_\{F\}can be derived using the inverse\-matrix bounds established above\.

‖𝑮⁡\(𝒁1\)‖F\\displaystyle\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\\\|\_\{F\}=nm​ϵ2​‖𝑸1−1​𝒁1−∑j=1k\(𝑸1j\)−1​𝒁1​𝚷j‖F\\displaystyle=\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\bm\{Z\}\_\{1\}\-\\sum\_\{j=1\}^\{k\}\(\\bm\{Q\}\_\{1\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{1\}\\bm\{\\Pi\}\_\{j\}\\right\\\|\_\{F\}\(166\)≤nm​ϵ2​\(‖𝑸1−1​𝒁1‖F\+‖∑j=1k\(𝑸1j\)−1​𝒁1​𝚷j‖F\)\\displaystyle\\leq\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\bm\{Z\}\_\{1\}\\\|\_\{F\}\+\\left\\\|\\sum\_\{j=1\}^\{k\}\(\\bm\{Q\}\_\{1\}^\{j\}\)^\{\-1\}\\bm\{Z\}\_\{1\}\\bm\{\\Pi\}\_\{j\}\\right\\\|\_\{F\}\\right\)\(167\)≤nm​ϵ2​\(‖𝑸1−1‖2​‖𝒁1‖F\+\(n\+1\)​‖𝒁1‖F\)\\displaystyle\\leq\\frac\{n\}\{m\\epsilon^\{2\}\}\\left\(\\\|\\bm\{Q\}\_\{1\}^\{\-1\}\\\|\_\{2\}\\\|\\bm\{Z\}\_\{1\}\\\|\_\{F\}\+\(n\+1\)\\\|\\bm\{Z\}\_\{1\}\\\|\_\{F\}\\right\)\(168\)≤2​n​\(n\+1\)m​ϵ2​‖𝒁1‖F\\displaystyle\\leq\\frac\{2n\(n\+1\)\}\{m\\epsilon^\{2\}\}\\\|\\bm\{Z\}\_\{1\}\\\|\_\{F\}\(169\)=2​n​\(n\+1\)m​ϵ2\.\\displaystyle=\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\.\(170\)Step \([168](https://arxiv.org/html/2608.20668#A4.E168)\) follows from‖\(𝑸1j\)−1‖2≤n\+1\\\|\(\\bm\{Q\}\_\{1\}^\{j\}\)^\{\-1\}\\\|\_\{2\}\\leq n\+1,j=1,2,…,kj=1,2,\\dots,k, together with∑j=1k𝚷j=𝑰\\sum\_\{j=1\}^\{k\}\\bm\{\\Pi\}\_\{j\}=\\bm\{I\}, and𝚷i​𝚷j=𝟎\\bm\{\\Pi\}\_\{i\}\\bm\{\\Pi\}\_\{j\}=\\bm\{0\}fori≠ji\\neq j\. Therefore, combining \([155](https://arxiv.org/html/2608.20668#A4.E155)\) and \([157](https://arxiv.org/html/2608.20668#A4.E157)\) gives the following explicit upper bound for \([165](https://arxiv.org/html/2608.20668#A4.E165)\):

m​‖𝑮⁡\(𝒁1\)‖F​‖𝒁1−𝒁2‖F\+m​‖𝒁2‖2​‖𝑮⁡\(𝒁1\)−𝑮⁡\(𝒁2\)‖F\\displaystyle\\sqrt\{m\}\\,\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\\\|\_\{F\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\+\\sqrt\{m\}\\,\\\|\\bm\{Z\}\_\{2\}\\\|\_\{2\}\\\|\\bm\{G\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}≤\(2​n​\(n\+1\)ϵ2\+m⁡\(k\+1\)​LE\)​‖𝒁1−𝒁2‖F\.\\displaystyle\\hskip 20\.00003pt\\leq\\left\(\\frac\{2n\(n\+1\)\}\{\\epsilon^\{2\}\}\+m\(k\+1\)L\_\{\\rm E\}\\right\)\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(171\)
To bound the second term in \([160](https://arxiv.org/html/2608.20668#A4.E160)\), let𝒛2,i\\bm\{z\}\_\{2,i\}and𝒈2,i\\bm\{g\}\_\{2,i\}denote theii\-th columns of𝒁2\\bm\{Z\}\_\{2\}and𝑮⁡\(𝒁2\)\\bm\{G\}\(\\bm\{Z\}\_\{2\}\), respectively\. Since

𝑫2\\displaystyle\\bm\{D\}\_\{2\}=diag⁡\(𝒁2T​𝑮​\(𝒁2\)\)\\displaystyle=\\operatorname\{diag\}\\left\(\\bm\{Z\}\_\{2\}^\{\\rm T\}\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\right\)\(172\)=diag⁡\(𝒛2,1T​𝒈2,1,…,𝒛2,mT​𝒈2,m\),\\displaystyle=\\operatorname\{diag\}\\left\(\\bm\{z\}\_\{2,1\}^\{\\rm T\}\\bm\{g\}\_\{2,1\},\\ldots,\\bm\{z\}\_\{2,m\}^\{\\rm T\}\\bm\{g\}\_\{2,m\}\\right\),\(173\)the spectral norm of𝑫2\\bm\{D\}\_\{2\}satisfies

‖𝑫2‖2\\displaystyle\\\|\\bm\{D\}\_\{2\}\\\|\_\{2\}=max1≤i≤m⁡\|𝒛2,iT​𝒈2,i\|\\displaystyle=\\max\_\{1\\leq i\\leq m\}\\left\|\\bm\{z\}\_\{2,i\}^\{\\rm T\}\\bm\{g\}\_\{2,i\}\\right\|\(174\)≤max1≤i≤m⁡\(‖𝒛2,i‖2​‖𝒈2,i‖2\)\\displaystyle\\leq\\max\_\{1\\leq i\\leq m\}\\left\(\\\|\\bm\{z\}\_\{2,i\}\\\|\_\{2\}\\\|\\bm\{g\}\_\{2,i\}\\\|\_\{2\}\\right\)\(175\)=max1≤i≤m⁡‖𝒈2,i‖2\\displaystyle=\\max\_\{1\\leq i\\leq m\}\\\|\\bm\{g\}\_\{2,i\}\\\|\_\{2\}\(176\)≤∑i=1m‖𝒈2,i‖22\\displaystyle\\leq\\sqrt\{\\sum\_\{i=1\}^\{m\}\\\|\\bm\{g\}\_\{2,i\}\\\|\_\{2\}^\{2\}\}\(177\)=‖𝑮⁡\(𝒁2\)‖F\\displaystyle=\\\|\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}\(178\)≤2​n​\(n\+1\)m​ϵ2\.\\displaystyle\\leq\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\.\(179\)Step \([175](https://arxiv.org/html/2608.20668#A4.E175)\) follows from the Cauchy\-Schwarz inequality\. Step \([177](https://arxiv.org/html/2608.20668#A4.E177)\) holds since each term is nonnegative, and the square of the maximum term does not exceed the sum of the squares of all terms\. The last inequality follows by applying the same bound as in \([170](https://arxiv.org/html/2608.20668#A4.E170)\) to𝑮⁡\(𝒁2\)\\bm\{G\}\(\\bm\{Z\}\_\{2\}\)\. Consequently,

‖𝒁1−𝒁2‖F​‖𝑫2‖2≤2​n​\(n\+1\)m​ϵ2​‖𝒁1−𝒁2‖F\.\\displaystyle\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\\\|\\bm\{D\}\_\{2\}\\\|\_\{2\}\\leq\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\.\(180\)
Finally, combining Equations \([146](https://arxiv.org/html/2608.20668#A4.E146)\), \([155](https://arxiv.org/html/2608.20668#A4.E155)\), \([157](https://arxiv.org/html/2608.20668#A4.E157)\), \([171](https://arxiv.org/html/2608.20668#A4.E171)\) and \([180](https://arxiv.org/html/2608.20668#A4.E180)\), the upper bound of‖𝑮T​\(𝒁1\)−𝑮T​\(𝒁2\)‖F\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}is given by

‖𝑮T​\(𝒁1\)−𝑮T​\(𝒁2\)‖F\\displaystyle\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{1\}\)\-\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\_\{2\}\)\\\|\_\{F\}≤\[\(m\+1\)​\(k\+1\)​LE\+2​n​\(n\+1\)ϵ2​\(1\+1m\)\]​‖𝒁1−𝒁2‖F\\displaystyle\\leq\\Bigg\[\(m\+1\)\(k\+1\)L\_\{\\rm E\}\+\\frac\{2n\(n\+1\)\}\{\\epsilon^\{2\}\}\\left\(1\+\\frac\{1\}\{\\sqrt\{m\}\}\\right\)\\Bigg\]\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\}\(181\)=Lgrad​‖𝒁1−𝒁2‖F,\\displaystyle=L\_\{\\rm grad\}\\\|\\bm\{Z\}\_\{1\}\-\\bm\{Z\}\_\{2\}\\\|\_\{F\},\(182\)where

Lgrad=\(m\+1\)​\(k\+1\)​LE\+2​n​\(n\+1\)ϵ2​\(1\+1m\)\.\\displaystyle L\_\{\\rm grad\}=\(m\+1\)\(k\+1\)L\_\{\\rm E\}\+\\frac\{2n\(n\+1\)\}\{\\epsilon^\{2\}\}\\left\(1\+\\frac\{1\}\{\\sqrt\{m\}\}\\right\)\.\(183\)This completes the proof\. ∎

## Appendix EProof of Proposition[1](https://arxiv.org/html/2608.20668#Thmproposition1)

Now, we are ready to establish Proposition[1](https://arxiv.org/html/2608.20668#Thmproposition1)\. Fix an arbitrary point𝒁=\[𝒛1,…,𝒛m\]∈ℳ\\bm\{Z\}=\[\\bm\{z\}\_\{1\},\\ldots,\\bm\{z\}\_\{m\}\]\\in\\mathcal\{M\}and an arbitrary tangent vector𝝃=\[𝝃1,…,𝝃m\]\\bm\{\\xi\}=\[\\bm\{\\xi\}\_\{1\},\\ldots,\\bm\{\\xi\}\_\{m\}\]\. Since𝝃∈T𝒁​ℳ\\bm\{\\xi\}\\in T\_\{\\bm\{Z\}\}\\mathcal\{M\}, we have

𝒛iT𝝃i=0,i=1,…,m\.\\displaystyle\\bm\{z\}\_\{i\}^\{\\rm T\}\\bm\{\\xi\}\_\{i\}=0,\\hskip 20\.00003pti=1,\\ldots,m\.\(184\)Consider

𝒁⁡\(s\)=Geo⁡\(𝒁,s​𝝃\),s∈\[0,1\],\\displaystyle\\bm\{Z\}\(s\)=\\operatorname\{Geo\}\(\\bm\{Z\},s\\bm\{\\xi\}\),\\hskip 10\.00002pts\\in\[0,1\],\(185\)which satisfies

𝒁⁡\(0\)=𝒁,𝒁⁡\(1\)=Geo⁡\(𝒁,𝝃\)\.\\displaystyle\\bm\{Z\}\(0\)=\\bm\{Z\},\\hskip 10\.00002pt\\bm\{Z\}\(1\)=\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\)\.\(186\)For𝝃i≠𝟎\\bm\{\\xi\}\_\{i\}\\neq\\bm\{0\}, theii\-th column of𝒁⁡\(s\)\\bm\{Z\}\(s\)is

𝒛i​\(s\)=cos⁡\(s​‖𝝃i‖2\)​𝒛i\+sin⁡\(s​‖𝝃i‖2\)​𝝃i‖𝝃i‖2,\\displaystyle\\bm\{z\}\_\{i\}\(s\)=\\cos\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\)\\bm\{z\}\_\{i\}\+\\sin\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\)\\frac\{\\bm\{\\xi\}\_\{i\}\}\{\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\},\(187\)whereas𝒛i​\(s\)=𝒛i\\bm\{z\}\_\{i\}\(s\)=\\bm\{z\}\_\{i\}when𝝃i=𝟎\\bm\{\\xi\}\_\{i\}=\\bm\{0\}\. Differentiating \([187](https://arxiv.org/html/2608.20668#A5.E187)\) with respect tossgives

d​𝒛i​\(s\)d​s=−‖𝝃i‖2​sin⁡\(s​‖𝝃i‖2\)​𝒛i\+𝝃i​cos⁡\(s​‖𝝃i‖2\)\.\\displaystyle\\frac\{\\mathrm\{d\}\\bm\{z\}\_\{i\}\(s\)\}\{\\mathrm\{d\}s\}=\-\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\sin\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\\bm\{z\}\_\{i\}\+\\bm\{\\xi\}\_\{i\}\\cos\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\.\(188\)Using‖𝒛i‖2=1\\\|\\bm\{z\}\_\{i\}\\\|\_\{2\}=1and𝒛iT​𝝃i=0\\bm\{z\}\_\{i\}^\{\\rm T\}\\bm\{\\xi\}\_\{i\}=0, we obtain

‖d​𝒛i​\(s\)d​s‖22\\displaystyle\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{z\}\_\{i\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\\|\_\{2\}^\{2\}=‖𝝃i‖22​sin2⁡\(s​‖𝝃i‖2\)​‖𝒛i‖22\+‖𝝃i‖22​cos2⁡\(s​‖𝝃i‖2\)\\displaystyle=\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\\sin^\{2\}\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\\\|\\bm\{z\}\_\{i\}\\\|\_\{2\}^\{2\}\+\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\\cos^\{2\}\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)−2​‖𝝃i‖2​sin⁡\(s​‖𝝃i‖2\)​cos⁡\(s​‖𝝃i‖2\)​𝒛iT​𝝃i\\displaystyle\\hskip 10\.00002pt\-2\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\sin\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\\cos\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\\bm\{z\}\_\{i\}^\{\\rm T\}\\bm\{\\xi\}\_\{i\}\(189\)=‖𝝃i‖22​\[sin2⁡\(s​‖𝝃i‖2\)\+cos2⁡\(s​‖𝝃i‖2\)\]\\displaystyle=\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\\left\[\\sin^\{2\}\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\+\\cos^\{2\}\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\\right\]\(190\)=‖𝝃i‖22\.\\displaystyle=\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\.\(191\)Consequently,

‖d​𝒁​\(s\)d​s‖F2\\displaystyle\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\\|\_\{F\}^\{2\}=∑i=1m‖d​𝒛i​\(s\)d​s‖22\\displaystyle=\\sum\_\{i=1\}^\{m\}\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{z\}\_\{i\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\\|\_\{2\}^\{2\}\(192\)=∑i=1m‖𝝃i‖22\\displaystyle=\\sum\_\{i=1\}^\{m\}\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\(193\)=‖𝝃‖F2\.\\displaystyle=\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\.\(194\)It follows from \([194](https://arxiv.org/html/2608.20668#A5.E194)\) that

‖d​𝒁​\(s\)d​s‖F=‖𝝃‖F\.\\displaystyle\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\\|\_\{F\}=\\\|\\bm\{\\xi\}\\\|\_\{F\}\.\(195\)Moreover, using2​\|sin⁡\(s​‖𝝃i‖2/2\)\|≤s​‖𝝃i‖22\\left\|\\sin\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}/2\\bigr\)\\right\|\\leq s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}and \([187](https://arxiv.org/html/2608.20668#A5.E187)\), we obtain

‖𝒁⁡\(s\)−𝒁‖F2\\displaystyle\\\|\\bm\{Z\}\(s\)\-\\bm\{Z\}\\\|\_\{F\}^\{2\}=∑i=1m‖𝒛i​\(s\)−𝒛i‖22\\displaystyle=\\sum\_\{i=1\}^\{m\}\\\|\\bm\{z\}\_\{i\}\(s\)\-\\bm\{z\}\_\{i\}\\\|\_\{2\}^\{2\}\(196\)=∑i=1m\(\[cos⁡\(s​‖𝝃i‖2\)−1\]2\+sin2⁡\(s​‖𝝃i‖2\)\)\\displaystyle=\\sum\_\{i=1\}^\{m\}\\left\(\\left\[\\cos\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\-1\\right\]^\{2\}\+\\sin^\{2\}\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\\right\)\(197\)=∑i=1m\(2−2​cos⁡\(s​‖𝝃i‖2\)\)\\displaystyle=\\sum\_\{i=1\}^\{m\}\\bigl\(2\-2\\cos\\bigl\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\bigr\)\\bigr\)\(198\)=∑i=1m\(4​sin2⁡\(s​‖𝝃i‖22\)\)\\displaystyle=\\sum\_\{i=1\}^\{m\}\\left\(4\\sin^\{2\}\\left\(\\frac\{s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\}\{2\}\\right\)\\right\)\(199\)≤s2​∑i=1m‖𝝃i‖22\\displaystyle\\leq s^\{2\}\\sum\_\{i=1\}^\{m\}\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\(200\)=s2​‖𝝃‖F2\.\\displaystyle=s^\{2\}\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\.\(201\)It follows from \([201](https://arxiv.org/html/2608.20668#A5.E201)\) that

‖𝒁⁡\(s\)−𝒁‖F≤s​‖𝝃‖F\.\\displaystyle\\\|\\bm\{Z\}\(s\)\-\\bm\{Z\}\\\|\_\{F\}\\leq s\\\|\\bm\{\\xi\}\\\|\_\{F\}\.\(202\)Similarly, it follows from \([188](https://arxiv.org/html/2608.20668#A5.E188)\) that

‖d​𝒛i​\(s\)d​s−𝝃i‖22\\displaystyle\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{z\}\_\{i\}\(s\)\}\{\\mathrm\{d\}s\}\-\\bm\{\\xi\}\_\{i\}\\right\\\|\_\{2\}^\{2\}=‖𝝃i‖22​\[sin2⁡\(s​‖𝝃i‖2\)\+\(cos⁡\(s​‖𝝃i‖2\)−1\)2\]\\displaystyle=\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\\left\[\\sin^\{2\}\\big\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\big\)\+\\big\(\\cos\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\)\-1\\big\)^\{2\}\\right\]\(203\)=‖𝝃i‖22​\(2−2​cos⁡\(s​‖𝝃i‖2\)\)\\displaystyle=\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\\Big\(2\-2\\cos\\big\(s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\\big\)\\Big\)\(204\)=4​‖𝝃i‖22​sin2⁡\(s​‖𝝃i‖22\)\\displaystyle=4\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\\sin^\{2\}\\left\(\\frac\{s\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}\}\{2\}\\right\)\(205\)≤s2​‖𝝃i‖24\.\\displaystyle\\leq s^\{2\}\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{4\}\.\(206\)Then,

‖d​𝒁​\(s\)d​s−𝝃‖F2\\displaystyle\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\-\\bm\{\\xi\}\\right\\\|\_\{F\}^\{2\}=∑i=1m‖d​𝒛i​\(s\)d​s−𝝃i‖22\\displaystyle=\\sum\_\{i=1\}^\{m\}\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{z\}\_\{i\}\(s\)\}\{\\mathrm\{d\}s\}\-\\bm\{\\xi\}\_\{i\}\\right\\\|\_\{2\}^\{2\}\(207\)≤∑i=1ms2​‖𝝃i‖24\\displaystyle\\leq\\sum\_\{i=1\}^\{m\}s^\{2\}\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{4\}\(208\)=s2​∑i=1m‖𝝃i‖24\\displaystyle=s^\{2\}\\sum\_\{i=1\}^\{m\}\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{4\}\(209\)≤s2​\(∑i=1m‖𝝃i‖22\)2\\displaystyle\\leq s^\{2\}\\left\(\\sum\_\{i=1\}^\{m\}\\\|\\bm\{\\xi\}\_\{i\}\\\|\_\{2\}^\{2\}\\right\)^\{2\}\(210\)=s2​‖𝝃‖F4\\displaystyle=s^\{2\}\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{4\}\(211\)It follows from \([211](https://arxiv.org/html/2608.20668#A5.E211)\) that

‖d​𝒁​\(s\)d​s−𝝃‖F≤s​‖𝝃‖F2\.\\displaystyle\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\-\\bm\{\\xi\}\\right\\\|\_\{F\}\\leq s\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\.\(212\)
Notice that𝒁⁡\(s\)\\bm\{Z\}\(s\),α⁡\(s\)\\alpha\(s\), andαj​\(s\)\\alpha\_\{j\}\(s\),j=1,…,kj=1,\\ldots,k, are all functions ofss\. Thus, by the definition of \([26](https://arxiv.org/html/2608.20668#S3.E26)\) and the chain rule, we have

dd​s​F​\(𝒁⁡\(s\)\)\\displaystyle\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}s\}F\(\\bm\{Z\}\(s\)\)=⟨∂Δ​R∂𝒁⁡\(s\),d​𝒁​\(s\)d​s⟩F\+∂Δ​R∂α⁡\(s\)​d​α​\(s\)d​s\+∑j=1k∂Δ​R∂αj​\(s\)​d​αj​\(s\)d​s\\displaystyle=\\left\\langle\\frac\{\\partial\\Delta R\}\{\\partial\\bm\{Z\}\(s\)\},\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\+\\frac\{\\partial\\Delta R\}\{\\partial\\alpha\(s\)\}\\frac\{\\mathrm\{d\}\\alpha\(s\)\}\{\\mathrm\{d\}s\}\+\\sum\_\{j=1\}^\{k\}\\frac\{\\partial\\Delta R\}\{\\partial\\alpha\_\{j\}\(s\)\}\\frac\{\\mathrm\{d\}\\alpha\_\{j\}\(s\)\}\{\\mathrm\{d\}s\}\(213\)=⟨𝑮⁡\(𝒁⁡\(s\)\),d​𝒁​\(s\)d​s⟩F\+12​tr​\(𝑸​\(s\)−1\)​α′​\(s\)\\displaystyle=\\left\\langle\\bm\{G\}\(\\bm\{Z\}\(s\)\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\+\\frac\{1\}\{2\}\{\\rm tr\}\\left\(\\bm\{Q\}\(s\)^\{\-1\}\\right\)\\alpha^\{\\prime\}\(s\)−∑j=1kmj2​mtr\(\(𝑸j\(s\)\)−1\)αj′\(s\)\.\\displaystyle\\hskip 10\.00002pt\-\\sum\_\{j=1\}^\{k\}\\frac\{m\_\{j\}\}\{2m\}\{\\rm tr\}\\left\(\(\\bm\{Q\}^\{j\}\(s\)\)^\{\-1\}\\right\)\\alpha\_\{j\}^\{\\prime\}\(s\)\.\(214\)where⟨⋅,⋅⟩F\\langle\\cdot,\\cdot\\rangle\_\{F\}denotes the Frobenius inner product\. Define

Eα​\(s\)\\displaystyle E\_\{\\alpha\}\(s\)=12​tr​\(𝑸​\(s\)−1\)​α′​\(s\)−∑j=1kmj2​m​tr​\(\(𝑸j​\(s\)\)−1\)​αj′​\(s\)\.\\displaystyle=\\frac\{1\}\{2\}\{\\rm tr\}\\left\(\\bm\{Q\}\(s\)^\{\-1\}\\right\)\\alpha^\{\\prime\}\(s\)\-\\sum\_\{j=1\}^\{k\}\\frac\{m\_\{j\}\}\{2m\}\{\\rm tr\}\\left\(\(\\bm\{Q\}^\{j\}\(s\)\)^\{\-1\}\\right\)\\alpha\_\{j\}^\{\\prime\}\(s\)\.\(215\)Sinced​𝒁​\(s\)/d​s∈T𝒁⁡\(s\)​ℳ\\mathrm\{d\}\\bm\{Z\}\(s\)/\\mathrm\{d\}s\\in T\_\{\\bm\{Z\}\(s\)\}\\mathcal\{M\}and𝑮T​\(𝒁​\(s\)\)\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\(s\)\)is the orthogonal projection of𝑮⁡\(𝒁⁡\(s\)\)\\bm\{G\}\(\\bm\{Z\}\(s\)\)ontoT𝒁⁡\(s\)​ℳT\_\{\\bm\{Z\}\(s\)\}\\mathcal\{M\}, we have

dd​s​F​\(𝒁⁡\(s\)\)=⟨𝑮T​\(𝒁⁡\(s\)\),d​𝒁​\(s\)d​s⟩F\+Eα​\(s\)\.\\displaystyle\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}s\}F\(\\bm\{Z\}\(s\)\)=\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\(s\)\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\+E\_\{\\alpha\}\(s\)\.\(216\)
To boundEα​\(s\)E\_\{\\alpha\}\(s\), we first consider the term associated withα⁡\(s\)\\alpha\(s\)\. By \([126](https://arxiv.org/html/2608.20668#A3.E126)\), we have𝑸⁡\(s\)⪰𝑨⁡\(s\)\\bm\{Q\}\(s\)\\succeq\\bm\{A\}\(s\), and hence𝑸​\(s\)−1⪯𝑨​\(s\)−1\\bm\{Q\}\(s\)^\{\-1\}\\preceq\\bm\{A\}\(s\)^\{\-1\}\. Therefore, using the expression forα′​\(s\)\\alpha^\{\\prime\}\(s\)derived in \([89](https://arxiv.org/html/2608.20668#A2.E89)\), we obtain

12​tr​\(𝑸​\(s\)−1\)​\|α′​\(s\)\|\\displaystyle\\frac\{1\}\{2\}\{\\rm tr\}\\left\(\\bm\{Q\}\(s\)^\{\-1\}\\right\)\|\\alpha^\{\\prime\}\(s\)\|=nm​tr⁡\(𝑸​\(s\)−1\)tr⁡\(𝑨​\(s\)−1\)​\|tr⁡\(𝒁​\(s\)T​𝑨​\(s\)−1​𝒁′​\(s\)\)\|\\displaystyle=\\frac\{n\}\{m\}\\frac\{\{\\rm tr\}\\left\(\\bm\{Q\}\(s\)^\{\-1\}\\right\)\}\{\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\\left\|\{\\rm tr\}\\left\(\\bm\{Z\}\(s\)^\{\\rm T\}\\bm\{A\}\(s\)^\{\-1\}\\bm\{Z\}^\{\\prime\}\(s\)\\right\)\\right\|\(217\)≤nm​\|tr⁡\(𝒁​\(s\)T​𝑨​\(s\)−1​𝒁′​\(s\)\)\|\\displaystyle\\leq\\frac\{n\}\{m\}\\left\|\{\\rm tr\}\\left\(\\bm\{Z\}\(s\)^\{\\rm T\}\\bm\{A\}\(s\)^\{\-1\}\\bm\{Z\}^\{\\prime\}\(s\)\\right\)\\right\|\(218\)≤nm​mn​tr​\(𝑨​\(s\)−1\)​‖𝒁′​\(s\)‖F\\displaystyle\\leq\\frac\{n\}\{m\}\\sqrt\{\\frac\{m\}\{n\}\{\\rm tr\}\\left\(\\bm\{A\}\(s\)^\{\-1\}\\right\)\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\(219\)≤n​n\+1m​‖𝒁′​\(s\)‖F\.\\displaystyle\\leq\\frac\{n\\sqrt\{n\+1\}\}\{\\sqrt\{m\}\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\.\(220\)Step \([218](https://arxiv.org/html/2608.20668#A5.E218)\) follows fromtr⁡\(𝑸​\(s\)−1\)≤tr⁡\(𝑨​\(s\)−1\)\{\\rm tr\}\(\\bm\{Q\}\(s\)^\{\-1\}\)\\leq\{\\rm tr\}\(\\bm\{A\}\(s\)^\{\-1\}\), step \([219](https://arxiv.org/html/2608.20668#A5.E219)\) from \([97](https://arxiv.org/html/2608.20668#A2.E97)\), and step \([220](https://arxiv.org/html/2608.20668#A5.E220)\) from Lemma[1](https://arxiv.org/html/2608.20668#Thmlemma1)\. For the class\-wise terms, we have

\|∑j=1kmj2​m​tr​\(\(𝑸j​\(s\)\)−1\)​αj′​\(s\)\|≤∑j=1kmj2​m​tr​\(\(𝑸j​\(s\)\)−1\)​\|αj′​\(s\)\|\\displaystyle\\left\|\\sum\_\{j=1\}^\{k\}\\frac\{m\_\{j\}\}\{2m\}\{\\rm tr\}\\left\(\(\\bm\{Q\}^\{j\}\(s\)\)^\{\-1\}\\right\)\\alpha\_\{j\}^\{\\prime\}\(s\)\\right\|\\leq\\sum\_\{j=1\}^\{k\}\\frac\{m\_\{j\}\}\{2m\}\{\\rm tr\}\\left\(\(\\bm\{Q\}^\{j\}\(s\)\)^\{\-1\}\\right\)\|\\alpha\_\{j\}^\{\\prime\}\(s\)\|\(221\)Step \([221](https://arxiv.org/html/2608.20668#A5.E221)\) follows from the triangle inequality andtr⁡\(\(𝑸j​\(s\)\)−1\)\>0\{\\rm tr\}\\big\(\(\\bm\{Q\}^\{j\}\(s\)\)^\{\-1\}\\big\)\>0\. Then, applying the same argument as in \([218](https://arxiv.org/html/2608.20668#A5.E218)\)–\([220](https://arxiv.org/html/2608.20668#A5.E220)\) to each class\-wise term, we obtain

\|∑j=1kmj2​m​tr​\(\(𝑸j​\(s\)\)−1\)​αj′​\(s\)\|\\displaystyle\\left\|\\sum\_\{j=1\}^\{k\}\\frac\{m\_\{j\}\}\{2m\}\{\\rm tr\}\\left\(\(\\bm\{Q\}^\{j\}\(s\)\)^\{\-1\}\\right\)\\alpha\_\{j\}^\{\\prime\}\(s\)\\right\|≤∑j=1kn​\(n\+1\)​mjm​‖\(𝒁j\)′​\(s\)‖F\\displaystyle\\leq\\sum\_\{j=1\}^\{k\}\\frac\{n\\sqrt\{\(n\+1\)m\_\{j\}\}\}\{m\}\\,\\\|\(\\bm\{Z\}^\{j\}\)^\{\\prime\}\(s\)\\\|\_\{F\}\(222\)=n​n\+1m​∑j=1kmj​‖\(𝒁j\)′​\(s\)‖F\\displaystyle=\\frac\{n\\sqrt\{n\+1\}\}\{m\}\\sum\_\{j=1\}^\{k\}\\sqrt\{m\_\{j\}\}\\,\\\|\(\\bm\{Z\}^\{j\}\)^\{\\prime\}\(s\)\\\|\_\{F\}\(223\)≤n​n\+1m​∑j=1kmj​∑j=1k‖\(𝒁j\)′​\(s\)‖F2\\displaystyle\\leq\\frac\{n\\sqrt\{n\+1\}\}\{m\}\\sqrt\{\\sum\_\{j=1\}^\{k\}m\_\{j\}\}\\sqrt\{\\sum\_\{j=1\}^\{k\}\\\|\(\\bm\{Z\}^\{j\}\)^\{\\prime\}\(s\)\\\|\_\{F\}^\{2\}\}\(224\)=n​n\+1m​‖𝒁′​\(s\)‖F\.\\displaystyle=\\frac\{n\\sqrt\{n\+1\}\}\{\\sqrt\{m\}\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\.\(225\)Step \([224](https://arxiv.org/html/2608.20668#A5.E224)\) follows from the Cauchy\-Schwarz inequality, while step \([225](https://arxiv.org/html/2608.20668#A5.E225)\) follows from∑j=1kmj=m\\sum\_\{j=1\}^\{k\}m\_\{j\}=mand∑j=1k‖\(𝒁j\)′​\(s\)‖F2=‖𝒁′​\(s\)‖F2\\sum\_\{j=1\}^\{k\}\\\|\(\\bm\{Z\}^\{j\}\)^\{\\prime\}\(s\)\\\|\_\{F\}^\{2\}=\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}^\{2\}, since the class\-wise selection matrices\{𝚷j\}j=1k\\\{\\bm\{\\Pi\}\_\{j\}\\\}\_\{j=1\}^\{k\}have mutually disjoint supports and partition all samples\.

Combining \([215](https://arxiv.org/html/2608.20668#A5.E215)\), \([220](https://arxiv.org/html/2608.20668#A5.E220)\) and \([225](https://arxiv.org/html/2608.20668#A5.E225)\), we obtain

\|Eα​\(s\)\|\\displaystyle\|E\_\{\\alpha\}\(s\)\|≤\|12​tr​\(𝑸​\(s\)−1\)​α′​\(s\)\|\+\|∑j=1kmj2​m​tr​\(\(𝑸j​\(s\)\)−1\)​αj′​\(s\)\|\\displaystyle\\leq\\left\|\\frac\{1\}\{2\}\{\\rm tr\}\\left\(\\bm\{Q\}\(s\)^\{\-1\}\\right\)\\alpha^\{\\prime\}\(s\)\\right\|\+\\left\|\\sum\_\{j=1\}^\{k\}\\frac\{m\_\{j\}\}\{2m\}\{\\rm tr\}\\left\(\(\\bm\{Q\}^\{j\}\(s\)\)^\{\-1\}\\right\)\\alpha\_\{j\}^\{\\prime\}\(s\)\\right\|\(226\)≤2​n​n\+1m​‖𝒁′​\(s\)‖F\.\\displaystyle\\leq\\frac\{2n\\sqrt\{n\+1\}\}\{\\sqrt\{m\}\}\\\|\\bm\{Z\}^\{\\prime\}\(s\)\\\|\_\{F\}\.\(227\)LetCα=2nn\+1m−1/2C\_\{\\alpha\}=2n\\sqrt\{n\+1\}\\,m^\{\-1/2\}\. From \([227](https://arxiv.org/html/2608.20668#A5.E227)\) and \([195](https://arxiv.org/html/2608.20668#A5.E195)\), we obtain

Eα​\(s\)≥−Cα​‖𝝃‖F\.\\displaystyle E\_\{\\alpha\}\(s\)\\geq\-C\_\{\\alpha\}\\\|\\bm\{\\xi\}\\\|\_\{F\}\.\(228\)
Integrating \([216](https://arxiv.org/html/2608.20668#A5.E216)\) overs∈\[0,1\]s\\in\[0,1\]and using𝒁⁡\(0\)=𝒁\\bm\{Z\}\(0\)=\\bm\{Z\}and𝒁⁡\(1\)=Geo⁡\(𝒁,𝝃\)\\bm\{Z\}\(1\)=\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\), we obtain

F⁡\(Geo⁡\(𝒁,𝝃\)\)−F⁡\(𝒁\)\\displaystyle F\\bigl\(\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\)\\bigr\)\-F\(\\bm\{Z\}\)=∫01dd​s​F​\(𝒁⁡\(s\)\)​𝑑s\\displaystyle=\\int\_\{0\}^\{1\}\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}s\}F\(\\bm\{Z\}\(s\)\)\\,\\mathrm\{d\}s=∫01\[⟨𝑮T​\(𝒁⁡\(s\)\),d​𝒁​\(s\)d​s⟩F\+Eα​\(s\)\]​𝑑s\.\\displaystyle=\\int\_\{0\}^\{1\}\\left\[\\left\\langle\\bm\{G\}\_\{\\rm T\}\\bigl\(\\bm\{Z\}\(s\)\\bigr\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\+E\_\{\\alpha\}\(s\)\\right\]\\,\\mathrm\{d\}s\.\(229\)
Subtracting⟨𝑮T​\(𝒁\),𝝃⟩F\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\bm\{\\xi\}\\rangle\_\{F\}from both sides of \([229](https://arxiv.org/html/2608.20668#A5.E229)\) yields

F⁡\(Geo⁡\(𝒁,𝝃\)\)−F⁡\(𝒁\)−⟨𝑮T​\(𝒁\),𝝃⟩F\\displaystyle F\\bigl\(\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\)\\bigr\)\-F\(\\bm\{Z\}\)\-\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\bm\{\\xi\}\\right\\rangle\_\{F\}=∫01\[⟨𝑮T​\(𝒁⁡\(s\)\),d​𝒁​\(s\)d​s⟩F\+Eα​\(s\)\]​𝑑s−∫01⟨𝑮T​\(𝒁\),𝝃⟩F​𝑑s\\displaystyle=\\int\_\{0\}^\{1\}\\left\[\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\(s\)\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\+E\_\{\\alpha\}\(s\)\\right\]\\mathrm\{d\}s\-\\int\_\{0\}^\{1\}\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\bm\{\\xi\}\\right\\rangle\_\{F\}\\mathrm\{d\}s\(230\)=∫01\[⟨𝑮T\(𝒁\(s\)\),d​𝒁​\(s\)d​s⟩F−⟨𝑮T\(𝒁\),d​𝒁​\(s\)d​s⟩F\\displaystyle=\\int\_\{0\}^\{1\}\\Bigg\[\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\(s\)\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\-\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\+⟨𝑮T\(𝒁\),d​𝒁​\(s\)d​s⟩F−⟨𝑮T\(𝒁\),𝝃⟩F\+Eα\(s\)\]ds\\displaystyle\\hskip 20\.00003pt\+\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\-\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\bm\{\\xi\}\\right\\rangle\_\{F\}\+E\_\{\\alpha\}\(s\)\\Bigg\]\\mathrm\{d\}s\(231\)=∫01⟨𝑮T​\(𝒁⁡\(s\)\)−𝑮T​\(𝒁\),d​𝒁​\(s\)d​s⟩F​𝑑s\\displaystyle=\\int\_\{0\}^\{1\}\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\(s\)\)\-\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\\mathrm\{d\}s\+∫01⟨𝑮T\(𝒁\),d​𝒁​\(s\)d​s−𝝃⟩Fds\\displaystyle\\hskip 10\.00002pt\+\\int\_\{0\}^\{1\}\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\-\\bm\{\\xi\}\\right\\rangle\_\{F\}\\mathrm\{d\}s\+∫01Eα\(s\)ds\.\\displaystyle\\hskip 10\.00002pt\+\\int\_\{0\}^\{1\}E\_\{\\alpha\}\(s\)\\,\\mathrm\{d\}s\.\(232\)For the first term in \([232](https://arxiv.org/html/2608.20668#A5.E232)\), we have

\|⟨𝑮T​\(𝒁⁡\(s\)\)−𝑮T​\(𝒁\),d​𝒁​\(s\)d​s⟩F\|\\displaystyle\\left\|\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\(s\)\)\-\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\rangle\_\{F\}\\right\|≤‖𝑮T​\(𝒁⁡\(s\)\)−𝑮T​\(𝒁\)‖F​‖d​𝒁​\(s\)d​s‖F\\displaystyle\\leq\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\(s\)\)\-\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)\\\|\_\{F\}\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\\|\_\{F\}\(233\)≤Lgrad​‖𝒁⁡\(s\)−𝒁‖F​‖d​𝒁​\(s\)d​s‖F\\displaystyle\\leq L\_\{\\rm grad\}\\\|\\bm\{Z\}\(s\)\-\\bm\{Z\}\\\|\_\{F\}\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\\right\\\|\_\{F\}\(234\)≤s​Lgrad​‖𝝃‖F2\.\\displaystyle\\leq sL\_\{\\rm grad\}\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\.\(235\)Step \([233](https://arxiv.org/html/2608.20668#A5.E233)\) follows from the Cauchy\-Schwarz inequality, step \([234](https://arxiv.org/html/2608.20668#A5.E234)\) follows from Lemma[4](https://arxiv.org/html/2608.20668#Thmlemma4), and step \([235](https://arxiv.org/html/2608.20668#A5.E235)\) follows from \([195](https://arxiv.org/html/2608.20668#A5.E195)\) and \([202](https://arxiv.org/html/2608.20668#A5.E202)\)\. Furthermore, since𝑮T​\(𝒁\)\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)is the orthogonal projection of𝑮⁡\(𝒁\)\\bm\{G\}\(\\bm\{Z\}\)ontoT𝒁​ℳT\_\{\\bm\{Z\}\}\\mathcal\{M\}, using \([170](https://arxiv.org/html/2608.20668#A4.E170)\), we have

‖𝑮T​\(𝒁\)‖F≤‖𝑮⁡\(𝒁\)‖F≤2​n​\(n\+1\)m​ϵ2\.\\displaystyle\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)\\\|\_\{F\}\\leq\\\|\\bm\{G\}\(\\bm\{Z\}\)\\\|\_\{F\}\\leq\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\.\(236\)For the second term in \([232](https://arxiv.org/html/2608.20668#A5.E232)\), combining \([236](https://arxiv.org/html/2608.20668#A5.E236)\) with \([212](https://arxiv.org/html/2608.20668#A5.E212)\), we obtain

\|⟨𝑮T​\(𝒁\),d​𝒁​\(s\)d​s−𝝃⟩F\|\\displaystyle\\left\|\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\-\\bm\{\\xi\}\\right\\rangle\_\{F\}\\right\|≤‖𝑮T​\(𝒁\)‖F​‖d​𝒁​\(s\)d​s−𝝃‖F\\displaystyle\\leq\\\|\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\)\\\|\_\{F\}\\left\\\|\\frac\{\\mathrm\{d\}\\bm\{Z\}\(s\)\}\{\\mathrm\{d\}s\}\-\\bm\{\\xi\}\\right\\\|\_\{F\}\(237\)≤2​n​\(n\+1\)​sm​ϵ2​‖𝝃‖F2\.\\displaystyle\\leq\\frac\{2n\(n\+1\)s\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\.\(238\)
Combining \([228](https://arxiv.org/html/2608.20668#A5.E228)\), \([235](https://arxiv.org/html/2608.20668#A5.E235)\), and \([238](https://arxiv.org/html/2608.20668#A5.E238)\), we obtain

F⁡\(Geo⁡\(𝒁,𝝃\)\)−F⁡\(𝒁\)−⟨𝑮T​\(𝒁\),𝝃⟩F\\displaystyle F\\bigl\(\\operatorname\{Geo\}\(\\bm\{Z\},\\bm\{\\xi\}\)\\bigr\)\-F\(\\bm\{Z\}\)\-\\left\\langle\\bm\{G\}\_\{\\rm T\}\(\\bm\{Z\}\),\\bm\{\\xi\}\\right\\rangle\_\{F\}≥−∫01s\[Lgrad\+2​n​\(n\+1\)m​ϵ2\]∥𝝃∥F2ds−∫01Cα∥𝝃∥Fds\\displaystyle\\geq\-\\int\_\{0\}^\{1\}s\\left\[L\_\{\\rm grad\}\+\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\\right\]\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\\,\\mathrm\{d\}s\-\\int\_\{0\}^\{1\}C\_\{\\alpha\}\\\|\\bm\{\\xi\}\\\|\_\{F\}\\,\\mathrm\{d\}s=−12​\[Lgrad\+2​n​\(n\+1\)m​ϵ2\]​‖𝝃‖F2−Cα​‖𝝃‖F\.\\displaystyle=\-\\frac\{1\}\{2\}\\left\[L\_\{\\rm grad\}\+\\frac\{2n\(n\+1\)\}\{\\sqrt\{m\}\\,\\epsilon^\{2\}\}\\right\]\\\|\\bm\{\\xi\}\\\|\_\{F\}^\{2\}\-C\_\{\\alpha\}\\\|\\bm\{\\xi\}\\\|\_\{F\}\.\(239\)Therefore, settingLsL\_\{\\rm s\}as in \([30](https://arxiv.org/html/2608.20668#S3.E30)\) yields \([27](https://arxiv.org/html/2608.20668#S3.E27)\)\. This completes the proof\.

## References

- \[1\]P\. Absil, R\. Mahony, and R\. Sepulchre\(2008\)Optimization algorithms on matrix manifolds\.Princeton University Press\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p4.1)\.
- \[2\]B\. Amirgaliyev, M\. Mussabek, T\. Rakhimzhanova, and A\. Zhumadillayeva\(2025\)A review of machine learning and deep learning methods for person detection, tracking and identification, and face recognition with applications\.Sensors25\(5\),pp\.1410\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[3\]M\. P\. Ayyar, J\. Benois\-Pineau, and A\. Zemmari\(2021\)Review of white box methods for explanations of convolutional neural networks in image classification tasks\.Journal of Electronic Imaging30\(5\),pp\.050901–050901\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[4\]N\. Boumal\(2023\)An introduction to optimization on smooth manifolds\.Cambridge University Press\.External Links:[Link](https://www.nicolasboumal.net/book),[Document](https://dx.doi.org/10.1017/9781009166164)Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p4.1)\.
- \[5\]K\. H\. R\. Chan, Y\. Yu, C\. You, H\. Qi, J\. Wright, and Y\. Ma\(2022\)ReduNet: a white\-box deep network from the principle of maximizing rate reduction\.The Journal of Machine Learning Research23\(1\),pp\.4907–5009\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.20668#S2.SS1.p3.2),[§2](https://arxiv.org/html/2608.20668#S2.p1.1),[§4\.1\.1](https://arxiv.org/html/2608.20668#S4.SS1.SSS1.p1.1),[§4](https://arxiv.org/html/2608.20668#S4.p1.1)\.
- \[6\]L\. N\. Darlow, E\. J\. Crowley, A\. Antoniou, and A\. J\. Storkey\(2018\)Cinic\-10 is not imagenet or cifar\-10\.arXiv preprint arXiv:1810\.03505\.Cited by:[§4](https://arxiv.org/html/2608.20668#S4.p1.2)\.
- \[7\]T\. M\. Getu, G\. Kaddoum, and M\. Bennis\(2025\)Semantic communication: a survey on research landscape, challenges, and future directions\.Proceedings of the IEEE112\(11\),pp\.1649–1685\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[8\]A\. Ghorbani, J\. Wexler, J\. Y\. Zou, and B\. Kim\(2019\)Towards automatic concept\-based explanations\.Advances in neural information processing systems32\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[9\]V\. Hassija, V\. Chamola, A\. Mahapatra, A\. Singal, D\. Goel, K\. Huang, S\. Scardapane, I\. Spinelli, M\. Mahmud, and A\. Hussain\(2024\)Interpreting black\-box models: a review on explainable artificial intelligence\.Cognitive Computation16\(1\),pp\.45–74\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[10\]Z\. Huang, Q\. Yan, B\. Dai, and X\. Tang\(2025\)A simple but accurate approximation for multivariate gaussian rate\-distortion function and its application in maximal coding rate reduction\.Tsinghua Science and Technology\.External Links:[Link](https://www.sciopen.com/article/10.26599/TST.2024.9010229),[Document](https://dx.doi.org/10.26599/TST.2024.9010229)Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p3.1),[§1](https://arxiv.org/html/2608.20668#S1.p4.1),[§2\.2](https://arxiv.org/html/2608.20668#S2.SS2.p1.2),[§2](https://arxiv.org/html/2608.20668#S2.p1.1)\.
- \[11\]A\. Krizhevsky G\. Hintonet al\.\(2009\)Learning multiple layers of features from tiny images\.Cited by:[§4](https://arxiv.org/html/2608.20668#S4.p1.2)\.
- \[12\]F\. Li and D\. ChuMulti\-redunet: interpretable class\-wise decomposition of redunet\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p3.1)\.
- \[13\]W\. Liu, Y\. Wen, Z\. Yu, M\. Li, B\. Raj, and L\. Song\(2017\)Sphereface: deep hypersphere embedding for face recognition\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\.212–220\.Cited by:[§3\.3](https://arxiv.org/html/2608.20668#S3.SS3.p3.2)\.
- \[14\]Y\. Ma, H\. Derksen, W\. Hong, and J\. Wright\(2007\)Segmentation of multivariate mixed data via lossy data coding and compression\.IEEE transactions on pattern analysis and machine intelligence29\(9\),pp\.1546–1562\.Cited by:[item 2](https://arxiv.org/html/2608.20668#S2.I1.i2.p1.1)\.
- \[15\]Y\. Meng, J\. Huang, G\. Wang, C\. Zhang, H\. Zhuang, L\. Kaplan, and J\. Han\(2019\)Spherical text embedding\.Advances in neural information processing systems32\.Cited by:[§3\.1](https://arxiv.org/html/2608.20668#S3.SS1.p3.2),[§3\.1](https://arxiv.org/html/2608.20668#S3.SS1.p4.1)\.
- \[16\]H\. Rabie and M\. A\. Akhloufi\(2025\)A review of machine learning and deep learning for parkinson’s disease detection\.Discover Artificial Intelligence5\(1\),pp\.24\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[17\]M\. T\. Ribeiro, S\. Singh, and C\. Guestrin\(2016\)" Why should i trust you?" explaining the predictions of any classifier\.InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining,pp\.1135–1144\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[18\]R\. R\. Selvaraju, M\. Cogswell, A\. Das, R\. Vedantam, D\. Parikh, and D\. Batra\(2017\)Grad\-cam: visual explanations from deep networks via gradient\-based localization\.InProceedings of the IEEE international conference on computer vision,pp\.618–626\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[19\]P\. Sharma, S\. Jain, S\. Gupta, and V\. Chamola\(2021\)Role of machine learning and deep learning in securing 5g\-driven industrial iot applications\.Ad Hoc Networks123,pp\.102685\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[20\]D\. Silver, A\. Huang, C\. J\. Maddison, A\. Guez, L\. Sifre, G\. Van Den Driessche, J\. Schrittwieser, I\. Antonoglou, V\. Panneershelvam, M\. Lanctot,et al\.\(2016\)Mastering the game of go with deep neural networks and tree search\.nature529\(7587\),pp\.484–489\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[21\]X\. Yu, H\. Zhang, L\. Peng, F\. Sun, and J\. Deng\(2024\)ESS\-redunet: enhancing subspace separability of redunet via dynamic expansion with bayesian inference\.arXiv preprint arXiv:2411\.17961\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p3.1)\.
- \[22\]Y\. Yu, K\. H\. R\. Chan, C\. You, C\. Song, and Y\. Ma\(2020\)Learning diverse and discriminative representations via the principle of maximal coding rate reduction\.Advances in Neural Information Processing Systems33,pp\.9422–9434\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.20668#S2.SS1.p1.1),[§2](https://arxiv.org/html/2608.20668#S2.p1.1)\.
- \[23\]H\. Zhang, M\. Li, Y\. Lai, M\. Wang, L\. Yu, and E\. Szczerbicki\(2025\)MSD\-rep: multi\-scale discriminative representation learning for chromosome classification with small datasets\.Knowledge\-Based Systems,pp\.114933\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p3.1)\.
- \[24\]J\. Zhao, Y\. Wu, R\. Deng, S\. Xu, J\. Gao, and A\. Burke\(2025\)A survey of autonomous driving from a deep learning perspective\.ACM Computing Surveys57\(10\),pp\.1–60\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p1.1)\.
- \[25\]S\. Zhao, X\. Li, J\. Ren, L\. Wang, and Z\. Huang\(2024\)Interpretable sar target recognition based on white box neural network\.In2024 IEEE International Conference on Signal, Information and Data Processing \(ICSIDP\),pp\.1–5\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p3.1)\.
- \[26\]Y\. Zhu, H\. Su, X\. Jing, R\. Yang, and H\. Wang\(2024\)An updateable network based on redunet for radar jamming recognition\.In2024 7th International Conference on Information Communication and Signal Processing \(ICICSP\),pp\.507–512\.Cited by:[§1](https://arxiv.org/html/2608.20668#S1.p3.1)\.