Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model

# Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model
Source: [https://arxiv.org/html/2606.28573](https://arxiv.org/html/2606.28573)
Andrea Montanari and Kangjie ZhouDepartment of Mathematics and Department of Statistics, Stanford UniversityDepartment of Statistics, Columbia University

###### Abstract

Modern machine learning models are trained by optimizing high\-dimensional non\-convex empirical risk functions\. Such cost functions can have a multitude of local optima and yet, gradient\-based optimization appears to converge to near\-global optima\.

Within a simple supervised learning setting, we develop a precise picture of which parts of the empirical risk landscape are accessible by polynomial\-time algorithms\. We are given i\.i\.d\. pairs\{\(𝒙i,yi\):1≤i≤n\}\\\{\(\\text\{$x$\}\_\{i\},y\_\{i\}\):\\;1\\leq i\\leq n\\\}with𝒙i∈ℝd\\text\{$x$\}\_\{i\}\\in\\mathbb\{R\}^\{d\}standard Gaussian feature vectors, andyi∈ℝy\_\{i\}\\in\\mathbb\{R\}response variables that depend on𝒙i\\text\{$x$\}\_\{i\}through their projections on an unknownkk\-dimensional subspace\. We use empirical risk minimization to learn a model that depends on anmm\-dimensional projection of the data \(e\.g\., anmm\-neurons neural network\)\.

We propose an incremental approximate message passing \(IAMP\) algorithm and precisely characterize the training error it achieves, as well as the relation between test and training error, in the high dimensional asymptoticsn,d→∞n,d\\to\\infty, withn/d→α∈\(0,\+∞\)n/d\\to\\alpha\\in\(0,\+\\infty\)\. Based on earlier work in related models, we expect that the performance achieved by our algorithm is optimal among polynomial\-time algorithms\.

###### Contents

1. [1Introduction](https://arxiv.org/html/2606.28573#S1)1. [1\.1Technical setting: Multi\-index model and ERM](https://arxiv.org/html/2606.28573#S1.SS1) 2. [1\.2Connection with projection pursuit](https://arxiv.org/html/2606.28573#S1.SS2) 3. [1\.3General approach](https://arxiv.org/html/2606.28573#S1.SS3) 4. [1\.4Summary of main results and paper organization](https://arxiv.org/html/2606.28573#S1.SS4) 5. [1\.5Definitions and notation](https://arxiv.org/html/2606.28573#S1.SS5)
2. [2Conjectures from statistical physics](https://arxiv.org/html/2606.28573#S2)
3. [3Main results \(I\): Algorithmic achievability](https://arxiv.org/html/2606.28573#S3)1. [3\.1Overview of the algorithm](https://arxiv.org/html/2606.28573#S3.SS1) 2. [3\.2Achievability results for AMP](https://arxiv.org/html/2606.28573#S3.SS2) 3. [3\.3Dual value𝒱m,α,φalg​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\(h\)and stochastic optimal control](https://arxiv.org/html/2606.28573#S3.SS3)
4. [4Replica symmetric regime form=k=1m=k=1](https://arxiv.org/html/2606.28573#S4)
5. [5Intermezzo: Largen/dn/dand tensor PCA equivalence](https://arxiv.org/html/2606.28573#S5)1. [5\.1Single\-index model](https://arxiv.org/html/2606.28573#S5.SS1) 2. [5\.2Replica prediction for global maximum](https://arxiv.org/html/2606.28573#S5.SS2) 3. [5\.3Achievability via AMP](https://arxiv.org/html/2606.28573#S5.SS3) 4. [5\.4The equivalent Gaussian model](https://arxiv.org/html/2606.28573#S5.SS4) 5. [5\.5Examples](https://arxiv.org/html/2606.28573#S5.SS5)
6. [6Main results \(II\): Parisi variational principle](https://arxiv.org/html/2606.28573#S6)
7. [7Two\-stage AMP algorithm: Proof of Theorem3\.1](https://arxiv.org/html/2606.28573#S7)1. [7\.1Approximate message passing](https://arxiv.org/html/2606.28573#S7.SS1) 2. [7\.2First stage: Fixed\-point AMP](https://arxiv.org/html/2606.28573#S7.SS2) 3. [7\.3Second stage: Incremental AMP](https://arxiv.org/html/2606.28573#S7.SS3) 4. [7\.4Combining the two stages](https://arxiv.org/html/2606.28573#S7.SS4)
8. [8Dual characterization and Parisi formula: Proof of Theorem6\.1](https://arxiv.org/html/2606.28573#S8)1. [8\.1Technical preliminaries](https://arxiv.org/html/2606.28573#S8.SS1) 2. [8\.2Proof of6\.1](https://arxiv.org/html/2606.28573#S8.SS2)
9. [References](https://arxiv.org/html/2606.28573#bib)
10. [AThe replica calculation](https://arxiv.org/html/2606.28573#A1)
11. [BAppendix forSection3](https://arxiv.org/html/2606.28573#A2)1. [B\.1Reduction to a random signal](https://arxiv.org/html/2606.28573#A2.SS1) 2. [B\.2AMP achievability results for the casem=k=1m=k=1](https://arxiv.org/html/2606.28573#A2.SS2) 3. [B\.3Auxiliary lemmas](https://arxiv.org/html/2606.28573#A2.SS3)
12. [CAppendix forSection4](https://arxiv.org/html/2606.28573#A3)1. [C\.1Proof of4\.1and4\.3](https://arxiv.org/html/2606.28573#A3.SS1) 2. [C\.2Estimation errors of ERM and Bayes AMP](https://arxiv.org/html/2606.28573#A3.SS2) 3. [C\.3Condition for replica symmetry](https://arxiv.org/html/2606.28573#A3.SS3)
13. [DAppendix forSection5](https://arxiv.org/html/2606.28573#A4)1. [D\.1Derivation of5\.1](https://arxiv.org/html/2606.28573#A4.SS1) 2. [D\.2Proof of5\.1,5\.2and5\.3](https://arxiv.org/html/2606.28573#A4.SS2) 3. [D\.3Appendix forSection5\.4](https://arxiv.org/html/2606.28573#A4.SS3)1. [D\.3\.1Parisi formula](https://arxiv.org/html/2606.28573#A4.SS3.SSS1) 2. [D\.3\.2Value ofH^dg\\widehat\{H\}\_\{d\}^\{g\}achieved by a two\-stage algorithm](https://arxiv.org/html/2606.28573#A4.SS3.SSS2) 4. [D\.4Proof of5\.4,D\.1andD\.2](https://arxiv.org/html/2606.28573#A4.SS4) 5. [D\.5Proof ofD\.3](https://arxiv.org/html/2606.28573#A4.SS5) 6. [D\.6Auxiliary lemmas](https://arxiv.org/html/2606.28573#A4.SS6)
14. [EAppendix for Section7](https://arxiv.org/html/2606.28573#A5)1. [E\.1Proof of7\.3and7\.4](https://arxiv.org/html/2606.28573#A5.SS1) 2. [E\.2Proof of Proposition7\.5](https://arxiv.org/html/2606.28573#A5.SS2) 3. [E\.3Proof of Theorem7\.1](https://arxiv.org/html/2606.28573#A5.SS3) 4. [E\.4Auxiliary lemmas](https://arxiv.org/html/2606.28573#A5.SS4)
15. [FAppendix forSection8](https://arxiv.org/html/2606.28573#A6)1. [F\.1Proof of8\.3](https://arxiv.org/html/2606.28573#A6.SS1) 2. [F\.2Auxiliary lemmas](https://arxiv.org/html/2606.28573#A6.SS2)

## 1Introduction

Empirical risk minimization \(ERM\) is the primary guiding principle in the design of learning algorithms\. However, its theoretical properties are still insufficiently understood in the regime where the model complexity111Characterized by the number of parameters or the Rademacher complexity\[[SSBD14](https://arxiv.org/html/2606.28573#bib.bibx37)\]\.scales proportionally with the sample size, while this scaling is precisely the one privileged by current AI trends\[[KMH\+20](https://arxiv.org/html/2606.28573#bib.bibx25),[HBM\+22](https://arxiv.org/html/2606.28573#bib.bibx20)\]\.

In the case of convex empirical risk functions \(namely, linear models with convex losses\), a rich theory allows us to derive sharp asymptotic results under simple data distributions\[[BM11b](https://arxiv.org/html/2606.28573#bib.bibx7),[TOH15](https://arxiv.org/html/2606.28573#bib.bibx40),[EK18](https://arxiv.org/html/2606.28573#bib.bibx15),[DM16](https://arxiv.org/html/2606.28573#bib.bibx13)\]\. Recent work has developed methods to handle non\-convexity when the empirical risk is locally convex in a neighborhood of the global minimizer, and there is only one or a small number of near\-global minima\[[AMS26](https://arxiv.org/html/2606.28573#bib.bibx4),[VDR\+25](https://arxiv.org/html/2606.28573#bib.bibx41),[MS26](https://arxiv.org/html/2606.28573#bib.bibx32)\]\. However, the empirical risk landscape of interest in machine learning is often qualitatively different, with many well\-separated local minima\.

An example of this type arises in the context of tensor PCA, where one tries to estimate a high\-dimensional rank\-one tensor, from observations corrupted by Gaussian noise\[[MR14](https://arxiv.org/html/2606.28573#bib.bibx30)\]\. In this case, the negative log\-likelihood function is a Gaussian processL^n​\(𝒘\)\\widehat\{L\}\_\{n\}\(\\text\{$w$\}\)indexed by the parameter vector𝒘∈𝕊d−1\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}, and can present an exponential number of local minima\[[AMMN19](https://arxiv.org/html/2606.28573#bib.bibx2)\]\. This type of cost functions is also at the core of the theory of mean\-field spin glasses\[[AMS23](https://arxiv.org/html/2606.28573#bib.bibx3)\]\.

Since global optimization of empirical risk functions with many local minima is out of reach of polynomial\-time algorithms in the worst case, we instead address the following central question:

- QWhich values of the training and test error are achievable by polynomial\-time algorithms with high probability with respect to a given data distribution?

To be definite, consider a simple setting where the observed i\.i\.d\. data\{\(𝒙i,yi\)\}i=1n\\\{\(\\text\{$x$\}\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}follow a Gaussian single\-index model:

yi=λ​φ​\(𝒘∗𝖳​𝒙i\)\+εi,𝒙i∼𝖭​\(𝟎,𝑰d\),εi∼𝖭​\(0,1\),y\_\{i\}=\\sqrt\{\\lambda\}\\,\\varphi\(\\text\{$w$\}\_\{\*\}^\{\{\\sf T\}\}\\text\{$x$\}\_\{i\}\)\+\\varepsilon\_\{i\}\\,,\\;\\;\\ \\text\{$x$\}\_\{i\}\\sim\{\\sf N\}\(\\mathrm\{\\bf 0\},\{\\boldsymbol\{I\}\}\_\{d\}\),\\,\\,\\varepsilon\_\{i\}\\sim\{\\sf N\}\(0,1\)\\,,\(1\)whereφ\\varphiis the link function,𝒘∗∈𝕊d−1\\text\{$w$\}\_\{\*\}\\in\\mathbb\{S\}^\{d\-1\}is the true signal, andλ\>0\\lambda\>0represents the signal\-to\-noise ratio\. We will be interested in the proportional scaling regime wheren,d→∞n,d\\to\\inftysuch thatn/d→α∈\(0,∞\)n/d\\to\\alpha\\in\(0,\\infty\)\. While our main results consider more general models, the above setup is already rich enough to illustrate the key points\.

To learn the true signal𝒘∗\\text\{$w$\}\_\{\*\}, we propose to solve the following correlation maximization problem over the unit sphere𝕊d−1\\mathbb\{S\}^\{d\-1\}:

maximizeH^n​\(𝒘\):=1n​λ​∑i=1nyi​σ​\(𝒘𝖳​𝒙i\),\\displaystyle\\widehat\{H\}\_\{n\}\(\)=\\frac\{1\}\{n\\sqrt\{\\lambda\}\}\\sum\_\{i=1\}^\{n\}y\_\{i\}\\sigma\(^\{\{\\sf T\}\}\_\{i\}\),\(2\)subject to𝒘∈𝕊d−1,\\displaystyle\\in\\mathbb\{S\}^\{d\-1\},for a given activation functionσ\\sigma\. Also in this respect, our main results are more general and we consider correlation loss here only as an illustration\. On the other hand, we do not assumeσ=φ\\sigma=\\varphibecause—in general—the data generating process is unknown\. We are particularly interested in characterizing the training and test errors achievable by efficient iterative algorithms, such as gradient\-based methods and approximate message passing \(AMP\)\.

We pause to note that many special cases of the model \([1](https://arxiv.org/html/2606.28573#S1.E1)\), and ERM methods analogous to \([2](https://arxiv.org/html/2606.28573#S1.E2)\) have been extensively investigated in the literature\. See, e\.g\.,\[[CC17](https://arxiv.org/html/2606.28573#bib.bibx8),[MM18](https://arxiv.org/html/2606.28573#bib.bibx27),[AMK\+18](https://arxiv.org/html/2606.28573#bib.bibx1),[BKM\+19](https://arxiv.org/html/2606.28573#bib.bibx5)\]\. Recall thatα=limn,d→∞n/d\\alpha=\\lim\_\{n,d\\to\\infty\}n/dis the limiting aspect ratio\. Existing results in literature\[[VDR\+25](https://arxiv.org/html/2606.28573#bib.bibx41),[MS26](https://arxiv.org/html/2606.28573#bib.bibx32),[MBB26](https://arxiv.org/html/2606.28573#bib.bibx26)\]suggest the following qualitative picture of the landscape ofH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\):

1. 1\.Forα\>αtr​\(λ,φ,σ\)\\alpha\>\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}\(\\lambda,\\varphi,\\sigma\), the empirical correlationH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)admits a unique global maximizer𝒘^∈𝕊d−1\\widehat\{\\boldsymbol\{w\}\}\\in\\mathbb\{S\}^\{d\-1\}\. Furthermore,𝒘^\\widehat\{\\boldsymbol\{w\}\}is positively correlated with the true signal𝒘∗\\text\{$w$\}\_\{\*\}, withH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)being strongly concave in a neighborhood of𝒘^\\widehat\{\\boldsymbol\{w\}\}\. This local maximum is accessible by suitable polynomial\-time algorithms that only use gradient or Hessian information ofH^n\\widehat\{H\}\_\{n\}, such as AMP\.
2. 2\.Forα<αtr​\(λ,φ,σ\)\\alpha<\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}\(\\lambda,\\varphi,\\sigma\), either all near\-global maximizers are roughly orthogonal to𝒘∗\\text\{$w$\}\_\{\*\}, or a global maximizer exists and is positively correlated with𝒘∗\\text\{$w$\}\_\{\*\}, but it cannot be found by polynomial\-time algorithms or does not have a strongly concave neighborhood\.

It is also possible thatαtr=0\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}=0orαtr=∞\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}=\\infty\. We refer to the phase transition atαtr\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}as the ‘trivialization’ phase transition\. In statistical physics terminology, when the landscape presents multiple well\-separated near\-optima, with roughly independent random values \(as is the case forα<αtr\\alpha<\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}\), one speaks of ‘replica symmetry breaking\.’

Existing results successfully characterize the Bayes optimal estimation error or analyze ERM forα\>α¯tr\\alpha\>\\overline\{\\alpha\}\_\{\\mbox\{\\tiny\\rm tr\}\}\(withα¯tr\\overline\{\\alpha\}\_\{\\mbox\{\\tiny\\rm tr\}\}an upper bound on the trivialization thresholdαtr\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}\)\[[BKM\+19](https://arxiv.org/html/2606.28573#bib.bibx5),[VDR\+25](https://arxiv.org/html/2606.28573#bib.bibx41),[AMS26](https://arxiv.org/html/2606.28573#bib.bibx4),[MS26](https://arxiv.org/html/2606.28573#bib.bibx32)\]\. However, these works fall short of answering QuestionQin the most challenging regimeα<αtr\\alpha<\\alpha\_\{\\mbox\{\\tiny\\rm tr\}\}\.

![Refer to caption](https://arxiv.org/html/2606.28573v1/x1.png)Figure 1:Left panel:The limiting overlap with𝒘∗\\text\{$w$\}\_\{\*\}for Bayes AMP, ERM \(value achieved by our two\-stage algorithm\), and projected gradient descent \(PGD\)\.Right panel:The limiting training error \(normalized empirical correlation\) and test error \(normalized population correlation\) for these three methods\. These values are computed under the limit whereα→∞\\alpha\\to\\inftyandλ→0\\lambda\\to 0such thatα​λ→α¯\\alpha\\lambda\\to\\overline\{\\alpha\}\. See[Section5](https://arxiv.org/html/2606.28573#S5)for a comprehensive description of the experimental setup and implementation details\.To illustrate the content of QuestionQ, Figure[1](https://arxiv.org/html/2606.28573#S1.F1)reports the results of a numerical simulation\. We consider the ERM problem \([2](https://arxiv.org/html/2606.28573#S1.E2)\) with data generated usingφ​\(x\)=ReLU⁡\(x\)=max⁡\(x,0\)\\varphi\(x\)=\\operatorname\{ReLU\}\(x\)=\\max\(x,0\)and learning with the misspecified activation functionσ​\(x\)=x\+2​x2\\sigma\(x\)=x\+\\sqrt\{2\}x^\{2\}\. We evaluate the performance of three algorithms:\(i\)\(i\)Bayes AMP;\(i​i\)\(ii\)ERM computed via our two\-stage algorithm, which we expect to be optimal among polynomial\-time algorithms \(see[Section3](https://arxiv.org/html/2606.28573#S3)for details\); and\(i​i​i\)\(iii\)Projected gradient descent \(PGD\) minimizing−H^n\-\\widehat\{H\}\_\{n\}\(maximizingH^n\\widehat\{H\}\_\{n\}\) over the unit sphere\. For\(i\)\(i\)we use the theoretical predictions from\[[BKM\+19](https://arxiv.org/html/2606.28573#bib.bibx5)\], while for\(i​i\)\(ii\)we plot the predictions from the present work\. Finally, for\(i​i​i\)\(iii\)we plot the results of numerical experiments\.

We plot suitably normalized versions of train and test errors \(see[Section5\.5](https://arxiv.org/html/2606.28573#S5.SS5)for details\), as well as the correlation with the true signal𝒘∗\\text\{$w$\}\_\{\*\}attained by each of the three methods\. We observe that numerical simulations with PGD are closely matched by the theoretical predictions for the optimal two\-stage algorithm\. In other words, our theory appears to capture the behavior of suitably tuned gradient descent\. On the other hand, a substantial gap exists between the test error of ERM based methods \(either PGD or our two\-stage algorithm\) and that of the Bayes optimal AMP\.

### 1\.1Technical setting: Multi\-index model and ERM

In the remainder of this paper, we aim to answer QuestionQby delineating the computationally feasible training and test errors for a broad class of iterative algorithms, under the following Gaussian multi\-index model\.

###### Assumption 1\.1\.

\{\(𝒙i,yi\)\}i∈\[n\]\\\{\(\\text\{$x$\}\_\{i\},y\_\{i\}\)\\\}\_\{i\\in\[n\]\}are i\.i\.d\. data such that for eachi∈\[n\]i\\in\[n\],𝐱i∼𝖭​\(𝟎,𝐈d\)\\text\{$x$\}\_\{i\}\\sim\{\\sf N\}\(\\mathrm\{\\bf 0\},\{\\boldsymbol\{I\}\}\_\{d\}\),yi=φ​\(𝐖∗𝖳​𝐱i,εi\)y\_\{i\}=\\varphi\(\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\text\{$x$\}\_\{i\},\\varepsilon\_\{i\}\), where𝐖∗∈ℝd×k\\text\{$W$\}\_\{\*\}\\in\\mathbb\{R\}^\{d\\times k\}is a deterministic signal matrix andφ:ℝk×ℝ→ℝ\\varphi:\\mathbb\{R\}^\{k\}\\times\\mathbb\{R\}\\to\\mathbb\{R\}is a deterministic link function\. Without loss of generality, we assume that𝐖∗∈O​\(d,k\)\\text\{$W$\}\_\{\*\}\\in O\(d,k\), the space ofd×kd\\times korthogonal matrices \(cf\.[B\.1](https://arxiv.org/html/2606.28573#A2.Thmlem1)\)\. Further, denoting𝐲=\(yi\)i∈\[n\]\\text\{$y$\}=\(y\_\{i\}\)\_\{i\\in\[n\]\},𝛆=\(εi\)i∈\[n\]\{\\boldsymbol\{\\varepsilon\}\}=\(\\varepsilon\_\{i\}\)\_\{i\\in\[n\]\}and𝐗=\[𝐱1,⋯,𝐱n\]⊤\\text\{$X$\}=\[\\text\{$x$\}\_\{1\},\\cdots,\\text\{$x$\}\_\{n\}\]^\{\\top\}, we assume that the noise vector𝛆\{\\boldsymbol\{\\varepsilon\}\}is independent of the data matrix𝐗X, withεi∼i\.i\.d\.Pε\\varepsilon\_\{i\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}P\_\{\\varepsilon\}such that theyiy\_\{i\}’s are sub\-Gaussian\.

Givenh:ℝ×ℝm→ℝh:\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\\to\\mathbb\{R\}a bounded Lipschitz function, we aim to characterize the achievable values of random optimization problems of the form

maximize1n​∑i=1nh​\(yi,𝑾𝖳​𝒙i\),subject to​𝑾∈O​\(d,m\),\\displaystyle\\mbox\{maximize\}\\quad\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\big\(y\_\{i\},\\text\{$W$\}^\{\\sf T\}\\text\{$x$\}\_\{i\}\\big\),\\quad\\mbox\{subject to\}\\ \\text\{$W$\}\\in O\(d,m\),\(3\)in the proportional asymptotics wheren/d→α∈\(0,∞\)n/d\\to\\alpha\\in\(0,\\infty\)asn,d→∞n,d\\to\\infty, whilem,k=O​\(1\)m,k=O\(1\)\. We will be interested both in the asymptotics of the maximum value \(possibly subject to constraints on𝑾𝖳​𝑾∗\\text\{$W$\}^\{\{\\sf T\}\}\\text\{$W$\}\_\{\*\}\), and on the maximum value achievable by polynomial\-time algorithms\. Note that, although in[Eq\.3](https://arxiv.org/html/2606.28573#S1.E3)we only optimize over orthogonal matrices, general inner product structures of𝑾Wcan be encoded into the definition ofhh\.

In statistical learning applications, one normally takesh=−ℓh=\-\\ellfor some loss functionℓ\\ell, and minimizes the empirical risk

L^n​\(𝑾\)=1n​∑i=1nℓ​\(yi,𝑾𝖳​𝒙i\),subject to​𝑾∈O​\(d,m\)\.\\widehat\{L\}\_\{n\}\(\\text\{$W$\}\)=\\,\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\ell\\big\(y\_\{i\},\\text\{$W$\}^\{\\sf T\}\\text\{$x$\}\_\{i\}\\big\),\\quad\\mbox\{subject to\}\\ \\text\{$W$\}\\in O\(d,m\)\.\(4\)Obviously, this minimization problem is entirely equivalent to the maximization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\)\.

###### Remark 1\(Two\-layer neural networks\)\.

As a special case of the ERM problem \([4](https://arxiv.org/html/2606.28573#S1.E4)\), we consider training a two\-layer neural network with squared loss\. This example corresponds to minimizing the following empirical risk function

L^n​\(𝑼;𝒂\)=12​n​∑i=1n\(yi−∑j=1maj​σ​\(⟨𝒖j,𝒙i⟩\)\)2\\displaystyle\\widehat\{L\}\_\{n\}\(\\text\{$U$\};\\text\{$a$\}\)=\\frac\{1\}\{2n\}\\sum\_\{i=1\}^\{n\}\\left\(y\_\{i\}\-\\sum\_\{j=1\}^\{m\}a\_\{j\}\\sigma\(\\langle\\text\{$u$\}\_\{j\},\\text\{$x$\}\_\{i\}\\rangle\)\\right\)^\{2\}\\,\(5\)over parametersa1,…,am∈ℝa\_\{1\},\\dots,a\_\{m\}\\in\\mathbb\{R\}and𝒖1,…,𝒖m∈ℝd\\text\{$u$\}\_\{1\},\\dots,\\text\{$u$\}\_\{m\}\\in\\mathbb\{R\}^\{d\}\. This can be performed in two steps:\(i\)\(i\)Optimization over𝑾∈O​\(d,m\)\\text\{$W$\}\\in O\(d,m\), an orthogonal basis for the span of𝒖1,…,𝒖m∈ℝd\\text\{$u$\}\_\{1\},\\dots,\\text\{$u$\}\_\{m\}\\in\\mathbb\{R\}^\{d\};\(i​i\)\(ii\)Optimization overa1,…,am∈ℝa\_\{1\},\\dots,a\_\{m\}\\in\\mathbb\{R\}, as well as over the representation of\(𝒖j\)j≤m\(\\text\{$u$\}\_\{j\}\)\_\{j\\leq m\}in the basis𝑾W, namely over𝒃1,⋯,𝒃m∈ℝm\\text\{$b$\}\_\{1\},\\cdots,\\text\{$b$\}\_\{m\}\\in\\mathbb\{R\}^\{m\}such that𝒖j=𝑾𝒃j\\text\{$u$\}\_\{j\}=\\text\{$W$\}\\text\{$b$\}\_\{j\}forj=1,⋯,mj=1,\\cdots,m\.

The inner optimization \(optimizing over𝑾Wat fixed𝒂,𝑩:=\(𝒃j\)j≤m\\text\{$a$\},\\text\{$B$\}:=\(\\text\{$b$\}\_\{j\}\)\_\{j\\leq m\}\) corresponds to the setting of our paper with the following choice ofℓ\\ell:

ℓ​\(y;𝒛\)=12​\(y−∑j=1maj​σ​\(⟨𝒃j,𝒛⟩\)\)2\.\\displaystyle\\ell\\big\(y;\\text\{$z$\}\\big\)=\\frac\{1\}\{2\}\\left\(y\-\\sum\_\{j=1\}^\{m\}a\_\{j\}\\sigma\(\\langle\\text\{$b$\}\_\{j\},\\text\{$z$\}\\rangle\)\\right\)^\{2\}\\,\.\(6\)We note that the outer step is substantially easier both from an optimization viewpoint, because it is a constant\-dimensional optimization problem, and from a statistical viewpoint\. Indeed, by Gaussian concentration, the optimal cost at fixed\(𝒂,𝑩\)\(\\text\{$a$\},\\text\{$B$\}\)converges uniformly to its expectation, asn,d→∞n,d\\to\\infty\. Hence, once we characterize the asymptotics of the inner optimization problem, the overall result follows by optimizing over the low\-dimensional parameters𝒂,𝑩\\text\{$a$\},\\text\{$B$\}\.

### 1\.2Connection with projection pursuit

The high\-dimensional asymptotics of the random optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\) is intimately related to projection pursuit, which aims to find low\-dimensional projections that are ‘informative’ or ‘atypical’\[[FT74](https://arxiv.org/html/2606.28573#bib.bibx16),[DF84](https://arxiv.org/html/2606.28573#bib.bibx12),[MZ22](https://arxiv.org/html/2606.28573#bib.bibx34)\]\. In order to formalize the projection pursuit problem, and clarify its relation to empirical risk minimization, let us define the set of feasible empirical joint distributions of theyiy\_\{i\}’s and low\-dimensional projections of the𝒙i\\text\{$x$\}\_\{i\}’s as

ℱm,α,φ:=\{P∈𝒫\(ℝ×ℝm\):∃𝑾=𝑾n,d​\(𝑿,𝒚,ω\)∈O​\(d,m\),s\.t\.1n∑i=1nδ\(yi,𝑾𝖳​𝒙i\)→wPin probability\},\\begin\{split\}\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}:=\\Big\\\{P\\in\\mathscr\{P\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\):\\,&\\exists\\text\{$W$\}=\\text\{$W$\}\_\{n,d\}\(\\text\{$X$\},\\text\{$y$\},\\omega\)\\in O\(d,m\),\\,\\,\\\\ &\\mbox\{s\.t\.\}\\,\\,\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\(y\_\{i\},\\text\{$W$\}^\{\\sf T\}\\text\{$x$\}\_\{i\}\)\}\\stackrel\{\{\\scriptstyle w\}\}\{\{\\to\}\}P\\,\\mbox\{ in probability\}\\Big\\\}\\,,\\end\{split\}\(7\)whereω\\omegarepresents some additional randomness independent of\(𝑿,𝒚\)\(\\text\{$X$\},\\text\{$y$\}\)\(without loss of generality, we takeω∼Unif​\(\[0,1\]\)\\omega\\sim\\mathrm\{Unif\}\(\[0,1\]\)\), and𝒫​\(ℝ×ℝm\)\\mathscr\{P\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\)is the set of all probability measures onℝ×ℝm\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\.

We emphasize one important feature of the feasible setℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}defined in \([7](https://arxiv.org/html/2606.28573#S1.E7)\), namely𝑾=𝑾n,d​\(𝑿,𝒚,ω\)\\text\{$W$\}=\\text\{$W$\}\_\{n,d\}\(\\text\{$X$\},\\text\{$y$\},\\omega\)does not depend on𝑾∗\\text\{$W$\}\_\{\*\}and hence is a true statistical estimator\.

To establish the connection between the random optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\) and the feasible setℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}, we define the dual functional𝒱m,α,φ​\(⋅\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(\\cdot\)ofℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}as

𝒱m,α,φ​\(h\):=supP∈ℱm,α,φ\{∫ℝ×ℝmh​\(y,z\)​P​\(d​y,d​z\)\},∀h∈Cb​\(ℝ×ℝm\),\\displaystyle\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\):=\\sup\_\{P\\in\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}\}\\left\\\{\\int\_\{\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\}h\(y,z\)P\(\\textup\{d\}y,\\textup\{d\}z\)\\right\\\},\\quad\\forall h\\in C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\),\(8\)whereCb​\(ℝ×ℝm\)C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\)denotes the collection of bounded continuous functions onℝ×ℝm\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\. According to\[[MZ22](https://arxiv.org/html/2606.28573#bib.bibx34), Proposition 4\.1\], for anyh∈Cb​\(ℝ×ℝm\)h\\in C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\),

p−lim​infn/d→α⁡max𝑾∈O​\(d,m\)⁡1n​∑i=1nh​\(yi,𝑾𝖳​𝒙i\)=𝒱m,α,φ​\(h\)\.\\operatorname\*\{p\-lim\\,inf\}\_\{n/d\\rightarrow\\alpha\}\\max\_\{\\text\{$W$\}\\in O\(d,m\)\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\text\{$W$\}^\{\\sf T\}\\text\{$x$\}\_\{i\}\\right\)=\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)\.\(9\)Therefore, characterizing the feasible setℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}would allow us to determine𝒱m,α,φ​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\), the asymptotics of the global maximum for all problems of the form \([3](https://arxiv.org/html/2606.28573#S1.E3)\)\. On the other hand, a direct application of geometric Hahn\-Banach theorem implies that, determining𝒱m,α,φ​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)for allh∈Cb​\(ℝ×ℝm\)h\\in C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\)provides a complete characterization of the closed convex hull ofℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}\(cf\.\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Theorem 1\.1\]\)\.

As emphasized above, we are equally interested in the values of the optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\) that can be achieved in polynomial time\. Correspondingly, we defineℱm,α,φalg\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}, the subset ofℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}that can be realized via polynomial\-time computable projections, namely

ℱm,α,φalg:=\{P∈𝒫\(ℝ×ℝm\):∃𝑾=𝑾n,d​\(𝑿,𝒚,ω\)∈O​\(d,m\)​polytime computable,s\.t\.1n∑i=1nδ\(yi,𝑾𝖳​𝒙i\)→wPin probability\}\.\\begin\{split\}\\mathscr\{F\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\_\{m,\\alpha,\\varphi\}:=\\Big\\\{P\\in\\mathscr\{P\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\):\\,&\\exists\\text\{$W$\}=\\text\{$W$\}\_\{n,d\}\(\\text\{$X$\},\\text\{$y$\},\\omega\)\\in O\(d,m\)\\mbox\{ polytime computable\},\\\\ &\\mbox\{s\.t\.\}\\,\\,\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\(y\_\{i\},\\text\{$W$\}^\{\\sf T\}\\text\{$x$\}\_\{i\}\)\}\\stackrel\{\{\\scriptstyle w\}\}\{\{\\to\}\}P\\,\\mbox\{ in probability\}\\Big\\\}\\,\.\\end\{split\}\(10\)In the above display,𝑾n,d​\(𝑿,𝒚,ω\)\\text\{$W$\}\_\{n,d\}\(\\text\{$X$\},\\text\{$y$\},\\omega\)is ‘polytime computable’ if there exists an algorithm accepting\(𝑿,𝒚,ω\)\(\\text\{$X$\},\\text\{$y$\},\\omega\)\(or its finite\-precision approximation\) as input and computing𝑾n,d​\(𝑿,𝒚,ω\)\\text\{$W$\}\_\{n,d\}\(\\text\{$X$\},\\text\{$y$\},\\omega\)in time at most polynomial inn,dn,d\. The associated dual functional is then defined as

𝒱m,α,φalg​\(h\):=supP∈ℱm,α,φalg\{∫ℝ×ℝmh​\(y,z\)​P​\(d​y,d​z\)\},∀h∈Cb​\(ℝ×ℝm\)\.\\displaystyle\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\(h\):=\\sup\_\{P\\in\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\}\\left\\\{\\int\_\{\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\}h\(y,z\)P\(\\textup\{d\}y,\\textup\{d\}z\)\\right\\\},\\quad\\forall h\\in C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\)\.\(11\)Analogous to[Eq\.9](https://arxiv.org/html/2606.28573#S1.E9), there is a direct correspondence between precisely characterizing the algorithmically feasible setℱm,α,φalg\\mathscr\{F\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\_\{m,\\alpha,\\varphi\}and exactly determining the values of the optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\) attainable via polynomial\-time algorithms\.

### 1\.3General approach

Over the last few years, a general approach has emerged to characterize the optimal value achieved by polynomial\-time algorithms in certain classes of random optimization problems\. This approach has two components \(see also\[[AMS23](https://arxiv.org/html/2606.28573#bib.bibx3),[Mon24](https://arxiv.org/html/2606.28573#bib.bibx29)\]for reviews\):

Achievability\.A rich class of first\-order optimization algorithms can be constructed using approximate message passing \(AMP\) or related strategies\[[CMW20](https://arxiv.org/html/2606.28573#bib.bibx10),[Sub21](https://arxiv.org/html/2606.28573#bib.bibx38),[Mon19](https://arxiv.org/html/2606.28573#bib.bibx28),[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14),[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\]\. Designing the optimal algorithm in this class is equivalent to solving a certain \(low dimensional\) stochastic optimal control problem\. This optimal control problem is dual to a Parisi\-style formula, thus connecting directly to fundamental concepts in spin glass theory\. Solving the optimal control problem yields the predicted algorithmic threshold\.

Impossibility\.Matching impossibility results are proven for algorithms that act as Lipschitz continuous functions of their input \(in our case,\(𝑿,𝒚\)\(\\text\{$X$\},\\text\{$y$\}\)\), with a bounded Lipschitz constant under suitable normalization\. The proof technique relies on establishing that near\-optima exhibit a certain geometric property—the ‘overlap gap property’—that cannot be reproduced by the output of Lipschitz algorithms\. This approach was initiated in\[[GS14](https://arxiv.org/html/2606.28573#bib.bibx19)\], applied to spin glass models related to the present setting in\[[GJ21](https://arxiv.org/html/2606.28573#bib.bibx17)\], and a sharp version based on the ‘branching overlap gap property’ was recently developed in\[[HS22](https://arxiv.org/html/2606.28573#bib.bibx21),[HS24](https://arxiv.org/html/2606.28573#bib.bibx22)\]\.

In this paper, we carry out the first part of this general program for the problem of learning a multi\-index model\. As the first statistical application of these techniques, our setting presents several distinctive challenges\. Most notably, we must account for the fact that we only have access to the empirical data\(𝑿,𝒚\)\(\\text\{$X$\},\\text\{$y$\}\), and not the true underlying parameters𝑾∗\\text\{$W$\}\_\{\*\}\.

### 1\.4Summary of main results and paper organization

The main contributions of this paper are summarized below:

Conjectures from statistical physics\.In Section[2](https://arxiv.org/html/2606.28573#S2), we use the non\-rigorous replica method from statistical physics to derive a Parisi\-type variational principle, which jointly predicts the feasible setℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}and𝒱m,α,φ​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\), the asymptotic value of \([3](https://arxiv.org/html/2606.28573#S1.E3)\)\. These predictions are formalized in Conjectures[2\.1](https://arxiv.org/html/2606.28573#S2.Thmconj1)and[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2), and serve as a useful benchmark for our rigorous results\.

Algorithmic achievability\.In Section[3](https://arxiv.org/html/2606.28573#S3), we present our two\-stage AMP algorithm and rigorously characterize its asymptotics when applied to the problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\)\. This analysis yields an explicit inner bound on the set of feasible distributionsℱm,α,φalg\\mathscr\{F\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\_\{m,\\alpha,\\varphi\}\([3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)\), which we expect to be tight over all polynomial\-time algorithms\.

Asymptotics of test and train error\.In particular, we precisely characterize the asymptotic training and test errors achievable by the two\-stage AMP algorithm for the ERM of Eq\. \([4](https://arxiv.org/html/2606.28573#S1.E4)\), in the high\-dimensional proportional limitn,d→∞n,d\\to\\inftywithn/d→αn/d\\to\\alpha\(see[3](https://arxiv.org/html/2606.28573#Thmrem3)\)\.

Largen/dn/dand tensor PCA equivalence\.As a special case, in Section[5](https://arxiv.org/html/2606.28573#S5)we derive simple expressions for the algorithmically achievable training and test errors in the single\-index model with correlation loss \(cf\.[Eq\.1](https://arxiv.org/html/2606.28573#S1.E1)\), under the double asymptoticsα→∞\\alpha\\to\\inftyandλ→0\\lambda\\to 0\. Interestingly, our results for the single\-index model admit a natural interpretation via an equivalent generalized tensor PCA problem, which we elaborate in[Section5\.4](https://arxiv.org/html/2606.28573#S5.SS4)\.

Duality and Parisi\-style formulas\.Our algorithmic inner bound onℱm,α,φalg\\mathscr\{F\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\_\{m,\\alpha,\\varphi\}yields a corresponding lower bound on the dual functional𝒱m,α,φalg​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\(h\), which we denote by𝒱m,α,φAMP​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)\. Namely,𝒱m,α,φAMP​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)is the limiting optimal value of the optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\) achieved by our two\-stage AMP algorithm\. In the casek=m=1k=m=1, we establish a Parisi\-type variational principle for𝒱1,α,φAMP​\(h\)\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)over an extended function space \([6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1)\)\.

[Section7](https://arxiv.org/html/2606.28573#S7)formally defines the two\-stage AMP algorithm and provides the proof of[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1), establishing our general AMP achievability results for bothℱm,α,φalg\\mathscr\{F\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\_\{m,\\alpha,\\varphi\}and𝒱m,α,φalg​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\(h\)\. In[Section8](https://arxiv.org/html/2606.28573#S8), we present the necessary technical preliminaries regarding the Parisi functional and prove[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1), the extended variational principle for𝒱m,α,φAMP​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)\. The proofs of all auxiliary lemmas and additional technical details are deferred to the appendices\.

### 1\.5Definitions and notation

We will follow the convention of using boldface letters for matrices or vectors whose dimensions diverge asn,d→∞n,d\\to\\infty, and normal fonts otherwise\. We denote the standard scalar product between two vectors𝒖,𝒗\\text\{$u$\},\\text\{$v$\}by⟨𝒖,𝒗⟩\\langle\\text\{$u$\},\\text\{$v$\}\\rangle, and the matrix scalar product by⟨𝑨,𝑩⟩=Tr​\(𝑨𝖳​𝑩\)\\langle\\text\{$A$\},\\text\{$B$\}\\rangle=\\mathrm\{Tr\}\(\\text\{$A$\}^\{\\sf T\}\\text\{$B$\}\)\. We use∥⋅∥2\\left\\\|\{\\cdot\}\\right\\\|\_\{2\}to denote the Euclidean norm of a vector\. We use∥⋅∥Lp\\left\\\|\{\\cdot\}\\right\\\|\_\{L^\{p\}\}to denote the standardLpL^\{p\}norm of a function forp∈\[1,\+∞\]p\\in\[1,\+\\infty\]\.

We denote by𝒮\+m\{\\mathcal\{S\}\}\_\{\+\}^\{m\}the convex cone ofm×mm\\times mpositive semi\-definite matrices\. Ford≥md\\geq m, we denote byO​\(d,m\)O\(d,m\)the set of alld×md\\times morthogonal matrices\. For a subsetSSin a topological space, we denote bycl​\(S\)\\mathrm\{cl\}\\ \(S\)its closure\. Forp≥1p\\geq 1, we denote byCk​\(ℝp\)C^\{k\}\(\\mathbb\{R\}^\{p\}\)the collection of all functions that have continuouskk\-th derivatives inℝp\\mathbb\{R\}^\{p\}\. We also denote byCb​\(ℝp\)C\_\{b\}\(\\mathbb\{R\}^\{p\}\)the set of all bounded continuous functions onℝp\\mathbb\{R\}^\{p\}, and byCc∞​\(ℝp\)C\_\{c\}^\{\\infty\}\(\\mathbb\{R\}^\{p\}\)the set of all infinitely differentiable functions with compact supports\. We use𝒫​\(ℝp\)\\mathscr\{P\}\(\\mathbb\{R\}^\{p\}\)to denote the set of all probability measures onℝp\\mathbb\{R\}^\{p\}equipped with the topology of weak convergence, unless otherwise stated\. For a sequence of probability measures\(νn\)n≥1\(\\nu\_\{n\}\)\_\{n\\geq 1\}and a probability measureν\\nu, we writeνn→wν\\nu\_\{n\}\\stackrel\{\{\\scriptstyle w\}\}\{\{\\to\}\}\\nuifνn\\nu\_\{n\}converges weakly toν\\nu\. We useLaw​\(X\)\\mathrm\{Law\}\(X\)to denote the law of a random variableXX\. For two random variablesXXandYY, we writeX⟂⟂YX\\perp\\\!\\\!\\\!\\perp YifXXis independent ofYY\.

Ford∈ℕd\\in\\mathbb\{N\}andr≥0r\\geq 0,Bd​\(r\)=\{x∈ℝd:‖x‖2≤r\}B\_\{d\}\(r\)=\\\{x\\in\\mathbb\{R\}^\{d\}:\\left\\\|\{x\}\\right\\\|\_\{2\}\\leq r\\\}\. For a functionhh, we denote byconc⁡\(h\)\\operatorname\{conc\}\(h\)the \(upper\) concave envelope ofhh\. Namely,conc⁡\(h\)\\operatorname\{conc\}\(h\)is the pointwise minimum of all concave functions that dominatehh\. Forl,k≥1l,k\\geq 1, and a differentiable mappingF:ℝk→ℝlF:\\mathbb\{R\}^\{k\}\\to\\mathbb\{R\}^\{l\}, we denote byJF∈ℝl×kJ\_\{F\}\\in\\mathbb\{R\}^\{l\\times k\}the Jacobian matrix ofFF, namely forx∈ℝkx\\in\\mathbb\{R\}^\{k\}:JF​\(x\)i​j=∂Fi/∂xjJ\_\{F\}\(x\)\_\{ij\}=\\partial F\_\{i\}/\\partial x\_\{j\}\. We occasionally useJFJ\_\{F\}as a shorthand forJF​\(x\)J\_\{F\}\(x\)whenever the variablexxis clear from the context\. We say that a functionψ:ℝp→ℝ\\psi:\\mathbb\{R\}^\{p\}\\to\\mathbb\{R\}is pseudo\-Lipschitz if there exists a constantCCsuch that, for allx,y∈ℝpx,y\\in\\mathbb\{R\}^\{p\},

\|ψ​\(x\)−ψ​\(y\)\|≤C​\(1\+‖x‖2\+‖y‖2\)​‖x−y‖2\.\|\\psi\(x\)\-\\psi\(y\)\|\\leq C\(1\+\\\|x\\\|\_\{2\}\+\\\|y\\\|\_\{2\}\)\\\|x\-y\\\|\_\{2\}\.Let\{Bt\}t∈\[0,1\]\\\{B\_\{t\}\\\}\_\{t\\in\[0,1\]\}be anmm\-dimensional standard Brownian motion, and let\{ℱt\}t∈\[0,1\]\\\{\\mathcal\{F\}\_\{t\}\\\}\_\{t\\in\[0,1\]\}be its canonical filtration\. Fors≤ts\\leq t, we denote byD​\[s,t\]D\[s,t\]the space of all admissible controls on the interval\[s,t\]\[s,t\], i\.e\., the collection of all progressively measurable processes\{Φr\}s≤r≤t\\\{\\Phi\_\{r\}\\\}\_\{s\\leq r\\leq t\}satisfying

σ​\(Φr\)⊂ℱr,∀r∈\[s,t\],and​𝔼​\[∫stΦr​Φr𝖳​d​r\]<∞\.\\sigma\(\\Phi\_\{r\}\)\\subset\\mathcal\{F\}\_\{r\},\\ \\forall r\\in\[s,t\],\\ \\mbox\{and\}\\ \\mathbb\{E\}\\left\[\\int\_\{s\}^\{t\}\\Phi\_\{r\}\\Phi\_\{r\}^\{\\sf T\}\\textup\{d\}r\\right\]<\\infty\.\(12\)

## 2Conjectures from statistical physics

In this section, we derive a prediction for𝒱m,α,φ​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)using physicists’ replica method, with detailed calculations deferred to Appendix[A](https://arxiv.org/html/2606.28573#A1)\. Recall that𝒱m,α,φ​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)is defined in Eq\. \([8](https://arxiv.org/html/2606.28573#S1.E8)\)\.

###### Conjecture 2\.1\(Replica prediction for𝒱m,α,φ​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)\)\.

For anyh∈Cb​\(ℝ×ℝm\)h\\in C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\), almost surely

limn→∞max𝑾∈O​\(d,m\)⁡1n​∑i=1nh​\(yi,𝑾𝖳​𝒙i\)=𝒱m,α,φ​\(h\)\.\\lim\_\{n\\to\\infty\}\\max\_\{\\text\{$W$\}\\in O\(d,m\)\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\text\{$W$\}^\{\\sf T\}\\text\{$x$\}\_\{i\}\\right\)=\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)\\,\.Further,𝒱m,α,φ​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)has the following variational representation:

𝒱m,α,φ​\(h\)=supR∈ℬk,minf\(μ,M,C\)∈𝒰×ℐm​\(R\)×𝒮\+m𝖥m​\(μ,M,C,R\)\.\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}\(h\)=\\sup\_\{R\\in\\mathscr\{B\}\_\{k,m\}\}\\inf\_\{\(\\mu,M,C\)\\in\\mathscr\{U\}\\times\\mathscr\{I\}\_\{m\}\(R\)\\times\{\\mathcal\{S\}\}\_\{\+\}^\{m\}\}\\mathsf\{F\}\_\{m\}\(\\mu,M,C,R\)\.In the above display,𝒮\+m\{\\mathcal\{S\}\}\_\{\+\}^\{m\}denotes the convex cone ofm×mm\\times mpositive semi\-definite matrices, and

ℬk,m\\displaystyle\\mathscr\{B\}\_\{k,m\}:=\{R∈ℝk×m:R𝖳​R⪯Im\},\\displaystyle:=\\left\\\{R\\in\\mathbb\{R\}^\{k\\times m\}:R^\{\\sf T\}R\\preceq I\_\{m\}\\right\\\},ℐm​\(R\)\\displaystyle\\mathscr\{I\}\_\{m\}\(R\):=\{M:\[0,1\)→𝒮\+m:∫01M​\(t\)​d​t=Im−R𝖳​R\},\\displaystyle:=\\left\\\{M:\[0,1\)\\to\{\\mathcal\{S\}\}\_\{\+\}^\{m\}:\\int\_\{0\}^\{1\}M\(t\)\\,\\textup\{d\}t=I\_\{m\}\-R^\{\\sf T\}R\\right\\\},𝒰\\displaystyle\\mathscr\{U\}:=\{μ:\[0,1\)→ℝ≥0:μ​non\-decreasing,∫01μ​\(t\)​d​t<∞\}\.\\displaystyle:=\\left\\\{\\mu:\[0,1\)\\to\\mathbb\{R\}\_\{\\geq 0\}:\\ \\mu\\ \\mbox\{\{\\rm non\-decreasing\}\},\\ \\int\_\{0\}^\{1\}\\mu\(t\)\\textup\{d\}t<\\infty\\right\\\}\.Themm\-dimensional Parisi functional𝖥m\\mathsf\{F\}\_\{m\}is defined as

𝖥m​\(μ,M,C,R\)=𝔼Y,G​\[fY,μ​\(0,R𝖳​G\)\]\+12​α​∫01Tr​\(M​\(t\)​\(C\+∫t1μ​\(s\)​M​\(s\)​d​s\)−1\)​d​t,\\mathsf\{F\}\_\{m\}\(\\mu,M,C,R\)=\\,\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}\\left\(0,R^\{\\sf T\}G\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{0\}^\{1\}\\mathrm\{Tr\}\\left\(M\(t\)\\left\(C\+\\int\_\{t\}^\{1\}\\mu\(s\)M\(s\)\\textup\{d\}s\\right\)^\{\-1\}\\right\)\\textup\{d\}t,where the random vector\(Y,G\)\(Y,G\)is distributed as follows:

G∼𝖭\(0,Ik\),Y=φ\(G,ε\),ε∼Pε,ε⟂⟂G\.G\\sim\{\\sf N\}\(0,I\_\{k\}\),\\quad Y=\\varphi\(G,\\varepsilon\),\\,\\varepsilon\\sim P\_\{\\varepsilon\},\\,\\varepsilon\\perp\\\!\\\!\\\!\\perp G\.Further, for any fixed\(y,μ\)\(y,\\mu\),fy,μ:\[0,1\]×ℝm→ℝf\_\{y,\\mu\}:\[0,1\]\\times\\mathbb\{R\}^\{m\}\\to\\mathbb\{R\}solves the followingmm\-dimensional Parisi PDE:

∂tfy,μ​\(t,x\)\+12​μ​\(t\)​⟨∇xfy,μ​\(t,x\),M​\(t\)​∇xfy,μ​\(t,x\)⟩\+12​Tr⁡\(M​\(t\)​∇x2fy,μ​\(t,x\)\)=0,fy,μ​\(1,x\)=supu∈ℝm\{h​\(y,x\+u\)−12​⟨u,C−1​u⟩\}\.\\begin\{split\}\\partial\_\{t\}f\_\{y,\\mu\}\(t,x\)\+\\,&\\frac\{1\}\{2\}\\mu\(t\)\\langle\\nabla\_\{x\}f\_\{y,\\mu\}\(t,x\),M\(t\)\\nabla\_\{x\}f\_\{y,\\mu\}\(t,x\)\\rangle\+\\frac\{1\}\{2\}\\operatorname\{Tr\}\\left\(M\(t\)\\nabla\_\{x\}^\{2\}f\_\{y,\\mu\}\(t,x\)\\right\)=\\,0,\\\\ f\_\{y,\\mu\}\(1,x\)=\\,&\\sup\_\{u\\in\\mathbb\{R\}^\{m\}\}\\left\\\{h\\left\(y,\\,x\+u\\right\)\-\\frac\{1\}\{2\}\\langle u,C^\{\-1\}u\\rangle\\right\\\}\\,\.\\end\{split\}\(13\)\(For notational simplicity, we omit the dependence offy,μf\_\{y,\\mu\}on\(M,C\)\(M,C\)\.\)

Form=1m=1, the replica formulas in[2\.1](https://arxiv.org/html/2606.28573#S2.Thmconj1)reduce to the following simpler form:

###### Conjecture 2\.2\(Replica prediction for𝒱1,α,φ​\(h\)\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}\(h\)\)\.

For anyh∈Cb​\(ℝ×ℝ\)h\\in C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}\), almost surely

limn→∞max𝒘∈𝕊d−1⁡1n​∑i=1nh​\(yi,⟨𝒘,𝒙i⟩\)=𝒱1,α,φ​\(h\)\.\\lim\_\{n\\to\\infty\}\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\langle\\text\{$w$\},\\text\{$x$\}\_\{i\}\\rangle\\right\)=\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}\(h\)\\,\.Further,𝒱1,α,φ​\(h\)\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}\(h\)has the following variational representation:

𝒱1,α,φ​\(h\)=supr∈Bk​\(1\)inf\(μ,c\)∈𝒰×ℝ\>0𝖥1​\(μ,c,r\)\.\\displaystyle\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}\(h\)=\\sup\_\{r\\in B\_\{k\}\(1\)\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\_\{1\}\(\\mu,c,r\)\.In the above display, the Parisi functional𝖥1:𝒰×ℝ\>0×Bk​\(1\)→ℝ\\mathsf\{F\}\_\{1\}:\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\\times B\_\{k\}\(1\)\\to\\mathbb\{R\}is defined as follows: Let\(Y,G\)\(Y,G\)be such thatG∼𝖭​\(0,Ik\)G\\sim\{\\sf N\}\(0,I\_\{k\}\),Y=φ​\(G,ε\)Y=\\varphi\(G,\\varepsilon\)whereε∼Pε\\varepsilon\\sim P\_\{\\varepsilon\}is independent ofGG\. For any fixedyyandμ\\mu, letfy,μ​\(t,x\)f\_\{y,\\mu\}\(t,x\)be the solution to the following Parisi PDE:

∂tfy,μ​\(t,x\)\+12​μ​\(t\)​\(∂xfy,μ​\(t,x\)\)2\+12​∂x2fy,μ​\(t,x\)=0,fy,μ​\(1,x\)=supu∈ℝ\{h​\(y,x\+u\)−u22​c\}\.\\begin\{split\}&\\partial\_\{t\}f\_\{y,\\mu\}\(t,x\)\+\\frac\{1\}\{2\}\\mu\(t\)\(\\partial\_\{x\}f\_\{y,\\mu\}\(t,x\)\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,\\mu\}\(t,x\)=\\,0,\\\\ &f\_\{y,\\mu\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,x\+u\\right\)\-\\frac\{u^\{2\}\}\{2c\}\\right\\\}\.\\end\{split\}\(14\)Note thatfy,μ​\(t,x\)f\_\{y,\\mu\}\(t,x\)also depends oncc, although we suppress this dependence in its definition to avoid heavy notation\. Finally, we define

𝖥1​\(μ,c,r\)=𝔼Y,G​\[fY,μ​\(‖r‖22,r𝖳​G\)\]\+12​α​∫‖r‖221d​tc\+∫t1μ​\(u\)​d​u\.\\mathsf\{F\}\_\{1\}\(\\mu,c,r\)=\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}\\left\(\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\},r^\{\\sf T\}G\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(u\)\\textup\{d\}u\}\\,\.\(15\)

In the following sections, we will drop the subscript “11” and use𝖥​\(μ,c,r\)\\mathsf\{F\}\(\\mu,c,r\)instead of𝖥1​\(μ,c,r\)\\mathsf\{F\}\_\{1\}\(\\mu,c,r\)\.

## 3Main results \(I\): Algorithmic achievability

In this section, we present our main results regardingℱm,α,φalg\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}, the set of computationally feasible probability distributions inℱm,α,φ\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}\. In Section[3\.1](https://arxiv.org/html/2606.28573#S3.SS1), we describe the class of two\-stage AMP algorithms to be analyzed, with detailed analysis deferred to[Section7](https://arxiv.org/html/2606.28573#S7)\. Section[3\.2](https://arxiv.org/html/2606.28573#S3.SS2)presents the characterization of the set of probability distributions inℱm,α,φalg\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}that can be realized using this class of algorithms\. Section[3\.3](https://arxiv.org/html/2606.28573#S3.SS3)then states our main achievability result for𝒱m,α,φalg​\(h\)\\mathscr\{V\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\_\{m,\\alpha,\\varphi\}\(h\), the dual functional ofℱm,α,φalg\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\.

### 3\.1Overview of the algorithm

We give a brief description of our AMP algorithm, and refer to Section[7](https://arxiv.org/html/2606.28573#S7)for further details222The notation used here is primarily for simplicity of exposition, and may differ slightly from that in Section[7](https://arxiv.org/html/2606.28573#S7)\.\. Our algorithm has two stages\. The first stage consists ofT1T\_\{1\}iterations with fixed step size, followed by an incremental stage ofT2T\_\{2\}iterations with small step sizes, whereT1T\_\{1\}andT2T\_\{2\}are two positive integers \(independent ofnnanddd\) to be determined\.

The first stage of our algorithm consists ofT1T\_\{1\}AMP iterations, where thett\-th iteration only depends on the\(t−1\)\(t\-1\)\-th one: fort=1,⋯,T1t=1,\\cdots,T\_\{1\}, we update𝑽t∈ℝn×m\\text\{$V$\}^\{t\}\\in\\mathbb\{R\}^\{n\\times m\},𝑾t∈ℝd×m\\text\{$W$\}^\{t\}\\in\\mathbb\{R\}^\{d\\times m\}according to

𝑾t\+1\\displaystyle\\text\{$W$\}^\{t\+1\}=1n​𝑿𝖳​Ft​\(𝑽t,𝒚\)−𝑾t​Kt𝖳,\\displaystyle=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}^\{\\sf T\}F\_\{t\}\(\\text\{$V$\}^\{t\},\\text\{$y$\}\)\-\\text\{$W$\}^\{t\}K\_\{t\}^\{\\sf T\},𝑽t\\displaystyle\\text\{$V$\}^\{t\}=1n​𝑿𝑾t−dn​Ft−1​\(𝑽t−1,𝒚\),\\displaystyle=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}\\text\{$W$\}^\{t\}\-\\frac\{d\}\{n\}F\_\{t\-1\}\(\\text\{$V$\}^\{t\-1\},\\text\{$y$\}\)\\,,with𝑾1=𝑿𝖳​F0​\(𝒚\)/n\\text\{$W$\}^\{1\}=\\text\{$X$\}^\{\\sf T\}F\_\{0\}\(\\text\{$y$\}\)/\\sqrt\{n\}\. Here,Ft:ℝm×ℝ→ℝmF\_\{t\}:\\mathbb\{R\}^\{m\}\\times\\mathbb\{R\}\\to\\mathbb\{R\}^\{m\}is understood to be applied row\-wise\. Namely, for𝑽∈ℝn×m\\text\{$V$\}\\in\\mathbb\{R\}^\{n\\times m\}\(with rows𝒗i\\text\{$v$\}\_\{i\}\),𝒚∈ℝn\\text\{$y$\}\\in\\mathbb\{R\}^\{n\}\(with entriesyiy\_\{i\}\),Ft​\(𝑽,𝒚\)∈ℝn×mF\_\{t\}\(\\text\{$V$\},\\text\{$y$\}\)\\in\\mathbb\{R\}^\{n\\times m\}is the matrix whoseii\-th row isFt​\(𝒗i,yi\)F\_\{t\}\(\\text\{$v$\}\_\{i\},y\_\{i\}\)\. Further, the Onsager correction termKt∈ℝm×mK\_\{t\}\\in\\mathbb\{R\}^\{m\\times m\}is given by

Kt=1n​∑i=1n∂Ft∂𝒗it​\(𝒗it,yi\),K\_\{t\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\frac\{\\partial F\_\{t\}\}\{\\partial\\text\{$v$\}\_\{i\}^\{t\}\}\(\\text\{$v$\}\_\{i\}^\{t\},y\_\{i\}\),where𝒗it\\text\{$v$\}\_\{i\}^\{t\}is theii\-th row of𝑽t\\text\{$V$\}^\{t\}, and∂Ft∂𝒗it​\(𝒗it,yi\)\\frac\{\\partial F\_\{t\}\}\{\\partial\\text\{$v$\}\_\{i\}^\{t\}\}\(\\text\{$v$\}\_\{i\}^\{t\},y\_\{i\}\)denotes the Jacobian matrix ofFtF\_\{t\}with respect to𝒗it\\text\{$v$\}\_\{i\}^\{t\}\. We will show \(using general tools from the analysis of AMP algorithms\) that, with proper choices of the non\-linearities and in the limit of largeT1T\_\{1\}aftern,d→∞n,d\\to\\infty,n/d→αn/d\\to\\alpha, this iteration converges to an approximate fixed point, which will be the starting point of the second stage of our algorithm\.

In the second stage, we allow each iterate to depend on all previous ones\. Denote𝑾≤t=\(𝑾s\)1≤s≤t\\text\{$W$\}^\{\\leq t\}=\(\\text\{$W$\}^\{s\}\)\_\{1\\leq s\\leq t\}and𝑽≤t=\(𝑽s\)1≤s≤t\\text\{$V$\}^\{\\leq t\}=\(\\text\{$V$\}^\{s\}\)\_\{1\\leq s\\leq t\}\. We iterate, forT1\+1≤t≤T1\+T2T\_\{1\}\+1\\leq t\\leq T\_\{1\}\+T\_\{2\}:

𝑾t\+1=1n​𝑿𝖳​Ft​\(𝑽≤t,𝒚\)−∑s=1tGs​\(𝑾≤s\)​Kt,s𝖳,𝑽t=1n​𝑿​Gt​\(𝑾≤t\)−∑s=1tFs−1​\(𝑽≤s−1,𝒚\)​Dt,s𝖳,\\begin\{split\}\\text\{$W$\}^\{t\+1\}&=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}^\{\\sf T\}F\_\{t\}\(\\text\{$V$\}^\{\\leq t\},\\text\{$y$\}\)\-\\sum\_\{s=1\}^\{t\}G\_\{s\}\(\\text\{$W$\}^\{\\leq s\}\)K\_\{t,s\}^\{\\sf T\},\\\\ \\text\{$V$\}^\{t\}&=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}G\_\{t\}\(\\text\{$W$\}^\{\\leq t\}\)\-\\sum\_\{s=1\}^\{t\}F\_\{s\-1\}\(\\text\{$V$\}^\{\\leq s\-1\},\\text\{$y$\}\)D\_\{t,s\}^\{\\sf T\},\\end\{split\}\(16\)whereFt:\(ℝm\)t×ℝ→ℝmF\_\{t\}:\(\\mathbb\{R\}^\{m\}\)^\{t\}\\times\\mathbb\{R\}\\to\\mathbb\{R\}^\{m\}andGt:\(ℝm\)t→ℝmG\_\{t\}:\(\\mathbb\{R\}^\{m\}\)^\{t\}\\to\\mathbb\{R\}^\{m\}are also understood to act row\-wise, and

Kt,s=1n​∑i=1n∂Ft∂𝒗is​\(𝒗i1,⋯,𝒗it,yi\),Dt,s=1n​∑i=1d∂Gt∂𝒘is​\(𝒘i1,⋯,𝒘it\),t≥s\.K\_\{t,s\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\frac\{\\partial F\_\{t\}\}\{\\partial\\text\{$v$\}\_\{i\}^\{s\}\}\\left\(\\text\{$v$\}\_\{i\}^\{1\},\\cdots,\\text\{$v$\}\_\{i\}^\{t\},y\_\{i\}\\right\),\\ D\_\{t,s\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{d\}\\frac\{\\partial G\_\{t\}\}\{\\partial\\text\{$w$\}\_\{i\}^\{s\}\}\\left\(\\text\{$w$\}\_\{i\}^\{1\},\\cdots,\\text\{$w$\}\_\{i\}^\{t\}\\right\),\\quad t\\geq s\.As before, we will overload the notations and letFtF\_\{t\}andGtG\_\{t\}operate on their argument matrices row\-wise\. We further assume thatFtF\_\{t\}andGtG\_\{t\}take the following specific structure:

Ft​\(𝒗1,⋯,𝒗t,y\)=\\displaystyle F\_\{t\}\\left\(\\text\{$v$\}^\{1\},\\cdots,\\text\{$v$\}^\{t\},y\\right\)=\\,𝒗t​Φt−1​\(𝒗1,⋯,𝒗t−1,y\),\\displaystyle\\text\{$v$\}^\{t\}\\Phi\_\{t\-1\}\\left\(\\text\{$v$\}^\{1\},\\cdots,\\text\{$v$\}^\{t\-1\},y\\right\),Gt​\(𝒘1,⋯,𝒘t\)=\\displaystyle G\_\{t\}\\left\(\\text\{$w$\}^\{1\},\\cdots,\\text\{$w$\}^\{t\}\\right\)=\\,𝒘t​Ψt−1​\(𝒘1,⋯,𝒘t−1\),\\displaystyle\\text\{$w$\}^\{t\}\\Psi\_\{t\-1\}\\left\(\\text\{$w$\}^\{1\},\\cdots,\\text\{$w$\}^\{t\-1\}\\right\),whereΦt−1\\Phi\_\{t\-1\}andΨt−1\\Psi\_\{t\-1\}are matrix\-valued mappings that satisfy certain moment constraints, which we spell out in[Section7](https://arxiv.org/html/2606.28573#S7)\. The second stage of our algorithm involvesT2T\_\{2\}iterations with the above choices ofFtF\_\{t\}andGtG\_\{t\}\.

Finally, the output of our two\-stage AMP algorithm is a linear combination of𝑾T1\\text\{$W$\}^\{T\_\{1\}\}and the incremental AMP iterations in the second stage\. To be concrete, we will show that

p−limn,d→∞⁡1n​\(𝑾T1\)𝖳​𝑾T1=Q,\\operatorname\*\{p\-lim\}\_\{n,d\\to\\infty\}\\frac\{1\}\{n\}\(\\text\{$W$\}^\{T\_\{1\}\}\)^\{\{\\sf T\}\}\\text\{$W$\}^\{T\_\{1\}\}=Q,whereQ∈𝒮\+mQ\\in\{\\mathcal\{S\}\}\_\{\+\}^\{m\}is a deterministicm×mm\\times mmatrix satisfying0⪯Q⪯Im0\\preceq Q\\preceq I\_\{m\}, which will be characterized in[Section7\.2](https://arxiv.org/html/2606.28573#S7.SS2)\. LetQ1,⋯,QT2Q\_\{1\},\\cdots,Q\_\{T\_\{2\}\}beT2T\_\{2\}deterministicm×mm\\times mmatrices such that

∑t=1T2Qt𝖳​Qt=Im−Q,\\sum\_\{t=1\}^\{T\_\{2\}\}Q\_\{t\}^\{\\sf T\}Q\_\{t\}=I\_\{m\}\-Q,and define

𝑾Q=1n​𝑾T1\+1n​∑t=1T2GT1\+t\+1​\(𝑾≤T1\+t\+1\)​Qt\\text\{$W$\}\_\{Q\}=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$W$\}^\{T\_\{1\}\}\+\\frac\{1\}\{\\sqrt\{n\}\}\\sum\_\{t=1\}^\{T\_\{2\}\}G\_\{T\_\{1\}\+t\+1\}\\left\(\\text\{$W$\}^\{\\leq T\_\{1\}\+t\+1\}\\right\)Q\_\{t\}\\,and set the final output of our algorithm to be𝑾^nAMP=𝑾Q​\(𝑾Q𝖳​𝑾Q\)−1/2\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}=\\text\{$W$\}\_\{Q\}\(\\text\{$W$\}\_\{Q\}^\{\\sf T\}\\text\{$W$\}\_\{Q\}\)^\{\-1/2\}, which is guaranteed to be ad×md\\times morthogonal matrix\. The set of\(α,m\)\(\\alpha,m\)\-feasible distributions achieved by our algorithm will be studied in the next section\.

### 3\.2Achievability results for AMP

We begin by presenting our main AMP achievability result for generalmmandkk\. We assumem≥km\\geq kwithout loss of generality; ifm<km<k, any projection matrix𝑾∈O​\(d,m\)\\text\{$W$\}\\in O\(d,m\)can be embedded into ad×kd\\times korthogonal matrix by considering only its firstmmcolumns\.

###### Definition 1\(Bayes AMP feasible region\)\.

Let the random vector\(Y,G,Z\)∈ℝ×ℝk×ℝm\(Y,G,Z\)\\in\\mathbb\{R\}\\times\\mathbb\{R\}^\{k\}\\times\\mathbb\{R\}^\{m\}be such that

\(Y,G\)⟂⟂Z,Z∼𝖭\(0,Im\),G∼𝖭\(0,Ik\),Y=φ\(G,ε\),ε∼Pε,ε⟂⟂G\.\\begin\{split\}&\(Y,G\)\\perp\\\!\\\!\\\!\\perp Z,\\,Z\\sim\{\\sf N\}\(0,I\_\{m\}\),\\,G\\sim\{\\sf N\}\(0,I\_\{k\}\),\\\\ &Y=\\varphi\(G,\\varepsilon\),\\,\\varepsilon\\sim P\_\{\\varepsilon\},\\,\\varepsilon\\perp\\\!\\\!\\\!\\perp G\.\\end\{split\}\(17\)Assume that𝔼​\[G\|Y\]\\mathbb\{E\}\\left\[G\|Y\\right\]is non\-zero and defineCBayes∈ℝk×mC\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\in\\mathbb\{R\}^\{k\\times m\}via

𝒞Bayes\\displaystyle\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}:=\{Γ∈𝒮\+k:Γ=α​𝔼​\[𝔼​\[G−Γ1/2​Z\|\(Γ1/2\)𝖳​G\+Z,Y\]​𝔼​\[G−Γ1/2​Z\|\(Γ1/2\)𝖳​G\+Z,Y\]𝖳\]\},\\displaystyle:=\\Big\\\{\{\\Gamma\}\\in\{\\mathcal\{S\}\}\_\{\+\}^\{k\}:\\;\{\\Gamma\}=\\alpha\\mathbb\{E\}\\left\[\\mathbb\{E\}\\Big\[G\-\{\\Gamma\}^\{1/2\}Z\\Big\|\(\{\\Gamma\}^\{1/2\}\)^\{\{\\sf T\}\}G\+Z,Y\\Big\]\\mathbb\{E\}\\Big\[G\-\{\\Gamma\}^\{1/2\}Z\\Big\|\(\{\\Gamma\}^\{1/2\}\)^\{\{\\sf T\}\}G\+Z,Y\\Big\]^\{\\sf T\}\\right\]\\Big\\\}\\,,CBayes:=ΓBayes1/2,ΓBayes:=min⁡𝒞Bayes⪯,\\displaystyle C\_\{\\mbox\{\\tiny\\rm Bayes\}\}:=\{\\Gamma\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{1/2\}\\,,\\;\\;\\;\\;\\;\\;\{\\Gamma\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}:=\\min\{\}\_\{\\preceq\}\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\,,\(18\)wheremin⪯\\min\{\}\_\{\\preceq\}denotes the minimum with respect to the semidefinite \(Loewner\) order\. It is understood that, forΓ∈ℝk×k\{\\Gamma\}\\in\\mathbb\{R\}^\{k\\times k\},C=Γ1/2∈ℝk×mC=\{\\Gamma\}^\{1/2\}\\in\\mathbb\{R\}^\{k\\times m\}is any matrix such thatC​C𝖳=ΓCC^\{\{\\sf T\}\}=\{\\Gamma\}\(by[B\.2](https://arxiv.org/html/2606.28573#A2.Thmlem2), the specific choice ofCCis immaterial to our results\)\.

We then define the*Bayes AMP feasible region*as

ABayes:=\{\(R,Q\)∈ℝk×m×𝒮\+m:R𝖳​R⪯Q⪯Im,R​\(Q−R𝖳​R\)−1​R𝖳⪯CBayes​CBayes𝖳\}\.A\_\{\\mbox\{\\tiny\\rm Bayes\}\}:=\\,\\left\\\{\(R,Q\)\\in\\mathbb\{R\}^\{k\\times m\}\\times\{\\mathcal\{S\}\}\_\{\+\}^\{m\}:\\,R^\{\\sf T\}R\\preceq Q\\preceq I\_\{m\},\\,R\(Q\-R^\{\\sf T\}R\)^\{\-1\}R^\{\\sf T\}\\preceq C\_\{\\mbox\{\\tiny\\rm Bayes\}\}C\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\sf T\}\\right\\\}\.If𝒞Bayes=∅\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\emptyset, the inequality constraintR​\(Q−R𝖳​R\)−1​R𝖳⪯CBayes​CBayes𝖳R\(Q\-R^\{\\sf T\}R\)^\{\-1\}R^\{\\sf T\}\\preceq C\_\{\\mbox\{\\tiny\\rm Bayes\}\}C\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\sf T\}in the above definition is vacuous and therefore removed\.

###### Remark 2\.

While the existence of the minimum in \([18](https://arxiv.org/html/2606.28573#S3.E18)\) is not immediately obvious, it is guaranteed by the following argument\. Define the sequence\{Γt\}t=0∞⊂𝒮\+k\\\{\{\\Gamma\}\_\{t\}\\\}\_\{t=0\}^\{\\infty\}\\subset\{\\mathcal\{S\}\}\_\{\+\}^\{k\}recursively via:

Γ0=0,Γt\+1=α𝔼\[𝔼\[G−Γt1/2Z\|\(Γt1/2\)𝖳G\+Z,Y\]𝔼\[G−Γt1/2Z\|\(Γt1/2\)𝖳G\+Z,Y\]𝖳\]=:𝖲\(Γt\)\.\\begin\{split\}\{\\Gamma\}\_\{0\}=\\,&0,\\\\ \{\\Gamma\}\_\{t\+1\}=\\,&\\alpha\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-\{\\Gamma\}^\{1/2\}\_\{t\}Z\\big\|\(\{\\Gamma\}^\{1/2\}\_\{t\}\)^\{\{\\sf T\}\}G\+Z,Y\\right\]\\mathbb\{E\}\\left\[G\-\{\\Gamma\}^\{1/2\}\_\{t\}Z\\big\|\(\{\\Gamma\}^\{1/2\}\_\{t\}\)^\{\{\\sf T\}\}G\+Z,Y\\right\]^\{\\sf T\}\\right\]=:\{\\sf S\}\(\{\\Gamma\}\_\{t\}\)\.\\end\{split\}\(19\)Then,[B\.2](https://arxiv.org/html/2606.28573#A2.Thmlem2)in the appendix implies thatΓ∗=limt→∞Γt\{\\Gamma\}\_\{\*\}=\\lim\_\{t\\to\\infty\}\{\\Gamma\}\_\{t\}exists, though it can be infinite\. Further,Γ∗∈𝒞Bayes\{\\Gamma\}\_\{\*\}\\in\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}by continuity of the mappingΓ↦𝖲​\(Γ\)\{\\Gamma\}\\mapsto\{\\sf S\}\(\{\\Gamma\}\)\. Finally, for anyΓ∗′∈𝒞Bayes\{\\Gamma\}\_\{\*\}^\{\\prime\}\\in\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}, we obviously haveΓ0⪯Γ∗′\{\\Gamma\}\_\{0\}\\preceq\{\\Gamma\}\_\{\*\}^\{\\prime\}and therefore \(by monotonicity of𝖲\{\\sf S\}with respect to the Loewner order established in[B\.2](https://arxiv.org/html/2606.28573#A2.Thmlem2)\)𝖲t​\(Γ0\)⪯𝖲t​\(Γ∗′\)\{\\sf S\}^\{t\}\(\{\\Gamma\}\_\{0\}\)\\preceq\{\\sf S\}^\{t\}\(\{\\Gamma\}^\{\\prime\}\_\{\*\}\)for alltt, i\.e\.,Γt⪯Γ∗′\{\\Gamma\}\_\{t\}\\preceq\{\\Gamma\}^\{\\prime\}\_\{\*\}for alltt\. By taking the limitt→∞t\\to\\infty, we conclude thatΓ∗⪯Γ∗′\{\\Gamma\}\_\{\*\}\\preceq\{\\Gamma\}^\{\\prime\}\_\{\*\}, whenceΓ∗=min⪯⁡𝒞Bayes\{\\Gamma\}\_\{\*\}=\\min\_\{\\preceq\}\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. We can therefore identifyΓBayes=Γ∗\{\\Gamma\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\{\\Gamma\}\_\{\*\}\.

We now introduce a family of Lipschitz functions, termed\(R,Q\)\(R,Q\)\-contractions, which are used in the first stage of our AMP algorithm\. The contraction property of these functions ensures that, during this stage, the AMP iterations converge to the solution of a set of fixed\-point equations\.

###### Definition 2\.

Let\(Y,G,Z\)\(Y,G,Z\)be as described in[Eq\.17](https://arxiv.org/html/2606.28573#S3.E17), and define

ZR,Q=R𝖳​G\+\(Q−R𝖳​R\)1/2​Z\.Z\_\{R,Q\}=R^\{\\sf T\}G\+\(Q\-R^\{\\sf T\}R\)^\{1/2\}Z\.We say that a Lipschitz functionF:ℝm×ℝ→ℝmF:\\mathbb\{R\}^\{m\}\\times\\mathbb\{R\}\\to\\mathbb\{R\}^\{m\}is an\(R,Q\)\(R,Q\)\-contraction, if

R𝖳=𝔼​\[∂F∂G​\(ZR,Q,φ​\(G,ε\)\)\],Q=R𝖳​R\+1α​𝔼​\[F​\(ZR,Q,φ​\(G,ε\)\)​F​\(ZR,Q,φ​\(G,ε\)\)𝖳\],\\displaystyle R^\{\\sf T\}=\\,\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial G\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\],\\quad Q=R^\{\\sf T\}R\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)F\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)^\{\\sf T\}\\right\],\(20\)and there exists someS∈𝒮\+mS\\in\{\\mathcal\{S\}\}\_\{\+\}^\{m\},S≻0S\\succ 0, such that

1α​𝔼​\[∂F∂ZR,Q​\(ZR,Q,φ​\(G,ε\)\)𝖳​S​∂F∂ZR,Q​\(ZR,Q,φ​\(G,ε\)\)\]⪯S\.\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{R,Q\}\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)^\{\\sf T\}S\\,\\frac\{\\partial F\}\{\\partial Z\_\{R,Q\}\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]\\preceq\\,S\.\(21\)\(Here, the partial derivatives are defined by treatingF​\(ZR,Q,φ​\(G,ε\)\)F\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\)as a function of the variables\(ZR,Q,G\)∈ℝm×ℝk\(Z\_\{R,Q\},G\)\\in\\mathbb\{R\}^\{m\}\\times\\mathbb\{R\}^\{k\}\.\)

We are now in position to state our main achievability results for AMP algorithms, whose proof is presented in[Section7](https://arxiv.org/html/2606.28573#S7)\. For the reader’s convenience, in Appendix[B\.2](https://arxiv.org/html/2606.28573#A2.SS2)we provide the explicit formulation of Theorem[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)for the special casem=k=1m=k=1\.

###### Theorem 3\.1\.

For any\(R,Q\)∈ABayes\(R,Q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, let\(Y,ZR,Q\)\(Y,Z\_\{R,Q\}\)be as described in[2](https://arxiv.org/html/2606.28573#Thmdefn2), andFFbe an\(R,Q\)\(R,Q\)\-contraction\. Let\(Bt\)0≤t≤1\(B\_\{t\}\)\_\{0\\leq t\\leq 1\}be anmm\-dimensional standard Brownian motion independent of\(Y,ZR,Q\)\(Y,Z\_\{R,Q\}\)\. Define the filtration\{ℱt\}0≤t≤1\\\{\\mathcal\{F\}\_\{t\}\\\}\_\{0\\leq t\\leq 1\}byℱt=σ​\(ZR,Q,Y,\(Bs\)0≤s≤t\)\\mathcal\{F\}\_\{t\}=\\sigma\\left\(Z\_\{R,Q\},Y,\(B\_\{s\}\)\_\{0\\leq s\\leq t\}\\right\)for all0≤t≤10\\leq t\\leq 1\.

AssumeQ​\(t\)∈L2​\(\[0,1\]→ℝm×m\)Q\(t\)\\in L^\{2\}\(\[0,1\]\\to\\mathbb\{R\}^\{m\\times m\}\)satisfies

∫01Q​\(t\)​Q​\(t\)𝖳​d​t=Im−Q,\\int\_\{0\}^\{1\}Q\(t\)Q\(t\)^\{\\sf T\}\\textup\{d\}t=I\_\{m\}\-Q,\(22\)and\{Φt\}0≤t≤1\\\{\\Phi\_\{t\}\\\}\_\{0\\leq t\\leq 1\}is anm×mm\\times mmatrix\-valued progressively measurable stochastic process with respect to the filtration\{ℱt\}0≤t≤1\\\{\\mathcal\{F\}\_\{t\}\\\}\_\{0\\leq t\\leq 1\}, satisfying

𝔼​\[Φt​Φt𝖳\]⪯Imα,∀0≤t≤1\.\\mathbb\{E\}\\left\[\\Phi\_\{t\}\\Phi\_\{t\}^\{\\sf T\}\\right\]\\preceq\\frac\{I\_\{m\}\}\{\\alpha\},\\;\\;\\;\\;\\forall 0\\leq t\\leq 1\\,\.\(23\)Then, there exists a two\-stage AMP algorithm \(described in[Section3\.1](https://arxiv.org/html/2606.28573#S3.SS1)and further detailed in[Section7](https://arxiv.org/html/2606.28573#S7)\) that outputs𝐖^nAMP∈O​\(d,m\)\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\in O\(d,m\), such that

- \(a\)limn→∞𝑾∗𝖳​𝑾^nAMP=R\\lim\_\{n\\to\\infty\}\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}=Ralmost surely\.
- \(b\)Defining U=ZR,Q\+1α​F​\(ZR,Q,Y\)\+∫01Q​\(t\)​\(Im\+Φt\)​d​Bt,U=\\,Z\_\{R,Q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{R,Q\},Y\\right\)\+\\int\_\{0\}^\{1\}Q\(t\)\\left\(I\_\{m\}\+\\Phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\},\(24\)then the following holds almost surely asn→∞n\\to\\infty: 1n​∑i=1nδ\(yi,\(𝑾^nAMP\)𝖳​𝒙i\)→wLaw⁡\(Y,U\),1n​∑i=1nh​\(yi,\(𝑾^nAMP\)𝖳​𝒙i\)→w𝔼​\[h​\(Y,U\)\],∀h∈Cb​\(ℝ×ℝm\)\.\\begin\{split\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\(y\_\{i\},\\,\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\)^\{\\sf T\}\\text\{$x$\}\_\{i\}\)\}\\stackrel\{\{\\scriptstyle w\}\}\{\{\\to\}\}\\,&\\operatorname\{Law\}\(Y,U\),\\\\ \\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\big\(y\_\{i\},\\,\\big\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\big\)^\{\\sf T\}\\text\{$x$\}\_\{i\}\\big\)\\stackrel\{\{\\scriptstyle w\}\}\{\{\\to\}\}\\,&\\mathbb\{E\}\[h\(Y,U\)\],\\quad\\forall h\\in C\_\{b\}\(\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\)\.\\end\{split\}Consequently, we haveLaw⁡\(Y,U\)∈ℱm,α,φalg\\operatorname\{Law\}\(Y,U\)\\in\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\.

###### Remark 3\(Test error\)\.

Leth=−ℓh=\-\\ellfor a given loss functionℓ\\ell, and consider the corresponding empirical risk minimization problem \([4](https://arxiv.org/html/2606.28573#S1.E4)\)\. We define the training and test errors of𝑾^nAMP\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}as follows:

L^n​\(𝑾^nAMP\)\\displaystyle\\widehat\{L\}\_\{n\}\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\):=1n​∑i=1nℓ​\(yi,\(𝑾^nAMP\)𝖳​𝒙i\),\\displaystyle:=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\ell\\Big\(y\_\{i\},\\,\\big\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\big\)^\{\\sf T\}\\text\{$x$\}\_\{i\}\\Big\)\\,,L​\(𝑾^nAMP\)\\displaystyle L\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\):=𝔼​\[ℓ​\(ytest,\(𝑾^nAMP\)𝖳​𝒙test\)\|𝒚,𝑿\],\\displaystyle:=\\mathbb\{E\}\\Big\[\\ell\\Big\(y\_\{\\mbox\{\\tiny\\rm test\}\},\\,\\big\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\big\)^\{\\sf T\}\\text\{$x$\}\_\{\\mbox\{\\tiny\\rm test\}\}\\Big\)\\Big\|\\text\{$y$\},\\text\{$X$\}\\Big\]\\,,where\(𝒙test,ytest\)\(\\text\{$x$\}\_\{\\rm test\},y\_\{\\rm test\}\)is a test data point independent of the training set\{\(𝒙i,yi\)\}i=1n\\\{\(\\text\{$x$\}\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}\.[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)then implies that, for any\(R,Q\)∈ABayes\(R,Q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}and\(Q​\(t\),Φt\)\(Q\(t\),\\Phi\_\{t\}\)satisfying Eqs\. \([22](https://arxiv.org/html/2606.28573#S3.E22)\) and \([23](https://arxiv.org/html/2606.28573#S3.E23)\), the following pair of training and test errors are achievable in the proportional limitn/d→αn/d\\to\\alpha\(withUUdefined as per Eq\. \([24](https://arxiv.org/html/2606.28573#S3.E24)\)\):

limn→∞L^n​\(𝑾^nAMP\)\\displaystyle\\lim\_\{n\\to\\infty\}\\widehat\{L\}\_\{n\}\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\)=𝔼​\[ℓ​\(Y,U\)\],\\displaystyle=\\mathbb\{E\}\[\\ell\(Y,U\)\]\\,,limn→∞L​\(𝑾^nAMP\)\\displaystyle\\lim\_\{n\\to\\infty\}L\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\)=𝔼​\[ℓ​\(Y,R𝖳​G\+\(Im−R𝖳​R\)1/2​Z\)\]\.\\displaystyle=\\mathbb\{E\}\\left\[\\ell\\left\(Y,\\,R^\{\{\\sf T\}\}G\+\(I\_\{m\}\-R^\{\{\\sf T\}\}R\)^\{1/2\}Z\\right\)\\right\]\\,\.In particular, the limiting test error only depends onRR, which captures the asymptotic correlation between𝑾^nAMP\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}and the ground truth𝑾∗\\text\{$W$\}\_\{\*\}\.

###### Remark 4\.

Sinceℱm,α,φalg\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}is closed under weak limits \(cf\.\[[MZ22](https://arxiv.org/html/2606.28573#bib.bibx34), Lemma E\.8\]\), we may allow an\(R,Q\)\(R,Q\)\-contractionFFto be only weakly differentiable\. In this case,∂F/∂G\\partial F/\\partial Gand∂F/∂ZR,Q\\partial F/\\partial Z\_\{R,Q\}in[2](https://arxiv.org/html/2606.28573#Thmdefn2)should be interpreted as weak derivatives\. Furthermore, by Stein’s identity, the first equality constraint in Eq\. \([20](https://arxiv.org/html/2606.28573#S3.E20)\), namely

R𝖳=𝔼​\[∂F∂G​\(ZR,Q,φ​\(G,ε\)\)\],\\displaystyle R^\{\\sf T\}=\\,\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial G\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]\\,,is equivalent to

R=\(Ik−R​Q−1​R𝖳\)−1​𝔼​\[\(G−R​Q−1​ZR,Q\)​F​\(ZR,Q,φ​\(G,ε\)\)𝖳\]\.R=\\,\(I\_\{k\}\-RQ^\{\-1\}R^\{\{\\sf T\}\}\)^\{\-1\}\\mathbb\{E\}\\big\[\\left\(G\-RQ^\{\-1\}Z\_\{R,Q\}\\right\)\\,F\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)^\{\{\\sf T\}\}\\big\]\\,\.\(25\)

Theorem[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)\(see also Theorem[B\.1](https://arxiv.org/html/2606.28573#A2.Thmthm1)for the casem=k=1m=k=1\) characterizes the asymptotic value of the optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\) achieved by our two\-stage AMP algorithm\. Optimizing this value involves a variational problem overFFandΦ\\Phi, which will be examined in greater detail in the following sections\.

### 3\.3Dual value𝒱m,α,φalg​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\(h\)and stochastic optimal control

We now discuss the implications of[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)on the optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\), which we restate here for future reference:

maximizeHn,d​\(𝑾\):=1n​∑i=1nh​\(yi,𝑾𝖳​𝒙i\),subject to​𝑾∈O​\(d,m\)\.\\mbox\{maximize\}\\quad H\_\{n,d\}\(\\text\{$W$\}\):=\\,\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\big\(y\_\{i\},\\text\{$W$\}^\{\\sf T\}\\text\{$x$\}\_\{i\}\\big\),\\quad\\mbox\{subject to\}\\ \\text\{$W$\}\\in O\(d,m\)\.To characterize its optimum achieved by our two\-stage AMP algorithm, define the dual functional

𝒱m,α,φAMP​\(h\):=sup𝔼​\[h​\(Y,ZR,Q\+1α​F​\(ZR,Q,Y\)\+∫01Q​\(t\)​\(Im\+Φt\)​d​Bt\)\],subject to​\(R,Q\)∈ABayes,Fis an\(R,Q\)\-contraction,∫01Q​\(t\)​Q​\(t\)𝖳​d​t=Im−Q,and​𝔼​\[Φt​Φt𝖳\]⪯Imα,∀t∈\[0,1\]\.\\begin\{split\}&\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\):=\\,\\sup\\,\\mathbb\{E\}\\left\[h\\left\(Y,\\,Z\_\{R,Q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{R,Q\},Y\\right\)\+\\int\_\{0\}^\{1\}Q\(t\)\\left\(I\_\{m\}\+\\Phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\],\\\\ &\\mbox\{subject to\}\\,\\,\(R,Q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\},\\,\\text\{$F$ is an $\(R,Q\)$\-contraction\},\\\\ &\\quad\\quad\\quad\\quad\\,\\,\\int\_\{0\}^\{1\}Q\(t\)Q\(t\)^\{\\sf T\}\\textup\{d\}t=I\_\{m\}\-Q,\\,\\text\{and\}\\,\\,\\mathbb\{E\}\\left\[\\Phi\_\{t\}\\Phi\_\{t\}^\{\\sf T\}\\right\]\\preceq\\frac\{I\_\{m\}\}\{\\alpha\},\\,\\forall t\\in\[0,1\]\.\\end\{split\}\(26\)Note that for any fixed choice of\(R,Q\)\(R,Q\),FFand\{Q​\(t\)\}t∈\[0,1\]\\\{Q\(t\)\\\}\_\{t\\in\[0,1\]\},[Eq\.26](https://arxiv.org/html/2606.28573#S3.E26)describes a stochastic optimal control problem for the control process\{Φt\}t∈\[0,1\]\\\{\\Phi\_\{t\}\\\}\_\{t\\in\[0,1\]\}\. The following theorem, which is a direct consequence of[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1), characterizes the asymptotic maximum ofHn,d​\(𝑾\)H\_\{n,d\}\(\\text\{$W$\}\)achieved by our two\-stage AMP algorithm, in terms of𝒱m,α,φAMP​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)\.

###### Theorem 3\.2\(Optimal value achieved by AMP algorithms\)\.

For anyh:ℝ×ℝm→ℝh:\\mathbb\{R\}\\times\\mathbb\{R\}^\{m\}\\to\\mathbb\{R\}continuous, bounded from above, and of at most linear growth at infinity, the following holds:

- \(a\)*Upper bound\.*If𝑾^nAMP\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}is the output of any AMP algorithm as described in Section[3\.1](https://arxiv.org/html/2606.28573#S3.SS1)\(and further elaborated in[Section7](https://arxiv.org/html/2606.28573#S7)\), then almost surely, limn→∞Hn,d​\(𝑾^nAMP\)≤𝒱m,α,φAMP​\(h\)\.\\lim\_\{n\\to\\infty\}H\_\{n,d\}\\big\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\big\)\\leq\\,\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)\.
- \(b\)*Achievability\.*For anyε\>0\\varepsilon\>0, there exists a two\-stage AMP algorithm outputting𝑾^nAMP\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}, such that almost surely, limn→∞Hn,d​\(𝑾^nAMP\)≥𝒱m,α,φAMP​\(h\)−ε\.\\lim\_\{n\\to\\infty\}H\_\{n,d\}\\big\(\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\big\)\\geq\\,\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)\-\\varepsilon\\,\.

Of course, since AMP is a polynomial\-time algorithm, we conclude that

𝒱m,α,φalg​\(h\)≥𝒱m,α,φAMP​\(h\)\.\\displaystyle\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\(h\)\\geq\\,\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)\\,\.Namely,𝒱m,α,φAMP​\(h\)\\mathscr\{V\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)is an algorithmic lower bound for the value of the optimization problem \([3](https://arxiv.org/html/2606.28573#S1.E3)\)\.

Henceforth, we focus on the casem=k=1m=k=1\. In this setting, the function parameterQ​\(t\)=q​\(t\)Q\(t\)=q\(t\)in[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)\(see also its specialization tom=k=1m=k=1in Appendix[B\.2](https://arxiv.org/html/2606.28573#A2.SS2)\) can be eliminated, as any choice ofq​\(t\)q\(t\)reduces toq​\(t\)=1q\(t\)=1via a reparametrization of the time variablett\. As a consequence, the stochastic control problem \([26](https://arxiv.org/html/2606.28573#S3.E26)\) can be further simplified, as stated below\.

###### Proposition 3\.1\.

Form=k=1m=k=1, we have

𝒱1,α,φAMP​\(h\)=\\displaystyle\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)=sup\{𝒱1,α,φAMP​\(r,q,h\):\(r,q\)∈ABayes\},\\displaystyle\\sup\\Big\\\{\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\):\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\Big\\\}\\,,\(27\)𝒱1,α,φAMP​\(r,q,h\):=\\displaystyle\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\):=sup𝔼​\[h​\(Y,Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+ϕt\)​d​Bt\)\],\\displaystyle\\;\\sup\\;\\mathbb\{E\}\\left\[h\\left\(Y,\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\],\(28\)subject to​Fis an\(r,q\)\-contraction,and​supt∈\[q,1\]𝔼​\[ϕt2\]≤1α,\\displaystyle\\mbox\{\\rm subject to\}\\,\\,\\text\{$F$ is an $\(r,q\)$\-contraction\},\\,\\text\{and\}\\,\\,\\sup\_\{t\\in\[q,1\]\}\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\\leq\\frac\{1\}\{\\alpha\}\\,,where we recall thatFFis an\(r,q\)\(r,q\)\-contraction if and only if

r=𝔼​\[∂F∂G​\(Zr,q,φ​\(G,ε\)\)\],q=r2\+1α​𝔼​\[F​\(Zr,q,φ​\(G,ε\)\)2\],\\displaystyle r=\\,\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial G\}\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\],\\,\\,q=\\,r^\{2\}\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\],1α​𝔼​\[∂F∂Zr,q​\(Zr,q,φ​\(G,ε\)\)2\]≤1\.\\displaystyle\\quad\\quad\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{r,q\}\}\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\]\\leq 1\\,\.

The proof of[3\.1](https://arxiv.org/html/2606.28573#S3.Thmlem1)closely follows that of\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Proposition 3\.2\], so we omit it here for simplicity\. Later in[Section6](https://arxiv.org/html/2606.28573#S6), we will present an extended variational principle for𝒱1,α,φAMP​\(h\)\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\), which parallels the replica prediction for𝒱1,α,φ​\(h\)\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}\(h\)established in[Section2](https://arxiv.org/html/2606.28573#S2)\.

## 4Replica symmetric regime form=k=1m=k=1

At sufficiently large aspect ratioα\\alpha, the variational principles described in the previous sections simplify significantly, along with the ERM landscape\. In statistical physics, this is usually referred to as a ‘replica symmetric’ phase\. In this section, we state the resulting simplified formulas, deferring their proofs to Appendix[C](https://arxiv.org/html/2606.28573#A3)\.

We begin with considering the variational principle for the optimizer of𝖥​\(μ,c,r\):=𝖥1​\(μ,c,r\)\\mathsf\{F\}\(\\mu,c,r\):=\\mathsf\{F\}\_\{1\}\(\\mu,c,r\)given in Conjecture[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)\. Definer∗∈\[−1,1\]r\_\{\*\}\\in\[\-1,1\]as

r∗=arg⁡maxr∈\[−1,1\]​inf\(μ,c\)∈𝒰×ℝ\>0𝖥​\(μ,c,r\)\.r\_\{\*\}=\\,\\arg\\max\_\{r\\in\[\-1,1\]\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\(\\mu,c,r\)\.The optimizerr∗r\_\{\*\}can be interpreted as the asymptotic limit of the inner product⟨𝒘^nERM,𝒘∗⟩\\langle\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\},\\text\{$w$\}\_\{\*\}\\rangle, where𝒘^nERM\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\}is the global maximizer of \([3](https://arxiv.org/html/2606.28573#S1.E3)\) \(or equivalently, the global minimizer of \([4](https://arxiv.org/html/2606.28573#S1.E4)\) withℓ=−h\\ell=\-h\)\. The next proposition establishes that for sufficiently largeα\\alpha, the Parisi variational principle is replica symmetric atr∗r\_\{\*\}\.

###### Proposition 4\.1\.

Assume𝔼​\[\(∂xh​\(Y,G\)\)2\]\>0\\mathbb\{E\}\[\(\\partial\_\{x\}h\(Y,G\)\)^\{2\}\]\>0, and forr∈\[−1,1\]r\\in\[\-1,1\], define the function

v\(r\)=𝔼\[h\(Y,rG\+1−r2Z\)\],Z∼𝖭\(0,1\),Z⟂⟂\(Y,G\)\.v\(r\)=\\,\\mathbb\{E\}\\left\[h\\left\(Y,rG\+\\sqrt\{1\-r^\{2\}\}Z\\right\)\\right\],\\quad Z\\sim\{\\sf N\}\(0,1\),\\,\\,Z\\perp\\\!\\\!\\\!\\perp\(Y,G\)\.Further assume thatr1:=arg⁡maxr∈\[−1,1\]⁡v​\(r\)∈\{−1,1\}r\_\{1\}:=\\arg\\max\_\{r\\in\[\-1,1\]\}v\(r\)\\in\\\{\-1,1\\\}, andr1​v′​\(r1\)\>0r\_\{1\}v^\{\\prime\}\(r\_\{1\}\)\>0\. Then, there exist constantsC,α0\>0C,\\alpha\_\{0\}\>0, such that for allα\>α0\\alpha\>\\alpha\_\{0\}, the following happens:

- \(i\)If1−r2≤C/α1\-r^\{2\}\\leq C/\\alpha, then the Parisi variational probleminf\(μ,c\)∈𝒰×ℝ\>0𝖥​\(μ,c,r\)\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\(\\mu,c,r\)has a replica symmetric solution, i\.e\., the infimum is achieved at\(μ∗=0,c∗\)\(\\mu\_\{\*\}=0,c\_\{\*\}\)\.
- \(ii\)r∗r\_\{\*\}satisfies1−r∗2≤C/α1\-r\_\{\*\}^\{2\}\\leq C/\\alpha\.

In this case,r∗r\_\{\*\}can be equivalently defined as

r∗=arg⁡maxr∈\[−1,1\]​sup𝔼​\[U2\]≤1/α𝔼​\[h​\(Y,r​G\+1−r2​\(Z\+U\)\)\]\.\\displaystyle r\_\{\*\}=\\arg\\max\_\{r\\in\[\-1,1\]\}\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha\}\\mathbb\{E\}\\left\[h\\left\(Y,rG\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\\right\]\.\(29\)

###### Remark 5\.

The conditionsr1∈\{−1,1\}r\_\{1\}\\in\\\{\-1,1\\\}andr1​v′​\(r1\)\>0r\_\{1\}v^\{\\prime\}\(r\_\{1\}\)\>0ensure that ERM achieves perfect recovery in the limitα=∞\\alpha=\\infty\. This requirement is essential\. Without this assumption, the estimation error is bounded away from zero asα→∞\\alpha\\to\\infty\(the empirical risk minimizer𝒘^nERM\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\}is an inconsistent estimator of𝒘∗\\text\{$w$\}\_\{\*\}\), thus changing the nature of the large\-α\\alphaasymptotics \(also making it less interesting from a statistical viewpoint\)\.

Recent works\[[VDR\+25](https://arxiv.org/html/2606.28573#bib.bibx41),[MS26](https://arxiv.org/html/2606.28573#bib.bibx32)\]derived high\-dimensional asymptotics for the empirical risk minimizer𝒘^nERM\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\}in the regimeα\>α0\\alpha\>\\alpha\_\{0\}for some constantα0\\alpha\_\{0\}\(independent ofnnanddd\), under certain technical conditions\. Combining their results with[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1), we obtain the following corollary\.

###### Corollary 4\.2\.

Letℓ=−h\\ell=\-h\. Under the setting of Proposition[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1), as well as the assumptions and rate trivialization conditions in\[[MS26](https://arxiv.org/html/2606.28573#bib.bibx32), Theorem 3\], there existsα0<∞\\alpha\_\{0\}<\\inftysuch that Conjecture[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)holds for allα\>α0\\alpha\>\\alpha\_\{0\}\. In particular, we have almost surely

limn→∞⟨𝒘^nERM,𝒘∗⟩=r∗\\lim\_\{n\\to\\infty\}\\left\\langle\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\},\\text\{$w$\}\_\{\*\}\\right\\rangle=r\_\{\*\}withr∗r\_\{\*\}defined as per[Eq\.29](https://arxiv.org/html/2606.28573#S4.E29)\. Further, the limiting training and test errors of𝐰^nERM\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\}are given by:

limn→∞L^n​\(𝒘^nERM\)=\\displaystyle\\lim\_\{n\\to\\infty\}\\widehat\{L\}\_\{n\}\\left\(\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\}\\right\)=\\,inf𝔼​\[U2\]≤1/α𝔼​\[ℓ​\(Y,r∗​G\+1−r∗2​\(Z\+U\)\)\],\\displaystyle\\inf\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha\}\\mathbb\{E\}\\left\[\\ell\\left\(Y,r\_\{\*\}G\+\\sqrt\{1\-r\_\{\*\}^\{2\}\}\(Z\+U\)\\right\)\\right\]\\,,\(30\)limn→∞L​\(𝒘^nERM\)=\\displaystyle\\lim\_\{n\\to\\infty\}L\\left\(\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\}\\right\)=\\,𝔼​\[ℓ​\(Y,r∗​G\+1−r∗2​Z\)\]\.\\displaystyle\\mathbb\{E\}\\left\[\\ell\\left\(Y,\\,r\_\{\*\}G\+\\sqrt\{1\-r\_\{\*\}^\{2\}\}Z\\right\)\\right\]\\,\.\(31\)

Finally, we show that for sufficiently largeα\\alpha, these training and test errors are also achievable by our two\-stage AMP algorithm\.

###### Proposition 4\.3\.

Letr∗r\_\{\*\}be defined by Eq\. \([29](https://arxiv.org/html/2606.28573#S4.E29)\)\. Under the settings of[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1), there existsα0<∞\\alpha\_\{0\}<\\inftysuch that for allα\>α0\\alpha\>\\alpha\_\{0\}, one can construct a two\-stage AMP algorithm as in Section[3\.2](https://arxiv.org/html/2606.28573#S3.SS2)outputting𝐰^nAMP\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}, such that

limn→∞⟨𝒘^nAMP,𝒘∗⟩=r∗\.\\lim\_\{n\\to\\infty\}\\left\\langle\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\},\\text\{$w$\}\_\{\*\}\\right\\rangle=r\_\{\*\}\.Further, Eqs\. \([30](https://arxiv.org/html/2606.28573#S4.E30)\) and \([31](https://arxiv.org/html/2606.28573#S4.E31)\) hold with𝐰^nERM\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm ERM\}\}replaced by𝐰^nAMP\\widehat\{\\boldsymbol\{w\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\.

## 5Intermezzo: Largen/dn/dand tensor PCA equivalence

In this section, we apply the general theory developed in previous sections to a single\-index model \(m=k=1m=k=1\) with additive noise, trained by optimizing a correlation loss objective\. We obtain a straightforward characterization of the algorithmically achievable training and test errors, in the large\-α\\alphalimit\. We then show that this characterization admits a natural interpretation, in terms of an equivalent tensor PCA problem\[[MR14](https://arxiv.org/html/2606.28573#bib.bibx30)\]\. Proofs of the results in this section are deferred to[AppendixD](https://arxiv.org/html/2606.28573#A4)\.

### 5\.1Single\-index model

Throughout this section, we assume the data\{\(𝒙i,yi\)\}i=1n\\\{\(\\text\{$x$\}\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}follow the single\-index model of Eq\. \([1](https://arxiv.org/html/2606.28573#S1.E1)\), and consider maximizing the empirical correlation objective of Eq\. \([2](https://arxiv.org/html/2606.28573#S1.E2)\) to learn the true signal𝒘∗\\text\{$w$\}\_\{\*\}\. Without loss of generality, we will assume𝔼​\[σ​\(G\)\]=0\\mathbb\{E\}\[\\sigma\(G\)\]=0forG∼𝖭​\(0,1\)G\\sim\{\\sf N\}\(0,1\); otherwise, one can simply consider the centered activation functionσ​\(z\)−𝔼​\[σ​\(G\)\]\\sigma\(z\)\-\\mathbb\{E\}\[\\sigma\(G\)\]\.

We first take the limitn,d→∞n,d\\to\\infty,n/d→αn/d\\to\\alpha, and then

α→∞,λ→0,α​λ→α¯,\\displaystyle\\alpha\\to\\infty,\\;\\;\\;\\;\\lambda\\to 0,\\;\\;\\;\\;\\alpha\\lambda\\to\\overline\{\\alpha\}\\,,\(32\)for some fixedα¯∈\(0,\+∞\)\\overline\{\\alpha\}\\in\(0,\+\\infty\)\. This turns out to be the correct scaling to avoid classical low\-dimensional behavior, or a trivial information\-theoretic limit\.

We will establish that—in this double limit— the single\-index model \([1](https://arxiv.org/html/2606.28573#S1.E1)\) is asymptotically equivalent to a Gaussian process model with asymptotically matching first and second moments\. Namely, we letH^dg​\(⋅\)\\widehat\{H\}^\{g\}\_\{d\}\(\\,\\cdot\\,\)be a Gaussian process on𝕊d−1\\mathbb\{S\}^\{d\-1\}with mean and covariance

𝔼​\[H^dg​\(𝒘\)\]=ξφ​σ​\(⟨𝒘∗,𝒘⟩\),𝖢𝗈𝗏​\(H^dg​\(𝒘1\),H^dg​\(𝒘2\)\)=1d​α¯​ξσ​σ​\(⟨𝒘1,𝒘2⟩\),\\begin\{split\}\\mathbb\{E\}\\big\[\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)\\big\]=\\,\\xi\_\{\\varphi\\sigma\}\(\\langle\\text\{$w$\}\_\{\*\},\\text\{$w$\}\\rangle\)\\,,\\quad\{\\sf Cov\}\\big\(\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\_\{1\}\),\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\_\{2\}\)\\big\)=\\,\\frac\{1\}\{d\\overline\{\\alpha\}\}\\,\\xi\_\{\\sigma\\sigma\}\(\\langle\\text\{$w$\}\_\{1\},\\text\{$w$\}\_\{2\}\\rangle\)\\,,\\end\{split\}\(33\)whereξσ​σ,ξφ​σ\\xi\_\{\\sigma\\sigma\},\\xi\_\{\\varphi\\sigma\}, andξφ​φ\\xi\_\{\\varphi\\varphi\}\(for future reference\) are defined as

ξσ​σ​\(q\)=𝔼​\[σ​\(G1\)​σ​\(Gq\)\],ξφ​σ​\(q\)=𝔼​\[φ​\(G1\)​σ​\(Gq\)\],ξφ​φ​\(q\)=𝔼​\[φ​\(G1\)​φ​\(Gq\)\],\\displaystyle\\xi\_\{\\sigma\\sigma\}\(q\)=\\mathbb\{E\}\\big\[\\sigma\(G\_\{1\}\)\\,\\sigma\(G\_\{q\}\)\\big\]\\,,\\,\\,\\xi\_\{\\varphi\\sigma\}\(q\)=\\mathbb\{E\}\\big\[\\varphi\(G\_\{1\}\)\\,\\sigma\(G\_\{q\}\)\\big\]\\,,\\,\\,\\xi\_\{\\varphi\\varphi\}\(q\)=\\mathbb\{E\}\\big\[\\varphi\(G\_\{1\}\)\\,\\varphi\(G\_\{q\}\)\\big\]\\,,for\(G1,Gq\)\(G\_\{1\},G\_\{q\}\)jointly centered Gaussian with𝔼​\[G12\]=𝔼​\[Gq2\]=1\\mathbb\{E\}\[G\_\{1\}^\{2\}\]=\\mathbb\{E\}\[G\_\{q\}^\{2\}\]=1and𝔼​\[G1​Gq\]=q\\mathbb\{E\}\[G\_\{1\}G\_\{q\}\]=q\. Denoting by\{φk\}k=0∞\\\{\\varphi\_\{k\}\\\}\_\{k=0\}^\{\\infty\}and\{σk\}k=0∞\\\{\\sigma\_\{k\}\\\}\_\{k=0\}^\{\\infty\}the Hermite coefficients of functionsφ\\varphiandσ\\sigma, we have

ξφ​φ​\(t\)=∑k=0∞φk2​tk,ξφ​σ​\(t\)=∑k=0∞φk​σk​tk,ξσ​σ​\(t\)=∑k=0∞σk2​tk\.\\xi\_\{\\varphi\\varphi\}\(t\)=\\sum\_\{k=0\}^\{\\infty\}\\varphi^\{2\}\_\{k\}t^\{k\}\\,,\\;\\;\\;\\;\\xi\_\{\\varphi\\sigma\}\(t\)=\\sum\_\{k=0\}^\{\\infty\}\\varphi\_\{k\}\\sigma\_\{k\}t^\{k\}\\,,\\;\\;\\;\\;\\xi\_\{\\sigma\\sigma\}\(t\)=\\sum\_\{k=0\}^\{\\infty\}\\sigma^\{2\}\_\{k\}t^\{k\}\\,\.\(34\)By straightforward calculation, we can verify thatH^dg\\widehat\{H\}^\{g\}\_\{d\}andH^n\\widehat\{H\}\_\{n\}\(the latter being defined in Eq\. \([2](https://arxiv.org/html/2606.28573#S1.E2)\)\) have the same mean, and their covariances are asymptotically equivalent in the double limitn/d→αn/d\\to\\alpha,α​λ→α¯\\alpha\\lambda\\to\\overline\{\\alpha\}\.

In Section[5\.4](https://arxiv.org/html/2606.28573#S5.SS4), we will see that the Gaussian process modelH^dg\\widehat\{H\}^\{g\}\_\{d\}has a natural statistical interpretation as a generalized tensor PCA problem\.

### 5\.2Replica prediction for global maximum

We now present the replica prediction for the asymptotic global maximum ofH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)over the unit sphere, under the double limit \([32](https://arxiv.org/html/2606.28573#S5.E32)\)\.

In this regime, the Parisi variational principle simplifies dramatically, as stated next\. The following claim is a consequence of Conjecture[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2): we derive it from that conjecture in Appendix[D\.1](https://arxiv.org/html/2606.28573#A4.SS1)\.

###### Conjecture 5\.1\.

Recall the function space𝒰\\mathscr\{U\}from[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)andH^n​\(𝐰\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)from[Eq\.2](https://arxiv.org/html/2606.28573#S1.E2)\. Under the double limit \([32](https://arxiv.org/html/2606.28573#S5.E32)\), we have

limα​λ→α¯limn/d→αmax𝒘∈𝕊d−1⁡H^n​\(𝒘\)=supr∈\[−1,1\]inf\(μ,c\)∈𝒰×ℝ\>0𝖥¯α¯​\(μ,c,r\),\\displaystyle\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}\\lim\_\{n/d\\to\\alpha\}\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)=\\,\\sup\_\{r\\in\[\-1,1\]\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\overline\{\\mathsf\{F\}\}\_\{\\overline\{\\alpha\}\}\(\\mu,c,r\),\(35\)where

𝖥¯α¯​\(μ,c,r\)=ξφ​σ​\(r\)\+12​∫r21μ​\(t\)​ξσ​σ′​\(t\)​d​t\+12​c​ξσ​σ′​\(1\)\+12​α¯​∫r21d​tc\+∫t1μ​\(s\)​d​s\.\\overline\{\\mathsf\{F\}\}\_\{\\overline\{\\alpha\}\}\(\\mu,c,r\)=\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\mu\(t\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(t\)\\textup\{d\}t\+\\frac\{1\}\{2\}c\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\}\\,\.\(36\)

We also note that the minimization of𝖥¯α¯​\(μ,c,r\)\\overline\{\\mathsf\{F\}\}\_\{\\overline\{\\alpha\}\}\(\\mu,c,r\)over the extended domain used in[Section6](https://arxiv.org/html/2606.28573#S6)below can be performed explicitly\. We state the resulting minimum in the following proposition and omit its proof, which follows from a straightforward application of integration by parts\.

###### Proposition 5\.1\.

Define forq∈\[0,1\]q\\in\[0,1\], the function spaces

ℒ\\displaystyle\{\\mathscr\{L\}\}:=\{\(μ,c\)∈L1​\[0,1\]×ℝ\>0:μ\|\[0,t\]∈L∞​\[0,t\]​and​c\+∫t1μ​\(s\)​d​s\>0,∀t∈\[0,1\)\},\\displaystyle:=\\left\\\{\(\\mu,c\)\\in L^\{1\}\[0,1\]\\times\\mathbb\{R\}\_\{\>0\}:\\,\\mu\|\_\{\[0,t\]\}\\in L^\{\\infty\}\[0,t\]\\ \\text\{and\}\\ c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\,\\textup\{d\}s\>0,\\,\\forall t\\in\[0,1\)\\right\\\},\(37\)ℒ​\(q\)\\displaystyle\{\\mathscr\{L\}\}\(q\):=\{\(μ,c\)∈ℒ:μ​\(t\)=0,∀t∈\[0,q\]\}\.\\displaystyle:=\\left\\\{\(\\mu,c\)\\in\{\\mathscr\{L\}\}:\\,\\mu\(t\)=0,\\ \\forall t\\in\[0,q\]\\right\\\}\.\(38\)Further define for any\(q,r\)∈\[0,1\]×\[−1,1\]\(q,r\)\\in\[0,1\]\\times\[\-1,1\]withr2≤q≤1r^\{2\}\\leq q\\leq 1, the functional

𝒱α¯∗​\(r,q\):=ξφ​σ​\(r\)\+1α¯​\(q−r2\)​ξσ​σ′​\(q\)\+1α¯​∫q1ξσ​σ′′​\(t\)​dt\.\\displaystyle\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\):=\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\sqrt\{\(q\-r^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{q\}^\{1\}\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\\,\{\\rm d\}t\\,\.\(39\)Then, we have

𝒱α¯∗​\(r,q\)=inf\(μ,c\)∈ℒ​\(q\)𝖥¯α¯​\(μ,c,r\)\.\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\)=\\inf\_\{\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)\}\\overline\{\\mathsf\{F\}\}\_\{\\overline\{\\alpha\}\}\(\\mu,c,r\)\\,\.\(40\)

### 5\.3Achievability via AMP

In this section, we characterize the asymptotic values ofH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)achieved by our two\-stage AMP algorithm, in the double limit \([32](https://arxiv.org/html/2606.28573#S5.E32)\)\. Throughout the analysis, we assumeξφ​φ′​\(0\)≠0\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\(0\)\\neq 0, namelyφ1:=𝔼​\[G​φ​\(G\)\]≠0\\varphi\_\{1\}:=\\mathbb\{E\}\[G\\varphi\(G\)\]\\neq 0\. We begin by defining the Bayes AMP feasible region for the single\-index model:

ABayes\\displaystyle A\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\{\(r,q\)∈ℝ2:q∈\[r2,1\],r2≤cBayes2​\(q−r2\)\},\\displaystyle=\\\{\(r,q\)\\in\\mathbb\{R\}^\{2\}:q\\in\[r^\{2\},1\],\\,r^\{2\}\\leq c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\(q\-r^\{2\}\)\\\},\(41\)cBayes\\displaystyle c\_\{\\mbox\{\\tiny\\rm Bayes\}\}:=inf\{c\>0:c2=α¯​ξφ​φ′​\(c21\+c2\)\}\.\\displaystyle:=\\inf\\left\\\{c\>0:c^\{2\}=\\overline\{\\alpha\}\\,\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{c^\{2\}\}\{1\+c^\{2\}\}\\right\)\\right\\\}\\,\.\(42\)Next, for\(q,r\)∈\[0,1\]×\[−1,1\]\(q,r\)\\in\[0,1\]\\times\[\-1,1\]withr2≤q≤1r^\{2\}\\leq q\\leq 1, define

vk=k​α¯c2​\(r2q\)\(k−1\)/2​φk,sk=k​σk​q\(k−1\)/2,k≥1,v\_\{k\}=\\sqrt\{\\frac\{k\\overline\{\\alpha\}\}\{c^\{2\}\}\}\\left\(\\frac\{r^\{2\}\}\{q\}\\right\)^\{\(k\-1\)/2\}\\varphi\_\{k\}\\,,\\quad s\_\{k\}=\\sqrt\{k\}\\sigma\_\{k\}q^\{\(k\-1\)/2\}\\,,\\,\\,k\\geq 1,\(43\)and

T0​\(r,q\):=max⁡\{⟨s,x⟩:‖x‖2≤1,⟨x,v⟩=1,∑k≥1\(k−1\)​xk2≤qq−r2\}\.\\displaystyle T\_\{0\}\(r,q\):=\\max\\Big\\\{\\langle s,x\\rangle:\\;\\;\\\|x\\\|\_\{2\}\\leq 1,\\,\\langle x,v\\rangle=1,\\,\\sum\_\{k\\geq 1\}\(k\-1\)x\_\{k\}^\{2\}\\leq\\frac\{q\}\{q\-r^\{2\}\}\\Big\\\}\\,\.\(44\)Then, for a fixed pair\(r,q\)\(r,q\), we establish that the asymptotic value ofH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)achieved by our two\-stage AMP algorithm takes the simple form:

𝒱α¯AMP​\(r,q\)\\displaystyle\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\):=ξφ​σ​\(r\)\+q−r2α¯​T0​\(r,q\)\+1α¯​∫q1ξσ​σ′′​\(t\)​dt,\\displaystyle:=\\xi\_\{\\varphi\\sigma\}\(r\)\+\\sqrt\{\\frac\{q\-r^\{2\}\}\{\\overline\{\\alpha\}\}\}\\,T\_\{0\}\(r,q\)\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{q\}^\{1\}\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\\,\{\\rm d\}t\\,,\(45\)see below for the precise statement\.

###### Proposition 5\.2\.

For any\(r,q\)∈ABayes\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}and\(α,λ\)∈ℝ\+2\(\\alpha,\\lambda\)\\in\\mathbb\{R\}\_\{\+\}^\{2\}, there exists a two\-stage AMP algorithm that returns𝐰^nα,λ∈𝕊d−1\{\\widehat\{\\boldsymbol\{w\}\}\}\_\{n\}^\{\\alpha,\\lambda\}\\in\\mathbb\{S\}^\{d\-1\}, such that

limα​λ→α¯limn/d→α⟨𝒘∗,𝒘^nα,λ⟩=r,limα​λ→α¯limn/d→αH^n​\(𝒘^nα,λ\)=𝒱α¯AMP​\(r,q\)\.\\displaystyle\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}\\lim\_\{n/d\\to\\alpha\}\\langle\\text\{$w$\}\_\{\*\},\{\\widehat\{\\boldsymbol\{w\}\}\}\_\{n\}^\{\\alpha,\\lambda\}\\rangle=r\\,,\\quad\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}\\lim\_\{n/d\\to\\alpha\}\\widehat\{H\}\_\{n\}\\big\(\{\\widehat\{\\boldsymbol\{w\}\}\}\_\{n\}^\{\\alpha,\\lambda\}\\big\)=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\,\.

Let𝒱α¯AMP=sup\(r,q\)∈ABayes𝒱α¯AMP​\(r,q\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}=\\sup\_\{\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)denote the asymptotic maximum value ofH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)achieved by our two\-stage AMP algorithm over all feasible choices of\(r,q\)\(r,q\)\. The following theorem establishes that, under certain optimality conditions,𝒱α¯AMP\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}admits a simpler expression that coincides with the formula in Eq\. \([39](https://arxiv.org/html/2606.28573#S5.E39)\)\.

###### Theorem 5\.1\.

Define𝒱α¯∗​\(r,q\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\)as per Eq\. \([39](https://arxiv.org/html/2606.28573#S5.E39)\)\. Then the following holds:

- \(a\)For all\(q,r\)\(q,r\)withr2≤q≤1r^\{2\}\\leq q\\leq 1,𝒱α¯AMP​\(r,q\)≤𝒱α¯∗​\(r,q\)\\mathscr\{V\}^\{\\mbox\{\\tiny\\rm AMP\}\}\_\{\\overline\{\\alpha\}\}\(r,q\)\\leq\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\)\.
- \(b\)For allr∈\[−1,1\]r\\in\[\-1,1\], define q∗​\(r\)=arg⁡minq∈\[r2,1\]⁡\{ξσ​σ′​\(q\)q−r2\}\.\\displaystyle q\_\{\*\}\(r\)=\\arg\\min\_\{q\\in\[r^\{2\},1\]\}\\left\\\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\{q\-r^\{2\}\}\\right\\\}\.\(46\)Then, we have supq:\(r,q\)∈ABayes𝒱α¯AMP​\(r,q\)≤𝒱α¯∗​\(r,q∗​\(r\)\)\.\\sup\_\{q:\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\.
- \(c\)Let r∗:=argmaxr:\(r,q∗​\(r\)\)∈ABayes𝒱α¯∗​\(r,q∗​\(r\)\),q∗=q∗​\(r∗\)\.r\_\{\*\}:=\\operatorname\*\{argmax\}\_\{r:\(r,q\_\{\*\}\(r\)\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\\,,\\,\\,\\;\\;q\_\{\*\}=q\_\{\*\}\(r\_\{\*\}\)\.\(47\)Then, we have𝒱α¯AMP​\(r∗,q∗\)=𝒱α¯∗​\(r∗,q∗\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r\_\{\*\},q\_\{\*\}\)=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r\_\{\*\},q\_\{\*\}\), i\.e\., the value𝒱α¯∗​\(r∗,q∗\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r\_\{\*\},q\_\{\*\}\)is achievable by our two\-stage AMP algorithm\. As a consequence, maxr:\(r,q∗​\(r\)\)∈ABayes⁡𝒱α¯∗​\(r,q∗​\(r\)\)≤max\(r,q\)∈ABayes⁡𝒱α¯AMP​\(r,q\)≤maxr:\(r,1\)∈ABayes⁡𝒱α¯∗​\(r,q∗​\(r\)\)\.\\max\_\{r:\(r,q\_\{\*\}\(r\)\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\\leq\\max\_\{\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\max\_\{r:\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\.\(48\)In particular, if the first and last terms in the above chain of inequalities coincide, then𝒱α¯AMP=𝒱α¯∗​\(r∗,q∗\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r\_\{\*\},q\_\{\*\}\)\.

While it is not guaranteed that condition \([48](https://arxiv.org/html/2606.28573#S5.E48)\) holds universally, we demonstrate below that it is satisfied for sufficiently largeα¯\\overline\{\\alpha\}\.

###### Proposition 5\.3\.

There exists a thresholdα¯∗​\(φ,σ\)\\overline\{\\alpha\}\_\{\*\}\(\\varphi,\\sigma\)depending only onφ\\varphiandσ\\sigma, such that[Eq\.48](https://arxiv.org/html/2606.28573#S5.E48)holds for allα¯≥α¯∗​\(φ,σ\)\\overline\{\\alpha\}\\geq\\overline\{\\alpha\}\_\{\*\}\(\\varphi,\\sigma\)\.

### 5\.4The equivalent Gaussian model

We next revisit the Gaussian process model defined by Eq\. \([33](https://arxiv.org/html/2606.28573#S5.E33)\), and show that it has the following tensor PCA interpretation\. Recall from[Section5\.1](https://arxiv.org/html/2606.28573#S5.SS1)that𝒘∗\\text\{$w$\}\_\{\*\}is the true signal,φk\\varphi\_\{k\}is thekk\-th Hermite coefficient ofφ\\varphi, andσk\\sigma\_\{k\}is thekk\-th Hermite coefficient ofσ\\sigma\. For eachk≥1k\\geq 1, we observe𝒀\(k\)∈\(ℝd\)⊗k\\text\{$Y$\}^\{\(k\)\}\\in\(\\mathbb\{R\}^\{d\}\)^\{\\otimes k\}as a rank\-one, order\-kktensor perturbed by i\.i\.d\. Gaussian noise:

𝒀\(k\)=φk​𝒘∗⊗k\+1d​α¯​𝑮\(k\),\\displaystyle\\text\{$Y$\}^\{\(k\)\}=\\varphi\_\{k\}\\text\{$w$\}\_\{\*\}^\{\\otimes k\}\+\\frac\{1\}\{\\sqrt\{d\\overline\{\\alpha\}\}\}\\text\{$G$\}^\{\(k\)\}\\,,where𝑮\(k\)=\(Gi1,…,ik\(k\)\)1≤i1,…,ik≤d∈\(ℝd\)⊗k\\text\{$G$\}^\{\(k\)\}=\(G^\{\(k\)\}\_\{i\_\{1\},\\dots,i\_\{k\}\}\)\_\{1\\leq i\_\{1\},\\dots,i\_\{k\}\\leq d\}\\in\(\\mathbb\{R\}^\{d\}\)^\{\\otimes k\}is a random tensor with i\.i\.d\.𝖭​\(0,1\)\{\\sf N\}\(0,1\)entries\. We propose to learn𝒘∗\\text\{$w$\}\_\{\*\}by maximizing the “generalized tensor PCA” objective:

H^dg​\(𝒘\)=∑k≥1σk​⟨𝒀\(k\),𝒘⊗k⟩,𝒘∈𝕊d−1,\\displaystyle\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)=\\,\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\langle\\text\{$Y$\}^\{\(k\)\},\\text\{$w$\}^\{\\otimes k\}\\rangle\\,,\\quad\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\\,,\(49\)The lemma below establishes the equivalence between this model and the Gaussian process model introduced in[Section5\.1](https://arxiv.org/html/2606.28573#S5.SS1)\.

###### Lemma 5\.4\.

The stochastic process\{H^dg​\(𝐰\)\}𝐰∈𝕊d−1\\big\\\{\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)\\big\\\}\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}defined as per Eq\. \([49](https://arxiv.org/html/2606.28573#S5.E49)\) is a Gaussian process with mean and covariance given by Eq\. \([33](https://arxiv.org/html/2606.28573#S5.E33)\)\.

In Appendix[D\.3](https://arxiv.org/html/2606.28573#A4.SS3), we show that this generalized tensor PCA model \(and the equivalent Gaussian process model\) share the same asymptotic global maximum—as well as the same algorithmic values achieved by a two\-stage AMP\-based procedure—as the single\-index model analyzed in[Section5\.1](https://arxiv.org/html/2606.28573#S5.SS1)\.

### 5\.5Examples

We now present several illustrative examples of the single\-index model and its tensor PCA equivalent\.

![Refer to caption](https://arxiv.org/html/2606.28573v1/x2.png)Figure 2:Left panel:The limiting overlap with𝒘∗\\text\{$w$\}\_\{\*\}for Bayes AMP, ERM and PGD\.Right panel:The normalized limiting empirical and population correlation for these three methods\. In our implementation of PGD, we setd=100d=100andα=200\\alpha=200, sweepingα¯∈\[0,10\]\\overline\{\\alpha\}\\in\[0,10\]so thatλ\\lambdaranges in\[0,0\.05\]\[0,0\.05\]\. All reported numerical results are averaged over100100independent runs of PGD\.##### Well\-specified model\.

We consider a simple well\-specified setting, withσ​\(z\)=φ​\(z\)=max⁡\(0,z\)\\sigma\(z\)=\\varphi\(z\)=\\max\(0,z\)being the standard ReLU activation function\. In this setting, the learning task is equivalent to training a single\-neuron ReLU network, and we have

ξσ​σ​\(x\)=ξφ​φ​\(x\)=ξφ​σ​\(x\)=\\displaystyle\\xi\_\{\\sigma\\sigma\}\(x\)=\\xi\_\{\\varphi\\varphi\}\(x\)=\\xi\_\{\\varphi\\sigma\}\(x\)=\\,x4\+x​arcsin⁡x\+1−x22​π\.\\displaystyle\\frac\{x\}\{4\}\+\\frac\{x\\arcsin x\+\\sqrt\{1\-x^\{2\}\}\}\{2\\pi\}\.In Figure[2](https://arxiv.org/html/2606.28573#S5.F2), we present theoretical predictions for both the Bayes optimal estimator, and the estimator that minimizes the training error within the class of two\-stage AMP algorithms considered in Proposition[5\.2](https://arxiv.org/html/2606.28573#S5.Thmlem2)\(ERM\)\. By direct calculation, we know that

x↦\(1−x\)​ξφ​φ′​\(x\)=\(1−x\)​\(14\+12​π​arcsin⁡x\)x\\mapsto\(1\-x\)\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\(x\)=\(1\-x\)\\left\(\\frac\{1\}\{4\}\+\\frac\{1\}\{2\\pi\}\\arcsin x\\right\)is decreasing on\[0,1\]\[0,1\], which verifies condition\(i\)\(i\)in the statement of[D\.6](https://arxiv.org/html/2606.28573#A4.Thmlem6)\. Therefore, the optimality condition \([48](https://arxiv.org/html/2606.28573#S5.E48)\) holds\. As a consequence, maximizing𝒱α¯∗​\(r,q∗​\(r\)\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\\big\(r,q\_\{\*\}\(r\)\\big\)over\(r,1\)∈ABayes\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}yields the theoretical predictions for ERM\. For both Bayes AMP and ERM, we plot their correlations with the true parameter𝒘∗\\text\{$w$\}\_\{\*\}, as well as the normalized training error

L^n​\(𝒘\):=−1d​α​ξσ​σ​\(1\)​∑i=1nyi​σ​\(𝒘𝖳​𝒙i\)\\displaystyle\\widehat\{L\}\_\{n\}\(\\text\{$w$\}\):=\-\\frac\{1\}\{d\\sqrt\{\\alpha\\xi\_\{\\sigma\\sigma\}\(1\)\}\}\\sum\_\{i=1\}^\{n\}y\_\{i\}\\sigma\(\\text\{$w$\}^\{\{\\sf T\}\}\\text\{$x$\}\_\{i\}\)\(50\)and the test error normalized in the same way\.

We also compare these theoretical predictions with numerical simulations for projected gradient descent \(PGD\) maximizing the empirical correlationH^n​\(𝒘\)\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)over the unit sphere\. We observe that the behavior of PGD closely matches our predictions for the optimal algorithm\.

![Refer to caption](https://arxiv.org/html/2606.28573v1/x3.png)Figure 3:Left panel:The limiting overlap with𝒘∗\\text\{$w$\}\_\{\*\}for Bayes AMP, ERM and PGD, under different choices of the activationσ\\sigma\.Right panel:The normalized limiting empirical and population correlation for these three methods\. In this experiment, we fixσ1=1\\sigma\_\{1\}=1and varyk∈\{2,3\}k\\in\\\{2,3\\\}andσk∈\{1,2,5\}\\sigma\_\{k\}\\in\\\{1,2,5\\\}\. The top and bottom rows present our results fork=2k=2andk=3k=3, respectively\. The remaining experimental setup is the same as in the well\-specified case\.
##### A family of misspecified models\.

In Figure[3](https://arxiv.org/html/2606.28573#S5.F3), we consider a misspecified scenario, where the true link function is still ReLU:φ​\(z\)=max⁡\(0,z\)\\varphi\(z\)=\\max\(0,z\), yet the activation function takes the formσ​\(z\)=σ1​He1​\(z\)\+σk​Hek​\(z\)\\sigma\(z\)=\\sigma\_\{1\}\{\\rm He\}\_\{1\}\(z\)\+\\sigma\_\{k\}\{\\rm He\}\_\{k\}\(z\)fork≥2k\\geq 2\. Here,Hek\{\\rm He\}\_\{k\}denotes thekk\-th orthonormal Hermite polynomial\. In this setting, we haveξσ​σ​\(t\)=σ12​t\+σk2​tk\\xi\_\{\\sigma\\sigma\}\(t\)=\\sigma\_\{1\}^\{2\}t\+\\sigma\_\{k\}^\{2\}t^\{k\}\. Consequently, the optimization landscape becomes more complicated as we increasekk\. Similar to the well\-specified setting, maximizing𝒱α¯∗​\(r,q∗​\(r\)\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\\big\(r,q\_\{\*\}\(r\)\\big\)over\(r,1\)∈ABayes\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}determines the limiting overlap with𝒘∗\\text\{$w$\}\_\{\*\}, as well as the empirical and population correlation for the optimal ERM algorithm\. We compute and plot these theoretical values across different choices ofkkand\(σ1,σk\)\(\\sigma\_\{1\},\\sigma\_\{k\}\), and compare them with Bayes AMP and numerical simulations of PGD in Figure[3](https://arxiv.org/html/2606.28573#S5.F3)\.

As in the well\-specified setting, the theoretical predictions for the optimal ERM algorithm closely match the empirical results with PGD\. However, the estimation and test errors of these methods are significantly larger than those of Bayes AMP, especially whenα¯\\overline\{\\alpha\}is small\. Furthermore, their performance deteriorates as we increasekkorσk\\sigma\_\{k\}, indicating an increasingly severe overfitting\.

## 6Main results \(II\): Parisi variational principle

In this section, we develop a variational principle that is dual to the stochastic optimal control problem \([28](https://arxiv.org/html/2606.28573#S3.E28)\)\. The resulting formula is closely related to the Parisi variational principle, which we derived heuristically in Section[2](https://arxiv.org/html/2606.28573#S2)\(namely, the simplified Parisi formula form=1m=1in[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)\)\.

We begin with defining two relevant function spaces\.

###### Definition 3\(Space of functional order parameters\)\.

Define the function spacesℒ\{\\mathscr\{L\}\}andℒ​\(q\)\{\\mathscr\{L\}\}\(q\)as per Eqs\. \([37](https://arxiv.org/html/2606.28573#S5.E37)\) and \([38](https://arxiv.org/html/2606.28573#S5.E38)\), and further define, for anyq∈\[0,1\)q\\in\[0,1\):

ℒ\#\\displaystyle\{\\mathscr\{L\}\_\{\\\#\}\}:=\{γ:\[0,1\]→ℝ\>0absolutely continuous:\(μ,c\)∈ℒforμ=γ′/γ2,c=1/γ\(1\)\},\\displaystyle:=\\Big\\\{\\gamma:\[0,1\]\\to\\mathbb\{R\}\_\{\>0\}\\mbox\{ \{\\rm absolutely continuous\}\}:\\,\(\\mu,c\)\\in\{\\mathscr\{L\}\}\\ \\mbox\{\{\\rm for \}\}\\mu=\\gamma^\{\\prime\}/\\gamma^\{2\},\\,c=1/\\gamma\(1\)\\Big\\\}\\,,\(51\)ℒ\#​\(q\)\\displaystyle\{\\mathscr\{L\}\_\{\\\#\}\}\(q\):=\{γ∈ℒ\#:γ\|\[0,q\]​is constant\}\.\\displaystyle:=\\left\\\{\\gamma\\in\{\\mathscr\{L\}\_\{\\\#\}\}:\\,\\gamma\|\_\{\[0,q\]\}\\ \\mbox\{is constant\}\\right\\\}\.\(52\)Then, we see thatγ∈ℒ\#​\(q\)\\gamma\\in\{\\mathscr\{L\}\_\{\\\#\}\}\(q\)if and only if\(μ,c\)∈ℒ​\(q\)\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)with\(μ,c\)=\(γ′/γ2,1/γ​\(1\)\)\(\\mu,c\)=\(\\gamma^\{\\prime\}/\\gamma^\{2\},1/\\gamma\(1\)\)\.

We now extend the definition of the Parisi functional in[Eq\.15](https://arxiv.org/html/2606.28573#S2.E15)\(we setk=1k=1and drop the subscript11for notational simplicity\) from𝒰×ℝ\>0\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}to the larger function spaceℒ\{\\mathscr\{L\}\}\. For\(μ,c\)∈ℒ\(\\mu,c\)\\in\{\\mathscr\{L\}\}andr∈\[−1,1\]r\\in\[\-1,1\], define

𝖥​\(μ,c,r\)=𝔼Y,G​\[fY,μ​\(r2,r​G\)\]\+12​α​∫r21d​tc\+∫t1μ​\(u\)​d​u,\\mathsf\{F\}\(\\mu,c,r\)=\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}\\left\(r^\{2\},rG\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(u\)\\textup\{d\}u\}\\,,where for fixedy∈ℝy\\in\\mathbb\{R\},fy,μf\_\{y,\\mu\}is the solution to the partial differential equation \([14](https://arxiv.org/html/2606.28573#S2.E14)\), which we copy here for the reader’s convenience:

∂tfy,μ​\(t,x\)\+12​μ​\(t\)​\(∂xfy,μ​\(t,x\)\)2\+12​∂x2fy,μ​\(t,x\)=0,fy,μ​\(1,x\)=supu∈ℝ\{h​\(y,x\+u\)−u22​c\}\.\\begin\{split\}&\\partial\_\{t\}f\_\{y,\\mu\}\(t,x\)\+\\frac\{1\}\{2\}\\mu\(t\)\(\\partial\_\{x\}f\_\{y,\\mu\}\(t,x\)\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,\\mu\}\(t,x\)=\\,0,\\\\ &f\_\{y,\\mu\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,x\+u\\right\)\-\\frac\{u^\{2\}\}\{2c\}\\right\\\}\.\\end\{split\}\(53\)Later in[Section8\.1](https://arxiv.org/html/2606.28573#S8.SS1), we will establish existence, uniqueness and regularity of solutions to the Parisi PDE, thus ensuring that the Parisi functional is indeed well\-defined onℒ\{\\mathscr\{L\}\}\.

We are now in position to state our main result establishing a Parisi\-type variational principle for𝒱1,α,φAMP​\(r,q,h\)\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\), defined via the stochastic optimal control problem \([28](https://arxiv.org/html/2606.28573#S3.E28)\)\.

###### Theorem 6\.1\.

Assume that for ally∈ℝy\\in\\mathbb\{R\},h​\(y,⋅\)h\(y,\\cdot\)isC2C^\{2\}, Lipschitz continuous and bounded from above\. Further assume that

𝔼​\[‖∂xh​\(Y,⋅\)‖∞2\]<∞\.\\mathbb\{E\}\\left\[\\left\\\|\{\\partial\_\{x\}h\(Y,\\cdot\)\}\\right\\\|\_\{\\infty\}^\{2\}\\right\]<\\infty\.Then, the following holds\.

- \(a\)*Variational formula\.*Fix\(r,q\)∈ABayes\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. For anyγ∈ℒ\#​\(q\)\\gamma\\in\{\\mathscr\{L\}\_\{\\\#\}\}\(q\)and\(μ,c\)∈ℒ​\(q\)\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)satisfyingμ=γ′/γ2\\mu=\\gamma^\{\\prime\}/\\gamma^\{2\}andc=1/γ​\(1\)c=1/\\gamma\(1\), we have: 𝖥\(μ,c,r\)=supF:ℝ2→ℝϕ∈D​\[q,1\]𝔼\[\\displaystyle\\mathsf\{F\}\(\\mu,c,r\)=\\,\\sup\_\{\\begin\{subarray\}\{c\}F:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}\\\\ \\phi\\in D\[q,1\]\\end\{subarray\}\}\\mathbb\{E\}\\Bigg\[h​\(Y,Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+ϕt\)​d​Bt\)−12​∫q1γ​\(t\)​\(ϕt2−1α\)​d​t\\displaystyle h\\left\(Y,\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\\phi\_\{t\}^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t−γ​\(q\)2​α\(F​\(Zr,q,Y\)2α−\(q−r2\)\)\]\.\\displaystyle\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(Z\_\{r,q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\Bigg\]\.
- \(b\)*Weak duality\.*For any\(r,q\)∈ABayes\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, we have 𝒱1,α,φAMP​\(r,q,h\)≤inf\(μ,c\)∈ℒ​\(q\)𝖥​\(μ,c,r\)\.\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\)\\leq\\,\\inf\_\{\(\\mu,c\)\\in\\mathscr\{L\}\(q\)\}\\mathsf\{F\}\(\\mu,c,r\)\.\(54\)
- \(c\)*Strong duality\.*For any fixedq∈\[0,1\]q\\in\[0,1\], denoteABayes​\(q\)=\{r∈\[−1,1\]:\(r,q\)∈ABayes\}A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(q\)=\\\{r\\in\[\-1,1\]:\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\\}\. Assume there exists\(μ∗,c∗\)∈ℒ​\(q\)\(\\mu\_\{\*\},c\_\{\*\}\)\\in\{\\mathscr\{L\}\}\(q\)andr∗∈ABayes​\(q\)r\_\{\*\}\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(q\), such that the following holds: - \(i\)The mappingr↦inf\(μ,c\)∈ℒ​\(q\)𝖥​\(μ,c,r\)r\\mapsto\\inf\_\{\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)\}\\mathsf\{F\}\(\\mu,c,r\)is differentiable atr∗r\_\{\*\}; - \(ii\)\(μ∗,c∗,r∗\)\(\\mu\_\{\*\},c\_\{\*\},r\_\{\*\}\)satisfies 𝖥​\(μ∗,c∗,r∗\)=inf\(μ,c\)∈ℒ​\(q\)𝖥​\(μ,c,r∗\)=supr∈ABayes​\(q\)inf\(μ,c\)∈ℒ​\(q\)𝖥​\(μ,c,r\)\.\\mathsf\{F\}\(\\mu\_\{\*\},c\_\{\*\},r\_\{\*\}\)=\\inf\_\{\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)\}\\mathsf\{F\}\(\\mu,c,r\_\{\*\}\)=\\sup\_\{r\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(q\)\}\\inf\_\{\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)\}\\mathsf\{F\}\(\\mu,c,r\)\. Then, there exists a feasible pair\(F∗,ϕ∗\)\(F^\{\*\},\\phi^\{\*\}\)such thatF∗F^\{\*\}is an\(r∗,q\)\(r\_\{\*\},q\)\-contraction, and 𝔼​\[\(ϕt∗\)2\]≤1α,∀t∈\[q,1\]\.\\mathbb\{E\}\[\(\\phi\_\{t\}^\{\*\}\)^\{2\}\]\\leq\\frac\{1\}\{\\alpha\},\\ \\forall t\\in\[q,1\]\.Furthermore, 𝖥​\(μ∗,c∗,r∗\)=𝔼​\[h​\(Y,Zr∗,q\+1α​F∗​\(Zr∗,q,Y\)\+∫q1\(1\+ϕt∗\)​d​Bt\)\]=𝒱1,α,φAMP​\(r∗,q,h\)\.\\mathsf\{F\}\(\\mu\_\{\*\},c\_\{\*\},r\_\{\*\}\)=\\,\\mathbb\{E\}\\left\[h\\left\(Y,\\,Z\_\{r\_\{\*\},q\}\+\\frac\{1\}\{\\alpha\}F^\{\*\}\\left\(Z\_\{r\_\{\*\},q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}^\{\*\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\]=\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r\_\{\*\},q,h\)\.\(55\)

The proof of Theorem[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1)is given in Section[8\.2](https://arxiv.org/html/2606.28573#S8.SS2), with certain technical details deferred to[SectionF\.1](https://arxiv.org/html/2606.28573#A6.SS1)\. Note that, under the conditions of point\(c\)\(c\), we have the dual characterization

𝒱1,α,φAMP​\(r∗,q,h\)=supr∈ABayes​\(q\)inf\(μ,c\)∈ℒ​\(q\)𝖥​\(μ,c,r\)\.\\displaystyle\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r\_\{\*\},q,h\)=\\sup\_\{r\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(q\)\}\\inf\_\{\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)\}\\mathsf\{F\}\(\\mu,c,r\)\\,\.Namely, the optimal value achievable by our two\-stage AMP algorithm is characterized by the Parisi variational principle on an extended function space\.

## 7Two\-stage AMP algorithm: Proof of Theorem[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)

This section will be devoted to establishing our general two\-stage AMP algorithm and the proof of Theorem[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1), with proofs of auxiliary results deferred to Appendix[E](https://arxiv.org/html/2606.28573#A5)\.

Recall from[1\.1](https://arxiv.org/html/2606.28573#S1.Thmass1)that\{\(𝒙i,yi\)\}i∈\[n\]\\\{\(\\text\{$x$\}\_\{i\},y\_\{i\}\)\\\}\_\{i\\in\[n\]\}are i\.i\.d\. pairs satisfying

𝒙i∼𝖭​\(𝟎,𝑰d\),yi=φ​\(𝑾∗𝖳​𝒙i,εi\),εi∼Pε\.\\text\{$x$\}\_\{i\}\\sim\{\\sf N\}\(\\mathrm\{\\bf 0\},\{\\boldsymbol\{I\}\}\_\{d\}\),\\quad y\_\{i\}=\\varphi\(\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\text\{$x$\}\_\{i\},\\varepsilon\_\{i\}\),\\,\\,\\varepsilon\_\{i\}\\sim P\_\{\\varepsilon\}\.Using vector notation, we will write𝒚=φ​\(𝑿𝑾∗,𝜺\)\\text\{$y$\}=\\varphi\(\\text\{$X$\}\\text\{$W$\}\_\{\*\},\{\\boldsymbol\{\\varepsilon\}\}\)occasionally, whereφ\\varphiis understood to act on its arguments row\-wise\. We further note that there is no real loss of generality in assuming𝑾∗∼Unif​\(O​\(d,k\)\)\\text\{$W$\}\_\{\*\}\\sim\\mathrm\{Unif\}\(O\(d,k\)\), the uniform distribution \(Haar measure\) on the Stiefel manifoldO​\(d,k\)O\(d,k\)\(see[B\.1](https://arxiv.org/html/2606.28573#A2.Thmlem1)for the precise statement\)\. Indeed, by rotation invariance of the covariates𝒙i\\text\{$x$\}\_\{i\},Unif​\(O​\(d,k\)\)\\mathrm\{Unif\}\(O\(d,k\)\)is the least favorable prior distribution\. Under this simplification, we will assume that the empirical distribution of the rows ofn​𝑾∗\\sqrt\{n\}\\text\{$W$\}\_\{\*\}converges inW2W\_\{2\}distance to𝖭​\(0,α​Ik\)\{\\sf N\}\(0,\\alpha I\_\{k\}\)asn,d→∞n,d\\to\\infty,n/d→αn/d\\to\\alpha\.

### 7\.1Approximate message passing

Following\[[BM11a](https://arxiv.org/html/2606.28573#bib.bibx6),[JM13](https://arxiv.org/html/2606.28573#bib.bibx23),[CMW20](https://arxiv.org/html/2606.28573#bib.bibx10)\], we define the general AMP algorithm as an iterative procedure that generates two sequences of matrices\{𝑾t\}t≥1⊂ℝd×m\\\{\\text\{$W$\}^\{t\}\\\}\_\{t\\geq 1\}\\subset\\mathbb\{R\}^\{d\\times m\}and\{𝑽t\}t≥1⊂ℝn×m\\\{\\text\{$V$\}^\{t\}\\\}\_\{t\\geq 1\}\\subset\\mathbb\{R\}^\{n\\times m\}according to:

𝑾t\+1=1n​𝑿𝖳​Ft​\(𝑽≤t,𝒚\)−∑s=1tGs​\(𝑾≤s\)​Kt,s𝖳,𝑽t=1n​𝑿​Gt​\(𝑾≤t\)−∑s=1tFs−1​\(𝑽≤s−1,𝒚\)​Dt,s𝖳,\\begin\{split\}\\text\{$W$\}^\{t\+1\}&=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}^\{\\sf T\}F\_\{t\}\(\\text\{$V$\}^\{\\leq t\},\\text\{$y$\}\)\-\\sum\_\{s=1\}^\{t\}G\_\{s\}\(\\text\{$W$\}^\{\\leq s\}\)K\_\{t,s\}^\{\\sf T\},\\\\ \\text\{$V$\}^\{t\}&=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}G\_\{t\}\(\\text\{$W$\}^\{\\leq t\}\)\-\\sum\_\{s=1\}^\{t\}F\_\{s\-1\}\(\\text\{$V$\}^\{\\leq s\-1\},\\text\{$y$\}\)D\_\{t,s\}^\{\\sf T\},\\end\{split\}\(56\)where𝑾1=𝑿𝖳​F0​\(𝒚\)/n\\text\{$W$\}^\{1\}=\\text\{$X$\}^\{\\sf T\}F\_\{0\}\(\\text\{$y$\}\)/\\sqrt\{n\}, and\{Ft:ℝm​t\+1→ℝm\}t≥0\\\{F\_\{t\}:\\mathbb\{R\}^\{mt\+1\}\\to\\mathbb\{R\}^\{m\}\\\}\_\{t\\geq 0\}and\{Gt:ℝm​t→ℝm\}t≥1\\\{G\_\{t\}:\\mathbb\{R\}^\{mt\}\\to\\mathbb\{R\}^\{m\}\\\}\_\{t\\geq 1\}are two sequences of Lipschitz functions\. Moreover, we let𝑾≤t=\(𝑾s\)1≤s≤t\\text\{$W$\}^\{\\leq t\}=\(\\text\{$W$\}^\{s\}\)\_\{1\\leq s\\leq t\},𝑽≤t=\(𝑽s\)1≤s≤t\\text\{$V$\}^\{\\leq t\}=\(\\text\{$V$\}^\{s\}\)\_\{1\\leq s\\leq t\}, and adopt the convention that the Lipschitz functionsGtG\_\{t\}andFtF\_\{t\}apply row\-wise to their arguments\. Them×mm\\times mmatricesDt,sD\_\{t,s\}andKt,sK\_\{t,s\}are defined as

Dt,s=1n​∑i=1d∂Gt∂𝒘is​\(𝒘i1,⋯,𝒘it\),Kt,s=1n​∑i=1n∂Ft∂𝒗is​\(𝒗i1,⋯,𝒗it,yi\),D\_\{t,s\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{d\}\\frac\{\\partial G\_\{t\}\}\{\\partial\\text\{$w$\}\_\{i\}^\{s\}\}\(\\text\{$w$\}\_\{i\}^\{1\},\\cdots,\\text\{$w$\}\_\{i\}^\{t\}\),\\ \\,\\,K\_\{t,s\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\frac\{\\partial F\_\{t\}\}\{\\partial\\text\{$v$\}\_\{i\}^\{s\}\}\(\\text\{$v$\}\_\{i\}^\{1\},\\cdots,\\text\{$v$\}\_\{i\}^\{t\},y\_\{i\}\),\(57\)where𝒘is\\text\{$w$\}\_\{i\}^\{s\}is theii\-th row of𝑾s\\text\{$W$\}^\{s\}and𝒗is\\text\{$v$\}\_\{i\}^\{s\}is theii\-th row of𝑽s\\text\{$V$\}^\{s\}, respectively\.

###### Remark 6\.

We will refer toDt,sD\_\{t,s\}andKt,sK\_\{t,s\}in[Eq\.57](https://arxiv.org/html/2606.28573#S7.E57)as “Onsager coefficients”\. The population versions of these coefficients are used in some of the earlier literature, where the empirical average overiiis replaced by an expectation over the asymptotic distributions of the𝒘it\\text\{$w$\}^\{t\}\_\{i\}’s and𝒗it\\text\{$v$\}^\{t\}\_\{i\}’s\. By an induction argument in\[[JM13](https://arxiv.org/html/2606.28573#bib.bibx23)\], the high\-dimensional asymptotics of these two versions of the general AMP algorithm are the same\.

Asn,d→∞n,d\\to\\inftyandn/d→αn/d\\to\\alpha, for any fixedt∈ℕt\\in\\mathbb\{N\}, the limiting joint distribution of the firstttAMP iterates is exactly characterized by the following proposition\.

###### Proposition 7\.1\(State evolution of AMP\)\.

LetY∈ℝY\\in\\mathbb\{R\}andZ¯0,V∈ℝk\\overline\{Z\}\_\{0\},V\\in\\mathbb\{R\}^\{k\}be row vectors\. DenoteZ¯≤t:=\(Z¯1,⋯,Z¯t\)∈ℝm​t\\overline\{Z\}\_\{\\leq t\}:=\(\\overline\{Z\}\_\{1\},\\cdots,\\overline\{Z\}\_\{t\}\)\\in\\mathbb\{R\}^\{mt\}andZ≤t:=\(Z1,⋯,Zt\)∈ℝm​tZ\_\{\\leq t\}:=\(Z\_\{1\},\\cdots,Z\_\{t\}\)\\in\\mathbb\{R\}^\{mt\}, where eachZt,Z¯t∈ℝmZ\_\{t\},\\overline\{Z\}\_\{t\}\\in\\mathbb\{R\}^\{m\}is a row vector\. Then, the distributions of the random row vectors\(Z¯0,Z¯≤t,Y\)∈ℝm​t\+k\+1\(\\overline\{Z\}\_\{0\},\\overline\{Z\}\_\{\\leq t\},Y\)\\in\\mathbb\{R\}^\{mt\+k\+1\}and\(Z≤t,V\)∈ℝm​t\+k\(Z\_\{\\leq t\},V\)\\in\\mathbb\{R\}^\{mt\+k\}are defined as follows: \(i\)V∼𝖭​\(0,α​Ik\)V\\sim\{\\sf N\}\(0,\\alpha I\_\{k\}\)is independent of\(Zi\)i≥1\(Z\_\{i\}\)\_\{i\\geq 1\}, andε∼Pε\\varepsilon\\sim P\_\{\\varepsilon\}is independent of\(Z¯i\)i≥0\(\\overline\{Z\}\_\{i\}\)\_\{i\\geq 0\}, \(ii\) both\(Z¯0,Z¯≤t\)\(\\overline\{Z\}\_\{0\},\\overline\{Z\}\_\{\\leq t\}\)andZ≤tZ\_\{\\leq t\}are multivariate Gaussian with zero mean, and their covariance structures are specified via the following recursion:

𝔼​\[Z¯i𝖳​Z¯j\]=1α​𝔼​\[Gi​\(V​R≤i\+Z≤i\)𝖳​Gj​\(V​R≤j\+Z≤j\)\],i,j≥1,𝔼​\[Z¯i𝖳​Z¯0\]=1α​𝔼​\[Gi​\(V​R≤i\+Z≤i\)𝖳​V\],𝔼​\[Z¯0𝖳​Z¯0\]=1α​𝔼​\[V𝖳​V\]=Ik,i≥1,𝔼​\[Zi𝖳​Zj\]=𝔼​\[Fi−1​\(Z¯≤i−1,Y\)𝖳​Fj−1​\(Z¯≤j−1,Y\)\],i,j≥1,Y=φ​\(Z¯0,ε\),Rt\+1=𝔼​\[∂Ft∂z¯0​\(Z¯≤t,φ​\(Z¯0,ε\)\)\]∈ℝk×m,t≥0\.\\begin\{split\}&\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{i\}^\{\\sf T\}\\overline\{Z\}\_\{j\}\\right\]=\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{i\}\\left\(VR\_\{\\leq i\}\+Z\_\{\\leq i\}\\right\)^\{\\sf T\}G\_\{j\}\\left\(VR\_\{\\leq j\}\+Z\_\{\\leq j\}\\right\)\\right\],\\quad i,j\\geq 1,\\\\ &\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{i\}^\{\\sf T\}\\overline\{Z\}\_\{0\}\\right\]=\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{i\}\\left\(VR\_\{\\leq i\}\+Z\_\{\\leq i\}\\right\)^\{\\sf T\}V\\right\],\\quad\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{0\}^\{\\sf T\}\\overline\{Z\}\_\{0\}\\right\]=\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[V^\{\\sf T\}V\\right\]=I\_\{k\},\\quad i\\geq 1,\\\\ &\\mathbb\{E\}\\left\[Z\_\{i\}^\{\\sf T\}Z\_\{j\}\\right\]=\\mathbb\{E\}\\left\[F\_\{i\-1\}\\left\(\\overline\{Z\}\_\{\\leq i\-1\},Y\\right\)^\{\\sf T\}F\_\{j\-1\}\\left\(\\overline\{Z\}\_\{\\leq j\-1\},Y\\right\)\\right\],\\quad i,j\\geq 1,\\\\ &Y=\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\),\\ R\_\{t\+1\}=\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{t\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{\\leq t\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]\\in\\mathbb\{R\}^\{k\\times m\},\\quad t\\geq 0\.\\end\{split\}\(58\)In the above display, we denoteR≤t=\(R1,⋯,Rt\)∈ℝk×m​tR\_\{\\leq t\}=\(R\_\{1\},\\cdots,R\_\{t\}\)\\in\\mathbb\{R\}^\{k\\times mt\}for allt≥1t\\geq 1, so thatV​R≤t\+Z≤t=\(V​R1\+Z1,⋯,V​Rt\+Zt\)∈ℝm​tVR\_\{\\leq t\}\+Z\_\{\\leq t\}=\(VR\_\{1\}\+Z\_\{1\},\\cdots,VR\_\{t\}\+Z\_\{t\}\)\\in\\mathbb\{R\}^\{mt\}, and in the definition ofRt\+1R\_\{t\+1\},∂Ft/∂z¯0\\partial F\_\{t\}/\\partial\\overline\{z\}\_\{0\}represents the Jacobian ofFtF\_\{t\}with respect toZ¯0∈ℝk\\overline\{Z\}\_\{0\}\\in\\mathbb\{R\}^\{k\}\. Under this specification, we have

p−limn→∞⁡Dt,s=1α​𝔼​\[∂Gt∂ws​\(V​R≤t\+Z≤t\)\],p−limn→∞⁡Kt,s=𝔼​\[∂Ft∂vs​\(Z¯≤t,Y\)\]\.\\operatorname\*\{p\-lim\}\_\{n\\to\\infty\}D\_\{t,s\}=\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial G\_\{t\}\}\{\\partial w^\{s\}\}\\left\(VR\_\{\\leq t\}\+Z\_\{\\leq t\}\\right\)\\right\],\\ \\operatorname\*\{p\-lim\}\_\{n\\to\\infty\}K\_\{t,s\}=\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{t\}\}\{\\partial v^\{s\}\}\\left\(\\overline\{Z\}\_\{\\leq t\},Y\\right\)\\right\]\.Furthermore, denoting by𝐯i∈ℝk\\text\{$v$\}\_\{i\}\\in\\mathbb\{R\}^\{k\}theii\-th row ofn​𝐖∗∈ℝd×k\\sqrt\{n\}\\text\{$W$\}\_\{\*\}\\in\\mathbb\{R\}^\{d\\times k\}fori∈\[d\]i\\in\[d\], we have for any pseudo\-Lipschitz functionsψ1:ℝm​t\+k→ℝ\\psi\_\{1\}:\\mathbb\{R\}^\{mt\+k\}\\to\\mathbb\{R\}andψ2:ℝm​t\+1→ℝ\\psi\_\{2\}:\\mathbb\{R\}^\{mt\+1\}\\to\\mathbb\{R\}:

limn→∞1d​∑i=1dψ1​\(𝒘i1,⋯,𝒘it,𝒗i\)=𝔼​\[ψ1​\(V​R≤t\+Z≤t,V\)\],limn→∞1n​∑i=1nψ2​\(𝒗i1,⋯,𝒗it,yi\)=𝔼​\[ψ2​\(Z¯≤t,Y\)\],\\begin\{split\}\\lim\_\{n\\to\\infty\}\\frac\{1\}\{d\}\\sum\_\{i=1\}^\{d\}\\psi\_\{1\}\\left\(\\text\{$w$\}\_\{i\}^\{1\},\\cdots,\\text\{$w$\}\_\{i\}^\{t\},\\text\{$v$\}\_\{i\}\\right\)&=\\mathbb\{E\}\\left\[\\psi\_\{1\}\\left\(VR\_\{\\leq t\}\+Z\_\{\\leq t\},V\\right\)\\right\],\\\\ \\lim\_\{n\\to\\infty\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\psi\_\{2\}\\left\(\\text\{$v$\}\_\{i\}^\{1\},\\cdots,\\text\{$v$\}\_\{i\}^\{t\},y\_\{i\}\\right\)&=\\mathbb\{E\}\\left\[\\psi\_\{2\}\\left\(\\overline\{Z\}\_\{\\leq t\},Y\\right\)\\right\],\\end\{split\}\(59\)almost surely asn,d→∞n,d\\to\\inftyandn/d→αn/d\\to\\alpha\.

###### Proof\.

This can be deduced from the results in\[[JM13](https://arxiv.org/html/2606.28573#bib.bibx23),[CMW20](https://arxiv.org/html/2606.28573#bib.bibx10),[MW22](https://arxiv.org/html/2606.28573#bib.bibx33)\]\. ∎

###### Remark 7\.

As mentioned in the introduction, we allow our two\-stage AMP algorithm to be randomized\. Within the framework of this section, randomization can be implemented by letting the functionsFtF\_\{t\}\(orGtG\_\{t\}\) depend on some additional randomness\. For example, one can replaceFt​\(𝒗i1,…,𝒗it,yi\)F\_\{t\}\(\\text\{$v$\}\_\{i\}^\{1\},\\dots,\\text\{$v$\}^\{t\}\_\{i\},y\_\{i\}\)byFt​\(𝒗i1,…,𝒗it,yi,ωi\)F\_\{t\}\(\\text\{$v$\}\_\{i\}^\{1\},\\dots,\\text\{$v$\}^\{t\}\_\{i\},y\_\{i\},\\omega\_\{i\}\)with\(ωi\)i≥1∼i\.i\.d\.Unif​\[0,1\]\(\\omega\_\{i\}\)\_\{i\\geq 1\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\\mathrm\{Unif\}\[0,1\], the uniform distribution on\[0,1\]\[0,1\]\. For simplicity of notation, we will leave this dependence implicit\. Expectations in the state evolution equations are understood to be taken with respect to these random variables as well\.

###### Corollary 7\.2\.

Asn,d→∞n,d\\to\\infty,n/d→αn/d\\to\\alpha, the empirical distribution of\(𝐯i1,⋯,𝐯it,yi\)1≤i≤n\(\\text\{$v$\}\_\{i\}^\{1\},\\cdots,\\text\{$v$\}\_\{i\}^\{t\},y\_\{i\}\)\_\{1\\leq i\\leq n\}almost surely weakly converges to the law of\(Z¯≤t,Y\)\(\\overline\{Z\}\_\{\\leq t\},Y\)\. Similarly, the empirical distribution of\(𝐰i1,⋯,𝐰it,𝐯i\)1≤i≤d\(\\text\{$w$\}\_\{i\}^\{1\},\\cdots,\\text\{$w$\}\_\{i\}^\{t\},\\text\{$v$\}\_\{i\}\)\_\{1\\leq i\\leq d\}almost surely weakly converges to the law of\(V​R≤t\+Z≤t,V\)\(VR\_\{\\leq t\}\+Z\_\{\\leq t\},V\)\.

###### Proof\.

Identical to the proof of\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Corollary 4\.2\]\. ∎

### 7\.2First stage: Fixed\-point AMP

In this section, we present the first stage of our general two\-stage AMP algorithm, which consists of several vanilla AMP iterations\. The AMP iterations in this stage eventually converge to a fixed point of the state evolution equations \([58](https://arxiv.org/html/2606.28573#S7.E58)\), which will be the starting point of the pure incremental part in the second stage\.

We now specify our choices ofFtF\_\{t\}andGtG\_\{t\}\. For this stage, we letFtF\_\{t\}be a function that only depends on\(𝑽t,𝒚\)\(\\text\{$V$\}^\{t\},\\text\{$y$\}\), and letGtG\_\{t\}be the identity mapping of𝑾t\\text\{$W$\}^\{t\}\. Consequently,Dt=\(d/n\)​ImD\_\{t\}=\(d/n\)I\_\{m\}and the AMP iterations reduce to

𝑾t\+1=1n​𝑿𝖳​Ft​\(𝑽t,𝒚\)−𝑾t​Kt𝖳,𝑽t=1n​𝑿𝑾t−dn​Ft−1​\(𝑽t−1,𝒚\),\\begin\{split\}\\text\{$W$\}^\{t\+1\}&=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}^\{\\sf T\}F\_\{t\}\(\\text\{$V$\}^\{t\},\\text\{$y$\}\)\-\\text\{$W$\}^\{t\}K\_\{t\}^\{\\sf T\},\\\\ \\text\{$V$\}^\{t\}&=\\frac\{1\}\{\\sqrt\{n\}\}\\text\{$X$\}\\text\{$W$\}^\{t\}\-\\frac\{d\}\{n\}F\_\{t\-1\}\(\\text\{$V$\}^\{t\-1\},\\text\{$y$\}\),\\end\{split\}\(60\)where we still have𝑾1=𝑿𝖳​F0​\(𝒚\)/n\\text\{$W$\}^\{1\}=\\text\{$X$\}^\{\\sf T\}F\_\{0\}\(\\text\{$y$\}\)/\\sqrt\{n\}, and

Kt=1n​∑i=1n∂Ft∂𝒗it​\(𝒗it,yi\)K\_\{t\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\frac\{\\partial F\_\{t\}\}\{\\partial\\text\{$v$\}\_\{i\}^\{t\}\}\(\\text\{$v$\}\_\{i\}^\{t\},y\_\{i\}\)\(61\)are the Onsager terms\. With this simple choice, the state evolution equations \([58](https://arxiv.org/html/2606.28573#S7.E58)\) reduce to

𝔼​\[Z¯i𝖳​Z¯j\]=Ri𝖳​Rj\+1α​𝔼​\[Zi𝖳​Zj\],i,j≥1,𝔼​\[Z¯i𝖳​Z¯0\]=Ri𝖳,𝔼​\[Z¯0𝖳​Z¯0\]=Ik,i≥1,𝔼​\[Z1𝖳​Z1\]=𝔼​\[F0​\(Y\)𝖳​F0​\(Y\)\],𝔼​\[Zi𝖳​Z1\]=𝔼​\[Fi−1​\(Z¯i−1,Y\)𝖳​F0​\(Y\)\],i≥2,𝔼​\[Zi𝖳​Zj\]=𝔼​\[Fi−1​\(Z¯i−1,Y\)𝖳​Fj−1​\(Z¯j−1,Y\)\],i,j≥2,Y=φ​\(Z¯0,ε\),R1=𝔼​\[∂F0∂z¯0​\(φ​\(Z¯0,ε\)\)\]∈ℝk×m,Rt\+1=𝔼​\[∂Ft∂z¯0​\(Z¯t,φ​\(Z¯0,ε\)\)\]∈ℝk×m,t≥1\.\\begin\{split\}&\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{i\}^\{\\sf T\}\\overline\{Z\}\_\{j\}\\right\]=R\_\{i\}^\{\\sf T\}R\_\{j\}\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[Z\_\{i\}^\{\\sf T\}Z\_\{j\}\\right\],\\quad i,j\\geq 1,\\\\ &\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{i\}^\{\\sf T\}\\overline\{Z\}\_\{0\}\\right\]=R\_\{i\}^\{\\sf T\},\\quad\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{0\}^\{\\sf T\}\\overline\{Z\}\_\{0\}\\right\]=I\_\{k\},\\quad i\\geq 1,\\\\ &\\mathbb\{E\}\\left\[Z\_\{1\}^\{\\sf T\}Z\_\{1\}\\right\]=\\mathbb\{E\}\\left\[F\_\{0\}\\left\(Y\\right\)^\{\\sf T\}F\_\{0\}\\left\(Y\\right\)\\right\],\\\\ &\\mathbb\{E\}\\left\[Z\_\{i\}^\{\\sf T\}Z\_\{1\}\\right\]=\\mathbb\{E\}\\left\[F\_\{i\-1\}\\left\(\\overline\{Z\}\_\{i\-1\},Y\\right\)^\{\\sf T\}F\_\{0\}\\left\(Y\\right\)\\right\],\\quad i\\geq 2,\\\\ &\\mathbb\{E\}\\left\[Z\_\{i\}^\{\\sf T\}Z\_\{j\}\\right\]=\\mathbb\{E\}\\left\[F\_\{i\-1\}\\left\(\\overline\{Z\}\_\{i\-1\},Y\\right\)^\{\\sf T\}F\_\{j\-1\}\\left\(\\overline\{Z\}\_\{j\-1\},Y\\right\)\\right\],\\quad i,j\\geq 2,\\\\ &Y=\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\),\\ R\_\{1\}=\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{0\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]\\in\\mathbb\{R\}^\{k\\times m\},\\\\ &R\_\{t\+1\}=\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{t\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{t\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]\\in\\mathbb\{R\}^\{k\\times m\},\\quad t\\geq 1\.\\end\{split\}\(62\)
In order to specify our choices of the Lipschitz functions\{Ft\}\\\{F\_\{t\}\\\}in this stage, we first define two crucial quantities that track the state evolution of AMP:

Rt=𝔼​\[Z¯0𝖳​Z¯t\]∈ℝk×m,Qt=𝔼​\[Z¯t𝖳​Z¯t\]∈𝒮\+m\.R\_\{t\}=\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{0\}^\{\\sf T\}\\overline\{Z\}\_\{t\}\\right\]\\in\\mathbb\{R\}^\{k\\times m\},\\quad Q\_\{t\}=\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{t\}^\{\\sf T\}\\overline\{Z\}\_\{t\}\\right\]\\in\{\\mathcal\{S\}\}\_\{\+\}^\{m\}\.Recall the Bayes AMP feasible region from[1](https://arxiv.org/html/2606.28573#Thmdefn1):

ABayes=\{\(R,Q\)∈ℝk×m×𝒮\+m:R𝖳​R⪯Q⪯Im,R​\(Q−R𝖳​R\)−1​R𝖳⪯CBayes​CBayes𝖳\}\.A\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\,\\left\\\{\(R,Q\)\\in\\mathbb\{R\}^\{k\\times m\}\\times\{\\mathcal\{S\}\}\_\{\+\}^\{m\}:\\,R^\{\\sf T\}R\\preceq Q\\preceq I\_\{m\},\\,R\(Q\-R^\{\\sf T\}R\)^\{\-1\}R^\{\\sf T\}\\preceq C\_\{\\mbox\{\\tiny\\rm Bayes\}\}C\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\sf T\}\\right\\\}\.The proposition below establishes the connection betweenABayesA\_\{\\mbox\{\\tiny\\rm Bayes\}\}and the collection of\(Rt,Qt\)\(R\_\{t\},Q\_\{t\}\)’s achievable via vanilla AMP iterations\.

###### Proposition 7\.3\.

Under state evolution \([62](https://arxiv.org/html/2606.28573#S7.E62)\), the following hold:

- \(a\)For allt∈ℕt\\in\\mathbb\{N\},\(Rt,Qt\)∈ℝk×m×𝒮\+m\(R\_\{t\},Q\_\{t\}\)\\in\\mathbb\{R\}^\{k\\times m\}\\times\{\\mathcal\{S\}\}\_\{\+\}^\{m\}satisfies Rt​\(Qt−Rt𝖳​Rt\)−1​Rt𝖳⪯CBayes​CBayes𝖳\.R\_\{t\}\\big\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\big\)^\{\-1\}R\_\{t\}^\{\\sf T\}\\preceq C\_\{\\mbox\{\\tiny\\rm Bayes\}\}C\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\sf T\}\.Further,\(Rt,Qt\)∈ABayes\(R\_\{t\},Q\_\{t\}\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}ifQt⪯ImQ\_\{t\}\\preceq I\_\{m\}\.
- \(b\)For any\(R,Q\)∈int⁡ABayes\(R,Q\)\\in\\operatorname\{int\}A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, there existsT11∈ℕT\_\{11\}\\in\\mathbb\{N\}and Lipschitz functions\{Ft\}t≤T11−1\\\{F\_\{t\}\\\}\_\{t\\leq T\_\{11\}\-1\}, such that\(RT11,QT11\)=\(R,Q\)\(R\_\{T\_\{11\}\},Q\_\{T\_\{11\}\}\)=\(R,Q\)\.

Now for any\(R,Q\)∈int⁡ABayes\(R,Q\)\\in\\operatorname\{int\}A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, we can find someT11∈ℕT\_\{11\}\\in\\mathbb\{N\}and\{Ft\}t≤T11−1\\\{F\_\{t\}\\\}\_\{t\\leq T\_\{11\}\-1\}such that\(RT11,QT11\)=\(R,Q\)\(R\_\{T\_\{11\}\},Q\_\{T\_\{11\}\}\)=\(R,Q\)\. LetFFbe an\(R,Q\)\(R,Q\)\-contraction as defined in[2](https://arxiv.org/html/2606.28573#Thmdefn2)\. We setFt=FF\_\{t\}=Ffor allT11≤t≤T1−1T\_\{11\}\\leq t\\leq T\_\{1\}\-1, whereT1∈ℕT\_\{1\}\\in\\mathbb\{N\}is a sufficiently large integer to be determined\.

###### Lemma 7\.4\.

DenotePt=𝔼​\[Z¯t\+1𝖳​Z¯t\]P\_\{t\}=\\mathbb\{E\}\[\\overline\{Z\}\_\{t\+1\}^\{\\sf T\}\\overline\{Z\}\_\{t\}\]\. Then for allT11\+1≤t≤T1T\_\{11\}\+1\\leq t\\leq T\_\{1\}, we have\(Rt,Qt\)=\(R,Q\)\(R\_\{t\},Q\_\{t\}\)=\(R,Q\), andlimT1→∞PT1−1=Q\\lim\_\{T\_\{1\}\\to\\infty\}P\_\{T\_\{1\}\-1\}=Q\.

As discussed in[Section3\.1](https://arxiv.org/html/2606.28573#S3.SS1), this fixed\-point AMP stage will end afterT1T\_\{1\}steps, withT1∈ℕT\_\{1\}\\in\\mathbb\{N\}to be determined later\.

### 7\.3Second stage: Incremental AMP

In this section, we describe the second stage of our algorithm, which is an incremental AMP \(IAMP\) procedure first introduced in\[[Mon19](https://arxiv.org/html/2606.28573#bib.bibx28)\]\. We will see that the asymptotics of this incremental stage admit a stochastic integral representation under a suitable scaling limit\.

For this IAMP stage, the non\-linear functions\{Ft\}t≥T1\\\{F\_\{t\}\\\}\_\{t\\geq T\_\{1\}\}and\{Gt\}t≥T1\+1\\\{G\_\{t\}\\\}\_\{t\\geq T\_\{1\}\+1\}depend on all previous AMP iterations, and are chosen to satisfy the following assumption:

###### Assumption 7\.1\.

Consider the random variables\(\(Vt\)t≥1,Y\)\(\(V^\{t\}\)\_\{t\\geq 1\},Y\)and\(\(Wt\)t≥1,V\)\(\(W^\{t\}\)\_\{t\\geq 1\},V\)defined as follows:

\(Vt\)1≤t≤T1=\(Z¯t\)1≤t≤T1,Y=φ\(Z¯0,ε\),Z¯0∼𝖭\(0,Ik\),ε∼Pε,Z¯0⟂⟂ε,\(Vt\)t≥T1\+1∼i\.i\.d\.𝖭\(0,Im\),\(Vt\)t≥T1\+1⟂⟂\(\(Vt\)1≤t≤T1,Y\),\(Wt\)1≤t≤T1=\(Zt\)1≤t≤T1,V⟂⟂\(Wt\)t≥1,V∼𝖭\(0,αIk\),\(Wt\)t≥T1\+1∼i\.i\.d\.𝖭\(0,Im\),\(Wt\)t≥T1\+1⟂⟂\(\(Wt\)1≤t≤T1,V\)\.\\begin\{split\}&\(V^\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\}=\(\\overline\{Z\}\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},\\ Y=\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\),\\ \\overline\{Z\}\_\{0\}\\sim\{\\sf N\}\\left\(0,I\_\{k\}\\right\),\\ \\varepsilon\\sim P\_\{\\varepsilon\},\\ \\overline\{Z\}\_\{0\}\\perp\\\!\\\!\\\!\\perp\\varepsilon,\\\\ &\(V^\{t\}\)\_\{t\\geq T\_\{1\}\+1\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\),\\ \(V^\{t\}\)\_\{t\\geq T\_\{1\}\+1\}\\perp\\\!\\\!\\\!\\perp\\left\(\(V^\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},Y\\right\),\\\\ &\(W^\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\}=\(Z\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},\\ V\\perp\\\!\\\!\\\!\\perp\(W^\{t\}\)\_\{t\\geq 1\},\\ V\\sim\{\\sf N\}\(0,\\alpha I\_\{k\}\),\\\\ &\(W^\{t\}\)\_\{t\\geq T\_\{1\}\+1\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\),\\ \(W^\{t\}\)\_\{t\\geq T\_\{1\}\+1\}\\perp\\\!\\\!\\\!\\perp\\left\(\(W^\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},V\\right\)\.\\end\{split\}We impose the following second moment constraints on\{Ft\}t≥T1\\\{F\_\{t\}\\\}\_\{t\\geq T\_\{1\}\}and\{Gt\}t≥T1\+1\\\{G\_\{t\}\\\}\_\{t\\geq T\_\{1\}\+1\}:

1. 1\.FT1F\_\{T\_\{1\}\}is only a function ofY=φ​\(Z¯0,ε\)Y=\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)with𝔼​\[FT1​\(Y\)𝖳​FT1​\(Y\)\]=Im\\mathbb\{E\}\[F\_\{T\_\{1\}\}\(Y\)^\{\\sf T\}F\_\{T\_\{1\}\}\(Y\)\]=I\_\{m\}\. Recall from[Eq\.58](https://arxiv.org/html/2606.28573#S7.E58)thatRT1\+1=𝔼​\[∂z¯0FT1​\(φ​\(Z¯0,ε\)\)\]R\_\{T\_\{1\}\+1\}=\\mathbb\{E\}\[\\partial\_\{\\overline\{z\}\_\{0\}\}F\_\{T\_\{1\}\}\(\\varphi\(\\overline\{Z\}\_\{0\},\\varepsilon\)\)\]\. We also require thatGT1\+1G\_\{T\_\{1\}\+1\}is only a function ofV​RT1\+1\+WT1\+1VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}satisfying 𝔼​\[GT1\+1​\(V​RT1\+1\+WT1\+1\)𝖳​GT1\+1​\(V​RT1\+1\+WT1\+1\)\]=α​Im\.\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)^\{\\sf T\}G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)\\right\]=\\alpha I\_\{m\}\.Furthermore, recallingFFand\(R,Q\)\(R,Q\)from[Section7\.2](https://arxiv.org/html/2606.28573#S7.SS2), we require 𝔼\(R,Q\)​\[FT1​\(φ​\(Z¯0,ε\)\)𝖳​F​\(Z¯,φ​\(Z¯0,ε\)\)\]=\\displaystyle\\mathbb\{E\}\_\{\(R,Q\)\}\\left\[F\_\{T\_\{1\}\}\\left\(\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)^\{\\sf T\}F\\left\(\\overline\{Z\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\,0,\\displaystyle 0,𝔼​\[GT1\+1​\(V​RT1\+1\+WT1\+1\)𝖳​V\]=\\displaystyle\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)^\{\\sf T\}V\\right\]=\\,0,\\displaystyle 0,where we recall that𝔼\(R,Q\)\\mathbb\{E\}\_\{\(R,Q\)\}represents the expectation taken under\(Z¯,Z¯0\)𝖳∼𝖭​\(0,\[QR𝖳RIk\]\)\(\\overline\{Z\},\\overline\{Z\}\_\{0\}\)^\{\\sf T\}\\sim\{\\sf N\}\\left\(0,\\begin\{bmatrix\}Q&R^\{\\sf T\}\\\\ R&I\_\{k\}\\end\{bmatrix\}\\right\), independent ofε∼Pε\\varepsilon\\sim P\_\{\\varepsilon\}\.
2. 2\.The functions\{Ft\}t≥T1\+1\\\{F\_\{t\}\\\}\_\{t\\geq T\_\{1\}\+1\}and\{Gt\}t≥T1\+2\\\{G\_\{t\}\\\}\_\{t\\geq T\_\{1\}\+2\}have the form: Ft​\(V≤t,Y\)=Vt​Φt−1​\(V≤t−1,Y\),Gt​\(W≤t\)=Wt​Ψt−1​\(W≤t−1\),F\_\{t\}\(V^\{\\leq t\},Y\)=V^\{t\}\\Phi\_\{t\-1\}\(V^\{\\leq t\-1\},Y\),\\ G\_\{t\}\(W^\{\\leq t\}\)=W^\{t\}\\Psi\_\{t\-1\}\(W^\{\\leq t\-1\}\),whereΦt−1:ℝm​\(t−1\)\+1→ℝm×m\\Phi\_\{t\-1\}:\\mathbb\{R\}^\{m\(t\-1\)\+1\}\\to\\mathbb\{R\}^\{m\\times m\}andΨt−1:ℝm​\(t−1\)→ℝm×m\\Psi\_\{t\-1\}:\\mathbb\{R\}^\{m\(t\-1\)\}\\to\\mathbb\{R\}^\{m\\times m\}arem×mm\\times mmatrix\-valued functions satisfying 𝔼​\[Φt−1​\(V≤t−1,Y\)𝖳​Φt−1​\(V≤t−1,Y\)\]=Im,𝔼​\[Ψt−1​\(\(V​Rs\+Ws\)s=1T1\+1,\(Ws\)s=T1\+2t−1\)𝖳​Ψt−1​\(\(V​Rs\+Ws\)s=1T1\+1,\(Ws\)s=T1\+2t−1\)\]=α​Im\.\\begin\{split\}&\\mathbb\{E\}\\left\[\\Phi\_\{t\-1\}\\left\(V^\{\\leq t\-1\},Y\\right\)^\{\\sf T\}\\Phi\_\{t\-1\}\\left\(V^\{\\leq t\-1\},Y\\right\)\\right\]=I\_\{m\},\\\\ &\\mathbb\{E\}\\left\[\\Psi\_\{t\-1\}\\left\(\(VR\_\{s\}\+W^\{s\}\)\_\{s=1\}^\{T\_\{1\}\+1\},\(W^\{s\}\)\_\{s=T\_\{1\}\+2\}^\{t\-1\}\\right\)^\{\\sf T\}\\Psi\_\{t\-1\}\\left\(\(VR\_\{s\}\+W^\{s\}\)\_\{s=1\}^\{T\_\{1\}\+1\},\(W^\{s\}\)\_\{s=T\_\{1\}\+2\}^\{t\-1\}\\right\)\\right\]=\\alpha I\_\{m\}\.\\end\{split\}

It is straightforward to verify that there exist functions\{Ft\}t≥T1\\\{F\_\{t\}\\\}\_\{t\\geq T\_\{1\}\}and\{Gt\}t≥T1\+1\\\{G\_\{t\}\\\}\_\{t\\geq T\_\{1\}\+1\}satisfying the above assumption\. Under such choices, the state evolution of the IAMP stage is characterized by the following:

###### Proposition 7\.5\.

Under Assumption[7\.1](https://arxiv.org/html/2606.28573#S7.Thmass1),\(Zt\)t≥T1\+1∼i\.i\.d\.𝖭​\(0,Im\)\(Z\_\{t\}\)\_\{t\\geq T\_\{1\}\+1\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\)is independent of\(\(Zt\)1≤t≤T1,V\)\(\(Z\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},V\), and\(Z¯t\)t≥T1\+1∼i\.i\.d\.𝖭​\(0,Im\)\(\\overline\{Z\}\_\{t\}\)\_\{t\\geq T\_\{1\}\+1\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\)is independent of\(\(Z¯t\)1≤t≤T1,Y\)\(\(\\overline\{Z\}\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},Y\)\. Further,Rt=0R\_\{t\}=0for allt≥T1\+2t\\geq T\_\{1\}\+2\.

The IAMP stage is run forT2T\_\{2\}steps after the fixed\-point AMP stage, for someT2∈ℕT\_\{2\}\\in\\mathbb\{N\}to be determined\. In the next section, we combine the two stages of our AMP algorithm to complete the proof of[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)\.

### 7\.4Combining the two stages

Let𝑾F=𝑾T1/n\\text\{$W$\}\_\{F\}=\\text\{$W$\}^\{T\_\{1\}\}/\\sqrt\{n\}be the output of the fixed\-point AMP stage\. By state evolution equations \([62](https://arxiv.org/html/2606.28573#S7.E62)\) and[7\.4](https://arxiv.org/html/2606.28573#S7.Thmlem4), we have

𝑾F𝖳​𝑾F=\\displaystyle\\text\{$W$\}\_\{F\}^\{\\sf T\}\\text\{$W$\}\_\{F\}=\\,1n​\(𝑾T1\)𝖳​𝑾T1=1n​∑i=1d\(𝒘iT1\)𝖳​𝒘iT1\\displaystyle\\frac\{1\}\{n\}\\left\(\\text\{$W$\}^\{T\_\{1\}\}\\right\)^\{\\sf T\}\\text\{$W$\}^\{T\_\{1\}\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{d\}\\left\(\\text\{$w$\}\_\{i\}^\{T\_\{1\}\}\\right\)^\{\\sf T\}\\text\{$w$\}\_\{i\}^\{T\_\{1\}\}→\\displaystyle\\to\\,1α​𝔼​\[\(V​R\+ZT1\)𝖳​\(V​R\+ZT1\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\left\(VR\+Z\_\{T\_\{1\}\}\\right\)^\{\\sf T\}\\left\(VR\+Z\_\{T\_\{1\}\}\\right\)\\right\]=\\displaystyle=\\,R𝖳​R\+1α​𝔼​\[ZT1𝖳​ZT1\]=𝔼​\[Z¯T1𝖳​Z¯T1\]=Q\\displaystyle R^\{\\sf T\}R\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\}^\{\\sf T\}Z\_\{T\_\{1\}\}\\right\]=\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\}^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\}\\right\]=Qalmost surely asn→∞n\\to\\infty\. LetQ1,⋯,QT2Q\_\{1\},\\cdots,Q\_\{T\_\{2\}\}beT2T\_\{2\}non\-randomm×mm\\times mmatrices such that∑t=1T2Qt𝖳​Qt=Im−Q\\sum\_\{t=1\}^\{T\_\{2\}\}Q\_\{t\}^\{\\sf T\}Q\_\{t\}=I\_\{m\}\-Q\. We then define

𝑾I\\displaystyle\\text\{$W$\}\_\{I\}=1n​∑t=1T2GT1\+t\+1​\(𝑾≤T1\+t\+1\)​Qt\\displaystyle=\\frac\{1\}\{\\sqrt\{n\}\}\\sum\_\{t=1\}^\{T\_\{2\}\}G\_\{T\_\{1\}\+t\+1\}\\left\(\\text\{$W$\}^\{\\leq T\_\{1\}\+t\+1\}\\right\)Q\_\{t\}\\,as the output of the IAMP stage\. Following[Section3\.1](https://arxiv.org/html/2606.28573#S3.SS1), we construct the final output of our two\-stage AMP algorithm by letting𝑾Q=𝑾F\+𝑾I\\text\{$W$\}\_\{Q\}=\\text\{$W$\}\_\{F\}\+\\text\{$W$\}\_\{I\}, and setting𝑾^nAMP=𝑾Q​\(𝑾Q𝖳​𝑾Q\)−1/2\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}=\\text\{$W$\}\_\{Q\}\(\\text\{$W$\}\_\{Q\}^\{\\sf T\}\\text\{$W$\}\_\{Q\}\)^\{\-1/2\}\. It is easy to see that𝑾^nAMP∈O​\(d,m\)\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\in O\(d,m\), as we required in the definition ofℱm,α,φalg\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\. The theorem below characterizes the set of\(α,m\)\(\\alpha,m\)\-feasible distributions achievable by our algorithm:

###### Theorem 7\.1\.

Let Assumption[7\.1](https://arxiv.org/html/2606.28573#S7.Thmass1)hold, and further assume that for allt≥0t\\geq 0,FtF\_\{t\}is continuous, and for allt≥1t\\geq 1,GtG\_\{t\}is Lipschitz continuous\. If the weight matrix𝐖^nAMP\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}is constructed as above, thenlimn→∞𝐖∗𝖳​𝐖^nAMP=R\\lim\_\{n\\to\\infty\}\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}=Ralmost surely\. Further,

1n​∑i=1nδ\(yi,𝒙i𝖳​𝑾^nAMP\)→wLaw⁡\(Y,Z¯T1\+1α​F​\(Z¯T1−1,Y\)\+∑t=1T2\(Z¯T1\+t\+1\+FT1\+t​\(Z¯≤T1\+t,Y\)​At\)​Qt\)\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\\left\(y\_\{i\},\\,\\text\{$x$\}\_\{i\}^\{\\sf T\}\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\right\)\}\\stackrel\{\{\\scriptstyle w\}\}\{\{\\to\}\}\\operatorname\{Law\}\\left\(Y,\\ \\overline\{Z\}\_\{T\_\{1\}\}\+\\frac\{1\}\{\\alpha\}F\\left\(\\overline\{Z\}\_\{T\_\{1\}\-1\},Y\\right\)\+\\sum\_\{t=1\}^\{T\_\{2\}\}\\left\(\\overline\{Z\}\_\{T\_\{1\}\+t\+1\}\+F\_\{T\_\{1\}\+t\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+t\},Y\\right\)A\_\{t\}\\right\)Q\_\{t\}\\right\)almost surely asn→∞n\\to\\infty, whereY=φ​\(Z¯0,ε\)Y=\\varphi\(\\overline\{Z\}\_\{0\},\\varepsilon\), and

At=1α​𝔼​\[ΨT1\+t​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+t\)\],1≤t≤T2\.A\_\{t\}=\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+t\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+t\}\\right\)\\right\],\\ 1\\leq t\\leq T\_\{2\}\.In fact, the above holds for any sequence ofm×mm\\times mmatrices\(At\)1≤t≤T2\(A\_\{t\}\)\_\{1\\leq t\\leq T\_\{2\}\}satisfyingAt𝖳​At⪯Im/αA\_\{t\}^\{\\sf T\}A\_\{t\}\\preceq I\_\{m\}/\\alpha, and functions\(FT1\+t\)1≤t≤T2\(F\_\{T\_\{1\}\+t\}\)\_\{1\\leq t\\leq T\_\{2\}\}satisfying point 2 of Assumption[7\.1](https://arxiv.org/html/2606.28573#S7.Thmass1)\(not necessarily continuous\)\.

We next take the limitT1→∞T\_\{1\}\\to\\infty\. Using[7\.4](https://arxiv.org/html/2606.28573#S7.Thmlem4), we know that

limT1→∞𝔼​\[‖Z¯T1−Z¯T1−1‖22\]=0,\\lim\_\{T\_\{1\}\\to\\infty\}\\mathbb\{E\}\\left\[\\left\\\|\\overline\{Z\}\_\{T\_\{1\}\}\-\\overline\{Z\}\_\{T\_\{1\}\-1\}\\right\\\|\_\{2\}^\{2\}\\right\]=0,and that\(Z¯T1,Z¯0\)𝖳\(\\overline\{Z\}\_\{T\_\{1\}\},\\overline\{Z\}\_\{0\}\)^\{\\sf T\}has the same distribution as\(ZR,Q,G\)\(Z\_\{R,Q\},G\)in[2](https://arxiv.org/html/2606.28573#Thmdefn2)\. Leveraging our assumptions onFFand\{FT1\+t\}t=1T2\\\{F\_\{T\_\{1\}\+t\}\\\}\_\{t=1\}^\{T\_\{2\}\}, the above characterization of AMP\-feasible distributions can be further simplified\. Note that in the theorem below, we recastT2T\_\{2\}asTTand theZ¯t\\overline\{Z\}\_\{t\}’s asVtV^\{t\}’s, and transpose the vectorsVtV^\{t\}and matrices\(Φt,Qt\)\(\\Phi\_\{t\},Q\_\{t\}\)to align with the notation of[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)\.

###### Theorem 7\.2\.

Let\(R,Q\)∈ABayes\(R,Q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\},\(Y,ZR,Q\)\(Y,Z\_\{R,Q\}\)be as described in[2](https://arxiv.org/html/2606.28573#Thmdefn2), andFFbe an\(R,Q\)\(R,Q\)\-contraction\. For anyT∈ℕ\+T\\in\\mathbb\{N\}\_\{\+\}, let\(Vt\)1≤t≤T∼i\.i\.d\.𝖭​\(0,Im\)\(V^\{t\}\)\_\{1\\leq t\\leq T\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\\mathsf\{N\}\(0,I\_\{m\}\)be independent of\(ZR,Q,Y\)\(Z\_\{R,Q\},Y\)\. Define

U=ZR,Q\+1α​F​\(ZR,Q,Y\)\+∑t=1TQt​\(Vt\+1\+Φt−1​\(V≤t−1,ZR,Q,Y\)​Vt\),U=Z\_\{R,Q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{R,Q\},Y\\right\)\+\\sum\_\{t=1\}^\{T\}Q\_\{t\}\\left\(V^\{t\+1\}\+\\Phi\_\{t\-1\}\\left\(V^\{\\leq t\-1\},Z\_\{R,Q\},Y\\right\)V^\{t\}\\right\),\(63\)where

𝔼​\[Φt−1​\(V≤t−1,ZR,Q,Y\)​Φt−1​\(V≤t−1,ZR,Q,Y\)𝖳\]⪯Imα,∀t≥1,∑t=1TQt​Qt𝖳=Im−Q\.\\mathbb\{E\}\\left\[\\Phi\_\{t\-1\}\\left\(V^\{\\leq t\-1\},Z\_\{R,Q\},Y\\right\)\\Phi\_\{t\-1\}\\left\(V^\{\\leq t\-1\},Z\_\{R,Q\},Y\\right\)^\{\\sf T\}\\right\]\\preceq\\frac\{I\_\{m\}\}\{\\alpha\},\\ \\forall t\\geq 1,\\ \\sum\_\{t=1\}^\{T\}Q\_\{t\}Q\_\{t\}^\{\\sf T\}=I\_\{m\}\-Q\.Then, the same conclusion as in[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)holds withUUdefined as per[Eq\.63](https://arxiv.org/html/2606.28573#S7.E63)\.

Finally, we take the scaling limitT→∞T\\to\\inftyfor the AMP\-feasible distribution described in Theorem[7\.2](https://arxiv.org/html/2606.28573#S7.Thmthm2), thus establishing the stochastic integral representation of[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)\. As the proof is nearly identical to the unsupervised setting in\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\], we omit it here for ease of presentation\.

## 8Dual characterization and Parisi formula: Proof of Theorem[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1)

This section will be devoted to the proof of Theorem[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1), thus yielding a dual characterization of𝒱1,α,φAMP​\(h\)\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(h\)\. We begin with some necessary technical preliminaries\.

### 8\.1Technical preliminaries

In this section, we gather several key technical results from\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\]that are essential for proving[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1)\. In particular, we establish the existence and uniqueness of a \(weak\) solution to the Parisi PDE, develop the verification argument which connects the Parisi PDE to the Hamilton\-Jacobi\-Bellman \(HJB\) equation, and compute the first\-order variation of the Parisi functional with respect to\(μ,c,r\)\(\\mu,c,r\)\.

##### Solution to the Parisi PDE\.

Theorem 3\.3 in\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\]implies the following:

###### Theorem 8\.1\.

Fixy∈ℝy\\in\\mathbb\{R\}, assume thath​\(y,⋅\)h\(y,\\cdot\)isC2C^\{2\}, Lipschitz and bounded from above\. Then, for any\(μ,c\)∈ℒ\(\\mu,c\)\\in\{\\mathscr\{L\}\}andγ∈ℒ\#\\gamma\\in\{\\mathscr\{L\}\_\{\\\#\}\}satisfyingμ=γ′/γ2\\mu=\\gamma^\{\\prime\}/\\gamma^\{2\}andc=1/γ​\(1\)c=1/\\gamma\(1\), the Parisi PDE \([53](https://arxiv.org/html/2606.28573#S6.E53)\) admits a unique weak solutionfy,μf\_\{y,\\mu\}such thatfy,μ​\(t,⋅\)∈C2​\(ℝ\)f\_\{y,\\mu\}\(t,\\cdot\)\\in C^\{2\}\(\\mathbb\{R\}\)for allt∈\[0,1\]t\\in\[0,1\]\. Further,

‖∂xfy,μ​\(t,⋅\)‖L∞​\(ℝ\)≤\\displaystyle\\left\\\|\{\\partial\_\{x\}f\_\{y,\\mu\}\(t,\\cdot\)\}\\right\\\|\_\{L^\{\\infty\}\(\\mathbb\{R\}\)\}\\leq\\,‖∂xh​\(y,⋅\)‖L∞​\(ℝ\),∀t∈\[0,1\],\\displaystyle\\left\\\|\{\\partial\_\{x\}h\(y,\\cdot\)\}\\right\\\|\_\{L^\{\\infty\}\(\\mathbb\{R\}\)\},\\,\\forall t\\in\[0,1\],∂x2fy,μ​\(t,x\)\>\\displaystyle\\partial\_\{x\}^\{2\}f\_\{y,\\mu\}\(t,x\)\>\\,−γ​\(t\),∀\(t,x\)∈\[0,1\]×ℝ\.\\displaystyle\-\\gamma\(t\),\\,\\forall\(t,x\)\\in\[0,1\]\\times\\mathbb\{R\}\.If, additionallysupz∈ℝ∂z2h​\(y,z\)<1/c\\sup\_\{z\\in\\mathbb\{R\}\}\\partial\_\{z\}^\{2\}h\(y,z\)<1/c, then we have

∂x2fy,μ​\(t,x\)≤C,∀\(t,x\)∈\[0,1\]×ℝ,\\partial\_\{x\}^\{2\}f\_\{y,\\mu\}\(t,x\)\\leq\\,C,\\,\\forall\(t,x\)\\in\[0,1\]\\times\\mathbb\{R\},where the constantCConly depends on\(μ,c\)\(\\mu,c\)andsupz∈ℝ∂z2h​\(y,z\)\\sup\_\{z\\in\\mathbb\{R\}\}\\partial\_\{z\}^\{2\}h\(y,z\)\.

According to[8\.1](https://arxiv.org/html/2606.28573#S8.Thmthm1), we know that the Parisi functional is well\-defined onℒ\{\\mathscr\{L\}\}\.

##### Verification argument\.

Define for any\(t,y,z\)∈\[0,1\]×ℝ×ℝ\(t,y,z\)\\in\[0,1\]\\times\\mathbb\{R\}\\times\\mathbb\{R\}, the following value function:

Vγ​\(t,y,z\)=supϕ∈D​\[t,1\]𝔼​\[h​\(y,z\+∫t1\(1\+ϕs\)​d​Bs\)−12​∫t1γ​\(s\)​\(ϕs2−1α\)​d​s\]\.V\_\{\\gamma\}\(t,y,z\)=\\,\\sup\_\{\\phi\\in D\[t,1\]\}\\mathbb\{E\}\\left\[h\\left\(y,z\+\\int\_\{t\}^\{1\}\\left\(1\+\\phi\_\{s\}\\right\)\\textup\{d\}B\_\{s\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{t\}^\{1\}\\gamma\(s\)\\left\(\\phi\_\{s\}^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}s\\right\]\.\(64\)Then, applying\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Proposition 5\.3\]for fixedy∈ℝy\\in\\mathbb\{R\}leads to the following dual relationship betweenVγV\_\{\\gamma\}andfy,μf\_\{y,\\mu\}:

###### Theorem 8\.2\.

Under the conditions of[8\.1](https://arxiv.org/html/2606.28573#S8.Thmthm1), we have for allt∈\[0,1\]t\\in\[0,1\]andy,x,z∈ℝy,x,z\\in\\mathbb\{R\}:

Vγ​\(t,y,z\)=infx∈ℝ\{fy,μ​\(t,x\)\+γ​\(t\)2​\(x−z\)2\}\+12​α​∫t1γ​\(s\)​d​s,fy,μ​\(t,x\)=supz∈ℝ\{Vγ​\(t,y,z\)−γ​\(t\)2​\(z−x\)2\}−12​α​∫t1γ​\(s\)​d​s\.\\begin\{split\}V\_\{\\gamma\}\(t,y,z\)=\\,&\\inf\_\{x\\in\\mathbb\{R\}\}\\left\\\{f\_\{y,\\mu\}\(t,x\)\+\\frac\{\\gamma\(t\)\}\{2\}\(x\-z\)^\{2\}\\right\\\}\+\\frac\{1\}\{2\\alpha\}\\int\_\{t\}^\{1\}\\gamma\(s\)\\textup\{d\}s,\\\\ f\_\{y,\\mu\}\(t,x\)=\\,&\\sup\_\{z\\in\\mathbb\{R\}\}\\left\\\{V\_\{\\gamma\}\(t,y,z\)\-\\frac\{\\gamma\(t\)\}\{2\}\(z\-x\)^\{2\}\\right\\\}\-\\frac\{1\}\{2\\alpha\}\\int\_\{t\}^\{1\}\\gamma\(s\)\\textup\{d\}s\.\\end\{split\}\(65\)

##### Analysis of the variational problem\.

Having established the existence and uniqueness of solutions to the Parisi PDE, and its dual relationship with the value function, we are now ready to compute the first\-order variations of the Parisi functional\. We summarize these results in the theorem below and defer its proof to[SectionF\.1](https://arxiv.org/html/2606.28573#A6.SS1)\.

###### Theorem 8\.3\.

Under the assumptions of[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1), letG∼𝖭​\(0,1\)G\\sim\{\\sf N\}\(0,1\),Y=φ​\(G,ε\)Y=\\varphi\(G,\\varepsilon\), and\(Xt\)t∈\[r2,1\]\(X\_\{t\}\)\_\{t\\in\[r^\{2\},1\]\}solve the SDE \(existence and uniqueness of the solution will be established in the proof\):

Xr2=r​G,d​Xt=μ​\(t\)​∂xfY,μ​\(t,Xt\)​d​t\+d​Bt,t∈\[r2,1\]X\_\{r^\{2\}\}=\\,rG,\\quad\\textup\{d\}X\_\{t\}=\\mu\(t\)\\partial\_\{x\}f\_\{Y,\\mu\}\(t,X\_\{t\}\)\\textup\{d\}t\+\\textup\{d\}B\_\{t\},\\,t\\in\[r^\{2\},1\]for\(μ,c\)∈ℒ​\(q\)\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)withq≥r2q\\geq r^\{2\}, where\(Bt\)t∈\[r2,1\]\(B\_\{t\}\)\_\{t\\in\[r^\{2\},1\]\}is a standard Brownian motion independent of\(Y,G\)\(Y,G\)\. Furthermore, fort∈\[r2,1\]t\\in\[r^\{2\},1\], define

Mt=1γ​\(t\)​∂xfY,μ​\(t,Xt\)\+Xt,M\_\{t\}=\\,\\frac\{1\}\{\\gamma\(t\)\}\\partial\_\{x\}f\_\{Y,\\mu\}\(t,X\_\{t\}\)\+X\_\{t\},then there exists\(ϕt\)t∈\[r2,1\]∈D​\[r2,1\]\(\\phi\_\{t\}\)\_\{t\\in\[r^\{2\},1\]\}\\in D\[r^\{2\},1\]satisfying

Mt=Mr2\+∫r2t\(1\+ϕs\)​d​Bs,∀t∈\[r2,1\]\.M\_\{t\}=\\,M\_\{r^\{2\}\}\+\\int\_\{r^\{2\}\}^\{t\}\(1\+\\phi\_\{s\}\)\\textup\{d\}B\_\{s\},\\,\\forall t\\in\[r^\{2\},1\]\.Finally, define for\(x,y\)∈ℝ2\(x,y\)\\in\\mathbb\{R\}^\{2\}:

F​\(x,y\)=αγ​\(q\)​∂xfy,μ​\(q,x\)=αγ​\(r2\)​∂xfy,μ​\(q,x\)\.F\(x,y\)=\\,\\frac\{\\alpha\}\{\\gamma\(q\)\}\\partial\_\{x\}f\_\{y,\\mu\}\(q,x\)=\\frac\{\\alpha\}\{\\gamma\(r^\{2\}\)\}\\partial\_\{x\}f\_\{y,\\mu\}\(q,x\)\.\(γ​\(q\)=γ​\(r2\)\\gamma\(q\)=\\gamma\(r^\{2\}\)sinceμ≡0\\mu\\equiv 0on\[r2,q\]\[r^\{2\},q\]\.\) The following hold:

- \(i\)\(Y,G,Xq\)=d\(Y,G,Zr,q\)\(Y,G,X\_\{q\}\)\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}\(Y,G,Z\_\{r,q\}\), and 𝖥\(μ,c,r\)=𝔼\[hc​\(Y,Xq\+1α​F​\(Xq,Y\)\+∫q1\(1\+ϕt\)​d​Bt\)−12​∫q1γ​\(t\)​\(ϕt2−1α\)​d​t−γ​\(q\)2​α\(F​\(Xq,Y\)2α−\(q−r2\)\)\],\\begin\{split\}\\mathsf\{F\}\(\\mu,c,r\)=\\,\\mathbb\{E\}\\Bigg\[&h\_\{c\}\\left\(Y,\\,X\_\{q\}\+\\frac\{1\}\{\\alpha\}F\\left\(X\_\{q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\\phi\_\{t\}^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t\\\\ &\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(X\_\{q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\Bigg\],\\end\{split\}where hc​\(y,x\)=conc⁡\(h​\(y,x\)−x22​c\)\+x22​c\.h\_\{c\}\(y,x\)=\\,\\operatorname\{conc\}\\left\(h\(y,x\)\-\\frac\{x^\{2\}\}\{2c\}\\right\)\+\\frac\{x^\{2\}\}\{2c\}\.
- \(ii\)∀0≤s<t≤1\\forall 0\\leq s<t\\leq 1, we have 𝔼​\[\(∂xfY,μ​\(t,Xt\)\)2\]−𝔼​\[\(∂xfY,μ​\(s,Xs\)\)2\]=∫stγ​\(u\)2​𝔼​\[ϕu2\]​d​u\.\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\(t,X\_\{t\}\)\\right\)^\{2\}\\right\]\-\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\(s,X\_\{s\}\)\\right\)^\{2\}\\right\]=\\,\\int\_\{s\}^\{t\}\\gamma\(u\)^\{2\}\\mathbb\{E\}\\left\[\\phi\_\{u\}^\{2\}\\right\]\\textup\{d\}u\.
- \(iii\)Define gc​\(y,x\)=∂hc​\(y,x\)∂c\.g\_\{c\}\(y,x\)=\\frac\{\\partial h\_\{c\}\(y,x\)\}\{\\partial c\}\.Assume thatδ:\[r2,1\]→ℝ\\delta:\[r^\{2\},1\]\\to\\mathbb\{R\}is inL1​\[r2,1\]L^\{1\}\[r^\{2\},1\]andL∞​\[r2,t\]L^\{\\infty\}\[r^\{2\},t\]for allt∈\[r2,1\)t\\in\[r^\{2\},1\)\. Then, dd​u​𝖥​\(μ\+u​δ,c,r\)\|u=0=12​∫r21δ​\(t\)​\(𝔼​\[\(∂xfY,μ​\(t,Xt\)\)2\]−1α​∫r2tγ​\(s\)2​d​s\)​d​t,dd​c​𝖥​\(μ,c,r\)=𝔼​\[gc​\(Y,M1\)\]\+12​\(𝔼​\[\(∂xfY,μ​\(1,X1\)\)2\]−1α​∫r21γ​\(t\)2​d​t\),dd​r​𝖥​\(μ,c,r\)=γ​\(r2\)α​\(1q−r2​𝔼​\[\(q​G−r​Zr,q\)​F​\(Zr,q,φ​\(G,ε\)\)\]−r\)\.\\begin\{split\}\\frac\{\\textup\{d\}\}\{\\textup\{d\}u\}\\mathsf\{F\}\\left\(\\mu\+u\\delta,c,r\\right\)\\bigg\|\_\{u=0\}=\\,&\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\delta\(t\)\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\(t,X\_\{t\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha\}\\int\_\{r^\{2\}\}^\{t\}\\gamma\(s\)^\{2\}\\textup\{d\}s\\right\)\\textup\{d\}t,\\\\ \\frac\{\\textup\{d\}\}\{\\textup\{d\}c\}\\mathsf\{F\}\\left\(\\mu,c,r\\right\)=\\,&\\mathbb\{E\}\\left\[g\_\{c\}\\left\(Y,M\_\{1\}\\right\)\\right\]\+\\frac\{1\}\{2\}\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)^\{2\}\\textup\{d\}t\\right\),\\\\ \\frac\{\\textup\{d\}\}\{\\textup\{d\}r\}\\mathsf\{F\}\\left\(\\mu,c,r\\right\)=\\,&\\frac\{\\gamma\(r^\{2\}\)\}\{\\alpha\}\\left\(\\frac\{1\}\{q\-r^\{2\}\}\\mathbb\{E\}\\left\[\\left\(qG\-rZ\_\{r,q\}\\right\)F\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]\-r\\right\)\.\\end\{split\}

### 8\.2Proof of[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1)

##### Proof of\(a\)\(a\): Variational formula\.

Sinceμ=0\\mu=0on\[r2,q\]\[r^\{2\},q\], we know that the Parisi PDE degenerates to a standard heat equation on this interval:

∂tfy,μ​\(t,x\)\+12​∂x2fy,μ​\(t,x\)=0,\(t,x\)∈\[r2,q\]×ℝ\.\\partial\_\{t\}f\_\{y,\\mu\}\(t,x\)\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,\\mu\}\(t,x\)=\\,0,\\,\(t,x\)\\in\[r^\{2\},q\]\\times\\mathbb\{R\}\.Therefore,

𝔼Y,G​\[fY,μ​\(r2,r​G\)\]=𝔼Y,G,Z​\[fY,μ​\(q,r​G\+q−r2​Z\)\],\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}\\left\(r^\{2\},rG\\right\)\\right\]=\\,\\mathbb\{E\}\_\{Y,G,Z\}\\left\[f\_\{Y,\\mu\}\\left\(q,rG\+\\sqrt\{q\-r^\{2\}\}Z\\right\)\\right\],whereZ∼𝖭​\(0,1\)Z\\sim\{\\sf N\}\(0,1\)is independent of\(Y,G\)\(Y,G\)\. Using the notation of[5](https://arxiv.org/html/2606.28573#Thmdefn5), we deduce that

𝖥​\(μ,c,r\)=\\displaystyle\\mathsf\{F\}\(\\mu,c,r\)=\\,𝔼Y,G​\[fY,μ​\(r2,r​G\)\]\+12​α​∫r21d​tc\+∫t1μ​\(u\)​d​u\\displaystyle\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}\\left\(r^\{2\},rG\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(u\)\\textup\{d\}u\}=\\displaystyle=\\,𝔼Y,G,Z​\[fY,μ​\(q,r​G\+q−r2​Z\)\]\+12​α​∫r21γ​\(t\)​d​t\\displaystyle\\mathbb\{E\}\_\{Y,G,Z\}\\left\[f\_\{Y,\\mu\}\\left\(q,rG\+\\sqrt\{q\-r^\{2\}\}Z\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t=\\displaystyle=\\,𝔼​\[fY,μ​\(q,Zr,q\)\]\+12​α​∫r21γ​\(t\)​d​t\.\\displaystyle\\mathbb\{E\}\\left\[f\_\{Y,\\mu\}\\left\(q,Z\_\{r,q\}\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t\.Applying[8\.2](https://arxiv.org/html/2606.28573#S8.Thmthm2)yields that

fy,μ​\(q,x\)=supz∈ℝ\{Vγ​\(q,y,z\)−γ​\(q\)2​\(z−x\)2\}−12​α​∫q1γ​\(s\)​d​s,f\_\{y,\\mu\}\(q,x\)=\\,\\sup\_\{z\\in\\mathbb\{R\}\}\\left\\\{V\_\{\\gamma\}\(q,y,z\)\-\\frac\{\\gamma\(q\)\}\{2\}\(z\-x\)^\{2\}\\right\\\}\-\\frac\{1\}\{2\\alpha\}\\int\_\{q\}^\{1\}\\gamma\(s\)\\textup\{d\}s,which leads to

𝖥​\(μ,c,r\)=\\displaystyle\\mathsf\{F\}\(\\mu,c,r\)=\\,𝔼​\[fY,μ​\(q,Zr,q\)\]\+12​α​∫r21γ​\(t\)​d​t\\displaystyle\\mathbb\{E\}\\left\[f\_\{Y,\\mu\}\\left\(q,Z\_\{r,q\}\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t=\\displaystyle=\\,𝔼​\[supz∈ℝ\{Vγ​\(q,Y,z\)−γ​\(q\)2​\(z−Zr,q\)2\}\]\+12​α​∫r2qγ​\(t\)​d​t\\displaystyle\\mathbb\{E\}\\left\[\\sup\_\{z\\in\\mathbb\{R\}\}\\left\\\{V\_\{\\gamma\}\(q,Y,z\)\-\\frac\{\\gamma\(q\)\}\{2\}\\left\(z\-Z\_\{r,q\}\\right\)^\{2\}\\right\\\}\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{q\}\\gamma\(t\)\\textup\{d\}t=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,𝔼​\[supu∈ℝ\{Vγ​\(q,Y,Zr,q\+u\)−γ​\(q\)2​\(u2−q−r2α\)\}\]\\displaystyle\\mathbb\{E\}\\left\[\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{V\_\{\\gamma\}\\left\(q,Y,Z\_\{r,q\}\+u\\right\)\-\\frac\{\\gamma\(q\)\}\{2\}\\left\(u^\{2\}\-\\frac\{q\-r^\{2\}\}\{\\alpha\}\\right\)\\right\\\}\\right\]=\\displaystyle=\\,supF:ℝ2→ℝ𝔼​\[Vγ​\(q,Y,Zr,q\+1α​F​\(Zr,q,Y\)\)−γ​\(q\)2​α​\(F​\(Zr,q,Y\)2α−\(q−r2\)\)\],\\displaystyle\\sup\_\{F:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}\}\\mathbb\{E\}\\left\[V\_\{\\gamma\}\\left\(q,Y,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\(Z\_\{r,q\},Y\)\\right\)\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(Z\_\{r,q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\right\],where\(i\)\(i\)is because ofγ​\(t\)=γ​\(q\)\\gamma\(t\)=\\gamma\(q\)fort∈\[r2,q\]t\\in\[r^\{2\},q\]\. For any fixedFF, we know that \(analogous to\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Lemma 5\.2\]\)

𝔼​\[Vγ​\(q,Y,Zr,q\+1α​F​\(Zr,q,Y\)\)\]\\displaystyle\\mathbb\{E\}\\left\[V\_\{\\gamma\}\\left\(q,Y,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\(Z\_\{r,q\},Y\)\\right\)\\right\]=\\displaystyle=\\,supϕ∈D​\[q,1\]𝔼​\[h​\(Y,Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+ϕt\)​d​Bt\)−12​∫q1γ​\(t\)​\(ϕt2−1α\)​d​t\]\.\\displaystyle\\sup\_\{\\phi\\in D\[q,1\]\}\\mathbb\{E\}\\left\[h\\left\(Y,\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\\phi\_\{t\}^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t\\right\]\.We thus obtain that

𝖥\(μ,c,r\)=supF:ℝ2→ℝϕ∈D​\[q,1\]𝔼\[\\displaystyle\\mathsf\{F\}\(\\mu,c,r\)=\\,\\sup\_\{\\begin\{subarray\}\{c\}F:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}\\\\ \\phi\\in D\[q,1\]\\end\{subarray\}\}\\mathbb\{E\}\\Bigg\[h​\(Y,Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+ϕt\)​d​Bt\)−12​∫q1γ​\(t\)​\(ϕt2−1α\)​d​t\\displaystyle h\\left\(Y,\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\\phi\_\{t\}^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t−γ​\(q\)2​α\(F​\(Zr,q,Y\)2α−\(q−r2\)\)\],\\displaystyle\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(Z\_\{r,q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\Bigg\],completing the proof of part\(a\)\(a\)\.

##### Proof of\(b\)\(b\): Weak duality\.

Define

𝒱¯1,α,φAMP​\(r,q,h\):=sup𝔼​\[h​\(Y,Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+ϕt\)​d​Bt\)\],s\.t\.q≥r2\+1α​𝔼​\[F​\(Zr,q,φ​\(G,ε\)\)2\],and​supt∈\[0,1\]𝔼​\[ϕt2\]≤1α\.\\begin\{split\}&\\overline\{\\mathscr\{V\}\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\):=\\,\\sup\\,\\mathbb\{E\}\\left\[h\\left\(Y,\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\],\\\\ &\\mbox\{s\.t\.\}\\quad q\\geq\\,r^\{2\}\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\],\\,\\text\{and\}\\sup\_\{t\\in\[0,1\]\}\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\\leq\\frac\{1\}\{\\alpha\}\.\\end\{split\}\(66\)By definition, we know that𝒱¯1,α,φAMP​\(r,q,h\)≥𝒱1,α,φAMP​\(r,q,h\)\\overline\{\\mathscr\{V\}\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\)\\geq\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\)\. Hence,

𝒱1,α,φAMP​\(r,q,h\)\\displaystyle\\mathscr\{V\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q,h\)≤\\displaystyle\\leq\\,supF:ℝ2→ℝϕ∈D​\[q,1\]infγ∈ℒ\#​\(q\)𝔼\[h\(Y,Zr,q\+1αF\(Zr,q,Y\)\+∫q1\(1\+ϕt\)dBt\)−12∫q1γ\(t\)\(ϕt2−1α\)dt\\displaystyle\\sup\_\{\\begin\{subarray\}\{c\}F:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}\\\\ \\phi\\in D\[q,1\]\\end\{subarray\}\}\\inf\_\{\\gamma\\in\{\\mathscr\{L\}\_\{\\\#\}\}\(q\)\}\\mathbb\{E\}\\Bigg\[h\\left\(Y,\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\\phi\_\{t\}^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t−γ​\(q\)2​α\(F​\(Zr,q,Y\)2α−\(q−r2\)\)\]\\displaystyle\\hskip 80\.00012pt\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(Z\_\{r,q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\Bigg\]≤\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{\\leq\}\}\\,infγ∈ℒ\#​\(q\)supF:ℝ2→ℝϕ∈D​\[q,1\]𝔼\[h\(Y,Zr,q\+1αF\(Zr,q,Y\)\+∫q1\(1\+ϕt\)dBt\)−12∫q1γ\(t\)\(ϕt2−1α\)dt\\displaystyle\\inf\_\{\\gamma\\in\{\\mathscr\{L\}\_\{\\\#\}\}\(q\)\}\\sup\_\{\\begin\{subarray\}\{c\}F:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}\\\\ \\phi\\in D\[q,1\]\\end\{subarray\}\}\\mathbb\{E\}\\Bigg\[h\\left\(Y,\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\\phi\_\{t\}^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t−γ​\(q\)2​α\(F​\(Zr,q,Y\)2α−\(q−r2\)\)\]\\displaystyle\\hskip 80\.00012pt\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(Z\_\{r,q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\Bigg\]=\(i​i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(ii\)\}\}\{\{=\}\}\\,inf\(μ,c\)∈ℒ​\(q\)𝖥​\(μ,c,r\),\\displaystyle\\inf\_\{\(\\mu,c\)\\in\{\\mathscr\{L\}\}\(q\)\}\\mathsf\{F\}\(\\mu,c,r\),where\(i\)\(i\)follows from the max\-min inequality,\(i​i\)\(ii\)follows from the variational formula of part\(a\)\(a\)\. This concludes the proof of part\(b\)\(b\)\.

##### Proof of\(c\)\(c\): Strong duality\.

By[8\.3](https://arxiv.org/html/2606.28573#S8.Thmthm3)\(iii\) and the envelope theorem, we know that\(μ∗,c∗,r∗\)\(\\mu\_\{\*\},c\_\{\*\},r\_\{\*\}\)verifies the first\-order conditions:

𝔼​\[\(∂xfY,μ∗​\(t,Xt∗\)\)2\]−1α​∫r∗2tγ∗​\(s\)2​d​s=0,∀t∈\[r∗2,1\],\\displaystyle\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\_\{\*\}\}\(t,X\_\{t\}^\{\*\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha\}\\int\_\{r\_\{\*\}^\{2\}\}^\{t\}\\gamma\_\{\*\}\(s\)^\{2\}\\textup\{d\}s=\\,0,\\quad\\forall t\\in\[r\_\{\*\}^\{2\},1\],\(67\)𝔼​\[gc∗​\(Y,M1∗\)\]\+12​\(𝔼​\[\(∂xfY,μ∗​\(1,X1∗\)\)2\]−1α​∫r∗21γ∗​\(t\)2​d​t\)=0,\\displaystyle\\mathbb\{E\}\\left\[g\_\{c\_\{\*\}\}\\left\(Y,M\_\{1\}^\{\*\}\\right\)\\right\]\+\\frac\{1\}\{2\}\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\_\{\*\}\}\(1,X\_\{1\}^\{\*\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha\}\\int\_\{r\_\{\*\}^\{2\}\}^\{1\}\\gamma\_\{\*\}\(t\)^\{2\}\\textup\{d\}t\\right\)=\\,0,\(68\)1q−r∗2​𝔼​\[\(q​G−r∗​Zr∗,q\)​F​\(Zr∗,q,φ​\(G,ε\)\)\]=r∗\.\\displaystyle\\frac\{1\}\{q\-r\_\{\*\}^\{2\}\}\\mathbb\{E\}\\left\[\\left\(qG\-r\_\{\*\}Z\_\{r\_\{\*\},q\}\\right\)F\\left\(Z\_\{r\_\{\*\},q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]=r\_\{\*\}\.\(69\)Using[8\.3](https://arxiv.org/html/2606.28573#S8.Thmthm3)\(ii\), we immediately obtain that𝔼​\[\(ϕt∗\)2\]=1/α\\mathbb\{E\}\[\(\\phi\_\{t\}^\{\*\}\)^\{2\}\]=1/\\alpha, thus establishing the feasibility ofϕ∗\\phi^\{\*\}\. Since we already have[Eq\.69](https://arxiv.org/html/2606.28573#S8.E69), to show thatF∗F^\{\*\}is feasible, it suffices to prove

q=r∗2\+1α​𝔼​\[F​\(Zr∗,q,φ​\(G,ε\)\)2\],1α​𝔼​\[∂F∂Zr∗,q​\(Zr∗,q,φ​\(G,ε\)\)2\]≤1\.q=\\,r\_\{\*\}^\{2\}\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(Z\_\{r\_\{\*\},q\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\],\\quad\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{r\_\{\*\},q\}\}\\left\(Z\_\{r\_\{\*\},q\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\]\\leq\\,1\.The first equation in the above display follows by takingt=qt=qin[Eq\.67](https://arxiv.org/html/2606.28573#S8.E67), while the second one can be proved using a similar argument as in\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Section 8\.2\]\. Having established the feasibility of\(F∗,ϕ∗\)\(F^\{\*\},\\phi^\{\*\}\),[8\.3](https://arxiv.org/html/2606.28573#S8.Thmthm3)\(i\) then implies that

𝖥​\(μ∗,c∗,r∗\)=𝔼​\[hc∗​\(Y,Zr∗,q\+1α​F​\(Zr∗,q,Y\)\+∫q1\(1\+ϕt∗\)​d​Bt\)\]=𝔼​\[hc∗​\(Y,M1∗\)\]\.\\mathsf\{F\}\(\\mu\_\{\*\},c\_\{\*\},r\_\{\*\}\)=\\,\\mathbb\{E\}\\left\[h\_\{c\_\{\*\}\}\\left\(Y,\\,Z\_\{r\_\{\*\},q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r\_\{\*\},q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}^\{\*\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\]=\\mathbb\{E\}\\left\[h\_\{c\_\{\*\}\}\\left\(Y,\\,M\_\{1\}^\{\*\}\\right\)\\right\]\.Note that[Eqs\.67](https://arxiv.org/html/2606.28573#S8.E67)and[68](https://arxiv.org/html/2606.28573#S8.E68)together imply that𝔼​\[gc∗​\(Y,M1∗\)\]=0\\mathbb\{E\}\[g\_\{c\_\{\*\}\}\(Y,M\_\{1\}^\{\*\}\)\]=0\. Applying Proposition C\.4 in\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\], we get that

𝔼​\[h​\(Y,M1∗\)\]=𝔼​\[hc∗​\(Y,M1∗\)\]=𝖥​\(μ∗,c∗,r∗\)\.\\mathbb\{E\}\\left\[h\(Y,M\_\{1\}^\{\*\}\)\\right\]=\\,\\mathbb\{E\}\\left\[h\_\{c\_\{\*\}\}\(Y,M\_\{1\}^\{\*\}\)\\right\]=\\mathsf\{F\}\(\\mu\_\{\*\},c\_\{\*\},r\_\{\*\}\)\.This completes the proof of part \(c\)\.

### Acknowledgments

AM was supported by the NSF through Award DMS\-2031883, the Simons Foundation through Award 814639 for the Collaboration on the Theoretical Foundations of Deep Learning, and the NSF Award MFAI\-2501597\. KZ was supported by the Founder’s Fellowship in Statistics at Columbia University\.

## References

- \[AMK\+18\]Benjamin Aubin, Antoine Maillard, Florent Krzakala, Nicolas Macris, and Lenka Zdeborová,*The committee machine: Computational to statistical gaps in learning a two\-layers neural network*, Advances in Neural Information Processing Systems31\(2018\)\.
- \[AMMN19\]Gerard Ben Arous, Song Mei, Andrea Montanari, and Mihai Nica,*The landscape of the spiked tensor model*, Communications on Pure and Applied Mathematics72\(2019\), no\. 11, 2282–2330\.
- \[AMS23\]Antonio Auffinger, Andrea Montanari, and Eliran Subag,*Optimization of random high\-dimensional functions: Structure and algorithms*, Spin Glass Theory and Far Beyond: Replica Symmetry Breaking After 40 Years, World Scientific, 2023, pp\. 609–633\.
- \[AMS26\]Kiana Asgari, Andrea Montanari, and Basil Saeed,*Local minima of the empirical risk in high dimension: General theorems and convex examples*, Annals of Statistics \(2026\)\.
- \[BKM\+19\]Jean Barbier, Florent Krzakala, Nicolas Macris, Léo Miolane, and Lenka Zdeborová,*Optimal errors and phase transitions in high\-dimensional generalized linear models*, Proceedings of the National Academy of Sciences116\(2019\), no\. 12, 5451–5460\.
- \[BM11a\]Mohsen Bayati and Andrea Montanari,*The dynamics of message passing on dense graphs, with applications to compressed sensing*, IEEE Transactions on Information Theory57\(2011\), no\. 2, 764–785\.
- \[BM11b\],*The lasso risk for gaussian matrices*, IEEE Transactions on Information Theory58\(2011\), no\. 4, 1997–2017\.
- \[CC17\]Yuxin Chen and Emmanuel J Candès,*Solving random quadratic systems of equations is nearly as easy as solving linear systems*, Communications on pure and applied mathematics70\(2017\), no\. 5, 822–883\.
- \[Che05\]Alexander S Cherny,*Singular stochastic differential equations*, Springer Science & Business Media, 2005\.
- \[CMW20\]Michael Celentano, Andrea Montanari, and Yuchen Wu,*The estimation error of general first order methods*, Conference on Learning Theory, PMLR, 2020, pp\. 1078–1141\.
- \[CS17\]Wei\-Kuo Chen and Arnab Sen,*Parisi formula, disorder chaos and fluctuation for the ground state energy in the spherical mixed p\-spin models*, Communications in Mathematical Physics350\(2017\), no\. 1, 129–173\.
- \[DF84\]Persi Diaconis and David Freedman,*Asymptotics of graphical projection pursuit*, The Annals of Statistics \(1984\), 793–815\.
- \[DM16\]David Donoho and Andrea Montanari,*High dimensional robust M\-estimation: Asymptotic variance via approximate message passing*, Probability Theory and Related Fields166\(2016\), 935–969\.
- \[EAMS21\]Ahmed El Alaoui, Andrea Montanari, and Mark Sellke,*Optimization of mean\-field spin glasses*, The Annals of Probability49\(2021\), no\. 6, 2922–2960\.
- \[EK18\]Noureddine El Karoui,*On the impact of predictor geometry on the performance on high\-dimensional ridge\-regularized generalized robust regression estimators*, Probability Theory and Related Fields170\(2018\), no\. 1, 95–175\.
- \[FT74\]Jerome H Friedman and John W Tukey,*A projection pursuit algorithm for exploratory data analysis*, IEEE Transactions on computers100\(1974\), no\. 9, 881–890\.
- \[GJ21\]David Gamarnik and Aukosh Jagannath,*The overlap gap property and approximate message passing algorithms for p\-spin models*, The Annals of Probability49\(2021\), no\. 1, 180–205\.
- \[GR00\]G Györgyi and P Reimann,*Beyond storage capacity in a single model neuron: Continuous replica symmetry breaking*, Journal of Statistical Physics101\(2000\), 679–702\.
- \[GS14\]David Gamarnik and Madhu Sudan,*Limits of local algorithms over sparse random graphs*, Proceedings of the 5th conference on Innovations in theoretical computer science, 2014, pp\. 369–376\.
- \[HBM\+22\]Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, and Aidan Clark,*Training compute\-optimal large language models*, arXiv:2203\.15556 \(2022\)\.
- \[HS22\]Brice Huang and Mark Sellke,*Tight Lipschitz hardness for optimizing mean field spin glasses*, 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science \(FOCS\), IEEE, 2022, pp\. 312–322\.
- \[HS24\],*Optimization algorithms for multi\-species spherical spin glasses*, Journal of Statistical Physics191\(2024\), no\. 2, 1–42\.
- \[JM13\]Adel Javanmard and Andrea Montanari,*State evolution for general approximate message passing algorithms, with applications to spatial coupling*, Information and Inference: A Journal of the IMA2\(2013\), no\. 2, 115–144\.
- \[JT17\]Aukosh Jagannath and Ian Tobasco,*Low temperature asymptotics of spherical mean field spin glasses*, Communications in Mathematical Physics352\(2017\), no\. 3, 979–1017\.
- \[KMH\+20\]Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei,*Scaling laws for neural language models*, arXiv preprint arXiv:2001\.08361 \(2020\)\.
- \[MBB26\]Antoine Maillard, Tony Bonnaire, and Giulio Biroli,*Topological exploration of high\-dimensional empirical risk landscapes: general approach, and applications to phase retrieval*, arXiv preprint arXiv:2602\.17779 \(2026\)\.
- \[MM18\]Marco Mondelli and Andrea Montanari,*Fundamental limits of weak recovery with applications to phase retrieval*, Conference On Learning Theory, PMLR, 2018, pp\. 1445–1450\.
- \[Mon19\]A\. Montanari,*Optimization of the sherrington\-kirkpatrick hamiltonian*, 2019 IEEE 60th Annual Symposium on Foundations of Computer Science \(FOCS\) \(Los Alamitos, CA, USA\), IEEE Computer Society, nov 2019, pp\. 1417–1433\.
- \[Mon24\]Andrea Montanari,*Optimization of random cost functions and statistical physics*, Proceedings of the 2023 Solvay Conference on Physics, 2024\.
- \[MR14\]Andrea Montanari and Emile Richard,*A statistical model for tensor pca*, Advances in neural information processing systems27\(2014\)\.
- \[MS24\]Andrea Montanari and Subhabrata Sen,*A friendly tutorial on mean\-field spin glass techniques for non\-physicists*, Foundations and Trends in Machine Learning17\(2024\), no\. 1, 1–173\.
- \[MS26\]Andrea Montanari and Basil Saeed,*Topological trivialization in non\-convex empirical risk minimization*, arXiv preprint arXiv:2602\.14969 \(2026\)\.
- \[MW22\]Andrea Montanari and Yuchen Wu,*Statistically optimal first order algorithms: A proof via orthogonalization*, arXiv preprint arXiv:2201\.05101 \(2022\)\.
- \[MZ22\]Andrea Montanari and Kangjie Zhou,*Overparametrized linear dimensionality reductions: From projection pursuit to two\-layer neural networks*, arXiv preprint arXiv:2206\.06526 \(2022\)\.
- \[MZ24\],*Which exceptional low\-dimensional projections of a gaussian point cloud can be found in polynomial time?*, arXiv preprint arXiv:2406\.02970 \(2024\)\.
- \[Nua06\]David Nualart,*The malliavin calculus and related topics*, Springer, 2006\.
- \[SSBD14\]Shai Shalev\-Shwartz and Shai Ben\-David,*Understanding machine learning: From theory to algorithms*, Cambridge University Press, 2014\.
- \[Sub21\]Eliran Subag,*Following the ground states of full\-rsb spherical spin glasses*, Communications on Pure and Applied Mathematics74\(2021\), no\. 5, 1021–1044\.
- \[Tal10\]Michel Talagrand,*Mean field models for spin glasses: Volume i: Basic examples*, vol\. 54, Springer Science & Business Media, 2010\.
- \[TOH15\]Christos Thrampoulidis, Samet Oymak, and Babak Hassibi,*Regularized linear regression: A precise analysis of the estimation error*, Conference on Learning Theory, PMLR, 2015, pp\. 1683–1709\.
- \[VDR\+25\]Matteo Vilucchio, Yatin Dandi, Matéo Pirio Rossignol, Cedric Gerbelot, and Florent Krzakala,*Asymptotics of non\-convex generalized linear models in high\-dimensions: A proof of the replica formula*, arXiv preprint arXiv:2502\.20003 \(2025\)\.

## Appendix AThe replica calculation

In this section, we carry out calculations using the non\-rigorous replica method from statistical physics, to support Conjectures[2\.1](https://arxiv.org/html/2606.28573#S2.Thmconj1)and[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)\. We will focus on the casem=1m=1\([2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)\), since the case of generalmmis almost identical, but less transparent\. We refer to\[[MS24](https://arxiv.org/html/2606.28573#bib.bibx31)\]for a friendly introduction to these techniques\. The derivation presented here is quite straightforward \(from the perspective of statistical physics\) and generalizes the replica calculation in\[[GR00](https://arxiv.org/html/2606.28573#bib.bibx18)\]\.

For the casem=1m=1, we define the Hamiltonian

Hn,d​\(𝒘\)=1n​∑i=1nh​\(yi,⟨𝒙i,𝒘⟩\),𝒘∈𝕊d−1\.H\_\{n,d\}\(\\text\{$w$\}\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\langle\\text\{$x$\}\_\{i\},\\text\{$w$\}\\rangle\\right\)\\,,\\,\\,\\,\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\\,\.Recall thatyi=φ​\(𝑾∗𝖳​𝒙i,εi\)y\_\{i\}=\\varphi\(\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\text\{$x$\}\_\{i\},\\varepsilon\_\{i\}\)where𝑾∗∈O​\(d,k\)\\text\{$W$\}\_\{\*\}\\in O\(d,k\)\. By rotational invariance, we may assume without loss of generality that𝑾∗=\[Ik,𝟎k×\(d−k\)\]𝖳\\text\{$W$\}\_\{\*\}=\[I\_\{k\},\\mathrm\{\\bf 0\}\_\{k\\times\(d\-k\)\}\]^\{\\sf T\}, and𝒙i=\(gi,𝒛i𝖳\)𝖳\\text\{$x$\}\_\{i\}=\(g\_\{i\},\\text\{$z$\}\_\{i\}^\{\\sf T\}\)^\{\\sf T\}wheregi∼𝖭​\(0,Ik\)g\_\{i\}\\sim\{\\sf N\}\(0,I\_\{k\}\), and𝒛i∼𝖭​\(𝟎,𝑰d−k\)\\text\{$z$\}\_\{i\}\\sim\{\\sf N\}\(\\mathrm\{\\bf 0\},\{\\boldsymbol\{I\}\}\_\{d\-k\}\)\. Denoting𝒘=\(w1,𝒘2𝖳\)𝖳\\text\{$w$\}=\(w\_\{1\},\\text\{$w$\}\_\{2\}^\{\\sf T\}\)^\{\\sf T\}wherew1∈ℝkw\_\{1\}\\in\\mathbb\{R\}^\{k\}and𝒘2∈ℝd−k\\text\{$w$\}\_\{2\}\\in\\mathbb\{R\}^\{d\-k\}, one can write

yi=φ​\(gi,εi\),⟨𝒙i,𝒘⟩=⟨w1,gi⟩\+⟨𝒘2,𝒛i⟩\.y\_\{i\}=\\varphi\(g\_\{i\},\\varepsilon\_\{i\}\),\\quad\\langle\\text\{$x$\}\_\{i\},\\text\{$w$\}\\rangle=\\langle w\_\{1\},g\_\{i\}\\rangle\+\\langle\\text\{$w$\}\_\{2\},\\text\{$z$\}\_\{i\}\\rangle\.Under this reduction, we have

Hn,d​\(𝒘\)=1n​∑i=1nh​\(yi,⟨𝒙i,𝒘⟩\)=1n​∑i=1nh​\(yi,⟨w1,gi⟩\+⟨𝒘2,𝒛i⟩\),yi=φ​\(gi,εi\)\.H\_\{n,d\}\(\\text\{$w$\}\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\langle\\text\{$x$\}\_\{i\},\\text\{$w$\}\\rangle\\right\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\langle w\_\{1\},g\_\{i\}\\rangle\+\\langle\\text\{$w$\}\_\{2\},\\text\{$z$\}\_\{i\}\\rangle\\right\),\\quad y\_\{i\}=\\varphi\(g\_\{i\},\\varepsilon\_\{i\}\)\.Since‖w1‖22\+‖𝒘2‖22=‖𝒘‖22=1\\left\\\|\{w\_\{1\}\}\\right\\\|\_\{2\}^\{2\}\+\\left\\\|\{\\text\{$w$\}\_\{2\}\}\\right\\\|\_\{2\}^\{2\}=\\left\\\|\{\\text\{$w$\}\}\\right\\\|\_\{2\}^\{2\}=1, it follows that

max𝒘∈𝕊d−1⁡Hn,d​\(𝒘\)=\\displaystyle\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}H\_\{n,d\}\(\\text\{$w$\}\)=\\,maxw1∈Bk​\(1\)⁡max‖𝒘2‖2=1−‖w1‖22⁡1n​∑i=1nh​\(yi,⟨w1,gi⟩\+⟨𝒘2,𝒛i⟩\)\\displaystyle\\max\_\{w\_\{1\}\\in B\_\{k\}\(1\)\}\\max\_\{\\left\\\|\{\\text\{$w$\}\_\{2\}\}\\right\\\|\_\{2\}=\\sqrt\{1\-\\left\\\|\{w\_\{1\}\}\\right\\\|\_\{2\}^\{2\}\}\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\langle w\_\{1\},g\_\{i\}\\rangle\+\\langle\\text\{$w$\}\_\{2\},\\text\{$z$\}\_\{i\}\\rangle\\right\)=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,maxr∈Bk​\(1\)⁡max𝒘∈𝕊d−k−1⁡1n​∑i=1nh​\(yi,⟨r,gi⟩\+1−‖r‖22​⟨𝒘,𝒛i⟩\),\\displaystyle\\max\_\{r\\in B\_\{k\}\(1\)\}\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-k\-1\}\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\langle r,g\_\{i\}\\rangle\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\langle\\text\{$w$\},\\text\{$z$\}\_\{i\}\\rangle\\right\),where in\(i\)\(i\)we recastw1w\_\{1\}asrrand𝒘2\\text\{$w$\}\_\{2\}as1−‖r‖22​𝒘\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\text\{$w$\}\. Sincelimn→∞n/\(d−k\)=α\\lim\_\{n\\to\\infty\}n/\(d\-k\)=\\alpha, it then suffices to compute the asymptotics of \(note that we also recastd−kd\-kasdd\)

max𝒘∈𝕊d−1⁡Hn,d​\(𝒘,r\):=max𝒘∈𝕊d−1⁡1n​∑i=1nh​\(yi,⟨r,gi⟩\+1−‖r‖22​⟨𝒘,𝒛i⟩\)\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}H\_\{n,d\}\(\\text\{$w$\},r\):=\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},\\langle r,g\_\{i\}\\rangle\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\langle\\text\{$w$\},\\text\{$z$\}\_\{i\}\\rangle\\right\)for each fixedr∈Bk​\(1\)r\\in B\_\{k\}\(1\)\.

To this end, we define the partition function forβ\>0\\beta\>0:

Zβ=∫𝕊d−1en​β​Hn,d​\(𝒘,r\)​ν0​\(d𝒘\),Z\_\{\\beta\}=\\int\_\{\\mathbb\{S\}^\{d\-1\}\}e^\{n\\beta H\_\{n,d\}\(\\text\{$w$\},r\)\}\\,\\nu\_\{0\}\(\\textup\{d\}\\text\{$w$\}\),whereν0\\nu\_\{0\}is the uniform measure on𝕊d−1\\mathbb\{S\}^\{d\-1\}\. Assume that

limn,d→∞n/d→αmax𝒘∈𝕊d−1⁡Hn,d​\(𝒘,r\)\\lim\_\{\\begin\{subarray\}\{c\}n,d\\to\\infty\\\\ n/d\\to\\alpha\\end\{subarray\}\}\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}H\_\{n,d\}\(\\text\{$w$\},r\)exists almost surely, and concentrates around its expectation \(this is true ifhhis Lipschitz\), then we would like to compute

limn→∞𝔼​\[max𝒘∈𝕊d−1⁡Hn,d​\(𝒘,r\)\]=limn→∞𝔼​\[limβ→∞1n​β​log⁡Zβ\]\.\\displaystyle\\lim\_\{n\\to\\infty\}\\mathbb\{E\}\\left\[\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}H\_\{n,d\}\(\\text\{$w$\},r\)\\right\]=\\lim\_\{n\\to\\infty\}\\mathbb\{E\}\\left\[\\lim\_\{\\beta\\to\\infty\}\\frac\{1\}\{n\\beta\}\\log Z\_\{\\beta\}\\right\]\.Here, the limitn→∞n\\to\\inftyshould be understood asn,d→∞n,d\\to\\inftysimultaneously withn/d→αn/d\\to\\alpha\. Within the scope of replica method, we will interchange expectations and limits arbitrarily\. It then suffices to compute the quantity

limβ→∞limn→∞1n​β​𝔼​\[log⁡Zβ\]=limβ→∞limn→∞limk→0\+1n​β​k​log⁡𝔼​\[Zβk\],\\lim\_\{\\beta\\to\\infty\}\\lim\_\{n\\to\\infty\}\\frac\{1\}\{n\\beta\}\\mathbb\{E\}\\left\[\\log Z\_\{\\beta\}\\right\]=\\lim\_\{\\beta\\to\\infty\}\\lim\_\{n\\to\\infty\}\\lim\_\{k\\to 0^\{\+\}\}\\frac\{1\}\{n\\beta k\}\\log\\mathbb\{E\}\\left\[Z\_\{\\beta\}^\{k\}\\right\],where we use the identity

𝔼​\[log⁡Z\]=limk→0\+1k​log⁡𝔼​\[Zk\]\\mathbb\{E\}\\left\[\\log Z\\right\]=\\lim\_\{k\\to 0^\{\+\}\}\\frac\{1\}\{k\}\\log\\mathbb\{E\}\\left\[Z^\{k\}\\right\]for any non\-negative random variableZZ\.

While the above interchange of limits is not justified in the present derivation, it is not the most problematic step in the replica calculation\. Indeed, the critical step is to first considerkkas an integer, and then extrapolate to non\-integer values ofkk\. Fork∈ℕk\\in\\mathbb\{N\}, we have

𝔼​\[Zβk\]=\\displaystyle\\mathbb\{E\}\\left\[Z\_\{\\beta\}^\{k\}\\right\]=\\,𝔼​\[∫\(𝕊d−1\)kexp⁡\(β⋅∑j=1k∑i=1nh​\(yi,r𝖳​gi\+1−‖r‖22​⟨𝒘j,𝒛i⟩\)\)⋅∏j=1kν0​\(d𝒘j\)\]\\displaystyle\\mathbb\{E\}\\left\[\\int\_\{\(\\mathbb\{S\}^\{d\-1\}\)^\{k\}\}\\exp\\left\(\\beta\\cdot\\sum\_\{j=1\}^\{k\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},r^\{\\sf T\}g\_\{i\}\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\langle\\text\{$w$\}\_\{j\},\\text\{$z$\}\_\{i\}\\rangle\\right\)\\right\)\\cdot\\prod\_\{j=1\}^\{k\}\\nu\_\{0\}\(\\textup\{d\}\\text\{$w$\}\_\{j\}\)\\right\]=\\displaystyle=\\,∫\(𝕊d−1\)k𝔼​\[exp⁡\(β⋅∑j=1k∑i=1nh​\(yi,r𝖳​gi\+1−‖r‖22​⟨𝒘j,𝒛i⟩\)\)\]⋅∏j=1kν0​\(d𝒘j\)\\displaystyle\\int\_\{\(\\mathbb\{S\}^\{d\-1\}\)^\{k\}\}\\mathbb\{E\}\\left\[\\exp\\left\(\\beta\\cdot\\sum\_\{j=1\}^\{k\}\\sum\_\{i=1\}^\{n\}h\\left\(y\_\{i\},r^\{\\sf T\}g\_\{i\}\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\langle\\text\{$w$\}\_\{j\},\\text\{$z$\}\_\{i\}\\rangle\\right\)\\right\)\\right\]\\cdot\\prod\_\{j=1\}^\{k\}\\nu\_\{0\}\(\\textup\{d\}\\text\{$w$\}\_\{j\}\)=\\displaystyle=\\,∫\(𝕊d−1\)k𝔼​\[exp⁡\(β⋅∑j=1kh​\(Y,r𝖳​G\+1−‖r‖22​⟨𝒘j,𝒛⟩\)\)\]n⋅∏j=1kν0​\(d𝒘j\),\\displaystyle\\int\_\{\(\\mathbb\{S\}^\{d\-1\}\)^\{k\}\}\\mathbb\{E\}\\left\[\\exp\\left\(\\beta\\cdot\\sum\_\{j=1\}^\{k\}h\\left\(Y,r^\{\\sf T\}G\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\langle\\text\{$w$\}\_\{j\},\\text\{$z$\}\\rangle\\right\)\\right\)\\right\]^\{n\}\\cdot\\prod\_\{j=1\}^\{k\}\\nu\_\{0\}\(\\textup\{d\}\\text\{$w$\}\_\{j\}\),where the inner expectation is taken under

\(Y,G\)⟂⟂𝒛,𝒛∼𝖭\(𝟎,𝑰d\),Y=φ\(G,ε\),G∼𝖭\(0,Ik\),ε∼Pε,ε⟂⟂G\.\(Y,G\)\\perp\\\!\\\!\\\!\\perp\\text\{$z$\},\\quad\\text\{$z$\}\\sim\{\\sf N\}\(\\mathrm\{\\bf 0\},\{\\boldsymbol\{I\}\}\_\{d\}\),\\,Y=\\varphi\(G,\\varepsilon\),\\,G\\sim\{\\sf N\}\(0,I\_\{k\}\),\\,\\varepsilon\\sim P\_\{\\varepsilon\},\\,\\varepsilon\\perp\\\!\\\!\\\!\\perp G\.Denoting byQQthe overlap matrix of the𝒘j\\text\{$w$\}\_\{j\}’s, namelyQi​j=⟨𝒘i,𝒘j⟩Q\_\{ij\}=\\langle\\text\{$w$\}\_\{i\},\\text\{$w$\}\_\{j\}\\ranglefor1≤i,j≤k1\\leq i,j\\leq k, then we have for𝒛∼𝖭​\(𝟎,𝑰d\)\\text\{$z$\}\\sim\{\\sf N\}\(\\mathrm\{\\bf 0\},\{\\boldsymbol\{I\}\}\_\{d\}\),

𝔼​\[exp⁡\(β⋅∑j=1kh​\(Y,r𝖳​G\+1−‖r‖22​⟨𝒘j,𝒛⟩\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\\exp\\left\(\\beta\\cdot\\sum\_\{j=1\}^\{k\}h\\left\(Y,r^\{\\sf T\}G\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\langle\\text\{$w$\}\_\{j\},\\text\{$z$\}\\rangle\\right\)\\right\)\\right\]=\\displaystyle=\\,𝔼​\[exp⁡\(β⋅∑j=1kh​\(Y,r𝖳​G\+1−‖r‖22​Zj\)\)\],\\displaystyle\\mathbb\{E\}\\left\[\\exp\\left\(\\beta\\cdot\\sum\_\{j=1\}^\{k\}h\\left\(Y,r^\{\\sf T\}G\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ Z\_\{j\}\\right\)\\right\)\\right\],whereZ=\(Zj\)j=1k∼𝖭​\(0,Q\)Z=\(Z\_\{j\}\)\_\{j=1\}^\{k\}\\sim\{\\sf N\}\(0,Q\)is independent of\(Y,G\)\(Y,G\)\. For future convenience, we denote the above quantity asfβ,h​\(Q\)f\_\{\\beta,h\}\(Q\), i\.e\.,

fβ,h​\(Q\)=𝔼​\[exp⁡\(β⋅∑j=1kh​\(Y,r𝖳​G\+1−‖r‖22​Zj\)\)\],f\_\{\\beta,h\}\(Q\)=\\mathbb\{E\}\\left\[\\exp\\left\(\\beta\\cdot\\sum\_\{j=1\}^\{k\}h\\left\(Y,r^\{\\sf T\}G\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ Z\_\{j\}\\right\)\\right\)\\right\],it then follows that

𝔼​\[Zβk\]=\\displaystyle\\mathbb\{E\}\\left\[Z\_\{\\beta\}^\{k\}\\right\]=\\,∫\(𝕊d−1\)kfβ,h​\(Q\)n⋅∏j=1kν0​\(d𝒘j\)\\displaystyle\\int\_\{\(\\mathbb\{S\}^\{d\-1\}\)^\{k\}\}f\_\{\\beta,h\}\(Q\)^\{n\}\\cdot\\prod\_\{j=1\}^\{k\}\\nu\_\{0\}\(\\textup\{d\}\\text\{$w$\}\_\{j\}\)=\\displaystyle=\\,∫𝒮\+k​\(1\)fβ,h​\(Q\)n​exp⁡\(d​Id​\(Q\)\)​d​Q,\\displaystyle\\int\_\{\{\\mathcal\{S\}\}\_\{\+\}^\{k\}\(1\)\}f\_\{\\beta,h\}\(Q\)^\{n\}\\exp\\left\(dI\_\{d\}\(Q\)\\right\)\\textup\{d\}Q,where𝒮\+k​\(1\)\{\\mathcal\{S\}\}\_\{\+\}^\{k\}\(1\)denotes the space of allk×kk\\times kpositive semidefinite matrices with all ones on the diagonal, andd​Q=∏1≤i<j≤kd​Qi​j\\textup\{d\}Q=\\prod\_\{1\\leq i<j\\leq k\}\\textup\{d\}Q\_\{ij\}represents the uniform probability measure on this space\. Moreover, we have for fixedkk,

limd→∞Id​\(Q\)=12​log​detQ,\\lim\_\{d\\to\\infty\}I\_\{d\}\(Q\)=\\frac\{1\}\{2\}\\log\\det Q,thus leading to

limn→∞1n​log⁡𝔼​\[Zβk\]=maxQ∈𝒮\+k​\(1\)⁡\{log⁡fβ,h​\(Q\)\+12​α​log​detQ\}\\displaystyle\\lim\_\{n\\to\\infty\}\\frac\{1\}\{n\}\\log\\mathbb\{E\}\\left\[Z\_\{\\beta\}^\{k\}\\right\]=\\,\\max\_\{Q\\in\{\\mathcal\{S\}\}\_\{\+\}^\{k\}\(1\)\}\\left\\\{\\log f\_\{\\beta,h\}\(Q\)\+\\frac\{1\}\{2\\alpha\}\\log\\det Q\\right\\\}=\\displaystyle=\\,maxQ∈𝒮\+k​\(1\)⁡\{log⁡𝔼​\[exp⁡\(β⋅∑j=1kh​\(Y,r𝖳​G\+1−‖r‖22​Zj\)\)\]\+12​α​log​detQ\},\\displaystyle\\max\_\{Q\\in\{\\mathcal\{S\}\}\_\{\+\}^\{k\}\(1\)\}\\left\\\{\\log\\mathbb\{E\}\\left\[\\exp\\left\(\\beta\\cdot\\sum\_\{j=1\}^\{k\}h\\left\(Y,r^\{\\sf T\}G\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ Z\_\{j\}\\right\)\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\log\\det Q\\right\\\},which we denote asSβ,h​\(α,k\)S\_\{\\beta,h\}\(\\alpha,k\)\. Assume again that we can interchange the limits arbitrarily, then we get that

limβ→∞limn→∞1n​β​𝔼​\[log⁡Zβ\]=\\displaystyle\\lim\_\{\\beta\\to\\infty\}\\lim\_\{n\\to\\infty\}\\frac\{1\}\{n\\beta\}\\mathbb\{E\}\\left\[\\log Z\_\{\\beta\}\\right\]=\\,limβ→∞limn→∞limk→0\+1n​β​k​log⁡𝔼​\[Zβk\]\\displaystyle\\lim\_\{\\beta\\to\\infty\}\\lim\_\{n\\to\\infty\}\\lim\_\{k\\to 0^\{\+\}\}\\frac\{1\}\{n\\beta k\}\\log\\mathbb\{E\}\\left\[Z\_\{\\beta\}^\{k\}\\right\]=\\displaystyle=\\,limβ→∞1β​limk→0\+1k​limn→∞1n​log⁡𝔼​\[Zβk\]\\displaystyle\\lim\_\{\\beta\\to\\infty\}\\frac\{1\}\{\\beta\}\\lim\_\{k\\to 0^\{\+\}\}\\frac\{1\}\{k\}\\lim\_\{n\\to\\infty\}\\frac\{1\}\{n\}\\log\\mathbb\{E\}\\left\[Z\_\{\\beta\}^\{k\}\\right\]=\\displaystyle=\\,limβ→∞1β​limk→0\+1k​Sβ,h​\(α,k\)\.\\displaystyle\\lim\_\{\\beta\\to\\infty\}\\frac\{1\}\{\\beta\}\\lim\_\{k\\to 0^\{\+\}\}\\frac\{1\}\{k\}S\_\{\\beta,h\}\(\\alpha,k\)\.To compute this limit, we resort to the full RSB \(full replica symmetry breaking\) ansatz described in Section 3 of\[[GR00](https://arxiv.org/html/2606.28573#bib.bibx18)\]\. Following their calculation, the limiting free energy can be expressed as the extreme value of a variational problem\. To be specific, we have

1β​limk→0\+1k​Sβ,h​\(α,k\)=infv∈𝒰​\[0,1\]𝖠​\(v,β\),\\displaystyle\\frac\{1\}\{\\beta\}\\lim\_\{k\\to 0^\{\+\}\}\\frac\{1\}\{k\}S\_\{\\beta,h\}\(\\alpha,k\)=\\inf\_\{v\\in\\mathscr\{U\}\[0,1\]\}\\mathsf\{A\}\(v,\\beta\)\\,,𝖠​\(v,β\):=𝔼Y,G​\[fY,G,r,v​\(0,0\)\]\+12​α​β​∫01\(1Dv​\(t\)−11−t\)​d​t,\\displaystyle\\mathsf\{A\}\(v,\\beta\):=\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,G,r,v\}\(0,0\)\\right\]\+\\frac\{1\}\{2\\alpha\\beta\}\\int\_\{0\}^\{1\}\\left\(\\frac\{1\}\{D\_\{v\}\(t\)\}\-\\frac\{1\}\{1\-t\}\\right\)\\textup\{d\}t,where𝒰​\[0,1\]\\mathscr\{U\}\[0,1\]is the space of all non\-decreasing functionsv:\[0,1\]→\[0,1\]v:\[0,1\]\\to\[0,1\],

Dv​\(t\)=∫t1v​\(s\)​d​s,D\_\{v\}\(t\)=\\int\_\{t\}^\{1\}v\(s\)\\textup\{d\}s,and for each fixed pair\(y,g\)\(y,g\),fy,g,r,v​\(t,x\)f\_\{y,g,r,v\}\(t,x\)satisfies the PDE:

∂tfy,g,r,v​\(t,x\)\+12​β​v​\(t\)​\(∂xfy,g,r,v​\(t,x\)\)2\+12​∂x2fy,g,r,v​\(t,x\)=0,fy,g,r,v​\(1,x\)=h​\(y,r𝖳​g\+1−‖r‖22​x\)\.\\begin\{split\}&\\partial\_\{t\}f\_\{y,g,r,v\}\(t,x\)\+\\frac\{1\}\{2\}\\beta v\(t\)\\left\(\\partial\_\{x\}f\_\{y,g,r,v\}\(t,x\)\\right\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,g,r,v\}\(t,x\)=\\,0,\\\\ &f\_\{y,g,r,v\}\(1,x\)=\\,h\\left\(y,r^\{\\sf T\}g\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ x\\right\)\.\\end\{split\}\(70\)The lemma below gives the zero\-temperature limit \(β→∞\\beta\\to\\infty\) of the variational functional𝖠​\(v,β\)\\mathsf\{A\}\(v,\\beta\), along specific sequences ofvβv\_\{\\beta\}\.

###### Lemma A\.1\.

Letc\>0c\>0, andμ​\(t\):\[0,1\)→ℝ≥0\\mu\(t\):\[0,1\)\\to\\mathbb\{R\}\_\{\\geq 0\}be a non\-decreasing function with∫01μ​\(t\)​d​t<∞\\int\_\{0\}^\{1\}\\mu\(t\)\\,\\textup\{d\}t<\\infty\. Further, assume thatv​\(t\)=vβ​\(t\)v\(t\)=v\_\{\\beta\}\(t\)has the following form:

vβ​\(t\)=μ​\(t\)β​𝟏t<1−cβ\+𝟏t≥1−cβ\.v\_\{\\beta\}\(t\)=\\frac\{\\mu\(t\)\}\{\\beta\}\\mathrm\{\\bf 1\}\_\{t<1\-\\frac\{c\}\{\\beta\}\}\+\\mathrm\{\\bf 1\}\_\{t\\geq 1\-\\frac\{c\}\{\\beta\}\}\.Then, we have

limβ→∞𝖠​\(vβ,β\)=𝖥1​\(μ,c,r\),\\displaystyle\\lim\_\{\\beta\\to\\infty\}\\mathsf\{A\}\(v\_\{\\beta\},\\beta\)=\\mathsf\{F\}\_\{1\}\(\\mu,c,r\)\\,,𝖥1​\(μ,c,r\):=𝔼Y,G​\[fY,G,r​\(0,0\)\]\+12​α​∫01d​tc\+∫t1μ​\(s\)​d​s,\\displaystyle\\mathsf\{F\}\_\{1\}\(\\mu,c,r\):=\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,G,r\}\(0,0\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{0\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\},where for each fixed pair\(y,g\)\(y,g\),fy,g,rf\_\{y,g,r\}solves the terminal\-value problem:

∂tfy,g,r​\(t,x\)\+12​μ​\(t\)​\(∂xfy,g,r​\(t,x\)\)2\+12​∂x2fy,g,r​\(t,x\)=0,fy,g,r​\(1,x\)=supu∈ℝ\{h​\(y,r𝖳​g\+1−‖r‖22​\(x\+u\)\)−u22​c\}\.\\begin\{split\}&\\partial\_\{t\}f\_\{y,g,r\}\(t,x\)\+\\frac\{1\}\{2\}\\mu\(t\)\\left\(\\partial\_\{x\}f\_\{y,g,r\}\(t,x\)\\right\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,g,r\}\(t,x\)=\\,0,\\\\ &f\_\{y,g,r\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,r^\{\\sf T\}g\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \(x\+u\)\\right\)\-\\frac\{u^\{2\}\}\{2c\}\\right\\\}\.\\end\{split\}\(71\)

###### Proof\.

In this proof, we will suppress the dependence offy,g,r,vf\_\{y,g,r,v\}onvvto simplify the notation\. Definingtβ=1−c/βt\_\{\\beta\}=1\-c/\\beta, Eq\. \([70](https://arxiv.org/html/2606.28573#A1.E70)\) then reduces to

∂tfy,g,r​\(t,x\)\+12​μ​\(t\)​\(∂xfy,g,r​\(t,x\)\)2\+12​∂x2fy,g,r​\(t,x\)=0,t∈\[0,tβ\),\\displaystyle\\partial\_\{t\}f\_\{y,g,r\}\(t,x\)\+\\frac\{1\}\{2\}\\mu\(t\)\\left\(\\partial\_\{x\}f\_\{y,g,r\}\(t,x\)\\right\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,g,r\}\(t,x\)=\\,0,\\ t\\in\[0,t\_\{\\beta\}\),∂tfy,g,r​\(t,x\)\+12​β​\(∂xfy,g,r​\(t,x\)\)2\+12​∂x2fy,g,r​\(t,x\)=0,t∈\[tβ,1\),\\displaystyle\\partial\_\{t\}f\_\{y,g,r\}\(t,x\)\+\\frac\{1\}\{2\}\\beta\\left\(\\partial\_\{x\}f\_\{y,g,r\}\(t,x\)\\right\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,g,r\}\(t,x\)=\\,0,\\ t\\in\[t\_\{\\beta\},1\),fy,g,r​\(1,x\)=h​\(y,r𝖳​g\+1−‖r‖22​x\)\.\\displaystyle f\_\{y,g,r\}\(1,x\)=\\,h\\left\(y,r^\{\\sf T\}g\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ x\\right\)\.Using Cole\-Hopf transform, we know that

fy,g,r​\(tβ,x\)=\\displaystyle f\_\{y,g,r\}\(t\_\{\\beta\},x\)=\\,1β​log⁡𝔼G∼𝖭​\(0,1\)​\[exp⁡\(β⋅h​\(y,r𝖳​g\+1−‖r‖22​\(x\+cβ​G\)\)\)\]\\displaystyle\\frac\{1\}\{\\beta\}\\log\\mathbb\{E\}\_\{G\\sim\{\\sf N\}\(0,1\)\}\\left\[\\exp\\left\(\\beta\\cdot h\\left\(y,r^\{\\sf T\}g\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\left\(x\+\\sqrt\{\\frac\{c\}\{\\beta\}\}G\\right\)\\right\)\\right\)\\right\]=\\displaystyle=\\,1β​log⁡\(12​π​∫ℝexp⁡\(β⋅h​\(y,r𝖳​g\+1−‖r‖22​\(x\+cβ​z\)\)−z22\)​d​z\)\\displaystyle\\frac\{1\}\{\\beta\}\\log\\left\(\\frac\{1\}\{\\sqrt\{2\\pi\}\}\\int\_\{\\mathbb\{R\}\}\\exp\\left\(\\beta\\cdot h\\left\(y,r^\{\\sf T\}g\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \\left\(x\+\\sqrt\{\\frac\{c\}\{\\beta\}\}z\\right\)\\right\)\-\\frac\{z^\{2\}\}\{2\}\\right\)\\textup\{d\}z\\right\)=\\displaystyle=\\,1β​log⁡\(β2​π​c​∫ℝexp⁡\(β⋅h​\(y,r𝖳​g\+1−‖r‖22​\(x\+u\)\)−β​u22​c\)​d​u\),\\displaystyle\\frac\{1\}\{\\beta\}\\log\\left\(\\frac\{\\sqrt\{\\beta\}\}\{\\sqrt\{2\\pi c\}\}\\int\_\{\\mathbb\{R\}\}\\exp\\left\(\\beta\\cdot h\\left\(y,r^\{\\sf T\}g\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \(x\+u\)\\right\)\-\\frac\{\\beta u^\{2\}\}\{2c\}\\right\)\\textup\{d\}u\\right\),which converges to

supu∈ℝ\{h​\(y,r𝖳​g\+1−‖r‖22​\(x\+u\)\)−u22​c\}\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,r^\{\\sf T\}g\+\\sqrt\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\ \(x\+u\)\\right\)\-\\frac\{u^\{2\}\}\{2c\}\\right\\\}asβ→∞\\beta\\to\\infty, uniformly over compact sets\. Moreover, sincetβ→1t\_\{\\beta\}\\to 1, we deduce that the limit offy,g,rf\_\{y,g,r\}asβ→∞\\beta\\to\\inftyindeed solves Eq\. \([71](https://arxiv.org/html/2606.28573#A1.E71)\)\. This establishes that the first term in𝖠​\(vβ,β\)\\mathsf\{A\}\(v\_\{\\beta\},\\beta\)converges to the first term in𝖥1​\(μ,c,r\)\\mathsf\{F\}\_\{1\}\(\\mu,c,r\)\. To compute the limit of the second term, we note thatDv​\(t\)=1−tD\_\{v\}\(t\)=1\-tift≥tβt\\geq t\_\{\\beta\}\. Therefore,

12​α​β​∫01\(1Dv​\(t\)−11−t\)​d​t=\\displaystyle\\frac\{1\}\{2\\alpha\\beta\}\\int\_\{0\}^\{1\}\\left\(\\frac\{1\}\{D\_\{v\}\(t\)\}\-\\frac\{1\}\{1\-t\}\\right\)\\textup\{d\}t=\\,12​α​β​∫0tβ\(1Dv​\(t\)−11−t\)​d​t\\displaystyle\\frac\{1\}\{2\\alpha\\beta\}\\int\_\{0\}^\{t\_\{\\beta\}\}\\left\(\\frac\{1\}\{D\_\{v\}\(t\)\}\-\\frac\{1\}\{1\-t\}\\right\)\\textup\{d\}t=\\displaystyle=\\,12​α​∫0tβd​tc\+∫ttβμ​\(s\)​d​s\+12​α​β​log⁡\(1−tβ\)\\displaystyle\\frac\{1\}\{2\\alpha\}\\int\_\{0\}^\{t\_\{\\beta\}\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{t\_\{\\beta\}\}\\mu\(s\)\\textup\{d\}s\}\+\\frac\{1\}\{2\\alpha\\beta\}\\log\\left\(1\-t\_\{\\beta\}\\right\)=\\displaystyle=\\,12​α​∫0tβd​tc\+∫ttβμ​\(s\)​d​s\+12​α​β​log⁡\(cβ\)\\displaystyle\\frac\{1\}\{2\\alpha\}\\int\_\{0\}^\{t\_\{\\beta\}\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{t\_\{\\beta\}\}\\mu\(s\)\\textup\{d\}s\}\+\\frac\{1\}\{2\\alpha\\beta\}\\log\\left\(\\frac\{c\}\{\\beta\}\\right\)→\\displaystyle\\to\\,12​α​∫01d​tc\+∫t1μ​\(s\)​d​s​as​β→∞\.\\displaystyle\\frac\{1\}\{2\\alpha\}\\int\_\{0\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\}\\ \\text\{as\}\\ \\beta\\to\\infty\.This completes the proof\. ∎

Defining

𝒰=\{μ:\[0,1\)→ℝ≥0:μ​non\-decreasing,∫01μ​\(t\)​d​t<∞\},\\mathscr\{U\}=\\left\\\{\\mu:\[0,1\)\\to\\mathbb\{R\}\_\{\\geq 0\}:\\ \\mu\\ \\text\{non\-decreasing\},\\ \\int\_\{0\}^\{1\}\\mu\(t\)\\textup\{d\}t<\\infty\\right\\\},our replica calculation leads to

limn→∞max𝒘∈𝕊d−1⁡Hn,d​\(𝒘\)=supr∈Bk​\(1\)inf\(μ,c\)∈𝒰×ℝ\>0𝖥1​\(μ,c,r\),\\lim\_\{n\\to\\infty\}\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}H\_\{n,d\}\(\\text\{$w$\}\)=\\,\\sup\_\{r\\in B\_\{k\}\(1\)\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\_\{1\}\(\\mu,c,r\),with𝖥1\\mathsf\{F\}\_\{1\}defined in[A\.1](https://arxiv.org/html/2606.28573#A1.Thmlem1)\. To establish the replica prediction in[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2), it suffices to show that the𝖥1\\mathsf\{F\}\_\{1\}defined as per[Eq\.15](https://arxiv.org/html/2606.28573#S2.E15)matches the one defined in[A\.1](https://arxiv.org/html/2606.28573#A1.Thmlem1)\. For fixedy∈ℝy\\in\\mathbb\{R\}and\(μ,c\)∈𝒰\(\\mu,c\)\\in\\mathscr\{U\}, letfy,μ\(r\):\[‖r‖22,1\]×ℝ→ℝf\_\{y,\\mu\}^\{\(r\)\}:\[\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\},1\]\\times\\mathbb\{R\}\\to\\mathbb\{R\}be the solution to the PDE:

∂tfy,μ\(r\)​\(t,x\)\+12​μ​\(t−‖r‖221−‖r‖22\)​\(∂xfy,μ\(r\)​\(t,x\)\)2\+12​∂x2fy,μ\(r\)​\(t,x\)=0,fy,μ\(r\)​\(1,x\)=supu∈ℝ\{h​\(y,x\+u\)−u22​\(1−‖r‖22\)​c\}\.\\begin\{split\}&\\partial\_\{t\}f\_\{y,\\mu\}^\{\(r\)\}\(t,x\)\+\\frac\{1\}\{2\}\\mu\\left\(\\frac\{t\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\right\)\(\\partial\_\{x\}f\_\{y,\\mu\}^\{\(r\)\}\(t,x\)\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,\\mu\}^\{\(r\)\}\(t,x\)=\\,0,\\\\ &f\_\{y,\\mu\}^\{\(r\)\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,x\+u\\right\)\-\\frac\{u^\{2\}\}\{2\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)c\}\\right\\\}\.\\end\{split\}Then, straightforward calculation reveals that

fy,g,r​\(t,x\)=fy,μ\(r\)​\(‖r‖22\+\(1−‖r‖22\)​t,\(1−‖r‖22\)1/2​x\+r𝖳​g\)⟹fy,g,r​\(0,0\)=fy,μ\(r\)​\(‖r‖22,r𝖳​g\),\\begin\{split\}&f\_\{y,g,r\}\(t,x\)=f\_\{y,\\mu\}^\{\(r\)\}\\big\(\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\+\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)t,\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)^\{1/2\}x\+r^\{\\sf T\}g\\big\)\\\\ \\implies\\,&f\_\{y,g,r\}\(0,0\)=f\_\{y,\\mu\}^\{\(r\)\}\(\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\},r^\{\\sf T\}g\),\\end\{split\}which leads to

𝖥1​\(μ,c,r\):=\\displaystyle\\mathsf\{F\}\_\{1\}\(\\mu,c,r\):=\\,𝔼Y,G​\[fY,μ\(r\)​\(‖r‖22,r𝖳​G\)\]\+12​α​∫01d​tc\+∫t1μ​\(s\)​d​s\\displaystyle\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}^\{\(r\)\}\(\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\},r^\{\\sf T\}G\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{0\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\}=\\displaystyle=\\,𝔼Y,G​\[fY,μ\(r\)​\(‖r‖22,r𝖳​G\)\]\+12​α​∫01\(1−‖r‖22\)​d​t\(1−‖r‖22\)​c\+∫t1\(1−‖r‖22\)​μ​\(s\)​d​s\\displaystyle\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}^\{\(r\)\}\(\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\},r^\{\\sf T\}G\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{0\}^\{1\}\\frac\{\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)\\textup\{d\}t\}\{\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)c\+\\int\_\{t\}^\{1\}\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)\\mu\(s\)\\textup\{d\}s\}=\\displaystyle=\\,𝔼Y,G​\[fY,μ\(r\)​\(‖r‖22,r𝖳​G\)\]\+12​α​∫‖r‖221d​t\(1−‖r‖22\)​c\+∫t1μ​\(s−‖r‖221−‖r‖22\)​d​s,\\displaystyle\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\mu\}^\{\(r\)\}\(\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\},r^\{\\sf T\}G\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)c\+\\int\_\{t\}^\{1\}\\mu\\left\(\\frac\{s\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\right\)\\textup\{d\}s\},where the last equality follows from the change of time variablet↦‖r‖22\+\(1−‖r‖22\)​tt\\mapsto\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\+\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)t\. Now, recasting\(1−‖r‖22\)​c\(1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\)casccandμ​\(t−‖r‖221−‖r‖22\)\\mu\\left\(\\frac\{t\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\{1\-\\left\\\|\{r\}\\right\\\|\_\{2\}^\{2\}\}\\right\)asμ​\(t\)\\mu\(t\)recovers[Eqs\.14](https://arxiv.org/html/2606.28573#S2.E14)and[15](https://arxiv.org/html/2606.28573#S2.E15)in[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)\. This establishes the replica prediction\.

## Appendix BAppendix for[Section3](https://arxiv.org/html/2606.28573#S3)

### B\.1Reduction to a random signal

###### Lemma B\.1\.

Under the multi\-index model specified in[1\.1](https://arxiv.org/html/2606.28573#S1.Thmass1), the distribution of the training and test errors of any machine learning method \(of the form𝐖^n=𝐖^n​\(𝐗,𝐲\)\\widehat\{\\text\{$W$\}\}\_\{n\}=\\widehat\{\\text\{$W$\}\}\_\{n\}\(\\text\{$X$\},\\text\{$y$\}\)\) is the same as the one for another multi\-index model, whereφ\\varphiis replaced by a new functionφ~\\widetilde\{\\varphi\}, and𝐖∗\\text\{$W$\}\_\{\*\}is replaced by a new signal matrix𝐖~∗\\widetilde\{\\text\{$W$\}\}\_\{\*\}that is uniformly random overO​\(d,k\)O\(d,k\)\(Haar distributed\)\.

In particular, the empirical distribution of the rows ofn​𝐖~∗\\sqrt\{n\}\\widetilde\{\\text\{$W$\}\}\_\{\*\}converges inW2W\_\{2\}distance \(and hence weakly\) to𝖭​\(0,α​Ik\)\{\\sf N\}\(0,\\alpha I\_\{k\}\)almost surely, asn,d→∞n,d\\to\\inftywithn/d→αn/d\\to\\alpha\.

###### Proof\.

First of all, let𝑾~∗=𝑾∗​\(𝑾∗𝖳​𝑾∗\)−1/2\\widetilde\{\\text\{$W$\}\}\_\{\*\}=\\text\{$W$\}\_\{\*\}\(\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\text\{$W$\}\_\{\*\}\)^\{\-1/2\}andφ~​\(𝒛,ε\)=φ​\(\(𝑾∗𝖳​𝑾∗\)1/2​𝒛,ε\)\\widetilde\{\\varphi\}\(\\text\{$z$\},\\varepsilon\)=\\varphi\(\(\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\text\{$W$\}\_\{\*\}\)^\{1/2\}\\text\{$z$\},\\varepsilon\)\. Then, we know that𝑾~∗∈O​\(d,k\)\\widetilde\{\\text\{$W$\}\}\_\{\*\}\\in O\(d,k\), and

φ~​\(𝑾~∗𝖳​𝒙i,εi\)=φ​\(𝑾∗𝖳​𝒙i,εi\)\.\\widetilde\{\\varphi\}\\big\(\\widetilde\{\\text\{$W$\}\}\_\{\*\}^\{\\sf T\}\\text\{$x$\}\_\{i\},\\varepsilon\_\{i\}\\big\)=\\varphi\\big\(\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\text\{$x$\}\_\{i\},\\varepsilon\_\{i\}\\big\)\.Hence, one can assume without loss of generality that𝑾∗∈O​\(d,k\)\\text\{$W$\}\_\{\*\}\\in O\(d,k\)\. Now for any fixed𝑾∗∈O​\(d,k\)\\text\{$W$\}\_\{\*\}\\in O\(d,k\), let𝑹Rbe a uniformly random orthogonal matrix of sized×dd\\times d, namely𝑹Ris sampled from the Haar measure onO​\(d,d\)O\(d,d\)\. Then, we know that𝑿𝑾∗=𝑿𝑹𝖳​𝑹𝑾∗\\text\{$X$\}\\text\{$W$\}\_\{\*\}=\\text\{$X$\}\\text\{$R$\}^\{\\sf T\}\\text\{$R$\}\\text\{$W$\}\_\{\*\}, which leads to

𝒚=φ​\(𝑿𝑾∗,𝜺\)=φ​\(𝑿𝑹𝖳​𝑹𝑾∗,𝜺\)\.\\text\{$y$\}=\\varphi\(\\text\{$X$\}\\text\{$W$\}\_\{\*\},\{\\boldsymbol\{\\varepsilon\}\}\)=\\varphi\(\\text\{$X$\}\\text\{$R$\}^\{\\sf T\}\\text\{$R$\}\\text\{$W$\}\_\{\*\},\{\\boldsymbol\{\\varepsilon\}\}\)\.Further,𝑾~∗=𝑹𝑾∗\\widetilde\{\\text\{$W$\}\}\_\{\*\}=\\text\{$R$\}\\text\{$W$\}\_\{\*\}is Haar distributed overO​\(d,k\)O\(d,k\)\. Using rotational invariance of standard Gaussian measure and the Haar measure onO​\(d,k\)O\(d,k\), we know that the empirical distribution of the rows ofn​𝑾~∗\\sqrt\{n\}\\widetilde\{\\text\{$W$\}\}\_\{\*\}weakly converges to𝖭​\(0,α​Ik\)\{\\sf N\}\(0,\\alpha I\_\{k\}\)almost surely, and that𝑿~=𝑿𝑹𝖳=d𝑿\\widetilde\{\\text\{$X$\}\}=\\text\{$X$\}\\text\{$R$\}^\{\\sf T\}\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}\\text\{$X$\}is independent of𝑾~∗\\widetilde\{\\text\{$W$\}\}\_\{\*\}\. As a consequence, using the estimator𝑾^n​\(𝑿~​𝑹,𝒚\)\\widehat\{\\text\{$W$\}\}\_\{n\}\(\\widetilde\{\\text\{$X$\}\}\\text\{$R$\},\\text\{$y$\}\)would yield the same training and test errors as in the original model\. This completes the proof\. ∎

### B\.2AMP achievability results for the casem=k=1m=k=1

In this appendix, we specialize our general AMP achievability result \(Theorem[3\.1](https://arxiv.org/html/2606.28573#S3.Thmthm1)\) to the casem=k=1m=k=1\. The following definitions are simplified versions of Definitions[1](https://arxiv.org/html/2606.28573#Thmdefn1)and[2](https://arxiv.org/html/2606.28573#Thmdefn2), respectively\.

###### Definition 4\(Bayes AMP feasible region\)\.

Let the random vector\(Y,G,Z\)∈ℝ3\(Y,G,Z\)\\in\\mathbb\{R\}^\{3\}be such that

\(Y,G\)⟂⟂Z,Z∼𝖭\(0,1\),G∼𝖭\(0,1\),Y=φ\(G,ε\),ε∼Pε,ε⟂⟂G\.\\begin\{array\}\[\]\{l\}\(Y,G\)\\perp\\\!\\\!\\\!\\perp Z,\\,Z\\sim\{\\sf N\}\(0,1\),\\,G\\sim\{\\sf N\}\(0,1\),\\\\ Y=\\varphi\(G,\\varepsilon\),\\,\\varepsilon\\sim P\_\{\\varepsilon\},\\,\\varepsilon\\perp\\\!\\\!\\\!\\perp G\.\\end\{array\}\(72\)Assume that𝔼​\[G\|Y\]≠0\\mathbb\{E\}\\left\[G\|Y\\right\]\\neq 0, and definecBayes∈ℝ≥0c\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\in\\mathbb\{R\}\_\{\\geq 0\}via

cBayes\\displaystyle c\_\{\\mbox\{\\tiny\\rm Bayes\}\}:=min⁡\{c≥0:c2=α​𝔼​\[𝔼​\[G−c​Z\|c​G\+Z,Y\]2\]\}\.\\displaystyle:=\\min\\Big\\\{c\\geq 0:\\;c^\{2\}=\\alpha\\mathbb\{E\}\\Big\[\\mathbb\{E\}\\big\[G\-cZ\\big\|cG\+Z,Y\\big\]^\{2\}\\Big\]\\Big\\\}\.\(73\)We then define the*Bayes AMP feasible region*as

ABayes=\{\(r,q\)∈ℝ2:q∈\[r2,1\],r2≤cBayes2​\(q−r2\)\}\.A\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\left\\\{\(r,q\)\\in\\mathbb\{R\}^\{2\}:q\\in\[r^\{2\},1\],\\,r^\{2\}\\leq c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\(q\-r^\{2\}\)\\right\\\}\.

###### Remark 8\.

As in the case of generalmmandkk,cBayesc\_\{\\mbox\{\\tiny\\rm Bayes\}\}can be equivalently defined as the limit of a certain one\-dimensional recursion\. Namely, we define the sequence\{ct\}t=0∞\\\{c\_\{t\}\\\}\_\{t=0\}^\{\\infty\}via

c0=0,ct\+1=α1/2​𝔼​\[𝔼​\[G−ct​Z\|ct​G\+Z,Y\]2\]1/2,t≥0\.c\_\{0\}=0,\\quad c\_\{t\+1\}=\\alpha^\{1/2\}\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-c\_\{t\}Z\|c\_\{t\}G\+Z,Y\\right\]^\{2\}\\right\]^\{1/2\},\\quad t\\geq 0\.\(74\)Then, we have similarlycBayes=limt→∞ctc\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\lim\_\{t\\to\\infty\}c\_\{t\}\(cf\.[B\.3](https://arxiv.org/html/2606.28573#A2.Thmlem3)\)\.

###### Definition 5\.

Let\(Y,G,Z\)\(Y,G,Z\)be as described in[Eq\.72](https://arxiv.org/html/2606.28573#A2.E72), and defineZr,q=r​G\+q−r2​ZZ\_\{r,q\}=rG\+\\sqrt\{q\-r^\{2\}\}Z\. We say thatF:ℝ2→ℝF:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}is an\(r,q\)\(r,q\)\-contraction, if

r=𝔼​\[∂F∂G​\(Zr,q,φ​\(G,ε\)\)\],q=r2\+1α​𝔼​\[F​\(Zr,q,φ​\(G,ε\)\)2\],\\displaystyle r=\\,\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial G\}\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\],\\quad q=\\,r^\{2\}\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\],and

1α​𝔼​\[∂F∂Zr,q​\(Zr,q,φ​\(G,ε\)\)2\]≤1\.\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{r,q\}\}\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\]\\leq 1\.

###### Theorem B\.1\.

For any\(r,q\)∈ABayes\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, let\(Y,Zr,q\)\(Y,Z\_\{r,q\}\)be as described in[5](https://arxiv.org/html/2606.28573#Thmdefn5), andFFbe an\(r,q\)\(r,q\)\-contraction\. Let\(Bt\)0≤t≤1\(B\_\{t\}\)\_\{0\\leq t\\leq 1\}be a standard Brownian motion independent of\(Y,Zr,q\)\(Y,Z\_\{r,q\}\)\. Define the filtration\{ℱt\}0≤t≤1\\\{\\mathcal\{F\}\_\{t\}\\\}\_\{0\\leq t\\leq 1\}by

ℱt=σ​\(Zr,q,Y,\(Bs\)0≤s≤t\),0≤t≤1\.\\mathcal\{F\}\_\{t\}=\\sigma\\left\(Z\_\{r,q\},Y,\(B\_\{s\}\)\_\{0\\leq s\\leq t\}\\right\),\\ 0\\leq t\\leq 1\.Assumeq​\(t\)∈L2​\[0,1\]q\(t\)\\in L^\{2\}\[0,1\]satisfies∫01q​\(t\)2​d​t=1−q\\int\_\{0\}^\{1\}q\(t\)^\{2\}\\textup\{d\}t=1\-q, and\{ϕt\}0≤t≤1\\\{\\phi\_\{t\}\\\}\_\{0\\leq t\\leq 1\}is a progressively measurable stochastic process with respect to the filtration\{ℱt\}0≤t≤1\\\{\\mathcal\{F\}\_\{t\}\\\}\_\{0\\leq t\\leq 1\}, satisfying

𝔼​\[ϕt2\]≤1α,∀0≤t≤1\.\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\\leq\\frac\{1\}\{\\alpha\},\\ \\forall 0\\leq t\\leq 1\.Then, there exists a two\-stage AMP algorithm \(described in[Section3\.1](https://arxiv.org/html/2606.28573#S3.SS1)and further detailed in[Section7](https://arxiv.org/html/2606.28573#S7)\) that outputs𝐰^nAMP∈𝕊d−1\\widehat\{\\text\{$w$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\in\\mathbb\{S\}^\{d\-1\}, satisfying:

- \(a\)limn→∞𝒘∗𝖳​𝒘^nAMP=r\\lim\_\{n\\to\\infty\}\\text\{$w$\}\_\{\*\}^\{\\sf T\}\\widehat\{\\text\{$w$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}=ralmost surely\.
- \(b\)Define U=Zr,q\+1α​F​\(Zr,q,Y\)\+∫01q​\(t\)​\(1\+ϕt\)​d​Bt\.U=\\,Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{0\}^\{1\}q\(t\)\(1\+\\phi\_\{t\}\)\\textup\{d\}B\_\{t\}\.Asn/d→αn/d\\to\\alpha, the following holds almost surely for anyh∈Cb​\(ℝ2\)h\\in C\_\{b\}\(\\mathbb\{R\}^\{2\}\): 1n​∑i=1nδ\(yi,\(𝒘^nAMP\)𝖳​𝒙i\)→wLaw⁡\(Y,U\),1n​∑i=1nh​\(yi,\(𝒘^nAMP\)𝖳​𝒙i\)→𝔼​\[h​\(Y,U\)\]\.\\begin\{split\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\(y\_\{i\},\\,\(\\widehat\{\\text\{$w$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\)^\{\\sf T\}\\text\{$x$\}\_\{i\}\)\}\\stackrel\{\{\\scriptstyle w\}\}\{\{\\to\}\}\\,\\operatorname\{Law\}\(Y,U\),\\quad\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}h\(y\_\{i\},\\,\(\\widehat\{\\text\{$w$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\)^\{\\sf T\}\\text\{$x$\}\_\{i\}\)\\to\\,\\mathbb\{E\}\[h\(Y,U\)\]\.\\end\{split\}Consequently, we haveLaw⁡\(Y,U\)∈ℱ1,α,φalg\\operatorname\{Law\}\(Y,U\)\\in\\mathscr\{F\}\_\{1,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}\.

### B\.3Auxiliary lemmas

###### Lemma B\.2\.

Let\(Y,G,Z\)\(Y,G,Z\)be as defined in[1](https://arxiv.org/html/2606.28573#Thmdefn1)\. ForC∈ℝk×mC\\in\\mathbb\{R\}^\{k\\times m\}, define

gBayes​\(C\)=𝔼​\[𝔼​\[G−C​Z\|C𝖳​G\+Z,Y\]​𝔼​\[G−C​Z\|C𝖳​G\+Z,Y\]𝖳\]\.g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\)=\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-CZ\\big\|C^\{\\sf T\}G\+Z,Y\\right\]\\mathbb\{E\}\\left\[G\-CZ\\big\|C^\{\\sf T\}G\+Z,Y\\right\]^\{\\sf T\}\\right\]\.Then,gBayes​\(C\)g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\)only depends onC​C𝖳CC^\{\\sf T\}, and is increasing inC​C𝖳CC^\{\\sf T\}in the following sense:

C1​C1𝖳⪯C2​C2𝖳⟹gBayes​\(C1\)⪯gBayes​\(C2\)\.C\_\{1\}C\_\{1\}^\{\\sf T\}\\preceq C\_\{2\}C\_\{2\}^\{\\sf T\}\\implies g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\_\{1\}\)\\preceq g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\_\{2\}\)\.As a consequence,Γ∗=limt→∞Γt\\Gamma\_\{\*\}=\\lim\_\{t\\to\\infty\}\\Gamma\_\{t\}exists for the sequence\{Γt\}t=0∞\\\{\\Gamma\_\{t\}\\\}\_\{t=0\}^\{\\infty\}defined in[Eq\.19](https://arxiv.org/html/2606.28573#S3.E19)\.

###### Proof\.

It suffices to show that for anyβ∈ℝk\\beta\\in\\mathbb\{R\}^\{k\}, the function

gBayes​\(C,β\)=\\displaystyle g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C,\\beta\)=\\,β𝖳​𝔼​\[𝔼​\[G−C​Z\|C𝖳​G\+Z,Y\]​𝔼​\[G−C​Z\|C𝖳​G\+Z,Y\]𝖳\]​β\\displaystyle\\beta^\{\\sf T\}\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-CZ\\big\|C^\{\\sf T\}G\+Z,Y\\right\]\\mathbb\{E\}\\left\[G\-CZ\\big\|C^\{\\sf T\}G\+Z,Y\\right\]^\{\\sf T\}\\right\]\\beta=\\displaystyle=\\,𝔼​\[𝔼​\[β𝖳​\(G−C​Z\)\|C𝖳​G\+Z,Y\]2\]\\displaystyle\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\(G\-CZ\)\\big\|C^\{\\sf T\}G\+Z,Y\\right\]^\{2\}\\right\]only depends onC​C𝖳CC^\{\\sf T\}, and is non\-decreasing inC​C𝖳CC^\{\\sf T\}\. To this end, we first establish an equivalent definition for this mapping\. According to Cauchy\-Schwarz inequality, we have

gBayes​\(C,β\)=max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​\(G−C​Z\)⋅f​\(C𝖳​G\+Z,Y\)\]2,g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C,\\beta\)=\\,\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\(G\-CZ\)\\cdot f\(C^\{\\sf T\}G\+Z,Y\)\\right\]^\{2\},whereffcan depend on some other randomness as well\. DenoteZC=C𝖳​G\+ZZ\_\{C\}=C^\{\\sf T\}G\+Zand recallY=φ​\(G,ε\)Y=\\varphi\(G,\\varepsilon\)\. We know that\(ZC𝖳,G𝖳\)𝖳\(Z\_\{C\}^\{\\sf T\},G^\{\\sf T\}\)^\{\\sf T\}is a\(k\+m\)\(k\+m\)\-dimensional Gaussian random vector with mean0and covariance matrix equal to

ΣC=\[Im\+C𝖳​CC𝖳CIk\]\.\\Sigma\_\{C\}=\\begin\{bmatrix\}I\_\{m\}\+C^\{\\sf T\}C&C^\{\\sf T\}\\\\ C&I\_\{k\}\\end\{bmatrix\}\.Stein’s lemma then implies that

𝔼​\[\(ZC𝖳,G𝖳\)𝖳​f​\(C𝖳​G\+Z,Y\)\]=𝔼​\[\(ZC𝖳,G𝖳\)𝖳​f​\(ZC,φ​\(G,ε\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\(Z\_\{C\}^\{\\sf T\},G^\{\\sf T\}\)^\{\\sf T\}f\(C^\{\\sf T\}G\+Z,Y\)\\right\]=\\,\\mathbb\{E\}\\left\[\(Z\_\{C\}^\{\\sf T\},G^\{\\sf T\}\)^\{\\sf T\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)\\right\]=\\displaystyle=\\,ΣC​𝔼​\[\(∇ZCf​\(ZC,φ​\(G,ε\)\)𝖳,∇Gf​\(ZC,φ​\(G,ε\)\)𝖳\)𝖳\]\.\\displaystyle\\Sigma\_\{C\}\\mathbb\{E\}\\left\[\\left\(\\nabla\_\{Z\_\{C\}\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)^\{\\sf T\},\\nabla\_\{G\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)^\{\\sf T\}\\right\)^\{\\sf T\}\\right\]\.Therefore,

𝔼​\[\(G−C​Z\)⋅f​\(C𝖳​G\+Z,Y\)\]=\[−CIk\+C​C𝖳\]​𝔼​\[\(ZC𝖳,G𝖳\)𝖳​f​\(C𝖳​G\+Z,Y\)\]\\displaystyle\\mathbb\{E\}\\left\[\(G\-CZ\)\\cdot f\(C^\{\\sf T\}G\+Z,Y\)\\right\]=\\,\\begin\{bmatrix\}\-C&I\_\{k\}\+CC^\{\\sf T\}\\end\{bmatrix\}\\mathbb\{E\}\\left\[\(Z\_\{C\}^\{\\sf T\},G^\{\\sf T\}\)^\{\\sf T\}f\(C^\{\\sf T\}G\+Z,Y\)\\right\]=\\displaystyle=\\,\[−CIk\+C​C𝖳\]​ΣC​𝔼​\[\(∇ZCf​\(ZC,φ​\(G,ε\)\)𝖳,∇Gf​\(ZC,φ​\(G,ε\)\)𝖳\)𝖳\]\\displaystyle\\begin\{bmatrix\}\-C&I\_\{k\}\+CC^\{\\sf T\}\\end\{bmatrix\}\\Sigma\_\{C\}\\mathbb\{E\}\\left\[\\left\(\\nabla\_\{Z\_\{C\}\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)^\{\\sf T\},\\nabla\_\{G\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)^\{\\sf T\}\\right\)^\{\\sf T\}\\right\]=\\displaystyle=\\,\[0Ik\]​𝔼​\[\(∇ZCf​\(ZC,φ​\(G,ε\)\)𝖳,∇Gf​\(ZC,φ​\(G,ε\)\)𝖳\)𝖳\]=𝔼​\[∇Gf​\(ZC,φ​\(G,ε\)\)\]\.\\displaystyle\\begin\{bmatrix\}0&I\_\{k\}\\end\{bmatrix\}\\mathbb\{E\}\\left\[\\left\(\\nabla\_\{Z\_\{C\}\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)^\{\\sf T\},\\nabla\_\{G\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)^\{\\sf T\}\\right\)^\{\\sf T\}\\right\]=\\mathbb\{E\}\\left\[\\nabla\_\{G\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)\\right\]\.As a consequence,

gBayes​\(C,β\)=max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(ZC,φ​\(G,ε\)\)\]2\.g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C,\\beta\)=\\,\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(Z\_\{C\},\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}\.Based on this definition, we can easily show thatgBayes​\(C,β\)g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C,\\beta\)only depends onC​C𝖳CC^\{\\sf T\}\. In fact, letC1C\_\{1\}andC2C\_\{2\}be twok×mk\\times mmatrices such thatC1​C1𝖳=C2​C2𝖳C\_\{1\}C\_\{1\}^\{\\sf T\}=C\_\{2\}C\_\{2\}^\{\\sf T\}, then there exists an orthogonal matrixO∈ℝm×mO\\in\\mathbb\{R\}^\{m\\times m\}such thatC2=C1​O𝖳C\_\{2\}=C\_\{1\}O^\{\\sf T\}\. Hence,

gBayes​\(C2,β\)=\\displaystyle g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\_\{2\},\\beta\)=\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(O​C1𝖳​G\+Z,φ​\(G,ε\)\)\]2\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(OC\_\{1\}^\{\\sf T\}G\+Z,\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,max𝔼​\[fO2\]≤1⁡𝔼​\[β𝖳​∇GfO​\(C1𝖳​G\+O𝖳​Z,φ​\(G,ε\)\)\]2\\displaystyle\\max\_\{\\mathbb\{E\}\[f\_\{O\}^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\_\{O\}\(C\_\{1\}^\{\\sf T\}G\+O^\{\\sf T\}Z,\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}=\(i​i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(ii\)\}\}\{\{=\}\}\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(C1𝖳​G\+Z,φ​\(G,ε\)\)\]2=gBayes​\(C1,β\),\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(C\_\{1\}^\{\\sf T\}G\+Z,\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}=g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\_\{1\},\\beta\),where in\(i\)\(i\)we definefO​\(x,y\)=f​\(O​x,y\)f\_\{O\}\(x,y\)=f\(Ox,y\), and\(i​i\)\(ii\)follows from the fact that\(C1𝖳​G\+O𝖳​Z,G\)=d\(C1𝖳​G\+Z,G\)\(C\_\{1\}^\{\\sf T\}G\+O^\{\\sf T\}Z,G\)\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}\(C\_\{1\}^\{\\sf T\}G\+Z,G\)\. To establish the monotonicity ofgBayes​\(⋅,β\)g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(\\cdot,\\beta\), note that it can be rewritten as

gBayes​\(C,β\)=\\displaystyle g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C,\\beta\)=\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(C𝖳​G\+Z,φ​\(G,ε\)\)\]2\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(C^\{\\sf T\}G\+Z,\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}=\\displaystyle=\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(G\+\(C​C𝖳\)−1​C​Z,φ​\(G,ε\)\)\]2\.\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(G\+\(CC^\{\\sf T\}\)^\{\-1\}CZ,\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}\.ConsiderC1,C2∈ℝk×mC\_\{1\},C\_\{2\}\\in\\mathbb\{R\}^\{k\\times m\}satisfyingC1​C1𝖳⪯C2​C2𝖳C\_\{1\}C\_\{1\}^\{\\sf T\}\\preceq C\_\{2\}C\_\{2\}^\{\\sf T\}\. DenoteUi=\(Ci​Ci𝖳\)−1​CiU\_\{i\}=\(C\_\{i\}C\_\{i\}^\{\\sf T\}\)^\{\-1\}C\_\{i\}fori=1,2i=1,2, then we haveU2​U2𝖳⪯U1​U1𝖳U\_\{2\}U\_\{2\}^\{\\sf T\}\\preceq U\_\{1\}U\_\{1\}^\{\\sf T\}\. Therefore, we can write

U1Z=U2Z\+εU,εU⟂⟂\(G,Z,ε\)\.U\_\{1\}Z=U\_\{2\}Z\+\\varepsilon\_\{U\},\\quad\\varepsilon\_\{U\}\\perp\\\!\\\!\\\!\\perp\(G,Z,\\varepsilon\)\.This decomposition further implies that

gBayes​\(C1,β\)=\\displaystyle g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\_\{1\},\\beta\)=\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(G\+U1​Z,φ​\(G,ε\)\)\]2\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(G\+U\_\{1\}Z,\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}=\\displaystyle=\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(G\+U2​Z\+εU,φ​\(G,ε\)\)\]2\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(G\+U\_\{2\}Z\+\\varepsilon\_\{U\},\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}≤\\displaystyle\\leq\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(G\+U2​Z,φ​\(G,ε\),εU\)\]2\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(G\+U\_\{2\}Z,\\varphi\(G,\\varepsilon\),\\varepsilon\_\{U\}\)\\right\]^\{2\}=\\displaystyle=\\,max𝔼​\[f2\]≤1⁡𝔼​\[β𝖳​∇Gf​\(G\+U2​Z,φ​\(G,ε\)\)\]2=gBayes​\(C2,β\),\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\beta^\{\\sf T\}\\nabla\_\{G\}f\(G\+U\_\{2\}Z,\\varphi\(G,\\varepsilon\)\)\\right\]^\{2\}=g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(C\_\{2\},\\beta\),which proves the monotonicity ofgBayes​\(⋅,β\)g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(\\cdot,\\beta\)\. Finally, let the sequence\{Γt\}t=0∞\\\{\{\\Gamma\}\_\{t\}\\\}\_\{t=0\}^\{\\infty\}be defined as in[Eq\.19](https://arxiv.org/html/2606.28573#S3.E19); then

Γ1=α​gBayes​\(0\),Γt\+1=α​gBayes​\(Γt1/2\)\.\{\\Gamma\}\_\{1\}=\\alpha g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(0\),\\quad\{\\Gamma\}\_\{t\+1\}=\\alpha g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\big\(\{\\Gamma\}\_\{t\}^\{1/2\}\\big\)\.Using induction, one can easily show that\{Γt\}t=0∞\\\{\{\\Gamma\}\_\{t\}\\\}\_\{t=0\}^\{\\infty\}is a non\-decreasing sequence\. Below, we consider two cases:

- \(a\)𝒞Bayes=∅\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\emptyset, i\.e\., there does not existΓ∈𝒮\+k\{\\Gamma\}\\in\{\\mathcal\{S\}\}\_\{\+\}^\{k\}such thatα​gBayes​\(Γ1/2\)=Γ\\alpha g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(\{\\Gamma\}^\{1/2\}\)=\{\\Gamma\}\. In this case,Γt\{\\Gamma\}\_\{t\}will diverge to infinity\. Otherwise, if it converges to some finiteΓ∗\{\\Gamma\}\_\{\*\},Γ∗\{\\Gamma\}\_\{\*\}must satisfyα​gBayes​\(Γ∗1/2\)=Γ∗\\alpha g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(\{\\Gamma\}\_\{\*\}^\{1/2\}\)=\{\\Gamma\}\_\{\*\}, a contradiction\.
- \(b\)𝒞Bayes≠∅\\mathscr\{C\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\neq\\emptyset, i\.e\., there exists someΓ∈𝒮\+k\{\\Gamma\}\\in\{\\mathcal\{S\}\}\_\{\+\}^\{k\}such thatα​gBayes​\(Γ1/2\)=Γ\\alpha g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(\{\\Gamma\}^\{1/2\}\)=\{\\Gamma\}\. In this case,Γt\{\\Gamma\}\_\{t\}will converge to one suchΓ\{\\Gamma\}\. Denote this limit byΓ∗\{\\Gamma\}\_\{\*\}\. ThenΓ∗\{\\Gamma\}\_\{\*\}is the smallest \(with respect to the Loewner order\) of allΓ∈𝒮\+k\{\\Gamma\}\\in\{\\mathcal\{S\}\}\_\{\+\}^\{k\}such thatα​gBayes​\(Γ1/2\)=Γ\\alpha g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(\{\\Gamma\}^\{1/2\}\)=\{\\Gamma\}, since it is the limit of a fixed\-point iteration starting from0\.

Combining these two cases completes the proof of[B\.2](https://arxiv.org/html/2606.28573#A2.Thmlem2)\. ∎

###### Lemma B\.3\.

For the sequence\{ct\}t=0∞\\\{c\_\{t\}\\\}\_\{t=0\}^\{\\infty\}defined in[Eq\.74](https://arxiv.org/html/2606.28573#A2.E74), we havecBayes=limt→∞ctc\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\lim\_\{t\\to\\infty\}c\_\{t\}\.

###### Proof\.

Similar to the case of generalkkandmm, let us define forx≥0x\\geq 0:

gBayes​\(x\)=𝔼​\[𝔼​\[G−x​Z\|x​G\+Z,Y\]2\]\.g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(x\)=\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-xZ\|xG\+Z,Y\\right\]^\{2\}\\right\]\.Then, the recursion \([74](https://arxiv.org/html/2606.28573#A2.E74)\) becomesct\+1=α1/2​gBayes​\(ct\)1/2c\_\{t\+1\}=\\alpha^\{1/2\}g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(c\_\{t\}\)^\{1/2\}\. Specializing the conclusions of[B\.2](https://arxiv.org/html/2606.28573#A2.Thmlem2)to the settingm=k=1m=k=1, we know thatgBayesg\_\{\\mbox\{\\tiny\\rm Bayes\}\}is monotone increasing on\[0,∞\)\[0,\\infty\)\. Furthermore, sincec1=α1/2​gBayes​\(0\)1/2c\_\{1\}=\\alpha^\{1/2\}g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(0\)^\{1/2\}, it follows that the sequence\{ct\}t=1∞\\\{c\_\{t\}\\\}\_\{t=1\}^\{\\infty\}is increasing\. Similarly, we consider the following two situations:

- \(a\)The mappingc↦α1/2​gBayes​\(c\)1/2c\\mapsto\\alpha^\{1/2\}g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(c\)^\{1/2\}does not have any fixed point\. In this case, we havecBayes=∞c\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\infty, andct→∞c\_\{t\}\\to\\inftyast→∞t\\to\\infty\.
- \(b\)The mappingc↦α1/2​gBayes​\(c\)1/2c\\mapsto\\alpha^\{1/2\}g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(c\)^\{1/2\}has a fixed point\. In this case,cBayes=limt→∞ctc\_\{\\mbox\{\\tiny\\rm Bayes\}\}=\\lim\_\{t\\to\\infty\}c\_\{t\}exists and is the first fixed point ofgBayesg\_\{\\mbox\{\\tiny\\rm Bayes\}\}\.

This completes the proof of[B\.3](https://arxiv.org/html/2606.28573#A2.Thmlem3)\. ∎

## Appendix CAppendix for[Section4](https://arxiv.org/html/2606.28573#S4)

### C\.1Proof of[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1)and[4\.3](https://arxiv.org/html/2606.28573#S4.Thmlem3)

###### Proof of[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1)\.

We first prove\(i\)\(i\)\. To this end, we invoke[C\.2](https://arxiv.org/html/2606.28573#A3.Thmlem2)and the condition \([78](https://arxiv.org/html/2606.28573#A3.E78)\) for replica symmetry\. It suffices to verify \([78](https://arxiv.org/html/2606.28573#A3.E78)\) for some smallc∗c\_\{\*\}whenα\\alphais large\. Note that for small enoughc∗c\_\{\*\}, the conditionhc∗=hh\_\{c\_\{\*\}\}=his automatically satisfied by definition ofhc∗h\_\{c\_\{\*\}\}\. It then suffices to show that there existsc∗=c∗​\(α\)→0c\_\{\*\}=c\_\{\*\}\(\\alpha\)\\to 0asα→∞\\alpha\\to\\infty, such that

𝔼​\[\(∂xfY,c∗​\(1,X1\)\)2\]=1α​c∗2​\(1−r2\),𝔼​\[\(∂x2fY,c∗​\(1,X1\)\)2\]≤1α​c∗2\.\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]=\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(1\-r^\{2\}\),\\quad\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}^\{2\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\\leq\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\.By definition and Lipschitzness ofhh, we know that

fy,c​\(1,x\)=supu∈ℝ\{h​\(y,x\+u\)−u22​c\}=supu∈ℝ\{h​\(y,x\+c​u\)−u22\}f\_\{y,c\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,x\+u\\right\)\-\\frac\{u^\{2\}\}\{2c\}\\right\\\}=\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,x\+\\sqrt\{c\}u\\right\)\-\\frac\{u^\{2\}\}\{2\}\\right\\\}converges toh​\(y,x\)h\(y,x\)asc→0c\\to 0\. Further, we can show that∂xfy,c​\(1,x\)→∂xh​\(y,x\)\\partial\_\{x\}f\_\{y,c\}\(1,x\)\\to\\partial\_\{x\}h\(y,x\)and∂x2fy,c​\(1,x\)→∂x2h​\(y,x\)\\partial\_\{x\}^\{2\}f\_\{y,c\}\(1,x\)\\to\\partial\_\{x\}^\{2\}h\(y,x\)\. Therefore, as long asc∗​\(α\)→0c\_\{\*\}\(\\alpha\)\\to 0and1−r2≤C/α1\-r^\{2\}\\leq C/\\alpha, we have

limα→∞𝔼​\[\(∂xfY,c∗​\(1,X1\)\)2\]=𝔼​\[\(∂xh​\(Y,G\)\)2\],\\displaystyle\\lim\_\{\\alpha\\to\\infty\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]=\\,\\mathbb\{E\}\\left\[\(\\partial\_\{x\}h\(Y,G\)\)^\{2\}\\right\],limα→∞𝔼​\[\(∂x2fY,c∗​\(1,X1\)\)2\]=𝔼​\[\(∂x2h​\(Y,G\)\)2\]\.\\displaystyle\\lim\_\{\\alpha\\to\\infty\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}^\{2\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]=\\,\\mathbb\{E\}\\left\[\(\\partial\_\{x\}^\{2\}h\(Y,G\)\)^\{2\}\\right\]\.This means that we can choose

c∗​\(α\)∝1−r2α​𝔼​\[\(∂xh​\(Y,G\)\)2\]c\_\{\*\}\(\\alpha\)\\propto\\sqrt\{\\frac\{1\-r^\{2\}\}\{\\alpha\\mathbb\{E\}\\left\[\(\\partial\_\{x\}h\(Y,G\)\)^\{2\}\\right\]\}\}to satisfy[Eq\.78](https://arxiv.org/html/2606.28573#A3.E78)\. This completes the proof of\(i\)\(i\)\.

We next prove\(i​i\)\(ii\)\. Note that for anyC\>0C\>0:

supr2<1−C/αinf\(μ,c\)∈𝒰×ℝ\>0𝖥​\(μ,c,r\)≤supr2<1−C/αinfc\>0𝖥​\(0,c,r\)\\displaystyle\\sup\_\{r^\{2\}<1\-C/\\alpha\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\(\\mu,c,r\)\\leq\\,\\sup\_\{r^\{2\}<1\-C/\\alpha\}\\inf\_\{c\>0\}\\mathsf\{F\}\(0,c,r\)=\\displaystyle=\\,supr2<1−C/αinfc\>0\{𝔼Y,G​\[fY,c​\(1,r​G\+1−r2​Z\)\]\+12​α​c​\(1−r2\)\}\\displaystyle\\sup\_\{r^\{2\}<1\-C/\\alpha\}\\inf\_\{c\>0\}\\left\\\{\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,c\}\\left\(1,rG\+\\sqrt\{1\-r^\{2\}\}Z\\right\)\\right\]\+\\frac\{1\}\{2\\alpha c\}\(1\-r^\{2\}\)\\right\\\}=\\displaystyle=\\,supr2<1−C/αsup𝔼​\[U2\]≤1/α𝔼​\[h​\(Y,r​G\+1−r2​\(Z\+U\)\)\],\\displaystyle\\sup\_\{r^\{2\}<1\-C/\\alpha\}\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha\}\\mathbb\{E\}\\left\[h\\left\(Y,rG\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\\right\],where the last equality follows from the definition offy,cf\_\{y,c\}and Lagrange duality\. Define

vα​\(r\)=sup𝔼​\[U2\]≤1/α𝔼​\[h​\(Y,r​G\+1−r2​\(Z\+U\)\)\]\.v\_\{\\alpha\}\(r\)=\\,\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha\}\\mathbb\{E\}\\left\[h\\left\(Y,rG\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\\right\]\.Sincehhis Lipschitz, we obtain that

\|vα​\(r\)−v​\(r\)\|≤L​1−r2α\.\\left\|v\_\{\\alpha\}\(r\)\-v\(r\)\\right\|\\leq L\\sqrt\{\\frac\{1\-r^\{2\}\}\{\\alpha\}\}\.Without loss of generality, we assume thatr1=1r\_\{1\}=1\. By our assumptions onvv, it follows that

vα​\(r\)−vα​\(1\)≤\\displaystyle v\_\{\\alpha\}\(r\)\-v\_\{\\alpha\}\(1\)\\leq\\,v​\(r\)−v​\(1\)\+L​1−r2α≤−min⁡\(ε0,c0​\(1−r\)\)\+L​1−r2α\\displaystyle v\(r\)\-v\(1\)\+L\\sqrt\{\\frac\{1\-r^\{2\}\}\{\\alpha\}\}\\leq\-\\min\\left\(\\varepsilon\_\{0\},c\_\{0\}\(1\-r\)\\right\)\+L\\sqrt\{\\frac\{1\-r^\{2\}\}\{\\alpha\}\}≤\\displaystyle\\leq\\,−c0​\(1−r\)\+L​2​\(1−r\)α=−1−r​\(c0​1−r−L​2α\)\.\\displaystyle\-c\_\{0\}\(1\-r\)\+L\\sqrt\{\\frac\{2\(1\-r\)\}\{\\alpha\}\}=\-\\sqrt\{1\-r\}\\left\(c\_\{0\}\\sqrt\{1\-r\}\-L\\sqrt\{\\frac\{2\}\{\\alpha\}\}\\right\)\.Note that1−r2\>C/α1\-r^\{2\}\>C/\\alphaimplies1−r\>C/2​α1\-r\>C/2\\alpha\. Therefore, for all suchrr, we have

vα​\(r\)−vα​\(1\)≤−1−r​\(c0​C2​α−L​2α\)<0\\displaystyle v\_\{\\alpha\}\(r\)\-v\_\{\\alpha\}\(1\)\\leq\\,\-\\sqrt\{1\-r\}\\left\(c\_\{0\}\\sqrt\{\\frac\{C\}\{2\\alpha\}\}\-L\\sqrt\{\\frac\{2\}\{\\alpha\}\}\\right\)<0forC\>4​L2/c02C\>4L^\{2\}/c\_\{0\}^\{2\}\. This implies that

supr2<1−C/αinf\(μ,c\)∈𝒰×ℝ\>0𝖥​\(μ,c,r\)<𝔼​\[h​\(Y,G\)\]=\(∗\)inf\(μ,c\)∈𝒰×ℝ\>0𝖥​\(μ,c,1\),\\displaystyle\\sup\_\{r^\{2\}<1\-C/\\alpha\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\(\\mu,c,r\)<\\,\\mathbb\{E\}\\left\[h\(Y,G\)\\right\]\\stackrel\{\{\\scriptstyle\(\*\)\}\}\{\{=\}\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\(\\mu,c,1\),where\(∗\)\(\*\)follows from our conclusion in part\(i\)\(i\)\. Hence,r∗r\_\{\*\}must satisfy1−r∗2≤C/α1\-r\_\{\*\}^\{2\}\\leq C/\\alpha, completing the proof of part\(i​i\)\(ii\)\.[Eq\.29](https://arxiv.org/html/2606.28573#S4.E29)then follows naturally\. ∎

###### Proof of[4\.3](https://arxiv.org/html/2606.28573#S4.Thmlem3)\.

We first show that\(r∗,1\)∈ABayes\(r\_\{\*\},1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}for sufficiently largeα\\alpha\. By[C\.1](https://arxiv.org/html/2606.28573#A3.Thmlem1), we know thatr∗≤rBayesr\_\{\*\}\\leq r\_\{\\mbox\{\\tiny\\rm Bayes\}\}for sufficiently largeα\\alpha, which implies\(r∗,1\)∈ABayes\(r\_\{\*\},1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. Therefore, the correlationr∗r\_\{\*\}with the true signal𝒘∗\\text\{$w$\}\_\{\*\}is achievable by our two\-stage AMP algorithm, and so is the limiting test error \([31](https://arxiv.org/html/2606.28573#S4.E31)\)\. Invoking[3\.1](https://arxiv.org/html/2606.28573#S3.Thmlem1)withr=r∗r=r\_\{\*\},q=1q=1, andh=−ℓh=\-\\ell, we know that the minimum limiting training error achieved by our two\-stage AMP algorithm is given by

inf𝔼​\[ℓ​\(Y,Zr∗,1\+1α​F​\(Zr∗,1,Y\)\)\],subject to​Fis an\(r∗,1\)\-contraction,\\inf\\;\\mathbb\{E\}\\left\[\\ell\\left\(Y,\\,Z\_\{r\_\{\*\},1\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r\_\{\*\},1\},Y\\right\)\\right\)\\right\],\\quad\\mbox\{subject to\}\\,\\,\\text\{$F$ is an $\(r\_\{\*\},1\)$\-contraction\},whereZr∗,1=r∗​G\+1−r∗2​ZZ\_\{r\_\{\*\},1\}=r\_\{\*\}G\+\\sqrt\{1\-r\_\{\*\}^\{2\}\}Z\. Further, the limiting training error of ERM given in[Eq\.30](https://arxiv.org/html/2606.28573#S4.E30)can be rewritten as

inf𝔼​\[U2\]≤1/α𝔼​\[ℓ​\(Y,r∗​G\+1−r∗2​\(Z\+U\)\)\]\\displaystyle\\inf\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha\}\\mathbb\{E\}\\left\[\\ell\\left\(Y,r\_\{\*\}G\+\\sqrt\{1\-r\_\{\*\}^\{2\}\}\(Z\+U\)\\right\)\\right\]=\\displaystyle=\\,inf𝔼​\[F​\(Zr∗,1,Y\)2\]≤α​\(1−r∗2\)𝔼​\[ℓ​\(Y,Zr∗,1\+1α​F​\(Zr∗,1,Y\)\)\]\.\\displaystyle\\inf\_\{\\mathbb\{E\}\[F\(Z\_\{r\_\{\*\},1\},Y\)^\{2\}\]\\leq\\alpha\(1\-r\_\{\*\}^\{2\}\)\}\\mathbb\{E\}\\left\[\\ell\\left\(Y,Z\_\{r\_\{\*\},1\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r\_\{\*\},1\},Y\\right\)\\right\)\\right\]\.Recallc∗c\_\{\*\}from the proof of[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1)\. According to that proof, we know that\(r∗,c∗\)\(r\_\{\*\},c\_\{\*\}\)satisfies[Eq\.78](https://arxiv.org/html/2606.28573#A3.E78)withr=r∗r=r\_\{\*\}\. Further, by definition offy,c​\(t,x\)f\_\{y,c\}\(t,x\)and Lagrange duality \(cf\. Proof of[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1)\), we know that the above infimum is achieved at

F​\(Zr∗,1,Y\)=α​c∗​∂xfY,c∗​\(1,Zr∗,1\)\.F\\left\(Z\_\{r\_\{\*\},1\},Y\\right\)=\\,\\alpha c\_\{\*\}\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\\big\(1,Z\_\{r\_\{\*\},1\}\\big\)\.It then suffices to verify thatFFis indeed an\(r∗,1\)\(r\_\{\*\},1\)\-contraction\. To this end, we note that the inequality constraint

1α​𝔼​\[∂F∂Zr∗,1​\(Zr∗,1,φ​\(G,ε\)\)2\]≤1\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{r\_\{\*\},1\}\}\\left\(Z\_\{r\_\{\*\},1\},\\varphi\(G,\\varepsilon\)\\right\)^\{2\}\\right\]\\leq 1is already verified in the proof of[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1), as part of condition \([78](https://arxiv.org/html/2606.28573#A3.E78)\) for replica symmetry\. Further, the equality constraint

r∗=𝔼​\[∂F∂G​\(Zr∗,1,φ​\(G,ε\)\)\]r\_\{\*\}=\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial G\}\\left\(Z\_\{r\_\{\*\},1\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]can be verified by using the envelope theorem to compute the first\-order condition for[Eq\.29](https://arxiv.org/html/2606.28573#S4.E29)with respect tor∗r\_\{\*\}, in a way similar to the proof of[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1)\(c\)\. This completes the proof of[4\.3](https://arxiv.org/html/2606.28573#S4.Thmlem3)\. ∎

### C\.2Estimation errors of ERM and Bayes AMP

We need the following lemma regarding the asymptotic estimation errors of ERM and Bayes optimal AMP asα→∞\\alpha\\to\\infty:

###### Lemma C\.1\.

Letℓ=−h\\ell=\-h\. Recallr∗r\_\{\*\}from[Eq\.29](https://arxiv.org/html/2606.28573#S4.E29)and letrBayesr\_\{\\mbox\{\\tiny\\rm Bayes\}\}be the largestr∈\[−1,1\]r\\in\[\-1,1\]such that\(r,1\)∈ABayes\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. Then, asα→∞\\alpha\\to\\infty, we have

𝖾𝗋𝗋ERM:=\\displaystyle\\mathsf\{err\}\_\{\\rm ERM\}:=\\,2​\(1−r∗\)∼𝔼​\[\(∂xℓ​\(Y,G\)\)2\]α​𝔼​\[∂y​xℓ​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]2,\\displaystyle\\sqrt\{2\(1\-r\_\{\*\}\)\}\\sim\\sqrt\{\\frac\{\\mathbb\{E\}\[\(\\partial\_\{x\}\\ell\(Y,G\)\)^\{2\}\]\}\{\\alpha\\mathbb\{E\}\[\\partial\_\{yx\}\\ell\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\]^\{2\}\}\},\(75\)𝖾𝗋𝗋Bayes:=\\displaystyle\\mathsf\{err\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}:=\\,2​\(1−rBayes\)∼minf:ℝ2→ℝ⁡𝔼​\[f​\(Y,G\)2\]α​𝔼​\[∂yf​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]2\.\\displaystyle\\sqrt\{2\(1\-r\_\{\\mbox\{\\tiny\\rm Bayes\}\}\)\}\\sim\\min\_\{f:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}\}\\sqrt\{\\frac\{\\mathbb\{E\}\[f\(Y,G\)^\{2\}\]\}\{\\alpha\\mathbb\{E\}\[\\partial\_\{y\}f\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\]^\{2\}\}\}\.\(76\)

###### Proof of[C\.1](https://arxiv.org/html/2606.28573#A3.Thmlem1)\.

We first compute the asymptotics of𝖾𝗋𝗋ERM\\mathsf\{err\}\_\{\\rm ERM\}asα→∞\\alpha\\to\\infty\. By[4\.1](https://arxiv.org/html/2606.28573#S4.Thmlem1)and[Eq\.29](https://arxiv.org/html/2606.28573#S4.E29), we have

r∗=\\displaystyle r\_\{\*\}=\\,arg⁡max1−r2≤C/α​infc\>0\{𝔼Y,G​\[fY,c​\(1,r​G\+1−r2​Z\)\]\+12​α​c​\(1−r2\)\}\\displaystyle\\arg\\max\_\{1\-r^\{2\}\\leq C/\\alpha\}\\inf\_\{c\>0\}\\left\\\{\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,c\}\\left\(1,rG\+\\sqrt\{1\-r^\{2\}\}Z\\right\)\\right\]\+\\frac\{1\}\{2\\alpha c\}\(1\-r^\{2\}\)\\right\\\}=\\displaystyle=\\,arg⁡max1−r2≤C/α​sup𝔼​\[U2\]≤1/α𝔼​\[h​\(Y,r​G\+1−r2​\(Z\+U\)\)\]\.\\displaystyle\\arg\\max\_\{1\-r^\{2\}\\leq C/\\alpha\}\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha\}\\mathbb\{E\}\\left\[h\\left\(Y,rG\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\\right\]\.Without loss of generality, we assumer1=1r\_\{1\}=1, and perform local Taylor expansions around11:

𝔼​\[h​\(Y,r​G\+1−r2​\(Z\+U\)\)\]=𝔼​\[h​\(Y,G\+\(r−1\)​G\+1−r2​\(Z\+U\)\)\]\\displaystyle\\mathbb\{E\}\\left\[h\\left\(Y,rG\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\\right\]=\\,\\mathbb\{E\}\\left\[h\\left\(Y,G\+\(r\-1\)G\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\\right\]=\\displaystyle=\\,𝔼​\[h​\(Y,G\)\+∂xh​\(Y,G\)​\(\(r−1\)​G\+1−r2​\(Z\+U\)\)\+12​∂x2h​\(Y,G\)​\(1−r2\)​\(Z\+U\)2\]\+o​\(1α\)\\displaystyle\\mathbb\{E\}\\left\[h\(Y,G\)\+\\partial\_\{x\}h\(Y,G\)\\left\(\(r\-1\)G\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}h\(Y,G\)\(1\-r^\{2\}\)\(Z\+U\)^\{2\}\\right\]\+o\\left\(\\frac\{1\}\{\\alpha\}\\right\)=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,𝔼​\[h​\(Y,G\)\]−tα​𝔼​\[G​∂xh​\(Y,G\)\]\+2​tα​𝔼​\[U​∂xh​\(Y,G\)\]\+tα​𝔼​\[∂x2h​\(Y,G\)\]\+o​\(1α\),\\displaystyle\\mathbb\{E\}\\left\[h\(Y,G\)\\right\]\-\\frac\{t\}\{\\alpha\}\\mathbb\{E\}\\left\[G\\partial\_\{x\}h\(Y,G\)\\right\]\+\\sqrt\{\\frac\{2t\}\{\\alpha\}\}\\mathbb\{E\}\\left\[U\\partial\_\{x\}h\(Y,G\)\\right\]\+\\frac\{t\}\{\\alpha\}\\mathbb\{E\}\\left\[\\partial\_\{x\}^\{2\}h\(Y,G\)\\right\]\+o\\left\(\\frac\{1\}\{\\alpha\}\\right\),where in\(i\)\(i\)we setr=1−t/αr=1\-t/\\alpha, and use the fact thatZZis independent of\(Y,G\)\(Y,G\)\. Maximizing the above expression overUUsubject to𝔼​\[U2\]≤1/α\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha, we obtain that

sup𝔼​\[U2\]≤1/α𝔼​\[h​\(Y,r​G\+1−r2​\(Z\+U\)\)\]\\displaystyle\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 1/\\alpha\}\\mathbb\{E\}\\left\[h\\left\(Y,rG\+\\sqrt\{1\-r^\{2\}\}\(Z\+U\)\\right\)\\right\]=\\displaystyle=\\,𝔼​\[h​\(Y,G\)\]−tα​𝔼​\[G​∂xh​\(Y,G\)\]\+2​tα​𝔼​\[\(∂xh​\(Y,G\)\)2\]1/2\+tα​𝔼​\[∂x2h​\(Y,G\)\]\+o​\(1α\)\\displaystyle\\mathbb\{E\}\\left\[h\(Y,G\)\\right\]\-\\frac\{t\}\{\\alpha\}\\mathbb\{E\}\\left\[G\\partial\_\{x\}h\(Y,G\)\\right\]\+\\frac\{\\sqrt\{2t\}\}\{\\alpha\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}h\(Y,G\)\\right\)^\{2\}\\right\]^\{1/2\}\+\\frac\{t\}\{\\alpha\}\\mathbb\{E\}\\left\[\\partial\_\{x\}^\{2\}h\(Y,G\)\\right\]\+o\\left\(\\frac\{1\}\{\\alpha\}\\right\)=\\displaystyle=\\,𝔼​\[h​\(Y,G\)\]\+1α​\(−t​𝔼​\[∂y​xh​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]\+2​t​𝔼​\[\(∂xh​\(Y,G\)\)2\]1/2\)\+o​\(1α\)\.\\displaystyle\\mathbb\{E\}\\left\[h\(Y,G\)\\right\]\+\\frac\{1\}\{\\alpha\}\\left\(\-t\\mathbb\{E\}\\left\[\\partial\_\{yx\}h\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\\right\]\+\\sqrt\{2t\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}h\(Y,G\)\\right\)^\{2\}\\right\]^\{1/2\}\\right\)\+o\\left\(\\frac\{1\}\{\\alpha\}\\right\)\.To ensure that the maximum overttexists, we need to assume

𝔼​\[∂y​xh​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]\>0\.\\mathbb\{E\}\\left\[\\partial\_\{yx\}h\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\\right\]\>\\,0\.Under this assumption, the above expression is maximized at

tERM∗≈𝔼​\[\(∂xh​\(Y,G\)\)2\]2​𝔼​\[∂y​xh​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]2=𝔼​\[\(∂xℓ​\(Y,G\)\)2\]2​𝔼​\[∂y​xℓ​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]2t\_\{\\rm ERM\}^\{\*\}\\approx\\,\\frac\{\\mathbb\{E\}\[\(\\partial\_\{x\}h\(Y,G\)\)^\{2\}\]\}\{2\\mathbb\{E\}\[\\partial\_\{yx\}h\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\]^\{2\}\}=\\frac\{\\mathbb\{E\}\[\(\\partial\_\{x\}\\ell\(Y,G\)\)^\{2\}\]\}\{2\\mathbb\{E\}\[\\partial\_\{yx\}\\ell\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\]^\{2\}\}sinceh=−ℓh=\-\\ell\. This finally gives the asymptotic estimation error for ERM:

𝖾𝗋𝗋ERM∼2​tERM∗α=𝔼​\[\(∂xℓ​\(Y,G\)\)2\]α​𝔼​\[∂y​xℓ​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]2\.\\mathsf\{err\}\_\{\\rm ERM\}\\sim\\sqrt\{\\frac\{2t\_\{\\rm ERM\}^\{\*\}\}\{\\alpha\}\}=\\sqrt\{\\frac\{\\mathbb\{E\}\[\(\\partial\_\{x\}\\ell\(Y,G\)\)^\{2\}\]\}\{\\alpha\\mathbb\{E\}\[\\partial\_\{yx\}\\ell\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\]^\{2\}\}\}\.
For Bayes AMP, the calculation is rather straightforward\. Using the AMP achievability results form=k=1m=k=1developed in[SectionB\.2](https://arxiv.org/html/2606.28573#A2.SS2)and settingq=1q=1, we can show that

limα→∞2​α​\(1−rBayes\)=limα→∞α​\(1−rBayes2\)=1gBayes​\(\+∞\),\\lim\_\{\\alpha\\to\\infty\}2\\alpha\\left\(1\-r\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\right\)=\\,\\lim\_\{\\alpha\\to\\infty\}\\alpha\\left\(1\-r\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\\right\)=\\frac\{1\}\{g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(\+\\infty\)\},where

gBayes​\(x\)=𝔼​\[𝔼​\[G−x​Z\|x​G\+Z,Y\]2\]\.g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(x\)=\\,\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-xZ\|xG\+Z,Y\\right\]^\{2\}\\right\]\.According to the proof of[B\.2](https://arxiv.org/html/2606.28573#A2.Thmlem2), we have

gBayes​\(x\)=\\displaystyle g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(x\)=\\,max𝔼​\[f2\]≤1⁡𝔼​\[\(G−x​Z\)⋅f​\(Y,x​G\+Z\)\]2\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\(G\-xZ\)\\cdot f\(Y,xG\+Z\)\\right\]^\{2\}=\\displaystyle=\\,max𝔼​\[f2\]≤1⁡𝔼​\[∂yf​\(Y,x​G\+Z\)⋅∂φ​\(G,ε\)∂G\]2\.\\displaystyle\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\partial\_\{y\}f\(Y,xG\+Z\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\\right\]^\{2\}\.Therefore,

limx→\+∞gBayes​\(x\)=max𝔼​\[f2\]≤1⁡𝔼​\[∂yf​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]2=maxf:ℝ2→ℝ⁡𝔼​\[∂yf​\(Y,G\)⋅∂φ​\(G,ε\)∂G\]2𝔼​\[f​\(Y,G\)2\],\\lim\_\{x\\to\+\\infty\}g\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(x\)=\\,\\max\_\{\\mathbb\{E\}\[f^\{2\}\]\\leq 1\}\\mathbb\{E\}\\left\[\\partial\_\{y\}f\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\\right\]^\{2\}=\\max\_\{f:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}\}\\frac\{\\mathbb\{E\}\\left\[\\partial\_\{y\}f\(Y,G\)\\cdot\\frac\{\\partial\\varphi\(G,\\varepsilon\)\}\{\\partial G\}\\right\]^\{2\}\}\{\\mathbb\{E\}\[f\(Y,G\)^\{2\}\]\},which leads to the claimed asymptotics for𝖾𝗋𝗋Bayes\\mathsf\{err\}\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. This completes the proof\. ∎

### C\.3Condition for replica symmetry

We present a necessary and sufficient condition for determining whether a Parisi variational problem has a replica symmetric solution\. To this end, we consider the Parisi functional𝖥​\(μ,c,r\)\\mathsf\{F\}\(\\mu,c,r\)defined in[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2), specialized to the casem=k=1m=k=1\.

###### Lemma C\.2\.

For any fixedr∈\[−1,1\]r\\in\[\-1,1\]andc\>0c\>0, letXt=r​G\+Bt−Br2X\_\{t\}=rG\+B\_\{t\}\-B\_\{r^\{2\}\}fort∈\[r2,1\]t\\in\[r^\{2\},1\], andfy,cf\_\{y,c\}solves the heat equation on\[r2,1\]\[r^\{2\},1\]:

∂tfy,c​\(t,x\)\+12​∂x2fy,c​\(t,x\)=0,fy,c​\(1,x\)=supu∈ℝ\{h​\(y,x\+u\)−u22​c\}\.\\partial\_\{t\}f\_\{y,c\}\(t,x\)\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,c\}\(t,x\)=\\,0,\\quad f\_\{y,c\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{h\\left\(y,x\+u\\right\)\-\\frac\{u^\{2\}\}\{2c\}\\right\\\}\.Recall from[8\.3](https://arxiv.org/html/2606.28573#S8.Thmthm3)that

hc​\(y,x\)=conc⁡\(h​\(y,x\)−x22​c\)\+x22​c,gc​\(y,x\)=∂hc​\(y,x\)∂c\.h\_\{c\}\(y,x\)=\\,\\operatorname\{conc\}\\left\(h\(y,x\)\-\\frac\{x^\{2\}\}\{2c\}\\right\)\+\\frac\{x^\{2\}\}\{2c\},\\quad g\_\{c\}\(y,x\)=\\frac\{\\partial h\_\{c\}\(y,x\)\}\{\\partial c\}\.Then, the following are equivalent for anyc∗\>0c\_\{\*\}\>0:

- \(a\)inf\(μ,c\)∈𝒰×ℝ\>0𝖥​\(μ,c,r\)\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\mathsf\{F\}\(\\mu,c,r\)is achieved at\(μ∗=0,c∗\)\(\\mu\_\{\*\}=0,c\_\{\*\}\)\.
- \(b\)c∗c\_\{\*\}satisfies ∫s1\(𝔼​\[\(∂xfY,c∗​\(t,Xt\)\)2\]−1α​c∗2​\(t−r2\)\)​d​t≥0,∀s∈\[r2,1\],\\displaystyle\\int\_\{s\}^\{1\}\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(t,X\_\{t\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(t\-r^\{2\}\)\\right\)\\textup\{d\}t\\geq 0,\\,\\,\\forall s\\in\[r^\{2\},1\],𝔼​\[gc∗​\(Y,M1\)\]\+12​\(𝔼​\[\(∂xfY,c∗​\(1,X1\)\)2\]−1α​c∗2​\(1−r2\)\)=0,\\displaystyle\\mathbb\{E\}\\left\[g\_\{c\_\{\*\}\}\\left\(Y,M\_\{1\}\\right\)\\right\]\+\\frac\{1\}\{2\}\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(1\-r^\{2\}\)\\right\)=0,whereM1=c∗​∂xfY,c∗​\(1,X1\)\+X1M\_\{1\}=c\_\{\*\}\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\+X\_\{1\}\.

###### Proof of[C\.2](https://arxiv.org/html/2606.28573#A3.Thmlem2)\.

We first prove\(b\)\(b\)implies\(a\)\(a\)\. It suffices to show that, for anyμ∈𝒰\\mu\\in\\mathscr\{U\}andc\>0c\>0, one always has𝖥​\(μ,c,r\)≥𝖥​\(0,c∗,r\)\\mathsf\{F\}\(\\mu,c,r\)\\geq\\mathsf\{F\}\(0,c\_\{\*\},r\)\. Using[8\.2](https://arxiv.org/html/2606.28573#S8.Thmthm2), we know that𝖥​\(μ,c,r\)\\mathsf\{F\}\(\\mu,c,r\)is a convex functional ofγ\\gamma, where we recall that

γ​\(t\)=1c\+∫t1μ​\(s\)​d​s\.\\gamma\(t\)=\\frac\{1\}\{c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\}\.For notational convenience, in this proof we recast𝖥​\(μ,c,r\)\\mathsf\{F\}\(\\mu,c,r\)asV​\(γ\)V\(\\gamma\)whenever the above equation holds\. We note that the condition\(μ,c\)∈𝒰×ℝ\>0\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}is equivalent toγ\\gammabeing strictly positive, non\-decreasing, and1/γ1/\\gammabeing concave\. It is easy to show that these properties are preserved under convex combinations ofγ\\gamma\. Therefore, to show thatV​\(γ\)≥V​\(1/c∗\)V\(\\gamma\)\\geq V\(1/c\_\{\*\}\)\(note thatγ≡1/c∗\\gamma\\equiv 1/c\_\{\*\}ifμ=0\\mu=0andc=c∗c=c\_\{\*\}\), it suffices to prove that

dd​u​𝖥​\(u​μ,c∗,r\)\|u=0\+≥0,dd​c​𝖥​\(0,c,r\)\|c=c∗=0\.\\frac\{\\textup\{d\}\}\{\\textup\{d\}u\}\\mathsf\{F\}\\left\(u\\mu,c\_\{\*\},r\\right\)\\bigg\|\_\{u=0^\{\+\}\}\\geq\\,0,\\quad\\frac\{\\textup\{d\}\}\{\\textup\{d\}c\}\\mathsf\{F\}\(0,c,r\)\\bigg\|\_\{c=c\_\{\*\}\}=0\.\(77\)By[8\.3](https://arxiv.org/html/2606.28573#S8.Thmthm3)\(iii\), we have

dd​u​𝖥​\(u​μ,c∗,r\)\|u=0\+=\\displaystyle\\frac\{\\textup\{d\}\}\{\\textup\{d\}u\}\\mathsf\{F\}\\left\(u\\mu,c\_\{\*\},r\\right\)\\bigg\|\_\{u=0^\{\+\}\}=\\,12​∫r21μ​\(t\)​\(𝔼​\[\(∂xfY,c∗​\(t,Xt\)\)2\]−1α​c∗2​\(t−r2\)\)​d​t,\\displaystyle\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\mu\(t\)\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(t,X\_\{t\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(t\-r^\{2\}\)\\right\)\\textup\{d\}t,dd​c​𝖥​\(0,c,r\)\|c=c∗=\\displaystyle\\frac\{\\textup\{d\}\}\{\\textup\{d\}c\}\\mathsf\{F\}\(0,c,r\)\\bigg\|\_\{c=c\_\{\*\}\}=\\,𝔼​\[gc∗​\(Y,M1\)\]\+12​\(𝔼​\[\(∂xfY,c∗​\(1,X1\)\)2\]−1α​c∗2​\(1−r2\)\)\.\\displaystyle\\mathbb\{E\}\\left\[g\_\{c\_\{\*\}\}\\left\(Y,M\_\{1\}\\right\)\\right\]\+\\frac\{1\}\{2\}\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(1\-r^\{2\}\)\\right\)\.Using our assumptions and integration by parts, we can show that[Eq\.77](https://arxiv.org/html/2606.28573#A3.E77)holds for anyμ∈𝒰\\mu\\in\\mathscr\{U\}\. The proof of “\(a\)\(a\)implies\(b\)\(b\)” follows similarly by computing the first\-order variations and using integration by parts\. This completes the proof\. ∎

By straightforward calculations, condition\(b\)\(b\)in[C\.2](https://arxiv.org/html/2606.28573#A3.Thmlem2)is equivalent to

𝔼​\[gc∗​\(Y,M1\)\]=0,𝔼​\[\(∂xfY,c∗​\(1,X1\)\)2\]−1α​c∗2​\(1−r2\)=0,\\displaystyle\\mathbb\{E\}\\left\[g\_\{c\_\{\*\}\}\\left\(Y,M\_\{1\}\\right\)\\right\]=\\,0,\\quad\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(1\-r^\{2\}\)=0,∫s1\(𝔼​\[\(∂xfY,c∗​\(t,Xt\)\)2\]−1α​c∗2​\(t−r2\)\)​d​t≥0,∀s∈\[r2,1\]\.\\displaystyle\\int\_\{s\}^\{1\}\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(t,X\_\{t\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(t\-r^\{2\}\)\\right\)\\textup\{d\}t\\geq 0,\\,\\,\\forall s\\in\[r^\{2\},1\]\.Using Proposition C\.4 in\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\], we know that

𝔼​\[gc∗​\(Y,M1\)\]=0⟹𝔼​\[hc∗​\(Y,M1\)\]=𝔼​\[h​\(Y,M1\)\]\.\\displaystyle\\mathbb\{E\}\\left\[g\_\{c\_\{\*\}\}\\left\(Y,M\_\{1\}\\right\)\\right\]=\\,0\\implies\\mathbb\{E\}\\left\[h\_\{c\_\{\*\}\}\\left\(Y,M\_\{1\}\\right\)\\right\]=\\,\\mathbb\{E\}\\left\[h\\left\(Y,M\_\{1\}\\right\)\\right\]\.SinceM1=c∗​∂xfY,c∗​\(1,X1\)\+X1M\_\{1\}=c\_\{\*\}\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\+X\_\{1\}andX1=r​G\+B1−Br2X\_\{1\}=rG\+B\_\{1\}\-B\_\{r^\{2\}\}, the above condition implies that

hc∗​\(y,c∗​∂xfy,c∗​\(1,x\)\+x\)=h​\(y,c∗​∂xfy,c∗​\(1,x\)\+x\)h\_\{c\_\{\*\}\}\\left\(y,c\_\{\*\}\\partial\_\{x\}f\_\{y,c\_\{\*\}\}\(1,x\)\+x\\right\)=\\,h\\left\(y,c\_\{\*\}\\partial\_\{x\}f\_\{y,c\_\{\*\}\}\(1,x\)\+x\\right\)for all\(y,x\)∈supp​\(Y\)×ℝ\(y,x\)\\in\\mathrm\{supp\}\(Y\)\\times\\mathbb\{R\}, which further implies thathc∗​\(y,x\)=h​\(y,x\)h\_\{c\_\{\*\}\}\(y,x\)=h\(y,x\)for all\(y,x\)∈supp​\(Y\)×ℝ\(y,x\)\\in\\mathrm\{supp\}\(Y\)\\times\\mathbb\{R\}as∂xfy,c∗​\(1,⋅\)\\partial\_\{x\}f\_\{y,c\_\{\*\}\}\(1,\\cdot\)is bounded and continuous\.

According to Itô’s formula, we deduce that the mapping

t↦𝔼​\[\(∂xfY,c∗​\(t,Xt\)\)2\]−1α​c∗2​\(t−r2\)t\\mapsto\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(t,X\_\{t\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(t\-r^\{2\}\)is convex\. Therefore, given

𝔼​\[\(∂xfY,c∗​\(1,X1\)\)2\]−1α​c∗2​\(1−r2\)=0,\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(1\-r^\{2\}\)=0,the condition

∫s1\(𝔼​\[\(∂xfY,c∗​\(t,Xt\)\)2\]−1α​c∗2​\(t−r2\)\)​d​t≥0,∀s∈\[r2,1\]\\int\_\{s\}^\{1\}\\left\(\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(t,X\_\{t\}\)\\right\)^\{2\}\\right\]\-\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(t\-r^\{2\}\)\\right\)\\textup\{d\}t\\geq 0,\\,\\,\\forall s\\in\[r^\{2\},1\]is equivalent to

𝔼​\[\(∂x2fY,c∗​\(1,X1\)\)2\]≤1α​c∗2\.\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}^\{2\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\\leq\\,\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\.We are now able to obtain a simpler equivalent form of condition\(b\)\(b\):

hc∗​\(y,x\)=h​\(y,x\)​for all​\(y,x\)∈supp​\(Y\)×ℝ,𝔼​\[\(∂xfY,c∗​\(1,X1\)\)2\]=1α​c∗2​\(1−r2\),𝔼​\[\(∂x2fY,c∗​\(1,X1\)\)2\]≤1α​c∗2,\\begin\{split\}&h\_\{c\_\{\*\}\}\(y,x\)=h\(y,x\)\\,\\,\\mbox\{for all\}\\,\\,\(y,x\)\\in\\mathrm\{supp\}\(Y\)\\times\\mathbb\{R\},\\\\ &\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]=\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\}\(1\-r^\{2\}\),\\quad\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}^\{2\}f\_\{Y,c\_\{\*\}\}\(1,X\_\{1\}\)\\right\)^\{2\}\\right\]\\leq\\frac\{1\}\{\\alpha c\_\{\*\}^\{2\}\},\\end\{split\}\(78\)whereY=φ​\(G,ε\)Y=\\varphi\(G,\\varepsilon\),X1=r​G\+B1−Br2X\_\{1\}=rG\+B\_\{1\}\-B\_\{r^\{2\}\}\.

## Appendix DAppendix for[Section5](https://arxiv.org/html/2606.28573#S5)

### D\.1Derivation of[5\.1](https://arxiv.org/html/2606.28573#S5.Thmconj1)

In this appendix, we derive the Parisi variational formula presented in[5\.1](https://arxiv.org/html/2606.28573#S5.Thmconj1)\. Recall from[2\.2](https://arxiv.org/html/2606.28573#S2.Thmconj2)that for fixedα,λ\>0\\alpha,\\lambda\>0, the replica method predicts that

limn/d→αmax𝒘∈𝕊d−1⁡H^n​\(𝒘\)=supr∈\[−1,1\]inf\(μ,c\)∈𝒰×ℝ\>01λ​𝖥​\(μ,c,r\),𝖥​\(μ,c,r\)=𝔼Y,G​\[fY,μ​\(r2,r​G\)\]\+12​α​∫r21d​tc\+∫t1μ​\(u\)​d​u,\\begin\{split\}&\\lim\_\{n/d\\to\\alpha\}\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}\\widehat\{H\}\_\{n\}\(\\text\{$w$\}\)=\\,\\sup\_\{r\\in\[\-1,1\]\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\mathsf\{F\}\(\\mu,c,r\),\\\\ &\\quad\\mathsf\{F\}\(\\mu,c,r\)=\\,\\mathbb\{E\}\_\{Y,G\}\[f\_\{Y,\\mu\}\(r^\{2\},rG\)\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(u\)\\textup\{d\}u\}\\,,\\end\{split\}whereY=λ​φ​\(G\)\+εY=\\sqrt\{\\lambda\}\\varphi\(G\)\+\\varepsilon, andfy,μf\_\{y,\\mu\}solves the PDE:

∂tfy,μ​\(t,x\)\+12​μ​\(t\)​\(∂xfy,μ​\(t,x\)\)2\+12​∂x2fy,μ​\(t,x\)=0,fy,μ​\(1,x\)=supu∈ℝ\{y​σ​\(x\+u\)−u22​c\}\.\\begin\{split\}&\\partial\_\{t\}f\_\{y,\\mu\}\(t,x\)\+\\frac\{1\}\{2\}\\mu\(t\)\(\\partial\_\{x\}f\_\{y,\\mu\}\(t,x\)\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,\\mu\}\(t,x\)=\\,0,\\\\ &f\_\{y,\\mu\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{y\\sigma\(x\+u\)\-\\frac\{u^\{2\}\}\{2c\}\\right\\\}\.\\end\{split\}We next compute the limiting Parisi formula asα→∞\\alpha\\to\\infty,λ→0\\lambda\\to 0,α​λ→α¯\\alpha\\lambda\\to\\overline\{\\alpha\}\. In order for the limit of\(1/λ\)​𝖥​\(μ,c,r\)\(1/\\sqrt\{\\lambda\}\)\\mathsf\{F\}\(\\mu,c,r\)to be meaningful, the function order parameters\(μ,c\)\(\\mu,c\)should scale likeλ\\sqrt\{\\lambda\}\. Hence, we replace\(μ,c\)\(\\mu,c\)with\(λ​μ,λ​c\)\(\\sqrt\{\\lambda\}\\mu,\\sqrt\{\\lambda\}c\), and obtain that

1λ​𝖥​\(λ​μ,λ​c,r\)=1λ​𝔼Y,G​\[fY,λ​μ​\(r2,r​G\)\]\+12​α​λ​∫r21d​tλ​c\+λ​∫t1μ​\(s\)​d​s,\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\mathsf\{F\}\\left\(\\sqrt\{\\lambda\}\\mu,\\sqrt\{\\lambda\}c,r\\right\)=\\,\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\sqrt\{\\lambda\}\\mu\}\(r^\{2\},rG\)\\right\]\+\\frac\{1\}\{2\\alpha\\sqrt\{\\lambda\}\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{\\sqrt\{\\lambda\}c\+\\sqrt\{\\lambda\}\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\},wherefy,λ​μf\_\{y,\\sqrt\{\\lambda\}\\mu\}is the solution to the PDE:

∂tfy,λ​μ​\(t,x\)\+12​λ​μ​\(t\)​\(∂xfy,λ​μ​\(t,x\)\)2\+12​∂x2fy,λ​μ​\(t,x\)=0,fy,λ​μ​\(1,x\)=supu∈ℝ\{y​σ​\(x\+u\)−u22​λ​c\}\.\\begin\{split\}&\\partial\_\{t\}f\_\{y,\\sqrt\{\\lambda\}\\mu\}\(t,x\)\+\\frac\{1\}\{2\}\\sqrt\{\\lambda\}\\mu\(t\)\(\\partial\_\{x\}f\_\{y,\\sqrt\{\\lambda\}\\mu\}\(t,x\)\)^\{2\}\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{y,\\sqrt\{\\lambda\}\\mu\}\(t,x\)=\\,0,\\\\ &f\_\{y,\\sqrt\{\\lambda\}\\mu\}\(1,x\)=\\,\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{y\\sigma\\left\(x\+u\\right\)\-\\frac\{u^\{2\}\}\{2\\sqrt\{\\lambda\}c\}\\right\\\}\.\\end\{split\}For any fixed\(μ,c\)∈𝒰×ℝ\>0\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}, it is straightforward to see thatfy,λ​μf\_\{y,\\sqrt\{\\lambda\}\\mu\}reduces tofε,0f\_\{\\varepsilon,0\}asλ→0\\lambda\\to 0, withfε,0f\_\{\\varepsilon,0\}being the solution to the heat equation:

∂tfε,0​\(t,x\)\+12​∂x2fε,0​\(t,x\)=0,fε,0​\(1,x\)=ε​σ​\(x\)\.\\begin\{split\}\\partial\_\{t\}f\_\{\\varepsilon,0\}\(t,x\)\+\\frac\{1\}\{2\}\\partial\_\{x\}^\{2\}f\_\{\\varepsilon,0\}\(t,x\)=\\,0,\\quad f\_\{\\varepsilon,0\}\(1,x\)=\\,\\varepsilon\\sigma\(x\)\.\\end\{split\}Of course, the above heat equation can be solved explicitly:

fε,0​\(t,x\)=ε​𝔼H∼𝖭​\(0,1\)​\[σ​\(x\+1−t​H\)\]\.f\_\{\\varepsilon,0\}\(t,x\)=\\,\\varepsilon\\mathbb\{E\}\_\{H\\sim\{\\sf N\}\(0,1\)\}\\big\[\\sigma\\big\(x\+\\sqrt\{1\-t\}H\\big\)\\big\]\.SinceY=λ​φ​\(G\)\+εY=\\sqrt\{\\lambda\}\\varphi\(G\)\+\\varepsilon, we can then perform Taylor expansions aroundλ=0\\lambda=0\(using[F\.1](https://arxiv.org/html/2606.28573#A6.Thmlem1)and Feynman\-Kac formula\) to deduce that

𝔼Y,G​\[fY,λ​μ​\(r2,r​G\)\]\\displaystyle\\mathbb\{E\}\_\{Y,G\}\\left\[f\_\{Y,\\sqrt\{\\lambda\}\\mu\}\(r^\{2\},rG\)\\right\]=\\displaystyle=\\,𝔼ε,G​\[fε,0​\(r2,r​G\)\]\+λ​𝔼​\[φ​\(G\)​σ​\(r​G\+∫r21d​Bt\)\]\\displaystyle\\mathbb\{E\}\_\{\\varepsilon,G\}\\left\[f\_\{\\varepsilon,0\}\(r^\{2\},rG\)\\right\]\+\\sqrt\{\\lambda\}\\mathbb\{E\}\\left\[\\varphi\(G\)\\sigma\\left\(rG\+\\int\_\{r^\{2\}\}^\{1\}\\textup\{d\}B\_\{t\}\\right\)\\right\]\+λ2​\(∫r21μ​\(t\)​𝔼​\[∂xfε,0​\(t,Bt\)2\]​d​t\+c​𝔼​\[ε2​σ′​\(B1\)2\]\)\+o​\(λ\)\\displaystyle\+\\frac\{\\sqrt\{\\lambda\}\}\{2\}\\left\(\\int\_\{r^\{2\}\}^\{1\}\\mu\(t\)\\mathbb\{E\}\\left\[\\partial\_\{x\}f\_\{\\varepsilon,0\}\(t,B\_\{t\}\)^\{2\}\\right\]\\textup\{d\}t\+c\\mathbb\{E\}\\left\[\\varepsilon^\{2\}\\sigma^\{\\prime\}\(B\_\{1\}\)^\{2\}\\right\]\\right\)\+o\(\\sqrt\{\\lambda\}\)=\\displaystyle=\\,𝔼​\[ε​σ​\(r​G\+∫r21d​Bt\)\]\+λ​ξφ​σ​\(r\)\+λ2​\(∫r21μ​\(t\)​ξσ​σ′​\(t\)​d​t\+c​ξσ​σ′​\(1\)\)\+o​\(λ\)\\displaystyle\\mathbb\{E\}\\left\[\\varepsilon\\sigma\\left\(rG\+\\int\_\{r^\{2\}\}^\{1\}\\textup\{d\}B\_\{t\}\\right\)\\right\]\+\\sqrt\{\\lambda\}\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{\\sqrt\{\\lambda\}\}\{2\}\\left\(\\int\_\{r^\{2\}\}^\{1\}\\mu\(t\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(t\)\\textup\{d\}t\+c\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\\right\)\+o\(\\sqrt\{\\lambda\}\)=\\displaystyle=\\,λ​\(ξφ​σ​\(r\)\+12​∫r21μ​\(t\)​ξσ​σ′​\(t\)​d​t\+12​c​ξσ​σ′​\(1\)\)\+o​\(λ\),\\displaystyle\\sqrt\{\\lambda\}\\left\(\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\mu\(t\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(t\)\\textup\{d\}t\+\\frac\{1\}\{2\}c\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\\right\)\+o\(\\sqrt\{\\lambda\}\),which leads to

limα​λ→α¯1λ​𝖥​\(λ​μ,λ​c,r\)=ξφ​σ​\(r\)\+12​∫r21μ​\(t\)​ξσ​σ′​\(t\)​d​t\+12​c​ξσ​σ′​\(1\)\+12​α¯​∫r21d​tc\+∫t1μ​\(s\)​d​s=𝖥¯α¯​\(μ,c,r\)\.\\begin\{split\}&\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\mathsf\{F\}\\left\(\\sqrt\{\\lambda\}\\mu,\\sqrt\{\\lambda\}c,r\\right\)\\\\ =\\,&\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\mu\(t\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(t\)\\textup\{d\}t\+\\frac\{1\}\{2\}c\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\textup\{d\}t\}\{c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\}=\\overline\{\\mathsf\{F\}\}\_\{\\overline\{\\alpha\}\}\(\\mu,c,r\)\.\\end\{split\}This completes the derivation of the replica prediction \([35](https://arxiv.org/html/2606.28573#S5.E35)\)\.

### D\.2Proof of[5\.1](https://arxiv.org/html/2606.28573#S5.Thmthm1),[5\.2](https://arxiv.org/html/2606.28573#S5.Thmlem2)and[5\.3](https://arxiv.org/html/2606.28573#S5.Thmlem3)

###### Proof of[5\.2](https://arxiv.org/html/2606.28573#S5.Thmlem2)\.

We begin with defining the value achievable by our two\-stage AMP algorithm at a fixed pair\(α,λ\)\(\\alpha,\\lambda\), based on the general results presented in[SectionB\.2](https://arxiv.org/html/2606.28573#A2.SS2)\. Let

ABayesα,λ=\{\(r,q\)∈ℝ2:q∈\[r2,1\],r2≤\(cBayesα,λ\)2​\(q−r2\)\},A\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}=\\\{\(r,q\)\\in\\mathbb\{R\}^\{2\}:q\\in\[r^\{2\},1\],\\,r^\{2\}\\leq\(c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}\)^\{2\}\(q\-r^\{2\}\)\\\},andcBayesα,λc\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}be the smallest positive solution to the fixed point equation \(again,cBayesα,λ=\+∞c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}=\+\\inftyif this equation has no solution\)

c2=α𝔼\[𝔼\[G−cZ\|cG\+Z,Y\]2\],Y=λφ\(G\)\+ε,Z∼𝖭\(0,1\),Z⟂⟂\(Y,G\)\.c^\{2\}=\\alpha\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-cZ\|cG\+Z,Y\\right\]^\{2\}\\right\],\\quad Y=\\sqrt\{\\lambda\}\\varphi\(G\)\+\\varepsilon,\\,\\,Z\\sim\{\\sf N\}\(0,1\),\\,\\,Z\\perp\\\!\\\!\\\!\\perp\(Y,G\)\.Further, define for\(r,q\)∈\[−1,1\]×\[0,1\]\(r,q\)\\in\[\-1,1\]\\times\[0,1\],q≥r2q\\geq r^\{2\}:

H∞α,λ​\(r,q\)=sup𝔼​\[Y​σ​\(Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+1α​ϕt\)​d​Bt\)\],subject to​Fis an\(r,q\)\-contraction,and​supt∈\[q,1\]𝔼​\[ϕt2\]≤1,\\begin\{split\}&H\_\{\\infty\}^\{\\alpha,\\lambda\}\(r,q\)=\\,\\sup\\,\\mathbb\{E\}\\left\[Y\\sigma\\left\(Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\frac\{1\}\{\\sqrt\{\\alpha\}\}\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\],\\\\ &\\mbox\{subject to\}\\,\\,\\text\{$F$ is an $\(r,q\)$\-contraction\},\\,\\text\{and\}\\,\\,\\sup\_\{t\\in\[q,1\]\}\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\\leq 1,\\end\{split\}\(79\)whereY=λ​φ​\(G\)\+εY=\\sqrt\{\\lambda\}\\varphi\(G\)\+\\varepsilon,Zr,q=r​G\+q−r2​ZZ\_\{r,q\}=rG\+\\sqrt\{q\-r^\{2\}\}Z\.

We first prove thatlim infα​λ→α¯cBayesα,λ≥cBayes\\liminf\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}\\geq c\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. To this end, note that asα→∞\\alpha\\to\\infty,λ→0\\lambda\\to 0andα​λ→α¯\\alpha\\lambda\\to\\overline\{\\alpha\}, we have:

𝔼​\[𝔼​\[G−c​Z\|c​G\+Z,Y\]2\]1/2\\displaystyle\\mathbb\{E\}\\left\[\\mathbb\{E\}\\big\[G\-cZ\\big\|cG\+Z,Y\\big\]^\{2\}\\right\]^\{1/2\}=\\displaystyle=\\,𝔼​\[𝔼​\[G−c​Z\|c​G\+Z,ε\]2\]1/2\+λ​𝔼​\[𝔼​\[\(G−c​Z\)​φ​\(G\)\|c​G\+Z\]2\]1/2\+O​\(λ\)\\displaystyle\\mathbb\{E\}\\left\[\\mathbb\{E\}\\big\[G\-cZ\\big\|cG\+Z,\\varepsilon\\big\]^\{2\}\\right\]^\{1/2\}\+\\sqrt\{\\lambda\}\\,\\mathbb\{E\}\\left\[\\mathbb\{E\}\\big\[\(G\-cZ\)\\varphi\(G\)\\big\|cG\+Z\\big\]^\{2\}\\right\]^\{1/2\}\+O\(\\lambda\)=\\displaystyle=\\,λ​𝔼​\[𝔼​\[\(G−c​Z\)​φ​\(G\)\|c​G\+Z\]2\]1/2\+O​\(λ\),\\displaystyle\\sqrt\{\\lambda\}\\,\\mathbb\{E\}\\left\[\\mathbb\{E\}\\big\[\(G\-cZ\)\\varphi\(G\)\\big\|cG\+Z\\big\]^\{2\}\\right\]^\{1/2\}\+O\(\\lambda\),where the last equality follows from the fact thatG−c​ZG\-cZis independent of\(c​G\+Z,ε\)\(cG\+Z,\\varepsilon\)\. DefineW1=\(c​G\+Z\)/1\+c2W\_\{1\}=\(cG\+Z\)/\\sqrt\{1\+c^\{2\}\},W2=\(G−c​Z\)/1\+c2W\_\{2\}=\(G\-cZ\)/\\sqrt\{1\+c^\{2\}\}, we then haveW1,W2∼i\.i\.d\.𝖭​\(0,1\)W\_\{1\},W\_\{2\}\\stackrel\{\{\\scriptstyle\\mathrm\{i\.i\.d\.\}\}\}\{\{\\sim\}\}\{\\sf N\}\(0,1\), and

𝔼​\[𝔼​\[\(G−c​Z\)​φ​\(G\)\|c​G\+Z\]2\]1/2\\displaystyle\\mathbb\{E\}\\left\[\\mathbb\{E\}\\big\[\(G\-cZ\)\\varphi\(G\)\\big\|cG\+Z\\big\]^\{2\}\\right\]^\{1/2\}=\\displaystyle=\\,𝔼​\[𝔼​\[1\+c2​W2​φ​\(\(c​W1\+W2\)/1\+c2\)\|W1\]2\]1/2\\displaystyle\\mathbb\{E\}\\left\[\\mathbb\{E\}\\big\[\\sqrt\{1\+c^\{2\}\}W\_\{2\}\\varphi\(\(cW\_\{1\}\+W\_\{2\}\)/\\sqrt\{1\+c^\{2\}\}\)\\big\|W\_\{1\}\\big\]^\{2\}\\right\]^\{1/2\}=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,𝔼W1​\[𝔼W2,W2′​\[\(1\+c2\)​W2​W2′​φ​\(c​W1\+W21\+c2\)​φ​\(c​W1\+W2′1\+c2\)\]\]1/2\\displaystyle\\mathbb\{E\}\_\{W\_\{1\}\}\\left\[\\mathbb\{E\}\_\{W\_\{2\},W\_\{2\}^\{\\prime\}\}\\left\[\(1\+c^\{2\}\)W\_\{2\}W\_\{2\}^\{\\prime\}\\,\\varphi\\left\(\\frac\{cW\_\{1\}\+W\_\{2\}\}\{\\sqrt\{1\+c^\{2\}\}\}\\right\)\\varphi\\left\(\\frac\{cW\_\{1\}\+W\_\{2\}^\{\\prime\}\}\{\\sqrt\{1\+c^\{2\}\}\}\\right\)\\right\]\\right\]^\{1/2\}=\(i​i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(ii\)\}\}\{\{=\}\}\\,𝔼W1​\[𝔼W2,W2′​\[φ′​\(c​W1\+W21\+c2\)​φ′​\(c​W1\+W2′1\+c2\)\]\]1/2,\\displaystyle\\mathbb\{E\}\_\{W\_\{1\}\}\\left\[\\mathbb\{E\}\_\{W\_\{2\},W\_\{2\}^\{\\prime\}\}\\left\[\\varphi^\{\\prime\}\\left\(\\frac\{cW\_\{1\}\+W\_\{2\}\}\{\\sqrt\{1\+c^\{2\}\}\}\\right\)\\varphi^\{\\prime\}\\left\(\\frac\{cW\_\{1\}\+W\_\{2\}^\{\\prime\}\}\{\\sqrt\{1\+c^\{2\}\}\}\\right\)\\right\]\\right\]^\{1/2\},whereW1,W2,W2′∼i\.i\.d\.𝖭​\(0,1\)W\_\{1\},W\_\{2\},W\_\{2\}^\{\\prime\}\\stackrel\{\{\\scriptstyle\\mathrm\{i\.i\.d\.\}\}\}\{\{\\sim\}\}\{\\sf N\}\(0,1\)in\(i\)\(i\), and in\(i​i\)\(ii\)we use Stein’s identity\. Recalling the definition ofξφ​φ\\xi\_\{\\varphi\\varphi\}, it follows that

𝔼​\[𝔼​\[\(G−c​Z\)​φ​\(G\)\|c​G\+Z\]2\]1/2=ξφ​φ′​\(c21\+c2\)1/2\.\\mathbb\{E\}\\left\[\\mathbb\{E\}\\big\[\(G\-cZ\)\\varphi\(G\)\\big\|cG\+Z\\big\]^\{2\}\\right\]^\{1/2\}=\\,\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\Big\(\\frac\{c^\{2\}\}\{1\+c^\{2\}\}\\Big\)^\{1/2\}\.Therefore, the fixed point equation definingcBayesα,λc\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}is just

c2=α¯​ξφ​φ′​\(c21\+c2\)\+O​\(λ\)\.c^\{2\}=\\overline\{\\alpha\}\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\Big\(\\frac\{c^\{2\}\}\{1\+c^\{2\}\}\\Big\)\+O\\big\(\\sqrt\{\\lambda\}\\big\)\.By definition ofcBayesc\_\{\\mbox\{\\tiny\\rm Bayes\}\}, we immediately obtain thatcBayesα,λ≥cBayes−O​\(λ\)c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}\\geq c\_\{\\mbox\{\\tiny\\rm Bayes\}\}\-O\(\\sqrt\{\\lambda\}\)asλ→0\\lambda\\to 0, which establishes the desired claim\.

Now we return to the proof of[5\.2](https://arxiv.org/html/2606.28573#S5.Thmlem2)\. By continuity, it suffices to consider\(r,q\)∈int⁡ABayes\(r,q\)\\in\\operatorname\{int\}A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. Sincelim infα​λ→α¯cBayesα,λ≥cBayes\\liminf\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}\\geq c\_\{\\mbox\{\\tiny\\rm Bayes\}\}, for sufficiently smallλ\\lambdawe have\(r,q\)∈ABayesα,λ\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\alpha,\\lambda\}\. It then suffices to show that

limα​λ→α¯1λ​H∞α,λ​\(r,q\)≥𝒱α¯AMP​\(r,q\),\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}\\frac\{1\}\{\\sqrt\{\\lambda\}\}H\_\{\\infty\}^\{\\alpha,\\lambda\}\(r,q\)\\geq\\,\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\),withH∞α,λ​\(r,q\)H\_\{\\infty\}^\{\\alpha,\\lambda\}\(r,q\)defined in[Eq\.79](https://arxiv.org/html/2606.28573#A4.E79)\. To this end, we define an auxiliary quantity:

H∞​\(r,q\):=sup\\displaystyle H\_\{\\infty\}\(r,q\):=\\,\\sup\\,\\,\{𝔼\[φ\(G\)σ\(Zr,q\+∫q1dBt\)\]\+1α¯𝔼\[εσ′\(Zr,q\+∫q1dBt\)F0\(Zr,q,ε\)\]\\displaystyle\\bigg\\\{\\mathbb\{E\}\\left\[\\varphi\(G\)\\,\\sigma\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)\\right\]\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\]\(80\)\+1α¯𝔼\[εσ′\(Zr,q\+∫q1dBs\)∫q1ϕtdBt\]\},\\displaystyle\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\int\_\{q\}^\{1\}\\phi\_\{t\}\{\\rm d\}B\_\{t\}\\right\]\\bigg\\\},subject tosupt∈\[q,1\]𝔼​\[ϕt2\]≤1,\\displaystyle\\sup\_\{t\\in\[q,1\]\}\\mathbb\{E\}\[\\phi\_\{t\}^\{2\}\]\\leq 1\\,,r=𝔼​\[∂F0​\(Zr,q,ε\)∂ε​φ′​\(G\)\],q=r2\+1α¯​𝔼​\[F0​\(Zr,q,ε\)2\],\\displaystyle r=\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\}\{\\partial\\varepsilon\}\\,\\varphi^\{\\prime\}\(G\)\\right\]\\,,\\,\\,q=r^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\big\[F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)^\{2\}\\big\]\\,,\(81\)1α¯​𝔼​\[∂F0∂Zr,q​\(Zr,q,ε\)2\]≤1\.\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{0\}\}\{\\partial Z\_\{r,q\}\}\(Z\_\{r,q\},\\varepsilon\)^\{2\}\\right\]\\leq 1\\,\.We will prove two claims: \(a\)limα​λ→α¯H∞α,λ​\(r,q\)/λ≥H∞​\(r,q\)\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}H\_\{\\infty\}^\{\\alpha,\\lambda\}\(r,q\)/\\sqrt\{\\lambda\}\\geq H\_\{\\infty\}\(r,q\); \(b\)H∞​\(r,q\)=𝒱α¯AMP​\(r,q\)H\_\{\\infty\}\(r,q\)=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\.

##### Proof of claim \(a\)\.

For anyF0F\_\{0\}andϕ\\phisatisfying the constraints in the optimization problem definingH∞​\(r,q\)H\_\{\\infty\}\(r,q\), let

F​\(Zr,q,Y\)=1λ​\(F0​\(Zr,q,Y\)\+∑i=13ai​\(λ\)​gi​\(Zr,q,Y\)\),F\(Z\_\{r,q\},Y\)=\\,\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\left\(F\_\{0\}\\left\(Z\_\{r,q\},Y\\right\)\+\\sum\_\{i=1\}^\{3\}a\_\{i\}\(\\lambda\)g\_\{i\}\(Z\_\{r,q\},Y\)\\right\),\(82\)wherea​\(λ\):=\{ai​\(λ\)\}i=13a\(\\lambda\):=\\\{a\_\{i\}\(\\lambda\)\\\}\_\{i=1\}^\{3\}areλ\\lambda\-dependent constants, and\{gi\}i=13\\\{g\_\{i\}\\\}\_\{i=1\}^\{3\}are differentiable functions, both to be determined\. Then, we have \(note thatY=λ​φ​\(G\)\+εY=\\sqrt\{\\lambda\}\\varphi\(G\)\+\\varepsilon\)

𝔼​\[∂F∂G​\(Zr,q,Y\)\]=𝔼​\[\(∂yF0​\(Zr,q,Y\)\+∑i=13ai​\(λ\)​∂ygi​\(Zr,q,Y\)\)​φ′​\(G\)\]:=A1​\(λ,a​\(λ\)\),\\displaystyle\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial G\}\\left\(Z\_\{r,q\},Y\\right\)\\right\]=\\,\\mathbb\{E\}\\left\[\\left\(\\partial\_\{y\}F\_\{0\}\\left\(Z\_\{r,q\},Y\\right\)\+\\sum\_\{i=1\}^\{3\}a\_\{i\}\(\\lambda\)\\partial\_\{y\}g\_\{i\}\\left\(Z\_\{r,q\},Y\\right\)\\right\)\\varphi^\{\\prime\}\(G\)\\right\]:=A\_\{1\}\(\\lambda,a\(\\lambda\)\),1α​𝔼​\[F​\(Zr,q,Y\)2\]=1α¯​𝔼​\[\(F0​\(Zr,q,Y\)\+∑i=13ai​\(λ\)​gi​\(Zr,q,Y\)\)2\]:=A2​\(λ,a​\(λ\)\),\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(Z\_\{r,q\},Y\\right\)^\{2\}\\right\]=\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[\\left\(F\_\{0\}\\left\(Z\_\{r,q\},Y\\right\)\+\\sum\_\{i=1\}^\{3\}a\_\{i\}\(\\lambda\)g\_\{i\}\(Z\_\{r,q\},Y\)\\right\)^\{2\}\\right\]:=A\_\{2\}\(\\lambda,a\(\\lambda\)\),1α​𝔼​\[∂F∂Zr,q​\(Zr,q,Y\)2\]=1α¯​𝔼​\[\(∂zF0​\(Zr,q,Y\)\+∑i=13ai​\(λ\)​∂zgi​\(Zr,q,Y\)\)2\]:=A3​\(λ,a​\(λ\)\)\.\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{r,q\}\}\\left\(Z\_\{r,q\},Y\\right\)^\{2\}\\right\]=\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{z\}F\_\{0\}\\left\(Z\_\{r,q\},Y\\right\)\+\\sum\_\{i=1\}^\{3\}a\_\{i\}\(\\lambda\)\\partial\_\{z\}g\_\{i\}\(Z\_\{r,q\},Y\)\\right\)^\{2\}\\right\]:=A\_\{3\}\(\\lambda,a\(\\lambda\)\)\.DenoteA​\(λ,a​\(λ\)\)=\{Ai​\(λ,a​\(λ\)\)\}i=13A\(\\lambda,a\(\\lambda\)\)=\\\{A\_\{i\}\(\\lambda,a\(\\lambda\)\)\\\}\_\{i=1\}^\{3\}\. Our goal is to show that for all sufficiently smallλ\\lambda, there existsa​\(λ\)a\(\\lambda\)such thatA​\(λ,a​\(λ\)\)=\(r,q−r2,b​\(λ\)\)A\(\\lambda,a\(\\lambda\)\)=\(r,q\-r^\{2\},b\(\\lambda\)\)withb​\(λ\)≤1b\(\\lambda\)\\leq 1, thus implying thatFFis an\(r,q\)\(r,q\)\-contraction\.

To this end, we invoke the implicit function theorem\. SinceF0F\_\{0\}satisfies[Eq\.81](https://arxiv.org/html/2606.28573#A4.E81), we know thatA​\(0,0\)=\(r,q−r2,b​\(0\)\)A\(0,0\)=\(r,q\-r^\{2\},b\(0\)\)withb​\(0\)≤1b\(0\)\\leq 1\. It suffices to show that there exists\{gi\}i=13\\\{g\_\{i\}\\\}\_\{i=1\}^\{3\}such that the Jacobian\(∂A/∂a\)​\(0,0\)\(\\partial A/\\partial a\)\(0,0\)is non\-singular, i\.e\., the vectors\{vi\}i=13⊂ℝ3\\\{v\_\{i\}\\\}\_\{i=1\}^\{3\}\\subset\\mathbb\{R\}^\{3\}are linearly independent, where

vi=\(𝔼​\[∂ygi​\(Zr,q,ε\)​φ′​\(G\)\],𝔼​\[gi​\(Zr,q,ε\)​F0​\(Zr,q,ε\)\],𝔼​\[∂zgi​\(Zr,q,ε\)​∂zF0​\(Zr,q,ε\)\]\)\.v\_\{i\}=\\,\\left\(\\mathbb\{E\}\\left\[\\partial\_\{y\}g\_\{i\}\(Z\_\{r,q\},\\varepsilon\)\\varphi^\{\\prime\}\(G\)\\right\],\\mathbb\{E\}\\left\[g\_\{i\}\(Z\_\{r,q\},\\varepsilon\)F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\],\\mathbb\{E\}\\left\[\\partial\_\{z\}g\_\{i\}\(Z\_\{r,q\},\\varepsilon\)\\partial\_\{z\}F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\]\\right\)\.This is equivalent to showing that the linear mapping

g↦v​\(g\):=\(𝔼​\[∂yg​\(Zr,q,ε\)​φ′​\(G\)\],𝔼​\[g​\(Zr,q,ε\)​F0​\(Zr,q,ε\)\],𝔼​\[∂zg​\(Zr,q,ε\)​∂zF0​\(Zr,q,ε\)\]\)g\\mapsto v\(g\):=\\left\(\\mathbb\{E\}\\left\[\\partial\_\{y\}g\(Z\_\{r,q\},\\varepsilon\)\\varphi^\{\\prime\}\(G\)\\right\],\\mathbb\{E\}\\left\[g\(Z\_\{r,q\},\\varepsilon\)F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\],\\mathbb\{E\}\\left\[\\partial\_\{z\}g\(Z\_\{r,q\},\\varepsilon\)\\partial\_\{z\}F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\]\\right\)is surjective, which can be easily proved using Gaussian integration by parts and noting thatZr,q≠GZ\_\{r,q\}\\neq Gsince\(r,q\)∈int⁡ABayes\(r,q\)\\in\\operatorname\{int\}A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\.

We have verified thatFFdefined as per[Eq\.82](https://arxiv.org/html/2606.28573#A4.E82)is an\(r,q\)\(r,q\)\-contraction\. Next we compute

limα​λ→α¯1λ​𝔼​\[Y​σ​\(Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+1α​ϕt\)​d​Bt\)\]\.\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\mathbb\{E\}\\left\[Y\\sigma\\left\(Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\frac\{1\}\{\\sqrt\{\\alpha\}\}\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\]\.Note that since

F​\(Zr,q,Y\)=\\displaystyle F\(Z\_\{r,q\},Y\)=\\,1λ​\(F0​\(Zr,q,Y\)\+∑i=13ai​\(λ\)​gi​\(Zr,q,Y\)\)\\displaystyle\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\left\(F\_\{0\}\\left\(Z\_\{r,q\},Y\\right\)\+\\sum\_\{i=1\}^\{3\}a\_\{i\}\(\\lambda\)g\_\{i\}\(Z\_\{r,q\},Y\)\\right\)=\\displaystyle=\\,1λ​\(F0​\(Zr,q,Y\)\+o​\(1\)\)=1λ​\(F0​\(Zr,q,ε\)\+o​\(1\)\),\\displaystyle\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\left\(F\_\{0\}\(Z\_\{r,q\},Y\)\+o\(1\)\\right\)=\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\left\(F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\+o\(1\)\\right\),we have

1λ​𝔼​\[Y​σ​\(Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+1α​ϕt\)​d​Bt\)\]\\displaystyle\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\mathbb\{E\}\\left\[Y\\sigma\\left\(Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\frac\{1\}\{\\sqrt\{\\alpha\}\}\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\]=\\displaystyle=\\,𝔼​\[\(φ​\(G\)\+ελ\)​σ​\(Zr,q\+λα¯​F0​\(Zr,q,ε\)\+∫q1\(1\+1α​ϕt\)​d​Bt\+o​\(λ\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\\left\(\\varphi\(G\)\+\\frac\{\\varepsilon\}\{\\sqrt\{\\lambda\}\}\\right\)\\sigma\\left\(Z\_\{r,q\}\+\\frac\{\\sqrt\{\\lambda\}\}\{\\overline\{\\alpha\}\}F\_\{0\}\\left\(Z\_\{r,q\},\\varepsilon\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\frac\{1\}\{\\sqrt\{\\alpha\}\}\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\+o\(\\sqrt\{\\lambda\}\)\\right\)\\right\]=\\displaystyle=\\,𝔼​\[\(φ​\(G\)\+ελ\)​\(σ​\(Zr,q\+∫q1d​Bt\)\+σ′​\(Zr,q\+∫q1d​Bt\)​\(λα¯​F0​\(Zr,q,ε\)\+1α​∫q1ϕt​d​Bt\)\+o​\(λ\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\\left\(\\varphi\(G\)\+\\frac\{\\varepsilon\}\{\\sqrt\{\\lambda\}\}\\right\)\\left\(\\sigma\\left\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\\textup\{d\}B\_\{t\}\\right\)\+\\sigma^\{\\prime\}\\left\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\\textup\{d\}B\_\{t\}\\right\)\\left\(\\frac\{\\sqrt\{\\lambda\}\}\{\\overline\{\\alpha\}\}F\_\{0\}\\left\(Z\_\{r,q\},\\varepsilon\\right\)\+\\frac\{1\}\{\\sqrt\{\\alpha\}\}\\int\_\{q\}^\{1\}\\phi\_\{t\}\\textup\{d\}B\_\{t\}\\right\)\+o\(\\sqrt\{\\lambda\}\)\\right\)\\right\]=\\displaystyle=\\,𝔼​\[φ​\(G\)​σ​\(Zr,q\+∫q1dBt\)\]\+1α¯​𝔼​\[ε​σ′​\(Zr,q\+∫q1dBt\)​F0​\(Zr,q,ε\)\]\+1α¯​𝔼​\[ε​σ′​\(Zr,q\+∫q1dBs\)​∫q1ϕt​dBt\]\+o​\(1\),\\displaystyle\\mathbb\{E\}\\left\[\\varphi\(G\)\\,\\sigma\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)\\right\]\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\]\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\int\_\{q\}^\{1\}\\phi\_\{t\}\{\\rm d\}B\_\{t\}\\right\]\+o\(1\),which leads to

limα​λ→α¯1λ​𝔼​\[Y​σ​\(Zr,q\+1α​F​\(Zr,q,Y\)\+∫q1\(1\+1α​ϕt\)​d​Bt\)\]\\displaystyle\\lim\_\{\\alpha\\lambda\\to\\overline\{\\alpha\}\}\\frac\{1\}\{\\sqrt\{\\lambda\}\}\\mathbb\{E\}\\left\[Y\\sigma\\left\(Z\_\{r,q\}\+\\frac\{1\}\{\\alpha\}F\\left\(Z\_\{r,q\},Y\\right\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\frac\{1\}\{\\sqrt\{\\alpha\}\}\\phi\_\{t\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\\right\]=\\displaystyle=\\,𝔼​\[φ​\(G\)​σ​\(Zr,q\+∫q1dBt\)\]\+1α¯​𝔼​\[ε​σ′​\(Zr,q\+∫q1dBt\)​F0​\(Zr,q,ε\)\]\\displaystyle\\mathbb\{E\}\\left\[\\varphi\(G\)\\,\\sigma\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)\\right\]\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\]\+1α¯​𝔼​\[ε​σ′​\(Zr,q\+∫q1dBs\)​∫q1ϕt​dBt\]\.\\displaystyle\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\int\_\{q\}^\{1\}\\phi\_\{t\}\{\\rm d\}B\_\{t\}\\right\]\.The above value is achievable for anyF0F\_\{0\}andϕ\\phisatisfyingsupt∈\[q,1\]𝔼​\[ϕt2\]≤1\\sup\_\{t\\in\[q,1\]\}\\mathbb\{E\}\[\\phi\_\{t\}^\{2\}\]\\leq 1and \([81](https://arxiv.org/html/2606.28573#A4.E81)\), thus proving our claim \(a\)\.

##### Proof of claim \(b\)\.

We consider the three terms in the definition ofH∞​\(r,q\)H\_\{\\infty\}\(r,q\), respectively\. For the first term, we note that by definition ofξφ​σ\\xi\_\{\\varphi\\sigma\}andZr,qZ\_\{r,q\},

𝔼​\[φ​\(G\)​σ​\(Zr,q\+∫q1d​Bt\)\]=ξφ​σ​\(r\)\.\\displaystyle\\mathbb\{E\}\\left\[\\varphi\(G\)\\sigma\\left\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\\textup\{d\}B\_\{t\}\\right\)\\right\]=\\xi\_\{\\varphi\\sigma\}\(r\)\\,\.
As for the third term in[Eq\.80](https://arxiv.org/html/2606.28573#A4.E80), we have

𝔼​\[ε​σ′​\(Zr,q\+∫q1dBs\)​∫q1ϕt​dBt\]=𝔼​\[∫q1ε​σ′​\(Zr,q\+∫q1dBs\)​ϕt​dBt\]\\displaystyle\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\int\_\{q\}^\{1\}\\phi\_\{t\}\{\\rm d\}B\_\{t\}\\right\]=\\,\\mathbb\{E\}\\left\[\\int\_\{q\}^\{1\}\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\phi\_\{t\}\{\\rm d\}B\_\{t\}\\right\]=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,𝔼​\[∫q1ε​ϕt​\(σ′​\(Zr,q\+∫q1dBs\)−𝔼​\[σ′​\(Zr,q\+∫q1dBs\)\|ℱt\]\)​dBt\]\\displaystyle\\mathbb\{E\}\\left\[\\int\_\{q\}^\{1\}\\varepsilon\\phi\_\{t\}\\left\(\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\-\\mathbb\{E\}\\left\[\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{t\}\\right\]\\right\)\{\\rm d\}B\_\{t\}\\right\]=\(i​i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(ii\)\}\}\{\{=\}\}\\,𝔼​\[∫q1ε​ϕt​\(∫t1𝔼​\[σ′′​\(Zr,q\+∫q1dBs\)\|ℱu\]​d​Bu\)​dBt\]\\displaystyle\\mathbb\{E\}\\left\[\\int\_\{q\}^\{1\}\\varepsilon\\phi\_\{t\}\\left\(\\int\_\{t\}^\{1\}\\mathbb\{E\}\\left\[\\sigma^\{\\prime\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{u\}\\right\]\\textup\{d\}B\_\{u\}\\right\)\{\\rm d\}B\_\{t\}\\right\]=\\displaystyle=\\,∫q1𝔼​\[ε​ϕt​𝔼​\[σ′′​\(Zr,q\+∫q1dBs\)\|ℱt\]\]​d​t,\\displaystyle\\int\_\{q\}^\{1\}\\mathbb\{E\}\\left\[\\varepsilon\\phi\_\{t\}\\mathbb\{E\}\\left\[\\sigma^\{\\prime\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{t\}\\right\]\\right\]\\textup\{d\}t,where\(i\)\(i\)is becauseε\\varepsilonandϕt\\phi\_\{t\}are adapted toℱt\\mathcal\{F\}\_\{t\}, and in\(i​i\)\(ii\)we use the Clark\-Ocone formula \(cf\.\[[Nua06](https://arxiv.org/html/2606.28573#bib.bibx36)\]\)\. By Cauchy\-Schwarz inequality, for anyt∈\[q,1\]t\\in\[q,1\]:

𝔼​\[ε​ϕt​𝔼​\[σ′′​\(Zr,q\+∫q1dBs\)\|ℱt\]\]\\displaystyle\\mathbb\{E\}\\left\[\\varepsilon\\phi\_\{t\}\\mathbb\{E\}\\left\[\\sigma^\{\\prime\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{t\}\\right\]\\right\]≤\\displaystyle\\leq\\,𝔼​\[ϕt2\]1/2​𝔼​\[ε2​𝔼​\[σ′′​\(Zr,q\+∫q1dBs\)\|ℱt\]2\]1/2≤𝔼​\[ε2​𝔼​\[σ′′​\(Zr,q\+∫q1dBs\)\|ℱt\]2\]1/2\\displaystyle\\mathbb\{E\}\[\\phi\_\{t\}^\{2\}\]^\{1/2\}\\mathbb\{E\}\\left\[\\varepsilon^\{2\}\\mathbb\{E\}\\left\[\\sigma^\{\\prime\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{t\}\\right\]^\{2\}\\right\]^\{1/2\}\\leq\\,\\mathbb\{E\}\\left\[\\varepsilon^\{2\}\\mathbb\{E\}\\left\[\\sigma^\{\\prime\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{t\}\\right\]^\{2\}\\right\]^\{1/2\}=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,𝔼ε,G1​\[ε2​𝔼G2,G2′​\[σ′′​\(t​G1\+1−t​G2\)​σ′′​\(t​G1\+1−t​G2′\)\]\]1/2\\displaystyle\\mathbb\{E\}\_\{\\varepsilon,G\_\{1\}\}\\left\[\\varepsilon^\{2\}\\mathbb\{E\}\_\{G\_\{2\},G\_\{2\}^\{\\prime\}\}\\left\[\\sigma^\{\\prime\\prime\}\(\\sqrt\{t\}G\_\{1\}\+\\sqrt\{1\-t\}G\_\{2\}\)\\sigma^\{\\prime\\prime\}\(\\sqrt\{t\}G\_\{1\}\+\\sqrt\{1\-t\}G\_\{2\}^\{\\prime\}\)\\right\]\\right\]^\{1/2\}=\\displaystyle=\\,𝔼G1,G2,G2′​\[σ′′​\(t​G1\+1−t​G2\)​σ′′​\(t​G1\+1−t​G2′\)\]1/2=ξσ​σ′′​\(t\)1/2,\\displaystyle\\mathbb\{E\}\_\{G\_\{1\},G\_\{2\},G\_\{2\}^\{\\prime\}\}\\left\[\\sigma^\{\\prime\\prime\}\(\\sqrt\{t\}G\_\{1\}\+\\sqrt\{1\-t\}G\_\{2\}\)\\sigma^\{\\prime\\prime\}\(\\sqrt\{t\}G\_\{1\}\+\\sqrt\{1\-t\}G\_\{2\}^\{\\prime\}\)\\right\]^\{1/2\}=\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)^\{1/2\},whereG1,G2,G2′∼i\.i\.d\.𝖭​\(0,1\)G\_\{1\},G\_\{2\},G\_\{2\}^\{\\prime\}\\stackrel\{\{\\scriptstyle\\mathrm\{i\.i\.d\.\}\}\}\{\{\\sim\}\}\{\\sf N\}\(0,1\)in\(i\)\(i\)\. Further, the equalities are achieved at

ϕt=𝔼​\[ε​σ′′​\(Zr,q\+∫q1dBs\)\|ℱt\]𝔼​\[𝔼​\[ε​σ′′​\(Zr,q\+∫q1dBs\)\|ℱt\]2\]1/2\.\\phi\_\{t\}=\\frac\{\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{t\}\\right\]\}\{\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{s\}\\Big\)\\Big\|\\mathcal\{F\}\_\{t\}\\right\]^\{2\}\\right\]^\{1/2\}\}\.Therefore, for any fixed\(r,q\)\(r,q\), the maximum of the third term in[Eq\.80](https://arxiv.org/html/2606.28573#A4.E80)is

1α¯​∫q1ξσ​σ′′​\(t\)1/2​d​t\.\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{q\}^\{1\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)^\{1/2\}\\textup\{d\}t\.
Finally, we consider optimizing the second term in[Eq\.80](https://arxiv.org/html/2606.28573#A4.E80)\. Our goal is to maximize

𝔼​\[ε​σ′​\(Zr,q\+∫q1dBt\)​F0​\(Zr,q,ε\)\],\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\],subject to the constraints in[Eq\.81](https://arxiv.org/html/2606.28573#A4.E81)\. Assume thatF0F\_\{0\}has the following Hermite polynomial decomposition:

F0​\(q​x,y\)=∑k,l=0∞fk,l​Hek​\(x\)​Hel​\(y\),F\_\{0\}\(\\sqrt\{q\}x,y\)=\\sum\_\{k,l=0\}^\{\\infty\}f\_\{k,l\}\\mathrm\{He\}\_\{k\}\(x\)\\mathrm\{He\}\_\{l\}\(y\),whereHek\\mathrm\{He\}\_\{k\}is thekk\-th normalized Hermite polynomial\. The\(r,q\)\(r,q\)\-contraction conditions \([81](https://arxiv.org/html/2606.28573#A4.E81)\) are equivalent to

r=∑k=0∞k\+1​φk\+1​fk,1​\(rq\)k,q=r2\+1α¯​∑k,l=0∞fk,l2,1α¯​q​∑k,l=0∞k​fk,l2≤1\.\\displaystyle r=\\sum\_\{k=0\}^\{\\infty\}\\sqrt\{k\+1\}\\varphi\_\{k\+1\}f\_\{k,1\}\\left\(\\frac\{r\}\{\\sqrt\{q\}\}\\right\)^\{k\},\\quad q=r^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k,l=0\}^\{\\infty\}f\_\{k,l\}^\{2\},\\quad\\frac\{1\}\{\\overline\{\\alpha\}q\}\\sum\_\{k,l=0\}^\{\\infty\}kf\_\{k,l\}^\{2\}\\leq 1\.Further,

𝔼​\[ε​σ′​\(Zr,q\+∫q1dBt\)​F0​\(Zr,q,ε\)\]=∑k=0∞k\+1​σk\+1​fk,1​\(q\)k\.\\mathbb\{E\}\\left\[\\varepsilon\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)F\_\{0\}\(Z\_\{r,q\},\\varepsilon\)\\right\]=\\,\\sum\_\{k=0\}^\{\\infty\}\\sqrt\{k\+1\}\\sigma\_\{k\+1\}f\_\{k,1\}\(\\sqrt\{q\}\)^\{k\}\.It is straightforward to see that this optimization problem is equivalent to the following:

maximize​∑k=0∞k\+1​σk\+1​fk,1​\(q\)k,s\.t\.​r=∑k=0∞k\+1​φk\+1​fk,1​\(rq\)k,q≥r2\+1α¯​∑k=0∞fk,12,1α¯​q​∑k=0∞k​fk,12≤1\.\\begin\{split\}&\\mbox\{maximize\}\\,\\,\\sum\_\{k=0\}^\{\\infty\}\\sqrt\{k\+1\}\\sigma\_\{k\+1\}f\_\{k,1\}\(\\sqrt\{q\}\)^\{k\},\\\\ &\\mbox\{s\.t\.\}\\,\\,r=\\sum\_\{k=0\}^\{\\infty\}\\sqrt\{k\+1\}\\varphi\_\{k\+1\}f\_\{k,1\}\\left\(\\frac\{r\}\{\\sqrt\{q\}\}\\right\)^\{k\},\\,\\,q\\geq r^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k=0\}^\{\\infty\}f\_\{k,1\}^\{2\},\\,\\,\\frac\{1\}\{\\overline\{\\alpha\}q\}\\sum\_\{k=0\}^\{\\infty\}kf\_\{k,1\}^\{2\}\\leq 1\.\\end\{split\}\(83\)Introducing the change of variablesxk\+1=fk,1/α¯​\(q−r2\)x\_\{k\+1\}=f\_\{k,1\}/\\sqrt\{\\overline\{\\alpha\}\(q\-r^\{2\}\)\},k≥0k\\geq 0, and recalling the definitions ofcc,vkv\_\{k\}’s andsks\_\{k\}’s from[Eq\.43](https://arxiv.org/html/2606.28573#S5.E43), it is straightforward to see that the value of the optimization problem \([83](https://arxiv.org/html/2606.28573#A4.E83)\) is equal toα¯​\(q−r2\)​T0​\(r,q\)\\sqrt\{\\overline\{\\alpha\}\(q\-r^\{2\}\)\}T\_\{0\}\(r,q\)\. This establishes our claim \(b\) and completes the proof of[5\.2](https://arxiv.org/html/2606.28573#S5.Thmlem2)\. ∎

###### Proof of[5\.1](https://arxiv.org/html/2606.28573#S5.Thmthm1)\.

Proof of\(a\)\(a\)\.Comparing the expressions for𝒱α¯AMP​\(r,q\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)and𝒱α¯∗​\(r,q\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\), it suffices to show thatT0​\(r,q\)≤ξσ​σ′​\(q\)T\_\{0\}\(r,q\)\\leq\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\. By definition ofT0​\(r,q\)T\_\{0\}\(r,q\)and Cauchy\-Schwarz inequality, we have

T0​\(r,q\)≤‖s‖2=∑k≥1k​σk2​qk−1=ξσ​σ′​\(q\)\.T\_\{0\}\(r,q\)\\leq\\,\\left\\\|\{s\}\\right\\\|\_\{2\}=\\sqrt\{\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}^\{2\}q^\{k\-1\}\}=\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\.This proves part\(a\)\(a\)\.

Proof of\(b\)\(b\)\.We will prove a stronger result: For all\(q,r\)\(q,r\)withr2≤q≤1r^\{2\}\\leq q\\leq 1,𝒱α¯AMP​\(r,q\)≤𝒱α¯∗​\(r,q∗​\(r\)\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\,\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\. To this end, denoteT​\(r,q\)=\(q−r2\)​T0​\(r,q\)/α¯T\(r,q\)=\\sqrt\{\(q\-r^\{2\}\)\}T\_\{0\}\(r,q\)/\\sqrt\{\\overline\{\\alpha\}\}, the second term in the expression for𝒱α¯AMP​\(r,q\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\. Then, the proof of[5\.2](https://arxiv.org/html/2606.28573#S5.Thmlem2)implies that,

T​\(r,q\)=max⁡1α¯​𝔼​\[σ′​\(Zr,q\+∫q1dBt\)​F0​\(Zr,q\)\],\\displaystyle T\(r,q\)=\\,\\max\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)F\_\{0\}\(Z\_\{r,q\}\)\\right\],s\.t\.​r=𝔼​\[F0​\(Zr,q\)​φ′​\(G\)\],q≥r2\+1α¯​𝔼​\[F0​\(Zr,q\)2\],1α¯​𝔼​\[F0′​\(Zr,q\)2\]≤1\.\\displaystyle\\mbox\{s\.t\.\}\\,\\,r=\\mathbb\{E\}\\left\[F\_\{0\}\(Z\_\{r,q\}\)\\varphi^\{\\prime\}\(G\)\\right\],\\,\\,q\\geq r^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\big\[F\_\{0\}\(Z\_\{r,q\}\)^\{2\}\\big\],\\,\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\\mathbb\{E\}\\left\[F\_\{0\}^\{\\prime\}\(Z\_\{r,q\}\)^\{2\}\\right\]\\leq 1\.We can viewr​GrGasBr2B\_\{r^\{2\}\}andZr,q=r​G\+q−r2​ZZ\_\{r,q\}=rG\+\\sqrt\{q\-r^\{2\}\}ZasBqB\_\{q\}, so that the objective function can be rewritten as \(using the Clark\-Ocone formula from\[[Nua06](https://arxiv.org/html/2606.28573#bib.bibx36)\]\)

𝔼​\[σ′​\(Zr,q\+∫q1dBt\)​F0​\(Zr,q\)\]=𝔼​\[σ′​\(B1\)​F0​\(Bq\)\]\\displaystyle\\mathbb\{E\}\\left\[\\sigma^\{\\prime\}\\Big\(Z\_\{r,q\}\+\\int\_\{q\}^\{1\}\{\\rm d\}B\_\{t\}\\Big\)F\_\{0\}\(Z\_\{r,q\}\)\\right\]=\\,\\mathbb\{E\}\\left\[\\sigma^\{\\prime\}\(B\_\{1\}\)F\_\{0\}\(B\_\{q\}\)\\right\]=\\displaystyle=\\,𝔼​\[\(𝔼​\[σ′​\(B1\)\|ℱr2\]\+∫r21𝔼​\[σ′′​\(B1\)\|ℱt\]​d​Bt\)​\(𝔼​\[F0​\(Bq\)\|ℱr2\]\+∫r2q𝔼​\[F0′​\(Bq\)\|ℱt\]​d​Bt\)\]\\displaystyle\\mathbb\{E\}\\left\[\\left\(\\mathbb\{E\}\[\\sigma^\{\\prime\}\(B\_\{1\}\)\|\\mathcal\{F\}\_\{r^\{2\}\}\]\+\\int\_\{r^\{2\}\}^\{1\}\\mathbb\{E\}\[\\sigma^\{\\prime\\prime\}\(B\_\{1\}\)\|\\mathcal\{F\}\_\{t\}\]\\textup\{d\}B\_\{t\}\\right\)\\left\(\\mathbb\{E\}\[F\_\{0\}\(B\_\{q\}\)\|\\mathcal\{F\}\_\{r^\{2\}\}\]\+\\int\_\{r^\{2\}\}^\{q\}\\mathbb\{E\}\[F\_\{0\}^\{\\prime\}\(B\_\{q\}\)\|\\mathcal\{F\}\_\{t\}\]\\textup\{d\}B\_\{t\}\\right\)\\right\]=\\displaystyle=\\,𝔼​\[𝔼​\[σ′​\(B1\)\|ℱr2\]​𝔼​\[F0​\(Bq\)\|ℱr2\]\]\+∫r2q𝔼​\[𝔼​\[σ′′​\(B1\)\|ℱt\]​𝔼​\[F0′​\(Bq\)\|ℱt\]\]​d​t\\displaystyle\\mathbb\{E\}\\left\[\\mathbb\{E\}\[\\sigma^\{\\prime\}\(B\_\{1\}\)\|\\mathcal\{F\}\_\{r^\{2\}\}\]\\mathbb\{E\}\[F\_\{0\}\(B\_\{q\}\)\|\\mathcal\{F\}\_\{r^\{2\}\}\]\\right\]\+\\int\_\{r^\{2\}\}^\{q\}\\mathbb\{E\}\\left\[\\mathbb\{E\}\[\\sigma^\{\\prime\\prime\}\(B\_\{1\}\)\|\\mathcal\{F\}\_\{t\}\]\\mathbb\{E\}\[F\_\{0\}^\{\\prime\}\(B\_\{q\}\)\|\\mathcal\{F\}\_\{t\}\]\\right\]\\textup\{d\}t=\\displaystyle=\\,𝔼​\[V​Ur2\]\+∫r2q𝔼​\[ϕt​ψt\]​d​t,\\displaystyle\\mathbb\{E\}\\left\[VU\_\{r^\{2\}\}\\right\]\+\\int\_\{r^\{2\}\}^\{q\}\\mathbb\{E\}\\left\[\\phi\_\{t\}\\psi\_\{t\}\\right\]\\textup\{d\}t,where we denoteUr2=𝔼​\[σ′​\(B1\)\|ℱr2\]U\_\{r^\{2\}\}=\\mathbb\{E\}\[\\sigma^\{\\prime\}\(B\_\{1\}\)\|\\mathcal\{F\}\_\{r^\{2\}\}\],V=𝔼​\[F0​\(Bq\)\|ℱr2\]V=\\mathbb\{E\}\[F\_\{0\}\(B\_\{q\}\)\|\\mathcal\{F\}\_\{r^\{2\}\}\],ψt=𝔼​\[σ′′​\(B1\)\|ℱt\]\\psi\_\{t\}=\\mathbb\{E\}\[\\sigma^\{\\prime\\prime\}\(B\_\{1\}\)\|\\mathcal\{F\}\_\{t\}\], andϕt=𝔼​\[F0′​\(Bq\)\|ℱt\]\\phi\_\{t\}=\\mathbb\{E\}\[F\_\{0\}^\{\\prime\}\(B\_\{q\}\)\|\\mathcal\{F\}\_\{t\}\]fort∈\[r2,q\]t\\in\[r^\{2\},q\]\. Our second and third constraints are equivalent to

supt∈\[r2,q\]𝔼​\[ϕt2\]≤α¯,and​𝔼​\[V2\]\+∫r2q𝔼​\[ϕt2\]​d​t≤α¯​\(q−r2\)\.\\sup\_\{t\\in\[r^\{2\},q\]\}\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\\leq\\overline\{\\alpha\},\\quad\\mbox\{and\}\\,\\,\\mathbb\{E\}\\left\[V^\{2\}\\right\]\+\\int\_\{r^\{2\}\}^\{q\}\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\\textup\{d\}t\\leq\\overline\{\\alpha\}\(q\-r^\{2\}\)\.Ignoring the first constraint, we can use Lagrange duality to deduce that

α¯T\(r,q\)≤supV,ϕinfγ,λ≥0\{\\displaystyle\\overline\{\\alpha\}T\(r,q\)\\leq\\,\\sup\_\{V,\\phi\}\\inf\_\{\\gamma,\\lambda\\geq 0\}\\bigg\\\{\\,𝔼​\[V​Ur2\]\+∫r2q𝔼​\[ϕt​ψt\]​d​t−12​∫r2qγ​\(t\)​\(𝔼​\[ϕt2\]−α¯\)​d​t\\displaystyle\\mathbb\{E\}\\left\[VU\_\{r^\{2\}\}\\right\]\+\\int\_\{r^\{2\}\}^\{q\}\\mathbb\{E\}\\left\[\\phi\_\{t\}\\psi\_\{t\}\\right\]\\textup\{d\}t\-\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{q\}\\gamma\(t\)\\left\(\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\-\\overline\{\\alpha\}\\right\)\\textup\{d\}t−λ2\(𝔼\[V2\]\+∫r2q\(𝔼\[ϕt2\]−α¯\)dt\)\}\\displaystyle\-\\frac\{\\lambda\}\{2\}\\left\(\\mathbb\{E\}\\left\[V^\{2\}\\right\]\+\\int\_\{r^\{2\}\}^\{q\}\\left\(\\mathbb\{E\}\\left\[\\phi\_\{t\}^\{2\}\\right\]\-\\overline\{\\alpha\}\\right\)\\textup\{d\}t\\right\)\\bigg\\\}≤infγ,λ≥0supV,ϕ\{\\displaystyle\\leq\\,\\inf\_\{\\gamma,\\lambda\\geq 0\}\\sup\_\{V,\\phi\}\\bigg\\\{\\,𝔼​\[V​Ur2−λ2​V2\]\+∫r2q𝔼​\[ϕt​ψt−γ​\(t\)\+λ2​ϕt2\]​d​t\\displaystyle\\mathbb\{E\}\\left\[VU\_\{r^\{2\}\}\-\\frac\{\\lambda\}\{2\}V^\{2\}\\right\]\+\\int\_\{r^\{2\}\}^\{q\}\\mathbb\{E\}\\left\[\\phi\_\{t\}\\psi\_\{t\}\-\\frac\{\\gamma\(t\)\+\\lambda\}\{2\}\\phi\_\{t\}^\{2\}\\right\]\\textup\{d\}t\+α¯2∫r2q\(γ\(t\)\+λ\)dt\}\\displaystyle\+\\frac\{\\overline\{\\alpha\}\}\{2\}\\int\_\{r^\{2\}\}^\{q\}\\left\(\\gamma\(t\)\+\\lambda\\right\)\\textup\{d\}t\\bigg\\\}=infγ,λ≥0\{12​λ\\displaystyle=\\,\\inf\_\{\\gamma,\\lambda\\geq 0\}\\bigg\\\{\\frac\{1\}\{2\\lambda\}\\,𝔼\[Ur22\]\+12∫r2q1γ​\(t\)\+λ𝔼\[ψt2\]dt\+α¯2∫r2q\(γ\(t\)\+λ\)dt\}\\displaystyle\\mathbb\{E\}\\left\[U\_\{r^\{2\}\}^\{2\}\\right\]\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{q\}\\frac\{1\}\{\\gamma\(t\)\+\\lambda\}\\mathbb\{E\}\\left\[\\psi\_\{t\}^\{2\}\\right\]\\textup\{d\}t\+\\frac\{\\overline\{\\alpha\}\}\{2\}\\int\_\{r^\{2\}\}^\{q\}\\left\(\\gamma\(t\)\+\\lambda\\right\)\\textup\{d\}t\\bigg\\\}=infγ,λ≥0\{12​λ\\displaystyle=\\,\\inf\_\{\\gamma,\\lambda\\geq 0\}\\bigg\\\{\\frac\{1\}\{2\\lambda\}\\,ξσ​σ′\(r2\)\+12∫r2q1γ​\(t\)\+λξσ​σ′′\(t\)dt\+α¯2∫r2q\(γ\(t\)\+λ\)dt\},\\displaystyle\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{q\}\\frac\{1\}\{\\gamma\(t\)\+\\lambda\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\\textup\{d\}t\+\\frac\{\\overline\{\\alpha\}\}\{2\}\\int\_\{r^\{2\}\}^\{q\}\\left\(\\gamma\(t\)\+\\lambda\\right\)\\textup\{d\}t\\bigg\\\}\\,,where by definition, we have

𝔼​\[ψt2\]=ξσ​σ′′​\(t\),𝔼​\[Ur22\]=ξσ​σ′​\(r2\)\.\\mathbb\{E\}\\left\[\\psi\_\{t\}^\{2\}\\right\]=\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\),\\quad\\mathbb\{E\}\\left\[U\_\{r^\{2\}\}^\{2\}\\right\]=\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\.This yields that for allq∈\[r2,1\]q\\in\[r^\{2\},1\]:

𝒱α¯AMP​\(r,q\)=\\displaystyle\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)=\\,ξφ​σ​\(r\)\+T​\(r,q\)\+1α¯​∫q1ξσ​σ′′​\(t\)​dt\\displaystyle\\xi\_\{\\varphi\\sigma\}\(r\)\+T\(r,q\)\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{q\}^\{1\}\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\\,\{\\rm d\}t≤\\displaystyle\\leq\\,ξφ​σ​\(r\)\+infγ,λ≥0\{12​α¯​λ​ξσ​σ′​\(r2\)\+12​α¯​∫r211γ​\(t\)\+λ​𝟏t≤q​ξσ​σ′′​\(t\)​d​t\+12​∫r21\(γ​\(t\)\+λ​𝟏t≤q\)​d​t\}\\displaystyle\\xi\_\{\\varphi\\sigma\}\(r\)\+\\inf\_\{\\gamma,\\lambda\\geq 0\}\\bigg\\\{\\frac\{1\}\{2\\overline\{\\alpha\}\\lambda\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{1\}\{\\gamma\(t\)\+\\lambda\\mathrm\{\\bf 1\}\_\{t\\leq q\}\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\\textup\{d\}t\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\left\(\\gamma\(t\)\+\\lambda\\mathrm\{\\bf 1\}\_\{t\\leq q\}\\right\)\\textup\{d\}t\\bigg\\\}\\,=\\displaystyle=\\,ξφ​σ​\(r\)\+infγ,λ≥0γ\|\[r2,q\]≥λ\{12​α¯​λ​ξσ​σ′​\(r2\)\+12​α¯​∫r211γ​\(t\)​ξσ​σ′′​\(t\)​d​t\+12​∫r21γ​\(t\)​d​t\}\.\\displaystyle\\xi\_\{\\varphi\\sigma\}\(r\)\+\\inf\_\{\\begin\{subarray\}\{c\}\\gamma,\\lambda\\geq 0\\\\ \\gamma\|\_\{\[r^\{2\},q\]\}\\geq\\lambda\\end\{subarray\}\}\\bigg\\\{\\frac\{1\}\{2\\overline\{\\alpha\}\\lambda\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{1\}\{\\gamma\(t\)\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\\textup\{d\}t\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t\\bigg\\\}\\,\.Therefore,

supq∈\[r2,1\]𝒱α¯AMP​\(r,q\)≤ξφ​σ​\(r\)\+infγ\|\[r2,1\]≥λ≥0\{12​α¯​λ​ξσ​σ′​\(r2\)\+12​α¯​∫r211γ​\(t\)​ξσ​σ′′​\(t\)​d​t\+12​∫r21γ​\(t\)​d​t\}\.\\displaystyle\\sup\_\{q\\in\[r^\{2\},1\]\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\inf\_\{\\gamma\|\_\{\[r^\{2\},1\]\}\\geq\\lambda\\geq 0\}\\bigg\\\{\\frac\{1\}\{2\\overline\{\\alpha\}\\lambda\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{1\}\{\\gamma\(t\)\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\\textup\{d\}t\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t\\bigg\\\}\\,\.To compute this infimum, let us consider two cases:

1. 1\.λ\>ξσ​σ′′​\(1\)/α¯\\lambda\>\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)/\\overline\{\\alpha\}\}\. Then the optimalγ\\gammais justλ\\lambda, and we obtain that supq∈\[r2,1\]𝒱α¯AMP​\(r,q\)≤ξφ​σ​\(r\)\+infλ\>ξσ​σ′′​\(1\)/α¯\{12​λ​α¯​ξσ​σ′​\(1\)\+λ2​\(1−r2\)\}\.\\displaystyle\\sup\_\{q\\in\[r^\{2\},1\]\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\inf\_\{\\lambda\>\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)/\\overline\{\\alpha\}\}\}\\bigg\\\{\\frac\{1\}\{2\\lambda\\overline\{\\alpha\}\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\+\\frac\{\\lambda\}\{2\}\(1\-r^\{2\}\)\\bigg\\\}\\,\.Ifξσ​σ′​\(1\)\>ξσ​σ′′​\(1\)​\(1−r2\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\>\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)\(1\-r^\{2\}\), the above infimum is achieved atλ=ξσ​σ′​\(1\)/α¯​\(1−r2\)\\lambda=\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)/\\overline\{\\alpha\}\(1\-r^\{2\}\)\}, and we obtain the upper bound supq∈\[r2,1\]𝒱α¯AMP​\(r,q\)≤ξφ​σ​\(r\)\+\(1−r2\)​ξσ​σ′​\(1\)α¯=𝒱α¯∗​\(r,1\)=𝒱α¯∗​\(r,q∗​\(r\)\),\\sup\_\{q\\in\[r^\{2\},1\]\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\sqrt\{\\frac\{\(1\-r^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\}\{\\overline\{\\alpha\}\}\}\\,=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,1\)=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\),whereq∗​\(r\)=1q\_\{\*\}\(r\)=1follows from[D\.4](https://arxiv.org/html/2606.28573#A4.Thmlem4)\. Otherwise, the infimum is achieved atλ=ξσ​σ′′​\(1\)/α¯\\lambda=\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)/\\overline\{\\alpha\}\}, which will be considered in the next case\.
2. 2\.λ≤ξσ​σ′′​\(1\)/α¯\\lambda\\leq\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)/\\overline\{\\alpha\}\}\. In this case, there exists somes∈\[r2,1\]s\\in\[r^\{2\},1\]such thatλ=ξσ​σ′′​\(s\)/α¯\\lambda=\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(s\)/\\overline\{\\alpha\}\}, and the optimalγ\\gammasatisfies γ​\(t\)=𝟏t≥s⋅ξσ​σ′′​\(t\)α¯\+𝟏t<s⋅λ\.\\gamma\(t\)=\\mathrm\{\\bf 1\}\_\{t\\geq s\}\\cdot\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\{\\overline\{\\alpha\}\}\}\+\\mathrm\{\\bf 1\}\_\{t<s\}\\cdot\\lambda\.We similarly obtain that supq∈\[r2,1\]𝒱α¯AMP​\(r,q\)≤ξφ​σ​\(r\)\+ξσ​σ′​\(s\)2​α¯​ξσ​σ′′​\(s\)\+ξσ​σ′′​\(s\)2​α¯​\(s−r2\)\+1α¯​∫s1ξσ​σ′′​\(t\)​d​t\.\\displaystyle\\sup\_\{q\\in\[r^\{2\},1\]\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(s\)\}\{2\\sqrt\{\\overline\{\\alpha\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(s\)\}\}\+\\frac\{\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(s\)\}\}\{2\\sqrt\{\\overline\{\\alpha\}\}\}\\left\(s\-r^\{2\}\\right\)\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{s\}^\{1\}\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\\textup\{d\}t\.Differentiating the above right\-hand side with respect tossand using[D\.4](https://arxiv.org/html/2606.28573#A4.Thmlem4), we deduce that it is minimized ats=q∗​\(r\)s=q\_\{\*\}\(r\)\. Hence, supq∈\[r2,1\]𝒱α¯AMP​\(r,q\)≤ξφ​σ​\(r\)\+\(s−r2\)​ξσ​σ′​\(s\)α¯\+1α¯​∫s1ξσ​σ′′​\(t\)​d​t=𝒱α¯∗​\(r,q∗​\(r\)\)\.\\sup\_\{q\\in\[r^\{2\},1\]\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\sqrt\{\\frac\{\(s\-r^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(s\)\}\{\\overline\{\\alpha\}\}\}\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{s\}^\{1\}\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\\textup\{d\}t=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\.

Combining the two cases and[D\.4](https://arxiv.org/html/2606.28573#A4.Thmlem4), we conclude thatsupq∈\[r2,1\]𝒱α¯AMP​\(r,q\)≤𝒱α¯∗​\(r,q∗​\(r\)\)\\sup\_\{q\\in\[r^\{2\},1\]\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\)\\leq\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\. This completes the proof of part\(b\)\(b\)\.

Proof of\(c\)\(c\)\.We first show that the set\{r∈\[−1,1\]:\(r,q∗​\(r\)\)∈ABayes\}\\\{r\\in\[\-1,1\]:\(r,q\_\{\*\}\(r\)\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\\}is an interval\. By definition,

\(r,q∗​\(r\)\)∈ABayes⟺q∗​\(r\)≥\(1\+cBayes−2\)​r2:=qBayes​\(r\)\.\(r,q\_\{\*\}\(r\)\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\Longleftrightarrow q\_\{\*\}\(r\)\\geq\\left\(1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\-2\}\\right\)r^\{2\}:=q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)\.Ifq∗​\(r\)=1q\_\{\*\}\(r\)=1, this is equivalent to\(r,1\)∈ABayes\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, which is already enforced by the constraint\(r,q∗​\(r\)\)∈ABayes\(r,q\_\{\*\}\(r\)\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. Now we consider the caseq∗​\(r\)<1q\_\{\*\}\(r\)<1\. According to[D\.4](https://arxiv.org/html/2606.28573#A4.Thmlem4),q∗​\(r\)q\_\{\*\}\(r\)uniquely solves the equation

r2=q−ξσ​σ′​\(q\)ξσ​σ′′​\(q\)\.r^\{2\}=\\,q\-\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)\}\.By direct calculation, we can verify that the mappingq↦q−ξσ​σ′​\(q\)/ξσ​σ′′​\(q\)q\\mapsto q\-\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)/\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)is monotone increasing\. Therefore,q∗​\(r\)≥qBayes​\(r\)q\_\{\*\}\(r\)\\geq q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)is equivalent to

qBayes​\(r\)−ξσ​σ′​\(qBayes​\(r\)\)ξσ​σ′′​\(qBayes​\(r\)\)≤r2⟺qBayes​\(r\)​ξσ​σ′′​\(qBayes​\(r\)\)ξσ​σ′​\(qBayes​\(r\)\)≤1\+cBayes2\.q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)\-\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)\)\}\\leq\\,r^\{2\}\\Longleftrightarrow\\frac\{q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)\)\}\\leq\\,1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\.It suffices to show that the set ofqBayes​\(r\)q\_\{\\mbox\{\\tiny\\rm Bayes\}\}\(r\)’s satisfying the above inequality is an interval\. Note that

q​ξσ​σ′′​\(q\)ξσ​σ′​\(q\)≤1\+cBayes2⟺∑k≥2k​\(k−1\)​σk2​qk−1≤\(1\+cBayes2\)​∑k≥1k​σk2​qk−1\\displaystyle\\frac\{q\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\\leq\\,1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\\Longleftrightarrow\\,\\sum\_\{k\\geq 2\}k\(k\-1\)\\sigma\_\{k\}^\{2\}q^\{k\-1\}\\leq\\,\\left\(1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\\right\)\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}^\{2\}q^\{k\-1\}⟺\\displaystyle\\Longleftrightarrow\\,∑k≥2\+cBayes2k​\(k−2−cBayes2\)​σk2​qk−1≤\(1\+cBayes2\)​σ12\+∑2≤k<2\+cBayes2k​\(2\+cBayes2−k\)​σk2​qk−1\.\\displaystyle\\sum\_\{k\\geq 2\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\}k\\left\(k\-2\-c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\\right\)\\sigma\_\{k\}^\{2\}q^\{k\-1\}\\leq\\,\\left\(1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\\right\)\\sigma\_\{1\}^\{2\}\+\\sum\_\{2\\leq k<2\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\}k\\left\(2\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\-k\\right\)\\sigma\_\{k\}^\{2\}q^\{k\-1\}\.Of course, ifqqsatisfies the above inequality, then anyq′≤qq^\{\\prime\}\\leq qalso satisfies it\. This proves our claim\.

We can now denote\{r∈\[−1,1\]:\(r,q∗​\(r\)\)∈ABayes\}=\[−rA,rA\]\\\{r\\in\[\-1,1\]:\(r,q\_\{\*\}\(r\)\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\\}=\[\-r\_\{A\},r\_\{A\}\]\. By continuity, we must have

rAq∗​\(rA\)−rA2=cBayes\.\\frac\{r\_\{A\}\}\{\\sqrt\{q\_\{\*\}\(r\_\{A\}\)\-r\_\{A\}^\{2\}\}\}=c\_\{\\mbox\{\\tiny\\rm Bayes\}\}\.We first derive the first\-order condition satisfied byr∗r\_\{\*\}\. On the one hand, in the region\{r:\(r,1\)∈ABayes,q∗​\(r\)=1\}\\\{r:\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\},q\_\{\*\}\(r\)=1\\\}we have

d​𝒱α¯∗d​r​\(r,q∗​\(r\)\)=ξφ​σ′​\(r\)−ξσ​σ′​\(1\)α¯​r1−r2\.\\displaystyle\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r,q\_\{\*\}\(r\)\)=\\,\\xi\_\{\\varphi\\sigma\}^\{\\prime\}\(r\)\-\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\}\{\\overline\{\\alpha\}\}\}\\frac\{r\}\{\\sqrt\{1\-r^\{2\}\}\}\.On the other hand, in the region\{r:\(r,1\)∈ABayes,q∗​\(r\)<1\}\\\{r:\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\},q\_\{\*\}\(r\)<1\\\}, by the envelope theorem we have

d​𝒱α¯∗d​r​\(r,q∗​\(r\)\)=ξφ​σ′​\(r\)−ξσ​σ′​\(q∗​\(r\)\)α¯​rq∗​\(r\)−r2\.\\displaystyle\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r,q\_\{\*\}\(r\)\)=\\,\\xi\_\{\\varphi\\sigma\}^\{\\prime\}\(r\)\-\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\(r\)\)\}\{\\overline\{\\alpha\}\}\}\\frac\{r\}\{\\sqrt\{q\_\{\*\}\(r\)\-r^\{2\}\}\}\.To summarize, we always have

d​𝒱α¯∗d​r​\(r,q∗​\(r\)\)=ξφ​σ′​\(r\)−ξσ​σ′​\(q∗​\(r\)\)α¯​rq∗​\(r\)−r2\.\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r,q\_\{\*\}\(r\)\)=\\,\\xi\_\{\\varphi\\sigma\}^\{\\prime\}\(r\)\-\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\(r\)\)\}\{\\overline\{\\alpha\}\}\}\\frac\{r\}\{\\sqrt\{q\_\{\*\}\(r\)\-r^\{2\}\}\}\.By definition ofcBayesc\_\{\\mbox\{\\tiny\\rm Bayes\}\}and Cauchy\-Schwarz inequality \(cf\.[D\.5](https://arxiv.org/html/2606.28573#A4.Thmlem5)\):

d​𝒱α¯∗d​r​\(rA,q∗​\(rA\)\)=ξφ​σ′​\(rA\)−ξσ​σ′​\(q∗​\(rA\)\)α¯​rAq∗​\(rA\)−rA2\\displaystyle\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r\_\{A\},q\_\{\*\}\(r\_\{A\}\)\)=\\,\\xi\_\{\\varphi\\sigma\}^\{\\prime\}\(r\_\{A\}\)\-\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\(r\_\{A\}\)\)\}\{\\overline\{\\alpha\}\}\}\\frac\{r\_\{A\}\}\{\\sqrt\{q\_\{\*\}\(r\_\{A\}\)\-r\_\{A\}^\{2\}\}\}≤\\displaystyle\\leq\\,ξσ​σ′​\(q∗​\(rA\)\)​ξφ​φ′​\(rA2q∗​\(rA\)\)−ξσ​σ′​\(q∗​\(rA\)\)α¯​rAq∗​\(rA\)−rA2\\displaystyle\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\(r\_\{A\}\)\)\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{r\_\{A\}^\{2\}\}\{q\_\{\*\}\(r\_\{A\}\)\}\\right\)\}\-\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\(r\_\{A\}\)\)\}\{\\overline\{\\alpha\}\}\}\\frac\{r\_\{A\}\}\{\\sqrt\{q\_\{\*\}\(r\_\{A\}\)\-r\_\{A\}^\{2\}\}\}=\\displaystyle=\\,ξσ​σ′​\(q∗​\(rA\)\)​\(ξφ​φ′​\(cBayes21\+cBayes2\)−cBayesα¯\)=0\.\\displaystyle\\sqrt\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\(r\_\{A\}\)\)\}\\left\(\\sqrt\{\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\}\{1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\}\\right\)\}\-\\frac\{c\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\right\)=0\.Similarly, we can show that

d​𝒱α¯∗d​r​\(−rA,q∗​\(−rA\)\)≥0\.\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(\-r\_\{A\},q\_\{\*\}\(\-r\_\{A\}\)\)\\geq\\,0\.Sincer∗=arg⁡max\[−rA,rA\]⁡𝒱α¯∗​\(r,q∗​\(r\)\)r\_\{\*\}=\\arg\\max\_\{\[\-r\_\{A\},r\_\{A\}\]\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\), we must have

0=d​𝒱α¯∗d​r​\(r∗,q∗​\(r∗\)\)=ξφ​σ′​\(r∗\)−ξσ​σ′​\(q∗\)α¯​r∗q∗−r∗2\.0=\\,\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r\_\{\*\},q\_\{\*\}\(r\_\{\*\}\)\)=\\xi\_\{\\varphi\\sigma\}^\{\\prime\}\(r\_\{\*\}\)\-\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}\{\\overline\{\\alpha\}\}\}\\frac\{r\_\{\*\}\}\{\\sqrt\{q\_\{\*\}\-r\_\{\*\}^\{2\}\}\}\.Further,q∗=q∗​\(r\)q\_\{\*\}=q\_\{\*\}\(r\)implies that\(q∗−r∗2\)​ξσ​σ′′​\(q∗\)≤ξσ​σ′​\(q∗\)\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\_\{\*\}\)\\leq\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\.

It now remains to show that

T​\(r∗,q∗\)=1α¯​\(q∗−r∗2\)​ξσ​σ′​\(q∗\)\.T\(r\_\{\*\},q\_\{\*\}\)=\\,\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\sqrt\{\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}\\,\.In[Eq\.83](https://arxiv.org/html/2606.28573#A4.E83), the choice of\{fk,1\}k=0∞\\\{f\_\{k,1\}\\\}\_\{k=0\}^\{\\infty\}achieving\(q∗−r∗2\)​ξσ​σ′​\(q∗\)/α¯\\sqrt\{\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)/\\overline\{\\alpha\}\}is given by

fk,1=α¯​\(q∗−r∗2\)ξσ​σ′​\(q∗\)​k\+1​σk\+1​q∗k/2\.f\_\{k,1\}=\\,\\sqrt\{\\frac\{\\overline\{\\alpha\}\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}\}\\sqrt\{k\+1\}\\sigma\_\{k\+1\}q\_\{\*\}^\{k/2\}\.It suffices to verify the first and third constraints\. For the first one, we have

∑k=0∞k\+1​φk\+1​fk,1​\(r∗q∗\)k=α¯​\(q∗−r∗2\)ξσ​σ′​\(q∗\)​∑k=0∞\(k\+1\)​φk\+1​σk\+1​r∗k=α¯​\(q∗−r∗2\)ξσ​σ′​\(q∗\)​ξφ​σ′​\(r∗\)=r∗,\\displaystyle\\sum\_\{k=0\}^\{\\infty\}\\sqrt\{k\+1\}\\varphi\_\{k\+1\}f\_\{k,1\}\\left\(\\frac\{r\_\{\*\}\}\{\\sqrt\{q\_\{\*\}\}\}\\right\)^\{k\}=\\,\\sqrt\{\\frac\{\\overline\{\\alpha\}\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}\}\\sum\_\{k=0\}^\{\\infty\}\(k\+1\)\\varphi\_\{k\+1\}\\sigma\_\{k\+1\}r\_\{\*\}^\{k\}=\\,\\sqrt\{\\frac\{\\overline\{\\alpha\}\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}\}\\xi\_\{\\varphi\\sigma\}^\{\\prime\}\(r\_\{\*\}\)=r\_\{\*\},where the last equality is due to the first\-order condition forr∗r\_\{\*\}\. The third constraint follows from the definition ofq∗q\_\{\*\}:

1α¯​q∗​∑k=0∞k​fk,12≤\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}q\_\{\*\}\}\\sum\_\{k=0\}^\{\\infty\}kf\_\{k,1\}^\{2\}\\leq\\,1α¯​q∗​α¯​\(q∗−r∗2\)ξσ​σ′​\(q∗\)​∑k=0∞k​\(k\+1\)​σk\+12​q∗k\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}q\_\{\*\}\}\\frac\{\\overline\{\\alpha\}\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}\\sum\_\{k=0\}^\{\\infty\}k\(k\+1\)\\sigma\_\{k\+1\}^\{2\}q\_\{\*\}^\{k\}=\\displaystyle=\\,1α¯​q∗​α¯​\(q∗−r∗2\)ξσ​σ′​\(q∗\)​q∗​ξσ​σ′′​\(q∗\)=\(q∗−r∗2\)​ξσ​σ′′​\(q∗\)ξσ​σ′​\(q∗\)≤1\.\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}q\_\{\*\}\}\\frac\{\\overline\{\\alpha\}\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}q\_\{\*\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\_\{\*\}\)=\\frac\{\(q\_\{\*\}\-r\_\{\*\}^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\_\{\*\}\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\)\}\\leq 1\.This establishes𝒱α¯AMP​\(r∗,q∗\)=𝒱α¯∗​\(r∗,q∗\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r\_\{\*\},q\_\{\*\}\)=\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r\_\{\*\},q\_\{\*\}\)\. Finally, the “as a consequence” part follows automatically, completing the proof of part\(c\)\(c\)\. ∎

###### Proof of[5\.3](https://arxiv.org/html/2606.28573#S5.Thmlem3)\.

From the proof of[5\.1](https://arxiv.org/html/2606.28573#S5.Thmthm1), we know that

\{r∈\[−1,1\]:\(r,q∗​\(r\)\)∈ABayes\}=\[−rA,rA\]\.\\\{r\\in\[\-1,1\]:\(r,q\_\{\*\}\(r\)\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\\}=\[\-r\_\{A\},r\_\{A\}\]\.Further, we have

\{r∈\[−1,1\]:\(r,1\)∈ABayes\}=\[−rB,rB\],rB=cBayes1\+cBayes2\.\\\{r\\in\[\-1,1\]:\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\\\}=\[\-r\_\{B\},r\_\{B\}\],\\quad r\_\{B\}=\\frac\{c\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\{\\sqrt\{1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\}\}\.Therefore,rA=rBr\_\{A\}=r\_\{B\}if and only ifq∗​\(rB\)=1q\_\{\*\}\(r\_\{B\}\)=1, which \(by[D\.4](https://arxiv.org/html/2606.28573#A4.Thmlem4)\) is equivalent to

ξσ​σ′​\(1\)≥ξσ​σ′′​\(1\)​\(1−rB2\)=11\+cBayes2​ξσ​σ′′​\(1\)\.\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\\geq\\,\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)\(1\-r\_\{B\}^\{2\}\)=\\frac\{1\}\{1\+c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)\.By definition,cBayesc\_\{\\mbox\{\\tiny\\rm Bayes\}\}is an increasing function ofα¯\\overline\{\\alpha\}\. Therefore, the above equation holds for large enoughα¯\\overline\{\\alpha\}, with the threshold depending only onφ\\varphiandσ\\sigma\. This completes the proof\. ∎

### D\.3Appendix for[Section5\.4](https://arxiv.org/html/2606.28573#S5.SS4)

#### D\.3\.1Parisi formula

From a statistical physics perspective, the objective functionH^dg​\(𝒘\)\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)for generalized tensor PCA can be viewed as the Hamiltonian of a spherical mixedpp\-spin model with a planted signal𝒘∗\\text\{$w$\}\_\{\*\}\. In this section, we derive the Parisi formula for predicting the asymptotic global maximum ofH^dg​\(𝒘\)\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)over the unit sphere𝕊d−1\\mathbb\{S\}^\{d\-1\}, building on foundational results from spin glass theory\[[CS17](https://arxiv.org/html/2606.28573#bib.bibx11),[JT17](https://arxiv.org/html/2606.28573#bib.bibx24)\]\.

Let𝒘=r​𝒘∗\+1−r2​𝒘⟂\\text\{$w$\}=r\\text\{$w$\}\_\{\*\}\+\\sqrt\{1\-r^\{2\}\}\\text\{$w$\}\_\{\\perp\}, with𝒘⟂∈𝕊d−1\\text\{$w$\}\_\{\\perp\}\\in\\mathbb\{S\}^\{d\-1\},𝒘⟂⟂𝒘∗\\text\{$w$\}\_\{\\perp\}\\perp\\text\{$w$\}\_\{\*\}, we obtain that

H^dg​\(𝒘\)=∑k≥1σk​φk​⟨𝒘∗,𝒘⟩k\+1d​α¯​∑k≥1σk​⟨𝑮\(k\),𝒘⊗k⟩:=ξφ​σ​\(r\)\+1d​Hσ,α¯​\(𝒘\),\\displaystyle\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)=\\,\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\varphi\_\{k\}\\langle\\text\{$w$\}\_\{\*\},\\text\{$w$\}\\rangle^\{k\}\+\\frac\{1\}\{\\sqrt\{d\\overline\{\\alpha\}\}\}\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\langle\\text\{$G$\}^\{\(k\)\},\\text\{$w$\}^\{\\otimes k\}\\rangle:=\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{\\sqrt\{d\}\}H\_\{\\sigma,\\overline\{\\alpha\}\}\(\\text\{$w$\}\),whereHσ,α¯H\_\{\\sigma,\\overline\{\\alpha\}\}is a mean\-zero Gaussian process on the unit sphere, with covariance

𝔼​\[Hσ,α¯​\(𝒘1\)​Hσ,α¯​\(𝒘2\)\]=1α¯​ξσ​σ​\(⟨𝒘1,𝒘2⟩\)\.\\mathbb\{E\}\\left\[H\_\{\\sigma,\\overline\{\\alpha\}\}\(\\text\{$w$\}\_\{1\}\)H\_\{\\sigma,\\overline\{\\alpha\}\}\(\\text\{$w$\}\_\{2\}\)\\right\]=\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\\xi\_\{\\sigma\\sigma\}\(\\langle\\text\{$w$\}\_\{1\},\\text\{$w$\}\_\{2\}\\rangle\)\.It thus follows that

max𝒘∈𝕊d−1⁡H^dg​\(𝒘\)=maxr∈\[−1,1\]𝒘⟂∈𝕊d−1,𝒘⟂⟂𝒘∗⁡\{ξφ​σ​\(r\)\+1d​Hσ,α¯​\(r​𝒘∗\+1−r2​𝒘⟂\)\}\.\\displaystyle\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)=\\max\_\{\\begin\{subarray\}\{c\}r\\in\[\-1,1\]\\\\ \\text\{$w$\}\_\{\\perp\}\\in\\mathbb\{S\}^\{d\-1\},\\,\\text\{$w$\}\_\{\\perp\}\\perp\\text\{$w$\}\_\{\*\}\\end\{subarray\}\}\\left\\\{\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{\\sqrt\{d\}\}H\_\{\\sigma,\\overline\{\\alpha\}\}\\left\(r\\text\{$w$\}\_\{\*\}\+\\sqrt\{1\-r^\{2\}\}\\text\{$w$\}\_\{\\perp\}\\right\)\\right\\\}\.Fixr∈\[−1,1\]r\\in\[\-1,1\], for any𝒘⟂,𝒘⟂′∈𝕊d−1\\text\{$w$\}\_\{\\perp\},\\text\{$w$\}\_\{\\perp\}^\{\\prime\}\\in\\mathbb\{S\}^\{d\-1\}orthogonal to𝒘∗\\text\{$w$\}\_\{\*\}, we have

𝔼​\[Hσ,α¯​\(r​𝒘∗\+1−r2​𝒘⟂\)​Hσ,α¯​\(r​𝒘∗\+1−r2​𝒘⟂′\)\]=1α¯​ξσ​σ​\(r2\+\(1−r2\)​⟨𝒘⟂,𝒘⟂′⟩\)\.\\mathbb\{E\}\\left\[H\_\{\\sigma,\\overline\{\\alpha\}\}\\left\(r\\text\{$w$\}\_\{\*\}\+\\sqrt\{1\-r^\{2\}\}\\text\{$w$\}\_\{\\perp\}\\right\)H\_\{\\sigma,\\overline\{\\alpha\}\}\\left\(r\\text\{$w$\}\_\{\*\}\+\\sqrt\{1\-r^\{2\}\}\\text\{$w$\}\_\{\\perp\}^\{\\prime\}\\right\)\\right\]=\\frac\{1\}\{\\overline\{\\alpha\}\}\\xi\_\{\\sigma\\sigma\}\(r^\{2\}\+\(1\-r^\{2\}\)\\langle\\text\{$w$\}\_\{\\perp\},\\text\{$w$\}\_\{\\perp\}^\{\\prime\}\\rangle\)\.Applying\[[CS17](https://arxiv.org/html/2606.28573#bib.bibx11), Theorem 1\]then implies that almost surely,

limd→∞max𝒘⟂∈𝕊d−1,𝒘⟂⟂𝒘∗⁡1d​Hσ,α¯​\(r​𝒘∗\+1−r2​𝒘⟂\)=minΓ∈𝒢⁡𝖴α¯​\(Γ,r\),\\displaystyle\\lim\_\{d\\to\\infty\}\\max\_\{\\text\{$w$\}\_\{\\perp\}\\in\\mathbb\{S\}^\{d\-1\},\\,\\text\{$w$\}\_\{\\perp\}\\perp\\text\{$w$\}\_\{\*\}\}\\frac\{1\}\{\\sqrt\{d\}\}H\_\{\\sigma,\\overline\{\\alpha\}\}\\left\(r\\text\{$w$\}\_\{\*\}\+\\sqrt\{1\-r^\{2\}\}\\text\{$w$\}\_\{\\perp\}\\right\)=\\,\\min\_\{\\Gamma\\in\\mathscr\{G\}\}\\mathsf\{U\}\_\{\\overline\{\\alpha\}\}\(\\Gamma,r\),where

𝖴α¯​\(Γ,r\):=Γ​\(0\)2​α¯​\(1−r2\)​ξσ​σ′​\(r2\)\+\(1−r2\)22​α¯​∫01Γ​\(t\)​ξσ​σ′′​\(r2\+\(1−r2\)​t\)​d​t\+12​∫01d​tΓ​\(t\),\\displaystyle\\mathsf\{U\}\_\{\\overline\{\\alpha\}\}\(\\Gamma,r\):=\\,\\frac\{\\Gamma\(0\)\}\{2\\overline\{\\alpha\}\}\(1\-r^\{2\}\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\+\\frac\{\(1\-r^\{2\}\)^\{2\}\}\{2\\overline\{\\alpha\}\}\\int\_\{0\}^\{1\}\\Gamma\(t\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(r^\{2\}\+\(1\-r^\{2\}\)t\)\\textup\{d\}t\+\\frac\{1\}\{2\}\\int\_\{0\}^\{1\}\\frac\{\\textup\{d\}t\}\{\\Gamma\(t\)\},𝒢:=\{Γ:\[0,1\)→ℝ\>0:Γ​\(t\)=L−∫0tα​\(s\)​d​s,α​non\-decreasing, right\-continuous, integrable\}\.\\displaystyle\\mathscr\{G\}:=\\,\\Big\\\{\\Gamma:\[0,1\)\\to\\mathbb\{R\}\_\{\>0\}:\\Gamma\(t\)=L\-\\int\_\{0\}^\{t\}\\alpha\(s\)\\,\\textup\{d\}s,\\,\\alpha\\mbox\{ non\-decreasing, right\-continuous, integrable\}\\Big\\\}\.Define fort∈\[r2,1\]t\\in\[r^\{2\},1\]:

γ​\(t\)=α¯1−r2​Γ​\(t−r21−r2\)−1,\\gamma\(t\)=\\,\\frac\{\\overline\{\\alpha\}\}\{1\-r^\{2\}\}\\Gamma\\left\(\\frac\{t\-r^\{2\}\}\{1\-r^\{2\}\}\\right\)^\{\-1\},we can rewrite𝖴α¯\\mathsf\{U\}\_\{\\overline\{\\alpha\}\}in terms ofγ\\gammaandrr:

𝖴α¯​\(γ,r\):=\\displaystyle\\mathsf\{U\}\_\{\\overline\{\\alpha\}\}\(\\gamma,r\):=\\,ξσ​σ′​\(r2\)2​γ​\(r2\)\+1−r22​∫01ξσ​σ′′​\(r2\+\(1−r2\)​t\)γ​\(r2\+\(1−r2\)​t\)​d​t\+1−r22​α¯​∫01γ​\(r2\+\(1−r2\)​t\)​d​t\\displaystyle\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\}\{2\\gamma\(r^\{2\}\)\}\+\\frac\{1\-r^\{2\}\}\{2\}\\int\_\{0\}^\{1\}\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(r^\{2\}\+\(1\-r^\{2\}\)t\)\}\{\\gamma\(r^\{2\}\+\(1\-r^\{2\}\)t\)\}\\textup\{d\}t\+\\frac\{1\-r^\{2\}\}\{2\\overline\{\\alpha\}\}\\int\_\{0\}^\{1\}\\gamma\(r^\{2\}\+\(1\-r^\{2\}\)t\)\\textup\{d\}t=\\displaystyle=\\,ξσ​σ′​\(r2\)2​γ​\(r2\)\+12​∫r21ξσ​σ′′​\(t\)γ​\(t\)​d​t\+12​α¯​∫r21γ​\(t\)​d​t\.\\displaystyle\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\}\{2\\gamma\(r^\{2\}\)\}\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\{\\gamma\(t\)\}\\textup\{d\}t\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t\.Further, by definition of𝒢\\mathscr\{G\}, we know thatγ\\gammahas the formγ​\(t\)=1/\(c\+∫t1μ​\(s\)​d​s\)\\gamma\(t\)=1/\(c\+\\int\_\{t\}^\{1\}\\mu\(s\)\\textup\{d\}s\), wherec\>0c\>0,μ\\muis non\-decreasing, right continuous, and integrable\. We finally obtain the prediction

limd→∞max𝒘∈𝕊d−1⁡H^dg​\(𝒘\)=\\displaystyle\\lim\_\{d\\to\\infty\}\\max\_\{\\text\{$w$\}\\in\\mathbb\{S\}^\{d\-1\}\}\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)=\\,supr∈\[−1,1\]inf\(μ,c\)∈𝒰×ℝ\>0\{ξφ​σ​\(r\)\+ξσ​σ′​\(r2\)2​γ​\(r2\)\+12​∫r21ξσ​σ′′​\(t\)γ​\(t\)​d​t\+12​α¯​∫r21γ​\(t\)​d​t\}\\displaystyle\\sup\_\{r\\in\[\-1,1\]\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\left\\\{\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(r^\{2\}\)\}\{2\\gamma\(r^\{2\}\)\}\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)\}\{\\gamma\(t\)\}\\textup\{d\}t\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t\\right\\\}=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,supr∈\[−1,1\]inf\(μ,c\)∈𝒰×ℝ\>0\{ξφ​σ​\(r\)\+ξσ​σ′​\(1\)2​γ​\(1\)\+12​∫r21μ​\(t\)​ξσ​σ′​\(t\)​d​t\+12​α¯​∫r21γ​\(t\)​d​t\}\\displaystyle\\sup\_\{r\\in\[\-1,1\]\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\left\\\{\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\}\{2\\gamma\(1\)\}\+\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\mu\(t\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(t\)\\textup\{d\}t\+\\frac\{1\}\{2\\overline\{\\alpha\}\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t\\right\\\}=\(i​i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(ii\)\}\}\{\{=\}\}\\,supr∈\[−1,1\]inf\(μ,c\)∈𝒰×ℝ\>0𝖥¯α¯​\(μ,c,r\),\\displaystyle\\sup\_\{r\\in\[\-1,1\]\}\\inf\_\{\(\\mu,c\)\\in\\mathscr\{U\}\\times\\mathbb\{R\}\_\{\>0\}\}\\overline\{\\mathsf\{F\}\}\_\{\\overline\{\\alpha\}\}\(\\mu,c,r\),where in\(i\)\(i\)we use integration by parts, and\(i​i\)\(ii\)follows from the definition of𝖥¯α¯\\overline\{\\mathsf\{F\}\}\_\{\\overline\{\\alpha\}\}in[Eq\.36](https://arxiv.org/html/2606.28573#S5.E36)\. This matches the Parisi formula for the single\-index model derived in[5\.1](https://arxiv.org/html/2606.28573#S5.Thmconj1)\.

#### D\.3\.2Value ofH^dg\\widehat\{H\}\_\{d\}^\{g\}achieved by a two\-stage algorithm

In this section, we describe and analyze a two\-stage algorithm to optimizeH^dg​\(𝒘\)\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)over the unit sphere\. Similar to our AMP algorithm for the single\-index model, the first stage consists of several tensor AMP iterations\. For the second stage, we use the Hessian ascent algorithm of\[[Sub21](https://arxiv.org/html/2606.28573#bib.bibx38)\]\.

Before presenting the actual tensor AMP algorithm, we need to introduce its state evolution, a set of recursive equations characterizing its asymptotic behavior asd→∞d\\to\\infty\. Let\{ak​\(ℓ\)\}k≥1,ℓ≥0\\\{a\_\{k\}\(\\ell\)\\\}\_\{k\\geq 1,\\,\\ell\\geq 0\}be a sequence of real numbers\. Forℓ,ℓ1,ℓ2≥0\\ell,\\ell\_\{1\},\\ell\_\{2\}\\geq 0, define

ξa​φℓ​\(t\)=∑k≥1ak​\(ℓ\)​φk​tk,ξa​aℓ1,ℓ2​\(t\)=∑k≥1ak​\(ℓ1\)​ak​\(ℓ1\)​tk\.\\xi\_\{a\\varphi\}^\{\\ell\}\(t\)=\\,\\sum\_\{k\\geq 1\}a\_\{k\}\(\\ell\)\\varphi\_\{k\}t^\{k\},\\quad\\xi\_\{aa\}^\{\\ell\_\{1\},\\ell\_\{2\}\}\(t\)=\\,\\sum\_\{k\\geq 1\}a\_\{k\}\(\\ell\_\{1\}\)a\_\{k\}\(\\ell\_\{1\}\)t^\{k\}\.Letp0p\_\{0\}be a probability distribution onℝ\\mathbb\{R\}andM0∼p0M^\{0\}\\sim p\_\{0\}\. For eachℓ≥1\\ell\\geq 1, letM≤ℓ=\(M1,⋯,Mℓ\)∈ℝℓM^\{\\leq\\ell\}=\(M^\{1\},\\cdots,M^\{\\ell\}\)\\in\\mathbb\{R\}^\{\\ell\}be jointly Gaussian with zero mean, independent ofM0M^\{0\}\. Further, letG∼𝖭​\(0,1\)G\\sim\{\\sf N\}\(0,1\)be independent of\{Mℓ\}ℓ≥0\\\{M^\{\\ell\}\\\}\_\{\\ell\\geq 0\}\. The covariance matrix ofM≤ℓM^\{\\leq\\ell\}is specified as follows:

1. 1\.Define the sequence\{rℓ\}ℓ=0∞\\\{r\_\{\\ell\}\\\}\_\{\\ell=0\}^\{\\infty\}iteratively by r0=0,rℓ\+1=\(ξa​φℓ\)′​\(rℓ\)\.r\_\{0\}=0,\\quad r\_\{\\ell\+1\}=\(\\xi\_\{a\\varphi\}^\{\\ell\}\)^\{\\prime\}\(r\_\{\\ell\}\)\.For eachℓ≥0\\ell\\geq 0, defineZℓ=rℓ​G\+MℓZ^\{\\ell\}=r\_\{\\ell\}G\+M^\{\\ell\}\.
2. 2\.The covariancesQℓ1,ℓ2=𝔼​\[Mℓ1​Mℓ2\]Q\_\{\\ell\_\{1\},\\ell\_\{2\}\}=\\mathbb\{E\}\[M^\{\\ell\_\{1\}\}M^\{\\ell\_\{2\}\}\]are defined by the recursion: Qℓ1\+1,ℓ2\+1=1α¯​\(ξa​aℓ1,ℓ2\)′​\(𝔼​\[Zℓ1​Zℓ2\]\)=1α¯​\(ξa​aℓ1,ℓ2\)′​\(rℓ1​rℓ2\+Qℓ1,ℓ2\),ℓ1,ℓ2≥0,Q\_\{\\ell\_\{1\}\+1,\\ell\_\{2\}\+1\}=\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\(\\xi\_\{aa\}^\{\\ell\_\{1\},\\ell\_\{2\}\}\)^\{\\prime\}\(\\mathbb\{E\}\[Z^\{\\ell\_\{1\}\}Z^\{\\ell\_\{2\}\}\]\)=\\frac\{1\}\{\\overline\{\\alpha\}\}\(\\xi\_\{aa\}^\{\\ell\_\{1\},\\ell\_\{2\}\}\)^\{\\prime\}\(r\_\{\\ell\_\{1\}\}r\_\{\\ell\_\{2\}\}\+Q\_\{\\ell\_\{1\},\\ell\_\{2\}\}\),\\quad\\ell\_\{1\},\\ell\_\{2\}\\geq 0,whereQℓ1,ℓ2=0Q\_\{\\ell\_\{1\},\\ell\_\{2\}\}=0ifℓ1=0\\ell\_\{1\}=0orℓ2=0\\ell\_\{2\}=0\.

Our tensor AMP algorithm takes the general form \(cf\.\[[MR14](https://arxiv.org/html/2606.28573#bib.bibx30),[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14)\]\):

𝒛ℓ\+1=∑k≥1k​ak​\(ℓ\)​𝒀\(k,s\)​\{𝒛ℓ\}−Bℓ​𝒛ℓ−1,ℓ≥0,\\text\{$z$\}^\{\\ell\+1\}=\\sum\_\{k\\geq 1\}ka\_\{k\}\(\\ell\)\\text\{$Y$\}^\{\(k,s\)\}\\\{\\text\{$z$\}^\{\\ell\}\\\}\-B\_\{\\ell\}\\text\{$z$\}^\{\\ell\-1\}\\,,\\quad\\ell\\geq 0,\(84\)where𝒀\(k,s\)\\text\{$Y$\}^\{\(k,s\)\}is the symmetrized version of𝒀\(k\)\\text\{$Y$\}^\{\(k\)\}, the initialization𝒛0∈ℝd\\text\{$z$\}^\{0\}\\in\\mathbb\{R\}^\{d\}has i\.i\.d\. coordinates drawn fromp0/dp\_\{0\}/\\sqrt\{d\}, and the Onsager terms\{Bℓ\}ℓ≥0\\\{B\_\{\\ell\}\\\}\_\{\\ell\\geq 0\}are defined as follows:B0=0B\_\{0\}=0, and forℓ≥1\\ell\\geq 1:

Bℓ=1α¯​\(ξa​aℓ,ℓ−1\)′′​\(rℓ​rℓ−1\+Qℓ,ℓ−1\)\.B\_\{\\ell\}=\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\(\\xi\_\{aa\}^\{\\ell,\\ell\-1\}\)^\{\\prime\\prime\}\(r\_\{\\ell\}r\_\{\\ell\-1\}\+Q\_\{\\ell,\\ell\-1\}\)\.\(85\)We are now ready to characterize the limiting empirical joint distribution of the coordinates of tensor AMP iterations asd→∞d\\to\\infty\.

###### Proposition D\.1\.

Assume thatp0p\_\{0\}has finite second moment\. For anyℓ≥0\\ell\\geq 0and pseudo\-Lipschitz functionψ:ℝℓ\+1→ℝ\\psi:\\mathbb\{R\}^\{\\ell\+1\}\\to\\mathbb\{R\}, we have

p−limd→∞⁡1d​∑i=1dψ​\(d​𝒛i0,d​𝒛i1,⋯,d​𝒛iℓ\)=𝔼​\[ψ​\(Z0,Z1,⋯,Zℓ\)\]\.\\begin\{split\}&\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\frac\{1\}\{d\}\\sum\_\{i=1\}^\{d\}\\psi\\big\(\\sqrt\{d\}\\text\{$z$\}\_\{i\}^\{0\},\\sqrt\{d\}\\text\{$z$\}\_\{i\}^\{1\},\\cdots,\\sqrt\{d\}\\text\{$z$\}\_\{i\}^\{\\ell\}\\big\)=\\,\\mathbb\{E\}\\big\[\\psi\\big\(Z^\{0\},Z^\{1\},\\cdots,Z^\{\\ell\}\\big\)\\big\]\\,\.\\end\{split\}

The proof of[D\.1](https://arxiv.org/html/2606.28573#A4.Thmlem1)is deferred to[SectionD\.4](https://arxiv.org/html/2606.28573#A4.SS4)\. We next characterize some important limiting quantities achieved by the first stage of our algorithm\.

###### Proposition D\.2\.

Recall the definition ofABayesA\_\{\\mbox\{\\tiny\\rm Bayes\}\}andcBayesc\_\{\\mbox\{\\tiny\\rm Bayes\}\}from[Eqs\.41](https://arxiv.org/html/2606.28573#S5.E41)and[42](https://arxiv.org/html/2606.28573#S5.E42)\. Assume that\(r,q\)∈ABayes\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, and\{ak\}k≥1\\\{a\_\{k\}\\\}\_\{k\\geq 1\}satisfying

r=∑k≥1k​ak​φk​rk−1,q≥r2\+1α¯​∑k≥1k​ak2​qk−1,1α¯​∑k≥2k​\(k−1\)​ak2​qk−2≤1,r=\\sum\_\{k\\geq 1\}ka\_\{k\}\\varphi\_\{k\}r^\{k\-1\},\\quad q\\geq r^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}ka\_\{k\}^\{2\}q^\{k\-1\},\\quad\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 2\}k\(k\-1\)a\_\{k\}^\{2\}q^\{k\-2\}\\leq 1,\(86\)Then, there exists a tensor AMP algorithm \(as described above\) returning𝐰^1\\widehat\{\\boldsymbol\{w\}\}\_\{1\}, such that

p−limd→∞⁡⟨𝒘^1,𝒘∗⟩=r,p−limd→∞⁡‖𝒘^1‖22=q,\\displaystyle\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\big\\langle\\widehat\{\\boldsymbol\{w\}\}\_\{1\},\\text\{$w$\}\_\{\*\}\\big\\rangle=r\\,,\\;\\;\\;\\;\\;\\;\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\big\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\big\\\|^\{2\}\_\{2\}=q\\,,p−limd→∞⁡H^dg​\(𝒘^1\)=ξφ​σ​\(r\)\+1α¯​∑k≥1k​σk​ak​qk−1\.\\displaystyle\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\widehat\{H\}^\{g\}\_\{d\}\\big\(\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\big\)=\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}a\_\{k\}q^\{k\-1\}\.

In the second stage, we run the Hessian ascent algorithm\[[Sub21](https://arxiv.org/html/2606.28573#bib.bibx38)\]on the Hamiltonian

𝒘⟂↦H^dg​\(𝒘^1\+1−‖𝒘^1‖2​𝒘⟂\)\\text\{$w$\}\_\{\\perp\}\\mapsto\\widehat\{H\}^\{g\}\_\{d\}\\big\(\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\+\\sqrt\{1\-\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\\|^\{2\}\}\\,\\text\{$w$\}\_\{\\perp\}\\big\)over𝒘⟂∈𝕊d−1\\text\{$w$\}\_\{\\perp\}\\in\\mathbb\{S\}^\{d\-1\},𝒘⟂⟂𝒘^1\\text\{$w$\}\_\{\\perp\}\\perp\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\. Since the limiting empirical spectral distribution of∇2H^dg\\nabla^\{2\}\\widehat\{H\}^\{g\}\_\{d\}, the spherical Hessian ofH^dg\\widehat\{H\}^\{g\}\_\{d\}, only depends on its pure noise part, the same argument as in the proof of\[[Sub21](https://arxiv.org/html/2606.28573#bib.bibx38), Theorem 4\]establishes that Hessian ascent achieves the value

H^dg​\(𝒘^1\+1−‖𝒘^1‖2​𝒘⟂\)≈H^dg​\(𝒘^1\)\+1α¯​∫01\(d2d​t2​ξσ​σ​\(‖𝒘^1‖2\+\(1−‖𝒘^1‖2\)​t\)\)1/2​d​t=H^dg​\(𝒘^1\)\+1α¯​∫‖𝒘^1‖21ξσ​σ′′​\(t\)1/2​d​t\.\\begin\{split\}\\widehat\{H\}^\{g\}\_\{d\}\\big\(\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\+\\sqrt\{1\-\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\\|^\{2\}\}\\,\\text\{$w$\}\_\{\\perp\}\\big\)\\approx\\,&\\widehat\{H\}^\{g\}\_\{d\}\(\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\)\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{0\}^\{1\}\\left\(\\frac\{\\textup\{d\}^\{2\}\}\{\\textup\{d\}t^\{2\}\}\\xi\_\{\\sigma\\sigma\}\\big\(\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\\|^\{2\}\+\(1\-\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\\|^\{2\}\)t\\big\)\\right\)^\{1/2\}\\textup\{d\}t\\\\ =\\,&\\widehat\{H\}^\{g\}\_\{d\}\(\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\)\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\\|^\{2\}\}^\{1\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)^\{1/2\}\\textup\{d\}t\.\\end\{split\}As‖𝒘^1‖2→pq\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\\|^\{2\}\\stackrel\{\{\\scriptstyle p\}\}\{\{\\to\}\}qasd→∞d\\to\\infty, we finally obtain the asymptotic value ofH^dg\\widehat\{H\}^\{g\}\_\{d\}achieved by our two\-stage \(tensor AMP \+ Hessian ascent\) algorithm:

p−limd→∞⁡H^dg​\(𝒘^1\+1−‖𝒘^1‖2​𝒘⟂\)=ξφ​σ​\(r\)\+1α¯​∑k≥1k​σk​ak​qk−1\+1α¯​∫q1ξσ​σ′′​\(t\)1/2​d​t\.\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\widehat\{H\}^\{g\}\_\{d\}\\big\(\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\+\\sqrt\{1\-\\\|\\widehat\{\\boldsymbol\{w\}\}\_\{1\}\\\|^\{2\}\}\\,\\text\{$w$\}\_\{\\perp\}\\big\)=\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}a\_\{k\}q^\{k\-1\}\+\\frac\{1\}\{\\sqrt\{\\overline\{\\alpha\}\}\}\\int\_\{q\}^\{1\}\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(t\)^\{1/2\}\\textup\{d\}t\.To further analyze this value, define

Tg​\(r,q\)=max⁡1α¯​∑k≥1k​σk​ak​qk−1,subject to[Eq\.86](https://arxiv.org/html/2606.28573#A4.E86)\.T^\{g\}\(r,q\)=\\,\\max\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}a\_\{k\}q^\{k\-1\},\\quad\\mbox\{subject to \\lx@cref\{creftype~refnum\}\{eq:Gaussian\_fixed\_pt\}\}\.Then, straightforward calculation yields that

Tg​\(r,q\)=q−r2α¯​T0​\(r,q\),T^\{g\}\(r,q\)=\\,\\sqrt\{\\frac\{q\-r^\{2\}\}\{\\overline\{\\alpha\}\}\}\\,T\_\{0\}\(r,q\),withT0​\(r,q\)T\_\{0\}\(r,q\)defined in[Eq\.44](https://arxiv.org/html/2606.28573#S5.E44)\. Consequently, for any fixed\(r,q\)∈ABayes\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, the asymptotic maximum ofH^dg\\widehat\{H\}\_\{d\}^\{g\}attained by our algorithm is exactly𝒱α¯AMP​\(r,q\)\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\\mbox\{\\tiny\\rm AMP\}\}\(r,q\), which matches the result established for the single\-index model\.

### D\.4Proof of[5\.4](https://arxiv.org/html/2606.28573#S5.Thmlem4),[D\.1](https://arxiv.org/html/2606.28573#A4.Thmlem1)and[D\.2](https://arxiv.org/html/2606.28573#A4.Thmlem2)

###### Proof of[5\.4](https://arxiv.org/html/2606.28573#S5.Thmlem4)\.

Obviously,H^dg\\widehat\{H\}^\{g\}\_\{d\}is a Gaussian process\. We now verify[Eq\.33](https://arxiv.org/html/2606.28573#S5.E33)\. By definition,

𝔼​\[H^dg​\(𝒘\)\]=\\displaystyle\\mathbb\{E\}\\left\[\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\)\\right\]=\\,∑k≥1σk​φk​⟨𝒘∗⊗k,𝒘⊗k⟩=ξφ​σ​\(⟨𝒘∗,𝒘⟩\),\\displaystyle\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\varphi\_\{k\}\\langle\\text\{$w$\}\_\{\*\}^\{\\otimes k\},\\text\{$w$\}^\{\\otimes k\}\\rangle=\\xi\_\{\\varphi\\sigma\}\(\\langle\\text\{$w$\}\_\{\*\},\\text\{$w$\}\\rangle\),𝖢𝗈𝗏​\(H^dg​\(𝒘1\),H^dg​\(𝒘2\)\)=\\displaystyle\{\\sf Cov\}\\big\(\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\_\{1\}\),\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$w$\}\_\{2\}\)\\big\)=\\,1d​α¯​𝖢𝗈𝗏​\(∑k≥1σk​⟨𝑮⊗k,𝒘1⊗k⟩,∑k≥1σk​⟨𝑮⊗k,𝒘2⊗k⟩\)\\displaystyle\\frac\{1\}\{d\\overline\{\\alpha\}\}\{\\sf Cov\}\\left\(\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\langle\\text\{$G$\}^\{\\otimes k\},\\text\{$w$\}\_\{1\}^\{\\otimes k\}\\rangle,\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\langle\\text\{$G$\}^\{\\otimes k\},\\text\{$w$\}\_\{2\}^\{\\otimes k\}\\rangle\\right\)=\\displaystyle=\\,1d​α¯​∑k≥1σk2​⟨𝒘1⊗k,𝒘2⊗k⟩=1d​α¯​ξσ​σ​\(⟨𝒘1,𝒘2⟩\),\\displaystyle\\frac\{1\}\{d\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}\\sigma\_\{k\}^\{2\}\\langle\\text\{$w$\}\_\{1\}^\{\\otimes k\},\\text\{$w$\}\_\{2\}^\{\\otimes k\}\\rangle=\\frac\{1\}\{d\\overline\{\\alpha\}\}\\xi\_\{\\sigma\\sigma\}\(\\langle\\text\{$w$\}\_\{1\},\\text\{$w$\}\_\{2\}\\rangle\),where in the above calculation we use[Eq\.34](https://arxiv.org/html/2606.28573#S5.E34)and the assumptionσ0=𝔼​\[σ​\(G\)\]=0\\sigma\_\{0\}=\\mathbb\{E\}\[\\sigma\(G\)\]=0\. This completes the proof\. ∎

###### Proof of[D\.1](https://arxiv.org/html/2606.28573#A4.Thmlem1)\.

This proof is based on the state evolution of general tensor AMP algorithms without a signal, which we establish below\. As in[SectionD\.3\.2](https://arxiv.org/html/2606.28573#A4.SS3.SSS2), we begin with introducing the Gaussian process that governs the limiting behavior of tensor AMP iterates asd→∞d\\to\\infty\.

Let\{σk\}k≥1\\\{\\sigma\_\{k\}\\\}\_\{k\\geq 1\}be a sequence of real numbers such that∑k≥1σk2​tk<\+∞\\sum\_\{k\\geq 1\}\\sigma\_\{k\}^\{2\}t^\{k\}<\+\\inftyfor somet\>1t\>1\. Let\{fℓk\}ℓ≥0,k≥1\\\{f\_\{\\ell\}^\{k\}\\\}\_\{\\ell\\geq 0,\\,k\\geq 1\}be a sequence of functions such that for eachℓ≥0\\ell\\geq 0andk≥1k\\geq 1,fℓk:ℝℓ\+1→ℝf\_\{\\ell\}^\{k\}:\\mathbb\{R\}^\{\\ell\+1\}\\to\\mathbb\{R\}is Lipschitz\. LetU0∼p0U^\{0\}\\sim p\_\{0\}, wherep0p\_\{0\}is a probability distribution onℝ\\mathbb\{R\}with finite second moment\. For eachℓ≥1\\ell\\geq 1,U≤ℓ=\(U1,⋯,Uℓ\)∈ℝℓU^\{\\leq\\ell\}=\(U^\{1\},\\cdots,U^\{\\ell\}\)\\in\\mathbb\{R\}^\{\\ell\}is a mean\-zero multivariate Gaussian vector, independent ofU0U^\{0\}\. Further, the covariance structure ofU≤ℓU^\{\\leq\\ell\}is recursively defined via:

𝔼​\[Uℓ1\+1​Uℓ2\+1\]=∑k≥1k​σk2​𝔼​\[fℓ1k​\(U0,⋯,Uℓ1\)​fℓ2k​\(U0,⋯,Uℓ2\)\]k−1,∀ℓ1,ℓ2≥0,𝔼​\[Uℓ1​Uℓ2\]=0,if​ℓ1=0​or​ℓ2=0\.\\begin\{split\}\\mathbb\{E\}\\left\[U^\{\\ell\_\{1\}\+1\}U^\{\\ell\_\{2\}\+1\}\\right\]=\\,&\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}^\{2\}\\,\\mathbb\{E\}\\left\[f\_\{\\ell\_\{1\}\}^\{k\}\\left\(U^\{0\},\\cdots,U^\{\\ell\_\{1\}\}\\right\)f\_\{\\ell\_\{2\}\}^\{k\}\\left\(U^\{0\},\\cdots,U^\{\\ell\_\{2\}\}\\right\)\\right\]^\{k\-1\},\\quad\\forall\\ell\_\{1\},\\ell\_\{2\}\\geq 0,\\\\ \\mathbb\{E\}\\left\[U^\{\\ell\_\{1\}\}U^\{\\ell\_\{2\}\}\\right\]=\\,&0,\\quad\\mbox\{ if \}\\ell\_\{1\}=0\\mbox\{ or \}\\ell\_\{2\}=0\.\\end\{split\}For eachk≥1k\\geq 1, let𝑮\(k\)=\(Gi1,…,ik\(k\)\)1≤i1,…,ik≤d∈\(ℝd\)⊗k\\text\{$G$\}^\{\(k\)\}=\(G^\{\(k\)\}\_\{i\_\{1\},\\dots,i\_\{k\}\}\)\_\{1\\leq i\_\{1\},\\dots,i\_\{k\}\\leq d\}\\in\(\\mathbb\{R\}^\{d\}\)^\{\\otimes k\}be a tensor with i\.i\.d\.𝖭​\(0,1\)\{\\sf N\}\(0,1\)entries, and let𝑮\(k,s\)\\text\{$G$\}^\{\(k,s\)\}be the symmetrization of𝑮\(k\)\\text\{$G$\}^\{\(k\)\}\. Consider the following general tensor AMP iterations:

𝒖ℓ\+1=1d​∑k≥1k​σk​𝑮\(k,s\)​\{fℓk​\(𝒖0,⋯,𝒖ℓ\)\}−∑k≥1∑j≤ℓBℓ,j,k​fj−1k​\(𝒖0,…,𝒖j−1\),ℓ≥0,\\text\{$u$\}^\{\\ell\+1\}=\\,\\frac\{1\}\{\\sqrt\{d\}\}\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}\\text\{$G$\}^\{\(k,s\)\}\\\{f\_\{\\ell\}^\{k\}\(\\text\{$u$\}^\{0\},\\cdots,\\text\{$u$\}^\{\\ell\}\)\\\}\-\\sum\_\{k\\geq 1\}\\sum\_\{j\\leq\\ell\}B\_\{\\ell,j,k\}f\_\{j\-1\}^\{k\}\(\\text\{$u$\}^\{0\},\\dots,\\text\{$u$\}^\{j\-1\}\),\\quad\\ell\\geq 0,\(87\)with𝒖0∈ℝd\\text\{$u$\}^\{0\}\\in\\mathbb\{R\}^\{d\}having coordinates i\.i\.d\. sampled fromp0p\_\{0\}, and the Onsager terms defined as

Bℓ,j,k=k​\(k−1\)​σk2​𝔼​\[fℓk​\(U0,⋯,Uℓ\)​fj−1k​\(U0,⋯,Uj−1\)\]k−2⋅𝔼​\[∂fℓk∂Uj​\(U0,⋯,Uℓ\)\]\.B\_\{\\ell,j,k\}=\\,k\(k\-1\)\\sigma\_\{k\}^\{2\}\\,\\mathbb\{E\}\[f\_\{\\ell\}^\{k\}\(U^\{0\},\\cdots,U^\{\\ell\}\)f\_\{j\-1\}^\{k\}\(U^\{0\},\\cdots,U^\{j\-1\}\)\]^\{k\-2\}\\cdot\\mathbb\{E\}\\left\[\\frac\{\\partial f\_\{\\ell\}^\{k\}\}\{\\partial U^\{j\}\}\(U^\{0\},\\cdots,U^\{\\ell\}\)\\right\]\.
###### Proposition D\.3\.

For anyℓ≥0\\ell\\geq 0and pseudo\-Lipschitz functionψ:ℝℓ\+1→ℝ\\psi:\\mathbb\{R\}^\{\\ell\+1\}\\to\\mathbb\{R\}, we have

p−limd→∞⁡1d​∑i=1dψ​\(d​𝒖i0,⋯,d​𝒖iℓ\)=𝔼​\[ψ​\(U0,⋯,Uℓ\)\]\.\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\frac\{1\}\{d\}\\sum\_\{i=1\}^\{d\}\\psi\\left\(\\sqrt\{d\}\\text\{$u$\}\_\{i\}^\{0\},\\cdots,\\sqrt\{d\}\\text\{$u$\}\_\{i\}^\{\\ell\}\\right\)=\\,\\mathbb\{E\}\\left\[\\psi\\left\(U^\{0\},\\cdots,U^\{\\ell\}\\right\)\\right\]\.

The proof of[D\.3](https://arxiv.org/html/2606.28573#A4.Thmlem3)closely follows that of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Theorem 6\], and will be presented in[SectionD\.5](https://arxiv.org/html/2606.28573#A4.SS5)\.

We are now in position to complete the proof of[D\.1](https://arxiv.org/html/2606.28573#A4.Thmlem1)\. In a similar spirit to[B\.1](https://arxiv.org/html/2606.28573#A2.Thmlem1), we can assume without loss of generality that the empirical distribution of the coordinates ofd​𝒘∗\\sqrt\{d\}\\text\{$w$\}\_\{\*\}converges inW2W\_\{2\}distance to𝖭​\(0,1\)\{\\sf N\}\(0,1\)asd→∞d\\to\\infty\. Define forℓ≥0\\ell\\geq 0andk≥1k\\geq 1:

fℓk​\(𝒖0,⋯,𝒖ℓ\)=\(ak​\(ℓ\)α¯​σk\)1/\(k−1\)​\(𝒖ℓ\+rℓ⋅𝒘∗\)=\(ak​\(ℓ\)α¯​σk\)1/\(k−1\)​\(𝒖ℓ\+\(ξa​φℓ−1\)′​\(rℓ−1\)⋅𝒘∗\),\\begin\{split\}f\_\{\\ell\}^\{k\}\(\\text\{$u$\}^\{0\},\\cdots,\\text\{$u$\}^\{\\ell\}\)=\\,&\\left\(\\frac\{a\_\{k\}\(\\ell\)\}\{\\sqrt\{\\overline\{\\alpha\}\}\\sigma\_\{k\}\}\\right\)^\{1/\(k\-1\)\}\\left\(\\text\{$u$\}^\{\\ell\}\+r\_\{\\ell\}\\cdot\\text\{$w$\}\_\{\*\}\\right\)\\\\ =\\,&\\left\(\\frac\{a\_\{k\}\(\\ell\)\}\{\\sqrt\{\\overline\{\\alpha\}\}\\sigma\_\{k\}\}\\right\)^\{1/\(k\-1\)\}\\left\(\\text\{$u$\}^\{\\ell\}\+\(\\xi\_\{a\\varphi\}^\{\\ell\-1\}\)^\{\\prime\}\(r\_\{\\ell\-1\}\)\\cdot\\text\{$w$\}\_\{\*\}\\right\),\\end\{split\}and consider the AMP algorithm \([87](https://arxiv.org/html/2606.28573#A4.E87)\) \(the base coefficients\{σk\}k≥1\\\{\\sigma\_\{k\}\\\}\_\{k\\geq 1\}may depend onℓ\\elland can be chosen such thatak​\(ℓ\)/σk\>0a\_\{k\}\(\\ell\)/\\sigma\_\{k\}\>0\)\. The state evolution equations then reduce to

𝔼​\[Uℓ1\+1​Uℓ2\+1\]=\\displaystyle\\mathbb\{E\}\\left\[U^\{\\ell\_\{1\}\+1\}U^\{\\ell\_\{2\}\+1\}\\right\]=\\,1α¯​∑k≥1k​ak​\(ℓ1\)​ak​\(ℓ2\)​\(rℓ1​rℓ2\+𝔼​\[Uℓ1​Uℓ2\]\)k−1\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}ka\_\{k\}\(\\ell\_\{1\}\)a\_\{k\}\(\\ell\_\{2\}\)\\,\\left\(r\_\{\\ell\_\{1\}\}r\_\{\\ell\_\{2\}\}\+\\mathbb\{E\}\[U^\{\\ell\_\{1\}\}U^\{\\ell\_\{2\}\}\]\\right\)^\{k\-1\}=\\displaystyle=\\,1α¯​\(ξa​aℓ1,ℓ2\)′​\(rℓ1​rℓ2\+𝔼​\[Uℓ1​Uℓ2\]\),\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}\}\(\\xi\_\{aa\}^\{\\ell\_\{1\},\\ell\_\{2\}\}\)^\{\\prime\}\(r\_\{\\ell\_\{1\}\}r\_\{\\ell\_\{2\}\}\+\\mathbb\{E\}\[U^\{\\ell\_\{1\}\}U^\{\\ell\_\{2\}\}\]\),which implies that𝔼​\[Uℓ1​Uℓ2\]=Qℓ1,ℓ2\\mathbb\{E\}\[U^\{\\ell\_\{1\}\}U^\{\\ell\_\{2\}\}\]=Q\_\{\\ell\_\{1\},\\ell\_\{2\}\}\. The AMP iterations can also be simplified to

𝒖ℓ\+1=1d​α¯​∑k≥1k​ak​\(ℓ\)​𝑮\(k,s\)​\{𝒖ℓ\+rℓ​𝒘∗\}−Bℓ​\(𝒖ℓ−1\+rℓ−1​𝒘∗\),\\text\{$u$\}^\{\\ell\+1\}=\\,\\frac\{1\}\{\\sqrt\{d\\overline\{\\alpha\}\}\}\\sum\_\{k\\geq 1\}ka\_\{k\}\(\\ell\)\\text\{$G$\}^\{\(k,s\)\}\\\{\\text\{$u$\}^\{\\ell\}\+r\_\{\\ell\}\\text\{$w$\}\_\{\*\}\\\}\-B\_\{\\ell\}\(\\text\{$u$\}^\{\\ell\-1\}\+r\_\{\\ell\-1\}\\text\{$w$\}\_\{\*\}\),where we recallBℓB\_\{\\ell\}from[Eq\.85](https://arxiv.org/html/2606.28573#A4.E85)\.

Define𝒎ℓ=𝒛ℓ−rℓ​𝒘∗\\text\{$m$\}^\{\\ell\}=\\,\\text\{$z$\}^\{\\ell\}\-r\_\{\\ell\}\\text\{$w$\}\_\{\*\}\. To prove our claim, it suffices to show that\{𝒎ℓ\}ℓ≥0\\\{\\text\{$m$\}^\{\\ell\}\\\}\_\{\\ell\\geq 0\}and\{𝒖ℓ\}ℓ≥0\\\{\\text\{$u$\}^\{\\ell\}\\\}\_\{\\ell\\geq 0\}have the same limiting empirical distribution\. To this end, we use induction to establish that

p−limd→∞⁡‖𝒖ℓ−𝒎ℓ‖22=p−limd→∞​∑i=1d\(𝒖iℓ−𝒎iℓ\)2=0,∀ℓ≥0\.\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\,\\left\\\|\{\\text\{$u$\}^\{\\ell\}\-\\text\{$m$\}^\{\\ell\}\}\\right\\\|\_\{2\}^\{2\}=\\,\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\,\\sum\_\{i=1\}^\{d\}\\left\(\\text\{$u$\}\_\{i\}^\{\\ell\}\-\\text\{$m$\}\_\{i\}^\{\\ell\}\\right\)^\{2\}=0,\\quad\\forall\\ell\\geq 0\.\(88\)The base caseℓ=0\\ell=0holds automatically since we can choose𝒖0=𝒛0=𝒎0\\text\{$u$\}^\{0\}=\\text\{$z$\}^\{0\}=\\text\{$m$\}^\{0\}\. Now assume that[Eq\.88](https://arxiv.org/html/2606.28573#A4.E88)holds for allℓ′≤ℓ\\ell^\{\\prime\}\\leq\\ell\. Forℓ\+1\\ell\+1, note that[Eq\.84](https://arxiv.org/html/2606.28573#A4.E84)implies

𝒛ℓ\+1=\(ξa​φℓ\)′​\(⟨𝒛ℓ,𝒘∗⟩\)⋅𝒘∗\+1d​α¯​∑k≥1k​ak​\(ℓ\)​𝑮\(k,s\)​\{𝒛ℓ\}−Bℓ​𝒛ℓ−1\.\\displaystyle\\text\{$z$\}^\{\\ell\+1\}=\\,\(\\xi\_\{a\\varphi\}^\{\\ell\}\)^\{\\prime\}\(\\langle\\text\{$z$\}^\{\\ell\},\\text\{$w$\}\_\{\*\}\\rangle\)\\cdot\\text\{$w$\}\_\{\*\}\+\\frac\{1\}\{\\sqrt\{d\\overline\{\\alpha\}\}\}\\sum\_\{k\\geq 1\}ka\_\{k\}\(\\ell\)\\text\{$G$\}^\{\(k,s\)\}\\\{\\text\{$z$\}^\{\\ell\}\\\}\-B\_\{\\ell\}\\text\{$z$\}^\{\\ell\-1\}\.Our induction assumption implies that

p−limd→∞⁡⟨𝒛ℓ,𝒘∗⟩=rℓ,p−limd→∞⁡‖𝒛ℓ′−𝒖ℓ′−rℓ′​𝒘∗‖22=0,∀ℓ′≤ℓ\.\\displaystyle\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\,\\langle\\text\{$z$\}^\{\\ell\},\\text\{$w$\}\_\{\*\}\\rangle=\\,r\_\{\\ell\},\\quad\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\,\\left\\\|\{\\text\{$z$\}^\{\\ell^\{\\prime\}\}\-\\text\{$u$\}^\{\\ell^\{\\prime\}\}\-r\_\{\\ell^\{\\prime\}\}\\text\{$w$\}\_\{\*\}\}\\right\\\|\_\{2\}^\{2\}=\\,0,\\,\\,\\forall\\ell^\{\\prime\}\\leq\\ell\.Therefore, it remains to show that

p−limd→∞⁡1d​‖𝑮\(k,s\)​\{𝒛ℓ\}−𝑮\(k,s\)​\{𝒖ℓ\}‖22=0,∀k≥1,\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\,\\frac\{1\}\{d\}\\left\\\|\{\\text\{$G$\}^\{\(k,s\)\}\\\{\\text\{$z$\}^\{\\ell\}\\\}\-\\text\{$G$\}^\{\(k,s\)\}\\\{\\text\{$u$\}^\{\\ell\}\\\}\}\\right\\\|\_\{2\}^\{2\}=\\,0,\\quad\\forall k\\geq 1,which easily follows from concentration bounds on the operator norm of Gaussian random tensors, i\.e\.,‖𝑮\(k,s\)‖op=Op​\(d\)\\\|\\text\{$G$\}^\{\(k,s\)\}\\\|\_\{\\mathrm\{op\}\}=O\_\{p\}\(\\sqrt\{d\}\)\(cf\.\[[MR14](https://arxiv.org/html/2606.28573#bib.bibx30), Lemma 2\]\)\. This completes our induction and the proof of[D\.1](https://arxiv.org/html/2606.28573#A4.Thmlem1)\. ∎

###### Proof of Proposition[D\.2](https://arxiv.org/html/2606.28573#A4.Thmlem2)\.

Forℓ≥0\\ell\\geq 0, defineqℓ=rℓ2\+Qℓ,ℓq\_\{\\ell\}=r\_\{\\ell\}^\{2\}\+Q\_\{\\ell,\\ell\}\. Then, we know that\(r0,q0\)=0\(r\_\{0\},q\_\{0\}\)=0and state evolution yields

rℓ\+1=\(ξa​φℓ\)′​\(rℓ\),qℓ\+1=rℓ\+12\+1α¯​\(ξa​aℓ,ℓ\)′​\(qℓ\)\.r\_\{\\ell\+1\}=\\,\(\\xi\_\{a\\varphi\}^\{\\ell\}\)^\{\\prime\}\(r\_\{\\ell\}\),\\quad q\_\{\\ell\+1\}=\\,r\_\{\\ell\+1\}^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\(\\xi\_\{aa\}^\{\\ell,\\ell\}\)^\{\\prime\}\(q\_\{\\ell\}\)\.\(89\)LetApcaA\_\{\\rm pca\}denote the set of pairs\(r,q\)∈\[−1,1\]×\[0,1\]\(r,q\)\\in\[\-1,1\]\\times\[0,1\]that can be achieved as limit points of the above iteration\. Sinceξa​φℓ\\xi\_\{a\\varphi\}^\{\\ell\}is linear in theak​\(ℓ\)a\_\{k\}\(\\ell\)’s andξa​aℓ,ℓ\\xi\_\{aa\}^\{\\ell,\\ell\}is quadratic in theak​\(ℓ\)a\_\{k\}\(\\ell\)’s, we see thatApcaA\_\{\\rm pca\}must be of the formApca=\{\(r,q\):r2/q≤cpca2/\(1\+cpca2\)\}A\_\{\\rm pca\}=\\\{\(r,q\):r^\{2\}/q\\leq c\_\{\\rm pca\}^\{2\}/\(1\+c\_\{\\rm pca\}^\{2\}\)\\\}for somecpca\>0c\_\{\\rm pca\}\>0\. To computecpcac\_\{\\rm pca\}, denotecℓ=rℓ/qℓ−rℓ2c\_\{\\ell\}=r\_\{\\ell\}/\\sqrt\{q\_\{\\ell\}\-r\_\{\\ell\}^\{2\}\}\. For any fixed\(rℓ,qℓ\)\(r\_\{\\ell\},q\_\{\\ell\}\),[Eq\.89](https://arxiv.org/html/2606.28573#A4.E89)implies that

cℓ\+12=rℓ\+12qℓ\+1−rℓ\+12=α¯​\(ξa​φℓ\)′​\(rℓ\)2\(ξa​aℓ,ℓ\)′​\(qℓ\)≤\(i\)α¯​ξφ​φ′​\(rℓ2qℓ\)=α¯​ξφ​φ′​\(cℓ21\+cℓ2\),\\displaystyle c\_\{\\ell\+1\}^\{2\}=\\,\\frac\{r\_\{\\ell\+1\}^\{2\}\}\{q\_\{\\ell\+1\}\-r\_\{\\ell\+1\}^\{2\}\}=\\overline\{\\alpha\}\\frac\{\(\\xi\_\{a\\varphi\}^\{\\ell\}\)^\{\\prime\}\(r\_\{\\ell\}\)^\{2\}\}\{\(\\xi\_\{aa\}^\{\\ell,\\ell\}\)^\{\\prime\}\(q\_\{\\ell\}\)\}\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{\\leq\}\}\\overline\{\\alpha\}\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{r\_\{\\ell\}^\{2\}\}\{q\_\{\\ell\}\}\\right\)=\\overline\{\\alpha\}\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{c\_\{\\ell\}^\{2\}\}\{1\+c\_\{\\ell\}^\{2\}\}\\right\),where in\(i\)\(i\)we use Cauchy\-Schwarz inequality \(cf\.[D\.5](https://arxiv.org/html/2606.28573#A4.Thmlem5)\), and the equality is achieved if

ak​\(ℓ\)∝φk​\(rℓqℓ\)k−1\.a\_\{k\}\(\\ell\)\\propto\\varphi\_\{k\}\\left\(\\frac\{r\_\{\\ell\}\}\{q\_\{\\ell\}\}\\right\)^\{k\-1\}\.This immediately implies thatcpcac\_\{\\rm pca\}should be the minimum positive solution to the equation

c2=α¯​ξφ​φ′​\(c21\+c2\),c^\{2\}=\\,\\overline\{\\alpha\}\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{c^\{2\}\}\{1\+c^\{2\}\}\\right\),leading tocpca=cBayesc\_\{\\rm pca\}=c\_\{\\mbox\{\\tiny\\rm Bayes\}\}andApca=ABayesA\_\{\\rm pca\}=A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\.

Now for any\(r,q\)∈int⁡ABayes\(r,q\)\\in\\operatorname\{int\}A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, there existsℓ1∈ℕ\\ell\_\{1\}\\in\\mathbb\{N\}such that afterℓ1\\ell\_\{1\}tensor AMP iterations, we achieve\(rℓ1,qℓ1\)=\(r,q\)\(r\_\{\\ell\_\{1\}\},q\_\{\\ell\_\{1\}\}\)=\(r,q\)\. By state evolution of tensor AMP, for any sequence\{ak\}k≥1\\\{a\_\{k\}\\\}\_\{k\\geq 1\}satisfying

r=ξa​φ′​\(r\),q=r2\+1α¯​ξa​a′​\(q\)r=\\xi\_\{a\\varphi\}^\{\\prime\}\(r\),\\quad q=r^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\xi\_\{aa\}^\{\\prime\}\(q\)\(whereξa​φ​\(t\):=∑k≥1ak​φk​tk\\xi\_\{a\\varphi\}\(t\):=\\sum\_\{k\\geq 1\}a\_\{k\}\\varphi\_\{k\}t^\{k\}andξa​a​\(t\):=∑k≥1ak2​tk\\xi\_\{aa\}\(t\):=\\sum\_\{k\\geq 1\}a\_\{k\}^\{2\}t^\{k\}\), usingak​\(ℓ\)=aka\_\{k\}\(\\ell\)=a\_\{k\}yields\(rℓ,qℓ\)=\(r,q\)\(r\_\{\\ell\},q\_\{\\ell\}\)=\(r,q\)for all iterationsℓ1≤ℓ≤ℓ2−1\\ell\_\{1\}\\leq\\ell\\leq\\ell\_\{2\}\-1, whereℓ2≥ℓ1\+1\\ell\_\{2\}\\geq\\ell\_\{1\}\+1is to be determined later\. Further, the covariance between successive AMP iterations satisfies

Qℓ\+1,ℓ=1α¯​ξa​a′​\(r2\+Qℓ,ℓ−1\)\.Q\_\{\\ell\+1,\\ell\}=\\,\\frac\{1\}\{\\overline\{\\alpha\}\}\\xi\_\{aa\}^\{\\prime\}\(r^\{2\}\+Q\_\{\\ell,\\ell\-1\}\)\.Therefore, as long asξa​a′′​\(q\)/α¯≤1\\xi\_\{aa\}^\{\\prime\\prime\}\(q\)/\\overline\{\\alpha\}\\leq 1, we haveQℓ2,ℓ2−1→q−r2Q\_\{\\ell\_\{2\},\\ell\_\{2\}\-1\}\\to q\-r^\{2\}asℓ2→∞\\ell\_\{2\}\\to\\infty, which further implies that𝔼​\[\(Zℓ2−Zℓ2−1\)2\]→0\\mathbb\{E\}\[\(Z^\{\\ell\_\{2\}\}\-Z^\{\\ell\_\{2\}\-1\}\)^\{2\}\]\\to 0\. In fact, by adding random noise to theaka\_\{k\}’s, it suffices to assume that

r=ξa​φ′​\(r\),q≥r2\+1α¯​ξa​a′​\(q\),1α¯​ξa​a′′​\(q\)≤1,r=\\xi\_\{a\\varphi\}^\{\\prime\}\(r\),\\quad q\\geq r^\{2\}\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\xi\_\{aa\}^\{\\prime\}\(q\),\\quad\\frac\{1\}\{\\overline\{\\alpha\}\}\\xi\_\{aa\}^\{\\prime\\prime\}\(q\)\\leq 1,which is equivalent to[Eq\.86](https://arxiv.org/html/2606.28573#A4.E86)\.

We are now in position to compute the asymptotic value ofH^dg\\widehat\{H\}^\{g\}\_\{d\}achieved in the first stage\. Fix\(r,q\)∈ABayes\(r,q\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, and let\{ak\}k≥1\\\{a\_\{k\}\\\}\_\{k\\geq 1\}satisfy[Eq\.86](https://arxiv.org/html/2606.28573#A4.E86)\. We compute

H^dg​\(𝒛ℓ2\)=∑k≥1σk​⟨𝒀\(k\),\(𝒛ℓ2\)⊗k⟩=∑k≥1σk​⟨𝒀\(k,s\),\(𝒛ℓ2\)⊗k⟩=⟨𝒛ℓ2,∑k≥1σk​𝒀\(k,s\)​\{𝒛ℓ2\}⟩\.\\displaystyle\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$z$\}^\{\\ell\_\{2\}\}\)=\\,\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\langle\\text\{$Y$\}^\{\(k\)\},\(\\text\{$z$\}^\{\\ell\_\{2\}\}\)^\{\\otimes k\}\\rangle=\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\langle\\text\{$Y$\}^\{\(k,s\)\},\(\\text\{$z$\}^\{\\ell\_\{2\}\}\)^\{\\otimes k\}\\rangle=\\Big\\langle\\text\{$z$\}^\{\\ell\_\{2\}\},\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\text\{$Y$\}^\{\(k,s\)\}\\\{\\text\{$z$\}^\{\\ell\_\{2\}\}\\\}\\Big\\rangle\.To this end, we “do” another step of tensor AMP:

𝒛ℓ2\+1=∑k≥1σk​𝒀\(k,s\)​\{𝒛ℓ2\}−\(1α¯​∑k≥1\(k−1\)​σk​ak​𝔼​\[Zℓ2​Zℓ2−1\]k−2\)⋅𝒛ℓ2−1,\\text\{$z$\}^\{\\ell\_\{2\}\+1\}=\\,\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\text\{$Y$\}^\{\(k,s\)\}\\\{\\text\{$z$\}^\{\\ell\_\{2\}\}\\\}\-\\Big\(\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}\(k\-1\)\\sigma\_\{k\}a\_\{k\}\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\}Z^\{\\ell\_\{2\}\-1\}\]^\{k\-2\}\\Big\)\\cdot\\text\{$z$\}^\{\\ell\_\{2\}\-1\},which is just[Eq\.84](https://arxiv.org/html/2606.28573#A4.E84)withak​\(ℓ2\)=σk/ka\_\{k\}\(\\ell\_\{2\}\)=\\sigma\_\{k\}/k, and we note thatak​\(ℓ2−1\)=aka\_\{k\}\(\\ell\_\{2\}\-1\)=a\_\{k\}by our choice\. It then follows that

H^dg​\(𝒛ℓ2\)=\\displaystyle\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$z$\}^\{\\ell\_\{2\}\}\)=\\,⟨𝒛ℓ2,∑k≥1σk​𝒀\(k,s\)​\{𝒛ℓ2\}⟩=⟨𝒛ℓ2,𝒛ℓ2\+1⟩\+⟨𝒛ℓ2−1,𝒛ℓ2⟩⋅1α¯​∑k≥1\(k−1\)​σk​ak​𝔼​\[Zℓ2​Zℓ2−1\]k−2\\displaystyle\\Big\\langle\\text\{$z$\}^\{\\ell\_\{2\}\},\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\text\{$Y$\}^\{\(k,s\)\}\\\{\\text\{$z$\}^\{\\ell\_\{2\}\}\\\}\\Big\\rangle=\\langle\\text\{$z$\}^\{\\ell\_\{2\}\},\\text\{$z$\}^\{\\ell\_\{2\}\+1\}\\rangle\+\\langle\\text\{$z$\}^\{\\ell\_\{2\}\-1\},\\text\{$z$\}^\{\\ell\_\{2\}\}\\rangle\\cdot\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}\(k\-1\)\\sigma\_\{k\}a\_\{k\}\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\}Z^\{\\ell\_\{2\}\-1\}\]^\{k\-2\}→p\\displaystyle\\stackrel\{\{\\scriptstyle p\}\}\{\{\\to\}\}\\,𝔼​\[Zℓ2\+1​Zℓ2\]\+1α¯​∑k≥1\(k−1\)​σk​ak​𝔼​\[Zℓ2​Zℓ2−1\]k−1​as​d→∞\.\\displaystyle\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\+1\}Z^\{\\ell\_\{2\}\}\]\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}\(k\-1\)\\sigma\_\{k\}a\_\{k\}\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\}Z^\{\\ell\_\{2\}\-1\}\]^\{k\-1\}\\,\\,\\mbox\{ as \}d\\to\\infty\.Since𝔼​\[\(Zℓ2−Zℓ2−1\)2\]→0\\mathbb\{E\}\[\(Z^\{\\ell\_\{2\}\}\-Z^\{\\ell\_\{2\}\-1\}\)^\{2\}\]\\to 0asℓ2→∞\\ell\_\{2\}\\to\\infty, we know that𝔼​\[Zℓ2​Zℓ2−1\]→q\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\}Z^\{\\ell\_\{2\}\-1\}\]\\to q\. Further,

rℓ2\+1=\\displaystyle r\_\{\\ell\_\{2\}\+1\}=\\,\(ξa​φℓ2\)′​\(rℓ2\)=∑k≥1k​ak​\(ℓ2\)​φk​rℓ2k−1=∑k≥1σk​φk​rℓ2k−1=1rℓ2​ξφ​σ​\(rℓ2\),\\displaystyle\(\\xi\_\{a\\varphi\}^\{\\ell\_\{2\}\}\)^\{\\prime\}\(r\_\{\\ell\_\{2\}\}\)=\\sum\_\{k\\geq 1\}ka\_\{k\}\(\\ell\_\{2\}\)\\varphi\_\{k\}r\_\{\\ell\_\{2\}\}^\{k\-1\}=\\sum\_\{k\\geq 1\}\\sigma\_\{k\}\\varphi\_\{k\}r\_\{\\ell\_\{2\}\}^\{k\-1\}=\\frac\{1\}\{r\_\{\\ell\_\{2\}\}\}\\xi\_\{\\varphi\\sigma\}\(r\_\{\\ell\_\{2\}\}\),Qℓ2\+1,ℓ2=\\displaystyle Q\_\{\\ell\_\{2\}\+1,\\ell\_\{2\}\}=\\,1α¯​\(ξa​aℓ2,ℓ2−1\)′​\(rℓ2​rℓ2−1\+Qℓ2,ℓ2−1\)=1α¯​∑k≥1k​ak​\(ℓ2\)​ak​\(ℓ2−1\)​𝔼​\[Zℓ2​Zℓ2−1\]k−1\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}\}\(\\xi\_\{aa\}^\{\\ell\_\{2\},\\ell\_\{2\}\-1\}\)^\{\\prime\}\(r\_\{\\ell\_\{2\}\}r\_\{\\ell\_\{2\}\-1\}\+Q\_\{\\ell\_\{2\},\\ell\_\{2\}\-1\}\)=\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}ka\_\{k\}\(\\ell\_\{2\}\)a\_\{k\}\(\\ell\_\{2\}\-1\)\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\}Z^\{\\ell\_\{2\}\-1\}\]^\{k\-1\}=\\displaystyle=\\,1α¯​∑k≥1σk​ak​𝔼​\[Zℓ2​Zℓ2−1\]k−1,\\displaystyle\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}\\sigma\_\{k\}a\_\{k\}\\,\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\}Z^\{\\ell\_\{2\}\-1\}\]^\{k\-1\},leading to

𝔼​\[Zℓ2\+1​Zℓ2\]=rℓ2\+1​rℓ2\+Qℓ2\+1,ℓ2=ξφ​σ​\(rℓ2\)\+1α¯​∑k≥1σk​ak​𝔼​\[Zℓ2​Zℓ2−1\]k−1→ξφ​σ​\(r\)\+1α¯​∑k≥1σk​ak​qk−1​as​ℓ2→∞\.\\begin\{split\}\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\+1\}Z^\{\\ell\_\{2\}\}\]=\\,&r\_\{\\ell\_\{2\}\+1\}r\_\{\\ell\_\{2\}\}\+Q\_\{\\ell\_\{2\}\+1,\\ell\_\{2\}\}=\\xi\_\{\\varphi\\sigma\}\(r\_\{\\ell\_\{2\}\}\)\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}\\sigma\_\{k\}a\_\{k\}\\,\\mathbb\{E\}\[Z^\{\\ell\_\{2\}\}Z^\{\\ell\_\{2\}\-1\}\]^\{k\-1\}\\\\ \\to\\,&\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}\\sigma\_\{k\}a\_\{k\}q^\{k\-1\}\\,\\,\\mbox\{ as \}\\ell\_\{2\}\\to\\infty\.\\end\{split\}We finally deduce that

limℓ2→∞p−limd→∞⁡H^dg​\(𝒛ℓ2\)=ξφ​σ​\(r\)\+1α¯​∑k≥1k​σk​ak​qk−1,\\lim\_\{\\ell\_\{2\}\\to\\infty\}\\operatorname\*\{p\-lim\}\_\{d\\to\\infty\}\\widehat\{H\}^\{g\}\_\{d\}\(\\text\{$z$\}^\{\\ell\_\{2\}\}\)=\\,\\xi\_\{\\varphi\\sigma\}\(r\)\+\\frac\{1\}\{\\overline\{\\alpha\}\}\\sum\_\{k\\geq 1\}k\\sigma\_\{k\}a\_\{k\}q^\{k\-1\},where\{ak\}k≥1\\\{a\_\{k\}\\\}\_\{k\\geq 1\}satisfy[Eq\.86](https://arxiv.org/html/2606.28573#A4.E86)\. This completes the proof\. ∎

### D\.5Proof of[D\.3](https://arxiv.org/html/2606.28573#A4.Thmlem3)

###### Proof\.

The tensor AMP algorithm \([87](https://arxiv.org/html/2606.28573#A4.E87)\) in[SectionD\.4](https://arxiv.org/html/2606.28573#A4.SS4)generalizes the one introduced in\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Appendix A\], by allowing for different non\-linearitiesfℓkf\_\{\\ell\}^\{k\}for different tensor orderskkat each iterationℓ\\ell\. Since the proof of state evolution in\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14)\]separately tracks the AMP iterates for differentkk, their argument extends naturally to our setting\. Therefore, instead of presenting a new proof here, we describe below how to adapt the proof of their state evolution result\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Theorem 6\]to establish our[D\.3](https://arxiv.org/html/2606.28573#A4.Thmlem3)\.

The proof of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Theorem 6\]proceeds in three main steps: \(i\) applying a Gaussian conditioning lemma\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Lemma A\.2\]to characterize the conditional distribution of the Gaussian tensors given their linear measurements from prior AMP iterations; \(ii\) deriving the state evolution for “Long AMP” \(LAMP\), a variant of tensor AMP that utilizes Gaussian tensors projected onto the orthogonal complement of the subspace spanned by previous iterates; and \(iii\) establishing the asymptotic equivalence between the standard tensor AMP and LAMP via an approximation argument\. In what follows, we detail how to adapt each of these steps to our setting\. We omit the details regarding the reduction to the well\-conditioned case and the extension toD=∞D=\\infty\(Appendices A\.7 and A\.8 in\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14)\]\), as these arguments apply directly to our setting without modification\.

##### Gaussian conditioning lemma\.

In this section, we adapt the statement and proof of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Lemma A\.2\]to our setting\. Introducing the notations forℓ≥0\\ell\\geq 0,k≥1k\\geq 1:

𝒇ℓk=fℓk​\(𝒖0,⋯,𝒖ℓ\),𝒗ℓk=1d​k​σk​𝑮\(k,s\)​\{𝒇ℓk\}−∑j≤ℓBℓ,j,k​𝒇j−1k,\\displaystyle\\text\{$f$\}\_\{\\ell\}^\{k\}=\\,f\_\{\\ell\}^\{k\}\(\\text\{$u$\}^\{0\},\\cdots,\\text\{$u$\}^\{\\ell\}\),\\quad\\text\{$v$\}\_\{\\ell\}^\{k\}=\\,\\frac\{1\}\{\\sqrt\{d\}\}k\\sigma\_\{k\}\\text\{$G$\}^\{\(k,s\)\}\\\{\\text\{$f$\}\_\{\\ell\}^\{k\}\\\}\-\\sum\_\{j\\leq\\ell\}B\_\{\\ell,j,k\}\\text\{$f$\}\_\{j\-1\}^\{k\},the key conclusion of this lemma is a characterization of the conditional distribution of the Gaussian random tensors\{𝑮\(k,s\)\}k≥1\\\{\\text\{$G$\}^\{\(k,s\)\}\\\}\_\{k\\geq 1\}, given theσ\\sigma\-algebraℱt=σ​\(\{𝒖ℓ,𝒇ℓk,𝒗ℓk\}ℓ≤t,k≤D\)\\mathcal\{F\}\_\{t\}=\\sigma\(\\\{\\text\{$u$\}^\{\\ell\},\\text\{$f$\}\_\{\\ell\}^\{k\},\\text\{$v$\}\_\{\\ell\}^\{k\}\\\}\_\{\\ell\\leq t,\\,k\\leq D\}\)generated by AMP iterations up to timett\. More accurately, we aim to derive an expression for𝔼​\[𝑮\(k,s\)\|ℱt\]\\mathbb\{E\}\[\\text\{$G$\}^\{\(k,s\)\}\|\\mathcal\{F\}\_\{t\}\]analogous to Eq\. \(A\.19\) in\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14)\]\.

Following the proof of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Lemma A\.2\], we know that𝔼​\[𝑮\(k,s\)\|ℱt\]\\mathbb\{E\}\[\\text\{$G$\}^\{\(k,s\)\}\|\\mathcal\{F\}\_\{t\}\]is just the conditional expectation of𝑮\(k,s\)\\text\{$G$\}^\{\(k,s\)\}given linear measurements:

𝑮\(k,s\)​\{𝒇ℓk\}=𝒚ℓk,ℓ≤t\.\\text\{$G$\}^\{\(k,s\)\}\\\{\\text\{$f$\}\_\{\\ell\}^\{k\}\\\}=\\text\{$y$\}\_\{\\ell\}^\{k\},\\quad\\ell\\leq t\.Using the method of Lagrange multipliers, one can write

𝔼​\[𝑮\(k,s\)\|ℱt\]=𝑮^\(k,s\):=∑ℓ=0t∑j=1k\(𝒇ℓk\)⊗\(j−1\)⊗𝒎ℓ⊗\(𝒇ℓk\)⊗\(k−j\),\\mathbb\{E\}\[\\text\{$G$\}^\{\(k,s\)\}\|\\mathcal\{F\}\_\{t\}\]=\\,\\widehat\{\\text\{$G$\}\}^\{\(k,s\)\}:=\\sum\_\{\\ell=0\}^\{t\}\\sum\_\{j=1\}^\{k\}\(\\text\{$f$\}\_\{\\ell\}^\{k\}\)^\{\\otimes\(j\-1\)\}\\otimes\\text\{$m$\}\_\{\\ell\}\\otimes\(\\text\{$f$\}\_\{\\ell\}^\{k\}\)^\{\\otimes\(k\-j\)\},where\{𝒎ℓ\}ℓ≤t\\\{\\text\{$m$\}\_\{\\ell\}\\\}\_\{\\ell\\leq t\}are vectors inℝd\\mathbb\{R\}^\{d\}\. Further,𝑮^\(k,s\)\\widehat\{\\text\{$G$\}\}^\{\(k,s\)\}must also satisfy the same set of linear constraints:

𝑮^\(k,s\)​\{𝒇ℓk\}=𝒚ℓk,ℓ≤t\.\\widehat\{\\text\{$G$\}\}^\{\(k,s\)\}\\\{\\text\{$f$\}\_\{\\ell\}^\{k\}\\\}=\\text\{$y$\}\_\{\\ell\}^\{k\},\\quad\\ell\\leq t\.The vectors\{𝒎ℓ\}ℓ≤t⊂ℝd\\\{\\text\{$m$\}\_\{\\ell\}\\\}\_\{\\ell\\leq t\}\\subset\\mathbb\{R\}^\{d\}can then be solved from the above linear constraints, in a similar way as in the proof of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Lemma A\.2\], completing the proof of the Gaussian conditioning lemma\.

##### State evolution of LAMP\.

We now work on adapting the proof of LAMP state evolution to our setting\. Similar to the definitions in Appendix A\.3 of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14)\], in LAMP we will use𝒫t,k⟂​𝑮\(k,s\)\\mathcal\{P\}\_\{t,k\}^\{\\perp\}\\text\{$G$\}^\{\(k,s\)\}instead of𝑮\(k,s\)\\text\{$G$\}^\{\(k,s\)\}in thett\-th iteration, where𝒫t,k⟂\\mathcal\{P\}\_\{t,k\}^\{\\perp\}denotes the orthogonal projection onto the linear subspace of all tensors𝑻\\boldsymbol\{T\}satisfying𝑻​\{𝒇sk\}=0\\boldsymbol\{T\}\\\{\\text\{$f$\}\_\{s\}^\{k\}\\\}=0for alls<ts<t\. The Onsager correction terms are now defined via the Gram matrices of the𝒇sk\\text\{$f$\}\_\{s\}^\{k\}’s\.

To establish state evolution for LAMP, we follow the proof strategy of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Theorem 7\]\. Part\(a\)\(a\)can be proved in exactly the same way by exploiting the independence between𝒫t,k⟂​𝑮\(k,s\)\\mathcal\{P\}\_\{t,k\}^\{\\perp\}\\text\{$G$\}^\{\(k,s\)\}andℱt\\mathcal\{F\}\_\{t\}\. Further, parts\(b\)\(b\)and\(c\)\(c\)directly follow from the conditional law and expectation\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Eq\.s \(A\.32\) and \(A\.33\)\]in part\(a\)\(a\), by replacing the𝒇s\\text\{$f$\}\_\{s\}’s with the𝒇sk\\text\{$f$\}\_\{s\}^\{k\}’s in the original proof\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14), Appendices A\.5\.2 and A\.5\.3\]\.

##### Asymptotic equivalence between AMP and LAMP\.

This asymptotic equivalence is established in Appendix A\.6 of\[[EAMS21](https://arxiv.org/html/2606.28573#bib.bibx14)\]for the case𝒇sk=𝒇s\\text\{$f$\}\_\{s\}^\{k\}=\\text\{$f$\}\_\{s\}for allkk\. In fact, a stronger result is proven there: not only are the AMP and LAMP iterates asymptotically equivalent, but this equivalence holds individually for the iterates corresponding to each tensor orderkk\. Examining their proof, we observe that it generalizes straightforwardly to our setting by simply replacing the𝒇s\\text\{$f$\}\_\{s\}’s with the𝒇sk\\text\{$f$\}\_\{s\}^\{k\}’s\. ∎

### D\.6Auxiliary lemmas

###### Lemma D\.4\.

Recall the definition ofq∗​\(r\)q\_\{\*\}\(r\)from[Eq\.46](https://arxiv.org/html/2606.28573#S5.E46)\. For anyr∈\[−1,1\]r\\in\[\-1,1\], we have

- \(a\)Ifξσ​σ′​\(1\)\>ξσ​σ′′​\(1\)​\(1−r2\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\>\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)\(1\-r^\{2\}\), thenq∗​\(r\)=1q\_\{\*\}\(r\)=1\.
- \(b\)Ifξσ​σ′​\(1\)≤ξσ​σ′′​\(1\)​\(1−r2\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\\leq\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)\(1\-r^\{2\}\), thenq∗​\(r\)q\_\{\*\}\(r\)is the unique solution to the equation ξσ​σ′​\(q\)=ξσ​σ′′​\(q\)​\(q−r2\),q∈\[r2,1\]\.\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)=\\,\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)\(q\-r^\{2\}\),\\quad q\\in\[r^\{2\},1\]\.

###### Proof\.

Note that

dd​q​\(ξσ​σ′​\(q\)q−r2\)=ξσ​σ′′​\(q\)\(q−r2\)2​\(q−ξσ​σ′​\(q\)ξσ​σ′′​\(q\)−r2\)\.\\frac\{\\textup\{d\}\}\{\\textup\{d\}q\}\\left\(\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\{q\-r^\{2\}\}\\right\)=\\,\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)\}\{\(q\-r^\{2\}\)^\{2\}\}\\left\(q\-\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)\}\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)\}\-r^\{2\}\\right\)\.It is easy to show that the mappingq↦q−ξσ​σ′​\(q\)/ξσ​σ′′​\(q\)q\\mapsto q\-\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)/\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)is monotone increasing\. Therefore, in case\(a\)\(a\)we have1−ξσ​σ′​\(1\)/ξσ​σ′′​\(1\)<r21\-\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)/\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)<r^\{2\}, henceq−ξσ​σ′​\(q\)/ξσ​σ′′​\(q\)<r2q\-\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)/\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)<r^\{2\}for allq∈\[r2,1\]q\\in\[r^\{2\},1\], which implies thatq∗​\(r\)=1q\_\{\*\}\(r\)=1\. Similarly, in case\(b\)\(b\)we know that the equationξσ​σ′​\(q\)=ξσ​σ′′​\(q\)​\(q−r2\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\)=\\,\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(q\)\(q\-r^\{2\}\)has a unique solution on\[r2,1\]\[r^\{2\},1\], andq∗​\(r\)q\_\{\*\}\(r\)must be this solution\. ∎

###### Lemma D\.5\.

For anyt,s∈\[−1,1\]t,s\\in\[\-1,1\], we have

ξφ​σ​\(t​s\)≤ξσ​σ​\(s2\)⋅ξφ​φ​\(t2\),\\displaystyle\\xi\_\{\\varphi\\sigma\}\(ts\)\\leq\\sqrt\{\\xi\_\{\\sigma\\sigma\}\(s^\{2\}\)\\cdot\\xi\_\{\\varphi\\varphi\}\(t^\{2\}\)\}\\,,with equality if and only if there existsb∈ℝb\\in\\mathbb\{R\}such thatσk​sk=b​φk​tk\\sigma\_\{k\}s^\{k\}=b\\varphi\_\{k\}t^\{k\}for allk∈ℕk\\in\\mathbb\{N\}\.

###### Proof\.

Follows from directly applying the Cauchy\-Schwarz inequality\. ∎

###### Lemma D\.6\.

Assume one of the following:

- \(i\)For anyα¯\>0\\overline\{\\alpha\}\>0, the equationx=α¯​\(1−x\)​ξφ​φ′​\(x\)x=\\overline\{\\alpha\}\(1\-x\)\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\(x\)has a unique solution over\[0,1\]\[0,1\]\.
- \(ii\)σ\\sigmasatisfiesξσ​σ′′​\(1\)≤ξσ​σ′​\(1\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)\\leq\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\.

Then, the optimality condition \([48](https://arxiv.org/html/2606.28573#S5.E48)\) holds for allα¯\>0\\overline\{\\alpha\}\>0\.

###### Proof\.

RecallrAr\_\{A\}andrBr\_\{B\}defined in the proof of[5\.1](https://arxiv.org/html/2606.28573#S5.Thmthm1)and[5\.3](https://arxiv.org/html/2606.28573#S5.Thmlem3)\. We already showed thatξσ​σ′​\(1\)≥ξσ​σ′′​\(1\)​\(1−rB2\)\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(1\)\\geq\\,\\xi\_\{\\sigma\\sigma\}^\{\\prime\\prime\}\(1\)\(1\-r\_\{B\}^\{2\}\)impliesrA=rBr\_\{A\}=r\_\{B\}\. Under our condition \(ii\), this is true for allα¯\>0\\overline\{\\alpha\}\>0, which implies[Eq\.48](https://arxiv.org/html/2606.28573#S5.E48)\.

We next assume condition \(i\), and let

r∗′=arg⁡maxr:\(r,1\)∈ABayes⁡𝒱α¯∗​\(r,q∗​\(r\)\)\.r\_\{\*\}^\{\\prime\}=\\,\\arg\\max\_\{r:\(r,1\)\\in A\_\{\\mbox\{\\tiny\\rm Bayes\}\}\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\(r,q\_\{\*\}\(r\)\)\.Then, from the proof of[5\.1](https://arxiv.org/html/2606.28573#S5.Thmthm1)we know that

d​𝒱α¯∗d​r​\(r,q∗​\(r\)\)=ξφ​σ′​\(r\)−ξσ​σ′​\(q∗​\(r\)\)α¯​rq∗​\(r\)−r2\.\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r,q\_\{\*\}\(r\)\)=\\,\\xi\_\{\\varphi\\sigma\}^\{\\prime\}\(r\)\-\\sqrt\{\\frac\{\\xi\_\{\\sigma\\sigma\}^\{\\prime\}\(q\_\{\*\}\(r\)\)\}\{\\overline\{\\alpha\}\}\}\\frac\{r\}\{\\sqrt\{q\_\{\*\}\(r\)\-r^\{2\}\}\}\.By our condition \(i\), we know that the fixed point equation

c2=α¯​ξφ​φ′​\(c21\+c2\)c^\{2\}=\\,\\overline\{\\alpha\}\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{c^\{2\}\}\{1\+c^\{2\}\}\\right\)has a unique solutionc=cBayesc=c\_\{\\mbox\{\\tiny\\rm Bayes\}\}\. As a consequence, for anyr∈\[rA,rB\]∪\[−rB,−rA\]r\\in\[r\_\{A\},r\_\{B\}\]\\cup\[\-r\_\{B\},\-r\_\{A\}\], we have

c2:=r2q∗​\(r\)−r2≥cBayes2⟹c2≥α¯​ξφ​φ′​\(c21\+c2\)\.c^\{2\}:=\\frac\{r^\{2\}\}\{q\_\{\*\}\(r\)\-r^\{2\}\}\\geq c\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{2\}\\implies c^\{2\}\\geq\\overline\{\\alpha\}\\xi\_\{\\varphi\\varphi\}^\{\\prime\}\\left\(\\frac\{c^\{2\}\}\{1\+c^\{2\}\}\\right\)\.Similar to the proof of[5\.1](https://arxiv.org/html/2606.28573#S5.Thmthm1), this implies that

d​𝒱α¯∗d​r​\(r,q∗​\(r\)\)≤0,r∈\[rA,rB\],d​𝒱α¯∗d​r​\(r,q∗​\(r\)\)≥0,r∈\[−rB,−rA\],\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r,q\_\{\*\}\(r\)\)\\leq 0,\\,\\,r\\in\[r\_\{A\},r\_\{B\}\],\\quad\\frac\{\{\\rm d\}\\mathscr\{V\}\_\{\\overline\{\\alpha\}\}^\{\*\}\}\{\{\\rm d\}r\}\(r,q\_\{\*\}\(r\)\)\\geq 0,\\,\\,r\\in\[\-r\_\{B\},\-r\_\{A\}\],leading to[Eq\.48](https://arxiv.org/html/2606.28573#S5.E48)\. This completes the proof\. ∎

## Appendix EAppendix for Section[7](https://arxiv.org/html/2606.28573#S7)

### E\.1Proof of[7\.3](https://arxiv.org/html/2606.28573#S7.Thmlem3)and[7\.4](https://arxiv.org/html/2606.28573#S7.Thmlem4)

###### Proof of[7\.3](https://arxiv.org/html/2606.28573#S7.Thmlem3)\.

Note that[Eq\.62](https://arxiv.org/html/2606.28573#S7.E62)implies the following recursive relation between\(Rt,Qt\)\(R\_\{t\},Q\_\{t\}\)and\(Rt\+1,Qt\+1\)\(R\_\{t\+1\},Q\_\{t\+1\}\):

Qt\+1=\\displaystyle Q\_\{t\+1\}=\\,Rt\+1𝖳​Rt\+1\+1α​𝔼\(Rt,Qt\)​\[Ft​\(Z¯t,Y\)𝖳​Ft​\(Z¯t,Y\)\],\\displaystyle R\_\{t\+1\}^\{\\sf T\}R\_\{t\+1\}\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[F\_\{t\}\\left\(\\overline\{Z\}\_\{t\},Y\\right\)^\{\\sf T\}F\_\{t\}\\left\(\\overline\{Z\}\_\{t\},Y\\right\)\\right\],Rt\+1=\\displaystyle R\_\{t\+1\}=\\,𝔼\(Rt,Qt\)​\[∂Ft∂z¯0​\(Z¯t,Y\)\],Y=φ​\(Z¯0,ε\),\\displaystyle\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[\\frac\{\\partial F\_\{t\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{t\},Y\\right\)\\right\],\\quad Y=\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\),where𝔼\(Rt,Qt\)\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}represents the expectation taken under\(Z¯t,Z¯0\)𝖳∼𝖭​\(0,\[QtRt𝖳RtIk\]\)\(\\overline\{Z\}\_\{t\},\\overline\{Z\}\_\{0\}\)^\{\\sf T\}\\sim\{\\sf N\}\\left\(0,\\begin\{bmatrix\}Q\_\{t\}&R\_\{t\}^\{\\sf T\}\\\\ R\_\{t\}&I\_\{k\}\\end\{bmatrix\}\\right\), independent ofε∼Pε\\varepsilon\\sim P\_\{\\varepsilon\}\. Applying Stein’s lemma, we obtain that

Rt\+1=𝔼\(Rt,Qt\)​\[∂Ft∂z¯0​\(Z¯t,φ​\(Z¯0,ε\)\)\]\\displaystyle R\_\{t\+1\}=\\,\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[\\frac\{\\partial F\_\{t\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{t\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\displaystyle=\\,𝔼\(Rt,Qt\)​\[\(Ik−Rt​Qt−1​Rt𝖳\)−1​\(Z¯0−Z¯t​Qt−1​Rt𝖳\)𝖳​Ft​\(Z¯t,φ​\(Z¯0,ε\)\)\]\.\\displaystyle\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[\\left\(I\_\{k\}\-R\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)^\{\-1\}\\left\(\\overline\{Z\}\_\{0\}\-\\overline\{Z\}\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)^\{\\sf T\}F\_\{t\}\\left\(\\overline\{Z\}\_\{t\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]\.DenotingHt=\(Z¯0−Z¯t​Qt−1​Rt𝖳\)​\(Ik−Rt​Qt−1​Rt𝖳\)−1∈ℝ1×kH\_\{t\}=\\left\(\\overline\{Z\}\_\{0\}\-\\overline\{Z\}\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)\\left\(I\_\{k\}\-R\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)^\{\-1\}\\in\\mathbb\{R\}^\{1\\times k\}, we can write

Rt\+1=𝔼\(Rt,Qt\)​\[Ht𝖳​Ft\]=𝔼\(Rt,Qt\)​\[𝔼​\[Ht\|Z¯t,Y\]𝖳​Ft\],R\_\{t\+1\}=\\,\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[H\_\{t\}^\{\\sf T\}F\_\{t\}\\right\]=\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[\\mathbb\{E\}\\left\[H\_\{t\}\|\\overline\{Z\}\_\{t\},Y\\right\]^\{\\sf T\}F\_\{t\}\\right\],sinceFtF\_\{t\}is only a function ofZ¯t\\overline\{Z\}\_\{t\}andYY\. Applying[E\.1](https://arxiv.org/html/2606.28573#A5.Thmlem1), we know that for fixed\(Rt,Qt\)\(R\_\{t\},Q\_\{t\}\), the range of all possible\(Rt\+1,Qt\+1\)\(R\_\{t\+1\},Q\_\{t\+1\}\)is the set of all\(R,Q\)∈ℝk×m×𝒮\+m\(R,Q\)\\in\\mathbb\{R\}^\{k\\times m\}\\times\{\\mathcal\{S\}\}\_\{\+\}^\{m\}satisfying

R𝖳​R⪯Q,R​\(Q−R𝖳​R\)−1​R𝖳⪯α​𝔼\(Rt,Qt\)​\[𝔼​\[Ht\|Z¯t,Y\]𝖳​𝔼​\[Ht\|Z¯t,Y\]\]\.R^\{\\sf T\}R\\preceq Q,\\quad R\(Q\-R^\{\\sf T\}R\)^\{\-1\}R^\{\\sf T\}\\preceq\\alpha\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[\\mathbb\{E\}\\left\[H\_\{t\}\|\\overline\{Z\}\_\{t\},Y\\right\]^\{\\sf T\}\\mathbb\{E\}\\left\[H\_\{t\}\|\\overline\{Z\}\_\{t\},Y\\right\]\\right\]\.We next show that the right\-hand side of the above inequality only depends on

Rt​\(Qt−Rt𝖳​Rt\)−1​Rt𝖳\.R\_\{t\}\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{\-1\}R\_\{t\}^\{\\sf T\}\.For future convenience, define for allt∈ℕt\\in\\mathbb\{N\}:

Mt=Rt​\(Qt−Rt𝖳​Rt\)−1/2∈ℝk×m,M\_\{t\}=R\_\{t\}\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{\-1/2\}\\in\\mathbb\{R\}^\{k\\times m\},and

G=Z¯0𝖳∈ℝk,Z=\(Qt−Rt𝖳​Rt\)−1/2​\(Z¯t−Z¯0​Rt\)𝖳∈ℝm\.G=\\overline\{Z\}\_\{0\}^\{\\sf T\}\\in\\mathbb\{R\}^\{k\},\\,Z=\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{\-1/2\}\\left\(\\overline\{Z\}\_\{t\}\-\\overline\{Z\}\_\{0\}R\_\{t\}\\right\)^\{\\sf T\}\\in\\mathbb\{R\}^\{m\}\.Note thatGGandZZare column vectors, and that the joint distribution of\(Y,G,Z\)\(Y,G,Z\)is as described in[Eq\.17](https://arxiv.org/html/2606.28573#S3.E17)of[1](https://arxiv.org/html/2606.28573#Thmdefn1)\. Further, one can write

Z¯t𝖳=Rt𝖳​G\+\(Qt−Rt𝖳​Rt\)1/2​Z=\(Qt−Rt𝖳​Rt\)1/2​\(Z\+Mt𝖳​G\),\\displaystyle\\overline\{Z\}\_\{t\}^\{\\sf T\}=\\,R\_\{t\}^\{\\sf T\}G\+\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{1/2\}Z=\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{1/2\}\\left\(Z\+M\_\{t\}^\{\\sf T\}G\\right\),and

Ht𝖳=\\displaystyle H\_\{t\}^\{\\sf T\}=\\,\(Ik−Rt​Qt−1​Rt𝖳\)−1​\(Z¯0𝖳−Rt​Qt−1​Z¯t𝖳\)\\displaystyle\\left\(I\_\{k\}\-R\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)^\{\-1\}\\left\(\\overline\{Z\}\_\{0\}^\{\\sf T\}\-R\_\{t\}Q\_\{t\}^\{\-1\}\\overline\{Z\}\_\{t\}^\{\\sf T\}\\right\)=\\displaystyle=\\,\(Ik−Rt​Qt−1​Rt𝖳\)−1​\(G−Rt​Qt−1​Z¯t𝖳\)\\displaystyle\\left\(I\_\{k\}\-R\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)^\{\-1\}\\left\(G\-R\_\{t\}Q\_\{t\}^\{\-1\}\\overline\{Z\}\_\{t\}^\{\\sf T\}\\right\)=\\displaystyle=\\,G−\(Ik−Rt​Qt−1​Rt𝖳\)−1​Rt​Qt−1​\(Qt−Rt𝖳​Rt\)1/2​Z\\displaystyle G\-\\left\(I\_\{k\}\-R\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)^\{\-1\}R\_\{t\}Q\_\{t\}^\{\-1\}\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{1/2\}Z=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,G−Rt​\(Qt−Rt𝖳​Rt\)−1/2​Z=G−Mt​Z,\\displaystyle G\-R\_\{t\}\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{\-1/2\}Z=G\-M\_\{t\}Z,where\(i\)\(i\)follows from the matrix identity

\(Ik−Rt​Qt−1​Rt𝖳\)−1​Rt​Qt−1​\(Qt−Rt𝖳​Rt\)=Rt\.\\left\(I\_\{k\}\-R\_\{t\}Q\_\{t\}^\{\-1\}R\_\{t\}^\{\\sf T\}\\right\)^\{\-1\}R\_\{t\}Q\_\{t\}^\{\-1\}\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)=R\_\{t\}\.We finally obtain that

α​𝔼\(Rt,Qt\)​\[𝔼​\[Ht\|Z¯t,Y\]𝖳​𝔼​\[Ht\|Z¯t,Y\]\]\\displaystyle\\alpha\\mathbb\{E\}\_\{\(R\_\{t\},Q\_\{t\}\)\}\\left\[\\mathbb\{E\}\\left\[H\_\{t\}\|\\overline\{Z\}\_\{t\},Y\\right\]^\{\\sf T\}\\mathbb\{E\}\\left\[H\_\{t\}\|\\overline\{Z\}\_\{t\},Y\\right\]\\right\]=\\displaystyle=\\,α​𝔼​\[𝔼​\[G−Mt​Z\|Mt𝖳​G\+Z,Y\]​𝔼​\[G−Mt​Z\|Mt𝖳​G\+Z,Y\]𝖳\]\.\\displaystyle\\alpha\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-M\_\{t\}Z\\Big\|M\_\{t\}^\{\\sf T\}G\+Z,Y\\right\]\\mathbb\{E\}\\left\[G\-M\_\{t\}Z\\Big\|M\_\{t\}^\{\\sf T\}G\+Z,Y\\right\]^\{\\sf T\}\\right\]\.Therefore,

Mt\+1​Mt\+1𝖳⪯α​𝔼​\[𝔼​\[G−Mt​Z\|Mt𝖳​G\+Z,Y\]​𝔼​\[G−Mt​Z\|Mt𝖳​G\+Z,Y\]𝖳\],M\_\{t\+1\}M\_\{t\+1\}^\{\\sf T\}\\preceq\\alpha\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\-M\_\{t\}Z\\Big\|M\_\{t\}^\{\\sf T\}G\+Z,Y\\right\]\\mathbb\{E\}\\left\[G\-M\_\{t\}Z\\Big\|M\_\{t\}^\{\\sf T\}G\+Z,Y\\right\]^\{\\sf T\}\\right\],and givenMtM\_\{t\}, anyMt\+1M\_\{t\+1\}satisfying the above inequality is achievable by someFtF\_\{t\}\.

Now we are in position to prove\(a\)\(a\)and\(b\)\(b\)\. By direct calculation, we know that

R1​\(Q1−R1𝖳​R1\)−1​R1𝖳=M1​M1𝖳⪯α​𝔼​\[𝔼​\[G\|Y\]​𝔼​\[G\|Y\]𝖳\]=C1​C1𝖳,R\_\{1\}\\left\(Q\_\{1\}\-R\_\{1\}^\{\\sf T\}R\_\{1\}\\right\)^\{\-1\}R\_\{1\}^\{\\sf T\}=M\_\{1\}M\_\{1\}^\{\\sf T\}\\preceq\\alpha\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[G\|Y\\right\]\\mathbb\{E\}\\left\[G\|Y\\right\]^\{\\sf T\}\\right\]=C\_\{1\}C\_\{1\}^\{\\sf T\},and every pair\(R1,Q1\)\(R\_\{1\},Q\_\{1\}\)satisfying the above inequality is achievable by some choice ofF0F\_\{0\}\. Here, the sequence\{Ct\}t=1∞\\\{C\_\{t\}\\\}\_\{t=1\}^\{\\infty\}is defined in[1](https://arxiv.org/html/2606.28573#Thmdefn1)\. Using induction, one can show that for anyt∈ℕt\\in\\mathbb\{N\},

Rt​\(Qt−Rt𝖳​Rt\)−1​Rt𝖳=Mt​Mt𝖳⪯Ct​Ct𝖳,R\_\{t\}\\left\(Q\_\{t\}\-R\_\{t\}^\{\\sf T\}R\_\{t\}\\right\)^\{\-1\}R\_\{t\}^\{\\sf T\}=M\_\{t\}M\_\{t\}^\{\\sf T\}\\preceq C\_\{t\}C\_\{t\}^\{\\sf T\},and that any pair\(Rt,Qt\)\(R\_\{t\},Q\_\{t\}\)satisfying the above inequality is achievable by some choice of\{Fs\}s=0t−1\\\{F\_\{s\}\\\}\_\{s=0\}^\{t\-1\}\.

##### Proof of\(a\)\(a\)\.

The conclusion directly follows from the fact that

CBayes​CBayes𝖳=limt→∞Ct​Ct𝖳⪰Mt​Mt𝖳\.C\_\{\\mbox\{\\tiny\\rm Bayes\}\}C\_\{\\mbox\{\\tiny\\rm Bayes\}\}^\{\\sf T\}=\\lim\_\{t\\to\\infty\}C\_\{t\}C\_\{t\}^\{\\sf T\}\\succeq M\_\{t\}M\_\{t\}^\{\\sf T\}\.

##### Proof of\(b\)\(b\)\.

For any\(R,Q\)∈int⁡ABayes\(R,Q\)\\in\\operatorname\{int\}A\_\{\\mbox\{\\tiny\\rm Bayes\}\}, there existsT11∈ℕT\_\{11\}\\in\\mathbb\{N\}such that

R​\(Q−R𝖳​R\)−1​R𝖳⪯CT11​CT11𝖳\.R\\left\(Q\-R^\{\\sf T\}R\\right\)^\{\-1\}R^\{\\sf T\}\\preceq C\_\{T\_\{11\}\}C\_\{T\_\{11\}\}^\{\\sf T\}\.Hence,\(R,Q\)\(R,Q\)is achievable by some sequence of Lipschitz functions\{Ft\}t≤T11−1\\\{F\_\{t\}\\\}\_\{t\\leq T\_\{11\}\-1\}, completing the proof of[7\.3](https://arxiv.org/html/2606.28573#S7.Thmlem3)\. ∎

###### Proof of[7\.4](https://arxiv.org/html/2606.28573#S7.Thmlem4)\.

SinceFFis an\(R,Q\)\(R,Q\)\-contraction, we know that

R𝖳=𝔼​\[∂F∂G​\(ZR,Q,φ​\(G,ε\)\)\],Q=R𝖳​R\+1α​𝔼​\[F​\(ZR,Q,φ​\(G,ε\)\)​F​\(ZR,Q,φ​\(G,ε\)\)𝖳\]\.\\displaystyle R^\{\\sf T\}=\\,\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial G\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\],\\quad Q=R^\{\\sf T\}R\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)F\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)^\{\\sf T\}\\right\]\.By our assumption,\(RT11,QT11\)=\(R,Q\)\(R\_\{T\_\{11\}\},Q\_\{T\_\{11\}\}\)=\(R,Q\), which further implies that\(Y,G,ZR,Q\)\(Y,G,Z\_\{R,Q\}\)has the same distribution as\(Y,Z¯0,Z¯T11\)𝖳\(Y,\\overline\{Z\}\_\{0\},\\overline\{Z\}\_\{T\_\{11\}\}\)^\{\\sf T\}\. Combining this fact withFT11=FF\_\{T\_\{11\}\}=F, we know that\(RT11\+1,QT11\+1\)=\(R,Q\)\(R\_\{T\_\{11\}\+1\},Q\_\{T\_\{11\}\+1\}\)=\(R,Q\)\. Repeating the same argument yields that\(Rt,Qt\)=\(R,Q\)\(R\_\{t\},Q\_\{t\}\)=\(R,Q\)for allT11\+1≤t≤T1T\_\{11\}\+1\\leq t\\leq T\_\{1\}\.

Next we show thatlimT1→∞PT1−1=Q\\lim\_\{T\_\{1\}\\to\\infty\}P\_\{T\_\{1\}\-1\}=Q\. According to AMP state evolution, we obtain the following recursive relationship:

Pt=\\displaystyle P\_\{t\}=\\,𝔼​\[Z¯t\+1𝖳​Z¯t\]=Rt\+1𝖳​Rt\+1α​𝔼​\[Ft​\(Z¯t,Y\)𝖳​Ft−1​\(Z¯t−1,Y\)\]\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{t\+1\}^\{\\sf T\}\\overline\{Z\}\_\{t\}\\right\]=R\_\{t\+1\}^\{\\sf T\}R\_\{t\}\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\_\{t\}\\left\(\\overline\{Z\}\_\{t\},Y\\right\)^\{\\sf T\}F\_\{t\-1\}\\left\(\\overline\{Z\}\_\{t\-1\},Y\\right\)\\right\]=\\displaystyle=\\,R𝖳​R\+1α​𝔼​\[F​\(Z¯t,Y\)𝖳​F​\(Z¯t−1,Y\)\],\\displaystyle R^\{\\sf T\}R\+\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[F\\left\(\\overline\{Z\}\_\{t\},Y\\right\)^\{\\sf T\}F\\left\(\\overline\{Z\}\_\{t\-1\},Y\\right\)\\right\],which is a function ofPt−1P\_\{t\-1\}for fixed\(R,Q\)\(R,Q\)\. We denote this function byψ\\psi, then the above recursion can be written asPt=ψ​\(Pt−1\)P\_\{t\}=\\psi\(P\_\{t\-1\}\)\. We further note that, the sequence of Lipschitz functions\{Ft\}t≤T11−1\\\{F\_\{t\}\\\}\_\{t\\leq T\_\{11\}\-1\}can be chosen such thatPT11\+1⪰R𝖳​RP\_\{T\_\{11\}\+1\}\\succeq R^\{\\sf T\}R\. Therefore, in order to show thatlimT1→∞PT1−1=Q\\lim\_\{T\_\{1\}\\to\\infty\}P\_\{T\_\{1\}\-1\}=Q, it suffices to prove thatlimt→∞ψt​\(P\)=Q\\lim\_\{t\\to\\infty\}\\psi^\{t\}\(P\)=Qfor allPPsatisfyingR𝖳​R⪯P⪯QR^\{\\sf T\}R\\preceq P\\preceq Q\. To this end, we first establish the following two claims:

- \(a\)ψ\\psiis increasing with respect to the matrix Loewner order, i\.e\., forR𝖳​R⪯A⪯B⪯QR^\{\\sf T\}R\\preceq A\\preceq B\\preceq Q, we haveψ​\(A\)⪯ψ​\(B\)\\psi\(A\)\\preceq\\psi\(B\)\.
- \(b\)For anyR𝖳​R⪯P⪯QR^\{\\sf T\}R\\preceq P\\preceq Q,ψ​\(P\)=P\\psi\(P\)=Pif and only ifP=QP=Q, i\.e\.,QQis the only fixed point ofψ\\psiin the region\{P∈𝕊\+m:R𝖳​R⪯P⪯Q\}\\\{P\\in\\mathbb\{S\}\_\{\+\}^\{m\}:R^\{\\sf T\}R\\preceq P\\preceq Q\\\}\.

##### Proof of \(a\)\.

DenoteH=B−A⪰0H=B\-A\\succeq 0, and define fort∈\[0,1\]t\\in\[0,1\]:

ψH​\(t\)=ψ​\(A\+t​H\)\.\\psi\_\{H\}\(t\)=\\psi\(A\+tH\)\.Then, it suffices to show thatψH​\(1\)⪰ψH​\(0\)\\psi\_\{H\}\(1\)\\succeq\\psi\_\{H\}\(0\)\. Using Gaussian interpolation \(see, e\.g\.,\[[Tal10](https://arxiv.org/html/2606.28573#bib.bibx39), Lemma 1\.3\.1\]\), we deduce that

ψH′​\(t\)=1α​𝔼A\+t​H​\[JF​\(Z¯,Y\)𝖳​H​JF​\(Z¯′,Y\)\],\\displaystyle\\psi\_\{H\}^\{\\prime\}\(t\)=\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{A\+tH\}\\left\[J\_\{F\}\\left\(\\overline\{Z\},Y\\right\)^\{\\sf T\}HJ\_\{F\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)\\right\],whereJFJ\_\{F\}represents the Jacobian ofFFtaken with respect to its first argument\. SinceA\+t​H⪰R𝖳​RA\+tH\\succeq R^\{\\sf T\}R, we know thatψH′​\(t\)⪰0\\psi\_\{H\}^\{\\prime\}\(t\)\\succeq 0\. This proves thatψH​\(1\)⪰ψH​\(0\)\\psi\_\{H\}\(1\)\\succeq\\psi\_\{H\}\(0\)and completes the proof of claim \(a\)\.

##### Proof of \(b\)\.

First note that, sinceℱm,α,φalg\\mathscr\{F\}\_\{m,\\alpha,\\varphi\}^\{\\mbox\{\\scriptsize\\rm alg\}\}is closed under weak limits, we can assume without loss of generality that[Eq\.21](https://arxiv.org/html/2606.28573#S3.E21)in[2](https://arxiv.org/html/2606.28573#Thmdefn2)holds with strict inequality\. Now, assume by contradiction thatψ​\(P\)=P\\psi\(P\)=Pfor someR𝖳​R⪯P⪯QR^\{\\sf T\}R\\preceq P\\preceq Q,P≠QP\\neq Q, and setH=Q−PH=Q\-P\. Define forβ∈ℝm\\\{0\}\\beta\\in\\mathbb\{R\}^\{m\}\\backslash\\\{0\\\}andt∈\[0,1\]t\\in\[0,1\]:

ψβ,H​\(t\)=β𝖳​ψ​\(P\+t​H\)​β\.\\psi\_\{\\beta,H\}\(t\)=\\beta^\{\\sf T\}\\psi\(P\+tH\)\\beta\.Similar to the proof of \(a\), we can use Gaussian interpolation to show thatψβ,H′​\(t\)≥0\\psi\_\{\\beta,H\}^\{\\prime\}\(t\)\\geq 0andψβ,H′′​\(t\)≥0\\psi\_\{\\beta,H\}^\{\\prime\\prime\}\(t\)\\geq 0\. Therefore

β𝖳​\(Q−P\)​β=\\displaystyle\\beta^\{\\sf T\}\\left\(Q\-P\\right\)\\beta=\\,β𝖳​\(ψ​\(Q\)−ψ​\(P\)\)​β=ψβ,H​\(1\)−ψβ,H​\(0\)=∫01ψβ,H′​\(t\)​d​t\\displaystyle\\beta^\{\\sf T\}\\left\(\\psi\(Q\)\-\\psi\(P\)\\right\)\\beta=\\psi\_\{\\beta,H\}\(1\)\-\\psi\_\{\\beta,H\}\(0\)=\\int\_\{0\}^\{1\}\\psi\_\{\\beta,H\}^\{\\prime\}\(t\)\\textup\{d\}t≤\\displaystyle\\leq\\,ψβ,H′​\(1\)=1α​𝔼Q​\[∇Fβ​\(Z¯,Y\)𝖳​H​∇Fβ​\(Z¯′,Y\)\]\\displaystyle\\psi\_\{\\beta,H\}^\{\\prime\}\(1\)=\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{Q\}\\left\[\\nabla F\_\{\\beta\}\\left\(\\overline\{Z\},Y\\right\)^\{\\sf T\}H\\nabla F\_\{\\beta\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)\\right\]=\\displaystyle=\\,1α​β𝖳​𝔼Q​\[JF​\(Z¯,Y\)𝖳​H​JF​\(Z¯′,Y\)\]​β\.\\displaystyle\\frac\{1\}\{\\alpha\}\\beta^\{\\sf T\}\\mathbb\{E\}\_\{Q\}\\left\[\{J\_\{F\}\}\\left\(\\overline\{Z\},Y\\right\)^\{\\sf T\}H\{J\_\{F\}\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)\\right\]\\beta\.The above calculation implies that for allβ∈ℝm\\\{0\}\\beta\\in\\mathbb\{R\}^\{m\}\\backslash\\\{0\\\},

β𝖳​H​β=β𝖳​\(Q−P\)​β≤1α​β𝖳​𝔼Q​\[JF​\(Z¯,Y\)𝖳​H​JF​\(Z¯′,Y\)\]​β\\displaystyle\\beta^\{\\sf T\}H\\beta=\\beta^\{\\sf T\}\\left\(Q\-P\\right\)\\beta\\leq\\frac\{1\}\{\\alpha\}\\beta^\{\\sf T\}\\mathbb\{E\}\_\{Q\}\\left\[\{J\_\{F\}\}\\left\(\\overline\{Z\},Y\\right\)^\{\\sf T\}H\{J\_\{F\}\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)\\right\]\\beta⟺\\displaystyle\\Longleftrightarrow\\,⟨H,β​β𝖳⟩≤⟨1α​𝔼Q​\[JF​\(Z¯,Y\)𝖳​H​JF​\(Z¯′,Y\)\],β​β𝖳⟩,\\displaystyle\\left\\langle H,\\beta\\beta^\{\\sf T\}\\right\\rangle\\leq\\left\\langle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{Q\}\\left\[\{J\_\{F\}\}\\left\(\\overline\{Z\},Y\\right\)^\{\\sf T\}H\{J\_\{F\}\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)\\right\],\\beta\\beta^\{\\sf T\}\\right\\rangle,thus leading to∀S∈𝒮\+m\\\{0\}\\forall S\\in\{\\mathcal\{S\}\}\_\{\+\}^\{m\}\\backslash\\\{0\\\},

⟨H,S⟩≤⟨1α​𝔼Q​\[JF​\(Z¯,Y\)𝖳​H​JF​\(Z¯′,Y\)\],S⟩=⟨H,1α​𝔼Q​\[JF​\(Z¯,Y\)​S​JF​\(Z¯′,Y\)𝖳\]⟩\.\\displaystyle\\left\\langle H,S\\right\\rangle\\leq\\left\\langle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{Q\}\\left\[\{J\_\{F\}\}\\left\(\\overline\{Z\},Y\\right\)^\{\\sf T\}H\{J\_\{F\}\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)\\right\],S\\right\\rangle=\\left\\langle H,\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{Q\}\\left\[\{J\_\{F\}\}\\left\(\\overline\{Z\},Y\\right\)S\{J\_\{F\}\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)^\{\\sf T\}\\right\]\\right\\rangle\.Recall the triple\(Y,G,Z\)\(Y,G,Z\)andZR,QZ\_\{R,Q\}in[2](https://arxiv.org/html/2606.28573#Thmdefn2)\. It is easy to check thatZ¯=Z¯′\\overline\{Z\}=\\overline\{Z\}^\{\\prime\}has the same conditional distribution asZR,Q𝖳Z\_\{R,Q\}^\{\\sf T\}givenYY\. Therefore,

1α​𝔼Q​\[JF​\(Z¯,Y\)​S​JF​\(Z¯′,Y\)𝖳\]=1α​𝔼​\[∂F∂ZR,Q​\(ZR,Q,φ​\(G,ε\)\)𝖳​S​∂F∂ZR,Q​\(ZR,Q,φ​\(G,ε\)\)\]\.\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{Q\}\\left\[\{J\_\{F\}\}\\left\(\\overline\{Z\},Y\\right\)S\{J\_\{F\}\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)^\{\\sf T\}\\right\]=\\,\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{R,Q\}\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)^\{\\sf T\}S\\,\\frac\{\\partial F\}\{\\partial Z\_\{R,Q\}\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]\.SinceFFis an\(R,Q\)\(R,Q\)\-contraction, the above argument yields that forS≻0S\\succ 0in[2](https://arxiv.org/html/2606.28573#Thmdefn2):

⟨H,S⟩≤\\displaystyle\\left\\langle H,S\\right\\rangle\\leq\\,⟨H,1α​𝔼Q​\[JF​\(Z¯,Y\)​S​JF​\(Z¯′,Y\)𝖳\]⟩\\displaystyle\\left\\langle H,\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\_\{Q\}\\left\[\{J\_\{F\}\}\\left\(\\overline\{Z\},Y\\right\)S\{J\_\{F\}\}\\left\(\\overline\{Z\}^\{\\prime\},Y\\right\)^\{\\sf T\}\\right\]\\right\\rangle=\\displaystyle=\\,⟨H,1α​𝔼​\[∂F∂ZR,Q​\(ZR,Q,φ​\(G,ε\)\)𝖳​S​∂F∂ZR,Q​\(ZR,Q,φ​\(G,ε\)\)\]⟩<⟨H,S⟩,\\displaystyle\\left\\langle H,\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial F\}\{\\partial Z\_\{R,Q\}\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)^\{\\sf T\}S\\,\\frac\{\\partial F\}\{\\partial Z\_\{R,Q\}\}\\left\(Z\_\{R,Q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]\\right\\rangle<\\left\\langle H,S\\right\\rangle,a contradiction\. This proves claim \(b\)\.

We are now in position to show thatlimt→∞ψt​\(P\)=Q\\lim\_\{t\\to\\infty\}\\psi^\{t\}\(P\)=Q\. Sinceψ\\psiis increasing and bounded, we know that\{ψt​\(P\)\}\\\{\\psi^\{t\}\(P\)\\\}is a bounded monotone \(in Loewner order\) sequence, and thus admits a unique limitP∗P\_\{\*\}\. Further, continuity implies thatP∗P\_\{\*\}is a fixed point ofψ\\psi\. By part \(b\), we must haveP∗=QP\_\{\*\}=Q\. This completes the proof of[7\.4](https://arxiv.org/html/2606.28573#S7.Thmlem4)\. ∎

### E\.2Proof of Proposition[7\.5](https://arxiv.org/html/2606.28573#S7.Thmlem5)

###### Proof\.

According to Proposition[7\.1](https://arxiv.org/html/2606.28573#S7.Thmlem1), we already know that\(Zt\)t≥T1\+1\(Z\_\{t\}\)\_\{t\\geq T\_\{1\}\+1\}and\(Z¯t\)t≥T1\+1\(\\overline\{Z\}\_\{t\}\)\_\{t\\geq T\_\{1\}\+1\}are centered multivariate Gaussians, hence it suffices to show that for anyk≥1k\\geq 1,

\(Zt\)T1\+1≤t≤T1\+k∼i\.i\.d\.𝖭\(0,Im\),\(Zt\)T1\+1≤t≤T1\+k⟂⟂\(\(Zt\)1≤t≤T1,V\),\(Z¯t\)T1\+1≤t≤T1\+k∼i\.i\.d\.𝖭\(0,Im\),\(Z¯t\)T1\+1≤t≤T1\+k⟂⟂\(\(Z¯t\)1≤t≤T1,Y\),\\begin\{split\}&\(Z\_\{t\}\)\_\{T\_\{1\}\+1\\leq t\\leq T\_\{1\}\+k\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\),\\quad\(Z\_\{t\}\)\_\{T\_\{1\}\+1\\leq t\\leq T\_\{1\}\+k\}\\perp\\\!\\\!\\\!\\perp\(\(Z\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},V\),\\\\ &\(\\overline\{Z\}\_\{t\}\)\_\{T\_\{1\}\+1\\leq t\\leq T\_\{1\}\+k\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\),\\quad\(\\overline\{Z\}\_\{t\}\)\_\{T\_\{1\}\+1\\leq t\\leq T\_\{1\}\+k\}\\perp\\\!\\\!\\\!\\perp\(\(\\overline\{Z\}\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},Y\),\\end\{split\}\(90\)andRT1\+k\+1=0R\_\{T\_\{1\}\+k\+1\}=0\. We prove the above claim via induction onkk\. Fork=1k=1, using Eq\. \([58](https://arxiv.org/html/2606.28573#S7.E58)\) and Assumption[7\.1](https://arxiv.org/html/2606.28573#S7.Thmass1), we obtain that

𝔼​\[ZT1\+1𝖳​ZT1\+1\]=𝔼​\[FT1​\(Y\)𝖳​FT1​\(Y\)\]=Im\.\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\+1\}^\{\\sf T\}Z\_\{T\_\{1\}\+1\}\\right\]=\\mathbb\{E\}\\left\[F\_\{T\_\{1\}\}\(Y\)^\{\\sf T\}F\_\{T\_\{1\}\}\(Y\)\\right\]=I\_\{m\}\.Further, for any1≤t≤T11\\leq t\\leq T\_\{1\}, we have

𝔼​\[ZT1\+1𝖳​Zt\]=\\displaystyle\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\+1\}^\{\\sf T\}Z\_\{t\}\\right\]=\\,𝔼​\[FT1​\(φ​\(Z¯0,ε\)\)𝖳​F​\(Z¯t−1,φ​\(Z¯0,ε\)\)\]\\displaystyle\\mathbb\{E\}\\left\[F\_\{T\_\{1\}\}\\left\(\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)^\{\\sf T\}F\\left\(\\overline\{Z\}\_\{t\-1\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\displaystyle=\\,𝔼\(R,Q\)​\[FT1​\(φ​\(Z¯0,ε\)\)𝖳​F​\(Z¯,φ​\(Z¯0,ε\)\)\]=0\.\\displaystyle\\mathbb\{E\}\_\{\(R,Q\)\}\\left\[F\_\{T\_\{1\}\}\\left\(\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)^\{\\sf T\}F\\left\(\\overline\{Z\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\,0\.Therefore,ZT1\+1∼𝖭​\(0,Im\)Z\_\{T\_\{1\}\+1\}\\sim\{\\sf N\}\(0,I\_\{m\}\), and is independent of\(\(Zt\)1≤t≤T1,V\)\(\(Z\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},V\)\. Similarly, by Eq\. \([58](https://arxiv.org/html/2606.28573#S7.E58)\) and Assumption[7\.1](https://arxiv.org/html/2606.28573#S7.Thmass1)we know that

𝔼​\[Z¯T1\+1𝖳​Z¯T1\+1\]=\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+1\}^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\+1\}\\right\]=\\,1α​𝔼​\[GT1\+1​\(V​RT1\+1\+WT1\+1\)𝖳​GT1\+1​\(V​RT1\+1\+WT1\+1\)\]=Im,\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)^\{\\sf T\}G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)\\right\]=I\_\{m\},𝔼​\[Z¯T1\+1𝖳​Z¯0\]=\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+1\}^\{\\sf T\}\\overline\{Z\}\_\{0\}\\right\]=\\,1α​𝔼​\[GT1\+1​\(V​RT1\+1\+WT1\+1\)𝖳​V\]=0,\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)^\{\\sf T\}V\\right\]=0,𝔼​\[Z¯T1\+1𝖳​Z¯t\]=\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+1\}^\{\\sf T\}\\overline\{Z\}\_\{t\}\\right\]=\\,1α​𝔼​\[GT1\+1​\(V​RT1\+1\+WT1\+1\)𝖳​\(V​R\+Wt\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)^\{\\sf T\}\\left\(VR\+W^\{t\}\\right\)\\right\]=\\displaystyle=\\,1α​𝔼​\[GT1\+1​\(V​RT1\+1\+WT1\+1\)𝖳​Wt\]=0,\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+1\}\\left\(VR\_\{T\_\{1\}\+1\}\+W^\{T\_\{1\}\+1\}\\right\)^\{\\sf T\}W^\{t\}\\right\]=0,where the last line follows from the fact thatWt⟂⟂\(V,WT1\+1\)W^\{t\}\\perp\\\!\\\!\\\!\\perp\(V,W^\{T\_\{1\}\+1\}\)\. As a consequence, we deduce thatZ¯T1\+1∼𝖭​\(0,Im\)\\overline\{Z\}\_\{T\_\{1\}\+1\}\\sim\{\\sf N\}\(0,I\_\{m\}\)and is independent of\(\(Z¯t\)1≤t≤T1,Y\)\(\(\\overline\{Z\}\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},Y\)\. Moreover,

RT1\+2=\\displaystyle R\_\{T\_\{1\}\+2\}=\\,𝔼​\[∂FT1\+1∂z¯0​\(Z¯≤T1\+1,φ​\(Z¯0,ε\)\)\]=𝔼​\[Z¯T1\+1​∂ΦT1∂z¯0​\(Z¯≤T1,φ​\(Z¯0,ε\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{T\_\{1\}\+1\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+1\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+1\}\\frac\{\\partial\\Phi\_\{T\_\{1\}\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\displaystyle=\\,𝔼​\[Z¯T1\+1\]​𝔼​\[∂ΦT1∂z¯0​\(Z¯≤T1,φ​\(Z¯0,ε\)\)\]=0\.\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+1\}\\right\]\\mathbb\{E\}\\left\[\\frac\{\\partial\\Phi\_\{T\_\{1\}\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=0\.This completes the base case of our induction\. Now assume that our claim \([90](https://arxiv.org/html/2606.28573#A5.E90)\) holds fork∈ℕk\\in\\mathbb\{N\}\. Fork\+1k\+1, we have

𝔼​\[ZT1\+k\+1𝖳​ZT1\+k\+1\]=𝔼​\[FT1\+k​\(Z¯≤T1\+k,Y\)𝖳​FT1\+k​\(Z¯≤T1\+k,Y\)\]\\displaystyle\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}Z\_\{T\_\{1\}\+k\+1\}\\right\]=\\mathbb\{E\}\\left\[F\_\{T\_\{1\}\+k\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\},Y\)^\{\\sf T\}F\_\{T\_\{1\}\+k\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\},Y\)\\right\]=\\displaystyle=\\,𝔼​\[ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)𝖳​Z¯T1\+k𝖳​Z¯T1\+k​ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)\]\\displaystyle\\mathbb\{E\}\\left\[\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\+k\}^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\+k\}\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)\\right\]=\\displaystyle=\\,𝔼​\[ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)𝖳​𝔼​\[Z¯T1\+k𝖳​Z¯T1\+k\]​ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)\]\\displaystyle\\mathbb\{E\}\\left\[\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)^\{\\sf T\}\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+k\}^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\+k\}\\right\]\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)\\right\]=\\displaystyle=\\,𝔼​\[ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)𝖳​ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)\]=Im,\\displaystyle\\mathbb\{E\}\\left\[\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)^\{\\sf T\}\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)\\right\]=I\_\{m\},and for allt≤T1\+kt\\leq T\_\{1\}\+k,

𝔼​\[ZT1\+k\+1𝖳​Zt\]=𝔼​\[FT1\+k​\(Z¯≤T1\+k,Y\)𝖳​Ft−1​\(Z¯≤t−1,Y\)\]\\displaystyle\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}Z\_\{t\}\\right\]=\\mathbb\{E\}\\left\[F\_\{T\_\{1\}\+k\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\},Y\)^\{\\sf T\}F\_\{t\-1\}\(\\overline\{Z\}\_\{\\leq t\-1\},Y\)\\right\]=\\displaystyle=\\,𝔼​\[ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)𝖳​Z¯T1\+k𝖳​Ft−1​\(Z¯≤t−1,Y\)\]\\displaystyle\\mathbb\{E\}\\left\[\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\+k\}^\{\\sf T\}F\_\{t\-1\}\(\\overline\{Z\}\_\{\\leq t\-1\},Y\)\\right\]=\\displaystyle=\\,𝔼​\[ΦT1\+k−1​\(Z¯≤T1\+k−1,Y\)𝖳​𝔼​\[Z¯T1\+k\]𝖳​Ft−1​\(Z¯≤t−1,Y\)\]=0\.\\displaystyle\\mathbb\{E\}\\left\[\\Phi\_\{T\_\{1\}\+k\-1\}\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\-1\},Y\)^\{\\sf T\}\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+k\}\\right\]^\{\\sf T\}F\_\{t\-1\}\(\\overline\{Z\}\_\{\\leq t\-1\},Y\)\\right\]=0\.This proves\(Zt\)T1\+1≤t≤T1\+k\+1∼i\.i\.d\.𝖭​\(0,Im\)\(Z\_\{t\}\)\_\{T\_\{1\}\+1\\leq t\\leq T\_\{1\}\+k\+1\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\), and are independent of\(\(Zt\)1≤t≤T1,V\)\(\(Z\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},V\)\. Proceeding similarly, we get that

𝔼​\[Z¯T1\+k\+1𝖳​Z¯T1\+k\+1\]=1α​𝔼​\[GT1\+k\+1​\(V​R≤T1\+k\+1\+Z≤T1\+k\+1\)𝖳​GT1\+k\+1​\(V​R≤T1\+k\+1\+Z≤T1\+k\+1\)\]\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\+k\+1\}\\right\]=\\,\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+k\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+k\+1\}\+Z\_\{\\leq T\_\{1\}\+k\+1\}\\right\)^\{\\sf T\}G\_\{T\_\{1\}\+k\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+k\+1\}\+Z\_\{\\leq T\_\{1\}\+k\+1\}\\right\)\\right\]=\(i\)\\displaystyle\\stackrel\{\{\\scriptstyle\(i\)\}\}\{\{=\}\}\\,1α​𝔼​\[GT1\+k\+1​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\+1\)𝖳​GT1\+k\+1​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\+1\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+k\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\+1\}\\right\)^\{\\sf T\}G\_\{T\_\{1\}\+k\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\+1\}\\right\)\\right\]=\\displaystyle=\\,1α​𝔼​\[ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)𝖳​ZT1\+k\+1𝖳​ZT1\+k\+1​ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)^\{\\sf T\}Z\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}Z\_\{T\_\{1\}\+k\+1\}\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)\\right\]=\\displaystyle=\\,1α​𝔼​\[ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)𝖳​𝔼​\[ZT1\+k\+1𝖳​ZT1\+k\+1\]​ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)^\{\\sf T\}\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}Z\_\{T\_\{1\}\+k\+1\}\\right\]\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)\\right\]=\\displaystyle=\\,1α​𝔼​\[ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)𝖳​ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)\]=Im,\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)^\{\\sf T\}\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)\\right\]=I\_\{m\},where\(i\)\(i\)is because ofRt=0R\_\{t\}=0forT1\+2≤t≤T1\+k\+1T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\+1\. For any1≤s≤T1\+k1\\leq s\\leq T\_\{1\}\+k, we deduce that

𝔼​\[Z¯T1\+k\+1𝖳​Z¯s\]=\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}\\overline\{Z\}\_\{s\}\\right\]=\\,1α​𝔼​\[GT1\+k\+1​\(V​R≤T1\+k\+1\+Z≤T1\+k\+1\)𝖳​Gs​\(V​R≤s\+Z≤s\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+k\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+k\+1\}\+Z\_\{\\leq T\_\{1\}\+k\+1\}\\right\)^\{\\sf T\}G\_\{s\}\\left\(VR\_\{\\leq s\}\+Z\_\{\\leq s\}\\right\)\\right\]=\\displaystyle=\\,1α​𝔼​\[GT1\+k\+1​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\+1\)𝖳​Gs​\(V​R≤s\+Z≤s\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+k\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\+1\}\\right\)^\{\\sf T\}G\_\{s\}\\left\(VR\_\{\\leq s\}\+Z\_\{\\leq s\}\\right\)\\right\]=\\displaystyle=\\,1α​𝔼​\[ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)𝖳​ZT1\+k\+1𝖳​Gs​\(V​R≤s\+Z≤s\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)^\{\\sf T\}Z\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}G\_\{s\}\\left\(VR\_\{\\leq s\}\+Z\_\{\\leq s\}\\right\)\\right\]=\\displaystyle=\\,1α​𝔼​\[ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)𝖳​𝔼​\[ZT1\+k\+1\]𝖳​Gs​\(V​R≤s\+Z≤s\)\]=0\.\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)^\{\\sf T\}\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\+k\+1\}\\right\]^\{\\sf T\}G\_\{s\}\\left\(VR\_\{\\leq s\}\+Z\_\{\\leq s\}\\right\)\\right\]=0\.Furthermore,

𝔼​\[Z¯T1\+k\+1𝖳​Z¯0\]=\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}\\overline\{Z\}\_\{0\}\\right\]=\\,1α​𝔼​\[GT1\+k\+1​\(V​R≤T1\+k\+1\+Z≤T1\+k\+1\)𝖳​V\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+k\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+k\+1\}\+Z\_\{\\leq T\_\{1\}\+k\+1\}\\right\)^\{\\sf T\}V\\right\]=\\displaystyle=\\,1α​𝔼​\[ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)𝖳​ZT1\+k\+1𝖳​V\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)^\{\\sf T\}Z\_\{T\_\{1\}\+k\+1\}^\{\\sf T\}V\\right\]=\\displaystyle=\\,1α​𝔼​\[ΨT1\+k​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+k\)𝖳​𝔼​\[ZT1\+k\+1\]𝖳​V\]=0\.\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+k\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+k\}\\right\)^\{\\sf T\}\\mathbb\{E\}\\left\[Z\_\{T\_\{1\}\+k\+1\}\\right\]^\{\\sf T\}V\\right\]=0\.This proves that\(Z¯t\)T1\+1≤t≤T1\+k\+1∼i\.i\.d\.𝖭​\(0,Im\)\(\\overline\{Z\}\_\{t\}\)\_\{T\_\{1\}\+1\\leq t\\leq T\_\{1\}\+k\+1\}\\sim\_\{\\mathrm\{i\.i\.d\.\}\}\{\\sf N\}\(0,I\_\{m\}\), and are independent of\(\(Z¯t\)1≤t≤T1,Y\)\(\(\\overline\{Z\}\_\{t\}\)\_\{1\\leq t\\leq T\_\{1\}\},Y\)\. Finally, we need to show thatRT1\+k\+2=0R\_\{T\_\{1\}\+k\+2\}=0\. Using Eq\. \([58](https://arxiv.org/html/2606.28573#S7.E58)\), it follows that

RT1\+k\+2=\\displaystyle R\_\{T\_\{1\}\+k\+2\}=\\,𝔼​\[∂FT1\+k\+1∂z¯0​\(Z¯≤T1\+k\+1,φ​\(Z¯0,ε\)\)\]=𝔼​\[Z¯T1\+k\+1​∂ΦT1\+k∂z¯0​\(Z¯≤T1\+k,φ​\(Z¯0,ε\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\\frac\{\\partial F\_\{T\_\{1\}\+k\+1\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\+1\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+k\+1\}\\frac\{\\partial\\Phi\_\{T\_\{1\}\+k\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=\\displaystyle=\\,𝔼​\[Z¯T1\+k\+1\]​𝔼​\[∂ΦT1\+k∂z¯0​\(Z¯≤T1\+k,φ​\(Z¯0,ε\)\)\]=0\.\\displaystyle\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+k\+1\}\\right\]\\mathbb\{E\}\\left\[\\frac\{\\partial\\Phi\_\{T\_\{1\}\+k\}\}\{\\partial\\overline\{z\}\_\{0\}\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+k\},\\varphi\\left\(\\overline\{Z\}\_\{0\},\\varepsilon\\right\)\\right\)\\right\]=0\.This completes the induction step and the proof of the proposition\. ∎

### E\.3Proof of Theorem[7\.1](https://arxiv.org/html/2606.28573#S7.Thmthm1)

###### Proof\.

By our assumption, we know that for all1≤s,t≤T1\\leq s,t\\leq T,Gt​GsG\_\{t\}G\_\{s\}is pseudo\-Lipschitz of order22, hence we have almost surely,

𝑾I𝖳​𝑾I=\\displaystyle\\text\{$W$\}\_\{I\}^\{\\sf T\}\\text\{$W$\}\_\{I\}=\\,1n​∑t=1T2∑s=1T2Qt𝖳​GT1\+t\+1​\(𝑾≤T1\+t\+1\)𝖳​GT1\+s\+1​\(𝑾≤T1\+s\+1\)​Qs\\displaystyle\\frac\{1\}\{n\}\\sum\_\{t=1\}^\{T\_\{2\}\}\\sum\_\{s=1\}^\{T\_\{2\}\}Q\_\{t\}^\{\\sf T\}G\_\{T\_\{1\}\+t\+1\}\\left\(\\text\{$W$\}^\{\\leq T\_\{1\}\+t\+1\}\\right\)^\{\\sf T\}G\_\{T\_\{1\}\+s\+1\}\\left\(\\text\{$W$\}^\{\\leq T\_\{1\}\+s\+1\}\\right\)Q\_\{s\}=\\displaystyle=\\,∑t=1T2∑s=1T2Qt𝖳​\(1n​∑i=1dGT1\+t\+1​\(𝒘i≤T1\+t\+1\)𝖳​GT1\+s\+1​\(𝒘i≤T1\+s\+1\)\)​Qs\\displaystyle\\sum\_\{t=1\}^\{T\_\{2\}\}\\sum\_\{s=1\}^\{T\_\{2\}\}Q\_\{t\}^\{\\sf T\}\\left\(\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{d\}G\_\{T\_\{1\}\+t\+1\}\(\\text\{$w$\}\_\{i\}^\{\\leq T\_\{1\}\+t\+1\}\)^\{\\sf T\}G\_\{T\_\{1\}\+s\+1\}\(\\text\{$w$\}\_\{i\}^\{\\leq T\_\{1\}\+s\+1\}\)\\right\)Q\_\{s\}→\\displaystyle\\to\\,1α​∑t=1T2∑s=1T2Qt𝖳​𝔼​\[GT1\+t\+1​\(V​R≤T1\+t\+1\+Z≤T1\+t\+1\)𝖳​GT1\+s\+1​\(V​R≤T1\+s\+1\+Z≤T1\+s\+1\)\]​Qs\\displaystyle\\frac\{1\}\{\\alpha\}\\sum\_\{t=1\}^\{T\_\{2\}\}\\sum\_\{s=1\}^\{T\_\{2\}\}Q\_\{t\}^\{\\sf T\}\\mathbb\{E\}\\left\[G\_\{T\_\{1\}\+t\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+t\+1\}\+Z\_\{\\leq T\_\{1\}\+t\+1\}\\right\)^\{\\sf T\}G\_\{T\_\{1\}\+s\+1\}\\left\(VR\_\{\\leq T\_\{1\}\+s\+1\}\+Z\_\{\\leq T\_\{1\}\+s\+1\}\\right\)\\right\]Q\_\{s\}=\\displaystyle=\\,∑t=1T2∑s=1T2Qt𝖳​𝔼​\[Z¯T1\+t\+1𝖳​Z¯T1\+s\+1\]​Qs=∑t=1T2Qt𝖳​Qt=Im−Q\.\\displaystyle\\sum\_\{t=1\}^\{T\_\{2\}\}\\sum\_\{s=1\}^\{T\_\{2\}\}Q\_\{t\}^\{\\sf T\}\\mathbb\{E\}\\left\[\\overline\{Z\}\_\{T\_\{1\}\+t\+1\}^\{\\sf T\}\\overline\{Z\}\_\{T\_\{1\}\+s\+1\}\\right\]Q\_\{s\}=\\sum\_\{t=1\}^\{T\_\{2\}\}Q\_\{t\}^\{\\sf T\}Q\_\{t\}=I\_\{m\}\-Q\.Similarly, we can show that𝑾F𝖳​𝑾I→0\\text\{$W$\}\_\{F\}^\{\\sf T\}\\text\{$W$\}\_\{I\}\\to 0almost surely\. Therefore,𝑾Q𝖳​𝑾Q→Im\\text\{$W$\}\_\{Q\}^\{\\sf T\}\\text\{$W$\}\_\{Q\}\\to I\_\{m\}almost surely asn→∞n\\to\\infty\. Further, one can show that𝑾∗𝖳​𝑾^nAMP→R\\text\{$W$\}\_\{\*\}^\{\\sf T\}\\widehat\{\\text\{$W$\}\}\_\{n\}^\{\\mbox\{\\tiny\\rm AMP\}\}\\to Ralmost surely\. Using Slutsky’s theorem, it now suffices to consider the empirical joint distribution of the rows of\(𝒚,𝑿𝑾Q\)\(\\text\{$y$\},\\text\{$X$\}\\text\{$W$\}\_\{Q\}\)\. By direct calculation, we obtain that

𝑿𝑾Q=\\displaystyle\\text\{$X$\}\\text\{$W$\}\_\{Q\}=\\,𝑿𝑾F\+𝑿𝑾I=𝑽T1\+dn​F​\(𝑽T1−1,𝒚\)\+1n​∑t=1T2𝑿​GT1\+t\+1​\(𝑾≤T1\+t\+1\)​Qt\\displaystyle\\text\{$X$\}\\text\{$W$\}\_\{F\}\+\\text\{$X$\}\\text\{$W$\}\_\{I\}=\\text\{$V$\}^\{T\_\{1\}\}\+\\frac\{d\}\{n\}F\(\\text\{$V$\}^\{T\_\{1\}\-1\},\\text\{$y$\}\)\+\\frac\{1\}\{\\sqrt\{n\}\}\\sum\_\{t=1\}^\{T\_\{2\}\}\\text\{$X$\}G\_\{T\_\{1\}\+t\+1\}\\left\(\\text\{$W$\}^\{\\leq T\_\{1\}\+t\+1\}\\right\)Q\_\{t\}=\\displaystyle=\\,𝑽T1\+dn​F​\(𝑽T1−1,𝒚\)\+∑t=1T2\(𝑽T1\+t\+1\+∑s=1T1\+t\+1Fs−1​\(𝑽≤s−1,𝒚\)​DT1\+t\+1,s𝖳\)​Qt,\\displaystyle\\text\{$V$\}^\{T\_\{1\}\}\+\\frac\{d\}\{n\}F\(\\text\{$V$\}^\{T\_\{1\}\-1\},\\text\{$y$\}\)\+\\sum\_\{t=1\}^\{T\_\{2\}\}\\left\(\\text\{$V$\}^\{T\_\{1\}\+t\+1\}\+\\sum\_\{s=1\}^\{T\_\{1\}\+t\+1\}F\_\{s\-1\}\(\\text\{$V$\}^\{\\leq s\-1\},\\text\{$y$\}\)D\_\{T\_\{1\}\+t\+1,s\}^\{\\sf T\}\\right\)Q\_\{t\},where by state evolution,

DT1\+t\+1,s=\\displaystyle D\_\{T\_\{1\}\+t\+1,s\}=\\,1n​∑i=1d∂GT1\+t\+1∂𝒘is​\(𝒘i1,⋯,𝒘iT1\+t\+1\)\\displaystyle\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{d\}\\frac\{\\partial G\_\{T\_\{1\}\+t\+1\}\}\{\\partial\\text\{$w$\}\_\{i\}^\{s\}\}\(\\text\{$w$\}\_\{i\}^\{1\},\\cdots,\\text\{$w$\}\_\{i\}^\{T\_\{1\}\+t\+1\}\)→a\.s\.\\displaystyle\\stackrel\{\{\\scriptstyle a\.s\.\}\}\{\{\\to\}\}\\,1α​𝔼​\[∂GT1\+t\+1∂ws​\(V​R≤T1\+t\+1\+Z≤T1\+t\+1\)\]\\displaystyle\\frac\{1\}\{\\alpha\}\\mathbb\{E\}\\left\[\\frac\{\\partial G\_\{T\_\{1\}\+t\+1\}\}\{\\partial w^\{s\}\}\\left\(VR\_\{\\leq T\_\{1\}\+t\+1\}\+Z\_\{\\leq T\_\{1\}\+t\+1\}\\right\)\\right\]=\\displaystyle=\\,𝟏s=T1\+t\+1α​𝔼​\[ΨT1\+t​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+t\)\]𝖳=𝟏s=T1\+t\+1​At𝖳,\\displaystyle\\frac\{\\mathrm\{\\bf 1\}\_\{s=T\_\{1\}\+t\+1\}\}\{\\alpha\}\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+t\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+t\}\\right\)\\right\]^\{\\sf T\}=\\mathrm\{\\bf 1\}\_\{s=T\_\{1\}\+t\+1\}A\_\{t\}^\{\\sf T\},where the last row follows from the definition ofGT1\+t\+1G\_\{T\_\{1\}\+t\+1\}\. Therefore, it suffices to consider the limiting empirical joint distribution of the rows of

\(𝒚,𝑽T1\+1α​F​\(𝑽T1−1,𝒚\)\+∑t=1T2\(𝑽T1\+t\+1\+FT1\+t​\(𝑽≤T1\+t,𝒚\)​At\)​Qt\),\\left\(\\text\{$y$\},\\ \\text\{$V$\}^\{T\_\{1\}\}\+\\frac\{1\}\{\\alpha\}F\(\\text\{$V$\}^\{T\_\{1\}\-1\},\\text\{$y$\}\)\+\\sum\_\{t=1\}^\{T\_\{2\}\}\\left\(\\text\{$V$\}^\{T\_\{1\}\+t\+1\}\+F\_\{T\_\{1\}\+t\}\\left\(\\text\{$V$\}^\{\\leq T\_\{1\}\+t\},\\text\{$y$\}\\right\)A\_\{t\}\\right\)Q\_\{t\}\\right\),which almost surely weakly converges to \(using continuous mapping theorem\)

Law⁡\(Y,Z¯T1\+1α​F​\(Z¯T1−1,Y\)\+∑t=1T2\(Z¯T1\+t\+1\+FT1\+t​\(Z¯≤T1\+t,Y\)​At\)​Qt\)\.\\operatorname\{Law\}\\left\(Y,\\ \\overline\{Z\}\_\{T\_\{1\}\}\+\\frac\{1\}\{\\alpha\}F\\left\(\\overline\{Z\}\_\{T\_\{1\}\-1\},Y\\right\)\+\\sum\_\{t=1\}^\{T\_\{2\}\}\\left\(\\overline\{Z\}\_\{T\_\{1\}\+t\+1\}\+F\_\{T\_\{1\}\+t\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+t\},Y\\right\)A\_\{t\}\\right\)Q\_\{t\}\\right\)\.This proves the first part of the theorem\.

Finally, note that for the IAMP stage, the only requirement for the functionΨT1\+t\\Psi\_\{T\_\{1\}\+t\}is that

𝔼​\[ΨT1\+t​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+t\)𝖳​ΨT1\+t​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+t\)\]=α​Im\.\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+t\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+t\}\\right\)^\{\\sf T\}\\Psi\_\{T\_\{1\}\+t\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+t\}\\right\)\\right\]=\\alpha I\_\{m\}\.Hence, for anyt≥1t\\geq 1andAt∈ℝm×mA\_\{t\}\\in\\mathbb\{R\}^\{m\\times m\}such thatAt𝖳​At⪯Im/αA\_\{t\}^\{\\sf T\}A\_\{t\}\\preceq I\_\{m\}/\\alpha, there exists a functionΨT1\+t\\Psi\_\{T\_\{1\}\+t\}that satisfies the condition of this theorem and that

𝔼​\[ΨT1\+t​\(V​R≤T1\+1\+Z≤T1\+1,\(Zt\)T1\+2≤t≤T1\+t\)\]=α​At\.\\mathbb\{E\}\\left\[\\Psi\_\{T\_\{1\}\+t\}\\left\(VR\_\{\\leq T\_\{1\}\+1\}\+Z\_\{\\leq T\_\{1\}\+1\},\(Z\_\{t\}\)\_\{T\_\{1\}\+2\\leq t\\leq T\_\{1\}\+t\}\\right\)\\right\]=\\alpha A\_\{t\}\.This proves that ifFFandFT1\+tF\_\{T\_\{1\}\+t\}are continuous, then

Law⁡\(Y,Z¯T1\+1α​F​\(Z¯T1−1,Y\)\+∑t=1T2\(Z¯T1\+t\+1\+FT1\+t​\(Z¯≤T1\+t,Y\)​At\)​Qt\)\\operatorname\{Law\}\\left\(Y,\\ \\overline\{Z\}\_\{T\_\{1\}\}\+\\frac\{1\}\{\\alpha\}F\\left\(\\overline\{Z\}\_\{T\_\{1\}\-1\},Y\\right\)\+\\sum\_\{t=1\}^\{T\_\{2\}\}\\left\(\\overline\{Z\}\_\{T\_\{1\}\+t\+1\}\+F\_\{T\_\{1\}\+t\}\\left\(\\overline\{Z\}\_\{\\leq T\_\{1\}\+t\},Y\\right\)A\_\{t\}\\right\)Q\_\{t\}\\right\)is\(α,m\)\(\\alpha,m\)\-feasible and can be achieved by our two\-stage AMP algorithm, where the only constraint on\{At\}1≤t≤T2\\\{A\_\{t\}\\\}\_\{1\\leq t\\leq T\_\{2\}\}is thatAt𝖳​At⪯Im/αA\_\{t\}^\{\\sf T\}A\_\{t\}\\preceq I\_\{m\}/\\alpha\. The second part of Theorem[7\.1](https://arxiv.org/html/2606.28573#S7.Thmthm1)follows immediately by combining this result and the fact that the set of\(α,m\)\(\\alpha,m\)\-feasible distributions achievable by our AMP algorithm is closed under weak limits, since we can approximate generalL2L^\{2\}\-integrable functions by continuous functions to arbitrary accuracy\. ∎

### E\.4Auxiliary lemmas

###### Lemma E\.1\.

LetA∈ℝ1×kA\\in\\mathbb\{R\}^\{1\\times k\}andX∈ℝ1×mX\\in\\mathbb\{R\}^\{1\\times m\}be two random vectors\. Then, we have

\{𝔼​\[A𝖳​X\]:𝔼​\[X𝖳​X\]⪯Im\}=\{R∈ℝk×m:R​R𝖳⪯𝔼​\[A𝖳​A\]\}\.\\left\\\{\\mathbb\{E\}\[A^\{\\sf T\}X\]:\\mathbb\{E\}\[X^\{\\sf T\}X\]\\preceq I\_\{m\}\\right\\\}=\\,\\left\\\{R\\in\\mathbb\{R\}^\{k\\times m\}:RR^\{\\sf T\}\\preceq\\mathbb\{E\}\[A^\{\\sf T\}A\]\\right\\\}\.

###### Proof\.

Define the following two subsets ofℝk×m\\mathbb\{R\}^\{k\\times m\}:

𝒴A=\{𝔼​\[A𝖳​X\]:𝔼​\[X𝖳​X\]⪯Im\},𝒵A=\{R∈ℝk×m:R​R𝖳⪯𝔼​\[A𝖳​A\]\}\.\\mathcal\{Y\}\_\{A\}=\\,\\left\\\{\\mathbb\{E\}\[A^\{\\sf T\}X\]:\\mathbb\{E\}\[X^\{\\sf T\}X\]\\preceq I\_\{m\}\\right\\\},\\quad\\mathcal\{Z\}\_\{A\}=\\,\\left\\\{R\\in\\mathbb\{R\}^\{k\\times m\}:RR^\{\\sf T\}\\preceq\\mathbb\{E\}\[A^\{\\sf T\}A\]\\right\\\}\.It is easy to see that both𝒴A\\mathcal\{Y\}\_\{A\}and𝒵A\\mathcal\{Z\}\_\{A\}are closed and convex\. By Hahn\-Banach theorem, it suffices to show that for anyB∈ℝk×mB\\in\\mathbb\{R\}^\{k\\times m\}:

supY∈𝒴ATr​\(B𝖳​Y\)=supZ∈𝒵ATr​\(B𝖳​Z\)\.\\sup\_\{Y\\in\\mathcal\{Y\}\_\{A\}\}\\mathrm\{Tr\}\(B^\{\\sf T\}Y\)=\\sup\_\{Z\\in\\mathcal\{Z\}\_\{A\}\}\\mathrm\{Tr\}\(B^\{\\sf T\}Z\)\.\(91\)To this end, we compute these two suprema respectively\. For the first supremum, we have

supY∈𝒴ATr​\(B𝖳​Y\)=\\displaystyle\\sup\_\{Y\\in\\mathcal\{Y\}\_\{A\}\}\\mathrm\{Tr\}\(B^\{\\sf T\}Y\)=\\,sup𝔼​\[X𝖳​X\]⪯ImTr​\(B𝖳​𝔼​\[A𝖳​X\]\)=sup𝔼​\[X𝖳​X\]⪯ImTr​\(𝔼​\[\(A​B\)𝖳​X\]\)\\displaystyle\\sup\_\{\\mathbb\{E\}\[X^\{\\sf T\}X\]\\preceq I\_\{m\}\}\\mathrm\{Tr\}\\left\(B^\{\\sf T\}\\mathbb\{E\}\[A^\{\\sf T\}X\]\\right\)=\\sup\_\{\\mathbb\{E\}\[X^\{\\sf T\}X\]\\preceq I\_\{m\}\}\\mathrm\{Tr\}\\left\(\\mathbb\{E\}\\left\[\(AB\)^\{\\sf T\}X\\right\]\\right\)=\\displaystyle=\\,sup𝔼​\[X𝖳​X\]⪯Im𝔼​\[Tr​\(\(A​B\)𝖳​X\)\]=sup𝔼​\[X𝖳​X\]⪯Im𝔼​\[X​\(A​B\)𝖳\]\\displaystyle\\sup\_\{\\mathbb\{E\}\[X^\{\\sf T\}X\]\\preceq I\_\{m\}\}\\mathbb\{E\}\\left\[\\mathrm\{Tr\}\\left\(\(AB\)^\{\\sf T\}X\\right\)\\right\]=\\sup\_\{\\mathbb\{E\}\[X^\{\\sf T\}X\]\\preceq I\_\{m\}\}\\mathbb\{E\}\\left\[X\(AB\)^\{\\sf T\}\\right\]=\\displaystyle=\\,supXminΛ⪰0⁡\{𝔼​\[X​\(A​B\)𝖳\]−Tr​\(Λ​\(𝔼​\[X𝖳​X\]−Im\)\)\}\\displaystyle\\sup\_\{X\}\\min\_\{\\Lambda\\succeq 0\}\\left\\\{\\mathbb\{E\}\\left\[X\(AB\)^\{\\sf T\}\\right\]\-\\mathrm\{Tr\}\\left\(\\Lambda\\left\(\\mathbb\{E\}\[X^\{\\sf T\}X\]\-I\_\{m\}\\right\)\\right\)\\right\\\}=\\displaystyle=\\,minΛ⪰0​supX\{𝔼​\[X​\(A​B\)𝖳\]−𝔼​\[X​Λ​X𝖳\]\+Tr​\(Λ\)\}\\displaystyle\\min\_\{\\Lambda\\succeq 0\}\\sup\_\{X\}\\left\\\{\\mathbb\{E\}\\left\[X\(AB\)^\{\\sf T\}\\right\]\-\\mathbb\{E\}\\left\[X\\Lambda X^\{\\sf T\}\\right\]\+\\mathrm\{Tr\}\(\\Lambda\)\\right\\\}=\\displaystyle=\\,minΛ⪰0⁡\{14​Tr​\(Λ−1​𝔼​\[\(A​B\)𝖳​A​B\]\)\+Tr​\(Λ\)\}\\displaystyle\\min\_\{\\Lambda\\succeq 0\}\\left\\\{\\frac\{1\}\{4\}\\mathrm\{Tr\}\\left\(\\Lambda^\{\-1\}\\mathbb\{E\}\\left\[\(AB\)^\{\\sf T\}AB\\right\]\\right\)\+\\mathrm\{Tr\}\(\\Lambda\)\\right\\\}=\\displaystyle=\\,minΛ⪰0⁡\{14​Tr​\(Λ​𝔼​\[\(A​B\)𝖳​A​B\]\)\+Tr​\(Λ−1\)\}\\displaystyle\\min\_\{\\Lambda\\succeq 0\}\\left\\\{\\frac\{1\}\{4\}\\mathrm\{Tr\}\\left\(\\Lambda\\mathbb\{E\}\\left\[\(AB\)^\{\\sf T\}AB\\right\]\\right\)\+\\mathrm\{Tr\}\\left\(\\Lambda^\{\-1\}\\right\)\\right\\\}=\\displaystyle=\\,Tr​\(𝔼​\[\(A​B\)𝖳​A​B\]1/2\)=Tr​\(B𝖳​𝔼​\[A𝖳​A\]​B\)1/2\.\\displaystyle\\mathrm\{Tr\}\\left\(\\mathbb\{E\}\\left\[\(AB\)^\{\\sf T\}AB\\right\]^\{1/2\}\\right\)=\\mathrm\{Tr\}\\left\(B^\{\\sf T\}\\mathbb\{E\}\[A^\{\\sf T\}A\]B\\right\)^\{1/2\}\.For the second one, the caculation follows similarly:

supZ∈𝒵ATr​\(B𝖳​Z\)=\\displaystyle\\sup\_\{Z\\in\\mathcal\{Z\}\_\{A\}\}\\mathrm\{Tr\}\(B^\{\\sf T\}Z\)=\\,supR​R𝖳⪯𝔼​\[A𝖳​A\]Tr​\(B𝖳​R\)=supRminΛ⪰0⁡\{Tr​\(B𝖳​R\)−Tr​\(Λ​\(R​R𝖳−𝔼​\[A𝖳​A\]\)\)\}\\displaystyle\\sup\_\{RR^\{\\sf T\}\\preceq\\mathbb\{E\}\[A^\{\\sf T\}A\]\}\\mathrm\{Tr\}\(B^\{\\sf T\}R\)=\\sup\_\{R\}\\min\_\{\\Lambda\\succeq 0\}\\left\\\{\\mathrm\{Tr\}\(B^\{\\sf T\}R\)\-\\mathrm\{Tr\}\\left\(\\Lambda\\left\(RR^\{\\sf T\}\-\\mathbb\{E\}\[A^\{\\sf T\}A\]\\right\)\\right\)\\right\\\}=\\displaystyle=\\,minΛ⪰0​supR\{Tr​\(B𝖳​R\)−Tr​\(Λ​\(R​R𝖳−𝔼​\[A𝖳​A\]\)\)\}\\displaystyle\\min\_\{\\Lambda\\succeq 0\}\\sup\_\{R\}\\left\\\{\\mathrm\{Tr\}\(B^\{\\sf T\}R\)\-\\mathrm\{Tr\}\\left\(\\Lambda\\left\(RR^\{\\sf T\}\-\\mathbb\{E\}\[A^\{\\sf T\}A\]\\right\)\\right\)\\right\\\}=\\displaystyle=\\,minΛ⪰0​supR\{Tr​\(B𝖳​R\)−Tr​\(Λ​R​R𝖳\)\+Tr​\(Λ​𝔼​\[A𝖳​A\]\)\}\\displaystyle\\min\_\{\\Lambda\\succeq 0\}\\sup\_\{R\}\\left\\\{\\mathrm\{Tr\}\(B^\{\\sf T\}R\)\-\\mathrm\{Tr\}\(\\Lambda RR^\{\\sf T\}\)\+\\mathrm\{Tr\}\\left\(\\Lambda\\mathbb\{E\}\[A^\{\\sf T\}A\]\\right\)\\right\\\}=\\displaystyle=\\,minΛ⪰0⁡\{14​Tr​\(B𝖳​Λ−1​B\)\+Tr​\(Λ​𝔼​\[A𝖳​A\]\)\}\\displaystyle\\min\_\{\\Lambda\\succeq 0\}\\left\\\{\\frac\{1\}\{4\}\\mathrm\{Tr\}\\left\(B^\{\\sf T\}\\Lambda^\{\-1\}B\\right\)\+\\mathrm\{Tr\}\\left\(\\Lambda\\mathbb\{E\}\[A^\{\\sf T\}A\]\\right\)\\right\\\}=\\displaystyle=\\,minΓ⪰0⁡\{14​Tr​\(Γ​𝔼​\[A𝖳​A\]1/2​B​B𝖳​𝔼​\[A𝖳​A\]1/2\)\+Tr​\(Γ−1\)\}\\displaystyle\\min\_\{\\Gamma\\succeq 0\}\\left\\\{\\frac\{1\}\{4\}\\mathrm\{Tr\}\\left\(\\Gamma\\mathbb\{E\}\[A^\{\\sf T\}A\]^\{1/2\}BB^\{\\sf T\}\\mathbb\{E\}\[A^\{\\sf T\}A\]^\{1/2\}\\right\)\+\\mathrm\{Tr\}\\left\(\\Gamma^\{\-1\}\\right\)\\right\\\}=\\displaystyle=\\,Tr​\(𝔼​\[A𝖳​A\]1/2​B​B𝖳​𝔼​\[A𝖳​A\]1/2\)1/2=Tr​\(B𝖳​𝔼​\[A𝖳​A\]​B\)1/2\.\\displaystyle\\mathrm\{Tr\}\\left\(\\mathbb\{E\}\[A^\{\\sf T\}A\]^\{1/2\}BB^\{\\sf T\}\\mathbb\{E\}\[A^\{\\sf T\}A\]^\{1/2\}\\right\)^\{1/2\}=\\mathrm\{Tr\}\\left\(B^\{\\sf T\}\\mathbb\{E\}\[A^\{\\sf T\}A\]B\\right\)^\{1/2\}\.This proves[Eq\.91](https://arxiv.org/html/2606.28573#A5.E91), and consequently𝒴A=𝒵A\\mathcal\{Y\}\_\{A\}=\\mathcal\{Z\}\_\{A\}\. ∎

## Appendix FAppendix for[Section8](https://arxiv.org/html/2606.28573#S8)

### F\.1Proof of[8\.3](https://arxiv.org/html/2606.28573#S8.Thmthm3)

###### Proof\.

We begin with defining a family of SDEs related to\(Xt\)t∈\[r2,1\]\(X\_\{t\}\)\_\{t\\in\[r^\{2\},1\]\}: For any fixed\(s,y,x\)∈\[0,1\]×ℝ×ℝ\(s,y,x\)\\in\[0,1\]\\times\\mathbb\{R\}\\times\\mathbb\{R\}, consider the SDE

d​Xty,x=μ​\(t\)​∂xfy,μ​\(t,Xty,x\)​d​t\+d​Bt,t∈\[s,1\]\\textup\{d\}X\_\{t\}^\{y,x\}=\\mu\(t\)\\partial\_\{x\}f\_\{y,\\mu\}\(t,X\_\{t\}^\{y,x\}\)\\textup\{d\}t\+\\textup\{d\}B\_\{t\},\\,t\\in\[s,1\]with initial conditionXsy,x=xX\_\{s\}^\{y,x\}=x\. Similar to\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\], we can show that the above SDE has a unique solution\{Xty,x\}t∈\[s,1\]\\\{X\_\{t\}^\{y,x\}\\\}\_\{t\\in\[s,1\]\}, which follows from the boundedness of∂xfy,μ\\partial\_\{x\}f\_\{y,\\mu\}and\[[Che05](https://arxiv.org/html/2606.28573#bib.bibx9), Proposition 1\.10\]\. As a consequence, we know thatXt=XtY,r​GX\_\{t\}=X\_\{t\}^\{Y,rG\}withs=r2s=r^\{2\}exists and is unique\.

Further, define for anyt∈\[s,1\]t\\in\[s,1\]:

Mty,x=1γ​\(t\)​∂xfy,μ​\(t,Xty,x\)\+Xty,x\.M\_\{t\}^\{y,x\}=\\,\\frac\{1\}\{\\gamma\(t\)\}\\partial\_\{x\}f\_\{y,\\mu\}\(t,X\_\{t\}^\{y,x\}\)\+X\_\{t\}^\{y,x\}\.Then, applying\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Lemma 7\.3\]and the martingale representation theorem, we know that there exists\(ϕty,x\)t∈\[s,1\]∈D​\[s,1\]\(\\phi\_\{t\}^\{y,x\}\)\_\{t\\in\[s,1\]\}\\in D\[s,1\]such that

Mty,x=Msy,x\+∫st\(1\+ϕuy,x\)​d​Bu,∀t∈\[s,1\]\.M\_\{t\}^\{y,x\}=\\,M\_\{s\}^\{y,x\}\+\\int\_\{s\}^\{t\}\\left\(1\+\\phi\_\{u\}^\{y,x\}\\right\)\\textup\{d\}B\_\{u\},\\,\\forall t\\in\[s,1\]\.This guarantees the existence of\(ϕt\)t∈\[r2,1\]\(\\phi\_\{t\}\)\_\{t\\in\[r^\{2\},1\]\}by lettings=r2s=r^\{2\}andϕt=ϕtY,r​G\\phi\_\{t\}=\\phi\_\{t\}^\{Y,rG\}\.

##### Proof of \(i\)\.

Sinceμ≡0\\mu\\equiv 0on\[r2,q\]\[r^\{2\},q\], we know that

Xq=Xr2\+Bq−Br2,X\_\{q\}=\\,X\_\{r^\{2\}\}\+B\_\{q\}\-B\_\{r^\{2\}\},which has the same distribution asZr,qZ\_\{r,q\}given\(Y,G\)\(Y,G\)\. Hence,\(Y,G,Xq\)=d\(Y,G,Zr,q\)\(Y,G,X\_\{q\}\)\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}\(Y,G,Z\_\{r,q\}\)\. To prove the second part, note that by[8\.2](https://arxiv.org/html/2606.28573#S8.Thmthm2)and our choice ofFF:

fy,μ​\(q,x\)\+12​α​∫q1γ​\(s\)​d​s=\\displaystyle f\_\{y,\\mu\}\(q,x\)\+\\frac\{1\}\{2\\alpha\}\\int\_\{q\}^\{1\}\\gamma\(s\)\\textup\{d\}s=\\,supu∈ℝ\{Vγ​\(q,y,x\+u\)−γ​\(q\)2​u2\}\\displaystyle\\sup\_\{u\\in\\mathbb\{R\}\}\\left\\\{V\_\{\\gamma\}\(q,y,x\+u\)\-\\frac\{\\gamma\(q\)\}\{2\}u^\{2\}\\right\\\}=\\displaystyle=\\,Vγ​\(q,y,x\+1α​F​\(x,y\)\)−γ​\(q\)2​α2​F​\(x,y\)2\.\\displaystyle V\_\{\\gamma\}\\left\(q,y,x\+\\frac\{1\}\{\\alpha\}F\(x,y\)\\right\)\-\\frac\{\\gamma\(q\)\}\{2\\alpha^\{2\}\}F\(x,y\)^\{2\}\.Further, similar to the verification argument in\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\], we know that the supremum in the definition ofVγ​\(q,y,x\+F​\(x,y\)/α\)V\_\{\\gamma\}\(q,y,x\+F\(x,y\)/\\alpha\)is achieved at\(ϕty,x\)t∈\[q,1\]\(\\phi\_\{t\}^\{y,x\}\)\_\{t\\in\[q,1\]\}, i\.e\.,

Vγ​\(q,y,x\+1α​F​\(x,y\)\)\\displaystyle V\_\{\\gamma\}\\left\(q,y,x\+\\frac\{1\}\{\\alpha\}F\(x,y\)\\right\)=\\displaystyle=\\,𝔼​\[h​\(y,x\+1α​F​\(x,y\)\+∫q1\(1\+ϕty,x\)​d​Bt\)−12​∫q1γ​\(t\)​\(\(ϕty,x\)2−1α\)​d​t\]\.\\displaystyle\\mathbb\{E\}\\left\[h\\left\(y,x\+\\frac\{1\}\{\\alpha\}F\(x,y\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}^\{y,x\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\(\\phi\_\{t\}^\{y,x\}\)^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t\\right\]\.From the proof of[6\.1](https://arxiv.org/html/2606.28573#S6.Thmthm1), we get that

𝖥​\(μ,c,r\)=𝔼​\[fY,μ​\(q,Zr,q\)\]\+12​α​∫r21γ​\(t\)​d​t,\\mathsf\{F\}\(\\mu,c,r\)=\\,\\mathbb\{E\}\\left\[f\_\{Y,\\mu\}\\left\(q,Z\_\{r,q\}\\right\)\\right\]\+\\frac\{1\}\{2\\alpha\}\\int\_\{r^\{2\}\}^\{1\}\\gamma\(t\)\\textup\{d\}t,which leads to

𝖥​\(μ,c,r\)=\\displaystyle\\mathsf\{F\}\(\\mu,c,r\)=\\,𝔼​\[Vγ​\(q,Y,Xq\+1α​F​\(Xq,Y\)\)−γ​\(q\)2​α​\(F​\(Xq,Y\)2α−\(q−r2\)\)\]\\displaystyle\\mathbb\{E\}\\left\[V\_\{\\gamma\}\\left\(q,Y,X\_\{q\}\+\\frac\{1\}\{\\alpha\}F\(X\_\{q\},Y\)\\right\)\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(X\_\{q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\right\]=\\displaystyle=\\,𝔼\[h\(Y,Xq\+1αF\(Xq,Y\)\+∫q1\(1\+ϕtY,Xq\)dBt\)−12∫q1γ\(t\)\(\(ϕtY,Xq\)2−1α\)dt\\displaystyle\\mathbb\{E\}\\bigg\[h\\left\(Y,X\_\{q\}\+\\frac\{1\}\{\\alpha\}F\(X\_\{q\},Y\)\+\\int\_\{q\}^\{1\}\\left\(1\+\\phi\_\{t\}^\{Y,X\_\{q\}\}\\right\)\\textup\{d\}B\_\{t\}\\right\)\-\\frac\{1\}\{2\}\\int\_\{q\}^\{1\}\\gamma\(t\)\\left\(\\left\(\\phi\_\{t\}^\{Y,X\_\{q\}\}\\right\)^\{2\}\-\\frac\{1\}\{\\alpha\}\\right\)\\textup\{d\}t−γ​\(q\)2​α\(F​\(Xq,Y\)2α−\(q−r2\)\)\]\.\\displaystyle\\quad\-\\frac\{\\gamma\(q\)\}\{2\\alpha\}\\left\(\\frac\{F\(X\_\{q\},Y\)^\{2\}\}\{\\alpha\}\-\(q\-r^\{2\}\)\\right\)\\bigg\]\.The proof is then completed by noting thatϕtY,Xq=ϕt\\phi\_\{t\}^\{Y,X\_\{q\}\}=\\phi\_\{t\}fort∈\[q,1\]t\\in\[q,1\]\.

##### Proof of \(ii\)\.

For any fixed pair\(Y,G\)\(Y,G\), the conclusion follows similarly as Proposition 5\.4 \(ii\) and Proposition 8\.2 of\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\]\. Applying the law of total expectation then completes the proof\.

##### Proof of \(iii\)\.

For the proof of this part, we need the following proposition which computes the first\-order variations offy,μ​\(t,x\)f\_\{y,\\mu\}\(t,x\)with respect toμ\\muandcc\(note thatfy,μ​\(t,x\)f\_\{y,\\mu\}\(t,x\)depends onccthrough the terminal condition\), whose proof is established by Proposition 5\.4 \(iii\), Proposition 8\.3 and Lemma 8\.4 in\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35)\]333Although\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Lemma 8\.4\]is only stated for the setting1/c≤supz∈ℝ∂z2h​\(y,z\)1/c\\leq\\sup\_\{z\\in\\mathbb\{R\}\}\\partial\_\{z\}^\{2\}h\(y,z\), its proof actually works for allc\>0c\>0\.\.

###### Proposition F\.1\.

Fix\(t,y,x\)∈\[0,1\]×ℝ×ℝ\(t,y,x\)\\in\[0,1\]\\times\\mathbb\{R\}\\times\\mathbb\{R\}\. Assume thatδ:\[t,1\]→ℝ\\delta:\[t,1\]\\to\\mathbb\{R\}is inL1​\[t,1\]L^\{1\}\[t,1\]andL∞​\[t,s\]L^\{\\infty\}\[t,s\]for anys∈\[t,1\)s\\in\[t,1\)\. Then, we have

dd​u​fy,μ\+u​δ​\(t,x\)\|u=0=12​∫t1δ​\(s\)​𝔼​\[\(∂xfy,μ​\(s,Xsy,x\)\)2\]​d​s,dd​c​fy,μ​\(t,x\)=𝔼​\[gc​\(y,M1y,x\)\]\+12​𝔼​\[\(∂xfy,μ​\(1,X1y,x\)\)2\]\.\\begin\{split\}\\frac\{\\textup\{d\}\}\{\\textup\{d\}u\}f\_\{y,\\mu\+u\\delta\}\(t,x\)\\bigg\|\_\{u=0\}=\\,&\\frac\{1\}\{2\}\\int\_\{t\}^\{1\}\\delta\(s\)\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{y,\\mu\}\(s,X\_\{s\}^\{y,x\}\)\\right\)^\{2\}\\right\]\\textup\{d\}s,\\\\ \\frac\{\\textup\{d\}\}\{\\textup\{d\}c\}f\_\{y,\\mu\}\(t,x\)=\\,&\\mathbb\{E\}\\left\[g\_\{c\}\\left\(y,M\_\{1\}^\{y,x\}\\right\)\\right\]\+\\frac\{1\}\{2\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{y,\\mu\}\\left\(1,X\_\{1\}^\{y,x\}\\right\)\\right\)^\{2\}\\right\]\.\\end\{split\}

Based on the above proposition, we are now able to compute the first\-order variations of𝖥\\mathsf\{F\}with respect toμ\\muandcc\. Note that, for any fixed\(Y,G\)\(Y,G\),[F\.1](https://arxiv.org/html/2606.28573#A6.Thmlem1)implies that

dd​u​fY,μ\+u​δ​\(r2,r​G\)\|u=0=12​∫r21δ​\(s\)​𝔼​\[\(∂xfY,μ​\(s,XsY,r​G\)\)2\]​d​s,dd​c​fY,μ​\(r2,r​G\)=𝔼​\[gc​\(Y,M1Y,r​G\)\]\+12​𝔼​\[\(∂xfY,μ​\(1,X1Y,r​G\)\)2\]\.\\begin\{split\}\\frac\{\\textup\{d\}\}\{\\textup\{d\}u\}f\_\{Y,\\mu\+u\\delta\}\(r^\{2\},rG\)\\bigg\|\_\{u=0\}=\\,&\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\delta\(s\)\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\(s,X\_\{s\}^\{Y,rG\}\)\\right\)^\{2\}\\right\]\\textup\{d\}s,\\\\ \\frac\{\\textup\{d\}\}\{\\textup\{d\}c\}f\_\{Y,\\mu\}\(r^\{2\},rG\)=\\,&\\mathbb\{E\}\\left\[g\_\{c\}\\left\(Y,M\_\{1\}^\{Y,rG\}\\right\)\\right\]\+\\frac\{1\}\{2\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\\left\(1,X\_\{1\}^\{Y,rG\}\\right\)\\right\)^\{2\}\\right\]\.\\end\{split\}Using[8\.1](https://arxiv.org/html/2606.28573#S8.Thmthm1)and[F\.2](https://arxiv.org/html/2606.28573#A6.Thmlem2), we obtain that

\|dd​u​fY,μ\+u​δ​\(r2,r​G\)\|u=0\|≤\\displaystyle\\left\|\\frac\{\\textup\{d\}\}\{\\textup\{d\}u\}f\_\{Y,\\mu\+u\\delta\}\(r^\{2\},rG\)\\bigg\|\_\{u=0\}\\right\|\\leq\\,12​‖δ‖L1​\[r2,1\]​‖∂xh​\(Y,⋅\)‖∞2,\\displaystyle\\frac\{1\}\{2\}\\left\\\|\{\\delta\}\\right\\\|\_\{L^\{1\}\[r^\{2\},1\]\}\\left\\\|\{\\partial\_\{x\}h\(Y,\\cdot\)\}\\right\\\|\_\{\\infty\}^\{2\},\|dd​c​fY,μ​\(r2,r​G\)\|≤\\displaystyle\\left\|\\frac\{\\textup\{d\}\}\{\\textup\{d\}c\}f\_\{Y,\\mu\}\(r^\{2\},rG\)\\right\|\\leq\\,52​‖∂xh​\(Y,⋅\)‖∞2\.\\displaystyle\\frac\{5\}\{2\}\\left\\\|\{\\partial\_\{x\}h\(Y,\\cdot\)\}\\right\\\|\_\{\\infty\}^\{2\}\.Of course, the above estimates hold within a neighborhood of\(μ,c\)\(\\mu,c\)\. Since𝔼​\[‖∂xh​\(Y,⋅\)‖∞2\]<∞\\mathbb\{E\}\[\\left\\\|\{\\partial\_\{x\}h\(Y,\\cdot\)\}\\right\\\|\_\{\\infty\}^\{2\}\]<\\inftyby our assumption, we can apply the dominated convergence theorem to deduce that

dd​u​𝔼​\[fY,μ\+u​δ​\(r2,r​G\)\]\|u=0=12​∫r21δ​\(s\)​𝔼​\[\(∂xfY,μ​\(s,Xs\)\)2\]​d​s,dd​c​𝔼​\[fY,μ​\(r2,r​G\)\]=𝔼​\[gc​\(Y,M1\)\]\+12​𝔼​\[\(∂xfY,μ​\(1,X1\)\)2\],\\begin\{split\}\\frac\{\\textup\{d\}\}\{\\textup\{d\}u\}\\mathbb\{E\}\\left\[f\_\{Y,\\mu\+u\\delta\}\(r^\{2\},rG\)\\right\]\\bigg\|\_\{u=0\}=\\,&\\frac\{1\}\{2\}\\int\_\{r^\{2\}\}^\{1\}\\delta\(s\)\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\(s,X\_\{s\}\)\\right\)^\{2\}\\right\]\\textup\{d\}s,\\\\ \\frac\{\\textup\{d\}\}\{\\textup\{d\}c\}\\mathbb\{E\}\\left\[f\_\{Y,\\mu\}\(r^\{2\},rG\)\\right\]=\\,&\\mathbb\{E\}\\left\[g\_\{c\}\\left\(Y,M\_\{1\}\\right\)\\right\]\+\\frac\{1\}\{2\}\\mathbb\{E\}\\left\[\\left\(\\partial\_\{x\}f\_\{Y,\\mu\}\\left\(1,X\_\{1\}\\right\)\\right\)^\{2\}\\right\],\\end\{split\}which concludes the calculation of the first two variations, since the variations of the entropic term follow from a straightforward calculation\.

Finally, we compute the first\-order variation of𝖥\\mathsf\{F\}with respect torr\. It suffices to show that

dd​r​𝔼​\[fY,μ​\(r2,r​G\)\]=γ​\(r2\)α​1q−r2​𝔼​\[\(q​G−r​Zr,q\)​F​\(Zr,q,φ​\(G,ε\)\)\]\.\\frac\{\\textup\{d\}\}\{\\textup\{d\}r\}\\mathbb\{E\}\\left\[f\_\{Y,\\mu\}\(r^\{2\},rG\)\\right\]=\\,\\frac\{\\gamma\(r^\{2\}\)\}\{\\alpha\}\\frac\{1\}\{q\-r^\{2\}\}\\mathbb\{E\}\\left\[\\left\(qG\-rZ\_\{r,q\}\\right\)F\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\]\.To this end, note that sinceμ≡0\\mu\\equiv 0on\[r2,q\]\[r^\{2\},q\], we have

fY,μ​\(r2,r​G\)=fY,μ​\(q,Xq\)=dfY,μ​\(q,Zr,q\)\.f\_\{Y,\\mu\}\(r^\{2\},rG\)=\\,f\_\{Y,\\mu\}\(q,X\_\{q\}\)\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}f\_\{Y,\\mu\}\(q,Z\_\{r,q\}\)\.For any fixed\(Y,G,Z\)\(Y,G,Z\),

dd​r​fY,μ​\(q,Zr,q\)=dd​r​fY,μ​\(q,r​G\+q−r2​Z\)=∂xfY,μ​\(q,Zr,q\)​\(G−rq−r2​Z\),\\displaystyle\\frac\{\\textup\{d\}\}\{\\textup\{d\}r\}f\_\{Y,\\mu\}\(q,Z\_\{r,q\}\)=\\,\\frac\{\\textup\{d\}\}\{\\textup\{d\}r\}f\_\{Y,\\mu\}\\left\(q,rG\+\\sqrt\{q\-r^\{2\}\}Z\\right\)=\\partial\_\{x\}f\_\{Y,\\mu\}\(q,Z\_\{r,q\}\)\\left\(G\-\\frac\{r\}\{\\sqrt\{q\-r^\{2\}\}\}Z\\right\),which is uniformly bounded by‖∂xh​\(Y,⋅\)‖∞​\|G−r​Z/q−r2\|∈L1\\left\\\|\{\\partial\_\{x\}h\(Y,\\cdot\)\}\\right\\\|\_\{\\infty\}\|G\-rZ/\\sqrt\{q\-r^\{2\}\}\|\\in L^\{1\}, since‖∂xh​\(Y,⋅\)‖∞∈L2\\left\\\|\{\\partial\_\{x\}h\(Y,\\cdot\)\}\\right\\\|\_\{\\infty\}\\in L^\{2\}by our assumption\. Applying again dominated convergence theorem, we obtain that

dd​r​𝔼​\[fY,μ​\(r2,r​G\)\]=\\displaystyle\\frac\{\\textup\{d\}\}\{\\textup\{d\}r\}\\mathbb\{E\}\\left\[f\_\{Y,\\mu\}\(r^\{2\},rG\)\\right\]=\\,dd​r​𝔼​\[fY,μ​\(q,Zr,q\)\]=𝔼​\[∂xfY,μ​\(q,Zr,q\)​\(G−rq−r2​Z\)\]\\displaystyle\\frac\{\\textup\{d\}\}\{\\textup\{d\}r\}\\mathbb\{E\}\\left\[f\_\{Y,\\mu\}\(q,Z\_\{r,q\}\)\\right\]=\\mathbb\{E\}\\left\[\\partial\_\{x\}f\_\{Y,\\mu\}\(q,Z\_\{r,q\}\)\\left\(G\-\\frac\{r\}\{\\sqrt\{q\-r^\{2\}\}\}Z\\right\)\\right\]=\\displaystyle=\\,γ​\(r2\)α​1q−r2​𝔼​\[\(q​G−r​Zr,q\)​F​\(Zr,q,φ​\(G,ε\)\)\],\\displaystyle\\frac\{\\gamma\(r^\{2\}\)\}\{\\alpha\}\\frac\{1\}\{q\-r^\{2\}\}\\mathbb\{E\}\\left\[\\left\(qG\-rZ\_\{r,q\}\\right\)F\\left\(Z\_\{r,q\},\\varphi\(G,\\varepsilon\)\\right\)\\right\],where the last line follows from the definition ofFF\. This completes the proof\. ∎

### F\.2Auxiliary lemmas

###### Lemma F\.2\.

For any fixedy∈ℝy\\in\\mathbb\{R\}, it holds

‖gc​\(y,⋅\)‖∞≤2​‖∂xh​\(y,⋅\)‖∞2\.\\left\\\|\{g\_\{c\}\(y,\\cdot\)\}\\right\\\|\_\{\\infty\}\\leq\\,2\\left\\\|\{\\partial\_\{x\}h\(y,\\cdot\)\}\\right\\\|\_\{\\infty\}^\{2\}\.

###### Proof\.

For anyc\>0c\>0,\[[MZ24](https://arxiv.org/html/2606.28573#bib.bibx35), Lemma C\.2\]implies that

hc​\(y,x\)=conc⁡\(h​\(y,x\)−x22​c\)\+x22​c=supU∈L2​\(Ω\),𝔼​\[U\]=0𝔼​\[h​\(y,x\+U\)−U22​c\]\.h\_\{c\}\(y,x\)=\\,\\operatorname\{conc\}\\left\(h\(y,x\)\-\\frac\{x^\{2\}\}\{2c\}\\right\)\+\\frac\{x^\{2\}\}\{2c\}=\\sup\_\{U\\in L^\{2\}\(\\Omega\),\\,\\mathbb\{E\}\[U\]=0\}\\mathbb\{E\}\\left\[h\(y,x\+U\)\-\\frac\{U^\{2\}\}\{2c\}\\right\]\.DenotingL=‖∂xh​\(y,⋅\)‖∞L=\\\|\\partial\_\{x\}h\(y,\\cdot\)\\\|\_\{\\infty\}, we first show that the above supremum must be achieved at someUUsuch that𝔼​\[U2\]≤4​c2​L2\\mathbb\{E\}\[U^\{2\}\]\\leq 4c^\{2\}L^\{2\}\. Indeed, suchUUmust satisfy

h​\(y,x\)≤𝔼​\[h​\(y,x\+U\)−U22​c\]≤h​\(y,x\)\+L​𝔼​\[\|U\|\]−12​c​𝔼​\[U2\]≤h​\(y,x\)\+L​𝔼​\[U2\]1/2−12​c​𝔼​\[U2\],\\displaystyle h\(y,x\)\\leq\\,\\mathbb\{E\}\\left\[h\(y,x\+U\)\-\\frac\{U^\{2\}\}\{2c\}\\right\]\\leq h\(y,x\)\+L\\mathbb\{E\}\[\|U\|\]\-\\frac\{1\}\{2c\}\\mathbb\{E\}\[U^\{2\}\]\\leq h\(y,x\)\+L\\mathbb\{E\}\[U^\{2\}\]^\{1/2\}\-\\frac\{1\}\{2c\}\\mathbb\{E\}\[U^\{2\}\],which implies that𝔼​\[U2\]≤4​c2​L2\\mathbb\{E\}\[U^\{2\}\]\\leq 4c^\{2\}L^\{2\}\. As a consequence,

hc​\(y,x\)=sup𝔼​\[U2\]≤4​c2​L2,𝔼​\[U\]=0𝔼​\[h​\(y,x\+U\)−U22​c\]\.h\_\{c\}\(y,x\)=\\,\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 4c^\{2\}L^\{2\},\\,\\mathbb\{E\}\[U\]=0\}\\mathbb\{E\}\\left\[h\(y,x\+U\)\-\\frac\{U^\{2\}\}\{2c\}\\right\]\.For any0<c1≤c20<c\_\{1\}\\leq c\_\{2\}, we have

\|hc1​\(y,x\)−hc2​\(y,x\)\|\\displaystyle\\left\|h\_\{c\_\{1\}\}\(y,x\)\-h\_\{c\_\{2\}\}\(y,x\)\\right\|=\\displaystyle=\\,\|sup𝔼​\[U2\]≤4​c12​L2,𝔼​\[U\]=0𝔼​\[h​\(y,x\+U\)−U22​c1\]−sup𝔼​\[U2\]≤4​c22​L2,𝔼​\[U\]=0𝔼​\[h​\(y,x\+U\)−U22​c2\]\|\\displaystyle\\left\|\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 4c\_\{1\}^\{2\}L^\{2\},\\,\\mathbb\{E\}\[U\]=0\}\\mathbb\{E\}\\left\[h\(y,x\+U\)\-\\frac\{U^\{2\}\}\{2c\_\{1\}\}\\right\]\-\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 4c\_\{2\}^\{2\}L^\{2\},\\,\\mathbb\{E\}\[U\]=0\}\\mathbb\{E\}\\left\[h\(y,x\+U\)\-\\frac\{U^\{2\}\}\{2c\_\{2\}\}\\right\]\\right\|≤\\displaystyle\\leq\\,sup𝔼​\[U2\]≤4​c22​L2,𝔼​\[U\]=0\|𝔼​\[U2\]2​c2−𝔼​\[U2\]2​c1\|≤2​c2​\(c2−c1\)c1​L2,\\displaystyle\\sup\_\{\\mathbb\{E\}\[U^\{2\}\]\\leq 4c\_\{2\}^\{2\}L^\{2\},\\,\\mathbb\{E\}\[U\]=0\}\\left\|\\frac\{\\mathbb\{E\}\[U^\{2\}\]\}\{2c\_\{2\}\}\-\\frac\{\\mathbb\{E\}\[U^\{2\}\]\}\{2c\_\{1\}\}\\right\|\\leq\\frac\{2c\_\{2\}\(c\_\{2\}\-c\_\{1\}\)\}\{c\_\{1\}\}L^\{2\},leading to

\|gc​\(y,x\)\|=\|∂hc​\(y,x\)∂c\|≤2​L2,∀x∈ℝ\.\\left\|g\_\{c\}\(y,x\)\\right\|=\\,\\left\|\\frac\{\\partial h\_\{c\}\(y,x\)\}\{\\partial c\}\\right\|\\leq 2L^\{2\},\\,\\forall x\\in\\mathbb\{R\}\.This completes the proof\. ∎