Connecting Score Matching, Maximum Likelihood, and Expectation-Maximization in Mixed Linear Regression

arXiv cs.LG 09/10/26, 04:00 AM Papers
Summary
This paper connects score matching, maximum likelihood, and expectation-maximization in mixed linear regression, providing theoretical guarantees and analyzing gradient behavior under diffusion noise levels.
arXiv:2609.05688v1 Announce Type: new Abstract: We study variance-preserving diffusion of the response in mixed linear regression (MLR) with unknown mixing weights. Our analysis separates the statistical guarantees of score matching from the loss geometry and optimization signal at a fixed diffusion noise level. The KL divergence links the denoising score matching objective integrated over the diffusion path with the likelihood and a terminal discrepancy. Under mild regularity conditions and terminal schedule, the resulting estimator converges up to the ground truth parameters of MLR, and its scaled error converges to the Gaussian limit of the maximum-likelihood estimator. At a fixed scale of the diffusion noise level, we derive a decomposition linking the score matching loss to cross-entropy and Expectation-Maximization (EM) operators. This decomposition yields an EM-related low-noise gradient expansion with additional correction terms of latent variance. In the high-noise limit, we further characterize gradient descent on this limiting loss under isotropic covariance. Along fixed high signal-to-noise ratio rays, the score matching imbalance gradient and the latent-variance term tend to zero pointwise. Numerical experiments illustrate our theoretical findings and statistical guarantees.
Original Article
View Cached Full Text
Cached at: 09/10/26, 08:23 AM
# Connecting Score Matching, Maximum Likelihood, andExpectation-Maximization in Mixed Linear Regression
Source: [https://arxiv.org/html/2609.05688](https://arxiv.org/html/2609.05688)
Zhankun Luoluo333@purdue\.eduAffiliation:Abolfazl Hashemiabolfazl@purdue\.eduAffiliation:School of Electrical and Computer EngineeringAffiliation:Purdue University, West Lafayette, IN, USA

###### Abstract

We study variance\-preserving diffusion of the response in mixed linear regression \(MLR\) with unknown mixing weights\. Our analysis separates the statistical guarantees of score matching from the loss geometry and optimization signal at a fixed diffusion noise level\. The KL divergence links the denoising score matching objective integrated over the diffusion path with the likelihood and a terminal discrepancy\. Under mild regularity conditions and terminal schedule, the resulting estimator converges up to the ground truth parameters of MLR, and its scaled error converges to the Gaussian limit of the maximum\-likelihood estimator\. At a fixed scale of the diffusion noise level, we derive a decomposition linking the score matching loss to cross\-entropy and Expectation\-Maximization \(EM\) operators\. This decomposition yields an EM\-related low\-noise gradient expansion with additional correction terms of latent variance\. In the high\-noise limit, we further characterize gradient descent on this limiting loss under isotropic covariance\. Along fixed high signal\-to\-noise ratio rays, the score matching imbalance gradient and the latent\-variance term tend to zero pointwise\. Numerical experiments illustrate our theoretical findings and statistical guarantees\.

††footnotetext:LLMs were used as a general\-purpose assistive tool during manuscript preparation for language editing and code assistance\. The authors independently verified theoretical and experimental results and take full responsibility for the manuscript\.### 1Introduction

For a response law with several modes, its mean can fall in a region of low probability and fail to describe a plausible outcome\([Bishop, 1994](https://arxiv.org/html/2609.05688#bib.bib3);[Han et al\., 2022](https://arxiv.org/html/2609.05688#bib.bib32)\)\. Diffusion models instead learn scores for Gaussian\-smoothed versions of the response law\([Ho et al\., 2020](https://arxiv.org/html/2609.05688#bib.bib35);[Song et al\., 2021b](https://arxiv.org/html/2609.05688#bib.bib34)\)\. Conditional variants retain the covariates while adding noise to the response\.[Batzolis et al\. \(2021\)](https://arxiv.org/html/2609.05688#bib.bib31)justified conditional denoising score estimation, while[Fu et al\. \(2024\)](https://arxiv.org/html/2609.05688#bib.bib33)and[Tang et al\. \(2025\)](https://arxiv.org/html/2609.05688#bib.bib48)established conditional\-score and distribution\-estimation guarantees under their respective assumptions\. Building on this distributional perspective, we study parameter recovery in a structured latent model, its connection to latent\-variable optimization, and the variation of parameter information and gradient signal with diffusion noise level and signal strength\. We study these questions in mixed linear regression \(MLR\), a classical latent model\([Quandt, 1972](https://arxiv.org/html/2609.05688#bib.bib45);[De Veaux, 1989](https://arxiv.org/html/2609.05688#bib.bib2);[Zilber and Nadler, 2023](https://arxiv.org/html/2609.05688#bib.bib61)\)\. Under Gaussian diffusion of the response, the conditional score and the posterior of the latent sign remain explicit\. We denote the covariate by𝐱∈ℝd\\mathbf\{x\}\\in\\mathbb\{R\}^\{d\}and the response byy0∈ℝy\_\{0\}\\in\\mathbb\{R\}in the MLR model\. The ground truth regression parametersθ∗\\theta^\{\\ast\}represent the separation between components,ν∗\\nu^\{\\ast\}characterizes the imbalance of the ground truth mixing weightsπ∗=\(π∗\(1\),π∗\(2\)\):=\(\(1\+tanh⁡ν∗\)/2,\(1−tanh⁡ν∗\)/2\)\\pi^\{\\ast\}=\(\\pi^\{\\ast\}\(1\),\\pi^\{\\ast\}\(2\)\):=\(\(1\+\\tanh\\nu^\{\\ast\}\)/2,\(1\-\\tanh\\nu^\{\\ast\}\)/2\), and the conditional law for the response is

y0\|𝐱∼π∗\(1\)𝒩\(⟨θ∗,𝐱⟩,1\)\+π∗\(2\)𝒩\(−⟨θ∗,𝐱⟩,1\)\.y\_\{0\}\\mid\\mathbf\{x\}\\sim\\pi^\{\\ast\}\(1\)\\,\\mathcal\{N\}\\\!\\left\(\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle,1\\right\)\+\\pi^\{\\ast\}\(2\)\\,\\mathcal\{N\}\\\!\\left\(\-\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle,1\\right\)\.\(1\)Questions:For this MLR model, under what conditions does the path\-integrated score matching estimator recover the regression parameters and mixing weights? How is the score matching loss related to Expectation\-Maximization, and how do the noise level and signal strength affect the optimization dynamics and parameter information retained by score matching?

##### Score matching and mixture learning\.

Score matching was introduced for unnormalized models, connected to maximum likelihood under perturbation, and linked to denoising objectives\([Hyvärinen, 2005](https://arxiv.org/html/2609.05688#bib.bib49);[Lyu, 2009](https://arxiv.org/html/2609.05688#bib.bib50);[Vincent, 2011](https://arxiv.org/html/2609.05688#bib.bib36)\)\. Analyses based on variational methods and maximum likelihood connected diffusion score objectives to likelihood under their respective reverse models and weightings\([Huang et al\., 2021](https://arxiv.org/html/2609.05688#bib.bib52);[Song et al\., 2021a](https://arxiv.org/html/2609.05688#bib.bib53)\)\. Results on learning densities and distributions with scores further related an integrated score objective to density and parameter estimation\([Li and Yan, 2024](https://arxiv.org/html/2609.05688#bib.bib58);[Chewi et al\., 2025a](https://arxiv.org/html/2609.05688#bib.bib18)\), while results for Gaussian mixtures provided end\-to\-end distribution\-learning guarantees for diffusion and score methods\([Chen et al\., 2025](https://arxiv.org/html/2609.05688#bib.bib13);[Gatmiry et al\., 2025](https://arxiv.org/html/2609.05688#bib.bib14)\)\.[Shah et al\. \(2023\)](https://arxiv.org/html/2609.05688#bib.bib23)analyzed balanced spherical Gaussian mixtures\. Their large\-noise proof used a spectral iteration, while their small\-noise proof compared a gradient step with the M\-step of Expectation\-Maximization \(EM\)\. Our conditional Gaussian components have centers that move with𝐱\\mathbf\{x\}, and both their regression parameters and mixing weights are unknown\. Prior work studied MLR through identifiability, spectral and moment methods, convex formulations, alternating minimization, approximate message passing, and EM\([Hennig, 2000](https://arxiv.org/html/2609.05688#bib.bib46);[Chaganty and Liang, 2013](https://arxiv.org/html/2609.05688#bib.bib4);[Zhong et al\., 2016](https://arxiv.org/html/2609.05688#bib.bib47);[Yi et al\., 2014](https://arxiv.org/html/2609.05688#bib.bib5);[Chen et al\., 2018](https://arxiv.org/html/2609.05688#bib.bib7);[Tan and Venkataramanan, 2023](https://arxiv.org/html/2609.05688#bib.bib62);[Balakrishnan et al\., 2017](https://arxiv.org/html/2609.05688#bib.bib6);[Klusowski et al\., 2019](https://arxiv.org/html/2609.05688#bib.bib8);[Kwon et al\., 2021](https://arxiv.org/html/2609.05688#bib.bib9);[Kwon et al\., 2024](https://arxiv.org/html/2609.05688#bib.bib16)\)\.[Zilber and Nadler \(2023\)](https://arxiv.org/html/2609.05688#bib.bib61)studied estimation of unknown imbalanced MLR, and[Luo and Hashemi \(2025\)](https://arxiv.org/html/2609.05688#bib.bib25);[Luo and Hashemi \(2026\)](https://arxiv.org/html/2609.05688#bib.bib26)characterized the coupled EM operators for the regression parameters and the imbalance of mixing weights\.[Ozkara et al\. \(2026\)](https://arxiv.org/html/2609.05688#bib.bib30)used a related conditional diffusion method to estimate client\-specific weights in a two\-component Gaussian mixture at one noise level with fixed component means\. Our setting estimates both the covariate\-dependent centers and the imbalance of mixing weights\.

##### Score blindness and the role of noise\.

At a high diffusion noise level, component separation can remove the information about the mixing weights from the score as established by[Wenliang and Kanagawa \(2020\)](https://arxiv.org/html/2609.05688#bib.bib55)in one\-dimensional Gaussian examples with two mixtures where the Fisher divergence between them vanished as the separation grew\.[Zhang et al\. \(2022\)](https://arxiv.org/html/2609.05688#bib.bib56)proposed a mixture Fisher divergence to mitigate this failure, while[Koehler et al\. \(2023\)](https://arxiv.org/html/2609.05688#bib.bib54)related score matching efficiency to isoperimetry\. The corresponding behavior of population EM in[Luo and Hashemi \(2024\)](https://arxiv.org/html/2609.05688#bib.bib24)showed that the EM update for the mixing weights was independent of the incoming imbalance of mixing weights in the noiseless setting but remained informative about the ground\-truth mixing imbalance through the alignment of the current and ground\-truth regression parameters\. For finite Gaussian mixtures with known mixture weights and a known shared covariance,[Qin and Risteski \(2024\)](https://arxiv.org/html/2609.05688#bib.bib57)proved a polynomial asymptotic bound on the statistical complexity of fitting the component means with a generalized score matching objective that used continuous tempering and was related to annealed score matching\. For mixing weights of a two\-Gaussian family with known component parameters,[Lütke Schwienhorst et al\. \(2026\)](https://arxiv.org/html/2609.05688#bib.bib60)showed that suitable horizon tuning removed separation dependence from their finite\-sample error bound, while[Dennehy et al\. \(2026\)](https://arxiv.org/html/2609.05688#bib.bib59)showed that mode\-overlapping intermediate scales of the diffusion noise level could retain information about the mixing weights\.

#### 1\.1Technical Overview and Contributions

The technical development combines the relative\-entropy and likelihood bridges by following the de Bruijn identity\([Stam, 1959](https://arxiv.org/html/2609.05688#bib.bib51)\), Expectation\-Maximization update\([Dempster et al\., 1977](https://arxiv.org/html/2609.05688#bib.bib1)\)with the coupled EM operators\([Weinberger and Bresler, 2022](https://arxiv.org/html/2609.05688#bib.bib12);[Luo and Hashemi, 2025](https://arxiv.org/html/2609.05688#bib.bib25)\), and standard M\-estimation arguments\([van der Vaart, 1998](https://arxiv.org/html/2609.05688#bib.bib15);[Chewi et al\., 2025b](https://arxiv.org/html/2609.05688#bib.bib17);[Lütke Schwienhorst et al\., 2026](https://arxiv.org/html/2609.05688#bib.bib60)\)\. The main contributions are as follows\.

- •Path\-Integrated Score Matching: Maximum\-Likelihood Asymptotics\(Section[3](https://arxiv.org/html/2609.05688#S3)\)\. Lemmas[A\.15](https://arxiv.org/html/2609.05688#A1.Thmstyle15)and[A\.16](https://arxiv.org/html/2609.05688#A1.Thmstyle16)establish population and empirical likelihood bridges for which terminal discrepancies form the only parameter\-dependent gap between the path\-integrated objective and likelihood\. Proposition[3\.1](https://arxiv.org/html/2609.05688#S3.Thmstyle1)shows that the terminal population KL divergence preserves identifiability, and Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)proves that controlling the terminal discrepancy transfers consistency and asymptotic normality with inverse\-Fisher covariance from maximum likelihood to the path\-integrated Score Matching estimator\.
- •Fixed\-Scale Score Matching: Cross\-Entropy and EM Decompositions\(Section[4](https://arxiv.org/html/2609.05688#S4)\)\. Propositions[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)and[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2)identify the diffused MLR Expectation\-Maximization \(EM\) operators and express the cross\-entropy gradients as EM residuals, while Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3)shows that the fixed\-scale loss admits equivalent decompositions of cross\-entropy and EM operators\. Corollaries[4\.4](https://arxiv.org/html/2609.05688#S4.Thmstyle4)and[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5), together with Proposition[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6), translate these decompositions into exact gradient identities, showing that the score matching gradient field involves EM operators, operator derivatives, and latent variance terms\. Equation \([15](https://arxiv.org/html/2609.05688#S4.E15)\) and Propositions[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)and[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)then characterize the two endpoint regimes: the low\-noise gradient limit retains this EM\-related correction structure, whereas the high\-noise limiting objective depends on the candidate parameters and the ground truth parameters\.
- •High\-Noise Gradient Dynamics and Score Matching Blindness\(Section[5](https://arxiv.org/html/2609.05688#S5)\)\. Main Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)gives an𝒪⁡\(K−2\)\\mathcal\{O\}\(K^\{\-2\}\)loss bound when the effective target is zero, whereKKdenotes the number of gradient\-descent iterations, and an𝒪⁡\(log⁡\(1/ϵ\)\)\\mathcal\{O\}\(\\log\(1/\\epsilon\)\)bound for generic initializations when the effective target is nonzero, with only a measure\-zero exceptional set attracted to a saddle\. Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)formalizes pointwise blindness to the mixing imbalance: as the ground\-truth signal diverges along a fixed ray, the gradient in the imbalance parameter vanishes at fixed scale of the diffusion noise level\.

##### Organization\.

In Section[2](https://arxiv.org/html/2609.05688#S2), we formulate the MLR model, score matching objectives, and associated EM operators\. In Section[3](https://arxiv.org/html/2609.05688#S3), we show how a likelihood bridge transfers maximum\-likelihood asymptotics to the path\-integrated Score Matching estimator\. In Section[4](https://arxiv.org/html/2609.05688#S4), we derive exact cross\-entropy and EM decompositions and use them to characterize the fixed\-scale loss and gradients across the low\- and high\-noise regimes\. In Section[5](https://arxiv.org/html/2609.05688#S5), we analyze high\-noise gradient dynamics and pointwise score matching blindness\. In Section[6](https://arxiv.org/html/2609.05688#S6), we consolidate the corresponding numerical implications\. In Appendix[A](https://arxiv.org/html/2609.05688#A1), we develop the auxiliary identities and bounds used in the main proofs for results of Sections[3](https://arxiv.org/html/2609.05688#S3)–[5](https://arxiv.org/html/2609.05688#S5)in Appendices[B](https://arxiv.org/html/2609.05688#A2)–[D](https://arxiv.org/html/2609.05688#A4)\. In Appendix[E](https://arxiv.org/html/2609.05688#A5), we record the experimental details for the numerical results to verify the theoretical findings of Sections[3](https://arxiv.org/html/2609.05688#S3)–[5](https://arxiv.org/html/2609.05688#S5)\.

### 2Problem Setup

In this section, we specify the mixed linear regression \(MLR\) model and its variance\-preserving diffusion of the response\. We then define the population and empirical score matching objectives\. We also introduce the cross\-entropy and EM operators and the notations required for the estimation and asymptotic analysis\. See the detailed derivations of the equations below in Appendix[A](https://arxiv.org/html/2609.05688#A1)\.

#### 2\.1Mixed Linear Regression and Variance\-Preserving Diffusion

In the two\-component mixed linear regression \(MLR\) model, the responsey0y\_\{0\}is generated from the regression parameterθ\\thetaand additive noiseε0\\varepsilon\_\{0\}according to

y0=\(−1\)z\+1⟨θ,𝐱⟩\+ε0,y\_\{0\}=\(\-1\)^\{z\+1\}\\langle\\theta,\\mathbf\{x\}\\rangle\+\\varepsilon\_\{0\},\(2\)where𝔼⁡\[𝐱𝐱⊤\]=Σ≻0\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]=\\Sigma\\succ 0andε0∼𝒩⁡\(0,1\)\\varepsilon\_\{0\}\\sim\\mathcal\{N\}\(0,1\)\. The latent variablez∈\{1,2\}z\\in\\\{1,2\\\}has probabilitiesℙ⁡\(z=1∣π\)=π⁡\(1\)\\mathbb\{P\}\(z=1\\mid\\pi\)=\\pi\(1\)andℙ⁡\(z=2∣π\)=π⁡\(2\)\\mathbb\{P\}\(z=2\\mid\\pi\)=\\pi\(2\)\. The mixing weightsπ=\(π⁡\(1\),π⁡\(2\)\)\\pi=\(\\pi\(1\),\\pi\(2\)\)are positive and sum to one\. We impose the independence assumptions\(z,π\)⟂⁣⟂θ\(z;\\pi\)\\mathrel\{\\perp\\\!\\\!\\\!\\perp\}\\theta,𝐱⟂⁣⟂\(z,θ,π\)\\mathbf\{x\}\\mathrel\{\\perp\\\!\\\!\\\!\\perp\}\(z;\\theta,\\pi\), andε0⟂⁣⟂\(𝐱,z,θ,π\)\\varepsilon\_\{0\}\\mathrel\{\\perp\\\!\\\!\\\!\\perp\}\(\\mathbf\{x\},z;\\theta,\\pi\)\. The half log\-odds parameterν:=\(ln⁡π⁡\(1\)−ln⁡π⁡\(2\)\)/2\\nu:=\(\\ln\\pi\(1\)\-\\ln\\pi\(2\)\)/2characterizes the mixing imbalance and satisfiestanh⁡ν=π⁡\(1\)−π⁡\(2\)\\tanh\\nu=\\pi\(1\)\-\\pi\(2\),π⁡\(1\)=\(1\+tanh⁡ν\)/2\\pi\(1\)=\(1\+\\tanh\\nu\)/2, andπ⁡\(2\)=\(1−tanh⁡ν\)/2\\pi\(2\)=\(1\-\\tanh\\nu\)/2\. The variance\-preserving diffusion process satisfies the stochastic differential equation \(SDE\)

dyt=−βt2ytdt\+βtdBt,\\mathrm\{d\}y\_\{t\}=\-\\frac\{\\beta\_\{t\}\}\{2\}y\_\{t\}\\mathrm\{d\}t\+\\sqrt\{\\beta\_\{t\}\}\\mathrm\{d\}B\_\{t\},\(3\)whereβt\>0\\beta\_\{t\}\>0is the noise schedule andBtB\_\{t\}is a standard Brownian motion satisfying𝔼⁡\[Bt\]=0\\mathbb\{E\}\[B\_\{t\}\]=0and𝔼⁡\[BtBs\]=min⁡\(t,s\)\\mathbb\{E\}\[B\_\{t\}B\_\{s\}\]=\\min\(t,s\)\. Noise schedules used or analyzed in the literature include the constant choiceβt=2\\beta\_\{t\}=2\([Chen et al\., 2023](https://arxiv.org/html/2609.05688#bib.bib27)\), the linear choiceβt=a\+bt\\beta\_\{t\}=a\+bt\([Ho et al\., 2020](https://arxiv.org/html/2609.05688#bib.bib35)\), and the power\-law familyβt=\(a\+bt\)ρ\\beta\_\{t\}=\(a\+bt\)^\{\\rho\}withρ≥1\\rho\\geq 1\([Gao et al\., 2025](https://arxiv.org/html/2609.05688#bib.bib28);[Gao and Zhu, 2025](https://arxiv.org/html/2609.05688#bib.bib29)\)\. The retained\-signal factorα¯t:=exp\(−∫0tβτdτ\)\\bar\{\\alpha\}\_\{t\}:=\\exp\\left\(\-\\int\_\{0\}^\{t\}\\beta\_\{\\tau\}\\mathrm\{d\}\\tau\\right\)encodes cumulative attenuation along the diffusion path, and its square rootα¯t\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}controls the decay of the signal fromy0y\_\{0\}\. The corresponding forward transition density isp⁡\(yt\|y0\)=𝒩⁡\(yt,y0α¯t,1−α¯t\)p\(y\_\{t\}\|y\_\{0\}\)=\\mathcal\{N\}\\left\(y\_\{t\};y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},1\-\\bar\{\\alpha\}\_\{t\}\\right\), so the response variableyty\_\{t\}at timettadmits the representation

yt=y0α¯t\+ξt1−α¯t=\(−1\)z\+1⟨θt,𝐱⟩\+εt,y\_\{t\}=y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\_\{t\}\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}=\(\-1\)^\{z\+1\}\\langle\\theta\_\{t\},\\mathbf\{x\}\\rangle\+\\varepsilon\_\{t\},\(4\)whereξt∼𝒩⁡\(0,1\)\\xi\_\{t\}\\sim\\mathcal\{N\}\(0,1\)andεt∼𝒩⁡\(0,1\)\\varepsilon\_\{t\}\\sim\\mathcal\{N\}\(0,1\)are additive Gaussian noises independent of𝐱\\mathbf\{x\}andzz, whileθt:=θα¯t\\theta\_\{t\}:=\\theta\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}andμt:=⟨θt,𝐱⟩\\mu\_\{t\}:=\\langle\\theta\_\{t\},\\mathbf\{x\}\\rangleprovide concise notation for the scaled parameters at timett\. Att=0t=0, these definitions giveθ0≡θ\\theta\_\{0\}\\equiv\\thetaandμ0≡⟨θ,𝐱⟩\\mu\_\{0\}\\equiv\\langle\\theta,\\mathbf\{x\}\\rangle\. These identities implyμt=μ0α¯t\\mu\_\{t\}=\\mu\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}anddμt/dt=−\(βt/2\)μt\\mathrm\{d\}\\mu\_\{t\}/\\mathrm\{d\}t=\-\(\\beta\_\{t\}/2\)\\mu\_\{t\}\. For the ground\-truth data\-generating parametersθ∗∈ℝd\\theta^\{\\ast\}\\in\\mathbb\{R\}^\{d\}andπ∗≡\(π∗\(1\),π∗\(2\)\)\\pi^\{\\ast\}\\equiv\(\\pi^\{\\ast\}\(1\),\\pi^\{\\ast\}\(2\)\), we haveν∗:=\(ln⁡π∗\(1\)−ln⁡π∗\(2\)\)/2\\nu^\{\\ast\}:=\(\\ln\\pi^\{\\ast\}\(1\)\-\\ln\\pi^\{\\ast\}\(2\)\)/2,θt∗:=θ∗α¯t\\theta\_\{t\}^\{\\ast\}:=\\theta^\{\\ast\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}, andμt∗:=⟨θt∗,𝐱⟩\\mu\_\{t\}^\{\\ast\}:=\\langle\\theta\_\{t\}^\{\\ast\},\\mathbf\{x\}\\rangle\.

#### 2\.2Score Function and Score Matching Loss

With the scaled regression parametersθt≡θα¯t\\theta\_\{t\}\\equiv\\theta\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}, mixing weightsπ\\pi, and shorthandμt≡μ0α¯t=⟨θ,𝐱⟩α¯t\\mu\_\{t\}\\equiv\\mu\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}=\\langle\\theta,\\mathbf\{x\}\\rangle\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}, the Stein score function with respect toyty\_\{t\}is

sθt,ν\(yt,𝐱\):=∇log⁡p\(yt∣𝐱;θt,π\)=−yt\+tanh⁡\(μtyt\+ν\)μt\.s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\):=\\nabla\\log p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\-y\_\{t\}\+\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\mu\_\{t\}\.\(5\)
The score matching lossℒt\(θ,ν\)\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)is the covariate expectation of the relative Fisher information𝖥𝖨\(p\(yt∣𝐱;θt∗,π∗\)∥p\(yt∣𝐱;θt,π\)\):=𝔼yt\|𝐱;θt∗,π∗∥sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)∥2\\FI\(p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\):=\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}between the model score with parametersθ,π\\theta,\\piand the true score with ground\-truth parametersθ∗,π∗\\theta^\{\\ast\},\\pi^\{\\ast\}under the same normalization of1/\(2α¯t\)1/\(2\\bar\{\\alpha\}\_\{t\}\)\.

ℒt\(θ,ν\):=12α¯t𝔼yt,𝐱\|θ∗,π∗∥sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)∥2=12α¯t𝔼𝐱𝖥𝖨\(p\(yt∣𝐱;θt∗,π∗\)∥p\(yt∣𝐱;θt,π\)\)\.\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\):=\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{y\_\{t\},\\mathbf\{x\}\|\\theta^\{\*\},\\pi^\{\*\}\}\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta^\{\*\}\_\{t\},\\nu^\{\*\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}=\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\FI\(p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta^\{\*\}\_\{t\},\\pi^\{\*\}\)\\parallel p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\)\.\(6\)
The auxiliary loss𝒥t\(θ,ν\)\\mathcal\{J\}\_\{t\}\(\\theta,\\nu\)compares the model score with the score of the forward transition\. The Gaussian transition isp⁡\(yt∣y0\)=𝒩⁡\(yt,y0α¯t,1−α¯t\)p\(y\_\{t\}\\mid y\_\{0\}\)=\\mathcal\{N\}\(y\_\{t\};y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},1\-\\bar\{\\alpha\}\_\{t\}\)\. Its score satisfies∇log⁡p\(yt∣y0\)=−yt−y0α¯t1−α¯t=𝖽−ξ1−α¯t\\nabla\\log p\(y\_\{t\}\\mid y\_\{0\}\)=\-\\frac\{y\_\{t\}\-y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\}\{1\-\\bar\{\\alpha\}\_\{t\}\}\\stackrel\{\{\\scriptstyle\\mathsf\{d\}\}\}\{\{=\}\}\-\\frac\{\\xi\}\{\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\}, whereξ=𝖽ξt∼𝒩⁡\(0,1\)\\xi\\stackrel\{\{\\scriptstyle\\mathsf\{d\}\}\}\{\{=\}\}\\xi\_\{t\}\\sim\\mathcal\{N\}\(0,1\)is independent of\(y0,𝐱\)\(y\_\{0\},\\mathbf\{x\}\)\. The equivalent representationyt=𝖽y0α¯t\+ξ1−α¯ty\_\{t\}\\stackrel\{\{\\scriptstyle\\mathsf\{d\}\}\}\{\{=\}\}y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}gives

𝒥t\(θ,ν\):=\\displaystyle\\mathcal\{J\}\_\{t\}\(\\theta,\\nu\):=12α¯t𝔼yt,y0,𝐱\|θ∗,π∗\[‖sθt,ν\(yt,𝐱\)‖2−2⟨sθt,ν\(yt,𝐱\),∇log⁡p\(yt∣y0\)⟩\]\\displaystyle\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{y\_\{t\},y\_\{0\},\\mathbf\{x\}\|\\theta^\{\*\},\\pi^\{\*\}\}\\left\[\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\-2\\langle s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\),\\nabla\\log p\(y\_\{t\}\\mid y\_\{0\}\)\\rangle\\right\]\(7\)=\\displaystyle=12α¯t𝔼y0,𝐱\|θ∗,π∗𝔼ξ\[‖sθt,ν\(y0α¯t\+ξ1−α¯t,𝐱\)‖2\+2⟨sθt,ν\(y0α¯t\+ξ1−α¯t,𝐱\),ξ1−α¯t⟩\]\.\\displaystyle\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\|\\theta^\{\*\},\\pi^\{\*\}\}\\mathbb\{E\}\_\{\\xi\}\\left\[\\Big\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\},\\mathbf\{x\}\)\\Big\\\|^\{2\}\+2\\left\\langle s\_\{\\theta\_\{t\},\\nu\}\(y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\},\\mathbf\{x\}\),\\frac\{\\xi\}\{\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\}\\right\\rangle\\right\]\.
For i\.i\.d\. observations\{\(y0\(i\),𝐱\(i\)\)\}i=1n∼i\.i\.d\.p\(y0,𝐱∣θ∗,π∗\)\\\{\(y\_\{0\}^\{\(i\)\},\\mathbf\{x\}^\{\(i\)\}\)\\\}\_\{i=1\}^\{n\}\\stackrel\{\{\\scriptstyle\\mathrm\{i\.i\.d\.\}\}\}\{\{\\sim\}\}p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta^\{\\ast\},\\pi^\{\\ast\}\), we defineℙn:=n−1∑i=1nδ\(y0\(i\),𝐱\(i\)\)\\mathbb\{P\}\_\{n\}:=n^\{\-1\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\(y\_\{0\}^\{\(i\)\},\\mathbf\{x\}^\{\(i\)\}\)\}and𝔼ℙn\[f⁡\(y0,𝐱\)\]:=n−1∑i=1nf⁡\(y0\(i\),𝐱\(i\)\)\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[f\(y\_\{0\},\\mathbf\{x\}\)\]:=n^\{\-1\}\\sum\_\{i=1\}^\{n\}f\(y\_\{0\}^\{\(i\)\},\\mathbf\{x\}^\{\(i\)\}\)\. Thus𝔼ℙn\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}averages over the observed pairs\. The inner expectation in𝔼ℙn𝔼yt\|y0\[⋅\]\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\[\\cdot\]averages over forward\-diffusion noise\. The finite\-sample loss is given by

𝒥tn\(θ,ν\):=\\displaystyle\\mathcal\{J\}\_\{t\}^\{n\}\(\\theta,\\nu\):=12α¯t𝔼ℙn𝔼yt\|y0\[‖sθt,ν\(yt,𝐱\)‖2−2⟨sθt,ν\(yt,𝐱\),∇log⁡p\(yt∣y0\)⟩\]\\displaystyle\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\\left\[\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\-2\\langle s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\),\\nabla\\log p\(y\_\{t\}\\mid y\_\{0\}\)\\rangle\\right\]\(8\)=\\displaystyle=12α¯t𝔼ℙn𝔼ξ\[‖sθt,ν\(y0α¯t\+ξ1−α¯t,𝐱\)‖2\+2⟨sθt,ν\(y0α¯t\+ξ1−α¯t,𝐱\),ξ1−α¯t⟩\]\.\\displaystyle\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{\\xi\}\\left\[\\Big\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\},\\mathbf\{x\}\)\\Big\\\|^\{2\}\+2\\left\\langle s\_\{\\theta\_\{t\},\\nu\}\(y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\},\\mathbf\{x\}\),\\frac\{\\xi\}\{\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\}\\right\\rangle\\right\]\.

#### 2\.3Negative Log\-Likelihood and Expectation\-Maximization Operators

The unit\-variance negative log\-likelihood and the associated population EM operators are given by the following equations, see detailed derivations in Appendix[A](https://arxiv.org/html/2609.05688#A1)and Proposition 3 and Appendix A of\([Luo and Hashemi, 2025](https://arxiv.org/html/2609.05688#bib.bib25)\)\. The negative log\-likelihood is

F⁡\(yt,μt,ν\):=−ln⁡p⁡\(yt∣𝐱;θt,π\)=12\(yt2\+μt2\)−ln⁡cosh⁡\(μtyt\+ν\)cosh⁡\(ν\)\+12ln⁡\(2π\)\.F\(y\_\{t\},\\mu\_\{t\},\\nu\):=\-\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\\frac\{1\}\{2\}\(y\_\{t\}^\{2\}\+\\mu\_\{t\}^\{2\}\)\-\\ln\\frac\{\\cosh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\}\{\\cosh\(\\nu\)\}\+\\frac\{1\}\{2\}\\ln\(2\\uppi\)\.\(9\)We use𝔼∗\[⋅\]:=𝔼yt,𝐱\|θ∗,π∗\[⋅\]\\mathbb\{E\}\_\{\\ast\}\[\\cdot\]:=\\mathbb\{E\}\_\{y\_\{t\},\\mathbf\{x\}\\mid\\theta^\{\\ast\},\\pi^\{\\ast\}\}\[\\cdot\]for expectation under the ground\-truth law\. The cross\-entropyℋ⁡\(θt,ν\):=𝔼∗\[−ln⁡p⁡\(yt∣𝐱;θt,π\)\]=𝔼∗\[F⁡\(yt,μt,ν\)\]\\mathcal\{H\}\(\\theta\_\{t\},\\nu\):=\\mathbb\{E\}\_\{\\ast\}\[\-\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\]=\\mathbb\{E\}\_\{\\ast\}\[F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]is the expected negative log\-likelihood\. The population Expectation\-Maximization \(EM\) operators forθt\\theta\_\{t\}and the imbalance of mixing weightstanh⁡ν≡π⁡\(1\)−π⁡\(2\)\\tanh\\nu\\equiv\\pi\(1\)\-\\pi\(2\)are

M⁡\(θt,ν\)=Σ−1𝔼∗\[𝐱yttanh⁡\(μtyt\+ν\)\],N⁡\(θt,ν\)=𝔼∗\[tanh⁡\(μtyt\+ν\)\]\.M\(\\theta\_\{t\},\\nu\)=\\Sigma^\{\-1\}\\mathbb\{E\}\_\{\\ast\}\\big\[\\mathbf\{x\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\big\],\\qquad N\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\\big\[\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\big\]\.\(10\)

#### 2\.4Notations on Metrics, Asymptotics, and Estimation Objectives

For probability measuresP,QP,Qon a common measurable space, we define𝖪𝖫\(P∥Q\):=∫ln\(dP/dQ\)dP\\KL\(P\\parallel Q\):=\\int\\ln\(\\mathrm\{d\}P/\\mathrm\{d\}Q\)\\,\\mathrm\{d\}PwhenP≪QP\\ll Qand𝖳𝖵⁡\(P,Q\):=supA\|P⁡\(A\)−Q⁡\(A\)\|\\TV\(P,Q\):=\\sup\_\{A\}\|P\(A\)\-Q\(A\)\|over measurable sets\. For probability measuresP,QP,Qwith finite second moments, the Wasserstein distance is defined asW22\(P,Q\):=infγ∈Γ⁡\(P,Q\)∫‖u−v‖22γ\(𝑑u,𝑑v\)W\_\{2\}^\{2\}\(P,Q\):=\\inf\_\{\\gamma\\in\\Gamma\(P,Q\)\}\\int\\\|u\-v\\\|\_\{2\}^\{2\}\\,\\gamma\(\\mathrm\{d\}u,\\mathrm\{d\}v\)over their set of couplingsΓ⁡\(P,Q\)\\Gamma\(P,Q\)\. We writeXn=𝒪P\(an\)X\_\{n\}=\\mathcal\{O\}\_\{P\}\(a\_\{n\}\)whenXn/anX\_\{n\}/a\_\{n\}is stochastically bounded andXn=oP\(an\)X\_\{n\}=o\_\{P\}\(a\_\{n\}\)whenXn/an→ℙ0X\_\{n\}/a\_\{n\}\\xrightarrow\{\\mathbb\{P\}\}0\. The model parametersϑ:=\(θ,ν\)\\vartheta:=\(\\theta,\\nu\)and the ground\-truth valuesϑ∗:=\(θ∗,ν∗\)\\vartheta^\{\\ast\}:=\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)are introduced for the convenience of notation\.TnT\_\{n\}denotes a deterministic terminal horizon for the total number of samplesnn, andα¯Tn\\bar\{\\alpha\}\_\{T\_\{n\}\}is the retained\-signal factor at the terminal horizon\. Usingℙn\\mathbb\{P\}\_\{n\}and𝒥tn\\mathcal\{J\}\_\{t\}^\{n\}defined above, the empirical negative log\-likelihood is defined asℛ^n𝖬𝖫𝖤\(ϑ\):=−𝔼ℙn\[ln⁡p⁡\(y0∣𝐱;θ,π\)\]\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\):=\-\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[\\ln p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta,\\pi\)\], and the path\-integrated score matching objective is denoted byℛ^n,Tn𝖲𝖬\(ϑ\):=∫α¯Tnα¯0𝒥tn\(θ,ν\)dα¯t\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\):=\\int\_\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}^\{\\bar\{\\alpha\}\_\{0\}\}\\mathcal\{J\}\_\{t\}^\{n\}\(\\theta,\\nu\)\\,\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\. On the event that this objective has a nonempty argmin, letϑ^n,Tn𝖲𝖬:=\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}:=\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)be a sample\-measurable exact minimizer; on the complementary event, setϑ^n,Tn𝖲𝖬:=\(𝟎,0\)\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}:=\(\\mathbf\{0\},0\)\. This convention defines the score matching estimator on the entire sample space, and Lemma[B\.2](https://arxiv.org/html/2609.05688#A2.Thmstyle2)supplies the measurable selection used under the conditions of Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)\. The empirical conditional terminal discrepancy is defined asΔn\(ϑ\):=𝔼ℙn\[𝖪𝖫\(p\(yTn∣y0\)∥p\(yTn∣𝐱;θTn,π\)\)\]≥0\\Delta\_\{n\}\(\\vartheta\):=\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\\!\\left\[\\KL\\\!\\left\(p\(y\_\{T\_\{n\}\}\\mid y\_\{0\}\)\\parallel p\(y\_\{T\_\{n\}\}\\mid\\mathbf\{x\};\\theta\_\{T\_\{n\}\},\\pi\)\\right\)\\right\]\\geq 0\. The constant term𝖼⁡\(α¯Tn\):=12\(ln⁡1−α¯Tnα¯Tn\+ln⁡\(2π\)\+1\)\\mathsf\{c\}\(\\bar\{\\alpha\}\_\{T\_\{n\}\}\):=\\frac\{1\}\{2\}\(\\ln\\frac\{1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\+\\ln\(2\\uppi\)\+1\)only depends on the retained\-signal factorα¯Tn\\bar\{\\alpha\}\_\{T\_\{n\}\}at the terminal horizon\.

### 3Path\-Integrated Score Matching: Maximum\-Likelihood Asymptotics

In this section, we show how integrating score matching along the diffusion path recovers the asymptotic guarantees of maximum likelihood\. We first derive population and empirical likelihood bridges that isolate the terminal discrepancy separating the integrated objective from likelihood and identify the terminal population KL divergence’s global minimizers\. We then control this discrepancy to transfer consistency up to joint sign and asymptotic normality with inverse\-Fisher covariance to the path\-integrated estimator in Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2), with the supporting arguments deferred to Appendices[A](https://arxiv.org/html/2609.05688#A1)and[B](https://arxiv.org/html/2609.05688#A2)\. by the two\-law relative analogue of de Bruijn identity\([Stam, 1959](https://arxiv.org/html/2609.05688#bib.bib51)\)for entropy and Fisher information under the variance\-preserving Ornstein–Uhlenbeck flow \(see the derivation of Lemma[A\.4](https://arxiv.org/html/2609.05688#A1.Thmstyle4)in Appendix[A](https://arxiv.org/html/2609.05688#A1)\)\.

∫α¯Tα¯0ℒt\(θ,ν\)dα¯t=𝖪𝖫\(p\(y0,𝐱∣θ∗,π∗\)∥p\(y0,𝐱∣θ,π\)\)−𝖪𝖫\(p\(yT,𝐱∣θT∗,π∗\)∥p\(yT,𝐱∣θT,π\)\)\.\\int\_\{\\bar\{\\alpha\}\_\{T\}\}^\{\\bar\{\\alpha\}\_\{0\}\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}=\\KL\(p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta^\{\*\},\\pi^\{\*\}\)\\parallel p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta,\\pi\)\)\-\\KL\(p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\}^\{\*\},\\pi^\{\*\}\)\\parallel p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\},\\pi\)\)\.\(11\)Its finite\-sample counterpart involves the empirical objectivesℛ^n,Tn𝖲𝖬\(ϑ\)\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\),ℛ^n𝖬𝖫𝖤\(ϑ\)\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\), andΔn\(ϑ\)\\Delta\_\{n\}\(\\vartheta\), which correspond to the left\-hand side and the first and second terms on the right\-hand side of the equation above, respectively, and differ from them only by the constant𝖼⁡\(α¯Tn\)\\mathsf\{c\}\(\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\(see Lemmas[A\.15](https://arxiv.org/html/2609.05688#A1.Thmstyle15)and[A\.16](https://arxiv.org/html/2609.05688#A1.Thmstyle16)in Appendix[A](https://arxiv.org/html/2609.05688#A1)\)\.

ℛ^n,Tn𝖲𝖬\(ϑ\)=ℛ^n𝖬𝖫𝖤\(ϑ\)−Δn\(ϑ\)−𝖼⁡\(α¯Tn\)\.\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\)=\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\)\-\\Delta\_\{n\}\(\\vartheta\)\-\\mathsf\{c\}\(\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\.\(12\)We can bound the terminal population KL divergence from the true law to the model law by the Wasserstein geometry of the discrete mixing measures supported on at most two pointsD≡D⁡\(𝐱,θ,π\)≡π⁡\(1\)δ⟨θ,𝐱⟩\+π⁡\(2\)δ−⟨θ,𝐱⟩D\\equiv D\(\\mathbf\{x\};\\theta,\\pi\)\\equiv\\pi\(1\)\\delta\_\{\\langle\\theta,\\mathbf\{x\}\\rangle\}\+\\pi\(2\)\\delta\_\{\-\\langle\\theta,\\mathbf\{x\}\\rangle\}andD∗≡D⁡\(𝐱,θ∗,π∗\)≡π∗\(1\)δ⟨θ∗,𝐱⟩\+π∗\(2\)δ−⟨θ∗,𝐱⟩D^\{\*\}\\equiv D\(\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\)\\equiv\\pi^\{\*\}\(1\)\\delta\_\{\\langle\\theta^\{\*\},\\mathbf\{x\}\\rangle\}\+\\pi^\{\*\}\(2\)\\delta\_\{\-\\langle\\theta^\{\*\},\\mathbf\{x\}\\rangle\}\(see Lemmas[A\.6](https://arxiv.org/html/2609.05688#A1.Thmstyle6),[A\.7](https://arxiv.org/html/2609.05688#A1.Thmstyle7),[A\.13](https://arxiv.org/html/2609.05688#A1.Thmstyle13)in Appendix[A](https://arxiv.org/html/2609.05688#A1)\)\.

𝖪𝖫\(p\(yT,𝐱∣θT∗,π∗\)∥p\(yT,𝐱∣θT,π\)\)≤α¯T2\(1−α¯T\)𝔼𝐱W22\(D,D∗\),\\KL\(p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\}^\{\*\},\\pi^\{\*\}\)\\parallel p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\},\\pi\)\)\\leq\\frac\{\\bar\{\\alpha\}\_\{T\}\}\{2\(1\-\\bar\{\\alpha\}\_\{T\}\)\}\\mathbb\{E\}\_\{\\mathbf\{x\}\}W\_\{2\}^\{2\}\(D,D^\{\*\}\),\(13\)Similarly, the finite\-sample counterpartΔn\(ϑ\)\\Delta\_\{n\}\(\\vartheta\)decays in the rate ofα¯Tn2\(1−α¯Tn\)\\frac\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{2\(1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\}asn→∞n\\to\\infty\(see Lemma[A\.8](https://arxiv.org/html/2609.05688#A1.Thmstyle8)in Appendix[A](https://arxiv.org/html/2609.05688#A1)\)\. Ifln⁡1−α¯Tnα¯Tn−ln⁡n→∞\\ln\\frac\{1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\-\\ln n\\to\\infty, thenα¯Tn2\(1−α¯Tn\)=o⁡\(n−1\)\\frac\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{2\(1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\}=o\(n^\{\-1\}\)andΔn\(ϑ\)\\Delta\_\{n\}\(\\vartheta\)isoP\(n−1\)o\_\{P\}\(n^\{\-1\}\)whenϑ\\varthetais bounded\.

The next proposition complements that the ground truth parameters are the global minimizers of the terminal population KL divergence\.

###### 3\.1Global Minimizers of the Terminal KL Divergence\.

The retained\-signal level satisfies0<α¯T≤10<\\bar\{\\alpha\}\_\{T\}\\leq 1, the ground\-truth parameters satisfyθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}andν∗∈ℝ\\nu^\{\\ast\}\\in\\mathbb\{R\}, and𝐱\\mathbf\{x\}has a density that is positive almost everywhere onℝd\\mathbb\{R\}^\{d\}and finite second moments\. For the terminal population KL divergence𝖪𝖫T\(θ,ν\):=𝖪𝖫\(p\(yT,𝐱∣θT∗,π∗\)∥p\(yT,𝐱∣θT,π\)\),\\KL\_\{T\}\(\\theta,\\nu\):=\\KL\\left\(p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\},\\pi\)\\right\),the set of global minimizers isargmin\(θ,ν\)∈ℝd×ℝ⁡𝖪𝖫T\(θ,ν\)=\{\(θ∗,ν∗\),\(−θ∗,−ν∗\)\}\.\\argmin\_\{\(\\theta,\\nu\)\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}\}\\KL\_\{T\}\(\\theta,\\nu\)=\\\{\(\\theta^\{\\ast\},\\nu^\{\\ast\}\),\(\-\\theta^\{\\ast\},\-\\nu^\{\\ast\}\)\\\}\.

###### \.

𝖪𝖫T⁡\(θ,ν\)=0\\KL\_\{T\}\(\\theta,\\nu\)=0only ifp\(⋅∣𝐱;θT∗,π∗\)=p\(⋅∣𝐱;θT,π\)p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\)=p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\},\\pi\)almost everywhere by Pinsker’s inequality\([Cover and Thomas, 2006](https://arxiv.org/html/2609.05688#bib.bib22)\)𝖪𝖫T\(θ,ν\)≥2𝔼𝐱𝖳𝖵2\(p\(⋅∣𝐱;θT∗,π∗\),p\(⋅∣𝐱;θT,π\)\)\.\\KL\_\{T\}\(\\theta,\\nu\)\\geq 2\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\TV^\{2\}\\left\(p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\),p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\},\\pi\)\\right\)\.Therefore,\(θ,ν\)∈\{\(θ∗,ν∗\),\(−θ∗,−ν∗\)\}\(\\theta,\\nu\)\\in\\\{\(\\theta^\{\\ast\},\\nu^\{\\ast\}\),\(\-\\theta^\{\\ast\},\-\\nu^\{\\ast\}\)\\\}when𝖪𝖫T⁡\(θ,ν\)=0\\KL\_\{T\}\(\\theta,\\nu\)=0by the positive\-density assumption on𝐱\\mathbf\{x\}, while𝔼𝐱W22\(D,D∗\)=0\\mathbb\{E\}\_\{\\mathbf\{x\}\}W\_\{2\}^\{2\}\(D,D^\{\*\}\)=0has the same two solutions\. Thus the lower bound on the terminal KL divergence and the upper bound on the expected discrete Wasserstein cost have the same global zero set, proving the stated minimizer set\.

The next theorem adapts the similar argument of asymtotic normality in[Chewi et al\. \(2025a, Assumption3, Theorem 3\.1 on page 25\)](https://arxiv.org/html/2609.05688#bib.bib18)to MLR but without the regularity condition on Lipschitz continuity of the log\-probability density\. Proposition[3\.1](https://arxiv.org/html/2609.05688#S3.Thmstyle1)suggests that the sign identification is needed for this model\.

###### 3\.2Consistency and Asymptotic Normality of Score Matching Estimator\.

The ground\-truth parameters\(θ∗,ν∗\)∈ℝd×ℝ\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}satisfyθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}\. The deterministic terminal\-horizon sequence\{Tn\}n≥1\\\{T\_\{n\}\\\}\_\{n\\geq 1\}satisfiesln⁡1−α¯Tnα¯Tn−ln⁡n→∞\\ln\\frac\{1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\-\\ln n\\to\\infty, and the covariate𝐱\\mathbf\{x\}has a density that is positive almost everywhere and satisfies𝔼‖𝐱‖2<∞\\mathbb\{E\}\\\|\\mathbf\{x\}\\\|^\{2\}<\\infty\. The estimatorϑ^n,Tn𝖲𝖬≡\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\equiv\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)isconsistentup to a joint sign change\. For the alignment mapsn∈argmins∈\{\+1,−1\}⁡‖s⁡\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)−\(θ∗,ν∗\)‖s\_\{n\}\\in\\argmin\_\{s\\in\\\{\+1,\-1\\\}\}\\left\\\|s\\,\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\right\)\-\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\right\\\|, asn→∞n\\to\\infty,sn\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)→ℙ\(θ∗,ν∗\)\.s\_\{n\}\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\right\)\\xrightarrow\{\\mathbb\{P\}\}\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\.If𝔼⁡\[‖𝐱‖4\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{4\}\]<\\infty, then the aligned Score Matching estimator isasymptotically normal\. Asn→∞n\\to\\infty,n\[sn\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)−\(θ∗,ν∗\)\]→𝖽𝒩⁡\(𝟎,I\(θ∗,ν∗\)−1\),\\sqrt\{n\}\\left\[s\_\{n\}\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\right\)\-\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\right\]\\xrightarrow\{\\mathsf\{d\}\}\\mathcal\{N\}\(\\mathbf\{0\},I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)^\{\-1\}\),whereIIdenotes the Fisher information matrix\.
###### Bridge between Score Matching and MLE\.

Lemma[B\.2](https://arxiv.org/html/2609.05688#A2.Thmstyle2)shows that the empirical score matching objectiveℛ^n,Tn𝖲𝖬\(ϑ\)\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\)admitsϑ^n,Tn𝖲𝖬\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}with probability tending to one, and the global minimizerϑ^n,Tn𝖲𝖬\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}is attained asymptotically almost surely\. See the detailed proof of Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)in Appendix[B](https://arxiv.org/html/2609.05688#A2)\. Step 1\. Boundedness and excess\-risk control\.On the attainment event, the empirical bridge \([12](https://arxiv.org/html/2609.05688#S3.E12)\) bounds the empirical likelihood excess risk by the terminal discrepancy such thatℛ^n𝖬𝖫𝖤\(ϑ^n,Tn𝖲𝖬\)−ℛ^n𝖬𝖫𝖤\(ϑ∗\)=ℛ^n,Tn𝖲𝖬\(ϑ^n,Tn𝖲𝖬\)−ℛ^n,Tn𝖲𝖬\(ϑ∗\)\+Δn\(ϑ^n,Tn𝖲𝖬\)−Δn\(ϑ∗\)≤Δn\(ϑ^n,Tn𝖲𝖬\)\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\-\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta^\{\\ast\}\)=\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\-\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta^\{\\ast\}\)\+\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\-\\Delta\_\{n\}\(\\vartheta^\{\\ast\}\)\\leq\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\. The transition\-kernel KL bound and a coercive likelihood bound give‖θ^n,Tn𝖲𝖬‖=𝒪P\(1\)\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|=\\mathcal\{O\}\_\{P\}\(1\)\. The terminal\-scale condition then givesΔn\(ϑ^n,Tn𝖲𝖬\)=oP\(n−1\)\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)=o\_\{P\}\(n^\{\-1\}\)\. Step 2\. Consistency up to joint sign\.The population likelihood excess risk is a conditional KL divergence whose zero set is\{ϑ∗,−ϑ∗\}\\\{\\vartheta^\{\\ast\},\-\\vartheta^\{\\ast\}\\\}\. The attainment result, the empirical bridge, and Step 1 make the estimator anoP\(n−1\)o\_\{P\}\(n^\{\-1\}\)\-near maximizer of the empirical likelihood\. Theorem 5\.14 of[van der Vaart \(1998, p\. 48\)](https://arxiv.org/html/2609.05688#bib.bib15)then gives consistency up to joint sign and‖ϑ^n,Tn𝖲𝖬‖=𝒪P\(1\)\\\|\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|=\\mathcal\{O\}\_\{P\}\(1\)\. Step 3\. Asymptotic normality\.After joint sign alignment, the estimator is consistent and remains anoP\(n−1\)o\_\{P\}\(n^\{\-1\}\)\-near maximizer\. Under the fourth\-moment condition, the population log\-likelihood has a nonsingular quadratic expansion with Hessian−I⁡\(θ∗,ν∗\)\-I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\), and the local log\-likelihood admits a square\-integrable Lipschitz envelope\. These properties verify the hypotheses of Theorem 5\.23 of[van der Vaart \(1998, p\. 53\)](https://arxiv.org/html/2609.05688#bib.bib15), which gives the stated Gaussian limit with covarianceI\(θ∗,ν∗\)−1I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)^\{\-1\}\.

![Refer to caption](https://arxiv.org/html/2609.05688v1/figures/finite_sample_estimation.png)Figure 1:Illustration of the finite\-sample estimation results in Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)\.The experiment uses a two\-parameter Gaussian 2MLR instance, the theorem’s joint sign alignment, and a terminal sequence whose terminal\-scale condition is verified in Appendix[E](https://arxiv.org/html/2609.05688#A5)\. Panel \(a\) reports the median and 90% quantile of the error after joint sign alignment in the Fisher metric, together with 95% bootstrap intervals and the corresponding limiting quantiles ofχ22/n\\sqrt\{\\chi^\{2\}\_\{2\}/n\}\. Over the three largest sample sizes, the fitted slopes are−0\.471\-0\.471and−0\.526\-0\.526, close to the predicted−1/2\-1/2in the rate ofn−1/2n^\{\-1/2\}\. Panel \(b\) compares the squared,nn\-scaled Fisher error withχ22\\chi^\{2\}\_\{2\}at the smallest and largest sample sizes\. Coverage of the joint 95% Fisher ellipse ranges from 93\.5% to 96\.5%, equals 95\.0% at the largest sample size, and has a corresponding covariance discrepancy of 0\.070\. These diagnostics provide finite\-sample illustrations under the alignment and terminal\-scale conditions of Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)\. Appendix[E](https://arxiv.org/html/2609.05688#A5)provides additional calibration details\.
### 4Fixed\-Scale Score Matching: Cross\-Entropy and EM Decompositions

In this section, we establish the connection between the EM operatorsM,NM,Nfor regression parameters and the imbalance of mixing weights, and the cross entropyℋ⁡\(θt,ν\)≡𝔼∗\[−log⁡p⁡\(yt∣𝐱;θt,π\)\]=𝔼∗\[F⁡\(yt,μt,ν\)\]\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\\equiv\\mathbb\{E\}\_\{\\ast\}\[\-\\log p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\]=\\mathbb\{E\}\_\{\\ast\}\[F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]and EM–latent\-variance decompositions of the loss, together with gradient identities that retain operator\-Jacobian and posterior\-variance corrections\. We then use these identities to characterize the loss and its gradients at the low\- and high\-noise endpoints\. Appendix[C](https://arxiv.org/html/2609.05688#A3)contains the proofs of Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3), Propositions[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)–[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2)and[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6)–[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8), and Corollaries[4\.4](https://arxiv.org/html/2609.05688#S4.Thmstyle4)–[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5)\.

By applying the established Lemma[A\.17](https://arxiv.org/html/2609.05688#A1.Thmstyle17)for the derivatives of the negative log\-likelihoodF⁡\(yt,μt,ν\)≡−ln⁡p⁡\(yt∣𝐱;θt,π\)F\(y\_\{t\},\\mu\_\{t\},\\nu\)\\equiv\-\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\), we can derive the following proposition for the EM update operatorsM,NM,Nfor the regression parametersθt\\theta\_\{t\}and the imbalance parameterν\\nu, and further formulate the relation between the derivatives of the EM update operators and derivatives of the cross entropyℋ⁡\(θt,ν\)=𝔼∗\[F⁡\(yt,μt,ν\)\]\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]\.

###### 4\.1Expectation\-Maximization Rules\.

The population EM operators for the regression parameterθt\\theta\_\{t\}and imbalance of mixing weightstanh⁡ν\\tanh\\nuare

M⁡\(θt,ν\)=Σ−1𝔼∗\[𝐱yttanh⁡\(μtyt\+ν\)\],N⁡\(θt,ν\)=𝔼∗\[tanh⁡\(μtyt\+ν\)\]\.M\(\\theta\_\{t\},\\nu\)=\\Sigma^\{\-1\}\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\],\\qquad N\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\.In addition to defining the EM updates, these operators satisfy the scalar identity

⟨ΣM\(θt,ν\),θt⟩=𝔼∗\[μtyttanh⁡\(μtyt\+ν\)\]=𝔼∗\[yt2−yt∂ytF⁡\(yt,μt,ν\)\],\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]=\\mathbb\{E\}\_\{\\ast\}\[y\_\{t\}^\{2\}\-y\_\{t\}\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\],while differentiation with respect toν\\nuyieldsΣ∇νM\(θt,ν\)=∇θtN\(θt,ν\)=𝔼∗\[𝐱ytsech2⁡\(μtyt\+ν\)\]\.\\Sigma\\nabla\_\{\\nu\}M\(\\theta\_\{t\},\\nu\)=\\nabla\_\{\\theta\_\{t\}\}N\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}y\_\{t\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\.

###### 4\.2Cross\-Entropy and EM Update Rules\.

The cross\-entropyℋ⁡\(θt,ν\)\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)satisfies

∇θtℋ\(θt,ν\)=Σ⁡\(θt−M⁡\(θt,ν\)\),∇νℋ\(θt,ν\)=tanh⁡ν−N⁡\(θt,ν\)\.\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)=\\Sigma\\left\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\\right\),\\qquad\\nabla\_\{\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)=\\tanh\\nu\-N\(\\theta\_\{t\},\\nu\)\.

From the established Proposition[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)and Proposition[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2), we can derive the following theorem for the information theoretical decomposition of the score matching loss, and show its connection with the EM update operatorsM,NM,Nand derivatives of the cross entropyℋ⁡\(θt,ν\)\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\.

###### 4\.3Information Decomposition of the Score Matching Loss\.

The expected squared activation is𝒜⁡\(θt,ν\):=12𝔼∗\[μt2tanh2⁡\(μtyt\+ν\)\]\\mathcal\{A\}\(\\theta\_\{t\},\\nu\):=\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}^\{2\}\\tanh^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]and its ground\-truth counterpart is𝒜⁡\(θt∗,ν∗\):=12𝔼∗\[\(μt∗\)2tanh2⁡\(μt∗yt\+ν∗\)\]\\mathcal\{A\}\(\\theta\_\{t\}^\{\*\},\\nu^\{\*\}\):=\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\ast\}\[\(\\mu\_\{t\}^\{\*\}\)^\{2\}\\tanh^\{2\}\(\\mu\_\{t\}^\{\*\}y\_\{t\}\+\\nu^\{\\ast\}\)\]\. With these definitions, the score matching loss satisfiesα¯tℒt\(θ,ν\)=⟨∇θtℋ\(θt,ν\),θt⟩−𝒜⁡\(θt,ν\)\+𝒜⁡\(θt∗,ν∗\)\.\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\langle\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\-\\mathcal\{A\}\(\\theta\_\{t\},\\nu\)\+\\mathcal\{A\}\(\\theta\_\{t\}^\{\*\},\\nu^\{\*\}\)\.Defining the latent varianceV⁡\(θt,ν\)≡12𝔼∗\[μt2sech2⁡\(μtyt\+ν\)\]V\(\\theta\_\{t\},\\nu\)\\equiv\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]gives an equivalent representation\.α¯tℒt\(θ,ν\)=‖θt‖Σ2/2−⟨ΣM\(θt,ν\),θt⟩\+‖θt∗‖Σ2/2\+V⁡\(θt,ν\)−V⁡\(θt∗,ν∗\)\.\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\\|\\theta\_\{t\}\\\|\_\{\\Sigma\}^\{2\}/2\-\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\+\\\|\\theta\_\{t\}^\{\\ast\}\\\|\_\{\\Sigma\}^\{2\}/2\+V\(\\theta\_\{t\},\\nu\)\-V\(\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)\.
###### \.

Lemmas[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)and[A\.3](https://arxiv.org/html/2609.05688#A1.Thmstyle3)express both scores as negativeyty\_\{t\}\-derivatives of the negative log\-likelihoodF⁡\(yt,μt,ν\)≡−ln⁡p⁡\(yt∣𝐱;θt,π\)F\(y\_\{t\},\\mu\_\{t\},\\nu\)\\equiv\-\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\. Expanding their squared discrepancy and integrating the mixed and squared ground\-truth\-score terms by parts with vanishing boundary terms yields an identity involving the first two derivatives\. The derivative formulas and the conditional second moment ofyty\_\{t\}then reduce this identity to quadratic andsech2\\sech^\{2\}terms\. Proposition[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)identifies the remaining mixed moment and gives the representation with EM operators and latent variance\. Proposition[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2), together withsech2⁡\(u\)=1−tanh2⁡\(u\)\\sech^\{2\}\(u\)=1\-\\tanh^\{2\}\(u\), converts it into the representation with the cross entropyℋ⁡\(θt,ν\)\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)and expected squared activation\.

###### 4\.4Gradients of the Score Matching Loss via Cross\-Entropy and Expected Squared Activation\.

The gradients of the score matching lossα¯tℒt\(θ,ν\)\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)with respect to the regression parameterθt\\theta\_\{t\}and the mixing imbalanceν\\nuare as follows\.

∇θt\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\theta\_\{t\}\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=∇θtℋ\(θt,ν\)\+∇θt2ℋ\(θt,ν\)θt−∇θt𝒜\(θt,ν\),\\displaystyle=\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\+\\nabla^\{2\}\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\\theta\_\{t\}\-\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{A\}\(\\theta\_\{t\},\\nu\),∇ν\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\nu\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=⟨∇θtν2ℋ\(θt,ν\),θt⟩−∇ν𝒜\(θt,ν\)\.\\displaystyle=\\langle\\nabla^\{2\}\_\{\\theta\_\{t\}\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\-\\nabla\_\{\\nu\}\\mathcal\{A\}\(\\theta\_\{t\},\\nu\)\.

###### 4\.5Gradients of the Score Matching Loss via EM Operators and Latent Variance\.

The gradients of the score matching lossα¯tℒt\(θ,ν\)\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)with respect toθt\\theta\_\{t\}andν\\nuare as follows\.

∇θt\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\theta\_\{t\}\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=Σ⁡\(θt−M⁡\(θt,ν\)\)−\(∇θtM\(θt,ν\)\)⊤Σθt\+∇θtV\(θt,ν\),\\displaystyle=\\Sigma\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\)\-\\big\(\\nabla\_\{\\theta\_\{t\}\}M\(\\theta\_\{t\},\\nu\)\\big\)^\{\\top\}\\Sigma\\theta\_\{t\}\+\\nabla\_\{\\theta\_\{t\}\}V\(\\theta\_\{t\},\\nu\),∇ν\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\nu\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=−⟨∇θtN\(θt,ν\),θt⟩\+∇νV\(θt,ν\)\.\\displaystyle=\-\\langle\\nabla\_\{\\theta\_\{t\}\}N\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\+\\nabla\_\{\\nu\}V\(\\theta\_\{t\},\\nu\)\.

The normalized EM operators areM¯t\(θ,ν\):=M⁡\(θt,ν\)/α¯t,N¯t\(θ,ν\):=\(N⁡\(θt,ν\)−\(1−α¯t\)tanh⁡ν\)/α¯t\.\\bar\{M\}\_\{t\}\(\\theta,\\nu\):=M\(\\theta\_\{t\},\\nu\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},\\quad\\bar\{N\}\_\{t\}\(\\theta,\\nu\):=\(N\(\\theta\_\{t\},\\nu\)\-\(1\-\\bar\{\\alpha\}\_\{t\}\)\\tanh\\nu\)/\\bar\{\\alpha\}\_\{t\}\.These definitions implyθ−M¯t\(θ,ν\)=\(θt−M⁡\(θt,ν\)\)/α¯t\\theta\-\\bar\{M\}\_\{t\}\(\\theta,\\nu\)=\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},tanh⁡ν−N¯t\(θ,ν\)=\(tanh⁡ν−N⁡\(θt,ν\)\)/α¯t\\tanh\\nu\-\\bar\{N\}\_\{t\}\(\\theta,\\nu\)=\(\\tanh\\nu\-N\(\\theta\_\{t\},\\nu\)\)/\\bar\{\\alpha\}\_\{t\}, andΣ∇νM¯t\(θ,ν\)=∇θN¯t\(θ,ν\)\\Sigma\\nabla\_\{\\nu\}\\bar\{M\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\theta\}\\bar\{N\}\_\{t\}\(\\theta,\\nu\), which yield the following gradient identities withVt\(θ,ν\):=V⁡\(θt,ν\)/α¯tV\_\{t\}\(\\theta,\\nu\):=V\(\\theta\_\{t\},\\nu\)/\\bar\{\\alpha\}\_\{t\}\.

###### 4\.6Gradients of the score matching Loss\.

The gradients of the score matching lossℒt\(θ,ν\)\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)are as follows:

∇θℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=Σ⁡\(θ−M¯t\(θ,ν\)\)−\(∇θM¯t\(θ,ν\)\)⊤Σθ\+∇θVt\(θ,ν\),\\displaystyle=\\Sigma\(\\theta\-\\bar\{M\}\_\{t\}\(\\theta,\\nu\)\)\-\\left\(\\nabla\_\{\\theta\}\\bar\{M\}\_\{t\}\(\\theta,\\nu\)\\right\)^\{\\top\}\\Sigma\\theta\+\\nabla\_\{\\theta\}V\_\{t\}\(\\theta,\\nu\),∇νℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=−⟨∇θN¯t\(θ,ν\),θ⟩\+∇νVt\(θ,ν\)\.\\displaystyle=\-\\langle\\nabla\_\{\\theta\}\\bar\{N\}\_\{t\}\(\\theta,\\nu\),\\theta\\rangle\+\\nabla\_\{\\nu\}V\_\{t\}\(\\theta,\\nu\)\.

###### 4\.7Limit at Low Noise and Expectation\-Maximization\.

If𝔼⁡\[‖𝐱‖4\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{4\}\]<\\infty, the gradients of the score matching lossℒt\(θ,ν\)\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)satisfy the following expansions asα¯t→1\\bar\{\\alpha\}\_\{t\}\\to 1\.

∇θℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=Σ⁡\(θ−M⁡\(θ,ν\)\)−\(∇θM\(θ,ν\)\)⊤Σθ\+∇θV\(θ,ν\)\+𝒪⁡\(1−α¯t\),\\displaystyle=\\Sigma\(\\theta\-M\(\\theta,\\nu\)\)\-\\left\(\\nabla\_\{\\theta\}M\(\\theta,\\nu\)\\right\)^\{\\top\}\\Sigma\\theta\+\\nabla\_\{\\theta\}V\(\\theta,\\nu\)\+\\mathcal\{O\}\(1\-\\bar\{\\alpha\}\_\{t\}\),∇νℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=−⟨∇θN\(θ,ν\),θ⟩\+∇νV\(θ,ν\)\+𝒪⁡\(1−α¯t\)\.\\displaystyle=\-\\langle\\nabla\_\{\\theta\}N\(\\theta,\\nu\),\\theta\\rangle\+\\nabla\_\{\\nu\}V\(\\theta,\\nu\)\+\\mathcal\{O\}\(1\-\\bar\{\\alpha\}\_\{t\}\)\.

###### 4\.8Effective Coefficient Limit at High Noise\.

If𝔼⁡\[‖𝐱‖2\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]<\\infty, the effective coefficientc∗:=θ∗tanhν∗c^\{\\ast\}:=\\theta^\{\\ast\}\\tanh\\nu^\{\\ast\}determines the limiting objective throughlimα¯t→0ℒt\(θ,ν\)=12‖θtanh⁡ν−c∗‖Σ2\.\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\frac\{1\}\{2\}\\\|\\theta\\tanh\\nu\-c^\{\\ast\}\\\|\_\{\\Sigma\}^\{2\}\.The condition𝔼⁡\[‖𝐱‖4\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{4\}\]<\\inftysuffices for the𝒪⁡\(α¯t\)\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\)gradient expansions asα¯t→0\\bar\{\\alpha\}\_\{t\}\\to 0

∇θℒt\(θ,ν\)=∇θlimα¯t→0ℒt\(θ,ν\)\+𝒪\(α¯t\),∇νℒt\(θ,ν\)=∇νlimα¯t→0ℒt\(θ,ν\)\+𝒪\(α¯t\)\.\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\theta\}\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\+\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\),\\qquad\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\nu\}\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\+\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\)\.

Figure 2:First\-order scaling of the endpoint gradient remainders in Propositions[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)and[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)\.At one fixed model pair, the panels plot componentwise remainders for the complete low\-noise expression in Proposition[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)and the effective high\-noise coefficient expression in Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)\. The fitted log–log slopes track the guides with unit slope, consistent with the respective𝒪⁡\(1−α¯t\)\\mathcal\{O\}\(1\-\\bar\{\\alpha\}\_\{t\}\)and𝒪⁡\(α¯t\)\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\)bounds\. The panels report one selected instance, whose settings and fitted values appear in Appendix[E](https://arxiv.org/html/2609.05688#A5)\.
### 5High\-Noise Gradient Dynamics and Score Matching Blindness

Building on the endpoint loss and gradient limits in Section[4](https://arxiv.org/html/2609.05688#S4), we further study the optimization dynamics and parameter\-information consequences in this section\. We first analyze fixed\-step gradient descent on the high\-noise limiting objective in Main Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)and then establish pointwise score matching blindness along fixed high\-SNR rays in Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)\. Complete arguments and details of proofs are provided in Appendix[D](https://arxiv.org/html/2609.05688#A4)\.

#### 5\.1High\-Noise Gradient Dynamics

As shown in Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8), the high\-noise limiting objective is given byℒ⁡\(θ,ν\):=limα¯t→0ℒt\(θ,ν\)\\mathcal\{L\}\(\\theta,\\nu\):=\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\. We consider the optimization dynamics of gradient descent with a fixed step sizeη\\eta:ϑk\+1←ϑk−η∇ℒ\(ϑk\),\\vartheta\_\{k\+1\}\\leftarrow\\vartheta\_\{k\}\-\\eta\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\),for the iterateϑk=\(θk,νk\)\\vartheta\_\{k\}=\(\\theta\_\{k\},\\nu\_\{k\}\)of parameters at time stepkk\. The following informal Main Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)demonstrates the convergence rates of the optimization dynamics\.

###### 5\.1Informal; Global Dynamics asα¯t→0\\bar\{\\alpha\}\_\{t\}\\to 0\.

On the high\-noise limiting lossℒ⁡\(θ,ν\)=12‖θtanh⁡ν−c∗‖2\\mathcal\{L\}\(\\theta,\\nu\)=\\frac\{1\}\{2\}\\\|\\theta\\tanh\\nu\-c^\{\\ast\}\\\|^\{2\}withc∗:=θ∗tanhν∗c^\{\\ast\}:=\\theta^\{\\ast\}\\tanh\\nu^\{\\ast\}, isotropic covariatesΣ=Id\\Sigma=I\_\{d\}, fixed\-step gradient descent with an admissible initialization has nonincreasing loss and obeys the following rates\.
\(c∗=𝟎c^\{\\ast\}=\\mathbf\{0\}\.\) The loss satisfiesℒ⁡\(θK,νK\)=𝒪⁡\(K−2\)\\mathcal\{L\}\(\\theta\_\{K\},\\nu\_\{K\}\)=\\mathcal\{O\}\(K^\{\-2\}\)\.
\(c∗≠𝟎c^\{\\ast\}\\neq\\mathbf\{0\}, generic initializations\.\) Every admissible initialization outside the exceptional saddle basin has geometric loss decay after a finite transient and reaches anyϵ\>0\\epsilon\>0in𝒪⁡\(log⁡\(1/ϵ\)\)\\mathcal\{O\}\(\\log\(1/\\epsilon\)\)iterations\.
\(c∗≠𝟎c^\{\\ast\}\\neq\\mathbf\{0\}, exceptional initializations\.\) The exceptional saddle basin consists of the admissible initializations whose trajectories converge to the saddle manifold\{\(θ,0\):⟨θ,c∗⟩=0\}\\\{\(\\theta,0\):\\langle\\theta,c^\{\\ast\}\\rangle=0\\\}\. This basin has\(d\+1\)\(d\+1\)\-dimensional Lebesgue measure zero, and every such trajectory has limiting loss‖c∗‖2/2\\\|c^\{\\ast\}\\\|^\{2\}/2\.###### \.

We establish the proof by showing that the coupled descent bounds keep the iterates in the admissible region\. Whenc∗=𝟎c^\{\\ast\}=\\mathbf\{0\}, a gradient bound of orderℒ3/2\(ϑk\)\\mathcal\{L\}^\{3/2\}\(\\vartheta\_\{k\}\)gives theK−2K^\{\-2\}upper bound\. Whenc∗≠𝟎c^\{\\ast\}\\neq\\mathbf\{0\}, the sign recurrences give geometric decay after the signs agree\. The local stable set of the saddle has lower dimension, and inverse images of null sets under the update map remain null\. Hence the admissible saddle basin has zero\(d\+1\)\(d\+1\)\-dimensional Lebesgue measure\. See Theorem[D\.1](https://arxiv.org/html/2609.05688#A4.Thmstyle1)in Appendix[D](https://arxiv.org/html/2609.05688#A4)for the precise admissibility condition for the step size, explicit constants in convergence rates, and the detailed proofs\.

![Refer to caption](https://arxiv.org/html/2609.05688v1/figures/high_noise_dynamics.png)Figure 3:Optimization Dynamics of Gradient Descent on the High\-Noise Objective in Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)\.The gradient descent update with a fixed step size is iterated from initializations satisfying the admissibility condition in Theorem[D\.1](https://arxiv.org/html/2609.05688#A4.Thmstyle1), and every displayed path preserves that condition and has nonincreasing loss\. \(a\) For the degenerate target, the computed loss remains below theK−2K^\{\-2\}upper bound in Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1), and its fitted log–log tail slope is−2−1\.50×10−2\-2\-1\.50\\times 10^\{\-2\}, tracking the anchoredK−2K^\{\-2\}guide over the displayed tail\. \(b\) Forc∗≠𝟎c^\{\\ast\}\\neq\\mathbf\{0\}, the three displayed generic initializations lie outside the exceptional saddle basin and show initialization\-dependent transients followed by geometric loss decay, consistent with the𝒪⁡\(log⁡\(1/ϵ\)\)\\mathcal\{O\}\(\\log\(1/\\epsilon\)\)conclusion\. \(c\) The phase portrait in\(⟨θK,c∗⟩/‖c∗‖,νK\)\(\\langle\\theta\_\{K\},c^\{\\ast\}\\rangle/\\\|c^\{\\ast\}\\\|,\\nu\_\{K\}\)coordinates uses open circles and filled diamonds to mark the initial and final iterates\. The dotted curves are the two branches of the global\-minimum set\{\(θ,ν\):θtanh⁡ν=c∗\}\\\{\(\\theta,\\nu\):\\theta\\tanh\\nu=c^\{\\ast\}\\\}\. By contrast, the saddle set\{\(θ,0\):⟨θ,c∗⟩=0\}\\\{\(\\theta,0\):\\langle\\theta,c^\{\\ast\}\\rangle=0\\\}projects to the single point\(0,0\)\(0,0\)in these coordinates and is marked by the black cross\. The observed degenerate slope diagnoses finite\-window agreement with theK−2K^\{\-2\}upper bound in Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)\. Appendix[E](https://arxiv.org/html/2609.05688#A5)provides the configurations for generating the figures\.
#### 5\.2Score Matching Blindness at High SNR

###### 5\.2Pointwise Score Matching Blindness Along a Fixed Ray at High SNR\.

The covariates satisfy𝔼⁡\[‖𝐱‖2\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]<\\infty\. The scale of the diffusion noise levelα¯t∈\(0,1\]\\bar\{\\alpha\}\_\{t\}\\in\(0,1\]and finite model pair\(θ,ν\)∈ℝd×ℝ\(\\theta,\\nu\)\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}are fixed\. Letθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}vary with‖θ∗‖→∞\\\|\\theta^\{\\ast\}\\\|\\to\\inftyalong a fixed ray, while the true mixing weights and all other model and data\-generating quantities remain fixed\. Whenθ≠𝟎\\theta\\neq\\mathbf\{0\}, we also impose the following projection condition\.

ℙ\(\{⟨θ,𝐱⟩=0\}∪\{⟨θ∗,𝐱⟩=0\}\)=0\.\\mathbb\{P\}\\\!\\left\(\\\{\\langle\\theta,\\mathbf\{x\}\\rangle=0\\\}\\cup\\\{\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle=0\\\}\\right\)=0\.The expectations definingℒt\\mathcal\{L\}\_\{t\}andVtV\_\{t\}are taken under the corresponding varying ground truth\. At the specified fixed model pair and fixed scale of the diffusion noise level, the following pointwise limits hold\.

lim‖θ∗‖→∞\|∇νℒt\(θ,ν\)\|=0,lim‖θ∗‖→∞Vt\(θ,ν\)=0\.\\lim\_\{\\\|\\theta^\{\\ast\}\\\|\\to\\infty\}\|\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\|=0,\\qquad\\lim\_\{\\\|\\theta^\{\\ast\}\\\|\\to\\infty\}V\_\{t\}\(\\theta,\\nu\)=0\.

![Refer to caption](https://arxiv.org/html/2609.05688v1/high_snr_blindness.png)Figure 4:Score Matching Blindness at High SNR in Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)\.The displayed instance usesd=1d=1,X∼𝒩⁡\(0,1\)X\\sim\\mathcal\{N\}\(0,1\),α¯t=12\\bar\{\\alpha\}\_\{t\}=\\tfrac\{1\}\{2\}, the fixed candidate\(θ,ν\)=\(0\.80,−0\.35\)\(\\theta,\\nu\)=\(0\.80,\-0\.35\), andν∗=0\.55\\nu^\{\\ast\}=0\.55, whileθ∗\>0\\theta^\{\\ast\}\>0grows along the positive ray\. This setting satisfies Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)\. \(a\)–\(b\) Following[Wenliang and Kanagawa \(2020\)](https://arxiv.org/html/2609.05688#bib.bib55), samples from the varying truths are overlaid on the fixed\-candidate sensitivity\|∂νsθt,ν\(yt,x\)\|=\|μt\|sech2⁡\(μtyt\+ν\)\|\\partial\_\{\\nu\}s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},x\)\|=\|\\mu\_\{t\}\|\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)on a log scale\. The normalized responses concentrate onyt/‖θ∗‖≃±α¯txy\_\{t\}/\\\|\\theta^\{\\ast\}\\\|\\simeq\\pm\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\,x, and the appreciable sensitivity concentrates near the ray intersection\. \(c\) Quadrature shows that\|∇νℒt\|\|\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\|and the latent variance termVt=V⁡\(θt,ν\)/α¯tV\_\{t\}=V\(\\theta\_\{t\},\\nu\)/\\bar\{\\alpha\}\_\{t\}decrease along the fixed ray, providing pointwise evidence at one fixed model and scale\. Appendix Figure[6](https://arxiv.org/html/2609.05688#A5.F6)provides fixed\-scale and cross\-entropy controls together with an auxiliary moving\-candidate diagnostic for a separate setting\. At high noise, Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)gives12‖θtanh⁡ν−c∗‖Σ2\\frac\{1\}\{2\}\\\|\\theta\\tanh\\nu\-c^\{\\ast\}\\\|\_\{\\Sigma\}^\{2\}, whose directional signal vanishes at balance\. At low noise, Proposition[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)retains operator\-Jacobian and posterior\-variance corrections beyond the EM residuals\.

### 6Numerical Experiments

In the section of numerical experiments, we first assess the finite\-sample behavior of the path\-integrated estimator and then check the two endpoint gradient expansions, the high\-noise gradient dynamics, and the pointwise high\-SNR blindness result\. Appendix[E](https://arxiv.org/html/2609.05688#A5)provides the complete configurations of numerical experiments for generating the figures, additional numerical checks, and an auxiliary finite\-noise spectral diagnostic\.

Under the terminal schedule of Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)and after applying its joint sign alignment, Figure[1](https://arxiv.org/html/2609.05688#S3.F1)illustrates the predicted parametric scaling and joint calibration of the integrated estimator under the assumptions stated in Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)\.

Figure[2](https://arxiv.org/html/2609.05688#S4.F2)checks the componentwise remainder orders in Propositions[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)and[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)at the fixed model pair\. Panel \(a\) uses the complete low\-noise expression, including the EM operator and latent variance, while panel \(b\) uses the high\-noise effective\-coefficient expression\.

For the displayed admissible initializations, Figure[3](https://arxiv.org/html/2609.05688#S5.F3)illustrates the polynomial upper bound, geometric decay, and exceptional saddle in Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)\.

For a one\-dimensional specialization of Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2), Figure[4](https://arxiv.org/html/2609.05688#S5.F4)shows the imbalance gradient and posterior\-variance term decreasing along the fixed\-candidate, fixed\-scale ray, thereby illustrating the proposition’s pointwise conclusion at one fixed candidate and scale\.

### 7Conclusions

This paper studies variance\-preserving diffusion of the response in two\-component mixed linear regression with unknown mixing weights and distinguishes the statistical information accumulated by integrating Score Matching along the diffusion path from the loss geometry and optimization signal available at a fixed scale of the diffusion noise level\. For the first question posed in the introduction, Proposition[3\.1](https://arxiv.org/html/2609.05688#S3.Thmstyle1)shows that, under the stated identifiability conditions, the terminal population KL divergence at every positive retained\-signal level has the ground truth as its global minimizers\. Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)proves that the path\-integrated score matching estimator recovers the regression parameters and mixing weights up to their joint\-sign symmetry\. The same Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)establishes asymptotic normality with inverse\-Fisher covariance, matching the limiting distribution of the maximum\-likelihood estimator\.

For the second question, Propositions[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)and[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2)identify the diffused MLR Expectation–Maximization \(EM\) operators and express the cross\-entropy gradients as EM residuals, while Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3)gives equivalent cross\-entropy and EM\-related decompositions of the fixed\-scale score matching loss\. Corollaries[4\.4](https://arxiv.org/html/2609.05688#S4.Thmstyle4)–[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5)and Proposition[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6)consequently show the decompositions of the fixed\-scale score matching gradient as a combination of EM residuals with operator\-Jacobian and latent variance terms\. The endpoint and signal\-strength results explain how noise changes both the optimization dynamics and the parameter information retained by score matching At low noise, Equation \([15](https://arxiv.org/html/2609.05688#S4.E15)\) identifies the limiting loss, and Proposition[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)shows that the gradient retains the EM\-related correction structure\. At high noise, Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)shows that the limiting loss depends on the regression vector and mixing imbalance only through their effective coefficient\. Under isotropic covariance, Main Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)proves an𝒪⁡\(K−2\)\\mathcal\{O\}\(K^\{\-2\}\)loss bound when this effective target is zero and eventual geometric decay for generic initializations when it is nonzero, with only a measure\-zero exceptional basin attracted to the saddle manifold\. Finally, Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)proves that, along a fixed high\-SNR ground\-truth ray and at each covered fixed candidate and diffusion scale, the imbalance gradient and latent\-variance term converge pointwise to zero, eliminating the local first\-order signal for changing the candidate mixing weights\.

### References

- S\. Balakrishnan, M\. J\. Wainwright, and B\. YuStatistical guarantees for the EM algorithm: from population to sample\-based analysis\.The Annals of Statistics45\(1\),pp\. 77–120\.External Links:[Document](https://dx.doi.org/10.1214/16-AOS1435),[Link](https://doi.org/10.1214/16-AOS1435)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Batzoliset al\.\(2021\)G\. Batzolis, J\. Stanczuk, C\. Schönlieb, and C\. EtmannConditional image generation with score\-based diffusion models\.External Links:2111\.13606,[Document](https://dx.doi.org/10.48550/arXiv.2111.13606),[Link](https://arxiv.org/abs/2111.13606)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- Bishop \(1994\)C\. M\. BishopMixture density networks\.Technical reportTechnical ReportNCRG/94/004,Aston University,Birmingham, UK\.External Links:[Link](https://publications.aston.ac.uk/id/eprint/373/)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- Chaganty and Liang \(2013\)A\. T\. Chaganty and P\. LiangSpectral experts for estimating mixtures of linear regressions\.InProceedings of the 30th International Conference on Machine Learning,S\. Dasgupta and D\. McAllester \(Eds\.\),Proceedings of Machine Learning Research, Vol\.28,Atlanta, Georgia, USA,pp\. 1040–1048\.External Links:[Link](https://proceedings.mlr.press/v28/tejasvichaganty13.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2023\)S\. Chen, S\. Chewi, H\. Lee, Y\. Li, J\. Lu, and A\. SalimThe probability flow ODE is provably fast\.Advances in Neural Information Processing Systems36,pp\. 68552–68575\.External Links:[Link](https://openreview.net/forum?id=KD6MFeWSAd)Cited by:[§2\.1](https://arxiv.org/html/2609.05688#S2.SS1.p1.3)\.
- Chenet al\.\(2025\)S\. Chen, V\. Kontonis, and K\. ShahLearning general Gaussian mixtures with efficient score matching\.InProceedings of Thirty Eighth Conference on Learning Theory,N\. Haghtalab and A\. Moitra \(Eds\.\),Proceedings of Machine Learning Research, Vol\.291,pp\. 1029–1090\.External Links:2404\.18893,[Link](https://proceedings.mlr.press/v291/chen25e.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2018\)Y\. Chen, X\. Yi, and C\. CaramanisConvex and nonconvex formulations for mixed regression with two components: minimax optimal rates\.IEEE Transactions on Information Theory64\(3\),pp\. 1738–1766\.External Links:[Document](https://dx.doi.org/10.1109/TIT.2017.2773474),[Link](https://doi.org/10.1109/TIT.2017.2773474)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Chewiet al\.\(2025a\)S\. Chewi, A\. Kalavasis, A\. Mehrotra, and O\. MontasserDDPM score matching and distribution learning\.Note:Extended abstract appeared at COLT 2026External Links:2504\.05161,[Document](https://dx.doi.org/10.48550/arXiv.2504.05161),[Link](https://arxiv.org/abs/2504.05161)Cited by:[Appendix B](https://arxiv.org/html/2609.05688#A2.p2.1.1),[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.05688#S3.p3.1)\.
- Chewiet al\.\(2025b\)S\. Chewi, A\. Kalavasis, A\. Mehrotra, and O\. MontasserDDPM score matching is asymptotically efficient\.InICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy,Note:Oral presentationExternal Links:[Link](https://iclr.cc/virtual/2025/35338)Cited by:[§1\.1](https://arxiv.org/html/2609.05688#S1.SS1.p1.1)\.
- Cover and Thomas \(2006\)T\. M\. Cover and J\. A\. ThomasElements of information theory\.2nd edition,John Wiley & Sons\.Cited by:[Appendix A](https://arxiv.org/html/2609.05688#A1.p14.2.1),[Appendix A](https://arxiv.org/html/2609.05688#A1.p6.6.1),[§3](https://arxiv.org/html/2609.05688#S3.Thmstylex1.p1.1)\.
- De Veaux \(1989\)R\. D\. De VeauxMixtures of linear regressions\.Computational Statistics & Data Analysis8\(3\),pp\. 227–245\.External Links:[Document](https://dx.doi.org/10.1016/0167-9473%2889%2990043-1),[Link](https://doi.org/10.1016/0167-9473(89)90043-1)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- Dempsteret al\.\(1977\)A\. P\. Dempster, N\. M\. Laird, and D\. B\. RubinMaximum likelihood from incomplete data via the EM algorithm\.Journal of the Royal Statistical Society: Series B \(Methodological\)39\(1\),pp\. 1–22\.External Links:[Document](https://dx.doi.org/10.1111/j.2517-6161.1977.tb01600.x),[Link](https://academic.oup.com/jrsssb/article/39/1/1/7027539)Cited by:[§1\.1](https://arxiv.org/html/2609.05688#S1.SS1.p1.1)\.
- Dennehyet al\.\(2026\)A\. Dennehy, R\. Muthukumar, R\. Willett, and N\. ChandramoorthyDiffusion models recover accurate mixture weights despite score function insensitivity\.External Links:2607\.15485,[Document](https://dx.doi.org/10.48550/arXiv.2607.15485),[Link](https://arxiv.org/abs/2607.15485)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px2.p1.1)\.
- Durrett \(2019\)R\. DurrettProbability: theory and examples\.5th edition,Cambridge Series in Statistical and Probabilistic Mathematics, Vol\.49,Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/9781108591034)Cited by:[Appendix A](https://arxiv.org/html/2609.05688#A1.p10.3.1),[Appendix B](https://arxiv.org/html/2609.05688#A2.p13.2.1),[Appendix B](https://arxiv.org/html/2609.05688#A2.p8.6.1),[Appendix B](https://arxiv.org/html/2609.05688#A2.p9.8),[Appendix C](https://arxiv.org/html/2609.05688#A3.p3.4.1),[Appendix C](https://arxiv.org/html/2609.05688#A3.p4.2.1)\.
- Evans and Gariepy \(2015\)L\. C\. Evans and R\. F\. GariepyMeasure theory and fine properties of functions\.Revised edition,CRC Press\.Cited by:[Appendix D](https://arxiv.org/html/2609.05688#A4.p29.1.1)\.
- Fanget al\.\(1990\)K\. Fang, S\. Kotz, and K\. W\. NgSymmetric multivariate and related distributions\.Monographs on Statistics and Applied Probability, Vol\.36,Chapman & Hall,London\.External Links:ISBN 0412314304,[Link](https://hub.hku.hk/handle/10722/130548)Cited by:[Appendix A](https://arxiv.org/html/2609.05688#A1.p22.1.1)\.
- Folland \(1999\)G\. B\. FollandReal analysis: modern techniques and their applications\.2nd edition,John Wiley & Sons\.External Links:ISBN 9780471317166,[Link](https://www.wiley.com/en-us/Real+Analysis/%3A+Modern+Techniques+and+Their+Applications/%2C+2nd+Edition-p-9780471317166)Cited by:[Appendix C](https://arxiv.org/html/2609.05688#A3.p12.3.1)\.
- Fuet al\.\(2024\)H\. Fu, Z\. Yang, M\. Wang, and M\. ChenUnveil conditional diffusion models with classifier\-free guidance: a sharp statistical theory\.External Links:2403\.11968,[Document](https://dx.doi.org/10.48550/arXiv.2403.11968),[Link](https://arxiv.org/abs/2403.11968)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- Gaoet al\.\(2025\)X\. Gao, H\. M\. Nguyen, and L\. ZhuWasserstein convergence guarantees for a general class of score\-based generative models\.Journal of Machine Learning Research26\(43\),pp\. 1–54\.External Links:[Link](https://www.jmlr.org/papers/v26/24-0902.html)Cited by:[§2\.1](https://arxiv.org/html/2609.05688#S2.SS1.p1.3)\.
- Gao and Zhu \(2025\)X\. Gao and L\. ZhuConvergence analysis for general probability flow ODEs of diffusion models in Wasserstein distances\.InProceedings of The 28th International Conference on Artificial Intelligence and Statistics,Y\. Li, S\. Mandt, S\. Agrawal, and E\. Khan \(Eds\.\),Proceedings of Machine Learning Research, Vol\.258,pp\. 1009–1017\.External Links:[Link](https://proceedings.mlr.press/v258/gao25c.html)Cited by:[§2\.1](https://arxiv.org/html/2609.05688#S2.SS1.p1.3)\.
- Gatmiryet al\.\(2025\)K\. Gatmiry, J\. Kelner, and H\. LeeLearning mixtures of Gaussians using diffusion models\.InProceedings of Thirty Eighth Conference on Learning Theory,N\. Haghtalab and A\. Moitra \(Eds\.\),Proceedings of Machine Learning Research, Vol\.291,pp\. 2403–2456\.External Links:2404\.18869,[Link](https://proceedings.mlr.press/v291/gatmiry25b.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Hanet al\.\(2022\)X\. Han, H\. Zheng, and M\. ZhouCARD: classification and regression diffusion models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 18100–18115\.External Links:[Document](https://dx.doi.org/10.52202/068431-1316),[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/72dad95a24fae750f8ab1cb3dab5e58d-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- Hennig \(2000\)C\. HennigIdentifiability of models for clusterwise linear regression\.Journal of Classification17\(2\),pp\. 273–296\.External Links:[Document](https://dx.doi.org/10.1007/s003570000022),[Link](https://doi.org/10.1007/s003570000022)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Hoet al\.\(2020\)J\. Ho, A\. Jain, and P\. AbbeelDenoising diffusion probabilistic models\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 6840–6851\.External Links:2006\.11239,[Link](https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.05688#S2.SS1.p1.3)\.
- Horn and Johnson \(2012\)R\. A\. Horn and C\. R\. JohnsonMatrix analysis\.2nd edition,Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/CBO9781139020411)Cited by:[Appendix B](https://arxiv.org/html/2609.05688#A2.p8.6.1)\.
- Huanget al\.\(2021\)C\. Huang, J\. H\. Lim, and A\. C\. CourvilleA variational perspective on diffusion\-based generative models and score matching\.InAdvances in Neural Information Processing Systems,Vol\.34,pp\. 22863–22876\.External Links:2106\.02808,[Link](https://proceedings.neurips.cc/paper/2021/hash/c11abfd29e4d9b4d4b566b01114d8486-Abstract.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Hyvärinen \(2005\)A\. HyvärinenEstimation of non\-normalized statistical models by score matching\.Journal of Machine Learning Research6\(24\),pp\. 695–709\.External Links:[Link](https://www.jmlr.org/papers/v6/hyvarinen05a.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Isserlis \(1918\)L\. IsserlisOn a formula for the product\-moment coefficient of any order of a normal frequency distribution in any number of variables\.Biometrika12\(1–2\),pp\. 134–139\.External Links:[Document](https://dx.doi.org/10.1093/biomet/12.1-2.134)Cited by:[Appendix A](https://arxiv.org/html/2609.05688#A1.p21.1.1)\.
- Josz and Ouyang \(2026\)C\. Josz and W\. OuyangComputing Kurdyka–Łojasiewicz exponents via composition and symmetry\.External Links:2602\.22553,[Document](https://dx.doi.org/10.48550/arXiv.2602.22553),[Link](https://arxiv.org/abs/2602.22553)Cited by:[§5\.1](https://arxiv.org/html/2609.05688#S5.Thmstylex2.p1.1)\.
- Klusowskiet al\.\(2019\)J\. M\. Klusowski, D\. Yang, and W\. D\. BrindaEstimating the coefficients of a mixture of two linear regressions by expectation maximization\.IEEE Transactions on Information Theory65\(6\),pp\. 3515–3524\.External Links:[Document](https://dx.doi.org/10.1109/TIT.2019.2891628),[Link](https://doi.org/10.1109/TIT.2019.2891628)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Koehleret al\.\(2023\)F\. Koehler, A\. Heckett, and A\. RisteskiStatistical efficiency of score matching: the view from isoperimetry\.InThe Eleventh International Conference on Learning Representations,External Links:2210\.00726,[Link](https://openreview.net/forum?id=TD7AnQjNzR6)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px2.p1.1)\.
- Krantz and Parks \(2002\)S\. G\. Krantz and H\. R\. ParksA primer of real analytic functions\.2nd edition,Birkhäuser Boston\.External Links:[Document](https://dx.doi.org/10.1007/978-0-8176-8134-0),[Link](https://link.springer.com/book/10.1007/978-0-8176-8134-0)Cited by:[Appendix D](https://arxiv.org/html/2609.05688#A4.p28.1.2)\.
- Kwonet al\.\(2021\)J\. Kwon, N\. Ho, and C\. CaramanisOn the minimax optimality of the EM algorithm for learning two\-component mixed linear regression\.InProceedings of The 24th International Conference on Artificial Intelligence and Statistics,A\. Banerjee and K\. Fukumizu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.130,pp\. 1405–1413\.External Links:[Link](https://proceedings.mlr.press/v130/kwon21b.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Kwonet al\.\(2024\)J\. Kwon, W\. Qian, Y\. Chen, C\. Caramanis, D\. Davis, and N\. HoGlobal optimality of the EM algorithm for mixtures of two\-component linear regressions\.IEEE Transactions on Information Theory70\(9\),pp\. 6519–6546\.External Links:[Document](https://dx.doi.org/10.1109/TIT.2024.3435522),[Link](https://doi.org/10.1109/TIT.2024.3435522)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Leeet al\.\(2016\)J\. D\. Lee, M\. Simchowitz, M\. I\. Jordan, and B\. RechtGradient descent only converges to minimizers\.In29th Annual Conference on Learning Theory,V\. Feldman, A\. Rakhlin, and O\. Shamir \(Eds\.\),Proceedings of Machine Learning Research, Vol\.49,pp\. 1246–1257\.External Links:[Link](https://proceedings.mlr.press/v49/lee16.html)Cited by:[Appendix D](https://arxiv.org/html/2609.05688#A4.p23.1.1.1)\.
- Li and Yan \(2024\)G\. Li and Y\. YanA score\-based density formula, with applications in diffusion generative models\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2408.16765),2408\.16765,[Link](https://arxiv.org/abs/2408.16765)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Luo and Hashemi \(2024\)Z\. Luo and A\. HashemiUnveiling the cycloid trajectory of EM iterations in mixed linear regression\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 33329–33382\.External Links:[Link](https://proceedings.mlr.press/v235/luo24c.html)Cited by:[Appendix C](https://arxiv.org/html/2609.05688#A3.p2.1.1),[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px2.p1.1),[§5\.2](https://arxiv.org/html/2609.05688#S5.Thmstylex3.p1.1)\.
- Luo and Hashemi \(2025\)Z\. Luo and A\. HashemiStructural properties, cycloid trajectories and non\-asymptotic guarantees of EM algorithm for mixed linear regression\.External Links:2511\.04937,[Document](https://dx.doi.org/10.48550/arXiv.2511.04937),[Link](https://arxiv.org/abs/2511.04937)Cited by:[A\.12](https://arxiv.org/html/2609.05688#A1.Thmstyle12.3),[A\.12](https://arxiv.org/html/2609.05688#A1.Thmstyle12.p1.1),[Appendix C](https://arxiv.org/html/2609.05688#A3.p2.1.1),[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1),[§1\.1](https://arxiv.org/html/2609.05688#S1.SS1.p1.1),[§2\.3](https://arxiv.org/html/2609.05688#S2.SS3.p1.1)\.
- Luo and Hashemi \(2026\)Z\. Luo and A\. HashemiCharacterizing evolution in expectation\-maximization estimates for overspecified mixed linear regression\.Transactions on Machine Learning Research\.External Links:ISSN 2835\-8856,2508\.10154,[Link](https://openreview.net/forum?id=mFdHMNFtrT)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Lütke Schwienhorstet al\.\(2026\)B\. Lütke Schwienhorst, N\. Klein, and J\. LedererDiffusion\-based denoising beats vanilla score matching in parameter estimation: a theoretical explanation\.External Links:2605\.22950,[Document](https://dx.doi.org/10.48550/arXiv.2605.22950),[Link](https://arxiv.org/abs/2605.22950)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px2.p1.1),[§1\.1](https://arxiv.org/html/2609.05688#S1.SS1.p1.1)\.
- Lyu \(2009\)S\. LyuInterpretation and generalization of score matching\.InProceedings of the Twenty\-Fifth Conference on Uncertainty in Artificial Intelligence,pp\. 359–366\.External Links:1205\.2629,[Link](https://arxiv.org/abs/1205.2629)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Ozkaraet al\.\(2026\)K\. Ozkara, R\. Zhou, and S\. DiggaviSPIRE: conditional personalization for federated diffusion generative models\.InProceedings of the 29th International Conference on Artificial Intelligence and Statistics,E\. Khan, Y\. Li, A\. Solin, and A\. Ramdas \(Eds\.\),Proceedings of Machine Learning Research, Vol\.300,pp\. 1666–1674\.External Links:2506\.12303,[Link](https://proceedings.mlr.press/v300/ozkara26a.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Qin and Risteski \(2024\)Y\. Qin and A\. RisteskiFit like you sample: sample\-efficient generalized score matching from fast mixing diffusions\.InProceedings of Thirty Seventh Conference on Learning Theory,S\. Agrawal and A\. Roth \(Eds\.\),Proceedings of Machine Learning Research, Vol\.247,pp\. 4413–4457\.External Links:2306\.09332,[Link](https://proceedings.mlr.press/v247/qin24a.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px2.p1.1)\.
- Quandt \(1972\)R\. E\. QuandtA new approach to estimating switching regressions\.Journal of the American Statistical Association67\(338\),pp\. 306–310\.External Links:[Document](https://dx.doi.org/10.1080/01621459.1972.10482378),[Link](https://doi.org/10.1080/01621459.1972.10482378)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- Rudin \(1976\)W\. RudinPrinciples of mathematical analysis\.3rd edition,McGraw\-Hill,New York\.Cited by:[Appendix B](https://arxiv.org/html/2609.05688#A2.p14.1.1.1)\.
- Santambrogio \(2015\)F\. SantambrogioOptimal transport for applied mathematicians: calculus of variations, PDEs, and modeling\.Progress in Nonlinear Differential Equations and Their Applications, Vol\.87,Birkhäuser\.External Links:[Document](https://dx.doi.org/10.1007/978-3-319-20828-2),[Link](https://link.springer.com/book/10.1007/978-3-319-20828-2)Cited by:[Appendix A](https://arxiv.org/html/2609.05688#A1.p7.3.1)\.
- Shahet al\.\(2023\)K\. Shah, S\. Chen, and A\. KlivansLearning mixtures of Gaussians using the DDPM objective\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 19636–19649\.External Links:[Document](https://dx.doi.org/10.52202/075280-0862),2307\.01178,[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3ec077b4af90f2556b517b556e186f64-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Shub \(1987\)M\. ShubGlobal stability of dynamical systems\.Springer\-Verlag,New York\.External Links:[Link](https://link.springer.com/book/10.1007/978-1-4757-1947-5)Cited by:[Appendix D](https://arxiv.org/html/2609.05688#A4.p26.1.1)\.
- Songet al\.\(2021a\)Y\. Song, C\. Durkan, I\. Murray, and S\. ErmonMaximum likelihood training of score\-based diffusion models\.InAdvances in Neural Information Processing Systems,Vol\.34,pp\. 1415–1428\.External Links:2101\.09258,[Link](https://proceedings.neurips.cc/paper_files/paper/2021/hash/0a9fdbb17feb6ccb7ec405cfb85222c4-Abstract.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Songet al\.\(2021b\)Y\. Song, J\. Sohl\-Dickstein, D\. P\. Kingma, A\. Kumar, S\. Ermon, and B\. PooleScore\-based generative modeling through stochastic differential equations\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=PxTIG12RRHS)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- Stam \(1959\)A\. J\. StamSome inequalities satisfied by the quantities of information of Fisher and Shannon\.Information and Control2\(2\),pp\. 101–112\.External Links:[Document](https://dx.doi.org/10.1016/S0019-9958%2859%2990348-1),[Link](https://doi.org/10.1016/S0019-9958(59)90348-1)Cited by:[§1\.1](https://arxiv.org/html/2609.05688#S1.SS1.p1.1),[§3](https://arxiv.org/html/2609.05688#S3.p1.1)\.
- Tan and Venkataramanan \(2023\)N\. Tan and R\. VenkataramananMixed regression via approximate message passing\.Journal of Machine Learning Research24\(317\),pp\. 1–44\.External Links:2304\.02229,[Link](https://www.jmlr.org/papers/v24/23-0473.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Tanget al\.\(2025\)R\. Tang, L\. Lin, and Y\. YangConditional diffusion models are minimax\-optimal and manifold\-adaptive for conditional distribution estimation\.InInternational Conference on Learning Representations,External Links:2409\.20124,[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/a5e2853be2ff6218a8d87a5ce4b65062-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.
- van der Vaart \(1998\)A\. W\. van der VaartAsymptotic statistics\.Cambridge Series in Statistical and Probabilistic Mathematics, Vol\.3,Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/CBO9780511802256)Cited by:[Appendix B](https://arxiv.org/html/2609.05688#A2.p11.1),[Appendix B](https://arxiv.org/html/2609.05688#A2.p15.1.1),[Appendix B](https://arxiv.org/html/2609.05688#A2.p9.13),[§1\.1](https://arxiv.org/html/2609.05688#S1.SS1.p1.1),[§3](https://arxiv.org/html/2609.05688#S3.Thmstylex2.p1.1)\.
- Villani \(2009\)C\. VillaniOptimal transport: old and new\.Grundlehren der mathematischen Wissenschaften, Vol\.338,Springer\.External Links:[Document](https://dx.doi.org/10.1007/978-3-540-71050-9),[Link](https://link.springer.com/book/10.1007/978-3-540-71050-9)Cited by:[Appendix A](https://arxiv.org/html/2609.05688#A1.p6.4.1)\.
- Vincent \(2011\)P\. VincentA connection between score matching and denoising autoencoders\.Neural Computation23\(7\),pp\. 1661–1674\.External Links:[Document](https://dx.doi.org/10.1162/NECO%5Fa%5F00142),[Link](https://doi.org/10.1162/NECO_a_00142)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Weinberger and Bresler \(2022\)N\. Weinberger and G\. BreslerThe EM algorithm is adaptively\-optimal for unbalanced symmetric gaussian mixtures\.Journal of Machine Learning Research23\(103\),pp\. 1–79\.Cited by:[§1\.1](https://arxiv.org/html/2609.05688#S1.SS1.p1.1)\.
- Wenliang and Kanagawa \(2020\)L\. K\. Wenliang and H\. KanagawaBlindness of score\-based methods to isolated components and mixing proportions\.Note:Presented at the NeurIPS 2021 Workshop “Your Model is Wrong: Robustness and Misspecification in Probabilistic Modeling”External Links:2008\.10087,[Document](https://dx.doi.org/10.48550/arXiv.2008.10087),[Link](https://arxiv.org/abs/2008.10087)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px2.p1.1),[Figure 4](https://arxiv.org/html/2609.05688#S5.F4)\.
- Yiet al\.\(2014\)X\. Yi, C\. Caramanis, and S\. SanghaviAlternating minimization for mixed linear regression\.InProceedings of the 31st International Conference on Machine Learning,E\. P\. Xing and T\. Jebara \(Eds\.\),Proceedings of Machine Learning Research, Vol\.32,pp\. 613–621\.External Links:[Link](https://proceedings.mlr.press/v32/yia14.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2022\)M\. Zhang, O\. Key, P\. Hayes, D\. Barber, B\. Paige, and F\. BriolTowards healing the blindness of score matching\.InNeurIPS 2022 Workshop on Score\-Based Methods,External Links:2209\.07396,[Link](https://openreview.net/forum?id=Ij8G_k0iuL)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px2.p1.1)\.
- Zhonget al\.\(2016\)K\. Zhong, P\. Jain, and I\. S\. DhillonMixed linear regression with multiple components\.InAdvances in Neural Information Processing Systems,Vol\.29,pp\. 2190–2198\.External Links:[Link](https://proceedings.neurips.cc/paper/2016/hash/8248a99e81e752cb9b41da3fc43fbe7f-Abstract.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1)\.
- Zilber and Nadler \(2023\)P\. Zilber and B\. NadlerImbalanced mixed linear regression\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 54540–54553\.External Links:[Document](https://dx.doi.org/10.52202/075280-2376),2301\.12559,[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/aad615d33ba5071045656ba24d800c7b-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.05688#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.05688#S1.p1.1)\.

## Appendices

The appendices are organized as follows\.

- •Appendix[A](https://arxiv.org/html/2609.05688#A1): We derive the diffused MLR density and score, establish the relative\-entropy dissipation and the population and empirical likelihood bridges for path\-integrated Score Matching, characterize the terminal KL zero set and its Wasserstein relation, and prove the covariance formulas for the results in Sections[2](https://arxiv.org/html/2609.05688#S2)–[6](https://arxiv.org/html/2609.05688#S6)\.
- •Appendix[B](https://arxiv.org/html/2609.05688#A2): Using the likelihood bridges from Appendix[A](https://arxiv.org/html/2609.05688#A1), we prove consistency up to joint sign, the aligned Gaussian limit with inverse\-Fisher covariance, and high\-probability attainment of the empirical Score Matching minimum for the results in Section[3](https://arxiv.org/html/2609.05688#S3)\.
- •Appendix[C](https://arxiv.org/html/2609.05688#A3): We prove the exact cross\-entropy–activation and EM–latent\-variance decompositions, derive their associated gradient identities, and establish the low\- and high\-noise endpoint results from Section[4](https://arxiv.org/html/2609.05688#S4)\.
- •Appendix[D](https://arxiv.org/html/2609.05688#A4): We prove the convergence rates of the high\-noise gradient dynamics, establish pointwise Score Matching blindness along fixed high\-SNR rays, and derive the hard\-assignment EM comparison for the results in Section[5](https://arxiv.org/html/2609.05688#S5)\.
- •Appendix[E](https://arxiv.org/html/2609.05688#A5): We report the numerical configurations, implementation details, calibration calculations, and additional spectral and high\-SNR blindness diagnostics for the results in Section[6](https://arxiv.org/html/2609.05688#S6)\.

### Appendix AAuxiliary Identities and Bounds for Diffused Mixed Linear Regression

This appendix develops the model\-specific identities and bounds used by the main proofs, proceeding from the diffused MLR representation and score formulae to the relative\-entropy, terminal\-identification, Wasserstein, and fourth\-order covariance results\.

###### A\.1Equation \([4](https://arxiv.org/html/2609.05688#S2.E4)\) derived from MLR \([2](https://arxiv.org/html/2609.05688#S2.E2)\) and SDE \([3](https://arxiv.org/html/2609.05688#S2.E3)\)\.

Lety0y\_\{0\}be generated by MLR \([2](https://arxiv.org/html/2609.05688#S2.E2)\) with fixed parametersθ,π\\theta,\\pi\. Suppose thatε0∼𝒩⁡\(0,1\)\\varepsilon\_\{0\}\\sim\\mathcal\{N\}\(0,1\)is independent of\(𝐱,z\)\(\\mathbf\{x\},z\)\. Letyty\_\{t\}be generated by SDE \([3](https://arxiv.org/html/2609.05688#S2.E3)\) with noise scheduleβt\\beta\_\{t\}, whose Brownian motion is independent of\(𝐱,z,ε0\)\(\\mathbf\{x\},z,\\varepsilon\_\{0\}\)\. Then, for everyt\>0t\>0,p⁡\(yt∣y0,𝐱;θ,π\)=p⁡\(yt∣y0\)=𝒩⁡\(yt,y0α¯t,1−α¯t\)p\(y\_\{t\}\\mid y\_\{0\},\\mathbf\{x\};\\theta,\\pi\)=p\(y\_\{t\}\\mid y\_\{0\}\)=\\mathcal\{N\}\(y\_\{t\};y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},1\-\\bar\{\\alpha\}\_\{t\}\), and Equation \([4](https://arxiv.org/html/2609.05688#S2.E4)\) holds\.

yt=y0α¯t\+ξt1−α¯t=\(−1\)z\+1⟨θt,𝐱⟩\+εt=\(−1\)z\+1μt\+εt,y\_\{t\}=y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\_\{t\}\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}=\(\-1\)^\{z\+1\}\\langle\\theta\_\{t\},\\mathbf\{x\}\\rangle\+\\varepsilon\_\{t\}=\(\-1\)^\{z\+1\}\\mu\_\{t\}\+\\varepsilon\_\{t\},whereα¯t:=exp\(−∫0tβτdτ\)\\bar\{\\alpha\}\_\{t\}:=\\exp\\left\(\-\\int\_\{0\}^\{t\}\\beta\_\{\\tau\}\\mathrm\{d\}\\tau\\right\)is the cumulative noise schedule,θt:=θα¯t\\theta\_\{t\}:=\\theta\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}is the scaled regression parameter, andμt:=⟨θt,𝐱⟩\\mu\_\{t\}:=\\langle\\theta\_\{t\},\\mathbf\{x\}\\rangleis the corresponding mean\. The Gaussian noisesξt,εt∼𝒩⁡\(0,1\)\\xi\_\{t\},\\varepsilon\_\{t\}\\sim\\mathcal\{N\}\(0,1\)are independent of\(𝐱,z\)\(\\mathbf\{x\},z\)\.

###### A\.2Negative Log\-Likelihood\.

For the model in Section[2\.1](https://arxiv.org/html/2609.05688#S2.SS1), the negative log\-likelihood at the scale of the diffusion noise levelttisF⁡\(yt,μt,ν\):=−ln⁡p⁡\(yt∣𝐱;θt,π\)F\(y\_\{t\},\\mu\_\{t\},\\nu\):=\-\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\), whereθt=θα¯t\\theta\_\{t\}=\\theta\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\. It has the form

F⁡\(yt,μt,ν\)=12\(yt2\+μt2\)−ln⁡cosh⁡\(μtyt\+ν\)cosh⁡ν\+12ln⁡\(2π\)\.F\(y\_\{t\},\\mu\_\{t\},\\nu\)=\\frac\{1\}\{2\}\(y\_\{t\}^\{2\}\+\\mu\_\{t\}^\{2\}\)\-\\ln\\frac\{\\cosh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\}\{\\cosh\\nu\}\+\\frac\{1\}\{2\}\\ln\(2\\uppi\)\.

###### A\.3Stein Score Function\.

The Stein score issθt,ν\(yt,𝐱\):=∇ytlnp\(yt∣𝐱;θt,π\)s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\):=\\nabla\_\{y\_\{t\}\}\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\. It has the form

sθt,ν\(yt,𝐱\)=−yt\+tanh⁡\(μtyt\+ν\)μt\.s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)=\-y\_\{t\}\+\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\mu\_\{t\}\.

###### Proof of Lemma[A\.1](https://arxiv.org/html/2609.05688#A1.Thmstyle1)\.

Applying Itô’s formula toyt/α¯ty\_\{t\}/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}, usingd⁡\[1/α¯t\]=βt2α¯tdt\\mathrm\{d\}\[1/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\]=\\frac\{\\beta\_\{t\}\}\{2\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\}\\mathrm\{d\}tforα¯t:=exp\(−∫0tβτdτ\)\\bar\{\\alpha\}\_\{t\}:=\\exp\\left\(\-\\int\_\{0\}^\{t\}\\beta\_\{\\tau\}\\mathrm\{d\}\\tau\\right\), and substitutingdyt=−βt2ytdt\+βtdBt\\mathrm\{d\}y\_\{t\}=\-\\frac\{\\beta\_\{t\}\}\{2\}y\_\{t\}\\mathrm\{d\}t\+\\sqrt\{\\beta\_\{t\}\}\\mathrm\{d\}B\_\{t\}from SDE \([3](https://arxiv.org/html/2609.05688#S2.E3)\) gives

d⁡\[ytα¯t\]=1α¯tdyt\+ytd\[1α¯t\]=βtα¯tdBt\.\\mathrm\{d\}\\left\[\\frac\{y\_\{t\}\}\{\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\}\\right\]=\\frac\{1\}\{\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\}\\mathrm\{d\}y\_\{t\}\+y\_\{t\}\\mathrm\{d\}\\left\[\\frac\{1\}\{\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\}\\right\]=\\sqrt\{\\frac\{\\beta\_\{t\}\}\{\\bar\{\\alpha\}\_\{t\}\}\}\\mathrm\{d\}B\_\{t\}\.Integrating from00tott, settingθt:=θα¯t\\theta\_\{t\}:=\\theta\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}, and substitutingy0=\(−1\)z\+1⟨θ,𝐱⟩\+ε0y\_\{0\}=\(\-1\)^\{z\+1\}\\langle\\theta,\\mathbf\{x\}\\rangle\+\\varepsilon\_\{0\}from MLR \([2](https://arxiv.org/html/2609.05688#S2.E2)\) gives

yt=α¯ty0\+α¯t∫0tβτα¯τdBτ=\(−1\)z\+1⟨θt,𝐱⟩⏟=:μt\+α¯t\[ε0\+∫0tβτα¯τdBτ\]⏟=:εt=\(−1\)z\+1μt\+εt\.y\_\{t\}=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}y\_\{0\}\+\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\int\_\{0\}^\{t\}\\sqrt\{\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\}\\mathrm\{d\}B\_\{\\tau\}=\(\-1\)^\{z\+1\}\\underbrace\{\\langle\\theta\_\{t\},\\mathbf\{x\}\\rangle\}\_\{=:\{\\color\[rgb\]\{0\.6133,0\.1523,0\.6914\}\\;\\mu\_\{t\}\}\}\+\\underbrace\{\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\left\[\\varepsilon\_\{0\}\+\\int\_\{0\}^\{t\}\\sqrt\{\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\}\\mathrm\{d\}B\_\{\\tau\}\\right\]\}\_\{=:\{\\color\[rgb\]\{0\.0195,0\.4883,0\.8047\}\\;\\varepsilon\_\{t\}\}\}=\(\-1\)^\{z\+1\}\{\\color\[rgb\]\{0\.6133,0\.1523,0\.6914\}\\mu\_\{t\}\}\+\{\\color\[rgb\]\{0\.0195,0\.4883,0\.8047\}\\varepsilon\_\{t\}\}\.Sinceβτ/α¯τ\\sqrt\{\\beta\_\{\\tau\}/\\bar\{\\alpha\}\_\{\\tau\}\}is deterministic, the stochastic integral is a zero\-mean Gaussian\. The initial noise satisfiesε0∼𝒩⁡\(0,1\)\\varepsilon\_\{0\}\\sim\\mathcal\{N\}\(0,1\), and the forward Brownian motion is independent of\(𝐱,z,ε0\)\(\\mathbf\{x\},z,\\varepsilon\_\{0\}\)\. Hence their linear combinationεt\\varepsilon\_\{t\}is also a zero\-mean Gaussian, with

𝔼⁡\[εt\]=α¯t\(𝔼⁡\[ε0\]\+𝔼⁡\[∫0tβτα¯τdBτ\]\)=α¯t\(0\+0\)=0\.\\mathbb\{E\}\[\{\\color\[rgb\]\{0\.0195,0\.4883,0\.8047\}\\varepsilon\_\{t\}\}\]=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\left\(\\mathbb\{E\}\[\\varepsilon\_\{0\}\]\+\\mathbb\{E\}\\left\[\\int\_\{0\}^\{t\}\\sqrt\{\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\}\\mathrm\{d\}B\_\{\\tau\}\\right\]\\right\)=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\left\(0\+0\\right\)=0\.Sinceε0∼𝒩⁡\(0,1\)\\varepsilon\_\{0\}\\sim\\mathcal\{N\}\(0,1\)is independent of\{Bτ:0≤τ≤t\}\\\{B\_\{\\tau\}:0\\leq\\tau\\leq t\\\}, the variance ofεt\\varepsilon\_\{t\}is the sum of the two variances\. Itô’s isometry givesVar⁡\[∫0tβτα¯τdBτ\]=∫0tβτα¯τ𝑑τ\\mathrm\{Var\}\\big\[\\int\_\{0\}^\{t\}\\sqrt\{\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\}\\mathrm\{d\}B\_\{\\tau\}\\big\]=\\int\_\{0\}^\{t\}\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\\mathrm\{d\}\\tau\. The identityd⁡\[1α¯τ\]=βτα¯τdτ\\mathrm\{d\}\\left\[\\frac\{1\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\\right\]=\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\\mathrm\{d\}\\tauthen gives∫0tβτα¯τdBτ∼𝒩⁡\(0,1/α¯t−1\)\\int\_\{0\}^\{t\}\\sqrt\{\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\}\\mathrm\{d\}B\_\{\\tau\}\\sim\\mathcal\{N\}\(0,1/\\bar\{\\alpha\}\_\{t\}\-1\)and

Var⁡\[εt\]=α¯t\(Var⁡\[ε0\]\+Var⁡\[∫0tβτα¯τdBτ\]\)=α¯t\(1\+∫0tβτα¯τ𝑑τ\)=α¯t\(1\+1α¯τ\|0t\)=1\.\\mathrm\{Var\}\[\{\\color\[rgb\]\{0\.0195,0\.4883,0\.8047\}\\varepsilon\_\{t\}\}\]=\\bar\{\\alpha\}\_\{t\}\\left\(\\mathrm\{Var\}\[\\varepsilon\_\{0\}\]\+\\mathrm\{Var\}\\left\[\\int\_\{0\}^\{t\}\\sqrt\{\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\}\\mathrm\{d\}B\_\{\\tau\}\\right\]\\right\)=\\bar\{\\alpha\}\_\{t\}\\left\(1\+\\int\_\{0\}^\{t\}\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\\mathrm\{d\}\\tau\\right\)=\\bar\{\\alpha\}\_\{t\}\\left\(1\+\\frac\{1\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\\Big\|\_\{0\}^\{t\}\\right\)=1\.Thus,εt∼𝒩⁡\(0,1\)\\varepsilon\_\{t\}\\sim\\mathcal\{N\}\(0,1\)andp⁡\(yt∣y0,𝐱;θ,π\)=p⁡\(yt∣y0\)=𝒩⁡\(yt,y0α¯t,1−α¯t\)p\(y\_\{t\}\\mid y\_\{0\},\\mathbf\{x\};\\theta,\\pi\)=p\(y\_\{t\}\\mid y\_\{0\}\)=\\mathcal\{N\}\(y\_\{t\};y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},1\-\\bar\{\\alpha\}\_\{t\}\)\. Fort\>0t\>0, it follows thatξt:=\(yt−y0α¯t\)/1−α¯t=α¯t1−α¯t∫0tβτα¯τdBτ∼𝒩⁡\(0,1\)\\xi\_\{t\}:=\(y\_\{t\}\-y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\)/\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}=\\sqrt\{\\frac\{\\bar\{\\alpha\}\_\{t\}\}\{1\-\\bar\{\\alpha\}\_\{t\}\}\}\\int\_\{0\}^\{t\}\\sqrt\{\\frac\{\\beta\_\{\\tau\}\}\{\\bar\{\\alpha\}\_\{\\tau\}\}\}\\mathrm\{d\}B\_\{\\tau\}\\sim\\mathcal\{N\}\(0,1\)\. The assumed independence ofε0\\varepsilon\_\{0\}, the Brownian motion, and\(𝐱,z\)\(\\mathbf\{x\},z\)shows thatξt\\xi\_\{t\}andεt\\varepsilon\_\{t\}are independent of\(𝐱,z\)\(\\mathbf\{x\},z\)\. ∎

###### Proofs of Lemmas[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)and[A\.3](https://arxiv.org/html/2609.05688#A1.Thmstyle3)\.

We first obtain the conditional density ofyty\_\{t\}by marginalizing the joint density overz∈\{1,2\}z\\in\\\{1,2\\\}\. The model assumptions giveℙ⁡\(z∣𝐱;θt,π\)=ℙ⁡\(z∣π\)=π⁡\(z\)\\mathbb\{P\}\(z\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\\mathbb\{P\}\(z\\mid\\pi\)=\\pi\(z\)\. Hence

F\(yt,μt,ν\):=−lnp\(yt∣𝐱;θt,π\)=−ln∑z∈\{1,2\}p\(yt,z∣𝐱;θt,π\)=−ln∑z∈\{1,2\}π\(z\)⋅p\(yt∣z,𝐱;θt,π\)\.F\(y\_\{t\},\\mu\_\{t\},\\nu\):=\-\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\-\\ln\\sum\_\{z\\in\\\{1,2\\\}\}p\(y\_\{t\},z\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\-\\ln\\sum\_\{z\\in\\\{1,2\\\}\}\\pi\(z\)\\cdot p\(y\_\{t\}\\mid z,\\mathbf\{x\};\\theta\_\{t\},\\pi\)\.Lemma[A\.1](https://arxiv.org/html/2609.05688#A1.Thmstyle1)givesεt∼𝒩⁡\(0,1\)\\varepsilon\_\{t\}\\sim\\mathcal\{N\}\(0,1\), independent of\(𝐱,z\)\(\\mathbf\{x\},z\)\. Hencep⁡\(yt∣z,𝐱;θt,π\)=p⁡\(yt∣z,μt\)=𝒩⁡\(\(−1\)z\+1μt,1\)p\(y\_\{t\}\\mid z,\\mathbf\{x\};\\theta\_\{t\},\\pi\)=p\(y\_\{t\}\\mid z,\\mu\_\{t\}\)=\\mathcal\{N\}\(\(\-1\)^\{z\+1\}\\mu\_\{t\},1\), whereμt=⟨θt,𝐱⟩\\mu\_\{t\}=\\langle\\theta\_\{t\},\\mathbf\{x\}\\rangle\.

F\(yt,μt,ν\)=−ln∑z∈\{1,2\}π\(z\)⋅p\(yt∣z,μt\)=−ln∑z∈\{1,2\}π\(z\)⋅𝒩\(\(−1\)z\+1μt,1\)\.F\(y\_\{t\},\\mu\_\{t\},\\nu\)=\-\\ln\\sum\_\{z\\in\\\{1,2\\\}\}\\pi\(z\)\\cdot p\(y\_\{t\}\\mid z,\\mu\_\{t\}\)=\-\\ln\\sum\_\{z\\in\\\{1,2\\\}\}\\pi\(z\)\\cdot\\mathcal\{N\}\(\(\-1\)^\{z\+1\}\\mu\_\{t\},1\)\.The imbalance parameterν:=\(ln⁡π⁡\(1\)−ln⁡π⁡\(2\)\)/2\\nu:=\(\\ln\\pi\(1\)\-\\ln\\pi\(2\)\)/2is in one\-to\-one correspondence with the mixing weights\. In particular,π⁡\(1\)=eν2cosh⁡ν\\pi\(1\)=\\frac\{\\mathrm\{e\}^\{\\nu\}\}\{2\\cosh\\nu\}andπ⁡\(2\)=e−ν2cosh⁡ν\\pi\(2\)=\\frac\{\\mathrm\{e\}^\{\-\\nu\}\}\{2\\cosh\\nu\}\. Substitution gives

F⁡\(yt,μt,ν\)\\displaystyle F\(y\_\{t\},\\mu\_\{t\},\\nu\)=\\displaystyle=−ln⁡\(eν2cosh⁡ν12πe−12\(yt2−2ytμt\+μt2\)\+e−ν2cosh⁡ν12πe−12\(yt2\+2ytμt\+μt2\)\)\\displaystyle\-\\ln\\left\(\\frac\{\\mathrm\{e\}^\{\\nu\}\}\{2\\cosh\\nu\}\\frac\{1\}\{\\sqrt\{2\\uppi\}\}\\mathrm\{e\}^\{\-\\frac\{1\}\{2\}\(y\_\{t\}^\{2\}\-2y\_\{t\}\\mu\_\{t\}\+\\mu\_\{t\}^\{2\}\)\}\+\\frac\{\\mathrm\{e\}^\{\-\\nu\}\}\{2\\cosh\\nu\}\\frac\{1\}\{\\sqrt\{2\\uppi\}\}\\mathrm\{e\}^\{\-\\frac\{1\}\{2\}\(y\_\{t\}^\{2\}\+2y\_\{t\}\\mu\_\{t\}\+\\mu\_\{t\}^\{2\}\)\}\\right\)=\\displaystyle=12\(yt2\+μt2\)−ln⁡\(eν\+ytμt\+e−\(ν\+ytμt\)\)\+ln⁡\(2cosh⁡ν\)\+12ln⁡\(2π\)\\displaystyle\\frac\{1\}\{2\}\(y\_\{t\}^\{2\}\+\\mu\_\{t\}^\{2\}\)\-\\ln\\left\(e^\{\\nu\+y\_\{t\}\\mu\_\{t\}\}\+e^\{\-\(\\nu\+y\_\{t\}\\mu\_\{t\}\)\}\\right\)\+\\ln\(2\\cosh\\nu\)\+\\frac\{1\}\{2\}\\ln\(2\\uppi\)=\\displaystyle=12\(yt2\+μt2\)−ln⁡cosh⁡\(ν\+ytμt\)cosh⁡ν\+12ln⁡\(2π\)\.\\displaystyle\\frac\{1\}\{2\}\(y\_\{t\}^\{2\}\+\\mu\_\{t\}^\{2\}\)\-\\ln\\frac\{\\cosh\(\\nu\+y\_\{t\}\\mu\_\{t\}\)\}\{\\cosh\\nu\}\+\\frac\{1\}\{2\}\\ln\(2\\uppi\)\.For Lemma[A\.3](https://arxiv.org/html/2609.05688#A1.Thmstyle3), write∇\\nablafor∇yt\\nabla\_\{y\_\{t\}\}\. The Stein score is the negative gradient ofFF\. Sinceμt\\mu\_\{t\},ν\\nu, and12ln⁡\(2π\)\\frac\{1\}\{2\}\\ln\(2\\uppi\)do not depend onyty\_\{t\}, the chain rule gives

sθt,ν\(yt,𝐱\):=∇lnp\(yt∣𝐱;θt,π\)=−∇F\(yt,μt,ν\)=−\(yt−∇lncosh\(μtyt\+ν\)\)=−yt\+tanh\(μtyt\+ν\)μt\.s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\):=\\nabla\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\-\\nabla F\(y\_\{t\},\\mu\_\{t\},\\nu\)=\-\\left\(y\_\{t\}\-\\nabla\\ln\\cosh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\right\)=\-y\_\{t\}\+\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\mu\_\{t\}\.∎

###### A\.4Conditional Relative Entropy Dissipation\.

The derivative of the KL divergence between the true and model distributions satisfies

−ddt𝖪𝖫\(p\(yt\|𝐱;θt∗,π∗\)∥p\(yt\|𝐱;θt,π\)\)=βt2𝔼yt\|𝐱;θt∗,π∗∥∇lnp\(yt\|𝐱;θt,π\)−∇lnp\(yt\|𝐱;θt∗,π∗\)∥2\.\\displaystyle\-\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\KL\\left\(p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)\\right\)=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\\|\\nabla\\ln p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)\-\\nabla\\ln p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)\\\|^\{2\}\.

###### A\.5Log\-likelihood Derivative and Score Matching Loss\.

Letℙn\\mathbb\{P\}\_\{n\}be the empirical measure defined before Equation \([8](https://arxiv.org/html/2609.05688#S2.E8)\)\. Then, for everyt\>0t\>0,

ddt𝔼ℙn𝔼yt\|y0ln⁡p⁡\(yt\|𝐱;θt,π\)=βt2𝔼ℙn𝔼yt\|y0\[‖∇ln⁡p\(yt\|𝐱;θt,π\)‖2−2⟨∇ln⁡p\(yt\|𝐱;θt,π\),∇ln⁡p\(yt\|y0\)⟩\+1\]\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\\ln p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\\left\[\\\|\\nabla\\ln p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)\\\|^\{2\}\-2\\langle\\nabla\\ln p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\),\\nabla\\ln p\(y\_\{t\}\|y\_\{0\}\)\\rangle\+1\\right\]\.

###### Proof of Lemma[A\.4](https://arxiv.org/html/2609.05688#A1.Thmstyle4)\.

Letpt:=p⁡\(yt\|𝐱;θt,π\)p\_\{t\}:=p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)denote the model density, and letpt∗:=p⁡\(yt\|𝐱;θt∗,π∗\)p^\{\*\}\_\{t\}:=p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\*\},\\pi^\{\*\}\)denote the ground truth density\. The negative time derivative of the KL divergence is

−ddt𝖪𝖫\(pt∗∥pt\)=−ddt𝔼pt∗lnpt∗pt=ddt𝔼pt∗\[lnpt\]−ddt𝔼pt∗\[lnpt∗\]\.\-\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\KL\(p^\{\*\}\_\{t\}\\parallel p\_\{t\}\)=\-\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\\ln\\frac\{p^\{\*\}\_\{t\}\}\{p\_\{t\}\}=\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\[\\ln p\_\{t\}\]\-\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\[\\ln p^\{\*\}\_\{t\}\]\.The forward Kolmogorov operator for SDE \([3](https://arxiv.org/html/2609.05688#S2.E3)\) is𝒜∗\[f\]:=−∇⋅\[f⋅\(−βt2yt\)\]\+12Δ\[f⋅\(βt\)2\]=βt2\[∇⋅\(fyt\)\+Δf\]\\mathcal\{A\}^\{\*\}\[f\]:=\-\\nabla\\cdot\\left\[f\\cdot\(\-\\frac\{\\beta\_\{t\}\}\{2\}y\_\{t\}\)\\right\]\+\\frac\{1\}\{2\}\\Delta\\left\[f\\cdot\(\\sqrt\{\\beta\_\{t\}\}\)^\{2\}\\right\]=\\frac\{\\beta\_\{t\}\}\{2\}\\left\[\\nabla\\cdot\(fy\_\{t\}\)\+\\Delta f\\right\]\. Lemma[A\.1](https://arxiv.org/html/2609.05688#A1.Thmstyle1)gives the transition densityp⁡\(yt∣y0\)=𝒩⁡\(yt,y0α¯t,1−α¯t\)p\(y\_\{t\}\\mid y\_\{0\}\)=\\mathcal\{N\}\(y\_\{t\};y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},1\-\\bar\{\\alpha\}\_\{t\}\)\. Its Fokker–Planck–Kolmogorov equation is

∂tp⁡\(yt∣y0\)=βt2\[∇⋅\(p⁡\(yt∣y0\)yt\)\+Δp\(yt∣y0\)\]≡𝒜∗\[p⁡\(yt∣y0\)\]\.\\partial\_\{t\}p\(y\_\{t\}\\mid y\_\{0\}\)=\\frac\{\\beta\_\{t\}\}\{2\}\\left\[\\nabla\\cdot\(p\(y\_\{t\}\\mid y\_\{0\}\)y\_\{t\}\)\+\\Delta p\(y\_\{t\}\\mid y\_\{0\}\)\\right\]\\equiv\\mathcal\{A\}^\{\*\}\[p\(y\_\{t\}\\mid y\_\{0\}\)\]\.The Chapman–Kolmogorov relation uses the fixed initial law and gives

pt=𝔼y0\|𝐱;θ,π\[p⁡\(yt∣y0\)\]=∑k=12π⁡\(k\)𝒩\(yt,\(−1\)k\+1μ0α¯t,1\)\.p\_\{t\}=\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\theta,\\pi\}\[p\(y\_\{t\}\\mid y\_\{0\}\)\]=\\sum\_\{k=1\}^\{2\}\\pi\(k\)\\,\\mathcal\{N\}\\\!\\left\(y\_\{t\};\(\-1\)^\{k\+1\}\\mu\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},1\\right\)\.Direct differentiation shows that each of these two Gaussian components satisfies the FPK equation above, and linearity of the forward operator therefore gives

∂tpt=𝒜∗\[pt\]=βt2\[∇⋅\(ptyt\)\+Δpt\]\.\\partial\_\{t\}p\_\{t\}=\\mathcal\{A\}^\{\*\}\[p\_\{t\}\]=\\frac\{\\beta\_\{t\}\}\{2\}\\left\[\\nabla\\cdot\(p\_\{t\}y\_\{t\}\)\+\\Delta p\_\{t\}\\right\]\.Dividing byptp\_\{t\}and using identities∇⋅\(ptyt\)pt=1\+⟨∇ln⁡pt,yt⟩\\frac\{\\nabla\\cdot\(p\_\{t\}y\_\{t\}\)\}\{p\_\{t\}\}=1\+\\langle\\nabla\\ln p\_\{t\},y\_\{t\}\\rangle,Δptpt=Δln⁡pt\+‖∇ln⁡pt‖2\\frac\{\\Delta p\_\{t\}\}\{p\_\{t\}\}=\\Delta\\ln p\_\{t\}\+\\\|\\nabla\\ln p\_\{t\}\\\|^\{2\}, we obtain the derivative of the log\-density\.

∂tln⁡pt=βt2\[1\+⟨∇ln⁡pt,yt⟩\+Δln⁡pt\+‖∇ln⁡pt‖2\]\.\\partial\_\{t\}\\ln p\_\{t\}=\\frac\{\\beta\_\{t\}\}\{2\}\\left\[1\+\\langle\\nabla\\ln p\_\{t\},y\_\{t\}\\rangle\+\\Delta\\ln p\_\{t\}\+\\\|\\nabla\\ln p\_\{t\}\\\|^\{2\}\\right\]\.Applying Itô’s formula toln⁡pt\\ln p\_\{t\}under the forward SDE gives

dln⁡pt\\displaystyle\\mathrm\{d\}\\ln p\_\{t\}=∂tln⁡ptdt\+⟨∇ln⁡pt,dyt⟩\+12Δln⁡pt\(dyt\)2\\displaystyle=\\partial\_\{t\}\\ln p\_\{t\}\\mathrm\{d\}t\+\\langle\\nabla\\ln p\_\{t\},\\mathrm\{d\}y\_\{t\}\\rangle\+\\frac\{1\}\{2\}\\Delta\\ln p\_\{t\}\(\\mathrm\{d\}y\_\{t\}\)^\{2\}=∂tln⁡ptdt\+⟨∇ln⁡pt,−βt2ytdt\+βtdBt⟩\+βt2Δln⁡ptdt\\displaystyle=\\partial\_\{t\}\\ln p\_\{t\}\\mathrm\{d\}t\+\\left\\langle\\nabla\\ln p\_\{t\},\-\\frac\{\\beta\_\{t\}\}\{2\}y\_\{t\}\\mathrm\{d\}t\+\\sqrt\{\\beta\_\{t\}\}\\mathrm\{d\}B\_\{t\}\\right\\rangle\+\\frac\{\\beta\_\{t\}\}\{2\}\\Delta\\ln p\_\{t\}\\mathrm\{d\}t=βt2\[‖∇ln⁡pt‖2\+2Δln⁡pt\+1\]dt\+βt⟨∇ln⁡pt,dBt⟩\.\\displaystyle=\\frac\{\\beta\_\{t\}\}\{2\}\\left\[\\\|\\nabla\\ln p\_\{t\}\\\|^\{2\}\+2\\Delta\\ln p\_\{t\}\+1\\right\]\\mathrm\{d\}t\+\\sqrt\{\\beta\_\{t\}\}\\langle\\nabla\\ln p\_\{t\},\\mathrm\{d\}B\_\{t\}\\rangle\.Under the ground truth path measureℙ∗\\mathbb\{P\}^\{\*\}, the state at eachτ∈\[0,t\]\\tau\\in\[0,t\]has marginal densitypτ∗p\_\{\\tau\}^\{\\ast\}\. Fubini’s theorem moves the expectation inside the time integral, and the Brownian stochastic integral has mean zero\.

𝔼pt∗\[ln⁡pt\]−𝔼p0∗\[ln⁡p0\]\\displaystyle\\mathbb\{E\}\_\{p\_\{t\}^\{\\ast\}\}\[\\ln p\_\{t\}\]\-\\mathbb\{E\}\_\{p\_\{0\}^\{\\ast\}\}\[\\ln p\_\{0\}\]=𝔼ℙ∗\[∫0tβτ2\[‖∇ln⁡pτ‖2\+2Δln⁡pτ\+1\]dτ\]\\displaystyle=\\mathbb\{E\}\_\{\\mathbb\{P\}^\{\\ast\}\}\\left\[\\int\_\{0\}^\{t\}\\frac\{\\beta\_\{\\tau\}\}\{2\}\\left\[\\\|\\nabla\\ln p\_\{\\tau\}\\\|^\{2\}\+2\\Delta\\ln p\_\{\\tau\}\+1\\right\]\\mathrm\{d\}\\tau\\right\]=∫0tβτ2𝔼pτ∗\[‖∇ln⁡pτ‖2\+2Δln⁡pτ\+1\]dτ\.\\displaystyle=\\int\_\{0\}^\{t\}\\frac\{\\beta\_\{\\tau\}\}\{2\}\\mathbb\{E\}\_\{p^\{\*\}\_\{\\tau\}\}\\left\[\\\|\\nabla\\ln p\_\{\\tau\}\\\|^\{2\}\+2\\Delta\\ln p\_\{\\tau\}\+1\\right\]\\mathrm\{d\}\\tau\.The fundamental theorem of calculus gives

ddt𝔼pt∗\[ln⁡pt\]=βt2𝔼pt∗\[‖∇ln⁡pt‖2\+2Δln⁡pt\+1\]\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p\_\{t\}^\{\\ast\}\}\[\\ln p\_\{t\}\]=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{p\_\{t\}^\{\\ast\}\}\\left\[\\\|\\nabla\\ln p\_\{t\}\\\|^\{2\}\+2\\Delta\\ln p\_\{t\}\+1\\right\]\.Integration by parts and the vanishing boundary terms give

𝔼pt∗\[Δln⁡pt\]=𝔼pt∗\[∇⋅\(∇ln⁡pt\)\]=−𝔼pt∗\[⟨∇ln⁡pt∗,∇ln⁡pt⟩\]\.\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\[\\Delta\\ln p\_\{t\}\]=\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\[\\nabla\\cdot\(\\nabla\\ln p\_\{t\}\)\]=\-\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\[\\langle\\nabla\\ln p^\{\*\}\_\{t\},\\nabla\\ln p\_\{t\}\\rangle\]\.Substitution gives

ddt𝔼pt∗\[ln⁡pt\]=βt2𝔼pt∗\[‖∇ln⁡pt‖2−2⟨∇ln⁡pt∗,∇ln⁡pt⟩\+1\]\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p\_\{t\}^\{\\ast\}\}\[\\ln p\_\{t\}\]=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{p\_\{t\}^\{\\ast\}\}\\left\[\\\|\\nabla\\ln p\_\{t\}\\\|^\{2\}\-2\\langle\\nabla\\ln p^\{\*\}\_\{t\},\\nabla\\ln p\_\{t\}\\rangle\+1\\right\]\.Applying the same calculation to the ground truth distribution gives

ddt𝔼pt∗\[ln⁡pt∗\]=βt2𝔼pt∗\[‖∇ln⁡pt∗‖2−2‖∇ln⁡pt∗‖2\+1\]=βt2𝔼pt∗\[−‖∇ln⁡pt∗‖2\+1\]\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\[\\ln p^\{\*\}\_\{t\}\]=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\\left\[\\\|\\nabla\\ln p^\{\*\}\_\{t\}\\\|^\{2\}\-2\\\|\\nabla\\ln p^\{\*\}\_\{t\}\\\|^\{2\}\+1\\right\]=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\\left\[\-\\\|\\nabla\\ln p^\{\*\}\_\{t\}\\\|^\{2\}\+1\\right\]\.Subtracting the entropy derivative from the cross\-entropy derivative gives

−ddt𝖪𝖫\(pt∗∥pt\)=ddt𝔼pt∗\[lnpt\]−ddt𝔼pt∗\[lnpt∗\]=βt2𝔼pt∗\[∥∇lnpt−∇lnpt∗∥2\]\.\-\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\KL\(p^\{\*\}\_\{t\}\\parallel p\_\{t\}\)=\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p\_\{t\}^\{\\ast\}\}\[\\ln p\_\{t\}\]\-\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{p\_\{t\}^\{\\ast\}\}\[\\ln p^\{\*\}\_\{t\}\]=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{p^\{\*\}\_\{t\}\}\\left\[\\\|\\nabla\\ln p\_\{t\}\-\\nabla\\ln p^\{\*\}\_\{t\}\\\|^\{2\}\\right\]\.Restoring the full notationpt≡p⁡\(yt\|𝐱;θt,π\)p\_\{t\}\\equiv p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)andpt∗≡p⁡\(yt\|𝐱;θt∗,π∗\)p\_\{t\}^\{\\ast\}\\equiv p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\*\},\\pi^\{\*\}\)yields the lemma statement\. ∎

###### Proof of Lemma[A\.5](https://arxiv.org/html/2609.05688#A1.Thmstyle5)\.

We apply the proof of Lemma[A\.4](https://arxiv.org/html/2609.05688#A1.Thmstyle4)withpt∗p^\{\\ast\}\_\{t\}replaced byp⁡\(yt∣y0\)p\(y\_\{t\}\\mid y\_\{0\}\)\. This gives

ddt𝔼yt\|y0\[ln⁡pt\]=βt2𝔼yt\|y0\[‖∇ln⁡pt‖2−2⟨∇ln⁡pt,∇ln⁡p\(yt\|y0\)⟩\+1\]\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\[\\ln p\_\{t\}\]=\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\[\\\|\\nabla\\ln p\_\{t\}\\\|^\{2\}\-2\\langle\\nabla\\ln p\_\{t\},\\nabla\\ln p\(y\_\{t\}\|y\_\{0\}\)\\rangle\+1\]\.Taking the expectation overℙn\\mathbb\{P\}\_\{n\}, interchanging𝔼ℙn\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}andddt\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}, and restoringpt≡p⁡\(yt\|𝐱;θt,π\)p\_\{t\}\\equiv p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)proves the result\. ∎

###### A\.6Decaying KL Divergence Bound\.

Suppose0<α¯t<10<\\bar\{\\alpha\}\_\{t\}<1\. Let the two conditional diffusions start from the initial probability lawsp⁡\(y0\|𝐱;θ∗,π∗\)p\(y\_\{0\}\|\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\)andp⁡\(y0\|𝐱;θ,π\)p\(y\_\{0\}\|\\mathbf\{x\};\\theta,\\pi\)\. Their time\-ttlaws satisfy

𝖪𝖫\(p\(yt\|𝐱;θt∗,π∗\)∥p\(yt\|𝐱;θt,π\)\)≤α¯t2\(1−α¯t\)W22\(p\(y0\|𝐱;θ∗,π∗\),p\(y0\|𝐱;θ,π\)\)\.\\KL\\left\(p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\*\},\\pi^\{\*\}\)\\parallel p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)\\right\)\\leq\\frac\{\\bar\{\\alpha\}\_\{t\}\}\{2\(1\-\\bar\{\\alpha\}\_\{t\}\)\}W\_\{2\}^\{2\}\\left\(p\(y\_\{0\}\|\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\),\\;p\(y\_\{0\}\|\\mathbf\{x\};\\theta,\\pi\)\\right\)\.

###### A\.7Wasserstein Distance for Conditional Distributions\.

Fix𝐱\\mathbf\{x\}\. Define the two discrete distributionsD:=D⁡\(𝐱,θ,π\)≡π⁡\(1\)δ⟨θ,𝐱⟩\+π⁡\(2\)δ−⟨θ,𝐱⟩D:=D\(\\mathbf\{x\};\\theta,\\pi\)\\equiv\\pi\(1\)\\delta\_\{\\langle\\theta,\\mathbf\{x\}\\rangle\}\+\\pi\(2\)\\delta\_\{\-\\langle\\theta,\\mathbf\{x\}\\rangle\}andD∗:=D⁡\(𝐱,θ∗,π∗\)≡π∗\(1\)δ⟨θ∗,𝐱⟩\+π∗\(2\)δ−⟨θ∗,𝐱⟩D^\{\*\}:=D\(\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\)\\equiv\\pi^\{\*\}\(1\)\\delta\_\{\\langle\\theta^\{\*\},\\mathbf\{x\}\\rangle\}\+\\pi^\{\*\}\(2\)\\delta\_\{\-\\langle\\theta^\{\*\},\\mathbf\{x\}\\rangle\}\. Then the squared 2\-Wasserstein distance betweenp⁡\(y0∣𝐱;θ∗,π∗\)p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\)andp⁡\(y0∣𝐱;θ,π\)p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta,\\pi\)is upper bounded by the squared 2\-Wasserstein distance between the discrete distributions\.

W22\(p⁡\(y0∣𝐱;θ∗,π∗\),p⁡\(y0∣𝐱;θ,π\)\)≤W22\(D,D∗\)\.W\_\{2\}^\{2\}\(p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\),p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta,\\pi\)\)\\leq W\_\{2\}^\{2\}\(D,D^\{\*\}\)\.

###### A\.8Decaying KL Divergence w\.r\.t\. Transition Kernel\.

Suppose0<α¯t<10<\\bar\{\\alpha\}\_\{t\}<1\. Letp⁡\(yt∣y0\)p\(y\_\{t\}\\mid y\_\{0\}\)be the transition kernel at timettfrom a fixed initial valuey0y\_\{0\}\. Setμ0=⟨θ,𝐱⟩\\mu\_\{0\}=\\langle\\theta,\\mathbf\{x\}\\rangleandν=\(ln⁡π⁡\(1\)−ln⁡π⁡\(2\)\)/2\\nu=\(\\ln\\pi\(1\)\-\\ln\\pi\(2\)\)/2\. Then

𝖪𝖫\(p\(yt∣y0\)∥p\(yt∣𝐱;θt,π\)\)≤α¯t2\(1−α¯t\)\(y02\+μ02−2y0μ0tanhν\+1\)\.\\KL\\left\(p\(y\_\{t\}\\mid y\_\{0\}\)\\parallel p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\\right\)\\leq\\frac\{\\bar\{\\alpha\}\_\{t\}\}\{2\(1\-\\bar\{\\alpha\}\_\{t\}\)\}\(y\_\{0\}^\{2\}\+\\mu\_\{0\}^\{2\}\-2y\_\{0\}\\mu\_\{0\}\\tanh\\nu\+1\)\.

###### Proof of Lemma[A\.6](https://arxiv.org/html/2609.05688#A1.Thmstyle6)\.

For any fixedu∈ℝu\\in\\mathbb\{R\}, Lemma[A\.1](https://arxiv.org/html/2609.05688#A1.Thmstyle1)gives the transition density

qt\(yt∣u\):=𝒩⁡\(yt,uα¯t,1−α¯t\)\.q\_\{t\}\(y\_\{t\}\\mid u\):=\\mathcal\{N\}\\left\(y\_\{t\};u\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},1\-\\bar\{\\alpha\}\_\{t\}\\right\)\.Define the initial probability lawsp0∗:=p⁡\(y0∣𝐱;θ∗,π∗\)p\_\{0\}^\{\*\}:=p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\)andp0:=p⁡\(y0∣𝐱;θ,π\)p\_\{0\}:=p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta,\\pi\)\. We usey~0\\widetilde\{y\}\_\{0\}for a value underp0∗p\_\{0\}^\{\*\}andy0y\_\{0\}for a value underp0p\_\{0\}\. Since the two transition densities have variance1−α¯t1\-\\bar\{\\alpha\}\_\{t\}, the Gaussian KL formula gives

𝖪𝖫\(qt\(⋅∣y~0\)∥qt\(⋅∣y0\)\)=α¯t2\(1−α¯t\)∥y~0−y0∥2\.\\KL\(q\_\{t\}\(\\,\\cdot\\mid\\widetilde\{y\}\_\{0\}\)\\parallel q\_\{t\}\(\\,\\cdot\\mid y\_\{0\}\)\)=\\frac\{\\bar\{\\alpha\}\_\{t\}\}\{2\(1\-\\bar\{\\alpha\}\_\{t\}\)\}\\\|\\widetilde\{y\}\_\{0\}\-y\_\{0\}\\\|^\{2\}\.Letpt:=p⁡\(yt\|𝐱;θt,π\)p\_\{t\}:=p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\},\\pi\)andpt∗:=p⁡\(yt\|𝐱;θt∗,π∗\)p\_\{t\}^\{\*\}:=p\(y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\*\},\\pi^\{\*\}\)\. Sinceθt=θα¯t\\theta\_\{t\}=\\theta\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}, we have

pt=𝔼y0∼p0\[qt\(yt∣y0\)\],pt∗=𝔼y~0∼p0∗\[qt\(yt∣y~0\)\]\.p\_\{t\}=\\mathbb\{E\}\_\{y\_\{0\}\\sim p\_\{0\}\}\[q\_\{t\}\(y\_\{t\}\\mid y\_\{0\}\)\],\\qquad p\_\{t\}^\{\*\}=\\mathbb\{E\}\_\{\\widetilde\{y\}\_\{0\}\\sim p\_\{0\}^\{\*\}\}\[q\_\{t\}\(y\_\{t\}\\mid\\widetilde\{y\}\_\{0\}\)\]\.Let𝒫⁡\(ℝ\)\\mathcal\{P\}\(\\mathbb\{R\}\)denote the space of probability measures onℝ\\mathbb\{R\}\. Forp0∗,p0∈𝒫⁡\(ℝ\)p\_\{0\}^\{\*\},p\_\{0\}\\in\\mathcal\{P\}\(\\mathbb\{R\}\), letΓ⁡\(p0∗,p0\)\\Gamma\(p\_\{0\}^\{\*\},p\_\{0\}\)be the set of joint probability measures onℝ×ℝ\\mathbb\{R\}\\times\\mathbb\{R\}with marginalsp0∗p\_\{0\}^\{\*\}andp0p\_\{0\}\. Thusγ∈Γ⁡\(p0∗,p0\)\\gamma\\in\\Gamma\(p\_\{0\}^\{\*\},p\_\{0\}\)if and only if every pair of Borel setsA,B⊆ℝA,B\\subseteq\\mathbb\{R\}satisfiesγ⁡\(A×ℝ\)=p0∗\(A\),γ⁡\(ℝ×B\)=p0\(B\)\\gamma\(A\\times\\mathbb\{R\}\)=p\_\{0\}^\{\*\}\(A\),\\,\\gamma\(\\mathbb\{R\}\\times B\)=p\_\{0\}\(B\)\. Theorem 4\.1 of[Villani \(2009\)](https://arxiv.org/html/2609.05688#bib.bib19)shows that a lower semicontinuous quadratic cost bounded from below attains its infimum\. Thus an optimal couplingγ⋆∈Γ⁡\(p0∗,p0\)\\gamma^\{\\star\}\\in\\Gamma\(p\_\{0\}^\{\*\},p\_\{0\}\)satisfies

𝔼\(y~0,y0\)∼γ⋆\[‖y~0−y0‖2\]=infγ∈Γ⁡\(p0∗,p0\)𝔼\(y~0,y0\)∼γ\[‖y~0−y0‖2\]=:W22\(p0∗,p0\)\.\\mathbb\{E\}\_\{\(\\widetilde\{y\}\_\{0\},y\_\{0\}\)\\sim\\gamma^\{\\star\}\}\[\\\|\\widetilde\{y\}\_\{0\}\-y\_\{0\}\\\|^\{2\}\]=\\inf\_\{\\gamma\\in\\Gamma\(p\_\{0\}^\{\*\},p\_\{0\}\)\}\\mathbb\{E\}\_\{\(\\widetilde\{y\}\_\{0\},y\_\{0\}\)\\sim\\gamma\}\[\\\|\\widetilde\{y\}\_\{0\}\-y\_\{0\}\\\|^\{2\}\]=:W\_\{2\}^\{2\}\(p\_\{0\}^\{\*\},p\_\{0\}\)\.The optimal couplingγ⋆\\gamma^\{\\star\}expresses both marginalspt∗p\_\{t\}^\{\*\}andptp\_\{t\}as expectations on the same probability space\.

pt∗=𝔼\(y~0,y0\)∼γ⋆\[qt\(yt∣y~0\)\],pt=𝔼\(y~0,y0\)∼γ⋆\[qt\(yt∣y0\)\]\.p\_\{t\}^\{\*\}=\\mathbb\{E\}\_\{\(\\widetilde\{y\}\_\{0\},y\_\{0\}\)\\sim\\gamma^\{\\star\}\}\[q\_\{t\}\(y\_\{t\}\\mid\\widetilde\{y\}\_\{0\}\)\],\\qquad p\_\{t\}=\\mathbb\{E\}\_\{\(\\widetilde\{y\}\_\{0\},y\_\{0\}\)\\sim\\gamma^\{\\star\}\}\[q\_\{t\}\(y\_\{t\}\\mid y\_\{0\}\)\]\.The integral log\-sum inequality is the continuous\-density form of joint convexity of relative entropy\([Cover and Thomas, 2006](https://arxiv.org/html/2609.05688#bib.bib22), Theorem 2\.7\.2\)\. It moves the expectation outside the divergence and gives the following upper bound\.

𝖪𝖫\(pt∗∥pt\)≤𝔼\(y~0,y0\)∼γ⋆\[𝖪𝖫\(qt\(⋅∣y~0\)∥qt\(⋅∣y0\)\)\]\.\\KL\(p\_\{t\}^\{\*\}\\parallel p\_\{t\}\)\\leq\\mathbb\{E\}\_\{\(\\widetilde\{y\}\_\{0\},y\_\{0\}\)\\sim\\gamma^\{\\star\}\}\\Big\[\\KL\(q\_\{t\}\(\\,\\cdot\\mid\\widetilde\{y\}\_\{0\}\)\\parallel q\_\{t\}\(\\,\\cdot\\mid y\_\{0\}\)\)\\Big\]\.Substituting the conditional KL divergence derived above gives the stated upper bound\.

𝖪𝖫\(pt∗∥pt\)≤𝔼\(y~0,y0\)∼γ⋆\[α¯t2\(1−α¯t\)∥y~0−y0∥2\]=α¯t2\(1−α¯t\)W22\(p0∗,p0\)\.\\KL\(p\_\{t\}^\{\*\}\\parallel p\_\{t\}\)\\leq\\mathbb\{E\}\_\{\(\\widetilde\{y\}\_\{0\},y\_\{0\}\)\\sim\\gamma^\{\\star\}\}\\left\[\\frac\{\\bar\{\\alpha\}\_\{t\}\}\{2\(1\-\\bar\{\\alpha\}\_\{t\}\)\}\\\|\\widetilde\{y\}\_\{0\}\-y\_\{0\}\\\|^\{2\}\\right\]=\\frac\{\\bar\{\\alpha\}\_\{t\}\}\{2\(1\-\\bar\{\\alpha\}\_\{t\}\)\}W\_\{2\}^\{2\}\(p\_\{0\}^\{\*\},p\_\{0\}\)\.Restoring the full notation for the four probability laws yields the lemma statement\. ∎

###### Proof of Lemma[A\.7](https://arxiv.org/html/2609.05688#A1.Thmstyle7)\.

Fix𝐱\\mathbf\{x\}and both parameter pairs\. Writep0:=p⁡\(y0\|𝐱;θ,π\)p\_\{0\}:=p\(y\_\{0\}\|\\mathbf\{x\};\\theta,\\pi\)andp0∗:=p⁡\(y0\|𝐱;θ∗,π∗\)p\_\{0\}^\{\*\}:=p\(y\_\{0\}\|\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\), and letD,D∗D,D^\{\*\}be the discrete laws in the lemma statement\. Sety¯0=\(−1\)z\+1⟨θ,𝐱⟩\\bar\{y\}\_\{0\}=\(\-1\)^\{z\+1\}\\langle\\theta,\\mathbf\{x\}\\ranglein Equation \([4](https://arxiv.org/html/2609.05688#S2.E4)\)\. Lemma[A\.1](https://arxiv.org/html/2609.05688#A1.Thmstyle1)givesy0=y¯0\+ε0y\_\{0\}=\\bar\{y\}\_\{0\}\+\\varepsilon\_\{0\}, withy¯0⟂⁣⟂ε0\\bar\{y\}\_\{0\}\\mathrel\{\\perp\\\!\\\!\\\!\\perp\}\\varepsilon\_\{0\}andε0∼η:=𝒩⁡\(0,1\)\\varepsilon\_\{0\}\\sim\\eta:=\\mathcal\{N\}\(0,1\), and hence

y¯0∼D,y0=y¯0\+ε0∼p0=D∗η\.\\bar\{y\}\_\{0\}\\sim D,\\quad y\_\{0\}=\\bar\{y\}\_\{0\}\+\\varepsilon\_\{0\}\\sim p\_\{0\}=D\*\\eta\.For the ground truth law, lety0∗=y¯0∗\+ε0∗y\_\{0\}^\{\*\}=\\bar\{y\}\_\{0\}^\{\*\}\+\\varepsilon\_\{0\}^\{\*\}, wherey¯0∗∼D∗\\bar\{y\}\_\{0\}^\{\*\}\\sim D^\{\*\},ε0∗∼η\\varepsilon\_\{0\}^\{\*\}\\sim\\eta, and the two terms are independent\. Thus

y0∗=y¯0∗\+ε0∗∼p0∗=D∗∗η\.y\_\{0\}^\{\*\}=\\bar\{y\}\_\{0\}^\{\*\}\+\\varepsilon\_\{0\}^\{\*\}\\sim p\_\{0\}^\{\*\}=D^\{\*\}\*\\eta\.Sinceη\\etais an even regularizing kernel, Lemma 5\.2 of[Santambrogio \(2015\)](https://arxiv.org/html/2609.05688#bib.bib20)gives

W22\(p0∗,p0\)=W22\(D∗∗η,D∗η\)≤W22\(D∗,D\)=W22\(D,D∗\)\.W\_\{2\}^\{2\}\(p\_\{0\}^\{\*\},p\_\{0\}\)=W\_\{2\}^\{2\}\(D^\{\*\}\*\\eta,D\*\\eta\)\\leq W\_\{2\}^\{2\}\(D^\{\*\},D\)=W\_\{2\}^\{2\}\(D,D^\{\*\}\)\.This establishes the desired upper bound and completes the proof\. ∎

###### Proof of Lemma[A\.8](https://arxiv.org/html/2609.05688#A1.Thmstyle8)\.

Letuudenote a dummy initial value\. We writept:=p⁡\(yt∣𝐱;θt,π\)=𝔼u\|𝐱;θ,π\[p⁡\(yt∣u\)\]p\_\{t\}:=p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)=\\mathbb\{E\}\_\{u\\mid\\mathbf\{x\};\\theta,\\pi\}\[p\(y\_\{t\}\\mid u\)\]andp⁡\(yt∣y0\)=𝔼u∼δy0\[p⁡\(yt∣u\)\]p\(y\_\{t\}\\mid y\_\{0\}\)=\\mathbb\{E\}\_\{u\\sim\\delta\_\{y\_\{0\}\}\}\[p\(y\_\{t\}\\mid u\)\]\. Lemma[A\.6](https://arxiv.org/html/2609.05688#A1.Thmstyle6)applies with the two initial lawsp⁡\(u∣𝐱;θ,π\)p\(u\\mid\\mathbf\{x\};\\theta,\\pi\)andδy0\\delta\_\{y\_\{0\}\}\. It gives

𝖪𝖫\(p\(yt∣y0\)∥p\(yt∣𝐱;θt,π\)\)≤α¯t2\(1−α¯t\)W22\(δy0,p\(u∣𝐱;θ,π\)\)\.\\KL\\left\(p\(y\_\{t\}\\mid y\_\{0\}\)\\parallel p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\\right\)\\leq\\frac\{\\bar\{\\alpha\}\_\{t\}\}\{2\(1\-\\bar\{\\alpha\}\_\{t\}\)\}W\_\{2\}^\{2\}\\left\(\\delta\_\{y\_\{0\}\},\\,p\(u\\mid\\mathbf\{x\};\\theta,\\pi\)\\right\)\.The squared 2\-Wasserstein distance betweenδy0\\delta\_\{y\_\{0\}\}and the two\-Gaussian mixture is

W22\(δy0,p⁡\(u∣𝐱;θ,π\)\)=𝔼u\|𝐱;θ,π\[\(y0−u\)2\]\\displaystyle W\_\{2\}^\{2\}\\left\(\\delta\_\{y\_\{0\}\},\\,p\(u\\mid\\mathbf\{x\};\\theta,\\pi\)\\right\)=\\mathbb\{E\}\_\{u\\mid\\mathbf\{x\};\\theta,\\pi\}\[\(y\_\{0\}\-u\)^\{2\}\]=\\displaystyle=π⁡\(1\)\[\(y0−μ0\)2\+1\]\+π⁡\(2\)\[\(y0\+μ0\)2\+1\]=y02\+μ02−2y0μ0tanh⁡ν\+1\.\\displaystyle\\pi\(1\)\\bigl\[\(y\_\{0\}\-\\mu\_\{0\}\)^\{2\}\+1\\bigr\]\+\\pi\(2\)\\bigl\[\(y\_\{0\}\+\\mu\_\{0\}\)^\{2\}\+1\\bigr\]=y\_\{0\}^\{2\}\+\\mu\_\{0\}^\{2\}\-2y\_\{0\}\\mu\_\{0\}\\tanh\\nu\+1\.The first equality holds because a point mass has one coupling with the second marginal\. The second equality usesu=\(−1\)z\+1μ0\+ε0u=\(\-1\)^\{z\+1\}\\mu\_\{0\}\+\\varepsilon\_\{0\}and𝔼⁡\[ε02\]=1\\mathbb\{E\}\[\\varepsilon\_\{0\}^\{2\}\]=1\. ∎

###### A\.9Two Losses of Stein Score Function\.

Setθt=θα¯t\\theta\_\{t\}=\\theta\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}andν=\(ln⁡π⁡\(1\)−ln⁡π⁡\(2\)\)/2\\nu=\(\\ln\\pi\(1\)\-\\ln\\pi\(2\)\)/2\. Define the candidate score by

sθt,ν\(yt,𝐱\):=∇ytlnp\(yt∣𝐱;θt,π\)\.s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\):=\\nabla\_\{y\_\{t\}\}\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\.Setθt∗=θ∗α¯t\\theta\_\{t\}^\{\\ast\}=\\theta^\{\\ast\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}andν∗=\(ln⁡π∗\(1\)−ln⁡π∗\(2\)\)/2\\nu^\{\\ast\}=\(\\ln\\pi^\{\\ast\}\(1\)\-\\ln\\pi^\{\\ast\}\(2\)\)/2\. Define the ground truth score by

sθt∗,ν∗\(yt,𝐱\):=∇ytlnp\(yt∣𝐱;θt∗,π∗\)\.s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\):=\\nabla\_\{y\_\{t\}\}\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)\.Then

𝔼yt\|𝐱;θt∗,π∗‖sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)‖2\\displaystyle\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}=𝔼y0\|𝐱;θ∗,π∗𝔼yt\|y0\[‖sθt,ν\(yt,𝐱\)‖2\]\\displaystyle=\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\\bigl\[\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\\bigr\]−2𝔼y0\|𝐱;θ∗,π∗𝔼yt\|y0\[⟨sθt,ν\(yt,𝐱\),∇ln⁡p\(yt∣y0\)⟩\]\+𝖢⁡\(μt∗,ν∗\)\.\\displaystyle\-2\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\\bigl\[\\langle s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\),\\nabla\\ln p\(y\_\{t\}\\mid y\_\{0\}\)\\rangle\\bigr\]\+\\mathsf\{C\}\(\\mu\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)\.Here𝖢⁡\(μt∗,ν∗\):=1−\[μt∗\]2𝔼yt\|𝐱;θt∗,π∗\[sech2⁡\(μt∗yt\+ν∗\)\]≥0\\mathsf\{C\}\(\\mu\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\):=1\-\[\\mu\_\{t\}^\{\\ast\}\]^\{2\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\sech^\{2\}\(\\mu\_\{t\}^\{\\ast\}y\_\{t\}\+\\nu^\{\\ast\}\)\]\\geq 0depends onμt∗=⟨θt∗,𝐱⟩\\mu\_\{t\}^\{\\ast\}=\\langle\\theta\_\{t\}^\{\\ast\},\\mathbf\{x\}\\rangleandν∗\\nu^\{\\ast\}alone\.

###### Proof\.

Differentiating the Gaussian\-mixture marginal and applying Bayes’ rule gives the denoising score identity, with∇:=∇yt\\nabla:=\\nabla\_\{y\_\{t\}\},

sθt∗,ν∗\(yt,𝐱\)≡∇lnp\(yt∣𝐱;θt∗,π∗\)=𝔼\[∇lnp\(yt∣y0\)∣yt,𝐱;θ∗,π∗\]\.s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\equiv\\nabla\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)=\\mathbb\{E\}\\\!\\left\[\\nabla\\ln p\(y\_\{t\}\\mid y\_\{0\}\)\\mid y\_\{t\},\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\\right\]\.The tower property therefore rewrites the cross\-term as

𝔼yt\|𝐱;θt∗,π∗\[⟨sθt,ν,sθt∗,ν∗⟩\]=𝔼y0\|𝐱;θ∗,π∗𝔼yt\|y0\[⟨sθt,ν,∇ln⁡p\(yt∣y0\)⟩\]\.\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\langle s\_\{\\theta\_\{t\},\\nu\},s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\\rangle\]=\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\[\\langle s\_\{\\theta\_\{t\},\\nu\},\\nabla\\ln p\(y\_\{t\}\\mid y\_\{0\}\)\\rangle\]\.Expanding the squared score difference and applying this cross\-term identity gives

𝖢:=\\displaystyle\\mathsf\{C\}:=𝔼yt\|𝐱;θt∗,π∗\[‖sθt,ν−sθt∗,ν∗‖2\]−𝔼y0\|𝐱;θ∗,π∗𝔼yt\|y0\[‖sθt,ν‖2\]\\displaystyle\\ \\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\\|s\_\{\\theta\_\{t\},\\nu\}\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\\\|^\{2\}\]\-\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\[\\\|s\_\{\\theta\_\{t\},\\nu\}\\\|^\{2\}\]\+2𝔼y0\|𝐱;θ∗,π∗𝔼yt\|y0\[⟨sθt,ν,∇ln⁡p\(yt∣y0\)⟩\]\\displaystyle\+2\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid y\_\{0\}\}\[\\langle s\_\{\\theta\_\{t\},\\nu\},\\nabla\\ln p\(y\_\{t\}\\mid y\_\{0\}\)\\rangle\]=\\displaystyle=𝔼yt\|𝐱;θt∗,π∗\[‖sθt∗,ν∗\(yt,𝐱\)‖2\],\\displaystyle\\ \\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\\|s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\],where suppressed score arguments in the first two lines are\(yt,𝐱\)\(y\_\{t\},\\mathbf\{x\}\)\. Integration by parts, the vanishing boundary term, and Lemma[A\.3](https://arxiv.org/html/2609.05688#A1.Thmstyle3)give

𝔼yt\|𝐱;θt∗,π∗\[‖sθt∗,ν∗\(yt,𝐱\)‖2\]=−𝔼yt\|𝐱;θt∗,π∗\[∇sθt∗,ν∗\(yt,𝐱\)\]=1−\[μt∗\]2𝔼yt\|𝐱;θt∗,π∗\[sech2⁡\(μt∗yt\+ν∗\)\]\.\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\\|s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\]=\-\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\nabla s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\]=1\-\[\\mu\_\{t\}^\{\\ast\}\]^\{2\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\sech^\{2\}\(\\mu\_\{t\}^\{\\ast\}y\_\{t\}\+\\nu^\{\\ast\}\)\]\.Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)shows that the conditional density is determined byyty\_\{t\},μt∗\\mu\_\{t\}^\{\\ast\}, andν∗\\nu^\{\\ast\}\. After integrating overyty\_\{t\},𝖢\\mathsf\{C\}therefore depends onμt∗\\mu\_\{t\}^\{\\ast\}andν∗\\nu^\{\\ast\}alone\. ∎

###### A\.10Asymptotic Behavior of the Scaled Score Matching Loss\.

Lety0y\_\{0\}be generated by MLR \([2](https://arxiv.org/html/2609.05688#S2.E2)\) under the ground truth parametersθ∗,π∗\\theta^\{\\ast\},\\pi^\{\\ast\}\. Letyty\_\{t\}follow SDE \([3](https://arxiv.org/html/2609.05688#S2.E3)\) at timet≥0t\\geq 0, with cumulative noise scheduleα¯t∈\(0,1\]\\bar\{\\alpha\}\_\{t\}\\in\(0,1\]\. Letμ0:=⟨θ,𝐱⟩\\mu\_\{0\}:=\\langle\\theta,\\mathbf\{x\}\\rangleandμ0∗:=⟨θ∗,𝐱⟩\\mu\_\{0\}^\{\\ast\}:=\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangledenote the estimated and ground truth means att=0t=0, such that the means at timettareμt=μ0α¯t\\mu\_\{t\}=\\mu\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}andμt∗=μ0∗α¯t\\mu\_\{t\}^\{\\ast\}=\\mu\_\{0\}^\{\\ast\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\. Letν:=\(ln⁡π⁡\(1\)−ln⁡π⁡\(2\)\)/2\\nu:=\(\\ln\\pi\(1\)\-\\ln\\pi\(2\)\)/2andν∗:=\(ln⁡π∗\(1\)−ln⁡π∗\(2\)\)/2\\nu^\{\\ast\}:=\(\\ln\\pi^\{\\ast\}\(1\)\-\\ln\\pi^\{\\ast\}\(2\)\)/2denote the estimated and ground truth imbalance parameters of the mixing weights\. Then

limα¯t→01α¯t𝔼yt\|𝐱;θt∗,π∗\[∥sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)∥2\]=\(μ0tanhν−μ0∗tanhν∗\)2,\\displaystyle\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\frac\{1\}\{\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\]=\\left\(\\mu\_\{0\}\\tanh\\nu\-\\mu\_\{0\}^\{\\ast\}\\tanh\\nu^\{\\ast\}\\right\)^\{2\},limα¯t→11α¯t𝔼yt\|𝐱;θt∗,π∗\[‖sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)‖2\]=𝔼y0\|𝐱;θ∗,π∗\[\(μ0tanh⁡\(μ0y0\+ν\)−μ0∗tanh⁡\(μ0∗y0\+ν∗\)\)2\]\.\\displaystyle\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 1\}\\frac\{1\}\{\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\]=\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\}\\left\[\\left\(\\mu\_\{0\}\\tanh\(\\mu\_\{0\}y\_\{0\}\+\\nu\)\-\\mu\_\{0\}^\{\\ast\}\\tanh\(\\mu\_\{0\}^\{\\ast\}y\_\{0\}\+\\nu^\{\\ast\}\)\\right\)^\{2\}\\right\]\.

###### Proof of Lemma[A\.10](https://arxiv.org/html/2609.05688#A1.Thmstyle10)\.

Using the Stein score functionsθt,ν\(yt,𝐱\)=−yt\+μttanh⁡\(μtyt\+ν\)s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)=\-y\_\{t\}\+\\mu\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)established in Lemma[A\.3](https://arxiv.org/html/2609.05688#A1.Thmstyle3), the difference between the estimated and ground truth score functions cancels the linear−yt\-y\_\{t\}terms\. We place the marginals at all scales of the diffusion noise level on a common probability space\. Lety¯0∗:=\(−1\)z\+1μ0∗\\bar\{y\}\_\{0\}^\{\\ast\}:=\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}, and letε∼𝒩⁡\(0,1\)\\varepsilon\\sim\\mathcal\{N\}\(0,1\)be independent ofzz\. At each scale,α¯ty¯0∗\+ε\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\,\\bar\{y\}\_\{0\}^\{\\ast\}\+\\varepsilonhas the same law asyt\|𝐱;θt∗,π∗y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\. Substituting this representation andμt=μ0α¯t\\mu\_\{t\}=\\mu\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}gives

1α¯t∥sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)∥2=\(μ0tanh⁡\(α¯tμ0y¯0∗\+α¯tμ0ε\+ν\)−μ0∗tanh⁡\(α¯tμ0∗y¯0∗\+α¯tμ0∗ε\+ν∗\)\)2⏟=:ℓμ0,μ0∗,ν,ν∗\(α¯t,y¯0∗,ε\)\.\\frac\{1\}\{\\bar\{\\alpha\}\_\{t\}\}\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}=\\underbrace\{\\left\(\\mu\_\{0\}\\tanh\\big\(\\bar\{\\alpha\}\_\{t\}\\mu\_\{0\}\\bar\{y\}\_\{0\}^\{\\ast\}\+\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\mu\_\{0\}\\varepsilon\+\\nu\\big\)\-\\mu\_\{0\}^\{\\ast\}\\tanh\\big\(\\bar\{\\alpha\}\_\{t\}\\mu\_\{0\}^\{\\ast\}\\bar\{y\}\_\{0\}^\{\\ast\}\+\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\mu\_\{0\}^\{\\ast\}\\varepsilon\+\\nu^\{\\ast\}\\big\)\\right\)^\{2\}\}\_\{=:\\ell\_\{\\mu\_\{0\},\\mu\_\{0\}^\{\\ast\},\\nu,\\nu^\{\\ast\}\}\(\\bar\{\\alpha\}\_\{t\},\\bar\{y\}\_\{0\}^\{\\ast\},\\varepsilon\)\}\.The triangle inequality and\|tanh⁡\(u\)\|≤1\|\\tanh\(u\)\|\\leq 1give the following bound for each fixed𝐱\\mathbf\{x\}\.

\|ℓμ0,μ0∗,ν,ν∗\(α¯t,y¯0∗,ε\)\|≤\(\|μ0\|\+\|μ0∗\|\)2<∞⟹𝔼yt\|𝐱;θt∗,π∗\[1α¯t‖sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)‖2\]<∞\.\|\\ell\_\{\\mu\_\{0\},\\mu\_\{0\}^\{\\ast\},\\nu,\\nu^\{\\ast\}\}\(\\bar\{\\alpha\}\_\{t\},\\bar\{y\}\_\{0\}^\{\\ast\},\\varepsilon\)\|\\leq\(\|\\mu\_\{0\}\|\+\|\\mu\_\{0\}^\{\\ast\}\|\)^\{2\}<\\infty\\implies\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\left\[\\frac\{1\}\{\\bar\{\\alpha\}\_\{t\}\}\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\\right\]<\\infty\.This integrable bound permits the limit to pass through the expectation by the dominated convergence theorem\([Durrett, 2019](https://arxiv.org/html/2609.05688#bib.bib10), Theorem 1\.5\.8\)\. The expectation over the common probability space is the iterated expectation overy¯0∗\\bar\{y\}\_\{0\}^\{\\ast\}andε\\varepsilon\. For eachc∈\{0,1\}c\\in\\\{0,1\\\}, the dominated convergence theorem gives

limα¯t→c𝔼yt\|𝐱;θt∗,π∗\[1α¯t‖sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)‖2\]=𝔼y¯0∗\|μ0∗,ν∗𝔼ε\[limα¯t→cℓμ0,μ0∗,ν,ν∗\(α¯t,y¯0∗,ε\)\]\.\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to c\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\left\[\\frac\{1\}\{\\bar\{\\alpha\}\_\{t\}\}\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\\right\]=\\mathbb\{E\}\_\{\\bar\{y\}\_\{0\}^\{\\ast\}\\mid\\mu\_\{0\}^\{\\ast\},\\nu^\{\\ast\}\}\\mathbb\{E\}\_\{\\varepsilon\}\\left\[\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to c\}\\ell\_\{\\mu\_\{0\},\\mu\_\{0\}^\{\\ast\},\\nu,\\nu^\{\\ast\}\}\(\\bar\{\\alpha\}\_\{t\},\\bar\{y\}\_\{0\}^\{\\ast\},\\varepsilon\)\\right\]\.The pointwise limit forc=0c=0is

limα¯t→0ℓμ0,μ0∗,ν,ν∗\(α¯t,y¯0∗,ε\)=\(μ0tanhν−μ0∗tanhν∗\)2\.\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\ell\_\{\\mu\_\{0\},\\mu\_\{0\}^\{\\ast\},\\nu,\\nu^\{\\ast\}\}\(\\bar\{\\alpha\}\_\{t\},\\bar\{y\}\_\{0\}^\{\\ast\},\\varepsilon\)=\\left\(\\mu\_\{0\}\\tanh\\nu\-\\mu\_\{0\}^\{\\ast\}\\tanh\\nu^\{\\ast\}\\right\)^\{2\}\.The pointwise limit forc=1c=1is

limα¯t→1ℓμ0,μ0∗,ν,ν∗\(α¯t,y¯0∗,ε\)=\(μ0tanh⁡\(μ0\(y¯0∗\+ε\)\+ν\)−μ0∗tanh⁡\(μ0∗\(y¯0∗\+ε\)\+ν∗\)\)2\.\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 1\}\\ell\_\{\\mu\_\{0\},\\mu\_\{0\}^\{\\ast\},\\nu,\\nu^\{\\ast\}\}\(\\bar\{\\alpha\}\_\{t\},\\bar\{y\}\_\{0\}^\{\\ast\},\\varepsilon\)=\\left\(\\mu\_\{0\}\\tanh\(\\mu\_\{0\}\(\\bar\{y\}\_\{0\}^\{\\ast\}\+\\varepsilon\)\+\\nu\)\-\\mu\_\{0\}^\{\\ast\}\\tanh\(\\mu\_\{0\}^\{\\ast\}\(\\bar\{y\}\_\{0\}^\{\\ast\}\+\\varepsilon\)\+\\nu^\{\\ast\}\)\\right\)^\{2\}\.Sincey¯0∗\+ε\\bar\{y\}\_\{0\}^\{\\ast\}\+\\varepsilonhas the ground truth law ofy0\|𝐱;θ∗,π∗y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}, the last display gives the second equality\. ∎

###### A\.11Wasserstein\-2 Distance between Two Discrete Distributions\.

Fix𝐱\\mathbf\{x\}\. Define the discrete distributionsD≡D⁡\(𝐱,θ,π\):=π⁡\(1\)δ⟨θ,𝐱⟩\+π⁡\(2\)δ−⟨θ,𝐱⟩D\\equiv D\(\\mathbf\{x\};\\theta,\\pi\):=\\pi\(1\)\\delta\_\{\\langle\\theta,\\mathbf\{x\}\\rangle\}\+\\pi\(2\)\\delta\_\{\-\\langle\\theta,\\mathbf\{x\}\\rangle\}andD∗≡D⁡\(𝐱,θ∗,π∗\):=π∗\(1\)δ⟨θ∗,𝐱⟩\+π∗\(2\)δ−⟨θ∗,𝐱⟩D^\{\*\}\\equiv D\(\\mathbf\{x\};\\theta^\{\*\},\\pi^\{\*\}\):=\\pi^\{\*\}\(1\)\\delta\_\{\\langle\\theta^\{\*\},\\mathbf\{x\}\\rangle\}\+\\pi^\{\*\}\(2\)\\delta\_\{\-\\langle\\theta^\{\*\},\\mathbf\{x\}\\rangle\}\. Then forμ0≡⟨θ,𝐱⟩,μ0∗≡⟨θ∗,𝐱⟩\\mu\_\{0\}\\equiv\\langle\\theta,\\mathbf\{x\}\\rangle,\\mu\_\{0\}^\{\\ast\}\\equiv\\langle\\theta^\{\*\},\\mathbf\{x\}\\rangleandν≡\(ln⁡π⁡\(1\)−ln⁡π⁡\(2\)\)/2,ν∗≡\(ln⁡π∗\(1\)−ln⁡π∗\(2\)\)/2\\nu\\equiv\(\\ln\\pi\(1\)\-\\ln\\pi\(2\)\)/2,\\nu^\{\\ast\}\\equiv\(\\ln\\pi^\{\*\}\(1\)\-\\ln\\pi^\{\*\}\(2\)\)/2, we have

W22\(D,D∗\)=\(\|μ0\|−\|μ0∗\|\)2\+2\|μ0\|\|μ0∗\|\|sgn\(μ0\)tanhν−sgn\(μ0∗\)tanhν∗\|\.W\_\{2\}^\{2\}\(D,D^\{\*\}\)=\(\|\\mu\_\{0\}\|\-\|\\mu\_\{0\}^\{\\ast\}\|\)^\{2\}\+2\|\\mu\_\{0\}\|\|\\mu\_\{0\}^\{\\ast\}\|\\left\|\\textnormal\{sgn\}\(\\mu\_\{0\}\)\\tanh\\nu\-\\textnormal\{sgn\}\(\\mu\_\{0\}^\{\\ast\}\)\\tanh\\nu^\{\\ast\}\\right\|\.

###### Proof of Lemma[A\.11](https://arxiv.org/html/2609.05688#A1.Thmstyle11)\.

We adoptsgn\(0\)=0\\textnormal\{sgn\}\(0\)=0and set

a:=\|μ0\|,a∗:=\|μ0∗\|,q:=sgn\(μ0\)tanhν,q∗:=sgn\(μ0∗\)tanhν∗\.a:=\|\\mu\_\{0\}\|,\\qquad a^\{\\ast\}:=\|\\mu\_\{0\}^\{\\ast\}\|,\\qquad q:=\\textnormal\{sgn\}\(\\mu\_\{0\}\)\\tanh\\nu,\\qquad q^\{\\ast\}:=\\textnormal\{sgn\}\(\\mu\_\{0\}^\{\\ast\}\)\\tanh\\nu^\{\\ast\}\.Ifaa∗=0aa^\{\\ast\}=0, the claimed identity follows\. Otherwise,DDis the law ofaSaS, whereS∈\{−1,1\}S\\in\\\{\-1,1\\\},ℙ⁡\(S=1\)=\(1\+q\)/2\\mathbb\{P\}\(S=1\)=\(1\+q\)/2, andtanh⁡ν=π⁡\(1\)−π⁡\(2\)\\tanh\\nu=\\pi\(1\)\-\\pi\(2\)\. Likewise,D∗D^\{\*\}is the law ofa∗S∗a^\{\\ast\}S^\{\\ast\}withℙ⁡\(S∗=1\)=\(1\+q∗\)/2\\mathbb\{P\}\(S^\{\\ast\}=1\)=\(1\+q^\{\\ast\}\)/2\. For any coupling ofSSandS∗S^\{\\ast\},

𝔼⁡\[\(aS−a∗S∗\)2\]=a2\+\(a∗\)2−2aa∗𝔼\[SS∗\]\.\\mathbb\{E\}\[\(aS\-a^\{\\ast\}S^\{\\ast\}\)^\{2\}\]=a^\{2\}\+\(a^\{\\ast\}\)^\{2\}\-2aa^\{\\ast\}\\mathbb\{E\}\[SS^\{\\ast\}\]\.The maximal coupling hasℙ⁡\(S≠S∗\)=\|q−q∗\|/2\\mathbb\{P\}\(S\\neq S^\{\\ast\}\)=\|q\-q^\{\\ast\}\|/2, and hence𝔼⁡\[SS∗\]=1−\|q−q∗\|\\mathbb\{E\}\[SS^\{\\ast\}\]=1\-\|q\-q^\{\\ast\}\|\. It therefore minimizes the quadratic cost, giving

W22\(D,D∗\)=\(a−a∗\)2\+2aa∗\|q−q∗\|,W\_\{2\}^\{2\}\(D,D^\{\*\}\)=\(a\-a^\{\\ast\}\)^\{2\}\+2aa^\{\\ast\}\|q\-q^\{\\ast\}\|,which is the stated formula\. ∎

###### A\.12Expectations of Gaussians, Lemma E\.2 of[Luo and Hashemi \(2025\)](https://arxiv.org/html/2609.05688#bib.bib25)\.

We use the following identities from Lemma E\.2 of[Luo and Hashemi \(2025\)](https://arxiv.org/html/2609.05688#bib.bib25)\. Letggandg′g^\{\\prime\}be standard Gaussian variables with𝔼⁡\[gg′\]=sin⁡φ\\mathbb\{E\}\[gg^\{\\prime\}\]=\\sin\\varphi, whereφ∈\[−π/2,π/2\]\\varphi\\in\[\-\\uppi/2,\\uppi/2\]\. Then

𝔼⁡\[sgn\(gg′\)\]=2πφ,𝔼⁡\[\|gg′\|\]=2π\[φsin⁡φ\+cos⁡φ\],𝔼⁡\[g2sgn\(gg′\)\]=2π\[φ\+sin⁡φcos⁡φ\]\.\\mathbb\{E\}\\left\[\\textnormal\{sgn\}\\left\(gg^\{\\prime\}\\right\)\\right\]=\\frac\{2\}\{\\uppi\}\\varphi,\\quad\\mathbb\{E\}\\left\[\\left\|gg^\{\\prime\}\\right\|\\right\]=\\frac\{2\}\{\\uppi\}\[\\varphi\\sin\\varphi\+\\cos\\varphi\],\\quad\\mathbb\{E\}\\left\[g^\{2\}\\textnormal\{sgn\}\\left\(gg^\{\\prime\}\\right\)\\right\]=\\frac\{2\}\{\\uppi\}\[\\varphi\+\\sin\\varphi\\cos\\varphi\]\.

###### A\.13Expectation of Wasserstein\-2 Distance between Two Discrete Distributions\.

Suppose that𝐱\\mathbf\{x\}is independent of\(z,ε0\)\(z,\\varepsilon\_\{0\}\)and𝔼⁡\[𝐱𝐱⊤\]=Id\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]=I\_\{d\}\. ThenD,D∗D,D^\{\*\}from Lemma[A\.11](https://arxiv.org/html/2609.05688#A1.Thmstyle11)satisfy

𝔼𝐱W22\(D,D∗\)\\displaystyle\\mathbb\{E\}\_\{\\mathbf\{x\}\}W\_\{2\}^\{2\}\(D,D^\{\*\}\)=‖θ‖2\+‖θ∗‖2−2⟨θ,θ∗⟩sgn\(νν∗\)tanh⁡\(\|ν\|∧\|ν∗\|\)−2𝔼𝐱\[\|μ0μ0∗\|\]\(1−tanh⁡\(\|ν\|∨\|ν∗\|\)\)\\displaystyle=\\\|\\theta\\\|^\{2\}\+\\\|\\theta^\{\*\}\\\|^\{2\}\-2\\langle\\theta,\\theta^\{\*\}\\rangle\\textnormal\{sgn\}\(\\nu\\nu^\{\*\}\)\\tanh\(\|\\nu\|\\wedge\|\\nu^\{\*\}\|\)\-2\\mathbb\{E\}\_\{\\mathbf\{x\}\}\[\|\\mu\_\{0\}\\mu\_\{0\}^\{\*\}\|\]\(1\-\\tanh\(\|\\nu\|\\vee\|\\nu^\{\*\}\|\)\)≤‖θ‖2\+‖θ∗‖2−2⟨θ,θ∗⟩sgn\(νν∗\)tanh⁡\(\|ν\|∧\|ν∗\|\)−2\|⟨θ,θ∗⟩\|\(1−tanh⁡\(\|ν\|∨\|ν∗\|\)\)\.\\displaystyle\\leq\\\|\\theta\\\|^\{2\}\+\\\|\\theta^\{\*\}\\\|^\{2\}\-2\\langle\\theta,\\theta^\{\*\}\\rangle\\textnormal\{sgn\}\(\\nu\\nu^\{\*\}\)\\tanh\(\|\\nu\|\\wedge\|\\nu^\{\*\}\|\)\-2\|\\langle\\theta,\\theta^\{\*\}\\rangle\|\(1\-\\tanh\(\|\\nu\|\\vee\|\\nu^\{\*\}\|\)\)\.In particular, suppose that𝐱∼𝒩⁡\(𝟎,Id\)\\mathbf\{x\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},I\_\{d\}\)andθ,θ∗≠𝟎\\theta,\\theta^\{\*\}\\neq\\mathbf\{0\}\. Defineρ:=⟨θ,θ∗⟩‖θ‖‖θ∗‖\\rho:=\\frac\{\\langle\\theta,\\theta^\{\*\}\\rangle\}\{\\\|\\theta\\\|\\\|\\theta^\{\*\}\\\|\}andφ:=arcsin⁡\|ρ\|\\varphi:=\\arcsin\|\\rho\|, sosin⁡φ=\|ρ\|\\sin\\varphi=\|\\rho\|\. Then

𝔼𝐱∼𝒩⁡\(𝟎,Id\)W22\(D,D∗\)\\displaystyle\\mathbb\{E\}\_\{\\mathbf\{x\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},I\_\{d\}\)\}W\_\{2\}^\{2\}\(D,D^\{\*\}\)=\\displaystyle=‖θ‖2\+‖θ∗‖2−2‖θ‖‖θ∗‖\[sgn\(ρνν∗\)tanh⁡\(\|ν\|∧\|ν∗\|\)sin⁡φ\+\(1−tanh⁡\(\|ν\|∨\|ν∗\|\)\)2π\[φsin⁡φ\+cos⁡φ\]\]\.\\displaystyle\\\|\\theta\\\|^\{2\}\+\\\|\\theta^\{\*\}\\\|^\{2\}\-2\\\|\\theta\\\|\\\|\\theta^\{\*\}\\\|\\left\[\\textnormal\{sgn\}\(\\rho\\nu\\nu^\{\\ast\}\)\\tanh\(\|\\nu\|\\wedge\|\\nu^\{\*\}\|\)\\sin\\varphi\+\(1\-\\tanh\(\|\\nu\|\\vee\|\\nu^\{\*\}\|\)\)\\frac\{2\}\{\\uppi\}\[\\varphi\\sin\\varphi\+\\cos\\varphi\]\\right\]\.

###### Proof of Lemma[A\.13](https://arxiv.org/html/2609.05688#A1.Thmstyle13)\.

WriteA:=μ0=⟨θ,𝐱⟩A:=\\mu\_\{0\}=\\langle\\theta,\\mathbf\{x\}\\rangle,B:=μ0∗=⟨θ∗,𝐱⟩B:=\\mu\_\{0\}^\{\\ast\}=\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle, and

m:=tanh⁡\(\|ν\|∧\|ν∗\|\),M:=tanh⁡\(\|ν\|∨\|ν∗\|\)\.m:=\\tanh\(\|\\nu\|\\wedge\|\\nu^\{\\ast\}\|\),\\qquad M:=\\tanh\(\|\\nu\|\\vee\|\\nu^\{\\ast\}\|\)\.Since\|tanh⁡ν\|=tanh⁡\|ν\|\|\\tanh\\nu\|=\\tanh\|\\nu\|, the identity\|a−b\|=max⁡\{\|a\|,\|b\|\}−sgn\(ab\)min⁡\{\|a\|,\|b\|\}\|a\-b\|=\\max\\\{\|a\|,\|b\|\\\}\-\\textnormal\{sgn\}\(ab\)\\min\\\{\|a\|,\|b\|\\\}gives

\|AB\|\|sgn\(A\)tanhν−sgn\(B\)tanhν∗\|=\|AB\|M−ABsgn\(νν∗\)m\.\|AB\|\\left\|\\textnormal\{sgn\}\(A\)\\tanh\\nu\-\\textnormal\{sgn\}\(B\)\\tanh\\nu^\{\\ast\}\\right\|=\|AB\|M\-AB\\,\\textnormal\{sgn\}\(\\nu\\nu^\{\\ast\}\)m\.Lemma[A\.11](https://arxiv.org/html/2609.05688#A1.Thmstyle11)therefore yields the pointwise identity

W22\(D,D∗\)=A2\+B2−2ABsgn\(νν∗\)m−2\|AB\|\(1−M\)\.W\_\{2\}^\{2\}\(D,D^\{\*\}\)=A^\{2\}\+B^\{2\}\-2AB\\,\\textnormal\{sgn\}\(\\nu\\nu^\{\\ast\}\)m\-2\|AB\|\(1\-M\)\.Taking expectations and using𝔼⁡\[𝐱𝐱⊤\]=Id\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]=I\_\{d\}proves the equality\. Moreover,𝔼\|AB\|≥\|𝔼⁡\[AB\]\|=\|⟨θ,θ∗⟩\|\\mathbb\{E\}\|AB\|\\geq\|\\mathbb\{E\}\[AB\]\|=\|\\langle\\theta,\\theta^\{\\ast\}\\rangle\|\. Since1−M≥01\-M\\geq 0, this proves the stated inequality\.

For the Gaussian specialization, the normalized projectionsG:=A/‖θ‖G:=A/\\\|\\theta\\\|andG∗:=B/‖θ∗‖G^\{\\ast\}:=B/\\\|\\theta^\{\\ast\}\\\|are standard Gaussian with correlationρ\\rho\. Lemma[A\.12](https://arxiv.org/html/2609.05688#A1.Thmstyle12), applied with the signed anglearcsin⁡ρ\\arcsin\\rho, gives

𝔼\|AB\|=2‖θ‖‖θ∗‖π\(φsin⁡φ\+cos⁡φ\),𝔼⁡\[AB\]=sgn\(ρ\)‖θ‖‖θ∗‖sin⁡φ\.\\mathbb\{E\}\|AB\|=\\frac\{2\\\|\\theta\\\|\\\|\\theta^\{\\ast\}\\\|\}\{\\uppi\}\(\\varphi\\sin\\varphi\+\\cos\\varphi\),\\qquad\\mathbb\{E\}\[AB\]=\\textnormal\{sgn\}\(\\rho\)\\\|\\theta\\\|\\\|\\theta^\{\\ast\}\\\|\\sin\\varphi\.Taking expectations in the pointwise identity and substituting these moments proves the Gaussian formula\. If either parameter vector is zero, the general identity instead gives𝔼W22\(D,D∗\)=‖θ‖2\+‖θ∗‖2\\mathbb\{E\}W\_\{2\}^\{2\}\(D,D^\{\*\}\)=\\\|\\theta\\\|^\{2\}\+\\\|\\theta^\{\\ast\}\\\|^\{2\}\. ∎

###### A\.14Proposition[3\.1](https://arxiv.org/html/2609.05688#S3.Thmstyle1)\.

Suppose that0<α¯T≤10<\\bar\{\\alpha\}\_\{T\}\\leq 1,θ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\},ν∗∈ℝ\\nu^\{\\ast\}\\in\\mathbb\{R\}, and that𝐱\\mathbf\{x\}has a density that is positive almost everywhere onℝd\\mathbb\{R\}^\{d\}and finite second moments\. Define the terminal population KL divergence

𝖪𝖫T\(θ,ν\):=𝖪𝖫\(p\(yT,𝐱∣θT∗,π∗\)∥p\(yT,𝐱∣θT,π\)\)\.\\KL\_\{T\}\(\\theta,\\nu\):=\\KL\\left\(p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\},\\pi\)\\right\)\.Then

argmin\(θ,ν\)∈ℝd×ℝ⁡𝖪𝖫T\(θ,ν\)=\{\(θ∗,ν∗\),\(−θ∗,−ν∗\)\}\.\\argmin\_\{\(\\theta,\\nu\)\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}\}\\KL\_\{T\}\(\\theta,\\nu\)=\\\{\(\\theta^\{\\ast\},\\nu^\{\\ast\}\),\(\-\\theta^\{\\ast\},\-\\nu^\{\\ast\}\)\\\}\.

###### Proof of Proposition[3\.1](https://arxiv.org/html/2609.05688#S3.Thmstyle1)\.

Because the marginal distribution of𝐱\\mathbf\{x\}does not depend on the model parameters, the chain rule for KL divergence yields

𝖪𝖫T\(θ,ν\)=𝔼𝐱𝖪𝖫\(p\(yT∣𝐱;θT∗,π∗\)∥p\(yT∣𝐱;θT,π\)\)\.\\KL\_\{T\}\(\\theta,\\nu\)=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\KL\\left\(p\(y\_\{T\}\\mid\\mathbf\{x\};\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{T\}\\mid\\mathbf\{x\};\\theta\_\{T\},\\pi\)\\right\)\.Applying Pinsker’s inequality\([Cover and Thomas, 2006](https://arxiv.org/html/2609.05688#bib.bib22)\)for each𝐱\\mathbf\{x\}and then taking the expectation over𝐱\\mathbf\{x\}gives

𝖪𝖫T\(θ,ν\)≥2𝔼𝐱𝖳𝖵2\(p\(⋅∣𝐱;θT∗,π∗\),p\(⋅∣𝐱;θT,π\)\)\.\\KL\_\{T\}\(\\theta,\\nu\)\\geq 2\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\TV^\{2\}\\left\(p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\),p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\},\\pi\)\\right\)\.
We next identify the equality case\. Suppose that𝖪𝖫T⁡\(θ,ν\)=0\\KL\_\{T\}\(\\theta,\\nu\)=0\. Since the conditional KL divergence is non\-negative, it follows that

p\(⋅∣𝐱;θT,π\)=p\(⋅∣𝐱;θT∗,π∗\)as distributions for almost every𝐱\.p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\},\\pi\)=p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\)\\quad\\text\{as distributions for almost every \}\\mathbf\{x\}\.By Lemma[A\.1](https://arxiv.org/html/2609.05688#A1.Thmstyle1), the first two raw moments of the terminal conditional distribution are

𝔼yT\|𝐱;θT,π\[yT\]=μTtanh⁡ν,𝔼yT\|𝐱;θT,π\[yT2\]=1\+μT2\.\\mathbb\{E\}\_\{y\_\{T\}\\mid\\mathbf\{x\};\\theta\_\{T\},\\pi\}\[y\_\{T\}\]=\\mu\_\{T\}\\tanh\\nu,\\qquad\\mathbb\{E\}\_\{y\_\{T\}\\mid\\mathbf\{x\};\\theta\_\{T\},\\pi\}\[y\_\{T\}^\{2\}\]=1\+\\mu\_\{T\}^\{2\}\.Equality of the terminal conditional distributions andα¯T\>0\\bar\{\\alpha\}\_\{T\}\>0therefore imply

μ0tanhν=μ0∗tanhν∗,μ02=\(μ0∗\)2for almost every𝐱\.\\mu\_\{0\}\\tanh\\nu=\\mu\_\{0\}^\{\\ast\}\\tanh\\nu^\{\\ast\},\\qquad\\mu\_\{0\}^\{2\}=\(\\mu\_\{0\}^\{\\ast\}\)^\{2\}\\quad\\text\{for almost every \}\\mathbf\{x\}\.The second identity can be rewritten as

𝐱⊤\(θθ⊤−θ∗\(θ∗\)⊤\)𝐱=0almost surely\.\\mathbf\{x\}^\{\\top\}\\left\(\\theta\\theta^\{\\top\}\-\\theta^\{\\ast\}\(\\theta^\{\\ast\}\)^\{\\top\}\\right\)\\mathbf\{x\}=0\\quad\\text\{almost surely\}\.Since the density of𝐱\\mathbf\{x\}is positive almost everywhere onℝd\\mathbb\{R\}^\{d\}, the quadratic polynomial on the left\-hand side is the zero polynomial\. Hence

θθ⊤=θ∗\(θ∗\)⊤\.\\theta\\theta^\{\\top\}=\\theta^\{\\ast\}\(\\theta^\{\\ast\}\)^\{\\top\}\.Becauseθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}, equality of these rank\-one matrices impliesθ=sθ∗\\theta=s\\theta^\{\\ast\}for somes∈\{\+1,−1\}s\\in\\\{\+1,\-1\\\}\. Substitution into the first\-moment identity givesstanh⁡ν=tanh⁡ν∗s\\tanh\\nu=\\tanh\\nu^\{\\ast\}\. Oddness and strict monotonicity of the hyperbolic tangent giveν=sν∗\\nu=s\\nu^\{\\ast\}\. The joint sign change\(θ,ν\)↦\(−θ,−ν\)\(\\theta,\\nu\)\\mapsto\(\-\\theta,\-\\nu\)exchanges the two mixture components and leaves the terminal conditional distribution unchanged\. Thus

𝖪𝖫T\(θ,ν\)=0⟺\(θ,ν\)∈\{\(θ∗,ν∗\),\(−θ∗,−ν∗\)\}\.\\KL\_\{T\}\(\\theta,\\nu\)=0\\quad\\Longleftrightarrow\\quad\(\\theta,\\nu\)\\in\\\{\(\\theta^\{\\ast\},\\nu^\{\\ast\}\),\(\-\\theta^\{\\ast\},\-\\nu^\{\\ast\}\)\\\}\.Lemma[A\.11](https://arxiv.org/html/2609.05688#A1.Thmstyle11)and the same positive\-density argument show that𝔼𝐱W22\(D,D∗\)=0\\mathbb\{E\}\_\{\\mathbf\{x\}\}W\_\{2\}^\{2\}\(D,D^\{\*\}\)=0has the same two solutions\. Combining these equivalences yields the claimed characterization of the global minimizers\. ∎

###### A\.15Integrated Relative de Bruijn Identity for Conditional MLR\.

Let the population Score Matching loss beℒt\(θ,ν\):=12α¯t𝔼yt,𝐱;θt∗,π∗\[‖sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)‖2\]\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\):=\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{y\_\{t\},\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\left\[\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\\right\]as defined in Equation \([6](https://arxiv.org/html/2609.05688#S2.E6)\)\. Suppose that𝐱\\mathbf\{x\}is independent of\(z,ε0\)\(z,\\varepsilon\_\{0\}\)and thatα¯t∈\(0,1\]\\bar\{\\alpha\}\_\{t\}\\in\(0,1\]\. Then everyT≥0T\\geq 0satisfies

𝖪𝖫\(p\(y0,𝐱∣θ∗,π∗\)∥p\(y0,𝐱∣θ,π\)\)=𝖪𝖫\(p\(yT,𝐱∣θT∗,π∗\)∥p\(yT,𝐱∣θT,π\)\)\+∫α¯Tα¯0ℒt\(θ,ν\)dα¯t\.\\KL\(p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta,\\pi\)\)=\\KL\(p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\},\\pi\)\)\+\\int\_\{\\bar\{\\alpha\}\_\{T\}\}^\{\\bar\{\\alpha\}\_\{0\}\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\.

###### A\.16Log\-likelihood and Finite Sample Score Matching Loss\.

Letℙn\\mathbb\{P\}\_\{n\}be the empirical measure defined before Equation \([8](https://arxiv.org/html/2609.05688#S2.E8)\), and let𝒥tn\\mathcal\{J\}\_\{t\}^\{n\}be the empirical auxiliary loss in Equation \([8](https://arxiv.org/html/2609.05688#S2.E8)\)\. Then everyT\>0T\>0with0<α¯T<10<\\bar\{\\alpha\}\_\{T\}<1satisfies

−𝔼ℙnlnp\(y0\|𝐱;θ,π\)=𝔼ℙn𝖪𝖫\(p\(yT∣y0\)∥p\(yT\|𝐱;θT,π\)\)\+∫α¯Tα¯0𝒥tn\(θ,ν\)dα¯t\+12\(ln1−α¯Tα¯T\+ln\(2π\)\+1\)\.\-\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\ln p\(y\_\{0\}\|\\mathbf\{x\};\\theta,\\pi\)=\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\KL\\left\(p\(y\_\{T\}\\mid y\_\{0\}\)\\parallel p\(y\_\{T\}\|\\mathbf\{x\};\\theta\_\{T\},\\pi\)\\right\)\+\\int\_\{\\bar\{\\alpha\}\_\{T\}\}^\{\\bar\{\\alpha\}\_\{0\}\}\\mathcal\{J\}\_\{t\}^\{n\}\(\\theta,\\nu\)\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\+\\frac\{1\}\{2\}\\left\(\\ln\\frac\{1\-\\bar\{\\alpha\}\_\{T\}\}\{\\bar\{\\alpha\}\_\{T\}\}\+\\ln\(2\\uppi\)\+1\\right\)\.

###### Proof\.

The independence assumption givesp⁡\(𝐱∣z;θ,π\)=p⁡\(𝐱∣z;θt,π\)=p⁡\(𝐱\)p\(\\mathbf\{x\}\\mid z;\\theta,\\pi\)=p\(\\mathbf\{x\}\\mid z;\\theta\_\{t\},\\pi\)=p\(\\mathbf\{x\}\)\. Thereforep\(y0,𝐱∣θ,π\)=p\(𝐱\)p\(y0∣𝐱;θ,π\)p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta,\\pi\)=p\(\\mathbf\{x\}\)p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta,\\pi\)andp\(yt,𝐱∣θt,π\)=p\(𝐱\)p\(yt∣𝐱;θt,π\)p\(y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\},\\pi\)=p\(\\mathbf\{x\}\)p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\.

Under the ground truth,p\(y0,𝐱∣θ∗,π∗\)=p\(𝐱\)p\(y0∣𝐱;θ∗,π∗\)p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta^\{\\ast\},\\pi^\{\\ast\}\)=p\(\\mathbf\{x\}\)p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\)andp\(yt,𝐱∣θt∗,π∗\)=p\(𝐱\)p\(yt∣𝐱;θt∗,π∗\)p\(y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)=p\(\\mathbf\{x\}\)p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)\. Hence𝔼y0,𝐱\|θ∗,π∗\[⋅\]=𝔼𝐱𝔼y0\|𝐱;θ∗,π∗\[⋅\]\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\theta^\{\\ast\},\\pi^\{\\ast\}\}\[\\cdot\]=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\mathbb\{E\}\_\{y\_\{0\}\|\\mathbf\{x\};\\theta^\{\\ast\},\\pi^\{\\ast\}\}\[\\cdot\]and𝔼yt,𝐱\|θt∗,π∗\[⋅\]=𝔼𝐱𝔼yt\|𝐱;θt∗,π∗\[⋅\]\\mathbb\{E\}\_\{y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\cdot\]=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\mathbb\{E\}\_\{y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\cdot\]\.

Interchangingddt\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}and𝔼𝐱\\mathbb\{E\}\_\{\\mathbf\{x\}\}, applying Lemma[A\.4](https://arxiv.org/html/2609.05688#A1.Thmstyle4), and usingdα¯tdt=−βtα¯t\\frac\{\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\}\{\\mathrm\{d\}t\}=\-\\beta\_\{t\}\\bar\{\\alpha\}\_\{t\}gives

−ddt𝖪𝖫\(p\(yt,𝐱∣θt∗,π∗\)∥p\(yt,𝐱∣θt,π\)\)=−𝔼𝐱ddt𝖪𝖫\(p\(yt∣𝐱;θt∗,π∗\)∥p\(yt∣𝐱;θt,π\)\)\\displaystyle\-\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\KL\(p\(y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\},\\pi\)\)=\-\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}t\}\\KL\(p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)\)=\\displaystyle=βt2𝔼𝐱𝔼yt\|𝐱;θt∗,π∗\[‖sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)‖2\]=βtα¯t⋅ℒt\(θ,ν\)=−ℒt\(θ,ν\)dα¯tdt\.\\displaystyle\\frac\{\\beta\_\{t\}\}\{2\}\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\mathbb\{E\}\_\{y\_\{t\}\|\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\left\[\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\\right\]=\\beta\_\{t\}\\bar\{\\alpha\}\_\{t\}\\cdot\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\-\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\\;\\frac\{\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\}\{\\mathrm\{d\}t\}\.Integrating from00toTT, withθ0=θ\\theta\_\{0\}=\\thetaandθ0∗=θ∗\\theta\_\{0\}^\{\\ast\}=\\theta^\{\\ast\}, gives

𝖪𝖫\(p\(y0,𝐱∣θ∗,π∗\)∥p\(y0,𝐱∣θ,π\)\)−𝖪𝖫\(p\(yT,𝐱∣θT∗,π∗\)∥p\(yT,𝐱∣θT,π\)\)=∫α¯Tα¯0ℒt\(θ,ν\)dα¯t\.\\KL\(p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{0\},\\mathbf\{x\}\\mid\\theta,\\pi\)\)\-\\KL\(p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\}^\{\\ast\},\\pi^\{\\ast\}\)\\parallel p\(y\_\{T\},\\mathbf\{x\}\\mid\\theta\_\{T\},\\pi\)\)=\\int\_\{\\bar\{\\alpha\}\_\{T\}\}^\{\\bar\{\\alpha\}\_\{0\}\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\.Rearranging the terms proves the result\. ∎

###### Proof of Lemma[A\.16](https://arxiv.org/html/2609.05688#A1.Thmstyle16)\.

Fixϵ∈\(0,T\)\\epsilon\\in\(0,T\)\. We apply Lemma[A\.5](https://arxiv.org/html/2609.05688#A1.Thmstyle5)to𝒥tn\\mathcal\{J\}\_\{t\}^\{n\}, usedα¯tdt=−βtα¯t\\frac\{\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\}\{\\mathrm\{d\}t\}=\-\\beta\_\{t\}\\bar\{\\alpha\}\_\{t\}, and integrate fromϵ\\epsilontoTT\. Under the common Gaussian coupling,yϵ→y0y\_\{\\epsilon\}\\to y\_\{0\}inL2L^\{2\}andθϵ→θ\\theta\_\{\\epsilon\}\\to\\thetaasϵ↓0\\epsilon\\downarrow 0\. The log\-density is continuous and has a quadratic envelope for each fixed empirical sample, so dominated convergence applies\. We therefore letϵ↓0\\epsilon\\downarrow 0, interpret the integral at its upper endpoint as an improper integral, and obtain

−𝔼ℙnln⁡p⁡\(y0∣𝐱;θ,π\)\+𝔼ℙn𝔼yT\|y0ln⁡p⁡\(yT\|𝐱;θT,π\)=∫α¯Tα¯0𝒥tn\(θ,ν\)dα¯t\+12ln1α¯T\.\-\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\ln p\(y\_\{0\}\\mid\\mathbf\{x\};\\theta,\\pi\)\+\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{y\_\{T\}\\mid y\_\{0\}\}\\ln p\(y\_\{T\}\|\\mathbf\{x\};\\theta\_\{T\},\\pi\)=\\int\_\{\\bar\{\\alpha\}\_\{T\}\}^\{\\bar\{\\alpha\}\_\{0\}\}\\mathcal\{J\}\_\{t\}^\{n\}\(\\theta,\\nu\)\\mathrm\{d\}\\bar\{\\alpha\}\_\{t\}\+\\frac\{1\}\{2\}\\ln\\frac\{1\}\{\\bar\{\\alpha\}\_\{T\}\}\.The Gaussian transition satisfies

p⁡\(yT∣y0\)\\displaystyle p\(y\_\{T\}\\mid y\_\{0\}\)=𝒩⁡\(yT,y0α¯T,1−α¯T\),\\displaystyle=\\mathcal\{N\}\(y\_\{T\};y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{T\}\},1\-\\bar\{\\alpha\}\_\{T\}\),−ln⁡p⁡\(yT∣y0\)\\displaystyle\-\\ln p\(y\_\{T\}\\mid y\_\{0\}\)=12ln⁡\(2π\(1−α¯T\)\)\+‖yT−y0α¯T‖22\(1−α¯T\),\\displaystyle=\\frac\{1\}\{2\}\\ln\\\!\\bigl\(2\\uppi\(1\-\\bar\{\\alpha\}\_\{T\}\)\\bigr\)\+\\frac\{\\\|y\_\{T\}\-y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{T\}\}\\\|^\{2\}\}\{2\(1\-\\bar\{\\alpha\}\_\{T\}\)\},𝔼yT\|y0\[‖yT−y0α¯T‖2\]\\displaystyle\\mathbb\{E\}\_\{y\_\{T\}\\mid y\_\{0\}\}\\\!\\left\[\\\|y\_\{T\}\-y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{T\}\}\\\|^\{2\}\\right\]=1−α¯T\.\\displaystyle=1\-\\bar\{\\alpha\}\_\{T\}\.These identities give

−𝔼yT\|y0ln⁡p⁡\(yT\|𝐱;θT,π\)\\displaystyle\-\\mathbb\{E\}\_\{y\_\{T\}\\mid y\_\{0\}\}\\ln p\(y\_\{T\}\|\\mathbf\{x\};\\theta\_\{T\},\\pi\)=𝖪𝖫\(p\(yT∣y0\)∥p\(yT\|𝐱;θT,π\)\)−𝔼yT\|y0\[lnp\(yT∣y0\)\]\\displaystyle=\\KL\(p\(y\_\{T\}\\mid y\_\{0\}\)\\parallel p\(y\_\{T\}\|\\mathbf\{x\};\\theta\_\{T\},\\pi\)\)\-\\mathbb\{E\}\_\{y\_\{T\}\\mid y\_\{0\}\}\[\\ln p\(y\_\{T\}\\mid y\_\{0\}\)\]=𝖪𝖫\(p\(yT∣y0\)∥p\(yT\|𝐱;θT,π\)\)\+12\(ln\(1−α¯T\)\+ln\(2π\)\+1\)\.\\displaystyle=\\KL\(p\(y\_\{T\}\\mid y\_\{0\}\)\\parallel p\(y\_\{T\}\|\\mathbf\{x\};\\theta\_\{T\},\\pi\)\)\+\\frac\{1\}\{2\}\\left\(\\ln\(1\-\\bar\{\\alpha\}\_\{T\}\)\+\\ln\(2\\uppi\)\+1\\right\)\.Taking𝔼ℙn\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}, substituting into the first identity, and rearranging the terms proves the result\. ∎

###### A\.17Derivatives of Negative Log\-Likelihood\.

LetF⁡\(yt,μt,ν\):=−ln⁡p⁡\(yt∣𝐱;θt,π\)F\(y\_\{t\},\\mu\_\{t\},\\nu\):=\-\\ln p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\},\\pi\)be the negative log\-likelihood from Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)\. Then

yt∂ytF⁡\(yt,μt,ν\)−yt2=μt∂μtF⁡\(yt,μt,ν\)−μt2=\(∂νF⁡\(yt,μt,ν\)−tanh⁡ν\)μtyt=−tanh⁡\(μtyt\+ν\)μtyt\.y\_\{t\}\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\-y\_\{t\}^\{2\}=\\mu\_\{t\}\\partial\_\{\\mu\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\-\\mu\_\{t\}^\{2\}=\(\\partial\_\{\\nu\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\-\\tanh\\nu\)\\mu\_\{t\}y\_\{t\}=\-\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\mu\_\{t\}y\_\{t\}\.Moreover,∂ytyt2F⁡\(yt,μt,ν\)=1−μt2sech2⁡\(μtyt\+ν\)\\partial^\{2\}\_\{y\_\{t\}y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)=1\-\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\), and

‖∂ytF⁡\(yt,μt,ν\)‖2−∂ytyt2F⁡\(yt,μt,ν\)=2yt∂ytF⁡\(yt,μt,ν\)\+\(μt2−yt2−1\)\.\\\|\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\\\|^\{2\}\-\\partial^\{2\}\_\{y\_\{t\}y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)=2y\_\{t\}\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\+\\left\(\\mu\_\{t\}^\{2\}\-y\_\{t\}^\{2\}\-1\\right\)\.

###### Proof\.

Differentiating the expression forFFfrom Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)with respect toyty\_\{t\},μt\\mu\_\{t\}, andν\\nugives

∂ytF⁡\(yt,μt,ν\)=yt−tanh⁡\(μtyt\+ν\)μt,∂μtF⁡\(yt,μt,ν\)=μt−tanh⁡\(μtyt\+ν\)yt,\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)=y\_\{t\}\-\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\mu\_\{t\},\\;\\partial\_\{\\mu\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)=\\mu\_\{t\}\-\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)y\_\{t\},∂νF⁡\(yt,μt,ν\)=tanh⁡ν−tanh⁡\(μtyt\+ν\)\.\\partial\_\{\\nu\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)=\\tanh\\nu\-\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\.We complete the proof of the first identity by rearranging terms in the above expressions and multiplying byyt,μty\_\{t\},\\mu\_\{t\}andμtyt\\mu\_\{t\}y\_\{t\}to obtain−tanh⁡\(μtyt\+ν\)μtyt\-\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\mu\_\{t\}y\_\{t\}\. For the second identity, usetanh2⁡\(⋅\)=1−sech2⁡\(⋅\)\\tanh^\{2\}\(\\cdot\)=1\-\\sech^\{2\}\(\\cdot\),dtanh\(⋅\)/d\(⋅\)=sech2\(⋅\)\\mathrm\{d\}\\tanh\(\\cdot\)/\\mathrm\{d\}\(\\cdot\)=\\sech^\{2\}\(\\cdot\), and∂ytF⁡\(yt,μt,ν\)=yt−tanh⁡\(μtyt\+ν\)μt\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)=y\_\{t\}\-\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\mu\_\{t\}\. Then

∂ytyt2F⁡\(yt,μt,ν\)\\displaystyle\\partial^\{2\}\_\{y\_\{t\}y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)=1−μt2sech2⁡\(μtyt\+ν\)=1−μt2\+μt2tanh2⁡\(μtyt\+ν\)=1−μt2\+‖yt−∂ytF⁡\(yt,μt,ν\)‖2\\displaystyle=1\-\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)=1\-\\mu\_\{t\}^\{2\}\+\\mu\_\{t\}^\{2\}\\tanh^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)=1\-\\mu\_\{t\}^\{2\}\+\\\|y\_\{t\}\-\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\\\|^\{2\}=‖∂ytF⁡\(yt,μt,ν\)‖2−2yt∂ytF⁡\(yt,μt,ν\)\+\(yt2\+1−μt2\)\.\\displaystyle=\\\|\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\\\|^\{2\}\-2y\_\{t\}\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\+\\left\(y\_\{t\}^\{2\}\+1\-\\mu\_\{t\}^\{2\}\\right\)\.Therefore, the last identity follows by rearranging terms in the above expressions\. ∎

###### A\.18Spiked Covariance under General Gaussian Covariance\.

Let𝐱∼𝒩⁡\(0,Σ\)\\mathbf\{x\}\\sim\\mathcal\{N\}\(0,\\Sigma\), whereΣ≻0\\Sigma\\succ 0, and define𝒞∗:=𝔼⁡\[𝐱𝐱⊤\(𝐱⊤θ∗\)2\]\\mathcal\{C\}^\{\\ast\}:=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\(\\mathbf\{x\}^\{\\top\}\\theta^\{\\ast\}\)^\{2\}\]\. Withc:=\(θ∗\)⊤Σθ∗c:=\(\\theta^\{\\ast\}\)^\{\\top\}\\Sigma\\theta^\{\\ast\}anda:=Σ1/2θ∗a:=\\Sigma^\{1/2\}\\theta^\{\\ast\},

𝒞∗=cΣ\+2Σθ∗\(Σθ∗\)⊤,Σ−1/2𝒞∗Σ−1/2=cId\+2aa⊤\.\\mathcal\{C\}^\{\\ast\}=c\\Sigma\+2\\Sigma\\theta^\{\\ast\}\(\\Sigma\\theta^\{\\ast\}\)^\{\\top\},\\qquad\\Sigma^\{\-1/2\}\\mathcal\{C\}^\{\\ast\}\\Sigma^\{\-1/2\}=cI\_\{d\}\+2aa^\{\\top\}\.Ifθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}, the whitened matrix has principal eigenspacespan⁡\{a\}\\operatorname\{span\}\\\{a\\\}, with eigenvalue3c3c, and eigenvalueccona⟂a^\{\\perp\}, with multiplicityd−1d\-1\. The equivalent generalized eigenproblem𝒞∗v=λΣv\\mathcal\{C\}^\{\\ast\}v=\\lambda\\Sigma vhas leading eigenspacespan⁡\{θ∗\}\\operatorname\{span\}\\\{\\theta^\{\\ast\}\\\}, lower eigenspace\{v:v⊤Σθ∗=0\}\\\{v:v^\{\\top\}\\Sigma\\theta^\{\\ast\}=0\\\}, and generalized spectral gap2c2c\. WhenΣ=Id\\Sigma=I\_\{d\}, these are the ordinary eigenspaces of𝒞∗\\mathcal\{C\}^\{\\ast\}\.

###### Proof\.

Isserlis’ identity gives𝔼⁡\[XiXjXkXl\]=ΣijΣkl\+ΣikΣjl\+ΣilΣjk\\mathbb\{E\}\[X\_\{i\}X\_\{j\}X\_\{k\}X\_\{l\}\]=\\Sigma\_\{ij\}\\Sigma\_\{kl\}\+\\Sigma\_\{ik\}\\Sigma\_\{jl\}\+\\Sigma\_\{il\}\\Sigma\_\{jk\}\([Isserlis, 1918](https://arxiv.org/html/2609.05688#bib.bib39)\)\. Contracting this identity withθk∗θl∗\\theta\_\{k\}^\{\\ast\}\\theta\_\{l\}^\{\\ast\}yields𝒞∗=cΣ\+2Σθ∗\(Σθ∗\)⊤\\mathcal\{C\}^\{\\ast\}=c\\Sigma\+2\\Sigma\\theta^\{\\ast\}\(\\Sigma\\theta^\{\\ast\}\)^\{\\top\}, and whitening yields the displayed rank\-one form\. Since‖a‖2=c\\\|a\\\|^\{2\}=c, that matrix mapsaato3ca3caand everyz⟂az\\perp atoczcz\. Underw=Σ1/2vw=\\Sigma^\{1/2\}v,𝒞∗v=λΣv\\mathcal\{C\}^\{\\ast\}v=\\lambda\\Sigma vis equivalent toΣ−1/2𝒞∗Σ−1/2w=λw\\Sigma^\{\-1/2\}\\mathcal\{C\}^\{\\ast\}\\Sigma^\{\-1/2\}w=\\lambda w, which gives the generalized eigenspaces and gap\. ∎

###### A\.19Spiked Covariance for Elliptical Distributions\.

Let𝐱∈ℝd\\mathbf\{x\}\\in\\mathbb\{R\}^\{d\}follow a zero\-mean elliptical distribution with covarianceΣ≻0\\Sigma\\succ 0and finite fourth moments\. Define the excess kurtosis parameter

κ:=𝔼\[∥Σ−1/2𝐱∥4\]d⁡\(d\+2\)−1\>−1,\\kappa\\;:=\\;\\frac\{\\mathbb\{E\}\[\\\|\\Sigma^\{\-1/2\}\\mathbf\{x\}\\\|^\{4\}\]\}\{d\(d\+2\)\}\-1\\;\>\\;\-1,whereΣ−1/2\\Sigma^\{\-1/2\}is the symmetric inverse square root ofΣ\\Sigma\. Then the fourth\-order moments satisfy

𝔼⁡\[XiXjXkXl\]=\(1\+κ\)\(ΣijΣkl\+ΣikΣjl\+ΣilΣjk\)\.\\mathbb\{E\}\[X\_\{i\}X\_\{j\}X\_\{k\}X\_\{l\}\]=\(1\+\\kappa\)\\bigl\(\\Sigma\_\{ij\}\\Sigma\_\{kl\}\+\\Sigma\_\{ik\}\\Sigma\_\{jl\}\+\\Sigma\_\{il\}\\Sigma\_\{jk\}\\bigr\)\.The fourth cumulant tensor

Δijkl:=𝔼⁡\[XiXjXkXl\]−\(ΣijΣkl\+ΣikΣjl\+ΣilΣjk\)\\Delta\_\{ijkl\}:=\\mathbb\{E\}\[X\_\{i\}X\_\{j\}X\_\{k\}X\_\{l\}\]\-\\bigl\(\\Sigma\_\{ij\}\\Sigma\_\{kl\}\+\\Sigma\_\{ik\}\\Sigma\_\{jl\}\+\\Sigma\_\{il\}\\Sigma\_\{jk\}\\bigr\)satisfiesΔijkl=κ⁡\(ΣijΣkl\+ΣikΣjl\+ΣilΣjk\)\\Delta\_\{ijkl\}=\\kappa\\,\(\\Sigma\_\{ij\}\\Sigma\_\{kl\}\+\\Sigma\_\{ik\}\\Sigma\_\{jl\}\+\\Sigma\_\{il\}\\Sigma\_\{jk\}\)\.

For𝒞∗:=𝔼⁡\[𝐱𝐱⊤\(𝐱⊤θ∗\)2\]\\mathcal\{C\}^\{\\ast\}:=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\(\\mathbf\{x\}^\{\\top\}\\theta^\{\\ast\}\)^\{2\}\], setc:=\(θ∗\)⊤Σθ∗c:=\(\\theta^\{\\ast\}\)^\{\\top\}\\Sigma\\theta^\{\\ast\}anda:=Σ1/2θ∗a:=\\Sigma^\{1/2\}\\theta^\{\\ast\}\. Then

𝒞∗=\(1\+κ\)\[cΣ\+2Σθ∗\(Σθ∗\)⊤\],Σ−1/2𝒞∗Σ−1/2=\(1\+κ\)\(cId\+2aa⊤\)\.\\mathcal\{C\}^\{\\ast\}=\(1\+\\kappa\)\\bigl\[c\\Sigma\+2\\Sigma\\theta^\{\\ast\}\(\\Sigma\\theta^\{\\ast\}\)^\{\\top\}\\bigr\],\\qquad\\Sigma^\{\-1/2\}\\mathcal\{C\}^\{\\ast\}\\Sigma^\{\-1/2\}=\(1\+\\kappa\)\(cI\_\{d\}\+2aa^\{\\top\}\)\.Ifθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}, the whitened eigenvalues are3\(1\+κ\)c3\(1\+\\kappa\)conspan⁡\{a\}\\operatorname\{span\}\\\{a\\\}and\(1\+κ\)c\(1\+\\kappa\)cona⟂a^\{\\perp\}\. The equivalent generalized problem𝒞∗v=λΣv\\mathcal\{C\}^\{\\ast\}v=\\lambda\\Sigma vhas leading eigenspacespan⁡\{θ∗\}\\operatorname\{span\}\\\{\\theta^\{\\ast\}\\\}, lower eigenspace\{v:v⊤Σθ∗=0\}\\\{v:v^\{\\top\}\\Sigma\\theta^\{\\ast\}=0\\\}, and generalized spectral gap2\(1\+κ\)c2\(1\+\\kappa\)c\. ForΣ=Id\\Sigma=I\_\{d\}, these are ordinary eigenspaces\.

###### Proof\.

Write𝐱=RΣ1/2𝐮\\mathbf\{x\}=R\\Sigma^\{1/2\}\\mathbf\{u\}, whereR≥0R\\geq 0is independent of the uniform direction𝐮∈𝕊d−1\\mathbf\{u\}\\in\\mathbb\{S\}^\{d\-1\}\([Fang et al\., 1990](https://arxiv.org/html/2609.05688#bib.bib43), Theorem 2\.5\(iii\), p\. 30\)\. Rotational invariance implies that the fourth\-moment tensor of𝐮\\mathbf\{u\}has the formcu\(δijδkl\+δikδjl\+δilδjk\)c\_\{u\}\(\\delta\_\{ij\}\\delta\_\{kl\}\+\\delta\_\{ik\}\\delta\_\{jl\}\+\\delta\_\{il\}\\delta\_\{jk\}\)for a scalarcuc\_\{u\}\. Taking the double trace and using‖𝐮‖=1\\\|\\mathbf\{u\}\\\|=1givescu=1/\[d⁡\(d\+2\)\]c\_\{u\}=1/\[d\(d\+2\)\], and therefore

𝔼⁡\[uiujukul\]=δijδkl\+δikδjl\+δilδjkd⁡\(d\+2\)\.\\mathbb\{E\}\[u\_\{i\}u\_\{j\}u\_\{k\}u\_\{l\}\]=\\frac\{\\delta\_\{ij\}\\delta\_\{kl\}\+\\delta\_\{ik\}\\delta\_\{jl\}\+\\delta\_\{il\}\\delta\_\{jk\}\}\{d\(d\+2\)\}\.Together withR=∥Σ−1/2𝐱∥R=\\\|\\Sigma^\{\-1/2\}\\mathbf\{x\}\\\|, this identity gives the stated fourth\-moment formula\. Contracting it withθk∗θl∗\\theta\_\{k\}^\{\\ast\}\\theta\_\{l\}^\{\\ast\}gives the formula for𝒞∗\\mathcal\{C\}^\{\\ast\}\. The remaining claims follow from the whitened calculation in Lemma[A\.18](https://arxiv.org/html/2609.05688#A1.Thmstyle18), multiplied by1\+κ1\+\\kappa\. ∎

### Appendix BProofs of Maximum\-Likelihood Asymptotics for Path\-Integrated Score Matching

This appendix uses the likelihood bridges from Appendix[A](https://arxiv.org/html/2609.05688#A1)to restate and prove Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2), including consistency modulo joint sign, the aligned Gaussian limit with inverse\-Fisher covariance, and the attainment argument required by the empirical estimator\.

###### B\.1Main Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)\.

The ground\-truth parameters\(θ∗,ν∗\)∈ℝd×ℝ\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}satisfyθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}\. The deterministic terminal\-horizon sequence\{Tn\}n≥1\\\{T\_\{n\}\\\}\_\{n\\geq 1\}satisfiesln⁡1−α¯Tnα¯Tn−ln⁡n→∞\\ln\\frac\{1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\-\\ln n\\to\\infty, and the covariate𝐱\\mathbf\{x\}has a density that is positive almost everywhere and satisfies𝔼‖𝐱‖2<∞\\mathbb\{E\}\\\|\\mathbf\{x\}\\\|^\{2\}<\\infty\. The estimatorϑ^n,Tn𝖲𝖬≡\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\equiv\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)isconsistentup to a joint sign change\. We choosesn∈argmins∈\{\+1,−1\}⁡‖s⁡\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)−\(θ∗,ν∗\)‖s\_\{n\}\\in\\argmin\_\{s\\in\\\{\+1,\-1\\\}\}\\left\\\|s\\,\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\right\)\-\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\right\\\|\. Then, asn→∞n\\to\\infty,sn\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)→ℙ\(θ∗,ν∗\)\.s\_\{n\}\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\right\)\\xrightarrow\{\\mathbb\{P\}\}\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\.If𝔼⁡\[‖𝐱‖4\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{4\}\]<\\infty, then the aligned Score Matching estimator isasymptotically normal\. Asn→∞n\\to\\infty,n\[sn\(θ^n,Tn𝖲𝖬,ν^n,Tn𝖲𝖬\)−\(θ∗,ν∗\)\]→𝖽𝒩⁡\(𝟎,I\(θ∗,ν∗\)−1\),\\sqrt\{n\}\\left\[s\_\{n\}\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\},\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\right\)\-\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\right\]\\xrightarrow\{\\mathsf\{d\}\}\\mathcal\{N\}\(\\mathbf\{0\},I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)^\{\-1\}\),whereIIdenotes the Fisher information matrix\.
###### Proof of Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)\.

This proof adapts the DDPM\-to\-MLE argument of[Chewi et al\. \(2025a\)](https://arxiv.org/html/2609.05688#bib.bib18)to conditional MLR with joint sign symmetry\. The model\-specific part identifies the two label\-equivalent limits\. The local M\-estimator argument is then applied after sign alignment\.

Proof\-specific notation\.In addition to the notation fixed in Section[2\.4](https://arxiv.org/html/2609.05688#S2.SS4), define

- •ℰn\(ϑ\):=ℛ^n𝖬𝖫𝖤\(ϑ\)−ℛ^n𝖬𝖫𝖤\(ϑ∗\)\\mathcal\{E\}\_\{n\}\(\\vartheta\):=\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\)\-\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta^\{\\ast\}\)andℰ⁡\(ϑ\):=𝔼⁡\[ℰn\(ϑ\)\]\\mathcal\{E\}\(\\vartheta\):=\\mathbb\{E\}\[\\mathcal\{E\}\_\{n\}\(\\vartheta\)\], where the expectation is over the i\.i\.d\. sample\. These quantities are the empirical contrast and the population excess risk\.
- •cn:=α¯Tn/\[2\(1−α¯Tn\)\]c\_\{n\}:=\\bar\{\\alpha\}\_\{T\_\{n\}\}/\[2\(1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\]andΣn:=𝔼ℙn\[𝐱𝐱⊤\]\\Sigma\_\{n\}:=\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]\.
- •ϑ~n,Tn𝖲𝖬:=snϑ^n,Tn𝖲𝖬\\widetilde\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}:=s\_\{n\}\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\. This estimator is sign\-aligned in both coordinates, withsns\_\{n\}as defined in the theorem\.
- •mϑ\(y0,𝐱\):=ln⁡p⁡\(y0∣𝐱;ϑ\)m\_\{\\vartheta\}\(y\_\{0\},\\mathbf\{x\}\):=\\ln p\(y\_\{0\}\\mid\\mathbf\{x\};\\vartheta\)\. We usem˙\(y0,𝐱\)\\dot\{m\}\(y\_\{0\},\\mathbf\{x\}\)for a measurable local Lipschitz envelope ofmϑm\_\{\\vartheta\}nearϑ∗\\vartheta^\{\\ast\}\.

We setT=TnT=T\_\{n\}in Lemma[A\.16](https://arxiv.org/html/2609.05688#A1.Thmstyle16)and define𝖼⁡\(α¯Tn\):=12\(ln⁡1−α¯Tnα¯Tn\+ln⁡\(2π\)\+1\)\\mathsf\{c\}\(\\bar\{\\alpha\}\_\{T\_\{n\}\}\):=\\frac\{1\}\{2\}\\left\(\\ln\\frac\{1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\+\\ln\(2\\uppi\)\+1\\right\)\. The lemma gives the empirical bridge

ℛ^n,Tn𝖲𝖬\(ϑ\)=ℛ^n𝖬𝖫𝖤\(ϑ\)−Δn\(ϑ\)−𝖼⁡\(α¯Tn\)\.\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\)=\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\)\-\\Delta\_\{n\}\(\\vartheta\)\-\\mathsf\{c\}\(\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\.Whenever the empirical minimum is attained, exact optimality givesℛ^n,Tn𝖲𝖬\(ϑ^n,Tn𝖲𝖬\)≤ℛ^n,Tn𝖲𝖬\(ϑ∗\)\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\\leq\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta^\{\\ast\}\)\. Substitution isolates the excess riskℰn\(ϑ^n,Tn𝖲𝖬\)≡ℛ^n𝖬𝖫𝖤\(ϑ^n,Tn𝖲𝖬\)−ℛ^n𝖬𝖫𝖤\(ϑ∗\)\\mathcal\{E\}\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\\equiv\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\-\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta^\{\\ast\}\)\. SinceΔn\(ϑ∗\)≥0\\Delta\_\{n\}\(\\vartheta^\{\\ast\}\)\\geq 0, we obtain

ℰn\(ϑ^n,Tn𝖲𝖬\)≤Δn\(ϑ^n,Tn𝖲𝖬\)−Δn\(ϑ∗\)≤Δn\(ϑ^n,Tn𝖲𝖬\)\.\\mathcal\{E\}\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\\leq\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\-\\Delta\_\{n\}\(\\vartheta^\{\\ast\}\)\\leq\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\.
Step 1\. Boundedness ofθ^n,Tn𝖲𝖬\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}and control of the excess risk\. Lemma[A\.8](https://arxiv.org/html/2609.05688#A1.Thmstyle8)bounds the decaying KL divergenceΔn\\Delta\_\{n\}\. We first work on samples with a nonempty score matching argmin\. Setcn≡α¯Tn2\(1−α¯Tn\)c\_\{n\}\\equiv\\frac\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{2\(1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\}\. The Cauchy–Schwarz inequality for the empirical covariance matrixΣn\\Sigma\_\{n\}, together with\|tanh⁡\(⋅\)\|≤1\|\\tanh\(\\cdot\)\|\\leq 1, gives

ℰn\(ϑ^n,Tn𝖲𝖬\)\\displaystyle\\mathcal\{E\}\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)≤Δn\(ϑ^n,Tn𝖲𝖬\)\\displaystyle\\leq\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)≤cn𝔼ℙn\[y02\+\(θ^n,Tn𝖲𝖬\)⊤𝐱𝐱⊤θ^n,Tn𝖲𝖬−2y0\(\(θ^n,Tn𝖲𝖬\)⊤𝐱\)tanhν^n,Tn𝖲𝖬\+1\]\\displaystyle\\leq c\_\{n\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\left\[y\_\{0\}^\{2\}\+\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)^\{\\top\}\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-2y\_\{0\}\(\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)^\{\\top\}\\mathbf\{x\}\)\\tanh\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\+1\\right\]≤cn\(‖θ^n,Tn𝖲𝖬‖Σn2\+2‖θ^n,Tn𝖲𝖬‖Σn𝔼ℙn\[y02\]\+𝔼ℙn\[y02\]\+1\)\.\\displaystyle\\leq c\_\{n\}\\left\(\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\+2\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}\\sqrt\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\}\+\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\+1\\right\)\.
Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)gives the negative log\-likelihood\. We apply\|x\|≥ln⁡cosh⁡\(x\+x′\)cosh⁡x′≥−\|x\|\|x\|\\geq\\ln\\frac\{\\cosh\(x\+x^\{\\prime\}\)\}\{\\cosh x^\{\\prime\}\}\\geq\-\|x\|and the Cauchy–Schwarz inequality forΣn\\Sigma\_\{n\}to obtain the lower bound

ℰn\(ϑ^n,Tn𝖲𝖬\)\\displaystyle\\mathcal\{E\}\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)=𝔼ℙn\[12\(θ^n,Tn𝖲𝖬\)⊤𝐱𝐱⊤θ^n,Tn𝖲𝖬−12\(θ∗\)⊤𝐱𝐱⊤θ∗−ln⁡cosh⁡\(\(θ^n,Tn𝖲𝖬\)⊤𝐱y0\+ν^n,Tn𝖲𝖬\)cosh⁡ν^n,Tn𝖲𝖬\+ln⁡cosh⁡\(\(θ∗\)⊤𝐱y0\+ν∗\)cosh⁡ν∗\]\\displaystyle=\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\left\[\\begin\{aligned\} &\\frac\{1\}\{2\}\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)^\{\\top\}\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\frac\{1\}\{2\}\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\\theta^\{\\ast\}\\\\ &\\quad\-\\ln\\frac\{\\cosh\(\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)^\{\\top\}\\mathbf\{x\}y\_\{0\}\+\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\}\{\\cosh\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\}\+\\ln\\frac\{\\cosh\(\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbf\{x\}y\_\{0\}\+\\nu^\{\\ast\}\)\}\{\\cosh\\nu^\{\\ast\}\}\\end\{aligned\}\\right\]≥12‖θ^n,Tn𝖲𝖬‖Σn2−‖θ^n,Tn𝖲𝖬‖Σn𝔼ℙn\[y02\]\\displaystyle\\geq\\frac\{1\}\{2\}\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\-\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}\\sqrt\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\}−‖θ∗‖Σn𝔼ℙn\[y02\]−12‖θ∗‖Σn2\.\\displaystyle\-\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}\\sqrt\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\}\-\\frac\{1\}\{2\}\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\.
Combining the quadratic lower bound with thecnc\_\{n\}\-scaled upper bound gives the following constraint on the empirical norm of the score matching estimator\.

\(12−cn\)⏟:=An∥θ^n,Tn𝖲𝖬∥Σn2−\(1\+2cn\)𝔼ℙn\[y02\]⏟:=Bn∥θ^n,Tn𝖲𝖬∥Σn≤cn\(𝔼ℙn\[y02\]\+1\)\+‖θ∗‖Σn𝔼ℙn\[y02\]\+12‖θ∗‖Σn2⏟Cn\.\\underbrace\{\\left\(\\frac\{1\}\{2\}\-c\_\{n\}\\right\)\}\_\{:=A\_\{n\}\}\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\-\\underbrace\{\(1\+2c\_\{n\}\)\\sqrt\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\}\}\_\{:=B\_\{n\}\}\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}\\leq\\underbrace\{c\_\{n\}\(\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\+1\)\+\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}\\sqrt\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\}\+\\frac\{1\}\{2\}\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\}\_\{C\_\{n\}\}\.
Under the time\-scaling conditionln⁡1−α¯Tnα¯Tn−ln⁡n→∞\\ln\\frac\{1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\-\\ln n\\to\\infty, it follows thatcn≡α¯Tn2\(1−α¯Tn\)=o⁡\(1/n\)c\_\{n\}\\equiv\\frac\{\\bar\{\\alpha\}\_\{T\_\{n\}\}\}\{2\(1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\)\}=o\(1/n\)\. SinceAn=12−cn→12A\_\{n\}=\\frac\{1\}\{2\}\-c\_\{n\}\\to\\frac\{1\}\{2\}, we haveAn\>0A\_\{n\}\>0for all largenn\. Since𝐱\\mathbf\{x\}has finite population second moments𝔼⁡\[𝐱𝐱⊤\]\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\], Markov’s inequality applied to the finite population expectations gives‖θ∗‖Σn2=𝒪P\(1\)\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}=\\mathcal\{O\}\_\{P\}\(1\)and𝔼ℙn\[y02\]=𝒪P\(1\)\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]=\\mathcal\{O\}\_\{P\}\(1\)\. First, consider the empirical weighted norm‖θ∗‖Σn2=\(θ∗\)⊤Σnθ∗\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}=\(\\theta^\{\\ast\}\)^\{\\top\}\\Sigma\_\{n\}\\theta^\{\\ast\}withΣn≡𝔼ℙn\[𝐱𝐱⊤\]\\Sigma\_\{n\}\\equiv\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]\. Its expectation under the data\-generating distribution is finite\. Markov’s inequality gives, for everyM\>0M\>0,

limM→∞lim supn→∞ℙ⁡\(‖θ∗‖Σn2\>M\)≤limM→∞𝔼⁡\[\(θ∗\)⊤𝔼ℙn\[𝐱𝐱⊤\]θ∗\]M=limM→∞\(θ∗\)⊤𝔼\[𝐱𝐱⊤\]θ∗M=0\.\\lim\_\{M\\to\\infty\}\\limsup\_\{n\\to\\infty\}\\mathbb\{P\}\\left\(\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\>M\\right\)\\leq\\lim\_\{M\\to\\infty\}\\frac\{\\mathbb\{E\}\\left\[\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]\\theta^\{\\ast\}\\right\]\}\{M\}=\\lim\_\{M\\to\\infty\}\\frac\{\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]\\theta^\{\\ast\}\}\{M\}=0\.Second, consider the empirical second moment𝔼ℙn\[y02\]\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\. Under the ground\-truth distribution, the response satisfiesy0=\(−1\)z\+1⟨θ∗,𝐱⟩\+ε0y\_\{0\}=\(\-1\)^\{z\+1\}\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle\+\\varepsilon\_\{0\}, where\(𝐱,z\)⟂⁣⟂ε0\(\\mathbf\{x\},z\)\\mathrel\{\\perp\\\!\\\!\\\!\\perp\}\\varepsilon\_\{0\}andε0∼𝒩⁡\(0,1\)\\varepsilon\_\{0\}\\sim\\mathcal\{N\}\(0,1\)\. Taking the expectation gives𝔼⁡\[𝔼ℙn\[y02\]\]=𝔼⁡\[y02\]=\(θ∗\)⊤𝔼\[𝐱𝐱⊤\]θ∗\+1\\mathbb\{E\}\\left\[\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\\right\]=\\mathbb\{E\}\[y\_\{0\}^\{2\}\]=\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]\\theta^\{\\ast\}\+1\. The finiteness of𝔼⁡\[𝐱𝐱⊤\]\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]and Markov’s inequality give

limM→∞lim supn→∞ℙ⁡\(𝔼ℙn\[y02\]\>M\)≤limM→∞𝔼⁡\[𝔼ℙn\[y02\]\]M=limM→∞\(θ∗\)⊤𝔼\[𝐱𝐱⊤\]θ∗\+1M=0\.\\lim\_\{M\\to\\infty\}\\limsup\_\{n\\to\\infty\}\\mathbb\{P\}\\left\(\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\>M\\right\)\\leq\\lim\_\{M\\to\\infty\}\\frac\{\\mathbb\{E\}\\left\[\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\\right\]\}\{M\}=\\lim\_\{M\\to\\infty\}\\frac\{\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]\\theta^\{\\ast\}\+1\}\{M\}=0\.The two inequalities above guarantee‖θ∗‖Σn2=𝒪P\(1\),𝔼ℙn\[y02\]=𝒪P\(1\)\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}=\\mathcal\{O\}\_\{P\}\(1\),\\,\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]=\\mathcal\{O\}\_\{P\}\(1\)\. It follows that

Bn:=\(1\+2cn\)𝔼ℙn\[y02\]=𝒪⁡\(1\)𝒪P\(1\)=𝒪P\(1\),B\_\{n\}:=\(1\+2c\_\{n\}\)\\sqrt\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\}=\\mathcal\{O\}\(1\)\\sqrt\{\\mathcal\{O\}\_\{P\}\(1\)\}=\\mathcal\{O\}\_\{P\}\(1\),Cn:=cn\(𝔼ℙn\[y02\]\+1\)\+‖θ∗‖Σn𝔼ℙn\[y02\]\+12‖θ∗‖Σn2=o⁡\(n−1\)𝒪P\(1\)\+𝒪P\(1\)=𝒪P\(1\)\.C\_\{n\}:=c\_\{n\}\(\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\+1\)\+\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}\\sqrt\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\}\+\\frac\{1\}\{2\}\\\|\\theta^\{\\ast\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}=o\(n^\{\-1\}\)\\mathcal\{O\}\_\{P\}\(1\)\+\\mathcal\{O\}\_\{P\}\(1\)=\\mathcal\{O\}\_\{P\}\(1\)\.This leaves a quadratic inequality of the formAn‖θ^n,Tn𝖲𝖬‖Σn2−Bn‖θ^n,Tn𝖲𝖬‖Σn≤CnA\_\{n\}\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\-B\_\{n\}\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}\\leq C\_\{n\}, whereAn→12A\_\{n\}\\to\\frac\{1\}\{2\}andBn,Cn=𝒪P\(1\)B\_\{n\},C\_\{n\}=\\mathcal\{O\}\_\{P\}\(1\)\. HenceAn≥14A\_\{n\}\\geq\\frac\{1\}\{4\}for all largenn\. Solving the quadratic inequality shows that the empirical norm is bounded in probability\.

‖θ^n,Tn𝖲𝖬‖Σn≤Bn\+Bn2\+4AnCn2An≤BnAn\+CnAn≤4𝒪P\(1\)\+4𝒪P\(1\)=𝒪P\(1\)\.\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}\\leq\\frac\{B\_\{n\}\+\\sqrt\{B\_\{n\}^\{2\}\+4A\_\{n\}C\_\{n\}\}\}\{2A\_\{n\}\}\\leq\\frac\{B\_\{n\}\}\{A\_\{n\}\}\+\\sqrt\{\\frac\{C\_\{n\}\}\{A\_\{n\}\}\}\\leq 4\\mathcal\{O\}\_\{P\}\(1\)\+\\sqrt\{4\\mathcal\{O\}\_\{P\}\(1\)\}=\\mathcal\{O\}\_\{P\}\(1\)\.On samples with an empty argmin, the convention in Section[2\.4](https://arxiv.org/html/2609.05688#S2.SS4)givesϑ^n,Tn𝖲𝖬=\(𝟎,0\)\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}=\(\\mathbf\{0\},0\)\. Hence the empirical norm bound holds unconditionally\. The transition\-kernel KL bound also holds for the fallback value\. In particular,Δn\(\(𝟎,0\)\)≤cn\(𝔼ℙn\[y02\]\+1\)=oP\(n−1\)\\Delta\_\{n\}\(\(\\mathbf\{0\},0\)\)\\leq c\_\{n\}\(\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[y\_\{0\}^\{2\}\]\+1\)=o\_\{P\}\(n^\{\-1\}\)\. Substitution into the KL discrepancy bound, together withcn=o⁡\(1/n\)c\_\{n\}=o\(1/n\), gives

Δn\(ϑ^n,Tn𝖲𝖬\)≤cn𝒪P\(1\)=oP\(n−1\)\.\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\\leq c\_\{n\}\\mathcal\{O\}\_\{P\}\(1\)=o\_\{P\}\(n^\{\-1\}\)\.On samples with a nonempty argmin, we also haveℰn\(ϑ^n,Tn𝖲𝖬\)≤Δn\(ϑ^n,Tn𝖲𝖬\)\\mathcal\{E\}\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\\leq\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\. Moreover, since𝐱\\mathbf\{x\}has a density that is positive almost everywhere onℝd\\mathbb\{R\}^\{d\},Σ:=𝔼⁡\[𝐱𝐱⊤\]\\Sigma:=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]is positive definite, and its smallest eigenvalue satisfiesλmin\(Σ\)\>0\\lambda\_\{\\min\}\(\\Sigma\)\>0\. Weyl’s Perturbation Theorem \(Theorem 4\.3\.1 on page 239 of[Horn and Johnson \(2012\)](https://arxiv.org/html/2609.05688#bib.bib21)\) and the Weak Law of Large Numbers \(Theorem 2\.2\.3 of[Durrett \(2019\)](https://arxiv.org/html/2609.05688#bib.bib10)\) give\|λmin\(Σn\)−λmin\(Σ\)\|≤‖Σn−Σ‖2=oP\(1\)\|\\lambda\_\{\\min\}\(\\Sigma\_\{n\}\)\-\\lambda\_\{\\min\}\(\\Sigma\)\|\\leq\\\|\\Sigma\_\{n\}\-\\Sigma\\\|\_\{2\}=o\_\{P\}\(1\)\. Therefore,\|λmin\(Σn\)−λmin\(Σ\)\|/λmin\(Σ\)=oP\(1\)\|\\lambda\_\{\\min\}\(\\Sigma\_\{n\}\)\-\\lambda\_\{\\min\}\(\\Sigma\)\|/\\lambda\_\{\\min\}\(\\Sigma\)=o\_\{P\}\(1\)\. For all largenn,λmin\(Σn\)≥λmin\(Σ\)−\|λmin\(Σn\)−λmin\(Σ\)\|=\(1−oP\(1\)\)λmin\(Σ\)\>0\\lambda\_\{\\min\}\(\\Sigma\_\{n\}\)\\geq\\lambda\_\{\\min\}\(\\Sigma\)\-\|\\lambda\_\{\\min\}\(\\Sigma\_\{n\}\)\-\\lambda\_\{\\min\}\(\\Sigma\)\|=\(1\-o\_\{P\}\(1\)\)\\lambda\_\{\\min\}\(\\Sigma\)\>0with high probability and

‖θ^n,Tn𝖲𝖬‖≤\(θ^n,Tn𝖲𝖬\)⊤Σn\(θ^n,Tn𝖲𝖬\)λmin\(Σn\)≤‖θ^n,Tn𝖲𝖬‖Σn\(1−oP\(1\)\)λmin\(Σ\)=𝒪P\(1\)\.\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\\leq\\sqrt\{\\frac\{\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)^\{\\top\}\\Sigma\_\{n\}\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\}\{\\lambda\_\{\\min\}\(\\Sigma\_\{n\}\)\}\}\\leq\\frac\{\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\_\{\\Sigma\_\{n\}\}\}\{\\sqrt\{\(1\-o\_\{P\}\(1\)\)\\lambda\_\{\\min\}\(\\Sigma\)\}\}=\\mathcal\{O\}\_\{P\}\(1\)\.
Step 2\. Convergence to the ground truth and boundedness ofϑ^n,Tn𝖲𝖬\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\. Taking the expectation of the empirical average under the i\.i\.d\. ground\-truth sample gives the population excess risk

ℰ\(ϑ\)=𝔼𝐱\[𝖪𝖫\(p\(⋅∣𝐱;ϑ∗\)∥p\(⋅∣𝐱;ϑ\)\)\]≥0\.\\mathcal\{E\}\(\\vartheta\)=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\\!\\left\[\\KL\\\!\\left\(p\(\\,\\cdot\\mid\\mathbf\{x\};\\vartheta^\{\\ast\}\)\\parallel p\(\\,\\cdot\\mid\\mathbf\{x\};\\vartheta\)\\right\)\\right\]\\geq 0\.The equality condition for KL divergence shows that

ℰ\(ϑ\)=0⇔p\(⋅∣𝐱;ϑ∗\)=p\(⋅∣𝐱;ϑ\)for almost every𝐱\.\\mathcal\{E\}\(\\vartheta\)=0\\Leftrightarrow p\(\\,\\cdot\\mid\\mathbf\{x\};\\vartheta^\{\\ast\}\)=p\(\\,\\cdot\\mid\\mathbf\{x\};\\vartheta\)\\quad\\text\{for almost every \}\\mathbf\{x\}\.Their conditional moment generating function is

𝔼y0\|𝐱;ϑ\[exp⁡\(ty0\)\]=\[cosh⁡\(tθ⊤𝐱\)\+tanh⁡ν⋅sinh⁡\(tθ⊤𝐱\)\]exp⁡\(t2/2\)\.\\mathbb\{E\}\_\{y\_\{0\}\\mid\\mathbf\{x\};\\vartheta\}\[\\exp\(ty\_\{0\}\)\]=\\left\[\\cosh\(t\\theta^\{\\top\}\\mathbf\{x\}\)\+\\tanh\\nu\\cdot\\sinh\(t\\theta^\{\\top\}\\mathbf\{x\}\)\\right\]\\exp\(t^\{2\}/2\)\.In particular, equality of the conditional distributions implies equality of their first two raw moments\.

\(θ⊤𝐱\)tanhν=\(\(θ∗\)⊤𝐱\)tanhν∗,\(θ⊤𝐱\)2=\(\(θ∗\)⊤𝐱\)2almost surely\.\(\\theta^\{\\top\}\\mathbf\{x\}\)\\tanh\\nu=\(\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbf\{x\}\)\\tanh\\nu^\{\\ast\},\\qquad\(\\theta^\{\\top\}\\mathbf\{x\}\)^\{2\}=\(\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbf\{x\}\)^\{2\}\\quad\\text\{almost surely\}\.The second identity is equivalent to

𝐱⊤\(θθ⊤−θ∗\(θ∗\)⊤\)𝐱=0almost surely\.\\mathbf\{x\}^\{\\top\}\\left\(\\theta\\theta^\{\\top\}\-\\theta^\{\\ast\}\(\\theta^\{\\ast\}\)^\{\\top\}\\right\)\\mathbf\{x\}=0\\quad\\text\{almost surely\}\.Since𝐱\\mathbf\{x\}has a density that is positive almost everywhere onℝd\\mathbb\{R\}^\{d\}, this quadratic polynomial is zero onℝd\\mathbb\{R\}^\{d\}, and henceθθ⊤=θ∗\(θ∗\)⊤\\theta\\theta^\{\\top\}=\\theta^\{\\ast\}\(\\theta^\{\\ast\}\)^\{\\top\}\. Becauseθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}, it follows thatθ=sθ∗\\theta=s\\theta^\{\\ast\}for somes∈\{\+1,−1\}s\\in\\\{\+1,\-1\\\}\. Substitution into the first\-moment identity givesstanh⁡ν=tanh⁡ν∗s\\tanh\\nu=\\tanh\\nu^\{\\ast\}, so the oddness and strict monotonicity oftanh\\tanhimplyν=sν∗\\nu=s\\nu^\{\\ast\}\. The reverse implication follows because the joint sign flip exchanges the two mixture components and leaves the conditional distribution unchanged\. This agrees with Proposition[3\.1](https://arxiv.org/html/2609.05688#S3.Thmstyle1)atα¯0=1\\bar\{\\alpha\}\_\{0\}=1and establishes identifiability for everyϑ∈ℝd×ℝ\\vartheta\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}\.

ℰ\(ϑ\)=0⟺ϑ≡\(θ,ν\)=s\(θ∗,ν∗\)=sϑ∗,s∈\{\+1,−1\}\.\\mathcal\{E\}\(\\vartheta\)=0\\quad\\Longleftrightarrow\\quad\\vartheta\\equiv\(\\theta,\\nu\)=s\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)=s\\vartheta^\{\\ast\},\\qquad s\\in\\\{\+1,\-1\\\}\.Step 1 gives‖θ^n,Tn𝖲𝖬‖=𝒪P\(1\)\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|=\\mathcal\{O\}\_\{P\}\(1\)\. Thus, for everyδ\>0\\delta\>0, we can choose a finite radiusRδ\>‖θ∗‖R\_\{\\delta\}\>\\\|\\theta^\{\\ast\}\\\|and an indexnδn\_\{\\delta\}such that

infn≥nδℙ⁡\(θ^n,Tn𝖲𝖬∈KRδ\)≥1−δ,KRδ:=\{θ∈ℝd∣‖θ‖≤Rδ\}\.\\inf\_\{n\\geq n\_\{\\delta\}\}\\mathbb\{P\}\\\!\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\in K\_\{R\_\{\\delta\}\}\\right\)\\geq 1\-\\delta,\\qquad K\_\{R\_\{\\delta\}\}:=\\\{\\theta\\in\\mathbb\{R\}^\{d\}\\mid\\\|\\theta\\\|\\leq R\_\{\\delta\}\\\}\.We work on the compact parameter setKδ:=KRδ×ℝ¯K\_\{\\delta\}:=K\_\{R\_\{\\delta\}\}\\times\\overline\{\\mathbb\{R\}\}, whereℝ¯:=ℝ∪\{−∞,\+∞\}\\overline\{\\mathbb\{R\}\}:=\\mathbb\{R\}\\cup\\\{\-\\infty,\+\\infty\\\}\. At its two boundary points, the conditional model has the limitsp\(⋅∣𝐱;θ,±∞\)=𝒩\(±⟨θ,𝐱⟩,1\)p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta,\\pm\\infty\)=\\mathcal\{N\}\(\\pm\\langle\\theta,\\mathbf\{x\}\\rangle,1\)\. Fatou’s lemma \(Theorem 1\.5\.5 of[Durrett \(2019\)](https://arxiv.org/html/2609.05688#bib.bib10)\) yields

lim inf\|ν\|→∞ℰ\(θ,ν\)≥𝔼𝐱\[mins∈\{\+1,−1\}𝖪𝖫\(p\(⋅∣𝐱;ϑ∗\)∥𝒩\(s⟨θ,𝐱⟩,1\)\)\]\>0\.\\liminf\_\{\|\\nu\|\\to\\infty\}\\mathcal\{E\}\(\\theta,\\nu\)\\geq\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\\!\\left\[\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\KL\\\!\\left\(p\(\\,\\cdot\\mid\\mathbf\{x\};\\vartheta^\{\\ast\}\)\\parallel\\mathcal\{N\}\\\!\\left\(s\\langle\\theta,\\mathbf\{x\}\\rangle,1\\right\)\\right\)\\right\]\>0\.Because the ground\-truth parameter is finite \(\|ν∗\|<∞\|\\nu^\{\\ast\}\|<\\inftyandθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}\), the true conditional distribution is a strict two\-component mixture for almost every𝐱\\mathbf\{x\}\. The identifiability of Gaussian mixtures implies that the KL divergence to either single Gaussian is positive for almost every𝐱\\mathbf\{x\}\. Define

g\(θ\):=𝔼𝐱\[mins∈\{\+1,−1\}𝖪𝖫\(p\(⋅∣𝐱;ϑ∗\)∥𝒩\(s⟨θ,𝐱⟩,1\)\)\]\.g\(\\theta\):=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\\!\\left\[\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\KL\\\!\\left\(p\(\\,\\cdot\\mid\\mathbf\{x\};\\vartheta^\{\\ast\}\)\\parallel\\mathcal\{N\}\\\!\\left\(s\\langle\\theta,\\mathbf\{x\}\\rangle,1\\right\)\\right\)\\right\]\.The integrand is continuous inθ\\theta\. Convexity of KL divergence in its first argument bounds it, forθ∈KRδ\\theta\\in K\_\{R\_\{\\delta\}\}, by12\(‖θ∗‖\+Rδ\)2‖𝐱‖2\\frac\{1\}\{2\}\(\\\|\\theta^\{\\ast\}\\\|\+R\_\{\\delta\}\)^\{2\}\\\|\\mathbf\{x\}\\\|^\{2\}\. The finite second moment of𝐱\\mathbf\{x\}and dominated convergence therefore makeggcontinuous onKRδK\_\{R\_\{\\delta\}\}\. We also haveg⁡\(θ\)\>0g\(\\theta\)\>0at everyθ∈KRδ\\theta\\in K\_\{R\_\{\\delta\}\}\. Compactness now givesκδ:=minθ∈KRδ⁡g⁡\(θ\)\>0\\kappa\_\{\\delta\}:=\\min\_\{\\theta\\in K\_\{R\_\{\\delta\}\}\}g\(\\theta\)\>0\.

infθ∈KRδlim inf\|ν\|→∞ℰ⁡\(θ,ν\)≥κδ\>0,minϑ∈Kδ⁡ℰ⁡\(ϑ\)=ℰ⁡\(ϑ∗\)=ℰ⁡\(−ϑ∗\)=0\.\\inf\_\{\\theta\\in K\_\{R\_\{\\delta\}\}\}\\liminf\_\{\|\\nu\|\\to\\infty\}\\mathcal\{E\}\(\\theta,\\nu\)\\geq\\kappa\_\{\\delta\}\>0,\\qquad\\min\_\{\\vartheta\\in K\_\{\\delta\}\}\\mathcal\{E\}\(\\vartheta\)=\\mathcal\{E\}\(\\vartheta^\{\\ast\}\)=\\mathcal\{E\}\(\-\\vartheta^\{\\ast\}\)=0\.Henceargminϑ∈Kδ⁡ℰ\(ϑ\)=\{ϑ∗,−ϑ∗\}\\argmin\_\{\\vartheta\\in K\_\{\\delta\}\}\\mathcal\{E\}\(\\vartheta\)=\\\{\\vartheta^\{\\ast\},\-\\vartheta^\{\\ast\}\\\}\. The ambient parameter spaceℝd×ℝ¯\\mathbb\{R\}^\{d\}\\times\\overline\{\\mathbb\{R\}\}containsKδK\_\{\\delta\}\. The global maximizers of the expected conditional log\-likelihood on the ambient space are𝒮∗:=\{ϑ∗,−ϑ∗\}\\mathcal\{S\}^\{\\ast\}:=\\\{\\vartheta^\{\\ast\},\-\\vartheta^\{\\ast\}\\\}\.

𝒮∗=\{ϑ′∈ℝd×ℝ¯\|𝔼y0,𝐱\|ϑ∗\[mϑ′\]=supϑ∈ℝd×ℝ¯𝔼y0,𝐱\|ϑ∗\[mϑ\]\}=\{ϑ′∈ℝd×ℝ¯\|ℰ⁡\(ϑ′\)=0\}\.\\mathcal\{S\}^\{\\ast\}=\\left\\\{\\vartheta^\{\\prime\}\\in\\mathbb\{R\}^\{d\}\\times\\overline\{\\mathbb\{R\}\}\\ \\middle\|\\ \\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[m\_\{\\vartheta^\{\\prime\}\}\]=\\sup\_\{\\vartheta\\in\\mathbb\{R\}^\{d\}\\times\\overline\{\\mathbb\{R\}\}\}\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[m\_\{\\vartheta\}\]\\right\\\}=\\left\\\{\\vartheta^\{\\prime\}\\in\\mathbb\{R\}^\{d\}\\times\\overline\{\\mathbb\{R\}\}\\ \\middle\|\\ \\mathcal\{E\}\(\\vartheta^\{\\prime\}\)=0\\right\\\}\.LetU⊂KδU\\subset K\_\{\\delta\}be a neighborhood and setRU:=sup\(θ,ν\)∈U‖θ‖<∞R\_\{U\}:=\\sup\_\{\(\\theta,\\nu\)\\in U\}\\\|\\theta\\\|<\\infty\. Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)and the inequalityln⁡\[cosh⁡\(x\+x′\)/cosh⁡x′\]≤\|x\|\\ln\[\\cosh\(x\+x^\{\\prime\}\)/\\cosh x^\{\\prime\}\]\\leq\|x\|, followed by passage to the boundary limits when needed, give

𝔼⁡\[supϑ∈U\{−mϑ\(y0,𝐱\)\}\]\\displaystyle\\mathbb\{E\}\\\!\\left\[\\sup\_\{\\vartheta\\in U\}\\\{\-m\_\{\\vartheta\}\(y\_\{0\},\\mathbf\{x\}\)\\\}\\right\]≤12𝔼\[y02\]\+12RU2𝔼\[‖𝐱‖2\]\+RU𝔼\[\|y0\|‖𝐱‖\]\+12ln⁡\(2π\)\\displaystyle\\leq\\frac\{1\}\{2\}\\mathbb\{E\}\[y\_\{0\}^\{2\}\]\+\\frac\{1\}\{2\}R\_\{U\}^\{2\}\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]\+R\_\{U\}\\mathbb\{E\}\[\|y\_\{0\}\|\\\|\\mathbf\{x\}\\\|\]\+\\frac\{1\}\{2\}\\ln\(2\\uppi\)≤12𝔼\[y02\]\+12RU2𝔼\[‖𝐱‖2\]\+RU𝔼⁡\[y02\]𝔼\[‖𝐱‖2\]\+12ln⁡\(2π\)<∞\.\\displaystyle\\leq\\frac\{1\}\{2\}\\mathbb\{E\}\[y\_\{0\}^\{2\}\]\+\\frac\{1\}\{2\}R\_\{U\}^\{2\}\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]\+R\_\{U\}\\sqrt\{\\mathbb\{E\}\[y\_\{0\}^\{2\}\]\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]\}\+\\frac\{1\}\{2\}\\ln\(2\\uppi\)<\\infty\.This integrable envelope gives the local domination condition in Wald’s consistency theorem for M\-estimators with nonunique maxima \(Theorem 5\.14 on page 48 of[van der Vaart \(1998\)](https://arxiv.org/html/2609.05688#bib.bib15)\)\. The conditional log\-likelihood is continuous in each finite parameter and has continuous limits atν=±∞\\nu=\\pm\\infty, so it is upper semicontinuous onKδK\_\{\\delta\}\.

The empirical bridge and score matching optimality give the required likelihood near\-optimality\. Let𝒜n\\mathcal\{A\}\_\{n\}denote the event that the empirical score matching argmin is nonempty, and defineDn:=supϑ𝔼ℙnmϑ−𝔼ℙnmϑ^n,Tn𝖲𝖬D\_\{n\}:=\\sup\_\{\\vartheta\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}m\_\{\\vartheta\}\-\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}m\_\{\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\}on the entire sample space using the estimator convention in Section[2\.4](https://arxiv.org/html/2609.05688#S2.SS4)\. Equivalently,Dn=ℛ^n𝖬𝖫𝖤\(ϑ^n,Tn𝖲𝖬\)−infϑℛ^n𝖬𝖫𝖤\(ϑ\)D\_\{n\}=\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\)\-\\inf\_\{\\vartheta\}\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\)\. On𝒜n\\mathcal\{A\}\_\{n\}, the score matching optimality givesℛ^n,Tn𝖲𝖬\(ϑ^\)−ℛ^n,Tn𝖲𝖬\(ϑ\)≤0\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\widehat\{\\vartheta\}\)\-\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\)\\leq 0, the bridge constant cancels, andΔn\(ϑ\)≥0\\Delta\_\{n\}\(\\vartheta\)\\geq 0; hence, for everyϑ\\vartheta,

ℛ^n𝖬𝖫𝖤\(ϑ^\)−ℛ^n𝖬𝖫𝖤\(ϑ\)=ℛ^n,Tn𝖲𝖬\(ϑ^\)−ℛ^n,Tn𝖲𝖬\(ϑ\)\+Δn\(ϑ^\)−Δn\(ϑ\)≤Δn\(ϑ^\)\.\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\widehat\{\\vartheta\}\)\-\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\)=\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\widehat\{\\vartheta\}\)\-\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\)\+\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\)\-\\Delta\_\{n\}\(\\vartheta\)\\leq\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\)\.Taking the supremum overϑ\\varthetagives0≤Dn≤Δn\(ϑ^\)0\\leq D\_\{n\}\\leq\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\)on𝒜n\\mathcal\{A\}\_\{n\}\. Step 1 establishedΔn\(ϑ^\)=oP\(n−1\)\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\)=o\_\{P\}\(n^\{\-1\}\)on the entire sample space, whileℙ⁡\(𝒜nc\)→0\\mathbb\{P\}\(\\mathcal\{A\}\_\{n\}^\{c\}\)\\to 0is ensured by the application of Lemma[B\.2](https://arxiv.org/html/2609.05688#A2.Thmstyle2)\. Therefore, for everyϵ\>0\\epsilon\>0,ℙ\(nDn\>ϵ\)≤ℙ\(𝒜nc\)\+ℙ\{nΔn\(ϑ^\)\>ϵ\}→0\\mathbb\{P\}\(nD\_\{n\}\>\\epsilon\)\\leq\\mathbb\{P\}\(\\mathcal\{A\}\_\{n\}^\{c\}\)\+\\mathbb\{P\}\\\{n\\Delta\_\{n\}\(\\widehat\{\\vartheta\}\)\>\\epsilon\\\}\\to 0, which provesDn=oP\(n−1\)D\_\{n\}=o\_\{P\}\(n^\{\-1\}\)\. We apply Wald’s theorem to the compact setKδ≡KRδ×ℝ¯K\_\{\\delta\}\\equiv K\_\{R\_\{\\delta\}\}\\times\\overline\{\\mathbb\{R\}\}\. The equivalenceϑ^n,Tn𝖲𝖬∈Kδ⇔θ^n,Tn𝖲𝖬∈KRδ\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\in K\_\{\\delta\}\\Leftrightarrow\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\in K\_\{R\_\{\\delta\}\}gives

limn→∞ℙ\(\{mins∈\{\+1,−1\}∥sϑ^n,Tn𝖲𝖬−ϑ∗∥≥ϵ\}∩\{θ^n,Tn𝖲𝖬∈KRδ\}\)=0for everyϵ\>0\.\\lim\_\{n\\to\\infty\}\\mathbb\{P\}\\\!\\left\(\\left\\\{\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\\|s\\,\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\vartheta^\{\\ast\}\\\|\\geq\\epsilon\\right\\\}\\cap\\left\\\{\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\in K\_\{R\_\{\\delta\}\}\\right\\\}\\right\)=0\\quad\\text\{for every \}\\epsilon\>0\.The union boundℙ⁡\(A\)=ℙ⁡\(A∩B\)\+ℙ⁡\(A∩B𝖼\)≤ℙ⁡\(A∩B\)\+1−ℙ⁡\(B\)\\mathbb\{P\}\(A\)=\\mathbb\{P\}\(A\\cap B\)\+\\mathbb\{P\}\(A\\cap B^\{\\mathsf\{c\}\}\)\\leq\\mathbb\{P\}\(A\\cap B\)\+1\-\\mathbb\{P\}\(B\), together with the definitions ofRδR\_\{\\delta\}andKRδK\_\{R\_\{\\delta\}\}, gives

lim supn→∞ℙ⁡\(mins∈\{\+1,−1\}⁡‖sϑ^n,Tn𝖲𝖬−ϑ∗‖≥ϵ\)≤1−lim infn→∞ℙ⁡\(θ^n,Tn𝖲𝖬∈KRδ\)≤δfor everyϵ\>0\.\\limsup\_\{n\\to\\infty\}\\mathbb\{P\}\\left\(\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\\|s\\,\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\vartheta^\{\\ast\}\\\|\\geq\\epsilon\\right\)\\leq 1\-\\liminf\_\{n\\to\\infty\}\\mathbb\{P\}\\left\(\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\in K\_\{R\_\{\\delta\}\}\\right\)\\leq\\delta\\quad\\text\{for every \}\\epsilon\>0\.Lettingδ\\deltatend to zero shows thatlimn→∞ℙ⁡\(mins∈\{\+1,−1\}⁡‖sϑ^n,Tn𝖲𝖬−ϑ∗‖≥ϵ\)=0\\lim\_\{n\\to\\infty\}\\mathbb\{P\}\\left\(\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\\|s\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\vartheta^\{\\ast\}\\\|\\geq\\epsilon\\right\)=0, and hence

mins∈\{\+1,−1\}⁡‖sϑ^n,Tn𝖲𝖬−ϑ∗‖→ℙ0\.\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\\|s\\,\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\vartheta^\{\\ast\}\\\|\\xrightarrow\{\\mathbb\{P\}\}0\.This also impliesmins∈\{\+1,−1\}⁡‖sθ^n,Tn𝖲𝖬−θ∗‖=oP\(1\)\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\\|s\\,\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\theta^\{\\ast\}\\\|=o\_\{P\}\(1\)\. The preceding convergence and the triangle inequality also give‖ϑ^n,Tn𝖲𝖬‖=𝒪P\(1\)\\\|\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|=\\mathcal\{O\}\_\{P\}\(1\)and\|ν^n,Tn𝖲𝖬\|=𝒪P\(1\)\|\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\|=\\mathcal\{O\}\_\{P\}\(1\)\.

‖ϑ^n,Tn𝖲𝖬‖≤‖ϑ∗‖\+mins∈\{\+1,−1\}⁡‖sϑ^n,Tn𝖲𝖬−ϑ∗‖=𝒪⁡\(1\)\+oP\(1\)=𝒪P\(1\),\\\|\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|\\leq\\\|\\vartheta^\{\\ast\}\\\|\+\\min\_\{s\\in\\\{\+1,\-1\\\}\}\\\|s\\,\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\vartheta^\{\\ast\}\\\|=\\mathcal\{O\}\(1\)\+o\_\{P\}\(1\)=\\mathcal\{O\}\_\{P\}\(1\),\|ν^n,Tn𝖲𝖬\|=‖ϑ^n,Tn𝖲𝖬‖2−‖θ^n,Tn𝖲𝖬‖2≤‖ϑ^n,Tn𝖲𝖬‖=𝒪P\(1\)\.\|\\widehat\{\\nu\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\|=\\sqrt\{\\\|\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|^\{2\}\-\\\|\\widehat\{\\theta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|^\{2\}\}\\leq\\\|\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\\|=\\mathcal\{O\}\_\{P\}\(1\)\.
Step 3\. Asymptotic normality\. We apply Theorem 5\.23 of[van der Vaart \(1998, p\. 53\)](https://arxiv.org/html/2609.05688#bib.bib15)\. The following paragraphs verify its assumptions for the aligned estimator\.

The aligned estimator is consistent\.Step 2 gives

ϑ~n,Tn𝖲𝖬:=snϑ^n,Tn𝖲𝖬→ℙϑ∗\.\\widetilde\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}:=s\_\{n\}\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\\xrightarrow\{\\mathbb\{P\}\}\\vartheta^\{\\ast\}\.The aligned estimator is near optimal\.Formϑ\(y0,𝐱\):=ln⁡p⁡\(y0∣𝐱;ϑ\)m\_\{\\vartheta\}\(y\_\{0\},\\mathbf\{x\}\):=\\ln p\(y\_\{0\}\\mid\\mathbf\{x\};\\vartheta\), Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)gives the symmetryℛ^n𝖬𝖫𝖤\(ϑ\)=ℛ^n𝖬𝖫𝖤\(−ϑ\)\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\\vartheta\)=\\widehat\{\\mathcal\{R\}\}\_\{n\}^\{\\,\\mathsf\{MLE\}\}\(\-\\vartheta\)\. Therefore, the near\-optimality bound from Step 2 also holds after sign alignment\.

𝔼ℙnmϑ~n,Tn𝖲𝖬≥supϑ𝔼ℙnmϑ−oP\(1/n\)\.\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}m\_\{\\widetilde\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\}\\geq\\sup\_\{\\vartheta\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}m\_\{\\vartheta\}\-o\_\{P\}\\left\(1/n\\right\)\.
The population criterion has a nonsingular quadratic expansion\.The conditional log\-likelihoodmϑ\(y0,𝐱\)=ln⁡p⁡\(y0∣𝐱;ϑ\)m\_\{\\vartheta\}\(y\_\{0\},\\mathbf\{x\}\)=\\ln p\(y\_\{0\}\\mid\\mathbf\{x\};\\vartheta\)is constructed fromln⁡cosh⁡\(⋅\)\\ln\\cosh\(\\cdot\)and linear maps \(Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)\)\. Hence the mapϑ↦mϑ\(y0,𝐱\)\\vartheta\\mapsto m\_\{\\vartheta\}\(y\_\{0\},\\mathbf\{x\}\)is smooth inϑ≡\(θ,ν\)\\vartheta\\equiv\(\\theta,\\nu\)for ally0,𝐱y\_\{0\},\\mathbf\{x\}\. In particular, the derivative∇mϑ∗\\nabla m\_\{\\vartheta^\{\\ast\}\}with respect toϑ\\varthetaatϑ=ϑ∗\\vartheta=\\vartheta^\{\\ast\}exists\. Writingu=θ⊤𝐱u=\\theta^\{\\top\}\\mathbf\{x\}, the Hessian blocks of the conditional log\-likelihood are

∇θθ2mϑ\\displaystyle\\nabla\_\{\\theta\\theta\}^\{2\}m\_\{\\vartheta\}=𝐱𝐱⊤\[y02sech2⁡\(y0u\+ν\)−1\],\\displaystyle=\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\\bigl\[y\_\{0\}^\{2\}\\sech^\{2\}\(y\_\{0\}u\+\\nu\)\-1\\bigr\],∇θν2mϑ\\displaystyle\\nabla\_\{\\theta\\nu\}^\{2\}m\_\{\\vartheta\}=𝐱y0sech2⁡\(y0u\+ν\),\\displaystyle=\\mathbf\{x\}y\_\{0\}\\sech^\{2\}\(y\_\{0\}u\+\\nu\),∇νν2mϑ\\displaystyle\\nabla\_\{\\nu\\nu\}^\{2\}m\_\{\\vartheta\}=sech2⁡\(y0u\+ν\)−sech2⁡ν\.\\displaystyle=\\sech^\{2\}\(y\_\{0\}u\+\\nu\)\-\\sech^\{2\}\\nu\.Their norms have a common integrable bound overϑ\\varthetagiven by a linear combination of‖𝐱‖2\(y02\+1\)\\\|\\mathbf\{x\}\\\|^\{2\}\(y\_\{0\}^\{2\}\+1\),‖𝐱‖\|y0\|\\\|\\mathbf\{x\}\\\|\|y\_\{0\}\|, and11\. Under the fourth\-moment condition and the representationy0=\(−1\)z\+1\(θ∗\)⊤𝐱\+ε0y\_\{0\}=\(\-1\)^\{z\+1\}\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbf\{x\}\+\\varepsilon\_\{0\}, this envelope is integrable\. Because the Hessian entries are continuous inϑ\\vartheta, the Dominated Convergence Theorem\([Durrett, 2019](https://arxiv.org/html/2609.05688#bib.bib10), Theorem 1\.5\.8\)justifies differentiating under the expectation twice and shows that the population criterion has a second\-order Taylor expansion atϑ∗\\vartheta^\{\\ast\}, governed by

Vϑ∗:=∇2𝔼y0,𝐱\|ϑ∗\[mϑ\]\|ϑ=ϑ∗=𝔼y0,𝐱\|ϑ∗\[∇2mϑ∗\]\.V\_\{\\vartheta^\{\\ast\}\}:=\\nabla^\{2\}\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[m\_\{\\vartheta\}\]\\big\|\_\{\\vartheta=\\vartheta^\{\\ast\}\}=\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[\\nabla^\{2\}m\_\{\\vartheta^\{\\ast\}\}\]\.The information matrix equality gives

I\(ϑ∗\):=𝔼y0,𝐱\|ϑ∗\[∇mϑ∗∇mϑ∗⊤\]=−𝔼y0,𝐱\|ϑ∗\[∇2mϑ∗\]=−𝔼y0,𝐱\|ϑ∗\[∇ϑ2lnp\(y0∣𝐱;ϑ\)\]=−Vϑ∗\.I\(\\vartheta^\{\\ast\}\):=\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[\\nabla m\_\{\\vartheta^\{\\ast\}\}\\nabla m\_\{\\vartheta^\{\\ast\}\}^\{\\top\}\]=\-\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[\\nabla^\{2\}m\_\{\\vartheta^\{\\ast\}\}\]=\-\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[\\nabla\_\{\\vartheta\}^\{2\}\\ln p\(y\_\{0\}\\mid\\mathbf\{x\};\\vartheta\)\]=\-V\_\{\\vartheta^\{\\ast\}\}\.We verify nonsingularity through the score coordinates\. For\(a,b\)∈ℝd×ℝ\(a,b\)\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}, setm=\(θ∗\)⊤𝐱m=\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbf\{x\}andq=a⊤𝐱q=a^\{\\top\}\\mathbf\{x\}\. If\(a,b\)⊤I\(ϑ∗\)\(a,b\)=0\(a,b\)^\{\\top\}I\(\\vartheta^\{\\ast\}\)\(a,b\)=0, then the corresponding linear combination of the score coordinates vanishes almost surely\.

\(qy0\+b\)tanh\(my0\+ν∗\)−qm−btanhν∗=0\.\(qy\_\{0\}\+b\)\\tanh\(my\_\{0\}\+\\nu^\{\\ast\}\)\-qm\-b\\tanh\\nu^\{\\ast\}=0\.For almost every𝐱\\mathbf\{x\}, the conditional densityy0↦p⁡\(y0∣𝐱;ϑ∗\)y\_\{0\}\\mapsto p\(y\_\{0\}\\mid\\mathbf\{x\};\\vartheta^\{\\ast\}\)is positive onℝ\\mathbb\{R\}\. The displayed expression is continuous iny0y\_\{0\}, so it vanishes for everyy0y\_\{0\}\. For every such𝐱\\mathbf\{x\}withm≠0m\\neq 0, division byy0y\_\{0\}followed byy0→\+∞y\_\{0\}\\to\+\\inftygivesq=0q=0, after which the nonconstancy ofy0↦tanh⁡\(my0\+ν∗\)y\_\{0\}\\mapsto\\tanh\(my\_\{0\}\+\\nu^\{\\ast\}\)givesb=0b=0\. Becauseθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}and the density of𝐱\\mathbf\{x\}is positive almost everywhere,m≠0m\\neq 0almost surely, anda⊤𝐱=0a^\{\\top\}\\mathbf\{x\}=0almost surely forcesa=0a=0\. Thus the Fisher information matrixI⁡\(ϑ∗\)I\(\\vartheta^\{\\ast\}\)is positive definite and nonsingular\.

A square\-integrable Lipschitz envelope exists\.LetUUbe a fixed convex neighborhood ofϑ∗\\vartheta^\{\\ast\}with radiusRR, and definem˙\(y0,𝐱\):=supϑ∈U‖∇mϑ\(y0,𝐱\)‖\\dot\{m\}\(y\_\{0\},\\mathbf\{x\}\):=\\sup\_\{\\vartheta\\in U\}\\\|\\nabla m\_\{\\vartheta\}\(y\_\{0\},\\mathbf\{x\}\)\\\|\. The Mean Value Theorem \(Theorem 9\.19 on page 218 of[Rudin \(1976\)](https://arxiv.org/html/2609.05688#bib.bib11)\) gives\|mϑ1\(y0,𝐱\)−mϑ2\(y0,𝐱\)\|≤m˙\(y0,𝐱\)‖ϑ1−ϑ2‖\|m\_\{\\vartheta\_\{1\}\}\(y\_\{0\},\\mathbf\{x\}\)\-m\_\{\\vartheta\_\{2\}\}\(y\_\{0\},\\mathbf\{x\}\)\|\\leq\\dot\{m\}\(y\_\{0\},\\mathbf\{x\}\)\\\|\\vartheta\_\{1\}\-\\vartheta\_\{2\}\\\|for everyϑ1,ϑ2∈U\\vartheta\_\{1\},\\vartheta\_\{2\}\\in U\. Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)gives∇θmϑ=∇θlnp\(y0∣𝐱;ϑ\)=𝐱⁡\(y0tanh⁡\(y0θ⊤𝐱\+ν\)−θ⊤𝐱\)\\nabla\_\{\\theta\}m\_\{\\vartheta\}=\\nabla\_\{\\theta\}\\ln p\(y\_\{0\}\\mid\\mathbf\{x\};\\vartheta\)=\\mathbf\{x\}\(y\_\{0\}\\tanh\(y\_\{0\}\\theta^\{\\top\}\\mathbf\{x\}\+\\nu\)\-\\theta^\{\\top\}\\mathbf\{x\}\)and∇νmϑ=∇νlnp\(y0∣𝐱,ϑ\)=tanh⁡\(y0θ⊤𝐱\+ν\)−tanh⁡ν\\nabla\_\{\\nu\}m\_\{\\vartheta\}=\\nabla\_\{\\nu\}\\ln p\(y\_\{0\}\\mid\\mathbf\{x\},\\vartheta\)=\\tanh\(y\_\{0\}\\theta^\{\\top\}\\mathbf\{x\}\+\\nu\)\-\\tanh\\nu\. We apply the triangle inequality and\|tanh⁡\(⋅\)\|≤1\|\\tanh\(\\cdot\)\|\\leq 1\. The boundssupϑ∈U‖θ‖≤R\+‖θ∗‖\\sup\_\{\\vartheta\\in U\}\\\|\\theta\\\|\\leq R\+\\\|\\theta^\{\\ast\}\\\|and\|y0\|≤‖θ∗‖‖𝐱‖\+\|ε0\|\|y\_\{0\}\|\\leq\\\|\\theta^\{\\ast\}\\\|\\\|\\mathbf\{x\}\\\|\+\|\\varepsilon\_\{0\}\|yield

m˙\(y0,𝐱\)≤‖𝐱‖\(‖θ∗‖‖𝐱‖\+\|ε0\|\+\(R\+‖θ∗‖\)‖𝐱‖\)\+2≤\(R\+2‖θ∗‖\+12\)‖𝐱‖2\+12ε02\+2\.\\dot\{m\}\(y\_\{0\},\\mathbf\{x\}\)\\leq\\\|\\mathbf\{x\}\\\|\(\\\|\\theta^\{\\ast\}\\\|\\\|\\mathbf\{x\}\\\|\+\|\\varepsilon\_\{0\}\|\+\(R\+\\\|\\theta^\{\\ast\}\\\|\)\\\|\\mathbf\{x\}\\\|\)\+2\\leq\\left\(R\+2\\\|\\theta^\{\\ast\}\\\|\+\\frac\{1\}\{2\}\\right\)\\\|\\mathbf\{x\}\\\|^\{2\}\+\\frac\{1\}\{2\}\\varepsilon\_\{0\}^\{2\}\+2\.Because the fourth moments are finite \(𝔼⁡\[‖𝐱‖24\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|\_\{2\}^\{4\}\]<\\infty\) and𝔼⁡\[ε04\]=3\\mathbb\{E\}\[\\varepsilon\_\{0\}^\{4\}\]=3, the Lipschitz envelope satisfies the square\-integrability requirement \(𝔼y0,𝐱\|ϑ∗\[m˙2\(y0,𝐱\)\]<∞\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[\\dot\{m\}^\{2\}\(y\_\{0\},\\mathbf\{x\}\)\]<\\infty\), noting that\(a\+b\+c\)2≤3\(a2\+b2\+c2\)\(a\+b\+c\)^\{2\}\\leq 3\(a^\{2\}\+b^\{2\}\+c^\{2\}\)\.

The aligned estimatorϑ~n,Tn𝖲𝖬:=snϑ^n,Tn𝖲𝖬\\widetilde\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}:=s\_\{n\}\\widehat\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}is consistent\. The event argument in Step 2 shows that its empirical likelihood suboptimality isoP\(n−1\)o\_\{P\}\(n^\{\-1\}\)\. The verified conditions allow Theorem 5\.23 of[van der Vaart \(1998\)](https://arxiv.org/html/2609.05688#bib.bib15)to be applied\. It gives

n\(ϑ~n,Tn𝖲𝖬−ϑ∗\)→𝖽𝒩\(𝟎,\(−Vϑ∗\)−1𝔼y0,𝐱\|ϑ∗\[∇mϑ∗∇mϑ∗⊤\]\(−Vϑ∗\)−1\)=𝒩\(𝟎,I\(θ∗,ν∗\)−1\)\.\\sqrt\{n\}\\left\(\\widetilde\{\\vartheta\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\-\\vartheta^\{\\ast\}\\right\)\\xrightarrow\{\\mathsf\{d\}\}\\mathcal\{N\}\\left\(\\mathbf\{0\},\(\-V\_\{\\vartheta^\{\\ast\}\}\)^\{\-1\}\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[\\nabla m\_\{\\vartheta^\{\\ast\}\}\\nabla m\_\{\\vartheta^\{\\ast\}\}^\{\\top\}\]\(\-V\_\{\\vartheta^\{\\ast\}\}\)^\{\-1\}\\right\)=\\mathcal\{N\}\(\\mathbf\{0\},I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)^\{\-1\}\)\.This completes the proof, noting thatI\(θ∗,ν∗\)≡I\(ϑ∗\)=𝔼y0,𝐱\|ϑ∗\[∇mϑ∗∇mϑ∗⊤\]=−Vϑ∗I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\equiv I\(\\vartheta^\{\\ast\}\)=\\mathbb\{E\}\_\{y\_\{0\},\\mathbf\{x\}\\mid\\vartheta^\{\\ast\}\}\[\\nabla m\_\{\\vartheta^\{\\ast\}\}\\nabla m\_\{\\vartheta^\{\\ast\}\}^\{\\top\}\]=\-V\_\{\\vartheta^\{\\ast\}\}\. ∎

###### B\.2High\-probability attainment of the empirical minimum\.

The dimensionddis fixed, and the observations are i\.i\.d\. from the MLR model withθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}andν∗∈ℝ\\nu^\{\\ast\}\\in\\mathbb\{R\}\. The covariate𝐱\\mathbf\{x\}has a density that is positive almost everywhere onℝd\\mathbb\{R\}^\{d\}and satisfies𝔼‖𝐱‖2<∞\\mathbb\{E\}\\\|\\mathbf\{x\}\\\|^\{2\}<\\infty\. The terminal horizon satisfiesα¯Tn/\(1−α¯Tn\)=o⁡\(n−1\)\\bar\{\\alpha\}\_\{T\_\{n\}\}/\(1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\)=o\(n^\{\-1\}\)\. Then the probability that the empirical Score Matching objective attains its minimum overℝd\+1\\mathbb\{R\}^\{d\+1\}tends to one\. A sample\-measurable exact minimizer can be selected whenever this minimum exists\.

###### Proof\.

Setan:=α¯Tna\_\{n\}:=\\bar\{\\alpha\}\_\{T\_\{n\}\},σn:=an\(1−an\)\\sigma\_\{n\}:=\\sqrt\{a\_\{n\}\(1\-a\_\{n\}\)\},Σn:=𝔼ℙn\[𝐱𝐱⊤\]\\Sigma\_\{n\}:=\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\], andmn:=𝔼ℙn\[𝐱y0\]m\_\{n\}:=\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[\\mathbf\{x\}y\_\{0\}\]\. Foruθ:=θ⊤𝐱u\_\{\\theta\}:=\\theta^\{\\top\}\\mathbf\{x\}, definerθ:=y0uθr\_\{\\theta\}:=y\_\{0\}u\_\{\\theta\}andqθ,ξ:=anrθ\+σnuθξq\_\{\\theta,\\xi\}:=a\_\{n\}r\_\{\\theta\}\+\\sigma\_\{n\}u\_\{\\theta\}\\xi, whereξ∼𝒩⁡\(0,1\)\\xi\\sim\\mathcal\{N\}\(0,1\)\. The terminal\-horizon condition givesan→0a\_\{n\}\\to 0\. The empirical bridge in Lemma[A\.16](https://arxiv.org/html/2609.05688#A1.Thmstyle16)and the Gaussian mixture formula in Lemma[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)show that, up to a parameter\-independent random term, the empirical score matching criterion is

𝖦n\(θ,ν\)=1−an2θ⊤Σnθ−𝔼ℙnlog⁡cosh⁡\(rθ\+ν\)\+𝔼ℙn𝔼ξlog⁡cosh⁡\(qθ,ξ\+ν\)\.\\mathsf\{G\}\_\{n\}\(\\theta,\\nu\)=\\frac\{1\-a\_\{n\}\}\{2\}\\theta^\{\\top\}\\Sigma\_\{n\}\\theta\-\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\log\\cosh\(r\_\{\\theta\}\+\\nu\)\+\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{\\xi\}\\log\\cosh\(q\_\{\\theta,\\xi\}\+\\nu\)\.
Setρ:=tanh⁡ν\\rho:=\\tanh\\nuandψρ\(z\):=log⁡\(cosh⁡z\+ρsinh⁡z\)\\psi\_\{\\rho\}\(z\):=\\log\(\\cosh z\+\\rho\\sinh z\)\. The identitylog⁡cosh⁡\(ν\+z\)−log⁡cosh⁡ν=ψρ\(z\)\\log\\cosh\(\\nu\+z\)\-\\log\\cosh\\nu=\\psi\_\{\\rho\}\(z\)gives the continuous compactification

𝖦¯n\(θ,ρ\)=1−an2θ⊤Σnθ−𝔼ℙnψρ\(rθ\)\+𝔼ℙn𝔼ξψρ\(qθ,ξ\),ρ∈\[−1,1\]\.\\overline\{\\mathsf\{G\}\}\_\{n\}\(\\theta,\\rho\)=\\frac\{1\-a\_\{n\}\}\{2\}\\theta^\{\\top\}\\Sigma\_\{n\}\\theta\-\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\psi\_\{\\rho\}\(r\_\{\\theta\}\)\+\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{\\xi\}\\psi\_\{\\rho\}\(q\_\{\\theta,\\xi\}\),\\qquad\\rho\\in\[\-1,1\]\.Indeed,cosh⁡z\+ρsinh⁡z\>0\\cosh z\+\\rho\\sinh z\>0on\[−1,1\]\[\-1,1\], andψ±1\(z\)=±z\\psi\_\{\\pm 1\}\(z\)=\\pm z\. For\|ρ\|<1\|\\rho\|<1, the derivative isψρ′\(z\)=tanh⁡\(z\+arctanh⁡ρ\)\\psi\_\{\\rho\}^\{\\prime\}\(z\)=\\tanh\(z\+\\operatorname\{arctanh\}\\rho\), while the endpoint derivatives are±1\\pm 1\. Thusψρ\\psi\_\{\\rho\}is11\-Lipschitz\. DefineLn:=\{𝔼ℙn\[\(\(1−an\)\|y0\|\+σn𝔼\|ξ\|\)2\]\}1/2L\_\{n\}:=\\\{\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\[\(\(1\-a\_\{n\}\)\|y\_\{0\}\|\+\\sigma\_\{n\}\\mathbb\{E\}\|\\xi\|\)^\{2\}\]\\\}^\{1/2\}\. Cauchy–Schwarz gives

𝖦¯n\(θ,ρ\)≥1−an2‖θ‖Σn2−Ln‖θ‖Σn\.\\overline\{\\mathsf\{G\}\}\_\{n\}\(\\theta,\\rho\)\\geq\\frac\{1\-a\_\{n\}\}\{2\}\\\|\\theta\\\|\_\{\\Sigma\_\{n\}\}^\{2\}\-L\_\{n\}\\\|\\theta\\\|\_\{\\Sigma\_\{n\}\}\.Thus the compactified criterion is coercive in fitted\-value spaceIm⁡\(X\)\\operatorname\{Im\}\(X\), where the design matrixX∈ℝn×dX\\in\\mathbb\{R\}^\{n\\times d\}has rows\(𝐱\(i\)\)⊤\(\\mathbf\{x\}^\{\(i\)\}\)^\{\\top\}\. It has a compact nonempty argmin there, andX\+X^\{\+\}gives its minimum\-norm lift toℝd\\mathbb\{R\}^\{d\}\. Forn≥dn\\geq d, the positive\-density assumption makesΣn\\Sigma\_\{n\}positive definite with probability one, so the criterion is coercive inθ\\thetaitself\.

Atρ=±1\\rho=\\pm 1, direct substitution gives

𝖦¯n\(θ,±1\)=\(1−an\)\(12θ⊤Σnθ∓θ⊤mn\),bn=−1−an2mn⊤Σn\+mn\.\\overline\{\\mathsf\{G\}\}\_\{n\}\(\\theta,\\pm 1\)=\(1\-a\_\{n\}\)\\left\(\\frac\{1\}\{2\}\\theta^\{\\top\}\\Sigma\_\{n\}\\theta\\mp\\theta^\{\\top\}m\_\{n\}\\right\),\\qquad b\_\{n\}=\-\\frac\{1\-a\_\{n\}\}\{2\}m\_\{n\}^\{\\top\}\\Sigma\_\{n\}^\{\+\}m\_\{n\}\.Herebnb\_\{n\}is the common boundary minimum becausemn∈Range⁡\(Σn\)m\_\{n\}\\in\\operatorname\{Range\}\(\\Sigma\_\{n\}\)\. Forn≥dn\\geq d, we haveΣn\+=Σn−1\\Sigma\_\{n\}^\{\+\}=\\Sigma\_\{n\}^\{\-1\}almost surely\. The weak law gives

Σn→ℙΣ,mn→ℙΣθ∗tanhν∗,bn→ℙb∞:=−12tanh2\(ν∗\)\(θ∗\)⊤Σθ∗\.\\Sigma\_\{n\}\\xrightarrow\{\\mathbb\{P\}\}\\Sigma,\\qquad m\_\{n\}\\xrightarrow\{\\mathbb\{P\}\}\\Sigma\\theta^\{\\ast\}\\tanh\\nu^\{\\ast\},\\qquad b\_\{n\}\\xrightarrow\{\\mathbb\{P\}\}b\_\{\\infty\}:=\-\\frac\{1\}\{2\}\\tanh^\{2\}\(\\nu^\{\\ast\}\)\(\\theta^\{\\ast\}\)^\{\\top\}\\Sigma\\theta^\{\\ast\}\.
Putu∗:=\(θ∗\)⊤𝐱u^\{\\ast\}:=\(\\theta^\{\\ast\}\)^\{\\top\}\\mathbf\{x\},r∗:=y0u∗r^\{\\ast\}:=y\_\{0\}u^\{\\ast\}, andHn:=𝔼ℙn𝔼ξlog⁡cosh⁡\(ν∗\+anr∗\+σnu∗ξ\)H\_\{n\}:=\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\\mathbb\{E\}\_\{\\xi\}\\log\\cosh\(\\nu^\{\\ast\}\+a\_\{n\}r^\{\\ast\}\+\\sigma\_\{n\}u^\{\\ast\}\\xi\)\. The Lipschitz bound yields

\|Hn−log⁡cosh⁡ν∗\|≤an𝔼ℙn\|r∗\|\+σn𝔼\|ξ\|𝔼ℙn\|u∗\|=oP\(1\)\.\|H\_\{n\}\-\\log\\cosh\\nu^\{\\ast\}\|\\leq a\_\{n\}\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\|r^\{\\ast\}\|\+\\sigma\_\{n\}\\mathbb\{E\}\|\\xi\|\\mathbb\{E\}\_\{\\mathbb\{P\}\_\{n\}\}\|u^\{\\ast\}\|=o\_\{P\}\(1\)\.The empirical moments on the right are𝒪P\(1\)\\mathcal\{O\}\_\{P\}\(1\)\. The weak law applied to the other terms therefore gives

𝖦n\(θ∗,ν∗\)→ℙg∗:=𝔼⁡\[12\(u∗\)2−log⁡cosh⁡\(r∗\+ν∗\)cosh⁡ν∗\]\.\\mathsf\{G\}\_\{n\}\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\xrightarrow\{\\mathbb\{P\}\}g\_\{\\ast\}:=\\mathbb\{E\}\\\!\\left\[\\frac\{1\}\{2\}\(u^\{\\ast\}\)^\{2\}\-\\log\\frac\{\\cosh\(r^\{\\ast\}\+\\nu^\{\\ast\}\)\}\{\\cosh\\nu^\{\\ast\}\}\\right\]\.The boundary gap has the distributional representation

b∞−g∗=𝔼𝐱𝖪𝖫\(p\(⋅∣𝐱;θ∗,ν∗\)∥𝒩\(tanhν∗u∗,1\)\)\>0\.b\_\{\\infty\}\-g\_\{\\ast\}=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\KL\\\!\\left\(p\(\\,\\cdot\\mid\\mathbf\{x\};\\theta^\{\\ast\},\\nu^\{\\ast\}\)\\parallel\\mathcal\{N\}\(\\tanh\\nu^\{\\ast\}u^\{\\ast\},1\)\\right\)\>0\.The inequality is strict because the true conditional variance is1\+sech2⁡\(ν∗\)\(u∗\)2\>11\+\\sech^\{2\}\(\\nu^\{\\ast\}\)\(u^\{\\ast\}\)^\{2\}\>1for almost every𝐱\\mathbf\{x\}, whereas the boundary Gaussian has variance one\. Consequently,

ℙ\{𝖦n\(θ∗,ν∗\)<bn\}⟶1\.\\mathbb\{P\}\\\{\\mathsf\{G\}\_\{n\}\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)<b\_\{n\}\\\}\\longrightarrow 1\.On this event, every compactified minimizer has\|ρ\|<1\|\\rho\|<1and maps throughν=arctanh⁡ρ\\nu=\\operatorname\{arctanh\}\\rhoto an exact finite minimizer\. Hence

ℙ\{argminϑ∈ℝd\+1ℛ^n,Tn𝖲𝖬\(ϑ\)≠∅\}⟶1\.\\mathbb\{P\}\\\!\\left\\\{\\argmin\_\{\\vartheta\\in\\mathbb\{R\}^\{d\+1\}\}\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\)\\neq\\varnothing\\right\\\}\\longrightarrow 1\.
For measurability, writeu=Xθ∈ℝnu=X\\theta\\in\\mathbb\{R\}^\{n\}\. The identityu∈Im⁡\(X\)u\\in\\operatorname\{Im\}\(X\)if and only ifXX\+u=uXX^\{\+\}u=ugives a measurable graph for the fitted\-value space\. We extend the compactified criterion by\+∞\+\\inftyoff this space\. The measurable minimum theorem then gives a measurable compact argmin correspondence onℝn×\[−1,1\]\\mathbb\{R\}^\{n\}\\times\[\-1,1\]\. We minimize\|ρ\|\|\\rho\|over this correspondence, take a measurable selector\(u~n,ρ~n\)\(\\widetilde\{u\}\_\{n\},\\widetilde\{\\rho\}\_\{n\}\), and setθ~n:=X\+u~n\\widetilde\{\\theta\}\_\{n\}:=X^\{\+\}\\widetilde\{u\}\_\{n\}\. This lift is measurable becauseX↦X\+X\\mapsto X^\{\+\}is Borel measurable\. The selector satisfies

\{argminϑ∈ℝd\+1ℛ^n,Tn𝖲𝖬\(ϑ\)≠∅\}=\{\|ρ~n\|<1\}\.\\left\\\{\\argmin\_\{\\vartheta\\in\\mathbb\{R\}^\{d\+1\}\}\\widehat\{\\mathcal\{R\}\}\_\{n,T\_\{n\}\}^\{\\,\\mathsf\{SM\}\}\(\\vartheta\)\\neq\\varnothing\\right\\\}=\\\{\|\\widetilde\{\\rho\}\_\{n\}\|<1\\\}\.The event on the left is therefore measurable, andarctanh⁡ρ~n\\operatorname\{arctanh\}\\widetilde\{\\rho\}\_\{n\}gives the claimed exact selection on that event\. ∎

### Appendix CProofs of Cross\-Entropy and EM Decompositions for Fixed\-Scale Score Matching

This appendix restates and proves the exact fixed\-scale decompositions, gradient identities, and noise\-limit results from Section[4](https://arxiv.org/html/2609.05688#S4): Propositions[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1),[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2),[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6),[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7), and[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8), Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3), and Corollaries[4\.4](https://arxiv.org/html/2609.05688#S4.Thmstyle4)and[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5)\.

###### C\.1Proposition[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)\.

The EM update rules for the regression parametersθt\\theta\_\{t\}and the imbalance parameterν\\nuare

M⁡\(θt,ν\)=Σ−1𝔼∗\[𝐱yttanh⁡\(μtyt\+ν\)\],N⁡\(θt,ν\)=𝔼∗\[tanh⁡\(μtyt\+ν\)\]\.M\(\\theta\_\{t\},\\nu\)=\\Sigma^\{\-1\}\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\],\\;\\;N\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\.They also satisfy

⟨ΣM\(θt,ν\),θt⟩=𝔼∗\[μtyttanh⁡\(μtyt\+ν\)\]=𝔼∗\[yt2−yt∂ytF⁡\(yt,μt,ν\)\]\.\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]=\\mathbb\{E\}\_\{\\ast\}\[y\_\{t\}^\{2\}\-y\_\{t\}\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]\.Their derivatives satisfy

Σ∇νM\(θt,ν\)=∇θtN\(θt,ν\)=𝔼∗\[𝐱ytsech2⁡\(μtyt\+ν\)\]\.\\Sigma\\nabla\_\{\\nu\}M\(\\theta\_\{t\},\\nu\)=\\nabla\_\{\\theta\_\{t\}\}N\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}y\_\{t\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\.

###### Proof for Proposition[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)\.

The calculation adapts Appendix B of[Luo and Hashemi \(2024\)](https://arxiv.org/html/2609.05688#bib.bib24)and Proposition 3 of[Luo and Hashemi \(2025\)](https://arxiv.org/html/2609.05688#bib.bib25)to the diffused parameterθt\\theta\_\{t\}\. Those results use the clean\-scale parameter\. Here we replace it withθt\\theta\_\{t\}from Equation \([4](https://arxiv.org/html/2609.05688#S2.E4)\)\.

The marginal law of𝐱\\mathbf\{x\}does not depend on\(θt,ν\)\(\\theta\_\{t\},\\nu\), so−lnp\(yt,𝐱∣θt,π\)=F\(yt,μt,ν\)−lnp\(𝐱\)\-\\ln p\(y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\},\\pi\)=F\(y\_\{t\},\\mu\_\{t\},\\nu\)\-\\ln p\(\\mathbf\{x\}\)\. Sinceμt=⟨θt,𝐱⟩\\mu\_\{t\}=\\langle\\theta\_\{t\},\\mathbf\{x\}\\rangle, differentiation gives

M\(θt,ν\)=θt−𝔼\[𝐱𝐱⊤\]−1∇θt𝔼𝐱𝔼yt\|𝐱;θt∗,π∗\[−lnp\(yt,𝐱∣θt,π\)\]=θt−Σ−1𝔼𝐱𝔼yt\|𝐱;θt∗,π∗\[𝐱∂μtF\(yt,μt,ν\)\],M\(\\theta\_\{t\},\\nu\)=\\theta\_\{t\}\-\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]^\{\-1\}\\nabla\_\{\\theta\_\{t\}\}\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\-\\ln p\(y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\},\\pi\)\]=\\theta\_\{t\}\-\\Sigma^\{\-1\}\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\mathbf\{x\}\\partial\_\{\\mu\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\],N\(θt,ν\)=tanhν−∇ν𝔼𝐱𝔼yt\|𝐱;θt∗,π∗\[−lnp\(yt,𝐱∣θt,π\)\]=tanhν−𝔼𝐱𝔼yt\|𝐱;θt∗,π∗\[∂νF\(yt,μt,ν\)\]\.N\(\\theta\_\{t\},\\nu\)=\\tanh\\nu\-\\nabla\_\{\\nu\}\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\-\\ln p\(y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\},\\pi\)\]=\\tanh\\nu\-\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\mathbb\{E\}\_\{y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\[\\partial\_\{\\nu\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]\.Lemma[A\.17](https://arxiv.org/html/2609.05688#A1.Thmstyle17)gives∂μtF=μt−yttanh⁡\(μtyt\+ν\)\\partial\_\{\\mu\_\{t\}\}F=\\mu\_\{t\}\-y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)and∂νF=tanh⁡ν−tanh⁡\(μtyt\+ν\)\\partial\_\{\\nu\}F=\\tanh\\nu\-\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\. Substitution yields

M⁡\(θt,ν\)=Σ−1𝔼∗\[𝐱yttanh⁡\(μtyt\+ν\)\],N⁡\(θt,ν\)=𝔼∗\[tanh⁡\(μtyt\+ν\)\]\.M\(\\theta\_\{t\},\\nu\)=\\Sigma^\{\-1\}\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\],\\;\\;N\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\.The same derivative formulas implyμt∂μtF−μt2=yt∂ytF−yt2=−μtyttanh⁡\(μtyt\+ν\)\\mu\_\{t\}\\partial\_\{\\mu\_\{t\}\}F\-\\mu\_\{t\}^\{2\}=y\_\{t\}\\partial\_\{y\_\{t\}\}F\-y\_\{t\}^\{2\}=\-\\mu\_\{t\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\. Therefore

⟨ΣM\(θt,ν\),θt⟩=𝔼∗\[μtyttanh⁡\(μtyt\+ν\)\]=𝔼∗\[yt2−yt∂ytF⁡\(yt,μt,ν\)\]\.\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}y\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]=\\mathbb\{E\}\_\{\\ast\}\[y\_\{t\}^\{2\}\-y\_\{t\}\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]\.The identitytanh′⁡\(u\)=sech2⁡\(u\)\\tanh^\{\\prime\}\(u\)=\\sech^\{2\}\(u\)and the relation∇θtμt=𝐱\\nabla\_\{\\theta\_\{t\}\}\\mu\_\{t\}=\\mathbf\{x\}give the derivative formula below\. The dominated convergence theorem justifies moving the derivatives through𝔼∗\\mathbb\{E\}\_\{\\ast\}\([Durrett, 2019](https://arxiv.org/html/2609.05688#bib.bib10), Theorem 1\.5\.8\)\.

Σ∇νM\(θt,ν\)=𝔼∗\[𝐱yt∇νtanh\(μtyt\+ν\)\]=𝔼∗\[𝐱ytsech2\(μtyt\+ν\)\]=𝔼∗\[∇θttanh\(μtyt\+ν\)\]=∇θtN\(θt,ν\)\.\\Sigma\\nabla\_\{\\nu\}M\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}y\_\{t\}\\nabla\_\{\\nu\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]=\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}y\_\{t\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]=\\mathbb\{E\}\_\{\\ast\}\[\\nabla\_\{\\theta\_\{t\}\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]=\\nabla\_\{\\theta\_\{t\}\}N\(\\theta\_\{t\},\\nu\)\.∎

###### C\.2Proposition[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2)\.

The cross\-entropyℋ⁡\(θt,ν\)\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)defined in Section[2\.3](https://arxiv.org/html/2609.05688#S2.SS3)satisfies

∇θtℋ\(θt,ν\)=Σ⁡\(θt−M⁡\(θt,ν\)\),∇νℋ\(θt,ν\)=tanh⁡ν−N⁡\(θt,ν\)\.\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)=\\Sigma\\left\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\\right\),\\;\\;\\nabla\_\{\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)=\\tanh\\nu\-N\(\\theta\_\{t\},\\nu\)\.

###### Proof for Proposition[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2)\.

The calculation in the preceding proof gives

M⁡\(θt,ν\)=θt−Σ−1𝔼∗\[𝐱∂μtF⁡\(yt,μt,ν\)\],N⁡\(θt,ν\)=tanh⁡ν−𝔼∗\[∂νF⁡\(yt,μt,ν\)\]\.M\(\\theta\_\{t\},\\nu\)=\\theta\_\{t\}\-\\Sigma^\{\-1\}\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}\\,\\partial\_\{\\mu\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\],\\qquad N\(\\theta\_\{t\},\\nu\)=\\tanh\\nu\-\\mathbb\{E\}\_\{\\ast\}\[\\partial\_\{\\nu\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]\.The same integrable bounds permit differentiation through the expectation\([Durrett, 2019](https://arxiv.org/html/2609.05688#bib.bib10), Theorem 1\.5\.8\)\. Since∇θtμt=𝐱\\nabla\_\{\\theta\_\{t\}\}\\mu\_\{t\}=\\mathbf\{x\}, rearrangement gives

Σ⁡\(θt−M⁡\(θt,ν\)\)=𝔼∗\[𝐱∂μtF⁡\(yt,μt,ν\)\]=𝔼∗\[∇θtF\(yt,μt,ν\)\]=∇θt𝔼∗\[F⁡\(yt,μt,ν\)\]=∇θtℋ\(θt,ν\),\\Sigma\\left\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\\right\)=\\mathbb\{E\}\_\{\\ast\}\[\\mathbf\{x\}\\partial\_\{\\mu\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]=\\mathbb\{E\}\_\{\\ast\}\[\\nabla\_\{\\theta\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]=\\nabla\_\{\\theta\_\{t\}\}\\mathbb\{E\}\_\{\\ast\}\[F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]=\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),tanh⁡ν−N⁡\(θt,ν\)=𝔼∗\[∂νF⁡\(yt,μt,ν\)\]=∇ν𝔼∗\[F⁡\(yt,μt,ν\)\]=∇νℋ\(θt,ν\)\.\\tanh\\nu\-N\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\partial\_\{\\nu\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]=\\nabla\_\{\\nu\}\\mathbb\{E\}\_\{\\ast\}\[F\(y\_\{t\},\\mu\_\{t\},\\nu\)\]=\\nabla\_\{\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\.∎

###### C\.3Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3)\.

We define the expected squared activation𝒜⁡\(θt,ν\):=12𝔼∗\[μt2tanh2⁡\(μtyt\+ν\)\]\\mathcal\{A\}\(\\theta\_\{t\},\\nu\):=\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}^\{2\}\\tanh^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]and its ground truth counterpart𝒜⁡\(θt∗,ν∗\):=12𝔼∗\[\(μt∗\)2tanh2⁡\(μt∗yt\+ν∗\)\]\\mathcal\{A\}\(\\theta\_\{t\}^\{\*\},\\nu^\{\*\}\):=\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\ast\}\[\(\\mu\_\{t\}^\{\*\}\)^\{2\}\\tanh^\{2\}\(\\mu\_\{t\}^\{\*\}y\_\{t\}\+\\nu^\{\\ast\}\)\]\. The Score Matching loss satisfiesα¯tℒt\(θ,ν\)=⟨∇θtℋ\(θt,ν\),θt⟩−𝒜⁡\(θt,ν\)\+𝒜⁡\(θt∗,ν∗\)\.\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\langle\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\-\\mathcal\{A\}\(\\theta\_\{t\},\\nu\)\+\\mathcal\{A\}\(\\theta\_\{t\}^\{\*\},\\nu^\{\*\}\)\.We also define the latent varianceV⁡\(θt,ν\)≡12𝔼∗\[μt2sech2⁡\(μtyt\+ν\)\]V\(\\theta\_\{t\},\\nu\)\\equiv\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\. The equivalent representation isα¯tℒt\(θ,ν\)=‖θt‖Σ2/2−⟨ΣM\(θt,ν\),θt⟩\+‖θt∗‖Σ2/2\+V⁡\(θt,ν\)−V⁡\(θt∗,ν∗\)\.\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\\|\\theta\_\{t\}\\\|\_\{\\Sigma\}^\{2\}/2\-\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\+\\\|\\theta\_\{t\}^\{\\ast\}\\\|\_\{\\Sigma\}^\{2\}/2\+V\(\\theta\_\{t\},\\nu\)\-V\(\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)\.HereV⁡\(θt∗,ν∗\)≡12𝔼∗\[\(μt∗\)2sech2⁡\(μt∗yt\+ν∗\)\]V\(\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)\\equiv\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\ast\}\[\(\\mu\_\{t\}^\{\\ast\}\)^\{2\}\\sech^\{2\}\(\\mu\_\{t\}^\{\\ast\}y\_\{t\}\+\\nu^\{\\ast\}\)\]does not depend on the candidate parameters\.###### Proof for Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3)\.

The candidate score issθt,ν\(yt,𝐱\)=−yt\+μttanh⁡\(μtyt\+ν\)s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)=\-y\_\{t\}\+\\mu\_\{t\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\. Together with its ground\-truth counterpart, it gives

ℒt\(θ,ν\)=12α¯t𝔼yt,𝐱\|θt∗,π∗‖sθt,ν\(yt,𝐱\)−sθt∗,ν∗\(yt,𝐱\)‖2\.\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{y\_\{t\},\\mathbf\{x\}\\mid\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\}\\\|s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},\\mathbf\{x\}\)\-s\_\{\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\}\(y\_\{t\},\\mathbf\{x\}\)\\\|^\{2\}\.Lemmas[A\.2](https://arxiv.org/html/2609.05688#A1.Thmstyle2)and[A\.3](https://arxiv.org/html/2609.05688#A1.Thmstyle3)givesθt,ν=−∂ytF\(yt,μt,ν\)s\_\{\\theta\_\{t\},\\nu\}=\-\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)andp⁡\(yt∣𝐱;θt∗,π∗\)=exp⁡\[−F⁡\(yt,μt∗,ν∗\)\]p\(y\_\{t\}\\mid\\mathbf\{x\};\\theta\_\{t\}^\{\\ast\},\\pi^\{\\ast\}\)=\\exp\[\-F\(y\_\{t\},\\mu\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)\]\. Hence

ℒt\(θ,ν\)=12α¯t𝔼𝐱∫ℝ∥−∂ytF\(yt,μt,ν\)\+∂ytF\(yt,μt∗,ν∗\)∥2exp\(−F\(yt,μt∗,ν∗\)\)dyt\.\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\int\_\{\\mathbb\{R\}\}\\\|\-\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\+\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)\\\|^\{2\}\\exp\(\-F\(y\_\{t\},\\mu\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)\)\\mathrm\{d\}y\_\{t\}\.Integration by parts inyty\_\{t\}is valid because the boundary terms vanish\. Applying it once to the mixed term and once to the ground\-truth score term gives

2α¯tℒt\(θ,ν\)=𝔼∗\[‖∂ytF⁡\(yt,μt,ν\)‖2−2∂ytyt2F⁡\(yt,μt,ν\)\+∂ytyt2F⁡\(yt,μt∗,ν∗\)\]\.2\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\mathbb\{E\}\_\{\\ast\}\\left\[\\\|\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\\\|^\{2\}\-2\\partial^\{2\}\_\{y\_\{t\}y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\+\\partial^\{2\}\_\{y\_\{t\}y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\}^\{\*\},\\nu^\{\*\}\)\\right\]\.Lemma[A\.17](https://arxiv.org/html/2609.05688#A1.Thmstyle17)gives‖∂ytF‖2−∂ytyt2F=2\[yt∂ytF−yt2\]\+\(μt2\+yt2−1\)\\\|\\partial\_\{y\_\{t\}\}F\\\|^\{2\}\-\\partial^\{2\}\_\{y\_\{t\}y\_\{t\}\}F=2\[y\_\{t\}\\partial\_\{y\_\{t\}\}F\-y\_\{t\}^\{2\}\]\+\(\\mu\_\{t\}^\{2\}\+y\_\{t\}^\{2\}\-1\)and∂ytyt2F=1−μt2sech2⁡\(μtyt\+ν\)\\partial^\{2\}\_\{y\_\{t\}y\_\{t\}\}F=1\-\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\. Equation \([4](https://arxiv.org/html/2609.05688#S2.E4)\) also gives𝔼⁡\[yt2∣𝐱\]=\(μt∗\)2\+1\\mathbb\{E\}\[y\_\{t\}^\{2\}\\mid\\mathbf\{x\}\]=\(\\mu\_\{t\}^\{\\ast\}\)^\{2\}\+1\. These identities yield

2α¯tℒt\(θ,ν\)=2𝔼∗\[yt∂ytF⁡\(yt,μt,ν\)−yt2\]\+𝔼𝐱\[μt2\+\(μt∗\)2\]\+𝔼∗\[μt2sech2⁡\(μtyt\+ν\)\]−𝔼∗\[\(μt∗\)2sech2⁡\(μt∗yt\+ν∗\)\]\.2\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=2\\mathbb\{E\}\_\{\\ast\}\\left\[y\_\{t\}\\partial\_\{y\_\{t\}\}F\(y\_\{t\},\\mu\_\{t\},\\nu\)\-y\_\{t\}^\{2\}\\right\]\+\\mathbb\{E\}\_\{\\mathbf\{x\}\}\[\\mu\_\{t\}^\{2\}\+\(\\mu\_\{t\}^\{\*\}\)^\{2\}\]\+\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\-\\mathbb\{E\}\_\{\\ast\}\[\(\\mu\_\{t\}^\{\*\}\)^\{2\}\\sech^\{2\}\(\\mu\_\{t\}^\{\*\}y\_\{t\}\+\\nu^\{\\ast\}\)\]\.Proposition[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)identifies the first expectation as−⟨ΣM\(θt,ν\),θt⟩\-\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\. The covariance identity gives𝔼⁡\[μt2\+\(μt∗\)2\]=‖θt‖Σ2\+‖θt∗‖Σ2\\mathbb\{E\}\[\\mu\_\{t\}^\{2\}\+\(\\mu\_\{t\}^\{\\ast\}\)^\{2\}\]=\\\|\\theta\_\{t\}\\\|\_\{\\Sigma\}^\{2\}\+\\\|\\theta\_\{t\}^\{\\ast\}\\\|\_\{\\Sigma\}^\{2\}\. Thus

2α¯tℒt\(θ,ν\)=‖θt‖Σ2−2⟨ΣM\(θt,ν\),θt⟩\+‖θt∗‖Σ2\+𝔼∗\[μt2sech2⁡\(μtyt\+ν\)\]−𝔼∗\[\(μt∗\)2sech2⁡\(μt∗yt\+ν∗\)\]\.2\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\\|\\theta\_\{t\}\\\|\_\{\\Sigma\}^\{2\}\-2\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\+\\\|\\theta\_\{t\}^\{\*\}\\\|\_\{\\Sigma\}^\{2\}\+\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\-\\mathbb\{E\}\_\{\\ast\}\[\(\\mu\_\{t\}^\{\*\}\)^\{2\}\\sech^\{2\}\(\\mu\_\{t\}^\{\*\}y\_\{t\}\+\\nu^\{\\ast\}\)\]\.This is the latent\-variance representation\. For the activation form, we use Proposition[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2), the identitysech2⁡\(u\)=1−tanh2⁡\(u\)\\sech^\{2\}\(u\)=1\-\\tanh^\{2\}\(u\), and the two covariance identities\. We obtain

2α¯tℒt\(θ,ν\)=2⟨∇θtℋ\(θt,ν\),θt⟩−𝔼∗\[μt2tanh2⁡\(μtyt\+ν\)\]\+𝔼∗\[\(μt∗\)2tanh2⁡\(μt∗yt\+ν∗\)\]\.2\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=2\\langle\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\-\\mathbb\{E\}\_\{\\ast\}\[\\mu\_\{t\}^\{2\}\\tanh^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\+\\mathbb\{E\}\_\{\\ast\}\[\(\\mu\_\{t\}^\{\*\}\)^\{2\}\\tanh^\{2\}\(\\mu\_\{t\}^\{\*\}y\_\{t\}\+\\nu^\{\\ast\}\)\]\.Division by two and the definitions of𝒜⁡\(θt,ν\)\\mathcal\{A\}\(\\theta\_\{t\},\\nu\)and𝒜⁡\(θt∗,ν∗\)\\mathcal\{A\}\(\\theta\_\{t\}^\{\\ast\},\\nu^\{\\ast\}\)complete the proof\. ∎

###### C\.4Corollary[4\.4](https://arxiv.org/html/2609.05688#S4.Thmstyle4)\.

The gradients ofα¯tℒt\(θ,ν\)\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)with respect toθt\\theta\_\{t\}andν\\nuare

∇θt\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\theta\_\{t\}\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=∇θtℋ\(θt,ν\)\+∇θt2ℋ\(θt,ν\)θt−∇θt𝒜\(θt,ν\),\\displaystyle=\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\+\\nabla^\{2\}\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\\theta\_\{t\}\-\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{A\}\(\\theta\_\{t\},\\nu\),∇ν\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\nu\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=⟨∇θtν2ℋ\(θt,ν\),θt⟩−∇ν𝒜\(θt,ν\)\.\\displaystyle=\\langle\\nabla^\{2\}\_\{\\theta\_\{t\}\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\-\\nabla\_\{\\nu\}\\mathcal\{A\}\(\\theta\_\{t\},\\nu\)\.

###### Proof for Corollary[4\.4](https://arxiv.org/html/2609.05688#S4.Thmstyle4)\.

Differentiate the first identity in Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3)with respect toθt\\theta\_\{t\}andν\\nu\. The product rule gives∇θt⟨∇θtℋ\(θt,ν\),θt⟩=∇θtℋ\(θt,ν\)\+∇θt2ℋ\(θt,ν\)θt\\nabla\_\{\\theta\_\{t\}\}\\langle\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\+\\nabla^\{2\}\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\\theta\_\{t\}and∇ν⟨∇θtℋ\(θt,ν\),θt⟩=⟨∇θtν2ℋ\(θt,ν\),θt⟩\\nabla\_\{\\nu\}\\langle\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\langle\\nabla^\{2\}\_\{\\theta\_\{t\}\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\. These formulas prove the two identities\. ∎

###### C\.5Corollary[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5)\.

The gradients ofα¯tℒt\(θ,ν\)\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)with respect toθt\\theta\_\{t\}andν\\nuare

∇θt\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\theta\_\{t\}\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=Σ⁡\(θt−M⁡\(θt,ν\)\)−\(∇θtM\(θt,ν\)\)⊤Σθt\+∇θtV\(θt,ν\),\\displaystyle=\\Sigma\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\)\-\\big\(\\nabla\_\{\\theta\_\{t\}\}M\(\\theta\_\{t\},\\nu\)\\big\)^\{\\top\}\\Sigma\\theta\_\{t\}\+\\nabla\_\{\\theta\_\{t\}\}V\(\\theta\_\{t\},\\nu\),∇ν\[α¯tℒt\(θ,ν\)\]\\displaystyle\\nabla\_\{\\nu\}\[\\bar\{\\alpha\}\_\{t\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\]=−⟨∇θtN\(θt,ν\),θt⟩\+∇νV\(θt,ν\)\.\\displaystyle=\-\\langle\\nabla\_\{\\theta\_\{t\}\}N\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\+\\nabla\_\{\\nu\}V\(\\theta\_\{t\},\\nu\)\.

###### Proof for Corollary[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5)\.

Differentiate the second identity in Main Theorem[4\.3](https://arxiv.org/html/2609.05688#S4.Thmstyle3)with respect toθt\\theta\_\{t\}andν\\nu\. The product rule gives∇θt‖θt‖Σ2=2Σθt\\nabla\_\{\\theta\_\{t\}\}\\\|\\theta\_\{t\}\\\|\_\{\\Sigma\}^\{2\}=2\\Sigma\\theta\_\{t\}and∇θt⟨ΣM\(θt,ν\),θt⟩=ΣM\(θt,ν\)\+\(∇θtM\(θt,ν\)\)⊤Σθt\\nabla\_\{\\theta\_\{t\}\}\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\Sigma M\(\\theta\_\{t\},\\nu\)\+\\big\(\\nabla\_\{\\theta\_\{t\}\}M\(\\theta\_\{t\},\\nu\)\\big\)^\{\\top\}\\Sigma\\theta\_\{t\}\. Proposition[4\.1](https://arxiv.org/html/2609.05688#S4.Thmstyle1)gives∇ν⟨ΣM\(θt,ν\),θt⟩=⟨Σ∇νM\(θt,ν\),θt⟩=⟨∇θtN\(θt,ν\),θt⟩\\nabla\_\{\\nu\}\\langle\\Sigma M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\langle\\Sigma\\nabla\_\{\\nu\}M\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle=\\langle\\nabla\_\{\\theta\_\{t\}\}N\(\\theta\_\{t\},\\nu\),\\theta\_\{t\}\\rangle\. These formulas prove the two identities\. ∎

We define the normalized EM operators at time stepttby

M¯t\(θ,ν\):=M⁡\(θt,ν\)/α¯t,N¯t\(θ,ν\):=\(N⁡\(θt,ν\)−\(1−α¯t\)tanh⁡ν\)/α¯t\.\\bar\{M\}\_\{t\}\(\\theta,\\nu\):=M\(\\theta\_\{t\},\\nu\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},\\quad\\bar\{N\}\_\{t\}\(\\theta,\\nu\):=\(N\(\\theta\_\{t\},\\nu\)\-\(1\-\\bar\{\\alpha\}\_\{t\}\)\\tanh\\nu\)/\\bar\{\\alpha\}\_\{t\}\.They satisfyθ−M¯t\(θ,ν\)=\(θt−M⁡\(θt,ν\)\)/α¯t\\theta\-\\bar\{M\}\_\{t\}\(\\theta,\\nu\)=\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},tanh⁡ν−N¯t\(θ,ν\)=\(tanh⁡ν−N⁡\(θt,ν\)\)/α¯t\\tanh\\nu\-\\bar\{N\}\_\{t\}\(\\theta,\\nu\)=\(\\tanh\\nu\-N\(\\theta\_\{t\},\\nu\)\)/\\bar\{\\alpha\}\_\{t\}, andΣ∇νM¯t\(θ,ν\)=∇θN¯t\(θ,ν\)\\Sigma\\nabla\_\{\\nu\}\\bar\{M\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\theta\}\\bar\{N\}\_\{t\}\(\\theta,\\nu\)\. We also setVt\(θ,ν\):=V⁡\(θt,ν\)/α¯tV\_\{t\}\(\\theta,\\nu\):=V\(\\theta\_\{t\},\\nu\)/\\bar\{\\alpha\}\_\{t\}\.

###### C\.6Proposition[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6)\.

The gradients of the Score Matching loss in Equation \([6](https://arxiv.org/html/2609.05688#S2.E6)\) are

∇θℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=Σ⁡\(θ−M¯t\(θ,ν\)\)−\(∇θM¯t\(θ,ν\)\)⊤Σθ\+∇θVt\(θ,ν\),\\displaystyle=\\Sigma\(\\theta\-\\bar\{M\}\_\{t\}\(\\theta,\\nu\)\)\-\\left\(\\nabla\_\{\\theta\}\\bar\{M\}\_\{t\}\(\\theta,\\nu\)\\right\)^\{\\top\}\\Sigma\\theta\+\\nabla\_\{\\theta\}V\_\{t\}\(\\theta,\\nu\),∇νℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=−⟨∇θN¯t\(θ,ν\),θ⟩\+∇νVt\(θ,ν\)\.\\displaystyle=\-\\langle\\nabla\_\{\\theta\}\\bar\{N\}\_\{t\}\(\\theta,\\nu\),\\theta\\rangle\+\\nabla\_\{\\nu\}V\_\{t\}\(\\theta,\\nu\)\.

###### Proof for Proposition[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6)\.

Recall the normalized EM operatorsM¯t\(θ,ν\):=M⁡\(θt,ν\)/α¯t\\bar\{M\}\_\{t\}\(\\theta,\\nu\):=M\(\\theta\_\{t\},\\nu\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},N¯t\(θ,ν\):=\(N⁡\(θt,ν\)−\(1−α¯t\)tanh⁡ν\)/α¯t\\bar\{N\}\_\{t\}\(\\theta,\\nu\):=\(N\(\\theta\_\{t\},\\nu\)\-\(1\-\\bar\{\\alpha\}\_\{t\}\)\\tanh\\nu\)/\\bar\{\\alpha\}\_\{t\}, andVt\(θ,ν\):=V⁡\(θt,ν\)/α¯tV\_\{t\}\(\\theta,\\nu\):=V\(\\theta\_\{t\},\\nu\)/\\bar\{\\alpha\}\_\{t\}\. These definitions guaranteeθ−M¯t\(θ,ν\)=\(θt−M⁡\(θt,ν\)\)/α¯t\\theta\-\\bar\{M\}\_\{t\}\(\\theta,\\nu\)=\(\\theta\_\{t\}\-M\(\\theta\_\{t\},\\nu\)\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\},tanh⁡ν−N¯t\(θ,ν\)=\(tanh⁡ν−N⁡\(θt,ν\)\)/α¯t\\tanh\\nu\-\\bar\{N\}\_\{t\}\(\\theta,\\nu\)=\(\\tanh\\nu\-N\(\\theta\_\{t\},\\nu\)\)/\\bar\{\\alpha\}\_\{t\}, andΣ∇νM¯t\(θ,ν\)=∇θN¯t\(θ,ν\)\\Sigma\\nabla\_\{\\nu\}\\bar\{M\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\theta\}\\bar\{N\}\_\{t\}\(\\theta,\\nu\)\. Dividing the identities of Corollary[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5)byα¯t\\bar\{\\alpha\}\_\{t\}and applying the chain rule withθt=α¯tθ\\theta\_\{t\}=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\,\\thetayields the stated gradients ofℒt\(θ,ν\)\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\. ∎

###### C\.7Proposition[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)\.

If𝔼⁡\[‖𝐱‖4\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{4\}\]<\\infty, then the gradients of the Score Matching loss satisfy the following expansions asα¯t→1\\bar\{\\alpha\}\_\{t\}\\to 1\.

∇θℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=Σ⁡\(θ−M⁡\(θ,ν\)\)−\(∇θM\(θ,ν\)\)⊤Σθ\+∇θV\(θ,ν\)\+𝒪⁡\(1−α¯t\),\\displaystyle=\\Sigma\(\\theta\-M\(\\theta,\\nu\)\)\-\\left\(\\nabla\_\{\\theta\}M\(\\theta,\\nu\)\\right\)^\{\\top\}\\Sigma\\theta\+\\nabla\_\{\\theta\}V\(\\theta,\\nu\)\+\\mathcal\{O\}\(1\-\\bar\{\\alpha\}\_\{t\}\),∇νℒt\(θ,ν\)\\displaystyle\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=−⟨∇θN\(θ,ν\),θ⟩\+∇νV\(θ,ν\)\+𝒪⁡\(1−α¯t\)\.\\displaystyle=\-\\langle\\nabla\_\{\\theta\}N\(\\theta,\\nu\),\\theta\\rangle\+\\nabla\_\{\\nu\}V\(\\theta,\\nu\)\+\\mathcal\{O\}\(1\-\\bar\{\\alpha\}\_\{t\}\)\.

###### Proof of Proposition[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)\.

We writeμ0=⟨θ,𝐱⟩\\mu\_\{0\}=\\langle\\theta,\\mathbf\{x\}\\rangle,μ0∗=⟨θ∗,𝐱⟩\\mu\_\{0\}^\{\\ast\}=\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle, anda=α¯ta=\\bar\{\\alpha\}\_\{t\}\. Since the loss depends only on the marginal law at each scale, Equation \([4](https://arxiv.org/html/2609.05688#S2.E4)\) permits the common coupling

yt=\(−1\)z\+1aμ0∗\+ε,μt=aμ0,μt∗=aμ0∗,y\_\{t\}=\(\-1\)^\{z\+1\}\\sqrt\{a\}\\,\\mu\_\{0\}^\{\\ast\}\+\\varepsilon,\\qquad\\mu\_\{t\}=\\sqrt\{a\}\\,\\mu\_\{0\},\\qquad\\mu\_\{t\}^\{\\ast\}=\\sqrt\{a\}\\,\\mu\_\{0\}^\{\\ast\},whereε∼𝒩⁡\(0,1\)\\varepsilon\\sim\\mathcal\{N\}\(0,1\)is independent of\(𝐱,z\)\(\\mathbf\{x\},z\)\. Equations \([5](https://arxiv.org/html/2609.05688#S2.E5)\) and \([6](https://arxiv.org/html/2609.05688#S2.E6)\) show that cancellation of the linear terms in the two scores gives the normalized score difference

Gt:=μ0tanh⁡\(μtyt\+ν\)−μ0∗tanh⁡\(μt∗yt\+ν∗\),ℒt\(θ,ν\)=12𝔼𝐱,z,ε\[Gt2\]\.G\_\{t\}:=\\mu\_\{0\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\-\\mu\_\{0\}^\{\\ast\}\\tanh\(\\mu\_\{t\}^\{\\ast\}y\_\{t\}\+\\nu^\{\\ast\}\),\\qquad\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\frac\{1\}\{2\}\\mathbb\{E\}\_\{\\mathbf\{x\},z,\\varepsilon\}\[G\_\{t\}^\{2\}\]\.The bounds below justify the identities

∇θℒt=𝔼⁡\[Gt∇θGt\],∇νℒt=𝔼⁡\[Gt∂νGt\]\.\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}=\\mathbb\{E\}\[G\_\{t\}\\nabla\_\{\\theta\}G\_\{t\}\],\\qquad\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}=\\mathbb\{E\}\[G\_\{t\}\\partial\_\{\\nu\}G\_\{t\}\]\.
We now control the dependence onaanear one\. Fora∈\[1/2,1\]a\\in\[1/2,1\], differentiation of the two activation arguments gives

∂a\(μtyt\)=μtyt2a\+\(−1\)z\+12μ0μ0∗,∂a\(μt∗yt\)=μt∗yt2a\+\(−1\)z\+12\(μ0∗\)2\.\\partial\_\{a\}\(\\mu\_\{t\}y\_\{t\}\)=\\frac\{\\mu\_\{t\}y\_\{t\}\}\{2a\}\+\\frac\{\(\-1\)^\{z\+1\}\}\{2\}\\mu\_\{0\}\\mu\_\{0\}^\{\\ast\},\\qquad\\partial\_\{a\}\(\\mu\_\{t\}^\{\\ast\}y\_\{t\}\)=\\frac\{\\mu\_\{t\}^\{\\ast\}y\_\{t\}\}\{2a\}\+\\frac\{\(\-1\)^\{z\+1\}\}\{2\}\(\\mu\_\{0\}^\{\\ast\}\)^\{2\}\.For each fixedc∈ℝc\\in\\mathbb\{R\}, the functiontanh⁡\(u\+c\)\\tanh\(u\+c\)and the quantitiesuktanh\(j\)⁡\(u\+c\)u^\{k\}\\tanh^\{\(j\)\}\(u\+c\), withj∈\{1,2\}j\\in\\\{1,2\\\}andk∈\{0,1,2\}k\\in\\\{0,1,2\\\}, are bounded overu∈ℝu\\in\\mathbb\{R\}\. The product and chain rules therefore give a constantCC, independent ofaa, such that

\|Gt\|\+‖∇θGt‖\+\|∂νGt\|≤C‖𝐱‖,\|∂aGt\|\+‖∂a∇θGt‖\+\|∂a∂νGt\|≤C⁡\(‖𝐱‖\+‖𝐱‖3\)\.\|G\_\{t\}\|\+\\\|\\nabla\_\{\\theta\}G\_\{t\}\\\|\+\|\\partial\_\{\\nu\}G\_\{t\}\|\\leq C\\\|\\mathbf\{x\}\\\|,\\qquad\|\\partial\_\{a\}G\_\{t\}\|\+\\\|\\partial\_\{a\}\\nabla\_\{\\theta\}G\_\{t\}\\\|\+\|\\partial\_\{a\}\\partial\_\{\\nu\}G\_\{t\}\|\\leq C\(\\\|\\mathbf\{x\}\\\|\+\\\|\\mathbf\{x\}\\\|^\{3\}\)\.Theaa\-derivatives of the two loss\-gradient integrandsGt∇θGtG\_\{t\}\\nabla\_\{\\theta\}G\_\{t\}andGt∂νGtG\_\{t\}\\partial\_\{\\nu\}G\_\{t\}are thus bounded byC⁡\(‖𝐱‖2\+‖𝐱‖4\)C\(\\\|\\mathbf\{x\}\\\|^\{2\}\+\\\|\\mathbf\{x\}\\\|^\{4\}\)\. The fourth\-moment assumption makes this envelope integrable\. Differentiation under the expectation is valid\([Folland, 1999](https://arxiv.org/html/2609.05688#bib.bib44), Theorem 2\.27\(b\)\), and

supa∈\[1/2,1\]\{‖∂a∇θℒt\(θ,ν\)‖\+\|∂a∇νℒt\(θ,ν\)\|\}<∞\.\\sup\_\{a\\in\[1/2,1\]\}\\left\\\{\\\|\\partial\_\{a\}\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\\\|\+\|\\partial\_\{a\}\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\|\\right\\\}<\\infty\.Integration fromaato one shows that both loss gradients equal their endpoint values plus𝒪⁡\(1−a\)\\mathcal\{O\}\(1\-a\)\. Ata=1a=1, the normalized operators reduce toMM,NN, andVV\. Proposition[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6)identifies the endpoint gradients with the two leading terms in the statement\. Substitutinga=α¯ta=\\bar\{\\alpha\}\_\{t\}completes the proof\.

∎

###### C\.8Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)\.

If𝔼⁡\[‖𝐱‖2\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]<\\infty, thenlimα¯t→0ℒt\(θ,ν\)=12∥θtanhν−θ∗tanhν∗∥Σ2\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\frac\{1\}\{2\}\\\|\\theta\\tanh\\nu\-\\theta^\{\\ast\}\\tanh\\nu^\{\\ast\}\\\|\_\{\\Sigma\}^\{2\}\. The condition𝔼⁡\[‖𝐱‖4\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{4\}\]<\\inftysuffices for the𝒪⁡\(α¯t\)\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\)gradient expansions

∇θℒt\(θ,ν\)=∇θlimα¯t→0ℒt\(θ,ν\)\+𝒪\(α¯t\),∇νℒt\(θ,ν\)=∇νlimα¯t→0ℒt\(θ,ν\)\+𝒪\(α¯t\)\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\theta\}\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\+\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\),\\qquad\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\nu\}\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\+\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\)asα¯t→0\\bar\{\\alpha\}\_\{t\}\\to 0\.

###### Proof of Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)\.

Withμ0=⟨θ,𝐱⟩\\mu\_\{0\}=\\langle\\theta,\\mathbf\{x\}\\rangleandμ0∗=⟨θ∗,𝐱⟩\\mu\_\{0\}^\{\\ast\}=\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle, set

f⁡\(w\):=12\[μ0tanh⁡\(ν\+μ0w\)−μ0∗tanh⁡\(ν∗\+μ0∗w\)\]2\.f\(w\):=\\frac\{1\}\{2\}\\left\[\\mu\_\{0\}\\tanh\(\\nu\+\\mu\_\{0\}w\)\-\\mu\_\{0\}^\{\\ast\}\\tanh\(\\nu^\{\\ast\}\+\\mu\_\{0\}^\{\\ast\}w\)\\right\]^\{2\}\.Under the ground\-truth model, Equation \([4](https://arxiv.org/html/2609.05688#S2.E4)\) and the normalization in Equation \([6](https://arxiv.org/html/2609.05688#S2.E6)\) give

ℒt\(θ,ν\)=𝔼𝐱,z,εt\[f⁡\(α¯tεt\+α¯t\(−1\)z\+1μ0∗\)\]\.\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\mathbb\{E\}\_\{\\mathbf\{x\},z,\\varepsilon\_\{t\}\}\\\!\\left\[f\\\!\\left\(\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\,\\varepsilon\_\{t\}\+\\bar\{\\alpha\}\_\{t\}\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}\\right\)\\right\]\.Since\|f⁡\(w\)\|≤12\(\|μ0\|\+\|μ0∗\|\)2\|f\(w\)\|\\leq\\frac\{1\}\{2\}\(\|\\mu\_\{0\}\|\+\|\\mu\_\{0\}^\{\\ast\}\|\)^\{2\}, dominated convergence under𝔼⁡\[‖𝐱‖2\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]<\\inftyyields

limα¯t→0ℒt\(θ,ν\)=𝔼𝐱\[f\(0\)\]=12∥θtanhν−θ∗tanhν∗∥Σ2\.\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\[f\(0\)\]=\\frac\{1\}\{2\}\\\|\\theta\\tanh\\nu\-\\theta^\{\\ast\}\\tanh\\nu^\{\\ast\}\\\|\_\{\\Sigma\}^\{2\}\.
Conditional on\(𝐱,z\)\(\\mathbf\{x\},z\), Gaussian integration by parts gives, fora\>0a\>0,

dda𝔼εt\[f⁡\(aεt\+a\(−1\)z\+1μ0∗\)\]=𝔼εt\[\(−1\)z\+1μ0∗f′\(aεt\+a\(−1\)z\+1μ0∗\)\+12f′′\(aεt\+a\(−1\)z\+1μ0∗\)\]\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}a\}\\mathbb\{E\}\_\{\\varepsilon\_\{t\}\}\\\!\\left\[f\\\!\\left\(\\sqrt\{a\}\\,\\varepsilon\_\{t\}\+a\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}\\right\)\\right\]=\\mathbb\{E\}\_\{\\varepsilon\_\{t\}\}\\\!\\left\[\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}f^\{\\prime\}\\\!\\left\(\\sqrt\{a\}\\,\\varepsilon\_\{t\}\+a\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}\\right\)\+\\frac\{1\}\{2\}f^\{\\prime\\prime\}\\\!\\left\(\\sqrt\{a\}\\,\\varepsilon\_\{t\}\+a\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}\\right\)\\right\]\.Forj∈\{1,2,3\}j\\in\\\{1,2,3\\\},k∈\{0,1,2\}k\\in\\\{0,1,2\\\}, andη\\etain any bounded set, the functionstanh\\tanhandvktanh\(j\)⁡\(η\+v\)v^\{k\}\\tanh^\{\(j\)\}\(\\eta\+v\)have a common finite bound\. Product and chain rules therefore give a constantC<∞C<\\infty, common to a bounded neighborhood of\(θ,ν\)\(\\theta,\\nu\), such that the derivatives inμ0\\mu\_\{0\}andν\\nuof the integrand on the right satisfy

‖𝐱‖\|∂μ0\{\(−1\)z\+1μ0∗f′\(w\)\+12f′′\(w\)\}\|\+\|∂ν\{\(−1\)z\+1μ0∗f′\(w\)\+12f′′\(w\)\}\|≤C⁡\(1\+‖𝐱‖4\)\\\|\\mathbf\{x\}\\\|\\left\|\\partial\_\{\\mu\_\{0\}\}\\left\\\{\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}f^\{\\prime\}\(w\)\+\\frac\{1\}\{2\}f^\{\\prime\\prime\}\(w\)\\right\\\}\\right\|\+\\left\|\\partial\_\{\\nu\}\\left\\\{\(\-1\)^\{z\+1\}\\mu\_\{0\}^\{\\ast\}f^\{\\prime\}\(w\)\+\\frac\{1\}\{2\}f^\{\\prime\\prime\}\(w\)\\right\\\}\\right\|\\leq C\\bigl\(1\+\\\|\\mathbf\{x\}\\\|^\{4\}\\bigr\)for everyw∈ℝw\\in\\mathbb\{R\}\. The fourth\-moment condition gives continuity of the parameter gradients ata=0a=0and permits differentiation under the expectations and integration of the preceding Gaussian identity from zero toα¯t\\bar\{\\alpha\}\_\{t\}\. Since∇θ=𝐱∂μ0\\nabla\_\{\\theta\}=\\mathbf\{x\}\\partial\_\{\\mu\_\{0\}\}, this gives

∇θℒt\(θ,ν\)=∇θlimα¯t→0ℒt\(θ,ν\)\+𝒪\(α¯t\),∇νℒt\(θ,ν\)=∇νlimα¯t→0ℒt\(θ,ν\)\+𝒪\(α¯t\),\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\theta\}\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\+\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\),\\qquad\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\nabla\_\{\\nu\}\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\+\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\),which proves the claim\. ∎

### Appendix DProofs for High\-Noise Gradient Dynamics and Score Matching Blindness

In this appendix we give the full quantitative statement \(Theorem[D\.1](https://arxiv.org/html/2609.05688#A4.Thmstyle1)\) and proof of Main Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1), prove the complementary high\-SNR blindness result in Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2), and establish the companion hard\-assignment limit that supports the comparison with population EM\.

###### D\.1Formal statement of Main Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)\.

AssumeΣ=𝔼⁡\[𝐱𝐱⊤\]=Id\\Sigma=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\]=I\_\{d\}, setc∗:=θ∗tanhν∗c^\{\*\}:=\\theta^\{\*\}\\tanh\\nu^\{\*\}, and define the high\-noise lossℒ⁡\(θ,ν\):=limα¯t→0ℒt\(θ,ν\)=12‖θtanh⁡ν−c∗‖2\.\\mathcal\{L\}\(\\theta,\\nu\):=\\lim\_\{\\bar\{\\alpha\}\_\{t\}\\to 0\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\\frac\{1\}\{2\}\\\|\\theta\\tanh\\nu\-c^\{\*\}\\\|^\{2\}\.Forϑ=\(θ,ν\)\\vartheta=\(\\theta,\\nu\), setJ⁡\(ϑ\):=12\(‖θ‖2−sinh2⁡ν\)J\(\\vartheta\):=\\frac\{1\}\{2\}\(\\\|\\theta\\\|^\{2\}\-\\sinh^\{2\}\\nu\)andx\+:=max⁡\{x,0\}x\_\{\+\}:=\\max\\\{x,0\\\}\. Givenη\>0\\eta\>0, letΦ\(ϑ\):=ϑ−η∇ℒ\(ϑ\)\\Phi\(\\vartheta\):=\\vartheta\-\\eta\\nabla\\mathcal\{L\}\(\\vartheta\)and𝒟η:=\{ϑ∈ℝd\+1:32ηmax⁡\{ℒ⁡\(ϑ\),\[J⁡\(ϑ\)\+ℒ⁡\(ϑ\)\]\+,‖c∗‖2,1\}≤1\}\.\\mathcal\{D\}\_\{\\eta\}:=\\left\\\{\\vartheta\\in\\mathbb\{R\}^\{d\+1\}:32\\eta\\max\\\!\\left\\\{\\mathcal\{L\}\(\\vartheta\),\[J\(\\vartheta\)\+\\mathcal\{L\}\(\\vartheta\)\]\_\{\+\},\\\|c^\{\*\}\\\|^\{2\},1\\right\\\}\\leq 1\\right\\\}\.The set𝒟η\\mathcal\{D\}\_\{\\eta\}is forward invariant underΦ\\Phi, and the loss is nonincreasing along its trajectories\. Ifϑ0∈𝒟η\\vartheta\_\{0\}\\in\\mathcal\{D\}\_\{\\eta\}andϑk\+1=Φ⁡\(ϑk\)\\vartheta\_\{k\+1\}=\\Phi\(\\vartheta\_\{k\}\), then, at every indexk∈ℤ≥0k\\in\\mathbb\{Z\}\_\{\\geq 0\},ϑk∈𝒟η\\vartheta\_\{k\}\\in\\mathcal\{D\}\_\{\\eta\}andℒ⁡\(ϑk\+1\)≤ℒ⁡\(ϑk\)\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\\leq\\mathcal\{L\}\(\\vartheta\_\{k\}\)\. Writeℒ0:=ℒ⁡\(ϑ0\)\\mathcal\{L\}\_\{0\}:=\\mathcal\{L\}\(\\vartheta\_\{0\}\)andB0:=4max⁡\{ℒ0,\[J⁡\(ϑ0\)\+ℒ0\]\+,‖c∗‖,1\}B\_\{0\}:=4\\max\\\{\\sqrt\{\\mathcal\{L\}\_\{0\}\},\\sqrt\{\[J\(\\vartheta\_\{0\}\)\+\\mathcal\{L\}\_\{0\}\]\_\{\+\}\},\\\|c^\{\*\}\\\|,1\\\}\.\(a\)Degenerate target\.Ifc∗=𝟎c^\{\*\}=\\mathbf\{0\}andℒ0\>0\\mathcal\{L\}\_\{0\}\>0, thenℒ\(ϑK\)≤\(ℒ0−1/2\+ηK6B0\)−2,K∈ℤ≥0\.\\mathcal\{L\}\(\\vartheta\_\{K\}\)\\leq\\left\(\\mathcal\{L\}\_\{0\}^\{\-1/2\}\+\\frac\{\\eta K\}\{\\sqrt\{6\}\\,B\_\{0\}\}\\right\)^\{\-2\},\\qquad K\\in\\mathbb\{Z\}\_\{\\geq 0\}\.Thusℒ⁡\(ϑK\)=𝒪⁡\(K−2\)\\mathcal\{L\}\(\\vartheta\_\{K\}\)=\\mathcal\{O\}\(K^\{\-2\}\)\. Ifℒ0=0\\mathcal\{L\}\_\{0\}=0, all subsequent losses vanish\.\(b\)Null saddle basin\.Supposec∗≠𝟎c^\{\*\}\\neq\\mathbf\{0\}, and define𝒮:=\{\(θ,0\):⟨θ,c∗⟩=0\},𝒲s\(𝒮\):=\{ϑ∈ℝd\+1:∃ϑs∈𝒮,limk→∞Φk\(ϑ\)=ϑs\}\.\\mathcal\{S\}:=\\\{\(\\theta,0\):\\langle\\theta,c^\{\*\}\\rangle=0\\\},\\qquad\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\):=\\left\\\{\\vartheta\\in\\mathbb\{R\}^\{d\+1\}:\\exists\\vartheta\_\{s\}\\in\\mathcal\{S\},\\ \\lim\_\{k\\to\\infty\}\\Phi^\{k\}\(\\vartheta\)=\\vartheta\_\{s\}\\right\\\}\.The exceptional set𝒟η∩𝒲s\(𝒮\)\\mathcal\{D\}\_\{\\eta\}\\cap\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)has zero\(d\+1\)\(d\+1\)\-dimensional Lebesgue measure\. Initializations in this set satisfyℒ⁡\(ϑk\)→‖c∗‖2/2\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\to\\\|c^\{\*\}\\\|^\{2\}/2\.\(c\)Generic geometric convergence\.Supposec∗≠𝟎c^\{\*\}\\neq\\mathbf\{0\}andϑ0∈𝒟η∖𝒲s\(𝒮\)\\vartheta\_\{0\}\\in\\mathcal\{D\}\_\{\\eta\}\\setminus\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)\. Then there existτ∈ℤ≥0\\tau\\in\\mathbb\{Z\}\_\{\\geq 0\}andu¯\>0\\underline\{u\}\>0, both depending onϑ0\\vartheta\_\{0\}, such thatℒ⁡\(ϑτ\+j\)≤e−ηu¯2jℒ\(ϑτ\),j∈ℤ≥0\.\\mathcal\{L\}\(\\vartheta\_\{\\tau\+j\}\)\\leq e^\{\-\\eta\\underline\{u\}^\{2\}j\}\\mathcal\{L\}\(\\vartheta\_\{\\tau\}\),\\qquad j\\in\\mathbb\{Z\}\_\{\\geq 0\}\.WithKϵ:=inf\{k∈ℤ≥0:ℒ⁡\(ϑk\)≤ϵ\}K\_\{\\epsilon\}:=\\inf\\\{k\\in\\mathbb\{Z\}\_\{\\geq 0\}:\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\leq\\epsilon\\\}, this bound givesKϵ=𝒪ϑ0,η,c∗\(log⁡\(1/ϵ\)\)K\_\{\\epsilon\}=\\mathcal\{O\}\_\{\\vartheta\_\{0\},\\eta,c^\{\*\}\}\(\\log\(1/\\epsilon\)\)asϵ↓0\\epsilon\\downarrow 0\.###### Proof of Theorem[5\.1](https://arxiv.org/html/2609.05688#S5.Thmstyle1)\.

Fixϑ0=\(θ0,ν0\)∈𝒟η\\vartheta\_\{0\}=\(\\theta\_\{0\},\\nu\_\{0\}\)\\in\\mathcal\{D\}\_\{\\eta\}, and writeJ0:=J⁡\(ϑ0\)J\_\{0\}:=J\(\\vartheta\_\{0\}\),ℒ0:=ℒ⁡\(ϑ0\)\\mathcal\{L\}\_\{0\}:=\\mathcal\{L\}\(\\vartheta\_\{0\}\),B:=B0B:=B\_\{0\}, andC:=‖c∗‖C:=\\\|c^\{\*\}\\\|\. The admissibility condition gives

2ηB2=32ηmax⁡\{ℒ0,\[J0\+ℒ0\]\+,C2,1\}≤1,η≤12B2≤132\.2\\eta B^\{2\}=32\\eta\\max\\\{\\mathcal\{L\}\_\{0\},\[J\_\{0\}\+\\mathcal\{L\}\_\{0\}\]\_\{\+\},C^\{2\},1\\\}\\leq 1,\\qquad\\eta\\leq\\frac\{1\}\{2B^\{2\}\}\\leq\\frac\{1\}\{32\}\.
Define the confinement set and its ambient convex cylinder by

Ω:=\{ϑ:ℒ\(ϑ\)≤ℒ0,J\(ϑ\)\+ℒ\(ϑ\)≤J0\+ℒ0\},Ω¯:=\{\(θ,ν\):∥θ∥≤B,ν∈ℝ\}\.\\Omega:=\\\{\\vartheta:\\mathcal\{L\}\(\\vartheta\)\\leq\\mathcal\{L\}\_\{0\},\\ J\(\\vartheta\)\+\\mathcal\{L\}\(\\vartheta\)\\leq J\_\{0\}\+\\mathcal\{L\}\_\{0\}\\\},\\qquad\\overline\{\\Omega\}:=\\\{\(\\theta,\\nu\):\\\|\\theta\\\|\\leq B,\\ \\nu\\in\\mathbb\{R\}\\\}\.We first proveΩ⊂Ω¯\\Omega\\subset\\overline\{\\Omega\}\. Neither set is claimed to be compact becauseν\\nuneed not be bounded\.

The definition ofBBgives

B≥4,C≤B4,C2≤B216,ℒ0≤B216,\[J0\+ℒ0\]\+≤B216\.B\\geq 4,\\qquad C\\leq\\frac\{B\}\{4\},\\qquad C^\{2\}\\leq\\frac\{B^\{2\}\}\{16\},\\qquad\\mathcal\{L\}\_\{0\}\\leq\\frac\{B^\{2\}\}\{16\},\\qquad\[J\_\{0\}\+\\mathcal\{L\}\_\{0\}\]\_\{\+\}\\leq\\frac\{B^\{2\}\}\{16\}\.For\(θ,ν\)∈Ω\(\\theta,\\nu\)\\in\\Omega, the triangle inequality and\(a\+b\)2≤2a2\+2b2\(a\+b\)^\{2\}\\leq 2a^\{2\}\+2b^\{2\}give

‖θ‖tanh\|ν\|≤\|θtanh⁡ν−c∗\|\+C≤2ℒ0\+C≤2C2\+4ℒ0\.\\\|\\theta\\\|\\tanh\|\\nu\|\\leq\\\|\\theta\\tanh\\nu\-c^\{\*\}\\\|\+C\\leq\\sqrt\{2\\mathcal\{L\}\_\{0\}\}\+C\\leq\\sqrt\{2C^\{2\}\+4\\mathcal\{L\}\_\{0\}\}\.Ifν=0\\nu=0, then‖θ‖2\+C2=2\[J⁡\(θ,0\)\+ℒ⁡\(θ,0\)\]\\\|\\theta\\\|^\{2\}\+C^\{2\}=2\[J\(\\theta,0\)\+\\mathcal\{L\}\(\\theta,0\)\], so membership inΩ\\Omegagives

‖θ‖2\+C2≤2\(J0\+ℒ0\)⟹J0\+ℒ0≥0,‖θ‖2≤2\(J0\+ℒ0\)≤18B2\.\\\|\\theta\\\|^\{2\}\+C^\{2\}\\leq 2\(J\_\{0\}\+\\mathcal\{L\}\_\{0\}\)\\quad\\Longrightarrow\\quad J\_\{0\}\+\\mathcal\{L\}\_\{0\}\\geq 0,\\qquad\\\|\\theta\\\|^\{2\}\\leq 2\(J\_\{0\}\+\\mathcal\{L\}\_\{0\}\)\\leq\\frac\{1\}\{8\}B^\{2\}\.Ifν≠0\\nu\\neq 0, thentanh−2⁡ν=1\+sinh−2⁡ν\\tanh^\{\-2\}\\nu=1\+\\sinh^\{\-2\}\\nu, so the loss bound above gives the first inequality below, while the energy bound definingΩ\\Omegagives the second inequality,

‖θ‖2≤\(2C2\+4ℒ0\)\(1\+sinh−2⁡ν\),‖θ‖2=sinh2⁡ν\+2J\(ϑ\)≤sinh2⁡ν\+2\(J0\+ℒ0\)\.\\\|\\theta\\\|^\{2\}\\leq\(2C^\{2\}\+4\\mathcal\{L\}\_\{0\}\)\(1\+\\sinh^\{\-2\}\\nu\),\\qquad\\\|\\theta\\\|^\{2\}=\\sinh^\{2\}\\nu\+2J\(\\vartheta\)\\leq\\sinh^\{2\}\\nu\+2\(J\_\{0\}\+\\mathcal\{L\}\_\{0\}\)\.Moreover,

2C2\+4ℒ0≤38B2,2\(J0\+ℒ0\)≤2\[J0\+ℒ0\]\+≤18B2\.2C^\{2\}\+4\\mathcal\{L\}\_\{0\}\\leq\\frac\{3\}\{8\}B^\{2\},\\qquad 2\(J\_\{0\}\+\\mathcal\{L\}\_\{0\}\)\\leq 2\[J\_\{0\}\+\\mathcal\{L\}\_\{0\}\]\_\{\+\}\\leq\\frac\{1\}\{8\}B^\{2\}\.Here we usedJ0\+ℒ0≤\[J0\+ℒ0\]\+J\_\{0\}\+\\mathcal\{L\}\_\{0\}\\leq\[J\_\{0\}\+\\mathcal\{L\}\_\{0\}\]\_\{\+\}\. Taking the minimum of the two preceding bounds, usingmin⁡\{a\+b,c\+d\}≤max⁡\{a,c\}\+min⁡\{b,d\}\\min\\\{a\+b,c\+d\\\}\\leq\\max\\\{a,c\\\}\+\\min\\\{b,d\\\}, and then usingmin⁡\{a/x,x\}≤a\\min\\\{a/x,x\\\}\\leq\\sqrt\{a\}fora≥0a\\geq 0andx\>0x\>0therefore yields

‖θ‖2≤max⁡\{2C2\+4ℒ0,2\(J0\+ℒ0\)\}\+min⁡\{2C2\+4ℒ0sinh2⁡ν,sinh2⁡ν\}≤38B2\+38B<34B2\.\\\|\\theta\\\|^\{2\}\\leq\\max\\\{2C^\{2\}\+4\\mathcal\{L\}\_\{0\},2\(J\_\{0\}\+\\mathcal\{L\}\_\{0\}\)\\\}\+\\min\\\!\\left\\\{\\frac\{2C^\{2\}\+4\\mathcal\{L\}\_\{0\}\}\{\\sinh^\{2\}\\nu\},\\sinh^\{2\}\\nu\\right\\\}\\leq\\frac\{3\}\{8\}B^\{2\}\+\\sqrt\{\\frac\{3\}\{8\}\}\\,B<\\frac\{3\}\{4\}B^\{2\}\.The last strict inequality follows fromB≥4B\\geq 4\. ThussupΩ‖θ‖≤3B/2<B\\sup\_\{\\Omega\}\\\|\\theta\\\|\\leq\\sqrt\{3\}B/2<B, provingΩ⊂Ω¯\\Omega\\subset\\overline\{\\Omega\}\. Putu:=tanh⁡ν∈\(−1,1\)u:=\\tanh\\nu\\in\(\-1,1\), so∂νu=1−u2\\partial\_\{\\nu\}u=1\-u^\{2\}\. The first derivatives are

∇θℒ=u⁡\(uθ−c∗\),∇νℒ=\(1−u2\)⟨θ,uθ−c∗⟩=\(1−u2\)\(u‖θ‖2−⟨θ,c∗⟩\)\.\\nabla\_\{\\theta\}\\mathcal\{L\}=u\(u\\theta\-c^\{\*\}\),\\qquad\\nabla\_\{\\nu\}\\mathcal\{L\}=\(1\-u^\{2\}\)\\langle\\theta,u\\theta\-c^\{\*\}\\rangle=\(1\-u^\{2\}\)\(u\\\|\\theta\\\|^\{2\}\-\\langle\\theta,c^\{\*\}\\rangle\)\.Differentiating once more gives

∇θθ2ℒ=u2Id,∇θν2ℒ=\(1−u2\)\{\(uθ−c∗\)\+uθ\}=\(1−u2\)\(2uθ−c∗\),\\nabla^\{2\}\_\{\\theta\\theta\}\\mathcal\{L\}=u^\{2\}I\_\{d\},\\qquad\\nabla^\{2\}\_\{\\theta\\nu\}\\mathcal\{L\}=\(1\-u^\{2\}\)\\\{\(u\\theta\-c^\{\*\}\)\+u\\theta\\\}=\(1\-u^\{2\}\)\(2u\\theta\-c^\{\*\}\),and

∇νν2ℒ=−2u\(1−u2\)\(u‖θ‖2−⟨θ,c∗⟩\)\+\(1−u2\)2‖θ‖2=\(1−u2\)\{2u⟨θ,c∗⟩\+\(1−3u2\)‖θ‖2\}\.\\nabla^\{2\}\_\{\\nu\\nu\}\\mathcal\{L\}=\-2u\(1\-u^\{2\}\)\(u\\\|\\theta\\\|^\{2\}\-\\langle\\theta,c^\{\*\}\\rangle\)\+\(1\-u^\{2\}\)^\{2\}\\\|\\theta\\\|^\{2\}=\(1\-u^\{2\}\)\\\{2u\\langle\\theta,c^\{\*\}\\rangle\+\(1\-3u^\{2\}\)\\\|\\theta\\\|^\{2\}\\\}\.Thus

∇2ℒ=\(u2Id\(1−u2\)\(2uθ−c∗\)\(1−u2\)\(2uθ−c∗\)⊤\(1−u2\)\{2u⟨θ,c∗⟩\+\(1−3u2\)‖θ‖2\}\)\.\\nabla^\{2\}\\mathcal\{L\}=\\begin\{pmatrix\}u^\{2\}I\_\{d\}&\(1\-u^\{2\}\)\(2u\\theta\-c^\{\*\}\)\\\\ \(1\-u^\{2\}\)\(2u\\theta\-c^\{\*\}\)^\{\\top\}&\(1\-u^\{2\}\)\\\{2u\\langle\\theta,c^\{\*\}\\rangle\+\(1\-3u^\{2\}\)\\\|\\theta\\\|^\{2\}\\\}\\end\{pmatrix\}\.By symmetry it suffices to consideru∈\[0,1\]u\\in\[0,1\]\. Sinceddu\[2u\(1−u2\)\]=2\(1−3u2\)\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}u\}\[2u\(1\-u^\{2\}\)\]=2\(1\-3u^\{2\}\), the first expression is maximized atu=1/3u=1/\\sqrt\{3\}\. In addition,\(1−u2\)\(1−3u2\)=3\(u2−2/3\)2−1/3∈\[−1/3,1\]\(1\-u^\{2\}\)\(1\-3u^\{2\}\)=3\(u^\{2\}\-2/3\)^\{2\}\-1/3\\in\[\-1/3,1\]\. Hence

max\|u\|≤1⁡\|2u\(1−u2\)\|=433,max\|u\|≤1⁡\|\(1−u2\)\(1−3u2\)\|=1\.\\max\_\{\|u\|\\leq 1\}\|2u\(1\-u^\{2\}\)\|=\\frac\{4\}\{3\\sqrt\{3\}\},\\qquad\\max\_\{\|u\|\\leq 1\}\|\(1\-u^\{2\}\)\(1\-3u^\{2\}\)\|=1\.OnΩ¯\\overline\{\\Omega\},‖θ‖≤B\\\|\\theta\\\|\\leq B,C≤B/4C\\leq B/4, andB≥4B\\geq 4, and therefore

‖∇θθ2ℒ‖2=u2≤1≤B216,\\\|\\nabla^\{2\}\_\{\\theta\\theta\}\\mathcal\{L\}\\\|\_\{2\}=u^\{2\}\\leq 1\\leq\\frac\{B^\{2\}\}\{16\},‖∇θν2ℒ‖2≤433‖θ‖\+C≤433B\+B4≤\(133\+116\)B2,\\\|\\nabla^\{2\}\_\{\\theta\\nu\}\\mathcal\{L\}\\\|\_\{2\}\\leq\\frac\{4\}\{3\\sqrt\{3\}\}\\\|\\theta\\\|\+C\\leq\\frac\{4\}\{3\\sqrt\{3\}\}B\+\\frac\{B\}\{4\}\\leq\\left\(\\frac\{1\}\{3\\sqrt\{3\}\}\+\\frac\{1\}\{16\}\\right\)B^\{2\},and

\|∇νν2ℒ\|≤433‖θ‖C\+‖θ‖2≤433BB4\+B2=\(1\+133\)B2\.\|\\nabla^\{2\}\_\{\\nu\\nu\}\\mathcal\{L\}\|\\leq\\frac\{4\}\{3\\sqrt\{3\}\}\\\|\\theta\\\|C\+\\\|\\theta\\\|^\{2\}\\leq\\frac\{4\}\{3\\sqrt\{3\}\}B\\frac\{B\}\{4\}\+B^\{2\}=\\left\(1\+\\frac\{1\}\{3\\sqrt\{3\}\}\\right\)B^\{2\}\.The last step in the mixed\-block bound usesB≤B2/4B\\leq B^\{2\}/4, which also givesB/4≤B2/16B/4\\leq B^\{2\}/16\. Decomposing the Hessian into its four block\-supported summands and applying the triangle inequality gives‖∇2ℒ‖2≤‖∇θθ2ℒ‖2\+2‖∇θν2ℒ‖2\+\|∇νν2ℒ\|\\\|\\nabla^\{2\}\\mathcal\{L\}\\\|\_\{2\}\\leq\\\|\\nabla^\{2\}\_\{\\theta\\theta\}\\mathcal\{L\}\\\|\_\{2\}\+2\\\|\\nabla^\{2\}\_\{\\theta\\nu\}\\mathcal\{L\}\\\|\_\{2\}\+\|\\nabla^\{2\}\_\{\\nu\\nu\}\\mathcal\{L\}\|\. Since1/3<3/41/\\sqrt\{3\}<3/4, the resulting coefficient satisfies19/16\+1/3<31/16<219/16\+1/\\sqrt\{3\}<31/16<2\. These bounds give

supϑ∈Ω¯‖∇2ℒ\(ϑ\)‖2≤\{116\+2\(133\+116\)\+\(1\+133\)\}B2=\(1916\+13\)B2<2B2\.\\sup\_\{\\vartheta\\in\\overline\{\\Omega\}\}\\\|\\nabla^\{2\}\\mathcal\{L\}\(\\vartheta\)\\\|\_\{2\}\\leq\\left\\\{\\frac\{1\}\{16\}\+2\\left\(\\frac\{1\}\{3\\sqrt\{3\}\}\+\\frac\{1\}\{16\}\\right\)\+\\left\(1\+\\frac\{1\}\{3\\sqrt\{3\}\}\\right\)\\right\\\}B^\{2\}=\\left\(\\frac\{19\}\{16\}\+\\frac\{1\}\{\\sqrt\{3\}\}\\right\)B^\{2\}<2B^\{2\}\.
Step 1\. Descent, confinement, and admissibility\.Sinceϑ0∈Ω\\vartheta\_\{0\}\\in\\Omega, the induction starts atk=0k=0\. Suppose as the induction hypothesis thatϑk∈Ω\\vartheta\_\{k\}\\in\\Omega, and setuk:=tanh⁡νku\_\{k\}:=\\tanh\\nu\_\{k\}andqk:=ukθk−c∗q\_\{k\}:=u\_\{k\}\\theta\_\{k\}\-c^\{\*\}\. The first component of the gradient is∇θℒ\(ϑk\)=ukqk\\nabla\_\{\\theta\}\\mathcal\{L\}\(\\vartheta\_\{k\}\)=u\_\{k\}q\_\{k\}\. Sinceη≤1/32\\eta\\leq 1/32, the residual bound is

‖qk‖≤\|uk\|‖θk‖\+C≤32B\+B4=\(32\+14\)B<54B,\\\|q\_\{k\}\\\|\\leq\|u\_\{k\}\|\\\|\\theta\_\{k\}\\\|\+C\\leq\\frac\{\\sqrt\{3\}\}\{2\}B\+\\frac\{B\}\{4\}=\\left\(\\frac\{\\sqrt\{3\}\}\{2\}\+\\frac\{1\}\{4\}\\right\)B<\\frac\{5\}\{4\}B,and the resulting one\-step movement satisfies

‖θk\+1‖=‖θk−ηukqk‖≤‖θk‖\+η\|uk\|\|qk\|≤\(32\+13254\)B=\(32\+5128\)B<B\.\\\|\\theta\_\{k\+1\}\\\|=\\\|\\theta\_\{k\}\-\\eta u\_\{k\}q\_\{k\}\\\|\\leq\\\|\\theta\_\{k\}\\\|\+\\eta\|u\_\{k\}\|\\\|q\_\{k\}\\\|\\leq\\left\(\\frac\{\\sqrt\{3\}\}\{2\}\+\\frac\{1\}\{32\}\\frac\{5\}\{4\}\\right\)B=\\left\(\\frac\{\\sqrt\{3\}\}\{2\}\+\\frac\{5\}\{128\}\\right\)B<B\.Here the last strict inequality follows from3/2<7/8\\sqrt\{3\}/2<7/8and7/8\+5/128=117/128<17/8\+5/128=117/128<1\. SinceΩ¯\\overline\{\\Omega\}imposes no restriction onν\\nu, this bound is sufficient to placeϑk\+1\\vartheta\_\{k\+1\}inΩ¯\\overline\{\\Omega\}\. Thusϑk,ϑk\+1∈Ω¯\\vartheta\_\{k\},\\vartheta\_\{k\+1\}\\in\\overline\{\\Omega\}, and convexity ofΩ¯\\overline\{\\Omega\}places the segment joining them inΩ¯\\overline\{\\Omega\}\. Taylor’s inequality on this segment and the preceding uniform Hessian bound give

ℒ\(ϑk\+1\)−ℒ\(ϑk\)≤⟨∇ℒ\(ϑk\),−η∇ℒ\(ϑk\)⟩\+B2∥η∇ℒ\(ϑk\)∥2=−η\(1−ηB2\)∥∇ℒ\(ϑk\)∥2≤−η2∥∇ℒ\(ϑk\)∥2\.\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\-\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\leq\\langle\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\),\-\\eta\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\rangle\+B^\{2\}\\\|\\eta\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}=\-\\eta\(1\-\\eta B^\{2\}\)\\\|\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\\leq\-\\frac\{\\eta\}\{2\}\\\|\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\.Here the Taylor remainder coefficient isB2=12\(2B2\)B^\{2\}=\\frac\{1\}\{2\}\(2B^\{2\}\), and the final inequality usesηB2≤1/2\\eta B^\{2\}\\leq 1/2\.

The component updates are

θk\+1=θk−ηukqk,νk\+1=νk−η⁡\(1−uk2\)⟨θk,qk⟩\.\\theta\_\{k\+1\}=\\theta\_\{k\}\-\\eta u\_\{k\}q\_\{k\},\\qquad\\nu\_\{k\+1\}=\\nu\_\{k\}\-\\eta\(1\-u\_\{k\}^\{2\}\)\\langle\\theta\_\{k\},q\_\{k\}\\rangle\.This identity gives

‖θk\+1‖2−‖θk‖2=−2ηuk⟨θk,qk⟩\+η2uk2‖qk‖2=η2uk2‖qk‖2\+2uk1−uk2\(νk\+1−νk\)\.\\\|\\theta\_\{k\+1\}\\\|^\{2\}\-\\\|\\theta\_\{k\}\\\|^\{2\}=\-2\\eta u\_\{k\}\\langle\\theta\_\{k\},q\_\{k\}\\rangle\+\\eta^\{2\}u\_\{k\}^\{2\}\\\|q\_\{k\}\\\|^\{2\}=\\eta^\{2\}u\_\{k\}^\{2\}\\\|q\_\{k\}\\\|^\{2\}\+\\frac\{2u\_\{k\}\}\{1\-u\_\{k\}^\{2\}\}\(\\nu\_\{k\+1\}\-\\nu\_\{k\}\)\.Convexity ofν↦sinh2⁡ν\\nu\\mapsto\\sinh^\{2\}\\nugives the exact supporting\-line comparison

sinh2⁡νk\+1−sinh2⁡νk≥sinh⁡\(2νk\)\(νk\+1−νk\)=2uk1−uk2\(νk\+1−νk\)\.\\sinh^\{2\}\\nu\_\{k\+1\}\-\\sinh^\{2\}\\nu\_\{k\}\\geq\\sinh\(2\\nu\_\{k\}\)\(\\nu\_\{k\+1\}\-\\nu\_\{k\}\)=\\frac\{2u\_\{k\}\}\{1\-u\_\{k\}^\{2\}\}\(\\nu\_\{k\+1\}\-\\nu\_\{k\}\)\.Combining these two displays and then using the descent inequality above yields

J⁡\(ϑk\+1\)−J⁡\(ϑk\)≤η22uk2‖qk‖2=η22‖∇θℒ\(ϑk\)‖2≤η22‖∇ℒ\(ϑk\)‖2≤η⁡\[ℒ⁡\(ϑk\)−ℒ⁡\(ϑk\+1\)\]\.J\(\\vartheta\_\{k\+1\}\)\-J\(\\vartheta\_\{k\}\)\\leq\\frac\{\\eta^\{2\}\}\{2\}u\_\{k\}^\{2\}\\\|q\_\{k\}\\\|^\{2\}=\\frac\{\\eta^\{2\}\}\{2\}\\\|\\nabla\_\{\\theta\}\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\\leq\\frac\{\\eta^\{2\}\}\{2\}\\\|\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\\leq\\eta\\bigl\[\\mathcal\{L\}\(\\vartheta\_\{k\}\)\-\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\\bigr\]\.Becauseη<1\\eta<1, combining the descent and energy\-drift inequalities gives

\[J⁡\(ϑk\+1\)\+ℒ⁡\(ϑk\+1\)\]−\[J⁡\(ϑk\)\+ℒ⁡\(ϑk\)\]≤−\(1−η\)\[ℒ⁡\(ϑk\)−ℒ⁡\(ϑk\+1\)\]≤0\.\[J\(\\vartheta\_\{k\+1\}\)\+\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\]\-\[J\(\\vartheta\_\{k\}\)\+\\mathcal\{L\}\(\\vartheta\_\{k\}\)\]\\leq\-\(1\-\\eta\)\[\\mathcal\{L\}\(\\vartheta\_\{k\}\)\-\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\]\\leq 0\.Henceϑk\+1∈Ω\\vartheta\_\{k\+1\}\\in\\Omega, and induction proves confinement\. In particular, monotonicity of the two trajectory\-dependent quantities gives the admissibility chain

max⁡\{ℒ⁡\(ϑk\+1\),\[J⁡\(ϑk\+1\)\+ℒ⁡\(ϑk\+1\)\]\+,C2,1\}≤max⁡\{ℒ0,\[J0\+ℒ0\]\+,C2,1\}≤132η\.\\max\\\{\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\),\[J\(\\vartheta\_\{k\+1\}\)\+\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\]\_\{\+\},C^\{2\},1\\\}\\leq\\max\\\{\\mathcal\{L\}\_\{0\},\[J\_\{0\}\+\\mathcal\{L\}\_\{0\}\]\_\{\+\},C^\{2\},1\\\}\\leq\\frac\{1\}\{32\\eta\}\.Thus admissibility is preserved\. In summary,

ϑk∈Ω⊂Ω¯,‖θk‖≤32B,ϑk∈𝒟η,k∈ℤ≥0\.\\vartheta\_\{k\}\\in\\Omega\\subset\\overline\{\\Omega\},\\qquad\\\|\\theta\_\{k\}\\\|\\leq\\frac\{\\sqrt\{3\}\}\{2\}B,\\qquad\\vartheta\_\{k\}\\in\\mathcal\{D\}\_\{\\eta\},\\qquad k\\in\\mathbb\{Z\}\_\{\\geq 0\}\.
Step 2\. Degenerate target\.Supposec∗=𝟎c^\{\*\}=\\mathbf\{0\}\. Ifℒ0=0\\mathcal\{L\}\_\{0\}=0, the descent inequality from Step 1 shows that all subsequent losses vanish\. We may therefore assumeℒ0\>0\\mathcal\{L\}\_\{0\}\>0and work until a possible finite iterate with zero loss, after which the same conclusion applies\.

Sincec∗=𝟎c^\{\*\}=\\mathbf\{0\}, direct calculation gives

∇θℒ=u2θ,∇νℒ=u⁡\(1−u2\)‖θ‖2,‖∇ℒ‖2=u4‖θ‖2\+u2\(1−u2\)2‖θ‖4=2ℒ\{u2\+‖θ‖2\(1−u2\)2\}\.\\nabla\_\{\\theta\}\\mathcal\{L\}=u^\{2\}\\theta,\\qquad\\nabla\_\{\\nu\}\\mathcal\{L\}=u\(1\-u^\{2\}\)\\\|\\theta\\\|^\{2\},\\qquad\\\|\\nabla\\mathcal\{L\}\\\|^\{2\}=u^\{4\}\\\|\\theta\\\|^\{2\}\+u^\{2\}\(1\-u^\{2\}\)^\{2\}\\\|\\theta\\\|^\{4\}=2\\mathcal\{L\}\\\{u^\{2\}\+\\\|\\theta\\\|^\{2\}\(1\-u^\{2\}\)^\{2\}\\\}\.Forϑ∈Ω\\vartheta\\in\\Omega, ifu2\>1/2u^\{2\}\>1/2, then

u2\+‖θ‖2\(1−u2\)2≥u2\>12≥‖θ‖\|u\|3B\.u^\{2\}\+\\\|\\theta\\\|^\{2\}\(1\-u^\{2\}\)^\{2\}\\geq u^\{2\}\>\\frac\{1\}\{2\}\\geq\\frac\{\\\|\\theta\\\|\|u\|\}\{\\sqrt\{3\}B\}\.Ifu2≤1/2u^\{2\}\\leq 1/2, then\(\|u\|−‖θ‖/2\)2≥0\(\|u\|\-\\\|\\theta\\\|/2\)^\{2\}\\geq 0gives

u2\+‖θ‖2\(1−u2\)2≥u2\+14‖θ‖2≥‖θ‖\|u\|≥‖θ‖\|u\|3B\.u^\{2\}\+\\\|\\theta\\\|^\{2\}\(1\-u^\{2\}\)^\{2\}\\geq u^\{2\}\+\\frac\{1\}\{4\}\\\|\\theta\\\|^\{2\}\\geq\\\|\\theta\\\|\|u\|\\geq\\frac\{\\\|\\theta\\\|\|u\|\}\{\\sqrt\{3\}B\}\.Here we used the confinement bound‖θ‖≤3B/2\\\|\\theta\\\|\\leq\\sqrt\{3\}B/2andB≥4B\\geq 4\. We obtain

‖∇ℒ\(ϑ\)‖2=2ℒ\(ϑ\)\{u2\+‖θ‖2\(1−u2\)2\}≥2ℒ\(ϑ\)2ℒ\(ϑ\)3B=223ℒ\(ϑ\)3/2B\.\\\|\\nabla\\mathcal\{L\}\(\\vartheta\)\\\|^\{2\}=2\\mathcal\{L\}\(\\vartheta\)\\\{u^\{2\}\+\\\|\\theta\\\|^\{2\}\(1\-u^\{2\}\)^\{2\}\\\}\\geq\\frac\{2\\mathcal\{L\}\(\\vartheta\)\\sqrt\{2\\mathcal\{L\}\(\\vartheta\)\}\}\{\\sqrt\{3\}B\}=2\\sqrt\{\\frac\{2\}\{3\}\}\\,\\frac\{\\mathcal\{L\}\(\\vartheta\)^\{3/2\}\}\{B\}\.As long asℒ⁡\(ϑk\+1\)\>0\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\>0, the supporting\-line inequality forx↦x−1/2x\\mapsto x^\{\-1/2\}, together with the descent inequality and the preceding gradient lower bound, gives

ℒ\(ϑk\+1\)−1/2−ℒ\(ϑk\)−1/2≥ℒ⁡\(ϑk\)−ℒ⁡\(ϑk\+1\)2ℒ\(ϑk\)3/2≥η‖∇ℒ\(ϑk\)‖24ℒ\(ϑk\)3/2≥η6B\.\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)^\{\-1/2\}\-\\mathcal\{L\}\(\\vartheta\_\{k\}\)^\{\-1/2\}\\geq\\frac\{\\mathcal\{L\}\(\\vartheta\_\{k\}\)\-\\mathcal\{L\}\(\\vartheta\_\{k\+1\}\)\}\{2\\mathcal\{L\}\(\\vartheta\_\{k\}\)^\{3/2\}\}\\geq\\frac\{\\eta\\\|\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\}\{4\\mathcal\{L\}\(\\vartheta\_\{k\}\)^\{3/2\}\}\\geq\\frac\{\\eta\}\{\\sqrt\{6\}\\,B\}\.Ifℒ⁡\(ϑK\)\>0\\mathcal\{L\}\(\\vartheta\_\{K\}\)\>0, telescoping and inversion prove

ℒ\(ϑK\)≤\(ℒ0−1/2\+ηK6B0\)−2\.\\mathcal\{L\}\(\\vartheta\_\{K\}\)\\leq\\left\(\\mathcal\{L\}\_\{0\}^\{\-1/2\}\+\\frac\{\\eta K\}\{\\sqrt\{6\}\\,B\_\{0\}\}\\right\)^\{\-2\}\.Ifℒ⁡\(ϑK\)=0\\mathcal\{L\}\(\\vartheta\_\{K\}\)=0, the same bound is immediate, and the descent inequality makes all later losses vanish\. Thus the displayed sublinear bound holds for everyK∈ℤ≥0K\\in\\mathbb\{Z\}\_\{\\geq 0\}\.

For Steps 3 through 6, assumec∗≠𝟎c^\{\*\}\\neq\\mathbf\{0\}, soC\>0C\>0\.

Step 3\. Saddle points and the boundary of the sign regions\.Ifϑ=\(θ,0\)∈𝒮\\vartheta=\(\\theta,0\)\\in\\mathcal\{S\}, thenu=0u=0,∇θℒ\(ϑ\)=u⁡\(uθ−c∗\)=0\\nabla\_\{\\theta\}\\mathcal\{L\}\(\\vartheta\)=u\(u\\theta\-c^\{\*\}\)=0, and∇νℒ\(ϑ\)=\(1−u2\)⟨θ,uθ−c∗⟩=−⟨θ,c∗⟩=0\\nabla\_\{\\nu\}\\mathcal\{L\}\(\\vartheta\)=\(1\-u^\{2\}\)\\langle\\theta,u\\theta\-c^\{\*\}\\rangle=\-\\langle\\theta,c^\{\*\}\\rangle=0\. HenceΦ⁡\(ϑ\)=ϑ\\Phi\(\\vartheta\)=\\vartheta, so an initialization in𝒮\\mathcal\{S\}remains fixed\.

Fix any boundary indexr∈ℤ≥0r\\in\\mathbb\{Z\}\_\{\\geq 0\}with⟨θr,c∗⟩νr=0\\langle\\theta\_\{r\},c^\{\*\}\\rangle\\nu\_\{r\}=0andϑr∉𝒮\\vartheta\_\{r\}\\notin\\mathcal\{S\}\. The boundary point enters the matching\-sign region after one update\. If⟨θr,c∗⟩=0\\langle\\theta\_\{r\},c^\{\*\}\\rangle=0andνr≠0\\nu\_\{r\}\\neq 0, thenurνr\>0u\_\{r\}\\nu\_\{r\}\>0,0<ur\(1−ur2\)/νr≤10<u\_\{r\}\(1\-u\_\{r\}^\{2\}\)/\\nu\_\{r\}\\leq 1, andη‖θr‖2≤3ηB2/4≤3/8\\eta\\\|\\theta\_\{r\}\\\|^\{2\}\\leq 3\\eta B^\{2\}/4\\leq 3/8\. Therefore,

⟨θr\+1,c∗⟩νr=ηC2urνr\>0,νr\+1νr=1−η‖θr‖2ur\(1−ur2\)νr≥1−η‖θr‖2≥58\>0,⟨θr\+1,c∗⟩νr\+1\>0\.\\langle\\theta\_\{r\+1\},c^\{\*\}\\rangle\\nu\_\{r\}=\\eta C^\{2\}u\_\{r\}\\nu\_\{r\}\>0,\\qquad\\frac\{\\nu\_\{r\+1\}\}\{\\nu\_\{r\}\}=1\-\\eta\\\|\\theta\_\{r\}\\\|^\{2\}\\frac\{u\_\{r\}\(1\-u\_\{r\}^\{2\}\)\}\{\\nu\_\{r\}\}\\geq 1\-\\eta\\\|\\theta\_\{r\}\\\|^\{2\}\\geq\\frac\{5\}\{8\}\>0,\\qquad\\langle\\theta\_\{r\+1\},c^\{\*\}\\rangle\\nu\_\{r\+1\}\>0\.If⟨θr,c∗⟩≠0\\langle\\theta\_\{r\},c^\{\*\}\\rangle\\neq 0andνr=0\\nu\_\{r\}=0, thenur=0u\_\{r\}=0andqr=−c∗q\_\{r\}=\-c^\{\*\}, so the same conclusion follows from the update

θr\+1=θr−ηurqr=θr,νr\+1=νr−η⁡\(1−ur2\)⟨θr,qr⟩=η⁡⟨θr,c∗⟩,⟨θr\+1,c∗⟩νr\+1=η⟨θr,c∗⟩2\>0\.\\theta\_\{r\+1\}=\\theta\_\{r\}\-\\eta u\_\{r\}q\_\{r\}=\\theta\_\{r\},\\qquad\\nu\_\{r\+1\}=\\nu\_\{r\}\-\\eta\(1\-u\_\{r\}^\{2\}\)\\langle\\theta\_\{r\},q\_\{r\}\\rangle=\\eta\\langle\\theta\_\{r\},c^\{\*\}\\rangle,\\qquad\\langle\\theta\_\{r\+1\},c^\{\*\}\\rangle\\nu\_\{r\+1\}=\\eta\\langle\\theta\_\{r\},c^\{\*\}\\rangle^\{2\}\>0\.Thus, in either boundary case,

⟨θr\+1,c∗⟩νr\+1\>0\.\\langle\\theta\_\{r\+1\},c^\{\*\}\\rangle\\nu\_\{r\+1\}\>0\.
Step 4\. Matching signs\.Becauseℒ⁡\(θ,ν\)=ℒ⁡\(−θ,−ν\)\\mathcal\{L\}\(\\theta,\\nu\)=\\mathcal\{L\}\(\-\\theta,\-\\nu\)andΦ⁡\(−θ,−ν\)=−Φ⁡\(θ,ν\)\\Phi\(\-\\theta,\-\\nu\)=\-\\Phi\(\\theta,\\nu\), it suffices to analyze a timerrat which⟨θr,c∗⟩\>0\\langle\\theta\_\{r\},c^\{\*\}\\rangle\>0andνr\>0\\nu\_\{r\}\>0\. In this and the next step, we use the normalized projectionwk:=⟨θk,c∗⟩/C2w\_\{k\}:=\\langle\\theta\_\{k\},c^\{\*\}\\rangle/C^\{2\}\. The component updates from Step 1 give

wk\+1=\(1−ηuk2\)wk\+ηuk,νk\+1−νk=η⁡\(1−uk2\)\(C2wk−uk‖θk‖2\)\.w\_\{k\+1\}=\(1\-\\eta u\_\{k\}^\{2\}\)w\_\{k\}\+\\eta u\_\{k\},\\qquad\\nu\_\{k\+1\}\-\\nu\_\{k\}=\\eta\(1\-u\_\{k\}^\{2\}\)\(C^\{2\}w\_\{k\}\-u\_\{k\}\\\|\\theta\_\{k\}\\\|^\{2\}\)\.
Definewmin,r:=min⁡\{wr,1\}\>0w\_\{\\min,r\}:=\\min\\\{w\_\{r\},1\\\}\>0andumin,r:=min⁡\{ur,C2wmin,r/\(2B2\)\}\>0u\_\{\\min,r\}:=\\min\\\{u\_\{r\},C^\{2\}w\_\{\\min,r\}/\(2B^\{2\}\)\\\}\>0\. The joint induction starts atk=rk=r\. Suppose thatwk≥wmin,rw\_\{k\}\\geq w\_\{\\min,r\}anduk≥umin,ru\_\{k\}\\geq u\_\{\\min,r\}for somek≥rk\\geq r\. Since1−ηuk2\>01\-\\eta u\_\{k\}^\{2\}\>0, the first matching\-sign recurrence gives

wk\+1=\(1−ηuk2\)wk\+ηuk≥\[\(1−ηuk2\)\+ηuk\]min\{wk,1\}=\[1\+ηuk\(1−uk\)\]min\{wk,1\}≥wmin,r\.w\_\{k\+1\}=\(1\-\\eta u\_\{k\}^\{2\}\)w\_\{k\}\+\\eta u\_\{k\}\\geq\\bigl\[\(1\-\\eta u\_\{k\}^\{2\}\)\+\\eta u\_\{k\}\\bigr\]\\min\\\{w\_\{k\},1\\\}=\\bigl\[1\+\\eta u\_\{k\}\(1\-u\_\{k\}\)\\bigr\]\\min\\\{w\_\{k\},1\\\}\\geq w\_\{\\min,r\}\.Moreover,0<C2wmin,r/B2≤C2/B2≤1/160<C^\{2\}w\_\{\\min,r\}/B^\{2\}\\leq C^\{2\}/B^\{2\}\\leq 1/16\. Ifuk≤C2wmin,r/B2u\_\{k\}\\leq C^\{2\}w\_\{\\min,r\}/B^\{2\}, the second matching\-sign recurrence and the confinement bound‖θk‖2≤3B2/4\\\|\\theta\_\{k\}\\\|^\{2\}\\leq 3B^\{2\}/4give

C2wk−uk‖θk‖2≥C2wmin,r−34C2wmin,r=14C2wmin,r\>0⟹νk\+1≥νk⟹uk\+1≥uk≥umin,r\.C^\{2\}w\_\{k\}\-u\_\{k\}\\\|\\theta\_\{k\}\\\|^\{2\}\\geq C^\{2\}w\_\{\\min,r\}\-\\frac\{3\}\{4\}C^\{2\}w\_\{\\min,r\}=\\frac\{1\}\{4\}C^\{2\}w\_\{\\min,r\}\>0\\quad\\Longrightarrow\\quad\\nu\_\{k\+1\}\\geq\\nu\_\{k\}\\quad\\Longrightarrow\\quad u\_\{k\+1\}\\geq u\_\{k\}\\geq u\_\{\\min,r\}\.Otherwise,uk\>C2wmin,r/B2u\_\{k\}\>C^\{2\}w\_\{\\min,r\}/B^\{2\}\. Dropping the nonnegative term involvingC2wkC^\{2\}w\_\{k\}in that recurrence and using2ηB2≤12\\eta B^\{2\}\\leq 1give

νk\+1≥arctanh⁡\(uk\)−η⁡\(1−uk2\)uk‖θk‖2≥arctanh⁡\(uk\)−ηB2uk≥arctanh⁡\(uk\)−uk2\.\\nu\_\{k\+1\}\\geq\\operatorname\{arctanh\}\(u\_\{k\}\)\-\\eta\(1\-u\_\{k\}^\{2\}\)u\_\{k\}\\\|\\theta\_\{k\}\\\|^\{2\}\\geq\\operatorname\{arctanh\}\(u\_\{k\}\)\-\\eta B^\{2\}u\_\{k\}\\geq\\operatorname\{arctanh\}\(u\_\{k\}\)\-\\frac\{u\_\{k\}\}\{2\}\.The mapx↦arctanh⁡x−x/2x\\mapsto\\operatorname\{arctanh\}x\-x/2has derivative\(1\+x2\)/\[2\(1−x2\)\]\>0\(1\+x^\{2\}\)/\[2\(1\-x^\{2\}\)\]\>0on\(0,1\)\(0,1\)\. The subtraction formula givestanh⁡\(a−b\)≥tanh⁡a−tanh⁡b\\tanh\(a\-b\)\\geq\\tanh a\-\\tanh bfora≥b≥0a\\geq b\\geq 0\. We also usetanh⁡x≤x\\tanh x\\leq xforx≥0x\\geq 0\. Hence

uk\+1≥tanh⁡\(arctanh⁡\(uk\)−uk2\)≥C2wmin,rB2−tanh⁡\(C2wmin,r2B2\)≥C2wmin,r2B2≥umin,r\.u\_\{k\+1\}\\geq\\tanh\\\!\\left\(\\operatorname\{arctanh\}\(u\_\{k\}\)\-\\frac\{u\_\{k\}\}\{2\}\\right\)\\geq\\frac\{C^\{2\}w\_\{\\min,r\}\}\{B^\{2\}\}\-\\tanh\\\!\\left\(\\frac\{C^\{2\}w\_\{\\min,r\}\}\{2B^\{2\}\}\\right\)\\geq\\frac\{C^\{2\}w\_\{\\min,r\}\}\{2B^\{2\}\}\\geq u\_\{\\min,r\}\.This completes the joint inductive step and yields

wk≥wmin,r,uk≥umin,r,k∈ℤ≥r\.w\_\{k\}\\geq w\_\{\\min,r\},\\qquad u\_\{k\}\\geq u\_\{\\min,r\},\\qquad k\\in\\mathbb\{Z\}\_\{\\geq r\}\.In particular, the matching\-sign region is forward invariant\. Fork≥rk\\geq r, the gradient satisfies

‖∇ℒ\(ϑk\)‖2≥‖∇θℒ\(ϑk\)‖2=uk2‖qk‖2=2uk2ℒ\(ϑk\)≥2umin,r2ℒ\(ϑk\)\.\\\|\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\\geq\\\|\\nabla\_\{\\theta\}\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}=u\_\{k\}^\{2\}\\\|q\_\{k\}\\\|^\{2\}=2u\_\{k\}^\{2\}\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\geq 2u\_\{\\min,r\}^\{2\}\\mathcal\{L\}\(\\vartheta\_\{k\}\)\.Combining this inequality with the descent inequality from Step 1 and using1−x≤e−x1\-x\\leq e^\{\-x\}yields

ℒ⁡\(ϑr\+j\)≤\(1−ηumin,r2\)jℒ\(ϑr\)≤e−ηumin,r2jℒ\(ϑr\),j∈ℤ≥0\.\\mathcal\{L\}\(\\vartheta\_\{r\+j\}\)\\leq\(1\-\\eta u\_\{\\min,r\}^\{2\}\)^\{j\}\\mathcal\{L\}\(\\vartheta\_\{r\}\)\\leq e^\{\-\\eta u\_\{\\min,r\}^\{2\}j\}\\mathcal\{L\}\(\\vartheta\_\{r\}\),\\qquad j\\in\\mathbb\{Z\}\_\{\\geq 0\}\.
Step 5\. Mixed signs, finite exit or convergence to𝒮\\mathcal\{S\}\.Suppose⟨θ0,c∗⟩ν0<0\\langle\\theta\_\{0\},c^\{\*\}\\rangle\\nu\_\{0\}<0, and define the exit timeT:=inf\{k∈ℤ≥0:⟨θk,c∗⟩νk≥0\}T:=\\inf\\\{k\\in\\mathbb\{Z\}\_\{\\geq 0\}:\\langle\\theta\_\{k\},c^\{\*\}\\rangle\\nu\_\{k\}\\geq 0\\\}, withinf∅:=∞\\inf\\varnothing:=\\infty\. ThusT∈ℤ≥0∪\{∞\}T\\in\\mathbb\{Z\}\_\{\\geq 0\}\\cup\\\{\\infty\\\}\. The estimates below apply at indicesk<Tk<T\.

Putθk⟂:=θk−wkc∗\\theta\_\{k\}^\{\\perp\}:=\\theta\_\{k\}\-w\_\{k\}c^\{\*\}andGk:=ℒ⁡\(ϑk\)−C2/2G\_\{k\}:=\\mathcal\{L\}\(\\vartheta\_\{k\}\)\-C^\{2\}/2\. Then⟨θk⟂,c∗⟩=0\\langle\\theta\_\{k\}^\{\\perp\},c^\{\*\}\\rangle=0,wkuk<0w\_\{k\}u\_\{k\}<0, and

Gk=12‖θk⟂‖2uk2\+C22\(wkuk−2\)wkuk\>0\.G\_\{k\}=\\frac\{1\}\{2\}\\\|\\theta\_\{k\}^\{\\perp\}\\\|^\{2\}u\_\{k\}^\{2\}\+\\frac\{C^\{2\}\}\{2\}\(w\_\{k\}u\_\{k\}\-2\)w\_\{k\}u\_\{k\}\>0\.The update has the form

θk\+1⟂=\(1−ηuk2\)θk⟂,wk\+1=\(1−ηuk2\)wk\+ηuk,νk\+1−νk=ηC2\(1−uk2\)\[wk−uk\(wk2\+‖θk⟂‖2C2\)\]\.\\theta\_\{k\+1\}^\{\\perp\}=\(1\-\\eta u\_\{k\}^\{2\}\)\\theta\_\{k\}^\{\\perp\},\\qquad w\_\{k\+1\}=\(1\-\\eta u\_\{k\}^\{2\}\)w\_\{k\}\+\\eta u\_\{k\},\\qquad\\nu\_\{k\+1\}\-\\nu\_\{k\}=\\eta C^\{2\}\(1\-u\_\{k\}^\{2\}\)\\left\[w\_\{k\}\-u\_\{k\}\\left\(w\_\{k\}^\{2\}\+\\frac\{\\\|\\theta\_\{k\}^\{\\perp\}\\\|^\{2\}\}\{C^\{2\}\}\\right\)\\right\]\.The first two mixed\-sign recurrences give‖θk⟂‖≤‖θ0⟂‖\\\|\\theta\_\{k\}^\{\\perp\}\\\|\\leq\\\|\\theta\_\{0\}^\{\\perp\}\\\|and\|wk\|≤max⁡\{\|w0\|,η\}\|w\_\{k\}\|\\leq\\max\\\{\|w\_\{0\}\|,\\eta\\\}\. These bounds hold at0≤k≤T0\\leq k\\leq TwhenT<∞T<\\infty, and at every finite index whenT=∞T=\\infty\. The final mixed\-sign recurrence has sign opposite toνk\\nu\_\{k\}, while its magnitude is at mostη⁡\[B2\+C2max⁡\{\|w0\|,η\}\]\\eta\[B^\{2\}\+C^\{2\}\\max\\\{\|w\_\{0\}\|,\\eta\\\}\]\. Hence the same index ranges satisfy\|νk\|≤max⁡\{\|ν0\|,η⁡\[B2\+C2max⁡\{\|w0\|,η\}\]\}\|\\nu\_\{k\}\|\\leq\\max\\\{\|\\nu\_\{0\}\|,\\eta\[B^\{2\}\+C^\{2\}\\max\\\{\|w\_\{0\}\|,\\eta\\\}\]\\\}\.

Setcν:=sech2⁡\(max⁡\{\|ν0\|,η⁡\[B2\+C2max⁡\{\|w0\|,η\}\]\}\)\>0c\_\{\\nu\}:=\\sech^\{2\}\(\\max\\\{\|\\nu\_\{0\}\|,\\eta\[B^\{2\}\+C^\{2\}\\max\\\{\|w\_\{0\}\|,\\eta\\\}\]\\\}\)\>0andcmix:=min⁡\{2C2/B2,cνC\}\>0c\_\{\\mathrm\{mix\}\}:=\\min\\\{2C^\{2\}/B^\{2\},c\_\{\\nu\}C\\\}\>0\. Orthogonality ofθk⟂\\theta\_\{k\}^\{\\perp\}andc∗c^\{\*\}gives‖∇θℒ\(ϑk\)‖2=C2uk2\+2uk2Gk\\\|\\nabla\_\{\\theta\}\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}=C^\{2\}u\_\{k\}^\{2\}\+2u\_\{k\}^\{2\}G\_\{k\}and\|∇νℒ\(ϑk\)\|≥cνC2\|wk\|\|\\nabla\_\{\\nu\}\\mathcal\{L\}\(\\vartheta\_\{k\}\)\|\\geq c\_\{\\nu\}C^\{2\}\|w\_\{k\}\|\. Also, the confinement bound from Step 1 givesGk≤3B2uk2/8\+C2\|wkuk\|G\_\{k\}\\leq 3B^\{2\}u\_\{k\}^\{2\}/8\+C^\{2\}\|w\_\{k\}u\_\{k\}\|\. Since14C2uk2\+cν2C4wk2≥cνC3\|wkuk\|\\frac\{1\}\{4\}C^\{2\}u\_\{k\}^\{2\}\+c\_\{\\nu\}^\{2\}C^\{4\}w\_\{k\}^\{2\}\\geq c\_\{\\nu\}C^\{3\}\|w\_\{k\}u\_\{k\}\|,

‖∇ℒ\(ϑk\)‖2≥34C2uk2\+cνC3\|wkuk\|≥cmix\(38B2uk2\+C2\|wkuk\|\)≥cmixGk\.\\\|\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\\geq\\frac\{3\}\{4\}C^\{2\}u\_\{k\}^\{2\}\+c\_\{\\nu\}C^\{3\}\|w\_\{k\}u\_\{k\}\|\\geq c\_\{\\mathrm\{mix\}\}\\left\(\\frac\{3\}\{8\}B^\{2\}u\_\{k\}^\{2\}\+C^\{2\}\|w\_\{k\}u\_\{k\}\|\\right\)\\geq c\_\{\\mathrm\{mix\}\}G\_\{k\}\.Herecmix≤2C2/B2≤1/8c\_\{\\mathrm\{mix\}\}\\leq 2C^\{2\}/B^\{2\}\\leq 1/8, so0<ηcmix/2<10<\\eta c\_\{\\mathrm\{mix\}\}/2<1\. Combining the descent inequality from Step 1 with the preceding mixed\-sign gradient lower bound yields

Gk\+1≤\(1−ηcmix2\)Gk≤e−ηcmix/2Gk,k<T\.G\_\{k\+1\}\\leq\\left\(1\-\\frac\{\\eta c\_\{\\mathrm\{mix\}\}\}\{2\}\\right\)G\_\{k\}\\leq e^\{\-\\eta c\_\{\\mathrm\{mix\}\}/2\}G\_\{k\},\\qquad k<T\.
Suppose now thatT=∞T=\\infty\. The preceding contraction estimate givesGk→0G\_\{k\}\\to 0, while‖∇ℒ\(ϑk\)‖2≥C2uk2\\\|\\nabla\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\\|^\{2\}\\geq C^\{2\}u\_\{k\}^\{2\}and the descent inequality from Step 1 gives the sharper bound below becauseℒ⁡\(ϑk\)\>C2/2\\mathcal\{L\}\(\\vartheta\_\{k\}\)\>C^\{2\}/2throughout the mixed\-sign region\.

∑k=0∞uk2≤2G0ηC2<∞\.\\sum\_\{k=0\}^\{\\infty\}u\_\{k\}^\{2\}\\leq\\frac\{2G\_\{0\}\}\{\\eta C^\{2\}\}<\\infty\.This summability bound impliesuk→0u\_\{k\}\\to 0, and henceνk→0\\nu\_\{k\}\\to 0\. Since0≤ηuk2≤1/320\\leq\\eta u\_\{k\}^\{2\}\\leq 1/32,−log⁡\(1−ηuk2\)≤\(32/31\)ηuk2\-\\log\(1\-\\eta u\_\{k\}^\{2\}\)\\leq\(32/31\)\\eta u\_\{k\}^\{2\}\. Thus the positive product∏k=0∞\(1−ηuk2\)\\prod\_\{k=0\}^\{\\infty\}\(1\-\\eta u\_\{k\}^\{2\}\)converges to a positive limit, and the first mixed\-sign recurrence shows thatθk⟂\\theta\_\{k\}^\{\\perp\}converges to someθs⟂c∗\\theta\_\{s\}\\perp c^\{\*\}\.

Becauseνk→0\\nu\_\{k\}\\to 0, its increments vanish\. In the mixed\-sign region, the final mixed\-sign recurrence satisfiesηC2\(1−uk2\)\|wk\|≤\|νk\+1−νk\|\\eta C^\{2\}\(1\-u\_\{k\}^\{2\}\)\|w\_\{k\}\|\\leq\|\\nu\_\{k\+1\}\-\\nu\_\{k\}\|, sowk→0w\_\{k\}\\to 0\. We conclude that

ϑk⟶\(θs,0\)∈𝒮\.\\vartheta\_\{k\}\\longrightarrow\(\\theta\_\{s\},0\)\\in\\mathcal\{S\}\.ThusT=∞T=\\inftyimpliesϑ0∈𝒲s\(𝒮\)\\vartheta\_\{0\}\\in\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)\.

Now supposeϑ0∉𝒲s\(𝒮\)\\vartheta\_\{0\}\\notin\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)\. If the initialization has matching signs, takeτ=0\\tau=0\. A boundary initialization outside𝒮\\mathcal\{S\}is handled by the boundary\-entry calculation in Step 3\. For a mixed\-sign initialization, the implicationT=∞⇒ϑ0∈𝒲s\(𝒮\)T=\\infty\\Rightarrow\\vartheta\_\{0\}\\in\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)proved above rules outT=∞T=\\infty\. After the finite exit, a landing in𝒮\\mathcal\{S\}would makeϑ0\\vartheta\_\{0\}belong to𝒲s\(𝒮\)\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)\. Thus the exit either has matching signs or lands on the boundary outside𝒮\\mathcal\{S\}, where the boundary\-entry calculation applies\. Thereforeτ:=inf\{k∈ℤ≥0:⟨θk,c∗⟩νk\>0\}<∞\\tau:=\\inf\\\{k\\in\\mathbb\{Z\}\_\{\\geq 0\}:\\langle\\theta\_\{k\},c^\{\*\}\\rangle\\nu\_\{k\}\>0\\\}<\\infty\.

Set

u¯:=min⁡\{\|tanh⁡ντ\|,min⁡\{\|⟨θτ,c∗⟩\|,C2\}2B2\}\>0\.\\underline\{u\}:=\\min\\\!\\left\\\{\|\\tanh\\nu\_\{\\tau\}\|,\\frac\{\\min\\\{\|\\langle\\theta\_\{\\tau\},c^\{\*\}\\rangle\|,C^\{2\}\\\}\}\{2B^\{2\}\}\\right\\\}\>0\.Applying the matching\-sign rate from Step 4 at timeτ\\taugives

ℒ⁡\(ϑτ\+j\)≤e−ηu¯2jℒ\(ϑτ\),j∈ℤ≥0\.\\mathcal\{L\}\(\\vartheta\_\{\\tau\+j\}\)\\leq e^\{\-\\eta\\underline\{u\}^\{2\}j\}\\mathcal\{L\}\(\\vartheta\_\{\\tau\}\),\\qquad j\\in\\mathbb\{Z\}\_\{\\geq 0\}\.Ifℒ⁡\(ϑτ\)=0\\mathcal\{L\}\(\\vartheta\_\{\\tau\}\)=0, thenKϵ≤τK\_\{\\epsilon\}\\leq\\tau\. Otherwise,

Kϵ≤τ\+⌈\[log⁡\(ℒ⁡\(ϑτ\)/ϵ\)\]\+ηu¯2⌉\.K\_\{\\epsilon\}\\leq\\tau\+\\left\\lceil\\frac\{\[\\log\(\\mathcal\{L\}\(\\vartheta\_\{\\tau\}\)/\\epsilon\)\]\_\{\+\}\}\{\\eta\\underline\{u\}^\{2\}\}\\right\\rceil\.This proves the pointwise logarithmic complexity\. We do not claim a uniform bound onτ\\tauoru¯\\underline\{u\}\.

Ifϑ0∈𝒲s\(𝒮\)\\vartheta\_\{0\}\\in\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\), continuity givesℒ⁡\(ϑk\)→C2/2\\mathcal\{L\}\(\\vartheta\_\{k\}\)\\to C^\{2\}/2\.

Step 6\. Null saddle basin\.We adapt the standard center\-stable\-manifold argument for avoiding saddle points under gradient descent\([Lee et al\., 2016](https://arxiv.org/html/2609.05688#bib.bib38)\)\. The argument below also treats points at which the update map is singular\. It remains to proveVold\+1\(𝒟η∩𝒲s\(𝒮\)\)=0\\mathrm\{Vol\}\_\{d\+1\}\(\\mathcal\{D\}\_\{\\eta\}\\cap\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)\)=0\. The claim is immediate if𝒟η\\mathcal\{D\}\_\{\\eta\}is empty\. Otherwise, its definition implies

η≤132max⁡\{C2,1\},ηC≤132\.\\eta\\leq\\frac\{1\}\{32\\max\\\{C^\{2\},1\\\}\},\\qquad\\eta C\\leq\\frac\{1\}\{32\}\.
*Reachable saddle limits\.*Letϑ0∈𝒟η∩𝒲s\(𝒮\)\\vartheta\_\{0\}\\in\\mathcal\{D\}\_\{\\eta\}\\cap\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)andΦk\(ϑ0\)→ϑs=\(θs,0\)∈𝒮\\Phi^\{k\}\(\\vartheta\_\{0\}\)\\to\\vartheta\_\{s\}=\(\\theta\_\{s\},0\)\\in\\mathcal\{S\}\. Applying the confinement argument above with this initialization and using the definition of𝒟η\\mathcal\{D\}\_\{\\eta\}gives‖θs‖2≤3/\(8η\)\\\|\\theta\_\{s\}\\\|^\{2\}\\leq 3/\(8\\eta\)\. Thus all such limits lie in the compact set𝒦η:=\{\(θ,0\)∈𝒮:‖θ‖2≤3/\(8η\)\}\\mathcal\{K\}\_\{\\eta\}:=\\\{\(\\theta,0\)\\in\\mathcal\{S\}:\\\|\\theta\\\|^\{2\}\\leq 3/\(8\\eta\)\\\}\.

*Local center\-stable disks\.*Atϑs=\(θs,0\)∈𝒮\\vartheta\_\{s\}=\(\\theta\_\{s\},0\)\\in\\mathcal\{S\}, the Hessian kernel is\{\(v,0\):v∈ℝd,⟨v,c∗⟩=0\}\\\{\(v,0\):v\\in\\mathbb\{R\}^\{d\},\\ \\langle v,c^\{\*\}\\rangle=0\\\}\. Direct diagonalization on its two\-dimensional complement gives

λ±\(ϑs\)=12\(‖θs‖2±‖θs‖4\+4C2\),λ−<0<λ\+\.\\lambda\_\{\\pm\}\(\\vartheta\_\{s\}\)=\\frac\{1\}\{2\}\\left\(\\\|\\theta\_\{s\}\\\|^\{2\}\\pm\\sqrt\{\\\|\\theta\_\{s\}\\\|^\{4\}\+4C^\{2\}\}\\right\),\\qquad\\lambda\_\{\-\}<0<\\lambda\_\{\+\}\.The derivativeDΦ\(ϑs\)D\\Phi\(\\vartheta\_\{s\}\)has eigenvalue11with multiplicityd−1d\-1, together with1−ηλ\+\(ϑs\)1\-\\eta\\lambda\_\{\+\}\(\\vartheta\_\{s\}\)and1−ηλ−\(ϑs\)1\-\\eta\\lambda\_\{\-\}\(\\vartheta\_\{s\}\)\. The inequalitya2\+4b2≤a\+2b\\sqrt\{a^\{2\}\+4b^\{2\}\}\\leq a\+2b, the global step\-size bounds above, andη‖θs‖2≤3/8\\eta\\\|\\theta\_\{s\}\\\|^\{2\}\\leq 3/8give0<ηλ\+\(ϑs\)≤13/32<10<\\eta\\lambda\_\{\+\}\(\\vartheta\_\{s\}\)\\leq 13/32<1\. Hence the eigenvalues ofDΦ\(ϑs\)D\\Phi\(\\vartheta\_\{s\}\)form center, stable, and unstable groups whose dimensions ared−1d\-1,11, and11\. In particular,DΦ\(ϑs\)D\\Phi\(\\vartheta\_\{s\}\)is nonsingular\. The Inverse Function Theorem makesΦ\\PhiaC1C^\{1\}local diffeomorphism near each point of𝒦η\\mathcal\{K\}\_\{\\eta\}\.

The local discrete\-time Center\-Stable Manifold Theorem\([Shub, 1987](https://arxiv.org/html/2609.05688#bib.bib40), Theorem III\.7, pp\. 65–66\)therefore supplies an open neighborhoodUϑsU\_\{\\vartheta\_\{s\}\}and add\-dimensional embeddedC1C^\{1\}diskWloccs\(ϑs\)W^\{cs\}\_\{\\mathrm\{loc\}\}\(\\vartheta\_\{s\}\)satisfying

\{ϑ∈Uϑs:∀n∈ℤ≥0,Φn\(ϑ\)∈Uϑs\}⊆Wloccs\(ϑs\)\.\\left\\\{\\vartheta\\in U\_\{\\vartheta\_\{s\}\}:\\forall n\\in\\mathbb\{Z\}\_\{\\geq 0\},\\quad\\Phi^\{n\}\(\\vartheta\)\\in U\_\{\\vartheta\_\{s\}\}\\right\\\}\\subseteq W^\{cs\}\_\{\\mathrm\{loc\}\}\(\\vartheta\_\{s\}\)\.Each such disk is Lebesgue\-null inℝd\+1\\mathbb\{R\}^\{d\+1\}\. Compactness gives pointsϑs,1,…,ϑs,N∈𝒦η\\vartheta\_\{s,1\},\\ldots,\\vartheta\_\{s,N\}\\in\\mathcal\{K\}\_\{\\eta\}such that𝒦η⊆⋃i=1NUϑs,i\\mathcal\{K\}\_\{\\eta\}\\subseteq\\bigcup\_\{i=1\}^\{N\}U\_\{\\vartheta\_\{s,i\}\}\. WriteUi:=Uϑs,iU\_\{i\}:=U\_\{\\vartheta\_\{s,i\}\}andWics:=Wloccs\(ϑs,i\)W\_\{i\}^\{cs\}:=W^\{cs\}\_\{\\mathrm\{loc\}\}\(\\vartheta\_\{s,i\}\)\.

If an admissible trajectory converges to a point of𝒦η\\mathcal\{K\}\_\{\\eta\}, it eventually remains in someUiU\_\{i\}\. The local trapping property then places a finite iterate inWicsW\_\{i\}^\{cs\}\. Thus

𝒟η∩𝒲s\(𝒮\)⊆⋃i=1N⋃ℓ=0∞Φ−ℓ\(Wics\)\.\\mathcal\{D\}\_\{\\eta\}\\cap\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)\\subseteq\\bigcup\_\{i=1\}^\{N\}\\bigcup\_\{\\ell=0\}^\{\\infty\}\\Phi^\{\-\\ell\}\(W\_\{i\}^\{cs\}\)\.
*Null preimages\.*We next show that inverse images of null sets remain null\. The determinant ofDΦD\\Phiis real analytic\. Its value at the origin isdetDΦ\(0,0\)=1−η2C2\>0\\det D\\Phi\(0,0\)=1\-\\eta^\{2\}C^\{2\}\>0, so the determinant is not the zero function\. Its critical setCrit⁡\(Φ\):=\{ϑ:detDΦ\(ϑ\)=0\}\\operatorname\{Crit\}\(\\Phi\):=\\\{\\vartheta:\\det D\\Phi\(\\vartheta\)=0\\\}is Lebesgue\-null\([Krantz and Parks, 2002](https://arxiv.org/html/2609.05688#bib.bib41), Section 4\.1, p\. 83\)\.

LetZ⊆ℝd\+1Z\\subseteq\\mathbb\{R\}^\{d\+1\}be Lebesgue\-null\. At every point outsideCrit⁡\(Φ\)\\operatorname\{Crit\}\(\\Phi\), the Inverse Function Theorem gives a neighborhoodUUon whichΦ\\Phihas aC1C^\{1\}inverse\. We choose an open ballBBwhose compact closure lies inΦ⁡\(U\)\\Phi\(U\), and setO=\(Φ\|U\)−1\(B\)O=\(\\Phi\|\_\{U\}\)^\{\-1\}\(B\)\. The inverse derivative is bounded onB¯\\overline\{B\}\. SinceBBis convex, the inverse is Lipschitz onBB\. Second countability gives a collection\{Oj\}j≥1\\\{O\_\{j\}\\\}\_\{j\\geq 1\}of these neighborhoods that covers the regular points\. Lipschitz maps preserve null sets\([Evans and Gariepy, 2015](https://arxiv.org/html/2609.05688#bib.bib42), Sections 2\.2 and 2\.4\), so

Φ−1\(Z\)∖Crit⁡\(Φ\)⊆⋃j=1∞\(Φ\|Oj\)−1\(Z∩Φ⁡\(Oj\)\),Vold\+1\(Φ−1\(Z\)∖Crit⁡\(Φ\)\)=0\.\\Phi^\{\-1\}\(Z\)\\setminus\\operatorname\{Crit\}\(\\Phi\)\\subseteq\\bigcup\_\{j=1\}^\{\\infty\}\(\\Phi\|\_\{O\_\{j\}\}\)^\{\-1\}\\bigl\(Z\\cap\\Phi\(O\_\{j\}\)\\bigr\),\\qquad\\mathrm\{Vol\}\_\{d\+1\}\\bigl\(\\Phi^\{\-1\}\(Z\)\\setminus\\operatorname\{Crit\}\(\\Phi\)\\bigr\)=0\.The part insideCrit⁡\(Φ\)\\operatorname\{Crit\}\(\\Phi\)is also null\. Therefore

Vold\+1\(Z\)=0⟹Vold\+1\(Φ−1\(Z\)\)=0\.\\mathrm\{Vol\}\_\{d\+1\}\(Z\)=0\\quad\\Longrightarrow\\quad\\mathrm\{Vol\}\_\{d\+1\}\\bigl\(\\Phi^\{\-1\}\(Z\)\\bigr\)=0\.
Repeating the argument shows that eachΦ−ℓ\(Wics\)\\Phi^\{\-\\ell\}\(W\_\{i\}^\{cs\}\)is null\. The countable union on the right of the stable\-basin cover above is measurable and null\. Its subset𝒟η∩𝒲s\(𝒮\)\\mathcal\{D\}\_\{\\eta\}\\cap\\mathcal\{W\}^\{s\}\(\\mathcal\{S\}\)is measurable and null by completeness of Lebesgue measure\. ∎

###### D\.2Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)\.

Assume𝔼⁡\[‖𝐱‖2\]<∞\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]<\\infty\. Fixα¯t∈\(0,1\]\\bar\{\\alpha\}\_\{t\}\\in\(0,1\], a finite model pair\(θ,ν\)∈ℝd×ℝ\(\\theta,\\nu\)\\in\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}\. Letθ∗≠𝟎\\theta^\{\\ast\}\\neq\\mathbf\{0\}vary with‖θ∗‖→∞\\\|\\theta^\{\\ast\}\\\|\\to\\inftyalong a fixed ray, while the ground\-truth mixing weights and all other model and data\-generating quantities remain fixed\. Ifθ≠𝟎\\theta\\neq\\mathbf\{0\}, assume

ℙ\(\{⟨θ,𝐱⟩=0\}∪\{⟨θ∗,𝐱⟩=0\}\)=0\.\\mathbb\{P\}\\\!\\left\(\\\{\\langle\\theta,\\mathbf\{x\}\\rangle=0\\\}\\cup\\\{\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle=0\\\}\\right\)=0\.The expectations definingℒt\\mathcal\{L\}\_\{t\}andVtV\_\{t\}are taken under the corresponding varying ground truth\. Then, pointwise at the fixed model pair and fixed scale of the diffusion noise level,

lim‖θ∗‖→∞\|∇νℒt\(θ,ν\)\|=0,lim‖θ∗‖→∞Vt\(θ,ν\)=0\.\\lim\_\{\\\|\\theta^\{\\ast\}\\\|\\to\\infty\}\|\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)\|=0,\\qquad\\lim\_\{\\\|\\theta^\{\\ast\}\\\|\\to\\infty\}V\_\{t\}\(\\theta,\\nu\)=0\.

###### Proof of Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)\.

We use the established scaled projections and forward observation

μt:=α¯t⟨θ,𝐱⟩,μt∗:=α¯t⟨θ∗,𝐱⟩,yt=\(−1\)z\+1μt∗\+εt,\\mu\_\{t\}:=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\langle\\theta,\\mathbf\{x\}\\rangle,\\qquad\\mu\_\{t\}^\{\\ast\}:=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle,\\qquad y\_\{t\}=\(\-1\)^\{z\+1\}\\mu\_\{t\}^\{\\ast\}\+\\varepsilon\_\{t\},whereεt∼𝒩⁡\(0,1\)\\varepsilon\_\{t\}\\sim\\mathcal\{N\}\(0,1\)is independent of\(𝐱,z\)\(\\mathbf\{x\},z\)\. Proposition[4\.6](https://arxiv.org/html/2609.05688#S4.Thmstyle6)and Corollary[4\.5](https://arxiv.org/html/2609.05688#S4.Thmstyle5)give the score\-matching and latent\-variance identities

Vt\(θ,ν\)=12α¯t𝔼\[μt2sech2⁡\(μtyt\+ν\)\],∇νℒt\(θ,ν\)=−1α¯t𝔼\[\(μtyt\+μt2tanh⁡\(μtyt\+ν\)\)sech2⁡\(μtyt\+ν\)\]\.V\_\{t\}\(\\theta,\\nu\)=\\frac\{1\}\{2\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\[\\mu\_\{t\}^\{2\}\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\],\\qquad\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(\\theta,\\nu\)=\-\\frac\{1\}\{\\bar\{\\alpha\}\_\{t\}\}\\mathbb\{E\}\[\(\\mu\_\{t\}y\_\{t\}\+\\mu\_\{t\}^\{2\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\)\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\.
Ifθ=𝟎\\theta=\\mathbf\{0\}, thenμt=0\\mu\_\{t\}=0almost surely, so both quantities vanish for every signal norm\. Hence assumeθ≠𝟎\\theta\\neq\\mathbf\{0\}\.

For everyx∈ℝx\\in\\mathbb\{R\},

\|x\|sech2x≤2e,sech2⁡x≤1,\|tanh⁡x\|sech2x≤1\.\|x\|\\sech^\{2\}x\\leq\\frac\{2\}\{e\},\\qquad\\sech^\{2\}x\\leq 1,\\qquad\|\\tanh x\|\\sech^\{2\}x\\leq 1\.Substitutingμtyt=\(μtyt\+ν\)−ν\\mu\_\{t\}y\_\{t\}=\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\-\\nutherefore gives the ground\-truth\-independent majorants

\|\(μtyt\+μt2tanh⁡\(μtyt\+ν\)\)sech2⁡\(μtyt\+ν\)\|≤2e\+\|ν\|\+μt2\.\|\(\\mu\_\{t\}y\_\{t\}\+\\mu\_\{t\}^\{2\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\)\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\|\\leq\\frac\{2\}\{e\}\+\|\\nu\|\+\\mu\_\{t\}^\{2\}\.The variance integrand is bounded in absolute value byμt2\\mu\_\{t\}^\{2\}\. Moreover,

𝔼⁡\[μt2\]=α¯t𝔼\[⟨θ,𝐱⟩2\]≤α¯t‖θ‖2𝔼\[‖𝐱‖2\]<∞\.\\mathbb\{E\}\[\\mu\_\{t\}^\{2\}\]=\\bar\{\\alpha\}\_\{t\}\\mathbb\{E\}\[\\langle\\theta,\\mathbf\{x\}\\rangle^\{2\}\]\\leq\\bar\{\\alpha\}\_\{t\}\\\|\\theta\\\|^\{2\}\\mathbb\{E\}\[\\\|\\mathbf\{x\}\\\|^\{2\}\]<\\infty\.Thus both absolute integrands have integrable majorants independent of‖θ∗‖\\\|\\theta^\{\\ast\}\\\|\. By the projection assumption, almost surely

μtyt\+ν=\(−1\)z\+1α¯t⟨θ,𝐱⟩⟨θ∗,𝐱⟩\+α¯t⟨θ,𝐱⟩εt\+ν,\\mu\_\{t\}y\_\{t\}\+\\nu=\(\-1\)^\{z\+1\}\\bar\{\\alpha\}\_\{t\}\\langle\\theta,\\mathbf\{x\}\\rangle\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle\+\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\langle\\theta,\\mathbf\{x\}\\rangle\\varepsilon\_\{t\}\+\\nu,whose absolute value tends to infinity\. Each integrand in the exact identities therefore tends to zero almost surely\. The dominated convergence theorem, followed by division by the fixed positiveα¯t\\bar\{\\alpha\}\_\{t\}, proves both limits\. ∎

###### D\.3Hard\-Assignment Limit of the EM Imbalance Operator\.

Under the assumptions of Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2), supposeθ≠𝟎\\theta\\neq\\mathbf\{0\}, and defineq⁡\(θ,θ∗\):=𝔼𝐱\[sgn\(⟨θ,𝐱⟩⟨θ∗,𝐱⟩\)\]q\(\\theta,\\theta^\{\\ast\}\):=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\[\\textnormal\{sgn\}\(\\langle\\theta,\\mathbf\{x\}\\rangle\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle\)\], which is constant along the fixed ray\. LetNθ∗N\_\{\\theta^\{\\ast\}\}andℋθ∗\\mathcal\{H\}\_\{\\theta^\{\\ast\}\}denote the EM imbalance operator and cross\-entropy when their ground\-truth expectations are taken under\(θ∗,ν∗\)\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)\. Then, as‖θ∗‖→∞\\\|\\theta^\{\\ast\}\\\|\\to\\inftyalong the fixed ray,

Nθ∗\(θt,ν\)⟶q\(θ,θ∗\)tanhν∗,∂νNθ∗\(θt,ν\)⟶0,N\_\{\\theta^\{\\ast\}\}\(\\theta\_\{t\},\\nu\)\\longrightarrow q\(\\theta,\\theta^\{\\ast\}\)\\tanh\\nu^\{\\ast\},\\qquad\\partial\_\{\\nu\}N\_\{\\theta^\{\\ast\}\}\(\\theta\_\{t\},\\nu\)\\longrightarrow 0,and

∇νℋθ∗\(θt,ν\)⟶tanhν−q\(θ,θ∗\)tanhν∗,∇ν2ℋθ∗\(θt,ν\)⟶sech2ν\>0\.\\nabla\_\{\\nu\}\\mathcal\{H\}\_\{\\theta^\{\\ast\}\}\(\\theta\_\{t\},\\nu\)\\longrightarrow\\tanh\\nu\-q\(\\theta,\\theta^\{\\ast\}\)\\tanh\\nu^\{\\ast\},\\qquad\\nabla\_\{\\nu\}^\{2\}\\mathcal\{H\}\_\{\\theta^\{\\ast\}\}\(\\theta\_\{t\},\\nu\)\\longrightarrow\\sech^\{2\}\\nu\>0\.If𝐱∼𝒩⁡\(𝟎,Id\)\\mathbf\{x\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},I\_\{d\}\)andρ:=⟨θ,θ∗⟩/\(‖θ‖‖θ∗‖\)\\rho:=\\langle\\theta,\\theta^\{\\ast\}\\rangle/\(\\\|\\theta\\\|\\\|\\theta^\{\\ast\}\\\|\), thenq⁡\(θ,θ∗\)=\(2/π\)arcsin⁡ρq\(\\theta,\\theta^\{\\ast\}\)=\(2/\\pi\)\\arcsin\\rho\.

###### Proof\.

Under the ground truth\(θ∗,ν∗\)\(\\theta^\{\\ast\},\\nu^\{\\ast\}\), the candidate activation isμtyt=\(−1\)z\+1α¯t⟨θ,𝐱⟩⟨θ∗,𝐱⟩\+α¯t⟨θ,𝐱⟩εt\\mu\_\{t\}y\_\{t\}=\(\-1\)^\{z\+1\}\\bar\{\\alpha\}\_\{t\}\\langle\\theta,\\mathbf\{x\}\\rangle\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle\+\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\langle\\theta,\\mathbf\{x\}\\rangle\\varepsilon\_\{t\}\. The projection condition givestanh⁡\(μtyt\+ν\)→\(−1\)z\+1sgn\(⟨θ,𝐱⟩⟨θ∗,𝐱⟩\)\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\to\(\-1\)^\{z\+1\}\\textnormal\{sgn\}\(\\langle\\theta,\\mathbf\{x\}\\rangle\\langle\\theta^\{\\ast\},\\mathbf\{x\}\\rangle\)andsech2⁡\(μtyt\+ν\)→0\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\to 0almost surely\. Bounded convergence, independence ofzzand𝐱\\mathbf\{x\}, and𝔼⁡\[\(−1\)z\+1\]=tanh⁡ν∗\\mathbb\{E\}\[\(\-1\)^\{z\+1\}\]=\\tanh\\nu^\{\\ast\}then give

Nθ∗\(θt,ν\)⟶q\(θ,θ∗\)tanhν∗,∂νNθ∗\(θt,ν\)=𝔼∗\[sech2\(μtyt\+ν\)\]⟶0\.N\_\{\\theta^\{\\ast\}\}\(\\theta\_\{t\},\\nu\)\\longrightarrow q\(\\theta,\\theta^\{\\ast\}\)\\tanh\\nu^\{\\ast\},\\qquad\\partial\_\{\\nu\}N\_\{\\theta^\{\\ast\}\}\(\\theta\_\{t\},\\nu\)=\\mathbb\{E\}\_\{\\ast\}\[\\sech^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]\\longrightarrow 0\.Proposition[4\.2](https://arxiv.org/html/2609.05688#S4.Thmstyle2)and its derivative give the two cross\-entropy limits\. For isotropic Gaussian covariates, Lemma[A\.12](https://arxiv.org/html/2609.05688#A1.Thmstyle12)applied to the standardized projections givesq⁡\(θ,θ∗\)=\(2/π\)arcsin⁡ρq\(\\theta,\\theta^\{\\ast\}\)=\(2/\\pi\)\\arcsin\\rho\. ∎

### Appendix EExperimental Details and Additional Numerical Results

This appendix develops the covariance interpretation for the auxiliary local spectral diagnostic and gives the numerical settings for the figures placed alongside the results in Sections[3](https://arxiv.org/html/2609.05688#S3),[4](https://arxiv.org/html/2609.05688#S4), and[5](https://arxiv.org/html/2609.05688#S5), together with their synthesis in Section[6](https://arxiv.org/html/2609.05688#S6)\. Unless stated otherwise,𝐱∼𝒩⁡\(0,Id\)\\mathbf\{x\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)\. All computations use double precision, unit response\-noise variance, and the convention for the mixing weights in Section[2\.1](https://arxiv.org/html/2609.05688#S2.SS1)\.

Experimental notation\.To avoid a second notation system, Table[1](https://arxiv.org/html/2609.05688#A5.T1)cross\-references quantities already introduced in the theorem statements or their preceding formal appendix\. Numerical diagnostics such as fitted slopes, quantiles, and coverage are described in words rather than assigned new symbols\.

Table 1:Experimental notation\.Main\-paper quantities reused in the numerical experiments\.Numerical settings\.Tables[2](https://arxiv.org/html/2609.05688#A5.T2)and[3](https://arxiv.org/html/2609.05688#A5.T3)collect the settings without introducing experiment\-specific mathematical notation\. A separate entry lists the spectral study as an auxiliary local\-mechanism diagnostic\.

Table 2:Finite\-sample, endpoint, and blindness settings\.Table 3:Limiting\-dynamics and spectral\-diagnostic settings\.Finite\-sample estimation and calibration\.For a data point\(y0,𝐱\)\(y\_\{0\},\\mathbf\{x\}\), defineμ0=⟨θ,𝐱⟩\\mu\_\{0\}=\\langle\\theta,\\mathbf\{x\}\\rangle,μt=α¯tμ0\\mu\_\{t\}=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\mu\_\{0\}, andyt=y0α¯t\+ξ1−α¯ty\_\{t\}=y\_\{0\}\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\+\\xi\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}as in Section[2\.1](https://arxiv.org/html/2609.05688#S2.SS1)\. After terms independent of\(θ,ν\)\(\\theta,\\nu\)are removed, the implemented conditional Hyvärinen integrand is the sum of−ytμ0tanh\(μtyt\+ν\)/α¯t\-y\_\{t\}\\mu\_\{0\}\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}andμ02\{2−tanh2⁡\(μtyt\+ν\)\}/2\\mu\_\{0\}^\{2\}\\\{2\-\\tanh^\{2\}\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\\\}/2\. It is integrated overα¯t∈\[α¯Tn,α¯0\]\\bar\{\\alpha\}\_\{t\}\\in\[\\bar\{\\alpha\}\_\{T\_\{n\}\},\\bar\{\\alpha\}\_\{0\}\]and overξ∼𝒩⁡\(0,1\)\\xi\\sim\\mathcal\{N\}\(0,1\)\. Sincenα¯Tn/\(1−α¯Tn\)=n−1/2/\(1−n−3/2\)→0n\\bar\{\\alpha\}\_\{T\_\{n\}\}/\(1\-\\bar\{\\alpha\}\_\{T\_\{n\}\}\)=n^\{\-1/2\}/\(1\-n^\{\-3/2\}\)\\to 0, the selected terminal scale satisfies the terminal\-scale condition in Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)\.

Each fit uses L\-BFGS\-B withθ∈\[−3,3\]\\theta\\in\[\-3,3\],ν∈\[−2\.5,2\.5\]\\nu\\in\[\-2\.5,2\.5\], at most 140 iterations, function tolerance10−1110^\{\-11\}, and projected\-gradient tolerance2×10−72\\times 10^\{\-7\}\. Two starts are a data\-moment estimate and its joint negative\. The first coordinate of the data\-moment estimate ismax⁡\{n−1∑i\(y0\(i\)\)2−1,8×10−2\}\\sqrt\{\\max\\\{n^\{\-1\}\\sum\_\{i\}\(y\_\{0\}^\{\(i\)\}\)^\{2\}\-1,\\,8\\times 10^\{\-2\}\\\}\}, and its second coordinate is obtained by applyingarctanh\\operatorname\{arctanh\}to the ratio of\(n−1∑i𝐱\(i\)y0\(i\)\)/max⁡\{n−1∑i‖𝐱\(i\)‖2,10−10\}\(n^\{\-1\}\\sum\_\{i\}\\mathbf\{x\}^\{\(i\)\}y\_\{0\}^\{\(i\)\}\)/\\max\\\{n^\{\-1\}\\sum\_\{i\}\\\|\\mathbf\{x\}^\{\(i\)\}\\\|^\{2\},10^\{\-10\}\\\}to this first coordinate, clipped to remain8×10−28\\times 10^\{\-2\}away from\{±1\}\\\{\\pm 1\\\}by restricting it to\[−1\+8×10−2,1−8×10−2\]\[\-1\+8\\times 10^\{\-2\},\\,1\-8\\times 10^\{\-2\}\]\. The other two starts are\(0\.55,0\.15\)\(0\.55,0\.15\)and\(−0\.55,−0\.15\)\(\-0\.55,\-0\.15\)\. Among the four starts, we retain the fit with the smallest objective and then apply the joint signsns\_\{n\}to both estimated coordinates\. The aligned error is this estimate minus the ground\-truth pair, and Figure[1](https://arxiv.org/html/2609.05688#S3.F1)\(a\) uses its norm induced byI⁡\(θ∗,ν∗\)I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\), called the Fisher norm\. Theorem[3\.2](https://arxiv.org/html/2609.05688#S3.Thmstyle2)gives convergence ofnntimes its square to the two\-degree\-of\-freedom chi\-square lawχ22\\chi^\{2\}\_\{2\}\. Hence the median and 90% references are the square roots of the correspondingχ22\\chi^\{2\}\_\{2\}quantiles divided bynn\. The empirical quantile bars are 95% percentile\-bootstrap intervals from 2,000 resamples with seed 17031\. Panel \(b\) compares all ordered values ofnntimes the squared Fisher error withχ22\\chi^\{2\}\_\{2\}quantiles atn=250n=250andn=4000n=4000\.

For each sample sizenn, Table[4](https://arxiv.org/html/2609.05688#A5.T4)reports the fraction of aligned estimates for whichnntimes the squared Fisher error does not exceed the 95%χ22\\chi^\{2\}\_\{2\}quantile\. Its covariance discrepancy is the Frobenius distance between the empirical covariance of then\\sqrt\{n\}\-scaled aligned errors andI\(θ∗,ν∗\)−1I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)^\{\-1\}, divided by the Frobenius norm ofI\(θ∗,ν∗\)−1I\(\\theta^\{\\ast\},\\nu^\{\\ast\}\)^\{\-1\}\. All 1,000 optimization runs terminated without solver failure\.

Table 4:Finite\-sample calibration\.Joint 95% Fisher\-ellipse coverage and relative covariance discrepancy\.Endpoint\-gradient checks\.We use an exact sum over the latent class and 140\-point Gauss–Hermite quadrature for both the Gaussian covariate and response\-noise expectations at every scale of the diffusion noise level\. At low noise, theθ\\thetacomponent of the complete reference in Proposition[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)is the sum ofΣ⁡\(θ−M⁡\(θ,ν\)\)\\Sigma\(\\theta\-M\(\\theta,\\nu\)\),−\(∇θM\(θ,ν\)\)⊤Σθ\-\\bigl\(\\nabla\_\{\\theta\}M\(\\theta,\\nu\)\\bigr\)^\{\\top\}\\Sigma\\theta, and∇θV\(θ,ν\)\\nabla\_\{\\theta\}V\(\\theta,\\nu\)\. Theν\\nucomponent is−⟨∇θN\(θ,ν\),θ⟩\+∇νV\(θ,ν\)\-\\langle\\nabla\_\{\\theta\}N\(\\theta,\\nu\),\\theta\\rangle\+\\nabla\_\{\\nu\}V\(\\theta,\\nu\)\. The plotted remainder uses the directα¯t=1\\bar\{\\alpha\}\_\{t\}=1gradient to stabilize the subtraction at the smallest scales\. As a separate check, that endpoint differs from the complete operator expression in Proposition[4\.7](https://arxiv.org/html/2609.05688#S4.Thmstyle7)by2\.24964×10−92\.24964\\times 10^\{\-9\}in Euclidean norm\. Itsθ\\theta\- andν\\nu\-component residuals are7\.606×10−107\.606\\times 10^\{\-10\}and2\.117×10−92\.117\\times 10^\{\-9\}\. This separate identity check confirms that the endpoint reference includes the correction terms and is not the bare EM residual\. At high noise, the reference is the gradient of the effective\-product limiting loss in Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)\.

Table[5](https://arxiv.org/html/2609.05688#A5.T5)records the componentwise log–log slope deviations from the first\-order benchmark\. Each fit uses the five smallest of the nine endpoint distances\.

Table 5:Endpoint\-remainder slopes\.Entries are written relative to the unit\-slope benchmark\.On the balanced sliceν∗=ν=0\\nu^\{\\ast\}=\\nu=0, the leading high\-noise objective is flat inθ\\theta, so Proposition[4\.8](https://arxiv.org/html/2609.05688#S4.Thmstyle8)supplies no leading\-order direction\. The𝒪⁡\(α¯t\)\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\)remainder test of that proposition is Figure[2](https://arxiv.org/html/2609.05688#S4.F2)\(b\), not Figure[5](https://arxiv.org/html/2609.05688#A5.F5)\. The latter is only an auxiliary next\-order directional comparison at small positiveα¯t\\bar\{\\alpha\}\_\{t\}\. Usingnncovariate draws\{𝐱\(i\)\}i=1n\\\{\\mathbf\{x\}^\{\(i\)\}\\\}\_\{i=1\}^\{n\}withΣ=Id\\Sigma=I\_\{d\}, form the sample spiked covariance

𝒞^∗:=1n∑i=1n𝐱\(i\)\(𝐱\(i\)\)⊤\(\(𝐱\(i\)\)⊤θ∗\)2\\widehat\{\\mathcal\{C\}\}^\{\\ast\}:=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\mathbf\{x\}^\{\(i\)\}\(\\mathbf\{x\}^\{\(i\)\}\)^\{\\top\}\\bigl\(\(\\mathbf\{x\}^\{\(i\)\}\)^\{\\top\}\\theta^\{\\ast\}\\bigr\)^\{2\}and the normalized power step

θ⟼\(Id\+2γ𝒞^∗\)θ‖\(Id\+2γ𝒞^∗\)θ‖,γ:=ηθα¯t\.\\theta\\;\\longmapsto\\;\\frac\{\(I\_\{d\}\+2\\gamma\\widehat\{\\mathcal\{C\}\}^\{\\ast\}\)\\theta\}\{\\\|\(I\_\{d\}\+2\\gamma\\widehat\{\\mathcal\{C\}\}^\{\\ast\}\)\\theta\\\|\},\\qquad\\gamma:=\\eta\_\{\\theta\}\\bar\{\\alpha\}\_\{t\}\.In parallel, withν\\nuclamped at zero, apply the finite\-noise score\-matching updateθ←θ−ηθ∇θℒt\(θ,0\)\\theta\\leftarrow\\theta\-\\eta\_\{\\theta\}\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,0\)and renormalize to a fixed radius\. The two trajectories use respective normalization radii11and2×10−22\\times 10^\{\-2\}, which do not affect the absolute cosine\|cos⁡∠⁡\(θk,θ∗\)\|\|\\cos\\angle\(\\theta\_\{k\},\\theta^\{\\ast\}\)\|plotted in Figure[5](https://arxiv.org/html/2609.05688#A5.F5)\. The labelt10t\_\{10\}denotes spherical Student\-ttcovariates with ten degrees of freedom \(covariance\-normalized\), contrasted with isotropic Gaussian covariates\. The experiment compares directions only: it asserts neither equality of the unnormalized updates nor that the comparison follows from the proposition’s𝒪⁡\(α¯t\)\\mathcal\{O\}\(\\bar\{\\alpha\}\_\{t\}\)bound\. IsotropicΣ=Id\\Sigma=I\_\{d\}gives the power step an ordinary\-eigenvector interpretation; anisotropic covariance would require the corresponding generalized\-eigenvector map\.

![Refer to caption](https://arxiv.org/html/2609.05688v1/figures/power_iteration.png)Figure 5:Auxiliary next\-order directional comparison on the balanced sliceν∗=ν=0\\nu^\{\\ast\}=\\nu=0\.Absolute cosine\|cos⁡∠⁡\(θk,θ∗\)\|\|\\cos\\angle\(\\theta\_\{k\},\\theta^\{\\ast\}\)\|for normalized finite\-noise score\-matching updatesθ←θ−ηθ∇θℒt\(θ,0\)\\theta\\leftarrow\\theta\-\\eta\_\{\\theta\}\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{t\}\(\\theta,0\)\(solid\) and normalized power stepsθ↦\(Id\+2γ𝒞^∗\)θ/‖\(Id\+2γ𝒞^∗\)θ‖\\theta\\mapsto\(I\_\{d\}\+2\\gamma\\widehat\{\\mathcal\{C\}\}^\{\\ast\}\)\\theta/\\\|\(I\_\{d\}\+2\\gamma\\widehat\{\\mathcal\{C\}\}^\{\\ast\}\)\\theta\\\|withγ=ηθα¯t\\gamma=\\eta\_\{\\theta\}\\bar\{\\alpha\}\_\{t\}\(dotted\)\. Curves labeledt10t\_\{10\}use spherical Student\-ttcovariates with ten degrees of freedom; the remaining curves use isotropic Gaussians\.Limiting\-dynamics diagnostics\.The implementation iteratesϑk\+1=Φ⁡\(ϑk\)\\vartheta\_\{k\+1\}=\\Phi\(\\vartheta\_\{k\}\)\. For the degenerate target,ℒ0=12sinh2⁡\(0\.70\)tanh2⁡\(0\.70\)≈0\.1050944\\mathcal\{L\}\_\{0\}=\\tfrac\{1\}\{2\}\\sinh^\{2\}\(0\.70\)\\tanh^\{2\}\(0\.70\)\\approx 0\.1050944,J⁡\(ϑ0\)=0J\(\\vartheta\_\{0\}\)=0, andB0=4B\_\{0\}=4\. Sinceη=1/50\\eta=1/50, the left\-hand side in the definition of𝒟η\\mathcal\{D\}\_\{\\eta\}is32η=16/25=0\.64<132\\eta=16/25=0\.64<1\. The reference curve in Figure[3](https://arxiv.org/html/2609.05688#S5.F3)\(a\) is the theorem’s bound\(ℒ0−1/2\+ηK/\(6B0\)\)−2\\bigl\(\\mathcal\{L\}\_\{0\}^\{\-1/2\}\+\\eta K/\(\\sqrt\{6\}\\,B\_\{0\}\)\\bigr\)^\{\-2\}\. We fit a linear regression oflog⁡ℒ⁡\(ϑK\)\\log\\mathcal\{L\}\(\\vartheta\_\{K\}\)onlog⁡K\\log Kfor every integerKKfrom3×1033\\times 10^\{3\}through3×1043\\times 10^\{4\}\. The fitted slope is−2−1\.5035×10−2\-2\-1\.5035\\times 10^\{\-2\}\. It is1\.5035×10−21\.5035\\times 10^\{\-2\}steeper than theK−2K^\{\-2\}reference\. The coefficient of determination is1−4\.6533×10−71\-4\.6533\\times 10^\{\-7\}\. This fit is a finite\-window diagnostic\. The pointwise comparison with the explicit bound checks the theorem’s𝒪⁡\(K−2\)\\mathcal\{O\}\(K^\{\-2\}\)upper bound\.

Forc∗≠𝟎c^\{\\ast\}\\neq\\mathbf\{0\}, the three displayed generic starts in Table[6](https://arxiv.org/html/2609.05688#A5.T6)are selected to show different entrance times while remaining in𝒟η\\mathcal\{D\}\_\{\\eta\}\. The exact saddle start satisfiesν0=0\\nu\_\{0\}=0and⟨θ0,c∗⟩=0\\langle\\theta\_\{0\},c^\{\\ast\}\\rangle=0and verifies the predicted plateau‖c∗‖2/2=0\.32\\\|c^\{\\ast\}\\\|^\{2\}/2=0\.32\. Across the one degenerate and four nondegenerate paths, the maximum admissibility left\-hand side is0\.640\.64and the largest one\-step loss increase is zero \(at the stationary saddle\), consistent along these trajectories with the theorem’s forward\-invariance and monotonicity conclusions\.

Table 6:Nondegenerate\-dynamics initializations\.For each displayed generic start, the analysis fits a linear regression oflog⁡ℒ⁡\(ϑK\)\\log\\mathcal\{L\}\(\\vartheta\_\{K\}\)onKKwhile the loss lies between10−310^\{\-3\}and10−1210^\{\-12\}\. HereK10−2K\_\{10^\{\-2\}\}, for example, is the first iterationKKwithℒ⁡\(ϑK\)≤10−2\\mathcal\{L\}\(\\vartheta\_\{K\}\)\\leq 10^\{\-2\}\. Table[7](https://arxiv.org/html/2609.05688#A5.T7)reports the four hitting times, the fitted slope, and one minus its coefficient of determination\.

Table 7:Hitting times and geometric tail fits\.We omit values below10−1410^\{\-14\}from the semilogarithmic panel as a roundoff display cutoff\.

Pointwise high\-SNR blindness\.The experiment in Figure[4](https://arxiv.org/html/2609.05688#S5.F4)uses the scalar covariatex∼𝒩⁡\(0,1\)x\\sim\\mathcal\{N\}\(0,1\)and a fixed positive ground\-truth direction\. This numerical derivation uses the boundary\-layer coordinateu=‖θ∗‖xu=\\sqrt\{\\\|\\theta^\{\\ast\}\\\|\}\\,x\. The diffused ground\-truth signal norm isα¯t‖θ∗‖\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\\|\\theta^\{\\ast\}\\\|\. Becauseα¯t\\bar\{\\alpha\}\_\{t\}is fixed in each sweep covered by Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2), it diverges as‖θ∗‖→∞\\\|\\theta^\{\\ast\}\\\|\\to\\infty\. With the established latent classz∈\{1,2\}z\\in\\\{1,2\\\}andεt∼𝒩⁡\(0,1\)\\varepsilon\_\{t\}\\sim\\mathcal\{N\}\(0,1\), the candidate activation is the sum of\(−1\)z\+1α¯tθu2\(\-1\)^\{z\+1\}\\bar\{\\alpha\}\_\{t\}\\theta u^\{2\},α¯tθuεt/‖θ∗‖\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\theta u\\varepsilon\_\{t\}/\\sqrt\{\\\|\\theta^\{\\ast\}\\\|\}, andν\\nu, so its nonsaturated region has fixed width inuu\. We evaluate∇νℒt\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}andVtV\_\{t\}by quadrature, multiplying the former by‖θ∗‖1/2\\\|\\theta^\{\\ast\}\\\|^\{1/2\}and the latter by‖θ∗‖3/2\\\|\\theta^\{\\ast\}\\\|^\{3/2\}before integrating over\(−∞,0\]∪\[0,∞\)\(\-\\infty,0\]\\cup\[0,\\infty\), and undo the scaling afterward\. This prevents the adaptive quadrature routine from having to resolve a shrinking neighborhood ofx=0x=0and keeps the reported components comparable in magnitude\. These factors are numerical preconditioners tailored to this one\-dimensional Gaussian design and are undone after integration\. The same adaptive call also evaluates two diagnostic components\. The first is a second copy of the preconditioned score matching gradient obtained by direct differentiation\. The second is the scaled boundary correction used for the cross\-entropy gradient\.

We use an exact sum over the two latent classes\. We evaluate the response\-noise expectation with paired 80\- and 160\-node Gauss–Hermite rules\. We compare their results without conflating this check with the adaptive integration inuu\. The latter uses vector quadrature with absolute tolerance10−1110^\{\-11\}, relative tolerance10−910^\{\-9\}, and subdivision limit 500\. Its returned error estimate covers the adaptiveuu\-integration, not Gauss–Hermite discretization\. A production run is rejected if any reported scaled component has paired\-order difference exceeding10−8\+10−610^\{\-8\}\+10^\{\-6\}times the larger absolute result from the 80\- and 160\-node rules\. At the publication settings, the direct\-score formula, which is derived without the integration\-by\-parts identity, differs from that identity by at most2\.85×10−102\.85\\times 10^\{\-10\}in absolute value over the displayed sweep\. The largest 80\-versus\-160\-node difference in the score gradient obtained by direct differentiation is also2\.85×10−102\.85\\times 10^\{\-10\}\. The largest adaptive error estimate for the scaled five\-component vector is1\.60×10−101\.60\\times 10^\{\-10\}\. We use the direct formula for the plotted score gradient because it preserves exact\-model cancellation, and retain the integration\-by\-parts formula as an independent identity check\. We take absolute values after integrating the signed quantities\.

Panels \(a\)–\(b\) of Figure[4](https://arxiv.org/html/2609.05688#S5.F4)use 1,200 common\-random\-number draws at‖θ∗‖=3\\\|\\theta^\{\\ast\}\\\|=3and‖θ∗‖=100\\\|\\theta^\{\\ast\}\\\|=100\. Their background is the fixed candidate’s response\-score sensitivity\|∂νsθt,ν\(yt,x\)\|\|\\partial\_\{\\nu\}s\_\{\\theta\_\{t\},\\nu\}\(y\_\{t\},x\)\|on a common logarithmic scale\. Panel \(c\) contains the imbalance gradient and posterior\-variance term stated in Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)\. At the largest displayed‖θ∗‖\\\|\\theta^\{\\ast\}\\\|, the computed values of\|∇νℒt\|\|\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\|andVtV\_\{t\}are1\.6688×10−31\.6688\\times 10^\{\-3\}and2\.9399×10−92\.9399\\times 10^\{\-9\}\.

The cross\-entropy control appears in Appendix Figure[6](https://arxiv.org/html/2609.05688#A5.F6)\(c\)\. It evaluates∇νℋ\(θt,ν\)=tanh⁡ν−𝔼∗\[tanh⁡\(μtyt\+ν\)\]\\nabla\_\{\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)=\\tanh\\nu\-\\mathbb\{E\}\_\{\\ast\}\[\\tanh\(\\mu\_\{t\}y\_\{t\}\+\\nu\)\]along the same fixed\-candidate ray\. For the displayed aligned one\-dimensional specialization, the candidate activation is the sum of\(−1\)z\+1α¯tθ‖θ∗‖x2\(\-1\)^\{z\+1\}\\bar\{\\alpha\}\_\{t\}\\theta\\\|\\theta^\{\\ast\}\\\|x^\{2\},α¯tθxεt\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\theta x\\varepsilon\_\{t\}, andν\\nu\. Its hyperbolic tangent therefore converges almost surely to\(−1\)z\+1\(\-1\)^\{z\+1\}, and bounded convergence gives\|∇νℋ\(θt,ν\)\|→\|tanh⁡\(−0\.35\)−tanh⁡\(0\.55\)\|≈0\.836896\|\\nabla\_\{\\nu\}\\mathcal\{H\}\(\\theta\_\{t\},\\nu\)\|\\to\|\\tanh\(\-0\.35\)\-\\tanh\(0\.55\)\|\\approx 0\.836896\. Its nonzero limit therefore agrees with Lemma[D\.3](https://arxiv.org/html/2609.05688#A4.Thmstyle3)and provides a contrast\. The score matching diagnostics lose their local imbalance signal, whereas this cross\-entropy gradient remains informative\. This is a fixed\-candidate comparison of population gradients, not a comparison of optimized estimators\.

Figure[6](https://arxiv.org/html/2609.05688#A5.F6)\(a\)–\(b\) provides two scope checks\. In the fixed\-scale comparisons, each value ofα¯t\\bar\{\\alpha\}\_\{t\}is held constant while‖θ∗‖\\\|\\theta^\{\\ast\}\\\|varies, so each curve is a separate specialization covered by Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)\. The moving\-candidate comparison instead setsθ=0\.8/‖θ∗‖\\theta=0\.8/\\\|\\theta^\{\\ast\}\\\|while retaining the main experiment’s scale of the diffusion noise level, ground\-truth ray, and imbalance parameters\. Because the candidate changes with‖θ∗‖\\\|\\theta^\{\\ast\}\\\|, this curve illustrates the proposition’s pointwise\-in\-candidate scope\.

Figure 6:Scope and objective controls for pointwise high\-SNR blindness\.\(a\)Fixed\-α¯t\\bar\{\\alpha\}\_\{t\}curves provide separate checks covered by the proposition as the true signal‖θ∗‖\\\|\\theta^\{\\ast\}\\\|grows, forα¯t∈\{10−1,12,1\}\\bar\{\\alpha\}\_\{t\}\\in\\\{10^\{\-1\},\\tfrac\{1\}\{2\},1\\\}\.\(b\)The moving\-candidate curve usesθ=0\.8/‖θ∗‖\\theta=0\.8/\\\|\\theta^\{\\ast\}\\\|, for which the candidate activation need not diverge and the moving\-candidate score matching gradient\|∇νℒt\(0\.8/‖θ∗‖,ν\)\|\|\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(0\.8/\\\|\\theta^\{\\ast\}\\\|,\\nu\)\|approaches0\.3023020\.302302\. That curve is outside the fixed\-candidate hypothesis of Proposition[5\.2](https://arxiv.org/html/2609.05688#S5.Thmstyle2)and illustrates its pointwise\-in\-candidate scope\.\(c\)The separate population cross\-entropy control is evaluated along the same fixed\-candidate ray as Figure[4](https://arxiv.org/html/2609.05688#S5.F4)\. It approaches\|tanh⁡ν−tanh⁡ν∗\|≈0\.836896\|\\tanh\\nu\-\\tanh\\nu^\{\\ast\}\|\\approx 0\.836896, showing that this selected cross\-entropy diagnostic retains imbalance signal\.Along the moving candidate used in the control, differentiation and dominated convergence give∇νℒt\(0\.8/‖θ∗‖,ν\)→−0\.8𝔼∗\[\(−1\)z\+1x2sech2⁡\(0\.8α¯t\(−1\)z\+1x2\+ν\)\]\\nabla\_\{\\nu\}\\mathcal\{L\}\_\{t\}\(0\.8/\\\|\\theta^\{\\ast\}\\\|,\\nu\)\\to\-0\.8\\,\\mathbb\{E\}\_\{\\ast\}\[\(\-1\)^\{z\+1\}x^\{2\}\\sech^\{2\}\(0\.8\\bar\{\\alpha\}\_\{t\}\(\-1\)^\{z\+1\}x^\{2\}\+\\nu\)\]\. A separate integration gives the signed limiting value−0\.3023024610\-0\.3023024610to ten decimal places, with magnitude0\.30230246100\.3023024610\. The plotted absolute gradient at‖θ∗‖=3×105\\\|\\theta^\{\\ast\}\\\|=3\\times 10^\{5\}differs from that magnitude by3\.2×10−123\.2\\times 10^\{\-12\}\. This is the discrepancy at the largest displayed signal norm\. The largest 80\-versus\-160\-node difference over the control sweep is8\.72×10−88\.72\\times 10^\{\-8\}\.
Connecting Score Matching, Maximum Likelihood, and Expectation-Maximization in Mixed Linear Regression

Similar Articles

High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

Enhancing Numerical Prediction in LLMs via Smooth MMD Alignment

DiffScore: Text Evaluation Beyond Autoregressive Likelihood

A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models

Diffusion models recover accurate mixture weights despite score function insensitivity

Submit Feedback

Similar Articles

High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence
Enhancing Numerical Prediction in LLMs via Smooth MMD Alignment
DiffScore: Text Evaluation Beyond Autoregressive Likelihood
A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
Diffusion models recover accurate mixture weights despite score function insensitivity