Barron最优传输（一）：生成建模

# Barron Optimal Transport I: Generative Modeling
Source: [https://arxiv.org/html/2610.10875](https://arxiv.org/html/2610.10875)
Evan Dogariu and Joan BrunaAffiliation:Courant Institute School of Mathematics, Computing, and Data Science New York University

###### Abstract

Motivated by recent applications in generative modeling and sampling, we introduce a framework for optimal measure transport where cost captures the notion of neural network complexity\. In transport\-based generative models, samples from a reference distribution \(e\.g\. Gaussian\) are mapped to samples of a target distribution along ordinary or stochastic differential equations\. These are implemented as deep residual networks when discretized in time, where each hidden layer approximates the associated instantaneous velocity\. Thus, given a pair of target and reference measures, a natural question is to search for the most efficient neural representation that implements this transport\.

Our starting point is the kinetic formulation of OT, due to Benamou and Brenier\. We replace the average kineticL2L^\{2\}energy by the*Barron*energy\[[7](https://arxiv.org/html/2610.10875#bib.bib46),[31](https://arxiv.org/html/2610.10875#bib.bib60)\], a natural norm which measures the complexity of representing a given vector field with a neural hidden layer, and which captures the adaptive properties of feature learning\. This defines a metric on the space of probability measures, complementing existing Wasserstein and Stein geometries\.

In this work we examine the properties of this metric in the context of generative modeling\. As a first application, we quantify the suboptimality of diffusion generative modeling in the Barron geometry by establishing super\-polynomial score approximation lower bounds for data generated by neural network pushforwards of the Gaussian\. We then investigate the benefit of adaptivity as a way to study alternative generative models\. In a companion paper\[[28](https://arxiv.org/html/2610.10875#bib.bib1)\]we leverage the Barron transport geometry for sampling applications, extending the scope of Stein variational gradient methods via feature adaptation\.

## Contents

## 1Introduction

Generative modeling and sampling have become central problems in modern machine learning and scientific computing\. Generative modeling – the task of using a finite dataset to learn to generate new samples from the same underlying data distribution – describes methods ranging from generative adversarial networks and diffusion/flow\-matching models to autoregressive large language models \(LLMs\)\[[39](https://arxiv.org/html/2610.10875#bib.bib59),[72](https://arxiv.org/html/2610.10875#bib.bib58),[11](https://arxiv.org/html/2610.10875#bib.bib57)\]\. Sampling – the task of using structural information about the target distribution \(such as the unnormalized density or its score\) to generate samples – is equally important in the computational sciences: for example, it is used to explore protein conformations and molecular free\-energy landscapes\[[70](https://arxiv.org/html/2610.10875#bib.bib56),[62](https://arxiv.org/html/2610.10875#bib.bib55)\], to compute equilibrium properties of many\-body systems\[[60](https://arxiv.org/html/2610.10875#bib.bib47)\], and to simulate lattice field theories\[[29](https://arxiv.org/html/2610.10875#bib.bib54)\]\.

Currently, the prevalent approaches in both tasks are rooted in a dynamical perspective, in which one starts with samples drawn from a simple base distributionμ\\mu\(such as the standard Gaussian for continuous data or the empty string for language\) and iteratively transports them during the sampling process toward a target distributionπ\\pi; this is the case for flow\-matching models\[[54](https://arxiv.org/html/2610.10875#bib.bib40),[56](https://arxiv.org/html/2610.10875#bib.bib48),[2](https://arxiv.org/html/2610.10875#bib.bib82)\]and diffusion models\[[43](https://arxiv.org/html/2610.10875#bib.bib52),[75](https://arxiv.org/html/2610.10875#bib.bib51)\], LLMs, and most sampling algorithms\. If we restrict to continuous\-time, then heuristically this dynamical transport from the base distribution to the target distribution is governed by the continuity equation111The dynamical transport perspective can also capture stochastic transports such as diffusion models, where \([1](https://arxiv.org/html/2610.10875#S1.E1)\) is replaced by a Fokker\-Planck equation and \([2](https://arxiv.org/html/2610.10875#S1.E2)\) is replaced by a stochastic differential equation \(SDE\)\. Under enough regularity, this can also be achieved by adding∇log⁡ρt\\nabla\\log\\rho\_\{t\}tovtv\_\{t\}\. Our results in this paper are in the ODE setting, but the high\-level ideas and many specific results transfer to the SDE setting\.

∂tρt\+∇⋅\(ρt​vt\)=0,\\partial\_\{t\}\\rho\_\{t\}\+\\nabla\\cdot\(\\rho\_\{t\}v\_\{t\}\)=0,\(1\)whereρ0=μ\\rho\_\{0\}=\\muis the base distribution,ρ1=π\\rho\_\{1\}=\\piis the target distribution,\(ρt\)t∈\(0,1\)\(\\rho\_\{t\}\)\_\{t\\in\(0,1\)\}describe the intermediate distributions along the sampling dynamics, andvt:ℝd→ℝdv\_\{t\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}is the instantaneous velocity field\. Under appropriate regularity, these dynamics may be implemented by the differential equation

X˙t=vt​\(Xt\),ρt=Law⁡\(Xt\),X0∼μ\.\\dot\{X\}\_\{t\}=v\_\{t\}\(X\_\{t\}\),\\qquad\\rho\_\{t\}=\\operatorname\{Law\}\(X\_\{t\}\),\\qquad X\_\{0\}\\sim\\mu\.\(2\)From this dynamical perspective, both generative modeling and sampling can be viewed as the task of designing an appropriate transport, parametrized by\(vt\)t∈\[0,1\]\(v\_\{t\}\)\_\{t\\in\[0,1\]\}, using available samples fromπ\\piin the former and structural information on its density in the latter\.

A key question is not merely whether one can transportμ\\mutoπ\\pidynamically, but whether one can do so using vector fields that are simple to approximate in a chosen model class\. For example, one may wish to restrictvtv\_\{t\}to neural networks of bounded size or to control its norm in a fixed reproducing kernel Hilbert space \(RKHS\), in a smoothness/Sobolev class, or in some other structured family of functions\. In order to quantify this, we may wish to penalizevtv\_\{t\}using some prescribed Banach norm‖vt‖ℬ\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}\. Minimizing the accumulated cost

∫01‖vt‖ℬ​𝑑t\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt\(3\)over all transports fromμ\\mutoπ\\piwhich satisfy \([1](https://arxiv.org/html/2610.10875#S1.E1)\) then measures the smallest complexity required to generateπ\\piwithin this model class, which informally defines an “optimal transport” relative to the complexity measureℬ\\mathcal\{B\}\.

In modern generative modeling, it has become standard to represent the driftsvtv\_\{t\}using deep residual neural networks, where each layer can be seen as an Euler discretization of an underlying neural ODE\[[17](https://arxiv.org/html/2610.10875#bib.bib38),[23](https://arxiv.org/html/2610.10875#bib.bib43)\]\. We can thus view the forward pass of each drift networkvtv\_\{t\}as a sequence of updates to the sampleXtX\_\{t\}, in which case the individual layers act as drifts in their own right\. Phrased differently, a discretization of the dynamics \([2](https://arxiv.org/html/2610.10875#S1.E2)\) where eachvtv\_\{t\}is represented by a deep residual network yields nothing more than a single very deep residual network pushingμ\\mutoπ\\pi\. This, in turn, is nothing more than a discretization of dynamics where each drift is a single layer of a neural network\. Hence, it is without loss of generality to consider dynamical transports like \([1](https://arxiv.org/html/2610.10875#S1.E1)\) and \([2](https://arxiv.org/html/2610.10875#S1.E2)\) for which eachvtv\_\{t\}is a shallow neural network\. Such neural transports operate on two timescales: samples evolve over longer time horizons according to the target dynamics, while microscopically they evolve according to the layers of learned drift networks\. In order to study the neural network size required to transportμ\\mutoπ\\pidynamically, it is therefore natural to consider \([3](https://arxiv.org/html/2610.10875#S1.E3)\) with a costℬ\\mathcal\{B\}which quantifies the size of a shallow neural network representation ofvtv\_\{t\}\.

For that purpose we instantiateℬ\\mathcal\{B\}as a*Barron*spaceℬϕ\\mathcal\{B\}\_\{\\phi\}\[[8](https://arxiv.org/html/2610.10875#bib.bib29),[7](https://arxiv.org/html/2610.10875#bib.bib46),[27](https://arxiv.org/html/2610.10875#bib.bib31)\], roughly defined as the convex hull of ridge functions of the formx↦ϕ⁡\(⟨x,θ⟩\)x\\mapsto\\phi\(\\langle x,\\theta\\rangle\)for an activationϕ\\phi\(see Section[3\.2](https://arxiv.org/html/2610.10875#S3.SS2)\)\. This space models the feature learning of neural networks by representing functions as adaptively\-chosen mixtures of these ridge functions \(i\.e\. shallow neural networks\), and theℬϕ\\mathcal\{B\}\_\{\\phi\}norm of a function describes the size of its smallest shallow neural network representation\. These properties makeℬ=ℬϕ\\mathcal\{B\}=\\mathcal\{B\}\_\{\\phi\}a natural choice to study what happens when transporting measures by deep networks, or equivalently by dynamics with shallow drifts\.

Minimizing the accumulated cost \([3](https://arxiv.org/html/2610.10875#S1.E3)\) over all transports fromμ\\mutoπ\\pisatisfying the continuity equation \([1](https://arxiv.org/html/2610.10875#S1.E1)\) yields a metricDℬϕ​\(μ,π\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\pi\), which we call theBarron transport metric\. In words,Dℬϕ​\(μ,π\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\pi\)captures the size of the most efficient neural network transport fromμ\\mutoπ\\pi, i\.e\. the smallest generative model forπ\\pi\. The formulation ofDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}as a dynamical optimal transport problem allows us to deploy tools from optimal transport and control theory for this analysis\. This will play only a minor role in the results of this paper, but in the companion paper\[[28](https://arxiv.org/html/2610.10875#bib.bib1)\]we will study the optimal transport geometry of the metric space\(𝒫⁡\(ℝd\),Dℬϕ\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{B\}\_\{\\phi\}\}\)in detail, which will help us construct and analyze new sampling algorithms \(see Section[5\.3](https://arxiv.org/html/2610.10875#S5.SS3)for a preview\)\.

### 1\.1Main results

Using the language of the Barron transport metric we can now state our main technical results, which are grouped into two categories\. We first study the commonly\-used denoising generative model, which defines a path\(ρtπ\)t∈\[0,1\]\(\\rho\_\{t\}^\{\\pi\}\)\_\{t\\in\[0,1\]\}fromγd\\gamma\_\{d\}toπ\\pias the marginal law ofXtX\_\{t\}, the time\-reversal of the Ornstein\-Uhlenbeck process with initial lawπ\\pi\. Our results show that depth is needed to model the associated drifts using neural networks – in other words, when we choose the denoising path as the macroscopic behavior of the transport we require alternative dynamics on a nontrivial microscopic timescale\. Afterward, we take a first step toward understanding such alternative behaviors by studying the role of feature learning in the broader class of dynamical generative models\.

##### Denoising transport suboptimality\.

Our first technical result asks whether the denoising transport fromγd\\gamma\_\{d\}to a target distributionπ\\piis efficient in terms of shallow neural network representation\. More precisely, we ask whether the denoising222These denoising drifts are also the time\-dependent scores of the marginals of the forward OU noising process \(up to affine transformations\), see Section[2\.3](https://arxiv.org/html/2610.10875#S2.SS3)and equation \([27](https://arxiv.org/html/2610.10875#S2.E27)\)\.drifts\(vtπ\)t∈\[0,1\]\(v\_\{t\}^\{\\pi\}\)\_\{t\\in\[0,1\]\}can be efficiently represented or approximated by single\-layer neural networks, and whether there exist other generative models forπ\\piwhose drifts admit smaller two\-layer representations\. This is equivalent to asking whetherDℬϕ​\(γd,π\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)is much smaller than∫01‖v^t‖ℬϕ​𝑑t\\int\_\{0\}^\{1\}\\\|\\hat\{v\}\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt, wherev^t\\hat\{v\}\_\{t\}is a suitable approximation ofvtπ\{v\}\_\{t\}^\{\\pi\}\. We show that the denoising transport can be very suboptimal in this regard:

###### Theorem 1\(Denoising suboptimality, informal\)\.

Ifπ\\piis the pushforward ofγd\\gamma\_\{d\}by a ResNet of depthΘ⁡\(d\)\\Theta\(d\)and widthO⁡\(1\)O\(1\)with bounded weights and two\-layer blocks using activationϕ\\phi, thenDℬϕ​\(γd,π\)≲dD\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\\lesssim d\. However, the following hold:

- •\(Theorem[3](https://arxiv.org/html/2610.10875#Thmtheorem3)\)\.There exists such a ResNet \(with someϕ\\phi\) for which∫01‖vtπ‖ℬϕ​𝑑t≳2d\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{\\pi\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt\\gtrsim 2^\{d\}, even though we have‖vtπ‖ℬϕ<∞\\\|v\_\{t\}^\{\\pi\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}<\\inftyfor alltt\.
- •\(Corollary[1](https://arxiv.org/html/2610.10875#Thmcorollary1)\)\.There exists such a ResNet \(withϕ=ReLU\\phi=\\operatorname\{ReLU\}\)333Our construction provides similar results for other activations\. For example, ifϕ=tanh\\phi=\\tanhthen we getdΩ⁡\(log⁡\(1/ε\)2\)d^\{\\Omega\(\\log\(1/\\varepsilon\)^\{2\}\)\}lower bounds, see Remark[8](https://arxiv.org/html/2610.10875#Thmremark8)\.such that foranyapproximate driftsv^t\\hat\{v\}\_\{t\}there is the implication ∫01∥v^t\(x\)−vtπ\(x\)∥2dρtπ\(x\)dt≤ε2⟹∫01∥v^t∥ℬϕdt≳dΩ\(ε−4/5\),\\int\_\{0\}^\{1\}\\\|\\hat\{v\}\_\{t\}\(x\)\-v\_\{t\}^\{\\pi\}\(x\)\\\|^\{2\}d\\rho\_\{t\}^\{\\pi\}\(x\)dt\\leq\\varepsilon^\{2\}\\implies\\int\_\{0\}^\{1\}\\\|\\hat\{v\}\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt\\gtrsim d^\{\\Omega\(\\varepsilon^\{\-4/5\}\)\}~,\(4\)and moreover if eachv^t\\hat\{v\}\_\{t\}is a two\-layer neural network then the largest width is at leastdΩ\(ε−4/5\)d^\{\\Omega\(\\varepsilon^\{\-4/5\}\)\}in such cases\. Here,ρtπ\\rho\_\{t\}^\{\\pi\}are the marginals along the ground truth denoising path\.

In words, exact shallow representation of the denoising drifts can require exponential width, and shallowL2L^\{2\}\-approximation to accuracyε\\varepsiloncan requiredpoly⁡\(1/ε\)d^\{\\operatorname\{poly\}\(1/\\varepsilon\)\}width\.444Early\-stopping does not fix this\. Indeed, we show in Lemma[32](https://arxiv.org/html/2610.10875#Thmlemma32)that if oneε\\varepsilon\-approximates scores up to an OU time whose marginal haspoly⁡\(ε\)\\operatorname\{poly\}\(\\varepsilon\)error in eitherTV\\operatorname\{TV\},W1W\_\{1\}, orKL\\operatorname\{KL\}, then ourdpoly⁡\(1/ε\)d^\{\\operatorname\{poly\}\(1/\\varepsilon\)\}width lower bound still applies\.

Despite the fact that the teacher model has widthO⁡\(1\)O\(1\), we find that achieving score\-matching squared error≤ε2\\leq\\varepsilon^\{2\}using two\-layer drift models requires width ofdpoly⁡\(1/ε\)d^\{\\operatorname\{poly\}\(1/\\varepsilon\)\}\. Unlike prior works showing hardness of score\-matching\[[73](https://arxiv.org/html/2610.10875#bib.bib17),[81](https://arxiv.org/html/2610.10875#bib.bib18)\], the difficulty does not come from cryptographic lower\-bounds or spatial regularity issues, but instead from a direct width lower bound using the Hermite rank method of\[[13](https://arxiv.org/html/2610.10875#bib.bib22)\]\. It therefore demonstrates a separation between using shallow and deep neural networks for the drifts/scores in denoising generative models, rather than general hardness\.

One way to interpret our result is as follows\. We can view the entire generative process as a single very deep ResNet, for which the score\-matching loss provides supervision at intermediate layers \(see Section[5\.2](https://arxiv.org/html/2610.10875#S5.SS2)\)\. From this perspective it is up to the practitioner how to allocate their computational budget, either by implementing many shallow drift models or a few deeper ones\. In the extremes these correspond to maximal supervision \(a flow driven by shallow denoising drifts\) and minimal supervision \(a deep single\-shot denoising model\), respectively\. Theorem[1](https://arxiv.org/html/2610.10875#Thmtheorem1)indicates that the former strategy can be suboptimal for the class of ResNet pushforward measures: some amount of non\-denoising depth is required for efficient generative modeling of these distributions\. From the two\-timescale perspective, this suggests that the number of neural network layers needs to be nonnegligible with respect to the number of noise scales; see Figure[1](https://arxiv.org/html/2610.10875#S1.F1)\.

γd\\gamma\_\{d\}π\\piρ^t\\hat\{\\rho\}\_\{t\}ρtπ\\rho\_\{t\}^\{\\pi\}

Figure 1:Diagram of the trajectoriesρtπ\\rho\_\{t\}^\{\\pi\}andρ^t\\hat\{\\rho\}\_\{t\}connectingγd\\gamma\_\{d\}toπ\\pi\. The ideal reverse OU pathρtπ\\rho\_\{t\}^\{\\pi\}is depicted in red, while the effective path obtained by regressing the target scores with deep ResNet models is depicted in purple\. The paths are matched at regular intervals, while the ResNet layers are free to implement arbitrary dynamics in between\. Theorem[1](https://arxiv.org/html/2610.10875#Thmtheorem1)suggests that this freedom is useful for parameter efficiency\.More generally, this construction is a toy model for a natural type of generative process in which features are revealed not by varying the noise level, but instead through nonlinear interactions of multi\-index type\. This motivates two directions for further work: \(1\) studying tradeoffs between network depth and denoising supervision and \(2\) designing algorithms which can learn more diverse types of generative models\. We discuss future directions of research in Section[5](https://arxiv.org/html/2610.10875#S5)\.

##### Feature learning\.

In light of Theorem[1](https://arxiv.org/html/2610.10875#Thmtheorem1)’s result that neural network\-based denoising transports can require non\-denoising dynamics between noise scales, it is natural to ask about the properties of such alternative dynamics\. To this end, our second technical result aims to understand the role of feature adaptation in generic dynamical generative models\. We do so by comparing the settings where each drift is a two\-layer network whose first layer is either fixed or adaptive, i\.e\. by comparing the kernel and feature learning regimes\.

For a fixed kernelk:ℝd×ℝd→ℝd×dk:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\\times d\}with RKHSℋk\\mathcal\{H\}\_\{k\}, we denote byDℋkD\_\{\\mathcal\{H\}\_\{k\}\}theStein metricon𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)which is obtained by the length minimization \([3](https://arxiv.org/html/2610.10875#S1.E3)\) with the choiceℬ=ℋk\\mathcal\{B\}=\\mathcal\{H\}\_\{k\}\. This was studied in\[[30](https://arxiv.org/html/2610.10875#bib.bib77)\]as the Riemannian geometry on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)whose gradient flow structure generates the Stein variational gradient descent sampling algorithm \(see Sections[1\.2](https://arxiv.org/html/2610.10875#S1.SS2)and[2\.2](https://arxiv.org/html/2610.10875#S2.SS2)\), but for our purposesDℋkD\_\{\\mathcal\{H\}\_\{k\}\}simply measures the cost of the most efficient transport fromμ\\mutoπ\\piin terms of the RKHS norm of the drifts\. The RKHS norm‖vt‖ℋk\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}captures the hardness of representingvtv\_\{t\}using frozen features \(e\.g\. ifkkis a random feature kernel\) in the same way that the Barron norm‖vt‖ℬϕ\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}captures the hardness of representingvtv\_\{t\}using two\-layer neural networks with adaptive features\. Hence, the comparison betweenDℋk​\(μ,π\)D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\pi\)andDℬϕ​\(μ,π\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\pi\)provides a tool to study the role of feature learning in the task of dynamical transport fromμ\\mutoπ\\pi\.

In analogy to the study of Gaussian single\-index models in regression\[[12](https://arxiv.org/html/2610.10875#bib.bib33)\], we consider single\-index flows with Gaussian source\. We perform this comparison in both the perturbative and nonperturbative regimes:

###### Theorem 2\(Feature learning comparison, informal\)\.

Letγd∈𝒫⁡\(ℝd\)\\gamma\_\{d\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)be the standard Gaussian and letk:ℝd×ℝd→ℝd×dk:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\\times d\}be rotation\-equivariant \(see Definition[1](https://arxiv.org/html/2610.10875#Thmdefinition1)\)\. Defineπε\\pi\_\{\\varepsilon\}as the time\-ε\\varepsilonmarginal of a flow initialized fromγd\\gamma\_\{d\}with single\-index drifts with activationϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}\.555In other wordsπε=Law⁡\(Xε\)\\pi\_\{\\varepsilon\}=\\operatorname\{Law\}\(X\_\{\\varepsilon\}\), whereX˙t=v⁡\(Xt\)\\dot\{X\}\_\{t\}=v\(X\_\{t\}\)withX0∼γdX\_\{0\}\\sim\\gamma\_\{d\}andv:ℝd→ℝdv:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}is a single\-index vector fieldv⁡\(x\)=σ​u​ϕ​\(⟨x,θ⟩\)v\(x\)=\\sigma u\\phi\(\\left\\langle x,\\theta\\right\\rangle\)for someu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}andσ\>0\\sigma\>0\.

- •\(Examples[2](https://arxiv.org/html/2610.10875#Thmexample2)and[3](https://arxiv.org/html/2610.10875#Thmexample3)\)\.For each activationϕ\\phi, denote Qϕ​\(d\):=infkrotation\-equivariant\{∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)⋅lim infε↓0Dℋk​\(γd,πε\)Dℬϕ​\(γd,πε\)\}\.Q\_\{\\phi\}\(d\):=\\inf\_\{\\begin\{subarray\}\{c\}\\text\{$k$ rotation\-equivariant\}\\end\{subarray\}\}\\left\\\{\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}\)\}\\right\\\}\.\(5\)Then, we have the scalingsQReLU​\(d\)=\+∞Q\_\{\\operatorname\{ReLU\}\}\(d\)=\+\\infty,Qtanh​\(d\)≍dd/4Q\_\{\\tanh\}\(d\)\\asymp d^\{d/4\},Qsin​\(d\)≍edQ\_\{\\sin\}\(d\)\\asymp e^\{\\sqrt\{d\}\}asd→∞d\\to\\infty\.
- •\(Example[4](https://arxiv.org/html/2610.10875#Thmexample4)\)\.There exists a rotation\-equivariantk⋆k\_\{\\star\}s\.t\. for each fixedε\>0\\varepsilon\>0 \(supt∈\[0,1\]∫ℝdtr⁡k⋆​\(x,x\)​d​ρt⋆​\(x\)\)⋅∫01‖vt⋆‖ℋk⋆​𝑑tDℬϕ​\(γd,πε\)≲poly⁡\(d\)\\left\(\\sup\_\{t\\in\[0,1\]\}\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\_\{\\star\}\(x,x\)d\\rho\_\{t\}^\{\\star\}\(x\)\}\\right\)\\cdot\\frac\{\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{\\star\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{\\star\}\}\}dt\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}\)\}\\lesssim\\operatorname\{poly\}\(d\)\(6\)asd→∞d\\to\\infty, where\(vt⋆,ρt⋆\)t∈\[0,1\]\(v\_\{t\}^\{\\star\},\\rho\_\{t\}^\{\\star\}\)\_\{t\\in\[0,1\]\}satisfies \([1](https://arxiv.org/html/2610.10875#S1.E1)\) withρ0⋆=γd\\rho\_\{0\}^\{\\star\}=\\gamma\_\{d\}andρ1⋆=πε\\rho\_\{1\}^\{\\star\}=\\pi\_\{\\varepsilon\}\.

In words, the separation betweenDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}and rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}is made superpolynomially\-large by perturbative single\-index flows, but not by nonperturbative ones\.

The first part is essentially the vector field version of the classical feature learning separation statement that single\-index functions are difficult to represent in rotationally\-invariant RKHSs\[[7](https://arxiv.org/html/2610.10875#bib.bib46),[38](https://arxiv.org/html/2610.10875#bib.bib65)\], and the proof is the same \(via spherical harmonics/Hermite decomposition\)\. However, the second part implies that this separation disappears once depth is allowed\. Phrased differently, the second part says that a single\-index function can be efficiently represented by a deep ResNet whose blocks use the same fixed feature basis\. This observation raises the question of whether depth can compensate for a lack of adaptive feature selection \(particularly in dynamical measure transport\), which we leave to future work\. Note that these quantitiesDℋk,DℬϕD\_\{\\mathcal\{H\}\_\{k\}\},D\_\{\\mathcal\{B\}\_\{\\phi\}\}consider an infimum over all transports fromγd\\gamma\_\{d\}toπ\\pi, which may be qualitatively different than the transports that are learned in practice \(e\.g\. Wasserstein optimal transports, denoising transports, etc\.\)\.

### 1\.2Related work

##### Dynamical optimal transport and associated metrics\.

The Benamou\-Brenier formulation of optimal transport\[[9](https://arxiv.org/html/2610.10875#bib.bib61)\]measures transport cost by minimizing an action subject to the continuity equation, and identifies the squared 2\-Wasserstein distance between two probability measures as the minimal possible action\. The textbook\[[5](https://arxiv.org/html/2610.10875#bib.bib69)\]thoroughly develops this perspective in the Wasserstein case\. The survey\[[10](https://arxiv.org/html/2610.10875#bib.bib81)\]covers some more general dynamical transport metrics coming from other choices of the action, see also\[[33](https://arxiv.org/html/2610.10875#bib.bib79)\]and\[[32](https://arxiv.org/html/2610.10875#bib.bib80)\]for control\-theoretic instantiations of this idea\.

The choice of a non\-LpL^\{p\}dynamical transport cost has direct application to the sampling problem\. In\[[30](https://arxiv.org/html/2610.10875#bib.bib77)\], a dynamical optimal transport metric is defined on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)using transport costs coming from RKHS norms; this is called theStein geometry, since the resulting Riemannian geometry on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)is used to analyze the mean\-field behavior of the Stein variational gradient descent algorithm\[[55](https://arxiv.org/html/2610.10875#bib.bib78)\]by casting it as a formal Riemannian gradient flow\. Similarly, in\[[42](https://arxiv.org/html/2610.10875#bib.bib71)\]a dynamical optimal transport metric is defined using the Radon transform, and the resulting Riemannian gradient flow structure yields a sampling algorithm with better per\-step computational cost\.

##### Dynamical optimal transport for generative modeling\.

Dynamical generative modeling requires choosing or learning a generative path from data\. Some recent works propose to do so using Wasserstein optimal transport\.\[[35](https://arxiv.org/html/2610.10875#bib.bib8)\]and\[[63](https://arxiv.org/html/2610.10875#bib.bib25)\]use optimal transport machinery to regularize neural ODEs and normalizing flow models, respectively\.\[[79](https://arxiv.org/html/2610.10875#bib.bib42)\]uses optimal couplings computed over minibatches in order to learn generative models\. Lastly,\[[56](https://arxiv.org/html/2610.10875#bib.bib48)\]proposes a procedure in which a generative model is iteratively “rectified” to reduce transport costs and straighten trajectories\.

Closer to our work, several recent papers propose alternative criteria for dynamical optimal transport based on velocity regularity with the goal of better generative modeling\. The paper\[[80](https://arxiv.org/html/2610.10875#bib.bib2)\]investigates rescheduling of the Wasserstein dynamical transport \(and more general straight\-line interpolations\), deriving a reparameterization of time which minimizes the largest Lipschitz constant of the drift fields\. The papers\[[65](https://arxiv.org/html/2610.10875#bib.bib49)\]and\[[59](https://arxiv.org/html/2610.10875#bib.bib5)\]propose to optimize dynamical transports using RKHS penalties on their velocity fields: the former derives some sample and model complexity bounds in this setting, while the latter highlights the optimal control perspective and practical algorithmic considerations\. While researchers are beginning to design and study non\-Wasserstein dynamical optimal transports for generative modeling, it remains unclear how the geometry of a target distribution relates to the notion of drift regularity we impose\.

##### Approximation theory for deep ResNets and neural ODEs\.

There is a growing body of work which studies the approximation power of deep ResNets and their continuous\-time/infinite\-depth limits, neural ODEs \(see\[[23](https://arxiv.org/html/2610.10875#bib.bib43)\]for a quantitative study of this convergence\)\.

One approach is based on defining function spaces with suitable norms or metrics\. The paper\[[31](https://arxiv.org/html/2610.10875#bib.bib60)\]notes that classical Barron spaces characterize shallow representations and do not explicitly encode depth or compositional structure; the authors subsequently design flow\-induced function spaces \(via neural ODE representations with sufficient regularity across depth\) and prove direct and inverse approximation theorems for residual networks along with statistical complexity bounds\. Related approaches include the ResNet norm of\[[46](https://arxiv.org/html/2610.10875#bib.bib37)\]and its connection to circuit complexity as well as the study of minimum\-norm representation paths in\[[45](https://arxiv.org/html/2610.10875#bib.bib50)\]\. These works focus on understanding which functions can be represented and approximated using ResNets and neural ODEs, largely by emphasizing the role of depth and the ODE structure\.

Complementary work examines which maps neural ODE architectures can represent; see\[[15](https://arxiv.org/html/2610.10875#bib.bib3), Section 2\]and\[[50](https://arxiv.org/html/2610.10875#bib.bib4)\]\. These results explain the role of augmentation and geometric restrictions, including the nonsingular Jacobian of finite\-time flows generated by smooth vector fields\. Such representability results do not by themselves give bounds on the norm or network size needed to approximate a particular measure transport\. Relatedly, the work\[[67](https://arxiv.org/html/2610.10875#bib.bib27)\]investigates the universal approximation property for neural ODEs through control\-theoretic arguments, and\[[52](https://arxiv.org/html/2610.10875#bib.bib6)\]shows that ODEs with two\-layer neural network drifts can approximate a wide class of dynamics or endpoint mappings\.

There is also a recent line of work which specifically investigates the use of neural ODEs in the representation and approximation of probability distributions via dynamical transport\. In\[[57](https://arxiv.org/html/2610.10875#bib.bib7)\]they establish statistical convergence guarantees for likelihood\-trained neural ODEs, including nearly minimax\-optimal density\-estimation rates under smoothness assumptions\. Their companion paper\[[58](https://arxiv.org/html/2610.10875#bib.bib26)\]constructs straight\-line flow representations for suitable classes of smooth transport maps and derives explicit neural ODE approximation bounds in Wasserstein distance and KL divergence, with dependence on the dimension and regularity of the source and target densities\.

To summarize, the aforementioned works develop approximation theories for deep ResNets and neural ODEs, both for function approximation and recently in the context of probability measure approximation\. These methods can yield approximation rates in the spirit of the shallow neural network theory, but it is still unclear which geometric properties of a data distribution allow neural ODE transports with few parameters\.

##### Optimality of reverse Ornstein\-Uhlenbeck transport\.

It is known that the reverse OU probability flow \(which is one of the most common approaches to generative modeling, see Section[2\.3](https://arxiv.org/html/2610.10875#S2.SS3)\) does not always minimize the Benamou\-Brenier action associated with Wasserstein optimal transport\. In\[[77](https://arxiv.org/html/2610.10875#bib.bib64)\]it is shown that the endpoint map of the related reverse heat flow transport of Kim\-Milman\[[49](https://arxiv.org/html/2610.10875#bib.bib44)\]can differ from the Brenier map, see also\[[51](https://arxiv.org/html/2610.10875#bib.bib66)\]for quadratic transport cost suboptimality\. However, from\[[61](https://arxiv.org/html/2610.10875#bib.bib75)\]it is also known that these reverse heat flow transports have endpoint maps with good spatial regularity\. These results leave open the question of how reverse OU transport compares under costs that measure the approximation/learning complexity of its velocity field\.

There are some notable recent works demonstrating some hardness of learning reverse OU transports\. The paper\[[73](https://arxiv.org/html/2610.10875#bib.bib17)\]leverages cryptographic hardness to show the hardness of learningL2L^\{2\}\-approximate scores; in that construction the hardness comes from the learning problem, and the author notes that the true scores are easy to approximate with neural networks\. The recent work of\[[81](https://arxiv.org/html/2610.10875#bib.bib18)\]constructs distributions for which \(1\) there exists anL2L^\{2\}approximate drift which does not sample well and \(2\) no poly\-time drift estimator with controlled Lipschitz constant can sample well\. Our approximation lower bounds are more structural as they do not rely on regularity or computational hardness: the Hermite rank argument directly lower bounds network size irrespective of the estimator’s regularity or computational complexity\.

##### Feature learning in RKHS and Barron spaces\.

For a common dictionary of neurons, an RKHS fixes a feature distribution, while a Barron norm optimizes over representations and can therefore reflect features adapted to the target\. In\[[7](https://arxiv.org/html/2610.10875#bib.bib46)\], this distinction is studied through variation\-norm regularization in order to prove adaptation to unknown low\-dimensional structure, i\.e\.feature learning\. The separation between kernel methods and neural networks is further investigated in\[[38](https://arxiv.org/html/2610.10875#bib.bib65)\]and\[[82](https://arxiv.org/html/2610.10875#bib.bib28)\]using tools from spherical harmonic analysis\. These works analyze feature learning separations in the shallow regression setting\.

Deep and compositional kernels have also been studied\[[69](https://arxiv.org/html/2610.10875#bib.bib11),[53](https://arxiv.org/html/2610.10875#bib.bib14)\], with particular focus on the effect of depth on the spectrum and complexity of their associated RKHSs\. These works demonstrate improved modeling properties arising from the compositional structure, but it is still unclear to what extent depth can compensate for the lack of adaptive feature selection in a fixed\-feature RKHS\. This question remains poorly understood not only in the setting of regression using deep models composed of RKHS layers, but also in the context of dynamical measure transport with RKHS drifts\.

### 1\.3Statement of AI Usage

Several mathematical ideas and proof arguments in this paper were suggested by ChatGPT 5\.6 and Astra and subsequently developed and refined by the authors\. These include the sharp scaling of RKHS tangent norms in Lemma[10](https://arxiv.org/html/2610.10875#Thmlemma10)and thetanh\\tanhdeconvolution argument in Lemma[17](https://arxiv.org/html/2610.10875#Thmlemma17)\. For the tail control of the averaging operator in Lemma[23](https://arxiv.org/html/2610.10875#Thmlemma23), the initial ideas originated with the authors, and subsequent exchanges with Astra helped develop and refine the argument\. The authors take full responsibility for all results and arguments in this paper\.

## 2Preliminaries

For some notation, we write𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)for the space of Borel probability measures onℝd\{\\mathbb\{R\}\}^\{d\}\(equipped with the narrow topology, see\[[5](https://arxiv.org/html/2610.10875#bib.bib69), Chapter 5\.1\]\),𝒫2​\(ℝd\)\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)for those with finite second moment, andγd:=𝒩⁡\(0,𝟙d\)\\gamma\_\{d\}:=\\mathcal\{N\}\(0,\{\\mathbbm\{1\}\}\_\{d\}\)for the standard Gaussian\. IfT:ℝd→ℝmT:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{m\}is measurable andμ∈𝒫⁡\(ℝd\)\\mu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), thenT\#​μ∈𝒫⁡\(ℝm\)T\_\{\\\#\}\\mu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{m\}\)denotes the pushforward ofμ\\mubyTT\. We denote by∥⋅∥∞\\\|\\cdot\\\|\_\{\\infty\}the sup norm and by∥⋅∥L∞\\\|\\cdot\\\|\_\{L^\{\\infty\}\}the essential supremum norm w\.r\.t\. Lebesgue\. All logarithms are natural\.

Throughout this paper, dynamical transport between probability measures is described by the continuity equation\. A narrowly continuous curve\(ρt\)t∈\[0,1\]⊆𝒫⁡\(ℝd\)\(\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}\\subseteq\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)and a Borel vector fieldvt:ℝd→ℝdv\_\{t\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}are said to satisfy the continuity equation if

∂tρt\+∇⋅\(ρt​vt\)=0\\partial\_\{t\}\\rho\_\{t\}\+\\nabla\\cdot\(\\rho\_\{t\}v\_\{t\}\)=0\(7\)in the weak sense, i\.e\. if for everyφ∈Cc∞​\(\(0,1\)×ℝd\)\\varphi\\in C\_\{c\}^\{\\infty\}\(\(0,1\)\\times\{\\mathbb\{R\}\}^\{d\}\)we have

∫01∫ℝd\(∂tφ⁡\(t,x\)\+⟨vt​\(x\),∇φ​\(t,x\)⟩\)​d​ρt​\(x\)​𝑑t=0,\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\(\\partial\_\{t\}\\varphi\(t,x\)\+\\langle v\_\{t\}\(x\),\\nabla\\varphi\(t,x\)\\rangle\\right\)d\\rho\_\{t\}\(x\)dt=0,\(8\)see \(8\.1\.3\) of\[[5](https://arxiv.org/html/2610.10875#bib.bib69)\]\. We always assume enough integrability for this expression to be well\-defined; for example,

∫01‖vt‖L1​\(ρt,ℝd\)​𝑑t<∞\.\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{L^\{1\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)\}dt<\\infty\.\(9\)Ifρ0=μ\\rho\_\{0\}=\\muandρ1=ν\\rho\_\{1\}=\\nu, we say that\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}transportsμ\\mutoν\\nu\. Whenvtv\_\{t\}is sufficiently regular, the same transport can be described by the ODE

X˙t=vt​\(Xt\),X0∼μ,ρt=Law⁡\(Xt\)\.\\dot\{X\}\_\{t\}=v\_\{t\}\(X\_\{t\}\),\\qquad X\_\{0\}\\sim\\mu,\\qquad\\rho\_\{t\}=\\operatorname\{Law\}\(X\_\{t\}\)\.\(10\)We use the continuity equation formulation because it does not require pointwise existence or uniqueness of the flow\.

### 2\.1Wasserstein metric

Forμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), the quadratic Wasserstein distance is defined by

W22​\(μ,ν\):=infΓ∈Π⁡\(μ,ν\)∫ℝd×ℝd\|x−y\|2​𝑑Γ​\(x,y\),W\_\{2\}^\{2\}\(\\mu,\\nu\):=\\inf\_\{\\Gamma\\in\\Pi\(\\mu,\\nu\)\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\}\|x\-y\|^\{2\}d\\Gamma\(x,y\),\(11\)whereΠ⁡\(μ,ν\)\\Pi\(\\mu,\\nu\)is the set of couplings ofμ\\muandν\\nu\. This is finite wheneverμ,ν∈𝒫2​\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)\. There are many interesting properties of the Wasserstein space\(𝒫2​\(ℝd\),W2\)\(\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\),W\_\{2\}\)which we will not require, see the second half of the textbook\[[5](https://arxiv.org/html/2610.10875#bib.bib69)\]for more details\. The main fact which will be relevant for us is the formula of Benamou\-Brenier\[[9](https://arxiv.org/html/2610.10875#bib.bib61)\], which gives an equivalent dynamical description:

W2​\(μ,ν\)2=infμ→ν∫01‖vt‖L2​\(ρt,ℝd\)2​𝑑t,W\_\{2\}\(\\mu,\\nu\)^\{2\}=\\inf\_\{\\mu\\to\\nu\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{L^\{2\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)\}^\{2\}dt,\(12\)where the infimum is over\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}which transportμ\\mutoν\\nu\(i\.e\. which satisfy the continuity equation \([7](https://arxiv.org/html/2610.10875#S2.E7)\) andρ0=μ\\rho\_\{0\}=\\mu,ρ1=ν\\rho\_\{1\}=\\nu\)\. Equivalently,

W2​\(μ,ν\)=infμ→ν∫01‖vt‖L2​\(ρt,ℝd\)​𝑑t\.W\_\{2\}\(\\mu,\\nu\)=\\inf\_\{\\mu\\to\\nu\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{L^\{2\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)\}dt\.\(13\)The equivalence of \([12](https://arxiv.org/html/2610.10875#S2.E12)\) and \([13](https://arxiv.org/html/2610.10875#S2.E13)\) follows by reparameterizing a path to have constant speed\.

In particular, the instantaneous Wasserstein cost of a velocity field is itsL2​\(ρt,ℝd\)L^\{2\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)norm, which measures the average speed of particles under the current distribution\. Otto\[[64](https://arxiv.org/html/2610.10875#bib.bib68)\]used this to recognize the metric geometry\(𝒫2​\(ℝd\),W2\)\(\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\),W\_\{2\}\)as a formal Riemannian geometry, in which the tangent vectors are identified as vector fields and the tangent norm atρ∈𝒫2​\(ℝd\)\\rho\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)is∥⋅∥L2​\(ρ,ℝd\)\\\|\\cdot\\\|\_\{L^\{2\}\(\\rho;\{\\mathbb\{R\}\}^\{d\}\)\}\. This allows us to draw inspiration from finite\-dimensional Riemannian geometry to reason about the dynamical evolution of probability measures\. As some examples:

- •\(Gradient flows\)\. For a sufficiently regular functionalℱ:𝒫2​\(ℝd\)→ℝ\\mathcal\{F\}:\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)\\to\{\\mathbb\{R\}\}and an initialρ0∈𝒫2​\(ℝd\)\\rho\_\{0\}\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\), one may define a curve of measures\(ρt\)t≥0⊆𝒫2​\(ℝd\)\(\\rho\_\{t\}\)\_\{t\\geq 0\}\\subseteq\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)as theW2W\_\{2\}gradient flow ofℱ\\mathcal\{F\}\. It can be shown that\(vt,ρt\)t≥0\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\geq 0\}satisfies the continuity equation \([7](https://arxiv.org/html/2610.10875#S2.E7)\) with vt\(x\)=−\(∇δℱ\(ρt\)\)\(x\),v\_\{t\}\(x\)=\-\(\\nabla\\delta\\mathcal\{F\}\(\\rho\_\{t\}\)\)\(x\),\(14\)whereδ​ℱ​\(ρ\):ℝd→ℝ\\delta\\mathcal\{F\}\(\\rho\):\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}denotes the first variation ofℱ\\mathcal\{F\}atρ\\rho\. An important special case is withℱ\(⋅\):=KL\(⋅∥π\)\\mathcal\{F\}\(\\cdot\):=\\operatorname\{KL\}\(\\cdot\\\|\\pi\)for a targetπ∈𝒫2​\(ℝd\)\\pi\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\), in which case the resulting curve\(ρt\)t≥0\(\\rho\_\{t\}\)\_\{t\\geq 0\}describes the evolution of marginal distributions along the Langevin dynamics\. See Chapter 11 of\[[5](https://arxiv.org/html/2610.10875#bib.bib69)\]for full details\.
- •\(Geodesics\)\. For two measuresμ,ν∈𝒫2​\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\), one can describe the optimal transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}which minimizes \([12](https://arxiv.org/html/2610.10875#S2.E12)\) via the geodesic PDE vt=∇φt,∂tφt\+12​\|∇φt\|2=0​a\.e\.,v\_\{t\}=\\nabla\\varphi\_\{t\},\\quad\\partial\_\{t\}\\varphi\_\{t\}\+\\frac\{1\}\{2\}\|\\nabla\\varphi\_\{t\}\|^\{2\}=0\\text\{ a\.e\.\},\(15\)whereφt:ℝd→ℝ\\varphi\_\{t\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}are the generating potentials \(see e\.g\.\[[5](https://arxiv.org/html/2610.10875#bib.bib69), Theorem 6\.2\.10\]\)\.

An important property of the Wasserstein geometry is its locality: the velocity of a gradient flow \(resp\. geodesic\) at a pointx∈ℝdx\\in\{\\mathbb\{R\}\}^\{d\}depends only on the Euclidean gradient of the generating potential∇δℱ\(ρt\)\\nabla\\delta\\mathcal\{F\}\(\\rho\_\{t\}\)\(resp\.φt\\varphi\_\{t\}\) atxx\. This lack of cooperation between particles is part of the reason why theW2W\_\{2\}gradient flow ofKL\(⋅∥π\)\\operatorname\{KL\}\(\\cdot\\\|\\pi\)can be simulated by the Langevin SDE, which is implemented with noninteracting particles\.

We emphasize that theL2​\(ρt,ℝd\)L^\{2\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)norm does not measure how complicated the functionx↦vt​\(x\)x\\mapsto v\_\{t\}\(x\)is in terms of smoothness, approximation difficulty, etc\. A transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}which is optimal w\.r\.t\.W2W\_\{2\}connects two measures in a way which moves particles minimally, but by doing so it can sacrifice the spatial regularity of the induced drift fieldsvtv\_\{t\}\.

### 2\.2Stein metric

Letk:ℝd×ℝd→ℝd×dk:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\\times d\}be a symmetric positive\-semidefinite matrix\-valued kernel, and letℋk\\mathcal\{H\}\_\{k\}be its vector\-valued reproducing kernel Hilbert space \(RKHS\)\. In particular, for everyx,a∈ℝdx,a\\in\{\\mathbb\{R\}\}^\{d\}we havek⁡\(⋅,x\)​a∈ℋkk\(\\cdot,x\)a\\in\\mathcal\{H\}\_\{k\}and

⟨v,k⁡\(⋅,x\)​a⟩ℋk=⟨v⁡\(x\),a⟩\(∀v∈ℋk\)\.\\langle v,k\(\\cdot,x\)a\\rangle\_\{\\mathcal\{H\}\_\{k\}\}=\\langle v\(x\),a\\rangle\\quad\(\\forall v\\in\\mathcal\{H\}\_\{k\}\)\.\(16\)
###### Assumptionkk\.

Throughout the rest of this paper, we assume the kernelk:ℝd×ℝd→ℝd×dk:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\\times d\}is jointly continuous \(w\.r\.t\. the operator norm\)\.

If\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}is a path withvt∈ℋkv\_\{t\}\\in\\mathcal\{H\}\_\{k\}we may consider its RKHS energy∫01‖vt‖ℋk2​𝑑t\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}^\{2\}dtin the spirit of \([12](https://arxiv.org/html/2610.10875#S2.E12)\), in which case the corresponding transport distance is

Dℋk​\(μ,ν\)2:=infμ→ν∫01‖vt‖ℋk2​𝑑t,D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\)^\{2\}:=\\inf\_\{\\mu\\to\\nu\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|^\{2\}\_\{\\mathcal\{H\}\_\{k\}\}dt,\(17\)with the infimum over solutions of the continuity equation \([7](https://arxiv.org/html/2610.10875#S2.E7)\) withvt∈ℋkv\_\{t\}\\in\\mathcal\{H\}\_\{k\}for almost everytt\. Equivalently, by constant\-speed reparameterization,

Dℋk​\(μ,ν\)=infμ→ν∫01‖vt‖ℋk​𝑑t\.D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\)=\\inf\_\{\\mu\\to\\nu\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}dt\.\(18\)Note that if∥⋅∥L2​\(ρ,ℝd\)≲∥⋅∥ℋk\\\|\\cdot\\\|\_\{L^\{2\}\(\\rho;\{\\mathbb\{R\}\}^\{d\}\)\}\\lesssim\\\|\\cdot\\\|\_\{\\mathcal\{H\}\_\{k\}\}\(which holds for e\.g\. bounded kernels\) thenW2​\(μ,ν\)≲Dℋk​\(μ,ν\)W\_\{2\}\(\\mu,\\nu\)\\lesssim D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\), see\[[30](https://arxiv.org/html/2610.10875#bib.bib77), Lemma 17\]\.

Under suitable regularity assumptions, this construction gives an extended metric \(i\.e\. a metric which may take the value\+∞\+\\infty\) which is referred to as theStein metricin the literature; see\[[30](https://arxiv.org/html/2610.10875#bib.bib77), Definition 15, Lemma 17\]for one precise formulation\. Importantly, the space\(𝒫⁡\(ℝd\),Dℋk\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{H\}\_\{k\}\}\)can be equipped with a formal Riemannian geometry \(since the tangent spacesℋk\\mathcal\{H\}\_\{k\}are Hilbert\) to enable analytical methods in analogy with the Wasserstein setting\. This geometry has been investigated recently in\[[30](https://arxiv.org/html/2610.10875#bib.bib77)\]as the metric on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)whose gradient flow structure induces the Stein variational gradient descent \(SVGD\) algorithm\[[55](https://arxiv.org/html/2610.10875#bib.bib78)\]\. In particular,\[[30](https://arxiv.org/html/2610.10875#bib.bib77)\]developed the Riemannian perspective and showed the following:

- •\(Gradient flows\)\. In contrast with the Wasserstein case \([14](https://arxiv.org/html/2610.10875#S2.E14)\), theDℋkD\_\{\\mathcal\{H\}\_\{k\}\}gradient flow ofℱ\\mathcal\{F\}is implemented by\(vt,ρt\)t≥0\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\geq 0\}satisfying the continuity equation \([7](https://arxiv.org/html/2610.10875#S2.E7)\) with vt\(x\)=−\(𝒯k,ρt∇δℱ\(ρt\)\)\(x\),v\_\{t\}\(x\)=\-\(\\mathcal\{T\}\_\{k,\\rho\_\{t\}\}\\nabla\\delta\\mathcal\{F\}\(\\rho\_\{t\}\)\)\(x\),\(19\)where𝒯k,ρt:L2​\(ρt,ℝd\)→L2​\(ρt,ℝd\)\\mathcal\{T\}\_\{k,\\rho\_\{t\}\}:L^\{2\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)\\to L^\{2\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)is the integral operator with kernelkkand measureρt\\rho\_\{t\}\.666In other words,\(𝒯k,ρ​v\)​\(x\)=∫ℝdk⁡\(x,y\)​v​\(y\)​𝑑ρ​\(y\)\(\\mathcal\{T\}\_\{k,\\rho\}v\)\(x\)=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}k\(x,y\)v\(y\)d\\rho\(y\)\.An important special case is withℱ\(⋅\):=KL\(⋅∥π\)\\mathcal\{F\}\(\\cdot\):=\\operatorname\{KL\}\(\\cdot\\\|\\pi\)for a targetπ∈𝒫⁡\(ℝd\)\\pi\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), in which case the resulting curve\(ρt\)t≥0\(\\rho\_\{t\}\)\_\{t\\geq 0\}describes the evolution of marginal distributions along the mean\-field limit of continuous\-time SVGD\. See Lemma 11 and Corollary 13 of\[[30](https://arxiv.org/html/2610.10875#bib.bib77)\]for these statements\.
- •\(Geodesics\)\. For two measuresμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), one can describe a transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}which is stationary for the energy \([17](https://arxiv.org/html/2610.10875#S2.E17)\) by the PDE vt=𝒯k,ρt∇φt,∂tφt\+⟨∇φt,𝒯k,ρt∇φt⟩=0a\.e\.,v\_\{t\}=\\mathcal\{T\}\_\{k,\\rho\_\{t\}\}\\nabla\\varphi\_\{t\},\\quad\\partial\_\{t\}\\varphi\_\{t\}\+\\left\\langle\\nabla\\varphi\_\{t\},\\mathcal\{T\}\_\{k,\\rho\_\{t\}\}\\nabla\\varphi\_\{t\}\\right\\rangle=0\\text\{ a\.e\.\},\(20\)whereφt:ℝd→ℝ\\varphi\_\{t\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}are the generating potentials, see\[[30](https://arxiv.org/html/2610.10875#bib.bib77), Proposition 20\]\. This can be contrasted with the Wasserstein case \([15](https://arxiv.org/html/2610.10875#S2.E15)\)\.

The difference between the Wasserstein and Stein geometries is easily understood through the tangent norms:

Wasserstein:∥vt∥L2​\(ρt,ℝd\),Stein:∥vt∥ℋk\.\\text\{\{Wasserstein\}: \}\\\|v\_\{t\}\\\|\_\{L^\{2\}\(\\rho\_\{t\};\{\\mathbb\{R\}\}^\{d\}\)\},\\qquad\\text\{\{Stein\}: \}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}\.\(21\)The first measures how far/fast particles move, while the second measures how large the velocity field is in a prescribed RKHS\. Crucially, the introduction of a kernelkkforces particles in different locations to interact in order to decrease the RKHS norm; this can be also seen through the gradient flow and geodesic identities \([19](https://arxiv.org/html/2610.10875#S2.E19)\) and \([20](https://arxiv.org/html/2610.10875#S2.E20)\), in which the kernel integral operator𝒯k,ρ\\mathcal\{T\}\_\{k,\\rho\}couples the velocities experienced by different particles\. This interaction \(and its implementable form using Stein integration by parts\) yields the advantages of the SVGD sampling algorithm\.

Any choice of kernelk:ℝd×ℝd→ℝd×dk:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\\times d\}induces its own Stein geometry through the metricDℋkD\_\{\\mathcal\{H\}\_\{k\}\}, which can be understood in two equivalent ways\. One perspective is that the kernel defines a notion of vector field complexity via its RKHSℋk\\mathcal\{H\}\_\{k\}, which yields a Benamou\-Brenier\-type energy minimization via \([17](https://arxiv.org/html/2610.10875#S2.E17)\)\. The other perspective is that the kernel couples spatial positions and the velocities experienced by particles at those positions via the operator𝒯k,ρ\\mathcal\{T\}\_\{k,\\rho\}\. Of course, this interaction and the resulting quantities/algorithms depend heavily on the choice of kernel\.

Some common choices are bilinear kernelsk⁡\(x,y\)=\(1\+⟨x,y⟩\)​𝟙dk\(x,y\)=\(1\+\\left\\langle x,y\\right\\rangle\)\{\\mathbbm\{1\}\}\_\{d\}, Gaussian/RBF kernelsk⁡\(x,y\)=e−‖x−y‖2​𝟙dk\(x,y\)=e^\{\-\\\|x\-y\\\|^\{2\}\}\{\\mathbbm\{1\}\}\_\{d\}, and the like\[[55](https://arxiv.org/html/2610.10875#bib.bib78)\]; these introduce isotropic interactions between particles\. As is the case in kernel\-based learning methods, much can be achieved if one were able to select the “perfect kernel” for the given data\. However, in practice one must choose a kernel in advance, in which case the kernel captures a fixed set of features and ought not to prefer any particular direction\. We will focus on the following class:

###### Definition 1\(Rotation\-equivariant kernel\)\.

A matrix\-valued kernelkkis rotation\-equivariant if

k⁡\(Q​x,Q​y\)=Q​k​\(x,y\)​Q⊤\(∀x,y∈ℝd\)k\(Qx,Qy\)=Qk\(x,y\)Q^\{\\top\}\\quad\(\\forall x,y\\in\{\\mathbb\{R\}\}^\{d\}\)\(22\)for every orthogonal matrixQ∈O⁡\(d\)Q\\in O\(d\)\. We denote the class of such kernels by𝒦rot\\mathcal\{K\}\_\{\\operatorname\{rot\}\}\.

We emphasize that in practice \(sampling with SVGD, learning dynamical transports via kernel methods or neural networks in the kernel regime, etc\.\) the kernels involved tend to be rotation\-equivariant\[[55](https://arxiv.org/html/2610.10875#bib.bib78)\]\. Informally,𝒦rot\\mathcal\{K\}\_\{\\operatorname\{rot\}\}can be viewed as the class of kernels which are “data\-agnostic” since they represent a fixed basis\-independent feature set and cannot know any features of the data\.

One of the main developments of this work will be the construction of a geometry on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)using Barron norms instead of RKHS norms, in which case the induced geometry can be understood as “adaptively choosing features based on the data”\. This perspective will yield dynamical transports and algorithms which leverage feature learning, and one contribution of this work will be the comparison between fixed features in the rotation\-equivariant Stein case and learned features in the Barron case\. We defer this construction until Section[3](https://arxiv.org/html/2610.10875#S3)\.

### 2\.3Generative models as dynamical transports

So far, we have discussed the geometry induced by minimization over dynamical transports fromμ\\mutoν\\nufor an arbitrary pairμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\. While these optimal transport constructions are mathematically meaningful on their own, we are interested in these geometries as analytical tools to study the task of generative modeling\.777In the companion paper\[[28](https://arxiv.org/html/2610.10875#bib.bib1)\]we will also leverage the same constructions to study the related task of sampling\.In this subsection we frame generative modeling as the learning of a dynamical transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}in order to draw practical conclusions from these geometric quantities\.

Letμ\\mube a base distribution which is easy to sample from \(such as the standard Gaussianγd\\gamma\_\{d\}\),888Generallyμ\\muwill be rotationally\-invariant, since it must also be data\-agnostic and ought not to depend on the choice of basis\.and letπ\\pibe the target data distribution\. For our purposes, a \(continuous\-time\) generative model consists of a velocity field\(vt\)t∈\[0,1\]\(v\_\{t\}\)\_\{t\\in\[0,1\]\}such that the ODE

X˙t=vt​\(Xt\),X0∼μ,\\dot\{X\}\_\{t\}=v\_\{t\}\(X\_\{t\}\),\\qquad X\_\{0\}\\sim\\mu,\(23\)satisfiesLaw⁡\(X1\)=π\\operatorname\{Law\}\(X\_\{1\}\)=\\pi\. This is the standard setup of aflow model\[[17](https://arxiv.org/html/2610.10875#bib.bib38),[41](https://arxiv.org/html/2610.10875#bib.bib39)\], and the continuity equation induced by \([23](https://arxiv.org/html/2610.10875#S2.E23)\) can also describe the marginals of a stochastic interpolant or SDE\-based generative model via its probability flow ODE\. Thus, the goal of the task of generative modeling is to learn a suitable\(vt\)t∈\[0,1\]\(v\_\{t\}\)\_\{t\\in\[0,1\]\}using samples from the targetπ\\pi\.

The endpoint measures do not determine the path: different generative models may connect the sameμ\\muandπ\\piusing very different intermediate distributions and velocity fields\. For example, Wasserstein optimal transport chooses a path minimizing \([13](https://arxiv.org/html/2610.10875#S2.E13)\), while flow\-matching and diffusion methods usually prescribe a convenient interpolation and learn the velocity field associated with \(the reversal of\) that interpolation\. Hence, the learning problem is not well\-specified unless one chooses a particular transport betweenμ\\muandπ\\pito attempt to learn\. We summarize the main choices below:

##### Reverse Ornstein\-Uhlenbeck transport\.

The main prescribed path studied in this paper is the denoising path to a targetπ∈𝒫⁡\(ℝd\)\\pi\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)used by modern diffusion models\. The Ornstein\-Uhlenbeck \(OU\) semigroup\(Pτ\)τ≥0\(P\_\{\\tau\}\)\_\{\\tau\\geq 0\}acts on functions according to

Pτ​f​\(x\):=𝔼Z∼γd​\[f⁡\(e−τ​x\+1−e−2​τ​Z\)\],P\_\{\\tau\}f\(x\):=\\mathbb\{E\}\_\{Z\\sim\\gamma\_\{d\}\}\\left\[f\\left\(e^\{\-\\tau\}x\+\\sqrt\{1\-e^\{\-2\\tau\}\}Z\\right\)\\right\],\(24\)and the adjoint operators\(Pτ∗\)τ≥0\(P\_\{\\tau\}^\{\*\}\)\_\{\\tau\\geq 0\}act on probability measures and describe the corresponding flow along the OU path\. Indeed, ifX0∼γdX\_\{0\}\\sim\\gamma\_\{d\}andX1∼πX\_\{1\}\\sim\\piare independent, then fort∈\[0,1\]t\\in\[0,1\]the interpolant

Xt:=t​X1\+1−t2​X0X\_\{t\}:=tX\_\{1\}\+\\sqrt\{1\-t^\{2\}\}X\_\{0\}\(25\)has law given by

ρt:=Law⁡\(Xt\)=P−log⁡t∗​π,t∈\(0,1\]\\rho\_\{t\}:=\\operatorname\{Law\}\(X\_\{t\}\)=P\_\{\-\\log t\}^\{\*\}\\pi,\\quad t\\in\(0,1\]\(26\)and satisfiesρ0=γd\\rho\_\{0\}=\\gamma\_\{d\}andρ1=π\\rho\_\{1\}=\\pi\. Here, we use the time changeτ=−log⁡t\\tau=\-\\log tto convert from transport timet∈\(0,1\]t\\in\(0,1\]to OU timeτ∈\[0,∞\)\\tau\\in\[0,\\infty\)\. Note thatρt\\rho\_\{t\}is absolutely continuous with smooth density, and ifπ\\piis also absolutely continuous then the relative Gaussian densities satisfyρtγd=P−log⁡t​\(πγd\)\\frac\{\\rho\_\{t\}\}\{\\gamma\_\{d\}\}=P\_\{\-\\log t\}\\left\(\\frac\{\\pi\}\{\\gamma\_\{d\}\}\\right\)\.

Thus, \([26](https://arxiv.org/html/2610.10875#S2.E26)\) gives a path from Gaussian noise to the target\. Its deterministic probability flow velocity \(see e\.g\.\[[1](https://arxiv.org/html/2610.10875#bib.bib53), Theorem 6\]\) is given by

vt​\(x\)=1t​\(∇log⁡ρtγd\)​\(x\),t∈\(0,1\]\.v\_\{t\}\(x\)=\\frac\{1\}\{t\}\\left\(\\nabla\\log\\frac\{\\rho\_\{t\}\}\{\\gamma\_\{d\}\}\\right\)\(x\),\\quad t\\in\(0,1\]\.\(27\)Indeed, one may check

∂tρt\+∇⋅\(ρt​vt\)=0\.\\partial\_\{t\}\\rho\_\{t\}\+\\nabla\\cdot\(\\rho\_\{t\}v\_\{t\}\)=0\.\(28\)The same velocity can be written in denoising form: Tweedie’s formula \(see equation \(5\.12\) of\[[2](https://arxiv.org/html/2610.10875#bib.bib82)\]\) gives

vt​\(x\)=𝔼⁡\[X1∣Xt=x\]−t​x1−t2\.v\_\{t\}\(x\)=\\frac\{\\mathbb\{E\}\[X\_\{1\}\\mid X\_\{t\}=x\]\-tx\}\{1\-t^\{2\}\}\.\(29\)Hence, the reverse OU probability flow moves each noised sample toward its posterior estimate of the clean sample\. These marginals\(ρt\)t∈\[0,1\]\(\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}are the same as those of the Anderson reverse SDE of a time\-change of the OU process, see Section 5\.1 of\[[1](https://arxiv.org/html/2610.10875#bib.bib53)\]\.

Given target samplesX1∼πX\_\{1\}\\sim\\pi, we can sample the interpolation \([25](https://arxiv.org/html/2610.10875#S2.E25)\) directly\. Moreover, if

Ut:=X1−t​Xt1−t2U\_\{t\}:=\\frac\{X\_\{1\}\-tX\_\{t\}\}\{1\-t^\{2\}\}\(30\)then \([29](https://arxiv.org/html/2610.10875#S2.E29)\) is equivalently

vt​\(x\)=𝔼⁡\[Ut∣Xt=x\]\.v\_\{t\}\(x\)=\\mathbb\{E\}\[U\_\{t\}\\mid X\_\{t\}=x\]\.\(31\)Hencevtv\_\{t\}is the population minimizer of the square\-loss regression problem

infv𝔼⁡\[‖v⁡\(t,Xt\)−Ut‖2\]\\inf\_\{v\}\\mathbb\{E\}\\left\[\\\|v\(t,X\_\{t\}\)\-U\_\{t\}\\\|^\{2\}\\right\]\(32\)which can be learned using the finite dataset of target samples\. Equivalently one may regress the Gaussian noise or score\. This gives the usualscore\-matching objectiveused in \(denoising\) diffusion models, see\[[72](https://arxiv.org/html/2610.10875#bib.bib58),[43](https://arxiv.org/html/2610.10875#bib.bib52),[75](https://arxiv.org/html/2610.10875#bib.bib51),[1](https://arxiv.org/html/2610.10875#bib.bib53)\]\.

Importantly, the usual score, noise\-prediction, denoising, reverse\-SDE, and probability flow parameterizations of this Gaussian denoising path are equivalent up to explicit affine transformations and time\-dependent scalar factors\. For concreteness, we work throughout with the probability flow velocity in \([27](https://arxiv.org/html/2610.10875#S2.E27)\), and we refer to the induced transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}withρ0=γd\\rho\_\{0\}=\\gamma\_\{d\}andρ1=π\\rho\_\{1\}=\\pias thereverse OU transport toπ\\pi\. It is known that the reverse OU transport is not Wasserstein\-optimal\[[77](https://arxiv.org/html/2610.10875#bib.bib64),[51](https://arxiv.org/html/2610.10875#bib.bib66)\], and in this work we will ask whether this prescribed transport is efficient in a neural network function space\.

##### Flow\-matching transport\.

More generally, one may prescribe an interpolation between the base and target distributions for which the samplewise velocity is known \(or estimated via finite\-differences\)\. IfXtX\_\{t\}denotes the interpolation andX˙t\\dot\{X\}\_\{t\}its samplewise velocity, then the marginal velocity field

vt​\(x\)=𝔼⁡\[X˙t∣Xt=x\]v\_\{t\}\(x\)=\\mathbb\{E\}\[\\dot\{X\}\_\{t\}\\mid X\_\{t\}=x\]\(33\)satisfies the continuity equation forρt=Law⁡\(Xt\)\\rho\_\{t\}=\\operatorname\{Law\}\(X\_\{t\}\)\. The fieldvtv\_\{t\}can again be learned by a square\-loss regression using samples of\(Xt,X˙t\)\(X\_\{t\},\\dot\{X\}\_\{t\}\)\. This is the basicflow\-matchingconstruction\[[54](https://arxiv.org/html/2610.10875#bib.bib40),[3](https://arxiv.org/html/2610.10875#bib.bib41),[1](https://arxiv.org/html/2610.10875#bib.bib53)\]\.

##### Wasserstein optimal transport\.

One can instead consider the transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}fromμ\\mutoπ\\pigiven by the inducedW2W\_\{2\}optimal transport, i\.e\. the minimizer of \([12](https://arxiv.org/html/2610.10875#S2.E12)\)\. While this transport has some useful properties \(particles move in straight lines, which can be helpful for discretization of the generative process\), it can be hard to directly learn from target samples since the exact Wasserstein coupling is not directly available\. Practical flow\-matching methods therefore approximate it \(for example using minibatch optimal transport\[[79](https://arxiv.org/html/2610.10875#bib.bib42)\]\), after which coupled points may be connected with linear interpolations and the drifts may be learned via flow matching\.

A common alternate dynamical approach, dubbedrectified flow\[[56](https://arxiv.org/html/2610.10875#bib.bib48)\], is to first learn a generative model from samples using something like the score\-matching objective \(as described above\)\. One then applies a rectification procedure which produces increasingly straight trajectories without increasing the Wasserstein transport cost\. In practice, this can provide useful approximations toW2W\_\{2\}optimal transports\.

## 3Barron Transport Metric

In the previous section, we framed the task of generative modeling as learning a collection of drifts\(vt\)t∈\[0,1\]\(v\_\{t\}\)\_\{t\\in\[0,1\]\}which transport a known base measureμ\\mu\(often the standard Gaussianγd\\gamma\_\{d\}\) to the target data distributionπ\\pivia the continuity equation \([7](https://arxiv.org/html/2610.10875#S2.E7)\)\. We surveyed two different notions of dynamical optimal transport \(Wasserstein and Stein\) which describe the “least complex” transport fromμ\\mutoπ\\pi, where complexity ofvtv\_\{t\}is measured inL2​\(ρt\)L^\{2\}\(\\rho\_\{t\}\)in the Wasserstein case and inℋk\\mathcal\{H\}\_\{k\}in the Stein case\.

The Wasserstein construction is natural when the goal of an efficient transport is to minimize the movement/speed of particles, and the Stein construction is natural to describe coherent transports which are efficient with respect to a fixed set of features encoded in the kernel\. However, since we will in practice be representing and approximating the driftsvtv\_\{t\}via neural networks, it makes sense to measure a transport’s complexity using a function\-space norm adapted to neural\-network approximation\.

This will be the subject of this section: in short, we define a metric on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)which asks “what is the minimal neural network size required to transport one measure to another”? To do so, we will leverage the notion of Barron spaces\[[31](https://arxiv.org/html/2610.10875#bib.bib60)\]\.

### 3\.1Abstract definition

Before doing so, we first show that the dynamical length minimizations of \([13](https://arxiv.org/html/2610.10875#S2.E13)\) and \([18](https://arxiv.org/html/2610.10875#S2.E18)\) can be generalized using any Banach norm \(satisfying a mild regularity condition, see Assumption[1](https://arxiv.org/html/2610.10875#Thmassumption1)\) to yield an extended metric on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\):

###### Definition 2\.

Letℬ\\mathcal\{B\}be a Banach space of functions fromℝd→ℝd\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}which satisfies Assumption[1](https://arxiv.org/html/2610.10875#Thmassumption1)\. For everyμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), define the quantity

Dℬ​\(μ,ν\):=infμ→ν∫01‖vt‖ℬ​𝑑t,D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\):=\\inf\_\{\\mu\\to\\nu\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt,\(34\)where the infimum is over\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}transportingμ\\mutoν\\nuvia the continuity equation \([7](https://arxiv.org/html/2610.10875#S2.E7)\)\.

In this work we will only prove the simplest properties of these metrics:

###### Proposition 1\.

\(𝒫⁡\(ℝd\),Dℬ\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{B\}\}\)is an extended metric space \(i\.e\. the metric may take the value\+∞\+\\infty\), and convergence inDℬD\_\{\\mathcal\{B\}\}implies narrow convergence of measures\.

###### Proof\.

See Section[A\.1](https://arxiv.org/html/2610.10875#A1.SS1)\. ∎

In the companion paper\[[28](https://arxiv.org/html/2610.10875#bib.bib1)\]we study a more general setting where the tangent normℬ\\mathcal\{B\}is allowed to depend onρt\\rho\_\{t\}, in which case we give a thorough treatment of absolutely continuous curves, geodesics, and metric gradient flows using the methods of Ambrosio\-Gigli\-Savaré\[[5](https://arxiv.org/html/2610.10875#bib.bib69)\]\.

### 3\.2Barron spaces and their associated transport metrics

Letℬϕ\\mathcal\{B\}\_\{\\phi\}be a representational Barron space, i\.e\.

∥w∥ℬϕ=inf\{∥μ∥TV:w\(x\)=∫Θϕθ\(x\)dμ\(θ\)∀x∈ℝd\}\\\|w\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\inf\\left\\\{\\\|\\mu\\\|\_\{\\operatorname\{TV\}\}:\\;w\(x\)=\\int\_\{\\Theta\}\\phi\_\{\\theta\}\(x\)d\\mu\(\\theta\)\\quad\\forall x\\in\{\\mathbb\{R\}\}^\{d\}\\right\\\}\(35\)for some parameterizationϕθ:ℝd→ℝ\\phi\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}andℝd\{\\mathbb\{R\}\}^\{d\}\-valued signed measuresμ\\muon a parameter spaceΘ\\Theta\. This is a Banach space of functions fromℝd→ℝd\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}, and if\{ϕθ\}θ∈Θ\\\{\\phi\_\{\\theta\}\\\}\_\{\\theta\\in\\Theta\}is uniformly bounded on compact sets then it satisfies Assumption[1](https://arxiv.org/html/2610.10875#Thmassumption1)\. We defineDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}according to Definition[2](https://arxiv.org/html/2610.10875#Thmdefinition2), i\.e\.

Dℬϕ​\(μ,ν\):=infμ→ν∫01‖vt‖ℬϕ​𝑑t\(∀μ,ν∈𝒫⁡\(ℝd\)\),D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\):=\\inf\_\{\\mu\\to\\nu\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt\\quad\(\\forall\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\),\(36\)and we refer to this as theBarron transport metric\.

We will be focused on parameterizations of the formϕθ​\(x\)=ϕ⁡\(⟨a,x⟩\+b\)\\phi\_\{\\theta\}\(x\)=\\phi\(\\left\\langle a,x\\right\\rangle\+b\)for a continuousϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}and parametersθ=\(a,b\)\\theta=\(a,b\)which describe two\-layer neural networks\.999Technically, to represent the concrete Barron spaces of Definitions[3](https://arxiv.org/html/2610.10875#Thmdefinition3)and[4](https://arxiv.org/html/2610.10875#Thmdefinition4)in the form of \([35](https://arxiv.org/html/2610.10875#S3.E35)\) we would takeϕθ​\(x\)=ϕ⁡\(⟨a,x⟩\+b\)/\(1\+‖a‖\+\|b\|\)\\phi\_\{\\theta\}\(x\)=\\phi\(\\left\\langle a,x\\right\\rangle\+b\)/\(1\+\\\|a\\\|\+\|b\|\)whenϕ\\phiis bounded andϕθ​\(x\)=ϕ⁡\(⟨a,x⟩\+b\)/\(‖a‖\+\|b\|\)s\\phi\_\{\\theta\}\(x\)=\\phi\(\\left\\langle a,x\\right\\rangle\+b\)/\(\\\|a\\\|\+\|b\|\)^\{s\}whenϕ\\phiis positivelyss\-homogeneous\.

###### Assumptionϕ\\phi\.

Throughout the rest of this paper, we assume the activationϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}is continuous and either bounded or positivelyss\-homogeneous for somes\>0s\>0\(i\.e\.ϕ⁡\(λ​z\)=λs​ϕ​\(z\)\\phi\(\\lambda z\)=\\lambda^\{s\}\\phi\(z\)for allλ≥0\\lambda\\geq 0\)\.

We will use the following two formulations of Barron space, depending on whetherϕ\\phiis bounded \(e\.g\.tanh\\tanh,sin\\sin, etc\.\) or positivelyss\-homogeneous \(e\.g\. a monomial, a power of the ReLU or leaky ReLU, etc\.\)\. In the bounded case we may need to additionally constrain the size of the neurons‖a‖\+\|b\|≤R\\\|a\\\|\+\|b\|\\leq R, so we allow our definition to accommodate this; unless otherwise stated we always takeR=∞R=\\infty\.

###### Definition 3\(Barron space, bounded\)\.

Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be continuous and bounded, and letR∈\(0,∞\]R\\in\(0,\\infty\]\. We denote byℬϕR\\mathcal\{B\}\_\{\\phi\}^\{R\}the Banach space of vector fieldsv:ℝd→ℝdv:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}with norm

‖v‖ℬϕR=infμ\{∫ℝd×ℝ\(1\+‖a‖\+\|b\|\)​d​\|μ\|​\(a,b\):v⁡\(⋅\)=∫ℝd×ℝϕ⁡\(⟨a,⋅⟩\+b\)​𝑑μ​\(a,b\)\}\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}^\{R\}\}=\\inf\_\{\\mu\}\\left\\\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}\}\(1\+\\\|a\\\|\+\|b\|\)d\|\\mu\|\(a,b\):\\;v\(\\cdot\)=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}\}\\phi\(\\left\\langle a,\\cdot\\right\\rangle\+b\)d\\mu\(a,b\)\\right\\\}\(39\)whereμ\\muranges over signed vector\-valued measures onℝd×ℝ\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}s\.t\.‖a‖\+\|b\|≤R\\\|a\\\|\+\|b\|\\leq Rforμ\\mu\-a\.e\.\(a,b\)\(a,b\)\. IfR=∞R=\\inftywe denote this space byℬϕ\\mathcal\{B\}\_\{\\phi\}for simplicity\.

In particular, the finite two\-layer neural network withμj,aj∈ℝd\\mu\_\{j\},a\_\{j\}\\in\{\\mathbb\{R\}\}^\{d\}andbj∈ℝb\_\{j\}\\in\{\\mathbb\{R\}\}given by

v⁡\(⋅\)=∑j=1mμj​ϕ​\(⟨aj,⋅⟩\+bj\)v\(\\cdot\)=\\sum\_\{j=1\}^\{m\}\\mu\_\{j\}\\phi\(\\left\\langle a\_\{j\},\\cdot\\right\\rangle\+b\_\{j\}\)\(40\)has Barron norm‖v‖ℬϕ≤∑j‖μj‖​\(1\+‖aj‖\+\|bj\|\)\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\sum\_\{j\}\\\|\\mu\_\{j\}\\\|\(1\+\\\|a\_\{j\}\\\|\+\|b\_\{j\}\|\)\.

###### Definition 4\(Barron space, homogeneous\)\.

Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be continuous and positivelyss\-homogeneous for somes\>0s\>0\. We denote byℬϕ\\mathcal\{B\}\_\{\\phi\}the Banach space of vector fieldsv:ℝd→ℝdv:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}with norm

‖v‖ℬϕ=infμ\{∫ℝd×ℝ\(‖a‖\+\|b\|\)s​d​\|μ\|​\(a,b\):v⁡\(⋅\)=∫ℝd×ℝϕ⁡\(⟨a,⋅⟩\+b\)​𝑑μ​\(a,b\)\}\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\inf\_\{\\mu\}\\left\\\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}\}\(\\\|a\\\|\+\|b\|\)^\{s\}d\|\\mu\|\(a,b\):\\;v\(\\cdot\)=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}\}\\phi\(\\left\\langle a,\\cdot\\right\\rangle\+b\)d\\mu\(a,b\)\\right\\\}\(41\)whereμ\\muranges over signed vector\-valued measures onℝd×ℝ\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}\.

In particular, the finite two\-layer neural network withμj,aj∈ℝd\\mu\_\{j\},a\_\{j\}\\in\{\\mathbb\{R\}\}^\{d\}andbj∈ℝb\_\{j\}\\in\{\\mathbb\{R\}\}given by

v⁡\(⋅\)=∑j=1mμj​ϕ​\(⟨aj,⋅⟩\+bj\)v\(\\cdot\)=\\sum\_\{j=1\}^\{m\}\\mu\_\{j\}\\phi\(\\left\\langle a\_\{j\},\\cdot\\right\\rangle\+b\_\{j\}\)\(42\)has Barron norm‖v‖ℬϕ≤∑j‖μj‖​\(‖aj‖\+\|bj\|\)s\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\sum\_\{j\}\\\|\\mu\_\{j\}\\\|\(\\\|a\_\{j\}\\\|\+\|b\_\{j\}\|\)^\{s\}\.

Note that ifϕ⁡\(z\)=z\\phi\(z\)=zis the identity then everyv∈ℬϕv\\in\\mathcal\{B\}\_\{\\phi\}must be affinev⁡\(x\)=M​x\+cv\(x\)=Mx\+c, in which case‖v‖ℬϕ=‖M‖∗\+‖c‖\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\\|M\\\|\_\{\*\}\+\\\|c\\\|with∥⋅∥∗\\\|\\cdot\\\|\_\{\*\}the nuclear norm\.

The Barron norm is an atomic variation norm whose atoms are normalized individual neurons; finite two\-layer networks correspond to finite atomic decompositions, while the Barron space permits arbitrary mixtures\. Unlike an RKHS with a fixed feature distribution, the Barron norm allows the relevant features to be chosen adaptively\. As we will see, thisDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}metric can \(1\) help us understand the dynamical transport of measures using two\-layer mean\-field neural network drifts and \(2\) produce interesting adaptive sampling algorithms\. The first point is the main subject of this paper, while the second point will be the subject of the companion paper\[[28](https://arxiv.org/html/2610.10875#bib.bib1)\]\.

In generalℬϕ\\mathcal\{B\}\_\{\\phi\}is a Banach rather than Hilbert space, so unlike the Stein geometry there is no canonical Riemannian structure which we use here\. Despite this, the spaces\(𝒫⁡\(ℝd\),Dℬϕ\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{B\}\_\{\\phi\}\}\)enjoy many dynamical properties akin to those ofW2W\_\{2\}andDℋkD\_\{\\mathcal\{H\}\_\{k\}\}; the companion paper develops the metric geometry of these spaces in greater detail\.

### 3\.3Interpretation as a ResNet norm

The Barron transport metric has a direct interpretation in terms of deep residual networks \(ResNets\)\. Recall that a ResNet withLLresidual blocks can be written in the form

hℓ\+1​\(x\)=hℓ​\(x\)\+1L​vℓ​\(hℓ​\(x\)\),\(ℓ∈\{0,…,L−1\}\),h^\{\\ell\+1\}\(x\)=h^\{\\ell\}\(x\)\+\\frac\{1\}\{L\}v\_\{\\ell\}\(h^\{\\ell\}\(x\)\),\\quad\(\\ell\\in\\\{0,\\ldots,L\-1\\\}\),\(43\)where eachvℓ:ℝd→ℝdv\_\{\\ell\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}is represented by a neural network\. AsL→∞L\\to\\infty, under some regularity this yields the Euler discretization of theneural ODE

X˙t=vt​\(Xt\),\(t∈\[0,1\]\)\\dot\{X\}\_\{t\}=v\_\{t\}\(X\_\{t\}\),\\quad\(t\\in\[0,1\]\)\(44\)under the relabelingt↔ℓ/Lt\\leftrightarrow\\ell/L\. This continuous\-depth interpretation of ResNets as neural ODEs is standard, see e\.g\.\[[31](https://arxiv.org/html/2610.10875#bib.bib60)\]and the quantitativeL→∞L\\to\\inftyconvergence theory of Chizat\[[23](https://arxiv.org/html/2610.10875#bib.bib43)\]\.

Suppose that the residual blocksvℓv\_\{\\ell\}are two\-layer networks with activationϕ\\phi\. A natural intrinsic complexity measure for the ResNet is

1L​∑ℓ=0L−1‖vℓ‖ℬϕ\.\\frac\{1\}\{L\}\\sum\_\{\\ell=0\}^\{L\-1\}\\\|v\_\{\\ell\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\.\(45\)In the continuous\-depth limit this becomes

∫01‖vt‖ℬϕ​𝑑t\.\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt\.\(46\)Thus, the quantity appearing in the definition ofDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}is exactly the continuous\-depth Barron ResNet norm of the corresponding neural ODE\.

This also has a direct interpretation in terms of network parameters\. For example, in the case of a bounded activation, if

vℓ​\(x\)=∑j=1mℓμj,ℓ​ϕ​\(⟨aj,ℓ,x⟩\+bj,ℓ\),v\_\{\\ell\}\(x\)=\\sum\_\{j=1\}^\{m\_\{\\ell\}\}\\mu\_\{j,\\ell\}\\phi\(\\left\\langle a\_\{j,\\ell\},x\\right\\rangle\+b\_\{j,\\ell\}\),\(47\)is a finite\-width two\-layer network then Definition[3](https://arxiv.org/html/2610.10875#Thmdefinition3)gives

‖vℓ‖ℬϕ≤∑j=1mℓ‖μj,ℓ‖​\(1\+‖aj,ℓ‖\+\|bj,ℓ\|\)\.\\\|v\_\{\\ell\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\sum\_\{j=1\}^\{m\_\{\\ell\}\}\\\|\\mu\_\{j,\\ell\}\\\|\\left\(1\+\\\|a\_\{j,\\ell\}\\\|\+\|b\_\{j,\\ell\}\|\\right\)\.\(48\)Hence the ResNet norm is bounded by the layer\-averaged parameter norm of the residual blocks\. The analogous statement holds for positivelyss\-homogeneous activations using Definition[4](https://arxiv.org/html/2610.10875#Thmdefinition4)\. Consequently, the quantity

Dℬϕ​\(μ,ν\)=infμ→ν∫01‖vt‖ℬϕ​𝑑tD\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\)=\\inf\_\{\\mu\\to\\nu\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt\(49\)can be interpreted as the minimum continuous\-depth ResNet norm required to transportμ\\mutoν\\nuusing a neural ODE with two\-layer blocks\.

This is the main reason we use the Barron transport geometry to study generative modeling:Dℬϕ​\(μ,ν\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\)measures the complexity of the simplest neural dynamics connecting the two distributions, rather than the amount by which particles move or the complexity of the dynamics in a fixed feature space\. In particular, the resultingDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}balls aroundμ\\muconsist of those measures which admit a neural network generative model of bounded size in the following sense:

- •Lower bounds onDℬϕ​\(μ,ν\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\)correspond to lower bounds on the average Barron norm of each layer of any deep ResNet/neural ODE which transportsμ\\mutoν\\nu\.
- •Conversely, upper bounds onDℬϕ​\(μ,ν\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\)prove the existence of a deep ResNet/neural ODE transportingμ\\mutoν\\nuwhose layer\-averaged Barron norm is upper bounded\.

Thus, bounds onDℬϕ​\(μ,ν\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\)provide a quantitative understanding of the approximation complexity for dynamical transport fromμ\\mutoν\\nuusing neural ODEs\.

###### Question 1\.

One could also define a finite\-depth distance viaDℬϕ\(L\)​\(μ,ν\):=inf1L​∑ℓ=1L‖vℓ‖ℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}^\{\(L\)\}\(\\mu,\\nu\):=\\inf\\frac\{1\}\{L\}\\sum\_\{\\ell=1\}^\{L\}\\\|v\_\{\\ell\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}, where the infimum is over blocks\(vℓ\)ℓ=1L\(v\_\{\\ell\}\)\_\{\\ell=1\}^\{L\}s\.t\.F\#​μ=νF\_\{\\\#\}\\mu=\\nuwithF:=\(𝟙\+1L​vL\)∘…∘\(𝟙\+1L​v1\)F:=\(\{\\mathbbm\{1\}\}\+\\frac\{1\}\{L\}v\_\{L\}\)\\circ\\ldots\\circ\(\{\\mathbbm\{1\}\}\+\\frac\{1\}\{L\}v\_\{1\}\)\. Quantifying the convergenceDℬϕ\(L\)→DℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}^\{\(L\)\}\\to D\_\{\\mathcal\{B\}\_\{\\phi\}\}asL→∞L\\to\\inftyis useful to transfer ourDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}results to real\-world ResNets, which we leave for future work\.

Our perspective of a continuous\-depth ResNet norm via a geometry on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)with tangent norm∥⋅∥ℬϕ\\\|\\cdot\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}is complementary to the approaches of\[[31](https://arxiv.org/html/2610.10875#bib.bib60),[46](https://arxiv.org/html/2610.10875#bib.bib37)\]\. These works study the approximation properties of deep ResNets by carefully designing function spaces that see the whole ResNet as a single function, which requires some way to capture the sensitivity of function outputs w\.r\.t\. changes at intermediate layers\. In\[[31](https://arxiv.org/html/2610.10875#bib.bib60)\]this is needed to control theL2L^\{2\}approximation error of finite\-width ResNets, since error accumulates across depth in a way that leads to a path norm/multiplicative norm\. In\[[46](https://arxiv.org/html/2610.10875#bib.bib37)\]this is used to describe circuit complexity of a function in terms of its ResNet representations\. By contrast, theDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}metric leverages the energy\-minimization perspective of dynamical optimal transport with fixed endpoints to encode depth, even though the cost itself is local and additive across depth\.

### 3\.4Comparisons with other notions of distance

As we increase the regularity ofϕ\\phiwe also increase the regularity of functions inℬϕ\\mathcal\{B\}\_\{\\phi\}, which can cause the distanceDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}to increase\. A useful first question to ask is how strong theDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}distance is under regularity assumptions onϕ\\phi, since this can help contextualize any quantitative results we obtain regardingDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}\. We note here a particularly clean lower bound on the Barron transport metricDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}in terms of either Wasserstein distance or KL divergence: the latter is interesting since it forgets all geometric information about a target distribution\.

###### Proposition 2\(Wasserstein and KL lower bounds\)\.

Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfy Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)and suppose that‖ϕ‖∞≤M<∞\\\|\\phi\\\|\_\{\\infty\}\\leq M<\\infty\. Then, there is the Wasserstein lower bound

Dℬϕ​\(μ,ν\)≥W∞​\(μ,ν\)M\(∀μ,ν∈𝒫⁡\(ℝd\)\)\.D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\)\\geq\\frac\{W\_\{\\infty\}\(\\mu,\\nu\)\}\{M\}\\quad\(\\forall\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\)\.\(50\)If we assume further thatϕ\\phiisMM\-Lipschitz then there is the KL lower bound

Dℬϕ​\(γd,π\)≥14​M⋅min⁡\{KL\(π∥γd\)d,KL\(π∥γd\)\}\(∀π∈𝒫⁡\(ℝd\)\)\.D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\\geq\\frac\{1\}\{4M\}\\cdot\\min\\left\\\{\\frac\{\\operatorname\{KL\}\(\\pi\\\|\\gamma\_\{d\}\)\}\{\\sqrt\{d\}\},\\;\\sqrt\{\\operatorname\{KL\}\(\\pi\\\|\\gamma\_\{d\}\)\}\\right\\\}\\quad\(\\forall\\pi\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\)\.\(51\)

###### Proof\.

See Section[A\.2](https://arxiv.org/html/2610.10875#A1.SS2)\. Similar lower bounds hold for anyDℬD\_\{\\mathcal\{B\}\}metric withℬ↪L∞\\mathcal\{B\}\\hookrightarrow L^\{\\infty\}andℬ↪W1,∞\\mathcal\{B\}\\hookrightarrow W^\{1,\\infty\}, respectively, as well as withγd\\gamma\_\{d\}replaced by any log\-smooth measure with sufficient tail control, see Lemma[3](https://arxiv.org/html/2610.10875#Thmlemma3)\. ∎

This result helps us understand the foliation structure of\(𝒫⁡\(ℝd\),Dℬϕ\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{B\}\_\{\\phi\}\}\)in the case of boundedϕ\\phi, since any measuresμ,ν\\mu,\\nuwhich have infiniteW∞W\_\{\\infty\}distance \(or KL under Lipschitzness and sufficient regularity of eitherμ\\muorν\\nu\) also have infiniteDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}distance\. For example,Dℬϕ​\(μ,ν\)=\+∞D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\)=\+\\inftyifμ\\muhas compact support butν\\nudoes not, ifμ\\muandν\\nuare mutually singular, ifμ\\muandν\\nuhave different tail scales at infinity, etc\.

### 3\.5Optimal transport distances between Gaussians

In some of our main results, we will be comparing the SteinDℋkD\_\{\\mathcal\{H\}\_\{k\}\}and BarronDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}metrics to gain some intuition about the complexity of generative modeling\. In order to perform an initial comparison between these distances, we will inspect the case where both endpoints are Gaussians\. If we restrict to bilinear kernelskkand linear activationsϕ\\phi, the spacesℋk\\mathcal\{H\}\_\{k\}andℬϕ\\mathcal\{B\}\_\{\\phi\}consist of affine drifts and so a Gaussian initial measure remains Gaussian along every finite\-length admissible flow\. This reduces the infinite\-dimensional minimization in \([36](https://arxiv.org/html/2610.10875#S3.E36)\) to a tractable finite\-dimensional matrix control problem, for which we can get matching upper and lower bounds\. For further simplicity we restrict one of the endpoints to be the standard Gaussianγd\\gamma\_\{d\}, leading to:

###### Proposition 3\.

Letπ:=𝒩⁡\(m,Σ\)\\pi:=\\mathcal\{N\}\(m,\\Sigma\)withΣ≻0\\Sigma\\succ 0\. Then, the following hold up to universal constants:

- •\(Wasserstein\)\.W2​\(γd,π\)=‖m‖2\+‖Σ1/2−𝟙d‖F2W\_\{2\}\(\\gamma\_\{d\},\\pi\)=\\sqrt\{\\\|m\\\|^\{2\}\+\\\|\\Sigma^\{1/2\}\-\{\\mathbbm\{1\}\}\_\{d\}\\\|\_\{F\}^\{2\}\}\.
- •\(Stein\)\.Dℋk​\(γd,π\)≍log⁡\(1\+‖m‖\)\+‖log⁡Σ‖FD\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\\asymp\\log\(1\+\\\|m\\\|\)\+\\\|\\log\\Sigma\\\|\_\{F\}fork⁡\(x,y\)=\(1\+⟨x,y⟩\)​𝟙dk\(x,y\)=\(1\+\\left\\langle x,y\\right\\rangle\)\{\\mathbbm\{1\}\}\_\{d\}\.
- •\(Barron\)\.Dℬϕ​\(γd,π\)≍log⁡\(1\+‖m‖\)\+‖log⁡Σ‖∗D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\\asymp\\log\(1\+\\\|m\\\|\)\+\\\|\\log\\Sigma\\\|\_\{\*\}forϕ⁡\(z\)=z\\phi\(z\)=z\.

Here,∥⋅∥F\\\|\\cdot\\\|\_\{F\}is the Frobenius norm and∥⋅∥∗\\\|\\cdot\\\|\_\{\*\}is the nuclear norm\.

###### Proof\.

This follows from Lemmas[4](https://arxiv.org/html/2610.10875#Thmlemma4),[5](https://arxiv.org/html/2610.10875#Thmlemma5), and[6](https://arxiv.org/html/2610.10875#Thmlemma6)\. Interestingly, the transports realizing the upper bounds for linear Stein and linear Barron are almost the same\. They move the mean and covariance in separate motions: the mean movement uses a spatial preconditioning bym​m⊤mm^\{\\top\}to get the logarithmic dependence, while the log\-covariance linearly interpolates from00tolog⁡Σ\\log\\Sigma\. See Remark[12](https://arxiv.org/html/2610.10875#Thmremark12)for more details\. ∎

Compared to the Wasserstein distance, the linear Stein and Barron distances have improved dependence onmm\(andΣ\\SigmaifΣ⪰𝟙d\\Sigma\\succeq\{\\mathbbm\{1\}\}\_\{d\}\)\. On the other hand, if there is a covariance eigenvalueσ2≪1\\sigma^\{2\}\\ll 1then linear Stein and Barron blow up likelog⁡1σ2\\log\\frac\{1\}\{\\sigma^\{2\}\}while Wasserstein stays bounded\. Furthermore, we haveDℋk≲DℬϕD\_\{\\mathcal\{H\}\_\{k\}\}\\lesssim D\_\{\\mathcal\{B\}\_\{\\phi\}\}with qualitatively similar behaviors\.

## 4Approximation Theory for Generative Paths

Having introduced the Barron transport metricDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}, we now use it to study two approximation questions for generative modeling\. In practice, we chooseμ\\muto be easy to sample from \(such as the standard Gaussianγd\\gamma\_\{d\}\) andν\\nua potentially complicated data distribution \(which we will denoteπ\\pi\)\.

First, we compare the reverse OU transport to the Barron optimal transport in order to study the role of depth in denoising models\. In particular, in Section[4\.1](https://arxiv.org/html/2610.10875#S4.SS1)we ask when the reverse OU transport is optimal \(up to constants\) w\.r\.t\. the Barron energy\. We answer in the affirmative for centered Gaussians and single\-index tilts, but in the negative for noncentered Gaussians and single\-index flows \(Proposition[4](https://arxiv.org/html/2610.10875#Thmproposition4)and Example[1](https://arxiv.org/html/2610.10875#Thmexample1)\)\. To clarify the situation, we construct a measureπ\\pi\(a pushforward ofγd\\gamma\_\{d\}by a deep narrow ResNet\) for which the denoising path has exponentialℬϕ\\mathcal\{B\}\_\{\\phi\}path norm, but which a simple time\-varying single\-index flow can easily generate and soDℬϕ​\(γd,π\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)is small \(Theorem[3](https://arxiv.org/html/2610.10875#Thmtheorem3)\)\. This suggests that the reverse OU transport is suboptimal for modeling neural network pushforwards of the Gaussian, but the obstruction we prove is in terms of exact representation and only for small OU times\. To finish, we strengthen this result by constructing another neural network pushforwardπ\\pifor which any shallow neural network thatε\\varepsilon\-approximates the reverse OU drifts inL2L^\{2\}must have widthdpoly⁡\(1/ε\)d^\{\\operatorname\{poly\}\(1/\\varepsilon\)\}\(Theorem[4](https://arxiv.org/html/2610.10875#Thmtheorem4)\)\. Hence, any shallow score networks which achieve small score\-matching loss on this data must have large width, irrespective of how they are learned\. This implies that deeper drift networks are needed to enable efficient generative modeling for these data distributions\.

In Section[4\.2](https://arxiv.org/html/2610.10875#S4.SS2)we instead aim to quantify the benefit of feature learning by comparing the BarronDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}distance to the SteinDℋkD\_\{\\mathcal\{H\}\_\{k\}\}distance, whereℋk\\mathcal\{H\}\_\{k\}is the RKHS of a rotation\-equivariant kernelkk\. We already saw that for Gaussian\-to\-Gaussian transports with linearϕ\\phiand bilinearkkthey behave similarly \(Proposition[3](https://arxiv.org/html/2610.10875#Thmproposition3)\)\. However, there are also the classical feature learning separation results of e\.g\.\[[7](https://arxiv.org/html/2610.10875#bib.bib46),[38](https://arxiv.org/html/2610.10875#bib.bib65)\]which exhibit functions that are easy to represent in Barron spaces \(i\.e\. using shallow neural networks\) but are difficult to represent in rotation\-invariant RKHS \(i\.e\. using random feature models\)\. In the generative modeling setting, we show that perturbatively\-small single\-index pushforwards/flows can yield measures in high dimension for which the ratioDℋk/Dℬϕ\{D\_\{\\mathcal\{H\}\_\{k\}\}\}/\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\}becomes very large \(Examples[2](https://arxiv.org/html/2610.10875#Thmexample2)and[3](https://arxiv.org/html/2610.10875#Thmexample3)\), but that this separation breaks down in the nonperturbative regime since deep RKHS transports can model single\-index flows \(Example[4](https://arxiv.org/html/2610.10875#Thmexample4)\)\. In other words, the classical shallow feature learning separation holds locally but fails once depth is introduced\. These preliminary results suggest that RKHS\-based generative models can be fairly expressive, and that a separation with neural network generative models \(if it exists\) might instead come from the learning procedure rather than approximation capabilities\. For now the comparison between RKHS\-based and neural network\-based generative modeling remains poorly understood, and we consider it important to compare these paradigms from both approximation and computational perspectives\.

To summarize, our results point toward two themes in the approximation theory of generative modeling/dynamical measure transport:

- •The reverse OU/denoising transport \(which is common in practice\) cannot always be efficiently modeled using shallow neural networks, even when there exist other transports which can be\. In such settings, non\-denoising depth is required\.
- •The Barron optimal transport is an efficient alternative when data has preferred directions/low intrinsic dimension, and an RKHS optimal transport which leverages depth may enjoy similar benefit\.

Taken together, these results demonstrate that the Barron/RKHS dynamical optimal transports have desirable properties for generative modeling that the current denoising approach can fail to capture\. Our study of the approximation question therefore motivates a generative modeling approach based on algorithms which \(approximately\) solve theDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}orDℋkD\_\{\\mathcal\{H\}\_\{k\}\}optimal transport problems; in Section[5\.1](https://arxiv.org/html/2610.10875#S5.SS1)we give some heuristic algorithmic ideas, but the computational question is wide open and left for future work\.

### 4\.1Reverse OU \(sub\)optimality

As mentioned in Section[2\.3](https://arxiv.org/html/2610.10875#S2.SS3), the reverse OU transport fromγd\\gamma\_\{d\}toπ\\piis a common design choice for generative models\. A natural question to ask is whether this transport is efficient to implement with shallow neural networks, or whether there can exist more economical alternative transports using smaller shallow networks\. Another way to phrase this question is to ask whether the reverse OU transport is optimal in terms of theDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}energy \([36](https://arxiv.org/html/2610.10875#S3.E36)\), perhaps up to multiplicative constants\. To help us approach this question, we define the following quantity:

###### Lemma 1\(Condition for reverse OU \(sub\)optimality\)\.

Letπ∈𝒫⁡\(ℝd\)\\pi\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)andℬ\\mathcal\{B\}a Banach space of functions fromℝd→ℝd\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}, and define

ℰℬ​\(π\):=∫0∞‖∇log⁡Pτ∗​πγd‖ℬ​𝑑τ,\\mathcal\{E\}\_\{\\mathcal\{B\}\}\(\\pi\):=\\int\_\{0\}^\{\\infty\}\\left\\\|\\nabla\\log\\frac\{P\_\{\\tau\}^\{\*\}\\pi\}\{\\gamma\_\{d\}\}\\right\\\|\_\{\\mathcal\{B\}\}d\\tau,\(52\)where\(Pτ\)τ\>0\(P\_\{\\tau\}\)\_\{\\tau\>0\}is the OU semigroup\. If\(vt\)t∈\[0,1\]\(v\_\{t\}\)\_\{t\\in\[0,1\]\}is any time\-reparameterization111111By this we mean thatvt=α˙​\(t\)α⁡\(t\)∇logρα⁡\(t\)γdv\_\{t\}=\\frac\{\\dot\{\\alpha\}\(t\)\}\{\\alpha\(t\)\}\\nabla\\log\\frac\{\\rho\_\{\\alpha\(t\)\}\}\{\\gamma\_\{d\}\}, whereρs=P−log⁡s∗​π\\rho\_\{s\}=P\_\{\-\\log s\}^\{\*\}\\piandα:\[0,1\]→\[0,1\]\\alpha:\[0,1\]\\to\[0,1\]is any absolutely continuous, strictly increasing function withα⁡\(0\)=0\\alpha\(0\)=0,α⁡\(1\)=1\\alpha\(1\)=1\.of the reverse OU transport fromγd\\gamma\_\{d\}toπ\\pi, then

ℰℬ​\(π\)=∫01‖vt‖ℬ​𝑑t≥Dℬ​\(γd,π\)\.\\mathcal\{E\}\_\{\\mathcal\{B\}\}\(\\pi\)=\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt\\geq D\_\{\\mathcal\{B\}\}\(\\gamma\_\{d\},\\pi\)\.\(53\)

###### Proof\.

This follows from reparameterizing from transport timet∈\[0,1\]t\\in\[0,1\]to OU timeτ∈\[0,∞\)\\tau\\in\[0,\\infty\)\. ∎

NaturallyDℬ​\(γd,π\)≤ℰℬ​\(π\)D\_\{\\mathcal\{B\}\}\(\\gamma\_\{d\},\\pi\)\\leq\\mathcal\{E\}\_\{\\mathcal\{B\}\}\(\\pi\), and they are comparable if and only if the reverse OU/denoising transport fromγd\\gamma\_\{d\}toπ\\pihas complexity \(w\.r\.t\.ℬ\\mathcal\{B\}\) comparable to the optimal one\. Hence, we can use this quantityℰℬ​\(π\)\\mathcal\{E\}\_\{\\mathcal\{B\}\}\(\\pi\)to understand the \(sub\)optimality of the reverse OU generative model for various choices of target measureπ∈𝒫⁡\(ℝd\)\\pi\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\.

As before, we will consider different target measuresπ\\piof increasing complexity\. We start with the linear Gaussian\-to\-Gaussian case, followed by simple nonlinear single\-index perturbations of the Gaussian, after which we look at nonperturbative pushforwards of the Gaussian by deep narrow ResNets\. In each setting, we will focus on the Barron caseℬ=ℬϕ\\mathcal\{B\}=\\mathcal\{B\}\_\{\\phi\}in order to draw conclusions about the implementation of score\-matching diffusion/denoising models using shallow neural network drifts\.121212Note that since the Barron norm is an infimum over random\-feature RKHS norms \(see Remark[1](https://arxiv.org/html/2610.10875#Thmremark1)\), many of these results will naturally extend to the corresponding kernel settings\.

We will see that for very simple data \(pushforwards of the Gaussian by a small teacher neural network\) the reverse OU transport can fail to see the correct relationship between coordinates, as the relevant structure is not visible by varying the noise level\. In such cases, it instead uses brute force and requires very wide networks to represent or approximate the OU scores, even though there exists a small alternative generative model \(namely, the teacher network\)\. These constructions provide a good toy model of a type of obstruction which denoising models with shallow neural network drifts can encounter on simple data\. Our hope is that they can \(1\) help us understand the structures in real\-world data which influence the role of depth in denoising models and \(2\) guide the design of other alternative generative modeling paradigms; we leave these investigations to future work\.

#### 4\.1\.1Gaussian targets

We again start by inspecting the setting of linear drift function spaces andπ\\pia Gaussian:

###### Proposition 4\(Reverse OU, Gaussian\-to\-Gaussian\)\.

Letπ:=𝒩⁡\(m,Σ\)\\pi:=\\mathcal\{N\}\(m,\\Sigma\)withΣ≻0\\Sigma\\succ 0\. Then, the following hold up to universal constants:

- •\(Wasserstein\)\.ℰW2​\(π\)≍W2​\(γd,π\)\\mathcal\{E\}\_\{W\_\{2\}\}\(\\pi\)\\asymp W\_\{2\}\(\\gamma\_\{d\},\\pi\)\.131313Here, we denoteℰW2​\(π\):=∫0∞‖∇log⁡Pτ∗​πγd‖L2​\(Pτ∗​π\)​𝑑τ\\mathcal\{E\}\_\{W\_\{2\}\}\(\\pi\):=\\int\_\{0\}^\{\\infty\}\\\|\\nabla\\log\\frac\{P\_\{\\tau\}^\{\*\}\\pi\}\{\\gamma\_\{d\}\}\\\|\_\{L^\{2\}\(P\_\{\\tau\}^\{\*\}\\pi\)\}d\\tauto be the unsquared Benamou\-Brenier kinetic energy of the reverse OU transport\.
- •\(Stein\)\.Ifk⁡\(x,y\)=\(1\+⟨x,y⟩\)​𝟙dk\(x,y\)=\(1\+\\left\\langle x,y\\right\\rangle\)\{\\mathbbm\{1\}\}\_\{d\}then ℰℋk​\(π\)≳‖m‖max⁡\{1,‖Σ‖op\}\+12​‖log⁡Σ‖F\.\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\\left\(\\pi\\right\)\\gtrsim\\frac\{\\\|m\\\|\}\{\\max\\\{1,\\;\\\|\\Sigma\\\|\_\{\\operatorname\{op\}\}\\\}\}\+\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{F\}\.\(54\)However, ifm=0m=0andΣ=diag⁡\(ed/1,…,ed/d\)\\Sigma=\\operatorname\{diag\}\(e^\{d/\\sqrt\{1\}\},\\ldots,e^\{d/\\sqrt\{d\}\}\)thenℰℋk​\(π\)≳log⁡d⋅Dℋk​\(γd,π\)\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\\left\(\\pi\\right\)\\gtrsim\\sqrt\{\\log d\}\\cdot D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\.
- •\(Barron\)\.Ifϕ⁡\(z\)=z\\phi\(z\)=zthen ℰℬϕ​\(π\)≳‖m‖max⁡\{1,‖Σ‖op\}\+12​‖log⁡Σ‖∗\.\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\\left\(\\pi\\right\)\\gtrsim\\frac\{\\\|m\\\|\}\{\\max\\\{1,\\;\\\|\\Sigma\\\|\_\{\\operatorname\{op\}\}\\\}\}\+\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}\.\(55\)Furthermore, ifm=0m=0thenℰℬϕ​\(π\)≍Dℬϕ​\(γd,π\)\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\\left\(\\pi\\right\)\\asymp D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\.

###### Proof\.

This follows from Lemma[7](https://arxiv.org/html/2610.10875#Thmlemma7)and Example[5](https://arxiv.org/html/2610.10875#Thmexample5)\. ∎

Hence, for Gaussians the reverse OU transport is always optimal up to constants w\.r\.t\.W2W\_\{2\}, and for centered Gaussians it is also optimal up to constants w\.r\.t\. linear Barron \(but curiously not linear Stein\)\. However, when moving the mean this transport can be arbitrarily suboptimal w\.r\.t\. linear Stein/Barron \(recall from Proposition[3](https://arxiv.org/html/2610.10875#Thmproposition3)that the optimal transport should have logarithmic dependence on‖m‖\\\|m\\\|\)\.

#### 4\.1\.2Perturbative single\-index flows and tilts

As our simplest nonlinear examples, we consider targetsπ\\piwhich are perturbations of the standard Gaussian by single\-index functions\. We consider two types of perturbations: flowing by a single\-index vector field and tilting by a single\-index scalar function\.

###### Example 1\(Reverse OU, perturbative single\-index flows and tilts\)\.

Fix a globally\-Lipschitzϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}which is bounded below, and fix anyσ∈\(0,∞\)\\sigma\\in\(0,\\infty\)andu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}s\.t\.\|⟨u,θ⟩\|<1\|\\left\\langle u,\\theta\\right\\rangle\|<1\. Forε\>0\\varepsilon\>0define the flow perturbation

πεflow:=Law⁡\(Xε\),X˙t=σ​u​ϕ​\(⟨Xt,θ⟩\),X0∼γd\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}:=\\operatorname\{Law\}\(X\_\{\\varepsilon\}\),\\quad\\dot\{X\}\_\{t\}=\\sigma u\\phi\(\\left\\langle X\_\{t\},\\theta\\right\\rangle\),\\quad X\_\{0\}\\sim\\gamma\_\{d\}\(56\)and forε\>0\\varepsilon\>0small enough that1\+ε​hϕ≥01\+\\varepsilon h\_\{\\phi\}\\geq 0define the tilt perturbation

πεtilt:=\(1\+ε​hϕ\)​γd,hϕ​\(x\):=ϕ⁡\(⟨x,θ⟩\)−𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ⁡\(ξ\)\]\.\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}:=\(1\+\\varepsilon h\_\{\\phi\}\)\\gamma\_\{d\},\\quad h\_\{\\phi\}\(x\):=\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\-\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\phi\(\\xi\)\]\.\(57\)Ifϕ∈\{ReLU,tanh,sin\}\\phi\\in\\\{\\operatorname\{ReLU\},\\tanh,\\sin\\\}then it holds that

lim infε↓0ℰℬϕ​\(πεflow\)Dℬϕ​\(γd,πεflow\)=\+∞,lim supε↓0ℰℬϕ​\(πεtilt\)Dℬϕ​\(γd,πεtilt\)≤23\.\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\\left\(\{\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}\}\\right\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}\)\}=\+\\infty,\\quad\\limsup\_\{\\varepsilon\\downarrow 0\}\\frac\{\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\\left\(\{\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}\}\\right\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}\)\}\\leq 23\.\(58\)

###### Proof\.

See Section[A\.6](https://arxiv.org/html/2610.10875#A1.SS6)\. Morally, the measureπεflow\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}from \([56](https://arxiv.org/html/2610.10875#S4.E56)\) has a reverse OU transport whose drifts have a term likeθ⁡⟨⋅,u⟩​ϕ′​\(⟨⋅,θ⟩\)\\theta\\left\\langle\\cdot,u\\right\\rangle\\phi^\{\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\); this is a product of two neurons, and it is not representable by two\-layer networks for the activations we consider\. This obstruction does not occur in the case of tilts, in which the reverse OU transport has a term likeθ​ϕ′​\(⟨⋅,θ⟩\)\\theta\\phi^\{\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\. ∎

Hence, the reverse OU transport is Barron\-suboptimal for small single\-index flows/pushforwards ofγd\\gamma\_\{d\}, whereas it is Barron\-optimal \(up to constants\) for small single\-index tilts\. This example hints that in nonlinear settings the reverse OU transport can sometimes be suboptimal w\.r\.t\. theL1​\(\[0,1\]→ℬϕ\)L^\{1\}\(\[0,1\]\\to\\mathcal\{B\}\_\{\\phi\}\)energy, but the mechanism identified here is perturbative and somewhat opaque\.

#### 4\.1\.3Intuition: triangular shear measures

In the following subsections, we aim to clarify the situation with nonperturbative constructions of target distributionsπ∈𝒫⁡\(ℝd\)\\pi\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)for which the reverse OU drifts \(i\.e\. the time\-dependent scores along the OU process, see Section[2\.3](https://arxiv.org/html/2610.10875#S2.SS3)\) are difficult to model with shallow neural networks despite the fact thatDℬϕ​\(γd,π\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)is small \(i\.e\. there are alternative generative models forπ\\piwith simple shallow drifts\)\. The high\-level intuition is that there can exist compositional/hierarchical structure in the data which is not visible by varying the noise level, in which case the reverse OU transport fails to leverage it\. We will show that this arises naturally for data which is drawn from a pushforward of the Gaussian by a deep neural network\.

A good toy example to understand this mechanism is a simple construction in whichπ∈𝒫⁡\(ℝd\)\\pi\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)is a pushforward of the Gaussianγd\\gamma\_\{d\}by a triangular shear mapT:ℝd→ℝdT:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}of the form

T⁡\(z,u\):=\(z,u\+Ψ⁡\(z\)\),\(z∈ℝd−1,u∈ℝ\)T\(z,u\):=\(z,u\+\\Psi\(z\)\),\\quad\(z\\in\{\\mathbb\{R\}\}^\{d\-1\},u\\in\{\\mathbb\{R\}\}\)\(59\)for some functionΨ:ℝd−1→ℝ\\Psi:\{\\mathbb\{R\}\}^\{d\-1\}\\to\{\\mathbb\{R\}\}\. Note thatπΨ:=T\#​γd∈𝒫⁡\(ℝd\)\\pi^\{\\Psi\}:=T\_\{\\\#\}\\gamma\_\{d\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)has Gaussian density

πΨγd​\(z,u\)=exp⁡\(u​Ψ​\(z\)−Ψ​\(z\)2/2\),\\frac\{\\pi^\{\\Psi\}\}\{\\gamma\_\{d\}\}\(z,u\)=\\exp\(u\\Psi\(z\)\-\\Psi\(z\)^\{2\}/2\),\(60\)and so the last coordinate of the Gaussian score∇log⁡πΨγd\\nabla\\log\\frac\{\\pi^\{\\Psi\}\}\{\\gamma\_\{d\}\}is equal toΨ\\Psi\. Hence, the reverse OU drifts must modelΨ\\Psi\(or an OU\-smoothed version of it\) and so they inheritΨ\\Psi’s complexity\.

Heuristically, the benefit of dynamical measure transport is that it allows us to avoid learning an entire endpoint map by instead learning drifts and using composition to increase the modeling capacity; this is analogous to how deep neural networks often have improved efficiency over shallow ones\[[34](https://arxiv.org/html/2610.10875#bib.bib63),[78](https://arxiv.org/html/2610.10875#bib.bib74)\]\. The beauty of the reverse OU/denoising generative model is that it reduces the task from learning the entire transport to regression of the drifts\[[18](https://arxiv.org/html/2610.10875#bib.bib72)\], which in our analogy may be viewed as a form of layerwise supervision of a deep model\. However, the observation of the previous paragraph implies that for pushforwards by triangular shears like \([59](https://arxiv.org/html/2610.10875#S4.E59)\), the scores are as complex as the endpoint map itself\. In our analogy, this means that the reverse OU supervision destroys the natural modeling benefit of depth which dynamical transport usually provides\.

Of course, this intuition we are describing need not hold at positive OU timesτ\>0\\tau\>0, since the actual drifts involve the scores of the OU\-smoothed targetPτ∗​πΨP\_\{\\tau\}^\{\*\}\\pi^\{\\Psi\}and may be simpler\. Thus, the technical part of our task is to show that the complexity ofΨ\\Psipersists for a non\-negligible amount of OU time\. In Section[A\.7](https://arxiv.org/html/2610.10875#A1.SS7)we identify two different mechanisms for this:

- •\(Smoothness\)\.By some elementary OU stability estimates, we show in Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)that large derivatives ofΨ\\Psiyield large derivatives of the drifts for positive OU times\. Although Gaussian smoothing removes oscillations quickly, we find that the size of the derivative outweighs the rate of smoothing for derivatives of order at least 3\. This third derivative obstruction becomes a Barron norm obstruction for activations with bounded third derivative, in which caseℰℬϕ​\(πΨ\)\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi^\{\\Psi\}\)can become large \(see Corollary[2](https://arxiv.org/html/2610.10875#Thmcorollary2)\)\. We emphasize that this lower bound is only encountered when exactly representing the full OU path and need not show up in practice\. For example, ifΨ\\Psihas oscillations of frequencyexp⁡\(d\)\\exp\(d\)then the third derivative of the score is of sizeexp⁡\(3​d\)\\exp\(3d\)for OU timesτ≲exp⁡\(−2​d\)\\tau\\lesssim\\exp\(\-2d\); with any early\-stopping which is≳exp⁡\(−2​d\)\\gtrsim\\exp\(\-2d\)this obstruction is avoided\. Moreover, by Girsanov one need only learnL2L^\{2\}\-approximate drifts for accurate sampling\[[18](https://arxiv.org/html/2610.10875#bib.bib72)\], which means that an approximate generative model does not need to match the third derivatives of the score\. Hence, while this mechanism suggests reverse OU suboptimality for some triangular shear measures, it leaves a lot to be desired\.
- •\(Hermite rank\)\.In\[[13](https://arxiv.org/html/2610.10875#bib.bib22), Theorem 4\.18\], it was proved thatL2L^\{2\}approximation of a function using two\-layer neural networks encounters a quantitative width lower bound in terms of the spectral tails of the function’s Hermite coefficient tensors; this is because shallow networks of small width have low\-rank Hermite coefficients\. Importantly, since the Wiener chaos decomposition diagonalizes the OU operator it is easy to track the evolution along the OU flow\. Thus, forΨ\\Psiwhich have heavy Hermite spectral tails across Hermite degrees, we can show that a low score matching loss with shallow network drifts implies large width \(see Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9)\)\. Unlike the smoothness obstruction, this lower bound has more practical significance since it \(1\) describes the hardness ofL2L^\{2\}\-approximate score modeling rather than exact representation, \(2\) directly targets the width, and \(3\) persists longer under OU flow\.

Both smoothness and Hermite rank can be viewed as notions of depth complexity ofΨ\\Psi, since they obstruct modeling by shallow networks but not deep ones \(depth can exponentially increase both spatial irregularity and rank, but width cannot\)\. By showing that the depth complexity ofΨ\\Psi\(which is fully present in the scores atτ=0\\tau=0\) propagates to the time\-dependent scores along the OU flow, we make precise the intuition that “modeling the reverse OU drifts is as hard as modeling the full endpoint map for these pushforward distributions”\. Hence, ifΨ\\Psihas large depth complexity then the reverse OU transport fails to decompose it into simpler pieces\.

It is easy to construct examples for which an alternative transport succeeds in this goal\. The obvious choice is to takeΨ\\Psiitself to be a deep neural network, in which case the triangular shear mapTTfrom \([59](https://arxiv.org/html/2610.10875#S4.E59)\) is a deep ResNet and an alternative transport with shallow drifts may be built from the layers of the network itself\. In the next two subsections, we will design such ResNet pushforwards to exhibit large third derivatives \(by using deep networks to compose many nonlinearities\) and high Hermite rank \(by using wide three\-layer networks and activations with heavy Hermite tails\), which together yield Theorem[1](https://arxiv.org/html/2610.10875#Thmtheorem1)\.

#### 4\.1\.4ResNet pushforwards: an exponential representation lower bound

We first instantiate a triangular shear measureπΨ\\pi^\{\\Psi\}whose generating mapΨ\\Psihas large third derivatives but admits a compositional decomposition into shallow pieces \(and therefore a Barron\-efficient alternative transport\)\. This is accomplished by chaining together many nonlinearities across the layers of a deep network, which can amplify the nonlinearity to create derivatives that are exponential in the depth\. Hence, we can construct a pushforward ofγd\\gamma\_\{d\}by a ResNet of depthΘ⁡\(d\)\\Theta\(d\)with two\-layer blocks of widthO⁡\(1\)O\(1\)for which the following holds:

###### Theorem 3\(Shallow representation lower bound, ResNet pushforward\)\.

Fixd≥4d\\geq 4and letϕ⁡\(t\):=max⁡\{0,t\}3\\phi\(t\):=\\max\\\{0,t\\\}^\{3\}be theReLU3\\operatorname\{ReLU\}^\{3\}activation\. Then, there exists a ResNetTθ:ℝd→ℝdT\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}of depthΘ⁡\(d\)\\Theta\(d\)and widthO⁡\(1\)O\(1\)\(which uses a smoothedReLU3\\operatorname\{ReLU\}^\{3\}activation\) s\.t\. ifπθ:=\(Tθ\)\#​γd∈𝒫⁡\(ℝd\)\\pi^\{\\theta\}:=\(T\_\{\\theta\}\)\_\{\\\#\}\\gamma\_\{d\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)then

ℰℬϕ​\(πθ\)≳2d,Dℬϕ​\(γd,πθ\)≲d\.\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi^\{\\theta\}\)\\gtrsim 2^\{d\},\\quad D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi^\{\\theta\}\)\\lesssim d\.\(61\)Furthermore, the OU scoressτθ:=∇log⁡Pτ∗​πθs\_\{\\tau\}^\{\\theta\}:=\\nabla\\log P\_\{\\tau\}^\{\*\}\\pi^\{\\theta\}andbτθ:=∇log⁡Pτ∗​πθγdb\_\{\\tau\}^\{\\theta\}:=\\nabla\\log\\frac\{P\_\{\\tau\}^\{\*\}\\pi^\{\\theta\}\}\{\\gamma\_\{d\}\}are inℬϕ\\mathcal\{B\}\_\{\\phi\}for everyτ∈\[0,∞\)\\tau\\in\[0,\\infty\)\.

###### Proof\.

The precise statement is given in Proposition[11](https://arxiv.org/html/2610.10875#Thmproposition11)\. The hard instance constructs a triangular shearTθ​\(z,u\)=\(z,ψ⁡\(u\)\+Ψθ​\(z\)\)T\_\{\\theta\}\(z,u\)=\(z,\\psi\(u\)\+\\Psi\_\{\\theta\}\(z\)\), whereΨθ:ℝd−1→ℝ\\Psi\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\-1\}\\to\{\\mathbb\{R\}\}is a deep narrow network of the form

Ψθ​\(z\)=ω⁡\(zd−1\+Λω​ω​\(zd−2\+Λω​ω​\(⋯\+Λω​ω​\(z1\)\)\)\)\\Psi\_\{\\theta\}\(z\)=\\omega\\left\(z\_\{d\-1\}\+\\Lambda\_\{\\omega\}\\omega\\left\(z\_\{d\-2\}\+\\Lambda\_\{\\omega\}\\omega\\left\(\\cdots\+\\Lambda\_\{\\omega\}\\omega\\left\(z\_\{1\}\\right\)\\right\)\\right\)\\right\)\(62\)for a scalar functionω∈Cb5​\(ℝ\)\\omega\\in C^\{5\}\_\{b\}\(\{\\mathbb\{R\}\}\)withΛω\\Lambda\_\{\\omega\}a constant depending only onω\\omega\. Alternatively, this can be viewed as a scalar readout \(from thedt​hd^\{th\}coordinate\) of a ResNet where each hidden layerℓ\\ellhas width 1, input featureeℓe\_\{\\ell\}, output featureeℓ\+1e\_\{\\ell\+1\}, and activationΛω​ω\\Lambda\_\{\\omega\}\\omega\.

The lower bound shows the third derivative ofΨθ\\Psi\_\{\\theta\}is large and uses some OU stability estimates to imply the same forsτθ,bτθs\_\{\\tau\}^\{\\theta\},b\_\{\\tau\}^\{\\theta\}for smallτ\\tau\(see Corollary[2](https://arxiv.org/html/2610.10875#Thmcorollary2)\)\. The upper bound simply computes \([62](https://arxiv.org/html/2610.10875#S4.E62)\) using an ODE with single\-index drifts \(namely, the layers ofΨθ\\Psi\_\{\\theta\}\), writes the result to theuucoordinate, and then reverses the computation to returnzzto its initial state\. The functionψ:ℝ→ℝ\\psi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}is designed in order to have the right polynomial tails, which is needed to ensure the OU scores are inℬReLU3\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{3\}\}\. The teacher activation implementingTθT\_\{\\theta\}is a smoothedReLU3\\operatorname\{ReLU\}^\{3\}function, which is chosen so that bothω\\omegaandψ\\psican be represented usingO⁡\(1\)O\(1\)width\. ∎

In words, the distributionπθ\\pi^\{\\theta\}encodes a sequence of very simple coordinatewise movements, and performing them in the correct order yields a generative model forπθ\\pi^\{\\theta\}which can be implemented with small shallow neural network drifts\. However, the reverse OU transport is unaware of this structure and attempts to adjust the complicated coordinate directly usingΨθ\\Psi\_\{\\theta\}, whose Barron norm increases exponentially with depth \(the composition of Barron functions can cause an exponential increase in Barron norm, see\[[34](https://arxiv.org/html/2610.10875#bib.bib63),[78](https://arxiv.org/html/2610.10875#bib.bib74),[31](https://arxiv.org/html/2610.10875#bib.bib60)\]\)\. Since the scores are Barron for all OU times, this exponential obstruction is quantitative rather than a statement that the scores escape the Barron space near the data side\.

![Refer to caption](https://arxiv.org/html/2610.10875v1/resnet_vs_reverse_ou_probability_flow.png)Figure 2:Marginal densities of 2\-dimensional slices \(first and last coordinates\) ofρt\\rho\_\{t\}, along with histograms showing which coordinates are in motion, for the ResNet and reverse OU transports fromγd\\gamma\_\{d\}to the lower bound instanceπθ\\pi^\{\\theta\}constructed in Theorem[3](https://arxiv.org/html/2610.10875#Thmtheorem3)\(withω=tanh\\omega=\\tanh,ψ=Id\\psi=\\operatorname\{Id\}, andd=14d=14for simplicity\)\. Starting fromt=1t=1and looking backward, we see that the OU process adjusts thexdx\_\{d\}coordinate immediately, whereas the single\-index ODE waits untilt≈0\.5t\\approx 0\.5to do so \(since it must first adjust all other coordinates\)\.In Figure[2](https://arxiv.org/html/2610.10875#S4.F2)we visually compare the reverse OU transport forπθ\\pi^\{\\theta\}to the more efficient generative model \(which is obtained by computingΨθ\\Psi\_\{\\theta\}using single\-index drifts, writing the output to the final coordinate, and reversing the computation to return the firstd−1d\-1coordinates to their initial state\)\. Our earlier intuition is displayed nicely: the ResNet transport greedily and efficiently moves one coordinate at a time and only touches the final coordinate once the computation is done, whereas the reverse OU transport attempts to move the hard coordinate instantly\.

#### 4\.1\.5ResNet pushforwards: an approximation lower bound

The prior subsection showed that for a certain deep ResNet pushforwardπθ\\pi^\{\\theta\}, exact representation of the reverse OU/denoising generative model using shallow neural network drifts with polynomially\-bounded weights requires exponential average width\. That being said, we emphasize that the exponential obstruction we identify only occurs at exponentially\-small OU times and relies on large third derivatives: a practical pipeline using early\-stopping andL2L^\{2\}\-accurate score estimates can yield a generative model which approximatesπθ\\pi^\{\\theta\}without encountering the lower bound of Theorem[3](https://arxiv.org/html/2610.10875#Thmtheorem3)\. In this subsection we will remove both of these caveats with a construction for which any fixed\-accuracy shallow drift approximation to the reverse OU generative model requires large width in high dimension\.

More precisely, we instantiate a triangular shear measureπΨ\\pi^\{\\Psi\}whose generating mapΨ\\Psihas high Hermite rank and therefore cannot be efficiently approximated by shallow models\. This is achieved by combining many independent features in a nonlinear way; the independent features create high rank, and a heavy Hermite tail of the nonlinear readout amplifies this effect by placing them in high Hermite degrees\. We do so using a pushforward ofγd\\gamma\_\{d\}by a ResNet of depthΘ⁡\(d\)\\Theta\(d\)with two\-layer blocks of widthO⁡\(1\)O\(1\), or alternatively by a three\-layer network of widthΘ⁡\(d\)\\Theta\(d\)\.

For notation, ifπθ=\(Tθ\)\#​γd∈𝒫⁡\(ℝd\)\\pi^\{\\theta\}=\(T\_\{\\theta\}\)\_\{\\\#\}\\gamma\_\{d\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)is the Gaussian pushforward by a ResNetTθ:ℝd→ℝdT\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}with parametersθ\\thetathen we denote byρτθ:=Pτ∗​πθ\\rho\_\{\\tau\}^\{\\theta\}:=P\_\{\\tau\}^\{\*\}\\pi^\{\\theta\}the OU flow forτ∈\[0,∞\)\\tau\\in\[0,\\infty\)\. We writesτθ:=∇log⁡ρτθs\_\{\\tau\}^\{\\theta\}:=\\nabla\\log\\rho\_\{\\tau\}^\{\\theta\}andbτθ:=∇log⁡ρτθγdb\_\{\\tau\}^\{\\theta\}:=\\nabla\\log\\frac\{\\rho\_\{\\tau\}^\{\\theta\}\}\{\\gamma\_\{d\}\}for the OU scores w\.r\.t\. Lebesgue and the Gaussian, respectively\.

###### Theorem 4\(Shallow approximation lower bound, ResNet pushforward\)\.

Fixε\>0\\varepsilon\>0smaller than a universal constant\. Then, there existsd0=d0​\(ε\)d\_\{0\}=d\_\{0\}\(\\varepsilon\)s\.t\. for eachd≥d0d\\geq d\_\{0\}there exists a ResNetTθ:ℝd→ℝdT\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}of depthΘ⁡\(d\)\\Theta\(d\)and widthO⁡\(1\)O\(1\)with ReLU activations with the following property for a universalc\>0c\>0:

If0≤τ≲ε4/30\\leq\\tau\\lesssim\\varepsilon^\{4/3\}ands^τ:ℝd→ℝd\\hat\{s\}\_\{\\tau\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}is any two\-layer neural network of widthmm\(i\.e\. a sum ofmmridge functions\) for which

‖sτθ−s^τ‖L2​\(ρτθ,ℝd\)≤ε,\\\|s\_\{\\tau\}^\{\\theta\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\theta\};\{\\mathbb\{R\}\}^\{d\}\)\}\\leq\\varepsilon,\(63\)then it holds thatm≳dcε−4/3m\\gtrsim d^\{c\\varepsilon^\{\-4/3\}\}and∥s^τ∥ℬϕ≳εdc2ε−4/3\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\gtrsim\\varepsilon d^\{\\frac\{c\}\{2\}\\varepsilon^\{\-4/3\}\}for allϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)\. The same statements hold withsτθs\_\{\\tau\}^\{\\theta\}replaced bybτθb\_\{\\tau\}^\{\\theta\}, and the implied constants are independent ofε,d,τ,s^τ,m\\varepsilon,d,\\tau,\\hat\{s\}\_\{\\tau\},m\.

###### Proof\.

The hard instance constructs a triangular shearTθ​\(z,u\)=\(z,u\+Ψθ​\(z\)\)T\_\{\\theta\}\(z,u\)=\(z,u\+\\Psi\_\{\\theta\}\(z\)\), whereΨθ:ℝd−1→ℝ\\Psi\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\-1\}\\to\{\\mathbb\{R\}\}is a wide three\-layer network of the form

Ψθ​\(z\)=χ⁡\(zd−1\+1d−2​∑ℓ=1d−2g⁡\(zℓ\)\)\\Psi\_\{\\theta\}\(z\)=\\chi\\left\(z\_\{d\-1\}\+\\frac\{1\}\{\\sqrt\{d\-2\}\}\\sum\_\{\\ell=1\}^\{d\-2\}g\(z\_\{\\ell\}\)\\right\)\(64\)for scalar functionsχ,g:ℝ→ℝ\\chi,g:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}\. Note thatTθT\_\{\\theta\}can be also expressed as a deep narrow ResNet, see Section[A\.8\.2](https://arxiv.org/html/2610.10875#A1.SS8.SSS2)\. In this ResNet each layer is a shear \(i\.e\. the input and output features are orthogonal\) and so it can be implemented by a ReLU flow; since the weights are1d−2\\frac\{1\}\{\\sqrt\{d\-2\}\}, this means thatDℬReLU​\(γd,πθ\)≲dD\_\{\\mathcal\{B\}\_\{\\operatorname\{ReLU\}\}\}\(\\gamma\_\{d\},\\pi^\{\\theta\}\)\\lesssim\\sqrt\{d\}\.

By Lemma[31](https://arxiv.org/html/2610.10875#Thmlemma31), withχ⁡\(t\)=max⁡\{0,t\}−max⁡\{0,t−1\}\\chi\(t\)=\\max\\\{0,t\\\}\-\\max\\\{0,t\-1\\\}and any centered non\-affine Lipschitzggwe get𝒱χ,g\(K,2K\)≳K−3/2\\mathcal\{V\}\_\{\\chi,g\}\(K,2K\)\\gtrsim K^\{\-3/2\}with constants depending only ongg, and henceK\(ε\)≳ε−4/3K\(\\varepsilon\)\\gtrsim\\varepsilon^\{\-4/3\}\. We chooseg⁡\(t\)=max⁡\{0,t\}−12​πg\(t\)=\\max\\\{0,t\\\}\-\\frac\{1\}\{\\sqrt\{2\\pi\}\}for simplicity\. The claim follows by applying Corollary[3](https://arxiv.org/html/2610.10875#Thmcorollary3)withL=d−2L=d\-2\. ∎

In words, Theorem[4](https://arxiv.org/html/2610.10875#Thmtheorem4)says that for a ReLU ResNet pushforwardπθ\\pi^\{\\theta\}of this sort and any fixedε\>0\\varepsilon\>0, any shallow network whichε\\varepsilon\-approximates the true score at any OU timeτ≲ε4/3\\tau\\lesssim\\varepsilon^\{4/3\}must have width≳dcε−4/3\\gtrsim d^\{c\\varepsilon^\{\-4/3\}\}\. Hence, a practical denoising sampling setup \(using early\-stopping andL2L^\{2\}\-accurate scores\) which aims to achieve small error must model scores in this window and will therefore encounter our lower bound\. More precisely, standard sampling guarantees for practical denoising models \(e\.g\.\[[18](https://arxiv.org/html/2610.10875#bib.bib72),[16](https://arxiv.org/html/2610.10875#bib.bib73)\]\) decompose the end\-to\-end sampling error along the lines of

sampling error≲early\-stopping error\+L2score\-estimation error\+discretization error\.\\text\{sampling error \}\\lesssim\\text\{ early\-stopping error\}\+\\text\{$L^\{2\}$ score\-estimation error\}\+\\text\{discretization error\}\.\(65\)In Lemma[32](https://arxiv.org/html/2610.10875#Thmlemma32), we show that early\-stopping at OU timeτ≍ε4/3\\tau\\asymp\\varepsilon^\{4/3\}incursKL\\sqrt\{\\operatorname\{KL\}\}error≳ε\\gtrsim\\varepsilonand bothW1W\_\{1\}andTV\\operatorname\{TV\}error≳ε4/3\\gtrsim\\varepsilon^\{4/3\}for the hard instance of Theorem[4](https://arxiv.org/html/2610.10875#Thmtheorem4)\. Our lower bound therefore implies that a denoising setup using shallow network drifts which gets the first two terms of \([65](https://arxiv.org/html/2610.10875#S4.E65)\) to≲δ\\lesssim\\deltarequires width≳dpoly⁡\(1/δ\)\\gtrsim d^\{\\operatorname\{poly\}\(1/\\delta\)\}\. This is a more substantial hardness result than that of Theorem[3](https://arxiv.org/html/2610.10875#Thmtheorem3), since it implies that the practical denoising score\-matching procedure will fail on these measures when using shallow drift networks of insufficient width\.

Theorem[4](https://arxiv.org/html/2610.10875#Thmtheorem4)is also different from the prior score\-matching hardness works of\[[73](https://arxiv.org/html/2610.10875#bib.bib17),[81](https://arxiv.org/html/2610.10875#bib.bib18)\], since the obstruction does not come from computational/cryptographic difficulty nor spatial regularity but rather from the approximation capabilities of shallow neural networks\. Instead of revealing an unavoidable issue that score\-matching encounters, this construction highlights an approximation\-theoretic separation between modeling scores with shallow and deep drift networks\. In Section[5\.2](https://arxiv.org/html/2610.10875#S5.SS2), we will discuss some conclusions and intuitions regarding the design of generative models which can be drawn from this result\.

We finish up by showing a simpler integrated form of the lower bound, which is the one quoted in the informal statement of Theorem[1](https://arxiv.org/html/2610.10875#Thmtheorem1)\.

###### Corollary 1\(Integrated shallow approximation lower bound\)\.

FixT\>0T\>0and0<ε<ε0​\(T\)0<\\varepsilon<\\varepsilon\_\{0\}\(T\)\. Then, for eachd≥d0​\(ε\)d\\geq d\_\{0\}\(\\varepsilon\)there exists a ResNetTθ:ℝd→ℝdT\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}of depthΘ⁡\(d\)\\Theta\(d\)and widthO⁡\(1\)O\(1\)with ReLU activations with the following property for a universalc\>0c\>0:

If\(s^τ\)τ∈\[0,T\]\(\\hat\{s\}\_\{\\tau\}\)\_\{\\tau\\in\[0,T\]\}is a measurable family of two\-layer neural networks of widthm⁡\(τ\)m\(\\tau\)for which

∫0T‖sτθ−s^τ‖L2​\(ρτθ,ℝd\)2​𝑑τ≤ε2,\\int\_\{0\}^\{T\}\\\|s\_\{\\tau\}^\{\\theta\}\-\\hat\{s\}\_\{\\tau\}\\\|^\{2\}\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\theta\};\{\\mathbb\{R\}\}^\{d\}\)\}d\\tau\\leq\\varepsilon^\{2\},\(66\)theness​supτ∈\[0,T\]m\(τ\)≳dcε−4/5\\esssup\_\{\\tau\\in\[0,T\]\}m\(\\tau\)\\gtrsim d^\{c\\varepsilon^\{\-4/5\}\}and for allϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)we have

∫0T∥s^τ∥ℬϕdτ≳ε7/5dc2ε−4/5and∫0T∥s^τ∥ℬϕ2dτ≳ε2dcε−4/5\.\\int\_\{0\}^\{T\}\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}d\\tau\\gtrsim\\varepsilon^\{7/5\}d^\{\\frac\{c\}\{2\}\\varepsilon^\{\-4/5\}\}\\quad\\text\{and\}\\quad\\int\_\{0\}^\{T\}\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}^\{2\}d\\tau\\gtrsim\\varepsilon^\{2\}d^\{\{c\}\\varepsilon^\{\-4/5\}\}\.\(67\)The same statements hold withsτθs\_\{\\tau\}^\{\\theta\}replaced bybτθb\_\{\\tau\}^\{\\theta\}, and the implied constants are independent ofT,ε,d,\(s^τ\)τ∈\[0,T\]T,\\varepsilon,d,\(\\hat\{s\}\_\{\\tau\}\)\_\{\\tau\\in\[0,T\]\}\.

###### Proof\.

We leta\>0a\>0be the constant s\.t\. Theorem[4](https://arxiv.org/html/2610.10875#Thmtheorem4)holds forτ∈\[0,a​δ4/3\]\\tau\\in\[0,a\\delta^\{4/3\}\]with pointwise accuracy parameterδ\\delta\. We takeδ=\(2​ε2/a\)3/10\\delta=\(2\\varepsilon^\{2\}/a\)^\{3/10\}andτδ=a​δ4/3\\tau\_\{\\delta\}=a\\delta^\{4/3\}\. This implies thatτδ​δ2=2​ε2\\tau\_\{\\delta\}\\delta^\{2\}=2\\varepsilon^\{2\}whileδ≍ε3/5\\delta\\asymp\\varepsilon^\{3/5\}andτδ≍ε4/5\\tau\_\{\\delta\}\\asymp\\varepsilon^\{4/5\}\. For sufficiently smallε\\varepsilon, Theorem[4](https://arxiv.org/html/2610.10875#Thmtheorem4)applies at accuracyδ\\deltaand timeτδ≤T\\tau\_\{\\delta\}\\leq T\. If we seteτ:=‖sτθ−s^τ‖L2​\(ρτθ,ℝd\)e\_\{\\tau\}:=\\\|s\_\{\\tau\}^\{\\theta\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\theta\};\{\\mathbb\{R\}\}^\{d\}\)\}andG:=\{τ∈\[0,τδ\]:eτ≤δ\}G:=\\\{\\tau\\in\[0,\\tau\_\{\\delta\}\]:\\;e\_\{\\tau\}\\leq\\delta\\\}then by \([66](https://arxiv.org/html/2610.10875#S4.E66)\) we have

\|\[0,τδ\]∖G\|≤1δ2​∫0τδeτ2​𝑑τ≤ε2δ2=τδ2\.\|\[0,\\tau\_\{\\delta\}\]\\setminus G\|\\leq\\frac\{1\}\{\\delta^\{2\}\}\\int\_\{0\}^\{\\tau\_\{\\delta\}\}e\_\{\\tau\}^\{2\}d\\tau\\leq\\frac\{\\varepsilon^\{2\}\}\{\\delta^\{2\}\}=\\frac\{\\tau\_\{\\delta\}\}\{2\}\.\(68\)Hence,\|G\|≥τδ/2\|G\|\\geq\\tau\_\{\\delta\}/2\. For everyτ∈G\\tau\\in Gwe havem\(τ\)≳dcδ−4/3m\(\\tau\)\\gtrsim d^\{c\\delta^\{\-4/3\}\}and∥s^τ∥ℬϕ≳δdc2δ−4/3\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\gtrsim\\delta d^\{\\frac\{c\}\{2\}\\delta^\{\-4/3\}\}, from which the claim \([67](https://arxiv.org/html/2610.10875#S4.E67)\) follows by integration overGG\. ∎

### 4\.2Feature learning separation

We now aim to study dimensional separations between theDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}and rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}distances as a way to understand the role of feature learning vs\. learning drifts in a rotation\-equivariant kernel space\. We do this first in a perturbatively\-small regime where matching upper and lower bounds are possible, after which we will present an initial result in the nonperturbative setting\. These results together yield Theorem[2](https://arxiv.org/html/2610.10875#Thmtheorem2)\.

We start in a simple setting whereπ\\piis a small pushforward of a rotation\-invariant measure by a single\-index ReLU map:

###### Example 2\(Perturbative feature learning, single\-index ReLU pushforward\)\.

Letd≥4d\\geq 4andμ∈𝒫2​\(ℝd\)\\mu\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)be rotationally\-invariant with∫ℝd\|x\|2​𝑑μ​\(x\)\>0\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\|x\|^\{2\}d\\mu\(x\)\>0\. Letϕ⁡\(z\)=max⁡\{0,z\}\\phi\(z\)=\\max\\\{0,z\\\}be the ReLU activation, fixu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}with⟨u,θ⟩≠0\\left\\langle u,\\theta\\right\\rangle\\neq 0, and forε\>0\\varepsilon\>0define

w⁡\(x\):=u​ϕ​\(⟨x,θ⟩\),νε:=\(𝟙\+ε​w\)\#​μ∈𝒫2​\(ℝd\)\.w\(x\):=u\\phi\(\\left\\langle x,\\theta\\right\\rangle\),\\quad\\nu\_\{\\varepsilon\}:=\(\{\\mathbbm\{1\}\}\+\\varepsilon w\)\_\{\\\#\}\\mu\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)\.\(69\)Then, for any rotation\-equivariantk∈𝒦rotk\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}which satisfies Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)we have

limε↓0Dℋk​\(μ,νε\)Dℬϕ​\(μ,νε\)=\+∞\.\\lim\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}=\+\\infty\.\(70\)

###### Proof\.

See Section[A\.5](https://arxiv.org/html/2610.10875#A1.SS5)\. ∎

This is essentially a restatement of the folklore fact that “a single\-index ReLU function does not lie in any rotation\-invariant RKHS ford≥4d\\geq 4” \(see\[[7](https://arxiv.org/html/2610.10875#bib.bib46), Appendix D\]\) in the setting of vector\-valued functions and rotation\-equivariant RKHS\. For completeness we provide a proof by combining a first\-order dual lower bound with a standard argument involving spherical harmonics; in the ReLU case the sum over harmonics diverges ford≥4d\\geq 4\.141414Sincex↦⟨x,u⟩\+x\\mapsto\\left\\langle x,u\\right\\rangle\_\{\+\}is inHs​\(𝕊d−1\)H^\{s\}\(\\mathbb\{S\}^\{d\-1\}\)for everys<32s<\\frac\{3\}\{2\}, it turns out that ford∈\{2,3\}d\\in\\\{2,3\\\}the ReLU neuron lies in a rotation\-invariant RKHS ford<4d<4\.

The proof method via spherical harmonics \(or Hermite decompositions when the source is Gaussian\) is general, and can be used to show matching upper and lower bounds for perturbatively\-small single\-index flows \(see Lemma[10](https://arxiv.org/html/2610.10875#Thmlemma10)for a general statement\)\. We apply it here to exhibit some quantitative separations for other activations:

###### Example 3\(Perturbative feature learning, autonomous single\-index flows\)\.

Letd≥3d\\geq 3andγd\\gamma\_\{d\}be the standard Gaussian\. Fixu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}, and forε\>0\\varepsilon\>0defineνε:=Law⁡\(Xε\)\\nu\_\{\\varepsilon\}:=\\operatorname\{Law\}\(X\_\{\\varepsilon\}\)using the flow

X˙t=u​ϕ​\(⟨Xt,θ⟩\),X0∼γd\.\\dot\{X\}\_\{t\}=u\\phi\(\\left\\langle X\_\{t\},\\theta\\right\\rangle\),\\quad X\_\{0\}\\sim\\gamma\_\{d\}\.\(71\)For an activationϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2), define

Qϕ,d:=infk∈𝒦rotk​satisfies Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)\{∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)⋅lim infε↓0Dℋk​\(γd,νε\)Dℬϕ​\(γd,νε\)\}\.Q\_\{\\phi,d\}:=\\inf\_\{\\begin\{subarray\}\{c\}k\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}\\\\ k\\text\{ satisfies Assumption \\ref\{assumption:kernel\}\}\\end\{subarray\}\}\\left\\\{\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\\right\\\}\.\(72\)Then, the following scalings hold asd→∞d\\to\\infty:

Qtanh,d≍dd4⋅\(1\+o⁡\(1\)\),Qsin,d≍ed⋅\(1\+o⁡\(1\)\)\.Q\_\{\\tanh,d\}\\asymp d^\{\\frac\{d\}\{4\}\\cdot\(1\+o\(1\)\)\},\\qquad Q\_\{\\sin,d\}\\asymp e^\{\\sqrt\{d\}\\cdot\(1\+o\(1\)\)\}\.\(73\)

###### Proof\.

See Section[A\.5](https://arxiv.org/html/2610.10875#A1.SS5)\. In Lemma[10](https://arxiv.org/html/2610.10875#Thmlemma10)we show matching upper and lower bounds \(up to constants\) for Gaussian single\-index flows in terms of the Hermite coefficients of the activation\. In Lemmas[12](https://arxiv.org/html/2610.10875#Thmlemma12)and[13](https://arxiv.org/html/2610.10875#Thmlemma13)we compute the asymptotics uniformly inu,θu,\\thetafortanh\\tanhandsin\\sin, respectively\. ∎

Examples[2](https://arxiv.org/html/2610.10875#Thmexample2)and[3](https://arxiv.org/html/2610.10875#Thmexample3)exhibit feature learning separations in a high dimensional perturbative regime for a nonsmooth ReLU transport \(where the ratio can grow unbounded in fixeddd\) and for smoothtanh\\tanhorsin\\sintransports \(where the ratio is superpolynomial indd\), respectively\. In Figure[3](https://arxiv.org/html/2610.10875#S4.F3)we plot two\-dimensional slices of some of these measures to help visualize their differences and the nature of the separation\.

![Refer to caption](https://arxiv.org/html/2610.10875v1/three_panel_distributions.png)Figure 3:Densities of 2\-dimensional slices \(theuu\-coordinate and one perpendicular touu\) for the standard Gaussianγd\\gamma\_\{d\}, the measureνε\\nu\_\{\\varepsilon\}constructed in Example[2](https://arxiv.org/html/2610.10875#Thmexample2)with a ReLU pushforward, and the measureνε\\nu\_\{\\varepsilon\}constructed in Example[3](https://arxiv.org/html/2610.10875#Thmexample3)with a smoothtanh\\tanhflow; we taked=6d=6andε=4\\varepsilon=4for visualization purposes\. The ReLU pushforward has a clear nonsmooth transition, while thetanh\\tanhflow almost looks like a Gaussian mixture\.Based on this initial perturbative result, it is tempting to conclude that “for smallε\\varepsilon, there is the classical superpolynomial\-in\-ddseparation between rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}andDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}when transporting to the endpoints ofε\\varepsilon\-length single\-index flows”\. Examples[2](https://arxiv.org/html/2610.10875#Thmexample2)and[3](https://arxiv.org/html/2610.10875#Thmexample3)indicate that this is the case whenε↓0\\varepsilon\\downarrow 0withddfixed, but it is entirely possible that this separation only kicks in whenε\\varepsilonis very tiny, sayε≲e−d\\varepsilon\\lesssim e^\{\-d\}\. After all, by takingε↓0\\varepsilon\\downarrow 0we are effectively constraining a transport to use time\-independent drifts, which does not allow an RKHS\-based transport to leverage depth\.151515For a more precise geometric perspective, by takingε↓0\\varepsilon\\downarrow 0we are measuring the norm of the original single\-index drift in the tangent spaces of the manifolds\(𝒫⁡\(ℝd\),Dℋk\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{H\}\_\{k\}\}\)and\(𝒫⁡\(ℝd\),Dℬϕ\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{B\}\_\{\\phi\}\}\)\. By doing so, we reason about the RKHS norm of the prescribed transport instead of the infimum over all transports\.

It turns out that this is exactly the case: in the macroscopic/nonperturbative setting, there exist shortcuts which can be taken to lower the total RKHS energy topoly⁡\(d\)\\operatorname\{poly\}\(d\)\. More precisely, we construct an efficient alternate flow which matches the endpoint map of a time\-invariant single\-index flow by paying a one\-time cost which ispoly⁡\(d\)\\operatorname\{poly\}\(d\)and independent ofε\\varepsilon\. We demonstrate this mechanism in the simple setting where⟨u,θ⟩=0\\left\\langle u,\\theta\\right\\rangle=0andϕ\\phiis smooth and bounded, but we expect it to apply more generally:

###### Example 4\(Nonperturbative feature learning, autonomous single\-index flows\)\.

Letd≥3d\\geq 3, fixϕ∈Cb∞​\(ℝ\)\\phi\\in C^\{\\infty\}\_\{b\}\(\{\\mathbb\{R\}\}\)andu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}with⟨u,θ⟩=0\\left\\langle u,\\theta\\right\\rangle=0, and defineνT:=Law⁡\(XT\)\\nu\_\{T\}:=\\operatorname\{Law\}\(X\_\{T\}\)for the flow \([71](https://arxiv.org/html/2610.10875#S4.E71)\)\. Then, there exists a smooth rotation\-equivariantk⋆∈𝒦rotk\_\{\\star\}\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}\(depending only ond,ϕ,Td,\\phi,T\) and a transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}fromγd\\gamma\_\{d\}toνT\\nu\_\{T\}usingpoly⁡\(d\)\\operatorname\{poly\}\(d\)\-Lipschitz drifts which satisfies

\(supt∈\[0,1\]∫ℝdtr⁡k⋆​\(x,x\)​d​ρt​\(x\)\)⋅∫01‖vt‖ℋk⋆​𝑑t≲poly⁡\(d\)\.\\left\(\\sup\_\{t\\in\[0,1\]\}\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\_\{\\star\}\(x,x\)d\\rho\_\{t\}\(x\)\}\\right\)\\cdot\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{\\star\}\}\}dt\\lesssim\\operatorname\{poly\}\(d\)\.\(74\)

###### Proof\.

The precise nonasymptotic statement is given in Proposition[7](https://arxiv.org/html/2610.10875#Thmproposition7), and it holds withγd\\gamma\_\{d\}replaced by any initialρ0∈𝒫⁡\(ℝd\)\\rho\_\{0\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)for which\(∫ℝd\|x\|d​d​ρ0​\(x\)\)1/d≲poly⁡\(d\)\(\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\|x\|^\{d\}d\\rho\_\{0\}\(x\)\)^\{1/d\}\\lesssim\\operatorname\{poly\}\(d\)\. The method is similar to how the linear Stein metric achieves logarithmic dependence on the mean in Proposition[3](https://arxiv.org/html/2610.10875#Thmproposition3): there is an initial compression/preconditioning phase in the directionuu, followed by an exponentially\-shrunk version of the original single\-index transport, followed by a decompression phase\. Since the compression/decompression phases have costs logarithmic in the compression factor, we may efficiently shrink the single\-index phase to achieve polynomial total cost\. ∎

Thus, endpoint maps of \(time\-invariant\) single\-index flows can be efficiently represented using drifts which lie in a rotation\-equivariant RKHS\. Taken literally, this result says that forε≳1poly⁡\(d\)\\varepsilon\\gtrsim\\frac\{1\}\{\\operatorname\{poly\}\(d\)\}we no longer have the superpolynomial separations between rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}andDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}that we saw in theε↓0\\varepsilon\\downarrow 0regime in Examples[2](https://arxiv.org/html/2610.10875#Thmexample2)and[3](https://arxiv.org/html/2610.10875#Thmexample3)\. More philosophically, it indicates depth can allow RKHS methods to model some functions which require feature learning in the shallow case\. In the setting of generative modeling, this result lends some credibility to the idea of learning dynamical transports by representing drifts in RKHS, see e\.g\.\[[36](https://arxiv.org/html/2610.10875#bib.bib16),[65](https://arxiv.org/html/2610.10875#bib.bib49),[59](https://arxiv.org/html/2610.10875#bib.bib5)\]\.

Our investigation only scratches the surface of the comparison between theDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}and rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}metrics: all we have shown is that the classical shallow separations persist locally in the dynamical measure transport problem, but they break down once depth and time\-dependent drifts are allowed\. We consider it an important direction for future work to investigate the efficiency of modeling more complex data using drifts which lie in rotation\-equivariant RKHS, particularly for targets which are flows/pushforwards by neural networks\.

## 5Outlook

In this paper, we introduced the Barron transport metricDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}as a technical tool to help us study approximation theory for generative modeling\. In this section, we highlight some open questions and directions for future research as well as a potential application beyond generative modeling\.

### 5\.1Algorithmic and computational ideas

Our results can be summarized as follows: representing drifts with neural networks \(which utilize feature learning\) can provably allow dynamical transports with more efficient representations, but the standard denoising score\-matching algorithm can fail to find such transports even in very natural cases\. Similarly, it is folklore that algorithms which attempt to learn the Wasserstein optimal transport with neural networks can suffer from lack of drift regularity\.161616See the counterexample of Caffarelli\[[14](https://arxiv.org/html/2610.10875#bib.bib34)\], which suffers from a discontinuous optimal transport map due to the target’s nonconvex support\. Any ODE with e\.g\. Lipschitz drifts cannot represent such a map, and so neither can any flow model with Barron drifts using a Lipschitz activation\.Hence, the current paradigms for learning dynamical generative models encounter neural network representation bottlenecks, even on easy data for which simple generative models with shallow drifts exist\.

This opens up the obvious next question: do there exist efficient algorithms which can \(approximately\) solve dynamical optimal transport problems likeDℋkD\_\{\\mathcal\{H\}\_\{k\}\}orDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}given access to uncoupled samples from source and target measures? Such an algorithm would yield an approach to dynamical generative modeling which is built to seek efficient representation via kernel methods or neural networks; this may prove useful on real\-world data in high dimension\. By contrast, the nonexistence of such an algorithm would yield some information about the fundamental difficulty of learning generative models from samples\. Furthermore, given the expressivity of transports with RKHS drifts \(see Example[4](https://arxiv.org/html/2610.10875#Thmexample4)\) it is also valuable to study the algorithmic differences between theDℋkD\_\{\\mathcal\{H\}\_\{k\}\}andDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}problems\. The algorithmic/computational aspects of this question are poorly understood and constitute a valuable direction for future work\.

In light of the deep ResNet pushforward counterexamples from Theorems[3](https://arxiv.org/html/2610.10875#Thmtheorem3)and[4](https://arxiv.org/html/2610.10875#Thmtheorem4), we can formulate the following slightly more precise ResNet version of this question:

###### Question 2\(Efficient generative modeling of ResNet pushforwards\)\.

Given a teacher ResNetF:ℝd→ℝdF:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}and samples from its Gaussian pushforwardπ=F\#​γd\\pi=F\_\{\\\#\}\\gamma\_\{d\}, is there an efficient algorithm which learns a drift field\(vt\)t∈\[0,1\]\(v\_\{t\}\)\_\{t\\in\[0,1\]\}transportingγd\\gamma\_\{d\}toπ\\pi? In other words, can we efficiently learn a deep ResNet/neural ODE which mapsγd\\gamma\_\{d\}toπ\\pi? What properties ofFFallow such an algorithm? Ideally, we want the learned transport to have path norm/Barron cost comparable to that ofFF\.

There are some strong prior works due to Chen et al\. regarding the computational hardness of learning pushforwards of the Gaussian by shallow neural networks: in\[[19](https://arxiv.org/html/2610.10875#bib.bib36)\]they show that a Gaussian pushed forward by a two\-layer neural network with polynomial activations can be efficiently learned under either quadratic or low\-rank structure, while in\[[20](https://arxiv.org/html/2610.10875#bib.bib35)\]they show by a moment\-matching argument that a Gaussian pushed forward by a two\-layer ReLU neural network withlog⁡\(d\)\\log\(d\)hidden neurons cannot always be efficiently learned in the statistical query model\.

These results mirror the regression setting, where small Barron norm of a target function does not necessarily imply efficient learnability\[[25](https://arxiv.org/html/2610.10875#bib.bib32), Theorem 4\]but a suitable sparse171717In fact, in the regression setting the sparsity threshold forpoly⁡\(d\)\\operatorname\{poly\}\(d\)complexity is alsolog⁡\(d\)\\log\(d\)hidden neurons without further assumptions, which aligns with the lower bound of\[[20](https://arxiv.org/html/2610.10875#bib.bib35)\]\.neural network representation or special activation structure does \(this is the subject of recent work on the multi\-index model, see e\.g\. the survey\[[12](https://arxiv.org/html/2610.10875#bib.bib33)\]\)\. Based on this, we can pose a question simpler than Question[2](https://arxiv.org/html/2610.10875#Thmquestion2)as a concrete next step toward understanding the computational landscape of generative modeling:

###### Question 3\(Efficient generative modeling of multi\-index pushforwards\)\.

Given a teacher multi\-index vector fieldF:ℝd→ℝdF:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}of the formF⁡\(x\)=∑j=1mσj​uj​ϕ​\(⟨x,θj⟩\)F\(x\)=\\sum\_\{j=1\}^\{m\}\\sigma\_\{j\}u\_\{j\}\\phi\(\\left\\langle x,\\theta\_\{j\}\\right\\rangle\)and samples from its Gaussian pushforwardπ=\(Id\+F\)\#​γd\\pi=\(\\operatorname\{Id\}\+F\)\_\{\\\#\}\\gamma\_\{d\}, is there an efficient algorithm which learns a generative model forπ\\pi? Which properties ofFFallow such an algorithm \(e\.g\.m=O⁡\(1\)m=O\(1\)\)?

Whenm=1m=1this problem is closely related to non\-Gaussian component analysis \(NGCA\), which is known to exhibit statistical\-computational gaps; see\[[26](https://arxiv.org/html/2610.10875#bib.bib12)\]and references therein\. The result of Example[1](https://arxiv.org/html/2610.10875#Thmexample1)shows that the standard denoising algorithm will have difficulty representing the reverse OU drifts of such a target using two\-layer neural networks,181818Note that a flow by a single\-index ReLU vector field can be represented by a single\-index ReLU pushforward due to homogeneity, and vice versa\.but the overall computational hardness of this problem is not completely understood\. A key difference from the multi\-index regression setting is that a learner can only observe moments from theπ\\pimarginal as opposed to cross\-moments of the induced coupling of\(γd,π\)\(\\gamma\_\{d\},\\pi\), which makes it easier to hide information by matching moments\. We leave the investigation into Question[3](https://arxiv.org/html/2610.10875#Thmquestion3)for future work\.

### 5\.2The role of depth and layerwise supervision

In a practical generative model, we are constrained by the amount of inference\-time computation required to generate a sample\. Consider a dynamical generative model \(a flow/SDE model performing e\.g\. denoising or optimal transport\), and suppose that the drifts are modeled using ResNets\. The layers of each ResNet can be viewed as a sequence of separate drifts in their own right: from the perspective of a particle, any update of the formx↦x\+η​F​\(x\)x\\mapsto x\+\\eta F\(x\)is a “drift”\. The natural quantity measuring sequential computation is the number of drift evaluations times the number of layers in each drift network, since this captures how many basic operations each particle must undergo during generation\. The major difference lies in how the transport is learned: each drift ResNet is trained to perform a specific operation \(e\.g\. in denoising models they attempt to match the reverse OU drifts\), while an individual layer of such a drift ResNet may implement any subroutine to help\.

Put simply, we may view the entire generative process as a single very deep ResNet which we train by supervised regression on various intermediate layers of the practitioner’s choosing\. Indeed, once the total depth is fixed by computational constraints, it is up to the designer whether to allocate more computation toward deeper drift models or more drift evaluations per generation; this is equivalent to less or more layerwise supervision, respectively\. This raises a natural algorithmic question, which we pose for denoising models specifically:

###### Question 4\(Allocation of depth/layerwise supervision\)\.

For a fixed total depth of the generative process of a denoising model, what is the “best” allocation between the number of neural network score evaluations and the depth of each score neural network? Here, “best” should be measured in terms of both training complexity and sample quality\.

The two extremes are \(1\) minimal supervision, in which case we train one deep ResNet pushing the Gaussian to the target distribution and \(2\) maximal supervision, in which case there are many shallow networks trained via score\-matching\. The first approach is reminiscent of generative adversarial networks \(GANs\)\[[39](https://arxiv.org/html/2610.10875#bib.bib59)\]; although these have been famously outperformed by iterative dynamical methods such as diffusion models\[[24](https://arxiv.org/html/2610.10875#bib.bib76)\], recently there has been much progress in minimizing the number of denoising steps required during generation\[[68](https://arxiv.org/html/2610.10875#bib.bib9),[74](https://arxiv.org/html/2610.10875#bib.bib10),[48](https://arxiv.org/html/2610.10875#bib.bib15)\]\. In Theorems[3](https://arxiv.org/html/2610.10875#Thmtheorem3)and[4](https://arxiv.org/html/2610.10875#Thmtheorem4)we showed the potential suboptimality of the second approach, since shallow score networks can cause inefficiencies due to the nature of the reverse OU supervision\. The optimal allocation is likely data\-dependent and somewhere in the middle of these two extremes, and it is a useful direction of future work to understand the underlying tradeoffs in both simple and real\-world settings\.

### 5\.3Application to sampling

We defined the Barron transport metricDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}as a dynamical optimal transport problem with tangent norm∥⋅∥ℬϕ\\\|\\cdot\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\. It turns out that any choice of tangent norm will yield a \(Finsler\) manifold geometry on𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), with an associated theory of absolutely continuous curves, geodesics, and gradient flows in terms of the continuity equation\. Importantly, the gradient flow ofKL\(⋅∥π\)\\operatorname\{KL\}\(\\cdot\\\|\\pi\)w\.r\.t\. such a geometry will always yield an algorithm for sampling fromπ\\pigiven query access to∇log⁡π\\nabla\\log\\pi\. This is the perspective underlying modern analysis of the SVGD sampling algorithm as aDℋkD\_\{\\mathcal\{H\}\_\{k\}\}gradient flow \(see\[[30](https://arxiv.org/html/2610.10875#bib.bib77)\]\), as well as the recently\-introduced Radon\-Wasserstein geometry and its induced sampling algorithm\[[42](https://arxiv.org/html/2610.10875#bib.bib71)\]\.

In the companion paper\[[28](https://arxiv.org/html/2610.10875#bib.bib1)\], we leverage this observation to design other sampling algorithms through careful selection of the tangent norms\. Doing so will require a rigorous study of the optimal transport geometry of these generalized dynamical metrics in the spirit of Ambrosio\-Gigli\-Savaré\[[5](https://arxiv.org/html/2610.10875#bib.bib69)\]\. In this subsection we will ignore mathematical rigor and the full generality, and instead we give a taste of the sampling application in the case of the Barron transport metricDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}in order to further motivate the study of the\(𝒫⁡\(ℝd\),Dℬϕ\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{B\}\_\{\\phi\}\}\)geometry\.

Indeed, by taking the gradient flow ofKL\(⋅∥π\)\\operatorname\{KL\}\(\\cdot\\\|\\pi\)in the metric space\(𝒫⁡\(ℝd\),Dℬϕ\)\(\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\),D\_\{\\mathcal\{B\}\_\{\\phi\}\}\)\(in the sense of\[[5](https://arxiv.org/html/2610.10875#bib.bib69), Chapter 1\]\) and discretizing it, we arrive at the following algorithm for sampling fromπ∝e−V\\pi\\propto e^\{\-V\}with potentialV:ℝd→ℝV:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}via a cloud of interacting particles:

Algorithm 1Barron transport samplerInput:Initial particles

X1,…,XN∈ℝdX\_\{1\},\\ldots,X\_\{N\}\\in\{\\mathbb\{R\}\}^\{d\}, activation

ϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}, step size

η\>0\\eta\>0\.

while*not converged*do

vn←∇V​\(Xn\)v\_\{n\}\\leftarrow\\nabla V\(X\_\{n\}\)for

n∈\[N\]n\\in\[N\];

\(σ⋆,u⋆,θ⋆\)←arg​minσ\>0u,θ∈𝕊d−1⁡\{σN​∑n=1N\[ϕ⁡\(⟨Xn,θ⟩\)​⟨u,vn⟩−⟨u,θ⟩​ϕ′​\(⟨Xn,θ⟩\)\]\+σ22\}\(\\sigma^\{\\star\},u^\{\\star\},\\theta^\{\\star\}\)\\leftarrow\\displaystyle\\argmin\_\{\\begin\{subarray\}\{c\}\\sigma\>0\\\\ u,\\theta\\in\\mathbb\{S\}^\{d\-1\}\\end\{subarray\}\}\\left\\\{\\frac\{\\sigma\}\{N\}\\sum\_\{n=1\}^\{N\}\\left\[\\phi\(\\left\\langle X\_\{n\},\\theta\\right\\rangle\)\\left\\langle u,v\_\{n\}\\right\\rangle\-\\left\\langle u,\\theta\\right\\rangle\\phi^\{\\prime\}\(\\left\\langle X\_\{n\},\\theta\\right\\rangle\)\\right\]\+\\frac\{\\sigma^\{2\}\}\{2\}\\right\\\};

Xn←Xn\+η⋅σ⋆​u⋆​ϕ​\(⟨Xn,θ⋆⟩\)X\_\{n\}\\leftarrow X\_\{n\}\+\\eta\\cdot\\sigma^\{\\star\}u^\{\\star\}\\phi\(\\left\\langle X\_\{n\},\\theta^\{\\star\}\\right\\rangle\)for

n∈\[N\]n\\in\[N\];

This interacting particle sampler regresses a single\-index vector field at each step, along which it coherently flows all the particles\. This is very similar in spirit to the SVGD algorithm, except that the velocity field is selected in a way which utilizes feature learning instead of kernel regression\. Algorithm[1](https://arxiv.org/html/2610.10875#algorithm1)has favorable behavior on target distributions which have hidden low\-dimensional structure\. As some illustrative examples which are strongly log\-concave:

1. \(1\)\(Hidden subspace\)\.If our target isπ\(1\):=𝒩⁡\(0,\(𝟙d−1log⁡d​P\)−1\)\\pi^\{\(1\)\}:=\\mathcal\{N\}\\left\(0,\(\{\\mathbbm\{1\}\}\_\{d\}\-\\frac\{1\}\{\\log d\}P\)^\{\-1\}\\right\)for any orthogonal projectionP=P⊤=P2P=P^\{\\top\}=P^\{2\}withrank⁡\(P\)=Θ⁡\(log2⁡d\)\\operatorname\{rank\}\(P\)=\\Theta\(\\log^\{2\}d\), then the following hold when initializing fromγd\\gamma\_\{d\}: - •Langevin Monte Carlo \(LMC\) with any deterministic, nonnegative step size schedule requiresΩ⁡\(d\)\\Omega\(\\sqrt\{d\}\)steps to decrease KL by a constant factor\. - •Mean\-field SVGD \(i\.e\. the infinite\-particle population version\) with a Gaussian or Matérn kernel and any bounded stable191919By this, we mean that a descent lemma is satisfied for the KL, i\.e\.KL\(ρn\+1∥π\)≤KL\(ρn∥π\)−cηn∥vn∥ℋk2\\operatorname\{KL\}\(\\rho\_\{n\+1\}\\\|\\pi\)\\leq\\operatorname\{KL\}\(\\rho\_\{n\}\\\|\\pi\)\-c\\eta\_\{n\}\\\|v\_\{n\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}^\{2\}withc\>0c\>0independent of dimension\.step size schedule requiresΩ⁡\(d\)\\Omega\(d\)steps to decrease KL by a constant factor\. - •The mean\-field Barron transport sampler \(with linear activationϕ⁡\(z\)=z\\phi\(z\)=zand step sizeη=1/2\\eta=1/2\) achieves KL≤ε\\leq\\varepsiloninO⁡\(\(log⁡d\)2​\(1\+log⁡log⁡\(1/ε\)\)\)O\\left\(\(\\log d\)^\{2\}\(1\+\\log\\log\(1/\\varepsilon\)\)\\right\)steps\.
2. \(2\)\(Single\-index tilt\)\.If our target isπ\(2\)∝e−a​ψ​\(⟨x,θ⟩\)−‖x‖2/2\\pi^\{\(2\)\}\\propto e^\{\-a\\psi\(\\left\\langle x,\\theta\\right\\rangle\)\-\\\|x\\\|^\{2\}/2\}for some hiddena\>0a\>0,θ∈𝕊d−1\\theta\\in\\mathbb\{S\}^\{d\-1\}, and an evenC2C^\{2\}ψ:ℝ→ℝ\\psi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}for whichπ\(2\)\\pi^\{\(2\)\}is strongly log\-concave, then the following hold when initializing fromγd\\gamma\_\{d\}assuming positive finite initial KL: - •LMC with any deterministic, nonnegative step size schedule requiresΩ⁡\(d\)\\Omega\(\\sqrt\{d\}\)steps to decrease KL by a constant factor\. - •Mean\-field SVGD with a Gaussian or Matérn kernel and any bounded stable step size schedule requiresΩ⁡\(d\)\\Omega\(d\)steps to decrease KL by a constant factor\. - •The mean\-field Barron transport sampler \(with activation202020Or any activationϕ\\phiwith‖ψ′‖ℬϕ<∞\\\|\\psi^\{\\prime\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}<\\infty\.ϕ=ψ′\\phi=\\psi^\{\\prime\}and a dimension\-independent step size\) achieves KL≤ε\\leq\\varepsiloninO⁡\(log⁡\(1/ε\)\)O\(\\log\(1/\\varepsilon\)\)steps\.

Herea,ψa,\\psi, and the step size bounds are independent of dimension, and Gaussian and Matérn kernels are normalized totr⁡k⁡\(x,x\)=d\\operatorname\{tr\}k\(x,x\)=d\. This behavior is numerically verified in Figure[4](https://arxiv.org/html/2610.10875#S5.F4)\.

Figure 4:Plots of the number of steps required so thatKL\(ρn∥π\)≤14KL\(ρ0∥π\)\\operatorname\{KL\}\(\\rho\_\{n\}\\\|\\pi\)\\leq\\frac\{1\}\{4\}\\operatorname\{KL\}\(\\rho\_\{0\}\\\|\\pi\)as a function ofddfor the constructionsπ∈\{π\(1\),π\(2\)\}\\pi\\in\\\{\\pi^\{\(1\)\},\\pi^\{\(2\)\}\\\}listed above\. For each fixed dimension, we sweep over fixed learning rates \(and bandwidth/smoothness in the SVGD cases\) and plot the best performance; triangles indicate a failure to reach the threshold KL in an allotted number of steps\. The hidden subspace is of dimension⌊log2⁡\(d\)/4⌋\\lfloor\\log^\{2\}\(d\)/4\\rfloor, and the single\-index tilt is constructed usinga=0\.1a=0\.1andψ⁡\(z\)=0\.256​cos⁡\(z\)\+1\.32​cos⁡\(2​z\)\\psi\(z\)=0\.256\\cos\(z\)\+1\.32\\cos\(2z\)\.In words, the Barron transport sampler is provably useful for discovering hidden directions and choosing adaptive step sizes in those directions, whereas other samplers can suffer from trying to apply similar step sizes in all directions or in a fixed feature basis\. This phenomenon persists for more complex target measures sharing these properties \(multi\-index tilts, products of these examples, etc\.\)\. Indeed, one can show dimension\-free convergence rates for the discretizedDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}sampler on log\-smooth targets satisfying a log\-Sobolev inequality for which the relative scores along the sampling path have small Barron norm\. We leave the derivation of this update rule, convergence rates, and the math/intuition behind the above examples to the companion paper\[[28](https://arxiv.org/html/2610.10875#bib.bib1)\]\.

## References

- \[1\]\(2025\)Stochastic interpolants: a unifying framework for flows and diffusions\.Journal of Machine Learning Research26\(209\),pp\. 1–80\.Cited by:[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p2.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p2.4),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p3.4),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px2.p1.2)\.
- \[2\]M\. S\. Albergo, N\. M\. Boffi, and E\. Vanden\-Eijnden\(2023\)Stochastic interpolants: a unifying framework for flows and diffusions\.External Links:2303\.08797,[Link](https://arxiv.org/abs/2303.08797)Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p2.3)\.
- \[3\]M\. S\. Albergo and E\. Vanden\-Eijnden\(2023\)Building normalizing flows with stochastic interpolants\.InInternational Conference on Learning Representations,Cited by:[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px2.p1.2),[Remark 11](https://arxiv.org/html/2610.10875#Thmremark11.p1.2.1)\.
- \[4\]J\. M\. Altschuler and S\. Chewi\(2026\)Shifted composition iii: local error framework for KL divergence\.Foundations of Computational Mathematics\.External Links:[Document](https://dx.doi.org/10.1007/s10208-026-09753-x)Cited by:[Remark 11](https://arxiv.org/html/2610.10875#Thmremark11.p1.1.1)\.
- \[5\]L\. Ambrosio, N\. Gigli, and G\. Savaré\(2008\)Gradient flows: in metric spaces and in the space of probability measures\.2 edition,Lectures in Mathematics ETH Zürich,Birkhäuser,Basel\.External Links:ISBN 978\-3\-7643\-8721\-1,[Document](https://dx.doi.org/10.1007/978-3-7643-8722-8),[Link](https://doi.org/10.1007/978-3-7643-8722-8)Cited by:[item \(iv\)](https://arxiv.org/html/2610.10875#A1.I1.ix4.p1.1),[§A\.1](https://arxiv.org/html/2610.10875#A1.SS1.p8.5.1),[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p1.1),[1st item](https://arxiv.org/html/2610.10875#S2.I1.i1.p1.2),[2nd item](https://arxiv.org/html/2610.10875#S2.I1.i2.p1.2),[§2\.1](https://arxiv.org/html/2610.10875#S2.SS1.p1.2),[§2](https://arxiv.org/html/2610.10875#S2.p1.1),[§2](https://arxiv.org/html/2610.10875#S2.p2.3),[§3\.1](https://arxiv.org/html/2610.10875#S3.SS1.p4.1),[§5\.3](https://arxiv.org/html/2610.10875#S5.SS3.p2.1),[§5\.3](https://arxiv.org/html/2610.10875#S5.SS3.p3.1)\.
- \[6\]K\. Atkinson and W\. Han\(2012\)Spherical harmonics and approximations on the unit sphere: an introduction\.Lecture Notes in Mathematics, Vol\.2044,Springer,Berlin, Heidelberg\.External Links:[Document](https://dx.doi.org/10.1007/978-3-642-25983-8),ISBN 978\-3\-642\-25983\-8Cited by:[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p16.9.1),[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p4.1),[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p6.1.1),[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p7.10.1),[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p7.16.1),[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p7.5.1),[§A\.5](https://arxiv.org/html/2610.10875#A1.SS5.p19.4.1),[§A\.5](https://arxiv.org/html/2610.10875#A1.SS5.p3.1.1),[§A\.5](https://arxiv.org/html/2610.10875#A1.SS5.p3.3.1),[§A\.5](https://arxiv.org/html/2610.10875#A1.SS5.p4.7.1),[Lemma 9](https://arxiv.org/html/2610.10875#Thmlemma9.p1.4.1)\.
- \[7\]F\. Bach\(2017\)Breaking the curse of dimensionality with convex neural networks\.Journal of Machine Learning Research18\(19\),pp\. 1–53\.External Links:[Link](https://www.jmlr.org/papers/v18/14-546.html)Cited by:[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p4.1),[§1\.1](https://arxiv.org/html/2610.10875#S1.SS1.SSS0.Px2.p4.1),[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.10875#S1.p5.1),[§4\.2](https://arxiv.org/html/2610.10875#S4.SS2.p4.1),[§4](https://arxiv.org/html/2610.10875#S4.p3.1),[Abstract](https://arxiv.org/html/2610.10875#abstract1.2)\.
- \[8\]A\. R\. Barron\(1993\)Universal approximation bounds for superpositions of a sigmoidal function\.IEEE Transactions on Information Theory39\(3\),pp\. 930–945\.External Links:[Document](https://dx.doi.org/10.1109/18.212500)Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p5.1)\.
- \[9\]J\. Benamou and Y\. Brenier\(2000\)A computational fluid mechanics solution to the monge–kantorovich mass transfer problem\.Numerische Mathematik84\(3\),pp\. 375–393\.External Links:[Document](https://dx.doi.org/10.1007/s002110050002)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p1.1),[§2\.1](https://arxiv.org/html/2610.10875#S2.SS1.p1.2)\.
- \[10\]L\. Brasco\(2012\)A survey on dynamical transport distances\.Journal of Mathematical Sciences181\(6\),pp\. 755–781\.External Links:[Document](https://dx.doi.org/10.1007/s10958-012-0713-7),[Link](https://doi.org/10.1007/s10958-012-0713-7)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p1.1)\.
- \[11\]T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.Advances in neural information processing systems33,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p1.1)\.
- \[12\]J\. Bruna and D\. Hsu\(2025\)Survey on algorithms for multi\-index models\.Statistical Science40\(3\),pp\. 378–391\.External Links:[Document](https://dx.doi.org/10.1214/25-STS993),[Link](https://doi.org/10.1214/25-STS993)Cited by:[§1\.1](https://arxiv.org/html/2610.10875#S1.SS1.SSS0.Px2.p3.1),[§5\.1](https://arxiv.org/html/2610.10875#S5.SS1.p5.1)\.
- \[13\]J\. Bruna\(2026\)The multiscale single\-index model: a stylized model for hierarchical feature learning\.arXiv preprint arXiv:2607\.03347\.Cited by:[§A\.7\.2](https://arxiv.org/html/2610.10875#A1.SS7.SSS2.p1.1),[§A\.7\.2](https://arxiv.org/html/2610.10875#A1.SS7.SSS2.p7.1),[§1\.1](https://arxiv.org/html/2610.10875#S1.SS1.SSS0.Px1.p2.1),[2nd item](https://arxiv.org/html/2610.10875#S4.I3.i2.p1.1)\.
- \[14\]L\. A\. Caffarelli\(1992\)The regularity of mappings with a convex potential\.Journal of the American Mathematical Society5\(1\),pp\. 99–104\.External Links:ISSN 08940347, 10886834,[Link](http://www.jstor.org/stable/2152752)Cited by:[footnote 16](https://arxiv.org/html/2610.10875#footnote16)\.
- \[15\]D\. Chemnitz, M\. Engel, C\. Kuehn, and S\. Kuntz\(2025\)A dynamical systems perspective on the analysis of neural networks\.arXiv preprint arXiv:2507\.05164\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p3.1)\.
- \[16\]F\. Chen, S\. Chewi, C\. C\. Daskalakis, and A\. Rakhlin\(2026\)High\-accuracy sampling for diffusion models and log\-concave distributions\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 13835–13869\.Cited by:[§4\.1\.5](https://arxiv.org/html/2610.10875#S4.SS1.SSS5.p6.1)\.
- \[17\]R\. T\. Q\. Chen, Y\. Rubanova, J\. Bettencourt, and D\. Duvenaud\(2018\)Neural ordinary differential equations\.InAdvances in Neural Information Processing Systems,Vol\.31\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.p2.2)\.
- \[18\]S\. Chen, S\. Chewi, J\. Li, Y\. Li, A\. Salim, and A\. R\. Zhang\(2023\)Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions\.InThe Eleventh International Conference on Learning Representations,Cited by:[§A\.7\.3](https://arxiv.org/html/2610.10875#A1.SS7.SSS3.p1.1),[1st item](https://arxiv.org/html/2610.10875#S4.I3.i1.p2.1),[§4\.1\.3](https://arxiv.org/html/2610.10875#S4.SS1.SSS3.p3.1),[§4\.1\.5](https://arxiv.org/html/2610.10875#S4.SS1.SSS5.p6.1)\.
- \[19\]S\. Chen, J\. Li, Y\. Li, and A\. R\. Zhang\(2023\)Learning polynomial transformations via generalized tensor decompositions\.InProceedings of the 55th Annual ACM Symposium on Theory of Computing,STOC 2023,New York, NY, USA,pp\. 1671–1684\.External Links:ISBN 9781450399135,[Link](https://doi.org/10.1145/3564246.3585209),[Document](https://dx.doi.org/10.1145/3564246.3585209)Cited by:[§5\.1](https://arxiv.org/html/2610.10875#S5.SS1.p4.1)\.
- \[20\]S\. Chen, J\. Li, and Y\. Li\(2022\)Learning \(very\) simple generative models is hard\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.35,pp\. 35269–35281\.Cited by:[§5\.1](https://arxiv.org/html/2610.10875#S5.SS1.p4.1),[footnote 17](https://arxiv.org/html/2610.10875#footnote17)\.
- \[21\]X\. Cheng, K\. Fu, and F\. Farnia\(2025\)On the hardness of sampling from mixture distributions via langevin dynamics\.External Links:2406\.02017,[Link](https://arxiv.org/abs/2406.02017)Cited by:[Remark 6](https://arxiv.org/html/2610.10875#Thmremark6.p1.1.1)\.
- \[22\]S\. Chewi, J\. Niles\-Weed, and P\. Rigollet\(2025\)Statistical optimal transport\.Lecture Notes in Mathematics, Vol\.2364,Springer,Cham, Switzerland\.External Links:ISBN 978\-3\-031\-85159\-9,[Document](https://dx.doi.org/10.1007/978-3-031-85160-5),[Link](https://doi.org/10.1007/978-3-031-85160-5)Cited by:[Lemma 4](https://arxiv.org/html/2610.10875#Thmlemma4)\.
- \[23\]L\. Chizat\(2026\)The hidden width of deep resnets: tight error bounds and phase diagram\.External Links:2509\.10167,[Link](https://arxiv.org/abs/2509.10167)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2610.10875#S1.p4.1),[§3\.3](https://arxiv.org/html/2610.10875#S3.SS3.p1.3),[Remark 3](https://arxiv.org/html/2610.10875#Thmremark3.p1.1.1)\.
- \[24\]P\. Dhariwal and A\. Nichol\(2021\)Diffusion models beat gans on image synthesis\.External Links:2105\.05233,[Link](https://arxiv.org/abs/2105.05233)Cited by:[§5\.2](https://arxiv.org/html/2610.10875#S5.SS2.p3.1)\.
- \[25\]I\. Diakonikolas, D\. M\. Kane, V\. Kontonis, and N\. Zarifis\(2020\)Algorithms and sq lower bounds for pac learning one\-hidden\-layer relu networks\.InProceedings of Thirty Third Conference on Learning Theory,J\. Abernethy and S\. Agarwal \(Eds\.\),Proceedings of Machine Learning Research, Vol\.125,pp\. 1514–1539\.External Links:[Link](https://proceedings.mlr.press/v125/diakonikolas20d.html)Cited by:[§5\.1](https://arxiv.org/html/2610.10875#S5.SS1.p5.1)\.
- \[26\]I\. Diakonikolas and D\. M\. Kane\(2022\)Non\-gaussian component analysis via lattice basis reduction\.InProceedings of the 35th Conference on Learning Theory,Proceedings of Machine Learning Research, Vol\.178,pp\. 4535–4547\.Cited by:[§5\.1](https://arxiv.org/html/2610.10875#S5.SS1.p6.1)\.
- \[27\]V\. Divol, J\. Niles\-Weed, and A\. Pooladian\(2025\)Optimal transport map estimation in general function spaces\.The Annals of Statistics53\(3\),pp\. 963 – 988\.External Links:[Document](https://dx.doi.org/10.1214/24-AOS2482),[Link](https://doi.org/10.1214/24-AOS2482)Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p5.1)\.
- \[28\]E\. Dogariu and J\. Bruna\(2026\)Barron optimal transport ii: sampling\.Note:in preparationCited by:[§1](https://arxiv.org/html/2610.10875#S1.p6.1),[§3\.1](https://arxiv.org/html/2610.10875#S3.SS1.p4.1),[§3\.2](https://arxiv.org/html/2610.10875#S3.SS2.p4.1),[§5\.3](https://arxiv.org/html/2610.10875#S5.SS3.p2.1),[§5\.3](https://arxiv.org/html/2610.10875#S5.SS3.p7.1),[Remark 10](https://arxiv.org/html/2610.10875#Thmremark10.p2.1.1),[Abstract](https://arxiv.org/html/2610.10875#abstract1.3),[footnote 7](https://arxiv.org/html/2610.10875#footnote7)\.
- \[29\]S\. Duane, A\. D\. Kennedy, B\. J\. Pendleton, and D\. Roweth\(1987\)Hybrid monte carlo\.Physics letters B195\(2\),pp\. 216–222\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p1.1)\.
- \[30\]A\. Duncan, N\. Nuesken, and L\. Szpruch\(2023\)On the geometry of stein variational gradient descent\.External Links:1912\.00894,[Link](https://arxiv.org/abs/1912.00894)Cited by:[§1\.1](https://arxiv.org/html/2610.10875#S1.SS1.SSS0.Px2.p2.1),[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p2.1),[1st item](https://arxiv.org/html/2610.10875#S2.I2.i1.p1.2),[2nd item](https://arxiv.org/html/2610.10875#S2.I2.i2.p1.2),[§2\.2](https://arxiv.org/html/2610.10875#S2.SS2.p2.3),[§2\.2](https://arxiv.org/html/2610.10875#S2.SS2.p3.1),[§5\.3](https://arxiv.org/html/2610.10875#S5.SS3.p1.1)\.
- \[31\]W\. E, C\. Ma, and L\. Wu\(2022\)The barron space and the flow\-induced function spaces for neural network models\.Constructive Approximation55\(1\),pp\. 369–406\.External Links:[Document](https://dx.doi.org/10.1007/s00365-021-09549-y)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p2.1),[§3\.3](https://arxiv.org/html/2610.10875#S3.SS3.p1.3),[§3\.3](https://arxiv.org/html/2610.10875#S3.SS3.p5.1),[§3](https://arxiv.org/html/2610.10875#S3.p3.1),[§4\.1\.4](https://arxiv.org/html/2610.10875#S4.SS1.SSS4.p4.1),[Remark 1](https://arxiv.org/html/2610.10875#Thmremark1.p1.2.1),[Abstract](https://arxiv.org/html/2610.10875#abstract1.2)\.
- \[32\]A\. Einav, Y\. Jiang, and A\. R\. Mészáros\(2026\)On the equivalence between static and dynamic optimal transport governed by linear control systems\.SIAM Journal on Mathematical Analysis\.Note:To appearCited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p1.1)\.
- \[33\]K\. Elamvazhuthi\(2025\)Benamou\-brenier formulation of optimal transport for nonlinear control systems onℝd\\mathbb\{R\}^\{d\}\.arXiv preprint arXiv:2407\.16088\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p1.1)\.
- \[34\]R\. Eldan and O\. Shamir\(2016\)The power of depth for feedforward neural networks\.InProceedings of the 29th Annual Conference on Learning Theory \(COLT\),V\. Feldman, A\. Rakhlin, and O\. Shamir \(Eds\.\),Proceedings of Machine Learning Research, Vol\.49,pp\. 907–940\.External Links:[Link](https://proceedings.mlr.press/v49/eldan16.html)Cited by:[§4\.1\.3](https://arxiv.org/html/2610.10875#S4.SS1.SSS3.p3.1),[§4\.1\.4](https://arxiv.org/html/2610.10875#S4.SS1.SSS4.p4.1)\.
- \[35\]C\. Finlay, J\. Jacobsen, L\. Nurbekyan, and A\. M\. Oberman\(2020\)How to train your neural ODE: the world of jacobian and kinetic regularization\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 3154–3164\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px2.p1.1)\.
- \[36\]M\. Fisher, T\. Nolan, M\. Graham, D\. Prangle, and C\. Oates\(2021\)Measure transport with kernel stein discrepancy\.InProceedings of the 24th International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.130,pp\. 1054–1062\.Cited by:[§4\.2](https://arxiv.org/html/2610.10875#S4.SS2.p11.1)\.
- \[37\]H\. Fu, T\. Guo, Y\. Bai, and S\. Mei\(2023\)What can a single attention layer learn? a study through the random features lens\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§A\.8\.2](https://arxiv.org/html/2610.10875#A1.SS8.SSS2.p8.1.1)\.
- \[38\]B\. Ghorbani, S\. Mei, T\. Misiakiewicz, and A\. Montanari\(2021\)When do neural networks outperform kernel methods?\.Journal of Statistical Mechanics: Theory and Experiment2021\(12\),pp\. 124009\.External Links:[Document](https://dx.doi.org/10.1088/1742-5468/ac3a81)Cited by:[§1\.1](https://arxiv.org/html/2610.10875#S1.SS1.SSS0.Px2.p4.1),[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px5.p1.1),[§4](https://arxiv.org/html/2610.10875#S4.p3.1)\.
- \[39\]I\. J\. Goodfellow, J\. Pouget\-Abadie, M\. Mirza, B\. Xu, D\. Warde\-Farley, S\. Ozair, A\. Courville, and Y\. Bengio\(2014\)Generative adversarial nets\.Advances in neural information processing systems27\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p1.1),[§5\.2](https://arxiv.org/html/2610.10875#S5.SS2.p3.1)\.
- \[40\]I\. S\. Gradshteyn and I\. M\. Ryzhik\(2014\)Table of integrals, series, and products\.8 edition,Academic Press,Amsterdam\.External Links:ISBN 9780123849335,[Document](https://dx.doi.org/10.1016/C2010-0-64839-5)Cited by:[§A\.5](https://arxiv.org/html/2610.10875#A1.SS5.p10.2.1)\.
- \[41\]W\. Grathwohl, R\. T\. Q\. Chen, J\. Bettencourt, I\. Sutskever, and D\. Duvenaud\(2019\)FFJORD: free\-form continuous dynamics for scalable reversible generative models\.InInternational Conference on Learning Representations,Cited by:[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.p2.2)\.
- \[42\]E\. Hess\-Childs, D\. Slepčev, and L\. Xu\(2026\)Radon–wasserstein gradient flows for interacting\-particle sampling in high dimensions\.External Links:2602\.05227,[Link](https://arxiv.org/abs/2602.05227)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p2.1),[§5\.3](https://arxiv.org/html/2610.10875#S5.SS3.p1.1)\.
- \[43\]J\. Ho, A\. Jain, and P\. Abbeel\(2020\)Denoising diffusion probabilistic models\.Advances in neural information processing systems33,pp\. 6840–6851\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p3.4)\.
- \[44\]K\. Hornik\(1991\)Approximation capabilities of multilayer feedforward networks\.Neural Networks4\(2\),pp\. 251–257\.External Links:[Document](https://dx.doi.org/10.1016/0893-6080%2891%2990009-T)Cited by:[§A\.7\.2](https://arxiv.org/html/2610.10875#A1.SS7.SSS2.p19.1),[§A\.7\.2](https://arxiv.org/html/2610.10875#A1.SS7.SSS2.p20.3.1),[§A\.7](https://arxiv.org/html/2610.10875#A1.SS7.p8.1)\.
- \[45\]A\. Jacot and A\. Kaiser\(2025\)Hamiltonian mechanics of feature learning: bottleneck structure in leaky resnets\.InConference on Parsimony and Learning,B\. Chen, S\. Liu, M\. Pilanci, W\. Su, J\. Sulam, Y\. Wang, and Z\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.280,pp\. 1255–1273\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p2.1)\.
- \[46\]A\. Jacot\(2026\)Deep learning as a convex paradigm of computation: minimizing circuit size with resnets\.External Links:2511\.20888,[Link](https://arxiv.org/abs/2511.20888)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p2.1),[§3\.3](https://arxiv.org/html/2610.10875#S3.SS3.p5.1),[Remark 3](https://arxiv.org/html/2610.10875#Thmremark3.p1.1.1)\.
- \[47\]S\. Janson\(1997\)Gaussian hilbert spaces\.Cambridge Tracts in Mathematics, Vol\.129,Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/CBO9780511526169)Cited by:[§A\.7\.2](https://arxiv.org/html/2610.10875#A1.SS7.SSS2.p3.1)\.
- \[48\]D\. Kim, C\. Lai, W\. Liao, N\. Murata, Y\. Takida, T\. Uesaka, Y\. He, Y\. Mitsufuji, and S\. Ermon\(2024\)Consistency trajectory models: learning probability flow ODE trajectory of diffusion\.InInternational Conference on Learning Representations,Cited by:[§5\.2](https://arxiv.org/html/2610.10875#S5.SS2.p3.1)\.
- \[49\]Y\. Kim and E\. Milman\(2012\)A generalization of caffarelli’s contraction theorem via \(reverse\) heat flow\.Mathematische Annalen354\(3\),pp\. 827–862\.External Links:[Document](https://dx.doi.org/10.1007/s00208-011-0749-x)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px4.p1.1)\.
- \[50\]C\. Kuehn and S\. Kuntz\(2026\)Analysis of the geometric structure of neural networks and neural odes via morse functions\.Advances in Computational Mathematics52\(1\),pp\. 9\.External Links:[Document](https://dx.doi.org/10.1007/s10444-025-10273-5)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p3.1)\.
- \[51\]H\. Lavenant and F\. Santambrogio\(2022\)The flow map of the fokker–planck equation does not provide optimal transport\.Applied Mathematics Letters133,pp\. 108225\.External Links:ISSN 0893\-9659,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.aml.2022.108225),[Link](https://www.sciencedirect.com/science/article/pii/S089396592200180X)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px4.p1.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p4.1)\.
- \[52\]Z\. Li, K\. Liu, L\. Liverani, and E\. Zuazua\(2026\)Universal approximation of dynamical systems by semi\-autonomous neural odes and applications\.arXiv preprint arXiv:2407\.17092\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p3.1)\.
- \[53\]T\. Liang and H\. Tran\-Bach\(2022\)Mehler’s formula, branching process, and compositional kernels of deep neural networks\.Journal of the American Statistical Association117\(539\),pp\. 1324–1337\.External Links:[Document](https://dx.doi.org/10.1080/01621459.2020.1853547)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px5.p2.1)\.
- \[54\]Y\. Lipman, R\. T\. Q\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. Le\(2023\)Flow matching for generative modeling\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px2.p1.2)\.
- \[55\]Q\. Liu\(2017\)Stein variational gradient descent as gradient flow\.External Links:1704\.07520,[Link](https://arxiv.org/abs/1704.07520)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px1.p2.1),[§2\.2](https://arxiv.org/html/2610.10875#S2.SS2.p3.1),[§2\.2](https://arxiv.org/html/2610.10875#S2.SS2.p6.1),[§2\.2](https://arxiv.org/html/2610.10875#S2.SS2.p7.1)\.
- \[56\]X\. Liu, C\. Gong, and Q\. Liu\(2022\)Flow straight and fast: learning to generate and transfer data with rectified flow\.arXiv preprint arXiv:2209\.03003\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.10875#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px3.p2.1)\.
- \[57\]Y\. Marzouk, Z\. Ren, S\. Wang, and J\. Zech\(2024\)Distribution learning via neural differential equations: a nonparametric statistical perspective\.Journal of Machine Learning Research25\(121\),pp\. 1–61\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p4.1)\.
- \[58\]Y\. Marzouk, Z\. Ren, and J\. Zech\(2025\)Distribution learning via neural differential equations: minimal energy regularization and approximation theory\.arXiv preprint arXiv:2502\.03795\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p4.1)\.
- \[59\]A\. Maurais, B\. Hosseini, and Y\. Marzouk\(2025\)Learning paths for dynamic measure transport: a control perspective\.arXiv preprint arXiv:2511\.03797\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px2.p2.1),[§4\.2](https://arxiv.org/html/2610.10875#S4.SS2.p11.1)\.
- \[60\]N\. Metropolis, A\. W\. Rosenbluth, M\. N\. Rosenbluth, A\. H\. Teller, and E\. Teller\(1953\)Equation of state calculations by fast computing machines\.The journal of chemical physics21\(6\),pp\. 1087–1092\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p1.1)\.
- \[61\]D\. Mikulincer and Y\. Shenfeld\(2022\)On the lipschitz properties of transportation along heat flows\.External Links:2201\.01382,[Link](https://arxiv.org/abs/2201.01382)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px4.p1.1)\.
- \[62\]F\. Noé, S\. Olsson, J\. Köhler, and H\. Wu\(2019\)Boltzmann generators: sampling equilibrium states of many\-body systems with deep learning\.Science365\(6457\),pp\. eaaw1147\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p1.1)\.
- \[63\]D\. Onken, S\. Wu Fung, X\. Li, and L\. Ruthotto\(2021\)OT\-Flow: fast and accurate continuous normalizing flows via optimal transport\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 9223–9232\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v35i10.17113)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px2.p1.1)\.
- \[64\]F\. Otto\(2001\)THE geometry of dissipative evolution equations: the porous medium equation\.Communications in Partial Differential Equations26\(1\-2\),pp\. 101–174\.External Links:[Document](https://dx.doi.org/10.1081/PDE-100002243)Cited by:[§2\.1](https://arxiv.org/html/2610.10875#S2.SS1.p2.1)\.
- \[65\]B\. Pandey, B\. Hosseini, P\. Batlle, and H\. Owhadi\(2025\)Diffeomorphic measure matching with kernels for generative modeling\.SIAM Journal on Mathematics of Data Science7\(3\),pp\. 937–964\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px2.p2.1),[§4\.2](https://arxiv.org/html/2610.10875#S4.SS2.p11.1)\.
- \[66\]R\. Parhi and R\. D\. Nowak\(2021\)Banach space representer theorems for neural networks and ridge splines\.Journal of Machine Learning Research22\(43\),pp\. 1–40\.Cited by:[§A\.7\.1](https://arxiv.org/html/2610.10875#A1.SS7.SSS1.p14.2.1)\.
- \[67\]D\. Ruiz\-Balet and E\. Zuazua\(2023\)Neural ODE control for classification, approximation, and transport\.SIAM Review65\(3\),pp\. 735–773\.External Links:[Document](https://dx.doi.org/10.1137/21M1411433)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px3.p3.1)\.
- \[68\]T\. Salimans and J\. Ho\(2022\)Progressive distillation for fast sampling of diffusion models\.InInternational Conference on Learning Representations,Cited by:[§5\.2](https://arxiv.org/html/2610.10875#S5.SS2.p3.1)\.
- \[69\]V\. Shankar, A\. Fang, W\. Guo, S\. Fridovich\-Keil, J\. Ragan\-Kelley, L\. Schmidt, and B\. Recht\(2020\)Neural kernels without tangents\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 8614–8623\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px5.p2.1)\.
- \[70\]D\. E\. Shaw, P\. Maragakis, K\. Lindorff\-Larsen, S\. Piana, R\. O\. Dror, M\. P\. Eastwood, J\. A\. Bank, J\. M\. Jumper, J\. K\. Salmon, Y\. Shan, and W\. Wriggers\(2010\)Atomic\-level characterization of the structural dynamics of proteins\.Science330\(6002\),pp\. 341–346\.External Links:[Document](https://dx.doi.org/10.1126/science.1187409),[Link](https://www.science.org/doi/abs/10.1126/science.1187409),https://www\.science\.org/doi/pdf/10\.1126/science\.1187409Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p1.1)\.
- \[71\]C\. Sheng, S\. Ma, H\. Li, L\. Wang, and L\. Jia\(2021\)Nontensorial generalised Hermite spectral methods for PDEs with fractional Laplacian and Schrödinger operators\.ESAIM: Mathematical Modelling and Numerical Analysis55\(5\),pp\. 2019–2047\.External Links:[Document](https://dx.doi.org/10.1051/m2an/2021049)Cited by:[§A\.4](https://arxiv.org/html/2610.10875#A1.SS4.p11.3.1)\.
- \[72\]J\. Sohl\-Dickstein, E\. Weiss, N\. Maheswaranathan, and S\. Ganguli\(2015\)Deep unsupervised learning using nonequilibrium thermodynamics\.InInternational conference on machine learning,pp\. 2256–2265\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p1.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p3.4)\.
- \[73\]M\. J\. Song\(2024\)Cryptographic hardness of score estimation\.arXiv preprint arXiv:2404\.03272\.Cited by:[§1\.1](https://arxiv.org/html/2610.10875#S1.SS1.SSS0.Px1.p2.1),[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px4.p2.1),[§4\.1\.5](https://arxiv.org/html/2610.10875#S4.SS1.SSS5.p7.1)\.
- \[74\]Y\. Song, P\. Dhariwal, M\. Chen, and I\. Sutskever\(2023\)Consistency models\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 32211–32252\.Cited by:[§5\.2](https://arxiv.org/html/2610.10875#S5.SS2.p3.1)\.
- \[75\]Y\. Song, J\. Sohl\-Dickstein, D\. P\. Kingma, A\. Kumar, S\. Ermon, and B\. Poole\(2020\)Score\-based generative modeling through stochastic differential equations\.arXiv preprint arXiv:2011\.13456\.Cited by:[§1](https://arxiv.org/html/2610.10875#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p3.4)\.
- \[76\]E\. M\. Stein and R\. Shakarchi\(2003\)Complex analysis\.Princeton Lectures in Analysis, Vol\.2,Princeton University Press,Princeton, NJ\.Cited by:[§A\.6](https://arxiv.org/html/2610.10875#A1.SS6.p13.20.1)\.
- \[77\]A\. Tanana\(2021\)Comparison of transport map generated by heat flow interpolation and the optimal transport brenier map\.Communications in Contemporary Mathematics23\(6\),pp\. 2050025\.External Links:[Document](https://dx.doi.org/10.1142/S021919972050025X)Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px4.p1.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px1.p4.1)\.
- \[78\]M\. Telgarsky\(2016\)Benefits of depth in neural networks\.In29th Annual Conference on Learning Theory,Proceedings of Machine Learning Research, Vol\.49,pp\. 1517–1539\.Cited by:[§4\.1\.3](https://arxiv.org/html/2610.10875#S4.SS1.SSS3.p3.1),[§4\.1\.4](https://arxiv.org/html/2610.10875#S4.SS1.SSS4.p4.1)\.
- \[79\]A\. Tong, K\. Fatras, N\. Malkin, G\. Huguet, Y\. Zhang, J\. Rector\-Brooks, G\. Wolf, and Y\. Bengio\(2024\)Improving and generalizing flow\-based generative models with minibatch optimal transport\.Transactions on Machine Learning Research\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px2.p1.1),[§2\.3](https://arxiv.org/html/2610.10875#S2.SS3.SSS0.Px3.p1.1)\.
- \[80\]P\. Tsimpos, Z\. Ren, J\. Zech, and Y\. Marzouk\(2025\)Optimal scheduling of dynamic transport\.InProceedings of the Thirty Eighth Conference on Learning Theory,N\. Haghtalab and A\. Moitra \(Eds\.\),Proceedings of Machine Learning Research, Vol\.291,pp\. 5441–5505\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px2.p2.1)\.
- \[81\]V\. Vu and A\. Montanari\(2026\)Computational bottlenecks for denoising diffusions\.InInternational Conference on Learning Representations,Cited by:[§1\.1](https://arxiv.org/html/2610.10875#S1.SS1.SSS0.Px1.p2.1),[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px4.p2.1),[§4\.1\.5](https://arxiv.org/html/2610.10875#S4.SS1.SSS5.p7.1)\.
- \[82\]L\. Wu and J\. Long\(2022\)A spectral\-based analysis of the separation between two\-layer neural networks and linear methods\.Journal of Machine Learning Research23\(119\),pp\. 1–34\.Cited by:[§1\.2](https://arxiv.org/html/2610.10875#S1.SS2.SSS0.Px5.p1.1)\.

## Appendix Contents

## Appendix AProofs

This section contains the proofs of various claims from the main paper\.

### A\.1Construction of theDℬD\_\{\\mathcal\{B\}\}metrics

Motivated by dynamical formulations of optimal transport, we define general metrics on the space𝒫⁡\(ℝd\)\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)of Borel probability measures onℝd\{\\mathbb\{R\}\}^\{d\}\. Informally, ifμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)then we say that\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}transportsμ\\mutoν\\nuif we have the continuity equation

∂tρt\+∇⋅\(ρt​vt\)=0,ρ0=μ,ρ1=ν\.\\partial\_\{t\}\\rho\_\{t\}\+\\nabla\\cdot\(\\rho\_\{t\}v\_\{t\}\)=0,\\quad\\rho\_\{0\}=\\mu,\\quad\\rho\_\{1\}=\\nu\.\(76\)Letℬ\\mathcal\{B\}be any Banach space of functions fromℝd→ℝd\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}\. We assume the following mild regularity ofℬ\\mathcal\{B\}to ensure that we can test against smooth compactly\-supported functions:

###### Assumption 1\.

Throughout this paper we assume that for everyφ∈Cc∞​\(ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\),

sup‖v‖ℬ≤1supx∈ℝd\|⟨v⁡\(x\),∇φ​\(x\)⟩\|<∞\.\\sup\_\{\\\|v\\\|\_\{\\mathcal\{B\}\}\\leq 1\}\\sup\_\{x\\in\{\\mathbb\{R\}\}^\{d\}\}\\left\|\\left\\langle v\(x\),\\nabla\\varphi\(x\)\\right\\rangle\\right\|<\\infty\.\(77\)We will always identify vectorsv∈ℬv\\in\\mathcal\{B\}with their representative functionsv⁡\(x\)v\(x\)\.

We can now define a notion of distance between the measuresμ,ν\\mu,\\nuby minimizing an accumulated cost∫01‖vt‖ℬp​𝑑t\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}^\{p\}dtover all transports fromμ\\mutoν\\nu\. A rigorous construction is given below, where we emphasize that the choice ofppdoes not matter due to time\-reparameterization:

###### Proposition 5\.

For anyμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), let𝒮μ→ν\\mathcal\{S\}\_\{\\mu\\to\\nu\}denote the class of pairs\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}s\.t\.:

1. \(i\)t↦ρtt\\mapsto\\rho\_\{t\}is narrowly continuous,
2. \(ii\)vt∈ℬv\_\{t\}\\in\\mathcal\{B\}for a\.e\.ttand the mapst↦‖vt‖ℬt\\mapsto\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}and\(t,x\)↦v⁡\(t,x\)\(t,x\)\\mapsto v\(t,x\)are Borel measurable,
3. \(iii\)∫01∫ℝd\|vt​\(x\)\|​d​ρt​\(x\)​𝑑t<∞,\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\|v\_\{t\}\(x\)\|d\\rho\_\{t\}\(x\)dt<\\infty,
4. \(iv\)the continuity equation \([76](https://arxiv.org/html/2610.10875#A1.E76)\) holds in the weak sense, see \(8\.1\.3\) of\[[5](https://arxiv.org/html/2610.10875#bib.bib69)\]\.

For everyp∈\[1,∞\)p\\in\[1,\\infty\), define the quantity

Dℬ,p​\(μ,ν\):=inf\(vt,ρt\)t∈\[0,1\]∈𝒮μ→ν\(∫01‖vt‖ℬp​𝑑t\)1/p,D\_\{\\mathcal\{B\},p\}\(\\mu,\\nu\):=\\inf\_\{\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}\}\\left\(\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}^\{p\}dt\\right\)^\{1/p\},\(78\)with a value of\+∞\+\\inftyif no admissible transport of finite energy exists\. Then, it holds that

Dℬ,p​\(μ,ν\)=Dℬ,p′​\(μ,ν\)\(∀p,p′∈\[1,∞\)\)\.D\_\{\\mathcal\{B\},p\}\(\\mu,\\nu\)=D\_\{\\mathcal\{B\},p^\{\\prime\}\}\(\\mu,\\nu\)\\quad\(\\forall p,p^\{\\prime\}\\in\[1,\\infty\)\)\.\(79\)

###### Proof\.

It suffices to prove thatDℬ,p​\(μ,ν\)=Dℬ,1​\(μ,ν\)D\_\{\\mathcal\{B\},p\}\(\\mu,\\nu\)=D\_\{\\mathcal\{B\},1\}\(\\mu,\\nu\)for everyp∈\[1,∞\)p\\in\[1,\\infty\)\. Naturally, for any admissible pair Hölder’s inequality implies

∫01‖vt‖ℬ​𝑑t≤\(∫01‖vt‖ℬp​𝑑t\)1/p\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt\\leq\\left\(\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}^\{p\}dt\\right\)^\{1/p\}\(80\)and soDℬ,1​\(μ,ν\)≤Dℬ,p​\(μ,ν\)D\_\{\\mathcal\{B\},1\}\(\\mu,\\nu\)\\leq D\_\{\\mathcal\{B\},p\}\(\\mu,\\nu\)\. IfDℬ,1​\(μ,ν\)=\+∞D\_\{\\mathcal\{B\},1\}\(\\mu,\\nu\)=\+\\inftythen the reverse inequality is trivial, and otherwise let\(vt,ρt\)t∈\[0,1\]∈𝒮μ→ν\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}be admissible with finite length

L:=∫01‖vt‖ℬ​𝑑t<∞\.L:=\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt<\\infty\.\(81\)Fixδ\>0\\delta\>0and define

aδ​\(t\):=∫0t‖vτ‖ℬ​𝑑τ\+δ​tL\+δ\.a\_\{\\delta\}\(t\):=\\frac\{\\int\_\{0\}^\{t\}\\\|v\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\}d\\tau\+\\delta t\}\{L\+\\delta\}\.\(82\)Then,aδ:\[0,1\]→\[0,1\]a\_\{\\delta\}:\[0,1\]\\to\[0,1\]is strictly increasing and absolutely continuous withaδ′​\(t\)=‖vt‖ℬ\+δL\+δ\>0a^\{\\prime\}\_\{\\delta\}\(t\)=\\frac\{\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}\+\\delta\}\{L\+\\delta\}\>0for a\.e\.tt, and so its inverseθδ:=aδ−1\\theta\_\{\\delta\}:=a\_\{\\delta\}^\{\-1\}isL\+δδ\\frac\{L\+\\delta\}\{\\delta\}\-Lipschitz and therefore absolutely continuous\. Define

ρ~τ:=ρθδ​\(τ\),v~τ:=θδ′​\(τ\)​vθδ​\(τ\)\.\\tilde\{\\rho\}\_\{\\tau\}:=\\rho\_\{\\theta\_\{\\delta\}\(\\tau\)\},\\quad\\tilde\{v\}\_\{\\tau\}:=\\theta^\{\\prime\}\_\{\\delta\}\(\\tau\)v\_\{\\theta\_\{\\delta\}\(\\tau\)\}\.\(83\)For anyφ∈Cc∞​\(\(0,1\)×ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\(0,1\)\\times\{\\mathbb\{R\}\}^\{d\}\), if we defineψ⁡\(t,x\):=φ⁡\(aδ​\(t\),x\)\\psi\(t,x\):=\\varphi\(a\_\{\\delta\}\(t\),x\)then testingψ\\psi212121Here, we may testψ\\psiby approximatingaδa\_\{\\delta\}inW1,1W^\{1,1\}by smooth functions and using dominated convergence due to \(iii\)\.against the original continuity equation \([76](https://arxiv.org/html/2610.10875#A1.E76)\) gives

0\\displaystyle 0=∫01∫ℝd\(aδ′​\(τ\)​∂tφ⁡\(aδ​\(τ\),x\)\+⟨vτ​\(x\),∇xφ​\(aδ​\(τ\),x\)⟩\)​d​ρτ​\(x\)​𝑑τ\\displaystyle=\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\(a\_\{\\delta\}^\{\\prime\}\(\\tau\)\\partial\_\{t\}\\varphi\(a\_\{\\delta\}\(\\tau\),x\)\+\\left\\langle v\_\{\\tau\}\(x\),\\nabla\_\{x\}\\varphi\(a\_\{\\delta\}\(\\tau\),x\)\\right\\rangle\\right\)d\\rho\_\{\\tau\}\(x\)d\\tau\(84\)=∫01∫ℝd\(∂tφ⁡\(t,x\)\+⟨θδ′​\(t\)​vθδ​\(t\)​\(x\),∇xφ​\(t,x\)⟩\)​d​ρ~t​\(x\)​𝑑t\\displaystyle=\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\(\\partial\_\{t\}\\varphi\(t,x\)\+\\left\\langle\\theta\_\{\\delta\}^\{\\prime\}\(t\)\{v\}\_\{\\theta\_\{\\delta\}\(t\)\}\(x\),\\nabla\_\{x\}\\varphi\(t,x\)\\right\\rangle\\right\)d\\tilde\{\\rho\}\_\{t\}\(x\)dt\(85\)=∫01∫ℝd\(∂tφ⁡\(t,x\)\+⟨v~t​\(x\),∇xφ​\(t,x\)⟩\)​d​ρ~t​\(x\)​𝑑t,\\displaystyle=\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\(\\partial\_\{t\}\\varphi\(t,x\)\+\\left\\langle\\tilde\{v\}\_\{t\}\(x\),\\nabla\_\{x\}\\varphi\(t,x\)\\right\\rangle\\right\)d\\tilde\{\\rho\}\_\{t\}\(x\)dt,\(86\)where the second line uses the absolutely continuous time changet=aδ​\(τ\)t=a\_\{\\delta\}\(\\tau\)along with the chain rule\. Similarly, change of variables implies

∫01∫ℝd\|v~t\|​d​ρ~t​𝑑t=∫01∫ℝdθδ′​\(t\)​\|vθδ​\(t\)\|​d​ρθδ​\(t\)​𝑑t=∫01∫ℝd\|vτ\|​d​ρτ​𝑑τ<∞\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\|\\tilde\{v\}\_\{t\}\|d\\tilde\{\\rho\}\_\{t\}dt=\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\theta\_\{\\delta\}^\{\\prime\}\(t\)\|v\_\{\\theta\_\{\\delta\}\(t\)\}\|d\\rho\_\{\\theta\_\{\\delta\}\(t\)\}dt=\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\|v\_\{\\tau\}\|d\\rho\_\{\\tau\}d\\tau<\\infty\(87\)and so\(v~t,ρ~t\)t∈\[0,1\]∈𝒮μ→ν\(\\tilde\{v\}\_\{t\},\\tilde\{\\rho\}\_\{t\}\)\_\{t\\in\[0,1\]\}\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}\. Furthermore, for a\.e\.t∈\[0,1\]t\\in\[0,1\]we know

‖v~t‖ℬ=θδ′​\(t\)​‖vθδ​\(t\)‖ℬ=L\+δ‖vθδ​\(t\)‖ℬ\+δ⋅‖vθδ​\(t\)‖ℬ≤L\+δ\\\|\\tilde\{v\}\_\{t\}\\\|\_\{\\mathcal\{B\}\}=\\theta^\{\\prime\}\_\{\\delta\}\(t\)\\\|v\_\{\\theta\_\{\\delta\}\(t\)\}\\\|\_\{\\mathcal\{B\}\}=\\frac\{L\+\\delta\}\{\\\|v\_\{\\theta\_\{\\delta\}\(t\)\}\\\|\_\{\\mathcal\{B\}\}\+\\delta\}\\cdot\\\|v\_\{\\theta\_\{\\delta\}\(t\)\}\\\|\_\{\\mathcal\{B\}\}\\leq L\+\\delta\(88\)and therefore

Dℬ,p​\(μ,ν\)≤\(∫01‖v~t‖ℬp​𝑑t\)1/p≤L\+δ\.D\_\{\\mathcal\{B\},p\}\(\\mu,\\nu\)\\leq\\left\(\\int\_\{0\}^\{1\}\\\|\\tilde\{v\}\_\{t\}\\\|\_\{\\mathcal\{B\}\}^\{p\}dt\\right\)^\{1/p\}\\leq L\+\\delta\.\(89\)Takingδ↓0\\delta\\downarrow 0and taking the infimum over all finite\-length admissible pairs proves the claim\. ∎

###### Definition 5\(DℬD\_\{\\mathcal\{B\}\}metric\)\.

Forμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)we denote the class𝒮μ→ν\\mathcal\{S\}\_\{\\mu\\to\\nu\}as in Proposition[5](https://arxiv.org/html/2610.10875#Thmproposition5)and we denote the shared quantity \([79](https://arxiv.org/html/2610.10875#A1.E79)\) by

Dℬ​\(μ,ν\):=inf\(vt,ρt\)t∈\[0,1\]∈𝒮μ→ν∫01‖vt‖ℬ​𝑑t,D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\):=\\inf\_\{\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt,\(90\)with a value of\+∞\+\\inftyif no admissible transport of finite length exists\.

It is elementary to check the following properties, which justify us callingDℬD\_\{\\mathcal\{B\}\}a metric\. Due to Assumption[1](https://arxiv.org/html/2610.10875#Thmassumption1), this metric produces a topology stronger than the narrow topology:

###### Proposition 6\.

DℬD\_\{\\mathcal\{B\}\}is an extended metric \(i\.e\. a metric which may take the value\+∞\+\\infty\)\. In particular, the following hold for allμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\):

1. \(i\)Dℬ​\(μ,ν\)=Dℬ​\(ν,μ\)D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)=D\_\{\\mathcal\{B\}\}\(\\nu,\\mu\)\.
2. \(ii\)Dℬ​\(μ,ν\)=infρ∈𝒫⁡\(ℝd\)\{Dℬ​\(μ,ρ\)\+Dℬ​\(ρ,ν\)\}D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)=\\inf\_\{\\rho\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\}\\left\\\{D\_\{\\mathcal\{B\}\}\(\\mu,\\rho\)\+D\_\{\\mathcal\{B\}\}\(\\rho,\\nu\)\\right\\\}\.
3. \(iii\)Dℬ​\(μ,ν\)=0⇔μ=νD\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)=0\\iff\\mu=\\nu\.

Furthermore, if\(μn\)n∈ℕ⊆𝒫⁡\(ℝd\)\(\\mu\_\{n\}\)\_\{n\\in\{\\mathbb\{N\}\}\}\\subseteq\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)andμ∈𝒫⁡\(ℝd\)\\mu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)are s\.t\.limn→∞Dℬ​\(μn,μ\)=0\\lim\_\{n\\to\\infty\}D\_\{\\mathcal\{B\}\}\(\\mu\_\{n\},\\mu\)=0then we have the narrow convergenceμn⇀μ\\mu\_\{n\}\\rightharpoonup\\mu\.

###### Proof\.

\(i\) follows from the observation that if\(vt,ρt\)∈𝒮μ→ν\(v\_\{t\},\\rho\_\{t\}\)\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}then\(−v1−t,ρ1−t\)∈𝒮ν→μ\(\-v\_\{1\-t\},\\rho\_\{1\-t\}\)\\in\\mathcal\{S\}\_\{\\nu\\to\\mu\}and‖−v1−t‖ℬ=‖v1−t‖ℬ\\\|\-v\_\{1\-t\}\\\|\_\{\\mathcal\{B\}\}=\\\|v\_\{1\-t\}\\\|\_\{\\mathcal\{B\}\}\.

Our proof of \(ii\) is taken from[this MathOverflow post](https://mathoverflow.net/questions/166307/pde-based-triangle-inequality-for-optimal-transportation)\. Suppose first that

d1:=Dℬ,1​\(μ,ρ\),d2:=Dℬ,1​\(ρ,ν\)d\_\{1\}:=D\_\{\\mathcal\{B\},1\}\(\\mu,\\rho\),\\quad d\_\{2\}:=D\_\{\\mathcal\{B\},1\}\(\\rho,\\nu\)\(91\)are both finite \(if not, the inequality is trivial\)\. Fixε\>0\\varepsilon\>0and let\(vt,ρt\)∈𝒮μ→ρ\(v\_\{t\},\\rho\_\{t\}\)\\in\\mathcal\{S\}\_\{\\mu\\to\\rho\}and\(wt,ηt\)∈𝒮ρ→ν\(w\_\{t\},\\eta\_\{t\}\)\\in\\mathcal\{S\}\_\{\\rho\\to\\nu\}be s\.t\.

∫01‖vt‖ℬ​𝑑t≤d1\+ε,∫01‖wt‖ℬ​𝑑t≤d2\+ε\.\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt\\leq d\_\{1\}\+\\varepsilon,\\quad\\int\_\{0\}^\{1\}\\\|w\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt\\leq d\_\{2\}\+\\varepsilon\.\(92\)Note that ifτ∈\(0,1\)\\tau\\in\(0,1\)and we define

v~t:=\{vt/τ/τt≤τw\(t−τ\)/\(1−τ\)/\(1−τ\)t\>τ,ρ~t:=\{ρt/τt≤τη\(t−τ\)/\(1−τ\)t\>τ,\\tilde\{v\}\_\{t\}:=\\begin\{cases\}v\_\{t/\\tau\}/\\tau&t\\leq\\tau\\\\ w\_\{\(t\-\\tau\)/\(1\-\\tau\)\}/\(1\-\\tau\)&t\>\\tau\\end\{cases\},\\quad\\tilde\{\\rho\}\_\{t\}:=\\begin\{cases\}\\rho\_\{t/\\tau\}&t\\leq\\tau\\\\ \\eta\_\{\(t\-\\tau\)/\(1\-\\tau\)\}&t\>\\tau\\end\{cases\},\(93\)thenρ~τ=ρ\\tilde\{\\rho\}\_\{\\tau\}=\\rhoand\(v~,ρ~\)∈𝒮μ→ν\(\\tilde\{v\},\\tilde\{\\rho\}\)\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}\. Furthermore,

∫01‖v~t‖ℬ​𝑑t\\displaystyle\\int\_\{0\}^\{1\}\\\|\\tilde\{v\}\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt=τ−1​∫0τ‖vt/τ‖ℬ​𝑑t\+\(1−τ\)−1​∫τ1‖w\(t−τ\)/\(1−τ\)‖ℬ​𝑑t\\displaystyle=\{\\tau^\{\-1\}\}\\int\_\{0\}^\{\\tau\}\\\|\{v\}\_\{t/\\tau\}\\\|\_\{\\mathcal\{B\}\}dt\+\{\(1\-\\tau\)^\{\-1\}\}\\int\_\{\\tau\}^\{1\}\\\|\{w\}\_\{\(t\-\\tau\)/\(1\-\\tau\)\}\\\|\_\{\\mathcal\{B\}\}dt\(94\)≤\(d1\+ε\)\+\(d2\+ε\),\\displaystyle\\leq\(d\_\{1\}\+\\varepsilon\)\+\(d\_\{2\}\+\\varepsilon\),\(95\)and therefore taking the infimum over transports implies

Dℬ,1​\(μ,ν\)≤d1\+d2\+2​ε\.D\_\{\\mathcal\{B\},1\}\(\\mu,\\nu\)\\leq d\_\{1\}\+d\_\{2\}\+2\\varepsilon\.\(96\)Takingε↓0\\varepsilon\\downarrow 0shows the≤\\leqdirection; for equality we may takeρ=μ\\rho=\\mu\.

For \(iii\), note that ifDℬ,1​\(μ,ν\)=0D\_\{\\mathcal\{B\},1\}\(\\mu,\\nu\)=0then for eachn∈ℕn\\in\{\\mathbb\{N\}\}there is some\(vtn,ρtn\)∈𝒮μ→ν\(v\_\{t\}^\{n\},\\rho\_\{t\}^\{n\}\)\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}s\.t\.

∫01‖vtn‖ℬ​𝑑t≤1n\.\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{n\}\\\|\_\{\\mathcal\{B\}\}dt\\leq\\frac\{1\}\{n\}\.\(97\)For anyφ∈Cc∞​\(ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\), by testing against the continuity equation we find

∫φ​𝑑ν−∫φ​𝑑μ=∫01∫⟨vtn,∇φ⟩​d​ρtn​𝑑t\.\\int\\varphi d\\nu\-\\int\\varphi d\\mu=\\int\_\{0\}^\{1\}\\int\\left\\langle v\_\{t\}^\{n\},\\nabla\\varphi\\right\\rangle d\\rho\_\{t\}^\{n\}dt\.\(98\)Applying Assumption[1](https://arxiv.org/html/2610.10875#Thmassumption1)and denoting byCφC\_\{\\varphi\}the implied constant, we find

\|∫φ​𝑑ν−∫φ​𝑑μ\|\\displaystyle\\left\|\\int\\varphi d\\nu\-\\int\\varphi d\\mu\\right\|≤Cφ⋅\(∫01‖vtn‖ℬ​𝑑t\)≤Cφn\.\\displaystyle\\leq C\_\{\\varphi\}\\cdot\\left\(\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{n\}\\\|\_\{\\mathcal\{B\}\}dt\\right\)\\leq\\frac\{C\_\{\\varphi\}\}\{n\}\.\(99\)Lettingn→∞n\\to\\inftyimplies∫φ​𝑑ν=∫φ​𝑑μ\\int\\varphi d\\nu=\\int\\varphi d\\mu, and since this holds for allφ∈Cc∞​\(ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\), we findμ=ν\\mu=\\nu\. Ifμ=ν\\mu=\\nuthen the null drift is admissible, proving \(iii\)\.

For the last statement, let\(vt,ρt\)t∈\[0,1\]∈𝒮μ→ν\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}be admissible\. Integrating the continuity equation againstφ\\varphiyields

∫ℝdφ​𝑑ν−∫ℝdφ​𝑑μ=∫01∫ℝd⟨vt,∇φ⟩​d​ρt​𝑑t\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\nu\-\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu=\\int\_\{0\}^\{1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle v\_\{t\},\\nabla\\varphi\\right\\rangle d\\rho\_\{t\}dt\(100\)and so

\|∫ℝdφ​𝑑ν−∫ℝdφ​𝑑μ\|\\displaystyle\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\nu\-\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu\\right\|≤Cφ​∫01‖vt‖ℬ​𝑑t,\\displaystyle\\leq C\_\{\\varphi\}\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt,\(101\)Taking the infimum over admissible transports,

\|∫ℝdφ​𝑑ν−∫ℝdφ​𝑑μ\|≤Cφ​Dℬ​\(μ,ν\)\.\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\nu\-\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu\\right\|\\leq C\_\{\\varphi\}D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)\.\(102\)Suppose now thatDℬ​\(μn,μ\)→0D\_\{\\mathcal\{B\}\}\(\\mu\_\{n\},\\mu\)\\to 0\. The above estimate implies that

∫ℝdφ​d​μn→∫ℝdφ​𝑑μ\(∀φ∈Cc∞​\(ℝd\)\),\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu\_\{n\}\\to\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu\\quad\(\\forall\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\)\),\(103\)from which narrow convergence follows from e\.g\.\[[5](https://arxiv.org/html/2610.10875#bib.bib69), Remark 5\.1\.6\]\. ∎

We finish by showing that theDℋkD\_\{\\mathcal\{H\}\_\{k\}\}andDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}constructions we consider satisfy Assumption[1](https://arxiv.org/html/2610.10875#Thmassumption1):

###### Lemma 2\.

Assumption[1](https://arxiv.org/html/2610.10875#Thmassumption1)is satisfied in either of the following cases:

- •if a kernelk:ℝd×ℝd→ℝd×dk:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\\times d\}satisfies Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)andℬ=ℋk\\mathcal\{B\}=\\mathcal\{H\}\_\{k\}\.
- •if an activationϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfies Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)andℬ=ℬϕ\\mathcal\{B\}=\\mathcal\{B\}\_\{\\phi\}\.

###### Proof\.

Letφ∈Cc∞​\(ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\), and letK:=spt⁡\(φ\)K:=\\operatorname\{spt\}\(\\varphi\)be its compact support set\. It suffices to show that there is someCK<∞C\_\{K\}<\\inftys\.t\.

supx∈K‖v⁡\(x\)‖≤CK​‖v‖ℬ\(∀v∈ℬ\)\.\\sup\_\{x\\in K\}\\\|v\(x\)\\\|\\leq C\_\{K\}\\\|v\\\|\_\{\\mathcal\{B\}\}\\quad\(\\forall v\\in\\mathcal\{B\}\)\.\(104\)In the Stein case, we know that

‖v⁡\(x\)‖≤‖v‖ℋk⋅‖k⁡\(x,x\)‖op1/2,\\\|v\(x\)\\\|\\leq\\\|v\\\|\_\{\\mathcal\{H\}\_\{k\}\}\\cdot\\\|k\(x,x\)\\\|\_\{\\operatorname\{op\}\}^\{1/2\},\(105\)and so compactness ofKKand continuity ofkkyield thatCK:=supx∈K‖k⁡\(x,x\)‖op1/2<∞C\_\{K\}:=\\sup\_\{x\\in K\}\\\|k\(x,x\)\\\|\_\{\\operatorname\{op\}\}^\{1/2\}<\\infty\. In the Barron case, ifϕ\\phiis bounded then we know

‖v⁡\(x\)‖≤‖v‖ℬϕ⋅‖ϕ‖∞,\\\|v\(x\)\\\|\\leq\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\cdot\\\|\\phi\\\|\_\{\\infty\},\(106\)while ifϕ\\phiis positivelyss\-homogeneous then

‖v⁡\(x\)‖≤‖v‖ℬϕ⋅max⁡\{\|ϕ⁡\(1\)\|,\|ϕ⁡\(−1\)\|\}⋅\(1\+‖x‖\)s\.\\\|v\(x\)\\\|\\leq\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\cdot\\max\\\{\|\\phi\(1\)\|,\|\\phi\(\-1\)\|\\\}\\cdot\(1\+\\\|x\\\|\)^\{s\}\.\(107\)∎

### A\.2Wasserstein and KL lower bounds onDℬD\_\{\\mathcal\{B\}\}

In this subsection we prove some general lower bounds on the dynamical transport distances: namely, ifℬ↪Cb​\(ℝd\)\\mathcal\{B\}\\hookrightarrow C\_\{b\}\(\{\\mathbb\{R\}\}^\{d\}\)with sup norm thenW∞≲DℬW\_\{\\infty\}\\lesssim D\_\{\\mathcal\{B\}\}while ifℬ↪W1,∞​\(ℝd\)\\mathcal\{B\}\\hookrightarrow W^\{1,\\infty\}\(\{\\mathbb\{R\}\}^\{d\}\)thenKL≲Dℬ\+Dℬ2\\operatorname\{KL\}\\lesssim D\_\{\\mathcal\{B\}\}\+D\_\{\\mathcal\{B\}\}^\{2\}for sufficiently regular endpoints\. These do not play a role in the results of the main paper, but they are useful tools and offer interesting consequences for our generative modeling applications\.

###### Lemma 3\(Wasserstein and KL control\)\.

Letℬ\\mathcal\{B\}be a Banach space of functions fromℝd→ℝd\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}s\.t\.

‖w‖∞≤M0​‖w‖ℬ\(∀w∈ℬ\)\\\|w\\\|\_\{\\infty\}\\leq M\_\{0\}\\\|w\\\|\_\{\\mathcal\{B\}\}\\quad\(\\forall w\\in\\mathcal\{B\}\)\(108\)for someM0<∞M\_\{0\}<\\infty\. Then, for everyp∈\[1,∞\]p\\in\[1,\\infty\]we have

Wp​\(μ,ν\)≤M0​Dℬ​\(μ,ν\)\(∀μ,ν∈𝒫⁡\(ℝd\)\)\.W\_\{p\}\(\\mu,\\nu\)\\leq M\_\{0\}D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)\\quad\(\\forall\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\)\.\(109\)Suppose further thatℬ↪W1,∞\\mathcal\{B\}\\hookrightarrow W^\{1,\\infty\}continuously and

‖div⁡w‖L∞≤M1​‖w‖ℬ\(∀w∈ℬ\)\\\|\\operatorname\{div\}w\\\|\_\{L^\{\\infty\}\}\\leq M\_\{1\}\\\|w\\\|\_\{\\mathcal\{B\}\}\\quad\(\\forall w\\in\\mathcal\{B\}\)\(110\)for someM1<∞M\_\{1\}<\\infty\. Letμ∝e−V\\mu\\propto e^\{\-V\}be s\.t\.∇V\\nabla Visβ\\beta\-Lipschitz andSμ:=𝔼μ​‖∇V‖<∞S\_\{\\mu\}:=\\mathbb\{E\}\_\{\\mu\}\\\|\\nabla V\\\|<\\infty\. Then,

KL\(ν∥μ\)≤\(M0Sμ\+M1\)Dℬ\(μ,ν\)\+β​M022Dℬ\(μ,ν\)2\(∀ν∈𝒫\(ℝd\)\)\.\\operatorname\{KL\}\(\\nu\\\|\\mu\)\\leq\(M\_\{0\}S\_\{\\mu\}\+M\_\{1\}\)D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)\+\\frac\{\\beta M\_\{0\}^\{2\}\}\{2\}D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)^\{2\}\\quad\(\\forall\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)\)\.\(111\)

###### Proof\.

Fix an admissible\(vt,ρt\)t∈\[0,1\]∈𝒮μ→ν\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}\\in\\mathcal\{S\}\_\{\\mu\\to\\nu\}and writeL:=∫01‖vt‖ℬ​𝑑tL:=\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\}dt\. By the superposition principle for the weak continuity equation, there is a probability measureℚ\\mathbb\{Q\}on absolutely continuous curvesω:\[0,1\]→ℝd\\omega:\[0,1\]\\to\{\\mathbb\{R\}\}^\{d\}s\.t\.\(et\)\#​ℚ=ρt\(e\_\{t\}\)\_\{\\\#\}\\mathbb\{Q\}=\\rho\_\{t\}andω˙t=vt​\(ωt\)\\dot\{\\omega\}\_\{t\}=v\_\{t\}\(\\omega\_\{t\}\)forℚ\\mathbb\{Q\}\-a\.e\.ω\\omega\. Hence,

\|ω1−ω0\|≤∫01‖vt‖∞​𝑑t≤M0​L\|\\omega\_\{1\}\-\\omega\_\{0\}\|\\leq\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\infty\}dt\\leq M\_\{0\}L\(112\)forℚ\\mathbb\{Q\}\-a\.e\.ω\\omega, which means\(e0,e1\)\#​ℚ\(e\_\{0\},e\_\{1\}\)\_\{\\\#\}\\mathbb\{Q\}is a coupling ofμ\\muandν\\nusupported on pairs with distance at mostM0​LM\_\{0\}L\. Taking an infimum over transports yieldsW∞​\(μ,ν\)≤M0​Dℬ​\(μ,ν\)W\_\{\\infty\}\(\\mu,\\nu\)\\leq M\_\{0\}D\_\{\\mathcal\{B\}\}\(\\mu,\\nu\)\.

Now, assume thatℬ↪W1,∞\\mathcal\{B\}\\hookrightarrow W^\{1,\\infty\}continuously and that we take Lipschitz representatives\. This means that

∫01\(‖vt‖∞\+Lip⁡\(vt\)\)​𝑑t<∞\\int\_\{0\}^\{1\}\(\\\|v\_\{t\}\\\|\_\{\\infty\}\+\\operatorname\{Lip\}\(v\_\{t\}\)\)dt<\\infty\(113\)for any finite\-action drifts\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}, which ensures that the corresponding ODE has a unique flowXtX\_\{t\}with\(Xt\)\#​μ=ρt\(X\_\{t\}\)\_\{\\\#\}\\mu=\\rho\_\{t\}; this lets us upgrade from the weak continuity equation to this ODE flow for any finite\-action path\. Now, the change of variables and Jacobian formulas imply

KL\(ν∥μ\)=∫ℝd\(V\(X1\(x\)\)−V\(x\)−∫01divvt\(Xt\(x\)\)dt\)dμ\(x\)\.\\operatorname\{KL\}\(\\nu\\\|\\mu\)=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\(V\(X\_\{1\}\(x\)\)\-V\(x\)\-\\int\_\{0\}^\{1\}\\operatorname\{div\}v\_\{t\}\(X\_\{t\}\(x\)\)dt\\right\)d\\mu\(x\)\.\(114\)Since∇V\\nabla Visβ\\beta\-Lipschitz we know

V⁡\(X1​\(x\)\)−V⁡\(x\)≤⟨∇V​\(x\),X1​\(x\)−x⟩\+β2​‖X1​\(x\)−x‖2\.V\(X\_\{1\}\(x\)\)\-V\(x\)\\leq\\left\\langle\\nabla V\(x\),X\_\{1\}\(x\)\-x\\right\\rangle\+\\frac\{\\beta\}\{2\}\\\|X\_\{1\}\(x\)\-x\\\|^\{2\}\.\(115\)We know‖X1​\(x\)−x‖≤∫01‖vt‖∞​𝑑t≤M0​L\\\|X\_\{1\}\(x\)\-x\\\|\\leq\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\infty\}dt\\leq M\_\{0\}Las before, and also

−∫01divvt\(Xt\(x\)\)dt≤∫01∥divvt∥L∞dt≤M1L\.\-\\int\_\{0\}^\{1\}\\operatorname\{div\}v\_\{t\}\(X\_\{t\}\(x\)\)dt\\leq\\int\_\{0\}^\{1\}\\\|\\operatorname\{div\}v\_\{t\}\\\|\_\{L^\{\\infty\}\}dt\\leq M\_\{1\}L\.\(116\)Plugging these into \([114](https://arxiv.org/html/2610.10875#A1.E114)\) yields that

KL\(ν∥μ\)≤M0L∫ℝd∥∇V∥dμ\+β​M022L2\+M1L,\\operatorname\{KL\}\(\\nu\\\|\\mu\)\\leq M\_\{0\}L\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|\\nabla V\\\|d\\mu\+\\frac\{\\beta M\_\{0\}^\{2\}\}\{2\}L^\{2\}\+M\_\{1\}L,\(117\)after which the claim follows by taking an infimum over transports\. ∎

In the Stein caseℬ=ℋk\\mathcal\{B\}=\\mathcal\{H\}\_\{k\}, the properties \([108](https://arxiv.org/html/2610.10875#A1.E108)\) and \([110](https://arxiv.org/html/2610.10875#A1.E110)\) are implied bysupx‖k⁡\(x,x\)‖op1/2≤M0\\sup\_\{x\}\\\|k\(x,x\)\\\|\_\{\\operatorname\{op\}\}^\{1/2\}\\leq M\_\{0\}andsupx\(divx⁡divy⁡k⁡\(x,y\)\|y=x\)1/2≤M1\\sup\_\{x\}\\left\(\\operatorname\{div\}\_\{x\}\\operatorname\{div\}\_\{y\}k\(x,y\)\|\_\{y=x\}\\right\)^\{1/2\}\\leq M\_\{1\}, respectively\. In the Barron caseℬ=ℬϕ\\mathcal\{B\}=\\mathcal\{B\}\_\{\\phi\}, they are implied by‖ϕ‖∞≤M0\\\|\\phi\\\|\_\{\\infty\}\\leq M\_\{0\}and‖ϕ′‖L∞≤M1\\\|\\phi^\{\\prime\}\\\|\_\{L^\{\\infty\}\}\\leq M\_\{1\}, respectively\. This directly implies Proposition[2](https://arxiv.org/html/2610.10875#Thmproposition2):

###### Proof of Proposition[2](https://arxiv.org/html/2610.10875#Thmproposition2)\.

TheW∞W\_\{\\infty\}bound follows from Lemma[3](https://arxiv.org/html/2610.10875#Thmlemma3)\. For the KL bound, the assumptions of Lemma[3](https://arxiv.org/html/2610.10875#Thmlemma3)hold withM0,M1=MM\_\{0\},M\_\{1\}=M, and withμ=γd\\mu=\\gamma\_\{d\}we haveβ=1\\beta=1andSγd≤dS\_\{\\gamma\_\{d\}\}\\leq\\sqrt\{d\}\. Thus, withK:=KL\(π∥γd\)K:=\\operatorname\{KL\}\(\\pi\\\|\\gamma\_\{d\}\)andD:=Dℬϕ​\(γd,π\)D:=D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)we find

K≤M⁡\(d\+1\)​D\+M22​D2≤2​M​d​D\+M22​D2\.K\\leq M\(\\sqrt\{d\}\+1\)D\+\\frac\{M^\{2\}\}\{2\}D^\{2\}\\leq 2M\\sqrt\{d\}D\+\\frac\{M^\{2\}\}\{2\}D^\{2\}\.\(118\)At least one term on the right must be at leastK/2K/2, and so eitherM​D≥K4​dMD\\geq\\frac\{K\}\{4\\sqrt\{d\}\}orM​D≥KMD\\geq\\sqrt\{K\}\. ∎

### A\.3Gaussian\-to\-Gaussian transport

We first analyze the optimal and reverse OU transports in the setting where the source and target are both Gaussian\. ForW2W\_\{2\}optimal transport and reverse OU the drifts are always affine, which means that intermediate measures remain Gaussian and so we can analyze mean and covariance dynamics explicitly without relying on perturbative arguments\. For theDℋkD\_\{\\mathcal\{H\}\_\{k\}\}andDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}settings we must impose this structure ourselves, i\.e\. restrict to kernels and activations which are affine\.

For simplicity, throughout this subsection we takeγd=𝒩⁡\(0,𝟙d\)\\gamma\_\{d\}=\\mathcal\{N\}\(0,\{\\mathbbm\{1\}\}\_\{d\}\)the standard Gaussian andπ=𝒩⁡\(m,Σ\)\\pi=\\mathcal\{N\}\(m,\\Sigma\)an arbitrary nondegenerate Gaussian withΣ≻0\\Sigma\\succ 0\. The general case should follow through similar methods\.

#### A\.3\.1Optimal transport distances

First, we recall that the Wasserstein distance has an exact form for Gaussians:

###### Lemma 4\(Example 1\.19 of\[[22](https://arxiv.org/html/2610.10875#bib.bib67)\]\)\.

W2​\(γd,π\)=‖m‖2\+‖Σ1/2−𝟙d‖F2W\_\{2\}\(\\gamma\_\{d\},\\pi\)=\\sqrt\{\\\|m\\\|^\{2\}\+\\\|\\Sigma^\{1/2\}\-\{\\mathbbm\{1\}\}\_\{d\}\\\|\_\{F\}^\{2\}\}\.

Moving on to the Stein case, we restrict to bilinear kernels in order to ensure affine drifts\. This lets us reduce to a finite\-dimensional matrix control problem which can be solved up to universal constants:

###### Lemma 5\(Stein distance between Gaussians\)\.

Letα\>0\\alpha\>0andA∈ℝd×dA\\in\{\\mathbb\{R\}\}^\{d\\times d\}be symmetric and positive definite, and letk⁡\(x,y\):=\(α\+x⊤​A​y\)​𝟙dk\(x,y\):=\(\\alpha\+x^\{\\top\}Ay\)\{\\mathbbm\{1\}\}\_\{d\}\(which implies Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)is satisfied\)\. If we denote

D:=1‖A‖op​\(log⁡\(1\+‖A‖opα​‖m‖\)\+12​‖log⁡Σ‖F\),D:=\\frac\{1\}\{\\sqrt\{\\\|A\\\|\_\{\\operatorname\{op\}\}\}\}\\left\(\\log\\left\(1\+\\sqrt\{\\frac\{\\\|A\\\|\_\{\\operatorname\{op\}\}\}\{\\alpha\}\}\\\|m\\\|\\right\)\+\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{F\}\\right\),\(119\)for∥⋅∥F\\\|\\cdot\\\|\_\{F\}the Frobenius norm, then withκ⁡\(A\)\\kappa\(A\)the condition number ofAAit holds that

12​D≤Dℋk​\(γd,π\)≤2​κ⁡\(A\)​D\.\\frac\{1\}\{2\}D\\leq\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\}\\leq 2\\sqrt\{\\kappa\(A\)\}D\.\(120\)In particular, withk⁡\(x,y\)=\(1\+⟨x,y⟩\)​𝟙dk\(x,y\)=\(1\+\\left\\langle x,y\\right\\rangle\)\{\\mathbbm\{1\}\}\_\{d\}we have

Dℋk​\(γd,π\)≍log⁡\(1\+‖m‖\)\+‖log⁡Σ‖F\.D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\\asymp\\log\\left\(1\+\\\|m\\\|\\right\)\+\\\|\\log\\Sigma\\\|\_\{F\}\.\(121\)

###### Proof\.

Let us denotea:=‖A‖opa:=\\\|A\\\|\_\{\\operatorname\{op\}\}\. Everyv∈ℋkv\\in\\mathcal\{H\}\_\{k\}is of the formv⁡\(x\)=M​x\+cv\(x\)=Mx\+cwith the norm

∥v∥ℋk2=∥MA−1/2∥F2\+‖c‖2α\.\\\|v\\\|\_\{\\mathcal\{H\}\_\{k\}\}^\{2\}=\\\|MA^\{\-1/2\}\\\|\_\{F\}^\{2\}\+\\frac\{\\\|c\\\|^\{2\}\}\{\\alpha\}\.\(122\)Since these are affine, any finite\-length admissible transport starting fromγd\\gamma\_\{d\}remains Gaussian; we write

ρt=𝒩⁡\(mt,Σt\),vt​\(x\)=Mt​x\+ct\\rho\_\{t\}=\\mathcal\{N\}\(m\_\{t\},\\Sigma\_\{t\}\),\\quad v\_\{t\}\(x\)=M\_\{t\}x\+c\_\{t\}\(123\)which follow the dynamics

m˙t=Mt​mt\+ct,Σ˙t=Mt​Σt\+Σt​Mt⊤,m0=0,Σ0=𝟙d\.\\dot\{m\}\_\{t\}=M\_\{t\}m\_\{t\}\+c\_\{t\},\\quad\\dot\{\\Sigma\}\_\{t\}=M\_\{t\}\\Sigma\_\{t\}\+\\Sigma\_\{t\}M\_\{t\}^\{\\top\},\\quad m\_\{0\}=0,\\Sigma\_\{0\}=\{\\mathbbm\{1\}\}\_\{d\}\.\(124\)Let us denote

ℓt:=∥vt∥ℋk=\(∥MtA−1/2∥F2\+‖ct‖2α\)1/2\.\\ell\_\{t\}:=\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}=\\left\(\\\|M\_\{t\}A^\{\-1/2\}\\\|\_\{F\}^\{2\}\+\\frac\{\\\|c\_\{t\}\\\|^\{2\}\}\{\\alpha\}\\right\)^\{1/2\}\.\(125\)If we letLtL\_\{t\}solve the dynamics

L˙t=Mt​Lt,L0=𝟙d\\dot\{L\}\_\{t\}=M\_\{t\}L\_\{t\},\\quad L\_\{0\}=\{\\mathbbm\{1\}\}\_\{d\}\(126\)thenΣt=Lt​Lt⊤\\Sigma\_\{t\}=L\_\{t\}L\_\{t\}^\{\\top\}\. Ifs1​\(t\),…,sd​\(t\)s\_\{1\}\(t\),\\ldots,s\_\{d\}\(t\)are the singular values ofLtL\_\{t\}then we may take an SVDLt=Ut​St​Vt⊤L\_\{t\}=U\_\{t\}S\_\{t\}V\_\{t\}^\{\\top\}withSt=diag⁡\(s1​\(t\),…,sd​\(t\)\)S\_\{t\}=\\operatorname\{diag\}\(s\_\{1\}\(t\),\\ldots,s\_\{d\}\(t\)\)\. In other words,Lt​vj=sj​\(t\)​ujL\_\{t\}v\_\{j\}=s\_\{j\}\(t\)u\_\{j\}withuj,vju\_\{j\},v\_\{j\}the left and right unit singular vectors, after which the standard formula yields

sj′​\(t\)=uj⊤​L˙t​vj=uj⊤​Mt​Lt​vj=sj​\(t\)​uj⊤​Mt​ujs\_\{j\}^\{\\prime\}\(t\)=u\_\{j\}^\{\\top\}\\dot\{L\}\_\{t\}v\_\{j\}=u\_\{j\}^\{\\top\}M\_\{t\}L\_\{t\}v\_\{j\}=s\_\{j\}\(t\)u\_\{j\}^\{\\top\}M\_\{t\}u\_\{j\}\(127\)whenever singular values do not cross, and so for a\.e\.ttwe have

dd​t​log⁡sj​\(t\)=uj⊤​Mt​uj\.\\frac\{d\}\{dt\}\\log s\_\{j\}\(t\)=u\_\{j\}^\{\\top\}M\_\{t\}u\_\{j\}\.\(128\)Sinceuj⊤​Mt​uju\_\{j\}^\{\\top\}M\_\{t\}u\_\{j\}are the diagonal entries ofUt⊤​Mt​UtU\_\{t\}^\{\\top\}M\_\{t\}U\_\{t\}we find

∑j=1d\|uj⊤​Mt​uj\|2≤∑i,j=1d\|\(Ut⊤​Mt​Ut\)i​j\|2=‖Ut⊤​Mt​Ut‖F2=‖Mt‖F2\\sum\_\{j=1\}^\{d\}\|u\_\{j\}^\{\\top\}M\_\{t\}u\_\{j\}\|^\{2\}\\leq\\sum\_\{i,j=1\}^\{d\}\\left\|\(U\_\{t\}^\{\\top\}M\_\{t\}U\_\{t\}\)\_\{ij\}\\right\|^\{2\}=\\\|U\_\{t\}^\{\\top\}M\_\{t\}U\_\{t\}\\\|\_\{F\}^\{2\}=\\\|M\_\{t\}\\\|\_\{F\}^\{2\}\(129\)and therefore we have

\(∑j=1d\|dd​t​log⁡sj​\(t\)\|2\)1/2≤‖Mt‖F\.\\left\(\\sum\_\{j=1\}^\{d\}\\left\|\\frac\{d\}\{dt\}\\log s\_\{j\}\(t\)\\right\|^\{2\}\\right\)^\{1/2\}\\leq\\\|M\_\{t\}\\\|\_\{F\}\.\(130\)Integrating over time and using thatlog⁡sj​\(0\)=0\\log s\_\{j\}\(0\)=0,

\(∑j=1d\|log⁡sj​\(1\)\|2\)1/2=‖log⁡s⁡\(1\)‖2≤∫01‖dd​t​log⁡s⁡\(t\)‖2​𝑑t≤∫01‖Mt‖F​𝑑t\.\\left\(\\sum\_\{j=1\}^\{d\}\|\\log s\_\{j\}\(1\)\|^\{2\}\\right\)^\{1/2\}=\\\|\\log s\(1\)\\\|\_\{2\}\\leq\\int\_\{0\}^\{1\}\\left\\\|\\frac\{d\}\{dt\}\\log s\(t\)\\right\\\|\_\{2\}dt\\leq\\int\_\{0\}^\{1\}\\\|M\_\{t\}\\\|\_\{F\}dt\.\(131\)In particular, since the eigenvalues ofΣ\\Sigmaares1​\(1\)2,…,sd​\(1\)2s\_\{1\}\(1\)^\{2\},\\ldots,s\_\{d\}\(1\)^\{2\}we find that

‖log⁡Σ‖F2=∑j=1d\|log⁡\(sj​\(1\)2\)\|2=4​‖log⁡s⁡\(1\)‖22\\\|\\log\\Sigma\\\|\_\{F\}^\{2\}=\\sum\_\{j=1\}^\{d\}\\left\|\\log\(s\_\{j\}\(1\)^\{2\}\)\\right\|^\{2\}=4\\\|\\log s\(1\)\\\|\_\{2\}^\{2\}\(132\)and hence

12​‖log⁡Σ‖F≤∫01‖Mt‖F​𝑑t≤a​∫01ℓt​𝑑t\.\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{F\}\\leq\\int\_\{0\}^\{1\}\\\|M\_\{t\}\\\|\_\{F\}dt\\leq\\sqrt\{a\}\\int\_\{0\}^\{1\}\\ell\_\{t\}dt\.\(133\)For the mean, ifrt:=‖mt‖r\_\{t\}:=\\\|m\_\{t\}\\\|then

r˙t≤∥Mtmt∥\+∥ct∥≤art∥MtA−1/2∥F\+α‖ct‖α≤a​rt2\+α⋅\(∥MtA−1/2∥F2\+‖ct‖2α\)1/2\\dot\{r\}\_\{t\}\\leq\\\|M\_\{t\}m\_\{t\}\\\|\+\\\|c\_\{t\}\\\|\\leq\\sqrt\{a\}r\_\{t\}\\\|M\_\{t\}A^\{\-1/2\}\\\|\_\{F\}\+\\sqrt\{\\alpha\}\\frac\{\\\|c\_\{t\}\\\|\}\{\\sqrt\{\\alpha\}\}\\leq\\sqrt\{ar\_\{t\}^\{2\}\+\\alpha\}\\cdot\\left\(\\\|M\_\{t\}A^\{\-1/2\}\\\|\_\{F\}^\{2\}\+\\frac\{\\\|c\_\{t\}\\\|^\{2\}\}\{\\alpha\}\\right\)^\{1/2\}\(134\)and so

1a​dd​t​\[sinh−1⁡\(aα​rt\)\]=r˙ta​rt2\+α≤ℓt\.\\frac\{1\}\{\\sqrt\{a\}\}\\frac\{d\}\{dt\}\\left\[\\sinh^\{\-1\}\\left\(\\sqrt\{\\frac\{a\}\{\\alpha\}\}r\_\{t\}\\right\)\\right\]=\\frac\{\\dot\{r\}\_\{t\}\}\{\\sqrt\{ar\_\{t\}^\{2\}\+\\alpha\}\}\\leq\\ell\_\{t\}\.\(135\)Integrating in time and using thatr0=0r\_\{0\}=0, we find

sinh−1⁡\(aα​‖m‖\)≤a​∫01ℓt​𝑑t\.\\sinh^\{\-1\}\\left\(\\sqrt\{\\frac\{a\}\{\\alpha\}\}\\\|m\\\|\\right\)\\leq\\sqrt\{a\}\\int\_\{0\}^\{1\}\\ell\_\{t\}dt\.\(136\)We may now take an infimum over finite\-energy transports\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}and use thatsinh−1⁡\(z\)≥log⁡\(1\+z\)\\sinh^\{\-1\}\(z\)\\geq\\log\(1\+z\)to find

Dℋk​\(γd,π\)≥1a​max​\{12​‖log⁡Σ‖F,log⁡\(1\+aα​‖m‖\)\}≥12​D\.D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\\geq\\frac\{1\}\{\\sqrt\{a\}\}\\max\\left\\\{\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{F\},\\;\\log\\left\(1\+\\sqrt\{\\frac\{a\}\{\\alpha\}\}\\\|m\\\|\\right\)\\right\\\}\\geq\\frac\{1\}\{2\}D\.\(137\)For the upper bound, we start with the triangle inequality: withπ0:=𝒩⁡\(0,Σ\)\\pi\_\{0\}:=\\mathcal\{N\}\(0,\\Sigma\)the centered Gaussian with correct variance we have

Dℋk​\(γd,π\)≤Dℋk​\(γd,π0\)\+Dℋk​\(π0,π\)D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\\leq D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\_\{0\}\)\+D\_\{\\mathcal\{H\}\_\{k\}\}\(\\pi\_\{0\},\\pi\)\(138\)by Proposition[6](https://arxiv.org/html/2610.10875#Thmproposition6)\(ii\)\. To bound the first term, we takeρt:=𝒩⁡\(0,Σt\)\\rho\_\{t\}:=\\mathcal\{N\}\(0,\\Sigma^\{t\}\)andvt​\(x\):=Mt​xv\_\{t\}\(x\):=M\_\{t\}xwithMt:=12​log⁡ΣM\_\{t\}:=\\frac\{1\}\{2\}\\log\\Sigma, so that we have

dd​t​Σt=Mt​Σt\+Σt​Mt\.\\frac\{d\}\{dt\}\\Sigma^\{t\}=M\_\{t\}\\Sigma^\{t\}\+\\Sigma^\{t\}M\_\{t\}\.\(139\)This is an admissible transport with time\-invariant drift, and we use \([125](https://arxiv.org/html/2610.10875#A1.E125)\) to calculate

Dℋk\(γd,π0\)≤∫01∥vt∥ℋkdt=12∥\(logΣ\)A−1/2∥F≤12​λmin​\(A\)∥logΣ∥F\.D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\_\{0\}\)\\leq\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}dt=\\frac\{1\}\{2\}\\\|\(\\log\\Sigma\)A^\{\-1/2\}\\\|\_\{F\}\\leq\\frac\{1\}\{2\\sqrt\{\\lambda\_\{\\text\{min\}\}\(A\)\}\}\\\|\\log\\Sigma\\\|\_\{F\}\.\(140\)Ifm=0m=0the desired bound trivially holds; otherwise, writeu:=m‖m‖u:=\\frac\{m\}\{\\\|m\\\|\}\. We define the affine maps

Tt​\(x\):=\(𝟙\+\(s⁡\(t\)−1\)​u​u⊤\)​x\+h⁡\(t\)​uT\_\{t\}\(x\):=\(\{\\mathbbm\{1\}\}\+\(s\(t\)\-1\)uu^\{\\top\}\)x\+h\(t\)u\(141\)fors:\[0,1\]→\(0,∞\)s:\[0,1\]\\to\(0,\\infty\)andh:\[0,1\]→ℝh:\[0,1\]\\to\{\\mathbb\{R\}\}to be determined\. We takes⁡\(0\)=s⁡\(1\)=1s\(0\)=s\(1\)=1,h⁡\(0\)=0h\(0\)=0, andh⁡\(1\)=‖m‖h\(1\)=\\\|m\\\|to ensure that if we defineρt:=\(Tt\)\#​π0\\rho\_\{t\}:=\(T\_\{t\}\)\_\{\\\#\}\\pi\_\{0\}thenρ0=π0\\rho\_\{0\}=\\pi\_\{0\}andρ1=π\\rho\_\{1\}=\\pi\. The Eulerian velocity of this path isvt​\(x\)=Mt​x\+ctv\_\{t\}\(x\)=M\_\{t\}x\+c\_\{t\}for

Mt=s′​\(t\)s⁡\(t\)​u​u⊤,ct=\(h′​\(t\)−s′​\(t\)s⁡\(t\)​h​\(t\)\)​uM\_\{t\}=\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}uu^\{\\top\},\\quad c\_\{t\}=\\left\(h^\{\\prime\}\(t\)\-\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}h\(t\)\\right\)u\(142\)and so by \([125](https://arxiv.org/html/2610.10875#A1.E125)\) we have

‖vt‖ℋk2=β​\(s′​\(t\)s⁡\(t\)\)2\+1α​\(h′​\(t\)−s′​\(t\)s⁡\(t\)​h​\(t\)\)2,\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}^\{2\}=\\beta\\left\(\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}\\right\)^\{2\}\+\\frac\{1\}\{\\alpha\}\\left\(h^\{\\prime\}\(t\)\-\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}h\(t\)\\right\)^\{2\},\(143\)where we denoteβ:=u⊤​A−1​u\\beta:=u^\{\\top\}A^\{\-1\}u\. Hence, if we change variables to

yt:=1s⁡\(t\),zt:=h⁡\(t\)α​β​s​\(t\)y\_\{t\}:=\\frac\{1\}\{s\(t\)\},\\quad z\_\{t\}:=\\frac\{h\(t\)\}\{\\sqrt\{\\alpha\\beta\}s\(t\)\}\(144\)then the derivatives satisfy

y˙t=−s′​\(t\)s​\(t\)2,z˙t=1α​β​h′​\(t\)​s​\(t\)−h⁡\(t\)​s′​\(t\)s​\(t\)2\.\\dot\{y\}\_\{t\}=\\frac\{\-s^\{\\prime\}\(t\)\}\{s\(t\)^\{2\}\},\\quad\\dot\{z\}\_\{t\}=\\frac\{1\}\{\\sqrt\{\\alpha\\beta\}\}\\frac\{h^\{\\prime\}\(t\)s\(t\)\-h\(t\)s^\{\\prime\}\(t\)\}\{s\(t\)^\{2\}\}\.\(145\)Using \([143](https://arxiv.org/html/2610.10875#A1.E143)\) we find

∫01‖vt‖ℋk​𝑑t=β⋅∫01y˙t2\+z˙t2yt​𝑑t\.\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}dt=\\sqrt\{\\beta\}\\cdot\\int\_\{0\}^\{1\}\\frac\{\\sqrt\{\\dot\{y\}\_\{t\}^\{2\}\+\\dot\{z\}\_\{t\}^\{2\}\}\}\{y\_\{t\}\}dt\.\(146\)This is exactlyβ\\sqrt\{\\beta\}times the length of the curvew⁡\(t\):=\(zt,yt\)∈ℍw\(t\):=\(z\_\{t\},y\_\{t\}\)\\in\\mathbb\{H\}, whereℍ:=\{\(x,y\):y\>0\}\\mathbb\{H\}:=\\\{\(x,y\):\\;y\>0\\\}is the Poincaré upper half\-plane with metricd​w2=d​x2\+d​y2y2dw^\{2\}=\\frac\{dx^\{2\}\+dy^\{2\}\}\{y^\{2\}\}\. Using the[formula for the distance in the metric spaceℍ\\mathbb\{H\}](https://en.wikipedia.org/wiki/Poincar%C3%A9_half-plane_model#Distance_calculation), ifw:\[0,1\]→ℍw:\[0,1\]\\to\\mathbb\{H\}is a geodesic then

∫01‖vt‖ℋk​𝑑t\\displaystyle\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}dt=β⋅dℍ​\(w⁡\(0\),w⁡\(1\)\)\\displaystyle=\\sqrt\{\\beta\}\\cdot d\_\{\\mathbb\{H\}\}\(w\(0\),w\(1\)\)\(147\)=β⋅2​sinh−1⁡\(‖w⁡\(1\)−w⁡\(0\)‖ℝ22​y0​y1\)\\displaystyle=\\sqrt\{\\beta\}\\cdot 2\\sinh^\{\-1\}\\left\(\\frac\{\\\|w\(1\)\-w\(0\)\\\|\_\{\{\\mathbb\{R\}\}^\{2\}\}\}\{2\\sqrt\{y\_\{0\}y\_\{1\}\}\}\\right\)\(148\)=β⋅2​sinh−1⁡\(‖m‖2​α​β\)\.\\displaystyle=\\sqrt\{\\beta\}\\cdot 2\\sinh^\{\-1\}\\left\(\\frac\{\\\|m\\\|\}\{2\\sqrt\{\\alpha\\beta\}\}\\right\)\.\(149\)We takew⁡\(t\)=\(zt,yt\)w\(t\)=\(z\_\{t\},y\_\{t\}\)to be any constant\-speed hyperbolic geodesic fromw⁡\(0\)=\(0,1\)w\(0\)=\(0,1\)tow⁡\(1\)=\(‖m‖/α​β,1\)w\(1\)=\(\\\|m\\\|/\\sqrt\{\\alpha\\beta\},1\)and defines=1/ys=1/yandh=α​β​z/yh=\\sqrt\{\\alpha\\beta\}z/yin order to ensure that this holds\. Sincesinh−1⁡\(z/2\)≤log⁡\(1\+z\)\\sinh^\{\-1\}\(z/2\)\\leq\\log\(1\+z\)and1a≤β≤1λmin​\(A\)\\frac\{1\}\{a\}\\leq\\beta\\leq\\frac\{1\}\{\\lambda\_\{\\text\{min\}\}\(A\)\}we find

Dℋk​\(π0,π\)≤1λmin​\(A\)⋅2​log⁡\(1\+aα​‖m‖\)\.D\_\{\\mathcal\{H\}\_\{k\}\}\(\\pi\_\{0\},\\pi\)\\leq\\frac\{1\}\{\\sqrt\{\\lambda\_\{\\text\{min\}\}\(A\)\}\}\\cdot 2\\log\\left\(1\+\\sqrt\{\\frac\{a\}\{\\alpha\}\}\\\|m\\\|\\right\)\.\(150\)Combining this with \([140](https://arxiv.org/html/2610.10875#A1.E140)\) and the triangle inequality, we find

Dℋk​\(γd,π\)≤1λmin​\(A\)⋅\(12​‖log⁡Σ‖F\+2​log⁡\(1\+aα​‖m‖\)\)≤2​κ⁡\(A\)​D\.D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\\leq\\frac\{1\}\{\\sqrt\{\\lambda\_\{\\text\{min\}\}\(A\)\}\}\\cdot\\left\(\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{F\}\+2\\log\\left\(1\+\\sqrt\{\\frac\{a\}\{\\alpha\}\}\\\|m\\\|\\right\)\\right\)\\leq 2\\sqrt\{\\kappa\(A\)\}D\.\(151\)∎

The Barron setting with identity activation has more or less the same behavior, with the key difference of changing the matrix norm from Frobenius \(ℓ2\\ell^\{2\}on singular values\) to nuclear \(ℓ1\\ell^\{1\}on singular values\):

###### Lemma 6\(Barron distance between Gaussians\)\.

Letϕ⁡\(z\)=z\\phi\(z\)=z\(which satisfies Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)\)\. Then, with∥⋅∥∗\\\|\\cdot\\\|\_\{\*\}denoting the nuclear norm it holds that

12​\(log⁡\(1\+‖m‖\)\+12​‖log⁡Σ‖∗\)≤Dℬϕ​\(γd,π\)≤3​\(log⁡\(1\+‖m‖\)\+12​‖log⁡Σ‖∗\)\.\\frac\{1\}\{2\}\\left\(\\log\\left\(1\+\\\|m\\\|\\right\)\+\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}\\right\)\\leq D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\\leq 3\\left\(\\log\\left\(1\+\\\|m\\\|\\right\)\+\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}\\right\)\.\(152\)

###### Proof\.

As in the proof of Lemma[5](https://arxiv.org/html/2610.10875#Thmlemma5), sincev∈ℬϕv\\in\\mathcal\{B\}\_\{\\phi\}hasv⁡\(x\)=M​x\+cv\(x\)=Mx\+cthen any finite\-length admissible transport starting fromγd\\gamma\_\{d\}remains Gaussian\. Letρt,vt,Mt,ct,mt,Σt,Lt\\rho\_\{t\},v\_\{t\},M\_\{t\},c\_\{t\},m\_\{t\},\\Sigma\_\{t\},L\_\{t\}be as in \([123](https://arxiv.org/html/2610.10875#A1.E123)\), \([124](https://arxiv.org/html/2610.10875#A1.E124)\), and \([126](https://arxiv.org/html/2610.10875#A1.E126)\), lets1​\(t\),…,sd​\(t\)s\_\{1\}\(t\),\\ldots,s\_\{d\}\(t\)be the singular values ofLtL\_\{t\}, and let us denote

ℓt:=‖vt‖ℬϕ=‖Mt‖∗\+‖ct‖\.\\ell\_\{t\}:=\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\\|M\_\{t\}\\\|\_\{\*\}\+\\\|c\_\{t\}\\\|\.\(153\)If we replace the estimate \([129](https://arxiv.org/html/2610.10875#A1.E129)\) with itsℓ1\\ell\_\{1\}version

∑j=1d\|uj⊤​Mt​uj\|≤‖Mt‖∗\\sum\_\{j=1\}^\{d\}\|u\_\{j\}^\{\\top\}M\_\{t\}u\_\{j\}\|\\leq\\\|M\_\{t\}\\\|\_\{\*\}\(154\)then we instead find that

‖log⁡s⁡\(1\)‖1≤∫01‖dd​t​log⁡s⁡\(t\)‖1​𝑑t≤∫01‖Mt‖∗​𝑑t\.\\\|\\log s\(1\)\\\|\_\{1\}\\leq\\int\_\{0\}^\{1\}\\left\\\|\\frac\{d\}\{dt\}\\log s\(t\)\\right\\\|\_\{1\}dt\\leq\\int\_\{0\}^\{1\}\\\|M\_\{t\}\\\|\_\{\*\}dt\.\(155\)In particular, since the eigenvalues ofΣ\\Sigmaares1​\(1\)2,…,sd​\(1\)2s\_\{1\}\(1\)^\{2\},\\ldots,s\_\{d\}\(1\)^\{2\}we have

12​‖log⁡Σ‖∗=‖log⁡s⁡\(1\)‖1≤∫01ℓt​𝑑t\.\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}=\\\|\\log s\(1\)\\\|\_\{1\}\\leq\\int\_\{0\}^\{1\}\\ell\_\{t\}dt\.\(156\)For the mean, withrt:=‖mt‖r\_\{t\}:=\\\|m\_\{t\}\\\|then

r˙t≤‖Mt​mt‖\+‖ct‖≤rt​‖Mt‖op\+‖ct‖≤rt​‖Mt‖∗\+\|ct\|≤\(1\+rt\)​ℓt,\\dot\{r\}\_\{t\}\\leq\\\|M\_\{t\}m\_\{t\}\\\|\+\\\|c\_\{t\}\\\|\\leq r\_\{t\}\\\|M\_\{t\}\\\|\_\{\\operatorname\{op\}\}\+\\\|c\_\{t\}\\\|\\leq r\_\{t\}\\\|M\_\{t\}\\\|\_\{\*\}\+\\\|c\_\{t\}\\\|\\leq\(1\+r\_\{t\}\)\\ell\_\{t\},\(157\)or in other words

dd​t​log⁡\(1\+rt\)≤ℓt\.\\frac\{d\}\{dt\}\\log\(1\+r\_\{t\}\)\\leq\\ell\_\{t\}\.\(158\)Integrating and applyingr0=0r\_\{0\}=0yields

log⁡\(1\+‖m‖\)≤∫01ℓt​𝑑t,\\log\(1\+\\\|m\\\|\)\\leq\\int\_\{0\}^\{1\}\\ell\_\{t\}dt,\(159\)and so we find

Dℬϕ​\(γd,π\)≥max⁡\{12​‖log⁡Σ‖∗,log⁡\(1\+‖m‖\)\}\.D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\\geq\\max\\left\\\{\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\},\\;\\log\\left\(1\+\\\|m\\\|\\right\)\\right\\\}\.\(160\)For the upper bound, we apply the same triangle inequality

Dℬϕ​\(γd,π\)≤Dℬϕ​\(γd,π0\)\+Dℬϕ​\(π0,π\)D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\\leq D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{0\}\)\+D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi\_\{0\},\\pi\)\(161\)withπ0:=𝒩⁡\(0,Σ\)\\pi\_\{0\}:=\\mathcal\{N\}\(0,\\Sigma\)\. For the first term, we use the interpolationρt=𝒩⁡\(0,Σt\)\\rho\_\{t\}=\\mathcal\{N\}\(0,\\Sigma^\{t\}\)andvt​\(x\)=Mt​xv\_\{t\}\(x\)=M\_\{t\}xwithMt=12​log⁡ΣM\_\{t\}=\\frac\{1\}\{2\}\\log\\Sigmato see that

Dℬϕ​\(γd,π0\)≤12​‖log⁡Σ‖∗\.D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{0\}\)\\leq\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}\.\(162\)Note that in light of our lower bound, the above is equality\.

For the mean term in the upper bound, fixs:\[0,1\]→\(0,∞\)s:\[0,1\]\\to\(0,\\infty\)andh:\[0,1\]→ℝh:\[0,1\]\\to\{\\mathbb\{R\}\}withs⁡\(0\)=s⁡\(1\)=1s\(0\)=s\(1\)=1,h⁡\(0\)=0h\(0\)=0, andh⁡\(1\)=‖m‖h\(1\)=\\\|m\\\|to be determined\. DefineTtT\_\{t\}as in \([141](https://arxiv.org/html/2610.10875#A1.E141)\) andρt:=\(Tt\)\#​π0\\rho\_\{t\}:=\(T\_\{t\}\)\_\{\\\#\}\\pi\_\{0\}, and letvt​\(x\)=Mt​x\+ctv\_\{t\}\(x\)=M\_\{t\}x\+c\_\{t\}forMt,ctM\_\{t\},c\_\{t\}as in \([142](https://arxiv.org/html/2610.10875#A1.E142)\)\. This is an admissible transport fromπ0\\pi\_\{0\}toπ\\piwith

‖vt‖ℬϕ=\|s′​\(t\)s⁡\(t\)\|\+\|h′​\(t\)−s′​\(t\)s⁡\(t\)​h​\(t\)\|\.\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\left\|\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}\\right\|\+\\left\|h^\{\\prime\}\(t\)\-\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}h\(t\)\\right\|\.\(163\)Ifm=0m=0then we are done, and form≠0m\\neq 0we take

s⁡\(t\):=\{\(1\+‖m‖\)−3​t0≤t<13,\(1\+‖m‖\)−113≤t<23,\(1\+‖m‖\)−3​\(1−t\)23≤t≤1,h⁡\(t\):=\{00≤t<13,\(3​t−1\)​‖m‖1\+‖m‖13≤t<23,‖m‖​s​\(t\)23≤t≤1\.s\(t\):=\\begin\{cases\}\(1\+\\\|m\\\|\)^\{\-3t\}&0\\leq t<\\frac\{1\}\{3\},\\\\ \(1\+\\\|m\\\|\)^\{\-1\}&\\frac\{1\}\{3\}\\leq t<\\frac\{2\}\{3\},\\\\ \(1\+\\\|m\\\|\)^\{\-3\(1\-t\)\}&\\frac\{2\}\{3\}\\leq t\\leq 1,\\end\{cases\}\\qquad h\(t\):=\\begin\{cases\}0&0\\leq t<\\frac\{1\}\{3\},\\\\ \\frac\{\(3t\-1\)\\\|m\\\|\}\{1\+\\\|m\\\|\}&\\frac\{1\}\{3\}\\leq t<\\frac\{2\}\{3\},\\\\ \\\|m\\\|s\(t\)&\\frac\{2\}\{3\}\\leq t\\leq 1\.\\end\{cases\}\(164\)Then, we haves⁡\(0\)=s⁡\(1\)=1s\(0\)=s\(1\)=1,h⁡\(0\)=0h\(0\)=0, andh⁡\(1\)=‖m‖h\(1\)=\\\|m\\\|\. Furthermore, using \([163](https://arxiv.org/html/2610.10875#A1.E163)\) we have

‖vt‖ℬϕ≤\{\|s′​\(t\)s⁡\(t\)\|0≤t<13,\|h′​\(t\)\|13≤t<23,\|s′​\(t\)s⁡\(t\)\|23≤t≤1,\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\begin\{cases\}\\left\|\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}\\right\|&0\\leq t<\\frac\{1\}\{3\},\\\\ \|h^\{\\prime\}\(t\)\|&\\frac\{1\}\{3\}\\leq t<\\frac\{2\}\{3\},\\\\ \\left\|\\frac\{s^\{\\prime\}\(t\)\}\{s\(t\)\}\\right\|&\\frac\{2\}\{3\}\\leq t\\leq 1,\\end\{cases\}\(165\)whereh=0h=0on the first interval,s′=0s^\{\\prime\}=0on the second, andh′−s′s​h=0h^\{\\prime\}\-\\frac\{s^\{\\prime\}\}\{s\}h=0on the third\. Hence, since\|s′s\|=\|\(log⁡s\)′\|=3​log⁡\(1\+‖m‖\)\|\\frac\{s^\{\\prime\}\}\{s\}\|=\|\(\\log s\)^\{\\prime\}\|=3\\log\(1\+\\\|m\\\|\)on the first and third intervals we have

Dℬϕ​\(π0,π\)\\displaystyle D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi\_\{0\},\\pi\)≤∫01‖vt‖ℬϕ​𝑑t\\displaystyle\\leq\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt\(166\)≤13⋅\(3​log⁡\(1\+‖m‖\)\)\+13⋅3​‖m‖1\+‖m‖\+13⋅\(3​log⁡\(1\+‖m‖\)\)\\displaystyle\\leq\\frac\{1\}\{3\}\\cdot\(3\\log\(1\+\\\|m\\\|\)\)\+\\frac\{1\}\{3\}\\cdot\\frac\{3\\\|m\\\|\}\{1\+\\\|m\\\|\}\+\\frac\{1\}\{3\}\\cdot\(3\\log\(1\+\\\|m\\\|\)\)\(167\)≤3​log⁡\(1\+‖m‖\),\\displaystyle\\leq 3\\log\(1\+\\\|m\\\|\),\(168\)where we used thatz1\+z≤log⁡\(1\+z\)\\frac\{z\}\{1\+z\}\\leq\\log\(1\+z\)\. Thus,

Dℬϕ​\(γd,π\)≤12​‖log⁡Σ‖∗\+3​log⁡\(1\+‖m‖\)\.D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\)\\leq\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}\+3\\log\(1\+\\\|m\\\|\)\.\(169\)∎

#### A\.3\.2Reverse OU energies

We will also take this opportunity to compute the action of the reverse OU transportℰℬ​\(π\)\\mathcal\{E\}\_\{\\mathcal\{B\}\}\(\{\\pi\}\)from Lemma[1](https://arxiv.org/html/2610.10875#Thmlemma1)for the caseπ=𝒩⁡\(m,Σ\)\\pi=\\mathcal\{N\}\(m,\\Sigma\)withΣ≻0\\Sigma\\succ 0\. Doing so helps us understand more about the reverse OU \(sub\)optimality:

###### Lemma 7\(Reverse OU energies for Gaussians\)\.

If we denoteCa:=\(1−a2\)​𝟙d\+a2​ΣC\_\{a\}:=\(1\-a^\{2\}\)\{\\mathbbm\{1\}\}\_\{d\}\+a^\{2\}\\Sigmathen the following bounds hold:

- •\(Wasserstein\)\.WithℰW2​\(π\)=∫0∞‖∇log⁡Pτ∗​πγd‖L2​\(Pτ∗​π\)​𝑑τ\\mathcal\{E\}\_\{W\_\{2\}\}\(\\pi\)=\\int\_\{0\}^\{\\infty\}\\\|\\nabla\\log\\frac\{P\_\{\\tau\}^\{\*\}\\pi\}\{\\gamma\_\{d\}\}\\\|\_\{L^\{2\}\(P\_\{\\tau\}^\{\*\}\\pi\)\}d\\tau, we have W2​\(γd,π\)≤ℰW2​\(π\)≤π2​W2​\(γd,π\)\.W\_\{2\}\(\\gamma\_\{d\},\\pi\)\\leq\\mathcal\{E\}\_\{W\_\{2\}\}\(\\pi\)\\leq\\frac\{\\pi\}\{2\}W\_\{2\}\(\\gamma\_\{d\},\\pi\)\.\(170\)
- •\(Stein\)\.Withℬ=ℋk\\mathcal\{B\}=\\mathcal\{H\}\_\{k\}fork⁡\(x,y\)=\(1\+⟨x,y⟩\)​𝟙dk\(x,y\)=\(1\+\\left\\langle x,y\\right\\rangle\)\{\\mathbbm\{1\}\}\_\{d\}, we have ℰℋk​\(π\)≥12​\(∫01‖Ca−1​m‖​𝑑a\+12​‖log⁡Σ‖F\)\.\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\(\\pi\)\\geq\\frac\{1\}\{\\sqrt\{2\}\}\\left\(\\int\_\{0\}^\{1\}\\\|C\_\{a\}^\{\-1\}m\\\|da\+\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{F\}\\right\)\.\(171\)
- •\(Barron\)\.Withℬ=ℬϕ\\mathcal\{B\}=\\mathcal\{B\}\_\{\\phi\}forϕ⁡\(z\)=z\\phi\(z\)=z, we have ℰℬϕ​\(π\)=∫01‖Ca−1​m‖​𝑑a\+12​‖log⁡Σ‖∗\.\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi\)=\\int\_\{0\}^\{1\}\\\|C\_\{a\}^\{\-1\}m\\\|da\+\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}\.\(172\)

###### Proof\.

Lettinga:=e−τa:=e^\{\-\\tau\}, we know thatPτ∗​π=𝒩⁡\(a​m,Ca\)P\_\{\\tau\}^\{\*\}\\pi=\\mathcal\{N\}\(am,C\_\{a\}\)and therefore that

\(∇log⁡Pτ∗​πγd\)​\(x\)=x−Ca−1​\(x−a​m\)=a2​\(Σ−𝟙d\)​Ca−1​x\+a​Ca−1​m\.\\left\(\\nabla\\log\\frac\{P\_\{\\tau\}^\{\*\}\\pi\}\{\\gamma\_\{d\}\}\\right\)\(x\)=x\-C\_\{a\}^\{\-1\}\(x\-am\)=a^\{2\}\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}x\+aC^\{\-1\}\_\{a\}m\.\(173\)This is an affine drift, and we will now compute the respective tangent norms\.

For the Wasserstein case, ifXa∼𝒩⁡\(a​m,Ca\)X\_\{a\}\\sim\\mathcal\{N\}\(am,C\_\{a\}\)then

\(∇log⁡Pτ∗​πγd\)​\(Xa\)=a​m\+a2​\(Σ−𝟙d\)​Ca−1​\(Xa−a​m\)\\left\(\\nabla\\log\\frac\{P\_\{\\tau\}^\{\*\}\\pi\}\{\\gamma\_\{d\}\}\\right\)\(X\_\{a\}\)=am\+a^\{2\}\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}\(X\_\{a\}\-am\)\(174\)and so we usedτ=−da/ad\\tau=\-da/ato find

ℰW2​\(π\)\\displaystyle\\mathcal\{E\}\_\{W\_\{2\}\}\(\\pi\)=∫01\(∥m∥2\+a2∥\(Σ−𝟙d\)Ca−1/2∥F2\)1/2da\\displaystyle=\\int\_\{0\}^\{1\}\\left\(\\\|m\\\|^\{2\}\+a^\{2\}\\\|\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1/2\}\\\|\_\{F\}^\{2\}\\right\)^\{1/2\}da\(175\)=∫01\(‖m‖2\+∑j=1d\|a⁡\(σj2−1\)1−a2\+a2​σj2\|2\)1/2​𝑑a,\\displaystyle=\\int\_\{0\}^\{1\}\\left\(\\\|m\\\|^\{2\}\+\\sum\_\{j=1\}^\{d\}\\left\|\\frac\{a\(\\sigma\_\{j\}^\{2\}\-1\)\}\{\\sqrt\{1\-a^\{2\}\+a^\{2\}\\sigma\_\{j\}^\{2\}\}\}\\right\|^\{2\}\\right\)^\{1/2\}da,\(176\)whereσ12,…,σd2\\sigma^\{2\}\_\{1\},\\ldots,\\sigma\_\{d\}^\{2\}are the eigenvalues ofΣ\\Sigma\. Now, for anya∈\[0,1\)a\\in\[0,1\)andσ\>0\\sigma\>0we have that

0\\displaystyle 0≤\(1−a2​\(σ\+1\)\)2\\displaystyle\\leq\(1\-a^\{2\}\(\\sigma\+1\)\)^\{2\}\(177\)=1−2​a2​\(σ\+1\)\+a4​\(σ\+1\)2\\displaystyle=1\-2a^\{2\}\(\\sigma\+1\)\+a^\{4\}\(\\sigma\+1\)^\{2\}\(178\)=1\+a2​\(σ2−1\)−a2​\(1−a2\)​\(σ\+1\)2,\\displaystyle=1\+a^\{2\}\(\\sigma^\{2\}\-1\)\-a^\{2\}\(1\-a^\{2\}\)\(\\sigma\+1\)^\{2\},\(179\)and so1−a2\+a2​σ2≥a2​\(1−a2\)​\(σ\+1\)21\-a^\{2\}\+a^\{2\}\\sigma^\{2\}\\geq a^\{2\}\(1\-a^\{2\}\)\(\\sigma\+1\)^\{2\}\. In other words,

a2​\(σ\+1\)21−a2\+a2​σ2≤11−a2\\frac\{a^\{2\}\(\\sigma\+1\)^\{2\}\}\{\{1\-a^\{2\}\+a^\{2\}\\sigma^\{2\}\}\}\\leq\\frac\{1\}\{\{1\-a^\{2\}\}\}\(180\)and so multiplying by\(σ−1\)2\(\\sigma\-1\)^\{2\}implies the inequality

\|a⁡\(σj2−1\)1−a2\+a2​σj2\|2≤\(σj−1\)21−a2\(∀a∈\[0,1\)\)\.\\displaystyle\\left\|\\frac\{a\(\\sigma\_\{j\}^\{2\}\-1\)\}\{\\sqrt\{1\-a^\{2\}\+a^\{2\}\\sigma\_\{j\}^\{2\}\}\}\\right\|^\{2\}\\leq\\frac\{\(\\sigma\_\{j\}\-1\)^\{2\}\}\{1\-a^\{2\}\}\\quad\(\\forall a\\in\[0,1\)\)\.\(181\)By Lemma[4](https://arxiv.org/html/2610.10875#Thmlemma4)we knowW2​\(γd,π\)=\(‖m‖2\+∑j=1d\(σj−1\)2\)1/2W\_\{2\}\(\\gamma\_\{d\},\\pi\)=\(\\\|m\\\|^\{2\}\+\\sum\_\{j=1\}^\{d\}\(\\sigma\_\{j\}\-1\)^\{2\}\)^\{1/2\}, and so

ℰW2​\(π\)≤W2​\(γd,π\)⋅∫0111−a2​𝑑a=π2​W2​\(γd,π\)\.\\displaystyle\\mathcal\{E\}\_\{W\_\{2\}\}\(\\pi\)\\leq W\_\{2\}\(\\gamma\_\{d\},\\pi\)\\cdot\\int\_\{0\}^\{1\}\\frac\{1\}\{\\sqrt\{1\-a^\{2\}\}\}da=\\frac\{\\pi\}\{2\}W\_\{2\}\(\\gamma\_\{d\},\\pi\)\.\(182\)The lower boundW2​\(γd,π\)≤ℰW2​\(π\)W\_\{2\}\(\\gamma\_\{d\},\\pi\)\\leq\\mathcal\{E\}\_\{W\_\{2\}\}\(\\pi\)is obvious becauseW2W\_\{2\}is the infimum over all transports\.

For the Stein case withk⁡\(x,y\)=\(1\+⟨x,y⟩\)​𝟙dk\(x,y\)=\(1\+\\left\\langle x,y\\right\\rangle\)\{\\mathbbm\{1\}\}\_\{d\}, we knowdτ=−da/ad\\tau=\-da/aand so

ℰℋk​\(π\)=∫01\(‖Ca−1​m‖2\+a2​‖\(Σ−𝟙d\)​Ca−1‖F2\)1/2​𝑑a,\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\(\\pi\)=\\int\_\{0\}^\{1\}\\left\(\\\|C\_\{a\}^\{\-1\}m\\\|^\{2\}\+a^\{2\}\\\|\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}\\\|\_\{F\}^\{2\}\\right\)^\{1/2\}da,\(183\)which implies that

ℰℋk​\(π\)≥12​∫01\(‖Ca−1​m‖\+a​‖\(Σ−𝟙d\)​Ca−1‖F\)​𝑑a\.\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\(\\pi\)\\geq\\frac\{1\}\{\\sqrt\{2\}\}\\int\_\{0\}^\{1\}\\left\(\\\|C\_\{a\}^\{\-1\}m\\\|\+a\\\|\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}\\\|\_\{F\}\\right\)da\.\(184\)Now, sincedd​a​log⁡Ca=2​a​\(Σ−𝟙d\)​Ca−1\\frac\{d\}\{da\}\\log C\_\{a\}=2a\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}\(note thatΣ−𝟙d\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}andCaC\_\{a\}commute\) andlog⁡C0=0,log⁡C1=log⁡Σ\\log C\_\{0\}=0,\\log C\_\{1\}=\\log\\Sigma, we have

12​‖log⁡Σ‖F=‖∫01a⁡\(Σ−𝟙d\)​Ca−1​𝑑a‖F≤∫01a​‖\(Σ−𝟙d\)​Ca−1‖F​𝑑a\.\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{F\}=\\left\\\|\\int\_\{0\}^\{1\}a\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}da\\right\\\|\_\{F\}\\leq\\int\_\{0\}^\{1\}a\\\|\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C^\{\-1\}\_\{a\}\\\|\_\{F\}da\.\(185\)
For the Barron case withϕ⁡\(z\)=z\\phi\(z\)=z, we have

ℰℬϕ​\(π\)=∫01\(‖Ca−1​m‖\+a​‖\(Σ−𝟙d\)​Ca−1‖∗\)​𝑑a,\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi\)=\\int\_\{0\}^\{1\}\\left\(\\\|C\_\{a\}^\{\-1\}m\\\|\+a\\\|\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}\\\|\_\{\*\}\\right\)da,\(186\)after which the result follows from the equalities

∫01a​‖\(Σ−𝟙d\)​Ca−1‖∗​𝑑a=∑j=1d∫01a​\|σj2−1\|1\+a2​\(σj2−1\)​𝑑a=12​∑j=1d\|log⁡σj2\|=12​‖log⁡Σ‖∗\.\\displaystyle\\int\_\{0\}^\{1\}a\\\|\(\\Sigma\-\{\\mathbbm\{1\}\}\_\{d\}\)C\_\{a\}^\{\-1\}\\\|\_\{\*\}da=\\sum\_\{j=1\}^\{d\}\\int\_\{0\}^\{1\}\\frac\{a\|\\sigma\_\{j\}^\{2\}\-1\|\}\{1\+a^\{2\}\(\\sigma\_\{j\}^\{2\}\-1\)\}da=\\frac\{1\}\{2\}\\sum\_\{j=1\}^\{d\}\|\\log\\sigma\_\{j\}^\{2\}\|=\\frac\{1\}\{2\}\\\|\\log\\Sigma\\\|\_\{\*\}\.\(187\)∎

Looking at this result, one might hope for a matching upper bound in the linear Stein case \(or at least for the‖log⁡Σ‖F\\\|\\log\\Sigma\\\|\_\{F\}dependence to be sharp up to constants\)\. We show that this cannot be the case, albeit in a very mild way:

###### Example 5\.

Letπ=𝒩⁡\(0,Σ\)\\pi=\\mathcal\{N\}\(0,\\Sigma\)forΣ=diag⁡\(ed/1,ed/2,…,ed/d\)\\Sigma=\\operatorname\{diag\}\(e^\{d/\\sqrt\{1\}\},e^\{d/\\sqrt\{2\}\},\\ldots,e^\{d/\\sqrt\{d\}\}\)\. Then, withk⁡\(x,y\)=\(1\+⟨x,y⟩\)​𝟙dk\(x,y\)=\(1\+\\left\\langle x,y\\right\\rangle\)\{\\mathbbm\{1\}\}\_\{d\}we have

ℰℋk​\(π\)≥log⁡d24⋅‖log⁡Σ‖F≥log⁡d12⋅Dℋk​\(γd,π\)\.\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\\left\(\\pi\\right\)\\geq\\frac\{\\sqrt\{\\log d\}\}\{24\}\\cdot\\\|\\log\\Sigma\\\|\_\{F\}\\geq\\frac\{\\sqrt\{\\log d\}\}\{12\}\\cdot D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)\.\(188\)

###### Proof\.

In the centered case, from \([183](https://arxiv.org/html/2610.10875#A1.E183)\) we find that ifσ12,…,σd2≥1\\sigma^\{2\}\_\{1\},\\ldots,\\sigma\_\{d\}^\{2\}\\geq 1are the eigenvalues ofΣ\\Sigmathen

ℰℋk​\(π\)=∫01\(∑j=1d\(a⁡\(σj2−1\)1\+a2​\(σj2−1\)\)2\)1/2​𝑑a\.\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\\left\(\\pi\\right\)=\\int\_\{0\}^\{1\}\\left\(\\sum\_\{j=1\}^\{d\}\\left\(\\frac\{a\(\\sigma\_\{j\}^\{2\}\-1\)\}\{1\+a^\{2\}\(\\sigma\_\{j\}^\{2\}\-1\)\}\\right\)^\{2\}\\right\)^\{1/2\}da\.\(189\)Fora∈\[σj−1,σj\+1−1\)a\\in\[\\sigma\_\{j\}^\{\-1\},\\sigma\_\{j\+1\}^\{\-1\}\)and alli≤ji\\leq jwe havea2\(σi2−1\)≥1−e−d/i≥1−e−d≥12a^\{2\}\(\\sigma\_\{i\}^\{2\}\-1\)\\geq 1\-e^\{\-d/\\sqrt\{i\}\}\\geq 1\-e^\{\-\\sqrt\{d\}\}\\geq\\frac\{1\}\{2\}, and so

a⁡\(σi2−1\)1\+a2​\(σi2−1\)=1a⋅a2​\(σi2−1\)1\+a2​\(σi2−1\)≥13​a\.\\frac\{a\(\\sigma\_\{i\}^\{2\}\-1\)\}\{1\+a^\{2\}\(\\sigma\_\{i\}^\{2\}\-1\)\}=\\frac\{1\}\{a\}\\cdot\\frac\{a^\{2\}\(\\sigma\_\{i\}^\{2\}\-1\)\}\{1\+a^\{2\}\(\\sigma\_\{i\}^\{2\}\-1\)\}\\geq\\frac\{1\}\{3a\}\.\(190\)Plugging this in and denotingσd\+12=ed/d\+1\\sigma\_\{d\+1\}^\{2\}=e^\{d/\\sqrt\{d\+1\}\}for simplicity, we split the integral to get

ℰℋk​\(π\)\\displaystyle\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\\left\(\\pi\\right\)≥∑j=1d∫σj−1σj\+1−1\(∑i=1j\(a⁡\(σi2−1\)1\+a2​\(σi2−1\)\)2\)1/2​𝑑a≥∑j=1d∫σj−1σj\+1−1j3​a​𝑑a\\displaystyle\\geq\\sum\_\{j=1\}^\{d\}\\int\_\{\\sigma\_\{j\}^\{\-1\}\}^\{\\sigma\_\{j\+1\}^\{\-1\}\}\\left\(\\sum\_\{i=1\}^\{j\}\\left\(\\frac\{a\(\\sigma\_\{i\}^\{2\}\-1\)\}\{1\+a^\{2\}\(\\sigma\_\{i\}^\{2\}\-1\)\}\\right\)^\{2\}\\right\)^\{1/2\}da\\geq\\sum\_\{j=1\}^\{d\}\\int\_\{\\sigma\_\{j\}^\{\-1\}\}^\{\\sigma\_\{j\+1\}^\{\-1\}\}\\frac\{\\sqrt\{j\}\}\{3a\}da\(191\)=13​∑j=1dj⋅log⁡\(σjσj\+1\)=16​∑j=1dj⋅\(dj−dj\+1\)\\displaystyle=\\frac\{1\}\{3\}\\sum\_\{j=1\}^\{d\}\\sqrt\{j\}\\cdot\\log\\left\(\\frac\{\\sigma\_\{j\}\}\{\\sigma\_\{j\+1\}\}\\right\)=\\frac\{1\}\{6\}\\sum\_\{j=1\}^\{d\}\\sqrt\{j\}\\cdot\\left\(\\frac\{d\}\{\\sqrt\{j\}\}\-\\frac\{d\}\{\\sqrt\{j\+1\}\}\\right\)\(192\)=d6​∑j=1d\(1−jj\+1\)≥d6​∑j=1d12​\(1\+j\)\\displaystyle=\\frac\{d\}\{6\}\\sum\_\{j=1\}^\{d\}\\left\(1\-\\sqrt\{\\frac\{j\}\{j\+1\}\}\\right\)\\geq\\frac\{d\}\{6\}\\sum\_\{j=1\}^\{d\}\\frac\{1\}\{2\(1\+j\)\}\(193\)≥d24​∑j=1d1j,\\displaystyle\\geq\\frac\{d\}\{24\}\\sum\_\{j=1\}^\{d\}\\frac\{1\}\{j\},\(194\)where we used that1−z1\+z≥12​\(1\+z\)1\-\\sqrt\{\\frac\{z\}\{1\+z\}\}\\geq\\frac\{1\}\{2\(1\+z\)\}forz\>0z\>0\. On the other hand, we have

‖log⁡Σ‖F=d​\(∑j=1d1j\)1/2\\\|\\log\\Sigma\\\|\_\{F\}=d\\left\(\\sum\_\{j=1\}^\{d\}\\frac\{1\}\{j\}\\right\)^\{1/2\}\(195\)and so

ℰℋk​\(π\)‖log⁡Σ‖F≥124​\(∑j=1d1j\)1/2≥log⁡d24\.\\frac\{\\mathcal\{E\}\_\{\\mathcal\{H\}\_\{k\}\}\\left\(\\pi\\right\)\}\{\\\|\\log\\Sigma\\\|\_\{F\}\}\\geq\\frac\{1\}\{24\}\\left\(\\sum\_\{j=1\}^\{d\}\\frac\{1\}\{j\}\\right\)^\{1/2\}\\geq\\frac\{\\sqrt\{\\log d\}\}\{24\}\.\(196\)The comparison withDℋk​\(γd,π\)D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\pi\)comes from \([140](https://arxiv.org/html/2610.10875#A1.E140)\)\. ∎

### A\.4Perturbative bounds for rotation\-equivariant Stein distances

We first prove a perturbative first\-order lower bound on theDℋkD\_\{\\mathcal\{H\}\_\{k\}\}distance along flows when the kernel is jointly continuous:

###### Lemma 8\(First\-orderDℋkD\_\{\\mathcal\{H\}\_\{k\}\}lower bound\)\.

Letk:ℝd×ℝd→ℝd×dk:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\\times d\}satisfy Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)\. Suppose thatφ∈Cc∞​\(ℝd\)\\varphi\\in C\_\{c\}^\{\\infty\}\(\{\\mathbb\{R\}\}^\{d\}\),μ∈𝒫⁡\(ℝd\)\\mu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), andw:\[0,1\]×ℝd→ℝdw:\[0,1\]\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}satisfy the following:

1. \(i\)L:=sup‖u‖ℋk≤1∫ℝd⟨u,∇φ⟩​𝑑μ∈\[0,∞\)L:=\\sup\_\{\\\|u\\\|\_\{\\mathcal\{H\}\_\{k\}\}\\leq 1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle u,\\nabla\\varphi\\right\\rangle d\\mu\\in\[0,\\infty\)\.
2. \(ii\)wwis jointly measurable and eachwtw\_\{t\}is globally\-Lipschitz with∫01\(\|wt​\(0\)\|\+Lip⁡\(wt\)\)​𝑑t<∞\\int\_\{0\}^\{1\}\(\|w\_\{t\}\(0\)\|\+\\operatorname\{Lip\}\(w\_\{t\}\)\)dt<\\infty\.

For eachε\>0\\varepsilon\>0defineνε:=Law⁡\(X1\)\\nu\_\{\\varepsilon\}:=\\operatorname\{Law\}\(X\_\{1\}\)according to the flow

X˙t=ε​wt​\(Xt\),X0∼μ\.\\dot\{X\}\_\{t\}=\\varepsilon w\_\{t\}\(X\_\{t\}\),\\quad X\_\{0\}\\sim\\mu\.\(197\)Then, withF1​\(x\):=∫01wt​\(x\)​𝑑tF\_\{1\}\(x\):=\\int\_\{0\}^\{1\}w\_\{t\}\(x\)dtwe have

lim infε↓0Dℋk​\(μ,νε\)ε≥\|∫ℝd⟨∇φ,F1⟩​𝑑μ\|L\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\frac\{\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi,F\_\{1\}\\right\\rangle d\\mu\\right\|\}\{L\}\(198\)where ifL=0L=0then the right hand side is∞\\inftyif\|∫ℝd⟨∇φ,F1⟩​𝑑μ\|\>0\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi,F\_\{1\}\\right\\rangle d\\mu\\right\|\>0and 0 otherwise\.

###### Proof\.

Forρ∈𝒫⁡\(ℝd\)\\rho\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)we denote

gρ:=∫ℝdk\(⋅,x\)∇φ\(x\)dρ\(x\)∈ℋk,G\(ρ\):=∥gρ∥ℋk\.g\_\{\\rho\}:=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}k\(\\cdot,x\)\\nabla\\varphi\(x\)d\\rho\(x\)\\in\\mathcal\{H\}\_\{k\},\\quad G\(\\rho\):=\\\|g\_\{\\rho\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}\.\(199\)By the reproducing property, we know that

G⁡\(ρ\)=sup‖u‖ℋk≤1∫ℝd⟨u,∇φ⟩​𝑑ρG\(\\rho\)=\\sup\_\{\\\|u\\\|\_\{\\mathcal\{H\}\_\{k\}\}\\leq 1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle u,\\nabla\\varphi\\right\\rangle d\\rho\(200\)and soG⁡\(μ\)=LG\(\\mu\)=L\. Moreover, the reproducing property also ensures

G\(ρ\)2=∬ℝd×ℝd⟨∇φ\(x\),k\(x,y\)∇φ\(y\)⟩dρ\(x\)dρ\(y\)\.G\(\\rho\)^\{2\}=\\iint\_\{\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\(x\),k\(x,y\)\\nabla\\varphi\(y\)\\right\\rangle d\\rho\(x\)d\\rho\(y\)\.\(201\)This integrand is bounded and continuous sincekkis jointly continuous and∇φ\\nabla\\varphiis compactly supported, and soGGis narrowly continuous\.

Assumption \(ii\) implies the Carathéodory conditions, yielding a unique flow\. LetTtεT\_\{t\}^\{\\varepsilon\}be the flow ofε​wt\\varepsilon w\_\{t\}, so thatνε=\(T1ε\)\#​μ\\nu\_\{\\varepsilon\}=\(T\_\{1\}^\{\\varepsilon\}\)\_\{\\\#\}\\mu\. For eachx∈ℝdx\\in\{\\mathbb\{R\}\}^\{d\}we know thatTtε​\(x\)→xT\_\{t\}^\{\\varepsilon\}\(x\)\\to xasε↓0\\varepsilon\\downarrow 0uniformly inttby Grönwall’s lemma\. By integrating the continuity equation, we have

φ⁡\(T1ε​\(x\)\)−φ⁡\(x\)ε=∫01⟨∇φ​\(Ttε​\(x\)\),wt​\(Ttε​\(x\)\)⟩​𝑑t\.\\frac\{\\varphi\(T\_\{1\}^\{\\varepsilon\}\(x\)\)\-\\varphi\(x\)\}\{\\varepsilon\}=\\int\_\{0\}^\{1\}\\left\\langle\\nabla\\varphi\(T\_\{t\}^\{\\varepsilon\}\(x\)\),w\_\{t\}\(T\_\{t\}^\{\\varepsilon\}\(x\)\)\\right\\rangle dt\.\(202\)DenotingK:=spt⁡\(∇φ\)K:=\\operatorname\{spt\}\(\\nabla\\varphi\), then by compactness ofKKwe know the integrand is bounded in absolute value by

‖∇φ‖∞​supy∈K\|wt​\(y\)\|≤CK​\(\|wt​\(0\)\|\+Lip⁡\(wt\)\)\.\\\|\\nabla\\varphi\\\|\_\{\\infty\}\\sup\_\{y\\in K\}\|w\_\{t\}\(y\)\|\\leq C\_\{K\}\(\|w\_\{t\}\(0\)\|\+\\operatorname\{Lip\}\(w\_\{t\}\)\)\.\(203\)This bound is integrable inttby \(ii\) and is independent ofx,εx,\\varepsilon, which by dominated convergence and integration overμ\\muimplies

limε↓0∫ℝdφ​d​νε−∫ℝdφ​𝑑με=∫ℝd⟨∇φ,F1⟩​𝑑μ\.\\lim\_\{\\varepsilon\\downarrow 0\}\\frac\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\nu\_\{\\varepsilon\}\-\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu\}\{\\varepsilon\}=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi,F\_\{1\}\\right\\rangle d\\mu\.\(204\)If we define

ω⁡\(r\):=supDℋk​\(μ,ρ\)≤rG⁡\(ρ\)\\omega\(r\):=\\sup\_\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\rho\)\\leq r\}G\(\\rho\)\(205\)then sinceDℋkD\_\{\\mathcal\{H\}\_\{k\}\}convergence implies narrow convergence \(Proposition[6](https://arxiv.org/html/2610.10875#Thmproposition6)\) andGGis narrowly continuous we have

limr↓0ω⁡\(r\)=G⁡\(μ\)=L\.\\lim\_\{r\\downarrow 0\}\\omega\(r\)=G\(\\mu\)=L\.\(206\)Let us denote

ℓ:=lim infε↓0Dℋk​\(μ,νε\)ε\.\\ell:=\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\.\(207\)Ifℓ=∞\\ell=\\inftythe claim is immediate, and otherwise we takeεn↓0\\varepsilon\_\{n\}\\downarrow 0realizing thelim inf\\liminfand admissible curves\(vtn,ρtn\)t∈\[0,1\]\(v\_\{t\}^\{n\},\\rho\_\{t\}^\{n\}\)\_\{t\\in\[0,1\]\}transportingμ\\mutoνεn\\nu\_\{\\varepsilon\_\{n\}\}for which

An:=∫01‖vtn‖ℋk​𝑑t≤Dℋk​\(μ,νεn\)\+εn2\.A\_\{n\}:=\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{n\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}dt\\leq D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\_\{n\}\}\)\+\\varepsilon\_\{n\}^\{2\}\.\(208\)Hence,An→0A\_\{n\}\\to 0and moreoverDℋk​\(μ,ρtn\)≤∫0t‖vsn‖ℋk​𝑑s≤AnD\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\rho\_\{t\}^\{n\}\)\\leq\\int\_\{0\}^\{t\}\\\|v\_\{s\}^\{n\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}ds\\leq A\_\{n\}\. Therefore,G⁡\(ρtn\)≤ω⁡\(An\)G\(\\rho\_\{t\}^\{n\}\)\\leq\\omega\(A\_\{n\}\)and so

\|∫ℝdφ​d​νεn−∫ℝdφ​𝑑μ\|≤∫01‖vtn‖ℋk​G​\(ρtn\)​𝑑t≤An​ω​\(An\)\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\nu\_\{\\varepsilon\_\{n\}\}\-\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu\\right\|\\leq\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{n\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}G\(\\rho\_\{t\}^\{n\}\)dt\\leq A\_\{n\}\\omega\(A\_\{n\}\)\(209\)by the continuity equation and duality\. Dividing byεn\\varepsilon\_\{n\}, we find

\|∫ℝdφ​d​νεn−∫ℝdφ​𝑑μεn\|≤Anεn​ω​\(An\)≤\(Dℋk​\(μ,νεn\)εn\+εn\)​ω​\(An\)\.\\left\|\\frac\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\nu\_\{\\varepsilon\_\{n\}\}\-\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\mu\}\{\\varepsilon\_\{n\}\}\\right\|\\leq\\frac\{A\_\{n\}\}\{\\varepsilon\_\{n\}\}\\omega\(A\_\{n\}\)\\leq\\left\(\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\_\{n\}\}\)\}\{\\varepsilon\_\{n\}\}\+\\varepsilon\_\{n\}\\right\)\\omega\(A\_\{n\}\)\.\(210\)Taking then→∞n\\to\\inftylimit and usingω⁡\(An\)→L\\omega\(A\_\{n\}\)\\to Lalong with \([204](https://arxiv.org/html/2610.10875#A1.E204)\) implies the claim\. ∎

Ifμ\\muis a rotation\-invariant measure andkkis a rotation\-equivariant kernel \(see Definition[1](https://arxiv.org/html/2610.10875#Thmdefinition1)\), then we may get a more concrete lower bound using test functions constructed via higher\-order spherical harmonics; this is inspired by the approach of Bach\[[7](https://arxiv.org/html/2610.10875#bib.bib46)\]\. This will require some theory of polynomials on the sphere, for which we refer to the textbook of Atkinson\-Han\[[6](https://arxiv.org/html/2610.10875#bib.bib62)\]\.

###### Lemma 9\(Rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}perturbative lower bound\)\.

Letd≥3d\\geq 3andμ∈𝒫2​\(ℝd\)\\mu\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)be rotationally\-invariant and notδ0\\delta\_\{0\}\. Letw:\[0,1\]×ℝd→ℝdw:\[0,1\]\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}be jointly measurable with eachwtw\_\{t\}globally\-Lipschitz with∫01\(\|wt​\(0\)\|\+Lip⁡\(wt\)\)​𝑑t<∞\\int\_\{0\}^\{1\}\(\|w\_\{t\}\(0\)\|\+\\operatorname\{Lip\}\(w\_\{t\}\)\)dt<\\infty, and forε\>0\\varepsilon\>0defineνε:=Law⁡\(X1\)\\nu\_\{\\varepsilon\}:=\\operatorname\{Law\}\(X\_\{1\}\)according to the flow

X˙t=ε​wt​\(Xt\),X0∼μ\.\\dot\{X\}\_\{t\}=\\varepsilon w\_\{t\}\(X\_\{t\}\),\\quad X\_\{0\}\\sim\\mu\.\(211\)Then, for anyk∈𝒦rotk\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}satisfying Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)with∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)∈\(0,∞\)\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)\\in\(0,\\infty\)we have

∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)⋅lim infε↓0Dℋk​\(μ,νε\)ε≥15⋅∑n∈ℕan,d​An∫ℝd\|x\|2​𝑑μ​\(x\),\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\frac\{1\}\{\\sqrt\{5\}\}\\cdot\\frac\{\\sum\_\{\\begin\{subarray\}\{c\}n\\in\{\\mathbb\{N\}\}\\end\{subarray\}\}\\sqrt\{a\_\{n,d\}\}A\_\{n\}\}\{\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\{\|x\|^\{2\}\}d\\mu\(x\)\}\},\(212\)where we define

an,d\\displaystyle a\_\{n,d\}:=1n⁡\(n\+d−2\)​\(n\+d−2n\),\\displaystyle:=\\frac\{1\}\{n\(n\+d\-2\)\}\{n\+d\-2\\choose n\},\(213\)An\\displaystyle A\_\{n\}:=supY∈𝕐nd‖Y‖L2​\(σ\)=1\|∫ℝd⟨∇x\[\|x\|2​Y​\(x\|x\|\)\],∫01wt​\(x\)​𝑑t⟩​𝑑μ​\(x\)\|\\displaystyle:=\\sup\_\{\\begin\{subarray\}\{c\}Y\\in\\mathbb\{Y\}\_\{n\}^\{d\}\\\\ \\\|Y\\\|\_\{L^\{2\}\(\\sigma\)\}=1\\end\{subarray\}\}\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\_\{x\}\\left\[\|x\|^\{2\}Y\\left\(\\frac\{x\}\{\|x\|\}\\right\)\\right\],\\int\_\{0\}^\{1\}w\_\{t\}\(x\)dt\\right\\rangle d\\mu\(x\)\\right\|\(214\)for𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}the space of degree\-nnspherical harmonics \(see\[[6](https://arxiv.org/html/2610.10875#bib.bib62), Definition 2\.7\]\) andσ\\sigmathe uniform probability measure on𝕊d−1\\mathbb\{S\}^\{d\-1\}\.

###### Proof of Lemma[9](https://arxiv.org/html/2610.10875#Thmlemma9)\.

In the statement and in what follows, we set\|x\|2​Y​\(x/\|x\|\)\|x\|^\{2\}Y\(x/\|x\|\),h⁡\(R\)/Rh\(R\)/R, and quantities like it to equal 0 whenx,R=0x,R=0\. LetF1​\(x\):=∫01wt​\(x\)​𝑑tF\_\{1\}\(x\):=\\int\_\{0\}^\{1\}w\_\{t\}\(x\)dt, which we know lies inL2​\(μ,ℝd\)L^\{2\}\(\\mu;\{\\mathbb\{R\}\}^\{d\}\)by the assumptions onww\. By Lemma[8](https://arxiv.org/html/2610.10875#Thmlemma8), we find that

lim infε↓0Dℋk​\(μ,νε\)ε≥supφ\|∫ℝd⟨∇φ,F1⟩​𝑑μ\|∫ℝd×ℝd⟨∇φ\(x\),k\(x,y\)∇φ\(y\)⟩dμ\(x\)dμ\(y\)\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\sup\_\{\\varphi\}\\frac\{\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi,F\_\{1\}\\right\\rangle d\\mu\|\}\{\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\(x\),k\(x,y\)\\nabla\\varphi\(y\)\\right\\rangle d\\mu\(x\)d\\mu\(y\)\}\}\(217\)where the supremum ranges overφ∈Cc∞​\(ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\)\. We will chooseφ\\varphias a weighted sum of spherical harmonics\.

Let𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}be the linear space of degree\-nnspherical harmonics on𝕊d−1\\mathbb\{S\}^\{d\-1\}\(see\[[6](https://arxiv.org/html/2610.10875#bib.bib62), Definition 2\.7\]\) andNn,d:=dim⁡𝕐ndN\_\{n,d\}:=\\operatorname\{dim\}\\mathbb\{Y\}\_\{n\}^\{d\}its dimension, which by \(2\.10\) of\[[6](https://arxiv.org/html/2610.10875#bib.bib62)\]satisfies

Nn,d=\(2​n\+d−2\)​\(n\+d−3\)\!n\!​\(d−2\)\!=2​n\+d−2\(d−2\)\!∏j=1d−3\(n\+j\)\(∀n∈ℕ,d≥3\)N\_\{n,d\}=\\frac\{\(2n\+d\-2\)\(n\+d\-3\)\!\}\{n\!\(d\-2\)\!\}=\\frac\{2n\+d\-2\}\{\(d\-2\)\!\}\\prod\_\{j=1\}^\{d\-3\}\(n\+j\)\\quad\(\\forall n\\in\{\\mathbb\{N\}\},\\;d\\geq 3\)\(218\)withN0,d=1N\_\{0,d\}=1andNn,2=2N\_\{n,2\}=2for alln∈ℕn\\in\{\\mathbb\{N\}\}\. For eachnn, let\{Yn,j:1≤j≤Nn,d\}\\\{Y\_\{n,j\}:\\;1\\leq j\\leq N\_\{n,d\}\\\}be anL2​\(σ\)L^\{2\}\(\\sigma\)\-orthonormal basis of𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}\.

SinceX∼μX\\sim\\muis rotationally\-invariant we may writeX=R​ΘX=R\\ThetaforΘ∼σ\\Theta\\sim\\sigmadrawn uniformly on𝕊d−1\\mathbb\{S\}^\{d\-1\}independently ofR∈\[0,∞\)R\\in\[0,\\infty\)\. Fixh∈Cc∞​\(\(0,∞\)\)h\\in C^\{\\infty\}\_\{c\}\(\(0,\\infty\)\)to be nonnegative with𝔼⁡\[h⁡\(R\)/R\]\>0\\mathbb\{E\}\[h\(R\)/R\]\>0and seth⁡\(0\)=0h\(0\)=0\. We consider𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}as a representation ofO⁡\(d\)O\(d\), where eachQ∈O⁡\(d\)Q\\in O\(d\)is represented by the automorphismρn​\(Q\)\\rho\_\{n\}\(Q\)on𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}sendingY↦Y∘Q⊤Y\\mapsto Y\\circ Q^\{\\top\}\. For each degreennwe define the linear map

Jn:𝕐nd→L2​\(μ,ℝd\),\(Jn​Y\)​\(x\):=∇x\[h⁡\(\|x\|\)​Y​\(x\|x\|\)\]\.J\_\{n\}:\\mathbb\{Y\}\_\{n\}^\{d\}\\to L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\),\\quad\(J\_\{n\}Y\)\(x\):=\\nabla\_\{x\}\\left\[h\(\|x\|\)Y\\left\(\\frac\{x\}\{\|x\|\}\\right\)\\right\]\.\(219\)By the chain rule, for everyQ∈O⁡\(d\)Q\\in O\(d\)andY∈𝕐ndY\\in\\mathbb\{Y\}\_\{n\}^\{d\}we have

Jn​\(Y∘Q⊤\)=Q∘\(Jn​Y\)∘Q⊤\.J\_\{n\}\(Y\\circ Q^\{\\top\}\)=Q\\circ\(J\_\{n\}Y\)\\circ Q^\{\\top\}\.\(220\)Sincekkis rotation\-equivariant andμ\\muis rotation\-invariant, we know that the kernel integral operator𝒯k,μ\\mathcal\{T\}\_\{k,\\mu\}commutes withf↦Q∘f∘Q⊤f\\mapsto Q\\circ f\\circ Q^\{\\top\}and so for anyY∈𝕐ndY\\in\\mathbb\{Y\}\_\{n\}^\{d\}andZ∈𝕐mdZ\\in\\mathbb\{Y\}\_\{m\}^\{d\}we have

⟨Jn​\(Y∘Q⊤\),𝒯k,μ​Jm​\(Z∘Q⊤\)⟩L2​\(μ,ℝd\)=⟨Jn​Y,𝒯k,μ​Jm​Z⟩L2​\(μ,ℝd\),\\left\\langle J\_\{n\}\(Y\\circ Q^\{\\top\}\),\\mathcal\{T\}\_\{k,\\mu\}J\_\{m\}\(Z\\circ Q^\{\\top\}\)\\right\\rangle\_\{L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)\}=\\left\\langle J\_\{n\}Y,\\mathcal\{T\}\_\{k,\\mu\}J\_\{m\}Z\\right\\rangle\_\{L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)\},\(221\)or in other words that

\(Jn∗​𝒯k,μ​Jm\)​ρm​\(Q\)=ρn​\(Q\)​\(Jn∗​𝒯k,μ​Jm\)\(∀Q∈O⁡\(d\)\)\.\(J\_\{n\}^\{\*\}\\mathcal\{T\}\_\{k,\\mu\}J\_\{m\}\)\\rho\_\{m\}\(Q\)=\\rho\_\{n\}\(Q\)\(J\_\{n\}^\{\*\}\\mathcal\{T\}\_\{k,\\mu\}J\_\{m\}\)\\quad\(\\forall Q\\in O\(d\)\)\.\(222\)Hence,Jn∗​𝒯k,μ​JmJ\_\{n\}^\{\*\}\\mathcal\{T\}\_\{k,\\mu\}J\_\{m\}is anO⁡\(d\)O\(d\)\-equivariant linear map from the representation\(𝕐md,ρm\)\(\\mathbb\{Y\}\_\{m\}^\{d\},\\rho\_\{m\}\)to\(𝕐nd,ρn\)\(\\mathbb\{Y\}\_\{n\}^\{d\},\\rho\_\{n\}\)\. By\[[6](https://arxiv.org/html/2610.10875#bib.bib62), Theorem 2\.10\]these representations are irreducible\. By \([218](https://arxiv.org/html/2610.10875#A1.E218)\) andd≥3d\\geq 3we know thatNn,dN\_\{n,d\}is strictly increasing innn, which means that𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\},𝕐md\\mathbb\{Y\}\_\{m\}^\{d\}are not isomorphic forn≠mn\\neq m\. Therefore, Schur’s lemma from representation theory implies that

Jn∗​𝒯k,μ​Jm=δn,m​qn​𝟙𝕐nd\(∀n,m∈ℕ\)J\_\{n\}^\{\*\}\\mathcal\{T\}\_\{k,\\mu\}J\_\{m\}=\\delta\_\{n,m\}q\_\{n\}\{\\mathbbm\{1\}\}\_\{\\mathbb\{Y\}\_\{n\}^\{d\}\}\\quad\(\\forall n,m\\in\{\\mathbb\{N\}\}\)\(223\)for some scalarsqn≥0q\_\{n\}\\geq 0\. Hence,

⟨JnY,𝒯k,μJmZ⟩L2​\(μ,ℝd\)=δn,mqn⟨Y,Z⟩L2​\(σ\)\(∀Y∈𝕐nd,Z∈𝕐md\)\.\\left\\langle J\_\{n\}Y,\\mathcal\{T\}\_\{k,\\mu\}J\_\{m\}Z\\right\\rangle\_\{L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)\}=\\delta\_\{n,m\}q\_\{n\}\\left\\langle Y,Z\\right\\rangle\_\{L^\{2\}\(\\sigma\)\}\\quad\(\\forall Y\\in\\mathbb\{Y\}\_\{n\}^\{d\},Z\\in\\mathbb\{Y\}\_\{m\}^\{d\}\)\.\(224\)Furthermore, spherical differentiation yields

\(Jn​Y\)​\(r​θ\)=h′​\(r\)​Y​\(θ\)​θ\+h⁡\(r\)r​∇𝕊d−1Y​\(θ\),\(J\_\{n\}Y\)\(r\\theta\)=h^\{\\prime\}\(r\)Y\(\\theta\)\\theta\+\\frac\{h\(r\)\}\{r\}\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\(\\theta\),\(225\)which implies that for everyY∈𝕐nd,Z∈𝕐mdY\\in\\mathbb\{Y\}\_\{n\}^\{d\},Z\\in\\mathbb\{Y\}\_\{m\}^\{d\}we have

⟨Jn​Y,Jm​Z⟩L2​\(μ,ℝd\)\\displaystyle\\left\\langle J\_\{n\}Y,J\_\{m\}Z\\right\\rangle\_\{L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)\}=𝔼R,Θ​⟨h′​\(R\)​Y​\(Θ\)​Θ\+h⁡\(R\)R​∇𝕊d−1Y​\(Θ\),h′​\(R\)​Z​\(Θ\)​Θ\+h⁡\(R\)R​∇𝕊d−1Z​\(Θ\)⟩\\displaystyle=\\mathbb\{E\}\_\{R,\\Theta\}\\left\\langle h^\{\\prime\}\(R\)Y\(\\Theta\)\\Theta\+\\frac\{h\(R\)\}\{R\}\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\(\\Theta\),\\;h^\{\\prime\}\(R\)Z\(\\Theta\)\\Theta\+\\frac\{h\(R\)\}\{R\}\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\(\\Theta\)\\right\\rangle\(226\)=𝔼R,Θ​\[h′​\(R\)2​Y​\(Θ\)​Z​\(Θ\)\]\+𝔼R,Θ​\[h​\(R\)2R2​⟨∇𝕊d−1Y​\(Θ\),∇𝕊d−1Z​\(Θ\)⟩ℝd\],\\displaystyle=\\mathbb\{E\}\_\{R,\\Theta\}\\left\[h^\{\\prime\}\(R\)^\{2\}Y\(\\Theta\)\{Z\(\\Theta\)\}\\right\]\+\\mathbb\{E\}\_\{R,\\Theta\}\\left\[\\frac\{h\(R\)^\{2\}\}\{R^\{2\}\}\\left\\langle\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\(\\Theta\),\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\(\\Theta\)\\right\\rangle\_\{\\mathbb\{R\}^\{d\}\}\\right\],\(227\)where the cross\-terms vanish in the second line because∇𝕊d−1Y​\(Θ\)\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\(\\Theta\)\(which is tangent to the sphere atΘ\\Theta\) andΘ\\Theta\(which is normal to the sphere atΘ\\Theta\) are orthogonal, and similarly forZZ\. By independence ofR,ΘR,\\Thetawe find

⟨Jn​Y,Jm​Z⟩L2​\(μ,ℝd\)\\displaystyle\\left\\langle J\_\{n\}Y,J\_\{m\}Z\\right\\rangle\_\{L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)\}=𝔼⁡\[h′​\(R\)2\]​⟨Y,Z⟩L2​\(σ\)\+𝔼⁡\[h​\(R\)2R2\]​⟨∇𝕊d−1Y,∇𝕊d−1Z⟩L2​\(σ,ℝd\)\\displaystyle=\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\\left\\langle Y,Z\\right\\rangle\_\{L^\{2\}\(\\sigma\)\}\+\\mathbb\{E\}\\left\[\\frac\{h\(R\)^\{2\}\}\{R^\{2\}\}\\right\]\\left\\langle\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\\right\\rangle\_\{L^\{2\}\(\\sigma;\\mathbb\{R\}^\{d\}\)\}\(228\)=𝔼⁡\[h′​\(R\)2\]​⟨Y,Z⟩L2​\(σ\)\+m⁡\(m\+d−2\)​𝔼​\[h​\(R\)2R2\]​⟨Y,Z⟩L2​\(σ\),\\displaystyle=\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\\left\\langle Y,Z\\right\\rangle\_\{L^\{2\}\(\\sigma\)\}\+m\(m\+d\-2\)\\mathbb\{E\}\\left\[\\frac\{h\(R\)^\{2\}\}\{R^\{2\}\}\\right\]\\left\\langle Y,Z\\right\\rangle\_\{L^\{2\}\(\\sigma\)\},\(229\)where the second line follows from equation \(3\.20\) of\[[6](https://arxiv.org/html/2610.10875#bib.bib62)\]\. Combining this with the fact that𝕐nd⟂𝕐md\\mathbb\{Y\}\_\{n\}^\{d\}\\perp\\mathbb\{Y\}\_\{m\}^\{d\}forn≠mn\\neq m\(see\[[6](https://arxiv.org/html/2610.10875#bib.bib62), Corollary 2\.15\]\), for everyY∈𝕐nd,Z∈𝕐mdY\\in\\mathbb\{Y\}\_\{n\}^\{d\},Z\\in\\mathbb\{Y\}\_\{m\}^\{d\}we find

⟨Jn​Y,Jm​Z⟩L2​\(μ,ℝd\)=δn,m​\(𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2R2\]\)​⟨Y,Z⟩L2​\(σ\)\.\\left\\langle J\_\{n\}Y,J\_\{m\}Z\\right\\rangle\_\{L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)\}=\\delta\_\{n,m\}\\left\(\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\\left\[\\frac\{h\(R\)^\{2\}\}\{R^\{2\}\}\\right\]\\right\)\\left\\langle Y,Z\\right\\rangle\_\{L^\{2\}\(\\sigma\)\}\.\(230\)Hence, the collection

\{Jn​Yn,j𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\]:n∈ℕ,1≤j≤Nn,d\}\\left\\\{\\frac\{J\_\{n\}Y\_\{n,j\}\}\{\\sqrt\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\}\}:\\quad n\\in\{\\mathbb\{N\}\},\\;1\\leq j\\leq N\_\{n,d\}\\right\\\}\(231\)is an orthonormal family inL2​\(μ,ℝd\)L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)\. Therefore, since𝒯k,μ\\mathcal\{T\}\_\{k,\\mu\}is positive and trace\-class we know

tr⁡\(𝒯k,μ\)\\displaystyle\\operatorname\{tr\}\(\\mathcal\{T\}\_\{k,\\mu\}\)≥∑n∈ℕ∑j=1Nn,d⟨Jn​Yn,j𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\],𝒯k,μ​Jn​Yn,j𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\]⟩\\displaystyle\\geq\\sum\_\{n\\in\{\\mathbb\{N\}\}\}\\sum\_\{j=1\}^\{N\_\{n,d\}\}\\left\\langle\\frac\{J\_\{n\}Y\_\{n,j\}\}\{\\sqrt\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\}\},\\mathcal\{T\}\_\{k,\\mu\}\\frac\{J\_\{n\}Y\_\{n,j\}\}\{\\sqrt\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\}\}\\right\\rangle\(232\)=∑n∈ℕNn,d​qn𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\],\\displaystyle=\\sum\_\{n\\in\{\\mathbb\{N\}\}\}\\frac\{N\_\{n,d\}q\_\{n\}\}\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\},\(233\)where in the second line we applied \([224](https://arxiv.org/html/2610.10875#A1.E224)\)\. In particular, using thattr⁡\(𝒯k,μ\)=∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)\\operatorname\{tr\}\(\\mathcal\{T\}\_\{k,\\mu\}\)=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)implies that

∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)≥∑n∈ℕNn,d​qn𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\]\.\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)\\geq\\sum\_\{n\\in\{\\mathbb\{N\}\}\}\\frac\{N\_\{n,d\}q\_\{n\}\}\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\}\.\(234\)Now, for eachnnlet us write

An\(h\):=supY∈𝕐nd‖Y‖L2​\(σ\)=1\|∫ℝd⟨Jn​Y,F1⟩​𝑑μ\|A\_\{n\}^\{\(h\)\}:=\\sup\_\{\\begin\{subarray\}\{c\}Y\\in\\mathbb\{Y\}\_\{n\}^\{d\}\\\\ \\\|Y\\\|\_\{L^\{2\}\(\\sigma\)\}=1\\end\{subarray\}\}\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle J\_\{n\}Y,F\_\{1\}\\right\\rangle d\\mu\\right\|\(235\)and let us take\(Wn,ℓ\(h\)\)ℓ∈ℕ⊆𝕐nd\(W\_\{n,\\ell\}^\{\(h\)\}\)\_\{\\ell\\in\{\\mathbb\{N\}\}\}\\subseteq\\mathbb\{Y\}\_\{n\}^\{d\}to be approximate maximizers with‖Wn,ℓ\(h\)‖L2​\(σ\)=1\\\|W\_\{n,\\ell\}^\{\(h\)\}\\\|\_\{L^\{2\}\(\\sigma\)\}=1and

An,ℓ\(h\):=∫ℝd⟨Jn​Wn,ℓ\(h\),F1⟩​𝑑μ≥An\(h\)−1ℓ,A\_\{n,\\ell\}^\{\(h\)\}:=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle J\_\{n\}W\_\{n,\\ell\}^\{\(h\)\},F\_\{1\}\\right\\rangle d\\mu\\geq A\_\{n\}^\{\(h\)\}\-\\frac\{1\}\{\\ell\},\(236\)where we note thatJnJ\_\{n\}depends onhh\. By equation \(3\.20\) of\[[6](https://arxiv.org/html/2610.10875#bib.bib62)\]we know that

‖∇𝕊d−1Wn,ℓ\(h\)‖L2​\(σ,ℝd\)2=n⁡\(n\+d−2\)\.\\\|\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}W\_\{n,\\ell\}^\{\(h\)\}\\\|\_\{L^\{2\}\(\\sigma;\{\\mathbb\{R\}\}^\{d\}\)\}^\{2\}=n\(n\+d\-2\)\.\(237\)Now, for each degreennand eachℓ\\ellwe define the functionφn,ℓ\(h\):ℝd→ℝ\\varphi\_\{n,\\ell\}^\{\(h\)\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}by

φn,ℓ\(h\)​\(r​θ\):=h⁡\(r\)​Wn,ℓ\(h\)​\(θ\)\.\\varphi\_\{n,\\ell\}^\{\(h\)\}\(r\\theta\):=h\(r\)W\_\{n,\\ell\}^\{\(h\)\}\(\\theta\)\.\(238\)Sincehhvanishes near00andWn,ℓ\(h\)W\_\{n,\\ell\}^\{\(h\)\}is a spherical polynomial we knowφn,ℓ\(h\)∈Cc∞​\(ℝd\)\\varphi\_\{n,\\ell\}^\{\(h\)\}\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\), and by definition we have∇φn,ℓ\(h\)=Jn​Wn,ℓ\(h\)\\nabla\\varphi\_\{n,\\ell\}^\{\(h\)\}=J\_\{n\}W\_\{n,\\ell\}^\{\(h\)\}\. In particular, applying \([224](https://arxiv.org/html/2610.10875#A1.E224)\) withn=mn=mandY=Z=Wn,ℓ\(h\)Y=Z=W\_\{n,\\ell\}^\{\(h\)\}implies

⟨∇φn,ℓ\(h\),𝒯k,μ∇φn,ℓ\(h\)⟩L2​\(μ,ℝd\)=qn∥Wn,ℓ\(h\)∥L2​\(σ\)2=qn\\left\\langle\\nabla\\varphi\_\{n,\\ell\}^\{\(h\)\},\\mathcal\{T\}\_\{k,\\mu\}\\nabla\\varphi\_\{n,\\ell\}^\{\(h\)\}\\right\\rangle\_\{L^\{2\}\(\\mu;\{\\mathbb\{R\}\}^\{d\}\)\}=q\_\{n\}\\\|W\_\{n,\\ell\}^\{\(h\)\}\\\|\_\{L^\{2\}\(\\sigma\)\}^\{2\}=q\_\{n\}\(239\)using that‖Wn,ℓ\(h\)‖L2​\(σ\)=1\\\|W\_\{n,\\ell\}^\{\(h\)\}\\\|\_\{L^\{2\}\(\\sigma\)\}=1, and so the scalarsqnq\_\{n\}are identified\.

We proceed by fixing a finite subsetΛ⊆ℕ\\Lambda\\subseteq\{\\mathbb\{N\}\}withqn\>0q\_\{n\}\>0for alln∈Λn\\in\\Lambdaand defining

φ\(ℓ,h\):=∑n∈ΛAn,ℓ\(h\)qn​φn,ℓ\(h\)∈Cc∞​\(ℝd\)\.\\varphi^\{\(\\ell,h\)\}:=\\sum\_\{n\\in\\Lambda\}\\frac\{A\_\{n,\\ell\}^\{\(h\)\}\}\{q\_\{n\}\}\\varphi\_\{n,\\ell\}^\{\(h\)\}\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\)\.\(240\)By definition ofAn,ℓ\(h\)A\_\{n,\\ell\}^\{\(h\)\}and \([224](https://arxiv.org/html/2610.10875#A1.E224)\),\([239](https://arxiv.org/html/2610.10875#A1.E239)\) we have

∫ℝd⟨∇φ\(ℓ,h\),F1⟩dμ=∑n∈Λ\(An,ℓ\(h\)\)2qn=⟨∇φ\(ℓ,h\),𝒯k,μ∇φ\(ℓ,h\)⟩L2​\(μ,ℝd\)\.\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi^\{\(\\ell,h\)\},F\_\{1\}\\right\\rangle d\\mu=\\sum\_\{n\\in\\Lambda\}\\frac\{\(A\_\{n,\\ell\}^\{\(h\)\}\)^\{2\}\}\{q\_\{n\}\}=\\left\\langle\\nabla\\varphi^\{\(\\ell,h\)\},\\mathcal\{T\}\_\{k,\\mu\}\\nabla\\varphi^\{\(\\ell,h\)\}\\right\\rangle\_\{L^\{2\}\(\\mu;\{\\mathbb\{R\}\}^\{d\}\)\}\.\(241\)The Stein lower bound of \([217](https://arxiv.org/html/2610.10875#A1.E217)\) therefore implies that

lim infε↓0Dℋk​\(μ,νε\)ε≥\(∑n∈Λ\(An,ℓ\(h\)\)2qn\)1/2,\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\left\(\\sum\_\{n\\in\\Lambda\}\\frac\{\(A\_\{n,\\ell\}^\{\(h\)\}\)^\{2\}\}\{q\_\{n\}\}\\right\)^\{1/2\},\(242\)which we combine with \([234](https://arxiv.org/html/2610.10875#A1.E234)\) to find that

∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)⋅lim infε↓0Dℋk​\(μ,νε\)ε\\displaystyle\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}≥\(∑n∈Λ\(An,ℓ\(h\)\)2qn\)1/2\\displaystyle\\geq\\left\(\\sum\_\{n\\in\\Lambda\}\\frac\{\(A\_\{n,\\ell\}^\{\(h\)\}\)^\{2\}\}\{q\_\{n\}\}\\right\)^\{1/2\}\(243\)⋅\(∑n∈ΛNn,d​qn𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\]\)1/2\.\\displaystyle\\quad\\cdot\\left\(\\sum\_\{n\\in\\Lambda\}\\frac\{N\_\{n,d\}q\_\{n\}\}\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\}\\right\)^\{1/2\}\.\(244\)Applying Cauchy\-Schwarz inℝ\|Λ\|\{\\mathbb\{R\}\}^\{\|\\Lambda\|\}, we find that for all finiteΛ⊆ℕ\\Lambda\\subseteq\{\\mathbb\{N\}\}withqn\>0q\_\{n\}\>0forn∈Λn\\in\\Lambdawe have

∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)⋅lim infε↓0Dℋk​\(μ,νε\)ε≥∑n∈ΛNn,d​\|An,ℓ\(h\)\|𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\]\.\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\sum\_\{n\\in\\Lambda\}\\frac\{\\sqrt\{N\_\{n,d\}\}\\left\|A\_\{n,\\ell\}^\{\(h\)\}\\right\|\}\{\\sqrt\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\}\}\.\(245\)Ifqn=0q\_\{n\}=0butAn,ℓ\(h\)≠0A\_\{n,\\ell\}^\{\(h\)\}\\neq 0then the left hand side must be infinite, and so this holds for all finiteΛ⊆ℕ\\Lambda\\subseteq\{\\mathbb\{N\}\}\. On any such finite subset we may takeℓ→∞\\ell\\to\\inftyand apply \([236](https://arxiv.org/html/2610.10875#A1.E236)\) to get

∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)⋅lim infε↓0Dℋk​\(μ,νε\)ε≥∑n∈ΛNn,d⋅An\(h\)𝔼⁡\[h′​\(R\)2\]\+n⁡\(n\+d−2\)​𝔼​\[h​\(R\)2/R2\]\.\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\sum\_\{n\\in\\Lambda\}\\frac\{\\sqrt\{N\_\{n,d\}\}\\cdot A\_\{n\}^\{\(h\)\}\}\{\\sqrt\{\\mathbb\{E\}\[h^\{\\prime\}\(R\)^\{2\}\]\+n\(n\+d\-2\)\\mathbb\{E\}\[h\(R\)^\{2\}/R^\{2\}\]\}\}\.\(246\)
We will now takeh⁡\(r\)h\(r\)to be smooth compactly\-supported approximations toh⋆​\(r\)=r2h\_\{\\star\}\(r\)=r^\{2\}\. Indeed, we define

Jn⋆:𝕐nd→L2​\(μ,ℝd\),\(Jn⋆​Y\)​\(x\):=∇x\[h⋆​\(\|x\|\)​Y​\(x\|x\|\)\],J\_\{n\}^\{\\star\}:\\mathbb\{Y\}\_\{n\}^\{d\}\\to L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\),\\quad\(J\_\{n\}^\{\\star\}Y\)\(x\):=\\nabla\_\{x\}\\left\[h\_\{\\star\}\(\|x\|\)Y\\left\(\\frac\{x\}\{\|x\|\}\\right\)\\right\],\(247\)takeχ∈Cc∞​\(ℝ\)\\chi\\in C\_\{c\}^\{\\infty\}\(\{\\mathbb\{R\}\}\)with0≤χ≤10\\leq\\chi\\leq 1equal to11on\[−1,1\]\[\-1,1\]and equal to00outside\[−2,2\]\[\-2,2\], and set

hj​\(r\):=r2​χ​\(r/j\)​\(1−χ⁡\(j​r\)\)\.h\_\{j\}\(r\):=r^\{2\}\\chi\(r/j\)\(1\-\\chi\(jr\)\)\.\(248\)Then,\|hj′​\(r\)\|\+\|hj​\(r\)/r\|≲r\|h^\{\\prime\}\_\{j\}\(r\)\|\+\|h\_\{j\}\(r\)/r\|\\lesssim runiformly inrrand converge pointwise to2​r2randrr, respectively\. By dominated convergence,hj′/2,hj/r→rh\_\{j\}^\{\\prime\}/2,h\_\{j\}/r\\to rinL2⁡\(Law⁡\(R\)\)\\operatorname\{L\}^\{2\}\(\\operatorname\{Law\}\(R\)\)\. Thus, if we defineAn\(hj\)A\_\{n\}^\{\(h\_\{j\}\)\}andJn\(hj\)J\_\{n\}^\{\(h\_\{j\}\)\}analogously to \([235](https://arxiv.org/html/2610.10875#A1.E235)\) and \([247](https://arxiv.org/html/2610.10875#A1.E247)\) then we haveJn\(hj\)→Jn⋆J\_\{n\}^\{\(h\_\{j\}\)\}\\to J\_\{n\}^\{\\star\}uniformly on the unit ball of𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}\. Therefore,

An\(hj\)→supY∈𝕐nd‖Y‖L2​\(σ\)=1\|∫ℝd⟨Jn⋆​Y,F1⟩​𝑑μ\|=:An\(∀n∈ℕ\)\.A\_\{n\}^\{\(h\_\{j\}\)\}\\to\\sup\_\{\\begin\{subarray\}\{c\}Y\\in\\mathbb\{Y\}\_\{n\}^\{d\}\\\\ \\\|Y\\\|\_\{L^\{2\}\(\\sigma\)\}=1\\end\{subarray\}\}\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle J\_\{n\}^\{\\star\}Y,F\_\{1\}\\right\\rangle d\\mu\\right\|=:A\_\{n\}\\quad\(\\forall n\\in\{\\mathbb\{N\}\}\)\.\(249\)We also find that

M2​\(μ\)2:=𝔼X∼μ​\|X\|2=14​𝔼​\[h⋆′​\(R\)2\]=𝔼⁡\[h⋆​\(R\)2R2\],M\_\{2\}\(\\mu\)^\{2\}:=\\mathbb\{E\}\_\{X\\sim\\mu\}\|X\|^\{2\}=\\frac\{1\}\{4\}\\mathbb\{E\}\[h^\{\\prime\}\_\{\\star\}\(R\)^\{2\}\]=\\mathbb\{E\}\\left\[\\frac\{h\_\{\\star\}\(R\)^\{2\}\}\{R^\{2\}\}\\right\],\(250\)which we plug into \([246](https://arxiv.org/html/2610.10875#A1.E246)\) to see

∫ℝdtr⁡k⁡\(x,x\)​𝑑μ​\(x\)⋅lim infε↓0Dℋk​\(μ,νε\)ε≥1M2​\(μ\)​∑n∈ΛNn,d⋅An4\+n⁡\(n\+d−2\)\.\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\mu\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\frac\{1\}\{M\_\{2\}\(\\mu\)\}\\sum\_\{n\\in\\Lambda\}\\frac\{\\sqrt\{N\_\{n,d\}\}\\cdot A\_\{n\}\}\{\\sqrt\{4\+n\(n\+d\-2\)\}\}\.\(251\)This holds for any finite subsetΛ⊆ℕ\\Lambda\\subseteq\{\\mathbb\{N\}\}, and we may increase it monotonically to hold for any subsetΛ⊆ℕ\\Lambda\\subseteq\{\\mathbb\{N\}\}\. The claim \([212](https://arxiv.org/html/2610.10875#A1.E212)\) then follows from combining \([218](https://arxiv.org/html/2610.10875#A1.E218)\) and2​n\+d−2≥n\+d−22n\+d\-2\\geq n\+d\-2to find

Nn,d4\+n⁡\(n\+d−2\)≥Nn,d5​n​\(n\+d−2\)≥15​n​\(n\+d−2\)\(n\+d−2d−2\)\(∀n∈ℕ,d≥3\)\.\\frac\{N\_\{n,d\}\}\{4\+n\(n\+d\-2\)\}\\geq\\frac\{N\_\{n,d\}\}\{5n\(n\+d\-2\)\}\\geq\\frac\{1\}\{5n\(n\+d\-2\)\}\{n\+d\-2\\choose d\-2\}\\quad\(\\forall n\\in\{\\mathbb\{N\}\},\\;d\\geq 3\)\.\(252\)∎

In the case whereμ=γd\\mu=\\gamma\_\{d\}is the Gaussian andwt=ww\_\{t\}=wis a single\-index drift, we can get matching lower and upper bounds on the infinitesimal behavior ofDℋkD\_\{\\mathcal\{H\}\_\{k\}\}in terms of the Hermite spectrum of the activation function:

###### Lemma 10\(Rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}bounds for single\-index Gaussian flows\)\.

Fixd≥3d\\geq 3\. Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}beCd\+2C^\{d\+2\}with‖ϕ′‖Cd\+1<∞\\\|\\phi^\{\\prime\}\\\|\_\{C^\{d\+1\}\}<\\inftyand fixσ\>0\\sigma\>0andu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}\. For eachε\>0\\varepsilon\>0defineνε:=Law⁡\(Xε\)\\nu\_\{\\varepsilon\}:=\\operatorname\{Law\}\(X\_\{\\varepsilon\}\)according to the flow

X˙t=σ​u​ϕ​\(⟨Xt,θ⟩\),X0∼γd\.\\dot\{X\}\_\{t\}=\\sigma u\\phi\(\\left\\langle X\_\{t\},\\theta\\right\\rangle\),\\quad X\_\{0\}\\sim\\gamma\_\{d\}\.\(253\)If we denote the quantity

Q:=infk∈𝒦rotk​satisfies Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)∈\(0,∞\)\{∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)⋅lim infε↓0Dℋk​\(γd,νε\)ε\}Q:=\\inf\_\{\\begin\{subarray\}\{c\}k\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}\\\\ k\\text\{ satisfies Assumption \\ref\{assumption:kernel\}\}\\\\ \\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\\in\(0,\\infty\)\\end\{subarray\}\}\\left\\\{\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\right\\\}\(254\)and use the probabilist’s Hermite222222By this, we mean that\{Hej\}j\\\{\\operatorname\{He\}\_\{j\}\\\}\_\{j\}are the monic Hermite polynomials with⟨Hei,Hej⟩L2​\(γ1\)=δi,j⋅j\!\\left\\langle\\operatorname\{He\}\_\{i\},\\operatorname\{He\}\_\{j\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{1\}\)\}=\\delta\_\{i,j\}\\cdot j\!\. In particular, there is the decompositionϕ=∑j=0∞ϕ^j/j\!​Hej\\phi=\\sum\_\{j=0\}^\{\\infty\}\{\\hat\{\\phi\}\_\{j\}\}/\{j\!\}\\operatorname\{He\}\_\{j\}inL2​\(γ1\)L^\{2\}\(\\gamma\_\{1\}\)\.coefficientsϕ^j:=𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ⁡\(ξ\)​Hej⁡\(ξ\)\]\\hat\{\\phi\}\_\{j\}:=\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\phi\(\\xi\)\\operatorname\{He\}\_\{j\}\(\\xi\)\]to define

𝒟ϕ​\(c\):=∑n=0∞Nn,d​\(∑ℓ=0n\+2​ℓ≥1∞\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)⋅\|ϕ^m−1\|2⋅\(c2\+\(1−c2\)​n⁡\(n\+d−2\)m2​\(d−1\)\)\)1/2\\mathcal\{D\}\_\{\\phi\}\(c\):=\\sum\_\{n=0\}^\{\\infty\}N\_\{n,d\}\\left\(\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{\\infty\}\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)\\cdot\|\\hat\{\\phi\}\_\{m\-1\}\|^\{2\}\\cdot\\left\(c^\{2\}\+\(1\-c^\{2\}\)\\frac\{n\(n\+d\-2\)\}\{m^\{2\}\(d\-1\)\}\\right\)\\right\)^\{1/2\}\(255\)withm=n\+2​ℓm=n\+2\\ell, then there are the lower and upper bounds

σ​𝒟ϕ​\(⟨u,θ⟩\)≤Q≤2​σ​𝒟ϕ​\(⟨u,θ⟩\)\.\\sigma\\mathcal\{D\}\_\{\\phi\}\(\\left\\langle u,\\theta\\right\\rangle\)\\leq Q\\leq\\sqrt\{2\}\\sigma\\mathcal\{D\}\_\{\\phi\}\(\\left\\langle u,\\theta\\right\\rangle\)\.\(256\)The lower bound holds under the weaker assumption thatϕ\\phiis globally\-Lipschitz\.

###### Proof\.

Fixw:ℝd→ℝdw:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}globally\-Lipschitz\. For everym∈ℕm\\in\{\\mathbb\{N\}\}, themt​hm^\{th\}scalar Hermite eigenspaceℋm\\mathcal\{H\}\_\{m\}splits orthogonally as

ℋm=⨁n,ℓ∈ℕ∪\{0\}m=n\+2​ℓℋn,ℓ\\mathcal\{H\}\_\{m\}=\\bigoplus\_\{\\begin\{subarray\}\{c\}n,\\ell\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\\\\ m=n\+2\\ell\\end\{subarray\}\}\\mathcal\{H\}\_\{n,\\ell\}\(257\)where eachℋn,ℓ\\mathcal\{H\}\_\{n,\\ell\}is isomorphic to𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}\(here,nnis the angular momentum/spherical harmonic degree andℓ\\ellis the radial index\)\. Choose the orthonormal basis\{Pn,ℓ,i\}i=1Nn,d\\\{P\_\{n,\\ell,i\}\\\}\_\{i=1\}^\{N\_\{n,d\}\}ofℋn,ℓ\\mathcal\{H\}\_\{n,\\ell\}w\.r\.t\.L2​\(γd\)L^\{2\}\(\\gamma\_\{d\}\)given by

Pn,ℓ,i​\(r​ω\):=\(−1\)ℓ​ℓ\!​Γ​\(d/2\)2n​Γ​\(n\+ℓ\+d/2\)​rn​Lℓ\(n\+d/2−1\)​\(r2/2\)​Yn,i​\(ω\)P\_\{n,\\ell,i\}\(r\\omega\):=\(\-1\)^\{\\ell\}\\sqrt\{\\frac\{\\ell\!\\Gamma\(d/2\)\}\{2^\{n\}\\Gamma\(n\+\\ell\+d/2\)\}\}r^\{n\}L\_\{\\ell\}^\{\(n\+d/2\-1\)\}\(r^\{2\}/2\)Y\_\{n,i\}\(\\omega\)\(258\)forr≥0r\\geq 0andω∈𝕊d−1\\omega\\in\\mathbb\{S\}^\{d\-1\}, where\{Yn,i\}i=1Nn,d\\\{Y\_\{n,i\}\\\}\_\{i=1\}^\{N\_\{n,d\}\}is theL2​\(σ\)L^\{2\}\(\\sigma\)\-orthonormal basis of𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}from the proof of Lemma[9](https://arxiv.org/html/2610.10875#Thmlemma9)forσ\\sigmathe uniform probability measure on the sphere, andLℓ\(α\)L\_\{\\ell\}^\{\(\\alpha\)\}are the generalized Laguerre polynomials\. This is the standard Laguerre\-spherical harmonic representation of the Hermite eigenspaces, see theμ=0\\mu=0case of\[[71](https://arxiv.org/html/2610.10875#bib.bib24), Definition 2\.1 and Theorem 2\.3\]after rescaling their Gaussian weighte−r2e^\{\-r^\{2\}\}toe−r2/2e^\{\-r^\{2\}/2\}and normalizing the Gaussian and spherical measures to be probability measures\. Forn\+2​ℓ≥1n\+2\\ell\\geq 1we set

En,ℓ,i:=1n\+2​ℓ∇Pn,ℓ,iE\_\{n,\\ell,i\}:=\\frac\{1\}\{\\sqrt\{n\+2\\ell\}\}\\nabla P\_\{n,\\ell,i\}\(259\)so that\{En,ℓ,i\}i=1Nn,d\\\{E\_\{n,\\ell,i\}\\\}\_\{i=1\}^\{N\_\{n,d\}\}forms an orthonormal basis of the Gaussian gradient fields \(i\.e\. the image ofℋn,ℓ\\mathcal\{H\}\_\{n,\\ell\}under the isometryf↦1n\+2​ℓ∇ff\\mapsto\\frac\{1\}\{\\sqrt\{n\+2\\ell\}\}\\nabla f, equipped with theL2​\(γd,ℝd\)L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)inner product\)\. For eachn∈ℕn\\in\{\\mathbb\{N\}\}, we define the operatorCn:ℝNn,d→ℓ2​\(ℕ∪\{0\}\)C\_\{n\}:\{\\mathbb\{R\}\}^\{N\_\{n,d\}\}\\to\\ell^\{2\}\(\{\\mathbb\{N\}\}\\cup\\\{0\\\}\)by the matrix elements

\(Cn\)ℓ,i:=⟨w,En,ℓ,i⟩L2​\(γd,ℝd\)\(C\_\{n\}\)\_\{\\ell,i\}:=\\left\\langle w,E\_\{n,\\ell,i\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\(260\)withℓ∈ℕ∪\{0\}\\ell\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}andi∈\{1,…,Nn,d\}i\\in\\\{1,\\ldots,N\_\{n,d\}\\\}\. Forn=0n=0, we do the same but withℓ∈ℕ\\ell\\in\{\\mathbb\{N\}\}andN0,d=1N\_\{0,d\}=1to getC0:ℝ→ℓ2​\(ℕ\)C\_\{0\}:\{\\mathbb\{R\}\}\\to\\ell^\{2\}\(\{\\mathbb\{N\}\}\)\. Finally, we set

𝒟~​\(w\):=∑n=0∞Nn,d​tr⁡\(Cn​Cn⊤\)\.\\widetilde\{\\mathcal\{D\}\}\(w\):=\\sum\_\{n=0\}^\{\\infty\}\\sqrt\{N\_\{n,d\}\}\\operatorname\{tr\}\\left\(\\sqrt\{C\_\{n\}C\_\{n\}^\{\\top\}\}\\right\)\.\(261\)We will first obtain matching lower and upper bounds to show thatQ=𝒟~​\(w\)Q=\\widetilde\{\\mathcal\{D\}\}\(w\)if we defineνε\\nu\_\{\\varepsilon\}using the more general flowX˙t=w⁡\(Xt\)\\dot\{X\}\_\{t\}=w\(X\_\{t\}\); the lower bound will hold for globally\-Lipschitzww, while the upper bound will require extra smoothness\. Afterward, we specialize tow⁡\(x\)=σ​u​ϕ​\(⟨x,θ⟩\)w\(x\)=\\sigma u\\phi\(\\left\\langle x,\\theta\\right\\rangle\)and relate𝒟~​\(w\)\\widetilde\{\\mathcal\{D\}\}\(w\)to our explicit quantity𝒟ϕ​\(c\)\\mathcal\{D\}\_\{\\phi\}\(c\)\.

To show thatQQis lower bounded by𝒟~​\(w\)\\widetilde\{\\mathcal\{D\}\}\(w\)for any globally\-Lipschitzww, we use the same Schur’s lemma trace calculation from the proof of Lemma[9](https://arxiv.org/html/2610.10875#Thmlemma9)\. Fix anyk∈𝒦rotk\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}satisfying Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)with∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)∈\(0,∞\)\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\\in\(0,\\infty\)\. We know that for a fixedn∈ℕn\\in\{\\mathbb\{N\}\}the spaces\(ℋn,ℓ\)ℓ=0∞\(\\mathcal\{H\}\_\{n,\\ell\}\)\_\{\\ell=0\}^\{\\infty\}are all isomorphic to𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}and transform in the same way under rotations \(as do their gradient spaces\)\. By Schur’s lemma and rotational equivariance of𝒯k,γd\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}, there is some matrixAn⪰0A\_\{n\}\\succeq 0for which

⟨En,ℓ,i,𝒯k,γd​En′,r,j⟩L2​\(γd,ℝd\)=𝟙n=n′​\(An\)ℓ,r​𝟙i=j\(∀ℓ,r∈ℕ∪\{0\}\)\\left\\langle E\_\{n,\\ell,i\},\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}E\_\{n^\{\\prime\},r,j\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}=\{\\mathbbm\{1\}\}\_\{n=n^\{\\prime\}\}\(A\_\{n\}\)\_\{\\ell,r\}\{\\mathbbm\{1\}\}\_\{i=j\}\\quad\(\\forall\\ell,r\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\)\(262\)for alli∈\{1,…,Nn,d\}i\\in\\\{1,\\ldots,N\_\{n,d\}\\\}andj∈\{1,…,Nn′,d\}j\\in\\\{1,\\ldots,N\_\{n^\{\\prime\},d\}\\\}, while ifn=0n=0the same is true forℓ,r∈ℕ\\ell,r\\in\{\\mathbb\{N\}\}\. By summing over the orthonormal basis of the gradient subspace ofL2​\(γd,ℝd\)L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)and applying \([262](https://arxiv.org/html/2610.10875#A1.E262)\), we find that

tr⁡\(𝒯k,γd\)\\displaystyle\\operatorname\{tr\}\(\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\)≥∑n=0∞∑ℓ=0n\+2​ℓ≥1∞∑i=1Nn,d⟨En,ℓ,i,𝒯k,γd​En,ℓ,i⟩L2​\(γd,ℝd\)\\displaystyle\\geq\\sum\_\{n=0\}^\{\\infty\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{\\infty\}\\sum\_\{i=1\}^\{N\_\{n,d\}\}\\left\\langle E\_\{n,\\ell,i\},\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}E\_\{n,\\ell,i\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\(263\)=∑n=0∞Nn,d​∑ℓ=0n\+2​ℓ≥1∞\(An\)ℓ,ℓ\\displaystyle=\\sum\_\{n=0\}^\{\\infty\}N\_\{n,d\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{\\infty\}\(A\_\{n\}\)\_\{\\ell,\\ell\}\(264\)=∑n=0∞Nn,d​tr⁡\(An\)\.\\displaystyle=\\sum\_\{n=0\}^\{\\infty\}N\_\{n,d\}\\operatorname\{tr\}\(A\_\{n\}\)\.\(265\)In particular, eachAnA\_\{n\}is trace\-class\. Now, fixL∈ℕL\\in\{\\mathbb\{N\}\}and truncateCn,L:ℝNn,d→ℝL\+1C\_\{n,L\}:\{\\mathbb\{R\}\}^\{N\_\{n,d\}\}\\to\{\\mathbb\{R\}\}^\{L\+1\}\(andC0,L:ℝ→ℝLC\_\{0,L\}:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}^\{L\}\)\. If we take any finite sequence of matrices\(Zn\)n=0N\(Z\_\{n\}\)\_\{n=0\}^\{N\}of the same shapes asCn,LC\_\{n,L\}supported onℓ≤L\\ell\\leq Land define

φZ:=∑n=0N∑ℓ=0n\+2​ℓ≥1L1n\+2​ℓ​∑i=1Nn,d\(Zn\)ℓ,i​Pn,ℓ,i\\varphi\_\{Z\}:=\\sum\_\{n=0\}^\{N\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{L\}\\frac\{1\}\{\\sqrt\{n\+2\\ell\}\}\\sum\_\{i=1\}^\{N\_\{n,d\}\}\(Z\_\{n\}\)\_\{\\ell,i\}P\_\{n,\\ell,i\}\(266\)then we have

∇φZ=∑n=0N∑ℓ=0n\+2​ℓ≥1L∑i=1Nn,d\(Zn\)ℓ,i​En,ℓ,i\.\\nabla\\varphi\_\{Z\}=\\sum\_\{n=0\}^\{N\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{L\}\\sum\_\{i=1\}^\{N\_\{n,d\}\}\(Z\_\{n\}\)\_\{\\ell,i\}E\_\{n,\\ell,i\}\.\(267\)Furthermore, by definition of\(Cn\)n\(C\_\{n\}\)\_\{n\}we know that

⟨∇φZ,w⟩L2​\(γd,ℝd\)\\displaystyle\\left\\langle\\nabla\\varphi\_\{Z\},w\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}=∑n=0N∑ℓ=0n\+2​ℓ≥1L∑i=1Nn,d⟨w,En,ℓ,i⟩L2​\(γd,ℝd\)​⟨∇φZ,En,ℓ,i⟩L2​\(γd,ℝd\)\\displaystyle=\\sum\_\{n=0\}^\{N\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{L\}\\sum\_\{i=1\}^\{N\_\{n,d\}\}\\left\\langle w,E\_\{n,\\ell,i\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\\left\\langle\\nabla\\varphi\_\{Z\},E\_\{n,\\ell,i\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\(268\)=∑n=0Ntr⁡\(Zn⊤​Cn,L\)\.\\displaystyle=\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}C\_\{n,L\}\\right\)\.\(269\)On the other hand, by using \([262](https://arxiv.org/html/2610.10875#A1.E262)\) and the same orthonormal basis we know that if we truncateAn,L:ℝL\+1→ℝL\+1A\_\{n,L\}:\{\\mathbb\{R\}\}^\{L\+1\}\\to\{\\mathbb\{R\}\}^\{L\+1\}\(withA0,L:ℝL→ℝLA\_\{0,L\}:\{\\mathbb\{R\}\}^\{L\}\\to\{\\mathbb\{R\}\}^\{L\}\) then

⟨∇φZ,𝒯k,γd∇φZ⟩L2​\(γd,ℝd\)\\displaystyle\\left\\langle\\nabla\\varphi\_\{Z\},\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\\nabla\\varphi\_\{Z\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}=∑n=0Ntr⁡\(Zn⊤​An,L​Zn\)\.\\displaystyle=\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}A\_\{n,L\}Z\_\{n\}\\right\)\.\(270\)If we take a cutoffχR∈Cc∞​\(ℝd\)\\chi\_\{R\}\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\)which is 1 onBRB\_\{R\}and 0 outsideB2​RB\_\{2R\}with‖∇χR‖≲1/R\\\|\\nabla\\chi\_\{R\}\\\|\\lesssim 1/Rand denoteφZ,R:=χR​φZ\\varphi\_\{Z,R\}:=\\chi\_\{R\}\\varphi\_\{Z\}, then we knowφZ,R∈Cc∞\\varphi\_\{Z,R\}\\in C^\{\\infty\}\_\{c\}and∇φZ,R→∇φZ\\nabla\\varphi\_\{Z,R\}\\to\\nabla\\varphi\_\{Z\}inL2​\(γd,ℝd\)L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\. By boundedness of𝒯k,γd\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}we also know⟨∇φZ,R,𝒯k,γd∇φZ,R⟩L2​\(γd,ℝd\)→⟨∇φZ,𝒯k,γd∇φZ⟩L2​\(γd,ℝd\)\\left\\langle\\nabla\\varphi\_\{Z,R\},\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\\nabla\\varphi\_\{Z,R\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\\to\\left\\langle\\nabla\\varphi\_\{Z\},\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\\nabla\\varphi\_\{Z\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\. Hence, we may apply Lemma[8](https://arxiv.org/html/2610.10875#Thmlemma8)toφZ,R\\varphi\_\{Z,R\}and takeR→∞R\\to\\inftyto get

lim infε↓0Dℋk​\(γd,νε\)ε≥⟨∇φZ,w⟩L2​\(γd,ℝd\)⟨∇φZ,𝒯k,γd∇φZ⟩L2​\(γd,ℝd\)=∑n=0Ntr⁡\(Zn⊤​Cn,L\)\(∑n=0Ntr⁡\(Zn⊤​An,L​Zn\)\)1/2\.\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\frac\{\\left\\langle\\nabla\\varphi\_\{Z\},w\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\}\{\\sqrt\{\\left\\langle\\nabla\\varphi\_\{Z\},\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\\nabla\\varphi\_\{Z\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\}\}=\\frac\{\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}C\_\{n,L\}\\right\)\}\{\\left\(\{\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}A\_\{n,L\}Z\_\{n\}\\right\)\}\\right\)^\{1/2\}\}\.\(271\)This holds for any\(Zn\)n=0N\(Z\_\{n\}\)\_\{n=0\}^\{N\}and anyN∈ℕN\\in\{\\mathbb\{N\}\}, i\.e\.

lim infε↓0Dℋk​\(γd,νε\)ε≥supN,L∈ℕ,\(Zn\)n=0N∑n=0Ntr⁡\(Zn⊤​Cn,L\)\(∑n=0Ntr⁡\(Zn⊤​An,L​Zn\)\)1/2\.\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\sup\_\{\\begin\{subarray\}\{c\}N,L\\in\{\\mathbb\{N\}\},\\\\ \(Z\_\{n\}\)\_\{n=0\}^\{N\}\\end\{subarray\}\}\\frac\{\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}C\_\{n,L\}\\right\)\}\{\\left\(\{\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}A\_\{n,L\}Z\_\{n\}\\right\)\}\\right\)^\{1/2\}\}\.\(272\)We may suppose without loss of generality thatim⁡\(Cn,L\)⊆im⁡\(An,L\)\\operatorname\{im\}\(C\_\{n,L\}\)\\subseteq\\operatorname\{im\}\(A\_\{n,L\}\); if not, then we can choose a test direction inker⁡\(An,L\)\\operatorname\{ker\}\(A\_\{n,L\}\)with nonzero numerator to make the right hand side of \([272](https://arxiv.org/html/2610.10875#A1.E272)\) infinite, and so the result is trivial\.

To evaluate this supremum, we fixN,L∈ℕN,L\\in\{\\mathbb\{N\}\}and first assumeAn,LA\_\{n,L\}is invertible for eachn≤Nn\\leq N\. PutWn:=An,L1/2​ZnW\_\{n\}:=A\_\{n,L\}^\{1/2\}Z\_\{n\}to see thattr\(Zn⊤Cn,L\)=⟨Wn,An,L−1/2Cn,L⟩F\\operatorname\{tr\}\(Z\_\{n\}^\{\\top\}C\_\{n,L\}\)=\\left\\langle W\_\{n\},A\_\{n,L\}^\{\-1/2\}C\_\{n,L\}\\right\\rangle\_\{F\}whiletr⁡\(Zn⊤​An,L​Zn\)=‖Wn‖F2\\operatorname\{tr\}\(Z\_\{n\}^\{\\top\}A\_\{n,L\}Z\_\{n\}\)=\\\|W\_\{n\}\\\|\_\{F\}^\{2\}\. Hence, this is simply the dual norm in the Frobenius inner product, i\.e\.

sup\(Zn\)n=0N∑n=0Ntr⁡\(Zn⊤​Cn,L\)\(∑n=0Ntr⁡\(Zn⊤​An,L​Zn\)\)1/2\\displaystyle\\sup\_\{\\begin\{subarray\}\{c\}\(Z\_\{n\}\)\_\{n=0\}^\{N\}\\end\{subarray\}\}\\frac\{\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}C\_\{n,L\}\\right\)\}\{\\left\(\{\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(Z\_\{n\}^\{\\top\}A\_\{n,L\}Z\_\{n\}\\right\)\}\\right\)^\{1/2\}\}=\(∑n=0N‖An,L−1/2Cn,L‖F2\)1/2\\displaystyle=\\left\(\\sum\_\{n=0\}^\{N\}\\left\\\|A\_\{n,L\}^\{\-1/2\}C\_\{n,L\}\\right\\\|\_\{F\}^\{2\}\\right\)^\{1/2\}\(273\)=\(∑n=0Ntr⁡\(An,L−1​Cn,L​Cn,L⊤\)\)1/2\.\\displaystyle=\\left\(\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(A\_\{n,L\}^\{\-1\}C\_\{n,L\}C\_\{n,L\}^\{\\top\}\\right\)\\right\)^\{1/2\}\.\(274\)IfAn,LA\_\{n,L\}is not invertible, the same holds with the pseudoinverse sinceim⁡\(Cn,L\)⊆im⁡\(An,L\)\\operatorname\{im\}\(C\_\{n,L\}\)\\subseteq\\operatorname\{im\}\(A\_\{n,L\}\)\. Hence, we always have

lim infε↓0Dℋk​\(γd,νε\)ε≥supN,L∈ℕ\(∑n=0Ntr⁡\(An,L†​Cn,L​Cn,L⊤\)\)1/2\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\sup\_\{N,L\\in\{\\mathbb\{N\}\}\}\\left\(\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(A\_\{n,L\}^\{\\dagger\}C\_\{n,L\}C\_\{n,L\}^\{\\top\}\\right\)\\right\)^\{1/2\}\(275\)and sincetr⁡\(𝒯k,γd\)≥∑n=0∞Nn,d​tr⁡\(An\)≥∑n=0NNn,d​tr⁡\(An,L\)\\operatorname\{tr\}\(\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\)\\geq\\sum\_\{n=0\}^\{\\infty\}N\_\{n,d\}\\operatorname\{tr\}\(A\_\{n\}\)\\geq\\sum\_\{n=0\}^\{N\}N\_\{n,d\}\\operatorname\{tr\}\(A\_\{n,L\}\)for allN,LN,Lwe know

tr⁡\(𝒯k,γd\)​lim infε↓0Dℋk​\(γd,νε\)ε\\displaystyle\\sqrt\{\\operatorname\{tr\}\(\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\)\}\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}≥\(∑n=0NNn,d​tr⁡\(An,L\)\)1/2⋅\(∑n=0Ntr⁡\(An,L†​Cn,L​Cn,L⊤\)\)1/2\\displaystyle\\geq\\left\(\\sum\_\{n=0\}^\{N\}N\_\{n,d\}\\operatorname\{tr\}\\left\(A\_\{n,L\}\\right\)\\right\)^\{1/2\}\\cdot\\left\(\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(A\_\{n,L\}^\{\\dagger\}C\_\{n,L\}C\_\{n,L\}^\{\\top\}\\right\)\\right\)^\{1/2\}\(276\)≥∑n=0NNn,d​tr⁡\(An,L\)​tr⁡\(An,L†​Cn,L​Cn,L⊤\),\\displaystyle\\geq\\sum\_\{n=0\}^\{N\}\\sqrt\{N\_\{n,d\}\}\\sqrt\{\\operatorname\{tr\}\\left\(A\_\{n,L\}\\right\)\\operatorname\{tr\}\\left\(A\_\{n,L\}^\{\\dagger\}C\_\{n,L\}C\_\{n,L\}^\{\\top\}\\right\)\},\(277\)where the second line uses Cauchy\-Schwarz\(∑nan\)​\(∑nbn\)≥∑nan​bn\\sqrt\{\(\\sum\_\{n\}a\_\{n\}\)\(\\sum\_\{n\}b\_\{n\}\)\}\\geq\\sum\_\{n\}\\sqrt\{a\_\{n\}b\_\{n\}\}\. ForA,B⪰0A,B\\succeq 0withim⁡\(B\)⊆im⁡\(A\)\\operatorname\{im\}\(B\)\\subseteq\\operatorname\{im\}\(A\)we know

tr\(B\)=tr\(A1/2A†/2B\)≤∥A1/2∥F∥A†/2B∥F=tr⁡\(A\)tr⁡\(A†​B\)\.\\operatorname\{tr\}\(\\sqrt\{B\}\)=\\operatorname\{tr\}\(A^\{1/2\}A^\{\\dagger/2\}\\sqrt\{B\}\)\\leq\\\|A^\{1/2\}\\\|\_\{F\}\\\|A^\{\\dagger/2\}\\sqrt\{B\}\\\|\_\{F\}=\\sqrt\{\\operatorname\{tr\}\(A\)\}\\sqrt\{\\operatorname\{tr\}\(A^\{\\dagger\}B\)\}\.\(278\)TakingA=An,LA=A\_\{n,L\}andB=Cn,L​Cn,L⊤B=C\_\{n,L\}C\_\{n,L\}^\{\\top\}yields

tr⁡\(𝒯k,γd\)​lim infε↓0Dℋk​\(γd,νε\)ε≥∑n=0NNn,d​tr⁡\(Cn,L​Cn,L⊤\)\.\\sqrt\{\\operatorname\{tr\}\(\\mathcal\{T\}\_\{k,\\gamma\_\{d\}\}\)\}\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\sum\_\{n=0\}^\{N\}\\sqrt\{N\_\{n,d\}\}\\operatorname\{tr\}\\left\(\\sqrt\{C\_\{n,L\}C\_\{n,L\}^\{\\top\}\}\\right\)\.\(279\)We first takeN,L→∞N,L\\to\\inftyusing monotone convergence and then an infimum overkkto arrive at the lower boundQ≥𝒟~​\(w\)Q\\geq\\widetilde\{\\mathcal\{D\}\}\(w\)\.

For the upper bound, assume further thatw∈Cd\+2​\(ℝd,ℝd\)w\\in C^\{d\+2\}\(\{\\mathbb\{R\}\}^\{d\};\{\\mathbb\{R\}\}^\{d\}\)with‖D​w‖Cd\+1<∞\\\|Dw\\\|\_\{C^\{d\+1\}\}<\\infty\. Suppose that𝒟~​\(w\)<∞\\widetilde\{\\mathcal\{D\}\}\(w\)<\\infty, since otherwise there is nothing to prove\. Let us define the Gaussian gradient subspace

𝒢:=\{∇φ:φ∈Cc∞​\(ℝd\)\}¯L2​\(γd,ℝd\)\\mathcal\{G\}:=\\overline\{\\left\\\{\\nabla\\varphi:\\;\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\)\\right\\\}\}^\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\(280\)withΠ𝒢:L2​\(γd,ℝd\)→𝒢\\Pi\_\{\\mathcal\{G\}\}:L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\\to\\mathcal\{G\}its orthogonal projection, and denoteg:=Π𝒢​wg:=\\Pi\_\{\\mathcal\{G\}\}w\. Since eachEn,ℓ,iE\_\{n,\\ell,i\}is a gradient field we know that

⟨w,En,ℓ,i⟩L2​\(γd,ℝd\)=⟨g,En,ℓ,i⟩L2​\(γd,ℝd\)=\(Cn\)ℓ,i\.\\left\\langle w,E\_\{n,\\ell,i\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}=\\left\\langle g,E\_\{n,\\ell,i\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}=\(C\_\{n\}\)\_\{\\ell,i\}\.\(281\)Let us denote by\(Φt\)t≥0\(\\Phi\_\{t\}\)\_\{t\\geq 0\}the flow of the ODE driven bywwandνt:=\(Φt\)\#​γd\\nu\_\{t\}:=\(\\Phi\_\{t\}\)\_\{\\\#\}\\gamma\_\{d\}the resulting trajectory of measures, and soD​Φt​\(x\)​w​\(x\)=w⁡\(Φt​\(x\)\)D\\Phi\_\{t\}\(x\)w\(x\)=w\(\\Phi\_\{t\}\(x\)\)\. If we define

gt​\(x\):=D​Φt​\(Φt−1​\(x\)\)​g​\(Φt−1​\(x\)\),g\_\{t\}\(x\):=D\\Phi\_\{t\}\(\\Phi\_\{t\}^\{\-1\}\(x\)\)g\(\\Phi\_\{t\}^\{\-1\}\(x\)\),\(282\)then writingy=Φt​\(x\)y=\\Phi\_\{t\}\(x\)impliesw⁡\(y\)−gt​\(y\)=D​Φt​\(x\)​\[w⁡\(x\)−g⁡\(x\)\]w\(y\)\-g\_\{t\}\(y\)=D\\Phi\_\{t\}\(x\)\[w\(x\)\-g\(x\)\]\. We claim that\(gt,νt\)t≥0\(g\_\{t\},\\nu\_\{t\}\)\_\{t\\geq 0\}satisfies the continuity equation\. Indeed, for fixedφ∈Cc∞​\(ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\)we have that

∫ℝd⟨∇φ​\(y\),w⁡\(y\)−gt​\(y\)⟩​d​νt​\(y\)\\displaystyle\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\(y\),w\(y\)\-g\_\{t\}\(y\)\\right\\rangle d\\nu\_\{t\}\(y\)=∫ℝd⟨∇φ​\(Φt​\(x\)\),D​Φt​\(x\)​\(w⁡\(x\)−g⁡\(x\)\)⟩​d​γd​\(x\)\\displaystyle=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\(\\Phi\_\{t\}\(x\)\),D\\Phi\_\{t\}\(x\)\(w\(x\)\-g\(x\)\)\\right\\rangle d\\gamma\_\{d\}\(x\)\(283\)=∫ℝd⟨∇\(φ∘Φt\),w−g⟩​d​γd\\displaystyle=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\(\\varphi\\circ\\Phi\_\{t\}\),w\-g\\right\\rangle d\\gamma\_\{d\}\(284\)=0\\displaystyle=0\(285\)since∇\(φ∘Φt\)∈𝒢\\nabla\(\\varphi\\circ\\Phi\_\{t\}\)\\in\\mathcal\{G\}\. This yields

∂tνt\+∇⋅\(νt​gt\)=0,\\partial\_\{t\}\\nu\_\{t\}\+\\nabla\\cdot\(\\nu\_\{t\}g\_\{t\}\)=0,\(286\)and so the drifts\(gt\)t≥0\(g\_\{t\}\)\_\{t\\geq 0\}generate the same trajectory of measures\. In particular, for anykksatisfying Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)if‖gt−g‖ℋk→0\\\|g\_\{t\}\-g\\\|\_\{\\mathcal\{H\}\_\{k\}\}\\to 0ast↓0t\\downarrow 0then

lim supε↓0Dℋk​\(γd,νε\)ε≤lim supε↓01ε​∫0ε‖gt‖ℋk​𝑑t=‖g‖ℋk\.\\limsup\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\leq\\limsup\_\{\\varepsilon\\downarrow 0\}\\frac\{1\}\{\\varepsilon\}\\int\_\{0\}^\{\\varepsilon\}\\\|g\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}dt=\\\|g\\\|\_\{\\mathcal\{H\}\_\{k\}\}\.\(287\)Note that‖g‖Wd\+1,4​\(γd,ℝd\)≲‖w‖Wd\+1,4​\(γd,ℝd\)<∞\\\|g\\\|\_\{W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\\lesssim\\\|w\\\|\_\{W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}<\\inftysinceΠ𝒢\\Pi\_\{\\mathcal\{G\}\}is a bounded operator onWd\+1,4​\(γd,ℝd\)W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\. We claim that our smoothness assumptions imply

limt↓0‖gt−g‖Wd\+1,2​\(γd,ℝd\)=0\.\\lim\_\{t\\downarrow 0\}\\\|g\_\{t\}\-g\\\|\_\{W^\{d\+1,2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}=0\.\(288\)To see this, by Grönwall we know that ifBt:=Φt−1B\_\{t\}:=\\Phi\_\{t\}^\{\-1\}andAt:=D​Φt∘BtA\_\{t\}:=D\\Phi\_\{t\}\\circ B\_\{t\}then

supt∈\[0,t0\]\{‖D​Bt‖Cd\+‖At‖Cd\+1\}<∞\\sup\_\{t\\in\[0,t\_\{0\}\]\}\\left\\\{\\\|DB\_\{t\}\\\|\_\{C^\{d\}\}\+\\\|A\_\{t\}\\\|\_\{C^\{d\+1\}\}\\right\\\}<\\infty\(289\)for somet0t\_\{0\}\. For anyf∈Cc∞​\(ℝd,ℝd\)f\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\};\{\\mathbb\{R\}\}^\{d\}\), the chain and product rules therefore imply that ifUt​f:=At⋅\(f∘Bt\)U\_\{t\}f:=A\_\{t\}\\cdot\(f\\circ B\_\{t\}\)then

∑\|α\|≤d\+1‖Dα​Ut​f​\(x\)‖≤C​∑\|β\|≤d\+1‖Dβ​f​\(Bt​\(x\)\)‖\(∀x∈ℝd\)\\sum\_\{\|\\alpha\|\\leq d\+1\}\\\|D^\{\\alpha\}U\_\{t\}f\(x\)\\\|\\leq C\\sum\_\{\|\\beta\|\\leq d\+1\}\\\|D^\{\\beta\}f\(B\_\{t\}\(x\)\)\\\|\\quad\(\\forall x\\in\{\\mathbb\{R\}\}^\{d\}\)\(290\)for a constantCCuniform inxxandt∈\[0,t0\]t\\in\[0,t\_\{0\}\]\. If we change variablesy=Bt​\(x\)y=B\_\{t\}\(x\)\(i\.e\.x=Φt​\(y\)x=\\Phi\_\{t\}\(y\)\) and define

Rt​\(y\):=\|det⁡D​Φt​\(y\)\|​exp⁡\(‖y‖2−‖Φt​\(y\)‖22\),R\_\{t\}\(y\):=\|\\operatorname\{det\}D\\Phi\_\{t\}\(y\)\|\\exp\\left\(\\frac\{\\\|y\\\|^\{2\}\-\\\|\\Phi\_\{t\}\(y\)\\\|^\{2\}\}\{2\}\\right\),\(291\)then Lipschitzness ofwwimplies thatRt​\(y\)≤eC′​t​\(1\+\|y\|2\)R\_\{t\}\(y\)\\leq e^\{C^\{\\prime\}t\(1\+\|y\|^\{2\}\)\}for some constantC′∈\(0,∞\)C^\{\\prime\}\\in\(0,\\infty\)\. In particular, after shrinkingt0t\_\{0\}we know that there is someC′′∈\(0,∞\)C^\{\\prime\\prime\}\\in\(0,\\infty\)for which

supt∈\[0,t0\]‖Rt‖L2​\(γd\)≤C′′<∞\.\\sup\_\{t\\in\[0,t\_\{0\}\]\}\\\|R\_\{t\}\\\|\_\{L^\{2\}\(\\gamma\_\{d\}\)\}\\leq C^\{\\prime\\prime\}<\\infty\.\(292\)Hence, \([290](https://arxiv.org/html/2610.10875#A1.E290)\) and the change of measure formula yield

‖Ut​f‖Wd\+1,2​\(γd,ℝd\)2\\displaystyle\\\|U\_\{t\}f\\\|\_\{W^\{d\+1,2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}^\{2\}≤C​∑\|β\|≤d\+1∫ℝd‖Dβ​f​\(Bt​\(x\)\)‖2​d​γd​\(x\)\\displaystyle\\leq C\\sum\_\{\|\\beta\|\\leq d\+1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|D^\{\\beta\}f\(B\_\{t\}\(x\)\)\\\|^\{2\}d\\gamma\_\{d\}\(x\)\(293\)≤C​∑\|β\|≤d\+1∫ℝd‖Dβ​f​\(y\)‖2​Rt​\(y\)​d​γd​\(y\)\\displaystyle\\leq C\\sum\_\{\|\\beta\|\\leq d\+1\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|D^\{\\beta\}f\(y\)\\\|^\{2\}R\_\{t\}\(y\)d\\gamma\_\{d\}\(y\)\(294\)≤C​‖f‖Wd\+1,4​\(γd,ℝd\)2⋅‖Rt‖L2​\(γd\)\\displaystyle\\leq C\\\|f\\\|\_\{W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}^\{2\}\\cdot\\\|R\_\{t\}\\\|\_\{L^\{2\}\(\\gamma\_\{d\}\)\}\(295\)≤C​C′′​‖f‖Wd\+1,4​\(γd,ℝd\)2,\\displaystyle\\leq CC^\{\\prime\\prime\}\\\|f\\\|\_\{W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}^\{2\},\(296\)where the second to last line applies Hölder’s inequality\. Redefining constants, this implies

supt∈\[0,t0\]‖Ut​f‖Wd\+1,2​\(γd,ℝd\)≤C​‖f‖Wd\+1,4​\(γd,ℝd\)\(∀f∈Cc∞​\(ℝd,ℝd\)\)\\sup\_\{t\\in\[0,t\_\{0\}\]\}\\\|U\_\{t\}f\\\|\_\{W^\{d\+1,2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\\leq C\\\|f\\\|\_\{W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}\\quad\(\\forall f\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\};\{\\mathbb\{R\}\}^\{d\}\)\)\(297\)and so the linear operatorsUt:Wd\+1,4​\(γd,ℝd\)→Wd\+1,2​\(γd,ℝd\)U\_\{t\}:W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\\to W^\{d\+1,2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)are uniformly bounded for small enoughtt\. Furthermore, forf∈Cc∞​\(ℝd,ℝd\)f\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\};\{\\mathbb\{R\}\}^\{d\}\)we knowUt​f→fU\_\{t\}f\\to fast↓0t\\downarrow 0inClocd\+1​\(ℝd,ℝd\)C^\{d\+1\}\_\{\\operatorname\{loc\}\}\(\{\\mathbb\{R\}\}^\{d\};\{\\mathbb\{R\}\}^\{d\}\)by the product rule and \([289](https://arxiv.org/html/2610.10875#A1.E289)\)\. The boundedness allows us to extend this convergence to generalWd\+1,4​\(γd,ℝd\)W^\{d\+1,4\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)maps, which we apply toggto prove \([288](https://arxiv.org/html/2610.10875#A1.E288)\)\.

Let us now fixN,L∈ℕN,L\\in\{\\mathbb\{N\}\}and define the matricesA^n,L:=1Nn,d​Cn,L​Cn,L⊤\\hat\{A\}\_\{n,L\}:=\\frac\{1\}\{\\sqrt\{N\_\{n,d\}\}\}\\sqrt\{C\_\{n,L\}C\_\{n,L\}^\{\\top\}\}and the kernel

kN,L​\(x,y\):=∑n=0N∑ℓ,r=0n\+2​ℓ≥1n\+2​r≥1L\(A^n,L\)ℓ,r​∑i=1Nn,dEn,ℓ,i​\(x\)​En,r,i​\(y\)⊤\.k\_\{N,L\}\(x,y\):=\\sum\_\{n=0\}^\{N\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell,r=0\\\\ n\+2\\ell\\geq 1\\\\ n\+2r\\geq 1\\end\{subarray\}\}^\{L\}\(\\hat\{A\}\_\{n,L\}\)\_\{\\ell,r\}\\sum\_\{i=1\}^\{N\_\{n,d\}\}E\_\{n,\\ell,i\}\(x\)E\_\{n,r,i\}\(y\)^\{\\top\}\.\(298\)This is jointly\-continuous sinceN,LN,Lare finite, it is rotation\-equivariant due to the sum overii, and sinceA^n,L⪰0\\hat\{A\}\_\{n,L\}\\succeq 0it is a positive kernel\. Furthermore, we compute

∫ℝdtr⁡kN,L​\(x,x\)​d​γd​\(x\)=∑n=0NNn,d​tr⁡\(A^n,L\)=∑n=0NNn,d​tr⁡\(Cn,L​Cn,L⊤\)\.\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\_\{N,L\}\(x,x\)d\\gamma\_\{d\}\(x\)=\\sum\_\{n=0\}^\{N\}\{N\_\{n,d\}\}\\operatorname\{tr\}\(\\hat\{A\}\_\{n,L\}\)=\\sum\_\{n=0\}^\{N\}\\sqrt\{N\_\{n,d\}\}\\operatorname\{tr\}\\left\(\\sqrt\{C\_\{n,L\}C\_\{n,L\}^\{\\top\}\}\\right\)\.\(299\)Moreover, if we let

gN,L:=∑n=0N∑ℓ=0n\+2​ℓ≥1L∑i=1Nn,d\(Cn,L\)ℓ,i​En,ℓ,ig\_\{N,L\}:=\\sum\_\{n=0\}^\{N\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{L\}\\sum\_\{i=1\}^\{N\_\{n,d\}\}\(C\_\{n,L\}\)\_\{\\ell,i\}E\_\{n,\\ell,i\}\(300\)be the truncation ofggthen by the same calculations as before we have

‖gN,L‖ℋkN,L2=∑n=0Ntr⁡\(A^n,L†​Cn,L​Cn,L⊤\)=∑n=0NNn,d​tr⁡\(Cn,L​Cn,L⊤\)\.\\\|g\_\{N,L\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{N,L\}\}\}^\{2\}=\\sum\_\{n=0\}^\{N\}\\operatorname\{tr\}\\left\(\\hat\{A\}\_\{n,L\}^\{\\dagger\}C\_\{n,L\}C\_\{n,L\}^\{\\top\}\\right\)=\\sum\_\{n=0\}^\{N\}\\sqrt\{N\_\{n,d\}\}\\operatorname\{tr\}\\left\(\\sqrt\{C\_\{n,L\}C\_\{n,L\}^\{\\top\}\}\\right\)\.\(301\)In particular, we find a sequence\(kN,N\)N∈ℕ⊆𝒦rot\(k\_\{N,N\}\)\_\{N\\in\{\\mathbb\{N\}\}\}\\subseteq\\mathcal\{K\}\_\{\\operatorname\{rot\}\}of jointly\-continuous rotation\-equivariant kernels for which

limN→∞TN⋅‖gN,N‖ℋkN,N=𝒟~​\(w\),TN:=∫ℝdtr⁡kN,N​\(x,x\)​d​γd​\(x\)<∞\.\\lim\_\{N\\to\\infty\}\\sqrt\{T\_\{N\}\}\\cdot\\\|g\_\{N,N\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{N,N\}\}\}=\\widetilde\{\\mathcal\{D\}\}\(w\),\\quad T\_\{N\}:=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\_\{N,N\}\(x,x\)d\\gamma\_\{d\}\(x\)<\\infty\.\(302\)Let us define the jointly\-continuous rotation\-invariant scalar Sobolev kernel

qSob​\(x,y\):=∑α∈\(ℕ∪\{0\}\)d\(1\+\|α\|\)−\(d\+1\)​hα​\(x\)​hα​\(y\),τSob:=d​∫ℝdqSob​\(x,x\)​d​γd​\(x\)<∞q\_\{\\operatorname\{Sob\}\}\(x,y\):=\\sum\_\{\\alpha\\in\(\{\\mathbb\{N\}\}\\cup\\\{0\\\}\)^\{d\}\}\(1\+\|\\alpha\|\)^\{\-\(d\+1\)\}h\_\{\\alpha\}\(x\)h\_\{\\alpha\}\(y\),\\quad\\tau\_\{\\operatorname\{Sob\}\}:=d\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}q\_\{\\operatorname\{Sob\}\}\(x,x\)d\\gamma\_\{d\}\(x\)<\\infty\(303\)with\{hα\}α∈\(ℕ∪\{0\}\)d\\\{h\_\{\\alpha\}\\\}\_\{\\alpha\\in\(\{\\mathbb\{N\}\}\\cup\\\{0\\\}\)^\{d\}\}the normalized multivariate Hermite basis ofL2​\(γd\)L^\{2\}\(\\gamma\_\{d\}\)\. Importantly,𝒯qSob,γd=\(𝟙−Δγ\)−\(d\+1\)\\mathcal\{T\}\_\{q\_\{\\operatorname\{Sob\}\},\\gamma\_\{d\}\}=\(\{\\mathbbm\{1\}\}\-\\Delta\_\{\\gamma\}\)^\{\-\(d\+1\)\}forΔγ=Δ−x⋅∇\\Delta\_\{\\gamma\}=\\Delta\-x\\cdot\\nablathe OU generator and therefore

∥⋅∥ℋqSob≍∥⋅∥Wd\+1,2​\(γd\)\\\|\\cdot\\\|\_\{\\mathcal\{H\}\_\{q\_\{\\operatorname\{Sob\}\}\}\}\\asymp\\\|\\cdot\\\|\_\{W^\{d\+1,2\}\(\\gamma\_\{d\}\)\}\(304\)by the Gaussian Meyer inequalities\. We denotekSob​\(x,y\):=1τSob​qSob​\(x,y\)​𝟙dk\_\{\\operatorname\{Sob\}\}\(x,y\):=\\frac\{1\}\{\\tau\_\{\\operatorname\{Sob\}\}\}q\_\{\\operatorname\{Sob\}\}\(x,y\)\{\\mathbbm\{1\}\}\_\{d\}and

δN:=‖gN,N−g‖ℋkSob2,\\delta\_\{N\}:=\\\|g\_\{N,N\}\-g\\\|\_\{\\mathcal\{H\}\_\{k\_\{\\operatorname\{Sob\}\}\}\}^\{2\},\(305\)and we note thatδN→0\\delta\_\{N\}\\to 0sinceg∈Wd\+1,2​\(γd,ℝd\)g\\in W^\{d\+1,2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)and the Hermite truncation diagonalizesΔγ\\Delta\_\{\\gamma\}\. We define the modified kernel

k~N​\(x,y\):=kN,N​\(x,y\)\+\(δN\+1N\)​kSob​\(x,y\)\\widetilde\{k\}\_\{N\}\(x,y\):=k\_\{N,N\}\(x,y\)\+\\left\(\\sqrt\{\\delta\_\{N\}\}\+\\frac\{1\}\{N\}\\right\)k\_\{\\operatorname\{Sob\}\}\(x,y\)\(306\)and compute that

∫ℝdtr⁡k~N​\(x,x\)​d​γd​\(x\)=TN\+δN\+1N\.\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}\\widetilde\{k\}\_\{N\}\(x,x\)d\\gamma\_\{d\}\(x\)=T\_\{N\}\+\\sqrt\{\\delta\_\{N\}\}\+\\frac\{1\}\{N\}\.\(307\)Furthermore, by the RKHS sum formula we know

‖g‖ℋk~N2\\displaystyle\\\|g\\\|\_\{\\mathcal\{H\}\_\{\\widetilde\{k\}\_\{N\}\}\}^\{2\}≤‖gN,N‖ℋkN,N2\+‖g−gN,N‖ℋ\(δN\+1N\)​kSob2\\displaystyle\\leq\\\|g\_\{N,N\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{N,N\}\}\}^\{2\}\+\\\|g\-g\_\{N,N\}\\\|^\{2\}\_\{\\mathcal\{H\}\_\{\(\\sqrt\{\\delta\_\{N\}\}\+\\frac\{1\}\{N\}\)k\_\{\\operatorname\{Sob\}\}\}\}\(308\)=‖gN,N‖ℋkN,N2\+δNδN\+1N\.\\displaystyle=\\\|g\_\{N,N\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{N,N\}\}\}^\{2\}\+\\frac\{\\delta\_\{N\}\}\{\\sqrt\{\\delta\_\{N\}\}\+\\frac\{1\}\{N\}\}\.\(309\)Putting these together, it holds that

lim supN→∞∫ℝdtr⁡k~N​\(x,x\)​d​γd​\(x\)⋅‖g‖ℋk~N2\\displaystyle\\limsup\_\{N\\to\\infty\}\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}\\widetilde\{k\}\_\{N\}\(x,x\)d\\gamma\_\{d\}\(x\)\}\\cdot\\\|g\\\|\_\{\\mathcal\{H\}\_\{\\widetilde\{k\}\_\{N\}\}\}^\{2\}≤lim supN→∞\(TN\+δN\+1N\)⋅\(‖gN,N‖ℋkN,N2\+δN\)\\displaystyle\\leq\\limsup\_\{N\\to\\infty\}\\left\(T\_\{N\}\+\\sqrt\{\\delta\_\{N\}\}\+\\frac\{1\}\{N\}\\right\)\\cdot\\left\(\\\|g\_\{N,N\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{N,N\}\}\}^\{2\}\+\\sqrt\{\\delta\_\{N\}\}\\right\)\(310\)=lim supN→∞TN​‖gN,N‖ℋkN,N2\\displaystyle=\\limsup\_\{N\\to\\infty\}T\_\{N\}\\\|g\_\{N,N\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{N,N\}\}\}^\{2\}\(311\)=𝒟~​\(w\)2\.\\displaystyle=\\widetilde\{\\mathcal\{D\}\}\(w\)^\{2\}\.\(312\)Lastly, since∥⋅∥ℋk~N≲∥⋅∥ℋkSob≲∥⋅∥Wd\+1,2​\(γd,ℝd\)\\\|\\cdot\\\|\_\{\\mathcal\{H\}\_\{\\widetilde\{k\}\_\{N\}\}\}\\lesssim\\\|\\cdot\\\|\_\{\\mathcal\{H\}\_\{k\_\{\\operatorname\{Sob\}\}\}\}\\lesssim\\\|\\cdot\\\|\_\{W^\{d\+1,2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}for each fixedNNwe knowlimt↓0‖gt−g‖ℋk~N=0\\lim\_\{t\\downarrow 0\}\\\|g\_\{t\}\-g\\\|\_\{\\mathcal\{H\}\_\{\\widetilde\{k\}\_\{N\}\}\}=0by \([288](https://arxiv.org/html/2610.10875#A1.E288)\)\. The upper bound then follows from \([287](https://arxiv.org/html/2610.10875#A1.E287)\)\.

Thus far, we have shown that for general flows driven byw:ℝd→ℝdw:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}we have

Q=𝒟~​\(w\),Q=\\widetilde\{\\mathcal\{D\}\}\(w\),\(313\)where the≥\\geqdirection holds ifwwis globally\-Lipschitz and the≤\\leqdirection also requiresw∈Cd\+2​\(ℝd,ℝd\)w\\in C^\{d\+2\}\(\{\\mathbb\{R\}\}^\{d\};\{\\mathbb\{R\}\}^\{d\}\)with‖D​w‖Cd\+1<∞\\\|Dw\\\|\_\{C^\{d\+1\}\}<\\infty\. Furthermore, by the definition in \([261](https://arxiv.org/html/2610.10875#A1.E261)\) we have that

𝒟~​\(w\):=∑n=0∞Nn,d⋅‖Cn‖𝒮1,\\widetilde\{\\mathcal\{D\}\}\(w\):=\\sum\_\{n=0\}^\{\\infty\}\\sqrt\{N\_\{n,d\}\}\\cdot\\\|C\_\{n\}\\\|\_\{\\mathcal\{S\}\_\{1\}\},\(314\)where∥⋅∥𝒮1\\\|\\cdot\\\|\_\{\\mathcal\{S\}\_\{1\}\}denotes the sum of singular value magnitudes\. To finish up, we specialize to the single\-index casew⁡\(x\)=σ​u​ϕ​\(⟨x,θ⟩\)w\(x\)=\\sigma u\\phi\(\\left\\langle x,\\theta\\right\\rangle\)and denotec:=⟨u,θ⟩c:=\\left\\langle u,\\theta\\right\\rangle\. We takev∈𝕊d−1v\\in\\mathbb\{S\}^\{d\-1\}s\.t\.⟨v,θ⟩=0\\left\\langle v,\\theta\\right\\rangle=0andu=c​θ\+1−c2​vu=c\\theta\+\\sqrt\{1\-c^\{2\}\}v\. This implies that for anyn,ℓ∈ℕ∪\{0\}n,\\ell\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}withm:=n\+2​ℓ≥1m:=n\+2\\ell\\geq 1andi∈\{1,…,Nn,d\}i\\in\\\{1,\\ldots,N\_\{n,d\}\\\}, if we denoteZ:=⟨ξ,θ⟩Z:=\\left\\langle\\xi,\\theta\\right\\rangleforξ∼γd\\xi\\sim\\gamma\_\{d\}ands:=1−c2s:=\\sqrt\{1\-c^\{2\}\}then we have

\(Cn\)ℓ,i\\displaystyle\(C\_\{n\}\)\_\{\\ell,i\}=⟨w,En,ℓ,i⟩L2​\(γd,ℝd\)=σm​𝔼​\[ϕ⁡\(Z\)​⟨u,∇Pn,ℓ,i​\(ξ\)⟩\]\\displaystyle=\\left\\langle w,E\_\{n,\\ell,i\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{d\};\{\\mathbb\{R\}\}^\{d\}\)\}=\\frac\{\\sigma\}\{\\sqrt\{m\}\}\\mathbb\{E\}\\left\[\\phi\(Z\)\\left\\langle u,\\nabla P\_\{n,\\ell,i\}\(\\xi\)\\right\\rangle\\right\]\(315\)=σ​ϕ^m−1m​\(m−1\)\!​𝔼​\[Hem−1⁡\(Z\)​⟨u,∇Pn,ℓ,i​\(ξ\)⟩\],\\displaystyle=\\frac\{\\sigma\\hat\{\\phi\}\_\{m\-1\}\}\{\\sqrt\{m\}\(m\-1\)\!\}\\mathbb\{E\}\\left\[\\operatorname\{He\}\_\{m\-1\}\(Z\)\\left\\langle u,\\nabla P\_\{n,\\ell,i\}\(\\xi\)\\right\\rangle\\right\],\(316\)where in the second line we usedϕ=∑j=0∞ϕ^jj\!​Hej\\phi=\\sum\_\{j=0\}^\{\\infty\}\\frac\{\\hat\{\\phi\}\_\{j\}\}\{j\!\}\\operatorname\{He\}\_\{j\}and thatx↦⟨u,∇Pn,ℓ,i​\(x\)⟩x\\mapsto\\left\\langle u,\\nabla P\_\{n,\\ell,i\}\(x\)\\right\\ranglebelongs to the\(m−1\)t​h\(m\-1\)^\{th\}Hermite chaos\. For anyf∈W1,2​\(γd\)f\\in W^\{1,2\}\(\\gamma\_\{d\}\)we have the Gaussian integration by parts identity

𝔼ξ∼γd​\[Hem−1⁡\(⟨ξ,θ⟩\)​⟨u,∇f​\(ξ\)⟩\]=⟨u,θ⟩​𝒜m​\(θ\)\+1m​⟨u,∇𝕊d−1𝒜m​\(θ\)⟩,\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\\left\[\\operatorname\{He\}\_\{m\-1\}\(\\left\\langle\\xi,\\theta\\right\\rangle\)\\left\\langle u,\\nabla f\(\\xi\)\\right\\rangle\\right\]=\\left\\langle u,\\theta\\right\\rangle\\mathcal\{A\}\_\{m\}\(\\theta\)\+\\frac\{1\}\{m\}\\left\\langle u,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}\\mathcal\{A\}\_\{m\}\(\\theta\)\\right\\rangle,\(317\)where we define𝒜m​\(θ\):=𝔼ξ∼γd​\[Hem⁡\(⟨ξ,θ⟩\)​f​\(ξ\)\]\\mathcal\{A\}\_\{m\}\(\\theta\):=\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\[\\operatorname\{He\}\_\{m\}\(\\left\\langle\\xi,\\theta\\right\\rangle\)f\(\\xi\)\]\. In the casef=Pn,ℓ,if=P\_\{n,\\ell,i\}, there is the identity

1m\!​𝔼ξ∼γd​\[Hem⁡\(⟨ξ,θ⟩\)​Pn,ℓ,i​\(ξ\)\]=βn,ℓ,d​Yn,i​\(θ\),βn,ℓ,d:=m\!​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\.\\frac\{1\}\{\\sqrt\{m\!\}\}\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\[\\operatorname\{He\}\_\{m\}\(\\left\\langle\\xi,\\theta\\right\\rangle\)P\_\{n,\\ell,i\}\(\\xi\)\]=\\beta\_\{n,\\ell,d\}Y\_\{n,i\}\(\\theta\),\\quad\\beta\_\{n,\\ell,d\}:=\\sqrt\{\\frac\{m\!\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\}\.\(318\)To see this, we start from \([258](https://arxiv.org/html/2610.10875#A1.E258)\) and note the Laguerre polynomials have leading coefficientLℓ\(α\)​\(a\)=\(−1\)ℓℓ\!​aℓ\+terms of degree<ℓL\_\{\\ell\}^\{\(\\alpha\)\}\(a\)=\\frac\{\(\-1\)^\{\\ell\}\}\{\\ell\!\}a^\{\\ell\}\+\\text\{terms of degree $<\\ell$\}\. Hence, the degree\-mmhomogeneous part ofPn,ℓ,iP\_\{n,\\ell,i\}, denoted\[Pn,ℓ,i\]m\[P\_\{n,\\ell,i\}\]\_\{m\}, is given by

\[Pn,ℓ,i\]m​\(x\)=12ℓ​ℓ\!​ℓ\!​Γ​\(d/2\)2n​Γ​\(n\+ℓ\+d/2\)⋅‖x‖m​Yn,i​\(x/‖x‖\)\.\[P\_\{n,\\ell,i\}\]\_\{m\}\(x\)=\\frac\{1\}\{2^\{\\ell\}\\ell\!\}\\sqrt\{\\frac\{\\ell\!\\Gamma\(d/2\)\}\{2^\{n\}\\Gamma\(n\+\\ell\+d/2\)\}\}\\cdot\\\|x\\\|^\{m\}Y\_\{n,i\}\(x/\\\|x\\\|\)\.\(319\)Applying Gaussian integration by partsmmtimes yields

𝔼ξ∼γd\[Hem\(⟨ξ,θ⟩\)Pn,ℓ,i\(ξ\)\]=𝔼ξ∼γd\[\(θ⋅∇\)mPn,ℓ,i\(ξ\)\]=m\!2ℓ​ℓ\!ℓ\!​Γ​\(d/2\)2n​Γ​\(n\+ℓ\+d/2\)Yn,i\(θ\)\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\[\\operatorname\{He\}\_\{m\}\(\\left\\langle\\xi,\\theta\\right\\rangle\)P\_\{n,\\ell,i\}\(\\xi\)\]=\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\[\(\\theta\\cdot\\nabla\)^\{m\}P\_\{n,\\ell,i\}\(\\xi\)\]=\\frac\{m\!\}\{2^\{\\ell\}\\ell\!\}\\sqrt\{\\frac\{\\ell\!\\Gamma\(d/2\)\}\{2^\{n\}\\Gamma\(n\+\\ell\+d/2\)\}\}Y\_\{n,i\}\(\\theta\)\(320\)which implies the claimed identity\. We find that

\(Cn\)ℓ,i\\displaystyle\(C\_\{n\}\)\_\{\\ell,i\}=σ​ϕ^m−1​m\!​βn,ℓ,dm​\(m−1\)\!​\(⟨u,θ⟩​Yn,i​\(θ\)\+1m​⟨u,∇𝕊d−1Yn,i​\(θ\)⟩\)\\displaystyle=\\frac\{\\sigma\\hat\{\\phi\}\_\{m\-1\}\\sqrt\{m\!\}\\beta\_\{n,\\ell,d\}\}\{\\sqrt\{m\}\(m\-1\)\!\}\\left\(\\left\\langle u,\\theta\\right\\rangle Y\_\{n,i\}\(\\theta\)\+\\frac\{1\}\{m\}\\left\\langle u,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\_\{n,i\}\(\\theta\)\\right\\rangle\\right\)\(321\)=σ​ϕ^m−1​m\!​βn,ℓ,dm​\(m−1\)\!​\(c​Yn,i​\(θ\)\+sm​⟨v,∇𝕊d−1Yn,i​\(θ\)⟩\),\\displaystyle=\\frac\{\\sigma\\hat\{\\phi\}\_\{m\-1\}\\sqrt\{m\!\}\\beta\_\{n,\\ell,d\}\}\{\\sqrt\{m\}\(m\-1\)\!\}\\left\(cY\_\{n,i\}\(\\theta\)\+\\frac\{s\}\{m\}\\left\\langle v,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\_\{n,i\}\(\\theta\)\\right\\rangle\\right\),\(322\)where we used that∇𝕊d−1Yn,i​\(θ\)\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\_\{n,i\}\(\\theta\)is tangent to the sphere atθ\\thetaand therefore orthogonal toθ\\theta\. By differentiating the addition theorem for spherical harmonics \(see\[[6](https://arxiv.org/html/2610.10875#bib.bib62), Theorem 2\.9\]\), we have

∑i=1Nn,dYn,i​\(θ\)2=Nn,d,∑i=1Nn,dYn,i​\(θ\)​⟨v,∇𝕊d−1Yn,i​\(θ\)⟩=0,\\displaystyle\\sum\_\{i=1\}^\{N\_\{n,d\}\}Y\_\{n,i\}\(\\theta\)^\{2\}=N\_\{n,d\},\\quad\\sum\_\{i=1\}^\{N\_\{n,d\}\}Y\_\{n,i\}\(\\theta\)\\left\\langle v,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\_\{n,i\}\(\\theta\)\\right\\rangle=0,\(323\)∑i=1Nn,d⟨v,∇𝕊d−1Yn,i​\(θ\)⟩2=Nn,d​n⁡\(n\+d−2\)d−1\.\\displaystyle\\sum\_\{i=1\}^\{N\_\{n,d\}\}\\left\\langle v,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Y\_\{n,i\}\(\\theta\)\\right\\rangle^\{2\}=N\_\{n,d\}\\frac\{n\(n\+d\-2\)\}\{d\-1\}\.\(324\)In particular, for fixedn,ℓn,\\ellwe find that

∑i=1Nn,d\(Cn\)ℓ,i2=σ2​\|ϕ^m−1\|2​βn,ℓ,d2​Nn,d\(m−1\)\!​\(c2\+s2m2​n⁡\(n\+d−2\)d−1\)\\sum\_\{i=1\}^\{N\_\{n,d\}\}\(C\_\{n\}\)\_\{\\ell,i\}^\{2\}=\\frac\{\\sigma^\{2\}\|\\hat\{\\phi\}\_\{m\-1\}\|^\{2\}\\beta\_\{n,\\ell,d\}^\{2\}N\_\{n,d\}\}\{\(m\-1\)\!\}\\left\(c^\{2\}\+\\frac\{s^\{2\}\}\{m^\{2\}\}\\frac\{n\(n\+d\-2\)\}\{d\-1\}\\right\)\(325\)By comparing with \([255](https://arxiv.org/html/2610.10875#A1.E255)\), we find that

σ​𝒟ϕ​\(⟨u,θ⟩\)=∑n=0∞Nn,d⋅‖Cn‖F\.\\sigma\\mathcal\{D\}\_\{\\phi\}\(\\left\\langle u,\\theta\\right\\rangle\)=\\sum\_\{n=0\}^\{\\infty\}\\sqrt\{N\_\{n,d\}\}\\cdot\\\|C\_\{n\}\\\|\_\{F\}\.\(326\)The final claim then follows from \([314](https://arxiv.org/html/2610.10875#A1.E314)\) and the fact that‖Cn‖F≤‖Cn‖𝒮1≤2​‖Cn‖F\\\|C\_\{n\}\\\|\_\{F\}\\leq\\\|C\_\{n\}\\\|\_\{\\mathcal\{S\}\_\{1\}\}\\leq\\sqrt\{2\}\\\|C\_\{n\}\\\|\_\{F\}since eachCnC\_\{n\}has rank at most22, see \([322](https://arxiv.org/html/2610.10875#A1.E322)\)\. ∎

Lastly, we show the following perturbative lower bound on the Barron transport metric with Gaussian source:

###### Lemma 11\(First\-orderDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}lower bound\)\.

Letϕ∈C1​\(ℝ\)\\phi\\in C^\{1\}\(\{\\mathbb\{R\}\}\)satisfy Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)withM:=max⁡\{‖ϕ‖∞,‖ϕ′‖∞\}∈\(0,∞\)M:=\\max\\\{\\\|\\phi\\\|\_\{\\infty\},\\\|\\phi^\{\\prime\}\\\|\_\{\\infty\}\\\}\\in\(0,\\infty\)\. Fixd≥2d\\geq 2andu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}, and for eachε\>0\\varepsilon\>0defineνε:=Law⁡\(X1\)\\nu\_\{\\varepsilon\}:=\\operatorname\{Law\}\(X\_\{1\}\)according to the flow

X˙t=ε​w​\(Xt\),w⁡\(x\):=u​ϕ​\(⟨x,θ⟩\),X0∼γd\.\\dot\{X\}\_\{t\}=\\varepsilon w\(X\_\{t\}\),\\quad w\(x\):=u\\phi\(\\left\\langle x,\\theta\\right\\rangle\),\\quad X\_\{0\}\\sim\\gamma\_\{d\}\.\(327\)Then, withc:=⟨u,θ⟩c:=\\left\\langle u,\\theta\\right\\ranglewe have that

lim infε↓0Dℬϕ​\(γd,νε\)ε≥1M​supa,b∈ℝ\|\(a​1−c2\+b​c\)​ϕ​\(b\)−c​ϕ′​\(b\)\|1\+a2\+b2\.\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\frac\{1\}\{M\}\\sup\_\{a,b\\in\{\\mathbb\{R\}\}\}\\frac\{\\left\|\(a\\sqrt\{1\-c^\{2\}\}\+bc\)\\phi\(b\)\-c\\phi^\{\\prime\}\(b\)\\right\|\}\{1\+\\sqrt\{a^\{2\}\+b^\{2\}\}\}\.\(328\)Ifϕ∈\{tanh,sin\}\\phi\\in\\\{\\tanh,\\sin\\\}then the right hand side is≥1\\geq 1for allc∈\[−1,1\]c\\in\[\-1,1\]\.

###### Proof\.

Note that since‖ϕ‖∞,‖ϕ′‖∞≤M\\\|\\phi\\\|\_\{\\infty\},\\\|\\phi^\{\\prime\}\\\|\_\{\\infty\}\\leq Mwe know

‖u‖∞,‖div⁡u‖∞≤M​‖u‖ℬϕ\(∀u∈ℬϕ\)\.\\\|u\\\|\_\{\\infty\},\\\|\\operatorname\{div\}u\\\|\_\{\\infty\}\\leq M\\\|u\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\quad\(\\forall u\\in\\mathcal\{B\}\_\{\\phi\}\)\.\(329\)Now, let\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}be an admissible transport fromγd\\gamma\_\{d\}toνε\\nu\_\{\\varepsilon\}with

L:=∫01‖vt‖ℬϕ​𝑑t<∞\.L:=\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt<\\infty\.\(330\)The prescribed flow has aC1C^\{1\}bi\-Lipschitz endpoint map \(sinceϕ\\phiisC1C^\{1\}and Lipschitz\), soνε\\nu\_\{\\varepsilon\}has a continuous positive density\. Now, if we take\(Ytx\)t∈\[0,1\]\(Y\_\{t\}^\{x\}\)\_\{t\\in\[0,1\]\}to be a characteristic ending at someY1x=x∈ℝdY\_\{1\}^\{x\}=x\\in\{\\mathbb\{R\}\}^\{d\}then there is the change of variables formula

log⁡νεγd​\(x\)=−‖Y0x‖2−‖x‖22−∫01div⁡vt​\(Ytx\)​𝑑t;\\log\\frac\{\\nu\_\{\\varepsilon\}\}\{\\gamma\_\{d\}\}\(x\)=\-\\frac\{\\\|Y\_\{0\}^\{x\}\\\|^\{2\}\-\\\|x\\\|^\{2\}\}\{2\}\-\\int\_\{0\}^\{1\}\\operatorname\{div\}v\_\{t\}\(Y\_\{t\}^\{x\}\)dt;\(331\)at first this holds for a\.e\.xxby Lipschitz regularity, which we extend by continuity to hold for everyxx\(note that eachdiv⁡vt\\operatorname\{div\}v\_\{t\}is continuous sinceϕ\\phiisC1C^\{1\}\)\. Since‖Y0x−x‖≤∫01‖vt‖∞​𝑑t≤M​L\\\|Y\_\{0\}^\{x\}\-x\\\|\\leq\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\infty\}dt\\leq MLwe know\|‖Y0x‖2−‖x‖22\|≤M2​L22\+M​L​‖x‖\\left\|\\frac\{\\\|Y\_\{0\}^\{x\}\\\|^\{2\}\-\\\|x\\\|^\{2\}\}\{2\}\\right\|\\leq\\frac\{M^\{2\}L^\{2\}\}\{2\}\+ML\\\|x\\\|, and so

\|log⁡νεγd​\(x\)\|≤M2​L22\+M​L​\(1\+‖x‖\)\.\\left\|\\log\\frac\{\\nu\_\{\\varepsilon\}\}\{\\gamma\_\{d\}\}\(x\)\\right\|\\leq\\frac\{M^\{2\}L^\{2\}\}\{2\}\+ML\(1\+\\\|x\\\|\)\.\(332\)However, for the flowX˙t=ε​w​\(Xt\)\\dot\{X\}\_\{t\}=\\varepsilon w\(X\_\{t\}\)which generatesνε\\nu\_\{\\varepsilon\}we know thatX1=X0\+ε​w​\(X0\)\+o⁡\(ε\)X\_\{1\}=X\_\{0\}\+\\varepsilon w\(X\_\{0\}\)\+o\(\\varepsilon\)and soνε=γd−ε​div⁡\(γd​w\)\+o⁡\(ε\)\\nu\_\{\\varepsilon\}=\\gamma\_\{d\}\-\\varepsilon\\operatorname\{div\}\(\\gamma\_\{d\}w\)\+o\(\\varepsilon\)\. In particular, usinglog⁡\(1\+a\)=a\+o⁡\(a\)\\log\(1\+a\)=a\+o\(a\)we find that the log density must satisfy

log⁡νεγd​\(x\)\\displaystyle\\log\\frac\{\\nu\_\{\\varepsilon\}\}\{\\gamma\_\{d\}\}\(x\)=ε⁡\(⟨x,w⁡\(x\)⟩−div⁡w⁡\(x\)\)\+ox​\(ε\)\\displaystyle=\\varepsilon\\left\(\\left\\langle x,w\(x\)\\right\\rangle\-\\operatorname\{div\}w\(x\)\\right\)\+o\_\{x\}\(\\varepsilon\)\(333\)=ε⁡\(⟨x,u⟩​ϕ​\(⟨x,θ⟩\)−⟨u,θ⟩​ϕ′​\(⟨x,θ⟩\)\)\+ox​\(ε\)\.\\displaystyle=\\varepsilon\\left\(\\left\\langle x,u\\right\\rangle\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\-\\left\\langle u,\\theta\\right\\rangle\\phi^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\\right\)\+o\_\{x\}\(\\varepsilon\)\.\(334\)Combining with \([332](https://arxiv.org/html/2610.10875#A1.E332)\), we find that

ε​\|⟨x,u⟩​ϕ​\(⟨x,θ⟩\)−⟨u,θ⟩​ϕ′​\(⟨x,θ⟩\)\|\+ox​\(ε\)≤M2​L22\+M​L​\(1\+‖x‖\)\\varepsilon\\left\|\\left\\langle x,u\\right\\rangle\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\-\\left\\langle u,\\theta\\right\\rangle\\phi^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\\right\|\+o\_\{x\}\(\\varepsilon\)\\leq\\frac\{M^\{2\}L^\{2\}\}\{2\}\+ML\(1\+\\\|x\\\|\)\(335\)and therefore

L≥\(1\+‖x‖\)2\+2​ε​\|⟨x,u⟩​ϕ​\(⟨x,θ⟩\)−⟨u,θ⟩​ϕ′​\(⟨x,θ⟩\)\|\+ox​\(ε\)−\(1\+‖x‖\)ML\\geq\\frac\{\\sqrt\{\(1\+\\\|x\\\|\)^\{2\}\+2\\varepsilon\\left\|\\left\\langle x,u\\right\\rangle\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\-\\left\\langle u,\\theta\\right\\rangle\\phi^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\\right\|\+o\_\{x\}\(\\varepsilon\)\}\-\(1\+\\\|x\\\|\)\}\{M\}\(336\)Taking an infimum over admissible paths followed byε↓0\\varepsilon\\downarrow 0withxxfixed implies that

lim infε↓0Dℬϕ​\(γd,νε\)ε≥\|⟨x,u⟩​ϕ​\(⟨x,θ⟩\)−⟨u,θ⟩​ϕ′​\(⟨x,θ⟩\)\|M⁡\(1\+‖x‖\)\.\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\geq\\frac\{\\left\|\\left\\langle x,u\\right\\rangle\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\-\\left\\langle u,\\theta\\right\\rangle\\phi^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\\right\|\}\{M\(1\+\\\|x\\\|\)\}\.\(337\)This holds for anyx∈ℝdx\\in\{\\mathbb\{R\}\}^\{d\}, and so we may take a supremum overxx\. It suffices to considerx∈span⁡\{u,θ\}x\\in\\operatorname\{span\}\\\{u,\\theta\\\}since otherwise the denominator is strictly suboptimal, and so we writec=⟨u,θ⟩c=\\left\\langle u,\\theta\\right\\rangle,u=c​θ\+1−c2​vu=c\\theta\+\\sqrt\{1\-c^\{2\}\}vfor a unit vectorvvorthogonal toθ\\theta, and we expressx=a​v\+b​θx=av\+b\\theta\. In this case the first claim follows due to

\|⟨x,u⟩​ϕ​\(⟨x,θ⟩\)−⟨u,θ⟩​ϕ′​\(⟨x,θ⟩\)\|1\+‖x‖=\|\(a​1−c2\+b​c\)​ϕ​\(b\)−c​ϕ′​\(b\)\|1\+a2\+b2\\frac\{\\left\|\\left\\langle x,u\\right\\rangle\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\-\\left\\langle u,\\theta\\right\\rangle\\phi^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\\right\|\}\{1\+\\\|x\\\|\}=\\frac\{\\left\|\(a\\sqrt\{1\-c^\{2\}\}\+bc\)\\phi\(b\)\-c\\phi^\{\\prime\}\(b\)\\right\|\}\{1\+\\sqrt\{a^\{2\}\+b^\{2\}\}\}\(338\)For the second claim, note first that ifϕ∈\{tanh,sin\}\\phi\\in\\\{\\tanh,\\sin\\\}thenM=1M=1\. Now, ifc=0c=0then the expression becomes\|a​ϕ​\(b\)\|1\+a2\+b2\\frac\{\|a\\phi\(b\)\|\}\{1\+\\sqrt\{a^\{2\}\+b^\{2\}\}\}, which goes to\|ϕ⁡\(b\)\|\|\\phi\(b\)\|as we takea→∞a\\to\\infty\. Hence, the right hand side equalssupb\|ϕ⁡\(b\)\|=1\\sup\_\{b\}\|\\phi\(b\)\|=1\. If insteadc≠0c\\neq 0then we takex=R​1−c2​v\+R​c​θ=R​ux=R\\sqrt\{1\-c^\{2\}\}v\+Rc\\theta=RuforR\>0R\>0to make the expression equal

\|R​ϕ​\(R​c\)−c​ϕ′​\(R​c\)\|1\+R\.\\frac\{\|R\\phi\(Rc\)\-c\\phi^\{\\prime\}\(Rc\)\|\}\{1\+R\}\.\(339\)Fortanh\\tanhwe letR→∞R\\to\\inftyto find that\|tanh⁡\(R​c\)\|→1\|\\tanh\(Rc\)\|\\to 1whiletanh′⁡\(R​c\)=sech⁡\(R​c\)2→0\\tanh^\{\\prime\}\(Rc\)=\\operatorname\{sech\}\(Rc\)^\{2\}\\to 0\. Forsin\\sinwe takeRj=π/2\+2​π​j\|c\|R\_\{j\}=\\frac\{\\pi/2\+2\\pi j\}\{\|c\|\}so that\|sin⁡\(Rj​c\)\|=1\|\\sin\(R\_\{j\}c\)\|=1whilecos⁡\(Rj​c\)=0\\cos\(R\_\{j\}c\)=0\. In either case, \([339](https://arxiv.org/html/2610.10875#A1.E339)\) tends to 1 and so the result follows\. ∎

### A\.5Feature learning comparisons

In this section we derive the results we present in the main paper regarding comparison ofDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}and rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}\. We first apply the spherical harmonic lower bound of Lemma[9](https://arxiv.org/html/2610.10875#Thmlemma9)to control the case of single\-index flows with ReLU activation:

###### Proof of Example[2](https://arxiv.org/html/2610.10875#Thmexample2)\.

Letw⁡\(x\):=q​ϕ​\(⟨x,u⟩\)w\(x\):=q\\phi\(\\left\\langle x,u\\right\\rangle\)withq,u∈𝕊d−1q,u\\in\\mathbb\{S\}^\{d\-1\}s\.t\.c:=⟨q,u⟩≠0c:=\\left\\langle q,u\\right\\rangle\\neq 0\(we relabel the input and output features for notational simplicity\)\. We start from the ReLU\-specific observation that𝟙\+ε​w\{\\mathbbm\{1\}\}\+\\varepsilon wis the time\-1 map of the flow

X˙t=log⁡\(1\+c​ε\)c​w​\(Xt\),\\dot\{X\}\_\{t\}=\\frac\{\\log\(1\+c\\varepsilon\)\}\{c\}w\(X\_\{t\}\),\(340\)where1\+c​ε\>01\+c\\varepsilon\>0for small enoughε\\varepsilon\. To see this, note that the induced dynamics ofat:=⟨Xt,u⟩a\_\{t\}:=\\left\\langle X\_\{t\},u\\right\\rangleare

a˙t=log⁡\(1\+c​ε\)​max​\{0,at\}\\dot\{a\}\_\{t\}=\{\\log\(1\+c\\varepsilon\)\}\\max\\\{0,a\_\{t\}\\\}\(341\)and thereforeat=elog⁡\(1\+c​ε\)​t​a0a\_\{t\}=e^\{\\log\(1\+c\\varepsilon\)t\}a\_\{0\}ifa0\>0a\_\{0\}\>0andat=a0a\_\{t\}=a\_\{0\}ifa0≤0a\_\{0\}\\leq 0\. Hence,

X1=X0\+1\+c​ε−1c​max⁡\{0,a0\}​q=\(𝟙\+ε​w\)​\(X0\)X\_\{1\}=X\_\{0\}\+\\frac\{1\+c\\varepsilon\-1\}\{c\}\\max\\\{0,a\_\{0\}\\\}q=\(\{\\mathbbm\{1\}\}\+\\varepsilon w\)\(X\_\{0\}\)\(342\)and so \([340](https://arxiv.org/html/2610.10875#A1.E340)\) is verified\. Since‖w‖ℬϕ≤1\\\|w\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq 1by construction \(see Definition[4](https://arxiv.org/html/2610.10875#Thmdefinition4)\), andlog⁡\(1\+c​ε\)c​ε→1\\frac\{\\log\(1\+c\\varepsilon\)\}\{c\\varepsilon\}\\to 1, we get the Barron upper bound

lim supε↓0Dℬϕ​\(μ,νε\)ε≤1\.\\limsup\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\mu,\\nu\_\{\\varepsilon\}\)\}\{\\varepsilon\}\\leq 1\.\(343\)
For the Stein lower bound, we will show for oddn≥dn\\geq dthatAn≳n−1A\_\{n\}\\gtrsim n^\{\-1\}\(which is≳n1−d/2\\gtrsim n^\{1\-d/2\}wheneverd≥4d\\geq 4\), after which the result will follow from Lemma[9](https://arxiv.org/html/2610.10875#Thmlemma9)\. To this end, letPn,d:\[−1,1\]→ℝP\_\{n,d\}:\[\-1,1\]\\to\{\\mathbb\{R\}\}be the Legendre polynomials defined in \(2\.19\) of\[[6](https://arxiv.org/html/2610.10875#bib.bib62)\]\. By the addition theorem for spherical harmonics \(see\[[6](https://arxiv.org/html/2610.10875#bib.bib62), Theorem 2\.9\]\), if\{Yn,j:1≤j≤Nn,d\}\\\{Y\_\{n,j\}:\\;1\\leq j\\leq N\_\{n,d\}\\\}is an orthonormal basis of𝕐nd\\mathbb\{Y\}\_\{n\}^\{d\}w\.r\.t\. the normalized uniform probability measureσ\\sigmaon𝕊d−1\\mathbb\{S\}^\{d\-1\}then

∑j=1Nn,dYn,j​\(ξ\)​Yn,j​\(η\)=Nn,d​Pn,d​\(⟨ξ,η⟩\)\(∀ξ,η∈𝕊d−1\)\.\\sum\_\{j=1\}^\{N\_\{n,d\}\}Y\_\{n,j\}\(\\xi\)\{Y\_\{n,j\}\(\\eta\)\}=N\_\{n,d\}P\_\{n,d\}\(\\left\\langle\\xi,\\eta\\right\\rangle\)\\quad\(\\forall\\xi,\\eta\\in\\mathbb\{S\}^\{d\-1\}\)\.\(344\)Let us take our input featureu∈𝕊d−1u\\in\\mathbb\{S\}^\{d\-1\}and defineZn,u:𝕊d−1→ℝZ\_\{n,u\}:\\mathbb\{S\}^\{d\-1\}\\to\{\\mathbb\{R\}\}by

Zn,u​\(θ\):=Nn,d​Pn,d​\(⟨θ,u⟩\),Z\_\{n,u\}\(\\theta\):=\\sqrt\{N\_\{n,d\}\}P\_\{n,d\}\(\\left\\langle\\theta,u\\right\\rangle\),\(345\)and soZn,u∈𝕐ndZ\_\{n,u\}\\in\\mathbb\{Y\}\_\{n\}^\{d\}\. By equation \(2\.40\) of\[[6](https://arxiv.org/html/2610.10875#bib.bib62)\]we know‖Zn,u‖L2​\(σ\)=1\\\|Z\_\{n,u\}\\\|\_\{L^\{2\}\(\\sigma\)\}=1\. We will takeY=Zn,uY=Z\_\{n,u\}as the comparator to lower boundAnA\_\{n\}\.

We claim that ifϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}is positively one\-homogeneous then withw⁡\(x\)=q​ϕ​\(⟨u,x⟩\)w\(x\)=q\\phi\(\\left\\langle u,x\\right\\rangle\)andφn​\(x\)=\|x\|2​Zn,u​\(x\|x\|\)\\varphi\_\{n\}\(x\)=\|x\|^\{2\}Z\_\{n,u\}\(\\frac\{x\}\{\|x\|\}\)we have

∫ℝd⟨∇φn,w⟩​𝑑μ=c​M2​\(μ\)2​\(2\+n⁡\(n\+d−2\)2\)⋅βn​\(ϕ\),\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\_\{n\},w\\right\\rangle d\\mu=cM\_\{2\}\(\\mu\)^\{2\}\\left\(2\+\\frac\{n\(n\+d\-2\)\}\{2\}\\right\)\\cdot\\beta\_\{n\}\(\\phi\),\(346\)where we denoteM2​\(μ\):=∫ℝd‖x‖2​𝑑μ​\(x\)M\_\{2\}\(\\mu\):=\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|x\\\|^\{2\}d\\mu\(x\)\}and we define the coefficient ofϕ\\phiby

βn\(ϕ\):=\(ϕ\(1\)\+\(−1\)n\+1ϕ\(−1\)\)∫\{⟨θ,u⟩\>0\}⟨θ,u⟩2Zn,u\(θ\)dσ\(θ\)\.\\beta\_\{n\}\(\\phi\):=\\left\(\\phi\(1\)\+\(\-1\)^\{n\+1\}\\phi\(\-1\)\\right\)\\int\_\{\\\{\\left\\langle\\theta,u\\right\\rangle\>0\\\}\}\\left\\langle\\theta,u\\right\\rangle^\{2\}Z\_\{n,u\}\(\\theta\)d\\sigma\(\\theta\)\.\(347\)To see this, we start from

∇φn​\(r​θ\)=2​r​Zn,u​\(θ\)​θ\+r2r​∇𝕊d−1Zn,u​\(θ\),\\nabla\\varphi\_\{n\}\(r\\theta\)=2rZ\_\{n,u\}\(\\theta\)\\theta\+\\frac\{r^\{2\}\}\{r\}\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\(\\theta\),\(348\)after which we apply positive homogeneity to find

∫ℝd⟨∇φn,w⟩​𝑑μ\\displaystyle\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\_\{n\},w\\right\\rangle d\\mu=𝔼R,Θ​\[2​R2​Zn,u​\(Θ\)​ϕ​\(⟨Θ,u⟩\)​⟨Θ,q⟩\+R2​ϕ​\(⟨Θ,u⟩\)​⟨q,∇𝕊d−1Zn,u​\(Θ\)⟩\]\\displaystyle=\\mathbb\{E\}\_\{R,\\Theta\}\\left\[2R^\{2\}Z\_\{n,u\}\(\\Theta\)\\phi\(\\left\\langle\\Theta,u\\right\\rangle\)\\left\\langle\\Theta,q\\right\\rangle\+R^\{2\}\\phi\(\\left\\langle\\Theta,u\\right\\rangle\)\\left\\langle q,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\(\\Theta\)\\right\\rangle\\right\]\(349\)=c​M2​\(μ\)2​\[2​∫𝕊d−1t​ϕ​\(t\)​Zn,u​𝑑σ\+∫𝕊d−1ϕ⁡\(t\)​⟨u,∇𝕊d−1Zn,u⟩​𝑑σ\],\\displaystyle=cM\_\{2\}\(\\mu\)^\{2\}\\left\[2\\int\_\{\\mathbb\{S\}^\{d\-1\}\}t\\phi\(t\)Z\_\{n,u\}d\\sigma\+\\int\_\{\\mathbb\{S\}^\{d\-1\}\}\\phi\(t\)\\left\\langle u,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\\right\\rangle d\\sigma\\right\],\(350\)where in the second line we denotet⁡\(Θ\):=⟨Θ,u⟩t\(\\Theta\):=\\left\\langle\\Theta,u\\right\\rangleand apply independence ofR,ΘR,\\Thetaand symmetry aboutuu\(which replacesqqwithc​ucu\)\. Now, since∇𝕊d−1t​\(θ\)=u−t⁡\(θ\)​θ\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}t\(\\theta\)=u\-t\(\\theta\)\\thetaand∇𝕊d−1Zn,u​\(θ\)\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\(\\theta\)is orthogonal toθ\\thetawe find that

⟨u,∇𝕊d−1Zn,u⟩=⟨∇𝕊d−1t,∇𝕊d−1Zn,u⟩\.\\left\\langle u,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\\right\\rangle=\\left\\langle\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}t,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\\right\\rangle\.\(351\)We now use that\(t​ϕ​\(t\)\)′=2​ϕ​\(t\)\(t\\phi\(t\)\)^\{\\prime\}=2\\phi\(t\)a\.e\. for positively one\-homogeneous functions to get

∫𝕊d−1ϕ⁡\(t\)​⟨u,∇𝕊d−1Zn,u⟩​𝑑σ\\displaystyle\\int\_\{\\mathbb\{S\}^\{d\-1\}\}\\phi\(t\)\\left\\langle u,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\\right\\rangle d\\sigma=∫𝕊d−1\(t​ϕ​\(t\)\)′2​⟨∇𝕊d−1t,∇𝕊d−1Zn,u⟩​𝑑σ\\displaystyle=\\int\_\{\\mathbb\{S\}^\{d\-1\}\}\\frac\{\(t\\phi\(t\)\)^\{\\prime\}\}\{2\}\\left\\langle\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}t,\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\\right\\rangle d\\sigma\(352\)=12​∫𝕊d−1⟨∇𝕊d−1\(t​ϕ​\(t\)\),∇𝕊d−1Zn,u⟩​𝑑σ\\displaystyle=\\frac\{1\}\{2\}\\int\_\{\\mathbb\{S\}^\{d\-1\}\}\\left\\langle\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}\(t\\phi\(t\)\),\\nabla\_\{\\mathbb\{S\}^\{d\-1\}\}Z\_\{n,u\}\\right\\rangle d\\sigma\(353\)=n⁡\(n\+d−2\)2​∫𝕊d−1t​ϕ​\(t\)​Zn,u​𝑑σ,\\displaystyle=\\frac\{n\(n\+d\-2\)\}\{2\}\\int\_\{\\mathbb\{S\}^\{d\-1\}\}t\\phi\(t\)Z\_\{n,u\}d\\sigma,\(354\)where the second line uses the chain rule and the third uses equation \(3\.20\) of\[[6](https://arxiv.org/html/2610.10875#bib.bib62)\]\. Thus,

∫ℝd⟨∇φn,w⟩​𝑑μ=c​M2​\(μ\)2​\(2\+n⁡\(n\+d−2\)2\)​∫𝕊d−1t​ϕ​\(t\)​Zn,u​𝑑σ\.\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\_\{n\},w\\right\\rangle d\\mu=cM\_\{2\}\(\\mu\)^\{2\}\\left\(2\+\\frac\{n\(n\+d\-2\)\}\{2\}\\right\)\\int\_\{\\mathbb\{S\}^\{d\-1\}\}t\\phi\(t\)Z\_\{n,u\}d\\sigma\.\(355\)Now, if we lets\+:=ϕ⁡\(1\)s\_\{\+\}:=\\phi\(1\)ands−:=−ϕ⁡\(−1\)s\_\{\-\}:=\-\\phi\(\-1\)be the slopes then

t​ϕ​\(t\)=\{s\+​t2,t\>0,s−​t2,t<0,t\\phi\(t\)=\\begin\{cases\}s\_\{\+\}t^\{2\},&t\>0,\\\\ s\_\{\-\}t^\{2\},&t<0,\\end\{cases\}\(356\)which we combine withZn,u​\(−θ\)=\(−1\)n​Zn,u​\(θ\)Z\_\{n,u\}\(\-\\theta\)=\(\-1\)^\{n\}Z\_\{n,u\}\(\\theta\)to get that

∫𝕊d−1t​ϕ​\(t\)​Zn,u​𝑑σ\\displaystyle\\int\_\{\\mathbb\{S\}^\{d\-1\}\}t\\phi\(t\)Z\_\{n,u\}d\\sigma=s\+∫\{t\>0\}t2Zn,udσ\+s−∫\{t<0\}t2Zn,udσ\\displaystyle=s\_\{\+\}\\int\_\{\\\{t\>0\\\}\}t^\{2\}Z\_\{n,u\}d\\sigma\+s\_\{\-\}\\int\_\{\\\{t<0\\\}\}t^\{2\}Z\_\{n,u\}d\\sigma\(357\)=\(s\+\+\(−1\)ns−\)∫\{t\>0\}t2Zn,udσ\.\\displaystyle=\(s\_\{\+\}\+\(\-1\)^\{n\}s\_\{\-\}\)\\int\_\{\\\{t\>0\\\}\}t^\{2\}Z\_\{n,u\}d\\sigma\.\(358\)Thus, the equality in \([346](https://arxiv.org/html/2610.10875#A1.E346)\) is proved and it remains to lower bound the integral in \([347](https://arxiv.org/html/2610.10875#A1.E347)\)\.

To this end, we claim that ifn≥d≥3n\\geq d\\geq 3andnnis odd then

\|∫\{⟨θ,u⟩\>0\}⟨θ,u⟩2Zn,u\(θ\)dσ\(θ\)\|≥d1/4​n−36\.\\left\|\\int\_\{\\\{\\left\\langle\\theta,u\\right\\rangle\>0\\\}\}\\left\\langle\\theta,u\\right\\rangle^\{2\}Z\_\{n,u\}\(\\theta\)d\\sigma\(\\theta\)\\right\|\\geq\\frac\{d^\{1/4\}n^\{\-3\}\}\{6\}\.\(359\)Ford≥3d\\geq 3andn≥3n\\geq 3withnnodd we have

\|∫\{⟨θ,u⟩\>0\}⟨θ,u⟩2Zn,u\(θ\)dσ\(θ\)\|\\displaystyle\\left\|\\int\_\{\\\{\\left\\langle\\theta,u\\right\\rangle\>0\\\}\}\\left\\langle\\theta,u\\right\\rangle^\{2\}Z\_\{n,u\}\(\\theta\)d\\sigma\(\\theta\)\\right\|=Γ⁡\(d/2\)4​π​\(2​n\+d−2\)​Γ​\(n\+d−2\)Γ⁡\(n\+1\)​Γ​\(d−1\)​Γ⁡\(\(n−2\)/2\)Γ⁡\(\(n\+d\+2\)/2\)\\displaystyle=\\frac\{\\Gamma\(d/2\)\}\{4\\pi\}\\sqrt\{\\frac\{\(2n\+d\-2\)\\Gamma\(n\+d\-2\)\}\{\\Gamma\(n\+1\)\\Gamma\(d\-1\)\}\}\\frac\{\\Gamma\(\(n\-2\)/2\)\}\{\\Gamma\(\(n\+d\+2\)/2\)\}\(360\)=π−3/44​2​2​n\+d−2​\(Γ⁡\(d/2\)Γ⁡\(\(d−1\)/2\)\)1/2⋅Γ⁡\(\(n−2\)/2\)Γ⁡\(\(n\+1\)/2\)​Γ​\(\(n\+2\)/2\)\\displaystyle=\\frac\{\\pi^\{\-3/4\}\}\{4\\sqrt\{2\}\}\\sqrt\{2n\+d\-2\}\\left\(\\frac\{\\Gamma\(d/2\)\}\{\\Gamma\(\(d\-1\)/2\)\}\\right\)^\{1/2\}\\cdot\\frac\{\\Gamma\(\(n\-2\)/2\)\}\{\\sqrt\{\\Gamma\(\(n\+1\)/2\)\\Gamma\(\(n\+2\)/2\)\}\}\(361\)⋅Γ⁡\(\(n\+d−2\)/2\)​Γ​\(\(n\+d−1\)/2\)Γ⁡\(\(n\+d\+2\)/2\)\\displaystyle\\qquad\\cdot\\frac\{\\sqrt\{\\Gamma\(\(n\+d\-2\)/2\)\\Gamma\(\(n\+d\-1\)/2\)\}\}\{\\Gamma\(\(n\+d\+2\)/2\)\}\(362\)≥π−3/44​22​n\+d−2⋅2−3/4d1/4⋅27/4n7/4⋅\(34\)1/427/4\(n\+d\)7/4\\displaystyle\\geq\\frac\{\\pi^\{\-3/4\}\}\{4\\sqrt\{2\}\}\\sqrt\{2n\+d\-2\}\\cdot 2^\{\-3/4\}d^\{1/4\}\\cdot\\frac\{2^\{7/4\}\}\{n^\{7/4\}\}\\cdot\\left\(\\frac\{3\}\{4\}\\right\)^\{1/4\}\\frac\{2^\{7/4\}\}\{\(n\+d\)^\{7/4\}\}\(363\)≥31/421/4​π3/4⋅d1/4n7/4​\(n\+d\)5/4\\displaystyle\\geq\\frac\{3^\{1/4\}\}\{2^\{1/4\}\\pi^\{3/4\}\}\\cdot\\frac\{d^\{1/4\}\}\{n^\{7/4\}\(n\+d\)^\{5/4\}\}\(364\)where the first line is[DLMF 18\.17\.37](https://dlmf.nist.gov/18.17#E37)withλ=\(d−2\)/2\\lambda=\(d\-2\)/2, the second uses the duplication formula, the third uses Wendel’s inequality\(xx\+s\)1−s≤Γ⁡\(x\+s\)xs​Γ​\(x\)≤1\(\\frac\{x\}\{x\+s\}\)^\{1\-s\}\\leq\\frac\{\\Gamma\(x\+s\)\}\{x^\{s\}\\Gamma\(x\)\}\\leq 1forx\>0x\>0,s∈\(0,1\)s\\in\(0,1\)on all threeΓ\\Gammafraction terms along with\(n\+d−2\)/2≥3/2\(n\+d\-2\)/2\\geq 3/2, and the fourth uses2​n\+d−2≥n\+d\\sqrt\{2n\+d\-2\}\\geq\\sqrt\{n\+d\}\. In particular, if we taken≥dn\\geq dthen

\|∫\{⟨θ,u⟩\>0\}⟨θ,u⟩2Zn,u\(θ\)dσ\(θ\)\|≥31/423/2​π3/4⋅d1/4n−3≥d1/4​n−36,\\left\|\\int\_\{\\\{\\left\\langle\\theta,u\\right\\rangle\>0\\\}\}\\left\\langle\\theta,u\\right\\rangle^\{2\}Z\_\{n,u\}\(\\theta\)d\\sigma\(\\theta\)\\right\|\\geq\\frac\{3^\{1/4\}\}\{2^\{3/2\}\\pi^\{3/4\}\}\\cdot d^\{1/4\}n^\{\-3\}\\geq\\frac\{d^\{1/4\}n^\{\-3\}\}\{6\},\(365\)and so \([359](https://arxiv.org/html/2610.10875#A1.E359)\) is proved\.

Combining \([346](https://arxiv.org/html/2610.10875#A1.E346)\) and \([359](https://arxiv.org/html/2610.10875#A1.E359)\), we use our particular choiceY=Zn,uY=Z\_\{n,u\}andφn=\|x\|2​Zn,u​\(x\|x\|\)\\varphi\_\{n\}=\|x\|^\{2\}Z\_\{n,u\}\(\\frac\{x\}\{\|x\|\}\)to find that for any oddn≥dn\\geq dand any positively one\-homogeneousϕ\\phiwe have

An\\displaystyle A\_\{n\}≥\|∫ℝd⟨∇φn,w⟩​𝑑μ\|\\displaystyle\\geq\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi\_\{n\},w\\right\\rangle d\\mu\\right\|\(366\)≥\|c\|​M2​\(μ\)2​\(2\+n⁡\(n\+d−2\)2\)​\|ϕ⁡\(1\)\+ϕ⁡\(−1\)\|⋅d1/4​n−36\\displaystyle\\geq\|c\|M\_\{2\}\(\\mu\)^\{2\}\\left\(2\+\\frac\{n\(n\+d\-2\)\}\{2\}\\right\)\\left\|\\phi\(1\)\+\\phi\(\-1\)\\right\|\\cdot\\frac\{d^\{1/4\}n^\{\-3\}\}\{6\}\(367\)≥\|c\|​M2​\(μ\)2​d1/412​n​\|ϕ⁡\(1\)\+ϕ⁡\(−1\)\|\.\\displaystyle\\geq\|c\|\\frac\{M\_\{2\}\(\\mu\)^\{2\}d^\{1/4\}\}\{12n\}\|\\phi\(1\)\+\\phi\(\-1\)\|\.\(368\)Thus,An≳n−1A\_\{n\}\\gtrsim n^\{\-1\}for oddn≥dn\\geq dand any positively one\-homogeneousϕ\\phiwhich is not a scalar multiple of the identity\. Therefore,

an,d​An≳d1/4\(d−2\)\!⋅n\(d−4\)/2−1,\\sqrt\{a\_\{n,d\}\}A\_\{n\}\\gtrsim\\frac\{d^\{1/4\}\}\{\\sqrt\{\(d\-2\)\!\}\}\\cdot n^\{\(d\-4\)/2\-1\},\(369\)which is not summable innnford≥4d\\geq 4\. Hence, the right hand side of the lower bound in \([212](https://arxiv.org/html/2610.10875#A1.E212)\) can be made as large as desired\. ∎

This lack of summability in anyd≥4d\\geq 4in the ReLU case is also visible from Lemma[10](https://arxiv.org/html/2610.10875#Thmlemma10)whenμ=γd\\mu=\\gamma\_\{d\}\. We will instead derive the correct scaling in thetanh\\tanhcase:

###### Lemma 12\.

Withϕ=tanh\\phi=\\tanh, asd→∞d\\to\\inftywe have that

log⁡𝒟tanh​\(c\)=d4​log⁡d\+O⁡\(d\)\\log\\mathcal\{D\}\_\{\\tanh\}\(c\)=\\frac\{d\}\{4\}\\log d\+O\(d\)\(370\)uniformly forc∈\[−1,1\]c\\in\[\-1,1\]\.

###### Proof\.

Sincetanh\\tanhis odd, the coefficientsϕ^m−1\\hat\{\\phi\}\_\{m\-1\}vanish whenm−1m\-1is even \(i\.e\. whenmmis odd\)\. In the other casesm=2​qm=2q, we claim that asq→∞q\\to\\inftythere is the asymptotic

log⁡\|ϕ^2​q−1\|=q​log⁡\(2​q\)−q−π2​q\+O⁡\(log⁡q\)\.\\log\|\\hat\{\\phi\}\_\{2q\-1\}\|=q\\log\(2q\)\-q\-\\frac\{\\pi\}\{\\sqrt\{2\}\}\\sqrt\{q\}\+O\(\\log q\)\.\(371\)Before proving this, we show how it implies the main claim; from here on, universal constants are labeledCjC\_\{j\}and can change from line to line\. Recall from \([218](https://arxiv.org/html/2610.10875#A1.E218)\) that

Nn,d=2​n\+d−2d−2​\(n\+d−3d−3\),N\_\{n,d\}=\\frac\{2n\+d\-2\}\{d\-2\}\{n\+d\-3\\choose d\-3\},\(372\)and since\(n\+d−3d−3\)≤\(e⁡\(n\+d\)d\)d\{n\+d\-3\\choose d\-3\}\\leq\(\\frac\{e\(n\+d\)\}\{d\}\)^\{d\}we find thatNn,d≤C0d​\(1\+nd\)dN\_\{n,d\}\\leq C\_\{0\}^\{d\}\(1\+\\frac\{n\}\{d\}\)^\{d\}for someC0<∞C\_\{0\}<\\infty\. Moreover,

Nn,d≥\(n\+d−3d−3\)≥nd−3\(d−3\)\!≥\(nd\)d−3\(∀n≥d\)\.N\_\{n,d\}\\geq\{n\+d\-3\\choose d\-3\}\\geq\\frac\{n^\{d\-3\}\}\{\(d\-3\)\!\}\\geq\\left\(\\frac\{n\}\{d\}\\right\)^\{d\-3\}\\quad\(\\forall n\\geq d\)\.\(373\)Next, withm=n\+2​ℓm=n\+2\\elleven we claim that

\|ϕ^m−1\|​\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)1/2≤\(d\+m\)C1​\(dm\)d/4​exp⁡\(−n24​m−C2​m\)\|\\hat\{\\phi\}\_\{m\-1\}\|\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)^\{1/2\}\\leq\(d\+m\)^\{C\_\{1\}\}\\left\(\\frac\{d\}\{m\}\\right\)^\{d/4\}\\exp\\left\(\-\\frac\{n^\{2\}\}\{4m\}\-C\_\{2\}\\sqrt\{m\}\\right\)\(374\)for someC1,C2∈\(0,∞\)C\_\{1\},C\_\{2\}\\in\(0,\\infty\)\. Indeed, by \([371](https://arxiv.org/html/2610.10875#A1.E371)\) and Stirling we have for evenmmthat

log⁡\(\|ϕ^m−1\|​\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)1/2\)\\displaystyle\\log\\left\(\|\\hat\{\\phi\}\_\{m\-1\}\|\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)^\{1/2\}\\right\)=log\|ϕ^m−1\|\+12​log⁡m\+12​log⁡Γ⁡\(d/2\)−m2​log⁡2\\displaystyle=\\log\|\\hat\{\\phi\}\_\{m\-1\}\|\+\\frac\{1\}\{2\}\\log m\+\\frac\{1\}\{2\}\\log\\Gamma\(d/2\)\-\\frac\{m\}\{2\}\\log 2\(375\)−12​log⁡Γ⁡\(m−n2\+1\)−12​log⁡Γ⁡\(m\+n\+d2\)\\displaystyle\\quad\-\\frac\{1\}\{2\}\\log\\Gamma\\left\(\\frac\{m\-n\}\{2\}\+1\\right\)\-\\frac\{1\}\{2\}\\log\\Gamma\\left\(\\frac\{m\+n\+d\}\{2\}\\right\)\(376\)=m2​log⁡m−m2−π2​m−m2​log⁡2\+d4​log⁡d2−d4\\displaystyle=\\frac\{m\}\{2\}\\log m\-\\frac\{m\}\{2\}\-\\frac\{\\pi\}\{2\}\\sqrt\{m\}\-\\frac\{m\}\{2\}\\log 2\+\\frac\{d\}\{4\}\\log\\frac\{d\}\{2\}\-\\frac\{d\}\{4\}\(377\)−m−n4​log⁡\(m−n2\)\+m−n4\\displaystyle\\quad\-\\frac\{m\-n\}\{4\}\\log\\left\(\\frac\{m\-n\}\{2\}\\right\)\+\\frac\{m\-n\}\{4\}\(378\)−m\+n\+d4​log⁡\(m\+n\+d2\)\+m\+n\+d4\\displaystyle\\quad\-\\frac\{m\+n\+d\}\{4\}\\log\\left\(\\frac\{m\+n\+d\}\{2\}\\right\)\+\\frac\{m\+n\+d\}\{4\}\(379\)\+O⁡\(log⁡\(d\+m\)\)\\displaystyle\\quad\+O\(\\log\(d\+m\)\)\(380\)=d4​log⁡dm−m−n4​log⁡\(1−nm\)\\displaystyle=\\frac\{d\}\{4\}\\log\\frac\{d\}\{m\}\-\\frac\{m\-n\}\{4\}\\log\\left\(1\-\\frac\{n\}\{m\}\\right\)\(381\)−m\+n\+d4​log⁡\(1\+n\+dm\)−π2​m\\displaystyle\\quad\-\\frac\{m\+n\+d\}\{4\}\\log\\left\(1\+\\frac\{n\+d\}\{m\}\\right\)\-\\frac\{\\pi\}\{2\}\\sqrt\{m\}\(382\)\+O⁡\(log⁡\(d\+m\)\),\\displaystyle\\quad\+O\(\\log\(d\+m\)\),\(383\)where we use the convention0​log⁡0=00\\log 0=0\. Therefore,

log⁡\(\|ϕ^m−1\|​\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)1/2\)\\displaystyle\\log\\left\(\|\\hat\{\\phi\}\_\{m\-1\}\|\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)^\{1/2\}\\right\)≤d4​log⁡dm−π2​m\+O⁡\(log⁡\(d\+m\)\)\\displaystyle\\leq\\frac\{d\}\{4\}\\log\\frac\{d\}\{m\}\-\\frac\{\\pi\}\{2\}\\sqrt\{m\}\+O\(\\log\(d\+m\)\)\(384\)−m4​\(\(1−nm\)​log⁡\(1−nm\)\+\(1\+nm\)​log⁡\(1\+nm\)\)\.\\displaystyle\\quad\-\\frac\{m\}\{4\}\\left\(\\left\(1\-\\frac\{n\}\{m\}\\right\)\\log\\left\(1\-\\frac\{n\}\{m\}\\right\)\+\\left\(1\+\\frac\{n\}\{m\}\\right\)\\log\\left\(1\+\\frac\{n\}\{m\}\\right\)\\right\)\.\(385\)Since\(1−x\)​log⁡\(1−x\)\+\(1\+x\)​log⁡\(1\+x\)≥x2\(1\-x\)\\log\(1\-x\)\+\(1\+x\)\\log\(1\+x\)\\geq x^\{2\}for\|x\|≤1\|x\|\\leq 1, takingx=n/mx=n/mimplies

log⁡\(\|ϕ^m−1\|​\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)1/2\)≤d4​log⁡dm−n24​m−π2​m\+O⁡\(log⁡\(d\+m\)\)\\log\\left\(\|\\hat\{\\phi\}\_\{m\-1\}\|\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)^\{1/2\}\\right\)\\leq\\frac\{d\}\{4\}\\log\\frac\{d\}\{m\}\-\\frac\{n^\{2\}\}\{4m\}\-\\frac\{\\pi\}\{2\}\\sqrt\{m\}\+O\(\\log\(d\+m\)\)\(386\)which proves \([374](https://arxiv.org/html/2610.10875#A1.E374)\)\.

For the upper bound, we note thatc2\+\(1−c2\)​n⁡\(n\+d−2\)m2​\(d−1\)≤1c^\{2\}\+\(1\-c^\{2\}\)\\frac\{n\(n\+d\-2\)\}\{m^\{2\}\(d\-1\)\}\\leq 1and that∑ℓxℓ≤∑ℓxℓ\\sqrt\{\\sum\_\{\\ell\}x\_\{\\ell\}\}\\leq\\sum\_\{\\ell\}\\sqrt\{x\_\{\\ell\}\}, and so applying our upper bound onNn,dN\_\{n,d\}and \([374](https://arxiv.org/html/2610.10875#A1.E374)\) yields

𝒟tanh​\(c\)\\displaystyle\\mathcal\{D\}\_\{\\tanh\}\(c\)≤∑n=0∞Nn,d​∑ℓ=0n\+2​ℓ≥1∞\|ϕ^m−1\|​\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)1/2\\displaystyle\\leq\\sum\_\{n=0\}^\{\\infty\}N\_\{n,d\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell=0\\\\ n\+2\\ell\\geq 1\\end\{subarray\}\}^\{\\infty\}\|\\hat\{\\phi\}\_\{m\-1\}\|\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)^\{1/2\}\(387\)≤C0d​∑m=1∞\(d\+m\)C1​\(dm\)d/4​e−C2​m⋅∑n=0m\(1\+nd\)d​e−n24​m\\displaystyle\\leq C\_\{0\}^\{d\}\\sum\_\{m=1\}^\{\\infty\}\(d\+m\)^\{C\_\{1\}\}\\left\(\\frac\{d\}\{m\}\\right\)^\{d/4\}e^\{\-C\_\{2\}\\sqrt\{m\}\}\\cdot\\sum\_\{n=0\}^\{m\}\\left\(1\+\\frac\{n\}\{d\}\\right\)^\{d\}e^\{\-\\frac\{n^\{2\}\}\{4m\}\}\(388\)≤C0d​∑m=1∞\(d\+m\)C1​\(dm\)d/4​e−C2​m⋅∑n=0∞\(1\+nd\)d​e−n24​m\\displaystyle\\leq C\_\{0\}^\{d\}\\sum\_\{m=1\}^\{\\infty\}\(d\+m\)^\{C\_\{1\}\}\\left\(\\frac\{d\}\{m\}\\right\)^\{d/4\}e^\{\-C\_\{2\}\\sqrt\{m\}\}\\cdot\\sum\_\{n=0\}^\{\\infty\}\\left\(1\+\\frac\{n\}\{d\}\\right\)^\{d\}e^\{\-\\frac\{n^\{2\}\}\{4m\}\}\(389\)after reorganizing the sums\. Splitting the inner sum, ifn≤dn\\leq dthen\(1\+nd\)d≤2d\(1\+\\frac\{n\}\{d\}\)^\{d\}\\leq 2^\{d\}and so∑n=0d\(1\+nd\)d​e−n24​m≤\(d\+1\)​2d\\sum\_\{n=0\}^\{d\}\\left\(1\+\\frac\{n\}\{d\}\\right\)^\{d\}e^\{\-\\frac\{n^\{2\}\}\{4m\}\}\\leq\(d\+1\)2^\{d\}\. Ifn\>dn\>dthen we have

∑n=d\+1∞\(1\+nd\)d​e−n24​m≤\(2d\)d​∑n=1∞nd​e−n24​m≤\(2d\)d​\(C4​d​m\)d/2​m=C5d​\(md\)d/2​m,\\sum\_\{n=d\+1\}^\{\\infty\}\\left\(1\+\\frac\{n\}\{d\}\\right\)^\{d\}e^\{\-\\frac\{n^\{2\}\}\{4m\}\}\\leq\\left\(\\frac\{2\}\{d\}\\right\)^\{d\}\\sum\_\{n=1\}^\{\\infty\}n^\{d\}e^\{\-\\frac\{n^\{2\}\}\{4m\}\}\\leq\\left\(\\frac\{2\}\{d\}\\right\)^\{d\}\(C\_\{4\}dm\)^\{d/2\}\\sqrt\{m\}=C\_\{5\}^\{d\}\\left\(\\frac\{m\}\{d\}\\right\)^\{d/2\}\\sqrt\{m\},\(390\)where the second inequality comes from comparison with the integral and Stirling\. In particular,

∑n=0∞\(1\+nd\)d​e−n24​m≤C6d​\(d\+m\)C7​\(1\+\(md\)d/2\)\\sum\_\{n=0\}^\{\\infty\}\\left\(1\+\\frac\{n\}\{d\}\\right\)^\{d\}e^\{\-\\frac\{n^\{2\}\}\{4m\}\}\\leq C\_\{6\}^\{d\}\(d\+m\)^\{C\_\{7\}\}\\left\(1\+\\left\(\\frac\{m\}\{d\}\\right\)^\{d/2\}\\right\)\(391\)and therefore

𝒟tanh​\(c\)≤C8d​∑m=1∞\(d\+m\)C9​\(\(dm\)d/4\+\(md\)d/4\)​e−C2​m\.\\mathcal\{D\}\_\{\\tanh\}\(c\)\\leq C\_\{8\}^\{d\}\\sum\_\{m=1\}^\{\\infty\}\(d\+m\)^\{C\_\{9\}\}\\left\(\\left\(\\frac\{d\}\{m\}\\right\)^\{d/4\}\+\\left\(\\frac\{m\}\{d\}\\right\)^\{d/4\}\\right\)e^\{\-C\_\{2\}\\sqrt\{m\}\}\.\(392\)We now split this sum intom≤dm\\leq dandm\>dm\>d\. Ifm≤dm\\leq dthen\(dm\)d/4≤dd/4\(\\frac\{d\}\{m\}\)^\{d/4\}\\leq d^\{d/4\}and\(md\)d/4≤1\(\\frac\{m\}\{d\}\)^\{d/4\}\\leq 1, and hence

∑m=1d\(d\+m\)C9​\(\(dm\)d/4\+\(md\)d/4\)​e−C2​m≤C10d​dd/4\.\\sum\_\{m=1\}^\{d\}\(d\+m\)^\{C\_\{9\}\}\\left\(\\left\(\\frac\{d\}\{m\}\\right\)^\{d/4\}\+\\left\(\\frac\{m\}\{d\}\\right\)^\{d/4\}\\right\)e^\{\-C\_\{2\}\\sqrt\{m\}\}\\leq C\_\{10\}^\{d\}d^\{d/4\}\.\(393\)Form\>dm\>dwe know\(dm\)d/4\+\(md\)d/4≤2​\(md\)d/4\\left\(\\frac\{d\}\{m\}\\right\)^\{d/4\}\+\\left\(\\frac\{m\}\{d\}\\right\)^\{d/4\}\\leq 2\\left\(\\frac\{m\}\{d\}\\right\)^\{d/4\}and hence

∑m=d\+1∞\(d\+m\)C9​\(\(dm\)d/4\+\(md\)d/4\)​e−C2​m\\displaystyle\\sum\_\{m=d\+1\}^\{\\infty\}\(d\+m\)^\{C\_\{9\}\}\\left\(\\left\(\\frac\{d\}\{m\}\\right\)^\{d/4\}\+\\left\(\\frac\{m\}\{d\}\\right\)^\{d/4\}\\right\)e^\{\-C\_\{2\}\\sqrt\{m\}\}≤C11dd−d/4∑m=d\+1∞md/4\+C12e−c​m\\displaystyle\\leq C\_\{11\}^\{d\}d^\{\-d/4\}\\sum\_\{m=d\+1\}^\{\\infty\}m^\{d/4\+C\_\{12\}\}e^\{\-c\\sqrt\{m\}\}\(394\)≤C13dd−d/4Γ\(d2\+2C12\+2\)\\displaystyle\\leq C\_\{13\}^\{d\}d^\{\-d/4\}\\Gamma\\left\(\\frac\{d\}\{2\}\+2C\_\{12\}\+2\\right\)\(395\)≤C14d​dd/4,\\displaystyle\\leq C\_\{14\}^\{d\}d^\{d/4\},\(396\)where the second line compares with the integral overx=mx=\\sqrt\{m\}and the third uses Stirling\. Combining this with \([393](https://arxiv.org/html/2610.10875#A1.E393)\) proves that uniformly incc,

log⁡𝒟tanh​\(c\)≤d4​log⁡d\+O⁡\(d\)\.\\log\\mathcal\{D\}\_\{\\tanh\}\(c\)\\leq\\frac\{d\}\{4\}\\log d\+O\(d\)\.\(397\)For the lower bound, if we takem≍d2m\\asymp d^\{2\}andn≍d3/2n\\asymp d^\{3/2\}both even then

Nn,d≥\(nd\)d−3=exp⁡\(d2​log⁡d−O⁡\(d\)\)N\_\{n,d\}\\geq\\left\(\\frac\{n\}\{d\}\\right\)^\{d\-3\}=\\exp\\left\(\\frac\{d\}\{2\}\\log d\-O\(d\)\\right\)\(398\)by \([373](https://arxiv.org/html/2610.10875#A1.E373)\), and the formula of \([381](https://arxiv.org/html/2610.10875#A1.E381)\)\-\([383](https://arxiv.org/html/2610.10875#A1.E383)\) simplifies to

log⁡\(\|ϕ^m−1\|​\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)1/2\)=d4​log⁡\(dm\)\+O⁡\(d\)\\log\\left\(\|\\hat\{\\phi\}\_\{m\-1\}\|\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)^\{1/2\}\\right\)=\\frac\{d\}\{4\}\\log\\left\(\\frac\{d\}\{m\}\\right\)\+O\(d\)\(399\)by expanding out the logs\. Sincem≍d2m\\asymp d^\{2\}, this yields

\|ϕ^m−1\|​\(m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)\)1/2≥exp⁡\(−d4​log⁡d−O⁡\(d\)\)\.\|\\hat\{\\phi\}\_\{m\-1\}\|\\left\(\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\\right\)^\{1/2\}\\geq\\exp\\left\(\-\\frac\{d\}\{4\}\\log d\-O\(d\)\\right\)\.\(400\)Lastly, sincec2\+\(1−c2\)​n⁡\(n\+d−2\)m2​\(d−1\)≥n⁡\(n\+d−2\)m2​\(d−1\)≳d−2c^\{2\}\+\(1\-c^\{2\}\)\\frac\{n\(n\+d\-2\)\}\{m^\{2\}\(d\-1\)\}\\geq\\frac\{n\(n\+d\-2\)\}\{m^\{2\}\(d\-1\)\}\\gtrsim d^\{\-2\}we may multiply \([398](https://arxiv.org/html/2610.10875#A1.E398)\) and \([400](https://arxiv.org/html/2610.10875#A1.E400)\) to lower bound this single term in the sum\. Hence, the lower bound

log⁡𝒟tanh​\(c\)≥d4​log⁡d−O⁡\(d\)\\log\\mathcal\{D\}\_\{\\tanh\}\(c\)\\geq\\frac\{d\}\{4\}\\log d\-O\(d\)\(401\)holds uniformly incc\.

We now prove the Hermite coefficient asymptotics \([371](https://arxiv.org/html/2610.10875#A1.E371)\)\. Applying Gaussian integration by parts2​q−12q\-1times gives that ifZ∼𝒩⁡\(0,1\)Z\\sim\\mathcal\{N\}\(0,1\)then

ϕ^2​q−1=𝔼⁡\[tanh⁡\(Z\)​He2​q−1⁡\(Z\)\]=𝔼⁡\[tanh\(2​q−1\)⁡\(Z\)\]\.\\hat\{\\phi\}\_\{2q\-1\}=\\mathbb\{E\}\[\\tanh\(Z\)\\operatorname\{He\}\_\{2q\-1\}\(Z\)\]=\\mathbb\{E\}\[\\tanh^\{\(2q\-1\)\}\(Z\)\]\.\(402\)Observe thattanh′=sech2\\tanh^\{\\prime\}=\\operatorname\{sech\}^\{2\}and that the Fourier transform satisfiessech2^​\(ξ\)=π​ξsinh⁡\(π​ξ/2\)\\widehat\{\\operatorname\{sech\}^\{2\}\}\(\\xi\)=\\frac\{\\pi\\xi\}\{\\sinh\(\\pi\\xi/2\)\}\(see \(3\.982\.1\) of\[[40](https://arxiv.org/html/2610.10875#bib.bib45)\]\), and so

ϕ^2​q−1\\displaystyle\\hat\{\\phi\}\_\{2q\-1\}=12​π∫ℝ\(sech2\)\(2​q−2\)\(x\)e−x2/2dx\\displaystyle=\\frac\{1\}\{\\sqrt\{2\\pi\}\}\\int\_\{\\mathbb\{R\}\}\(\\operatorname\{sech\}^\{2\}\)^\{\(2q\-2\)\}\(x\)e^\{\-x^\{2\}/2\}dx\(403\)=1\(2​π\)3/2​∫ℝ\(sech2\)\(2​q−2\)^​\(ξ\)​e−x2/2^​\(ξ\)​𝑑ξ\\displaystyle=\\frac\{1\}\{\(2\\pi\)^\{3/2\}\}\\int\_\{\\mathbb\{R\}\}\\widehat\{\(\\operatorname\{sech\}^\{2\}\)^\{\(2q\-2\)\}\}\(\\xi\)\\widehat\{e^\{\-x^\{2\}/2\}\}\(\\xi\)d\\xi\(404\)=12∫ℝ\(iξ\)2​q−2ξsinh⁡\(π​ξ/2\)e−ξ2/2dξ\\displaystyle=\\frac\{1\}\{2\}\\int\_\{\\mathbb\{R\}\}\(i\\xi\)^\{2q\-2\}\\frac\{\\xi\}\{\\sinh\(\\pi\\xi/2\)\}e^\{\-\\xi^\{2\}/2\}d\\xi\(405\)=\(−1\)2​q2−12​∫ℝξ2​q−1e−ξ2/2sinh⁡\(π​ξ/2\)​𝑑ξ,\\displaystyle=\\frac\{\(\-1\)^\{\\frac\{2q\}\{2\}\-1\}\}\{2\}\\int\_\{\\mathbb\{R\}\}\\frac\{\\xi^\{2q\-1\}e^\{\-\\xi^\{2\}/2\}\}\{\\sinh\(\\pi\\xi/2\)\}d\\xi,\(406\)where the second line uses Parseval’s identity and the fourth uses that2​q2qis even\. In particular,

\|ϕ^2​q−1\|\\displaystyle\|\\hat\{\\phi\}\_\{2q\-1\}\|=∫0∞ξ2​q−1e−ξ2/2sinh⁡\(π​ξ/2\)​𝑑ξ=2​∫0∞ξ2​q−1​exp⁡\(−ξ22−π​ξ2\)1−e−π​ξ​𝑑ξ\.\\displaystyle=\\int\_\{0\}^\{\\infty\}\\frac\{\\xi^\{2q\-1\}e^\{\-\\xi^\{2\}/2\}\}\{\\sinh\(\\pi\\xi/2\)\}d\\xi=2\\int\_\{0\}^\{\\infty\}\\frac\{\\xi^\{2q\-1\}\\exp\\left\(\-\\frac\{\\xi^\{2\}\}\{2\}\-\\frac\{\\pi\\xi\}\{2\}\\right\)\}\{1\-e^\{\-\\pi\\xi\}\}d\\xi\.\(407\)Hence, since the regionξ∈\[0,1\]\\xi\\in\[0,1\]contributesO⁡\(1\)O\(1\)to this integral we find that

log⁡\|ϕ^2​q−1\|=log⁡\(∫1∞eF⁡\(ξ\)​𝑑ξ\)\+O⁡\(1\)\\log\|\\hat\{\\phi\}\_\{2q\-1\}\|=\\log\\left\(\\int\_\{1\}^\{\\infty\}e^\{F\(\\xi\)\}d\\xi\\right\)\+O\(1\)\(408\)where we denoteF⁡\(ξ\):=\(2​q−1\)​log⁡ξ−ξ22−π​ξ2F\(\\xi\):=\(2q\-1\)\\log\\xi\-\\frac\{\\xi^\{2\}\}\{2\}\-\\frac\{\\pi\\xi\}\{2\}\. We compute that

F′​\(2​q\)=−π2−12​q=O⁡\(1\),supξ\>0F′′​\(ξ\)=supξ\>0\{−2​q−1ξ2−1\}=−1\.F^\{\\prime\}\(\\sqrt\{2q\}\)=\-\\frac\{\\pi\}\{2\}\-\\frac\{1\}\{\\sqrt\{2q\}\}=O\(1\),\\quad\\sup\_\{\\xi\>0\}F^\{\\prime\\prime\}\(\\xi\)=\\sup\_\{\\xi\>0\}\\left\\\{\-\\frac\{2q\-1\}\{\\xi^\{2\}\}\-1\\right\\\}=\-1\.\(409\)By Taylor’s theorem, for everyξ≥1\\xi\\geq 1we haveF⁡\(ξ\)≤F⁡\(2​q\)\+O⁡\(1\)−14​\(ξ−2​q\)2F\(\\xi\)\\leq F\(\\sqrt\{2q\}\)\+O\(1\)\-\\frac\{1\}\{4\}\(\\xi\-\\sqrt\{2q\}\)^\{2\}and therefore

log⁡\(∫1∞eF⁡\(ξ\)​𝑑ξ\)≤F⁡\(2​q\)\+O⁡\(1\)\.\\log\\left\(\\int\_\{1\}^\{\\infty\}e^\{F\(\\xi\)\}d\\xi\\right\)\\leq F\(\\sqrt\{2q\}\)\+O\(1\)\.\(410\)For the lower bound, we note that

supξ∈\[2​q,2​q\+1\]\{F⁡\(2​q\)−F⁡\(ξ\)\}=O⁡\(1\)\\sup\_\{\\xi\\in\[\\sqrt\{2q\},\\sqrt\{2q\}\+1\]\}\\left\\\{F\(\\sqrt\{2q\}\)\-F\(\\xi\)\\right\\\}=O\(1\)\(411\)and therefore that

log⁡\(∫1∞eF⁡\(ξ\)​𝑑ξ\)≥log⁡\(∫2​q2​q\+1eF⁡\(ξ\)​𝑑ξ\)≥F⁡\(2​q\)−O⁡\(1\)\.\\log\\left\(\\int\_\{1\}^\{\\infty\}e^\{F\(\\xi\)\}d\\xi\\right\)\\geq\\log\\left\(\\int\_\{\\sqrt\{2q\}\}^\{\\sqrt\{2q\}\+1\}e^\{F\(\\xi\)\}d\\xi\\right\)\\geq F\(\\sqrt\{2q\}\)\-O\(1\)\.\(412\)Hence, \([371](https://arxiv.org/html/2610.10875#A1.E371)\) follows from the computation

F⁡\(2​q\)=\(2​q−1\)​log⁡\(2​q\)−q−π2​2​q=q​log⁡\(2​q\)−q−π2​q−12​log⁡\(2​q\)\.\\displaystyle F\(\\sqrt\{2q\}\)=\(2q\-1\)\\log\(\\sqrt\{2q\}\)\-q\-\\frac\{\\pi\}\{2\}\\sqrt\{2q\}=q\\log\(2q\)\-q\-\\frac\{\\pi\}\{\\sqrt\{2\}\}\\sqrt\{q\}\-\\frac\{1\}\{2\}\\log\(2q\)\.\(413\)
∎

We can do the same for other activations of interest, such assin\\sin:

###### Lemma 13\.

Withϕ=sin\\phi=\\sin, asd→∞d\\to\\inftywe have that

log⁡𝒟sin​\(c\)=d\+o⁡\(d\)\\log\\mathcal\{D\}\_\{\\sin\}\(c\)=\\sqrt\{d\}\+o\(\\sqrt\{d\}\)\(414\)uniformly forc∈\[−1,1\]c\\in\[\-1,1\]\.

###### Proof\.

First, we use the fact that

𝔼ξ∼𝒩⁡\(0,1\)\[ei​t​ξHej\(ξ\)\]=\(it\)je−t2/2\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[e^\{it\\xi\}\\operatorname\{He\}\_\{j\}\(\\xi\)\]=\(it\)^\{j\}e^\{\-t^\{2\}/2\}\(415\)and take imaginary parts att=1t=1to see that

𝔼ξ∼𝒩⁡\(0,1\)\[sin\(ξ\)Hej\(ξ\)\]=e−1/2Im\(ij\)\.\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\sin\(\\xi\)\\operatorname\{He\}\_\{j\}\(\\xi\)\]=e^\{\-1/2\}\\operatorname\{Im\}\(i^\{j\}\)\.\(416\)Hence, the Hermite coefficients satisfy

\|ϕ^j\|2=\{0j​even1/ej​odd\|\\hat\{\\phi\}\_\{j\}\|^\{2\}=\\begin\{cases\}0&j\\text\{ even\}\\\\ 1/e&j\\text\{ odd\}\\end\{cases\}\(417\)and so only evenm=n\+2​ℓm=n\+2\\ellcontribute \(and therefore only evennn\)\.

Now, forn≥2n\\geq 2we denote

Tn,d:=Nn,d​\(n​Γ​\(d/2\)2n​Γ​\(n\+d/2\)\)1/2\.T\_\{n,d\}:=N\_\{n,d\}\\left\(\\frac\{n\\Gamma\(d/2\)\}\{2^\{n\}\\Gamma\(n\+d/2\)\}\\right\)^\{1/2\}\.\(418\)Form=n\+2​ℓm=n\+2\\ellwe know that

m​Γ​\(d/2\)2m​ℓ\!​Γ​\(n\+ℓ\+d/2\)n​Γ​\(d/2\)2n​Γ​\(n\+d/2\)=mn​14ℓ​ℓ\!​Γ⁡\(n\+d/2\)Γ⁡\(n\+ℓ\+d/2\)≤mn​14ℓ​ℓ\!​\(2d\)ℓ\.\\frac\{\\frac\{m\\Gamma\(d/2\)\}\{2^\{m\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}\}\{\\frac\{n\\Gamma\(d/2\)\}\{2^\{n\}\\Gamma\(n\+d/2\)\}\}=\\frac\{m\}\{n\}\\frac\{1\}\{4^\{\\ell\}\\ell\!\}\\frac\{\\Gamma\(n\+d/2\)\}\{\\Gamma\(n\+\\ell\+d/2\)\}\\leq\\frac\{m\}\{n\}\\frac\{1\}\{4^\{\\ell\}\\ell\!\}\\left\(\\frac\{2\}\{d\}\\right\)^\{\\ell\}\.\(419\)In particular, the sum overℓ\\ellbecomes

∑ℓ=0∞\(n\+2​ℓ\)​Γ​\(d/2\)2n\+2​ℓ​ℓ\!​Γ​\(n\+ℓ\+d/2\)=\(1\+O⁡\(d−1\)\)​n​Γ​\(d/2\)2n​Γ​\(n\+d/2\),\\sum\_\{\\ell=0\}^\{\\infty\}\\frac\{\(n\+2\\ell\)\\Gamma\(d/2\)\}\{2^\{n\+2\\ell\}\\ell\!\\Gamma\(n\+\\ell\+d/2\)\}=\(1\+O\(d^\{\-1\}\)\)\\frac\{n\\Gamma\(d/2\)\}\{2^\{n\}\\Gamma\(n\+d/2\)\},\(420\)i\.e\. it is dominated by theℓ=0\\ell=0term\. Moreover, we have

0≤c2\+\(1−c2\)​n⁡\(n\+d−2\)\(n\+2​ℓ\)2​\(d−1\)≤10\\leq c^\{2\}\+\(1\-c^\{2\}\)\\frac\{n\(n\+d\-2\)\}\{\(n\+2\\ell\)^\{2\}\(d\-1\)\}\\leq 1\(421\)while atℓ=0\\ell=0we have

c2\+\(1−c2\)​n\+d−2n⁡\(d−1\)≥1n\.c^\{2\}\+\(1\-c^\{2\}\)\\frac\{n\+d\-2\}\{n\(d\-1\)\}\\geq\\frac\{1\}\{n\}\.\(422\)Lastly, then=0n=0term hasΓ⁡\(d/2\)Γ⁡\(d/2\+1\)≍d−1\\frac\{\\Gamma\(d/2\)\}\{\\Gamma\(d/2\+1\)\}\\asymp d^\{\-1\}and so it contributesO\(d−1/2\)O\(d^\{\-1/2\}\)uniformly inccafter the square root\. Therefore, uniformly inccwe have

e−1/2∑n≥2n​evenTn,dn≤𝒟sin\(c\)≤O\(d−1/2\)\+e−1/2\(1\+O\(d−1\)\)∑n≥2n​evenTn,d\.e^\{\-1/2\}\\sum\_\{\\begin\{subarray\}\{c\}n\\geq 2\\\\ n\\text\{ even\}\\end\{subarray\}\}\\frac\{T\_\{n,d\}\}\{\\sqrt\{n\}\}\\leq\\mathcal\{D\}\_\{\\sin\}\(c\)\\leq O\(d^\{\-1/2\}\)\+e^\{\-1/2\}\(1\+O\(d^\{\-1\}\)\)\\sum\_\{\\begin\{subarray\}\{c\}n\\geq 2\\\\ n\\text\{ even\}\\end\{subarray\}\}T\_\{n,d\}\.\(423\)Forn≤d2/3n\\leq d^\{2/3\}we know that

log⁡Nn,d=n​log⁡d−log⁡\(n\!\)\+O⁡\(n2d\+1\)\\log N\_\{n,d\}=n\\log d\-\\log\(n\!\)\+O\\left\(\\frac\{n^\{2\}\}\{d\}\+1\\right\)\(424\)and that

12​log⁡\(n​Γ​\(d/2\)2n​Γ​\(n\+d/2\)\)=−n2​log⁡d\+12​log⁡n\+O⁡\(n2d\)\.\\frac\{1\}\{2\}\\log\\left\(\\frac\{n\\Gamma\(d/2\)\}\{2^\{n\}\\Gamma\(n\+d/2\)\}\\right\)=\-\\frac\{n\}\{2\}\\log d\+\\frac\{1\}\{2\}\\log n\+O\\left\(\\frac\{n^\{2\}\}\{d\}\\right\)\.\(425\)Combining these two and using Stirling,

log⁡Tn,d\\displaystyle\\log T\_\{n,d\}=n⁡\(1−log⁡nd\)\+O⁡\(n2d\+1\)\\displaystyle=n\\left\(1\-\\log\\frac\{n\}\{\\sqrt\{d\}\}\\right\)\+O\\left\(\\frac\{n^\{2\}\}\{d\}\+1\\right\)\(426\)=n⁡\(1−log⁡nd\)\+o⁡\(d\)\\displaystyle=n\\left\(1\-\\log\\frac\{n\}\{\\sqrt\{d\}\}\\right\)\+o\(\\sqrt\{d\}\)\(427\)uniformly overn≤d2/3n\\leq d^\{2/3\}\.

For the upper bound, we split

∑n≥2n​evenTn,d=∑2≤n≤d2/3n​evenTn,d\+∑n\>d2/3n​evenTn,d\.\\displaystyle\\sum\_\{\\begin\{subarray\}\{c\}n\\geq 2\\\\ n\\text\{ even\}\\end\{subarray\}\}T\_\{n,d\}=\\sum\_\{\\begin\{subarray\}\{c\}2\\leq n\\leq d^\{2/3\}\\\\ n\\text\{ even\}\\end\{subarray\}\}T\_\{n,d\}\+\\sum\_\{\\begin\{subarray\}\{c\}n\>d^\{2/3\}\\\\ n\\text\{ even\}\\end\{subarray\}\}T\_\{n,d\}\.\(428\)In the first sum, we use thatx⁡\(1−log⁡x\)≤1x\(1\-\\log x\)\\leq 1withx=n/dx=n/\\sqrt\{d\}to get thatTn,d≤exp⁡\(d\+o⁡\(d\)\)T\_\{n,d\}\\leq\\exp\(\\sqrt\{d\}\+o\(\\sqrt\{d\}\)\)forn≤d2/3n\\leq d^\{2/3\}\. Since there are onlyd2/3d^\{2/3\}many terms, we have

∑2≤n≤d2/3n​evenTn,d≤exp⁡\(d\+o⁡\(d\)\)\.\\sum\_\{\\begin\{subarray\}\{c\}2\\leq n\\leq d^\{2/3\}\\\\ n\\text\{ even\}\\end\{subarray\}\}T\_\{n,d\}\\leq\\exp\(\\sqrt\{d\}\+o\(\\sqrt\{d\}\)\)\.\(429\)For the second sum, we use the definition \([418](https://arxiv.org/html/2610.10875#A1.E418)\) ofTn,dT\_\{n,d\}to see that

Tn\+1,dTn,d=\(n\+d−2\)​2​n\+d\(2​n\+d−2\)​n⁡\(n\+1\)\.\\frac\{T\_\{n\+1,d\}\}\{T\_\{n,d\}\}=\\frac\{\(n\+d\-2\)\\sqrt\{2n\+d\}\}\{\(2n\+d\-2\)\\sqrt\{n\(n\+1\)\}\}\.\(430\)Ifd2/3<n≤dd^\{2/3\}<n\\leq dthen this is≤3​dn≤3d−1/6\\leq\\frac\{\\sqrt\{3d\}\}\{n\}\\leq\\sqrt\{3\}d^\{\-1/6\}, and ifn\>dn\>dthen this is≤3n≤3d−1/3\\leq\\sqrt\{\\frac\{3\}\{n\}\}\\leq\\sqrt\{3\}d^\{\-1/3\}\. In any case, for sufficiently largeddwe haveTn\+1,dTn,d≤12\\frac\{T\_\{n\+1,d\}\}\{T\_\{n,d\}\}\\leq\\frac\{1\}\{2\}forn\>d2/3n\>d^\{2/3\}, and so this sum is geometric\. Plugging these in proves the upper bound\. For the lower bound, letnnbe the nearest even integer tod\\sqrt\{d\}and use \([427](https://arxiv.org/html/2610.10875#A1.E427)\) to find that

log⁡Tn,d=d\+o⁡\(d\)\.\\log T\_\{n,d\}=\\sqrt\{d\}\+o\(\\sqrt\{d\}\)\.\(431\)∎

These results describe a superpolynomial separation between the Barron and rotation\-equivariant Stein transport metrics along single\-index flows in the perturbatively small regime\. To summarize, we have shown that ifνε:=Law⁡\(Xε\)\\nu\_\{\\varepsilon\}:=\\operatorname\{Law\}\(X\_\{\\varepsilon\}\)for the time\-invariant single\-index flow

X˙t=u​ϕ​\(⟨Xt,θ⟩\),X0∼γd,\\dot\{X\}\_\{t\}=u\\phi\(\\left\\langle X\_\{t\},\\theta\\right\\rangle\),\\quad X\_\{0\}\\sim\\gamma\_\{d\},\(432\)then for anyϕ∈\{ReLU,tanh,sin\}\\phi\\in\\\{\\operatorname\{ReLU\},\\tanh,\\sin\\\}and any rotation\-equivariantk∈𝒦rotk\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}satisfying Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)with∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)∈\(0,∞\)\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\\in\(0,\\infty\)we apply these results along with Lemma[11](https://arxiv.org/html/2610.10875#Thmlemma11)to find that

∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)⋅lim infε↓0Dℋk​\(γd,νε\)Dℬϕ​\(γd,νε\)≳\{\+∞ϕ=ReLUdd4​\(1−o​\(1\)\)ϕ=tanh,ed​\(1−o​\(1\)\)ϕ=sin,\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\}\\cdot\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\\gtrsim\\begin\{cases\}\+\\infty&\\phi=\\operatorname\{ReLU\}\\\\ d^\{\\frac\{d\}\{4\}\(1\-o\(1\)\)\}&\\phi=\\tanh,\\\\ e^\{\\sqrt\{d\}\(1\-o\(1\)\)\}&\\phi=\\sin,\\end\{cases\}\(433\)ifd≥4d\\geq 4and⟨u,θ⟩≠0\\left\\langle u,\\theta\\right\\rangle\\neq 0, and that these scalings are sharp in the sense that there are rotation\-equivariant kernels matching them\. We emphasize that in this perturbative setting we are effectively constraining transports to use time\-independent drifts, thereby linearizing the problem and reducing to control of the RKHS norms of single\-index functions\.

This result makes it tempting to claim that “for smallε\\varepsilon, there is a superpolynomial\-in\-ddseparation between rotation\-equivariantDℋkD\_\{\\mathcal\{H\}\_\{k\}\}and BarronDℬϕD\_\{\\mathcal\{B\}\_\{\\phi\}\}in terms of the representation ofε\\varepsilon\-length single\-index flows”\. By \([433](https://arxiv.org/html/2610.10875#A1.E433)\) this is true asε↓0\\varepsilon\\downarrow 0withddfixed\. However, this has the potential to be misleading: the superpolynomial separation may only kick in whenε\\varepsilonis smaller than any1poly⁡\(d\)\\frac\{1\}\{\\operatorname\{poly\}\(d\)\}\.

In fact, it turns out that this is the case\. More precisely, the next result shows that

ε≳1poly⁡\(d\)⟹∫ℝdtr⁡k⁡\(x,x\)​d​γd​\(x\)⋅Dℋk​\(γd,νε\)Dℬϕ​\(γd,νε\)≲poly⁡\(d\)\\varepsilon\\gtrsim\\frac\{1\}\{\\operatorname\{poly\}\(d\)\}\\implies\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\(x,x\)d\\gamma\_\{d\}\(x\)\}\\cdot\\frac\{D\_\{\\mathcal\{H\}\_\{k\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\nu\_\{\\varepsilon\}\)\}\\lesssim\\operatorname\{poly\}\(d\)\(434\)for some rotation\-equivariantk∈𝒦rotk\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}satisfying Assumption[k](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt1)\. Put differently, we can construct an alternate flow which matches the endpoint map of a time\-invariant single\-index flow by paying a one\-time cost which ispoly⁡\(d\)\\operatorname\{poly\}\(d\)and independent ofε\\varepsilon\. We demonstrate this mechanism in the simplest setting of shears, i\.e\. single\-index flows with⟨u,θ⟩=0\\left\\langle u,\\theta\\right\\rangle=0:

###### Proposition 7\(Nonperturbative shortcuts in rotation\-equivariant RKHS\)\.

Letd≥2d\\geq 2andϕ∈Cb∞​\(ℝ\)\\phi\\in C^\{\\infty\}\_\{b\}\(\{\\mathbb\{R\}\}\)\. FixQ,R∈\(0,∞\)Q,R\\in\(0,\\infty\)and denote

β:=1\+log⁡\(1\+Q\)\+d​log⁡\(d\+2\+R\+Q​‖ϕ‖∞\)\+log⁡\(1\+‖ϕ‖C⌈d/2⌉\)\.\\beta:=1\+\\log\(1\+Q\)\+d\\log\(d\+2\+R\+Q\\\|\\phi\\\|\_\{\\infty\}\)\+\\log\(1\+\\\|\\phi\\\|\_\{C^\{\\lceil d/2\\rceil\}\}\)\.\(435\)For everyρ∈𝒫⁡\(ℝd\)\\rho\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)we also denote itsLdL^\{d\}\-radius by

radd⁡\(ρ\):=\(∫ℝd‖x‖d​𝑑ρ​\(x\)\)1/d\.\\operatorname\{rad\}\_\{d\}\(\\rho\):=\\left\(\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|x\\\|^\{d\}d\\rho\(x\)\\right\)^\{1/d\}\.\(436\)Then, there exists a smooth rotation\-equivariant positive kernelkd,ϕ,Q,Rk\_\{d,\\phi,Q,R\}with the following property:

For everyu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}with⟨u,θ⟩=0\\left\\langle u,\\theta\\right\\rangle=0, every\|q\|≤Q\|q\|\\leq Q, and everyρ0∈𝒫⁡\(ℝd\)\\rho\_\{0\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)satisfyingradd⁡\(ρ0\)≤R\\operatorname\{rad\}\_\{d\}\(\\rho\_\{0\}\)\\leq R, there is a family of drifts\(vt\)t∈\[0,1\]⊆ℋkd,ϕ,Q,R\(v\_\{t\}\)\_\{t\\in\[0,1\]\}\\subseteq\\mathcal\{H\}\_\{k\_\{d,\\phi,Q,R\}\}s\.t\. the following hold:

1. \(i\)Eachvtv\_\{t\}is globally\-Lipschitz with supt∈\[0,1\]Lip⁡\(vt\)≤12​β\.\\sup\_\{t\\in\[0,1\]\}\\operatorname\{Lip\}\(v\_\{t\}\)\\leq 12\\beta\.\(437\)These drifts are in three phases, i\.e\.t↦vtt\\mapsto v\_\{t\}is piecewise\-constant with three pieces\.
2. \(ii\)If\(Φt\)t∈\[0,1\]\(\\Phi\_\{t\}\)\_\{t\\in\[0,1\]\}denote the flow maps generated by\(vt\)t∈\[0,1\]\(v\_\{t\}\)\_\{t\\in\[0,1\]\}, then Φ1​\(x\)=x\+q​u​ϕ​\(⟨x,θ⟩\)\.\\Phi\_\{1\}\(x\)=x\+qu\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\.\(438\)
3. \(iii\)Ifρt=\(Φt\)\#​ρ0\\rho\_\{t\}=\(\\Phi\_\{t\}\)\_\{\\\#\}\\rho\_\{0\}denote the marginals along the flow \(and so\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}satisfy the continuity equation\), then supt∈\[0,1\]radd⁡\(ρt\)≤R\+\|q\|​‖ϕ‖∞\.\\sup\_\{t\\in\[0,1\]\}\\operatorname\{rad\}\_\{d\}\(\\rho\_\{t\}\)\\leq R\+\|q\|\\\|\\phi\\\|\_\{\\infty\}\.\(439\)
4. \(iv\)There is the trace bound along the flow supt∈\[0,1\]∫ℝdtr⁡kd,ϕ,Q,R​\(x,x\)​d​ρt​\(x\)≤1\+d​\(R\+\|q\|​‖ϕ‖∞\)2\.\\sup\_\{t\\in\[0,1\]\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\_\{d,\\phi,Q,R\}\(x,x\)d\\rho\_\{t\}\(x\)\\leq 1\+d\(R\+\|q\|\\\|\\phi\\\|\_\{\\infty\}\)^\{2\}\.\(440\)
5. \(v\)The RKHS length of this transport satisfies ∫01‖vt‖ℋkd,ϕ,Q,R​𝑑t≤8​β\.\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{d,\\phi,Q,R\}\}\}dt\\leq 8\\beta\.\(441\)

In particular, if‖ϕ‖∞,log⁡\(1\+‖ϕ‖C⌈d/2⌉\)≲poly⁡\(d\)\\\|\\phi\\\|\_\{\\infty\},\\log\(1\+\\\|\\phi\\\|\_\{C^\{\\lceil d/2\\rceil\}\}\)\\lesssim\\operatorname\{poly\}\(d\)then there is a smooth rotation\-equivariantkd,ϕ∈𝒦rotk\_\{d,\\phi\}\\in\\mathcal\{K\}\_\{\\operatorname\{rot\}\}s\.t\. for anyT≤d10000T\\leq d^\{10000\}, anyρ0∈𝒫⁡\(ℝd\)\\rho\_\{0\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\)withradd⁡\(ρ0\)≤d10000\\operatorname\{rad\}\_\{d\}\(\\rho\_\{0\}\)\\leq d^\{10000\}, and any single\-index shear

νT:=Law⁡\(XT\),X˙t=u​ϕ​\(⟨Xt,θ⟩\),X0∼ρ0,⟨u,θ⟩=0,\\nu\_\{T\}:=\\operatorname\{Law\}\(X\_\{T\}\),\\quad\\dot\{X\}\_\{t\}=u\\phi\(\\left\\langle X\_\{t\},\\theta\\right\\rangle\),\\quad X\_\{0\}\\sim\\rho\_\{0\},\\quad\\left\\langle u,\\theta\\right\\rangle=0,\(442\)there exists a transport\(vt,ρt\)t∈\[0,1\]\(v\_\{t\},\\rho\_\{t\}\)\_\{t\\in\[0,1\]\}fromρ0\\rho\_\{0\}toνT\\nu\_\{T\}using three phases ofpoly⁡\(d\)\\operatorname\{poly\}\(d\)\-Lipschitz drifts with an RKHS cost of

supt∈\[0,1\]∫ℝdtr⁡kd,ϕ​\(x,x\)​d​ρt​\(x\)⋅∫01‖vt‖ℋkd,ϕ​𝑑t≲poly⁡\(d\)\.\\sup\_\{t\\in\[0,1\]\}\\sqrt\{\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}k\_\{d,\\phi\}\(x,x\)d\\rho\_\{t\}\(x\)\}\\cdot\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{d,\\phi\}\}\}dt\\lesssim\\operatorname\{poly\}\(d\)\.\(443\)

###### Proof\.

For notation, we define

m:=⌈d2⌉,‖ϕ‖Cm:=max0≤j≤m⁡‖ϕ\(j\)‖∞,Cd:=\(1\+d⁡\(d−1\)2​m2\)m≤d4​d\.m:=\\left\\lceil\\frac\{d\}\{2\}\\right\\rceil,\\quad\\\|\\phi\\\|\_\{C^\{m\}\}:=\\max\_\{0\\leq j\\leq m\}\\\|\\phi^\{\(j\)\}\\\|\_\{\\infty\},\\quad C\_\{d\}:=\\left\(1\+\\frac\{d\(d\-1\)\}\{2\}m^\{2\}\\right\)^\{m\}\\leq d^\{4d\}\.\(444\)LetHd/2​\(𝕊d−1\)H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)be the spectral Sobolev space of indexd/2d/2w\.r\.t\. the normalized uniform measureσ\\sigmaon𝕊d−1\\mathbb\{S\}^\{d\-1\}\. More precisely, if\{Yn,j:1≤j≤Nn,d,n∈ℕ∪\{0\}\}\\\{Y\_\{n,j\}:\\;1\\leq j\\leq N\_\{n,d\},\\;n\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\\\}is the orthonormal basis ofL2​\(σ\)L^\{2\}\(\\sigma\)composed of spherical harmonics \(withNn,d=dim𝕐ndN\_\{n,d\}=\\dim\\mathbb\{Y\}\_\{n\}^\{d\}, see \([218](https://arxiv.org/html/2610.10875#A1.E218)\)\), then

−Δ𝕊d−1​Yn,j=λn​Yn,j,λn:=n⁡\(n\+d−2\)\-\\Delta\_\{\\mathbb\{S\}^\{d\-1\}\}Y\_\{n,j\}=\\lambda\_\{n\}Y\_\{n,j\},\\quad\\lambda\_\{n\}:=n\(n\+d\-2\)\(445\)and we may identify the Sobolev norm by

‖f‖Hd/2​\(𝕊d−1\)2=∑n=0∞∑j=1Nn,d\(1\+λn\)d/2​\|⟨f,Yn,j⟩L2​\(σ\)\|2\.\\\|f\\\|\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\}^\{2\}=\\sum\_\{n=0\}^\{\\infty\}\\sum\_\{j=1\}^\{N\_\{n,d\}\}\(1\+\\lambda\_\{n\}\)^\{d/2\}\|\\left\\langle f,Y\_\{n,j\}\\right\\rangle\_\{L^\{2\}\(\\sigma\)\}\|^\{2\}\.\(446\)By the addition formula\[[6](https://arxiv.org/html/2610.10875#bib.bib62), Theorem 2\.9\]we know∑j=1Nn,d\|Yn,j​\(η\)\|2=Nn,d\\sum\_\{j=1\}^\{N\_\{n,d\}\}\|Y\_\{n,j\}\(\\eta\)\|^\{2\}=N\_\{n,d\}for everyη∈𝕊d−1\\eta\\in\\mathbb\{S\}^\{d\-1\}, and so the squared norm of point evaluationf↦f⁡\(η\)f\\mapsto f\(\\eta\)onHd/2​\(𝕊d−1\)H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)is

Ed=∑n=0∞Nn,d\(1\+λn\)d/2≤1\+∑n=1∞2​dd−2​nd−2nd=1\+π23​dd−2≤2​dd,E\_\{d\}=\\sum\_\{n=0\}^\{\\infty\}\\frac\{N\_\{n,d\}\}\{\(1\+\\lambda\_\{n\}\)^\{d/2\}\}\\leq 1\+\\sum\_\{n=1\}^\{\\infty\}\\frac\{2d^\{d\-2\}n^\{d\-2\}\}\{n^\{d\}\}=1\+\\frac\{\\pi^\{2\}\}\{3\}d^\{d\-2\}\\leq 2d^\{d\},\(447\)where we used thatNn,d≤2​\(n\+d−1\)d−2≤2​dd−2​nd−2N\_\{n,d\}\\leq 2\(n\+d\-1\)^\{d\-2\}\\leq 2d^\{d\-2\}n^\{d\-2\}and1\+λn≥n21\+\\lambda\_\{n\}\\geq n^\{2\}forn≥1n\\geq 1andd≥2d\\geq 2\.

Now, for eachx∈ℝdx\\in\{\\mathbb\{R\}\}^\{d\}we define

Ψx​\(ω\):=ϕ⁡\(⟨x,ω⟩\)\(∀ω∈𝕊d−1\)\\Psi\_\{x\}\(\\omega\):=\\phi\(\\left\\langle x,\\omega\\right\\rangle\)\\quad\(\\forall\\omega\\in\\mathbb\{S\}^\{d\-1\}\)\(448\)and we claim the Sobolev bound

‖Ψx‖Hd/2​\(𝕊d−1\)2≤Cd​\(1\+‖ϕ‖Cm\)2​\(1\+‖x‖\)d\.\\\|\\Psi\_\{x\}\\\|\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\}^\{2\}\\leq C\_\{d\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+\\\|x\\\|\)^\{d\}\.\(449\)To show \([449](https://arxiv.org/html/2610.10875#A1.E449)\), we decomposeΔ𝕊d−1=∑1≤i<j≤dXi​j2\\Delta\_\{\\mathbb\{S\}^\{d\-1\}\}=\\sum\_\{1\\leq i<j\\leq d\}X\_\{ij\}^\{2\}for the rotation fieldsXi​j=ωi∂ωj−ωj∂ωiX\_\{ij\}=\\omega\_\{i\}\\partial\_\{\\omega\_\{j\}\}\-\\omega\_\{j\}\\partial\_\{\\omega\_\{i\}\}\. SinceΔ𝕊d−1\\Delta\_\{\\mathbb\{S\}^\{d\-1\}\}commutes with everyXi​jX\_\{ij\}, we can expand\(𝟙−Δ𝕊d−1\)m\(\{\\mathbbm\{1\}\}\-\\Delta\_\{\\mathbb\{S\}^\{d\-1\}\}\)^\{m\}to get

∥f∥Hm​\(𝕊d−1\)2=∥f∥L2​\(σ\)2\+∑a=1m\(ma\)∑1≤i1<j1≤d⋯∑1≤ia<ja≤d∥Xi1,j1⋯Xia,jaf∥L2​\(σ\)2\.\\\|f\\\|\_\{H^\{m\}\(\\mathbb\{S\}^\{d\-1\}\)\}^\{2\}=\\\|f\\\|\_\{L^\{2\}\(\\sigma\)\}^\{2\}\+\\sum\_\{a=1\}^\{m\}\{m\\choose a\}\\sum\_\{1\\leq i\_\{1\}<j\_\{1\}\\leq d\}\\cdots\\sum\_\{1\\leq i\_\{a\}<j\_\{a\}\\leq d\}\\\|X\_\{i\_\{1\},j\_\{1\}\}\\cdots X\_\{i\_\{a\},j\_\{a\}\}f\\\|\_\{L^\{2\}\(\\sigma\)\}^\{2\}\.\(450\)We have‖Ψx‖L2​\(σ\)2≤‖ϕ‖∞2\\\|\\Psi\_\{x\}\\\|\_\{L^\{2\}\(\\sigma\)\}^\{2\}\\leq\\\|\\phi\\\|\_\{\\infty\}^\{2\}and by the chain rule we know that

∥Xi1,j1⋯Xia,jaΨx∥∞≤aa\(1\+∥ϕ∥Cm\)\(1\+∥x∥\)a,\\\|X\_\{i\_\{1\},j\_\{1\}\}\\cdots X\_\{i\_\{a\},j\_\{a\}\}\\Psi\_\{x\}\\\|\_\{\\infty\}\\leq a^\{a\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)\(1\+\\\|x\\\|\)^\{a\},\(451\)since there are at mostaaa^\{a\}partitions of\{1,…,a\}\\\{1,\\ldots,a\\\}and we bound the factors⟨x,Xi1,j1⋯Xia,jaω⟩\\left\\langle x,X\_\{i\_\{1\},j\_\{1\}\}\\cdots X\_\{i\_\{a\},j\_\{a\}\}\\omega\\right\\rangleby‖x‖\\\|x\\\|\. Since there are\(d⁡\(d−1\)/2\)a\(d\(d\-1\)/2\)^\{a\}such terms, we find that

‖Ψx‖Hm​\(𝕊d−1\)2\\displaystyle\\\|\\Psi\_\{x\}\\\|\_\{H^\{m\}\(\\mathbb\{S\}^\{d\-1\}\)\}^\{2\}≤\(1\+‖ϕ‖Cm\)2​\(1\+∑a=1m\(ma\)​\(d⁡\(d−1\)2\)a​a2​a​\(1\+‖x‖\)2​a\)\\displaystyle\\leq\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\\left\(1\+\\sum\_\{a=1\}^\{m\}\{m\\choose a\}\\left\(\\frac\{d\(d\-1\)\}\{2\}\\right\)^\{a\}a^\{2a\}\(1\+\\\|x\\\|\)^\{2a\}\\right\)\(452\)≤\(1\+‖ϕ‖Cm\)2​\(1\+‖x‖\)2​m​\(1\+∑a=1m\(ma\)​\(m2​d⁡\(d−1\)2\)a\)\\displaystyle\\leq\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+\\\|x\\\|\)^\{2m\}\\left\(1\+\\sum\_\{a=1\}^\{m\}\{m\\choose a\}\\left\(m^\{2\}\\frac\{d\(d\-1\)\}\{2\}\\right\)^\{a\}\\right\)\(453\)=\(1\+‖ϕ‖Cm\)2​\(1\+‖x‖\)2​m​\(1\+m2​d⁡\(d−1\)2\)m\\displaystyle=\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+\\\|x\\\|\)^\{2m\}\\left\(1\+m^\{2\}\\frac\{d\(d\-1\)\}\{2\}\\right\)^\{m\}\(454\)=Cd​\(1\+‖ϕ‖Cm\)2​\(1\+‖x‖\)2​m\.\\displaystyle=C\_\{d\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+\\\|x\\\|\)^\{2m\}\.\(455\)Denotingν:=d2​m∈\(0,1\]\\nu:=\\frac\{d\}\{2m\}\\in\(0,1\]and using the interpolation inequality

‖f‖Hd/2​\(𝕊d−1\)≤‖f‖L2​\(σ\)1−ν​‖f‖Hm​\(𝕊d−1\)ν,\\\|f\\\|\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\}\\leq\\\|f\\\|\_\{L^\{2\}\(\\sigma\)\}^\{1\-\\nu\}\\\|f\\\|\_\{H^\{m\}\(\\mathbb\{S\}^\{d\-1\}\)\}^\{\\nu\},\(456\)we obtain

‖Ψx‖Hd/2​\(𝕊d−1\)2\\displaystyle\\\|\\Psi\_\{x\}\\\|\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\}^\{2\}≤\(1\+‖ϕ‖Cm\)2​\(1−ν\)​\[Cd​\(1\+‖ϕ‖Cm\)2​\(1\+‖x‖\)2​m\]ν\\displaystyle\\leq\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\(1\-\\nu\)\}\\left\[C\_\{d\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+\\\|x\\\|\)^\{2m\}\\right\]^\{\\nu\}\(457\)=Cdν​\(1\+‖ϕ‖Cm\)2​\(1\+‖x‖\)2​m​ν\.\\displaystyle=C\_\{d\}^\{\\nu\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+\\\|x\\\|\)^\{2m\\nu\}\.\(458\)Since2​m​ν=d2m\\nu=dandCd≥1C\_\{d\}\\geq 1, we get \([449](https://arxiv.org/html/2610.10875#A1.E449)\)\.

Using these ingredients, we define the scalar kernelκϕ:ℝd×ℝd→ℝ\\kappa\_\{\\phi\}:\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}by

κϕ​\(x,y\):=⟨Ψx,Ψy⟩Hd/2​\(𝕊d−1\)\.\\kappa\_\{\\phi\}\(x,y\):=\\left\\langle\\Psi\_\{x\},\\Psi\_\{y\}\\right\\rangle\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\}\.\(459\)This is positive semidefinite and sinceΨU​x​\(ω\)=Ψx​\(U⊤​ω\)\\Psi\_\{Ux\}\(\\omega\)=\\Psi\_\{x\}\(U^\{\\top\}\\omega\)it is rotation\-invariant\. Also, since∂xαΨx​\(ω\)=ωα​ϕ\(\|α\|\)​\(⟨x,ω⟩\)\\partial\_\{x\}^\{\\alpha\}\\Psi\_\{x\}\(\\omega\)=\\omega^\{\\alpha\}\\phi^\{\(\|\\alpha\|\)\}\(\\left\\langle x,\\omega\\right\\rangle\)the same Sobolev estimate showsx↦Ψxx\\mapsto\\Psi\_\{x\}is smooth as anHd/2H^\{d/2\}\-valued map; thus,κϕ\\kappa\_\{\\phi\}is smooth\. For anyη∈𝕊d−1\\eta\\in\\mathbb\{S\}^\{d\-1\}, we know that

ϕ⁡\(⟨x,η⟩\)=Ψx​\(η\)=⟨Ψx,eη⟩Hd/2​\(𝕊d−1\),\\phi\(\\left\\langle x,\\eta\\right\\rangle\)=\\Psi\_\{x\}\(\\eta\)=\\left\\langle\\Psi\_\{x\},e\_\{\\eta\}\\right\\rangle\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\},\(460\)and so by the feature map construction ofκϕ\\kappa\_\{\\phi\}we have

‖ϕ⁡\(⟨⋅,η⟩\)‖ℋκϕ2≤‖eη‖Hd/2​\(𝕊d−1\)2=Ed≤2​dd\.\\\|\\phi\(\\left\\langle\\cdot,\\eta\\right\\rangle\)\\\|\_\{\\mathcal\{H\}\_\{\\kappa\_\{\\phi\}\}\}^\{2\}\\leq\\\|e\_\{\\eta\}\\\|^\{2\}\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\}=E\_\{d\}\\leq 2d^\{d\}\.\(461\)Now, we setM:=\(1\+‖ϕ‖Cm\)2​\(1\+2​dd\+1​Cd​\(1\+R\+Q​‖ϕ‖∞\)d\)M:=\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\\left\(1\+2d^\{d\+1\}C\_\{d\}\(1\+R\+Q\\\|\\phi\\\|\_\{\\infty\}\)^\{d\}\\right\)and define the kernel

kd,ϕ,Q,R​\(x,y\):=⟨x,y⟩​𝟙d\+2​dd1\+M​κϕ​\(x,y\)​𝟙d\.k\_\{d,\\phi,Q,R\}\(x,y\):=\\left\\langle x,y\\right\\rangle\{\\mathbbm\{1\}\}\_\{d\}\+\\frac\{2d^\{d\}\}\{1\+M\}\\kappa\_\{\\phi\}\(x,y\)\{\\mathbbm\{1\}\}\_\{d\}\.\(462\)This kernel is smooth, positive semidefinite, and rotation\-equivariant\. Furthermore, by construction we knowx↦u​⟨x,u⟩x\\mapsto u\\left\\langle x,u\\right\\ranglehasℋkd,ϕ,Q,R\\mathcal\{H\}\_\{k\_\{d,\\phi,Q,R\}\}norm at most 1 whilex↦u​ϕ​\(⟨x,θ⟩\)x\\mapsto u\\phi\(\\left\\langle x,\\theta\\right\\rangle\)hasℋkd,ϕ,Q,R\\mathcal\{H\}\_\{k\_\{d,\\phi,Q,R\}\}norm≤1\+M\\leq\\sqrt\{1\+M\}\. Hence, if we definer:=max⁡\{1,\|q\|​1\+M2\}r:=\\max\\left\\\{1,\\;\\frac\{\|q\|\\sqrt\{1\+M\}\}\{2\}\\right\\\}and

vt​\(x\):=\{−3​log⁡r​u​⟨x,u⟩0≤t<133​qr​u​ϕ​\(⟨x,θ⟩\)13≤t<23,3​log⁡r​u​⟨x,u⟩23≤t≤1\.v\_\{t\}\(x\):=\\begin\{cases\}\-3\\log ru\\left\\langle x,u\\right\\rangle&0\\leq t<\\frac\{1\}\{3\}\\\\ \\frac\{3q\}\{r\}u\\phi\(\\left\\langle x,\\theta\\right\\rangle\)&\\frac\{1\}\{3\}\\leq t<\\frac\{2\}\{3\},\\\\ 3\\log ru\\left\\langle x,u\\right\\rangle&\\frac\{2\}\{3\}\\leq t\\leq 1\.\\end\{cases\}\(463\)then it holds that

∫01‖vt‖ℋkd,ϕ,Q,R​𝑑t≤2​log⁡r\+\|q\|​1\+Mr≤2\+2​log⁡\(1\+\|q\|​1\+M\),\\int\_\{0\}^\{1\}\\\|v\_\{t\}\\\|\_\{\\mathcal\{H\}\_\{k\_\{d,\\phi,Q,R\}\}\}dt\\leq 2\\log r\+\\frac\{\|q\|\\sqrt\{1\+M\}\}\{r\}\\leq 2\+2\\log\(1\+\|q\|\\sqrt\{1\+M\}\),\(464\)where the second inequality follows from definition ofrr\. Next, since1\+M≥‖ϕ‖Cm≥‖ϕ′‖∞\\sqrt\{1\+M\}\\geq\\\|\\phi\\\|\_\{C^\{m\}\}\\geq\\\|\\phi^\{\\prime\}\\\|\_\{\\infty\}we know thatLip⁡\(3​qr​u​ϕ​\(⟨⋅,θ⟩\)\)≤6\\operatorname\{Lip\}\(\\frac\{3q\}\{r\}u\\phi\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\)\\leq 6and so

supt∈\[0,1\]Lip⁡\(vt\)≤6\+3​log⁡\(1\+\|q\|​1\+M\)\.\\sup\_\{t\\in\[0,1\]\}\\operatorname\{Lip\}\(v\_\{t\}\)\\leq 6\+3\\log\(1\+\|q\|\\sqrt\{1\+M\}\)\.\(465\)By definition ofMMandCd≤d4​dC\_\{d\}\\leq d^\{4d\}we know that

1\+M≤2​\(1\+‖ϕ‖Cm\)​d\(5​d\+1\)/2​\(1\+R\+Q​‖ϕ‖∞\)d/2\\sqrt\{1\+M\}\\leq 2\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)d^\{\(5d\+1\)/2\}\(1\+R\+Q\\\|\\phi\\\|\_\{\\infty\}\)^\{d/2\}\(466\)from which we can bound

log⁡\(1\+\|q\|​1\+M\)\\displaystyle\\log\(1\+\|q\|\\sqrt\{1\+M\}\)≤1\+log⁡\(1\+\|q\|\)\+log⁡\(1\+‖ϕ‖Cm\)\+4​d​log⁡\(d\+2\+R\+Q​‖ϕ‖∞\)\.\\displaystyle\\leq 1\+\\log\(1\+\|q\|\)\+\\log\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)\+4d\\log\(d\+2\+R\+Q\\\|\\phi\\\|\_\{\\infty\}\)\.\(467\)Combining this with \([464](https://arxiv.org/html/2610.10875#A1.E464)\) and \([465](https://arxiv.org/html/2610.10875#A1.E465)\) yields \(v\) and \(i\), respectively\. Looking back at \([463](https://arxiv.org/html/2610.10875#A1.E463)\), we see that the first phase multiplies theuucoordinate by1/r1/r\. The second phase addsqr​u​ϕ​\(⟨x,θ⟩\)\\frac\{q\}\{r\}u\\phi\(\\left\\langle x,\\theta\\right\\rangle\)and preserves⟨x,θ⟩\\left\\langle x,\\theta\\right\\ranglesinceu⟂θu\\perp\\theta\. Lastly, the third phase multiplies the resultinguucoordinate byrr, which proves \(ii\)\.

It remains to prove the moment and trace estimates\. Write

x=x⟂\+⟨x,u⟩​u,⟨x⟂,u⟩=0\.x=x\_\{\\perp\}\+\\left\\langle x,u\\right\\rangle u,\\quad\\left\\langle x\_\{\\perp\},u\\right\\rangle=0\.\(468\)During the first phase, the absolute value of theuucoordinate decreases whilex⟂x\_\{\\perp\}is unchanged\. During the middle phase, the displacement has magnitude at most\|q\|r​‖ϕ‖∞\\frac\{\|q\|\}\{r\}\\\|\\phi\\\|\_\{\\infty\}\. At the beginning of the last phase, theuucoordinate equals

⟨x,u⟩\+q​ϕ​\(⟨x,θ⟩\)r\.\\frac\{\\left\\langle x,u\\right\\rangle\+q\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\}\{r\}\.\(469\)During the final phase this coordinate is multiplied by a factor which starts at 1 and ends atrr, and so its magnitude never exceeds\|⟨x,u⟩\+q​ϕ​\(⟨x,θ⟩\)\|\|\\left\\langle x,u\\right\\rangle\+q\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\|\. Thus,

∥Φt\(x\)∥≤∥x∥\+\|q\|∥ϕ∥∞\(∀t∈\[0,1\],∀x∈ℝd\)\.\\\|\\Phi\_\{t\}\(x\)\\\|\\leq\\\|x\\\|\+\|q\|\\\|\\phi\\\|\_\{\\infty\}\\quad\(\\forall t\\in\[0,1\],\\;\\forall x\\in\{\\mathbb\{R\}\}^\{d\}\)\.\(470\)Applying the triangle inequality inLd​\(ρ0\)L^\{d\}\(\\rho\_\{0\}\)impliesradd⁡\(ρt\)≤R\+\|q\|​‖ϕ‖∞\\operatorname\{rad\}\_\{d\}\(\\rho\_\{t\}\)\\leq R\+\|q\|\\\|\\phi\\\|\_\{\\infty\}, proving \(iii\)\. To finish, for the linear part of the kernel we know

d​∫ℝd‖x‖2​d​ρt​\(x\)≤d​radd​\(ρt\)2≤d​\(R\+\|q\|​‖ϕ‖∞\)2\.d\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|x\\\|^\{2\}d\\rho\_\{t\}\(x\)\\leq d\\operatorname\{rad\}\_\{d\}\(\\rho\_\{t\}\)^\{2\}\\leq d\(R\+\|q\|\\\|\\phi\\\|\_\{\\infty\}\)^\{2\}\.\(471\)For the nonlinear part, we have that

∫ℝdκϕ​\(x,x\)​d​ρt​\(x\)\\displaystyle\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\kappa\_\{\\phi\}\(x,x\)d\\rho\_\{t\}\(x\)=∫ℝd‖Ψx‖Hd/2​\(𝕊d−1\)2\\displaystyle=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|\\Psi\_\{x\}\\\|^\{2\}\_\{H^\{d/2\}\(\\mathbb\{S\}^\{d\-1\}\)\}\(472\)≤Cd​\(1\+‖ϕ‖Cm\)2​∫ℝd\(1\+‖x‖\)d​d​ρt​\(x\)\\displaystyle\\leq C\_\{d\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\(1\+\\\|x\\\|\)^\{d\}d\\rho\_\{t\}\(x\)\(473\)≤Cd​\(1\+‖ϕ‖Cm\)2​\(1\+radd⁡\(ρt\)\)d\\displaystyle\\leq C\_\{d\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+\\operatorname\{rad\}\_\{d\}\(\\rho\_\{t\}\)\)^\{d\}\(474\)≤Cd​\(1\+‖ϕ‖Cm\)2​\(1\+R\+Q​‖ϕ‖∞\)d\\displaystyle\\leq C\_\{d\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+R\+Q\\\|\\phi\\\|\_\{\\infty\}\)^\{d\}\(475\)which implies that

2​dd1\+M​∫ℝdtr⁡\(κϕ​\(x,x\)​𝟙d\)​d​ρt​\(x\)≤2​dd\+1​Cd​\(1\+‖ϕ‖Cm\)2​\(1\+R\+Q​‖ϕ‖∞\)d1\+M≤1\\displaystyle\\frac\{2d^\{d\}\}\{1\+M\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\operatorname\{tr\}\(\\kappa\_\{\\phi\}\(x,x\)\{\\mathbbm\{1\}\}\_\{d\}\)d\\rho\_\{t\}\(x\)\\leq\\frac\{2d^\{d\+1\}C\_\{d\}\(1\+\\\|\\phi\\\|\_\{C^\{m\}\}\)^\{2\}\(1\+R\+Q\\\|\\phi\\\|\_\{\\infty\}\)^\{d\}\}\{1\+M\}\\leq 1\(476\)by definition ofMM\. Combining \([471](https://arxiv.org/html/2610.10875#A1.E471)\) and \([476](https://arxiv.org/html/2610.10875#A1.E476)\) yields \(iv\)\.

The final “in particular” follows by takingkd,ϕ:=kd,ϕ,d10000,d10000k\_\{d,\\phi\}:=k\_\{d,\\phi,d^\{10000,d^\{10000\}\}\}, observing thatdd​t​⟨Xt,θ⟩=0\\frac\{d\}\{dt\}\\left\\langle X\_\{t\},\\theta\\right\\rangle=0and soXT=X0\+T​u​ϕ​\(⟨X0,θ⟩\)X\_\{T\}=X\_\{0\}\+Tu\\phi\(\\left\\langle X\_\{0\},\\theta\\right\\rangle\), and applying the result withq=Tq=T\. ∎

### A\.6Reverse OU \(sub\)optimality for single\-index flows and tilts

We first show the flow lower bound\. The Barron norm will turn out to be\+∞\+\\inftyfor the examples we consider, but for now we define an abstract quantity which captures this lower bound\.

###### Lemma 14\(Single\-index flows\)\.

Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be globally\-Lipschitz, nonzero, and satisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)\. Letu,θ∈𝕊d−1u,\\theta\\in\\mathbb\{S\}^\{d\-1\}andσ∈\(0,∞\)\\sigma\\in\(0,\\infty\), and for eachε\>0\\varepsilon\>0define

πεflow:=Law⁡\(X1\),X˙t=ε​σ​u​ϕ​\(⟨Xt,θ⟩\),X0∼γd\.\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}:=\\operatorname\{Law\}\(X\_\{1\}\),\\quad\\dot\{X\}\_\{t\}=\\varepsilon\\sigma u\\phi\(\\left\\langle X\_\{t\},\\theta\\right\\rangle\),\\quad X\_\{0\}\\sim\\gamma\_\{d\}\.\(477\)Then, it holds that

lim infε↓0ℰℬϕ​\(πεflow\)Dℬϕ​\(γd,πεflow\)≥𝔉ϕflow​\(c\)2\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\\left\(\{\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}\}\\right\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}\)\}\\geq\\frac\{\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}\(c\)\}\{2\}\(478\)wherec:=⟨u,θ⟩c:=\\left\\langle u,\\theta\\right\\rangleand we define the quantity

𝔉ϕflow​\(c\):=∫01‖u​Qr​\(⟨⋅,θ⟩\)\+θ⁡⟨⋅,u⟩​Qr′​\(⟨⋅,θ⟩\)−c​θ​Qr′′​\(⟨⋅,θ⟩\)‖ℬϕ​𝑑r\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}\(c\):=\\int\_\{0\}^\{1\}\\left\\\|uQ\_\{r\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\+\\theta\\left\\langle\\cdot,u\\right\\rangle Q\_\{r\}^\{\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\-c\\theta Q\_\{r\}^\{\\prime\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\\right\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dr\(479\)for the function

Qr​\(t\):=𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ⁡\(r​t\+1−r2​ξ\)\]\.Q\_\{r\}\(t\):=\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\phi\\left\(rt\+\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]\.\(480\)

###### Proof\.

Let us denote

w⁡\(x\):=σ​u​ϕ​\(⟨x,θ⟩\),fε:=πεflowγd,c:=⟨u,θ⟩,w\(x\):=\\sigma u\\phi\(\\left\\langle x,\\theta\\right\\rangle\),\\quad f\_\{\\varepsilon\}:=\\frac\{\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}\}\{\\gamma\_\{d\}\},\\quad c:=\\left\\langle u,\\theta\\right\\rangle,\(481\)where we know thatπεflow\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}is absolutely continuous since the flow generated bywwis bi\-Lipschitz \(due to Lipschitzness ofww\)\. By the continuity equation, for eachφ∈Cc∞​\(ℝd\)\\varphi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{d\}\)we have

dd​ε\|ε=0​∫ℝdφ​d​πεflow=∫ℝd⟨∇φ,w⟩​d​γd=∫ℝdφ​h​d​γd,\\left\.\\frac\{d\}\{d\\varepsilon\}\\right\|\_\{\\varepsilon=0\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi d\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle\\nabla\\varphi,w\\right\\rangle d\\gamma\_\{d\}=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\varphi hd\\gamma\_\{d\},\(482\)where we use Gaussian integration by parts and denote the function

h⁡\(x\):=⟨x,w⁡\(x\)⟩−div⁡w⁡\(x\)=σ⁡\(⟨x,u⟩​ϕ​\(⟨x,θ⟩\)−c​ϕ′​\(⟨x,θ⟩\)\)\.h\(x\):=\\left\\langle x,w\(x\)\\right\\rangle\-\\operatorname\{div\}w\(x\)=\\sigma\\left\(\\left\\langle x,u\\right\\rangle\\phi\(\\left\\langle x,\\theta\\right\\rangle\)\-c\\phi^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\\right\)\.\(483\)Hence,fε=1\+ε​h\+o⁡\(ε\)f\_\{\\varepsilon\}=1\+\\varepsilon h\+o\(\\varepsilon\)in the sense of distributions\. For each fixedτ\>0\\tau\>0, the linear growth ofwwand OU regularization232323LettingKτK\_\{\\tau\}denote the Mehler kernel \(which is smooth with Gaussian decay of all its derivatives\) andTεT\_\{\\varepsilon\}the flow map for timeε\\varepsilon, sincePτ​fε=∫ℝdKτ​\(⋅,Tε​\(y\)\)​d​γd​\(y\)P\_\{\\tau\}f\_\{\\varepsilon\}=\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}K\_\{\\tau\}\(\\cdot,T\_\{\\varepsilon\}\(y\)\)d\\gamma\_\{d\}\(y\)andTε​\(y\)=y\+ε​w​\(y\)\+o⁡\(ε\)T\_\{\\varepsilon\}\(y\)=y\+\\varepsilon w\(y\)\+o\(\\varepsilon\), we may differentiate under the integral to get local uniform convergence of derivatives\.turn this intoPτ​fε=1\+ε​Pτ​h\+o⁡\(ε\)P\_\{\\tau\}f\_\{\\varepsilon\}=1\+\\varepsilon P\_\{\\tau\}h\+o\(\\varepsilon\)inCloc1C^\{1\}\_\{\\operatorname\{loc\}\}, or in other words

limε↓01ε∇logPτfε=∇Pτhuniformly on compact sets\.\\lim\_\{\\varepsilon\\downarrow 0\}\\frac\{1\}\{\\varepsilon\}\\nabla\\log P\_\{\\tau\}f\_\{\\varepsilon\}=\\nabla P\_\{\\tau\}h\\quad\\text\{uniformly on compact sets\}\.\(484\)By lower\-semicontinuity of the Barron norm w\.r\.t\. local uniform convergence \(Lemma[15](https://arxiv.org/html/2610.10875#Thmlemma15)\) and Fatou’s lemma, we find

lim infε↓0ℰℬϕ​\(πεflow\)ε=lim infε↓0∫0∞‖1ε∇logPτfε‖ℬϕdτ≥∫0∞∥∇Pτh∥ℬϕdτ\.\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}\)\}\{\\varepsilon\}=\\liminf\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{\\infty\}\\left\\\|\\frac\{1\}\{\\varepsilon\}\\nabla\\log P\_\{\\tau\}f\_\{\\varepsilon\}\\right\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}d\\tau\\geq\\int\_\{0\}^\{\\infty\}\\\|\\nabla P\_\{\\tau\}h\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}d\\tau\.\(485\)We now compute the right hand side\. Withr=e−τr=e^\{\-\\tau\}, by \([483](https://arxiv.org/html/2610.10875#A1.E483)\) and the Mehler formula we have

\(Pτ​h\)​\(x\)\\displaystyle\(P\_\{\\tau\}h\)\(x\)=𝔼ξ∼γd​\[h⁡\(r​x\+1−r2​ξ\)\]\\displaystyle=\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\\left\[h\\left\(rx\+\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]\(486\)=σ​r​⟨x,u⟩​𝔼​\[ϕ⁡\(r⁡⟨x,θ⟩\+1−r2​⟨ξ,θ⟩\)\]\\displaystyle=\\sigma r\\left\\langle x,u\\right\\rangle\\mathbb\{E\}\\left\[\\phi\\left\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}\\left\\langle\\xi,\\theta\\right\\rangle\\right\)\\right\]\(487\)\+σ​1−r2​𝔼​\[⟨ξ,u⟩​ϕ​\(r⁡⟨x,θ⟩\+1−r2​⟨ξ,θ⟩\)\]\\displaystyle\\quad\+\\sigma\\sqrt\{1\-r^\{2\}\}\\mathbb\{E\}\\left\[\\left\\langle\\xi,u\\right\\rangle\\phi\\left\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}\\left\\langle\\xi,\\theta\\right\\rangle\\right\)\\right\]\(488\)−σ​c​𝔼​\[ϕ′​\(r⁡⟨x,θ⟩\+1−r2​⟨ξ,θ⟩\)\]\.\\displaystyle\\quad\-\\sigma c\\mathbb\{E\}\\left\[\\phi^\{\\prime\}\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}\\left\\langle\\xi,\\theta\\right\\rangle\)\\right\]\.\(489\)Now, by Gaussian integration by parts we know

𝔼⁡\[⟨ξ,u⟩​F​\(⟨ξ,θ⟩\)\]=⟨u,θ⟩​𝔼​\[F′​\(⟨ξ,θ⟩\)\]\(∀F∈W1,2​\(γ1\)\)\.\\mathbb\{E\}\[\\left\\langle\\xi,u\\right\\rangle F\(\\left\\langle\\xi,\\theta\\right\\rangle\)\]=\\left\\langle u,\\theta\\right\\rangle\\mathbb\{E\}\[F^\{\\prime\}\(\\left\\langle\\xi,\\theta\\right\\rangle\)\]\\quad\(\\forall F\\in W^\{1,2\}\(\\gamma\_\{1\}\)\)\.\(490\)We apply this withF⁡\(z\)=ϕ⁡\(r⁡⟨x,θ⟩\+1−r2​z\)F\(z\)=\\phi\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}z\)andF′​\(z\)=1−r2​ϕ′​\(r⁡⟨x,θ⟩\+1−r2​z\)F^\{\\prime\}\(z\)=\\sqrt\{1\-r^\{2\}\}\\phi^\{\\prime\}\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}z\)to get that

𝔼⁡\[⟨ξ,u⟩​ϕ​\(r⁡⟨x,θ⟩\+1−r2​⟨ξ,θ⟩\)\]=c​1−r2​𝔼​\[ϕ′​\(r⁡⟨x,θ⟩\+1−r2​⟨ξ,θ⟩\)\],\\mathbb\{E\}\\left\[\\left\\langle\\xi,u\\right\\rangle\\phi\\left\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}\\left\\langle\\xi,\\theta\\right\\rangle\\right\)\\right\]=c\\sqrt\{1\-r^\{2\}\}\\mathbb\{E\}\\left\[\\phi^\{\\prime\}\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}\\left\\langle\\xi,\\theta\\right\\rangle\)\\right\],\(491\)which we plug into our earlier expression to find

1σ​\(Pτ​h\)​\(x\)\\displaystyle\\frac\{1\}\{\\sigma\}\(P\_\{\\tau\}h\)\(x\)=r⁡⟨x,u⟩​Qr​\(⟨x,θ⟩\)−c​r2​𝔼​\[ϕ′​\(r⁡⟨x,θ⟩\+1−r2​⟨ξ,θ⟩\)\]\\displaystyle=r\\left\\langle x,u\\right\\rangle Q\_\{r\}\(\\left\\langle x,\\theta\\right\\rangle\)\-cr^\{2\}\\mathbb\{E\}\\left\[\\phi^\{\\prime\}\(r\\left\\langle x,\\theta\\right\\rangle\+\\sqrt\{1\-r^\{2\}\}\\left\\langle\\xi,\\theta\\right\\rangle\)\\right\]\(492\)=r⁡⟨x,u⟩​Qr​\(⟨x,θ⟩\)−c​r​Qr′​\(⟨x,θ⟩\)\.\\displaystyle=r\\left\\langle x,u\\right\\rangle Q\_\{r\}\(\\left\\langle x,\\theta\\right\\rangle\)\-crQ\_\{r\}^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\.\(493\)Differentiating, we find that

\(∇Pτh\)\(x\)=σr\(uQr\(⟨x,θ⟩\)\+θ⟨x,u⟩Qr′\(⟨x,θ⟩\)−cθQr′′\(⟨x,θ⟩\)\)\(\\nabla P\_\{\\tau\}h\)\(x\)=\\sigma r\\left\(uQ\_\{r\}\(\\left\\langle x,\\theta\\right\\rangle\)\+\\theta\\left\\langle x,u\\right\\rangle Q^\{\\prime\}\_\{r\}\(\\left\\langle x,\\theta\\right\\rangle\)\-c\\theta Q\_\{r\}^\{\\prime\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\\right\)\(494\)and therefore that

∫0∞∥∇Pτh∥ℬϕdτ=σ𝔉ϕflow\(c\)\.\\int\_\{0\}^\{\\infty\}\\\|\\nabla P\_\{\\tau\}h\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}d\\tau=\\sigma\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}\(c\)\.\(495\)Since the flow definingπεflow\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}is admissible we know thatDℬϕ​\(γd,πεflow\)≤2​σ​εD\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}^\{\\operatorname\{flow\}\}\)\\leq 2\\sigma\\varepsilon\(where the factor of 2 accounts for the weight1\+‖θ‖1\+\\\|\\theta\\\|in the bounded\-activation Barron definition\), yielding the claim\. ∎

Before moving on, we fill in the claim that the Barron norm is lower semicontinuous under locally uniform convergence\. The proof uses a standard compactification trick:

###### Lemma 15\.

Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfy Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2), and let\(wn\)n∈ℕ⊆ℬϕ\(w\_\{n\}\)\_\{n\\in\{\\mathbb\{N\}\}\}\\subseteq\\mathcal\{B\}\_\{\\phi\}andw∈Cloc​\(ℝd,ℝd\)w\\in C\_\{\\operatorname\{loc\}\}\(\{\\mathbb\{R\}\}^\{d\};\{\\mathbb\{R\}\}^\{d\}\)be s\.t\.wn→ww\_\{n\}\\to wuniformly on compact sets\. Then,

‖w‖ℬϕ≤lim infn→∞‖wn‖ℬϕ\.\\\|w\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\liminf\_\{n\\to\\infty\}\\\|w\_\{n\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\.\(496\)

###### Proof\.

SetΘ^:=ℝd×ℝ\\hat\{\\Theta\}:=\{\\mathbb\{R\}\}^\{d\}\\times\{\\mathbb\{R\}\}to be the parameter space\. Ifϕ\\phiis bounded, letΘ:=Θ^∪\{∞\}\\Theta:=\\hat\{\\Theta\}\\cup\\\{\\infty\\\}be its one\-point compactification and define

ψa,b​\(x\):=ϕ⁡\(⟨a,x⟩\+b\)1\+‖a‖\+\|b\|,ψ∞​\(x\):=0\.\\psi\_\{a,b\}\(x\):=\\frac\{\\phi\(\\left\\langle a,x\\right\\rangle\+b\)\}\{1\+\\\|a\\\|\+\|b\|\},\\quad\\psi\_\{\\infty\}\(x\):=0\.\(497\)The mapθ↦ψθ\\theta\\mapsto\\psi\_\{\\theta\}is continuous fromΘ\\ThetaintoCloc​\(ℝd\)C\_\{\\operatorname\{loc\}\}\(\{\\mathbb\{R\}\}^\{d\}\): forθ∈Θ^\\theta\\in\\hat\{\\Theta\}we get continuity atθ\\thetafrom continuity ofϕ\\phi, while for continuity atθ=∞\\theta=\\inftywe use that for each compactKKwe have

supx∈K\|ψa,b​\(x\)\|≤‖ϕ‖∞1\+‖a‖\+\|b\|→0\\sup\_\{x\\in K\}\|\\psi\_\{a,b\}\(x\)\|\\leq\\frac\{\\\|\\phi\\\|\_\{\\infty\}\}\{1\+\\\|a\\\|\+\|b\|\}\\to 0\(498\)as‖a‖\+\|b\|→∞\\\|a\\\|\+\|b\|\\to\\infty\. Similarly, ifϕ\\phiis positivelyss\-homogeneous we define

Θ:=\{\(a,b\)∈Θ^:‖a‖\+\|b\|=1\},ψa,b​\(x\):=ϕ⁡\(⟨a,x⟩\+b\)\.\\Theta:=\\\{\(a,b\)\\in\\hat\{\\Theta\}:\\;\\\|a\\\|\+\|b\|=1\\\},\\quad\\psi\_\{a,b\}\(x\):=\\phi\(\\left\\langle a,x\\right\\rangle\+b\)\.\(499\)ThisΘ\\Thetais compact, and againθ↦ψθ\\theta\\mapsto\\psi\_\{\\theta\}is a continuous map fromΘ\\ThetaintoCloc​\(ℝd\)C\_\{\\operatorname\{loc\}\}\(\{\\mathbb\{R\}\}^\{d\}\)\. Hence, in either case the Barron norm admits the equivalent representation

‖u‖ℬϕ=inf\{‖ν‖TV:u⁡\(x\)=∫Θψθ​\(x\)​𝑑ν​\(θ\)\}\(∀u∈ℬϕ\)\\\|u\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\inf\\left\\\{\\\|\\nu\\\|\_\{\\operatorname\{TV\}\}:\\;u\(x\)=\\int\_\{\\Theta\}\\psi\_\{\\theta\}\(x\)d\\nu\(\\theta\)\\right\\\}\\quad\(\\forall u\\in\\mathcal\{B\}\_\{\\phi\}\)\(500\)withν\\nuranging over finiteℝd\{\\mathbb\{R\}\}^\{d\}\-valued measures on the compact Polish spaceΘ\\Theta\.

With this, letL:=lim infn→∞‖wn‖ℬϕL:=\\liminf\_\{n\\to\\infty\}\\\|w\_\{n\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}and assume without loss of generality thatL<∞L<\\infty\. Passing to a subsequence, assume‖wn‖ℬϕ→L\\\|w\_\{n\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\to Land chooseνn\\nu\_\{n\}s\.t\.

wn​\(x\)=∫Θψθ​\(x\)​d​νn​\(θ\),‖νn‖TV≤‖wn‖ℬϕ\+1n\.w\_\{n\}\(x\)=\\int\_\{\\Theta\}\\psi\_\{\\theta\}\(x\)d\\nu\_\{n\}\(\\theta\),\\quad\\\|\\nu\_\{n\}\\\|\_\{\\operatorname\{TV\}\}\\leq\\\|w\_\{n\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\+\\frac\{1\}\{n\}\.\(501\)SinceΘ\\Thetais compact and Polish,C⁡\(Θ,ℝd\)C\(\\Theta;\{\\mathbb\{R\}\}^\{d\}\)is separable andℳ⁡\(Θ,ℝd\)=C​\(Θ,ℝd\)∗\\mathcal\{M\}\(\\Theta;\{\\mathbb\{R\}\}^\{d\}\)=C\(\\Theta;\{\\mathbb\{R\}\}^\{d\}\)^\{\*\}is the space of signed vector\-valued measures equipped with the TV norm\. Since\(νn\)n∈ℕ\(\\nu\_\{n\}\)\_\{n\\in\{\\mathbb\{N\}\}\}is bounded inℳ⁡\(Θ,ℝd\)\\mathcal\{M\}\(\\Theta;\{\\mathbb\{R\}\}^\{d\}\), by Banach\-Alaoglu we may pass to a subsequence satisfyingνn⇀ν\\nu\_\{n\}\\rightharpoonup\\nuweak\-\* for someν∈ℳ⁡\(Θ,ℝd\)\\nu\\in\\mathcal\{M\}\(\\Theta;\{\\mathbb\{R\}\}^\{d\}\)\. Sinceθ↦ψθ​\(x\)\\theta\\mapsto\\psi\_\{\\theta\}\(x\)is bounded continuous for everyxx, we know

w⁡\(x\)=limn→∞wn​\(x\)=∫Θψθ​\(x\)​𝑑ν​\(θ\)w\(x\)=\\lim\_\{n\\to\\infty\}w\_\{n\}\(x\)=\\int\_\{\\Theta\}\\psi\_\{\\theta\}\(x\)d\\nu\(\\theta\)\(502\)and soν\\nurepresentsww\. Since‖ν‖TV≤lim infn→∞‖νn‖TV≤L\\\|\\nu\\\|\_\{\\operatorname\{TV\}\}\\leq\\liminf\_\{n\\to\\infty\}\\\|\\nu\_\{n\}\\\|\_\{\\operatorname\{TV\}\}\\leq Lby lower\-semicontinuity of the dual norm under weak\-\* convergence, the claim follows\. ∎

Next, we show the tilt upper bound in terms of a second abstract quantity:

###### Lemma 16\(Single\-index tilts\)\.

Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be globally\-Lipschitz, nonconstant, bounded below, and suppose it satisfies Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)and𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ′​\(ξ\)\]≠0\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\phi^\{\\prime\}\(\\xi\)\]\\neq 0\. Fixθ∈𝕊d−1\\theta\\in\\mathbb\{S\}^\{d\-1\}and define

πεtilt:=\(1\+ε​hϕ\)​γd,hϕ​\(x\):=ϕ⁡\(⟨x,θ⟩\)−𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ⁡\(ξ\)\]\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}:=\(1\+\\varepsilon h\_\{\\phi\}\)\\gamma\_\{d\},\\quad h\_\{\\phi\}\(x\):=\{\\phi\}\(\\left\\langle x,\\theta\\right\\rangle\)\-\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\phi\(\\xi\)\]\(503\)forε\>0\\varepsilon\>0small enough that1\+ε​hϕ≥01\+\\varepsilon h\_\{\\phi\}\\geq 0\. Then, it holds that

lim supε↓0ℰℬϕ​\(πεtilt\)Dℬϕ​\(γd,πεtilt\)≤𝔉ϕtilt⋅min⁡\{‖ϕ‖∞,max⁡\{\|ϕ⁡\(0\)\|,Lip⁡\(ϕ\)\}\}\|𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ′​\(ξ\)\]\|\\limsup\_\{\\varepsilon\\downarrow 0\}\\frac\{\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\\left\(\{\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}\}\\right\)\}\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}\)\}\\leq\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\\cdot\\frac\{\\min\\left\\\{\\\|\\phi\\\|\_\{\\infty\},\\;\\max\\left\\\{\|\\phi\(0\)\|,\\;\\operatorname\{Lip\}\(\\phi\)\\right\\\}\\right\\\}\}\{\\left\|\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\phi^\{\\prime\}\(\\xi\)\]\\right\|\}\(504\)whereQrQ\_\{r\}is as in \([480](https://arxiv.org/html/2610.10875#A1.E480)\) and we define the quantity

𝔉ϕtilt:=lim supε↓0∫01‖θ​Qr′​\(⟨⋅,θ⟩\)r⁡\(1\+ε​Qr​\(⟨⋅,θ⟩\)−ε​𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ⁡\(ξ\)\]\)‖ℬϕ​𝑑r\.\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}:=\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}\\left\\\|\\theta\\frac\{Q\_\{r\}^\{\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\}\{r\(1\+\\varepsilon Q\_\{r\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\-\\varepsilon\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\phi\(\\xi\)\]\)\}\\right\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dr\.\(505\)

###### Proof\.

Let us denote

mϕ:=𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ⁡\(ξ\)\],fε:=πεtiltγd=1\+ε​hϕ\.m\_\{\\phi\}:=\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\phi\(\\xi\)\],\\quad f\_\{\\varepsilon\}:=\\frac\{\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}\}\{\\gamma\_\{d\}\}=1\+\\varepsilon h\_\{\\phi\}\.\(506\)Forr=e−τr=e^\{\-\\tau\}, by the Mehler formula we know

\(Pτ​fε\)​\(x\)=1\+ε⁡\(Qr​\(⟨x,θ⟩\)−mϕ\)\(P\_\{\\tau\}f\_\{\\varepsilon\}\)\(x\)=1\+\\varepsilon\(Q\_\{r\}\(\\left\\langle x,\\theta\\right\\rangle\)\-m\_\{\\phi\}\)\(507\)and therefore that forr∈\(0,1\)r\\in\(0,1\)we have

\(∇log⁡Pτ​fε\)​\(x\)=ε​θ​Qr′​\(⟨x,θ⟩\)1\+ε​Qr​\(⟨x,θ⟩\)−ε​mϕ\.\(\\nabla\\log P\_\{\\tau\}f\_\{\\varepsilon\}\)\(x\)=\\varepsilon\\theta\\frac\{Q\_\{r\}^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\}\{1\+\\varepsilon Q\_\{r\}\(\\left\\langle x,\\theta\\right\\rangle\)\-\\varepsilon m\_\{\\phi\}\}\.\(508\)Usingdτ=−dr/rd\\tau=\-dr/rand integrating,

lim supε↓0ℰℬϕ​\(πεtilt\)ε=𝔉ϕtilt\.\\limsup\_\{\\varepsilon\\downarrow 0\}\\frac\{\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\\left\(\{\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}\}\\right\)\}\{\\varepsilon\}=\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\.\(509\)
For the Barron transport distance lower bound, we use an argument similar to that of Lemma[3](https://arxiv.org/html/2610.10875#Thmlemma3)adapted for unboundedϕ\\phi\. Denote the quantities

L:=Lip⁡\(ϕ\),C0:=max⁡\{\|ϕ⁡\(0\)\|,L\},M:=‖ϕ‖∞,Cϕ:=min⁡\{M,C0\}L:=\\operatorname\{Lip\}\(\\phi\),\\quad C\_\{0\}:=\\max\\\{\|\\phi\(0\)\|,\\;L\\\},\\quad M:=\\\|\\phi\\\|\_\{\\infty\},\\quad C\_\{\\phi\}:=\\min\\\{M,C\_\{0\}\\\}\(510\)withM=\+∞M=\+\\inftyfor unboundedϕ\\phi\. We claim that for everyw∈ℬϕw\\in\\mathcal\{B\}\_\{\\phi\}and everyρ∈𝒫1​\(ℝd\)\\rho\\in\\mathcal\{P\}\_\{1\}\(\{\\mathbb\{R\}\}^\{d\}\)we have

∫ℝd‖w⁡\(x\)‖​𝑑ρ​\(x\)≤\(C0\+L​W1​\(γd,ρ\)\)​‖w‖ℬϕ\.\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|w\(x\)\\\|d\\rho\(x\)\\leq\(C\_\{0\}\+LW\_\{1\}\(\\gamma\_\{d\},\\rho\)\)\\\|w\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\.\(511\)To prove this, note that sinceϕ\\phiisLL\-Lipschitz then

𝔼ξ∼γd​\|ϕ⁡\(⟨a,ξ⟩\+b\)\|≤\|ϕ⁡\(0\)\|\+L​𝔼ξ∼γd​\|⟨a,ξ⟩\+b\|≤\|ϕ⁡\(0\)\|\+L⁡\(‖a‖\+\|b\|\)\.\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\\left\|\\phi\\left\(\\left\\langle a,\\xi\\right\\rangle\+b\\right\)\\right\|\\leq\|\\phi\(0\)\|\+L\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{d\}\}\|\\left\\langle a,\\xi\\right\\rangle\+b\|\\leq\|\\phi\(0\)\|\+L\(\\\|a\\\|\+\|b\|\)\.\(512\)Sinceϕ⁡\(0\)=0\\phi\(0\)=0for positively one\-homogeneousϕ\\phi, if we define

ω⁡\(a,b\):=\{1\+‖a‖\+\|b\|ϕ​bounded,‖a‖\+\|b\|ϕ​positively one\-homogeneous\\omega\(a,b\):=\\begin\{cases\}1\+\\\|a\\\|\+\|b\|&\\phi\\text\{ bounded\},\\\\ \\\|a\\\|\+\|b\|&\\phi\\text\{ positively one\-homogeneous\}\\end\{cases\}\(513\)then \([512](https://arxiv.org/html/2610.10875#A1.E512)\) implies that

∫ℝd\|ϕ⁡\(⟨a,x⟩\+b\)\|​d​γd​\(x\)≤C0​ω​\(a,b\)\.\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\|\\phi\\left\(\\left\\langle a,x\\right\\rangle\+b\\right\)\\right\|d\\gamma\_\{d\}\(x\)\\leq C\_\{0\}\\omega\(a,b\)\.\(514\)Sincex↦\|ϕ⁡\(⟨a,x⟩\+b\)\|x\\mapsto\|\\phi\(\\left\\langle a,x\\right\\rangle\+b\)\|isL​‖a‖L\\\|a\\\|\-Lipschitz and‖a‖≤ω⁡\(a,b\)\\\|a\\\|\\leq\\omega\(a,b\), by Kantorovich\-Rubinstein we have

∫ℝd\|ϕ⁡\(⟨a,x⟩\+b\)\|​𝑑ρ​\(x\)\\displaystyle\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\|\\phi\(\\left\\langle a,x\\right\\rangle\+b\)\|d\\rho\(x\)≤∫ℝd\|ϕ⁡\(⟨a,x⟩\+b\)\|​d​γd​\(x\)\+L​‖a‖​W1​\(γd,ρ\)\\displaystyle\\leq\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\|\\phi\(\\left\\langle a,x\\right\\rangle\+b\)\|d\\gamma\_\{d\}\(x\)\+L\\\|a\\\|W\_\{1\}\(\\gamma\_\{d\},\\rho\)\(515\)≤\(C0\+L​W1​\(γd,ρ\)\)​ω​\(a,b\)\.\\displaystyle\\leq\\left\(C\_\{0\}\+LW\_\{1\}\(\\gamma\_\{d\},\\rho\)\\right\)\\omega\(a,b\)\.\(516\)Integrating this over any vector measure representingwwand taking the infimum over such representations proves \([511](https://arxiv.org/html/2610.10875#A1.E511)\)\.

We now claim that

lim infε↓0Dℬϕ​\(γd,πεtilt\)ε≥\|𝔼ξ∼𝒩⁡\(0,1\)​ϕ′​\(ξ\)\|Cϕ,\\liminf\_\{\\varepsilon\\downarrow 0\}\\frac\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\}^\{\\operatorname\{tilt\}\}\)\}\{\\varepsilon\}\\geq\\frac\{\\left\|\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\phi^\{\\prime\}\(\\xi\)\\right\|\}\{C\_\{\\phi\}\},\(517\)which would yield the main result after combining with \([509](https://arxiv.org/html/2610.10875#A1.E509)\)\. To prove this, letεn↓0\\varepsilon\_\{n\}\\downarrow 0be any sequence realizing thelim inf\\liminf, let\(vtn,ρtn\)t∈\[0,1\]\(v\_\{t\}^\{n\},\\rho\_\{t\}^\{n\}\)\_\{t\\in\[0,1\]\}be any admissible transport fromγd\\gamma\_\{d\}toπεntilt\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}with

ℓn:=∫01‖vtn‖ℬϕ​𝑑t≤Dℬϕ​\(γd,πεntilt\)\+εn2,\\ell\_\{n\}:=\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{n\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dt\\leq D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\)\+\\varepsilon\_\{n\}^\{2\},\(518\)and define

An​\(t\):=∫0t∫ℝd‖vsn​\(x\)‖​d​ρsn​\(x\)​𝑑s\.A\_\{n\}\(t\):=\\int\_\{0\}^\{t\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\\|v\_\{s\}^\{n\}\(x\)\\\|d\\rho\_\{s\}^\{n\}\(x\)ds\.\(519\)Assume thatlimn→∞Dℬϕ​\(γd,πεntilt\)/εn<∞\\lim\_\{n\\to\\infty\}D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\)/\\varepsilon\_\{n\}<\\infty\(since \([517](https://arxiv.org/html/2610.10875#A1.E517)\) holds trivially otherwise\), in which caseℓn≤εn\\ell\_\{n\}\\leq\\sqrt\{\\varepsilon\_\{n\}\}for all large enoughnn\. By the standard continuity equation estimate \(i\.e\. the length of an absolutely continuous curve in\(𝒫1​\(ℝd\),W1\)\(\\mathcal\{P\}\_\{1\}\(\{\\mathbb\{R\}\}^\{d\}\),W\_\{1\}\)upper bounds theW1W\_\{1\}distance\) we knowW1​\(γd,ρtn\)≤An​\(t\)W\_\{1\}\(\\gamma\_\{d\},\\rho\_\{t\}^\{n\}\)\\leq A\_\{n\}\(t\)\. Applying \([511](https://arxiv.org/html/2610.10875#A1.E511)\) yields that

An′​\(t\)≤\(C0\+L​An​\(t\)\)​‖vtn‖ℬϕA^\{\\prime\}\_\{n\}\(t\)\\leq\(C\_\{0\}\+LA\_\{n\}\(t\)\)\\\|v\_\{t\}^\{n\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\(520\)for a\.e\.tt, after which Grönwall’s inequality implies

An​\(1\)≤C0L​\(eL​ℓn−1\),A\_\{n\}\(1\)\\leq\\frac\{C\_\{0\}\}\{L\}\\left\(e^\{L\\ell\_\{n\}\}\-1\\right\),\(521\)whereL\>0L\>0sinceϕ\\phiis nonconstant\. Ifϕ\\phiis bounded, then we may instead use thatW1​\(γd,πεntilt\)≤M​Dℬϕ​\(γd,πεntilt\)≤M​ℓnW\_\{1\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\)\\leq MD\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\)\\leq M\\ell\_\{n\}\(see Proposition[2](https://arxiv.org/html/2610.10875#Thmproposition2)\)\. Hence, in either case we find

W1​\(γd,πεntilt\)≤ℓn⋅min⁡\{M,C0​eL​ℓn−1L​ℓn\}\.W\_\{1\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\)\\leq\\ell\_\{n\}\\cdot\\min\\left\\\{M,\\;C\_\{0\}\\frac\{e^\{L\\ell\_\{n\}\}\-1\}\{L\\ell\_\{n\}\}\\right\\\}\.\(522\)On the other hand, applying Kantorovich\-Rubinstein with the11\-Lipschitz functionx↦⟨x,θ⟩x\\mapsto\\left\\langle x,\\theta\\right\\rangleimplies

W1​\(γd,πεntilt\)\\displaystyle W\_\{1\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\)≥\|∫ℝd⟨x,θ⟩​d​πεntilt​\(x\)−∫ℝd⟨x,θ⟩​d​γd​\(x\)\|=\|εn​∫ℝd⟨x,θ⟩​hϕ​\(x\)​d​γd​\(x\)\|\\displaystyle\\geq\\left\|\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle x,\\theta\\right\\rangle d\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\(x\)\-\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle x,\\theta\\right\\rangle d\\gamma\_\{d\}\(x\)\\right\|=\\left\|\\varepsilon\_\{n\}\\int\_\{\{\\mathbb\{R\}\}^\{d\}\}\\left\\langle x,\\theta\\right\\rangle h\_\{\\phi\}\(x\)d\\gamma\_\{d\}\(x\)\\right\|\(523\)=\|εn​𝔼ξ∼𝒩⁡\(0,1\)​\[ξ⁡\(ϕ⁡\(ξ\)−mϕ\)\]\|=\|εn​𝔼ξ∼𝒩⁡\(0,1\)​\[ξ​ϕ​\(ξ\)\]\|\\displaystyle=\\left\|\\varepsilon\_\{n\}\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\xi\(\\phi\(\\xi\)\-m\_\{\\phi\}\)\\right\]\\right\|=\\left\|\\varepsilon\_\{n\}\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\xi\\phi\(\\xi\)\\right\]\\right\|\(524\)=εn​\|𝔼ξ∼𝒩⁡\(0,1\)​ϕ′​\(ξ\)\|\\displaystyle=\\varepsilon\_\{n\}\\left\|\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\phi^\{\\prime\}\(\\xi\)\\right\|\(525\)by Gaussian integration by parts\. Combining this with \([522](https://arxiv.org/html/2610.10875#A1.E522)\) yields

ℓnεn≥\|𝔼ξ∼𝒩⁡\(0,1\)​ϕ′​\(ξ\)\|min⁡\{M,C0​eL​ℓn−1L​ℓn\}\.\\frac\{\\ell\_\{n\}\}\{\\varepsilon\_\{n\}\}\\geq\\frac\{\\left\|\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\phi^\{\\prime\}\(\\xi\)\\right\|\}\{\\min\\left\\\{M,\\;C\_\{0\}\\frac\{e^\{L\\ell\_\{n\}\}\-1\}\{L\\ell\_\{n\}\}\\right\\\}\}\.\(526\)Taking an infimum over transports fromγd\\gamma\_\{d\}toπεntilt\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}and using thatℓn≤εn\\ell\_\{n\}\\leq\\sqrt\{\\varepsilon\_\{n\}\}for all large enoughnnimplies that

Dℬϕ​\(γd,πεntilt\)εn≥\|𝔼ξ∼𝒩⁡\(0,1\)​ϕ′​\(ξ\)\|min⁡\{M,C0​eL​εn−1L​εn\}−εn\\frac\{D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi\_\{\\varepsilon\_\{n\}\}^\{\\operatorname\{tilt\}\}\)\}\{\\varepsilon\_\{n\}\}\\geq\\frac\{\\left\|\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\phi^\{\\prime\}\(\\xi\)\\right\|\}\{\\min\\left\\\{M,\\;C\_\{0\}\\frac\{e^\{L\\sqrt\{\\varepsilon\_\{n\}\}\}\-1\}\{L\\sqrt\{\\varepsilon\_\{n\}\}\}\\right\\\}\}\-\\varepsilon\_\{n\}\(527\)for allnnlarge enough\. In particular, asn→∞n\\to\\inftywe know

min⁡\{M,C0​eL​εn−1L​εn\}→min⁡\{M,C0\}=Cϕsinceεn↓0\\min\\left\\\{M,\\;C\_\{0\}\\frac\{e^\{L\\sqrt\{\\varepsilon\_\{n\}\}\}\-1\}\{L\\sqrt\{\\varepsilon\_\{n\}\}\}\\right\\\}\\to\\min\\\{M,\\;C\_\{0\}\\\}=C\_\{\\phi\}\\quad\\text\{since $\\varepsilon\_\{n\}\\downarrow 0$\}\(528\)and so \([517](https://arxiv.org/html/2610.10875#A1.E517)\) is proved\. ∎

We now specialize to some common activations and compute the quantities𝔉ϕflow\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}and𝔉ϕtilt\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\. First, we show a simpler form of𝔉ϕtilt\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}for thesin\\sinandtanh\\tanhcases:

###### Lemma 17\.

Ifλ∈\(0,∞\)\\lambda\\in\(0,\\infty\)andϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\)orϕ⁡\(z\)=tanh⁡\(λ​z\)\\phi\(z\)=\\tanh\(\\lambda z\), then

𝔉ϕtilt=∫011r​‖θ​Qr′​\(⟨⋅,θ⟩\)‖ℬϕ​𝑑r\.\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}=\\int\_\{0\}^\{1\}\\frac\{1\}\{r\}\\left\\\|\\theta Q\_\{r\}^\{\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\\right\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dr\.\(529\)

###### Proof\.

We will show that forϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\)andϕ⁡\(z\)=tanh⁡\(λ​z\)\\phi\(z\)=\\tanh\(\\lambda z\)it holds that if

Fr,ε​\(t\):=Qr′​\(t\)r⁡\(1\+ε​Qr​\(t\)\)−Qr′​\(t\)r=−ε​Qr​\(t\)​Qr′​\(t\)r⁡\(1\+ε​Qr​\(t\)\)=1r​dd​t​\[log⁡\(1\+ε​Qr\)ε−Qr\],F\_\{r,\\varepsilon\}\(t\):=\\frac\{Q\_\{r\}^\{\\prime\}\(t\)\}\{r\(1\+\\varepsilon Q\_\{r\}\(t\)\)\}\-\\frac\{Q^\{\\prime\}\_\{r\}\(t\)\}\{r\}=\-\\varepsilon\\frac\{Q\_\{r\}\(t\)Q\_\{r\}^\{\\prime\}\(t\)\}\{r\(1\+\\varepsilon Q\_\{r\}\(t\)\)\}=\\frac\{1\}\{r\}\\frac\{d\}\{dt\}\\left\[\\frac\{\\log\(1\+\\varepsilon Q\_\{r\}\)\}\{\\varepsilon\}\-Q\_\{r\}\\right\],\(530\)then there is the scalar Barron norm estimate

limε↓0∫01‖Fr,ε‖ℬϕ​𝑑r=0,\\lim\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}\\left\\\|F\_\{r,\\varepsilon\}\\right\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dr=0,\(531\)after which the main claim follows by the reverse triangle inequality\.

Forϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\), we may usesin⁡\(a\+b\)=sin⁡\(a\)​cos⁡\(b\)\+cos⁡\(a\)​sin⁡\(b\)\\sin\(a\+b\)=\\sin\(a\)\\cos\(b\)\+\\cos\(a\)\\sin\(b\)to find

Qr​\(t\)\\displaystyle Q\_\{r\}\(t\)=𝔼ξ∼𝒩⁡\(0,1\)​\[sin⁡\(λ​r​t\+λ​1−r2​ξ\)\]\\displaystyle=\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\sin\\left\(\\lambda rt\+\\lambda\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]\(532\)=sin⁡\(λ​r​t\)​𝔼ξ∼𝒩⁡\(0,1\)​\[cos⁡\(λ​1−r2​ξ\)\]\+cos⁡\(λ​r​t\)​𝔼ξ∼𝒩⁡\(0,1\)​\[sin⁡\(λ​1−r2​ξ\)\]\\displaystyle=\\sin\(\\lambda rt\)\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\cos\\left\(\\lambda\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]\+\\cos\(\\lambda rt\)\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\sin\\left\(\\lambda\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]\(533\)=sin⁡\(λ​r​t\)​𝔼ξ∼𝒩⁡\(0,1\)​\[cos⁡\(λ​1−r2​ξ\)\]\\displaystyle=\\sin\(\\lambda rt\)\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\cos\\left\(\\lambda\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]\(534\)=Ar​sin⁡\(λ​r​t\),\\displaystyle=A\_\{r\}\\sin\(\\lambda rt\),\(535\)whereAr:=exp⁡\(−λ22​\(1−r2\)\)≤1A\_\{r\}:=\\exp\(\-\\frac\{\\lambda^\{2\}\}\{2\}\(1\-r^\{2\}\)\)\\leq 1\. In particular,

Fr,ε\(t\)=−ελAr2sin⁡\(λ​r​t\)​cos⁡\(λ​r​t\)1\+ε​Ar​sin⁡\(λ​r​t\)=−ελAr2∑j=0∞\(−εAr\)jsin\(λrt\)j\+1cos\(λrt\),F\_\{r,\\varepsilon\}\(t\)=\-\\varepsilon\\lambda A\_\{r\}^\{2\}\\frac\{\\sin\(\\lambda rt\)\\cos\(\\lambda rt\)\}\{1\+\\varepsilon A\_\{r\}\\sin\(\\lambda rt\)\}=\-\\varepsilon\\lambda A\_\{r\}^\{2\}\\sum\_\{j=0\}^\{\\infty\}\(\-\\varepsilon A\_\{r\}\)^\{j\}\\sin\(\\lambda rt\)^\{j\+1\}\\cos\(\\lambda rt\),\(536\)where for the second equality we takeε<1\\varepsilon<1and use geometric series11−x=∑j=0∞xj\\frac\{1\}\{1\-x\}=\\sum\_\{j=0\}^\{\\infty\}x^\{j\}\. Denoting

hj​\(x\):=sin⁡\(x\)j\+1​cos⁡\(x\)=12j\+2​ij\+1​\(ei​x−e−i​x\)j\+1​\(ei​x\+e−i​x\)=∑n=−\(j\+2\)j\+2aj,n​sin⁡\(n​x\+bj,n\)h\_\{j\}\(x\):=\\sin\(x\)^\{j\+1\}\\cos\(x\)=\\frac\{1\}\{2^\{j\+2\}i^\{j\+1\}\}\(e^\{ix\}\-e^\{\-ix\}\)^\{j\+1\}\(e^\{ix\}\+e^\{\-ix\}\)=\\sum\_\{n=\-\(j\+2\)\}^\{j\+2\}a\_\{j,n\}\\sin\(nx\+b\_\{j,n\}\)\(537\)where\|bj,n\|≤π2\|b\_\{j,n\}\|\\leq\\frac\{\\pi\}\{2\}and∑n\|aj,n\|≤1\\sum\_\{n\}\|a\_\{j,n\}\|\\leq 1\. Hence,

∥hj\(λr⋅\)∥ℬϕ≤∑n=−\(j\+2\)j\+2\|aj,n\|\(1\+r\|n\|\+\|bj,n\|λ\)≤1\+r\(j\+2\)\+π2​λ\.\\\|h\_\{j\}\(\\lambda r\\cdot\)\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\sum\_\{n=\-\(j\+2\)\}^\{j\+2\}\|a\_\{j,n\}\|\\left\(1\+r\|n\|\+\\frac\{\|b\_\{j,n\}\|\}\{\\lambda\}\\right\)\\leq 1\+r\(j\+2\)\+\\frac\{\\pi\}\{2\\lambda\}\.\(538\)SinceFr,ε\(t\)=−ελAr2∑j=0∞\(−εAr\)jhj\(λrt\)F\_\{r,\\varepsilon\}\(t\)=\-\\varepsilon\\lambda A\_\{r\}^\{2\}\\sum\_\{j=0\}^\{\\infty\}\(\-\\varepsilon A\_\{r\}\)^\{j\}h\_\{j\}\(\\lambda rt\), we find

‖Fr,ε‖ℬϕ\\displaystyle\\\|F\_\{r,\\varepsilon\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}≤ε​λ​Ar2​∑j=0∞\(ε​Ar\)j​\(1\+r⁡\(j\+2\)\+π2​λ\)\\displaystyle\\leq\\varepsilon\\lambda A\_\{r\}^\{2\}\\sum\_\{j=0\}^\{\\infty\}\(\\varepsilon A\_\{r\}\)^\{j\}\\left\(1\+r\(j\+2\)\+\\frac\{\\pi\}\{2\\lambda\}\\right\)\(539\)≤ε​λ​∑j=0∞εj​\(j\+3\+π2​λ\)\\displaystyle\\leq\\varepsilon\\lambda\\sum\_\{j=0\}^\{\\infty\}\\varepsilon^\{j\}\\left\(j\+3\+\\frac\{\\pi\}\{2\\lambda\}\\right\)\(540\)≤\(8​λ\+π\)​ε\\displaystyle\\leq\(8\\lambda\+\\pi\)\\varepsilon\(541\)for allr∈\(0,1\)r\\in\(0,1\)andε∈\(0,1/2\)\\varepsilon\\in\(0,1/2\)\. Hence,∫01‖Fr,ε‖ℬϕ​𝑑r→0\\int\_\{0\}^\{1\}\\\|F\_\{r,\\varepsilon\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dr\\to 0\.

Forϕ⁡\(z\)=tanh⁡\(λ​z\)\\phi\(z\)=\\tanh\(\\lambda z\)we apply a deconvolution argument with the Fourier conventionf^​\(ξ\)=∫ℝf⁡\(x\)​e−2​π​i​x​ξ​𝑑x\\hat\{f\}\(\\xi\)=\\int\_\{\\mathbb\{R\}\}f\(x\)e^\{\-2\\pi ix\\xi\}dx\(note that this is different than the rest of the paper\)\. If we denote

Gr,ε​\(z\):=log⁡\(1\+ε​Qr​\(z/r\)\)ε−Qr​\(z/r\)G\_\{r,\\varepsilon\}\(z\):=\\frac\{\\log\(1\+\\varepsilon Q\_\{r\}\(z/r\)\)\}\{\\varepsilon\}\-Q\_\{r\}\(z/r\)\(542\)thenFr,ε​\(t\)=Gr,ε′​\(r​t\)F\_\{r,\\varepsilon\}\(t\)=G\_\{r,\\varepsilon\}^\{\\prime\}\(rt\)and so for eachjjwe have

∂ξjFr,ε′^​\(ξ\)=r−j​\(−2​π​i​x\)j​Gr,ε′′^​\(ξr\)\.\\partial\_\{\\xi\}^\{j\}\\widehat\{F\_\{r,\\varepsilon\}^\{\\prime\}\}\(\\xi\)=r^\{\-j\}\\widehat\{\(\-2\\pi ix\)^\{j\}G\_\{r,\\varepsilon\}^\{\\prime\\prime\}\}\\left\(\\frac\{\\xi\}\{r\}\\right\)\.\(543\)Fix anya∈\(π8​λ,π2​λ\)a\\in\(\\frac\{\\pi\}\{8\\lambda\},\\frac\{\\pi\}\{2\\lambda\}\)\. We claim there areDλ,a,ελ,a∈\(0,∞\)D\_\{\\lambda,a\},\\varepsilon\_\{\\lambda,a\}\\in\(0,\\infty\)s\.t\. for allε∈\(0,ελ,a\)\\varepsilon\\in\(0,\\varepsilon\_\{\\lambda,a\}\)and allr∈\(0,1\)r\\in\(0,1\)the functionGr,εG\_\{r,\\varepsilon\}is holomorphic on an open neighborhood of\|Im⁡\(z\)\|≤a\|\\operatorname\{Im\}\(z\)\|\\leq awith the estimate

sup\|y\|≤a\|x\+i​y\|j​\|Gr,ε′′​\(x\+i​y\)\|≤Dλ,a​ε1\+x2\(∀x∈ℝ\)\\sup\_\{\|y\|\\leq a\}\|x\+iy\|^\{j\}\\left\|G\_\{r,\\varepsilon\}^\{\\prime\\prime\}\(x\+iy\)\\right\|\\leq\\frac\{D\_\{\\lambda,a\}\\varepsilon\}\{1\+x^\{2\}\}\\quad\(\\forall x\\in\{\\mathbb\{R\}\}\)\(544\)forj∈\{0,1,2\}j\\in\\\{0,1,2\\\}\. To see this, we writeu=λ​xu=\\lambda x,v=λ​yv=\\lambda yand start from the identities

\|cosh⁡\(u\+i​v\)\|2=sinh⁡\(u\)2\+cos⁡\(v\)2,\|sinh⁡\(u\+i​v\)\|2=sinh⁡\(u\)2\+sin⁡\(v\)2\.\|\\cosh\(u\+iv\)\|^\{2\}=\\sinh\(u\)^\{2\}\+\\cos\(v\)^\{2\},\\quad\|\\sinh\(u\+iv\)\|^\{2\}=\\sinh\(u\)^\{2\}\+\\sin\(v\)^\{2\}\.\(545\)Forz=x\+i​yz=x\+iywith\|y\|≤a\|y\|\\leq awe know\|v\|≤λ​a<π/2\|v\|\\leq\\lambda a<\\pi/2and so

\|cosh\(u\+iv\)\|2≥cos\(v\)2\(sinh\(u\)2\+1\)=cos\(v\)2cosh\(u\)2≥cos\(λa\)2cosh\(u\)2\\displaystyle\|\\cosh\(u\+iv\)\|^\{2\}\\geq\\cos\(v\)^\{2\}\(\\sinh\(u\)^\{2\}\+1\)=\\cos\(v\)^\{2\}\\cosh\(u\)^\{2\}\\geq\\cos\(\\lambda a\)^\{2\}\\cosh\(u\)^\{2\}\(546\)and also

\|sinh⁡\(u\+i​v\)\|2≤sinh⁡\(u\)2\+1=cosh⁡\(u\)2\.\|\\sinh\(u\+iv\)\|^\{2\}\\leq\\sinh\(u\)^\{2\}\+1=\\cosh\(u\)^\{2\}\.\(547\)Dividing these two,

\|tanh⁡\(λ​z\)\|=\|sinh⁡\(u\+i​v\)\|\|cosh⁡\(u\+i​v\)\|≤sec⁡\(λ​a\)\.\|\\tanh\(\\lambda z\)\|=\\frac\{\|\\sinh\(u\+iv\)\|\}\{\|\\cosh\(u\+iv\)\|\}\\leq\\sec\(\\lambda a\)\.\(548\)Also, taking the reciprocal of \([546](https://arxiv.org/html/2610.10875#A1.E546)\) yields

\|sech\(λz\)\|2=1\|cosh⁡\(u\+i​v\)\|2≤sec\(λa\)2sech\(λx\)2≤4sec\(λa\)2e−2​λ​\|x\|,\|\\operatorname\{sech\}\(\\lambda z\)\|^\{2\}=\\frac\{1\}\{\|\\cosh\(u\+iv\)\|^\{2\}\}\\leq\\sec\(\\lambda a\)^\{2\}\\operatorname\{sech\}\(\\lambda x\)^\{2\}\\leq 4\\sec\(\\lambda a\)^\{2\}e^\{\-2\\lambda\|x\|\},\(549\)where we used thatsech⁡\(t\)2=4\(et\+e−t\)2≤4​e−2​\|t\|\\operatorname\{sech\}\(t\)^\{2\}=\\frac\{4\}\{\(e^\{t\}\+e^\{\-t\}\)^\{2\}\}\\leq 4e^\{\-2\|t\|\}\. Now, we know

\(tanh⁡\(λ​z\)\)′=λ​sech⁡\(λ​z\)2,\(tanh⁡\(λ​z\)\)′′=−2​λ2​tanh⁡\(λ​z\)​sech⁡\(λ​z\)2\.\(\\tanh\(\\lambda z\)\)^\{\\prime\}=\\lambda\\operatorname\{sech\}\(\\lambda z\)^\{2\},\\quad\(\\tanh\(\\lambda z\)\)^\{\\prime\\prime\}=\-2\\lambda^\{2\}\\tanh\(\\lambda z\)\\operatorname\{sech\}\(\\lambda z\)^\{2\}\.\(550\)and so if we defineqr​\(z\):=Qr​\(z/r\)=𝔼ξ∼𝒩⁡\(0,1\)​\[tanh⁡\(λ​z\+λ​1−r2​ξ\)\]q\_\{r\}\(z\):=Q\_\{r\}\(z/r\)=\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[\\tanh\(\\lambda z\+\\lambda\\sqrt\{1\-r^\{2\}\}\\xi\)\]and differentiate under the expectation then

\|qr\(z\)\|≤sec\(λa\),\|qr′\(z\)\|≤4λsec\(λa\)2𝔼\[e−2​λ​\|x\+1−r2​ξ\|\]\|q\_\{r\}\(z\)\|\\leq\\sec\(\\lambda a\),\\quad\|q\_\{r\}^\{\\prime\}\(z\)\|\\leq 4\\lambda\\sec\(\\lambda a\)^\{2\}\\mathbb\{E\}\[e^\{\-2\\lambda\|x\+\\sqrt\{1\-r^\{2\}\}\\xi\|\}\]\(551\)and similarly

\|qr′′\(z\)\|≤8λ2sec\(λa\)3𝔼\[e−2​λ​\|x\+1−r2​ξ\|\]\.\|q\_\{r\}^\{\\prime\\prime\}\(z\)\|\\leq 8\\lambda^\{2\}\\sec\(\\lambda a\)^\{3\}\\mathbb\{E\}\[e^\{\-2\\lambda\|x\+\\sqrt\{1\-r^\{2\}\}\\xi\|\}\]\.\(552\)Note that\|x\+1−r2​ξ\|≥\|x\|−1−r2​\|ξ\|≥\|x\|−\|ξ\|\|x\+\\sqrt\{1\-r^\{2\}\}\\xi\|\\geq\|x\|\-\\sqrt\{1\-r^\{2\}\}\|\\xi\|\\geq\|x\|\-\|\\xi\|and so

𝔼𝒩⁡\(0,1\)​\[e−2​λ​\|x\+1−r2​ξ\|\]≤e−2​λ​\|x\|⋅𝔼ξ∼𝒩⁡\(0,1\)​\[e2​λ​\|ξ\|\]\.\\mathbb\{E\}\_\{\\mathcal\{N\}\(0,1\)\}\[e^\{\-2\\lambda\|x\+\\sqrt\{1\-r^\{2\}\}\\xi\|\}\]\\leq e^\{\-2\\lambda\|x\|\}\\cdot\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[e^\{2\\lambda\|\\xi\|\}\]\.\(553\)Hence, if we define the constant

Cλ,a:=sec\(λa\)\+\(4λsec\(λa\)2\+8λ2sec\(λa\)3\)⋅𝔼ξ∼𝒩⁡\(0,1\)\[e2​λ​\|ξ\|\]∈\(0,∞\)C\_\{\\lambda,a\}:=\\sec\(\\lambda a\)\+\\left\(4\\lambda\\sec\(\\lambda a\)^\{2\}\+8\\lambda^\{2\}\\sec\(\\lambda a\)^\{3\}\\right\)\\cdot\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\[e^\{2\\lambda\|\\xi\|\}\]\\in\(0,\\infty\)\(554\)then for allx∈ℝx\\in\{\\mathbb\{R\}\},y∈\[−a,a\]y\\in\[\-a,a\], andr∈\(0,1\)r\\in\(0,1\)we have

\|qr​\(x\+i​y\)\|≤Cλ,a,\|qr′​\(x\+i​y\)\|\+\|qr′′​\(x\+i​y\)\|≤Cλ,a​e−2​λ​\|x\|\.\|q\_\{r\}\(x\+iy\)\|\\leq C\_\{\\lambda,a\},\\quad\|q\_\{r\}^\{\\prime\}\(x\+iy\)\|\+\|q\_\{r\}^\{\\prime\\prime\}\(x\+iy\)\|\\leq C\_\{\\lambda,a\}e^\{\-2\\lambda\|x\|\}\.\(555\)Take0<ε≤min⁡\{1,1/\(2​Cλ,a\)\}=:ελ,a0<\\varepsilon\\leq\\min\\\{1,\\;1/\(2C\_\{\\lambda,a\}\)\\\}=:\\varepsilon\_\{\\lambda,a\}\. Since

Gr,ε=log⁡\(1\+ε​qr\)ε−qr,G\_\{r,\\varepsilon\}=\\frac\{\\log\(1\+\\varepsilon q\_\{r\}\)\}\{\\varepsilon\}\-q\_\{r\},\(556\)and throughout\|y\|≤a\|y\|\\leq awe have\|ε​qr​\(x\+i​y\)\|≤12\|\\varepsilon q\_\{r\}\(x\+iy\)\|\\leq\\frac\{1\}\{2\}, we know that the principal branch of the logarithm is holomorphic on a neighborhood of the strip\|Im⁡\(z\)\|≤a\|\\operatorname\{Im\}\(z\)\|\\leq aand therefore so too isGr,εG\_\{r,\\varepsilon\}\. Furthermore, we compute

Gr,ε′′=−ε​\(qr′\)2\+qr​qr′′1\+ε​qr\+ε2​qr​\(qr′\)2\(1\+ε​qr\)2G\_\{r,\\varepsilon\}^\{\\prime\\prime\}=\-\\varepsilon\\frac\{\(q\_\{r\}^\{\\prime\}\)^\{2\}\+q\_\{r\}q\_\{r\}^\{\\prime\\prime\}\}\{1\+\\varepsilon q\_\{r\}\}\+\\varepsilon^\{2\}\\frac\{q\_\{r\}\(q\_\{r\}^\{\\prime\}\)^\{2\}\}\{\(1\+\\varepsilon q\_\{r\}\)^\{2\}\}\(557\)after which we apply the bound \([555](https://arxiv.org/html/2610.10875#A1.E555)\) to get

\|Gr,ε′′​\(x\+i​y\)\|\\displaystyle\|G\_\{r,\\varepsilon\}^\{\\prime\\prime\}\(x\+iy\)\|≤2​ε​\(\|qr′​\(x\+i​y\)\|2\+\|qr​\(x\+i​y\)\|​\|qr′′​\(x\+i​y\)\|\)\+4​ε2​\|qr​\(x\+i​y\)\|​\|qr′​\(x\+i​y\)\|2\\displaystyle\\leq 2\\varepsilon\\left\(\|q^\{\\prime\}\_\{r\}\(x\+iy\)\|^\{2\}\+\|q\_\{r\}\(x\+iy\)\|\|q\_\{r\}^\{\\prime\\prime\}\(x\+iy\)\|\\right\)\+4\\varepsilon^\{2\}\|q\_\{r\}\(x\+iy\)\|\|q\_\{r\}^\{\\prime\}\(x\+iy\)\|^\{2\}\(558\)≤4​ε​\(Cλ,a2\+Cλ,a3\)​e−2​λ​\|x\|\.\\displaystyle\\leq 4\\varepsilon\(C\_\{\\lambda,a\}^\{2\}\+C\_\{\\lambda,a\}^\{3\}\)e^\{\-2\\lambda\|x\|\}\.\(559\)If we define

C~λ,a:=max⁡supx∈ℝj∈\{0,1,2\}⁡\(1\+x2\)​\(\|x\|\+a\)j​e−2​λ​\|x\|<∞\\widetilde\{C\}\_\{\\lambda,a\}:=\\max\_\{j\\in\\\{0,1,2\\\}\}\\sup\_\{x\\in\{\\mathbb\{R\}\}\}\(1\+x^\{2\}\)\(\|x\|\+a\)^\{j\}e^\{\-2\\lambda\|x\|\}<\\infty\(560\)then we have shown \([544](https://arxiv.org/html/2610.10875#A1.E544)\) withDλ,a:=\(2​π\)2​4​\(Cλ,a2\+Cλ,a3\)​C~λ,aD\_\{\\lambda,a\}:=\(2\\pi\)^\{2\}4\(C\_\{\\lambda,a\}^\{2\}\+C\_\{\\lambda,a\}^\{3\}\)\\widetilde\{C\}\_\{\\lambda,a\}\. Since the mapz↦\(−2​π​i​z\)j​Gr,ε′′​\(z\)z\\mapsto\(\-2\\pi iz\)^\{j\}G\_\{r,\\varepsilon\}^\{\\prime\\prime\}\(z\)is holomorphic in the strip and satisfies the decay estimate \([544](https://arxiv.org/html/2610.10875#A1.E544)\), we get the exponential Fourier decay

\|\(−2​π​i​x\)j​Gr,ε′′^\(ξ\)\|≤πDλ,aεe−2​π​a​\|ξ\|\(∀ξ∈ℝ,∀j∈\{0,1,2\}\),\\left\|\\widehat\{\(\-2\\pi ix\)^\{j\}G^\{\\prime\\prime\}\_\{r,\\varepsilon\}\}\(\\xi\)\\right\|\\leq\\pi D\_\{\\lambda,a\}\\varepsilon e^\{\-2\\pi a\|\\xi\|\}\\quad\(\\forall\\xi\\in\{\\mathbb\{R\}\},\\;\\forall j\\in\\\{0,1,2\\\}\),\(561\)see e\.g\. Theorem 2\.1 of Chapter 4 in\[[76](https://arxiv.org/html/2610.10875#bib.bib30)\]\. In light of \([543](https://arxiv.org/html/2610.10875#A1.E543)\), this implies that

\|∂ξjFr,ε′^\(ξ\)\|≤r−jπDλ,aεe−2πa\|ξ\|/r\(∀ξ∈ℝ,∀j∈\{0,1,2\}\)\\left\|\\partial\_\{\\xi\}^\{j\}\\widehat\{F\_\{r,\\varepsilon\}^\{\\prime\}\}\(\\xi\)\\right\|\\leq r^\{\-j\}\\pi D\_\{\\lambda,a\}\\varepsilon e^\{\-2\\pi a\|\\xi\|/r\}\\quad\(\\forall\\xi\\in\{\\mathbb\{R\}\},\\;\\forall j\\in\\\{0,1,2\\\}\)\(562\)for allr∈\(0,1\)r\\in\(0,1\)andε∈\(0,ελ,a\]\\varepsilon\\in\(0,\\varepsilon\_\{\\lambda,a\}\]\. Let us now define

kr​\(t\):=tanh⁡\(4​λ​r​t\),kr′^​\(ξ\)=π2​ξ/\(2​λ​r\)sinh⁡\(π2​ξ/\(4​λ​r\)\),k\_\{r\}\(t\):=\\tanh\(4\\lambda rt\),\\quad\\widehat\{k\_\{r\}^\{\\prime\}\}\(\\xi\)=\\frac\{\\pi^\{2\}\\xi/\(2\\lambda r\)\}\{\\sinh\(\\pi^\{2\}\\xi/\(4\\lambda r\)\)\},\(563\)which implies the estimate

\|∂ξj1kr′^​\(ξ\)\|≤12\(π24​λ​r\)jeπ2​\|ξ\|/\(4​λ​r\)\(∀ξ∈ℝ,∀j∈\{0,1,2\}\)\\left\|\\partial\_\{\\xi\}^\{j\}\\frac\{1\}\{\\widehat\{k\_\{r\}^\{\\prime\}\}\(\\xi\)\}\\right\|\\leq\\frac\{1\}\{2\}\\left\(\\frac\{\\pi^\{2\}\}\{4\\lambda r\}\\right\)^\{j\}e^\{\\pi^\{2\}\|\\xi\|/\(4\\lambda r\)\}\\quad\(\\forall\\xi\\in\{\\mathbb\{R\}\},\\;\\forall j\\in\\\{0,1,2\\\}\)\(564\)To see this, note thatsinh⁡\(x/2\)x=12​∫01cosh⁡\(t​x2\)​𝑑t\\frac\{\\sinh\(x/2\)\}\{x\}=\\frac\{1\}\{2\}\\int\_\{0\}^\{1\}\\cosh\(\\frac\{tx\}\{2\}\)dtand so differentiating under the integral yields

\|djd​xj​sinh⁡\(x/2\)x\|≤12j\+1​∫01tj​e\|x\|/2​𝑑t≤2−j−1​e\|x\|/2\.\\left\|\\frac\{d^\{j\}\}\{dx^\{j\}\}\\frac\{\\sinh\(x/2\)\}\{x\}\\right\|\\leq\\frac\{1\}\{2^\{j\+1\}\}\\int\_\{0\}^\{1\}t^\{j\}e^\{\|x\|/2\}dt\\leq 2^\{\-j\-1\}e^\{\|x\|/2\}\.\(565\)If we define the function

mr,ε^​\(ξ\):=Fr,ε′^​\(ξ\)kr′^​\(ξ\)\(∀ξ∈ℝ\),\\widehat\{m\_\{r,\\varepsilon\}\}\(\\xi\):=\\frac\{\\widehat\{F^\{\\prime\}\_\{r,\\varepsilon\}\}\(\\xi\)\}\{\\widehat\{k\_\{r\}^\{\\prime\}\}\(\\xi\)\}\\quad\(\\forall\\xi\\in\{\\mathbb\{R\}\}\),\(566\)then by \([562](https://arxiv.org/html/2610.10875#A1.E562)\), \([564](https://arxiv.org/html/2610.10875#A1.E564)\) and the product rule, if we denoteMλ,a:=π2​Dλ,a​maxj∈\{0,1,2\}​\(1\+π24​λ\)jM\_\{\\lambda,a\}:=\\frac\{\\pi\}\{2\}D\_\{\\lambda,a\}\\max\_\{j\\in\\\{0,1,2\\\}\}\(1\+\\frac\{\\pi^\{2\}\}\{4\\lambda\}\)^\{j\}then

\|∂ξjmr,ε^\(ξ\)\|≤Mλ,aεr−je−\(2πa−π24​λ\)\|ξ\|/r\(∀ξ∈ℝ,∀j∈\{0,1,2\}\)\.\\left\|\\partial\_\{\\xi\}^\{j\}\\widehat\{m\_\{r,\\varepsilon\}\}\(\\xi\)\\right\|\\leq M\_\{\\lambda,a\}\\varepsilon r^\{\-j\}e^\{\-\\left\(2\\pi a\-\\frac\{\\pi^\{2\}\}\{4\\lambda\}\\right\)\|\\xi\|/r\}\\quad\(\\forall\\xi\\in\{\\mathbb\{R\}\},\\;\\forall j\\in\\\{0,1,2\\\}\)\.\(567\)Note that2​π​a−π24​λ\>02\\pi a\-\\frac\{\\pi^\{2\}\}\{4\\lambda\}\>0sincea\>π8​λa\>\\frac\{\\pi\}\{8\\lambda\}\. The important consequence is that if we denotecλ,a:=Mλ,a2​π​a−π24​λc\_\{\\lambda,a\}:=\\frac\{M\_\{\\lambda,a\}\}\{\\sqrt\{2\\pi a\-\\frac\{\\pi^\{2\}\}\{4\\lambda\}\}\}then we have

‖∂ξjmr,ε^‖L2​\(d​ξ\)≤cλ,a⋅ε​r12−j\(∀j∈\{0,1,2\}\)\\left\\\|\\partial\_\{\\xi\}^\{j\}\\widehat\{m\_\{r,\\varepsilon\}\}\\right\\\|\_\{L^\{2\}\(d\\xi\)\}\\leq c\_\{\\lambda,a\}\\cdot\\varepsilon r^\{\\frac\{1\}\{2\}\-j\}\\quad\(\\forall j\\in\\\{0,1,2\\\}\)\(568\)for allε∈\(0,ελ,a\)\\varepsilon\\in\(0,\\varepsilon\_\{\\lambda,a\}\)andr∈\(0,1\)r\\in\(0,1\)\. Hence, we may apply Fourier inversion to definemr,ε:ℝ→ℝm\_\{r,\\varepsilon\}:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}, and so by Plancherel

‖mr,ε‖L2​\(d​x\)\\displaystyle\\left\\\|m\_\{r,\\varepsilon\}\\right\\\|\_\{L^\{2\}\(dx\)\}=‖mr,ε^‖L2​\(d​ξ\)≤cλ,a​ε​r1/2,\\displaystyle=\\left\\\|\\widehat\{m\_\{r,\\varepsilon\}\}\\right\\\|\_\{L^\{2\}\(d\\xi\)\}\\leq c\_\{\\lambda,a\}\\varepsilon r^\{1/2\},\(569\)‖x​mr,ε‖L2​\(d​x\)\\displaystyle\\left\\\|xm\_\{r,\\varepsilon\}\\right\\\|\_\{L^\{2\}\(dx\)\}=12​π‖∂ξmr,ε^‖L2​\(d​ξ\)≤cλ,a2​πεr−1/2,\\displaystyle=\\frac\{1\}\{2\\pi\}\\left\\\|\\partial\_\{\\xi\}\\widehat\{m\_\{r,\\varepsilon\}\}\\right\\\|\_\{L^\{2\}\(d\\xi\)\}\\leq\\frac\{c\_\{\\lambda,a\}\}\{2\\pi\}\\varepsilon r^\{\-1/2\},\(570\)‖x2​mr,ε‖L2​\(d​x\)\\displaystyle\\left\\\|x^\{2\}m\_\{r,\\varepsilon\}\\right\\\|\_\{L^\{2\}\(dx\)\}=14​π2‖∂ξ2mr,ε^‖L2​\(d​ξ\)≤cλ,a4​π2εr−3/2\.\\displaystyle=\\frac\{1\}\{4\\pi^\{2\}\}\\left\\\|\\partial\_\{\\xi\}^\{2\}\\widehat\{m\_\{r,\\varepsilon\}\}\\right\\\|\_\{L^\{2\}\(d\\xi\)\}\\leq\\frac\{c\_\{\\lambda,a\}\}\{4\\pi^\{2\}\}\\varepsilon r^\{\-3/2\}\.\(571\)Furthermore, by Cauchy\-Schwarz

∫ℝ\(1\+r​\|x\|\)​\|mr,ε​\(x\)\|​𝑑x\\displaystyle\\int\_\{\\mathbb\{R\}\}\(1\+r\|x\|\)\|m\_\{r,\\varepsilon\}\(x\)\|dx≤\(∫ℝ\(1\+r​\|x\|\)2\(1\+r2​x2\)2​𝑑x\)1/2⋅\(∫ℝ\(1\+r2​x2\)2​\|mr,ε​\(x\)\|2​𝑑x\)1/2\\displaystyle\\leq\\left\(\\int\_\{\\mathbb\{R\}\}\\frac\{\(1\+r\|x\|\)^\{2\}\}\{\(1\+r^\{2\}x^\{2\}\)^\{2\}\}dx\\right\)^\{1/2\}\\cdot\\left\(\\int\_\{\\mathbb\{R\}\}\(1\+r^\{2\}x^\{2\}\)^\{2\}\|m\_\{r,\\varepsilon\}\(x\)\|^\{2\}dx\\right\)^\{1/2\}\(572\)=π\+2r⋅\(‖mr,ε‖L2​\(d​x\)2\+2​r2​‖x​mr,ε‖L2​\(d​x\)2\+r4​‖x2​mr,ε‖L2​\(d​x\)2\)1/2\\displaystyle=\\sqrt\{\\frac\{\\pi\+2\}\{r\}\}\\cdot\\left\(\\\|m\_\{r,\\varepsilon\}\\\|\_\{L^\{2\}\(dx\)\}^\{2\}\+2r^\{2\}\\\|xm\_\{r,\\varepsilon\}\\\|\_\{L^\{2\}\(dx\)\}^\{2\}\+r^\{4\}\\\|x^\{2\}m\_\{r,\\varepsilon\}\\\|\_\{L^\{2\}\(dx\)\}^\{2\}\\right\)^\{1/2\}\(573\)≤π\+2r⋅cλ,a​ε⋅\(r\+2​r24​π2​r\+r416​π4​r3\)1/2\\displaystyle\\leq\\sqrt\{\\frac\{\\pi\+2\}\{r\}\}\\cdot c\_\{\\lambda,a\}\\varepsilon\\cdot\\left\(r\+\\frac\{2r^\{2\}\}\{4\\pi^\{2\}r\}\+\\frac\{r^\{4\}\}\{16\\pi^\{4\}r^\{3\}\}\\right\)^\{1/2\}\(574\)=π\+2⋅\(1\+14​π2\)⋅cλ,a​ε,\\displaystyle=\\sqrt\{\\pi\+2\}\\cdot\\left\(1\+\\frac\{1\}\{4\\pi^\{2\}\}\\right\)\\cdot c\_\{\\lambda,a\}\\varepsilon,\(575\)where the second line used that∫ℝ\(1\+r​\|x\|\)2\(1\+r2​x2\)2​𝑑x=1r​∫ℝ\(1\+\|y\|\)2\(1\+y2\)2​𝑑y=π\+2r\\int\_\{\\mathbb\{R\}\}\\frac\{\(1\+r\|x\|\)^\{2\}\}\{\(1\+r^\{2\}x^\{2\}\)^\{2\}\}dx=\\frac\{1\}\{r\}\\int\_\{\\mathbb\{R\}\}\\frac\{\(1\+\|y\|\)^\{2\}\}\{\(1\+y^\{2\}\)^\{2\}\}dy=\\frac\{\\pi\+2\}\{r\}\. Now, by \([566](https://arxiv.org/html/2610.10875#A1.E566)\) and the convolution theorem we knowFr,ε′=kr′∗mr,εF\_\{r,\\varepsilon\}^\{\\prime\}=k\_\{r\}^\{\\prime\}\*m\_\{r,\\varepsilon\}and therefore that

Fr,ε=kr∗mr,ε\+CF\_\{r,\\varepsilon\}=k\_\{r\}\*m\_\{r,\\varepsilon\}\+C\(576\)for some constantC∈ℝC\\in\{\\mathbb\{R\}\}\. SinceFr,ε=−ε​Qr​Qr′r⁡\(1\+ε​Qr\)F\_\{r,\\varepsilon\}=\-\\varepsilon\\frac\{Q\_\{r\}Q\_\{r\}^\{\\prime\}\}\{r\(1\+\\varepsilon Q\_\{r\}\)\}and‖Qr‖∞<∞\\\|Q\_\{r\}\\\|\_\{\\infty\}<\\inftywhilelim\|t\|→∞Qr′​\(t\)=0\\lim\_\{\|t\|\\to\\infty\}Q\_\{r\}^\{\\prime\}\(t\)=0, we know thatlim\|t\|→∞Fr,ε​\(t\)=0\\lim\_\{\|t\|\\to\\infty\}F\_\{r,\\varepsilon\}\(t\)=0\. Since‖kr‖∞<∞\\\|k\_\{r\}\\\|\_\{\\infty\}<\\inftyandmr,ε∈L1​\(d​x\)m\_\{r,\\varepsilon\}\\in L^\{1\}\(dx\), by dominated convergence

limt→∞\(kr∗mr,ε\)​\(t\)=∫ℝmr,ε​\(x\)​𝑑x=mr,ε^​\(0\)=Fr,ε′^​\(0\)kr′^​\(0\)=Fr,ε​\(∞\)−Fr,ε​\(−∞\)2=0\.\\lim\_\{t\\to\\infty\}\(k\_\{r\}\*m\_\{r,\\varepsilon\}\)\(t\)=\\int\_\{\\mathbb\{R\}\}m\_\{r,\\varepsilon\}\(x\)dx=\\widehat\{m\_\{r,\\varepsilon\}\}\(0\)=\\frac\{\\widehat\{F^\{\\prime\}\_\{r,\\varepsilon\}\}\(0\)\}\{\\widehat\{k^\{\\prime\}\_\{r\}\}\(0\)\}=\\frac\{F\_\{r,\\varepsilon\}\(\\infty\)\-F\_\{r,\\varepsilon\}\(\-\\infty\)\}\{2\}=0\.\(577\)Thus,C=0C=0and so \([576](https://arxiv.org/html/2610.10875#A1.E576)\) reads

Fr,ε​\(t\)=∫ℝϕ⁡\(4​r​t−4​r​x\)​mr,ε​\(x\)​𝑑x\.F\_\{r,\\varepsilon\}\(t\)=\\int\_\{\\mathbb\{R\}\}\\phi\(4rt\-4rx\)m\_\{r,\\varepsilon\}\(x\)dx\.\(578\)This is a representation ofFr,εF\_\{r,\\varepsilon\}as a mixture of neurons, and so

‖Fr,ε‖ℬϕ\\displaystyle\\\|F\_\{r,\\varepsilon\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}≤∫ℝ\(1\+4​r\+4​r​\|x\|\)\|mr,ε​\(x\)\|𝑑x≤5​∫ℝ\(1\+r​\|x\|\)​\|mr,ε​\(x\)\|​𝑑x\\displaystyle\\leq\\int\_\{\\mathbb\{R\}\}\(1\+4r\+4r\|x\|\)\|m\_\{r,\\varepsilon\}\(x\)\|dx\\leq 5\\int\_\{\\mathbb\{R\}\}\(1\+r\|x\|\)\|m\_\{r,\\varepsilon\}\(x\)\|dx\(579\)≤5​π\+2⋅\(1\+14​π2\)⋅cλ,a​ε\.\\displaystyle\\leq 5\\sqrt\{\\pi\+2\}\\cdot\\left\(1\+\\frac\{1\}\{4\\pi^\{2\}\}\\right\)\\cdot c\_\{\\lambda,a\}\\varepsilon\.\(580\)This holds for allε∈\(0,ελ,a\)\\varepsilon\\in\(0,\\varepsilon\_\{\\lambda,a\}\)and allr∈\(0,1\)r\\in\(0,1\), which yields the claim\. ∎

We may now compute the quantities of interest for the activation functions we consider:

###### Proposition 8\.

Let𝔉ϕflow\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}and𝔉ϕtilt\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}be as in \([479](https://arxiv.org/html/2610.10875#A1.E479)\) and \([505](https://arxiv.org/html/2610.10875#A1.E505)\), respectively\.

- •\(Sinusoidal activation\)\.Ifϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\)for someλ∈\(0,∞\)\\lambda\\in\(0,\\infty\), then it holds that 𝔉ϕflow​\(c\)=\+∞\(∀c∈\[−1,1\]\),𝔉ϕtilt≤1\+2π\+π2\.\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}\(c\)=\+\\infty\\quad\(\\forall c\\in\[\-1,1\]\),\\quad\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\\leq 1\+\\sqrt\{\\frac\{2\}\{\\pi\}\}\+\\sqrt\{\\frac\{\\pi\}\{2\}\}\.\(581\)
- •\(Tanh activation\)\.Ifϕ⁡\(z\)=tanh⁡\(λ​z\)\\phi\(z\)=\\tanh\(\\lambda z\)for someλ∈\(0,∞\)\\lambda\\in\(0,\\infty\), then it holds that 𝔉ϕflow​\(c\)=\+∞\(∀c∈\(−1,1\)\),𝔉ϕtilt≤1\+2π\+π2\.\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}\(c\)=\+\\infty\\quad\(\\forall c\\in\(\-1,1\)\),\\quad\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\\leq 1\+\\sqrt\{\\frac\{2\}\{\\pi\}\}\+\\sqrt\{\\frac\{\\pi\}\{2\}\}\.\(582\)
- •\(ReLU activation\)\.Ifϕ⁡\(z\)=max⁡\{0,z\}\\phi\(z\)=\\max\\\{0,z\\\}, then it holds that 𝔉ϕflow​\(c\)=\+∞\(∀c∈\(−1,1\)\),𝔉ϕtilt≤9\+2π\+π2\.\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}\(c\)=\+\\infty\\quad\(\\forall c\\in\(\-1,1\)\),\\quad\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\\leq 9\+\\sqrt\{\\frac\{2\}\{\\pi\}\}\+\\sqrt\{\\frac\{\\pi\}\{2\}\}\.\(583\)

###### Proof\.

First we will show that𝔉ϕflow​\(c\)=\+∞\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{flow\}\}\(c\)=\+\\inftyin the stated cases, for which we denote

Vr​\(x\):=u​Qr​\(⟨x,θ⟩\)\+θ⁡⟨x,u⟩​Qr′​\(⟨x,θ⟩\)−c​θ​Qr′′​\(⟨x,θ⟩\)\.V\_\{r\}\(x\):=uQ\_\{r\}\(\\left\\langle x,\\theta\\right\\rangle\)\+\\theta\\left\\langle x,u\\right\\rangle Q\_\{r\}^\{\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\-c\\theta Q\_\{r\}^\{\\prime\\prime\}\(\\left\\langle x,\\theta\\right\\rangle\)\.\(584\)If\|c\|<1\|c\|<1and we takev:=u−c​θ1−c2v:=\\frac\{u\-c\\theta\}\{\\sqrt\{1\-c^\{2\}\}\}then⟨v,θ⟩=0\\left\\langle v,\\theta\\right\\rangle=0while⟨v,u⟩=1−c2\>0\\left\\langle v,u\\right\\rangle=\\sqrt\{1\-c^\{2\}\}\>0\. Hence,

Vr​\(a​v\)=u​Qr​\(0\)\+a⋅θ​1−c2​Qr′​\(0\)−c​θ​Qr′′​\(0\)\(∀a\>0\)\.V\_\{r\}\(av\)=uQ\_\{r\}\(0\)\+a\\cdot\\theta\\sqrt\{1\-c^\{2\}\}Q\_\{r\}^\{\\prime\}\(0\)\-c\\theta Q\_\{r\}^\{\\prime\\prime\}\(0\)\\quad\(\\forall a\>0\)\.\(585\)In the casesϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\)andϕ⁡\(z\)=tanh⁡\(λ​z\)\\phi\(z\)=\\tanh\(\\lambda z\)we know‖ϕ‖∞,‖ϕ′‖∞,‖ϕ′′‖∞<∞\\\|\\phi\\\|\_\{\\infty\},\\\|\\phi^\{\\prime\}\\\|\_\{\\infty\},\\\|\\phi^\{\\prime\\prime\}\\\|\_\{\\infty\}<\\inftyand so‖Qr‖∞,‖Qr′‖∞,‖Qr′′‖∞<∞\\\|Q\_\{r\}\\\|\_\{\\infty\},\\\|Q\_\{r\}^\{\\prime\}\\\|\_\{\\infty\},\\\|Q\_\{r\}^\{\\prime\\prime\}\\\|\_\{\\infty\}<\\inftyfor allr∈\(0,1\)r\\in\(0,1\)\. Forϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\)we know from the proof of Lemma[17](https://arxiv.org/html/2610.10875#Thmlemma17)that

Qr′​\(0\)=λ​r​Ar​cos⁡\(0\)\>0\(∀r∈\(0,1\)\)Q^\{\\prime\}\_\{r\}\(0\)=\\lambda rA\_\{r\}\\cos\(0\)\>0\\quad\(\\forall r\\in\(0,1\)\)\(586\)and forϕ⁡\(z\)=tanh⁡\(λ​z\)\\phi\(z\)=\\tanh\(\\lambda z\)we directly compute

Qr′​\(0\)=r​λ​𝔼ξ∼𝒩⁡\(0,1\)​\[sech⁡\(λ​1−r2​ξ\)2\]\>0\(∀r∈\(0,1\]\)\.Q^\{\\prime\}\_\{r\}\(0\)=r\\lambda\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\operatorname\{sech\}\\left\(\\lambda\\sqrt\{1\-r^\{2\}\}\\xi\\right\)^\{2\}\\right\]\>0\\quad\(\\forall r\\in\(0,1\]\)\.\(587\)Hence,a↦Vr​\(a​v\)a\\mapsto V\_\{r\}\(av\)is unbounded\. Since‖Vr‖∞≤‖ϕ‖∞​‖Vr‖ℬϕ\\\|V\_\{r\}\\\|\_\{\\infty\}\\leq\\\|\\phi\\\|\_\{\\infty\}\\\|V\_\{r\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}, this implies‖Vr‖ℬϕ=∞\\\|V\_\{r\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\inftyfor allr∈\(0,1\)r\\in\(0,1\)\. If instead\|c\|=1\|c\|=1thenu=c​θu=c\\thetain which case

Vr​\(a​θ\)=u​Qr​\(a\)\+a​u​Qr′​\(a\)−u​Qr′′​\(a\)\(∀a\>0\)\.V\_\{r\}\(a\\theta\)=uQ\_\{r\}\(a\)\+auQ\_\{r\}^\{\\prime\}\(a\)\-uQ\_\{r\}^\{\\prime\\prime\}\(a\)\\quad\(\\forall a\>0\)\.\(588\)In the caseϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\)we knowa↦a​Qr′​\(a\)=λ​r​Ar⋅a​cos⁡\(λ​r​a\)a\\mapsto aQ\_\{r\}^\{\\prime\}\(a\)=\\lambda rA\_\{r\}\\cdot a\\cos\(\\lambda ra\)is unbounded, and‖Vr‖ℬϕ=\+∞\\\|V\_\{r\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\+\\inftyis true even when\|c\|=1\|c\|=1\. Lastly, in the caseϕ⁡\(z\)=max⁡\{0,z\}\\phi\(z\)=\\max\\\{0,z\\\}and\|c\|<1\|c\|<1we again definev:=u−c​θ1−c2v:=\\frac\{u\-c\\theta\}\{\\sqrt\{1\-c^\{2\}\}\}to get

⟨Vr​\(a​v\+b​θ\),θ⟩=c​Qr​\(b\)\+\(a​1−c2\+b​c\)​Qr′​\(b\)−c​Qr′′​\(b\)\(∀a,b\>0\)\.\\left\\langle V\_\{r\}\(av\+b\\theta\),\\theta\\right\\rangle=cQ\_\{r\}\(b\)\+\\left\(a\\sqrt\{1\-c^\{2\}\}\+bc\\right\)Q\_\{r\}^\{\\prime\}\(b\)\-cQ\_\{r\}^\{\\prime\\prime\}\(b\)\\quad\(\\forall a,b\>0\)\.\(589\)Hence, comparing withb=0b=0gives

⟨Vr​\(a​v\+b​θ\)−Vr​\(a​v\),θ⟩=\(a​1−c2\)​\(Qr′​\(b\)−Qr′​\(0\)\)\+Cr,b,c\\left\\langle V\_\{r\}\(av\+b\\theta\)\-V\_\{r\}\(av\),\\theta\\right\\rangle=\\left\(a\\sqrt\{1\-c^\{2\}\}\\right\)\\left\(Q\_\{r\}^\{\\prime\}\(b\)\-Q\_\{r\}^\{\\prime\}\(0\)\\right\)\+C\_\{r,b,c\}\(590\)for some constantCr,b,cC\_\{r,b,c\}independent ofaa\. Hence,

‖Vr​\(a​v\+b​θ\)−Vr​\(a​v\)‖‖\(a​v\+b​θ\)−a​v‖\\displaystyle\\frac\{\\\|V\_\{r\}\(av\+b\\theta\)\-V\_\{r\}\(av\)\\\|\}\{\\\|\(av\+b\\theta\)\-av\\\|\}=‖Vr​\(a​v\+b​θ\)−Vr​\(a​v\)‖b≥\|⟨Vr​\(a​v\+b​θ\)−Vr​\(a​v\),θ⟩\|b\\displaystyle=\\frac\{\\\|V\_\{r\}\(av\+b\\theta\)\-V\_\{r\}\(av\)\\\|\}\{b\}\\geq\\frac\{\\left\|\\left\\langle V\_\{r\}\(av\+b\\theta\)\-V\_\{r\}\(av\),\\theta\\right\\rangle\\right\|\}\{b\}\(591\)≥a​1−c2b​\|Qr′​\(b\)−Qr′​\(0\)\|−\|Cr,b,c\|b\.\\displaystyle\\geq\\frac\{a\\sqrt\{1\-c^\{2\}\}\}\{b\}\\left\|Q\_\{r\}^\{\\prime\}\(b\)\-Q^\{\\prime\}\_\{r\}\(0\)\\right\|\-\\frac\{\|C\_\{r,b,c\}\|\}\{b\}\.\(592\)For ReLU,Qr′​\(b\)=r​Φ​\(r​b1−r2\)Q\_\{r\}^\{\\prime\}\(b\)=r\\Phi\(\\frac\{rb\}\{\\sqrt\{1\-r^\{2\}\}\}\)is strictly increasing and soQr′​\(b\)−Qr′​\(0\)≠0Q\_\{r\}^\{\\prime\}\(b\)\-Q^\{\\prime\}\_\{r\}\(0\)\\neq 0, which means this quantity can be made as large as possible \(by takingaalarge withbbfixed\)\. Hence,VrV\_\{r\}cannot be globally\-Lipschitz; since‖ϕ′‖L∞<∞\\\|\\phi^\{\\prime\}\\\|\_\{L^\{\\infty\}\}<\\inftythis means that‖Vr‖ℬϕ=∞\\\|V\_\{r\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}=\\inftyas desired\.

We now bound the energy of the tilts\. For any Lipschitzϕ\\phiwe know

1r​Qr′​\(t\)=𝔼ξ∼𝒩⁡\(0,1\)​\[ϕ′​\(r​t\+1−r2​ξ\)\]=11−r2​𝔼ξ∼𝒩⁡\(0,1\)​\[ξ​ϕ​\(r​t\+1−r2​ξ\)\]\\frac\{1\}\{r\}Q\_\{r\}^\{\\prime\}\(t\)=\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\phi^\{\\prime\}\\left\(rt\+\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]=\\frac\{1\}\{\\sqrt\{1\-r^\{2\}\}\}\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\xi\\phi\\left\(rt\+\\sqrt\{1\-r^\{2\}\}\\xi\\right\)\\right\]\(593\)by Gaussian integration by parts\. This is a valid Barron representation, and so

1r​‖θ​Qr′​\(⟨⋅,θ⟩\)‖ℬϕ\\displaystyle\\frac\{1\}\{r\}\\\|\\theta Q\_\{r\}^\{\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}≤11−r2​𝔼ξ∼𝒩⁡\(0,1\)​\[\|ξ\|​\(1\+r\+1−r2​\|ξ\|\)\]\\displaystyle\\leq\\frac\{1\}\{\\sqrt\{1\-r^\{2\}\}\}\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\|\\xi\|\\left\(1\+r\+\\sqrt\{1\-r^\{2\}\}\|\\xi\|\\right\)\\right\]\(594\)=2π​1\+r1−r2\+1\.\\displaystyle=\\sqrt\{\\frac\{2\}\{\\pi\}\}\\frac\{1\+r\}\{\\sqrt\{1\-r^\{2\}\}\}\+1\.\(595\)Using that∫0111−r2​𝑑r=π2\\int\_\{0\}^\{1\}\\frac\{1\}\{\\sqrt\{1\-r^\{2\}\}\}dr=\\frac\{\\pi\}\{2\}and∫01r1−r2​𝑑r=1\\int\_\{0\}^\{1\}\\frac\{r\}\{\\sqrt\{1\-r^\{2\}\}\}dr=1, we find

∫011r​‖θ​Qr′​\(⟨⋅,θ⟩\)‖ℬϕ​𝑑r≤2π​\(π2\+1\)\+1\.\\int\_\{0\}^\{1\}\\frac\{1\}\{r\}\\\|\\theta Q\_\{r\}^\{\\prime\}\(\\left\\langle\\cdot,\\theta\\right\\rangle\)\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}dr\\leq\\sqrt\{\\frac\{2\}\{\\pi\}\}\\left\(\\frac\{\\pi\}\{2\}\+1\\right\)\+1\.\(596\)The tilt bounds then follow for theϕ⁡\(z\)=sin⁡\(λ​z\)\\phi\(z\)=\\sin\(\\lambda z\)andϕ⁡\(z\)=tanh⁡\(λ​z\)\\phi\(z\)=\\tanh\(\\lambda z\)cases from Lemma[17](https://arxiv.org/html/2610.10875#Thmlemma17)\.

For the ReLU caseϕ⁡\(z\)=max⁡\{0,z\}\\phi\(z\)=\\max\\\{0,z\\\}, we use the fact that iff:ℝ→ℝf:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}isC2C^\{2\}with

limx→−∞f⁡\(x\)=limx→−∞f′​\(x\)=0,∫ℝ\(1\+\|x\|\)​\|f′′​\(x\)\|​𝑑x<∞\\lim\_\{x\\to\-\\infty\}f\(x\)=\\lim\_\{x\\to\-\\infty\}f^\{\\prime\}\(x\)=0,\\quad\\int\_\{\\mathbb\{R\}\}\(1\+\|x\|\)\|f^\{\\prime\\prime\}\(x\)\|dx<\\infty\(597\)then there is the representation

f⁡\(x\)=∫ℝmax⁡\{0,x−a\}​f′′​\(a\)​𝑑a\.f\(x\)=\\int\_\{\\mathbb\{R\}\}\\max\\\{0,x\-a\\\}f^\{\\prime\\prime\}\(a\)da\.\(598\)This comes from the identification ofϕ\\phias the Green’s function of the 1D Laplacian, i\.e\.

∂x2ϕ⁡\(x−a\)=∂x2\[max⁡\{0,x−a\}\]=∂x\[𝟙x\>a\]=δx−a\\partial\_\{x\}^\{2\}\\phi\(x\-a\)=\\partial^\{2\}\_\{x\}\\left\[\\max\\\{0,x\-a\\\}\\right\]=\\partial\_\{x\}\\left\[\{\\mathbbm\{1\}\}\_\{x\>a\}\\right\]=\\delta\_\{x\-a\}\(599\)in the sense of distributions\. In particular, we have

‖f‖ℬϕ≤∫ℝ\(1\+\|a\|\)​\|f′′​\(a\)\|​𝑑a\.\\\|f\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\int\_\{\\mathbb\{R\}\}\(1\+\|a\|\)\|f^\{\\prime\\prime\}\(a\)\|da\.\(600\)Hence, if we note that𝔼ξ∼𝒩⁡\(0,1\)​max⁡\{0,ξ\}=12​π\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\max\\\{0,\\xi\\\}=\\frac\{1\}\{\\sqrt\{2\\pi\}\}and we define the remainder

Rr,ε​\(t\)\\displaystyle R\_\{r,\\varepsilon\}\(t\):=Qr′​\(t\)r⁡\(1\+ε​Qr​\(t\)−ε2​π\)−Qr′​\(t\)r\\displaystyle:=\\frac\{Q^\{\\prime\}\_\{r\}\(t\)\}\{r\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)\}\-\\frac\{Q\_\{r\}^\{\\prime\}\(t\)\}\{r\}\(601\)=Qr′​\(t\)r⋅\(11\+ε​Qr​\(t\)−ε2​π−1\),\\displaystyle=\\frac\{Q^\{\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\left\(\\frac\{1\}\{1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\-1\\right\),\(602\)then the same argument of \([596](https://arxiv.org/html/2610.10875#A1.E596)\) yields

𝔉ϕtilt≤π2\+2π\+1\+lim supε↓0∫01∫ℝ\(1\+\|a\|\)​\|Rr,ε′′​\(a\)\|​𝑑a​𝑑r,\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\\leq\\sqrt\{\\frac\{\\pi\}\{2\}\}\+\\sqrt\{\\frac\{2\}\{\\pi\}\}\+1\+\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}\\int\_\{\\mathbb\{R\}\}\(1\+\|a\|\)\|R\_\{r,\\varepsilon\}^\{\\prime\\prime\}\(a\)\|dadr,\(603\)where we use thatlimt→−∞Rr,ε​\(t\)=limt→−∞Rr,ε′​\(t\)=0\\lim\_\{t\\to\-\\infty\}R\_\{r,\\varepsilon\}\(t\)=\\lim\_\{t\\to\-\\infty\}R^\{\\prime\}\_\{r,\\varepsilon\}\(t\)=0sinceQr​\(t\),Qr′​\(t\),Qr′′​\(t\)→0Q\_\{r\}\(t\),Q^\{\\prime\}\_\{r\}\(t\),Q^\{\\prime\\prime\}\_\{r\}\(t\)\\to 0ast→−∞t\\to\-\\inftyby dominated convergence\. By the product rule,

Rr,ε′​\(t\)=Qr′′​\(t\)r⋅\(11\+ε​Qr​\(t\)−ε2​π−1\)−Qr′​\(t\)r⋅ε​Qr′​\(t\)\(1\+ε​Qr​\(t\)−ε2​π\)2R^\{\\prime\}\_\{r,\\varepsilon\}\(t\)=\\frac\{Q^\{\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\left\(\\frac\{1\}\{1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\-1\\right\)\-\\frac\{Q^\{\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\frac\{\\varepsilon Q^\{\\prime\}\_\{r\}\(t\)\}\{\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{2\}\}\(604\)and similarly

Rr,ε′′​\(t\)\\displaystyle R^\{\\prime\\prime\}\_\{r,\\varepsilon\}\(t\)=Qr′′′​\(t\)r⋅\(11\+ε​Qr​\(t\)−ε2​π−1\)−Qr′′​\(t\)r⋅ε​Qr′​\(t\)\(1\+ε​Qr​\(t\)−ε2​π\)2\\displaystyle=\\frac\{Q^\{\\prime\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\left\(\\frac\{1\}\{1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\-1\\right\)\-\\frac\{Q^\{\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\frac\{\\varepsilon Q^\{\\prime\}\_\{r\}\(t\)\}\{\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{2\}\}\(605\)−Qr′′​\(t\)r⋅ε​Qr′​\(t\)\(1\+ε​Qr​\(t\)−ε2​π\)2\\displaystyle\\quad\-\\frac\{Q^\{\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\frac\{\\varepsilon Q^\{\\prime\}\_\{r\}\(t\)\}\{\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{2\}\}\(606\)−Qr′​\(t\)r⋅\(1\+ε​Qr​\(t\)−ε2​π\)2​ε​Qr′′​\(t\)−2​ε2​\(Qr′​\(t\)\)2​\(1\+ε​Qr​\(t\)−ε2​π\)\(1\+ε​Qr​\(t\)−ε2​π\)4\\displaystyle\\quad\-\\frac\{Q^\{\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\frac\{\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{2\}\\varepsilon Q^\{\\prime\\prime\}\_\{r\}\(t\)\-2\\varepsilon^\{2\}\(Q\_\{r\}^\{\\prime\}\(t\)\)^\{2\}\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)\}\{\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{4\}\}\(607\)=Qr′′′​\(t\)r⋅\(11\+ε​Qr​\(t\)−ε2​π−1\)−3​Qr′′​\(t\)r⋅ε​Qr′​\(t\)\(1\+ε​Qr​\(t\)−ε2​π\)2\\displaystyle=\\frac\{Q^\{\\prime\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\left\(\\frac\{1\}\{1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\-1\\right\)\-\\frac\{3Q^\{\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\frac\{\\varepsilon Q^\{\\prime\}\_\{r\}\(t\)\}\{\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{2\}\}\(608\)\+2​ε2​\(Qr′​\(t\)\)3r​\(1\+ε​Qr​\(t\)−ε2​π\)3\.\\displaystyle\\quad\+\\frac\{2\\varepsilon^\{2\}\(Q\_\{r\}^\{\\prime\}\(t\)\)^\{3\}\}\{r\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{3\}\}\.\(609\)This splitsRr,ε′′R\_\{r,\\varepsilon\}^\{\\prime\\prime\}into three terms\. Using thatQr​\(t\)≥0Q\_\{r\}\(t\)\\geq 0and so1\+ε​Qr​\(t\)−ε2​π≥121\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\geq\\frac\{1\}\{2\}forε≤π2\\varepsilon\\leq\\sqrt\{\\frac\{\\pi\}\{2\}\}, we find that

\|Qr′′′​\(t\)r⋅\(11\+ε​Qr​\(t\)−ε2​π−1\)\|\\displaystyle\\left\|\\frac\{Q^\{\\prime\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\left\(\\frac\{1\}\{1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\-1\\right\)\\right\|=\|Qr′′′​\(t\)r⋅ε​\(12​π−Qr​\(t\)\)1\+ε​Qr​\(t\)−ε2​π\|\\displaystyle=\\left\|\\frac\{Q\_\{r\}^\{\\prime\\prime\\prime\}\(t\)\}\{r\}\\cdot\\frac\{\\varepsilon\\left\(\\frac\{1\}\{\\sqrt\{2\\pi\}\}\-Q\_\{r\}\(t\)\\right\)\}\{1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|\(610\)≤2​εr⋅\|Qr′′′​\(t\)\|⋅\(12​π\+Qr​\(t\)\)\\displaystyle\\leq\\frac\{2\\varepsilon\}\{r\}\\cdot\|Q\_\{r\}^\{\\prime\\prime\\prime\}\(t\)\|\\cdot\\left\(\\frac\{1\}\{\\sqrt\{2\\pi\}\}\+Q\_\{r\}\(t\)\\right\)\(611\)and that

\|3​Qr′′​\(t\)r⋅ε​Qr′​\(t\)\(1\+ε​Qr​\(t\)−ε2​π\)2\|≤12​εr⋅\|Qr′′​\(t\)\|⋅\|Qr′​\(t\)\|\\left\|\\frac\{3Q^\{\\prime\\prime\}\_\{r\}\(t\)\}\{r\}\\cdot\\frac\{\\varepsilon Q^\{\\prime\}\_\{r\}\(t\)\}\{\\left\(1\+\\varepsilon Q\_\{r\}\(t\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{2\}\}\\right\|\\leq\\frac\{12\\varepsilon\}\{r\}\\cdot\\left\|Q\_\{r\}^\{\\prime\\prime\}\(t\)\\right\|\\cdot\\left\|Q\_\{r\}^\{\\prime\}\(t\)\\right\|\(612\)Forϕ⁡\(z\)=max⁡\{0,z\}\\phi\(z\)=\\max\\\{0,z\\\}we may compute that

Qr′​\(t\)=r​Φ​\(r​t1−r2\),Qr′′​\(t\)=r21−r2​φ​\(r​t1−r2\),Q\_\{r\}^\{\\prime\}\(t\)=r\\Phi\\left\(\\frac\{rt\}\{\\sqrt\{1\-r^\{2\}\}\}\\right\),\\quad Q\_\{r\}^\{\\prime\\prime\}\(t\)=\\frac\{r^\{2\}\}\{\\sqrt\{1\-r^\{2\}\}\}\\varphi\\left\(\\frac\{rt\}\{\\sqrt\{1\-r^\{2\}\}\}\\right\),\(613\)Qr′′′​\(t\)=−r4​t\(1−r2\)3/2​φ​\(r​t1−r2\),Qr​\(t\)≤r​\|t\|\+1−r22​πQ\_\{r\}^\{\\prime\\prime\\prime\}\(t\)=\-\\frac\{r^\{4\}t\}\{\(1\-r^\{2\}\)^\{3/2\}\}\\varphi\\left\(\\frac\{rt\}\{\\sqrt\{1\-r^\{2\}\}\}\\right\),\\quad Q\_\{r\}\(t\)\\leq r\|t\|\+\\frac\{\\sqrt\{1\-r^\{2\}\}\}\{\\sqrt\{2\\pi\}\}\(614\)forΦ\\Phiandφ\\varphithe Gaussian CDF and PDF, respectively\. Hence,

1r​∫ℝ\(1\+\|t\|\)​\(\|Qr′′′​\(t\)\|⋅\(12​π\+Qr​\(t\)\)\+\|Qr′′​\(t\)\|⋅\|Qr′​\(t\)\|\)​𝑑t≤C⁡\(1\+r\+r1−r2\+1−r2\)\\frac\{1\}\{r\}\\int\_\{\\mathbb\{R\}\}\(1\+\|t\|\)\\left\(\|Q\_\{r\}^\{\\prime\\prime\\prime\}\(t\)\|\\cdot\\left\(\\frac\{1\}\{\\sqrt\{2\\pi\}\}\+Q\_\{r\}\(t\)\\right\)\+\|Q\_\{r\}^\{\\prime\\prime\}\(t\)\|\\cdot\|Q\_\{r\}^\{\\prime\}\(t\)\|\\right\)dt\\leq C\\left\(1\+r\+\\frac\{r\}\{\\sqrt\{1\-r^\{2\}\}\}\+\\sqrt\{1\-r^\{2\}\}\\right\)\(615\)and since∫01r1−r2​𝑑r<∞\\int\_\{0\}^\{1\}\\frac\{r\}\{\\sqrt\{1\-r^\{2\}\}\}dr<\\inftywe find

∫011r​∫ℝ\(1\+\|t\|\)​\(\|Qr′′′​\(t\)\|⋅\(12​π\+Qr​\(t\)\)\+\|Qr′′​\(t\)\|⋅\|Qr′​\(t\)\|\)​𝑑t​𝑑r<∞\.\\int\_\{0\}^\{1\}\\frac\{1\}\{r\}\\int\_\{\\mathbb\{R\}\}\(1\+\|t\|\)\\left\(\|Q\_\{r\}^\{\\prime\\prime\\prime\}\(t\)\|\\cdot\\left\(\\frac\{1\}\{\\sqrt\{2\\pi\}\}\+Q\_\{r\}\(t\)\\right\)\+\|Q\_\{r\}^\{\\prime\\prime\}\(t\)\|\\cdot\|Q\_\{r\}^\{\\prime\}\(t\)\|\\right\)dtdr<\\infty\.\(616\)This implies that the first two terms in the expansion ofRr,ε′′R^\{\\prime\\prime\}\_\{r,\\varepsilon\}stay first order inε\\varepsilon, and do not contribute to \([603](https://arxiv.org/html/2610.10875#A1.E603)\), i\.e\.

𝔉ϕtilt≤π2\+2π\+1\+lim supε↓0∫012​ε2r​∫ℝ\(1\+\|a\|\)⋅\|Qr′​\(a\)1\+ε​Qr​\(a\)−ε2​π\|3​𝑑a​𝑑r\.\\mathfrak\{F\}\_\{\\phi\}^\{\\operatorname\{tilt\}\}\\leq\\sqrt\{\\frac\{\\pi\}\{2\}\}\+\\sqrt\{\\frac\{2\}\{\\pi\}\}\+1\+\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}\\frac\{2\\varepsilon^\{2\}\}\{r\}\\int\_\{\\mathbb\{R\}\}\(1\+\|a\|\)\\cdot\\left\|\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|^\{3\}dadr\.\(617\)Now, sinceQr′\>0Q\_\{r\}^\{\\prime\}\>0forr∈\(0,1\)r\\in\(0,1\)andQr​\(−∞\)=0Q\_\{r\}\(\-\\infty\)=0,Qr​\(∞\)=∞Q\_\{r\}\(\\infty\)=\\infty, we may substituteu=Qr​\(a\)−12​πu=Q\_\{r\}\(a\)\-\\frac\{1\}\{\\sqrt\{2\\pi\}\}\(withd​u=Qr′​\(a\)​d​adu=Q^\{\\prime\}\_\{r\}\(a\)da\) to find that

∫ℝ\|Qr′​\(a\)1\+ε​Qr​\(a\)−ε2​π\|3​𝑑a\\displaystyle\\int\_\{\\mathbb\{R\}\}\\left\|\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|^\{3\}da≤r2​∫ℝQr′​\(a\)\|1\+ε​Qr​\(a\)−ε2​π\|3​𝑑a\\displaystyle\\leq r^\{2\}\\int\_\{\\mathbb\{R\}\}\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{\\left\|1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\|^\{3\}\}da\(618\)=r2​∫−12​π∞1\(1\+ε​u\)3​𝑑u\\displaystyle=r^\{2\}\\int\_\{\-\\frac\{1\}\{\\sqrt\{2\\pi\}\}\}^\{\\infty\}\\frac\{1\}\{\(1\+\\varepsilon u\)^\{3\}\}du\(619\)=r2​12​ε​\(1−ε2​π\)2\\displaystyle=r^\{2\}\\frac\{1\}\{2\\varepsilon\\left\(1\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{2\}\}\(620\)≤2​r2ε,\\displaystyle\\leq\\frac\{2r^\{2\}\}\{\\varepsilon\},\(621\)where we used thatQr′​\(a\)≤rQ\_\{r\}^\{\\prime\}\(a\)\\leq randε2​π≤12\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\leq\\frac\{1\}\{2\}\. In particular,

lim supε↓0∫012​ε2r​∫ℝ\|Qr′​\(a\)1\+ε​Qr​\(a\)−ε2​π\|3​𝑑a​𝑑r≤lim supε↓0∫014​ε​r​𝑑r=0\.\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}\\frac\{2\\varepsilon^\{2\}\}\{r\}\\int\_\{\\mathbb\{R\}\}\\left\|\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|^\{3\}dadr\\leq\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}4\\varepsilon rdr=0\.\(622\)We split the remaining integral intoa<0a<0anda≥0a\\geq 0\. Indeed, by using0≤Qr′​\(a\)≤r0\\leq Q\_\{r\}^\{\\prime\}\(a\)\\leq rand lower bounding the denominator we have

∫−∞0\|a\|⋅\|Qr′​\(a\)1\+ε​Qr​\(a\)−ε2​π\|3da≤−8r2∫−∞0aQr′\(a\)da=8r2∫−∞0Qr\(a\)da\\int\_\{\-\\infty\}^\{0\}\|a\|\\cdot\\left\|\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|^\{3\}da\\leq\-8r^\{2\}\\int\_\{\-\\infty\}^\{0\}aQ\_\{r\}^\{\\prime\}\(a\)da=8r^\{2\}\\int\_\{\-\\infty\}^\{0\}Q\_\{r\}\(a\)da\(623\)using integration by parts\. We calculate

∫−∞0Qr​\(a\)​𝑑a\\displaystyle\\int\_\{\-\\infty\}^\{0\}Q\_\{r\}\(a\)da=𝔼ξ∼𝒩⁡\(0,1\)\[𝟙ξ\>0∫−1−r2ξ/r0\(ra\+1−r2ξ\)da\]\\displaystyle=\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\{\\mathbbm\{1\}\}\_\{\\xi\>0\}\\int\_\{\-\\sqrt\{1\-r^\{2\}\}\\xi/r\}^\{0\}\\left\(ra\+\\sqrt\{1\-r^\{2\}\}\\xi\\right\)da\\right\]\(624\)=1−r22​r​𝔼ξ∼𝒩⁡\(0,1\)​\[ξ2​𝟙ξ\>0\]≤14​r,\\displaystyle=\\frac\{1\-r^\{2\}\}\{2r\}\\mathbb\{E\}\_\{\\xi\\sim\\mathcal\{N\}\(0,1\)\}\\left\[\\xi^\{2\}\{\\mathbbm\{1\}\}\_\{\\xi\>0\}\\right\]\\leq\\frac\{1\}\{4r\},\(625\)which means that

lim supε↓0∫012​ε2r​∫−∞0\|a\|⋅\|Qr′​\(a\)1\+ε​Qr​\(a\)−ε2​π\|3​𝑑a​𝑑r≤lim supε↓0∫014​ε2​𝑑r=0\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}\\frac\{2\\varepsilon^\{2\}\}\{r\}\\int\_\{\-\\infty\}^\{0\}\|a\|\\cdot\\left\|\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|^\{3\}dadr\\leq\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}4\\varepsilon^\{2\}dr=0\(626\)and so the only contribution comes froma≥0a\\geq 0\. Ifa≥0a\\geq 0then we knowQr​\(a\)≥r​aQ\_\{r\}\(a\)\\geq raand so

\|a\|⋅\|Qr′​\(a\)1\+ε​Qr​\(a\)−ε2​π\|3≤a​r3\(1\+ε​r​a−ε2​π\)3≤a​r3\(12\+ε​r​a\)3≤8​a​r3\(1\+ε​r​a\)3\|a\|\\cdot\\left\|\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|^\{3\}\\leq\\frac\{ar^\{3\}\}\{\\left\(1\+\\varepsilon ra\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\right\)^\{3\}\}\\leq\\frac\{ar^\{3\}\}\{\\left\(\\frac\{1\}\{2\}\+\\varepsilon ra\\right\)^\{3\}\}\\leq\\frac\{8ar^\{3\}\}\{\(1\+\\varepsilon ra\)^\{3\}\}\(627\)since0≤Qr′​\(a\)≤r0\\leq Q\_\{r\}^\{\\prime\}\(a\)\\leq randε2​π≤12\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\\leq\\frac\{1\}\{2\}\. Hence,

lim supε↓0∫012​ε2r​∫0∞\|a\|⋅\|Qr′​\(a\)1\+ε​Qr​\(a\)−ε2​π\|3​𝑑a​𝑑r\\displaystyle\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}\\frac\{2\\varepsilon^\{2\}\}\{r\}\\int\_\{0\}^\{\\infty\}\|a\|\\cdot\\left\|\\frac\{Q\_\{r\}^\{\\prime\}\(a\)\}\{1\+\\varepsilon Q\_\{r\}\(a\)\-\\frac\{\\varepsilon\}\{\\sqrt\{2\\pi\}\}\}\\right\|^\{3\}dadr≤lim supε↓0∫0116​ε2​r2​∫0∞a\(1\+ε​r​a\)3​𝑑a​𝑑r\\displaystyle\\leq\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}16\\varepsilon^\{2\}r^\{2\}\\int\_\{0\}^\{\\infty\}\\frac\{a\}\{\(1\+\\varepsilon ra\)^\{3\}\}dadr\(628\)=lim supε↓0∫0116​ε2​r2⋅12​ε2​r2​𝑑r\\displaystyle=\\limsup\_\{\\varepsilon\\downarrow 0\}\\int\_\{0\}^\{1\}16\\varepsilon^\{2\}r^\{2\}\\cdot\\frac\{1\}\{2\\varepsilon^\{2\}r^\{2\}\}dr\(629\)=8\.\\displaystyle=8\.\(630\)∎

###### Proof of Example[1](https://arxiv.org/html/2610.10875#Thmexample1)\.

We combine Lemmas[14](https://arxiv.org/html/2610.10875#Thmlemma14)and[16](https://arxiv.org/html/2610.10875#Thmlemma16)with Proposition[8](https://arxiv.org/html/2610.10875#Thmproposition8)to find that if\|⟨u,θ⟩\|<1\|\\left\\langle u,\\theta\\right\\rangle\|<1the flow ratios are infinite and the tilt ratios are always bounded above by

\(9\+2π\+π2\)⋅1\|𝔼𝒩⁡\(0,1\)​\[ϕ′​\(ξ\)\]\|\\left\(9\+\\sqrt\{\\frac\{2\}\{\\pi\}\}\+\\sqrt\{\\frac\{\\pi\}\{2\}\}\\right\)\\cdot\\frac\{1\}\{\\left\|\\mathbb\{E\}\_\{\\mathcal\{N\}\(0,1\)\}\[\\phi^\{\\prime\}\(\\xi\)\]\\right\|\}\(631\)sinceϕ⁡\(0\)=0\\phi\(0\)=0andLip⁡\(ϕ\)≤1\\operatorname\{Lip\}\(\\phi\)\\leq 1forϕ⁡\(z\)=sin⁡\(z\)\\phi\(z\)=\\sin\(z\),ϕ⁡\(z\)=tanh⁡\(z\)\\phi\(z\)=\\tanh\(z\), andϕ⁡\(z\)=max⁡\{0,z\}\\phi\(z\)=\\max\\\{0,z\\\}\. In either case, we may confirm that\|𝔼𝒩⁡\(0,1\)​\[ϕ′​\(ξ\)\]\|≥12\\left\|\\mathbb\{E\}\_\{\\mathcal\{N\}\(0,1\)\}\[\\phi^\{\\prime\}\(\\xi\)\]\\right\|\\geq\\frac\{1\}\{2\}, from which the claim follows\. ∎

### A\.7Reverse OU suboptimality for triangular shear measures

In the main paper, we present lower bounds for the representation and approximation by two\-layer neural networks of the OU scores of certain ResNet pushforwards\. These proof techniques extend to any measure with a triangular shear structure, which may be of wider interest\. We study the general case in this section, and in Section[A\.8](https://arxiv.org/html/2610.10875#A1.SS8)we specialize these general lower bounds to the ResNet pushforward examples\.

###### Definition 6\(Triangular shear measure\)\.

Fixn∈ℕn\\in\{\\mathbb\{N\}\}and letψ:ℝ→ℝ\\psi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}andΨ:ℝn→ℝ\\Psi:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}be measurable\. Using coordinates\(z,u\)∈ℝn\+1\(z,u\)\\in\{\\mathbb\{R\}\}^\{n\+1\}withz∈ℝnz\\in\{\\mathbb\{R\}\}^\{n\}andu∈ℝu\\in\{\\mathbb\{R\}\}, we define the induced triangular shear measureπψ,Ψ∈𝒫⁡\(ℝn\+1\)\\pi^\{\\psi,\\Psi\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{n\+1\}\)by

πψ,Ψ:=Law⁡\(Z,ψ⁡\(U\)\+Ψ⁡\(Z\)\),\(Z,U\)∼γn\+1\.\\pi^\{\\psi,\\Psi\}:=\\operatorname\{Law\}\(Z,\\psi\(U\)\+\\Psi\(Z\)\),\\quad\(Z,U\)\\sim\\gamma\_\{n\+1\}\.\(632\)

We recall the following preliminaries from Section[2\.3](https://arxiv.org/html/2610.10875#S2.SS3)\. The OU semigroup\(Pτ\)τ≥0\(P\_\{\\tau\}\)\_\{\\tau\\geq 0\}and its adjoint\(Pτ∗\)τ≥0\(P\_\{\\tau\}^\{\*\}\)\_\{\\tau\\geq 0\}act on functionsf:ℝn→ℝf:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}and probability measuresρ∈𝒫⁡\(ℝn\)\\rho\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{n\}\), respectively, by

\(Pτ​f\)​\(x\):=𝔼Z∼γn​\[f⁡\(e−τ​x\+1−e−2​τ​Z\)\],Pτ∗​ρ:=Law⁡\(e−τ​X\+1−e−2​τ​Z\)\(P\_\{\\tau\}f\)\(x\):=\\mathbb\{E\}\_\{Z\\sim\\gamma\_\{n\}\}\\left\[f\\left\(e^\{\-\\tau\}x\+\\sqrt\{1\-e^\{\-2\\tau\}\}Z\\right\)\\right\],\\quad P\_\{\\tau\}^\{\*\}\\rho:=\\operatorname\{Law\}\\left\(e^\{\-\\tau\}X\+\\sqrt\{1\-e^\{\-2\\tau\}\}Z\\right\)\(633\)withX∼ρX\\sim\\rhoandZ∼γnZ\\sim\\gamma\_\{n\}independently\. In this section we will use the same notation for the corresponding semigroups onℝn\+1\{\\mathbb\{R\}\}^\{n\+1\}, with the dimensions clear from context\.

Ifρτψ,Ψ=Pτ∗​πψ,Ψ\\rho\_\{\\tau\}^\{\\psi,\\Psi\}=P\_\{\\tau\}^\{\*\}\\pi^\{\\psi,\\Psi\}denotes the OU flow initialized fromπψ,Ψ\\pi^\{\\psi,\\Psi\}and we define the time\-dependent scores relative to Lebesgue and the Gaussian forτ\>0\\tau\>0by

sτψ,Ψ:=∇log⁡ρτψ,Ψ,bτψ,Ψ:=∇log⁡ρτψ,Ψγn\+1=sτψ,Ψ\+Idn\+1,s\_\{\\tau\}^\{\\psi,\\Psi\}:=\\nabla\\log\\rho\_\{\\tau\}^\{\\psi,\\Psi\},\\quad b\_\{\\tau\}^\{\\psi,\\Psi\}:=\\nabla\\log\\frac\{\\rho\_\{\\tau\}^\{\\psi,\\Psi\}\}\{\\gamma\_\{n\+1\}\}=s\_\{\\tau\}^\{\\psi,\\Psi\}\+\\operatorname\{Id\}\_\{n\+1\},\(634\)respectively, then\(vtψ,Ψ,ρ~tψ,Ψ\)t∈\[0,1\]\(v\_\{t\}^\{\\psi,\\Psi\},\\tilde\{\\rho\}\_\{t\}^\{\\psi,\\Psi\}\)\_\{t\\in\[0,1\]\}solves the continuity equation withρ~0ψ,Ψ=γn\+1\\tilde\{\\rho\}\_\{0\}^\{\\psi,\\Psi\}=\\gamma\_\{n\+1\},ρ~1ψ,Ψ=πψ,Ψ\\tilde\{\\rho\}\_\{1\}^\{\\psi,\\Psi\}=\\pi^\{\\psi,\\Psi\}and

vtψ,Ψ:=1tb−log⁡tψ,Ψ,ρ~tψ,Ψ:=ρ−log⁡tψ,Ψ\(∀t∈\(0,1\)\)\.v\_\{t\}^\{\\psi,\\Psi\}:=\\frac\{1\}\{t\}b\_\{\-\\log t\}^\{\\psi,\\Psi\},\\quad\\tilde\{\\rho\}\_\{t\}^\{\\psi,\\Psi\}:=\\rho\_\{\-\\log t\}^\{\\psi,\\Psi\}\\quad\(\\forall t\\in\(0,1\)\)\.\(635\)Hence,vtψ,Ψv\_\{t\}^\{\\psi,\\Psi\}is the standard probability flow denoising generative model andbτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}is a time\-change of it\. Lastly, we recall our definition of the reverse OU energy for a Banach norm∥⋅∥ℬ\\\|\\cdot\\\|\_\{\\mathcal\{B\}\}to be

ℰℬ​\(πψ,Ψ\)=∫01‖vtψ,Ψ‖ℬ​𝑑t=∫0∞‖bτψ,Ψ‖ℬ​𝑑τ,\\mathcal\{E\}\_\{\\mathcal\{B\}\}\(\\pi^\{\\psi,\\Psi\}\)=\\int\_\{0\}^\{1\}\\\|v\_\{t\}^\{\\psi,\\Psi\}\\\|\_\{\\mathcal\{B\}\}dt=\\int\_\{0\}^\{\\infty\}\\\|b\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\mathcal\{B\}\}d\\tau,\(636\)where we takeℬ=ℬϕ\\mathcal\{B\}=\\mathcal\{B\}\_\{\\phi\}orℬ=ℬϕR\\mathcal\{B\}=\\mathcal\{B\}\_\{\\phi\}^\{R\}, depending on the situation\.

In this section, we prove lower bounds for the representation and approximation of the generative driftsvtψ,Ψv\_\{t\}^\{\\psi,\\Psi\}using two\-layer neural networks in terms of the properties of the maps\(ψ,Ψ\)\(\\psi,\\Psi\)which generate the shear\. These lower bounds will also hold for the Lebesgue and Gaussian scoressτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}andbτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}and therefore apply similarly to SDE\-based denoising generative models, other time parameterizations, variance\-exploding formulations, etc\. We summarize these results here:

###### Theorem 5\(Shallow representation and approximation lower bounds, quantitative\)\.

Fixψ,Ψ\\psi,\\Psiand definesτψ,Ψ,bτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\},b\_\{\\tau\}^\{\\psi,\\Psi\}as above\. Then, the following hold:

1. \(i\)IfΨ∈Cb5​\(ℝn\)\\Psi\\in C^\{5\}\_\{b\}\(\{\\mathbb\{R\}\}^\{n\}\)andψ\\psiis increasing and equal to the identity on\[−2​‖Ψ‖∞,2​‖Ψ‖∞\]\[\-2\\\|\\Psi\\\|\_\{\\infty\},2\\\|\\Psi\\\|\_\{\\infty\}\], then ‖∂13sτψ,Ψ‖∞≳‖∂13Ψ‖∞for all0<τ≲‖∂13Ψ‖∞n​‖Ψ‖C5\.\\\|\\partial\_\{1\}^\{3\}s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\infty\}\\gtrsim\\\|\\partial\_\{1\}^\{3\}\\Psi\\\|\_\{\\infty\}\\quad\\text\{ for all $0<\\tau\\lesssim\\frac\{\\\|\\partial\_\{1\}^\{3\}\\Psi\\\|\_\{\\infty\}\}\{n\\\|\\Psi\\\|\_\{C^\{5\}\}\}$\}\.\(637\)In particular, ifϕ∈Cb3​\(ℝ\)\\phi\\in C^\{3\}\_\{b\}\(\{\\mathbb\{R\}\}\)theninfR\>0R2​‖sτψ,Ψ‖ℬϕR≳‖∂13Ψ‖∞\\inf\_\{R\>0\}R^\{2\}\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}^\{R\}\}\\gtrsim\\\|\\partial\_\{1\}^\{3\}\\Psi\\\|\_\{\\infty\}, while ifϕ\\phiis positively33\-homogeneous then‖sτψ,Ψ‖ℬϕ≳‖∂13Ψ‖∞\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\gtrsim\\\|\\partial\_\{1\}^\{3\}\\Psi\\\|\_\{\\infty\}\. The implied constants depend only on‖Ψ‖∞\\\|\\Psi\\\|\_\{\\infty\}\.
2. \(ii\)LetΨ\\Psibe bounded andLL\-Lipschitz andψ\\psilinear with positive slope, and fixτ≥0\\tau\\geq 0,ε\>0\\varepsilon\>0\. Ifs^τ\\hat\{s\}\_\{\\tau\}is a two\-layer neural network of widthmm, then ‖sτψ,Ψ−s^τ‖L2​\(ρτψ,Ψ,ℝn\+1\)≤ε⟹m≥mΨ​\(Q⁡\(ε\+L2​τ\),τ\),\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\leq\\varepsilon\\implies m\\geq m\_\{\\Psi\}\(Q\(\\varepsilon\+L^\{2\}\\tau\),\\tau\),\(638\)wheremΨm\_\{\\Psi\}is defined in Definition[8](https://arxiv.org/html/2610.10875#Thmdefinition8)using the Hermite decomposition ofΨ\\PsiandQQis a constant depending only on‖Ψ‖∞\\\|\\Psi\\\|\_\{\\infty\}andψ\\psi\. In such cases, for anyϕ\\phisatisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)we have ‖s^τ‖ℬϕ≳ε​mΨ​\(Q⁡\(2​ε\+L2​τ\),τ\)−1\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\gtrsim\\varepsilon\\sqrt\{m\_\{\\Psi\}\(Q\(2\\varepsilon\+L^\{2\}\\tau\),\\tau\)\-1\}\(639\)with implied constant depending onϕ,‖Ψ‖∞,ψ\\phi,\\\|\\Psi\\\|\_\{\\infty\},\\psi\.

These statements also hold withsτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}replaced bybτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\.

In words, our representation result \(Corollary[2](https://arxiv.org/html/2610.10875#Thmcorollary2)\) lower bounds the third derivatives of the scores \(and therefore the Barron norm for sufficiently regularϕ\\phi\) in terms of the derivatives ofΨ\\Psi\. The approximation result \(Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9)\) is stronger, as it leverages the spectral tails of the Hermite tensor decomposition ofΨ\\Psiin order to lower bound the width of any shallowL2L^\{2\}approximation; this can be converted to a lower bound on the Barron norm of the approximation\.

In both cases we allow the added flexibility ofψ\\psidifferent from the identity: in the representation case we allowψ\\psito have nonlinear tails \(in which case the last coordinate ofπψ,Ψ\\pi^\{\\psi,\\Psi\}has non\-Gaussian tails\), and in the approximation case we allowψ\\psito be a scalar multiple of the identity \(in which case the last coordinate ofπψ,Ψ\\pi^\{\\psi,\\Psi\}isΨ⁡\(Z\)\\Psi\(Z\)smoothed by a Gaussian of any fixed variance\)\.

To help interpret these lower bounds, we also demonstrate some settings where the corresponding upper bounds hold in a nonquantitative sense:

###### Theorem 6\(Shallow representation and approximation upper bounds, nonquantitative\)\.

Fixψ,Ψ\\psi,\\Psiand definesτψ,Ψ,bτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\},b\_\{\\tau\}^\{\\psi,\\Psi\}as above\. Then, the following hold:

1. \(i\)IfΨ⁡\(z\)=∑r=1MΦr​\(PEr​z\)\\Psi\(z\)=\\sum\_\{r=1\}^\{M\}\\Phi\_\{r\}\(P\_\{E\_\{r\}\}z\)for subspacesEr⊆ℝnE\_\{r\}\\subseteq\{\\mathbb\{R\}\}^\{n\}withΦr∈Cc∞​\(Er\)\\Phi\_\{r\}\\in C^\{\\infty\}\_\{c\}\(E\_\{r\}\)andψ\\psiis a smooth increasing diffeomorphism withψ⁡\(u\)∼up\\psi\(u\)\\sim u^\{p\}outside of a compact set for somep\>2p\>2, thensτψ,Ψ∈ℬReLUss\_\{\\tau\}^\{\\psi,\\Psi\}\\in\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{s\}\}for anys∈ℕs\\in\{\\mathbb\{N\}\}and allτ≥0\\tau\\geq 0\.
2. \(ii\)For anyψ,Ψ\\psi,\\Psi, anyε,τ\>0\\varepsilon,\\tau\>0, and any nonpolynomial activationϕ\\phi, there is some two\-layer neural networks^τ\\hat\{s\}\_\{\\tau\}of finite width which uses activationϕ\\phiand satisfies‖sτψ,Ψ−s^τ‖L2​\(ρτψ,Ψ,ℝn\+1\)≤ε\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\leq\\varepsilon\.

This first statement also holds withsτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}replaced bybτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}, and the second statement does ifπψ,Ψ∈𝒫2​\(ℝd\)\\pi^\{\\psi,\\Psi\}\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{d\}\)\.

In words, we provide conditions under which the OU\-smoothed scoressτψ,Ψ,bτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\},b\_\{\\tau\}^\{\\psi,\\Psi\}are \(i\) themselves Barron and therefore representable by infinite\-width two\-layer neural networks of finite path norm \(Lemma[21](https://arxiv.org/html/2610.10875#Thmlemma21)\) and \(ii\) are approximable to arbitraryL2L^\{2\}accuracy by finite\-width two\-layer neural networks \(Proposition[10](https://arxiv.org/html/2610.10875#Thmproposition10)\)\. These are proved by a Sobolev\-to\-Barron embedding \(Lemmas[19](https://arxiv.org/html/2610.10875#Thmlemma19)and[20](https://arxiv.org/html/2610.10875#Thmlemma20)\) and by the classic universalL2L^\{2\}approximation result of Hornik\[[44](https://arxiv.org/html/2610.10875#bib.bib21)\], respectively\. We do not provide quantitative upper bounds on the widths or Barron norms; these results merely serve as sanity checks to complement our quantitative lower bounds\.

#### A\.7\.1Representation lower bound via third derivative

We first lower bound the homogeneous Sobolev seminorm∥⋅∥W˙3,∞​\(ℝn\+1,ℝn\+1\)\\\|\\cdot\\\|\_\{\\dot\{W\}^\{3,\\infty\}\(\{\\mathbb\{R\}\}^\{n\+1\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}of the drifts and their integrated valueℰW˙3,∞​\(ℝn\+1,ℝn\+1\)​\(πψ,Ψ\)\\mathcal\{E\}\_\{\\dot\{W\}^\{3,\\infty\}\(\{\\mathbb\{R\}\}^\{n\+1\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\(\\pi^\{\\psi,\\Psi\}\)\. The mechanism is elementary: ifΨ\\Psiis smooth and bounded with large derivatives andψ\\psiis the identity near the origin, then a standard OU stability argument transfers those large derivatives tosτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}for small OU times\.

###### Lemma 18\(Third derivative representation lower bound\)\.

Letn∈ℕn\\in\{\\mathbb\{N\}\}, fixΨ∈Cb5​\(ℝn,ℝ\)\\Psi\\in C^\{5\}\_\{b\}\(\{\\mathbb\{R\}\}^\{n\};\{\\mathbb\{R\}\}\), and letβ,D∈\(0,∞\)\\beta,D\\in\(0,\\infty\)be s\.t\.D\>1D\>1and

∥Ψ∥∞≤β,∥∂αΨ∥∞≤D\|α\|\(∀1≤\|α\|≤5\)\.\\\|\\Psi\\\|\_\{\\infty\}\\leq\\beta,\\quad\\\|\\partial^\{\\alpha\}\\Psi\\\|\_\{\\infty\}\\leq D^\{\|\\alpha\|\}\\quad\(\\forall 1\\leq\|\\alpha\|\\leq 5\)\.\(640\)Fixψ:ℝ→ℝ\\psi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}measurable and nondecreasing s\.t\.ψ⁡\(u\)=u\\psi\(u\)=ufor allu∈\[−2​β,2​β\]u\\in\[\-2\\beta,2\\beta\]\. Then, there is someLβ∈\(0,∞\)L\_\{\\beta\}\\in\(0,\\infty\)s\.t\. for everyη∈\{1,2,3\}\\eta\\in\\\{1,2,3\\\}we have

‖∂1ηsτψ,Ψ‖∞≥‖∂1ηΨ‖∞2\(∀τ∈\(0,Lβ​‖∂1ηΨ‖∞n​Dη\+2\]\)\.\\\|\\partial\_\{1\}^\{\\eta\}s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\infty\}\\geq\\frac\{\\\|\\partial\_\{1\}^\{\\eta\}\\Psi\\\|\_\{\\infty\}\}\{2\}\\quad\\left\(\\forall\\tau\\in\\left\(0,\\;L\_\{\\beta\}\\frac\{\\\|\\partial\_\{1\}^\{\\eta\}\\Psi\\\|\_\{\\infty\}\}\{nD^\{\\eta\+2\}\}\\right\]\\right\)\.\(641\)This also holds withsτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}replaced bybτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\.

###### Proof\.

Denote byφa2\\varphi\_\{a^\{2\}\}the density of𝒩⁡\(0,a2\)\\mathcal\{N\}\(0,a^\{2\}\)w\.r\.t\. Lebesgue and byqτq\_\{\\tau\}the density ofe−τ​ψ​\(U\)\+1−e−2​τ​He^\{\-\\tau\}\\psi\(U\)\+\\sqrt\{1\-e^\{\-2\\tau\}\}HforU,H∼γ1U,H\\sim\\gamma\_\{1\}i\.i\.d\. If we denoteσ:=1−e−2​τ\\sigma:=\\sqrt\{1\-e^\{\-2\\tau\}\}then

qτ​\(w\)=∫ℝφ1​\(u\)​φσ2​\(w−e−τ​ψ​\(u\)\)​𝑑u\.q\_\{\\tau\}\(w\)=\\int\_\{\\mathbb\{R\}\}\\varphi\_\{1\}\(u\)\\varphi\_\{\\sigma^\{2\}\}\(w\-e^\{\-\\tau\}\\psi\(u\)\)du\.\(642\)Sincee−τ​U\+σ​H∼γ1e^\{\-\\tau\}U\+\\sigma H\\sim\\gamma\_\{1\}we know

φ1​\(w\)=∫ℝφ1​\(u\)​φσ2​\(w−e−τ​u\)​𝑑u\\varphi\_\{1\}\(w\)=\\int\_\{\\mathbb\{R\}\}\\varphi\_\{1\}\(u\)\\varphi\_\{\\sigma^\{2\}\}\(w\-e^\{\-\\tau\}u\)du\(643\)and hence forj∈\{0,…,4\}j\\in\\\{0,\\ldots,4\\\}we have

qτ\(j\)​\(w\)−φ1\(j\)​\(w\)=∫\|u\|\>2​βφ1​\(u\)​\[φσ2\(j\)​\(w−e−τ​ψ​\(u\)\)−φσ2\(j\)​\(w−e−τ​u\)\]​𝑑u\.q\_\{\\tau\}^\{\(j\)\}\(w\)\-\\varphi\_\{1\}^\{\(j\)\}\(w\)=\\int\_\{\|u\|\>2\\beta\}\\varphi\_\{1\}\(u\)\\left\[\\varphi\_\{\\sigma^\{2\}\}^\{\(j\)\}\(w\-e^\{\-\\tau\}\\psi\(u\)\)\-\\varphi\_\{\\sigma^\{2\}\}^\{\(j\)\}\(w\-e^\{\-\\tau\}u\)\\right\]du\.\(644\)Ifτ∈\(0,1/2\]\\tau\\in\(0,1/2\]\(and soτ≤σ2≤2​τ\\tau\\leq\\sigma^\{2\}\\leq 2\\tau\) and\|t\|≥β/2\|t\|\\geq\\beta/2then for allj∈\{0,…,4\}j\\in\\\{0,\\ldots,4\\\}we claim that

\|φσ2\(j\)​\(t\)\|≤Eβ2​τ,Eβ:=200​\(112e​β2\)7/2\.\\displaystyle\\left\|\\varphi^\{\(j\)\}\_\{\{\\sigma^\{2\}\}\}\(t\)\\right\|\\leq\\frac\{E\_\{\\beta\}\}\{2\}\\tau,\\quad E\_\{\\beta\}:=200\\left\(\\frac\{112\}\{e\\beta^\{2\}\}\\right\)^\{7/2\}\.\(645\)This follows from the observation\|φ1\(j\)\(x\)\|≤10\(1\+\|x\|4\)e−x2/2≤100e−x2/4\|\\varphi\_\{1\}^\{\(j\)\}\(x\)\|\\leq 10\(1\+\|x\|^\{4\}\)e^\{\-x^\{2\}/2\}\\leq 100e^\{\-x^\{2\}/4\}and therefore\|φσ2\(j\)\(t\)\|≤100σ−\(j\+1\)e−t2/\(4σ2\)≤100τ−5/2e−β2/\(32τ\)\|\\varphi\_\{\\sigma^\{2\}\}^\{\(j\)\}\(t\)\|\\leq 100\\sigma^\{\-\(j\+1\)\}e^\{\-t^\{2\}/\(4\\sigma^\{2\}\)\}\\leq 100\\tau^\{\-5/2\}e^\{\-\\beta^\{2\}/\(32\\tau\)\}\. Combining this with the elementary calculus boundsupτ≤1/2τ−7/2e−β2/\(32τ\)≤\(112/\(eβ2\)\)7/2\\sup\_\{\\tau\\leq 1/2\}\\tau^\{\-7/2\}e^\{\-\\beta^\{2\}/\(32\\tau\)\}\\leq\(112/\(e\\beta^\{2\}\)\)^\{7/2\}yields \([645](https://arxiv.org/html/2610.10875#A1.E645)\) and consequently

max⁡sup\|w\|≤e−τ​β0≤j≤4⁡\|qτ\(j\)​\(w\)−φ1\(j\)​\(w\)\|≤Eβ​τ\(∀τ∈\(0,1/2\]\),\\max\_\{0\\leq j\\leq 4\}\\sup\_\{\|w\|\\leq e^\{\-\\tau\}\\beta\}\\left\|q\_\{\\tau\}^\{\(j\)\}\(w\)\-\\varphi^\{\(j\)\}\_\{1\}\(w\)\\right\|\\leq E\_\{\\beta\}\\tau\\quad\(\\forall\\tau\\in\(0,1/2\]\),\(646\)where we note that\|w\|≤e−τ​β\|w\|\\leq e^\{\-\\tau\}\\betaimplies thatmin⁡\{\|w−e−τ​ψ​\(u\)\|,\|w−e−τ​u\|\}≥e−τ​β≥β/2\\min\\\{\|w\-e^\{\-\\tau\}\\psi\(u\)\|,\|w\-e^\{\-\\tau\}u\|\\\}\\geq e^\{\-\\tau\}\\beta\\geq\\beta/2by monotonicity ofψ\\psiand\|u\|\>2​β\|u\|\>2\\beta\.

Now, withρτψ,Ψ=Pτ∗​πψ,Ψ∈𝒫⁡\(ℝn\+1\)\\rho\_\{\\tau\}^\{\\psi,\\Psi\}=P\_\{\\tau\}^\{\*\}\\pi^\{\\psi,\\Psi\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{n\+1\}\)the Mehler formula implies thatρτψ,Ψ=Law⁡\(\(Zτ,Uτ\)\)\\rho\_\{\\tau\}^\{\\psi,\\Psi\}=\\operatorname\{Law\}\\left\(\(Z\_\{\\tau\},U\_\{\\tau\}\)\\right\)for the random variables

Zτ:=e−τ​Z\+1−e−2​τ​G,Uτ:=e−τ​\(ψ⁡\(U\)\+Ψ⁡\(Z\)\)\+1−e−2​τ​H,Z\_\{\\tau\}:=e^\{\-\\tau\}Z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G,\\quad U\_\{\\tau\}:=e^\{\-\\tau\}\\left\(\\psi\(U\)\+\\Psi\(Z\)\\right\)\+\\sqrt\{1\-e^\{\-2\\tau\}\}H,\(647\)whereZ,G∼γnZ,G\\sim\\gamma\_\{n\}andU,H∼γ1U,H\\sim\\gamma\_\{1\}are all independent\. SinceLaw⁡\(Zτ\)=γn\\operatorname\{Law\}\(Z\_\{\\tau\}\)=\\gamma\_\{n\}we have the joint density

ρτψ,Ψ​\(z,u\)=γn​\(z\)​Fτ​\(z,u\),\\rho\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)=\\gamma\_\{n\}\(z\)F\_\{\\tau\}\(z,u\),\(648\)where we defineFτ​\(z,⋅\)F\_\{\\tau\}\(z,\\cdot\)to be the density ofLaw⁡\(Uτ∣Zτ=z\)\\operatorname\{Law\}\(U\_\{\\tau\}\\mid Z\_\{\\tau\}=z\)\. Now, we knowZ∣\{Zτ=z\}∼𝒩\(e−τz,\(1−e−2​τ\)𝟙n\)Z\\mid\\\{Z\_\{\\tau\}=z\\\}\\sim\\mathcal\{N\}\(e^\{\-\\tau\}z,\(1\-e^\{\-2\\tau\}\)\{\\mathbbm\{1\}\}\_\{n\}\)by Gaussian regression and so

Fτ​\(z,u\)=𝔼G∼γn​\[qτ​\(u−e−τ​Ψ​\(e−τ​z\+1−e−2​τ​G\)\)\]F\_\{\\tau\}\(z,u\)=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[q\_\{\\tau\}\\left\(u\-e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\\right\)\\right\]\(649\)sinceqτq\_\{\\tau\}is the density ofe−τ​ψ​\(U\)\+1−e−2​τ​He^\{\-\\tau\}\\psi\(U\)\+\\sqrt\{1\-e^\{\-2\\tau\}\}H\. Let us also define the Gaussian comparator

F¯τ​\(z,u\):=𝔼G∼γn​\[φ1​\(u−e−τ​Ψ​\(e−τ​z\+1−e−2​τ​G\)\)\]\\bar\{F\}\_\{\\tau\}\(z,u\):=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[\\varphi\_\{1\}\\left\(u\-e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\\right\)\\right\]\(650\)Forℓ∈\{0,1\}\\ell\\in\\\{0,1\\\}we have

\|∂uℓ\(Fτ−F¯τ\)​\(eτ​z,0\)\|=\|𝔼G∼γn​\[\(qτ\(ℓ\)−φ1\(ℓ\)\)​\(−e−τ​Ψ​\(z\+1−e−2​τ​G\)\)\]\|\.\\left\|\\partial\_\{u\}^\{\\ell\}\(F\_\{\\tau\}\-\\bar\{F\}\_\{\\tau\}\)\(e^\{\\tau\}z,0\)\\right\|=\\left\|\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[\(q\_\{\\tau\}^\{\(\\ell\)\}\-\\varphi\_\{1\}^\{\(\\ell\)\}\)\(\-e^\{\-\\tau\}\\Psi\(z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\)\\right\]\\right\|\.\(651\)Differentiating three times in the first coordinate ofeτ​ze^\{\\tau\}zand applying \([640](https://arxiv.org/html/2610.10875#A1.E640)\) and \([646](https://arxiv.org/html/2610.10875#A1.E646)\) yield

supz∈ℝnmaxℓ∈\{0,1\}\|∂1k∂uℓ\(Fτ−F¯τ\)\(eτz,0\)\|≤5EβDkτ\(∀k∈\{0,…,3\},∀τ∈\(0,1/2\]\)\.\\sup\_\{z\\in\{\\mathbb\{R\}\}^\{n\}\}\\max\_\{\\ell\\in\\\{0,1\\\}\}\\left\|\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}\(F\_\{\\tau\}\-\\bar\{F\}\_\{\\tau\}\)\(e^\{\\tau\}z,0\)\\right\|\\leq 5E\_\{\\beta\}D^\{k\}\\tau\\quad\(\\forall k\\in\\\{0,\\ldots,3\\\},\\;\\forall\\tau\\in\(0,1/2\]\)\.\(652\)Next, fork∈\{0,…,3\}k\\in\\\{0,\\ldots,3\\\}andℓ∈\{0,1\}\\ell\\in\\\{0,1\\\}, each summand of∂1k∂uℓF¯τ​\(eτ​z,0\)\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}\\bar\{F\}\_\{\\tau\}\(e^\{\\tau\}z,0\)has the form

c​e−a​τ​𝔼G∼γn​\[fj,α,τ​\(z\+1−e−2​τ​G\)\],fj,α,τ​\(y\):=φ1\(j\)​\(−e−τ​Ψ​\(y\)\)​∏i=1m∂1αiΨ⁡\(y\),ce^\{\-a\\tau\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[f\_\{j,\\alpha,\\tau\}\(z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\\right\],\\quad f\_\{j,\\alpha,\\tau\}\(y\):=\\varphi\_\{1\}^\{\(j\)\}\\left\(\-e^\{\-\\tau\}\\Psi\(y\)\\right\)\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\Psi\(y\),\(653\)where0≤a≤60\\leq a\\leq 6,j∈\{0,…,4\}j\\in\\\{0,\\ldots,4\\\},∑i=1mαi=k\\sum\_\{i=1\}^\{m\}\\alpha\_\{i\}=k, and the sum of\|c\|\|c\|across all the summands is≤5\\leq 5\. Note the Laplacian bound

‖Δ​fj,α,τ‖∞≤n​B​\(2\+2​m\+m2\)​Dk\+2≤17​B​n​Dk\+2,\\\|\\Delta f\_\{j,\\alpha,\\tau\}\\\|\_\{\\infty\}\\leq nB\(2\+2m\+m^\{2\}\)D^\{k\+2\}\\leq 17BnD^\{k\+2\},\(654\)where we denoteB:=max0≤j≤6⁡‖φ1\(j\)‖∞≤20000B:=\\max\_\{0\\leq j\\leq 6\}\\\|\\varphi\_\{1\}^\{\(j\)\}\\\|\_\{\\infty\}\\leq 20000\. Consequently, the difference between corresponding summands of∂1k∂uℓF¯τ​\(eτ​z,0\)\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}\\bar\{F\}\_\{\\tau\}\(e^\{\\tau\}z,0\)and∂1k∂uℓF¯0​\(z,0\)\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}\\bar\{F\}\_\{0\}\(z,0\)is upper bounded in absolute value by

\|ce−a​τ𝔼G∼γn\\displaystyle\|ce^\{\-a\\tau\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\[fj,α,τ​\(z\+1−e−2​τ​G\)\]−c​fj,α,0​\(z\)\|≤\\displaystyle\\left\[f\_\{j,\\alpha,\\tau\}\(z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\\right\]\-cf\_\{j,\\alpha,0\}\(z\)\|\\leq\(655\)\|c\|​e−a​τ​\|𝔼G∼γn​\[fj,α,τ​\(z\+1−e−2​τ​G\)\]−fj,α,τ​\(z\)\|\+\\displaystyle\\quad\|c\|e^\{\-a\\tau\}\\left\|\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[f\_\{j,\\alpha,\\tau\}\(z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\\right\]\-f\_\{j,\\alpha,\\tau\}\(z\)\\right\|\+\(656\)\|c\|​\|e−a​τ​fj,α,τ​\(z\)−fj,α,0​\(z\)\|\.\\displaystyle\\quad\|c\|\\left\|e^\{\-a\\tau\}f\_\{j,\\alpha,\\tau\}\(z\)\-f\_\{j,\\alpha,0\}\(z\)\\right\|\.\(657\)We know that

\|e−a​τ​φ1\(j\)​\(−e−τ​Ψ​\(z\)\)−φ1\(j\)​\(−Ψ⁡\(z\)\)\|\\displaystyle\\left\|e^\{\-a\\tau\}\\varphi\_\{1\}^\{\(j\)\}\(\-e^\{\-\\tau\}\\Psi\(z\)\)\-\\varphi\_\{1\}^\{\(j\)\}\(\-\\Psi\(z\)\)\\right\|≤\(1−e−a​τ\)​‖φ1\(j\)‖∞\+\|φ1\(j\)​\(−e−τ​Ψ​\(z\)\)−φ1\(j\)​\(−Ψ⁡\(z\)\)\|\\displaystyle\\leq\(1\-e^\{\-a\\tau\}\)\\\|\\varphi\_\{1\}^\{\(j\)\}\\\|\_\{\\infty\}\+\\left\|\\varphi\_\{1\}^\{\(j\)\}\(\-e^\{\-\\tau\}\\Psi\(z\)\)\-\\varphi\_\{1\}^\{\(j\)\}\(\-\\Psi\(z\)\)\\right\|\(658\)≤\(1−e−a​τ\)​‖φ1\(j\)‖∞\+‖φ1\(j\+1\)‖∞​\(1−e−τ\)​\|Ψ⁡\(z\)\|\\displaystyle\\leq\(1\-e^\{\-a\\tau\}\)\\\|\\varphi\_\{1\}^\{\(j\)\}\\\|\_\{\\infty\}\+\\\|\\varphi\_\{1\}^\{\(j\+1\)\}\\\|\_\{\\infty\}\(1\-e^\{\-\\tau\}\)\|\\Psi\(z\)\|\(659\)≤B⁡\(a\+β\)​τ\\displaystyle\\leq B\(a\+\\beta\)\\tau\(660\)and therefore the second term in \([657](https://arxiv.org/html/2610.10875#A1.E657)\) is bounded by

\|e−a​τ​fj,α,τ​\(z\)−fj,α,0​\(z\)\|≤B⁡\(6\+β\)​τ​Dk\.\\left\|e^\{\-a\\tau\}f\_\{j,\\alpha,\\tau\}\(z\)\-f\_\{j,\\alpha,0\}\(z\)\\right\|\\leq B\(6\+\\beta\)\\tau D^\{k\}\.\(661\)For the first term, we use the heat semigroup identity

\|𝔼G∼γn​\[fj,α,τ​\(z\+1−e−2​τ​G\)\]−fj,α,τ​\(z\)\|\\displaystyle\\left\|\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[f\_\{j,\\alpha,\\tau\}\(z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\\right\]\-f\_\{j,\\alpha,\\tau\}\(z\)\\right\|=\|12​∫01−e−2​τ𝔼G∼γn​\[Δ​fj,α,τ​\(z\+t​G\)\]​𝑑t\|\\displaystyle=\\left\|\\frac\{1\}\{2\}\\int\_\{0\}^\{1\-e^\{\-2\\tau\}\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[\\Delta f\_\{j,\\alpha,\\tau\}\(z\+\\sqrt\{t\}G\)\\right\]dt\\right\|\(662\)≤1−e−2​τ2​‖Δ​fj,α,τ‖∞\\displaystyle\\leq\\frac\{1\-e^\{\-2\\tau\}\}\{2\}\\\|\\Delta f\_\{j,\\alpha,\\tau\}\\\|\_\{\\infty\}\(663\)≤1−e−2​τ2​\(17​B​n​Dk\+2\)\\displaystyle\\leq\\frac\{1\-e^\{\-2\\tau\}\}\{2\}\(17BnD^\{k\+2\}\)\(664\)≤17​B​n​Dk\+2​τ,\\displaystyle\\leq 17BnD^\{k\+2\}\\tau,\(665\)where for the third line we plugged in \([654](https://arxiv.org/html/2610.10875#A1.E654)\)\. Summing over the summands, we find that

supz∈ℝnmaxℓ∈\{0,1\}⁡\|∂1k∂uℓF¯τ​\(eτ​z,0\)−∂1k∂uℓF¯0​\(z,0\)\|\\displaystyle\\sup\_\{z\\in\{\\mathbb\{R\}\}^\{n\}\}\\max\_\{\\ell\\in\\\{0,1\\\}\}\\left\|\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}\\bar\{F\}\_\{\\tau\}\(e^\{\\tau\}z,0\)\-\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}\\bar\{F\}\_\{0\}\(z,0\)\\right\|≤5⋅\(B⁡\(6\+β\)​τ​Dk\+17​B​n​Dk\+2​τ\)\\displaystyle\\leq 5\\cdot\(B\(6\+\\beta\)\\tau D^\{k\}\+17BnD^\{k\+2\}\\tau\)\(666\)≤B⁡\(115\+5​β\)​n​Dk\+2​τ\.\\displaystyle\\leq B\(115\+5\\beta\)nD^\{k\+2\}\\tau\.\(667\)Combining with \([652](https://arxiv.org/html/2610.10875#A1.E652)\), we find that

supz∈ℝnmaxℓ∈\{0,1\}⁡\|∂1k∂uℓFτ​\(eτ​z,0\)−∂1k∂uℓF0​\(z,0\)\|≤Jβ​n​Dk\+2​τ\\sup\_\{z\\in\{\\mathbb\{R\}\}^\{n\}\}\\max\_\{\\ell\\in\\\{0,1\\\}\}\\left\|\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}F\_\{\\tau\}\(e^\{\\tau\}z,0\)\-\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}F\_\{0\}\(z,0\)\\right\|\\leq J\_\{\\beta\}nD^\{k\+2\}\\tau\(668\)for everyk∈\{0,…,3\}k\\in\\\{0,\\ldots,3\\\}andτ∈\(0,1/2\]\\tau\\in\(0,1/2\], where we defineJβ:=B⁡\(115\+5​β\)\+5​EβJ\_\{\\beta\}:=B\(115\+5\\beta\)\+5E\_\{\\beta\}and the Gaussian comparatorF0​\(z,u\):=F¯0​\(z,u\)=φ1​\(u−Ψ⁡\(z\)\)F\_\{0\}\(z,u\):=\\bar\{F\}\_\{0\}\(z,u\)=\\varphi\_\{1\}\(u\-\\Psi\(z\)\)\.

Armed with this stability along the OU flow, we note that the last coordinate of the score satisfies

Sτ:=\(sτψ,Ψ\)n\+1=∂ulog⁡ρτψ,Ψ=∂uFτFτS\_\{\\tau\}:=\(s\_\{\\tau\}^\{\\psi,\\Psi\}\)\_\{n\+1\}=\\partial\_\{u\}\\log\\rho\_\{\\tau\}^\{\\psi,\\Psi\}=\\frac\{\\partial\_\{u\}F\_\{\\tau\}\}\{F\_\{\\tau\}\}\(669\)due to the joint density \([648](https://arxiv.org/html/2610.10875#A1.E648)\)\. Denote also the comparatorS0​\(z,u\):=Ψ⁡\(z\)−uS\_\{0\}\(z,u\):=\\Psi\(z\)\-u\. By the quotient rule, for anyS=∂uFFS=\\frac\{\\partial\_\{u\}F\}\{F\}we have

∂1S=F1​uF−F1​FuF2,∂12S=F11​uF−2​F1​u​F1\+Fu​F11F2\+2​Fu​F12F3,\\partial\_\{1\}S=\\frac\{F\_\{1u\}\}\{F\}\-\\frac\{F\_\{1\}F\_\{u\}\}\{F^\{2\}\},\\quad\\partial\_\{1\}^\{2\}S=\\frac\{F\_\{11u\}\}\{F\}\-\\frac\{2F\_\{1u\}F\_\{1\}\+F\_\{u\}F\_\{11\}\}\{F^\{2\}\}\+\\frac\{2F\_\{u\}F\_\{1\}^\{2\}\}\{F^\{3\}\},\(670\)∂13S=F111​uF−3​F11​u​F1\+3​F1​u​F11\+Fu​F111F2\+6​F1​u​F12\+6​Fu​F1​F11F3−6​Fu​F13F4\.\\partial\_\{1\}^\{3\}S=\\frac\{F\_\{111u\}\}\{F\}\-\\frac\{3F\_\{11u\}F\_\{1\}\+3F\_\{1u\}F\_\{11\}\+F\_\{u\}F\_\{111\}\}\{F^\{2\}\}\+\\frac\{6F\_\{1u\}F\_\{1\}^\{2\}\+6F\_\{u\}F\_\{1\}F\_\{11\}\}\{F^\{3\}\}\-\\frac\{6F\_\{u\}F\_\{1\}^\{3\}\}\{F^\{4\}\}\.\(671\)Denotingmβ:=φ1​\(β\)m\_\{\\beta\}:=\\varphi\_\{1\}\(\\beta\), from here on we fixn​D2​τ≤mβ2​JβnD^\{2\}\\tau\\leq\\frac\{m\_\{\\beta\}\}\{2J\_\{\\beta\}\}\. Applying \([668](https://arxiv.org/html/2610.10875#A1.E668)\) withk=ℓ=0k=\\ell=0implies that

Fτ​\(eτ​z,0\)≥F0​\(z,0\)−Jβ​n​D2​τ≥mβ2\.F\_\{\\tau\}\(e^\{\\tau\}z,0\)\\geq F\_\{0\}\(z,0\)\-J\_\{\\beta\}nD^\{2\}\\tau\\geq\\frac\{m\_\{\\beta\}\}\{2\}\.\(672\)Moreover, since we haveF0​\(z,u\)=φ1​\(u−Ψ⁡\(z\)\)F\_\{0\}\(z,u\)=\\varphi\_\{1\}\(u\-\\Psi\(z\)\)the chain rule implies

maxℓ∈\{0,1\}⁡\|∂1k∂uℓF0​\(z,0\)\|≤5​B​Dk\(∀k∈\{0,…,3\}\),\\max\_\{\\ell\\in\\\{0,1\\\}\}\\left\|\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}F\_\{0\}\(z,0\)\\right\|\\leq 5BD^\{k\}\\quad\(\\forall k\\in\\\{0,\\ldots,3\\\}\),\(673\)after which another application of \([668](https://arxiv.org/html/2610.10875#A1.E668)\) yields

maxℓ∈\{0,1\}⁡\|∂1k∂uℓFτ​\(eτ​z,0\)\|≤5​B​Dk\+Jβ​n​Dk\+2​τ≤Jβ​Dk\(∀k∈\{0,…,3\}\)\.\\max\_\{\\ell\\in\\\{0,1\\\}\}\\left\|\\partial\_\{1\}^\{k\}\\partial\_\{u\}^\{\\ell\}F\_\{\\tau\}\(e^\{\\tau\}z,0\)\\right\|\\leq 5BD^\{k\}\+J\_\{\\beta\}nD^\{k\+2\}\\tau\\leq J\_\{\\beta\}D^\{k\}\\quad\(\\forall k\\in\\\{0,\\ldots,3\\\}\)\.\(674\)For any term in \([670](https://arxiv.org/html/2610.10875#A1.E670)\), \([671](https://arxiv.org/html/2610.10875#A1.E671)\) of the formc​∏i=1m∂1αi∂uℓiFFqc\\frac\{\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\}\{F^\{q\}\}with∑i=1mαi=k\\sum\_\{i=1\}^\{m\}\\alpha\_\{i\}=kandℓi∈\{0,1\}\\ell\_\{i\}\\in\\\{0,1\\\}we have the bound

\|∏i=1m∂1αi∂uℓiFτFτq−∏i=1m∂1αi∂uℓiF0F0q\|\\displaystyle\\left\|\\frac\{\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{\\tau\}\}\{F\_\{\\tau\}^\{q\}\}\-\\frac\{\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{0\}\}\{F\_\{0\}^\{q\}\}\\right\|≤1Fτq​\|∏i=1m∂1αi∂uℓiFτ−∏i=1m∂1αi∂uℓiF0\|\\displaystyle\\leq\\frac\{1\}\{F\_\{\\tau\}^\{q\}\}\\left\|\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{\\tau\}\-\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{0\}\\right\|\(675\)\+\|∏i=1m∂1αi∂uℓiF0\|⋅\|1Fτq−1F0q\|,\\displaystyle\\quad\+\\left\|\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{0\}\\right\|\\cdot\\left\|\\frac\{1\}\{F\_\{\\tau\}^\{q\}\}\-\\frac\{1\}\{F\_\{0\}^\{q\}\}\\right\|,\(676\)where we recall that theFτF\_\{\\tau\}terms takeeτ​ze^\{\\tau\}zas their argument while theF0F\_\{0\}terms usezz\. If we apply the telescoping bound

\|∏i=1mai−∏i=1mbi\|≤∑i=1m\|ai−bi\|​∏j≠imax⁡\{\|aj\|,\|bj\|\},\\left\|\\prod\_\{i=1\}^\{m\}a\_\{i\}\-\\prod\_\{i=1\}^\{m\}b\_\{i\}\\right\|\\leq\\sum\_\{i=1\}^\{m\}\|a\_\{i\}\-b\_\{i\}\|\\prod\_\{j\\neq i\}\\max\\\{\|a\_\{j\}\|,\|b\_\{j\}\|\\\},\(677\)then by applying \([668](https://arxiv.org/html/2610.10875#A1.E668)\) and \([674](https://arxiv.org/html/2610.10875#A1.E674)\) we find

\|∏i=1m∂1αi∂uℓiFτ−∏i=1m∂1αi∂uℓiF0\|≤m​Jβm​n​Dk\+2​τ\.\\left\|\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{\\tau\}\-\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{0\}\\right\|\\leq mJ\_\{\\beta\}^\{m\}nD^\{k\+2\}\\tau\.\(678\)Furthermore, we may apply\|1aq−1bq\|≤qmin⁡\{a,b\}q\+1​\|a−b\|\|\\frac\{1\}\{a^\{q\}\}\-\\frac\{1\}\{b^\{q\}\}\|\\leq\\frac\{q\}\{\\min\\\{a,b\\\}^\{q\+1\}\}\|a\-b\|fora,b\>0a,b\>0to find

\|1Fτq−1F0q\|≤q​\(2mβ\)q\+1​\|Fτ−F0\|≤q​Jβ​\(2mβ\)q\+1​n​D2​τ,\\left\|\\frac\{1\}\{F\_\{\\tau\}^\{q\}\}\-\\frac\{1\}\{F\_\{0\}^\{q\}\}\\right\|\\leq q\\left\(\\frac\{2\}\{m\_\{\\beta\}\}\\right\)^\{q\+1\}\|F\_\{\\tau\}\-F\_\{0\}\|\\leq qJ\_\{\\beta\}\\left\(\\frac\{2\}\{m\_\{\\beta\}\}\\right\)^\{q\+1\}nD^\{2\}\\tau,\(679\)after which we use that\|∏i=1m∂1αi∂uℓiF0\|≤Jβm​Dk\\left\|\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{0\}\\right\|\\leq J\_\{\\beta\}^\{m\}D^\{k\}to find

\|∏i=1m∂1αi∂uℓiFτFτq−∏i=1m∂1αi∂uℓiF0F0q\|≤\(m​Jβm​\(2mβ\)q\+q​Jβm\+1​\(2mβ\)q\+1\)​n​Dk\+2​τ\.\\left\|\\frac\{\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{\\tau\}\}\{F\_\{\\tau\}^\{q\}\}\-\\frac\{\\prod\_\{i=1\}^\{m\}\\partial\_\{1\}^\{\\alpha\_\{i\}\}\\partial\_\{u\}^\{\\ell\_\{i\}\}F\_\{0\}\}\{F\_\{0\}^\{q\}\}\\right\|\\leq\\left\(mJ\_\{\\beta\}^\{m\}\\left\(\\frac\{2\}\{m\_\{\\beta\}\}\\right\)^\{q\}\+qJ\_\{\\beta\}^\{m\+1\}\\left\(\\frac\{2\}\{m\_\{\\beta\}\}\\right\)^\{q\+1\}\\right\)nD^\{k\+2\}\\tau\.\(680\)In particular, from \([670](https://arxiv.org/html/2610.10875#A1.E670)\), \([671](https://arxiv.org/html/2610.10875#A1.E671)\) we see that\|c\|≤6\|c\|\\leq 6and so

supz∈ℝn\|∂1ηSτ​\(eτ​z,0\)−∂1ηS0​\(z,0\)\|≤Kβ​n​Dη\+2​τ\(∀η∈\{1,2,3\}\)\\sup\_\{z\\in\{\\mathbb\{R\}\}^\{n\}\}\\left\|\\partial\_\{1\}^\{\\eta\}S\_\{\\tau\}\(e^\{\\tau\}z,0\)\-\\partial\_\{1\}^\{\\eta\}S\_\{0\}\(z,0\)\\right\|\\leq K\_\{\\beta\}nD^\{\\eta\+2\}\\tau\\quad\(\\forall\\eta\\in\\\{1,2,3\\\}\)\(681\)for allτ≤1Kβ​n​D2\\tau\\leq\\frac\{1\}\{K\_\{\\beta\}nD^\{2\}\}, where we define the constant

Kβ:=2​Jβmβ\+60​Jβ2mβ2\+400​Jβ3mβ3\+960​Jβ4mβ4\+768​Jβ5mβ5≥2​Jβmβ\.K\_\{\\beta\}:=\\frac\{2J\_\{\\beta\}\}\{m\_\{\\beta\}\}\+\\frac\{60J\_\{\\beta\}^\{2\}\}\{m\_\{\\beta\}^\{2\}\}\+\\frac\{400J\_\{\\beta\}^\{3\}\}\{m\_\{\\beta\}^\{3\}\}\+\\frac\{960J\_\{\\beta\}^\{4\}\}\{m\_\{\\beta\}^\{4\}\}\+\\frac\{768J\_\{\\beta\}^\{5\}\}\{m\_\{\\beta\}^\{5\}\}\\geq\\frac\{2J\_\{\\beta\}\}\{m\_\{\\beta\}\}\.\(682\)Since we have∂1ηS0=∂1ηΨ\\partial\_\{1\}^\{\\eta\}S\_\{0\}=\\partial\_\{1\}^\{\\eta\}\\Psi, we find that

‖∂1ηsτψ,Ψ‖∞≥‖∂1ηΨ‖∞−Kβ​n​Dη\+2​τ,\\\|\\partial\_\{1\}^\{\\eta\}s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\infty\}\\geq\{\\\|\\partial\_\{1\}^\{\\eta\}\\Psi\\\|\_\{\\infty\}\}\-K\_\{\\beta\}nD^\{\\eta\+2\}\\tau,\(683\)from which the claim follows\. Since\(bτψ,Ψ​\(z,u\)\)n\+1=\(sτψ,Ψ​\(z,u\)\)n\+1\+u\(b\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)\)\_\{n\+1\}=\(s\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)\)\_\{n\+1\}\+u, the∂1\\partial\_\{1\}derivatives of the last coordinate are unchanged if we replacesτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}withbτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\.

It remains to find a more explicit upper bound forKβK\_\{\\beta\}\. SinceJβ≥115​BJ\_\{\\beta\}\\geq 115Bwe knowJβ≥mβJ\_\{\\beta\}\\geq m\_\{\\beta\}, and so

Kβ≤2190​\(Jβmβ\)5\.K\_\{\\beta\}\\leq 2190\\left\(\\frac\{J\_\{\\beta\}\}\{m\_\{\\beta\}\}\\right\)^\{5\}\.\(684\)SinceJβ≤20000​\(115\+5​β\)\+1000​\(112e​β2\)7/2≤4\.5⋅108​\(1\+β\+β−7\)J\_\{\\beta\}\\leq 20000\(115\+5\\beta\)\+1000\\left\(\\frac\{112\}\{e\\beta^\{2\}\}\\right\)^\{7/2\}\\leq 4\.5\\cdot 10^\{8\}\(1\+\\beta\+\\beta^\{\-7\}\)and there are the inequalities\(1\+β\+β−7\)≤2​eβ2/2​\(1\+β−7\)\(1\+\\beta\+\\beta^\{\-7\}\)\\leq 2e^\{\\beta^\{2\}/2\}\(1\+\\beta^\{\-7\}\),\(1\+β−7\)5≤16​\(1\+β−35\)\(1\+\\beta^\{\-7\}\)^\{5\}\\leq 16\(1\+\\beta^\{\-35\}\), we may usemβ=12​πe−β2/2m\_\{\\beta\}=\\frac\{1\}\{\\sqrt\{2\\pi\}\}e^\{\-\\beta^\{2\}/2\}to find that

Kβ≤1052​e5​β2​\(1\+β−35\)\.K\_\{\\beta\}\\leq 10^\{52\}e^\{5\\beta^\{2\}\}\(1\+\\beta^\{\-35\}\)\.\(685\)DefiningLβ:=10−52​e−5​β2​\(1\+β−35\)−1/2L\_\{\\beta\}:=10^\{\-52\}e^\{\-5\\beta^\{2\}\}\(1\+\\beta^\{\-35\}\)^\{\-1\}/2yields the claim\. ∎

For a Barron space which controls third derivatives, this implies a lower bound on the Barron norms of the scores:

###### Corollary 2\(Shallow representation lower bound\)\.

Letψ,Ψ\\psi,\\Psibe as in Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)and defineD,LβD,L\_\{\\beta\}accordingly\. Then, the following hold:

1. \(i\)Letϕ∈C3​\(ℝ\)\\phi\\in C^\{3\}\(\{\\mathbb\{R\}\}\)be bounded with‖ϕ′′′‖∞<∞\\\|\\phi^\{\\prime\\prime\\prime\}\\\|\_\{\\infty\}<\\inftyand letℬϕR\\mathcal\{B\}\_\{\\phi\}^\{R\}be its Barron space with input weights bounded byRR\(see Definition[3](https://arxiv.org/html/2610.10875#Thmdefinition3)\)\. Then, infR∈\(0,∞\)R2​ℰℬϕR​\(πψ,Ψ\)≥Lβ2​n​D5⋅maxi∈\{1,…,n\}⁡‖∂i3Ψ‖∞2\.\\inf\_\{R\\in\(0,\\infty\)\}R^\{2\}\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}^\{R\}\}\(\\pi^\{\\psi,\\Psi\}\)\\geq\\frac\{L\_\{\\beta\}\}\{2nD^\{5\}\}\\cdot\\max\_\{i\\in\\\{1,\\ldots,n\\\}\}\{\\\|\\partial\_\{i\}^\{3\}\\Psi\\\|\_\{\\infty\}^\{2\}\}\.\(686\)
2. \(ii\)Letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be positively 3\-homogeneous \(e\.g\.ReLU3\\operatorname\{ReLU\}^\{3\}\) and letℬϕ\\mathcal\{B\}\_\{\\phi\}be its Barron space \(see Definition[4](https://arxiv.org/html/2610.10875#Thmdefinition4)\)\. Then, ℰℬϕ​\(πψ,Ψ\)≥Lβ2​n​D5⋅maxi∈\{1,…,n\}⁡‖∂i3Ψ‖∞2\.\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi^\{\\psi,\\Psi\}\)\\geq\\frac\{L\_\{\\beta\}\}\{2nD^\{5\}\}\\cdot\\max\_\{i\\in\\\{1,\\ldots,n\\\}\}\{\\\|\\partial\_\{i\}^\{3\}\\Psi\\\|\_\{\\infty\}^\{2\}\}\.\(687\)

###### Proof\.

The lower bound in Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)holds for any coordinatei∈\{1,…,n\}i\\in\\\{1,\\ldots,n\\\}\. Ifϕ\\phiis as in \(i\) then any Barron functionv⁡\(x\)=∑jμj​ϕ​\(⟨aj,x⟩\+bj\)v\(x\)=\\sum\_\{j\}\\mu\_\{j\}\\phi\(\\left\\langle a\_\{j\},x\\right\\rangle\+b\_\{j\}\)has

\|∂13v⁡\(x\)\|≤‖ϕ′′′‖∞​∑j‖μj‖​‖aj‖3≤‖ϕ′′′‖∞​R2​‖v‖ℬϕR\.\|\\partial\_\{1\}^\{3\}v\(x\)\|\\leq\\\|\\phi^\{\\prime\\prime\\prime\}\\\|\_\{\\infty\}\\sum\_\{j\}\\\|\\mu\_\{j\}\\\|\\\|a\_\{j\}\\\|^\{3\}\\leq\\\|\\phi^\{\\prime\\prime\\prime\}\\\|\_\{\\infty\}R^\{2\}\\\|v\\\|\_\{\\mathcal\{B\}\_\{\\phi\}^\{R\}\}\.\(688\)If insteadϕ\\phiis33\-homogeneous as in \(ii\), then the same argument works since‖ϕ′′′‖L∞<∞\\\|\\phi^\{\\prime\\prime\\prime\}\\\|\_\{L^\{\\infty\}\}<\\infty\. ∎

We also verify that the scores can lie in the Barron space under sufficient smoothness and tail behavior\. This is a sanity check to ensure that the result of Corollary[2](https://arxiv.org/html/2610.10875#Thmcorollary2)is quantitative and not simply a disguised version of the statement that “the scores are not Barron”\. First, we define a class of functions for which this is simple to verify:

###### Lemma 19\(Cylindrical Schwartz functions are Barron w\.r\.t\.ReLUs\\operatorname\{ReLU\}^\{s\}\)\.

Letn∈ℕn\\in\{\\mathbb\{N\}\}andλ∈\(0,∞\)\\lambda\\in\(0,\\infty\)\. We define𝒞λ\\mathcal\{C\}\_\{\\lambda\}to be the class of functionsf:ℝn×ℝ→ℝf:\{\\mathbb\{R\}\}^\{n\}\\times\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}which can be written as a finite sum

f⁡\(z,u\)=∑r=1MFr​\(PEr​z,u\)f\(z,u\)=\\sum\_\{r=1\}^\{M\}F\_\{r\}\(P\_\{E\_\{r\}\}z,u\)\(689\)where eachEr⊆ℝnE\_\{r\}\\subseteq\{\\mathbb\{R\}\}^\{n\}is a linear subspace \(with orthogonal projectionPErP\_\{E\_\{r\}\}\) and eachFr:Er×ℝ→ℝF\_\{r\}:E\_\{r\}\\times\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfies the seminorm bounds

\|∂xα∂ujFr\(x,u\)\|≲α,j,N\(1\+∥x∥\)−N\(1\+\|u\|\)−λ−j\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}F\_\{r\}\(x,u\)\|\\lesssim\_\{\\alpha,j,N\}\(1\+\\\|x\\\|\)^\{\-N\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\(690\)for allN∈ℕN\\in\{\\mathbb\{N\}\}, all multi\-indicesα\\alpha, and allj∈ℕ∪\{0\}j\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\.

The class𝒞λ\\mathcal\{C\}\_\{\\lambda\}is an algebra which is closed under smooth composition and OU smoothing, i\.e\. the following properties hold:

1. \(i\)Iff,g∈𝒞λf,g\\in\\mathcal\{C\}\_\{\\lambda\}, thenf​g∈𝒞λfg\\in\\mathcal\{C\}\_\{\\lambda\}with seminorms controlled by those off,gf,gand the angles between the involved subspaces\.
2. \(ii\)Iff∈𝒞λf\\in\\mathcal\{C\}\_\{\\lambda\}andη∈C∞​\(ℝ\)\\eta\\in C^\{\\infty\}\(\{\\mathbb\{R\}\}\)satisfiesη⁡\(0\)=0\\eta\(0\)=0and all derivatives ofη\\etaare bounded, thenη∘f∈𝒞λ\\eta\\circ f\\in\\mathcal\{C\}\_\{\\lambda\}with seminorms controlled by those offf, the derivative bounds onη\\eta, and the angles between the involved subspaces\.
3. \(iii\)Iff∈𝒞λf\\in\\mathcal\{C\}\_\{\\lambda\}andg⁡\(z,u\)=𝔼G∼γn​\[f⁡\(a​z\+b​G,u\)\]g\(z,u\)=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\[f\(az\+bG,u\)\]for some0<a−≤a≤a\+<∞0<a\_\{\-\}\\leq a\\leq a\_\{\+\}<\\inftyand0≤b≤b\+<∞0\\leq b\\leq b\_\{\+\}<\\infty, theng∈𝒞λg\\in\\mathcal\{C\}\_\{\\lambda\}with seminorms controlled by those offfwith constants depending only ona−,a\+,b\+a\_\{\-\},a\_\{\+\},b\_\{\+\}, and subspace dimensions\.

Furthermore, for anys∈ℕs\\in\{\\mathbb\{N\}\}withϕ⁡\(t\)=max⁡\{0,t\}s\\phi\(t\)=\\max\\\{0,t\\\}^\{s\}theReLUs\\operatorname\{ReLU\}^\{s\}activation andℬϕ\\mathcal\{B\}\_\{\\phi\}its associated Barron space of functions fromℝn\+1→ℝ\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}\(using the scalar version of Definition[4](https://arxiv.org/html/2610.10875#Thmdefinition4)\), we have𝒞λ⊆ℬϕ\\mathcal\{C\}\_\{\\lambda\}\\subseteq\\mathcal\{B\}\_\{\\phi\}with Barron norm controlled by the𝒞λ\\mathcal\{C\}\_\{\\lambda\}seminorms\.

###### Proof\.

For any profileF:E×ℝ→ℝF:E\\times\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}, we denote the seminorm

\[F\]α,j,N:=supx∈E,u∈ℝ\(1\+‖x‖\)N​\(1\+\|u\|\)λ\+j​\|∂xα∂ujF⁡\(x,u\)\|\.\[F\]\_\{\\alpha,j,N\}:=\\sup\_\{x\\in E,\\;u\\in\{\\mathbb\{R\}\}\}\(1\+\\\|x\\\|\)^\{N\}\(1\+\|u\|\)^\{\\lambda\+j\}\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}F\(x,u\)\|\.\(691\)
\(i\)\.For this, it suffices to consider two summands

f⁡\(x,u\)=F⁡\(PE​x,u\),g⁡\(x,u\)=G⁡\(PV​x,u\)\.f\(x,u\)=F\(P\_\{E\}x,u\),\\quad g\(x,u\)=G\(P\_\{V\}x,u\)\.\(692\)Denote the subspaceW:=E\+VW:=E\+Vand writeH⁡\(x,u\):=F⁡\(PE​x,u\)​G​\(PV​x,u\)H\(x,u\):=F\(P\_\{E\}x,u\)G\(P\_\{V\}x,u\)\. Then,\(f​g\)​\(x,u\)=H⁡\(PW​x,u\)\(fg\)\(x,u\)=H\(P\_\{W\}x,u\)\. The linear mapW∋x↦\(PE​x,PV​x\)W\\ni x\\mapsto\(P\_\{E\}x,P\_\{V\}x\)is injective, and hence by finite\-dimensionality‖x‖≤CE,V​\(‖PE​x‖\+‖PV​x‖\)\\\|x\\\|\\leq C\_\{E,V\}\(\\\|P\_\{E\}x\\\|\+\\\|P\_\{V\}x\\\|\)and so

1\+‖x‖≲\(1\+‖PE​x‖\)​\(1\+‖PV​x‖\)\.1\+\\\|x\\\|\\lesssim\(1\+\\\|P\_\{E\}x\\\|\)\(1\+\\\|P\_\{V\}x\\\|\)\.\(693\)By the product rule,∂xα∂ujH⁡\(x,u\)\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}H\(x,u\)is a finite sum of products\(∂xα1∂uj1F\)​\(PE​x,u\)​\(∂xα2∂uj2G\)​\(PV​x,u\)\(\\partial\_\{x\}^\{\\alpha\_\{1\}\}\\partial\_\{u\}^\{j\_\{1\}\}F\)\(P\_\{E\}x,u\)\(\\partial\_\{x\}^\{\\alpha\_\{2\}\}\\partial\_\{u\}^\{j\_\{2\}\}G\)\(P\_\{V\}x,u\)withα1\+α2=α\\alpha\_\{1\}\+\\alpha\_\{2\}=\\alphaandj1\+j2=jj\_\{1\}\+j\_\{2\}=j\. Using \([693](https://arxiv.org/html/2610.10875#A1.E693)\),

\|∂xα∂ujH\(x,u\)\|≲α,j,N\(1\+∥x∥\)−N\(1\+\|u\|\)−2​λ−j≲\(1\+∥x∥\)−N\(1\+\|u\|\)−λ−j\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}H\(x,u\)\|\\lesssim\_\{\\alpha,j,N\}\(1\+\\\|x\\\|\)^\{\-N\}\(1\+\|u\|\)^\{\-2\\lambda\-j\}\\lesssim\(1\+\\\|x\\\|\)^\{\-N\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\(694\)sinceλ\>0\\lambda\>0\. Hence,f​g∈𝒞λfg\\in\\mathcal\{C\}\_\{\\lambda\}with controlled seminorms\. The result extends tof,gf,gbeing any finite sums by linearity and the triangle inequality\.

\(ii\)\.Writef⁡\(z,u\)=∑r=1MFr​\(PEr​z,u\)f\(z,u\)=\\sum\_\{r=1\}^\{M\}F\_\{r\}\(P\_\{E\_\{r\}\}z,u\)and defineB:=∑r=1M‖Fr‖∞B:=\\sum\_\{r=1\}^\{M\}\\\|F\_\{r\}\\\|\_\{\\infty\}\. We use the anchored multivariate fundamental theorem of calculus: if

Θ⁡\(y1,…,yM\):=η⁡\(∑r=1Myr\)\\Theta\(y\_\{1\},\\ldots,y\_\{M\}\):=\\eta\\left\(\\sum\_\{r=1\}^\{M\}y\_\{r\}\\right\)\(695\)thenΘ⁡\(0\)=η⁡\(0\)=0\\Theta\(0\)=\\eta\(0\)=0and

Θ⁡\(y\)=∑∅≠J⊆\{1,…,M\}\(∏r∈Jyr\)​∫\[0,1\]\|J\|η\(\|J\|\)​\(∑r∈Jtr​yr\)​𝑑t\.\\Theta\(y\)=\\sum\_\{\\emptyset\\neq J\\subseteq\\\{1,\\ldots,M\\\}\}\\left\(\\prod\_\{r\\in J\}y\_\{r\}\\right\)\\int\_\{\[0,1\]^\{\|J\|\}\}\\eta^\{\(\|J\|\)\}\\left\(\\sum\_\{r\\in J\}t\_\{r\}y\_\{r\}\\right\)dt\.\(696\)Substituting inyr=Fr​\(PEr​z,u\)y\_\{r\}=F\_\{r\}\(P\_\{E\_\{r\}\}z,u\)gives a finite decomposition ofη∘f\\eta\\circ f\. Fix a nonemptyJ⊆\{1,…,M\}J\\subseteq\\\{1,\\ldots,M\\\}and define the subspaceEJ:=∑r∈JEr⊆ℝnE\_\{J\}:=\\sum\_\{r\\in J\}E\_\{r\}\\subseteq\{\\mathbb\{R\}\}^\{n\}\. The corresponding term inη∘f\\eta\\circ f, denotedHJH\_\{J\}, has

HJ​\(x,u\)=∏r∈JFr​\(PEr​x,u\)​∫\[0,1\]\|J\|η\(\|J\|\)​\(∑r∈Jtr​Fr​\(PEr​x,u\)\)​𝑑tH\_\{J\}\(x,u\)=\\prod\_\{r\\in J\}F\_\{r\}\(P\_\{E\_\{r\}\}x,u\)\\int\_\{\[0,1\]^\{\|J\|\}\}\\eta^\{\(\|J\|\)\}\\left\(\\sum\_\{r\\in J\}t\_\{r\}F\_\{r\}\(P\_\{E\_\{r\}\}x,u\)\\right\)dt\(697\)for everyx∈EJx\\in E\_\{J\}\. Since the arguments ofη\(\|J\|\)\\eta^\{\(\|J\|\)\}lie in\[−B,B\]\[\-B,B\], all derivatives of the integral factor are bounded due to the assumed bounds onη\\eta, Faà di Bruno, and the seminorm bounds onFrF\_\{r\}\. As before, the mapEJ∋x↦\(PEr​x\)r∈JE\_\{J\}\\ni x\\mapsto\(P\_\{E\_\{r\}\}x\)\_\{r\\in J\}is injective and therefore

1\+‖x‖≲∏r∈JJ⁡\(1\+‖PEr​x‖\)\.1\+\\\|x\\\|\\lesssim\_\{J\}\\prod\_\{r\\in J\}\(1\+\\\|P\_\{E\_\{r\}\}x\\\|\)\.\(698\)Differentiating \([697](https://arxiv.org/html/2610.10875#A1.E697)\) and applying Faà di Bruno, each resulting term is a product of finitely many derivatives ofFr​\(PEr​x,u\)F\_\{r\}\(P\_\{E\_\{r\}\}x,u\)multiplied by a bounded derivative ofη\\eta\. Due to the prefactor, every such term contains at least one value or derivative ofFr​\(PEr​x,u\)F\_\{r\}\(P\_\{E\_\{r\}\}x,u\)for eachr∈Jr\\in J\. Hence, using the seminorm definition of eachFrF\_\{r\}implies

\|∂xα∂ujHJ\(x,u\)\|≲α,j,N\(1\+∥x∥\)−N\(1\+\|u\|\)−\|J\|​λ−j\.\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}H\_\{J\}\(x,u\)\|\\lesssim\_\{\\alpha,j,N\}\(1\+\\\|x\\\|\)^\{\-N\}\(1\+\|u\|\)^\{\-\|J\|\\lambda\-j\}\.\(699\)SinceJJis nonempty andλ\>0\\lambda\>0, this proves the second claim\.

\(iii\)\.By linearity, it suffices to consider a single summandf⁡\(z,u\)=F⁡\(PE​z,u\)f\(z,u\)=F\(P\_\{E\}z,u\)\. SincePE​G∼γEP\_\{E\}G\\sim\\gamma\_\{E\}, we have

g⁡\(z,u\)=𝔼G∼γn​\[F⁡\(PE​\(a​z\+b​G\),u\)\]=F~​\(PE​z,u\),F~​\(x,u\):=𝔼ξ∼γE​\[F⁡\(a​x\+b​ξ,u\)\]\.g\(z,u\)=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\[F\(P\_\{E\}\(az\+bG\),u\)\]=\\widetilde\{F\}\(P\_\{E\}z,u\),\\quad\\widetilde\{F\}\(x,u\):=\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{E\}\}\[F\(ax\+b\\xi,u\)\]\.\(700\)Hence,ggis cylindrical on the same subspace\. Furthermore, by differentiating under the expectation we find

∂xα∂ujF~​\(x,u\)=a\|α\|​𝔼ξ∼γE​\[∂xα∂ujF⁡\(a​x\+b​ξ,u\)\]\.\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}\\widetilde\{F\}\(x,u\)=a^\{\|\\alpha\|\}\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{E\}\}\[\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}F\(ax\+b\\xi,u\)\]\.\(701\)Sincea≥a−a\\geq a\_\{\-\}andb≤b\+b\\leq b\_\{\+\}we know that

‖x‖=a−1​‖a​x‖≤a−−1​‖a​x\+b​ξ‖\+a−−1​b\+​‖ξ‖\\\|x\\\|=a^\{\-1\}\\\|ax\\\|\\leq a\_\{\-\}^\{\-1\}\\\|ax\+b\\xi\\\|\+a\_\{\-\}^\{\-1\}b\_\{\+\}\\\|\\xi\\\|\(702\)by the triangle inequality, and so

\(1\+∥x∥\)N≲N,a−,b\+\(1\+∥ax\+bξ∥\)N\(1\+∥ξ∥\)N\.\(1\+\\\|x\\\|\)^\{N\}\\lesssim\_\{N,a\_\{\-\},b\_\{\+\}\}\(1\+\\\|ax\+b\\xi\\\|\)^\{N\}\(1\+\\\|\\xi\\\|\)^\{N\}\.\(703\)Combining this with \([701](https://arxiv.org/html/2610.10875#A1.E701)\) anda≤a\+a\\leq a\_\{\+\}, we get

\(1\+‖x‖\)N​\(1\+\|u\|\)λ\+j​\|∂xα∂ujF~​\(x,u\)\|\\displaystyle\(1\+\\\|x\\\|\)^\{N\}\(1\+\|u\|\)^\{\\lambda\+j\}\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}\\widetilde\{F\}\(x,u\)\|≲a\+\|α\|​𝔼ξ∼γE​\[\(1\+‖ξ‖\)N​\(1\+‖a​x\+b​ξ‖\)N​\(1\+\|u\|\)λ\+j​\|∂xα∂ujF⁡\(a​x\+b​ξ,u\)\|\]\\displaystyle\\lesssim a\_\{\+\}^\{\|\\alpha\|\}\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{E\}\}\\left\[\(1\+\\\|\\xi\\\|\)^\{N\}\(1\+\\\|ax\+b\\xi\\\|\)^\{N\}\(1\+\|u\|\)^\{\\lambda\+j\}\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}\{F\}\(ax\+b\\xi,u\)\|\\right\]\(704\)≲α,N,a−,a\+,b\+\[F\]α,j,N𝔼ξ∼γE\[\(1\+∥ξ∥\)N\]\.\\displaystyle\\lesssim\_\{\\alpha,N,a\_\{\-\},a\_\{\+\},b\_\{\+\}\}\[F\]\_\{\\alpha,j,N\}\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{E\}\}\[\(1\+\\\|\\xi\\\|\)^\{N\}\]\.\(705\)Since the Gaussian has all moments finite, this impliesg∈𝒞λg\\in\\mathcal\{C\}\_\{\\lambda\}with the claimed seminorm control\.

\(Barron embedding\)\.We will apply a global Sobolev\-to\-Barron embedding, whose proof we defer to Lemma[20](https://arxiv.org/html/2610.10875#Thmlemma20)\. By the triangle inequality, it suffices to consider a single summandf⁡\(z,u\)=F⁡\(PE​z,u\)f\(z,u\)=F\(P\_\{E\}z,u\)\. WriteD=dim\(E\)D=\\dim\(E\)and identifyEEwithℝD\{\\mathbb\{R\}\}^\{D\}\. FixN\>D\+sN\>D\+sandAF:=∑\|α\|\+j≤D\+s\+2\[F\]α,j,NA\_\{F\}:=\\sum\_\{\|\\alpha\|\+j\\leq D\+s\+2\}\[F\]\_\{\\alpha,j,N\}\. Choose a radial cutoffχ∈Cc∞​\(ℝ\)\\chi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}\)equal to 1 on\[−1,1\]\[\-1,1\]and supported in\[−2,2\]\[\-2,2\], and set

ζ⁡\(u\):=χ⁡\(u/2\)−χ⁡\(u\)\.\\zeta\(u\):=\\chi\(u/2\)\-\\chi\(u\)\.\(706\)Then,ζ\\zetais supported on1≤\|u\|≤41\\leq\|u\|\\leq 4and

1=χ⁡\(u\)\+∑q=0∞ζ⁡\(2−q​u\)\(∀u∈ℝ\)\.1=\\chi\(u\)\+\\sum\_\{q=0\}^\{\\infty\}\\zeta\(2^\{\-q\}u\)\\quad\(\\forall u\\in\{\\mathbb\{R\}\}\)\.\(707\)Using this, we define

Fq\(x,u\):=ζ\(2−qu\)F\(x,u\),F−1\(x,u\)=χ\(u\)F\(x,u\)\(∀q∈ℕ∪\{0\}\)\.F\_\{q\}\(x,u\):=\\zeta\(2^\{\-q\}u\)F\(x,u\),\\quad F\_\{\-1\}\(x,u\)=\\chi\(u\)F\(x,u\)\\quad\(\\forall q\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\)\.\(708\)Forq∈ℕ∪\{0\}q\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}we rescale the last coordinate

F~q​\(x,u\):=Fq​\(x,2q​u\)=ζ⁡\(u\)​F​\(x,2q​u\)\.\\widetilde\{F\}\_\{q\}\(x,u\):=F\_\{q\}\(x,2^\{q\}u\)=\\zeta\(u\)F\(x,2^\{q\}u\)\.\(709\)By the product and chain rules in theuucoordinate,

∂xα∂ujF~q​\(x,u\)=∑ℓ=0j\(jℓ\)​ζ\(j−ℓ\)​\(u\)​2q​ℓ​\(∂xα∂uℓF\)​\(x,2q​u\)\.\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}\\widetilde\{F\}\_\{q\}\(x,u\)=\\sum\_\{\\ell=0\}^\{j\}\{j\\choose\\ell\}\\zeta^\{\(j\-\\ell\)\}\(u\)2^\{q\\ell\}\(\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{\\ell\}F\)\(x,2^\{q\}u\)\.\(710\)Now, on the support ofζ\(j−ℓ\)\\zeta^\{\(j\-\\ell\)\}we have\|u\|≥1\|u\|\\geq 1, in which case2q​ℓ​\(1\+2q​\|u\|\)−λ−ℓ≤2−q​λ2^\{q\\ell\}\(1\+2^\{q\}\|u\|\)^\{\-\\lambda\-\\ell\}\\leq 2^\{\-q\\lambda\}\. This means that for all\|α\|\+j≤D\+s\+2\|\\alpha\|\+j\\leq D\+s\+2we have

\|∂xα∂ujF~q\(x,u\)\|≲D,sAF2−q​λ\(1\+∥x∥\)−N\\displaystyle\\left\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}\\widetilde\{F\}\_\{q\}\(x,u\)\\right\|\\lesssim\_\{D,s\}A\_\{F\}2^\{\-q\\lambda\}\(1\+\\\|x\\\|\)^\{\-N\}\(711\)and these derivatives vanish for\|u\|\>4\|u\|\>4\. SinceN\>D\+sN\>D\+s, we find

∑\|α\|\+j≤D\+s\+2∫ℝD×ℝ\(1\+‖\(x,u\)‖\)s​\|∂xα∂ujF~q​\(x,u\)\|​𝑑x​𝑑u\\displaystyle\\sum\_\{\|\\alpha\|\+j\\leq D\+s\+2\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\\times\{\\mathbb\{R\}\}\}\(1\+\\\|\(x,u\)\\\|\)^\{s\}\\left\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}\\widetilde\{F\}\_\{q\}\(x,u\)\\right\|dxdu≲D,s,NAF2−q​λ∫ℝD\(1\+∥x∥\)s−Ndx\\displaystyle\\lesssim\_\{D,s,N\}A\_\{F\}2^\{\-q\\lambda\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\(1\+\\\|x\\\|\)^\{s\-N\}dx\(712\)≲D,s,NAF2−q​λ\.\\displaystyle\\lesssim\_\{D,s,N\}A\_\{F\}2^\{\-q\\lambda\}\.\(713\)Applying Lemma[20](https://arxiv.org/html/2610.10875#Thmlemma20)yields

∥F~q∥ℬϕ≲D,s,NAF2−q​λ\.\\\|\\widetilde\{F\}\_\{q\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\lesssim\_\{D,s,N\}A\_\{F\}2^\{\-q\\lambda\}\.\(714\)Now, sinceFq​\(x,u\)=F~q​\(x,2−q​u\)F\_\{q\}\(x,u\)=\\widetilde\{F\}\_\{q\}\(x,2^\{\-q\}u\)we may obtain a Barron representation ofFqF\_\{q\}from that ofF~q\\widetilde\{F\}\_\{q\}by replacingϕ⁡\(⟨a,x⟩\+a′​u\+b\)\\phi\(\\left\\langle a,x\\right\\rangle\+a^\{\\prime\}u\+b\)withϕ⁡\(⟨a,x⟩\+2−q​a′​u\+b\)\\phi\(\\left\\langle a,x\\right\\rangle\+2^\{\-q\}a^\{\\prime\}u\+b\)\. This can only decrease the Barron cost, and so

∥Fq∥ℬϕ≲D,s,NAF2−q​λ\.\\\|F\_\{q\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\lesssim\_\{D,s,N\}A\_\{F\}2^\{\-q\\lambda\}\.\(715\)A similar application of Lemma[20](https://arxiv.org/html/2610.10875#Thmlemma20)yields that

∥F−1∥ℬϕ≲D,s,NAF\.\\\|F\_\{\-1\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\lesssim\_\{D,s,N\}A\_\{F\}\.\(716\)We may apply \([707](https://arxiv.org/html/2610.10875#A1.E707)\) to find that

F−1​\(x,u\)\+∑q=0QFq​\(x,u\)=χ⁡\(2−Q−1​u\)​F​\(x,u\)F\_\{\-1\}\(x,u\)\+\\sum\_\{q=0\}^\{Q\}F\_\{q\}\(x,u\)=\\chi\(2^\{\-Q\-1\}u\)F\(x,u\)\(717\)and so they converge locally uniformly toFFasQ→∞Q\\to\\infty\. By lower\-semicontinuity of the Barron norm under local uniform convergence \([15](https://arxiv.org/html/2610.10875#Thmlemma15)\) and the triangle inequality,

∥F∥ℬϕ≤∥F−1∥ℬϕ\+∑q=0∞∥Fq∥ℬϕ≲D,s,NAF\(1\+11−2−λ\)<∞\.\\displaystyle\\\|F\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\\|F\_\{\-1\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\+\\sum\_\{q=0\}^\{\\infty\}\\\|F\_\{q\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\lesssim\_\{D,s,N\}A\_\{F\}\\left\(1\+\\frac\{1\}\{1\-2^\{\-\\lambda\}\}\\right\)<\\infty\.\(718\)This gives a Barron representation onE×ℝ⊆ℝn×ℝE\\times\{\\mathbb\{R\}\}\\subseteq\{\\mathbb\{R\}\}^\{n\}\\times\{\\mathbb\{R\}\}, which extends to one on the whole space without changing the parameter norms \(since the neurons only seeE×ℝE\\times\{\\mathbb\{R\}\}\)\. ∎

###### Lemma 20\(Global Sobolev\-to\-Barron embedding forReLUs\\operatorname\{ReLU\}^\{s\}\)\.

Fix anyD,s∈ℕD,s\\in\{\\mathbb\{N\}\}withϕ⁡\(t\)=max⁡\{0,t\}s\\phi\(t\)=\\max\\\{0,t\\\}^\{s\}theReLUs\\operatorname\{ReLU\}^\{s\}activation andℬϕ\\mathcal\{B\}\_\{\\phi\}its associated Barron space of functions fromℝD→ℝ\{\\mathbb\{R\}\}^\{D\}\\to\{\\mathbb\{R\}\}\(using the scalar version of Definition[4](https://arxiv.org/html/2610.10875#Thmdefinition4)\)\.

Then, there is someCD,s∈\(0,∞\)C\_\{D,s\}\\in\(0,\\infty\)s\.t\. for anyf∈WD\+s\+1,1​\(ℝD\)f\\in W^\{D\+s\+1,1\}\(\{\\mathbb\{R\}\}^\{D\}\)if we choose its continuous representative we have

‖f‖ℬϕ≤CD,s​∑\|α\|≤D\+s\+1∫ℝD\(1\+\|x\|\)s​\|∂αf⁡\(x\)\|​𝑑x\.\\\|f\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq C\_\{D,s\}\\sum\_\{\|\\alpha\|\\leq D\+s\+1\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\(1\+\|x\|\)^\{s\}\|\\partial^\{\\alpha\}f\(x\)\|dx\.\(719\)

###### Proof\.

First, suppose thatDDis odd andf∈Cc∞​\(ℝD\)f\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{D\}\)\. Write its Radon transform by

R​f​\(θ,t\):=∫θ⟂f⁡\(t​θ\+y\)​𝑑y\.Rf\(\\theta,t\):=\\int\_\{\\theta^\{\\perp\}\}f\(t\\theta\+y\)dy\.\(720\)By the odd\-dimensional Radon inversion formula \(see \(27\) and \(28\) of\[[66](https://arxiv.org/html/2610.10875#bib.bib20)\]\), we have

f⁡\(x\)=cD​∫𝕊D−1\(∂tD−1R​f\)​\(θ,⟨θ,x⟩\)​𝑑θ,cD=\(−1\)\(D−1\)/22​\(2​π\)D−1\.f\(x\)=c\_\{D\}\\int\_\{\\mathbb\{S\}^\{D\-1\}\}\(\\partial\_\{t\}^\{D\-1\}Rf\)\(\\theta,\\left\\langle\\theta,x\\right\\rangle\)d\\theta,\\quad c\_\{D\}=\\frac\{\(\-1\)^\{\(D\-1\)/2\}\}\{2\(2\\pi\)^\{D\-1\}\}\.\(721\)We have seen that for everyg∈Cc∞​\(ℝ\)g\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}\), repeated integration by parts gives

g⁡\(v\)=1s\!​∫ℝmax⁡\{0,v−t\}s​g\(s\+1\)​\(t\)​𝑑t\.g\(v\)=\\frac\{1\}\{s\!\}\\int\_\{\\mathbb\{R\}\}\\max\\\{0,v\-t\\\}^\{s\}g^\{\(s\+1\)\}\(t\)dt\.\(722\)Applying this tog⁡\(⋅\)=∂tD−1R​f​\(θ,⋅\)g\(\\cdot\)=\\partial\_\{t\}^\{D\-1\}Rf\(\\theta,\\cdot\)gives the global representation

f⁡\(x\)=cDs\!​∫𝕊D−1∫ℝmax⁡\{0,⟨θ,x⟩−t\}s​\(∂tD\+sR​f\)​\(θ,t\)​𝑑t​𝑑θ\.f\(x\)=\\frac\{c\_\{D\}\}\{s\!\}\\int\_\{\\mathbb\{S\}^\{D\-1\}\}\\int\_\{\\mathbb\{R\}\}\\max\\\{0,\\left\\langle\\theta,x\\right\\rangle\-t\\\}^\{s\}\(\\partial\_\{t\}^\{D\+s\}Rf\)\(\\theta,t\)dtd\\theta\.\(723\)By differentiating under the integral, there is the Radon transform identity

\(∂tkR​f\)​\(θ,t\)\\displaystyle\(\\partial\_\{t\}^\{k\}Rf\)\(\\theta,t\)=∫θ⟂∂tk\[f⁡\(t​θ\+y\)\]​𝑑y\\displaystyle=\\int\_\{\\theta^\{\\perp\}\}\\partial\_\{t\}^\{k\}\[f\(t\\theta\+y\)\]dy\(724\)=∫θ⟂\(θ⋅∇\)kf\(tθ\+y\)dy\\displaystyle=\\int\_\{\\theta^\{\\perp\}\}\(\\theta\\cdot\\nabla\)^\{k\}f\(t\\theta\+y\)dy\(725\)=R\[\(θ⋅∇\)kf\]\(θ,t\),\\displaystyle=R\[\(\\theta\\cdot\\nabla\)^\{k\}f\]\(\\theta,t\),\(726\)where\(θ⋅∇\)kf\(\\theta\\cdot\\nabla\)^\{k\}fis thekt​hk^\{th\}directional derivative offfin directionθ\\theta\. Since∂tD\+sRf=R\[\(θ⋅∇\)D\+sf\]\\partial\_\{t\}^\{D\+s\}Rf=R\[\(\\theta\\cdot\\nabla\)^\{D\+s\}f\], the homogeneous Barron cost of this representation is

‖f‖ℬϕ\\displaystyle\\\|f\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}≤\|cD\|s\!∫𝕊D−1∫ℝ\(1\+\|t\|\)s\|R\[\(θ⋅∇\)D\+sf\]\(θ,t\)\|dtdθ\\displaystyle\\leq\\frac\{\|c\_\{D\}\|\}\{s\!\}\\int\_\{\\mathbb\{S\}^\{D\-1\}\}\\int\_\{\\mathbb\{R\}\}\(1\+\|t\|\)^\{s\}\|R\[\(\\theta\\cdot\\nabla\)^\{D\+s\}f\]\(\\theta,t\)\|dtd\\theta\(727\)=\|cD\|s\!∫𝕊D−1∫ℝ\(1\+\|t\|\)s\|∫θ⟂\(θ⋅∇\)D\+sf\(tθ\+y\)dy\|dtdθ\\displaystyle=\\frac\{\|c\_\{D\}\|\}\{s\!\}\\int\_\{\\mathbb\{S\}^\{D\-1\}\}\\int\_\{\\mathbb\{R\}\}\(1\+\|t\|\)^\{s\}\\left\|\\int\_\{\\theta^\{\\perp\}\}\(\\theta\\cdot\\nabla\)^\{D\+s\}f\(t\\theta\+y\)dy\\right\|dtd\\theta\(728\)≤\|cD\|s\!∫𝕊D−1∫ℝ∫θ⟂\(1\+\|t\|\)s\|\(θ⋅∇\)D\+sf\(tθ\+y\)\|dydtdθ\\displaystyle\\leq\\frac\{\|c\_\{D\}\|\}\{s\!\}\\int\_\{\\mathbb\{S\}^\{D\-1\}\}\\int\_\{\\mathbb\{R\}\}\\int\_\{\\theta^\{\\perp\}\}\(1\+\|t\|\)^\{s\}\\left\|\(\\theta\\cdot\\nabla\)^\{D\+s\}f\(t\\theta\+y\)\\right\|dydtd\\theta\(729\)=\|cD\|s\!∫𝕊D−1∫ℝD\(1\+\|⟨θ,x⟩\|\)s\|\(θ⋅∇\)D\+sf\(x\)\|dxdθ,\\displaystyle=\\frac\{\|c\_\{D\}\|\}\{s\!\}\\int\_\{\\mathbb\{S\}^\{D\-1\}\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\(1\+\|\\left\\langle\\theta,x\\right\\rangle\|\)^\{s\}\\left\|\(\\theta\\cdot\\nabla\)^\{D\+s\}f\(x\)\\right\|dxd\\theta,\(730\)where the last line fixesθ\\thetaand uses the change of coordinatesx=t​θ\+yx=t\\theta\+y\(which has Jacobian 1\)\. By the multinomial formula, we know

\(θ⋅∇\)D\+sf=∑\|α\|=D\+s\(D\+s\)\!∏jαj\!θα∂αf\.\(\\theta\\cdot\\nabla\)^\{D\+s\}f=\\sum\_\{\|\\alpha\|=D\+s\}\\frac\{\(D\+s\)\!\}\{\\prod\_\{j\}\\alpha\_\{j\}\!\}\\theta^\{\\alpha\}\\partial^\{\\alpha\}f\.\(731\)Since\|θα\|≤1\|\\theta^\{\\alpha\}\|\\leq 1,\|⟨θ,x⟩\|≤‖x‖\|\\left\\langle\\theta,x\\right\\rangle\|\\leq\\\|x\\\|, and∏jαj\!≥1\\prod\_\{j\}\\alpha\_\{j\}\!\\geq 1, we find that

‖f‖ℬϕ≤\|cD\|⋅\|𝕊D−1\|​\(D\+s\)\!s\!​∑\|α\|=D\+s∫ℝD\(1\+‖x‖\)s​\|∂αf⁡\(x\)\|​𝑑x\.\\\|f\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\frac\{\|c\_\{D\}\|\\cdot\|\\mathbb\{S\}^\{D\-1\}\|\(D\+s\)\!\}\{s\!\}\\sum\_\{\|\\alpha\|=D\+s\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\(1\+\\\|x\\\|\)^\{s\}\|\\partial^\{\\alpha\}f\(x\)\|dx\.\(732\)SettingCD,s:=\|cD\|⋅\|𝕊D−1\|​\(D\+s\)\!s\!C\_\{D,s\}:=\\frac\{\|c\_\{D\}\|\\cdot\|\\mathbb\{S\}^\{D\-1\}\|\(D\+s\)\!\}\{s\!\}yields the claim for oddDDandf∈Cc∞​\(ℝD\)f\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{D\}\)\.

Letting nowDDbe even andf∈Cc∞​\(ℝD\)f\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{D\}\), we fixη∈Cc∞​\(ℝ\)\\eta\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}\)withη⁡\(0\)=1\\eta\(0\)=1and define

F⁡\(x,u\):=f⁡\(x\)​η​\(u\)\.F\(x,u\):=f\(x\)\\eta\(u\)\.\(733\)Restricting any Barron representation ofFFtou=0u=0gives a Barron representation offfwith no increase in parameter norm, since eachϕ⁡\(⟨a,x⟩\+a′​u\+b\)\\phi\(\\left\\langle a,x\\right\\rangle\+a^\{\\prime\}u\+b\)becomesϕ⁡\(⟨a,x⟩\+b\)\\phi\(\\left\\langle a,x\\right\\rangle\+b\)\. Hence,

‖f‖ℬϕ≤‖F‖ℬϕ\.\\\|f\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\\|F\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\.\(734\)Let us define for evenDDthe constant

CD,s:=CD\+1,s​max⁡∫ℝ0≤j≤D\+s\+1⁡\(1\+\|u\|\)s​\|η\(j\)​\(u\)\|​𝑑uC\_\{D,s\}:=C\_\{D\+1,s\}\\max\_\{0\\leq j\\leq D\+s\+1\}\\int\_\{\\mathbb\{R\}\}\(1\+\|u\|\)^\{s\}\|\\eta^\{\(j\)\}\(u\)\|du\(735\)for the fixedη\\eta\(which we will use for allf:ℝD→ℝf:\{\\mathbb\{R\}\}^\{D\}\\to\{\\mathbb\{R\}\}\)\. Using thatD\+1D\+1is odd, we apply \([732](https://arxiv.org/html/2610.10875#A1.E732)\) along with the product rule to see that

‖F‖ℬϕ\\displaystyle\\\|F\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}≤CD\+1,s​∑\|α\|\+j=D\+s\+1∫ℝD∫ℝ\(1\+‖\(x,u\)‖\)s​\|∂αf⁡\(x\)\|​\|η\(j\)​\(u\)\|​𝑑u​𝑑x\\displaystyle\\leq C\_\{D\+1,s\}\\sum\_\{\|\\alpha\|\+j=D\+s\+1\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\\int\_\{\\mathbb\{R\}\}\(1\+\\\|\(x,u\)\\\|\)^\{s\}\|\\partial^\{\\alpha\}f\(x\)\|\|\\eta^\{\(j\)\}\(u\)\|dudx\(736\)≤CD\+1,s​∑\|α\|\+j=D\+s\+1\(∫ℝD\(1\+‖x‖\)s​\|∂αf⁡\(x\)\|​𝑑x\)​\(∫ℝ\(1\+\|u\|\)s​\|η\(j\)​\(u\)\|​𝑑u\)\\displaystyle\\leq C\_\{D\+1,s\}\\sum\_\{\|\\alpha\|\+j=D\+s\+1\}\\left\(\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\(1\+\\\|x\\\|\)^\{s\}\|\\partial^\{\\alpha\}f\(x\)\|dx\\right\)\\left\(\\int\_\{\\mathbb\{R\}\}\(1\+\|u\|\)^\{s\}\|\\eta^\{\(j\)\}\(u\)\|du\\right\)\(737\)≤CD,s​∑\|α\|≤D\+s\+1∫ℝD\(1\+‖x‖\)s​\|∂αf⁡\(x\)\|​𝑑x,\\displaystyle\\leq\{C\}\_\{D,s\}\\sum\_\{\|\\alpha\|\\leq D\+s\+1\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\(1\+\\\|x\\\|\)^\{s\}\|\\partial^\{\\alpha\}f\(x\)\|dx,\(738\)where we used that1\+‖\(x,u\)‖≤\(1\+‖x‖\)​\(1\+\|u\|\)1\+\\\|\(x,u\)\\\|\\leq\(1\+\\\|x\\\|\)\(1\+\|u\|\)\. Hence, the claim holds for anyD∈ℕD\\in\{\\mathbb\{N\}\}and anyf∈Cc∞​\(ℝD\)f\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{D\}\)\. For generalffwith finite right hand side, we takefj∈Cc∞​\(ℝD\)f\_\{j\}\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}^\{D\}\)s\.t\.fj→ff\_\{j\}\\to flocally uniformly and

∑\|α\|≤D\+s\+1∫ℝD\(1\+\|x\|\)s​\|∂α\(fj−f\)​\(x\)\|​𝑑x→0\\sum\_\{\|\\alpha\|\\leq D\+s\+1\}\\int\_\{\{\\mathbb\{R\}\}^\{D\}\}\(1\+\|x\|\)^\{s\}\|\\partial^\{\\alpha\}\(f\_\{j\}\-f\)\(x\)\|dx\\to 0\(739\)using cutoff and mollification\. By lower\-semicontinuity of the Barron norm w\.r\.t\. local uniform convergence \(Lemma[15](https://arxiv.org/html/2610.10875#Thmlemma15)\), the claim follows\. ∎

###### Lemma 21\(Shallow representation upper bound, nonquantitative\)\.

Fixs∈ℕs\\in\{\\mathbb\{N\}\}and letϕ⁡\(t\)=max⁡\{0,t\}s\\phi\(t\)=\\max\\\{0,t\\\}^\{s\}be theReLUs\\operatorname\{ReLU\}^\{s\}activation withℬϕ\\mathcal\{B\}\_\{\\phi\}its associated Barron space \(see Definition[4](https://arxiv.org/html/2610.10875#Thmdefinition4)\)\. Letn∈ℕn\\in\{\\mathbb\{N\}\}and supposeΨ⁡\(z\)=∑r=1MΦr​\(PEr​z\)\\Psi\(z\)=\\sum\_\{r=1\}^\{M\}\\Phi\_\{r\}\(P\_\{E\_\{r\}\}z\), where eachEr⊆ℝnE\_\{r\}\\subseteq\{\\mathbb\{R\}\}^\{n\}is a linear subspace \(with orthogonal projectionPErP\_\{E\_\{r\}\}\) andΦr∈Cc∞​\(Er\)\\Phi\_\{r\}\\in C^\{\\infty\}\_\{c\}\(E\_\{r\}\)\. Letψ∈C∞​\(ℝ\)\\psi\\in C^\{\\infty\}\(\{\\mathbb\{R\}\}\)be a strictly increasing diffeomorphism and choosea±,R∈\(0,∞\)a\_\{\\pm\},R\\in\(0,\\infty\),b±∈ℝb\_\{\\pm\}\\in\{\\mathbb\{R\}\}, andp\>2p\>2s\.t\.

ψ⁡\(u\)=a\+​up\+b\+\(∀u≥R\),ψ⁡\(u\)=−a−​\(−u\)p\+b−\(∀u≤−R\)\.\\psi\(u\)=a\_\{\+\}u^\{p\}\+b\_\{\+\}\\quad\(\\forall u\\geq R\),\\quad\\psi\(u\)=\-a\_\{\-\}\(\-u\)^\{p\}\+b\_\{\-\}\\quad\(\\forall u\\leq\-R\)\.\(740\)Then,sτψ,Ψ,bτψ,Ψ∈ℬϕs\_\{\\tau\}^\{\\psi,\\Psi\},b\_\{\\tau\}^\{\\psi,\\Psi\}\\in\\mathcal\{B\}\_\{\\phi\}for allτ≥0\\tau\\geq 0with

supτ∈\[0,T\]max⁡\{‖sτψ,Ψ‖ℬϕ,‖bτψ,Ψ‖ℬϕ\}<∞\(∀T∈\[0,∞\)\)\.\\sup\_\{\\tau\\in\[0,T\]\}\\max\\left\\\{\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\},\\\|b\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\right\\\}<\\infty\\quad\(\\forall T\\in\[0,\\infty\)\)\.\(741\)

###### Proof\.

FixT\>0T\>0and letqτq\_\{\\tau\}be the density ofe−τ​ψ​\(U\)\+σ​He^\{\-\\tau\}\\psi\(U\)\+\\sigma HforU,H∼γ1U,H\\sim\\gamma\_\{1\}independent andσ:=1−e−2​τ\\sigma:=\\sqrt\{1\-e^\{\-2\\tau\}\}\(as in the proof of Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)\)\. Denote its score byhτ:=qτ′/qτh\_\{\\tau\}:=q\_\{\\tau\}^\{\\prime\}/q\_\{\\tau\}\. Sinceψ\\psiis exactly a power function of exponentppoutside of a compact set, the change of variables formula gives that

q0\(u\)=12​π​p​a\+1/p\(u−b\+\)−\(p−1\)/pexp\(−a\+−2/p2\(u−b\+\)2/p\)q\_\{0\}\(u\)=\\frac\{1\}\{\\sqrt\{2\\pi\}pa\_\{\+\}^\{1/p\}\}\(u\-b\_\{\+\}\)^\{\-\(p\-1\)/p\}\\exp\\left\(\-\\frac\{a\_\{\+\}^\{\-2/p\}\}\{2\}\(u\-b\_\{\+\}\)^\{2/p\}\\right\)\(742\)foruularge and therefore ifλ:=1−2p\\lambda:=1\-\\frac\{2\}\{p\}we have

h0​\(u\)=−a\+−2/pp​\(u−b\+\)−λ−p−1p⁡\(u−b\+\)\.h\_\{0\}\(u\)=\-\\frac\{a\_\{\+\}^\{\-2/p\}\}\{p\}\(u\-b\_\{\+\}\)^\{\-\\lambda\}\-\\frac\{p\-1\}\{p\(u\-b\_\{\+\}\)\}\.\(743\)The analogous formula holds on the negative tail\. Hence, for everyj∈ℕ∪\{0\}j\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}we have

\|h0\(j\)\(u\)\|≲j\(1\+\|u\|\)−λ−j\.\|h\_\{0\}^\{\(j\)\}\(u\)\|\\lesssim\_\{j\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\.\(744\)We claim that this estimate persists under OU smoothing, i\.e\. that

supτ∈\[0,T\]supu∈ℝ\(1\+\|u\|\)λ\+j​\|hτ\(j\)​\(u\)\|<∞\(∀j∈ℕ∪\{0\}\)\.\\sup\_\{\\tau\\in\[0,T\]\}\\sup\_\{u\\in\{\\mathbb\{R\}\}\}\(1\+\|u\|\)^\{\\lambda\+j\}\|h\_\{\\tau\}^\{\(j\)\}\(u\)\|<\\infty\\quad\(\\forall j\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\)\.\(745\)Indeed, sinceqτ\(⋅\)=eτq0\(eτ⋅\)∗φσ2q\_\{\\tau\}\(\\cdot\)=e^\{\\tau\}q\_\{0\}\(e^\{\\tau\}\\cdot\)\*\\varphi\_\{\\sigma^\{2\}\}we may denoteq~τ​\(u\):=eτ​q0​\(eτ​u\)\\tilde\{q\}\_\{\\tau\}\(u\):=e^\{\\tau\}q\_\{0\}\(e^\{\\tau\}u\)and write

qτ​\(u\)=q~τ​\(u\)​Gτ​\(u\),Gτ​\(u\):=𝔼⁡\[exp⁡\(ℓτ​\(u−σ​H\)−ℓτ​\(u\)\)\],ℓτ:=log⁡q~τ\.q\_\{\\tau\}\(u\)=\\tilde\{q\}\_\{\\tau\}\(u\)G\_\{\\tau\}\(u\),\\quad G\_\{\\tau\}\(u\):=\\mathbb\{E\}\\left\[\\exp\\left\(\\ell\_\{\\tau\}\(u\-\\sigma H\)\-\\ell\_\{\\tau\}\(u\)\\right\)\\right\],\\quad\\ell\_\{\\tau\}:=\\log\\tilde\{q\}\_\{\\tau\}\.\(746\)From \([742](https://arxiv.org/html/2610.10875#A1.E742)\) we know that

\|ℓτ\(j\+1\)\(u\)\|≲j,T\(1\+\|u\|\)−λ−j\|\\ell\_\{\\tau\}^\{\(j\+1\)\}\(u\)\|\\lesssim\_\{j,T\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\(747\)uniformly forτ∈\[0,T\]\\tau\\in\[0,T\]\. Since‖ℓτ′‖∞≤CT\\\|\\ell\_\{\\tau\}^\{\\prime\}\\\|\_\{\\infty\}\\leq C\_\{T\}is uniformly bounded forτ∈\[0,T\]\\tau\\in\[0,T\]we knowGτ​\(u\)≤𝔼⁡\[eCT​\|H\|\]<∞G\_\{\\tau\}\(u\)\\leq\\mathbb\{E\}\[e^\{C\_\{T\}\|H\|\}\]<\\infty\. Similarly, we knowGτ\(u\)≥e−CTℙ\(\|H\|≤1\)≳T1G\_\{\\tau\}\(u\)\\geq e^\{\-C\_\{T\}\}\\mathbb\{P\}\(\|H\|\\leq 1\)\\gtrsim\_\{T\}1, and therefore

Gτ\(u\)≍T1G\_\{\\tau\}\(u\)\\asymp\_\{T\}1\(748\)uniformly forτ∈\[0,T\]\\tau\\in\[0,T\]\. On the event\|σ​H\|≤\|u\|/2\|\\sigma H\|\\leq\|u\|/2, by the mean value theorem and \([747](https://arxiv.org/html/2610.10875#A1.E747)\) we have

\|ℓτ\(j\)\(u−σH\)−ℓτ\(j\)\(u\)\|≲j,T\|H\|\(1\+\|u\|\)−λ−j\(∀j∈ℕ∪\{0\}\),\\left\|\\ell\_\{\\tau\}^\{\(j\)\}\(u\-\\sigma H\)\-\\ell\_\{\\tau\}^\{\(j\)\}\(u\)\\right\|\\lesssim\_\{j,T\}\|H\|\(1\+\|u\|\)^\{\-\\lambda\-j\}\\quad\(\\forall j\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\),\(749\)By Faà di Bruno, forj∈ℕj\\in\{\\mathbb\{N\}\}on this event we know that

\|∂ujeℓτ​\(u−σ​H\)−ℓτ​\(u\)\|\\displaystyle\|\\partial\_\{u\}^\{j\}e^\{\\ell\_\{\\tau\}\(u\-\\sigma H\)\-\\ell\_\{\\tau\}\(u\)\}\|≲j,TeCT​\|H\|∑m,r1,…,rm∈ℕr1\+…\+rm=j\|H\|m\(1\+\|u\|\)−m​λ−j\\displaystyle\\lesssim\_\{j,T\}e^\{C\_\{T\}\|H\|\}\\sum\_\{\\begin\{subarray\}\{c\}m,r\_\{1\},\\ldots,r\_\{m\}\\in\{\\mathbb\{N\}\}\\\\ r\_\{1\}\+\\ldots\+r\_\{m\}=j\\end\{subarray\}\}\|H\|^\{m\}\(1\+\|u\|\)^\{\-m\\lambda\-j\}\(750\)≲j,TeCT​\|H\|\(1\+\|H\|\)j\(1\+\|u\|\)−λ−j,\\displaystyle\\lesssim\_\{j,T\}e^\{C\_\{T\}\|H\|\}\(1\+\|H\|\)^\{j\}\(1\+\|u\|\)^\{\-\\lambda\-j\},\(751\)where the second inequality usesλ\>0\\lambda\>0\. On the other hand, the bound \([747](https://arxiv.org/html/2610.10875#A1.E747)\) implies that

\|∂ujeℓτ​\(u−σ​H\)−ℓτ​\(u\)\|≲j,TecT​\|H\|\(1\+\|H\|\)j,\|\\partial\_\{u\}^\{j\}e^\{\\ell\_\{\\tau\}\(u\-\\sigma H\)\-\\ell\_\{\\tau\}\(u\)\}\|\\lesssim\_\{j,T\}e^\{c\_\{T\}\|H\|\}\(1\+\|H\|\)^\{j\},\(752\)and since

𝔼\[eCT​\|H\|\(1\+\|H\|\)j𝟙\|σ​H\|\>\|u\|/2\]≲j,Te−cT​u2\\mathbb\{E\}\\left\[e^\{C\_\{T\}\|H\|\}\(1\+\|H\|\)^\{j\}\{\\mathbbm\{1\}\}\_\{\|\\sigma H\|\>\|u\|/2\}\\right\]\\lesssim\_\{j,T\}e^\{\-c\_\{T\}u^\{2\}\}\(753\)for some constantcTc\_\{T\}we find that

\|Gτ\(j\)\(u\)\|≲j,T\(1\+\|u\|\)−λ−j\(∀j∈ℕ\)\.\|G\_\{\\tau\}^\{\(j\)\}\(u\)\|\\lesssim\_\{j,T\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\\quad\(\\forall j\\in\{\\mathbb\{N\}\}\)\.\(754\)SinceGτ≍T1G\_\{\\tau\}\\asymp\_\{T\}1, we find that

\|∂jlogGτ\(u\)\|≲j,T\(1\+\|u\|\)−λ−j\(∀j∈ℕ\)\.\|\\partial^\{j\}\\log G\_\{\\tau\}\(u\)\|\\lesssim\_\{j,T\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\\quad\(\\forall j\\in\{\\mathbb\{N\}\}\)\.\(755\)However, sincelog⁡qτ=ℓτ\+log⁡Gτ\\log q\_\{\\tau\}=\\ell\_\{\\tau\}\+\\log G\_\{\\tau\}and sohτ\(j\)=ℓτ\(j\+1\)\+∂\(j\+1\)log⁡Gτh\_\{\\tau\}^\{\(j\)\}=\\ell\_\{\\tau\}^\{\(j\+1\)\}\+\\partial^\{\(j\+1\)\}\\log G\_\{\\tau\}, we may apply \([755](https://arxiv.org/html/2610.10875#A1.E755)\) and \([747](https://arxiv.org/html/2610.10875#A1.E747)\) to show the claim \([745](https://arxiv.org/html/2610.10875#A1.E745)\)\.

Moving on, \([745](https://arxiv.org/html/2610.10875#A1.E745)\) implies that\(1\+\|u\|\)s​\|hτ\(s\+1\)​\(u\)\|≲\(1\+\|u\|\)−1−λ\(1\+\|u\|\)^\{s\}\|h\_\{\\tau\}^\{\(s\+1\)\}\(u\)\|\\lesssim\(1\+\|u\|\)^\{\-1\-\\lambda\}and so

supτ∈\[0,T\]∫ℝ\(1\+\|u\|\)s​\|hτ\(s\+1\)​\(u\)\|​𝑑u<∞\.\\sup\_\{\\tau\\in\[0,T\]\}\\int\_\{\\mathbb\{R\}\}\(1\+\|u\|\)^\{s\}\|h\_\{\\tau\}^\{\(s\+1\)\}\(u\)\|du<\\infty\.\(756\)The bound \([745](https://arxiv.org/html/2610.10875#A1.E745)\) also implies thathτ\(j\)→0h\_\{\\tau\}^\{\(j\)\}\\to 0asu→−∞u\\to\-\\infty, and so repeated application of the fundamental theorem of calculus yields

hτ​\(v\)=1s\!​∫ℝhτ\(s\+1\)​\(u\)​max⁡\{0,v−u\}s​𝑑u\.h\_\{\\tau\}\(v\)=\\frac\{1\}\{s\!\}\\int\_\{\\mathbb\{R\}\}h\_\{\\tau\}^\{\(s\+1\)\}\(u\)\\max\\\{0,v\-u\\\}^\{s\}du\.\(757\)This is a valid Barron representation, and so \([756](https://arxiv.org/html/2610.10875#A1.E756)\) yieldssupτ∈\[0,T\]‖hτ‖ℬReLUs<∞\\sup\_\{\\tau\\in\[0,T\]\}\\\|h\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{s\}\}\}<\\infty\.

Next, using the notation from the proof of Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)we have the density

ρτψ,Ψ​\(z,u\)=γn​\(z\)​Fτ​\(z,u\),Fτ​\(z,u\)=𝔼G∼γn​\[qτ​\(u−e−τ​Ψ​\(e−τ​z\+1−e−2​τ​G\)\)\]\.\\rho\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)=\\gamma\_\{n\}\(z\)F\_\{\\tau\}\(z,u\),\\quad F\_\{\\tau\}\(z,u\)=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\[q\_\{\\tau\}\(u\-e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\)\)\]\.\(758\)If we define the quantity

Rτ​\(z,u\):=Fτ​\(z,u\)qτ​\(u\),Sτ:=∇log⁡Rτ,R\_\{\\tau\}\(z,u\):=\\frac\{F\_\{\\tau\}\(z,u\)\}\{q\_\{\\tau\}\(u\)\},\\quad S\_\{\\tau\}:=\\nabla\\log R\_\{\\tau\},\(759\)then using that

log⁡ρτψ,Ψ​\(z,u\)=log⁡γn​\(z\)\+log⁡qτ​\(u\)\+log⁡Rτ​\(z,u\)\\log\\rho\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)=\\log\\gamma\_\{n\}\(z\)\+\\log q\_\{\\tau\}\(u\)\+\\log R\_\{\\tau\}\(z,u\)\(760\)we find that the scores satisfy

sτψ,Ψ​\(z,u\)=\[−zhτ​\(u\)\]\+Sτ​\(z,u\),bτψ,Ψ=\[0u\+hτ​\(u\)\]\+Sτ​\(z,u\)\.s\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)=\\begin\{bmatrix\}\-z\\\\ h\_\{\\tau\}\(u\)\\end\{bmatrix\}\+S\_\{\\tau\}\(z,u\),\\quad b\_\{\\tau\}^\{\\psi,\\Psi\}=\\begin\{bmatrix\}0\\\\ u\+h\_\{\\tau\}\(u\)\\end\{bmatrix\}\+S\_\{\\tau\}\(z,u\)\.\(761\)We claim that affine vector fields are inℬReLUs\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{s\}\}; to see this, note that for anyr0,…,rsr\_\{0\},\\ldots,r\_\{s\}distinct we have thatt↦\(t\+r0\)s,…,t↦\(t\+rs\)st\\mapsto\(t\+r\_\{0\}\)^\{s\},\\ldots,t\\mapsto\(t\+r\_\{s\}\)^\{s\}span the space of polynomials of degree at mostss\(since the coefficient matrix is Vandermonde and therefore invertible\)\. Hence, every affine map is a linear combination of theset↦\(t\+rj\)st\\mapsto\(t\+r\_\{j\}\)^\{s\}polynomials\. However, since

\(t\+r\)s=max⁡\{0,t\+r\}s\+\(−1\)s​max⁡\{0,−t−r\}s\(t\+r\)^\{s\}=\\max\\\{0,t\+r\\\}^\{s\}\+\(\-1\)^\{s\}\\max\\\{0,\-t\-r\\\}^\{s\}\(762\)we can apply it coordinatewise to find that affine vector fields are Barron\. In light of \([761](https://arxiv.org/html/2610.10875#A1.E761)\) it remains to show thatsupτ∈\[0,T\]‖Sτ‖ℬReLUs<∞\\sup\_\{\\tau\\in\[0,T\]\}\\\|S\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{s\}\}\}<\\infty\.

To this end, if we denoteB:=∑r=1M‖Φr‖∞B:=\\sum\_\{r=1\}^\{M\}\\\|\\Phi\_\{r\}\\\|\_\{\\infty\}andKτ​\(u,c\):=qτ​\(u−c\)/qτ​\(u\)K\_\{\\tau\}\(u,c\):=q\_\{\\tau\}\(u\-c\)/q\_\{\\tau\}\(u\)then by definition we have

Rτ​\(z,u\)=𝔼G∼γn​\[Kτ​\(u,e−τ​Ψ​\(e−τ​z\+σ​G\)\)\]\.R\_\{\\tau\}\(z,u\)=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\[K\_\{\\tau\}\(u,e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sigma G\)\)\]\.\(763\)Differentiating under the expectation, forα≠0\\alpha\\neq 0every term∂zα∂ujRτ​\(z,u\)\\partial\_\{z\}^\{\\alpha\}\\partial\_\{u\}^\{j\}R\_\{\\tau\}\(z,u\)has the form

C​e−\(\|α\|\+k\)​τ​𝔼G∼γn​\[\(∂ck∂ujKτ\)​\(u,e−τ​Ψ​\(e−τ​z\+σ​G\)\)​∏ℓ=1k∂βℓΨ⁡\(e−τ​z\+σ​G\)\]Ce^\{\-\(\|\\alpha\|\+k\)\\tau\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\\left\[\(\\partial\_\{c\}^\{k\}\\partial\_\{u\}^\{j\}K\_\{\\tau\}\)\(u,e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sigma G\)\)\\prod\_\{\\ell=1\}^\{k\}\\partial^\{\\beta\_\{\\ell\}\}\\Psi\(e^\{\-\\tau\}z\+\\sigma G\)\\right\]\(764\)withk≥1k\\geq 1,\|βℓ\|≥1\|\\beta\_\{\\ell\}\|\\geq 1, and∑ℓβℓ=α\\sum\_\{\\ell\}\\beta\_\{\\ell\}=\\alpha\. Forα=0\\alpha=0, we use instead that

∂uj\(Rτ−1\)=𝔼G∼γn​\[∂uj\(Kτ​\(u,e−τ​Ψ​\(e−τ​z\+σ​G\)\)−1\)\]\.\\partial\_\{u\}^\{j\}\(R\_\{\\tau\}\-1\)=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\[\\partial\_\{u\}^\{j\}\(K\_\{\\tau\}\(u,e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sigma G\)\)\-1\)\]\.\(765\)Now, since we have

logKτ\(u,c\)=logqτ\(u−c\)−logqτ\(u\)=−∫u−cuhτ\(v\)dv,\\log K\_\{\\tau\}\(u,c\)=\\log q\_\{\\tau\}\(u\-c\)\-\\log q\_\{\\tau\}\(u\)=\-\\int\_\{u\-c\}^\{u\}h\_\{\\tau\}\(v\)dv,\(766\)the estimate \([745](https://arxiv.org/html/2610.10875#A1.E745)\) onhτh\_\{\\tau\}implies thatKτ\(u,c\)≍T1K\_\{\\tau\}\(u,c\)\\asymp\_\{T\}1uniformly in\|c\|≤B\|c\|\\leq Band furthermore

\|∂ck∂ujKτ\(u,c\)\|≲k,j,T\(1\+\|u\|\)−λ−j\(∀k\+j≥1\)\|\\partial\_\{c\}^\{k\}\\partial\_\{u\}^\{j\}K\_\{\\tau\}\(u,c\)\|\\lesssim\_\{k,j,T\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\\quad\(\\forall k\+j\\geq 1\)\(767\)uniformly in\|c\|≤B\|c\|\\leq B\. SinceKτ​\(u,0\)=1K\_\{\\tau\}\(u,0\)=1, the fundamental theorem of calculus in thecccoordinate implies that for allj∈ℕ∪\{0\}j\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}we have

\|∂uj\(Kτ\(u,c\)−1\)\|≲j,T\|c\|\(1\+\|u\|\)−λ−j\.\|\\partial\_\{u\}^\{j\}\(K\_\{\\tau\}\(u,c\)\-1\)\|\\lesssim\_\{j,T\}\|c\|\(1\+\|u\|\)^\{\-\\lambda\-j\}\.\(768\)We define the new functionAτ:ℝn×ℝ→ℝA\_\{\\tau\}:\{\\mathbb\{R\}\}^\{n\}\\times\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}by

Aτ​\(z,u\):=Kτ​\(u,e−τ​Ψ​\(z\)\)−1\.A\_\{\\tau\}\(z,u\):=K\_\{\\tau\}\(u,e^\{\-\\tau\}\\Psi\(z\)\)\-1\.\(769\)Apply the anchored fundamental theorem of calculus, as in the proof of Lemma[19](https://arxiv.org/html/2610.10875#Thmlemma19)\(ii\), to\(y1,…,yM\)↦Kτ​\(u,e−τ​∑r=1Myr\)−1\(y\_\{1\},\\ldots,y\_\{M\}\)\\mapsto K\_\{\\tau\}\(u,e^\{\-\\tau\}\\sum\_\{r=1\}^\{M\}y\_\{r\}\)\-1\. This function vanishes aty=0y=0\(sinceKτ​\(u,0\)=1K\_\{\\tau\}\(u,0\)=1\) and so

Aτ​\(z,u\)=∑∅≠J⊆\{1,…,M\}Hτ,J​\(PEJ​z,u\),EJ:=∑r∈JEr,A\_\{\\tau\}\(z,u\)=\\sum\_\{\\emptyset\\neq J\\subseteq\\\{1,\\ldots,M\\\}\}H\_\{\\tau,J\}\(P\_\{E\_\{J\}\}z,u\),\\quad E\_\{J\}:=\\sum\_\{r\\in J\}E\_\{r\},\(770\)where forx∈EJx\\in E\_\{J\}we define

Hτ,J​\(x,u\):=e−\|J\|​τ​\(∏r∈JΦr​\(PEr​x\)\)​∫\[0,1\]\|J\|\(∂c\|J\|Kτ\)​\(u,e−τ​∑r∈Jtr​Φr​\(PEr​x\)\)​𝑑t\.H\_\{\\tau,J\}\(x,u\):=e^\{\-\|J\|\\tau\}\\left\(\\prod\_\{r\\in J\}\\Phi\_\{r\}\(P\_\{E\_\{r\}\}x\)\\right\)\\int\_\{\[0,1\]^\{\|J\|\}\}\(\\partial\_\{c\}^\{\|J\|\}K\_\{\\tau\}\)\\left\(u,e^\{\-\\tau\}\\sum\_\{r\\in J\}t\_\{r\}\\Phi\_\{r\}\(P\_\{E\_\{r\}\}x\)\\right\)dt\.\(771\)For eachJJthe support ofHτ,JH\_\{\\tau,J\}in thexxcoordinate is contained in the set

\{x∈EJ:PEr​x∈spt⁡\(Φr\)​∀r∈J\},\\\{x\\in E\_\{J\}:\\;P\_\{E\_\{r\}\}x\\in\\operatorname\{spt\}\(\\Phi\_\{r\}\)\\;\\forall r\\in J\\\},\(772\)which is compact \(since‖x‖≲∑r∈JJ⁡‖PEr​x‖\\\|x\\\|\\lesssim\_\{J\}\\sum\_\{r\\in J\}\\\|P\_\{E\_\{r\}\}x\\\|\)\. Since we know\|e−τ​∑r∈Jtr​Φr​\(PEr​x\)\|≤B\|e^\{\-\\tau\}\\sum\_\{r\\in J\}t\_\{r\}\\Phi\_\{r\}\(P\_\{E\_\{r\}\}x\)\|\\leq B\(and so the estimate \([767](https://arxiv.org/html/2610.10875#A1.E767)\) applies\), by the product and chain rule each derivative ofHτ,JH\_\{\\tau,J\}is a finite sum of products of values/derivatives ofΦr\\Phi\_\{r\}with those ofKτK\_\{\\tau\}with theuuderivatives only applying toKτK\_\{\\tau\}and therefore

\|∂xα∂ujHτ,J\(x,u\)\|≲α,j,J,T\(1\+\|u\|\)−λ−j\.\|\\partial\_\{x\}^\{\\alpha\}\\partial\_\{u\}^\{j\}H\_\{\\tau,J\}\(x,u\)\|\\lesssim\_\{\\alpha,j,J,T\}\(1\+\|u\|\)^\{\-\\lambda\-j\}\.\(773\)SinceHτ,JH\_\{\\tau,J\}is compactly\-supported inxx, we find thatAτ∈𝒞λA\_\{\\tau\}\\in\\mathcal\{C\}\_\{\\lambda\}uniformly overτ∈\[0,T\]\\tau\\in\[0,T\]\.

Next, sinceRτ​\(z,u\)−1=𝔼G∼γn​\[Aτ​\(e−τ​z\+σ​G,u\)\]R\_\{\\tau\}\(z,u\)\-1=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{n\}\}\[A\_\{\\tau\}\(e^\{\-\\tau\}z\+\\sigma G,u\)\]we apply Lemma[19](https://arxiv.org/html/2610.10875#Thmlemma19)\(iii\) to find thatrτ:=Rτ−1∈𝒞λr\_\{\\tau\}:=R\_\{\\tau\}\-1\\in\\mathcal\{C\}\_\{\\lambda\}uniformly overτ∈\[0,T\]\\tau\\in\[0,T\]\. Next, sinceKτ\(u,c\)≍T1K\_\{\\tau\}\(u,c\)\\asymp\_\{T\}1uniformly in\|c\|≤B\|c\|\\leq Bwe also haveRτ\(z,u\)≍T1R\_\{\\tau\}\(z,u\)\\asymp\_\{T\}1\. Hence, ifη∈Cc∞​\(ℝ\)\\eta\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}\)is s\.t\.η⁡\(0\)=0\\eta\(0\)=0and

η⁡\(t\):=1t\+1−1\\eta\(t\):=\\frac\{1\}\{t\+1\}\-1\\quad\(774\)for allttin a neighborhood of the image ofrτr\_\{\\tau\}for allτ∈\[0,T\]\\tau\\in\[0,T\]\(which is possible since1\+rτ=Rτ1\+r\_\{\\tau\}=R\_\{\\tau\}is bounded above and away from 0 uniformly forτ∈\[0,T\]\\tau\\in\[0,T\]\), then by Lemma[19](https://arxiv.org/html/2610.10875#Thmlemma19)\(ii\) we knowη∘rτ∈𝒞λ\\eta\\circ r\_\{\\tau\}\\in\\mathcal\{C\}\_\{\\lambda\}\. Furthermore, each coordinate of∇rτ\\nabla r\_\{\\tau\}is also in𝒞λ\\mathcal\{C\}\_\{\\lambda\}since taking derivatives only helps with theuudecay\. Finally, since

Sτ=∇log⁡Rτ=∇RτRτ=∇rτ\+∇rτ​\(1Rτ−1\)=∇rτ\+∇rτ​\(η∘rτ\),S\_\{\\tau\}=\\nabla\\log R\_\{\\tau\}=\\frac\{\\nabla R\_\{\\tau\}\}\{R\_\{\\tau\}\}=\\nabla r\_\{\\tau\}\+\\nabla r\_\{\\tau\}\\left\(\\frac\{1\}\{R\_\{\\tau\}\}\-1\\right\)=\\nabla r\_\{\\tau\}\+\\nabla r\_\{\\tau\}\(\\eta\\circ r\_\{\\tau\}\),\(775\)by the algebra property in Lemma[19](https://arxiv.org/html/2610.10875#Thmlemma19)\(i\) we find that each coordinate ofSτS\_\{\\tau\}is in𝒞λ\\mathcal\{C\}\_\{\\lambda\}, uniformly inτ∈\[0,T\]\\tau\\in\[0,T\]\. The claim follows by applying the Barron embedding statement of Lemma[19](https://arxiv.org/html/2610.10875#Thmlemma19)coordinatewise\. ∎

#### A\.7\.2Approximation lower bound via Hermite spectral tails

Throughout this subsection we will takeψ⁡\(u\)=σ​u\\psi\(u\)=\\sigma ufor someσ\>0\\sigma\>0\(i\.e\.ψ⁡\(U\)\\psi\(U\)is centered Gaussian\) and focus on the dependence onΨ\\Psi\. We will now lower bound the widths and Barron norms of two\-layer neural networkapproximationsto the scoressτψ,Ψ/bτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}/b\_\{\\tau\}^\{\\psi,\\Psi\}, as opposed to exact representations\. This approach is more robust, leveraging high Hermite rank ofΨ\\Psito directly lower bound the width of any sum of ridge functions approximating the score\. The technique borrows largely from the recent paper\[[13](https://arxiv.org/html/2610.10875#bib.bib22)\], and we adapt the general lower bound method to the reverse OU application\.

Let us first define a class of functions which contains two\-layer neural networks of widthmm\. Since the Hermite method controls width in a general sense, the neurons can be very general ridge functions which need not share an activation function\.

###### Definition 7\(Shallow function class\)\.

For eachn,m∈ℕn,m\\in\{\\mathbb\{N\}\}, we define the classℛm\(n\)\\mathcal\{R\}\_\{m\}^\{\(n\)\}of sums of at mostmmsquare\-integrable ridge functions onℝn\{\\mathbb\{R\}\}^\{n\}, i\.e\.

ℛm\(n\):=\{g:ℝn→ℝ:g\(x\)=∑j=1mfj\(⟨x,θj⟩\)withfj∈L2\(γ1\)andθj∈𝕊n−1\}⊆L2\(γn\)\.\\mathcal\{R\}\_\{m\}^\{\(n\)\}:=\\left\\\{g:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}:\\;g\(x\)=\\sum\_\{j=1\}^\{m\}f\_\{j\}\(\\left\\langle x,\\theta\_\{j\}\\right\\rangle\)\\text\{ with \}f\_\{j\}\\in L^\{2\}\(\\gamma\_\{1\}\)\\text\{ and \}\\theta\_\{j\}\\in\\mathbb\{S\}^\{n\-1\}\\right\\\}\\subseteq L^\{2\}\(\\gamma\_\{n\}\)\.\(776\)The functionsfj:ℝ→ℝf\_\{j\}:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}encode both the activations and the output weights of each neuron\.

We will make use of some notation from Gaussian/Hermite analysis; for references, see\[[47](https://arxiv.org/html/2610.10875#bib.bib23), Definition 2\.1, Example 3\.18, Theorem 4\.1\]of the textbook of Janson\. Fixn∈ℕn\\in\{\\mathbb\{N\}\}and letΨ:ℝn→ℝ\\Psi:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}be s\.t\.Ψ∈L2​\(γn\)\\Psi\\in L^\{2\}\(\\gamma\_\{n\}\)\. For eachk∈ℕ∪\{0\}k\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}, denote byTk​\(Ψ\)∈Symk⁡\(ℝn\)T\_\{k\}\(\\Psi\)\\in\\operatorname\{Sym\}^\{k\}\(\{\\mathbb\{R\}\}^\{n\}\)its degree\-kkHermite coefficient tensor, normalized s\.t\.

‖Tk​\(Ψ\)‖F=‖Πk​Ψ‖L2​\(γn\)\\\|T\_\{k\}\(\\Psi\)\\\|\_\{F\}=\\\|\\Pi\_\{k\}\\Psi\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}\(777\)forΠk\\Pi\_\{k\}the orthogonal projection inL2​\(γn\)L^\{2\}\(\\gamma\_\{n\}\)onto thekt​hk^\{th\}Hermite chaos\. In other words,Tk​\(Ψ\):=𝔼ξ∼γn​\[Ψ⁡\(ξ\)​𝐇𝐞k​\(ξ\)/k\!\]T\_\{k\}\(\\Psi\):=\\mathbb\{E\}\_\{\\xi\\sim\\gamma\_\{n\}\}\[\\Psi\(\\xi\)\\mathbf\{He\}\_\{k\}\(\\xi\)/\\sqrt\{k\!\}\]where𝐇𝐞k:ℝn→Symk⁡\(ℝn\)\\mathbf\{He\}\_\{k\}:\{\\mathbb\{R\}\}^\{n\}\\to\\operatorname\{Sym\}^\{k\}\(\{\\mathbb\{R\}\}^\{n\}\)are the standard monic probabilist’s Hermite polynomial tensors which satisfy𝔼⁡\[⟨A,𝐇𝐞k​\(ξ\)⟩F​⟨B,𝐇𝐞k​\(ξ\)⟩F\]=k\!​⟨A,B⟩F\\mathbb\{E\}\\left\[\\langle A,\\mathbf\{He\}\_\{k\}\(\\xi\)\\rangle\_\{F\}\\langle B,\\mathbf\{He\}\_\{k\}\(\\xi\)\\rangle\_\{F\}\\right\]=k\!\\langle A,B\\rangle\_\{F\}forA,B∈Symk⁡\(ℝn\)A,B\\in\\operatorname\{Sym\}^\{k\}\(\{\\mathbb\{R\}\}^\{n\}\)\. LetMk​\(Ψ\)∈ℝn⌊k/2⌋×n⌈k/2⌉M\_\{k\}\(\\Psi\)\\in\{\\mathbb\{R\}\}^\{n^\{\\lfloor k/2\\rfloor\}\\times n^\{\\lceil k/2\\rceil\}\}be the balanced matricization ofTk​\(Ψ\)T\_\{k\}\(\\Psi\), and denote its singular values by

σ1​\(Mk​\(Ψ\)\)≥…≥σn⌊k/2⌋​\(Mk​\(Ψ\)\)≥0\.\\sigma\_\{1\}\(M\_\{k\}\(\\Psi\)\)\\geq\\ldots\\geq\\sigma\_\{n^\{\\lfloor k/2\\rfloor\}\}\(M\_\{k\}\(\\Psi\)\)\\geq 0\.\(778\)SinceTk​\(Ψ\)T\_\{k\}\(\\Psi\)is symmetric, the singular values of its balanced matricization are independent of which partition of thekkindices into groups of sizes⌊k/2⌋\\lfloor k/2\\rfloorand⌈k/2⌉\\lceil k/2\\rceilwe use\. Also, note that‖Mk​\(Ψ\)‖F=‖Tk​\(Ψ\)‖F\\\|M\_\{k\}\(\\Psi\)\\\|\_\{F\}=\\\|T\_\{k\}\(\\Psi\)\\\|\_\{F\}, and so∑jσj​\(Mk​\(Ψ\)\)2=‖Πk​Ψ‖L2​\(γn\)2\\sum\_\{j\}\\sigma\_\{j\}\(M\_\{k\}\(\\Psi\)\)^\{2\}=\\\|\\Pi\_\{k\}\\Psi\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}^\{2\}\.

To state our general approximation result for the reverse OU transports to triangular shear measures, we make the following definition:

###### Definition 8\(Hermite width envelope\)\.

LetΨ:ℝn→ℝ\\Psi:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}be s\.t\.Ψ∈L2​\(γn\)\\Psi\\in L^\{2\}\(\\gamma\_\{n\}\)\. Define the functionaΨ:ℕ∪\{0\}×\[0,∞\)→\[0,∞\)a\_\{\\Psi\}:\{\\mathbb\{N\}\}\\cup\\\{0\\\}\\times\[0,\\infty\)\\to\[0,\\infty\)by the envelope

aΨ​\(m,τ\):=\(∑k=0∞e−2​\(k\+1\)​τ​∑j=m\+1n⌊k/2⌋σj​\(Mk​\(Ψ\)\)2\)1/2\.a\_\{\\Psi\}\(m,\\tau\):=\\left\(\\sum\_\{k=0\}^\{\\infty\}e^\{\-2\(k\+1\)\\tau\}\\sum\_\{j=\{m\+1\}\}^\{n^\{\\lfloor k/2\\rfloor\}\}\\sigma\_\{j\}\(M\_\{k\}\(\\Psi\)\)^\{2\}\\right\)^\{1/2\}\.\(779\)It is nonincreasing in its first argument, and we definemΨ:\(0,∞\)×\[0,∞\)→ℕ∪\{0,∞\}m\_\{\\Psi\}:\(0,\\infty\)\\times\[0,\\infty\)\\to\{\\mathbb\{N\}\}\\cup\\\{0,\\infty\\\}to be its generalized inverse, i\.e\.

mΨ​\(δ,τ\):=inf\{m∈ℕ∪\{0\}:aΨ​\(m,τ\)≤δ\}m\_\{\\Psi\}\(\\delta,\\tau\):=\\inf\\\{m\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}:\\;a\_\{\\Psi\}\(m,\\tau\)\\leq\\delta\\\}\(780\)with the convention thatinf∅=∞\\inf\\emptyset=\\infty\. We callmΨm\_\{\\Psi\}the Hermite width envelope ofΨ\\Psi\.

Using this definition, we get a particularly clean lower bound on the width of shallow approximations to the OU\-smoothed scores of the triangular shear measure generated by\(ψ,Ψ\)\(\\psi,\\Psi\)in the caseψ\\psiis linear:

###### Proposition 9\(Hermite tail shallow approximation lower bound\)\.

Letn∈ℕn\\in\{\\mathbb\{N\}\}, fixΨ:ℝn→ℝ\\Psi:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}bounded and Lipschitz, and letβ,L∈\(0,∞\)\\beta,L\\in\(0,\\infty\)be s\.t\.

‖Ψ‖∞≤β,Lip⁡\(Ψ\)≤L\.\\\|\\Psi\\\|\_\{\\infty\}\\leq\\beta,\\quad\\operatorname\{Lip\}\(\\Psi\)\\leq L\.\(781\)Fixψ⁡\(u\)=σ​u\\psi\(u\)=\\sigma ufor someσ∈\(0,∞\)\\sigma\\in\(0,\\infty\)\.

Fix anyτ∈\[0,∞\)\\tau\\in\[0,\\infty\)andε∈\(0,∞\)\\varepsilon\\in\(0,\\infty\), and suppose that there is somes^τ:ℝn\+1→ℝn\+1\\hat\{s\}\_\{\\tau\}:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}for which⟨en\+1,s^τ​\(⋅\)⟩∈ℛm\(n\+1\)\\left\\langle e\_\{n\+1\},\\hat\{s\}\_\{\\tau\}\(\\cdot\)\\right\\rangle\\in\\mathcal\{R\}\_\{m\}^\{\(n\+1\)\}for some widthm∈ℕm\\in\{\\mathbb\{N\}\}\(such as a two\-layer neural network of widthmm\) and

‖sτψ,Ψ−s^τ‖L2​\(ρτψ,Ψ,ℝn\+1\)≤ε\.\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\leq\\varepsilon\.\(782\)Then, there is someQβ,σ∈\(0,∞\)Q\_\{\\beta,\\sigma\}\\in\(0,\\infty\)depending only onβ,σ\\beta,\\sigmas\.t\.

m≥mΨ​\(Qβ,σ​\(ε\+L2​τ\),τ\)\.m\\geq m\_\{\\Psi\}\\left\(Q\_\{\\beta,\\sigma\}\(\\varepsilon\+L^\{2\}\\tau\),\\;\\tau\\right\)\.\(783\)Moreover, for any nonzeroϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)there is someKϕ,β,σ∈\(0,∞\)K\_\{\\phi,\\beta,\\sigma\}\\in\(0,\\infty\)s\.t\.

‖s^τ‖ℬϕ≥Kϕ,β,σ⋅ε​max⁡\{0,mΨ​\(Qβ,σ​\(2​ε\+L2​τ\),τ\)−1\}\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\geq K\_\{\\phi,\\beta,\\sigma\}\\cdot\\varepsilon\\sqrt\{\\max\\left\\\{0,\\;m\_\{\\Psi\}\(Q\_\{\\beta,\\sigma\}\(2\\varepsilon\+L^\{2\}\\tau\),\\;\\tau\)\-1\\right\\\}\}\(784\)These statements also hold withsτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}replaced bybτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\.

In words, ifs^τ\\hat\{s\}\_\{\\tau\}is a two\-layer neural network for which‖sτψ,Ψ−s^τ‖L2​\(ρτψ,Ψ,ℝn\+1\)≤ε\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\leq\\varepsilon, then its width must be at leastmΨ​\(Qβ,σ​\(ε\+L2​τ\),τ\)m\_\{\\Psi\}\(Q\_\{\\beta,\\sigma\}\(\\varepsilon\+L^\{2\}\\tau\),\\tau\)\. The rest of this subsection is devoted to the proof of this result\.

We start with a general approximation lower bound in terms of the spectral tails of Hermite tensors, which is a direct extension of\[[13](https://arxiv.org/html/2610.10875#bib.bib22), Lemma 4\.14\]\. Informally, functions of high Hermite rank cannot be approximated using sums of ridge functions without sufficient width\. This is quantified byaΨa\_\{\\Psi\}in a way which is robust to OU smoothing, yielding:

###### Lemma 22\(Shallow approximation lower bound inL2​\(γn\)L^\{2\}\(\\gamma\_\{n\}\)via Hermite tail\)\.

For everyΨ∈L2​\(γn\)\\Psi\\in L^\{2\}\(\\gamma\_\{n\}\)and everym∈ℕm\\in\{\\mathbb\{N\}\}andτ∈\[0,∞\)\\tau\\in\[0,\\infty\), we have

infg∈ℛm\(n\)‖e−τ​Pτ​Ψ−g‖L2​\(γn\)≥aΨ​\(m,τ\)\.\\inf\_\{g\\in\\mathcal\{R\}\_\{m\}^\{\(n\)\}\}\\\|e^\{\-\\tau\}P\_\{\\tau\}\\Psi\-g\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}\\geq a\_\{\\Psi\}\(m,\\tau\)\.\(785\)

###### Proof\.

Fix someg⁡\(⋅\)=∑j=1mfj​\(⟨⋅,θj⟩\)∈ℛm\(n\)g\(\\cdot\)=\\sum\_\{j=1\}^\{m\}f\_\{j\}\(\\left\\langle\\cdot,\\theta\_\{j\}\\right\\rangle\)\\in\\mathcal\{R\}\_\{m\}^\{\(n\)\}, where eachfj∈L2​\(γ1\)f\_\{j\}\\in L^\{2\}\(\\gamma\_\{1\}\)\. Withhk:=Hek/k\!h\_\{k\}:=\\operatorname\{He\}\_\{k\}/\\sqrt\{k\!\}the orthonormalized Hermite basis ofL2​\(γ1\)L^\{2\}\(\\gamma\_\{1\}\), writef^j​\(k\):=⟨fj,hk⟩L2​\(γ1\)\\hat\{f\}\_\{j\}\(k\):=\\left\\langle f\_\{j\},h\_\{k\}\\right\\rangle\_\{L^\{2\}\(\\gamma\_\{1\}\)\}so that

fj=∑k=0∞f^j​\(k\)​hk​inL2​\(γ1\)\(∀j∈\{1,…,m\}\)\.f\_\{j\}=\\sum\_\{k=0\}^\{\\infty\}\\hat\{f\}\_\{j\}\(k\)h\_\{k\}\\text\{ in $L^\{2\}\(\\gamma\_\{1\}\)$\}\\quad\(\\forall j\\in\\\{1,\\ldots,m\\\}\)\.\(786\)Sinceθj∈𝕊n−1\\theta\_\{j\}\\in\\mathbb\{S\}^\{n\-1\}, the defining property of the multivariate Hermite tensors gives

hk​\(⟨x,θj⟩\)=⟨𝐇𝐞k​\(x\)k\!,θj⊗k⟩F\.h\_\{k\}\(\\left\\langle x,\\theta\_\{j\}\\right\\rangle\)=\\left\\langle\\frac\{\\mathbf\{He\}\_\{k\}\(x\)\}\{\\sqrt\{k\!\}\},\\theta\_\{j\}^\{\\otimes k\}\\right\\rangle\_\{F\}\.\(787\)Hence,Tk​\(fj​\(⟨⋅,θj⟩\)\)=f^j​\(k\)​θj⊗kT\_\{k\}\(f\_\{j\}\(\\left\\langle\\cdot,\\theta\_\{j\}\\right\\rangle\)\)=\\hat\{f\}\_\{j\}\(k\)\\theta\_\{j\}^\{\\otimes k\}and therefore

Tk​\(g\)=∑j=1mf^j​\(k\)​θj⊗k\.T\_\{k\}\(g\)=\\sum\_\{j=1\}^\{m\}\\hat\{f\}\_\{j\}\(k\)\\theta\_\{j\}^\{\\otimes k\}\.\(788\)In particular,rank⁡\(Mk​\(g\)\)≤m\\operatorname\{rank\}\(M\_\{k\}\(g\)\)\\leq mfor everyk∈ℕ∪\{0\}k\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\. Since the OU semigroupPτP\_\{\\tau\}acts on thekt​hk^\{th\}Wiener chaos as multiplication bye−k​τe^\{\-k\\tau\}, we find thatTk​\(e−τ​Pτ​Ψ\)=e−\(k\+1\)​τ​Tk​\(Ψ\)T\_\{k\}\(e^\{\-\\tau\}P\_\{\\tau\}\\Psi\)=e^\{\-\(k\+1\)\\tau\}T\_\{k\}\(\\Psi\)and so

‖e−τ​Pτ​Ψ−g‖L2​\(γn\)2\\displaystyle\\\|e^\{\-\\tau\}P\_\{\\tau\}\\Psi\-g\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}^\{2\}=∑k=0∞‖e−\(k\+1\)​τ​Tk​\(Ψ\)−Tk​\(g\)‖F2\\displaystyle=\\sum\_\{k=0\}^\{\\infty\}\\\|e^\{\-\(k\+1\)\\tau\}T\_\{k\}\(\\Psi\)\-T\_\{k\}\(g\)\\\|\_\{F\}^\{2\}\(789\)=∑k=0∞‖e−\(k\+1\)​τ​Mk​\(Ψ\)−Mk​\(g\)‖F2\\displaystyle=\\sum\_\{k=0\}^\{\\infty\}\\\|e^\{\-\(k\+1\)\\tau\}M\_\{k\}\(\\Psi\)\-M\_\{k\}\(g\)\\\|^\{2\}\_\{F\}\(790\)≥∑k=0∞e−2​\(k\+1\)​τ​∑j=m\+1n⌊k/2⌋σj​\(Mk​\(Ψ\)\)2\\displaystyle\\geq\\sum\_\{k=0\}^\{\\infty\}e^\{\-2\(k\+1\)\\tau\}\\sum\_\{j=m\+1\}^\{n^\{\\lfloor k/2\\rfloor\}\}\\sigma\_\{j\}\(M\_\{k\}\(\\Psi\)\)^\{2\}\(791\)=aΨ​\(m,τ\)2,\\displaystyle=a\_\{\\Psi\}\(m,\\tau\)^\{2\},\(792\)where the inequality is due to Eckart\-Young and the factrank⁡\(Mk​\(g\)\)≤m\\operatorname\{rank\}\(M\_\{k\}\(g\)\)\\leq m\. ∎

The above result operates inL2​\(γn\)L^\{2\}\(\\gamma\_\{n\}\)in order to deploy the methods of Hermite analysis\. However, our object of interest is the approximation of OU scores inL2​\(ρτψ,Ψ,ℝn\+1\)L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\(whereρτψ,Ψ=Pτ∗​πψ,Ψ\\rho\_\{\\tau\}^\{\\psi,\\Psi\}=P\_\{\\tau\}^\{\*\}\\pi^\{\\psi,\\Psi\}are the OU marginals\)\. In order to apply Lemma[22](https://arxiv.org/html/2610.10875#Thmlemma22), we define the following operator which averages over the non\-Gaussian coordinate:

###### Definition 9\(Truncated Gaussian averaging operator\)\.

FixR\>0R\>0\. LetF:ℝn\+1→ℝn\+1F:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}be a measurable vector field\. Using coordinates\(z,u\)∈ℝn\+1\(z,u\)\\in\{\\mathbb\{R\}\}^\{n\+1\}withz∈ℝnz\\in\{\\mathbb\{R\}\}^\{n\}andu∈ℝu\\in\{\\mathbb\{R\}\}, assume thatFn\+1​\(z,⋅\)F\_\{n\+1\}\(z,\\cdot\)is integrable w\.r\.t\.γ1\\gamma\_\{1\}on\[−R,R\]\[\-R,R\]for a\.e\.zz, and define𝒜R​F:ℝn→ℝ\\mathcal\{A\}\_\{R\}F:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}by

\(𝒜R​F\)​\(z\):=𝔼U∼γ1​\[Fn\+1​\(z,U\)∣\|U\|≤R\]\.\(\\mathcal\{A\}\_\{R\}F\)\(z\):=\\mathbb\{E\}\_\{U\\sim\\gamma\_\{1\}\}\[F\_\{n\+1\}\(z,U\)\\mid\|U\|\\leq R\]\.\(793\)We call𝒜R\\mathcal\{A\}\_\{R\}the averaging operator with truncationRR\.

For our triangular shear measures of interest, all other coordinates are Gaussian\. This means that using some estimates on the averaging operator, we can lower boundL2​\(ρτψ,Ψ,ℝn\+1\)L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)score approximation in terms ofL2​\(γn\)L^\{2\}\(\\gamma\_\{n\}\)function approximation:

###### Lemma 23\.

Fixσ,β,L,R∈\(0,∞\)\\sigma,\\beta,L,R\\in\(0,\\infty\), letψ⁡\(u\)=σ​u\\psi\(u\)=\\sigma u, and letΨ:ℝn→ℝ\\Psi:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}satisfy‖Ψ‖∞≤β\\\|\\Psi\\\|\_\{\\infty\}\\leq\\betaandLip⁡\(Ψ\)≤L\\operatorname\{Lip\}\(\\Psi\)\\leq L\. Then, for everyτ∈\[0,∞\)\\tau\\in\[0,\\infty\)and every measurableF:ℝn\+1→ℝn\+1F:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}for which𝒜R​F\\mathcal\{A\}\_\{R\}Fis defined and inL2​\(γn\)L^\{2\}\(\\gamma\_\{n\}\), it holds that

‖sτψ,Ψ−F‖L2​\(ρτψ,Ψ,ℝn\+1\)≥αR,β,σ⋅\(‖e−τ​Pτ​Ψ−cσ,τ​𝒜R​F‖L2​\(γn\)−CR,β,σ​L2​τ\)\\displaystyle\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-F\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\geq\\alpha\_\{R,\\beta,\\sigma\}\\cdot\\left\(\\\|e^\{\-\\tau\}P\_\{\\tau\}\\Psi\-c\_\{\\sigma,\\tau\}\\mathcal\{A\}\_\{R\}F\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}\-C\_\{R,\\beta,\\sigma\}L^\{2\}\\tau\\right\)\(794\)where we define the explicit constants

αR,β,σ\\displaystyle\\alpha\_\{R,\\beta,\\sigma\}:=γ1​\(\[−R,R\]\)max\{1,σ\}−5/2exp\(−R​β\+β2/2\+R2​max⁡\{0,1−σ2\}/22​min⁡\{1,σ\}2\),\\displaystyle:=\\sqrt\{\\gamma\_\{1\}\(\[\-R,R\]\)\}\\max\\\{1,\\sigma\\\}^\{\-5/2\}\\exp\\left\(\-\\frac\{R\\beta\+\\beta^\{2\}/2\+R^\{2\}\\max\\\{0,1\-\\sigma^\{2\}\\\}/2\}\{2\\min\\\{1,\\sigma\\\}^\{2\}\}\\right\),\(795\)CR,β,σ\\displaystyle C\_\{R,\\beta,\\sigma\}:=2​\(R\+β\)min⁡\{1,σ\}2​exp⁡\(\(R\+β\)22​min⁡\{1,σ\}2\),\\displaystyle:=\\frac\{2\(R\+\\beta\)\}\{\\min\\\{1,\\sigma\\\}^\{2\}\}\\exp\\left\(\\frac\{\(R\+\\beta\)^\{2\}\}\{2\\min\\\{1,\\sigma\\\}^\{2\}\}\\right\),\(796\)cσ,τ\\displaystyle c\_\{\\sigma,\\tau\}:=e−2​τ​σ2\+1−e−2​τ\.\\displaystyle:=e^\{\-2\\tau\}\\sigma^\{2\}\+1\-e^\{\-2\\tau\}\.\(797\)The same is true withsτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}replaced bybτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\.

###### Proof\.

The claim follows directly from the following two estimates on𝒜R\\mathcal\{A\}\_\{R\}:

‖e−τ​Pτ​Ψ−cσ,τ​𝒜R​sτψ,Ψ‖∞≤CR,β,σ​L2​τ\.\\\|e^\{\-\\tau\}P\_\{\\tau\}\\Psi\-c\_\{\\sigma,\\tau\}\\mathcal\{A\}\_\{R\}s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{\\infty\}\\leq C\_\{R,\\beta,\\sigma\}L^\{2\}\\tau\.\(798\)‖sτψ,Ψ−F‖L2​\(ρτψ,Ψ,ℝn\+1\)≥αR,β,σ​cσ,τ​‖𝒜R​sτψ,Ψ−𝒜R​F‖L2​\(γn\),\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-F\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\geq\\alpha\_\{R,\\beta,\\sigma\}c\_\{\\sigma,\\tau\}\\\|\\mathcal\{A\}\_\{R\}s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\mathcal\{A\}\_\{R\}F\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\},\(799\)The same will be true forbτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}, after which the result will follow from the reverse triangle inequality\.

To show these claims, let us denoteq−:=min⁡\{1,σ\}q\_\{\-\}:=\\min\\\{1,\\sigma\\\},q\+:=max⁡\{1,σ\}q\_\{\+\}:=\\max\\\{1,\\sigma\\\},c=cσ,τc=c\_\{\\sigma,\\tau\}and observe thatq−2≤c≤q\+2q\_\{\-\}^\{2\}\\leq c\\leq q\_\{\+\}^\{2\}\. For eachz∈ℝnz\\in\{\\mathbb\{R\}\}^\{n\}we define

ηz:=e−τ​Ψ​\(e−τ​z\+1−e−2​τ​X\),X∼γn\.\\eta\_\{z\}:=e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sqrt\{1\-e^\{\-2\\tau\}\}X\),\\quad X\\sim\\gamma\_\{n\}\.\(800\)Note that\|ηz\|≤β\|\\eta\_\{z\}\|\\leq\\betaand𝔼⁡\[ηz\]=e−τ​Pτ​Ψ​\(z\)\\mathbb\{E\}\[\\eta\_\{z\}\]=e^\{\-\\tau\}P\_\{\\tau\}\\Psi\(z\)\. If we denote

η~z​\(u\):=𝔼⁡\[ηz​exp⁡\(−\(u−ηz\)22​c\)\]𝔼⁡\[exp⁡\(−\(u−ηz\)22​c\)\]\\widetilde\{\\eta\}\_\{z\}\(u\):=\\frac\{\\mathbb\{E\}\\left\[\\eta\_\{z\}\\exp\\left\(\-\\frac\{\(u\-\\eta\_\{z\}\)^\{2\}\}\{2c\}\\right\)\\right\]\}\{\\mathbb\{E\}\\left\[\\exp\\left\(\-\\frac\{\(u\-\\eta\_\{z\}\)^\{2\}\}\{2c\}\\right\)\\right\]\}\(801\)then we claim that

\(sτψ,Ψ​\(z,u\)\)n\+1=η~z​\(u\)−uc\.\(s\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)\)\_\{n\+1\}=\\frac\{\\widetilde\{\\eta\}\_\{z\}\(u\)\-u\}\{c\}\.\(802\)Indeed, as was the case in the proof of Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18), by Gaussian regression we know that conditionally on the firstnncoordinates being equal tozzthe last coordinate underρτψ,Ψ\\rho\_\{\\tau\}^\{\\psi,\\Psi\}has the same law as

c​W\+ηz,W∼γ1​independently ofηz\.\\sqrt\{c\}W\+\\eta\_\{z\},\\quad W\\sim\\gamma\_\{1\}\\text\{ independently of $\\eta\_\{z\}$\}\.\(803\)The conditional density of the last coordinate is thereforepz​\(u\)=12​π​c​𝔼​\[exp⁡\(−\(u−ηz\)22​c\)\]p\_\{z\}\(u\)=\\frac\{1\}\{\\sqrt\{2\\pi c\}\}\\mathbb\{E\}\\left\[\\exp\\left\(\-\\frac\{\(u\-\\eta\_\{z\}\)^\{2\}\}\{2c\}\\right\)\\right\], and so

\(sτψ,Ψ​\(z,u\)\)n\+1=∂ulog⁡pz​\(u\)=𝔼⁡\[−u−ηzc​exp⁡\(−\(u−ηz\)22​c\)\]𝔼⁡\[exp⁡\(−\(u−ηz\)22​c\)\]=η~z​\(u\)−uc\.\\displaystyle\(s\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)\)\_\{n\+1\}=\\partial\_\{u\}\\log p\_\{z\}\(u\)=\\frac\{\\mathbb\{E\}\\left\[\-\\frac\{u\-\\eta\_\{z\}\}\{c\}\\exp\\left\(\-\\frac\{\(u\-\\eta\_\{z\}\)^\{2\}\}\{2c\}\\right\)\\right\]\}\{\\mathbb\{E\}\\left\[\\exp\\left\(\-\\frac\{\(u\-\\eta\_\{z\}\)^\{2\}\}\{2c\}\\right\)\\right\]\}=\\frac\{\\widetilde\{\\eta\}\_\{z\}\(u\)\-u\}\{c\}\.\(804\)This also implies that\(bτψ,Ψ​\(z,u\)\)n\+1=η~z​\(u\)−uc\+u\(b\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)\)\_\{n\+1\}=\\frac\{\\widetilde\{\\eta\}\_\{z\}\(u\)\-u\}\{c\}\+u, and since the standard Gaussian conditioned on\[−R,R\]\[\-R,R\]is centered we find

\(c​𝒜R​sτψ,Ψ\)​\(z\)=\(c​𝒜R​bτψ,Ψ\)​\(z\)=𝔼U∼γ1​\[η~z​\(U\)∣\|U\|≤R\],\(c\\mathcal\{A\}\_\{R\}s\_\{\\tau\}^\{\\psi,\\Psi\}\)\(z\)=\(c\\mathcal\{A\}\_\{R\}b\_\{\\tau\}^\{\\psi,\\Psi\}\)\(z\)=\\mathbb\{E\}\_\{U\\sim\\gamma\_\{1\}\}\[\\widetilde\{\\eta\}\_\{z\}\(U\)\\mid\|U\|\\leq R\],\(805\)and in particular𝒜R​sτψ,Ψ=𝒜R​bτψ,Ψ\\mathcal\{A\}\_\{R\}s\_\{\\tau\}^\{\\psi,\\Psi\}=\\mathcal\{A\}\_\{R\}b\_\{\\tau\}^\{\\psi,\\Psi\}\. We now compareη~z​\(u\)\\widetilde\{\\eta\}\_\{z\}\(u\)and𝔼⁡\[ηz\]\\mathbb\{E\}\[\\eta\_\{z\}\]\. Foru,y∈ℝu,y\\in\{\\mathbb\{R\}\}we let

wu​\(y\):=exp⁡\(−\(u−y\)22​c\)w\_\{u\}\(y\):=\\exp\\left\(\-\\frac\{\(u\-y\)^\{2\}\}\{2c\}\\right\)\(806\)to see thatη~z​\(u\)=𝔼⁡\[ηz​wu​\(ηz\)\]/𝔼⁡\[wu​\(ηz\)\]\\widetilde\{\\eta\}\_\{z\}\(u\)=\\mathbb\{E\}\[\\eta\_\{z\}w\_\{u\}\(\\eta\_\{z\}\)\]/\\mathbb\{E\}\[w\_\{u\}\(\\eta\_\{z\}\)\]and therefore

η~z​\(u\)−𝔼⁡\[ηz\]=Cov⁡\(ηz,wu​\(ηz\)\)𝔼⁡\[wu​\(ηz\)\]\.\\widetilde\{\\eta\}\_\{z\}\(u\)\-\\mathbb\{E\}\[\\eta\_\{z\}\]=\\frac\{\\operatorname\{Cov\}\(\\eta\_\{z\},w\_\{u\}\(\\eta\_\{z\}\)\)\}\{\\mathbb\{E\}\[w\_\{u\}\(\\eta\_\{z\}\)\]\}\.\(807\)Now, since\|ηz\|≤β\|\\eta\_\{z\}\|\\leq\\betaand\|u\|≤R\|u\|\\leq Rwe know that𝔼⁡\[wu​\(ηz\)\]≥exp⁡\(−\(R\+β\)22​c\)\\mathbb\{E\}\[w\_\{u\}\(\\eta\_\{z\}\)\]\\geq\\exp\\left\(\-\\frac\{\(R\+\\beta\)^\{2\}\}\{2c\}\\right\)\. Furthermore, fory∈\[−β,β\]y\\in\[\-\\beta,\\beta\]we know\|wu′​\(y\)\|=\|u−y\|c​wu​\(y\)≤R\+βc\|w^\{\\prime\}\_\{u\}\(y\)\|=\\frac\{\|u\-y\|\}\{c\}w\_\{u\}\(y\)\\leq\\frac\{R\+\\beta\}\{c\}\. Hence, the covariance inequality\|Cov⁡\(Y,f⁡\(Y\)\)\|≤Lip⁡\(f\)​Var⁡\(Y\)\|\\operatorname\{Cov\}\(Y,f\(Y\)\)\|\\leq\\operatorname\{Lip\}\(f\)\\operatorname\{Var\}\(Y\)implies that

supu∈\[−R,R\]\|η~z​\(u\)−𝔼⁡\[ηz\]\|≤R\+βc​exp⁡\(\(R\+β\)22​c\)​Var⁡\(ηz\)\.\\sup\_\{u\\in\[\-R,R\]\}\\left\|\\widetilde\{\\eta\}\_\{z\}\(u\)\-\\mathbb\{E\}\[\\eta\_\{z\}\]\\right\|\\leq\\frac\{R\+\\beta\}\{c\}\\exp\\left\(\\frac\{\(R\+\\beta\)^\{2\}\}\{2c\}\\right\)\\operatorname\{Var\}\(\\eta\_\{z\}\)\.\(808\)However, the functionx↦e−τ​Ψ​\(e−τ​z\+1−e−2​τ​x\)x\\mapsto e^\{\-\\tau\}\\Psi\(e^\{\-\\tau\}z\+\\sqrt\{1\-e^\{\-2\\tau\}\}x\)isL​e−τ​1−e−2​τLe^\{\-\\tau\}\\sqrt\{1\-e^\{\-2\\tau\}\}\-Lipschitz and so the Gaussian Poincaré inequality yields

Var⁡\(ηz\)≤L2​e−2​τ​\(1−e−2​τ\)≤2​L2​τ\.\\operatorname\{Var\}\(\\eta\_\{z\}\)\\leq L^\{2\}e^\{\-2\\tau\}\(1\-e^\{\-2\\tau\}\)\\leq 2L^\{2\}\\tau\.\(809\)Combining this with \([808](https://arxiv.org/html/2610.10875#A1.E808)\) andc≥q−2c\\geq q\_\{\-\}^\{2\}and using𝔼⁡\[ηz\]=e−τ​Pτ​Ψ​\(z\)\\mathbb\{E\}\[\\eta\_\{z\}\]=e^\{\-\\tau\}P\_\{\\tau\}\\Psi\(z\), we find

supz∈ℝnsupu∈\[−R,R\]\|η~z​\(u\)−e−τ​Pτ​Ψ​\(z\)\|≤CR,β,σ​L2​τ\.\\sup\_\{z\\in\{\\mathbb\{R\}\}^\{n\}\}\\sup\_\{u\\in\[\-R,R\]\}\\left\|\\widetilde\{\\eta\}\_\{z\}\(u\)\-e^\{\-\\tau\}P\_\{\\tau\}\\Psi\(z\)\\right\|\\leq C\_\{R,\\beta,\\sigma\}L^\{2\}\\tau\.\(810\)Averaging overuuand using \([805](https://arxiv.org/html/2610.10875#A1.E805)\) gives the estimate \([798](https://arxiv.org/html/2610.10875#A1.E798)\) for bothsτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}andbτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\.

To prove the second estimate \([799](https://arxiv.org/html/2610.10875#A1.E799)\), we note that the density ofρτψ,Ψ\\rho\_\{\\tau\}^\{\\psi,\\Psi\}is given by

ρτψ,Ψ​\(z,u\)=γn​\(z\)​pz​\(u\)\\rho\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)=\\gamma\_\{n\}\(z\)p\_\{z\}\(u\)\(811\)since the firstnncoordinates have marginal lawγn\\gamma\_\{n\}and the last coordinate has densitypzp\_\{z\}conditioned on the event where the firstnncoordinates equalzz\. Therefore,

‖sτψ,Ψ−F‖L2​\(ρτψ,Ψ,ℝn\+1\)2\\displaystyle\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-F\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}^\{2\}≥∫ℝn∫−∞∞\|\(sτψ,Ψ​\(z,u\)\)n\+1−Fn\+1​\(z,u\)\|2​pz​\(u\)​𝑑u​d​γn​\(z\)\\displaystyle\\geq\\int\_\{\{\\mathbb\{R\}\}^\{n\}\}\\int\_\{\-\\infty\}^\{\\infty\}\\left\|\(s\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)\)\_\{n\+1\}\-F\_\{n\+1\}\(z,u\)\\right\|^\{2\}p\_\{z\}\(u\)dud\\gamma\_\{n\}\(z\)\(812\)≥κR,β,σ​∫ℝn∫−RR\|\(sτψ,Ψ​\(z,u\)\)n\+1−Fn\+1​\(z,u\)\|2​d​γ1​\(u\)​d​γn​\(z\),\\displaystyle\\geq\\kappa\_\{R,\\beta,\\sigma\}\\int\_\{\{\\mathbb\{R\}\}^\{n\}\}\\int\_\{\-R\}^\{R\}\\left\|\(s\_\{\\tau\}^\{\\psi,\\Psi\}\(z,u\)\)\_\{n\+1\}\-F\_\{n\+1\}\(z,u\)\\right\|^\{2\}d\\gamma\_\{1\}\(u\)d\\gamma\_\{n\}\(z\),\(813\)where we use that

pz​\(u\)γ1​\(u\)=1c​𝔼​\[exp⁡\(c−12​c​u2\+u​ηzc−ηz22​c\)\]≥κR,β,σ\\frac\{p\_\{z\}\(u\)\}\{\\gamma\_\{1\}\(u\)\}=\\frac\{1\}\{\\sqrt\{c\}\}\\mathbb\{E\}\\left\[\\exp\\left\(\\frac\{c\-1\}\{2c\}u^\{2\}\+\\frac\{u\\eta\_\{z\}\}\{c\}\-\\frac\{\\eta\_\{z\}^\{2\}\}\{2c\}\\right\)\\right\]\\geq\\kappa\_\{R,\\beta,\\sigma\}\(814\)for all\|u\|≤R\|u\|\\leq Rand we denote

κR,β,σ:=1q\+​exp⁡\(−R​β\+β2/2\+R2​max⁡\{0,1−σ2\}/2q−2\)\.\\kappa\_\{R,\\beta,\\sigma\}:=\\frac\{1\}\{q\_\{\+\}\}\\exp\\left\(\-\\frac\{R\\beta\+\\beta^\{2\}/2\+R^\{2\}\\max\\\{0,1\-\\sigma^\{2\}\\\}/2\}\{q\_\{\-\}^\{2\}\}\\right\)\.\(815\)Hence, by applying Jensen’s inequality under the conditional lawγ1\(⋅∣\[−R,R\]\)\\gamma\_\{1\}\(\\cdot\\mid\[\-R,R\]\)we find

‖sτψ,Ψ−F‖L2​\(ρτψ,Ψ,ℝn\+1\)2\\displaystyle\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-F\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}^\{2\}≥κR,β,σ​γ1​\(\[−R,R\]\)​∫ℝn\|\(𝒜R​sτψ,Ψ\)​\(z\)−\(𝒜R​F\)​\(z\)\|2​d​γn​\(z\)\\displaystyle\\geq\\kappa\_\{R,\\beta,\\sigma\}\\gamma\_\{1\}\(\[\-R,R\]\)\\int\_\{\{\\mathbb\{R\}\}^\{n\}\}\\left\|\(\\mathcal\{A\}\_\{R\}s\_\{\\tau\}^\{\\psi,\\Psi\}\)\(z\)\-\(\\mathcal\{A\}\_\{R\}F\)\(z\)\\right\|^\{2\}d\\gamma\_\{n\}\(z\)\(816\)=κR,β,σ​γ1​\(\[−R,R\]\)​‖𝒜R​sτψ,Ψ−𝒜R​F‖L2​\(γn\)2\.\\displaystyle=\\kappa\_\{R,\\beta,\\sigma\}\\gamma\_\{1\}\(\[\-R,R\]\)\\\|\\mathcal\{A\}\_\{R\}s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\mathcal\{A\}\_\{R\}F\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}^\{2\}\.\(817\)UsingκR,β,σ​γ1​\(\[−R,R\]\)/c2≥αR,β,σ2\\kappa\_\{R,\\beta,\\sigma\}\\gamma\_\{1\}\(\[\-R,R\]\)/c^\{2\}\\geq\\alpha\_\{R,\\beta,\\sigma\}^\{2\}, we get \([799](https://arxiv.org/html/2610.10875#A1.E799)\)\. The same argument applies tobτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\. ∎

Hence, for any candidate vector fieldF:ℝn\+1→ℝn\+1F:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}which attempts to approximatesτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}orbτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}inL2​\(ρτψ,Ψ\)L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\}\), if𝒜R​F∈ℛm\(n\)\\mathcal\{A\}\_\{R\}F\\in\\mathcal\{R\}\_\{m\}^\{\(n\)\}then combining Lemmas[22](https://arxiv.org/html/2610.10875#Thmlemma22)and[23](https://arxiv.org/html/2610.10875#Thmlemma23)yields an approximation lower bound\. We next show that𝒜R\\mathcal\{A\}\_\{R\}preserves the ridge width constraint:

###### Lemma 24\.

Fixm∈ℕm\\in\{\\mathbb\{N\}\}and letF:ℝn\+1→ℝn\+1F:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}be measurable s\.t\. its last coordinate satisfiesFn\+1∈ℛm\(n\+1\)F\_\{n\+1\}\\in\\mathcal\{R\}\_\{m\}^\{\(n\+1\)\}\. Then,𝒜R​F∈ℛm\(n\)\\mathcal\{A\}\_\{R\}F\\in\\mathcal\{R\}\_\{m\}^\{\(n\)\}for allR\>0R\>0\.

###### Proof\.

Let us express

Fn\+1​\(z,u\)=∑j=1mfj​\(⟨z,vj⟩\+bj​u\)F\_\{n\+1\}\(z,u\)=\\sum\_\{j=1\}^\{m\}f\_\{j\}\(\\left\\langle z,v\_\{j\}\\right\\rangle\+b\_\{j\}u\)\(818\)forfj∈L2​\(γ1\)f\_\{j\}\\in L^\{2\}\(\\gamma\_\{1\}\)and‖vj‖2\+\|bj\|2=1\\\|v\_\{j\}\\\|^\{2\}\+\|b\_\{j\}\|^\{2\}=1\. Chooseθj∈𝕊n−1\\theta\_\{j\}\\in\\mathbb\{S\}^\{n\-1\}s\.t\.vj=‖vj‖​θjv\_\{j\}=\\\|v\_\{j\}\\\|\\theta\_\{j\}\(where if‖vj‖=0\\\|v\_\{j\}\\\|=0then we choose arbitraryθj\\theta\_\{j\}\) and define

f~j​\(t\):=𝔼U∼γ1​\[fj​\(‖vj‖​t\+bj​U\)∣\|U\|≤R\]\.\\widetilde\{f\}\_\{j\}\(t\):=\\mathbb\{E\}\_\{U\\sim\\gamma\_\{1\}\}\[f\_\{j\}\(\\\|v\_\{j\}\\\|t\+b\_\{j\}U\)\\mid\|U\|\\leq R\]\.\(819\)Naturally, this means that

\(𝒜R​F\)​\(z\)=∑j=1mf~j​\(⟨z,θj⟩\)\.\(\\mathcal\{A\}\_\{R\}F\)\(z\)=\\sum\_\{j=1\}^\{m\}\\widetilde\{f\}\_\{j\}\(\\left\\langle z,\\theta\_\{j\}\\right\\rangle\)\.\(820\)It remains to check thatf~j∈L2​\(γ1\)\\widetilde\{f\}\_\{j\}\\in L^\{2\}\(\\gamma\_\{1\}\)\. WithY,U∼γ1Y,U\\sim\\gamma\_\{1\}i\.i\.d\. we have

‖f~j‖L2​\(γ1\)2\\displaystyle\\\|\\widetilde\{f\}\_\{j\}\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}^\{2\}≤𝔼⁡\[fj​\(‖vj‖​Y\+bj​U\)2∣\|U\|≤R\]\\displaystyle\\leq\\mathbb\{E\}\[f\_\{j\}\(\\\|v\_\{j\}\\\|Y\+b\_\{j\}U\)^\{2\}\\mid\|U\|\\leq R\]\(821\)≤1γ1​\(\[−R,R\]\)​𝔼​\[fj​\(‖vj‖​Y\+bj​U\)2\]\\displaystyle\\leq\\frac\{1\}\{\\gamma\_\{1\}\(\[\-R,R\]\)\}\\mathbb\{E\}\[f\_\{j\}\(\\\|v\_\{j\}\\\|Y\+b\_\{j\}U\)^\{2\}\]\(822\)=1γ1​\(\[−R,R\]\)​‖fj‖L2​\(γ1\)2,\\displaystyle=\\frac\{1\}\{\\gamma\_\{1\}\(\[\-R,R\]\)\}\\\|f\_\{j\}\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}^\{2\},\(823\)where the first line is Jensen and the third is becauseLaw⁡\(‖vj‖​Y\+bj​U\)=γ1\\operatorname\{Law\}\(\\\|v\_\{j\}\\\|Y\+b\_\{j\}U\)=\\gamma\_\{1\}\. ∎

We may combine these statements to yield a proof of Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9):

###### Proof of Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9)\.

Lettingq−,q\+q\_\{\-\},q\_\{\+\}be as before, we chooseR:=q−2β\+q−R:=\\frac\{q\_\{\-\}^\{2\}\}\{\\beta\+q\_\{\-\}\}, and soR≤q−≤1R\\leq q\_\{\-\}\\leq 1,β​Rq−2≤1\\frac\{\\beta R\}\{q\_\{\-\}^\{2\}\}\\leq 1, andR2q−2≤1\\frac\{R^\{2\}\}\{q\_\{\-\}^\{2\}\}\\leq 1\. We first show that

Qβ,σ:=9​\(1\+β\)​max⁡\{1,σ\}5/2min⁡\{1,σ\}2​exp⁡\(β22​min⁡\{1,σ\}2\)≥max⁡\{αR,β,σ−1,CR,β,σ\}\.Q\_\{\\beta,\\sigma\}:=9\(1\+\\beta\)\\frac\{\\max\\\{1,\\sigma\\\}^\{5/2\}\}\{\\min\\\{1,\\sigma\\\}^\{2\}\}\\exp\\left\(\\frac\{\\beta^\{2\}\}\{2\\min\\\{1,\\sigma\\\}^\{2\}\}\\right\)\\geq\\max\\\{\\alpha\_\{R,\\beta,\\sigma\}^\{\-1\},C\_\{R,\\beta,\\sigma\}\\\}\.\(824\)SinceR≤1R\\leq 1we knowγ1\(\[−R,R\]\)≥2Re−1/2/2​π=2π​eR\\gamma\_\{1\}\(\[\-R,R\]\)\\geq 2Re^\{\-1/2\}/\\sqrt\{2\\pi\}=\\sqrt\{\\frac\{2\}\{\\pi e\}\}R, and so

αR,β,σ−1\\displaystyle\\alpha\_\{R,\\beta,\\sigma\}^\{\-1\}=γ1\(\[−R,R\]\)−1/2q\+5/2exp\(R​β\+β2/2\+R2​max⁡\{0,1−σ2\}/22​q−2\)\\displaystyle=\\gamma\_\{1\}\(\[\-R,R\]\)^\{\-1/2\}q\_\{\+\}^\{5/2\}\\exp\\left\(\\frac\{R\\beta\+\\beta^\{2\}/2\+R^\{2\}\\max\\\{0,1\-\\sigma^\{2\}\\\}/2\}\{2q\_\{\-\}^\{2\}\}\\right\)\(825\)≤e\(π2\)1/4R−1/2q\+5/2exp\(β24​q−2\)\\displaystyle\\leq e\\left\(\\frac\{\\pi\}\{2\}\\right\)^\{1/4\}R^\{\-1/2\}q\_\{\+\}^\{5/2\}\\exp\\left\(\\frac\{\\beta^\{2\}\}\{4q\_\{\-\}^\{2\}\}\\right\)\(826\)≤e​\(π2\)1/4​1\+βq−2​q\+5/2​exp⁡\(β22​q−2\)\\displaystyle\\leq e\\left\(\\frac\{\\pi\}\{2\}\\right\)^\{1/4\}\\frac\{1\+\\beta\}\{q\_\{\-\}^\{2\}\}q\_\{\+\}^\{5/2\}\\exp\\left\(\\frac\{\\beta^\{2\}\}\{2q\_\{\-\}^\{2\}\}\\right\)\(827\)≤Qβ,σ\.\\displaystyle\\leq Q\_\{\\beta,\\sigma\}\.\(828\)Similarly, we have

CR,β,σ\\displaystyle C\_\{R,\\beta,\\sigma\}=2​\(R\+β\)q−2​exp⁡\(β22​q−2\+β​Rq−2\+R22​q−2\)\\displaystyle=\\frac\{2\(R\+\\beta\)\}\{q\_\{\-\}^\{2\}\}\\exp\\left\(\\frac\{\\beta^\{2\}\}\{2q\_\{\-\}^\{2\}\}\+\\frac\{\\beta R\}\{q\_\{\-\}^\{2\}\}\+\\frac\{R^\{2\}\}\{2q\_\{\-\}^\{2\}\}\\right\)\(829\)≤2​e3/2​1\+βq−2​exp⁡\(β22​q−2\)\\displaystyle\\leq 2e^\{3/2\}\\frac\{1\+\\beta\}\{q\_\{\-\}^\{2\}\}\\exp\\left\(\\frac\{\\beta^\{2\}\}\{2q\_\{\-\}^\{2\}\}\\right\)\(830\)≤Qβ,σ\.\\displaystyle\\leq Q\_\{\\beta,\\sigma\}\.\(831\)We now prove the width lower bound\. Applying Lemma[23](https://arxiv.org/html/2610.10875#Thmlemma23)toF=s^τF=\\hat\{s\}\_\{\\tau\}gives

ε≥αR,β,σ​\(‖e−τ​Pτ​Ψ−cσ,τ​𝒜R​s^τ‖L2​\(γn\)−CR,β,σ​L2​τ\)\\varepsilon\\geq\\alpha\_\{R,\\beta,\\sigma\}\\left\(\\\|e^\{\-\\tau\}P\_\{\\tau\}\\Psi\-c\_\{\\sigma,\\tau\}\\mathcal\{A\}\_\{R\}\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}\-C\_\{R,\\beta,\\sigma\}L^\{2\}\\tau\\right\)\(832\)and therefore

‖e−τ​Pτ​Ψ−cσ,τ​𝒜R​s^τ‖L2​\(γn\)≤αR,β,σ−1​ε\+CR,β,σ​L2​τ≤Qβ,σ​\(ε\+L2​τ\)\.\\\|e^\{\-\\tau\}P\_\{\\tau\}\\Psi\-c\_\{\\sigma,\\tau\}\\mathcal\{A\}\_\{R\}\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}\\leq\\alpha\_\{R,\\beta,\\sigma\}^\{\-1\}\\varepsilon\+C\_\{R,\\beta,\\sigma\}L^\{2\}\\tau\\leq Q\_\{\\beta,\\sigma\}\(\\varepsilon\+L^\{2\}\\tau\)\.\(833\)Since\(s^τ\)n\+1∈ℛm\(n\+1\)\(\\hat\{s\}\_\{\\tau\}\)\_\{n\+1\}\\in\\mathcal\{R\}\_\{m\}^\{\(n\+1\)\}, Lemma[24](https://arxiv.org/html/2610.10875#Thmlemma24)ensures thatcσ,τ​𝒜R​s^τ∈ℛm\(n\)c\_\{\\sigma,\\tau\}\\mathcal\{A\}\_\{R\}\\hat\{s\}\_\{\\tau\}\\in\\mathcal\{R\}\_\{m\}^\{\(n\)\}\. By Lemma[22](https://arxiv.org/html/2610.10875#Thmlemma22), we know that

aΨ​\(m,τ\)≤Qβ,σ​\(ε\+L2​τ\)a\_\{\\Psi\}\(m,\\tau\)\\leq Q\_\{\\beta,\\sigma\}\(\\varepsilon\+L^\{2\}\\tau\)\(834\)and thereforem≥mΨ​\(Qβ,σ​\(ε\+L2​τ\),τ\)m\\geq m\_\{\\Psi\}\(Q\_\{\\beta,\\sigma\}\(\\varepsilon\+L^\{2\}\\tau\),\\tau\)\.

It remains to prove the Barron lower bound\. Letℋ:=L2​\(ρτψ,Ψ\)\\mathcal\{H\}:=L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\}\), denote

κs:=\(𝔼G∼γ1​\|G\|2​s\)1/\(2​s\)=\(2s​Γ​\(s\+1/2\)π\)1/\(2​s\),\\kappa\_\{s\}:=\(\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\|G\|^\{2s\}\)^\{1/\(2s\)\}=\\left\(\\frac\{2^\{s\}\\Gamma\(s\+1/2\)\}\{\\sqrt\{\\pi\}\}\\right\)^\{1/\(2s\)\},\(835\)and define the constant

Λs,β,σ:=\{\(\(κs​q\+\)2​s\+max⁡\{1,β\}2​s\)1/20<s<1/2,max⁡\{κs​q\+\+β,1\}ss≥1/2\.\\Lambda\_\{s,\\beta,\\sigma\}:=\\begin\{cases\}\\left\(\(\\kappa\_\{s\}q\_\{\+\}\)^\{2s\}\+\\max\\\{1,\\beta\\\}^\{2s\}\\right\)^\{1/2\}&0<s<1/2,\\\\ \\max\\\{\\kappa\_\{s\}q\_\{\+\}\+\\beta,1\\\}^\{s\}&s\\geq 1/2\.\\end\{cases\}\(836\)Finally, set

Aϕ,β,σ:=\{‖ϕ‖∞ϕbounded,max⁡\{\|ϕ⁡\(1\)\|,\|ϕ⁡\(−1\)\|\}​Λs,β,σϕpositivelys\-homogeneous\.A\_\{\\phi,\\beta,\\sigma\}:=\\begin\{cases\}\\\|\\phi\\\|\_\{\\infty\}&\\text\{$\\phi$ bounded\},\\\\ \\max\\\{\|\\phi\(1\)\|,\|\\phi\(\-1\)\|\\\}\\Lambda\_\{s,\\beta,\\sigma\}&\\text\{$\\phi$ positively $s$\-homogeneous\.\}\\end\{cases\}\(837\)We show that every normalized neuron hasℋ\\mathcal\{H\}\-norm at mostAϕ,β,σA\_\{\\phi,\\beta,\\sigma\}\. Ifϕ\\phiis bounded then obviously

‖ϕ⁡\(⟨⋅,a⟩\+b\)1\+‖a‖\+\|b\|‖ℋ≤‖ϕ‖∞1\+‖a‖\+\|b\|≤Aϕ,β,σ\.\\left\\\|\\frac\{\\phi\(\\left\\langle\\cdot,a\\right\\rangle\+b\)\}\{1\+\\\|a\\\|\+\|b\|\}\\right\\\|\_\{\\mathcal\{H\}\}\\leq\\frac\{\\\|\\phi\\\|\_\{\\infty\}\}\{1\+\\\|a\\\|\+\|b\|\}\\leq A\_\{\\phi,\\beta,\\sigma\}\.\(838\)Suppose now thatϕ\\phiis positivelyss\-homogeneous\. LetXτ∼ρτψ,ΨX\_\{\\tau\}\\sim\\rho\_\{\\tau\}^\{\\psi,\\Psi\}so that

Xτ=e−τ​\(Z,σ​U\+Ψ⁡\(Z\)\)\+1−e−2​τ​GX\_\{\\tau\}=e^\{\-\\tau\}\(Z,\\sigma U\+\\Psi\(Z\)\)\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\(839\)forZ∼γnZ\\sim\\gamma\_\{n\},U∼γ1U\\sim\\gamma\_\{1\},G∼γn\+1G\\sim\\gamma\_\{n\+1\}independent\. For parametersa=\(v,t\)∈ℝn×ℝa=\(v,t\)\\in\{\\mathbb\{R\}\}^\{n\}\\times\{\\mathbb\{R\}\}andb∈ℝb\\in\{\\mathbb\{R\}\}, write

⟨Xτ,a⟩\+b=Ha\+Da,b\\left\\langle X\_\{\\tau\},a\\right\\rangle\+b=H\_\{a\}\+D\_\{a,b\}\(840\)where

Ha:=e−τ​\(⟨Z,v⟩\+σ​t​U\)\+1−e−2​τ​⟨G,a⟩H\_\{a\}:=e^\{\-\\tau\}\(\\left\\langle Z,v\\right\\rangle\+\\sigma tU\)\+\\sqrt\{1\-e^\{\-2\\tau\}\}\\left\\langle G,a\\right\\rangle\(841\)is centered Gaussian and

Da,b:=e−τ​t​Ψ​\(Z\)\+b\.D\_\{a,b\}:=e^\{\-\\tau\}t\\Psi\(Z\)\+b\.\(842\)Since we have

𝔼⁡\[Ha2\]=‖v‖2\+cσ,τ​t2≤q\+2​‖a‖2\\mathbb\{E\}\[H\_\{a\}^\{2\}\]=\\\|v\\\|^\{2\}\+c\_\{\\sigma,\\tau\}t^\{2\}\\leq q\_\{\+\}^\{2\}\\\|a\\\|^\{2\}\(843\)we find that‖Ha‖L2​s≤κs​q\+​‖a‖\\\|H\_\{a\}\\\|\_\{L^\{2s\}\}\\leq\\kappa\_\{s\}q\_\{\+\}\\\|a\\\|\. Moreover,

\|Da,b\|≤\|t\|​β\+\|b\|≤β\|a\|\+\|b\|\.\|D\_\{a,b\}\|\\leq\|t\|\\beta\+\|b\|\\leq\\beta\\\|a\\\|\+\|b\|\.\(844\)Suppose first thats≥1/2s\\geq 1/2\. Since\|ϕ⁡\(w\)\|≤max⁡\{\|ϕ⁡\(1\)\|,\|ϕ⁡\(−1\)\|\}​\|w\|s\|\\phi\(w\)\|\\leq\\max\\\{\|\\phi\(1\)\|,\|\\phi\(\-1\)\|\\\}\|w\|^\{s\}we have

‖ϕ⁡\(⟨Xτ,a⟩\+b\)‖L2\\displaystyle\\\|\\phi\(\\left\\langle X\_\{\\tau\},a\\right\\rangle\+b\)\\\|\_\{L^\{2\}\}≤max⁡\{\|ϕ⁡\(1\)\|,\|ϕ⁡\(−1\)\|\}⋅‖Ha\+Da,b‖L2​ss\\displaystyle\\leq\\max\\\{\|\\phi\(1\)\|,\|\\phi\(\-1\)\|\\\}\\cdot\\\|H\_\{a\}\+D\_\{a,b\}\\\|\_\{L^\{2s\}\}^\{s\}\(845\)≤max⁡\{\|ϕ⁡\(1\)\|,\|ϕ⁡\(−1\)\|\}⋅\(κs​q\+​‖a​‖\+β‖​a‖\+\|b\|\)s\\displaystyle\\leq\\max\\\{\|\\phi\(1\)\|,\|\\phi\(\-1\)\|\\\}\\cdot\(\\kappa\_\{s\}q\_\{\+\}\\\|a\\\|\+\\beta\\\|a\\\|\+\|b\|\)^\{s\}\(846\)≤Aϕ,β,σ​\(‖a‖\+\|b\|\)s\.\\displaystyle\\leq A\_\{\\phi,\\beta,\\sigma\}\(\\\|a\\\|\+\|b\|\)^\{s\}\.\(847\)If we instead have0<s<1/20<s<1/2we know\|x\+y\|2​s≤\|x\|2​s\+\|y\|2​s\|x\+y\|^\{2s\}\\leq\|x\|^\{2s\}\+\|y\|^\{2s\}and so

𝔼​\|Ha\+Da,b\|2​s\\displaystyle\\mathbb\{E\}\|H\_\{a\}\+D\_\{a,b\}\|^\{2s\}≤𝔼​\|Ha\|2​s\+𝔼​\|Da,b\|2​s\\displaystyle\\leq\\mathbb\{E\}\|H\_\{a\}\|^\{2s\}\+\\mathbb\{E\}\|D\_\{a,b\}\|^\{2s\}\(848\)≤\(κs​q\+\)2​s​‖a‖2​s\+\(β​‖a‖\+\|b\|\)2​s\\displaystyle\\leq\(\\kappa\_\{s\}q\_\{\+\}\)^\{2s\}\\\|a\\\|^\{2s\}\+\(\\beta\\\|a\\\|\+\|b\|\)^\{2s\}\(849\)≤Λs,β,σ2​\(‖a‖\+\|b\|\)2​s\.\\displaystyle\\leq\\Lambda\_\{s,\\beta,\\sigma\}^\{2\}\(\\\|a\\\|\+\|b\|\)^\{2s\}\.\(850\)Hence, in either case we know that

‖ϕ⁡\(⟨⋅,a⟩\+b\)\(‖a‖\+\|b\|\)s‖ℋ≤Aϕ,β,σ\.\\left\\\|\\frac\{\\phi\(\\left\\langle\\cdot,a\\right\\rangle\+b\)\}\{\(\\\|a\\\|\+\|b\|\)^\{s\}\}\\right\\\|\_\{\\mathcal\{H\}\}\\leq A\_\{\\phi,\\beta,\\sigma\}\.\(851\)We finish by noting that if𝒩ϕ,k\(n\+1,n\+1\)\\mathcal\{N\}\_\{\\phi,k\}^\{\(n\+1,n\+1\)\}denotes the class of vector\-valued two\-layer neural networks of width at mostkkwith activationϕ\\phi, then

infG∈𝒩ϕ,k\(n\+1,n\+1\)‖s^τ−G‖ℋ≤Aϕ,β,σ​‖s^τ‖ℬϕk\(∀k∈ℕ\)\.\\inf\_\{G\\in\\mathcal\{N\}\_\{\\phi,k\}^\{\(n\+1,n\+1\)\}\}\\\|\\hat\{s\}\_\{\\tau\}\-G\\\|\_\{\\mathcal\{H\}\}\\leq\\frac\{A\_\{\\phi,\\beta,\\sigma\}\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\}\{\\sqrt\{k\}\}\\quad\(\\forall k\\in\{\\mathbb\{N\}\}\)\.\(852\)Indeed, given a Barron representations^τ​\(⋅\)=∫ℝn\+1×ℝϕ⁡\(⟨⋅,a⟩\+b\)​𝑑μ​\(a,b\)\\hat\{s\}\_\{\\tau\}\(\\cdot\)=\\int\_\{\{\\mathbb\{R\}\}^\{n\+1\}\\times\{\\mathbb\{R\}\}\}\\phi\(\\left\\langle\\cdot,a\\right\\rangle\+b\)d\\mu\(a,b\)with

Bμ:=∫ℝn\+1×ℝw⁡\(a,b\)​𝑑μ​\(a,b\)≤‖s^τ‖ℬϕ\+δ,w⁡\(a,b\):=\{1\+‖a‖\+\|b\|ϕbounded,\(‖a‖\+\|b\|\)sϕs\-homogeneous,B\_\{\\mu\}:=\\int\_\{\{\\mathbb\{R\}\}^\{n\+1\}\\times\{\\mathbb\{R\}\}\}w\(a,b\)d\\mu\(a,b\)\\leq\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\+\\delta,\\quad w\(a,b\):=\\begin\{cases\}1\+\\\|a\\\|\+\|b\|&\\text\{$\\phi$ bounded\},\\\\ \(\\\|a\\\|\+\|b\|\)^\{s\}&\\text\{$\\phi$ $s$\-homogeneous\},\\end\{cases\}\(853\)we normalizeμ\\muto a probability measure, samplekki\.i\.d\. normalized neuronsYj:=ϕ⁡\(⟨⋅,aj⟩\+bj\)w⁡\(aj,bj\)Y\_\{j\}:=\\frac\{\\phi\(\\left\\langle\\cdot,a\_\{j\}\\right\\rangle\+b\_\{j\}\)\}\{w\(a\_\{j\},b\_\{j\}\)\}, and take

G^​\(⋅\):=Bμk​∑j=1kYj\.\\hat\{G\}\(\\cdot\):=\\frac\{B\_\{\\mu\}\}\{k\}\\sum\_\{j=1\}^\{k\}Y\_\{j\}\.\(854\)We may calculate that

𝔼​‖s^τ−G^‖ℋ2=Bμ2k​𝔼​‖Y1−𝔼​Y1‖ℋ2≤Bμ2​Aϕ,β,σ2k≤Aϕ,β,σ2​\(‖s^τ‖ℬϕ\+δ\)2k,\\mathbb\{E\}\\\|\\hat\{s\}\_\{\\tau\}\-\\hat\{G\}\\\|\_\{\\mathcal\{H\}\}^\{2\}=\\frac\{B\_\{\\mu\}^\{2\}\}\{k\}\\mathbb\{E\}\\\|Y\_\{1\}\-\\mathbb\{E\}Y\_\{1\}\\\|^\{2\}\_\{\\mathcal\{H\}\}\\leq\\frac\{B\_\{\\mu\}^\{2\}A\_\{\\phi,\\beta,\\sigma\}^\{2\}\}\{k\}\\leq\\frac\{A\_\{\\phi,\\beta,\\sigma\}^\{2\}\(\\\|\\widehat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\+\\delta\)^\{2\}\}\{k\},\(855\)after which takingδ↓0\\delta\\downarrow 0proves \([852](https://arxiv.org/html/2610.10875#A1.E852)\)\. Define

M:=mΨ​\(Qβ,σ​\(2​ε\+L2​τ\),τ\)\.M:=m\_\{\\Psi\}\(Q\_\{\\beta,\\sigma\}\(2\\varepsilon\+L^\{2\}\\tau\),\\tau\)\.\(856\)IfM≤1M\\leq 1then the claim is trivial, so we supposeM≥2M\\geq 2\. For everyk<Mk<M, the width lower bound we already proved implies

infG∈𝒩ϕ,k\(n\+1,n\+1\)‖sτψ,Ψ−G‖ℋ≥2​ε\\inf\_\{G\\in\\mathcal\{N\}\_\{\\phi,k\}^\{\(n\+1,n\+1\)\}\}\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-G\\\|\_\{\\mathcal\{H\}\}\\geq 2\\varepsilon\(857\)since the last coordinate satisfiesGn\+1∈ℛk\(n\+1\)G\_\{n\+1\}\\in\\mathcal\{R\}^\{\(n\+1\)\}\_\{k\}for allG∈𝒩ϕ,k\(n\+1,n\+1\)G\\in\\mathcal\{N\}\_\{\\phi,k\}^\{\(n\+1,n\+1\)\}\. On the other hand, by the triangle inequality and \([852](https://arxiv.org/html/2610.10875#A1.E852)\) we have

infG∈𝒩ϕ,k\(n\+1,n\+1\)‖sτψ,Ψ−G‖ℋ≤‖sτψ,Ψ−s^τ‖ℋ\+infG∈𝒩ϕ,k\(n\+1,n\+1\)‖s^τ−G‖ℋ≤ε\+Aϕ,β,σ​‖s^τ‖ℬϕk\\inf\_\{G\\in\\mathcal\{N\}\_\{\\phi,k\}^\{\(n\+1,n\+1\)\}\}\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-G\\\|\_\{\\mathcal\{H\}\}\\leq\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{H\}\}\+\\inf\_\{G\\in\\mathcal\{N\}\_\{\\phi,k\}^\{\(n\+1,n\+1\)\}\}\\\|\\hat\{s\}\_\{\\tau\}\-G\\\|\_\{\\mathcal\{H\}\}\\leq\\varepsilon\+\\frac\{A\_\{\\phi,\\beta,\\sigma\}\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\}\{\\sqrt\{k\}\}\(858\)for everyk<Mk<M\. Combining \([857](https://arxiv.org/html/2610.10875#A1.E857)\) and \([858](https://arxiv.org/html/2610.10875#A1.E858)\) implies that

‖s^τ‖ℬϕ≥ε​kAϕ,β,σ\(∀k<M\)\.\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\geq\\frac\{\\varepsilon\\sqrt\{k\}\}\{A\_\{\\phi,\\beta,\\sigma\}\}\\quad\(\\forall k<M\)\.\(859\)Takingk=M−1k=M\-1\(ork↑∞k\\uparrow\\inftyifM=∞M=\\infty\) and settingKϕ,β,σ=1/Aϕ,β,σK\_\{\\phi,\\beta,\\sigma\}=1/A\_\{\\phi,\\beta,\\sigma\}yields the claim\. ∎

There is also a nonquantitative upper bound in the sense that for everyε,τ,n\\varepsilon,\\tau,nthere exists a two\-layer neural networks^τ\\hat\{s\}\_\{\\tau\}of finite width and Barron norm for which‖sτψ,Ψ−s^τ‖L2​\(ρτψ,Ψ,ℝn\+1\)≤ε\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\leq\\varepsilon\. This is nothing more than the standard universalL2L^\{2\}approximation statement for two\-layer networks, due originally to Hornik in\[[44](https://arxiv.org/html/2610.10875#bib.bib21)\]\. For completeness, we record a precise formulation below:

###### Proposition 10\(Shallow approximation upper bound, nonquantitative\)\.

Letn∈ℕn\\in\{\\mathbb\{N\}\}andψ:ℝ→ℝ\\psi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\},Ψ:ℝn→ℝ\\Psi:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}be measurable, and letϕ:ℝ→ℝ\\phi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be nonpolynomial and satisfy Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)\. Then, for anyε,τ∈\(0,∞\)\\varepsilon,\\tau\\in\(0,\\infty\)there is somem∈ℕm\\in\{\\mathbb\{N\}\}and neural networks^τ:ℝn\+1→ℝn\+1\\hat\{s\}\_\{\\tau\}:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}of the forms^τ​\(x\)=∑j=1muj​ϕ​\(⟨x,aj⟩\+bj\)\\hat\{s\}\_\{\\tau\}\(x\)=\\sum\_\{j=1\}^\{m\}u\_\{j\}\\phi\(\\left\\langle x,a\_\{j\}\\right\\rangle\+b\_\{j\}\)withuj,aj∈ℝn\+1u\_\{j\},a\_\{j\}\\in\{\\mathbb\{R\}\}^\{n\+1\}andbj∈ℝb\_\{j\}\\in\{\\mathbb\{R\}\}for which‖s^τ‖ℬϕ<∞\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}<\\inftyand

‖sτψ,Ψ−s^τ‖L2​\(ρτψ,Ψ,ℝn\+1\)≤ε\.\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}\\leq\\varepsilon\.\(860\)Ifπψ,Ψ∈𝒫2​\(ℝn\+1\)\\pi^\{\\psi,\\Psi\}\\in\\mathcal\{P\}\_\{2\}\(\{\\mathbb\{R\}\}^\{n\+1\}\), then the same statement holds withsτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\}replaced bybτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\.

###### Proof\.

Let us write the OU interpolant by

Xτ:=e−τ​X\+1−e−2​τ​Z,X∼πψ,Ψ,Z∼γn\+1X\_\{\\tau\}:=e^\{\-\\tau\}X\+\\sqrt\{1\-e^\{\-2\\tau\}\}Z,\\quad X\\sim\\pi^\{\\psi,\\Psi\},\\;Z\\sim\\gamma\_\{n\+1\}\(861\)withX,ZX,Zindependent\. Fixτ\>0\\tau\>0\. By the Tweedie formula \([29](https://arxiv.org/html/2610.10875#S2.E29)\) we havesτψ,Ψ​\(x\)=−11−e−2​τ​𝔼​\[Z∣Xτ=x\]s\_\{\\tau\}^\{\\psi,\\Psi\}\(x\)=\-\\frac\{1\}\{\\sqrt\{1\-e^\{\-2\\tau\}\}\}\\mathbb\{E\}\[Z\\mid X\_\{\\tau\}=x\]\. Conditional Jensen implies that

‖sτψ,Ψ‖L2​\(ρτψ,Ψ,ℝn\+1\)2=11−e−2​τ​𝔼​\[‖𝔼⁡\[Z∣Xτ\]‖2\]≤𝔼​‖Z‖21−e−2​τ=n\+11−e−2​τ<∞\\displaystyle\\\|s\_\{\\tau\}^\{\\psi,\\Psi\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\}^\{2\}=\\frac\{1\}\{1\-e^\{\-2\\tau\}\}\\mathbb\{E\}\\left\[\\\|\\mathbb\{E\}\[Z\\mid X\_\{\\tau\}\]\\\|^\{2\}\\right\]\\leq\\frac\{\\mathbb\{E\}\\\|Z\\\|^\{2\}\}\{1\-e^\{\-2\\tau\}\}=\\frac\{n\+1\}\{1\-e^\{\-2\\tau\}\}<\\infty\(862\)and hencesτψ,Ψ∈L2​\(ρτψ,Ψ,ℝn\+1\)s\_\{\\tau\}^\{\\psi,\\Psi\}\\in L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\)\. Ifϕ\\phiis nonconstant and bounded, then\[[44](https://arxiv.org/html/2610.10875#bib.bib21), Theorem 1\]directly implies the result\. Ifϕ\\phiis instead positivelyss\-homogeneous fors∈ℕs\\in\{\\mathbb\{N\}\}, then the\(s\+1\)t​h\(s\+1\)^\{th\}discrete derivative ofϕ\\phiis bounded, nonconstant \(sinceϕ\\phiis nonpolynomial\), and representable usings\+2s\+2neurons, so the same universal approximation statement applies at the cost of increasing the width\. Lastly, ifπψ,Ψ\\pi^\{\{\\psi,\\Psi\}\}has finite second moment thenbτψ,Ψ=sτψ,Ψ\+Idb\_\{\\tau\}^\{\\psi,\\Psi\}=s\_\{\\tau\}^\{\\psi,\\Psi\}\+\\operatorname\{Id\}is also inL2​\(ρτψ,Ψ,ℝn\+1\)L^\{2\}\(\\rho\_\{\\tau\}^\{\\psi,\\Psi\};\{\\mathbb\{R\}\}^\{n\+1\}\), and so we may apply the same argument\. ∎

#### A\.7\.3Early\-stopping lower bounds

The presented lower bounds describe the difficulty of score\-matching using shallow neural networks for OU times in a certain windowτ∈\[0,τ0\]\\tau\\in\[0,\\tau\_\{0\}\]\. However, in practice it is common to avoid modeling scores all the way toτ↓0\\tau\\downarrow 0by applying a cutoff\. Qualitatively speaking, to approximate the target distributionπ\\pito accuracyδ\\deltait suffices to fixτ1\>0\\tau\_\{1\}\>0for which the distance betweenPτ1∗​πP\_\{\\tau\_\{1\}\}^\{\*\}\\piandπ\\piis≲δ\\lesssim\\deltaand to only learn approximate scores‖sτ−s^τ‖L2​\(ρτ\)≲δ\\\|s\_\{\\tau\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\(\\rho\_\{\\tau\}\)\}\\lesssim\\deltaforτ∈\[τ1,∞\)\\tau\\in\[\\tau\_\{1\},\\infty\), see e\.g\.\[[18](https://arxiv.org/html/2610.10875#bib.bib72)\]\. Since we have shown that approximating scores on\[0,τ0\]\[0,\\tau\_\{0\}\]with shallow networks is hard, it remains to show that choosingτ1\>τ0\\tau\_\{1\}\>\\tau\_\{0\}introduces a lot of early\-stopping error\. We will provide some general tools for triangular shears here, which we apply to our ResNet pushforward examples in Section[A\.8](https://arxiv.org/html/2610.10875#A1.SS8)\.

We start with a lower bound on total variation andW1W\_\{1\}distances which uses only the triangular shear structure\. Forμ,ν∈𝒫⁡\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{d\}\), we setTV⁡\(μ,ν\)=sup\|μ⁡\(A\)−ν⁡\(A\)\|\\operatorname\{TV\}\(\\mu,\\nu\)=\\sup\|\\mu\(A\)\-\\nu\(A\)\|over measurable setsAA\.

###### Lemma 25\(Early\-stopping error,TV\\operatorname\{TV\}andW1W\_\{1\}\)\.

Letn≥2n\\geq 2andΨ∈L1​\(γn\)\\Psi\\in L^\{1\}\(\\gamma\_\{n\}\)and define

π:=πId,Ψ=Law⁡\(Z,U\+Ψ⁡\(Z\)\),ρτ:=Pτ∗​π,\\pi:=\\pi^\{\\operatorname\{Id\},\\Psi\}=\\operatorname\{Law\}\(Z,U\+\\Psi\(Z\)\),\\quad\\rho\_\{\\tau\}:=P\_\{\\tau\}^\{\*\}\\pi,\(863\)withZ∼γnZ\\sim\\gamma\_\{n\}andU∼γ1U\\sim\\gamma\_\{1\}independent\. If we denoteMΨ:=𝔼Z∼γn\[Ψ\(Z\)2e−Ψ\(Z\)2/4\]M\_\{\\Psi\}:=\\mathbb\{E\}\_\{Z\\sim\\gamma\_\{n\}\}\[\\Psi\(Z\)^\{2\}e^\{\-\\Psi\(Z\)^\{2\}/4\}\]then

min⁡\{TV⁡\(π,ρτ\),W1​\(π,ρτ\)\}≥MΨ4​2​\(1−e−2​τ\)\(∀τ≥0\)\.\\min\\left\\\{\\operatorname\{TV\}\(\\pi,\\rho\_\{\\tau\}\),\\;W\_\{1\}\(\\pi,\\rho\_\{\\tau\}\)\\right\\\}\\geq\\frac\{M\_\{\\Psi\}\}\{4\\sqrt\{2\}\}\(1\-e^\{\-2\\tau\}\)\\quad\(\\forall\\tau\\geq 0\)\.\(864\)

###### Proof\.

SinceΨ∈L1​\(γn\)\\Psi\\in L^\{1\}\(\\gamma\_\{n\}\), bothπ\\piandρτ\\rho\_\{\\tau\}have finite first moments\. Consider

f\(z,u\):=e−u2/2\.f\(z,u\):=e^\{\-u^\{2\}/2\}\.\(865\)We know0≤f≤10\\leq f\\leq 1and∥∇f∥∞=supu∈ℝ\|u\|e−u2/2≤1\\\|\\nabla f\\\|\_\{\\infty\}=\\sup\_\{u\\in\{\\mathbb\{R\}\}\}\|u\|e^\{\-u^\{2\}/2\}\\leq 1\. So, by the dual characterizations of total variation andW1W\_\{1\}, we know

min⁡\{TV⁡\(π,ρτ\),W1​\(π,ρτ\)\}≥∫ℝn\+1f​d​ρτ−∫ℝn\+1f​𝑑π\.\\min\\left\\\{\\operatorname\{TV\}\(\\pi,\\rho\_\{\\tau\}\),\\;W\_\{1\}\(\\pi,\\rho\_\{\\tau\}\)\\right\\\}\\geq\\int\_\{\{\\mathbb\{R\}\}^\{n\+1\}\}fd\\rho\_\{\\tau\}\-\\int\_\{\{\\mathbb\{R\}\}^\{n\+1\}\}fd\\pi\.\(866\)The last\-coordinate marginal ofρτ\\rho\_\{\\tau\}has law equal toLaw⁡\(U\+e−τ​Ψ​\(W\)\)\\operatorname\{Law\}\(U\+e^\{\-\\tau\}\\Psi\(W\)\)forU∼γ1U\\sim\\gamma\_\{1\}independent ofW∼γnW\\sim\\gamma\_\{n\}\. Using the fact that

𝔼U∼γ1\[e−\(U\+a\)2/2\]=e−a2/42,\\mathbb\{E\}\_\{U\\sim\\gamma\_\{1\}\}\\left\[e^\{\-\(U\+a\)^\{2\}/2\}\\right\]=\\frac\{e^\{\-a^\{2\}/4\}\}\{\\sqrt\{2\}\},\(867\)we obtain that

∫ℝn\+1fdρτ−∫ℝn\+1fdπ=12𝔼Z∼γn\[e−e−2​τΨ\(Z\)2/4−e−Ψ\(Z\)2/4\]\.\\int\_\{\{\\mathbb\{R\}\}^\{n\+1\}\}fd\\rho\_\{\\tau\}\-\\int\_\{\{\\mathbb\{R\}\}^\{n\+1\}\}fd\\pi=\\frac\{1\}\{\\sqrt\{2\}\}\\mathbb\{E\}\_\{Z\\sim\\gamma\_\{n\}\}\\left\[e^\{\-e^\{\-2\\tau\}\\Psi\(Z\)^\{2\}/4\}\-e^\{\-\\Psi\(Z\)^\{2\}/4\}\\right\]\.\(868\)Sinceex−1≥xe^\{x\}\-1\\geq xforx≥0x\\geq 0, pointwise we have

e−e−2​τΨ\(z\)2/4−e−Ψ\(z\)2/4=e−Ψ\(z\)2/4\(e\(1−e−2​τ\)​Ψ​\(z\)2/4−1\)≥1−e−2​τ4Ψ\(z\)2e−Ψ\(z\)2/4,e^\{\-e^\{\-2\\tau\}\\Psi\(z\)^\{2\}/4\}\-e^\{\-\\Psi\(z\)^\{2\}/4\}=e^\{\-\\Psi\(z\)^\{2\}/4\}\\left\(e^\{\(1\-e^\{\-2\\tau\}\)\\Psi\(z\)^\{2\}/4\}\-1\\right\)\\geq\\frac\{1\-e^\{\-2\\tau\}\}\{4\}\\Psi\(z\)^\{2\}e^\{\-\\Psi\(z\)^\{2\}/4\},\(869\)from which the claim follows by taking an expectation\. ∎

By Pinsker’s inequality, thisTV\\operatorname\{TV\}lower bound implies a lower bound on both forward and reverseKL\\sqrt\{\\operatorname\{KL\}\}\. However, we can get a sharper lower bound in terms of the Hermite mass ofΨ\\Psi:

###### Lemma 26\(Early\-stopping error,KL\{\\operatorname\{KL\}\}\)\.

LetΨ,π,ρτ\\Psi,\\pi,\\rho\_\{\\tau\}be as in Lemma[25](https://arxiv.org/html/2610.10875#Thmlemma25), and assume further that‖Ψ‖∞≤β\\\|\\Psi\\\|\_\{\\infty\}\\leq\\beta\. LetDτD\_\{\\tau\}denote eitherKL\(ρτ∥π\)\\operatorname\{KL\}\(\\rho\_\{\\tau\}\\\|\\pi\)orKL\(π∥ρτ\)\\operatorname\{KL\}\(\\pi\\\|\\rho\_\{\\tau\}\)\. Then,

Dτ≥12​\(1\+β2\)​maxm∈ℕ∪\{0\}​\{\(1−e−ℓn​\(m\)​τ\)​aΨ​\(m,0\)\}\(∀τ≥0\),\\sqrt\{D\_\{\\tau\}\}\\geq\\frac\{1\}\{\\sqrt\{2\(1\+\\beta^\{2\}\)\}\}\\max\_\{m\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}\}\\left\\\{\(1\-e^\{\-\\ell\_\{n\}\(m\)\\tau\}\)a\_\{\\Psi\}\(m,0\)\\right\\\}\\quad\(\\forall\\tau\\geq 0\),\(870\)where we denoteℓn​\(m\):=1\+2​⌈log⁡\(m\+1\)/log⁡\(n\)⌉\\ell\_\{n\}\(m\):=1\+2\\lceil\\log\(m\+1\)/\\log\(n\)\\rceil\.

###### Proof\.

The claim is immediate forτ=0\\tau=0, so fixτ\>0\\tau\>0\. Bothπ\\piandρτ\\rho\_\{\\tau\}have first\-coordinate marginalγn\\gamma\_\{n\}, and conditional on the first coordinate beingz∈ℝnz\\in\{\\mathbb\{R\}\}^\{n\}the last coordinate underπ\\pihas law𝒩⁡\(Ψ⁡\(z\),1\)\\mathcal\{N\}\(\\Psi\(z\),1\)and underρτ\\rho\_\{\\tau\}has law

Law⁡\(W\+e−τ​Ψ​\(e−τ​z\+1−e−2​τ​G\)\)\\operatorname\{Law\}\\left\(W\+e^\{\-\\tau\}\\Psi\\left\(e^\{\-\\tau\}z\+\\sqrt\{1\-e^\{\-2\\tau\}\}G\\right\)\\right\)\(871\)withW∼γ1W\\sim\\gamma\_\{1\},G∼γnG\\sim\\gamma\_\{n\}independent; this can be shown via Gaussian regression, see the proof of Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)\. In particular, both conditional laws are a standard Gaussian plus an independent shift supported on\[−β,β\]\[\-\\beta,\\beta\]\. Thus, withUUa random variable drawn from the conditional law of the last coordinate under eitherπ\\piorρτ\\rho\_\{\\tau\}we know

log⁡𝔼​et⁡\(U−𝔼​U\)≤1\+β22​t2\(∀t∈ℝ\),\\log\\mathbb\{E\}e^\{t\(U\-\\mathbb\{E\}U\)\}\\leq\\frac\{1\+\\beta^\{2\}\}\{2\}t^\{2\}\\quad\(\\forall t\\in\{\\mathbb\{R\}\}\),\(872\)where the bounded shift contributesβ2​t2/2\\beta^\{2\}t^\{2\}/2by Hoeffding’s lemma and the Gaussian contributest2/2t^\{2\}/2\. Now, we know the variational inequality that for probability measuresP,QP,Qonℝ\{\\mathbb\{R\}\}we have

KL\(P∥Q\)\\displaystyle\\operatorname\{KL\}\(P\\\|Q\)≥supf​Lipschitz\{𝔼P​\[f\]−log⁡𝔼Q​\[ef\]\}\\displaystyle\\geq\\sup\_\{f\\text\{ Lipschitz\}\}\\left\\\{\\mathbb\{E\}\_\{P\}\[f\]\-\\log\\mathbb\{E\}\_\{Q\}\\left\[e^\{f\}\\right\]\\right\\\}\(873\)≥supt∈ℝ\{t⁡\(𝔼P​Y−𝔼Q​Y\)−log⁡𝔼Q​et⁡\(Y−𝔼Q​Y\)\},\\displaystyle\\geq\\sup\_\{t\\in\{\\mathbb\{R\}\}\}\\left\\\{t\(\\mathbb\{E\}\_\{P\}Y\-\\mathbb\{E\}\_\{Q\}Y\)\-\\log\\mathbb\{E\}\_\{Q\}e^\{t\(Y\-\\mathbb\{E\}\_\{Q\}Y\)\}\\right\\\},\(874\)where we apply the first line withf⁡\(y\)=t⁡\(y−𝔼Q​\[y\]\)f\(y\)=t\(y\-\\mathbb\{E\}\_\{Q\}\[y\]\)to get the second\. Thus, withPPandQQthe distributions of the last coordinate underπ\\piandρτ\\rho\_\{\\tau\}conditioned on the first beingzz, we have

KL\(P∥Q\)≥supt∈ℝ\{t\(𝔼PY−𝔼QY\)−1\+β22t2\}=\|𝔼P​Y−𝔼Q​Y\|22​\(1\+β2\)=\|Ψ⁡\(z\)−e−τ​Pτ​Ψ​\(z\)\|22​\(1\+β2\)\.\\operatorname\{KL\}\(P\\\|Q\)\\geq\\sup\_\{t\\in\{\\mathbb\{R\}\}\}\\left\\\{t\(\\mathbb\{E\}\_\{P\}Y\-\\mathbb\{E\}\_\{Q\}Y\)\-\\frac\{1\+\\beta^\{2\}\}\{2\}t^\{2\}\\right\\\}=\\frac\{\\left\|\\mathbb\{E\}\_\{P\}Y\-\\mathbb\{E\}\_\{Q\}Y\\right\|^\{2\}\}\{2\(1\+\\beta^\{2\}\)\}=\\frac\{\|\\Psi\(z\)\-e^\{\-\\tau\}P\_\{\\tau\}\\Psi\(z\)\|^\{2\}\}\{2\(1\+\\beta^\{2\}\)\}\.\(875\)The same argument works withPPandQQswapped\. Since the marginals of the first coordinates coincide, by the chain rule for relative entropy we find

Dτ≥‖Ψ−e−τ​Pτ​Ψ‖L2​\(γn\)22​\(1\+β2\)=12​\(1\+β2\)​∑k=0∞\(1−e−\(k\+1\)​τ\)2​‖Mk​\(Ψ\)‖F2,D\_\{\\tau\}\\geq\\frac\{\\\|\\Psi\-e^\{\-\\tau\}P\_\{\\tau\}\\Psi\\\|\_\{L^\{2\}\(\\gamma\_\{n\}\)\}^\{2\}\}\{2\(1\+\\beta^\{2\}\)\}=\\frac\{1\}\{2\(1\+\\beta^\{2\}\)\}\\sum\_\{k=0\}^\{\\infty\}\(1\-e^\{\-\(k\+1\)\\tau\}\)^\{2\}\\\|M\_\{k\}\(\\Psi\)\\\|\_\{F\}^\{2\},\(876\)where we expand in Hermite chaoses\. For eachm∈ℕ∪\{0\}m\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}, by definition ofℓn​\(m\)\\ell\_\{n\}\(m\)we knowk\+1<ℓn​\(m\)⟹n⌊k/2⌋≤mk\+1<\\ell\_\{n\}\(m\)\\implies n^\{\\lfloor k/2\\rfloor\}\\leq m\. Hence,

Dτ\\displaystyle D\_\{\\tau\}≥12​\(1\+β2\)​∑k=0∞\(1−e−\(k\+1\)​τ\)2​∑j=m\+1n⌊k/2⌋σj​\(Mk​\(Ψ\)\)2\\displaystyle\\geq\\frac\{1\}\{2\(1\+\\beta^\{2\}\)\}\\sum\_\{k=0\}^\{\\infty\}\(1\-e^\{\-\(k\+1\)\\tau\}\)^\{2\}\\sum\_\{j=m\+1\}^\{n^\{\\lfloor k/2\\rfloor\}\}\\sigma\_\{j\}\(M\_\{k\}\(\\Psi\)\)^\{2\}\(877\)≥\(1−e−ℓn​\(m\)​τ\)22​\(1\+β2\)​∑k=0∞∑j=m\+1n⌊k/2⌋σj​\(Mk​\(Ψ\)\)2\\displaystyle\\geq\\frac\{\(1\-e^\{\-\\ell\_\{n\}\(m\)\\tau\}\)^\{2\}\}\{2\(1\+\\beta^\{2\}\)\}\\sum\_\{k=0\}^\{\\infty\}\\sum\_\{j=m\+1\}^\{n^\{\\lfloor k/2\\rfloor\}\}\\sigma\_\{j\}\(M\_\{k\}\(\\Psi\)\)^\{2\}\(878\)=\(1−e−ℓn​\(m\)​τ\)22​\(1\+β2\)​aΨ​\(m,0\)2\.\\displaystyle=\\frac\{\(1\-e^\{\-\\ell\_\{n\}\(m\)\\tau\}\)^\{2\}\}\{2\(1\+\\beta^\{2\}\)\}a\_\{\\Psi\}\(m,0\)^\{2\}\.\(879\)Taking square roots and maximizing overmmproves the claim\. SinceaΨ​\(m,0\)→0a\_\{\\Psi\}\(m,0\)\\to 0asm→∞m\\to\\infty, this maximum is attained at a finitemm\. ∎

### A\.8Reverse OU suboptimality for ResNet pushforwards

In the previous section, we dealt with the general class of Gaussian triangular shear measures

πψ,Ψ=Law⁡\(Z,ψ⁡\(U\)\+Ψ⁡\(Z\)\)∈𝒫⁡\(ℝn\+1\),\(Z,U\)∼γn\+1\\pi^\{\\psi,\\Psi\}=\\operatorname\{Law\}\(Z,\\psi\(U\)\+\\Psi\(Z\)\)\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{n\+1\}\),\\quad\(Z,U\)\\sim\\gamma\_\{n\+1\}\(880\)forψ:ℝ→ℝ\\psi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}andΨ:ℝn→ℝ\\Psi:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}\. For this class, we developed some tools to get lower and upper bounds for the representation and approximation of the reverse OU scoressτψ,Ψ,bτψ,Ψs\_\{\\tau\}^\{\\psi,\\Psi\},b\_\{\\tau\}^\{\\psi,\\Psi\}by two\-layer neural networks\. In this section, we will \(1\) describe a very natural subclass ofπψ,Ψ\\pi^\{\\psi,\\Psi\}given by pushforwards ofγn\+1\\gamma\_\{n\+1\}by certain ResNets and \(2\) verify that the induced shear maps\(ψ,Ψ\)\(\\psi,\\Psi\)satisfy the assumptions needed to apply the general lower and upper bounds\.

Our class of ResNets will have blocks which each perform a shear operation \(i\.e\.x↦x\+u​ϕ​\(⟨x,θ⟩\)x\\mapsto x\+u\\phi\(\\left\\langle x,\\theta\\right\\rangle\)with⟨u,θ⟩=0\\left\\langle u,\\theta\\right\\rangle=0\)\. In words, the architecture is:

1. 1\.First, there areLLlayers which separately perform computation on the firstnncoordinates \(labeledzz\) and on the last coordinate \(labeleduu\)\. We denote the input by\(z\(0\),u\(0\)\)∈ℝn\+1\(z^\{\(0\)\},u^\{\(0\)\}\)\\in\{\\mathbb\{R\}\}^\{n\+1\}, and after layerℓ\\ellwe denote the hidden state by\(z\(ℓ\),u\(ℓ\)\)∈ℝn\+1\(z^\{\(\\ell\)\},u^\{\(\\ell\)\}\)\\in\{\\mathbb\{R\}\}^\{n\+1\}\. The computations in thezzanduuchannels are performed independently\. After theseLLlayers, the value in theuucoordinate isu\(L\)=ψ⁡\(u\(0\)\)u^\{\(L\)\}=\\psi\(u^\{\(0\)\}\)\.
2. 2\.Next, there is a “readout” layer which uses the statez\(L\)z^\{\(L\)\}of thezzcoordinate to computeΨ\\Psiand add its output to theuucoordinate\. After this layer, the last coordinate containsu\(L\+1\)=ψ⁡\(u\(0\)\)\+Ψ⁡\(z\(0\)\)u^\{\(L\+1\)\}=\\psi\(u^\{\(0\)\}\)\+\\Psi\(z^\{\(0\)\}\)while thezzcoordinate remainsz\(L\+1\)=z\(L\)z^\{\(L\+1\)\}=z^\{\(L\)\}\.
3. 3\.Lastly, there are anotherLLlayers which only act on thezzcoordinate and leave theuucoordinate unchanged\. These layers implement the inverses of the originalLLlayers in reverse order\. In particular,u\(2​L\+1\)=u\(L\+1\)u^\{\(2L\+1\)\}=u^\{\(L\+1\)\}andz\(2​L\+1\)=z\(0\)z^\{\(2L\+1\)\}=z^\{\(0\)\}\.

Combining these three stages produces a neural network of depth2​L\+12L\+1whose input\-output mapF:ℝn\+1→ℝn\+1F:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}is given byF⁡\(z\(0\),u\(0\)\)=\(z\(0\),ψ⁡\(u\(0\)\)\+Ψ⁡\(z\(0\)\)\)F\(z^\{\(0\)\},u^\{\(0\)\}\)=\(z^\{\(0\)\},\\psi\(u^\{\(0\)\}\)\+\\Psi\(z^\{\(0\)\}\)\)\. Hence,F\#​γn\+1=πψ,ΨF\_\{\\\#\}\\gamma\_\{n\+1\}=\\pi^\{\\psi,\\Psi\}\. This architecture allows us to representπψ,Ψ\\pi^\{\\psi,\\Psi\}as a ResNet pushforward of the Gaussian assuming that we can \(1\) implementΨ\\PsiusingLLlayers of single\-index shears in thezzchannel along with a readout layer and \(2\) implementψ\\psiusingLLlayers in theuuchannel\. The shear structure of thezzchannel is crucial, since it allows us to invert the entire stage\. As an added bonus, it will let us represent each layer using a continuous\-time flow\.

We now introduce notation for this construction\. For an activationφ:ℝ→ℝ\\varphi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}and a widthm∈ℕm\\in\{\\mathbb\{N\}\}, we define a class of functions fromℝ→ℝ\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}by

𝒩φ,m:=\{x↦c0\+∑j=1mcjφ\(αjx\+βj\);\(cj\)j=0m,\(αj\)j=1m,\(βj\)j=1m⊆ℝ\}\.\\mathcal\{N\}\_\{\\varphi,m\}:=\\left\\\{x\\mapsto c\_\{0\}\+\\sum\_\{j=1\}^\{m\}c\_\{j\}\\varphi\(\\alpha\_\{j\}x\+\\beta\_\{j\}\);\\quad\(c\_\{j\}\)\_\{j=0\}^\{m\},\(\\alpha\_\{j\}\)\_\{j=1\}^\{m\},\(\\beta\_\{j\}\)\_\{j=1\}^\{m\}\\subseteq\{\\mathbb\{R\}\}\\right\\\}\.\(881\)These are the functions which can be implemented by width\-mmneural networks with activationφ\\varphi\. Anyφ~∈𝒩φ,m\\widetilde\{\\varphi\}\\in\\mathcal\{N\}\_\{\\varphi,m\}can be treated as a compatible activation, in the sense that any layer of widthwwwith activationφ~\\widetilde\{\\varphi\}can be implemented using a layer of widthw​mwmwith activationφ\\varphi\.

###### Definition 10\(Triangular ResNet\)\.

Fix an activationφ:ℝ→ℝ\\varphi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}andn,W,L∈ℕn,W,L\\in\{\\mathbb\{N\}\}\. Choose scalar functionsg,χ∈𝒩φ,Wg,\\chi\\in\\mathcal\{N\}\_\{\\varphi,W\}\. For eachℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\}, choosehℓ∈𝒩φ,Wh\_\{\\ell\}\\in\\mathcal\{N\}\_\{\\varphi,W\}and input/output featuresaℓ,bℓ∈ℝna\_\{\\ell\},b\_\{\\ell\}\\in\{\\mathbb\{R\}\}^\{n\}with⟨aℓ,bℓ⟩=0\\left\\langle a\_\{\\ell\},b\_\{\\ell\}\\right\\rangle=0\. Choose also a readout directionv∈𝕊n−1v\\in\\mathbb\{S\}^\{n\-1\}\. Using these choices, we define a ResNet:

1. 1\.With\(z\(0\),u\(0\)\)∈ℝn\+1\(z^\{\(0\)\},u^\{\(0\)\}\)\\in\{\\mathbb\{R\}\}^\{n\+1\}the input, the first stage ofLLlayers is z\(ℓ\)=z\(ℓ−1\)\+bℓ​g​\(⟨z\(ℓ−1\),aℓ⟩\),u\(ℓ\)=u\(ℓ−1\)\+hℓ​\(u\(ℓ−1\)\),\(ℓ∈\{1,…,L\}\)\.\\begin\{aligned\} z^\{\(\\ell\)\}&=z^\{\(\\ell\-1\)\}\+b\_\{\\ell\}g\\left\(\\left\\langle z^\{\(\\ell\-1\)\},a\_\{\\ell\}\\right\\rangle\\right\),\\\\ u^\{\(\\ell\)\}&=u^\{\(\\ell\-1\)\}\+h\_\{\\ell\}\\left\(u^\{\(\\ell\-1\)\}\\right\),\\end\{aligned\}\\qquad\(\\ell\\in\\\{1,\\ldots,L\\\}\)\.\(882\)
2. 2\.The middle readout stage is z\(L\+1\)\\displaystyle z^\{\(L\+1\)\}=z\(L\),\\displaystyle=z^\{\(L\)\},\(883\)u\(L\+1\)\\displaystyle u^\{\(L\+1\)\}=u\(L\)\+χ⁡\(⟨z\(L\),v⟩\)\.\\displaystyle=u^\{\(L\)\}\+\\chi\\left\(\\left\\langle z^\{\(L\)\},v\\right\\rangle\\right\)\.
3. 3\.Denotingℓ:=L\+1−j\\ell:=L\+1\-j, the third stage ofLLlayers is z\(L\+j\+1\)=z\(L\+j\)−bℓ​g​\(⟨z\(L\+j\),aℓ⟩\),u\(L\+j\+1\)=u\(L\+j\),\(j∈\{1,…,L\}\)\.\\begin\{aligned\} z^\{\(L\+j\+1\)\}&=z^\{\(L\+j\)\}\-b\_\{\\ell\}g\\left\(\\left\\langle z^\{\(L\+j\)\},a\_\{\\ell\}\\right\\rangle\\right\),\\\\ u^\{\(L\+j\+1\)\}&=u^\{\(L\+j\)\},\\end\{aligned\}\\qquad\(j\\in\\\{1,\\ldots,L\\\}\)\.\(884\)

Denoting the parameters byθ:=\(g,χ,\(hℓ,aℓ,bℓ\)ℓ=1L,v\)\\theta:=\\left\(g,\\chi,\(h\_\{\\ell\},a\_\{\\ell\},b\_\{\\ell\}\)\_\{\\ell=1\}^\{L\},v\\right\), we writeTResNetθ:ℝn\+1→ℝn\+1\\operatorname\{TResNet\}\_\{\\theta\}:\{\\mathbb\{R\}\}^\{n\+1\}\\to\{\\mathbb\{R\}\}^\{n\+1\}to represent the mapping sending\(z\(0\),u\(0\)\)↦\(z\(2​L\+1\),u\(2​L\+1\)\)\(z^\{\(0\)\},u^\{\(0\)\}\)\\mapsto\(z^\{\(2L\+1\)\},u^\{\(2L\+1\)\}\)\.

We also denote the Gaussian pushforward byπθ:=\(TResNetθ\)\#​γn\+1\\pi^\{\\theta\}:=\(\\operatorname\{TResNet\}\_\{\\theta\}\)\_\{\\\#\}\\gamma\_\{n\+1\}, the OU flow byρτθ:=Pτ∗​πθ\\rho\_\{\\tau\}^\{\\theta\}:=P\_\{\\tau\}^\{\*\}\\pi^\{\\theta\}, and the Lebesgue and Gaussian scores respectively bysτθ:=∇log⁡ρτθs\_\{\\tau\}^\{\\theta\}:=\\nabla\\log\\rho\_\{\\tau\}^\{\\theta\}andbτθ:=∇log⁡ρτθγn\+1b\_\{\\tau\}^\{\\theta\}:=\\nabla\\log\\frac\{\\rho\_\{\\tau\}^\{\\theta\}\}\{\\gamma\_\{n\+1\}\}\.

This is a standard ResNet of depth2​L\+12L\+1using blocks which each have one hidden layer of2​W2Wneurons with activationφ\\varphi\. Throughout this section, we will useφ\\varphito denote the base activation that the neural network is implemented with\. We use the notation “TResNet” to emphasize the triangular structure, since we compute thezzanduucoordinates separately, combine their outputs, and roll thezzcomputation back to its initial state\. In the\(ψ,Ψ\)\(\\psi,\\Psi\)notation from earlier, we have:

###### Lemma 27\(Triangular structure\)\.

Letθ=\(g,χ,\(hℓ,aℓ,bℓ\)ℓ=1L,v\)\\theta=\\left\(g,\\chi,\(h\_\{\\ell\},a\_\{\\ell\},b\_\{\\ell\}\)\_\{\\ell=1\}^\{L\},v\\right\)\. Then,

TResNetθ⁡\(z,u\)=\(z,ψθ​\(u\)\+Ψθ​\(z\)\)\\operatorname\{TResNet\}\_\{\\theta\}\(z,u\)=\(z,\\psi\_\{\\theta\}\(u\)\+\\Psi\_\{\\theta\}\(z\)\)\(885\)where we defineψθ:ℝ→ℝ\\psi\_\{\\theta\}:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}byψθ:=\(Id\+hL\)∘⋯∘\(Id\+h1\)\\psi\_\{\\theta\}:=\(\\operatorname\{Id\}\+h\_\{L\}\)\\circ\\cdots\\circ\(\\operatorname\{Id\}\+h\_\{1\}\)andΨθ:ℝn→ℝ\\Psi\_\{\\theta\}:\{\\mathbb\{R\}\}^\{n\}\\to\{\\mathbb\{R\}\}by

Ψθ​\(z\):=χ⁡\(⟨GL​\(z\),v⟩\),Gℓ​\(z\):=Gℓ−1​\(z\)\+bℓ​g​\(⟨Gℓ−1​\(z\),aℓ⟩\),G0​\(z\):=z\.\\Psi\_\{\\theta\}\(z\):=\\chi\(\\left\\langle G\_\{L\}\(z\),v\\right\\rangle\),\\quad G\_\{\\ell\}\(z\):=G\_\{\\ell\-1\}\(z\)\+b\_\{\\ell\}g\\left\(\\left\\langle G\_\{\\ell\-1\}\(z\),a\_\{\\ell\}\\right\\rangle\\right\),\\quad G\_\{0\}\(z\):=z\.\(886\)In particular,πθ=πψθ,Ψθ\\pi^\{\\theta\}=\\pi^\{\\psi\_\{\\theta\},\\Psi\_\{\\theta\}\},sτθ=sτψθ,Ψθs\_\{\\tau\}^\{\\theta\}=s\_\{\\tau\}^\{\\psi\_\{\\theta\},\\Psi\_\{\\theta\}\}, andbτθ=bτψθ,Ψθb\_\{\\tau\}^\{\\theta\}=b\_\{\\tau\}^\{\\psi\_\{\\theta\},\\Psi\_\{\\theta\}\}\.

###### Proof\.

Defining the mapSℓ​\(x\):=x\+bℓ​g​\(⟨x,aℓ⟩\)S\_\{\\ell\}\(x\):=x\+b\_\{\\ell\}g\(\\left\\langle x,a\_\{\\ell\}\\right\\rangle\), since⟨aℓ,bℓ⟩=0\\left\\langle a\_\{\\ell\},b\_\{\\ell\}\\right\\rangle=0we know that

Sℓ−1​\(x\)=x−bℓ​g​\(⟨x,aℓ⟩\)\.S\_\{\\ell\}^\{\-1\}\(x\)=x\-b\_\{\\ell\}g\(\\left\\langle x,a\_\{\\ell\}\\right\\rangle\)\.\(887\)SinceGℓ=Sℓ∘⋯∘S1G\_\{\\ell\}=S\_\{\\ell\}\\circ\\cdots\\circ S\_\{1\}, the claim follows by construction\. ∎

#### A\.8\.1Representation lower and upper bounds

We first apply Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)for a shallow representation lower bound of the OU scoressτθ,bτθs\_\{\\tau\}^\{\\theta\},b\_\{\\tau\}^\{\\theta\}\. The mechanism is simple: composing nonlinearities can amplify derivatives exponentially\. We will first present a general condition, followed by a concrete construction\.

###### Lemma 28\(Shallow representation lower bound\)\.

Fixω∈Cb5​\(ℝ\)∩𝒩φ,W\\omega\\in C^\{5\}\_\{b\}\(\{\\mathbb\{R\}\}\)\\cap\\mathcal\{N\}\_\{\\varphi,W\}s\.t\.

ω⁡\(0\)=0,ω′​\(0\)\>0,ω′′​\(0\)=0,ω′′′​\(0\)≠0\.\\omega\(0\)=0,\\quad\\omega^\{\\prime\}\(0\)\>0,\\quad\\omega^\{\\prime\\prime\}\(0\)=0,\\quad\\omega^\{\\prime\\prime\\prime\}\(0\)\\neq 0\.\(888\)Then, there is a constantΛω∈\(0,∞\)\\Lambda\_\{\\omega\}\\in\(0,\\infty\)depending only onω\\omegafor which the following holds\. Letθ=\(g,χ,\(hℓ,aℓ,bℓ\)ℓ=1L,v\)\\theta=\\left\(g,\\chi,\(h\_\{\\ell\},a\_\{\\ell\},b\_\{\\ell\}\)\_\{\\ell=1\}^\{L\},v\\right\)satisfy the following properties:

1. \(i\)We haveg=Λω​ωg=\\Lambda\_\{\\omega\}\\omegaandχ=ω\\chi=\\omega\.
2. \(ii\)For everyℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\},Id\+hℓ\\operatorname\{Id\}\+h\_\{\\ell\}is increasing withhℓ=0h\_\{\\ell\}=0on\[−2​‖ω‖∞,2​‖ω‖∞\]\[\-2\\\|\\omega\\\|\_\{\\infty\},2\\\|\\omega\\\|\_\{\\infty\}\]\.
3. \(iii\)For everyℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\}we have‖aℓ‖=‖bℓ‖=1\\\|a\_\{\\ell\}\\\|=\\\|b\_\{\\ell\}\\\|=1and⟨aℓ,bℓ⟩=0\\left\\langle a\_\{\\ell\},b\_\{\\ell\}\\right\\rangle=0\. Also,aℓ\+1=bℓa\_\{\\ell\+1\}=b\_\{\\ell\}for everyℓ∈\{1,…,L−1\}\\ell\\in\\\{1,\\ldots,L\-1\\\}along withbL=vb\_\{L\}=v\.

Then, there are constantsBω∈\(0,∞\)B\_\{\\omega\}\\in\(0,\\infty\),cω,Cω∈\(1,∞\)c\_\{\\omega\},C\_\{\\omega\}\\in\(1,\\infty\)depending only onω\\omegas\.t\.Cω/cω≥2C\_\{\\omega\}/c\_\{\\omega\}\\geq 2and

6​‖sτθ‖ℬReLU3≥‖∂a13sτθ‖∞≥Bω​CωL\(∀τ∈\(0,Bω​cω−L\]\)\.6\\\|s\_\{\\tau\}^\{\\theta\}\\\|\_\{\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{3\}\}\}\\geq\\\|\\partial\_\{a\_\{1\}\}^\{3\}s\_\{\\tau\}^\{\\theta\}\\\|\_\{\\infty\}\\geq B\_\{\\omega\}C\_\{\\omega\}^\{L\}\\quad\\left\(\\forall\\tau\\in\\left\(0,\{B\_\{\\omega\}\}\{c\_\{\\omega\}^\{\-L\}\}\\right\]\\right\)\.\(889\)The same statement holds forbτθb\_\{\\tau\}^\{\\theta\}\. In particular,ℰℬReLU3​\(πθ\)≥Bω2​2L\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{3\}\}\}\(\\pi^\{\\theta\}\)\\geq B\_\{\\omega\}^\{2\}2^\{L\}and ifϕ∈C3​\(ℝ\)\\phi\\in C^\{3\}\(\{\\mathbb\{R\}\}\)satisfies Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)with‖ϕ‖∞,‖ϕ′′′‖∞<∞\\\|\\phi\\\|\_\{\\infty\},\\\|\\phi^\{\\prime\\prime\\prime\}\\\|\_\{\\infty\}<\\inftytheninfR\>0R2​ℰℬϕR​\(πθ\)≥Bω2​2L/‖ϕ′′′‖∞\\inf\_\{R\>0\}R^\{2\}\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}^\{R\}\}\(\\pi^\{\\theta\}\)\\geq B\_\{\\omega\}^\{2\}2^\{L\}/\\\|\\phi^\{\\prime\\prime\\prime\}\\\|\_\{\\infty\}\.

###### Proof\.

Let us denoteκ1:=ω′​\(0\)\\kappa\_\{1\}:=\\omega^\{\\prime\}\(0\),Kω:=52​\(1\+‖ω‖C5\)K\_\{\\omega\}:=52\(1\+\\\|\\omega\\\|\_\{C^\{5\}\}\), and

Λω:=max⁡\{1,6κ1,1458​Kω5κ16\}\.\\Lambda\_\{\\omega\}:=\\max\\left\\\{1,\\;\\frac\{6\}\{\\kappa\_\{1\}\},\\;\\frac\{1458K\_\{\\omega\}^\{5\}\}\{\\kappa\_\{1\}^\{6\}\}\\right\\\}\.\(891\)We also setCω:=\(κ1​Λω−2\)3C\_\{\\omega\}:=\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)^\{3\}andcω:=2​Kω5​\(1\+Λω\)5/Cωc\_\{\\omega\}:=2K\_\{\\omega\}^\{5\}\(1\+\\Lambda\_\{\\omega\}\)^\{5\}/C\_\{\\omega\}as well as

Bω:=\|ω′′′​\(0\)\|​min⁡\{12,LβKω5\}B\_\{\\omega\}:=\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|\\min\\left\\\{\\frac\{1\}\{2\},\\;\\frac\{L\_\{\\beta\}\}\{K\_\{\\omega\}^\{5\}\}\\right\\\}\(892\)whereβ:=‖ω‖∞\\beta:=\\\|\\omega\\\|\_\{\\infty\}andLβL\_\{\\beta\}is the constant from Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18)\. Sinceκ1​Λω≥6\\kappa\_\{1\}\\Lambda\_\{\\omega\}\\geq 6andΛω≥1\\Lambda\_\{\\omega\}\\geq 1we knowκ1​Λω−2≥23​κ1​Λω\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\\geq\\frac\{2\}\{3\}\\kappa\_\{1\}\\Lambda\_\{\\omega\}and1\+Λω≤2​Λω1\+\\Lambda\_\{\\omega\}\\leq 2\\Lambda\_\{\\omega\}\. Hence,

Cωcω=\(κ1​Λω−2\)62​Kω5​\(1\+Λω\)5≥κ16​Λω729​Kω5≥2\.\\frac\{C\_\{\\omega\}\}\{c\_\{\\omega\}\}=\\frac\{\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)^\{6\}\}\{2K\_\{\\omega\}^\{5\}\(1\+\\Lambda\_\{\\omega\}\)^\{5\}\}\\geq\\frac\{\\kappa\_\{1\}^\{6\}\\Lambda\_\{\\omega\}\}\{729K\_\{\\omega\}^\{5\}\}\\geq 2\.\(893\)Also,Kω≥κ1K\_\{\\omega\}\\geq\\kappa\_\{1\}and soKω​\(1\+Λω\)\>κ1​Λω\>κ1​Λω−2≥4K\_\{\\omega\}\(1\+\\Lambda\_\{\\omega\}\)\>\\kappa\_\{1\}\\Lambda\_\{\\omega\}\>\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\\geq 4\. Thus,

cω=2​Kω5​\(1\+Λω\)5\(κ1​Λω−2\)3\>2​\(κ1​Λω−2\)2≥32c\_\{\\omega\}=\\frac\{2K\_\{\\omega\}^\{5\}\(1\+\\Lambda\_\{\\omega\}\)^\{5\}\}\{\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)^\{3\}\}\>2\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)^\{2\}\\geq 32\(894\)and so1<cω≤Cω/21<c\_\{\\omega\}\\leq C\_\{\\omega\}/2\. Now, choose an orthonormal basis ofℝn\{\\mathbb\{R\}\}^\{n\}whose first vector isa1a\_\{1\}and set

D:=KωL\+1​\(1\+Λω\)L\.D:=K\_\{\\omega\}^\{L\+1\}\(1\+\\Lambda\_\{\\omega\}\)^\{L\}\.\(895\)By Faà di Bruno applied inductively to the recursion definingGℓG\_\{\\ell\}, we have

∥∂αGℓ∥∞≤\(Kω\(1\+Λω\)\)ℓ​\|α\|\(1≤\|α\|≤5,∀ℓ∈\{1,…,L\}\),\\\|\\partial^\{\\alpha\}G\_\{\\ell\}\\\|\_\{\\infty\}\\leq\(K\_\{\\omega\}\(1\+\\Lambda\_\{\\omega\}\)\)^\{\\ell\|\\alpha\|\}\\quad\(1\\leq\|\\alpha\|\\leq 5,\\;\\forall\\ell\\in\\\{1,\\ldots,L\\\}\),\(896\)since the number of partitions of a set of at most 5 elements is≤52\\leq 52\. Applying this formula once more toΨθ​\(z\)=ω⁡\(⟨GL​\(z\),v⟩\)\\Psi\_\{\\theta\}\(z\)=\\omega\(\\left\\langle G\_\{L\}\(z\),v\\right\\rangle\)gives

∥Ψθ∥∞≤β,∥∂αΨθ∥∞≤D\|α\|\(1≤\|α\|≤5\)\.\\\|\\Psi\_\{\\theta\}\\\|\_\{\\infty\}\\leq\\beta,\\quad\\\|\\partial^\{\\alpha\}\\Psi\_\{\\theta\}\\\|\_\{\\infty\}\\leq D^\{\|\\alpha\|\}\\quad\(1\\leq\|\\alpha\|\\leq 5\)\.\(897\)It remains to lower bound‖∂a13Ψθ‖∞\\\|\\partial\_\{a\_\{1\}\}^\{3\}\\Psi\_\{\\theta\}\\\|\_\{\\infty\}\. For this, we denote

w0:=a1,wℓ:=bℓ,Jℓ:=𝟙n\+κ1​Λω​wℓ​wℓ−1⊤w\_\{0\}:=a\_\{1\},\\quad w\_\{\\ell\}:=b\_\{\\ell\},\\quad J\_\{\\ell\}:=\{\\mathbbm\{1\}\}\_\{n\}\+\\kappa\_\{1\}\\Lambda\_\{\\omega\}w\_\{\\ell\}w\_\{\\ell\-1\}^\{\\top\}\(898\)forℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\}\. For0≤i<j≤L0\\leq i<j\\leq Lwe define

Pi,j:=⟨wj,Jj⋯Ji\+1wi⟩,Pi,i:=1\.P\_\{i,j\}:=\\left\\langle w\_\{j\},J\_\{j\}\\cdots J\_\{i\+1\}w\_\{i\}\\right\\rangle,\\quad P\_\{i,i\}:=1\.\(899\)We claim that

Pi,j≥\(κ1​Λω−2\)j−i\.P\_\{i,j\}\\geq\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)^\{j\-i\}\.\(900\)Indeed, starting fromui:=wiu\_\{i\}:=w\_\{i\}we recursively setuk:=Jk​uk−1u\_\{k\}:=J\_\{k\}u\_\{k\-1\}fork∈\{i\+1,…,j\}k\\in\\\{i\+1,\\ldots,j\\\}andAk:=⟨wk−1,uk−1⟩A\_\{k\}:=\\left\\langle w\_\{k\-1\},u\_\{k\-1\}\\right\\ranglefork∈\{i\+1,…,j\+1\}k\\in\\\{i\+1,\\ldots,j\+1\\\}\. We prove inductively thatAk\>0A\_\{k\}\>0and‖uk−1‖≤2​Ak\\\|u\_\{k\-1\}\\\|\\leq 2A\_\{k\}\. Fork=i\+1k=i\+1this is trivial, and since

uk=uk−1\+κ1​Λω​Ak​wku\_\{k\}=u\_\{k\-1\}\+\\kappa\_\{1\}\\Lambda\_\{\\omega\}A\_\{k\}w\_\{k\}\(901\)we have that

Ak\+1\\displaystyle A\_\{k\+1\}=⟨wk,uk⟩=⟨wk,uk−1⟩\+κ1​Λω​Ak≥−‖uk−1‖\+κ1​Λω​Ak≥\(κ1​Λω−2\)​Ak\>0\.\\displaystyle=\\left\\langle w\_\{k\},u\_\{k\}\\right\\rangle=\\left\\langle w\_\{k\},u\_\{k\-1\}\\right\\rangle\+\\kappa\_\{1\}\\Lambda\_\{\\omega\}A\_\{k\}\\geq\-\\\|u\_\{k\-1\}\\\|\+\\kappa\_\{1\}\\Lambda\_\{\\omega\}A\_\{k\}\\geq\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)A\_\{k\}\>0\.\(902\)Moreover,

‖uk‖≤‖uk−1‖\+κ1​Λω​Ak≤\(κ1​Λω\+2\)​Ak≤2​\(κ1​Λω−2\)​Ak≤2​Ak\+1,\\displaystyle\\\|u\_\{k\}\\\|\\leq\\\|u\_\{k\-1\}\\\|\+\\kappa\_\{1\}\\Lambda\_\{\\omega\}A\_\{k\}\\leq\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\+2\)A\_\{k\}\\leq 2\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)A\_\{k\}\\leq 2A\_\{k\+1\},\(903\)where the second to last inequality uses thatκ1​Λω≥6\\kappa\_\{1\}\\Lambda\_\{\\omega\}\\geq 6\. This closes the induction, and hence\(κ1​Λω−2\)​Ak≤Ak\+1\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)A\_\{k\}\\leq A\_\{k\+1\}for allkk\. SinceAj\+1=Pi,jA\_\{j\+1\}=P\_\{i,j\}, \([900](https://arxiv.org/html/2610.10875#A1.E900)\) follows\.

Now, let us define

F⁡\(t\):=⟨wL,GL​\(t​w0\)⟩\.F\(t\):=\\left\\langle w\_\{L\},G\_\{L\}\(tw\_\{0\}\)\\right\\rangle\.\(904\)From the recursion definingGℓG\_\{\\ell\}, we know that

Gℓ​\(t​w0\)=Gℓ−1​\(t​w0\)\+Λω​wℓ​ω​\(⟨wℓ−1,Gℓ−1​\(t​w0\)⟩\)\.G\_\{\\ell\}\(tw\_\{0\}\)=G\_\{\\ell\-1\}\(tw\_\{0\}\)\+\\Lambda\_\{\\omega\}w\_\{\\ell\}\\omega\(\\left\\langle w\_\{\\ell\-1\},G\_\{\\ell\-1\}\(tw\_\{0\}\)\\right\\rangle\)\.\(905\)SinceGℓ​\(0\)=0G\_\{\\ell\}\(0\)=0\(due toω⁡\(0\)=0\\omega\(0\)=0\), differentiating once att=0t=0yields

dd​t​Gℓ​\(t​w0\)\|t=0=Jℓ​dd​t​Gℓ−1​\(t​w0\)\|t=0\\left\.\\frac\{d\}\{dt\}G\_\{\\ell\}\(tw\_\{0\}\)\\right\|\_\{t=0\}=J\_\{\\ell\}\\left\.\\frac\{d\}\{dt\}G\_\{\\ell\-1\}\(tw\_\{0\}\)\\right\|\_\{t=0\}\(906\)which we iterate to find that

F′\(0\)=⟨wL,JL⋯J1w0⟩=P0,L\.F^\{\\prime\}\(0\)=\\left\\langle w\_\{L\},J\_\{L\}\\cdots J\_\{1\}w\_\{0\}\\right\\rangle=P\_\{0,L\}\.\(907\)Similarly, differentiating the recursion twice yields

d2d​t2​Gℓ​\(t​w0\)\|t=0=Jℓ​d2d​t2​Gℓ−1​\(t​w0\)\|t=0\+Λω​wℓ​ω′′​\(0\)​⟨wℓ−1,dd​t​Gℓ−1​\(t​w0\)\|t=0⟩2\.\\left\.\\frac\{d^\{2\}\}\{dt^\{2\}\}G\_\{\\ell\}\(tw\_\{0\}\)\\right\|\_\{t=0\}=J\_\{\\ell\}\\left\.\\frac\{d^\{2\}\}\{dt^\{2\}\}G\_\{\\ell\-1\}\(tw\_\{0\}\)\\right\|\_\{t=0\}\+\\Lambda\_\{\\omega\}w\_\{\\ell\}\\omega^\{\\prime\\prime\}\(0\)\\left\\langle w\_\{\\ell\-1\},\\left\.\\frac\{d\}\{dt\}G\_\{\\ell\-1\}\(tw\_\{0\}\)\\right\|\_\{t=0\}\\right\\rangle^\{2\}\.\(908\)Sinceω′′​\(0\)=0\\omega^\{\\prime\\prime\}\(0\)=0andG0​\(t​w0\)=t​w0G\_\{0\}\(tw\_\{0\}\)=tw\_\{0\}has zero second derivative att=0t=0, by induction we findd2d​t2​Gℓ​\(t​w0\)\|t=0=0\\left\.\\frac\{d^\{2\}\}\{dt^\{2\}\}G\_\{\\ell\}\(tw\_\{0\}\)\\right\|\_\{t=0\}=0for eachℓ\\elland thereforeF′′​\(0\)=0F^\{\\prime\\prime\}\(0\)=0\. Differentiating a third time, all terms involvingω′′​\(0\)\\omega^\{\\prime\\prime\}\(0\)or a second derivative ofGℓ−1​\(t​w0\)G\_\{\\ell\-1\}\(tw\_\{0\}\)vanish\. Thus,

d3d​t3​Gℓ​\(t​w0\)\|t=0\\displaystyle\\left\.\\frac\{d^\{3\}\}\{dt^\{3\}\}G\_\{\\ell\}\(tw\_\{0\}\)\\right\|\_\{t=0\}=Jℓ​d3d​t3​Gℓ−1​\(t​w0\)\|t=0\+Λω​wℓ​ω′′′​\(0\)​⟨wℓ−1,dd​t​Gℓ−1​\(t​w0\)\|t=0⟩3\\displaystyle=J\_\{\\ell\}\\left\.\\frac\{d^\{3\}\}\{dt^\{3\}\}G\_\{\\ell\-1\}\(tw\_\{0\}\)\\right\|\_\{t=0\}\+\\Lambda\_\{\\omega\}w\_\{\\ell\}\\omega^\{\\prime\\prime\\prime\}\(0\)\\left\\langle w\_\{\\ell\-1\},\\left\.\\frac\{d\}\{dt\}G\_\{\\ell\-1\}\(tw\_\{0\}\)\\right\|\_\{t=0\}\\right\\rangle^\{3\}\(909\)=Jℓ​d3d​t3​Gℓ−1​\(t​w0\)\|t=0\+Λω​wℓ​ω′′′​\(0\)​P0,ℓ−13\.\\displaystyle=J\_\{\\ell\}\\left\.\\frac\{d^\{3\}\}\{dt^\{3\}\}G\_\{\\ell\-1\}\(tw\_\{0\}\)\\right\|\_\{t=0\}\+\\Lambda\_\{\\omega\}w\_\{\\ell\}\\omega^\{\\prime\\prime\\prime\}\(0\)P\_\{0,\\ell\-1\}^\{3\}\.\(910\)By iterating this and then projecting ontowLw\_\{L\}, we find that

F′′′​\(0\)=Λω​ω′′′​\(0\)​∑ℓ=1LP0,ℓ−13​Pℓ,LF^\{\\prime\\prime\\prime\}\(0\)=\\Lambda\_\{\\omega\}\\omega^\{\\prime\\prime\\prime\}\(0\)\\sum\_\{\\ell=1\}^\{L\}P\_\{0,\\ell\-1\}^\{3\}P\_\{\\ell,L\}\(911\)since the cubic termwℓ​P0,ℓ−13w\_\{\\ell\}P\_\{0,\\ell\-1\}^\{3\}is propagated byJL⋯Jℓ\+1J\_\{L\}\\cdots J\_\{\\ell\+1\}\. Finally, sinceΨθ​\(t​w0\)=ω⁡\(F⁡\(t\)\)\\Psi\_\{\\theta\}\(tw\_\{0\}\)=\\omega\(F\(t\)\)we apply the chain rule to find

d3d​t3​ω​\(F⁡\(t\)\)=ω′′′​\(F⁡\(t\)\)​F′​\(t\)3\+3​ω′′​\(F⁡\(t\)\)​F′​\(t\)​F′′​\(t\)\+ω′​\(F⁡\(t\)\)​F′′′​\(t\)\.\\frac\{d^\{3\}\}\{dt^\{3\}\}\\omega\(F\(t\)\)=\\omega^\{\\prime\\prime\\prime\}\(F\(t\)\)F^\{\\prime\}\(t\)^\{3\}\+3\\omega^\{\\prime\\prime\}\(F\(t\)\)F^\{\\prime\}\(t\)F^\{\\prime\\prime\}\(t\)\+\\omega^\{\\prime\}\(F\(t\)\)F^\{\\prime\\prime\\prime\}\(t\)\.\(912\)Att=0t=0we haveF⁡\(0\)=F′′​\(0\)=ω′′​\(0\)=0F\(0\)=F^\{\\prime\\prime\}\(0\)=\\omega^\{\\prime\\prime\}\(0\)=0andω′​\(0\)=κ1\\omega^\{\\prime\}\(0\)=\\kappa\_\{1\}, and so

∂a13Ψθ​\(0\)=ω′′′​\(0\)​F′​\(0\)3\+κ1​F′′′​\(0\)=ω′′′​\(0\)​\(P0,L3\+κ1​Λω​∑ℓ=1LP0,ℓ−13​Pℓ,L\)\.\\partial\_\{a\_\{1\}\}^\{3\}\\Psi\_\{\\theta\}\(0\)=\\omega^\{\\prime\\prime\\prime\}\(0\)F^\{\\prime\}\(0\)^\{3\}\+\\kappa\_\{1\}F^\{\\prime\\prime\\prime\}\(0\)=\\omega^\{\\prime\\prime\\prime\}\(0\)\\left\(P\_\{0,L\}^\{3\}\+\\kappa\_\{1\}\\Lambda\_\{\\omega\}\\sum\_\{\\ell=1\}^\{L\}P\_\{0,\\ell\-1\}^\{3\}P\_\{\\ell,L\}\\right\)\.\(913\)Since all terms in the parentheses are positive, there is no cancellation and so

‖∂a13Ψθ‖∞≥\|∂a13Ψθ​\(0\)\|≥\|ω′′′​\(0\)\|​P0,L3≥\|ω′′′​\(0\)\|\(κ1​Λω−2\)3​L=\|ω′′′​\(0\)\|​CωL,\\\|\\partial\_\{a\_\{1\}\}^\{3\}\\Psi\_\{\\theta\}\\\|\_\{\\infty\}\\geq\|\\partial\_\{a\_\{1\}\}^\{3\}\\Psi\_\{\\theta\}\(0\)\|\\geq\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|P\_\{0,L\}^\{3\}\\geq\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|\(\\kappa\_\{1\}\\Lambda\_\{\\omega\}\-2\)^\{3L\}=\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|C\_\{\\omega\}^\{L\},\(914\)where we applied \([900](https://arxiv.org/html/2610.10875#A1.E900)\)\. By assumption \(ii\) we know thatψθ\\psi\_\{\\theta\}is increasing and the identity on\[−2​β,2​β\]\[\-2\\beta,2\\beta\]\. By Lemma[18](https://arxiv.org/html/2610.10875#Thmlemma18), we have that

‖∂a13sτθ‖∞≥\|ω′′′​\(0\)\|2​CωL≥Bω​CωL\\\|\\partial\_\{a\_\{1\}\}^\{3\}s\_\{\\tau\}^\{\\theta\}\\\|\_\{\\infty\}\\geq\\frac\{\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|\}\{2\}C\_\{\\omega\}^\{L\}\\geq B\_\{\\omega\}C\_\{\\omega\}^\{L\}\(915\)wheneverτ≤Lβ​\|ω′′′​\(0\)\|​CωL\(L\+1\)​D5\\tau\\leq L\_\{\\beta\}\\frac\{\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|C\_\{\\omega\}^\{L\}\}\{\(L\+1\)D^\{5\}\}\. Note thatΨθ\\Psi\_\{\\theta\}depends on a subspace of dimension at mostL\+1L\+1and so we replacennwithL\+1L\+1for this application \(since the OU semigroup tensorizes\)\. SinceL\+1≤2LL\+1\\leq 2^\{L\}, we find that

Lβ​\|ω′′′​\(0\)\|​CωL\(L\+1\)​D5\\displaystyle L\_\{\\beta\}\\frac\{\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|C\_\{\\omega\}^\{L\}\}\{\(L\+1\)D^\{5\}\}=Lβ​\|ω′′′​\(0\)\|\(L\+1\)​Kω5⋅\(CωKω5​\(1\+Λω\)5\)L\\displaystyle=L\_\{\\beta\}\\frac\{\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|\}\{\(L\+1\)K\_\{\\omega\}^\{5\}\}\\cdot\\left\(\\frac\{C\_\{\\omega\}\}\{K\_\{\\omega\}^\{5\}\(1\+\\Lambda\_\{\\omega\}\)^\{5\}\}\\right\)^\{L\}\(916\)≥Lβ​\|ω′′′​\(0\)\|Kω5⋅\(Cω2​Kω5​\(1\+Λω\)5\)L\\displaystyle\\geq L\_\{\\beta\}\\frac\{\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|\}\{K\_\{\\omega\}^\{5\}\}\\cdot\\left\(\\frac\{C\_\{\\omega\}\}\{2K\_\{\\omega\}^\{5\}\(1\+\\Lambda\_\{\\omega\}\)^\{5\}\}\\right\)^\{L\}\(917\)=Lβ​\|ω′′′​\(0\)\|Kω5⋅cω−L\\displaystyle=L\_\{\\beta\}\\frac\{\|\\omega^\{\\prime\\prime\\prime\}\(0\)\|\}\{K\_\{\\omega\}^\{5\}\}\\cdot c\_\{\\omega\}^\{\-L\}\(918\)≥Bω​cω−L\.\\displaystyle\\geq B\_\{\\omega\}c\_\{\\omega\}^\{\-L\}\.\(919\)The rest of the result follows from the proof of Corollary[2](https://arxiv.org/html/2610.10875#Thmcorollary2)\. ∎

We may also ensure that the Barron norms of the OU scores are finite for the examples produced in Lemma[28](https://arxiv.org/html/2610.10875#Thmlemma28)\. By Lemma[21](https://arxiv.org/html/2610.10875#Thmlemma21), it is sufficient to show thatΨθ\\Psi\_\{\\theta\}is a finite sum of smooth compactly\-supported cylindrical functions andψθ\\psi\_\{\\theta\}has polynomial tails\. Both of these can be achieved using a suitable base activationφ\\varphi:

###### Lemma 29\(Shallow representation upper bound\)\.

Assume the setting of Lemma[28](https://arxiv.org/html/2610.10875#Thmlemma28)\. Assume further thatω∈Cc∞∩𝒩φ,W\\omega\\in C^\{\\infty\}\_\{c\}\\cap\\mathcal\{N\}\_\{\\varphi,W\}and thatψ:=\(Id\+hL\)∘⋯∘\(Id\+h1\)\\psi:=\(\\operatorname\{Id\}\+h\_\{L\}\)\\circ\\cdots\\circ\(\\operatorname\{Id\}\+h\_\{1\}\)is aC∞C^\{\\infty\}diffeomorphism with

ψ⁡\(u\)=\{−a−​\(−u\)p\+b−u≤−Ra\+​up\+b\+u≥R\\psi\(u\)=\\begin\{cases\}\-a\_\{\-\}\(\-u\)^\{p\}\+b\_\{\-\}&u\\leq\-R\\\\ a\_\{\+\}u^\{p\}\+b\_\{\+\}&u\\geq R\\end\{cases\}\(920\)for somea±∈\(0,∞\)a\_\{\\pm\}\\in\(0,\\infty\),b±∈ℝb\_\{\\pm\}\\in\{\\mathbb\{R\}\},R\>2​‖ω‖∞R\>2\\\|\\omega\\\|\_\{\\infty\}, andp\>2p\>2\. Then,sτθ∈ℬReLUss\_\{\\tau\}^\{\\theta\}\\in\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{s\}\}for allτ≥0\\tau\\geq 0ands∈ℕs\\in\{\\mathbb\{N\}\}with the uniform bound

supτ∈\[0,T\]‖sτθ‖ℬReLUs<∞\(∀T∈\(0,∞\)\)\.\\sup\_\{\\tau\\in\[0,T\]\}\\\|s\_\{\\tau\}^\{\\theta\}\\\|\_\{\\mathcal\{B\}\_\{\\operatorname\{ReLU\}^\{s\}\}\}<\\infty\\quad\(\\forall T\\in\(0,\\infty\)\)\.\(921\)The same statements hold withsτθs\_\{\\tau\}^\{\\theta\}replaced bybτθb\_\{\\tau\}^\{\\theta\}\.

###### Proof\.

We know thatψθ=ψ\\psi\_\{\\theta\}=\\psi\. By the assumed tail behavior, we know thatψθ\\psi\_\{\\theta\}satisfies the assumptions of Lemma[21](https://arxiv.org/html/2610.10875#Thmlemma21)\. It remains to verify the assumptions onΨθ\\Psi\_\{\\theta\}\. We first show that iff⁡\(z\)=F⁡\(PE​z\)f\(z\)=F\(P\_\{E\}z\)is a cylindrical function withF∈Cc∞​\(E\)F\\in C^\{\\infty\}\_\{c\}\(E\)andS⁡\(z\):=z\+b​g​\(⟨a,z⟩\)S\(z\):=z\+bg\(\\left\\langle a,z\\right\\rangle\)is a single\-index field \(witha,b∈ℝna,b\\in\{\\mathbb\{R\}\}^\{n\}\) thenf∘Sf\\circ Sis a sum of two cylindrical functions which areCc∞C^\{\\infty\}\_\{c\}on their active subspaces\. Indeed, withV:=E\+span⁡\{a\}V:=E\+\\operatorname\{span\}\\\{a\\\}we define forx∈Vx\\in Vthe function

H⁡\(x\):=F⁡\(PE​x\+PE​b​g​\(⟨a,x⟩\)\)−F⁡\(PE​x\)H\(x\):=F\(P\_\{E\}x\+P\_\{E\}bg\(\\left\\langle a,x\\right\\rangle\)\)\-F\(P\_\{E\}x\)\(922\)and sof⁡\(S⁡\(z\)\)=F⁡\(PE​z\)\+H⁡\(PV​z\)f\(S\(z\)\)=F\(P\_\{E\}z\)\+H\(P\_\{V\}z\)\. Clearly,HHis smooth, and since∥x∥≲E,a∥PEx∥\+\|⟨a,x⟩\|\\\|x\\\|\\lesssim\_\{E,a\}\\\|P\_\{E\}x\\\|\+\|\\left\\langle a,x\\right\\rangle\|we find thatH∈Cc∞​\(V\)H\\in C^\{\\infty\}\_\{c\}\(V\)\. By linearity, ifffis a finite sum of cylindrical functions, each of which isCc∞C^\{\\infty\}\_\{c\}on its active subspace, then the same is true forf∘Sf\\circ S\.

Now, in the present construction we haveg,χ∈Cc∞​\(ℝ\)g,\\chi\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}\)andΨθ=f0∘SL∘⋯∘S1\\Psi\_\{\\theta\}=f\_\{0\}\\circ S\_\{L\}\\circ\\cdots\\circ S\_\{1\}withf0​\(z\)=ω⁡\(⟨v,z⟩\)f\_\{0\}\(z\)=\\omega\(\\left\\langle v,z\\right\\rangle\)\. Since shears preserve the finite structure of being a sum of smooth cylindrical functions,Ψθ\\Psi\_\{\\theta\}satisfies the assumptions of Lemma[21](https://arxiv.org/html/2610.10875#Thmlemma21)\. ∎

We will now instantiate these so that the lower and upper bounds both apply to the same teacher ResNet\. We chooseϕ=ReLU3\\phi=\\operatorname\{ReLU\}^\{3\}for the student activation, and we must takeφ\\varphito be a smooth mollification ofϕ\\phito ensure smoothness ofω\\omegaandψ\\psi:

###### Proposition 11\(Shallow representation bounds forReLU3\\operatorname\{ReLU\}^\{3\}\)\.

Letϕ⁡\(t\)=max⁡\{0,t\}3\\phi\(t\)=\\max\\\{0,t\\\}^\{3\}be theReLU3\\operatorname\{ReLU\}^\{3\}activation, and defineφ=ϕ∗η\\varphi=\\phi\*\\etausing some nonnegative evenη∈Cc∞​\(ℝ\)\\eta\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}\)with∫ℝη=1\\int\_\{\\mathbb\{R\}\}\\eta=1which is supported in\[−δ,δ\]\[\-\\delta,\\delta\]for someδ∈\(0,1/4\)\\delta\\in\(0,1/4\)\.

Then, for eachd≥4d\\geq 4there exists aTResNetθ:ℝd→ℝd\\operatorname\{TResNet\}\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}of depth2​d−32d\-3and width1616with two\-layer blocks using activationφ\\varphis\.t\.

ℰℬϕ\(πθ\)≳2d,Dℬϕ\(γd,πθ\)≲d,andsτθ,bτθ∈ℬϕ\(∀τ≥0\),\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi^\{\\theta\}\)\\gtrsim 2^\{d\},\\quad D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi^\{\\theta\}\)\\lesssim d,\\quad\\text\{and \}s\_\{\\tau\}^\{\\theta\},b\_\{\\tau\}^\{\\theta\}\\in\\mathcal\{B\}\_\{\\phi\}\\quad\(\\forall\\tau\\geq 0\),\(923\)where the implied constants depend only onη\\eta\.

###### Proof\.

By mollification, if we denotem2:=∫ℝr2​η​\(r\)​𝑑r≤δ2m\_\{2\}:=\\int\_\{\\mathbb\{R\}\}r^\{2\}\\eta\(r\)dr\\leq\\delta^\{2\}thenφ⁡\(t\)=0\\varphi\(t\)=0for allt≤−δt\\leq\-\\delta, while fort≥δt\\geq\\deltawe have

φ⁡\(t\)\\displaystyle\\varphi\(t\)=∫ℝ\(t−r\)3​η​\(r\)​𝑑r\\displaystyle=\\int\_\{\\mathbb\{R\}\}\(t\-r\)^\{3\}\\eta\(r\)dr\(924\)=t3​∫ℝη⁡\(r\)​𝑑r−3​t2​∫rr​η​\(r\)​𝑑r\+3​t​∫ℝr2​η​\(r\)​𝑑r−∫ℝr3​η​\(r\)​𝑑r\\displaystyle=t^\{3\}\\int\_\{\\mathbb\{R\}\}\\eta\(r\)dr\-3t^\{2\}\\int\_\{r\}r\\eta\(r\)dr\+3t\\int\_\{\\mathbb\{R\}\}r^\{2\}\\eta\(r\)dr\-\\int\_\{\\mathbb\{R\}\}r^\{3\}\\eta\(r\)dr\(925\)=t3\+3​t​m2\\displaystyle=t^\{3\}\+3tm\_\{2\}\(926\)sinceη\\etais even\. We define the scalar functionω∈𝒩φ,10\\omega\\in\\mathcal\{N\}\_\{\\varphi,10\}using 10 neurons in the following way:

ω⁡\(t\):=B⁡\(t−12\)−B⁡\(t\+12\),B⁡\(t\):=16​∑j=04\(−1\)j​\(4j\)​φ​\(t\+2−j\)\.\\omega\(t\):=B\\left\(t\-\\frac\{1\}\{2\}\\right\)\-B\\left\(t\+\\frac\{1\}\{2\}\\right\),\\quad B\(t\):=\\frac\{1\}\{6\}\\sum\_\{j=0\}^\{4\}\(\-1\)^\{j\}\{4\\choose j\}\\varphi\(t\+2\-j\)\.\(927\)Note thatω∈Cc∞​\(ℝ\)\\omega\\in C\_\{c\}^\{\\infty\}\(\{\\mathbb\{R\}\}\)\(sinceB⁡\(⋅\)B\(\\cdot\)is a mollification of the discrete fourth derivative ofReLU3\\operatorname\{ReLU\}^\{3\}, which is compactly supported\), and in particular‖ω‖∞<∞\\\|\\omega\\\|\_\{\\infty\}<\\infty\. Furthermore, since the unsmoothed discrete derivative equals54​t−t3\\frac\{5\}\{4\}t\-t^\{3\}near zero, convolution with the even mollifier implies that

ω⁡\(t\)=\(54−3​m2\)​t−t3\(∀\|t\|<12−δ\)\.\\omega\(t\)=\\left\(\\frac\{5\}\{4\}\-3m\_\{2\}\\right\)t\-t^\{3\}\\quad\\left\(\\forall\|t\|<\\frac\{1\}\{2\}\-\\delta\\right\)\.\(928\)Thus,ω⁡\(0\)=0\\omega\(0\)=0,ω′​\(0\)=54−3​m2\>0\\omega^\{\\prime\}\(0\)=\\frac\{5\}\{4\}\-3m\_\{2\}\>0,ω′′​\(0\)=0\\omega^\{\\prime\\prime\}\(0\)=0, andω′′′​\(0\)=−6≠0\\omega^\{\\prime\\prime\\prime\}\(0\)=\-6\\neq 0, i\.e\.ω\\omegasatisfies \([888](https://arxiv.org/html/2610.10875#A1.E888)\)\.

Now, takeR:=2​‖ω‖∞\+1\>2​‖ω‖∞\+δR:=2\\\|\\omega\\\|\_\{\\infty\}\+1\>2\\\|\\omega\\\|\_\{\\infty\}\+\\deltaand set

σ:=233​R2−4​m2,r=R\+σ2\.\\sigma:=\\sqrt\{\\frac\{23\}\{3\}R^\{2\}\-4m\_\{2\}\},\\quad r=\\frac\{R\+\\sigma\}\{2\}\.\(929\)Define the scalar functionh∈𝒩φ,6h\\in\\mathcal\{N\}\_\{\\varphi,6\}using 6 neurons in the following way:

h⁡\(u\):=G⁡\(u\)−G⁡\(−u\)3​R​σ,G⁡\(u\):=2​φ​\(u−r\)−3​φ​\(u−r−R\)\+φ⁡\(u−r−2​R\)\.h\(u\):=\\frac\{G\(u\)\-G\(\-u\)\}\{3R\\sigma\},\\quad G\(u\):=2\\varphi\(u\-r\)\-3\\varphi\(u\-r\-R\)\+\\varphi\(u\-r\-2R\)\.\(930\)Sinceσ\>R\\sigma\>Rwe haver\>Rr\>R, which means for every\|u\|≤2​‖ω‖∞\|u\|\\leq 2\\\|\\omega\\\|\_\{\\infty\}we knowG⁡\(u\)=0G\(u\)=0and soh⁡\(u\)=0h\(u\)=0\. Also, sinceG′G^\{\\prime\}is the mollification of

OPEN3​\(2​max⁡\{0,u−r\}\)2−3​max⁡\{0,u−r−R\}2\+max⁡\{0,u−r−2​R\}2\),3\(2\\max\\\{0,u\-r\\\}\)^\{2\}\-3\\max\\\{0,u\-r\-R\\\}^\{2\}\+\\max\\\{0,u\-r\-2R\\\}^\{2\}\),\(931\)which is everywhere nonnegative, we knowG′≥0G^\{\\prime\}\\geq 0\. Foru≥r\+2​R\+δu\\geq r\+2R\+\\deltawe knowG⁡\(−u\)=0G\(\-u\)=0and we may expand the cubic tails to find that ifv=u−rv=u\-rthen

G⁡\(u\)\\displaystyle G\(u\)=2​v3\+6​v​m2−3​\(v−R\)3−9​\(v−R\)​m2\+\(v−2​R\)3\+3​\(v−2​R\)​m2\\displaystyle=2v^\{3\}\+6vm\_\{2\}\-3\(v\-R\)^\{3\}\-9\(v\-R\)m\_\{2\}\+\(v\-2R\)^\{3\}\+3\(v\-2R\)m\_\{2\}\(932\)=2​v3−3​\(v−R\)3\+\(v−2​R\)3\+3​m2​\(2​v−3​\(v−R\)\+\(v−2​R\)\)\\displaystyle=2v^\{3\}\-3\(v\-R\)^\{3\}\+\(v\-2R\)^\{3\}\+3m\_\{2\}\(2v\-3\(v\-R\)\+\(v\-2R\)\)\(933\)=3​R​v2\+3​R2​v−5​R3\+3​m2​R\\displaystyle=3Rv^\{2\}\+3R^\{2\}v\-5R^\{3\}\+3m\_\{2\}R\(934\)=3​R​\(\(u−r\)2\+R⁡\(u−r\)−53​R2\+m2\)\\displaystyle=3R\\left\(\(u\-r\)^\{2\}\+R\(u\-r\)\-\\frac\{5\}\{3\}R^\{2\}\+m\_\{2\}\\right\)\(935\)=3​R​\(u2−σ​u\),\\displaystyle=3R\(u^\{2\}\-\\sigma u\),\(936\)where we used that2​r−R=σ2r\-R=\\sigmaandr2−R​r=53​R2−m2r^\{2\}\-Rr=\\frac\{5\}\{3\}R^\{2\}\-m\_\{2\}\. By oddness, this implies

u\+h⁡\(u\)=u​\|u\|σ\(∀\|u\|≥r\+2​R\+δ\)\.u\+h\(u\)=\\frac\{u\|u\|\}\{\\sigma\}\\quad\(\\forall\|u\|\\geq r\+2R\+\\delta\)\.\(937\)
We taken=d−1n=d\-1,L=d−2L=d\-2, and choose ResNet parametersθ=\(g,χ,\(hℓ,aℓ,bℓ\)ℓ=1L,v\)\\theta=\(g,\\chi,\(h\_\{\\ell\},a\_\{\\ell\},b\_\{\\ell\}\)\_\{\\ell=1\}^\{L\},v\)according to the following:

1. \(i\)We takeg=Λω​ωg=\\Lambda\_\{\\omega\}\\omegaandχ=ω\\chi=\\omega, whereΛω\\Lambda\_\{\\omega\}is defined in Lemma[28](https://arxiv.org/html/2610.10875#Thmlemma28)\. Note thatω∈Cc∞​\(ℝ\)\\omega\\in C^\{\\infty\}\_\{c\}\(\{\\mathbb\{R\}\}\)satisfies \([888](https://arxiv.org/html/2610.10875#A1.E888)\) and can be made with 10 neurons with activationφ\\varphi, i\.e\.ω∈𝒩φ,10\\omega\\in\\mathcal\{N\}\_\{\\varphi,10\}\.
2. \(ii\)For eachℓ∈\{3,…,L\}\\ell\\in\\\{3,\\ldots,L\\\}we takehℓ=0h\_\{\\ell\}=0, and we takeh1=h2=hh\_\{1\}=h\_\{2\}=has defined in \([930](https://arxiv.org/html/2610.10875#A1.E930)\); this can be made with 6 neurons with activationφ\\varphi, i\.e\.h1,h2∈𝒩φ,6h\_\{1\},h\_\{2\}\\in\\mathcal\{N\}\_\{\\varphi,6\}\. Note that\(Id\+h2\)∘\(Id\+h1\)\(\\operatorname\{Id\}\+h\_\{2\}\)\\circ\(\\operatorname\{Id\}\+h\_\{1\}\)is an increasingC∞C^\{\\infty\}diffeomorphism which satisfies the tail condition \([920](https://arxiv.org/html/2610.10875#A1.E920)\) withp=4p=4\.
3. \(iii\)For eachℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\}we takeaℓ=eℓa\_\{\\ell\}=e\_\{\\ell\}andbℓ=eℓ\+1b\_\{\\ell\}=e\_\{\\ell\+1\}, along withv=eL\+1v=e\_\{L\+1\}\.

The resulting mapTResNetθ:ℝd→ℝd\\operatorname\{TResNet\}\_\{\\theta\}:\{\\mathbb\{R\}\}^\{d\}\\to\{\\mathbb\{R\}\}^\{d\}is a ResNet of depth2​L\+1=2​d−32L\+1=2d\-3which uses two\-layer blocks with activationφ\\varphi: the first two have width 16, and all subsequent ones have width 10\. The assumptions of Lemmas[28](https://arxiv.org/html/2610.10875#Thmlemma28)and[29](https://arxiv.org/html/2610.10875#Thmlemma29)are both satisfied, and soℰℬϕ​\(πθ\)≥Bω2​2L=Bω24​2d≳2d\\mathcal\{E\}\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\pi^\{\\theta\}\)\\geq B\_\{\\omega\}^\{2\}2^\{L\}=\\frac\{B\_\{\\omega\}^\{2\}\}\{4\}2^\{d\}\\gtrsim 2^\{d\}whilesτθ,bτθ∈ℬϕs\_\{\\tau\}^\{\\theta\},b\_\{\\tau\}^\{\\theta\}\\in\\mathcal\{B\}\_\{\\phi\}for allτ≥0\\tau\\geq 0\(uniformly on compact OU time intervals\)\.

We finish by constructing a Barron flow transportingγd\\gamma\_\{d\}toπθ\\pi^\{\\theta\}which realizesDℬϕ​\(γd,πθ\)≲dD\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{d\},\\pi^\{\\theta\}\)\\lesssim d\. Since the ResNet layers in thezzcoordinate which computeΨθ\\Psi^\{\\theta\}are orthogonal shears \(as is the readout layer\), we can represent them by an ODE with drifts given by the layers themselves at cost≲d\\lesssim d; this flow computesΨθ\\Psi^\{\\theta\}, writes the readout, and uncomputesΨθ\\Psi^\{\\theta\}\. Since we can prepend a Barron flow on theuucoordinate before this, the claim follows if we can show that

Dℬϕ​\(γ1,ψ\#​γ1\)<∞,ψ:=\(Id\+h\)∘\(Id\+h\)\.D\_\{\\mathcal\{B\}\_\{\\phi\}\}\(\\gamma\_\{1\},\\psi\_\{\\\#\}\\gamma\_\{1\}\)<\\infty,\\quad\\psi:=\(\\operatorname\{Id\}\+h\)\\circ\(\\operatorname\{Id\}\+h\)\.\(938\)It suffices to exhibit a finite\-cost flow implementingF=Id\+hF=\\operatorname\{Id\}\+h, which we will run twice\. To start, note that the ODE

u˙=−12​u3\\dot\{u\}=\-\\frac\{1\}\{2\}u^\{3\}\(939\)has a time\-1 mapC:ℝ→\(−1,1\)C:\{\\mathbb\{R\}\}\\to\(\-1,1\)given by the smooth increasing diffeomorphism

C⁡\(u\)=u1\+u2\.C\(u\)=\\frac\{u\}\{\\sqrt\{1\+u^\{2\}\}\}\.\(940\)If we choose a smooth cutoff0≤ζ≤10\\leq\\zeta\\leq 1which vanishes on\(−∞,1/4\]\(\-\\infty,1/4\]and equals 1 on\[1/2,∞\)\[1/2,\\infty\)and define the velocity

v⁡\(u\):=u2​\(ζ⁡\(u\)−ζ⁡\(−u\)\),v\(u\):=u^\{2\}\(\\zeta\(u\)\-\\zeta\(\-u\)\),\(941\)then the ODEu˙=v⁡\(u\)\\dot\{u\}=v\(u\)has a time\-1 mapE:\(−1,1\)→ℝE:\(\-1,1\)\\to\{\\mathbb\{R\}\}which is an odd increasing diffeomorphism with

E⁡\(u\)=u1−u\(∀u∈\[1/2,1\)\)\.E\(u\)=\\frac\{u\}\{1\-u\}\\quad\(\\forall u\\in\[1/2,1\)\)\.\(942\)Indeed, on this region the ODE isu˙=u2\\dot\{u\}=u^\{2\}which is well\-posed \(since the blowup time is1/u\>11/u\>1\), and for otheru∈\[0,1/2\)u\\in\[0,1/2\)comparison withu˙=u2\\dot\{u\}=u^\{2\}ensures existence through time 1\. By oddness, the same is true for negative initial conditions\. Now, we define the smooth increasing diffeomorphismK:=E−1∘F∘C−1:\(−1,1\)→\(−1,1\)K:=E^\{\-1\}\\circ F\\circ C^\{\-1\}:\(\-1,1\)\\to\(\-1,1\)\. Foruusufficiently close to 1, the quadratic tails ofFFgive

F⁡\(C−1​\(u\)\)=u2σ⁡\(1−u2\),K⁡\(u\)=u2σ\+\(1−σ\)​u2\.F\(C^\{\-1\}\(u\)\)=\\frac\{u^\{2\}\}\{\\sigma\(1\-u^\{2\}\)\},\\quad K\(u\)=\\frac\{u^\{2\}\}\{\\sigma\+\(1\-\\sigma\)u^\{2\}\}\.\(943\)Hence,KKextends smoothly to 1 withK⁡\(1\)=1K\(1\)=1andK′​\(1\)=2​σ\>0K^\{\\prime\}\(1\)=2\\sigma\>0\. By oddness, the same holds at−1\-1\. Thus,K⁡\(±1\)=±1K\(\\pm 1\)=\\pm 1andmin\[−1,1\]⁡K′\>0\\min\_\{\[\-1,1\]\}K^\{\\prime\}\>0\. Defining the linear interpolation

Kt​\(u\):=\(1−t\)​u\+t​K​\(u\),wt​\(u\):=\(K−Id\)​\(Kt−1​\(u\)\),K\_\{t\}\(u\):=\(1\-t\)u\+tK\(u\),\\quad w\_\{t\}\(u\):=\(K\-\\operatorname\{Id\}\)\(K\_\{t\}^\{\-1\}\(u\)\),\(944\)we find that∂tKt​\(u\)=wt​\(Kt​\(u\)\)\\partial\_\{t\}K\_\{t\}\(u\)=w\_\{t\}\(K\_\{t\}\(u\)\)and thatwt​\(u\)w\_\{t\}\(u\)is jointly smooth in\(t,u\)\(t,u\)with fixed compact support\. In particular, using theReLU3\\operatorname\{ReLU\}^\{3\}representation

f⁡\(u\)=16​∫ℝϕ⁡\(u−a\)​f\(4\)​\(a\)​𝑑af\(u\)=\\frac\{1\}\{6\}\\int\_\{\\mathbb\{R\}\}\\phi\(u\-a\)f^\{\(4\)\}\(a\)da\(945\)forffwhich vanish on some\(−∞,u0\]\(\-\\infty,u\_\{0\}\]with compactly supported fourth derivative, we may extendwtw\_\{t\}to all ofℝ\{\\mathbb\{R\}\}in a way that

supt∈\[0,1\]‖wt‖ℬϕ≤16​supt∈\[0,1\]∫ℝ\(1\+\|u\|\)3​\|∂u4wt​\(u\)\|​𝑑u<∞\.\\sup\_\{t\\in\[0,1\]\}\\\|w\_\{t\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\leq\\frac\{1\}\{6\}\\sup\_\{t\\in\[0,1\]\}\\int\_\{\\mathbb\{R\}\}\(1\+\|u\|\)^\{3\}\|\\partial\_\{u\}^\{4\}w\_\{t\}\(u\)\|du<\\infty\.\(946\)The flow \([939](https://arxiv.org/html/2610.10875#A1.E939)\) definingCChas theReLU3\\operatorname\{ReLU\}^\{3\}representationϕ⁡\(−u\)−ϕ⁡\(u\)2\\frac\{\\phi\(\-u\)\-\\phi\(u\)\}\{2\}\. Lastly, the flow \([941](https://arxiv.org/html/2610.10875#A1.E941)\) definingEEhasv⁡\(u\)=q⁡\(u\)−q⁡\(−u\)v\(u\)=q\(u\)\-q\(\-u\)withq⁡\(u\):=u2​ζ​\(u\)q\(u\):=u^\{2\}\\zeta\(u\)\. Therefore,q⁡\(u\)=0q\(u\)=0foru≤1/4u\\leq 1/4andq⁡\(u\)=u2q\(u\)=u^\{2\}foru≥1/2u\\geq 1/2, which meansq\(4\)q^\{\(4\)\}is smooth and supported on\[1/4,1/2\]\[1/4,1/2\]\. Using the representation \([945](https://arxiv.org/html/2610.10875#A1.E945)\), this implies thatqqhas finite Barron norm which meansvvdoes too\. SinceF=E∘K∘CF=E\\circ K\\circ C, the result follows by concatenating these Barron flows\. ∎

#### A\.8\.2Approximation lower bounds

We will apply Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9)for a shallow approximation lower bound of the OU scoressτθ,bτθs\_\{\\tau\}^\{\\theta\},b\_\{\\tau\}^\{\\theta\}\. To do so, we require a lower bound on the Hermite width envelopemΨθm\_\{\\Psi\_\{\\theta\}\}, which is defined in terms of spectral tails of the Hermite modes \(see Definition[8](https://arxiv.org/html/2610.10875#Thmdefinition8)\)\.

First, we demonstrate that three\-layer neural networks of the form \([947](https://arxiv.org/html/2610.10875#A1.E947)\) produce heavy spectral tails in high Hermite degrees\. The mechanism is Hermite branching: the nonlinear Hermite modes ofgggenerate many independent tensor factors along the orthogonal feature directions, producing high\-rank Hermite matricizations\. A heavy Hermite tail ofχ\\chiensures that these high\-rank obstructions occur at high Hermite degrees, where the rank can be large\. Recalling the definition ofaΨa\_\{\\Psi\}from Definition[8](https://arxiv.org/html/2610.10875#Thmdefinition8)and thatHej\\operatorname\{He\}\_\{j\}denote the monic probabilist’s Hermite polynomials, a precise statement is given below:

###### Lemma 30\(Hermite branching lemma\)\.

Letg:ℝ→ℝg:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be Lipschitz, centered \(i\.e\.𝔼G∼γ1​\[g​\(G\)\]=0\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[g\(G\)\]=0\), and non\-affine inL2​\(γ1\)L^\{2\}\(\\gamma\_\{1\}\)\(i\.e\.‖g‖L2​\(γ1\)\>\|𝔼G∼γ1​\[G​g​\(G\)\]\|\\\|g\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}\>\|\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[Gg\(G\)\]\|\)\. Letχ:ℝ→ℝ\\chi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be bounded and Lipschitz\. FixL∈ℕL\\in\{\\mathbb\{N\}\}and an orthonormal basis\{e0,…,eL\}\\\{e\_\{0\},\\ldots,e\_\{L\}\\\}ofℝL\+1\{\\mathbb\{R\}\}^\{L\+1\}, and defineFL:ℝL\+1→ℝF\_\{L\}:\{\\mathbb\{R\}\}^\{L\+1\}\\to\{\\mathbb\{R\}\}by

FL​\(z\):=χ⁡\(z0\+1L​∑ℓ=1Lg⁡\(zℓ\)\)\.F\_\{L\}\(z\):=\\chi\\left\(z\_\{0\}\+\\frac\{1\}\{\\sqrt\{L\}\}\\sum\_\{\\ell=1\}^\{L\}g\(z\_\{\\ell\}\)\\right\)\.\(947\)For positive integersK1≤K2K\_\{1\}\\leq K\_\{2\}we define the quantity

𝒱χ,g​\(K1,K2\):=∑j=K1K21j\!​𝔼G∼γ1​\[χ⁡\(λg​G\)​Hej⁡\(G\)\]2,λg:=1\+‖g‖L2​\(γ1\)2\.\\mathcal\{V\}\_\{\\chi,g\}\(K\_\{1\},K\_\{2\}\):=\\sum\_\{j=K\_\{1\}\}^\{K\_\{2\}\}\\frac\{1\}\{j\!\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\\left\[\\chi\(\\sqrt\{\\lambda\_\{g\}\}G\)\\operatorname\{He\}\_\{j\}\(G\)\\right\]^\{2\},\\quad\\lambda\_\{g\}:=\{1\+\\\|g\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}^\{2\}\}\.\(948\)Then, there are somebg,cg∈\(0,∞\)b\_\{g\},c\_\{g\}\\in\(0,\\infty\)andKg∈ℕK\_\{g\}\\in\{\\mathbb\{N\}\}\(depending only ongg\) s\.t\. for anyKg≤K1≤K2K\_\{g\}\\leq K\_\{1\}\\leq K\_\{2\}there is someL0=L0​\(g,χ,K1,K2\)L\_\{0\}=L\_\{0\}\(g,\\chi,K\_\{1\},K\_\{2\}\)s\.t\.

aFL​\(m,τ\)≥12​e−bg​K2​τ​𝒱χ,g​\(K1,K2\)a\_\{F\_\{L\}\}\(m,\\tau\)\\geq\\frac\{1\}\{2\}e^\{\-b\_\{g\}K\_\{2\}\\tau\}\\sqrt\{\\mathcal\{V\}\_\{\\chi,g\}\(K\_\{1\},K\_\{2\}\)\}\(949\)for everyL≥L0L\\geq L\_\{0\}, everyτ≥0\\tau\\geq 0, and every integer0≤m≤Lcg​K10\\leq m\\leq L^\{c\_\{g\}K\_\{1\}\}\.

We defer the proof of Lemma[30](https://arxiv.org/html/2610.10875#Thmlemma30), which uses a probabilistic construction to control the crossing behavior of matricized Hermite tensors, to the end of this subsection\. Combining this result with Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9)yields a robust approximation lower bound for the scoressτId,FL,bτId,FLs\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\},b\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\}along the OU flow initialized from the triangular shear measureπId,FL∈𝒫⁡\(ℝL\+2\)\\pi^\{\\operatorname\{Id\},F\_\{L\}\}\\in\\mathcal\{P\}\(\{\\mathbb\{R\}\}^\{L\+2\}\)\.

Note thatΨθ=FL\\Psi\_\{\\theta\}=F\_\{L\}andψθ=Id\\psi\_\{\\theta\}=\\operatorname\{Id\}\(up to a rotation\) in the case wheren=L\+1n=L\+1and the parametersθ=\(g,χ,\(hℓ,aℓ,bℓ\)ℓ=1L,v\)\\theta=\(g,\\chi,\(h\_\{\\ell\},a\_\{\\ell\},b\_\{\\ell\}\)\_\{\\ell=1\}^\{L\},v\)are s\.t\.\{a1,…,aL,v\}\\\{a\_\{1\},\\ldots,a\_\{L\},v\\\}is an orthonormal basis ofℝn\{\\mathbb\{R\}\}^\{n\},bℓ=1L​vb\_\{\\ell\}=\\frac\{1\}\{\\sqrt\{L\}\}v, andhℓ=0h\_\{\\ell\}=0\. For suchθ\\theta, we have thatπθ,sτθ,bτθ\\pi^\{\\theta\},s^\{\\theta\}\_\{\\tau\},b^\{\\theta\}\_\{\\tau\}equalπId,FL,sτId,FL,bτId,FL\\pi^\{\\operatorname\{Id\},F\_\{L\}\},s\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\},b\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\}\. Hence, even though the mapFLF\_\{L\}is expressed in \([947](https://arxiv.org/html/2610.10875#A1.E947)\) in a parallelized form as a three\-layer network, this same construction can be viewed as a deep narrow triangular ResNet in the sense of Definition[10](https://arxiv.org/html/2610.10875#Thmdefinition10)\.

###### Corollary 3\(Shallow approximation width lower bound\)\.

Letg,χ:ℝ→ℝg,\\chi:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}satisfy the assumptions of Lemma[30](https://arxiv.org/html/2610.10875#Thmlemma30)\. Suppose further that there are constantsc,K0∈\(1,∞\)c,K\_\{0\}\\in\(1,\\infty\)and a functionV:ℕ→\(0,∞\)V:\{\\mathbb\{N\}\}\\to\(0,\\infty\)for which

𝒱χ,g​\(K,⌊c​K⌋\)≥V⁡\(K\)\(∀K≥K0\)\.\\mathcal\{V\}\_\{\\chi,g\}\(K,\\lfloor cK\\rfloor\)\\geq V\(K\)\\quad\(\\forall K\\geq K\_\{0\}\)\.\(950\)For everyε\>0\\varepsilon\>0, we denote the quantity

K⁡\(ε\):=max⁡\{K≥K0:V⁡\(K\)≥Cχ,g​ε2\},K\(\\varepsilon\):=\\max\\left\\\{K\\geq K\_\{0\}:\\;V\(K\)\\geq C\_\{\\chi,g\}\\varepsilon^\{2\}\\right\\\},\(951\)whereCχ,g:=64​Q‖χ‖∞,12​\(1\+Lip⁡\(χ\)2​\(1\+Lip⁡\(g\)2\)\)2C\_\{\\chi,g\}:=64Q\_\{\\\|\\chi\\\|\_\{\\infty\},1\}^\{2\}\\left\(1\+\\operatorname\{Lip\}\(\\chi\)^\{2\}\(1\+\\operatorname\{Lip\}\(g\)^\{2\}\)\\right\)^\{2\}and our convention here is thatmax⁡∅=0\\max\\emptyset=0\.

Then, there are constantsε0,c1,c2∈\(0,∞\)\\varepsilon\_\{0\},c\_\{1\},c\_\{2\}\\in\(0,\\infty\)depending only ong,χ,V,c,K0g,\\chi,V,c,K\_\{0\}s\.t\. the following holds: for eachε∈\(0,ε0\]\\varepsilon\\in\(0,\\varepsilon\_\{0\}\]there is someL0=L0​\(g,χ,ε\)L\_\{0\}=L\_\{0\}\(g,\\chi,\\varepsilon\)s\.t\. for everyL≥L0L\\geq L\_\{0\}, everyτ∈\[0,c1​min⁡\{ε,1/K⁡\(ε\)\}\]\\tau\\in\[0,c\_\{1\}\\min\\\{\\varepsilon,1/K\(\\varepsilon\)\\\}\], and every two\-layer neural networks^τ:ℝL\+2→ℝL\+2\\hat\{s\}\_\{\\tau\}:\{\\mathbb\{R\}\}^\{L\+2\}\\to\{\\mathbb\{R\}\}^\{L\+2\}of widthmmwith

‖sτId,FL−s^τ‖L2​\(ρτId,FL,ℝL\+2\)≤ε\\\|s\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\}\-\\hat\{s\}\_\{\\tau\}\\\|\_\{L^\{2\}\\left\(\\rho\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\};\{\\mathbb\{R\}\}^\{L\+2\}\\right\)\}\\leq\\varepsilon\(952\)we must have

m≥Lc2​K​\(ε\)and‖s^τ‖ℬϕ≥Dϕ,χ​ε​Lc2​K​\(ε\)/2∀ϕ​satisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2),m\\geq L^\{c\_\{2\}K\(\\varepsilon\)\}\\quad\\text\{and\}\\quad\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\geq D\_\{\\phi,\\chi\}\\varepsilon L^\{c\_\{2\}K\(\\varepsilon\)/2\}\\quad\\forall\\phi\\text\{ satisfying Assumption \\ref\{assumption:activationfunciton\}\},\(953\)whereDϕ,χ\>0D\_\{\\phi,\\chi\}\>0is a constant depending only onϕ\\phiandχ\\chi\.

The same statements hold withsτId,FLs\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\}replaced bybτId,FLb\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\}\. Moreover, if we setn=L\+1n=L\+1andθ=\(g,χ,\(hℓ,aℓ,bℓ\)ℓ=1L,v\)\\theta=\(g,\\chi,\(h\_\{\\ell\},a\_\{\\ell\},b\_\{\\ell\}\)\_\{\\ell=1\}^\{L\},v\)is s\.t\.

\{a1,…,aL,v\}⊆ℝnis orthonormal,bℓ=1Lv,hℓ=0\(∀ℓ∈\{1,…,L\}\),\\\{a\_\{1\},\\ldots,a\_\{L\},v\\\}\\subseteq\{\\mathbb\{R\}\}^\{n\}\\text\{ is orthonormal\},\\quad b\_\{\\ell\}=\\frac\{1\}\{\\sqrt\{L\}\}v,\\quad h\_\{\\ell\}=0\\quad\(\\forall\\ell\\in\\\{1,\\ldots,L\\\}\),\(954\)then the same statements hold withsτId,FLs\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\}replaced by either ofsτθs\_\{\\tau\}^\{\\theta\},bτθb\_\{\\tau\}^\{\\theta\}andρτId,FL\\rho\_\{\\tau\}^\{\\operatorname\{Id\},F\_\{L\}\}replaced byρτθ\\rho\_\{\\tau\}^\{\\theta\}\.

###### Proof\.

SinceΨθ=FL\\Psi\_\{\\theta\}=F\_\{L\}andψθ=Id\\psi\_\{\\theta\}=\\operatorname\{Id\}under \([954](https://arxiv.org/html/2610.10875#A1.E954)\), it suffices to prove the claim forFLF\_\{L\}\. DenoteQ:=Q‖χ‖∞,1Q:=Q\_\{\\\|\\chi\\\|\_\{\\infty\},1\}forQβ,σQ\_\{\\beta,\\sigma\}from Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9)along withΛ:=Lip⁡\(χ\)​1\+Lip⁡\(g\)2\\Lambda:=\\operatorname\{Lip\}\(\\chi\)\\sqrt\{1\+\\operatorname\{Lip\}\(g\)^\{2\}\}\. Then, for everyLLby the chain rule we have

‖FL‖∞≤‖χ‖∞,Lip⁡\(FL\)≤Λ\.\\\|F\_\{L\}\\\|\_\{\\infty\}\\leq\\\|\\chi\\\|\_\{\\infty\},\\quad\\operatorname\{Lip\}\(F\_\{L\}\)\\leq\\Lambda\.\(955\)Chooseε0\\varepsilon\_\{0\}sufficiently small thatK⁡\(ε\)≥max⁡\{Kg,⌈K0⌉\}K\(\\varepsilon\)\\geq\\max\\\{K\_\{g\},\\lceil K\_\{0\}\\rceil\\\}for allε∈\(0,ε0\]\\varepsilon\\in\(0,\\varepsilon\_\{0\}\], which is possible sinceVVis strictly positive\. For anyε∈\(0,ε0\]\\varepsilon\\in\(0,\\varepsilon\_\{0\}\]by definition ofK⁡\(ε\)K\(\\varepsilon\)we have

𝒱χ,g​\(K⁡\(ε\),⌊c​K​\(ε\)⌋\)≥V⁡\(K⁡\(ε\)\)≥Cχ,g​ε2=64​Q2​\(1\+Λ2\)2​ε2\.\\mathcal\{V\}\_\{\\chi,g\}\(K\(\\varepsilon\),\\lfloor cK\(\\varepsilon\)\\rfloor\)\\geq V\(K\(\\varepsilon\)\)\\geq C\_\{\\chi,g\}\\varepsilon^\{2\}=64Q^\{2\}\(1\+\\Lambda^\{2\}\)^\{2\}\\varepsilon^\{2\}\.\(956\)Applying Lemma[30](https://arxiv.org/html/2610.10875#Thmlemma30)withK1=K⁡\(ε\)K\_\{1\}=K\(\\varepsilon\)andK2=⌊c​K​\(ε\)⌋K\_\{2\}=\\lfloor cK\(\\varepsilon\)\\rfloor, we obtain someL0=L0​\(g,χ,ε\)L\_\{0\}=L\_\{0\}\(g,\\chi,\\varepsilon\)s\.t\. for allL≥L0L\\geq L\_\{0\}and all integers0≤r≤Lcg​K​\(ε\)0\\leq r\\leq L^\{c\_\{g\}K\(\\varepsilon\)\}we have

aFL​\(r,τ\)≥12​e−bg​c​K​\(ε\)​τ​𝒱χ,g​\(K⁡\(ε\),⌊c​K​\(ε\)⌋\)≥4​Q​\(1\+Λ2\)​e−bg​c​K​\(ε\)​τ​ε\.a\_\{F\_\{L\}\}\(r,\\tau\)\\geq\\frac\{1\}\{2\}e^\{\-b\_\{g\}cK\(\\varepsilon\)\\tau\}\\sqrt\{\\mathcal\{V\}\_\{\\chi,g\}\(K\(\\varepsilon\),\\lfloor cK\(\\varepsilon\)\\rfloor\)\}\\geq 4Q\(1\+\\Lambda^\{2\}\)e^\{\-b\_\{g\}cK\(\\varepsilon\)\\tau\}\\varepsilon\.\(957\)Choosingc1≤min⁡\{1,log⁡\(2\)/\(bg​c\)\}c\_\{1\}\\leq\\min\\\{1,\\log\(2\)/\(b\_\{g\}c\)\\\}, ifτ∈\[0,c1​min⁡\{ε,1/K⁡\(ε\)\}\]\\tau\\in\[0,c\_\{1\}\\min\\\{\\varepsilon,1/K\(\\varepsilon\)\\\}\]thene−bg​c​K​\(ε\)​τ≥1/2e^\{\-b\_\{g\}cK\(\\varepsilon\)\\tau\}\\geq 1/2andτ≤ε\\tau\\leq\\varepsilon\. This means that

aFL​\(r,τ\)≥2​Q​\(1\+Λ2\)​ε\.a\_\{F\_\{L\}\}\(r,\\tau\)\\geq 2Q\(1\+\\Lambda^\{2\}\)\\varepsilon\.\(958\)SinceQ⁡\(2​ε\+Λ2​τ\)≤2​Q​\(1\+Λ2\)​εQ\(2\\varepsilon\+\\Lambda^\{2\}\\tau\)\\leq 2Q\(1\+\\Lambda^\{2\}\)\\varepsilon, it follows from the definition of the Hermite width envelope \(see Definition[8](https://arxiv.org/html/2610.10875#Thmdefinition8)\) that

mFL​\(Q⁡\(ε\+Λ2​τ\),τ\),mFL​\(2​Q​\(ε\+Λ2​τ\),τ\)≥⌊Lcg​K​\(ε\)⌋\.m\_\{F\_\{L\}\}\\left\(Q\(\\varepsilon\+\\Lambda^\{2\}\\tau\),\\;\\tau\\right\),m\_\{F\_\{L\}\}\\left\(2Q\(\\varepsilon\+\\Lambda^\{2\}\\tau\),\\;\\tau\\right\)\\geq\\lfloor L^\{c\_\{g\}K\(\\varepsilon\)\}\\rfloor\.\(959\)Applying Proposition[9](https://arxiv.org/html/2610.10875#Thmproposition9)withψ=Id\\psi=\\operatorname\{Id\},Ψ=FL\\Psi=F\_\{L\}, andn=L\+1n=L\+1, we find that the width lower bound holds withc2=cgc\_\{2\}=c\_\{g\}and that for anyϕ\\phisatisfying Assumption[ϕ](https://arxiv.org/html/2610.10875#Thmassumption0.Thmassumptionalt2)we have

‖s^τ‖ℬϕ≥Kϕ,‖χ‖∞,1​ε​⌊Lcg​K​\(ε\)⌋\\\|\\hat\{s\}\_\{\\tau\}\\\|\_\{\\mathcal\{B\}\_\{\\phi\}\}\\geq K\_\{\\phi,\\\|\\chi\\\|\_\{\\infty\},1\}\\varepsilon\\sqrt\{\\lfloor L^\{c\_\{g\}K\(\\varepsilon\)\}\\rfloor\}\(960\)with the same holding forbτψ,Ψb\_\{\\tau\}^\{\\psi,\\Psi\}\. IncreasingL0L\_\{0\}if needed, we may take⌊Lcg​K​\(ε\)⌋≥Lcg​K​\(ε\)/2\\lfloor L^\{c\_\{g\}K\(\\varepsilon\)\}\\rfloor\\geq L^\{c\_\{g\}K\(\\varepsilon\)\}/2, in which case the Barron norm claim holds withDϕ,χ=Kϕ,‖χ‖∞,1/2D\_\{\\phi,\\chi\}=K\_\{\\phi,\\\|\\chi\\\|\_\{\\infty\},1\}/\\sqrt\{2\}\. ∎

We finish these approximation lower bounds by getting concrete scalings ofV⁡\(K\)V\(K\)for our activations of interest\. Importantly,VVdepends only onχ\\chiand on the Gaussian variance ofgg:

###### Lemma 31\(Hermite decay for ReLU difference and tanh\)\.

Letg:ℝ→ℝg:\{\\mathbb\{R\}\}\\to\{\\mathbb\{R\}\}be centered withλg:=1\+‖g‖L2​\(γ1\)2<∞\\lambda\_\{g\}:=1\+\\\|g\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}^\{2\}<\\infty\. Then, there is someK0∈\(0,∞\)K\_\{0\}\\in\(0,\\infty\)depending only onλg\\lambda\_\{g\}s\.t\.𝒱χ,g​\(K,2​K\)≥V⁡\(K\)\\mathcal\{V\}\_\{\\chi,g\}\(K,2K\)\\geq V\(K\)for allK≥K0K\\geq K\_\{0\}, where we defineV:ℕ→\(0,∞\)V:\{\\mathbb\{N\}\}\\to\(0,\\infty\)as follows:

1. \(i\)Ifχ⁡\(t\)=max⁡\{0,t\}−max⁡\{0,t−a\}a\\chi\(t\)=\\frac\{\\max\\\{0,t\\\}\-\\max\\\{0,t\-a\\\}\}\{a\}for some fixeda∈\(0,∞\)a\\in\(0,\\infty\)then we takeV\(K\)=bK−3/2V\(K\)=bK^\{\-3/2\}for someb∈\(0,∞\)b\\in\(0,\\infty\)depending onλg,a\\lambda\_\{g\},a\.
2. \(ii\)Ifχ⁡\(t\)=tanh⁡\(t\)\\chi\(t\)=\\tanh\(t\)then we takeV⁡\(K\)=b​e−π​K/λgV\(K\)=be^\{\-\\pi\\sqrt\{K/\\lambda\_\{g\}\}\}for someb∈\(0,∞\)b\\in\(0,\\infty\)depending onλg\\lambda\_\{g\}\.

###### Proof\.

Fixλ\>0\\lambda\>0\(which we take asλ=λg\\lambda=\\lambda\_\{g\}\), and let us denoteαj:=𝔼G∼γ1​\[χ⁡\(λ​G\)​Hej⁡\(G\)\]/j\!\\alpha\_\{j\}:=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[\\chi\(\\sqrt\{\\lambda\}G\)\\operatorname\{He\}\_\{j\}\(G\)\]/\\sqrt\{j\!\}\.

For the ReLU difference, we simply apply\[[37](https://arxiv.org/html/2610.10875#bib.bib19), Lemma A\.3\]to see that

1j\!​𝔼G∼γ1​\[max⁡\{0,G−w\}​Hej⁡\(G\)\]=e−w2/2Hej−2\(w\)2​π​j\!\(∀j≥2\)\\frac\{1\}\{\\sqrt\{j\!\}\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\\left\[\\max\\\{0,G\-w\\\}\\operatorname\{He\}\_\{j\}\(G\)\\right\]=\\frac\{e^\{\-w^\{2\}/2\}\\operatorname\{He\}\_\{j\-2\}\(w\)\}\{\\sqrt\{2\\pi j\!\}\}\\quad\(\\forall j\\geq 2\)\(961\)for anyw∈ℝw\\in\{\\mathbb\{R\}\}\. By homogeneity and linearity, for everyj≥2j\\geq 2this gives

αj=λa​2​π​j\!\[Hej−2\(0\)−e−a2/\(2λ\)Hej−2\(aλ\)\]\.\\alpha\_\{j\}=\\frac\{\\sqrt\{\\lambda\}\}\{a\\sqrt\{2\\pi j\!\}\}\\left\[\\operatorname\{He\}\_\{j\-2\}\(0\)\-e^\{\-a^\{2\}/\(2\\lambda\)\}\\operatorname\{He\}\_\{j\-2\}\\left\(\\frac\{a\}\{\\sqrt\{\\lambda\}\}\\right\)\\right\]\.\(962\)If we denoteu\(t\):=e−t2/4Hen⁡\(t\)Hen⁡\(0\)u\(t\):=e^\{\-t^\{2\}/4\}\\frac\{\\operatorname\{He\}\_\{n\}\(t\)\}\{\\operatorname\{He\}\_\{n\}\(0\)\}for an evenn∈ℕ∪\{0\}n\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}, then the Hermite equationHen′′−t​Hen′\+n​Hen=0\\operatorname\{He\}\_\{n\}^\{\\prime\\prime\}\-t\\operatorname\{He\}\_\{n\}^\{\\prime\}\+n\\operatorname\{He\}\_\{n\}=0implies that

u′′​\(t\)\+\(n\+12−t24\)​u​\(t\)=0,u⁡\(0\)=1,u′​\(0\)=0\.u^\{\\prime\\prime\}\(t\)\+\\left\(n\+\\frac\{1\}\{2\}\-\\frac\{t^\{2\}\}\{4\}\\right\)u\(t\)=0,\\quad u\(0\)=1,\\quad u^\{\\prime\}\(0\)=0\.\(963\)Consequently, for allt≥0t\\geq 0we have

dd​t​\[u′​\(t\)2\+\(n\+12−t24\)​u​\(t\)2\]\\displaystyle\\frac\{d\}\{dt\}\\left\[u^\{\\prime\}\(t\)^\{2\}\+\\left\(n\+\\frac\{1\}\{2\}\-\\frac\{t^\{2\}\}\{4\}\\right\)u\(t\)^\{2\}\\right\]=2​u′​\(t\)​\(u′′​\(t\)\+\(n\+12−t24\)​u​\(t\)\)−t2​u​\(t\)2\\displaystyle=2u^\{\\prime\}\(t\)\\left\(u^\{\\prime\\prime\}\(t\)\+\\left\(n\+\\frac\{1\}\{2\}\-\\frac\{t^\{2\}\}\{4\}\\right\)u\(t\)\\right\)\-\\frac\{t\}\{2\}u\(t\)^\{2\}\(964\)=−t2​u​\(t\)2≤0\.\\displaystyle=\-\\frac\{t\}\{2\}u\(t\)^\{2\}\\leq 0\.\(965\)Hence,

\(n\+12−t24\)​u​\(t\)2≤u′​\(t\)2\+\(n\+12−t24\)​u​\(t\)2≤n\+12\\left\(n\+\\frac\{1\}\{2\}\-\\frac\{t^\{2\}\}\{4\}\\right\)u\(t\)^\{2\}\\leq u^\{\\prime\}\(t\)^\{2\}\+\\left\(n\+\\frac\{1\}\{2\}\-\\frac\{t^\{2\}\}\{4\}\\right\)u\(t\)^\{2\}\\leq n\+\\frac\{1\}\{2\}\(966\)and so whenevern\+1/2\>t2/4n\+1/2\>t^\{2\}/4we have

\|e−t2/2Hen\(t\)Hen⁡\(0\)\|≤e−t2/4n\+12n\+12−t24\.\\left\|\\frac\{e^\{\-t^\{2\}/2\}\\operatorname\{He\}\_\{n\}\(t\)\}\{\\operatorname\{He\}\_\{n\}\(0\)\}\\right\|\\leq e^\{\-t^\{2\}/4\}\\sqrt\{\\frac\{n\+\\frac\{1\}\{2\}\}\{n\+\\frac\{1\}\{2\}\-\\frac\{t^\{2\}\}\{4\}\}\}\.\(967\)Therefore, asn=j−2→∞n=j\-2\\to\\inftywe find that

lim supj→∞j​even\|e−a2/\(2λ\)Hej−2\(a/λ\)Hej−2⁡\(0\)\|≤e−a2/\(4λ\)<1\.\\limsup\_\{\\begin\{subarray\}\{c\}j\\to\\infty\\\\ j\\text\{ even\}\\end\{subarray\}\}\\left\|\\frac\{e^\{\-a^\{2\}/\(2\\lambda\)\}\\operatorname\{He\}\_\{j\-2\}\(a/\\sqrt\{\\lambda\}\)\}\{\\operatorname\{He\}\_\{j\-2\}\(0\)\}\\right\|\\leq e^\{\-a^\{2\}/\(4\\lambda\)\}<1\.\(968\)In particular, asj→∞j\\to\\inftywithj=2​r\+2j=2r\+2even we have the asymptotic

αj2≍λ,aHej−2⁡\(0\)2j\!=\(2​rr\)4r​\(2​r\+1\)​\(2​r\+2\)≍1π​r​\(2​r\+1\)​\(2​r\+2\)≍j−5/2,\\alpha\_\{j\}^\{2\}\\asymp\_\{\\lambda,a\}\\frac\{\\operatorname\{He\}\_\{j\-2\}\(0\)^\{2\}\}\{j\!\}=\\frac\{\{2r\\choose r\}\}\{4^\{r\}\(2r\+1\)\(2r\+2\)\}\\asymp\\frac\{1\}\{\\sqrt\{\\pi r\}\(2r\+1\)\(2r\+2\)\}\\asymp j^\{\-5/2\},\(969\)where we used that\(2​rr\)/4r≍1/π​r\{2r\\choose r\}/4^\{r\}\\asymp 1/\\sqrt\{\\pi r\}asr→∞r\\to\\infty\. Hence,αj2≍j−5/2\\alpha\_\{j\}^\{2\}\\asymp j^\{\-5/2\}forjjlarge and even\. Summing over a band gives

𝒱χ,g\(K,2K\)≳λ,a∑K≤j≤2​Kj​evenj−5/2≍K−3/2\.\\mathcal\{V\}\_\{\\chi,g\}\(K,2K\)\\gtrsim\_\{\\lambda,a\}\\sum\_\{\\begin\{subarray\}\{c\}K\\leq j\\leq 2K\\\\ j\\text\{ even\}\\end\{subarray\}\}j^\{\-5/2\}\\asymp K^\{\-3/2\}\.\(970\)Note that by \([962](https://arxiv.org/html/2610.10875#A1.E962)\) we know that\|αj\|\>0\|\\alpha\_\{j\}\|\>0for allj≥2j\\geq 2even, and so by decreasingbbour result holds even up toK0=2K\_\{0\}=2\.

Fortanh\\tanh, we replicate the argument of Lemma[12](https://arxiv.org/html/2610.10875#Thmlemma12)\. Note thatαj=0\\alpha\_\{j\}=0for evenjj\. By the same Fourier transform formula as in Lemma[12](https://arxiv.org/html/2610.10875#Thmlemma12)we have

\(tanh\(λ⋅\)\)′^​\(ξ\)=π​ξ/λsinh⁡\(π​ξ/\(2​λ\)\),\\widehat\{\(\\tanh\(\\sqrt\{\\lambda\}\\cdot\)\)^\{\\prime\}\}\(\\xi\)=\\frac\{\\pi\\xi/\\sqrt\{\\lambda\}\}\{\\sinh\(\\pi\\xi/\(2\\sqrt\{\\lambda\}\)\)\},\(971\)and so the same integration by parts and Parseval argument implies that for oddjj

αj=\(−1\)\(j−1\)/2λ​j\!​∫0∞ξje−ξ2/2sinh⁡\(π​ξ/\(2​λ\)\)​𝑑ξ\.\\alpha\_\{j\}=\\frac\{\(\-1\)^\{\(j\-1\)/2\}\}\{\\sqrt\{\\lambda j\!\}\}\\int\_\{0\}^\{\\infty\}\\frac\{\\xi^\{j\}e^\{\-\\xi^\{2\}/2\}\}\{\\sinh\(\\pi\\xi/\(2\\sqrt\{\\lambda\}\)\)\}d\\xi\.\(972\)We again setF⁡\(ξ\):=j​log⁡ξ−ξ22−π​ξ2​λF\(\\xi\):=j\\log\\xi\-\\frac\{\\xi^\{2\}\}\{2\}\-\\frac\{\\pi\\xi\}\{2\\sqrt\{\\lambda\}\}\. Splitting atξ=1\\xi=1, we find that as oddj→∞j\\to\\inftythere is the asymptotic

log\|αj\|=−12log\(λj\!\)\+log∫1∞eF⁡\(ξ\)dξ\+Oλ\(1\)\.\\log\|\\alpha\_\{j\}\|=\-\\frac\{1\}\{2\}\\log\(\\lambda j\!\)\+\\log\\int\_\{1\}^\{\\infty\}e^\{F\(\\xi\)\}d\\xi\+O\_\{\\lambda\}\(1\)\.\(973\)The same Taylor estimates as in Lemma[12](https://arxiv.org/html/2610.10875#Thmlemma12)\(with2​q2qreplaced byjj\) apply becauseF′​\(j\)=−π2​λF^\{\\prime\}\(\\sqrt\{j\}\)=\-\\frac\{\\pi\}\{2\\sqrt\{\\lambda\}\}andF′′​\(ξ\)=−jξ2−1≤−1F^\{\\prime\\prime\}\(\\xi\)=\-\\frac\{j\}\{\\xi^\{2\}\}\-1\\leq\-1for allξ\\xi\. In particular,

log⁡\(∫1∞eF⁡\(ξ\)​𝑑ξ\)=F⁡\(j\)\+Oλ​\(1\)\\log\\left\(\\int\_\{1\}^\{\\infty\}e^\{F\(\\xi\)\}d\\xi\\right\)=F\(\\sqrt\{j\}\)\+O\_\{\\lambda\}\(1\)\(974\)and consequently

log⁡\|αj\|=j2​log⁡j−j2−π2​λ​j−12​log⁡\(λ​j\!\)\+Oλ​\(1\)\.\\log\|\\alpha\_\{j\}\|=\\frac\{j\}\{2\}\\log j\-\\frac\{j\}\{2\}\-\\frac\{\\pi\}\{2\\sqrt\{\\lambda\}\}\\sqrt\{j\}\-\\frac\{1\}\{2\}\\log\(\\lambda j\!\)\+O\_\{\\lambda\}\(1\)\.\(975\)Applying Stirling, we get

log⁡\|αj\|=−π2​λ​j−14​log⁡j\+Oλ​\(1\)\\log\|\\alpha\_\{j\}\|=\-\\frac\{\\pi\}\{2\\sqrt\{\\lambda\}\}\\sqrt\{j\}\-\\frac\{1\}\{4\}\\log j\+O\_\{\\lambda\}\(1\)\(976\)or in other words that

αj2≍λj−1/2e−π​j/λ\(j→∞odd\)\.\\alpha\_\{j\}^\{2\}\\asymp\_\{\\lambda\}j^\{\-1/2\}e^\{\-\\pi\\sqrt\{j/\\lambda\}\}\\quad\(j\\to\\infty\\text\{ odd\}\)\.\(977\)We sum over the band to find that

𝒱tanh,g\(K,2K\)≍λ∑K≤j≤2​Kj​oddj−1/2e−π​j/λ≍λe−π​K/λ,\\mathcal\{V\}\_\{\\tanh,g\}\(K,2K\)\\asymp\_\{\\lambda\}\\sum\_\{\\begin\{subarray\}\{c\}K\\leq j\\leq 2K\\\\ j\\text\{ odd\}\\end\{subarray\}\}j^\{\-1/2\}e^\{\-\\pi\\sqrt\{j/\\lambda\}\}\\asymp\_\{\\lambda\}e^\{\-\\pi\\sqrt\{K/\\lambda\}\},\(978\)where the last asymptotic is due to the integral equality

∫K2​Kx−1/2e−π​x/λdx=2​λπ\(e−π​K/λ−e−π​2​K/λ\)\.\\int\_\{K\}^\{2K\}x^\{\-1/2\}e^\{\-\\pi\\sqrt\{x/\\lambda\}\}dx=\\frac\{2\\sqrt\{\\lambda\}\}\{\\pi\}\\left\(e^\{\-\\pi\\sqrt\{K/\\lambda\}\}\-e^\{\-\\pi\\sqrt\{2K/\\lambda\}\}\\right\)\.\(979\)
∎

We can also apply the lower bounds from Section[A\.7\.3](https://arxiv.org/html/2610.10875#A1.SS7.SSS3)to verify that early\-stopping cannot sidestep our approximation lower bound: one must model the scores in the window of OU timesτ∈\[0,c1min\{ε,1/K\(ε\)\]\}\\tau\\in\[0,c\_\{1\}\\min\\\{\\varepsilon,1/K\(\\varepsilon\)\]\\\}in which Corollary[3](https://arxiv.org/html/2610.10875#Thmcorollary3)applies\.

###### Lemma 32\(Early\-stopping lower bound\)\.

Assume the setting of Corollary[3](https://arxiv.org/html/2610.10875#Thmcorollary3)\. Then, for everya∈\(0,∞\)a\\in\(0,\\infty\)there exist constantsε0,b1,b2∈\(0,∞\)\\varepsilon\_\{0\},b\_\{1\},b\_\{2\}\\in\(0,\\infty\)s\.t\. for everyε∈\(0,ε0\]\\varepsilon\\in\(0,\\varepsilon\_\{0\}\]there is anL0​\(ε\)L\_\{0\}\(\\varepsilon\)for which the following holds:

For everyL≥L0​\(ε\)L\\geq L\_\{0\}\(\\varepsilon\)and everyτ0≥a​min⁡\{ε,1/K⁡\(ε\)\}\\tau\_\{0\}\\geq a\\min\\\{\\varepsilon,1/K\(\\varepsilon\)\\\}, if we denoteπ=πId,FL\\pi=\\pi^\{\\operatorname\{Id\},F\_\{L\}\}andρτ0=ρτ0Id,FL\\rho\_\{\\tau\_\{0\}\}=\\rho\_\{\\tau\_\{0\}\}^\{\\operatorname\{Id\},F\_\{L\}\}then we have the lower bounds

min⁡\{TV⁡\(π,ρτ0\),W1​\(π,ρτ0\)\}\\displaystyle\\min\\left\\\{\\operatorname\{TV\}\(\\pi,\\rho\_\{\\tau\_\{0\}\}\),\\;W\_\{1\}\(\\pi,\\rho\_\{\\tau\_\{0\}\}\)\\right\\\}≥b1​min⁡\{ε,1/K⁡\(ε\)\},\\displaystyle\\geq b\_\{1\}\\min\\\{\\varepsilon,1/K\(\\varepsilon\)\\\},\(980\)min⁡\{KL\(π∥ρτ0\),KL\(ρτ0∥π\)\}\\displaystyle\\min\\left\\\{\\sqrt\{\\operatorname\{KL\}\(\\pi\\\|\\rho\_\{\\tau\_\{0\}\}\)\},\\;\\sqrt\{\\operatorname\{KL\}\(\\rho\_\{\\tau\_\{0\}\}\\\|\\pi\)\}\\right\\\}≥b2​ε\.\\displaystyle\\geq b\_\{2\}\\varepsilon\.\(981\)The constants depend only ona,g,χ,c,K0,Va,g,\\chi,c,K\_\{0\},V\.

###### Proof\.

Chooseε0≤1\\varepsilon\_\{0\}\\leq 1sufficiently small thatK⁡\(ε\)≥KgK\(\\varepsilon\)\\geq K\_\{g\}\. Applying Lemma[30](https://arxiv.org/html/2610.10875#Thmlemma30)at time 0, both to the band\[K⁡\(ε\),⌊c​K​\(ε\)⌋\]\[K\(\\varepsilon\),\\lfloor cK\(\\varepsilon\)\\rfloor\]and to one fixed admissible band with positive mass, gives

aFL​\(⌊Lcg​K​\(ε\)⌋,0\)≳ε,aFL​\(0,0\)=‖FL‖L2​\(γL\+1\)≳1a\_\{F\_\{L\}\}\(\\lfloor L^\{c\_\{g\}K\(\\varepsilon\)\}\\rfloor,0\)\\gtrsim\\varepsilon,\\quad a\_\{F\_\{L\}\}\(0,0\)=\\\|F\_\{L\}\\\|\_\{L^\{2\}\(\\gamma\_\{L\+1\}\)\}\\gtrsim 1\(982\)forL≥L0​\(ε\)L\\geq L\_\{0\}\(\\varepsilon\)\. Here and below, the implied constants are independent ofε,L,τ0\\varepsilon,L,\\tau\_\{0\}\. Since‖FL‖∞≤‖χ‖∞\\\|F\_\{L\}\\\|\_\{\\infty\}\\leq\\\|\\chi\\\|\_\{\\infty\}, we have

MFL≥e−∥χ∥2∞/4∥FL∥L2​\(γL\+1\)2≳1,M\_\{F\_\{L\}\}\\geq e^\{\-\\\|\\chi\\\|^\{2\}\_\{\\infty\}/4\}\\\|F\_\{L\}\\\|\_\{L^\{2\}\(\\gamma\_\{L\+1\}\)\}^\{2\}\\gtrsim 1,\(983\)and forL≥2L\\geq 2we know

ℓL\+1​\(⌊Lcg​K​\(ε\)⌋\)≥2​cg​K​\(ε\)​log⁡Llog⁡\(L\+1\)≥cg​K​\(ε\)\.\\ell\_\{L\+1\}\(\\lfloor L^\{c\_\{g\}K\(\\varepsilon\)\}\\rfloor\)\\geq 2c\_\{g\}K\(\\varepsilon\)\\frac\{\\log L\}\{\\log\(L\+1\)\}\\geq c\_\{g\}K\(\\varepsilon\)\.\(984\)Lemma[25](https://arxiv.org/html/2610.10875#Thmlemma25)therefore implies that

min⁡\{TV⁡\(π,ρτ0\),W1​\(π,ρτ0\)\}\\displaystyle\\min\\\{\\operatorname\{TV\}\(\\pi,\\rho\_\{\\tau\_\{0\}\}\),W\_\{1\}\(\\pi,\\rho\_\{\\tau\_\{0\}\}\)\\\}≳1−e−2​τ0\\displaystyle\\gtrsim 1\-e^\{\-2\\tau\_\{0\}\}\(985\)≥1−e−2​a​min⁡\{ε,1/K⁡\(ε\)\}\\displaystyle\\geq 1\-e^\{\-2a\\min\\\{\\varepsilon,1/K\(\\varepsilon\)\\\}\}\(986\)≥\(1−e−2​a\)​min⁡\{ε,1/K⁡\(ε\)\}\.\\displaystyle\\geq\(1\-e^\{\-2a\}\)\\min\\\{\\varepsilon,1/K\(\\varepsilon\)\\\}\.\(987\)Similarly, Lemma[26](https://arxiv.org/html/2610.10875#Thmlemma26)implies that withDτ0D\_\{\\tau\_\{0\}\}as the KL in either direction,

Dτ0\\displaystyle\\sqrt\{D\_\{\\tau\_\{0\}\}\}≳max⁡\{1−e−τ0,ε⁡\(1−e−cg​K​\(ε\)​τ0\)\}\.\\displaystyle\\gtrsim\\max\\left\\\{1\-e^\{\-\\tau\_\{0\}\},\\;\\varepsilon\\left\(1\-e^\{\-c\_\{g\}K\(\\varepsilon\)\\tau\_\{0\}\}\\right\)\\right\\\}\.\(988\)Ifε≤1/K⁡\(ε\)\\varepsilon\\leq 1/K\(\\varepsilon\)then1−e−τ0≥1−e−a​ε≥\(1−e−a\)​ε1\-e^\{\-\\tau\_\{0\}\}\\geq 1\-e^\{\-a\\varepsilon\}\\geq\(1\-e^\{\-a\}\)\\varepsilon, while ifε\>1/K⁡\(ε\)\\varepsilon\>1/K\(\\varepsilon\)then we have that1−e−cg​K​\(ε\)​τ0≥1−e−cg​a1\-e^\{\-c\_\{g\}K\(\\varepsilon\)\\tau\_\{0\}\}\\geq 1\-e^\{\-c\_\{g\}a\}\. In either case,Dτ0≳ε\\sqrt\{D\_\{\\tau\_\{0\}\}\}\\gtrsim\\varepsilon\. ∎

We now turn to the proof of Lemma[30](https://arxiv.org/html/2610.10875#Thmlemma30)\. In a few words, we perform a probabilistic counting argument using the number of crossings of the Hermite coefficient tensors ofFLF\_\{L\}\. This allows us to lower bound the Hermite rank in a way which translates to our width lower bounds\.

###### Proof of Lemma[30](https://arxiv.org/html/2610.10875#Thmlemma30)\.

Let us denoteVg:=‖g‖L2​\(γ1\)2V\_\{g\}:=\\\|g\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}^\{2\}andβj:=𝔼G∼γ1​\[g⁡\(G\)​hj​\(G\)\]\\beta\_\{j\}:=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[g\(G\)h\_\{j\}\(G\)\]forj∈ℕ∪\{0\}j\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\}, wherehj:=Hej/j\!h\_\{j\}:=\\operatorname\{He\}\_\{j\}/\\sqrt\{j\!\}are theL2​\(γ1\)L^\{2\}\(\\gamma\_\{1\}\)\-orthonormal Hermite polynomials\. Becauseggis centered we knowβ0=0\\beta\_\{0\}=0and thereforeVg=∑j∈ℕβj2V\_\{g\}=\\sum\_\{j\\in\{\\mathbb\{N\}\}\}\\beta\_\{j\}^\{2\}\. Moreover, sinceβ1=𝔼G∼γ1​\[G​g​\(G\)\]\\beta\_\{1\}=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[Gg\(G\)\]then

0<𝔳g:=∑j≥2βj28​λg≤18,λg:=1\+Vg,0<\\mathfrak\{v\}\_\{g\}:=\\frac\{\\sum\_\{j\\geq 2\}\\beta\_\{j\}^\{2\}\}\{8\\lambda\_\{g\}\}\\leq\\frac\{1\}\{8\},\\quad\\lambda\_\{g\}:=1\+V\_\{g\},\(989\)sinceggis non\-affine inL2​\(γ1\)L^\{2\}\(\\gamma\_\{1\}\)\. The heart of this proof is to show that for all⌈4/𝔳g⌉≤K1≤K2\\lceil 4/\\mathfrak\{v\}\_\{g\}\\rceil\\leq K\_\{1\}\\leq K\_\{2\}, we have the estimate

lim infL→∞∑k=K1⌈Bg​K2⌉∑j\>L⌊𝔳g​K1⌋−1σj​\(Mk​\(FL\)\)2≥1348​𝒱χ,g​\(K1,K2\)\\liminf\_\{L\\to\\infty\}\\sum\_\{k=K\_\{1\}\}^\{\\lceil B\_\{g\}K\_\{2\}\\rceil\}\\;\\sum\_\{j\>L^\{\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}\\geq\\frac\{13\}\{48\}\\mathcal\{V\}\_\{\\chi,g\}\(K\_\{1\},K\_\{2\}\)\(990\)whereMk​\(FL\)M\_\{k\}\(F\_\{L\}\)are the balanced matricizations of the Hermite tensor coefficients ofFLF\_\{L\}\(see Definition[8](https://arxiv.org/html/2610.10875#Thmdefinition8)\) and we denote

Bg:=8​\(1\+Lip⁡\(g\)\)2λg\.B\_\{g\}:=\\frac\{8\(1\+\\operatorname\{Lip\}\(g\)\)^\{2\}\}\{\\lambda\_\{g\}\}\.\(991\)Indeed, from \([990](https://arxiv.org/html/2610.10875#A1.E990)\) we find anL0=L0​\(g,χ,K1,K2\)L\_\{0\}=L\_\{0\}\(g,\\chi,K\_\{1\},K\_\{2\}\)s\.t\.

∑k=K1⌈Bg​K2⌉∑j\>L⌊𝔳g​K1⌋−1σj​\(Mk​\(FL\)\)2≥14​𝒱χ,g​\(K1,K2\)\(∀L≥L0\)\.\\sum\_\{k=K\_\{1\}\}^\{\\lceil B\_\{g\}K\_\{2\}\\rceil\}\\;\\sum\_\{j\>L^\{\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}\\geq\\frac\{1\}\{4\}\\mathcal\{V\}\_\{\\chi,g\}\(K\_\{1\},K\_\{2\}\)\\quad\(\\forall L\\geq L\_\{0\}\)\.\(992\)Moreover,K1≥⌈4/𝔳g⌉K\_\{1\}\\geq\\lceil 4/\\mathfrak\{v\}\_\{g\}\\rceilimplies⌊𝔳g​K1⌋−1≥𝔳g​K1−2≥cg​K1\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\-1\\geq\\mathfrak\{v\}\_\{g\}K\_\{1\}\-2\\geq c\_\{g\}K\_\{1\}withcg:=𝔳g/2c\_\{g\}:=\\mathfrak\{v\}\_\{g\}/2, and so for everym≤Lcg​K1m\\leq L^\{c\_\{g\}K\_\{1\}\}andτ≥0\\tau\\geq 0we find

aFL​\(m,τ\)2≥14​e−2​\(⌈Bg​K2⌉\+1\)​τ​𝒱χ,g​\(K1,K2\)\.a\_\{F\_\{L\}\}\(m,\\tau\)^\{2\}\\geq\\frac\{1\}\{4\}e^\{\-2\(\\lceil B\_\{g\}K\_\{2\}\\rceil\+1\)\\tau\}\\mathcal\{V\}\_\{\\chi,g\}\(K\_\{1\},K\_\{2\}\)\.\(993\)Takingbg:=Bg\+1b\_\{g\}:=B\_\{g\}\+1yields the claim\. The proof of \([990](https://arxiv.org/html/2610.10875#A1.E990)\) proceeds in the following high\-level steps:

1. 1\.For eachk∈ℕk\\in\{\\mathbb\{N\}\}, we fix a balanced cutSk⊆\{1,…,k\}S\_\{k\}\\subseteq\\\{1,\\ldots,k\\\}with\|Sk\|=⌊k/2⌋\|S\_\{k\}\|=\\lfloor k/2\\rfloorto define the matricizationMk​\(FL\)M\_\{k\}\(F\_\{L\}\)\. We show that our Hermite spectral tail of interest is controlled by the elements of the Hermite tensor coefficients whose indices cross the cut\. More precisely, for any integerR\>1R\>1we show that lim infL→∞∑j\>LR−1σj​\(Mk​\(FL\)\)2≥lim infL→∞∑𝐩∈ℐkCSk​\(𝐩\)≥R\|d𝐩,L\|2\(∀k∈ℕ\),\\liminf\_\{L\\to\\infty\}\\sum\_\{j\>L^\{R\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}\\geq\\liminf\_\{L\\to\\infty\}\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}\|d\_\{\\mathbf\{p\},L\}\|^\{2\}\\quad\(\\forall k\\in\{\\mathbb\{N\}\}\),\(994\)whereℐk⊆\{0,…,k\}k\\mathcal\{I\}\_\{k\}\\subseteq\\\{0,\\ldots,k\\\}^\{k\}is a certain collection ofkk\-tuples,CSk​\(𝐩\)C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)denotes the number of distinct positive entries of𝐩\\mathbf\{p\}appearing on both sides of the cutSkS\_\{k\}, andd𝐩,Ld\_\{\\mathbf\{p\},L\}is proportional to the Hermite coefficient tensor entryTk​\(FL\)​\[𝐩\]T\_\{k\}\(F\_\{L\}\)\[\\mathbf\{p\}\]\.
2. 2\.By tensor symmetry, the fraction of entries satisfying the crossing condition can be computed using a uniformly random balanced cut\. Using this, we show that limL→∞∑𝐩∈ℐkCSk​\(𝐩\)≥R\|d𝐩,L\|2=∑J=1kαJ2⋅ℙJ​\(N=k,CS≥R\)\(∀k∈ℕ\)\\lim\_\{L\\to\\infty\}\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}\|d\_\{\\mathbf\{p\},L\}\|^\{2\}=\\sum\_\{J=1\}^\{k\}\\alpha\_\{J\}^\{2\}\\cdot\\mathbb\{P\}\_\{J\}\(N=k,\\;C\_\{S\}\\geq R\)\\quad\(\\forall k\\in\{\\mathbb\{N\}\}\)\(995\)with the notation αJ:=𝔼G∼γ1​\[χ⁡\(λg​G\)​hJ​\(G\)\]\.\\alpha\_\{J\}:=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\\left\[\\chi\(\\sqrt\{\\lambda\_\{g\}\}G\)h\_\{J\}\(G\)\\right\]\.\(996\)Here,ℙJ\\mathbb\{P\}\_\{J\}is an auxiliary probability law describing the Hermite degrees and uniformly random balanced cut, and underℙJ\\mathbb\{P\}\_\{J\}we denote byNNthe total degree and byCSC\_\{S\}the crossing count for a uniformly random balanced cutSS\. Note that𝒱χ,g​\(K1,K2\)=∑J=K1K2αJ2\\mathcal\{V\}\_\{\\chi,g\}\(K\_\{1\},K\_\{2\}\)=\\sum\_\{J=K\_\{1\}\}^\{K\_\{2\}\}\\alpha\_\{J\}^\{2\}\.
3. 3\.By simple probabilistic estimates and plugging in our quantities, we show that ℙJ​\(N≤Bg​K2,CS≥𝔳g​K1\)≥1348\(∀K1≤J≤K2\)\.\\mathbb\{P\}\_\{J\}\(N\\leq B\_\{g\}K\_\{2\},\\;C\_\{S\}\\geq\\mathfrak\{v\}\_\{g\}K\_\{1\}\)\\geq\\frac\{13\}\{48\}\\quad\(\\forall K\_\{1\}\\leq J\\leq K\_\{2\}\)\.\(997\)Combining this with the prior steps proves the main estimate \([990](https://arxiv.org/html/2610.10875#A1.E990)\)\.

\(Step 1\)\.Fixk∈ℕk\\in\{\\mathbb\{N\}\}and letL\>kL\>k\. Choose a subsetSk⊆\{1,…,k\}S\_\{k\}\\subseteq\\\{1,\\ldots,k\\\}of size\|Sk\|=⌊k/2⌋\|S\_\{k\}\|=\\lfloor k/2\\rfloor;SkS\_\{k\}defines the row indices of the balanced matricization andSkcS\_\{k\}^\{c\}defines the column indices\. For each tuple𝐢=\(i1,…,ik\)∈\{0,…,L\}k\\mathbf\{i\}=\(i\_\{1\},\\ldots,i\_\{k\}\)\\in\\\{0,\\ldots,L\\\}^\{k\}, we denote𝐢S=\(ir\)r∈S\\mathbf\{i\}\_\{S\}=\(i\_\{r\}\)\_\{r\\in S\}and observe that by symmetry ofTk​\(FL\)T\_\{k\}\(F\_\{L\}\)we have

Mk​\(FL\)​\[𝐢Sk,𝐢Skc\]=Tk​\(FL\)​\[𝐢\]=∏ℓ=0Lνℓ​\(𝐢\)\!k\!​𝔼Z∼γL\+1​\[FL​\(Z\)​∏ℓ=0Lhνℓ​\(𝐢\)​\(Zℓ\)\],M\_\{k\}\(F\_\{L\}\)\[\\mathbf\{i\}\_\{S\_\{k\}\},\\mathbf\{i\}\_\{S\_\{k\}^\{c\}\}\]=T\_\{k\}\(F\_\{L\}\)\[\\mathbf\{i\}\]=\\sqrt\{\\frac\{\\prod\_\{\\ell=0\}^\{L\}\\nu\_\{\\ell\}\(\\mathbf\{i\}\)\!\}\{k\!\}\}\\mathbb\{E\}\_\{Z\\sim\\gamma\_\{L\+1\}\}\\left\[F\_\{L\}\(Z\)\\prod\_\{\\ell=0\}^\{L\}h\_\{\\nu\_\{\\ell\}\(\\mathbf\{i\}\)\}\(Z\_\{\\ell\}\)\\right\],\(998\)where we denoteνℓ​\(𝐢\)=\#⁡\{r:ir=ℓ\}\\nu\_\{\\ell\}\(\\mathbf\{i\}\)=\\\#\\\{r:\\;i\_\{r\}=\\ell\\\}\. We define the crossing quantity

CS​\(𝐢\):=\|\{ir:r∈S\}∩\{ir:r∈Sc\}∩\{1,…,L\}\|C\_\{S\}\(\\mathbf\{i\}\):=\\left\|\\\{i\_\{r\}:\\;r\\in S\\\}\\cap\\\{i\_\{r\}:\\;r\\in S^\{c\}\\\}\\cap\\\{1,\\ldots,L\\\}\\right\|\(999\)to be the number of distinct coordinate labels \(other than 0\) which appear in both the row and column indices for a balanced cutSS\. Now, letℐk⊆\{0,…,k\}k\\mathcal\{I\}\_\{k\}\\subseteq\\\{0,\\ldots,k\\\}^\{k\}be the set of tuples whose distinct positive labels, listed in order of first appearance, are\{1,2,…,s\}\\\{1,2,\\ldots,s\\\}for somes≤ks\\leq k\. Equivalently, we require that the distinct nonzero values of𝐩∈ℐk\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}are\{1,2,…,s\}\\\{1,2,\\ldots,s\\\}and

min⁡\{r:pr=1\}<min⁡\{r:pr=2\}<…<min⁡\{r:pr=s\}\.\\min\\\{r:\\;p\_\{r\}=1\\\}<\\min\\\{r:\\;p\_\{r\}=2\\\}<\\ldots<\\min\\\{r:\\;p\_\{r\}=s\\\}\.\(1000\)In particular, a positive entry may repeat a previously introduced label or introduce the next unused label, but it cannot skip a label\.242424By this, we mean tuples like\(0,1,1,2,0,2,3\)\(0,1,1,2,0,2,3\)or\(0,1,2,1,…\)\(0,1,2,1,\\ldots\)but not\(0,2,…\)\(0,2,\\ldots\)\.Note thatssdepends on𝐩\\mathbf\{p\}\. For every𝐩∈ℐk\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}we define two matrices of the same dimensions asMk​\(FL\)M\_\{k\}\(F\_\{L\}\)by

U𝐩,L\[𝐢Sk,𝐢Skc\]:=L−s/2∑ℓ1,…,ℓs=1L∏r=1k𝟙ir=ℓpr,U𝐩,L≠\[𝐢Sk,𝐢Skc\]:=L−s/2∑ℓ1,…,ℓs=1ℓ1,…,ℓs​distinctL∏r=1k𝟙ir=ℓpr,U\_\{\\mathbf\{p\},L\}\[\\mathbf\{i\}\_\{S\_\{k\}\},\\mathbf\{i\}\_\{S\_\{k\}^\{c\}\}\]:=L^\{\-s/2\}\\sum\_\{\\ell\_\{1\},\\ldots,\\ell\_\{s\}=1\}^\{L\}\\;\\prod\_\{r=1\}^\{k\}\{\\mathbbm\{1\}\}\_\{i\_\{r\}=\\ell\_\{p\_\{r\}\}\},\\quad U\_\{\\mathbf\{p\},L\}^\{\\neq\}\[\\mathbf\{i\}\_\{S\_\{k\}\},\\mathbf\{i\}\_\{S\_\{k\}^\{c\}\}\]:=L^\{\-s/2\}\\sum\_\{\\begin\{subarray\}\{c\}\\ell\_\{1\},\\ldots,\\ell\_\{s\}=1\\\\ \\ell\_\{1\},\\ldots,\\ell\_\{s\}\\text\{ distinct\}\\end\{subarray\}\}^\{L\}\\;\\prod\_\{r=1\}^\{k\}\{\\mathbbm\{1\}\}\_\{i\_\{r\}=\\ell\_\{p\_\{r\}\}\},\(1001\)where we denoteℓ0:=0\\ell\_\{0\}:=0\. Thus,U𝐩,L≠U\_\{\\mathbf\{p\},L\}^\{\\neq\}enforces exactly the coordinate equalities specified by𝐩\\mathbf\{p\}whereasU𝐩,LU\_\{\\mathbf\{p\},L\}also permits different positive labels of𝐩\\mathbf\{p\}to receive the same coordinate\. We set

d𝐩,L:=Ls/2​Tk​\(FL\)​\[𝐩\]\.d\_\{\\mathbf\{p\},L\}:=L^\{s/2\}T\_\{k\}\(F\_\{L\}\)\[\\mathbf\{p\}\]\.\(1002\)BecauseFLF\_\{L\}is invariant under permutations of the coordinates1,…,L1,\\ldots,L, tuples whose nonzero entries are equivalent up to relabeling have the same tensor coefficients\. Hence,

Mk​\(FL\)=∑𝐩∈ℐkd𝐩,L​U𝐩,L≠\.M\_\{k\}\(F\_\{L\}\)=\\sum\_\{\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\}d\_\{\\mathbf\{p\},L\}U\_\{\\mathbf\{p\},L\}^\{\\neq\}\.\(1003\)Fix an integerR\>1R\>1\. We decomposeMk​\(FL\)=Ak,L\+Hk,L\+Ek,LM\_\{k\}\(F\_\{L\}\)=A\_\{k,L\}\+H\_\{k,L\}\+E\_\{k,L\}using the matrices

Ak,L:=∑𝐩∈ℐkCSk​\(𝐩\)<Rd𝐩,L​U𝐩,L,Hk,L:=∑𝐩∈ℐkCSk​\(𝐩\)≥Rd𝐩,L​U𝐩,L,Ek,L:=∑𝐩∈ℐkd𝐩,L​\(U𝐩,L≠−U𝐩,L\)\.A\_\{k,L\}:=\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)<R\\end\{subarray\}\}d\_\{\\mathbf\{p\},L\}U\_\{\\mathbf\{p\},L\},\\quad H\_\{k,L\}:=\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}d\_\{\\mathbf\{p\},L\}U\_\{\\mathbf\{p\},L\},\\quad E\_\{k,L\}:=\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\end\{subarray\}\}d\_\{\\mathbf\{p\},L\}\(U\_\{\\mathbf\{p\},L\}^\{\\neq\}\-U\_\{\\mathbf\{p\},L\}\)\.\(1004\)We will now prove some bounds on these matrices\. If we denoteat:=\#⁡\{r∈Sk:pr=t\}a\_\{t\}:=\\\#\\\{r\\in S\_\{k\}:\\;p\_\{r\}=t\\\}andbt:=\#⁡\{r∈Skc:pr=t\}b\_\{t\}:=\\\#\\\{r\\in S\_\{k\}^\{c\}:\\;p\_\{r\}=t\\\}, then we find

U𝐩,L=\(e0⊗a0​\(e0⊗b0\)⊤\)⊗⨂t=1sDt,Dt:=1L​∑ℓ=1Leℓ⊗at​\(eℓ⊗bt\)⊤U\_\{\\mathbf\{p\},L\}=\\left\(e\_\{0\}^\{\\otimes a\_\{0\}\}\(e\_\{0\}^\{\\otimes b\_\{0\}\}\)^\{\\top\}\\right\)\\otimes\\bigotimes\_\{t=1\}^\{s\}D\_\{t\},\\quad D\_\{t\}:=\\frac\{1\}\{\\sqrt\{L\}\}\\sum\_\{\\ell=1\}^\{L\}e\_\{\\ell\}^\{\\otimes a\_\{t\}\}\(e\_\{\\ell\}^\{\\otimes b\_\{t\}\}\)^\{\\top\}\(1005\)up to row and column permutations, witheℓ⊗0:=1e\_\{\\ell\}^\{\\otimes 0\}:=1\. These permutations preserve rank and singular values\. Now, ifat,bt≥1a\_\{t\},b\_\{t\}\\geq 1then both families\{eℓ⊗at\}ℓ=1L\\\{e\_\{\\ell\}^\{\\otimes a\_\{t\}\}\\\}\_\{\\ell=1\}^\{L\}and\{eℓ⊗bt\}ℓ=1L\\\{e\_\{\\ell\}^\{\\otimes b\_\{t\}\}\\\}\_\{\\ell=1\}^\{L\}are orthonormal, and so the displayed expression forDtD\_\{t\}is an SVD with all singular values equal to1/L1/\\sqrt\{L\}\. If eitherat=0a\_\{t\}=0orbt=0b\_\{t\}=0, thenDtD\_\{t\}is a row or column vector of norm 1 \(since its squared norm is1L​∑ℓ=1L1=1\\frac\{1\}\{L\}\\sum\_\{\\ell=1\}^\{L\}1=1\)\. It therefore has one nonzero singular value equal to 1, as does the terme0⊗a0​\(e0⊗b0\)⊤e\_\{0\}^\{\\otimes a\_\{0\}\}\(e\_\{0\}^\{\\otimes b\_\{0\}\}\)^\{\\top\}\. We also have

CSk\(𝐩\)=\#\{t∈\{1,…,s\}:at,bt\>0\}\.C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)=\\\#\\\{t\\in\\\{1,\\ldots,s\\\}:\\;a\_\{t\},b\_\{t\}\>0\\\}\.\(1006\)Since the nonzero singular values of a Kronecker product are all the possible products of those of the factors, we find thatU𝐩,LU\_\{\\mathbf\{p\},L\}has exactlyLCSk​\(𝐩\)L^\{C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\}nonzero singular values which are all equal toL−CSk\(𝐩\)/2L^\{\-C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)/2\}\. Hence,

rank\(U𝐩,L\)=LCSk​\(𝐩\),∥U𝐩,L∥op=L−CSk\(𝐩\)/2\.\\operatorname\{rank\}\(U\_\{\\mathbf\{p\},L\}\)=L^\{C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\},\\quad\\\|U\_\{\\mathbf\{p\},L\}\\\|\_\{\\operatorname\{op\}\}=L^\{\-C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)/2\}\.\(1007\)Next, we claim that

\|d𝐩,L\|≤‖χ‖∞​Vgs/2≤‖χ‖∞​max⁡\{1,Vgk/2\}=:Bk\.\|d\_\{\\mathbf\{p\},L\}\|\\leq\\\|\\chi\\\|\_\{\\infty\}V\_\{g\}^\{s/2\}\\leq\\\|\\chi\\\|\_\{\\infty\}\\max\\\{1,V\_\{g\}^\{k/2\}\\\}=:B\_\{k\}\.\(1008\)To show this, denotej:=ν0​\(𝐩\)j:=\\nu\_\{0\}\(\\mathbf\{p\}\)andqt:=νt​\(𝐩\)≥1q\_\{t\}:=\\nu\_\{t\}\(\\mathbf\{p\}\)\\geq 1\. We start from the fact that ifZ=\(Z0,…,ZL\)∼γL\+1Z=\(Z\_\{0\},\\ldots,Z\_\{L\}\)\\sim\\gamma\_\{L\+1\}then

d𝐩,L=Ls/2​j\!​∏t=1sqt\!k\!​𝔼​\[FL​\(Z\)​hj​\(Z0\)​∏t=1shqt​\(Zt\)\]\.d\_\{\\mathbf\{p\},L\}=L^\{s/2\}\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\mathbb\{E\}\\left\[F\_\{L\}\(Z\)h\_\{j\}\(Z\_\{0\}\)\\prod\_\{t=1\}^\{s\}h\_\{q\_\{t\}\}\(Z\_\{t\}\)\\right\]\.\(1009\)If we defineQj​\(y\):=𝔼G∼γ1​\[χ⁡\(G\+y\)​hj​\(G\)\]Q\_\{j\}\(y\):=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[\\chi\(G\+y\)h\_\{j\}\(G\)\]and differentiate the convolution, we find

Qj\(s\)​\(y\)=\(j\+s\)\!j\!​𝔼G∼γ1​\[χ⁡\(G\+y\)​hj\+s​\(G\)\]Q\_\{j\}^\{\(s\)\}\(y\)=\\sqrt\{\\frac\{\(j\+s\)\!\}\{j\!\}\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[\\chi\(G\+y\)h\_\{j\+s\}\(G\)\]\(1010\)and therefore‖Qj\(s\)‖∞≤‖χ‖∞​\(j\+s\)\!j\!\\\|Q\_\{j\}^\{\(s\)\}\\\|\_\{\\infty\}\\leq\\\|\\chi\\\|\_\{\\infty\}\\sqrt\{\\frac\{\(j\+s\)\!\}\{j\!\}\}\. If we define\(Δu​Q\)​\(y\):=Q⁡\(y\+u\)−Q⁡\(y\)\(\\Delta\_\{u\}Q\)\(y\):=Q\(y\+u\)\-Q\(y\), then repeated application of the fundamental theorem of calculus yields

\(Δu1⋯ΔusQ\)\(y\)=\(∏t=1sut\)∫\[0,1\]sQ\(s\)\(y\+∑t=1sutvt\)dv,\(\\Delta\_\{u\_\{1\}\}\\cdots\\Delta\_\{u\_\{s\}\}Q\)\(y\)=\\left\(\\prod\_\{t=1\}^\{s\}u\_\{t\}\\right\)\\int\_\{\[0,1\]^\{s\}\}Q^\{\(s\)\}\\left\(y\+\\sum\_\{t=1\}^\{s\}u\_\{t\}v\_\{t\}\\right\)dv,\(1011\)which we apply toQjQ\_\{j\}to find that

∥Δu1⋯ΔusQj∥∞≤∥Qj\(s\)∥∞∏t=1s\|ut\|≤∥χ∥∞\(j\+s\)\!j\!∏t=1s\|ut\|\.\\\|\\Delta\_\{u\_\{1\}\}\\cdots\\Delta\_\{u\_\{s\}\}Q\_\{j\}\\\|\_\{\\infty\}\\leq\\\|Q\_\{j\}^\{\(s\)\}\\\|\_\{\\infty\}\\prod\_\{t=1\}^\{s\}\|u\_\{t\}\|\\leq\\\|\\chi\\\|\_\{\\infty\}\\sqrt\{\\frac\{\(j\+s\)\!\}\{j\!\}\}\\prod\_\{t=1\}^\{s\}\|u\_\{t\}\|\.\(1012\)On the other hand, denotingYL:=1L​∑ℓ=s\+1Lg⁡\(Zℓ\)Y\_\{L\}:=\\frac\{1\}\{\\sqrt\{L\}\}\\sum\_\{\\ell=s\+1\}^\{L\}g\(Z\_\{\\ell\}\)and integrating overZ0Z\_\{0\}yields

d𝐩,L\\displaystyle d\_\{\\mathbf\{p\},L\}=Ls/2​j\!​∏t=1sqt\!k\!​𝔼​\[χ⁡\(Z0\+YL\+∑t=1sg⁡\(Zt\)L\)​hj​\(Z0\)​∏t=1shqt​\(Zt\)\]\\displaystyle=L^\{s/2\}\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\mathbb\{E\}\\left\[\\chi\\left\(Z\_\{0\}\+Y\_\{L\}\+\\sum\_\{t=1\}^\{s\}\\frac\{g\(Z\_\{t\}\)\}\{\\sqrt\{L\}\}\\right\)h\_\{j\}\(Z\_\{0\}\)\\prod\_\{t=1\}^\{s\}h\_\{q\_\{t\}\}\(Z\_\{t\}\)\\right\]\(1013\)=Ls/2​j\!​∏t=1sqt\!k\!​𝔼​\[Qj​\(YL\+∑t=1sg⁡\(Zt\)L\)​∏t=1shqt​\(Zt\)\]\.\\displaystyle=L^\{s/2\}\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\mathbb\{E\}\\left\[Q\_\{j\}\\left\(Y\_\{L\}\+\\sum\_\{t=1\}^\{s\}\\frac\{g\(Z\_\{t\}\)\}\{\\sqrt\{L\}\}\\right\)\\prod\_\{t=1\}^\{s\}h\_\{q\_\{t\}\}\(Z\_\{t\}\)\\right\]\.\(1014\)Now, use the finite differences identity\(Δu1⋯ΔusQ\)\(y\)=∑A⊆\{1,…,s\}\(−1\)s−\|A\|Q\(y\+∑t∈Aut\)\(\\Delta\_\{u\_\{1\}\}\\cdots\\Delta\_\{u\_\{s\}\}Q\)\(y\)=\\sum\_\{A\\subseteq\\\{1,\\ldots,s\\\}\}\(\-1\)^\{s\-\|A\|\}Q\(y\+\\sum\_\{t\\in A\}u\_\{t\}\)along with the fact

𝔼⁡\[Qj​\(YL\+∑t∈Ag⁡\(Zt\)L\)​∏t=1shqt​\(Zt\)\]=0​ifA⊊\{1,…,s\},\\displaystyle\\mathbb\{E\}\\left\[Q\_\{j\}\\left\(Y\_\{L\}\+\\sum\_\{t\\in A\}\\frac\{g\(Z\_\{t\}\)\}\{\\sqrt\{L\}\}\\right\)\\prod\_\{t=1\}^\{s\}h\_\{q\_\{t\}\}\(Z\_\{t\}\)\\right\]=0\\text\{ if $A\\subsetneq\\\{1,\\ldots,s\\\}$\},\(1015\)which is true since for anyt∉At\\notin Awe may integrate overZtZ\_\{t\}and use𝔼⁡\[hqt​\(Zt\)\]=0\\mathbb\{E\}\[h\_\{q\_\{t\}\}\(Z\_\{t\}\)\]=0\. So, only theA=\{1,…,s\}A=\\\{1,\\ldots,s\\\}term of the finite differences identity survives, which we plug into \([1014](https://arxiv.org/html/2610.10875#A1.E1014)\) to find that

d𝐩,L\\displaystyle d\_\{\\mathbf\{p\},L\}=Ls/2j\!​∏t=1sqt\!k\!𝔼\[\(Δg⁡\(Z1\)/L⋯Δg⁡\(Zs\)/LQj\)\(YL\)∏t=1shqt\(Zt\)\]\\displaystyle=L^\{s/2\}\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\mathbb\{E\}\\left\[\\left\(\\Delta\_\{g\(Z\_\{1\}\)/\\sqrt\{L\}\}\\cdots\\Delta\_\{g\(Z\_\{s\}\)/\\sqrt\{L\}\}Q\_\{j\}\\right\)\(Y\_\{L\}\)\\prod\_\{t=1\}^\{s\}h\_\{q\_\{t\}\}\(Z\_\{t\}\)\\right\]\(1016\)and therefore

\|d𝐩,L\|\\displaystyle\|d\_\{\\mathbf\{p\},L\}\|≤‖χ‖∞​j\!​∏t=1sqt\!k\!​\(j\+s\)\!j\!​∏t=1s𝔼⁡\[\|g⁡\(Zt\)​hqt​\(Zt\)\|\]\\displaystyle\\leq\\\|\\chi\\\|\_\{\\infty\}\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\sqrt\{\\frac\{\(j\+s\)\!\}\{j\!\}\}\\prod\_\{t=1\}^\{s\}\\mathbb\{E\}\\left\[\|g\(Z\_\{t\}\)h\_\{q\_\{t\}\}\(Z\_\{t\}\)\|\\right\]\(1017\)≤‖χ‖∞​\(j\+s\)\!​∏t=1sqt\!k\!​‖g‖L2​\(γ1\)s\\displaystyle\\leq\\\|\\chi\\\|\_\{\\infty\}\\sqrt\{\\frac\{\(j\+s\)\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\\|g\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}^\{s\}\(1018\)≤‖χ‖∞​Vgs/2,\\displaystyle\\leq\\\|\\chi\\\|\_\{\\infty\}V\_\{g\}^\{s/2\},\(1019\)where we use Cauchy\-Schwarz and that‖hqt‖L2​\(γ1\)=1\\\|h\_\{q\_\{t\}\}\\\|\_\{L^\{2\}\(\\gamma\_\{1\}\)\}=1in the second line and that\(j\+s\)\!​∏t=1sqt\!k\!≤1\\sqrt\{\\frac\{\(j\+s\)\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\leq 1in the third\. This proves \([1008](https://arxiv.org/html/2610.10875#A1.E1008)\)\.

Now, since\|ℐk\|≤\(k\+1\)k\|\\mathcal\{I\}\_\{k\}\|\\leq\(k\+1\)^\{k\}we find that

rank\(Ak,L\)≤\(k\+1\)kLR−1,∥Hk,L∥op≤\(k\+1\)kBkL−R/2\.\\operatorname\{rank\}\(A\_\{k,L\}\)\\leq\(k\+1\)^\{k\}L^\{R\-1\},\\quad\\\|H\_\{k,L\}\\\|\_\{\\operatorname\{op\}\}\\leq\(k\+1\)^\{k\}B\_\{k\}L^\{\-R/2\}\.\(1020\)Also, sinceU𝐩,L−U𝐩,L≠U\_\{\\mathbf\{p\},L\}\-U\_\{\\mathbf\{p\},L\}^\{\\neq\}’s nonzero entries come from exactly the assignments with a repeated coordinate label we find

∥U𝐩,L−U𝐩,L≠∥F2=L−s\(Ls−L\(L−1\)⋯\(L−s\+1\)\)≤\(s2\)L≤\(k2\)L\.\\\|U\_\{\\mathbf\{p\},L\}\-U\_\{\\mathbf\{p\},L\}^\{\\neq\}\\\|\_\{F\}^\{2\}=L^\{\-s\}\\left\(L^\{s\}\-L\(L\-1\)\\cdots\(L\-s\+1\)\\right\)\\leq\\frac\{\{s\\choose 2\}\}\{L\}\\leq\\frac\{\{k\\choose 2\}\}\{L\}\.\(1021\)In particular,‖Ek,L‖F2≤\(k\+1\)2​k​Bk2​\(k2\)/L=Ok​\(L−1\)\\\|E\_\{k,L\}\\\|\_\{F\}^\{2\}\\leq\(k\+1\)^\{2k\}B\_\{k\}^\{2\}\{k\\choose 2\}/L=O\_\{k\}\(\{L^\{\-1\}\}\)\. If we definedm​\(B\):=∑j\>mσj​\(B\)2d\_\{m\}\(B\):=\\sqrt\{\\sum\_\{j\>m\}\\sigma\_\{j\}\(B\)^\{2\}\}and observe that this is a distance to the set of matrices of rank at mostmm\(w\.r\.t\. Frobenius norm\), then by the triangle inequality we have that

dm​\(B\)2≤\(1\+η\)​dm​\(B\+E\)2\+\(1\+η−1\)​‖E‖F2\.d\_\{m\}\(B\)^\{2\}\\leq\(1\+\\eta\)d\_\{m\}\(B\+E\)^\{2\}\+\(1\+\\eta^\{\-1\}\)\\\|E\\\|\_\{F\}^\{2\}\.\(1022\)Therefore, takingB=Ak,L\+Hk,LB=A\_\{k,L\}\+H\_\{k,L\}andE=Ek,LE=E\_\{k,L\}gives

∑j\>LR−1σj​\(Mk​\(FL\)\)2≥11\+η​∑j\>LR−1σj​\(Ak,L\+Hk,L\)2−Ok,η​\(L−1\)\.\\sum\_\{j\>L^\{R\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}\\geq\\frac\{1\}\{1\+\\eta\}\\sum\_\{j\>L^\{R\-1\}\}\\sigma\_\{j\}\(A\_\{k,L\}\+H\_\{k,L\}\)^\{2\}\-O\_\{k,\\eta\}\(L^\{\-1\}\)\.\(1023\)Using the elementary inequality

∑j\>mσj​\(A\+H\)2≥‖H‖F2−\(m\+rank⁡\(A\)\)​‖H‖op2\(∀A,H∈ℝM×M′,∀M,M′∈ℕ\),\\sum\_\{j\>m\}\\sigma\_\{j\}\(A\+H\)^\{2\}\\geq\\\|H\\\|\_\{F\}^\{2\}\-\(m\+\\operatorname\{rank\}\(A\)\)\\\|H\\\|\_\{\\operatorname\{op\}\}^\{2\}\\quad\\left\(\\forall A,H\\in\{\\mathbb\{R\}\}^\{M\\times M^\{\\prime\}\},\\;\\forall M,M^\{\\prime\}\\in\{\\mathbb\{N\}\}\\right\),\(1024\)if we takem=LR−1m=L^\{R\-1\}along withA=Ak,LA=A\_\{k,L\}andH=Hk,LH=H\_\{k,L\}then

∑j\>LR−1σj​\(Mk​\(FL\)\)2≥11\+η​‖Hk,L‖F2−Ok,η​\(L−1\)\.\\displaystyle\\sum\_\{j\>L^\{R\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}\\geq\\frac\{1\}\{1\+\\eta\}\\\|H\_\{k,L\}\\\|\_\{F\}^\{2\}\-O\_\{k,\\eta\}\(L^\{\-1\}\)\.\(1025\)TakingL→∞L\\to\\infty, we find that

lim infL→∞∑j\>LR−1σj​\(Mk​\(FL\)\)2≥lim infL→∞11\+η​‖Hk,L‖F2\.\\liminf\_\{L\\to\\infty\}\\sum\_\{j\>L^\{R\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}\\geq\\liminf\_\{L\\to\\infty\}\\frac\{1\}\{1\+\\eta\}\\\|H\_\{k,L\}\\\|\_\{F\}^\{2\}\.\(1026\)By \([1021](https://arxiv.org/html/2610.10875#A1.E1021)\), replacingU𝐩,LU\_\{\\mathbf\{p\},L\}byU𝐩,L≠U\_\{\\mathbf\{p\},L\}^\{\\neq\}inHk,LH\_\{k,L\}changes the matrix by at mostOk\(L−1/2\)O\_\{k\}\(L^\{\-1/2\}\)in Frobenius norm\. Since the matricesU𝐩,L≠U\_\{\\mathbf\{p\},L\}^\{\\neq\}have disjoint entry supports for distinct𝐩\\mathbf\{p\}, we find

‖Hk,L‖F2=∑𝐩∈ℐkCSk​\(𝐩\)≥R\|d𝐩,L\|2​L\(L−1\)⋯\(L−s\+1\)Ls\+o⁡\(1\)\\\|H\_\{k,L\}\\\|\_\{F\}^\{2\}=\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}\|d\_\{\\mathbf\{p\},L\}\|^\{2\}\\frac\{L\(L\-1\)\\cdots\(L\-s\+1\)\}\{L^\{s\}\}\+o\(1\)\(1027\)withs=maxr⁡prs=\\max\_\{r\}p\_\{r\}\. Combining this with \([1026](https://arxiv.org/html/2610.10875#A1.E1026)\), takingη↓0\\eta\\downarrow 0, and noting thatL\(L−1\)⋯\(L−s\+1\)/Ls→1L\(L\-1\)\\cdots\(L\-s\+1\)/L^\{s\}\\to 1yields

lim infL→∞∑j\>LR−1σj​\(Mk​\(FL\)\)2≥lim infL→∞∑𝐩∈ℐkCSk​\(𝐩\)≥R\|d𝐩,L\|2\.\\liminf\_\{L\\to\\infty\}\\sum\_\{j\>L^\{R\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}\\geq\\liminf\_\{L\\to\\infty\}\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}\|d\_\{\\mathbf\{p\},L\}\|^\{2\}\.\(1028\)\(Step 2\)\.We first fix𝐩\\mathbf\{p\}and compute the limit ofd𝐩,Ld\_\{\\mathbf\{p\},L\}\. Recalling our notationj=ν0​\(𝐩\)j=\\nu\_\{0\}\(\\mathbf\{p\}\),qt=νt​\(𝐩\)q\_\{t\}=\\nu\_\{t\}\(\\mathbf\{p\}\), andYL=1L​∑ℓ=s\+1Lg⁡\(Zℓ\)Y\_\{L\}=\\frac\{1\}\{\\sqrt\{L\}\}\\sum\_\{\\ell=s\+1\}^\{L\}g\(Z\_\{\\ell\}\)forZ=\(Z0,…,ZL\)∼γL\+1Z=\(Z\_\{0\},\\ldots,Z\_\{L\}\)\\sim\\gamma\_\{L\+1\}, we start from \([1016](https://arxiv.org/html/2610.10875#A1.E1016)\) and the finite difference integral representation \([1011](https://arxiv.org/html/2610.10875#A1.E1011)\) to get that

d𝐩,L\\displaystyle d\_\{\\mathbf\{p\},L\}=j\!​∏t=1sqt\!k\!​𝔼​\[\(∫\[0,1\]sQj\(s\)​\(YL\+1L​∑t=1sg⁡\(Zt\)​vt\)​𝑑v\)​∏t=1sg⁡\(Zt\)​hqt​\(Zt\)\]\.\\displaystyle=\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\mathbb\{E\}\\left\[\\left\(\\int\_\{\[0,1\]^\{s\}\}Q^\{\(s\)\}\_\{j\}\\left\(Y\_\{L\}\+\\frac\{1\}\{\\sqrt\{L\}\}\\sum\_\{t=1\}^\{s\}g\(Z\_\{t\}\)v\_\{t\}\\right\)dv\\right\)\\prod\_\{t=1\}^\{s\}g\(Z\_\{t\}\)h\_\{q\_\{t\}\}\(Z\_\{t\}\)\\right\]\.\(1029\)Noting thatQj\(s\+1\)Q\_\{j\}^\{\(s\+1\)\}is bounded \(and soQj\(s\)Q\_\{j\}^\{\(s\)\}is Lipschitz\),ggis Lipschitz, and thehqth\_\{q\_\{t\}\}are polynomials of fixed degree, we find that

d𝐩,L\\displaystyle d\_\{\\mathbf\{p\},L\}=j\!​∏t=1sqt\!k\!⋅𝔼\[\(∫\[0,1\]sQj\(s\)\(YL\)dv\)∏t=1sg\(Zt\)hqt\(Zt\)\]\+Ok\(L−1/2\)\\displaystyle=\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\cdot\\mathbb\{E\}\\left\[\\left\(\\int\_\{\[0,1\]^\{s\}\}Q^\{\(s\)\}\_\{j\}\\left\(Y\_\{L\}\\right\)dv\\right\)\\prod\_\{t=1\}^\{s\}g\(Z\_\{t\}\)h\_\{q\_\{t\}\}\(Z\_\{t\}\)\\right\]\+O\_\{k\}\(L^\{\-1/2\}\)\(1030\)=j\!​∏t=1sqt\!k\!⋅𝔼\[Qj\(s\)\(YL\)\]⋅∏t=1sβqt\+Ok\(L−1/2\),\\displaystyle=\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\cdot\\mathbb\{E\}\\left\[Q^\{\(s\)\}\_\{j\}\\left\(Y\_\{L\}\\right\)\\right\]\\cdot\\prod\_\{t=1\}^\{s\}\\beta\_\{q\_\{t\}\}\+O\_\{k\}\(L^\{\-1/2\}\),\(1031\)where the second line uses thatYLY\_\{L\}is independent ofσ⁡\(Z1,…,Zs\)\\sigma\(Z\_\{1\},\\ldots,Z\_\{s\}\)\. Sinceggis centered with finite Gaussian moments, by the central limit theorem we know thatLaw⁡\(YL\)→𝒩⁡\(0,Vg\)\\operatorname\{Law\}\(Y\_\{L\}\)\\to\\mathcal\{N\}\(0,V\_\{g\}\)narrowly asL→∞L\\to\\infty\. SinceQj\(s\)Q\_\{j\}^\{\(s\)\}is bounded and continuous, we find that

limL→∞d𝐩,L=j\!​∏t=1sqt\!k\!⋅𝔼G∼γ1​\[Qj\(s\)​\(Vg​G\)\]⋅∏t=1sβqt\.\\displaystyle\\lim\_\{L\\to\\infty\}d\_\{\\mathbf\{p\},L\}=\\sqrt\{\\frac\{j\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\cdot\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\\left\[Q\_\{j\}^\{\(s\)\}\(\\sqrt\{V\_\{g\}\}G\)\\right\]\\cdot\\prod\_\{t=1\}^\{s\}\\beta\_\{q\_\{t\}\}\.\(1032\)Convolving the two Gaussians \(whose variances add toλg=1\+Vg\\lambda\_\{g\}=1\+V\_\{g\}\) implies that

𝔼G∼γ1\[Qj\(s\)\(VgG\)\]=1j\!dj\+sd​yj\+s𝔼G∼γ1\[χ\(λgG\+y\)\]\|y=0=\(j\+s\)\!j\!λg−\(j\+s\)/2αj\+s,\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\\left\[Q\_\{j\}^\{\(s\)\}\(\\sqrt\{V\_\{g\}\}G\)\\right\]=\\frac\{1\}\{\\sqrt\{j\!\}\}\\left\.\\frac\{d^\{j\+s\}\}\{dy^\{j\+s\}\}\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\[\\chi\(\\sqrt\{\\lambda\_\{g\}\}G\+y\)\]\\right\|\_\{y=0\}=\\sqrt\{\\frac\{\(j\+s\)\!\}\{j\!\}\}\\lambda\_\{g\}^\{\-\(j\+s\)/2\}\\alpha\_\{j\+s\},\(1033\)and therefore withJ:=j\+sJ:=j\+swe have

limL→∞d𝐩,L=αJλg−J/2J\!​∏t=1sqt\!k\!∏t=1sβqt\.\\lim\_\{L\\to\\infty\}d\_\{\\mathbf\{p\},L\}=\\alpha\_\{J\}\\lambda\_\{g\}^\{\-J/2\}\\sqrt\{\\frac\{J\!\\prod\_\{t=1\}^\{s\}q\_\{t\}\!\}\{k\!\}\}\\prod\_\{t=1\}^\{s\}\\beta\_\{q\_\{t\}\}\.\(1034\)Now, for each integerJ∈ℕJ\\in\{\\mathbb\{N\}\}we assign to integer configurationsj∈ℕ∪\{0\}j\\in\{\\mathbb\{N\}\}\\cup\\\{0\\\},\(mq\)q∈ℕ⊆ℕ∪\{0\}\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\subseteq\{\\mathbb\{N\}\}\\cup\\\{0\\\}satisfyingj\+∑q∈ℕmq=Jj\+\\sum\_\{q\\in\{\\mathbb\{N\}\}\}m\_\{q\}=Jthe probabilities

ℙJ​\(j,\(mq\)q∈ℕ\):=J\!j\!​∏q∈ℕmq\!​\(1λg\)j​∏q∈ℕ\(βq2λg\)mq\.\\mathbb\{P\}\_\{J\}\\left\(j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\):=\\frac\{J\!\}\{j\!\\prod\_\{q\\in\{\\mathbb\{N\}\}\}m\_\{q\}\!\}\\left\(\\frac\{1\}\{\\lambda\_\{g\}\}\\right\)^\{j\}\\prod\_\{q\\in\{\\mathbb\{N\}\}\}\\left\(\\frac\{\\beta\_\{q\}^\{2\}\}\{\\lambda\_\{g\}\}\\right\)^\{m\_\{q\}\}\.\(1035\)These sum to 1 by the multinomial formula, since1λg\+∑q∈ℕβq2λg=1\+Vgλg=1\\frac\{1\}\{\\lambda\_\{g\}\}\+\\sum\_\{q\\in\{\\mathbb\{N\}\}\}\\frac\{\\beta\_\{q\}^\{2\}\}\{\\lambda\_\{g\}\}=\\frac\{1\+V\_\{g\}\}\{\\lambda\_\{g\}\}=1\. For each configuration drawn fromℙJ\\mathbb\{P\}\_\{J\}, we set

s:=∑q∈ℕmq,N:=j\+∑q∈ℕq​mq,s:=\\sum\_\{q\\in\{\\mathbb\{N\}\}\}m\_\{q\},\\quad N:=j\+\\sum\_\{q\\in\{\\mathbb\{N\}\}\}qm\_\{q\},\(1036\)chooseq1≤…≤qsq\_\{1\}\\leq\\ldots\\leq q\_\{s\}with exactlymqm\_\{q\}occurrences of eachqq, and form the tuple

𝐢:=\(0,…,0⏟j,1,…,1⏟q1,…,s,…,s⏟qs\)∈ℐN\.\\mathbf\{i\}:=\(\\underbrace\{0,\\ldots,0\}\_\{j\},\\underbrace\{1,\\ldots,1\}\_\{q\_\{1\}\},\\ldots,\\underbrace\{s,\\ldots,s\}\_\{q\_\{s\}\}\)\\in\\mathcal\{I\}\_\{N\}\.\(1037\)ChooseS⊆\{1,…,N\}S\\subseteq\\\{1,\\ldots,N\\\}uniformly at random among subsets of size⌊N/2⌋\\lfloor N/2\\rfloorand define

CS:=CS​\(𝐢\)=\|\{ir:r∈S\}∩\{ir:r∈Sc\}∖\{0\}\|\.C\_\{S\}:=C\_\{S\}\(\\mathbf\{i\}\)=\|\\\{i\_\{r\}:\\;r\\in S\\\}\\cap\\\{i\_\{r\}:\\;r\\in S^\{c\}\\\}\\setminus\\\{0\\\}\|\.\(1038\)To identify \([1028](https://arxiv.org/html/2610.10875#A1.E1028)\) probabilistically, fix any configurationj,\(mq\)q∈ℕj,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}withN=kN=k\. There are

k\!j\!​∏q∈ℕ\(q\!\)mq\\frac\{k\!\}\{j\!\\prod\_\{q\\in\{\\mathbb\{N\}\}\}\(q\!\)^\{m\_\{q\}\}\}\(1039\)distinct permutations of the tuple𝐢\\mathbf\{i\}\. Each canonical tuple𝐩∈ℐk\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}with these multiplicities arises from exactly∏q∈ℕmq\!\\prod\_\{q\\in\{\\mathbb\{N\}\}\}m\_\{q\}\!such permutations, as labels with equal multiplicities can be arbitrarily permuted without changing their canonical representative𝐩\\mathbf\{p\}\. Hence, the number of canonicalkk\-tuples with the fixed multiplicities\(mq\)q∈ℕ\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}is equal to

k\!j\!​∏q∈ℕ\(q\!\)mq​mq\!\.\\frac\{k\!\}\{j\!\\prod\_\{q\\in\{\\mathbb\{N\}\}\}\(q\!\)^\{m\_\{q\}\}m\_\{q\}\!\}\.\(1040\)Letπ\\pibe a uniformly random permutation of\{1,…,k\}\\\{1,\\ldots,k\\\}\. By the above counting argument, the canonical representative of\(iπ⁡\(1\),…,iπ⁡\(k\)\)\(i\_\{\\pi\(1\)\},\\ldots,i\_\{\\pi\(k\)\}\)produces a uniformly random canonical tuple𝐩π∈ℐk\\mathbf\{p\}\_\{\\pi\}\\in\\mathcal\{I\}\_\{k\}with the given multiplicities\. Note that relabeling preserves crossing counts, i\.e\.

CSk​\(𝐩π\)=CSk​\(iπ⁡\(1\),…,iπ⁡\(k\)\)=Cπ⁡\(Sk\)​\(𝐢\)\.C\_\{S\_\{k\}\}\(\\mathbf\{p\}\_\{\\pi\}\)=C\_\{S\_\{k\}\}\(i\_\{\\pi\(1\)\},\\ldots,i\_\{\\pi\(k\)\}\)=C\_\{\\pi\(S\_\{k\}\)\}\(\\mathbf\{i\}\)\.\(1041\)Sinceπ⁡\(Sk\)\\pi\(S\_\{k\}\)is a uniformly random balanced cut, the fraction of canonical tuples𝐩π\\mathbf\{p\}\_\{\\pi\}satisfyingCSk​\(𝐩π\)≥RC\_\{S\_\{k\}\}\(\\mathbf\{p\}\_\{\\pi\}\)\\geq Ris exactlyℙJ​\(CS≥R∣j,\(mq\)q∈ℕ\)\\mathbb\{P\}\_\{J\}\(C\_\{S\}\\geq R\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\)\. From \([1034](https://arxiv.org/html/2610.10875#A1.E1034)\) we know that

limL→∞\|d𝐩π,L\|2=αJ2​J\!k\!​λgJ​∏q∈ℕ\(q\!​βq2\)mq,J=j\+∑q∈ℕmq\.\\lim\_\{L\\to\\infty\}\|d\_\{\\mathbf\{p\}\_\{\\pi\},L\}\|^\{2\}=\\frac\{\\alpha\_\{J\}^\{2\}J\!\}\{k\!\\lambda\_\{g\}^\{J\}\}\\prod\_\{q\\in\{\\mathbb\{N\}\}\}\(q\!\\beta\_\{q\}^\{2\}\)^\{m\_\{q\}\},\\quad J=j\+\\sum\_\{q\\in\{\\mathbb\{N\}\}\}m\_\{q\}\.\(1042\)Hence, averaging overπ\\pifor a fixed configurationj,\(mq\)q∈ℕj,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}withN=kN=kyields

j\!​∏q∈ℕ\(q\!\)mq​mq\!k\!∑𝐩∈ℐk,ν0​\(𝐩\)=j\#⁡\{1≤t≤s:νt​\(𝐩\)=q\}=mq​∀q∈ℕCSk​\(𝐩\)≥RlimL→∞\|d𝐩,L\|2=αJ2​J\!k\!​λgJ∏q∈ℕ\(q\!βq2\)mqℙJ\(CS≥R∣j,\(mq\)q∈ℕ\)\\displaystyle\\frac\{j\!\\prod\_\{q\\in\{\\mathbb\{N\}\}\}\(q\!\)^\{m\_\{q\}\}m\_\{q\}\!\}\{k\!\}\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\},\\nu\_\{0\}\(\\mathbf\{p\}\)=j\\\\ \\\#\\\{1\\leq t\\leq s:\\;\\nu\_\{t\}\(\\mathbf\{p\}\)=q\\\}=m\_\{q\}\\;\\forall q\\in\{\\mathbb\{N\}\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}\\lim\_\{L\\to\\infty\}\|d\_\{\\mathbf\{p\},L\}\|^\{2\}=\\frac\{\\alpha\_\{J\}^\{2\}J\!\}\{k\!\\lambda\_\{g\}^\{J\}\}\\prod\_\{q\\in\{\\mathbb\{N\}\}\}\(q\!\\beta\_\{q\}^\{2\}\)^\{m\_\{q\}\}\\mathbb\{P\}\_\{J\}\\left\(C\_\{S\}\\geq R\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\)\(1043\)or in other words that

∑𝐩∈ℐk,ν0​\(𝐩\)=j\#⁡\{1≤t≤s:νt​\(𝐩\)=q\}=mq​∀q∈ℕCSk​\(𝐩\)≥RlimL→∞\|d𝐩,L\|2\\displaystyle\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\},\\nu\_\{0\}\(\\mathbf\{p\}\)=j\\\\ \\\#\\\{1\\leq t\\leq s:\\;\\nu\_\{t\}\(\\mathbf\{p\}\)=q\\\}=m\_\{q\}\\;\\forall q\\in\{\\mathbb\{N\}\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}\\lim\_\{L\\to\\infty\}\|d\_\{\\mathbf\{p\},L\}\|^\{2\}=αJ2​J\!j\!​∏q∈ℕmq\!​\(1λg\)j​∏q∈ℕ\(βq2λg\)mq\\displaystyle=\\frac\{\\alpha\_\{J\}^\{2\}J\!\}\{j\!\\prod\_\{q\\in\{\\mathbb\{N\}\}\}m\_\{q\}\!\}\\left\(\\frac\{1\}\{\\lambda\_\{g\}\}\\right\)^\{j\}\\prod\_\{q\\in\{\\mathbb\{N\}\}\}\\left\(\\frac\{\\beta\_\{q\}^\{2\}\}\{\\lambda\_\{g\}\}\\right\)^\{m\_\{q\}\}\(1044\)⋅ℙJ​\(CS≥R∣j,\(mq\)q∈ℕ\)\\displaystyle\\quad\\cdot\\mathbb\{P\}\_\{J\}\\left\(C\_\{S\}\\geq R\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\)\(1045\)=αJ2​ℙJ​\(j,\(mq\)q∈ℕ\)⋅ℙJ​\(CS≥R∣j,\(mq\)q∈ℕ\),\\displaystyle=\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\\left\(j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\)\\cdot\\mathbb\{P\}\_\{J\}\\left\(C\_\{S\}\\geq R\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\),\(1046\)where the second line applied the definition ofℙJ\\mathbb\{P\}\_\{J\}in \([1035](https://arxiv.org/html/2610.10875#A1.E1035)\)\. We exchange the limit and the finite sum and then sum over all configurations for whichN=kN=k\(which includes summing overJJ\)\. However, sinceJ≤NJ\\leq Nwe need only considerJ≤kJ\\leq k, which yields

limL→∞∑𝐩∈ℐkCSk​\(𝐩\)≥R\|d𝐩,L\|2=∑J=1kαJ2​ℙJ​\(N=k,CS≥R\)\.\\lim\_\{L\\to\\infty\}\\sum\_\{\\begin\{subarray\}\{c\}\\mathbf\{p\}\\in\\mathcal\{I\}\_\{k\}\\\\ C\_\{S\_\{k\}\}\(\\mathbf\{p\}\)\\geq R\\end\{subarray\}\}\|d\_\{\\mathbf\{p\},L\}\|^\{2\}=\\sum\_\{J=1\}^\{k\}\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\(N=k,\\;C\_\{S\}\\geq R\)\.\(1047\)\(Step 3\)\.UnderℙJ\\mathbb\{P\}\_\{J\}, letB:=∑q≥2mqB:=\\sum\_\{q\\geq 2\}m\_\{q\}be the number of positive labels occurring at least twice \(i\.e\. the number of nonlinear factors\)\. Because the probability mass function ofℙJ\\mathbb\{P\}\_\{J\}from \([1035](https://arxiv.org/html/2610.10875#A1.E1035)\) is a multinomial law, we find thatB∼Binomial⁡\(J,8​𝔳g\)B\\sim\\operatorname\{Binomial\}\(J,8\\mathfrak\{v\}\_\{g\}\)\(since∑q≥2βq2/λg=8​𝔳g\\sum\_\{q\\geq 2\}\\beta\_\{q\}^\{2\}/\\lambda\_\{g\}=8\\mathfrak\{v\}\_\{g\}\)\. Moreover, sinceggis Lipschitz we know that

∑q∈ℕq​βq2=𝔼G∼γ1​\|g′​\(G\)\|2≤Lip⁡\(g\)2,\\sum\_\{q\\in\{\\mathbb\{N\}\}\}q\\beta\_\{q\}^\{2\}=\\mathbb\{E\}\_\{G\\sim\\gamma\_\{1\}\}\|g^\{\\prime\}\(G\)\|^\{2\}\\leq\\operatorname\{Lip\}\(g\)^\{2\},\(1048\)which implies that

𝔼J​\[N\]=𝔼J​\[j\+∑q∈ℕq​mq\]=Jλg​\(1\+∑q∈ℕq​βq2\)≤J​Bg8\.\\mathbb\{E\}\_\{J\}\[N\]=\\mathbb\{E\}\_\{J\}\\left\[j\+\\sum\_\{q\\in\{\\mathbb\{N\}\}\}qm\_\{q\}\\right\]=\\frac\{J\}\{\\lambda\_\{g\}\}\\left\(1\+\\sum\_\{q\\in\{\\mathbb\{N\}\}\}q\\beta\_\{q\}^\{2\}\\right\)\\leq\\frac\{JB\_\{g\}\}\{8\}\.\(1049\)By Markov’s inequality, we find that

ℙJ​\(N\>Bg​J\)≤18\.\\mathbb\{P\}\_\{J\}\(N\>B\_\{g\}J\)\\leq\\frac\{1\}\{8\}\.\(1050\)Next, by the binomial Chernoff bound we know

ℙJ​\(B<4​𝔳g​J\)≤e−𝔳g​J,\\mathbb\{P\}\_\{J\}\(B<4\\mathfrak\{v\}\_\{g\}J\)\\leq e^\{\-\\mathfrak\{v\}\_\{g\}J\},\(1051\)and so a union bound gives that forJ≥KgJ\\geq K\_\{g\}we have

ℙJ​\(N≤Bg​J,B≥4​𝔳g​J\)≥1−18−e−𝔳g​J≥1316,\\mathbb\{P\}\_\{J\}\(N\\leq B\_\{g\}J,\\;B\\geq 4\\mathfrak\{v\}\_\{g\}J\)\\geq 1\-\\frac\{1\}\{8\}\-e^\{\-\\mathfrak\{v\}\_\{g\}J\}\\geq\\frac\{13\}\{16\},\(1052\)where we denoteKg:=⌈4/𝔳g⌉K\_\{g\}:=\\lceil 4/\\mathfrak\{v\}\_\{g\}\\rceil\. Now, condition on a configurationj,\(mq\)q∈ℕj,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}withB≥1B\\geq 1\. For each positive label occurring at least twice, select two positions carrying that label\. A uniformly balanced random cut separates these positions with probability≥12\\geq\\frac\{1\}\{2\}, which implies

𝔼\[CS∣j,\(mq\)q∈ℕ\]≥B2\.\\mathbb\{E\}\\left\[C\_\{S\}\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\]\\geq\\frac\{B\}\{2\}\.\(1053\)Next, we know that

CS=CS​𝟙CS<B/4\+CS​𝟙CS≥B/4≤B4\+3​B4​𝟙CS≥B/4,C\_\{S\}=C\_\{S\}\{\\mathbbm\{1\}\}\_\{C\_\{S\}<B/4\}\+C\_\{S\}\{\\mathbbm\{1\}\}\_\{C\_\{S\}\\geq B/4\}\\leq\\frac\{B\}\{4\}\+\\frac\{3B\}\{4\}\{\\mathbbm\{1\}\}\_\{C\_\{S\}\\geq B/4\},\(1054\)where we use that0≤CS≤B0\\leq C\_\{S\}\\leq B\(since any pair contributing toCSC\_\{S\}must also contribute toBB\)\. Taking conditional expectations,

B2≤𝔼\[CS∣j,\(mq\)q∈ℕ\]≤B4\+3​B4ℙJ\(CS≥B/4∣j,\(mq\)q∈ℕ\)\.\\frac\{B\}\{2\}\\leq\\mathbb\{E\}\\left\[C\_\{S\}\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\]\\leq\\frac\{B\}\{4\}\+\\frac\{3B\}\{4\}\\mathbb\{P\}\_\{J\}\\left\(C\_\{S\}\\geq B/4\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\)\.\(1055\)SinceB\>0B\>0, we rearrange to find that

ℙJ​\(CS≥B/4∣j,\(mq\)q∈ℕ\)≥13\.\\mathbb\{P\}\_\{J\}\\left\(C\_\{S\}\\geq B/4\\mid j,\(m\_\{q\}\)\_\{q\\in\{\\mathbb\{N\}\}\}\\right\)\\geq\\frac\{1\}\{3\}\.\(1056\)For a configuration withB=0B=0this holds trivially, and therefore it always holds\. Combining with \([1052](https://arxiv.org/html/2610.10875#A1.E1052)\) gives

ℙJ​\(N≤Bg​J,CS≥𝔳g​J\)≥1348\.\\mathbb\{P\}\_\{J\}\(N\\leq B\_\{g\}J,\\;C\_\{S\}\\geq\\mathfrak\{v\}\_\{g\}J\)\\geq\\frac\{13\}\{48\}\.\(1057\)
We now finish the proof of the main claim \([990](https://arxiv.org/html/2610.10875#A1.E990)\)\. Combining the estimates \([1028](https://arxiv.org/html/2610.10875#A1.E1028)\) and \([1047](https://arxiv.org/html/2610.10875#A1.E1047)\) from steps 1 and 2, respectively, gives

lim infL→∞∑j\>LR−1σj​\(Mk​\(FL\)\)2\\displaystyle\\liminf\_\{L\\to\\infty\}\\sum\_\{j\>L^\{R\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}≥∑J=1kαJ2​ℙJ​\(N=k,CS≥R\)\.\\displaystyle\\geq\\sum\_\{J=1\}^\{k\}\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\(N=k,\\;C\_\{S\}\\geq R\)\.\(1058\)ForKg≤K1≤K2K\_\{g\}\\leq K\_\{1\}\\leq K\_\{2\}, summing overK1≤k≤⌈Bg​K2⌉K\_\{1\}\\leq k\\leq\\lceil B\_\{g\}K\_\{2\}\\rceiland takingR=⌊𝔳g​K1⌋R=\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloorgives

lim infL→∞∑k=K1⌈Bg​K2⌉∑j\>L⌊𝔳g​K1⌋−1σj​\(Mk​\(FL\)\)2\\displaystyle\\liminf\_\{L\\to\\infty\}\\sum\_\{k=K\_\{1\}\}^\{\\lceil B\_\{g\}K\_\{2\}\\rceil\}\\sum\_\{j\>L^\{\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\-1\}\}\\sigma\_\{j\}\(M\_\{k\}\(F\_\{L\}\)\)^\{2\}≥∑k=K1⌈Bg​K2⌉∑J=1kαJ2​ℙJ​\(N=k,CS≥⌊𝔳g​K1⌋\)\\displaystyle\\geq\\sum\_\{k=K\_\{1\}\}^\{\\lceil B\_\{g\}K\_\{2\}\\rceil\}\\sum\_\{J=1\}^\{k\}\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\(N=k,\\;C\_\{S\}\\geq\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\)\(1059\)=∑J=1⌈Bg​K2⌉αJ2​∑k=max⁡\{K1,J\}⌈Bg​K2⌉ℙJ​\(N=k,CS≥⌊𝔳g​K1⌋\)\\displaystyle=\\sum\_\{J=1\}^\{\\lceil B\_\{g\}K\_\{2\}\\rceil\}\\alpha\_\{J\}^\{2\}\\sum\_\{k=\\max\\\{K\_\{1\},J\\\}\}^\{\\lceil B\_\{g\}K\_\{2\}\\rceil\}\\mathbb\{P\}\_\{J\}\(N=k,\\;C\_\{S\}\\geq\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\)\(1060\)=∑J=1⌈Bg​K2⌉αJ2ℙJ\(K1≤N≤⌈BgK2⌉,CS≥⌊𝔳gK1⌋\)\\displaystyle=\\sum\_\{J=1\}^\{\\lceil B\_\{g\}K\_\{2\}\\rceil\}\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\(K\_\{1\}\\leq N\\leq\\lceil B\_\{g\}K\_\{2\}\\rceil,\\;C\_\{S\}\\geq\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\)\(1061\)≥∑J=K1K2αJ2ℙJ\(K1≤N≤⌈BgK2⌉,CS≥⌊𝔳gK1⌋\)\\displaystyle\\geq\\sum\_\{J=K\_\{1\}\}^\{K\_\{2\}\}\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\(K\_\{1\}\\leq N\\leq\\lceil B\_\{g\}K\_\{2\}\\rceil,\\;C\_\{S\}\\geq\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\)\(1062\)=∑J=K1K2αJ2​ℙJ​\(N≤⌈Bg​K2⌉,CS≥⌊𝔳g​K1⌋\)\\displaystyle=\\sum\_\{J=K\_\{1\}\}^\{K\_\{2\}\}\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\(N\\leq\\lceil B\_\{g\}K\_\{2\}\\rceil,\\;C\_\{S\}\\geq\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\)\(1063\)≥∑J=K1K2αJ2​ℙJ​\(N≤Bg​J,CS≥𝔳g​J\)\\displaystyle\\geq\\sum\_\{J=K\_\{1\}\}^\{K\_\{2\}\}\\alpha\_\{J\}^\{2\}\\mathbb\{P\}\_\{J\}\(N\\leq B\_\{g\}J,\\;C\_\{S\}\\geq\\mathfrak\{v\}\_\{g\}J\)\(1064\)≥1348​∑J=K1K2αJ2,\\displaystyle\\geq\\frac\{13\}\{48\}\\sum\_\{J=K\_\{1\}\}^\{K\_\{2\}\}\\alpha\_\{J\}^\{2\},\(1065\)where the third and fifth lines use thatN≥JN\\geq Jalmost surely, the fourth uses thatVg=Var⁡\(g\)≤Lip⁡\(g\)2V\_\{g\}=\\operatorname\{Var\}\(g\)\\leq\\operatorname\{Lip\}\(g\)^\{2\}and soBg≥1B\_\{g\}\\geq 1, the sixth uses that⌈Bg​K2⌉≥Bg​J\\lceil B\_\{g\}K\_\{2\}\\rceil\\geq B\_\{g\}Jand⌊𝔳g​K1⌋≤𝔳g​J\\lfloor\\mathfrak\{v\}\_\{g\}K\_\{1\}\\rfloor\\leq\\mathfrak\{v\}\_\{g\}J, and the last uses \([1057](https://arxiv.org/html/2610.10875#A1.E1057)\) from step 3\. This is exactly the claim \([990](https://arxiv.org/html/2610.10875#A1.E990)\)\. ∎