From Score Approximation to Distribution Approximation in Score-Based Diffusion Models

# From Score Approximation to Distribution Approximation in Score-Based Diffusion Models
Source: [https://arxiv.org/html/2607.22199](https://arxiv.org/html/2607.22199)
Lan V\. Truong Faculty of Computer Science and Engineering Ho Chi Minh City University of Technology \(HCMUT\) Vietnam National University Ho Chi Minh City \(VNU\-HCM\), Vietnam lantv@hcmut\.edu\.vn

###### Abstract

Score\-based diffusion models have achieved remarkable empirical success in generative modeling, yet their approximation\-theoretic foundations remain incomplete\. In particular, although classical universal approximation theorems guarantee that neural networks can approximate score functions, it remains unclear whether such approximation guarantees translate into approximation of the probability distributions generated by reverse diffusion processes\. In this paper, we establish a rigorous quantitative connection between these two notions\. Specifically, we prove that if a neural network approximates the true score function sufficiently accurately, then the probability distribution generated by the corresponding reverse diffusion model is close to the target data distribution in Kullback–Leibler \(KL\) divergence, up to an irreducible mismatch between the terminal distribution of the forward diffusion process and the prior used to initialize the reverse process\. More precisely, we derive an explicit upper bound on the distribution approximation error in terms of the score approximation error, the diffusion noise schedule, and the terminal prior mismatch\. Our analysis combines Hornik’s universal approximation theorem, Girsanov’s theorem on path space, and the data processing inequality for relative entropy\. Complementary to recent work that studies score approximation under finite\-sample statistical settings and structural assumptions on the data distribution, our work develops an approximation\-theoretic analysis based on classical neural network approximation theory\. The resulting theorem provides a simple and explicit guarantee linking neural network approximation of score functions to approximation of the probability distributions generated by reverse diffusion models\.

###### Contents

1. [1Introduction](https://arxiv.org/html/2607.22199#S1)
2. [2Related Work](https://arxiv.org/html/2607.22199#S2)
3. [3Background](https://arxiv.org/html/2607.22199#S3)1. [3\.1Forward Diffusion Process](https://arxiv.org/html/2607.22199#S3.SS1) 2. [3\.2Reverse\-Time Diffusion](https://arxiv.org/html/2607.22199#S3.SS2) 3. [3\.3Problem Formulation](https://arxiv.org/html/2607.22199#S3.SS3)
4. [4Assumptions](https://arxiv.org/html/2607.22199#S4)
5. [5Score Approximation to Path Measure Approximation](https://arxiv.org/html/2607.22199#S5)
6. [6Path Measure Approximation to Distribution Approximation](https://arxiv.org/html/2607.22199#S6)
7. [7From Score Approximation to Distribution Approximation](https://arxiv.org/html/2607.22199#S7)
8. [8Extensions and Future Directions](https://arxiv.org/html/2607.22199#S8)1. [8\.1Alternative Probability Metrics](https://arxiv.org/html/2607.22199#S8.SS1) 2. [8\.2Approximation Rates](https://arxiv.org/html/2607.22199#S8.SS2) 3. [8\.3General Diffusion Models](https://arxiv.org/html/2607.22199#S8.SS3) 4. [8\.4Beyond Euclidean Data](https://arxiv.org/html/2607.22199#S8.SS4) 5. [8\.5Learning Theory for Diffusion Models](https://arxiv.org/html/2607.22199#S8.SS5)
9. [References](https://arxiv.org/html/2607.22199#bib)

## 1Introduction

Score\-based diffusion models have recently emerged as one of the most successful frameworks for generative modeling, achieving state\-of\-the\-art performance in image generation, audio synthesis, molecular design, and scientific machine learning\. Representative examples include denoising diffusion probabilistic models \(DDPMs\) and score\-based generative models formulated through stochastic differential equations \(SDEs\)\. Their remarkable empirical performance has motivated growing interest in understanding their mathematical foundations\.

A central object in score\-based diffusion models is the*score function*

s​\(x,t\)=∇xlog⁡pt​\(x\),s\(x,t\)=\\nabla\_\{x\}\\log p\_\{t\}\(x\),which characterizes the gradient of the log\-density of the forward diffusion process\. In practice, this score function is unknown and is approximated by a deep neural network trained using denoising score matching\. Consequently, the quality of the learned generative model fundamentally depends on how accurately the neural network approximates the true score function\.

Although classical universal approximation theorems guarantee that neural networks can approximate a broad class of functions, these results concern only function approximation\. They do not directly imply that approximating the score function yields an accurate approximation of the probability distribution generated by the reverse diffusion process\. Establishing such a connection remains an important theoretical challenge\.

The objective of this paper is to establish a rigorous quantitative relationship between*score approximation*and*distribution approximation*for score\-based diffusion models\. Specifically, we prove that an arbitrarily accurate approximation of the score function in theL2L^\{2\}sense leads to an arbitrarily small Kullback–Leibler \(KL\) divergence between the target data distribution and the distribution generated by the learned reverse diffusion process\.

Our analysis proceeds in two steps\.

First, we study the path measures induced by the true and learned reverse\-time stochastic differential equations\. Using Girsanov’s theorem, we derive an explicit identity for the Kullback–Leibler divergence between the corresponding path measures:

KL⁡\(P∥Qθ\)=KL⁡\(pT∥pprior\)\+12​∫0Tg2​\(t\)​𝔼pt​\[‖sθ​\(Xt,t\)−s​\(Xt,t\)‖2\]​𝑑t\.\\operatorname\{KL\}\(P\\\|Q\_\{\\theta\}\)=\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\+\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}g^\{2\}\(t\)\\,\\mathbb\{E\}\_\{p\_\{t\}\}\\\!\\left\[\\\|s\_\{\\theta\}\(X\_\{t\},t\)\-s\(X\_\{t\},t\)\\\|^\{2\}\\right\]dt\.
Second, we apply the data processing inequality for relative entropy to show that the KL divergence between the generated data distributions is bounded above by the corresponding path\-space divergence\. Combining these two results with Hornik’s universal approximation theorem yields our main result: for everyε\>0\\varepsilon\>0, there exists a neural network satisfying

‖sθ−s‖L2​\(μ\)d2<ε,\\\|s\_\{\\theta\}\-s\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}^\{2\}<\\varepsilon,such that

KL⁡\(μ∥μθ\)<δ\+12​gmax2​ε,\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)<\\delta\+\\frac\{1\}\{2\}g\_\{\\max\}^\{2\}\\varepsilon,where

δ=KL⁡\(pT∥pprior\)\\delta=\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)quantifies the discrepancy between the terminal distribution of the forward diffusion process and the prior used to initialize the reverse diffusion process\.

Recent work by Chen et al\.\[[3](https://arxiv.org/html/2607.22199#bib.bib3)\]investigated the relationship between score approximation and distribution approximation from a statistical learning perspective, deriving approximation guarantees under finite\-sample settings and structural assumptions on the underlying data distribution, including low\-dimensional linear subspaces\. Our work complements this line of research by developing an approximation\-theoretic perspective\. Rather than analyzing finite\-sample estimation error, we combine classical results from neural network approximation theory, stochastic analysis, and information theory to derive a simple and explicit quantitative bound connecting score approximation with approximation of the generated probability distribution\. This provides a mathematical justification for the expressive power of neural score models and offers an approximation\-theoretic foundation for score\-based diffusion models\.

The remainder of the paper is organized as follows\. Section 2 discusses the related literature\. Section 3 reviews score\-based diffusion models and the corresponding reverse\-time stochastic differential equation\. Section 4 presents the assumptions used throughout the paper and discusses their validity in several important settings\. Section 5 establishes the relationship between score approximation and path measure approximation\. Section 6 establishes that approximation of path measures implies approximation of the generated data distribution\. Section 7 combines these results with Hornik’s universal approximation theorem to establish our main approximation theorem for reverse diffusion models\. Finally, Section 8 presents several extensions of the proposed framework and discusses directions for future research\.

## 2Related Work

#### Score\-Based Diffusion Models\.

Diffusion models have recently become one of the dominant approaches to generative modeling\. Early score\-based generative models were introduced by Song and Ermon\[[12](https://arxiv.org/html/2607.22199#bib.bib12)\], who proposed learning the score function of perturbed data distributions using denoising score matching\. Ho et al\.\[[8](https://arxiv.org/html/2607.22199#bib.bib8)\]subsequently introduced Denoising Diffusion Probabilistic Models \(DDPMs\), which achieved remarkable empirical performance by learning the reverse of a discrete diffusion process\. Song et al\.\[[13](https://arxiv.org/html/2607.22199#bib.bib13)\]unified score\-based models and DDPMs under a continuous\-time stochastic differential equation \(SDE\) framework, showing that sample generation can be viewed as solving a reverse\-time SDE driven by the score function\. Since then, the SDE formulation has become the standard mathematical framework for analyzing score\-based diffusion models\.

#### Reverse\-Time Diffusion Processes\.

The mathematical foundation of score\-based diffusion models can be traced back to the classical theory of time reversal of diffusion processes\. Anderson\[[1](https://arxiv.org/html/2607.22199#bib.bib1)\]first derived the reverse\-time stochastic differential equation for diffusion processes, showing that the reverse drift depends on the gradient of the logarithm of the evolving probability density\. Later, Haussmann and Pardoux\[[6](https://arxiv.org/html/2607.22199#bib.bib6)\]established a rigorous probabilistic treatment of reverse\-time diffusions under suitable regularity conditions\. These results provide the theoretical basis for the reverse SDE used in modern diffusion models\.

#### Universal Approximation of Neural Networks\.

The expressive power of neural networks has been extensively studied in the approximation theory literature\. Cybenko\[[5](https://arxiv.org/html/2607.22199#bib.bib5)\]first established the universal approximation property for single\-hidden\-layer neural networks\. Hornik\[[9](https://arxiv.org/html/2607.22199#bib.bib9)\]subsequently proved much stronger density results, showing that standard feedforward neural networks are dense inLp​\(μ\)L^\{p\}\(\\mu\)spaces under mild assumptions on the activation function\. These results guarantee that neural networks can approximate the score function arbitrarily well, but they do not directly imply approximation of the probability distribution generated by the reverse diffusion process\.

#### Theoretical Analysis of Diffusion Models\.

Recent years have witnessed increasing interest in establishing rigorous mathematical foundations for diffusion models\. Song et al\.\[[13](https://arxiv.org/html/2607.22199#bib.bib13)\]derived the continuous\-time SDE formulation together with the associated probability flow ordinary differential equation\. De Bortoli et al\.\[[2](https://arxiv.org/html/2607.22199#bib.bib2)\]investigated Schrödinger bridge formulations and their connections to score\-based generative modeling\. These works primarily focus on the stochastic dynamics, sampling algorithms, and score estimation underlying diffusion models, providing the mathematical foundations for modern score\-based generative methods\.

#### Distribution Approximation in Diffusion Models\.

More recently, theoretical attention has turned to understanding how approximation errors in the learned score function affect the probability distribution generated by a diffusion model\. In particular, Chen et al\.\[[3](https://arxiv.org/html/2607.22199#bib.bib3)\]investigated the relationship between score approximation and distribution approximation, establishing quantitative guarantees on how score estimation errors propagate to the generated distribution in finite\-sample settings under low\-dimensional linear subspace assumptions\. Their work provides important insights into the statistical complexity of score\-based generative models\.

Our work is complementary to this line of research\. Rather than studying finite\-sample statistical estimation under structural assumptions on the data distribution, we investigate the approximation capability of neural networks for score\-based diffusion models in a general Euclidean setting\. By combining Hornik’s universal approximation theorem, Girsanov’s theorem on path space, and the data processing inequality for relative entropy, we establish an explicit bound showing that anL2L^\{2\}approximation of the score function directly yields a quantitative upper bound on the Kullback–Leibler divergence between the target distribution and the distribution generated by the learned reverse diffusion model\.

#### Our Contribution\.

The present work establishes a rigorous theoretical connection between score approximation and distribution approximation for score\-based diffusion models\. We prove that arbitrarily accurate approximation of the score function inL2​\(μ\)L^\{2\}\(\\mu\)by neural networks implies arbitrarily small Kullback–Leibler divergence between the true data distribution and the distribution generated by the learned reverse diffusion model\. The proof combines three fundamental ingredients from approximation theory, stochastic analysis, and information theory: Hornik’s universal approximation theorem, Girsanov’s theorem for changes of path measures, and the contraction property of relative entropy under measurable projections\. This yields a general quantitative approximation theorem for reverse diffusion models, providing a theoretical foundation that links neural network approximation of score functions to approximation of the induced probability distributions\.

## 3Background

In this section, we briefly review score\-based diffusion models and introduce the notation used throughout the paper\.

### 3\.1Forward Diffusion Process

Let

whereμ∈𝒫​\(ℝd\)\\mu\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{d\}\)denotes the unknown data distribution\.

The forward diffusion process is governed by the Itô stochastic differential equation

d​Xt=f​\(Xt,t\)​d​t\+g​\(t\)​d​Wt,t∈\[0,T\],dX\_\{t\}=f\(X\_\{t\},t\)\\,dt\+g\(t\)\\,dW\_\{t\},\\qquad t\\in\[0,T\],where

- •f:ℝd×\[0,T\]→ℝdf:\\mathbb\{R\}^\{d\}\\times\[0,T\]\\rightarrow\\mathbb\{R\}^\{d\}is the drift function;
- •g:\[0,T\]→\(0,∞\)g:\[0,T\]\\rightarrow\(0,\\infty\)is the diffusion coefficient;
- •WtW\_\{t\}is a standard Brownian motion\.

Letpt​\(x\)p\_\{t\}\(x\)denote the probability density function ofXtX\_\{t\}\. Asttincreases, the forward diffusion gradually transforms the data distribution into a tractable terminal distributionpTp\_\{T\}\. In practical diffusion models, the reverse process is initialized from a prescribed prior distribution

pprior,p\_\{\\mathrm\{prior\}\},which is typically chosen to be the standard Gaussian distribution\. Ideally,ppriorp\_\{\\mathrm\{prior\}\}should coincide withpTp\_\{T\}; however, in general they may differ\. Throughout this paper, we explicitly quantify this mismatch by the Kullback–Leibler divergence

KL⁡\(pT∥pprior\)\.\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\.
The score function associated with the forward diffusion is defined by

s​\(x,t\)=∇xlog⁡pt​\(x\)\.s\(x,t\)=\\nabla\_\{x\}\\log p\_\{t\}\(x\)\.
Throughout the paper, we assume

s∈L2​\(μ\)d\.s\\in L^\{2\}\(\\mu\)^\{d\}\.

### 3\.2Reverse\-Time Diffusion

A fundamental result due to Anderson\[[1](https://arxiv.org/html/2607.22199#bib.bib1)\]states that the reverse\-time diffusion process satisfies

d​Xt=\(f​\(Xt,t\)−g2​\(t\)​s​\(Xt,t\)\)​d​t\+g​\(t\)​d​W¯t,dX\_\{t\}=\\bigl\(f\(X\_\{t\},t\)\-g^\{2\}\(t\)s\(X\_\{t\},t\)\\bigr\)\\,dt\+g\(t\)\\,d\\bar\{W\}\_\{t\},whereW¯t\\bar\{W\}\_\{t\}denotes a Brownian motion with respect to the reverse\-time filtration\.

Since the score function is generally unknown, it is approximated by a neural network

sθ​\(x,t\)\.s\_\{\\theta\}\(x,t\)\.
Starting from the prior distribution

XTθ∼pprior,X\_\{T\}^\{\\theta\}\\sim p\_\{\\mathrm\{prior\}\},replacing the true score function by its approximation yields the learned reverse diffusion process

d​Xtθ=\(f​\(Xtθ,t\)−g2​\(t\)​sθ​\(Xtθ,t\)\)​d​t\+g​\(t\)​d​W¯t\.dX\_\{t\}^\{\\theta\}=\\bigl\(f\(X\_\{t\}^\{\\theta\},t\)\-g^\{2\}\(t\)s\_\{\\theta\}\(X\_\{t\}^\{\\theta\},t\)\\bigr\)\\,dt\+g\(t\)\\,d\\bar\{W\}\_\{t\}\.
LetPPandQθQ\_\{\\theta\}denote the probability measures induced by the true and learned reverse diffusion processes, respectively, on the path space

C​\(\[0,T\];ℝd\)\.C\(\[0,T\];\\mathbb\{R\}^\{d\}\)\.Furthermore, letμθ\\mu\_\{\\theta\}denote the marginal distribution ofX0θX\_\{0\}^\{\\theta\}generated by the learned reverse diffusion process\.

The objective of this paper is to quantify how approximation of the score function affects the discrepancy between the path measuresPPandQθQ\_\{\\theta\}, and consequently between the generated data distributionsμ\\muandμθ\\mu\_\{\\theta\}\.

### 3\.3Problem Formulation

The objective of score\-based generative modeling is to learn a neural networksθs\_\{\\theta\}such that the generated distributionμθ\\mu\_\{\\theta\}accurately approximates the true data distributionμ\\mu\.

Classical universal approximation theorems establish that neural networks can approximate the score function arbitrarily well in suitable function spaces\. However, these results do not directly imply that the generated distribution approximates the true data distribution\.

The goal of this paper is therefore to establish a quantitative relationship between

and

distribution approximation\.\\text\{distribution approximation\}\.
Specifically, we show that a sufficiently accurate approximation of the score function guarantees a small Kullback–Leibler divergence between the true data distribution and the distribution generated by the learned reverse diffusion process\.

## 4Assumptions

Throughout this paper, we impose the following assumptions\.

###### Assumption 1\(Regularity\)\.

The driftf​\(⋅,t\)f\(\\cdot,t\), the true score functions​\(⋅,t\)s\(\\cdot,t\), and the learned score functionsθ​\(⋅,t\)s\_\{\\theta\}\(\\cdot,t\)are globally Lipschitz continuous in the state variablexx, uniformly for allt∈\[0,T\]t\\in\[0,T\]\.

Assumption[1](https://arxiv.org/html/2607.22199#Thmassumption1)is satisfied by a broad class of neural networks commonly used in practice\. Throughout this paper, we consider feedforward neural networks with an activation function satisfying

σ∈C1​\(ℝ\),supz∈ℝ\|σ′​\(z\)\|<∞\.\\sigma\\in C^\{1\}\(\\mathbb\{R\}\),\\qquad\\sup\_\{z\\in\\mathbb\{R\}\}\|\\sigma^\{\\prime\}\(z\)\|<\\infty\.A typical example is the softplus activation

σ​\(z\)=log⁡\(1\+ez\),\\sigma\(z\)=\\log\(1\+e^\{z\}\),which is unbounded, nonconstant, continuously differentiable, and has bounded derivative\. Consequently, it satisfies the assumptions of Theorem 1 in Hornik\[[9](https://arxiv.org/html/2607.22199#bib.bib9)\]\.

Since the score function depends on both the spatial variablexxand the time variablett, we regard\(x,t\)\(x,t\)as an element ofℝd\+1\\mathbb\{R\}^\{d\+1\}\. Under the standing assumption thats∈L2​\(μ\)ds\\in L^\{2\}\(\\mu\)^\{d\}, Hornik’s Universal Approximation Theorem\[[9](https://arxiv.org/html/2607.22199#bib.bib9)\]implies that, for everyε\>0\\varepsilon\>0, there exists a finite feedforward neural network of the form

sθ​\(x,t\)=∑i=1Nαi​σ​\(ui⊤​x\+ci​t\+bi\),s\_\{\\theta\}\(x,t\)=\\sum\_\{i=1\}^\{N\}\\alpha\_\{i\}\\sigma\\\!\\left\(u\_\{i\}^\{\\top\}x\+c\_\{i\}t\+b\_\{i\}\\right\),where

ui∈ℝd,ci,bi∈ℝ,αi∈ℝd,u\_\{i\}\\in\\mathbb\{R\}^\{d\},\\qquad c\_\{i\},b\_\{i\}\\in\\mathbb\{R\},\\qquad\\alpha\_\{i\}\\in\\mathbb\{R\}^\{d\},such that

‖s−sθ‖L2​\(μ\)d<ε\.\\\|s\-s\_\{\\theta\}\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}<\\varepsilon\.
Furthermore, since the network contains finitely many neurons and all parameters are finite,

∇xsθ​\(x,t\)=∑i=1Nαi​σ′​\(ui⊤​x\+ci​t\+bi\)​ui⊤\.\\nabla\_\{x\}s\_\{\\theta\}\(x,t\)=\\sum\_\{i=1\}^\{N\}\\alpha\_\{i\}\\sigma^\{\\prime\}\\\!\\left\(u\_\{i\}^\{\\top\}x\+c\_\{i\}t\+b\_\{i\}\\right\)u\_\{i\}^\{\\top\}\.Therefore,

‖∇xsθ​\(x,t\)‖≤supz∈ℝ\|σ′​\(z\)\|​∑i=1N‖αi‖​‖ui‖<∞\.\\\|\\nabla\_\{x\}s\_\{\\theta\}\(x,t\)\\\|\\leq\\sup\_\{z\\in\\mathbb\{R\}\}\|\\sigma^\{\\prime\}\(z\)\|\\sum\_\{i=1\}^\{N\}\\\|\\alpha\_\{i\}\\\|\\,\\\|u\_\{i\}\\\|<\\infty\.Hence every finite neural network in this class is globally Lipschitz continuous with respect to the spatial variablexx, uniformly int∈\[0,T\]t\\in\[0,T\]\. Consequently, the approximating network provided by Hornik’s theorem also satisfies Assumption[1](https://arxiv.org/html/2607.22199#Thmassumption1)\.

The global Lipschitz assumption on the score function is commonly adopted in the analysis of reverse\-time stochastic differential equations \(see, e\.g\.,\[[1](https://arxiv.org/html/2607.22199#bib.bib1),[7](https://arxiv.org/html/2607.22199#bib.bib7),[10](https://arxiv.org/html/2607.22199#bib.bib10)\]\)\. Although this assumption does not hold for arbitrary data distributions, it is satisfied in several important settings\.

1. 1\.If the marginal distributionptp\_\{t\}is Gaussian, then s​\(x,t\)=∇xlog⁡pt​\(x\)s\(x,t\)=\\nabla\_\{x\}\\log p\_\{t\}\(x\)is an affine function ofxx, and therefore is globally Lipschitz\.
2. 2\.More generally, suppose the marginal distributions are uniformly strongly log\-concave, i\.e\., pt​\(x\)=e−Vt​\(x\),p\_\{t\}\(x\)=e^\{\-V\_\{t\}\(x\)\},where the potential functionVtV\_\{t\}satisfies sup\(x,t\)∈ℝd×\[0,T\]‖∇x2Vt​\(x\)‖<∞\.\\sup\_\{\(x,t\)\\in\\mathbb\{R\}^\{d\}\\times\[0,T\]\}\\\|\\nabla\_\{x\}^\{2\}V\_\{t\}\(x\)\\\|<\\infty\.Then s​\(x,t\)=−∇xVt​\(x\),s\(x,t\)=\-\\nabla\_\{x\}V\_\{t\}\(x\),and hence ∇xs​\(x,t\)=−∇x2Vt​\(x\)\.\\nabla\_\{x\}s\(x,t\)=\-\\nabla\_\{x\}^\{2\}V\_\{t\}\(x\)\.Consequently, sup\(x,t\)∈ℝd×\[0,T\]‖∇xs​\(x,t\)‖<∞,\\sup\_\{\(x,t\)\\in\\mathbb\{R\}^\{d\}\\times\[0,T\]\}\\\|\\nabla\_\{x\}s\(x,t\)\\\|<\\infty,which implies thats​\(⋅,t\)s\(\\cdot,t\)is globally Lipschitz uniformly overt∈\[0,T\]t\\in\[0,T\]\.

###### Assumption 2\(Novikov Condition\)\.

The exponential martingale associated with the drift difference satisfies Novikov’s condition:

𝔼P​\[exp⁡\(12​∫0Tg2​\(t\)​‖sθ​\(Xt,t\)−s​\(Xt,t\)‖2​𝑑t\)\]<∞\.\\mathbb\{E\}\_\{P\}\\left\[\\exp\\left\(\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}g^\{2\}\(t\)\\\|s\_\{\\theta\}\(X\_\{t\},t\)\-s\(X\_\{t\},t\)\\\|^\{2\}dt\\right\)\\right\]<\\infty\.

Assumption[2](https://arxiv.org/html/2607.22199#Thmassumption2)ensures that the exponential local martingale is a true martingale, allowing the application of Girsanov’s theorem\.

###### Example 1\(Variance\-Preserving Diffusion Models\)\.

For the variance\-preserving \(VP\) diffusion introduced by Ho et al\., the forward process is Gaussian:

Xt=αt​X0\+σt​Z,Z∼𝒩​\(0,I\)\.X\_\{t\}=\\alpha\_\{t\}X\_\{0\}\+\\sigma\_\{t\}Z,\\qquad Z\\sim\\mathcal\{N\}\(0,I\)\.If the data distribution is sub\-Gaussian and the score network has at most linear growth,

‖sθ​\(x,t\)‖≤C​\(1\+‖x‖\),\\displaystyle\\\|s\_\{\\theta\}\(x,t\)\\\|\\leq C\(1\+\\\|x\\\|\),\(1\)then the Gaussian tails ofXtX\_\{t\}imply the required exponential integrability, and Assumption[2](https://arxiv.org/html/2607.22199#Thmassumption2)holds\. The growth condition

‖sθ​\(x,t\)‖≤C​\(1\+‖x‖\)\\\|s\_\{\\theta\}\(x,t\)\\\|\\leq C\(1\+\\\|x\\\|\)is satisfied by a broad class of neural\-network architectures\. Indeed, finite\-depth feedforward networks with affine layers and activations satisfying

\|σ​\(z\)\|≤a\+b​\|z\|\|\\sigma\(z\)\|\\leq a\+b\|z\|grow at most linearly with respect to the input, by induction over the network layers\. This includes ReLU, GELU, SiLU, Softplus, ELU, and Leaky\-ReLU activations\. Moreover, finite\-depth convolutional networks \(including U\-Nets\) satisfy the same property because convolution is a bounded linear operator\. Finally, Transformer score networks equipped with bounded projection matrices also satisfy linear growth since the softmax attention operator is uniformly bounded\.

###### Example 2\(Variance\-Exploding Diffusion Models\)\.

For the variance\-exploding \(VE\) diffusion considered by Song et al\.\[[13](https://arxiv.org/html/2607.22199#bib.bib13)\], the forward process is

Xt=X0\+∫0tg​\(s\)​𝑑Ws,X\_\{t\}=X\_\{0\}\+\\int\_\{0\}^\{t\}g\(s\)\\,dW\_\{s\},whose marginal distribution is the Gaussian convolution

pt=μ∗𝒩​\(0,∫0tg2​\(s\)​𝑑s​I\)\.p\_\{t\}=\\mu\*\\mathcal\{N\}\\\!\\left\(0,\\int\_\{0\}^\{t\}g^\{2\}\(s\)\\,ds\\,I\\right\)\.Suppose that the score function satisfies the linear\-growth condition \([1](https://arxiv.org/html/2607.22199#S4.E1)\)\. If the data distribution possesses finite exponential moments, then every marginal distributionptp\_\{t\}also possesses finite exponential moments because Gaussian convolution preserves exponential integrability\. Consequently, the exponential integrability required in Assumption[2](https://arxiv.org/html/2607.22199#Thmassumption2)can be verified under these conditions\.

## 5Score Approximation to Path Measure Approximation

In this section, we establish a quantitative relationship between the approximation error of the score function and the discrepancy between the corresponding diffusion path measures\. The key result shows that the Kullback–Leibler divergence between the true and approximate diffusion processes is completely characterized by theL2L^\{2\}approximation error of their score functions\. This provides the fundamental link between function approximation and distribution approximation developed in this paper\.

Throughout the remainder of this paper, let

d​μ​\(x,t\)=pt​\(x\)​d​x​d​t,d\\mu\(x,t\)=p\_\{t\}\(x\)\\,dx\\,dt,wherept​\(x\)p\_\{t\}\(x\)denotes the marginal density of the forward diffusion process\. Sinceptp\_\{t\}is a probability density for everyt∈\[0,T\]t\\in\[0,T\],

∫ℝdpt​\(x\)​𝑑x=1,\\int\_\{\\mathbb\{R\}^\{d\}\}p\_\{t\}\(x\)\\,dx=1,the measureμ\\muis a finite Borel measure onℝd×\[0,T\]\\mathbb\{R\}^\{d\}\\times\[0,T\]with total mass

μ​\(ℝd×\[0,T\]\)=∫0T∫ℝdpt​\(x\)​𝑑x​𝑑t=T\.\\mu\(\\mathbb\{R\}^\{d\}\\times\[0,T\]\)=\\int\_\{0\}^\{T\}\\int\_\{\\mathbb\{R\}^\{d\}\}p\_\{t\}\(x\)\\,dx\\,dt=T\.Consequently, the Hilbert spaceL2​\(μ\)dL^\{2\}\(\\mu\)^\{d\}is well defined\.

We consider the true reverse\-time stochastic differential equation

d​Xt=\(f​\(Xt,t\)−g2​\(t\)​s​\(Xt,t\)\)​d​t\+g​\(t\)​d​W¯t,dX\_\{t\}=\\bigl\(f\(X\_\{t\},t\)\-g^\{2\}\(t\)s\(X\_\{t\},t\)\\bigr\)\\,dt\+g\(t\)\\,d\\bar\{W\}\_\{t\},and the approximate reverse\-time stochastic differential equation

d​Xtθ=\(f​\(Xtθ,t\)−g2​\(t\)​sθ​\(Xtθ,t\)\)​d​t\+g​\(t\)​d​W¯t,dX\_\{t\}^\{\\theta\}=\\bigl\(f\(X\_\{t\}^\{\\theta\},t\)\-g^\{2\}\(t\)s\_\{\\theta\}\(X\_\{t\}^\{\\theta\},t\)\\bigr\)\\,dt\+g\(t\)\\,d\\bar\{W\}\_\{t\},where both equations are driven by the same Brownian motionW¯t\\bar\{W\}\_\{t\}on the interval\[0,T\]\[0,T\]\.

First, we prove the following lemma\.

###### Lemma 1\.

Suppose that Assumption[1](https://arxiv.org/html/2607.22199#Thmassumption1)holds\. Then both reverse SDEs admit unique strong solutions, meaning the path measuresPPandQθQ\_\{\\theta\}are well\-defined onC​\(\[0,T\];ℝd\)C\(\[0,T\];\\mathbb\{R\}^\{d\}\)\.

###### Proof\.

Let us check the conditions for the true reverse driftb​\(x,t\)=f​\(x,t\)−g​\(t\)2​s​\(x,t\)b\(x,t\)=f\(x,t\)\-g\(t\)^\{2\}s\(x,t\)\. For anyx,y∈ℝdx,y\\in\\mathbb\{R\}^\{d\}, by the triangle inequality:

‖b​\(x,t\)−b​\(y,t\)‖≤‖f​\(x,t\)−f​\(y,t\)‖\+g​\(t\)2​‖s​\(x,t\)−s​\(y,t\)‖\.\\displaystyle\\\|b\(x,t\)\-b\(y,t\)\\\|\\leq\\\|f\(x,t\)\-f\(y,t\)\\\|\+g\(t\)^\{2\}\\\|s\(x,t\)\-s\(y,t\)\\\|\.\(2\)By assumption,ffandssare globally Lipschitz continuous in their spatial arguments with constantsLfL\_\{f\}andLsL\_\{s\}\. Sinceg​\(t\)g\(t\)is continuous on the compact interval\[0,T\]\[0,T\], it is bounded by somegmax=maxt∈\[0,T\]⁡\|g​\(t\)\|g\_\{\\max\}=\\max\_\{t\\in\[0,T\]\}\|g\(t\)\|\. Thus:

‖b​\(x,t\)−b​\(y,t\)‖≤\(Lf\+gmax2​Ls\)​‖x−y‖\.\\displaystyle\\\|b\(x,t\)\-b\(y,t\)\\\|\\leq\\left\(L\_\{f\}\+g\_\{\\max\}^\{2\}L\_\{s\}\\right\)\\\|x\-y\\\|\.\(3\)Therefore,b​\(x,t\)b\(x,t\)is globally Lipschitz inxx, uniformly intt\. Combining this property with the boundedness ofb​\(0,t\)b\(0,t\)immediately implies the linear growth condition:

‖b​\(x,t\)‖≤‖b​\(0,t\)‖\+\(Lf\+gmax2​Ls\)​‖x‖≤C​\(1\+‖x‖\)\.\\displaystyle\\\|b\(x,t\)\\\|\\leq\\\|b\(0,t\)\\\|\+\\left\(L\_\{f\}\+g\_\{\\max\}^\{2\}L\_\{s\}\\right\)\\\|x\\\|\\leq C\(1\+\\\|x\\\|\)\.\(4\)The diffusion coefficientσ​\(x,t\)=g​\(t\)​𝐈d\\sigma\(x,t\)=g\(t\)\\mathbf\{I\}\_\{d\}is spatially constant, meaning it is trivially Lipschitz continuous with constant 0, and bounded\.

By the standard Itô SDE Existence and Uniqueness Theorem\[[11](https://arxiv.org/html/2607.22199#bib.bib11), Theorem 5\.2\.1\], given a well\-defined initial conditionXT∼pTX\_\{T\}\\sim p\_\{T\}, there exists a unique strong solutionXtX\_\{t\}fort∈\[0,T\]t\\in\[0,T\]\. An identical argument applies to the approximate driftbθ​\(x,t\)=f​\(x,t\)−g​\(t\)2​sθ​\(x,t\)b\_\{\\theta\}\(x,t\)=f\(x,t\)\-g\(t\)^\{2\}s\_\{\\theta\}\(x,t\)sincesθs\_\{\\theta\}is also globally Lipschitz\. Consequently, the corresponding stochastic processes\(Xt\)0≤t≤T\(X\_\{t\}\)\_\{0\\leq t\\leq T\}and\(Xtθ\)0≤t≤T\(X\_\{t\}^\{\\theta\}\)\_\{0\\leq t\\leq T\}induce unique probability measuresPPandQθQ\_\{\\theta\}on the path spaceC​\(\[0,T\];ℝd\)C\(\[0,T\];\\mathbb\{R\}^\{d\}\)\. ∎

###### Proposition 1\(Path\-space Stability under Score Approximation\)\.

Assume that the reverse\-time SDEs satisfy Assumptions[1](https://arxiv.org/html/2607.22199#Thmassumption1)and[2](https://arxiv.org/html/2607.22199#Thmassumption2)\. Then the path measuresPPandQθQ\_\{\\theta\}satisfy

KL\(P∥Qθ\)=KL\(pT∥pprior\)\+12∫0Tg2\(t\)𝔼pt\[∥sθ\(Xt,t\)−s\(Xt,t\)∥2\]dt\.\\boxed\{\\operatorname\{KL\}\(P\\\|Q\_\{\\theta\}\)=\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\+\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}g^\{2\}\(t\)\\mathbb\{E\}\_\{p\_\{t\}\}\\\!\\left\[\\\|s\_\{\\theta\}\(X\_\{t\},t\)\-s\(X\_\{t\},t\)\\\|^\{2\}\\right\]dt\.\}

###### Proof\.

By Lemma[1](https://arxiv.org/html/2607.22199#Thmlemma1), both reverse\-time SDEs admit unique strong solutions\. Therefore, the corresponding probability laws are uniquely determined\.

Let

\(Ω,ℱ,\(ℱt\)0≤t≤T,P,W¯\)\(\\Omega,\\mathcal\{F\},\(\\mathcal\{F\}\_\{t\}\)\_\{0\\leq t\\leq T\},P,\\bar\{W\}\)be a filtered probability space supporting the unique strong solution of the true reverse\-time SDE

d​Xt=\(f​\(Xt,t\)−g2​\(t\)​s​\(Xt,t\)\)​d​t\+g​\(t\)​d​W¯t\.dX\_\{t\}=\\bigl\(f\(X\_\{t\},t\)\-g^\{2\}\(t\)s\(X\_\{t\},t\)\\bigr\)\\,dt\+g\(t\)\\,d\\bar\{W\}\_\{t\}\.
On the same filtered probability space, consider the auxiliary SDE

d​Yt=\(f​\(Yt,t\)−g2​\(t\)​sθ​\(Yt,t\)\)​d​t\+g​\(t\)​d​W¯t,dY\_\{t\}=\\bigl\(f\(Y\_\{t\},t\)\-g^\{2\}\(t\)s\_\{\\theta\}\(Y\_\{t\},t\)\\bigr\)\\,dt\+g\(t\)\\,d\\bar\{W\}\_\{t\},
with terminal condition

Since the coefficients satisfy Assumption[1](https://arxiv.org/html/2607.22199#Thmassumption1), by Lemma[1](https://arxiv.org/html/2607.22199#Thmlemma1), the above SDE admits a unique strong solution\.

Observe that this auxiliary SDE has exactly the same drift, diffusion coefficient, and terminal distribution as the learned reverse\-time SDE

d​Xtθ=\(f​\(Xtθ,t\)−g2​\(t\)​sθ​\(Xtθ,t\)\)​d​t\+g​\(t\)​d​W¯t\.dX\_\{t\}^\{\\theta\}=\\bigl\(f\(X\_\{t\}^\{\\theta\},t\)\-g^\{2\}\(t\)s\_\{\\theta\}\(X\_\{t\}^\{\\theta\},t\)\\bigr\)\\,dt\+g\(t\)\\,d\\bar\{W\}\_\{t\}\.
Since strong uniqueness holds, every strong solution of the learned reverse\-time SDE defined on an arbitrary filtered probability space has the same probability law\. Therefore,

ℒ​\(Y\)=ℒ​\(Xθ\),\\mathcal\{L\}\(Y\)=\\mathcal\{L\}\(X^\{\\theta\}\),and hence the auxiliary processYYinduces precisely the path measureQθQ\_\{\\theta\}\.

Consequently, it is sufficient to compare the processesXXandYY, which are realized on the same filtered probability space and driven by the same Brownian motionW¯t\\bar\{W\}\_\{t\}\.

For every fixed terminal valueXT=xX\_\{T\}=x, both processes start from the same deterministic terminal state\. Therefore, under Assumption[2](https://arxiv.org/html/2607.22199#Thmassumption2), Girsanov’s theorem\[[11](https://arxiv.org/html/2607.22199#bib.bib11), Theorem 8\.6\.4\]applies to the conditional path measures

P\(⋅∣XT=x\)andQθ\(⋅∣XT=x\)\.P\(\\cdot\\mid X\_\{T\}=x\)\\quad\\text\{and\}\\quad Q\_\{\\theta\}\(\\cdot\\mid X\_\{T\}=x\)\.
Define

ut=g​\(t\)​\(sθ​\(Xt,t\)−s​\(Xt,t\)\)\.u\_\{t\}=g\(t\)\\bigl\(s\_\{\\theta\}\(X\_\{t\},t\)\-s\(X\_\{t\},t\)\\bigr\)\.
By Girsanov’s theorem\[[11](https://arxiv.org/html/2607.22199#bib.bib11), Theorem 8\.6\.4\],

dP\(⋅∣XT\)dQθ\(⋅∣XT\)=exp⁡\(∫0Tut​𝑑W¯t\+12​∫0T‖ut‖2​𝑑t\)\.\\displaystyle\\frac\{dP\(\\cdot\\mid X\_\{T\}\)\}\{dQ\_\{\\theta\}\(\\cdot\\mid X\_\{T\}\)\}=\\exp\\\!\\left\(\\int\_\{0\}^\{T\}u\_\{t\}\\,d\\bar\{W\}\_\{t\}\+\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}\\\|u\_\{t\}\\\|^\{2\}\\,dt\\right\)\.\(5\)
Using Bayes’ rule,

P​\(d​X\)=P​\(d​X∣XT\)​pT​\(XT\),P\(dX\)=P\(dX\\mid X\_\{T\}\)\\,p\_\{T\}\(X\_\{T\}\),
and

Qθ​\(d​X\)=Qθ​\(d​X∣XT\)​pprior​\(XT\),Q\_\{\\theta\}\(dX\)=Q\_\{\\theta\}\(dX\\mid X\_\{T\}\)\\,p\_\{\\mathrm\{prior\}\}\(X\_\{T\}\),
we obtain

d​Pd​Qθ=dP\(⋅∣XT\)dQθ\(⋅∣XT\)⋅pT​\(XT\)pprior​\(XT\)\.\\displaystyle\\frac\{dP\}\{dQ\_\{\\theta\}\}=\\frac\{dP\(\\cdot\\mid X\_\{T\}\)\}\{dQ\_\{\\theta\}\(\\cdot\\mid X\_\{T\}\)\}\\cdot\\frac\{p\_\{T\}\(X\_\{T\}\)\}\{p\_\{\\mathrm\{prior\}\}\(X\_\{T\}\)\}\.\(6\)
Therefore, from \([5](https://arxiv.org/html/2607.22199#S5.E5)\) and \([6](https://arxiv.org/html/2607.22199#S5.E6)\) we obtain

d​Pd​Qθ=pT​\(XT\)pprior​\(XT\)​exp⁡\(∫0Tut​𝑑W¯t\+12​∫0T‖ut‖2​𝑑t\)\.\\displaystyle\\frac\{dP\}\{dQ\_\{\\theta\}\}=\\frac\{p\_\{T\}\(X\_\{T\}\)\}\{p\_\{\\mathrm\{prior\}\}\(X\_\{T\}\)\}\\exp\\\!\\left\(\\int\_\{0\}^\{T\}u\_\{t\}\\,d\\bar\{W\}\_\{t\}\+\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}\\\|u\_\{t\}\\\|^\{2\}\\,dt\\right\)\.\(7\)
Taking logarithms,

log⁡d​Pd​Qθ=log⁡pT​\(XT\)pprior​\(XT\)\+∫0Tut​𝑑W¯t\+12​∫0T‖ut‖2​𝑑t\.\\displaystyle\\log\\frac\{dP\}\{dQ\_\{\\theta\}\}=\\log\\frac\{p\_\{T\}\(X\_\{T\}\)\}\{p\_\{\\mathrm\{prior\}\}\(X\_\{T\}\)\}\+\\int\_\{0\}^\{T\}u\_\{t\}\\,d\\bar\{W\}\_\{t\}\+\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}\\\|u\_\{t\}\\\|^\{2\}\\,dt\.\(8\)
By taking expectation with respect toPP, from \([8](https://arxiv.org/html/2607.22199#S5.E8)\) we have

KL⁡\(P∥Qθ\)=𝔼P​\[log⁡pT​\(XT\)pprior​\(XT\)\]\+𝔼P​\[∫0Tut​𝑑W¯t\]\+12​𝔼P​\[∫0T‖ut‖2​𝑑t\]\.\\displaystyle\\begin\{aligned\} \\operatorname\{KL\}\(P\\\|Q\_\{\\theta\}\)&=\\mathbb\{E\}\_\{P\}\\\!\\left\[\\log\\frac\{p\_\{T\}\(X\_\{T\}\)\}\{p\_\{\\mathrm\{prior\}\}\(X\_\{T\}\)\}\\right\]\\\\ &\\quad\+\\mathbb\{E\}\_\{P\}\\\!\\left\[\\int\_\{0\}^\{T\}u\_\{t\}\\,d\\bar\{W\}\_\{t\}\\right\]\\\\ &\\quad\+\\frac\{1\}\{2\}\\mathbb\{E\}\_\{P\}\\\!\\left\[\\int\_\{0\}^\{T\}\\\|u\_\{t\}\\\|^\{2\}\\,dt\\right\]\.\\end\{aligned\}\(9\)Sinceutu\_\{t\}is progressively measurable and square integrable, the Itô integral

∫0Tut​𝑑W¯t\\int\_\{0\}^\{T\}u\_\{t\}\\,d\\bar\{W\}\_\{t\}is a square\-integrable martingale \(see, e\.g\.,\[[11](https://arxiv.org/html/2607.22199#bib.bib11), Chapter 3\]\)\. Therefore,

𝔼P​\[∫0Tut​𝑑W¯t\]=0\.\\displaystyle\\mathbb\{E\}\_\{P\}\\\!\\left\[\\int\_\{0\}^\{T\}u\_\{t\}\\,d\\bar\{W\}\_\{t\}\\right\]=0\.\(10\)
Moreover,

𝔼P​\[log⁡pT​\(XT\)pprior​\(XT\)\]=KL⁡\(pT∥pprior\)\.\\displaystyle\\mathbb\{E\}\_\{P\}\\\!\\left\[\\log\\frac\{p\_\{T\}\(X\_\{T\}\)\}\{p\_\{\\mathrm\{prior\}\}\(X\_\{T\}\)\}\\right\]=\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\.\(11\)
Finally,

KL⁡\(P∥Qθ\)=KL⁡\(pT∥pprior\)\+12​∫0Tg2​\(t\)​𝔼pt​\[‖sθ​\(Xt,t\)−s​\(Xt,t\)‖2\]​𝑑t,\\displaystyle\\begin\{aligned\} \\operatorname\{KL\}\(P\\\|Q\_\{\\theta\}\)&=\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\\\\ &\\quad\+\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}g^\{2\}\(t\)\\,\\mathbb\{E\}\_\{p\_\{t\}\}\\\!\\left\[\\\|s\_\{\\theta\}\(X\_\{t\},t\)\-s\(X\_\{t\},t\)\\\|^\{2\}\\right\]dt,\\end\{aligned\}\(12\)
which completes the proof\.

∎

## 6Path Measure Approximation to Distribution Approximation

The previous section established an upper bound on the Kullback–Leibler divergence between the path measures of the true and learned reverse diffusion processes\. We now show that this immediately implies an approximation result for the generated data distributions\.

###### Lemma 2\(Marginal Radon–Nikodym Identity\)\.

LetPPandQθQ\_\{\\theta\}denote the path measures of the true and learned reverse diffusion processes, and letμ\\muandμθ\\mu\_\{\\theta\}denote their corresponding marginal distributions at timet=0t=0\. Then

d​μd​μθ\(X0\)=𝔼Qθ\[d​Pd​Qθ\|X0\]\.\\boxed\{\\frac\{d\\mu\}\{d\\mu\_\{\\theta\}\}\(X\_\{0\}\)=\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\.\}

###### Proof\.

Letf:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\rightarrow\\mathbb\{R\}be any bounded measurable function\. Sinceμ\\muis the marginal of the path measurePP,

𝔼μ​\[f​\(X0\)\]=𝔼P​\[f​\(X0\)\]\.\\displaystyle\\mathbb\{E\}\_\{\\mu\}\[f\(X\_\{0\}\)\]=\\mathbb\{E\}\_\{P\}\[f\(X\_\{0\}\)\]\.\(13\)
Applying the Radon–Nikodym theorem gives

𝔼P​\[f​\(X0\)\]=𝔼Qθ​\[f​\(X0\)​d​Pd​Qθ\]\.\\displaystyle\\mathbb\{E\}\_\{P\}\[f\(X\_\{0\}\)\]=\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[f\(X\_\{0\}\)\\frac\{dP\}\{dQ\_\{\\theta\}\}\\right\]\.\(14\)
Using the tower property,

𝔼Qθ​\[f​\(X0\)​d​Pd​Qθ\]\\displaystyle\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[f\(X\_\{0\}\)\\frac\{dP\}\{dQ\_\{\\theta\}\}\\right\]=𝔼Qθ\[𝔼Qθ\[f\(X0\)d​Pd​Qθ\|X0\]\]\\displaystyle=\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[f\(X\_\{0\}\)\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\\right\]=𝔼Qθ\[f\(X0\)𝔼Qθ\[d​Pd​Qθ\|X0\]\],\\displaystyle=\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[f\(X\_\{0\}\)\\,\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\\right\],\(15\)
sincef​\(X0\)f\(X\_\{0\}\)is measurable with respect toσ​\(X0\)\\sigma\(X\_\{0\}\)\.

Therefore,

∫f\(x\)dμ\(x\)=∫f\(x\)𝔼Qθ\[d​Pd​Qθ\|X0=x\]dμθ\(x\)\.\\displaystyle\\int f\(x\)\\,d\\mu\(x\)=\\int f\(x\)\\,\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}=x\\right\]d\\mu\_\{\\theta\}\(x\)\.\(16\)
On the other hand,

∫f​\(x\)​𝑑μ​\(x\)=∫f​\(x\)​d​μd​μθ​\(x\)​𝑑μθ​\(x\)\.\\displaystyle\\int f\(x\)\\,d\\mu\(x\)=\\int f\(x\)\\frac\{d\\mu\}\{d\\mu\_\{\\theta\}\}\(x\)\\,d\\mu\_\{\\theta\}\(x\)\.\(17\)
From \([16](https://arxiv.org/html/2607.22199#S6.E16)\) and \([17](https://arxiv.org/html/2607.22199#S6.E17)\) we obtain

∫f\(x\)𝔼Qθ\[d​Pd​Qθ\|X0=x\]dμθ\(x\)=∫f\(x\)d​μd​μθ\(x\)dμθ\(x\)\.\\displaystyle\\int f\(x\)\\,\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}=x\\right\]d\\mu\_\{\\theta\}\(x\)=\\int f\(x\)\\frac\{d\\mu\}\{d\\mu\_\{\\theta\}\}\(x\)\\,d\\mu\_\{\\theta\}\(x\)\.\(18\)Since the identity \([18](https://arxiv.org/html/2607.22199#S6.E18)\) holds for every bounded measurable functionff, the Radon–Nikodym derivatives coincideμθ\\mu\_\{\\theta\}\-almost everywhere, proving

d​μd​μθ\(X0\)=𝔼Qθ\[d​Pd​Qθ\|X0\]\.\\displaystyle\\frac\{d\\mu\}\{d\\mu\_\{\\theta\}\}\(X\_\{0\}\)=\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\.\(19\)
∎

###### Proposition 2\.

The Kullback–Leibler divergence between the generated data distributions is bounded by the Kullback–Leibler divergence between the corresponding path measures:

KL\(μ∥μθ\)≤KL\(P∥Qθ\)\.\\boxed\{\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)\\leq\\operatorname\{KL\}\(P\\\|Q\_\{\\theta\}\)\.\}

###### Proof\.

By Lemma[2](https://arxiv.org/html/2607.22199#Thmlemma2),

d​μd​μθ\(X0\)=𝔼Qθ\[d​Pd​Qθ\|X0\]\.\\displaystyle\\frac\{d\\mu\}\{d\\mu\_\{\\theta\}\}\(X\_\{0\}\)=\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\.\(20\)
Hence,

KL\(μ∥μθ\)=𝔼P\[log𝔼Qθ\[d​Pd​Qθ\|X0\]\]\.\\displaystyle\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)=\\mathbb\{E\}\_\{P\}\\left\[\\log\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\\right\]\.\(21\)
Applying Jensen’s inequality to the concave functionlog\\logyields

log𝔼Qθ\[d​Pd​Qθ\|X0\]≤𝔼P\[logd​Pd​Qθ\|X0\]\.\\displaystyle\\log\\mathbb\{E\}\_\{Q\_\{\\theta\}\}\\left\[\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\\leq\\mathbb\{E\}\_\{P\}\\left\[\\log\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\.\(22\)
From \([21](https://arxiv.org/html/2607.22199#S6.E21)\) and \([22](https://arxiv.org/html/2607.22199#S6.E22)\) we have

KL⁡\(μ∥μθ\)\\displaystyle\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)≤𝔼P\[𝔼P\[logd​Pd​Qθ\|X0\]\]\\displaystyle\\leq\\mathbb\{E\}\_\{P\}\\left\[\\mathbb\{E\}\_\{P\}\\left\[\\log\\frac\{dP\}\{dQ\_\{\\theta\}\}\\,\\middle\|\\,X\_\{0\}\\right\]\\right\]=𝔼P​\[log⁡d​Pd​Qθ\]\\displaystyle=\\mathbb\{E\}\_\{P\}\\left\[\\log\\frac\{dP\}\{dQ\_\{\\theta\}\}\\right\]=KL⁡\(P∥Qθ\),\\displaystyle=\\operatorname\{KL\}\(P\\\|Q\_\{\\theta\}\),\(23\)
which completes the proof\.

∎

## 7From Score Approximation to Distribution Approximation

Let

𝒳=ℝd×\[0,T\]\.\\mathcal\{X\}=\\mathbb\{R\}^\{d\}\\times\[0,T\]\.We consider the vector\-valued Hilbert spaceL2​\(μ\)dL^\{2\}\(\\mu\)^\{d\}equipped with the norm

‖g‖L2​\(μ\)d2=∫0T𝔼pt​\[‖g​\(Xt,t\)‖2\]​𝑑t=∫0T∫ℝd‖g​\(x,t\)‖2​pt​\(x\)​𝑑x​𝑑t\.\\\|g\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}^\{2\}=\\int\_\{0\}^\{T\}\\mathbb\{E\}\_\{p\_\{t\}\}\\\!\\left\[\\\|g\(X\_\{t\},t\)\\\|^\{2\}\\right\]dt=\\int\_\{0\}^\{T\}\\int\_\{\\mathbb\{R\}^\{d\}\}\\\|g\(x,t\)\\\|^\{2\}p\_\{t\}\(x\)\\,dx\\,dt\.
Since the true score function

s​\(x,t\)=∇xlog⁡pt​\(x\)s\(x,t\)=\\nabla\_\{x\}\\log p\_\{t\}\(x\)belongs toL2​\(μ\)dL^\{2\}\(\\mu\)^\{d\}, Hornik’s universal approximation theorem\[[9](https://arxiv.org/html/2607.22199#bib.bib9), Theorem 1\]implies that feedforward neural networks with an unbounded and nonconstant activation function \(e\.g\. softplus\) are dense inL2​\(μ\)dL^\{2\}\(\\mu\)^\{d\}\. Consequently, for everyε\>0\\varepsilon\>0, there exists a neural networksθs\_\{\\theta\}, which is globally Lipschitz inxx, uniformly int∈\[0,T\]t\\in\[0,T\], satisfying

‖sθ−s‖L2​\(μ\)d2<ε\.\\\|s\_\{\\theta\}\-s\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}^\{2\}<\\varepsilon\.\(24\)\(See the discussion in Assumption[1](https://arxiv.org/html/2607.22199#Thmassumption1)\)\.

###### Theorem 1\(Approximation of Reverse Diffusion Models\)\.

Letμ∈𝒫​\(ℝd\)\\mu\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{d\}\)be the data distribution and letptp\_\{t\}denote the marginal density of the forward diffusion process with score function

s​\(x,t\)=∇xlog⁡pt​\(x\)∈L2​\(μ\)d\.s\(x,t\)=\\nabla\_\{x\}\\log p\_\{t\}\(x\)\\in L^\{2\}\(\\mu\)^\{d\}\.
Assume that

1. 1\.The driftf​\(⋅,t\)f\(\\cdot,t\)and the score functions​\(⋅,t\)s\(\\cdot,t\)satisfy all the conditions in Assumptions[1](https://arxiv.org/html/2607.22199#Thmassumption1)and[2](https://arxiv.org/html/2607.22199#Thmassumption2)\.
2. 2\.The reverse diffusion is initialized from a prior distributionppriorp\_\{\\mathrm\{prior\}\}satisfying KL⁡\(pT∥pprior\)≤δ\.\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\\leq\\delta\.

Then, for everyε\>0\\varepsilon\>0, there exists a neural networksθs\_\{\\theta\}satisfying all the conditions in Assumptions[1](https://arxiv.org/html/2607.22199#Thmassumption1)and[2](https://arxiv.org/html/2607.22199#Thmassumption2)and

‖sθ−s‖L2​\(μ\)d2<ε,\\\|s\_\{\\theta\}\-s\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}^\{2\}<\\varepsilon,such that the probability distributionμθ\\mu\_\{\\theta\}generated by the learned reverse diffusion model satisfies

KL\(μ∥μθ\)<δ\+12gmax2ε,\\boxed\{\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)<\\delta\+\\frac\{1\}\{2\}g\_\{\\max\}^\{2\}\\varepsilon,\}where

gmax=maxt∈\[0,T\]⁡\|g​\(t\)\|\.g\_\{\\max\}=\\max\_\{t\\in\[0,T\]\}\|g\(t\)\|\.

###### Proof\.

Letsθs\_\{\\theta\}be the neural network satisfying all the conditions in Assumptions[1](https://arxiv.org/html/2607.22199#Thmassumption1)and[2](https://arxiv.org/html/2607.22199#Thmassumption2)and

‖sθ−s‖L2​\(μ\)d2<ε,\\\|s\_\{\\theta\}\-s\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}^\{2\}<\\varepsilon,whose existence is guaranteed by \([24](https://arxiv.org/html/2607.22199#S7.E24)\)\.

By Proposition[2](https://arxiv.org/html/2607.22199#Thmproposition2)and Proposition[1](https://arxiv.org/html/2607.22199#Thmproposition1),

KL⁡\(μ∥μθ\)\\displaystyle\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)≤KL⁡\(P∥Qθ\)\\displaystyle\\leq\\operatorname\{KL\}\(P\\\|Q\_\{\\theta\}\)≤KL⁡\(pT∥pprior\)\+12​∫0Tg2​\(t\)​𝔼pt​\[‖sθ​\(Xt,t\)−s​\(Xt,t\)‖2\]​𝑑t\.\\displaystyle\\leq\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\+\\frac\{1\}\{2\}\\int\_\{0\}^\{T\}g^\{2\}\(t\)\\,\\mathbb\{E\}\_\{p\_\{t\}\}\\\!\\left\[\\\|s\_\{\\theta\}\(X\_\{t\},t\)\-s\(X\_\{t\},t\)\\\|^\{2\}\\right\]dt\.\(25\)
Since

g2​\(t\)≤gmax2,∀t∈\[0,T\],g^\{2\}\(t\)\\leq g\_\{\\max\}^\{2\},\\qquad\\forall\\,t\\in\[0,T\],we obtain

KL⁡\(μ∥μθ\)\\displaystyle\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)≤KL⁡\(pT∥pprior\)\+12​gmax2​∫0T𝔼pt​\[‖sθ​\(Xt,t\)−s​\(Xt,t\)‖2\]​𝑑t\\displaystyle\\leq\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\+\\frac\{1\}\{2\}g\_\{\\max\}^\{2\}\\int\_\{0\}^\{T\}\\mathbb\{E\}\_\{p\_\{t\}\}\\\!\\left\[\\\|s\_\{\\theta\}\(X\_\{t\},t\)\-s\(X\_\{t\},t\)\\\|^\{2\}\\right\]dt=KL⁡\(pT∥pprior\)\+12​gmax2​‖sθ−s‖L2​\(μ\)d2\.\\displaystyle=\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\+\\frac\{1\}\{2\}g\_\{\\max\}^\{2\}\\\|s\_\{\\theta\}\-s\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}^\{2\}\.\(26\)
Finally, using

KL⁡\(pT∥pprior\)≤δ\\operatorname\{KL\}\(p\_\{T\}\\\|p\_\{\\mathrm\{prior\}\}\)\\leq\\deltaand

‖sθ−s‖L2​\(μ\)d2<ε,\\\|s\_\{\\theta\}\-s\\\|\_\{L^\{2\}\(\\mu\)^\{d\}\}^\{2\}<\\varepsilon,yields

KL⁡\(μ∥μθ\)<δ\+12​gmax2​ε,\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)<\\delta\+\\frac\{1\}\{2\}g\_\{\\max\}^\{2\}\\varepsilon,which completes the proof\.

∎

Theorem[1](https://arxiv.org/html/2607.22199#Thmtheorem1)provides a theoretical justification for score\-based diffusion models\. It shows that accurate approximation of the score function in theL2​\(μ\)L^\{2\}\(\\mu\)sense guarantees approximation of the generated data distribution in Kullback–Leibler divergence\. Thus, universal approximation of neural networks directly translates into universal approximation of diffusion models\.

## 8Extensions and Future Directions

The present work establishes a direct connection between approximation of the score function and approximation of the generated probability distribution in Kullback–Leibler divergence\. Although our analysis focuses on score\-based diffusion models driven by reverse\-time stochastic differential equations, the proof technique is considerably more general\. In this section, we briefly discuss several possible extensions\.

### 8\.1Alternative Probability Metrics

Our analysis employs the Kullback–Leibler divergence because Girsanov’s theorem naturally provides an explicit expression for the Radon–Nikodym derivative between the path measures\. It would be interesting to investigate whether similar approximation guarantees can be established under other probability metrics, including

Wp​\(μ,μθ\),TV​\(μ,μθ\),DJS​\(μ,μθ\),W\_\{p\}\(\\mu,\\mu\_\{\\theta\}\),\\qquad\\mathrm\{TV\}\(\\mu,\\mu\_\{\\theta\}\),\\qquad D\_\{\\mathrm\{JS\}\}\(\\mu,\\mu\_\{\\theta\}\),
whereWpW\_\{p\}denotes the Wasserstein distance,TV\\mathrm\{TV\}denotes the total variation distance, andDJSD\_\{\\mathrm\{JS\}\}denotes the Jensen–Shannon divergence\. Such results may provide stronger geometric guarantees for generative modeling\.

### 8\.2Approximation Rates

Hornik’s theorem guarantees the existence of neural network approximations but does not quantify the approximation rate\. Combining our framework with quantitative approximation theory for neural networks may lead to explicit convergence rates of the form

KL⁡\(μ∥μθ\)=O​\(N−α\),\\operatorname\{KL\}\(\\mu\\\|\\mu\_\{\\theta\}\)=O\(N^\{\-\\alpha\}\),
whereNNdenotes the network size\. Such results would provide a theoretical characterization of the expressive power of neural score models\.

### 8\.3General Diffusion Models

The present analysis considers continuous\-time score\-based diffusion models represented by stochastic differential equations\. Since denoising diffusion probabilistic models \(DDPMs\) arise as discrete approximations of these SDEs, it would be interesting to derive analogous approximation theorems directly for discrete\-time diffusion processes\.

Another promising direction is to extend the analysis to probability flow ordinary differential equations, where the reverse diffusion is replaced by a deterministic transport equation\.

### 8\.4Beyond Euclidean Data

The current framework assumes that the data lie in the Euclidean spaceℝd\\mathbb\{R\}^\{d\}\. Extending the analysis to diffusion models defined on manifolds, graphs, or other structured state spaces remains an important direction for future research\. Such extensions may provide theoretical foundations for geometric diffusion models and scientific machine learning applications\.

### 8\.5Learning Theory for Diffusion Models

The present paper establishes an approximation theorem for score\-based diffusion models\. A natural next step is to study statistical learning guarantees, including sample complexity, estimation error, and generalization error of neural score estimators\. Developing a complete statistical learning theory for diffusion models remains an important open problem\.

## References

- \[1\]Brian D\. O\. Anderson\.Reverse\-time diffusion equation models\.Stochastic Processes and their Applications, 12\(3\):313–326, 1982\.
- \[2\]Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet\.Diffusion schrödinger bridge with applications to score\-based generative modeling\.InAdvances in Neural Information Processing Systems \(NeurIPS\), 2021\.
- \[3\]Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang\.Score approximation, estimation and distribution recovery of diffusion models on low\-dimensional data\.InInternational Conference on Machine Learning \(ICML\), 2023\.
- \[4\]Thomas M\. Cover and Joy A\. Thomas\.Elements of Information Theory\.Wiley, 2 edition, 2006\.
- \[5\]George Cybenko\.Approximation by superpositions of a sigmoidal function\.Mathematics of Control, Signals and Systems, 2\(4\):303–314, 1989\.
- \[6\]U\. G\. Haussmann and E\. Pardoux\.Time reversal of diffusion processes\.InStochastic Differential Systems: Filtering and Control, volume 69 ofLecture Notes in Control and Information Sciences, pages 176–182\. Springer, 1985\.
- \[7\]U\. G\. Haussmann and E\. Pardoux\.Time reversal of diffusions\.The Annals of Probability, 14\(4\):1188–1205, 1986\.
- \[8\]Jonathan Ho, Ajay Jain, and Pieter Abbeel\.Denoising diffusion probabilistic models\.InAdvances in Neural Information Processing Systems \(NeurIPS\), 2020\.
- \[9\]Kurt Hornik\.Approximation capabilities of multilayer feedforward networks\.Neural Networks, 4\(2\):251–257, 1991\.
- \[10\]Ioannis Karatzas and Steven E\. Shreve\.Brownian Motion and Stochastic Calculus, volume 113 ofGraduate Texts in Mathematics\.Springer, New York, 2 edition, 1991\.
- \[11\]Bernt Øksendal\.Stochastic Differential Equations: An Introduction with Applications\.Springer, 5 edition, 1998\.
- \[12\]Yang Song and Stefano Ermon\.Generative modeling by estimating gradients of the data distribution\.InAdvances in Neural Information Processing Systems \(NeurIPS\), 2019\.
- \[13\]Yang Song, Jascha Sohl\-Dickstein, Diederik P\. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole\.Score\-based generative modeling through stochastic differential equations\.InInternational Conference on Learning Representations \(ICLR\), 2021\.