Quantitative Sobolev Approximation Bounds for Neural Operators with Empirical Validation on Burgers Equation

# Quantitative Sobolev Approximation Bounds for Neural Operators with Empirical Validation on Burgers’ Equation
Source: [https://arxiv.org/html/2605.08170](https://arxiv.org/html/2605.08170)
Nicole Hao Department of Mathematics, Cornell University yh397@cornell\.edu

###### Abstract

Neural operators have emerged as a powerful tool for learning mappings between infinite\-dimensional function spaces\. However, their approximation properties in Sobolev norms remain poorly quantified, even though these norms control both function values and derivatives and are the natural metrics for PDE well\-posedness, stability, and generalization\. We develop a functional\-analytic framework for operator learning in Sobolev spaces and connect it to the numerical behavior of Fourier Neural Operators \(FNOs\) on a prototypical PDE\. First, for a continuous nonlinear operator𝒢:Hs​\(D\)→Ht​\(D′\)\\mathcal\{G\}:H^\{s\}\(D\)\\to H^\{t\}\(D^\{\\prime\}\)withs\>d/2s\>d/2and inputs restricted to a compact subset ofHs​\(D\)H^\{s\}\(D\), we prove that𝒢\\mathcal\{G\}can be uniformly approximated inHtH^\{t\}\-norm by a neural operator with𝒪​\(ε−d/s\)\\mathcal\{O\}\(\\varepsilon^\{\-d/s\}\)trainable parameters\. This yields an explicit complexity–error relation of the form‖𝒢−𝒢θ‖Ht≲C​N−s/d\\\|\\mathcal\{G\}\-\\mathcal\{G\}\_\{\\theta\}\\\|\_\{H^\{t\}\}\\lesssim CN^\{\-s/d\}\. We then study the one\-dimensional viscous Burgers’ solution operator𝒢:u0↦u​\(⋅,1\)\\mathcal\{G\}:u\_\{0\}\\mapsto u\(\\cdot,1\)on a boundedH1H^\{1\}\-ball and train FNOs with anH1H^\{1\}\-loss\. Across a sweep of model sizes, we obtain testH1H^\{1\}\-errors down to𝒪​\(10−7\)\\mathcal\{O\}\(10^\{\-7\}\)and relative errors of order10−310^\{\-3\}, with predictions accurately matching both solutions and spatial derivatives on held\-out data\. A log–log plot of Sobolev error versus parameter count exhibits an approximate power law‖𝒢−𝒢θ‖H1≈C​N−α\\\|\\mathcal\{G\}\-\\mathcal\{G\}\_\{\\theta\}\\\|\_\{H^\{1\}\}\\approx CN^\{\-\\alpha\}with empirical exponentα≈1\.4\\alpha\\approx 1\.4, and long\-horizon training reveals optimization instabilities in large FNOs, providing quantitative evidence that Sobolev\-space approximation theory meaningfully predicts neural\-operator scaling behavior\.

## 1Introduction

Neural operators are powerful but poorly understood from a complexity standpoint\. In scientific machine learning, they are increasingly used as resolution\-invariant surrogates for PDE solvers, yet practitioners still choose model sizes heuristically, with little guidance on how Sobolev\-norm error scales with parameters\. We bridge that gap by proving Sobolev\-space approximation bounds and validating them on a canonical PDE, with an eye toward principled model sizing and accuracy control in operator\-learning pipelines\.

Operator learning refers to the task of approximating mappings between infinite\-dimensional function spaces, such as those arising from solutions of partial differential equations parameterized by initial conditions or coefficients\. Formally, we seek to learn an operator

𝒢:𝒳→𝒴,\\mathcal\{G\}:\\mathcal\{X\}\\to\\mathcal\{Y\},where𝒳\\mathcal\{X\}and𝒴\\mathcal\{Y\}are typically Banach or Hilbert spaces of functions, such as Sobolev spaces\.

This learning paradigm has gained significant attention in the scientific machine learning community because it can bypass traditional numerical solvers and directly model complex solution operators from data\. From the perspective of functional analysis, learning an operator between Sobolev spaces𝒳=Hs​\(D\)\\mathcal\{X\}=H^\{s\}\(D\)and𝒴=Ht​\(D′\)\\mathcal\{Y\}=H^\{t\}\(D^\{\\prime\}\)raises important questions about approximation theory, continuity, and generalization in high\-dimensional regimes\. Although neural operators like DeepONetLu et al\. \([2021](https://arxiv.org/html/2605.08170#bib.bib4)\)show empirical success, their approximation properties in Sobolev norms, which control both function values and derivatives, remain relatively underdeveloped\. Clarifying these properties is directly relevant for ML practice\. It provides a principled link between architecture size, regularity of the underlying PDE, and the accuracy of learned surrogates used in downstream simulation, control, and design tasks\.

This paper aims to connect studies in functional analysis, specifically Sobolev spaces, with operator learning in scientific machine learning\. Our contributions are:

1. 1\.Establishing approximation boundsfor deep neural networks mapping between Sobolev spaces𝒳=Hs​\(D\)→𝒴=Ht​\(D′\)\\mathcal\{X\}=H^\{s\}\(D\)\\to\\mathcal\{Y\}=H^\{t\}\(D^\{\\prime\}\), with explicit dependence on domain dimension and smoothness, as a continuation and refinement of universal approximation results for neural networks\.
2. 2\.Validating the theory numericallyin a PDE solution\-operator learning setting, showing how the Sobolev\-based bounds manifest in concrete scaling laws for Fourier Neural Operators \(see Conclusions and Future Work for a link to the code and ongoing numerical updates in the associated GitHub repository\)\.

These goals have direct implications for scientific machine learning\. Many physical systems, especially those governed by PDEs, are naturally described by mappings between functions\. For example, solving a PDE often amounts to computing a solutionu​\(x,t\)u\(x,t\)given an initial or boundary conditionu0​\(x\)u\_\{0\}\(x\)\. Such mappings are not pointwise but involve entire functions as both input and output, making them*operators*\. Without approximation bounds, we have no rigorous guarantee that a neural network can learn the target operator to a prescribed Sobolev accuracy, nor a quantitative sense of how model size must grow to achieve a given error tolerance\.

To rigorously model such mappings, we specifically considerSobolev spaces\. Sobolev spaces naturally incorporate weak derivatives, making them well\-suited for PDE solutions that may fail to be classically differentiable \(e\.g\., shocks\)\. SpacesHs​\(D\)H^\{s\}\(D\)are Hilbert spaces, providing inner products, orthonormal bases, and projection theorems that we exploit in our construction\. Finally, by the Rellich–Kondrachov compactness theorem, bounded subsets ofHsH^\{s\}embed compactly into lower\-order Sobolev or continuous spaces, making it mathematically possible to approximate infinite\-dimensional mappings using finite\-dimensional neural networks\. We will dedicate an entire section to the Rellich–Kondrachov theorem and explain how it underpins our Sobolev approximation bounds for neural operators\.

## 2Sobolev Spaces and Operator Learning

### 2\.1Problem Setup

LetD⊂ℝdD\\subset\\mathbb\{R\}^\{d\}be a bounded[Lipschitz domain](https://arxiv.org/html/2605.08170#A1.SS1)\. Fors∈ℕs\\in\\mathbb\{N\}, the[Sobolev space](https://arxiv.org/html/2605.08170#A1.SS2)Hs​\(D\)H^\{s\}\(D\)consists of[square\-integrable functions](https://arxiv.org/html/2605.08170#A1.SS4)with weak derivatives up to orderssalso square\-integrable:

Hs​\(D\):=\{f∈L2​\(D\)\|∂αf∈L2​\(D\),∀\|α\|≤s\},H^\{s\}\(D\):=\\left\\\{f\\in L^\{2\}\(D\)\\ \\middle\|\\ \\partial^\{\\alpha\}f\\in L^\{2\}\(D\),\\ \\forall\|\\alpha\|\\leq s\\right\\\},with the norm

‖f‖Hs​\(D\):=\(∑\|α\|≤s∫D\|∂αf​\(x\)\|2​𝑑x\)1/2\.\\\|f\\\|\_\{H^\{s\}\(D\)\}:=\\left\(\\sum\_\{\|\\alpha\|\\leq s\}\\int\_\{D\}\|\\partial^\{\\alpha\}f\(x\)\|^\{2\}\\,dx\\right\)^\{1/2\}\.Let

𝒳=Hs​\(D\)\\mathcal\{X\}=H^\{s\}\(D\)and

𝒴=Ht​\(D′\)\\mathcal\{Y\}=H^\{t\}\(D^\{\\prime\}\), where

D′⊂ℝd′D^\{\\prime\}\\subset\\mathbb\{R\}^\{d^\{\\prime\}\}is another bounded Lipschitz domain and

t∈ℕt\\in\\mathbb\{N\}\.

Given a nonlinear operator

𝒢:𝒳→𝒴\\mathcal\{G\}:\\mathcal\{X\}\\to\\mathcal\{Y\}, the current objective, as defined in the introduction, is to approximate

𝒢\\mathcal\{G\}using a deep neural network

𝒢θ\\mathcal\{G\}\_\{\\theta\}, where

θ\\thetarepresents the parameters, such that the approximation is uniformly accurate on a compact set

𝒦⊂𝒳\\mathcal\{K\}\\subset\\mathcal\{X\}\.

supf∈𝒦‖𝒢​\(f\)−𝒢θ​\(f\)‖Ht​\(D′\)<ε\.\\sup\_\{f\\in\\mathcal\{K\}\}\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\_\{\\theta\}\(f\)\\\|\_\{H^\{t\}\(D^\{\\prime\}\)\}<\\varepsilon\.This formulation is a classic formulation of operator learning in function spaces, where the goal is to learn a map between infinite\-dimensional spaces with controlled approximation error\. In this paper, we will be using finite\-dimensional NNs\.

### 2\.2Compactness via Rellich–Kondrachov Theorem

As mentioned previously, an important reason why I decided to choose Sobolev spaces is because of the finite\-dimensional approximability of

𝒢\\mathcal\{G\}hinges on the compactness of Sobolev embeddings\.

Rellich–Kondrachov Theorem: Let

D⊂ℝdD\\subset\\mathbb\{R\}^\{d\}be a bounded Lipschitz domain\. If

s\>t\+d/2s\>t\+d/2, then the embedding

Hs​\(D\)↪Ht​\(D\)H^\{s\}\(D\)\\hookrightarrow H^\{t\}\(D\)is compact\.

###### Proof Sketch\.

Let\{fn\}⊂Hs​\(D\)\\\{f\_\{n\}\\\}\\subset H^\{s\}\(D\)be bounded\. By the Banach–Alaoglu theorem, it has a weakly convergent subsequence inHsH^\{s\}\. The Rellich–Kondrachov theorem guarantees strong convergence inHt​\(D\)H^\{t\}\(D\), hence precompactness\. We will use this theorem later in 2 proofs \(see Section[3](https://arxiv.org/html/2605.08170#S3)and Section[4](https://arxiv.org/html/2605.08170#S4)\.\) ∎

This compactness implies that for any compact𝒦⊂Hs\\mathcal\{K\}\\subset H^\{s\}, the image under𝒢\\mathcal\{G\}can be uniformly approximated inHtH^\{t\}by finite\-dimensional projections, a key step in neural operator approximation\.

## 3Reformulation of the Universal Approximation in Sobolev Norms

We now formalize a universal approximation result for operator learning in the Sobolev setting\. I will begin with a formal approximation result\. Instead of a grid\-based proof like theorem 3\.10 from A Mathematical Analysis of Neural Operator BehaviorsLe and Dik \([2024](https://arxiv.org/html/2605.08170#bib.bib2)\), I tried to contribute by reformulating it using functional projection and basis expansion\.

Let

𝒢:Hs​\(D\)→Ht​\(D′\)\\mathcal\{G\}:H^\{s\}\(D\)\\to H^\{t\}\(D^\{\\prime\}\)be a continuous nonlinear operator, and let

𝒦⊂Hs​\(D\)\\mathcal\{K\}\\subset H^\{s\}\(D\)be compact\.

Then for any

ε\>0\\varepsilon\>0, there exists a ReLU neural network

𝒢θ\\mathcal\{G\}\_\{\\theta\}such that:

supf∈𝒦‖𝒢​\(f\)−𝒢θ​\(f\)‖Ht​\(D′\)<ε\.\\sup\_\{f\\in\\mathcal\{K\}\}\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\_\{\\theta\}\(f\)\\\|\_\{H^\{t\}\(D^\{\\prime\}\)\}<\\varepsilon\.
###### Proof\.

We construct the approximation in three steps:

We let

\{ϕk\}k=1∞⊂Hs​\(D\)\\\{\\phi\_\{k\}\\\}\_\{k=1\}^\{\\infty\}\\subset H^\{s\}\(D\)be an orthonormal basis\. Define a projection operator

PN​\(f\)P\_\{N\}\(f\)as

PN​\(f\)=∑k=1N⟨f,ϕk⟩​ϕk\.P\_\{N\}\(f\)=\\sum\_\{k=1\}^\{N\}\\langle f,\\phi\_\{k\}\\rangle\\phi\_\{k\}\.Since𝒦⊂Hs​\(D\)\\mathcal\{K\}\\subset H^\{s\}\(D\)is compact andPN​f→fP\_\{N\}f\\to finHsH^\{s\}, we also have \(by continuity of𝒢\\mathcal\{G\}\) that

𝒢​\(PN​\(f\)\)→𝒢​\(f\)\\mathcal\{G\}\(P\_\{N\}\(f\)\)\\to\\mathcal\{G\}\(f\)in

HtH^\{t\}uniformly on

𝒦\\mathcal\{K\}To reduce to finite\-dimensional learning, we let

𝐜N​\(f\)=\(⟨f,ϕ1⟩,…,⟨f,ϕN⟩\)∈ℝN\.\\mathbf\{c\}\_\{N\}\(f\)=\(\\langle f,\\phi\_\{1\}\\rangle,\\dots,\\langle f,\\phi\_\{N\}\\rangle\)\\in\\mathbb\{R\}^\{N\}\.Then𝒢∘PN\\mathcal\{G\}\\circ P\_\{N\}can be viewed as a map

ℝN→Ht​\(D′\)\.\\mathbb\{R\}^\{N\}\\to H^\{t\}\(D^\{\\prime\}\)\.Defineψk\\psi\_\{k\}as a basis forHt​\(D′\),H^\{t\}\(D^\{\\prime\}\),and define

𝒢N​\(f\):=∑j=1Mgj​\(𝐜N​\(f\)\)​ψj\\mathcal\{G\}\_\{N\}\(f\):=\\sum\_\{j=1\}^\{M\}g\_\{j\}\(\\mathbf\{c\}\_\{N\}\(f\)\)\\psi\_\{j\}for suitable continuous

gjg\_\{j\}\.

Since

𝐜N​\(𝒦\)⊂ℝN\\mathbf\{c\}\_\{N\}\(\\mathcal\{K\}\)\\subset\\mathbb\{R\}^\{N\}is compact and

gjg\_\{j\}are continuous, we may approximate each

gjg\_\{j\}uniformly on this set by a ReLU network

gj,θg\_\{j,\\theta\}, using the universal approximation theorem in finite dimensions\. Define:

𝒢θ​\(f\):=∑j=1Mgj,θ​\(𝐜N​\(f\)\)​ψj\.\\displaystyle\\mathcal\{G\}\_\{\\theta\}\(f\):=\\sum\_\{j=1\}^\{M\}g\_\{j,\\theta\}\(\\mathbf\{c\}\_\{N\}\(f\)\)\\psi\_\{j\}\.\(1\)Then forf∈𝒦f\\in\\mathcal\{K\}:

‖𝒢​\(f\)−𝒢θ​\(f\)‖Ht≤‖𝒢​\(f\)−𝒢​\(PN​f\)‖Ht\+‖𝒢​\(PN​f\)−𝒢θ​\(f\)‖Ht\.\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\_\{\\theta\}\(f\)\\\|\_\{H^\{t\}\}\\leq\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\(P\_\{N\}f\)\\\|\_\{H^\{t\}\}\+\\\|\\mathcal\{G\}\(P\_\{N\}f\)\-\\mathcal\{G\}\_\{\\theta\}\(f\)\\\|\_\{H^\{t\}\}\.So, if we choose

NNlarge enough that the first term is less than

ε/2\\varepsilon/2, and approximate

gjg\_\{j\}well enough to make the second term

<ε/2<\\varepsilon/2\.

Hence the total error is less than

ε\\varepsilon, uniformly on

𝒦\\mathcal\{K\}\. This concludes the proof\. ∎

In this section, we combined the compact Sobolev embeddingHs↪HtH^\{s\}\\hookrightarrow H^\{t\}\(via the Rellich–Kondrachov theorem\) with the universal approximation property of neural networks in finite\-dimensional spaces\. More intuitively, we used projection onto finite\-dimensional bases to reduce the infinite\-dimensional operator learning problem to a standard function\-approximation task that can be implemented by neural networks\.

## 4Quantitative Approximation Error Bounds for Operator Learning

Now that we have explained the universal approximation in sobolev norms, we can build on it and derive an original theorem that gives an explicit bound on the neural network complexity required to approximate a nonlinear operator between Sobolev spaces\.

Let𝒢:Hs​\(D\)→Ht​\(D′\)\\mathcal\{G\}:H^\{s\}\(D\)\\to H^\{t\}\(D^\{\\prime\}\)be a continuous operator between Sobolev spaces, whereD,D′⊂ℝdD,D^\{\\prime\}\\subset\\mathbb\{R\}^\{d\}are bounded Lipschitz domains ands\>d/2s\>d/2\. Let𝒦⊂Hs​\(D\)\\mathcal\{K\}\\subset H^\{s\}\(D\)be compact\. Then for anyε\>0\\varepsilon\>0, there exists a ReLU neural network operator𝒢θ\\mathcal\{G\}\_\{\\theta\}with𝒪​\(ε−d/s\)\\mathcal\{O\}\(\\varepsilon^\{\-d/s\}\)parameters such that

supf∈𝒦‖𝒢​\(f\)−𝒢θ​\(f\)‖Ht​\(D′\)<ε\.\\sup\_\{f\\in\\mathcal\{K\}\}\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\_\{\\theta\}\(f\)\\\|\_\{H^\{t\}\(D^\{\\prime\}\)\}<\\varepsilon\.
###### Proof\.

Let\{ϕk\}⊂Hs​\(D\)\\\{\\phi\_\{k\}\\\}\\subset H^\{s\}\(D\)be an orthonormal basis, and define the projection

PN​f:=∑k=1N⟨f,ϕk⟩​ϕk\.P\_\{N\}f:=\\sum\_\{k=1\}^\{N\}\\langle f,\\phi\_\{k\}\\rangle\\phi\_\{k\}\.By theRellich–Kondrachov theoremand continuity of𝒢\\mathcal\{G\}, there existsNNsuch that

supf∈𝒦‖𝒢​\(f\)−𝒢​\(PN​f\)‖Ht​\(D′\)<ε/2\.\\sup\_\{f\\in\\mathcal\{K\}\}\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\(P\_\{N\}f\)\\\|\_\{H^\{t\}\(D^\{\\prime\}\)\}<\\varepsilon/2\.F​\(𝐜\)=∑j=1Mgj​\(𝐜\)​ψj\.F\(\\mathbf\{c\}\)=\\sum\_\{j=1\}^\{M\}g\_\{j\}\(\\mathbf\{c\}\)\\psi\_\{j\}\.Eachgjg\_\{j\}is continuous on the compact set𝐜​\(𝒦\)⊂ℝN\\mathbf\{c\}\(\\mathcal\{K\}\)\\subset\\mathbb\{R\}^\{N\}, so we can approximate it uniformly by a ReLU neural networkgj,θg\_\{j,\\theta\}such that

sup𝐜∈𝐜​\(𝒦\)\|gj​\(𝐜\)−gj,θ​\(𝐜\)\|<ε2​M\.\\sup\_\{\\mathbf\{c\}\\in\\mathbf\{c\}\(\\mathcal\{K\}\)\}\|g\_\{j\}\(\\mathbf\{c\}\)\-g\_\{j,\\theta\}\(\\mathbf\{c\}\)\|<\\frac\{\\varepsilon\}\{2M\}\.Define the neural network operator:

𝒢θ​\(f\):=∑j=1Mgj,θ​\(𝐜​\(f\)\)​ψj\.\\mathcal\{G\}\_\{\\theta\}\(f\):=\\sum\_\{j=1\}^\{M\}g\_\{j,\\theta\}\(\\mathbf\{c\}\(f\)\)\\psi\_\{j\}\.Then for allf∈𝒦f\\in\\mathcal\{K\},

‖𝒢​\(f\)−𝒢θ​\(f\)‖Ht≤‖𝒢​\(f\)−𝒢​\(PN​f\)‖Ht\+‖𝒢​\(PN​f\)−𝒢θ​\(f\)‖Ht<ε\.\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\_\{\\theta\}\(f\)\\\|\_\{H^\{t\}\}\\leq\\\|\\mathcal\{G\}\(f\)\-\\mathcal\{G\}\(P\_\{N\}f\)\\\|\_\{H^\{t\}\}\+\\\|\\mathcal\{G\}\(P\_\{N\}f\)\-\\mathcal\{G\}\_\{\\theta\}\(f\)\\\|\_\{H^\{t\}\}<\\varepsilon\.Finally, from classical results on approximation by neural networks inℝN\\mathbb\{R\}^\{N\}, approximating Lipschitz functionsgjg\_\{j\}to accuracyδ\\deltarequires𝒪​\(δ−N\)\\mathcal\{O\}\(\\delta^\{\-N\}\)parameters\. Since the projection error scales like𝒪​\(N−s/d\)\\mathcal\{O\}\(N^\{\-s/d\}\), we chooseN=𝒪​\(ε−d/s\)N=\\mathcal\{O\}\(\\varepsilon^\{\-d/s\}\)\. This yields the stated bound\. ∎

Building upon the theoretical analysis in Sections[3](https://arxiv.org/html/2605.08170#S3)and[4](https://arxiv.org/html/2605.08170#S4), we formulate several hypotheses about neural operator approximation in Sobolev spaces and test them numerically on the viscous Burgers solution operator

G:u0↦u​\(⋅,1\),G:u\_\{0\}\\mapsto u\(\\cdot,1\),whereu0∈H1​\(\[0,1\]\)u\_\{0\}\\in H^\{1\}\(\[0,1\]\)denotes the initial condition andu​\(⋅,1\)u\(\\cdot,1\)is the corresponding final–time solution\.

### 4\.1Discussion of Hypotheses

We now summarize the main hypotheses that link the Sobolev\-space theory in Sections[3](https://arxiv.org/html/2605.08170#S3)and[4](https://arxiv.org/html/2605.08170#S4)to the behavior of neural operators trained on Burgers’ equation, and describe what we expect to observe numerically\.

#### Universal approximability in Sobolev norms\.

Our first hypothesis is that a neural operator \(here, a Fourier Neural Operator\) can approximate the target PDE solution operatorGGuniformly on compact subsetsK⊂H1​\(\[0,1\]\)K\\subset H^\{1\}\(\[0,1\]\), in the sense that for anyε\>0\\varepsilon\>0there exists a sufficiently large model such that

supf∈K‖G​\(f\)−Gθ​\(f\)‖H1​\(\[0,1\]\)<ε\.\\sup\_\{f\\in K\}\\\|G\(f\)\-G\_\{\\theta\}\(f\)\\\|\_\{H^\{1\}\(\[0,1\]\)\}<\\varepsilon\.In light of the universal approximation results in Sobolev norms, we expect that training an FNO on data pairs\(u0,u​\(⋅,1\)\)\(u\_\{0\},u\(\\cdot,1\)\)will yield uniformly small approximation errors when these errors are measured directly in the SobolevH1H^\{1\}\-norm\.

#### Quantitative approximation rate\.

The second hypothesis concerns not just the possibility of approximation, but the*rate*at which Sobolev error decays as the model size increases\. We posit that the approximation error inH1H^\{1\}\-norm decays with model size according to a power law of the form

‖G​\(f\)−Gθ​\(f\)‖H1≈C​N−α,\\\|G\(f\)\-G\_\{\\theta\}\(f\)\\\|\_\{H^\{1\}\}\\approx CN^\{\-\\alpha\},whereNNdenotes an effective model size \(e\.g\. the number of trainable parameters\) andα\>0\\alpha\>0is an empirical exponent\. The bound of Section[4](https://arxiv.org/html/2605.08170#S4)suggests an idealized rate of orderN−s/dN^\{\-s/d\}; in our Burgers settings=d=1s=d=1, so the benchmark exponent iss/d=1s/d=1\. Accordingly, on a log–log plot of Sobolev error versus model size, we expect the empirical curve to be approximately linear over a suitable range ofNN, with slope−α\-\\alpha\. Comparing the fittedα\\alphatos/ds/dquantifies to what extent architectural and optimization constraints slow down the worst\-case theoretical rate\.

#### Compactness of the input set\.

The third hypothesis is that the data distribution used in the experiments respects the compactness assumptions required by the theory\. Concretely, we assume that the initial data for training and testing lie in a compact subsetK⊂H1​\(\[0,1\]\)K\\subset H^\{1\}\(\[0,1\]\)\. In practice, this means that all sampled initial conditionsu0u\_\{0\}have uniformly boundedH1H^\{1\}\-norm, and that the sampling procedure explicitly enforces such a bound\. This is precisely the setting in which the Rellich–Kondrachov based arguments from Sections[3](https://arxiv.org/html/2605.08170#S3)and[4](https://arxiv.org/html/2605.08170#S4)apply\.

#### Sobolev\-norm fidelity\.

Finally, we hypothesize that the learned operator captures not only function values but also derivative information, so that convergence occurs in the full Sobolev norm rather than merely inL2L^\{2\}\. In other words, good performance inH1H^\{1\}should translate into accurate prediction of bothu​\(⋅,1\)u\(\\cdot,1\)and its spatial derivative∂xu​\(⋅,1\)\\partial\_\{x\}u\(\\cdot,1\)\. For held\-out test samples, we therefore expect the predicted solutionuθ​\(⋅,1\)u\_\{\\theta\}\(\\cdot,1\)and its derivative∂xuθ​\(⋅,1\)\\partial\_\{x\}u\_\{\\theta\}\(\\cdot,1\)to closely match the ground\-truthu​\(⋅,1\)u\(\\cdot,1\)and∂xu​\(⋅,1\)\\partial\_\{x\}u\(\\cdot,1\), with derivative errors remaining small\. This Sobolev\-norm fidelity is essential if the learned operator is to be used as a stable surrogate in downstream scientific computing tasks\.

## 5Numerical Experiments

### 5\.1Experimental Setup

To test these hypotheses, we train Fourier Neural Operators \(FNOs\) to learn the mappingG:u0↦u​\(⋅,1\)G:u\_\{0\}\\mapsto u\(\\cdot,1\)for the one\-dimensional viscous Burgers equation with periodic boundary conditions and viscosityν\>0\\nu\>0\. Training data pairs\(u0,u​\(⋅,1\)\)\(u\_\{0\},u\(\\cdot,1\)\)are generated using a spectral solver with random smooth initial conditions drawn from a boundedH1H^\{1\}\-ball:

‖u0‖H1​\(\[0,1\]\)≤R,\\\|u\_\{0\}\\\|\_\{H^\{1\}\(\[0,1\]\)\}\\leq R,for a fixed radiusR=0\.3R=0\.3\. This construction enforces the compactness assumption underlying Hypothesis 3\.

We discretize the spatial domain on a uniform grid and represent each initial condition and solution as a periodic function on this grid\. The loss is the discreteH1H^\{1\}\-norm,

‖u−u^‖H12≈‖u−u^‖L22\+‖∂xu−∂xu^‖L22,\\\|u\-\\hat\{u\}\\\|\_\{H^\{1\}\}^\{2\}\\approx\\\|u\-\\hat\{u\}\\\|\_\{L^\{2\}\}^\{2\}\+\\\|\\partial\_\{x\}u\-\\partial\_\{x\}\\hat\{u\}\\\|\_\{L^\{2\}\}^\{2\},implemented via periodic finite differences for the gradient term, so that the optimization objective exactly matches the Sobolev norm used in the theory\.

We consider FNOs with four Fourier layers and vary the number of retained modes and the channel width, leading to the following configurations:

\(modes, width\)∈\{\(8,32\),\(12,48\),\(16,64\),\(24,96\)\},\\text\{\(modes, width\)\}\\in\\\{\(8,32\),\(12,48\),\(16,64\),\(24,96\)\\\},with corresponding parameter counts

N∈\{74,209,237,137,549,569,1,819,553\}\.N\\in\\\{74\{,\}209,\\;237\{,\}137,\\;549\{,\}569,\\;1\{,\}819\{,\}553\\\}\.Each model is trained with the Adam optimizer on256256training examples and evaluated on a held\-out test set of6464examples\. Unless otherwise stated, we train for 100 epochs and record the evolution of the train and testH1H^\{1\}\-loss\. For the largest model we additionally perform long runs of 500 and 1000 epochs to probe optimization stability\.

### 5\.2Numerical Results

#### Single\-sample Sobolev fidelity\.

Figure[1](https://arxiv.org/html/2605.08170#S5.F1)illustrates a representative test sample for the largest FNO \(modes=24=24, width=96=96\) in a regime where the test loss is very small\. The left panel shows the initial conditionu​\(x,0\)u\(x,0\), the ground\-truth solutionu​\(x,1\)u\(x,1\), and the FNO predictionu^​\(x,1\)\\hat\{u\}\(x,1\); the right panel compares the corresponding spatial derivatives∂xu​\(x,1\)\\partial\_\{x\}u\(x,1\)and∂xu^​\(x,1\)\\partial\_\{x\}\\hat\{u\}\(x,1\)\. The predicted curves are visually indistinguishable from the ground truth in both value and derivative, confirming that smallH1H^\{1\}\-loss indeed corresponds to accurate recovery of both the function and its gradient and providing strong evidence for Hypothesis 4 \(Sobolev\-norm fidelity\)\.

![Refer to caption](https://arxiv.org/html/2605.08170v1/fno_prediction.png)Figure 1:Qualitative evaluation of the learned FNO on a representative test sample\.Left:initial conditionu​\(x,0\)u\(x,0\), ground\-truth final\-time solutionu​\(x,1\)u\(x,1\), and FNO predictionu^​\(x,1\)\\hat\{u\}\(x,1\)\.Right:comparison of spatial derivatives∂xu​\(x,1\)\\partial\_\{x\}u\(x,1\)and∂xu^​\(x,1\)\\partial\_\{x\}\\hat\{u\}\(x,1\)\. The close alignment in both plots is consistent with the very small measuredH1H^\{1\}\-error\.
#### Learning curves for different model sizes\.

Figure[2](https://arxiv.org/html/2605.08170#S5.F2)shows the testH1H^\{1\}\-loss as a function of epoch for the four FNO sizes trained for 100 epochs\. Increasing the number of modes and the width accelerates optimization and reduces the lowest test error attained during training\. The three smaller models \(modes=8,12,16=8,12,16\) exhibit smooth, monotone decay of the test loss, with the larger widths converging faster and achieving lower error\. The largest model \(modes=24=24, width=96=96\) initially dominates and reaches the smallest test loss by around 60–80 epochs, but then undergoes a sharp instability around epoch 90, where the test loss increases by almost two orders of magnitude before partially recovering\. If one naïvely reads off the error at epoch 100, the largest model appears significantly worse than the smaller ones, even though it passes through a strictly better regime earlier in training\.

![Refer to caption](https://arxiv.org/html/2605.08170v1/fno_learning_curves_by_size.png)Figure 2:Learning curves \(testH1H^\{1\}\-loss\) for FNOs of increasing size, trained for 100 epochs on the Burgers dataset\. Larger models converge faster and attain lower test error, but the largest architecture \(modes=24=24, width=96=96\) becomes unstable near epoch 90, causing its loss to spike despite having achieved the best performance earlier in training\.
#### Long\-run training and optimization instabilities\.

To better understand the behavior of the largest model, we train it for 100, 500, and 1000 epochs\. The resulting testH1H^\{1\}\-loss curves are shown in Figure[3](https://arxiv.org/html/2605.08170#S5.F3)\. All three runs initially decrease rapidly to errors around10−710^\{\-7\}\. Beyond roughly 150–200 epochs, however, the loss becomes highly non\-monotone: we repeatedly observe spikes where the error increases by several orders of magnitude, followed by recovery to a low\-error regime \(sometimes below10−810^\{\-8\}\)\. The 1000\-epoch run attains a minimum test loss of order10−910^\{\-9\}, but also exhibits multiple catastrophic bursts where the loss rises to10−410^\{\-4\}before returning to the small\-loss region\. This behavior points to pronounced optimization instabilities for this architecture and learning rate: the optimizer moves through a sequence of sharp minima and occasionally crosses into unstable regions of parameter space, even though excellent generalization is still achievable at certain epochs\.

![Refer to caption](https://arxiv.org/html/2605.08170v1/fno_longrun_learning_curves.png)Figure 3:Long\-run learning curves for the largest FNO \(modes=24=24, width=96=96\) trained for 100, 500, and 1000 epochs\. All runs reach very small testH1H^\{1\}\-loss, but longer training reveals repeated spikes where the loss increases by several orders of magnitude before recovering, indicating substantial optimization instability\.For each model, we record both the final and the*best*testH1H^\{1\}\-loss over the first 100 epochs\. The best\-epoch test losses are

Nminepoch⁡test​H1​\-loss74,2096\.87×10−7237,1376\.01×10−7549,5694\.80×10−71,819,5534\.93×10−7,\\begin\{array\}\[\]\{c\|c\}N&\\min\_\{\\text\{epoch\}\}\\text\{ test \}H^\{1\}\\text\{\-loss\}\\\\ \\hline\\cr 74\{,\}209&6\.87\\times 10^\{\-7\}\\\\ 237\{,\}137&6\.01\\times 10^\{\-7\}\\\\ 549\{,\}569&4\.80\\times 10^\{\-7\}\\\\ 1\{,\}819\{,\}553&4\.93\\times 10^\{\-7\},\\end\{array\}with corresponding global relativeH1H^\{1\}\-errors of order10−310^\{\-3\}\. The final\-epoch loss for the largest model is much worse \(8\.63×10−58\.63\\times 10^\{\-5\}\), reflecting the instability near epoch 100\.

Figure[4](https://arxiv.org/html/2605.08170#S5.F4)plots the final testH1H^\{1\}\-loss and the global relativeH1H^\{1\}\-error as a function ofNNon a log–log scale\. Because the largest model ends its run in an unstable phase, its final error is roughly two orders of magnitude larger than that of the smaller networks, producing an apparent “U\-shaped” curve\. Fitting a line through these final values yields an exponent that is essentially meaningless, since it is dominated by the pathological last point\.

To obtain a more faithful picture of the*approximation*capacity, we instead fit a power law‖G−Gθ‖H1≈C​N−α\\\|G\-G\_\{\\theta\}\\\|\_\{H^\{1\}\}\\approx CN^\{\-\\alpha\}to the best\-epoch errors for each model\. Figure[4](https://arxiv.org/html/2605.08170#S5.F4)shows the corresponding log–log plot\. A least\-squares fit gives

α≈0\.11,‖G−Gθ‖H1≈2\.4×10−6​N−0\.11\.\\alpha\\approx 0\.11,\\qquad\\\|G\-G\_\{\\theta\}\\\|\_\{H^\{1\}\}\\approx 2\.4\\times 10^\{\-6\}\\,N^\{\-0\.11\}\.Thus, while larger models do achieve slightly smaller Sobolev error, the observed decay withNNis*much flatter*than the benchmark rateN−1N^\{\-1\}suggested by the worst–case complexity bound\.

![Refer to caption](https://arxiv.org/html/2605.08170v1/fno_error_vs_params.png)Figure 4:Final test error versus number of parametersNNfor FNO approximations of the Burgers solution operator\. We plot both the final testH1H^\{1\}\-loss and the global relativeH1H^\{1\}\-error\. The largest model \(N≈1\.8×106N\\approx 1\.8\\times 10^\{6\}\) is an outlier due to training instability, yielding a misleading “U\-shaped” curve when only final\-epoch errors are used\.![Refer to caption](https://arxiv.org/html/2605.08170v1/figure_1.png)Figure 5:Log–log plot of testH1H^\{1\}\-loss versus number of trainable parametersNNfor the four FNO models, using the*best*test loss attained within the first 100 epochs for each model\. A least\-squares fit yields an empirical exponentα≈0\.11\\alpha\\approx 0\.11in‖G−Gθ‖H1≈C​N−α\\\|G\-G\_\{\\theta\}\\\|\_\{H^\{1\}\}\\approx CN^\{\-\\alpha\}, indicating a very slow improvement of Sobolev error with model size compared to the benchmark rateN−1N^\{\-1\}suggested by the theoretical complexity bound\.

## 6Conclusions

In this work, we have established a universal approximation result for operator learning in Sobolev spaces\. Specifically, for a continuous operator𝒢:Hs​\(D\)→Ht​\(D′\)\\mathcal\{G\}:H^\{s\}\(D\)\\to H^\{t\}\(D^\{\\prime\}\)and a compact subset𝒦⊂Hs​\(D\)\\mathcal\{K\}\\subset H^\{s\}\(D\), we showed that𝒢\\mathcal\{G\}can be uniformly approximated inHtH^\{t\}\-norm by a finite\-dimensional neural network operator𝒢θ\\mathcal\{G\}\_\{\\theta\}\. The proof relies on three ingredients: \(i\) compact Sobolev embeddings via the Rellich–Kondrachov theorem, \(ii\) projection onto finite\-dimensional bases inHsH^\{s\}andHtH^\{t\}, and \(iii\) the classical universal approximation theorem for ReLU networks in finite dimensions\.

Building on this qualitative statement, we derived a quantitative complexity bound: to achieve accuracyε\\varepsiloninHtH^\{t\}\-norm it suffices to use𝒪​\(ε−d/s\)\\mathcal\{O\}\(\\varepsilon^\{\-d/s\}\)parameters\. Inverting this relation suggests an asymptotic error law

‖𝒢−𝒢θ‖Ht≲C​N−s/d,\\\|\\mathcal\{G\}\-\\mathcal\{G\}\_\{\\theta\}\\\|\_\{H^\{t\}\}\\lesssim CN^\{\-s/d\},whereNNdenotes the number of trainable parameters\. In the one\-dimensional Burgers experiment, the solution operator𝒢:H1​\(\[0,1\]\)→H1​\(\[0,1\]\)\\mathcal\{G\}:H^\{1\}\(\[0,1\]\)\\to H^\{1\}\(\[0,1\]\)satisfies the assumptions withs=1s=1andd=1d=1, so the theoretical benchmark is a rate of orderN−1N^\{\-1\}when initial data are restricted to a compact subset ofH1H^\{1\}\.

To connect theory with practice, we considered the viscous Burgers equation with periodic boundary conditions and studied its solution operator

𝒢:u0↦u​\(⋅,1\),u0∈H1​\(\[0,1\]\)\.\\mathcal\{G\}:u\_\{0\}\\mapsto u\(\\cdot,1\),\\quad u\_\{0\}\\in H^\{1\}\(\[0,1\]\)\.Using a spectral solver, we generated training and test data by sampling smooth initial conditions from a boundedH1H^\{1\}\-ball and propagating them to timet=1t=1\. Fourier Neural Operators with up to1\.8×1061\.8\\times 10^\{6\}parameters were trained using anH1H^\{1\}\-loss\. In favorable parts of the optimization trajectory, the largest model achieved testH1H^\{1\}\-loss down to10−710^\{\-7\}–10−910^\{\-9\}, with predicted solutions and derivatives almost indistinguishable from the ground truth \(see Figure[1](https://arxiv.org/html/2605.08170#S5.F1)\), empirically realizing the qualitative universal approximation result in this PDE setting\.

A key finding is that the dependence of Sobolev error on model size is governed by a power law, but with a much smaller exponent than the theoretical benchmark\. When we fit‖𝒢−𝒢θ‖H1≈C​N−α\\\|\\mathcal\{G\}\-\\mathcal\{G\}\_\{\\theta\}\\\|\_\{H^\{1\}\}\\approx CN^\{\-\\alpha\}to the*best\-epoch*errors of FNOs of increasing size, we obtainα≈0\.11\\alpha\\approx 0\.11\(Figure[5](https://arxiv.org/html/2605.08170#S5.F5)\)\. Thus, increasing the parameter count from7\.4×1047\.4\\times 10^\{4\}to1\.8×1061\.8\\times 10^\{6\}yields only a modest improvement in Sobolev error\. If one instead uses final\-epoch errors, the picture is dominated by optimization instabilities of the largest model and can even suggest a misleading*increase*in error with model size\. This reinforces the point that quantitative approximation theory describes what is*possible in principle*, whereas actual performance is constrained by architecture, optimization, and regularization\.

Taken together, the theoretical results and numerical experiments support the following conclusions: Continuous operators between Sobolev spacesHs​\(D\)H^\{s\}\(D\)andHt​\(D′\)H^\{t\}\(D^\{\\prime\}\)can be uniformly approximated on compact sets by finite\-dimensional neural networks, with approximation measured directly in Sobolev norms\. In a concrete PDE setting \(the viscous Burgers equation\), neural operators \(FNOs\) achieve extremely smallH1H^\{1\}\-errors on a compact family of initial conditions, and qualitative plots confirm that both solutions and derivatives are well reproduced, providing strong evidence for Sobolev\-norm fidelity\. The observed decay of Sobolev error with model size follows a power law but with an empirical exponentα≈0\.11\\alpha\\approx 0\.11, significantly smaller than the benchmarks/d=1s/d=1\. In practice, approximation quality is therefore dominated by optimization dynamics and architectural biases rather than by the worst–case complexity bound\. The numerical setting respects the compactness assumptions underlying the theoretical arguments, but long\-run learning curves reveal substantial optimization instabilities, especially for the largest models\. This suggests that controlling the optimization trajectory \(e\.g\. via early stopping, adaptive learning rates, or regularization\) is as important as increasing model size when the goal is to reduce Sobolev error\.

## 7Discussion and Future Work

There are several natural directions for extending this work\. First, the complexity estimate𝒪​\(ε−d/s\)\\mathcal\{O\}\(\\varepsilon^\{\-d/s\}\)derived here relies on relatively coarse, worst–case arguments\. A natural next step is to refine this bound by incorporating additional regularity assumptions on𝒢\\mathcal\{G\}\(for example, Lipschitz or Hölder continuity\) and by exploiting architecture\-specific properties such as the spectral bias of FNOs\. The overarching goal is to narrow the gap between the benchmark exponents/ds/dand the significantly smaller empirical exponents observed in practice\.

Second, the experiments in this paper provide qualitative evidence of power\-law convergence for a single one\-dimensional PDE over a limited range of model sizes\. A more systematic empirical study— sweeping over architectures, training regimes, and PDE families \(such as higher\-dimensional Burgers, Navier–Stokes, or elliptic problems\), would enable more robust estimation of empirical exponents and a tighter comparison to theoretical predictions\.

Third, while this work focuses on Fourier Neural Operators, the approximation results apply more generally to neural operators built from finite\-dimensional networks\. A careful comparison of architectures such as DeepONet, graph neural operators, and FNOs under common Sobolev error metrics could help clarify which design choices most effectively exploit the functional\-analytic structure of the underlying problem\.

Finally, it would be valuable to investigate how Sobolev\-space regularity interacts with generalization beyond the training distribution \(for instance, to rougher initial conditions or different viscosity parameters\), and to what extent one can obtain guarantees that couple approximation properties with optimization stability\. The pronounced spikes observed in the long\-run learning curves suggest that understanding the geometry of the loss landscape in Sobolev norm is an important open problem for neural operator theory, with direct implications for robust training and deployment\.

## Code Availability

## Related Works

This project was mainly inspired byA Mathematical Analysis of Neural Operator Behaviors \(Le & Dik, 2024\)Le and Dik \([2024](https://arxiv.org/html/2605.08170#bib.bib2)\)and can be seen as an extension of the paper, but on the operator learning problem\.

Here’s a list of other papers I read to set up this project \(not ranked by relevance\):

Deep Operator Networks \(DeepONet\)Lu et al\. \([2021](https://arxiv.org/html/2605.08170#bib.bib4)\)introduced one of the first architectures for learning nonlinear operators from data\. Based on the universal approximation theorem for operators, DeepONet consists of a branch network for encoding input functions and a trunk network for output evaluation, showing strong performance on a variety of differential equations\.

Fourier Neural Operators \(FNO\)Li et al\. \([2020](https://arxiv.org/html/2605.08170#bib.bib3)\)this paper showed a different approach by parameterizing integral kernels directly in Fourier space\. The focus of this method was on achieving fast and accurate operator learning \(for parametric PDEs like turbulent flow simulations\)\.

A Mathematical Guide to Operator LearningKovachki et al\. \([2021](https://arxiv.org/html/2605.08170#bib.bib1)\)a comprehensive review of neural operator theory and practice\. It covers key architectures, approximation theory, and insights from infinite\-dimensional analysis\.

## Acknowledgements

This work did not receive any specific grant from funding agencies in the public, commercial, or not\-for\-profit sectors\. The author is deeply grateful to Professor Yunan Yang \(Department of Mathematics, Cornell University\), whose functional analysis course and guidance provided the mathematical foundation and motivation for this project\.

The author declares no known competing financial interests or personal relationships that could have influenced the work reported in this paper\. All numerical experiments are carried out on synthetically generated data produced by the PDE solvers described in the manuscript\. Code and scripts to reproduce the datasets and experiments will be made publicly available in the GitHub repository referenced in the paper upon publication\.

## References

- Kovachki et al\. \[2021\]Nikola B Kovachki, Zongyi Li, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M Stuart, and Anima Anandkumar\.Neural operator: Learning maps between function spaces\.*arXiv preprint arXiv:2108\.08481*, 2021\.
- Le and Dik \[2024\]Vu\-Anh Le and Mehmet Dik\.A mathematical analysis of neural operator behaviors, 2024\.URL[https://arxiv\.org/abs/2410\.21481](https://arxiv.org/abs/2410.21481)\.
- Li et al\. \[2020\]Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar\.Fourier neural operator for parametric partial differential equations\.*arXiv preprint arXiv:2010\.08895*, 2020\.
- Lu et al\. \[2021\]Lu Lu, Pengzhan Jin, Guofei Pang, Zhiping Zhang, and George Em Karniadakis\.Learning nonlinear operators via deeponet based on the universal approximation theorem of operators\.*Nature Machine Intelligence*, 3\(3\):218–229, 2021\.

## Appendix AAppendix: Definitions and Theorems

### A\.1Lipschitz Domain

A domainD⊂ℝdD\\subset\\mathbb\{R\}^\{d\}is called a*Lipschitz domain*if, near every point on its boundary, it can be locally represented as the region above the graph of a Lipschitz continuous function\. That is, for everyx0∈∂Dx\_\{0\}\\in\\partial D, there exists a neighborhoodUUofx0x\_\{0\}and a Lipschitz functionφ:ℝd−1→ℝ\\varphi:\\mathbb\{R\}^\{d\-1\}\\to\\mathbb\{R\}such that \(after a coordinate change\):

D∩U=\{x=\(x′,xd\)∈U∣xd\>φ​\(x′\)\}\.D\\cap U=\\left\\\{x=\(x^\{\\prime\},x\_\{d\}\)\\in U\\mid x\_\{d\}\>\\varphi\(x^\{\\prime\}\)\\right\\\}\.

### A\.2Sobolev Space

The Sobolev spaceHs​\(D\)H^\{s\}\(D\)consists of functionsf∈L2​\(D\)f\\in L^\{2\}\(D\)such that all weak partial derivatives∂αf∈L2​\(D\)\\partial^\{\\alpha\}f\\in L^\{2\}\(D\)for\|α\|≤s\|\\alpha\|\\leq s\. These spaces are Hilbert spaces equipped with the norm:

‖f‖Hs​\(D\):=\(∑\|α\|≤s∫D\|∂αf​\(x\)\|2​𝑑x\)1/2\.\\\|f\\\|\_\{H^\{s\}\(D\)\}:=\\left\(\\sum\_\{\|\\alpha\|\\leq s\}\\int\_\{D\}\|\\partial^\{\\alpha\}f\(x\)\|^\{2\}\\,dx\\right\)^\{1/2\}\.

### A\.3Weak Derivative

Letf∈Lloc1​\(D\)f\\in L^\{1\}\_\{\\text\{loc\}\}\(D\), whereD⊂ℝdD\\subset\\mathbb\{R\}^\{d\}is open\. We say thatg∈Lloc1​\(D\)g\\in L^\{1\}\_\{\\text\{loc\}\}\(D\)is the*weak derivative*offfwith respect toxix\_\{i\}if:

∫Df​\(x\)​∂iφ​\(x\)​d​x=−∫Dg​\(x\)​φ​\(x\)​𝑑xfor all​φ∈Cc∞​\(D\)\.\\int\_\{D\}f\(x\)\\,\\partial\_\{i\}\\varphi\(x\)\\,dx=\-\\int\_\{D\}g\(x\)\\,\\varphi\(x\)\\,dx\\quad\\text\{for all \}\\varphi\\in C\_\{c\}^\{\\infty\}\(D\)\.
In this case, we write∂if=g\\partial\_\{i\}f=gin the weak sense\.

More generally, for a multi\-indexα∈ℕd\\alpha\\in\\mathbb\{N\}^\{d\},ffhas weak derivative∂αf∈Lloc1​\(D\)\\partial^\{\\alpha\}f\\in L^\{1\}\_\{\\text\{loc\}\}\(D\)if:

∫Df​\(x\)​∂αφ​\(x\)​d​x=\(−1\)\|α\|​∫D∂αf​\(x\)​φ​\(x\)​d​x∀φ∈Cc∞​\(D\)\.\\int\_\{D\}f\(x\)\\,\\partial^\{\\alpha\}\\varphi\(x\)\\,dx=\(\-1\)^\{\|\\alpha\|\}\\int\_\{D\}\\partial^\{\\alpha\}f\(x\)\\,\\varphi\(x\)\\,dx\\quad\\forall\\varphi\\in C\_\{c\}^\{\\infty\}\(D\)\.
Remarks:Weak derivatives generalize classical derivatives to functions that may not be differentiable in the usual sense\. The spaceHs​\(D\)H^\{s\}\(D\)is defined using these weak derivatives, allowing for the inclusion of solutions to PDEs that are not classically smooth\.

### A\.4Square\-Integrable Function

A functionf:D→ℝf:D\\to\\mathbb\{R\}is called*square\-integrable*if:

∫D\|f​\(x\)\|2​𝑑x<∞\.\\int\_\{D\}\|f\(x\)\|^\{2\}\\,dx<\\infty\.The space of such functions is denotedL2​\(D\)L^\{2\}\(D\), a Hilbert space with inner product⟨f,g⟩=∫Df​\(x\)​g​\(x\)​𝑑x\\langle f,g\\rangle=\\int\_\{D\}f\(x\)g\(x\)\\,dx\.

## Appendix BAppendix: Fourier Neural Operators \(FNOs\)

The Fourier Neural Operator \(FNO\), introduced by Li et al\.Li et al\. \[[2020](https://arxiv.org/html/2605.08170#bib.bib3)\], is a deep learning architecture designed to learn mappings between infinite\-dimensional function spaces—especially solution operators of parametric partial differential equations \(PDEs\)\.

Unlike traditional neural networks that act on finite\-dimensional vectors, FNOs learn operators of the form:

𝒢:f​\(x\)↦u​\(x\),f∈𝒳,u∈𝒴,\\mathcal\{G\}:f\(x\)\\mapsto u\(x\),\\quad f\\in\\mathcal\{X\},\\ u\\in\\mathcal\{Y\},where𝒳,𝒴\\mathcal\{X\},\\mathcal\{Y\}are typically subsets ofL2​\(D\)L^\{2\}\(D\)orHs​\(D\)H^\{s\}\(D\)over a spatial domainD⊂ℝdD\\subset\\mathbb\{R\}^\{d\}\. The central innovation of FNO is to parameterize the action of the operator in theFourier domain, allowing it to efficiently capture long\-range dependencies and smooth functional structure\.

FNO layers consist of:

- •AFourier transformto move the function into frequency space,
- •A learned,diagonal multiplier\(analogous to a convolution kernel\) that acts on each frequency mode,
- •Aninverse Fourier transformto return to the spatial domain,
- •Pointwise non\-linearities and potential skip connections\.

### Mathematical Structure of an FNO Layer

Letv:D→ℝCv:D\\to\\mathbb\{R\}^\{C\}be a function withCCchannels\. An FNO layer updatesvvas follows:

FNO​\(v\)​\(x\)=ℱ−1​\(R​\(v^\)\)​\(x\)\+W​\(v​\(x\)\),\\text\{FNO\}\(v\)\(x\)=\\mathcal\{F\}^\{\-1\}\\left\(R\(\\hat\{v\}\)\\right\)\(x\)\+W\(v\(x\)\),where:

- •v^=ℱ​\(v\)\\hat\{v\}=\\mathcal\{F\}\(v\)is the Fourier transform,
- •RRis a learned transformation applied mode\-wise \(typically a complex\-valued linear layer on each frequency\),
- •WWis a learned pointwise linear transformation\.

The number of retained modeskkis typically truncated, introducing an implicit low\-pass filter that stabilizes training and improves generalization\.