Neural Slack Variables for Shape Constraints

# Neural Slack Variables for Shape Constraints
Source: [https://arxiv.org/html/2606.13803](https://arxiv.org/html/2606.13803)
Ruben Wiedemann Imperial College London London, UK r\.wiedemann22@imperial\.ac\.uk &Antoine Jacquier Imperial College London London, UK a\.jacquier@imperial\.ac\.uk &Lukas Gonon University of St\. Gallen St\. Gallen, Switzerland lukas\.gonon@unisg\.ch

###### Abstract

Enforcing functional inequality constraints such as monotonicity and convexity in neural networks is a fundamental challenge in many industrial and scientific applications\. Classical one\-sided penalty methods, along with primal\-dual methods gated by complementary slackness, provide constraint gradients only at violated locations, resulting in fragile satisfaction\. Architectures that guarantee feasibility by construction, on the other hand, remain largely limited to elementary cases and impose additional inductive biases\. We introduce*neural slack variables*, a deep learning native primal\-side approach that converts constraint enforcement into a regression problem by coupling the primary network with a jointly learned auxiliary network\. The auxiliary network serves as a valid target for the primary network’s constraint quantities, inducing feasibility and regularity\. Neural slack variables achieve zero measured violations on dense\-grid monotonicity and convexity test cases, where penalty and primal\-dual baselines leave residual violations, and enable arbitrage\-free learning of volatility surfaces, an open industrial challenge in quantitative finance\.

## 1Introduction

In scientific computing and industrial domains such as control systems and quantitative finance, neural networks serve as learned function approximators on low\-dimensional continuous domains\. The learned map must both reproduce data with high fidelity and respect known structural laws such as monotonicity, convexity, stability, or absence of arbitrage\. These are*shape constraints*: conditions on the function and its derivatives that must hold everywhere on the domain, expressed as functional inequalities\. Standard neural architectures offer no intrinsic mechanism to preserve them, and unconstrained training routinely produces models that fit the data while remaining operationally inconsistent\.

Specialized architectures that guarantee feasibility by construction \(*architectural constraints*\) exist for elementary constraint types: Constrained Monotonic Neural Networks \(CMNN\)Runje and Shankaranarayana \([2023](https://arxiv.org/html/2606.13803#bib.bib42)\); Sartoret al\.\([2025](https://arxiv.org/html/2606.13803#bib.bib44)\)for monotonicity, Input\-Convex Neural Networks \(ICNN\)Amoset al\.\([2017](https://arxiv.org/html/2606.13803#bib.bib3)\)for convexity\. These architectures restrict the hypothesis class in ways that can make them stiffer to train and bias the fit toward low\-frequency solutions\(Sivaprasadet al\.,[2021](https://arxiv.org/html/2606.13803#bib.bib46)\)\.

For general constraints and architectures, constraints must be enforced through the training procedure instead \(*soft constraints*\)\. The default method in deep learning is the*penalty method*: a violation cost term is added to the objective, computed as an aggregate of the constraint residual; its weight determines how strongly constraint satisfaction is enforced relative to data fit\.Ramirezet al\.\([2025](https://arxiv.org/html/2606.13803#bib.bib41)\)advocate primal\-dual methods as the principled alternative, which introduce Lagrangian multipliers that adaptively balance fit and feasibility\. In the deep learning regime, however, both methods exhibit the same “constraint drifting” failure mode in practice: For the penalty method, the violation term supplies no gradient wherever the network is already feasible\. Subsequent training steps can then reintroduce violations, which are corrected only after they appear, leading to repeated drift rather than stable satisfaction\. Primal\-dual methods inherit constraint drifting for analogous structural reasons: under complementary slackness, the multiplier is zero in feasible regions, again eliminating the constraint gradient \([Section˜2\.3](https://arxiv.org/html/2606.13803#S2.SS3)\)\. The outcome for both methods is often spurious violations in trained models, which is a real practical challenge\. In control systems, a learned barrier certificate carries no guarantee unless it is exact \(formal verification is an open research direction\(Daiet al\.,[2020](https://arxiv.org/html/2606.13803#bib.bib11),[2021](https://arxiv.org/html/2606.13803#bib.bib12); Edwardset al\.,[2024](https://arxiv.org/html/2606.13803#bib.bib16)\)\)\. In quantitative finance, an implied volatility surface that admits even slight arbitrage is a significant issue for any downstream pricing and risk model\(Gatheral and Jacquier,[2014](https://arxiv.org/html/2606.13803#bib.bib20); Deschâtres,[2024](https://arxiv.org/html/2606.13803#bib.bib15); Buehleret al\.,[2026](https://arxiv.org/html/2606.13803#bib.bib5)\)\. Ensuring fully satisfied constraints is a high\-impact industrial challenge even for low\-dimensional parametric approaches\.

![Refer to caption](https://arxiv.org/html/2606.13803v1/x1.png)Figure 1:∂xfθ​\(x\)\\partial\_\{x\}f^\{\\theta\}\(x\)over last 500 training epochs for monotonicity test case in[Section˜4\.1\.1](https://arxiv.org/html/2606.13803#S4.SS1.SSS1)\. White \(empty\) regions indicate violations \(negative derivatives\), feasible values are colormapped\. Penalty and Lagrangian methods exhibit “constraint drift”; neural slack variables maintain stable constraint profiles\.To overcome the issue of vanishing constraint gradients observed in the penalty method and primal\-dual methods, we propose to tie the primary network’s constraint profile to a jointly learned valid target, rather than penalizing only its violations\. Writingfθf^\{\\theta\}for the primary network, letcθ:=𝒞​\[fθ\]:Ω→ℝmc\_\{\\theta\}:=\\mathcal\{C\}\[f^\{\\theta\}\]\\colon\\Omega\\to\\mathbb\{R\}^\{m\}denote the constrained quantities\. We introduce an auxiliary neural networksϕs^\{\\phi\}—a learned analogue of the classical slack variable in constrained optimization—whose outputs are non\-negative by construction \(through an appropriate output activation function\) and provide the valid target in constraint space\. The two networks are trained jointly, enforcing the slack\-variable form

𝒞​\[fθ\]​\(x\)−sϕ​\(x\)=0,sϕ​\(x\)≥0,\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)\-s^\{\\phi\}\(x\)=0,\\qquad s^\{\\phi\}\(x\)\\geq 0,\(1\)softly through a quadratic matching loss in constraint space\. The matching term couples the two networks:sϕs^\{\\phi\}adapts to the data\-driven shape ofcθc\_\{\\theta\}, whilecθc\_\{\\theta\}is steered toward the valid targetsϕs^\{\\phi\}\. As the matching residual shrinks,cθc\_\{\\theta\}inherits the positivity ofsϕs^\{\\phi\}, makingfθf^\{\\theta\}feasible\. Crucially, becausesϕs^\{\\phi\}is a learned, finite\-capacity approximation, the matching residual generally does not fully vanish, which leaves a constraint\-space gradient after violations disappear and prevents the constraint profile from drifting back into violation\.

We termsϕs^\{\\phi\}a*neural slack variable*\. In our experiments, neural slack variables overcome the drifting failure mode that we observe for penalty and primal\-dual methods \([Figure˜1](https://arxiv.org/html/2606.13803#S1.F1)\)\. We identify an additional mechanism of the proposed method: the neural slack loss transfers the regularity ofsϕs^\{\\phi\}to the constraint profilecθc\_\{\\theta\}\([Figure˜6](https://arxiv.org/html/2606.13803#A4.F6)\), making the architecture ofsϕs^\{\\phi\}an explicit, controllable inductive bias on that profile\. We use this lever to constrain spectrally expressive primary networks \(including SIRENsSitzmannet al\.\([2020](https://arxiv.org/html/2606.13803#bib.bib45)\)and Fourier featuresTanciket al\.\([2020](https://arxiv.org/html/2606.13803#bib.bib50)\)\); in fixed\-grid settings, this can help keep satisfaction stable between collocation points\.

Viewed structurally, neural slack variables decompose the feasibility problem at the constraint operator\. Once the profilecθ=𝒞​\[fθ\]c\_\{\\theta\}=\\mathcal\{C\}\[f^\{\\theta\}\]is formed, feasibility is simply its pointwise non\-negativity; the difficulty lies in realizing a valid profile through𝒞\\mathcal\{C\}\. Loosely, this is an inverse problem, and solving it by construction requires an architecture that can only represent feasible functions, known only for a few elementary constraint types\. Neural slack variables sidestep this: non\-negativity is parameterized exactly on the slack side, while steering the primary network’s profile toward the valid target is left to joint training\.

##### Contributions

- •We propose*neural slack variables*: a soft constraint method that introduces a non\-negative auxiliary networksϕs^\{\\phi\}, jointly trained with the primary network through a matching loss in constraint space\.
- •We identify a drifting failure mode shared by penalty and primal\-dual methods, supported by its visual signature in[Figure˜1](https://arxiv.org/html/2606.13803#S1.F1)and an ablation that removes its source \([Section˜4\.2](https://arxiv.org/html/2606.13803#S4.SS2)\), and show that neural slack variables overcome it in the data\-driven setting\.
- •We identify regularity transfer fromsϕs^\{\\phi\}tofθf^\{\\theta\}as a controllable inductive bias on the constraint profile, enabling constrained learning with spectrally expressive architectures \(SIREN, Fourier features\)\.
- •We demonstrate stronger constraint satisfaction on synthetic monotonicity and convexity benchmarks, and analyze neural slack dynamics on a data\-free certification task from neural verification \(FOSSIL Barr3\)\.
- •As the main application, we apply neural slack variables to implied volatility surface modeling and obtain a robustly arbitrage\-free generative model, addressing a high\-impact industrial challenge\.

## 2Background and related work

An extended discussion of related work is provided in[Appendix˜B](https://arxiv.org/html/2606.13803#A2)\.

### 2\.1Problem setup

On a bounded domainΩ⊂ℝd\\Omega\\subset\\mathbb\{R\}^\{d\}, we learn a neural networkfθ:Ω→ℝf^\{\\theta\}\\colon\\Omega\\to\\mathbb\{R\}from data𝒟=\{\(xi,yi\)\}i=1N\\mathcal\{D\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}, subject to a functionalmm\-dimensional inequality constraint

𝒞​\[fθ\]​\(x\)≥0,for all​x∈Ω,\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)\\geq 0,\\quad\\text\{for all \}x\\in\\Omega,\(2\)where𝒞​\[⋅\]\\mathcal\{C\}\[\\cdot\]is a constraint operator mapping the function to constraint quantities \(derivatives, eigenvalues, or other functional properties\)\. We writeℒdata​\(θ\)\\mathcal\{L\}\_\{\\text\{data\}\}\(\\theta\)for the data\-fitting loss \(e\.g\.,𝔼\(x,y\)∼𝒟​‖fθ​\(x\)−y‖2\\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\}\\\|f^\{\\theta\}\(x\)\-y\\\|^\{2\}for the MSE loss\)\.

### 2\.2Penalty method

The penalty method is the de facto default for enforcing constraints in deep learning\. It augments the data loss with

ℒhinge,p​\(θ\)=1p​𝔼x∼𝒰​\(Ω\)​\[‖max⁡\(0,ε−𝒞​\[fθ\]​\(x\)\)‖pp\],\\mathcal\{L\}\_\{\\text\{hinge\},p\}\(\\theta\)=\\frac\{1\}\{p\}\\mathbb\{E\}\_\{x\\sim\\mathcal\{U\}\(\\Omega\)\}\\\!\\left\[\\left\\\|\\max\\\!\\bigl\(0,\\;\\varepsilon\-\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)\\bigr\)\\right\\\|\_\{p\}^\{p\}\\right\],\(3\)where𝒰​\(Ω\)\\mathcal\{U\}\(\\Omega\)is the Uniform distribution overΩ\\Omega,p≥1p\\geq 1\(typicallyp=1p=1to encourage sparse violation patterns\), the marginε\>0\\varepsilon\>0encourages strict satisfaction of the constraint, and “hinge” signals the use of the positive\-part function to compute the constraint residual\. The total loss becomesℒ​\(θ\):=ℒdata​\(θ\)\+ρ​ℒhinge,p​\(θ\)\\mathcal\{L\}\(\\theta\):=\\mathcal\{L\}\_\{\\text\{data\}\}\(\\theta\)\+\\rho\\,\\mathcal\{L\}\_\{\\text\{hinge\},p\}\(\\theta\), whereρ\>0\\rho\>0weights the penalty term versus the data loss\. On a gridπ=\{xj\}j=1n\\pi=\\\{x\_\{j\}\\\}\_\{j=1\}^\{n\}, this discretizes to the classical exterior penalty method applied to the sampled constraints𝒞​\[fθ\]​\(xj\)≥ε\\mathcal\{C\}\[f^\{\\theta\}\]\(x\_\{j\}\)\\geq\\varepsilon\(Nocedal and Wright,[2006](https://arxiv.org/html/2606.13803#bib.bib34), Ch\. 17\)\. Here, the level ofρ\\rhogoverns*exactness*\(whether the minimizer coincides with the constrained optimum\), soρ\\rhois a crucial tuning parameter in practice\. For the linear penalty \(p=1p=1\), exactness may be attained at sufficiently large finiteρ\\rho, while the quadratic penalty \(p=2p=2\) is generally exact only in the limitρ→∞\\rho\\to\\infty, motivating schedules that increaseρ\\rhoover training\. These considerations are, however, largely set aside in deep learning practice;Ramirezet al\.\([2025](https://arxiv.org/html/2606.13803#bib.bib41)\)advocate primal\-dual methods instead, citing in particular the burden of penalty weight tuning\.

### 2\.3Primal\-dual methods

The*primal\-dual*or*Lagrangian*formulation of \([2](https://arxiv.org/html/2606.13803#S2.E2)\) associates a non\-negative multiplier functionλ:Ω→ℝ≥0m\\lambda\\colon\\Omega\\to\\mathbb\{R\}^\{m\}\_\{\\geq 0\}with the constraint:

minθ⁡maxλ​\(⋅\)≥0⁡ℒdata​\(θ\)\+𝔼x∼𝒰​\(Ω\)​\[⟨λ​\(x\),ε−𝒞​\[fθ\]​\(x\)⟩\]\.\\min\_\{\\theta\}\\max\_\{\\lambda\(\\cdot\)\\geq 0\}\\;\\mathcal\{L\}\_\{\\text\{data\}\}\(\\theta\)\+\\mathbb\{E\}\_\{x\\sim\\mathcal\{U\}\(\\Omega\)\}\\\!\\left\[\\left\\langle\\lambda\(x\),\\;\\varepsilon\-\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)\\right\\rangle\\right\]\.\(4\)On a gridπ=\{xj\}j=1n\\pi=\\\{x\_\{j\}\\\}\_\{j=1\}^\{n\}, this discretizes to one multiplier vectorλj∈ℝ≥0m\\lambda\_\{j\}\\in\\mathbb\{R\}\_\{\\geq 0\}^\{m\}per grid point, equivalently one scalar multiplier per grid point and constraint component\.*Gradient descent\-ascent*\(GDA\) on the multiplier vector is the standard primal\-dual scheme in constrained deep learning\(Ramirezet al\.,[2025](https://arxiv.org/html/2606.13803#bib.bib41); Gallego\-Posadaet al\.,[2025](https://arxiv.org/html/2606.13803#bib.bib19)\)\. The*augmented Lagrangian*adds a quadratic violation penalty to \([4](https://arxiv.org/html/2606.13803#S2.E4)\), convexifying the local saddle and stabilizing GDA\(Platt and Barr,[1987](https://arxiv.org/html/2606.13803#bib.bib37)\); controller\-based multiplier updates offer an alternative\(Stookeet al\.,[2020](https://arxiv.org/html/2606.13803#bib.bib49); Sohrabiet al\.,[2024](https://arxiv.org/html/2606.13803#bib.bib47)\)\.

WhenΩ\\Omegais high\-dimensional or extended by conditioning variables, materializing one scalar per constraint is impractical, motivating direct parametrization ofλ\\lambdaas a networkλϕ:Ω→ℝ≥0m\\lambda\_\{\\phi\}\\colon\\Omega\\to\\mathbb\{R\}^\{m\}\_\{\\geq 0\}, a*neural multiplier*\.Narasimhanet al\.\([2020](https://arxiv.org/html/2606.13803#bib.bib32)\)introduce neural multipliers for heavily\-constrained classification, where the multiplier is indexed by a constraint feature vector \(directly analogous to the domain coordinatexxin our setting\)\. They study how the capacity ofλϕ\\lambda\_\{\\phi\}distorts the constrained problem: under\-parameterized models effectively enforce averaged transformations of the constraint set rather than individual constraints\.

### 2\.4Architectural constraints

Finally, it is possible to guarantee feasibility by explicit architecture choices\. Input\-Convex Neural Networks\(Amoset al\.,[2017](https://arxiv.org/html/2606.13803#bib.bib3)\)enforce convexity via non\-negative weights and convex activations; Constrained Monotonic Neural Networks\(Runje and Shankaranarayana,[2023](https://arxiv.org/html/2606.13803#bib.bib42); Sartoret al\.,[2025](https://arxiv.org/html/2606.13803#bib.bib44)\)guarantee monotonicity through paired sign\-restrictions on the weights and increasing activation functions\.

## 3Method

![Refer to caption](https://arxiv.org/html/2606.13803v1/x2.png)Figure 2:Monotone regression experiment with sinusoidal representation network \(SIREN\) and fixed constraint grid\. Fitted networkfθf^\{\\theta\}against noisy observations \(left\) and derivative profile∂xfθ\\partial\_\{x\}f^\{\\theta\}\(right\)\. The penalty method exhibits constraint\-grid overfitting \(the penalty loss vanishes as violations persist;[Figure˜5](https://arxiv.org/html/2606.13803#A4.F5)\), while neural slack variables limit the frequency content of the derivative profile, eliminating violations between collocation points\.##### Neural slack variables

We propose to enforce inequality constraints by augmenting the data loss with the relaxation termℒslack​\(θ,ϕ\)\\mathcal\{L\}\_\{\\text\{slack\}\}\(\\theta,\\phi\)\. Here,ϕ∈Φ\\phi\\in\\Phiare the parameters of an auxiliary networksϕ:Ω→ℝms^\{\\phi\}\\colon\\Omega\\to\\mathbb\{R\}^\{m\}\(mmthe dimension of the constraint\), which is matched tofθf^\{\\theta\}’s constraint profile:

ℒslack​\(θ,ϕ\):=12​𝔼x∼𝒰​\(Ω\)​\[‖𝒞​\[fθ\]​\(x\)−sϕ​\(x\)‖22\]\.\\mathcal\{L\}\_\{\\text\{slack\}\}\(\\theta,\\phi\):=\\frac\{1\}\{2\}\\mathbb\{E\}\_\{x\\sim\\mathcal\{U\}\(\\Omega\)\}\\left\[\\left\\\|\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)\-s^\{\\phi\}\(x\)\\right\\\|\_\{2\}^\{2\}\\right\]\.\(5\)Crucially,sϕs^\{\\phi\}is architecturally constrained to take values inK=\[ε,∞\)mK=\[\\varepsilon,\\infty\)^\{m\}through its output activation function\. We callsϕs^\{\\phi\}a*neural slack variable*: it carries the feasibility role of a slack variable, while its agreement with𝒞​\[fθ\]\\mathcal\{C\}\[f^\{\\theta\}\]is learned through the matching objective rather than imposed exactly\. The total loss becomesℒ​\(θ,ϕ\):=ℒdata​\(θ\)\+ρ​ℒslack​\(θ,ϕ\)\\mathcal\{L\}\(\\theta,\\phi\):=\\mathcal\{L\}\_\{\\text\{data\}\}\(\\theta\)\+\\rho\\,\\mathcal\{L\}\_\{\\text\{slack\}\}\(\\theta,\\phi\), a relaxation of the original constrained problem, minimized jointly in\(θ,ϕ\)\(\\theta,\\phi\)\. In practice we replaceρ​ℒslack\\rho\\,\\mathcal\{L\}\_\{\\text\{slack\}\}by the relativized lossℒslackρmax\\mathcal\{L\}^\{\\rho\_\{\\text\{max\}\}\}\_\{\\text\{slack\}\}\([Appendix˜A](https://arxiv.org/html/2606.13803#A1)\), whereρmax\\rho\_\{\\text\{max\}\}sets the maximum effective weight near the constraint boundary and no separate multiplier is used\.

##### Gradient persistence via variable splitting

The construction can be understood as variable splitting on the constraint quantity\. Letcθ​\(x\):=𝒞​\[fθ\]​\(x\)c\_\{\\theta\}\(x\):=\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)denote the constraint profile of the primary network\. Introduce an auxiliary variablessonΩ\\Omegaand replace the original constraintcθ∈Kc\_\{\\theta\}\\in Kby the constraint pair\(s=cθ\)∧\(s∈K\)\(s=c\_\{\\theta\}\)\\wedge\(s\\in K\)\. Variable splitting schemes relax the couplings=cθs=c\_\{\\theta\}\(e\.g\., via penalties or primal\-dual methods\) and thereby decouple the optimization of the primary model parametersθ\\thetafrom the formation of a feasible constraint profiless\. As the equality residual is reduced, feasibility transfers back tocθc\_\{\\theta\}\. The simple square penalty variant yields the constrained problem

minθ,s​\(⋅\)\\displaystyle\\min\_\{\\theta,s\(\\cdot\)\}\{ℒdata​\(θ\)\+12​𝔼x∼𝒰​\(Ω\)​\[‖cθ​\(x\)−s​\(x\)‖2\]\}\\displaystyle\\Big\\\{\\mathcal\{L\}\_\{\\text\{data\}\}\(\\theta\)\+\\frac\{1\}\{2\}\\,\\mathbb\{E\}\_\{x\\sim\\mathcal\{U\}\(\\Omega\)\}\\left\[\\left\\\|c\_\{\\theta\}\(x\)\-s\(x\)\\right\\\|^\{2\}\\right\]\\Big\\\}subject tos​\(⋅\)∈K\.\\displaystyle s\(\\cdot\)\\in K\.\(6\)For fixedθ\\theta, the exactss\-subproblem solved by the pointwise Euclidean projectionΠK​\(cθ​\(x\)\)=max⁡\(ε,cθ​\(x\)\)\\Pi\_\{K\}\(c\_\{\\theta\}\(x\)\)=\\max\(\\varepsilon,c\_\{\\theta\}\(x\)\)\. Substituted into \([3](https://arxiv.org/html/2606.13803#S3.Ex1)\) this gives precisely the objective of the quadratic penalty method\.

Instead of optimizing \([3](https://arxiv.org/html/2606.13803#S3.Ex1)\) over all feasible auxiliary profiles, we restrict the auxiliary profile to the neural slack class𝒮=\{sϕ:ϕ∈Φ\}\\mathcal\{S\}=\\\{s^\{\\phi\}\\colon\\phi\\in\\Phi\\\}and optimize\(θ,ϕ\)\(\\theta,\\phi\)jointly\. Neural slack variables replace the exactss\-projection by a learned, finite\-capacity, jointly optimized surrogate\. The finite capacity “approximate projection” is the key: The residualrθ,ϕ​\(x\)=cθ​\(x\)−sϕ​\(x\)r\_\{\\theta,\\phi\}\(x\)=c\_\{\\theta\}\(x\)\-s^\{\\phi\}\(x\)generally does not collapse to zero throughout the slack region, so thatℒslack\\mathcal\{L\}\_\{\\text\{slack\}\}continues to provide an “anchoring” gradient even ascθ≥εc\_\{\\theta\}\\geq\\varepsilon, stabilizing feasible constraint profiles and preventing drift\.

##### Inductive bias on the constraint profile

Restricting the auxiliary profile to the finite\-capacity class𝒮\\mathcal\{S\}adds a term to the inner\-minimized slack loss:

infϕℒslack​\(θ,ϕ\)≥ℒhinge,2​\(θ\)\+12​distL22⁡\(ΠK​\(cθ\),𝒮\)\.\\inf\_\{\\phi\}\\mathcal\{L\}\_\{\\text\{slack\}\}\(\\theta,\\phi\)\\;\\geq\\;\\mathcal\{L\}\_\{\\text\{hinge\},2\}\(\\theta\)\\;\+\\;\\frac\{1\}\{2\}\\,\\operatorname\{dist\}\_\{L^\{2\}\}^\{2\}\\\!\\bigl\(\\Pi\_\{K\}\(c\_\{\\theta\}\),\\,\\mathcal\{S\}\\bigr\)\.\(7\)The bound is the projection decomposition of‖cθ−sϕ‖2\\\|c\_\{\\theta\}\-s^\{\\phi\}\\\|^\{2\}, valid because𝒮⊂K\\mathcal\{S\}\\subset Kby construction\. On feasibleθ\\theta, the quadratic violation penalty term vanishes and the inner\-minimized slack loss reduces to12​distL22⁡\(cθ,𝒮\)\\tfrac\{1\}\{2\}\\,\\operatorname\{dist\}\_\{L^\{2\}\}^\{2\}\(c\_\{\\theta\},\\mathcal\{S\}\)\. Therefore, the slack loss selects, among feasibleθ\\thetaof equal data fit, one whose constraint profile lies closest to𝒮\\mathcal\{S\}\. This is an explicit inductive bias on the shape ofcθc\_\{\\theta\}, set by the architecture ofsϕs^\{\\phi\}rather thanfθf^\{\\theta\}\.

The strength of the selection depends on the capacity of𝒮\\mathcal\{S\}\. A small𝒮\\mathcal\{S\}imposes a strong shape prior at the cost of underfitting, while excessive𝒮\\mathcal\{S\}destabilizes joint optimization\. The useful regime lies between these extremes:𝒮\\mathcal\{S\}matched to the constraint profile offθf^\{\\theta\}on the data\-relevant subclass\.

## 4Experiments

We evaluate neural slack variables against penalty and \(augmented\) Lagrangian methods; for monotonicity and convexity we additionally report CMNN and ICNN as architectural baselines\. The differential constraint operators we consider amplify high\-frequency content, in tension with an MLP\-based neural slack class\(Rahamanet al\.,[2019](https://arxiv.org/html/2606.13803#bib.bib38); Ramasinghe and Lucey,[2022](https://arxiv.org/html/2606.13803#bib.bib40)\)\. For the neural slack variable architecture we therefore default to sinusoidal representation networks \(SIRENs;\(Sitzmannet al\.,[2020](https://arxiv.org/html/2606.13803#bib.bib45)\)\) with input frequency scaleω0=5\\omega\_\{0\}=5, biasing the neural slack variable toward smoothness rather than high\-bandwidth representation, and ablate this choice in[Section˜4\.1\.1](https://arxiv.org/html/2606.13803#S4.SS1.SSS1)and[Section˜4\.2](https://arxiv.org/html/2606.13803#S4.SS2)\. For outputs valued inK=\[ε,∞\)mK=\[\\varepsilon,\\infty\)^\{m\}, we use the componentwise shifted squareε\+\(⋅\)2\\varepsilon\+\(\\cdot\)^\{2\}by default\.

We also consider higher bandwidth architectures \(moderate SIRENs and Fourier features\) forfθf^\{\\theta\}in selected experiments; by reducing the low\-frequency bias of plain MLPs, these settings arguably better isolate the effect of the constraint\-enforcement mechanism\.

##### Training and hyperparameters

With the exception of Experiment[4\.3](https://arxiv.org/html/2606.13803#S4.SS3), the datasets are small enough to resolveℒdata\\mathcal\{L\}\_\{\\text\{data\}\}on the full training set \(full\-batch gradient descent\)\. All method\-specific constraint objectives are evaluated on a separate constraint gridπtrain\\pi\_\{\\text\{train\}\}, with marginε=10−6\\varepsilon=10^\{\-6\}\. For the penalty method, we use the linear variant \(p=1p=1\)\. For penalty and augmented\-Lagrangian baselines, we update the scalar penalty coefficient with a progress\-based multiplicative rule driven by the maximum constraint violation onπtrain\\pi\_\{\\text\{train\}\}, capped at a maximum value\. Per\-experiment training details \(number of epochs, learning rate schedule, network architecture\) are reported alongside each benchmark\.

##### Evaluation metrics

We report the train mean absolute errorδmae\\delta\_\{\\text\{mae\}\}and three constraint statistics computed from a fine evaluation gridπeval\\pi\_\{\\text\{eval\}\}: the violation rateηrate\\eta\_\{\\text\{rate\}\}, the average violation magnitudeηmean\\eta\_\{\\text\{mean\}\}, and the worst\-case magnitudeηmax\\eta\_\{\\text\{max\}\}\(definitions in[Appendix˜C](https://arxiv.org/html/2606.13803#A3)\)\.

### 4\.1Testcases: Monotonicity and Convexity

#### 4\.1\.1Monotonicity \(d=1d=1\)

We learn a monotonically increasing function onΩ=\[−1,1\]\\Omega=\[\-1,1\]from100100noisy observations of a handcrafted target with an adversarial, slightly negatively sloped terrace in the center of the interval \(discernible from[Figure˜2](https://arxiv.org/html/2606.13803#S3.F2); full details in[Section˜D\.1](https://arxiv.org/html/2606.13803#A4.SS1)\)\. The constraint𝒞​\[f\]​\(x\)=∂f​\(x\)≥0\\mathcal\{C\}\[f\]\(x\)=\\partial f\(x\)\\geq 0is enforced on a200200\-point training gridπtrain\\pi\_\{\\text\{train\}\}and verified on a10,00010\{,\}000\-point evaluation gridπeval\\pi\_\{\\text\{eval\}\}\. We train full\-batch for10,00010\{,\}000epochs with Adam \(learning rate10−310^\{\-3\}, cosine\-annealed to zero on the final30%30\\%\)\.fθf^\{\\theta\}is a width\-1616four\-layer softplus MLP, sized to match the CMNN baseline’s minimum depth requirement\.

##### Results

[Table˜1](https://arxiv.org/html/2606.13803#S4.T1)aggregates results over100100seeds\. Among the soft methods, only neural slack variables achieve zero violations on every seed\. The augmented Lagrangian clears73/10073/100runs against2/1002/100for both the penalty method and the plain Lagrangian; among the latter two, the plain Lagrangian fares strictly worse than the penalty method on every violation metric\.[Figure˜1](https://arxiv.org/html/2606.13803#S1.F1)traces the mechanism: the penalty and \(augmented\) Lagrangian methods continue to drift across the constraint inequality after the data fit converges, while the neural slack variable stabilizes on a strictly non\-negative profile\.

Table 1:Monotone \(MLPfθf^\{\\theta\}\): mean±\\pmstd overN=100N=100seeds;nsat/Nn\_\{\\text\{sat\}\}/Ncounts seeds withηrate=0\\eta\_\{\\text\{rate\}\}=0\. The neural slack variable induces feasibility on every seed\. Apple M3 Pro \(CPU\)\.Among the feasible methods \([Table˜1](https://arxiv.org/html/2606.13803#S4.T1)above\), CMNN reaches the bestδmae\\delta\_\{\\text\{mae\}\}with an MLPfθf^\{\\theta\}while, in this configuration, the neural slack variable’s constraint enforcement costs the most accuracy\. In the following two paragraphs we investigate the role of the neural slack architecture as well as the choice of the primary network on these metrics\.

##### Ablation: Neural slack variable architecture and capacity

The neural slack variable used for[Table˜1](https://arxiv.org/html/2606.13803#S4.T1)is a SIREN withω0=5\\omega\_\{0\}=5and three hidden layers of width 16\. We ablate this choice and by contrasting withsoftplus\\mathrm\{softplus\}\-MLPs and sweeping both architectures’ widths\.[Figure˜3\(a\)](https://arxiv.org/html/2606.13803#S4.F3.sf1)reportsδmae\\delta\_\{\\text\{mae\}\},ℒslack​\(θ,ϕ\)\\mathcal\{L\}\_\{\\text\{slack\}\}\(\\theta,\\phi\)andcminθc^\{\\theta\}\_\{\\min\}across55seeds; constraint satisfaction is exact at every width for both architectures\.

The SIREN slack attains a lowerδmae\\delta\_\{\\text\{mae\}\}than the MLP slack across the full sweep, with the gap closing atW=16W=16\. Both architectures exhibit a U\-shape inδmae\\delta\_\{\\text\{mae\}\}: capacity is needed at the lower end, where a narrow neural slack variable overly restricts the expressivity of the primary network, and excess capacity destabilises the optimization\. The failure modes atW=256W=256differ in quality \([Figures˜6](https://arxiv.org/html/2606.13803#A4.F6)and[7](https://arxiv.org/html/2606.13803#A4.F7)\)\. Thesoftplus\\mathrm\{softplus\}\-MLP slack collapses to zero, which forcesfθf^\{\\theta\}itself toward a constant predictor and raisesδmae\\delta\_\{\\text\{mae\}\}above one\. The SIREN slack instead settles into a pathological microscopic\-oscillation state that still traces a data\-compatible profile on a macroscopic level, soδmae\\delta\_\{\\text\{mae\}\}rises modestly without catastrophic collapse\.

![Refer to caption](https://arxiv.org/html/2606.13803v1/x3.png)\(a\)Monotonicity: Neural slack variable, hidden width sweep\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x4.png)\(b\)Convexity: Increasing dimension

Figure 3:\(a\)Neural slack variable width sweep on the monotone MLPfθf^\{\\theta\}, comparing a SIREN slack against a softplus\-MLP slack\. Low neural slack capacity hurtsδmae\\delta\_\{\\text\{mae\}\}for the MLP slack disproportionally; excessive capacity destabilizes the MLP slack atW=256W=256\. The SIREN slack stays flat across the sweep; both choices keepcminθc^\{\\theta\}\_\{\\min\}positive at every width\.\(b\)Convex\-ND \(MLPfθf^\{\\theta\}\),55seeds: data MAE \(left\) and worst\-case violationηmax\\eta\_\{\\text\{max\}\}\(right\) versus input dimensiondd, comparing the neural slack variable against the penalty method, an unconstrained reference, and the architecturally constrained ICNN\. Bands are±1\\pm 1std forδmae\\delta\_\{\\text\{mae\}\}and seed min/max forηmax\\eta\_\{\\text\{max\}\}\. Feasibility degrades withddas the constraint training grid thins for the penalty method and neural slack variable; the ICNN remains feasible by construction but trails onδmae\\delta\_\{\\text\{mae\}\}at everydd\.
##### Using a SIREN forfθf^\{\\theta\}

Using a width\-1616three\-layer SIREN forfθf^\{\\theta\}widens the spectral envelope available to the regression itself\.[Figure˜2](https://arxiv.org/html/2606.13803#S3.F2)contrastsfθf^\{\\theta\}and∂xfθ\\partial\_\{x\}f^\{\\theta\}in this setting\. The fixed\-grid run exposes constraint\-grid overfitting for the penalty method: its penalty loss term vanishes while violations persist \([Figure˜5](https://arxiv.org/html/2606.13803#A4.F5)\), whereas the neural slack variable selects a lower\-frequency derivative profile that remains positive between collocation points\. The corresponding table in[Section˜D\.3](https://arxiv.org/html/2606.13803#A4.SS3)reportsδmae=0\.071\\delta\_\{\\text\{mae\}\}=0\.071for the neural slack variable against0\.0810\.081for CMNN, with both feasible on every seed, whereas the penalty method leaves substantial residual violations and the Lagrangian methods suffer failures, with primal\-dual updates struggling to handle the SIREN’s constraint profile\. Using neural slack variables moves constraint enforcement out of the primary architecture, freeing enough capacity forfθf^\{\\theta\}to overtake CMNN onδmae\\delta\_\{\\text\{mae\}\}while preserving feasibility in every run\.

#### 4\.1\.2Convexity \(d∈\{2,3,4,5\}d\\in\\\{2,3,4,5\\\}\)

We learn convex functions onΩ=\[−1,1\]d\\Omega=\[\-1,1\]^\{d\}ford∈\{2,3,4,5\}d\\in\\\{2,3,4,5\\\}from a smoothed log\-sum\-exp target withK=10K=10random affine pieces \(full details in[Section˜E\.1](https://arxiv.org/html/2606.13803#A5.SS1)\)\. The constraint𝒞​\[f\]​\(x\)=eig​∇2f​\(x\)≥0\\mathcal\{C\}\[f\]\(x\)=\\operatorname\{eig\}\\nabla^\{2\}f\(x\)\\geq 0requires every Hessian eigenvalue to be non\-negative\. We train full\-batch with Adam onN=103N=10^\{3\}noisy observations for10,00010\{,\}000epochs \(learning rate10−310^\{\-3\}, cosine\-annealed to zero on the final30%30\\%\)\. At each optimizer step, the constraint objective is evaluated on a newly drawn Sobol gridπtrain\\pi\_\{\\text\{train\}\}with\|πtrain\|=104\|\\pi\_\{\\text\{train\}\}\|=10^\{4\}\. Evaluation uses an independent Sobol gridπeval\\pi\_\{\\text\{eval\}\}with\|πeval\|=107\|\\pi\_\{\\text\{eval\}\}\|=10^\{7\}\. The primary modelfθf^\{\\theta\}is a width\-128128three\-layersoftplus\\mathrm\{softplus\}\-MLP, matching the activation class required by the ICNN baseline; the neural slack variablesϕs^\{\\phi\}is a width\-128128three\-layer SIREN with input frequencyω0=5\\omega\_\{0\}=5\. We run55seeds per dimension, alongside an unconstrained reference\. Per\-dimension tables andd=2d=2surface plots are in[Appendix˜E](https://arxiv.org/html/2606.13803#A5)\.

##### Results

[Figure˜3\(b\)](https://arxiv.org/html/2606.13803#S4.F3.sf2)shows the scaling across input dimension; bands are±1\\pm 1std forδmae\\delta\_\{\\text\{mae\}\}and seed min/max forηmax\\eta\_\{\\text\{max\}\}, so anηmax\\eta\_\{\\text\{max\}\}band whose lower edge is at zero contains at least one feasible seed\. Atd=2d=2neural slack variables ensure feasibility on every seed, while the penalty method fails on every seed \(ηmax≈2×10−3\\eta\_\{\\text\{max\}\}\\approx 2\\times 10^\{\-3\}\)\. Atd=3d=3neural slack variables start to leak violations for some seeds, while the penalty method grows by an order of magnitude \(ηmax≈2×10−2\\eta\_\{\\text\{max\}\}\\approx 2\\times 10^\{\-2\}\); atd=4,5d=4,5neither method clears any seed, and the penalty’s worst\-case violation is three to five times the slack variable’s\.δmae\\delta\_\{\\text\{mae\}\}remains comparable between both methods at everydd\. We read this as the curse of dimensionality acting on learned constraint enforcement: at fixed grid budget the per\-volume sample density thins exponentially, and methods whose feasibility relies on dense pointwise supervision degrade accordingly\. ICNNs are convex by construction but underfit atd=2,3d=2,3, consistent with the slower training dynamics documented for input\-convex architectures\(Sivaprasadet al\.,[2021](https://arxiv.org/html/2606.13803#bib.bib46); Hoedt and Klambauer,[2023](https://arxiv.org/html/2606.13803#bib.bib24)\)\.

### 4\.2Neural barrier certificates

We turn to a multi\-component safety certificate drawn from the FOSSIL Barr3 benchmark\(Edwardset al\.,[2024](https://arxiv.org/html/2606.13803#bib.bib16)\)\. On the autonomous polynomial systemx˙=Φ​\(x\)\\dot\{x\}=\\Phi\(x\)withΦ​\(x\)=\(x2,−x1−x2\+13​x13\)\\Phi\(x\)=\(x\_\{2\},\\;\-x\_\{1\}\-x\_\{2\}\+\\tfrac\{1\}\{3\}\\,x\_\{1\}^\{3\}\)overΩ=\[−3,2\.5\]×\[−2,1\]\\Omega=\[\-3,2\.5\]\\times\[\-2,1\], we trainfθ:Ω→ℝf^\{\\theta\}\\colon\\Omega\\to\\mathbb\{R\}whose certified\-safe set\{x:fθ​\(x\)≥0\}\\\{x:f^\{\\theta\}\(x\)\\geq 0\\\}separates an initial setΩi\\Omega\_\{\\mathrm\{i\}\}from an unsafe setΩu\\Omega\_\{\\mathrm\{u\}\}, both unions of axis\-aligned rectangles and disks \(full geometry in[Section˜F\.1](https://arxiv.org/html/2606.13803#A6.SS1)\)\. Folding the boundary sign anchors and the Nagumo / control\-barrier inequality\(Ameset al\.,[2019](https://arxiv.org/html/2606.13803#bib.bib2); Dawsonet al\.,[2023](https://arxiv.org/html/2606.13803#bib.bib13)\)into a single vector\-valued constraint:

𝒞​\[fθ\]​\(x\)=\(∇fθ​\(x\)⋅Φ​\(x\)\+γ​fθ​\(x\),1Ωi​\(x\)​\(fθ​\(x\)−μi\),1Ωu​\(x\)​\(−fθ​\(x\)−μu\)\)≥0,\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)=\\bigl\(\\nabla f^\{\\theta\}\(x\)\\cdot\\Phi\(x\)\+\\gamma\\,f^\{\\theta\}\(x\),\\;\\mathbf\{1\}\_\{\\Omega\_\{\\mathrm\{i\}\}\}\(x\)\\bigl\(f^\{\\theta\}\(x\)\-\\mu\_\{\\mathrm\{i\}\}\\bigr\),\\;\\mathbf\{1\}\_\{\\Omega\_\{\\mathrm\{u\}\}\}\(x\)\\bigl\(\-f^\{\\theta\}\(x\)\-\\mu\_\{\\mathrm\{u\}\}\\bigr\)\\bigr\)\\;\\geq\\;0,

\(8\)withγ=1\\gamma=1and strict\-positivity marginsμi=μu=10−3\\mu\_\{\\mathrm\{i\}\}=\\mu\_\{\\mathrm\{u\}\}=10^\{\-3\}on the boundary anchors so that the certificate’s safe set has non\-empty interior onΩi\\Omega\_\{\\mathrm\{i\}\}and a strict separation fromΩu\\Omega\_\{\\mathrm\{u\}\}\. We give details about training and evaluation grids in[Section˜F\.1](https://arxiv.org/html/2606.13803#A6.SS1)\.

This setting removes the data\-fitting term \(ℒdata≡0\\mathcal\{L\}\_\{\\text\{data\}\}\\equiv 0\), leaving the constraint\-enforcement term as the only signal forθ\\theta\. It therefore removes the source of constraint drift identified in[Figure˜1](https://arxiv.org/html/2606.13803#S1.F1)\. Indeed, the literature\-default two\-layer width\-1010squared\-activation MLP\(Edwardset al\.,[2024](https://arxiv.org/html/2606.13803#bib.bib16); de Moura and Bjørner,[2008](https://arxiv.org/html/2606.13803#bib.bib14)\)reaches feasibility within about2,0002\{,\}000gradient steps under the penalty method; at this point, with no data term and zero penalty loss, its parameters freeze\. This setting is therefore the controlled counterpart to[Figure˜1](https://arxiv.org/html/2606.13803#S1.F1): the penalty method suffers from fragile constraint satisfaction under a competing data fit, but not in its absence\. This setting is therefore the controlled counterpart to[Figure˜1](https://arxiv.org/html/2606.13803#S1.F1): the penalty method exhibits fragile constraint satisfaction under a competing data fit, but not when the data term is absent\.

![Refer to caption](https://arxiv.org/html/2606.13803v1/)Figure 4:Neural slack variable width sweep on FOSSIL Barr3,55seeds per cell \(lines = seed mean, bands = seed min/max\)\. Linear panel:cminθc^\{\\theta\}\_\{\\min\}\(dashed line marks feasibility; the band lower edge is the worst\-seed feasibility\)\. Log panels: slack lossℒslack\\mathcal\{L\}\_\{\\text\{slack\}\}, its tail oscillationστ\\sigma\_\{\\tau\}, the relative parameter displacement‖Δ​ϕ‖2/‖ϕ0‖2\\\|\\Delta\\phi\\\|\_\{2\}/\\\|\\phi\_\{0\}\\\|\_\{2\}, and the relative output displacement‖Δ​sϕ‖2/‖s0ϕ‖2\\\|\\Delta s^\{\\phi\}\\\|\_\{2\}/\\\|s^\{\\phi\}\_\{0\}\\\|\_\{2\}on a fixed grid\. SIREN is feasible only at intermediate widthsh∈\{8,16,32\}h\\in\\\{8,16,32\\\}; thetanh\\tanh\-MLP slack is infeasible at every width tried\.##### Neural slack variable dynamics

For neural slack variables, the absence of a data term makes Barr3 a diagnostic of the method’s own training dynamics\.[Figure˜4](https://arxiv.org/html/2606.13803#S4.F4)sweeps the hidden width of a three\-layer neural slack variablesϕs^\{\\phi\}over\{1,2,…,256\}\\\{1,2,\\ldots,256\\\}for SIREN \(ω0=5\\omega\_\{0\}=5\) andtanh\\tanh\-MLP architectures, trained full\-batch for5,0005\{,\}000epochs \(details in[Section˜F\.2](https://arxiv.org/html/2606.13803#A6.SS2)\)\. Feasibility requiressϕs^\{\\phi\}to represent an inflated, sink\-shaped constraint profile, even though the certificatefθf^\{\\theta\}itself remains small in magnitude\.[Figure˜4](https://arxiv.org/html/2606.13803#S4.F4)identifies the collapse regime: since the neural slack variable is initialized at constant unit output,s0ϕ≡1s^\{\\phi\}\_\{0\}\\equiv 1, collapse toward zero gives‖Δ​sϕ‖2/‖s0ϕ‖2≈1\\\|\\Delta s^\{\\phi\}\\\|\_\{2\}/\\\|s^\{\\phi\}\_\{0\}\\\|\_\{2\}\\approx 1\(rightmost panel\)\. Thetanh\\tanh\-MLP slack variable remains in this regime at every width, failing to induce feasibility\. The SIREN slack variable leaves this regime only at intermediate widthsh∈\{8,16,32\}h\\in\\\{8,16,32\\\}, with peak feasibility ath=16h=16\. The tail oscillationστ\\sigma\_\{\\tau\}\(standard deviation ofℒslack\\mathcal\{L\}\_\{\\text\{slack\}\}over the last1,5001\{,\}500epochs\) is lowest in the same intermediate\-width regime and rises at both extremes\. The high\-width end aligns with the capacity failure seen in the monotone ablation \([Figure˜7](https://arxiv.org/html/2606.13803#A4.F7)\), wheresϕs^\{\\phi\}appears to move through SIREN feature combinations\. The low\-width end suggests that joint training offθf^\{\\theta\}andsϕs^\{\\phi\}fails to settle into a stable configuration, with no data\-fitting gradient to anchor it\.

### 4\.3No\-arbitrage volatility surfaces

Constructing arbitrage\-free implied volatility surfaces is a longstanding challenge in equity\-options trading: even standard low\-dimensional parametrizations\(Gatheral and Jacquier,[2014](https://arxiv.org/html/2606.13803#bib.bib20); Lucic,[2021](https://arxiv.org/html/2606.13803#bib.bib30)\)require careful constraint handling\. Deep\-learning surrogates predominantly enforce no\-arbitrage softly through penalty methods\(Ackereret al\.,[2020](https://arxiv.org/html/2606.13803#bib.bib1); Chataigneret al\.,[2020](https://arxiv.org/html/2606.13803#bib.bib8); Zhenget al\.,[2021](https://arxiv.org/html/2606.13803#bib.bib54); Bergeronet al\.,[2022](https://arxiv.org/html/2606.13803#bib.bib4); Yanget al\.,[2025](https://arxiv.org/html/2606.13803#bib.bib52); Wiedemannet al\.,[2025](https://arxiv.org/html/2606.13803#bib.bib51)\); satisfaction rates are either reported as imperfect or not reported at all, implying residual arbitrage at deployment\. Architecturally constrained models are arbitrage\-free by construction, but their mismatch with market data is well documented and has limited their adoption in practice\(Chataigneret al\.,[2020](https://arxiv.org/html/2606.13803#bib.bib8)\)\. We adopt this as the paper’s main application:251251daily NDX \(Nasdaq\-100\) option snapshots from OptionMetrics, fit jointly through a variational autodecoder\(Zadehet al\.,[2021](https://arxiv.org/html/2606.13803#bib.bib53)\)that maps a per\-surface latent codezi∈ℝ32z\_\{i\}\\in\\mathbb\{R\}^\{32\}and grid coordinates\(τ,k\)\(\\tau,k\)\(time\-to\-expiry and log\-moneyness\) to implied volatility through a FiLM\-ed SIRENfθ​\(τ,k;zi\)f^\{\\theta\}\(\\tau,k;z\_\{i\}\)\(Perezet al\.\([2018](https://arxiv.org/html/2606.13803#bib.bib36)\); architecture details in[Section˜G\.3](https://arxiv.org/html/2606.13803#A7.SS3)\)\.

Writingv​\(τ,k\):=fθ​\(τ,k;z\)​τv\(\\tau,k\):=f^\{\\theta\}\(\\tau,k;z\)\\sqrt\{\\tau\}for*total volatility*andw:=v2w:=v^\{2\}for*total variance*, the no\-arbitrage constraint is two\-dimensional:

𝒞​\[fθ\]=\(𝒞cal​\[fθ\],𝒞str​\[fθ\]\)=\(∂τw,\(1\+d1​∂kv\)​\(1\+d2​∂kv\)\+v​∂k2v\)≥0,\\mathcal\{C\}\[f^\{\\theta\}\]=\\bigl\(\\mathcal\{C\}\_\{\\text\{cal\}\}\[f^\{\\theta\}\],\\;\\mathcal\{C\}\_\{\\text\{str\}\}\[f^\{\\theta\}\]\\bigr\)=\\bigl\(\\partial\_\{\\tau\}w,\\;\(1\+d\_\{1\}\\,\\partial\_\{k\}v\)\(1\+d\_\{2\}\\,\\partial\_\{k\}v\)\+v\\,\\partial\_\{k\}^\{2\}v\\bigr\)\\;\\geq\\;0,\(9\)whered1,2​\(τ,k,v\):=−k/v±v/2d\_\{1,2\}\(\\tau,k,v\):=\-k/v\\pm v/2\(details in[Section˜G\.1](https://arxiv.org/html/2606.13803#A7.SS1)\)\. The*local volatility*a​\(τ,k\)a\(\\tau,k\)derived from𝒞​\[fθ\]\\mathcal\{C\}\[f^\{\\theta\}\]via Dupire’s equation \(a2=𝒞cal/𝒞stra^\{2\}=\\mathcal\{C\}\_\{\\text\{cal\}\}/\\mathcal\{C\}\_\{\\text\{str\}\}\) is a standard diffusion model for pricing exotic derivatives consistently with the observed option prices; residual violations of either constraint leaveaaundefined\. Deployment\-grade surfaces therefore require strict enforcement of both constraints\.

The setting introduces a qualitatively new challenge: a high\-dimensional, structured constraint domain\. The shape constraint targets the surface coordinates\(τ,k\)∈\[τmin,τmax\]×\[kmin,kmax\]\(\\tau,k\)\\in\[\\tau\_\{\\text\{min\}\},\\tau\_\{\\text\{max\}\}\]\\times\[k\_\{\\text\{min\}\},k\_\{\\text\{max\}\}\]but is extended pointwise by conditioning on the latent, so the effective constraint domain isΩ⊆ℝ34\\Omega\\subseteq\\mathbb\{R\}^\{34\}withx=\(τ,k,z\)x=\(\\tau,k,z\)\. Mini\-batches of88surfaces per step further make the constraint grid seen by each update very sparse, so the constraint enforcement method must interpolate cleanly through both the spatial and the latent dimensions under that noise\. Finally, feasibility must hold beyond the trained codes\{zi\}i\\\{z\_\{i\}\\\}\_\{i\}\(train\) to make the generative setting useful; we additionally evaluate it on linear interpolations of pairwise sampled trained codes \(interpolation\) and at samples from the latent prior \(prior\), the deployment regime for unconditional sampling\.

##### Results

[Table˜2](https://arxiv.org/html/2606.13803#S4.T2)compares the penalty method, the neural augmented Lagrangian, and neural slack variables\. Neural slack variables are the only method to fully eliminate violations across all three regimes \(train,interpolation, and the deployment\-regimeprior\)\. Moreover, it achieves the best data fit; the augmented Lagrangian attains the next\-best fit but the largest residual violations across all regimes\. Neural slack variables thereby establish a viable approach to arbitrage\-free learning of volatility surfaces, in our view an open practical challenge of high industrial impact\.

Table 2:IV\-surface autodecoder: method comparison\. Lower is better\. Only the neural slack variable is feasible on every regime, at the lowest fit error\. Measured on Apple M3 Pro \(MPS\)\.trainpairwisepriorMethodδmae\\delta\_\{\\text\{mae\}\}ηrate\\eta\_\{\\text\{rate\}\}ηmax\\eta\_\{\\text\{max\}\}ηrate\\eta\_\{\\text\{rate\}\}ηmax\\eta\_\{\\text\{max\}\}ηrate\\eta\_\{\\text\{rate\}\}ηmax\\eta\_\{\\text\{max\}\}Time \(h\)Penalty1\.0e\-025\.6e\-042\.7e\-033\.3e\-041\.6e\-034\.9e\-042\.8e\-031\.1Neural Aug\. Lag\.7\.8e\-031\.2e\-034\.9e\-031\.1e\-034\.4e\-031\.5e\-037\.7e\-031\.2Neural Slack Var\.6\.9e\-030000001\.3

## 5Conclusion

Neural slack variables enforce shape constraints by pairing the primary network with an auxiliary network that is feasible by construction and jointly trained as a target for the constraint profile offθf^\{\\theta\}\. The construction overcomes the drifting failure mode of penalty and primal\-dual methods \([Figure˜1](https://arxiv.org/html/2606.13803#S1.F1)\); removing the data loss in the barrier experiment isolates the cause \([Section˜4\.2](https://arxiv.org/html/2606.13803#S4.SS2)\)\. On dense\-grid synthetic benchmarks the method achieves zero measured violations on every seed; with a spectrally expressive primary network \(SIREN, Fourier features\) it maintains constraint enforcement while achieving a lower data error than the architectural baselines, a regime not reached by the penalty or primal\-dual baselines in our experiments\. On the high\-dimensional volatility surface application \([Section˜4\.3](https://arxiv.org/html/2606.13803#S4.SS3)\), a high\-profile open challenge in deep learning for quantitative finance, it is the only method to satisfy the no\-arbitrage constraints on every evaluation regime while attaining the lowest data fit error of the three methods\.

The neural slack architecture is an explicit inductive bias on the constraint profile, with a U\-shaped capacity tradeoff\. Because differential constraint operators amplify high frequencies, the neural slack variable typically needs more spectral expressiveness thanfθf^\{\\theta\}\.

##### Limitations

Learned constraint enforcement on a sampled grid scales poorly with input dimension; neural slack variables degrade more slowly than the penalty method but do not avoid this scaling issue \([Section˜4\.1\.2](https://arxiv.org/html/2606.13803#S4.SS1.SSS2)\)\. The neural slack variable roughly doubles parameter count and adds an architectural design axis the penalty method does not carry, though dense grid constraint evaluation dominates the per\-step cost in either case\. Off\-grid feasibility is not formally certified, so architectural constraints remain the appropriate default where available and applicable\.

## Acknowledgments and Disclosure of Funding

AJ gratefully acknowledges financial support from the EPSRC grants EP/W032643/1 and EP/T032146/1\. RW was supported by the Department of Mathematics, Imperial College, through the Roth scholarship scheme\. Computational resources and support were provided by the Imperial College Research Computing Service\(Harvey,[2017](https://arxiv.org/html/2606.13803#bib.bib22)\)\. For the purpose of open access, the authors have applied a Creative Commons Attribution \(CC BY\) licence to any Author Accepted Manuscript version arising from this work\.

## References

- Deep smoothing of the implied volatility surface\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 11552–11563\.Cited by:[§G\.4](https://arxiv.org/html/2606.13803#A7.SS4.p1.15),[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- A\. D\. Ames, S\. Coogan, M\. Egerstedt, G\. Notomista, K\. Sreenath, and P\. Tabuada \(2019\)Control barrier functions: Theory and applications\.In17th European Control Conference, ECC 2019, Naples, Italy, June 25\-28, 2019,pp\. 3420–3431\.External Links:[Document](https://dx.doi.org/10.23919/ECC.2019.8796030)Cited by:[§F\.2](https://arxiv.org/html/2606.13803#A6.SS2.p2.2),[§4\.2](https://arxiv.org/html/2606.13803#S4.SS2.p1.7)\.
- B\. Amos, L\. Xu, and J\. Z\. Kolter \(2017\)Input convex neural networks\.InProceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6\-11 August 2017,D\. Precup and Y\. W\. Teh \(Eds\.\),Proceedings of Machine Learning Research, Vol\.70,pp\. 146–155\.Cited by:[§E\.2](https://arxiv.org/html/2606.13803#A5.SS2.p1.3),[§1](https://arxiv.org/html/2606.13803#S1.p2.1),[§2\.4](https://arxiv.org/html/2606.13803#S2.SS4.p1.1)\.
- M\. Bergeron, N\. Fung, J\. Hull, Z\. Poulos, and A\. Veneris \(2022\)Variational autoencoders: a hands\-off approach to volatility\.The Journal of Financial Data Science4,pp\. jfds\.2022\.1\.093\.External Links:[Document](https://dx.doi.org/10.3905/jfds.2022.1.093)Cited by:[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- H\. Buehler, B\. Horvath, A\. Kratsios, Y\. Limmer, and R\. Saqur \(2026\)SANOS \(Smooth strictly Arbitrage\-free Non\-parametric Option Surfaces\)\.SSRN Scholarly Paper,Social Science Research Network,Rochester, NY\.External Links:6080287,[Document](https://dx.doi.org/10.2139/ssrn.6080287)Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p3.1)\.
- L\. F\. O\. Chamon, S\. Paternain, M\. Calvo\-Fullana, and A\. Ribeiro \(2023\)Constrained learning with non\-convex losses\.IEEE Transactions on Information Theory69\(3\),pp\. 1739–1760\.External Links:[Document](https://dx.doi.org/10.1109/TIT.2022.3187948)Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px1.p1.1)\.
- L\. F\. O\. Chamon and A\. Ribeiro \(2020\)Probably approximately correct constrained learning\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, Virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px1.p1.1)\.
- M\. Chataigner, S\. Crépey, and M\. Dixon \(2020\)Deep Local Volatility\.Risks8\(3\),pp\. 82\.External Links:ISSN 2227\-9091,[Document](https://dx.doi.org/10.3390/risks8030082)Cited by:[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- N\. Cho, J\. Ryu, and H\. J\. Hwang \(2025\)Sobolev training for operator learning\.Journal of Computational Physics543,pp\. 114408\.External Links:ISSN 0021\-9991,[Document](https://dx.doi.org/10.1016/j.jcp.2025.114408)Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px4.p1.3)\.
- W\. M\. Czarnecki, S\. Osindero, M\. Jaderberg, G\. Swirszcz, and R\. Pascanu \(2017\)Sobolev training for neural networks\.InAdvances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4\-9, 2017, Long Beach, CA, USA,I\. Guyon, U\. von Luxburg, S\. Bengio, H\. M\. Wallach, R\. Fergus, S\. V\. N\. Vishwanathan, and R\. Garnett \(Eds\.\),pp\. 4278–4287\.Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px4.p1.3)\.
- H\. Dai, B\. Landry, M\. Pavone, and R\. Tedrake \(2020\)Counter\-example guided synthesis of neural network Lyapunov functions for piecewise linear systems\.In59th IEEE Conference on Decision and Control, CDC 2020, Jeju Island, South Korea, December 14\-18, 2020,pp\. 1274–1281\.External Links:[Document](https://dx.doi.org/10.1109/CDC42340.2020.9304201)Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p3.1)\.
- H\. Dai, B\. Landry, L\. Yang, M\. Pavone, and R\. Tedrake \(2021\)Lyapunov\-stable neural\-network control\.InRobotics: Science and Systems XVII, Virtual Event, July 12\-16, 2021,D\. A\. Shell, M\. Toussaint, and M\. A\. Hsieh \(Eds\.\),External Links:[Document](https://dx.doi.org/10.15607/RSS.2021.XVII.063)Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p3.1)\.
- C\. Dawson, S\. Gao, and C\. Fan \(2023\)Safe control with learned certificates: a survey of neural lyapunov, barrier, and contraction methods for robotics and control\.IEEE Transactions on Robotics39\(3\),pp\. 1749–1767\.Note:This one has hinge losses for positive and negative semi\-definitenessExternal Links:[Document](https://dx.doi.org/10.1109/TRO.2022.3232542)Cited by:[§4\.2](https://arxiv.org/html/2606.13803#S4.SS2.p1.7)\.
- L\. de Moura and N\. Bjørner \(2008\)Z3: An efficient SMT solver\.InTools and Algorithms for the Construction and Analysis of Systems,C\. R\. Ramakrishnan and J\. Rehof \(Eds\.\),Berlin, Heidelberg,pp\. 337–340\.External Links:ISBN 978\-3\-540\-78800\-3Cited by:[§4\.2](https://arxiv.org/html/2606.13803#S4.SS2.p2.4)\.
- F\. Deschâtres \(2024\)Convex Volatility Interpolation\.SSRN Scholarly Paper,Social Science Research Network,Rochester, NY\.External Links:4831218,[Document](https://dx.doi.org/10.2139/ssrn.4831218)Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p3.1)\.
- A\. Edwards, A\. Peruffo, and A\. Abate \(2024\)Fossil 2\.0: Formal certificate synthesis for the verification and control of dynamical models\.InProceedings of the 27th ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2024, Hong Kong SAR, China, May 14\-16, 2024,E\. Ábrahám and M\. M\. Jr\. \(Eds\.\),pp\. 26:1–26:10\.External Links:[Document](https://dx.doi.org/10.1145/3641513.3651398)Cited by:[§F\.1](https://arxiv.org/html/2606.13803#A6.SS1.p1.5),[§1](https://arxiv.org/html/2606.13803#S1.p3.1),[§4\.2](https://arxiv.org/html/2606.13803#S4.SS2.p1.7),[§4\.2](https://arxiv.org/html/2606.13803#S4.SS2.p2.4)\.
- J\. Gallego\-Posada, J\. Ramirez, M\. Hashemizadeh, and S\. Lacoste\-Julien \(2025\)Cooper: A Library for Constrained Optimization in Deep Learning\.arXiv\.External Links:2504\.01212,[Document](https://dx.doi.org/10.48550/arXiv.2504.01212)Cited by:[§2\.3](https://arxiv.org/html/2606.13803#S2.SS3.p1.3)\.
- J\. Gatheral and A\. Jacquier \(2014\)Arbitrage\-free SVI volatility surfaces\.Quantitative Finance14\(1\),pp\. 59–71\.External Links:ISSN 1469\-7688,[Document](https://dx.doi.org/10.1080/14697688.2013.819986)Cited by:[§G\.1](https://arxiv.org/html/2606.13803#A7.SS1.p1.6),[§1](https://arxiv.org/html/2606.13803#S1.p3.1),[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- R\. Golder, B\. N\. Roy, and M\. M\. F\. Hasan \(2025\)DAE\-HardNet: A Physics Constrained Neural Network Enforcing Differential\-Algebraic Hard Constraints\.arXiv\.External Links:2512\.05881,[Document](https://dx.doi.org/10.48550/arXiv.2512.05881)Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px3.p1.1)\.
- M\. Harvey \(2017\)Imperial College Research Computing Service\.Imperial College London\.External Links:[Document](https://dx.doi.org/10.14469/HPC/2232)Cited by:[Acknowledgments and Disclosure of Funding](https://arxiv.org/html/2606.13803#Sx1.p1.1)\.
- P\. Hoedt and G\. Klambauer \(2023\)Principled weight initialisation for input\-convex neural networks\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Cited by:[§4\.1\.2](https://arxiv.org/html/2606.13803#S4.SS1.SSS2.Px1.p1.12)\.
- I\. Hounie, A\. Ribeiro, and L\. F\. O\. Chamon \(2023\)Resilient constrained learning\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px1.p1.1)\.
- A\. Iftakher, R\. Golder, B\. N\. Roy, and M\.M\. Faruque Hasan \(2026\)Physics\-informed neural networks with hard nonlinear equality and inequality constraints\.Computers & Chemical Engineering204,pp\. 109418\.External Links:ISSN 0098\-1354,[Document](https://dx.doi.org/10.1016/j.compchemeng.2025.109418)Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px3.p1.1)\.
- P\. Jäckel \(2015\)Let’s Be Rational: Let’s Be Rational\.Wilmott2015\(75\),pp\. 40–53\.External Links:ISSN 15406962,[Document](https://dx.doi.org/10.1002/wilm.10395)Cited by:[§G\.2](https://arxiv.org/html/2606.13803#A7.SS2.SSS0.Px2.p1.2)\.
- J\. Kotary, F\. Fioretto, P\. V\. Hentenryck, and B\. Wilder \(2021\)End\-to\-end constrained optimization learning: A survey\.InProceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19\-27 August 2021,Z\. Zhou \(Ed\.\),pp\. 4475–4482\.External Links:[Document](https://dx.doi.org/10.24963/IJCAI.2021/610)Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px2.p1.1)\.
- V\. Lucic \(2021\)Normalizing Volatility Transforms and Parameterization of Volatility Smile\.SSRN Scholarly Paper,Social Science Research Network,Rochester, NY\.External Links:3835233,[Document](https://dx.doi.org/10.2139/ssrn.3835233)Cited by:[§G\.1](https://arxiv.org/html/2606.13803#A7.SS1.p1.6),[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- Y\. Min and N\. Azizan \(2025\)HardNet: Hard\-Constrained Neural Networks with Universal Approximation Guarantees\.arXiv\.External Links:2410\.10807,[Document](https://dx.doi.org/10.48550/arXiv.2410.10807)Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px3.p1.1)\.
- H\. Narasimhan, A\. Cotter, Y\. Zhou, S\. L\. Wang, and W\. Guo \(2020\)Approximate heavily\-constrained learning with lagrange multiplier models\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, Virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),Cited by:[§2\.3](https://arxiv.org/html/2606.13803#S2.SS3.p2.5)\.
- J\. Nocedal and S\. J\. Wright \(2006\)Numerical optimization\.2\. ed edition,Springer Series in Operations Research and Financial Engineering,Springer,New York\.External Links:ISBN 978\-0\-387\-30303\-1Cited by:[§2\.2](https://arxiv.org/html/2606.13803#S2.SS2.p1.16)\.
- S\. Park and P\. V\. Hentenryck \(2023\)Self\-supervised primal\-dual learning for constrained optimization\.InThirty\-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty\-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7\-14, 2023,B\. Williams, Y\. Chen, and J\. Neville \(Eds\.\),pp\. 4052–4060\.External Links:[Document](https://dx.doi.org/10.1609/AAAI.V37I4.25520)Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px2.p1.1)\.
- E\. Perez, F\. Strub, H\. De Vries, V\. Dumoulin, and A\. Courville \(2018\)FiLM: Visual Reasoning with a General Conditioning Layer\.Proceedings of the AAAI Conference on Artificial Intelligence32\(1\)\.External Links:ISSN 2374\-3468, 2159\-5399,[Document](https://dx.doi.org/10.1609/aaai.v32i1.11671)Cited by:[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- J\. C\. Platt and A\. H\. Barr \(1987\)Constrained differential optimization\.InNeural Information Processing Systems, Denver, Colorado, USA, 1987,D\. Z\. Anderson \(Ed\.\),pp\. 612–621\.Cited by:[§2\.3](https://arxiv.org/html/2606.13803#S2.SS3.p1.3)\.
- N\. Rahaman, A\. Baratin, D\. Arpit, F\. Draxler, M\. Lin, F\. A\. Hamprecht, Y\. Bengio, and A\. C\. Courville \(2019\)On the spectral bias of neural networks\.InProceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\-15 June 2019, Long Beach, California, USA,K\. Chaudhuri and R\. Salakhutdinov \(Eds\.\),Proceedings of Machine Learning Research,pp\. 5301–5310\.Cited by:[§4](https://arxiv.org/html/2606.13803#S4.p1.3)\.
- S\. Ramasinghe and S\. Lucey \(2022\)Beyond periodicity: Towards a unifying framework for activations in coordinate\-mlps\.InComputer Vision \- ECCV 2022 \- 17th European Conference, Tel Aviv, Israel, October 23\-27, 2022, Proceedings, Part XXXIII,S\. Avidan, G\. J\. Brostow, M\. Cissé, G\. M\. Farinella, and T\. Hassner \(Eds\.\),Lecture Notes in Computer Science,pp\. 142–158\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-19827-4%5F9)Cited by:[§4](https://arxiv.org/html/2606.13803#S4.p1.3)\.
- J\. Ramirez, M\. Hashemizadeh, and S\. Lacoste\-Julien \(2025\)Position: Adopt constraints over penalties in deep learning\.CoRRabs/2505\.20628\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2505.20628)Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p3.1),[§2\.2](https://arxiv.org/html/2606.13803#S2.SS2.p1.16),[§2\.3](https://arxiv.org/html/2606.13803#S2.SS3.p1.3)\.
- D\. Runje and S\. M\. Shankaranarayana \(2023\)Constrained monotonic neural networks\.InInternational Conference on Machine Learning, ICML 2023, 23\-29 July 2023, Honolulu, Hawaii, USA,A\. Krause, E\. Brunskill, K\. Cho, B\. Engelhardt, S\. Sabato, and J\. Scarlett \(Eds\.\),Proceedings of Machine Learning Research, Vol\.202,pp\. 29338–29353\.Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p2.1),[§2\.4](https://arxiv.org/html/2606.13803#S2.SS4.p1.1)\.
- D\. Sartor, A\. Sinigaglia, and G\. A\. Susto \(2025\)Advancing constrained monotonic neural networks: Achieving universal approximation beyond bounded activations\.InForty\-Second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,Cited by:[§D\.2](https://arxiv.org/html/2606.13803#A4.SS2.p1.2),[§1](https://arxiv.org/html/2606.13803#S1.p2.1),[§2\.4](https://arxiv.org/html/2606.13803#S2.SS4.p1.1)\.
- V\. Sitzmann, J\. N\. P\. Martel, A\. W\. Bergman, D\. B\. Lindell, and G\. Wetzstein \(2020\)Implicit neural representations with periodic activation functions\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, Virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p5.4),[§4](https://arxiv.org/html/2606.13803#S4.p1.3)\.
- S\. Sivaprasad, A\. Singh, N\. Manwani, and V\. Gandhi \(2021\)The curious case of convex neural networks\.InMachine Learning and Knowledge Discovery in Databases\. Research Track,N\. Oliver, F\. Pérez\-Cruz, S\. Kramer, J\. Read, and J\. A\. Lozano \(Eds\.\),Cham,pp\. 738–754\.External Links:ISBN 978\-3\-030\-86486\-6Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p2.1),[§4\.1\.2](https://arxiv.org/html/2606.13803#S4.SS1.SSS2.Px1.p1.12)\.
- M\. Sohrabi, J\. Ramirez, T\. H\. Zhang, S\. Lacoste\-Julien, and J\. Gallego\-Posada \(2024\)On PI controllers for updating lagrange multipliers in constrained optimization\.InForty\-First International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21\-27, 2024,R\. Salakhutdinov, Z\. Kolter, K\. A\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research,pp\. 45922–45954\.Cited by:[§2\.3](https://arxiv.org/html/2606.13803#S2.SS3.p1.3)\.
- M\. C\. Stoian, S\. Dyrmishi, M\. Cordy, T\. Lukasiewicz, and E\. Giunchiglia \(2024\)How realistic is your synthetic data? Constraining deep generative models for tabular data\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,Cited by:[Appendix B](https://arxiv.org/html/2606.13803#A2.SS0.SSS0.Px3.p2.1)\.
- A\. Stooke, J\. Achiam, and P\. Abbeel \(2020\)Responsive safety in reinforcement learning by PID lagrangian methods\.InProceedings of the 37th International Conference on Machine Learning, ICML 2020, 13\-18 July 2020, Virtual Event,Proceedings of Machine Learning Research,pp\. 9133–9143\.Cited by:[§2\.3](https://arxiv.org/html/2606.13803#S2.SS3.p1.3)\.
- M\. Tancik, P\. P\. Srinivasan, B\. Mildenhall, S\. Fridovich\-Keil, N\. Raghavan, U\. Singhal, R\. Ramamoorthi, J\. T\. Barron, and R\. Ng \(2020\)Fourier features let networks learn high frequency functions in low dimensional domains\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, Virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),Cited by:[§1](https://arxiv.org/html/2606.13803#S1.p5.4)\.
- R\. Wiedemann, A\. Jacquier, and L\. Gonon \(2025\)Operator deep smoothing for implied volatility\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,Cited by:[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- Y\. Yang, W\. Chen, C\. Shu, and T\. M\. Hospedales \(2025\)HyperIV: Real\-time implied volatility smoothing\.InForty\-Second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,Cited by:[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- A\. Zadeh, Y\. Lim, P\. P\. Liang, and L\. Morency \(2021\)Variational Auto\-Decoder: A Method for Neural Generative Modeling from Incomplete Data\.arXiv\.Note:Comment: Link to code and data available from https://github\.com/A2Zadeh/Variational\-AutodecoderExternal Links:1903\.00840,[Document](https://dx.doi.org/10.48550/arXiv.1903.00840)Cited by:[§G\.3](https://arxiv.org/html/2606.13803#A7.SS3.p1.7),[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.
- Y\. Zheng, Y\. Yang, and B\. Chen \(2021\)Incorporating prior financial domain knowledge into neural networks for implied volatility surface prediction\.InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining,pp\. 3968–3975\.Cited by:[§4\.3](https://arxiv.org/html/2606.13803#S4.SS3.p1.4)\.

## Appendix ANeural slack variables: training and implementation details

Algorithm 1Neural slack variable training1:Input:dataset

𝒟\\mathcal\{D\}; primary network

fθ:Ω→ℝf^\{\\theta\}\\colon\\Omega\\to\\mathbb\{R\}; neural slack variable

sϕ:Ω→Ks^\{\\phi\}\\colon\\Omega\\to K; constraint operator

𝒞\\mathcal\{C\}; data batch size

BdB\_\{\\text\{d\}\}, grid batch size

BgB\_\{\\text\{g\}\}; maximum effective weight

ρmax\\rho\_\{\\text\{max\}\}; iterations

TT\.

2:Initialize:

θ\(0\),ϕ\(0\)\\theta^\{\(0\)\},\\phi^\{\(0\)\}; set

δ←1/ρmax\\delta\\leftarrow 1/\\sqrt\{\\rho\_\{\\text\{max\}\}\}\.

3:for

t=1,…,Tt=1,\\dots,Tdo

4:Sample

\{\(xn,yn\)\}n=1Bd⊂𝒟\\\{\(x\_\{n\},y\_\{n\}\)\\\}\_\{n=1\}^\{B\_\{\\text\{d\}\}\}\\subset\\mathcal\{D\}and

\{ξn\}n=1Bg∼𝒰​\(Ω\)\\\{\\xi\_\{n\}\\\}\_\{n=1\}^\{B\_\{\\text\{g\}\}\}\\sim\\mathcal\{U\}\(\\Omega\)\.

5:Evaluate

cn←𝒞​\[fθ\]​\(ξn\)c\_\{n\}\\leftarrow\\mathcal\{C\}\[f^\{\\theta\}\]\(\\xi\_\{n\}\)and

sn←sϕ​\(ξn\)s\_\{n\}\\leftarrow s^\{\\phi\}\(\\xi\_\{n\}\)\.

6:

dn←max⁡\(δ,sg​\[sn\]\)d\_\{n\}\\leftarrow\\max\\bigl\(\\delta,\\ \\mathrm\{sg\}\[s\_\{n\}\]\\bigr\)\.

7:

ℒdata←1Bd​∑n=1Bdℓ​\(fθ​\(xn\),yn\)\\mathcal\{L\}\_\{\\text\{data\}\}\\leftarrow\\tfrac\{1\}\{B\_\{\\text\{d\}\}\}\\sum\_\{n=1\}^\{B\_\{\\text\{d\}\}\}\\ell\\bigl\(f^\{\\theta\}\(x\_\{n\}\),y\_\{n\}\\bigr\)\.

8:

ℒslack←12​Bg​∑n=1Bg‖asinh⁡\(cn/dn\)−asinh⁡\(sn/dn\)‖2\\mathcal\{L\}\_\{\\text\{slack\}\}\\leftarrow\\tfrac\{1\}\{2B\_\{\\text\{g\}\}\}\\sum\_\{n=1\}^\{B\_\{\\text\{g\}\}\}\\bigl\\\|\\operatorname\{asinh\}\(c\_\{n\}/d\_\{n\}\)\-\\operatorname\{asinh\}\(s\_\{n\}/d\_\{n\}\)\\bigr\\\|^\{2\}\.

9:Optimizer step on

ℒdata\+ℒslack\\mathcal\{L\}\_\{\\text\{data\}\}\+\\mathcal\{L\}\_\{\\text\{slack\}\}jointly in

\(θ,ϕ\)\(\\theta,\\phi\)\.

10:endfor

##### Scale\-balanced slack matching

A plain squared matching loss betweencθ=𝒞​\[fθ\]c\_\{\\theta\}=\\mathcal\{C\}\[f^\{\\theta\}\]andsϕs^\{\\phi\}is dominated by satisfied margins: inequality constraints typically have wide dynamic range, and large values ofcθc\_\{\\theta\}contribute uninformative gradient\. We instead matchcθc\_\{\\theta\}andsϕs^\{\\phi\}on asϕs^\{\\phi\}\-relative scale\. The choice has a self\-consistent justification\. Letc^\\hat\{c\}denote the constraint profile of the feasible solution selected by the data and the inductive bias of𝒮\\mathcal\{S\}\. Late in training one expectssϕ≈c^s^\{\\phi\}\\approx\\hat\{c\}, sosϕs^\{\\phi\}is itself an estimator of the natural denominatorc^\\hat\{c\}\. Wherec^\\hat\{c\}is small, the relative metric weights residuals heavily, exactly the regime where drift can occur; wherec^\\hat\{c\}is large, deviations are penalized in proportion, so satisfied margins do not dominate\. The architectural lower boundsϕ≥εs^\{\\phi\}\\geq\\varepsilonalso keeps the denominator strictly positive\.

##### Adaptive interpretation

The relative scaling caps the effective penalty weight at a chosenρmax\\rho\_\{\\text\{max\}\}\. Withδ:=1/ρmax\\delta:=1/\\sqrt\{\\rho\_\{\\text\{max\}\}\}and

dϕ​\(x\):=max⁡\(δ,sg​\[sϕ​\(x\)\]\),d\_\{\\phi\}\(x\):=\\max\\bigl\(\\delta,\\ \\mathrm\{sg\}\[s^\{\\phi\}\(x\)\]\\bigr\),\(10\)wheresg​\[⋅\]\\mathrm\{sg\}\[\\,\\cdot\\,\]is a stop\-gradient onsϕs^\{\\phi\}, the relativized matching loss

ℒslackρmax​\(θ,ϕ\):=12​𝔼x∼𝒰​\(Ω\)​\[‖cθ​\(x\)−sϕ​\(x\)dϕ​\(x\)‖2\]\\mathcal\{L\}\_\{\\text\{slack\}\}^\{\\rho\_\{\\text\{max\}\}\}\(\\theta,\\phi\)\\;:=\\;\\frac\{1\}\{2\}\\,\\mathbb\{E\}\_\{x\\sim\\mathcal\{U\}\(\\Omega\)\}\\\!\\left\[\\biggl\\\|\\frac\{c\_\{\\theta\}\(x\)\-s^\{\\phi\}\(x\)\}\{d\_\{\\phi\}\(x\)\}\\biggr\\\|^\{2\}\\right\]\(11\)satisfiesℒslackρmax=ρmax​ℒslack\\mathcal\{L\}\_\{\\text\{slack\}\}^\{\\rho\_\{\\text\{max\}\}\}=\\rho\_\{\\text\{max\}\}\\,\\mathcal\{L\}\_\{\\text\{slack\}\}in the binding regimesϕ≤δs^\{\\phi\}\\leq\\delta, and is scale\-invariant forsϕ\>δs^\{\\phi\}\>\\delta\. There is therefore no separate multiplier onℒslack\\mathcal\{L\}\_\{\\text\{slack\}\}in the practical algorithm;ρmax\\rho\_\{\\text\{max\}\}sets the maximum effective quadratic weight near the constraint boundary\.

##### Asinh stabilization

For additional training stability, we apply anasinh\\operatorname\{asinh\}transform to the rescaled arguments before differencing, replacing the squared residual in \([11](https://arxiv.org/html/2606.13803#A1.E11)\) by

‖asinh⁡\(cθ​\(x\)dϕ​\(x\)\)−asinh⁡\(sϕ​\(x\)dϕ​\(x\)\)‖2\.\\biggl\\\|\\operatorname\{asinh\}\\\!\\biggl\(\\frac\{c\_\{\\theta\}\(x\)\}\{d\_\{\\phi\}\(x\)\}\\biggr\)\-\\operatorname\{asinh\}\\\!\\biggl\(\\frac\{s^\{\\phi\}\(x\)\}\{d\_\{\\phi\}\(x\)\}\\biggr\)\\biggr\\\|^\{2\}\.\(12\)asinh\\operatorname\{asinh\}is locally the identity at the origin, so the binding\-regime equivalence is preserved to leading order\. We found the transform beneficial in our experiments\.

[Algorithm˜1](https://arxiv.org/html/2606.13803#alg1)summarizes the joint training procedure for the primary networkfθf^\{\\theta\}and the neural slack variablesϕs^\{\\phi\}\.

## Appendix BRelated work

##### Theory

The PAC\-learning framework gives distributional guarantees for constrained learning through Lagrangian dualityChamon and Ribeiro \[[2020](https://arxiv.org/html/2606.13803#bib.bib7)\], Chamonet al\.\[[2023](https://arxiv.org/html/2606.13803#bib.bib6)\]\. In this setting,Hounieet al\.\[[2023](https://arxiv.org/html/2606.13803#bib.bib26)\]study adaptive constraint relaxation, with slack adjusted during training according to a user\-specified relaxation cost\. Neural slack variables also relax constraints, but through a learned slack network rather than per\-sample slack updates\.

##### Constrained optimization learning

A related line of work uses neural networks to amortize finite\-dimensional constrained optimization, where the network learns a surrogate solver from problem instances to optimal decisionsKotaryet al\.\[[2021](https://arxiv.org/html/2606.13803#bib.bib29)\]\. Our setting instead trains one function under functional inequality constraints, often involving derivatives, over a continuous domain\.Park and Hentenryck \[[2023](https://arxiv.org/html/2606.13803#bib.bib35)\]also learn primal and dual networks for instance\-amortized constrained optimization\. The closest connection to our method is their target\-based dual training, where the dual network is trained toward a projected update rather than only through a violation penalty\. Neural slack variables make a related move on the primal side: they learn an auxiliary target and match the constraint operator to it, giving gradient signal beyond the currently violated set\.

##### Projection\-based hard constraint methods

Several recent works enforce hard constraints through differentiable projection or solver layersMin and Azizan \[[2025](https://arxiv.org/html/2606.13803#bib.bib31)\], Iftakheret al\.\[[2026](https://arxiv.org/html/2606.13803#bib.bib27)\], Golderet al\.\[[2025](https://arxiv.org/html/2606.13803#bib.bib21)\]\. These methods resemble neural slack variables because both form explicit feasible targets, but they address different settings\. Projection\-based methods typically use fixed, non\-learnable operators and often require an optimization or solve inside the forward pass\. Neural slack variables instead target functional inequality constraints with a learnable slack network, where the joint learning of the matching objective replaces the explicit projection\.

Related neuro\-symbolic work also uses symbolic constraint descriptions to enforce linear inequalities in generative models for tabular dataStoianet al\.\[[2024](https://arxiv.org/html/2606.13803#bib.bib48)\]\. Such approaches suggest a route toward more general hard\-constraint mechanisms, which remain limited for many applied constrained learning problems\.

##### Sobolev training

When the constraint operator𝒞​\[⋅\]\\mathcal\{C\}\[\\cdot\]involves derivatives \(monotonicity via first derivatives, convexity via Hessian spectra\), neural slack variables are related to Sobolev trainingCzarneckiet al\.\[[2017](https://arxiv.org/html/2606.13803#bib.bib10)\], Choet al\.\[[2025](https://arxiv.org/html/2606.13803#bib.bib9)\]\. Both introduce derivative\-based matching terms, but neural slack variables learn the target profilesϕs^\{\\phi\}jointly withfθf^\{\\theta\}rather than taking it from derivative labels\.

## Appendix CExperimental evaluation metrics

We report approximation quality via mean absolute error

δmae:=𝔼\(x,y\)∼𝒟​\[\|fθ​\(x\)−y\|\],\\delta\_\{\\text\{mae\}\}:=\\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\}\\\!\\left\[\\left\|f^\{\\theta\}\(x\)\-y\\right\|\\right\],computed on the full training dataset𝒟\\mathcal\{D\}\.

Constraint satisfaction is measured on a fine evaluation gridπeval\\pi\_\{\\text\{eval\}\}coveringΩ\\Omega\. For a constraint operator𝒞​\[fθ\]:Ω→ℝm\\mathcal\{C\}\[f^\{\\theta\}\]\\colon\\Omega\\to\\mathbb\{R\}^\{m\}with components𝒞j​\[fθ\]\\mathcal\{C\}\_\{j\}\[f^\{\\theta\}\],j=1,…,mj=1,\\dots,m, we report

ηrate\\displaystyle\\eta\_\{\\text\{rate\}\}:=𝔼x∼πeval​\[𝟏\{𝒞​\[fθ\]​\(x\)≱0\}\],\\displaystyle:=\\mathbb\{E\}\_\{x\\sim\\pi\_\{\\text\{eval\}\}\}\\\!\\left\[\\mathbf\{1\}\_\{\\left\\\{\\mathcal\{C\}\[f^\{\\theta\}\]\(x\)\\not\\geq 0\\right\\\}\}\\right\],ηmean\\displaystyle\\eta\_\{\\text\{mean\}\}:=𝔼x∼πeval​\[1m​∑j=1mmax⁡\{−𝒞j​\[fθ\]​\(x\),0\}\],\\displaystyle:=\\mathbb\{E\}\_\{x\\sim\\pi\_\{\\text\{eval\}\}\}\\\!\\left\[\\frac\{1\}\{m\}\\sum\_\{j=1\}^\{m\}\\max\\\!\\left\\\{\-\\mathcal\{C\}\_\{j\}\[f^\{\\theta\}\]\(x\),\\,0\\right\\\}\\right\],ηmax\\displaystyle\\eta\_\{\\text\{max\}\}:=maxx∈πeval⁡maxj=1,…,m⁡max⁡\{−𝒞j​\[fθ\]​\(x\),0\},\\displaystyle:=\\max\_\{x\\in\\pi\_\{\\text\{eval\}\}\}\\,\\max\_\{j=1,\\dots,m\}\\,\\max\\\!\\left\\\{\-\\mathcal\{C\}\_\{j\}\[f^\{\\theta\}\]\(x\),\\,0\\right\\\},where𝟏\{⋅\}\\mathbf\{1\}\_\{\\\{\\cdot\\\}\}is the indicator function\.ηrate\\eta\_\{\\text\{rate\}\}is the fraction of evaluation points where \(any component of\) the inequality constraint is violated;ηmean\\eta\_\{\\text\{mean\}\}averages the elementwise shortfall over both the evaluation grid and the constraint components, so non\-violating components contribute zero;ηmax\\eta\_\{\\text\{max\}\}is the worst\-case shortfall\. For seed\-aggregated tables we additionally reportnsat/Nn\_\{\\text\{sat\}\}/N, the count of seeds withηrate=0\\eta\_\{\\text\{rate\}\}=0exactly\.

## Appendix DAdditional details: Monotonicity \(d=1d=1\)

### D\.1Dataset and target function

The target is the squared piecewise linear profile

f​\(x\)=b​\(x\)2,b​\(x\)=\{\(2−s/3\)\+3​\(x\+13\),−1≤x<−13,2\+s​x,−13≤x<13,\(2\+s/3\)\+\(x−13\),13≤x≤1,f\(x\)=b\(x\)^\{2\},\\qquad b\(x\)=\\begin\{cases\}\(2\-s/3\)\+3\\,\(x\+\\tfrac\{1\}\{3\}\),&\-1\\leq x<\-\\tfrac\{1\}\{3\},\\\\\[2\.0pt\] 2\+s\\,x,&\-\\tfrac\{1\}\{3\}\\leq x<\\tfrac\{1\}\{3\},\\\\\[2\.0pt\] \(2\+s/3\)\+\(x\-\\tfrac\{1\}\{3\}\),&\\tfrac\{1\}\{3\}\\leq x\\leq 1,\\end\{cases\}onΩ=\[−1,1\]\\Omega=\[\-1,1\], where the inner\-slope parametersscontrols the central segment\. The two outer segments retain slopes33\(left\) and11\(right\); their vertical anchors are chosen sobbis continuous atx=±13x=\\pm\\tfrac\{1\}\{3\}for anyss\. Withs=0s=0the central segment is a flat plateau atf≡4f\\equiv 4\. We uses=−0\.1s=\-0\.1, tilting the plateau into a slight downward terrace; this violates∂xf≥0\\partial\_\{x\}f\\geq 0in the ground truth, so the constraint actively opposes the data fit\.

The training dataset consists ofN=100N=100noisy observations on a uniform grid overΩ\\Omega, with additive zero\-mean Gaussian noise \(standard deviation0\.10\.1\)\. The constraint is enforced on a separate uniform training gridπtrain\\pi\_\{\\text\{train\}\}of200200points covering\[−1\.05,1\.05\]\[\-1\.05,1\.05\]\(a small0\.050\.05pad past the data domain\) and verified on an evaluation gridπeval\\pi\_\{\\text\{eval\}\}of10,00010\{,\}000uniformly spaced points overΩ\\Omega\. Initialization seeds and noise seeds are paired across runs; exact reproduction is possible from the supplemental code\.

### D\.2Architectural baseline \(CMNN\)

As an architectural reference we use \(advanced\) Constrained Monotonic Neural Networks \(CMNNs\)Sartoret al\.\[[2025](https://arxiv.org/html/2606.13803#bib.bib44)\]\. Following the reference implementation,111[https://github\.com/AMCO\-UniPD/monotonic](https://github.com/AMCO-UniPD/monotonic)the first hidden\-layer activation is the identity and subsequent layers use a monotone nonlinearity; we useSoftplus\\operatorname\{Softplus\}\. We match the primary network’s capacity: four hidden layers of width1616, with bias terms enabled\.

### D\.3SIRENfθf^\{\\theta\}results

Counterpart to[Table˜1](https://arxiv.org/html/2606.13803#S4.T1)withfθf^\{\\theta\}andsϕs^\{\\phi\}both three\-layer SIRENs of width1616andω0=15\\omega\_\{0\}=15; other settings match[Section˜4\.1\.1](https://arxiv.org/html/2606.13803#S4.SS1.SSS1)\.

Table 3:Monotone \(SIRENfθf^\{\\theta\}\): mean±\\pmstd over1010seeds\. The neural slack variable achieves a lowerδmae\\delta\_\{\\text\{mae\}\}than CMNN at full feasibility \(10/10 seeds\): using a SIREN forfθf^\{\\theta\}extracts more variance from the data than CMNN can, with the neural slack variable still enforcing feasibility\. Counterpart to[Table˜1](https://arxiv.org/html/2606.13803#S4.T1); the penalty method still leaves small residual violations, while the Lagrangian methods fail, with primal\-dual updates struggling to handle the SIREN’s constraint profile \(the augmented Lagrangian method was feasible on 73/100 seeds with an MLP\-based primary network\)\. Time measured on Apple M3 Pro \(CPU\)\.The qualitative picture from the MLPfθf^\{\\theta\}table carries over: aside from CMNN, the neural slack variable is the only run to retain feasibility at a competitive fit; the penalty method leaves a positive violation rate; the dense Lagrangian fails to fit the data; the augmented Lagrangian is bimodal across seeds, with high variance in both fit and feasibility\.

![Refer to caption](https://arxiv.org/html/2606.13803v1/x6.png)Figure 5:Training dynamics over ten thousand epochs for the monotonicity experiment with a SIREN primary model, with zoom insets \(last500500epochs\) on the right\. For the representative seed shown, the sampled violation loss becomes small while dense\-grid violations persist, whereas the neural slack variable maintains zero dense\-grid violations while its matching loss remains active\.
### D\.4Neural slack variable capacity sweep

Complementing[Section˜4\.1\.1](https://arxiv.org/html/2606.13803#S4.SS1.SSS1.Px2),[Figure˜6](https://arxiv.org/html/2606.13803#A4.F6)shows the finalfθf^\{\\theta\}\(left column\) and∂xfθ\\partial\_\{x\}f^\{\\theta\}\(right column\) for an MLP slack variable at every width in the sweepW∈\{1,4,16,64,256\}W\\in\\\{1,4,16,64,256\\\}, on the monotone MLP setup with seed4242\. The two columns track the U\-shape visible in[Figure˜3\(a\)](https://arxiv.org/html/2606.13803#S4.F3.sf1)\. At small widths \(W=1W=1\) the slack variable has so little capacity that it can only represent a near\-constant slack: the constraint pressure pulls∂xfθ\\partial\_\{x\}f^\{\\theta\}down to a smooth proxy and the plateau is rounded off into a near\-linear ramp, costing data fit\. At the optimum \(W=4W=4,1616\) the slack tracks the constraint slack accurately andfθf^\{\\theta\}resolves both the outer slopes and the central plateau\. At large widths \(W=64W=64,256256\) the optimization collapses on this seed:fθf^\{\\theta\}degenerates toward a constant predictor and∂xfθ\\partial\_\{x\}f^\{\\theta\}vanishes almost everywhere onΩ\\Omega, so the constraint is trivially satisfied at the cost of any data fit\. The collapse is clearly visible in the per\-row plots and is the source of the risingδmae\\delta\_\{\\text\{mae\}\}tail in[Figure˜3\(a\)](https://arxiv.org/html/2606.13803#S4.F3.sf1)\.

![Refer to caption](https://arxiv.org/html/2606.13803v1/x7.png)\(a\)W=1W=1\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x8.png)\(b\)W=4W=4\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x9.png)\(c\)W=16W=16\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x10.png)\(d\)W=64W=64\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x11.png)\(e\)W=256W=256\.

Figure 6:Monotone MLP\+\+neural slack variable, hidden width sweep\. Each row pairs the final function fit \(left half\) with the final derivative profile \(right half\) at hidden widthW∈\{1,4,16,64,256\}W\\in\\\{1,4,16,64,256\\\}\(seed4242\)\. Shaded grey bands on the full panels mark the zoom region\[−0\.1,0\.1\]\[\-0\.1,0\.1\]rendered alongside\.The SIREN slack variable exhibits no analogous U\-shape across the sweep:[Figure˜7](https://arxiv.org/html/2606.13803#A4.F7)renders the per\-width fits and derivatives for a SIREN slack on the same MLPfθf^\{\\theta\}setup, seed4242\. The slack tracks the constraint profile at everyWW,fθf^\{\\theta\}retains both the outer slopes and the central plateau, and the collapse modes seen at largeWWfor the MLP slack do not appear\.

![Refer to caption](https://arxiv.org/html/2606.13803v1/x12.png)\(a\)W=1W=1\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x13.png)\(b\)W=4W=4\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x14.png)\(c\)W=16W=16\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x15.png)\(d\)W=64W=64\.
![Refer to caption](https://arxiv.org/html/2606.13803v1/x16.png)\(e\)W=256W=256\.

Figure 7:Monotone MLP\+\+SIREN slack variable, hidden width sweep\. Counterpart to[Figure˜6](https://arxiv.org/html/2606.13803#A4.F6)withsϕs^\{\\phi\}a SIREN \(ω0=5\\omega\_\{0\}=5\) instead of a softplus MLP\. Shaded grey bands on the full panels mark the zoom region\[−0\.5,0\.5\]\[\-0\.5,0\.5\]rendered alongside\.

## Appendix EAdditional details: Convexity \(d∈\{2,3,4,5\}d\\in\\\{2,3,4,5\\\}\)

### E\.1Dataset and target function

The target is the log\-sum\-exp ofK=10K=10affine functions plus a small quadratic anchor:

f​\(x\)=τ​log⁡\(∑k=1Kexp⁡\(ak⊤​x\+bkτ\)\)\+μ2​‖x‖22,x∈Ω=\[−1,1\]d\.f\(x\)=\\tau\\log\\\!\\left\(\\sum\_\{k=1\}^\{K\}\\exp\\\!\\left\(\\frac\{a\_\{k\}^\{\\top\}x\+b\_\{k\}\}\{\\tau\}\\right\)\\right\)\+\\frac\{\\mu\}\{2\}\\\|x\\\|^\{2\}\_\{2\},\\qquad x\\in\\Omega=\[\-1,1\]^\{d\}\.The affine coefficientsak∈ℝda\_\{k\}\\in\\mathbb\{R\}^\{d\}and offsetsbk∈ℝb\_\{k\}\\in\\mathbb\{R\}are drawn i\.i\.d\. from𝒩​\(0,Id\)\\mathcal\{N\}\(0,I\_\{d\}\)and𝒩​\(0,1\)\\mathcal\{N\}\(0,1\)at a fixed target seed; the log\-sum\-exp temperature isτ=0\.2\\tau=0\.2and the quadratic anchor isμ=10−4\\mu=10^\{\-4\}\. Training observations are noisy samples drawn as a scrambled Sobol low\-discrepancy sequence onΩ\\Omega\(additive zero\-mean Gaussian noise, standard deviation0\.050\.05\); constraint enforcement uses a separate scrambled Sobol grid \(with an optional small absolute pad pastΩ\\Omega\); evaluation uses a denser Sobol Monte Carlo sample\. Eachddis run with five seeds; the initialization, noise, training\-grid, and target seeds advance jointly across the five repetitions, so the same seed index gives a reproducible\(ak,bk,init,noise,grid\)\(\{a\_\{k\}\},\{b\_\{k\}\},\\text\{init\},\\text\{noise\},\\text\{grid\}\)tuple\.

### E\.2Architectural baseline \(ICNN\)

As an architectural reference we use Input Convex Neural NetworksAmoset al\.\[[2017](https://arxiv.org/html/2606.13803#bib.bib3)\]with three hidden layers of width128128andSoftplus\\operatorname\{Softplus\}activations; the output is linear\. The ICNN trains under the same10,00010\{,\}000\-epoch Adam schedule as the other comparison runs\.

### E\.3Per\-dimension tables

[Table˜4](https://arxiv.org/html/2606.13803#A5.T4)reports final metrics at eachddas mean±\\pmstd across the five seeds, alongside the unconstrained reference \(“None”\)\. Violation metrics \(ηrate\\eta\_\{\\text\{rate\}\},ηmean\\eta\_\{\\text\{mean\}\},ηmax\\eta\_\{\\text\{max\}\},nsat/Nn\_\{\\text\{sat\}\}/N\) are defined in[Appendix˜C](https://arxiv.org/html/2606.13803#A3); here the constraint components are the Hessian eigenvalues\.

Table 4:Convex\-dd,d∈\{2,3,4,5\}d\\in\\\{2,3,4,5\\\}\(mean±\\pmstd over 5 seeds\)\. Apple M3 Pro \(CPU\); total wall\-clock∼20\\sim\\\!20h, of which the unconstrained baseline and ICNN training are negligible\.δmae\\delta\_\{\\text\{mae\}\}ηrate\\eta\_\{\\text\{rate\}\}ηmean\\eta\_\{\\text\{mean\}\}ηmax\\eta\_\{\\text\{max\}\}nsat/Nn\_\{\\text\{sat\}\}/NDimMethod2None4\.1e\-02 ± 6\.9e\-043\.8e\-01 ± 1\.3e\-013\.4e\-02 ± 4\.0e\-029\.8e\-01 ± 1\.6e\+000/5Penalty4\.1e\-02 ± 6\.4e\-043\.3e\-04 ± 6\.8e\-054\.3e\-08 ± 2\.4e\-082\.2e\-03 ± 1\.7e\-030/5NSV4\.2e\-02 ± 1\.4e\-030005/5ICNN5\.4e\-02 ± 6\.1e\-030005/53None4\.1e\-02 ± 1\.2e\-038\.0e\-01 ± 1\.3e\-015\.0e\-02 ± 3\.5e\-021\.1e\+00 ± 6\.8e\-010/5Penalty4\.3e\-02 ± 1\.7e\-031\.3e\-03 ± 6\.8e\-041\.2e\-06 ± 1\.7e\-062\.1e\-02 ± 2\.5e\-020/5NSV4\.4e\-02 ± 2\.9e\-033\.0e\-07 ± 5\.2e\-071\.4e\-10 ± 2\.9e\-101\.0e\-03 ± 1\.8e\-033/5ICNN6\.5e\-02 ± 1\.4e\-020005/54None4\.1e\-02 ± 1\.1e\-039\.2e\-01 ± 6\.7e\-026\.8e\-02 ± 2\.0e\-022\.3e\+00 ± 3\.4e\-010/5Penalty4\.7e\-02 ± 1\.9e\-032\.9e\-03 ± 1\.7e\-034\.8e\-06 ± 4\.9e\-062\.7e\-01 ± 4\.4e\-010/5NSV4\.8e\-02 ± 2\.8e\-031\.8e\-05 ± 1\.0e\-053\.8e\-08 ± 3\.6e\-085\.8e\-02 ± 4\.7e\-020/5ICNN7\.8e\-02 ± 1\.9e\-020005/55None4\.4e\-02 ± 1\.2e\-039\.6e\-01 ± 1\.4e\-021\.1e\-01 ± 1\.9e\-025\.9e\+00 ± 2\.4e\+000/5Penalty5\.9e\-02 ± 6\.1e\-036\.0e\-03 ± 1\.7e\-031\.3e\-05 ± 8\.0e\-062\.6e\-01 ± 1\.1e\-010/5NSV6\.2e\-02 ± 8\.3e\-039\.9e\-05 ± 8\.3e\-051\.8e\-07 ± 2\.1e\-077\.5e\-02 ± 3\.2e\-020/5ICNN1\.1e\-01 ± 1\.0e\-020005/5

### E\.4Neural slack variable surfaces \(d=2d=2\)

[Figure˜8](https://arxiv.org/html/2606.13803#A5.F8)visualizes the targetff, the fittedfθf^\{\\theta\}, the actual eigenvalues of∇2fθ\\nabla^\{2\}f^\{\\theta\}, and the per\-eigenvalue neural slack variable outputss1ϕ,s2ϕs^\{\\phi\}\_\{1\},s^\{\\phi\}\_\{2\}atd=2d=2for the neural slack variable \(rel\) run\. The slack outputs are non\-negative by construction; the actual eigenvalues track them acrossΩ\\Omega\.

![Refer to caption](https://arxiv.org/html/2606.13803v1/x17.png)Figure 8:Dimensiond=2d=2: targetff, fittedfθf^\{\\theta\}, sorted Hessian eigenvaluesσ1≤σ2\\sigma\_\{1\}\\leq\\sigma\_\{2\}, and neural slack variable outputss1ϕ,s2ϕs^\{\\phi\}\_\{1\},s^\{\\phi\}\_\{2\}\.

## Appendix FAdditional details: neural barrier certificates

### F\.1Domain and constraint geometry

The benchmark is the autonomous polynomial system from FOSSIL’sBarr3suite\[Edwardset al\.,[2024](https://arxiv.org/html/2606.13803#bib.bib16)\],222[https://github\.com/oxford\-oxcav/fossil](https://github.com/oxford-oxcav/fossil)x˙1=x2\\dot\{x\}\_\{1\}=x\_\{2\},x˙2=−x1−x2\+13​x13\\dot\{x\}\_\{2\}=\-x\_\{1\}\-x\_\{2\}\+\\tfrac\{1\}\{3\}x\_\{1\}^\{3\}, onΩ=\[−3,2\.5\]×\[−2,1\]\\Omega=\[\-3,2\.5\]\\times\[\-2,1\]\. The initial setΩi\\Omega\_\{\\mathrm\{i\}\}and unsafe setΩu\\Omega\_\{\\mathrm\{u\}\}are unions of axis\-aligned rectangles and disks, lifted verbatim from the FOSSIL reference configuration:

Ωi\\displaystyle\\Omega\_\{\\mathrm\{i\}\}=B​\(\(1\.5,0\),0\.5\)∪\[−1\.8,−1\.2\]×\[−0\.1,0\.1\]∪\[−1\.4,−1\.2\]×\[−0\.5,0\.1\],\\displaystyle=B\\\!\\bigl\(\(1\.5,0\),\\,0\.5\\bigr\)\\;\\cup\\;\[\-1\.8,\-1\.2\]\\\!\\times\\\!\[\-0\.1,0\.1\]\\;\\cup\\;\[\-1\.4,\-1\.2\]\\\!\\times\\\!\[\-0\.5,0\.1\],Ωu\\displaystyle\\Omega\_\{\\mathrm\{u\}\}=B​\(\(−1,−1\),0\.4\)∪\[0\.4,0\.6\]×\[0\.1,0\.5\]∪\[0\.4,0\.8\]×\[0\.1,0\.3\],\\displaystyle=B\\\!\\bigl\(\(\-1,\-1\),\\,0\.4\\bigr\)\\;\\cup\\;\[0\.4,0\.6\]\\\!\\times\\\!\[0\.1,0\.5\]\\;\\cup\\;\[0\.4,0\.8\]\\\!\\times\\\!\[0\.1,0\.3\],whereB​\(c,r\)B\(c,r\)denotes the closed Euclidean disk\. Boundary samples are drawn by Sobol rejection on each component, with400400accepted points each \(independent Sobol streams forΩi\\Omega\_\{\\mathrm\{i\}\}andΩu\\Omega\_\{\\mathrm\{u\}\}\)\. The Nagumo training points are1,0001\{,\}000scrambled Sobol samples onΩ\\Omega, padded outward by0\.0250\.025on each axis to soften boundary effects \(evaluation uses the unpaddedΩ\\Omega\); points are resampled at each training step\.

### F\.2Margins, belt option, and Nagumo decay

The strict\-positivity marginsμi=μu=10−3\\mu\_\{\\mathrm\{i\}\}=\\mu\_\{\\mathrm\{u\}\}=10^\{\-3\}enter[Equation˜8](https://arxiv.org/html/2606.13803#S4.E8)as fixed shifts on the boundary components, so the enforced inequalities arehθ≥μih^\{\\theta\}\\geq\\mu\_\{\\mathrm\{i\}\}onΩi\\Omega\_\{\\mathrm\{i\}\}andhθ≤−μuh^\{\\theta\}\\leq\-\\mu\_\{\\mathrm\{u\}\}onΩu\\Omega\_\{\\mathrm\{u\}\}\. Margins are distinct from the loss\-sideε=10−6\\varepsilon=10^\{\-6\}used inside the violation and slack losses for numerical conditioning\. The Nagumo decay rate isγ=1\\gamma=1throughout\.

FOSSIL’s Barr3 configuration enforces the Nagumo row only on a*belt*around\{hθ=0\}\\\{h^\{\\theta\}=0\\\}, the relaxation justified by Nagumo’s theorem requiringh˙≥0\\dot\{h\}\\geq 0only on the boundary of the safe set\[Ameset al\.,[2019](https://arxiv.org/html/2606.13803#bib.bib2)\]\. We report the unrestricted variant as the default since it provides a stricter operational requirement and a cleaner comparison across methods\.

### F\.3Slack\-network width sweep

[Tables˜5](https://arxiv.org/html/2606.13803#A6.T5)and[6](https://arxiv.org/html/2606.13803#A6.T6)report the seed\-aggregated metrics behind[Figure˜4](https://arxiv.org/html/2606.13803#S4.F4): feasibility margincminθc^\{\\theta\}\_\{\\min\}, slack lossℒslack\\mathcal\{L\}\_\{\\text\{slack\}\}at the final epoch, late\-stage oscillationστ\\sigma\_\{\\tau\}\(std ofℒslack\\mathcal\{L\}\_\{\\text\{slack\}\}over the last1,5001\{,\}500epochs\), parameter and output displacements relative to initialization, and the countnsatn\_\{\\text\{sat\}\}of seeds withcminθ≥0c^\{\\theta\}\_\{\\min\}\\geq 0\. Boldnsat/Nn\_\{\\text\{sat\}\}/Nentries flag widths feasible on every seed\.

Table 5:Slack\-network width sweep, SIREN slack \(ω0=5\\omega\_\{0\}=5\): per\-width seed mean±\\pmstd over55seeds on FOSSIL Barr3\. Feasibility holds on every seed at intermediate widthsh∈\{8,16,32\}h\\in\\\{8,16,32\\\}, peaking ath=16h=16; at the swept extremes the slack collapses to a flat\-zero profile \(relative output displacement≈1\\approx 1\), and tail oscillationστ\\sigma\_\{\\tau\}traces a U\-shape across widths\. Apple M3 Pro \(CPU\);∼2\\sim\\\!2h per sweep\.Table 6:Slack\-network width sweep,tanh\\tanh\-MLP slack: per\-width seed mean±\\pmstd over55seeds on FOSSIL Barr3\. The slack collapses to a flat\-zero profile at every swept width \(relative output displacement≈1\\approx 1\), failing to inflate\. Apple M3 Pro \(CPU\);∼2\\sim\\\!2h per sweep\.

## Appendix GAdditional details: Generative volatility surfaces

### G\.1Options and implied volatility

Options are used to hedge against or speculate on price movements of an underlying asset, and they trade in record volumes on stocks, indices, currencies, and commodities\. We focus on options on a major equity index \(e\.g\. the Nasdaq\-100\)\. At a given instantt0t\_\{0\}, the collection of traded options on such an underlying gives a dataset𝒟=\{\(\(τi,ki\),yi\)\}i=1N\\mathcal\{D\}=\\\{\(\(\\tau\_\{i\},k\_\{i\}\),y\_\{i\}\)\\\}\_\{i=1\}^\{N\}, where

- •τi=Ti−t0\\tau\_\{i\}=T\_\{i\}\-t\_\{0\}is the*time\-to\-expiration*, the remaining duration until the option’s expiryTiT\_\{i\};
- •ki=log⁡\(Ki/FTi\)k\_\{i\}=\\log\(K\_\{i\}/F\_\{T\_\{i\}\}\)is the*log\-moneyness*, with strikeKiK\_\{i\}and forwardFTiF\_\{T\_\{i\}\};333The forwardFTiF\_\{T\_\{i\}\}is the price of the underlying stripped of interest\-rate and dividend effects, projected to expiryTiT\_\{i\}\.
- •yiy\_\{i\}is the observed*implied volatility*, the standard quoting convention for option prices\.

*Implied volatility smoothing*is the task of nowcasting a continuous, positive surface\(τ,k\)↦f​\(τ,k\)\(\\tau,k\)\\mapsto f\(\\tau,k\)from𝒟\\mathcal\{D\}\[Gatheral and Jacquier,[2014](https://arxiv.org/html/2606.13803#bib.bib20), Lucic,[2021](https://arxiv.org/html/2606.13803#bib.bib30)\]\. The surface must respect a pair of shape constraints encoding financial consistency\. With*total volatility*v​\(τ,k\):=f​\(τ,k\)​τv\(\\tau,k\):=f\(\\tau,k\)\\sqrt\{\\tau\}and*total variance*w=v2w=v^\{2\}, a surface must be

- •free from*calendar arbitrage*:𝒞cal​\[f\]:=∂τw≥0\\mathcal\{C\}\_\{\\text\{cal\}\}\[f\]:=\\partial\_\{\\tau\}w\\geq 0;
- •free from*butterfly arbitrage*:𝒞str​\[f\]≥0\\mathcal\{C\}\_\{\\text\{str\}\}\[f\]\\geq 0, where 𝒞str​\[f\]:=\(1\+d1​\(⋅,v\)​∂kv\)​\(1\+d2​\(⋅,v\)​∂kv\)\+v​∂k2v,\\mathcal\{C\}\_\{\\text\{str\}\}\[f\]:=\(1\+d\_\{1\}\(\\cdot,v\)\\,\\partial\_\{k\}v\)\(1\+d\_\{2\}\(\\cdot,v\)\\,\\partial\_\{k\}v\)\+v\\,\\partial\_\{k\}^\{2\}v,andd1,2​\(τ,k,v\):=−k/v±v/2d\_\{1,2\}\(\\tau,k,v\):=\-k/v\\pm v/2\.

The full constraint is𝒞​\[f\]:=\(𝒞cal​\[f\],𝒞str​\[f\]\)\\mathcal\{C\}\[f\]:=\(\\mathcal\{C\}\_\{\\text\{cal\}\}\[f\],\\mathcal\{C\}\_\{\\text\{str\}\}\[f\]\)\.

##### Local volatility

A no\-arbitrage surface yields a*local volatility*model via Dupire’s equation,

a2​\(τ,k\)=𝒞cal​\[f\]​\(τ,k\)𝒞str​\[f\]​\(τ,k\),a^\{2\}\(\\tau,k\)=\\frac\{\\mathcal\{C\}\_\{\\text\{cal\}\}\[f\]\(\\tau,k\)\}\{\\mathcal\{C\}\_\{\\text\{str\}\}\[f\]\(\\tau,k\)\},\(13\)the workhorse diffusion model for pricing exotic derivatives consistently with observed European option prices\. While no\-arbitrage ensures the quotient is well defined, the local volatility inherits its ill\-conditioning, often producing irregular surfaces unsuitable for pricing — small residual violations of𝒞\\mathcal\{C\}at the level of the implied surface translate into locally undefined or negative local volatility, which makes exact constraint satisfaction crucial in this application\.

### G\.2Data

##### Source

We use NDX \(Nasdaq\-100\) option snapshots from OptionMetrics, accessed via WRDS, covering allN=251N=251trading days from 2024\-08\-29 to 2025\-08\-29\. Each daily snapshot carries roughly2,3002\{,\}300mid\-quote observations spread across1313–1818listed expiries, with time\-to\-expiration extending up to∼4\.6\\sim 4\.6years and log\-moneyness spanning roughly\[−1\.7,0\.3\]\[\-1\.7,0\.3\]\(NDX puts are listed further from the forward than calls\)\.

##### Processing

We average bid and ask quotes for the underlying and the options to obtain mid prices\. Forward prices and discount factors are inferred per expiry from box\-spread pricing, with put\-call parity as a cross\-check\. Time\-to\-maturity is expressed in year fractions and log\-moneyness is computed as in[Section˜G\.1](https://arxiv.org/html/2606.13803#A7.SS1)\. Implied volatilities are obtained from the*py\-vollib\-vectorized*library, a vectorized implementation of the “Let’s\-be\-rational” algorithm ofJäckel \[[2015](https://arxiv.org/html/2606.13803#bib.bib28)\]\.444[https://pypi\.org/project/py\-vollib\-vectorized/](https://pypi.org/project/py-vollib-vectorized/)We retain out\-of\-the\-money quotes \(Put for log\-moneynessk≤0k\\leq 0, Call fork\>0k\>0\) and discard the rest\. The end\-to\-end preparation pipeline ships with the supplemental code\.

##### Constraint domain

𝒞​\[f\]\\mathcal\{C\}\[f\]is a property of the decoded surface and must hold at any latent code, not only at observed\(τ,k\)\(\\tau,k\), so the constraint domain is decoupled from any per\-surface empirical hull\. We enforce constraints on a fixed rectangular box\[τmin,τmax\]×\[kmin,kmax\]=\[0\.01,1\.0\]×\[−0\.5,0\.5\]\[\\tau\_\{\\text\{min\}\},\\tau\_\{\\text\{max\}\}\]\\times\[k\_\{\\text\{min\}\},k\_\{\\text\{max\}\}\]=\[0\.01,1\.0\]\\times\[\-0\.5,0\.5\]\. The training data are restricted to the sameτ\\taurange \(the closest expiry slice on each side is retained for boundary stability\), with nokkfilter applied; this leaves roughly2,0002\{,\}000observations across1212–1313expiries per surface\.

### G\.3Autodecoder model

Each surface receives a learned latent codezi∈ℝ32z\_\{i\}\\in\\mathbb\{R\}^\{32\}, optimized jointly with the decoder by gradient descent \(no encoder, following the variational\-autodecoder formulation ofZadehet al\.[2021](https://arxiv.org/html/2606.13803#bib.bib53)\)\. The decoder maps\(zi,τ,k\)\(z\_\{i\},\\tau,k\)to implied volatility, with moneyness rescaled byτ\\sqrt\{\\tau\}to exploit smile\-shape stationarity across expiries andτ\\tauencoded via a log\-spaced Fourier embedding\. Its backbone is a learnable tanh feature layer followed by a two\-hidden\-layer SIREN tail\. The tanh layer can be read as a low\-frequency \(ω0≈1\\omega\_\{0\}\\approx 1\) SIREN layer that saturates rather than oscillating outside the active region, giving graceful asymptotics at large absolute log\-moneyness; the higher\-frequency SIREN tail then represents fine smile structure\. The latent code and the time embedding are concatenated into a per\-slice conditioning vector for a small FiLM modulator that produces\(γℓ,βℓ\)\(\\gamma\_\{\\ell\},\\beta\_\{\\ell\}\)for every backbone layer, including the tanh layer;γℓ\\gamma\_\{\\ell\}uses a learnable scaled sigmoid so FiLM can both attenuate and amplify\.

### G\.4Training

100100epochs of Adam with separate learning rates for decoder/slack parameters \(3×10−43\\times 10^\{\-4\}\) and per\-surface latent codes \(10−310^\{\-3\}\); batch size88surfaces, gradient clipping at norm11\. A KL term \(weightβ=10−2\\beta=10^\{\-2\}\) regularizes eachziz\_\{i\}towards𝒩​\(0,I\)\\mathcal\{N\}\(0,I\), controlling the geometry of the trained aggregate posterior used by thepriorevaluation regime below\. The constraint grid is a Sobol cloud of10,00010\{,\}000\(τ,k\)\(\\tau,k\)points per step over\[τmin,τmax\]×\[kmin,kmax\]\[\\tau\_\{\\text\{min\}\},\\tau\_\{\\text\{max\}\}\]\\times\[k\_\{\\text\{min\}\},k\_\{\\text\{max\}\}\], withτ\\tauwarped throughu↦\(τmin\+u​\(τmax−τmin\)\)2u\\mapsto\(\\sqrt\{\\tau\_\{\\text\{min\}\}\}\+u\(\\sqrt\{\\tau\_\{\\text\{max\}\}\}\-\\sqrt\{\\tau\_\{\\text\{min\}\}\}\)\)^\{2\}\. For the penalty method and the augmented Lagrangian, the constraint weight is increased multiplicatively on a tolerance trigger and capped at100100; under this long\-run minibatch regime with a sparse constraint grid, larger caps introduce optimization instabilities, consistent with the practice ofAckereret al\.\[[2020](https://arxiv.org/html/2606.13803#bib.bib1)\], who report100100as the largest weight tested\.

### G\.5Evaluation regimes

We evaluate constraint satisfaction on a200×200200\\times 200Cartesian\(τ,k\)\(\\tau,k\)grid for each of three latent populations:

- •train: the251251trained latent codes themselves;
- •interpolation:100100samplesz=α​μi\+\(1−α\)​μjz=\\alpha\\mu\_\{i\}\+\(1\-\\alpha\)\\mu\_\{j\}, withμi,μj\\mu\_\{i\},\\mu\_\{j\}a uniformly sampled pair of trained codes andα∼Uniform​\(0,1\)\\alpha\\sim\\mathrm\{Uniform\}\(0,1\);
- •prior:100100samples from𝒩​\(μ¯,Σ^\)\\mathcal\{N\}\(\\bar\{\\mu\},\\widehat\{\\Sigma\}\), the moment\-matched Gaussian approximation to the trained aggregate posteriorq​\(z\)=1N​∑iq​\(z∣xi\)q\(z\)=\\tfrac\{1\}\{N\}\\sum\_\{i\}q\(z\\mid x\_\{i\}\)\. Hereμ¯\\bar\{\\mu\}is the empirical mean of the trainedμi\\mu\_\{i\}andΣ^\\widehat\{\\Sigma\}is their empirical covariance plus the average per\-surface posterior variance on the diagonal \(the within/between decomposition ofCovq​\(z\)​\[z\]\\mathrm\{Cov\}\_\{q\(z\)\}\[z\]\)\. Sampling from𝒩​\(μ¯,Σ^\)\\mathcal\{N\}\(\\bar\{\\mu\},\\widehat\{\\Sigma\}\)rather than the formal𝒩​\(0,I\)\\mathcal\{N\}\(0,I\)prior accounts for the gap that the small KL weightβ=10−2\\beta=10^\{\-2\}leaves betweenq​\(z\)q\(z\)and the prior\.

### G\.6Neural augmented Lagrangian and neural slack variable

The penalty baseline adds the standardp=1p\\\!=\\\!1hinge penalty from \([3](https://arxiv.org/html/2606.13803#S2.E3)\) to the calendar and strike no\-arbitrage constraint heads\. The adaptive penalty weight is capped at100100in the reported run\.

The neural augmented Lagrangian and neural slack variable baselines use the same FiLM conditioning convention as the volatility decoder, but use plain SIREN networks with two output channels and no initialtanh\\tanhfeature layer\. They takek/τk/\\sqrt\{\\tau\}as the scalar network input and use the concatenated vector\[zi,Φ​\(τ\)\]\[z\_\{i\},\\Phi\(\\tau\)\]for FiLM conditioning\.

For the neural augmented Lagrangian the two channels are the multipliers for the calendar and strike no\-arbitrage constraints\. We train the multiplier network jointly with the autodecoder under the augmented Lagrangian objective, using joint optimization for the primal and multiplier parameters\. The adaptive penalty weight is also capped at100100\.

For the neural slack variable baseline, the two output channels are activated withε\+exp⁡\(⋅\)\\varepsilon\+\\exp\(\\cdot\)\. This gives one positive slack variable in\[ε,∞\)\[\\varepsilon,\\infty\)for each no\-arbitrage constraint\.