ORACLE：面向约束学习的优化器相对对齐方法

# ORACLE: Optimizer-Relative Alignment for Constrained LEarning
Source: [https://arxiv.org/html/2610.09040](https://arxiv.org/html/2610.09040)
Wyatt MackeyAffiliation:DEVCOM Army Research LabEmail:[khua@usf\.edu](mailto:[email protected])Kaixun HuaAffiliation:University of South FloridaEmail:[chang5@usf\.edu](mailto:[email protected])J\. Morris ChangAffiliation:University of South FloridaEmail:[xlin2@usf\.edu](mailto:[email protected])Xiaomin LinAffiliation:University of South FloridaEmail:[wyatt\.t\.mackey\.civ@army\.mil](mailto:)

###### Abstract

Constraint\-handling methods typically intervene before the optimizer acts, by modifying the objective or the gradient\. Yet momentum, adaptive scaling, and structured preconditioning can substantially reshape that signal before it becomes a parameter update\. We formulate*optimizer relative constrained learning*, where constraint compatibility is assessed on the post\-optimizer update\. Building on this view, we introduceORACLE, which evaluates the native optimizer’s realized step through a joint endpoint linearization of heterogeneous constraint families, constructs the resulting alignment in the optimizer’s own geometry, bounds its authority, and commits it only after validation\. We evaluate ORACLE across eight Partial Differential Equation benchmarks and four optimizers spanning Euclidean, diagonal adaptive, and structured preconditioned geometries, where it improves or matches native optimizer in 94% of configurations\. Cross\-model analysis shows the same behavior in 92% of configurations, while matched comparisons show improvements over alternative constraint\-handling methods acting at the objective, gradient, and post\-optimizer levels\.

## 1Introduction

Modern learning systems increasingly operate under requirements that extend beyond minimizing a single task loss\. Physics\-informed learning must reconcile data with governing equations and structural constraints[Wang et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib8);[Moro and Chamon \(2025\)](https://arxiv.org/html/2610.09040#bib.bib39); safety\-critical robotics must optimize behavior while satisfying safety requirements[Nguyen et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib41);[Zhang et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib40); and decision\-focused learning couples prediction with constrained downstream optimization, where predictive accuracy alone may not yield feasible or high\-quality decisions[Mandi et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib42);[Mandi et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib44);[Kong et al\. \(2023\)](https://arxiv.org/html/2610.09040#bib.bib43)\. In each case the learner must improve task performance while remaining compatible with additional physical, safety, or decision\-level requirements\.

The most direct way to impose such requirements is to write them into the quantity being minimized\. Penalty terms, constraint multipliers, primal\-dual formulations, and augmented\-Lagrangian methods encode constraints in the objective itself[Moro and Chamon \(2025\)](https://arxiv.org/html/2610.09040#bib.bib39);[Nandwani et al\. \(2019\)](https://arxiv.org/html/2610.09040#bib.bib15);[Dener et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib16);[Fioretto et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib18), and adaptive weighting schemes adjust their relative influence as training proceeds[Bischof and Kraus \(2025\)](https://arxiv.org/html/2610.09040#bib.bib32)\. These methods control how much each requirement contributes to a scalar trade\-off, but they cannot express which*directions*of parameter movement respect it: competing terms may remain directionally incompatible even after their magnitudes are balanced[Wang et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib8)\.

A second family therefore acts on directions, the gradient itself\. Multi\-objective and conflict\-resolution methods remove or reconcile components of competing gradients before a step is taken[Yu et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib19);[Liu et al\. \(2021\)](https://arxiv.org/html/2610.09040#bib.bib20);[Liu et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib37);[Bu et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib38), and projection or SQP\-type methods correct a proposed movement using local constraint geometry[Rosen \(1961\)](https://arxiv.org/html/2610.09040#bib.bib22);[Byrd et al\. \(2008\)](https://arxiv.org/html/2610.09040#bib.bib23)\. This resolves directional conflict that reweighting cannot\. Both families, however, hand their corrected signal to an optimizer, and treat that handover as though the optimizer will apply what it receives\.

Each optimizer defines its own geometry by scaling and prioritizing parameter magnitude and directions differently\. Momentum, adaptive coordinate statistics, and structured preconditioning transform the incoming gradient using information accumulated across training, so the displacement ultimately applied to the parameters can differ from that gradient in both scale and direction[Kingma and Ba \(2014\)](https://arxiv.org/html/2610.09040#bib.bib1);[Gupta et al\. \(2018\)](https://arxiv.org/html/2610.09040#bib.bib3);[Vyas et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib2);[Si et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib4);[Du and Su \(2026\)](https://arxiv.org/html/2610.09040#bib.bib5)\. Adaptive optimizers descend in a geometry that is not Euclidean[Xie et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib47), and recent work names the resulting gradient\-update mismatch directly in physics\-informed training[Xiao et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib6)\. A constraint enforced before this transformation is therefore enforced in a geometry that is not the geometry of the step actually taken, and this gap widens precisely for the stateful, preconditioned optimizers on which modern training relies\. This leaves one position in the pipeline unoccupied: the realized update, after the optimizer has committed to how the parameters should move\.

We introduceORACLE\(*Optimizer\-Relative Alignment for Constrained LEarning*\), illustrated in Figure[1](https://arxiv.org/html/2610.09040#S1.F1), which occupies that position\. Fromθt\\theta\_\{t\}, the native optimizer uses its internal state to produce the realized stepsts\_\{t\}and proposed endpointθ¯t=θt\+st\\bar\{\\theta\}\_\{t\}=\\theta\_\{t\}\+s\_\{t\}\. At this endpoint, ORACLE jointly linearizes the sampled constraint families so that their local responses to the same parameter movement are considered together\. The resulting local model is then used to construct an alignment in the geometry of the optimizer that producedsts\_\{t\}\. The quantityct⊤​stc\_\{t\}^\{\\top\}s\_\{t\}identifies locally adverse realized movements; when it is positive, ORACLE constructs a bounded optimizer\-relative alignment and commits it only after nonlinear validation and an independent audit\. Thus,θt\+1=θt\+st\+dt\\theta\_\{t\+1\}=\\theta\_\{t\}\+s\_\{t\}\+d\_\{t\}when the alignment is accepted andθt\+1=θ¯t\\theta\_\{t\+1\}=\\bar\{\\theta\}\_\{t\}otherwise\.

We study this in physics\-informed Partial Differential Equation learning, where the gap is most exposed\. PINNs[Raissi et al\. \(2019\)](https://arxiv.org/html/2610.09040#bib.bib7)combine PDE residuals with initial and boundary conditions whose gradients differ sharply in scale, conditioning, and direction, and modern PINN training increasingly depends on adaptive and structured optimizers[Wang et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib8);[Rathore et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib9)\. Heterogeneous constraint sensitivities therefore meet stateful preconditioned updates in the same training loop\. We ask five questions of this setting: whether optimizer\-relative alignment holds across native optimizer geometries; whether it persists when the model formulation changes; what’s the additional computation costs and what it buys; whether intervening on the realized update outperforms intervening earlier under a matched protocol; and which of ORACLE’s mechanisms are responsible\.

Our contributions are as follows\.\(i\)We formulate*optimizer\-relative constrained learning*, in which constraint compatibility is defined on the realized update rather than the gradient preceding it\.\(ii\)We instantiate it as ORACLE, which jointly linearizes heterogeneous constraint families at the proposed endpoint and constructs a single alignment in the certified geometry of the native optimizer\. A matrix\-free residual\-space solve computes this alignment without forming the parameter\-space Jacobian, while native\-relative authority and held\-out validation control whether it is committed\.\(iii\)We evaluate it across optimizer geometries, model formulations, computational cost, intervention points, and internal mechanisms, improving or matching the native optimizer in 94% valid PDE optimizer configurations and 92% model optimizer configurations\.

![Refer to caption](https://arxiv.org/html/2610.09040v1/oracle_toy_landscape_clear.png)Figure 1:Toy illustration of ORACLE\.\(a\) Fromθt\\theta\_\{t\}, the native optimizer proposes the endpointθ¯t\\bar\{\\theta\}\_\{t\}\. \(b\) ORACLE checks the proposal against the local constraint geometry\. \(c\) A validated alignment producesθt\+1\\theta\_\{t\+1\}; otherwise the native proposal is retained\.
## 2Related Work

Modern optimizers separate the raw gradient from the parameter displacement ultimately applied to the model\. AdaGrad, RMSProp, Adam, and Adafactor use gradient history for coordinate\-wise adaptation[Ward et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib10);[Kingma and Ba \(2014\)](https://arxiv.org/html/2610.09040#bib.bib1);[Shazeer and Stern \(2018\)](https://arxiv.org/html/2610.09040#bib.bib11), while natural gradient, K\-FAC, Shampoo, SOAP, and Newton\-type methods introduce metric, structured\-preconditioning, or curvature information[Kakade \(2001\)](https://arxiv.org/html/2610.09040#bib.bib12);[Grosse and Martens \(2016\)](https://arxiv.org/html/2610.09040#bib.bib13);[Gupta et al\. \(2018\)](https://arxiv.org/html/2610.09040#bib.bib3);[Vyas et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib2);[Botev et al\. \(2017\)](https://arxiv.org/html/2610.09040#bib.bib14)\. The realized update can therefore differ from the raw gradient in both direction and scale, making the optimizer’s induced geometry consequential for where and how constraint information is introduced\.

Penalty, Lagrangian, primal\-dual, and augmented\-Lagrangian methods encode constraints directly in the optimization problem[Nandwani et al\. \(2019\)](https://arxiv.org/html/2610.09040#bib.bib15);[Dener et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib16);[Lavado et al\. \(2023\)](https://arxiv.org/html/2610.09040#bib.bib17);[Fioretto et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib18), while multi\-objective methods reconcile competing gradients[Yu et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib19);[Liu et al\. \(2021\)](https://arxiv.org/html/2610.09040#bib.bib20);[Navon et al\. \(2022\)](https://arxiv.org/html/2610.09040#bib.bib21)and projection or SQP\-type methods correct candidate movement using local constraint geometry[Rosen \(1961\)](https://arxiv.org/html/2610.09040#bib.bib22);[Byrd et al\. \(2008\)](https://arxiv.org/html/2610.09040#bib.bib23)\. These approaches span objective, gradient, and movement\-level intervention, but generally do not make the native optimizer’s realized geometry the reference geometry for constraint correction\.

Recent methods enforce nonlinear requirements through constrained formulations and feasibility mechanisms[Moro and Chamon \(2025\)](https://arxiv.org/html/2610.09040#bib.bib39);[Lastrucci and Schweidtmann \(2025\)](https://arxiv.org/html/2610.09040#bib.bib45);[Iftakher et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib46), or by modifying task gradients[Liu et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib37);[Bu et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib38)\. Yet in PINNs, heterogeneous constraint sensitivities already induce difficult optimization geometry[Wang et al\. \(2021a\)](https://arxiv.org/html/2610.09040#bib.bib24);[Yao et al\. \(2023\)](https://arxiv.org/html/2610.09040#bib.bib25);[Rathore et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib9);[Wang et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib8), which adaptive and structured optimizers can further reshape[Kingma and Ba \(2014\)](https://arxiv.org/html/2610.09040#bib.bib1);[Vyas et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib2)\. This creates a missing interface between nonlinear constraint geometry and the geometry of the realized optimizer update, together with the problem of deciding when a locally constructed correction should be trusted\.

ORACLE addresses this interface by recovering and certifying the native optimizer’s realized metric and expressing sampled residual geometry within it\. It applies a bounded correction only when the realized step is locally adverse, then separates candidate construction from acceptance through an independent audit support\. Detailed comparisons with post\-optimizer alignment, projection methods, and sample\-splitting approaches appear in the Extended Related Work[A](https://arxiv.org/html/2610.09040#A1)\.

## 3ORACLE: Optimizer\-Relative Alignment for Constrained LEarning

ORACLE wraps a native optimizer\. At iterationttthe optimizer proposes moving fromθt\\theta\_\{t\}toθ¯t=θt\+st\\bar\{\\theta\}\_\{t\}=\\theta\_\{t\}\+s\_\{t\}; ORACLE commits either that proposal or a corrected one,

θt\+1=θt\+st\+dt,dt=𝟏\{Certt\}1\{Trigt\}1\{Valt\}αtd^t\.\\boxed\{\\;\\theta\_\{t\+1\}=\\theta\_\{t\}\+s\_\{t\}\+d\_\{t\},\\qquad d\_\{t\}=\\mathbf\{1\}\\\{\\mathrm\{Cert\}\_\{t\}\\\}\\,\\mathbf\{1\}\\\{\\mathrm\{Trig\}\_\{t\}\\\}\\,\\mathbf\{1\}\\\{\\mathrm\{Val\}\_\{t\}\\\}\\;\\alpha\_\{t\}\\,\\widehat\{d\}\_\{t\}\.\\;\}\(1\)Each factor ofdtd\_\{t\}is one mechanism \(Fig\.[1](https://arxiv.org/html/2610.09040#S1.F1)\)\. The*alignment direction*d^t\\widehat\{d\}\_\{t\}reduces the local constraint model in the native optimizer’s own geometry; the*corrective authority*αt\\alpha\_\{t\}bounds how far the proposal may be redirected; and three gates decide whether the correction is applied:*geometry certification*Certt\\mathrm\{Cert\}\_\{t\}, the*alignment trigger*Trigt\\mathrm\{Trig\}\_\{t\}, and*validation*Valt\\mathrm\{Val\}\_\{t\}\. If any gate is closed,dt=0d\_\{t\}=0and ORACLE returns the native step exactly\. Algorithm[1](https://arxiv.org/html/2610.09040#alg1)lists the procedure; the rest of this section defines each factor in turn\.

Constraints are organized intoKKsemantic families\. In a PINN, for example, these may correspond to the PDE residual, boundary conditions, initial conditions, or observed data\. For familykk,rk​\(θ,z\)r\_\{k\}\(\\theta;z\)measures the violation at evaluation pointzz,z∼νkz\\sim\\nu\_\{k\}specifies how those points are sampled, andωk,t\>0\\omega\_\{k,t\}\>0controls the family’s relative contribution\.

### 3\.1Native Proposal and Constraint Model

Letgtg\_\{t\}be the gradient supplied to the native optimizer, andξt\\xi\_\{t\}its internal state, such as momentum, running moments, or structured preconditioning statistics\. The native optimizer produces

st=𝒮⁡\(gt,ξt\),ξt\+1=𝒰⁡\(gt,ξt\),θ¯t=θt\+st,s\_\{t\}=\\mathcal\{S\}\(g\_\{t\},\\xi\_\{t\}\),\\qquad\\xi\_\{t\+1\}=\\mathcal\{U\}\(g\_\{t\},\\xi\_\{t\}\),\\qquad\\bar\{\\theta\}\_\{t\}=\\theta\_\{t\}\+s\_\{t\},\(2\)wherests\_\{t\}is the realized parameter displacement andθ¯t\\bar\{\\theta\}\_\{t\}the native proposal\. ORACLE leaves bothsts\_\{t\}and the state transition untouched\. Withdt=0d\_\{t\}=0, Eq\.[1](https://arxiv.org/html/2610.09040#S3.E1)is the native updateθt\+1=θ¯t\\theta\_\{t\+1\}=\\bar\{\\theta\}\_\{t\}; this is the default that each mechanism below modifies only under a stated condition\.

To evaluate this proposal, we summarize the population\-level constraint violation as

Φt​\(θ\)=∑k=1Kωk,t​𝔼z∼νk​\[rk​\(θ,z\)2\]\.\\Phi\_\{t\}\(\\theta\)=\\sum\_\{k=1\}^\{K\}\\omega\_\{k,t\}\\,\\mathbb\{E\}\_\{z\\sim\\nu\_\{k\}\}\\\!\\left\[r\_\{k\}\(\\theta;z\)^\{2\}\\right\]\.\(3\)Afterθ¯t\\bar\{\\theta\}\_\{t\}is fixed, ORACLE evaluates its constraint violations on a fresh support: for each familykkit drawsmkm\_\{k\}sampleszk,j∼νkz\_\{k,j\}\\sim\\nu\_\{k\}, collects them inStS\_\{t\}, stacks the residuals intorSt​\(θ\)r\_\{S\_\{t\}\}\(\\theta\), and measures

ΦSt​\(θ\)=12​rSt​\(θ\)⊤​Wt​rSt​\(θ\),\\Phi\_\{S\_\{t\}\}\(\\theta\)=\\tfrac\{1\}\{2\}\\,r\_\{S\_\{t\}\}\(\\theta\)^\{\\top\}W\_\{t\}\\,r\_\{S\_\{t\}\}\(\\theta\),\(4\)whereWtW\_\{t\}is diagonal and assigns each residual from familykkthe weight2​ωk,t/mk2\\omega\_\{k,t\}/m\_\{k\}\. Becauseθ¯t\\bar\{\\theta\}\_\{t\}is fixed beforeStS\_\{t\}is sampled,ΦSt​\(θ¯t\)\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\)is an unbiased estimate of the population constraint energy \(Appendix[B\.1](https://arxiv.org/html/2610.09040#A2.SS1)\)\. The sampled energy tells us how much violation remains at the proposed endpoint\. SincerStr\_\{S\_\{t\}\}stacks PDE, boundary, initial\-condition, and other residuals into a single endpoint model, ORACLE next asks how these constraint families respond jointly to the same parameter movement\. Answering this requires their local sensitivities and the geometry in which the optimizer producedsts\_\{t\}\.

### 3\.2Checking the Proposal in the Native Optimizer Geometry

Different optimizers induce different geometries on parameter updates: Adam rescales coordinates using adaptive statistics, while SOAP and PSGD\-Kron additionally apply structured preconditioning\. Rather than constructing a new geometry, ORACLE reuses the transformation already encoded by the native optimizer and writes the realized step as

st=−Bt​ut,Pt=Bt−1≻0,s\_\{t\}=\-B\_\{t\}u\_\{t\},\\qquad P\_\{t\}=B\_\{t\}^\{\-1\}\\succ 0,\(5\)whereutu\_\{t\}is the optimizer’s effective direction andBtB\_\{t\}is the symmetric positive\-definite \(SPD\) scaling or preconditioning operator induced by its update rule and state\. For SGD,BtB\_\{t\}is scalar learning\-rate scaling; for Adam, diagonal adaptive scaling; for structured optimizers, it acts through their existing factors\. The inversePtP\_\{t\}defines the optimizer\-relative metric‖v‖Pt2=v⊤​Pt​v\\\|v\\\|\_\{P\_\{t\}\}^\{2\}=v^\{\\top\}P\_\{t\}v, which reflects the native optimizer’s directional preferences\. NeitherBtB\_\{t\}norPtP\_\{t\}is formed explicitly; ORACLE uses them only through optimizer\-specific vector products\. The geometry is meaningful only if it faithfully describes the step actually produced, so ORACLE certifies it against the realized displacement,

‖st\+Bt​ut‖2≤τrel​‖st‖2\+τabs,\\\|s\_\{t\}\+B\_\{t\}u\_\{t\}\\\|\_\{2\}\\leq\\tau\_\{\\mathrm\{rel\}\}\\\|s\_\{t\}\\\|\_\{2\}\+\\tau\_\{\\mathrm\{abs\}\},\(6\)together with optimizer\-specific validity checks \(Appendix[B\.2](https://arxiv.org/html/2610.09040#A2.SS2)\);Certt\\mathrm\{Cert\}\_\{t\}is the event that both hold\. On failure ORACLE preserves the native proposal and never replaces an uncertified geometry withPt=IP\_\{t\}=I\. This supplies the gate𝟏​\{Certt\}\\mathbf\{1\}\\\{\\mathrm\{Cert\}\_\{t\}\\\}and the norm in which every quantity below is measured\.

ORACLE now asks whether the movement fromθt\\theta\_\{t\}toθ¯t\\bar\{\\theta\}\_\{t\}is locally adverse to the sampled constraints\. At the proposed endpoint, define

rt=rSt​\(θ¯t\),Jt=∂rSt​\(θ\)∂θ\|θ=θ¯t,ct=Jt⊤​Wt​rt=∇ΦSt​\(θ¯t\),r\_\{t\}=r\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\),\\qquad J\_\{t\}=\\left\.\\frac\{\\partial r\_\{S\_\{t\}\}\(\\theta\)\}\{\\partial\\theta\}\\right\|\_\{\\theta=\\bar\{\\theta\}\_\{t\}\},\\qquad c\_\{t\}=J\_\{t\}^\{\\top\}W\_\{t\}r\_\{t\}=\\nabla\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\),\(7\)wherertr\_\{t\}measures the sampled violations at the endpoint,JtJ\_\{t\}their local sensitivity to parameter movement, andctc\_\{t\}the first\-order direction of increasing constraint energy\. The trigger is the directional interaction betweenctc\_\{t\}and the realized step:

Trigt=\{ct⊤st\>0\}\.\\mathrm\{Trig\}\_\{t\}=\\\{\\,c\_\{t\}^\{\\top\}s\_\{t\}\>0\\,\\\}\.\(8\)When it fires, the native movement is locally adverse to the sampled constraint model and ORACLE opens the alignment path; otherwise the native proposal is preserved\.JtJ\_\{t\}is never formed explicitly: a Jacobian\-vector productJt​vJ\_\{t\}vmeasures how a parameter movementvvchanges the residuals, and a vector\-Jacobian productJt⊤​qJ\_\{t\}^\{\\top\}qmaps a residual\-space direction back to parameter space, both by automatic differentiation\. Formal analysis of the endpoint trigger appears in Appendix[B\.4](https://arxiv.org/html/2610.09040#A2.SS4)\. This supplies the gate𝟏​\{Trigt\}\\mathbf\{1\}\\\{\\mathrm\{Trig\}\_\{t\}\\\}\.

### 3\.3Optimizer\-Relative Alignment and Validated Commitment

Once the trigger fires, the certified metric determines how the proposal is corrected\. Atθ¯t\\bar\{\\theta\}\_\{t\}, letbt=Wt1/2​rtb\_\{t\}=W\_\{t\}^\{1/2\}r\_\{t\}andCt=Wt1/2​JtC\_\{t\}=W\_\{t\}^\{1/2\}J\_\{t\}\. ORACLE computes the alignment direction by solving

d^t=arg⁡mind∈ℝp​12​‖bt\+Ct​d‖22\+λt2​‖d‖Pt2,\\widehat\{d\}\_\{t\}=\\arg\\min\_\{d\\in\\mathbb\{R\}^\{p\}\}\\ \\tfrac\{1\}\{2\}\\\|b\_\{t\}\+C\_\{t\}d\\\|\_\{2\}^\{2\}\+\\tfrac\{\\lambda\_\{t\}\}\{2\}\\\|d\\\|\_\{P\_\{t\}\}^\{2\},\(9\)with dampingλt\>0\\lambda\_\{t\}\>0\. The first term jointly reduces the local linearized violations of the stacked constraint families; the second keeps the adjustment close to the native proposal in the optimizer’s own geometry\. Rather than solving Eq\.[9](https://arxiv.org/html/2610.09040#S3.E9)in parameter space, ORACLE uses

Gt=Ct​Pt−1​Ct⊤=Wt1/2​Jt​Pt−1​Jt⊤​Wt1/2,\(Gt\+λt​I\)​zt=bt,d^t=−Pt−1​Ct⊤​zt\.G\_\{t\}=C\_\{t\}P\_\{t\}^\{\-1\}C\_\{t\}^\{\\top\}=W\_\{t\}^\{1/2\}J\_\{t\}P\_\{t\}^\{\-1\}J\_\{t\}^\{\\top\}W\_\{t\}^\{1/2\},\\qquad\(G\_\{t\}\+\\lambda\_\{t\}I\)\\,z\_\{t\}=b\_\{t\},\\qquad\\widehat\{d\}\_\{t\}=\-P\_\{t\}^\{\-1\}C\_\{t\}^\{\\top\}z\_\{t\}\.\(10\)NeitherJtJ\_\{t\}norGtG\_\{t\}is materialized: a product withGtG\_\{t\}uses one VJP, one application ofPt−1P\_\{t\}^\{\-1\}, and one JVP\. The resulting direction is a strict first\-order constraint\-descent direction; Appendix[B\.5](https://arxiv.org/html/2610.09040#A2.SS5)gives the exact and finite\-CG results\. This supplies the direction ofdtd\_\{t\}\.

ORACLE next limits how far the constraints may redirect the native proposal, scalingd^t\\widehat\{d\}\_\{t\}by

αt=min⁡\{ct⊤​st−ct⊤​d^t\+εauth,βt​‖st‖Pt‖d^t‖Pt\},d~t=αt​d^t,\\alpha\_\{t\}=\\min\\\!\\left\\\{\\frac\{c\_\{t\}^\{\\top\}s\_\{t\}\}\{\-c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t\}\+\\varepsilon\_\{\\mathrm\{auth\}\}\},\\;\\frac\{\\beta\_\{t\}\\\|s\_\{t\}\\\|\_\{P\_\{t\}\}\}\{\\\|\\widehat\{d\}\_\{t\}\\\|\_\{P\_\{t\}\}\}\\right\\\},\\qquad\\widetilde\{d\}\_\{t\}=\\alpha\_\{t\}\\widehat\{d\}\_\{t\},\(11\)whereβt∈\[0,1\)\\beta\_\{t\}\\in\[0,1\)controls the maximum corrective authority relative to the native step andεauth\>0\\varepsilon\_\{\\mathrm\{auth\}\}\>0is a small numerical stabilizer\. The first term is the scale needed to neutralize the detected first\-order adverse signal; the second caps the correction in the optimizer\-relative metric\. Consequently‖d~t‖Pt≤βt​‖st‖Pt\\\|\\widetilde\{d\}\_\{t\}\\\|\_\{P\_\{t\}\}\\leq\\beta\_\{t\}\\\|s\_\{t\}\\\|\_\{P\_\{t\}\}, so the aligned update remains bounded relative to the movement selected by the native optimizer\. This supplies the magnitude ofdtd\_\{t\}:dt∈\{0,d~t\}d\_\{t\}\\in\\\{0,\\widetilde\{d\}\_\{t\}\\\}\.

Because the alignment is derived from a local linear model, ORACLE verifies that its predicted improvement persists under the nonlinear constraints,

Predt=ΦSt​\(θ¯t\)−12​∥bt\+Ct​d~t∥22,Aredt=ΦSt​\(θ¯t\)−ΦSt​\(θ¯t\+d~t\)\.\\mathrm\{Pred\}\_\{t\}=\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\)\-\\frac\{1\}\{2\}\\lVert b\_\{t\}\+C\_\{t\}\\widetilde\{d\}\_\{t\}\\rVert\_\{2\}^\{2\},\\qquad\\mathrm\{Ared\}\_\{t\}=\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\)\-\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\+\\widetilde\{d\}\_\{t\}\)\.\(12\)and the candidate proceeds only ifPredt\>0\\mathrm\{Pred\}\_\{t\}\>0,Aredt\>0\\mathrm\{Ared\}\_\{t\}\>0, andAredt/Predt≥ρmin\\mathrm\{Ared\}\_\{t\}/\\mathrm\{Pred\}\_\{t\}\\geq\\rho\_\{\\min\}\. Because this check reusesStS\_\{t\}, it tests local\-model agreement rather than independent improvement\. ORACLE therefore fixes the candidate and evaluates it on an independent audit supportSt′S\_\{t\}^\{\\prime\}, accepting only if

ΦSt′​\(θ¯t\+d~t\)≤ΦSt′​\(θ¯t\)\.\\Phi\_\{S\_\{t\}^\{\\prime\}\}\(\\bar\{\\theta\}\_\{t\}\+\\widetilde\{d\}\_\{t\}\)\\leq\\Phi\_\{S\_\{t\}^\{\\prime\}\}\(\\bar\{\\theta\}\_\{t\}\)\.\(13\)Valt\\mathrm\{Val\}\_\{t\}is the event that both checks pass\. Finally,

dt=\{d~t,if all checks pass,0,otherwise,θt\+1=θt\+st\+dt,d\_\{t\}=\\begin\{cases\}\\widetilde\{d\}\_\{t\},&\\text\{if all checks pass\},\\\\ 0,&\\text\{otherwise\},\\end\{cases\}\\qquad\\theta\_\{t\+1\}=\\theta\_\{t\}\+s\_\{t\}\+d\_\{t\},\(14\)which is equation[1](https://arxiv.org/html/2610.09040#S3.E1)\. Any rejection recovers the exact native update while leavingξt\+1\\xi\_\{t\+1\}unchanged\. In summary, the native optimizer*proposes*, the constraints*check*, ORACLE*aligns*in the certified native geometry, and validation determines whether the aligned proposal is committed\.

Algorithm 1ORACLE: Optimizer\-Relative Alignment for Constrained Learning0:Parameters

θt\\theta\_\{t\}, optimizer state

ξt\\xi\_\{t\}, task gradient

gtg\_\{t\};

0:constraint sampling laws and ORACLE hyperparameters

1:Form the native proposal

\(st,ξt\+1\)←NativeStep⁡\(gt,ξt\)\(s\_\{t\},\\xi\_\{t\+1\}\)\\leftarrow\\mathrm\{NativeStep\}\(g\_\{t\},\\xi\_\{t\}\),

θ¯t←θt\+st\\bar\{\\theta\}\_\{t\}\\leftarrow\\theta\_\{t\}\+s\_\{t\}
2:Recover and certify the optimizer geometry

\(Bt,Pt,ut\)\(B\_\{t\},P\_\{t\},u\_\{t\}\)via Eq\.[5](https://arxiv.org/html/2610.09040#S3.E5)\- Eq\.[6](https://arxiv.org/html/2610.09040#S3.E6);return

\(θ¯t,ξt\+1\)\(\\bar\{\\theta\}\_\{t\},\\xi\_\{t\+1\}\)if certification fails

3:Draw construction support

StS\_\{t\}and evaluate the local constraint model at

θ¯t\\bar\{\\theta\}\_\{t\}to obtain

\(rt,ct\)\(r\_\{t\},c\_\{t\}\)and Jacobian products with

JtJ\_\{t\}
4:return

\(θ¯t,ξt\+1\)\(\\bar\{\\theta\}\_\{t\},\\xi\_\{t\+1\}\)if

ct⊤​st≤0c\_\{t\}^\{\\top\}s\_\{t\}\\leq 0\{trigger Eq\.[8](https://arxiv.org/html/2610.09040#S3.E8)\}

5:Construct the optimizer\-relative alignment direction

d^t\\widehat\{d\}\_\{t\}using Eq\.[10](https://arxiv.org/html/2610.09040#S3.E10)

6:return

\(θ¯t,ξt\+1\)\(\\bar\{\\theta\}\_\{t\},\\xi\_\{t\+1\}\)if

ct⊤​d^t≥0c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t\}\\geq 0\{finite\-precision guard\}

7:Apply the native\-relative authority bound Eq\.[11](https://arxiv.org/html/2610.09040#S3.E11)to obtain

d~t\\widetilde\{d\}\_\{t\}
8:Validate

d~t\\widetilde\{d\}\_\{t\}on

StS\_\{t\}using Eq\.[12](https://arxiv.org/html/2610.09040#S3.E12);return

\(θ¯t,ξt\+1\)\(\\bar\{\\theta\}\_\{t\},\\xi\_\{t\+1\}\)if validation fails

9:Fix

d~t\\widetilde\{d\}\_\{t\}, draw independent audit support

St′S^\{\\prime\}\_\{t\}, andreturn

\(θ¯t,ξt\+1\)\(\\bar\{\\theta\}\_\{t\},\\xi\_\{t\+1\}\)if the held\-out audit Eq\.[13](https://arxiv.org/html/2610.09040#S3.E13)fails

10:return

\(θ¯t\+d~t,ξt\+1\)\(\\bar\{\\theta\}\_\{t\}\+\\widetilde\{d\}\_\{t\},\\xi\_\{t\+1\}\)

## 4Experiment

The central claim of ORACLE is that constraint alignment should be defined relative to the movement produced by the native optimizer\. We test this claim from breadth to mechanism, and structure our experiments around the following research questions: Q1 varies optimizer geometry; Q2 varies model formulation; Q3 quantifies the cost and solution\-level benefit of alignment; Q4 compares objective, gradient, and post\-optimizer intervention; and Q5 isolates ORACLE’s geometry, authority, and validation mechanisms\.

Within every Native ORACLE pair, we hold the architecture, native objective, optimizer configuration, learning\-rate schedule, sampling pipeline, and training budget fixed; ORACLE acts only after the native optimizer forms its realized update\. Our primary metric is relativeℓ2\\ell\_\{2\}solution error \(rel\-L2\), with lower values indicating greater solution accuracy\. Complete architectures, optimizer settings, sampling protocols, and ORACLE hyperparameters appear in Appendix[C](https://arxiv.org/html/2610.09040#A3)\.

Figure 2:Performance across optimizer geometries\.Endpoint rel\-L2L\_\{2\}error across eight PDE benchmarks for Adam, SOAP, SGD, and PSGD\-Kron, comparing each Native VS Native\+ORACLE\. Bars show means; the standard deviation across three independent runs is<5%<5\\%for all reported entries\.### 4\.1Q1: Does ORACLE Work Across Optimizer Geometries?

ORACLE defines alignment relative to the geometry of the native optimizer\. Its first test is therefore whether the same alignment principle remains effective when that geometry changes\. We pair ORACLE with SGD[Robbins and Monro \(1951\)](https://arxiv.org/html/2610.09040#bib.bib34), Adam[Kingma and Ba \(2014\)](https://arxiv.org/html/2610.09040#bib.bib1), SOAP[Vyas et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib2), and PSGD\-Kron[Li \(2017\)](https://arxiv.org/html/2610.09040#bib.bib35)across eight PDE benchmarks, fixing PirateNet[Wang et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib36)throughout\. These optimizers span Euclidean, diagonal\-adaptive, and structured preconditioned update geometries\.

##### Results\.

Figure[2](https://arxiv.org/html/2610.09040#S4.F2)shows that ORACLE improves or matches Native in30/3230/32valid PDE optimizer pairs:2323improve,77match, and22degrade\. Improvements occur under every optimizer family, spanning Euclidean SGD, diagonal\-adaptive Adam, and the structured geometries of SOAP and PSGD\-Kron\. ORACLE therefore does not depend on a single optimizer or a shared update geometry\.

Q1 establishes breadth across native optimizer geometries, but all runs use PirateNet\. Q2 tests whether the same behavior persists when the model formulation changes\. Complete results appear in Appendix[D\.1](https://arxiv.org/html/2610.09040#A4.SS1)\.

Figure 3:Performance across model formulations\.Endpoint rel\-L2L\_\{2\}error on Wave, Burgers, Allen\-Cahn, and KdV for MLP, PINN, and PirateNet under Adam and SOAP, VS ORACLE\. Lower is better\. Each entry is mean, the std\-dev across three independent runs < 7% for reported entries\.

### 4\.2Q2: Does Alignment Persist Across Model Formulations?

Q1 varies the optimizer while fixing PirateNet, leaving open whether the effect depends on that model\. We therefore evaluate a standard MLP, a conventional PINN[Raissi et al\. \(2019\)](https://arxiv.org/html/2610.09040#bib.bib7), and PirateNet[Wang et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib36)on Wave, Burgers, Allen\-Cahn, and KdV under Adam and SOAP\. The MLP and PINN share the same backbone but differ in whether the PDE residual enters the native objective, while PirateNet changes both the architecture and its physics\-specific inductive structure\.

##### Results\.

Figure[3](https://arxiv.org/html/2610.09040#S4.F3)shows that ORACLE improves or matches Native in22/2422/24configurations\. The effect spans all three model formulations and both native optimizers, with gains reaching60\.6%60\.6\\%for PINN\+Adam on Burgers and90\.6%90\.6\\%for PirateNet\+SOAP on KdV\.

The two degradations occur for Pure MLP on Allen\-Cahn under Adam and SOAP, showing that ORACLE does not override limitations of the underlying learning formulation\. Nevertheless, Q2 extends the Q1 result beyond PirateNet and shows that optimizer\-relative alignment persists across substantially different model formulations\.

##### Where do the gains appear?

Aggregate rel\-L2L\_\{2\}obscures where solution error is reduced\. Figure[4](https://arxiv.org/html/2610.09040#S4.F4)compares the reference solution with matched Native and ORACLE pointwise error fields\. On Allen–Cahn, Native error is concentrated along the phase interfaces, where ORACLE substantially attenuates the error bands\. On Wave, the reduction is distributed more broadly across space time\. Thus, the endpoint gains reflect meaningful changes in the learned solution rather than only improvements in an aggregate metric\. Complete results appear in Appendix[D\.2](https://arxiv.org/html/2610.09040#A4.SS2)\.

![Refer to caption](https://arxiv.org/html/2610.09040v1/fig_b_error_fields.png)Figure 4:Representative solution and error fields\.Reference solution \(left\), Native pointwise error \(middle\), and ORACLE pointwise error \(right\) for Allen–Cahn and Wave\. ORACLE reduces interface localized error on Allen–Cahn and broader space time error on Wave\. Native and ORACLE use the same error scale within each PDE\.Figure 5:Cost\-benefit of ORACLE\.Endpoint accuracy, per\-step runtime, temporal win rate, and equal\-wall\-clock performance across model formulations and optimizer geometries\.

### 4\.3Q3: What Does ORACLE Cost, and What Does That Cost Buy?

Q1 and Q2 establish that ORACLE remains effective across optimizer geometries and model formulations\. We next ask what these gains cost, whether they persist beyond endpoint error, and whether they amortize under a fixed compute budget\.

##### Results\.

Figure[5](https://arxiv.org/html/2610.09040#S4.F5)\(a\) shows that ORACLE reduces normalized endpoint error to0\.90×0\.90\\times,0\.75×0\.75\\times, and0\.38×0\.38\\timesNative for MLP, PINN, and PirateNet, respectively\. These gains come at a cost: ORACLE requires approximately4\.04\.0–5\.8×5\.8\\timesmore time per optimization step\.\(Fig\.[5](https://arxiv.org/html/2610.09040#S4.F5)b\)\.

Panel[5](https://arxiv.org/html/2610.09040#S4.F5)\(c\) asks whether the improvement persists through the physical evolution of the PDE rather than appearing only in the final aggregate error\. At each physical time slice, we compare Native and ORACLE using the relativeℓ2\\ell\_\{2\}error over the complete spatial field; ORACLE wins a slice only when its full spatial prediction is more accurate\. Across matched configurations, ORACLE wins81\.1%81\.1\\%,88\.8%88\.8\\%, and82\.9%82\.9\\%of evaluated time slices for MLP, PINN, and PirateNet, respectively\. Thus, the endpoint gains reflect persistent improvements throughout the predicted dynamics rather than a small set of favorable spatial locations or times\.

Panel[5](https://arxiv.org/html/2610.09040#S4.F5)\(d\) imposes the stricter equal\-wall\-clock test, giving Native and ORACLE the same elapsed time rather than the same number of optimization steps\. This comparison combines update quality with update throughput\. ORACLE still reduces aggregate error by approximately73%73\\%with SGD, showing that its alignment benefit can exceed its correction cost\. For Adam, SOAP, and PSGD\-Kron, the additional correction does not yet recover its runtime overhead within the same budget\.

This sharpens Q1: optimizer\-relative alignment remains effective across all tested geometries, while its compute\-normalized benefit depends on how quickly that geometry\-aware correction can be realized\. Together, the results separate*alignment effectiveness*from*compute efficiency*\. For detailed analysis see Appendix[D\.3](https://arxiv.org/html/2610.09040#A4.SS3)\.

### 4\.4Q4: Where Should Constraint Intervention Occur?

Q1, Q2 and Q3 establish breadth, but do not determine whether the gains arise from optimizer\-relative alignment or merely from additional constraint handling\. We therefore compare objective, gradient, and post\-optimizer interventions on Allen\-Cahn and Wave\. Every method branches from the same5,0005\{,\}000\-step Native and receives an identical additional5,0005\{,\}000optimization steps, controlling initialization, optimizer state, and training horizon\.

##### Results\.

Figure[6](https://arxiv.org/html/2610.09040#S4.F6)shows that ORACLE achieves the lowest endpoint rel\-L2L\_\{2\}error on both PDEs, improving over Native by61\.2%61\.2\\%on Allen–Cahn and44\.6%44\.6\\%on Wave\. The post\-fork trajectories strengthen this result: from the shared55k Native, ORACLE remains below Native at every recorded milestone and decreases monotonically to the1010k endpoint on both problems\. Objective\- and gradient\-level interventions do not show the same consistency\. Post\-optimizer intervention alone does not explain which part of ORACLE produces the gain\. Q5 therefore fixes*where*constraints act and examines*how*the resulting update is aligned\.

Figure 6:Matched Q4 intervention comparison on Wave and Allen\-Cahn\.Panels \(a,c\) show endpoint rel\-L2L\_\{2\}error; panels \(b,d\) show trajectories from the shared55k Native checkpoint to1010k\. ORACLE achieves the lowest endpoint error on both PDEs\. Each entry is mean, the std\-dev across three independent runs < 7% for all reported entries\.

### 4\.5Q5: What Makes Optimizer\-Relative Alignment Work?

Q5 therefore examines three mechanisms: optimizer\-relative geometry, corrective authority, and validation\. We compare five settings on Allen\-Cahn and Wave:*Native*applies no post\-optimizer correction;Pt=IP\_\{t\}=Ireplaces the optimizer metric with Euclidean geometry;*First\-order*retains the native metric but uses the direct correctiond∝−Pt−1​ctd\\propto\-P\_\{t\}^\{\-1\}c\_\{t\};*Unbounded*removes the authority cap; and*ORACLE*combines the full optimizer\-relative, bounded, and validated alignment\. Figure[7](https://arxiv.org/html/2610.09040#S4.F7)traces these choices from alignment behavior to final solution error\.

To compare how each mechanism affects the final solution on a common scale, we normalize endpoint error by the matched Native optimizer,Rerr=relL2⁡\(method\)relL2⁡\(Native\),R\_\{\\mathrm\{err\}\}=\\frac\{\\mathrm\{relL2\}\(\\mathrm\{method\}\)\}\{\\mathrm\{relL2\}\(\\mathrm\{Native\}\)\},whereRerr<1R\_\{\\mathrm\{err\}\}<1denotes improvement\. Replacing the optimizer\-relative metric withPt=IP\_\{t\}=Iincreases error to18\.6×18\.6\\timesNative on Allen–Cahn and5\.4×5\.4\\timeson Wave \(Fig\.[7](https://arxiv.org/html/2610.09040#S4.F7)a\)\. In contrast, the metric\-aware first\-order variant and full ORACLE remain below Native on both PDEs\. Thus, successful alignment depends strongly on measuring movement in the geometry induced by the native optimizer rather than in a common Euclidean metric\. ORACLE bounds the aligned adjustment in the same geometry,‖d~t‖Pt≤βt​‖st‖Pt\.\\\|\\widetilde\{d\}\_\{t\}\\\|\_\{P\_\{t\}\}\\leq\\beta\_\{t\}\\\|s\_\{t\}\\\|\_\{P\_\{t\}\}\.This bound is active on100%/99%100\\%/99\\%of accepted ORACLE alignments on Allen\-Cahn/Wave and100%/95%100\\%/95\\%for the metric\-aware first\-order variant, compared with38%/2\.5%38\\%/2\.5\\%underPt=IP\_\{t\}=I\(Fig\.[7](https://arxiv.org/html/2610.09040#S4.F7)b\)\. The optimizer metric therefore affects not only the alignment direction but also what constitutes a permissible adjustment relative to the native step\. ORACLE evaluates each fixed candidate on independent support before commitment\. Over55k active steps, the audit rejects875875candidates underPt=IP\_\{t\}=Iversus only44under ORACLE on Allen–Cahn, and1111versus00on Wave \(Fig\.[7](https://arxiv.org/html/2610.09040#S4.F7)c\)\. The Euclidean ablation therefore produces substantially more candidates whose apparent local improvement does not persist on held\-out constraints\. This diagnostic links the geometry used to construct the alignment with its off\-support reliability\. Figure[7](https://arxiv.org/html/2610.09040#S4.F7)\(d\) shows the resulting endpoint error\. Removing the authority bound gives the lowest Allen\-Cahn error, but provides a smaller gain on Wave; full ORACLE improves both PDEs substantially\. We therefore do not interpret every mechanism as independently necessary\. Rather, ORACLE combines three roles: the native optimizer geometry determines*how*alignment is measured, the authority bound limits*how far*the proposal may be redirected, and held\-out validation determines*whether*the aligned proposal is committed\. Extended mechanism analysis appears in Appendix[E](https://arxiv.org/html/2610.09040#A5)\.

Figure 7:Mechanism analysis of ORACLE on Allen–Cahn and Wave\.\(a\) Native optimizer geometry is essential; \(b\) the authority bound is active for nearly all accepted alignments; \(c\) held\-out validation rejects unreliable candidates; and \(d\) full ORACLE improves both PDEs more consistently

## 5Conclusion and Limitations

We introduced ORACLE, which changes both where and how constraints interact with optimization: it acts on the realized native update, constructs a joint local model of heterogeneous constraints at the proposed endpoint, and aligns the resulting update in the geometry of the optimizer that produced the step\. Across five experimental questions, ORACLE improves or matches Native in 94% of optimizer\-PDE settings and 92% of model\-formulation settings, with endpoint error reductions of up to 90\.6% and improvements that persist across the solution domain and physical time\. These improvements come with a clear trade\-off: ORACLE incurs additional computation and depends on the fidelity of its local correction model\. Future work will target cheaper, selective interventions through adaptive correction budgets and reuse of native optimizer geometry, together with richer local models for strongly nonlinear regimes\. Extending the current single\-step analysis to trajectory\-level guarantees and testing optimizer\-relative alignment beyond PDE learning are natural next steps\.

## Acknowledgements

This work was supported by CRADA22\-0034\-J003 and and NVIDIA Academic Grant Program\.

## References

- Berahaset al\.\(2016\)A\. S\. Berahas, J\. Nocedal, and M\. TakácA multi\-batch l\-bfgs method for machine learning\.Advances in Neural Information Processing Systems29\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1)\.
- Bischof and Kraus \(2025\)R\. Bischof and M\. A\. KrausMulti\-objective loss balancing for physics\-informed deep learning\.Computer Methods in Applied Mechanics and Engineering439,pp\. 117914\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1),[§1](https://arxiv.org/html/2610.09040#S1.p2.1)\.
- Botevet al\.\(2017\)A\. Botev, H\. Ritter, and D\. BarberPractical gauss\-newton optimisation for deep learning\.InInternational Conference on Machine Learning,pp\. 557–565\.Cited by:[§2](https://arxiv.org/html/2610.09040#S2.p1.1)\.
- Bourtouleet al\.\(2021\)L\. Bourtoule, V\. Chandrasekaran, C\. A\. Choquette\-Choo, H\. Jia, A\. Travers, B\. Zhang, D\. Lie, and N\. PapernotMachine unlearning\.In2021 IEEE symposium on security and privacy \(SP\),pp\. 141–159\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Brohanet al\.\(2023\)A\. Brohan, N\. Brown, J\. Carbajal, Y\. Chebotar, X\. Chen, K\. Choromanski, T\. Ding, D\. Driess, A\. Dubey, C\. Finn,et al\.Rt\-2: vision\-language\-action models transfer web knowledge to robotic control\.arXiv preprint arXiv:2307\.15818\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Buet al\.\(2026\)D\. Bu, Y\. Byun, and J\. LEEHarmonized cone for feasible and non\-conflict directions in training physics\-informed neural networks\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 112050–112093\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p3.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Byrdet al\.\(2008\)R\. H\. Byrd, F\. E\. Curtis, and J\. NocedalAn inexact sqp method for equality constrained optimization\.SIAM Journal on Optimization19\(1\),pp\. 351–369\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p3.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Cao and Yang \(2015\)Y\. Cao and J\. YangTowards making systems forget with machine unlearning\.In2015 IEEE symposium on security and privacy,pp\. 463–480\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Chenet al\.\(2018\)Z\. Chen, V\. Badrinarayanan, C\. Lee, and A\. RabinovichGradnorm: gradient normalization for adaptive loss balancing in deep multitask networks\.InInternational conference on machine learning,pp\. 794–803\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p2.1)\.
- Deneret al\.\(2020\)A\. Dener, M\. A\. Miller, R\. M\. Churchill, T\. Munson, and C\. ChangTraining neural networks under physical constraints using a stochastic augmented lagrangian approach\.arXiv preprint arXiv:2009\.07330\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p2.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Désidéri \(2012\)J\. DésidériMultiple\-gradient descent algorithm \(mgda\) for multiobjective optimization\.Comptes Rendus\. Mathématique350\(5\-6\),pp\. 313–318\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p2.1)\.
- Du and Su \(2026\)Z\. Du and W\. SuThe newton\-muon optimizer\.arXiv preprint arXiv:2604\.01472\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p4.1)\.
- Fiorettoet al\.\(2020\)F\. Fioretto, P\. Van Hentenryck, T\. W\. Mak, C\. Tran, F\. Baldo, and M\. LombardiLagrangian duality for constrained deep learning\.InJoint European conference on machine learning and knowledge discovery in databases,pp\. 118–135\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p2.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Grosse and Martens \(2016\)R\. Grosse and J\. MartensA kronecker\-factored approximate fisher matrix for convolution layers\.InInternational Conference on Machine Learning,pp\. 573–582\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1),[§2](https://arxiv.org/html/2610.09040#S2.p1.1)\.
- Groveret al\.\(2026a\)U\. Grover, R\. Ranjan, M\. Mao, T\. T\. Dong, S\. Praveen, Z\. Wu, J\. M\. Chang, T\. Mohsenin, Y\. Sheng, A\. Polyzou,et al\.Embodied foundation models at the edge: a survey of deployment constraints and mitigation strategies\.arXiv preprint arXiv:2603\.16952\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Groveret al\.\(2026b\)U\. Grover, R\. Ranjan, A\. Polyzou, W\. T\. Mackey, J\. M\. Chang, L\. Bobadilla, and X\. LinDePICT: decision\-preserving interface for constrained downstream tasks\.External Links:2610\.03945,[Link](https://arxiv.org/abs/2610.03945)Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Guet al\.\(2024\)S\. Gu, L\. Yang, Y\. Du, G\. Chen, F\. Walter, J\. Wang, and A\. KnollA review of safe reinforcement learning: methods, theories, and applications\.IEEE Transactions on Pattern Analysis and Machine Intelligence46\(12\),pp\. 11216–11235\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Guptaet al\.\(2018\)V\. Gupta, T\. Koren, and Y\. SingerShampoo: preconditioned stochastic tensor optimization\.InInternational Conference on Machine Learning,pp\. 1842–1850\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1),[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1),[§1](https://arxiv.org/html/2610.09040#S1.p4.1),[§2](https://arxiv.org/html/2610.09040#S2.p1.1)\.
- Haoet al\.\(2024\)Z\. Hao, J\. Yao, C\. Su, H\. Su, Z\. Wang, F\. Lu, Z\. Xia, Y\. Zhang, S\. Liu, L\. Lu,et al\.Pinnacle: a comprehensive benchmark of physics\-informed neural networks for solving pdes\.Advances in neural information processing systems37,pp\. 76721–76774\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1)\.
- Huanget al\.\(2021\)D\. Huang, H\. Zhang, X\. Song, and R\. ShibasakiDifferentiable projection for constrained deep learning\.arXiv preprint arXiv:2111\.10785\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p3.1)\.
- Iftakheret al\.\(2025\)A\. Iftakher, R\. Golder, B\. N\. Roy, and M\. F\. HasanPhysics\-informed neural networks with hard nonlinear equality and inequality constraints\.Computers & Chemical Engineering,pp\. 109418\.Cited by:[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Kakade \(2001\)S\. M\. KakadeA natural policy gradient\.Advances in neural information processing systems14\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1),[§2](https://arxiv.org/html/2610.09040#S2.p1.1)\.
- Kimet al\.\(2024\)M\. J\. Kim, K\. Pertsch, S\. Karamcheti, T\. Xiao, A\. Balakrishna, S\. Nair, R\. Rafailov, E\. Foster, G\. Lam, P\. Sanketi,et al\.Openvla: an open\-source vision\-language\-action model\.arXiv preprint arXiv:2406\.09246\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Kingma and Ba \(2014\)D\. P\. Kingma and J\. BaAdam: a method for stochastic optimization\.arXiv preprint arXiv:1412\.6980\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p4.1),[§2](https://arxiv.org/html/2610.09040#S2.p1.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1),[§4\.1](https://arxiv.org/html/2610.09040#S4.SS1.p1.1)\.
- Konget al\.\(2023\)L\. Kong, W\. Mu, J\. Cui, Y\. Zhuang, B\. A\. Prakash, B\. Dai, and C\. ZhangDf2: distribution\-free decision\-focused learning\.arXiv preprint arXiv:2308\.05889\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p1.1)\.
- Lastrucci and Schweidtmann \(2025\)G\. Lastrucci and A\. M\. SchweidtmannEnforce: nonlinear constrained learning with adaptive\-depth neural projection\.arXiv preprint arXiv:2502\.06774\.Cited by:[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Lavadoet al\.\(2023\)D\. Lavado, C\. Soares, and A\. MichelettiAchieving constraints in neural networks: a stochastic augmented lagrangian approach\.arXiv preprint arXiv:2310\.16647\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p1.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Li \(2017\)X\. LiPreconditioned stochastic gradient descent\.IEEE transactions on neural networks and learning systems29\(5\),pp\. 1454–1466\.Cited by:[§4\.1](https://arxiv.org/html/2610.09040#S4.SS1.p1.1)\.
- Liuet al\.\(2021\)B\. Liu, X\. Liu, X\. Jin, P\. Stone, and Q\. LiuConflict\-averse gradient descent for multi\-task learning\.Advances in neural information processing systems34,pp\. 18878–18890\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p2.1),[§1](https://arxiv.org/html/2610.09040#S1.p3.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Liuet al\.\(2025\)Q\. Liu, M\. Chu, and N\. ThuereyConfig: towards conflict\-free training of physics informed neural networks\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 59531–59566\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p3.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Lopez\-Paz and Ranzato \(2017\)D\. Lopez\-Paz and M\. RanzatoGradient episodic memory for continual learning\.Advances in neural information processing systems30\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p3.1)\.
- Mandiet al\.\(2026\)J\. Mandi, M\. Defresne, S\. Berden, and T\. GunsFeasibility\-aware decision\-focused learning for predicting parameters in the constraints\.Advances in Neural Information Processing Systems38,pp\. 163285–163309\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p1.1)\.
- Mandiet al\.\(2024\)J\. Mandi, J\. Kotary, S\. Berden, M\. Mulamba, V\. Bucarey, T\. Guns, and F\. FiorettoDecision\-focused learning: foundations, state of the art, benchmark and future opportunities\.Journal of Artificial Intelligence Research80,pp\. 1623–1701\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p1.1)\.
- Moro and Chamon \(2025\)V\. Moro and L\. ChamonSolving differential equations with constrained learning\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 44906–44948\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p2.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Nandwaniet al\.\(2019\)Y\. Nandwani, A\. Pathak, and P\. SinglaA primal dual formulation for deep learning with constraints\.Advances in neural information processing systems32\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p2.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Navonet al\.\(2022\)A\. Navon, A\. Shamsian, I\. Achituve, H\. Maron, K\. Kawaguchi, G\. Chechik, and E\. FetayaMulti\-task learning as a bargaining game\.arXiv preprint arXiv:2202\.01017\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p2.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Nguyenet al\.\(2024\)D\. P\. Nguyen, K\. Hsu, W\. Yu, J\. Tan, and J\. F\. FisacGameplay filters: robust zero\-shot safety through adversarial imagination\.arXiv preprint arXiv:2405\.00846\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p1.1)\.
- Raissiet al\.\(2019\)M\. Raissi, P\. Perdikaris, and G\. E\. KarniadakisPhysics\-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations\.Journal of Computational physics378,pp\. 686–707\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p6.1),[§4\.2](https://arxiv.org/html/2610.09040#S4.SS2.p1.1)\.
- Ranjanet al\.\(2026a\)R\. Ranjan, U\. Grover, X\. Lin, and A\. PolyzouG\-drift mia: membership inference via gradient\-induced feature drift in llms\.InInternational Conference on Pattern Recognition,pp\. 359–374\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Ranjanet al\.\(2026b\)R\. Ranjan, U\. Grover, X\. Lin, and A\. PolyzouPERSA: reinforcement learning for professor\-style personalized feedback with llms\.arXiv preprint arXiv:2605\.0112315\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Ranjanet al\.\(2026c\)R\. Ranjan, U\. Grover, X\. Lin, and A\. PolyzouRazor: ratio\-aware layer editing for targeted unlearning in vision transformers and diffusion models\.arXiv preprint arXiv:2603\.14819\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Rathoreet al\.\(2024\)P\. Rathore, W\. Lei, Z\. Frangella, L\. Lu, and M\. UdellChallenges in training pinns: a loss landscape perspective\.arXiv preprint arXiv:2402\.01868\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1),[§1](https://arxiv.org/html/2610.09040#S1.p6.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Robbins and Monro \(1951\)H\. Robbins and S\. MonroA stochastic approximation method\.The annals of mathematical statistics,pp\. 400–407\.Cited by:[§4\.1](https://arxiv.org/html/2610.09040#S4.SS1.p1.1)\.
- Rosen \(1961\)J\. B\. RosenThe gradient projection method for nonlinear programming\. part ii\. nonlinear constraints\.Journal of the Society for Industrial and Applied Mathematics9\(4\),pp\. 514–532\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p3.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Shazeer and Stern \(2018\)N\. Shazeer and M\. SternAdafactor: adaptive learning rates with sublinear memory cost\.InInternational conference on machine learning,pp\. 4596–4604\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1),[§2](https://arxiv.org/html/2610.09040#S2.p1.1)\.
- Siet al\.\(2025\)C\. Si, D\. Zhang, and W\. ShenAdamuon: adaptive muon optimizer\.arXiv preprint arXiv:2507\.11005\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p4.1)\.
- Tayalet al\.\(2026\)M\. Tayal, M\. Tayal, and R\. PrakashSafe flow q\-learning: offline safe reinforcement learning with reachability\-based flow policies\.arXiv preprint arXiv:2603\.15136\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Vyaset al\.\(2025\)N\. Vyas, D\. Morwani, R\. Zhao, I\. Shapira, D\. Brandfonbrener, L\. Janson, and S\. KakadeSOAP: improving and stabilizing shampoo using adam for language modeling\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 93423–93444\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1),[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1),[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p4.1),[§2](https://arxiv.org/html/2610.09040#S2.p1.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1),[§4\.1](https://arxiv.org/html/2610.09040#S4.SS1.p1.1)\.
- Wanget al\.\(2024\)S\. Wang, B\. Li, Y\. Chen, and P\. PerdikarisPiratenets: physics\-informed deep learning with residual adaptive networks\.Journal of Machine Learning Research25\(402\),pp\. 1–51\.Cited by:[§4\.1](https://arxiv.org/html/2610.09040#S4.SS1.p1.1),[§4\.2](https://arxiv.org/html/2610.09040#S4.SS2.p1.1)\.
- Wanget al\.\(2026\)S\. Wang, B\. Li, P\. Perdikaris,et al\.Gradient alignment in physics\-informed neural networks: a second\-order optimization perspective\.Advances in Neural Information Processing Systems38,pp\. 168482–168532\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1),[§1](https://arxiv.org/html/2610.09040#S1.p1.1),[§1](https://arxiv.org/html/2610.09040#S1.p2.1),[§1](https://arxiv.org/html/2610.09040#S1.p6.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Wanget al\.\(2021a\)S\. Wang, Y\. Teng, and P\. PerdikarisUnderstanding and mitigating gradient flow pathologies in physics\-informed neural networks\.SIAM Journal on Scientific Computing43\(5\),pp\. A3055–A3081\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Wanget al\.\(2021b\)S\. Wang, H\. Wang, and P\. PerdikarisOn the eigenvector bias of fourier feature networks: from regression to solving multi\-scale pdes with physics\-informed neural networks\.Computer Methods in Applied Mechanics and Engineering384,pp\. 113938\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1)\.
- Wardet al\.\(2020\)R\. Ward, X\. Wu, and L\. BottouAdagrad stepsizes: sharp convergence over nonconvex landscapes\.Journal of Machine Learning Research21\(219\),pp\. 1–30\.Cited by:[§A\.1](https://arxiv.org/html/2610.09040#A1.SS1.p1.1),[§2](https://arxiv.org/html/2610.09040#S2.p1.1)\.
- Xiaoet al\.\(2026\)J\. Xiao, X\. Chen, Q\. Wang, M\. Jia, Z\. Lai, D\. Li, J\. Liu, and T\. LiGradient\-update mismatch: rethinking conflict\-free training of physics\-informed neural networks\.arXiv preprint arXiv:2609\.01558\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p4.1)\.
- Xieet al\.\(2026\)S\. Xie, T\. Wang, B\. Wu, and Z\. LiA tale of two geometries: adaptive optimizers and non\-euclidean descent\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 76867–76913\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p4.1)\.
- Yanget al\.\(2026\)L\. Yang, B\. Werner, M\. de Sa, and A\. D\. AmesCBF\-rl: safety filtering reinforcement learning in training with control barrier functions\.In2026 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 17825–17832\.Cited by:[§A\.5](https://arxiv.org/html/2610.09040#A1.SS5.p1.1)\.
- Yaoet al\.\(2023\)J\. Yao, C\. Su, Z\. Hao, S\. Liu, H\. Su, and J\. ZhuMultiadam: parameter\-wise scale\-invariant optimizer for multiscale training of physics\-informed neural networks\.InInternational conference on machine learning,pp\. 39702–39721\.Cited by:[§A\.3](https://arxiv.org/html/2610.09040#A1.SS3.p2.1),[§2](https://arxiv.org/html/2610.09040#S2.p3.1)\.
- Yuet al\.\(2020\)T\. Yu, S\. Kumar, A\. Gupta, S\. Levine, K\. Hausman, and C\. FinnGradient surgery for multi\-task learning\.Advances in neural information processing systems33,pp\. 5824–5836\.Cited by:[§A\.2](https://arxiv.org/html/2610.09040#A1.SS2.p2.1),[§1](https://arxiv.org/html/2610.09040#S1.p3.1),[§2](https://arxiv.org/html/2610.09040#S2.p2.1)\.
- Zhanget al\.\(2026\)B\. Zhang, Y\. Zhang, J\. Ji, Y\. Lei, J\. Dai, Y\. Chen, and Y\. YangSafevla: towards safety alignment of vision\-language\-action model via constrained learning\.Advances in Neural Information Processing Systems38,pp\. 153335–153373\.Cited by:[§1](https://arxiv.org/html/2610.09040#S1.p1.1)\.

## Appendix AExtended Related Work

The main text positions ORACLE relative to the principal families of constraint\-handling methods\. Here we expand that discussion around the interface central to this work: the distinction between the gradient supplied to an optimizer and the parameter displacement that the optimizer ultimately produces\.

### A\.1Optimizer Geometry and the Realized Update

Modern optimizers transform the instantaneous gradient using information accumulated across training\. AdaGrad adapts coordinate scales from gradient history[Ward et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib10); Adam combines first\- and second\-moment statistics[Kingma and Ba \(2014\)](https://arxiv.org/html/2610.09040#bib.bib1); and Adafactor factorizes adaptive statistics to reduce memory[Shazeer and Stern \(2018\)](https://arxiv.org/html/2610.09040#bib.bib11)\. More structured methods alter the update through non\-diagonal geometry: natural gradient uses the Fisher metric[Kakade \(2001\)](https://arxiv.org/html/2610.09040#bib.bib12), K\-FAC approximates Fisher blocks with Kronecker factors[Grosse and Martens \(2016\)](https://arxiv.org/html/2610.09040#bib.bib13), Shampoo maintains tensor\-structured preconditioners[Gupta et al\. \(2018\)](https://arxiv.org/html/2610.09040#bib.bib3), and SOAP combines structured preconditioning with Adam\-style adaptation in an evolving basis[Vyas et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib2)\. Newton, Gauss Newton, and quasi\-Newton methods likewise reshape parameter motion using curvature information[Berahas et al\. \(2016\)](https://arxiv.org/html/2610.09040#bib.bib26)\.

Across these methods, the gradient is an input to the optimizer rather than the parameter movement itself\. Momentum, adaptive scaling, and structured preconditioning can change both the scale and direction of the resulting step\. ORACLE refers to this optimizer\-transformed displacement as the*realized optimizer update*\. This distinction becomes consequential when constraint information must interact with an optimizer whose state already defines a nontrivial geometry\.

### A\.2Constraint Handling and the Intervention Point

Classical constrained optimization typically introduces constraint information before or during construction of the optimization trajectory\. Penalty methods add violation terms to the objective; primal dual methods jointly evolve parameter and multiplier variables[Nandwani et al\. \(2019\)](https://arxiv.org/html/2610.09040#bib.bib15); and augmented\-Lagrangian approaches combine multiplier dynamics with explicit feasibility penalties[Dener et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib16);[Lavado et al\. \(2023\)](https://arxiv.org/html/2610.09040#bib.bib17)\. Related formulations extend these mechanisms to constrained neural networks[Fioretto et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib18)\. In each case, constraint information affects the signal from which the optimizer subsequently constructs its step\.

Multi\-objective methods intervene more directly on gradients\. GradNorm balances objectives using gradient magnitudes[Chen et al\. \(2018\)](https://arxiv.org/html/2610.09040#bib.bib27); MGDA constructs a common descent direction[Désidéri \(2012\)](https://arxiv.org/html/2610.09040#bib.bib28); PCGrad removes pairwise conflicting components[Yu et al\. \(2020\)](https://arxiv.org/html/2610.09040#bib.bib19); CAGrad trades average descent against worst\-objective conflict[Liu et al\. \(2021\)](https://arxiv.org/html/2610.09040#bib.bib20); and Nash\-MTL derives a shared direction through bargaining[Navon et al\. \(2022\)](https://arxiv.org/html/2610.09040#bib.bib21)\. These methods address imbalance or directional conflict before the native optimizer applies its state\-dependent transformation\.

Projection and local constrained\-optimization methods act closer to the parameter movement itself\. Projected\-gradient methods map candidate steps toward feasible sets, SQP constructs locally constrained subproblems, and trust\-region mechanisms restrict movement to regions where local models remain reliable\. Related learning methods include GEM, which projects gradients that would interfere with previously learned tasks[Lopez\-Paz and Ranzato \(2017\)](https://arxiv.org/html/2610.09040#bib.bib29), and differentiable projection layers that explicitly map predictions or parameters toward feasibility[Huang et al\. \(2021\)](https://arxiv.org/html/2610.09040#bib.bib30)\. These methods separate a proposed direction from a constraint\-induced modification, but the proposal is often defined before a stateful optimizer has produced its realized update\.

This ordering matters when the optimizer itself carries useful structure\. For Adam, SOAP, Shampoo, or other stateful optimizers, modifying the raw gradient does not directly specify the parameter displacement that will follow\. ORACLE instead takes the realized native step as the reference movement and asks whether that movement should be aligned with the constraints\.

### A\.3Constraint Geometry in Physics\-Informed Learning

PINNs provide a demanding setting for this distinction because their training objectives combine PDE residuals, initial conditions, boundary conditions, and possibly observations[Raissi et al\. \(2019\)](https://arxiv.org/html/2610.09040#bib.bib7)\. These components can differ substantially in scale, conditioning, spatial support, and gradient direction\.

Several methods address this difficulty before the optimizer forms its step\. Gradient\-statistics\-based annealing adapts loss weights from constituent gradient magnitudes[Wang et al\. \(2021a\)](https://arxiv.org/html/2610.09040#bib.bib24); NTK\-based weighting accounts for unequal convergence rates[Wang et al\. \(2021b\)](https://arxiv.org/html/2610.09040#bib.bib31); ReLoBRaLo adapts weights using relative loss progress[Bischof and Kraus \(2025\)](https://arxiv.org/html/2610.09040#bib.bib32); and MultiAdam maintains separate adaptive statistics for different PINN loss components[Yao et al\. \(2023\)](https://arxiv.org/html/2610.09040#bib.bib25)\. Other work instead strengthens the optimizer geometry itself through curvature\-aware or structured methods, including NysNewton\-CG[Rathore et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib9), Shampoo[Gupta et al\. \(2018\)](https://arxiv.org/html/2610.09040#bib.bib3), and SOAP[Vyas et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib2)\. Recent analyses further show that PINN difficulty is not explained by gradient magnitude alone: constituent gradients can remain directionally incompatible even after loss balancing[Wang et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib8), while benchmark studies show substantial variation in optimizer behavior across PDE families[Hao et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib33)\.

These developments create the setting studied by ORACLE\. The native optimizer may already encode useful task\-dependent scaling or preconditioning, while the physical constraints provide a separate collection of local sensitivities\. Combining the two before optimization changes the signal from which the native update is constructed; operating only on the raw gradient acts before the optimizer geometry has been expressed\. ORACLE instead preserves the realized native update as the reference movement\.

### A\.4Positioning ORACLE

ORACLE operates after the native optimizer has formed its realized step\. It first recovers and certifies the geometry associated with that step, then evaluates whether the movement is locally adverse to a sampled constraint model\. When alignment is required, ORACLE constructs the adjustment in the same optimizer\-relative metric, bounds its authority relative to the native step, and commits it only after nonlinear validation and evaluation on independent support\.

The distinction is therefore not merely*when*constraint information is introduced\. ORACLE separates two forms of structure:*optimizer geometry*, which determines how the native optimizer prefers to move, and*constraint geometry*, which determines how that realized movement affects local constraint violation\. Matrix\-free Jacobian vector and vector Jacobian products allow the latter to be represented without materializing the parameter\-space Jacobian, while native\-relative authority control and held\-out validation regulate whether the resulting alignment is committed\.

ORACLE therefore complements rather than replaces the underlying optimizer or constraint model\. Its contribution is the interface between them: constraints are evaluated against the movement the optimizer actually produced, and any alignment is expressed relative to the geometry of that same movement\.

### A\.5Connecting ORACLE to different fields

ORACLE shifts constraint intervention from the raw gradient to the realized parameter update, computing a matrix\-free, bounded alignment strictly within the native optimizer’s certified geometry\. This post\-optimizer architecture seamlessly integrates into parametric constrained optimization frameworks like DePICT by ensuring decision\-preserving interfaces are not distorted by the underlying preconditioner\. By evaluating constraints at the proposed endpoint rather than the objective[Grover et al\. \(2026b\)](https://arxiv.org/html/2610.09040#bib.bib48), it enables unlearning models[Ranjan et al\. \(2026c\)](https://arxiv.org/html/2610.09040#bib.bib49);[Bourtoule et al\. \(2021\)](https://arxiv.org/html/2610.09040#bib.bib50);[Ranjan et al\. \(2026a\)](https://arxiv.org/html/2610.09040#bib.bib58);[Cao and Yang \(2015\)](https://arxiv.org/html/2610.09040#bib.bib59)to maximize target forgetting while independently treating retention as an endpoint geometric constraint\. Additionally, it provides a critical safeguard for multi\-agent systems and Vision\-Language\-Action \(VLA\) models by explicitly blocking adverse policy steps into unsafe states[Kim et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib51);[Brohan et al\. \(2023\)](https://arxiv.org/html/2610.09040#bib.bib52);[Grover et al\. \(2026a\)](https://arxiv.org/html/2610.09040#bib.bib57)\. Furthermore, because ORACLE dynamically recovers the metric of stateful optimizers before intervening, which is currently being explored in modern safe reinforcement learning[Gu et al\. \(2024\)](https://arxiv.org/html/2610.09040#bib.bib53);[Yang et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib54);[Tayal et al\. \(2026\)](https://arxiv.org/html/2610.09040#bib.bib55);[Ranjan et al\. \(2026b\)](https://arxiv.org/html/2610.09040#bib.bib56), it naturally extends to physics\-informed deep learning tasks relying on advanced second\-order algorithms like SOAP, Muon ensuring boundary and initial conditions are satisfied without disrupting the optimizer’s core convergence properties[Vyas et al\. \(2025\)](https://arxiv.org/html/2610.09040#bib.bib2);[Du and Su \(2026\)](https://arxiv.org/html/2610.09040#bib.bib5)\.

## Appendix BSupporting Theory and Derivations

This appendix provides the technical details supporting Section[3](https://arxiv.org/html/2610.09040#S3)\. We first formalize the finite\-support constraint estimator, then derive the optimizer\-specific geometry adapters used to recoverPtP\_\{t\}, characterize when the PSGD–Kron adapter must abstain, and justify the endpoint signal used to open the ORACLE alignment path\. Subsequent subsections derive the optimizer\-relative alignment, authority bound, and validation mechanisms\.

### B\.1Finite\-Support Constraint Estimator

Fix iterationttand letℱt\\mathcal\{F\}\_\{t\}contain all randomness determined before the construction supportStS\_\{t\}is drawn\. In particular,θt\\theta\_\{t\}, the realized native stepsts\_\{t\}, the proposed endpointθ¯t=θt\+st\\bar\{\\theta\}\_\{t\}=\\theta\_\{t\}\+s\_\{t\}, the family weightsωk,t\\omega\_\{k,t\}, and the sample countsmkm\_\{k\}areℱt\\mathcal\{F\}\_\{t\}\-measurable\. For each constraint familykk, draw

zk,1,…,zk,mk​∼i\.i\.d\.​νk\.z\_\{k,1\},\\ldots,z\_\{k,m\_\{k\}\}\\overset\{\\mathrm\{i\.i\.d\.\}\}\{\\sim\}\\nu\_\{k\}\.
The sampled constraint energy is

ΦSt​\(θ\)=∑k=1Kωk,t​1mk​∑j=1mkrk​\(θ,zk,j\)2\.\\Phi\_\{S\_\{t\}\}\(\\theta\)=\\sum\_\{k=1\}^\{K\}\\omega\_\{k,t\}\\frac\{1\}\{m\_\{k\}\}\\sum\_\{j=1\}^\{m\_\{k\}\}r\_\{k\}\(\\theta;z\_\{k,j\}\)^\{2\}\.\(15\)This is equivalent to the weighted residual form12​rSt​\(θ\)⊤​Wt​rSt​\(θ\)\\frac\{1\}\{2\}r\_\{S\_\{t\}\}\(\\theta\)^\{\\top\}W\_\{t\}r\_\{S\_\{t\}\}\(\\theta\)used in the main text, with each residual from familykkassigned weight2​ωk,t/mk2\\omega\_\{k,t\}/m\_\{k\}\.

For anyℱt\\mathcal\{F\}\_\{t\}\-measurable evaluation pointΘt\\Theta\_\{t\},

𝔼⁡\[ΦSt​\(Θt\)∣ℱt\]\\displaystyle\\mathbb\{E\}\\left\[\\Phi\_\{S\_\{t\}\}\(\\Theta\_\{t\}\)\\mid\\mathcal\{F\}\_\{t\}\\right\]=∑k=1Kωk,t​𝔼z∼νk​\[rk​\(Θt,z\)2\]\\displaystyle=\\sum\_\{k=1\}^\{K\}\\omega\_\{k,t\}\\mathbb\{E\}\_\{z\\sim\\nu\_\{k\}\}\\left\[r\_\{k\}\(\\Theta\_\{t\};z\)^\{2\}\\right\]\(16\)=Φt​\(Θt\)\.\\displaystyle=\\Phi\_\{t\}\(\\Theta\_\{t\}\)\.\(17\)Hence

𝔼⁡\[ΦSt​\(Θt\)∣ℱt\]=Φt​\(Θt\)\\boxed\{\\mathbb\{E\}\\left\[\\Phi\_\{S\_\{t\}\}\(\\Theta\_\{t\}\)\\mid\\mathcal\{F\}\_\{t\}\\right\]=\\Phi\_\{t\}\(\\Theta\_\{t\}\)\}\(18\)and, whenever differentiation and expectation may be interchanged,

𝔼⁡\[∇ΦSt​\(Θt\)∣ℱt\]=∇Φt​\(Θt\)\.\\mathbb\{E\}\\left\[\\nabla\\Phi\_\{S\_\{t\}\}\(\\Theta\_\{t\}\)\\mid\\mathcal\{F\}\_\{t\}\\right\]=\\nabla\\Phi\_\{t\}\(\\Theta\_\{t\}\)\.\(19\)
The ordering used by ORACLE satisfies this condition becauseθ¯t\\bar\{\\theta\}\_\{t\}is fixed beforeStS\_\{t\}is sampled\. For finite stored constraint sets,νk\\nu\_\{k\}denotes the corresponding empirical distribution\.

### B\.2Certified Optimizer Geometry

Letutu\_\{t\}denote the optimizer’s effective direction after its stateful transformation but before application of its local mobility\. ORACLE uses an optimizer\-relative geometry only when it can identify an SPD operatorBtB\_\{t\}satisfying

st=−Bt​ut,Pt=Bt−1≻0,s\_\{t\}=\-B\_\{t\}u\_\{t\},\\qquad P\_\{t\}=B\_\{t\}^\{\-1\}\\succ 0,\(20\)and this representation reconstructs the realized deployed step\.

##### Certification\.

Define the reconstruction defect

et=st\+Bt​ut\.e\_\{t\}=s\_\{t\}\+B\_\{t\}u\_\{t\}\.ORACLE accepts the recovered geometry only if

∥et∥2≤τrel∥st∥2\+τabs,τrel=10−4,τabs=10−9\.\\boxed\{\\lVert e\_\{t\}\\rVert\_\{2\}\\leq\\tau\_\{\\mathrm\{rel\}\}\\lVert s\_\{t\}\\rVert\_\{2\}\+\\tau\_\{\\mathrm\{abs\}\},\\qquad\\tau\_\{\\mathrm\{rel\}\}=10^\{\-4\},\\quad\\tau\_\{\\mathrm\{abs\}\}=10^\{\-9\}\.\}\(21\)If this test or an optimizer\-specific validity condition fails, ORACLE abstains and preserves the Native proposal; it never substitutesPt=IP\_\{t\}=I\.

For nonzero native steps, the corresponding relative reconstruction residual is

εcert,t=∥st\+Bt​ut∥2∥st∥2\.\\varepsilon\_\{\\mathrm\{cert\},t\}=\\frac\{\\lVert s\_\{t\}\+B\_\{t\}u\_\{t\}\\rVert\_\{2\}\}\{\\lVert s\_\{t\}\\rVert\_\{2\}\}\.\(22\)Runtime certification is evaluated on the deployed numerical path; exact algebraic identities alone are not treated as production certification\.

##### SGD\.

For SGD without momentum,

st=−ηt​gt\.s\_\{t\}=\-\\eta\_\{t\}g\_\{t\}\.Thus

ut=gt,Bt=ηt​I,Pt=ηt−1​I\.u\_\{t\}=g\_\{t\},\\qquad B\_\{t\}=\\eta\_\{t\}I,\\qquad P\_\{t\}=\\eta\_\{t\}^\{\-1\}I\.\(23\)Forηt\>0\\eta\_\{t\}\>0, both operators are SPD and equation[20](https://arxiv.org/html/2610.09040#A2.E20)holds exactly\.

##### Adam\.

Letμ^t\\widehat\{\\mu\}\_\{t\}andν^t\\widehat\{\\nu\}\_\{t\}denote the bias\-corrected moments used by the deployed Adam update, and define

Dt=diag⁡\(ν^t\+εroot\+ε\)≻0\.D\_\{t\}=\\operatorname\{diag\}\\left\(\\sqrt\{\\widehat\{\\nu\}\_\{t\}\+\\varepsilon\_\{\\mathrm\{root\}\}\}\+\\varepsilon\\right\)\\succ 0\.Then

st=−ηt​Dt−1​μ^t,s\_\{t\}=\-\\eta\_\{t\}D\_\{t\}^\{\-1\}\\widehat\{\\mu\}\_\{t\},so

ut=μ^t,Bt=ηt​Dt−1,Pt=ηt−1​Dt\.u\_\{t\}=\\widehat\{\\mu\}\_\{t\},\\qquad B\_\{t\}=\\eta\_\{t\}D\_\{t\}^\{\-1\},\\qquad P\_\{t\}=\\eta\_\{t\}^\{\-1\}D\_\{t\}\.\(24\)The adapter follows the numerical stabilizers and state timing of the deployed optimizer exactly\.

##### SOAP\.

LetQtQ\_\{t\}denote the orthogonal basis that generated the realized SOAP step,Dt≻0D\_\{t\}\\succ 0its adaptive denominator,ht\>0h\_\{t\}\>0the scalar step factor, andutsoapu\_\{t\}^\{\\mathrm\{soap\}\}the effective momentum in parameter coordinates\. The realized step is

st=−ht​Qt⊤​Dt−1​Qt​utsoap,s\_\{t\}=\-h\_\{t\}Q\_\{t\}^\{\\top\}D\_\{t\}^\{\-1\}Q\_\{t\}u\_\{t\}^\{\\mathrm\{soap\}\},which gives

ut=utsoap,Bt=ht​Qt⊤​Dt−1​Qt,Pt=ht−1​Qt⊤​Dt​Qt\.u\_\{t\}=u\_\{t\}^\{\\mathrm\{soap\}\},\\qquad B\_\{t\}=h\_\{t\}Q\_\{t\}^\{\\top\}D\_\{t\}^\{\-1\}Q\_\{t\},\\qquad P\_\{t\}=h\_\{t\}^\{\-1\}Q\_\{t\}^\{\\top\}D\_\{t\}Q\_\{t\}\.\(25\)BecauseQtQ\_\{t\}is orthogonal andDt≻0D\_\{t\}\\succ 0, bothBtB\_\{t\}andPtP\_\{t\}are SPD\.

The basis timing is essential\. If SOAP refreshes its basis after constructing the current step, certification must use the basis that actually generatedsts\_\{t\}; using the refreshed basis describes a different operator and may fail the reconstruction test\.

##### PSGD–Kron\.

In the unclipped linear regime, let

Kt\(ℓ\)=⨂iAi,t\(ℓ\),Ai,t\(ℓ\)≻0,K\_\{t\}^\{\(\\ell\)\}=\\bigotimes\_\{i\}A\_\{i,t\}^\{\(\\ell\)\},\\qquad A\_\{i,t\}^\{\(\\ell\)\}\\succ 0,and

Kt=⨁ℓKt\(ℓ\)\.K\_\{t\}=\\bigoplus\_\{\\ell\}K\_\{t\}^\{\(\\ell\)\}\.Kronecker products and direct sums of SPD operators remain SPD\. If

st=−ηt​Kt​utkron,s\_\{t\}=\-\\eta\_\{t\}K\_\{t\}u\_\{t\}^\{\\mathrm\{kron\}\},then

ut=utkron,Bt=ηt​Kt,Pt=ηt−1​Kt−1\.u\_\{t\}=u\_\{t\}^\{\\mathrm\{kron\}\},\\qquad B\_\{t\}=\\eta\_\{t\}K\_\{t\},\\qquad P\_\{t\}=\\eta\_\{t\}^\{\-1\}K\_\{t\}^\{\-1\}\.\(26\)This representation is valid only while the deployed update remains linear in the preconditioned direction\.

### B\.3PSGD–Kron Clipping and Abstention

Let

p=Kt​utkron∈ℝnp=K\_\{t\}u\_\{t\}^\{\\mathrm\{kron\}\}\\in\\mathbb\{R\}^\{n\}denote the linear preconditioned direction before clipping\. The deployed implementation first applies the global normalization

γ⁡\(p\)=p​nmax⁡\(n,∥p∥2\)\\gamma\(p\)=p\\,\\frac\{\\sqrt\{n\}\}\{\\max\(\\sqrt\{n\},\\lVert p\\rVert\_\{2\}\)\}\(27\)and then the elementwise map

h​\(y\)j=min⁡\{1,max⁡\{−1,yj\}\}\.h\(y\)\_\{j\}=\\min\\\{1,\\max\\\{\-1,y\_\{j\}\\\}\\\}\.\(28\)The realized step is therefore

st=−ηt​h​\(γ⁡\(p\)\)\.s\_\{t\}=\-\\eta\_\{t\}h\(\\gamma\(p\)\)\.
A global norm rescaling alone would preserve an SPD linear mobility\. In the deployed rule, however,

∥p∥2\>n⟹∥p∥∞≥∥p∥2n\>1,\\lVert p\\rVert\_\{2\}\>\\sqrt\{n\}\\quad\\Longrightarrow\\quad\\lVert p\\rVert\_\{\\infty\}\\geq\\frac\{\\lVert p\\rVert\_\{2\}\}\{\\sqrt\{n\}\}\>1,and hence

global clipping active⟹elementwise clipping active\.\\boxed\{\\text\{global clipping active\}\\quad\\Longrightarrow\\quad\\text\{elementwise clipping active\}\.\}\(29\)
Once the elementwise clip binds, the optimizer response is coordinatewise nonlinear\. The linear Kronecker operator can no longer represent the mobility used to evaluate arbitrary alignment directions\. ORACLE therefore rejects the Kron adapter on such steps rather than replacing it with an unverified geometry\.

### B\.4Endpoint Constraint Signal

Suppress the iteration index and write

θ=θt,s=st,θ¯=θ\+s,Φ=ΦSt,c=∇Φ​\(θ¯\),P=Pt\.\\theta=\\theta\_\{t\},\\qquad s=s\_\{t\},\\qquad\\bar\{\\theta\}=\\theta\+s,\\qquad\\Phi=\\Phi\_\{S\_\{t\}\},\\qquad c=\\nabla\\Phi\(\\bar\{\\theta\}\),\\qquad P=P\_\{t\}\.
Assume that∇Φ\\nabla\\Phiis Lipschitz along the realized native segment\{θ\+τ​s:τ∈\[0,1\]\}\\\{\\theta\+\\tau s:\\tau\\in\[0,1\]\\\}in the optimizer metric:

∥∇Φ​\(x\)−∇Φ​\(y\)∥P−1≤LΦP​∥x−y∥P\.\\lVert\\nabla\\Phi\(x\)\-\\nabla\\Phi\(y\)\\rVert\_\{P^\{\-1\}\}\\leq L\_\{\\Phi\}^\{P\}\\lVert x\-y\\rVert\_\{P\}\.\(30\)By the fundamental theorem of calculus,

Φ⁡\(θ¯\)−Φ⁡\(θ\)−c⊤​s\\displaystyle\\Phi\(\\bar\{\\theta\}\)\-\\Phi\(\\theta\)\-c^\{\\top\}s=∫01\[∇Φ​\(θ\+τ​s\)−∇Φ​\(θ¯\)\]⊤​s​𝑑τ\.\\displaystyle=\\int\_\{0\}^\{1\}\\left\[\\nabla\\Phi\(\\theta\+\\tau s\)\-\\nabla\\Phi\(\\bar\{\\theta\}\)\\right\]^\{\\top\}s\\,d\\tau\.\(31\)Applying dual Cauchy–Schwarz and equation[30](https://arxiv.org/html/2610.09040#A2.E30)yields

\|ΦSt​\(θ¯t\)−ΦSt​\(θt\)−ct⊤​st\|≤LΦ,tP2​∥st∥Pt2\.\\boxed\{\\left\|\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\)\-\\Phi\_\{S\_\{t\}\}\(\\theta\_\{t\}\)\-c\_\{t\}^\{\\top\}s\_\{t\}\\right\|\\leq\\frac\{L\_\{\\Phi,t\}^\{P\}\}\{2\}\\lVert s\_\{t\}\\rVert\_\{P\_\{t\}\}^\{2\}\.\}\(32\)Thusct⊤​stc\_\{t\}^\{\\top\}s\_\{t\}is the first\-order term associated with the sampled constraint\-energy change along the realized native movement\. Because ORACLE does not estimateLΦ,tPL\_\{\\Phi,t\}^\{P\}at runtime, the deployed rule

ct⊤​st\>0c\_\{t\}^\{\\top\}s\_\{t\}\>0is a first\-order trigger, not an unconditional finite\-step sign certificate\.

##### Why evaluate at the endpoint?

Define

h⁡\(τ\)=Φ⁡\(θ\+τ​s\)\.h\(\\tau\)=\\Phi\(\\theta\+\\tau s\)\.IfΦ\\Phiis convex along the realized native segment, thenh′h^\{\\prime\}is nondecreasing and

h′​\(0\)≤h⁡\(1\)−h⁡\(0\)≤h′​\(1\)\.h^\{\\prime\}\(0\)\\leq h\(1\)\-h\(0\)\\leq h^\{\\prime\}\(1\)\.Equivalently,

∇Φ\(θ\)⊤s≤Φ\(θ\+s\)−Φ\(θ\)≤∇Φ\(θ\+s\)⊤s\.\\boxed\{\\nabla\\Phi\(\\theta\)^\{\\top\}s\\leq\\Phi\(\\theta\+s\)\-\\Phi\(\\theta\)\\leq\\nabla\\Phi\(\\theta\+s\)^\{\\top\}s\.\}\(33\)Therefore,

∇Φ\(θ\+s\)⊤s≤0⟹Φ\(θ\+s\)≤Φ\(θ\)\\nabla\\Phi\(\\theta\+s\)^\{\\top\}s\\leq 0\\quad\\Longrightarrow\\quad\\Phi\(\\theta\+s\)\\leq\\Phi\(\\theta\)\(34\)under segment convexity\.

The endpoint signal is therefore asymmetric\. A non\-positive value provides a one\-sided condition for preserving the Native proposal under the stated assumption, whereas a positive value only opens the alignment path\. The resulting candidate must still satisfy the alignment\-descent, authority, nonlinear\-agreement, and held\-out validation conditions before commitment\.

### B\.5Optimizer\-Relative Alignment

This subsection derives the optimizer\-relative alignment used in Section[3\.3](https://arxiv.org/html/2610.09040#S3.SS3)\. We establish the equivalence between its parameter\- and residual\-space forms, show that both exact and finite\-CG solutions produce first\-order constraint\-descent directions on triggered steps, and give the matrix\-free operator used in implementation\.

##### Parameter\- and residual\-space formulations\.

At the proposed endpointθ¯t\\bar\{\\theta\}\_\{t\}, define

Ct=Wt1/2​Jt,bt=Wt1/2​rt\.C\_\{t\}=W\_\{t\}^\{1/2\}J\_\{t\},\\qquad b\_\{t\}=W\_\{t\}^\{1/2\}r\_\{t\}\.Before authority control, ORACLE computes

d^t=arg⁡mind​\{12​∥bt\+Ct​d∥22\+λt2​∥d∥Pt2\},λt\>0\.\\widehat\{d\}\_\{t\}=\\arg\\min\_\{d\}\\left\\\{\\frac\{1\}\{2\}\\lVert b\_\{t\}\+C\_\{t\}d\\rVert\_\{2\}^\{2\}\+\\frac\{\\lambda\_\{t\}\}\{2\}\\lVert d\\rVert\_\{P\_\{t\}\}^\{2\}\\right\\\},\\qquad\\lambda\_\{t\}\>0\.\(35\)BecausePt≻0P\_\{t\}\\succ 0, the objective is strictly convex\. Its optimality condition is

\(Ct⊤​Ct\+λt​Pt\)​d^t=−Ct⊤​bt=−ct\.\\left\(C\_\{t\}^\{\\top\}C\_\{t\}\+\\lambda\_\{t\}P\_\{t\}\\right\)\\widehat\{d\}\_\{t\}=\-C\_\{t\}^\{\\top\}b\_\{t\}=\-c\_\{t\}\.\(36\)
Define the optimizer\-relative residual Gram operator

Gt=Ct​Pt−1​Ct⊤=Wt1/2​Jt​Pt−1​Jt⊤​Wt1/2⪰0\.G\_\{t\}=C\_\{t\}P\_\{t\}^\{\-1\}C\_\{t\}^\{\\top\}=W\_\{t\}^\{1/2\}J\_\{t\}P\_\{t\}^\{\-1\}J\_\{t\}^\{\\top\}W\_\{t\}^\{1/2\}\\succeq 0\.\(37\)If

\(Gt\+λt​I\)​zt=bt,\(G\_\{t\}\+\\lambda\_\{t\}I\)z\_\{t\}=b\_\{t\},\(38\)then

d^t=−Pt−1​Ct⊤​zt=−Pt−1​Jt⊤​Wt1/2​zt\.\\boxed\{\\widehat\{d\}\_\{t\}=\-P\_\{t\}^\{\-1\}C\_\{t\}^\{\\top\}z\_\{t\}=\-P\_\{t\}^\{\-1\}J\_\{t\}^\{\\top\}W\_\{t\}^\{1/2\}z\_\{t\}\.\}\(39\)To verify the equivalence,

\(Ct⊤​Ct\+λt​Pt\)​\(−Pt−1​Ct⊤​zt\)\\displaystyle\\left\(C\_\{t\}^\{\\top\}C\_\{t\}\+\\lambda\_\{t\}P\_\{t\}\\right\)\\left\(\-P\_\{t\}^\{\-1\}C\_\{t\}^\{\\top\}z\_\{t\}\\right\)=−Ct⊤​\(Ct​Pt−1​Ct⊤\+λt​I\)​zt\\displaystyle\\qquad=\-C\_\{t\}^\{\\top\}\\left\(C\_\{t\}P\_\{t\}^\{\-1\}C\_\{t\}^\{\\top\}\+\\lambda\_\{t\}I\\right\)z\_\{t\}\(40\)=−Ct⊤​\(Gt\+λt​I\)​zt=−Ct⊤​bt=−ct\.\\displaystyle\\qquad=\-C\_\{t\}^\{\\top\}\(G\_\{t\}\+\\lambda\_\{t\}I\)z\_\{t\}=\-C\_\{t\}^\{\\top\}b\_\{t\}=\-c\_\{t\}\.\(41\)Hence equation[39](https://arxiv.org/html/2610.09040#A2.E39)satisfies equation[36](https://arxiv.org/html/2610.09040#A2.E36)\.

##### Exact\-solve descent\.

Sincect=Ct⊤​btc\_\{t\}=C\_\{t\}^\{\\top\}b\_\{t\},

−ct⊤​d^t=bt⊤​Gt​\(Gt\+λt​I\)−1​bt\.\-c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t\}=b\_\{t\}^\{\\top\}G\_\{t\}\(G\_\{t\}\+\\lambda\_\{t\}I\)^\{\-1\}b\_\{t\}\.\(42\)Let

Gt=Q​diag⁡\(σi\)​Q⊤,βi=qi⊤​bt,σi≥0\.G\_\{t\}=Q\\operatorname\{diag\}\(\\sigma\_\{i\}\)Q^\{\\top\},\\qquad\\beta\_\{i\}=q\_\{i\}^\{\\top\}b\_\{t\},\\qquad\\sigma\_\{i\}\\geq 0\.Then

−ct⊤​d^t=∑iσiσi\+λt​βi2\.\-c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t\}=\\sum\_\{i\}\\frac\{\\sigma\_\{i\}\}\{\\sigma\_\{i\}\+\\lambda\_\{t\}\}\\beta\_\{i\}^\{2\}\.\(43\)Moreover,

bt⊤​Gt​bt=ct⊤​Pt−1​ct=∥ct∥Pt−12\.b\_\{t\}^\{\\top\}G\_\{t\}b\_\{t\}=c\_\{t\}^\{\\top\}P\_\{t\}^\{\-1\}c\_\{t\}=\\lVert c\_\{t\}\\rVert\_\{P\_\{t\}^\{\-1\}\}^\{2\}\.\(44\)
On a triggered step,ct⊤​st\>0c\_\{t\}^\{\\top\}s\_\{t\}\>0, soct≠0c\_\{t\}\\neq 0\. Thereforebt⊤​Gt​bt\>0b\_\{t\}^\{\\top\}G\_\{t\}b\_\{t\}\>0, implying thatbtb\_\{t\}has nonzero mass on at least one eigenspace withσi\>0\\sigma\_\{i\}\>0\. Equation[43](https://arxiv.org/html/2610.09040#A2.E43)is therefore strictly positive, and

ct⊤​d^t<0\.\\boxed\{c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t\}<0\.\}\(45\)Thus the exact optimizer\-relative solution is a strict first\-order constraint\-descent direction whenever the alignment path is triggered\.

##### Finite\-CG descent\.

The deployed method solves the residual system with conjugate gradients rather than forming\(Gt\+λt​I\)−1\(G\_\{t\}\+\\lambda\_\{t\}I\)^\{\-1\}\. Let

At=Gt\+λt​I≻0,A\_\{t\}=G\_\{t\}\+\\lambda\_\{t\}I\\succ 0,and letzt,kz\_\{t,k\}be thekk\-th CG iterate forAt​z=btA\_\{t\}z=b\_\{t\}, initialized atzt,0=0z\_\{t,0\}=0\. Define

d^t,k=−Pt−1​Ct⊤​zt,k\.\\widehat\{d\}\_\{t,k\}=\-P\_\{t\}^\{\-1\}C\_\{t\}^\{\\top\}z\_\{t,k\}\.\(46\)
Let

δt,k=bt−At​zt,k\\delta\_\{t,k\}=b\_\{t\}\-A\_\{t\}z\_\{t,k\}denote the CG residual\. Fork≥1k\\geq 1, CG orthogonality gives

δt,k⟂𝒦k​\(At,bt\),\\delta\_\{t,k\}\\perp\\mathcal\{K\}\_\{k\}\(A\_\{t\},b\_\{t\}\),while bothbtb\_\{t\}andzt,kz\_\{t,k\}belong to𝒦k​\(At,bt\)\\mathcal\{K\}\_\{k\}\(A\_\{t\},b\_\{t\}\)\. Hence

bt⊤​At​zt,k\\displaystyle b\_\{t\}^\{\\top\}A\_\{t\}z\_\{t,k\}=bt⊤​\(bt−δt,k\)=∥bt∥22,\\displaystyle=b\_\{t\}^\{\\top\}\(b\_\{t\}\-\\delta\_\{t,k\}\)=\\lVert b\_\{t\}\\rVert\_\{2\}^\{2\},\(47\)bt⊤​zt,k\\displaystyle b\_\{t\}^\{\\top\}z\_\{t,k\}=\(At​zt,k\+δt,k\)⊤​zt,k=∥zt,k∥At2\.\\displaystyle=\(A\_\{t\}z\_\{t,k\}\+\\delta\_\{t,k\}\)^\{\\top\}z\_\{t,k\}=\\lVert z\_\{t,k\}\\rVert\_\{A\_\{t\}\}^\{2\}\.\(48\)
UsingGt=At−λt​IG\_\{t\}=A\_\{t\}\-\\lambda\_\{t\}I,

−ct⊤​d^t,k\\displaystyle\-c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t,k\}=bt⊤​Gt​zt,k\\displaystyle=b\_\{t\}^\{\\top\}G\_\{t\}z\_\{t,k\}\(49\)=∥bt∥22−λt​∥zt,k∥At2\.\\displaystyle=\\lVert b\_\{t\}\\rVert\_\{2\}^\{2\}\-\\lambda\_\{t\}\\lVert z\_\{t,k\}\\rVert\_\{A\_\{t\}\}^\{2\}\.\(50\)
From equation[48](https://arxiv.org/html/2610.09040#A2.E48),

∥zt,k∥At2=⟨At−1​bt,zt,k⟩At\.\\lVert z\_\{t,k\}\\rVert\_\{A\_\{t\}\}^\{2\}=\\left\\langle A\_\{t\}^\{\-1\}b\_\{t\},\\,z\_\{t,k\}\\right\\rangle\_\{A\_\{t\}\}\.Cauchy–Schwarz therefore gives

∥zt,k∥At2≤bt⊤​At−1​bt\.\\lVert z\_\{t,k\}\\rVert\_\{A\_\{t\}\}^\{2\}\\leq b\_\{t\}^\{\\top\}A\_\{t\}^\{\-1\}b\_\{t\}\.\(51\)
In the eigenbasis ofGtG\_\{t\},

λt​bt⊤​At−1​bt=∑iλtσi\+λt​βi2\.\\lambda\_\{t\}b\_\{t\}^\{\\top\}A\_\{t\}^\{\-1\}b\_\{t\}=\\sum\_\{i\}\\frac\{\\lambda\_\{t\}\}\{\\sigma\_\{i\}\+\\lambda\_\{t\}\}\\beta\_\{i\}^\{2\}\.\(52\)The coefficient equals one only whenσi=0\\sigma\_\{i\}=0and is strictly smaller than one whenσi\>0\\sigma\_\{i\}\>0\. By equation[44](https://arxiv.org/html/2610.09040#A2.E44), a triggered step has nonzeroβi\\beta\_\{i\}for at least oneσi\>0\\sigma\_\{i\}\>0\. Hence

λt​bt⊤​At−1​bt<∥bt∥22\.\\lambda\_\{t\}b\_\{t\}^\{\\top\}A\_\{t\}^\{\-1\}b\_\{t\}<\\lVert b\_\{t\}\\rVert\_\{2\}^\{2\}\.\(53\)Combining equation[50](https://arxiv.org/html/2610.09040#A2.E50), equation[51](https://arxiv.org/html/2610.09040#A2.E51), and equation[53](https://arxiv.org/html/2610.09040#A2.E53)yields

ct⊤d^t,k<0,k≥1\.\\boxed\{c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t,k\}<0,\\qquad k\\geq 1\.\}\(54\)
Thus every nonzero finite\-CG iterate is a strict first\-order constraint\-descent direction on a triggered step in exact arithmetic\. The runtime sign test in Algorithm[1](https://arxiv.org/html/2610.09040#alg1)therefore serves as a finite\-precision safeguard rather than an additional modeling assumption\.

##### Relative damping\.

ORACLE scales the absolute damping to the residual geometry:

λt=λrel,t​ϱt,ϱt=bt⊤​Gt​btbt⊤​bt\.\\lambda\_\{t\}=\\lambda\_\{\\mathrm\{rel\},t\}\\varrho\_\{t\},\\qquad\\varrho\_\{t\}=\\frac\{b\_\{t\}^\{\\top\}G\_\{t\}b\_\{t\}\}\{b\_\{t\}^\{\\top\}b\_\{t\}\}\.\(55\)On a triggered step,ct≠0c\_\{t\}\\neq 0, and thereforebt≠0b\_\{t\}\\neq 0, so the quotient is well defined\. Using equation[44](https://arxiv.org/html/2610.09040#A2.E44)and

bt⊤​bt=2​ΦSt​\(θ¯t\),b\_\{t\}^\{\\top\}b\_\{t\}=2\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\),we obtain

ϱt=∥ct∥Pt−122​ΦSt​\(θ¯t\)\.\\boxed\{\\varrho\_\{t\}=\\frac\{\\lVert c\_\{t\}\\rVert\_\{P\_\{t\}^\{\-1\}\}^\{2\}\}\{2\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\)\}\.\}\(56\)Equivalently,ϱt\\varrho\_\{t\}is the Rayleigh quotient ofGtG\_\{t\}along the current weighted\-residual direction\. It therefore provides a residual\-dependent scale for the regularization rather than using a fixed absolute damping across problems and optimizer geometries\. The deployed controller forλrel,t\\lambda\_\{\\mathrm\{rel\},t\}is specified in Section[3](https://arxiv.org/html/2610.09040#S3)\.

##### Matrix\-free application\.

NeitherJtJ\_\{t\}norGtG\_\{t\}is materialized\. For any residual\-space vectorvv,

Gt​v=Wt1/2​Jt​\[Pt−1​\(Jt⊤​Wt1/2​v\)\]\.\\boxed\{G\_\{t\}v=W\_\{t\}^\{1/2\}J\_\{t\}\\left\[P\_\{t\}^\{\-1\}\\left\(J\_\{t\}^\{\\top\}W\_\{t\}^\{1/2\}v\\right\)\\right\]\.\}\(57\)Each application ofGtG\_\{t\}therefore requires one VJP throughJt⊤J\_\{t\}^\{\\top\}, one application of the certified mobilityPt−1P\_\{t\}^\{\-1\}, and one JVP throughJtJ\_\{t\}\. CG can consequently solve equation[38](https://arxiv.org/html/2610.09040#A2.E38)without forming either the residual Jacobian or a parameter\-space matrix\.

### B\.6Authority, Validation, and Exact Native Reduction

This subsection supports the authority and validation mechanisms in Section[3](https://arxiv.org/html/2610.09040#S3)\. We establish the native\-relative authority bound, derive its geometric interpretation, formalize the nonlinear\-agreement and held\-out validation tests, and show that any rejected alignment recovers the exact Native update from the current optimizer state\.

##### Native\-relative alignment authority\.

On a triggered step, define

ht=ct⊤​st\>0,κt=−ct⊤​d^t\>0\.h\_\{t\}=c\_\{t\}^\{\\top\}s\_\{t\}\>0,\\qquad\\kappa\_\{t\}=\-c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t\}\>0\.Herehth\_\{t\}measures the detected first\-order constraint increase along the Native step, whileκt\\kappa\_\{t\}measures the first\-order decrease provided by the optimizer\-relative alignment direction\.

ORACLE defines

αneed,t=htκt\+εauth,αsafe,t=βt​∥st∥Pt∥d^t∥Pt,\\alpha\_\{\\mathrm\{need\},t\}=\\frac\{h\_\{t\}\}\{\\kappa\_\{t\}\+\\varepsilon\_\{\\mathrm\{auth\}\}\},\\qquad\\alpha\_\{\\mathrm\{safe\},t\}=\\frac\{\\beta\_\{t\}\\lVert s\_\{t\}\\rVert\_\{P\_\{t\}\}\}\{\\lVert\\widehat\{d\}\_\{t\}\\rVert\_\{P\_\{t\}\}\},\(58\)withεauth=10−30\\varepsilon\_\{\\mathrm\{auth\}\}=10^\{\-30\}, and sets

αt=min⁡\{αneed,t,αsafe,t\},d~t=αt​d^t\.\\alpha\_\{t\}=\\min\\\{\\alpha\_\{\\mathrm\{need\},t\},\\alpha\_\{\\mathrm\{safe\},t\}\\\},\\qquad\\widetilde\{d\}\_\{t\}=\\alpha\_\{t\}\\widehat\{d\}\_\{t\}\.\(59\)
Without the numerical guard,αneed,t=ht/κt\\alpha\_\{\\mathrm\{need\},t\}=h\_\{t\}/\\kappa\_\{t\}is exactly the scale required to neutralize the detected first\-order constraint signal:

ct⊤​\(st\+αneed,t​d^t\)=ht−αneed,t​κt=0\.c\_\{t\}^\{\\top\}\\left\(s\_\{t\}\+\\alpha\_\{\\mathrm\{need\},t\}\\widehat\{d\}\_\{t\}\\right\)=h\_\{t\}\-\\alpha\_\{\\mathrm\{need\},t\}\\kappa\_\{t\}=0\.The numerical guard keeps the deployed scale infinitesimally below this exact cancellation point\.

Sinceαt≤αsafe,t\\alpha\_\{t\}\\leq\\alpha\_\{\\mathrm\{safe\},t\},

∥d~t∥Pt\\displaystyle\\lVert\\widetilde\{d\}\_\{t\}\\rVert\_\{P\_\{t\}\}=αt​∥d^t∥Pt\\displaystyle=\\alpha\_\{t\}\\lVert\\widehat\{d\}\_\{t\}\\rVert\_\{P\_\{t\}\}\(60\)≤βt​∥st∥Pt\.\\displaystyle\\leq\\beta\_\{t\}\\lVert s\_\{t\}\\rVert\_\{P\_\{t\}\}\.\(61\)Therefore,

∥d~t∥Pt≤βt​∥st∥Pt\.\\boxed\{\\lVert\\widetilde\{d\}\_\{t\}\\rVert\_\{P\_\{t\}\}\\leq\\beta\_\{t\}\\lVert s\_\{t\}\\rVert\_\{P\_\{t\}\}\.\}\(62\)The denominator ofαsafe,t\\alpha\_\{\\mathrm\{safe\},t\}is nonzero on the triggered alignment path becausect⊤​d^t<0c\_\{t\}^\{\\top\}\\widehat\{d\}\_\{t\}<0impliesd^t≠0\\widehat\{d\}\_\{t\}\\neq 0\.

##### Geometric interpretation of the authority bound\.

Let

x=Pt1/2​st,y=Pt1/2​d~t\.x=P\_\{t\}^\{1/2\}s\_\{t\},\\qquad y=P\_\{t\}^\{1/2\}\\widetilde\{d\}\_\{t\}\.Equation[62](https://arxiv.org/html/2610.09040#A2.E62)implies

∥y∥2≤βt​∥x∥2\.\\lVert y\\rVert\_\{2\}\\leq\\beta\_\{t\}\\lVert x\\rVert\_\{2\}\.Moreover, thePtP\_\{t\}\-angle betweensts\_\{t\}andst\+d~ts\_\{t\}\+\\widetilde\{d\}\_\{t\}equals the Euclidean angle betweenxxandx\+yx\+y\.

Normalize∥x∥2=1\\lVert x\\rVert\_\{2\}=1and decompose

y=a​x\+b​q,q⊤​x=0,∥q∥2=1,a2\+b2≤βt2\.y=ax\+bq,\\qquad q^\{\\top\}x=0,\\qquad\\lVert q\\rVert\_\{2\}=1,\\qquad a^\{2\}\+b^\{2\}\\leq\\beta\_\{t\}^\{2\}\.If

ϑ=∠⁡\(x,x\+y\),\\vartheta=\\angle\(x,x\+y\),then

sin2⁡ϑ=b2\(1\+a\)2\+b2\.\\sin^\{2\}\\vartheta=\\frac\{b^\{2\}\}\{\(1\+a\)^\{2\}\+b^\{2\}\}\.\(63\)For fixedaa, this quantity increases withb2b^\{2\}, so its maximum lies ona2\+b2=βt2a^\{2\}\+b^\{2\}=\\beta\_\{t\}^\{2\}\. Substitutingb2=βt2−a2b^\{2\}=\\beta\_\{t\}^\{2\}\-a^\{2\}gives

f⁡\(a\)=βt2−a21\+2​a\+βt2\.f\(a\)=\\frac\{\\beta\_\{t\}^\{2\}\-a^\{2\}\}\{1\+2a\+\\beta\_\{t\}^\{2\}\}\.The interior maximizer isa=−βt2a=\-\\beta\_\{t\}^\{2\}, for which

f⁡\(−βt2\)=βt2\.f\(\-\\beta\_\{t\}^\{2\}\)=\\beta\_\{t\}^\{2\}\.Hence, for0≤βt<10\\leq\\beta\_\{t\}<1,

∠Pt​\(st,st\+d~t\)≤arcsin⁡\(βt\)\.\\boxed\{\\angle\_\{P\_\{t\}\}\\left\(s\_\{t\},\\,s\_\{t\}\+\\widetilde\{d\}\_\{t\}\\right\)\\leq\\arcsin\(\\beta\_\{t\}\)\.\}\(64\)
The bound is sharp\. Equality is attained in transformed coordinates by

y=−βt2​x\+βt​1−βt2​q,q⟂x\.y=\-\\beta\_\{t\}^\{2\}x\+\\beta\_\{t\}\\sqrt\{1\-\\beta\_\{t\}^\{2\}\}\\,q,\\qquad q\\perp x\.Thusβt\\beta\_\{t\}has a direct geometric interpretation: it limits how far the aligned movement may rotate away from the direction selected by the Native optimizer, measured in that optimizer’s own metric\.

##### Nonlinear agreement\.

The alignment direction is constructed from a local linearization, so ORACLE checks whether the predicted reduction agrees with the true nonlinear constraint energy before proceeding\.

On the same construction supportStS\_\{t\}, define

Predt\\displaystyle\\mathrm\{Pred\}\_\{t\}=ΦSt​\(θ¯t\)−12​‖bt\+Ct​d~t‖22,\\displaystyle=\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\)\-\\frac\{1\}\{2\}\\left\\lVert b\_\{t\}\+C\_\{t\}\\widetilde\{d\}\_\{t\}\\right\\rVert\_\{2\}^\{2\},\(65\)Aredt\\displaystyle\\mathrm\{Ared\}\_\{t\}=ΦSt​\(θ¯t\)−ΦSt​\(θ¯t\+d~t\)\.\\displaystyle=\\Phi\_\{S\_\{t\}\}\(\\bar\{\\theta\}\_\{t\}\)\-\\Phi\_\{S\_\{t\}\}\\left\(\\bar\{\\theta\}\_\{t\}\+\\widetilde\{d\}\_\{t\}\\right\)\.\(66\)Recall thatCt=Wt1/2​JtC\_\{t\}=W\_\{t\}^\{1/2\}J\_\{t\}andbt=Wt1/2​rtb\_\{t\}=W\_\{t\}^\{1/2\}r\_\{t\}\.

The candidate proceeds only if

Predt\>0,Aredt\>0,ρt=AredtPredt≥ρmin\.\\mathrm\{Pred\}\_\{t\}\>0,\\qquad\\mathrm\{Ared\}\_\{t\}\>0,\\qquad\\rho\_\{t\}=\\frac\{\\mathrm\{Ared\}\_\{t\}\}\{\\mathrm\{Pred\}\_\{t\}\}\\geq\\rho\_\{\\min\}\.\(67\)BecauseStS\_\{t\}is used both to construct and evaluate the candidate, this test measures agreement between the local model and the nonlinear constraints; it does not provide independent evidence that the alignment improves unseen constraint samples\.

##### Independent held\-out validation\.

ORACLE therefore fixesd~t\\widetilde\{d\}\_\{t\}before drawing an independent audit supportSt′S^\{\\prime\}\_\{t\}\. Let𝒢t\\mathcal\{G\}\_\{t\}contain all information available after the candidate is fixed but beforeSt′S^\{\\prime\}\_\{t\}is sampled\. Conditional on𝒢t\\mathcal\{G\}\_\{t\}, the audit samples are independent draws from their prescribed constraint\-family distributions\.

For familykk, define

Xk,t​\(z\)=ωk,t​\[rk​\(θ¯t\+d~t,z\)2−rk​\(θ¯t,z\)2\]\.X\_\{k,t\}\(z\)=\\omega\_\{k,t\}\\left\[r\_\{k\}\(\\bar\{\\theta\}\_\{t\}\+\\widetilde\{d\}\_\{t\};z\)^\{2\}\-r\_\{k\}\(\\bar\{\\theta\}\_\{t\};z\)^\{2\}\\right\]\.\(68\)Withmk′m^\{\\prime\}\_\{k\}audit samples from familykk, the empirical change is

Δ^taudit=∑k=1K1mk′​∑j=1mk′Xk,t​\(zk,j′\)\.\\widehat\{\\Delta\}^\{\\,\\mathrm\{audit\}\}\_\{t\}=\\sum\_\{k=1\}^\{K\}\\frac\{1\}\{m^\{\\prime\}\_\{k\}\}\\sum\_\{j=1\}^\{m^\{\\prime\}\_\{k\}\}X\_\{k,t\}\(z^\{\\prime\}\_\{k,j\}\)\.\(69\)The corresponding population change is

Δtpop\\displaystyle\\Delta\_\{t\}^\{\\mathrm\{pop\}\}=∑k=1K𝔼⁡\[Xk,t​\(z\)∣𝒢t\]\\displaystyle=\\sum\_\{k=1\}^\{K\}\\mathbb\{E\}\\left\[X\_\{k,t\}\(z\)\\mid\\mathcal\{G\}\_\{t\}\\right\]=Φt​\(θ¯t\+d~t\)−Φt​\(θ¯t\)\.\\displaystyle=\\Phi\_\{t\}\\left\(\\bar\{\\theta\}\_\{t\}\+\\widetilde\{d\}\_\{t\}\\right\)\-\\Phi\_\{t\}\(\\bar\{\\theta\}\_\{t\}\)\.\(70\)Therefore,

𝔼⁡\[Δ^taudit∣𝒢t\]=Δtpop\.\\boxed\{\\mathbb\{E\}\\left\[\\widehat\{\\Delta\}^\{\\,\\mathrm\{audit\}\}\_\{t\}\\mid\\mathcal\{G\}\_\{t\}\\right\]=\\Delta\_\{t\}^\{\\mathrm\{pop\}\}\.\}\(71\)
Assume that, conditional on𝒢t\\mathcal\{G\}\_\{t\}, eachXk,t​\(z\)X\_\{k,t\}\(z\)lies in an interval of widthRk,tR\_\{k,t\}\. Hoeffding’s inequality then gives, for anyu\>0u\>0,

Pr⁡\[Δ^taudit−Δtpop≤−u\|𝒢t\]≤exp⁡\(−2​u2∑k=1KRk,t2/mk′\)\.\\Pr\\\!\\left\[\\widehat\{\\Delta\}^\{\\,\\mathrm\{audit\}\}\_\{t\}\-\\Delta\_\{t\}^\{\\mathrm\{pop\}\}\\leq\-u\\,\\middle\|\\,\\mathcal\{G\}\_\{t\}\\right\]\\leq\\exp\\\!\\left\(\-\\frac\{2u^\{2\}\}\{\\displaystyle\\sum\_\{k=1\}^\{K\}R\_\{k,t\}^\{2\}/m^\{\\prime\}\_\{k\}\}\\right\)\.\(72\)
The deployed zero\-margin audit accepts when

Δ^taudit≤0\.\\widehat\{\\Delta\}^\{\\,\\mathrm\{audit\}\}\_\{t\}\\leq 0\.Suppose instead that the true population constraint energy increases by at leastuu, so thatΔtpop≥u\\Delta\_\{t\}^\{\\mathrm\{pop\}\}\\geq u\. If the audit nevertheless accepts, then

Δ^taudit−Δtpop≤−u\.\\widehat\{\\Delta\}^\{\\,\\mathrm\{audit\}\}\_\{t\}\-\\Delta\_\{t\}^\{\\mathrm\{pop\}\}\\leq\-u\.Hence

Δtpop≥u⟹Pr⁡\[Δ^taudit≤0\|𝒢t\]≤exp⁡\(−2​u2∑k=1KRk,t2/mk′\)\.\\boxed\{\\Delta\_\{t\}^\{\\mathrm\{pop\}\}\\geq u\\Longrightarrow\\Pr\\\!\\left\[\\widehat\{\\Delta\}^\{\\,\\mathrm\{audit\}\}\_\{t\}\\leq 0\\,\\middle\|\\,\\mathcal\{G\}\_\{t\}\\right\]\\leq\\exp\\\!\\left\(\-\\frac\{2u^\{2\}\}\{\\displaystyle\\sum\_\{k=1\}^\{K\}R\_\{k,t\}^\{2\}/m^\{\\prime\}\_\{k\}\}\\right\)\.\}\(73\)
Setting the right\-hand side toδ\\deltagives the finite\-sample resolution

uδ=log⁡\(1/δ\)2​∑k=1KRk,t2mk′\.\\boxed\{u\_\{\\delta\}=\\sqrt\{\\frac\{\\log\(1/\\delta\)\}\{2\}\\sum\_\{k=1\}^\{K\}\\frac\{R\_\{k,t\}^\{2\}\}\{m^\{\\prime\}\_\{k\}\}\}\.\}\(74\)Thus, conditional on the fixed candidate, a population increase of at leastuδu\_\{\\delta\}passes the zero\-margin audit with probability at mostδ\\delta\. For comparable range widths and balanced family counts,

uδ=O⁡\(log⁡\(1/δ\)maudit\)\.u\_\{\\delta\}=O\\\!\\left\(\\sqrt\{\\frac\{\\log\(1/\\delta\)\}\{m\_\{\\mathrm\{audit\}\}\}\}\\right\)\.
This is a finite\-sample statement rather than an exact sign guarantee: population changes below this resolution may not be distinguished reliably by the zero\-margin audit\.

##### Scope of the held\-out guarantee\.

The preceding result applies to one fixed alignment candidate relative to its current proposed endpoint\. After an accepted alignment, the parameters change, and so do subsequent gradients, Native proposals, sampled supports, and potentially the constraint\-family weights\. The single\-step audit guarantee therefore does not telescope into a terminal guarantee comparing an entire ORACLE trajectory with an independently trained Native trajectory\.

##### Exact one\-step reduction to Native\.

LetCertt\\mathrm\{Cert\}\_\{t\},Trigt\\mathrm\{Trig\}\_\{t\},Desct\\mathrm\{Desc\}\_\{t\},NLt\\mathrm\{NL\}\_\{t\}, andAuditt\\mathrm\{Audit\}\_\{t\}denote successful geometry certification, trigger, alignment\-descent, nonlinear\-agreement, and held\-out validation events, respectively\. Define

Acct=Certt∧Trigt∧Desct∧NLt∧Auditt\.\\mathrm\{Acc\}\_\{t\}=\\mathrm\{Cert\}\_\{t\}\\land\\mathrm\{Trig\}\_\{t\}\\land\\mathrm\{Desc\}\_\{t\}\\land\\mathrm\{NL\}\_\{t\}\\land\\mathrm\{Audit\}\_\{t\}\.\(75\)ORACLE commits

dt=d~t​𝟏​\{Acct\}\.d\_\{t\}=\\widetilde\{d\}\_\{t\}\\mathbf\{1\}\\\{\\mathrm\{Acc\}\_\{t\}\\\}\.\(76\)If any gate fails,dt=0d\_\{t\}=0, and therefore

θt\+1=θt\+st\.\\theta\_\{t\+1\}=\\theta\_\{t\}\+s\_\{t\}\.Because ORACLE never writes its alignment adjustment into the Native optimizer state,

ξt\+1=𝒰⁡\(gt,ξt\)\.\\xi\_\{t\+1\}=\\mathcal\{U\}\(g\_\{t\},\\xi\_\{t\}\)\.Consequently,

¬Acct⟹\(θt\+1,ξt\+1\)=\(θt\+st,𝒰⁡\(gt,ξt\)\)\.\\boxed\{\\neg\\mathrm\{Acc\}\_\{t\}\\Longrightarrow\(\\theta\_\{t\+1\},\\xi\_\{t\+1\}\)=\\left\(\\theta\_\{t\}\+s\_\{t\},\\,\\mathcal\{U\}\(g\_\{t\},\\xi\_\{t\}\)\\right\)\.\}\(77\)
Thus, from the current state\(θt,ξt\)\(\\theta\_\{t\},\\xi\_\{t\}\), rejection of any ORACLE gate recovers exactly the update that the Native optimizer had already proposed\. This is a one\-step identity\. It does not imply equality with an independently run Native trajectory after an earlier accepted ORACLE alignment, because an accepted alignment changes the parameters from which subsequent gradients and Native updates are generated\.

## Appendix CExperimental Details

This appendix provides the experimental specification underlying Section[4](https://arxiv.org/html/2610.09040#S4)\. The central claim of ORACLE is that constraint alignment should be defined relative to the movement produced by the native optimizer\. We test this claim from breadth to mechanism\. Q1 varies optimizer geometry; Q2 varies model formulation; Q3 quantifies the cost and solution\-level benefit of alignment; Q4 compares objective\-, gradient\-, and post\-optimizer intervention; and Q5 isolates ORACLE’s geometry, authority, and validation mechanisms\. We report architectures, optimizer settings, training and sampling protocols, ORACLE hyperparameters, controlled baselines, and compute\-accounting conventions below\.

### C\.1Reference Solution Generation

Parameter settings and numerical configurations used to generate the reference solutions for the PDE benchmarks as given in Table[1](https://arxiv.org/html/2610.09040#A3.T1)\.

Table 1:Parameter settings and numerical configurations used to generate the reference solutions for the eight PDE benchmarks\. Resolutions are those of the shipped reference arrays; the evaluation pipeline discards the final temporal row for the windowed systems \(Kuramoto–Sivashinsky, Gray–Scott\), so the evaluated horizons are250250and100100respectively\.*Segmentation*gives the training decomposition: single\-segment problems are trained over one contiguous horizon, windowed problems over ten sequential time windows, and the lid\-driven cavity over a five\-stage Reynolds\-number curriculum\.
### C\.2Common Evaluation Protocol

ORACLE is evaluated as a wrapper around a Native optimizer rather than as a replacement optimizer\. Within each paired Native–ORACLE comparison, we hold the architecture, native objective, optimizer configuration, learning\-rate schedule, sampling pipeline, initialization or shared checkpoint, and optimization budget fixed\. ORACLE acts only after the Native optimizer has formed its realized parameter update\.

Unless otherwise stated, comparisons are matched by optimizer steps rather than wall\-clock time\. The primary accuracy metric is relative solution error,

relL2=‖uθ−u⋆‖2‖u⋆‖2,\\mathrm\{relL2\}=\\frac\{\\\|u\_\{\\theta\}\-u^\{\\star\}\\\|\_\{2\}\}\{\\\|u^\{\\star\}\\\|\_\{2\}\},\(78\)whereu⋆u^\{\\star\}denotes the reference solution\. For paired comparisons we also report

Δrel=100​\(1−relL2methodrelL2Native\)%,\\Delta\_\{\\mathrm\{rel\}\}=100\\left\(1\-\\frac\{\\mathrm\{relL2\}\_\{\\mathrm\{method\}\}\}\{\\mathrm\{relL2\}\_\{\\mathrm\{Native\}\}\}\\right\)\\%,\(79\)so that positive values indicate lower error than the corresponding Native configuration\.

Table 2:Overview of the experimental design\.
### C\.3Q1: Variation in Native Optimizer Geometry

Q1 fixes the model to PirateNet and varies the Native optimizer among SGD, Adam, SOAP, and PSGD–Kron\. These optimizers span isotropic Euclidean, diagonal\-adaptive, and structured preconditioned update geometries\.

Each optimizer is evaluated on Burgers, KdV, Wave, Allen–Cahn, lid\-driven cavity, Kuramoto–Sivashinsky, Gray–Scott, and Kolmogorov flow, giving a nominal8×4=328\\times 4=32PDE–optimizer grid\. Every configuration uses a100,000100\{,\}000\-step training horizon\. ORACLE becomes active at the fork listed in Table[7](https://arxiv.org/html/2610.09040#A3.T7); the trajectory before that point is Native\.

Table 3:Native optimizer families used in Q1\.
### C\.4Q2: Variation in Model Formulation

Q2 changes the model formulation while retaining Adam and SOAP as two qualitatively different Native optimizer geometries\. We evaluate Pure MLP, Simple PINN, and PirateNet on Wave, Burgers, Allen–Cahn, and KdV, producing

4​PDEs×3​model formulations×2​optimizers=244\\;\\text\{PDEs\}\\times 3\\;\\text\{model formulations\}\\times 2\\;\\text\{optimizers\}=24paired configurations, each with a fixed10,00010\{,\}000\-step training horizon\.

Pure MLP and Simple PINN deliberately share the same4×2564\\times 256tanh\\tanhbackbone\. Pure MLP optimizes only the prescribed initial\- and boundary\-condition terms, whereas Simple PINN additionally includes the PDE residual in the Native objective\. PirateNet changes both the representation and its physics\-specific inductive structure through Fourier features, random weight factorization, and gated residual blocks\.

### C\.5Network Architecture Details

Table[4](https://arxiv.org/html/2610.09040#A3.T4)specifies the three model formulations used in Q2\. The same PirateNet configuration is used throughout Q1\.

Table 4:Architecture configurations used in Q2\. All models use hidden width256256\.aWave and Burgers use 2\-D inputs\.bAllen–Cahn and KdV use a non\-trainable periodic embedding that lifts the input to 3\-D without introducing additional parameters\.

The88\-PDE×\\times44\-optimizer Q1 study uses PirateNet exclusively\. For the executed PirateNet configurations, the experiment path does not apply the optionaluse\_pi\_initinitialization flag\.

### C\.6Native Optimizer Configuration

Native and Native\+ORACLE use identical learning rates, schedules, momentum parameters, numerical stabilizers, and preconditioner\-update rules\. Table[5](https://arxiv.org/html/2610.09040#A3.T5)reports the Q1 configurations\.

Table 5:Native optimizer configurations used in Q1\.a10−410^\{\-4\}for Burgers and lid\-driven cavity;10−310^\{\-3\}for all remaining PDEs\.bMomentum coefficient; PSGD–Kron does not use an Adam\-style second\-moment coefficient\.

#### C\.6\.1Learning\-Rate Schedule

Within each Q1 PDE, all four optimizers use the same learning\-rate schedule\. The learning rate increases linearly from zero to its configured initial value during warmup and then follows exponential decay,

ηt=ExpDecay⁡\(ηbase,rate=0\.9,decay steps\),\\eta\_\{t\}=\\operatorname\{ExpDecay\}\\left\(\\eta\_\{\\mathrm\{base\}\},\\text\{rate\}=0\.9,\\text\{decay steps\}\\right\),\(80\)with no learning\-rate floor\. The schedule is indexed by the global training step and does not restart at curriculum or time\-window boundaries\.

Table 6:Learning\-rate schedule parameters in Q1\.
#### C\.6\.2Geometry Certification for PSGD–Kron

ORACLE uses an optimizer\-relative metric only when the realized Native update admits the certified linear representation derived in Appendix[B\.2](https://arxiv.org/html/2610.09040#A2.SS2)\. The deployed PSGD–Kron update applies global\-norm and elementwise clipping to the preconditioned direction\. When elementwise clipping is active, the resulting update is no longer represented by the linear Kronecker mobility used by ORACLE\. The adapter is therefore treated as uncertified and ORACLE preserves the Native proposal rather than falling back toPt=IP\_\{t\}=I\.

The algebraic reason for this abstention rule is given in Appendix[B\.3](https://arxiv.org/html/2610.09040#A2.SS3)\. Empirical certification statistics are reported with the corresponding Q1 results rather than repeated here\.

### C\.7PDE Training, Sampling, and Evaluation

Table[7](https://arxiv.org/html/2610.09040#A3.T7)gives the Q1 training and evaluation settings\. Windowed PDEs use a time\-marching curriculum with a fresh model for each window; the lid\-driven cavity uses a Reynolds\-number curriculum while retaining state\. “Interior” denotes residual collocation points sampled per optimization step\.

Table 7:Q1 training and evaluation settings across PDE benchmarks\.Note\.Interior residual points are resampled uniformly at each step\. For windowed problems, sampling is restricted to the active space–time window\. Boundary and initial\-condition data are fixed where applicable, except for Kolmogorov flow\.aSampled with replacement from the65,53665\{,\}536\-point reference grid\.

### C\.8Q4: Matched Intervention Comparison

Q4 trades breadth for control\. For Allen–Cahn and Wave, every method begins from the same exact5,0005\{,\}000\-step Native checkpoint, including model parameters and optimizer state\. Each arm then receives exactly5,0005\{,\}000additional method\-active optimization steps and is evaluated at the common global endpoint of step10,00010\{,\}000\.

The comparison is therefore matched in initialization, optimizer state, sampling protocol, and optimization horizon\. Methods differ only in how constraint information enters the optimization pipeline\.

Table 8:Constraint\-handling methods used in Q4\.Objective\-level methods alter the objective before differentiation; gradient\-level methods alter the signal supplied to the Native optimizer; and post\-optimizer methods act after the optimizer has formed a realized parameter displacement\.

### C\.9ORACLE Experimental Configuration

The mathematical construction of ORACLE is given in Appendix[B](https://arxiv.org/html/2610.09040#A2)\. Here we report only the experimental settings used by the reference configuration\. The alignment system is solved matrix\-free with1616CG iterations using a construction support of10241024residuals\. A candidate that passes the nonlinear\-agreement test is evaluated on an independent10241024\-point audit support before commitment\.

Table 9:Reference ORACLE hyperparameters\.QuantitySymbolValue*Residual\-space alignment*Construction support sizemm10241024Held\-out audit sizemauditm\_\{\\mathrm\{audit\}\}10241024CG iterationsKK1616Relative LM damping, initialλrel\(0\)\\lambda\_\{\\mathrm\{rel\}\}^\{\(0\)\}10−210^\{\-2\}Relative LM damping bounds\[λmin,λmax\]\[\\lambda\_\{\\min\},\\lambda\_\{\\max\}\]\[10−4,10−1\]\[10^\{\-4\},10^\{\-1\}\]Damping grow / shrink–4\.04\.0/0\.50\.5*Alignment authority*Initial authority fractionβ0\\beta\_\{0\}0\.100\.10Authority bounds\[βmin,βmax\]\[\\beta\_\{\\min\},\\beta\_\{\\max\}\]\[0\.01,0\.25\]\[0\.01,0\.25\]Agreement thresholdρmin\\rho\_\{\\min\}0\.100\.10Shrink threshold / factorρshrink\\rho\_\{\\mathrm\{shrink\}\}0\.250\.25/×0\.5\\times 0\.5Expand threshold / factorρexpand\\rho\_\{\\mathrm\{expand\}\}0\.750\.75/×1\.25\\times 1\.25Numerical floorεauth\\varepsilon\_\{\\mathrm\{auth\}\}10−3010^\{\-30\}Archived experiment directories may refer to this reference configuration asH2\-S; throughout the paper we use the final nameORACLE\.

### C\.10Q5: Controlled ORACLE Ablations

Q5 isolates the mechanisms used by the reference ORACLE configuration\. Each ablation inherits the same Native checkpoint and experimental protocol and changes only the component listed in Table[10](https://arxiv.org/html/2610.09040#A3.T10)\.

Table 10:Controlled ORACLE ablations used in Q5\.
### C\.11Mechanism Diagnostics

The Q5 diagnostics characterize the internal behavior of ORACLE but are not used as substitutes for endpoint solution accuracy\.

##### Authority\-bound activation\.

For each accepted alignment, we record whether the native\-relative authority limit determines the applied scale,

Iauth,t=𝟏\[αsafe,t≤αneed,t\]\.I\_\{\\mathrm\{auth\},t\}=\\mathbf\{1\}\\left\[\\alpha\_\{\\mathrm\{safe\},t\}\\leq\\alpha\_\{\\mathrm\{need\},t\}\\right\]\.\(81\)The reported authority\-bound rate is the fraction of accepted alignments for whichIauth,t=1I\_\{\\mathrm\{auth\},t\}=1\.

##### Held\-out rejection\.

The number of fixed alignment candidates rejected by the independent audit is

Naudit=∑t𝟏\[Auditt=reject\]\.N\_\{\\mathrm\{audit\}\}=\\sum\_\{t\}\\mathbf\{1\}\[\\mathrm\{Audit\}\_\{t\}=\\mathrm\{reject\}\]\.\(82\)
These quantities measure different mechanisms: authority activation describes how often the candidate is limited relative to the Native movement, whereas audit rejection records how often an already constructed candidate fails to generalize to fresh constraint support\.

## Appendix DComplete Experimental Results

### D\.1Q1: Optimizer Geometry

Table[11](https://arxiv.org/html/2610.09040#A4.T11)reports the complete Q1 endpoint results\. PirateNet is fixed throughout, while the Native optimizer varies among SGD, Adam, SOAP, and PSGD–Kron\. Within each Native–ORACLE pair, the architecture, objective, optimizer configuration, learning\-rate schedule, sampling pipeline, and training budget are unchanged\.

Of the3232nominal PDE–optimizer configurations,3232produce valid scientific endpoints\.

Table 11:Complete Q1 results across optimizer geometries\.Endpoint relativeℓ2\\ell\_\{2\}error across eight PDE benchmarks for four Native optimizers, with and without ORACLE; lower is better\.Boldmarks the better endpoint for configurations classified as an improvement or degradation, while unbolded pairs denote matched outcomes under the Q1 reporting criterion\. Each entry is mean, the standard deviation across three independent runs < 5% for all reported entries\.PirateNet is used for every Q1 PDE–optimizer pair\. Full architecture, optimizer, and PDE configurations are reported in Tables[4](https://arxiv.org/html/2610.09040#A3.T4),[5](https://arxiv.org/html/2610.09040#A3.T5), and[7](https://arxiv.org/html/2610.09040#A3.T7)\.

##### Outcome summary\.

Under the Q1 reporting criterion, ORACLE improves2323configurations, matches Native in77, and degrades22, corresponding to improvement or matching behavior in30/3230/32valid PDE\-optimizer pairs\. Table[12](https://arxiv.org/html/2610.09040#A4.T12)gives the breakdown by Native optimizer\.

Table 12:Q1 outcome summary by Native optimizer\.The complete results show that the effect is not confined to one update geometry\. Improvements occur under isotropic SGD, diagonal\-adaptive Adam, and the structured preconditioned geometries of SOAP and PSGD–Kron\. The two degradations are retained as measured outcomes rather than excluded\.

### D\.2Q2: Model Formulation

Table[13](https://arxiv.org/html/2610.09040#A4.T13)reports the complete Q2 results for Pure MLP, Simple PINN, and PirateNet under Adam and SOAP\. Pure MLP and Simple PINN share the same4×2564\\times 256tanh\\tanhbackbone but differ in whether the PDE residual participates in the Native objective; PirateNet additionally changes the architecture and physics\-specific inductive structure\. The study contains

4​PDEs×3​model formulations×2​optimizers=244\\;\\text\{PDEs\}\\times 3\\;\\text\{model formulations\}\\times 2\\;\\text\{optimizers\}=24paired Native–ORACLE configurations\.

Table 13:Complete Q2 results across model formulations\.Endpoint relativeℓ2\\ell\_\{2\}error for Pure MLP, Simple PINN, and PirateNet under Adam and SOAP; lower is better\. Improvement is100​\(1−relL2ORACLE/relL2Native\)100\(1\-\\mathrm\{relL2\}\_\{\\mathrm\{ORACLE\}\}/\\mathrm\{relL2\}\_\{\\mathrm\{Native\}\}\)\.*Trigger*is the fraction of optimization steps on which the realized Native movement opens the ORACLE alignment path;*Accept*is the fraction of steps on which an aligned candidate is ultimately committed\. Each entry is mean, the std\-dev across three independent runs < 7% for all reported entries\.ORACLE lowers endpoint rel\-L2 across the large majority of configurations, with improvements spanning all three model formulations and both Native optimizers\. The largest gains include60\.6%60\.6\\%for Simple PINN\+Adam on Burgers,76\.1%76\.1\\%for PirateNet\+Adam on Burgers, and90\.6%90\.6\\%for PirateNet\+SOAP on KdV\.

The strongest degradations occur for Pure MLP on Allen–Cahn under both Adam and SOAP\. Because Pure MLP and Simple PINN share the same backbone, whereas Simple PINN additionally includes the PDE residual in its Native objective, this comparison shows that ORACLE does not override the underlying learning formulation\.

Trigger and acceptance rates are reported as mechanism diagnostics rather than performance metrics\. A high trigger rate does not imply a large endpoint gain, and triggering does not imply automatic commitment\. For example, Simple PINN\+SOAP on Burgers opens the alignment path on97%97\\%of steps but commits an aligned candidate on only32%32\\%\.

### D\.3Detailed Cost–Benefit Analysis

Figure[5](https://arxiv.org/html/2610.09040#S4.F5)summarizes the cost and solution\-level benefit of ORACLE across the model\-formulation study\. Here we define each quantity used in that figure and report how the configuration\-level measurements are aggregated\.

##### Evaluation setting\.

Let

denote one experimental configuration, whereaais the model formulation,ppis the PDE, andoois the native optimizer\. We evaluate three model formulations,

a∈\{Pure MLP,Simple PINN,PirateNet\},a\\in\\\{\\text\{Pure MLP\},\\text\{Simple PINN\},\\text\{PirateNet\}\\\},four PDEs,

p∈\{Wave,Burgers,Allen–Cahn,KdV\},p\\in\\\{\\text\{Wave\},\\text\{Burgers\},\\text\{Allen\-\-Cahn\},\\text\{KdV\}\\\},and two native optimizers,

o∈\{Adam,SOAP\}\.o\\in\\\{\\text\{Adam\},\\text\{SOAP\}\\\}\.Thus, each architecture contains

\|𝒞a\|=4×2=8\|\\mathcal\{C\}\_\{a\}\|=4\\times 2=8matched Native–ORACLE comparisons, for2424configurations overall\. Native and ORACLE use the same model formulation, PDE, optimizer, training budget, and evaluation grid within each configuration; the only difference is the ORACLE intervention\.

Letu^cN\\hat\{u\}^\{N\}\_\{c\}andu^cO\\hat\{u\}^\{O\}\_\{c\}denote the solutions produced by Native and ORACLE, respectively, and letuc⋆u\_\{c\}^\{\\star\}denote the reference solution\.

#### D\.3\.1Normalized Endpoint Performance

For each configuration, we first compute the relativeℓ2\\ell\_\{2\}solution error at the final checkpoint\. For methodm∈\{N,O\}m\\in\\\{N,O\\\},

Ecm=‖u^cm−uc⋆‖2‖uc⋆‖2\.E\_\{c\}^\{m\}=\\frac\{\\left\\\|\\hat\{u\}^\{m\}\_\{c\}\-u^\{\\star\}\_\{c\}\\right\\\|\_\{2\}\}\{\\left\\\|u^\{\\star\}\_\{c\}\\right\\\|\_\{2\}\}\.\(83\)For a discrete evaluation grid𝒵c=\{zq\}q=1Nc\\mathcal\{Z\}\_\{c\}=\\\{z\_\{q\}\\\}\_\{q=1\}^\{N\_\{c\}\}, this is

Ecm=∑q=1Nc‖u^cm​\(zq\)−uc⋆​\(zq\)‖22∑q=1Nc‖uc⋆​\(zq\)‖22\.E\_\{c\}^\{m\}=\\sqrt\{\\frac\{\\sum\_\{q=1\}^\{N\_\{c\}\}\\left\\\|\\hat\{u\}^\{m\}\_\{c\}\(z\_\{q\}\)\-u^\{\\star\}\_\{c\}\(z\_\{q\}\)\\right\\\|\_\{2\}^\{2\}\}\{\\sum\_\{q=1\}^\{N\_\{c\}\}\\left\\\|u^\{\\star\}\_\{c\}\(z\_\{q\}\)\\right\\\|\_\{2\}^\{2\}\}\}\.\(84\)
Because the absolute error scale differs substantially across PDEs and model formulations, we normalize ORACLE by its matched Native baseline:

ρc=EcOEcN\.\\rho\_\{c\}=\\frac\{E\_\{c\}^\{O\}\}\{E\_\{c\}^\{N\}\}\.\(85\)Hence,

indicates that ORACLE improves over Native,ρc=1\\rho\_\{c\}=1denotes equal endpoint error, andρc\>1\\rho\_\{c\}\>1indicates degradation\.

For each architectureaa, panel \(a\) of Figure[5](https://arxiv.org/html/2610.09040#S4.F5)reports the geometric mean of the eight matched error ratios:

ρ¯a=\(∏c∈𝒞aρc\)1/\|𝒞a\|=exp⁡\[1\|𝒞a\|​∑c∈𝒞alog⁡ρc\]\.\\bar\{\\rho\}\_\{a\}=\\left\(\\prod\_\{c\\in\\mathcal\{C\}\_\{a\}\}\\rho\_\{c\}\\right\)^\{1/\|\\mathcal\{C\}\_\{a\}\|\}=\\exp\\left\[\\frac\{1\}\{\|\\mathcal\{C\}\_\{a\}\|\}\\sum\_\{c\\in\\mathcal\{C\}\_\{a\}\}\\log\\rho\_\{c\}\\right\]\.\(86\)We use a geometric rather than arithmetic mean because the quantity being aggregated is a multiplicative ratio\. This treats proportional improvements consistently across problems with substantially different absolute error scales\.

Native is therefore represented by the normalized reference

ρ¯aN=1,\\bar\{\\rho\}^\{N\}\_\{a\}=1,while the ORACLE bar showsρ¯a\\bar\{\\rho\}\_\{a\}\.

The corresponding architecture\-level values are

ρ¯MLP=0\.897,ρ¯PINN=0\.748,ρ¯PirateNet=0\.379,\\bar\{\\rho\}\_\{\\mathrm\{MLP\}\}=0\.897,\\qquad\\bar\{\\rho\}\_\{\\mathrm\{PINN\}\}=0\.748,\\qquad\\bar\{\\rho\}\_\{\\mathrm\{PirateNet\}\}=0\.379,reported in the main figure as0\.90×0\.90\\times,0\.75×0\.75\\times, and0\.38×0\.38\\timesNative, respectively\.

Equivalently, the relative endpoint\-error reduction can be written as

Δaerr=100​\(1−ρ¯a\)%\.\\Delta^\{\\mathrm\{err\}\}\_\{a\}=100\\left\(1\-\\bar\{\\rho\}\_\{a\}\\right\)\\%\.\(87\)This corresponds to approximately10\.3%10\.3\\%,25\.2%25\.2\\%, and62\.1%62\.1\\%lower normalized endpoint error for Pure MLP, Simple PINN, and PirateNet, respectively\.

#### D\.3\.2Per\-Step Runtime Overhead

We measure runtime relative to the matched Native configuration so that differences in architecture, PDE, and optimizer cost do not confound the comparison\. Let

denote the wall\-clock duration of optimization stepkkfor methodm∈\{N,O\}m\\in\\\{N,O\\\}\. Over theKcK\_\{c\}timed steps, the mean per\-step runtime is

τ¯cm=1Kc​∑k=1Kcτc,km\.\\bar\{\\tau\}^\{m\}\_\{c\}=\\frac\{1\}\{K\_\{c\}\}\\sum\_\{k=1\}^\{K\_\{c\}\}\\tau^\{m\}\_\{c,k\}\.\(88\)
The matched runtime ratio is then

rc=τ¯cOτ¯cN\.r\_\{c\}=\\frac\{\\bar\{\\tau\}^\{O\}\_\{c\}\}\{\\bar\{\\tau\}^\{N\}\_\{c\}\}\.\(89\)Thus,rc=1r\_\{c\}=1corresponds to Native runtime, whilerc=4r\_\{c\}=4means that an ORACLE step requires four times the wall\-clock time of its matched Native step\.

Panel \(b\) reports the unweighted architecture\-level mean

r¯a=1\|𝒞a\|​∑c∈𝒞arc\.\\bar\{r\}\_\{a\}=\\frac\{1\}\{\|\\mathcal\{C\}\_\{a\}\|\}\\sum\_\{c\\in\\mathcal\{C\}\_\{a\}\}r\_\{c\}\.\(90\)Every PDE–optimizer configuration therefore contributes equally rather than allowing a more expensive PDE or model to dominate the aggregate\.

The resulting runtime ratios are

r¯MLP=3\.99,r¯PINN=5\.74,r¯PirateNet=4\.41,\\bar\{r\}\_\{\\mathrm\{MLP\}\}=3\.99,\\qquad\\bar\{r\}\_\{\\mathrm\{PINN\}\}=5\.74,\\qquad\\bar\{r\}\_\{\\mathrm\{PirateNet\}\}=4\.41,reported as approximately4\.0×4\.0\\times,5\.8×5\.8\\times, and4\.4×4\.4\\timesNative runtime per step\.

If expressed as percentage overhead rather than a multiplicative ratio, the same quantity is

Δatime=100​\(r¯a−1\)%\.\\Delta^\{\\mathrm\{time\}\}\_\{a\}=100\\left\(\\bar\{r\}\_\{a\}\-1\\right\)\\%\.\(91\)We use the multiplicative ratio in the main paper because it is easier to compare directly against the normalized performance ratio\.

#### D\.3\.3Pointwise Domain Win Rate

Endpoint relativeℓ2\\ell\_\{2\}error compresses the full solution into one scalar\. To determine whether ORACLE’s advantage is distributed across the solution domain, we additionally compare Native and ORACLE point by point\.

For evaluation pointzq∈𝒵cz\_\{q\}\\in\\mathcal\{Z\}\_\{c\}, define the local prediction error for methodmmas

dc,qm=‖u^cm​\(zq\)−uc⋆​\(zq\)‖2\.d^\{m\}\_\{c,q\}=\\left\\\|\\hat\{u\}^\{m\}\_\{c\}\(z\_\{q\}\)\-u^\{\\star\}\_\{c\}\(z\_\{q\}\)\\right\\\|\_\{2\}\.\(92\)For scalar PDEs this reduces to the absolute error

dc,qm=\|u^cm​\(zq\)−uc⋆​\(zq\)\|\.d^\{m\}\_\{c,q\}=\\left\|\\hat\{u\}^\{m\}\_\{c\}\(z\_\{q\}\)\-u^\{\\star\}\_\{c\}\(z\_\{q\}\)\\right\|\.
ORACLE wins at pointzqz\_\{q\}when

dc,qO<dc,qN\.d^\{O\}\_\{c,q\}<d^\{N\}\_\{c,q\}\.\(93\)Using the indicator function𝟙​\[⋅\]\\mathbbm\{1\}\[\\cdot\], the configuration\-level pointwise win rate is

Sc=100Nc∑q=1Nc𝟙\[dc,qO<dc,qN\]\.S\_\{c\}=\\frac\{100\}\{N\_\{c\}\}\\sum\_\{q=1\}^\{N\_\{c\}\}\\mathbbm\{1\}\\left\[d^\{O\}\_\{c,q\}<d^\{N\}\_\{c,q\}\\right\]\.\(94\)
A value ofSc=50%S\_\{c\}=50\\%indicates equal pointwise coverage, whileSc\>50%S\_\{c\}\>50\\%means that ORACLE is more accurate than Native on the majority of evaluated points\.

We first computeScS\_\{c\}separately for every matched configuration and then average the eight configuration\-level rates within each architecture:

S¯a=1\|𝒞a\|​∑c∈𝒞aSc\.\\bar\{S\}\_\{a\}=\\frac\{1\}\{\|\\mathcal\{C\}\_\{a\}\|\}\\sum\_\{c\\in\\mathcal\{C\}\_\{a\}\}S\_\{c\}\.\(95\)This configuration\-first aggregation prevents a PDE with a denser evaluation grid from receiving greater weight solely because it contains more points\.

S¯MLP=67\.4%,S¯PINN=61\.6%,S¯PirateNet=70\.8%\.\\bar\{S\}\_\{\\mathrm\{MLP\}\}=67\.4\\%,\\qquad\\bar\{S\}\_\{\\mathrm\{PINN\}\}=61\.6\\%,\\qquad\\bar\{S\}\_\{\\mathrm\{PirateNet\}\}=70\.8\\%\.Thus, ORACLE is more accurate than Native on a majority of evaluated domain points for all three model formulations\.

For visualization, the Native share is shown as the complementary quantity

SaN=100−S¯a,S^\{N\}\_\{a\}=100\-\\bar\{S\}\_\{a\},\(96\)under the negligible\-tie convention used in the aggregate plot\.

#### D\.3\.4Physical Time\-Slice Win Rate

The pointwise metric above measures where ORACLE improves the solution\. We separately ask whether this advantage persists across the physical time dimension of the PDE\.

Let

\{tj\}j=1Tc\\\{t\_\{j\}\\\}\_\{j=1\}^\{T\_\{c\}\}denote the physical time slices in the evaluation grid\. At each timetjt\_\{j\}, we compute the spatial relativeℓ2\\ell\_\{2\}error

ec,jm=‖u^cm​\(⋅,tj\)−uc⋆​\(⋅,tj\)‖2‖uc⋆​\(⋅,tj\)‖2\.e^\{m\}\_\{c,j\}=\\frac\{\\left\\\|\\hat\{u\}^\{m\}\_\{c\}\(\\cdot,t\_\{j\}\)\-u^\{\\star\}\_\{c\}\(\\cdot,t\_\{j\}\)\\right\\\|\_\{2\}\}\{\\left\\\|u^\{\\star\}\_\{c\}\(\\cdot,t\_\{j\}\)\\right\\\|\_\{2\}\}\.\(97\)
For a discrete spatial grid𝒳c,j=\{xi\}i=1Nx,j\\mathcal\{X\}\_\{c,j\}=\\\{x\_\{i\}\\\}\_\{i=1\}^\{N\_\{x,j\}\},

ec,jm=∑i=1Nx,j‖u^cm​\(xi,tj\)−uc⋆​\(xi,tj\)‖22∑i=1Nx,j‖uc⋆​\(xi,tj\)‖22\.e^\{m\}\_\{c,j\}=\\sqrt\{\\frac\{\\sum\_\{i=1\}^\{N\_\{x,j\}\}\\left\\\|\\hat\{u\}^\{m\}\_\{c\}\(x\_\{i\},t\_\{j\}\)\-u^\{\\star\}\_\{c\}\(x\_\{i\},t\_\{j\}\)\\right\\\|\_\{2\}^\{2\}\}\{\\sum\_\{i=1\}^\{N\_\{x,j\}\}\\left\\\|u^\{\\star\}\_\{c\}\(x\_\{i\},t\_\{j\}\)\\right\\\|\_\{2\}^\{2\}\}\}\.\(98\)
ORACLE wins time slicetjt\_\{j\}when

ec,jO<ec,jN\.e^\{O\}\_\{c,j\}<e^\{N\}\_\{c,j\}\.\(99\)The configuration\-level time\-slice win rate is therefore

Tc=100Tceval∑j=1Tceval𝟙\[ec,jO<ec,jN\],T\_\{c\}=\\frac\{100\}\{T\_\{c\}^\{\\mathrm\{eval\}\}\}\\sum\_\{j=1\}^\{T\_\{c\}^\{\\mathrm\{eval\}\}\}\\mathbbm\{1\}\\left\[e^\{O\}\_\{c,j\}<e^\{N\}\_\{c,j\}\\right\],\(100\)whereTcevalT\_\{c\}^\{\\mathrm\{eval\}\}is the number of evaluated physical time slices\.

Importantly, this metric refers to*physical PDE time*, not optimization steps or training checkpoints\. It therefore measures whether ORACLE’s solution\-quality advantage persists throughout the predicted dynamics\.

As with the pointwise metric, we first compute the win rate independently for each matched configuration and then average across the eight configurations within an architecture:

T¯a=1\|𝒞a\|​∑c∈𝒞aTc\.\\bar\{T\}\_\{a\}=\\frac\{1\}\{\|\\mathcal\{C\}\_\{a\}\|\}\\sum\_\{c\\in\\mathcal\{C\}\_\{a\}\}T\_\{c\}\.\(101\)Panel \(c\) reports

T¯MLP=81\.1%,T¯PINN=88\.8%,T¯PirateNet=82\.9%\.\\bar\{T\}\_\{\\mathrm\{MLP\}\}=81\.1\\%,\\qquad\\bar\{T\}\_\{\\mathrm\{PINN\}\}=88\.8\\%,\\qquad\\bar\{T\}\_\{\\mathrm\{PirateNet\}\}=82\.9\\%\.Hence, ORACLE produces lower spatial solution error than Native on more than80%80\\%of evaluated physical time slices for every model formulation\.

Analogously to the spatial plot, the Native share is displayed as

TaN=100−T¯aT^\{N\}\_\{a\}=100\-\\bar\{T\}\_\{a\}\(102\)under the negligible\-tie convention\.

### D\.4Matched Wall\-Clock Evaluation

Panel \(d\) reports The primary experiments compare Native and ORACLE after the same number of optimization steps, isolating the effect of modifying the realized update\. Because ORACLE performs additional constraint evaluation, linear solves, and validation, we additionally ask whether these gains amortize when both methods receive the same elapsed\-time budget\.

For each optimizer–PDE configurationcc, Native and ORACLE are therefore compared at a common wall\-clock budgetTcT\_\{c\}\. LetEcN​\(Tc\)E\_\{c\}^\{\\mathrm\{N\}\}\(T\_\{c\}\)andEcO​\(Tc\)E\_\{c\}^\{\\mathrm\{O\}\}\(T\_\{c\}\)denote their relativeℓ2\\ell\_\{2\}solution errors at that budget\. We report the normalized equal\-time ratio

ρctime=EcO​\(Tc\)EcN​\(Tc\),\\rho^\{\\mathrm\{time\}\}\_\{c\}=\\frac\{E\_\{c\}^\{\\mathrm\{O\}\}\(T\_\{c\}\)\}\{E\_\{c\}^\{\\mathrm\{N\}\}\(T\_\{c\}\)\},\(103\)whereρctime<1\\rho^\{\\mathrm\{time\}\}\_\{c\}<1means that ORACLE achieves lower error within the same elapsed time\. Unlike the step\-matched comparison, ORACLE generally completes fewer optimization steps because each step includes the additional alignment procedure\.

To summarize across PDEs, we compute the geometric mean separately for each native optimizer,

ρ¯otime=exp⁡\(1\|𝒞o\|​∑c∈𝒞olog⁡ρctime\),\\bar\{\\rho\}^\{\\mathrm\{time\}\}\_\{o\}=\\exp\\\!\\left\(\\frac\{1\}\{\|\\mathcal\{C\}\_\{o\}\|\}\\sum\_\{c\\in\\mathcal\{C\}\_\{o\}\}\\log\\rho^\{\\mathrm\{time\}\}\_\{c\}\\right\),\(104\)where𝒞o\\mathcal\{C\}\_\{o\}contains the available PDE configurations for optimizeroo\. The geometric mean is appropriate because the comparison is expressed as a multiplicative error ratio and the underlying errors span several orders of magnitude\.

#### D\.4\.1Summary of Aggregated Quantities

Table[14](https://arxiv.org/html/2610.09040#A4.T14)reports the architecture\-level values used directly in Figure[5](https://arxiv.org/html/2610.09040#S4.F5)\.

Table 14:Architecture\-level quantities used in the cost–benefit analysis\.Endpoint performance is the geometric mean of matched ORACLE/Native relative\-ℓ2\\ell\_\{2\}ratios across four PDEs and two optimizers\. Runtime, spatial win rate, and time\-slice win rate are arithmetic means of the corresponding eight configuration\-level quantities\. Lower is better for endpoint ratio and runtime; higher is better for the two win rates\.Taken together, these metrics separate four distinct questions\. The endpoint ratio measures*how much*final solution quality changes; the runtime ratio measures*what that improvement costs*; the pointwise win rate measures*how broadly*the improvement is distributed over the solution domain; and the time\-slice win rate measures*how persistently*it holds across physical time\. The main\-text conclusion therefore does not rely on the endpoint metric alone: ORACLE incurs additional computation, but the resulting accuracy gains extend across both the domain and the temporal evolution of the solution\.

## Appendix EExtended Mechanism Analysis

The main paper isolates three mechanisms within ORACLE: the geometry used to construct alignment, the authority permitted relative to the Native step, and the held\-out validation used before commitment\. This appendix reports the corresponding exact values and provides additional analysis of the Native optimizer geometry\. We first examine endpoint behavior under controlled ablations, then quantify the anisotropy of the optimizer\-induced metric, and finally relate that geometry to authority activation and held\-out rejection\.

### E\.1Optimizer Geometry and Endpoint Behavior

Because Allen–Cahn and Wave have different absolute error scales, we compare mechanism ablations using endpoint error normalized by the matched Native result,

Rerr=relL2⁡\(method\)relL2⁡\(Native\),R\_\{\\mathrm\{err\}\}=\\frac\{\\mathrm\{relL2\}\(\\mathrm\{method\}\)\}\{\\mathrm\{relL2\}\(\\mathrm\{Native\}\)\},\(105\)whereRerr<1R\_\{\\mathrm\{err\}\}<1denotes lower endpoint error than Native\.

Table[15](https://arxiv.org/html/2610.09040#A5.T15)reports the exact values underlying Fig\.[7](https://arxiv.org/html/2610.09040#S4.F7)\(a,d\)\. Replacing the optimizer\-relative metric withPt=IP\_\{t\}=Iincreases endpoint error to18\.60×18\.60\\timesNative on Allen–Cahn and5\.40×5\.40\\timeson Wave\. By contrast, the metric\-aware First\-order variant reaches0\.572×0\.572\\timesand0\.707×0\.707\\timesNative, while full ORACLE reaches0\.388×0\.388\\timesand0\.554×0\.554\\times, respectively\.

The Euclidean ablation therefore changes the outcome substantially rather than acting as a benign simplification of the ORACLE metric\. At the same time, geometry alone does not determine the final endpoint: the Unbounded variant achieves the lowest Allen–Cahn error but provides a smaller gain on Wave\. This motivates separating the geometry used to construct alignment from the authority assigned to that alignment\.

Table 15:Endpoint behavior under controlled ORACLE ablations\.Relativeℓ2\\ell\_\{2\}error and error normalized by the matched Native endpoint on Allen–Cahn and Wave\. Lower is better\.
### E\.2Anisotropy of the Native Optimizer Geometry

The strong degradation underPt=IP\_\{t\}=Iraises a natural question: how different is the Native optimizer geometry from an isotropic Euclidean metric in practice? For Adam, the optimizer\-relative metric is determined by its coordinate\-wise adaptive denominator\. Writing

Dt=v^t\+ϵ,D\_\{t\}=\\sqrt\{\\widehat\{v\}\_\{t\}\}\+\\epsilon,we quantify its anisotropy using both the full coordinate condition number,

κ⁡\(Dt\)=maxi⁡\(Dt\)imini⁡\(Dt\)i,\\kappa\(D\_\{t\}\)=\\frac\{\\max\_\{i\}\(D\_\{t\}\)\_\{i\}\}\{\\min\_\{i\}\(D\_\{t\}\)\_\{i\}\},\(106\)and the robust percentile ratio

ρ95/05​\(Dt\)=p95​\(Dt\)p05​\(Dt\)\.\\rho\_\{95/05\}\(D\_\{t\}\)=\\frac\{p\_\{95\}\(D\_\{t\}\)\}\{p\_\{05\}\(D\_\{t\}\)\}\.\(107\)
Table[16](https://arxiv.org/html/2610.09040#A5.T16)shows substantial anisotropy on both PDEs\. The full coordinate spread exceeds10610^\{6\}at every reported checkpoint and reaches1\.94×1071\.94\\times 10^\{7\}on Allen–Cahn at step1010k\. Even after reducing sensitivity to extreme coordinates, the9595th\-to\-55th percentile ratio ranges from16\.716\.7to33\.833\.8\.

Thus, replacing the Native Adam geometry withPt=IP\_\{t\}=Idiscards a large amount of coordinate\-dependent scaling\. This does not by itself prove that anisotropy causes the endpoint differences in Table[15](https://arxiv.org/html/2610.09040#A5.T15), but it shows that the Euclidean ablation constitutes a substantial change in the geometry rather than a small approximation to the Native metric\.

Table 16:Anisotropy of the Adam\-induced geometry\.The full condition numberκ⁡\(Dt\)\\kappa\(D\_\{t\}\)measures the total coordinate\-wise spread, whilep95​\(Dt\)/p05​\(Dt\)p\_\{95\}\(D\_\{t\}\)/p\_\{05\}\(D\_\{t\}\)provides a robust measure less sensitive to extreme coordinates\.The same geometry enters ORACLE twice: it determines the alignment direction throughPt−1P\_\{t\}^\{\-1\}and determines the scale of admissible alignment through thePtP\_\{t\}\-norm\. We therefore next examine how often the native\-relative authority bound becomes active\.

### E\.3Geometry and Alignment Authority

ORACLE bounds the aligned adjustment relative to the Native movement in the same optimizer\-induced metric,

∥d~t∥Pt≤βt​∥st∥Pt,∥v∥Pt2=v⊤​Pt​v\.\\lVert\\widetilde\{d\}\_\{t\}\\rVert\_\{P\_\{t\}\}\\leq\\beta\_\{t\}\\lVert s\_\{t\}\\rVert\_\{P\_\{t\}\},\\qquad\\lVert v\\rVert\_\{P\_\{t\}\}^\{2\}=v^\{\\top\}P\_\{t\}v\.\(108\)
Table[17](https://arxiv.org/html/2610.09040#A5.T17)reports how often this authority boundary is active among accepted alignments\. For full ORACLE, the boundary binds on100\.0%100\.0\\%of accepted Allen–Cahn alignments and99\.2%99\.2\\%on Wave\. The metric\-aware First\-order variant behaves similarly, with rates of99\.7%99\.7\\%and94\.8%94\.8\\%\. Under thePt=IP\_\{t\}=Iablation, the corresponding rates fall to37\.6%37\.6\\%and2\.5%2\.5\\%\.

The authority mechanism therefore depends materially on the geometry in which the Native and aligned movements are measured\. ChangingPtP\_\{t\}affects not only the direction produced by the alignment objective but also whether a proposed adjustment reaches the native\-relative authority boundary\.

The Unbounded variant provides the complementary case: its authority boundary is absent by construction\. Its strong Allen–Cahn endpoint shows that greater alignment authority can help in some regimes, whereas its smaller improvement on Wave shows that unrestricted authority is not uniformly preferable across problems\. We therefore treat the authority bound as a mechanism for regulating alignment magnitude rather than claim that it independently improves every endpoint\.

Table 17:Activation of the native\-relative authority boundary\.Percentage of accepted alignments for which Eq\. equation[108](https://arxiv.org/html/2610.09040#A5.E108)is active in the matched mechanism study\. Native constructs no alignment candidate, while Unbounded removes the authority boundary by definition\.
### E\.4Held\-Out Validation and Commitment

The authority bound controls the magnitude of an alignment candidate but does not establish that its locally predicted improvement persists on unseen constraint samples\. ORACLE therefore separates candidate construction from commitment by evaluating the fixed candidate on an independent supportSt′S^\{\\prime\}\_\{t\}\.

We count candidates rejected at this stage as

Naudit=∑t=1T𝟏\[Auditt=reject\]\.N\_\{\\mathrm\{audit\}\}=\\sum\_\{t=1\}^\{T\}\\mathbf\{1\}\\left\[\\mathrm\{Audit\}\_\{t\}=\\mathrm\{reject\}\\right\]\.\(109\)
Table[18](https://arxiv.org/html/2610.09040#A5.T18)reports the resulting counts over the matched55k\-step mechanism study\. On Allen–Cahn, thePt=IP\_\{t\}=Ivariant produces875875held\-out rejections, compared with1919for First\-order,1919for Unbounded, and only44for ORACLE\. On Wave, the corresponding counts are1111,66,11, and00\.

These counts provide information not captured by endpoint rel\-L2\. In particular, the Euclidean ablation not only ends with substantially larger solution error but also constructs many more candidates whose local improvement does not persist on independent support\. This diagnostic does not establish that audit rejection itself causes the endpoint difference; rather, it shows that the geometry used to construct alignment is associated with substantially different off\-support behavior\.

Table 18:Held\-out validation rejections in the matched mechanism study\.Number of fixed alignment candidates rejected on independent support over55k active optimization steps\. Native constructs no alignment candidate and therefore has no audit decision\.
### E\.5Mechanism Summary

The extended diagnostics support a sequential interpretation of ORACLE\. The Native optimizer first determines the metric in which alignment is constructed\. The same metric defines the scale of the aligned adjustment relative to the Native proposal\. Finally, held\-out validation determines whether the fixed candidate is committed:

Pt⟶alignment⟶authority⟶held\-out validation⟶commitment\.\\boxed\{P\_\{t\}\\;\\longrightarrow\\;\\text\{alignment\}\\;\\longrightarrow\\;\\text\{authority\}\\;\\longrightarrow\\;\\text\{held\-out validation\}\\;\\longrightarrow\\;\\text\{commitment\}\.\}
This sequence clarifies the roles of the individual mechanisms\. ReplacingPtP\_\{t\}by the identity changes both alignment behavior and authority activation; removing the authority bound changes the allowable intervention scale; and independent validation can reject candidates that appear locally acceptable on the construction support\.

We therefore do not interpret every component as independently necessary for every problem\. Instead, the complete ORACLE design assigns distinct roles to the three mechanisms: optimizer geometry determines*how*alignment is measured, the authority bound limits*how far*the Native movement may be redirected, and held\-out validation determines*whether*the aligned candidate is ultimately committed\.

### E\.6Optimizer\-Level Gate Statistics

Table[19](https://arxiv.org/html/2610.09040#A5.T19)reports how frequently each stage of ORACLE is reached across the eight Q1 PDE configurations for each optimizer\.*Certified*denotes steps for which the recovered optimizer geometry passes certification;*Trigger*denotes steps satisfyingct⊤​st\>0c\_\{t\}^\{\\top\}s\_\{t\}\>0;*Proposal*denotes steps for which a valid alignment candidate is constructed; and*Accept*denotes steps for which the candidate passes all validation gates and is committed\.

Table 19:ORACLE gate statistics across optimizer geometries\.Rates are aggregated across the eight PDE configurations associated with each optimizer\. Certification checks whether the recovered geometry faithfully represents the realized native update; trigger marks locally adverse native steps; proposal records successfully constructed alignment candidates; and accept records corrections ultimately committed after validation and held\-out audit\.Geometry certification succeeds throughout these evaluated configurations, showing that ORACLE can recover the realized optimizer geometry reliably for all four optimizer families\. The subsequent behavior is optimizer dependent: the alignment trigger fires on59\.56%59\.56\\%of SGD steps but on88\.4688\.46–94\.73%94\.73\\%of Adam, SOAP, and PSGD–Kron steps\. Once triggered, a valid alignment proposal is constructed almost always, while the lower final acceptance rates reflect the nonlinear\-agreement and held\-out validation gates\. Thus, ORACLE’s selectivity arises primarily from detecting whether alignment is needed and from validating the resulting candidate, rather than from failures to construct the optimizer\-relative correction\.