GradRepair-ODE: Certified Gradient Repair for Neural ODE Training

# GradRepair-ODE: Certified Gradient Repair for Neural ODE Training
Source: [https://arxiv.org/html/2609.13204](https://arxiv.org/html/2609.13204)
\[ BoldFont = texgyretermes\-bold\.otf, ItalicFont = texgyretermes\-italic\.otf, BoldItalicFont = texgyretermes\-bolditalic\.otf \]

###### Abstract

Neural ordinary differential equations use numerical solvers inside the training loop\. The solver determines the forward trajectory and also affects the gradient passed to the optimizer\. That coupling creates a reliability problem for scientific machine learning and continuous\-time generative modeling, including diffusion probability\-flow ordinary differential equations and flow\-matching models\. Under loose step sizes, stiff dynamics, chaotic sensitivity, or event discontinuities, a differentiable ODE pipeline can return a finite gradient whose direction is numerically suspect\. We introduce GradRepair\-ODE, a reliability framework for checking, repairing, and rejecting ODE gradients at the optimizer step\. The method computes several gradient candidates, compares them with directional finite\-difference checks and solver diagnostics, diagnoses likely numerical failure modes, repairs selected gradients through path switching or stricter recomputation, and rejects steps whose descent direction cannot be certified\. In six synthetic ODE systems, GradRepair\-ODE leaves low\-risk systems unchanged, repairs Robertson and Lorenz gradients to cosine similarity1\.0001\.000against a strict reference, reduces unsafe accepted steps from3737to00, and rejects an event\-discontinuous case instead of applying an uncertified update\. The paper argues for a simple change in the training contract: an ODE gradient should reach the optimizer with numerical evidence attached\.

Short title:GradRepair\-ODEAuthors:Z\. Bi and X\. L\. Chia

Keywords\.Neural ordinary differential equations; differentiable simulation; adjoint sensitivity; numerical stability; gradient reliability; scientific machine learning; diffusion models; flow matching; continuous normalizing flows\.

Mathematics Subject Classification\.65L05; 65L06; 65L20; 65L50; 68T07; 90C30\.

SISC section\.Machine Learning Methods for Scientific Computing\.

## 1Introduction

Neural ordinary differential equations \(Neural ODEs\) replace a discrete stack of transformations with a continuous\-time dynamical system whose trajectory is computed by a numerical ordinary differential equation \(ODE\) solver\[[4](https://arxiv.org/html/2609.13204#bib.bib1)\]\. This formulation gives the model an appealing mathematical interface and allows solver tolerance to trade numerical precision for speed\. It also moves a large part of training into numerical analysis: the optimizer does not receive an exact gradient, but a gradient shaped by solver error, interpolation, stiffness, horizon length, and the chosen sensitivity path\.

Modern differentiable ODE methods expose both direct backpropagation through solver operations and adjoint\-style methods with different memory and accuracy tradeoffs\[[4](https://arxiv.org/html/2609.13204#bib.bib1),[14](https://arxiv.org/html/2609.13204#bib.bib14)\]\. The same numerical issue reaches beyond classical Neural ODE benchmarks\. Continuous normalizing flows \(CNFs\) use ODE dynamics for density modeling\[[7](https://arxiv.org/html/2609.13204#bib.bib2)\]; diffusion probabilistic models and score\-based stochastic differential equation \(SDE\) models connect generative modeling to continuous\-time dynamics and probability\-flow ODEs\[[23](https://arxiv.org/html/2609.13204#bib.bib3),[13](https://arxiv.org/html/2609.13204#bib.bib4)\]; flow matching trains CNFs by regressing vector fields along probability paths and explicitly includes diffusion paths as an important case\[[15](https://arxiv.org/html/2609.13204#bib.bib5)\]; rectified flow learns ODE transports that aim to make generative paths straight and solver\-efficient\[[16](https://arxiv.org/html/2609.13204#bib.bib6)\]\. In all of these settings, a learned vector field is ultimately paired with numerical integration\. When such models are trained through differentiable solves or evaluated by ODE\-based likelihood or sampling routines, gradient reliability is no longer a niche implementation detail; it is part of the numerical foundation of continuous\-time generative modeling\.

Prior work has shown that optimize\-then\-discretize adjoints and discretize\-then\-optimize gradients can disagree in ways that matter for Neural ODE training\[[6](https://arxiv.org/html/2609.13204#bib.bib13),[18](https://arxiv.org/html/2609.13204#bib.bib15)\]\. Classical numerical analysis has long treated step\-size control, dense output, stiffness, backward differentiation formulae, finite\-precision effects, and sensitivity analysis as solver\-level concerns\[[5](https://arxiv.org/html/2609.13204#bib.bib7),[9](https://arxiv.org/html/2609.13204#bib.bib8),[22](https://arxiv.org/html/2609.13204#bib.bib9),[2](https://arxiv.org/html/2609.13204#bib.bib10),[11](https://arxiv.org/html/2609.13204#bib.bib11),[12](https://arxiv.org/html/2609.13204#bib.bib17),[10](https://arxiv.org/html/2609.13204#bib.bib19),[3](https://arxiv.org/html/2609.13204#bib.bib20)\]\. General scientific computing libraries likewise distinguish non\-stiff explicit solvers from stiff\-aware implicit methods and warn that divergence or unusually many iterations can indicate stiffness\[[20](https://arxiv.org/html/2609.13204#bib.bib12)\]\. These facts are well understood at the solver level, but the optimizer interface usually receives only the resulting tensor\. Once automatic differentiation returns a finite array with the expected shape, most training loops have little additional evidence about its directional reliability\.

The failure is easy to miss\. A gradient tensor can be finite, shaped correctly, and still point in the wrong direction\. If that direction disagrees with other sensitivity paths or with local directional checks, the optimizer may increase the loss while the training loop records only a normal update\. The relevant question is whether the current numerical evidence supports the next step\.

GradRepair\-ODE answers that question at the optimizer boundary\. It computes multiple gradient candidates, builds a reliability certificate from path disagreement and finite\-difference residuals, diagnoses numerical risk, repairs selected gradients, and certifies whether the final step is accepted\. The method does not propose another ODE architecture or another general\-purpose solver\. It adds a missing interface: gradient trust, repair, and rejection become recorded training events\.

The point is sharper than a debugging aid\. Solver tolerance, an adjoint method, and a loss curve do not say whether the next update is defensible\. GradRepair\-ODE places a certificate where numerical approximation becomes optimization action\. The certificate can trust a benign gradient, route a repairable gradient to a safer path, or withhold an update whose smooth\-gradient evidence has collapsed\. This matters in scientific computing because the damaging failure is often silent: the computation continues with a plausible number whose direction has lost numerical meaning\.

The experiments use controlled mechanisms rather than decorative applications\. The suite contains smooth, stiff, chaotic, event\-driven, and neural\-vector\-field dynamics, so the same reliability layer faces the pathologies it claims to expose\. The main result is the conversion of hidden optimizer risk into explicit decisions\. GradRepair\-ODE leaves low\-risk systems untouched, repairs Robertson and Lorenz gradients to cosine similarity1\.0001\.000against a strict reference, reduces unsafe accepted steps from3737to00, and rejects the event\-discontinuous case when repair would overstate the evidence\. A useful reliability method should have exactly this profile: keep benign steps cheap, pay for accuracy when the direction can be recovered, and refuse an update when the differentiated object no longer matches the optimized trajectory\.

The paper contributes four pieces\. It defines an optimizer\-facing gradient reliability certificate for differentiable ODE training, using sensitivity\-path disagreement, directional finite differences, and solver diagnostics\. It formulates repair as constrained selection among gradient paths, not as a blanket move to the strictest computation\. It derives a local descent certificate that connects an empirical gradient\-error radius to the next optimizer step\. It also reports a controlled numerical study, with gradient clipping and Armijo\-style line search included as safeguards, showing when the certificate trusts, repairs, or rejects\. The outcome is not a training\-stability anecdote\. It is a per\-step numerical account of the update stream\.

## 2Method

### 2\.1Problem Setup

We consider an ODE parameterized byθ\\theta,

d​xd​t=f⁡\(t,x,θ\),x⁡\(t0\)=x0,\\frac\{dx\}\{dt\}=f\(t,x,\\theta\),\\qquad x\(t\_\{0\}\)=x\_\{0\},\(1\)with a trajectory\-level loss

L\(θ\)=ℓ\(x\(t0:t1;θ\),y\)\.L\(\\theta\)=\\ell\(x\(t\_\{0\}:t\_\{1\};\\theta\),y\)\.\(2\)The training loop receives an approximate gradientg^\\hat\{g\}produced by a numerical solver and a sensitivity path\. GradRepair\-ODE tests whetherg^\\hat\{g\}can support the update−η​g^\-\\eta\\hat\{g\}\.

The distinction between a derivative and an update is practical, not semantic\. A sensitivity path can produce a parameter\-space vector even when the trajectory used to construct it is fragile\. Let𝒮\\mathcal\{S\}denote the solver configuration, including the integration rule, tolerance, step\-size policy, interpolation rule, and event handling\. Let𝒫\\mathcal\{P\}denote the sensitivity path\. The returned gradient is better written as

g^=G⁡\(θ,𝒮,𝒫\),\\hat\{g\}=G\(\\theta;\\mathcal\{S\},\\mathcal\{P\}\),\(3\)instead of simply∇L​\(θ\)\\nabla L\(\\theta\)\. GradRepair\-ODE treats the gap betweenG⁡\(θ,𝒮,𝒫\)G\(\\theta;\\mathcal\{S\},\\mathcal\{P\}\)and∇L​\(θ\)\\nabla L\(\\theta\)as an observable numerical quantity\. The framework is solver\-agnostic in the same sense that a line search is optimizer\-agnostic: it leaves the underlying numerical method in place and changes the condition under which the optimizer trusts its output\.

Each training step produces gradient\-path evidence, directional evidence, and solver evidence\. Gradient\-path evidence compares independent or partially independent approximations of the same derivative\. Directional evidence compares inner products against local centered finite differences\. Solver evidence records numerical stress, such as step\-size collapse, high function\-evaluation counts, failed steps, or known discontinuities\. A gradient supported by these signals passes cheaply\. A gradient that fails them is repaired or withheld\.

### 2\.2Gradient Candidates

Each diagnostic step compares the default coarse solver\-path gradient with gradients from more accurate recomputation paths\. A finer discrete path tests whether the direction survives reduced integration error\. A checkpointed\-style recomputation path tests whether greater trajectory consistency changes the gradient\. A strict path provides a high\-cost reference for certification or last\-resort repair\. This design borrows the memory\-computation tradeoff familiar from reverse\-mode differentiation, adjoint sensitivity analysis, and checkpointing\[[8](https://arxiv.org/html/2609.13204#bib.bib16),[12](https://arxiv.org/html/2609.13204#bib.bib17),[21](https://arxiv.org/html/2609.13204#bib.bib18)\]\. Here the goal is the update decision: accept, repair, or reject before the parameter changes\.

The candidate set is ordered by cost and expected numerical fidelity:

𝒞=\{gcoarse,gdisc,gckpt,gstrict\}\.\\mathcal\{C\}=\\\{g\_\{\\mathrm\{coarse\}\},g\_\{\\mathrm\{disc\}\},g\_\{\\mathrm\{ckpt\}\},g\_\{\\mathrm\{strict\}\}\\\}\.\(4\)The coarse path is the gradient a standard training loop would consume\. The discrete path recomputes the solve at smaller steps and checks whether the direction survives refinement\. The checkpointed\-style path represents a more trajectory\-consistent recomputation with additional cost\. The strict path is used as a repair target or evaluation reference in the controlled study\. In a larger implementation, these symbols may correspond to continuous adjoints, direct differentiation through solver operations, checkpointed discrete adjoints, direct sensitivity equations, or stiff\-aware solvers\. The claim does not depend on a particular solver; it depends on multiple numerical routes whose disagreement carries information\.

### 2\.3Reliability Certificate

For a candidate gradientgig\_\{i\}, GradRepair\-ODE computes pairwise cosine disagreement

Dcos​\(gi,gj\)=1−⟨gi,gj⟩‖gi‖​‖gj‖\+δ,D\_\{\\cos\}\(g\_\{i\},g\_\{j\}\)=1\-\\frac\{\\langle g\_\{i\},g\_\{j\}\\rangle\}\{\\\|g\_\{i\}\\\|\\\|g\_\{j\}\\\|\+\\delta\},\(5\)relative norm disagreement

Dnorm​\(gi,gj\)=‖gi−gj‖‖gi‖\+‖gj‖\+δ,D\_\{\\mathrm\{norm\}\}\(g\_\{i\},g\_\{j\}\)=\\frac\{\\\|g\_\{i\}\-g\_\{j\}\\\|\}\{\\\|g\_\{i\}\\\|\+\\\|g\_\{j\}\\\|\+\\delta\},\(6\)and directional finite\-difference \(FD\) residual

RFD​\(g,v\)=\|⟨g,v⟩−FD⁡\(θ,v\)\|\|FD⁡\(θ,v\)\|\+δ\.R\_\{\\mathrm\{FD\}\}\(g,v\)=\\frac\{\|\\langle g,v\\rangle\-\\mathrm\{FD\}\(\\theta,v\)\|\}\{\|\\mathrm\{FD\}\(\\theta,v\)\|\+\\delta\}\.\(7\)These terms are combined with solver instability and stiffness proxies into an empirical uncertainty score

ϵ^i=wcos​Dcos\+wnorm​Dnorm\+wFD​RFD\+wsolver​S\+wstiff​K,\\widehat\{\\epsilon\}\_\{i\}=w\_\{\\cos\}D\_\{\\cos\}\+w\_\{\\mathrm\{norm\}\}D\_\{\\mathrm\{norm\}\}\+w\_\{\\mathrm\{FD\}\}R\_\{\\mathrm\{FD\}\}\+w\_\{\\mathrm\{solver\}\}S\+w\_\{\\mathrm\{stiff\}\}K,\(8\)whereSSdenotes solver instability evidence andKKdenotes stiffness evidence\. The certificate assigns one of four states:Trusted,Repairable,Unsafe, orFailed\. The certificate is a visible reliability estimate, not a claimed mathematical upper bound\.

ATrustedgradient is applied without repair\. ARepairablegradient justifies additional computation but not the original update\. AnUnsafegradient lacks a certified descent direction after the available repairs\. AFailedgradient contains nonfinite values, invalid shapes, or broken differentiation state\. These states keep warnings, repairs, acceptances, and failures separate\.

The certificate also diagnoses the failure mode\. Large finite\-difference residuals indicate directional inconsistency\. Large pairwise disagreement indicates sensitivity\-path instability\. Large solver stress indicates numerical difficulty before the gradient is inspected\. Event flags indicate that the smooth adjoint model may be mismatched to the trajectory\. When the evidence is mixed, the diagnosis reports numerical instability instead of assigning a precise physical cause\.

### 2\.4Repair and Step Certification

If the naive gradient is trusted, the optimizer can use it directly\. If the certificate reports repairable risk, GradRepair\-ODE switches to a more reliable gradient path\. If the repaired certificate remains unsafe, the step is rejected\. At step level, the decision compares the predicted gradient signal against the estimated numerical uncertainty:

η​‖g^‖2\>η​ϵ​‖g^‖\+m,\\eta\\\|\\hat\{g\}\\\|^\{2\}\>\\eta\\epsilon\\\|\\hat\{g\}\\\|\+m,\(9\)whereϵ\\epsilonis the empirical error proxy andmmis a small margin\. This rule is sufficient for the step certificate used here; it is not a global convergence theorem\.

Repair is selected by a cost\-aware policy\. Tolerance tightening is used when the evidence points to loose integration\. Path switching is used when sensitivity routes disagree but a refined path is available\. Strict recomputation is used for stiff or chaotic cases that remain repairable under additional numerical work\. Rejection is selected when the trajectory contains an event discontinuity without an event\-aware derivative, or when no candidate produces a positive descent margin\. The policy minimizes computation subject to a reliability constraint:

mingi∈𝒞⁡Cisuch thatCert⁡\(gi\)=TrustedandΔi\>0\.\\min\_\{g\_\{i\}\\in\\mathcal\{C\}\}C\_\{i\}\\quad\\text\{such that\}\\quad\\mathrm\{Cert\}\(g\_\{i\}\)=\\mathrm\{Trusted\}\\quad\\text\{and\}\\quad\\Delta\_\{i\}\>0\.\(10\)If the feasible set is empty, the update is rejected\. This separates GradRepair\-ODE from ordinary optimizer safeguards\. Clipping and line search operate on the proposed move; GradRepair\-ODE checks the numerical origin of the gradient that produced the move\.

## 3Theory and Certification

GradRepair\-ODE uses a local theoretical object: a certificate for the next optimizer step\. The certificate does not prove that an ordinary differential equation solver returned the exact gradient\. It combines a computed gradient, an empirical uncertainty radius, and local descent geometry\. Global guarantees are usually out of reach for stiff, chaotic, or event\-driven dynamics\. Step\-level evidence is still enough to decide whether a gradient can be used, repaired, or rejected\.

### 3\.1Uncertainty Sets

LetL⁡\(θ\)L\(\\theta\)denote the training loss,g=∇L​\(θ\)g=\\nabla L\(\\theta\)the exact gradient, andg^i\\hat\{g\}\_\{i\}the gradient candidate produced by pathii\. GradRepair\-ODE assigns the candidate an empirical uncertainty radiusϵ^i\\widehat\{\\epsilon\}\_\{i\}obtained from path disagreement, directional finite\-difference residuals, solver instability, and stiffness evidence\. The associated local uncertainty set is

𝒢i=\{g′:‖g′−g^i‖2≤ϵ^i\}\.\\mathcal\{G\}\_\{i\}=\\\{g^\{\\prime\}:\\\|g^\{\\prime\}\-\\hat\{g\}\_\{i\}\\\|\_\{2\}\\leq\\widehat\{\\epsilon\}\_\{i\}\\\}\.\(11\)This set is local\. It does not describe future gradients; it describes the numerical ambiguity attached to the current candidate at the current step\.

###### Assumption 1\(Local smoothness\)\.

In a neighborhood of the proposed update,LLis differentiable and its gradient is locallyMM\-Lipschitz:

‖∇L​\(θa\)−∇L​\(θb\)‖2≤M​‖θa−θb‖2\.\\\|\\nabla L\(\\theta\_\{a\}\)\-\\nabla L\(\\theta\_\{b\}\)\\\|\_\{2\}\\leq M\\\|\\theta\_\{a\}\-\\theta\_\{b\}\\\|\_\{2\}\.\(12\)The assumption is local to the step\. It does not require global smoothness of the full dynamical system\.

###### Assumption 2\(Certificate calibration\)\.

The scoreϵ^i\\widehat\{\\epsilon\}\_\{i\}is calibrated as an empirical proxy for current\-step gradient error\. Larger path disagreement, larger finite\-difference residuals, larger solver instability, and stronger stiffness evidence produce a larger radius\. The analysis below is conditional on this radius\. Weak calibration makes the certificate conservative\.

### 3\.2Descent Geometry

The acceptance test is a robust first\-order statement\. The update−g^i\-\\hat\{g\}\_\{i\}is certified only when every gradient in𝒢i\\mathcal\{G\}\_\{i\}gives a negative directional derivative along that update with marginm≥0m\\geq 0:

supg′∈𝒢i⟨g′,−g^i⟩<−m\.\\sup\_\{g^\{\\prime\}\\in\\mathcal\{G\}\_\{i\}\}\\langle g^\{\\prime\},\-\\hat\{g\}\_\{i\}\\rangle<\-m\.\(13\)
###### Lemma 1\(Support function of the gradient error ball\)\.

For𝒢i=\{g′:‖g′−g^i‖2≤ϵ^i\}\\mathcal\{G\}\_\{i\}=\\\{g^\{\\prime\}:\\\|g^\{\\prime\}\-\\hat\{g\}\_\{i\}\\\|\_\{2\}\\leq\\widehat\{\\epsilon\}\_\{i\}\\\},

supg′∈𝒢i⟨g′,−g^i⟩=−‖g^i‖22\+ϵ^i​‖g^i‖2\.\\sup\_\{g^\{\\prime\}\\in\\mathcal\{G\}\_\{i\}\}\\langle g^\{\\prime\},\-\\hat\{g\}\_\{i\}\\rangle=\-\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}^\{2\}\+\\widehat\{\\epsilon\}\_\{i\}\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\.\(14\)

###### Proof\.

Writeg′=g^i\+eg^\{\\prime\}=\\hat\{g\}\_\{i\}\+ewith‖e‖2≤ϵ^i\\\|e\\\|\_\{2\}\\leq\\widehat\{\\epsilon\}\_\{i\}\. Then

⟨g′,−g^i⟩=−‖g^i‖22−⟨e,g^i⟩\.\\langle g^\{\\prime\},\-\\hat\{g\}\_\{i\}\\rangle=\-\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}^\{2\}\-\\langle e,\\hat\{g\}\_\{i\}\\rangle\.\(15\)The supremum occurs wheneeis anti\-parallel tog^i\\hat\{g\}\_\{i\}, giving−⟨e,g^i⟩=ϵ^i​‖g^i‖2\-\\langle e,\\hat\{g\}\_\{i\}\\rangle=\\widehat\{\\epsilon\}\_\{i\}\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\. ∎

###### Proposition 1\(Certified descent margin\)\.

Under the uncertainty set above, condition \([13](https://arxiv.org/html/2609.13204#S3.E13)\) is equivalent to

Δi=‖g^i‖22−ϵ^i​‖g^i‖2−m\>0\.\\Delta\_\{i\}=\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}^\{2\}\-\\widehat\{\\epsilon\}\_\{i\}\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\-m\>0\.\(16\)If Assumption 1 holds, then for a learning rateη\>0\\eta\>0,

L⁡\(θ−η​g^i\)≤L⁡\(θ\)−η⁡\(‖g^i‖22−ϵ^i​‖g^i‖2\)\+M​η22​‖g^i‖22\.L\(\\theta\-\\eta\\hat\{g\}\_\{i\}\)\\leq L\(\\theta\)\-\\eta\\left\(\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}^\{2\}\-\\widehat\{\\epsilon\}\_\{i\}\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\\right\)\+\\frac\{M\\eta^\{2\}\}\{2\}\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}^\{2\}\.\(17\)Thus a positive margin yields a descent certificate for sufficiently smallη\\eta\.

###### Corollary 1\(Rejection is informative\)\.

If‖g^i‖2≤ϵ^i\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\\leq\\widehat\{\\epsilon\}\_\{i\}, the uncertainty ball contains gradients whose projection along−g^i\-\\hat\{g\}\_\{i\}is nonnegative\. In that case, rejecting the update is not a failed optimization step; it is a recorded absence of descent evidence\.

###### Proposition 2\(Monotonicity of evidence\)\.

For a fixed candidate norm‖g^i‖2\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}and marginmm, the descent marginΔi\\Delta\_\{i\}is strictly decreasing in the uncertainty radiusϵ^i\\widehat\{\\epsilon\}\_\{i\}whenever‖g^i‖2\>0\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\>0\. Consequently, adding evidence that increases numerical uncertainty can only make the certificate more conservative, never more permissive\.

###### Proof\.

Equation \([16](https://arxiv.org/html/2609.13204#S3.E16)\) gives

∂Δi∂ϵ^i=−‖g^i‖2\.\\frac\{\\partial\\Delta\_\{i\}\}\{\\partial\\widehat\{\\epsilon\}\_\{i\}\}=\-\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\.\(18\)The derivative is negative for every nonzero candidate gradient\. ∎

###### Proposition 3\(Descent cone\)\.

If the true error satisfies‖g−g^i‖2≤ϵ<‖g^i‖2\\\|g\-\\hat\{g\}\_\{i\}\\\|\_\{2\}\\leq\\epsilon<\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}, then every admissible true gradient lies in a cone aroundg^i\\hat\{g\}\_\{i\}with half\-angleα\\alphasatisfying

sin⁡α≤ϵ‖g^i‖2\.\\sin\\alpha\\leq\\frac\{\\epsilon\}\{\\\|\\hat\{g\}\_\{i\}\\\|\_\{2\}\}\.\(19\)The certificate evaluates direction as well as magnitude\. A low\-norm candidate can be rejected even with small absolute error, while a high\-norm candidate can remain certified under larger numerical uncertainty\.

### 3\.3Directional Evidence

Finite differences are not treated as ground truth\. They are local directional witnesses\. Forvvwith‖v‖2=1\\\|v\\\|\_\{2\}=1, define

Dh​\(θ,v\)=L⁡\(θ\+h​v\)−L⁡\(θ−h​v\)2​h\.D\_\{h\}\(\\theta;v\)=\\frac\{L\(\\theta\+hv\)\-L\(\\theta\-hv\)\}\{2h\}\.\(20\)
###### Proposition 4\(Centered directional consistency\)\.

Ifϕ⁡\(α\)=L⁡\(θ\+α​v\)\\phi\(\\alpha\)=L\(\\theta\+\\alpha v\)is three times continuously differentiable near00, then

Dh​\(θ,v\)=⟨∇L​\(θ\),v⟩\+O⁡\(h2\)\+O⁡\(u/h\),D\_\{h\}\(\\theta;v\)=\\langle\\nabla L\(\\theta\),v\\rangle\+O\(h^\{2\}\)\+O\(u/h\),\(21\)whereuudenotes the floating\-point roundoff scale\. Therefore, a sign disagreement between⟨g^i,v⟩\\langle\\hat\{g\}\_\{i\},v\\rangleandDh​\(θ,v\)D\_\{h\}\(\\theta;v\)is meaningful only whenhhis chosen outside both the truncation\-dominated and roundoff\-dominated regimes\.

###### Corollary 2\(Multi\-direction directional risk\)\.

For independent directionsv1,…,vkv\_\{1\},\\ldots,v\_\{k\}, define

Ak\(g^i\)=1k∑j=1k\[sign⟨g^i,vj⟩=signDh\(θ;vj\)\]\.A\_\{k\}\(\\hat\{g\}\_\{i\}\)=\\frac\{1\}\{k\}\\sum\_\{j=1\}^\{k\}\\mathbf\{1\}\\\!\\left\[\\operatorname\{sign\}\\langle\\hat\{g\}\_\{i\},v\_\{j\}\\rangle=\\operatorname\{sign\}D\_\{h\}\(\\theta;v\_\{j\}\)\\right\]\.\(22\)LowAkA\_\{k\}is direct evidence that the candidate is directionally fragile\. It is evidence rather than proof, but it is more informative than accepting a gradient only because automatic differentiation returned a finite tensor\.

###### Proposition 5\(Directional rejection witness\)\.

Assume there exists a tested directionvvsuch that the centered finite difference is in its consistency regime and

sign⁡⟨g^i,v⟩≠sign⁡Dh​\(θ,v\),\|Dh​\(θ,v\)\|\>ρ,\\operatorname\{sign\}\\langle\\hat\{g\}\_\{i\},v\\rangle\\neq\\operatorname\{sign\}D\_\{h\}\(\\theta;v\),\\qquad\|D\_\{h\}\(\\theta;v\)\|\>\\rho,\(23\)for a toleranceρ\\rholarger than the estimated finite\-difference noise floor\. Then the candidate cannot be certified by directional evidence alone; any acceptance must be justified by stronger path or solver evidence\. GradRepair\-ODE uses this witness to move the candidate from trusted to repairable or unsafe\.

###### Proof\.

In the finite\-difference consistency regime,Dh​\(θ,v\)D\_\{h\}\(\\theta;v\)estimates⟨g,v⟩\\langle g,v\\rangleup to truncation and roundoff errors belowρ\\rho\. A sign mismatch above that floor means the candidate and the local directional witness disagree on whether moving alongvvincreases or decreases the loss\. The candidate may still be correct if the witness is corrupted, but the available evidence is no longer sufficient for trust\. ∎

### 3\.4Repair as Constrained Selection

Letℐ\\mathcal\{I\}be the available set of gradient paths and letCiC\_\{i\}denote the cost of candidateii\. GradRepair\-ODE selects

i⋆∈arg⁡mini∈ℐ⁡Cisubject toΔi\>0,Ak​\(g^i\)≥τ\.i^\{\\star\}\\in\\arg\\min\_\{i\\in\\mathcal\{I\}\}C\_\{i\}\\quad\\text\{subject to\}\\quad\\Delta\_\{i\}\>0,\\quad A\_\{k\}\(\\hat\{g\}\_\{i\}\)\\geq\\tau\.\(24\)If no candidate satisfies the constraints, the optimizer step is rejected\. Repair is constrained selection under numerical evidence, not an unconditional move toward the strictest computation\.

###### Proposition 6\(Selective repair cost\)\.

Let the naive path cost beC0C\_\{0\}, diagnostic costCdC\_\{d\}, repair costCrC\_\{r\}, and always\-strict costCsC\_\{s\}, all measured in function evaluations\. If a fractionppof steps trigger repair, the expected GradRepair cost is

CGR=C0\+Cd\+p​Cr\.C\_\{\\mathrm\{GR\}\}=C\_\{0\}\+C\_\{d\}\+pC\_\{r\}\.\(25\)An always\-strict strategy is more expensive whenever

p<Cs−C0−CdCr\.p<\\frac\{C\_\{s\}\-C\_\{0\}\-C\_\{d\}\}\{C\_\{r\}\}\.\(26\)This gives the cost\-reliability premise of GradRepair\-ODE: use cheap gradients when the evidence is coherent, pay for repair when evidence breaks, and reject when no path produces a certified step\.

###### Proposition 7\(Dominance over warning\-only diagnostics\)\.

Consider two policies that observe the same certificate\. A warning\-only policy always applies the original candidate, while a certificate\-controlled policy applies only candidates satisfyingΔi\>0\\Delta\_\{i\}\>0and rejects otherwise\. On any step where the original candidate hasΔi≤0\\Delta\_\{i\}\\leq 0, the warning\-only policy accepts an uncertified update and the certificate\-controlled policy does not\. Therefore, for a fixed sequence of certificates, certificate control weakly reduces the number of uncertified accepted steps relative to warning\-only diagnostics\.

###### Proof\.

The statement follows directly from the policy definitions\. Steps withΔi\>0\\Delta\_\{i\}\>0may be accepted by both policies\. Steps withΔi≤0\\Delta\_\{i\}\\leq 0are accepted by the warning\-only policy but withheld by the certificate\-controlled policy\. Summing over the sequence gives the weak reduction\. ∎

###### Corollary 3\(Irreparable steps\)\.

For discontinuous or hybrid dynamics, an ordinary smooth\-gradient path may remain uncertified even after recomputation\. Preventing an unsupported update is a valid certified action\.

### 3\.5What the Certificate Establishes

The certificate establishes a step\-local statement:

certified⟺∃i∈ℐsuch thatΔi\>0andAk\(g^i\)≥τ\.\\text\{certified\}\\quad\\Longleftrightarrow\\quad\\exists i\\in\\mathcal\{I\}\\text\{ such that \}\\Delta\_\{i\}\>0\\text\{ and \}A\_\{k\}\(\\hat\{g\}\_\{i\}\)\\geq\\tau\.\(27\)It does not establish global convergence, exact adjoint correctness, or universal solver reliability\. This narrowness is a strength of the formulation\. The object being certified is precisely the object consumed by the optimizer: the next update direction\. When the candidate fails the test, the output is not an ambiguous warning but an optimizer action: repair with a different path, reduce trust in the step, or reject the update\.

The reliability claim can fail at three levels\. At the gradient level, path disagreement and finite\-difference evidence can contradict the candidate\. At the step level, the descent margin can fail\. At the system level, the cost of repair can exceed the value of using a cheap path\. The certification boundary is local, numerical, optimizer\-facing, and still strong enough to change training behavior\.

Appendix A connects the diagnostics to standard inequalities\. Pairwise path disagreement lower\-bounds the error of at least one candidate\. Finite\-difference residuals lower\-bound directional projection error after accounting for the finite\-difference noise floor\. Sensitivity growth follows the variational equation through a Gronwall\-type amplification bound\. These results do not turn the empirical proxy into an oracle; they explain why the proxy uses these numerical quantities\.

## 4Experiments

### 4\.1Systems and Baselines

We evaluate six synthetic systems: a harmonic oscillator, Van der Pol oscillator, Robertson chemical kinetics, Lorenz dynamics, a bouncing ball with event discontinuities, and a neural right\-hand side \(RHS\) system\. The systems cover smooth dynamics, stiffness, chaotic sensitivity, hybrid events, and a small neural vector field\.

The design stresses the gradient interface\. It includes cases where a coarse sensitivity path is harmless, cases where it is directionally wrong, and cases where no smooth\-gradient update is defensible\. We compare a naive coarse solver path, a discrete finer solver path, a checkpointed\-style repair path, and GradRepair\-ODE\. The strict reference is used only to measure agreement\. The same deterministic protocol generates all reported numbers\.

Each system isolates a numerical mechanism that a scientific\-computing reader can inspect\. Smooth periodic dynamics should not trigger repair\. Stiffness should expose tolerance and path sensitivity\. Chaotic dynamics should challenge long\-horizon directionality\. Event discontinuities should prevent an ordinary smooth\-gradient certificate from making an unsupported claim\. The neural RHS checks that the reliability layer stays quiet when the learned vector field is numerically benign\. The benchmark tests mechanism recognition as well as final loss\.

The baselines address different objections\. The naive solver path tests the common training interface in which a returned gradient is applied without further numerical evidence\. Gradient clipping tests whether magnitude control is enough\. Armijo\-style line search tests whether sufficient decrease on the proposed move can substitute for gradient\-source repair\. GradRepair\-ODE faces a stricter test: it must reduce unsafe accepted updates, preserve benign updates, repair cases with coherent numerical alternatives, and reject cases where smooth\-gradient evidence is inadequate\.

### 4\.2Deterministic Protocol

All tables and figures come from the same recorded diagnostic run\. The random seed fixes the initial conditions, parameter perturbations, and finite\-difference directions\. For each system, the experiment records the loss, gradient candidates, cosine similarity to the strict reference, finite\-difference residuals, failure diagnosis, repair action, acceptance decision, wall\-clock time, and function\-evaluation counts\. Training\-level summaries are computed from the saved step records\. The ablation table replays the same evidence, so changes across rows reflect decision logic rather than a different numerical run\.

The protocol centers the optimizer decision instead of a post\-hoc curve\. Repair without cost accounting is incomplete\. Detection without control is incomplete\. Rejecting every suspicious step is safe but uninformative\. The recorded protocol keeps those cases separate\.

### 4\.3Gradient Reliability

Table[1](https://arxiv.org/html/2609.13204#S4.T1)shows the reliability pattern\. Harmonic, Van der Pol, and neural RHS dynamics have cosine agreement close to one and no repair\. Robertson and bouncing\-ball gradients reverse direction under the naive path, with cosine similarity−1\.000\-1\.000against the strict reference\. Lorenz does not fully reverse, but its naive cosine similarity drops to0\.6980\.698\. GradRepair restores Robertson and Lorenz to cosine similarity1\.0001\.000and refuses the event\-discontinuous bouncing\-ball update, where the certificate cannot justify a smooth\-gradient claim\.

Table 1:Gradient reliability across ODE systems\. Gray cells mark quantities that triggered certificate attention: large finite\-difference risk or a nonzero repair rate\. A dash indicates that no repaired gradient was accepted\. Cosine similarity is measured against a strict reference gradient\. Finite\-difference risk islog10⁡\(1\+finite​\-​difference​residual\)\\log\_\{10\}\(1\+\\mathrm\{finite\\mbox\{\-\}difference\\ residual\}\), with the raw residual preserved in the experiment log\.Figure 1:Gradient direction agreement against a strict reference\. GradRepair\-ODE leaves smooth low\-risk systems unchanged, while exposing failure or repair needs in stiff, chaotic, and event\-driven systems\.Table 2:Failure routing induced by the certificate\. Gray cells mark final accepted or rejected interventions selected by the certificate\. The table reports the primary numerical evidence, selected repair action, final optimizer decision, and function\-evaluation multiplier for each system\.
### 4\.4Training Stability

Table[3](https://arxiv.org/html/2609.13204#S4.T3)reports the optimizer\-facing consequence\. The naive loop accepts 37 uncertified updates across Robertson, Lorenz, and bouncing ball\. GradRepair reduces that count to zero\. The method is not conservative by default: low\-risk systems are trusted, Lorenz and Robertson are repaired, and the event\-discontinuous system is rejected\. The intervention occurs where the evidence changes the status of the step\.

Table 3:Training stability summary\. Red cells mark unsafe updates accepted by the naive loop, green cells mark repaired or cleared unsafe counts, yellow cells mark certified rejection, and gray cells mark trusted no\-intervention decisions\. Unsafe accepted counts steps that the naive loop applied despite a non\-trusted certificate\.Figure 2:Loss trajectories under naive training and GradRepair\-ODE\. The goal is reliability control rather than aggressive optimization; rejected steps preserve evidence when no safe gradient is available\.
### 4\.5Ablation and Sensitivity

Table[4](https://arxiv.org/html/2609.13204#S4.T4)isolates the reliability components\. The ablation asks how many unsafe updates remain when detection, repair routing, finite\-difference evidence, or step certification is removed\. Detection without action leaves all unsafe accepted updates in place\. Removing step certification allows the event\-discontinuous case to pass after a failed repair\. Removing repair routing avoids unsafe updates but rejects every non\-trusted step, including repairable Robertson and Lorenz updates\. The full system repairs repairable steps, rejects the irreparable event case, and leaves zero unsafe accepted updates\.

Table 4:Ablation study from the same recorded step evidence\. Red cells mark unsafe accepted updates, green cells mark repaired or cleared unsafe counts, and yellow cells mark rejected updates\.Figure 3:Threshold sensitivity for the certificate decision\. The reliability layer has a stable operating region: stricter thresholds increase repair and rejection, while unsafe accepted steps disappear once suspicious gradients are no longer trusted by default\.Figure 4:Robertson case study\. The naive and discrete paths point in the opposite direction from the strict reference, while GradRepair\-ODE routes to strict recomputation and restores cosine similarity to1\.0001\.000\.
### 4\.6Cost and Classical Safeguards

Table[5](https://arxiv.org/html/2609.13204#S4.T5)reports wall\-clock time and function\-evaluation overhead for the diagnostic step\. Low\-risk systems incur essentially no repair overhead, Lorenz is repaired at moderate cost, and Robertson shows the expensive edge of strict recomputation\. GradRepair\-ODE does not claim that certification is free\. It makes the price of reliable gradients explicit and pays it only where the evidence requires it\.

Table 5:Measured diagnostic cost\. Yellow cells mark time or function\-evaluation multipliers above5×5\\times\. Wall\-clock measurements are generated by the same deterministic experiment script and reported in milliseconds\.Table[6](https://arxiv.org/html/2609.13204#S4.T6)compares GradRepair\-ODE with two standard optimizer safeguards\. Gradient clipping controls magnitude but cannot repair a wrong direction\[[19](https://arxiv.org/html/2609.13204#bib.bib23)\]\. Line search checks whether a proposed move decreases the measured loss\[[1](https://arxiv.org/html/2609.13204#bib.bib21),[17](https://arxiv.org/html/2609.13204#bib.bib22)\], but it does not identify the numerical source of the gradient error and may reject without producing a reliable replacement direction\. GradRepair\-ODE addresses a different failure mode: it uses numerical evidence to decide whether to accept, repair, or reject the gradient before the optimizer consumes it\.

Table 6:Comparison with classical safeguards\. Red cells mark unsafe accepted updates, green cells mark cleared unsafe counts, yellow cells mark rejected updates, and light red cells mark loss spikes\.Figure 5:Per\-step GradRepair decisions\. Trusted steps are left unchanged, repaired steps are routed to a safer gradient path, and rejected steps are withheld when no smooth\-gradient claim is certified\.Figure 6:Repair cost measured as the multiplier in function evaluations relative to the naive gradient path, shown on a logarithmic scale to preserve the full Robertson cost\.Figure 7:Reliability gain measured as cosine improvement relative to the naive gradient path\. The largest gains occur where the naive direction is inconsistent with the strict reference\.Figure 8:Directional finite\-difference residuals\. Large residuals identify candidates whose directional derivative evidence is inconsistent with the proposed gradient\.

## 5Discussion

The experiments show that gradient reliability can be handled at the optimizer boundary\. GradRepair\-ODE changes the update stream: low\-risk harmonic and neural RHS systems pass without intervention, Robertson and Lorenz are repaired to agreement with a strict reference, and the event\-discontinuous system is rejected when the smooth\-gradient evidence is insufficient\. The repair\-controlled loop eliminates unsafe accepted steps in these runs, from3737under the naive policy to00under GradRepair\-ODE\.

The design separates detection, diagnosis, repair, and step acceptance\. A scalar loss curve rarely reveals whether instability comes from the model, the solver, the sensitivity path, or an event discontinuity\. GradRepair\-ODE records finite\-difference residuals, pairwise gradient disagreement, solver failure, repair action, and step acceptance for each update\.

This is more precise than a solver\-tolerance recommendation\. Tight tolerances can be wasteful when the gradient is already coherent, and they do not fix a wrong differentiability assumption, as the event\-discontinuous case shows\. GradRepair\-ODE asks what evidence supports this step\. That question gives the method its cost\-reliability profile: harmonic, Van der Pol, and neural RHS systems do not pay for repair; Robertson and Lorenz do; the bouncing\-ball system is not allowed to turn a discontinuity into a smooth\-gradient update\.

The comparison with clipping and line search clarifies the gap\. Clipping modifies vector length; it does not repair direction\. Line search can reject a bad move, but it does not say whether the failure came from stiffness, chaos, interpolation, event structure, or sensitivity\-path mismatch\. GradRepair\-ODE answers the earlier numerical question: whether the vector used to define the move deserves trust\.

The claim has a clear boundary\. Finite differences are directional checks with truncation and floating\-point error\. The certificate is an empirical proxy, not a formal upper bound\. Within that boundary, the experiments show that a small evidence layer can distinguish trusted, repairable, unsafe, and failed gradient states while preserving the option to refuse a step when repair would overclaim\.

For scientific computing, the practical message is direct\. When a differentiable solver supplies a gradient to an optimizer, it should also supply enough numerical evidence to make the update accountable\. GradRepair\-ODE gives one design for that evidence: multiple gradient candidates, calibrated disagreement, repair routing, and a descent certificate attached to the step itself\.

## 6Limitations and Scope

GradRepair\-ODE is a reliability layer for optimizer steps, not a universal solver correctness theorem\. The experiments are synthetic and diagnostic; they expose numerical failure modes under controlled conditions rather than establish application\-level superiority\. That scope matches the claim: the paper studies whether unreliable ODE gradients can be detected, repaired, or rejected before they become optimizer updates\.

Several boundaries are visible in the formulation\. Directional finite\-difference checks become expensive in very high\-dimensional parameter spaces\. Random directional checks reduce the cost, but they do not replace full\-gradient verification\. Stochastic objectives or noisy mini\-batches require additional calibration because finite\-difference residuals can mix numerical error with sampling noise\. Hybrid systems with event\-time derivatives may require event\-aware adjoints; the bouncing\-ball experiment is therefore rejected instead of repaired\. Strict recomputation can be expensive, as Robertson shows\. The method reports this cost so reliability and computation can be compared directly\.

The controlled benchmark suite should be read as a mechanism study\. Six systems do not cover all scientific machine learning\. They cover the failure modes most likely to corrupt ODE gradients: benign smooth flow, stiffness, chaos, event discontinuity, and neural\-vector\-field training\. A domain application would add modeling assumptions; it would still need the reliability question studied here\.

These limitations define the operating regime for the certificate: deterministic or low\-noise differentiable ODE training, observable solver diagnostics, multiple feasible gradient paths, and optimizer steps whose safety can be evaluated locally\. In that regime, GradRepair\-ODE makes numerical gradient reliability a measured training variable, with direct consequences for repair, rejection, and unsafe accepted updates\. The formulation is conservative when evidence is thin and assertive when evidence is coherent\.

## 7Conclusion

GradRepair\-ODE treats differentiable ODE training as a numerical optimization problem in the literal sense: the optimizer receives gradients produced by approximate trajectories, approximate sensitivity paths, and solver configurations that may be under stress\. A finite tensor is not enough\. By combining gradient\-path disagreement, directional finite\-difference checks, solver diagnostics, repair routing, and descent\-safe step decisions, GradRepair\-ODE makes numerical fragility visible at the update level\.

The evidence in this paper is step\-level\. Low\-risk systems are left alone\. Repairable stiff and chaotic gradients are routed to safer computations\. An event\-discontinuous case is rejected when ordinary smooth\-gradient evidence is insufficient\. The training loop records why a gradient was trusted, how it was repaired, or why it was withheld\. That record is the contribution: an ODE gradient reaches the optimizer with numerical evidence attached\.

## References

- \[1\]L\. Armijo\(1966\)Minimization of functions having lipschitz continuous first partial derivatives\.Pacific Journal of Mathematics16\(1\),pp\. 1–3\.External Links:[Document](https://dx.doi.org/10.2140/pjm.1966.16.1)Cited by:[§A\.8](https://arxiv.org/html/2609.13204#A1.SS8.p1.1),[§4\.6](https://arxiv.org/html/2609.13204#S4.SS6.p2.1)\.
- \[2\]U\. M\. Ascher and L\. R\. Petzold\(1998\)Computer methods for ordinary differential equations and differential\-algebraic equations\.SIAM\.External Links:[Document](https://dx.doi.org/10.1137/1.9781611971392)Cited by:[§A\.6](https://arxiv.org/html/2609.13204#A1.SS6.p1.1),[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[3\]K\. E\. Brenan, S\. L\. Campbell, and L\. R\. Petzold\(1996\)Numerical solution of initial\-value problems in differential\-algebraic equations\.Classics in Applied Mathematics, Vol\.14,SIAM\.External Links:[Document](https://dx.doi.org/10.1137/1.9781611971224)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[4\]R\. T\. Q\. Chen, Y\. Rubanova, J\. Bettencourt, and D\. K\. Duvenaud\(2018\)Neural ordinary differential equations\.InAdvances in Neural Information Processing Systems,Vol\.31\.External Links:[Link](https://proceedings.neurips.cc/paper/2018/hash/69386f6bb1dfed68692a24c8686939b9-Abstract.html)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p1.1),[§1](https://arxiv.org/html/2609.13204#S1.p2.1)\.
- \[5\]J\. R\. Dormand and P\. J\. Prince\(1980\)A family of embedded Runge–Kutta formulae\.Journal of Computational and Applied Mathematics6\(1\),pp\. 19–26\.External Links:[Document](https://dx.doi.org/10.1016/0771-050X%2880%2990013-3)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[6\]A\. Gholaminejad, K\. Keutzer, and G\. Biros\(2019\)ANODE: unconditionally accurate memory\-efficient gradients for neural ODEs\.InProceedings of the Twenty\-Eighth International Joint Conference on Artificial Intelligence,pp\. 730–736\.External Links:[Document](https://dx.doi.org/10.24963/ijcai.2019/103)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[7\]W\. Grathwohl, R\. T\. Q\. Chen, J\. Bettencourt, I\. Sutskever, and D\. Duvenaud\(2019\)FFJORD: free\-form continuous dynamics for scalable reversible generative models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rJxgknCcK7)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p2.1)\.
- \[8\]A\. Griewank and A\. Walther\(2000\)Algorithm 799: Revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation\.ACM Transactions on Mathematical Software26\(1\),pp\. 19–45\.External Links:[Document](https://dx.doi.org/10.1145/347837.347846)Cited by:[§2\.2](https://arxiv.org/html/2609.13204#S2.SS2.p1.1)\.
- \[9\]E\. Hairer, S\. P\. Nørsett, and G\. Wanner\(1993\)Solving ordinary differential equations I: nonstiff problems\.2 edition,Springer Series in Computational Mathematics, Vol\.8,Springer\.External Links:[Document](https://dx.doi.org/10.1007/978-3-540-78862-1)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[10\]E\. Hairer and G\. Wanner\(1996\)Solving ordinary differential equations II: stiff and differential\-algebraic problems\.2 edition,Springer Series in Computational Mathematics, Vol\.14,Springer\.External Links:[Document](https://dx.doi.org/10.1007/978-3-642-05221-7)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[11\]N\. J\. Higham\(2002\)Accuracy and stability of numerical algorithms\.2 edition,SIAM\.External Links:ISBN 978\-0\-89871\-521\-7Cited by:[§A\.3](https://arxiv.org/html/2609.13204#A1.SS3.p1.3),[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[12\]A\. C\. Hindmarsh, P\. N\. Brown, K\. E\. Grant, S\. L\. Lee, R\. Serban, D\. E\. Shumaker, and C\. S\. Woodward\(2005\)SUNDIALS: suite of nonlinear and differential/algebraic equation solvers\.ACM Transactions on Mathematical Software31\(3\),pp\. 363–396\.External Links:[Document](https://dx.doi.org/10.1145/1089014.1089020)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1),[§2\.2](https://arxiv.org/html/2609.13204#S2.SS2.p1.1)\.
- \[13\]J\. Ho, A\. Jain, and P\. Abbeel\(2020\)Denoising diffusion probabilistic models\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 6840–6851\.External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p2.1)\.
- \[14\]P\. Kidger, R\. T\. Q\. Chen, and T\. J\. Lyons\(2021\)“Hey, that’s not an ODE”: faster ODE adjoints via seminorms\.InProceedings of the 38th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.139,pp\. 5443–5452\.External Links:[Link](https://proceedings.mlr.press/v139/kidger21a.html)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p2.1)\.
- \[15\]Y\. Lipman, R\. T\. Q\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. Le\(2023\)Flow matching for generative modeling\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=PqvMRDCJT9t)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p2.1)\.
- \[16\]X\. Liu, C\. Gong, and Q\. Liu\(2023\)Flow straight and fast: learning to generate and transfer data with rectified flow\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=XVjTT1nw5z)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p2.1)\.
- \[17\]J\. Nocedal and S\. J\. Wright\(2006\)Numerical optimization\.2 edition,Springer Series in Operations Research and Financial Engineering,Springer\.External Links:[Document](https://dx.doi.org/10.1007/978-0-387-40065-5)Cited by:[§A\.8](https://arxiv.org/html/2609.13204#A1.SS8.p1.1),[§4\.6](https://arxiv.org/html/2609.13204#S4.SS6.p2.1)\.
- \[18\]D\. Onken and L\. Ruthotto\(2020\)Discretize\-optimize vs\. optimize\-discretize for time\-series regression and continuous normalizing flows\.External Links:2005\.13420,[Document](https://dx.doi.org/10.48550/arXiv.2005.13420)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[19\]R\. Pascanu, T\. Mikolov, and Y\. Bengio\(2013\)On the difficulty of training recurrent neural networks\.InProceedings of the 30th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.28,pp\. 1310–1318\.External Links:[Link](https://proceedings.mlr.press/v28/pascanu13.html)Cited by:[§A\.8](https://arxiv.org/html/2609.13204#A1.SS8.p1.1),[§4\.6](https://arxiv.org/html/2609.13204#S4.SS6.p2.1)\.
- \[20\]SciPy Developersscipy\.integrate\.solve\_ivp documentation\.Note:[https://docs\.scipy\.org/doc/scipy/reference/generated/scipy\.integrate\.solve\_ivp\.html](https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.solve_ivp.html)Accessed 2026\-08\-14Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[21\]R\. Serban and A\. C\. Hindmarsh\(2005\)CVODES: the sensitivity\-enabled ODE solver in SUNDIALS\.InProceedings of the ASME 2005 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference,Long Beach, California, USA\.External Links:[Document](https://dx.doi.org/10.1115/DETC2005-85597)Cited by:[§A\.6](https://arxiv.org/html/2609.13204#A1.SS6.p1.1),[§2\.2](https://arxiv.org/html/2609.13204#S2.SS2.p1.1)\.
- \[22\]L\. F\. Shampine and M\. W\. Reichelt\(1997\)The MATLAB ODE suite\.SIAM Journal on Scientific Computing18\(1\),pp\. 1–22\.External Links:[Document](https://dx.doi.org/10.1137/S1064827594276424)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p3.1)\.
- \[23\]Y\. Song, J\. Sohl\-Dickstein, D\. P\. Kingma, A\. Kumar, S\. Ermon, and B\. Poole\(2021\)Score\-based generative modeling through stochastic differential equations\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=PxTIG12RRHS)Cited by:[§1](https://arxiv.org/html/2609.13204#S1.p2.1)\.

## Appendix AAdditional Mathematical Details

### A\.1From Gradient Error to Robust Descent

Letg^\\hat\{g\}be the gradient consumed by the optimizer and let the unknown exact gradient beg=g^\+eg=\\hat\{g\}\+ewith‖e‖2≤ϵ\\\|e\\\|\_\{2\}\\leq\\epsilon\. The first\-order change along the proposed updated=−g^d=\-\\hat\{g\}is

⟨g,d⟩=−‖g^‖22−⟨e,g^⟩\.\\langle g,d\\rangle=\-\\\|\\hat\{g\}\\\|\_\{2\}^\{2\}\-\\langle e,\\hat\{g\}\\rangle\.\(28\)The worst admissible error is anti\-parallel tog^\\hat\{g\}, so

sup‖e‖2≤ϵ⟨g,−g^⟩=−‖g^‖22\+ϵ​‖g^‖2\.\\sup\_\{\\\|e\\\|\_\{2\}\\leq\\epsilon\}\\langle g,\-\\hat\{g\}\\rangle=\-\\\|\\hat\{g\}\\\|\_\{2\}^\{2\}\+\\epsilon\\\|\\hat\{g\}\\\|\_\{2\}\.\(29\)Therefore, the sufficient condition for robust first\-order descent is

‖g^‖2\>ϵ\.\\\|\\hat\{g\}\\\|\_\{2\}\>\\epsilon\.\(30\)With a marginm≥0m\\geq 0, GradRepair\-ODE uses the stricter condition

‖g^‖22−ϵ​‖g^‖2\>m\.\\\|\\hat\{g\}\\\|\_\{2\}^\{2\}\-\\epsilon\\\|\\hat\{g\}\\\|\_\{2\}\>m\.\(31\)This gives the descent\-safe step certificate used in the main text\.

### A\.2Learning\-Rate Dependence

Under localMM\-smoothness,

L⁡\(θ−η​g^\)≤L⁡\(θ\)\+η⁡⟨g,−g^⟩\+M​η22​‖g^‖22\.L\(\\theta\-\\eta\\hat\{g\}\)\\leq L\(\\theta\)\+\\eta\\langle g,\-\\hat\{g\}\\rangle\+\\frac\{M\\eta^\{2\}\}\{2\}\\\|\\hat\{g\}\\\|\_\{2\}^\{2\}\.\(32\)Substituting the worst\-case bound gives

L⁡\(θ−η​g^\)−L⁡\(θ\)≤−η​‖g^‖2​\(‖g^‖2−ϵ\)\+M​η22​‖g^‖22\.L\(\\theta\-\\eta\\hat\{g\}\)\-L\(\\theta\)\\leq\-\\eta\\\|\\hat\{g\}\\\|\_\{2\}\(\\\|\\hat\{g\}\\\|\_\{2\}\-\\epsilon\)\+\\frac\{M\\eta^\{2\}\}\{2\}\\\|\\hat\{g\}\\\|\_\{2\}^\{2\}\.\(33\)Thus a certified direction remains a descent step for any

0<η<2​\(‖g^‖2−ϵ\)M​‖g^‖2\.0<\\eta<\\frac\{2\(\\\|\\hat\{g\}\\\|\_\{2\}\-\\epsilon\)\}\{M\\\|\\hat\{g\}\\\|\_\{2\}\}\.\(34\)GradRepair\-ODE does not need to estimateMMexactly; the inequality explains why the step certificate and the learning\-rate scale interact\.

### A\.3Directional Finite\-Difference Error

Forϕ⁡\(α\)=L⁡\(θ\+α​v\)\\phi\(\\alpha\)=L\(\\theta\+\\alpha v\), Taylor expansion gives

ϕ⁡\(h\)\\displaystyle\\phi\(h\)=ϕ⁡\(0\)\+h​ϕ′​\(0\)\+h22​ϕ′′​\(0\)\+h36​ϕ′′′​\(ξ\+\),\\displaystyle=\\phi\(0\)\+h\\phi^\{\\prime\}\(0\)\+\\frac\{h^\{2\}\}\{2\}\\phi^\{\\prime\\prime\}\(0\)\+\\frac\{h^\{3\}\}\{6\}\\phi^\{\\prime\\prime\\prime\}\(\\xi\_\{\+\}\),\(35\)ϕ⁡\(−h\)\\displaystyle\\phi\(\-h\)=ϕ⁡\(0\)−h​ϕ′​\(0\)\+h22​ϕ′′​\(0\)−h36​ϕ′′′​\(ξ−\)\.\\displaystyle=\\phi\(0\)\-h\\phi^\{\\prime\}\(0\)\+\\frac\{h^\{2\}\}\{2\}\\phi^\{\\prime\\prime\}\(0\)\-\\frac\{h^\{3\}\}\{6\}\\phi^\{\\prime\\prime\\prime\}\(\\xi\_\{\-\}\)\.\(36\)Subtracting and dividing by2​h2hyields

ϕ⁡\(h\)−ϕ⁡\(−h\)2​h=ϕ′​\(0\)\+O⁡\(h2\)\.\\frac\{\\phi\(h\)\-\\phi\(\-h\)\}\{2h\}=\\phi^\{\\prime\}\(0\)\+O\(h^\{2\}\)\.\(37\)Floating\-point evaluation adds a competingO⁡\(u/h\)O\(u/h\)term, whereuuis the roundoff scale\[[11](https://arxiv.org/html/2609.13204#bib.bib11)\]\. This produces the classical finite\-difference tradeoff: too large a step is truncation dominated, while too small a step is roundoff dominated\. GradRepair\-ODE therefore uses finite differences as directional evidence, not as absolute ground truth\.

### A\.4Path Disagreement as an Error Witness

Letggdenote the exact gradient and letg^a,g^b\\hat\{g\}\_\{a\},\\hat\{g\}\_\{b\}be two independently computed candidates\. Even without knowinggg, their disagreement imposes a deterministic lower bound on the error of at least one candidate:

max⁡\{‖g^a−g‖2,‖g^b−g‖2\}≥12​‖g^a−g^b‖2\.\\max\\\{\\\|\\hat\{g\}\_\{a\}\-g\\\|\_\{2\},\\\|\\hat\{g\}\_\{b\}\-g\\\|\_\{2\}\\\}\\geq\\frac\{1\}\{2\}\\\|\\hat\{g\}\_\{a\}\-\\hat\{g\}\_\{b\}\\\|\_\{2\}\.\(38\)This follows immediately from the triangle inequality,

‖g^a−g^b‖2≤‖g^a−g‖2\+‖g^b−g‖2\.\\\|\\hat\{g\}\_\{a\}\-\\hat\{g\}\_\{b\}\\\|\_\{2\}\\leq\\\|\\hat\{g\}\_\{a\}\-g\\\|\_\{2\}\+\\\|\\hat\{g\}\_\{b\}\-g\\\|\_\{2\}\.\(39\)The inequality gives a direct reason to inspect path disagreement\. A large separation between two sensitivity paths proves that at least one path is inaccurate at the current step\. It does not identify which path is wrong, so GradRepair\-ODE combines disagreement with directional checks and solver diagnostics before selecting a repair\.

### A\.5Finite\-Difference Residual and Projection Error

Letqv=⟨g,v⟩q\_\{v\}=\\langle g,v\\ranglebe the exact directional derivative and letDh​\(θ,v\)D\_\{h\}\(\\theta;v\)be the centered finite\-difference estimate\. Suppose the finite\-difference error alongvvis bounded byβh\\beta\_\{h\}:

\|Dh​\(θ,v\)−qv\|≤βh\.\|D\_\{h\}\(\\theta;v\)\-q\_\{v\}\|\\leq\\beta\_\{h\}\.\(40\)For a candidateg^\\hat\{g\}, the observed residual satisfies

\|⟨g^,v⟩−Dh​\(θ,v\)\|≤\|⟨g^−g,v⟩\|\+βh,\|\\langle\\hat\{g\},v\\rangle\-D\_\{h\}\(\\theta;v\)\|\\leq\|\\langle\\hat\{g\}\-g,v\\rangle\|\+\\beta\_\{h\},\(41\)and therefore

\|⟨g^−g,v⟩\|≥\|⟨g^,v⟩−Dh​\(θ,v\)\|−βh\.\|\\langle\\hat\{g\}\-g,v\\rangle\|\\geq\|\\langle\\hat\{g\},v\\rangle\-D\_\{h\}\(\\theta;v\)\|\-\\beta\_\{h\}\.\(42\)Thus a large finite\-difference residual, after accounting for the finite\-difference noise floor, lower\-bounds the candidate’s error in at least one tested direction\. This is why the certificate treats finite\-difference residuals as directional witnesses\.

### A\.6Sensitivity Growth and Long\-Horizon Risk

For a smooth ODE, the state sensitivityS⁡\(t\)=∂x⁡\(t\)/∂θS\(t\)=\\partial x\(t\)/\\partial\\thetasatisfies the variational equation used in classical sensitivity analysis\[[2](https://arxiv.org/html/2609.13204#bib.bib10),[21](https://arxiv.org/html/2609.13204#bib.bib18)\]:

d​Sd​t=Jx​\(t\)​S​\(t\)\+Jθ​\(t\),Jx​\(t\)=∂f∂x​\(t,x⁡\(t\),θ\),Jθ​\(t\)=∂f∂θ​\(t,x⁡\(t\),θ\)\.\\frac\{dS\}\{dt\}=J\_\{x\}\(t\)S\(t\)\+J\_\{\\theta\}\(t\),\\qquad J\_\{x\}\(t\)=\\frac\{\\partial f\}\{\\partial x\}\(t,x\(t\),\\theta\),\\quad J\_\{\\theta\}\(t\)=\\frac\{\\partial f\}\{\\partial\\theta\}\(t,x\(t\),\\theta\)\.\(43\)Gronwall’s inequality gives the bound

‖S⁡\(t\)‖2≤exp⁡\(∫t0t‖Jx​\(s\)‖2​𝑑s\)​\(‖S⁡\(t0\)‖2\+∫t0t‖Jθ​\(r\)‖2​𝑑r\)\.\\\|S\(t\)\\\|\_\{2\}\\leq\\exp\\\!\\left\(\\int\_\{t\_\{0\}\}^\{t\}\\\|J\_\{x\}\(s\)\\\|\_\{2\}\\,ds\\right\)\\left\(\\\|S\(t\_\{0\}\)\\\|\_\{2\}\+\\int\_\{t\_\{0\}\}^\{t\}\\\|J\_\{\\theta\}\(r\)\\\|\_\{2\}\\,dr\\right\)\.\(44\)We do not use this inequality as a tight numerical estimate\. Large Jacobian norms, stiff transients, and long horizons can amplify small trajectory or sensitivity errors into large gradient discrepancies\. The certificate’s stiffness and solver\-instability terms follow this variational structure\.

### A\.7Ablation Replay Definitions

The ablation table replays the same recorded diagnostic evidence under counterfactual decision policies\. The ODE solves, gradient candidates, finite\-difference residuals, and step labels are held fixed, so each row changes the reliability mechanism rather than the numerical episode\. In the detect\-only replay, the certificate is observed but the naive update is still applied; this measures the gap between warning and control\. Removing finite\-difference evidence suppresses the event\-discontinuity signal and shows why path agreement alone can be over\-permissive\. Removing repair routing converts every non\-trusted step into rejection, which exposes the cost of treating reliability as a binary gate\. Removing step certification applies repaired gradients even when the final margin remains unsafe, isolating the last optimizer\-level decision\. The full GradRepair\-ODE policy accepts trusted gradients, repairs repairable gradients, and rejects gradients whose descent direction is not certified\. The replay compares decision logic without changing the underlying numerical evidence\.

### A\.8Safeguard Definitions

The clipping baseline rescales the naive gradient to unit norm when its Euclidean norm exceeds one, following the standard role of clipping as magnitude control\[[19](https://arxiv.org/html/2609.13204#bib.bib23)\]\. The line\-search baseline uses a backtracking sufficient\-decrease check inspired by classical Armijo\-style methods\[[1](https://arxiv.org/html/2609.13204#bib.bib21),[17](https://arxiv.org/html/2609.13204#bib.bib22)\]\. Both safeguards operate on the proposed update\. They do not compute a gradient reliability certificate, compare sensitivity paths, or repair the numerical source of a wrong gradient direction\.