Hamiltonian Neural Networks from a Differential Geometry Perspective

# Hamiltonian Neural Networks from a Differential Geometry Perspective Source: [https://abscondita.com/blog/symplectic-sledgehammer-for-a-spring](https://abscondita.com/blog/symplectic-sledgehammer-for-a-spring) *Because when given the simplest nail in the universe, sometimes you just need a nuclear powered sledgehammer\.* We have committed a crime\. What do you do when you have a frontier concept that is just a few years old and is already complicated enough? Well, you set up a Tinder date between that concept and one of the most haunting\[·\]branches of mathematics, of course\. So we are taking a look at Hamiltonian Neural Networks from a differential\-geometry perspective\. Look, we didn't ask for this\. But nowadays it seems every single piece of ML research boils down to the same thing:  Do they get the job done? Usually — at significant compute cost\. But they might as well be the single most uninspired architecture we have\. So this time we're doing the opposite\. We're going to chase something beautiful and let the*structure of the problem*do the heavy lifting, instead of throwing a billion parameters at it and praying to hyperparameter deities that our losses converge\. ★ Achievement Unlocked Offended the Entire ML Community We are not the first to think the aesthetics matter: > “It is more important to have beauty in one's equations than to have them fit experiment\. ” —Paul Dirac, Physicist, professional aesthete We are going to work on what is possibly the simplest physical system in the universe — the kind engineering students see in their second physics course and never think about again\. And to talk about it, we are going to need symplectic manifolds, differential forms, the Lie derivative, Cartan's magic formula, the Poisson bracket, and Noether's theorem\. Mathematical overkill level3/10 concept: Differential Geometry If that ratio of machinery\-to\-mass strikes you as deranged, we never claimed otherwise\. But we promise: somewhere underneath this absurdity is a genuinely beautiful idea about machine learning — you can teach a neural network to obey a conservation law**it is never shown**, purely by changing the*shape*of the thing it is allowed to learn\. ℹINFO Every terrifying word above has a precise, checkable meaning, and we are going to define each one in terms of things that are easier to digest — partial derivatives, dot products, determinants, the chain rule — and then*compute*with it, instead of waving our hands and hoping you nod along\. If you can differentiatex2x^2and remember what a determinant is, you should be able to follow**every line**here\. The whole post runs on six tools: - a**partial derivative**∂H∂q\\tfrac\{\\partial H\}\{\\partial q\}— howHHchanges when you nudgeqqand freeze everything else; - the**gradient**∇H=\(∂H∂q,∂H∂p\)\\nabla H = \\left\(\\tfrac\{\\partial H\}\{\\partial q\}, \\tfrac\{\\partial H\}\{\\partial p\}\\right\)— every partial stacked into one vector; - the**dot product**, for turning two vectors into a number; - the multivariable**chain rule**,ddtH\(q\(t\),p\(t\)\)=∂H∂qdqdt\+∂H∂pdpdt\\tfrac\{d\}\{dt\}H\(q\(t\),p\(t\)\) = \\tfrac\{\\partial H\}\{\\partial q\}\\tfrac\{dq\}\{dt\} \+ \\tfrac\{\\partial H\}\{\\partial p\}\\tfrac\{dp\}\{dt\}; - the**determinant**of a2×22\\times 2matrix, read as a*signed area*; - **Green's theorem**, which trades a loop integral for the area it encloses\. Everything else — forms, the symplectic structure, Poisson brackets — we build out of those, in front of you\. A bit like a cooking show, I guess\. ## The crime scene Here's the setup that started it\. Take that mass on a spring\. Unit mass, unit spring constant, no friction\. Its entire universe is two numbers: positionqqand momentumpp\. The total energy is H\(q,p\)=12p2⏟kinetic\+12q2⏟potentialH\(q, p\) = \\underbrace\{\\tfrac\{1\}\{2\} p^\{2\}\}\_\{\\text\{kinetic\}\} \\;\+\\; \\underbrace\{\\tfrac\{1\}\{2\} q^\{2\}\}\_\{\\text\{potential\}\} and the equations of motion are the cleanest things in all of physics\. Heredqdt\\tfrac\{dq\}\{dt\}is the velocity \(the rate position changes\) anddpdt\\tfrac\{dp\}\{dt\}is the force \(the rate momentum changes\)\. The first equation below just says*velocity equals momentum*— true when the mass is11\. The second is Hooke's law,F=−kqF = \-kq: pull the mass to the right, and the spring hauls it back left\. dqdt=p,dpdt=−q\\frac\{dq\}\{dt\} = p, \\qquad \\frac\{dp\}\{dt\} = \-q Watch the two pictures move together — the mass bobbing in real space on the left, and the state\(q,p\)\(q, p\)tracing a closed loop on the right\. That loop, the system drawn in its own coordinates, is the**phase space**we'll obsess over for the rest of the post: Now do the obvious machine\-learning thing\. NO\. Nooooo\. Put down the transformer\. Not**that**obvious thing\. Collect a pile of\(q,p\)\(q, p\)states and their measured time\-derivatives\(dqdt,dpdt\)\(\\tfrac\{dq\}\{dt\}, \\tfrac\{dp\}\{dt\}\), and train a perfectly reasonable little MLP to map one to the other\. It fits the training data to five decimal places\. You feel great\. You leave your room and tell your mom "Mama, I did it," as she gives you a confused look\. Then you let it predict forward in time for a few hundred swings, and oh no… It does not stay constant\. The orbit, which should be a perfect closed circle in phase space, slowly**spirals**— either inward until the mass grinds to a halt, or outward until your frictionless spring is somehow flinging a mass around with more energy than you gave it\. Your neural network has, depending on the sign of its errors, either invented heat death or built a perpetual\-motion machine\. ℹINFO The network is not bad at*physics*\. It is bad at*geometry*\. It learned a vector field with no reason to conserve anything, because nothing in its architecture knew that "conservation" was even a category of thing\. So it's not the model's fault\. It's yours\. Fixing that is not a matter of more data or a cleverer loss; it's a matter of giving the network the right*shape*to learn into\. That shape has a name, and the name is*symplectic*\. So here is the plan\. We are going to explain why that spiral is inevitable for a naive model — and how to make it*impossible*— three times over: once as**geometry**, once as a**learning problem**, and once as a**symmetry**\. They turn out to be the same sentence wearing three different hats\. Helmets on\. --- ## Movement I — The arena is a manifold ## Phase space is not a plane Physics is often built on small white lies we tell ourselves to make a problem easier to hold\. It's why, when you ask a physicist "what is spin," they say "it's like a ball spinning, except it's not a ball and it isn't spinning\." ★ Achievement Unlocked Offended the Entire Physics Community Too One such lie we tell undergrads is that the state of the spring "is a point in the\(q,p\)\(q, p\)plane\." It's close enough to compute with — but calling phase space "a plane" is doing about as much heavy lifting as calling a 747 "a plane\." Same word; wildly different object underneath\. Here is the honest version, and then immediately the plain\-language version\. Position lives in a*configuration space*QQ\(for the spring, just the line of possible positions\)\. Momentum is**not**another position — it's the thing you pair*with*a velocity to extract a number \(a kinetic energy\)\. An object whose whole job is "eat a vector, return a number" is a**covector**, and it lives in a separate space, the dual\. Glue one dual space onto every point ofQQand you get the**cotangent bundle**T∗QT^\{\*\}Q\.*That*is phase space\. ℹINFO If "covector" made your eyes glaze, good news — you already use one constantly\. In ML, when you writeΔL≈∇L⋅Δw\\Delta L \\approx \\nabla L \\cdot \\Delta w, the gradient∇L\\nabla Lisn't really "an arrow in weight space\." It's the gadget you*feed a step*Δw\\Delta win order to get back a*number*, the change in loss\. That "eat a vector → return a number" gadget**is**a covector \(a**1\-form**\)\. A plain vector is a direction you move; a covector is a ruler you measure against\. Momentum is a ruler, not a move — that's the entire reason phase space isn't a naive plane\. Mathematical overkill level5/10 concept: The Cotangent Bundle T\*Q This sounds like pedantry until it bites\. Ask the question the naive picture cannot answer:*what does it even mean for a flow to "preserve volume" here, when the two axes carry different units?*Position times momentum is an**action**, not an area — you can't lay a ruler across phase space\. You need a purpose\-built instrument\. Building that instrument, carefully, is the rest of this movement\. ## The instrument: a symplectic form That instrument is the**symplectic form**ω\\omega\. In canonical coordinates it is ω=dq∧dp\.\\omega = dq \\wedge dp \. We'll define the pieces from the ground up\. A**1\-form**likedqdqis a little machine that eats a vectorv=\(vq,vp\)v=\(v\_q,v\_p\)and reports one number — here, how far you moved inqq:dq\(v\)=vqdq\(v\)=v\_q\. A general 1\-form isα=adq\+bdp\\alpha = a\\,dq \+ b\\,dp, acting byα\(v\)=avq\+bvp\\alpha\(v\) = a\\,v\_q \+ b\\,v\_p\. With that, the differential of our energy is nothing but the total\-derivative formula you already know, finally given a name: dH=∂H∂qdq\+∂H∂pdp\.dH = \\frac\{\\partial H\}\{\\partial q\}\\,dq \+ \\frac\{\\partial H\}\{\\partial p\}\\,dp \. A**2\-form**likedq∧dpdq \\wedge dpeats*two*vectors and returns a number — and this particular one returns the**signed area**of the parallelogram they span: ℹINFOω\(u,v\)=uqvp−upvq=det⁡\(uqvqupvp\)\\omega\(u, v\) = u\_q v\_p \- u\_p v\_q = \\det\\begin\{pmatrix\} u\_q & v\_q \\\\ u\_p & v\_p \\end\{pmatrix\} That's just the2×22\\times 2determinant you've seen a hundred times — the area of the parallelogram with sidesuuandvv\. Two facts read straight off it, both of which we cash in later: - **Antisymmetry:**swap the inputs and the sign flips,ω\(v,u\)=−ω\(u,v\)\\omega\(v,u\) = \-\\omega\(u,v\)\. - **Zero on repeats:**feed it the same vector twice and you getω\(v,v\)=0\\omega\(v,v\)=0— a "parallelogram" with two identical sides is flat, it has no area\. Tattoo that second fact somewhere visible\. It is, almost single\-handedly, why energy is conserved\. ω\\omegacarries two structural properties: - it is**closed**, writtendω=0d\\omega = 0— there's no "source" of area anywhere \(we cash this in exactly once, in a slick proof later\), and - it is**non\-degenerate**— the only vector that measures zero against*every*other vector is the zero vector itself\. Non\-degeneracy is the magic clause, because it makesω\\omegaa perfect, invertible**dictionary between vectors and 1\-forms**: hand it a vector and you get a 1\-form; because nothing is invisible to it, you can run the dictionary backwards and turn any 1\-form into exactly one vector\. A metricgg\(think: the ordinary dot product\) gives the*same*kind of dictionary — that's literally how the gradient∇H\\nabla His defined, by translating the 1\-formdHdHinto a vector\. The only difference is thatggis**symmetric**whileω\\omegais**antisymmetric**, and that one sign flip is the whole post\. ## The 90° turn, derived \(not asserted\) Our network learns a scalarHH, whose differentialdHdHis a 1\-form — at every point it points "uphill" on the energy landscape\. We now turn that 1\-form into a flow, throughω\\omega\. The vector fieldXHX\_Hwe want is*defined*by one deceptively small equation: ιXHω=dH\.\\iota\_\{X\_H\}\\,\\omega = dH \. Let's not admire it — let's solve it\. The symbolιX\\iota\_X\(the**interior product**\) just means "plugXXinto the first slot ofω\\omegaand leave the second open," which leaves a 1\-form behind\. Compute it directly: \(ιXω\)\(v\)=ω\(X,v\)=Xqvp−Xpvq=\(−Xp\)vq\+\(Xq\)vp⟹ιXω=−Xpdq\+Xqdp\.\(\\iota\_X \\omega\)\(v\) = \\omega\(X, v\) = X\_q v\_p \- X\_p v\_q = \(\-X\_p\)\\,v\_q \+ \(X\_q\)\\,v\_p \\;\\Longrightarrow\\; \\iota\_X\\omega = \-X\_p\\,dq \+ X\_q\\,dp \. Now set that equal todH=∂H∂qdq\+∂H∂pdpdH = \\tfrac\{\\partial H\}\{\\partial q\}\\,dq \+ \\tfrac\{\\partial H\}\{\\partial p\}\\,dpand match thedqdqanddpdppieces: −Xp=∂H∂q,Xq=∂H∂p⟹XH=\(∂H∂p,−∂H∂q\)\.\-X\_p = \\frac\{\\partial H\}\{\\partial q\}, \\qquad X\_q = \\frac\{\\partial H\}\{\\partial p\} \\quad\\Longrightarrow\\quad X\_H = \\left\( \\frac\{\\partial H\}\{\\partial p\},\\; \-\\frac\{\\partial H\}\{\\partial q\} \\right\)\. There they are —**Hamilton's equations, derived in three lines\.**Plug in the spring'sH=12\(q2\+p2\)H = \\tfrac12\(q^2\+p^2\)and you getXH=\(p,−q\)X\_H = \(p, \-q\), exactlydqdt=p,dpdt=−q\\tfrac\{dq\}\{dt\} = p,\\ \\tfrac\{dp\}\{dt\} = \-q\. And now the "90° rotation" is not a slogan but a computation\. Stack the two dictionaries side by side: ∇H=\(∂H∂q,∂H∂p\),XH=\(∂H∂p,−∂H∂q\)=\(01−10\)⏟J∇H\.\\nabla H = \\left\( \\frac\{\\partial H\}\{\\partial q\}, \\frac\{\\partial H\}\{\\partial p\} \\right\), \\qquad X\_H = \\left\( \\frac\{\\partial H\}\{\\partial p\}, \-\\frac\{\\partial H\}\{\\partial q\} \\right\) = \\underbrace\{\\begin\{pmatrix\} 0 & 1 \\\\ \-1 & 0 \\end\{pmatrix\}\}\_\{J\}\\,\\nabla H \. That matrixJJrotates any vector by−90∘\-90^\\circ\. SoXHX\_His*literally the gradient, turned a quarter turn\.*The gradient climbs straight up the energy bowl; rotate it and it runs*along*the contour rings instead\. Drag the white dot below and flip between the two dictionaries — samedHdH, same rings, opposite fate: ## Why the quarter\-turn conserves energy Here's the payoff, short enough to do twice\. How fast does energy change as the system flows alongXHX\_H? Multivariable chain rule, then substitutedqdt=∂H/∂p\\tfrac\{dq\}\{dt\} = \\partial H/\\partial panddpdt=−∂H/∂q\\tfrac\{dp\}\{dt\} = \-\\partial H/\\partial q: dHdt=∂H∂qdqdt\+∂H∂pdpdt=∂H∂q∂H∂p\+∂H∂p\(−∂H∂q\)=0\.\\frac\{dH\}\{dt\} = \\frac\{\\partial H\}\{\\partial q\}\\,\\frac\{dq\}\{dt\} \+ \\frac\{\\partial H\}\{\\partial p\}\\,\\frac\{dp\}\{dt\} = \\frac\{\\partial H\}\{\\partial q\}\\frac\{\\partial H\}\{\\partial p\} \+ \\frac\{\\partial H\}\{\\partial p\}\\left\(\-\\frac\{\\partial H\}\{\\partial q\}\\right\) = 0 \. The two terms are identical with opposite signs\. They cancel**for anyHHwhatsoever**— that's the entire content of energy conservation, and you just checked it with first\-year calculus\. The slick, coordinate\-free way to say precisely the same thing is dHdt=dH\(XH\)=ω\(XH,XH\)=0,\\frac\{dH\}\{dt\} = dH\(X\_H\) = \\omega\(X\_H, X\_H\) = 0, where the final step is free becauseω\\omegais antisymmetric, and*anything antisymmetric, fed two copies of one vector, returns zero*— the "zero on repeats" fact from the primer\.**Energy is conserved because a quantity, paired with itself, backwards, is zero\.**We brought a cotangent bundle to a knife fight and the knife wasω\(X,X\)=0\\omega\(X,X\)=0\. ## The flow preserves area, two ways Energy conservation pins each trajectory to one ring\. There's a second, deeper invariant: the flow preserves*area*in phase space\. The beginner\-friendly proof is one line of partial derivatives\. A flow inflates or shrinks volume according to the**divergence**of its vector field, so just compute it: ∇⋅XH=∂∂q⁣\(∂H∂p\)\+∂∂p⁣\(−∂H∂q\)=∂2H∂q∂p−∂2H∂p∂q=0\.\\nabla\\cdot X\_H = \\frac\{\\partial\}\{\\partial q\}\\\!\\left\(\\frac\{\\partial H\}\{\\partial p\}\\right\) \+ \\frac\{\\partial\}\{\\partial p\}\\\!\\left\(\-\\frac\{\\partial H\}\{\\partial q\}\\right\) = \\frac\{\\partial^2 H\}\{\\partial q\\,\\partial p\} \- \\frac\{\\partial^2 H\}\{\\partial p\\,\\partial q\} = 0 \. It vanishes because mixed partials commute\[·\]— the most boring fact in calculus\. A divergence\-free flow is**incompressible**: it can stir phase space but never compress or inflate a patch of it\. That is**Liouville's theorem**, and for a Hamiltonian system you get it for free\. Mathematical overkill level8/10 concept: Cartan's Magic Formula Drop a small square patch of initial conditions into phase space and let it flow\. A real \(Hamiltonian\) flow just turns the patch rigidly — same shape, same area forever; the MLP's divergent field balloons it\. Toggle between them and watch the area readout: For the spring, the Hamiltonian flow is a rigid rotation, so Liouville's theorem here is the thrilling claim that*spinning a shape doesn't change its area\.*We**know\.**But the one\-line proof that makes it obvious for the spring is the identical proof that makes it true for a galaxy of stars or a Hamiltonian Monte Carlo sampler\. That is the whole point of the sledgehammer: the spring is where you can*check, by hand,*that the machinery is honest\. --- ## Movement II — The network never sees an energy Here is the part about Hamiltonian Neural Networks that, in our opinion, just doesn't get enough love\. ## The training target is a lie of omission A[Hamiltonian Neural Network](https://arxiv.org/abs/1906.01563)\(Greydanus, Dzamba & Yosinski, 2019\) is almost insultingly simple to state: instead of a network that outputs the vector field\(dqdt,dpdt\)\(\\tfrac\{dq\}\{dt\}, \\tfrac\{dp\}\{dt\}\), you build a networkHθ\(q,p\)H\_\\theta\(q, p\)that outputs a**single number**, and you*define*the dynamics as its symplectic gradient — the exactιXHω=dH\\iota\_\{X\_H\}\\omega = dHconstruction we derived above, computed with autodiff\. But look hard at what you train it on\. You have trajectory data: states\(q,p\)\(q, p\)and their measured slopes\(dqdt,dpdt\)\(\\tfrac\{dq\}\{dt\}, \\tfrac\{dp\}\{dt\}\)\. You do**not**have energy labels\. Nobody ever measuredHH\. The loss compares the network's*derivatives*to the observed slopes: L\(θ\)=∥∂Hθ∂p−dqdt∥2\+∥−∂Hθ∂q−dpdt∥2\\mathcal\{L\}\(\\theta\) = \\left\\lVert\\, \\frac\{\\partial H\_\\theta\}\{\\partial p\} \- \\frac\{dq\}\{dt\} \\,\\right\\rVert^\{2\} \+ \\left\\lVert\\, \-\\frac\{\\partial H\_\\theta\}\{\\partial q\} \- \\frac\{dp\}\{dt\} \\,\\right\\rVert^\{2\} Every term is a*derivative*of the network\. The network's actual output — the scalarHθH\_\\theta— appears**nowhere**in the loss\. It is never supervised, never compared to a target, never even named in the data\. The Hamiltonian is a**latent variable**: a hidden scalar the network is forced to invent, whose only job is to have the right slopes\. ⚠WARN You are doing supervised regression on the*gradient*of a function while leaving the*function itself*completely unobserved\. It's like being handed thousands of slope readings from across a mountain range and asked to reconstruct the terrain\. You can do it — but only up to one thing slopes can never tell you: the**absolute altitude**\. Sea level is a free parameter\. So is the additive constant inHH\(add55toHHanddHdHdoesn't budge\)\. The dynamics don't care, because they only ever seedHdH, neverHH\. ## "Is there an energy at all?" is a geometry question Now the deep bit, and we are going to earn it rather than assert it\. Recovering a scalar from its slopes is only possible**if the slopes are genuinely the slopes of something\.**Not every field of arrows is a gradient\. So: given a learned fieldX=\(Xq,Xp\)X = \(X\_q, X\_p\), when does a potentialHHexist withX=XHX = X\_H? Run it through the same machine\. The 1\-form attached toXXisιXω=−Xpdq\+Xqdp\\iota\_X\\omega = \-X\_p\\,dq \+ X\_q\\,dp\. For this to equaldHdH, it has to be**closed**— its exterior derivative must vanish\. For any 1\-formα=adq\+bdp\\alpha = a\\,dq \+ b\\,dpthat derivative is dα=\(∂b∂q−∂a∂p\)dq∧dp\.d\\alpha = \\left\(\\frac\{\\partial b\}\{\\partial q\} \- \\frac\{\\partial a\}\{\\partial p\}\\right\) dq \\wedge dp \. Substitutea=−Xpa = \-X\_p,b=Xqb = X\_q: d\(ιXω\)=\(∂Xq∂q\+∂Xp∂p\)dq∧dp=\(∇⋅X\)dq∧dp\.d\(\\iota\_X \\omega\) = \\left\(\\frac\{\\partial X\_q\}\{\\partial q\} \+ \\frac\{\\partial X\_p\}\{\\partial p\}\\right\) dq \\wedge dp = \(\\nabla\\cdot X\)\\; dq \\wedge dp \. So the abstract condition "ιXω\\iota\_X\\omegais closed" is, in plain coordinates, exactly "XXis**divergence\-free\.**" And on a simply\-connected patch like our plane, closed implies exact \(the**Poincaré lemma**\), so a potentialHHexists**if and only if the field has zero divergence\.**The symplectic field\(p,−q\)\(p,\-q\)has divergence∂p/∂q\+∂\(−q\)/∂p=0\\partial p/\\partial q \+ \\partial\(\-q\)/\\partial p = 0and passes\. A generic MLP that regresses\(dqdt,dpdt\)\(\\tfrac\{dq\}\{dt\}, \\tfrac\{dp\}\{dt\}\)directly has no reason on earth to be divergence\-free, so its 1\-form is not closed, so**noHHexists**, so there is nothing for a conservation law to even be*about\.* ℹINFO You can*watch*the obstruction\. By**Green's theorem**, integrating the recovered slope\-form around a closed loop equals the divergence piled up inside it: ∮∂RιXω=∬R\(∇⋅X\)dqdp\.\\oint\_\{\\partial R\} \\iota\_X\\omega = \\iint\_R \(\\nabla\\cdot X\)\\; dq\\,dp \. IfXXis divergence\-free the right side is zero, so the loop integral comes home to where it started and a single\-valuedHHexists\. If not, you get a non\-zero**gap**— the recovered "energy" is path\-dependent, which is mathematician for*it isn't a function at all\.*The slider below dials the divergence; the gap you see is literally that area integral\. Drag the slider below from real physics toward what an MLP learns, and watch the recovered altitude refuse to return to where it started: ✕DANGER A direct\-field MLP doesn't fail to conserve energy because it was under\-trained\. It fails because the field it learned is**not closed**, so the loop integral is path\-dependent, so there is**no single\-valued energy function in existence**for it to conserve\. The HNN conserves energy for the opposite reason: by only ever emittingdHdHof an actual scalar, every field it can possibly produce is exact*by construction*— divergence\-free, area\-preserving, energy\-conserving, with no extra loss term and no penalty\. Conservation isn't learned\. It's the only thing the architecture can express\. The only thing this test can never pin down is the overall "sea level" — the additive constant inHHfrom earlier\. Which is fine, because the dynamics never depended on it\. --- ## Movement III — Conservation is a symmetry in disguise This is the section that makes us question: "If Emmy Noether was alive to see her work utilized by cavemen like us, would she be excited or horrified?" We have energy conservation\. Time for the theorem that says energy was never special to begin with\. Mathematical overkill level10/10 concept: Noether's Theorem \+ the Poisson Bracket ## The Poisson bracket, derived [Noether's theorem](https://en.wikipedia.org/wiki/Noether%27s_theorem)says every continuous**symmetry**of a system has a matching conserved**quantity**, and vice versa\. In our symplectic language it becomes startlingly mechanical\. First, one new gadget — the**Poisson bracket**\[·\]of two functions, defined asω\\omegaevaluated on their two Hamiltonian fields,\{F,G\}=ω\(XF,XG\)\\\{F, G\\\} = \\omega\(X\_F, X\_G\)\. Expand it with the coordinate formulas we already derived \(XF=\(Fp,−Fq\)X\_F = \(F\_p, \-F\_q\), same forGG\): \{F,G\}=ω\(XF,XG\)=\(Fp\)\(−Gq\)−\(−Fq\)\(Gp\)=∂F∂q∂G∂p−∂F∂p∂G∂q\.\\\{F, G\\\} = \\omega\(X\_F, X\_G\) = \(F\_p\)\(\-G\_q\) \- \(\-F\_q\)\(G\_p\) = \\frac\{\\partial F\}\{\\partial q\}\\frac\{\\partial G\}\{\\partial p\} \- \\frac\{\\partial F\}\{\\partial p\}\\frac\{\\partial G\}\{\\partial q\} \. Now the one identity that runs the whole show\. How does*any*quantityGGchange along the dynamics? Chain rule once more, then substitute Hamilton's equations: dGdt=∂G∂qdqdt\+∂G∂pdpdt=∂G∂q∂H∂p−∂G∂p∂H∂q=\{G,H\}\.\\frac\{dG\}\{dt\} = \\frac\{\\partial G\}\{\\partial q\}\\frac\{dq\}\{dt\} \+ \\frac\{\\partial G\}\{\\partial p\}\\frac\{dp\}\{dt\} = \\frac\{\\partial G\}\{\\partial q\}\\frac\{\\partial H\}\{\\partial p\} \- \\frac\{\\partial G\}\{\\partial p\}\\frac\{\\partial H\}\{\\partial q\} = \\\{G, H\\\} \. Read it both ways and Noether falls out for free: - \{G,H\}=0⟺G\\\{G, H\\\} = 0 \\iff G**is conserved**as the system evolves underHH\. - By antisymmetry\{G,H\}=−\{H,G\}\\\{G, H\\\} = \-\\\{H, G\\\}, that's the*same equation*as "HHis unchanged by the flow generated byGG" — i\.e\.GGgenerates a**symmetry**\. Symmetry and conservation are, quite literally, one equation\{G,H\}=0\\\{G, H\\\} = 0read left\-to\-right or right\-to\-left\. Energy is conserved because\{H,H\}=0\\\{H, H\\\} = 0— antisymmetry,*again*, the gift that keeps on giving\. Pick a quantity below and watch its value as the spring swings\. The widget offers three: the**energy**12\(q2\+p2\)\\tfrac12\(q^2\+p^2\)with\{G,H\}=0\\\{G,H\\\}=0, the**position**qqwith\{G,H\}=p\\\{G,H\\\}=p, and a**stretch**qpqpwith\{G,H\}=p2−q2\\\{G,H\\\}=p^2\-q^2\. Only the energy has a vanishing bracket — and only the energy stays flat: ## Uhhh\.\. So? Here is the thing we find genuinely lovely, and the reason the latent scalar from Movement II matters so much: ℹINFO Let's be precise, because the honest version is better than the hype\. The HNN conserves**energy**no matter what, because it always has*some*scalarHθH\_\\thetaand\{Hθ,Hθ\}=0\\\{H\_\\theta, H\_\\theta\\\} = 0— antisymmetry, one final time\. It does**not**automatically conserve momentum or angular momentum; a plainHθ\(q,p\)H\_\\theta\(q,p\)has no reason to\. But here is the lever a direct\-field MLP will never have:**impose a symmetry on the learned energy, and Noether hands you back the matching conserved quantity — exactly, for free\.**Build anHθH\_\\thetathat is blind to absolute position, and momentum is conserved\. Blind to absolute orientation, and angular momentum is\. The MLP has no scalar, so no bracket, so no Noether — it is not that it conserves the*wrong*things, it has**nowhere for a conserved quantity to live\.** On the spring, that lever has exactly one notch\. A single degree of freedom bolted to a wall has one continuous symmetry — the phase\-space rotation, generated \(in a coincidence special to the oscillator\) byHHitself — so Noether politely hands us back the energy we already proved in Movement I\. The wall is precisely why*momentum*leaks: it pins the potential12q2\\tfrac12 q^2to an absolute position, breaking translation symmetry\. Cut the wall, let two masses interact freely, and translation symmetry — and conserved momentum — come right back\. The real power shows up the moment a system is big enough to own more than one symmetry\. ## Why an ML person should care: this is weight\-sharing for physics YOU BETTER NOT SKIP THIS ONE Okay — this one box is the entire reason we wrote this article\. We know most of you are just thumb\-scrolling by now, so we made it as loud and obnoxious as humanly possible, purely so you would stop and go*what in god's name is happening here\.*Now that we have your attention: Tell an ML person you built a model that conserves energy and you get a polite nod\. But every conservation law is, secretly, a**generalization guarantee**— that is the entire content of Noether's theorem\. "Conserves energy" already*means*"the learned dynamics don't depend on absolute time: fit them on one window and they hold for all time\." Make the energy blind to*position*too, and the same theorem hands you conserved**momentum**, which reads "fit the system in one place and it is correct everywhere\." That second sentence is a CNN's weight\-sharing, restated for things that move\. Conservation was never the selling point: it was generalization wearing a physicist's hat\. Here somewhere, buried deep in these gorgeous mechanics, there has to be the holy grail of ML and generalization that we are unfortunately collectively just too dumb to figure out and utilize to its potential\. If "a symmetry gives you a conservation law" still sounds like a physicist's party trick, notice that you already bet your career on the same idea\. A**CNN**bakes in translation symmetry through weight\-sharing: a feature learned in one corner of an image works in every corner\. You don't*hope*the network discovers that a cat is a cat wherever it lands — you build it in, shrinking the hypothesis space to functions that already respect the symmetry\. The payoff is the whole reason CNNs won: far fewer parameters, and a pattern seen once generalizes across the entire plane\. A symmetry\-constrained HNN is the identical move, one level up\. MakeHθH\_\\thetadepend only on**relative**coordinates and it goes blind to rigid shifts of the whole system — and because invariance of the scalar becomes equivariance of the flowXH=J∇HθX\_H = J\\nabla H\_\\theta, the two things you actually want fall straight out: - **Generalization, for free\.**Train near the origin, and the shift\-invariant energy makes the learned dynamics automatically correct for the same system placed*anywhere*\. You never had to show it examples at every location — the symmetry transports them for you\. That is a CNN's "learn the pattern once, apply everywhere," but for motion\. - **An exact invariant, as a bonus\.**Noether upgrades that same shift\-invariance into conserved momentum — held to machine precision along every trajectory\. A CNN's equivariance only ever buys generalization; in a*dynamical*setting the identical constraint buys generalization**and**a conservation law\. You get to cash the symmetry twice\. This is just the[geometric deep learning](https://arxiv.org/abs/2104.13478)program — CNNs, GNNs, equivariant nets — restated for things that*move*: name the symmetries your problem has, build them into the energy, and let the geometry hand back both the invariances and the sample\-efficiency\. The HNN is simply the member of that family whose scalar happens to be an energy, and Noether is the receipt\. --- ## Let's actually build the thing Enough cathedral\-building\. The astonishing part is how little code turns all of that geometry into a working network\. The symplectic gradientιXHω=dH\\iota\_\{X\_H\}\\omega = dH— the conceptual heart of three movements — is**two lines of autograd\.**We deployed a cotangent bundle, Cartan's magic formula, and Noether's theorem to justify two lines of code\. We regret nothing\. ``` import torch import torch.nn as nn class HNN(nn.Module): """Outputs a single number — the latent energy H(q, p) — and nothing else.""" def __init__(self, hidden=128): super().__init__() self.H = nn.Sequential( nn.Linear(2, hidden), nn.Tanh(), nn.Linear(hidden, hidden), nn.Tanh(), nn.Linear(hidden, 1), ) def field(self, x): """The symplectic gradient J @ grad(H). This *is* the equation iota_{X_H} w = dH, in code.""" x = x.requires_grad_(True) H = self.H(x).sum() # one scalar to differentiate dH, = torch.autograd.grad(H, x, create_graph=True) dHdq, dHdp = dH[:, 0], dH[:, 1] return torch.stack([dHdp, -dHdq], dim=1) # the 90-degree turn. yes, we had an entire section about a minus sign. ``` That`torch\.stack\(\[dHdp, \-dHdq\]\)`is the90∘90^\\circrotation\. The minus sign is the antisymmetry ofω\\omega\. Everything we proved about conservation is now structurally true of this network, and there's no way to write it down*without*conservation — the architecture literally cannot represent a divergence\-bearing field\. Training matches slopes, never energies — the unsupervised target from Movement II: ``` model = HNN() opt = torch.optim.Adam(model.parameters(), lr=1e-3) for step in range(2000): x, dx_true = sample_spring_batch() # states and their *observed* slopes dx_pred = model.field(x) loss = ((dx_pred - dx_true) ** 2).mean() # we never tell it what the energy is opt.zero_grad() loss.backward() opt.step() ``` And the baseline that invents free energy — same capacity, same data, but it emits a field directly, with no scalar and therefore no geometry: ``` class MLP(nn.Module): """Learns the vector field directly. There is no H anywhere in here.""" def __init__(self, hidden=128): super().__init__() self.net = nn.Sequential( nn.Linear(2, hidden), nn.Tanh(), nn.Linear(hidden, hidden), nn.Tanh(), nn.Linear(hidden, 2), # (dq/dt, dp/dt), and not a scalar in sight ) def field(self, x): return self.net(x) ``` Train both, then integrate each forward for fifty full periods with**RK4**— a plain,*non\-symplectic*integrator, so nothing is quietly conserving energy*for*the networks\. \(Reach for a symplectic integrator here and it would conserve energy even for the MLP, hiding the entire effect\.\) Here is an actual, reproducible run — every number below came out of the script, none were typed by hand: Both networks fit the training slopes essentially perfectly — to`~4e\-5`, so the MLP is*not*a worse function approximator\. The`true field`row is the control: RK4 itself leaks essentially nothing over fifty periods, so any drift below it is the*model's*doing, not the integrator's\. And the two models could not be more different\. The HNN holds the true energy to**0\.2%**at its very worst and keeps phase\-space area pinned at**×1\.001**\[·\]\. The MLP**bleeds away 27% of its energy**— its orbit spiralling inward as its phase\-space area collapses to**×0\.66**, a slow heat\-death, exactly the non\-zero divergence Movement I warned about\. The difference was never accuracy\. It was that one of them was allowed to learn a field with non\-zero divergence, and the other, by construction, was not\. --- ## Where the geometry runs out We would be bad guests if we wheeled out this much machinery and pretended it solved everything\. The symplectic story is exact, beautiful, and**assumes a frictionless, energy\-conserving, canonically\-coordinatized universe\.**Reality is rarely so polite\. --- ## Closing thoughts Hamilton wrote down these equations roughly…ago, to reformulate a mechanics that already worked\. Joseph\-Louis Lagrange set up the foundations with Lagrangian Mechanics…ago when he mailed his work to Euler\. They were not thinking about gradient descent, automatic differentiation, or the regrettable tendency of neural networks to manufacture energy from nothing\. And yet the cleanest way we have today to stop a network from violating thermodynamics is to hand it their 200\-year\-old geometry and let the antisymmetry of a 2\-form do the rest\. That's the lesson we keep relearning, in security and in ML alike: the win usually isn't a bigger hammer\. It's noticing what*shape*the problem already has, and refusing to learn anything that doesn't fit\. Give the network a scalar instead of a field and conservation stops being a thing you hope for and becomes a thing that*cannot not happen*\. We didn't teach it to conserve energy\. We made non\-conservation inexpressible\. Good luck creating free energy now, bozo\. “ Are you all quite finished? A A mass on a frictionless springUnbothered We are\. The spring, of course, is still going\. --- *Normally this is where we pitch you a consultation\. But if you read a differential\-geometry\-of\-neural\-networks post to the end, you are not a sales lead — you are one of us\.* *Didn't like our post?[Send your hatemails here\.](https://abscondita.com/cdn-cgi/l/email-protection#caa0a5a8b98aaba8b9a9a5a4aea3beabe4a9a5a7)* *Are you named Sam Greydanus? We absolutely**love**your work\.*👉👈