运动等变卷积 LSTM：当运动成为一种规范对称性

# Motion Equivariant Convolutional LSTM:When Motion Becomes a Gauge Symmetry
Source: [https://arxiv.org/html/2610.28771](https://arxiv.org/html/2610.28771)
Vahid HosseinzadehT\. Anderson KellerAffiliation:The Kempner Institute at Harvard UniversityAffiliation:Cusp AIMichael PuthawalaAffiliation:Department of Mathematics & StatisticsAffiliation:South Dakota State University

###### Abstract

Objects move, cameras pan, and weather drifts with the wind: much of what changes in sequential data is motion\. Motion changes where things are, not what they are, and is therefore a symmetry of the data\. Yet standard recurrent networks are not equivariant to it: their memory stays put while the input moves on\. Flow equivariant recurrent networks first brought this symmetry into sequence models, for motions of constant velocity, or flows, by enumerating a bank of velocities\. We extend this idea in three directions: from simple recurrent networks to convolutional LSTMs, from flows to motions whose velocity changes in time, and from enumerating velocities to estimating them\. Treating a sequence as a field over time, we show that motion becomes a gauge symmetry: a global symmetry made local in time\. Built on this view, our motion equivariant convolutional LSTM \(MELSTM\) uses its velocity model as the gauge field and is provably equivariant to every motion\. Across future\-frame prediction, classification of digits visible only through their motion, and action recognition under camera shake, MELSTM outperforms a convolutional LSTM and its flow equivariant counterpart with the same number of parameters: it is more accurate, generalizes to longer horizons and to unseen motions, and carries a fraction of the flow equivariant model’s state\.

## 1Introduction

Symmetry is one of the most effective inductive biases in machine learning\. A model is*equivariant*if transforming its input transforms its output in the same way, so that its response to every transformed input follows from its response to the original\. Equivariance is usually built in by constraining the weights\. A linear map between fields is equivariant to translations exactly when it is a convolution, whose weights are shared across positions\([LeCun et al\., 1989](https://arxiv.org/html/2610.28771#bib.bib15);[Kondor & Trivedi, 2018](https://arxiv.org/html/2610.28771#bib.bib14)\), and equivariant toG=ℝn⋊HG=\\mathbb\{R\}^\{n\}\\rtimes Hwhen its kernel also satisfies a steerability constraint\([Cohen & Welling, 2017](https://arxiv.org/html/2610.28771#bib.bib6);[Weiler & Cesa, 2019](https://arxiv.org/html/2610.28771#bib.bib26);[Cohen et al\., 2019a](https://arxiv.org/html/2610.28771#bib.bib7)\)\. The same principle yields group convolutions\([Cohen & Welling, 2016](https://arxiv.org/html/2610.28771#bib.bib5)\), convolutions on general Lie groups\([Finzi et al\., 2020](https://arxiv.org/html/2610.28771#bib.bib11)\), and gauge equivariant convolutions on manifolds\([Cohen et al\., 2019b](https://arxiv.org/html/2610.28771#bib.bib8);[Weiler et al\., 2026](https://arxiv.org/html/2610.28771#bib.bib27)\);[Bronstein et al\. \(2021\)](https://arxiv.org/html/2610.28771#bib.bib4)give a unified account\. When the data have the symmetry, equivariance provably improves generalization and sample efficiency\([Elesedy & Zaidi, 2021](https://arxiv.org/html/2610.28771#bib.bib10);[Bietti et al\., 2021](https://arxiv.org/html/2610.28771#bib.bib2)\)\. In almost all of this work, however, the symmetry is static: when the input is a sequence, every frame is transformed in the same way\.

Sequential data transform in time: objects move and turn, cameras pan and shake, and weather systems drift with the wind\. The transformation that relates a sequence to its transformed copy is then not one group element but a patht↦γtt\\mapsto\\gamma\_\{t\}inGG\. These paths form the group𝒢=Gℝ\{\\cal\{G\}\}=G^\{\\mathbb\{R\}\}of dynamical symmetries \(Section[2](https://arxiv.org/html/2610.28771#S2)\), which contains the static, or global, symmetries as constant paths, the*flows*as paths of constant velocity, and the*motions*as continuous paths whose velocity may change in time\. A recurrent model withGG\-equivariant layers, such as the convolutional LSTM\([Shi et al\., 2015](https://arxiv.org/html/2610.28771#bib.bib21), LSTM;\)with steerable kernels, is equivariant to the constant paths but not, in general, to the others: it updates its memory in place rather than transporting it, so a moving input leaves a decaying trail in the hidden state \(Figure[1](https://arxiv.org/html/2610.28771#S1.F1), second row\)\. One may ask whether flow or motion equivariance can also be obtained by constraining the weights\. On space\-time, flows of translations are Galilean boosts\(x,t\)↦\(x\+ν​t,t\)\(x,t\)\\mapsto\(x\+\\nu t,t\), whose orbits are entire time slices, so the steerability constraint forces a space\-time kernel to be constant in space at every nonzero time lag; Lorentz boosts, in contrast, have hyperbolic orbits, and Lorentz invariant kernels can vary in space \(Remark[1](https://arxiv.org/html/2610.28771#Thmremark1)\)\. For the LSTM, flow equivariance restricts the kernels to a set of measure zero \(Proposition[1](https://arxiv.org/html/2610.28771#Thmproposition1)\)\. The weights alone cannot provide this symmetry: the memory itself has to move with the input\.

![Refer to caption](https://arxiv.org/html/2610.28771v1/figures/Motion_vs_flows8.png)Figure 1:Hidden\-state transport under RNN, Flow Equivariant RNN \(FERNN\), and Motion Equivariant RNN \(MERNN\) models\.The first row shows the transformed input,ϕ⊳𝒳t\\phi\\triangleright\{\\cal\{X\}\}\_\{t\}in the flow column andψ⊳𝒳t\\psi\\triangleright\{\\cal\{X\}\}\_\{t\}in the motion column\. Here an MNIST digit undergoes two simultaneous changes: a geometric transformation along the group orbit \(the motion, which differs across columns\) and an*intrinsic appearance change*transverse to the group action \(identical across columns\)\. The second row shows the hidden state of an untrained leaky accumulator,ht\+1=λ​ht\+𝒳th\_\{t\+1\}=\\lambda\\,h\_\{t\}\+\{\\cal\{X\}\}\_\{t\}\(λ=0\.92\\lambda=0\.92\), which can be viewed as a convolutional RNN,ht\+1=W⋆ht\+U⋆𝒳th\_\{t\+1\}=W\\star h\_\{t\}\+U\\star\{\\cal\{X\}\}\_\{t\}, with kernelsW⁡\(x\)=λ​δ​\(x\)W\(x\)=\\lambda\\delta\(x\)andU⁡\(x\)=δ⁡\(x\)U\(x\)=\\delta\(x\)\. The third row shows FERNN’s hidden state, which is lifted to velocity copiesg∈\{−3,…,3\}2g\\in\\\{\-3,\\dots,3\\\}^\{2\}and evolves asht\+1g=λ​g⋅htg\+𝒳th^\{g\}\_\{t\+1\}=\\lambda\\,g\\cdot h^\{g\}\_\{t\}\+\{\\cal\{X\}\}\_\{t\}; we show the copy of maximum energy, i\.e\., the copy with the largest‖htg‖22\\\|h\_\{t\}^\{g\}\\\|\_\{2\}^\{2\}\. The fourth row shows MERNN’s hidden state,ht\+1=λ​Tvt​ht\+𝒳th\_\{t\+1\}=\\lambda T\_\{v\_\{t\}\}\\,h\_\{t\}\+\{\\cal\{X\}\}\_\{t\}, whereTvtT\_\{v\_\{t\}\}is the transport operator andvtv\_\{t\}is the digit’s velocity at timett\. Matched transport stacks evidence into a sharp digit at the true location, whereas mismatched transport leaves a decaying ghost trail along the residual motion\.[Keller \(2025\)](https://arxiv.org/html/2610.28771#bib.bib12)took the first step beyond static symmetries with flow equivariant RNNs \(FERNNs\)\. FERNN lifts the hidden state to a bank of copies, one for each velocity of a fixed lattice and each moving with it, so an input moving with velocityν\\nuis static in copyν\\nuand accumulates there as if it did not move\. Flows are, however, a narrow class\. They form a group only whenGGis abelian \(Lemma[1](https://arxiv.org/html/2610.28771#Thmlemma1)\); when the velocity changes, the content of one copy is handed over to another and the history accumulated in the first is lost \(Figure[1](https://arxiv.org/html/2610.28771#S1.F1), third row\); and the bank must enumerate every velocity the data may contain, which multiplies the size of the state and leaves velocities outside the lattice unrepresented\.

We take a different route: rather than enumerating velocities, we estimate them\. A velocity model, trainable or parameter\-free, reads the sequence and returns at every step a Lie algebra elementut∈𝔤u\_\{t\}\\in\\mathfrak\{g\}, which generates a transporterTt∈GT\_\{t\}\\in Gthat carries the hidden and cell states from one step to the next \(Figure[1](https://arxiv.org/html/2610.28771#S1.F1), fourth row\)\. The resulting motion equivariant LSTM \(MELSTM\) is equivariant to all motions, provided the velocity model transforms asut↦ψt​ut​ψt−1\+ψ˙t​ψt−1u\_\{t\}\\mapsto\\psi\_\{t\}u\_\{t\}\\psi\_\{t\}^\{\-1\}\+\\dot\{\\psi\}\_\{t\}\\psi\_\{t\}^\{\-1\}when its input is transformed by a motionψ\\psi\(Theorem[2](https://arxiv.org/html/2610.28771#Thmtheorem2)\)\. This is the transformation law of a gauge connection\. In physics, making a global symmetry local, so that the transformation may vary from point to point, requires a connection and the parallel transport it defines\([Yang & Mills, 1954](https://arxiv.org/html/2610.28771#bib.bib30);[Nakahara, 2003](https://arxiv.org/html/2610.28771#bib.bib17)\)\. Time\-dependent transformations are the global symmetries made local in time: a sequence is a section of a bundle over the time axis, motions act on it as gauge transformations, the velocity model is a connection, and the transporter is its parallel transport \(Appendix[A\.8](https://arxiv.org/html/2610.28771#A1.SS8)\)\. Gauge equivariant networks apply the same principle in space, to the choice of frame on a manifold\([Cohen et al\., 2019b](https://arxiv.org/html/2610.28771#bib.bib8);[Weiler et al\., 2026](https://arxiv.org/html/2610.28771#bib.bib27)\); here the gauge freedom is the choice of frame at each instant\.

So far, a single motion acts on the whole frame\. Scenes, however, usually consist of many independent motions: objects move on their own, against a background that moves with the camera\. The symmetry is then a*local motion*, which moves each component by its own motion, and a motion equivariant model that respects superposition is also equivariant to local motions \(Section[3](https://arxiv.org/html/2610.28771#S3)\)\. MELSTM represents such scenes with a few slots, each transported by its own velocity\. With*slot tracking*, each slot estimates this velocity by matching its own hidden state against the incoming frame, so the scene is segmented within the state and no correspondence between frames has to be solved \(Appendix[B\.1\.4](https://arxiv.org/html/2610.28771#A2.SS1.SSS4)\)\. Our contributions are:

- •Weight constraints are not enough\.With steerable kernels, the LSTM is globallyGG\-equivariant, yet it is flow equivariant only for a measure\-zero set of local kernels\. For space\-time convolutions, Galilean steerability forces the kernel to be constant in space at every nonzero time lag \(Proposition[1](https://arxiv.org/html/2610.28771#Thmproposition1), Remark[1](https://arxiv.org/html/2610.28771#Thmremark1), Appendices[A\.2](https://arxiv.org/html/2610.28771#A1.SS2)–[A\.3](https://arxiv.org/html/2610.28771#A1.SS3)\)\.
- •Flow equivariance by lifting\.Flows form a group only whenGGis abelian \(Lemma[1](https://arxiv.org/html/2610.28771#Thmlemma1)\)\. We characterize the equivariant linear maps between transport fields and, for an abelian subgroupV≤GV\\leq G, build from them FELSTM, a flow equivariant LSTM; the proof of its equivariance isolates the single step that requires constant velocity \(Theorem[1](https://arxiv.org/html/2610.28771#Thmtheorem1), Appendices[A\.4](https://arxiv.org/html/2610.28771#A1.SS4)–[A\.5](https://arxiv.org/html/2610.28771#A1.SS5)\)\.
- •Motion equivariance by transport\.A transporter is motion equivariant if and only if its velocity transforms as a gauge connection, and MELSTM, which transports its states with such a transporter, is equivariant to all motions, which act on the data as gauge transformations over the time axis \(Theorem[2](https://arxiv.org/html/2610.28771#Thmtheorem2), Appendices[A\.6](https://arxiv.org/html/2610.28771#A1.SS6)–[A\.8](https://arxiv.org/html/2610.28771#A1.SS8)\)\.
- •Local motions\.A motion equivariant model that respects superposition is also equivariant to local motions, which move the components of a scene independently \(Section[3](https://arxiv.org/html/2610.28771#S3)\)\. MELSTM represents such scenes with a few transported slots, each estimating the velocity of its own content, which removes the correspondence problem of frame\-to\-frame motion estimation \(Appendix[B\.1\.4](https://arxiv.org/html/2610.28771#A2.SS1.SSS4)\)\.
- •Experiments\.We compare LSTM, FELSTM and MELSTM at equal parameter counts on three tasks with time\-dependent motion: future\-frame prediction on Moving MNIST\([Srivastava et al\., 2015](https://arxiv.org/html/2610.28771#bib.bib23)\), classification of Motion\-Only MNIST digits, which no single frame reveals because only the common fate of their pixels defines them\([Upadhyay et al\., 2026](https://arxiv.org/html/2610.28771#bib.bib24)\), and action recognition on KTH\([Schuldt et al\., 2004](https://arxiv.org/html/2610.28771#bib.bib20)\)under a shaking camera\. Since FELSTM is flow but not motion equivariant, the comparison isolates what a changing velocity costs a flow equivariant model\. MELSTM is the most accurate model on all three tasks and the only one whose performance does not degrade when the motion statistics change; on Moving MNIST, it trains about six times faster per epoch than FELSTM and extrapolates to8\.58\.5times the trained horizon with3737times lower error \(Section[5](https://arxiv.org/html/2610.28771#S5)\)\. Code is available at[https://github\.com/VahidHosseinzadeh/MELSTM](https://github.com/VahidHosseinzadeh/MELSTM)\.

Section[6](https://arxiv.org/html/2610.28771#S6)discusses related work and future directions\.

## 2Equivariance; static vs dynamical symmetries

The data used in machine learning often exhibit structure and symmetry\. Incorporating these symmetries as inductive bias leads to equivariant models, which are typically more efficient and generalize better\. Most existing work focuses on*static symmetries*, where we have a time independent signalX:M→VX:M\\to Vwhich is a vector\-valued function on the domainMM\. The space of these signals is denoted byFFwhich admits a group action of a symmetry groupGG\. The equivariant model in this case is a functionΦ\\Phithat commutes with the group action:

Φ⁡\[g⋅X\]=g⋅Φ⁡\[X\],∀g∈G,X∈F\.\\displaystyle\\Phi\[g\\cdot X\]=g\\cdot\\Phi\[X\],\\qquad\\forall g\\in G,\\;X\\in F\.\(1\)This static setup can be generalized to the case of time dependent signals or trajectories𝒳:ℝ→F\{\\cal\{X\}\}:\\mathbb\{R\}\\to F, which map timettto a function𝒳t≡𝒳⁡\(t\)∈F\{\\cal\{X\}\}\_\{t\}\\equiv\{\\cal\{X\}\}\(t\)\\in F\. We denote the space of time dependent signals byℱ=\{𝒳:ℝ→F\}≡Fℝ\{\\cal F\}=\\\{\{\\cal\{X\}\}:\\mathbb\{R\}\\to F\\\}\\equiv F^\{\\mathbb\{R\}\}\. The natural question is how to generalize the static symmetries to this dynamical setting\. The most general way is to extend the groupGGto a group of time dependent transformations𝒢:=\{γ:ℝ→G\}≡Gℝ\\mathcal\{G\}:=\\\{\\gamma:\\mathbb\{R\}\\to G\\\}\\equiv G^\{\\mathbb\{R\}\}where at each timett, we have a group elementγt≡γ⁡\(t\)∈G\\gamma\_\{t\}\\equiv\\gamma\(t\)\\in G, and the group composition is defined pointwise\(γ\(1\)⋅γ\(2\)\)t:=γt\(1\)​γt\(2\)\{\(\\gamma^\{\(1\)\}\\cdot\\gamma^\{\(2\)\}\)\}\_\{t\}:=\\gamma^\{\(1\)\}\_\{t\}\\gamma^\{\(2\)\}\_\{t\}\. Now the action of a time dependent transformationγ∈𝒢\\gamma\\in\\mathcal\{G\}on a time dependent signal𝒳∈ℱ\{\\cal\{X\}\}\\in\\cal Fis given by

\(γ⊳𝒳\)t:=γt⋅𝒳t,∀t∈ℝ\.\\displaystyle\{\(\\gamma\\triangleright\{\\cal\{X\}\}\)\}\_\{t\}:=\\gamma\_\{t\}\\cdot\{\\cal\{X\}\}\_\{t\},\\qquad\\forall t\\in\\mathbb\{R\}\.\(2\)This defines a*dynamical symmetry*, where transformations may vary arbitrarily over time\.

##### Motions𝒢m​o​t\{\\cal G\}\_\{mot\}\.

While𝒢=Gℝ\\mathcal\{G\}=G^\{\\mathbb\{R\}\}allows arbitrary time dependence, many physically and geometrically meaningful transformations are obtained by imposing regularity conditions such as continuity or smoothness\. A continuous mapψ:ℝ→G\\psi:\\mathbb\{R\}\\to G, withGGa topological group, is called a motion\. WhenGGis a Lie group, we assume a motion is differentiable and generated by a time\-dependent Lie algebra valued functionv:ℝ→𝔤v:\\mathbb\{R\}\\to\\mathfrak\{g\}, which can be interpreted as the*velocity*of the motion\. The evolution is governed bydd​t​ψt=vt​ψt\\frac\{d\}\{dt\}\\psi\_\{t\}=v\_\{t\}\\psi\_\{t\}with initial conditionψ0=e\\psi\_\{0\}=e, whereeeis the identity element ofGG\. The motionψ\\psigenerated byvvcan be expressed as a time\-ordered exponential, defined as the continuum limit of an ordered product of infinitesimal exponentials:

ψt=𝒯exp\(∫0tvsds\)=limN→∞\[ev⁡\(τN\)​δ​t⋯ev⁡\(τ1\)​δ​t\]\\displaystyle\\psi\_\{t\}=\\mathcal\{T\}\\exp\\left\(\\int\_\{0\}^\{t\}v\_\{s\}\\,ds\\right\)=\\lim\_\{N\\to\\infty\}\\left\[e^\{v\(\\tau\_\{N\}\)\\delta t\}\\cdots e^\{v\(\\tau\_\{1\}\)\\delta t\}\\right\]\(3\)whereδ​t=tN\\delta t=\\frac\{t\}\{N\}andτi=i​δ​t\{\\tau\_\{i\}\}=i\\delta t\. Note that the composition of two motions and the inverse of any motion are again motions\. Therefore the set of motions is a subgroup of𝒢\\mathcal\{G\}, denoted𝒢m​o​t≤𝒢\{\\mathcal\{G\}\}\_\{mot\}\\leq\\mathcal\{G\}\. A proof appears in Appendix[A\.1](https://arxiv.org/html/2610.28771#A1.SS1)\.

##### Flows𝒢f​l​o​w\{\\cal G\}\_\{flow\}\.

An interesting subset of motions are motions with constant velocityvt=νv\_\{t\}=\\nufor alltt\. We call these motions*flows*and denote them byϕ\\phi\. In this case, equation \([3](https://arxiv.org/html/2610.28771#S2.E3)\) reduces toϕt=exp⁡\(t​ν\)\\phi\_\{t\}=\\exp\(t\\nu\)\. Equivalently, flows are motionsϕ:ℝ→G\\phi:\\mathbb\{R\}\\to Gthat correspond to continuous*group homomorphisms*fromℝ\\mathbb\{R\}, viewed as the additive abelian group of time translations\. The group homomorphism condition impliesϕt\+s=ϕt​ϕs\\phi\_\{t\+s\}=\\phi\_\{t\}\\phi\_\{s\}, which follows directly from equation \([3](https://arxiv.org/html/2610.28771#S2.E3)\) as well\. Note that flows form a subgroup of𝒢\\mathcal\{G\}if and only ifGGis an abelian group\. See Appendix[A\.1](https://arxiv.org/html/2610.28771#A1.SS1)for proof\.

##### Global symmetries𝒢g​l​o​b\{\\cal G\}\_\{glob\}\.

The group𝒢\\cal Gcontains the subgroup of time\-independent transformations, i\.e\., constant functionsγ⁡\(t\)=g∈G\\gamma\(t\)=g\\in Gfor alltt, which we call*global transformations*\. This subgroup is isomorphic toGG, with elements denoted simply byggwhere it acts uniformly across time via\(g⋅𝒳\)t=g⋅𝒳t\(g\\cdot\{\\cal\{X\}\}\)\_\{t\}=g\\cdot\{\\cal\{X\}\}\_\{t\}\.

##### Discrete vs Continuous time\.

In practice we often have a finite sample from the trajectory𝒳t\{\\cal\{X\}\}\_\{t\}as\{𝒳t\}t=0T\\\{\{\\cal\{X\}\}\_\{t\}\\\}\_\{t=0\}^\{T\}\. The definitions of symmetries and equivariance can be adapted to this setting by replacingℝ\\mathbb\{R\}with the discrete set\{0,1,…,T\}\\\{0,1,\\ldots,T\\\}and considering functionsg:\{0,1,…,T\}→Gg:\\\{0,1,\\ldots,T\\\}\\to G\. The same classification into motion, flow and global symmetries applies in the discrete case as well\. In the case of motions we have the recursive relationψt=exp⁡\(vt\)​ψt−1\\psi\_\{t\}=\\exp\(v\_\{t\}\)\\psi\_\{t\-1\}withψ0=e\\psi\_\{0\}=eor equivalentlyψt=∏s=1texp⁡\(vs\)\\psi\_\{t\}=\\prod\_\{s=1\}^\{t\}\\exp\(v\_\{s\}\)for a sequence of velocities\{vt\}t=1T\{\\\{v\_\{t\}\\\}\}\_\{t=1\}^\{T\}, which is the discrete analogue of equation \([3](https://arxiv.org/html/2610.28771#S2.E3)\)\. In the case of flows we haveϕt=exp⁡\(t​ν\)=\(ϕ1\)t\\phi\_\{t\}=\\exp\(t\\nu\)=\{\(\\phi\_\{1\}\)\}^\{t\}\.

## 3Spatiotemporal data and Equivariant Sequential Models

##### Spatial data and convolution\.

The formalism above is general and applies to any type of data and symmetries\. However, here we focus on spatiotemporal data where the domain of signalsMMisℝ2\\mathbb\{R\}^\{2\}, the feature spaceVVisℝc\\mathbb\{R\}^\{c\}, andF=L2​\(ℝ2,ℝc\)F=\{L\}^\{2\}\(\\mathbb\{R\}^\{2\};\\mathbb\{R\}^\{c\}\)is the space of square\-integrable functions onℝ2\\mathbb\{R\}^\{2\}with values inℝc\\mathbb\{R\}^\{c\}\. The groupGGis typically spatial translations\(ℝ2,\+\)\(\\mathbb\{R\}^\{2\},\+\),S​E​\(2\)=ℝ2⋊S​O​\(2\)SE\(2\)=\\mathbb\{R\}^\{2\}\\rtimes SO\(2\)\(including rotations\), or more generallyℝ2⋊H\\mathbb\{R\}^\{2\}\\rtimes HwhereHHis a subgroup ofG​L​\(2\)GL\(2\)\. We consider regular fields where the group action is defined as\(g⋅X\)​\(x\)=X⁡\(g−1⋅x\)\(g\\cdot X\)\(x\)=X\(g^\{\-1\}\\cdot x\)for ag∈Gg\\in G\. Note that this means we have a direct sum ofccnumber of scalar regular fields as the channels are not mixing under the group action\.

Now, consider a linear map from a regular fieldX∈Fi​n=ℒ2​\(ℝ2,ℝci​n\)X\\in F\_\{in\}=\{\\cal L\}^\{2\}\(\\mathbb\{R\}^\{2\};\\mathbb\{R\}^\{c\_\{in\}\}\)to another regular field inFo​u​t=L2​\(ℝ2,ℝco​u​t\)F\_\{out\}=\{L\}^\{2\}\(\\mathbb\{R\}^\{2\};\\mathbb\{R\}^\{c\_\{out\}\}\)via an integral operatorℐ⁡\[X\]=∫ℝ2W⁡\(x,y\)​X​\(y\)​𝑑y\{\\cal I\}\[X\]=\\int\_\{\\mathbb\{R\}^\{2\}\}W\(x,y\)X\(y\)dywith kernelW:ℝ2×ℝ2→ℝco​u​t×ci​nW:\\mathbb\{R\}^\{2\}\\times\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}^\{c\_\{out\}\\times c\_\{in\}\}\. For this map to be equivariant, it must be a convolution

\[W⋆X\]​\(x\)=∫ℝ2W⁡\(x−y\)​X​\(y\)​𝑑y\\displaystyle\[W\\star X\]\(x\)=\\int\_\{\\mathbb\{R\}^\{2\}\}W\(x\-y\)X\(y\)dy\(4\)with a kernel satisfying the steerability constraintW⁡\(h⋅x\)=1\|det\(h\)\|​W​\(x\)W\(h\\cdot x\)=\\frac\{1\}\{\|\{\\det\(h\)\}\|\}W\(x\)for allh∈Hh\\in Handx∈ℝ2x\\in\\mathbb\{R\}^\{2\}\. For instance, whenG=ℝ2G=\\mathbb\{R\}^\{2\}, the equivariant map is standard convolution with no kernel constraints, while forG=S​E​\(2\)G=SE\(2\), it is convolution with a rotation invariant kernelW⁡\(R⋅x\)=W⁡\(x\)W\(R\\cdot x\)=W\(x\)for allR∈S​O​\(2\)R\\in SO\(2\)\.

##### Spatiotemporal data and equivariant sequential models\.

We model spatiotemporal data as a trajectory𝒳:ℝ→F\{\\cal\{X\}\}:\\mathbb\{R\}\\to F, whereF=L2​\(ℝ2,ℝc\)F=\{L\}^\{2\}\(\\mathbb\{R\}^\{2\};\\mathbb\{R\}^\{c\}\)is the space of regular fields\. In discrete time, the temporal domain is replaced by\{0,1,…,T\}\\\{0,1,\\ldots,T\\\}and the trajectory is represented by the sequence\(𝒳0,𝒳1,…,𝒳T\)≡𝒳≤T\(\{\\cal\{X\}\}\_\{0\},\{\\cal\{X\}\}\_\{1\},\\ldots,\{\\cal\{X\}\}\_\{T\}\)\\equiv\{\\cal\{X\}\}\_\{\\leq T\}where each𝒳t∈L2​\(ℝ2,ℝc\)\{\\cal\{X\}\}\_\{t\}\\in\{L\}^\{2\}\(\\mathbb\{R\}^\{2\};\\mathbb\{R\}^\{c\}\)is a spatial field at timett\. Given the action ofGGonFF, we obtain an induced action of the trajectory group𝒢\{\\cal G\}on the trajectory spaceℱ\{\\cal F\}defined by

\(γ⊳𝒳\)t​\(x\)=\(γt⋅𝒳t\)​\(x\)=𝒳t​\(γt−1⋅x\)\\displaystyle\{\(\\gamma\\triangleright\{\\cal\{X\}\}\)\}\_\{t\}\(x\)=\(\\gamma\_\{t\}\\cdot\{\\cal\{X\}\}\_\{t\}\)\(x\)=\{\\cal\{X\}\}\_\{t\}\(\\gamma\_\{t\}^\{\-1\}\\cdot x\)\(5\)forγ∈𝒢\\gamma\\in\{\\cal G\}and𝒳∈ℱ\{\\cal\{X\}\}\\in\{\\cal F\}\.

A sequential model is a mapΦ\\Phithat takes as input the history𝒳≤t\{\\cal\{X\}\}\_\{\\leq t\}and outputs a regular fieldΦ⁡\[𝒳≤t\]∈F\\Phi\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\in F\. Such a representation may then be used for forecasting, sequence classification, or other downstream tasks\. We say that the model is*motion equivariant*if

Φ⁡\[ψ⊳𝒳≤t\]=ψ⊳Φ⁡\[𝒳≤t\]∀ψ∈𝒢m​o​t,∀𝒳∈ℱ\.\\displaystyle\\Phi\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\\triangleright\\Phi\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\qquad\\forall\\psi\\in\{\\cal G\}\_\{mot\},\\;\\forall\{\\cal\{X\}\}\\in\{\\cal F\}\.\(6\)Similarly, replacingψ∈𝒢m​o​t\\psi\\in\{\\cal G\}\_\{mot\}withϕ∈𝒢f​l​o​w\\phi\\in\{\\cal G\}\_\{flow\}org∈𝒢g​l​o​bg\\in\{\\cal G\}\_\{glob\}yields the notions of flow equivariance and globalGG\-equivariance, respectively\.

##### Local motion equivariance\.

The transformations above act on the whole signal at once\. In practice, however, the data of interest is often a superposition of several objects, each carrying its own motion\. Given a decomposition𝒳t=∑α=1K𝒳tα\{\\cal\{X\}\}\_\{t\}=\\sum\_\{\\alpha=1\}^\{K\}\{\\cal\{X\}\}^\{\\alpha\}\_\{t\}and a tupleψ=\(ψ1,…,ψK\)∈𝒢m​o​tK\\psi=\(\\psi^\{1\},\\dots,\\psi^\{K\}\)\\in\{\\cal\{G\}\}\_\{mot\}^\{K\}, a*local motion*acts by\(ψ⊳𝒳\)t:=∑α\(ψα⊳𝒳α\)t\(\\psi\\triangleright\{\\cal\{X\}\}\)\_\{t\}\\;:=\\;\\sum\_\{\\alpha\}\(\\psi^\{\\alpha\}\\triangleright\{\\cal\{X\}\}^\{\\alpha\}\)\_\{t\}, and we speak of a*local flow*when everyψα\\psi^\{\\alpha\}is a flow\. Now suppose that the model respects the superposition property, in the sense thatΦ⁡\[∑α𝒳≤tα\]=∑αΦ⁡\[𝒳≤tα\]\\Phi\[\\sum\_\{\\alpha\}\{\\cal\{X\}\}\_\{\\leq t\}^\{\\alpha\}\]=\\sum\_\{\\alpha\}\\Phi\[\{\\cal\{X\}\}\_\{\\leq t\}^\{\\alpha\}\]\. Then, if the model is motion equivariant in the sense above, it is automatically equivariant under local motions as well:

Φ⁡\[∑αψα⊳𝒳≤tα\]=∑αΦ⁡\[ψα⊳𝒳≤tα\]=∑αψα⊳Φ⁡\[𝒳≤tα\]∀ψ∈𝒢m​o​tK,∀𝒳∈ℱ,\\displaystyle\\Phi\\Big\[\\sum\_\{\\alpha\}\\psi^\{\\alpha\}\\triangleright\{\\cal\{X\}\}^\{\\alpha\}\_\{\\leq t\}\\Big\]=\\sum\_\{\\alpha\}\\Phi\\Big\[\\psi^\{\\alpha\}\\triangleright\{\\cal\{X\}\}^\{\\alpha\}\_\{\\leq t\}\\Big\]=\\sum\_\{\\alpha\}\\psi^\{\\alpha\}\\triangleright\\Phi\\big\[\{\\cal\{X\}\}^\{\\alpha\}\_\{\\leq t\}\\big\]\\qquad\\forall\\,\\psi\\in\{\\cal G\}\_\{mot\}^\{K\},\\;\\forall\{\\cal\{X\}\}\\in\{\\cal F\},\(7\)where the first equality uses the superposition property and the second uses motion equivariance\.

## 4Motion Equivariant Convolutional LSTM

##### LSTM\.

The Convolutional LSTM \(LSTM\)\([Shi et al\., 2015](https://arxiv.org/html/2610.28771#bib.bib21)\)is a first\-order Markovian recurrent model with two state variables, the cell state\{𝒞t\}\\\{\{\\cal\{C\}\}\_\{t\}\\\}and hidden state\{ℋt\}\\\{\{\\cal\{H\}\}\_\{t\}\\\}, evolving according to

zt\\displaystyle z\_\{t\}=az​\(Wz⋆ℋt\+Uz⋆𝒳t\)∀z∈\{i,f,o,c\}\\displaystyle=a\_\{z\}\\big\(W\_\{z\}\\star\{\\cal H\}\_\{t\}\+U\_\{z\}\\star\{\\cal X\}\_\{t\}\\big\)\\qquad\\forall z\\in\\\{i,f,o,c\\\}\(8\)𝒞t\+1\\displaystyle\{\\cal C\}\_\{t\+1\}=𝒞t⊙ft\+it⊙ct\\displaystyle=\{\\cal C\}\_\{t\}\\odot f\_\{t\}\+i\_\{t\}\\odot c\_\{t\}\(9\)ℋt\+1\\displaystyle\{\\cal H\}\_\{t\+1\}=ot⊙tanh⁡\(𝒞t\+1\)\\displaystyle=o\_\{t\}\\odot\\tanh\(\{\\cal C\}\_\{t\+1\}\)\(10\)where⊙\\odotdenotes element\-wise multiplication,\{it,ft,ot,ct\}\\\{i\_\{t\},f\_\{t\},o\_\{t\},c\_\{t\}\\\}are the input, forget, output, and cell gates, and the nonlinearityaza\_\{z\}isσ\\sigmaforz∈\{i,f,o\}z\\in\\\{i,f,o\\\}andtanh\\tanhforz=cz=c\.

The LSTM is globallyGG\-equivariant when the convolutional kernelsWzW\_\{z\}andUzU\_\{z\}are chosen to satisfy the appropriate steerability constraints for the groupGG\. See Appendix[A\.2](https://arxiv.org/html/2610.28771#A1.SS2)for detailed statements and proof\.

The LSTM, however, is not flow equivariant for generic kernels \(Proposition[1](https://arxiv.org/html/2610.28771#Thmproposition1), Appendix[A\.3](https://arxiv.org/html/2610.28771#A1.SS3)\)\. The obstruction is structural: LSTM updates its memory in place rather than transporting it with the flow\. As illustrated for a simple RNN in the second row of Figure[1](https://arxiv.org/html/2610.28771#S1.F1), this leaves a decaying trail, or left\-behind memory, instead of transporting the accumulated state with moving features\. The appendix shows that eliminating this residual memory requires an exact cancellation among the local convolution kernels, satisfied only by a measure\-zero set of parameters\. Transporting the hidden and cell states with the flow removes this obstruction, motivating the architectures introduced next\.

##### Channel based flow equivariant LSTM \(FELSTM\)\.

In this section, following the Flow Equivariant RNN \(FERNN\) introduced by[Keller \(2025\)](https://arxiv.org/html/2610.28771#bib.bib12), we introduce the channel based FELSTM, a generalization of the LSTM architecture that achieves flow equivariance by lifting the regular fields onMMto the larger spaceM×GM\\times G\. In particular the gate variables transform under𝒢\{\\cal\{G\}\}as

\(γ⊳z\)t​\(x,g\)=γt⋅zt​\(x,γt−1​g​γt−1\)\\displaystyle\{\(\\gamma\\triangleright z\)\}\_\{t\}\(x,g\)=\\gamma\_\{t\}\\cdot z\_\{t\}\\big\(x,\\,\\gamma\_\{t\}^\{\-1\}g\\,\\gamma\_\{t\-1\}\\big\)=zt​\(γt−1⋅x,γt−1​g​γt−1\)\\displaystyle=z\_\{t\}\\big\(\\gamma\_\{t\}^\{\-1\}\\cdot x,\\,\\gamma\_\{t\}^\{\-1\}g\\,\\gamma\_\{t\-1\}\\big\)\(11\)and the state variables𝒮∈\{ℋ,𝒞\}\\mathcal\{S\}\\in\\\{\{\\cal\{H\}\},\{\\cal\{C\}\}\\\}transform as111Note that the state variable at timettis a function of the input sequence up to timet−1t\-1so it transforms according toγt−1\\gamma\_\{t\-1\}instead ofγt\\gamma\_\{t\}\.

\(γ⊳𝒮\)t​\(x,g\)=γt−1⋅𝒮t​\(x,γt−1​g​γt−1\)\\displaystyle\{\(\\gamma\\triangleright\\mathcal\{S\}\)\}\_\{t\}\(x,g\)=\\gamma\_\{t\-1\}\\cdot\\mathcal\{S\}\_\{t\}\\big\(x,\\,\\gamma\_\{t\}^\{\-1\}g\\,\\gamma\_\{t\-1\}\\big\)=𝒮t​\(γt−1−1⋅x,γt−1​g​γt−1\)\.\\displaystyle=\\mathcal\{S\}\_\{t\}\\big\(\\gamma\_\{t\-1\}^\{\-1\}\\cdot x,\\,\\gamma\_\{t\}^\{\-1\}g\\,\\gamma\_\{t\-1\}\\big\)\.\(12\)We refer to the transformationg↦γt−1​g​γt−1g\\mapsto\\gamma\_\{t\}^\{\-1\}g\\,\\gamma\_\{t\-1\}as*transport conjugation*, and fields onM×GM\\times Gwhose transformation law involves this action on the velocity coordinate are called*transport fields*\.222This terminology is inspired by the parallel transport concept in gauge theories which is the theoretical basis of our transporter based model in the next section\.As we saw earlier,𝒢f​l​o​w\{\\cal\{G\}\}\_\{flow\}is a group \(subgroup of𝒢\{\\cal\{G\}\}\) if and only ifGGis abelian \(see Lemma[1](https://arxiv.org/html/2610.28771#Thmlemma1)\)\. So for general non\-abelian groups, we restrict to an abelian subgroupV≤GV\\leq Gand build the flow equivariant model for𝒱f​l​o​w≤𝒢\{\\cal V\}\_\{flow\}\\leq\\cal Gas the symmetry group\. Therefore we lift to transport fields onM×VM\\times Vand modify update equations as follows

zt​\(x,𝗏\)\\displaystyle z\_\{t\}\(x,\\mathsf\{v\}\)=az​\(𝗏⋅\[Wz⋆Fℋt\]​\(x,𝗏\)\+\[Uz​⋆^​𝒳t\]​\(x,𝗏\)\)∀z∈\{i,f,o,c\}\\displaystyle=a\_\{z\}\\Big\(\\mathsf\{v\}\\cdot\[W\_\{z\}\\star\_\{F\}\{\\cal H\}\_\{t\}\]\(x,\\mathsf\{v\}\)\+\[U\_\{z\}\\,\\hat\{\\star\}\\,\{\\cal X\}\_\{t\}\]\(x,\\mathsf\{v\}\)\\Big\)\\qquad\\forall z\\in\\\{i,f,o,c\\\}\(13\)𝒞t\+1​\(x,𝗏\)\\displaystyle\{\\cal C\}\_\{t\+1\}\(x,\\mathsf\{v\}\)=\[𝗏⋅𝒞t​\(x,𝗏\)\]⊙ft​\(x,𝗏\)\+it​\(x,𝗏\)⊙ct​\(x,𝗏\)\\displaystyle=\[\\mathsf\{v\}\\cdot\{\\cal C\}\_\{t\}\(x,\\mathsf\{v\}\)\]\\odot f\_\{t\}\(x,\\mathsf\{v\}\)\+i\_\{t\}\(x,\\mathsf\{v\}\)\\odot c\_\{t\}\(x,\\mathsf\{v\}\)\(14\)ℋt\+1​\(x,𝗏\)\\displaystyle\{\\cal H\}\_\{t\+1\}\(x,\\mathsf\{v\}\)=ot​\(x,𝗏\)⊙tanh⁡\(𝒞t\+1​\(x,𝗏\)\)\\displaystyle=o\_\{t\}\(x,\\mathsf\{v\}\)\\odot\\tanh\(\{\\cal C\}\_\{t\+1\}\(x,\\mathsf\{v\}\)\)\(15\)for allz∈\{i,f,o,c\}z\\in\\\{i,f,o,c\\\}and𝗏∈V\\mathsf\{v\}\\in V\. Here⋆^\\hat\{\\star\}is the flow\-lifting convolution operator which is an equivariant linear map that lifts the input signal𝒳t\{\\cal\{X\}\}\_\{t\}to the transport field space\. It turns out that this map is simply given by copying the steerable convolution to all the channels indexed by𝗏\\mathsf\{v\}as follows\[U​⋆^​𝒳t\]​\(x,𝗏\)=\[U⋆𝒳t\]​\(x\)\[U\\,\\hat\{\\star\}\\,\{\\cal\{X\}\}\_\{t\}\]\(x,\\mathsf\{v\}\)=\[U\\\!\\star\\\!\{\\cal\{X\}\}\_\{t\}\]\(x\)\. Also⋆F\\star\_\{F\}is the flow convolution which is an equivariant linear map between transport fields and it is defined as follows

\[W⋆Fℋt\]​\(x,𝗏\)=∫ℳ∫VW⁡\(x−y,𝗏−𝗎\)​ℋt​\(y,𝗎\)​𝑑𝗎​𝑑y\\displaystyle\[W\\star\_\{F\}\{\\cal\{H\}\}\_\{t\}\]\(x,\\mathsf\{v\}\)=\\int\_\{\\cal M\}\\int\_\{V\}W\(x\-y,\\mathsf\{v\}\-\\mathsf\{u\}\)\{\\cal\{H\}\}\_\{t\}\(y,\\mathsf\{u\}\)d\\mathsf\{u\}dy\(16\)where the kernelWWmust additionally satisfy a steerability constraint in its spatial argument\. See Appendix[A\.4](https://arxiv.org/html/2610.28771#A1.SS4)for the details of the proof of equivariance\.

###### Theorem 1\(FELSTM is flow equivariant\)\.

Consider the channel\-based FELSTM defined by equations \([13](https://arxiv.org/html/2610.28771#S4.E13)\)–\([15](https://arxiv.org/html/2610.28771#S4.E15)\), with invariant initial state variables satisfyingϕ⊳ℋ0=ℋ0\\phi\\triangleright\{\\cal\{H\}\}\_\{0\}=\{\\cal\{H\}\}\_\{0\}andϕ⊳𝒞0=𝒞0\\phi\\triangleright\{\\cal\{C\}\}\_\{0\}=\{\\cal\{C\}\}\_\{0\}for allϕ∈𝒱f​l​o​w\\phi\\in\{\\cal V\}\_\{flow\}\. Then the model is flow equivariant, i\.e\., for any flowϕ∈𝒱f​l​o​w\\phi\\in\{\\cal V\}\_\{flow\}and any trajectory𝒳∈ℱ\{\\cal\{X\}\}\\in\{\\cal F\}, we have

ℋt​\[ϕ⊳𝒳≤t\]=ϕ⊳ℋt​\[𝒳≤t\]∀t\.\\displaystyle\{\\cal\{H\}\}\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\phi\\triangleright\{\\cal\{H\}\}\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\qquad\\forall t\.\(17\)

See Appendix[A\.5](https://arxiv.org/html/2610.28771#A1.SS5)for the proof\.

##### Transporter based motion equivariant LSTM \(MELSTM\)\.

The channel\-based FELSTM achieves equivariance under flows by lifting regular fields onMMto transport fields onM×GM\\times G, which can be interpreted as introducing a bank of moving frames indexed by the groupGG\. However, this construction is equivariant only under flows𝒱f​l​o​w\{\\cal V\}\_\{flow\}of an abelian subgroupV≤GV\\leq G, not under general time\-dependent motions𝒢m​o​t\{\\cal\{G\}\}\_\{mot\}; in practice,VVis even truncated to a finite grid of velocities\. Moreover, it increases the dimensionality of the hidden representation by introducing an additional group dimension\. Here, we take a different approach\. Instead of lifting the representation space, we explicitly transport the hidden states between consecutive time steps using an estimated group\-valued transformation\. This leads to a transporter\-based architecture that achieves equivariance under general motions while keeping the fields regular and defined only on the original spatial domainMM\.

This estimated transformation is the*transporter*Tt2,t1∈GT\_\{t\_\{2\},t\_\{1\}\}\\in G, which describes how the signal transforms between timest1t\_\{1\}andt2t\_\{2\}\. It is generated by a Lie algebra valued velocityut∈𝔤u\_\{t\}\\in\\mathfrak\{g\}through the ODEdd​t​Tt,t0=ut​Tt,t0\\frac\{d\}\{dt\}T\_\{t,t\_\{0\}\}=u\_\{t\}T\_\{t,t\_\{0\}\}withTt0,t0=eT\_\{t\_\{0\},t\_\{0\}\}=e, so that, like the motionψt\\psi\_\{t\}generated byvtv\_\{t\}in \([3](https://arxiv.org/html/2610.28771#S2.E3)\), it is a time\-ordered exponential:

Tt2,t1=𝒯exp\(∫t1t2usds\)=limN→∞\[eu⁡\(τN\)​δ​t⋯eu⁡\(τ1\)​δ​t\]\\displaystyle T\_\{t\_\{2\},t\_\{1\}\}=\\mathcal\{T\}\\exp\\left\(\\int\_\{t\_\{1\}\}^\{t\_\{2\}\}u\_\{s\}ds\\right\)=\\lim\_\{N\\to\\infty\}\\left\[e^\{u\(\\tau\_\{N\}\)\\delta t\}\\cdots e^\{u\(\\tau\_\{1\}\)\\delta t\}\\right\]\(18\)whereδ​t=t2−t1N\\delta t=\\frac\{t\_\{2\}\-t\_\{1\}\}\{N\}andτi=t1\+i​δ​t\{\\tau\_\{i\}\}=t\_\{1\}\+i\\delta t\. We also need the transporter to satisfy the equivariance relation as

Tt2,t1​\[ψ⊳𝒳≤t2\]=ψt2​Tt2,t1​\[𝒳≤t2\]​ψt1−1\\displaystyle T\_\{t\_\{2\},t\_\{1\}\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\_\{2\}\}\]=\\psi\_\{t\_\{2\}\}\\,T\_\{t\_\{2\},t\_\{1\}\}\[\{\\cal\{X\}\}\_\{\\leq t\_\{2\}\}\]\\,\\psi\_\{t\_\{1\}\}^\{\-1\}\(19\)for every motionψ∈𝒢m​o​t\\psi\\in\{\\cal G\}\_\{mot\}\. In practice, we might model the instantaneous velocityutu\_\{t\}instead of the transporterTt2,t1T\_\{t\_\{2\},t\_\{1\}\}directly\. Using the ODE governing the transporter, we can show that this equivariance relation is satisfied if and only if the velocity modelutu\_\{t\}satisfies the following equivariance relation \(see Appendix[A\.6](https://arxiv.org/html/2610.28771#A1.SS6)for proof\):

ut​\[ψ⊳𝒳≤t\]=ψt​ut​\[𝒳≤t\]​ψt−1\+ψ˙t​ψt−1\\displaystyle u\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\_\{t\}u\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\}^\{\-1\}\+\\dot\{\\psi\}\_\{t\}\\psi\_\{t\}^\{\-1\}\(20\)for every motionψ∈𝒢m​o​t\\psi\\in\{\\cal G\}\_\{mot\}\. This is precisely how a gauge connection transforms\. Viewing a sequence as a section of a bundle over the time axis, the spatial coordinates of each𝒳t\{\\cal\{X\}\}\_\{t\}may be chosen independently at every instant: motions change these coordinates as gauge transformations, the velocity modelutu\_\{t\}is the connection that relates coordinates at neighboring times, and the transporter is its parallel transport \(Appendix[A\.8](https://arxiv.org/html/2610.28771#A1.SS8)\)\.

In the discrete\-time setting, we denoteTt≡Tt,t−1T\_\{t\}\\equiv T\_\{t,t\-1\}as the transporter between two consecutive time steps\. In this case, we simply haveTt=exp⁡\(ut\)T\_\{t\}=\\exp\(u\_\{t\}\)\. Using the transporter, the MELSTM is defined by the following equations

zt​\(x\)\\displaystyle z\_\{t\}\(x\)=az​\(Tt⋅\[Wz⋆ℋt\]​\(x\)\+\[Uz⋆𝒳t\]​\(x\)\)∀z∈\{i,f,o,c\}\\displaystyle=a\_\{z\}\\Big\(T\_\{t\}\\cdot\[W\_\{z\}\\star\{\\cal H\}\_\{t\}\]\(x\)\+\[U\_\{z\}\\star\{\\cal X\}\_\{t\}\]\(x\)\\Big\)\\qquad\\forall z\\in\\\{i,f,o,c\\\}\(21\)𝒞t\+1​\(x\)\\displaystyle\{\\cal C\}\_\{t\+1\}\(x\)=\[Tt⋅𝒞t​\(x\)\]⊙ft​\(x\)\+it​\(x\)⊙ct​\(x\)\\displaystyle=\[T\_\{t\}\\cdot\{\\cal C\}\_\{t\}\(x\)\]\\odot f\_\{t\}\(x\)\+i\_\{t\}\(x\)\\odot c\_\{t\}\(x\)\(22\)ℋt\+1​\(x\)\\displaystyle\{\\cal H\}\_\{t\+1\}\(x\)=ot​\(x\)⊙tanh⁡\(𝒞t\+1​\(x\)\)\\displaystyle=o\_\{t\}\(x\)\\odot\\tanh\(\{\\cal C\}\_\{t\+1\}\(x\)\)\(23\)
###### Theorem 2\(MELSTM is motion equivariant\)\.

Consider the transporter\-based MELSTM defined by equations \([21](https://arxiv.org/html/2610.28771#S4.E21)\)–\([23](https://arxiv.org/html/2610.28771#S4.E23)\), with invariant initial state variables satisfyingψ⊳ℋ0=ℋ0\\psi\\triangleright\{\\cal\{H\}\}\_\{0\}=\{\\cal\{H\}\}\_\{0\}andψ⊳𝒞0=𝒞0\\psi\\triangleright\{\\cal\{C\}\}\_\{0\}=\{\\cal\{C\}\}\_\{0\}for allψ∈𝒢m​o​t\\psi\\in\{\\cal G\}\_\{mot\}\. Then the model is motion equivariant, i\.e\., for any motionψ∈𝒢m​o​t\\psi\\in\{\\cal G\}\_\{mot\}and any trajectory𝒳∈ℱ\{\\cal\{X\}\}\\in\{\\cal F\}, we have

ℋt​\[ψ⊳𝒳≤t\]=ψ⊳ℋt​\[𝒳≤t\]∀t\.\\displaystyle\{\\cal\{H\}\}\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\\triangleright\{\\cal\{H\}\}\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\qquad\\forall t\.\(24\)

###### Proof sketch\.

The proof follows the same inductive structure as the flow equivariance result for FELSTM\. The base case is immediate from the invariance of the initial state variables\. Note that the transporter equivariance relation \([19](https://arxiv.org/html/2610.28771#S4.E19)\) in the discrete\-time setting reduces toTt​\[ψ⊳𝒳≤t\]=ψt​Tt​\[𝒳≤t\]​ψt−1−1T\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\_\{t\}\\,T\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\,\\psi\_\{t\-1\}^\{\-1\}\. Assuming that the state variables are motion equivariant at timett, we first show that the gate variables remain motion equivariant up to timett\. The key step is the cancellation of theψt−1\\psi\_\{t\-1\}factor between the transporter and the hidden state in the gate equations\. We then apply the same argument to the cell\-state and hidden\-state updates to conclude motion equivariance at timet\+1t\+1\. See Appendix[A\.7](https://arxiv.org/html/2610.28771#A1.SS7)for the complete proof\. ∎

## 5Experiments

We evaluate the proposed motion equivariant LSTM \(MELSTM\) against the flow equivariant LSTM \(FELSTM\) and a standard LSTM baseline on prediction and classification under time\-dependent motion\. The three models share an architecture, a parameter count, and a training protocol, and differ only in how the hidden state is transported between timesteps, so any difference in performance is attributable to that mechanism alone\. Our aim is not only to show that equivariance helps, which is established for flows\([Keller, 2025](https://arxiv.org/html/2610.28771#bib.bib12)\), but to ask what form it should take when the motion itself varies in time: whether it is better to*enumerate*a lattice of candidate velocities or to*estimate*a small number of them\.

### 5\.1Moving MNIST

Both experiments in this section use the same synthetic motion and the same three encoders\. They differ only in how the frames are rendered and in what the final hidden state is used for\.

##### Time\-dependent motion\.

MNIST digits move on a36×3636\\times 36canvas with periodic boundary conditions\. Velocities are discrete, drawn from the integer latticeV2V\_\{2\}whereVn=\{v∈ℤ2∣‖v‖∞≤n\}V\_\{n\}=\\\{v\\in\\mathbb\{Z\}^\{2\}\\mid\\\|v\\\|\_\{\\infty\}\\leq n\\\}, so that translation by anyv∈V2v\\in V\_\{2\}acts exactly on the periodic grid without interpolation\. Motions are piecewise constant: a velocity is held for a random33–66frames and then resampled\. Every moving componentα\\alphacarries its own velocity schedulevt\(α\)∈V2v^\{\(\\alpha\)\}\_\{t\}\\in V\_\{2\}and hence its own motionψ\(α\)\\psi^\{\(\\alpha\)\}, and the models read a context ofT=15T=15frames\.

##### Models\.

The three models share one recurrent cell and differ only in how its hidden state is transported: not at all \(LSTM\), by enumeration over\|V2\|=25\|V\_\{2\}\|=25velocity channels \(FELSTM\), or byK=2K=2slots warped by velocitiesut​\[𝒳≤t\]u\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]estimated online by parameter\-free Fourier phase correlation \(MELSTM; Appendix[B\.1\.4](https://arxiv.org/html/2610.28771#A2.SS1.SSS4)\)\. The velocity dimension is max\-pooled and read by a task headgθg\_\{\\theta\}: a decoder that produces each predicted frame,𝒳^t=gθ​\(maxv⁡ht​\(v\)\)\\hat\{\\cal\{X\}\}\_\{t\}=g\_\{\\theta\}\\bigl\(\\max\_\{v\}h\_\{t\}\(v\)\\bigr\)\(Section[5\.1\.1](https://arxiv.org/html/2610.28771#S5.SS1.SSS1)\), or a classifier applied once to the final state,y^=gθ​\(maxv⁡hT​\(v\)\)\\hat\{y\}=g\_\{\\theta\}\\bigl\(\\max\_\{v\}h\_\{T\}\(v\)\\bigr\)\(Section[5\.1\.2](https://arxiv.org/html/2610.28771#S5.SS1.SSS2)\)\. Weights are shared across channels and slots, so for each task the three models have identical parameter counts, and they are trained identically\.

#### 5\.1\.1Future\-Frame Prediction

![Refer to caption](https://arxiv.org/html/2610.28771v1/paper_training_efficiency_seeds.png)

Figure 2:Training efficiency and test error\.Right: test MSE over the trained horizon and att=100t=100; mean±\\pms\.d\. over five seeds\.![Refer to caption](https://arxiv.org/html/2610.28771v1/paper_generalization.png)Figure 3:Rollout stability and context\-boundary sensitivity\.Left: median rollout MSE\.Middle: MSE att=16t=16of sequences whose last velocity change came11–33frames before the context boundary \(boundary velocity age\), relative to settled sequences \(age≥4\\geq 4\)\.Right: the age\-11group over the rollout; the FELSTM penalty persists through mid\-rollout before gradually decaying\.Figure 4:Motion generalization sensitivity\.Each model normalized by its own training error \(1\.01\.0= no degradation\)\.##### Data and task\.

Two digits move independently, so a sequence is a sum of two independently transported components,ψ\(1\)⊳𝒳\(1\)\+ψ\(2\)⊳𝒳\(2\)\\psi^\{\(1\)\}\\triangleright\{\\cal\{X\}\}^\{\(1\)\}\+\\psi^\{\(2\)\}\\triangleright\{\\cal\{X\}\}^\{\(2\)\}\. Every sequence has two phases: during the firstT=15T=15steps, the context, each digit moves under its own time\-dependent velocity schedule, and at the context boundary the velocities of local motions are frozen,vs=vTv\_\{s\}=v\_\{T\}for alls\>Ts\>T\. Models are trained autoregressively on this structure: given the1515context steps of𝒳\{\\cal\{X\}\}, predict the following1010, over which each local motion reduces to a constant flow\. FELSTM is flow\-equivariant by construction, so its task is to have encoded, by the end of the context, a good representation of each digit in the velocity channel that matches the digit’s final velocity\. MELSTM must encode each digit in one of its slots*and*recover the digit’s final velocity, which it then uses to transport that slot through the frozen phase\. Parameters are optimized to minimize MSE\+\+L1 error against the ground truth overt∈\[16,25\]t\\in\[16,25\]for5050epochs, identical for every model\. Figure[2](https://arxiv.org/html/2610.28771#S5.F2)shows the validation loss of the three models: MELSTM outperforms both FELSTM and the standard LSTM, demonstrating the benefits of motion equivariance in handling time\-dependent motions\.

##### Slot tracking\.

For prediction, MELSTM takesut​\[𝒳≤t\]=ut​\[ℋt​\[𝒳<t\],𝒳t\]u\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]=u\_\{t\}\\big\[\{\\cal\{H\}\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\],\\,\{\\cal\{X\}\}\_\{t\}\\big\]: each slot correlates its own hidden state with the current frame\. Since a slot carries a single coherently moving component, the local problem of a velocity field over a multi\-object frame reduces toKKglobal displacements, one per slot, each given in closed form by phase correlation\. This also solves the correspondence \(or assignment\) problem\. Correlating two consecutive frames yieldsKKvelocities per step, but as an unordered set that does not say which velocity atttcontinues which att−1t\-1; here each estimate is bound to the slot that requested it\. We call this*slot tracking*\(Appendix[B\.1\.4](https://arxiv.org/html/2610.28771#A2.SS1.SSS4)\)\. Note that early in training, slot tracking can also be replaced by frame\-pair velocities, withdrawn gradually under a stochastic or curriculum schedule \(Appendix[B\.1\.5](https://arxiv.org/html/2610.28771#A2.SS1.SSS5)\)\.

##### Length generalization\.

Figure[3](https://arxiv.org/html/2610.28771#S5.F3)shows the prediction error when the frozen phase is extended from1010to8585steps: MELSTM is lowest at every timestep, by37×37\\timesover FELSTM on average and on99\.7%99\.7\\%of individual predicted frames, with all models scored on the same10,00010\{,\}000sequences\. Stratifying by how recently a digit’s velocity last changed before the context ended locates where that advantage comes from \(Figure[3](https://arxiv.org/html/2610.28771#S5.F3)\): a change one frame before the boundary costs FELSTM roughly2\.5×2\.5\\timesits error at the first predicted frame, above the baseline’s own sensitivity to the same event, while MELSTM pays nothing measurable\. The enumerating model is therefore the most sensitive of the three to a recent change in motion and the estimating model the least, even though MELSTM is the only one that must recover the velocity and so the one that would be expected to suffer most from a single frame of new evidence\.

##### Motion generalization\.

We evaluate each model as trained on benchmarks whose motion statistics differ from the training distribution, scoring MSE over the same trained horizon and dividing each model by its own error at the training configuration; Figure[4](https://arxiv.org/html/2610.28771#S5.F4)thus shows sensitivity, not accuracy\. Two compensated sweeps vary switching rate and jump size independently\. FELSTM keeps 25 hidden\-state copies, one per possible velocity; when the velocity changes, it must read from a different copy and abandon what had accumulated\. The penalty is roughly the product of how disruptive the velocity changes are, which grows with both how often and how far the velocity jumps, and how much the abandoned copy was worth\. The rate sweep raises the first factor directly and costs FELSTM3\.45×3\.45\\times; larger jumps raise it less and cost1\.35×1\.35\\times\. MELSTM transports its two slots by an estimated velocity parameter, so a velocity change moves the same representation differently rather than transferring between velocity\-indexed copies; it stays flat to within1%1\\%on both axes\. LSTM has no velocity\-indexed state to disturb and tracks only motion difficulty\. Seven fixed regimes probe combinations of rate and jump size beyond the swept axes\. LSTM’s rise measures intrinsic difficulty, while the ratio of FELSTM to LSTM isolates architecture\-specific cost\. Figure[4](https://arxiv.org/html/2610.28771#S5.F4)also orders these regimes by velocity unpredictabilityρ\\rho, the error of using the previous velocity relative to assuming no motion\.ρ\\rhomeasures the disruption and1−ρ1\-\\rhothe value of the abandoned history, so the penalty should followρ⁡\(1−ρ\)\\rho\(1\-\\rho\)\(Appendix[B\.1\.3](https://arxiv.org/html/2610.28771#A2.SS1.SSS3)\); FELSTM’s penalty indeed peaks nearρ=0\.5\\rho=0\.5\.

#### 5\.1\.2Motion\-Only MNIST Classification

![Refer to caption](https://arxiv.org/html/2610.28771v1/training_time_row_h16_all.png)

Figure 5:Motion\-Only MNIST: Training efficiency and test accuracy\.Right:five\-seed mean±\\pms\.d\. of the best validation and the test accuracy and of the median time per epoch\. Dotted lines: chance \(10%10\\%\)\.Figure 6:Motion generalization sensitivity in classification\.Relative to their own training accuracy \(far lower for LSTM\), LSTM and FELSTM respond alike, except at speed11, where LSTM improves because its3×33\\times 3recurrence can move its memory with the digit\.##### Data and task\.

If every frame is noise, an object can still be defined by the*common fate*of its pixels, which move together; all information about it then lies in the correlations between frames\([Upadhyay et al\., 2026](https://arxiv.org/html/2610.28771#bib.bib24)\)\. Motion\-Only MNIST renders a single digit in this way, as a glyph\-shaped window onto its own white\-noise texture, which moves with it, over an independent white\-noise background with a motion of its own \(Appendix[B\.2](https://arxiv.org/html/2610.28771#A2.SS2)\)\. No single frame contains the digit \(a single\-frame CNN scores12%12\\%, against10%10\\%chance\), and the model must name the digit after the context\.

##### Velocity model and training\.

MELSTM reads both slot velocities from the two strongest phase\-correlation peaks of each frame pair\(𝒳t−1,𝒳t\)\(\{\\cal\{X\}\}\_\{t\-1\},\{\\cal\{X\}\}\_\{t\}\), one from the background and one from the digit\. The estimator is still parameter\-free, and with slot tracking instead, accuracy is only slightly lower \(Appendix[B\.2](https://arxiv.org/html/2610.28771#A2.SS2)\)\. All models are trained with cross\-entropy for8080epochs on five seeds\.

##### Transport reveals the digit\.

A state transported with the digit accumulates its texture, while the background, moving differently, averages out \(Figure[14](https://arxiv.org/html/2610.28771#A2.F14)\)\. Both equivariant models exploit this \(Figure[5](https://arxiv.org/html/2610.28771#S5.F5)\), but MELSTM is slightly more accurate, trains several times faster with22state copies instead of2525, and generalizes better \(Figure[6](https://arxiv.org/html/2610.28771#S5.F6)\)\. LSTM compares consecutive frames within its3×33\\times 3recurrence but cannot carry its memory with a digit moving faster than one pixel per step, so it must find motion detectors in noise, which it does only on some seeds \(Appendix[B\.2](https://arxiv.org/html/2610.28771#A2.SS2)\)\.

##### Motion generalization\.

We evaluate the trained models as the switching rate, jump size and maximum speed move away from training \(Figure[6](https://arxiv.org/html/2610.28771#S5.F6)\)\. As in prediction \(Figure[4](https://arxiv.org/html/2610.28771#S5.F4)\), FELSTM loses accuracy as velocity changes become frequent, since each change moves its evidence to another channel, and it does not generalize beyond its velocity grid; MELSTM, which re\-estimates its velocities at every step, generalizes on all three axes\. LSTM’s3×33\\times 3recurrence moves its memory by one pixel per step, so at speed11it can follow the digit and improves; faster digits leave it only the comparison of the last two frames, which fails beyond33pixels per frame\. Like FELSTM, it also loses accuracy as velocity changes become more frequent, which suggests that it too accumulates evidence only while the velocity stays the same\.

### 5\.2KTH Action Recognition under Camera Motion

![Refer to caption](https://arxiv.org/html/2610.28771v1/kth_val_and_shake_k2.png)Figure 7:KTH: training and generalization to camera shake\.Left: validation accuracy under the training shake\. Right: test accuracy under shakes of amplitudeAAand frequencyff\(A=0A=0: static camera\); mean±\\pms\.d\. over three seeds\.##### Data and task\.

KTH\([Schuldt et al\., 2004](https://arxiv.org/html/2610.28771#bib.bib20)\)contains videos of2525people performing six actions, such as walking, boxing and hand waving, filmed by a mostly static camera\. We simulate a moving camera by translating every frame, which moves the whole scene but leaves the action unchanged \(Appendix[B\.3](https://arxiv.org/html/2610.28771#A2.SS3)\)\. The models are trained to recognize the action under a small, slow camera shake, a sinusoid on each axis with amplitudeA≤2A\\leq 2px and frequencyf=0\.4f=0\.4–0\.80\.8Hz, and tested under larger and faster shakes and other camera motions\. FELSTM enumerates the99velocities with‖v‖∞≤1\\\|v\\\|\_\{\\infty\}\\leq 1, which contain every step of the training shake\. MELSTM has two slots and uses slot tracking, which is equivariant to any camera path; early in training it also uses frame\-pair velocities, with the stochastic schedule of Appendix[B\.1\.5](https://arxiv.org/html/2610.28771#A2.SS1.SSS5)\. All models are trained for500500epochs with cross\-entropy loss\.

##### Camera\-motion generalization\.

With a static camera \(A=0A=0\) the three models are comparable\. Under the training shake, MELSTM learns more slowly while frame\-pair velocities are withdrawn, but ends with the highest validation accuracy \(Figure[7](https://arxiv.org/html/2610.28771#S5.F7)\)\. As the shake grows, LSTM and FELSTM lose accuracy, already inside the training region, and approach chance for large, slow shakes, while MELSTM keeps its static accuracy everywhere, up to camera steps of1111pixels per frame\. FELSTM gains little over LSTM, although its lattice contains every step of the training shake: each of its copies moves at one fixed velocity, while a shake keeps changing velocity\.

## 6Discussion

##### Related work\.

Most equivariant recurrent and dynamical models are equivariant to static transformations, applied to every frame alike\([Azari & Erdoğmuş, 2022](https://arxiv.org/html/2610.28771#bib.bib1);[Nguyen et al\., 2023](https://arxiv.org/html/2610.28771#bib.bib18)\)\. Symmetries that mix space and time, such as Galilean boosts in fluid dynamics\([Wang et al\., 2021](https://arxiv.org/html/2610.28771#bib.bib25)\)and Lorentz boosts in particle physics\([Bogatskiy et al\., 2020](https://arxiv.org/html/2610.28771#bib.bib3)\), act on the whole space\-time signal as one group element\. FERNNs\([Keller, 2025](https://arxiv.org/html/2610.28771#bib.bib12)\)and their extensions to world models with known self\-motion\([Lillemark et al\., 2026](https://arxiv.org/html/2610.28771#bib.bib16)\)and to transformers\([Khaliliya & Keller, 2026](https://arxiv.org/html/2610.28771#bib.bib13)\)are flow equivariant by construction; MELSTM instead estimates the motion from the data and is equivariant to motions whose velocity changes at every step\. Recurrent video models such as TrajGRU\([Shi et al\., 2017](https://arxiv.org/html/2610.28771#bib.bib22)\)and MotionRNN\([Wu et al\., 2021](https://arxiv.org/html/2610.28771#bib.bib28)\)also move their hidden states along flow fields that they learn from the data, which improves the prediction of complex motion\. Here we treat motion as a gauge symmetry, under which the recurrent model is exactly equivariant whenever its transporter is \(Theorem[2](https://arxiv.org/html/2610.28771#Thmtheorem2)\)\. Equation \([20](https://arxiv.org/html/2610.28771#S4.E20)\) states what this requires of the velocity model, and our experiments measure what the symmetry buys by comparing models that share an architecture and differ only in their transport\.

##### Future directions\.

Our symmetry groups act on each frame as a whole or, through slots, on a few components, each moved by a single group element, and the velocity model in our experiments is parameter\-free\. We see three directions that follow from the construction\.

*Trainable velocity models\.*Any velocity model that satisfies \([20](https://arxiv.org/html/2610.28771#S4.E20)\) keeps MELSTM exactly equivariant, including a trainable one, such as a recurrent network coupled to MELSTM\. Phase correlation on learned convolutional features instead of raw pixels still satisfies it, and would let the model learn which part of the change between frames is motion and which is intrinsic \(Figure[1](https://arxiv.org/html/2610.28771#S1.F1)\)\. A recurrent velocity model could also predict the motion after the context\.

*Diffeomorphisms\.*Atmospheric fields deform as they move, so their motion is better described by a time\-dependent diffeomorphism\. ForG=Diff⁡\(M\)G=\\mathrm\{Diff\}\(M\), the transporter advects the states along a velocity field\. Replacing convolutions by diffeomorphism\-equivariant maps, approximated by canonicalization through image registration\([Oettinger et al\., 2026](https://arxiv.org/html/2610.28771#bib.bib19)\), would give a model suited to weather forecasting\.

*Transformers\.*Attention from framettto framesscan read the keys and values of framesstransported byTt,sT\_\{t,s\}\. By \([19](https://arxiv.org/html/2610.28771#S4.E19)\) these transform withψt\\psi\_\{t\}, like the query, which would extend flow equivariant transformers\([Khaliliya & Keller, 2026](https://arxiv.org/html/2610.28771#bib.bib13)\)from flows to motions\.

#### Acknowledgments

V\.H\. and M\.P\. received support from National Science Foundation Award \#2418909 LEAPS\-MPS: Topological Machine Learning and Cryo\-EM\. T\.A\.K\. acknowledges that this work has been made possible in part by a gift from the Chan Zuckerberg Initiative Foundation to establish the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University\. M\.P\. recieved support from CAPITAL Services\.

## References

- Azari & Erdoğmuş \(2022\)Bahar Azari and Deniz Erdoğmuş\.Equivariant deep dynamical model for motion prediction\.In*Proceedings of the 25th International Conference on Artificial Intelligence and Statistics \(AISTATS\)*, volume 151 of*Proceedings of Machine Learning Research*, 2022\.
- Bietti et al\. \(2021\)Alberto Bietti, Luca Venturi, and Joan Bruna\.On the sample complexity of learning under geometric stability\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 34, 2021\.
- Bogatskiy et al\. \(2020\)Alexander Bogatskiy, Brandon Anderson, Jan T\. Offermann, Marwah Roussi, David W\. Miller, and Risi Kondor\.Lorentz group equivariant neural network for particle physics\.In*Proceedings of the 37th International Conference on Machine Learning \(ICML\)*, volume 119 of*Proceedings of Machine Learning Research*, 2020\.
- Bronstein et al\. \(2021\)Michael M\. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković\.Geometric deep learning: Grids, groups, graphs, geodesics, and gauges\.*arXiv preprint arXiv:2104\.13478*, 2021\.
- Cohen & Welling \(2016\)Taco Cohen and Max Welling\.Group equivariant convolutional networks\.In*Proceedings of the 33rd International Conference on Machine Learning \(ICML\)*, volume 48 of*Proceedings of Machine Learning Research*, pp\. 2990–2999, 2016\.
- Cohen & Welling \(2017\)Taco S\. Cohen and Max Welling\.Steerable CNNs\.In*International Conference on Learning Representations \(ICLR\)*, 2017\.
- Cohen et al\. \(2019a\)Taco S\. Cohen, Mario Geiger, and Maurice Weiler\.A general theory of equivariant CNNs on homogeneous spaces\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 32, 2019a\.
- Cohen et al\. \(2019b\)Taco S\. Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling\.Gauge equivariant convolutional networks and the icosahedral CNN\.In*Proceedings of the 36th International Conference on Machine Learning \(ICML\)*, volume 97 of*Proceedings of Machine Learning Research*, 2019b\.
- Dummit & Foote \(2004\)David S\. Dummit and Richard M\. Foote\.*Abstract Algebra*\.John Wiley & Sons, 3 edition, 2004\.
- Elesedy & Zaidi \(2021\)Bryn Elesedy and Sheheryar Zaidi\.Provably strict generalisation benefit for equivariant models\.In*Proceedings of the 38th International Conference on Machine Learning \(ICML\)*, volume 139 of*Proceedings of Machine Learning Research*, 2021\.
- Finzi et al\. \(2020\)Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson\.Generalizing convolutional neural networks for equivariance to Lie groups on arbitrary continuous data\.In*Proceedings of the 37th International Conference on Machine Learning \(ICML\)*, volume 119 of*Proceedings of Machine Learning Research*, 2020\.
- Keller \(2025\)Andy Keller\.Flow equivariant recurrent neural networks\.In D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(eds\.\),*Advances in Neural Information Processing Systems*, volume 38, Main Conference, pp\. 156934–156979\. Curran Associates, Inc\., 2025\.doi:10\.52202/085713\-5244\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2025/file/e637029c42aa593850eeebf46616444d\-Paper\-Conference\.pdf](https://proceedings.neurips.cc/paper_files/paper/2025/file/e637029c42aa593850eeebf46616444d-Paper-Conference.pdf)\.
- Khaliliya & Keller \(2026\)Ibrahim Khaliliya and T\. Anderson Keller\.Flow equivariant transformers\.In*ICML 2026 Workshop on Weight\-Space Symmetries: from Foundations to Practical Applications*, 2026\.URL[https://openreview\.net/forum?id=lXTPf5HDHy](https://openreview.net/forum?id=lXTPf5HDHy)\.
- Kondor & Trivedi \(2018\)Risi Kondor and Shubhendu Trivedi\.On the generalization of equivariance and convolution in neural networks to the action of compact groups\.In*Proceedings of the 35th International Conference on Machine Learning \(ICML\)*, volume 80 of*Proceedings of Machine Learning Research*, 2018\.
- LeCun et al\. \(1989\)Yann LeCun, Bernhard Boser, John S\. Denker, Donnie Henderson, Richard E\. Howard, Wayne Hubbard, and Lawrence D\. Jackel\.Backpropagation applied to handwritten zip code recognition\.*Neural Computation*, 1\(4\):541–551, 1989\.
- Lillemark et al\. \(2026\)Hansen Lillemark, Benhao Huang, Fangneng Zhan, Yilun Du, and T\. Anderson Keller\.Flow equivariant world models: Structured memory for dynamic environments\.In*International Conference on Machine Learning \(ICML\)*, 2026\.arXiv:2601\.01075\.
- Nakahara \(2003\)Mikio Nakahara\.*Geometry, Topology and Physics*\.Institute of Physics Publishing, Bristol, second edition, 2003\.
- Nguyen et al\. \(2023\)Hai Huu Nguyen, Andrea Baisero, David Klee, Dian Wang, Robert Platt, and Christopher Amato\.Equivariant reinforcement learning under partial observability\.In*Proceedings of the 7th Conference on Robot Learning \(CoRL\)*, volume 229 of*Proceedings of Machine Learning Research*, 2023\.
- Oettinger et al\. \(2026\)Josephine Elisabeth Oettinger, Zakhar Shumaylov, Johannes Bostelmann, Jan Lellmann, and Carola\-Bibiane Schönlieb\.Diffeomorphism\-equivariant neural networks\.*arXiv preprint arXiv:2602\.06695*, 2026\.
- Schuldt et al\. \(2004\)Christian Schuldt, Ivan Laptev, and Barbara Caputo\.Recognizing human actions: A local SVM approach\.In*Proceedings of the 17th International Conference on Pattern Recognition \(ICPR\)*, volume 3, pp\. 32–36, 2004\.
- Shi et al\. \(2015\)Xingjian Shi, Zhourong Chen, Hao Wang, Dit\-Yan Yeung, Wai\-Kin Wong, and Wang\-chun Woo\.Convolutional LSTM network: A machine learning approach for precipitation nowcasting\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 28, 2015\.
- Shi et al\. \(2017\)Xingjian Shi, Zhihan Gao, Leonard Lausen, Hao Wang, Dit\-Yan Yeung, Wai\-kin Wong, and Wang\-chun Woo\.Deep learning for precipitation nowcasting: A benchmark and a new model\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 30, 2017\.
- Srivastava et al\. \(2015\)Nitish Srivastava, Elman Mansimov, and Ruslan Salakhutdinov\.Unsupervised learning of video representations using LSTMs\.In*Proceedings of the 32nd International Conference on Machine Learning \(ICML\)*, volume 37 of*Proceedings of Machine Learning Research*, 2015\.
- Upadhyay et al\. \(2026\)Ujjwal Upadhyay, Mukul Ranjan, Zhiqiang Shen, and Mohamed Elhoseiny\.Time Blindness: Why video\-language models can’t see what humans can?In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2026\.
- Wang et al\. \(2021\)Rui Wang, Robin Walters, and Rose Yu\.Incorporating symmetry into deep dynamics models for improved generalization\.In*International Conference on Learning Representations \(ICLR\)*, 2021\.
- Weiler & Cesa \(2019\)Maurice Weiler and Gabriele Cesa\.General E\(2\)\-equivariant steerable CNNs\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 32, 2019\.
- Weiler et al\. \(2026\)Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling\.*Equivariant and Coordinate Independent Convolutional Networks: A Gauge Field Theory of Neural Networks*\.Progress in Data Science\. World Scientific, New Jersey, 2026\.doi:10\.1142/14143\.
- Wu et al\. \(2021\)Haixu Wu, Zhiyu Yao, Jianmin Wang, and Mingsheng Long\.MotionRNN: A flexible model for video prediction with spacetime\-varying motions\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pp\. 15435–15444, 2021\.
- Wu & Johnson \(2021\)Yuxin Wu and Justin Johnson\.Rethinking “Batch” in BatchNorm\.*arXiv preprint arXiv:2105\.07576*, 2021\.
- Yang & Mills \(1954\)Chen\-Ning Yang and Robert L\. Mills\.Conservation of isotopic spin and isotopic gauge invariance\.*Physical Review*, 96\(1\):191–195, 1954\.

###### Appendix Contents

1. [1Introduction](https://arxiv.org/html/2610.28771#S1)
2. [2Equivariance; static vs dynamical symmetries](https://arxiv.org/html/2610.28771#S2)
3. [3Spatiotemporal data and Equivariant Sequential Models](https://arxiv.org/html/2610.28771#S3)
4. [4Motion Equivariant Convolutional LSTM](https://arxiv.org/html/2610.28771#S4)
5. [5Experiments](https://arxiv.org/html/2610.28771#S5)1. [5\.1Moving MNIST](https://arxiv.org/html/2610.28771#S5.SS1)1. [5\.1\.1Future\-Frame Prediction](https://arxiv.org/html/2610.28771#S5.SS1.SSS1) 2. [5\.1\.2Motion\-Only MNIST Classification](https://arxiv.org/html/2610.28771#S5.SS1.SSS2) 2. [5\.2KTH Action Recognition under Camera Motion](https://arxiv.org/html/2610.28771#S5.SS2)
6. [6Discussion](https://arxiv.org/html/2610.28771#S6)
7. [References](https://arxiv.org/html/2610.28771#bib)
8. [AProofs](https://arxiv.org/html/2610.28771#A1)1. [A\.1The structure of subgroups of𝒢\\mathcal\{G\}](https://arxiv.org/html/2610.28771#A1.SS1) 2. [A\.2Global G\-equivariance of LSTM model](https://arxiv.org/html/2610.28771#A1.SS2) 3. [A\.3LSTM is not flow equivariant](https://arxiv.org/html/2610.28771#A1.SS3) 4. [A\.4Transport and flow convolution](https://arxiv.org/html/2610.28771#A1.SS4) 5. [A\.5Flow equivariance of FELSTM](https://arxiv.org/html/2610.28771#A1.SS5) 6. [A\.6Equivariance of the Velocity model](https://arxiv.org/html/2610.28771#A1.SS6) 7. [A\.7Motion equivariance of the Velocity Based MELSTM](https://arxiv.org/html/2610.28771#A1.SS7) 8. [A\.8Time\-dependent data and motion equivariance as a gauge theory](https://arxiv.org/html/2610.28771#A1.SS8)
9. [BExperimental Details](https://arxiv.org/html/2610.28771#A2)1. [B\.1Future\-Frame Prediction](https://arxiv.org/html/2610.28771#A2.SS1)1. [B\.1\.1Moving MNIST Dataset](https://arxiv.org/html/2610.28771#A2.SS1.SSS1) 2. [B\.1\.2Training and Evaluation Setup](https://arxiv.org/html/2610.28771#A2.SS1.SSS2) 3. [B\.1\.3Length and Motion Generalization](https://arxiv.org/html/2610.28771#A2.SS1.SSS3) 4. [B\.1\.4Velocity Model](https://arxiv.org/html/2610.28771#A2.SS1.SSS4) 5. [B\.1\.5Training with Frame\-Pair Velocities](https://arxiv.org/html/2610.28771#A2.SS1.SSS5) 2. [B\.2Motion\-Only MNIST Classification](https://arxiv.org/html/2610.28771#A2.SS2) 3. [B\.3KTH Action Recognition under Camera Motion](https://arxiv.org/html/2610.28771#A2.SS3)

## Appendix AProofs

### A\.1The structure of subgroups of𝒢\\mathcal\{G\}

###### Lemma 1\.

The motions form a subgroup of𝒢\\mathcal\{G\}, while the flows form a subgroup of𝒢\\mathcal\{G\}if and only ifGGis an abelian group\.

###### Proof\.

We assume thatψ\(1\)\\psi^\{\(1\)\}andψ\(2\)\\psi^\{\(2\)\}are two motions generated by the velocitiesv\(1\)v^\{\(1\)\}andv\(2\)v^\{\(2\)\}, respectively\. We show thatψt=ψt\(1\)​ψt\(2\)\\psi\_\{t\}=\\psi^\{\(1\)\}\_\{t\}\\psi^\{\(2\)\}\_\{t\}is again a motion\. Differentiating with respect to time and using the product rule givesdd​t​ψt=ψ˙t\(1\)​ψt\(2\)\+ψt\(1\)​ψ˙t\(2\)\\frac\{d\}\{dt\}\\psi\_\{t\}=\\dot\{\\psi\}^\{\(1\)\}\_\{t\}\\psi^\{\(2\)\}\_\{t\}\+\\psi^\{\(1\)\}\_\{t\}\\dot\{\\psi\}^\{\(2\)\}\_\{t\}\. Substituting the evolution equations forψ\(1\)\\psi^\{\(1\)\}andψ\(2\)\\psi^\{\(2\)\}yields

dd​t​\(ψt\(1\)​ψt\(2\)\)=ψ˙t\\displaystyle\\frac\{d\}\{dt\}\(\\psi^\{\(1\)\}\_\{t\}\\psi^\{\(2\)\}\_\{t\}\)=\\dot\{\\psi\}\_\{t\}=vt\(1\)​ψt\(1\)​ψt\(2\)\+ψt\(1\)​vt\(2\)​ψt\(2\)\\displaystyle=v^\{\(1\)\}\_\{t\}\\psi^\{\(1\)\}\_\{t\}\\psi^\{\(2\)\}\_\{t\}\+\\psi^\{\(1\)\}\_\{t\}v^\{\(2\)\}\_\{t\}\\psi^\{\(2\)\}\_\{t\}\(25\)=vt\(1\)​ψt\(1\)​ψt\(2\)\+ψt\(1\)​vt\(2\)​ψt\(1\)−1​ψt\(1\)​ψt\(2\)\\displaystyle=v^\{\(1\)\}\_\{t\}\\psi^\{\(1\)\}\_\{t\}\\psi^\{\(2\)\}\_\{t\}\+\\psi^\{\(1\)\}\_\{t\}v^\{\(2\)\}\_\{t\}\{\\psi^\{\(1\)\}\_\{t\}\}^\{\-1\}\\psi^\{\(1\)\}\_\{t\}\\psi^\{\(2\)\}\_\{t\}\(26\)Using the definition of the adjoint actionAdg⁡\(v\)=g​v​g−1\\operatorname\{Ad\}\_\{g\}\(v\)=gvg^\{\-1\}forg∈Gg\\in Gandv∈𝔤v\\in\\mathfrak\{g\}, we conclude thatψt\\psi\_\{t\}is again a motion generated by the velocity

vt=vt\(1\)\+Adψt\(1\)⁡\(vt\(2\)\)\\displaystyle v\_\{t\}=v^\{\(1\)\}\_\{t\}\+\\operatorname\{Ad\}\_\{\\psi^\{\(1\)\}\_\{t\}\}\(v^\{\(2\)\}\_\{t\}\)\(27\)which is simplyvt=vt\(1\)\+vt\(2\)v\_\{t\}=v^\{\(1\)\}\_\{t\}\+v^\{\(2\)\}\_\{t\}in the case of an abelian group where the adjoint action is trivial\. Similarly, we can show that the inverse of a motion is again a motion, and therefore the motions form a subgroup of𝒢\\mathcal\{G\}\. This is proved as follows\. Starting with the identityψt−1​ψt=e\\psi\_\{t\}^\{\-1\}\\psi\_\{t\}=eand differentiating givesψ˙t−1​ψt\+ψt−1​ψ˙t=0\\dot\{\\psi\}\_\{t\}^\{\-1\}\\psi\_\{t\}\+\\psi\_\{t\}^\{\-1\}\\dot\{\\psi\}\_\{t\}=0\. Substitutingψ˙t=vt​ψt\\dot\{\\psi\}\_\{t\}=v\_\{t\}\\psi\_\{t\}yieldsψ˙t−1​ψt\+ψt−1​vt​ψt=0\\dot\{\\psi\}\_\{t\}^\{\-1\}\\psi\_\{t\}\+\\psi\_\{t\}^\{\-1\}v\_\{t\}\\psi\_\{t\}=0\. Multiplying on the right byψt−1\\psi\_\{t\}^\{\-1\}givesψ˙t−1\+ψt−1​vt=0\\dot\{\\psi\}\_\{t\}^\{\-1\}\+\\psi\_\{t\}^\{\-1\}v\_\{t\}=0, soψ˙t−1=−ψt−1​vt\\dot\{\\psi\}\_\{t\}^\{\-1\}=\-\\psi\_\{t\}^\{\-1\}v\_\{t\}\. Rewriting in right\-invariant form givesψ˙t−1=−Adψt−1⁡\(vt\)​ψt−1\\dot\{\\psi\}\_\{t\}^\{\-1\}=\-\\operatorname\{Ad\}\_\{\\psi\_\{t\}^\{\-1\}\}\(v\_\{t\}\)\\psi\_\{t\}^\{\-1\}, showing thatψt−1\\psi\_\{t\}^\{\-1\}is a motion generated by the velocityvtinv=−Adψt−1⁡\(vt\)v^\{\\mathrm\{inv\}\}\_\{t\}=\-\\operatorname\{Ad\}\_\{\\psi\_\{t\}^\{\-1\}\}\(v\_\{t\}\)\.

For the second part of the lemma we proceed as follows\. For the forward direction, ifGGis abelian then the composition of two flowsϕt\(1\)=exp⁡\(t​ν1\)\\phi^\{\(1\)\}\_\{t\}=\\exp\(t\\nu\_\{1\}\)andϕt\(2\)=exp⁡\(t​ν2\)\\phi^\{\(2\)\}\_\{t\}=\\exp\(t\\nu\_\{2\}\)is given byϕt=ϕt\(1\)​ϕt\(2\)=exp⁡\(t⁡\(ν1\+ν2\)\)\\phi\_\{t\}=\\phi^\{\(1\)\}\_\{t\}\\phi^\{\(2\)\}\_\{t\}=\\exp\(t\(\\nu\_\{1\}\+\\nu\_\{2\}\)\), which is again a flow\. The inverse of a flowϕt=exp⁡\(t​ν\)\\phi\_\{t\}=\\exp\(t\\nu\)is given byϕt−1=exp⁡\(−t​ν\)\\phi\_\{t\}^\{\-1\}=\\exp\(\-t\\nu\), which is also a flow\. Thus, ifGGis abelian, the flows form a subgroup of𝒢\\mathcal\{G\}\. For the converse direction, if the flows form a subgroup of𝒢\\mathcal\{G\}, then for anyν1,ν2∈𝔤\\nu\_\{1\},\\nu\_\{2\}\\in\\mathfrak\{g\}, the composition of the corresponding flowsϕt\(1\)=exp⁡\(t​ν1\)\\phi^\{\(1\)\}\_\{t\}=\\exp\(t\\nu\_\{1\}\)andϕt\(2\)=exp⁡\(t​ν2\)\\phi^\{\(2\)\}\_\{t\}=\\exp\(t\\nu\_\{2\}\)must be a flow\. This implies thatexp⁡\(t​ν1\)​exp⁡\(t​ν2\)=exp⁡\(t​ν\)\\exp\(t\\nu\_\{1\}\)\\exp\(t\\nu\_\{2\}\)=\\exp\(t\\nu\)for someν∈𝔤\\nu\\in\\mathfrak\{g\}\. Differentiating att=0t=0givesν=ν1\+ν2\\nu=\\nu\_\{1\}\+\\nu\_\{2\}\. Using the Baker\-Campbell\-Hausdorff formula,exp⁡\(t​ν1\)​exp⁡\(t​ν2\)=exp⁡\(t⁡\(ν1\+ν2\)\+t22​\[ν1,ν2\]\+𝒪⁡\(t3\)\)\\exp\(t\\nu\_\{1\}\)\\exp\(t\\nu\_\{2\}\)=\\exp\\Big\(t\(\\nu\_\{1\}\+\\nu\_\{2\}\)\+\\tfrac\{t^\{2\}\}\{2\}\[\\nu\_\{1\},\\nu\_\{2\}\]\+\\mathcal\{O\}\(t^\{3\}\)\\Big\)shows then that equality for allttis possible only if\[ν1,ν2\]=0\[\\nu\_\{1\},\\nu\_\{2\}\]=0for allν1,ν2∈𝔤\\nu\_\{1\},\\nu\_\{2\}\\in\\mathfrak\{g\}\. Therefore, the Lie algebra𝔤\\mathfrak\{g\}is abelian, and hence the Lie groupGGis abelian\. ∎

##### Group structure of𝒢\{\\cal G\}\.

Consider the subgroup𝒢0=\{γ∈𝒢:γ0=e\}\{\\cal G\}\_\{0\}=\\\{\\gamma\\in\{\\cal G\}:\\gamma\_\{0\}=e\\\}consisting of time\-dependent transformations initialized at the identity\. This is a normal subgroup of𝒢\{\\cal G\}and satisfies𝒢0∩𝒢g​l​o​b=\{e\}\{\\cal G\}\_\{0\}\\cap\{\\cal G\}\_\{glob\}=\\\{e\\\}\. Moreover, every elementγ∈𝒢\\gamma\\in\{\\cal G\}admits a unique decomposition into a component in𝒢0\{\\cal G\}\_\{0\}and a global component in𝒢g​l​o​b\{\\cal G\}\_\{glob\}, sinceγt​γ0−1∈𝒢0\\gamma\_\{t\}\\gamma\_\{0\}^\{\-1\}\\in\{\\cal G\}\_\{0\}andγ0∈𝒢g​l​o​b\\gamma\_\{0\}\\in\{\\cal G\}\_\{glob\}\. Hence, everyγ∈𝒢\\gamma\\in\{\\cal G\}can be written uniquely asγ=\(γ​γ0−1\)​γ0\\gamma=\(\\gamma\\gamma\_\{0\}^\{\-1\}\)\\gamma\_\{0\}, which gives the internal product decomposition𝒢=𝒢0​𝒢g​l​o​b\{\\cal\{G\}\}=\{\\cal\{G\}\}\_\{0\}\{\\cal\{G\}\}\_\{glob\}\. Therefore, by[Dummit & Foote \(2004, Theorem 12, Section 5\.5\)](https://arxiv.org/html/2610.28771#bib.bib9),𝒢=𝒢0⋊𝒢g​l​o​b\{\\cal\{G\}\}=\{\\cal\{G\}\}\_\{0\}\\rtimes\{\\cal\{G\}\}\_\{glob\}\. The semidirect product structure is determined by the conjugation action of𝒢g​l​o​b\{\\cal G\}\_\{glob\}on𝒢0\{\\cal G\}\_\{0\}, given by\(g⋅γ\)t=g​γt​g−1\{\(g\\cdot\\gamma\)\}\_\{t\}=g\\gamma\_\{t\}g^\{\-1\}forg∈𝒢g​l​o​bg\\in\{\\cal G\}\_\{glob\}andγ∈𝒢0\\gamma\\in\{\\cal G\}\_\{0\}\. Since motions and flows satisfyψ0=e\\psi\_\{0\}=eandϕ0=e\\phi\_\{0\}=e, respectively, they are naturally subsets of𝒢0\{\\cal G\}\_\{0\}\. Consequently,𝒢f​l​o​w⊂𝒢m​o​t≤𝒢0​⊲​𝒢\{\\cal G\}\_\{flow\}\\subset\{\\cal G\}\_\{mot\}\\leq\{\\cal G\}\_\{0\}\\vartriangleleft\{\\cal G\}, and𝒢g​l​o​b≤𝒢\{\\cal G\}\_\{glob\}\\leq\{\\cal G\}\.

### A\.2Global G\-equivariance of LSTM model

###### Theorem 3\(LSTM is globallyGG\-equivariant\)\.

Consider the LSTM model defined by equations \([8](https://arxiv.org/html/2610.28771#S4.E8)\)–\([10](https://arxiv.org/html/2610.28771#S4.E10)\), with invariant initial state variables satisfyingg⊳ℋ0=ℋ0g\\triangleright\{\\cal\{H\}\}\_\{0\}=\{\\cal\{H\}\}\_\{0\}andg⊳𝒞0=𝒞0g\\triangleright\{\\cal\{C\}\}\_\{0\}=\{\\cal\{C\}\}\_\{0\}for allg∈𝒢g​l​o​bg\\in\{\\cal G\}\_\{glob\}\. Then the model is globallyGG\-equivariant, i\.e\., for any global transformationg∈𝒢g​l​o​bg\\in\{\\cal G\}\_\{glob\}and any trajectory𝒳∈ℱ\{\\cal\{X\}\}\\in\{\\cal F\},

ℋt​\[g⊳𝒳≤t\]=g⊳ℋt​\[𝒳≤t\]∀t\.\\displaystyle\{\\cal\{H\}\}\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=g\\triangleright\{\\cal\{H\}\}\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\qquad\\forall t\.\(28\)

###### Proof\.

We want to prove

ℋt​\[g⊳𝒳≤t\]=g⊳ℋt​\[𝒳≤t\]∀t,∀g∈𝒢g​l​o​b\\displaystyle\{\\cal H\}\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=g\\triangleright\{\\cal H\}\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\qquad\\forall t,\\qquad\\forall g\\in\{\\cal\{G\}\}\_\{glob\}\(29\)The proof is by induction ontt\. For the base case, we consider the initial states𝒞0\{\\cal C\}\_\{0\}andℋ0\{\\cal H\}\_\{0\}to be invariant under the group action, sog⊳𝒞0​\(x\)=𝒞0​\(x\)g\\triangleright\{\\cal\{C\}\}\_\{0\}\(x\)=\{\\cal\{C\}\}\_\{0\}\(x\)andg⊳ℋ0​\(x\)=ℋ0​\(x\)g\\triangleright\{\\cal\{H\}\}\_\{0\}\(x\)=\{\\cal\{H\}\}\_\{0\}\(x\)\. Moreover, the empty sequence𝒳<0\{\\cal\{X\}\}\_\{<0\}is trivially invariant under the group action, so we have the equivariance at timet=0t=0as followsg⊳ℋ0​\[𝒳<0\]=ℋ0​\[𝒳<0\]=ℋ0​\[g⊳𝒳<0\]g\\triangleright\{\\cal\{H\}\}\_\{0\}\[\{\\cal\{X\}\}\_\{<0\}\]=\{\\cal\{H\}\}\_\{0\}\[\{\\cal\{X\}\}\_\{<0\}\]=\{\\cal\{H\}\}\_\{0\}\[g\\triangleright\{\\cal\{X\}\}\_\{<0\}\]\(similarly for𝒞0\{\\cal C\}\_\{0\}\)\. Now, assuming that the equivariance holds at timettwe want to show that it also holds at timet\+1t\+1\. First note that for the gate variables we have

zt​\[g⊳𝒳≤t\]​\(x\)\\displaystyle z\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)=az​\[\[Wz⋆ℋt​\[g⊳𝒳<t\]\]​\(x\)\+\[Uz⋆\(g⊳𝒳t\)\]​\(x\)\]\\displaystyle=a\_\{z\}\\Big\[\[W\_\{z\}\\star\{\\cal H\}\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{<t\}\]\]\(x\)\+\[U\_\{z\}\\star\(g\\triangleright\{\\cal X\}\_\{t\}\)\]\(x\)\\Big\]\(30\)=az​\[\[Wz⋆\(g⋅ℋt​\[𝒳<t\]\)\]​\(x\)\+\[Uz⋆\(g⋅𝒳t\)\]​\(x\)\]\\displaystyle=a\_\{z\}\\Big\[\[W\_\{z\}\\star\(g\\cdot\{\\cal H\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\)\]\(x\)\+\[U\_\{z\}\\star\(g\\cdot\{\\cal X\}\_\{t\}\)\]\(x\)\\Big\]\(31\)=az​\[g⋅\[Wz⋆ℋt​\[𝒳<t\]\]​\(x\)\+g⋅\[Uz⋆𝒳t\]​\(x\)\]\\displaystyle=a\_\{z\}\\Big\[g\\cdot\[W\_\{z\}\\star\{\\cal H\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\]\(x\)\+g\\cdot\[U\_\{z\}\\star\{\\cal X\}\_\{t\}\]\(x\)\\Big\]\(32\)=g⊳zt​\[𝒳≤t\]​\(x\)\\displaystyle=g\\triangleright z\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(33\)for allz∈\{i,f,o,c\}z\\in\\\{i,f,o,c\\\}\. Here in the second line, we used the induction hypothesis for the hidden state and in third line the equivariance of the steerable convolution under regular action of the group\. For the cell state we have

𝒞t\+1​\[g⊳𝒳≤t\]​\(x\)\\displaystyle\{\\cal\{C\}\}\_\{t\+1\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)=𝒞t​\[g⊳𝒳<t\]​\(x\)⊙ft​\[g⊳𝒳≤t\]​\(x\)\+it​\[g⊳𝒳≤t\]​\(x\)⊙ct​\[g⊳𝒳≤t\]​\(x\)\\displaystyle=\{\\cal C\}\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{<t\}\]\(x\)\\odot f\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\+i\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot c\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(34\)=\[g⋅𝒞t​\[𝒳<t\]\]​\(x\)⊙\[g⋅ft​\[𝒳≤t\]\]​\(x\)\+\[g⋅it​\[𝒳≤t\]\]​\(x\)⊙\[g⋅ct​\[𝒳≤t\]\]​\(x\)\\displaystyle=\[g\\cdot\{\\cal C\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\]\(x\)\\odot\[g\\cdot f\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\]\(x\)\+\[g\\cdot i\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\]\(x\)\\odot\[g\\cdot c\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\]\(x\)\(35\)=g⋅\[𝒞t​\[𝒳<t\]⊙ft​\[𝒳≤t\]\]​\(x\)\+g⋅\[it​\[𝒳≤t\]⊙ct​\[𝒳≤t\]\]​\(x\)\\displaystyle=g\\cdot\[\{\\cal C\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\\odot f\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\]\(x\)\+g\\cdot\[i\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\odot c\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\]\(x\)\(36\)=g⊳𝒞t\+1​\[𝒳≤t\]​\(x\)\\displaystyle=g\\triangleright\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(37\)where in the second line we used the induction hypothesis for the cell state and the gate variables and in the third line we used the fact that the group action distributes over the Hadamard product\. Finally for the hidden state we have

ℋt\+1​\[g⊳𝒳≤t\]​\(x\)\\displaystyle\{\\cal\{H\}\}\_\{t\+1\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)=ot​\[g⊳𝒳≤t\]​\(x\)⊙tanh⁡\(𝒞t\+1​\[g⊳𝒳≤t\]​\(x\)\)\\displaystyle=o\_\{t\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot\\tanh\(\{\\cal C\}\_\{t\+1\}\[g\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\)\(38\)=g⋅ot​\[𝒳≤t\]​\(x\)⊙tanh⁡\(g⋅𝒞t\+1​\[𝒳≤t\]​\(x\)\)\\displaystyle=g\\cdot o\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot\\tanh\(g\\cdot\{\\cal C\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\)\(39\)=g⊳ℋt\+1​\[𝒳≤t\]​\(x\)\\displaystyle=g\\triangleright\{\\cal\{H\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(40\)where in the second line we used the induction hypothesis for the gate variableooand the cell state and in the third line we used the fact that the group action distributes over the Hadamard product and also commutes with the pointwise nonlinearitytanh\\tanh\. ∎

### A\.3LSTM is not flow equivariant

###### Proposition 1\.

Let𝒩\\cal Nbe a LSTM defined by equations \([8](https://arxiv.org/html/2610.28771#S4.E8)\)–\([10](https://arxiv.org/html/2610.28771#S4.E10)\), with invariant initial statesℋ0=𝒞0=0\{\\cal\{H\}\}\_\{0\}=\{\\cal\{C\}\}\_\{0\}=0whose kernels are supported in a ball of radiusRR, onℝ2\\mathbb\{R\}^\{2\}or on a torus of widthL\>8​RL\>8R\. If𝒩\\cal Nis flow equivariant, then

𝒞1⊙σ⁡\(Wf⋆ℋ1\)\+σ⁡\(Wi⋆ℋ1\)⊙tanh⁡\(Wc⋆ℋ1\)=0,𝒞1=σ⁡\(Ui\)⊙tanh⁡\(Uc\),ℋ1=σ⁡\(Uo\)⊙tanh⁡\(𝒞1\)\.\{\\cal\{C\}\}\_\{1\}\\odot\\sigma\\bigl\(W\_\{f\}\\star\{\\cal\{H\}\}\_\{1\}\\bigr\)\+\\sigma\\bigl\(W\_\{i\}\\star\{\\cal\{H\}\}\_\{1\}\\bigr\)\\odot\\tanh\\bigl\(W\_\{c\}\\star\{\\cal\{H\}\}\_\{1\}\\bigr\)=\\;0,\\qquad\\begin\{aligned\} \{\\cal\{C\}\}\_\{1\}&=\\sigma\(U\_\{i\}\)\\odot\\tanh\(U\_\{c\}\),\\\\ \{\\cal\{H\}\}\_\{1\}&=\\sigma\(U\_\{o\}\)\\odot\\tanh\(\{\\cal\{C\}\}\_\{1\}\)\.\\end\{aligned\}\(41\)Condition \([41](https://arxiv.org/html/2610.28771#A1.E41)\) fails for all kernels outside a Lebesgue measure zero set \(e\.g\. it fails wheneverWi=Wf=Wc=0W\_\{i\}=W\_\{f\}=W\_\{c\}=0andUc≠0U\_\{c\}\\neq 0\); hence𝒩\\cal Nis not flow equivariant for generic kernels\.

###### Proof\.

As a probe for flow equivariance, consider the sequence of static dots𝒳t=δ0\{\\cal\{X\}\}\_\{t\}=\\delta\_\{0\}for allttwhere a flow of velocityν\\numaps it to the moving dot\(ϕ⊳𝒳\)t=δt​ν\(\\phi\\triangleright\{\\cal\{X\}\}\)\_\{t\}=\\delta\_\{t\\nu\}\. Since\(U⋆δa\)​\(x\)=U⁡\(x−a\)\(U\\star\\delta\_\{a\}\)\(x\)=U\(x\-a\)and𝒞0=ℋ0=0\{\\cal\{C\}\}\_\{0\}=\{\\cal\{H\}\}\_\{0\}=0, the first time step gives𝒞1=σ⁡\(Ui\)​tanh⁡\(Uc\)\{\\cal\{C\}\}\_\{1\}=\\sigma\(U\_\{i\}\)\\tanh\(U\_\{c\}\)andℋ1=σ⁡\(Uo\)​tanh⁡\(𝒞1\)\{\\cal\{H\}\}\_\{1\}=\\sigma\(U\_\{o\}\)\\tanh\(\{\\cal\{C\}\}\_\{1\}\), supported in the ballBRB\_\{R\}\(they vanish whereUcU\_\{c\}does asσ⁡\(x\)\>0\\sigma\(x\)\>0andtanh⁡\(x\)=0\\tanh\(x\)=0iffx=0x=0\), so the memory termsPz:=Wz⋆ℋ1P\_\{z\}:=W\_\{z\}\\star\{\\cal\{H\}\}\_\{1\}are supported inB2​RB\_\{2R\}\. Choose4​R<\|ν\|<L−4​R4R<\|\\nu\|<L\-4Rand evaluate the equivariance condition at timet=1t=1,

ℋ2​\[\(δ0,δν\)\]​\(x\)=ℋ2​\[\(δ0,δ0\)\]​\(x−ν\),\{\\cal\{H\}\}\_\{2\}\\bigl\[\(\\delta\_\{0\},\\delta\_\{\\nu\}\)\\bigr\]\(x\)=\{\\cal\{H\}\}\_\{2\}\\bigl\[\(\\delta\_\{0\},\\delta\_\{0\}\)\\bigr\]\(x\-\\nu\),at points\|x\|≤2​R\|x\|\\leq 2R\. On the left, all time\-11input termsUz​\(x−ν\)U\_\{z\}\(x\-\\nu\)vanish, since\|x−ν\|≥\|ν\|−\|x\|\>2​R≥R\|x\-\\nu\|\\geq\|\\nu\|\-\|x\|\>2R\\geq R; the gates see only their memory, leavingσ⁡\(Po\)​tanh⁡\(𝒞1​σ​\(Pf\)\+σ⁡\(Pi\)​tanh⁡\(Pc\)\)​\(x\)\\sigma\(P\_\{o\}\)\\,\\tanh\\bigl\(\{\\cal\{C\}\}\_\{1\}\\,\\sigma\(P\_\{f\}\)\+\\sigma\(P\_\{i\}\)\\tanh\(P\_\{c\}\)\\bigr\)\(x\)\. On the right, every argument of the static response vanishes atx−νx\-\\nu\(distance\>2​R\>2Rfrom the dot\), givingσ⁡\(0\)​tanh⁡\(0\)=0\\sigma\(0\)\\tanh\(0\)=0\. Sinceσ\>0\\sigma\>0andtanh⁡\(s\)=0\\tanh\(s\)=0only ats=0s=0, so the memory content𝒞2m​e​m=𝒞1​σ​\(Pf\)\+σ⁡\(Pi\)​tanh⁡\(Pc\)\{\\cal\{C\}\}\_\{2\}^\{mem\}=\{\\cal\{C\}\}\_\{1\}\\,\\sigma\(P\_\{f\}\)\+\\sigma\(P\_\{i\}\)\\tanh\(P\_\{c\}\)must vanish on\|x\|≤2​R\|x\|\\leq 2Rwhere it is also zero trivially outside ofB2​RB\_\{2R\}\. This gives \([41](https://arxiv.org/html/2610.28771#A1.E41)\) as a necessary condition for flow equivariance\.

In practice, the convolution kernels are finitek×kk\\times karrays, so all parameters can be collected into a vector in a finite\-dimensional spaceθ∈ℝD\\theta\\in\\mathbb\{R\}^\{D\}, whereD=8​k2D=8k^\{2\}\. Since𝒞2m​e​m\{\\cal\{C\}\}\_\{2\}^\{mem\}is a real\-analytic function of the kernel parameters and it is not identically zero \(as it is not zero wheneverUc​\(x\)≠0U\_\{c\}\(x\)\\neq 0\), it follows that its zero set has Lebesgue measure zero inℝD\\mathbb\{R\}^\{D\}\. Since every flow\-equivariant LSTM must satisfy \([41](https://arxiv.org/html/2610.28771#A1.E41)\), the set of flow\-equivariant kernels is contained in this measure\-zero set\. ∎

### A\.4Transport and flow convolution

In this section we introduce the transport convolution and the transport\-lifting convolution: the𝒢\{\\cal\{G\}\}\-equivariant linear maps between transport fields onM×GM\\times G, and from regular fields onMMto transport fields onM×GM\\times G, respectively\. As special cases we then obtain the flow\-lifting convolution⋆^\\hat\{\\star\}and the flow convolution⋆F\\star\_\{F\}, the two operations used in the FELSTM architecture\. Throughout, we assume thatGGis unimodular, so that its Haar measure is both left and right invariant\. ForG=ℝ2⋊HG=\\mathbb\{R\}^\{2\}\\rtimes H, unimodularity is equivalent to\|deth\|=1\|\\det h\|=1for allh∈Hh\\in H; in that caseGGalso preserves the Lebesgue measure onM=ℝ2M=\\mathbb\{R\}^\{2\}, so all Jacobian factors\|detJk\|\|\\det J\_\{k\}\|appearing below are equal to one\.

##### Transport\-lifting convolution\.

We begin with a general linear map that takes as input a regular field onMMand produces a transport field onM×GM\\times G:

ℐU​\[𝒳t\]​\(x,g\)=∫MU⁡\(x,y,g\)​𝒳t​\(y\)​𝑑y\\displaystyle\{\\cal I\}\_\{U\}\[\{\\cal\{X\}\}\_\{t\}\]\(x,g\)=\\int\_\{M\}U\(x,y;g\)\{\\cal\{X\}\}\_\{t\}\(y\)dy\(43\)whereU:M×M×G→ℝco​u​t×ci​nU:M\\times M\\times G\\to\\mathbb\{R\}^\{c\_\{out\}\\times c\_\{in\}\}is the kernel\. Requiring this map to be equivariant means thatℐU​\[γ⊳𝒳t\]=γ⊳ℐU​\[𝒳t\]\{\\cal I\}\_\{U\}\[\\gamma\\triangleright\{\\cal\{X\}\}\_\{t\}\]=\\gamma\\triangleright\{\\cal I\}\_\{U\}\[\{\\cal\{X\}\}\_\{t\}\]for allγ∈𝒢\\gamma\\in\{\\cal G\}\. Expanding both sides gives

∫MU⁡\(x,y,g\)​𝒳t​\(γt−1⋅y\)​𝑑y=∫MU⁡\(γt−1⋅x,y,γt−1​g​γt−1\)​𝒳t​\(y\)​𝑑y\\displaystyle\\int\_\{M\}U\(x,y;g\)\{\\cal\{X\}\}\_\{t\}\(\\gamma\_\{t\}^\{\-1\}\\cdot y\)dy=\\int\_\{M\}U\(\\gamma^\{\-1\}\_\{t\}\\cdot x,y;\\gamma^\{\-1\}\_\{t\}g\\gamma\_\{t\-1\}\)\{\\cal\{X\}\}\_\{t\}\(y\)dy\(44\)for allγ∈𝒢\\gamma\\in\{\\cal G\}, all𝒳∈ℱ\{\\cal\{X\}\}\\in\{\\cal F\}and allt≥0t\\geq 0\. Using the change of variablesy′=γt−1⋅yy^\{\\prime\}=\\gamma\_\{t\}^\{\-1\}\\cdot y, the left\-hand side becomes∫MU⁡\(x,γt⋅y′,g\)​𝒳t​\(y′\)​\|detJγt\|​d​y′\\int\_\{M\}U\(x,\\gamma\_\{t\}\\cdot y^\{\\prime\};g\)\{\\cal\{X\}\}\_\{t\}\(y^\{\\prime\}\)\|\\det J\_\{\\gamma\_\{t\}\}\|dy^\{\\prime\}, whereJγtJ\_\{\\gamma\_\{t\}\}is the Jacobian∂y∂y′\\frac\{\\partial y\}\{\\partial y^\{\\prime\}\}of the transformationγt∈G\\gamma\_\{t\}\\in G\. Since the above equality must hold for all inputs𝒳∈ℱ\{\\cal\{X\}\}\\in\{\\cal F\}, we obtain the kernel constraint

U⁡\(γt⋅x,γt⋅y,γt​g​γt−1−1\)=1\|detJγt\|​U​\(x,y,g\)\\displaystyle U\(\\gamma\_\{t\}\\cdot x,\\gamma\_\{t\}\\cdot y;\\gamma\_\{t\}g\\gamma\_\{t\-1\}^\{\-1\}\)=\\frac\{1\}\{\|\\det J\_\{\\gamma\_\{t\}\}\|\}U\(x,y;g\)\(45\)which must hold for allγ∈𝒢\\gamma\\in\{\\cal G\}\. Takingt=0t=0, whereγ0=e\\gamma\_\{0\}=eforγ∈𝒢0\\gamma\\in\{\\cal\{G\}\}\_\{0\}, the spatial factors drop out and the constraint reduces toU⁡\(x,y,g​γ−1−1\)=U⁡\(x,y,g\)U\(x,y;\\,g\\,\\gamma\_\{\-1\}^\{\-1\}\)=U\(x,y;g\)for allγ−1∈G\\gamma\_\{\-1\}\\in G\. Since the right action ofGGon itself is transitive, it follows thatUUmust be independent of its group argument, i\.e\.U⁡\(x,y,g\)=U⁡\(x,y\)U\(x,y;g\)=U\(x,y\)\. The remaining constraint is precisely the usual equivariance condition on the spatial variables, implying thatUUis a steerable convolution kernel\. In particular, global transformationsγ∈𝒢g​l​o​b\\gamma\\in\{\\cal\{G\}\}\_\{glob\}impose no further condition, sinceUUno longer carries a group argument on which conjugation could act\. Conversely, any kernel satisfying these conditions defines a𝒢\{\\cal\{G\}\}\-equivariant lifting operator\. Hence the above characterization ofUUis both necessary and sufficient for𝒢\{\\cal\{G\}\}\-equivariance, and the lifting map reduces to the standard steerable convolution

\[U​⋆^​𝒳t\]​\(x,g\)=∫MU⁡\(x−y\)​𝒳t​\(y\)​𝑑y=\[U⋆𝒳t\]​\(x\)\\displaystyle\[U\\,\\hat\{\\star\}\\,\{\\cal\{X\}\}\_\{t\}\]\(x,g\)=\\int\_\{M\}U\(x\-y\)\{\\cal\{X\}\}\_\{t\}\(y\)dy=\[U\\star\{\\cal\{X\}\}\_\{t\}\]\(x\)\(46\)where we copy the result into the extra channels indexed byGG\. Note that this map is equivariant under the full group𝒢\{\\cal\{G\}\}, including the flow, motion and global transformations\.

##### Transport convolution\.

Following the same logic, we define the transport convolution as a𝒢\{\\cal\{G\}\}\-equivariant linear map between transport fields onM×GM\\times G\. A general such map takes the form

ℐW​\[ℋt\]​\(x,g\)=∫M∫GW⁡\(x,y,g,g~\)​ℋt​\(y,g~\)​𝑑g~​𝑑y\\displaystyle\{\\cal I\}\_\{W\}\[\{\\cal\{H\}\}\_\{t\}\]\(x,g\)=\\int\_\{M\}\\int\_\{G\}W\(x,y;g,\{\\tilde\{g\}\}\)\{\\cal\{H\}\}\_\{t\}\(y,\{\\tilde\{g\}\}\)\\,d\{\\tilde\{g\}\}\\,dy\(47\)for a kernelW:M×M×G×G→ℝco​u​t×ci​nW:M\\times M\\times G\\times G\\to\\mathbb\{R\}^\{c\_\{out\}\\times c\_\{in\}\}and the Haar measured​g~d\{\\tilde\{g\}\}onGG\. The equivariance conditionℐW​\[γ⊳ℋt\]=γ⊳ℐW​\[ℋt\]\{\\cal I\}\_\{W\}\[\\gamma\\triangleright\{\\cal\{H\}\}\_\{t\}\]=\\gamma\\triangleright\{\\cal I\}\_\{W\}\[\{\\cal\{H\}\}\_\{t\}\]results in the constraint

W⁡\(γt−1⋅x,γt−1⋅y,γt​g​γt−1−1,γt​g~​γt−1−1\)=1\|detJγt−1\|​W​\(x,y,g,g~\)\\displaystyle W\(\\gamma\_\{t\-1\}\\cdot x,\\gamma\_\{t\-1\}\\cdot y;\\,\\gamma\_\{t\}g\\gamma\_\{t\-1\}^\{\-1\},\\,\\gamma\_\{t\}\{\\tilde\{g\}\}\\gamma\_\{t\-1\}^\{\-1\}\)=\\frac\{1\}\{\|\\det J\_\{\\gamma\_\{t\-1\}\}\|\}W\(x,y;g,\{\\tilde\{g\}\}\)\(48\)for allγ∈𝒢\\gamma\\in\{\\cal G\}, allx,y∈Mx,y\\in Mand allg,g~∈Gg,\{\\tilde\{g\}\}\\in G\. Again, a necessary condition is obtained by takingt=1t=1, which givesW⁡\(x,y,k​g,k​g~\)=W⁡\(x,y,g,g~\)W\(x,y;kg,k\{\\tilde\{g\}\}\)=W\(x,y;g,\{\\tilde\{g\}\}\)for allk,g,g~∈Gk,g,\{\\tilde\{g\}\}\\in G\. Settingk=g~−1k=\{\\tilde\{g\}\}^\{\-1\}shows that the kernel depends on the group arguments only throughg~−1​g\{\\tilde\{g\}\}^\{\-1\}g, i\.e\.W⁡\(x,y,g,g~\)=W⁡\(x,y,g~−1​g\)W\(x,y;g,\{\\tilde\{g\}\}\)=W\(x,y;\\,\{\\tilde\{g\}\}^\{\-1\}g\), so the integration overGGtakes the form of a group convolution\. Fort=2t=2the constraint becomes

W⁡\(k⋅x,k⋅y,k​q​k−1\)=1\|detJk\|​W​\(x,y,q\)\\displaystyle W\(k\\cdot x,k\\cdot y;\\,kqk^\{\-1\}\)=\\frac\{1\}\{\|\\det J\_\{k\}\|\}W\(x,y;q\)\(49\)for allk,q∈Gk,q\\in G, withq=g~−1​gq=\{\\tilde\{g\}\}^\{\-1\}gandk=γ1k=\\gamma\_\{1\}; the constraints att≥3t\\geq 3follow by iterating this argument and impose no additional structure\. Since the samekkacts on the spatial and on the group argument, \([49](https://arxiv.org/html/2610.28771#A1.E49)\) is a single joint condition and does not in general factor into a spatial equivariance condition and a separate conjugation invariance\. Equivalently,WWis an equivariant function onM×M×GM\\times M\\times Gunder the diagonal action\(x,y,q\)↦\(k⋅x,k⋅y,k​q​k−1\)\(x,y,q\)\\mapsto\(k\\cdot x,\\,k\\cdot y,\\,kqk^\{\-1\}\), and is therefore freely specifiable on a set of orbit representatives\. Rather than solving this condition in general, we restrict to the case used in the FELSTM, in which the bank is an abelian subgroupV≤GV\\leq G\. Transport fields then live onM×VM\\times V, and the relevant transformations are those taking values inVV, namely the motions𝒱m​o​t\{\\cal V\}\_\{mot\}and in particular the flows𝒱f​l​o​w\{\\cal V\}\_\{flow\}\. Conjugation by an element ofVVacts trivially onVV, so the velocity arguments are only translated, and \([49](https://arxiv.org/html/2610.28771#A1.E49)\) restricted tok∈Vk\\in Vreduces to invariance ofWWunder a common spatial translation\. This givesW⁡\(x,y,q\)=W⁡\(x−y,q\)W\(x,y;q\)=W\(x\-y;q\)and hence \([16](https://arxiv.org/html/2610.28771#S4.E16)\): a convolution in both arguments, where we write𝗏−𝗎\\mathsf\{v\}\-\\mathsf\{u\}for𝗎−1​𝗏\\mathsf\{u\}^\{\-1\}\\mathsf\{v\}sinceVVis abelian\. In particular,⋆F\\star\_\{F\}is equivariant under all of𝒱m​o​t\{\\cal V\}\_\{mot\}, and not only under𝒱f​l​o​w\{\\cal V\}\_\{flow\}\. Note that if we keep the same domainM×VM\\times Vbut also want the operation to be equivariant under global transformations, that is, under𝒱m​o​t⋊𝒢g​l​o​b\{\\cal V\}\_\{mot\}\\rtimes\{\\cal\{G\}\}\_\{glob\}, then two further conditions are needed\. First,𝒢g​l​o​b\{\\cal\{G\}\}\_\{glob\}must act on𝒱m​o​t\{\\cal V\}\_\{mot\}by conjugation, so the action𝗏↦k​𝗏​k−1\\mathsf\{v\}\\mapsto k\\mathsf\{v\}k^\{\-1\}ofk∈Gk\\in Gon the bank must land inVV; that is,VVmust be normal inGG\. Second, \([49](https://arxiv.org/html/2610.28771#A1.E49)\) must now hold for allk∈Gk\\in G, and not only fork∈Vk\\in V\. ForG=ℝ2⋊HG=\\mathbb\{R\}^\{2\}\\rtimes HwithV=ℝ2V=\\mathbb\{R\}^\{2\}both conditions are satisfied, and the conjugation can be computed explicitly: writingk=\(τ,h\)k=\(\\tau,h\)and𝗏=\(b,I\)\\mathsf\{v\}=\(b,I\),

k​𝗏​k−1=\(τ,h\)​\(b,I\)​\(−h−1​τ,h−1\)=\(τ\+h​b,h\)​\(−h−1​τ,h−1\)=\(h​b,I\),\\displaystyle k\\,\\mathsf\{v\}\\,k^\{\-1\}=\(\\tau,h\)\(b,I\)\(\-h^\{\-1\}\\tau,\\,h^\{\-1\}\)=\(\\tau\+hb,\\;h\)\(\-h^\{\-1\}\\tau,\\,h^\{\-1\}\)=\(hb,\\;I\),\(50\)soVVis indeed normal and conjugation acts on the bank through the point group alone,𝗏↦h​𝗏\\mathsf\{v\}\\mapsto h\\,\\mathsf\{v\}, independently ofτ\\tau\. This is why the analysis above is unaffected: translations act only on the spatial argument\. The remaininghh\-dependence in \([49](https://arxiv.org/html/2610.28771#A1.E49)\) then yields the joint steerability constraint

W⁡\(h⋅z,h​q\)=W⁡\(z,q\),∀h∈H\.\\displaystyle W\(h\\cdot z;\\;h\\,q\)=W\(z;\\,q\),\\qquad\\forall\\,h\\in H\.\(51\)

### A\.5Flow equivariance of FELSTM

In this section, we prove Theorem[1](https://arxiv.org/html/2610.28771#Thmtheorem1)which says that the channel based FELSTM is flow equivariant\.

###### Proof\.

The proof proceeds by induction ontt\. The base caset=0t=0is immediate since, by assumption, the initial state variablesℋ0\{\\cal\{H\}\}\_\{0\}and𝒞0\{\\cal\{C\}\}\_\{0\}are invariant under the action of flows, i\.e\.,ϕ⊳ℋ0=ℋ0\\phi\\triangleright\{\\cal\{H\}\}\_\{0\}=\{\\cal\{H\}\}\_\{0\}andϕ⊳𝒞0=𝒞0\\phi\\triangleright\{\\cal\{C\}\}\_\{0\}=\{\\cal\{C\}\}\_\{0\}for allϕ∈𝒱f​l​o​w\\phi\\in\{\\cal V\}\_\{flow\}whereV≤GV\\leq Gis an abelian subgroup ofGG\. Moreover, the empty sequence𝒳<0\{\\cal\{X\}\}\_\{<0\}is trivially invariant under the action of flows\. Therefore,

ϕ⊳ℋ0​\[𝒳<0\]=ℋ0​\[𝒳<0\]=ℋ0​\[ϕ⊳𝒳<0\]\\displaystyle\\phi\\triangleright\{\\cal\{H\}\}\_\{0\}\[\{\\cal\{X\}\}\_\{<0\}\]=\{\\cal\{H\}\}\_\{0\}\[\{\\cal\{X\}\}\_\{<0\}\]=\{\\cal\{H\}\}\_\{0\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{<0\}\]\(52\)and similarly for𝒞0\{\\cal\{C\}\}\_\{0\}, establishing the base case of the induction\. For the inductive step, assume that the state variablesℋt\{\\cal\{H\}\}\_\{t\}and𝒞t\{\\cal\{C\}\}\_\{t\}are flow equivariant at timett, namely,ϕ⊳ℋt​\[𝒳<t\]=ℋt​\[ϕ⊳𝒳<t\]\\phi\\triangleright\{\\cal\{H\}\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]=\{\\cal\{H\}\}\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{<t\}\]andϕ⊳𝒞t​\[𝒳<t\]=𝒞t​\[ϕ⊳𝒳<t\]\\phi\\triangleright\{\\cal\{C\}\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]=\{\\cal\{C\}\}\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{<t\}\]for allϕ∈𝒱f​l​o​w\\phi\\in\{\\cal V\}\_\{flow\}\. We now show that the state variables at timet\+1t\+1are also flow equivariant\.

We first show that the gate variables are flow equivariant:

zt​\[ϕ⊳𝒳≤t\]​\(x,𝗏\)\\displaystyle z\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,\\mathsf\{v\}\)=az​\[𝗏⋅\[Wz⋆Fℋt​\[ϕ⊳𝒳<t\]\]​\(x,𝗏\)\+\[Uz​⋆^​\(ϕ⊳𝒳\)t\]​\(x,𝗏\)\]\\displaystyle=a\_\{z\}\\Big\[\\mathsf\{v\}\\cdot\\left\[W\_\{z\}\\star\_\{F\}\{\\cal H\}\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{<t\}\]\\right\]\(x,\\mathsf\{v\}\)\+\[U\_\{z\}\\,\\hat\{\\star\}\\,\{\(\\phi\\triangleright\{\\cal X\}\)\}\_\{t\}\]\(x,\\mathsf\{v\}\)\\Big\]\(53\)=az​\[𝗏​ϕt−1⋅\[Wz⋆Fℋt​\[𝒳<t\]\]​\(x,ϕt−1​𝗏​ϕt−1\)\+ϕt⋅\[Uz​⋆^​𝒳t\]​\(x,ϕt−1​𝗏​ϕt−1\)\]\\displaystyle=a\_\{z\}\\Big\[\\mathsf\{v\}\\phi\_\{t\-1\}\\cdot\\left\[W\_\{z\}\\star\_\{F\}\{\\cal H\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\\right\]\\\!\\big\(x,\\phi\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\phi\_\{t\-1\}\\big\)\+\\phi\_\{t\}\\cdot\[U\_\{z\}\\,\\hat\{\\star\}\\,\{\\cal X\}\_\{t\}\]\\\!\\big\(x,\\phi\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\phi\_\{t\-1\}\\big\)\\Big\]\(54\)=ϕt⋅az​\[ϕt−1​𝗏​ϕt−1⋅\[Wz⋆Fℋt​\[𝒳<t\]\]​\(x,ϕt−1​𝗏​ϕt−1\)\+\[Uz​⋆^​𝒳t\]​\(x,ϕt−1​𝗏​ϕt−1\)\]\\displaystyle=\\phi\_\{t\}\\cdot a\_\{z\}\\Big\[\\phi\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\phi\_\{t\-1\}\\cdot\\left\[W\_\{z\}\\star\_\{F\}\{\\cal H\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\\right\]\\\!\\big\(x,\\phi\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\phi\_\{t\-1\}\\big\)\+\[U\_\{z\}\\,\\hat\{\\star\}\\,\{\\cal X\}\_\{t\}\]\\\!\\big\(x,\\phi\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\phi\_\{t\-1\}\\big\)\\Big\]\(55\)=ϕt⋅zt​\[𝒳≤t\]​\(x,ϕt−1​𝗏​ϕt−1\)\\displaystyle=\\phi\_\{t\}\\cdot z\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\\!\\big\(x,\\phi\_\{t\}^\{\-1\}\\mathsf\{v\}\\phi\_\{t\-1\}\\big\)\(56\)=ϕ⊳zt​\[𝒳≤t\]​\(x,𝗏\)\\displaystyle=\\phi\\triangleright z\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,\\mathsf\{v\}\)\(57\)for allz∈\{i,f,o,c\}z\\in\\\{i,f,o,c\\\}\. Here, in the first line, we used the definition of the gate variables\. In the second line, we applied the inductive hypothesis for the hidden state variable together with the flow equivariance of the flow convolution and flow\-lifting convolution operators\. In the third line, we inserted the identity element in the formϕt​ϕt−1=e\\phi\_\{t\}\\phi\_\{t\}^\{\-1\}=einto the first term and factored outϕt\\phi\_\{t\}, using the fact that the nonlinearityaza\_\{z\}acts pointwise and is equivariant under the action of group elements\. In the fourth line, we used the definition of the gate variables once more, and in the final line we used the definition of the action of flows on transport fields\.

Next we show that the cell state𝒞t\+1\{\\cal\{C\}\}\_\{t\+1\}equation is flow equivariant as well\. Starting with the left hand side, we have

𝒞t\+1​\[ϕ⊳𝒳≤t\]​\(x,g\)\\displaystyle\{\\cal\{C\}\}\_\{t\+1\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)=g⋅𝒞t​\[ϕ⊳𝒳<t\]​\(x,g\)⊙ft​\[ϕ⊳𝒳≤t\]​\(x,g\)\+it​\[ϕ⊳𝒳≤t\]​\(x,g\)⊙ct​\[ϕ⊳𝒳≤t\]​\(x,g\)\\displaystyle=g\\cdot\{\\cal C\}\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{<t\}\]\(x,g\)\\odot f\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)\+i\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)\\odot c\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)\(58\)=g​ϕt−1⋅𝒞t​\[𝒳<t\]​\(x,ϕt−1​g​ϕt−1\)⊙ϕt⋅ft​\[𝒳≤t\]​\(x,ϕt−1​g​ϕt−1\)\\displaystyle=g\\phi\_\{t\-1\}\\cdot\{\\cal C\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\\odot\\phi\_\{t\}\\cdot f\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\+ϕt⋅it\[𝒳≤t\]\(x,ϕt−1gϕt−1\)⊙ϕt⋅ct\[𝒳≤t\]\(x,ϕt−1gϕt−1\)\\displaystyle\\hskip 85\.35826pt\+\\phi\_\{t\}\\cdot i\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\\odot\\phi\_\{t\}\\cdot c\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\(59\)=ϕt⋅\[ϕt−1gϕt−1⋅𝒞t\[𝒳<t\]\(x,ϕt−1gϕt−1\)⊙ft\[𝒳≤t\]\(x,ϕt−1gϕt−1\)\\displaystyle=\\phi\_\{t\}\\cdot\\Big\[\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\cdot\{\\cal C\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\\odot f\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\phi\_\{t\-1\}\\big\)\+it\[𝒳≤t\]\(x,ϕt−1gϕt−1\)⊙ct\[𝒳≤t\]\(x,ϕt−1gϕt−1\)\]\\displaystyle\\hskip 85\.35826pt\+i\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\\odot c\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\\Big\]\(60\)=ϕt⋅𝒞t\+1​\[𝒳≤t\]​\(x,ϕt−1​g​ϕt−1\)\\displaystyle=\\phi\_\{t\}\\cdot\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\\!\\big\(x,\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}\\big\)\(61\)=ϕt⋅𝒞t\+1​\[𝒳≤t\]​\(x,ϕt\+1−1​g​ϕt\)\\displaystyle=\\phi\_\{t\}\\cdot\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\\!\\big\(x,\\phi\_\{t\+1\}^\{\-1\}g\\,\\phi\_\{t\}\\big\)\(62\)=ϕ⊳𝒞t\+1​\[𝒳≤t\]​\(x,g\)\\displaystyle=\\phi\\triangleright\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)\(63\)where in the second line we used the inductive hypothesis for the cell state at timettvariable together with the flow equivariance of the gate variables which we just proved\. In the third line, we inserted the identity element in the formϕt​ϕt−1=e\\phi\_\{t\}\\phi\_\{t\}^\{\-1\}=einto the first term and factored outϕt\\phi\_\{t\}, using the fact that regular action distributes over the Hadamard product\. In the fourth line, we used the definition of the cell state variable at timet\+1t\+1\. In the fifth line we used thatϕ\\phiis a flow in the abelian groupVV, soϕt−1​g​ϕt−1=g​ϕ1−1=ϕt\+1−1​g​ϕt\\phi\_\{t\}^\{\-1\}g\\,\\phi\_\{t\-1\}=g\\,\\phi\_\{1\}^\{\-1\}=\\phi\_\{t\+1\}^\{\-1\}g\\,\\phi\_\{t\}; this is the only step that uses the flow property\. In the final line we used the definition of the action of flows on transport fields\.

Finally we show that the hidden stateℋ\{\\cal\{H\}\}is flow equivariant\. Starting with the left hand side, we have

ℋt\+1​\[ϕ⊳𝒳≤t\]​\(x,g\)\\displaystyle\{\\cal\{H\}\}\_\{t\+1\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)=ot​\[ϕ⊳𝒳≤t\]​\(x,g\)⊙tanh⁡\(𝒞t\+1​\[ϕ⊳𝒳≤t\]​\(x,g\)\)\\displaystyle=o\_\{t\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)\\odot\\tanh\\Big\(\{\\cal\{C\}\}\_\{t\+1\}\[\\phi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)\\Big\)\(64\)=ϕt⋅ot​\[𝒳≤t\]​\(x,ϕt−1​g​ϕt−1\)⊙tanh⁡\(ϕt⋅𝒞t\+1​\[𝒳≤t\]​\(x,ϕt−1​g​ϕt−1\)\)\\displaystyle=\\phi\_\{t\}\\cdot o\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\\!\\left\(x,\\phi\_\{t\}^\{\-1\}g\\phi\_\{t\-1\}\\right\)\\odot\\tanh\\Big\(\\phi\_\{t\}\\cdot\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\big\(x,\\phi\_\{t\}^\{\-1\}g\\phi\_\{t\-1\}\\big\)\\Big\)\(65\)=ϕt⋅ℋt\+1​\[𝒳≤t\]​\(x,ϕt−1​g​ϕt−1\)\\displaystyle=\\phi\_\{t\}\\cdot\{\\cal\{H\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\\!\\left\(x,\\phi\_\{t\}^\{\-1\}g\\phi\_\{t\-1\}\\right\)\(66\)=ϕt⋅ℋt\+1​\[𝒳≤t\]​\(x,ϕt\+1−1​g​ϕt\)\\displaystyle=\\phi\_\{t\}\\cdot\{\\cal\{H\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\\!\\left\(x,\\phi\_\{t\+1\}^\{\-1\}g\\phi\_\{t\}\\right\)\(67\)=ϕ⊳ℋt\+1​\[𝒳≤t\]​\(x,g\)\\displaystyle=\\phi\\triangleright\{\\cal\{H\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x,g\)\(68\)where in the second line we used the flow equivariance of the output gate variable which we proved earlier together with the flow equivariance of the cell state variable which we also proved\. In the third line, we used the fact thattanh\\tanhis a pointwise nonlinearity and is equivariant under the action of group elements\. We also factored outϕt\\phi\_\{t\}using the fact that the regular action distributes over the Hadamard product, and used the definition of the hidden state variable at timet\+1t\+1\. In the fourth line we used the flow property, exactly as for the cell state, and in the final line we used the definition of the action of flows on transport fields\. This completes the inductive step and hence the proof of flow equivariance for the channel based FELSTM model\. ∎

##### Global and motion equivariance of the FELSTM\.

Since the flowϕ\\phitakes values in the abelian subgroupVV, the transport conjugation could equally well have been written in the reduced formϕt−1​𝗏​ϕt−1=𝗏​ϕ1−1\\phi\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\phi\_\{t\-1\}=\\mathsf\{v\}\\,\\phi\_\{1\}^\{\-1\}\. We deliberately kept the unreduced form, because it isolates the single place where the proof uses thatϕ\\phiis a flow, and therefore shows exactly why the model fails to be motion equivariant\.

The induction produces a field whose velocity argument isψt−1​𝗏​ψt−1\\psi\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\psi\_\{t\-1\}, whereas the transport\-field law at the next time step requiresψt\+1−1​𝗏​ψt\\psi\_\{t\+1\}^\{\-1\}\\mathsf\{v\}\\,\\psi\_\{t\}\. Closing the induction therefore requires

ψt−1​𝗏​ψt−1=ψt\+1−1​𝗏​ψt∀𝗏∈V\.\\displaystyle\\psi\_\{t\}^\{\-1\}\\,\\mathsf\{v\}\\,\\psi\_\{t\-1\}=\\psi\_\{t\+1\}^\{\-1\}\\,\\mathsf\{v\}\\,\\psi\_\{t\}\\qquad\\forall\\,\\mathsf\{v\}\\in V\.\(69\)Multiplying on the left byψt\+1\\psi\_\{t\+1\}, on the right byψt−1−1\\psi\_\{t\-1\}^\{\-1\}, and usingψt=evt​ψt−1\\psi\_\{t\}=e^\{v\_\{t\}\}\\psi\_\{t\-1\}, this is equivalent to

evt\+1​𝗏=𝗏​evt∀𝗏∈V\.\\displaystyle e^\{v\_\{t\+1\}\}\\,\\mathsf\{v\}=\\mathsf\{v\}\\,e^\{v\_\{t\}\}\\qquad\\forall\\,\\mathsf\{v\}\\in V\.\(70\)Taking𝗏=e\\mathsf\{v\}=egivesevt\+1=evte^\{v\_\{t\+1\}\}=e^\{v\_\{t\}\}, so the velocity must be constant in time; the remaining conditionevt​𝗏=𝗏​evte^\{v\_\{t\}\}\\mathsf\{v\}=\\mathsf\{v\}\\,e^\{v\_\{t\}\}is then automatic, sinceevt∈Ve^\{v\_\{t\}\}\\in Vby assumption\. Hence \([70](https://arxiv.org/html/2610.28771#A1.E70)\) holds precisely when the velocity is constant, that is, whenψ\\psiis a flow\. For global transformations the condition holds trivially\. Ifγ∈𝒢g​l​o​b\\gamma\\in\{\\cal\{G\}\}\_\{glob\}thenγt=γ\\gamma\_\{t\}=\\gammafor alltt, so the transport conjugationγt−1​𝗏​γt−1=γ−1​𝗏​γ\\gamma\_\{t\}^\{\-1\}\\mathsf\{v\}\\,\\gamma\_\{t\-1\}=\\gamma^\{\-1\}\\mathsf\{v\}\\,\\gammadoes not depend onttand the induction closes\. The FELSTM is therefore also𝒢g​l​o​b\{\\cal\{G\}\}\_\{glob\}\-equivariant\.

### A\.6Equivariance of the Velocity model

Suppose first thatut​\[ψ⊳𝒳≤t\]=ψt​ut​\[𝒳≤t\]​ψt−1\+ψ˙t​ψt−1u\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\_\{t\}u\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\}^\{\-1\}\+\\dot\{\\psi\}\_\{t\}\\psi\_\{t\}^\{\-1\}, and defineT~t,t0=ψt​Tt,t0​\[𝒳≤t\]​ψt0−1\\widetilde\{T\}\_\{t,t\_\{0\}\}=\\psi\_\{t\}T\_\{t,t\_\{0\}\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\_\{0\}\}^\{\-1\}\. Then

dd​t​T~t,t0\\displaystyle\\frac\{d\}\{dt\}\\widetilde\{T\}\_\{t,t\_\{0\}\}=ψ˙t​Tt,t0​ψt0−1\+ψt​dd​t​Tt,t0​ψt0−1\\displaystyle=\\dot\{\\psi\}\_\{t\}T\_\{t,t\_\{0\}\}\\psi\_\{t\_\{0\}\}^\{\-1\}\+\\psi\_\{t\}\\frac\{d\}\{dt\}T\_\{t,t\_\{0\}\}\\psi\_\{t\_\{0\}\}^\{\-1\}=\(ψ˙t​ψt−1\+ψt​ut​\[𝒳≤t\]​ψt−1\)​T~t,t0=ut​\[ψ⊳𝒳≤t\]​T~t,t0\.\\displaystyle=\\left\(\\dot\{\\psi\}\_\{t\}\\psi\_\{t\}^\{\-1\}\+\\psi\_\{t\}u\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\}^\{\-1\}\\right\)\\widetilde\{T\}\_\{t,t\_\{0\}\}=u\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\\,\\widetilde\{T\}\_\{t,t\_\{0\}\}\.SinceT~t0,t0=ψt0​e​ψt0−1=e\\widetilde\{T\}\_\{t\_\{0\},t\_\{0\}\}=\\psi\_\{t\_\{0\}\}e\\psi\_\{t\_\{0\}\}^\{\-1\}=e, uniqueness of solutions to the ODE impliesTt,t0​\[ψ⊳𝒳≤t\]=ψt​Tt,t0​\[𝒳≤t\]​ψt0−1T\_\{t,t\_\{0\}\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\_\{t\}T\_\{t,t\_\{0\}\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\_\{0\}\}^\{\-1\}\. Conversely, supposeTt,t0​\[ψ⊳𝒳≤t\]=ψt​Tt,t0​\[𝒳≤t\]​ψt0−1T\_\{t,t\_\{0\}\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\_\{t\}T\_\{t,t\_\{0\}\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\_\{0\}\}^\{\-1\}\. Differentiating both sides and usingdd​t​Tt,t0=ut​Tt,t0\\frac\{d\}\{dt\}T\_\{t,t\_\{0\}\}=u\_\{t\}T\_\{t,t\_\{0\}\}gives

ut​\[ψ⊳𝒳≤t\]​Tt,t0​\[ψ⊳𝒳≤t\]=\(ψ˙t​ψt−1\+ψt​ut​\[𝒳≤t\]​ψt−1\)​Tt,t0​\[ψ⊳𝒳≤t\],\\displaystyle u\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\\,T\_\{t,t\_\{0\}\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\left\(\\dot\{\\psi\}\_\{t\}\\psi\_\{t\}^\{\-1\}\+\\psi\_\{t\}u\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\}^\{\-1\}\\right\)T\_\{t,t\_\{0\}\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\],and multiplying on the right byTt,t0−1​\[ψ⊳𝒳≤t\]T\_\{t,t\_\{0\}\}^\{\-1\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]yieldsut​\[ψ⊳𝒳≤t\]=ψt​ut​\[𝒳≤t\]​ψt−1\+ψ˙t​ψt−1u\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]=\\psi\_\{t\}u\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\}^\{\-1\}\+\\dot\{\\psi\}\_\{t\}\\psi\_\{t\}^\{\-1\}\.

### A\.7Motion equivariance of the Velocity Based MELSTM

Here, in this section we prove Theorem[2](https://arxiv.org/html/2610.28771#Thmtheorem2)which states that the transporter based MELSTM is motion equivariant\.

###### Proof\.

The proof proceeds by induction ontt\. Similar to the flow\-equivariance proof of FELSTM, the base caset=0t=0is trivial since, by assumption, the initial state variablesℋ0\{\\cal\{H\}\}\_\{0\},𝒞0\{\\cal\{C\}\}\_\{0\}and the empty sequence𝒳<0\{\\cal\{X\}\}\_\{<0\}are invariant under the action of motion transformations which establishes the equivariance at timet=0t=0\. Then, we assume that the state variablesℋt\{\\cal\{H\}\}\_\{t\}and𝒞t\{\\cal\{C\}\}\_\{t\}are motion equivariant at timettand we show that they are also motion equivariant at timet\+1t\+1\. For all the gate variablesz∈\{i,f,o,c\}z\\in\\\{i,f,o,c\\\}we have

zt​\[ψ⊳𝒳≤t\]​\(x\)\\displaystyle z\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)=az​\[Tt​\[ψ⊳𝒳≤t\]⋅\[Wz⋆ℋt​\[ψ⊳𝒳<t\]\]​\(x\)\+\[Uz⋆\(ψ⊳𝒳t\)\]​\(x\)\]\\displaystyle=a\_\{z\}\\Big\[T\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\\cdot\\left\[W\_\{z\}\\star\{\\cal H\}\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{<t\}\]\\right\]\\\!\(x\)\+\[U\_\{z\}\\star\(\\psi\\triangleright\{\\cal X\}\_\{t\}\)\]\(x\)\\Big\]\(71\)=az​\[ψt​Tt​\[𝒳≤t\]​ψt−1−1​ψt−1⋅\[Wz⋆ℋt​\[𝒳<t\]\]​\(x\)\+ψt⋅\[Uz⋆𝒳t\]​\(x\)\]\\displaystyle=a\_\{z\}\\Big\[\\psi\_\{t\}T\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\{\\psi\_\{t\-1\}\}^\{\-1\}\{\\psi\_\{t\-1\}\}\\cdot\\left\[W\_\{z\}\\star\{\\cal H\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\\right\]\\\!\(x\)\+\\psi\_\{t\}\\cdot\[U\_\{z\}\\star\{\\cal X\}\_\{t\}\]\(x\)\\Big\]\(72\)=ψt⋅az​\[Tt​\[𝒳≤t\]⋅\[Wz⋆ℋt​\[𝒳<t\]\]​\(x\)\+\[Uz⋆𝒳t\]​\(x\)\]\\displaystyle=\\psi\_\{t\}\\cdot a\_\{z\}\\Big\[T\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\cdot\[W\_\{z\}\\star\{\\cal H\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\]\(x\)\+\[U\_\{z\}\\star\{\\cal X\}\_\{t\}\]\(x\)\\Big\]\(73\)=ψt⋅zt​\[𝒳≤t\]​\(x\)\\displaystyle=\\psi\_\{t\}\\cdot z\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(74\)=ψ⊳zt​\[𝒳≤t\]​\(x\)\\displaystyle=\\psi\\triangleright z\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(75\)where in the second line we used the induction hypothesis for the hidden state variable together with the equivariance of the convolution under regular action of the group and also the equivariance of the transporterTtT\_\{t\}under the motion transformation\. In the third line we used associativity of the group action and the fact that the nonlinearityaza\_\{z\}is pointwise and is equivariant under the action of group elements\. In the fourth line we used the definition of the gate variables\. For the cell state we have

𝒞t\+1​\[ψ⊳𝒳≤t\]​\(x\)\\displaystyle\{\\cal\{C\}\}\_\{t\+1\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)=Tt​\[ψ⊳𝒳≤t\]⋅𝒞t​\[ψ⊳𝒳<t\]​\(x\)⊙ft​\[ψ⊳𝒳≤t\]​\(x\)\+it​\[ψ⊳𝒳≤t\]​\(x\)⊙ct​\[ψ⊳𝒳≤t\]​\(x\)\\displaystyle=T\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\\cdot\{\\cal C\}\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{<t\}\]\(x\)\\odot f\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\+i\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot c\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(76\)=ψt​Tt​\[𝒳≤t\]​ψt−1−1​ψt−1⋅𝒞t​\[𝒳<t\]​\(x\)⊙ψt⋅ft​\[𝒳≤t\]​\(x\)\\displaystyle=\\psi\_\{t\}T\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\psi\_\{t\-1\}^\{\-1\}\\psi\_\{t\-1\}\\cdot\{\\cal C\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\(x\)\\odot\\psi\_\{t\}\\cdot f\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\+ψt⋅it\[𝒳≤t\]\(x\)⊙ψt⋅ct\[𝒳≤t\]\(x\)\\displaystyle\\hskip 170\.71652pt\+\\psi\_\{t\}\\cdot i\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot\\psi\_\{t\}\\cdot c\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(77\)=ψt⋅\[Tt​\[𝒳≤t\]⋅𝒞t​\[𝒳<t\]⊙ft​\[𝒳≤t\]\+it​\[𝒳≤t\]⊙ct​\[𝒳≤t\]\]​\(x\)\\displaystyle=\\psi\_\{t\}\\cdot\\Big\[T\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\cdot\{\\cal C\}\_\{t\}\[\{\\cal\{X\}\}\_\{<t\}\]\\odot f\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\+i\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\odot c\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\\Big\]\(x\)\(78\)=ψt⋅𝒞t\+1​\[𝒳≤t\]​\(x\)\\displaystyle=\\psi\_\{t\}\\cdot\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(79\)=ψ⊳𝒞t\+1​\[𝒳≤t\]​\(x\)\\displaystyle=\\psi\\triangleright\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(80\)where in the second line we used the induction hypothesis for the cell state variable at timett, the equivariance of the transporterTtT\_\{t\}together with the equivariance of the gate variables which we just proved\. In the third line, we factored outψt\\psi\_\{t\}using the fact that the regular action distributes over the Hadamard product\. Finally for the hidden state we have

ℋt\+1​\[ψ⊳𝒳≤t\]​\(x\)\\displaystyle\{\\cal\{H\}\}\_\{t\+1\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)=ot​\[ψ⊳𝒳≤t\]​\(x\)⊙tanh⁡\(𝒞t\+1​\[ψ⊳𝒳≤t\]​\(x\)\)\\displaystyle=o\_\{t\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot\\tanh\(\{\\cal\{C\}\}\_\{t\+1\}\[\\psi\\triangleright\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\)\(81\)=ψt⋅ot​\[𝒳≤t\]​\(x\)⊙tanh⁡\(ψt⋅𝒞t\+1​\[𝒳≤t\]​\(x\)\)\\displaystyle=\\psi\_\{t\}\\cdot o\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot\\tanh\(\\psi\_\{t\}\\cdot\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\)\(82\)=ψt⋅\[ot​\[𝒳≤t\]​\(x\)⊙tanh⁡\(𝒞t\+1​\[𝒳≤t\]​\(x\)\)\]\\displaystyle=\\psi\_\{t\}\\cdot\\Big\[o\_\{t\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\\odot\\tanh\(\{\\cal\{C\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\)\\Big\]\(83\)=ψt⋅ℋt\+1​\[𝒳≤t\]​\(x\)\\displaystyle=\\psi\_\{t\}\\cdot\{\\cal\{H\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(84\)=ψ⊳ℋt\+1​\[𝒳≤t\]​\(x\)\\displaystyle=\\psi\\triangleright\{\\cal\{H\}\}\_\{t\+1\}\[\{\\cal\{X\}\}\_\{\\leq t\}\]\(x\)\(85\)where in the second line we used the motion equivariance of the output gate variable and cell state variable which we also proved\. In the third line we used the fact thattanh\\tanhis a pointwise nonlinearity and is equivariant under the action of group elements\. We also factored outψt\\psi\_\{t\}using the fact that the regular action distributes over the Hadamard product\. We then just replace the definition of the hidden state variable at timet\+1t\+1\. This completes the inductive step and hence the proof of motion equivariance for the MELSTM model\. ∎

### A\.8Time\-dependent data and motion equivariance as a gauge theory

In this section, we provide a geometric interpretation of time\-dependent data and motion equivariance in terms of principal bundles and gauge connections\. This perspective naturally identifies time\-dependent data as sections of an associated bundle, motions as gauge transformations, and the velocity model as a gauge connection that defines parallel transport along time through the transporter\. Motion equivariance of the model can then be understood as equivariance under gauge transformations\.

##### Temporal data as sections of an associated bundle\.

As our base manifold is justℝ\\mathbb\{R\}, our bundles are trivial and we can work with global trivializations\. We define the trivial principalGG\-bundle

P=ℝ×G,πP​\(t,g\)=t,\\displaystyle P=\\mathbb\{R\}\\times G,\\qquad\\pi\_\{P\}\(t,g\)=t,\(86\)with right action\(t,g\)⋅h=\(t,g​h\)\(t,g\)\\cdot h=\(t,gh\)forh∈Gh\\in G\. From this, we construct the associated bundle with fiberFFwhich is the feature space of our model:

ℰ=P×GF:=\(P×F\)/∼,\\displaystyle\\mathcal\{E\}=P\\times\_\{G\}F:=\(P\\times F\)/\\sim,\(87\)where the equivalence relation is\(p⋅g,f\)∼\(p,g⋅f\)\(p\\cdot g,f\)\\sim\(p,g\\cdot f\)forg∈Gg\\in G,p∈Pp\\in Pandf∈Ff\\in F\. Here our associated bundle is also trivial and can be identified withℝ×F\\mathbb\{R\}\\times F\. A time\-dependent signal𝒳:ℝ→F\\mathcal\{X\}:\\mathbb\{R\}\\to Fis then naturally identified with a section of this associated bundles𝒳​\(t\)=\[\(t,e\),𝒳t\]∈ℰs\_\{\\mathcal\{X\}\}\(t\)=\[\(t,e\),\\mathcal\{X\}\_\{t\}\]\\in\\mathcal\{E\}whereeeis the identity inGG\. This means that the space of time\-dependent signalsℱ\\cal Fcan be identified with the space of sections of the associated bundleΓ⁡\(ℰ\)\\Gamma\(\\mathcal\{E\}\)\.

##### Gauge transformations\.

A gauge transformation of the principal bundlePPis a bundle automorphism commuting with the rightGG\-action and preserving the base spaceℝ\\mathbb\{R\}\. In the trivializationP=ℝ×GP=\\mathbb\{R\}\\times G, every gauge transformation is uniquely determined by a smooth mapγ:ℝ→G\\gamma:\\mathbb\{R\}\\to G, acting fiberwise as\(t,h\)↦\(t,γt​h\)\(t,h\)\\mapsto\(t,\\gamma\_\{t\}h\)\. Therefore, the group of gauge transformations is naturally identified with the group of smoothGG\-valued functions on time which in our case is𝒢m​o​t⋊𝒢g​l​o​b\{\\cal G\}\_\{mot\}\\rtimes\{\\cal G\}\_\{glob\}where𝒢m​o​t\{\\cal G\}\_\{mot\}consists of smooth motions initialized at the identity and𝒢g​l​o​b≅G\{\\cal G\}\_\{glob\}\\cong Gis the subgroup of time\-independent transformations\.

A gauge transformation induces an action on the associated bundleℰ\\mathcal\{E\}and thus on sections ofℰ\\mathcal\{E\}, which, in our trivial bundle case, is simply given by

\(γ⊳𝒳\)t=γt⋅𝒳t∀γ∈𝒢m​o​t⋊𝒢g​l​o​b\\displaystyle\{\(\\gamma\\triangleright\\mathcal\{X\}\)\}\_\{t\}=\\gamma\_\{t\}\\cdot\\mathcal\{X\}\_\{t\}\\qquad\\forall\\gamma\\in\{\\cal G\}\_\{mot\}\\rtimes\{\\cal G\}\_\{glob\}\(88\)which is exactly the action of motion transformations on time\-dependent signals that we defined in the main text\. Therefore, motions are naturally identified with gauge transformations in this geometric framework\.

##### Gauge connection and parallel transport\.

A connection on the principal bundlePPis a𝔤\\mathfrak\{g\}\-valued one\-form onPPsatisfying the standard equivariance and normalization properties, and it defines parallel transport along the time direction\. In the global trivializationP≅ℝ×GP\\cong\\mathbb\{R\}\\times G, it is represented by a𝔤\\mathfrak\{g\}\-valued one\-form onℝ\\mathbb\{R\}of the formut​d​tu\_\{t\}\\,dt, whereut∈𝔤u\_\{t\}\\in\\mathfrak\{g\}\. Under a gauge transformationγ:ℝ→G\\gamma:\\mathbb\{R\}\\to G, the connection transforms as

γ⊳ut=Adγt⁡\(ut\)\+γ˙t​γt−1\\displaystyle\\gamma\\triangleright u\_\{t\}=\\operatorname\{Ad\}\_\{\\gamma\_\{t\}\}\(u\_\{t\}\)\+\\dot\{\\gamma\}\_\{t\}\\gamma\_\{t\}^\{\-1\}\(89\)whereAdγt⁡\(ut\)=γt​ut​γt−1\\operatorname\{Ad\}\_\{\\gamma\_\{t\}\}\(u\_\{t\}\)=\\gamma\_\{t\}u\_\{t\}\\gamma\_\{t\}^\{\-1\}denotes the adjoint action ofGGon its Lie algebra𝔤\\mathfrak\{g\}\. Given a connectionutu\_\{t\}, one can define a covariant derivative and the associated parallel transport along time\. The covariant derivative measures the intrinsic time variation of a section, i\.e\. the variation after subtracting the contribution induced by gauge transformations which in our case are motions\.

The parallel transport from timet1t\_\{1\}tot2t\_\{2\}induced by the connectionutu\_\{t\}is given by the associated Wilson line, i\.e\., the time\-ordered exponential of the connection:

Tt2,t1=𝒯​exp⁡\(∫t1t2us​𝑑s\)\.\\displaystyle T\_\{t\_\{2\},t\_\{1\}\}=\\mathcal\{T\}\\exp\\left\(\\int\_\{t\_\{1\}\}^\{t\_\{2\}\}u\_\{s\}\\,ds\\right\)\.\(90\)Equivalently, it is the unique solution of the differential equationdd​t​Tt,t0=ut​Tt,t0\\frac\{d\}\{dt\}T\_\{t,t\_\{0\}\}=u\_\{t\}T\_\{t,t\_\{0\}\}with initial conditionTt0,t0=eT\_\{t\_\{0\},t\_\{0\}\}=e\. Its action on a time\-dependent signal \(viewed as a section\) transports features from timet1t\_\{1\}to timet2t\_\{2\}by parallel transport along the connectionutu\_\{t\}, meaning that the transported section remains covariantly constant with respect to the induced covariant derivative\.

##### Parallel transport and model dynamics\.

Given a gauge connection in the continuous\-time limit, we can define a covariant time derivative and express the model dynamics in terms of this covariant derivative\. This formulation yields an evolution equation that is equivariant under gauge transformations, and hence under motions as well\. In the discrete\-time setting, we instead use the parallel transporter induced by the connection to modify the update equations\. Concretely, the feature produced at timettis transported to timet\+1t\+1in a parallel manner and then combined with the newly computed features at timet\+1t\+1to form the updated hidden state\. This construction ensures that the resulting discrete\-time dynamics are motion equivariant by design\. The mechanism echoes the ‘gauge principle,’ whereby equivariance is achieved through the introduction of an additional gauge degree of freedom\.

## Appendix BExperimental Details

### B\.1Future\-Frame Prediction

#### B\.1\.1Moving MNIST Dataset

We construct a synthetic dataset ofndn\_\{d\}MNIST digits moving with time\-dependent velocities on the discrete torus𝕋S2\\mathbb\{T\}^\{2\}\_\{S\}, obtained by zero\-padding the28×2828\\times 28MNIST digits toS×SS\\times SwithS≥28S\\geq 28\. Each digiti=1,…,ndi=1,\\ldots,n\_\{d\}carries a velocityvt\(i\)∈VNv^\{\(i\)\}\_\{t\}\\in V\_\{N\}drawn from the velocity latticeVN=\{v∈ℤ2∣‖v‖∞≤N\}V\_\{N\}=\\\{\\,v\\in\\mathbb\{Z\}^\{2\}\\mid\\\|v\\\|\_\{\\infty\}\\leq N\\,\\\}\. A frame is the superposition of the individual digits,

𝒳t=∑i=1nd𝒳t\(i\),𝒳t\(i\)=ψt\(i\)⋅𝒳0\(i\),\\displaystyle\{\\cal\{X\}\}\_\{t\}=\\sum\_\{i=1\}^\{n\_\{d\}\}\{\\cal\{X\}\}^\{\(i\)\}\_\{t\},\\qquad\{\\cal\{X\}\}^\{\(i\)\}\_\{t\}=\\psi^\{\(i\)\}\_\{t\}\\cdot\{\\cal\{X\}\}^\{\(i\)\}\_\{0\},\(91\)whereψ\(i\)\\psi^\{\(i\)\}is the discrete motion generated by the velocity sequence\{vt\(i\)\}\\\{v^\{\(i\)\}\_\{t\}\\\}as defined in Section[2](https://arxiv.org/html/2610.28771#S2)\.

##### Motions\.

We construct a stochastic generative process that produces a rich distribution of motions\. To govern the temporal dynamics, we introduce two key parameters:pchangep\_\{\\text\{change\}\}to control the frequency of velocity updates, andpsmoothp\_\{\\text\{smooth\}\}to control the spatial smoothness of the motion\. At each time steptt, a velocity update is triggered independently with probabilitypchangep\_\{\\text\{change\}\}\. When an update occurs, the new velocity transitions to an immediate neighboring state on the lattice with probabilitypsmoothp\_\{\\text\{smooth\}\}\. With probability1−psmooth1\-p\_\{\\text\{smooth\}\}, it is instead sampled uniformly from the full latticeVNV\_\{N\}\(excluding the current velocity\), allowing for a long\-tail of sudden, larger accelerations\. Formally, the velocity evolves as:

vt=\{update⁡\(vt−1\)w\.p\.​pchange,vt−1otherwise,update⁡\(v\)=\{nbr⁡\(v\)w\.p\.​psmooth,𝒰⁡\(VN∖\{v\}\)w\.p\.​1−psmooth\.v\_\{t\}=\\begin\{cases\}\\mathrm\{update\}\(v\_\{t\-1\}\)&\\text\{w\.p\. \}p\_\{\\text\{change\}\},\\\\ v\_\{t\-1\}&\\text\{otherwise,\}\\end\{cases\}\\qquad\\mathrm\{update\}\(v\)=\\begin\{cases\}\\mathrm\{nbr\}\(v\)&\\text\{w\.p\. \}p\_\{\\text\{smooth\}\},\\\\ \\mathcal\{U\}\(V\_\{N\}\\setminus\\\{v\\\}\)&\\text\{w\.p\. \}1\-p\_\{\\text\{smooth\}\}\.\\end\{cases\}\(92\)wherenbr⁡\(v\)=v\+ε\\mathrm\{nbr\}\(v\)=v\+\\varepsilondrawsε\\varepsilonuniformly from the eight unit spatial offsets\. If the proposal leaves the latticeVNV\_\{N\}, the update holds \(nbr⁡\(v\)=v\\mathrm\{nbr\}\(v\)=v\), ensuring the stationary velocity distribution remains perfectly uniform\. Alternatively, to generate trajectories with more structured durations, we also employ a piecewise\-constant formulation where velocity is strictly held for a durationτ∼𝒰⁡\(smin,smax\)\\tau\\sim\\mathcal\{U\}\(s\_\{\\text\{min\}\},s\_\{\\text\{max\}\}\)before an update is triggered using the identical rules\.

##### Kinematic Metrics\.

Because out\-of\-bounds proposals are held rather than resampled, a triggered update does not strictly guarantee a change in velocity\. To accurately classify the motion regimes, we define two metrics measured from the realised trajectories\. First, the*realised*switching raterr\(the empirical probability of a velocity transition on a given frame\) is strictly lower than the nominal trigger ratepchangep\_\{\\text\{change\}\}:

r=pchange​\(1−psmooth​\(1−accN\)\),r=p\_\{\\text\{change\}\}\\bigl\(1\-p\_\{\\text\{smooth\}\}\(1\-\\mathrm\{acc\}\_\{N\}\)\\bigr\),\(93\)whereaccN\\mathrm\{acc\}\_\{N\}is the lattice\-dependent probability that a unit\-offset stays withinVNV\_\{N\}\. Second, to strictly decouple the frequency of these transitions from their severity, we define the jump magnitude\|Δ​v\|\|\\Delta v\|as theL∞L\_\{\\infty\}norm of the velocity difference,max⁡\(\|vx,t−vx,t−1\|,\|vy,t−vy,t−1\|\)\\max\(\|v\_\{x,t\}\-v\_\{x,t\-1\}\|,\|v\_\{y,t\}\-v\_\{y,t\-1\}\|\), averaged strictly over the subset of frames where a transition actually occurred\. By conditioning the expectation onvt≠vt−1v\_\{t\}\\neq v\_\{t\-1\}, the jump magnitude isolates the purely spatial severity of the transitions independently of their frequency\.

##### Velocity Freezing and Model Evaluation\.

As outlined in the main text, every sequence setsfreeze\_afterequal to the context length \(Tc​o​n​t​e​x​tT\_\{context\}\)\. This yields a two\-phase trajectory: a time\-varying phase during the encoding context, followed by a constant\-velocity phase over the entire predicted rollout\. Neither model is given access to the ground truth velocities; they must infer them solely from the video frames\. By freezing the rollout velocity, we establish a fair, perfectly balanced competition between the two architectures:

1. 1\.MELSTM \(Explicit Estimation\):To succeed, MELSTM must learn a robust encoding of the digits*and*successfully estimate the correct boundary velocity \(via phase correlation\) in order to explicitly transport those representations into the future\.
2. 2\.FELSTM \(Implicit Routing\):FELSTM does not estimate a single velocity vector, but instead maintains separate spatial channels for all valid velocities\. To succeed, it must learn a robust encoding of the digits*and*successfully route those representations into the correct velocity channels by the end of the context\.

Because the rollout velocity is constant, it is theoretically observable from the final context frames for both models\. By strictly freezing the kinematics during the rollout, the evaluation isolates their ability to encode and assign velocities, decoupling the competition from the impossible task of guessing unpredictable future velocity changes\.

#### B\.1\.2Training and Evaluation Setup

##### Dataset Configuration and Splits\.

Our standard training configuration utilizes the piecewise motion regime on𝕋362\\mathbb\{T\}^\{2\}\_\{36\}withτ∼𝒰⁡\(3,6\)\\tau\\sim\\mathcal\{U\}\(3,6\),psmooth=0\.80p\_\{\\text\{smooth\}\}=0\.80, andV2V\_\{2\}\(whereacc2=2/3\\mathrm\{acc\}\_\{2\}=2/3\)\. The expected segment duration of 4\.5 frames implies a nominal switching rate of2/9≈0\.2222/9\\approx 0\.222\. Applying the realized rate correction, the training dataset operates at a realized switching rate ofr=0\.163r=0\.163changes per frame, with a mean realized jump magnitude of\|Δ​v\|=1\.40\|\\Delta v\|=1\.40\. We use54,000/6,00054\{,\}000/6\{,\}000training/validation sequences \(a90/1090/10split of the MNIST training set\) and10,00010\{,\}000test sequences \(the MNIST test set\), ensuring digit identity never crosses between training and evaluation\. Training and validation sequences are generated on the fly: fresh digit pairs, placements, and velocity schedules are drawn on every access so no sequence is seen twice\. The test set and every evaluation benchmark are instead pre\-generated from fixed seeds\.

##### Optimization and Hyperparameters\.

All three models are trained identically:2525\-frame sequences \(1515context,1010predicted\), optimized via MSE\+\{\}\+\{\}L1 loss computed exclusively on the predicted frames\. We use the Adam optimizer with an initial learning rate of10−310^\{\-3\}andReduceLROnPlateauscheduling \(factor0\.50\.5, patience55, floor10−610^\{\-6\}\), along with gradient clipping at1\.01\.0\. Training proceeds with a batch size of3232for5050epochs \(totaling84,40084\{,\}400optimizer steps\) on a single NVIDIA A100 80 GB GPU\. Early stopping was never triggered\.

##### Parameter Budget\.

All three models have exactly47,68147\{,\}681parameters, all trainable:38,14438\{,\}144in the recurrent cell and9,5379\{,\}537in the decoder\. The counts agree exactly rather than approximately, because weights are shared across FELSTM’s2525velocity channels and MELSTM’sK=2K=2slots, and because MELSTM’s velocity model contributes no parameters at all \(Appendix[B\.1\.4](https://arxiv.org/html/2610.28771#A2.SS1.SSS4)\)\. The three architectures therefore differ only in how many hidden\-state copies they carry per step:11,2525and22respectively, and not in their number of learnable parameters\.

##### Validation Tracking\.

Validation loss is tracked at two resolutions: a fixed256256\-sequence probe scored every2525batches, which resolves training dynamics without resampling noise, and the full6,0006\{,\}000\-sequence validation split scored once per epoch\. The two curves agree closely throughout training, confirming that the high\-frequency probe is unbiased\.

#### B\.1\.3Length and Motion Generalization

![Refer to caption](https://arxiv.org/html/2610.28771v1/rollout_as_trained_seq0.png)

![Refer to caption](https://arxiv.org/html/2610.28771v1/rollout_as_trained_seq12.png)

Figure 8:Length generalization beyond the trained horizon\.Two held\-out sequences; rows are the ground truth, LSTM, FELSTM, and MELSTM\. The dashed box marks the2525frames seen in training \(1515context\+\+1010predicted\); the rest is extrapolation, run out tot=140t=140, past the8585\-step horizon scored in Figure[3](https://arxiv.org/html/2610.28771#S5.F3)\. LSTM collapses almost immediately, FELSTM smears into high\-frequency artifacts late in the rollout, and MELSTM stays sharp throughout\.##### Length Generalization\.

We extend the 10\-frame trained rollout to an 85\-frame free\-running rollout \(8\.5×8\.5\\timesthe trained horizon\), predictingt=16t=16throught=100t=100after1515context frames; Figure[3](https://arxiv.org/html/2610.28771#S5.F3)reports the resulting rollout MSE\. Because velocity is frozen throughout \(the velocity active at the last context transition is held constant\), the test isolates stable autoregressive spatial extrapolation from the ability to predict future velocity changes\. Figure[8](https://arxiv.org/html/2610.28771#A2.F8)shows the same behaviour qualitatively, run past the scored horizon tot=140t=140\.

##### Motion Generalization\.

To test robustness to changes in motion, we rescore the trained checkpoints, without retraining, on test sequences whose context motion differs from training; the velocity is still frozen at the context boundary\. Each model is scored by its median per\-sequence MSE over the trained horizon \(t=16t=16–2525\), divided by its own median on a reference motion with the training switching rate and jump sizes \(marked “trained” in Figure[4](https://arxiv.org/html/2610.28771#S5.F4)\), so11means the new motion costs the model nothing\. This removes the differences in overall error between the models, but not the difficulty of the new motion itself, which every model faces: LSTM’s curve reflects that difficulty, and the ratio of FELSTM to LSTM \(Figure[9](https://arxiv.org/html/2610.28771#A2.F9)\) isolates the cost specific to the FELSTM architecture\.

Figure 9:The channel\-handover penalty across the seven motion regimes\.*Top:*kinematics of the seven evaluation regimes and the training baseline\. The unpredictabilityρ\\rhocollapses the switching raterrand the jump magnitude\|Δ​v\|\|\\Delta v\|into a single number; net context displacement falls asρ\\rhorises, because the trajectories become diffusive\.*Bottom:*FELSTM’s architectural penalty \(MSE normalised by the LSTM baseline\) againstρ⁡\(1−ρ\)\\rho\(1\-\\rho\)\. Because FELSTM allocates separate channels to discrete velocities, a velocity change forces a handover that strands accumulated temporal history; we model its cost as the product of the disruption \(ρ\\rho\) and the predictive value of what is stranded \(1−ρ1\-\\rho\)\. The penalty is negligible up toρ⁡\(1−ρ\)≈0\.1\\rho\(1\-\\rho\)\\approx 0\.1, including under pure chaos \(churn\_max,ρ→1\\rho\\to 1\), where the stranded history is worthless, and grows steeply above it\.random\_walk\(frequent tiny jitters\) andchurn\_half\(less frequent but far larger jumps\) are kinematically opposite, yet sit at nearly symmetric points of the parabola and show nearly equal penalties\.
##### The Channel\-Handover Penalty\.

In the main text, we ordered the motion regimes by their kinematic unpredictability \(ρ\\rho\)\. To quantify this metric, letTdT\_\{d\}denote circular translation byddandm⁡\(d\)=1H​W​𝔼x​‖x−Td​x‖2m\(d\)=\\tfrac\{1\}\{HW\}\\,\\mathbb\{E\}\_\{x\}\\\|x\-T\_\{d\}x\\\|^\{2\}the mean squared image difference induced by displacementdd\. We define:

ρ=𝔼⁡\[m⁡\(Δ​v\)\]𝔼⁡\[m⁡\(vt\)\],\\rho\\;=\\;\\frac\{\\mathbb\{E\}\\bigl\[m\(\\Delta v\)\\bigr\]\}\{\\mathbb\{E\}\\bigl\[m\(v\_\{t\}\)\\bigr\]\},\(94\)where the numerator is the one\-step error of persisting the previous velocity, and the denominator is the error of assuming no motion\. Thus,ρ=0\\rho=0indicates pure flow, whileρ→1\\rho\\to 1indicates the past velocity offers no predictive advantage\.

This definition makes the product described in the main text concrete\. Handovers happen at the switching raterr, andρ\\rhoalso weights each one by the size of its jump, so it measures how disruptive the handovers are, while the predictive value of the history abandoned at a handover is roughly proportional to1−ρ1\-\\rho\. We therefore useρ⁡\(1−ρ\)\\rho\(1\-\\rho\)as a simple model of the FELSTM penalty\. It is a heuristic rather than a derived law, and the relation is not linear: the penalty stays near11for the four regimes withρ⁡\(1−ρ\)≤0\.11\\rho\(1\-\\rho\)\\leq 0\.11\(constant,rare\_teleport,churn\_maxand the training motion\) and then grows throughregular\_fast,random\_walkandchurn\_halftochurn\_mid, the regime closest toρ=0\.5\\rho=0\.5\(Figure[9](https://arxiv.org/html/2610.28771#A2.F9)\)\.

#### B\.1\.4Velocity Model

![Refer to caption](https://arxiv.org/html/2610.28771v1/figures/states_val_sample15.png)

![Refer to caption](https://arxiv.org/html/2610.28771v1/figures/states_train_sample4.png)

Figure 10:Slot specialisation\.Hidden\-state evolution for a validation sequence \(top\) and a training sequence \(bottom\)\. Within each block the first two rows show the channel\-averaged statesℋ¯1,ℋ¯2\\bar\{\\cal\{H\}\}^\{1\},\\bar\{\\cal\{H\}\}^\{2\}of theK=2K=2slots and the third the corresponding frame; column titles givett, in red once the decoder takes over\. Each slot localises onto one digit and tracks it, and this is the template that Eq\. \([97](https://arxiv.org/html/2610.28771#A2.E97)\) correlates against the incoming frame\.As introduced in the main text, we estimate local velocities using a slot tracking mechanism\. Specifically, each slot’s velocity is computed via phase correlation between its own hidden state and the current frame\.

##### Phase correlation\.

Fourier phase correlation is a global frequency\-domain technique for estimating the relative translation between two images or signals\. Given a reference image𝒳1\{\\cal\{X\}\}\_\{1\}and a shifted image𝒳2\{\\cal\{X\}\}\_\{2\}, the normalized cross\-power spectrum is computed as

R=𝔉⁡\[𝒳1\]⊙𝔉⁡\[𝒳2\]¯\|𝔉⁡\[𝒳1\]⊙𝔉⁡\[𝒳2\]¯\|,R\\;=\\;\\frac\{\\mathfrak\{F\}\[\{\\cal\{X\}\}\_\{1\}\]\\odot\\overline\{\\mathfrak\{F\}\[\{\\cal\{X\}\}\_\{2\}\]\}\}\{\\bigl\|\\mathfrak\{F\}\[\{\\cal\{X\}\}\_\{1\}\]\\odot\\overline\{\\mathfrak\{F\}\[\{\\cal\{X\}\}\_\{2\}\]\}\\bigr\|\},\(95\)where𝔉\\mathfrak\{F\}denotes the two\-dimensional Fourier transform,⊙\\odotis the Hadamard product, and the overline represents complex conjugation\. Applying the inverse Fourier transform toRRyields a correlation surface whose peak indicates the estimated translation between the two images:

u∗=argmaxu𝔉−1​\[R\]​\(u\)\.u^\{\*\}=\\argmax\_\{u\}\\mathfrak\{F\}^\{\-1\}\[R\]\(u\)\.\(96\)The height of this peak serves as a confidence measure for the estimate\. We denote this entire procedure as

u∗=𝒫1​\(𝒳1,𝒳2\),u^\{\*\}=\\mathcal\{P\}\_\{1\}\(\{\\cal\{X\}\}\_\{1\},\{\\cal\{X\}\}\_\{2\}\),where𝒫1\\mathcal\{P\}\_\{1\}is the phase correlation operator returning the translation vector that corresponds to the dominant peak of the correlation surface\. The subscript indicates that only the single strongest peak is considered\.

##### From a local problem to a global one\.

A frame containingNNindependently moving objects does not possess a single uniform velocity\. Consequently, recovering motion strictly from consecutive frames\(Xt−1,Xt\)\(X\_\{t\-1\},X\_\{t\}\)is inherently a*local*problem: the frame must be segmented and the correspondence resolved region by region, which is the primary challenge addressed by optical flow models\. By contrast, a global estimator like phase correlation is better suited for determining a single global motion\. Applying it directly to raw frames and extracting theKKstrongest peaks via𝒫K​\(Xt−1,Xt\)\\mathcal\{P\}\_\{K\}\(X\_\{t\-1\},X\_\{t\}\)does yieldKKvelocities at each timesteptt\. However, these velocities form an unordered set; there is no inherent mechanism to associate a peak at timettwith its corresponding peak at timet−1t\-1\. Converting these sets into coherent trajectories requires a separate assignment step at every timestep, typically relying on continuity heuristics or matching algorithms, where any error becomes unrecoverable\.

The introduction of the hidden state circumvents these difficulties\. Fort≥2t\\geq 2, each slot correlates its own channel\-averaged hidden state against the incoming frame:

utk=𝒫1​\(ℋ¯tk,𝒳t\),ℋ¯k=1Ch​∑cℋk​\[c\]\.u^\{k\}\_\{t\}\\;=\\;\\mathcal\{P\}\_\{1\}\\bigl\(\\bar\{\\cal\{H\}\}^\{k\}\_\{t\},\\,\{\\cal\{X\}\}\_\{t\}\\bigr\),\\qquad\\bar\{\\cal\{H\}\}^\{k\}=\\tfrac\{1\}\{C\_\{h\}\}\\textstyle\\sum\_\{c\}\{\\cal\{H\}\}^\{k\}\[c\]\.\(97\)Through this operation, slotkkeffectively queries where*its specific*content has moved\. Once a slot encapsulates a single coherently moving object, this query has exactly one valid answer: a single global displacement between the two fields, which phase correlation computes in closed form\. In this framework, the local tracking problem is not merely solved, but rather absorbed: the recurrent dynamics resolve the segmentation within the state itself, allowing the estimator to face only a global displacement problem\. Furthermore, each computed velocity is directly bound to the slot that requested it\. Consequently, the slot’s identity inherently acts as the tracking mechanism, naturally dissolving the correspondence problem without requiring explicit matching\.

##### Bootstrap\.

Att=0t=0, all slot states are initialized to zero\. Since warping a zero state has no effect and every slot receives identical input, theKKslots would ordinarily remain identical copies of one another indefinitely\. Breaking this symmetry requires an initial velocity estimate that does not originate from the hidden state\. We resolve this by computingu1\.\.K=𝒫K​\(X0,X1\)u^\{1\.\.K\}=\\mathcal\{P\}\_\{K\}\(X\_\{0\},X\_\{1\}\), which seeds the slots with theKKstrongest peaks from the first frame pair\. This initial step is the only instance where correlation is performed directly between raw frames, and the only time more than one peak is extracted\. For every timestept≥2t\\geq 2onward, the velocities are individually determined via Eq\. \([97](https://arxiv.org/html/2610.28771#A2.E97)\)\.

##### The state and the estimate improve together\.

Because the phase correlation estimator contains no trainable parameters, the observed velocity accuracy in Figure[11](https://arxiv.org/html/2610.28771#A2.F11)—which rises from≈66%\\approx 66\\%to over99\.9%99\.9\\%—indicates that the underlying*template*is improving\. Specifically, the hidden state progressively evolves into a cleaner, spatially localized representation of the object that each slot tracks \(as visualized in Figure[10](https://arxiv.org/html/2610.28771#A2.F10)\)\. These two quantities are coupled in a mutually reinforcing loop: a more accurate velocity transports the state more faithfully, a better\-transported state forms a cleaner template, and a cleaner template subsequently yields a sharper velocity estimate\. No external mechanism closes this loop, and no explicit motion supervision is provided\. Instead, the prediction loss shapes the hidden statehh, and the estimator directly consumes this increasingly refined state\. Therefore, the upward trend in accuracy serves as a direct probe of the model’s representation quality, rather than a learning curve for the estimator itself\.

Figure 11:Velocity accuracy\(one training run\)\. The estimator is parameter\-free and receives no gradient, so the rise from≈66%\\approx 66\\%at epoch55to above99\.5%99\.5\\%by epoch88reflects the hidden\-state template improving, not the estimator being fitted\. Final values are means over the last five epochs\. In the same run, the validation loss stays on a plateau near7×10−27\\times 10^\{\-2\}until about epoch66and then falls by an order of magnitude while the accuracy climbs: the model leaves the plateau when the slots lock onto their digits\. The five seeds of Figure[2](https://arxiv.org/html/2610.28771#S5.F2)leave it at different times, which smooths their mean\.

#### B\.1\.5Training with Frame\-Pair Velocities

Slot tracking needs a hidden state that is a good template, which takes several epochs to form \(Figure[11](https://arxiv.org/html/2610.28771#A2.F11)\), whereas velocities read from a frame pair do not depend on the state\. We therefore also trained MELSTM with frame\-pair velocities that are withdrawn by epoch2525: the two strongest peaks of𝒫2​\(𝒳t−1,𝒳t\)\\mathcal\{P\}\_\{2\}\(\{\\cal\{X\}\}\_\{t\-1\},\{\\cal\{X\}\}\_\{t\}\)are matched to the two slots by closeness to their previous velocities and replace slot tracking \(Eq\. \([97](https://arxiv.org/html/2610.28771#A2.E97)\)\)\. In the*stochastic*schedule, each stept≥2t\\geq 2of each training sequence uses them with a probability that decays from11to00along a half cosine\. In the*curriculum*, the steps2≤t<Tx2\\leq t<T\_\{x\}use them, whereTxT\_\{x\}decreases linearly from1515, the whole context, to22, so the latest steps switch to slot tracking first\. From epoch2525on, training uses slot tracking only; validation and testing always do\. With the stochastic schedule, every step is occasionally slot\-tracked from the second epoch on, and the validation loss falls steadily from the start \(Figure[12](https://arxiv.org/html/2610.28771#A2.F12)\)\. With the curriculum, the earliest steps switch last: the validation loss stays near that of LSTM until they do, at epoch2525, and then drops sharply, while the training loss jumps briefly \(Figure[13](https://arxiv.org/html/2610.28771#A2.F13)\)\. Both schedules end at about the test error of training with slot tracking throughout \(Figure[2](https://arxiv.org/html/2610.28771#S5.F2)\)\.

![Refer to caption](https://arxiv.org/html/2610.28771v1/appendix_val_train_xstocos.png)

Figure 12:Stochastic frame\-pair velocities\.In training, MELSTM uses frame\-pair velocities with a probability that decays to00by epoch2525; validation uses slot tracking only\. Losses \(MSE\+\{\}\+\{\}L1\) are means over three seeds with the min–max range shaded; faint traces are recorded every2525batches and circles once per epoch \(Appendix[B\.1\.2](https://arxiv.org/html/2610.28771#A2.SS1.SSS2)\)\. Table: test MSE over the trained horizon and att=100t=100; mean±\\pms\.d\.![Refer to caption](https://arxiv.org/html/2610.28771v1/appendix_val_train_xcur.png)

Figure 13:Frame\-pair curriculum\.As Figure[12](https://arxiv.org/html/2610.28771#A2.F12), but training uses frame\-pair velocities on the first steps of the context, a window that shrinks from the whole context to none by epoch2525\.

### B\.2Motion\-Only MNIST Classification

Classification shares with the future\-frame prediction experiment the36×3636\\times 36torus, the piecewise motion of Appendix[B\.1\.1](https://arxiv.org/html/2610.28771#A2.SS1.SSS1)with segments of33–66frames on the latticeV2V\_\{2\}, the recurrent cell, and the optimizer and learning\-rate schedule of Appendix[B\.1\.2](https://arxiv.org/html/2610.28771#A2.SS1.SSS2)\. It differs in how frames are rendered, in the task head and in the loss\.

##### Data\.

Each sequence holds one digit\. Its MNIST glyph is binarised at intensity0\.30\.3into a maskχ\\chi, equal to one on the digit and zero elsewhere, and placed uniformly at random on𝕋362\\mathbb\{T\}^\{2\}\_\{36\}\. The digit and the background have independent white\-noise texturesηd\\eta^\{\\mathrm\{d\}\}andηb\\eta^\{\\mathrm\{b\}\}\(unit variance, independent across pixels\) and their own motionsψd\\psi^\{\\mathrm\{d\}\}andψb\\psi^\{\\mathrm\{b\}\}, generated by velocitiesvtdv^\{\\mathrm\{d\}\}\_\{t\}andvtbv^\{\\mathrm\{b\}\}\_\{t\}\. Framettis

𝒳t=ψtd⋅\(χ⊙ηd\)\+\(1−ψtd⋅χ\)⊙\(ψtb⋅ηb\),\{\\cal\{X\}\}\_\{t\}\\;=\\;\\psi^\{\\mathrm\{d\}\}\_\{t\}\\cdot\\bigl\(\\chi\\odot\\eta^\{\\mathrm\{d\}\}\\bigr\)\\;\+\\;\\bigl\(1\-\\psi^\{\\mathrm\{d\}\}\_\{t\}\\cdot\\chi\\bigr\)\\odot\\bigl\(\\psi^\{\\mathrm\{b\}\}\_\{t\}\\cdot\\eta^\{\\mathrm\{b\}\}\\bigr\),\(98\)clipped at±3\\pm 3and mapped affinely to\[0,1\]\[0,1\]\. The first term is the digit’s texture inside its moved mask; the factor1−ψtd⋅χ1\-\\psi^\{\\mathrm\{d\}\}\_\{t\}\\cdot\\chiis zero on the digit, so the second term is the moved background with the digit cut out\. The velocities of the digit and of the background are each drawn from the motion generator of the prediction experiment, except that smooth updates are drawn uniformly from the valid lattice neighbours, giving a realised switching rate of0\.1930\.193and a mean jump of1\.191\.19\. The54,00054\{,\}000/6,0006\{,\}000training/validation glyphs are a90/1090/10split of the MNIST training set and the10,00010\{,\}000test glyphs are the MNIST test set, so no glyph is shared between training and evaluation\. Training sequences are rendered afresh on every access; validation and test sequences are fixed\. No single frame reveals the digit: binarisation discards its intensity, both layers have the same noise statistics, and white noise has no correlation between neighbouring pixels, so nothing marks the digit’s outline\. A single\-frame CNN \(three circular stride\-22convolutions with GroupNorm and global pooling\) accordingly scores12\.2%12\.2\\%against10%10\\%chance, so the digit can only be recovered from how its pixels move together\. Smoothing the textures with a Gaussian filter would leak the outline, because the correlation it creates breaks where the two textures meet: with a standard deviation of0\.50\.5,11or22pixels, the same CNN reaches13%13\\%,24%24\\%and36%36\\%\.

##### Classifier and training\.

The headgθg\_\{\\theta\}applies three3×33\\times 3convolutions with stride22, circular padding,1616channels, BatchNorm and ReLU, concatenates global average and max pooling, and ends in an MLP32→32→1032\\to 32\\to 10; with the recurrent cell this gives48,84248\{,\}842parameters for every model\. The velocity axis is max\-pooled as well\. Before every evaluation, the head’s BatchNorm statistics are re\-estimated on5050training batches with the current weights \(precise BatchNorm;[Wu & Johnson, 2021](https://arxiv.org/html/2610.28771#bib.bib29)\)\. Training differs from the future\-frame prediction experiment in the cross\-entropy loss, batch size6464, plateau patience44and8080epochs \(67,52067\{,\}520steps\); the checkpoint with the best validation accuracy is tested once\.

##### Velocity estimation in MELSTM\.

Att=1t=1, the two peaks of𝒫2​\(𝒳0,𝒳1\)\\mathcal\{P\}\_\{2\}\(\{\\cal\{X\}\}\_\{0\},\{\\cal\{X\}\}\_\{1\}\)\(Appendix[B\.1\.4](https://arxiv.org/html/2610.28771#A2.SS1.SSS4)\) seed the two slots, one for the digit and one for the background\. At later steps, each new peak goes to the slot whose previous velocity is nearest, which solves the assignment problem as long as velocities change little from one frame to the next\. With slot tracking \(Eq\. \([97](https://arxiv.org/html/2610.28771#A2.E97)\)\) instead, test accuracy is about one point lower, and the digit also appears in the hidden state\.

##### Hidden states\.

Figure[14](https://arxiv.org/html/2610.28771#A2.F14)shows the hidden states on example sequences: the digit builds up clearly in the MELSTM slot that follows it, but only faintly in FELSTM’s copies, since each velocity change moves it to another copy\.

![Refer to caption](https://arxiv.org/html/2610.28771v1/figures/cf_states_melstm.png)\(a\)MELSTM
![Refer to caption](https://arxiv.org/html/2610.28771v1/figures/cf_states_felstm.png)\(b\)FELSTM

Figure 14:Hidden states on Motion\-Only MNIST\.Channel\-averaged state of each MELSTM slot \(a\) and of the FELSTM copies at the velocities the digit takes during the context \(b\); the bottom rows show the input frames\. The digit’s outline is drawn only for the reader: the frames themselves are pure noise\. MELSTM’s slots1s\_\{1\}follows the digit and accumulates its shape, whiles0s\_\{0\}follows the background; because the digit’s velocity keeps changing, no FELSTM copy accumulates the shape for long\.
##### Why LSTM can leave chance\.

Each step of LSTM sees the current frame within11pixel, through its3×33\\times 3input kernel, and the previous frame within22pixels, through the hidden state and its3×33\\times 3recurrent kernel\. One step can therefore compare the two frames at every displacement up to33pixels, which covers every velocity in the data, so LSTM can learn local motion detectors that pick out the digit\. Finding such detectors in noise is hard, however, and LSTM does so only on some seeds\.

##### Motion\-generalization protocol\.

The sweep is evaluation only: each cell has50005000sequences on MNIST test glyphs, shared by the three models\. BatchNorm keeps the statistics stored with each checkpoint, estimated on the training motion; re\-estimating them on each cell would let the model partly adapt to the new motion\. The switching\-rate and jump\-size axes are those of the future\-frame prediction experiment\. That experiment also orders motion regimes by the unpredictabilityρ\\rho\(Eq\. \([94](https://arxiv.org/html/2610.28771#A2.E94)\)\), which compares the one\-step error of reusing the previous velocity with the error of assuming no motion\. On white noise every nonzero shift changes the image by the same amount, so both errors simply count the frames with a nonzero shift,ρ\\rhoequals the switching rate, and aρ\\rhopanel would repeat the switching\-rate panel\. For the speed axis, velocities are drawn fromVnV\_\{n\}with maximum speedn∈\{1,3,4,5\}n\\in\\\{1,3,4,5\\\}instead of22; FELSTM’s lattice contains only the velocities of speed at most22\. Relative accuracy divides each model’s accuracy in a cell by its accuracy on the training motion\.

### B\.3KTH Action Recognition under Camera Motion

##### Data\.

KTH has599599videos of2525people performing six actions in four scenarios, at2525frames/s and160×120160\\times 120pixels\. Each video is annotated with the segments in which the action is performed\. As in[Keller \(2025\)](https://arxiv.org/html/2610.28771#bib.bib12), the clips are the segments of at least3232frames, converted to grayscale and resized to32×3232\\times 32without keeping the aspect ratio\. People11–1616,1717–2020and2121–2525give1,5171\{,\}517training,380380validation and472472test clips\. The model sees1616frames of a clip, taken at every second frame, that is, at12\.512\.5frames/s\. In training, this window is drawn at random, and with probability1/21/2all its frames are mirrored left to right\. In validation and testing, each clip has one fixed window, not mirrored and shared by all test sets, so the test sets differ only in the camera motion\.

##### Camera motion\.

The frame at timettis translated by a whole\-pixel displacementd⁡\(t\)d\(t\), withd⁡\(0\)=0d\(0\)=0and wrap\-around\. Since every model uses circular padding, this is an exact group action without boundary effects\. In a shake, each axis ofd⁡\(t\)d\(t\)followsA​sin⁡\(2​π​t/P\+φ\)A\\sin\(2\\pi t/P\+\\varphi\), rounded to whole pixels and shifted so thatd⁡\(0\)=0d\(0\)=0; the amplitudeAA, the periodPPin frames and the phaseφ\\varphiare drawn once per clip and per axis, and the frequency isf=12\.5/Pf=12\.5/PHz\. The training shake hasA∈\[0,2\]A\\in\[0,2\]px andP∈\[16,32\]P\\in\[16,32\]\(f=0\.4f=0\.4–0\.80\.8Hz\)\. Its fastest step,2​A​sin⁡\(π/P\)2A\\sin\(\\pi/P\), stays below11px, so every step lies inV1V\_\{1\}, and the scene never moves more than44px from its first position\. Amplitudes below0\.50\.5px round to no motion, so6%6\\%of the training clips are static\. Validation uses the training shake\. See Figure[15](https://arxiv.org/html/2610.28771#A2.F15)for an example of a sequence with and without camera shake\.

![Refer to caption](https://arxiv.org/html/2610.28771v1/kth_handwaving_shake.png)Figure 15:Example KTH clip with and without camera shake\.All 16 frames of a handwaving clip from the test set\. Top: original clip \(static camera\)\. Bottom: the same clip with the camera shaking horizontally, withA=8A=8px andP=8P=8frames \(1\.6 Hz\)\.
##### Models\.

All models use the same convolutional encoder, with6464hidden channels and3×33\\times 3circular kernels \(150,016150\{,\}016parameters\), the max over copies or slots of the final hidden state, and the classifier head of Appendix[B\.2](https://arxiv.org/html/2610.28771#A2.SS2)with6464channels and an MLP128→128→6128\\to 128\\to 6with leaky ReLU \(128,262128\{,\}262parameters\)\. FELSTM carries99copies at the velocities ofV1V\_\{1\}while MELSTM has22slots, whose velocities are estimated to sub\-pixel precision and applied with an interpolating warp, and slot tracking considers every displacement up to half the frame\. Its frame\-pair velocities are the two strongest peaks of𝒫2​\(𝒳t−1,𝒳t\)\\mathcal\{P\}\_\{2\}\(\{\\cal\{X\}\}\_\{t\-1\},\{\\cal\{X\}\}\_\{t\}\)\. They come as an unordered set, so each is given to the slot whose previous velocity is nearest\. A constant camera shifts the new peaks and the previous velocities alike and leaves this matching unchanged, but a camera whose velocity changes does not, and a slot can then be given the velocity of another slot’s content\. Slot tracking needs no matching, since each slot finds its own content in the next frame, and is equivariant to any camera path\. Frame\-pair velocities are therefore used only early in training: each stept≥2t\\geq 2of each training sequence uses them with a probability that decays from11to00along a half cosine over the first200200epochs, and slot tracking otherwise\. Evaluation uses slot tracking only\.

##### Training\.

Every model is trained for500500epochs of4848batches of3232clips, about24,00024\{,\}000steps, with Adam, a learning rate decaying from3⋅10−43\\cdot 10^\{\-4\}to10−510^\{\-5\}along a cosine, gradient clipping at11and cross\-entropy\. Before every validation and test pass, the head’s BatchNorm statistics are re\-estimated on3030training batches \(precise BatchNorm;[Wu & Johnson, 2021](https://arxiv.org/html/2610.28771#bib.bib29)\), and the checkpoint with the best validation accuracy is tested\. The seed sets the weights, the data order, the training windows and the mirroring; evaluation windows and camera paths are the same for all seeds\. On one NVIDIA A100 GPU, the training pass of an epoch takes1\.31\.3s for LSTM,9\.19\.1s for FELSTM and3\.83\.8s for MELSTM \(median over epochs\)\.

##### Shake map\.

Figure[7](https://arxiv.org/html/2610.28771#S5.F7)usesA∈\{0,1,1\.5,2,3,4,6,8\}A\\in\\\{0,1,1\.5,2,3,4,6,8\\\}px andP∈\{32,24,16,12,8,6,4\}P\\in\\\{32,24,16,12,8,6,4\\\}frames, with a random phase per axis and clip;A=0\.5A=0\.5is left out because it rounds to no motion\. The fastest shake moves the camera by up to1111px between frames\. A shake stays on FELSTM’s lattice when its fastest step2​A​sin⁡\(π/P\)2A\\sin\(\\pi/P\)is at most11px, roughlyA≤P/2​πA\\leq P/2\\pi, which holds in1616of the moving cells\.