Geometry-Constrained Kolmogorov-Arnold Networks: Learning Edge Geometry via Banach Duality

K S Sesh Kumar https://t.co/frcSgYwUwx [𝚌𝚜.𝙻𝙶 𝚜𝚝𝚊𝚝.𝙼𝙻] https://t.co/51fDySQIsY

---

Learning Edge Geometry via Banach Duality

Source: https://arxiv.org/html/2608.25807

Geometry-Constrained Kolmogorov–Arnold Networks: Learning Edge Geometry via Banach Duality

K. S. Sesh KumarAffiliation:Brevan Howard Centre for Financial AnalysisAffiliation:Imperial Business SchoolEmail:[email protected]

August 26, 2026

Abstract

Kolmogorov–Arnold Networks (KANs) replace fixed activations in deep architectures with learnable univariate edge functions, making the choice of edge parametrisation central. Existing variants rely on fixed bases such as splines, polynomials, or Fourier features, which impose a function-space geometry before data are observed. We introduce geometry-constrained KANs, a family of edge activations derived from Banach duality maps in which the geometry itself is learned through a scalar exponentp>1p>1per edge. This exponent controls the qualitative response: sub-Euclidean values produce sharp, threshold-like behaviour reminiscent of theℓ1\ell_{1}(LASSO) geometry,p=2p=2recovers the linear regime, and larger values produce flatter responses near the origin. Across 50 symbolic-regression targets (4040from the AI Feynman benchmark plus1010synthetic stress tests), geometry-constrained KANs match or beat every fixed-basis baseline on median NRMSE (Banach-KAN0.0300.030, tying Chebyshev and improving on splines); on average rank Banach-KAN is best on the1818-equation core (2.002.00) and statistically tied with the strongest spline on the full benchmark (2.322.32vs.2.342.34). The clearest gains appear under measurement noise: asσ\sigmagrows from00to11,ℓp\ell^{p}-KAN degrades only3.7×3.7\times—below even a cross-validated spline (≈11×\approx\!11\times)—while an unregularised spline degrades21.6×21.6\times; Banach-KAN degrades8.8×8.8\times, comparable to a tuned spline but far more stable than the unregularised one. Banach-KAN also takes the most per-equation wins in the small-sample regime, with fixed-basis models catching up only as the training set grows. Learned exponents provide an interpretable, relative signal: at a fixed initialisation they reveal a consistent, target-dependent geometric ordering across equation families and input dimensions.

1Introduction

Many scientific regression problems are governed by low-dimensional nonlinear dependencies: pendulum periods depend on amplitude, radiation laws on temperature, and traffic speeds on density. Kolmogorov–Arnold Networks (KANs)[15]are designed for this setting by replacing fixed activations in standard neural networks with learnable univariate edge functions, summing their responses at each node. The central modelling decision in KANs is therefore the choice of edge function.

Existing approaches fix this choice through a basis: B-splines[15], radial basis functions[14], wavelets[5], Chebyshev polynomials[20], Jacobi polynomials[1], or Fourier features[22]. Each basis implicitly imposes a function-space geometry—such as smoothness, periodicity, or locality—before any data are observed. This choice is not benign.

Figure1shows that even at a matched parameter budget, different fixed bases fail in different ways when fitting simple discontinuous targets such as a Heaviside step or a multi-level staircase. No fixed basis performs uniformly well across both tasks. The limitation is not representational capacity but geometric mismatch: smooth bases approximate discontinuities by spreading them across coefficients, leading to oscillations or bias.

Figure 1:Fixed bases lock in geometry. All columns use a matched budget of2121trainable parameters, and each target is fit fromN=2000N=2000samples under label noiseσ=0.03\sigma=0.03, so every panel is a noisy fit rather than a noiseless interpolation. Different bases fail differently on the same targets, so no single fixed geometry is uniformly appropriate across both; the geometry-adaptiveℓp\ell^{p}-KAN adapts to each rather than committing to one. A budget sweep across{13,21,41,125}\{13,21,41,125\}parameters is reported inAppendixM.These observations suggest that the key modelling choice is not the basis itself, but the geometry it induces. Rather than fixing this geometry a priori, we seek a way to adapt it to the data.

The behaviour of a function class—its smoothness, sharpness, and growth—is governed by the geometry of the space it inhabits, which is determined by its underlying norm. A natural way to parameterise such geometries is through a one-parameter family that continuously interpolates between qualitatively different regimes.

This perspective leads to theℓp\ell^{p}family (Section2.4), where a single exponentppcontrols the geometry. Rather than selecting a basis, we parameterise each edge using the duality map associated with this family and learnppdirectly from data.Our goal is not to develop a full generalisation theory, but to provide a minimal, interpretable parameterisation of function-space geometry whose empirical effects we study.

This yieldsgeometry-constrained KANs: edge activations whose form is fixed by the underlying geometry but whose geometry is adaptive. The exponentppbecomes a simple, interpretable control that moves each edge between sharp, linear, and saturating regimes.

Contributions.

• •We introduce geometry-constrained KANs, where edge activations are derived from duality maps with a learned exponent.
• •We provide a unified interpretation of KAN edge functions as geometry-controlled mappings rather than basis expansions.
• •We show empirically that learning geometry (i) improves robustness under measurement noise—theℓp\ell^{p}edge degrades less than cross-validated spline regularisation—and (ii) yields better approximation than basis-based KANs in the small-sample regime, outperforming fixed bases and explicit spline regularisation.
• •We derive a testable prediction from the geometry—a trainable-depth conditionp>3/2p>3/2(Proposition1)—and isolate the geometry effect from generic parametric flexibility through matched-budget controls (Section5.1).
• •We demonstrate that learned exponents give an interpretable, init-anchored signal that reflects the structure of the underlying task.

2Background and Geometry

We develop a geometric perspective on KANs in which the choice of edge function is understood as a choice of function-space geometry. This viewpoint allows us to move from fixed basis constructions to a parameterisation in which the geometry itself is learned.

2.1Kolmogorov–Arnold Networks

Kolmogorov–Arnold Networks (KANs) replace fixed nonlinear activations with learnable univariate functions on edges. A layer is defined as

(Φℓ(x))j=∑i=1dℓ−1ϕj,i(ℓ)(xi),j=1,…,dℓ,(\Phi_{\ell}(x))_{j}=\sum_{i=1}^{d_{\ell-1}}\phi_{j,i}^{(\ell)}(x_{i}),\qquad j=1,\dots,d_{\ell},(1)so the model class is determined by the edge functionsϕj,i\phi_{j,i}. In contrast to standard neural networks, which fix the activation and learn affine weights, KANs fix the aggregation and learn the nonlinear transformations.

2.2Fixed bases as implicit geometry

Existing KAN variants parameteriseϕj,i\phi_{j,i}using fixed basis expansions, including B-splines[15], radial basis functions[14], wavelets[5], Chebyshev polynomials[20], Jacobi polynomials[1], and Fourier features[22].

Although these constructions differ, they share a common limitation: the function space is fixed before observing any data. Each basis encodes a particular inductive bias—such as smoothness, periodicity, or locality—which determines how functions are represented.

Figure1illustrates the consequences. Even with matched parameter budgets, different bases exhibit distinct failure modes when fitting discontinuous targets such as steps and staircases. Smooth bases approximate discontinuities by distributing them across coefficients, leading to oscillations or bias. This reflects a mismatch between the imposed geometry and the structure of the target.

Figure 2:Geometry induces activation. Convex potentials define the geometry, their gradients act as activations, and the unit balls illustrate how the geometry varies.

2.3Adaptive activations

A related line makes the activation itself trainable: adaptive-activation methods[11]learn an inputslopeinside a fixed-shape nonlinearityσ⁡(a​x)\sigma(ax), which accelerates convergence without leaving the activation’s regularity class. The construction we develop differs in kind. The exponentppchanges theshape and regularity class itself—threshold-like asp→1p\to 1, linear atp=2p=2, and flattened for largepp—so it is the induced geometry, not the input scale, that adapts. The learnable-shape-parameter idea is therefore not new, but adapting geometry rather than slope is; we isolate the two effects empirically inSection5.1, where at a matched per-edge budget a slope-adaptive edge is never competitive with the geometry-adaptive one.

2.4Geometry, duality, and activations

To move beyond fixed bases, we consider geometry at a more fundamental level. Let us consider Banach spaces[17,7], where geometry is induced by a norm: the norm determines the shape of the unit ball and therefore how distances, variations, and gradients are measured.

A norm also induces a canonical convex potential that encodes this geometry. For theℓp\ell^{p}family, this potential is

Ψp​(z)=1p​|z|p,\Psi_{p}(z)=\tfrac{1}{p}|z|^{p},whose level sets coincide with scaled unit balls. The gradient of this potential defines the associated duality map,

Jp​(z)=∇Ψp​(z)=sign⁡(z)​(|z|+ε)p−1,J_{p}(z)=\nabla\Psi_{p}(z)=\operatorname{sign}(z)(|z|+\varepsilon)^{p-1},which converts geometric structure into a nonlinear transformation[7].

This establishes a direct correspondence:

norm⇒geometry⇒convex potential⇒activation.\text{norm}\;\Rightarrow\;\text{geometry}\;\Rightarrow\;\text{convex potential}\;\Rightarrow\;\text{activation}. Figure2visualises this relationship. The convex potentials define the geometry, their gradients define the activations, and the unit balls illustrate how geometry varies withpp. Smaller values ofppproduce sharper responses, while larger values flatten behaviour near the origin and amplify tails.

This perspective extends beyondℓp\ell^{p}geometries. For example, the convex potential

Ψ⁡(z)=log⁡cosh⁡(z)\Psi(z)=\log\cosh(z)induces the activation

∇Ψ​(z)=tanh⁡(z).\nabla\Psi(z)=\tanh(z).Unlike the power-law growth ofℓp\ell^{p}, this potential grows quadratically near the origin and linearly in the tails, leading to bounded, saturating activations. As shown in Figure2, these two geometries induce qualitatively different response behaviours.

Taken together, this yields a unifying view: activation functions arise as gradients of convex potentials determined by geometry. Rather than selecting an activation directly, one selects a geometry, and the activation follows. Learning the exponentpptherefore corresponds to learning the underlying geometry.

2.5A design gap

The preceding discussion highlights a structural limitation in existing approaches. Current KAN variants implicitly fix the underlying function-space geometry through their choice of basis, while standard neural networks fix the activation and learn affine weights. In both cases, the geometry is specified a priori and remains unchanged during training.

This is restrictive: the experiments in Figure1show that no single fixed geometry is well-suited across different target behaviours. Smooth bases struggle with discontinuities, while bounded activations cannot capture unbounded growth. These limitations arise not from insufficient capacity, but from a mismatch between the imposed geometry and the structure of the data.

This suggests that the key modelling choice is not the basis or the activation itself, but the geometry it induces. A more flexible approach would allow this geometry to adapt during learning.

Concretely, we seek a parametrisation that:

• •retains the simple edge-wise structure of KANs,
• •avoids discrete basis selection or architectural search,
• •exposes geometry through a small number of continuous, interpretable parameters.

In the next section, we show that this can be achieved by promoting the exponentppto a learnable parameter on each edge, yielding geometry-constrained KANs.

3Geometry-Constrained KANs

Figure 3:(a)The two duality maps compared: Orlicz duality (tanh\tanh) saturates at±1\pm 1, providing bounded expressivity; ball duality (J2.5J_{2.5}) grows as|z|1.5|z|^{1.5}, providing adaptive power-law geometry.(b)Banach-KAN edge functions span diverse shapes by combining both: saturating curves (Orlicz-led), power-law growth (ball-led), non-monotone functions (mixed), asymmetric responses (highpp).We now define three KAN architectures that form a controlled family, each isolating a specific aspect of the geometry–expressivity trade-off (see Figures2and3). In all three cases, the edge functionϕj,i\phi_{j,i}in Eq. (1) takes the form of a parameterized univariate activation with an affine pre-activationz=w​x+bz=wx+b.

3.1Tanh-KAN: Expressivity Without Geometric Adaptation

Each edge function is

ϕ⁡(x)=a⋅tanh⁡(w​x+b)+d,\phi(x)=a\cdot\tanh(wx+b)+d,(2)with44learnable parameters{a,w,b,d}\{a,w,b,d\}per edge.

As discussed in Section2,tanh\tanharises from Orlicz duality, and is also one of the most widely studied and deployed activations in practice: the canonical gating nonlinearity in LSTM and GRU units[9,6], theoretically analysed in classical feedforward training studies[13], and a universal approximator for sigmoidal networks[8,10]. Because Banach-KAN contains this branch as itsa2=0a_{2}=0special case (Table1), the family inherits universal approximation, so expressivity rather than universality is the binding constraint at a fixed budget. This yields bounded, smooth edge functions with strong expressivity, but offers nolearnablegeometric constraint.

3.2ℓp\ell^{p}-KAN: Geometric Adaptation

Each edge function is

ϕ⁡(x)=a⋅Jp​(w​x+b)+d,Jp​(z)=sign⁡(z)​(|z|+ε)p−1,\phi(x)=a\cdot J_{p}(wx+b)+d,\qquad J_{p}(z)=\operatorname{sign}(z)(|z|+\varepsilon)^{p-1},(3)with55learnable parameters{a,w,b,d,p}\{a,w,b,d,p\}per edge; hereε=10−8\varepsilon=10^{-8}is a numerical smoothing, and the everywhere-C1C^{1}form ofJpJ_{p}actually used in training is given inSection4. The exponent is parameterised asp=1+exp⁡(θ)p=1+\exp(\theta)withθ∈ℝ\theta\in\mathbb{R}, allowing unconstrained optimisation.

The exponentppgoverns how each edge responds to its input. Figure3shows how varyingppproduces qualitatively different behaviours: sharp, threshold-like responses forp→1p\to 1, approximately linear behaviour nearp=2p=2, and flatter responses near the origin for largerpp. Unlike fixed-basis constructions, this control is continuous and learned directly from data.

This makesppa compact descriptor of local regularity: instead of selecting a basis or tuning a regularization parameter, the model adapts its geometry directly.

3.3Banach-KAN: Dual-Geometry Composition

The two constructions above correspond to distinct duality structures: Orlicz geometry (bounded, saturating) andℓp\ell^{p}geometry (unbounded, power-law). A natural question is whether these can be combined in a principled way.

Both activations are gradients of convex potentials:

tanh=∇Ψ1,Ψ1​(z)=log⁡cosh⁡(z),Jp=∇Ψ2,Ψ2​(z)=|z|pp.\tanh=\nabla\Psi_{1},\quad\Psi_{1}(z)=\log\cosh(z),\qquad J_{p}=\nabla\Psi_{2},\quad\Psi_{2}(z)=\frac{|z|^{p}}{p}. The Banach-KAN edge combines them:

ϕ⁡(x)=a1​tanh⁡(w1​x+b1)+a2​Jp​(w2​x+b2)+d,\phi(x)=a_{1}\tanh(w_{1}x+b_{1})+a_{2}J_{p}(w_{2}x+b_{2})+d,(4)with88learnable parameters{a1,a2,w1,w2,b1,b2,d,p}\{a_{1},a_{2},w_{1},w_{2},b_{1},b_{2},d,p\}per edge.

If both branches share the same pre-activationu=w​x+bu=wx+b, the edge reduces to∇(Ψ1+Ψ2)​(u)\nabla(\Psi_{1}+\Psi_{2})(u). By standard convex duality[3], the corresponding dual representation is given by the infimal convolutionΨ1∗​□​Ψ2∗\Psi_{1}^{*}\square\Psi_{2}^{*}, producing a principled blend of entropic (near-zero) and power-law (tail) behaviour.

In practice, the architecture uses independent affine parameters(w1,b1)(w_{1},b_{1})and(w2,b2)(w_{2},b_{2}), which breaks this exact identity but strictly generalizes it: each geometry can attend to a different projection of the input. This trades exact convex structure for increased expressivity while retaining the geometric interpretation. We are explicit that the reported model is only loosely tied to the duality construction: the tied-affine edge that preserves the exact∇(Ψ1+Ψ2)\nabla(\Psi_{1}+\Psi_{2})identity is reported as a control inAppendixN, where it is competitive—matching or beating the independent form under moderate noise and degrading more gently—but trails on clean and small-sample data, so the independent form remains the model we report.

The resulting edge operates in a product of two geometric families, combining bounded and unbounded responses. Structurally,tanh\tanhhas bounded derivative whileJpJ_{p}grows as a power law, so the combined gradient spans response regimes that neither branch reaches alone—which is why the conjunction, rather than either branch, is the operative model (Section5.1). Figure3shows that this produces a diverse class of behaviours not achievable by either geometry alone.

3.4Geometry-dependent sensitivity

The exponentppdoes not only change the shape of the edge function; it also controls how perturbations are amplified through the activation. Consider anℓp\ell^{p}edge

ϕ⁡(x)=a​Jp​(w​x+b)+d,Jp​(z)=sign⁡(z)​(|z|+ε)p−1.\phi(x)=aJ_{p}(wx+b)+d,\qquad J_{p}(z)=\operatorname{sign}(z)(|z|+\varepsilon)^{p-1}.The smoothed duality map has derivative

Jp′​(z)=(p−1)​(|z|+ε)p−2,z≠0.J_{p}^{\prime}(z)=(p-1)(|z|+\varepsilon)^{p-2},\qquad z\neq 0.On a bounded input domain|x|≤R|x|\leq Rwith|z|≤M:=|w|​R+|b||z|\leq M:=|w|R+|b|, the edgeϕ⁡(x)=a​Jp​(w​x+b)+d\phi(x)=aJ_{p}(wx+b)+dis therefore Lipschitz with

Lip⁡(ϕ)≤|a|​|w|​(p−1)​sup|z|≤M(|z|+ε)p−2.\operatorname{Lip}(\phi)\leq|a|\,|w|\,(p-1)\sup_{|z|\leq M}(|z|+\varepsilon)^{p-2}.(5) For Banach-KAN, thetanh\tanhbranch is|a1|​|w1||a_{1}||w_{1}|-Lipschitz, since|tanh′⁡(z)|≤1|\tanh^{\prime}(z)|\leq 1. Combining the two branches gives

Lip⁡(ϕ)≤|a1|​|w1|+|a2|​|w2|​(p−1)​sup|z|≤M2(|z|+ε)p−2.\operatorname{Lip}(\phi)\leq|a_{1}||w_{1}|+|a_{2}||w_{2}|(p-1)\sup_{|z|\leq M_{2}}(|z|+\varepsilon)^{p-2}.(6) The bound in Eq. (5) is non-monotone inpp: it is small forppnear22and large in both limitsp→1+p\to 1^{+}(becauseεp−2→∞\varepsilon^{p-2}\to\infty) andp→∞p\to\infty(becauseMp−2→∞M^{p-2}\to\inftyforM>1M>1).

3.5What the geometry predicts: trainable depth

The sensitivity analysis above is local. A complementary, global consequence of the exponent is a condition for trainability at depth, and it is the one place where the geometric perspective makes a derived, testable prediction rather than motivating a form. Consider the exact mapJpJ_{p}in a plain feedforward stack with Gaussian preactivations, and letχ1\chi_{1}be the mean squared singular value of a layer’s Jacobian—the standard order parameter for signal propagation.

Proposition 1(Trainable depth).

For the exact duality mapJp​(z)=sign⁡(z)​|z|p−1J_{p}(z)=\operatorname{sign}(z)|z|^{p-1}in a plain feedforward network whose preactivations are Gaussian with any varianceq>0q>0,

χ1=σw2​(p−1)2​qp−2​𝔼Z∼𝒩⁡(0,1)​[|Z|2​p−4],\chi_{1}\;=\;\sigma_{w}^{2}\,(p-1)^{2}\,q^{\,p-2}\;\mathbb{E}_{Z\sim\mathcal{N}(0,1)}\!\left[|Z|^{2p-4}\right],which is finite if and only ifp>3/2p>3/2, for everyqq. Whenp>3/2p>3/2a unique critical weight scaleσw\sigma_{w}solvingχ1=1\chi_{1}=1exists; at or belowp=3/2p=3/2none exists at anyqq.

Proof.

With weights i.i.d.𝒩⁡(0,σw2/fan-in)\mathcal{N}(0,\sigma_{w}^{2}/\text{fan-in})the preactivation isz=q​Zz=\sqrt{q}\,Z,Z∼𝒩⁡(0,1)Z\sim\mathcal{N}(0,1), andχ1=σw2​𝔼​[Jp′​(z)2]\chi_{1}=\sigma_{w}^{2}\,\mathbb{E}[J_{p}^{\prime}(z)^{2}]withJp′​(z)=(p−1)​|z|p−2J_{p}^{\prime}(z)=(p-1)|z|^{p-2}; henceχ1=σw2​(p−1)2​qp−2​𝔼​|Z|2​p−4\chi_{1}=\sigma_{w}^{2}(p-1)^{2}q^{\,p-2}\,\mathbb{E}|Z|^{2p-4}. Since𝔼​|Z|s=2s/2​Γ​(s+12)/π\mathbb{E}|Z|^{s}=2^{s/2}\Gamma(\tfrac{s+1}{2})/\sqrt{\pi}is finite iffs>−1s>-1, the moment𝔼​|Z|2​p−4=2p−2​Γ​(p−32)/π\mathbb{E}|Z|^{2p-4}=2^{\,p-2}\Gamma(p-\tfrac{3}{2})/\sqrt{\pi}is finite iffp>3/2p>3/2, for everyqq. Forp>3/2p>3/2the coefficient ofσw2\sigma_{w}^{2}is a finite positive constant, soχ1=1\chi_{1}=1has the unique solutionσw2=[(p−1)2​qp−2​𝔼​|Z|2​p−4]−1>0\sigma_{w}^{2}=[(p-1)^{2}q^{\,p-2}\mathbb{E}|Z|^{2p-4}]^{-1}>0; at or belowp=3/2p=3/2the moment diverges, soχ1=+∞\chi_{1}=+\inftyfor everyσw>0\sigma_{w}>0and no critical scale exists at anyqq. ∎

Table 1:Members of the geometry-constrained KAN family. Top: special-value members of Banach-KAN. Bottom: standalone variants.

4Practical Implementation and Inference

We now describe how geometry-constrained KANs are fit in practice. The goal is to estimate the per-edge exponentθ\thetatogether with the standard activation parameters from training data.

Loss and optimiser.Given training pairs{(xi,yi)}i=1n⊂ℝd×ℝ\{(x_{i},y_{i})\}_{i=1}^{n}\subset\mathbb{R}^{d}\times\mathbb{R}, we minimise the empirical mean squared error

ℒ⁡(Θ)=1n​∑i=1n(fΘ​(xi)−yi)2,\mathcal{L}(\Theta)\;=\;\frac{1}{n}\sum_{i=1}^{n}\big(f_{\Theta}(x_{i})-y_{i}\big)^{2},withΘ\Thetacollecting the per-edge parameters(a,w,b,d,θ)(a,w,b,d,\theta). Optimisation is unconstrained onθ\theta, withp=1+exp⁡(θ)p=1+\exp(\theta)enforcingp>1p>1. We use Adam[12]with cosine annealing for5,0005{,}000steps and gradient clipping at1010. The optimisation is low-dimensional per edge, withθ\thetaas the only non-standard parameter.

Initialisation from the Euclidean reference.The exponent admits a natural reference point atp=2.5p=2.5, slightly above the Euclidean value, where the activation is sufficiently nonlinear to provide useful expressivity but not so peaked as to cause numerical difficulties atz=0z=0. This corresponds toθ=log⁡(1.5)≈0.405\theta=\log(1.5)\approx 0.405. In practice we initialise every edge from this point and rely on gradient descent to redistribute exponents across edges. Empirically, this initialisation gives stable training across all our experiments in Section5and recovers a wide spread of final exponents. In the benchmark-equations experiment of Section5.1, only1.2%1.2\%of edges remain within±0.01\pm 0.01of the initialisation, and edges genuinely explore both the sub-Euclidean and super-Euclidean regimes.

Numerical stabilisation and theC1C^{1}map.The exactℓp\ell^{p}duality mapJp​(z)=sign⁡(z)​|z|p−1J_{p}(z)=\operatorname{sign}(z)|z|^{p-1}has a singular derivative atz=0z=0forp<2p<2. The additive smoothingsign⁡(z)​(|z|+ε)p−1\operatorname{sign}(z)(|z|+\varepsilon)^{p-1}that appears in the display equations removes the singular value but is not itself differentiable at the origin: for1<p<21<p<2it retains a jump of2​εp−12\varepsilon^{p-1}there, and the corresponding potential1p​(|z|+ε)p\tfrac{1}{p}(|z|+\varepsilon)^{p}has a corner. We therefore implement the everywhere-C1C^{1}duality map

Jpε​(z)=z​(z2+ε2)(p−2)/2=∇[1p​(z2+ε2)p/2],J_{p}^{\varepsilon}(z)\;=\;z\,(z^{2}+\varepsilon^{2})^{(p-2)/2}\;=\;\nabla\!\left[\tfrac{1}{p}(z^{2}+\varepsilon^{2})^{p/2}\right],the gradient of a convex, everywhere-differentiable potential, withε=10−8\varepsilon=10^{-8}and no per-task tuning. This is the correct realisation of differentiability atz=0z=0and preserves the geometric interpretation, agreeing with|z|p−1|z|^{p-1}in the tails (Figure4); empirically it reproduces the additive-ε\varepsilonnumbers—clean medians within0.5%0.5\%and the same noise-degradation ratios (Table2)—so the correction changes no result-table entry. Gradient clipping at1010controls the near-origin sensitivity: the slopeJpε′(0)=εp−2≈104J_{p}^{\varepsilon\prime}(0)=\varepsilon^{p-2}\approx 10^{4}for(p,ε)=(1.5,10−8)(p,\varepsilon)=(1.5,10^{-8}), so the clip fires routinely on sub-Euclidean edges and keeps training stable.

Figure 4:TheC1C^{1}duality map approximates the exact map and regularises its gradient.Top:the exact mapJp​(z)=sign⁡(z)​|z|p−1J_{p}(z)=\operatorname{sign}(z)|z|^{p-1}(dashed), the additive-ε\varepsilonsmoothingsign⁡(z)​(|z|+ε)p−1\operatorname{sign}(z)(|z|+\varepsilon)^{p-1}, and the adoptedC1C^{1}mapz​(z2+ε2)(p−2)/2z(z^{2}+\varepsilon^{2})^{(p-2)/2}coincide away from the origin; nearz=0z=0the additive-ε\varepsilonmap has a jump of2​εp−12\varepsilon^{p-1}forp<2p<2, whereas theC1C^{1}map passes through the origin smoothly.Bottom:the gradient—the exact(p−1)​|z|p−2(p-1)|z|^{p-2}is singular atz=0z=0forp<2p<2, and theC1C^{1}map replaces it with a bounded, smooth peak of heightεp−2\varepsilon^{p-2}, the quantity the gradient clip acts on. A visibleε=0.1\varepsilon=0.1is shown for legibility; the experiments useε=10−8\varepsilon=10^{-8}, for which the curves are indistinguishable outside anε\varepsilon-neighbourhood of00.Table 2:TheC1C^{1}map reproduces theε\varepsilon-map: noise-degradation ratio (σ=0→1\sigma{=}0\to 1) under three realisations ofJpJ_{p}; clean medians agree within0.5%0.5\%, so the differentiability correction changes no result-table entry.Remarks.Additional implementation details are provided in the appendices: the preregistered Spline–Adam control schedule (AppendixA), per-equation training-time breakdowns (AppendixB), and the CV protocol for the spline regularisation baseline (AppendixC).

5Experiments

In this section we report results of geometry-constrained KANs on synthetic and real datasets.

Protocol.Symbolic regression — predominantly the AI Feynman benchmark[21]augmented with custom stress-test targets — is the primary evaluation; PMLB[19]is a tabular sanity check. All benchmark-equation results report median NRMSE=RMSE/std⁡(ytest)=\mathrm{RMSE}/\mathrm{std}(y_{\mathrm{test}}); all methods share the same train/test splits and input normalisation.

Baselines.Four comparators: Spline G=3 (B-spline KAN[15]via pykan,G=3G=3,k=3k=3, L-BFGS), Spline CV (cross-validated regularised B-spline KAN; details inAppendixC), Cheby-KAN[20](degree-44Chebyshev polynomials), and a5050-unit ReLU MLP. An L-BFGS-vs-Adam optimiser control for the spline baseline (AppendixA) does not affect the overall results.

Datasets.

• •Benchmark equations:5050targets in22–66D,4040from AI Feynman[21]and1010synthetic stress tests (AppendixL). Types: polynomial, rational, trigonometric, compositional, exponential, logarithmic, hyperbolic. The Core 18 subset is used for noise, small-sample, and fixed-ppsweeps.
• •PMLB[19]:1212regression datasets,d∈[2,10]d\in[2,10],n∈[108,1,000]n\in[108,1{,}000], covering geophysics, pollution monitoring, computing, and epidemiology.

All KAN models use architecture[d,K,1][d,K,1]withK=5K=5ford≤2d\leq 2,K=4K=4ford∈{3,4}d\in\{3,4\}, andK=3K=3ford≥5d\geq 5. Counted as trainable weights, the pykan spline uses88per edge—the same as Banach-KAN, and more than Cheby-KAN’s77(the spline’s larger nominal footprint is non-trained grid, mask, and bias buffers)—so we make no parameter-efficiency claim; the fairness of the comparison rests instead on the matched-budget controls ofSection5.1. The full5050-equation list (AppendixL), PMLB descriptions (AppendixJ), per-dimension parameter counts (AppendixB), and a pendulum-period interpretability case study (AppendixI; a near-boundary logarithmic singularity that local-basis splines cannot extrapolate to) are provided in the appendices.

Table 3:Clean-data baseline (n=500n=500,σ=0\sigma=0). Two column groups: “Full5050equations” uses the complete benchmark, “Core1818equations” uses the subset re-used in the noise and small-sample sweeps.### 5.1Benchmark equations

Baseline.Table3shows three diagnostic patterns. First, Spline G=3 takes the most wins (21/5021/50) yet loses on median NRMSE: a bimodal error signature in which the spline is sharp where its grid resolution suffices and brittle elsewhere. Cheby-KAN is the inverse—uniformly close to the front, rarely strictly best, reflecting global polynomial expressivity without local adaptivity. The wins-vs-median gap therefore separates sharp-but-fragile bases from uniformly competent ones. Second, Banach-KAN attains the best average rank on both subsets (2.322.32on5050eq,2.002.00on the core1818) and is the only family member that wins consistently under both aggregations; Tanh-KAN andℓp\ell^{p}-KAN, each carrying only one ingredient (bounded saturation or adaptive power-law geometry), never win outright on clean data, so the Banach conjunction is what is operative. Third, MLP becomes competitive only at2020–30×30\timesmore parameters, so its ranks are bought rather than earned at parity. Stratifying by input dimension makes the regime explicit: Spline G=3 dominates at lowddwhere its77control points per edge cover the input, while atd≥4d\geq 4the budget runs out and wins fragment across Banach, Cheby, and MLP with no single dominant model—exactly the regime where adaptive geometry should pay over a fixed prior (AppendixB).

Learned exponents on the same set are interpretable and target-specific. Across4,4654{,}465Banach-KAN edges,18.7%18.7\%learnp<2p<2, only1.2%1.2\%remain within±0.01\pm 0.01of the initialisationp=2.5p=2.5, and the within-run standard deviation ofppexceeds0.50.5in77%77\%of runs—edges specialise rather than collapse. The distribution by equation type is consistent across runs: families with sharper local behaviour (logarithmic, hyperbolic, exponential, trigonometric) push edges belowp=2p=2, while smoother polynomial and power targets stay above. The sub-Euclidean fraction grows monotonically with input dimension (11%11\%at22D to34%34\%at66D), and exponents are stable under noise (mean absolute shift fromσ=0\sigma=0toσ=0.3\sigma=0.3is0.0940.094), sopptracks the target rather than the noise level (AppendixH). We are precise about what this asserts: the per-equation mean exponent is seed-noisy (between-equation spread exceeds within-equation spread), and the sub-Euclidean fraction is stable across seeds (across-seed SD0.0790.079) only at a fixed initialisation—initialising nearp=2p=2leaves almost none—so the claim is a relative, aggregate, init-anchored signal (sharper targets receive lowerpp), not a per-edge or absolute-threshold property.

Table 4:Left: median NRMSE under noise atn=500n=500(best per column inbold). Right: number of equations won atσ=0\sigma=0for each sample size (out of1818).55seeds.Noise robustness (n=500n=500, median NRMSE)Small-sample wins out of1818(σ=0\sigma=0)Modelσ=0\sigma\!=\!00.10.10.30.30.50.51.01.0n=50n\!=\!5010010020020050050010001000Spline G=30.0370.0770.1970.3460.78932426Spline CV0.0360.0540.1320.1870.40303161Tanh0.0890.0940.1120.1850.28300000Banach0.0300.0410.0980.1450.26387553ℓp\ell^{p}0.0680.0770.1080.1380.24932000MLP0.0480.0730.1480.2000.38143344Cheby0.0290.0470.1200.2020.43501514Noisy data.Table4(left) sweepsσ∈{0,0.1,0.3,0.5,1.0}\sigma\in\{0,0.1,0.3,0.5,1.0\}atn=500n=500. Fixed-basis models degrade more under measurement noise (Spline G=321.6×21.6\times, Cheby-KAN14.9×14.9\timesfromσ=0\sigma=0toσ=1.0\sigma=1.0) than the geometry-constrained KANs (ℓp\ell^{p}-KAN3.7×3.7\times, Tanh-KAN3.2×3.2\times, Banach-KAN8.8×8.8\times). Among the three geometry-constrained variants, Banach-KAN is the lowest-NRMSE model at low noise, whereas atσ=1.0\sigma=1.0theℓp\ell^{p}and Tanh variants overtake it. The comparison holds against recent fixed-basis KANs as well: adding RBF-[14], Fourier-[22], Wavelet-[5], and Jacobi-KAN[1]at matched per-edge budget under a single Adam protocol, all four degrade by1010–17×17\timesfromσ=0\sigma=0toσ=1\sigma=1, joining Spline and Cheby, while the geometry family degrades least; averaged over theσ\sigma-sweep Banach-KAN holds the best rank (2.122.12), ahead of every fixed basis, and atσ≥0.5\sigma\geq 0.5no fixed basis wins a single equation (Table5). Per-equation breakdowns are inAppendixE.

Table 5:Median NRMSE under noise, unified Adam (Core1818,55seeds); “Degr.” is theσ=0→1\sigma{=}0\to 1ratio. Right column: mean rank over theσ\sigma-sweep{0,0.1,0.3,0.5,1.0}\{0,0.1,0.3,0.5,1.0\}across the eight edge models.Small-sample approximation.Table4(right) sweepsn∈{50,100,200,500,1000}n\in\{50,100,200,500,1000\}atσ=0\sigma=0and reports per-condition wins (out of1818).Banach-KAN approximates targets better than basis-based KANs in the small-sample regime:8/188/18wins atn=50n=50and7/187/18atn=100n=100, the most of any single model. Basis-based KANs need data—Spline G=3 climbs from3/183/18atn=50n=50to6/186/18atn=1000n=1000; Cheby-KAN from00to44; Spline CV takes most of its wins atn=500n=500(6/186/18). Across the9090cells, Banach-KAN wins2828(Spline G=31717, Spline CV1111, MLP1818, Cheby1111,ℓp\ell^{p}55, Tanh00); Banach-KAN is the only family member uniformly competitive across the budget range (per-equation breakdowns inAppendixF).

Architecture dissection.The three geometry-constrained models—Tanh-KAN (bounded expressivity, no geometric adaptation),ℓp\ell^{p}-KAN (geometric adaptation, no bounded saturation), and Banach-KAN (the conjunction)—form a controlled ablation of the expressivity–geometry trade-off. In the three-way comparison (AppendixG), Banach wins clean data,ℓp\ell^{p}wins in the scarce noisy regime, and Tanh contributes under moderate noise. Freezinga2=0a_{2}=0disablesJpJ_{p}while keepingtanh\tanhtrainable: median clean-data NRMSE degrades by1.91×1.91\times, all18/1818/18equations worsen, with the largest gap5.0×5.0\times(AppendixD). Withinℓp\ell^{p}-KAN, learnedppwins17/1817/18against any fixedppchoice (AppendixH), confirming that the departure from Euclidean geometry—not a power-law activation per se—is the operative mechanism. A single-edge probe (AppendixK) shows Banach reproducing four univariate targets (sigmoid, cubic, scaled BesselJ0J_{0}, exponential) at NRMSE<0.04<0.04whileℓp\ell^{p}fails on the bounded sigmoid—direct evidence that thetanh\tanhbranch supplies expressivity theJpJ_{p}branch lacks.

Isolating geometry from parametric flexibility.Banach-KAN can also be read as a compact parametric edge, which raises the question of whether its gains come from the geometry or simply from a more flexible activation with more parameters. We isolate this with a panel of parametric-activation controls—a learnable-power edge, a rational/Padé edge[18,4], a mixture of standard activations[16], and a slope-adaptive edge in the style of adaptive activations[11]—each at its family’s natural per-edge budget under one Adam/cosine/gradient-clip protocol (Core1818,55seeds). Banach’s88parameters per edge sit at the upper end of this range; the two controls at its own budget, mixture (77) and rational (88), still lose (0.0520.052and0.0660.066clean median NRMSE against Banach’s0.0290.029), so the effect is not a parameter-count artefact. Against the four controls Banach-KAN wins clean15/1815/18(rank1.171.17within the seven-model set) and small-sample60/9060/90; a bare learnable-power edge sits close toℓp\ell^{p}-KAN and far from Banach (0.0860.086), and the slope-adaptive edge, swept over three learning rates and three slope initialisations, is best at∼2.8×\sim\!2.8\timesBanach’s error (0.0710.071vs0.0260.026)—never competitive, and not a tuning artefact. Together with the single-scalar learned-ppvs fixed-pptest (learnedppwins17/1817/18), this places the gain on the learned geometry rather than on generic parametric flexibility (Table6).

Table 6:Parametric-activation controls, clean data, unified Adam (Core1818,55seeds). “Rank” is the mean rank within the seven-model set; “Small-sample” is wins out of9090. The Banach–mixture clean gap is−0.023-0.023(95%95\%CI[−0.039,−0.001][-0.039,-0.001]) by paired equation-bootstrap.Extrapolation.BecauseJpJ_{p}is globally power-law, geometry-constrained edges carry a controlled behaviour outside the training range that local bases lack. On a Core-1818protocol that trains on the inner60%60\%of each input box and tests out-of-range (33seeds), Banach-KAN andℓp\ell^{p}-KAN attain the lowest median NRMSE (0.350.35and0.420.42), ahead of Spline (0.730.73), Cheby (0.800.80), RBF (0.860.86), and Fourier (1.661.66); the bounded Tanh edge is worst among the geometry variants (1.101.10), consistent with saturation. This mirrors the pendulum-period case study ofAppendixI, where a near-boundary logarithmic singularity defeats local-basis splines (Table7).

Table 7:Extrapolation (train inner60%60\%, test out-of-range): median NRMSE and per-model wins out of1818.Scaling and scope.Our scope is deliberately low-to-mid-dimensional regression, and we do not compete with deep architectures on their own ground, where an MLP matches these targets only at2020–30×30\timesthe parameters (Table3). As a boundary probe, replacing only the final-layer activation of a ResNet backbone by the geometry map—leaving all other activations as ReLU—gives Banach-KAN0.8810.881vs ReLU0.9110.911on CIFAR-10 and0.5980.598vs0.6340.634on CIFAR-100, with a larger gap on STL-10 (0.440.44–0.490.49vs0.790.79). This is a readout rather than a depth test: it sits outsideProposition1, and the plausible limiter is the sensitivity of an unbounded power-law readout, whose Lipschitz constant grows withpp(Section3.4), which we flag as a hypothesis. The faithful inner-layer variant did not complete within budget, so a direct depth test remains future work. We report the negative to mark the boundary of the regime, not because we entered a contest for it (Table8).

Table 8:Test accuracy, matched parameters; final-layer geometry only, ReLU backbone.Beyond symbolic regression: tabular data.To check that the mechanism extends beyond synthetic targets, we evaluate on1212PMLB regression datasets[19](d∈[2,10]d\in[2,10],n∈[108,1,000]n\in[108,1{,}000]),55seeds,80/2080/20split, standardised features and target.

Table 9:Per-dataset median NRMSE (55seeds, lower is better, best per row inbold) on1212PMLB regression datasets. The geometry-constrained family wins10/1210/12datasets; Spline G=3 wins zero. Last two rows aggregate over the table.DatasetDomainddnnSplineTanhBanachℓp\ell^{p}MLP712_chscase_geyser1geophysics22221.020.460.480.460.48678_vis_environmentalenvironmental31111.800.860.990.910.921027_ESLordinal44880.520.370.400.410.401030_ERAordinal410000.810.810.810.810.81210_cloudmeteorology51080.690.410.510.410.35230_machine_cpucomputing62090.760.270.310.330.27665_sleuth_case2002biology61472.011.141.211.031.22561_cpucomputing72090.080.050.040.060.10522_pm10pollution75001.560.941.170.921.04547_no2pollution75001.340.690.910.750.87666_rmftsa_ladataepidemiology105081.080.692.050.660.831028_SWDordinal1010000.820.800.810.810.82Wins (out of 12)–––05141Mean rank–––4.831.673.082.253.17The pattern matches the benchmark-equation results (Table9): Tanh-KAN is the strongest single model (55wins, mean rank1.671.67);ℓp\ell^{p}-KAN takes44datasets with power-law/heavy-tailed structure. Spline G=3 wins zero and has the worst mean rank (4.834.83), often with NRMSE above11where geometry-aware models stay below0.50.5. The geometry-constrained family wins10/1210/12; MLP wins one despite several-times-more parameters. Learnedppon the Banach branch concentrates near[2.5,3.3][2.5,3.3], consistent with super-Euclidean geometry on noisy real-world data.

6Discussion and Conclusion

The operative choice in KANs isfunction-space geometry, not the basis or activation. A learned per-edgeppmatches or improves upon fixed alternatives without tuning. On the clean-data benchmark, Banach-KAN takes the best average rank on the Core 18 (2.002.00) and is statistically tied with the strongest spline on the full 50-equation set (2.322.32vs.2.342.34), while matching or beating fixed-basis baselines on the median NRMSE. Under measurement noise the geometry-constrained variants degrade less than fixed-basis baselines asσ\sigmarises (ℓp\ell^{p}3.7×3.7\timesand Tanh3.2×3.2\times—below even a cross-validated spline (≈11×\approx\!11\times)—versus21.6×21.6\timesfor the unregularised spline, with Banach at8.8×8.8\times, fromσ=0\sigma=0toσ=1\sigma=1), and Banach-KAN takes the highest number of per-equation wins in the small-sample regime. The three geometry variants play distinct empirical roles: Banach is the lowest-NRMSE model at low noise and smallnn,ℓp\ell^{p}takes over at high noise, and Tanh contributes a bounded-saturation component thatℓp\ell^{p}alone cannot supply. Learned exponents provide an interpretable, init-anchored signal across equation families and input dimensions: edges specialise rather than collapse, the sub-Euclidean fraction grows monotonically with input dimension, and the per-equation exponentdistributionis preserved under noise, even though individual per-edge values are seed-dependent and the signal is relative to a fixed initialisation. Our robustness claims concern measurement noise, not worst-case perturbations, which lie outside the symbolic-regression setting; the rigorous object for the latter is the per-edge Lipschitz certificate ofSection3.4, and empirically the worst noise-degrader (the spline) carries the larger Lipschitz constant under noise—a bounded mechanism rather than a uniform law. We also do not claim reduced spectral bias: our edges are global rather than local, so the Gram conditioning grows with basis size and the spline-locality property does not transfer.

Our approach has two limitations, each suggesting a concrete future-work direction. First, the per-edge form has a capacity ceiling that fixed-basis KANs close asnngrows; we plan to mitigate this by composing geometry-constrained edges with a small learnable basis component, retaining the geometric prior while expanding local expressive power, and by exploring deeper geometry-constrained compositions. Second, our evaluation covers symbolic regression and tabular data only; we will scale geometry-constrained KANs to higher-dimensional modalities (sequence and image data) by using them as drop-in replacements for MLP heads inside standard backbones. Beyond these, we plan to characterise KAN and MLP function classes through the variation-norm space[2]to formalise the differences observed empirically, and to develop a more structured combination of thetanh\tanhandℓp\ell^{p}branches (for instance tied-affine, gated, or regime-adaptive) so that each branch contributes where its empirical strengths lie, rather than summing both with independent affines.

The contribution is orthogonal to the choice of basis. Casting edge activations as gradients of convex potentials makes existing activations special cases of one construction rather than competing bases, and promoting the exponent to a learnable per-edge parameter makes geometry itself the adaptable degree of freedom—so the learnedppis a geometric readout of the target, not a tuned knob. We do not position this against deep architectures on their own ground: an MLP matches these targets only at2020–30×30\timesthe parameters, and the ResNet readout marks the boundary of the regime rather than a contest we entered. The claim is a design principle at an operating point—at comparable per-edge budget, under measurement noise, and at smallnn, it is the induced geometry, not raw expressivity, that governs what is learned well, and at that budget learned geometry beats both generic bases and generic parametric activations. What we offer is a principle, an interpretable mechanism, and a regime where learned geometry is the right tool—not a state-of-the-art benchmark number.

References

• [1]Alireza Afzal Aghaei.fKAN: Fractional Kolmogorov–Arnold networks with trainable Jacobi basis functions.Neurocomputing, 623:129414, 2025.
• [2]Francis Bach.Breaking the Curse of Dimensionality with Convex Neural Networks.Journal of Machine Learning Research, 18(19):1–53, 2017.
• [3]Heinz H. Bauschke and Patrick L. Combettes.Convex Analysis and Monotone Operator Theory in Hilbert Spaces.CMS Books in Mathematics. Springer, 2nd edition, 2017.
• [4]Nicolas Boullé, Yuji Nakatsukasa, and Alex Townsend.Rational neural networks.InAdvances in Neural Information Processing Systems (NeurIPS), 2020.
• [5]Zavareh Bozorgasl and Hao Chen.Wav-KAN: Wavelet Kolmogorov-Arnold networks.arXiv preprint arXiv:2405.12832, 2024.
• [6]Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio.Learning phrase representations using RNN encoder–decoder for statistical machine translation.InProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734, Doha, Qatar, October 2014. Association for Computational Linguistics.
• [7]Ioana Ciorănescu.Geometry of Banach Spaces, Duality Mappings and Nonlinear Problems, volume 62 ofMathematics and Its Applications.Kluwer Academic Publishers, Dordrecht, 1990.
• [8]G. Cybenko.Approximation by superpositions of a sigmoidal function.Mathematics of Control, Signals and Systems, 2(4):303–314, 1989.
• [9]S. Hochreiter and J. Schmidhuber.Long short-term memory.Neural Computation, 9(8):1735–1780, 1997.
• [10]K. Hornik, M. Stinchcombe, and H. White.Multilayer feedforward networks are universal approximators.Neural Networks, 2(5):359–366, 1989.
• [11]Ameya D. Jagtap, Kenji Kawaguchi, and George Em Karniadakis.Adaptive activation functions accelerate convergence in deep and physics-informed neural networks.Journal of Computational Physics, 404:109136, 2020.
• [12]Diederik P. Kingma and Jimmy Ba.Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014.
• [13]Yann LeCun, Léon Bottou, Genevieve B. Orr, and Klaus-Robert Müller.Efficient BackProp.InNeural Networks: Tricks of the Trade, Lecture Notes in Computer Science, pages 9–50. Springer-Verlag, Berlin, Heidelberg, 1998.
• [14]Ziyao Li.Kolmogorov-Arnold networks are radial basis function networks.arXiv preprint arXiv:2405.06721, 2024.
• [15]Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljacic, Thomas Y. Hou, and Max Tegmark.KAN: Kolmogorov–Arnold networks.InThe Thirteenth International Conference on Learning Representations (ICLR), 2025.
• [16]Franco Manessi and Alessandro Rozza.Learning combinations of activation functions.InInternational Conference on Pattern Recognition (ICPR), 2018.
• [17]R. E. Megginson.An Introduction to Banach Space Theory, volume 183 ofGraduate Texts in Mathematics.Springer, 1998.
• [18]Alejandro Molina, Patrick Schramowski, and Kristian Kersting.Padé activation units: End-to-end learning of flexible activation functions in deep networks.InInternational Conference on Learning Representations (ICLR), 2020.
• [19]Joseph D Romano, Trang T Le, William La Cava, John T Gregg, Daniel J Goldberg, Praneel Chakraborty, Natasha L Ray, Daniel Himmelstein, Weixuan Fu, and Jason H Moore.PMLB v1.0: an open-source dataset collection for benchmarking machine learning methods.Bioinformatics, 38(3):878–880, 2022.
• [20]Sidharth SS, Keerthana AR, Gokul R, and Anas KP.Chebyshev polynomial-based Kolmogorov-Arnold networks: An efficient architecture for nonlinear function approximation.arXiv preprint arXiv:2405.07200, 2024.
• [21]Silviu-Marian Udrescu and Max Tegmark.AI Feynman: a physics-inspired method for symbolic regression.arXiv preprint arXiv:1905.11481, 2020.
• [22]Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Wei Wang, Xiping Hu, and Edith Ngai.Enhancing graph collaborative filtering with FourierKAN feature transformation.arXiv preprint arXiv:2406.01034, 2025.

Appendix

This document collects the exhaustive empirical results and ablations referenced from the main paper. Per-equation breakdowns and full sweeps for every experiment in the main paper are reported here. Section labels are reused from the main paper where natural; figures and tables are numbered fresh.

Appendix APreregistered Spline–Adam sweep

The Spline†configuration in the main paper (Adam,10,00010{,}000steps,lr=10−2\text{lr}=10^{-2},λ=0\lambda=0) was selectedbeforerunning the 18-equation ablation, by a preliminary sweep on two held-out equations (I.12.1, 2D polynomial, andI.34.14, 3D power). Table10reports the sweep results.

Table 10:Preregistered Spline–Adam sweep on two held-out equations (5 seeds each). Step count and learning rate for the main run (bold) selected before observing the 18-equation results.At 5,000 steps, the best learning rate (lr=3×10−3\text{lr}=3\times 10^{-3}) leaves the harder 3D equation a factor of 2 short of converged. Doubling to 10,000 steps atlr=10−2\text{lr}=10^{-2}closes that gap. Larger step counts were not explored beyond 10,000 because the per-equation cost (93 s/config) already exceeds the L-BFGS baseline’s cost (34.7 s/config) by2.7×2.7\times.

Per-equation clean-data ratio.

On the 18-equation clean baseline, the Spline†/ Spline-LBFGS ratio is highly equation-dependent. Adam matches or beats L-BFGS on harder equations (I.34.10:0.28×0.28\times;I.8.14:0.55×0.55\times;III.17.37:0.80×0.80\times;III.14.14:0.83×0.83\times) and loses on simple 2D polynomials (I.12.1:3.81×3.81\times;I.14.4:3.49×3.49\times;I.25.13:3.62×3.62\times). The>3×>3\timesgap is confined to the 2–3 simplest 2D polynomial equations; on the remaining 15 equations the ratio is at or below2×2\times, with 8 of 18 showing near-parity (⩽1.2×\leqslant 1.2\times).

Appendix BArchitectures and parameter counts

All KAN models use architecture[d,K,1][d,K,1]withK=5K=5ford≤2d\leq 2,K=4K=4ford∈{3,4}d\in\{3,4\}, andK=3K=3ford≥5d\geq 5. Per-edge parameter counts followTable1: Tanh-KAN (44/edge),ℓp\ell^{p}-KAN (55/edge), Banach-KAN (88/edge), Cheby-KAN (degree44,77/edge withtanh\tanhnormalisation), Spline G=3 (k=3k=3,G+k+1=7G+k+1=7control points per edge), and a5050-unit ReLU MLP. Total parameter counts per model and per input dimension are summarised in Table11. Counted as trainable weights, Banach-KAN matches the pykan spline at88per edge (Cheby-KAN’s77is cheaper), so we make no parameter-efficiency claim over the spline; the fairness of the comparison rests instead on the matched per-edge budget. The MLP, by contrast, uses2020–30×30\timesmore parameters than any KAN atd≥4d\geq 4.

Table 11:Parameter counts per model by input dimension.#### Wins by input dimension.

Stratifying the 50-equation clean baseline by input dimension shows a clean regime split (Table12). Spline G=3 dominates at low dimensions (7/77/7at22D,8/178/17at33D); atd≥4d\geq 4the wins split among Banach-KAN, Cheby-KAN, and the MLP, with no single model dominating. The MLP becomes competitive at55D (4/114/11wins) at the cost of2020–30×30\timesmore parameters than the KAN models.

Table 12:Optimal geometry by input dimension (5050equations, clean data). Entries are equations won.

Compute resources.

All experiments run on CPU only (no GPU). On a single2222-core node, the supplementary reproducer’s Core 18 single-seed clean baseline (108108trainings,5,0005{,}000Adam steps each) completes in about7575minutes of wall-clock time and consumes6161CPU-minutes in total. Per-training wall-clock by model is summarised in Table13; spline LBFGS dominates the budget. The full paper’s experimental matrix (50-equation clean baseline, noise sweep, small-sample sweep, dissection and fixed-ppablations, five seeds) comprises roughly7,0007{,}000trainings and was run as PBS array jobs on a6464-core node with7272-hour walltime, approximately6060CPU-hours of effective compute.

Table 13:Per-training wall-clock (seconds) on a single2222-core node, Core 18 equations, single seed,5,0005{,}000Adam steps. Statistics over1818equations per model.

Appendix CSpline regularisation under noise: CV vs oracleλ\lambda

We evaluate Spline G=3 with L1-entropy regularisation atλ∈{0,10−3,10−2,10−1}\lambda\in\{0,10^{-3},10^{-2},10^{-1}\}across noise levels (Table14). The optimalλ\lambdascales with noise: no regularisation for clean data, light regularisation (λ=10−3\lambda=10^{-3}) atσ=0.1\sigma=0.1, and moderate regularisation (λ=10−2\lambda=10^{-2}) fromσ=0.3\sigma=0.3onward.

Table 14:Median NRMSE across 18 equations for eachλ\lambda. Bestλ\lambdaper row inbold.#### CV vs oracle.

CV-selectedλ\lambda(5-fold cross-validation on the training set) tracks the oracle-tunedλ\lambda(bestλ\lambdaper equation×\timesnoise) within 3% at every noise level. The gap between regularised spline and geometry-constrained KANs is therefore not an oracle-tuning artefact.

Win counts: best regularised spline vs geometry-constrained.

Even with the bestλ\lambdaper equation (oracle), the regularised spline still loses the per-equation race onceσ⩾0.1\sigma\geqslant 0.1(Table15).

Table 15:Per-equation wins against oracle-tuned regularised spline (18 equations).

Appendix DBanacha2=0a_{2}=0ablation

To verify that Banach-KAN’s dominance is not driven entirely by thetanh\tanhcomponent, we freezea2=0a_{2}=0to disable theJpJ_{p}branch while keeping thetanh\tanhbranch trainable (denoted Banacha2=0{}_{a_{2}=0}).

Table 16:JpJ_{p}branch ablation: Banach-KAN witha2a_{2}frozen at 0 (tanh-only) vs. full Banach-KAN, by input dimension (18 core equations, clean data, median NRMSE over 5 seeds).The median degradation is1.91×1.91\times; full Banach-KAN beats the ablated variant onall18 equations by at least 10%, with the largest single-equation gap reaching5.00×5.00\timeson the trigonometric equationIII.17.37. The degradation is present at every dimension, with the largest ratios at 2D (2.74×2.74\times) and 5D (2.30×2.30\times): theJpJ_{p}branch is not redundant at any scale.

Banach with fixedp=2p=2.

Within the Banach-KAN family, the analogue ablation freezesp=2p=2(disabling geometry learning while keeping both branches active). The learned-ppBanach beats the fixed-p=2p=2Banach on 15/18 clean and 14/18 noisy equations; the fixed wins are marginal (Capacitor, Doppler, Barometric).

Appendix EPer-equation noise tables

We report the median NRMSE per equation per noise level for all 18 core benchmark equations. Seeds: 1729–1733. Best per row inbold.

E.1σ=0\sigma=0(clean)

Per-equation median NRMSE on the 18 core equations atσ=0\sigma=0,n=500n=500, across55seeds. Best of the four models inbold; Spline G=3 takes10/1810/18wins, Banach-KAN8/188/18, while Tanh-KAN andℓp\ell^{p}-KAN take none on the clean benchmark. The pattern is consistent with theTable3reading: the spline takes localised wins (sharp where its grid resolution matches), while Banach is uniformly competent and wins the average rank.

Table 17:Clean-data NRMSE per equation (18 core equations).

E.2σ∈{0.1,0.3,0.5,1.0}\sigma\in\{0.1,0.3,0.5,1.0\}

Aggregate medians by noise level (Table18); winner counts shift from Banach-led (lowσ\sigma) toℓp\ell^{p}-led (highσ\sigma), confirming the geometry transition.

Table 18:Aggregate noise-by-model median NRMSE (18 equations, 5 seeds). Best per row inbold.#### Win counts per noise level.

Within the geometry-constrained family the leader transitions from Banach toℓp\ell^{p}asσ\sigmagrows (Table19).

Table 19:Number of equations (out of 18) where each model achieves the lowest median NRMSE.

Degradation analysis.

The ratio of median NRMSE atσ=1.0\sigma=1.0toσ=0\sigma=0separates fixed-basis from learnable-geometry models cleanly (Table20).

Table 20:Degradation factor (NRMSE atσ=1.0\sigma=1.0/ NRMSE atσ=0\sigma=0).ℓp\ell^{p}-KAN and Tanh-KAN degrade least (33–4×4\times); Spline G=3 degrades most (21.6×21.6\times). All spline variants converge to NRMSE around0.750.75–0.800.80atσ=1.0\sigma=1.0, rendering them essentially useless.

Appendix FPer-equation small-sample tables

Table 21:Aggregate small-sample median NRMSE (σ=0\sigma=0, 18 core equations, 5 seeds). Best per row inbold.Table 22:Number of equations (out of 18) where each model wins per sample size atσ=0\sigma=0(across88models including spline grids G=5,10,20).Banach-KAN takes the most wins at everyn≤500n\leq 500and remains close to the front atn=1000n=1000. Spline G=3 increases its win count monotonically withnn(3→73\to 7), consistent with basis-based KANs needing more samples to identify their per-edge parameters; the larger spline grids (G=5, G=10, G=20) win rarely at anynnin this range. Geometry-constrained models (Banach +ℓp\ell^{p}) and the small spline (G=3) together account for the bulk of wins at every sample size.

Per-equation best model.

Table23reports the winning model for each equation at three representative sample sizes (underσ=0\sigma=0).

Table 23:Per-equation best model in the clean small-sample regime (σ=0\sigma=0).

Appendix GArchitecture dissection per equation

The 3-way race between Tanh,ℓp\ell^{p}, and Banach across regimes (Table24) shows that the conjunction story holds equation by equation, not just on aggregate.

Table 24:Dissection winner per equation by regime.B= Banach,L=ℓp\ell^{p},T= Tanh; columns are clean (σ=0\sigma=0,n=500n=500) and noisy (σ=0.3\sigma=0.3,n=500n=500).In the clean regime, Banach wins18/1818/18; under noise (σ=0.3\sigma=0.3) Banach wins1010,ℓp\ell^{p}wins66, and Tanh wins22. The pattern matches the design intuition: power-law geometry is most useful when label information is weak, while the boundedtanh\tanhbranch is most useful when fine compositional structure must be tracked.

Appendix HLearned geometry: full distributional analysis

Sub-Euclidean fraction by dimension.

Across4,4654{,}465Banach-KAN edges (50 equations×\times5 seeds), the fraction of edges withp<2p<2scales monotonically with input dimension (Figure5and Table25).

Figure 5:Distribution of learned exponentsppacross4,4654{,}465Banach-KAN edges, broken out by input dimension.Table 25:Fraction of edges withp<2p<2and edge counts by dimension.

Within-run specialisation.

In 77% of training runs the within-run standard deviation of learnedppacross edges exceeds 0.5, confirming that distinct edges converge to distinct geometries rather than collapsing to a shared value. By equation type, hyperbolic (31%), logarithmic (28%), and trigonometric (26%) equations push edges towardp<2p<2, while polynomial (11%) and power (9%) equations keep most edges above 2.

Mean learnedppby equation type.

Aggregate means obscure the within-type spread documented above (Table26).

Table 26:Learned exponentppaggregated by equation type (Banach-KAN, 50 equations, clean data).

Stability under noise.

Mean learnedppacross 18 equations is stable asσ\sigmagrows (Table27); the standard deviation widens only slightly.

Table 27:Mean learnedppacross 18 equations at each noise level (with std across equations).

Fixed vs learnedpp(full per-equation, clean).

Forℓp\ell^{p}-KAN with frozenppvs learnedppon1818equations under clean data, learnedppwins17/1817/18outright. The single fixed-ppwin lands onI.29.16 Cosines, wherep=3.0p=3.0achieves0.2690.269vs. learned at0.2830.283(within5%5\%).

Table 28:ℓp\ell^{p}-KAN fixed-ppvs learned-ppNRMSE per equation (clean,n=500n=500). Best per row inbold.The fixedp=2p=2column (Euclidean) is consistently the worst choice, confirming that the departure from Hilbert geometry is beneficial.

Appendix IPendulum interpretability case study

As an illustrative interpretability experiment, we consider the nonlinear correction to the small-angle pendulum period. WithT=4​L/g​K​(sin2⁡(θ0/2))T=4\sqrt{L/g}\,K(\sin^{2}(\theta_{0}/2))andT0=2​π​L/gT_{0}=2\pi\sqrt{L/g}, the correction factorR⁡(θ0)=T/T0=(2/π)​K​(sin2⁡(θ0/2))R(\theta_{0})=T/T_{0}=(2/\pi)K(\sin^{2}(\theta_{0}/2))is smooth, monotone, and has a logarithmic singularity asθ0→π\theta_{0}\to\pi.

Visual comparison: spline vs.JpJ_{p}on the pendulum target.

Figure6contrasts how a small B-spline stack and a smallJpJ_{p}-stack approximateR⁡(θ0)R(\theta_{0})when trained on[0.01,2.0][0.01,2.0]atn=500n=500and asked to extrapolate on(2.0,2.8](2.0,2.8]toward the singularity atθ0=π\theta_{0}=\pi. The spline (a local basis tied to its grid) collapses past the training boundary because the basis carries no information about behaviour outside the grid; theJpJ_{p}-stack (a global functional form parameterised bypp) extrapolates with the correct power-law shape because the activation itself encodes the singular geometry. The same contrast appears at a cuspf⁡(x)=sign⁡(x)​|x|0.7f(x)=\operatorname{sign}(x)|x|^{0.7}, where a singleJpJ_{p}edge with learnedp≈1.7p\approx 1.7reproduces the cusp exactly while the spline rounds it. The figure uses parameter-matched pedagogical implementations (a few-edge spline stack and a few-edgeJpJ_{p}stack) rather than the full pykan / Banach-KAN architectures of Tables29–30, so the absolute extrapolation NRMSE values shown in the legend are larger than those reported in the tables; the qualitative gap between local and global parameterisations is the point.

Figure 6:Two regularity-mismatch failure modes for fixed bases (parameter-matched pedagogical models; train[0.01,2.0][0.01,2.0]atn=500n=500, extrapolate on(2.0,2.8](2.0,2.8]).Left: pendulum correctionR⁡(θ0)R(\theta_{0})with a log singularity atθ0=π\theta_{0}=\pi. The spline (blue) extrapolates flat past the training interval; theJpJ_{p}-stack (red) extrapolates with the correct power-law shape. Extrapolation NRMSE is reported in the legend; absolute values exceed those in Table30because of the smaller per-edge parameter budget used here for visual clarity, but the relative ordering matches the full experiments.Right: algebraic cuspf⁡(x)=sign⁡(x)​|x|0.7f(x)=\operatorname{sign}(x)|x|^{0.7}. A singleJpJ_{p}edge with learnedp≈1.7p\approx 1.7recovers the cusp; a same-budget spline cannot.

Setup.

Two formulations: 1D (θ0→R\theta_{0}\to R) and 3D ((L,g,θ0)→T(L,g,\theta_{0})\to T). Three regimes: interpolation on[0.01,2.8][0.01,2.8], extrapolation (train on[0.01,2.0][0.01,2.0], test on[2.0,2.8][2.0,2.8]), and noise on the interpolation regime.

1D interpolation.

ℓp\ell^{p}-KAN dominates at every sample size (Table29), achieving NRMSE0.00120.0012atn=500n=500—a relative error below 0.2%.

Table 29:1D pendulum interpolation NRMSE (median over 5 seeds).

1D extrapolation.

Banach-KAN dominates the out-of-distribution regime, despite training only on[0.01,2.0][0.01,2.0].

Table 30:1D pendulum extrapolation NRMSE (train on[0.01,2.0][0.01,2.0], test on[2.0,2.8][2.0,2.8]).The Tanh and MLP baselines exceed NRMSE 1 (worse than mean prediction) on extrapolation. Spline reaches∼0.7\sim 0.7. Banach’s dual-geometry composition transfers best to the unseen near-singularity region.

Learned exponents.

For Banach-KAN on 1D pendulum, the learnedppacross 5 seeds is2.9±0.12.9\pm 0.1(mean), with individual edges ranging from 2.3 to 4.9. This is consistent with the regularity structure ofK⁡(m)K(m)near the upper training boundary.

Connection toK⁡(m)K(m)singularity.

K⁡(m)∼12​log⁡(4/(1−m))K(m)\sim\tfrac{1}{2}\log(4/(1-m))asm→1m\to 1, so withm=sin2⁡(θ0/2)m=\sin^{2}(\theta_{0}/2)andε=π−θ0\varepsilon=\pi-\theta_{0},

1−m=cos2⁡(θ0/2)∼ε2/4,K⁡(m)∼log⁡(4/ε),R⁡(θ0)∼(2/π)​log⁡(4/ε).1-m=\cos^{2}(\theta_{0}/2)\sim\varepsilon^{2}/4,\qquad K(m)\sim\log(4/\varepsilon),\qquad R(\theta_{0})\sim(2/\pi)\log(4/\varepsilon).Derivatives nearθ0=π\theta_{0}=\pi:R′​(θ0)∼(1/π)​log⁡(4/ε)/εR^{\prime}(\theta_{0})\sim(1/\pi)\log(4/\varepsilon)/\varepsilonandR′′​(θ0)∼(1/π)​(1+log⁡(4/ε))/ε2R^{\prime\prime}(\theta_{0})\sim(1/\pi)(1+\log(4/\varepsilon))/\varepsilon^{2}. At the upper training boundaryθ0=2.8\theta_{0}=2.8,ε=0.34\varepsilon=0.34,R′′∼9.5R^{\prime\prime}\sim 9.5—bounded but large. AlthoughRRisC∞C^{\infty}on[0.01,2.8][0.01,2.8], derivatives scale asε−k​log⁡(1/ε)\varepsilon^{-k}\log(1/\varepsilon), controlled by the distance to the singularity. The local Hölder-like exponent ofRRnear the boundary is encoded in this distance-to-singularity structure. The activationJp​(z)=sign⁡(z)​|z|p−1J_{p}(z)=\operatorname{sign}(z)|z|^{p-1}has nonlinearity orderp−1p-1. ForRR’s local behaviour (log⁡(1/ε)/ε\log(1/\varepsilon)/\varepsilon, a composite of log and inverse-power), the optimalppwould satisfyp−1∼αp-1\sim\alphafor some effective regularity exponentα\alphathat absorbs the log factor; the observedp≈2.9p\approx 2.9is consistent with this picture. We frame the precise approximation-theoretic derivation as an open problem.

Appendix JPMLB tabular benchmark: additional notes

The per-dataset table is reported inTable9. Two additional observations not in the main paper:

• •Learnedppclusters around2.52.5–3.03.0across PMLB datasets, with the environmental dataset pushing higher (p≈3.9p\approx 3.9), consistent with smoother target functions favouring higher-order polynomial growth.
• •The dominance of Tanh-KAN on tabular data suggests Orlicz geometry is sufficient when compositional structure is weak;ℓp\ell^{p}and hybrid geometries provide their largest advantages in structured scientific settings.

Appendix KMemorisation and edge-expressivity toy experiments

To complement the symbolic regression results, four toy memorisation tasks evaluate noise robustness on synthetic targets:exp⁡(sin⁡x1+x22)\exp(\sin x_{1}+x_{2}^{2}),J0​(20​x)J_{0}(20x)(Bessel),x1​x2x_{1}x_{2}(multiplication), and a 4D compositional target. Across these tasks Tanh-KAN, Banach-KAN, and Spline-KAN show the same pattern as on Feynman:

• •No universal winner across all four; the choice depends on smoothness and dimensionality.
• •Banach-KAN is the best all-rounder at moderate-to-large data sizes.
• •Tanh-KAN outperforms Spline on noisy 2D and 3D targets, consistent with the implicit-regularisation reading.

The edge-expressivity ablation fits a single learnable activation to four univariate targets (sigmoid, cubic, scaled BesselJ0J_{0}, and an exponential). Banach-KAN reproduces all four (NRMSE<0.04<0.04on each), whileℓp\ell^{p}-KAN succeeds on the two power-law-shaped targets and underperforms on the bounded sigmoid—providing a geometric explanation for the conjunction story: thetanh\tanhbranch is needed precisely when the target saturates.

Appendix LFull benchmark (50 equations)

The first1818equations form the core subset used for noise, small-sample, dissection, and fixed-ppablations; all5050are used for the clean baseline. Of the5050equations,4040are taken from the AI Feynman benchmark[21]with the original equation IDs preserved (so that the formula at anyI.#.#/II.#.#/III.#.#entry matches the corresponding Feynman entry verbatim); the remaining1010entries (prefixedF.) are synthetic stress-test targets we constructed to broaden coverage of the polynomial, trigonometric, exponential, logarithmic, hyperbolic and compositional types in33–66D.

Table 31:All 50 benchmark equations. Entries with IDsI.#.#,II.#.#,III.#.#are taken from[21]; entries prefixedF.are synthetic stress-test targets we constructed for this work. “Core” denotes membership in the 18-equation subset.#IDNameTypeddFormulaCore1I.12.1Coulomb (linear)poly2μ​N\mu\,N✓\checkmark2I.14.4Spring energypoly212​k​x2\tfrac{1}{2}kx^{2}✓\checkmark3I.25.13Capacitor voltagerat2q/Cq/C✓\checkmark4I.34.14Relativistic Dopplerpow3ω​(1+v/c)/(1−v/c)\omega\sqrt{(1+v/c)/(1-v/c)}✓\checkmark5I.14.3Gravitational PEpoly3m​g​zmgz✓\checkmark6I.18.12Torquetrig3r​F​sin⁡θrF\sin\theta✓\checkmark7I.34.10Doppler effectrat3ω0/(1−v/c)\omega_{0}/(1-v/c)✓\checkmark8I.47.23Speed of soundpow3γ​p/ρ\sqrt{\gamma p/\rho}✓\checkmark9III.17.37Angular distributiontrig3β⁡(1+α​cos⁡θ)\beta(1+\alpha\cos\theta)✓\checkmark10I.8.14Euclidean distancecomp4(x2−x1)2+(y2−y1)2\sqrt{(x_{2}-x_{1})^{2}+(y_{2}-y_{1})^{2}}✓\checkmark11I.13.4Kinetic energypoly412​m​(v2+u2+w2)\tfrac{1}{2}m(v^{2}+u^{2}+w^{2})✓\checkmark12I.29.16Law of cosinescomp4x12+x22−2​x1​x2​cos⁡(t1−t2)\sqrt{x_{1}^{2}+x_{2}^{2}-2x_{1}x_{2}\cos(t_{1}-t_{2})}✓\checkmark13III.4.32Bose–Einsteinexp41/(exp⁡(h​ω/2​π​kB​T)−1)1/(\exp(h\omega/2\pi k_{B}T)-1)✓\checkmark14I.12.11Lorentz forcetrig5q⁡(Ef+B​v​sin⁡θ)q(E_{f}+Bv\sin\theta)✓\checkmark15I.44.4Entropy changelog5n​kB​T​ln⁡(V2/V1)nk_{B}T\ln(V_{2}/V_{1})✓\checkmark16II.35.21Magnetisationhyp5nρ​μ​tanh⁡(μ​B/kB​T)n_{\rho}\mu\tanh(\mu B/k_{B}T)✓\checkmark17III.14.14Diode equationexp5I0​(exp⁡(q​V/kB​T)−1)I_{0}(\exp(qV/k_{B}T)-1)✓\checkmark18I.40.1Barometric formulaexp6n0exp(−mgx/kBT)n_{0}\exp(-mgx/k_{B}T)✓\checkmark19II.6.15aDipole fieldtrig3p​cos⁡θ/r2p\cos\theta/r^{2}20I.12.4Electric fieldrat2q/r2q/r^{2}21I.10.7Relativistic masspow2m0/1−β2m_{0}/\sqrt{1-\beta^{2}}22I.6.20aNormal distributionexp2e−θ2/2σ2/(2​πσ)e^{-\theta^{2}/2\sigma^{2}}/(\sqrt{2\pi}\sigma)23F.43Velocity correctionpoly3v+u+α​v​uv+u+\alpha vu24I.16.6Velocity additionrat3(u+v)/(1+u​v/c2)(u+v)/(1+uv/c^{2})25I.27.6Thin lensrat31/(1/d1+n/d2)1/(1/d_{1}+n/d_{2})26II.2.42Heat conductionrat3κ​Δ​T/d\kappa\,\Delta T/d27II.15.4Magnetic PEtrig3μ​B​cos⁡θ\mu B\cos\theta28III.15.12Tight bindingtrig32​U​(1−cos⁡(k​d))2U(1-\cos(kd))29I.18.16Angular momentumtrig4m​r​v​sin⁡θmrv\sin\theta30II.11.17Density fluctuationtrig4n0​(1+pd​cos⁡θ/kB​T)n_{0}(1+p_{d}\cos\theta/k_{B}T)31I.37.4Interferencecomp3I1+I2+2​I1​I2​cos⁡δI_{1}+I_{2}+2\sqrt{I_{1}I_{2}}\cos\delta32I.26.2Snell’s lawtrig2arcsin⁡(n​sin⁡θ)\arcsin(n\sin\theta)33F.33Mixed trig producttrig5a​b​c​sin⁡θ​cos⁡ϕabc\sin\theta\cos\phi34F.34Phase-shift producttrig5a​b​sin⁡(ϕ1+ϕ2)​cab\sin(\phi_{1}+\phi_{2})c35III.10.19Magnetic momentcomp4μ​Bx2+By2+Bz2\mu\sqrt{B_{x}^{2}+B_{y}^{2}+B_{z}^{2}}36I.15.3tLorentz timecomp4(t−u​x/c2)/1−u2/c2(t-ux/c^{2})/\sqrt{1-u^{2}/c^{2}}37I.24.6Oscillator energypoly414​m​(ω2+ω02)​x2\tfrac{1}{4}m(\omega^{2}+\omega_{0}^{2})x^{2}38I.18.4Centre of massrat4(m1​r1+m2​r2)/(m1+m2)(m_{1}r_{1}+m_{2}r_{2})/(m_{1}+m_{2})39I.13.12Grav. PE differencerat5G​m1​m2​(1/r2−1/r1)Gm_{1}m_{2}(1/r_{2}-1/r_{1})40F.40Inverse-sqrt fieldcomp4q​k/r12+r22qk/\sqrt{r_{1}^{2}+r_{2}^{2}}41I.39.11Polytropic energyrat3p​V/(γ−1)pV/(\gamma-1)42I.48.20Relativistic energypow3m​c2/1−v2/c2mc^{2}/\sqrt{1-v^{2}/c^{2}}43II.11.27Boltzmann factorexp4n0exp(−μEf/kBT)n_{0}\exp(-\mu E_{f}/k_{B}T)44F.44Fermi–Diracexp41/(exp⁡(E​ℏ/kB​T)+1)1/(\exp(E\hbar/k_{B}T)+1)45F.45Saturating exp.exp3A(1−exp(−t/τ))A(1-\exp(-t/\tau))46F.46Saturating responsehyp5n​μ​tanh⁡(B/kB​T)n\mu\tanh(B/k_{B}T)47F.31Polarisationrat5q​Ef/(m⁡(ω02−ω2))qE_{f}/(m(\omega_{0}^{2}-\omega^{2}))48F.48Nested logarithmlog5n​ln⁡(1+E​V/kB​T)n\ln(1+EV/k_{B}T)49F.496D multiplicativeexp6n0Aexp(−mg/kBT)n_{0}A\exp(-mg/k_{B}T)50III.21.1Current densitypoly5ρ​q​A​v/m\rho qAv/m

Appendix MBudget-matched reproduction of Figure1

Figure1fixes every column at a matched2121-parameter budget;Table32sweeps that budget to{13,21,41,125}\{13,21,41,125\}parameters under the same noisy protocol (σ=0.03\sigma=0.03,N=2000N=2000, seed17291729). On the Heaviside step the geometry-adaptive edge sits at the noise floor and beats every smooth basis at every budget; a Haar wavelet, whose piecewise-constant geometry matches a step exactly, is indistinguishable from it there. On the five-level staircase the geometry edge instead scales with budget (0.1770.177at1313p→0.124\to 0.124at2121p→0.090\to 0.090at125125p, vs. B-spline0.1350.135and RBF0.1410.141at2121p), while Haar loses the staircase (0.1770.177at2121p)—no single fixed geometry is uniformly appropriate.

Table 32:Heaviside step, NRMSE by parameter budget (seed17291729). The geometry-adaptive edge is budget-invariant at the noise floor; smooth bases need budget to catch up.

Appendix NTied-affine exact-duality variant

The tied-affine edge shares one(w,b)(w,b)between thetanh\tanhandJpJ_{p}branches (66parameters/edge) and is the exact∇(Ψ1+Ψ2)\nabla(\Psi_{1}+\Psi_{2})variant; the reported model uses independent affines (Section3.3).Table33compares the three (they compete only against each other, which is why Banach-independent’s small-sample count differs fromTable6). The exact-duality construction is competitive—matching or beating the independent form at moderate noise and degrading more gently—but trails on clean and small-sample data, so we report the independent form.

Table 33:Tied- vs. independent-affine (median NRMSE); “Degr.” is theσ=0→1\sigma{=}0\to 1ratio, “Small-sample” wins out of9090.