When and How to Canonize: A Generalization Perspective

# When and How to Canonize: A Generalization Perspective
Source: [https://arxiv.org/html/2605.11008](https://arxiv.org/html/2605.11008)
Yonatan Sverdlov Technion – Israel Institute of Technology &Benjamin Friedman Technion – Israel Institute of Technology &Snir Hordan Technion – Israel Institute of Technology &Nadav Dym Technion – Israel Institute of TechnologyCoresponding author, Emails:yonatans@campus\.technion\.ac\.il,benjamin\.fri@campus\.technion\.ac\.il,snirhordan@campus\.technion\.ac\.il,nadavdym@technion\.ac\.il\.

###### Abstract

While invariant architectures are standard for processing symmetric data, there is growing interest in achieving invariance by applying group averaging or canonization to non\-invariant backbones\. However, the theoretical generalization properties of these alternative strategies remain poorly understood\. We introduce a theoretical framework to analyze the generalization error of these methods by bounding their covering numbers\. We establish a rigorous generalization hierarchy: the error bounds of canonized models are at best equal to the error bounds of structurally invariant and group\-averaged models, and at worst equal to the bounds of non\-invariant baselines\. Furthermore, we show that there exist “optimal” canonizations which attain the optimal error bounds, and “poor” canonizations which attain the non\-invariant error bounds, and that this depends on the regularity of the canonization\. Finally, applying this framework to permutation groups in point cloud processing, we rigorously prove that the covering number of lexicographical sorting grows exponentially with point cloud dimension, whereas Hilbert curve canonization guarantees polynomial growth\. This provides the first formal theoretical justification for the empirical success of Hilbert curve serialization in state\-of\-the\-art point cloud architectures\. We conclude with experiments that support our theoretical claims\.

Our Code is available at[https://github\.com/yonatansverdlov/Canonization](https://github.com/yonatansverdlov/Canonization)

## 1Introduction

The integration of geometric priors into neural networks has become a cornerstone of modern machine learning, particularly for applications involving graphs, 3D point clouds, and molecular structures\. In these domains, the underlying data often exhibits inherent symmetries, meaning the target function to be learned is invariant to a specific group action\. Exploiting these symmetries is known to improve sample complexity and generalizationElesedy \([2021](https://arxiv.org/html/2605.11008#bib.bib25)\); Brehmeret al\.\([2025](https://arxiv.org/html/2605.11008#bib.bib24)\)\.

While specialized architectures, which are invariant by design, are a common method for processing symmetric data, there is growing interest in achieving invariance by applying generalized group averaging methods to non\-invariant backbones\. This family of methods includes full group averaging, which is theoretically sound but intractable for all but very small groups, group augmentation, which can be seen as an efficient approximation of group averaging, frame averagingPunyet al\.\([2021](https://arxiv.org/html/2605.11008#bib.bib19)\), which enables invariant averaging over subsets of the group, and canonization, which maps every group orbit to a single, consistent "canonical" element\.

Canonization is the most efficient of all generalized group averaging methods, as it only involves processing a single orbit representative\. At the same time, it enjoys the same universal approximation guarantees as full group averagingKabaet al\.\([2023](https://arxiv.org/html/2605.11008#bib.bib20)\)\. These observations have motivated researchers to define canonizations in several diverse domains\. These include sign canonizations for spectral embeddings of graphsMaet al\.\([2023](https://arxiv.org/html/2605.11008#bib.bib18),[2024a](https://arxiv.org/html/2605.11008#bib.bib17)\); Hordanet al\.\([2025](https://arxiv.org/html/2605.11008#bib.bib15)\), canonizations of graphsLinet al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib9)\)and point cloudsKabaet al\.\([2023](https://arxiv.org/html/2605.11008#bib.bib20)\); Bakeret al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib10)\); Friedmann and Werman \([2025](https://arxiv.org/html/2605.11008#bib.bib16)\); Zhouet al\.\([2026](https://arxiv.org/html/2605.11008#bib.bib6)\)including the celebrated Point Transformer V3Wuet al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib21)\), and canonization of the SMILES representation of small moleculesWeiningeret al\.\([1989](https://arxiv.org/html/2605.11008#bib.bib12)\)\.

In this paper, we provide a theoretical study of canonization from the perspective of generalization\. Our focus is both on comparing canonization with other options for enforcing invariance and on comparing the quality of different canonizations\. Our main contributions are threefold:

1. 1\.Canonization vs\. Group Averaging:We prove that the generalization bounds of canonized models are at best equal to the bounds of group\-averaged models, and at worst equal to the bounds of non\-invariant models \(Section[2](https://arxiv.org/html/2605.11008#S2)\)\.
2. 2\.Canonizations and continuity:We show that continuous, isometric canonizations attain the*optimal*generalization error achieved by group averaging, while discontinuous canonizations can lead to*poor*canonizations whose error bounds are identical to non\-invariant models \(Section[3](https://arxiv.org/html/2605.11008#S3)\)\.
3. 3\.Hilbert Canonization:We prove that canonizing point clouds with respect to permutation via lexicographical sorting is inferior to the Hilbert canonizations used in Point Transformer V3Wuet al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib21)\), thus establishing the first theoretical justification for the empirical success of this method \(Section[4](https://arxiv.org/html/2605.11008#S4)\)\.

### 1\.1Related Work

#### Canonization vs\. Group Averaging

It was observed that randomized SMILES can often outperform canonized SMILESArús\-Pouset al\.\([2019](https://arxiv.org/html/2605.11008#bib.bib13)\); Bjerrum \([2017](https://arxiv.org/html/2605.11008#bib.bib11)\)\(see alsoItoet al\.\([2026](https://arxiv.org/html/2605.11008#bib.bib35)\)\)\. Our first contribution gives a theoretical justification for this empirical observation\.

#### Discontinuity

It was shown inDymet al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib22)\); Bakeret al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib10)\)that in many invariant learning scenarios, continuous canonizations are mathematically impossible\. Attempts to mitigate this issue were made inTahmasebi and Jegelka \([2025b](https://arxiv.org/html/2605.11008#bib.bib7)\); Lin and Levie \([2026](https://arxiv.org/html/2605.11008#bib.bib8)\)\. Our second contribution complements these works by showing how discontinuity of canonization hurts the generalization of the canonized model\.

#### Invariance and Generalization

Many papers have considered generalization analysis of invariant modelsVasileiouet al\.\([2025](https://arxiv.org/html/2605.11008#bib.bib4)\); Petrache and Trivedi \([2023](https://arxiv.org/html/2605.11008#bib.bib3)\); Maskeyet al\.\([2025](https://arxiv.org/html/2605.11008#bib.bib36)\); Frankset al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib37)\), but they do not consider canonizations\. To our knowledge, the only paper considering the generalization of canonizations isTahmasebi and Jegelka \([2025a](https://arxiv.org/html/2605.11008#bib.bib5)\)\. They prove that in some setting the*approximation error*of canonized models is lower than group averaged models, and as a result the*expected error*of canonized models can be lower when the model is presented with enough samples\. In contrast, our first conclusion is that the*generalization error bound*of group average models is always lower than the generalization error bound of canonized models\.

### 1\.2Notation

Throughout the paper, we will consider a metric space\(K,ρ\)\(K,\\rho\), whereKKis endowed by the action of a group𝔾\\mathbb\{G\}\. We will assume that the metric is𝔾\\mathbb\{G\}invariant, i\.e\.,ρ​\(g⋅x,g⋅y\)=ρ​\(x,y\)\\rho\(g\\cdot x,g\\cdot y\)=\\rho\(x,y\)for allx,y∈Kx,y\\in Kandg∈𝔾g\\in\\mathbb\{G\}\.

#### Quotient space

We denote the orbit ofx∈Kx\\in Kby\[x\]=\{g⋅x\|g∈G\}\[x\]=\\\{g\\cdot x\|\\quad g\\in G\\\}, and denote the quotient space to be the space of orbitsK/𝔾:=\{\[x\]\|x∈K\}K/\\mathbb\{G\}:=\\\{\[x\]\|x\\in K\\\}\. On this space, we define a metricρ𝔾​\(\[x\],\[y\]\):=ming∈𝔾⁡ρ​\(g⋅x,y\)\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\):=\\min\_\{g\\in\\mathbb\{G\}\}\\rho\(g\\cdot x,y\), where we explicitly assume that the minimum in the definition ofρ𝔾\\rho\_\{\\mathbb\{G\}\}exists, which is the case in most examples of practical interest\. This assumption, together with the𝔾\\mathbb\{G\}invariance ofρ\\rho, guarantees thatρ𝔾\\rho\_\{\\mathbb\{G\}\}is a metric\. We call a triplet\(K,ρ,𝔾\)\(K,\\rho,\\mathbb\{G\}\)satisfying the conditions of the last two paragraphs a*module*\.

Covering numberLet\(K,ρ\)\(K,\\rho\)be a metric space\. We say thatC⊂KC\\subset Kis anϵ\\epsiloncover ofKK, if∀x∈K,∃y∈C:ρ​\(x,y\)≤ϵ\\forall x\\in K,\\exists y\\in C:\\rho\(x,y\)\\leq\\epsilon\. The minimalNNsuch that there exists anϵ\\epsiloncover of cardinalityNNis called theϵ\\epsiloncovering number ofKK, and is denoted by𝒩​\(K,ρ,ϵ\)\\mathcal\{N\}\(K,\\rho,\\epsilon\)\.

#### Canonization

We say thatc:K→Kc:K\\rightarrow Kis a canonization if for everyx∈Kx\\in K, \(i\)c​\(x\)∈\[x\]c\(x\)\\in\[x\], and \(ii\) ally∈\[x\]y\\in\[x\]satisfyc​\(y\)=c​\(x\)c\(y\)=c\(x\)\. We will sometimes use canon\. as an abbreviation for canonization\.

## 2Generalization, Canonization and Group Averaging

Our discussion of the generalization properties of canonizations and other invariant models is based on a popular framework fromXu and Mannor \([2012](https://arxiv.org/html/2605.11008#bib.bib26)\)for studying generalization via metric properties like Lipschitz continuity and covering numbers\. We begin with a short review of this framework\.

Consider a supervised learning problem of learning an unknown functionf:X→Yf:X\\to Yfrom a finite set ofnnsamplesS=\{\(xi,f\(xi\)\|i=1,…,n\)\}S=\\\{\(x\_\{i\},f\(x\_\{i\}\)\|\\quad i=1,\\ldots,n\)\\\}of the function\. Assume that we have some algorithm \(e\.g\., gradient descent\) which, givenSS, returns a hypothesishS:X→Yh\_\{S\}:X\\to Y\. Letℓ:Y×Y→ℝ\\ell:Y\\times Y\\to\\mathbb\{R\}be a function which will be used as a loss function\. The*expected loss*ℒ\\mathcal\{L\}and the*empirical loss*ℒemp\\mathcal\{L\}\_\{\\mathrm\{emp\}\}are defined via

ℒexp​\(hS\)≜𝔼x∼μ​ℓ​\(hS​\(x\),f​\(x\)\),ℒemp​\(hS\)≜1n​∑i=1nℓ​\(hS​\(xi\),f​\(xi\)\)\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\\triangleq\\mathbb\{E\}\_\{x\\sim\\mu\}\\ell\(h\_\{S\}\(x\),f\(x\)\),\\quad\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\\triangleq\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\ell\(h\_\{S\}\(x\_\{i\}\),f\(x\_\{i\}\)\)
The*generalization*error measures how much the empirical error ofhSh\_\{S\}on the training data can deviate from its true loss \(the expected loss\) on the data distribution\. It can be bounded using the Lipschitz constants of the functionhS,f,ℓh\_\{S\},f,\\elland the covering number of the domainXX, as follows:

###### Theorem 2\.1\.

\[Proof in Appendix[A\.1](https://arxiv.org/html/2605.11008#A1.SS1), based onXu and Mannor \([2012](https://arxiv.org/html/2605.11008#bib.bib26)\)\] Let\(X,ρ\)\(X,\\rho\)and\(Y,ρY\)\(Y,\\rho\_\{Y\}\)be metric spaces and assumeXXis compact\. Letf:X→Yf:X\\to Ybe acfc\_\{f\}Lipschitz function, and letℓ:Y×Y→\[0,M\]\\ell:Y\\times Y\\to\[0,M\]be acℓc\_\{\\ell\}Lipschitz function\. Then for anyϵ,δ\>0\\epsilon,\\delta\>0and naturalnn, for a training sample setSSgenerated bynnIID draws from a distributionμ\\mu, with probability of at least1−δ1\-\\delta, we have

\|ℒexp​\(hS\)−ℒemp​\(hS\)\|≤2​cℓ​\(chS\+cf\)​ϵ\+M​2​𝒩​\(X,ρ,ϵ\)​ln⁡2\+2​ln⁡\(1/δ\)n\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\|\\leq 2c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\\epsilon\+M\\sqrt\{\\frac\{2\{\\color\[rgb\]\{0,0,1\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,1\}\\mathcal\{N\}\(X,\\rho,\\epsilon\)\}\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}wherechSc\_\{h\_\{S\}\}denotes the Lipschitz constant ofhSh\_\{S\}\.

We now apply this theorem in the setting where the learned functionffis invariant to a group action\. Our focus is on the setting where we have a non\-invariant Lipschitz backboneh~\\tilde\{h\}, and we consider three types of models: \(i\) models that do not incorporate symmetryh=h~h=\\tilde\{h\}, \(ii\) models that incorporate symmetries by canonization,h=h~∘ch=\\tilde\{h\}\\circ cand \(iii\) models that incorporate symmetries by group averagingh​\(x\)=∫Gh~​\(g​x\)​𝑑gh\(x\)=\\int\_\{G\}\\tilde\{h\}\(gx\)dg\. In this setting, the averaged functionhhis invariant and has the same Lipschitz constant ash~\\tilde\{h\}\(see Proposition[A\.2](https://arxiv.org/html/2605.11008#A1.Thmdefinition2)in the appendix\)\. Our proposition below will be relevant for any Lipschitz invariant model, whether obtained via averaging or via a specialized invariant model:

###### Proposition 2\.2\.

\[Proof in Appendix[A\.1](https://arxiv.org/html/2605.11008#A1.SS1)\] Let\(K,ρ,𝔾\)\(K,\\rho,\\mathbb\{G\}\)be a module\. Let\(Y,ρY\)\(Y,\\rho\_\{Y\}\)be a metric space, and assumeKKis compact\. Letf:K→Yf:K\\to Ybe acfc\_\{f\}Lipschitz functionwhich is𝔾\\mathbb\{G\}invariant, and letℓ:Y×Y→\[0,M\]\\ell:Y\\times Y\\to\[0,M\]be acℓc\_\{\\ell\}Lipschitz function , then for anyϵ,δ\>0\\epsilon,\\delta\>0and naturalnn, for a training sample setSSgenerated bynnIID draws from the distributionμ\\mu, with probability of at least1−δ1\-\\delta, we have

\|ℒexp​\(hS\)−ℒemp​\(hS\)\|≤2​cℓ​\(chS\+cf\)​ϵ\+M​2​𝒩​\(hS\)​ln⁡2\+2​ln⁡\(1/δ\)n\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\|\\leq 2c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\\epsilon\+M\\sqrt\{\\frac\{2\{\\color\[rgb\]\{0,0,1\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,1\}\\mathcal\{N\}\(h\_\{S\}\)\}\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}
where

𝒩​\(hS\)=\{𝒩​\(K,ρ,ϵ\)if​hS​is​chS​Lipschitz𝒩​\(c​\(K\),ρ,ϵ\)if​hS=h~S∘c,h~S​is​chS​Lipschitz, and​c​is a canon\.𝒩​\(K/𝔾,ρ𝔾,ϵ\)if​hS​is​chS​Lipschitz and​𝔾​invariant\{\\color\[rgb\]\{0,0,1\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,1\}\\mathcal\{N\}\(h\_\{S\}\)\}=\\left\\\{\\begin\{array\}\[\]\{lll\}\\mathcal\{N\}\(K,\\rho,\\epsilon\)&\\mbox\{if \}h\_\{S\}\\text\{ is \}c\_\{h\_\{S\}\}\\text\{ Lipschitz \}\\\\ \\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)&\\mbox\{if \}h\_\{S\}=\\tilde\{h\}\_\{S\}\\circ c,\\tilde\{h\}\_\{S\}\\text\{ is \}c\_\{h\_\{S\}\}\\text\{ Lipschitz, and \}c\\text\{ is a canon\.\}\\\\ \\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)&\\mbox\{if \}h\_\{S\}\\text\{ is \}c\_\{h\_\{S\}\}\\text\{ Lipschitz and \}\\mathbb\{G\}\\text\{ invariant \}\\end\{array\}\\right\.\(1\)

###### Proof idea\.

The claim for a Lipschitz functionhSh\_\{S\}without any invariant structure follows immediately from Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)\. For a function of the formhS=h~S∘ch\_\{S\}=\\tilde\{h\}\_\{S\}\\circ c, the claim follows from Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)when replacing the domainKKwithc​\(K\)c\(K\)and the functionhSh\_\{S\}withh~S\\tilde\{h\}\_\{S\}\. The claim forhSh\_\{S\}which is both Lipschitz and invariant follows from the fact that due to invariancehS,fh\_\{S\},fcan be identified with functionh^S,f^:K/𝔾→Y\\hat\{h\}\_\{S\},\\hat\{f\}:K/\\mathbb\{G\}\\to Ysatisfyingh^S​\(\[x\]\)=hS​\(x\),f^​\(\[x\]\)=f​\(x\),∀x∈K\\hat\{h\}\_\{S\}\(\[x\]\)=h\_\{S\}\(x\),\\hat\{f\}\(\[x\]\)=f\(x\),\\forall x\\in K, and the Lipschitz constant remains unchanged after the identification \(see Lemma 20 inSiegelet al\.\([2026](https://arxiv.org/html/2605.11008#bib.bib23)\)\)\. ∎

For each of the three model types considered in Proposition[2\.2](https://arxiv.org/html/2605.11008#S2.Thmdefinition2), a different value of𝒩​\(hS\)\{\\color\[rgb\]\{0,0,1\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,1\}\\mathcal\{N\}\(h\_\{S\}\)\}is provided\. The following simple lemma establishes the relationship between these three values:

###### Proposition 2\.3\.

For any module\(K,ρ,𝔾\)\(K,\\rho,\\mathbb\{G\}\), and anyϵ\>0\\epsilon\>0, the following inequality holds:

𝒩​\(K/𝔾,ρ𝔾,ϵ\)≤𝒩​\(c​\(K\),ρ,ϵ\)≤𝒩​\(K,ρ,ϵ\)\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\\leq\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)\\leq\\mathcal\{N\}\(K,\\rho,\\epsilon\)\(2\)

###### Proof\.

The inequality𝒩​\(c​\(K\),ρ,ϵ\)≤𝒩​\(K,ρ,ϵ\)\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)\\leq\\mathcal\{N\}\(K,\\rho,\\epsilon\)is immediate sincec​\(K\)⊆Kc\(K\)\\subseteq K\. The inequality𝒩​\(K/𝔾,ρ𝔾,ϵ\)≤𝒩​\(c​\(K\),ρ,ϵ\)\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\\leq\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)is due to the quotient mappingq​\(x\)=\[x\]q\(x\)=\[x\]being onto and11\-Lipschitz, and therefore it transforms anyϵ\\epsiloncover ofc​\(K\)c\(K\)to anϵ\\epsiloncover ofK/𝔾K/\\mathbb\{G\}\. ∎

Proposition[2\.2](https://arxiv.org/html/2605.11008#S2.Thmdefinition2)and Proposition[2\.3](https://arxiv.org/html/2605.11008#S2.Thmdefinition3)lead to the following conclusion:

Conclusion 1The generalization bounds of canonized models are at best equal to the bounds of group\-averaged models, and at worst equal to the bounds of non\-invariant models\.

In Section[5](https://arxiv.org/html/2605.11008#S5)we give empirical support for this claim, by providing several experiments in which group averaging outperforms canonized models\. At the same time, we note that group averaging requires more computational resources and is infeasible for large groups\. Thus, Conclusion 1 implies that the improved complexity of the canonized model comes at the price of reduced generalization, but it is not a conclusive statement that canonization should be avoided\.

We conclude this section by showing that the ratio between the largest and smallest covering numbers in \([2](https://arxiv.org/html/2605.11008#S2.E2)\) is bounded by the group’s cardinality, implying that for larger groups the gap between non\-invariant, canonized, and averaged models will be more pronounced:

###### Proposition 2\.4\.

\[See proof in[A\.2](https://arxiv.org/html/2605.11008#A1.SS2)\] Let\(K,ρ,𝔾\)\(K,\\rho,\\mathbb\{G\}\)be a module, and assume𝔾\\mathbb\{G\}is finite\. Then

𝒩​\(K,ρ,ϵ\)≤\|𝔾\|⋅𝒩​\(K/𝔾,ρ𝔾,ϵ\)\.\\mathcal\{N\}\(K,\\rho,\\epsilon\)\\leq\|\\mathbb\{G\}\|\\cdot\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\.

## 3Canonizations and Continuity

In the previous section, we established upper and lower bounds for the covering numbers of canonizations\. Our focus in this section is to study when these bounds are achieved\.

###### Definition 3\.1\.

Let\(K,ρ,𝔾\)\(K,\\rho,\\mathbb\{G\}\)be a module and letϵ\\epsilonbe a positive number\. We say thatccis\(K,ϵ\)\(K,\\epsilon\)*optimal*if it attains the lower bound in \([2](https://arxiv.org/html/2605.11008#S2.E2)\), namely𝒩​\(c​\(K\),ρ,ϵ\)=𝒩​\(K/𝔾,ρ𝔾,ϵ\)\.\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)=\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\.We say thatccis\(K,ϵ\)\(K,\\epsilon\)*poor*if it attains the upper bound in \([2](https://arxiv.org/html/2605.11008#S2.E2)\), namely𝒩​\(c​\(K\),ρ,ϵ\)=𝒩​\(K,ρ,ϵ\)\.\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)=\\mathcal\{N\}\(K,\\rho,\\epsilon\)\.

We will next show conditions for optimal and poor canonizations, and show how these conditions relate to the continuity of the canonization\.

Optimal canonizations:First, we prove that canonizations, which are isometries, are optimal\. We say that a canonizationc:K→Kc:K\\to Kis an*isometry*ifρ​\(c​\(x\),c​\(y\)\)=ρ𝔾​\(\[x\],\[y\]\)\\rho\(c\(x\),c\(y\)\)=\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\)for allx,y∈Kx,y\\in K\.

###### Proposition 3\.2\.

\[proof in[A\.2](https://arxiv.org/html/2605.11008#A1.SS2)\] Any isometric canonizationc:K→Kc:K\\to Kis\(K,ϵ\)\(K,\\epsilon\)optimal for allϵ\>0\\epsilon\>0\.

We next use this claim to give several examples of optimal canonizations:

###### Example 3\.3\.

The following canonizations are isometries \(see proof in[A\.2](https://arxiv.org/html/2605.11008#A1.SS2)\)

1. 1\.The canon\.c​\(x\)=\|x\|c\(x\)=\|x\|with respect to the action of\{−1,1\}\\\{\-1,1\\\}onℝ\\mathbb\{R\}and the standard metric\.
2. 2\.The canon\.sort:ℝn→ℝn\\mathrm\{sort\}:\\mathbb\{R\}^\{n\}\\to\\mathbb\{R\}^\{n\}with respect to the action of permutations and a metricρ\\rhoinduced by appnorm\.
3. 3\.The centralization canon\. defined for point clouds inℝd×n\\mathbb\{R\}^\{d\\times n\}with respect to the action of translation by vectors inℝd\\mathbb\{R\}^\{d\}, with the metricρ\\rhoinduced by the Frobenius norm\.

In Proposition[A\.3](https://arxiv.org/html/2605.11008#A1.Thmdefinition3)in the appendix, we prove that a canonization is isometric if and only if it is11\-Lipschitz in the standard sense\. In particular, this means that an isometric canonization is always continuous\. As a result, in the setting where there can be no continuous canonizations, such as those discussed inDymet al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib22)\); Bakeret al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib10)\), there can be no isometric canonizations\.Poor canonizations:We next show that discontinuities in canonizations can lead to poor canonizations\. This occurs under two conditions: firstly, the canonization is ‘strongly discontinuous’ in the sense that it has\|G\|\|G\|partial limits\. Secondly, the domainKKof the group action is chosen adversarially to exploit the discontinuity of the canonization\.

###### Proposition 3\.4\.

\[proof in[A\.2](https://arxiv.org/html/2605.11008#A1.SS2)\] Let\(V,ρ,𝔾\)\(V,\\rho,\\mathbb\{G\}\)be a module, where𝔾\\mathbb\{G\}is a finite group\. Letccbe a canonization that has\|G\|\|G\|partial limits at a pointx∈Vx\\in V\. Then for any small enoughϵ\>0\\epsilon\>0there exists a𝔾\\mathbb\{G\}invariant setK⊆VK\\subseteq Vsuch that𝒩​\(c​\(K\),ρ,ϵ\)=𝒩​\(K,ρ,ϵ\)=\|𝔾\|⋅𝒩​\(K/𝔾,ρ,ϵ\)\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)=\\mathcal\{N\}\(K,\\rho,\\epsilon\)=\|\\mathbb\{G\}\|\\cdot\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho,\\epsilon\)\.

Based on this proposition, we give two examples of poor canonizations:

###### Example 3\.5\.

Consider the action of𝔾=\{−1,1\}\\mathbb\{G\}=\\\{\-1,1\\\}onℝ\\mathbb\{R\}\. Consider the canonizations

c1​\(t\)=\{\|t\|if​\|t\|\>1/2−\|t\|if​\|t\|≤1/2,c∞​\(t\)=\{\|t\|if​t∈ℚ−\|t\|if​t∉ℚc\_\{1\}\(t\)=\\left\\\{\\begin\{array\}\[\]\{ll\}\|t\|&\\mbox\{if \}\|t\|\>1/2\\\\ \-\|t\|&\\mbox\{if \}\|t\|\\leq 1/2\\end\{array\}\\right\.,\\quad c\_\{\\infty\}\(t\)=\\left\\\{\\begin\{array\}\[\]\{ll\}\|t\|&\\mbox\{if \}t\\in\\mathbb\{Q\}\\\\ \-\|t\|&\\mbox\{if \}t\\not\\in\\mathbb\{Q\}\\end\{array\}\\right\.visualized in Figure[1](https://arxiv.org/html/2605.11008#S3.F1)\. Both of these canonizations are discontinuous and have a point with\|G\|=2\|G\|=2partial limits\. Therefore, by Proposition[3\.4](https://arxiv.org/html/2605.11008#S3.Thmdefinition4)for every small enoughϵ\>0\\epsilon\>0we can choose an adversarial domain for which the canonization is poor\. Forc1c\_\{1\}we can choose for everyϵ∈\(0,1\)\\epsilon\\in\(0,1\)the setK=Bϵ​\(−12\)∪Bϵ​\(12\)K=B\_\{\\epsilon\}\(\-\\frac\{1\}\{2\}\)\\cup B\_\{\\epsilon\}\(\\frac\{1\}\{2\}\)and then𝒩​\(K,ρ,ϵ\)=𝒩​\(c1​\(K\),ρ,ϵ\)=2\\mathcal\{N\}\(K,\\rho,\\epsilon\)=\\mathcal\{N\}\(c\_\{1\}\(K\),\\rho,\\epsilon\)=2and so the canonization is poor\. In the canonizationc∞c\_\{\\infty\}the discontinuities are much more extreme\. In this case, we can choose a non\-adversarial domainK=\[−1,1\]K=\[\-1,1\], and the canonization will be poor for anyϵ\>0\\epsilon\>0because the covering number of a set and its closure are the same, and in our case, the closure ofc∞​\(K\)c\_\{\\infty\}\(K\)will be all ofKK\.

Figure 1:Visualization of three canonizations for the action of𝔾=\{−1,1\}\\mathbb\{G\}=\\\{\-1,1\\\}onℝ\\mathbb\{R\}\. The natural canonizationc​\(x\)=\|x\|c\(x\)=\|x\|, which is shown to be optimal \(Example[3\.3](https://arxiv.org/html/2605.11008#S3.Thmdefinition3)\), and two discontinuous canonizations, which on an appropriate domain are poor \(Example[3\.5](https://arxiv.org/html/2605.11008#S3.Thmdefinition5)\)\.![Refer to caption](https://arxiv.org/html/2605.11008v1/x1.png)###### Example 3\.6\.

Consider the action of the permutation group𝔾=Sn\\mathbb\{G\}=S\_\{n\}on the space of matricesℝd×n\\mathbb\{R\}^\{d\\times n\}by permuting the columns of the matrices\. We call such matrices ‘point clouds’\. We assume thatn,d≥2n,d\\geq 2\(the cased=1d=1sorting is an isometry by Example[3\.3](https://arxiv.org/html/2605.11008#S3.Thmdefinition3)\)\. A natural canonization in this scenario isclexc\_\{\\mathrm\{lex\}\}, which permutes the columns of the matrices so that the first row of the matrix is sorted from small to large, and in the event of ties in the first row, sorts according to the second row, and continues in this fashion until a permutation is uniquely defined\. The canonizationclexc\_\{\\mathrm\{lex\}\}is discontinuous\. Moreover, the canonization hasn\!=\|𝔾\|n\!=\|\\mathbb\{G\}\|partial limits at any matrixM∈ℝd×nM\\in\\mathbb\{R\}^\{d\\times n\}satisfying that the matrix does not contain two identical columns, butM1,1=M1,2=…=M1,nM\_\{1,1\}=M\_\{1,2\}=\\ldots=M\_\{1,n\}\. Thus, according to Proposition[3\.4](https://arxiv.org/html/2605.11008#S3.Thmdefinition4), for every small enoughϵ\>0\\epsilon\>0there exists an adversarial domain for which the canonization is poor\.

Conclusion 2We show that isometric canonizations are optimal, while discontinuous canonizations can yield poor canonizations\. These results give a rigorous justification for the importance of continuous canonizations\.

## 4Case study: Permutation Canonizations

In Example[3\.6](https://arxiv.org/html/2605.11008#S3.Thmdefinition6), we showed that lexicographical sorting is a poor canonization on an adversarial domain\. In this section, we analyze the covering numbers of lexicographical sorting when considering a fixed, ‘natural’ domain of point cloudsK=\[0,1\]d×nK=\[0,1\]^\{d\\times n\}\. Throughout this section, we will consider the action of the group𝔾=Sn\\mathbb\{G\}=S\_\{n\}, and the distance induced by the element\-wise infinity normρ∞​\(X,Y\)=‖X−Y‖∞\\rho\_\{\\infty\}\(X,Y\)=\\\|X\-Y\\\|\_\{\\infty\}\. We will show that \(for fixedϵ,d\\epsilon,d\) the covering number of Lexsort grows exponentially innn, the number of points in the point cloud, while the covering number of the quotient space only grows polynomially innn\. Moreover, we will consider an alternative canonization based on Hilbert curves, and show that its covering numbers also grow only polynomially innn\. The significance of this result is that it gives a theoretical justification for the choice of Point Transformer V3Wuet al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib21)\)to use Hilbert canonizations for point cloud serialization\.

We begin by proving the exponential growth of Lexsort canonizations:

###### Lemma 4\.1\.

\[proof in[A\.3](https://arxiv.org/html/2605.11008#A1.SS3)\] Letk,d,n≥2k,d,n\\geq 2be natural numbers, and setϵ=1/\(2​k\),K=\[0,1\]d×n\\epsilon=1/\(2k\),K=\[0,1\]^\{d\\times n\}and𝔾=Sn\\mathbb\{G\}=S\_\{n\}\. Then the covering number of the lexicographical sorting canonization satisfies

\(12​ϵ\)\(d−1\)⋅n\+1≤𝒩​\(clex​\(K\),ρ∞,ϵ\)\.\\left\(\\frac\{1\}\{2\\epsilon\}\\right\)^\{\(d\-1\)\\cdot n\+1\}\\leq\\mathcal\{N\}\(c\_\{\\mathrm\{lex\}\}\(K\),\\rho\_\{\\infty\},\\epsilon\)\.\(3\)

###### Proof idea\.

The Lexsort canonization is ineffective on the set of its ‘severe discontinuities’\- point clouds whose first row is constant\. These point clouds form a\(d−1\)⋅n\+1\(d\-1\)\\cdot n\+1dimensional hypercube\. The covering number ofclex​\(K\)c\_\{\\mathrm\{lex\}\}\(K\)is upper bounded by the covering number of this hypercube, which is the left\-hand side of \([3](https://arxiv.org/html/2605.11008#S4.E3)\)\. ∎

Next, we prove that for fixedϵ,d\\epsilon,d, the quotient space’s covering number grows polynomially innn:

###### Lemma 4\.2\.

Letn,dn,dbe natural numbers andϵ∈\(0,1\)\\epsilon\\in\(0,1\)\. SetK=\[0,1\]d×nK=\[0,1\]^\{d\\times n\}and𝔾=Sn\\mathbb\{G\}=S\_\{n\}\. Then the covering number of the quotient space satisfies

𝒩​\(K/𝔾,ρ𝔾,ϵ\)≤\(n\+kd−1n\),where​k=⌈12​ϵ⌉\.\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\\leq\\binom\{n\+k^\{d\}\-1\}\{n\},\\text\{ where \}k=\\left\\lceil\\frac\{1\}\{2\\epsilon\}\\right\\rceil\.\(4\)

###### Proof idea\.

Setk=⌈1/\(2​ϵ\)⌉k=\\lceil 1/\(2\\epsilon\)\\rceil\. The setKKcan be covered bykdk^\{d\}hypercubes of diameterkkand radius1/\(2​k\)≤ϵ1/\(2k\)\\leq\\epsilon\. LetCCdenote the collection of centers of these hypercubes, and after applying the quotient mapπ​\(c\)=\[c\]\\pi\(c\)=\[c\], we obtain pointsπ​\(C\)\\pi\(C\)in the quotient space which are anϵ\\epsiloncover ofK/𝔾K/\\mathbb\{G\}\. Note that points inCCwhich are permutations of each other are mapped byπ\\pito the same orbit\. Therefore, the cardinality of the coverπ​\(C\)\\pi\(C\)is the number of equivalence classes with respect to the relation onCCdefined by the group:c1≡c2⇔∃π∈Sn:c1=π​\(c2\)c\_\{1\}\\equiv c\_\{2\}\\iff\\exists\\pi\\in S\_\{n\}:c\_\{1\}=\\pi\(c\_\{2\}\)\. As explained in more detail in the proof, the number of equivalence classes is the combinatorial expression in \([4](https://arxiv.org/html/2605.11008#S4.E4)\)\. ∎

### 4\.1Hilbert Curve Canonization

Finally, we will provide an intuitive explanation of the Hilbert canonization and formulate the claim that its covering number exhibits only polynomial growth with respect to the point cloud dimensionality\. For a full definition of the canonization and proof, see Appendix[A\.4](https://arxiv.org/html/2605.11008#A1.SS4)\.

Figure 2:The Hilbert curvesHmH\_\{m\}ford=2d=2andm=1,2,3m=1,2,3induce an ordered on a grid in\[0,1\]d\[0,1\]^\{d\}with2m​d2^\{md\}points\. On the right we see the discontinuous Lexsort ordering on the grid\.![Refer to caption](https://arxiv.org/html/2605.11008v1/x2.png)The Hilbert curve is a space\-filling curve, mapping the unit interval\[0,1\]\[0,1\]continuously onto a hypercube\[0,1\]d\[0,1\]^\{d\}\. It is defined as a limit of curvesHm:\[0,1\]→\[0,1\]dH\_\{m\}:\[0,1\]\\to\[0,1\]^\{d\}which are not onto but whose image becomes more and more ‘dense’ in\[0,1\]d\[0,1\]^\{d\}asmmgrows, as shown in Figure[2](https://arxiv.org/html/2605.11008#S4.F2)for the case whered=2d=2\. The curvesHmH\_\{m\}can be used to define an order on\[0,1\]d\[0,1\]^\{d\}\(at least on points in the image of the curve\) and thus induce a canonization\. Intuitively, the advantage of this order over the Lexsort ordering is that this ordering is continuous \(see Figure[2](https://arxiv.org/html/2605.11008#S4.F2)\) again\. However, while previous workWuet al\.\([2024](https://arxiv.org/html/2605.11008#bib.bib21)\); Chenet al\.\([2022](https://arxiv.org/html/2605.11008#bib.bib1)\)cited this continuity as justification for Hilbert ordering, a formal justification of why this continuity is important for learning has not yet been provided\. We address this caveat by showing the polynomial growth of the covering number of Hilbert canonizations:

###### Theorem 4\.3\.

Letd,n,m≥2d,n,m\\geq 2be natural numbers, letϵ∈\(0,1\)\\epsilon\\in\(0,1\), and assume thatϵ\>2−m−1\\epsilon\>2^\{\-m\-1\}\. SetK=\[0,1\]d×nK=\[0,1\]^\{d\\times n\}and𝔾=Sn\\mathbb\{G\}=S\_\{n\}\. Then the covering number of the Hilbert canonizationcmc\_\{m\}satisfies

𝒩​\(cm​\(K\),ρ∞,ϵ\)≤\(n\+⌈1/\(2​δ\)⌉−1n\),where​δ=\(ϵ−2−m−1\)d4\\mathcal\{N\}\(c\_\{m\}\(K\),\\rho\_\{\\infty\},\\epsilon\)\\leq\\binom\{n\+\\lceil 1/\(2\\delta\)\\rceil\-1\}\{n\},\\text\{ where \}\\delta=\\frac\{\\left\(\\epsilon\-2^\{\-m\-1\}\\right\)^\{d\}\}\{4\}

###### Proof idea\.

When restricting tonn\-tuples inKKwhich are all in the image ofHmH\_\{m\}, the Hilbert canonization can be written asHm∘sort∘Hm−1H\_\{m\}\\circ\\mathrm\{sort\}\\circ H\_\{m\}^\{\-1\}\. Sincesort\\mathrm\{sort\}\(Proposition[3\.3](https://arxiv.org/html/2605.11008#S3.Thmdefinition3)\) is an isometry, the image ofsort∘Hm−1\\mathrm\{sort\}\\circ H\_\{m\}^\{\-1\}can be covered efficiently\. This cover can then be pushed forward to cover the image ofHm∘sort∘Hm−1H\_\{m\}\\circ\\mathrm\{sort\}\\circ H\_\{m\}^\{\-1\}, using the fact thatHmH\_\{m\}is Holder continuous\. ∎

Theorem[4\.3](https://arxiv.org/html/2605.11008#S4.Thmdefinition3)shows that the covering number of the Hilbert canonization grows at worst polynomially innn, for fixedϵ\>0\\epsilon\>0\. This is in contrast with the exponential growth of the sorting canonization, proving a substantial gap between them\.

Conclusion 3Our analysis proves that the covering number of Hilbert canonizations is far superior to Lexsort canonization, giving a theoretical justification for its superior performance in practice\.

To obtain more intuition for the gap between our bounds for the different covering numbers, in Section[B\.2](https://arxiv.org/html/2605.11008#A2.SS2)in the appendix, we give a table showing these values computed for specific values ofϵ,d,n\\epsilon,d,n\.

## 5Experiments

We conduct several experiments to verify our theoretical insights and their manifestation in practical learning scenarios\. We focus on validating the hierarchy between group averaging, canonization, and non\-invariant models \(Conclusion 1\), and on the hierarchy between Hilbert and Lexsort canonizations \(Conclusion 3\)\.

#### Coverage on ModelNet

In our first experiment, we considered the ModelNetWuet al\.\([2015](https://arxiv.org/html/2605.11008#bib.bib32)\)point cloud classification task\. Our first goal was to see whether the covering number hierarchy between the four different methods considered in Table[B\.2](https://arxiv.org/html/2605.11008#A2.SS2)is preserved when replacing the large domain\[0,1\]3×n\[0,1\]^\{3\\times n\}with the ‘ModelNet point\-cloud domain’KK\.

As we do not have access to the ‘true point cloud’ domainKK, but only to the train and test sets, we do not compute covering numbers, but a related notion we call the coverage\. For given setsTT\(representing the test set\) andRR\(the train set\), we define the coverageϵ​\(R,T\)\\epsilon\(R,T\)to be the smallestϵ\\epsilonsuch thatRRis anϵ\\epsiloncover ofTT\. We compute the coverage by choosing, for each test samplet∈Tt\\in T, the minimum distance fromttto all training samples \(for computational efficiency, we consider only samples with the same label astt\)\. We call this numberqtq\_\{t\}\. The coverage is then given byϵ​\(R,T\)=maxt∈T⁡qt\\epsilon\(R,T\)=\\max\_\{t\\in T\}q\_\{t\}\. We call this measure the coverage or*max coverage*\. We also consider a more robust measure, the*mean coverage*ϵmean​\(R,T\)=mean​\{qt\|t∈Test\}\\epsilon\_\{\\mathrm\{mean\}\}\(R,T\)=\\mathrm\{mean\}\\\{q\_\{t\}\|t\\in\\mathrm\{Test\}\\\}\.

Table 1:Estimates of covering number on ModelNet via mean and max coverage, when using Euclidean distances, Lexsort or Hilbert canonizations, and the group distance\.We compute the max and mean coverage on ModelNet10 and ModelNet40, using four different distances: \(i\) the standard Euclidean distance, the Euclidean distance after applying \(ii\) Lexsort or \(iii\) Hilbert canonizations to the train and test sets, and \(iv\) the quotient metricd𝔾d\_\{\\mathbb\{G\}\}\(in this case, the Wasserstein distance\)\. The results, shown in Table[1](https://arxiv.org/html/2605.11008#S5.T1), show that the theoretical covering number hierarchy we derived is also preserved when considering coverage of real data: the coverage ofd𝔾d\_\{\\mathbb\{G\}\}is the lowest, followed by Hilbert, Lexsort, and finally standard Euclidean distance\.

#### Classification on ModelNet

Next, we consider whether the hierarchy in covering numbers between the different methods translates into a corresponding hierarchy in accuracy and generalization error\. We choose a simple, purely non\-invariant architecture: a Global MLP, and compare the performance of this network \(i\) without canonization, \(ii\) with Lexsort canonization, and \(iii\) with Hilbert canonization \(we cannot implement group averaging here since the group is very large\)\. The results, shown in Table[2](https://arxiv.org/html/2605.11008#S5.T2), reveal that the expected hierarchy is obtained, with Hilbert performing best both on ModelNet40 and ModelNet10, followed by the Lexsort canonization and a plain MLP\. The Generalization error column \(defined as Test acc\.\-Train acc\.\) indicates that this performance gap is due to improved generalization\.

Table 2:On the ModelNet classification task, we compare vanilla MLP architectures in three configurations: \(a\) no canon\. \(b\) Lexsort canon\. and \(c\) Hilbert canon\. As predicted by our theory, the Hilbert canon\. leads to better generalization, which translates into improved overall accuracy\.To further verify the generality of our findings, we also trained the three alternatives on smaller samples from the ModelNet40 dataset\. We find that theHilbert\>Lexsort\>no canon\.\\text\{Hilbert\}\>\\text\{Lexsort\}\>\\text\{no canon\.\}hierarchy is preserved in all experiments, as shown in Table[9](https://arxiv.org/html/2605.11008#A3.T9)in the appendix\. We note that this experiment is only illustrative; the results attained by simple vanilla MLPs, even with Hilbert canonization, are far from state\-of\-the\-art\. In the context of our theory, this can also be explained through the fact that successful models for this task are typically permutation invariant, and hence their generalization is governed by the covering number of the quotient space, which is always better than the covering number of canonizations \(Conclusion 1\)\.

We next conduct further experiments to corroborate conclusion 1, which claims that, in terms of generalization, group averaging is preferable to canonizations, which are in turn preferable to ignoring invariance altogether\.

Group averaging on PCA framesOur first experiment is still with the ModelNet40 classification task, but while the previous experiments investigates canonization w\.r\.t\. permutations, and implicitly relies on the fact that ModelNet models are aligned w\.r\.t rotations, in this experiment we use the DeepsetsZaheeret al\.\([2017](https://arxiv.org/html/2605.11008#bib.bib33)\)model to deal with permutation invariance, and will focus on rotation invariance: we will assume the ModelNet models rotation degree of freedom is initially unknown, and is fixed using the PCA axes as suggested inPunyet al\.\([2021](https://arxiv.org/html/2605.11008#bib.bib19)\)\. Assuming the point cloud singular values are pairwise distinct, this procedure determines the point cloud orientation uniquely up to a\{−1,1\}3\\\{\-1,1\\\}^\{3\}symmetry\.

We consider four ways to deal with this symmetry: \(i\) Pure PCA, where the symmetry is ignored \(ii\) Skewness canonization, where the global sign of thex,yx,yandzzcoordinates is chosen so that their third moment is positive \(iii\) group averaging and \(iv\) group augmentation where in each epoch a random group element is applied to the input\. This can be seen as an approximation of group averaging\. The results of this experiment, detailed in Table[4](https://arxiv.org/html/2605.11008#S5.T4)demonstrate a clear performance hierarchy\. Group averaging achieves the highest overall accuracy, while group augmentation and skewness canonization yield comparable final results, with all three methods significantly outperforming the pure PCA baseline\. We note that, as highlighted in Figure[3](https://arxiv.org/html/2605.11008#A3.F3)in the appendix, the group augmentation approach suffers from a noticeably slower convergence rate compared to the other three methods\.

Group averaging for image classificationNext, we check our approach on image classification\. We take the rotated MNISTLarochelleet al\.\([2007](https://arxiv.org/html/2605.11008#bib.bib34)\)dataset, and consider the action of thep​4p4group, which is the four\-element group generated by9090degree rotations\. We consider 4 approaches: simple CNN, the canonization fromKabaet al\.\([2023](https://arxiv.org/html/2605.11008#bib.bib20)\)with random or learned weights \(named CN\(p4 frozen\)\-CNN and CN\(p4\)\-CNN, respectively\), and a group average over the44elements of the group\. As shown in Table[4](https://arxiv.org/html/2605.11008#S5.T4), the canonizations outperform the simple CNN, but are outperformed by group averaging, as predicted in Conclusion 1\. We also see in the table that the accuracy is correlated with the mean coverage of the different methods\.

Table 3:Test accuracy on ModelNet40 with a DeepSets model with orientation determined by PCA, and sign ambiguity handled via averaging, sign augmentation, canonization, or Pure PCA \(ignoring the ambiguity\)\.
Table 4:Accuracy and mean coverage on Rotated\-MNIST, which is invariant with respect to the groupp​4p4of9090rotations\. We consider a non\-invariant CNN, two canonizations fromKabaet al\.\([2023](https://arxiv.org/html/2605.11008#bib.bib20)\), and group averaging\.

Group Averaging for Spectral EmbeddingsIn the Appendix Section[B](https://arxiv.org/html/2605.11008#A2)we present an additional experiment showing the advantage of averaging over previously proposed canonizations, in the context of sign ambiguity present in spectral embeddings for graph learning\.

## 6Conclusion

In this paper, we establish a methodology for comparing different canonizations with group averaging and other invariant models by studying their covering number\. We show the covering number of canonizations is at best the covering number of the quotient space \(for isometric canonizations\), and at worst the covering number of the original domain \(for certain discontinuous canonizations\)\. These results suggest that group\-averaged models are preferable when computationally feasible\. We also show that the covering number of Hilbert canonizations for point clouds is substantially better than that of Lexsort canonizations\.

#### Limitations and Future Work

A limitation of our analysis is that it relies on upper bounds on the generalization error \(Proposition[2\.2](https://arxiv.org/html/2605.11008#S2.Thmdefinition2)\), which may not be tight\. Nonetheless, our experiments indicate that lower covering numbers do correlate with better generalization and learning\. Looking towards future developments, a central contribution of this work is the demonstration that a rigorous theoretical comparison between competing canonization methods can be established via their covering numbers\. This framework complements the traditional focus on a model’s ability to distinguish between complex symmetric examples, instead highlighting properties that arguably exert a more significant influence on the ultimate success of the learning process\. In future work, we intend to apply our framework to other canonizations and extend it to enable a rigorous treatment of the generalization of frame averaging\.

## 7Acknowledgments

All authors are funded by Israeli Science Foundation grant no\.272/23\. N\.D\. and S\.H\. wish to thank Jonathan W Siegel and Hannah Lawrence for many interesting discussions related to the topics presented in the paper\.

## References

- \[1\]J\. Arús\-Pous, S\. V\. Johansson, O\. Prykhodko, E\. J\. Bjerrum, C\. Tyrchan, J\. Reymond, H\. Chen, and O\. Engkvist\(2019\)Randomized SMILES strings improve the quality of molecular generative models\.Journal of Cheminformatics11\(1\),pp\. 71\.External Links:[Document](https://dx.doi.org/10.1186/s13321-019-0393-0),[Link](https://doi.org/10.1186/s13321-019-0393-0)Cited by:[§B\.1](https://arxiv.org/html/2605.11008#A2.SS1.p1.1),[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px1.p1.1)\.
- \[2\]\(2024\-21–27 Jul\)An explicit frame construction for normalizing 3D point clouds\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 2456–2473\.External Links:[Link](https://proceedings.mlr.press/v235/baker24a.html)Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2605.11008#S1.p3.1),[§3](https://arxiv.org/html/2605.11008#S3.p5.3)\.
- \[3\]E\. J\. Bjerrum\(2017\)SMILES enumeration as data augmentation for neural network modeling of molecules\.ArXivabs/1703\.07076\.External Links:[Link](https://api.semanticscholar.org/CorpusID:11060416)Cited by:[§B\.1](https://arxiv.org/html/2605.11008#A2.SS1.p1.1),[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px1.p1.1)\.
- \[4\]S\. Bobkov and M\. Ledoux\(2019\)One\-dimensional empirical measures, order statistics, and kantorovich transport distances\.Vol\.261,American Mathematical Society\.Cited by:[§A\.2](https://arxiv.org/html/2605.11008#A1.SS2.7.p2.3)\.
- \[5\]J\. Brehmer, S\. Behrends, P\. de Haan, and T\. Cohen\(2025\)Does equivariance matter at scale?\.External Links:2410\.23179,[Link](https://arxiv.org/abs/2410.23179)Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p1.1)\.
- \[6\]W\. Chen, X\. Zhu, G\. Chen, and B\. Yu\(2022\)Efficient point cloud analysis using hilbert curve\.InComputer Vision – ECCV 2022,S\. Avidan, G\. Brostow, M\. Cissé, G\. M\. Farinella, and T\. Hassner \(Eds\.\),Cham,pp\. 730–747\.External Links:ISBN 978\-3\-031\-20086\-1Cited by:[§4\.1](https://arxiv.org/html/2605.11008#S4.SS1.p2.8)\.
- \[7\]N\. Dym, H\. Lawrence, and J\. W\. Siegel\(2024\)Equivariant frames and the impossibility of continuous canonicalization\.InInternational Conference on Machine Learning,pp\. 12228–12267\.Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2605.11008#S3.p5.3)\.
- \[8\]B\. Elesedy\(2021\)Provably strict generalisation benefit for invariance in kernel methods\.InAdvances in Neural Information Processing Systems,A\. Beygelzimer, Y\. Dauphin, P\. Liang, and J\. W\. Vaughan \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=yKdYdQbo22W)Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p1.1)\.
- \[9\]B\. J\. Franks, C\. Morris, A\. Velingker, and F\. Geerts\(2024\)Weisfeiler\-leman at the margin: when more expressivity matters\.InProceedings of the 41st International Conference on Machine Learning,pp\. 13885–13926\.Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px3.p1.1)\.
- \[10\]B\. Friedmann and M\. Werman\(2025\)CanonNet: canonical ordering and curvature learning for point cloud analysis\.External Links:2504\.02763,[Link](https://arxiv.org/abs/2504.02763)Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p3.1)\.
- \[11\]Z\. He and A\. B\. Owen\(2016\)Extensible grids: uniform sampling on a space filling curve\.Journal of the Royal Statistical Society: Series B \(Statistical Methodology\)78\(4\),pp\. 917–931\.Cited by:[§A\.4](https://arxiv.org/html/2605.11008#A1.SS4.p2.14),[§A\.4](https://arxiv.org/html/2605.11008#A1.SS4.p4.4)\.
- \[12\]S\. Hordan, M\. Bechler\-Speicher, G\. Lifshitz, and N\. Dym\(2025\)Spectral graph neural networks are incomplete on graphs with a simple spectrum\.arXiv preprint arXiv:2506\.05530\.Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p3.1)\.
- \[13\]M\. Ito, D\. Koutra, and J\. Wiens\(2026\)Random search neural networks for efficient and expressive graph learning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=2zHrRV4J1O)Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px1.p1.1)\.
- \[14\]S\. Kaba, A\. K\. Mondal, Y\. Zhang, Y\. Bengio, and S\. Ravanbakhsh\(2023\)Equivariance with learned canonicalization functions\.InInternational Conference on Machine Learning,pp\. 15546–15566\.Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p3.1),[§5](https://arxiv.org/html/2605.11008#S5.SS0.SSS0.Px2.p6.3),[Table 4](https://arxiv.org/html/2605.11008#S5.T4.7.2.2),[Table 4](https://arxiv.org/html/2605.11008#S5.T4.9.4.2)\.
- \[15\]H\. Larochelle, D\. Erhan, A\. Courville, J\. Bergstra, and Y\. Bengio\(2007\)An empirical evaluation of deep architectures on problems with many factors of variation\.InProceedings of the 24th international conference on Machine learning,pp\. 473–480\.Cited by:[§5](https://arxiv.org/html/2605.11008#S5.SS0.SSS0.Px2.p6.3)\.
- \[16\]Y\. E\. Lin and R\. Levie\(2026\)Adaptive canonicalization with application to invariant anisotropic geometric networks\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=j2DHdrsRXI)Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px2.p1.1)\.
- \[17\]Y\. Lin, J\. Helwig, S\. Gui, and S\. Ji\(2024\)Equivariance via minimal frame averaging for more symmetries and efficiency\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=guFsTBXsov)Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p3.1)\.
- \[18\]G\. Ma, Y\. Wang, D\. Lim, S\. Jegelka, and Y\. Wang\(2024\)A canonicalization perspective on invariant and equivariant learning\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=jjcY92FX4R)Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p3.1)\.
- \[19\]G\. Ma, Y\. Wang, D\. Lim, S\. Jegelka, and Y\. Wang\(2024\)A canonicalization perspective on invariant and equivariant learning\.Advances in Neural Information Processing Systems37,pp\. 60936–60979\.Cited by:[§B\.1](https://arxiv.org/html/2605.11008#A2.SS1.SSS0.Px2.p1.18)\.
- \[20\]G\. Ma, Y\. Wang, and Y\. Wang\(2023\)Laplacian canonization: a minimalist approach to sign and basis invariant spectral embedding\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=1mAYtdoYw6)Cited by:[§B\.1](https://arxiv.org/html/2605.11008#A2.SS1.SSS0.Px2.p1.18),[§1](https://arxiv.org/html/2605.11008#S1.p3.1)\.
- \[21\]S\. Maskey, G\. Kutyniok, and R\. Levie\(2025\)Generalization bounds for message passing networks on mixture of graphons\.SIAM Journal on Mathematics of Data Science7\(2\),pp\. 802–825\.Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px3.p1.1)\.
- \[22\]M\. Petrache and S\. Trivedi\(2023\)Approximation\-generalization trade\-offs under \(approximate\) group equivariance\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=DnO6LTQ77U)Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px3.p1.1)\.
- \[23\]O\. Puny, M\. Atzmon, H\. Ben\-Hamu, I\. Misra, A\. Grover, E\. J\. Smith, and Y\. Lipman\(2021\)Frame averaging for invariant and equivariant network design\.arXiv preprint arXiv:2110\.03336\.Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p2.1),[§5](https://arxiv.org/html/2605.11008#S5.SS0.SSS0.Px2.p4.1)\.
- \[24\]J\. W\. Siegel, S\. Hordan, H\. Lawrence, A\. Syed, and N\. Dym\(2026\)Quantitative approximation rates for group equivariant learning\.External Links:2602\.20370,[Link](https://arxiv.org/abs/2602.20370)Cited by:[§A\.1](https://arxiv.org/html/2605.11008#A1.SS1.7.p4.19),[§2](https://arxiv.org/html/2605.11008#S2.1.p1.10)\.
- \[25\]R\. P\. Stanley\(2011\)Enumerative combinatorics volume 1 second edition\.Cambridge studies in advanced mathematics\.Cited by:[§A\.3](https://arxiv.org/html/2605.11008#A1.SS3.1.p1.6)\.
- \[26\]B\. Tahmasebi and S\. Jegelka\(2025\)Generalization bounds for canonicalization: a comparative study with group averaging\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px3.p1.1)\.
- \[27\]B\. Tahmasebi and S\. Jegelka\(2025\)Regularity in canonicalized models: a theoretical perspective\.InInternational Conference on Artificial Intelligence and Statistics,pp\. 4789–4797\.Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px2.p1.1)\.
- \[28\]A\. Vasileiou, S\. Jegelka, R\. Levie, and C\. Morris\(2025\)Survey on generalization theory for graph neural networks\.arXiv preprint arXiv:2503\.15650\.Cited by:[§1\.1](https://arxiv.org/html/2605.11008#S1.SS1.SSS0.Px3.p1.1)\.
- \[29\]D\. Weininger, A\. Weininger, and J\. L\. Weininger\(1989\)SMILES\. 2\. algorithm for generation of unique smiles notation\.J\. Chem\. Inf\. Comput\. Sci\.29,pp\. 97–101\.External Links:[Link](https://api.semanticscholar.org/CorpusID:6621315)Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p3.1)\.
- \[30\]X\. Wu, L\. Jiang, P\. Wang, Z\. Liu, X\. Liu, Y\. Qiao, W\. Ouyang, T\. He, and H\. Zhao\(2024\)Point transformer v3: simpler, faster, stronger\.InCVPR,Cited by:[item 3](https://arxiv.org/html/2605.11008#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2605.11008#S1.p3.1),[§4\.1](https://arxiv.org/html/2605.11008#S4.SS1.p2.8),[§4](https://arxiv.org/html/2605.11008#S4.p1.7)\.
- \[31\]Z\. Wu, S\. Song, A\. Khosla, F\. Yu, L\. Zhang, X\. Tang, and J\. Xiao\(2015\)3d shapenets: a deep representation for volumetric shapes\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 1912–1920\.Cited by:[§5](https://arxiv.org/html/2605.11008#S5.SS0.SSS0.Px1.p1.2)\.
- \[32\]H\. Xu and S\. Mannor\(2012\)Robustness and generalization\.Machine learning86\(3\),pp\. 391–423\.Cited by:[§A\.1](https://arxiv.org/html/2605.11008#A1.SS1.p1.1),[Theorem A\.1](https://arxiv.org/html/2605.11008#A1.Thmdefinition1),[Theorem 2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1.p1.13.13),[§2](https://arxiv.org/html/2605.11008#S2.p1.1)\.
- \[33\]M\. Zaheer, S\. Kottur, S\. Ravanbakhsh, B\. Poczos, R\. R\. Salakhutdinov, and A\. J\. Smola\(2017\)Deep sets\.Advances in neural information processing systems30\.Cited by:[§5](https://arxiv.org/html/2605.11008#S5.SS0.SSS0.Px2.p4.1)\.
- \[34\]C\. Zhou, Z\. Chen, Z\. Li, J\. Wang, K\. Jiang, P\. Li, R\. Yu, M\. Zhang, S\. Bates, and T\. Jaakkola\(2026\)Rethinking diffusion models with symmetries through canonicalization with applications to molecular graph generation\.External Links:2602\.15022,[Link](https://arxiv.org/abs/2602.15022)Cited by:[§1](https://arxiv.org/html/2605.11008#S1.p3.1)\.

## Appendix AProofs

In this section, we provide the proofs that are missing in the main text\.

### A\.1Generalization and Covering Numbers

Our generalization results are based on the results of\[[32](https://arxiv.org/html/2605.11008#bib.bib26)\]: a combination of Theorem 1 and Example 4 from\[[32](https://arxiv.org/html/2605.11008#bib.bib26)\]yields the following theorem:

###### Theorem A\.1\(\[[32](https://arxiv.org/html/2605.11008#bib.bib26)\]\)\.

IfXXis compact w\.r\.t\. metricρ\\rho, andl​\(hS,⋅\)l\(h\_\{S\},\\cdot\)is Lipschitz continuous with Lipschitz constantc​\(s\)c\(s\), i\.e\.,

\|l​\(hS,x1\)−l​\(hS,x2\)\|≤c​\(S\)​ρ​\(x1,x2\),∀x1,x2∈X,\|l\(h\_\{S\},x\_\{1\}\)\-l\(h\_\{S\},x\_\{2\}\)\|\\leq c\(S\)\\rho\(x\_\{1\},x\_\{2\}\),\\quad\\forall x\_\{1\},x\_\{2\}\\in X,and the training sample setSSis generated bynnIID draws fromμ\\mu, then for anyδ,ϵ\>0\\delta,\\epsilon\>0, with probability at least1−δ1\-\\deltawe have

\|ℒexp​\(hS\)−ℒemp​\(hS\)\|≤2​c​\(S\)​ϵ\+M​2​𝒩​\(X,ρ,ϵ\)​ln⁡2\+2​ln⁡\(1/δ\)n\.\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\|\\leq 2c\(S\)\\epsilon\+M\\sqrt\{\\frac\{2\\mathcal\{N\}\(X,\\rho,\\epsilon\)\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}\.\(5\)

We now explain how Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)from the main text follows from Theorem[A\.1](https://arxiv.org/html/2605.11008#A1.Thmdefinition1)\. We recall Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1): See[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)

###### Proof\.

We will apply Theorem[A\.1](https://arxiv.org/html/2605.11008#A1.Thmdefinition1), where to apply it to to the setting discussed in Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1), we set

l​\(hS,x\)=ℓ​\(hS​\(x\),f​\(x\)\)\.l\(h\_\{S\},x\)=\\ell\(h\_\{S\}\(x\),f\(x\)\)\.
We note thatllis Lipscitz with a Lipschitz constant ofcℓ​\(chS\+cf\)c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)as the following shows:

\|l​\(hS,x\)−l​\(hS,x′\)\|\\displaystyle\|l\(h\_\{S\},x\)\-l\(h\_\{S\},x^\{\\prime\}\)\|=\|ℓ​\(hS​\(x\),f​\(x\)\)−ℓ​\(hS​\(x′\),f​\(x′\)\)\|\\displaystyle=\|\\ell\(h\_\{S\}\(x\),f\(x\)\)\-\\ell\(h\_\{S\}\(x^\{\\prime\}\),f\(x^\{\\prime\}\)\)\|≤cℓ​\(ρY​\(hS​\(x\),hS​\(x′\)\)\+ρY​\(f​\(x\),f​\(x′\)\)\)\\displaystyle\\leq c\_\{\\ell\}\\left\(\\rho\_\{Y\}\(h\_\{S\}\(x\),h\_\{S\}\(x^\{\\prime\}\)\)\+\\rho\_\{Y\}\(f\(x\),f\(x^\{\\prime\}\)\)\\right\)≤cℓ​\(chS​ρ​\(x,x′\)\+cf⋅ρ​\(x,x′\)\)\\displaystyle\\leq c\_\{\\ell\}\\left\(c\_\{h\_\{S\}\}\\rho\(x,x^\{\\prime\}\)\+c\_\{f\}\\cdot\\rho\(x,x^\{\\prime\}\)\\right\)=cℓ​\(chS\+cf\)​\(ρ​\(x,x′\)\)\\displaystyle=c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\(\\rho\(x,x^\{\\prime\}\)\)
Now, to conclude the proof, choose anyϵ,δ\>0\\epsilon,\\delta\>0\. Sincelliscℓ​\(chS\+cf\)c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)Lipschitz, we have

\|ℒexp​\(hS\)−ℒemp​\(hS\)\|\\displaystyle\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\|≤\([5](https://arxiv.org/html/2605.11008#A1.E5)\)2​cℓ​\(chS\+cf\)​ϵ\+M​2​𝒩​\(X,ρ,ϵ\)​ln⁡2\+2​ln⁡\(1/δ\)n\\displaystyle\\stackrel\{\{\\scriptstyle\\eqref\{eq:intermediate\}\}\}\{\{\\leq\}\}2c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\\epsilon\+M\\sqrt\{\\frac\{2\\mathcal\{N\}\(X,\\rho,\\epsilon\)\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}which proves Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)\. ∎

Next, we prove Proposition[2\.2](https://arxiv.org/html/2605.11008#S2.Thmdefinition2)\. See[2\.2](https://arxiv.org/html/2605.11008#S2.Thmdefinition2)

###### Proof\.

The first case wherehSh\_\{S\}is Lipschitz \(but not invariant\) follows directly from Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)\.

Next, we consider the second casehS=h~S∘ch\_\{S\}=\\tilde\{h\}\_\{S\}\\circ c, whereh~S\\tilde\{h\}\_\{S\}ischSc\_\{h\_\{S\}\}Lipschitz\. We note that sinceh~S\\tilde\{h\}\_\{S\}operates onc​\(S\)c\(S\), we can think of it as defined by the setS~=c​\(S\)\\tilde\{S\}=c\(S\), and so we can writeh~S:=h~S~\\tilde\{h\}\_\{S\}:=\\tilde\{h\}\_\{\\tilde\{S\}\}\. This notation is more convenient for the proof\.

We want to consider the domain to bec​\(K\)c\(K\), so consider the following distribution measure onc​\(K\)c\(K\):

μ~​\(A\)=μ​\(c−1​\(A\)\)\\displaystyle\\tilde\{\\mu\}\(A\)=\\mu\(c^\{\-1\}\(A\)\)Note that

∀x∈c​\(K\),hS​\(x\)=h~S~​\(x\)\\forall x\\in c\(K\),h\_\{S\}\(x\)=\\tilde\{h\}\_\{\\tilde\{S\}\}\(x\)Next, for any fixednnsamples fromKK, which we denote byS=\{x1,x2,…,xn\}S=\\\{x\_\{1\},x\_\{2\},\\ldots,x\_\{n\}\\\}, denote its image byS~=\{c​\(x1\),…,c​\(xn\)\)\\tilde\{S\}=\\\{c\(x\_\{1\}\),\\ldots,c\(x\_\{n\}\)\)\. Then

ℒemp\(hS\)=1n∑i=1nℓ\(h~S~\(c\(xi\)\),f\(xi\)=1n∑i=1nℓ\(h~S~\(c\(xi\)\),f\(c\(xi\)\)=ℒemp\(h~S~\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\ell\(\\tilde\{h\}\_\{\\tilde\{S\}\}\(c\(x\_\{i\}\)\),f\(x\_\{i\}\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\ell\(\\tilde\{h\}\_\{\\tilde\{S\}\}\(c\(x\_\{i\}\)\),f\(c\(x\_\{i\}\)\)=\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)So, the empirical losses are the same, and next we show the expected losses are the same:

ℒexp​\(h~S~\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)=𝔼x∼μ~ℓ\(h~S~\(x\),f\(x\)\)=∫c​\(K\)ℓ\(h~S~\(x\),f\(x\)\)dμ~=∫Kℓ\(h~S~\(c\(x\)\),f\(c\(x\)\)dμ\\displaystyle=\\mathbb\{E\}\_\{x\\sim\\tilde\{\\mu\}\}\\ell\(\\tilde\{h\}\_\{\\tilde\{S\}\}\(x\),f\(x\)\)=\\int\_\{c\(K\)\}\\ell\(\\tilde\{h\}\_\{\\tilde\{S\}\}\(x\),f\(x\)\)d\\tilde\{\\mu\}=\\int\_\{K\}\\ell\(\\tilde\{h\}\_\{\\tilde\{S\}\}\(c\(x\)\),f\(c\(x\)\)d\\mu=∫Kℓ​\(hS​\(x\)\),f​\(x\)​d​μ=ℒexp​\(hS\)\\displaystyle=\\int\_\{K\}\\ell\(h\_\{S\}\(x\)\),f\(x\)d\\mu=\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)Finally, we apply Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)onh~S\\tilde\{h\}\_\{S\}and as it’schSc\_\{h\_\{S\}\}lipschiz we obtain that with probablity at least1−δ1\-\\delta, according toμ~\\tilde\{\\mu\}, it holds that:

\|ℒexp​\(h~S\)−ℒemp​\(h~S\)\|≤2​cℓ​\(chS\+cf\)​ϵ\+M​2​𝒩​\(c​\(K\),ρX,ϵ\)​ln⁡2\+2​ln⁡\(1/δ\)n\\displaystyle\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(\\tilde\{h\}\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(\\tilde\{h\}\_\{S\}\)\|\\leq 2c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\\epsilon\+M\\sqrt\{\\frac\{2\\mathcal\{N\}\(c\(K\),\\rho\_\{X\},\\epsilon\)\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}Let us abbreviate the right hand side of the equation above byη=2​cℓ​\(chS\+cf\)​ϵ\+M​2​𝒩​\(c​\(K\),ρX,ϵ\)​ln⁡2\+2​ln⁡\(1/δ\)n\\eta=2c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\\epsilon\+M\\sqrt\{\\frac\{2\\mathcal\{N\}\(c\(K\),\\rho\_\{X\},\\epsilon\)\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}\. We then have

1−δ\\displaystyle 1\-\\delta≤ℙμ~​\{S~⊆c​\(K\)\|\|ℒexp​\(h~S~\)−ℒemp​\(h~S~\)\|≤η\}\\displaystyle\\leq\\mathbb\{P\}\_\{\\tilde\{\\mu\}\}\\\{\\tilde\{S\}\\subseteq c\(K\)\|\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)\|\\leq\\eta\\\}=ℙμ\{S=\(x1,…,xn\)\|\|ℒexp\(h~S~∘c\)−ℒemp\(h~S~∘c\)\|≤η\}\\displaystyle=\\mathbb\{P\}\_\{\\mu\}\\\{S=\(x\_\{1\},\\ldots,x\_\{n\}\)\|\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\\circ c\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\\circ c\)\|\\quad\\leq\\eta\\\}=ℙμ\{S=\(x1,…,xn\)\|\|ℒexp\(hS\)−ℒemp\(hS\)\|≤η\}\\displaystyle=\\mathbb\{P\}\_\{\\mu\}\\\{S=\(x\_\{1\},\\ldots,x\_\{n\}\)\|\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\|\\quad\\leq\\eta\\\}which concludes the proof of the second case\.

In the third case, wherehSh\_\{S\}ischSc\_\{h\_\{S\}\}lipshiz and invariant, we want to consider the domainK/𝔾K/\\mathbb\{G\}and defineπ:K→K/𝔾\\pi:K\\rightarrow K/\\mathbb\{G\}byπ​\(k\)=\[k\]\\pi\(k\)=\[k\]\. Define the following distribution onK/𝔾K/\\mathbb\{G\}by:

μ~​\(A\)=μ​\(π−1​\(A\)\)\\displaystyle\\tilde\{\\mu\}\(A\)=\\mu\(\\pi^\{\-1\}\(A\)\)DefineS~=π​\(S\)\\tilde\{S\}=\\pi\(S\)and

h~S~,f~:k/𝔾→ℝ\\tilde\{h\}\_\{\\tilde\{S\}\},\\tilde\{f\}:k/\\mathbb\{G\}\\rightarrow\\mathbb\{R\}by

h~S~​\(\[x\]\)=hS​\(x\),f~​\(\[x\]\)=f​\(x\)\\tilde\{h\}\_\{\\tilde\{S\}\}\(\[x\]\)=h\_\{S\}\(x\),\\tilde\{f\}\(\[x\]\)=f\(x\)Note that by invariance, these functions are well defined\. Next, for any setSSofnnsamplesx1,x2,…,xnx\_\{1\},x\_\{2\},\\ldots,x\_\{n\}fromKK, denoteS~=\{π​\(x1\),…,π​\(xn\)\}\\tilde\{S\}=\\\{\\pi\(x\_\{1\}\),\\ldots,\\pi\(x\_\{n\}\)\\\}, then

ℒemp\(hS\)=1n∑i=1nℓ\(hS\(xi\),f\(xi\)\)=1n∑i=1nℓ\(h~S\(π\(xi\),f~\(π\(xi\)\)=ℒemp\(h~S~\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\ell\(h\_\{S\}\(x\_\{i\}\),f\(x\_\{i\}\)\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\ell\(\\tilde\{h\}\_\{S\}\(\\pi\(x\_\{i\}\),\\tilde\{f\}\(\\pi\(x\_\{i\}\)\)=\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)So, the empirical losses are the same, and next we show the expected losses:

ℒexp​\(h~S~\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)=𝔼x∼μ~​ℓ​\(h~S~​\(x\),f~​\(x\)\)=∫K/𝔾ℓ​\(h~S~​\(x\),f~​\(x\)\)​𝑑μ~​\(x\)\\displaystyle=\\mathbb\{E\}\_\{x\\sim\\tilde\{\\mu\}\}\\ell\(\\tilde\{h\}\_\{\\tilde\{S\}\}\(x\),\\tilde\{f\}\(x\)\)=\\int\_\{K/\\mathbb\{G\}\}\\ell\(\\tilde\{h\}\_\{\\tilde\{S\}\}\(x\),\\tilde\{f\}\(x\)\)d\\tilde\{\\mu\}\(x\)=∫Kℓ​\(h~S​\(π​\(x\)\),f~​\(π​\(x\)\)\)​𝑑μ\\displaystyle=\\int\_\{K\}\\ell\(\\tilde\{h\}\_\{S\}\(\\pi\(x\)\),\\tilde\{f\}\(\\pi\(x\)\)\)d\\mu=∫Kℓ​\(hS​\(x\)\),f​\(x\)​d​μ=ℒexp​\(hS\)\\displaystyle=\\int\_\{K\}\\ell\(h\_\{S\}\(x\)\),f\(x\)d\\mu=\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)Note thatf~\\tilde\{f\}iscfc\_\{f\}Lipschitz function andh~S\\tilde\{h\}\_\{S\}ischSc\_\{h\_\{S\}\}Lipschitz function \(see Lemma 20 in\[[24](https://arxiv.org/html/2605.11008#bib.bib23)\]\. Therefore, we can apply Theorem[2\.1](https://arxiv.org/html/2605.11008#S2.Thmdefinition1)onh~S\\tilde\{h\}\_\{S\}andf~\\tilde\{f\}and we obtain that with probablity at least1−δ1\-\\deltait holds that:

\|ℒexp​\(h~S\)−ℒemp​\(h~S\)\|≤2​cℓ​\(chS\+cf\)​ϵ\+M​2​𝒩​\(K/𝔾,ρ𝔾,ϵ\)​ln⁡2\+2​ln⁡\(1/δ\)n\\displaystyle\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(\\tilde\{h\}\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(\\tilde\{h\}\_\{S\}\)\|\\leq 2c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\\epsilon\+M\\sqrt\{\\frac\{2\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}And as\|ℒexp​\(h~S~\)−ℒemp​\(h~S~\)\|=\|ℒexp​\(hS\)−ℒemp​\(hS\)\|\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(\\tilde\{h\}\_\{\\tilde\{S\}\}\)\|=\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\|\(see the more formal argument used for bounding the generalization ofhS∘ch\_\{S\}\\circ c\) we obtain that with probability of at least1−δ1\-\\delta,

\|ℒexp​\(hS\)−ℒemp​\(hS\)\|≤2​cℓ​\(chS\+cf\)​ϵ\+M​2​𝒩​\(k/𝔾,ρ𝔾,ϵ\)​ln⁡2\+2​ln⁡\(1/δ\)n\\displaystyle\|\\mathcal\{L\}\_\{\\mathrm\{exp\}\}\(h\_\{S\}\)\-\\mathcal\{L\}\_\{\\mathrm\{emp\}\}\(h\_\{S\}\)\|\\leq 2c\_\{\\ell\}\(c\_\{h\_\{S\}\}\+c\_\{f\}\)\\epsilon\+M\\sqrt\{\\frac\{2\\mathcal\{N\}\(k/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\\ln 2\+2\\ln\(1/\\delta\)\}\{n\}\}∎

We next state and prove a result mentioned in the main text: that the Lipschitz constant of a backbone model is preserved under averaging

###### Proposition A\.2\.

Let\(X,ρ,𝔾\)\(X,\\rho,\\mathbb\{G\}\)be a module, and assume that𝔾\\mathbb\{G\}is a compact group with a Haar probability measureμ\\mu\. LetYYbe a normed space and assume thath:X→Yh:X\\to Yischc\_\{h\}Lipschitz\. Then the averaged function

q​\(x\)=∫Gh​\(g​x\)​𝑑μ​\(g\)q\(x\)=\\int\_\{G\}h\(gx\)d\\mu\(g\)ischc\_\{h\}Lipschitz as well\.

###### Proof\.

For everyx1,x2∈Xx\_\{1\},x\_\{2\}\\in X

‖q​\(x1\)−q​\(x2\)‖\\displaystyle\\\|q\(x\_\{1\}\)\-q\(x\_\{2\}\)\\\|=‖∫Gh​\(g​x1\)​𝑑μ​\(g\)−∫Gh​\(g​x2\)​𝑑μ​\(g\)‖\\displaystyle=\\left\\\|\\int\_\{G\}h\(gx\_\{1\}\)d\\mu\(g\)\-\\int\_\{G\}h\(gx\_\{2\}\)d\\mu\(g\)\\right\\\|≤Jensen∫G‖h​\(g​x1\)−h​\(g​x2\)‖​𝑑μ​\(g\)\\displaystyle\\stackrel\{\{\\scriptstyle\\text\{Jensen\}\}\}\{\{\\leq\}\}\\int\_\{G\}\\left\\\|h\(gx\_\{1\}\)\-h\(gx\_\{2\}\)\\right\\\|d\\mu\(g\)≤∫Gch​ρ​\(g​x1,g​x2\)​𝑑μ​\(g\)\\displaystyle\\leq\\int\_\{G\}c\_\{h\}\\rho\(gx\_\{1\},gx\_\{2\}\)d\\mu\(g\)=∫Gch​ρ​\(x1,x2\)​𝑑μ​\(g\)\\displaystyle=\\int\_\{G\}c\_\{h\}\\rho\(x\_\{1\},x\_\{2\}\)d\\mu\(g\)=ch​ρ​\(x1,x2\)\\displaystyle=c\_\{h\}\\rho\(x\_\{1\},x\_\{2\}\)∎

### A\.2Proofs for Section[3](https://arxiv.org/html/2605.11008#S3)

We restate and prove Proposition[2\.4](https://arxiv.org/html/2605.11008#S2.Thmdefinition4)\. See[2\.4](https://arxiv.org/html/2605.11008#S2.Thmdefinition4)

###### Proof\.

LetCCbe anϵ\\epsilon\-cover ofK/𝔾K/\\mathbb\{G\}underρ𝔾\\rho\_\{\\mathbb\{G\}\}, and define

C^:=\{g⋅y∣y∈C,g∈𝔾\}\.\\widehat\{C\}:=\\\{g\\cdot y\\mid y\\in C,\\ g\\in\\mathbb\{G\}\\\}\.Then of course\|C^\|≤\|𝔾\|⋅\|C\|\|\\widehat\{C\}\|\\leq\|\\mathbb\{G\}\|\\cdot\|C\|\.

We claim thatC^\\widehat\{C\}is anϵ\\epsilon\-cover ofKKunderdd\. Letx∈Kx\\in K\. SinceCCcoversK/𝔾K/\\mathbb\{G\}, there existsy∈Cy\\in Csuch that

ρ𝔾​\(\[x\],\[y\]\)≤ϵ\.\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\)\\leq\\epsilon\.Because𝔾\\mathbb\{G\}is finite, the minimum in the definition ofρ𝔾\\rho\_\{\\mathbb\{G\}\}is attained, so there existsg∈𝔾g\\in\\mathbb\{G\}such that

ρ​\(x,g⋅y\)≤ϵ\.\\rho\(x,g\\cdot y\)\\leq\\epsilon\.By construction,g⋅y∈C^g\\cdot y\\in\\widehat\{C\}, henceC^\\widehat\{C\}coversKK\. Therefore𝒩​\(K,ρ,ϵ\)≤\|C^\|≤\|𝔾\|⋅\|C\|=\|𝔾\|⋅𝒩​\(K/𝔾,ρ𝔾,ϵ\)\\mathcal\{N\}\(K,\\rho,\\epsilon\)\\leq\|\\widehat\{C\}\|\\leq\|\\mathbb\{G\}\|\\cdot\|C\|=\|\\mathbb\{G\}\|\\cdot\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\. ∎

We next prove Proposition[3\.2](https://arxiv.org/html/2605.11008#S3.Thmdefinition2)\. See[3\.2](https://arxiv.org/html/2605.11008#S3.Thmdefinition2)

###### Proof\.

As it always holds that

𝒩​\(K/𝔾,ρ𝔾,ϵ\)≤𝒩​\(c​\(K\),ρ,ϵ\)\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho\_\{\\mathbb\{G\}\},\\epsilon\)\\leq\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)we just need to prove the opposite direction\.

LetCCbe a cover ofK/𝔾K/\\mathbb\{G\}with𝒩​\(K/𝔾,ρ,ϵ\)\\mathcal\{N\}\(K/\\mathbb\{G\},\\rho,\\epsilon\)elements\. Consider

C^:=\{c​\(x\)\|x∈C\}\\hat\{C\}:=\\\{c\(x\)\|x\\in C\\\}and we will prove thatC^\\hat\{C\}is anϵ\\epsiloncover ofc​\(K\)c\(K\)\.

Lety∈c​\(K\)y\\in c\(K\), then by definition∃x∈K:y=c​\(x\)\\exists x\\in K:y=c\(x\)\. By the definition of cover∃z∈C\\exists z\\in Csuch thatρ​\(\[x\],\[z\]\)≤ϵ\\rho\(\[x\],\[z\]\)\\leq\\epsilon\. But asρ𝔾​\(\[x\],\[z\]\)=ρ​\(c​\(x\),c​\(z\)\)\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[z\]\)=\\rho\(c\(x\),c\(z\)\)we obtain thatρ​\(y,c​\(z\)\)≤ϵ\\rho\(y,c\(z\)\)\\leq\\epsilon\. SoC^\\hat\{C\}is anϵ\\epsiloncover ofc​\(K\)c\(K\)as desired\. ∎

See[3\.3](https://arxiv.org/html/2605.11008#S3.Thmdefinition3)

###### Proof\.

The canonizationc​\(x\)=\|x\|c\(x\)=\|x\|is an isometry as

∀x,y∈K,ρ𝔾​\(\[x\],\[y\]\)=min⁡\{\|x−y\|,\|x\+y\|\}=\|\|x\|−\|y\|\|=\|c​\(x\)−c​\(y\)\|\\forall x,y\\in K,\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\)=\\min\\\{\|x\-y\|,\|x\+y\|\\\}=\|\|x\|\-\|y\|\|=\|c\(x\)\-c\(y\)\|
The fact that sorting is an isometry with respect to the metricρ𝔾\\rho\_\{\\mathbb\{G\}\}induced from appnorm \(which is thepp\-Wasserstein metric\) is shown in Lemma 4\.2 in\[[4](https://arxiv.org/html/2605.11008#bib.bib14)\]\.

consider the action ofℝd\\mathbb\{R\}^\{d\}onℝd×n\\mathbb\{R\}^\{d\\times n\}by translating all columns of a matrix inℝd×n\\mathbb\{R\}^\{d\\times n\}by a vector inℝd\\mathbb\{R\}^\{d\}\. Letρ\\rhobe the metric induced from the Frobenius normρ​\(X,Y\)=‖X−Y‖F\\rho\(X,Y\)=\\\|X\-Y\\\|\_\{F\}, and letccbe the centralization mappingc​\(X\)=X−1n​X​1n×nc\(X\)=X\-\\frac\{1\}\{n\}X1\_\{n\\times n\}\. This mapping is a canonization and an isometry\. To see it is an isometry, note that we have for any canonizationρ​\(c​\(X\),c​\(Y\)\)≤ρ𝔾​\(\[X\],\[Y\]\)\\rho\(c\(X\),c\(Y\)\)\\leq\\rho\_\{\\mathbb\{G\}\}\\left\(\[X\],\[Y\]\\right\), and we get the converse inequality by noting thatccis the orthogonal projection from the inner product spaceℝd×n\\mathbb\{R\}^\{d\\times n\}onto the subspace\{X∈ℝd×n\|X1n×n=0\}\\\{X\\in\\mathbb\{R\}^\{d\\times n\}\|\\quad X1\_\{n\\times n\}=0\\\}\. Thus, letg∈𝔾g\\in\\mathbb\{G\}be a translation vector such thatρ𝔾​\(\[X\],\[Y\]\)=ρ​\(X,g​Y\)\\rho\_\{\\mathbb\{G\}\}\(\[X\],\[Y\]\)=\\rho\(X,gY\)\. Then

ρ𝔾​\(\[X\],\[Y\]\)=ρ​\(X,g​Y\)=‖X−g​Y‖F≤‖c​\(X\)−c​\(g​Y\)‖F=‖c​\(X\)−c​\(Y\)‖F=ρ​\(c​\(X\),c​\(Y\)\)\\rho\_\{\\mathbb\{G\}\}\(\[X\],\[Y\]\)=\\rho\(X,gY\)=\\\|X\-gY\\\|\_\{F\}\\leq\\\|c\(X\)\-c\(gY\)\\\|\_\{F\}=\\\|c\(X\)\-c\(Y\)\\\|\_\{F\}=\\rho\(c\(X\),c\(Y\)\)
∎

We next formulate and prove a result mentioned in the main text

###### Proposition A\.3\.

Let\(K,ρ,𝔾\)\(K,\\rho,\\mathbb\{G\}\)be a module, andc:K→Kc:K\\to Ka canonization\. Thenccis an isometric canonization if and only ifccis11\-Lipschitz\.

###### Proof\.

Assumeccis an isometric canonization\. Recall that this means thatρ​\(c​\(x\),c​\(y\)\)=ρ𝔾​\(\[x\],\[y\]\)\\rho\(c\(x\),c\(y\)\)=\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\)\. This implies thatccis Lipschitz because

ρ​\(c​\(x\),c​\(y\)\)=ρ𝔾​\(\[x\],\[y\]\)≤ρ​\(x,y\),∀x,y∈K\\rho\(c\(x\),c\(y\)\)=\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\)\\leq\\rho\(x,y\),\\quad\\forall x,y\\in K
In the other direction, assume thatccis11\-Lipschitz\. This implies that for allg∈𝔾g\\in\\mathbb\{G\}

ρ​\(c​\(x\),c​\(y\)\)=ρ​\(c​\(g​x\),c​\(y\)\)≤ρ​\(g​x,y\)\.\\rho\(c\(x\),c\(y\)\)=\\rho\(c\(gx\),c\(y\)\)\\leq\\rho\(gx,y\)\.Since this is true for allg∈Gg\\in G, we can minimize overggand obtain the inequalityρ​\(c​\(x\),c​\(y\)\)≤ρ𝔾​\(\[x\],\[y\]\)\\rho\(c\(x\),c\(y\)\)\\leq\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\)\. The converse of this inequality is always true for any canonization, namely

ρ​\(c​\(x\),c​\(y\)\)≥ρ𝔾​\(\[x\],\[y\]\)\\rho\(c\(x\),c\(y\)\)\\geq\\rho\_\{\\mathbb\{G\}\}\(\[x\],\[y\]\)and so we obtain thatccis an isometric canonization\. ∎

We restate and prove[3\.4](https://arxiv.org/html/2605.11008#S3.Thmdefinition4)\. See[3\.4](https://arxiv.org/html/2605.11008#S3.Thmdefinition4)

###### Proof\.

LLis a partial limit ofccif there is a sequencexn→xx\_\{n\}\\rightarrow xsuch thatc​\(xn\)→Lc\(x\_\{n\}\)\\rightarrow L\. We claim thatLLmust be in the orbit ofxx\. Indeed,c​\(xn\)=gn​xnc\(x\_\{n\}\)=g\_\{n\}x\_\{n\}for appropriategn∈Gg\_\{n\}\\in G\. Since𝔾\\mathbb\{G\}is finite, by passing to a subsequence, we can assume thatgn=g∈Gg\_\{n\}=g\\in Gdoes not depend onnn\. Sincex↦g​xx\\mapsto gxis an isometry, we know thatg​xn→g​x=Lgx\_\{n\}\\rightarrow gx=L\. In particular,g​x≠xgx\\neq xfor everyg≠eg\\neq e\. Choose someϵ<m​i​ng≠e​\|x−g⋅x\|\\epsilon<min\_\{g\\neq e\}\|x\-g\\cdot x\|, and define theGGinvariant setK=K​\(ϵ\)K=K\(\\epsilon\)byK:=∪g∈GB​\(g⋅x,ϵ\)K:=\\cup\_\{g\\in G\}B\(g\\cdot x,\\epsilon\)\. Note thatc​\(K\)c\(K\)contains all partial limits ofccatxx, and thereforeg​x∈c​\(K\)gx\\in c\(K\)for allg∈Gg\\in G\. Since the distance between any two elements in the orbit ofxxis larger thanϵ\\epsilon, anϵ\\epsiloncover ofc​\(K\)c\(K\)must contain at least\|𝔾\|\|\\mathbb\{G\}\|elements\. On the other hand,KKitself is a union of\|𝔾\|\|\\mathbb\{G\}\|balls of radiusϵ\\epsilon, and therefore

𝒩​\(c​\(K\),ρ,ϵ\)≤𝒩​\(K,ρ,ϵ\)≤\|𝔾\|≤𝒩​\(c​\(K\),ρ,ϵ\),\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\)\\leq\\mathcal\{N\}\(K,\\rho,\\epsilon\)\\leq\|\\mathbb\{G\}\|\\leq\\mathcal\{N\}\(c\(K\),\\rho,\\epsilon\),and therefore all these inequalities are equalities and the covering number is exactly\|𝔾\|\|\\mathbb\{G\}\|\.

Finally, note thatρ​\(K/G,ρG,ϵ\)=1\\rho\(K/G,\\rho\_\{G\},\\epsilon\)=1as one ball of radiusϵ\\epsilonis enough for covering the quotient space, as all\|𝔾\|\|\\mathbb\{G\}\|balls are the same up to group action\. ∎

### A\.3Proofs for Section[4](https://arxiv.org/html/2605.11008#S4)

For proving Lemma[4\.2](https://arxiv.org/html/2605.11008#S4.Thmdefinition2), we need the following lemma:

###### Lemma A\.4\.

LetS:=\[m\]nS:=\[m\]^\{n\}all vectors withnnelements, each between11andmm\. Define two vectorsv≡u⇔∃π∈Sn:u=π⋅vv\\equiv u\\iff\\exists\\pi\\in S\_\{n\}:u=\\pi\\cdot v\. Then the number of equivalent classes is exactly\(n\+m−1n\)\\binom\{n\+m\-1\}\{n\}\.

###### Proof\.

Define∀i∈\[m\]\\forall i\\in\[m\],ciuc^\{u\}\_\{i\}to be the number of timesiiappears inuu\. Note thatu≡v⇔∀i∈\[m\]:ciu=civu\\equiv v\\iff\\forall i\\in\[m\]:c^\{u\}\_\{i\}=c^\{v\}\_\{i\}\. So we want to ask how many solutions there are to the following system:

∑i=1mci=n\\displaystyle\\sum^\{m\}\_\{i=1\}c\_\{i\}=n∀i,ci≥0\\displaystyle\\forall i,c\_\{i\}\\geq 0By\[[25](https://arxiv.org/html/2605.11008#bib.bib27)\]we know the number of solutions to the equation is\(n\+m−1n\)\\binom\{n\+m\-1\}\{n\}\. ∎

We now restate and prove[4\.2](https://arxiv.org/html/2605.11008#S4.Thmdefinition2)\. See[4\.2](https://arxiv.org/html/2605.11008#S4.Thmdefinition2)

###### Proof\.

Setk=⌈1/\(2​ϵ\)⌉k=\\lceil 1/\(2\\epsilon\)\\rceil\. The setKKcan be covered bykdk^\{d\}hypercubes of diameterkkand radius1/\(2​k\)≤ϵ1/\(2k\)\\leq\\epsilon\. LetCCdenote the collection of centers of these hypercubes, namely

C=\{\[2​i1\+12​k,2​i2\+12​k,…,2​in\+12​k\]\|ij∈\{0,1,…,k−1\},1≤j≤n\}\.\\displaystyle C=\\left\\\{\\left\[\\frac\{2i\_\{1\}\+1\}\{2k\},\\frac\{2i\_\{2\}\+1\}\{2k\},\\ldots,\\frac\{2i\_\{n\}\+1\}\{2k\}\\right\]\|\\quad i\_\{j\}\\in\\\{0,1,\\ldots,k\-1\\\},1\\leq j\\leq n\\right\\\}\.Applying the quotient mapπ​\(c\)=\[c\]\\pi\(c\)=\[c\]we obtain pointsπ​\(C\)\\pi\(C\)in the quotient space which are anϵ\\epsiloncover ofK/𝔾K/\\mathbb\{G\}\. Note that points inCCwhich are permutations of each other are mapped byπ\\pito the same orbit\. Therefore, the cardinality of the coverπ​\(C\)\\pi\(C\)is the number of equivalence classes with respect to the relation onCCdefined by the group:c1≡c2⇔∃π∈Sn:c1=π​\(c2\)c\_\{1\}\\equiv c\_\{2\}\\iff\\exists\\pi\\in S\_\{n\}:c\_\{1\}=\\pi\(c\_\{2\}\)\. Using Lemma[A\.4](https://arxiv.org/html/2605.11008#A1.Thmdefinition4), we know that the number of equivalence classes is

\|π​\(C\)\|=\(n\+kd−1n\)\|\\pi\(C\)\|=\\binom\{n\+k^\{d\}\-1\}\{n\}∎

We now restate and prove[4\.1](https://arxiv.org/html/2605.11008#S4.Thmdefinition1)\. See[4\.1](https://arxiv.org/html/2605.11008#S4.Thmdefinition1)

###### Proof\.

We consider the setU⊂\[0,1\]d×nU\\subset\[0,1\]^\{d\\times n\}defined by

U=\{X∈\[0,1\]d×n\|X11=X12=…=X1​n\}\\displaystyle U=\\\{X\\in\[0,1\]^\{d\\times n\}\|\\quad X\_\{11\}=X\_\{12\}=\\ldots=X\_\{1n\}\\\}The setUUis not a subset ofclex​\(K\)c\_\{\\mathrm\{lex\}\}\(K\), but it is a subset of its closureclex​\(K\)¯\\overline\{c\_\{\\mathrm\{lex\}\}\(K\)\}\. This is because for everyX∈UX\\in Uwe can apply an arbitrarily small perturbation of the first row ofXXso that it is ordered from small to large, and theXXwill be unaffected by the canonization\. We then obtain

𝒩​\(clex​\(K\),ρ,ϵ\)=𝒩​\(clex​\(K\)¯,ρ,ϵ\)≥𝒩​\(U,ρ,ϵ\)=\(∗\)\(12​ϵ\)\(d−1\)⋅n\+1,\\mathcal\{N\}\(c\_\{\\mathrm\{lex\}\}\(K\),\\rho,\\epsilon\)=\\mathcal\{N\}\(\\overline\{c\_\{\\mathrm\{lex\}\}\(K\)\},\\rho,\\epsilon\)\\geq\\mathcal\{N\}\(U,\\rho,\\epsilon\)\\stackrel\{\{\\scriptstyle\(\*\)\}\}\{\{=\}\}\\left\(\\frac\{1\}\{2\\epsilon\}\\right\)^\{\(d\-1\)\\cdot n\+1\},where\(∗\)\(\*\)follows from the fact that the hypercube\[0,1\]\(d−1\)⋅n\+1\[0,1\]^\{\(d\-1\)\\cdot n\+1\}can be mapped isometrically \(in the∞\\inftynorm\) toUUviaf:\[0,1\]\(d−1\)⋅n\+1→Uf:\[0,1\]^\{\(d\-1\)\\cdot n\+1\}\\rightarrow Uby repeating the first entryn−1n\-1times, obtaining ann⋅dn\\cdot dvector, and then transposing to be of shapen×dn\\times d, and from the fact that the covering number of the hypercube is\(12​ϵ\)\(d−1\)⋅n\+1\\left\(\\frac\{1\}\{2\\epsilon\}\\right\)^\{\(d\-1\)\\cdot n\+1\}\. ∎

### A\.4Hilbert Canonization

In this appendix, we give a formal definition of Hilbert curves and a full proof of Theorem[4\.3](https://arxiv.org/html/2605.11008#S4.Thmdefinition3)\.

The Hilbert curveH​\(x\)H\(x\)is defined as a limit of the functionHm:\[0,1\]→\[0,1\]dH\_\{m\}:\[0,1\]\\to\[0,1\]^\{d\}, and theseHmH\_\{m\}are our main focus\. We will now define these curves, loosely following the terminology from\[[11](https://arxiv.org/html/2605.11008#bib.bib28)\]\. Firstly, for a given integerLL, we consider a partition of the unit interval into2L2^\{L\}disjoint intervals: the semi\-closed intervalsIL​\(ℓ\)=\[ℓ​2−L,\(ℓ\+1\)​2−L\)I\_\{L\}\(\\ell\)=\[\\ell 2^\{\-L\},\(\\ell\+1\)2^\{\-L\}\)forℓ=0,…,2L−2\\ell=0,\\ldots,2^\{L\}\-2, and the closed intervalI​\(2L−1\)=\[\(2L−1\)​2−L,1\]I\(2^\{L\}\-1\)=\[\(2^\{L\}\-1\)2^\{\-L\},1\]\. We denote the collection of all these intervals byℐL\\mathcal\{I\}\_\{L\}\. For a naturald\>1d\>1, we also consider the partition of the unit hypercube\[0,1\]d\[0,1\]^\{d\}into2L⋅d2^\{L\\cdot d\}disjoint hypercubes of the formELd​\(ℓ→\)=I​\(ℓ1\)×…×I​\(ℓd\)E\_\{L\}^\{d\}\(\\vec\{\\ell\}\)=I\(\\ell\_\{1\}\)\\times\\ldots\\times I\(\\ell\_\{d\}\)\. We denote the collection of these hypercubes byℰLd\\mathcal\{E\}\_\{L\}^\{d\}\.

Next, for any givenmm, we defineHmH\_\{m\}to be a bijection fromℐd⋅m\\mathcal\{I\}\_\{d\\cdot m\}toℰmd\\mathcal\{E\}\_\{m\}^\{d\}, such that \(i\) adjacent intervals are mapped to adjacent hypercubes and \(ii\) ifId⋅\(m\+1\)​\(k′\)⊆Id⋅m​\(k\)I\_\{d\\cdot\(m\+1\)\}\(k^\{\\prime\}\)\\subseteq I\_\{d\\cdot m\}\(k\)thenHm\+1​\(Id⋅\(m\+1\)​\(k′\)\)⊆Hm​\(Id⋅m​\(k\)\)H\_\{m\+1\}\\left\(I\_\{d\\cdot\(m\+1\)\}\(k^\{\\prime\}\)\\right\)\\subseteq H\_\{m\}\\left\(I\_\{d\\cdot m\}\(k\)\\right\)\. An illustration ofHm,m=1,2,3H\_\{m\},m=1,2,3ford=2d=2is given in Figure[2](https://arxiv.org/html/2605.11008#S4.F2)\. EachHmH\_\{m\}maps a finite collection of intervals bijectively to a finite collection of hypercubes\. This naturally induces a bijection from the grid defined by the intervals centroids, to the grid defined by the hypercube centroids\. We denote these grids by𝔸​\[d⋅m\]⊆\[0,1\]\\mathbb\{A\}\[d\\cdot m\]\\subseteq\[0,1\]and𝔹​\[d,m\]⊆ℝd\\mathbb\{B\}\[d,m\]\\subseteq\\mathbb\{R\}^\{d\}, respectively\.

The Hilbert curveHHis obtained fromHmH\_\{m\}by a limiting procedure and shown to be Holder continuous\[[11](https://arxiv.org/html/2605.11008#bib.bib28)\]\. Following similar ideas we show that each Holder functionHmH\_\{m\}is Holder continuous with constant that do not depend onmm\. This property will be crucial for the analysis of the covering number of the Hilbert canonization\.

###### Lemma A\.5\.

For every naturalm,dm,d, we have that

∀x,y∈𝔸​\[d⋅m\],‖Hm​\(x\)−Hm​\(y\)‖∞≤4​\|x−y\|1/d\\forall x,y\\in\\mathbb\{A\}\[d\\cdot m\],\\quad\\\|H\_\{m\}\(x\)\-H\_\{m\}\(y\)\\\|\_\{\\infty\}\\leq 4\|x\-y\|^\{1/d\}

###### Proof\.

Letx≠yx\\neq ybe points in the one dimensional grid𝔸​\[d⋅m\]\\mathbb\{A\}\[d\\cdot m\], and assume without loss of generality thatx<yx<y\. Note thaty−x≥12d⋅my\-x\\geq\\frac\{1\}\{2^\{d\\cdot m\}\}, since this is the minimal distance between points in the grid\. On the other hand,y−x≤1y\-x\\leq 1\. Therefore, there exists somekkwithm−1≤k≤0m\-1\\leq k\\leq 0, such that

2−d⋅\(k\+1\)≤\|x−y\|≤2−d⋅k\.2^\{\-d\\cdot\(k\+1\)\}\\leq\|x\-y\|\\leq 2^\{\-d\\cdot k\}\.\(6\)It follows thatxxandyyare in two adjacent intervals in thekk\-th level \(or in the same interval\), and by the nesting property of Hilbert’s curve, we know thatHm​\(x\)H\_\{m\}\(x\)andHm​\(y\)H\_\{m\}\(y\)are in two adjacent hypercubes \(or in the same hypercube\)\. Therefore, their distances is at most the sum of the diameters of those two cubes, so

‖Hm​\(x\)−Hm​\(y\)‖∞≤2⋅2−k≤\([6](https://arxiv.org/html/2605.11008#A1.E6)\)4​\|x−y\|1/d\\displaystyle\\\|H\_\{m\}\(x\)\-H\_\{m\}\(y\)\\\|\_\{\\infty\}\\leq 2\\cdot 2^\{\-k\}\\stackrel\{\{\\scriptstyle\\eqref\{eq:interval\}\}\}\{\{\\leq\}\}4\|x\-y\|^\{1/d\}∎

Hilbert CanonizationAs a first step, we define a canonization not on all of\[0,1\]d×n\[0,1\]^\{d\\times n\}, but rather on the finite set ofnntuples of grid elements𝔹n​\[d,m\]\\mathbb\{B\}^\{n\}\[d,m\]\. This is done by mapping each tuple\-element to the unit interval usingHm−1H\_\{m\}^\{\-1\}, then sorting the resultingnn\-dimensional array, and mapping back to thedd\-dimensional grid usingHmH\_\{m\}\. We denote this canonization bycmc\_\{m\}\. We can writecm=Hm∘sort∘Hm−1c\_\{m\}=H\_\{m\}\\circ\\mathrm\{sort\}\\circ H\_\{m\}^\{\-1\}, with the understanding thatHmH\_\{m\}andHm−1H\_\{m\}^\{\-1\}are applied elementwise tonn\-tuples of grid elements\.

To extendHmH\_\{m\}to all ofK=\[0,1\]d×nK=\[0,1\]^\{d\\times n\}, we employ the following procedure: For a givenX=\(x1,…,xn\)∈KX=\(x\_\{1\},\\ldots,x\_\{n\}\)\\in K, \(i\)Rounding:Eachxjx\_\{j\}resides in a unique hypercube inℰmd\\mathcal\{E\}\_\{m\}^\{d\}\. Replacexjx\_\{j\}with the centroid of this cubeyj∈𝔹n​\[d,m\]y\_\{j\}\\in\\mathbb\{B\}^\{n\}\[d,m\]\. \(ii\)Sorting:find the permutationτ∈Sn\\tau\\in S\_\{n\}which sorts\(Hm−1​\(y1\),…,Hm−1​\(yn\)\)\\left\(H\_\{m\}^\{\-1\}\(y\_\{1\}\),\\ldots,H\_\{m\}^\{\-1\}\(y\_\{n\}\)\\right\)from small to large\. \(iii\)CanonizingApply the permutationτ\\tautoXX\. We note that ifXXcontains several points in the same hypercube, the permutationτ\\tauis not uniquely defined\. In this case, the order of points with the hypercube will be determined by lexicographical sorting\.

We now restate and prove our theorem regarding the covering number of the Hilbert canonization:

See[4\.3](https://arxiv.org/html/2605.11008#S4.Thmdefinition3)

###### Proof\.

We first begin by a reduction from coveringKKto covering the grid𝔹n​\[d,m\]\\mathbb\{B\}^\{n\}\[d,m\]For convenience we abbreviate𝔹n:=𝔹n​\[d,m\]\\mathbb\{B\}^\{n\}:=\\mathbb\{B\}^\{n\}\[d,m\]\. We note that for anyX∈KX\\in K, the matrixYYobtained by the rounding procedure satisfies‖X−Y‖∞≤2−m−1\\\|X\-Y\\\|\_\{\\infty\}\\leq 2^\{\-m\-1\}\. This stays true also after applying the canonization,‖cm​\(X\)−cm​\(Y\)‖∞≤2−m−1\\\|c\_\{m\}\(X\)\-c\_\{m\}\(Y\)\\\|\_\{\\infty\}\\leq 2^\{\-m\-1\}, since the canonization will apply the same permutation to bothXXandYY\. It follows that

𝒩​\(cm​\(K\),ρ∞,ϵ\)≤𝒩​\(cm​\(𝔹n\),ρ∞,ϵ−2−m−1\)\.\\mathcal\{N\}\\left\(c\_\{m\}\\left\(K\\right\),\\rho\_\{\\infty\},\\epsilon\\right\)\\leq\\mathcal\{N\}\\left\(c\_\{m\}\\left\(\\mathbb\{B\}^\{n\}\\right\),\\rho\_\{\\infty\},\\epsilon\-2^\{\-m\-1\}\\right\)\.It remains to bound the right\-hand side:

𝒩​\(cm​\(𝔹n\),ρ∞,ϵ−2−m−1\)\\displaystyle\\mathcal\{N\}\\left\(c\_\{m\}\\left\(\\mathbb\{B\}^\{n\}\\right\),\\rho\_\{\\infty\},\\epsilon\-2^\{\-m\-1\}\\right\)=𝒩​\(Hm∘sort∘Hm−1​\(𝔹n\),ρ∞,ϵ−2−m−1\)\\displaystyle=\\mathcal\{N\}\\left\(H\_\{m\}\\circ\\mathrm\{sort\}\\circ H\_\{m\}^\{\-1\}\\left\(\\mathbb\{B\}^\{n\}\\right\),\\rho\_\{\\infty\},\\epsilon\-2^\{\-m\-1\}\\right\)≤\(∗\)𝒩​\(sort∘Hm−1​\(𝔹n\),ρ∞,δ\)\\displaystyle\\stackrel\{\{\\scriptstyle\(\*\)\}\}\{\{\\leq\}\}\\mathcal\{N\}\\left\(\\mathrm\{sort\}\\circ H\_\{m\}^\{\-1\}\\left\(\\mathbb\{B\}^\{n\}\\right\),\\rho\_\{\\infty\},\\delta\\right\)≤𝒩​\(sort​\(\[0,1\]n\),ρ∞,δ\)\\displaystyle\\leq\\mathcal\{N\}\\left\(\\mathrm\{sort\}\\left\(\[0,1\]^\{n\}\\right\),\\rho\_\{\\infty\},\\delta\\right\)≤\(∗∗\)\(n\+⌈1/\(2​δ\)⌉−1n\)\\displaystyle\\stackrel\{\{\\scriptstyle\(\*\*\)\}\}\{\{\\leq\}\}\\binom\{n\+\\lceil 1/\(2\\delta\)\\rceil\-1\}\{n\}Where \(\*\) follows from the Holder properties ofHmH\_\{m\}proven in Lemma[A\.5](https://arxiv.org/html/2605.11008#A1.Thmdefinition5), and \(\*\*\) follows from the fact that one\-dimensional sorting is an isometry \(Proposition[3\.3](https://arxiv.org/html/2605.11008#S3.Thmdefinition3)\) and the upper bound on the covering number of\[0,1\]n/𝔾\[0,1\]^\{n\}/\\mathbb\{G\}\(Lemma[4\.2](https://arxiv.org/html/2605.11008#S4.Thmdefinition2)\)\. ∎

###### Lemma A\.6\.

BeK=\[0,1\]n×dK=\[0,1\]^\{n\\times d\}andϵ=12⋅k\>0\\epsilon=\\frac\{1\}\{2\\cdot k\}\>0such thatk∈ℕk\\in\\mathbb\{N\}\. Then𝒩​\(K,d∞,ϵ\)=1\(2​ϵ\)n​d\\mathcal\{N\}\(K,d\_\{\\infty\},\\epsilon\)=\\frac\{1\}\{\(2\\epsilon\)^\{nd\}\}

###### Proof\.

It’s easy to see that each line we can cover using12⋅ϵ\\frac\{1\}\{2\\cdot\\epsilon\}ϵ\\epsilonsized lines, so using the product of size covering forn⋅dn\\cdot dtimes we get a covering forKKthat uses1\(2​ϵ\)n​d\\frac\{1\}\{\(2\\epsilon\)^\{nd\}\}balls\. On the other hand, denote byFFsome other covering\. Note thatK⊆∪c∈FB​\(c,ϵ\)K\\subseteq\\cup\_\{c\\in F\}B\(c,\\epsilon\)So1=μ​\(K\)≤μ​\(∪c∈FB​\(c,ϵ\)\)≤∑c∈Fμ​\(B​\(c,ϵ\)\)=\|F\|⋅\(2​ϵ\)n⋅d1=\\mu\(K\)\\leq\\mu\(\\cup\_\{c\\in F\}B\(c,\\epsilon\)\)\\leq\\sum\_\{c\\in F\}\\mu\(B\(c,\\epsilon\)\)=\|F\|\\cdot\(2\\epsilon\)^\{n\\cdot d\}so1\(2⋅ϵ\)n​d≤\|F\|\\frac\{1\}\{\(2\\cdot\\epsilon\)^\{nd\}\}\\leq\|F\|\. Thus, we got the lower bound and the upper bound as desired\. ∎

## Appendix BAdditional Experiments

### B\.1Laplacian Eigenvector Sign Ambiguity: Augmentation vs\. Canonization onogbg\-molpcba

This appendix presents an additional empirical comparison between deterministic canonization and training\-time random group augmentation, on a setting distinct from the main\-text experiments and complementary to the PCA experiment of Section[4](https://arxiv.org/html/2605.11008#S4)\. The goal is to ask whether the empirical behavior of these two strategies on a standard graph property prediction benchmark is qualitatively consistent with our Proposition[2\.2](https://arxiv.org/html/2605.11008#S2.Thmdefinition2)and the covering\-number inequality \([2](https://arxiv.org/html/2605.11008#S2.E2)\); we do not claim a quantitative test of the inequality, since the test AP on a finite\-sample\-trained GIN mixes optimization, capacity, and generalization effects that the inequality does not isolate\. The experiment is also consistent with the empirical observation cited in the Related Work that randomized SMILES can outperform canonized SMILES\[[1](https://arxiv.org/html/2605.11008#bib.bib13),[3](https://arxiv.org/html/2605.11008#bib.bib11)\]\.

#### Module\.

We study graph property prediction onogbg\-molpcbausing a55\-layer Graph Isomorphism Network \(GIN\) with Laplacian positional encoding \(LapPE\)\. For each graphGGwith normalized LaplacianLGL\_\{G\}, we compute thekksmallest non\-trivial eigenpairs\{\(λi,vi\)\}i=1k\\\{\(\\lambda\_\{i\},v\_\{i\}\)\\\}\_\{i=1\}^\{k\}and concatenate the eigenvectors as a node\-level positional encoding\. The resulting LapPE is invariant to graph isomorphism only up to the eigenvector*ambiguity group*

𝔾LapPE=∏i:mi=1\{−1,\+1\}×∏i:mi\>1O​\(mi\),\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}\\;=\\;\\prod\_\{i\\,:\\,m\_\{i\}=1\}\\\{\-1,\+1\\\}\\,\\times\\,\\prod\_\{i\\,:\\,m\_\{i\}\>1\}O\(m\_\{i\}\),wheremim\_\{i\}denotes the multiplicity of eigenvalueλi\\lambda\_\{i\}: a sign flip is admissible for each simple eigenvalue and an arbitrary orthogonal change of basis is admissible within each multiplicity\-mim\_\{i\}block\. On molecular graphs, the spectrum is almost everywhere simple, so in practice𝔾LapPE\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}reduces to\{−1,\+1\}k\\\{\-1,\+1\\\}^\{k\}for almost all graphs\. The natural questions are then \(i\) whether to canonize this ambiguity, and \(ii\) whether augmentation\-based approximations of group averaging behave consistently with our framework\.

#### Three methods compared\.

We compare three strategies for resolving𝔾LapPE\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}, all implemented as faithful numpy ports of the reference torch code from the Laplacian canonization release of\[[20](https://arxiv.org/html/2605.11008#bib.bib18)\]\. The first,map\[[20](https://arxiv.org/html/2605.11008#bib.bib18)\]\(Maximal Axis Projection\), resolves a simple\-eigenvalue sign by forming the rank\-one projectorPi=ui​ui⊤P\_\{i\}=u\_\{i\}u\_\{i\}^\{\\top\}, grouping the coordinate indicesjjby the rounded column norms‖Pi​\[:,j\]‖\\\|P\_\{i\}\[:,j\]\\\|\(1414\-decimal rounding\), and walking these groups from smallest to largest norm to find the first group whose indicator vectorXX\(the sum of standard basis vectors over the group, plus a uniform shift\) gives‖Pi​X‖≠0\\\|P\_\{i\}X\\\|\\\!\\neq\\\!0; the canonical sign is thensign​\(ui⊤​X\)\\mathrm\{sign\}\(u\_\{i\}^\{\\top\}X\)\. For a multiplicity\-mim\_\{i\}block,mapformsP=Ui​Ui⊤P=U\_\{i\}U\_\{i\}^\{\\top\}, selects themim\_\{i\}*largest*\-norm coordinate groups, and builds the canonical orthonormal basis incrementally by projecting each group\-indicator vector onto the residual orthogonal complement \(with plain QR inside the complementary\-space step\)\. The second,oap\[[19](https://arxiv.org/html/2605.11008#bib.bib2)\]\(Orthogonalized Axis Projection\), differs frommapin two places: the sign step rounds the column norms to66decimals \(coarser tie\-grouping under near\-degenerate norms\), and the basis step replaces the largest\-norm groupings ofmapby a hash\-based grouping of columns and uses signed\-QR in the complementary\-space iteration\. The third,random\_augmented, flips the sign of each eigenvector with probability1/21/2at every training\-time forward pass, so the network sees a uniformly random element of\{−1,\+1\}k\\\{\-1,\+1\\\}^\{k\}at each step\. This is a finite\-group augmentation that approximates group averaging over𝔾LapPE\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}without summing over its2k2^\{k\}elements explicitly\. At evaluation time therandom\_augmentedmodel is fed the deterministic eigsh\-sign convention from the cache \(no fresh random draws\), yielding a single deterministic prediction per graph; this is the number reported in therandom\_augmentedcolumn of Table[5](https://arxiv.org/html/2605.11008#A2.T5)\. Note that this is an*adversarial*test\-time deployment of an averaging\-trained model \(it scores a single point of the orbit rather than the orbit mean\); theaug\-KKcolumn reports the alternative Reynolds\-style sample ofKKorbit elements, and we discuss the gap between the two below\.

#### Test\-time averagingaug\-KK\.

Forrandom\_augmentedmodels we additionally evaluate a test\-time*Reynolds\-style*averaged metric\. At evaluation, for each test graph we drawKKindependent group elementsg1,…,gK∈𝔾LapPEg\_\{1\},\\ldots,g\_\{K\}\\in\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}\(eachgjg\_\{j\}a sign\-flip pattern on the simple\-eigenvalue subspace and a Haar\-random orthogonal matrix on each multiplicity\-mmblock\); we forward\-pass the model on each sign\-flipped LapPE; we average the predicted per\-task probabilities; and we score the averaged predictions with the official OGB AP evaluator\. We useK=8K\\\!=\\\!8atk=3k\\\!=\\\!3\(which approximately covers\{−1,\+1\}3\\\{\-1,\+1\\\}^\{3\}under uniform i\.i\.d\. sampling:88draws cover∼65%\\sim\\\!65\\%of the88\-element group on average\) andK=16K\\\!=\\\!16atk∈\{8,16\}k\\\!\\in\\\!\\\{8,16\\\}\(sparse Monte\-Carlo over\{−1,\+1\}8\\\{\-1,\+1\\\}^\{8\}and\{−1,\+1\}16\\\{\-1,\+1\\\}^\{16\}respectively\)\. Onogbg\-molpcbathe spectrum is almost everywhere simple, so the multiplicity\-block factor∏iO​\(mi\)\\prod\_\{i\}O\(m\_\{i\}\)is degenerate in practice and thegjg\_\{j\}samples reduce to sign\-flip patterns\. This is the empirical group\-averaging operator of Section[2](https://arxiv.org/html/2605.11008#S2), restricted to a finite sample\.

#### Hyperparameters and protocol\.

GIN,55layers, hidden dimh∈\{16,128,512\}h\\\!\\in\\\!\\\{16,128,512\\\}, dropout0\.50\.5, batch size3232, optimizer Adam with cosine learning\-rate schedule and initial rate10−310^\{\-3\},33seeds per cell \(\{0,1,2\}\\\{0,1,2\\\}\), eigval\-scaling off, eigenvector cache size1515\. Checkpoint selection: the model state at the epoch with the highest validation AP across all training epochs is saved \(ties broken by the earliest epoch\); the final test AP reported in Tables[5](https://arxiv.org/html/2605.11008#A2.T5)and[6](https://arxiv.org/html/2605.11008#A2.T6)is this checkpoint’s score on the OGB test split\. Software stack: PyTorch2\.10\.02\.10\.0, CUDA12\.812\.8,torch\_geometric2\.7\.02\.7\.0,numpy2\.2\.62\.2\.6,scipy1\.15\.31\.15\.3,ogb1\.3\.61\.3\.6\(OGB evaluator:Evaluator\(name=‘ogbg\-molpcba’\)\)\. Per\-arm seeding viatorch\.manual\_seed,numpy\.random\.seed, andrandom\.seed;cudnn\.deterministicis not enforced, so two runs at the same seed on different hardware are not guaranteed to be bit\-identical \(we observe the resulting variability absorbed into the seed std at the magnitudes shown in Table[6](https://arxiv.org/html/2605.11008#A2.T6)\)\. Standard deviations reported below are sample standard deviations \(ddof=1\\mathrm\{ddof\}\\\!=\\\!1\)\. Code, rawresults\.jsonfiles, theorbit\_stability\.jsonfiles behind Table[7](https://arxiv.org/html/2605.11008#A2.T7), and reproduction commands are released at the project repository\.

#### Compute budget\.

Atk∈\{3,8\}k\\\!\\in\\\!\\\{3,8\\\}all three arms are trained for200200epochs with early\-stop patience1515\. Atk=16k\\\!=\\\!16, where validation AP forrandom\_augmentedcontinues to improve well past200200epochs, all three arms are trained for500500epochs with early stopping disabled \(patience999999\)\. Tables[5](https://arxiv.org/html/2605.11008#A2.T5)and[6](https://arxiv.org/html/2605.11008#A2.T6)therefore report a matched\-compute comparison at every\(k,h\)\(k,h\)cell\. Both schedules are released in the project repository\.

#### Results\.

Table[5](https://arxiv.org/html/2605.11008#A2.T5)reports mean test AP at all nine\(k,h\)\(k,h\)cells for the three methods, alongside theaug\-KKtest\-time\-averaged metric forrandom\_augmented\. Table[6](https://arxiv.org/html/2605.11008#A2.T6)zooms in on thek=16k\\\!=\\\!16slice with sample standard deviations\.

Table 5:Mean test AP onogbg\-molpcbaat matched compute,33seeds per cell \(k∈\{3,8\}k\\\!\\in\\\!\\\{3,8\\\}at200200epochs / patience1515,k=16k\\\!=\\\!16at500500epochs / patience999999for all three arms\)\. Per\-cell row\-best margins are mostly within seed std and none survives a Bonferroni correction across the nine cells atα=0\.05\\alpha\\\!=\\\!0\.05\(see Table[6](https://arxiv.org/html/2605.11008#A2.T6)for thek=16k\\\!=\\\!16Welchpp\-values\); the bolding, therefore, marks the nominal row\-best mean, not a significance claim\. Theaug\-KKcolumn reports test\-time averaging overKKambiguity\-group draws applied to therandom\_augmented\-trained model \(see text\)\.Table 6:Matched\-computek=16k\\\!=\\\!16slice of Table[5](https://arxiv.org/html/2605.11008#A2.T5): all three arms trained for500500epochs with patience999999\. Reported are mean±\\pmsample standard deviation over33seeds, plus the two\-sided Welchtt\-testpp\-value for the row\-best mean against the row\-second\-best mean\. None of the three row\-best margins reachesp<0\.05p\\\!<\\\!0\.05, and after a Bonferroni correction across the nine cells in Table[5](https://arxiv.org/html/2605.11008#A2.T5)\(α=0\.05/9≈0\.0056\\alpha\\\!=\\\!0\.05/9\\\!\\approx\\\!0\.0056\) none of the row\-best margins anywhere in the appendix is significant\.
#### Test\-time averagingΔ\\Deltais small\.

Across all nine\(k,h\)\(k,h\)cells the difference between the single\-pass deterministic\-eigsh evaluation of therandom\_augmentedmodel and itsaug\-KKaveraged evaluation \(Table[5](https://arxiv.org/html/2605.11008#A2.T5), right\-most two columns\) lies in\[−0\.0002,\+0\.0005\]\[\-0\.0002,\+0\.0005\]AP, within the seed noise at every cell\. Theaug\-KKΔ\\Deltais an aggregate metric, not a direct measurement of model invariance; we test invariance directly in the next paragraph\.

#### Per\-graph orbit stability\.

For each test graph we drawK=64K\\\!=\\\!64i\.i\.d\. elementsgjg\_\{j\}of the sign\-flip subgroup of𝔾LapPE\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}, forward\-pass the trainedrandom\_augmentedmodel on eachgj​\(LapPE\)g\_\{j\}\(\\mathrm\{LapPE\}\), and record the per\-task predicted probability\. The resulting per\-graph spread \(max−\-min predicted probability across theK=64K\\\!=\\\!64draws\) is a direct measure of how much the model’s output varies across orbit elements\. Aggregating across all43,79343\{,\}793ogbg\-molpcbatest graphs and128128tasks, at hidden dimh=128h\\\!=\\\!128and over all33training seeds we observe a clean monotone scaling of per\-graph spread withkk\(Table[7](https://arxiv.org/html/2605.11008#A2.T7)\): mean per\-graph spread is0\.0013±0\.00050\.0013\\\!\\pm\\\!0\.0005atk=3k\\\!=\\\!3,0\.0027±0\.00120\.0027\\\!\\pm\\\!0\.0012atk=8k\\\!=\\\!8, and0\.0048±0\.00080\.0048\\\!\\pm\\\!0\.0008atk=16k\\\!=\\\!16\. This is qualitatively consistent with\|𝔾LapPE\|=2k\|\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}\|\\\!=\\\!2^\{k\}: more orbit elements give more room to vary, and the model is closer to be invariant for smallerkk\. The within\-seed standard deviation of the OGB AP across theK=64K\\\!=\\\!64orbit elements is at most0\.00060\.0006atk=16k\\\!=\\\!16and below0\.00040\.0004atk∈\{3,8\}k\\\!\\in\\\!\\\{3,8\\\}, in every case much smaller than the between\-seed standard deviation of the same quantity \(0\.00620\.0062atk∈\{3,8\}k\\\!\\in\\\!\\\{3,8\\\},0\.00150\.0015atk=16k\\\!=\\\!16\): the model’s output varies less when we resample the ambiguity group at fixed seed than when we change the training seed at fixed ambiguity element\. The Reynolds\-style \(mean\-probability\) AP across theK=64K\\\!=\\\!64draws is0\.1923±0\.00170\.1923\\\!\\pm\\\!0\.0017atk=16k\\\!=\\\!16/h=128h\\\!=\\\!128, within seed noise of the matched\-compute single\-pass number \(0\.1916±0\.00090\.1916\\\!\\pm\\\!0\.0009forrandom\_augmentedand0\.1923±0\.00050\.1923\\\!\\pm\\\!0\.0005for matched\-computemapat the same cell\)\. We read this as direct evidence that therandom\_augmentedmodel is approximately𝔾LapPE\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}\-invariant on the test distribution at the configurations tested, with the residual non\-invariance growing predictably with the size of the ambiguity group\. Along the hidden\-dim axis atk=16k\\\!=\\\!16\(Table[7](https://arxiv.org/html/2605.11008#A2.T7), lower block,33seeds per cell\), the per\-graph spread is non\-monotone inhh:0\.0032±0\.00150\.0032\\\!\\pm\\\!0\.0015ath=16h\\\!=\\\!16,0\.0048±0\.00080\.0048\\\!\\pm\\\!0\.0008ath=128h\\\!=\\\!128,0\.0033±0\.00040\.0033\\\!\\pm\\\!0\.0004ath=512h\\\!=\\\!512\. We attribute this to a capacity floor and a capacity ceiling: ath=16h\\\!=\\\!16the network underfits \(test AP≈0\.07\\approx\\\!0\.07, near a constant\-prediction baseline\) so per\-graph predictions are bunched near a few values and have little room to spread; ath=512h\\\!=\\\!512the network has enough capacity to learn invariance well; the middle caseh=128h\\\!=\\\!128shows the largest residual non\-invariance, although theh=16h\\\!=\\\!16row’s larger seed std \(0\.00150\.0015\) suggests this conclusion is sensitive to which random initialization one looks at\.

Table 7:Per\-graph orbit stability of the trainedrandom\_augmentedmodel underK=64K\\\!=\\\!64uniform i\.i\.d\. draws from the sign\-flip subgroup of𝔾LapPE\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}, evaluated on theogbg\-molpcbatest set \(43,79343\{,\}793graphs×\\times128128tasks\)\. All entries are the mean±\\pmsample standard deviation over the33training seeds\. “Per\-graph spread” is the mean over graphs and tasks of\(max−min\)\(\\max\\\!\-\\\!\\min\)predicted probability across theK=64K\\\!=\\\!64orbit elements\. “Within\-seed AP std” is the standard deviation of the OGB AP scored on each single orbit draw, averaged across seeds\. “Reynolds AP” scores the mean probability over theK=64K\\\!=\\\!64draws\. Per\-graph spread scales monotonically with\|𝔾LapPE\|=2k\|\\mathbb\{G\}\_\{\\mathrm\{LapPE\}\}\|\\\!=\\\!2^\{k\}\.
#### Reading the tables\.

Reading along the hidden\-dim axis at fixedkkin Table[5](https://arxiv.org/html/2605.11008#A2.T5), the row\-best AP grows monotonically withhhat everykk, and the row\-best arm is most oftenrandom\_augmented\(six of nine cells\), withmapnominally row\-best atk=16k\\\!=\\\!16/h∈\{16,128\}h\\\!\\in\\\!\\\{16,128\\\}andoapnominally row\-best atk=3k\\\!=\\\!3/h=512h\\\!=\\\!512\. The matched\-computek=16k\\\!=\\\!16slice in Table[6](https://arxiv.org/html/2605.11008#A2.T6)resolves these margins against seed variance: the smallest sample std on each row is consistently therandom\_augmentedstd \(0\.00080\.0008,0\.00090\.0009,0\.00260\.0026\), whilemapcarries the largest std at smallhh\(0\.00670\.0067ath=16h\\\!=\\\!16\)\. Under a two\-sided Welchtt\-test for unequal variances, the row\-best versus row\-second\-best margins atk=16k\\\!=\\\!16havepp\-values of0\.830\.83,0\.320\.32, and0\.480\.48ath=16,128,512h\\\!=\\\!16,128,512respectively, none significant\. Atk∈\{3,8\}k\\\!\\in\\\!\\\{3,8\\\}the row\-best margins are similar in magnitude or smaller, withrandom\_augmentedwinning five of six rows there but at margins of the same order as the seed std\. We therefore present Table[5](https://arxiv.org/html/2605.11008#A2.T5)as a nominal\-rank summary, not as a benchmark claim\.

#### Tie to the theory\.

Onogbg\-molpcbaat matched compute, with33seeds per cell, we did not detect a statistically significant difference between training\-time random sign\-flip augmentation and projection\-based deterministic canonization at any of the nine\(k,h\)\(k,h\)cells; this is qualitatively consistent with \([2](https://arxiv.org/html/2605.11008#S2.E2)\) of Proposition[2\.2](https://arxiv.org/html/2605.11008#S2.Thmdefinition2), which suggests that augmentation can match canonization on tasks where the covering number of𝔾\\mathbb\{G\}is small, but it is not a positive test of the inequality, since AP gaps of order10−310^\{\-3\}are dominated by optimization and seed noise rather than by the input\-space covering complexity, the inequality bounds\. Theaug\-KKcolumn moves AP by at most\+0\.0005\+0\.0005across the nine cells; this is consistent with the cached deterministic eigsh\-sign PE already lying close to the orbit\-mean the network was trained on, but is not itself a direct test of model invariance\.

### B\.2Explicit values of Covering Numbers

In this appendix, we compute the covering number bounds from Section[4](https://arxiv.org/html/2605.11008#S4)for specific values ofϵ,d,n\\epsilon,d,n\. We give bounds on the covering number of the quotient spaceK/𝔾K/\\mathbb\{G\}whenK=\[0,1\]d×nK=\[0,1\]^\{d\\times n\}and𝔾=Sn\\mathbb\{G\}=S\_\{n\}, as well as the covering number of the image ofKKunder the Hilbert canonization and the Lexsort canonization, and the covering number of the hypercube\[0,1\]d×n\[0,1\]^\{d\\times n\}which is known to bekn​dk^\{nd\}whenϵ=1/\(2​k\)\\epsilon=1/\(2k\)\(see proof in[A\.6](https://arxiv.org/html/2605.11008#A1.Thmdefinition6)\)\. We take characteristic values encountered in point cloud learning scenarios: point dimensions ared=3d=3, a very modest value ofϵ=1/6\\epsilon=1/6, andnnvarying between250250and20002000\. The results are shown in Table[B\.2](https://arxiv.org/html/2605.11008#A2.SS2)

Table 8:Comparison of covering number bounds as a function ofnnbetween the quotient space, the Hilbert and Lexsort canonizations, and the hypercubeWe note that our bounds for the quotient space and Hilbert canonizations are upper bounds, while the Lexsort bound is a lower bound\. Thus, the gap between the covering number of Lexsort and that of the other methods may be even larger than shown in the table\. We also note that despite the conservative choice ofϵ=1/6\\epsilon=1/6, all covering numbers in the table are enormous\. This picture may be over\-pessimistic, as the ‘true data distribution’ will be supported on a ‘small’ setK⊆\[0,1\]d×nK\\subseteq\[0,1\]^\{d\\times n\}of ‘realistic point clouds’\. An experiment estimating the covering number \(in fact, the coverage\) on the Modelnet40 dataset was shown in Table[1](https://arxiv.org/html/2605.11008#S5.T1)\.

## Appendix CExperimental Setup and Hyperparameter Details

### C\.1Experimental Setup for the covering number experiment

We applied the following preprocessing for*each*sample: we first sample256256points, then we shift it to the positive axis, and then divide by the maximum axis alongx,y,zx,y,z, obtaining all entries to be positive and between0and11\. The metric we considered on the point cloud space is

ρ𝔾​\(X,Y\)=1n​∑i=1n‖Xi−Yi‖2\.\\rho\_\{\\mathbb\{G\}\}\(X,Y\)=\\frac\{1\}\{n\}\\sum^\{n\}\_\{i=1\}\\\|X\_\{i\}\-Y\_\{i\}\\\|\_\{2\}\.The same metric is used for the two canonizations, and its quotientρ𝔾\\rho\_\{\\mathbb\{G\}\}is used for the quotient space\.

### C\.2Experiment setup: ModelNet40 classification

Table 9:Test accuracy of the Global MLP under varying training set sizes\. The performance gap between Hilbert and lexicographical sorting widens as data becomes scarcer, highlighting the regularizing effect of optimal canonization\.In the ModelNet40 classification tasks, we sampled from each input CAD model point cloud, which consists of10241024points, and applied an initial batched affine normalization to\[0,1\]3\[0,1\]^\{3\}\. The data is then ordered using one of three strategies: no canonization \(baseline\), lexicographical sorting \(i\.e\., sorting points sequentially by theirxx, thenyy, thenzzcoordinates\), or Hilbert curve sorting \(using a space size of2122^\{12\}\)\. Following ordering, the coordinates are projected into a higher\-dimensional space using Random Fourier Features to capture high\-frequency geometric details\. The feature vector is then passed through a bottlenecked MLP consisting of residual blocks with dimensions\[256,128,64\]\[256,128,64\], yielding the final classification logits\. The MLPs were trained for 100 epochs with a fixed batch size of 256, and were trained using AdamW\.

To ensure a rigorous and fair comparison between the different ordering strategies, we conducted comprehensive grid sweeps to identify the optimal hyperparameters for the uncanonized baseline \(ply\), lexicographical sorting \(lex\), and Hilbert curve sorting \(hilbert\)\. The specific optimal hyperparameters extracted from our sweeps and utilized for the final evaluation are detailed in Table[10](https://arxiv.org/html/2605.11008#A3.T10)\. The best hyperparameters for each model on ModlenNet40 are used for training the models on ModelNet10\.

Table 10:Optimal hyperparameters identified via grid search for each ordering strategy\.#### Robustness Across Seeds

To account for variance in weight initialization and optimization dynamics, we evaluated the final optimal configurations across 5 independent random seeds\. The averaged test accuracy and standard deviations are reported in Table[11](https://arxiv.org/html/2605.11008#A3.T11)\. The results demonstrate that the Hilbert curve canonization not only achieves the highest average accuracy but also exhibits the lowest variance across different initializations\.

Table 11:Test accuracy across 5 independent random seeds using the optimal hyperparameters\. Hilbert canonization yields both higher performance and greater stability\.

### C\.3Experimental Setup: Modelnet 40 Data scarcity experiment

To simulate data scarcity, we subsample the ModelNet40 training set using strides of 1, 2, 4, and 8, resulting in training subsets of 9840, 4920, 2460, and 1230 samples, respectively\. Results are shown in Table[9](https://arxiv.org/html/2605.11008#A3.T9)\.

### C\.4Experimental Setup: PCA group averaging

We use a DeepSets architecture that is fully invariant to point order permutations but strictly non\-invariant to the\{−1,1\}3\\\{\-1,1\\\}^\{3\}sign flips\. We intentionally employ a lightweight, low\-capacity version of this architecture\. A highly over\-parameterized model might achieve higher absolute accuracies, potentially masking the generalization gaps we aim to observe; by constraining the network capacity, we ensure that any performance differences are strictly attributable to the chosen alignment strategy\. Each input point cloud consists of10241024points from the ModelNet40 dataset\. We compare four approaches: \(1\)Pure PCA, which leaves the sign ambiguity unresolved \(acting as a baseline or potentially*poor*canonization due to eigensolver discontinuities\); \(2\)Skewness, a deterministic canonization mappingc:K→Kc:K\\to Kthat resolves the ambiguity by orienting axes based on the third moment; \(3\)Frame Averaging, an explicitly𝔾\\mathbb\{G\}\-invariant model constructed by pooling the network’s predictions across all88elements of the group using log\-sum\-exp; and \(4\)Random Frame, which applies elements of𝔾\\mathbb\{G\}as random data augmentation during training\. All models are optimized using Adam for 1600 epochs and evaluated across 5 different random seeds\.

![Refer to caption](https://arxiv.org/html/2605.11008v1/x3.png)Figure 3:Rotation Symmetry Accuracy\.The curves illustrate the comparative performance of the four approaches: Random Frame \(blue\), Frame Averaging \(red\), Skewness \(orange\), and the Pure PCA baseline \(green\)\. Frame Averaging achieves the strongest overall performance\. While Random Frame and Skewness reach comparable final results—both significantly outperforming the Pure PCA baseline—the augmentation\-based Random Frame method exhibits a noticeably slower convergence rate than deterministic canonization via Skewness\.
### C\.5Experimental Setup: Rotated MNIST

In the last set of the experiments, we compare the performance of several image rotation invariant models\. We considered three models: simple CNN, CN\(p4\)\-CNN, and AvgCNN\. The backbone for all three models is the same and additional layers are added for CN\(p4\)\-CNN, AvgCNN\. In order to have the same setting, including the data split we rerun all models, including CN\(p4\)\-CNN\. We used learning rate of1​e−31e^\{\-3\}and a batch size of256256and an AdamW optimizer\. Each experiment is run for55different seeds, and mean and standard deviation are reported\.