SIEVE: Overcoming Topological Obstruction in Equivariant Self-Supervised Learning

# SIEVE: Overcoming Topological Obstruction in Equivariant Self-Supervised Learning
Source: [https://arxiv.org/html/2610.21087](https://arxiv.org/html/2610.21087)
Jongann Lee††thanks:Equal contribution\.Sun Woo Park11footnotemark:1Affiliation:Max Planck Institute of MathematicsAffiliation:Bonn, GermanyEmail:[s\.park@mpim\-bonn\.mpg\.de](mailto:)Yun Young Choi††thanks:Corresponding author:young@solverx\.ai\.Affiliation:SolverXAffiliation:Seoul, Republic of KoreaEmail:[young@solverx\.ai](mailto:)

###### Abstract

We identify atheoretical incompatibilitybetween 3D point cloud masked autoencoding and SE\(3\)\-equivariant networks\. We prove that standard masking triggers a*topological collapse*, effectively forcing the model to hallucinate orientation without a reference frame\. To resolve this, we introduceSIEVE\(Scalar Invariant Extraction Via Equivariance\), an SE\(3\)\-equivariant self\-supervised framework that compresses point cloud geometry into invariant scalar embeddings\. At its core is the*spherical projection mask*, which preserves global pose while obfuscating local semantics, sidestepping the obstruction by construction\. Experiments confirm that SIEVE circumvents the predicted collapse where constant and noise masks fail\. On downstream point cloud tasks, the resulting embeddings improve over coordinate\-based baselines and hand\-crafted geometric descriptors, with the largest gains on generalization to unseen shape categories\. Code is available at[https://github\.com/voltwin\-dev/SIEVE\_NeurIPS26](https://github.com/voltwin-dev/SIEVE_NeurIPS26)

## 1Introduction

Point clouds serve as the fundamental data structure for 3D understanding in various domains, including robotics\[[18](https://arxiv.org/html/2610.21087#bib.bib2)\]and autonomous driving\[[6](https://arxiv.org/html/2610.21087#bib.bib1)\]\. Unlike 2D images, where*position*\(the pixel grid\) and*content*\(RGB values\) are naturally disentangled, point clouds present a unique challenge: the geometry*is*the feature\. The coordinate𝐱∈ℝ3\\mathbf\{x\}\\in\\mathbb\{R\}^\{3\}encodes both the local structure and the global spatial location\. This entanglement necessitates representations that are not only topologically rich, a goal also pursued in graph representation learning\[[13](https://arxiv.org/html/2610.21087#bib.bib45),[12](https://arxiv.org/html/2610.21087#bib.bib46)\], but also consistent under global rigid\-body transformations\.

This separation naturally motivates the use of SE\(3\)\-equivariant neural networks, whose architectures are constrained to ensure that a rigid\-body transformation applied to the input results in an identical transformation of the output\[[39](https://arxiv.org/html/2610.21087#bib.bib14),[4](https://arxiv.org/html/2610.21087#bib.bib15),[16](https://arxiv.org/html/2610.21087#bib.bib12),[5](https://arxiv.org/html/2610.21087#bib.bib9),[3](https://arxiv.org/html/2610.21087#bib.bib16)\]\. However, because these models cannot rely on absolute coordinate “shortcuts”\[[43](https://arxiv.org/html/2610.21087#bib.bib24),[47](https://arxiv.org/html/2610.21087#bib.bib8)\], they often face more difficult optimization landscapes than their non\-equivariant counterparts\[[49](https://arxiv.org/html/2610.21087#bib.bib23)\]\. We propose self\-supervised learning \(SSL\) as the path forward: the equivariant pipeline can distill geometric structure into SE\(3\)\-invariant embeddings that enrich downstream representations, sidestepping the need to impose equivariance on the downstream model\.

However, we identify a fundamental theoretical flaw when applying standard masking strategies to SE\(3\)\-equivariant neural networks\. Conventional SSL methods replace masked features with a learnable, isotropic token\[[24](https://arxiv.org/html/2610.21087#bib.bib3),[50](https://arxiv.org/html/2610.21087#bib.bib4),[32](https://arxiv.org/html/2610.21087#bib.bib5)\]\. Utilizing the theory of pullbacks of vector bundles, we demonstrate that the use of such constant tokens induces a topological obstruction, leading to a collapse in the vector pipeline that inhibits the reconstruction of the input point cloud\. We empirically validate this obstruction through a controlled experiment on a synthetic dataset, highlighting the catastrophic failure of isotropic masking in equivariant settings\.

To resolve this, we propose the equivariant SSL frameworkSIEVE\(Scalar Invariant Extraction Via Equivariance\)\. Instead of masking information completely, SIEVE employs a*spherical projection masking strategy*that*sieves*the information: it projects the ‘where’ \(global pose\) onto the sphere while forcing the encoder to compress the ‘what’ \(intrinsic semantics\) into invariant scalars\. This masking scheme is a direct consequence of our vector bundle analysis: it materializes the pullback of non\-trivial vector bundles that standard masking cannot reconstruct\. We validate these embeddings on several downstream point cloud tasks, where SIEVE improves over coordinate\-based baselines and hand\-crafted descriptors with the largest gains appearing on generalization to unseen categories\.

## 2Theoretical Framework

![Refer to caption](https://arxiv.org/html/2610.21087v1/figures/SIEVE_Comparison_Random.png)

Figure 1:Random masking causes representational collapse; spherical projection masking does not\.Random masking functions ignore subtle geometric data inherent in point clouds\. This imposes topological obstructions in reconstructing the desired non\-trivial vector bundles characterizing the underlying geometry of point clouds, as demonstrated in Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)\. As a result, representational collapse can occur as shown in Section[4\.2](https://arxiv.org/html/2610.21087#S4.SS2)\. SIEVE utilizes a spherical projection mask, thereby possessing capabilities in reconstructing a class of pullbacks of vector bundles defined over a wide variety of 2 and 3\-dimensional manifolds as demonstrated in Theorem[3\.1](https://arxiv.org/html/2610.21087#S3.Thmtheorem1)\.### 2\.1Preliminaries: Vector Bundles

Effective analysis of properties of 3D point clouds rests upon understanding entanglement betweenpositionandcontent\. For this purpose, we seek suitable mathematical concepts or frameworks that encompass both topological features of 3D point clouds as well as contents measured over such point clouds as real vectors\. We pay particular focus on using vector bundles from algebraic topology among many possible candidates that can harbor the subtlety of analyzing such entanglements\.

###### Definition 2\.1\.

LetXXbe a topological space\. Annn\-dimensional vector bundleπ:V→X\\pi:V\\to Xis a topological space satisfying the following conditions\.\(1\)The mapπ\\piis a continuous surjection\.\(2\)For everyx∈Xx\\in X, the fiberπ−1​\(x\)\\pi^\{\-1\}\(x\)is isomorphic toℝn\\mathbb\{R\}^\{n\}\.\(3\)For everyx∈Xx\\in X, there exists an open neighborhoodU⊂XU\\subset Xsuch that satisfies the following three conditions\.\(3a\)There exists a homeomorphismφ:U×ℝn→π−1​\(U\)\\varphi:U\\times\\mathbb\{R\}^\{n\}\\to\\pi^\{\-1\}\(U\)\.\(3b\)For everyv∈ℝnv\\in\\mathbb\{R\}^\{n\}, we haveπ⁡\(φ⁡\(x,v\)\)=x\\pi\(\\varphi\(x,v\)\)=x\.\(3c\)Every isomorphismℝn→π−1​\(x\)\\mathbb\{R\}^\{n\}\\to\\pi^\{\-1\}\(x\)can be rewritten as an invertiblen×nn\\times nreal matrix\.

Interested readers may refer to Appendix[A\.1](https://arxiv.org/html/2610.21087#A1.SS1)for a detailed motivation for using the notion of vector bundles to formulate strengths and capabilities of deep learning algorithms\. Given a vector bundleπ:V→X\\pi:V\\to X, a continuous functionf:X→Vf:X\\to Vsuch thatπ⁡\(f⁡\(x\)\)=x\\pi\(f\(x\)\)=xfor everyx∈Xx\\in Xis called a global section\. With these definitions, we propose the following topological reformulation of 3D point clouds, up to some perturbations or negligible errors\.\(1\)3D point cloudscorrespond to points on a smooth compact orientable 3\-dimensionalmanifoldYYembedded insideℝ3\\mathbb\{R\}^\{3\}, whose boundaryX:=∂YX:=\\partial Yis a smooth compact orientable surface embedded insideℝ3\\mathbb\{R\}^\{3\}\.\(2\)Because 3D Point clouds consist of discrete set of points, we may require all the point clouds to lie over a complement of a small tubular neighborhood of the wedge sum of some circles∨i=1gS1\\vee\_\{i=1\}^\{g\}S^\{1\}insideYY, whereggis the genus ofXX\. Such a space, denoted asY′Y^\{\\prime\}, is homotopy equivalent toX×IX\\times I\.\(3\)Measurementsconsisting ofnn\-dimensional feature vectors over each point cloudx∈Y′x\\in Y^\{\\prime\}corresponds toglobal sectionsof annn\-dimensional vector bundleV→Y′V\\to Y^\{\\prime\}\.

It is natural to understand vector bundles that can harbor non\-trivial entanglements between geometry and intrinsic semantics which 3D point clouds contain\. This brings us to the notion of trivial vector bundles over topological spaces\.

###### Definition 2\.2\.

LetXXbe a topological space\. Annn\-dimensional trivial vector bundleV→XV\\to Xis a vector bundle overXXwhich is isomorphic to the vector bundleX×ℝn→XX\\times\\mathbb\{R\}^\{n\}\\to X\. Any continuous functionf:X→ℝnf:X\\to\\mathbb\{R\}^\{n\}is a global section of the trivial vector bundle\.

The geometry ofXXdoes not impose any restrictions on the image of the global sectionf:X→ℝnf:X\\to\\mathbb\{R\}^\{n\}\. We hence obtain the following insight:trivial vector bundlesover topological spaces are suitable for analyzing data whosepositionandcontentaredisentangled\. We also observe that vector bundles over topological spaces which arenon\-trivialare potential candidates that can illuminateentanglementsbetween geometry and intrinsic semantics of 3D point clouds\. In light of these insights, we ask the following question:What are effective ways to learn global sections of real oriented non\-trivial vector bundles over a given topological spaceYY\(orY′Y^\{\\prime\}\)?

### 2\.2Interpretation: Simplifications with Projections toS2S^\{2\}using Pullback of Vector Bundles

To understand the question, we analyze vector bundles overYY\(orY′Y^\{\\prime\}\) that may be underlying spaces for 3D point clouds\. The problem of classifying real oriented vector bundles over smooth manifolds, however, is a highly non\-trivial problem\. Nevertheless, we can construct some of these desired vector bundles by using pullbacks of vector bundles between topological spaces\.

###### Definition 2\.3\.

Given a continuous mapf:X→Yf:X\\to Y, and a vector bundleπ:V→Y\\pi:V\\to Y, the pullback ofVVis the vector bundlef∗​π:f∗​V→Xf^\{\*\}\\pi:f^\{\*\}V\\to X, wheref∗​V:=\{\(x,e\)∈X×V:f⁡\(x\)=π⁡\(e\)\}\.f^\{\*\}V:=\\left\\\{\(x,e\)\\in X\\times V:f\(x\)=\\pi\(e\)\\right\\\}\.The fiber off∗​Vf^\{\*\}Vatx∈Xx\\in Xis isomorphic to the fiber ofVVatf⁡\(x\)∈Yf\(x\)\\in Y\.

Without loss of generality, we consider an auxiliary \(possibly unknown\)nn\-dimensional vector bundleVY→Y′V\_\{Y\}\\to Y^\{\\prime\}underlying 3D point clouds, wheren≥3n\\geq 3\. BecauseY′Y^\{\\prime\}andXXare homotopy equivalent, there exists a bijection of isomorphism classes of vector bundles ofY′Y^\{\\prime\}andXX\(see Corollary 1\.8 of\[[23](https://arxiv.org/html/2610.21087#bib.bib35)\]\)\. Hence, we can identify thenn\-dimensional vector bundleVY→Y′V\_\{Y\}\\to Y^\{\\prime\}with annn\-dimensional vector bundleVX→XV\_\{X\}\\to X\. \(This is one of the reasons why we choseY′Y^\{\\prime\}instead ofYY, because we may reduce the dimension of the underlying topological space of our interest from33to22\.\)

We can use pullbacks of vector bundles to construct trivial and non\-trivial vector bundles overXX\. Given any continuous mapf:X→S2f:X\\to S^\{2\}, we can pullback somenn\-dimensional vector bundleV→S2V\\to S^\{2\}to construct a vector bundlef∗​V→Xf^\{\*\}V\\to X\. Thankfully, the classification of real oriented vector bundles overS2S^\{2\}is very well understood, whose proof is in Appendix[A\.2](https://arxiv.org/html/2610.21087#A1.SS2)\.

###### Theorem 2\.4\.

LetV→S2V\\to S^\{2\}be a real oriented vector bundle of dimensionn≥3n\\geq 3\. SupposeVVis a non\-trivial vector bundle, i\.e\.VVcontains the tautological bundleπt:T→S2\\pi\_\{t\}:T\\to S^\{2\}as a sub\-vector bundle \(see Appendix[A\.2](https://arxiv.org/html/2610.21087#A1.SS2)for its definition\)\. ThenVVis isomorphic to theuniquenon\-trivial vector bundle of dimensionnn, whose fibers are isomorphic toℝn−2⊕πt−1​\(x\)\\mathbb\{R\}^\{n\-2\}\\oplus\\pi\_\{t\}^\{\-1\}\(x\)for everyx∈S2x\\in S^\{2\}\.

There are two isomorphism classes of vector bundles overXXwe can construct using pullback of vector bundles ofS2S^\{2\}: the trivial vector bundleX×ℝn→XX\\times\\mathbb\{R\}^\{n\}\\to X, obtained as a pullback of the trivial vector bundleS2×ℝn→S2S^\{2\}\\times\\mathbb\{R\}^\{n\}\\to S^\{2\}, and a non\-trivial vector bundleVX→XV\_\{X\}\\to X, obtained as a pullback of the unique non\-trivial vector bundleV→S2V\\to S^\{2\}\.

### 2\.3Applications: Topological Obstructions in Constant and Randomized Noise Masking

The main application of our paper will focus on effectively approximating 3D point clouds, which shall be regarded as a set of discrete points𝒳\\mathcal\{X\}overY′Y^\{\\prime\}\. Given an injectioni:𝒳↪Y′i:\\mathcal\{X\}\\hookrightarrow Y^\{\\prime\}, we consider the restriction of the global sectionf∘i:𝒳→Y′→VY′f\\circ i:\\mathcal\{X\}\\to Y^\{\\prime\}\\to V\_\{Y\}^\{\\prime\}\. We regard the problem of approximating 3D point clouds as a problem of approximating the restriction of the global sectionf∘if\\circ i\.

To make the notion of approximationf∘if\\circ irigorous, it suffices to clarify what it means for anSE⁡\(3\)\\mathrm\{SE\}\(3\)\-equivariant neural networkΨ\\Psito effectively approximate a global sectionf:Y′→VY′f:Y^\{\\prime\}\\to V\_\{Y\}^\{\\prime\}\. To do this, we identify a global sectionf:Y′→VY′f:Y^\{\\prime\}\\to V\_\{Y\}^\{\\prime\}of anNN\-dimensional vector bundleVY′V\_\{Y\}^\{\\prime\}over a33\-dimensional manifoldY′Y^\{\\prime\}constructed in Section[2\.2](https://arxiv.org/html/2610.21087#S2.SS2)as a restriction of the continuous functionF:ℝ3→ℝmNF:\\mathbb\{R\}^\{3\}\\to\\mathbb\{R\}^\{m\_\{N\}\}toY′Y^\{\\prime\}\(or any connected compact subspace ofℝ3\\mathbb\{R\}^\{3\}homeomorphic toY′Y^\{\\prime\}\) that is equivariant under the action ofSE⁡\(3\)\\mathrm\{SE\}\(3\)\. Here, the quantitymNm\_\{N\}may not necessarily be equal toNN, but can be defined as the minimal dimensionmN≥Nm\_\{N\}\\geq Nof a trivial vector bundle𝒯Y′\\mathcal\{T\}\_\{Y\}^\{\\prime\}overY′Y^\{\\prime\}which containsVY′V\_\{Y\}^\{\\prime\}as a vector sub\-bundle\. We note that this quantity is guaranteed to exist using Proposition[A\.2](https://arxiv.org/html/2610.21087#A1.Thmtheorem2), whose statement appears in the next subsection\. More concretely, the relation betweenffandFFis given as follows\. Given an inclusionj:Y′→ℝ3j:Y^\{\\prime\}\\to\\mathbb\{R\}^\{3\}and a projectionπ:𝒯Y′→VY′\\pi:\\mathcal\{T\}\_\{Y\}^\{\\prime\}\\to V\_\{Y\}^\{\\prime\}, we have

f=π∘F∘j:Y′→ℝ3→ℝmN→ℝN\.f=\\pi\\circ F\\circ j:Y^\{\\prime\}\\to\\mathbb\{R\}^\{3\}\\to\\mathbb\{R\}^\{m\_\{N\}\}\\to\\mathbb\{R\}^\{N\}\.\(1\)Using the analogous argument, we note that the restriction of a global sectionf∘if\\circ iover a 3D point cloud𝒳\\mathcal\{X\}can be obtained from a continuous functionF:ℝ3→ℝmNF:\\mathbb\{R\}^\{3\}\\to\\mathbb\{R\}^\{m\_\{N\}\}via the compositionπ∘F∘j∘i\\pi\\circ F\\circ j\\circ i\.

Based on this identification, we define what it means for anSE⁡\(3\)\\mathrm\{SE\}\(3\)\-equivariant neural network to effectively approximate a global section\. Similar definitions can be found in Theorem 11 of\[[27](https://arxiv.org/html/2610.21087#bib.bib50)\]and Theorem 4\.1 of\[[35](https://arxiv.org/html/2610.21087#bib.bib51)\]\.

###### Definition 2\.5\.

LetΨ\\Psibe anSE⁡\(3\)\\mathrm\{SE\}\(3\)\-equivariant neural network whose activation functions are non\-constant and Lipschitz continuous\. We say thatΨ\\Psieffectively approximates a global sectionf:Y′→VY′f:Y^\{\\prime\}\\to V\_\{Y\}^\{\\prime\}if for anyϵ\>0\\epsilon\>0the neural networkΨ\\Psisatisfies the following inequality:

supz∈Y′⊂ℝ3\|Ψ⁡\(z\)−F⁡\(z\)\|<ϵ\.\\sup\_\{z\\in Y^\{\\prime\}\\subset\\mathbb\{R\}^\{3\}\}\|\\Psi\(z\)\-F\(z\)\|<\\epsilon\.\(2\)Given a discrete point cloud𝒳\\mathcal\{X\}onY′Y^\{\\prime\}endowed with an injectioni:𝒳→Y′i:\\mathcal\{X\}\\to Y^\{\\prime\}, we say thatΨ\\Psieffectively approximates a restriction of a global sectionf∘i:𝒳→VY′f\\circ i:\\mathcal\{X\}\\to V\_\{Y\}^\{\\prime\}if for anyϵ\>0\\epsilon\>0the neural networkΨ\\Psisatisfies the following inequality:

supz∈𝒳⊂ℝ3\|Ψ⁡\(z\)−F⁡\(z\)\|<ϵ\.\\sup\_\{z\\in\\mathcal\{X\}\\subset\\mathbb\{R\}^\{3\}\}\|\\Psi\(z\)\-F\(z\)\|<\\epsilon\.\(3\)

By formulating the notion of effective approximation using continuous functions over manifolds, instead of discrete point clouds, we can apply Definition[2\.5](https://arxiv.org/html/2610.21087#S2.Thmtheorem5)to any 3D point cloud𝒳\\mathcal\{X\}onY′Y^\{\\prime\}once we specify an injectioni:𝒳→Y′i:\\mathcal\{X\}\\to Y^\{\\prime\}\. By doing so, one can hope to ensure robustness in approximating the desired restrictionf∘if\\circ igiven any arbitrary 3D point cloud𝒳\\mathcal\{X\}overY′Y^\{\\prime\}\.

With the theoretic framework in place, we shall analyze potential topological obstructions imposed on certain masking functions\. Given an integerNN, let𝔪:X→ℝN\\mathfrak\{m\}:X\\to\\mathbb\{R\}^\{N\}be a masking function that does not lift to a section of a non\-trivial vector bundleVX→XV\_\{X\}\\to X\. As an example, we may consider constant and randomized masking functions, which are default choices for reconstruction based SSL in both images and point clouds\. We note that the image𝔪⁡\(x\)\\mathfrak\{m\}\(\\textbf\{x\}\)is chosen randomly independent of the geometric features of𝐱\\mathbf\{x\}\. We can hence interpret the application of the masking function𝔪\\mathfrak\{m\}as replacingVXV\_\{X\}withℝN×X\\mathbb\{R\}^\{N\}\\times X\. Because the trivial vector bundle overXXis the pullback of the trivial vector bundle overS2S^\{2\}, the masking operator by𝔪\\mathfrak\{m\}corresponds to replacingVXV\_\{X\}withf∗​\(ℝN×S2\)f^\{\*\}\(\\mathbb\{R\}^\{N\}\\times S^\{2\}\)\. There is a wide class of non\-trivial vector bundles overXXwhich may not admit a smooth embedding insidef∗​\(ℝN×S2\)f^\{\*\}\(\\mathbb\{R\}^\{N\}\\times S^\{2\}\)\. A summary of our aforementioned findings, focusing on constant masking functions, leads to the following result, whose proof is in Appendix[A\.3](https://arxiv.org/html/2610.21087#A1.SS3)\.

###### Theorem 2\.6\.

LetY′Y^\{\\prime\}be the underlying33\-dimensional smooth orientable manifold constructed from Section[2\.2](https://arxiv.org/html/2610.21087#S2.SS2)which represents the underlying space for a given 3D point cloud\. Let𝔪:X→ℝ\\mathfrak\{m\}:X\\to\\mathbb\{R\}be any constant masking function\. Then anySE⁡\(3\)\\mathrm\{SE\}\(3\)\-equivariant neural network with the masking function𝔪\\mathfrak\{m\}cannot effectively approximate global sections of any non\-trivial vector bundlesVY′→Y′V\_\{Y\}^\{\\prime\}\\to Y^\{\\prime\}\(as in Definition[2\.5](https://arxiv.org/html/2610.21087#S2.Thmtheorem5)\)\.

AnySE⁡\(3\)\\mathrm\{SE\}\(3\)\-equivariant neural network with the randomized masking function𝔪\\mathfrak\{m\}cannot effectively approximate global sections of any non\-trivial vector bundle of dimension at leastNN\. Even for lower dimensional non\-trivial vector bundles, there are many challenges that would need to be addressed to effectively approximate such global sections, see Figure[1](https://arxiv.org/html/2610.21087#S2.F1)and Appendix[A\.3](https://arxiv.org/html/2610.21087#A1.SS3)for more details\.

## 3SIEVE: Scalar Invariant Extraction Via Equivariance

![Refer to caption](https://arxiv.org/html/2610.21087v1/sieve-main-diagramv2.png)

Figure 2:SIEVE Framework\.SIEVE extracts SE\(3\)\-invariant embeddings using self\-supervised reconstruction on an equivariant bottleneck\. The equivariant encoder compresses raw point coordinates into an invariant scalar encoding, while the vector pipeline is blocked\. The spherical projection mask is applied to the input point cloud and supplied as the new vector embedding\. It and the semantic details from the scalar embedding are used by the equivariant decoder to reconstruct the point cloud on a node\-level\.To circumvent the topological obstruction stated in Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6), we proposeSIEVE, an SSL framework that utilizes the*spherical projection mask*to maintain a continuous section in the vector bundle, thereby ensuring the representational completeness of the equivariant pipeline\. SIEVE adopts a framework inspired by the Masked Autoencoder \(MAE\), comprising an encoder, a geometric masking operator, and a decoder\[[24](https://arxiv.org/html/2610.21087#bib.bib3)\]\. The framework is based on two core design principles\.

Equivariant Feature Streams\.We leverage SE\(3\)\-equivariant neural networks to process the data through two streams: a vector pipeline and a scalar pipeline \(Figure[2](https://arxiv.org/html/2610.21087#S3.F2)\)\. The encoder maps the input point cloud𝒳\\mathcal\{X\}to an SE\(3\)\-invariant scalar embeddingsc​o​d​es\_\{code\}, capturing semantic information independent of the point cloud’s global pose\.

Semantic\-Geometric Decomposition\.Unlike standard MAE, which masks a subset of nodes entirely, SIEVE employs a*spherical projection mask*\. This operator maps all the points in𝒳\\mathcal\{X\}to a unit sphere, obfuscating local semantic features \(the “what”\) while preserving global spatial properties such as orientation and position \(the “where”\)\. This sidesteps the topological obstructions of constant or randomized vector masks, admitting a wider range of geometric entanglements for reconstruction\.

### 3\.1Spherical Projection Mask

Standard masking in SSL typically replaces masked node features with a learnable constant token𝐳∈ℝC\\mathbf\{z\}\\in\\mathbb\{R\}^\{C\}\. However, applying such an isotropic mask to the vector pipeline of an SE\(3\)\-equivariant neural network leads to a collapse of the vector output, as stated in Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)\.

To circumvent this collapse, we propose the*spherical projection mask*\. Given an input point cloud𝒳=\{𝐱i\}i=1Np⊂ℝ3\\mathcal\{X\}=\\\{\\mathbf\{x\}\_\{i\}\\\}\_\{i=1\}^\{N\_\{p\}\}\\subset\\mathbb\{R\}^\{3\}, let𝐱¯=1Np​∑i=1Np𝐱i\\overline\{\\mathbf\{x\}\}=\\frac\{1\}\{N\_\{p\}\}\\sum\_\{i=1\}^\{N\_\{p\}\}\\mathbf\{x\}\_\{i\}denote the mean of the point cloud\. The spherical projection operator projects each point𝐱i\\mathbf\{x\}\_\{i\}onto a unit sphereS2S^\{2\}centered on the mean𝐱¯\\overline\{\\mathbf\{x\}\}:

SphericalMask⁡\(𝐱i\)=𝐱¯\+𝐱i−𝐱¯‖𝐱i−𝐱¯‖2\.\\mathrm\{SphericalMask\}\(\\mathbf\{x\}\_\{i\}\)=\\overline\{\\mathbf\{x\}\}\+\\frac\{\\mathbf\{x\}\_\{i\}\-\\overline\{\\mathbf\{x\}\}\}\{\\\|\\mathbf\{x\}\_\{i\}\-\\overline\{\\mathbf\{x\}\}\\\|\_\{2\}\}\.\(4\)Using the mean𝐱¯\\overline\{\\mathbf\{x\}\}as the center guarantees that projected nodes translate identically to the input nodes\. Thus, the spherical projection mask is an SE\(3\)\-equivariant operator, ensuring that SIEVE’s vector pipeline is equivariant\.

A naive projection leaves the scalar radiusr=‖𝐱i−𝐱¯‖r=\\\|\\mathbf\{x\}\_\{i\}\-\\overline\{\\mathbf\{x\}\}\\\|as a trivial shortcut, leading to a representational collapse of the embeddings as demonstrated in Appendix[C\.3](https://arxiv.org/html/2610.21087#A3.SS3)\. To mitigate this, we inject i\.i\.d\. Gaussian noise into the projected coordinates\. Further mitigation strategies are discussed in Appendix[B\.1](https://arxiv.org/html/2610.21087#A2.SS1)\.

### 3\.2SE\(3\)\-Equivariant Encoder and Decoder

The encoder and decoder in SIEVE are SE\(3\)\-equivariant neural networks\. Each layer has two feature streams: a scalar stream representing invariant features and a vector stream representing steerable features, with cross\-stream interaction at every layer\. We denote these feature pairs as\[s,𝐯\]\[s,\\mathbf\{v\}\], wheres∈ℝNp×h1s\\in\\mathbb\{R\}^\{N\_\{p\}\\times h\_\{1\}\}and𝐯∈ℝNp×h2×3\\mathbf\{v\}\\in\\mathbb\{R\}^\{N\_\{p\}\\times h\_\{2\}\\times 3\}\.

Encoder:The SIEVE encoder extracts the SE\(3\)\-invariant semantic information from the input geometry\. The point cloud𝒳\\mathcal\{X\}is the vector input\. The choice of scalar input becomes critical: we demonstrate in Appendix[C\.2](https://arxiv.org/html/2610.21087#A3.SS2)that uniform scalar inputs prevent SIEVE from reconstructing the point cloud, so we provide i\.i\.d\. Gaussian noisesi​n∼𝒩⁡\(0,I\)∈ℝNps\_\{in\}\\sim\\mathcal\{N\}\(0,I\)\\in\\mathbb\{R\}^\{N\_\{p\}\}as the scalar feature\. The encoding process is defined as

\[sc​o​d​e,𝐯c​o​d​e\]=Encoder⁡\(\[si​n,𝒳\]\)\[s\_\{code\},\\mathbf\{v\}\_\{code\}\]=\\mathrm\{Encoder\}\(\[s\_\{in\},\\mathcal\{X\}\]\)\(5\)
Following the forward pass, we impose an equivariant bottleneck by discarding the vector output𝐯c​o​d​e\\mathbf\{v\}\_\{code\}\. The invariant scalar embeddingsc​o​d​es\_\{code\}is retained as the compressed semantic representation of the point cloud\.

Decoder:The decoder aims to reconstruct the original point cloud𝒳\\mathcal\{X\}using the semantic encodingsc​o​d​es\_\{code\}and the masked geometry𝐱m​a​s​k​e​d\\mathbf\{x\}\_\{masked\}as defined in section[3\.1](https://arxiv.org/html/2610.21087#S3.SS1)\. The decoder’s vector output𝒳^\\hat\{\\mathcal\{X\}\}serves as the reconstructed point cloud, while the scalar outputso​u​ts\_\{out\}is discarded\.

\[so​u​t,𝒳^\]=Decoder⁡\(\[sc​o​d​e,𝐱m​a​s​k​e​d\]\)\[s\_\{out\},\\hat\{\\mathcal\{X\}\}\]=\\mathrm\{Decoder\}\(\[s\_\{code\},\\mathbf\{x\}\_\{masked\}\]\)\(6\)
Backbone Selection:While SIEVE is agnostic to specific SE\(3\)\-equivariant architectures, we employ the Geometric Algebra Transformer \(GATr\) as our backbone\[[5](https://arxiv.org/html/2610.21087#bib.bib9)\]\. GATr’s global attention captures long\-range dependencies\. It also uses memory efficient attention\[[28](https://arxiv.org/html/2610.21087#bib.bib10)\]and avoids expensive spherical harmonics, allowing for better scalability compared to other equivariant neural networks\.

### 3\.3Learning Objective

The objective of SIEVE is to perform node level reconstruction, wherein each individual point𝐱i∈𝒳\\mathbf\{x\}\_\{i\}\\in\\mathcal\{X\}is mapped back to its original position\. We accomplish this by using a standardℒ​1\\mathcal\{L\}1loss between the original point cloud𝒳\\mathcal\{X\}and the reconstructed point cloud𝒳^\\hat\{\\mathcal\{X\}\}:

ℒ​1=1Np​∑i=1Np‖𝐱i−𝐱^i‖1\.\\mathcal\{L\}1=\\frac\{1\}\{N\_\{p\}\}\\sum\_\{i=1\}^\{N\_\{p\}\}\\\|\\mathbf\{x\}\_\{i\}\-\\hat\{\\mathbf\{x\}\}\_\{i\}\\\|\_\{1\}\.\(7\)

### 3\.4A Mathematical Reformulation of SIEVE

Using the theoretical motivation for SIEVE, we deduce theoretical strengths of our proposed method in comparison to other masking schemes, whose proof is in Appendix[A\.4](https://arxiv.org/html/2610.21087#A1.SS4)\.

###### Theorem 3\.1\.

LetY′Y^\{\\prime\}be the underlying33\-dimensional smooth orientable manifold constructed from Section[2\.2](https://arxiv.org/html/2610.21087#S2.SS2)\. Then SIEVE can approximate global sections of any vector bundles which are isomorphic to pullbacks of any vector bundles ofS2S^\{2\}\.

The idea of the proof originates from an observation that application ofSphericalMask\\mathrm\{SphericalMask\}corresponds to utilizing pullback of vector bundleV→S2V\\to S^\{2\}which contains the tautological bundleT→S2T\\to S^\{2\}as a vector sub\-bundle\. We thus deduce that SIEVE, unlike constant or random masking techniques, has theoretical capabilities in approximating all possible vector bundles over a wide range of classes of 2\-dimensional or 3\-dimensional closed oriented manifolds which arise from pullback of both trivial and non\-trivial vector bundles overS2S^\{2\}\. To support this, we present experimental demonstrations of SIEVE’s theoretical capabilities in Section[4\.2](https://arxiv.org/html/2610.21087#S4.SS2)\.

## 4Experiments

### 4\.1Pre\-training

![Refer to caption](https://arxiv.org/html/2610.21087v1/SIEVE_Embeddings.png)

Figure 3:Pre\-trained SIEVE Encoder Embeddings\.We visualize the PCA components of the pre\-trained SIEVE embeddings\. These embedding are invariant to point cloud rotation and translation\.We pre\-train SIEVE on the OpenShape dataset, which consists of approximately 876k 3D models\[[30](https://arxiv.org/html/2610.21087#bib.bib44)\]\. Each point cloud is generated by sampling 2048 points via furthest point sampling\. We utilize the entire dataset for pre\-training without a train\-validation split to maximize geometric diversity\. Full pre\-training details are in Appendix[B\.1](https://arxiv.org/html/2610.21087#A2.SS1)\.

### 4\.2Empirical Verification of Collapse

In Section[2](https://arxiv.org/html/2610.21087#S2), we theoretically established that SE\(3\) equivariant networks cannot output two point clouds of different topology with a constant or noise vector token as input\. To verify this, we perform an experiment using a synthetic dataset of two objects with differing topologies: a solid cuboid \(homeomorphic to the 3\-diskD3D^\{3\}\) and a square toroid \(homeomorphic toS1×S1S^\{1\}\\times S^\{1\}\)\. The dataset consists of 120 instances per class, partitioned into a 100/20 train\-test split\. Failure on a dataset this simple indicates a fundamental representational issue rather than a capacity or data limitation\.

To demonstrate that this limitation is inherent to the equivariance constraint rather than a specific architecture, we evaluate two distinct backbones: GATr\[[5](https://arxiv.org/html/2610.21087#bib.bib9)\]and SE\(3\)\-Transformer\[[16](https://arxiv.org/html/2610.21087#bib.bib12)\]\. We compare three masking strategies: a constant vector mask utilizing a learnable isotropic token, an ellipsoid noise mask sampled from an ellipsoid spectrally aligned with the point cloud’s principal components \(and thus is SE\(3\)\-equivariant\), and our proposed spherical projection mask\. All three models use identical initialization and hyperparameter settings to ensure a fair comparison\.

Figure 4:Validation Loss of SIEVE on two baselines: GATr and SE\(3\)\-Transformer\. Trained on the synthetic dataset\.As shown in Figure[4](https://arxiv.org/html/2610.21087#S4.F4), models that utilize the spherical projection mask successfully converge, achieving a validationℒ​1\\mathcal\{L\}1loss below 0\.01\. Conversely, models employing the constant vector mask or the ellipsoid noise mask fail to converge, with validation and training losses stagnating around 0\.5\. This failure to minimize even the training loss demonstrates a fundamental representational collapse rather than a deficit in model capacity or data volume\.

A visual inspection of the reconstructed point clouds \(Figure[5](https://arxiv.org/html/2610.21087#S4.F5)\) further demonstrates the predicted representational collapse of Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)\. While SIEVE with the spherical projection mask achieves near\-perfect reconstruction, SIEVE with the constant vector mask or the ellipsoid noise mask outputs a near zero\-dimensional ‘blob’\. As the proof of Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)suggests, these two masking schemes can only represent global sections of vector bundles of dimension at most11\. Hence, they cannot effectively represent local coordinate functions for 3D point clouds\.

![Refer to caption](https://arxiv.org/html/2610.21087v1/Mask_Ablation.png)

Figure 5:Reconstruction Comparison and Representational Collapse\.We evaluate the reconstruction*\(blue\)*of the input point cloud*\(red\)*under different masking strategies\. Using our spherical projection mask, the model achieves near\-perfect reconstruction\. Reconstructing with a constant vector mask or the ellipsoid noise mask results in a catastrophic representational collapse\. We refer to Appendix[A\.3](https://arxiv.org/html/2610.21087#A1.SS3)for further discussions\.Reconstruction experiments on a dataset with identical topology confirms that it is the topological variety that causes the collapse; constant and noise mask SIEVE have no issues learning to reconstruct such datasets\. \(Appendix[C\.1](https://arxiv.org/html/2610.21087#A3.SS1)\)

### 4\.3Downstream Tasks

This section evaluates whethersc​o​d​es\_\{code\}contains useful, transferable geometric information\. We do not target SOTA on any specific benchmark, as SOTA architectures are heavily tuned to their target tasks\. Instead we test whether SIEVE consistently improves generalist point cloud architectures \(PointNet\+\+, DGCNN\[[37](https://arxiv.org/html/2610.21087#bib.bib27),[45](https://arxiv.org/html/2610.21087#bib.bib28)\]\)\. Integration is minimal: the SIEVE encoder is frozen, andsc​o​d​es\_\{code\}is concatenated with the raw coordinates at the input layer\. This drop\-in usage is enabled by SIEVE’s node\-level granularity, which aligns one\-to\-one with input points, unlike patch\-based SSL\. We therefore evaluate on downstream tasks where such granularity is critical\.

#### 4\.3\.1Molecular Segmentation

We first evaluate the learned representations on the molecular segmentation task which consists of 640 RNA molecules segmented into 120 categories\[[36](https://arxiv.org/html/2610.21087#bib.bib26)\]\. Following the protocol in\[[21](https://arxiv.org/html/2610.21087#bib.bib25)\]we resample each mesh to 2048 points and utilize the standard 80\-20 train test split\.

To demonstrate that SIEVE embeddings transfer across architectures, we evaluate with three distinct architectures: PointNet\+\+\[[37](https://arxiv.org/html/2610.21087#bib.bib27)\], DeltaNet\[[46](https://arxiv.org/html/2610.21087#bib.bib29)\], and DiffusionNet\[[40](https://arxiv.org/html/2610.21087#bib.bib30)\]\. For each model, the frozen SIEVE scalar embeddingsc​o​d​es\_\{code\}is concatenated with the raw node coordinates𝐱i\\mathbf\{x\}\_\{i\}as input\. We compare our learned representation against two SE\(3\)\-invariant geometric representations: the distance from meanrrand curvatureκ1,κ2\\kappa\_\{1\},\\kappa\_\{2\}\. As shown in Table[1](https://arxiv.org/html/2610.21087#S4.T1), SIEVE embeddings consistently outperform the “position\-only” baselines and the geometry\-enhanced models across all three architectures\.

Table 1:Test accuracies\(%\) on the RNA Segmentation Task
#### 4\.3\.23D Shape Completion

3D\-EPN is a point cloud completion dataset consisting of 16 categories: 8 categories are included in both the training and test sets \(“Seen”\), while the remaining 8 appear only in the test set \(“Unseen”\)\[[14](https://arxiv.org/html/2610.21087#bib.bib31)\]\. We utilize PCN as our baseline and compare the performance with and without the addition of SIEVE encodings\[[51](https://arxiv.org/html/2610.21087#bib.bib32)\]\.

As shown in Table[2](https://arxiv.org/html/2610.21087#S4.T2), SIEVE consistently reduces the Chamfer Distance \(CD\) loss across both seen and unseen categories\. The performance gain is most pronounced on unseen categories, with a12\.5%12\.5\\%reduction in the average loss compared to the1\.62%1\.62\\%reduction on seen categories\. This suggests that the geometric rigor distilled into SIEVE embeddings provides a more robust structural prior than coordinate\-based features alone, facilitating superior zero\-shot generalization to novel topological structures\.

Table 2:Reconstructionℒ​2\\mathcal\{L\}2CD Loss\(×104\)\(\\times 10^\{4\}\)on the 3D\-EPN Dataset
#### 4\.3\.3Geometric Registration

Geometric registration is the prediction of a rigid body transformation to align one shape to another\. We follow the task and architecture in Deep Closest Point \(DCP\)\[[44](https://arxiv.org/html/2610.21087#bib.bib39)\]\. A point cloud is sampled from the ModelNet40 dataset\[[48](https://arxiv.org/html/2610.21087#bib.bib33)\], which is then cloned, point order shuffled, then transformed with a random rigid body transformation\. We embed the SIEVE embeddings to DCP baseline and test with two train\-test splits: one where all 40 categories are in both sets, and the other where 20 are used in train and the other 20 are reserved for testing\.

The results show that SIEVE embeddings improve both the shared and separate category registration, with more dramatic gains in the separate category\. The geometric registration performance with SIEVE is much more robust to unseen objects\. This pattern is consistent with the 3D\-EPN results: SIEVE embeddings encode intrinsic geometric structure that transfers across categories\.

Table 3:Point Cloud Registration Results on the ModelNet40 Dataset

## 5Related Work

Point Cloud Self\-Supervised Learning\.Inspired by the success of Masked Autoencoders in images\[[24](https://arxiv.org/html/2610.21087#bib.bib3)\], reconstruction based point cloud SSL has largely adopted the 3D patch reconstruction framework established by PointBERT\[[50](https://arxiv.org/html/2610.21087#bib.bib4)\]\. These methods typically employ a binary masking strategy on local patches, where specific regions of the point cloud are entirely removed from the input\[[32](https://arxiv.org/html/2610.21087#bib.bib5),[9](https://arxiv.org/html/2610.21087#bib.bib6),[29](https://arxiv.org/html/2610.21087#bib.bib7)\]\. However, these frameworks lack inherent SE\(3\)\-symmetry and their embeddings remain sensitive to global rotations and translations\. RI\-MAE pursues rotation invariance without equivariant architectures by PCA\-aligning patches and reconstructing in latent space\[[41](https://arxiv.org/html/2610.21087#bib.bib40)\]\. RIVQ\-VAE learns a discretized, SE\(3\)\-invariant patch representation using TFN\[[31](https://arxiv.org/html/2610.21087#bib.bib49)\]\. All of these methods produce patch embeddings, requiring fine tuning with a task\-specific head for downstream tasks\. SIEVE instead uses equivariant networks and a spherical projection mask, producing node\-level SE\(3\)\-invariant embeddings that concatenate cleanly with raw coordinates as input to any downstream architecture, without fine\-tuning the encoder\.

Equivariant Neural Networks\.To date, numerous equivariant architectures have been proposed\[[15](https://arxiv.org/html/2610.21087#bib.bib47),[7](https://arxiv.org/html/2610.21087#bib.bib18),[34](https://arxiv.org/html/2610.21087#bib.bib13),[2](https://arxiv.org/html/2610.21087#bib.bib17),[10](https://arxiv.org/html/2610.21087#bib.bib48)\]\. However, many of these models have focused on molecular biology applications\[[20](https://arxiv.org/html/2610.21087#bib.bib19)\]; QM9 and MD17 molecular prediction datasets are some of the most popular benchmarks for equivariant neural networks\[[38](https://arxiv.org/html/2610.21087#bib.bib20),[11](https://arxiv.org/html/2610.21087#bib.bib21)\]\. Molecules possess a significant SE\(3\)\-invariant node feature, the atomic number, which distinguishes the domain from purely geometric data like point clouds\. Consequently, the only existing work that combines SSL with SE\(3\)\-equivariant networks is not directly applicable to point clouds due to this difference in domain\[[1](https://arxiv.org/html/2610.21087#bib.bib22)\]\. To our knowledge, this is the first work to combine point cloud SSL with SE\(3\)\-equivariant neural networks for general geometric learning\.

## 6Conclusion

In this work, we theoretically established that conventional masking strategies in geometric self\-supervised learning induce a representational collapse when applied to SE\(3\)\-equivariant architectures\. Using the theory of vector bundles, we showed that isotropic or randomized masking functions which ignore innate geometric features of point clouds do not have enough capabilities to reconstruct such point clouds, because they cannot effectively reconstruct a wide range of classes of non\-isomorphic vector bundles\.

Based on these theoretical observations, we proposed SIEVE, a pioneering framework that bridges the gap between equivariance and SSL for point clouds\. By utilizing a novel spherical projection masking strategy, SIEVE circumvented the topological obstructions inherent in isotropic masking, enabling robust geometric reconstruction\. Our experiments empirically confirmed the representational collapse and demonstrated that SIEVE learns highly generalizable embeddings that outperform traditional descriptors in downstream tasks\. To our knowledge, SIEVE is the first framework to combine self\-supervised learning with SE\(3\)\-equivariant networks for general point cloud geometry, and we hope it spurs further work at this intersection\.

## Acknowledgments and Disclosure of Funding

This research was supported by the “Advanced GPU Utilization Support Program” funded by the Government of the Republic of Korea \(Ministry of Science and ICT\)\.

This work was initiated while Jongann Lee was a paid intern at SolverX in the Republic of Korea and continued after the internship concluded\. Following the internship, Jongann Lee was no longer employed by SolverX and received no monetary compensation from SolverX for his subsequent contributions\. SolverX continued to provide in\-kind research support through access to GPU computing resources and LLM\-based tools\.

Sun Woo Park thanks the Max Planck Institute for Mathematics for its generous support\.

Competing interests\.Jongann Lee was previously a paid intern at SolverX in the Republic of Korea\. SolverX subsequently continued to provide the in\-kind research resources described above\. The authors declare no other competing interests\.

## References

- \[1\]J\. An, C\. Qu, Y\. Shi, X\. Liu, Q\. Tang, F\. Cao, and Y\. Qi\(2025\)Equivariant masked position prediction for efficient molecular representation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[2\]S\. Aykent and T\. Xia\(2025\)GotenNet: rethinking efficient 3D equivariant graph neural networks\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[3\]E\. J\. Bekkers, S\. Vadgama, R\. D\. Hesselink, P\. A\. V\. der Linden, and D\. W\. Romero\(2023\)Fast, expressive SE\(n\)\(n\)equivariant networks through weight\-sharing in position\-orientation space\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p2.1)\.
- \[4\]J\. Brandstetter, R\. Hesselink, E\. van der Pol, E\. Bekkers, and M\. Welling\(2022\)Geometric and physical quantities improve E\(3\) equivariant message passing\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p2.1)\.
- \[5\]J\. Brehmer, P\. de Haan, S\. Behrends, and T\. Cohen\(2023\)Geometric algebra transformer\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§B\.1](https://arxiv.org/html/2610.21087#A2.SS1.p1.1),[§B\.2](https://arxiv.org/html/2610.21087#A2.SS2.p1.1),[§1](https://arxiv.org/html/2610.21087#S1.p2.1),[§3\.2](https://arxiv.org/html/2610.21087#S3.SS2.p7.1),[§4\.2](https://arxiv.org/html/2610.21087#S4.SS2.p2.1)\.
- \[6\]H\. Caesar, V\. Bankiti, A\. H\. Lang, S\. Vora, V\. E\. Liong, Q\. Xu, A\. Krishnan, Y\. Pan, G\. Baldan, and O\. Beijbom\(2020\)NuScenes: a multimodal dataset for autonomous driving\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition \(CVPR\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p1.1)\.
- \[7\]J\. Cen, A\. Li, N\. Lin, Y\. Ren, Z\. Wang, and W\. Huang\(2024\)Are high\-degree representations really unnecessary in equivariant graph neural networks?\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[8\]A\. X\. Chang, T\. Funkhouser, L\. Guibas, P\. Hanrahan, Q\. Huang, Z\. Li, S\. Savarese, M\. Savva, S\. Song, H\. Su, J\. Xiao, L\. Yi, and F\. Yu\(2015\)ShapeNet: an information\-rich 3D model repository\.arXiv preprint arXiv:1512\.03012\.External Links:1512\.03012Cited by:[§B\.1](https://arxiv.org/html/2610.21087#A2.SS1.p4.1),[§B\.1](https://arxiv.org/html/2610.21087#A2.SS1.p6.1)\.
- \[9\]G\. Chen, M\. Wang, Y\. Yang, K\. Yu, L\. Yuan, and Y\. Yue\(2023\)PointGPT: auto\-regressively generative pre\-training from point clouds\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p1.1)\.
- \[10\]H\. Chen, S\. Liu, W\. Chen, H\. Li, and R\. Hill\(2021\)Equivariant point network for 3D point cloud analysis\.IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)\.Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[11\]S\. Chmiela, A\. Tkatchenko, H\. E\. Sauceda, I\. Poltavsky, K\. T\. Schütt, and K\. Müller\(2017\)Machine learning of accurate energy\-conserving molecular force fields\.Science advances3\(5\),pp\. e1603015\.Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[12\]Y\. Y\. Choi, S\. W\. Park, M\. Lee, and Y\. Woo\(2024\)Topology\-informed graph transformer\.arXiv preprint arXiv:2402\.02005\.Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p1.1)\.
- \[13\]Y\. Y\. Choi, S\. W\. Park, Y\. Woo, and U\. J\. Choi\(2022\)Cycle to clique \(cy2c\) graph neural network: a sight to see beyond neighborhood aggregation\.InThe eleventh international conference on learning representations,Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p1.1)\.
- \[14\]A\. Dai, C\. Ruizhongtai Qi, and M\. Nießner\(2017\)Shape completion using 3D\-encoder\-predictor cnns and shape synthesis\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§4\.3\.2](https://arxiv.org/html/2610.21087#S4.SS3.SSS2.p1.1)\.
- \[15\]C\. Deng, O\. Litany, Y\. Duan, A\. Poulenard, A\. Tagliasacchi, and L\. J\. Guibas\(2021\)Vector Neurons: a general framework for SO\(3\)\-equivariant networks\.Proceedings of the IEEE/CVF international conference on computer vision \(ICCV\)\.Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[16\]F\. B\. Fuchs, D\. E\. Worrall, V\. Fischer, and M\. Welling\(2020\)SE\(3\)\-transformers: 3D roto\-translation equivariant attention networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§B\.2](https://arxiv.org/html/2610.21087#A2.SS2.p1.1),[§1](https://arxiv.org/html/2610.21087#S1.p2.1),[§4\.2](https://arxiv.org/html/2610.21087#S4.SS2.p2.1)\.
- \[17\]A\. Ghanwat and D\. M\. Pancholi\(2020\)Embeddings of44–manifolds inℂ​ℙ3\\mathbb\{CP\}^\{3\}\.arXiv preprint arXiv:2002\.11299\.External Links:2002\.11299Cited by:[§A\.4](https://arxiv.org/html/2610.21087#A1.SS4.p2.1)\.
- \[18\]J\. Gu, F\. Xiang, X\. Li, Z\. Ling, X\. Liu, T\. Mu, Y\. Tang, S\. Tao, X\. Wei, Y\. Yao, X\. Yuan, P\. Xie, Z\. Huang, R\. Chen, and H\. Su\(2023\)ManiSkill2: a unified benchmark for generalizable manipulation skills\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p1.1)\.
- \[19\]A\. Haefliger and M\. W\. Hirsch\(1963\)On the existence and classification of differentiable embeddings\.Topology2\(1\-2\),pp\. 129–135\.Cited by:[§A\.4](https://arxiv.org/html/2610.21087#A1.SS4.p2.1)\.
- \[20\]J\. Han, J\. Cen, L\. Wu, Z\. Li, X\. Kong, R\. Jiao, Z\. Yu, T\. Xu, F\. Wu, Z\. Wang,et al\.\(2025\)A survey of geometric graph neural networks: data structures, models and applications\.Frontiers of Computer Science19\(11\),pp\. 1911375\.Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[21\]J\. Harrison, J\. Benn, and M\. Sermesant\(2024\)Improving neural network surface processing with principal curvatures\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.3\.1](https://arxiv.org/html/2610.21087#S4.SS3.SSS1.p1.1)\.
- \[22\]A\. Hatcher\(2002\)Algebraic topology\.Cambridge University Press,Cambridge, England\.Cited by:[§A\.2](https://arxiv.org/html/2610.21087#A1.SS2.p2.2),[§A\.2](https://arxiv.org/html/2610.21087#A1.SS2.p2.3),[Appendix A](https://arxiv.org/html/2610.21087#A1.p1.1)\.
- \[23\]A\. Hatcher\(2017\)Vector bundles and K\-theory\.External Links:[Link](https://pi.math.cornell.edu/~hatcher/VBKT/VBpage.html)Cited by:[§A\.2](https://arxiv.org/html/2610.21087#A1.SS2.p2.1),[§A\.3](https://arxiv.org/html/2610.21087#A1.SS3.p1.1),[Appendix A](https://arxiv.org/html/2610.21087#A1.p1.1),[§2\.2](https://arxiv.org/html/2610.21087#S2.SS2.p2.1)\.
- \[24\]K\. He, X\. Chen, S\. Xie, Y\. Li, P\. Dollár, and R\. Girshick\(2022\)Masked autoencoders are scalable vision learners\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition \(CVPR\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p3.1),[§3](https://arxiv.org/html/2610.21087#S3.p1.1),[§5](https://arxiv.org/html/2610.21087#S5.p1.1)\.
- \[25\]S\. Hong, Y\. Kwon, D\. Shin, J\. Park, and N\. Kang\(2025\)DeepJEB: 3d deep learning\-based synthetic jet engine bracket dataset\.Journal of Mechanical Design147\(4\),pp\. 041703\.Cited by:[§C\.1](https://arxiv.org/html/2610.21087#A3.SS1.p1.1)\.
- \[26\]A\. Kratsios and L\. Papon\(2022\)Universal approximation theorems for differentiable geometric deep learning\.Journal of Machine Learning Research23,pp\. 1–73\.Cited by:[§A\.1](https://arxiv.org/html/2610.21087#A1.SS1.p6.1)\.
- \[27\]W\. Kumagai and A\. Sannai\(2020\)Universal Approximation Theorem for Equivariant Maps by Group CNNs\.arXiv preprint arXiv:2012\.13882\.Cited by:[§2\.3](https://arxiv.org/html/2610.21087#S2.SS3.p3.1)\.
- \[28\]B\. Lefaudeux, F\. Massa, D\. Liskovich, W\. Xiong, V\. Caggiano, S\. Naren, M\. Xu, J\. Hu, M\. Tintore, S\. Zhang, P\. Labatut, D\. Haziza, L\. Wehrstedt, J\. Reizenstein, and G\. Sizov\(2022\)XFormers: a modular and hackable transformer modelling library\.Note:[https://github\.com/facebookresearch/xformers](https://github.com/facebookresearch/xformers)Cited by:[§3\.2](https://arxiv.org/html/2610.21087#S3.SS2.p7.1)\.
- \[29\]D\. Liang, X\. Zhou, W\. Xu, X\. Zhu, Z\. Zou, X\. Ye, X\. Tan, and X\. Bai\(2024\)PointMAMBA: a simple state space model for point cloud analysis\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p1.1)\.
- \[30\]M\. Liu, R\. Shi, K\. Kuang, Y\. Zhu, X\. Li, S\. Han, H\. Cai, F\. Porikli, and H\. Su\(2023\)OpenShape: scaling up 3D shape representation towards open\-world understanding\.Advances in neural information processing systems \(NeurIPS\)\.Cited by:[§B\.1](https://arxiv.org/html/2610.21087#A2.SS1.p10.1),[§4\.1](https://arxiv.org/html/2610.21087#S4.SS1.p1.1)\.
- \[31\]M\. Mezghanni, M\. Boulkenafed, and M\. Ovsjanikov\(2024\)RIVQ\-VAE: discrete rotation\-invariant 3D representation learning\.2024 International Conference on 3D Vision \(3DV\)\.Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p1.1)\.
- \[32\]Y\. Pang, W\. Wang, F\. E\.H\. Tay, W\. Liu, Y\. Tian, and L\. Yuan\(2022\)Masked autoencoders for point cloud self\-supervised learning\.InEuropean Conference on Computer Vision \(ECCV\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p3.1),[§5](https://arxiv.org/html/2610.21087#S5.p1.1)\.
- \[33\]S\. W\. Park, Y\. Y\. Choi, U\. J\. Choi, and Y\. Woo\(2026\)A functorial formulation of neighborhood aggregating deep learning\.arXiv preprint arXiv:2604\.24672\.External Links:2604\.24672Cited by:[§A\.1](https://arxiv.org/html/2610.21087#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2610.21087#A1.SS1.p6.1)\.
- \[34\]R\. P\. Pelaez, G\. Simeon, R\. Galvelis, A\. Mirarchi, P\. Eastman, S\. Doerr, P\. Thölke, T\. E\. Markland, and G\. D\. Fabritiis\(2024\)TorchMD\-net 2\.0: fast neural network potentials for molecular simulations\.arXiv preprint arXiv:2402\.17660\.External Links:2402\.17660Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[35\]P\. Petersen and F\. Voigtlaender\(2020\)Equivalence of approximation by Convolutional Neural Networks and Fully\-connected Networks\.Proceedings of the American Mathematical Society148\(4\),pp\. 1567–1581\.Cited by:[§2\.3](https://arxiv.org/html/2610.21087#S2.SS3.p3.1)\.
- \[36\]A\. Poulenard, MJ\. Rakotosaona, Y\. Ponty, and M\. Ovsjanikov\(2019\)Effective rotation\-invariant point CNN with spherical harmonics kernels\.InInternational Conference on 3D Vision \(3DV\),Cited by:[§4\.3\.1](https://arxiv.org/html/2610.21087#S4.SS3.SSS1.p1.1)\.
- \[37\]C\. R\. Qi, L\. Yi, H\. Su, and L\. J\. Guibas\(2017\)PointNet\+\+: deep hierarchical feature learning on point sets in a metric space\.InAdvances in neural information processing systems \(NeurIPS\),Cited by:[§4\.3\.1](https://arxiv.org/html/2610.21087#S4.SS3.SSS1.p2.1),[§4\.3](https://arxiv.org/html/2610.21087#S4.SS3.p1.1)\.
- \[38\]R\. Ramakrishnan, P\. O\. Dral, M\. Rupp, and O\. A\. Von Lilienfeld\(2014\)Quantum chemistry structures and properties of 134 kilo molecules\.Scientific data1\(1\),pp\. 1–7\.Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p2.1)\.
- \[39\]V\. G\. Satorras, E\. Hoogeboom, and M\. Welling\(2021\)E\(n\) equivariant graph neural networks\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p2.1)\.
- \[40\]N\. Sharp, S\. Attaiki, K\. Crane, and M\. Ovsjanikov\(2022\)DiffusionNet: discretization agnostic learning on surfaces\.ACM Transactions on Graphics \(TOG\)41\(3\),pp\. 1–16\.Cited by:[§4\.3\.1](https://arxiv.org/html/2610.21087#S4.SS3.SSS1.p2.1)\.
- \[41\]K\. Su, Q\. Wu, P\. Cai, X\. Zhu, X\. Lu, Z\. Wang, and K\. Hu\(2025\)RI\-MAE: rotation\-invariant masked autoencoders for self\-supervised point cloud representation learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Cited by:[§5](https://arxiv.org/html/2610.21087#S5.p1.1)\.
- \[42\]C\. Wall\(1965\)All 3\-manifolds imbed in 5\-space\.Bulletin of the American Mathematical Society71\(3\),pp\. 564–567\.Cited by:[§A\.4](https://arxiv.org/html/2610.21087#A1.SS4.p2.1)\.
- \[43\]D\. Wang, X\. Zhu, JY\. Park, M\. Jia, G\. Su, R\. Platt, and R\. Walters\(2023\)A general theory of correct, incorrect, and extrinsic equivariance\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p2.1)\.
- \[44\]Y\. Wang and J\. M\. Solomon\(2019\)Deep closest point: learning representations for point cloud registration\.InProceedings of the IEEE/CVF international conference on computer vision \(ICCV\),Cited by:[§B\.3](https://arxiv.org/html/2610.21087#A2.SS3.p1.1),[§4\.3\.3](https://arxiv.org/html/2610.21087#S4.SS3.SSS3.p1.1)\.
- \[45\]Y\. Wang, Y\. Sun, Z\. Liu, S\. E\. Sarma, M\. M\. Bronstein, and J\. M\. Solomon\(2019\)Dynamic Graph CNN for learning on point clouds\.ACM Transactions on Graphics \(TOG\)\.Cited by:[§4\.3](https://arxiv.org/html/2610.21087#S4.SS3.p1.1)\.
- \[46\]R\. Wiersma, A\. Nasikun, and E\. E\. K\. Hildebrandt\(2022\)DeltaConv: anisotropic operators for geometric deep learning on point clouds\.Transactions on Graphics41\(4\)\.Cited by:[§4\.3\.1](https://arxiv.org/html/2610.21087#S4.SS3.SSS1.p2.1)\.
- \[47\]X\. Wu, D\. DeTone, D\. Frost, T\. Shen, C\. Xie, N\. Yang, J\. Engel, R\. Newcombe, H\. Zhao, and J\. Straub\(2025\)Sonata: self\-supervised learning of reliable point representations\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p2.1)\.
- \[48\]Z\. Wu, S\. Song, A\. Khosla, F\. Yu, L\. Zhang, X\. Tang, and J\. Xiao\(2015\)3D shapenets: a deep representation for volumetric shapes\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§4\.3\.3](https://arxiv.org/html/2610.21087#S4.SS3.SSS3.p1.1)\.
- \[49\]Y\. Xie and T\. Smidt\(2025\)A tale of two symmetries: exploring the loss landscape of equivariant models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p2.1)\.
- \[50\]X\. Yu, L\. Tang, Y\. Rao, T\. Huang, J\. Zhou, and J\. Lu\(2022\)Point\-BERT: pre\-training 3D point cloud transformers with masked point modeling\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§1](https://arxiv.org/html/2610.21087#S1.p3.1),[§5](https://arxiv.org/html/2610.21087#S5.p1.1)\.
- \[51\]W\. Yuan, T\. Khot, D\. Held, C\. Mertz, and M\. Hebert\(2018\)PCN: point completion network\.InInternational Conference on 3D Vision \(3DV\),Cited by:[§B\.3](https://arxiv.org/html/2610.21087#A2.SS3.p1.1),[§4\.3\.2](https://arxiv.org/html/2610.21087#S4.SS3.SSS2.p1.1)\.

## Appendix AMathematical Background

We recommend interested readers to review\[[22](https://arxiv.org/html/2610.21087#bib.bib34)\]for an introduction to algebraic topology \(where definitions of homotopy, homotopy equivalence, homotopy groups, closed orientable manifolds, and fiber bundles can be found\), and Section 1 of\[[23](https://arxiv.org/html/2610.21087#bib.bib35)\]for an introduction to the theory of vector bundles over topological spaces\.

### A\.1Motivation for using vector bundles

We motivate the formulation of deep learning techniques as an approximator of global section of vector bundles by utilizing insights from the main results of Park et al\[[33](https://arxiv.org/html/2610.21087#bib.bib41)\]\.

Universal approximation theorem: The classical formulations of universal approximation theorem states that a feedforward neural network of arbitrary depth with non\-polynomial activation function can approximate any given continuous functionf:ℝm→ℝnf:\\mathbb\{R\}^\{m\}\\to\\mathbb\{R\}^\{n\}\. Such an approximation theorem ensures a neural network with arbitrary depth \(or arbitrarily many weights and parameters\) to find any auxiliary relation among collections of datasets, as long as they embed in Euclidean spaceℝm\\mathbb\{R\}^\{m\}\.

However, such a formulation of the universal approximation theorem does not necessarily imply that a neural network with arbitrary depth \(or arbitrarily many weights and parameters\) can find auxiliary relations among collections of datasets defined over manifoldsMMwhich are not homeomorphic toℝm\\mathbb\{R\}^\{m\}\. For example, given a manifoldMMwith open affine charts\{Uα\}α∈𝒜\\\{U\_\{\\alpha\}\\\}\_\{\\alpha\\in\\mathcal\{A\}\}, a neural network can approximate a continuous functionfα:Uα→ℝmf\_\{\\alpha\}:U\_\{\\alpha\}\\to\\mathbb\{R\}^\{m\}for eachα∈𝒜\\alpha\\in\\mathcal\{A\}, but one cannot guarantee that such a neural network will learn how to coherently glue such functionsfα:Uα→ℝmf\_\{\\alpha\}:U\_\{\\alpha\}\\to\\mathbb\{R\}^\{m\}\. In terms of datasets associated to 3D point clouds, the functionsfα:Uα→ℝmf\_\{\\alpha\}:U\_\{\\alpha\}\\to\\mathbb\{R\}^\{m\}correspond to measurements taken over 3D point clouds which constitute a part of \(but not all\) underlying geometric space\.

Using vector bundles: By using the notion of vector bundles \(see Definition[2\.1](https://arxiv.org/html/2610.21087#S2.Thmtheorem1)for its definition\), we can reformulate the problem of gluing functionsfαf\_\{\\alpha\}as follows\. Given a vector bundleV→MV\\to M, each functionfα:Uα→ℝmf\_\{\\alpha\}:U\_\{\\alpha\}\\to\\mathbb\{R\}^\{m\}corresponds to local sections of the vector bundle restricted to the open affine chartUαU\_\{\\alpha\}\. Here, the term “local” means that the functionfαf\_\{\\alpha\}is only well defined over the neighborhoodUαU\_\{\\alpha\}, and it is not necessarily defined overMM\. One can construct a global sectionf:M→Vf:M\\to Vby ensuring that the collection of functions\{fα\}α∈𝒜\\\{f\_\{\\alpha\}\\\}\_\{\\alpha\\in\\mathcal\{A\}\}satisfy the condition that given any two indicesα,β∈𝒜\\alpha,\\beta\\in\\mathcal\{A\}, the restrictions of functionsfαf\_\{\\alpha\}andfβf\_\{\\beta\}overUα∩UβU\_\{\\alpha\}\\cap U\_\{\\beta\}are equal\. By allowing such a condition to hold, one extends the function fromUαU\_\{\\alpha\}toUα∪UβU\_\{\\alpha\}\\cup U\_\{\\beta\}\. Iterating the procedure for all indicesα∈𝒜\\alpha\\in\\mathcal\{A\}allows us to construct the global sectionf:M→Vf:M\\to V\.

Non\-triviality of vector bundles: We note that the global section need not be a functionf:M→ℝmf:M\\to\\mathbb\{R\}^\{m\}\. If a global section is given by a functionf:M→ℝmf:M\\to\\mathbb\{R\}^\{m\}, then the vector bundleV→MV\\to Mis a trivial bundle, i\.e\.V≅M×ℝmV\\cong M\\times\\mathbb\{R\}^\{m\}\. Otherwise, we say that the vector bundleV→MV\\to Mis non\-trivial\. Such non\-triviality of vector bundle implies that there are obstructions in naively gluing local sectionsfα:Uα→ℝmf\_\{\\alpha\}:U\_\{\\alpha\}\\to\\mathbb\{R\}^\{m\}\. These obstructions can be detected by computing topological invariants of vector bundles, such as their homology or cohomology groups\. Loosely speaking, a global section of a non\-trivial vector bundle can be thought of as of datasets whose measurements depend crucially on their underlying geometric space \(or an underlying manifoldMM\)\. Meanwhile, a global section of a trivial vector bundle can be considered as of datasets whose measurements can be well approximated by classical deep neural networks without making significant reference to underlying geometric space of the datasets\. In particular, this observation hints that even if the measurements of 3D point clouds defined over an underlying manifoldMMtake values inℝm\\mathbb\{R\}^\{m\}, we cannot guarantee that such measurements can be continuously extended to a functionf:M→ℝmf:M\\to\\mathbb\{R\}^\{m\}\.

Strength of deep learning algorithms: We note that there are variants of universal approximation theorem which regards approximating functions between differentiable manifolds with neural networks of arbitrary width and depth, see for example the work of Kratsios and Papon\[[26](https://arxiv.org/html/2610.21087#bib.bib42)\]\. On the other hand, deep learning algorithms of finite depth and width may not have enough capabilities to approximate global sections of any vector bundleV→MV\\to Mof a manifold\. In light of this vein, the main result of Park et al\[[33](https://arxiv.org/html/2610.21087#bib.bib41)\]\(see for example Theorem 1\.2, Sections 2 and 3\) gives a presheaf/copresheaf theoretic characterization of capabilities of message passing neural networks: they approximate global sections of presheaf \(and copresheaf\) of continuous functions induced from direct sums of skyscraper cosheaves \(and respectively sheaves\)\. The obstructions of these presheaves \(and copresheaves\) from being sheaves \(and cosheaves, respectively\) can be utilized to explain limitations of a wide range of classes of deep learning algorithms, such as non\-unique gluing of local sections of such presheaves \(and copresheaves\), and adversarial attacks\.

Deep learning algorithms and vector bundles: Given that limitations of such deep learning algorithms originate from obstructions of a presheaf \(and copresheaf\) from being a sheaf \(and respectively cosheaf\), the paper hence calls for a need to design neural networks which are capable of approximating global sections of sheaves \(or cosheaves\) other than the aformentioned classes of presheaves and copresheaves\. In particular, vector bundles over manifolds are examples of sheaves over manifolds\. A neural network which has capabilities to approximate global sections of vector bundles over manifolds hence has potential to overcome limitations that other previously studied deep learning algorithms may have experienced\. We can ask a valid question on what isomorphism classes of vector bundlesV→MV\\to Mof a manifold a given neural network can approximate their global sections\. The capability that a neural network can effectively analyze dataset dependent on its intrinsic geometry can be rephrased as the capability to approximate global sections of non\-trivial isomorphism classes of vector bundles overMM\. In turn, it implies that the neural network has capabilities in detecting exotic underlying geometric features of a given dataset, and can perform on par with or beyond other neural network algorithms\.

### A\.2Proof of Theorem[2\.4](https://arxiv.org/html/2610.21087#S2.Thmtheorem4)

We first recall the definition of tautological bundle\.

###### Definition A\.1\.

Recall the homeomorphism of complex manifoldsS2≅ℂ​ℙ1S^\{2\}\\cong\\mathbb\{C\}\\mathbb\{P\}^\{1\}, which is the Grassmannian of complex projective lines inℂ2\\mathbb\{C\}^\{2\}\. The tautological bundleπt:T→S2\\pi\_\{t\}:T\\to S^\{2\}is the vector bundle defined as follows: for eachx∈S2x\\in S^\{2\}, the homeomorphismS2≅ℂ​ℙ1S^\{2\}\\cong\\mathbb\{CP\}^\{1\}identifiesxxwith a point that represents a complex projective lineℓ⊂ℂ2\\ell\\subset\\mathbb\{C\}^\{2\}\. The fiberπt−1​\(x\)\\pi\_\{t\}^\{\-1\}\(x\)is given by the 1\-dimensional complex vector spaceℓ\\ellitself\. We note thatπt:T→S2\\pi\_\{t\}:T\\to S^\{2\}can also be interpreted as a 2\-dimensional real vector bundle by using the fact thatℂ\\mathbb\{C\}is a 2\-dimensionalℝ\\mathbb\{R\}\-vector space\.

The proof follows from applying Proposition 1\.14 of Hatcher\[[23](https://arxiv.org/html/2610.21087#bib.bib35)\]\. The isomorphism classes ofnn\-dimensional real oriented vector bundles overS2S^\{2\}is isomorphic to the fundamental groupπ1​\(S​O​\(n\)\)\\pi\_\{1\}\(SO\(n\)\)\. This group is isomorphic toπ1​\(S​O​\(3\)\)\\pi\_\{1\}\(SO\(3\)\)\. To see this, we consider the short exact sequence of topological spaces obtained from fiber bundle ofSnS^\{n\},

S​O​\(n\)→S​O​\(n\+1\)→Sn,SO\(n\)\\to SO\(n\+1\)\\to S^\{n\},\(8\)from which the long exact sequence of homotopy groups can be obtained using Theorem 4\.41 of\[[22](https://arxiv.org/html/2610.21087#bib.bib34)\]:

⋯→π2​\(Sn\)→π1​\(S​O​\(n\)\)→π1​\(S​O​\(n\+1\)\)→π1​\(Sn\)→⋯\.\\cdots\\to\\pi\_\{2\}\(S^\{n\}\)\\to\\pi\_\{1\}\(SO\(n\)\)\\to\\pi\_\{1\}\(SO\(n\+1\)\)\\to\\pi\_\{1\}\(S^\{n\}\)\\to\\cdots\.\(9\)The triviality of higher homotopy groupsπ2​\(Sn\)=π1​\(Sn\)=0\\pi\_\{2\}\(S^\{n\}\)=\\pi\_\{1\}\(S^\{n\}\)=0forn≥3n\\geq 3, see in particular Corollary 4\.9 of\[[22](https://arxiv.org/html/2610.21087#bib.bib34)\], implies that we haveπ1​\(S​O​\(n\)\)≅π1​\(S​O​\(n\+1\)\)\\pi\_\{1\}\(SO\(n\)\)\\cong\\pi\_\{1\}\(SO\(n\+1\)\)for alln≥3n\\geq 3\. By induction we haveπ1​\(S​O​\(n\)\)≅π1​\(S​O​\(3\)\)\\pi\_\{1\}\(SO\(n\)\)\\cong\\pi\_\{1\}\(SO\(3\)\)for alln≥3n\\geq 3\. But using the homeomorphismS​O​\(3\)≃ℝ​ℙ3SO\(3\)\\simeq\\mathbb\{RP\}^\{3\}, we obtain

π1​\(S​O​\(3\)\)≅π1​\(ℝ​ℙ3\)≅ℤ/2​ℤ\.\\pi\_\{1\}\(SO\(3\)\)\\cong\\pi\_\{1\}\(\\mathbb\{RP\}^\{3\}\)\\cong\\mathbb\{Z\}/2\\mathbb\{Z\}\.\(10\)Hence, every vector bundle overS2S^\{2\}is either isomorphic to the trivial vector bundleℝn×S2→S2\\mathbb\{R\}^\{n\}\\times S^\{2\}\\to S^\{2\}, or a non\-trivial vector bundle of dimensionnn\. We note that becauseπt:T→S2\\pi\_\{t\}:T\\to S^\{2\}is a non\-trivial real vector bundle of dimension 2\. \(As a complex line bundle,L⊗LL\\otimes Lgives the holomorphic tangent bundle ofS2S^\{2\}\.\) Becausen≥3n\\geq 3, every non\-trivial vector bundle of dimensionnnis isomorphic to the vector bundle obtained by taking direct sum of fibers ofLLwith those of the trivial vector bundleℝn−2×S2→S2\\mathbb\{R\}^\{n\-2\}\\times S^\{2\}\\to S^\{2\}\. This finishes the proof of the theorem\.

### A\.3Additional elaboration and proof of Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)

We recall the following application of Proposition 1\.4 to our setup from\[[23](https://arxiv.org/html/2610.21087#bib.bib35)\]\.

###### Proposition A\.2\.

Given a vector bundleπ:VY′→Y′\\pi:V\_\{Y\}^\{\\prime\}\\to Y^\{\\prime\}over a closed smooth manifoldY′⊂ℝ3Y^\{\\prime\}\\subset\\mathbb\{R\}^\{3\}, there exists a vector bundleE→Y′E\\to Y^\{\\prime\}such thatVY′⊕EV\_\{Y\}^\{\\prime\}\\oplus Eis the trivial vector bundle overY′Y^\{\\prime\}\.

The construction of the vector bundleEEis obtained by finding a finite open cover\{Ui\}i=1J\\\{U\_\{i\}\\\}\_\{i=1\}^\{J\}ofY′Y^\{\\prime\}, over which the restriction of the vector bundleVY′→UiV\_\{Y\}^\{\\prime\}\\to U\_\{i\}is the trivial vector bundleUi×ℝm→UiU\_\{i\}\\times\\mathbb\{R\}^\{m\}\\to U\_\{i\}\(often denoted as local trivialization ofVY′V\_\{Y\}^\{\\prime\}\)\. Using Urysohn’s lemma, there exists a functiongi:VY′→ℝmg\_\{i\}:V\_\{Y\}^\{\\prime\}\\to\\mathbb\{R\}^\{m\}which factors through the local trivializationVY′→UiV\_\{Y\}^\{\\prime\}\\to U\_\{i\}\. One can then define a mapg:VY′→ℝm​J=∏i=1Jℝmg:V\_\{Y\}^\{\\prime\}\\to\\mathbb\{R\}^\{mJ\}=\\prod\_\{i=1\}^\{J\}\\mathbb\{R\}^\{m\}by definingg:=\(g1,g2,⋯,gJ\)g:=\(g\_\{1\},g\_\{2\},\\cdots,g\_\{J\}\)\. Then the map\(g,π\):VY′→ℝm​J×Y′\(g,\\pi\):V\_\{Y\}^\{\\prime\}\\to\\mathbb\{R\}^\{mJ\}\\times Y^\{\\prime\}defines the isomorphism of vector bundles\. We note that the vector bundleE→Y′E\\to Y^\{\\prime\}is of dimension\(J−1\)​m\(J\-1\)m\. BecauseY′Y^\{\\prime\}is homotopy equivalent to a smooth orientable closed surface of genusgg, there must be at least22open coversU1,U2U\_\{1\},U\_\{2\}, over each of which one obtains a trivialization of vector bundleVY′→UiV\_\{Y\}^\{\\prime\}\\to U\_\{i\}\.

In the case of constant masking function, we may consider the subspaceℝ⊂ℝN\\mathbb\{R\}\\subset\\mathbb\{R\}^\{N\}spanned by the fixed choice of a vectorv∈ℝNv\\in\\mathbb\{R\}^\{N\}\. Hence, constant masking function corresponds to replacing the vector bundleVXV\_\{X\}withℝ×X\\mathbb\{R\}\\times X, because the range of the masking function lies inℝ×X\\mathbb\{R\}\\times Xinstead ofℝN×X\\mathbb\{R\}^\{N\}\\times X\. This implies that if𝔪:X→ℝ\\mathfrak\{m\}:X\\to\\mathbb\{R\}is a constant masking function, then the only possible way to embed a non\-trivial vector bundleVY′→Y′V\_\{Y\}^\{\\prime\}\\to Y^\{\\prime\}intoY′×ℝ→Y′Y^\{\\prime\}\\times\\mathbb\{R\}\\to Y^\{\\prime\}is by establishing an isomorphism of vector bundles betweenVY′≅Y′×ℝV\_\{Y\}^\{\\prime\}\\cong Y^\{\\prime\}\\times\\mathbb\{R\}, a contradiction\. This finishes the proof of Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)\.

We note that in the case where𝔪:X→ℝN\\mathfrak\{m\}:X\\to\\mathbb\{R\}^\{N\}is a masking function that is not a section of a non\-trivial vector bundle, the proof of the above proposition requires that the dimension ofEEhas to be at leastmm\. Therefore, unlessN≥2​mN\\geq 2m, the above method cannot be utilized to ensure that the non\-trivial vector bundleVY′→Y′V\_\{Y\}^\{\\prime\}\\to Y^\{\\prime\}can be regarded as a vector sub\-bundle of a trivial vector bundle of dimensionNN\. Even if one enlarges the parameterNNto be greater than2​m2m, finding exactly what vector sub\-bundle of the trivial bundle is the non\-trivial bundleVY′V\_\{Y\}^\{\\prime\}is a challenging task\. This is because the construction of the map\(g,π\)\(g,\\pi\)depends crucially on the choice of the open cover\{Ui\}i=1J\\\{U\_\{i\}\\\}\_\{i=1\}^\{J\}over which the non\-trivial vector bundleVY′V\_\{Y\}^\{\\prime\}admits local trivialization\. It would be a very difficult task for the neural network to predict such geometric data regarding open coversUiU\_\{i\}, especially in cases where even the auxiliary spaceY′Y^\{\\prime\}parametrizing the point clouds is not understood correctly\.

We also elaborate more on the relation between Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)and Figure[5](https://arxiv.org/html/2610.21087#S4.F5)\. We thank an anonymous reviewer for pointing out the need to clarify the relation between these two results\. Letf:Y′→VY′f:Y^\{\\prime\}\\to V\_\{Y\}^\{\\prime\}be a non\-trivial global section\. We note thatffshall be regarded as a restriction of a continuous functionF:ℝ3→ℝmNF:\\mathbb\{R\}^\{3\}\\to\\mathbb\{R\}^\{m\_\{N\}\}toY′Y^\{\\prime\}\. BecauseY′Y^\{\\prime\}is compact, Heine\-Cantor theorem implies thatffis uniformly continuous\. Hence, given anyϵ\>0\\epsilon\>0, there existsδ\>0\\delta\>0such that for anyx,y∈Y′x,y\\in Y^\{\\prime\}with\|x−y\|<δ\|x\-y\|<\\delta, we have\|f⁡\(x\)−f⁡\(y\)\|<ϵ\|f\(x\)\-f\(y\)\|<\\epsilon\. Given that the learning objective of SIEVE is to optimizeL1L\_\{1\}norm, we will denote by\|⋅\|\|\\cdot\|theL1L\_\{1\}norm overℝ3\\mathbb\{R\}^\{3\}andℝmN\\mathbb\{R\}^\{m\_\{N\}\}, respectively\.

LetΦ\\Phibe theSE⁡\(3\)\\mathrm\{SE\}\(3\)\-equivariant neural network with constant masking function and scalar pipeline perturbed by Gaussian noise\. SupposeΦ\\Phieffectively approximatesff\. Then for everyx∈Y′x\\in Y^\{\\prime\}we have\|Φ⁡\(x\)−f⁡\(x\)\|<ϵ\|\\Phi\(x\)\-f\(x\)\|<\\epsilon\. Fix a pointx∈Y′x\\in Y^\{\\prime\}\. Using triangle inequality and uniform continuity offf, for every suchy∈Y′y\\in Y^\{\\prime\}with\|x−y\|<δ\|x\-y\|<\\deltawe have

ϵ\>\|Φ⁡\(x\)−f⁡\(x\)\|≥\|Φ⁡\(x\)−Φ⁡\(y\)​\|−\|Φ⁡\(y\)−f⁡\(y\)\|−\|​f​\(y\)−f⁡\(x\)\|≥\|Φ⁡\(x\)−Φ⁡\(y\)\|−2​ϵ\.\\epsilon\>\|\\Phi\(x\)\-f\(x\)\|\\geq\|\\Phi\(x\)\-\\Phi\(y\)\|\-\|\\Phi\(y\)\-f\(y\)\|\-\|f\(y\)\-f\(x\)\|\\geq\|\\Phi\(x\)\-\\Phi\(y\)\|\-2\\epsilon\.\(11\)Hence, we require that for everyy∈Y′y\\in Y^\{\\prime\}such that\|x−y\|<δ\|x\-y\|<\\delta, we have\|Φ⁡\(x\)−Φ⁡\(y\)\|<3​ϵ\|\\Phi\(x\)\-\\Phi\(y\)\|<3\\epsilon, i\.e\.Φ\\Phiis also uniformly continuous\. Since the masking function is constant, there exists a vectorv∈ℝmv\\in\\mathbb\{R\}^\{m\}for somem\>0m\>0such that

Φ⁡\(x\)=Decoder⁡\(v,Encoder⁡\(\[μx,x\]\)\),\\Phi\(x\)=\\mathrm\{Decoder\}\(v,\\mathrm\{Encoder\}\(\[\\mu\_\{x\},x\]\)\),\(12\)whereμx\\mu\_\{x\}denotes perturbation by Gaussian noise𝒩⁡\(0,I\)\\mathcal\{N\}\(0,I\), and the appropriate projection maps for encoder and decoder functions are abbreviated\. Here, givena∈ℝn1a\\in\\mathbb\{R\}^\{n\_\{1\}\}andb∈ℝn2b\\in\\mathbb\{R\}^\{n\_\{2\}\}, the notation\[a,b\]∈ℝn1\+n2\[a,b\]\\in\\mathbb\{R\}^\{n\_\{1\}\+n\_\{2\}\}denotes concatenation of vectorsaaandbb\. Both encoder and decoder functions from GATr are continuous\. Hence, there exists someδ∗\>0\\delta^\{\*\}\>0such that

\|\[μx,x\]−\[μy,y\]\|<δ∗⇒\|Φ⁡\(x\)−Φ⁡\(y\)\|<3​ϵ\.\|\[\\mu\_\{x\},x\]\-\[\\mu\_\{y\},y\]\|<\\delta^\{\*\}\\Rightarrow\|\\Phi\(x\)\-\\Phi\(y\)\|<3\\epsilon\.\(13\)From our assumption, we had\|x−y\|<δ\|x\-y\|<\\delta\. Hence, by the definition ofL1L\_\{1\}norm and triangle inequality, we have

δ∗\>\|\[μx,x\]−\[μy,y\]\|=\|\[μx−μy,x−y\]\|=\|μx−μy\|\+\|x−y\|\.\\delta^\{\*\}\>\|\[\\mu\_\{x\},x\]\-\[\\mu\_\{y\},y\]\|=\|\[\\mu\_\{x\}\-\\mu\_\{y\},x\-y\]\|=\|\\mu\_\{x\}\-\\mu\_\{y\}\|\+\|x\-y\|\.\(14\)This implies that for anyy∈Y′y\\in Y^\{\\prime\}such that\|x−y\|<δ\|x\-y\|<\\delta, we have\|μx−μy\|<δ∗−δ\|\\mu\_\{x\}\-\\mu\_\{y\}\|<\\delta^\{\*\}\-\\delta\. However, becauseμx\\mu\_\{x\}andμy\\mu\_\{y\}are identically independently distributed, and there are arbitrarily many points lying in an open neighborhood ofxxof radiusδ\\delta, the inequality\|μx−μy\|<δ∗−δ\|\\mu\_\{x\}\-\\mu\_\{y\}\|<\\delta^\{\*\}\-\\deltacannot hold uniformly for all suchy∈Y′y\\in Y^\{\\prime\}almost surely, unlessδ∗\\delta^\{\*\}is arbitrarily large\. Hence, with suchδ∗\\delta^\{\*\}to be chosen to be arbitrarily large, in order for the inequality\|Φ⁡\(x\)−Φ⁡\(y\)\|<3​ϵ\|\\Phi\(x\)\-\\Phi\(y\)\|<3\\epsilonto be satisfied, we require that there exists some arbitrarily smallϵ′′\>0\\epsilon^\{\\prime\\prime\}\>0such that eithersupx∈Y′\|Encoder⁡\(\[μx,x\]\)\|<ϵ′′\\sup\_\{x\\in Y^\{\\prime\}\}\|\\mathrm\{Encoder\}\(\[\\mu\_\{x\},x\]\)\|<\\epsilon^\{\\prime\\prime\}, orsupx∈Y′\|Φ⁡\(x\)\|<ϵ′′\\sup\_\{x\\in Y^\{\\prime\}\}\|\\Phi\(x\)\|<\\epsilon^\{\\prime\\prime\}\. Suppose the former holds\. \(We note that the latter case implies thatffis a trivial function, contradicting our assumption onff\)\. This implies that the domain of the decoder function \(the last component ofΦ\\Phi\) is contained in a compact subset corresponding to a ball of radiusϵ′′\>0\\epsilon^\{\\prime\\prime\}\>0from the constant vectorv∈ℝmv\\in\\mathbb\{R\}^\{m\}that we chose earlier\. Within this domain, the decoder function is uniformly continuous, hence one can chooseϵ\>0\\epsilon\>0to be arbitrarily small so that the image ofΦ\\Phiis mapped to a ball of arbitrarily small radius centered atΦ⁡\(v\)\\Phi\(v\)\. This explain the empirical collapse in Figure[5](https://arxiv.org/html/2610.21087#S4.F5)\.

In relation to Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6), the above argument shows that ifΦ\\Phiwere to approximate the global sectionf:Y′→VY′f:Y^\{\\prime\}\\to V\_\{Y\}^\{\\prime\}effectively, thenffwould have been a constant functionΦ⁡\(v\)\\Phi\(v\)for some fixed vectorvvwith arbitrarily small perturbations\. We can then proceed as in the proof of Theorem[2\.6](https://arxiv.org/html/2610.21087#S2.Thmtheorem6)stated above to conclude that the only global sections of vector bundles suchΦ\\Phican approximate \(almost surely\) are rank11trivial bundle of formℝ×Y′→Y′\\mathbb\{R\}\\times Y^\{\\prime\}\\to Y^\{\\prime\}\.

### A\.4Proof of Theorem[3\.1](https://arxiv.org/html/2610.21087#S3.Thmtheorem1)

The theorem follows from considering the reformulation of the architectural components of SIEVE into the following steps as follows\.

1. 1\.Using 3D point clouds, consider an auxiliary \(possibly unknown\)nn\-dimensional vector bundleVY→YV\_\{Y\}\\to Y’ of some 3\-dimensional closed orientable manifoldYY’ homeomorphic toX×IX\\times Ifor some closed orientable surfaceX⊂ℝ3X\\subset\\mathbb\{R\}^\{3\}of genusgg\. We regard thenn\-dimensional measurements over 3D point clouds correspond as a global sectionf:Y′→VYf:Y^\{\\prime\}\\to V\_\{Y\}of thenn\-dimensional vector bundle overY′Y^\{\\prime\}\.
2. 2\.\(Encoder\):Find anmm\-dimensional vector bundleVYV\_\{Y\}’→Y\\to Y’ withm≥2​n\+5m\\geq 2n\+5such that the fiber ofVYV\_\{Y\}injects into a vector subspace of the fiber ofVYV\_\{Y\}’ at everyy∈Yy\\in Y’\.
3. 3\.Consider an auxiliarymm\-dimensional vector bundleVX→XV\_\{X\}\\to Xsuch thatVXV\_\{X\}andVY′V\_\{Y\}^\{\\prime\}are equivalent vector bundles with respect to homotopy equivalenceX×I→XX\\times I\\to X\.
4. 4\.\(Spherical Masking\):Using the spherical projectionπ2:X→S2\\pi\_\{2\}:X\\to S^\{2\}which is the continuous map, replace themm\-dimensional vector bundleVX→XV\_\{X\}\\to Xwith the pullback of somemm\-dimensional vector bundleV→S2V\\to S^\{2\}\. This replacement operator is well defined ifVXV\_\{X\}is isomorphic to the pullback bundleπ2∗​V\\pi\_\{2\}^\{\*\}Vas vector bundles overXX, which is why the statement of the theorem makes technical assumption on isomorphism class of vector bundles that SIEVE can approximate their global sections\. By definition of spherical masking,VVcontains the dimension22tautological bundleπt:T→S2\\pi\_\{t\}:T\\to S^\{2\}as its sub\-vector bundle\. Using classification of vector bundles overS2S^\{2\}, the vector bundleV→S2V\\to S^\{2\}must be isomorphic to either the trivial vector bundle or the unique non\-trivial vector bundle of dimensionmm, whose fibers given by a summand of formℝm−2⊕πt−1​\(x\)\\mathbb\{R\}^\{m\-2\}\\oplus\\pi\_\{t\}^\{\-1\}\(x\)for everyx∈S2x\\in S^\{2\}\.
5. 5\.The spherical projection transforms thenn\-dimensional measurements over 3D point clouds via the composition of maps f∗:Y′→VY↪VY′≅VX≅π2∗​V→π2∗​T\.f^\{\*\}:Y^\{\\prime\}\\to V\_\{Y\}\\hookrightarrow V\_\{Y\}^\{\\prime\}\\cong V\_\{X\}\\cong\\pi\_\{2\}^\{\*\}V\\to\\pi\_\{2\}^\{\*\}T\.\(15\)Here, the first map is our global sectionf:Y′→VYf:Y^\{\\prime\}\\to V\_\{Y\}\. The second map is the inclusion of fibers ofVYV\_\{Y\}intoVY′V\_\{Y\}^\{\\prime\}\(i\.e\. the encoder\)\. The third map originates from equivalence of vector bundles betweenXXandY′Y^\{\\prime\}\. The fourth and fifth map originates from replacing the vector bundleVXV\_\{X\}with the pullback bundle ofV→S2V\\to S^\{2\}using the spherical masking function\. Note that the fifth map originates from the projecting the vector bundleπ2∗​V\\pi\_\{2\}^\{\*\}Vto its vector sub\-bundleπ2∗​T\\pi\_\{2\}^\{\*\}T\. \(The previous step implies thatπ2∗​V≅ℝm−2⊕π2∗​T\\pi\_\{2\}^\{\*\}V\\cong\\mathbb\{R\}^\{m\-2\}\\oplus\\pi\_\{2\}^\{\*\}T, so the fifth projection map can be explicitly define by mappingℝm−2\\mathbb\{R\}^\{m\-2\}to\{0\}\\\{0\\\}\)\.
6. 6\.\(Decoder\):Reconstruct thenn\-dimensional vector bundleVY→YV\_\{Y\}\\to Y’ as a closed orientablen\+2n\+2dimensional submanifold ofℝm\+3\\mathbb\{R\}^\{m\+3\}, and approximate the global sectionf:Y′→VYf:Y^\{\\prime\}\\to V\_\{Y\}using the spherical projection of measurements of 3D point cloudf∗:Y′→π2∗​Tf^\{\*\}:Y^\{\\prime\}\\to\\pi\_\{2\}^\{\*\}T\.

The fact that spherical projection masking corresponds to replacing the vector bundle with the pullback of the non\-trivial vector bundle overS2S^\{2\}can be shown from the following reasoning\. Using the definition of pullback of vector bundles and tautological bundles, the feature vectors obtained from the spherical masking functionSphericalMask\\mathrm\{SphericalMask\}lie in the span of a linear subspace ofℂ2\\mathbb\{C\}^\{2\}that passes through the mean𝐱¯\\overline\{\\mathbf\{x\}\}\. Up to linear change of coordinates that sends𝐱¯\\overline\{\\mathbf\{x\}\}to the point of origin0∈ℂ10\\in\\mathbb\{C\}^\{1\}, we obtain that the spherical masking functionSphericalMask\\mathrm\{SphericalMask\}corresponds to defining a global sectiong:X→π2∗​Tg:X\\to\\pi\_\{2\}^\{\*\}T, where for eachx∈Xx\\in X, we haveg⁡\(x\)=SphericalMask⁡\(x\)g\(x\)=\\mathrm\{SphericalMask\}\(x\)\. For general vector bundlesVX→XV\_\{X\}\\to X, we can hence useSphericalMask\\mathrm\{SphericalMask\}to replace them with pullback of vector bundleV→S2V\\to S^\{2\}which containsT→S2T\\to S^\{2\}as a vector sub\-bundle\.

The requirement form≥2​n\+5m\\geq 2n\+5need not be satisfied, but we explain the motivation behind such a choice\. By construction, we may assume that the vector bundleVY→YV\_\{Y\}\\to Y’ is a smooth compact orientable manifold of dimensionn\+2n\+2\. By Haefliger\-Hirsch, C\.T\.C\. Wall, and Ghanwat\-Pancholi’s results, every such manifold embed intoℝ2​\(n\+2\)−1=ℝ2​n\+3\.\\mathbb\{R\}^\{2\(n\+2\)\-1\}=\\mathbb\{R\}^\{2n\+3\}\.\[[19](https://arxiv.org/html/2610.21087#bib.bib36)\]\[[42](https://arxiv.org/html/2610.21087#bib.bib37)\]\[[17](https://arxiv.org/html/2610.21087#bib.bib38)\]Using the fact thatVVcontains fibers isomorphic to the Euclidean spaceℝh\\mathbb\{R\}^\{h\}of dimension at leastm−2m\-2, the condition onmmensures to identify the desired vector bundleVYV\_\{Y\}as a smoothly embedded submanifold of the vector bundleV→S2V\\to S^\{2\}\. We note that the reconstruction of the desired vector bundleVY→YV\_\{Y\}\\to Y’ can be obtained from applying the universal approximation theorem for approximating the desired continuous functionF:ℝm\+3→ℝm\+3F:\\mathbb\{R\}^\{m\+3\}\\to\\mathbb\{R\}^\{m\+3\}whose image is homeomorphic to the vector bundleVYV\_\{Y\}\.

The approximation of the global sectionf:Y′→VYf:Y^\{\\prime\}\\to V\_\{Y\}usingf∗:Y′→π2∗​Tf^\{\*\}:Y^\{\\prime\}\\to\\pi\_\{2\}^\{\*\}Tfollows from using universal approximation theorem forℝm\\mathbb\{R\}^\{m\}\. Without loss of generality, we assumeVYV\_\{Y\}is a non\-trivial vector bundle which are isomorphic to pullback of a non\-trivial vector bundle ofS2S^\{2\}\. The decoder satisfying conditions of universal approximation theorem for approximating continuous functionsf:ℝm→ℝnf:\\mathbb\{R\}^\{m\}\\to\\mathbb\{R\}^\{n\}can be used to approximate global sections of trivial vector bundleg:Y′→ℝn−2g:Y^\{\\prime\}\\to\\mathbb\{R\}^\{n\-2\}by approximating local restrictions ofggto an open affine chart\{Uα\}α∈𝒜\\\{U\_\{\\alpha\}\\\}\_\{\\alpha\\in\\mathcal\{A\}\}ofY′Y^\{\\prime\}and gluing such local restrictions ofggwithout any obstructions\. Because the vector bundleVYV\_\{Y\}is isomorphic toℝn−2⊕π2∗​T\\mathbb\{R\}^\{n\-2\}\\oplus\\pi\_\{2\}^\{\*\}T, we can use the product of functions\(g,f∗\):Y′→ℝn−2⊕π2∗​T≅VY\(g,f^\{\*\}\):Y^\{\\prime\}\\to\\mathbb\{R\}^\{n\-2\}\\oplus\\pi\_\{2\}^\{\*\}T\\cong V\_\{Y\}to approximate any global section ofVYV\_\{Y\}\.

### A\.5Theoretical Limitations

Our theoretical results suggest that SIEVE is capable of approximating global sections of a wide range of classes of non\-trivial vector bundles over manifolds\. Nevertheless, as similar to other state\-of\-the\-art techniques, SIEVE has some theoretical limitations, in that it does not guarantee outperformance in analyzing datasets which are global sections of vector bundles which do not contain pullback of tautological bundles as vector sub\-bundle\. For instance, there may be many non\-trivial isomorphism classes of vector bundles over manifolds which do not originate from pullback of non\-trivial vector bundles overS2S^\{2\}\. Further studies would be required to devise new neural networks which are capable of approximating such global sections of nontrivial vector bundles\.

## Appendix BImplementation Details

### B\.1Pre\-Training

Model Parameters:The backbone of pre\-trained SIEVE is GATr, a model based on multivector representations\[[5](https://arxiv.org/html/2610.21087#bib.bib9)\]\. We embed the position of each node as a trivector inside the multivector\.

The encoder comprises a 9\-layer GATr with 16 multivector and 64 scalar channels\. The input is 1 multivector and 1 scalar channel and the output is 1 multivector and 32 scalar channel\. The 32\-dimensional scalar output constitutes the SIEVE embeddingsc​o​d​es\_\{code\}\. The encoder has 949,825 parameters\.

The decoder is a 3\-layer GATr with 16 multivector and 64 scalar channels\. The input is 1 multivector and 1 scalar channel and the output is a single\-channel multivector and scalar output for coordinate reconstruction\. The decoder has 318,690 parameters\.

Note that for variants of SIEVE trained with ShapeNet\[[8](https://arxiv.org/html/2610.21087#bib.bib11)\], the model parameters from Table[4](https://arxiv.org/html/2610.21087#A2.T4)are used\.

Training Method:The model is trained for 10 epochs using the AdamW optimizer with a batch size of 200\. To accelarate training, bf16 quantization is used\. We employ a cosine learning rate schedule with a warm\-up phase, reaching a maximum learning rate of3×10−43\\times 10^\{\-4\}\. To stabilize training, we apply gradient clipping with a maximum norm of 10\. We train on a single Nvidia B200 GPU over 4 days\. Memory consumption with a batch size of 200 is 154 GB\.

Additional Techniques:Three distinct pre\-training strategies have been tested on the smaller ShapeNet dataset\[[8](https://arxiv.org/html/2610.21087#bib.bib11)\]\.

The first strategy utilizes simple position reconstruction as outlined in the main text, with pre\-training performed on a volumetric ShapeNet dataset\. The standard deviation of the spherical projection mask noise isσ=0\.02\\sigma=0\.02\.

The second strategy is to reconstruct the normal vector and the curvature in addition to the position of the node\. The normal vector is the output multivector’s vector, and the curvature is the output scalar\. The reconstruction loss for this strategy is

ℒ=L​1​\(PredPos,Pos\)\+10⋅\(1−cos⁡\(PredNorm⋅Norm\)\+5⋅L​1​\(PredCurv,Curv\)CLOSE\.\\mathcal\{L\}=L1\(\\mathrm\{PredPos,Pos\}\)\+10\\cdot\(1\-\\cos\(\\mathrm\{PredNorm\}\\cdot\\mathrm\{Norm\}\)\+5\\cdot L1\(\\mathrm\{PredCurv,Curv\}\)\.\(16\)We use the surface ShapeNet dataset for this strategy, as neither curvature and normal vectors can be defined in the interior\.

The third strategy involves the application of a nearest\-neighbor attention mask within the decoder\. To ensure the encoder extracts global embeddings, the decoder is forced to reconstruct the nodes without local information by masking70%70\\%of the nearest nodes during self\-attention\. This strategy is implemented on a volumetric ShapeNet dataset\.

As discussed in Appendix[C\.3](https://arxiv.org/html/2610.21087#A3.SS3)and Table[6](https://arxiv.org/html/2610.21087#A3.T6), the third strategy of using a nearest\-neighbor attention mask yielded the best results\. Therefore, the final OpenShape\[[30](https://arxiv.org/html/2610.21087#bib.bib44)\]pretrained SIEVE utilizes a nearest neighbor attention mask with ratio0\.70\.7along with a spherical projection mask\.

### B\.2Synthetic Data Experiment

Model Parameters:The mask ablation study \(Section[4\.2](https://arxiv.org/html/2610.21087#S4.SS2)\) evaluates two different equivariant backbone architectures: GATr\[[5](https://arxiv.org/html/2610.21087#bib.bib9)\]and SE\(3\)\-Transformer\[[16](https://arxiv.org/html/2610.21087#bib.bib12)\]\. As detailed in Table[4](https://arxiv.org/html/2610.21087#A2.T4), both models utilize matching hidden channels and layer numbers to ensure a fair comparison\. The positions are embedded as trivectors for GATr and as type\-1 vectors for the SE\(3\)\-Transformer\. For the SE\(3\)\-Transformer, we generate edges between the nodes usingkk\-nearest neighbor withk=50k=50\.

Table 4:Model Hyperparameters for the Mask Ablation StudyMask Details:The constant mask is a single vector or multivector token that is applied uniformly over all of the nodes\. This token is trainable\. The noise mask is a uniform sample of points on an ellipsoid surface with semi\-axes\(2,1,0\.5\)\(2,1,0\.5\)\. The ellipsoid is centered at each point cloud’s centroid, and the orientation is determined by PCA of the original point cloud: x\-axis for the largest variance direction, y\-axis for the middle, and z\-axis for the smallest\. This ensures that the noise mask is also an equivariant operator\. The spherical projection mask is as described in Section[3\.1](https://arxiv.org/html/2610.21087#S3.SS1), but no noise is added to the projected position\.

Training Method:Both models are trained with a simpleℒ​1\\mathcal\{L\}1loss on predicted node positions\. The models are trained over 200 epochs, with a batch size of 4\. We employ a cosine learning rate decay that reaches a maximum learning rate of3×10−43\\times 10^\{\-4\}after a10%10\\%warm up period\. To ensure numerical stability, the gradients are clipped at a threshold of 10\.0\. No weight decay is applied\.

### B\.3Downstream Tasks

Each downstream task utilizes a domain specific architecture\. For the 3D\-EPN point cloud completion task, we utilize PCN\[[51](https://arxiv.org/html/2610.21087#bib.bib32)\]; for the point cloud registration task on ModelNet40, we utilize DCP\[[44](https://arxiv.org/html/2610.21087#bib.bib39)\]\. All downstream models use the hyperparameters specified in their original publications and official implementations\. The only modification made is to the input channel to accommodate the additional 32\-dimensionsc​o​d​es\_\{code\}embeddings\. The SIEVE embeddings are concatenated along with the position as the input\. The SIEVE encoder is frozen when used in a downstream task; we do not fine\-tune it\. The RNA segmentation task and the geometric registration task use the OpenShape pretrained encoder, and the 3D shape completion task uses the ShapeNet pretrained encoder with the "simple noise" setting\.

The added downstream task training time is a single forward pass through the frozen SIEVE encoder\. On the RNA segmentation task with PointNet\+\+, memory increases from 2134 MiB to 2166 MiB, and per\-epoch time from 92s to 123s \(\+31s\)\. On 3D\-EPN with PCN, per\-epoch time increases from 4 min 56s to 5 min 43 sec \(\+47s\)\. The absolute overhead is comparable in both cases, so its relative share falls as the downstream backbone becomes more expensive: 34% for PointNet\+\+ against 16% for PCN\.

## Appendix CAblation Study

### C\.1Reconstruction of Point Clouds with Identical Topology

To demonstrate that the topological obstruction is the cause of the representational collapse, we train SIEVE to reconstruct a point cloud dataset with identical topology\. We use DeepJEB, which contains 2138 meshes of jet engine brackets\[[25](https://arxiv.org/html/2610.21087#bib.bib43)\]\. These engine brackets all have two holes at the top and 4 holes at the bottom; therefore all of the meshes have the same topology\.

\(a\)Reconstruction Validation Loss![Refer to caption](https://arxiv.org/html/2610.21087v1/DeepJEB_reconstruction.png)\(b\)Reconstruction Result

Figure 6:SIEVE on DeepJEB\.DeepJEB is a point cloud dataset that contains objects with identical topology\. Therefore, both the constant mask SIEVE and the spherical mask SIEVE is able to do near\-perfect reconstruction\.The results \(Figure[6](https://arxiv.org/html/2610.21087#A3.F6)\) show that both constant mask SIEVE and spherical mask SIEVE are able to reconstruct point clouds in the DeepJEB dataset\. Hence we confirm that it is the topological obstruction that is causing the representational collapse in Section[4\.2](https://arxiv.org/html/2610.21087#S4.SS2), not an inherent architectural limitation\.

### C\.2Scalar Input of Encoder

Our choice to use Gaussian noise as the scalar input of the equivariant encoder is based on its capability and ease of use\. This subsection is an ablation study on the scalar input of the encoder to justify this selection\.

We compare noise to two other scalar encoder inputs: a constant array of ones𝟏=\[1,1,⋯,1\]T\\mathbf\{1\}=\[1,1,\\cdots,1\]^\{T\}and a handpicked invariant scalar feature of the point cloud, for which we chose surface variance \(SV\)\. Given a node𝐱i\\mathbf\{x\}\_\{i\}, and its 16 closest neighbors𝒩⁡\(𝐱i\)\\mathcal\{N\}\(\\mathbf\{x\}\_\{i\}\), the covariance matrix of the 16 neighbors’ relative position𝐩i​j=𝐱j−𝐱i,𝐱j∈𝒩⁡\(𝐱i\)\\mathbf\{p\}\_\{ij\}=\\mathbf\{x\}\_\{j\}\-\\mathbf\{x\}\_\{i\},\\mathbf\{x\}\_\{j\}\\in\\mathcal\{N\}\(\\mathbf\{x\}\_\{i\}\)is defined asKK\. Denoting the three eigenvalue ofKKasλ1≥λ2≥λ3\\lambda\_\{1\}\\geq\\lambda\_\{2\}\\geq\\lambda\_\{3\}, sv is defined assv=λ1/\(λ1\+λ2\+λ3\)\\mathrm\{sv\}=\\lambda\_\{1\}/\(\\lambda\_\{1\}\+\\lambda\_\{2\}\+\\lambda\_\{3\}\)\.

Three distinct SIEVE models were trained with different encoder inputs\. The ModelNet40 dataset was used to pre\-train all of the models, and the backbone model for SIEVE was GATr\. Training was conducted over 100 epochs with a batch size of 32, using a cosine learning rate decay with a maximum learning rate of3×10−43\\times 10^\{\-4\}\.

![Refer to caption](https://arxiv.org/html/2610.21087v1/SIEVE_input_comparison_ModelNet40.png)

Figure 7:Input Noise Ablation\.We train SIEVE on ModelNet40 with three different input: local point cloud feature sv, a constant array of ones, and noise\. Either noise or sv input enables SIEVE to reconstruct the input point cloud\. SIEVE with constant input shows significantly worse performance when reconstructing the input point cloud\.Figure[7](https://arxiv.org/html/2610.21087#A3.F7)showcases the reconstruction results of SIEVE with different encoder inputs\. Both sv and noise are shown to be valid inputs; SIEVE is capable of reconstructing the input point cloud nearly perfectly with either\. The constant array input is shown to be suboptimal\. While it does not yield a complete collapse, it demonstrates difficulties in reconstructing the geometric details of the point cloud\.

We choose noise over sv because it does not require pre\-processing of the input point cloud\. Relying on handpicked SE\(3\)\-invariant point cloud features increases the difficulty of implementing SIEVE across a wide range of datasets\.

It is interesting to note that the use of noise as input does not lead to the scalar input being completely ignored\. Figure[8](https://arxiv.org/html/2610.21087#A3.F8)showcases the result when SIEVE trained with noise as input receives sv as input instead\. The reconstruction is distorted, which would not be the case if the scalar input was simply ignored by the model\. We therefore conclude that the encoder utilizes the specific distribution of the scalar input to construct the embedding, further highlighting the importance of the input signal even when it is stochastic\.

![Refer to caption](https://arxiv.org/html/2610.21087v1/figures/ModelNet40_spherical_sv_input.png)

Figure 8:Noise Input Trained SIEVE with SV as Input\.The above figure demonstrates the result*\(blue, right\)*of reconstructing the input point cloud*\(red, left\)*using a noise\-input pre\-trained SIEVE with SV as scalar input\. The distorted reconstruction implies that the encoder does not simply ignore the scalar noise input\.The use of noise as an encoder input does not lead to SIEVE simply memorizing a specific noise realization for a given point cloud\. Table[5](https://arxiv.org/html/2610.21087#A3.T5)provides the results when using different random seeds to generate the noise input for the encoder\. While each seed yields a unique input noise for the encoder, these differences have almost no effect on SIEVE’s ability to reconstruct the point cloud\. The three different seeds yield nearly identical validation losses, demonstrating that the model can consistently generate embeddings over various samples of noise\.

Table 5:Reconstruction Validation Loss\(ℒ​1\)\(\\mathcal\{L\}1\)of SIEVE with different random seeds on the ModelNet40 dataset
### C\.3SIEVE Embeddings do not Collapse to the Radius

![Refer to caption](https://arxiv.org/html/2610.21087v1/SIEVE_PCA_Analysis.png)

Figure 9:SIEVE Embedding PCA Analysis\.We perform PCA on the SIEVE embeddingsc​o​d​es\_\{code\}and plot the percentage of each eigenvalueλi\\lambda\_\{i\}over the sum of all the eigenvalues\. In addition, the cumulative percentage is also plotted\. From the top left, we plot the analysis results for SIEVE with no noise, ‘simple noise’ strategy, ‘curv and norm’ strategy, and ‘nearest neighbor attention mask’ strategy\.In this subsection, we rigorously analyze the embeddings to demonstrate that they do not collapse into trivial representations of the radiusr=‖𝐱i−𝐱¯‖r=\\\|\\mathbf\{x\}\_\{i\}\-\\overline\{\\mathbf\{x\}\}\\\|\. We compare SIEVE using a simple spherical projection mask without any noise to a variant with noise added to the spherical projection\. We refer to this variant as ‘simple noise’\.

Additionally, we compare two further variations of SIEVE\. The first reconstructs the normal vector and the curvature in addition to the position\. We refer to this variant as ‘curv and norm’\. The second applies an additional self\-attention mask to the decoder, where for every node the nearest70%70\\%of nodes are masked during self\-attention\. We refer to this variant as ‘nearest neighbor attention mask’\. The models are pre\-trained with the settings outlined in Appendix[B\.1](https://arxiv.org/html/2610.21087#A2.SS1)\.

We first analyze the embeddings using principal component analysis \(PCA\)\. Given an input point cloud and its SIEVE embeddingsc​o​d​e∈ℝNp×16s\_\{code\}\\in\\mathbb\{R\}^\{N\_\{p\}\\times 16\}, the covariance of the embedding isK=1Np−1​\[sc​o​d​e−mean⁡\(sc​o​d​e\)\]T​\[sc​o​d​e−mean⁡\(sc​o​d​e\)\]∈ℝ16×16K=\\frac\{1\}\{N\_\{p\}\-1\}\[s\_\{code\}\-\\mathrm\{mean\}\(s\_\{code\}\)\]^\{T\}\[s\_\{code\}\-\\mathrm\{mean\}\(s\_\{code\}\)\]\\in\\mathbb\{R\}^\{16\\times 16\}\. SinceKKis a positive semi\-definite matrix, all the eigenvalues are non\-negative\. We sort the eigenvalues in decreasing orderλ1≥λ2≥⋯≥λ16\\lambda\_\{1\}\\geq\\lambda\_\{2\}\\geq\\cdots\\geq\\lambda\_\{16\}, and calculate the ratio of each eigenvalue relative to the total varianceλi∑i=116λi\\frac\{\\lambda\_\{i\}\}\{\\sum\_\{i=1\}^\{16\}\\lambda\_\{i\}\}\.

PCA analysis \(Figure[9](https://arxiv.org/html/2610.21087#A3.F9)\) showcases that not adding any noise to the spherical projection mask yields a dimensional collapse, with95%95\\%of the variation contained in the first principal component\. The addition of a simple Gaussian noise prevents this collapse, decreasing the first principal component’s variation percentage to84\.2%84\.2\\%, and increasing the second principal component’s variation percentage to9\.6%9\.6\\%\. Furthermore, using more difficult reconstruction strategies such as ‘curv and norm’ and ‘nearest neighbor attention mask’ mitigates the dimensional collapse even further, with both strategies showing variation being spread out over multiple principal components\.

Next, we directly compare the learned embeddings to the radial distancer=‖𝐱i−𝐱¯‖r=\\\|\\mathbf\{x\}\_\{i\}\-\\overline\{\\mathbf\{x\}\}\\\|\. We calculate the Pearson correlation coefficient between each of the 16 channels ofsc​o​d​es\_\{code\}and the radiusrrfor a representative point cloud sample\.

![Refer to caption](https://arxiv.org/html/2610.21087v1/SIEVE_radius_correlation.png)

Figure 10:Embedding Radius Correlation\.This plot illustrates the correlation between each channel of the SIEVE embedding and the radiusr=‖𝐱i−𝐱¯‖r=\\\|\\mathbf\{x\}\_\{i\}\-\\overline\{\\mathbf\{x\}\}\\\|\.The correlation analysis \(Figure[10](https://arxiv.org/html/2610.21087#A3.F10)\) demonstrates that when no noise is added to the masking process, the embeddings collapse and correlate heavily with the radiusrr\. In this configuration, 12 out of 16 channels exhibit an absolute correlation value above 0\.75\. The addition of simple noise reduces this count to 9, indicating that even basic noise helps prevent a complete dimensional collapse to the radial component\. The more sophisticated reconstruction strategies further decouple the embeddings from the radius; notably, the ‘nearest neighbor attention mask’ variant results in only 2 channels possessing an absolute correlation value above 0\.75\.

Finally, we apply each strategy’s embedding to the RNA segmentation downstream task with PointNet\+\+ to evaluate the marginal utility of each variant compared to a raw radial feature\. The results are in table[6](https://arxiv.org/html/2610.21087#A3.T6)\.

Table 6:Test accuracies\(%\) on the RNA segmentation task with different SIEVE strategiesThe SIEVE embedding pre\-trained without noise yields results nearly identical to those using the radiusrras a feature, confirming the dimensional collapse identified in our PCA analysis\. While the addition of simple noise disrupts this radial dependency, it initially results in a marginal performance decrease\. In contrast, the more sophisticated strategies yield significantly higher accuracies\. This indicates that these strategies have learned richer features beyond the simple radiusrr\.

### C\.4SIEVE Encoder Needs to be Trained

Since SIEVE uses SE\(3\)\-equivariant neural networks, even a randomly initialized SIEVE encoder provides an SE\(3\)\-invariant embedding to the downstream task model\. To isolate the contribution of the pretraining, we compare the downstream task performance of a randomly initialized SIEVE encoder against a pre\-trained one\. The setup reproduces Table[1](https://arxiv.org/html/2610.21087#S4.T1)exactly: same encoder architecture,sc​o​d​es\_\{code\}integration and hyperparameters\. The only difference is the weights loaded into the SIEVE encoder\. In both arms,sc​o​d​es\_\{code\}is passed to the downstream network exactly as the encoder emits it, with no rescaling or normalization\. The results are below\.

Table 7:Comparison of untrained SIEVE and pre\-trained SIEVE on the RNA segmentation task \(%\)Adding a randomly initialized SIEVE encoder to the downstream task does not meaningfully improve the performance\. DiffusionNet’s accuracy increases marginally \(\+1\.32\), but both DeltaNet \(\-9\.62\) and PointNet\+\+ \(\-4\.08\) suffer performance loss with a randomly initialized SIEVE encoder\. Across all three backbones, neither the equivariant architecture nor the additional input width accounts for the gain in Table[1](https://arxiv.org/html/2610.21087#S4.T1); those gains are attributable to the pretraining itself\.

### C\.5SIEVE Embedding Injection Location

The SIEVE encoder is frozen downstream, and every downstream task uses the original published hyperparameters; the only modification is widening the input layer\. The choice that does matter is where the embedding is injected\. We have done an ablation study on the geometric registration task where the ShapeNet pre\-trained SIEVE embeddings are injected at different locations\.

Table 8:sc​o​d​es\_\{code\}injection location comparison on the geometric registration taskWe have found that concatenating at the start yields the best results\. Therefore, all of the downstream tasks concatenatesc​o​d​es\_\{code\}at the downstream network input\.