MABLE: Masked Autoencoding with Bi-Lipschitz Decoding for Embeddings and Graph Metric Learning

# MABLE: Masked Autoencoding with Bi-Lipschitz Decoding for Embeddings and Graph Metric Learning
Source: [https://arxiv.org/html/2607.02990](https://arxiv.org/html/2607.02990)
Yaniv ShulmanCorrespondence:[yaniv@shulman\.info](https://arxiv.org/html/2607.02990v1/mailto:[email protected])\. This work was done while Yaniv Shulman was working at Fleet Space Technologies\.Fleet Space Technologies, Adelaide, Australia [https://www\.fleet\.space](https://www.fleet.space/)Shaghayegh AkbarpourJack B\. Muirjack\.muir@fleet\.spaceFleet Space Technologies, Adelaide, Australia [https://www\.fleet\.space](https://www.fleet.space/)Research School of Earth Sciences, Australian National University, Acton, Australia

###### Abstract

We propose MABLE \(MaskedAutoencoding withBi\-Lipschitz Decoding forEmbeddings and Graph Metric Learning\), a self\-supervised framework for learning node and graph embeddings from large, heterogeneous graphs, demonstrated here on geospatial mineral\-exploration data\. MABLE combines masked reconstruction with fixed cosine\-similarity losses that align matched augmented views while keeping unpaired embeddings well spread\. A bi\-Lipschitz feature decoder ties a low\-dimensional reconstruction component of each node embedding to feature similarity, while matched\-node consistency shapes the remaining context used by graph pooling\. Lipschitz\-controlled pooling helps stabilize graph\-level representations under perturbations of retained node embeddings, while augmentation alignment trains robustness to masking, node dropping, and sampling variation\. Across local copper and regional Arabian Shield studies, MABLE embeddings provide complementary downstream signal and produce coherent embedding\-derived layers for hypothesis generation without learned discriminators or hard\-negative selection\.

## 1Introduction

Recent advances in self\-supervised learning have fueled the development of foundation models in domains ranging from computer vision\[Heet al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib6)\]to geospatial analysis\[Darunaet al\.,[2024](https://arxiv.org/html/2607.02990#bib.bib5)\], where tasks like semantic similarity search of distant regions or mineral prospectivity mapping require robust representations of spatially distributed data\. In particular,*masked autoencoders*\(MAEs\) and*contrastive*methods have emerged as two prominent paradigms\. MAEs reconstruct missing content from partially observed inputs, thereby learning localized semantics\[Heet al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib6), Liet al\.,[2023](https://arxiv.org/html/2607.02990#bib.bib2), Zhanget al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib1), Houet al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib26),[2023](https://arxiv.org/html/2607.02990#bib.bib27)\], while contrastive approaches push away “unrelated” samples to avoid representational collapse\[van den Oordet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib7), Chenet al\.,[2020](https://arxiv.org/html/2607.02990#bib.bib8)\]\. It has recently been shown that plain MAEs can still suffer from partial dimensional collapse in feature space; thus, adding a*uniformity or repulsion term*can spread out different samples’ embeddings\[Zhanget al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib1)\]\.

In geospatial applications, data are often geographically uneven, and when absolute location is provided as an input feature, it may spuriously dominate learned representations if not properly controlled\[Wuet al\.,[2024](https://arxiv.org/html/2607.02990#bib.bib11)\]\. Our work*unifies*masked reconstruction and metric learning in a single end\-to\-end framework on*graphs*\. Here, metric learning refers to learning an embedding geometry in which distances or cosine similarities reflect useful node\- and graph\-level relationships, using self\-supervised objectives rather than supervised pair or triplet labels\. To avoid trivial geospatial shortcuts, we exclude absolute/global position signals from the input features, and learn representations from observed features, relative positions, and graph structure alone\. Specifically, we encourage each node’s latent feature representation to be accurate for reconstructing its masked features, a*positive alignment*principle in the reconstruction\-relevant component; to remain consistent across matched nodes in augmented views, a*cross\-view node\-correspondence*principle in the complementary contextual component; and to remain well spread relative to other sampled nodes in the batch, a*node uniformity*concept, as well as from entirely different graphs, a*graph uniformity*concept\. We impose a*bi\-Lipschitz*continuity constraint on the feature decoder \(acting on a low\-dimensional reconstruction component\), so that accurate reconstruction encourages proximity in that reconstruction subspace, and we employ*Lipschitz*pooling operators, notably an adaptive softmax attention pooling mechanism \(§[2\.4](https://arxiv.org/html/2607.02990#S2.SS4)\), to aggregate node embeddings into robust graph\-level representations\. These Lipschitz\-continuous aggregators ensure that small local perturbations in node embeddings produce proportionally bounded shifts in the resulting graph embedding, even though they do not in general admit a lower\-Lipschitz bound or guarantee semantic separation\.

Our framework employs normalized/cosine contrastive loss terms inspired by the Spectral Contrastive Loss \(SCL\) of\[HaoChenet al\.,[2021](https://arxiv.org/html/2607.02990#bib.bib9)\]\. Because MABLE uses cosine similarity rather than unconstrained raw inner products, the resulting fixed\-cosine surrogate removes scale ambiguity and admits a heuristic second\-order Donsker–Varadhan \(DV\)\-style interpretation under boundedness, approximate symmetry, and small\-variance assumptions \(see Appendix[A\.3](https://arxiv.org/html/2607.02990#A1.SS3)for details\)\. In place of a learned discriminator and the InfoNCE log softmax\[van den Oordet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib7)\], we use a fixed squared\-similarity surrogate, and introduce two key advancements: First, we impose spectral constraints on the decoder, bounding its singular values to enforce both upper and lower Lipschitz continuity so that small reconstruction errors encourage proximity in the reconstruction subspace\. Second, we embed the surrogate in a masked autoencoder and apply it at both node and graph scales, while also adding an explicit node\-level correspondence objective across augmented views in a contextual component of the node embedding, all without explicit hard\-negative selection, crucial for geospatial data where “negatives” can still be semantically related\.

## 2A Unified End\-to\-End Framework

Our masked\-graph pretraining pipeline addresses three challenges in one pass: \(i\) encouraging node similarity in a reconstruction\-relevant component via masked feature reconstruction and a bi\-Lipschitz feature decoder, while also encouraging cross\-view consistency of matched nodes in a complementary contextual component; \(ii\) encouraging graph uniformity by pushing graph embeddings toward a well\-spread, approximately isotropic distribution in normalized embedding space; and \(iii\) suppressing geospatial bias by dispensing with explicit hard\-negative selection and absolute/global locations\.

Conceptually, the loss is the masked\-graph counterpart of SCL, but expressed with cosine or squared\-distance penalties instead of any learned discriminator or InfoNCE term\. Combined with the spectral constraints, this fixed\-surrogate design encourages reconstruction\-driven alignment in the reconstruction component, cross\-view node consistency in the contextual component, and graph\-level alignment/uniformity without a separate substructure\-to\-graph module, while avoiding the instability of adversarial training\. We draw inspiration from mutual\-information methods such as Deep InfoMax\[Hjelmet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib12), Veličkovićet al\.,[2019](https://arxiv.org/html/2607.02990#bib.bib13)\]while remaining fully non\-adversarial, and from recent graph self\-supervised learning methods including GraphCL, GRACE, and BGRL\[Youet al\.,[2020](https://arxiv.org/html/2607.02990#bib.bib28), Zhuet al\.,[2020](https://arxiv.org/html/2607.02990#bib.bib29), Thakooret al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib30)\]\.

Throughout, we use “graph” to denote a variable\-sized collection of node observations with optional explicit edges\. The experiments in this paper instantiate the edge\-free case: no hand\-crafted adjacency matrix is supplied, and interactions are induced by the attention mechanism over node tokens \(see §[4\.1](https://arxiv.org/html/2607.02990#S4.SS1)\)\. Explicit edge\-aware variants are a natural extension when reliable connectivity is available\. The objective itself is not tied to geospatial inputs or to a particular encoder: the mineral\-exploration setting below is the experimental demonstration and validation domain\.

### 2\.1Notation and Node Feature Similarity Preservation

We begin by formalizing the encoder–decoder model and illustrating how masked reconstruction can help preserve node\-level similarity in the latent space, provided the decoder is suitably constrained\. By hiding portions of each node’s features, we encourage the model to learn robust, cross\-modal or cross\-attribute relationships, rather than simply memorizing direct inputs\.

##### Encoder–Decoder Setup\.

LetG=\(V​\(G\),E​\(G\)\)G=\(V\(G\),E\(G\)\)denote a graph with vertex setV​\(G\)V\(G\)and edge setE​\(G\)E\(G\)\. Each nodeu∈V​\(G\)u\\in V\(G\)is associated with a feature vectorxu∈ℝdxx\_\{u\}\\in\\mathbb\{R\}^\{d\_\{x\}\}, wheredxd\_\{x\}denotes the feature \(attribute\) dimension; we writeX​\(G\):=\{xu\}u∈V​\(G\)X\(G\):=\\\{x\_\{u\}\\\}\_\{u\\in V\(G\)\}for the collection of node features, and by abuse of notation also denote byX​\(G\)∈ℝ\|V​\(G\)\|×dxX\(G\)\\in\\mathbb\{R\}^\{\|V\(G\)\|\\times d\_\{x\}\}the matrix obtained by stackingxu⊤x\_\{u\}^\{\\top\}as rows in an arbitrary but fixed node ordering\. WhenGGandXXare clear from context, we writeX:=X​\(G\)X:=X\(G\)\. Herefθf\_\{\\theta\}is a graph\-level encoder \(e\.g\. a GNN\) that processes the whole graphGGtogether with an input feature collectionXX\(which may be masked\) and returns a contextualised embedding for every node; for notational simplicity, any additional auxiliary per\-node context supplied to the encoder is left implicit\. We writef​\(u\)f\(u\)as shorthand for the more explicit\(fθ​\(G,X\)\)u\(f\_\{\\theta\}\(G,X\)\)\_\{u\}whenGGandXXare clear from context\. Our model consists of an encoder that produces node embeddings of dimensionddand a feature decoder that acts on a reconstruction\-facing subspace of dimensionde≤dd\_\{e\}\\leq d:

fθ​\(G,X​\(G\)\)∈ℝ\|V​\(G\)\|×dandg:ℝde→ℝdx,f\_\{\\theta\}\(G,X\(G\)\)\\in\\mathbb\{R\}^\{\|V\(G\)\|\\times d\}\\quad\\text\{and\}\\quad g:\\mathbb\{R\}^\{d\_\{e\}\}\\;\\to\\;\\mathbb\{R\}^\{d\_\{x\}\},wherezu:=f​\(u\):=\(fθ​\(G,X​\(G\)\)\)u∈ℝdz\_\{u\}:=f\(u\):=\(f\_\{\\theta\}\(G,X\(G\)\)\)\_\{u\}\\in\\mathbb\{R\}^\{d\}denotes the*node embedding*\. When a minibatch contains multiple graphs, we distinguish*node instances*by writing

ℬnode:=\{\(G,u\):G∈batch,u∈V​\(G\)\},\\mathcal\{B\}\_\{\\mathrm\{node\}\}:=\\\{\(G,u\):\\,G\\in\\mathrm\{batch\},\\ u\\in V\(G\)\\\},and for\(G,u\)∈ℬnode\(G,u\)\\in\\mathcal\{B\}\_\{\\mathrm\{node\}\}we write

f​\(G,u\):=\(fθ​\(G,X​\(G\)\)\)u\.f\(G,u\):=\(f\_\{\\theta\}\(G,X\(G\)\)\)\_\{u\}\.Thus, the shorthandf​\(u\)f\(u\)is used only when the underlying graphGG\(and input viewXX\) are fixed by context\.

To reconstruct node features, we define a low\-dimensional*feature embedding*eu:=π​\(zu\)∈ℝdee\_\{u\}:=\\pi\(z\_\{u\}\)\\in\\mathbb\{R\}^\{d\_\{e\}\}as a coordinate sliceπ​\(z\)=z1:de\\pi\(z\)=z\_\{1:d\_\{e\}\}\. We denote the complementary*contextual slice*by

cu:=ρ​\(zu\):=zde\+1:d∈ℝd−de\.c\_\{u\}:=\\rho\(z\_\{u\}\):=z\_\{d\_\{e\}\+1:d\}\\in\\mathbb\{R\}^\{d\-d\_\{e\}\}\.The feature decoderggreconstructs node features fromeue\_\{u\}, and the autoencoder ish​\(u\):=g​\(eu\)=g​\(π​\(f​\(u\)\)\)h\(u\):=g\(e\_\{u\}\)=g\(\\pi\(f\(u\)\)\)\. In the sequel, masked reconstruction acts on the feature sliceeue\_\{u\}, while optional cross\-view node\-consistency terms may be imposed on the contextual slicecuc\_\{u\}\. This split embeds a constrained reconstruction pathway inside the broader graph encoder: the feature slice is geometrically tied to reconstructible feature similarity through the decoder, while the contextual slice remains available for view\-stable information used by pooling and downstream graph representations\. It also decouples the full node\-embedding dimension from the reconstruction dimension, allowing high\-dimensional contextual node embeddings while keeping the decoder input sized relative to the actual feature dimensiondxd\_\{x\}\.

Finally, letPϕP\_\{\\phi\}be a permutation\-invariant pooling operator mapping multisets of node embeddings to a graph embedding:

Pϕ:\(ℝd\)\|V​\(G\)\|→ℝdG,sG:=Pϕ​\(\{zu\}u∈V​\(G\)\)\.P\_\{\\phi\}:\\ \(\\mathbb\{R\}^\{d\}\)^\{\|V\(G\)\|\}\\to\\mathbb\{R\}^\{d\_\{G\}\},\\qquad s\_\{G\}:=P\_\{\\phi\}\(\\\{z\_\{u\}\\\}\_\{u\\in V\(G\)\}\)\.
In words,PϕP\_\{\\phi\}pools the unordered multiset of node embeddings into a single fixed\-dimensional*graph embedding*sGs\_\{G\}, used for graph\-level losses and downstream tasks\. Permutation invariance ensures thatsGs\_\{G\}depends only on the graph content, not on the arbitrary ordering of its nodes, consistent with the general set\-function viewpoint of Deep Sets\[Zaheeret al\.,[2017](https://arxiv.org/html/2607.02990#bib.bib31)\]\.

##### View operator for masking and augmentation\.

We model masking and augmentation as a random view operatorT∼𝒜T\\sim\\mathcal\{A\}acting on the graph–feature pair:

\(G\(a\),X\(a\),ι\(a\)\)=T​\(G,X\)\.\(G^\{\(a\)\},X^\{\(a\)\},\\iota^\{\(a\)\}\)=T\(G,X\)\.HereG\(a\)G^\{\(a\)\}andX\(a\)X^\{\(a\)\}are the augmented graph and feature collection, whileι\(a\):V​\(G\(a\)\)→V​\(G\)\\iota^\{\(a\)\}:V\(G^\{\(a\)\}\)\\to V\(G\)is an injective map recording the original node identity of each retained node\. Feature\-only masking is the special caseG\(a\)=GG^\{\(a\)\}=Gandι\(a\)=id\\iota^\{\(a\)\}=\\mathrm\{id\}; node or subgraph drops remove vertices fromG\(a\)G^\{\(a\)\}and restrictι\(a\)\\iota^\{\(a\)\}to the retained nodes\. Unless otherwise stated, when a single graphGGand input viewXXare fixed by context, we continue to writef​\(u\):=\(fθ​\(G,X\)\)uf\(u\):=\(f\_\{\\theta\}\(G,X\)\)\_\{u\}for the node embedding under the*current*inputXX\(masked or unmasked\)\. In minibatch settings involving multiple graphs, we instead writef​\(G,u\)f\(G,u\)\. For two sampled view operatorsT1,T2T\_\{1\},T\_\{2\}, we write\(G\(a\),X\(a\),ι\(a\)\)=Ta​\(G,X\)\(G^\{\(a\)\},X^\{\(a\)\},\\iota^\{\(a\)\}\)=T\_\{a\}\(G,X\)fora∈\{1,2\}a\\in\\\{1,2\\\}\. In our masked setting,hhpredicts missing or masked features from partially observed inputs, forcing the learned representation to capture meaningful dependencies within and across features\.

##### Matrix norms and singular values\.

For any matrixW∈ℝp×qW\\in\\mathbb\{R\}^\{p\\times q\}, define

σmax​\(W\):=max‖v‖2=1⁡‖W​v‖2=λmax​\(W⊤​W\),σmin​\(W\):=min‖v‖2=1⁡‖W​v‖2=λmin​\(W⊤​W\)\.\\sigma\_\{\\max\}\(W\)\\;:=\\;\\max\_\{\\\|v\\\|\_\{2\}=1\}\\\|Wv\\\|\_\{2\}\\;=\\;\\sqrt\{\\lambda\_\{\\max\}\(W^\{\\top\}W\)\},\\qquad\\sigma\_\{\\min\}\(W\)\\;:=\\;\\min\_\{\\\|v\\\|\_\{2\}=1\}\\\|Wv\\\|\_\{2\}\\;=\\;\\sqrt\{\\lambda\_\{\\min\}\(W^\{\\top\}W\)\}\.Whenp≥qp\\geq q\(tall or square\) andWWhas full column rank,σmin​\(W\)\>0\\sigma\_\{\\min\}\(W\)\>0and coincides with the smallest \(nonzero\) singular value in the usual ordering\. Whenp<qp<q\(wide\),σmin​\(W\)=0\\sigma\_\{\\min\}\(W\)=0\.

##### Cosine similarity\.

We define cosine similarity as

cos⁡\(z,z′\):=\{z⊤​z′‖z‖2​‖z′‖2,if​z≠0​and​z′≠0,0,otherwise\.\\cos\(z,z^\{\\prime\}\)\\;:=\\;\\begin\{cases\}\\dfrac\{z^\{\\top\}z^\{\\prime\}\}\{\\\|z\\\|\_\{2\}\\,\\\|z^\{\\prime\}\\\|\_\{2\}\},&\\text\{if \}z\\neq 0\\text\{ and \}z^\{\\prime\}\\neq 0,\\\\\[6\.0pt\] 0,&\\text\{otherwise\.\}\\end\{cases\}

##### Positive Alignment via Masking\.

Masked autoencoders, including graph variants\[Heet al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib6), Zhanget al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib1), Houet al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib26),[2023](https://arxiv.org/html/2607.02990#bib.bib27)\], suggest that masking can*align*nodes in the reconstruction\-relevant feature\-embedding space when local structures are similar, because reconstructing missing features or, in graph\-autoencoding settings, masked edges/structure from context often forces the model to group similar inputs together\. If two nodesuuandvvshare similar local context, features, or incident structure, then under sufficient masking,h​\(u\)h\(u\)andh​\(v\)h\(v\)must correctly fill in similar content or structure\. This typically helps their learned representationsf​\(u\)f\(u\)andf​\(v\)f\(v\)to become proximate\. In this paragraph we usef​\(u\)f\(u\)andg​\(⋅\)g\(\\cdot\)in the standard autoencoder sense \(not MABLE\-specific notation\)\.

However, if the decoderggis*unconstrained*, the reconstruction loss only requiresh​\(u\)=g​\(f​\(u\)\)h\(u\)=g\\bigl\(f\(u\)\\bigr\)to matchuuin the*input–output*space\. That alone does not strictly guarantee thatf​\(u\)≈f​\(v\)f\(u\)\\approx f\(v\)for two nodesu,vu,vwith similar original features\. In other words,ggmight learn to “invert”f​\(u\)f\(u\)in arbitrary ways without necessarily preserving pairwise distances in the latent space\. This scenario shares conceptual parallels with issues like posterior collapse in Variational Autoencoders \(VAEs\), where an overly powerful or flexible decoder can learn to ignore the latent variables, rendering them uninformative, because it can model the data distribution well without relying on structured latent codes\[Wanget al\.,[2021](https://arxiv.org/html/2607.02990#bib.bib21)\]\. To ensure that the reconstruction\-relevant latent space captured byeu=π​\(f​\(u\)\)e\_\{u\}=\\pi\(f\(u\)\)reflects meaningful similarities and is not trivialized by the decoder’s capacity, we turn to explicit constraints ongg\.

##### Bi\-Lipschitz Feature Decoding for Latent Alignment\.

To promote embedding\-level alignment, we impose a bi\-Lipschitz constraint ongg\. The next proposition shows that, whenggis bi\-Lipschitz and reconstruction is accurate, similarity in the*input*domain translates into controlled proximity in the reconstruction subspace\.

###### Proposition 1\(Node Feature Similarity Preservation\)\.

Letzu:=f​\(u\)∈ℝdz\_\{u\}:=f\(u\)\\in\\mathbb\{R\}^\{d\}denote the node embedding and leteu:=π​\(zu\)∈ℝdee\_\{u\}:=\\pi\(z\_\{u\}\)\\in\\mathbb\{R\}^\{d\_\{e\}\}denote the feature embedding\. Leth​\(u\):=g​\(eu\)h\(u\):=g\(e\_\{u\}\)be an autoencoder whose feature decoderggis*bi\-Lipschitz*; i\.e\. there exist constants0<m≤L0<m\\leq Lsuch that

m​‖e1−e2‖≤‖g​\(e1\)−g​\(e2\)‖≤L​‖e1−e2‖,∀e1,e2∈ℝde\.m\\,\\\|e\_\{1\}\-e\_\{2\}\\\|\\;\\leq\\;\\\|g\(e\_\{1\}\)\-g\(e\_\{2\}\)\\\|\\;\\leq\\;L\\,\\\|e\_\{1\}\-e\_\{2\}\\\|,\\quad\\forall\\,e\_\{1\},e\_\{2\}\\in\\mathbb\{R\}^\{d\_\{e\}\}\.Supposeh​\(u\)=g​\(eu\)h\(u\)=g\(e\_\{u\}\)reconstructs each nodeuuto withinε\\varepsilonof its features:

‖h​\(u\)−xu‖≤ε∀u\.\\\|h\(u\)\-x\_\{u\}\\\|\\leq\\varepsilon\\quad\\forall\\,u\.Then, for any nodesu,vu,v,

‖eu−ev‖≤1m​‖xu−xv‖\+2​εm\.\\\|e\_\{u\}\-e\_\{v\}\\\|\\leq\\frac\{1\}\{m\}\\,\\\|x\_\{u\}\-x\_\{v\}\\\|\+\\frac\{2\\varepsilon\}\{m\}\.Moreover,

‖eu−ev‖≥max⁡\{0,‖xu−xv‖−2​εL\}\.\\\|e\_\{u\}\-e\_\{v\}\\\|\\geq\\max\\\!\\left\\\{0,\\;\\frac\{\\\|x\_\{u\}\-x\_\{v\}\\\|\-2\\varepsilon\}\{L\}\\right\\\}\.
\(This is a pointwise accuracy assumption onhh; we specify practical training losses and masking\-dependent instantiations later\. The same inequality holds in any norm providedggsatisfies the corresponding co‑Lipschitz bound in that norm\.\)

See proof in Appendix[A\.1](https://arxiv.org/html/2607.02990#A1.SS1)\. Thus,*node feature similarity*is approximately preserved in the feature\-embedding space used for reconstruction when the decoder is bi\-Lipschitz and reconstruction error is small\. The proposition is a pointwise geometric statement, while in training we optimize an empirical/expected mask\-dependent reconstruction objective as its practical surrogate\. Masking can amplify this effect because the model must predict missing information from partial inputs, implicitly aligning nodes that fill in similar content\[Zhanget al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib1)\]\.

Scope of the guarantee\.Proposition[1](https://arxiv.org/html/2607.02990#Thmproposition1)is a statement about the reconstruction\-relevant feature sliceeu=π​\(zu\)e\_\{u\}=\\pi\(z\_\{u\}\)under a bi\-Lipschitz decoder and small reconstruction error\. It does not by itself imply that the full node embeddingzuz\_\{u\}preserves all task\-relevant similarities, nor that masking alone guarantees alignment in the full latent space\.

Practical note\.We approximate the bi\-Lipschitz property with a single linear decoderg​\(e\)=Wd​e​c​e\+bg\(e\)=W\_\{dec\}e\+bwhose singular values are kept in a fixed bandm≤σi​\(Wd​e​c\)≤Lm\\\!\\leq\\\!\\sigma\_\{i\}\(W\_\{dec\}\)\\\!\\leq\\\!L\. Implementation notes and computational details are discussed in §[2\.5](https://arxiv.org/html/2607.02990#S2.SS5)\. For a linear map, achieving a non\-trivial lower boundm\>0m\>0requiresσmin​\(Wd​e​c\)\>0\\sigma\_\{\\min\}\(W\_\{dec\}\)\>0, i\.e\.Wd​e​cW\_\{dec\}has full column rank \(and thusde≤dxd\_\{e\}\\leq d\_\{x\}in the Euclidean setting\)\. Ifde\>dxd\_\{e\}\>d\_\{x\}, thenσmin​\(Wd​e​c\)=0\\sigma\_\{\\min\}\(W\_\{dec\}\)=0and no nontrivial lower bound exists\.

### 2\.2Fixed\-Cosine Surrogates

InfoMax–style objectives learn representations by contrasting the joint distributionppos​\(z,z\+\)p\_\{\\text\{pos\}\}\(z,z^\{\+\}\), withz=f​\(x\)z=f\(x\)andz\+=f​\(x\+\)z^\{\+\}=f\(x^\{\+\}\)an augmented view, against the product distributionp​\(z\)2p\(z\)^\{2\}of unpaired embeddings\[Hjelmet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib12), Veličkovićet al\.,[2019](https://arxiv.org/html/2607.02990#bib.bib13), van den Oordet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib7), Heet al\.,[2019](https://arxiv.org/html/2607.02990#bib.bib15)\]\. Positive pairs\(z,z\+\)\(z,z^\{\+\}\)are encouraged to score higher than uncorrelated pairs\(z,z−\)\(z,z^\{\-\}\)treated as approximately independent negatives \(e\.g\., in\-batch\), typically by maximising a mutual‑information lower bound such as InfoNCE, a Jensen–Shannon variant, or the binary cross‑entropy \(BCE\) objective\[Gutmann and Hyvärinen,[2010](https://arxiv.org/html/2607.02990#bib.bib14)\]:

ℒBCE=𝔼\(z,z\+\)​\[−log⁡σ​\(D​\(z,z\+\)\)\]\+𝔼\(z,z−\)​\[−log⁡\(1−σ​\(D​\(z,z−\)\)\)\],\\mathcal\{L\}\_\{\\text\{BCE\}\}=\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\\bigl\[\-\\log\\sigma\\bigl\(D\(z,z^\{\+\}\)\\bigr\)\\bigr\]\+\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\\bigl\[\-\\log\\bigl\(1\-\\sigma\\bigl\(D\(z,z^\{\-\}\)\\bigr\)\\bigr\)\\bigr\],whereσ​\(u\)=1/\(1\+e−u\)\\sigma\(u\)=1/\(1\+e^\{\-u\}\)andDDis a learned discriminator\. Besides introducing additional parameters, this formulation assumes that in\-batch negatives are semantically unrelated, an assumption that often fails for real\-world geospatial corpora, where nearby coordinates typically share land\-cover, climate, or geology\. In practice, when absolute coordinates are available to the model, a powerful discriminator may latch onto absolute location: the simplest way to separate a random negative from its positive is often just coordinate distance, not a meaningful semantic cue\. In our setting, we avoid this shortcut at the source by*not*feeding absolute/global positions as input features\. We still prefer fixed\-similarity surrogates over learned discriminators to reduce the risk of other trivial separation cues dominating training dynamics in geographically uneven corpora\.

To address these limitations and avoid the learned discriminator, we use fixed cosine\-similarity objectives\. The raw inner\-product form of the Spectral Contrastive Loss \(SCL\) ofHaoChenet al\.\[[2021](https://arxiv.org/html/2607.02990#bib.bib9)\]already expresses the desired pairwise geometry:

ℒSCL,raw=−2​𝔼\(z,z\+\)​\[z⊤​z\+\]\+𝔼\(z,z−\)​\[\(z⊤​z−\)2\]\.\\mathcal\{L\}\_\{\\text\{SCL,raw\}\}=\-2\\,\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\\bigl\[z^\{\\top\}z^\{\+\}\\bigr\]\+\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\\bigl\[\(z^\{\\top\}z^\{\-\}\)^\{2\}\\bigr\]\.The positive term rewards large positive\-pair inner products, while the quadratic negative term penalizes nonzero inner products for unpaired embeddings\. The original population objective is written in terms of raw inner products, but the empirical setting inHaoChenet al\.\[[2021](https://arxiv.org/html/2607.02990#bib.bib9)\]still uses norm control, including projection to a bounded ball with experimentally chosen radius, and the finite\-sample analysis assumes norm\-controlled function classes\. Without normalization, however, this objective mixes angular structure with vector scale: after angular alignment saturates, the positive term can still be improved by increasing norms, and the negative term can be reduced either by orthogonality or by shrinking norms\.

In MABLE we remove this scale ambiguity by working directly with cosine similarity,cos⁡\(z,z′\)∈\[−1,1\]\\cos\(z,z^\{\\prime\}\)\\in\[\-1,1\]\. For unit\-normalized embeddings,cos⁡\(z,z′\)=z⊤​z′\\cos\(z,z^\{\\prime\}\)=z^\{\\top\}z^\{\\prime\}, and SCL becomes the normalized/cosine objective

ℒSCL=−2𝔼\(z,z\+\)\[cos\(z,z\+\)\]\+𝔼\(z,z−\)\[cos\(z,z−\)2\]\.\\mathcal\{L\}\_\{\\text\{SCL\}\}=\-2\\,\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\\bigl\[\\cos\(z,z^\{\+\}\)\\bigr\]\+\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\\bigl\[\\cos\(z,z^\{\-\}\)^\{2\}\\bigr\]\.This cleanly targets angular alignment of positive pairs \(cos⁡\(z,z\+\)→1\\cos\(z,z^\{\+\}\)\\to 1\) and angular decorrelation of unpaired embeddings \(cos⁡\(z,z−\)→0\\cos\(z,z^\{\-\}\)\\to 0\)\.

The same geometry can be motivated from a fixed log\-based surrogate\. For positive pairs, choose an argument whose negative log is minimized ascos⁡\(z,z\+\)→1\\cos\(z,z^\{\+\}\)\\to 1, namelyexp⁡\(cos⁡\(z,z\+\)−1\)\\exp\(\\cos\(z,z^\{\+\}\)\-1\)\. For unpaired embeddings, choose an argument whose negative log is minimized ascos⁡\(z,z−\)→0\\cos\(z,z^\{\-\}\)\\to 0, namely1−12cos\(z,z−\)21\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\. This gives

ℒSurrogate=−𝔼\(z,z\+\)\[log\(exp\(cos\(z,z\+\)−1\)\)\]−𝔼\(z,z−\)\[log\(1−12cos\(z,z−\)2\)\]\.\\mathcal\{L\}\_\{\\text\{Surrogate\}\}=\-\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\[\\log\(\\exp\(\\cos\(z,z^\{\+\}\)\-1\)\)\]\-\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\[\\log\(1\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\)\]\.The positive term simplifies, up to an additive constant, to−cos⁡\(z,z\+\)\-\\cos\(z,z^\{\+\}\)\. The negative term has the Taylor expansion

−log\(1−12cos\(z,z−\)2\)=12cos\(z,z−\)2\+O\(cos\(z,z−\)4\)\-\\log\\\!\\left\(1\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\\right\)=\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\+O\(\\cos\(z,z^\{\-\}\)^\{4\}\)near the desired negative optimumcos⁡\(z,z−\)=0\\cos\(z,z^\{\-\}\)=0\.

Thus the local quadratic approximation to the log\-based surrogate is the scaled normalized SCL objective\. LetℒSSCL=12​ℒSCL\\mathcal\{L\}\_\{\\text\{SSCL\}\}=\\tfrac\{1\}\{2\}\\,\\mathcal\{L\}\_\{\\text\{SCL\}\}:

ℒSSCL=−𝔼\(z,z\+\)\[cos\(z,z\+\)\]\+12𝔼\(z,z−\)\[cos\(z,z−\)2\]\.\\mathcal\{L\}\_\{\\text\{SSCL\}\}=\-\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\\bigl\[\\cos\(z,z^\{\+\}\)\\bigr\]\+\\tfrac\{1\}\{2\}\\,\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\\bigl\[\\cos\(z,z^\{\-\}\)^\{2\}\\bigr\]\.This is the fixed\-cosine geometry used throughout MABLE: the first term aligns true pairs, while the quadratic penalty pushes unpaired embeddings toward orthogonality, without requiring a learned discriminator or an InfoNCE softmax\.

##### Alignment, Uniformity, and Downstream Separability\.

Empirically, achieving strong intra\-class “alignment” \(similar samples are close in the latent space\) and cross\-class “uniformity” \(dissimilar samples are spread out across the latent space, often on the hypersphere\) often correlates with improved linear separability on downstream tasks \(see, e\.g\.,\[Toshet al\.,[2020](https://arxiv.org/html/2607.02990#bib.bib10)\]; under additional multi\-view redundancy assumptions, their Theorem 3 shows that linear functions of suitable embeddings can approximate the Bayes\-optimal predictor\)\. Uniformity on the hypersphere is particularly valuable in a fixed\-cosine similarity space because random embeddings then have a near\-zero expected dot product\. This near\-zero baseline for random dissimilar pairs provides a natural separation from concentrated positive pairs that exhibit high similarity\. Here, we enforce alignment and uniformity via fixed similarity\-based objectives at the graph level \(SCL\-style alignment/uniformity\), while at the node level alignment is induced by reconstruction in the feature\-embedding subspace, cross\-view consistency is encouraged in the complementary contextual slice via matched\-node correspondence across augmentations, and uniformity is encouraged by a squared\-cosine repulsion term\. This design shifts the representational burden primarily onto the encoder, which must learn node embeddings whose reconstruction\-relevant slice preserves local feature similarity and whose contextual slice remains stable across views while still spreading unrelated samples apart, all without requiring a learned discriminator or the InfoNCE objective itself\. In practice, the resulting representation spaces \(in particular the graph embeddings, the feature\-embedding component used for reconstruction, and the contextual component used for cross\-view consistency\) often become easier to separate with simple downstream models, so a linear classifier or even akk\-NN rule can perform competitively\[Wang and Isola,[2020](https://arxiv.org/html/2607.02990#bib.bib16)\]\. Recent refinements like*MIM\-Refiner*\[Alkinet al\.,[2025](https://arxiv.org/html/2607.02990#bib.bib4)\]further demonstrate that intermediate\-layers’ representations can be re\-trained with an instance discrimination head to improve cluster coherence and alignment, thereby making the final embedding layers more suitable for low\-shot classification and other downstream tasks\.

### 2\.3From Node Embeddings to Graph Embeddings via Lipschitz Pooling

After obtaining node embeddings\{zu\}u∈V​\(G\)\\\{z\_\{u\}\\\}\_\{u\\in V\(G\)\}, many applications require distilling them into a single*graph\-level*representation\. In this subsection, we discuss how to preserve alignment in the embedding space by employing a*Lipschitz*pooling function\.

##### Generic Lipschitz Pooling\.

Fix a norm∥⋅∥\\\|\\cdot\\\|onℝd\\mathbb\{R\}^\{d\}and equip\(ℝd\)n\(\\mathbb\{R\}^\{d\}\)^\{n\}with the product metric‖\(zi\)−\(wi\)‖sum:=∑i=1n‖zi−wi‖\\\|\(z\_\{i\}\)\-\(w\_\{i\}\)\\\|\_\{\\mathrm\{sum\}\}:=\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-w\_\{i\}\\\|\. We say a functionP:\(ℝd\)n→ℝdGP:\(\\mathbb\{R\}^\{d\}\)^\{n\}\\to\\mathbb\{R\}^\{d\_\{G\}\}is*Lipschitz*if there exists a constantL≥0L\\geq 0such that for all ordered collections\(zi\)i=1n,\(wi\)i=1n\(z\_\{i\}\)\_\{i=1\}^\{n\},\(w\_\{i\}\)\_\{i=1\}^\{n\},

‖P​\(z1,…,zn\)−P​\(w1,…,wn\)‖≤L​‖\(zi\)−\(wi\)‖sum\.\\\|P\(z\_\{1\},\\dots,z\_\{n\}\)\-P\(w\_\{1\},\\dots,w\_\{n\}\)\\\|\\;\\leq\\;L\\,\\\|\(z\_\{i\}\)\-\(w\_\{i\}\)\\\|\_\{\\mathrm\{sum\}\}\.In the context of pooling, we view a graph as a*multiset*of node embeddingsZ=\{zi\}i=1n⊂ℝdZ=\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\}\\subset\\mathbb\{R\}^\{d\}, where node order is arbitrary\.

Remark\.In this section,PPdenotes a generic pooling map\. Our model uses a \(possibly learnable\) pooling operatorPϕP\_\{\\phi\}as defined in §[2\.1](https://arxiv.org/html/2607.02990#S2.SS1), and all results apply toPϕP\_\{\\phi\}whenever it satisfies the stated assumptions \(permutation invariance and Lipschitz continuity\)\.

##### Permutation invariance and matching distance\.

In our setting, the pooling operator acts on the \(orderless\) multiset of node embeddings produced by the encoder, so we assumePPis*permutation\-invariant*:P​\(z1,…,zn\)=P​\(zξ​\(1\),…,zξ​\(n\)\)P\(z\_\{1\},\\dots,z\_\{n\}\)=P\(z\_\{\\xi\(1\)\},\\dots,z\_\{\\xi\(n\)\}\)for any permutationξ∈Sn\\xi\\in S\_\{n\}\. Accordingly, whenPPis permutation\-invariant we writeP​\(Z\)P\(Z\)to denote the value ofPPon any ordering of the multisetZZ\. To compare two graphs without a predefined node correspondence, we use the optimal matching distance

dmatch​\(Z,Z′\):=minξ∈Sn​∑i=1n‖zi−zξ​\(i\)′‖,Z=\{zi\}i=1n,Z′=\{zi′\}i=1n\.d\_\{\\mathrm\{match\}\}\(Z,Z^\{\\prime\}\)\\;:=\\;\\min\_\{\\xi\\in S\_\{n\}\}\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{\\xi\(i\)\}\\\|,\\qquad Z=\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\},\\;Z^\{\\prime\}=\\\{z^\{\\prime\}\_\{i\}\\\}\_\{i=1\}^\{n\}\.\(When graph sizes differ, as happens under node\-dropping or subgraph augmentations, one can pad the smaller set with a fixed null embedding or use a partial\-matching variant; in the main text we use Proposition[2](https://arxiv.org/html/2607.02990#Thmproposition2)as an idealized equal\-cardinality stability result rather than as a full theorem for every augmentation regime\.\)

###### Proposition 2\(Lipschitz Pooling is Stable Under Optimal Matching\)\.

LetP:\(ℝd\)n→ℝdGP:\(\\mathbb\{R\}^\{d\}\)^\{n\}\\to\\mathbb\{R\}^\{d\_\{G\}\}be permutation\-invariant and Lipschitz with constantLLwith respect to∥⋅∥sum\\\|\\cdot\\\|\_\{\\mathrm\{sum\}\}\. Then for any two multisets of node embeddingsZ=\{zi\}i=1nZ=\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\}andZ′=\{zi′\}i=1nZ^\{\\prime\}=\\\{z^\{\\prime\}\_\{i\}\\\}\_\{i=1\}^\{n\},

‖P​\(Z\)−P​\(Z′\)‖≤L​dmatch​\(Z,Z′\)\.\\bigl\\\|P\(Z\)\-P\(Z^\{\\prime\}\)\\bigr\\\|\\;\\leq\\;L\\,d\_\{\\mathrm\{match\}\}\(Z,Z^\{\\prime\}\)\.Consequently, for any chosen correspondence, if the matched node embeddings differ only slightly, then the pooled graph embeddings differ proportionally\. In applications, correspondences may be inherited from node identity, augmentation bookkeeping, or coordinate\-based matching, but the guarantee is in the matched embedding space rather than in coordinate space itself\.

The proof is given in Appendix[A\.4](https://arxiv.org/html/2607.02990#A1.SS4)\. We emphasize that Proposition[2](https://arxiv.org/html/2607.02990#Thmproposition2)is stated for equal\-cardinality multisets; when augmentations change graph size, we use it as the natural equal\-cardinality stability result, while practical implementations may instead rely on padding or partial\-matching variants as noted above\.

##### Examples of Lipschitz Pooling: Sum, Mean, and Max\.

Many standard pooling operators, such as sum, mean, and coordinate‐wise max, are Lipschitz transformations\. Informally, each uses a linear or piecewise‐linear aggregation over the embeddings, ensuring bounded output changes when inputs are perturbed slightly\. Moreover, since these operators are permutation\-invariant, their Lipschitz bounds on ordered tuples immediately imply stability for unordered multisets under the optimal matching distancedmatchd\_\{\\mathrm\{match\}\}via Proposition[2](https://arxiv.org/html/2607.02990#Thmproposition2)\(withdmatchd\_\{\\mathrm\{match\}\}defined using the same underlying norm as in the input metric\)\.

###### Proposition 3\(Sum, Mean, and Max Pooling are Lipschitz\)\.

Consider the following three operators on node embeddings\{zi\}i=1n⊂ℝd\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\}\\subset\\mathbb\{R\}^\{d\}:

SumPool​\(\{zi\}\)\\displaystyle\\text\{SumPool\}\(\\\{z\_\{i\}\\\}\)\\;=∑i=1nzi,\\displaystyle=\\;\\sum\_\{i=1\}^\{n\}z\_\{i\},MeanPool​\(\{zi\}\)\\displaystyle\\text\{MeanPool\}\(\\\{z\_\{i\}\\\}\)\\;=1n​∑i=1nzi,\\displaystyle=\\;\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}z\_\{i\},MaxPool​\(\{zi\}\)k\\displaystyle\\text\{MaxPool\}\(\\\{z\_\{i\}\\\}\)\_\{k\}\\;=max1≤i≤n\(zi\)kfor each coordinatek\.\\displaystyle=\\;\\max\_\{1\\leq i\\leq n\}\(z\_\{i\}\)\_\{k\}\\quad\\text\{for each coordinate \}k\.Then each of these is Lipschitz, with constants that depend onnnand the norm\. Specifically:

- •With output normℓ2\\ell\_\{2\}and input metric∑i‖zi−zi′‖2\\sum\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{2\}, SumPool has Lipschitz constant at most11\.
- •With output normℓ2\\ell\_\{2\}and input metric∑i‖zi−zi′‖2\\sum\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{2\}, MeanPool has Lipschitz constant at most1/n1/n\.
- •With output normℓ∞\\ell\_\{\\infty\}and input metric∑i=1n‖zi−zi′‖2\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{2\}, MaxPool has Lipschitz constant at most11\(sincemaxi⁡‖zi−zi′‖∞≤∑i‖zi−zi′‖2\\max\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\infty\}\\leq\\sum\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{2\}\)\. If one instead measures the output inℓ2\\ell\_\{2\}, the constant is at mostd\\sqrt\{d\}\.

The detailed proof is given in Appendix[A\.5](https://arxiv.org/html/2607.02990#A1.SS5)\. The essence is a triangle‐inequality argument\. While these operators provide basic Lipschitz aggregation, more expressive mechanisms like Softmax Attention Pooling \(detailed in §[2\.4](https://arxiv.org/html/2607.02990#S2.SS4)\) can offer enhanced representational power while maintaining controlled Lipschitz properties\.

### 2\.4Adaptive Graph Pooling with Softmax Attention

While the pooling operators discussed in §[2\.3](https://arxiv.org/html/2607.02990#S2.SS3)\(sum, mean, max\) offer simplicity and clear Lipschitz bounds, more adaptive mechanisms can provide enhanced flexibility and empirical performance in generating graph\-level embeddings, in the spirit of learned set/graph aggregation methods such as Set Transformer and Graph Multiset Transformer pooling\[Leeet al\.,[2019](https://arxiv.org/html/2607.02990#bib.bib32), Baeket al\.,[2021](https://arxiv.org/html/2607.02990#bib.bib33)\]\. This is particularly relevant for graphs with heterogeneous node densities or complex relational structures where uniform weighting of nodes may be suboptimal\. Motivated by these learned aggregation methods, we use Softmax Attention Pooling as a learnable, context\-aware aggregation mechanism:

Pattn​\(\{zi\}i=1n\)=Wp​o​o​l​\(∑i=1nαi​\(\{zk\}k=1n\)​zi\),P\_\{\\mathrm\{attn\}\}\(\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\}\)=W\_\{pool\}\\left\(\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z\_\{i\}\\right\),where\{zi\}\\\{z\_\{i\}\\\}are the node embeddings,Wp​o​o​l∈ℝdG×dW\_\{pool\}\\in\\mathbb\{R\}^\{d\_\{G\}\\times d\}is a trainable projection matrix, andαi≥0\\alpha\_\{i\}\\geq 0\(with∑iαi=1\\sum\_\{i\}\\alpha\_\{i\}=1\) are attention weights\. These weightsαi\\alpha\_\{i\}dynamically depend on the full set of node embeddings\{zk\}k=1n\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}, often computed via a softmax over scores derived from small neural networks \(e\.g\., MLPs\) applied to node embeddings\.

ForPattnP\_\{\\mathrm\{attn\}\}to serve as a robust Lipschitz pooling operator within the MABLE framework, we state the guarantee on bounded subsets of the node\-embedding space rather than globally on all of\(ℝd\)n\(\\mathbb\{R\}^\{d\}\)^\{n\}\. In our implementation, the attention\-score network is fed unit\-normalized inputs

z¯i:=zimax⁡\(‖zi‖2,ε\),\\bar\{z\}\_\{i\}:=\\frac\{z\_\{i\}\}\{\\max\(\\\|z\_\{i\}\\\|\_\{2\},\\varepsilon\)\},its linear maps are constrained only from above via spectral\-norm clamping, and the pre\-softmax logit gaps are clipped in a shift\-invariant way:

s~i:=ψ​\(z¯i\),si:=clip​\(s~i−maxj⁡s~j,−c,0\),αi:=softmax​\(s\)i\.\\tilde\{s\}\_\{i\}:=\\psi\(\\bar\{z\}\_\{i\}\),\\qquad s\_\{i\}:=\\mathrm\{clip\}\\\!\\bigl\(\\tilde\{s\}\_\{i\}\-\\max\_\{j\}\\tilde\{s\}\_\{j\},\\,\-c,\\,0\\bigr\),\\qquad\\alpha\_\{i\}:=\\mathrm\{softmax\}\(s\)\_\{i\}\.Assume additionally that the pooled values lie in a bounded region,‖zi‖2≤B\\\|z\_\{i\}\\\|\_\{2\}\\leq Bfor alliiin the domain of interest, that the normalization uses a fixedε\>0\\varepsilon\>0, and that the score networkψ\\psiis composed of standard Lipschitz activations together with linear maps whose operator norms are clamped from above\. Because the score network acts on normalized inputs, its Lipschitz constant is controlled by the product of the clamped operator norms of its linear layers, while clipping places the logits in a compact set on which softmax is Lipschitz\. Under these conditions, the attention aggregation map

A​\(\{zk\}k=1n\):=∑i=1nαi​\(\{zk\}k=1n\)​ziA\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\;:=\\;\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z\_\{i\}admits a Lipschitz constant on the bounded domain of interest, with respect to the chosen product metric on\(ℝd\)n\(\\mathbb\{R\}^\{d\}\)^\{n\}\(e\.g\.,∥⋅∥sum\\\|\\cdot\\\|\_\{\\mathrm\{sum\}\}\), depending on the value boundBB, the logit\-gap boundcc, the set sizenn, and the score networkψ\\psi\. Consequently, if the final linear projection satisfiesσmax​\(Wp​o​o​l\)≤LW\\sigma\_\{\\max\}\(W\_\{pool\}\)\\leq L\_\{W\}, then the composed pooling operatorPattn:=Wp​o​o​l∘AP\_\{\\mathrm\{attn\}\}:=W\_\{pool\}\\circ Ais also Lipschitz on that bounded domain\. Note that because we do*not*normalize the pooled valuesziz\_\{i\}themselves, this is a bounded\-domain \(local\) Lipschitz statement, not a global one over unbounded node embeddings\.

###### Lemma 1\(Bounded\-domain Lipschitzness of clipped softmax attention\)\.

Fixnnand consider the bounded domain𝒟B=\{\(z1,…,zn\):‖zi‖2≤B\}\\mathcal\{D\}\_\{B\}=\\\{\(z\_\{1\},\\dots,z\_\{n\}\):\\\|z\_\{i\}\\\|\_\{2\}\\leq B\\\}\. Suppose the normalization usesε\>0\\varepsilon\>0, the score mapψ\\psiisLψL\_\{\\psi\}\-Lipschitz, logits are shifted by their maximum and clipped to\[−c,0\]\[\-c,0\]before softmax, and attention weights are used to form

A​\(Z\)=∑i=1nαi​\(Z\)​zi\.A\(Z\)=\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(Z\)z\_\{i\}\.ThenAAis Lipschitz on𝒟B\\mathcal\{D\}\_\{B\}with respect to the product metric∑i‖zi−zi′‖2\\sum\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{2\}\. Consequently, ifσmax​\(Wp​o​o​l\)≤LW\\sigma\_\{\\max\}\(W\_\{pool\}\)\\leq L\_\{W\}, thenPattn=Wp​o​o​l∘AP\_\{\\mathrm\{attn\}\}=W\_\{pool\}\\circ Ais also Lipschitz on𝒟B\\mathcal\{D\}\_\{B\}\.

The proof is a direct composition argument and is given in Appendix[A\.6](https://arxiv.org/html/2607.02990#A1.SS6)\.

Beyond this upper\-Lipschitz/stability guarantee, one may additionally ask whether the pooling head preserves some pairwise separation\. In our implementation, the*final pooling projection*Wp​o​o​lW\_\{pool\}is explicitly conditioned by clamping its singular values to a band\[m,L\]\[m,L\]in the regime where the projection is square or full\-column\-rank, while the*score\-network*matrices used to produce the attention logits are constrained only from above\. Accordingly, the corollary below isolates a direct consequence relevant in our setting: once the attention\-weighted sum has achieved a non\-zero separation for a given pair, the lower singular\-value bound onWp​o​o​lW\_\{pool\}helps ensure that this separation is not collapsed by the final projection, though this does not by itself yield a general lower bound with respect to a standard set metric\.

###### Corollary 1\(Conditioned Projection Preserves Attention\-Induced Separation\)\.

LetPattn​\(\{zi\}\)=W​\(∑i=1nαi​\(\{zk\}k=1n\)​zi\)P\_\{\\mathrm\{attn\}\}\(\\\{z\_\{i\}\\\}\)=W\\\!\\bigl\(\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z\_\{i\}\\bigr\)define the pooling operation, whereαi​\(⋅\)≥0\\alpha\_\{i\}\(\\cdot\)\\geq 0are attention weights satisfying∑i=1nαi​\(\{zk\}k=1n\)=1\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)=1, andW∈ℝdG×dW\\in\\mathbb\{R\}^\{d\_\{G\}\\times d\}withσmin​\(W\)≥m\>0\\sigma\_\{\\min\}\(W\)\\geq m\>0\(hence co\-Lipschitz\)\. Consider two distinct sets of embeddings\{zi\}\\\{z\_\{i\}\\\}and\{zi′\}\\\{z^\{\\prime\}\_\{i\}\\\}from the data distribution\. Letδpair\\delta\_\{\\text\{pair\}\}denote the actual separation between their attention\-weighted sums produced by the mechanismαi​\(⋅\)\\alpha\_\{i\}\(\\cdot\):

δpair=‖∑i=1nαi​\(\{zk\}k=1n\)​zi−∑i=1nαi​\(\{zk′\}k=1n\)​zi′‖\.\\delta\_\{\\text\{pair\}\}=\\Bigl\\\|\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z\_\{i\}\\;\-\\;\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z^\{\\prime\}\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z^\{\\prime\}\_\{i\}\\Bigr\\\|\.If this achieved separation is positive \(δpair\>0\\delta\_\{\\text\{pair\}\}\>0\), then the pooled embeddings for these specific sets satisfy

‖Pattn​\(\{zi\}\)−Pattn​\(\{zi′\}\)‖≥m⋅δpair\.\\bigl\\\|P\_\{\\mathrm\{attn\}\}\(\\\{z\_\{i\}\\\}\)\-P\_\{\\mathrm\{attn\}\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\bigr\\\|\\;\\;\\geq\\;\\;m\\cdot\\delta\_\{\\text\{pair\}\}\.HencePattnP\_\{\\mathrm\{attn\}\}preserves the separation induced by the attention mechanism for this pair\{zi\},\{zi′\}\\\{z\_\{i\}\\\},\\\{z^\{\\prime\}\_\{i\}\\\}up to the factormm, provided the attention mechanism yields a non\-zero separationδpair\\delta\_\{\\text\{pair\}\}\.

This does not provide a uniform lower bound in terms of a standard set metric; it states only that any separation realized by the attention\-weighted sum is not collapsed by the lower\-bounded final projection\. In our implementation this lower\-bound condition is enforced onWp​o​o​lW\_\{pool\}, whereas the attention*scoring*network itself is controlled only through input normalization, upper spectral\-norm constraints, and logit clipping \(see Appendix[A\.7](https://arxiv.org/html/2607.02990#A1.SS7)\)\.

### 2\.5Condition Number Control for Linear Decoders

While a general Lipschitz or bi\-Lipschitz decoder can be implemented via architectural constraints, which often introduce complexity or complicate training\[Miyatoet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib17), Cisseet al\.,[2017](https://arxiv.org/html/2607.02990#bib.bib18), Behrmannet al\.,[2019](https://arxiv.org/html/2607.02990#bib.bib19), Gulrajaniet al\.,[2017](https://arxiv.org/html/2607.02990#bib.bib20)\], a particularly simple yet powerful approach uses a single linear decoding layer,g​\(e\)=Wd​e​c​e\+bg\(e\)=W\_\{dec\}e\+b, whereWd​e​c∈ℝdx×deW\_\{dec\}\\in\\mathbb\{R\}^\{d\_\{x\}\\times d\_\{e\}\}\. For such a linear map from feature\-embedding space \(ℝde\\mathbb\{R\}^\{d\_\{e\}\}\) to feature space \(ℝdx\\mathbb\{R\}^\{d\_\{x\}\}\), the maximum singular valueσmax​\(Wd​e​c\)=‖Wd​e​c‖2\\sigma\_\{\\max\}\(W\_\{dec\}\)=\\\|W\_\{dec\}\\\|\_\{2\}dictates its Lipschitz constant \(bounding expansion\), and the minimum singular valueσmin​\(Wd​e​c\)\\sigma\_\{\\min\}\(W\_\{dec\}\)determines its co\-Lipschitz constant \(bounding collapse\)\. Specifically, a non\-zeroσmin​\(Wd​e​c\)\>0\\sigma\_\{\\min\}\(W\_\{dec\}\)\>0implies injectivity for the linear map \(which in particular requiresde≤dxd\_\{e\}\\leq d\_\{x\}\), preventing distinct latent vectors from collapsing to the same output\. By ensuring both0<m≤σmin​\(Wd​e​c\)0<m\\leq\\sigma\_\{\\min\}\(W\_\{dec\}\)andσmax​\(Wd​e​c\)≤L<∞\\sigma\_\{\\max\}\(W\_\{dec\}\)\\leq L<\\infty\(which in particular requiresde≤dxd\_\{e\}\\leq d\_\{x\}\), the linear decoder becomes bi\-Lipschitz, guaranteeing that distances are preserved up to bounded factors:m​‖e1−e2‖≤‖g​\(e1\)−g​\(e2\)‖≤L​‖e1−e2‖m\\\|e\_\{1\}\-e\_\{2\}\\\|\\leq\\\|g\(e\_\{1\}\)\-g\(e\_\{2\}\)\\\|\\leq L\\\|e\_\{1\}\-e\_\{2\}\\\|\.

Controlling these singular values in practice adds computational cost\. Calculating the full Singular Value Decomposition \(SVD\) of thedx×ded\_\{x\}\\times d\_\{e\}matrixWd​e​cW\_\{dec\}is typicallyO​\(min⁡\(dx,de\)⋅dx​de\)O\(\\min\(d\_\{x\},d\_\{e\}\)\\cdot d\_\{x\}d\_\{e\}\), which can be expensive, especially if performed at every training step for large matrices\. However, practical methods exist to approximate or constrain singular values without full SVD\. Controllingσmax​\(Wd​e​c\)\\sigma\_\{\\max\}\(W\_\{dec\}\), the spectral norm, is relatively efficient; techniques like the Power Method are used in Spectral Normalization\[Miyatoet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib17)\]to approximate this value iteratively\. Controllingσmin​\(Wd​e​c\)\\sigma\_\{\\min\}\(W\_\{dec\}\)or the condition numberκ​\(Wd​e​c\)=σmax​\(Wd​e​c\)/σmin​\(Wd​e​c\)\\kappa\(W\_\{dec\}\)=\\sigma\_\{\\max\}\(W\_\{dec\}\)/\\sigma\_\{\\min\}\(W\_\{dec\}\)is generally more challenging\. Nevertheless, for decoder matrices, whose dimensionsdxd\_\{x\}andded\_\{e\}are often manageable \(hundreds or low thousands\) compared to other parts of a model, the computational cost may be acceptable or can be mitigated by techniques like singular value clamping or projection\-based updates\. This approach ensures that the primary representational learning burden resides with the encoderf​\(⋅\)f\(\\cdot\), while the well\-conditioned linear decoderWd​e​cW\_\{dec\}serves as a predictable mapping that prevents trivial inversions or representational collapse in the output layer\.

### 2\.6Hypothesis: Output Constraints and Encoder Regularity

Explicitly imposing Lipschitz or bi\-Lipschitz constraints layer\-by\-layer throughout a deep neural network is computationally expensive, often requiring per\-layer spectral norm calculations\[Miyatoet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib17), Cisseet al\.,[2017](https://arxiv.org/html/2607.02990#bib.bib18)\]\. We treat output\-side conditioning as a practical design hypothesis: constraining the feature decoder \(§[2\.5](https://arxiv.org/html/2607.02990#S2.SS5)\) and using a Lipschitz\-controlled graph pooling operation \(§[2\.3](https://arxiv.org/html/2607.02990#S2.SS3), §[2\.4](https://arxiv.org/html/2607.02990#S2.SS4)\) may encourage more stable encoder outputs without requiring layer\-wise constraints throughout the network\.

This is not a formal guarantee on the encoder\. Rather, it motivates the architecture: bounded decoder and pooling stages make the final representation\-to\-loss map less sensitive, while leaving the encoder expressive enough to learn the data\-dependent interactions required by the task\. Directly measuring how such output\-side constraints affect encoder spectra and optimization dynamics is left for future work\.

### 2\.7Discussion and Practical Considerations

The choice of pooling operator itself presents several options, each with distinct characteristics\. Foundational Lipschitz operators such as mean, sum, and max pooling \(detailed in §[2\.3](https://arxiv.org/html/2607.02990#S2.SS3)\) offer a starting point\. Empirically, mean pooling is simple and stable, and it avoids the scaling issues of sum pooling when\|V​\(G\)\|\|V\(G\)\|varies widely\. Sum pooling precisely captures node multiplicities \(beneficial if the number of nodes is meaningful\), but can become unbounded with large graphs\. Max pooling discards duplicates, effectively capturing the “set” of embeddings\. Each of these simpler methods has advantages depending on how one wishes to represent the entire graph and whether exact cardinality or distributional cues are prioritized\. See\[Xuet al\.,[2019](https://arxiv.org/html/2607.02990#bib.bib3)\]for a deeper discussion comparing sum, mean, and max aggregator behaviors in Graph Neural Networks\. In MABLE, reconstruction constrains the low\-dimensional feature embeddingeu=π​\(zu\)e\_\{u\}=\\pi\(z\_\{u\}\), while pooling operates on the full node embeddingzuz\_\{u\}; the complementary contextual slice can therefore contribute to graph\-level representations while also being regularized by cross\-view matched\-node correspondence\.

In summary, while these simpler Lipschitz operators \(mean, sum, or max\) provide robust baseline aggregation, more adaptive mechanisms such as Softmax Attention Pooling \(§[2\.4](https://arxiv.org/html/2607.02990#S2.SS4)\) often yield superior flexibility and empirical performance, especially for graphs with complex or heterogeneous structures\. Softmax Attention Pooling, as discussed, can be constructed to maintain overall Lipschitz continuity on bounded embedding domains when the score network uses normalized inputs, upper spectral\-norm control, and clipped logits\. In our implementation, the final pooling projection is additionally conditioned with both lower and upper singular\-value bounds, yielding the partial co\-Lipschitz guarantee discussed above\.

Furthermore, practical implementations often involve choices about processing the pooled vector before its use in downstream tasks or loss functions\. For instance, when angle\-based similarity measures are used \(such as cosine similaritycos⁡\(⋅,⋅\)\\cos\(\\cdot,\\cdot\)employed inℒgraph​\_​align\\mathcal\{L\}\_\{\\mathrm\{graph\\\_align\}\}§[3\.2](https://arxiv.org/html/2607.02990#S3.SS2), or normalized SCL variants\), the final loss signal primarily depends on embedding direction, not magnitude\. However, the magnitude of the pooled vectors can still affect numerical stability and optimization dynamics\. Applying a bounded activation function like element\-wisetanh\\tanhdirectly after the primary aggregation step \(e\.g\.,sG=tanh⁡\(Pϕ​\(\{zu\}u∈V​\(G\)\)\)∈ℝdGs\_\{G\}=\\tanh\\\!\\bigl\(P\_\{\\phi\}\(\\\{z\_\{u\}\\\}\_\{u\\in V\(G\)\}\)\\bigr\)\\in\\mathbb\{R\}^\{d\_\{G\}\}\) can mitigate these issues by constraining the components of the pre\-normalized vectors to the range\(−1,1\)\(\-1,1\)\. This ensures inputs to the normalization step are well\-scaled, potentially improving stability and removing an unconstrained degree of freedom from the optimization\. While introducing saturation \(which might compress informative large magnitudes\), this approach provides component\-wise regularization, maintains the overall Lipschitz nature of the graph embedding function \(astanh\\tanhis 1\-Lipschitz\), and produces outputs more readily adaptable for efficient similarity search via binarization techniques\. As such, it represents a viable practical strategy within the MABLE toolkit, complementing the choice of pooling operator and decoder constraints\.

The considerations for condition\-number control on a linear feature decoderg​\(e\)=Wd​e​c​e\+bg\(e\)=W\_\{dec\}e\+b\(§[2\.5](https://arxiv.org/html/2607.02990#S2.SS5)\) also involve a conceptual balancing act when setting bounds for its singular values: the minimumm=σmin​\(Wd​e​c\)m=\\sigma\_\{\\min\}\(W\_\{dec\}\)and maximumL=σmax​\(Wd​e​c\)L=\\sigma\_\{\\max\}\(W\_\{dec\}\)\. The objective is a decoder that is robustly bi\-Lipschitz \(requiringm\>0m\>0and a finiteLL\) while remaining sufficiently expressive for accurate feature reconstruction\. In practical terms, boundingLLprevents the decoder from hallucinating large feature variances from small latent perturbations, while enforcing a strictly positivemmprevents the decoder from ignoring the latent space entirely \(a failure mode akin to posterior collapse\)\. Smaller values forLL, particularly approaching1\.01\.0, are theoretically appealing as they can enhance training stability by ensuring the decoder is non\-expansive or only mildly expansive, fostering more regularized mappings\. However, an overly restrictiveLLmight hypothetically curtail the decoder’s ability to reconstruct features if significant scaling or transformation from the latent space is necessary, suggesting a need for adequate ”slack” or expressiveness\. Simultaneously,mmmust be strictly positive to uphold the bi\-Lipschitz property, ensuring distinct latent states can lead to separable reconstructions\. The ratioL/mL/m\(the condition number\) dictates the decoder’s allowance for anisotropic distortion; a larger condition number implies more ”slack” but less uniform geometric preservation, whereas a smaller condition number \(largermmrelative toLL\) makes the decoder more rigid, strengthening the geometric coupling outlined in Proposition[1](https://arxiv.org/html/2607.02990#Thmproposition1)\. Thus, the selection ofmmandLLinvolves a hypothetical trade\-off between decoder flexibility for reconstruction, the stability afforded by tighter singular value control, and the desired strength of the geometric link between the latent and feature spaces, with optimal ranges likely being sensitive to specific dataset characteristics and modeling goals\.

## 3End\-to\-End Training Objectives for Node & Graph Embeddings

We now present the end\-to\-end objectives for learning both node\-level and graph\-level representations\. By training*end\-to\-end*, the model enforces similarity among similar nodes, encourages consistency of matched nodes across augmented views, aggregates those nodes into a graph\-level embedding, and encourages graph\-level uniformity among unpaired embeddings\. Our system uses in\-batch sampling for node embeddings and leverages augmented/un\-augmented pairs at the graph level\.

Drawing inspiration from fixed\-cosine surrogate principles \(alignment, uniformity, avoiding learned discriminators and the InfoNCE objective itself\), our framework employs loss functions instantiated differently across scales\. At the node level, we combine a masked reconstruction loss with a bi\-Lipschitz feature decoder to promote alignment in the reconstruction\-relevant feature slice, an explicit cross\-view node\-correspondence loss to promote consistency in the complementary contextual slice, and a node\-level uniformity penalty\. At the graph level, we enforce both alignment of augmented graph views and uniformity among unpaired graph embeddings using penalties analogous to those derived for the fixed\-cosine surrogate\. This design provides a scalable, geometrically interpretable implementation of contrastive principles, adapted to masked graph data\.

### 3\.1Node Alignment, Node Correspondence, and Node Uniformity

We start with a*masked*reconstruction objective that encourages the learned representation to capture meaningful dependencies within and across attributes, and to align similar nodes in the reconstruction\-relevant embedding space\. This loss penalizes the model’s ability to accurately reconstruct the original or masked features of a node from its latent embedding:

ℒnode​\_​align=𝔼G∼𝒟​𝔼T∼𝒜​𝔼v∼V​\(G\(a\)\)​\[ℓrecon​\(g​\(π​\(fθ​\(G\(a\),X\(a\)\)v\)\),xι\(a\)​\(v\);T\)\],\\mathcal\{L\}\_\{\\mathrm\{node\\\_align\}\}=\\mathbb\{E\}\_\{G\\sim\\mathcal\{D\}\}\\ \\mathbb\{E\}\_\{T\\sim\\mathcal\{A\}\}\\ \\mathbb\{E\}\_\{v\\sim V\(G^\{\(a\)\}\)\}\\Bigl\[\\ell\_\{\\mathrm\{recon\}\}\\bigl\(g\(\\pi\(f\_\{\\theta\}\(G^\{\(a\)\},X^\{\(a\)\}\)\_\{v\}\)\),x\_\{\\iota^\{\(a\)\}\(v\)\};T\\bigr\)\\Bigr\],
where𝒟\\mathcal\{D\}is the training distribution over graphs,T∼𝒜T\\sim\\mathcal\{A\}is the view operator from §[2\.1](https://arxiv.org/html/2607.02990#S2.SS1), and\(G\(a\),X\(a\),ι\(a\)\)=T​\(G,X\)\(G^\{\(a\)\},X^\{\(a\)\},\\iota^\{\(a\)\}\)=T\(G,X\)\. The encoderfθf\_\{\\theta\}consumes the augmented input\(G\(a\),X\(a\)\)\(G^\{\(a\)\},X^\{\(a\)\}\)and outputs a contextualized embedding for each retained nodev∈V​\(G\(a\)\)v\\in V\(G^\{\(a\)\}\), whilexι\(a\)​\(v\)x\_\{\\iota^\{\(a\)\}\(v\)\}is the original reconstruction target associated with that retained node identity\. The sliceπ\\piselects the feature\-embedding subspace used for reconstruction andggis the \(bi\-Lipschitz\) feature decoder \(§[2\.5](https://arxiv.org/html/2607.02990#S2.SS5)\)\. The reconstruction lossℓrecon​\(⋅,⋅;T\)\\ell\_\{\\mathrm\{recon\}\}\(\\cdot,\\cdot;T\)is a norm\-based penalty in feature space and may depend on the view mask \(e\.g\., evaluated only on masked components, only on unmasked components, or on both with feature\-type weights\)\.

Masking encourages the encoder to learn feature embeddings that support consistent feature prediction from context for nodes with similar local structure or attributes, thereby promoting grouping in the reconstruction\-relevant feature\-embedding space\. Importantly, due to the bi\-Lipschitz nature ofgg\(see Proposition[1](https://arxiv.org/html/2607.02990#Thmproposition1)\), minimizing reconstruction error acts as a geometric regularizer in the learned feature\-embedding space\. If two nodes have similar reconstruction targets and both are reconstructed accurately, then their feature\-embedding slices are forced to be close up to the reconstruction\-error slack in Proposition[1](https://arxiv.org/html/2607.02990#Thmproposition1)\. Without the lower\-Lipschitz component of the decoder constraint, accurate reconstruction alone would not rule out arbitrarily arranged latent codes\.

##### Cross\-View Node Correspondence in the Contextual Slice\.

Reconstruction constrains only the feature sliceeu=π​\(zu\)e\_\{u\}=\\pi\(z\_\{u\}\)\. To encourage the remaining contextual slicecu=ρ​\(zu\)c\_\{u\}=\\rho\(z\_\{u\}\)to encode view\-stable information rather than augmentation\-specific variation, we additionally align matched nodes across two augmented views of the same graph, in the spirit of bootstrap\-style cross\-view consistency methods such as BGRL\[Thakooret al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib30)\]\. Let\(G\(a\),X\(a\),ι\(a\)\)=Ta​\(G,X\)\(G^\{\(a\)\},X^\{\(a\)\},\\iota^\{\(a\)\}\)=T\_\{a\}\(G,X\)fora∈\{1,2\}a\\in\\\{1,2\\\}denote two masked/augmented views of the same underlying graphGG\. We define the common retained identity set

𝒞​\(G,T1,T2\):=ι\(1\)​\(V​\(G\(1\)\)\)∩ι\(2\)​\(V​\(G\(2\)\)\)⊆V​\(G\)\.\\mathcal\{C\}\(G,T\_\{1\},T\_\{2\}\):=\\iota^\{\(1\)\}\(V\(G^\{\(1\)\}\)\)\\cap\\iota^\{\(2\)\}\(V\(G^\{\(2\)\}\)\)\\subseteq V\(G\)\.Foru∈𝒞​\(G,T1,T2\)u\\in\\mathcal\{C\}\(G,T\_\{1\},T\_\{2\}\), writeva​\(u\):=\(ι\(a\)\)−1​\(u\)v\_\{a\}\(u\):=\(\\iota^\{\(a\)\}\)^\{\-1\}\(u\)for the corresponding node in viewaa\.111In practice,𝒞​\(G,T1,T2\)\\mathcal\{C\}\(G,T\_\{1\},T\_\{2\}\)is the set of nodes that survive the two augmentations and can be aligned by identity or inherited indexing; for efficiency we may evaluate the loss on a random subset of𝒞​\(G,T1,T2\)\\mathcal\{C\}\(G,T\_\{1\},T\_\{2\}\)in each minibatch\.We define

ℒnode​\_​corr=𝔼G∼𝒟​𝔼T1,T2∼𝒜​𝔼u∼𝒞​\(G,T1,T2\)​\[ℓcorr​\(ρ​\(fθ​\(G\(1\),X\(1\)\)v1​\(u\)\),ρ​\(fθ​\(G\(2\),X\(2\)\)v2​\(u\)\)\)\],\\mathcal\{L\}\_\{\\mathrm\{node\\\_corr\}\}\\;=\\;\\mathbb\{E\}\_\{G\\sim\\mathcal\{D\}\}\\ \\mathbb\{E\}\_\{T\_\{1\},T\_\{2\}\\sim\\mathcal\{A\}\}\\ \\mathbb\{E\}\_\{u\\sim\\mathcal\{C\}\(G,T\_\{1\},T\_\{2\}\)\}\\Bigl\[\\ell\_\{\\mathrm\{corr\}\}\\\!\\bigl\(\\rho\(f\_\{\\theta\}\(G^\{\(1\)\},X^\{\(1\)\}\)\_\{v\_\{1\}\(u\)\}\),\\,\\rho\(f\_\{\\theta\}\(G^\{\(2\)\},X^\{\(2\)\}\)\_\{v\_\{2\}\(u\)\}\)\\bigr\)\\Bigr\],whereℓcorr\\ell\_\{\\mathrm\{corr\}\}is a similarity\-matching loss on the contextual slice \(e\.g\., squaredℓ2\\ell\_\{2\}distance between normalized contextual vectors, or equivalently a cosine\-alignment penalty up to constants\)\. This term encourages matched nodes across augmentations to remain close in the contextual subspace while leaving the reconstruction\-relevant feature slice to be shaped primarily by the masked autoencoding objective\.

##### In\-Batch Node Sampling for Node Uniformity\.

We then promote uniformity among node embeddings by randomly pairing node instances*within the same batch*and penalizing the squared cosine similarity between embeddings of unpaired nodes\. This term is analogous to the negative term in the Spectral Contrastive Loss \(SCL\) and the log\-based surrogate, which encourages orthogonality among negative pairs:

ℒnode​\_​uniform=𝔼η,η′∼𝒵node,η≠η′\[cos\(η,η′\)2\]\.\\mathcal\{L\}\_\{\\mathrm\{node\\\_uniform\}\}\\;=\\;\\mathbb\{E\}\_\{\\eta,\\eta^\{\\prime\}\\sim\\mathcal\{Z\}\_\{\\mathrm\{node\}\},\\ \\eta\\neq\\eta^\{\\prime\}\}\\Bigl\[\\cos\(\\eta,\\eta^\{\\prime\}\)^\{2\}\\Bigr\]\.Here𝒵node\\mathcal\{Z\}\_\{\\mathrm\{node\}\}denotes the current minibatch pool of valid retained node embeddings, distinct from the node\-instance setℬnode\\mathcal\{B\}\_\{\\mathrm\{node\}\}defined above\. In our implementation,𝒵node\\mathcal\{Z\}\_\{\\mathrm\{node\}\}contains token embeddings from both augmented views of all graphs in the minibatch, sampled uniformly from this pooled token set\. Exact cross\-view matches of the same retained node identity are rare under this uniform sampling in large token pools, so we use this term as a lightweight isotropy/decorrelation regularizer rather than as a strict semantic\-negative objective\. This objective penalizes large absolute dot products between sampled token embeddings, pushing their cosine similarity towards orthogonality \(zero\)\.

### 3\.2Graph\-Level Pooling and Positive Pair Augmentation

Next, we consider the graph\-level embeddingsGs\_\{G\}\. For a given view\(G\(a\),X\(a\),ι\(a\)\)\(G^\{\(a\)\},X^\{\(a\)\},\\iota^\{\(a\)\}\), this embedding is typically generated using the Adaptive Graph Pooling with Softmax Attention mechanism \(PattnP\_\{\\mathrm\{attn\}\}\), as detailed in §[2\.4](https://arxiv.org/html/2607.02990#S2.SS4), applied to the set of retained node embeddings:

sG\(a\)=Pattn​\(\{fθ​\(G\(a\),X\(a\)\)v:v∈V​\(G\(a\)\)\}\),s\_\{G\}^\{\(a\)\}=P\_\{\\mathrm\{attn\}\}\\bigl\(\\\{f\_\{\\theta\}\(G^\{\(a\)\},X^\{\(a\)\}\)\_\{v\}:\\,v\\in V\(G^\{\(a\)\}\)\\\}\\bigr\),yielding a single vector per graph view\. This adaptive mechanism is often favored for its flexibility in handling varied graph structures and for its potential empirical performance benefits\. While simpler Lipschitz operators likeMeanPool\\mathrm\{MeanPool\}\(cf\. §[2\.3](https://arxiv.org/html/2607.02990#S2.SS3)\) can serve as alternatives or for comparative studies, any chosen Lipschitz pooling operator helps control sensitivity to changes in graph size or node embedding scale\. However, pooling alone is insufficient to guarantee invariance to structured perturbations such as node masking, feature noise, or irregular subgraph sampling\. The role of positive pair augmentation is thus critical: by explicitly training the encoder and pooling to align perturbed views of the same graph, the model learns to ignore incidental variations and focus on stable, semantically meaningful patterns\. We create positive graph\-view pairs by samplingT1,T2∼𝒜T\_\{1\},T\_\{2\}\\sim\\mathcal\{A\}from the same underlying\(G,X\)\(G,X\), e\.g\.:

1. 1\.Augmented copy:a view might keep the same node set but apply partial node masking, node dropping, or slight feature corruption\.
2. 2\.Subgraph sampling:IfGGis large, we can sample subgraphs \(e\.g\., via node dropping\) to form augmented views, treating the embedding of a subgraph as a positive view of the entire graph, provided they remain semantically related\.

Using a penalty based on cosine similarity \(for nonzero vectorsa,ba,b, writinga^:=a/‖a‖2\\hat\{a\}:=a/\\\|a\\\|\_\{2\}andb^:=b/‖b‖2\\hat\{b\}:=b/\\\|b\\\|\_\{2\}, so that the usual identity1−cos⁡\(a,b\)=12​‖a^−b^‖221\-\\cos\(a,b\)=\\tfrac\{1\}\{2\}\\\|\\hat\{a\}\-\\hat\{b\}\\\|\_\{2\}^\{2\}holds\), we define the inter\-graph alignment loss:

ℒgraph​\_​align=𝔼G∼𝒟​𝔼T1,T2∼𝒜​\[1−cos⁡\(sG\(1\),sG\(2\)\)\]\.\\mathcal\{L\}\_\{\\mathrm\{graph\\\_align\}\}=\\mathbb\{E\}\_\{G\\sim\\mathcal\{D\}\}\\ \\mathbb\{E\}\_\{T\_\{1\},T\_\{2\}\\sim\\mathcal\{A\}\}\\left\[1\-\\cos\\\!\\bigl\(s\_\{G\}^\{\(1\)\},s\_\{G\}^\{\(2\)\}\\bigr\)\\right\]\.HeresG\(a\)s\_\{G\}^\{\(a\)\}is computed from the view\(G\(a\),X\(a\),ι\(a\)\)=Ta​\(G,X\)\(G^\{\(a\)\},X^\{\(a\)\},\\iota^\{\(a\)\}\)=T\_\{a\}\(G,X\)as above\.

Minimizing this loss is equivalent to maximizing the cosine similarity between the two view embeddings\. When one view is a subgraph of the other \(a possible outcome of our augmentation strategy, as previously described\), this alignment objective specifically encourages the larger\-view embedding to be predictive of the representation of its own local \(subgraph\) structure\. Such an alignment between global and local views of the same data instance is a core tenet of mutual information maximization strategies like Deep InfoMax\[Hjelmet al\.,[2018](https://arxiv.org/html/2607.02990#bib.bib12), Veličkovićet al\.,[2019](https://arxiv.org/html/2607.02990#bib.bib13)\]\. This alignment is particularly important in geospatial and geological datasets, where differences in node count or spatial density often reflect sampling artifacts rather than true semantic divergence\. Together, the pooling and augmentation strategy is designed to improve robustness to cardinality mismatches, topological edits, and local feature inconsistencies\.

### 3\.3Graph Uniformity

Lastly, to promote inter\-graph uniformity, for an anchor graph identityGG, a view embeddingsG\(a\)s\_\{G\}^\{\(a\)\}is pushed away from embeddings of views of other underlying graphs sampled from the current batch\. Concretely, for a minibatchℬgraph\\mathcal\{B\}\_\{\\mathrm\{graph\}\}of graphs and their augmented views, the anchor identity is the underlying graphGG; its own views include all embeddingssG\(a\)s\_\{G\}^\{\(a\)\}derived from that same graph\. We takeNegPool​\(G\)\\mathrm\{NegPool\}\(G\)to be the empirical distribution over all graph\-level view embeddings in the batch excluding those views ofGGitself\. In particular, it may comprise full\-graph or subgraph view embeddings derived from other graphsH≠GH\\neq G\. Analogous to the SCL negative term, we penalize the squared cosine similarity:

ℒgraph​\_​uniform=𝔼G,Ta,Z−∈NegPool​\(G\)\[cos\(sG\(a\),Z−\)2\]\.\\mathcal\{L\}\_\{\\mathrm\{graph\\\_uniform\}\}=\\mathbb\{E\}\_\{G,\\,T\_\{a\},\\,Z^\{\-\}\\in\\mathrm\{NegPool\}\(G\)\}\\bigl\[\\cos\\\!\\bigl\(s\_\{G\}^\{\(a\)\},Z^\{\-\}\\bigr\)^\{2\}\\bigr\]\.Here, for each anchor graphGG,sG\(a\)s\_\{G\}^\{\(a\)\}is computed from a sampled anchor viewTa​\(G,X\)T\_\{a\}\(G,X\), andZ−Z^\{\-\}is sampled from the empirical distribution that is uniform over elements ofNegPool​\(G\)\\mathrm\{NegPool\}\(G\)\. The importance of such explicit uniformity\-promoting terms for enhancing feature diversity and preventing issues like representation collapse has also been highlighted in masked autoencoder frameworks such as U\-MAE\[Zhanget al\.,[2022](https://arxiv.org/html/2607.02990#bib.bib1)\]\. In practice, these unpaired in\-batch graphs are a heuristic source of contrast rather than guaranteed semantic negatives\. WhenZ−Z^\{\-\}includes subgraph view embeddings from another graphHH, this objective contributes to making views ofGGdiscriminable from local views of other graphs, complementing the positive local\-global alignment encouraged byℒgraph​\_​align\\mathcal\{L\}\_\{\\mathrm\{graph\\\_align\}\}\.

### 3\.4Combined Objective

Putting it all together, we train*end\-to\-end*with a single combined objective:

ℒ=α​ℒnode​\_​align\+β​ℒnode​\_​corr\+γ​ℒnode​\_​uniform\+λ​ℒgraph​\_​align\+μ​ℒgraph​\_​uniform\.\\mathcal\{L\}\\;=\\;\\alpha\\,\\mathcal\{L\}\_\{\\mathrm\{node\\\_align\}\}\\;\+\\;\\beta\\,\\mathcal\{L\}\_\{\\mathrm\{node\\\_corr\}\}\\;\+\\;\\gamma\\,\\mathcal\{L\}\_\{\\mathrm\{node\\\_uniform\}\}\\;\+\\;\\lambda\\,\\mathcal\{L\}\_\{\\mathrm\{graph\\\_align\}\}\\;\+\\;\\mu\\,\\mathcal\{L\}\_\{\\mathrm\{graph\\\_uniform\}\}\.
By tuning the above scalars, we balance local \(node\) alignment via reconstruction, cross\-view node correspondence in the contextual slice, node\-level uniformity, graph\-level augmentation alignment, and graph uniformity across the dataset\.

### 3\.5Holistic Benefits

This fully integrated pipeline yields robust node and graph embeddings with the following advantages:

- •Principled Surrogates:By using fixed\-cosine based penalties analogous to normalized SCL, we avoid learned discriminators and their associated stability and bias issues\. The normalized/cosine SCL\-style surrogate also admits a heuristic second\-order connection to a DV\-style mutual\-information objective under boundedness, approximate symmetry, and small\-variance assumptions \([SectionA\.3](https://arxiv.org/html/2607.02990#A1.SS3)\)\.
- •Local Quality:Masked node autoencoding plus bi\-Lipschitz decoding encourages the*feature\-embedding slice*used for reconstruction to reflect local similarity: similar node attributes lead to nearby feature embeddings up to the reconstruction\-error slack in Proposition[1](https://arxiv.org/html/2607.02990#Thmproposition1)\. A complementary node\-correspondence loss encourages the remaining contextual slice to stay consistent across matched nodes in augmented views\. The node uniformity penalty shapes the full node embeddings by encouraging unpaired in\-batch node embeddings to stay near\-orthogonal\. Together, these terms encourage node representations that are locally faithful, cross\-view stable, and well spread\.
- •Reliable Graph Embeddings:The use of Lipschitz\-controlled pooling mechanisms, particularly adaptive Softmax Attention Pooling \(§[2\.4](https://arxiv.org/html/2607.02990#S2.SS4)\) as well as simpler operators like mean or sum \(cf\.[Proposition3](https://arxiv.org/html/2607.02990#Thmproposition3)\), ensures that small changes in node embeddings lead to proportionally bounded changes insGs\_\{G\}\. Furthermore, the graph alignment penalty \(ℒgraph​\_​align\\mathcal\{L\}\_\{\\mathrm\{graph\\\_align\}\}\) contributes to the robustness of these embeddings against augmentation variations\.
- •Global Uniformity:By drawing data from the same batch or dataset, both at node and graph level, the uniformity penalties encourage a well\-spread, approximately isotropic embedding distribution across the collection of graphs, complementing local alignment and helping prevent representation collapse\.
- •Implicit Deep InfoMax\-Style Learning:By leveraging subgraph augmentations for positive pairs inℒgraph​\_​align\\mathcal\{L\}\_\{\\mathrm\{graph\\\_align\}\}\(aligning two views of the same underlying graph, potentially including full\-graph and local\-subgraph views\) and employing a broad in\-batch contrastive strategy forℒgraph​\_​uniform\\mathcal\{L\}\_\{\\mathrm\{graph\\\_uniform\}\}\(contrasting views of one graph against full\-graph or subgraph views of other graphs\), MABLE encourages a learning dynamic similar to Deep InfoMax\. It learns global representations that are predictive of their own local structures while being distinct from the local structures of other graphs, all achieved without a learned discriminator due to the SCL\-based fixed\-cosine surrogates\.
- •End\-to\-End Efficiency:Because node\- and graph\-level objectives are co\-trained, we do not need separate staging or complex hard\-negative selection steps\. In\-batch sampling, straightforward augmentation, and sampled matched\-node correspondence provide the necessary positive and regularizing signals within a single training loop\.
- •Output\-Side Stability:Constraining the bi\-Lipschitz decoder and Lipschitz pooling layers provides controlled final\-stage mappings from embeddings to reconstruction and graph\-level objectives, while leaving encoder\-level regularity as a design hypothesis rather than a formal guarantee\.
- •Conditioned Decoder Control:Ifggis a single linear layer with constrained singular values \(§[2\.5](https://arxiv.org/html/2607.02990#S2.SS5)\), then the final reconstruction map is protected against trivial collapse or unbounded expansion, reducing the risk of poorly conditioned latent\-to\-feature mappings\.

Overall, this holistic design encourages similarity, correspondence, and uniformity at both the node and graph levels in one end\-to\-end pass\. Crucially, by leveraging principled fixed\-cosine surrogates, simple augmentations, in\-batch sampling, sampled matched\-node correspondence, and a well\-conditioned decoder, the burden of representation is shifted to the encoder, which must learn meaningful and robust features\. This principle is strongly supported by findings like MIM\-Refiner\[Alkinet al\.,[2025](https://arxiv.org/html/2607.02990#bib.bib4)\], which show that lightweight decoders lead to more discriminative intermediate representations, ultimately improving performance in downstream tasks\.

## 4Experiments

We evaluate MABLE at two spatial scales \(prospect and regional; details deferred to subsequent subsections\)\. Here we first describe the shared backbone and training procedure used across both experiments, including masking, augmentation, and the instantiation of the losses introduced in §[3](https://arxiv.org/html/2607.02990#S3)\.

### 4\.1GeoReformer backbone, tokenization, and training procedure

##### Backbone \(GeoReformer\)\.

All experiments use a Reformer\-style encoder\[Kitaevet al\.,[2020](https://arxiv.org/html/2607.02990#bib.bib25)\]with LSH self\-attention, adapted to geospatial node sets and referred to as theGeoReformer\. GeoReformer is the encoder instantiation used in our experiments; the MABLE objectives only require an encoder that maps a graph or node collection to node embeddings, so other GNN, graph\-transformer, or set\-transformer backbones could be substituted\. Each graph is encoded as a padded sequence of node tokens and processed by a stack of LSH attention blocks to support thousands of nodes per sample\. We remove sequence/axial positional encodings because node order is arbitrary\.

##### Design rationale: edge\-free graphs via induced connectivity\.

We represent each sample as an*edge\-free*set of node observations and avoid constructing an explicit adjacency matrix\. In mineral prospectivity settings, heuristic edges \(e\.g\.,kk\-NN or radius graphs\) can inject strong, dataset\-specific priors and become brittle under heterogeneous sampling densities and multi\-sensor geometry\[Sihombinget al\.,[2024](https://arxiv.org/html/2607.02990#bib.bib23), Darunaet al\.,[2024](https://arxiv.org/html/2607.02990#bib.bib5), Zuoet al\.,[2023](https://arxiv.org/html/2607.02990#bib.bib24)\]\. Instead, we let the model*induce*an example\-specific sparse connectivity pattern through its attention mechanism: the Reformer\-style LSH routing performs a hard partition of tokens into candidate neighborhoods, yielding a block\-sparse attention graph that scales to tens of thousands of nodes while remaining agnostic to arbitrary node ordering\. This shifts the burden of discovering useful spatial interactions from a hand\-crafted graph construction rule to learned attention over data\-dependent neighborhoods\.

##### Node tokenization and input projections\.

Each node token concatenates \(i\) a feature\-type identifierτu∈\{1,…,K\}\\tau\_\{u\}\\in\\\{1,\\dots,K\\\}\(stored as a one\-hot vector overKKfeature IDs, and optionally mapped to a fixed unit\-norm code vectorq​\(τu\)∈ℝKq\(\\tau\_\{u\}\)\\in\\mathbb\{R\}^\{K\}for training stability\), \(ii\) an optional scalar feature valuerur\_\{u\}\(or a sentinel when not applicable\), and \(iii\) relative geometric features \(e\.g\., coordinates expressed relative to the graph center, with dataset\-specific scaling\)\. The resulting encoder token augments the reconstructible feature vector with relative geometric information\. Concretely, the reconstructible feature vectorxux\_\{u\}comprises the feature\-type identifier together with the scalar value when present, while relative geometry is provided to the encoder as auxiliary positional/context information concatenated to this feature vector\. The full token is then projected into the model hidden dimensionddby a learned linear embedder,tu=E​\(⋅\)∈ℝdt\_\{u\}=E\(\\cdot\)\\in\\mathbb\{R\}^\{d\}, providing a minimal and modality\-agnostic input interface for heterogeneous geospatial observations\.

##### Relative geometry only; no absolute spatial identity\.

To avoid trivial location\-based shortcuts and to improve transfer across regions, we do not inject absolute or global position embeddings into the sequence\. Geometric information enters only through*relative*positional features and through augmentation, which preserves the local spatial relationships while discouraging memorization of absolute coordinates\.

##### LSH self\-attention and routing regularization\.

GeoReformer uses LSH self\-attention to approximate full self\-attention with sub\-quadratic cost\. Concretely, tokens are assigned to buckets by hashing projected representations; these bucket assignments define the*support*of the attention computation \(i\.e\., which token pairs are eligible to interact\) and thus act as a hard, example\-specific sparsification of the attention adjacency\. During training, we regularize this routing by sampling bucket assignments from a temperature\-scaled distribution over the hash projection scores, rather than using a deterministic argmax\. This stochastic routing reduces brittleness of the induced block structure and improves robustness\. At evaluation time, routing reverts to deterministic argmax assignment, recovering the standard Reformer behavior\.

##### Masking with feature\-type specific mask tokens\.

We use a feature\-type aware masking scheme with a dedicated learned mask token \(a trainable replacement vector\) per feature type\. When a node is masked, we keep its relative geometric features unchanged but replace both the feature\-ID block and the scalar value \(when present\) with the corresponding mask token\. In this experimental tokenization, the model is trained to reconstruct the original feature\-type identifier \(represented either as a one\-hot vector or its fixed codeq​\(τu\)q\(\\tau\_\{u\}\)\) together with the scalar value for feature types that admit one; MABLE itself only requires a reconstructible target appropriate to the domain\. For feature types without an associated scalar value, the scalar reconstruction term is omitted\. Because the mask token is feature\-type specific by design, the nontrivial learning signal comes primarily from reconstructing masked values/content from cross\-feature context and relative spatial structure, rather than from inferring the feature type itself\.

##### Two\-view augmentation \(mask\-only vs drop–rotate–mask\)\.

Training forms positive view pairs from the same underlying graph\. The*original*view applies node masking only\. The*positive*view combines \(i\) stochastic node dropping, \(ii\) random Z\-axis rotations around a local reference point \(e\.g\., the graph center\), and \(iii\) masking\. Dropping is applied both*globally*and in*spatial patches*: with some probability we sample a patch \(random center and radius\) and remove nodes within that region\. Patch dropping can be*uniform*\(the entire patch is dropped\) or*feature\-conditional*, where each feature type is independently retained or removed with a sampled probability, effectively simulating missing modalities or incomplete acquisition within localized areas\. A minimum\-retained\-node constraint avoids degenerate views under aggressive dropping\. Rotation augmentation is used specifically to encourage the learned representations, especially graph\-level embeddings, to be insensitive to local orientation, i\.e\., to favor rotational invariance in the embedding space\. This is desirable in the mineral\-exploration setting considered here, where local patch orientation is typically not itself the quantity of interest, whereas relative spatial configuration and cross\-modal context are\.

##### Decoder conditioning and bounded attention scores\.

We use a bi\-Lipschitz feature decoder for reconstruction by clamping the singular values of the decoder matrix to\[σmin,σmax\]\[\\sigma\_\{\\min\},\\sigma\_\{\\max\}\]\. For graph pooling, we use standard softmax attention: the attention MLP receives unit\-normalized node embeddings, its internal linear maps are clamped only in*maximum*singular value, and the pre\-softmax logits are shifted by their within\-graph maximum and clipped to\[−c,0\]\[\-c,0\], thereby bounding relative logit gaps in a shift\-invariant way\. The final pooling projectionWp​o​o​lW\_\{pool\}is separately conditioned by clamping its singular values to\[m,L\]\[m,L\]\. Thus, the score network is controlled for upper\-Lipschitz stability, while the output projection additionally provides the partial co\-Lipschitz guarantee stated in §[2\.4](https://arxiv.org/html/2607.02990#S2.SS4)\.

##### Losses and optimization\.

The training objective combines reconstruction with representation\-shaping terms: node\-level correspondence and uniformity, graph\-level alignment between two views of the same underlying graph, and graph\-level uniformity within a batch\. Throughout, the decoder is trained to predict the full node reconstruction targetxux\_\{u\}\.222We usexux\_\{u\}to denote the reconstruction target; the encoder input is formed by masking/corrupting the reconstructible components ofxux\_\{u\}\(where selected components are replaced by learned mask tokens\) and concatenating the resulting vector with the relative geometric context\.Masking is applied to the*encoder input*by corrupting a subset of feature dimensions/types with learned mask tokens, so that predicting the corrupted entries is inherently context\-dependent and does not admit trivial copy\-through shortcuts\. In training we use \(i\) a*masked*reconstruction loss, evaluated on the masked entries, and \(ii\) an auxiliary reconstruction loss on the unmasked entries; together these form a weighted reconstruction loss on the full vectorxux\_\{u\}, with weights chosen as a task\- and modality\-dependent implementation choice\. We use anℓ2\\ell\_\{2\}\-style reconstruction loss \(MSE\) on all reconstructed components, including the feature\-ID representation \(one\-hot or fixed code\) and the scalar value when present\. This keeps the reconstruction target in a common Euclidean space consistent with the geometric role of the decoder\. In addition, for two augmented views of the same graph we apply a node\-correspondence loss on the*contextual slice*of matched nodes retained in both views; for efficiency and to avoid over\-constraining the representation, this term is evaluated on a random subset of valid matched nodes per graph in each minibatch\. Uniformity terms use squared cosine similarity surrogates, and node\-level uniformity is likewise estimated by sampling valid tokens \(optionally from both views\) for efficiency\. Optimization uses Adam with a cosine learning\-rate schedule and linear warmup; some experiment\-specific hyperparameters are reported with each experimental setting\.

##### Implementation\-detail availability\.

The experiments were run in a proprietary commercial environment\. We report the architectural choices, data splits, embedding dimensions, downstream feature construction, and evaluation protocols needed to interpret the results, but most internal low\-level implementation details and training hyperparameters are not fully disclosed here\. All reported comparisons use the same internal training configuration unless an ablation explicitly states otherwise, so the experimental claims should be read as controlled comparisons within that fixed implementation rather than as a complete public reproduction recipe\.

### 4\.2Experiment: Regional\-scale Arabian Shield dataset

##### Study area and geological context\.

Our second experiment targets the*Arabian Shield*, a structurally complex Neoproterozoic shield comprising diverse accreted terranes, sutures, and intrusive bodies\. This setting provides a large, heterogeneous multi\-modal corpus for node and graph\-level pretraining at regional scale, where robustness to uneven coverage and strong geological non\-stationarity are central\.

##### Data modalities\.

We assemble a multi\-sensor stack over the Arabian Shield using:

- •Surface geochemistry:Saudi Geological Survey \(SGS\) surface geochemistry\.
- •Optical imagery:Sentinel\-2 raw optical channels \(Channel 1 removed\), with invalid pixels masked using the Sentinel\-2 Scene Classification Layer\.
- •Geophysics:Total Magnetic Intensity \(TMI\) Reduced\-to\-Pole \(RTP\), with partial coverage over the Shield\.
- •Regional seismic:KSA seismicVSV\_\{S\}data resampled to 600 m horizontal spacing, truncated to 10 km depth, and spatially filtered to the Shield footprint\.
- •Topography:DEM data \(used for dataset collation but*not*used as an explicit feature\)\.

##### Graph construction \(random\-patch sampling\)\.

Graphs are constructed by randomly selecting a horizontal center\(y0,x0\)\(y\_\{0\},x\_\{0\}\)within the dataset footprint and extracting a fixed\-radius patch\. Each sample is an*edge\-free*graph whose nodes correspond to observations from the available modalities falling inside a 200 m radius disc in the horizontal plane:

\(y−y0\)2\+\(x−x0\)2≤200​m\.\\sqrt\{\(y\-y\_\{0\}\)^\{2\}\+\(x\-x\_\{0\}\)^\{2\}\}\\leq 200~\\text\{m\}\.We retain each observation’s vertical coordinatezzas part of the node token, but do not usezzfor inclusion, i\.e\., the receptive field is a cylinder in\(y,x\)\(y,x\)with full retained vertical structure\. Node geometry is encoded in a local relative frame centered at\(y0,x0\)\(y\_\{0\},x\_\{0\}\), consistent with the relative\-geometry\-only design of §[4\.1](https://arxiv.org/html/2607.02990#S4.SS1)\.

##### Dataset size and structure\.

The resulting dataset contains approximately145145k graphs and951951M total nodes\. Graphs contain about6\.66\.6k nodes on average \(std\.≈1\.3\\approx 1\.3k\), ranging from roughly5050to8\.08\.0k nodes per graph\. Across these modalities there are 71 distinct feature types, which are tokenized using the feature\-type identifier scheme described in §[4\.1](https://arxiv.org/html/2607.02990#S4.SS1)\.

##### Train/validation splits\.

We use a 90% / 10% split into training and validation graphs for model selection\. Subsequent subsections report embedding diagnostics and downstream analyses under this split\.

#### 4\.2\.1Study 1: Regional Embedding Structure and Emergent Zonation Patterns

We evaluate the structure of the 256\-dimensional embeddings learned by MABLE over the Arabian Shield using a complementary set of spatial, statistical, spectral, and graph\-based analyses\. Although the model is trained without access to absolute/global geographic coordinates or geological labels, these diagnostics indicate that the learned representation contains spatially coherent, qualitatively geologically plausible structure across multiple scales\.

##### Dimension\-Level Spatial Structure and Latent Geometry\.

We begin by visualizing individual embedding dimensions geospatially \(Fig\.[1](https://arxiv.org/html/2607.02990#S4.F1)\)\. Most dimensions exhibit clear spatial coherence, large contiguous patches, gradients, and sharp transitions, rather than unstructured noise\. Several dimensions appear to respond to broad terrane\-scale domains, while others emphasize finer lithological texture, suggesting meaningful specialization across embedding channels and a genuinely distributed representation\.

A complementary geometric sanity check is the cosine\-similarity distribution between 5,000 randomly sampled embedding pairs \(Fig\.[2](https://arxiv.org/html/2607.02990#S4.F2)\)\. Many representation learning pipelines suffer from*anisotropy*\(a “cone effect”\), where embeddings concentrate around a dominant direction and cosine similarity becomes less discriminative\. Here, the near\-zero mean \(μ≈0\.054\\mu\\approx 0\.054\) is consistent with low anisotropy and broad angular coverage, and there is no evident collapse in the learned embedding distribution\. The pronounced positive tail, in turn, reflects dense regional domains in latent space: randomly sampled pairs occasionally fall within the same large\-scale terrane or lithological regime, yielding high similarity\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/dim_maps_grid.png)Figure 1:Spatial activation maps for selected Arabian Shield graph\-embedding dimensions \(Experiment 2\)\. Most dimensions exhibit coherent, non\-random spatial structure, consistent with structured geoscience signal\.![Refer to caption](https://arxiv.org/html/2607.02990v1/x1.png)Figure 2:Cosine\-similarity distribution for 5,000 randomly sampled Arabian Shield graph embedding pairs \(Experiment 2, Study 1\)\. The near\-zero mean \(μ≈0\.054\\mu\\approx 0\.054\) is consistent with low anisotropy in the learned embedding distribution, avoiding the common failure mode in self\-supervised and contrastive models where representations collapse toward a dominant direction\. The approximately symmetric central mass suggests broad angular coverage of the latent space, while the pronounced long tail reflects dense regional clusters \(e\.g\., terrane\-scale domains\) encoded without sacrificing global diversity\.
##### Manifold Projections Reveal Regional Organization\.

Low\-dimensional projections provide a compact view of how the embedding space is organized at regional scale \(Fig\.[3](https://arxiv.org/html/2607.02990#S4.F3)\)\. The UMAP RGB map \(cosine metric,nneighbors=30n\_\{\\mathrm\{neighbors\}\}\{=\}30\) exhibits sharply bounded, spatially coherent domains, consistent with terrane\-scale segmentation emerging from signal content alone \(no absolute/global coordinates were used during training\)\. In contrast, the PCA RGB map captures a broad west–east gradient but appears substantially more diffuse\. This combination suggests that the embedding geometry is strongly non\-linear: regional structure is present in the representation, but it is not well described by a small number of linear modes\. Consistent with this, the first three principal components explain only∼\\sim17% of the variance, indicating that information is distributed across many embedding dimensions rather than concentrated into a few dominant axes\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/x2.png)\(a\)UMAP \(cosine\),nneighbors=30n\_\{\\mathrm\{neighbors\}\}=30\.
![Refer to caption](https://arxiv.org/html/2607.02990v1/x3.png)\(b\)PCA \(top 3 components\)\.

Figure 3:Low\-dimensional projections of the 256\-dimensional Arabian Shield graph embeddings \(Experiment 2, Study 1\), visualized by mapping the first three coordinates of each projection to RGB\. UMAP reveals sharply bounded, spatially coherent domains, consistent with terrane\-scale segmentation emerging without absolute/global coordinate input during training\. PCA provides a linear baseline that recovers a broad west–east gradient but appears noisier, consistent with variance being distributed across many embedding dimensions rather than concentrated in a few linear modes\.
##### Similarity Search Supports Anomalies as Structured Signals\.

To test whether outlier scores reflect repeatable structure rather than preprocessing artifacts, we run a cosine\-similarity retrieval over the full validation set using representative*anomalous*and*typical*query embeddings selected by Isolation Forest \(global rarity\) and Local Outlier Factor \(local density\)\. For each query, the similarity\-score distribution is sharply concentrated near zero with a heavy right tail \(high kurtosis\), indicating that high\-similarity neighbors are sparse and highly selective\. Importantly, the spatial footprint of the top matches differs by query type: anomaly queries concentrate into a small number of compact, geographically coherent patches, whereas normal queries return broader, higher\-coverage regions consistent with membership in a dense manifold region\. Because ranking uses*only*embedding similarity \(coordinates are used solely for plotting\), the emergence of contiguous match regions supports the interpretation that the detected anomalies correspond to rare but repeatable, spatially structured signatures in the learned representation \(Fig\.[4](https://arxiv.org/html/2607.02990#S4.F4)\)\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/x4.png)\(a\)Isolation Forest: anomaly query\.
![Refer to caption](https://arxiv.org/html/2607.02990v1/x5.png)\(b\)Isolation Forest: normal query\.
![Refer to caption](https://arxiv.org/html/2607.02990v1/x6.png)\(c\)LOF: anomaly query\.
![Refer to caption](https://arxiv.org/html/2607.02990v1/x7.png)\(d\)LOF: normal query\.

Figure 4:Similarity\-search diagnostics for Arabian Shield graph embedding\-space outliers \(Experiment 2, Study 1\)\. For each query \(anomalous vs\. typical; selected by Isolation Forest or LOF\), nearest neighbors are retrieved by cosine similarity in the 256\-dimensional embedding space and visualized as \(left\) the spatial footprint of similarity scores and \(right\) the similarity\-score histogram\. In all cases the score distributions are strongly leptokurtic \(kurtosis≈45\.0\\approx 45\.0for IF anomaly,13\.813\.8for IF normal,8\.28\.2for LOF anomaly, and9\.49\.4for LOF normal\), with means close to zero and a heavy right tail, indicating that high\-similarity neighbors are sparse and highly selective\. Anomaly queries exhibit weaker near\-duplicate mass in the extreme\-high similarity range and concentrate into compact, spatially coherent patches, whereas normal queries show broader high\-similarity support consistent with dense manifold regions\. Coordinates are not used for retrieval, only for visualization, so spatial coherence reflects structure learned in the embedding space rather than an explicit spatial prior\.
##### Spatial Correlation and Characteristic Scale\.

A global spatial correlogram computed over random embedding pairs shows a clear decay of cosine similarity with physical distance, crossing zero at about 300 km and flattening at approximately 400–600 km \(Fig\.[5](https://arxiv.org/html/2607.02990#S4.F5)\)\. This provides an estimate of the characteristic correlation length of the learned representation, plausibly associated with regional geological structure\. Local radial profiles further differentiate anomalies from normal points: anomalous embeddings exhibit rapid decay to near\-orthogonality, while normal points retain weak correlation over longer distances \(Fig\.[6](https://arxiv.org/html/2607.02990#S4.F6)\)\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/x8.png)Figure 5:Global spatial correlogram of Arabian Shield graph embedding similarity versus geographic distance \(Experiment 2, Study 1\)\. Each blue point is the whitened cosine similarity of a sampled pair; the red curve shows the binned mean similarity, with shaded band indicating±1\\pm 1standard deviation\. Similarity decays rapidly with distance, crossing≈0\\approx 0near∼\\sim300 km and approaching a near\-flat regime by∼\\sim400–600 km, consistent with terrane\-scale continuity captured by the learned representation\.![Refer to caption](https://arxiv.org/html/2607.02990v1/x9.png)\(a\)Radial similarity decay from a LOF\-selected anomaly reference point\.
![Refer to caption](https://arxiv.org/html/2607.02990v1/x10.png)\(b\)Radial similarity decay from a LOF\-selected normal reference point\.

Figure 6:Local radial correlograms comparing an Arabian Shield graph embedding outlier versus a typical embedding \(Experiment 2, Study 1\)\. Blue points show whitened cosine similarity between the reference embedding and surrounding locations as a function of distance; the red curve is a moving\-average trend \(dashed line denotes zero similarity\)\. The anomaly decorrelates rapidly to near\-orthogonality with many near\-zero similarities, indicating a spatially localized signature, whereas the normal point retains weak positive correlation over longer ranges, consistent with membership in a broader regional embedding mode\.
##### Spectral Analysis: Scale, Texture, and Anisotropy\.

To quantify the spatial scales encoded by the embeddings, we analyze their frequency content in both 1D and 2D\. Along a latitudinal transect \(Lat≈20\.30∘\\approx 20\.30^\{\\circ\}\), the mean embedding power spectrum follows an approximate power law with slope close to1/f0\.751/f^\{0\.75\}over the resolvable band \(Fig\.[7](https://arxiv.org/html/2607.02990#S4.F7)\), consistent with multi\-scale geological organization rather than white noise or an overly smoothed field\. Because the transect sampling is sparse \(grid spacingΔ​x≈2\.05\\Delta x\\approx 2\.05km\), this result is band\-limited by the Nyquist wavelength \(λN≈2​Δ​x≈4\.1\\lambda\_\{N\}\\approx 2\\Delta x\\approx 4\.1km\) and should be interpreted as a regional\-scale signature\. A complementary 2D Fourier analysis over a dense local patch reveals structured texture in the latent RMS field and direction\-dependent spectral decay \(Fig\.[8](https://arxiv.org/html/2607.02990#S4.F8)\), indicating that the embeddings preserve both spatial texture and a weak\-to\-moderate anisotropic fabric aligned with structural trends\. Such approximate power\-law spectra are commonly observed in natural spatial fields; here we treat it as evidence of multi\-scale organization rather than a strict fractal model\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/x11.png)Figure 7:1D power spectral density \(PSD\) of the Arabian Shield 256\-dimensional graph embeddings along a latitudinal transect \(Lat≈20\.30∘\\approx 20\.30^\{\\circ\}; Experiment 2, Study 1\)\. The green curve shows mean power across embedding dimensions \(shaded:±1\\pm 1s\.d\.\), compared to a1/f0\.751/f^\{0\.75\}reference trend\. Over the resolved band the spectrum exhibits an approximately linear decay in log–log space, consistent with scale\-rich \(pink\-noise\-like\) structure rather than white noise\. The analysis is limited by the transect sampling \(Δ​x≈2\.05\\Delta x\\approx 2\.05km; Nyquist wavelengthλN≈4\.1\\lambda\_\{N\}\\approx 4\.1km\), so sub\-kilometre texture learned by the encoder is not directly observable here\.![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/fft_anisotropy_grid.png)Figure 8:2D spectral analysis of a dense local patch of Arabian Shield graph embeddings \(Experiment 2, Study 1\)\.Top\-left:approximate surface\-domain reference from satellite imagery\.Top\-right:latent RMS energy map \(aggregated embedding amplitude\), showing coherent spatial texture in feature space\.Bottom\-left:2D FFT power spectrum of the latent field, highlighting non\-uniform frequency content\.Bottom\-right:directional radial PSD profiles \(0∘, 45∘, 90∘, 135∘\) with a power\-law reference; systematic separation between directions indicates anisotropy \(directional fabric\) rather than isotropic texture, while the broad\-band decay indicates multi\-scale structure\.
##### Graph Spectral Structure and Distributed Representation\.

Finally, we analyze embedding smoothness using Dirichlet energy over a spatial nearest\-neighbor graph\. Sorting dimensions by energy reveals a near\-linear ramp \(Fig\.[9](https://arxiv.org/html/2607.02990#S4.F9)\), with no sharp separation between low\- and high\-frequency channels\. Even the smoothest dimensions exhibit substantial variation, while the highest\-energy dimensions encode fine\-scale texture\. This confirms a distributed, multi\-frequency representation rather than a small set of dominant components\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/x12.png)Figure 9:Dirichlet energy \(graph Laplacian quadratic form\) computed for each Arabian Shield graph embedding dimension on akk\-NN spatial graph, sorted from smoothest to roughest \(Experiment 2, Study 1\)\. The absence of a sharp elbow and the near\-linear ramp indicate that spatial frequency content is distributed across channels rather than concentrated in a small subset; even the lowest\-energy dimensions retain nontrivial spatial structure, while the highest\-energy dimensions capture localized texture\.
##### Qualitative Comparison via Unsupervised Segmentation\.

To qualitatively compare embedding clusters with visible surface domains, we apply K\-Means clustering \(k=3k=3\) to the embeddings within a dense local patch and compare the result to satellite imagery \(Fig\.[10](https://arxiv.org/html/2607.02990#S4.F10)\)\. The segmentation aligns closely with visible domains: a dark, rugged western belt consistent with mafic lithologies; a high\-albedo eastern domain consistent with felsic or sandy cover; and an intermediate unit concentrated near contacts\. Importantly, this spatial coherence emerges from the embedding vectors alone, with coordinates and imagery used only for visualization, suggesting that the embedding space captures surface\-domain structure\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/x13.png)Figure 10:Left: optical satellite imagery of the patch, used here as a qualitative surface reference \(Experiment 2, Study 1\)\. Right: unsupervised K\-Means clustering \(k=3k\{=\}3\) applied to the 256\-dimensional Arabian Shield graph embeddings for the same area\. The resulting segments are spatially coherent and align with major visible domains, including a dark, rugged western belt, a lighter high\-albedo eastern domain, and an intermediate/contact\-associated unit\. From an ML perspective, the agreement is consistent with improved*clusterability*of the learned representation: a simple centroid\-based method \(Euclidean Voronoi partition in embedding space\) recovers coherent, separable groups without supervision\. Clustering is performed purely in embedding space \(coordinates and imagery are used only for visualization\)\.
##### Summary\.

Across independent diagnostics, the Arabian Shield embeddings exhibit consistent, spatially coherent and qualitatively geologically plausible structure without using absolute/global coordinates or labels during training\. Individual dimensions form coherent spatial fields \(Fig\.[1](https://arxiv.org/html/2607.02990#S4.F1)\), while random\-pair similarities have near\-zero mean \(μ≈0\.054\\mu\\approx 0\.054\), consistent with no strong embedding collapse and with a heavy right tail reflecting clustered terrane\-scale domains \(Fig\.[2](https://arxiv.org/html/2607.02990#S4.F2)\)\. Nonlinear organization is evident in UMAP, which yields sharply bounded regional zones compared to a diffuse linear PCA baseline \(only∼\\sim17% variance in the first three PCs; Fig\.[3](https://arxiv.org/html/2607.02990#S4.F3)\)\. In a local patch, even a simple K\-Means partition \(k=3k\{=\}3\) recovers spatially coherent units visually aligned with visible surface domains \(Fig\.[10](https://arxiv.org/html/2607.02990#S4.F10)\)\. Outliers selected by IF/LOF are supported by selective, leptokurtic similarity distributions and geographically compact match footprints \(Fig\.[4](https://arxiv.org/html/2607.02990#S4.F4)\)\. Finally, similarity decays with distance at terrane scales \(zero\-crossing near∼\\sim300 km; Fig\.[5](https://arxiv.org/html/2607.02990#S4.F5)\) and spectral/graph analyses indicate multi\-scale texture with distributed spatial frequencies across embedding channels \(Figs\.[7](https://arxiv.org/html/2607.02990#S4.F7)–[9](https://arxiv.org/html/2607.02990#S4.F9)\)\.

#### 4\.2\.2Study 2: Prospect\-scale transfer via a dense embedding field over Suwaj

Study 1 characterized embedding structure at regional scale on the Arabian Shield\. Here we test whether the same pretrained encoder*transfers across scale*when evaluated densely over a focused area of interest \(AOI\) on a new dense evaluation grid, and whether the resulting embedding field supports reproducible, GIS\-consumable*unsupervised domain maps*and*novelty proxies*\(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)\)\.

Because the model is trained without explicit absolute/global geographic coordinates or geological labels, the key scientific question is whether the learned representation nevertheless induces a*spatially coherent latent field*at prospect scale\. Concretely, we test whether dense evaluation yields: \(i\) stable, contiguous domains rather than fragmented “salt\-and\-pepper” structure; \(ii\) sharp transitions localized in space \(consistent with boundaries in the underlying signals\); and \(iii\) controllable granularity under explicit analysis knobs \(e\.g\., varyingKKin K\-Means\) instead of unstable partitioning\.

##### Dense embedding grid \(100 m\)\.

We evaluate the encoder trained in Study 1 \(trained on random\-patch samples distributed across the Arabian Shield\) on a regular 2D grid over the Suwaj AOI at 100 m spacing\. The resulting field contains1,250,4141\{,\}250\{,\}414embedding vectors of dimension 256, defining a dense latent fieldE​\(x,y\)∈ℝ256E\(x,y\)\\in\\mathbb\{R\}^\{256\}over the AOI\. The AOI spans approximately 23\.26–24\.40∘N and 43\.55–44\.51∘E \(CRS: EPSG:4979\), with vertical extentz∈\[−10,000,1,372\]z\\in\[\-10\{,\}000,\\,1\{,\}372\]m\. Importantly, the model and preprocessing are held fixed; only the evaluation sampling density and spatial support change, making this a controlled test of representation stability under a new spatial regime\.

##### Derived layers as controlled probes of representation structure\.

To probe the organization of the dense embedding field, we derive several families of raster layers \(exported as Cloud\-Optimized GeoTIFFs for reproducible GIS analysis\)\. \(1\)Manifold projections\(PCA, UMAP\) and corresponding RGB composites provide qualitative visualization of neighborhood structure in embedding space \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)a–b\), while individual principal components provide interpretable scalar fields \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)e–f\)\. \(2\)Unsupervised zonationvia K\-Means acrossK∈\{4,8,16,32,64\}K\\in\\\{4,8,16,32,64\\\}supplies an explicit control knob on domain granularity; categorical maps at fixedKKsummarize domain partitions \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)c\)\. \(3\)Novelty / boundary proxiesinclude distance\-to\-centroid under K\-Means \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)d\) and multi\-scale anomaly scores \(Isolation Forest pyramid; Figure[11](https://arxiv.org/html/2607.02990#S4.F11)g\) that emphasize signatures that are rare in embedding space and/or concentrated near transitions\. Finally, texture proxies such as gray\-level co\-occurrence matrix \(GLCM\) statistics computed on selected components provide an additional controlled view of spatial heterogeneity \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)h\)\.

##### Qualitative evidence of coherent domains and multi\-scale structure\.

The Suwaj rasters indicate that the embedding\-derived layers are not spatially random: both nonlinear \(UMAP\) and linear \(PCA\) composites form contiguous regions separated by sharp boundaries \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)a–b\)\. Consistent with the strongly non\-linear geometry observed at regional scale \(Study 1\), the UMAP composite tends to emphasize discrete, sharply bounded domains, while PCA captures smoother large\-scale variation\. K\-Means zonation exhibits the expected coarse\-to\-fine behavior asKKincreases: lowKKrecovers macro\-domains, whereas higherKKrefines these into more detailed subdomains \(example atK=8K\{=\}8in Figure[11](https://arxiv.org/html/2607.02990#S4.F11)c\)\. The distance\-to\-centroid layers provide a complementary continuous view, highlighting mixed/transition zones and locally atypical signatures that are difficult to capture with categorical maps alone \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)d\)\. The anomaly and texture proxies further concentrate on spatially localized deviations and heterogeneous structure \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)g–h\), serving as hypothesis\-generating layers for downstream validation rather than geological labels\.

##### Summary\.

This study demonstrates a scale\-transfer property of the learned representation: a regional pretrained encoder, trained without absolute/global coordinate input, can be evaluated densely on a 2D grid at 100 m spacing to yield a spatially coherent embedding field over Suwaj \(Figure[11](https://arxiv.org/html/2607.02990#S4.F11)\)\. While these layers are not geological labels, their consistency across projection methods, the controllable granularity viaKK, and the agreement between categorical \(zonation\) and continuous \(distance/anomaly/texture\) probes provide evidence that the embedding space captures structured, multi\-scale variation suitable for downstream interpretation and targeting\.

![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/umap_rgb_suwaj.png)\(a\)UMAP RGB
![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/pca_rgb_suwaj.png)\(b\)PCA RGB
![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/kmeans_label_8_suwaj.png)\(c\)KK\-Means labels \(K=8K\{=\}8\)
![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/kmeans_dist_8_suwaj.png)\(d\)Dist\. to centroid \(K=8K\{=\}8\)
![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/pc1_suwaj.png)\(e\)PC1
![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/pc2_suwaj.png)\(f\)PC2
![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/anomaly_f8_suwaj.png)\(g\)Anomaly \(f=8f\{=\}8\)
![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/pc1_glcm_contrast_w11_suwaj.png)\(h\)PC1 GLCM contrast

Figure 11:Dense graph embedding\-derived layers over the Suwaj AOI evaluated on a 2D regular grid at 100 m spacing \(Experiment 2, Study 2\)\. Top row: RGB composites of the embedding manifold under UMAP and PCA, andKK\-Means zonation withK=8K\{=\}8with the corresponding continuous distance\-to\-centroid proxy \(highlighting mixed/transition areas and locally atypical signatures\)\. Bottom row: principal component rasters \(PC1–PC2\), a multi\-scale Isolation Forest anomaly score at window factorf=8f\{=\}8\(the displayed anomaly\-pyramid scale\), and a texture probe computed from PC1 \(GLCM contrast, windoww=11w\{=\}11\), emphasizing local heterogeneity and edge/structural texture\.

## 5Conclusion

We presentedMABLE, a self\-supervised representation learning pipeline for large, heterogeneous geospatial graphs that encourages robust*node*and*graph*\-level embeddings using a tightly integrated set of objectives and stability constraints\. The method couples masked node reconstruction with representation\-shaping losses that explicitly encourage useful geometry: \(i\) node correspondence and node uniformity to promote cross\-view consistency while discouraging collapse and improving feature diversity, \(ii\) graph alignment to enforce invariance across augmented views \(including subgraph views\), and \(iii\) graph uniformity to maintain global spread across graphs within a batch\. On the architectural side, we combine a modality\-agnostic token interface with a Lipschitz\-controlled pooling operator and a well\-conditioned \(bi\-Lipschitz\) reconstruction head, linking practical stability to the theoretical sensitivity bounds established for the reconstruction slice and pooling components discussed throughout the paper\.

Across the mineral\-prospectivity experiments, MABLE embeddings are most useful as complementary contextual representations rather than as standalone predictors under strong geographic distribution shift\. On the local\-scale copper benchmark \(redacted Experiment 1\), embeddings alone are not sufficient to extrapolate into a geographically distinct holdout region, but compact embedding components improve coordinate\-only and engineered geophysical baselines, indicating transferable context not fully summarized by local MAG/ANT statistics\. On the regional\-scale Arabian Shield corpus, the learned 256\-dimensional graph embeddings exhibit strong spatial coherence without using absolute/global coordinates or geological labels during training: manifold projections, similarity search, anomaly detection, and graph/spectral diagnostics consistently indicate a distributed, multi\-scale representation that organizes the Shield into stable domains and highlights rare signatures\. Finally, a dense 100 m grid evaluation over Suwaj demonstrates*scale transfer*: a regional pretrained encoder can be evaluated prospect\-scale to produce GIS\-consumable, embedding\-derived layers \(zonation, novelty proxies, and texture probes\) that are spatially coherent and controllably granular\. These regional and prospect\-scale products should be interpreted as hypothesis\-generating latent layers rather than geological labels\.

A core architectural differentiator explored in this work is the*feature\-embedding slice*for reconstruction with a constrained decoder, together with a complementary contextual slice regularized by cross\-view node correspondence\. Ablations indicate that this architectural choice, when properly tuned, improves training dynamics and reduces overfitting pressure\. These results support the practical use of the slice\+bi\-Lipschitz decoder together with contextual cross\-view consistency as stability\-oriented design choices that need not materially sacrifice predictive utility when appropriately parameterized\.

##### Limitations and future work\.

Several directions remain open\. First, while embeddings can complement engineered context under spatial distribution shift, stronger downstream models \(e\.g\., residual or hierarchical predictors, or end\-to\-end fine\-tuning\) may better exploit the learned context in the strictest extrapolation regimes\. Second, the current construction uses edge\-free graphs; incorporating lightweight, uncertainty\-aware relational structure \(e\.g\., learned sparse edges or physics\-informed neighborhoods\) is a natural extension when explicit connectivity is meaningful\. Third, prospect\-scale products derived from embeddings \(zonation, anomalies, textures\) should be validated more directly against mapped geology and known mineral systems to quantify correspondence beyond qualitative alignment\. More broadly, evaluating transfer across regions, sensor stacks, sampling densities and characterizing failure modes under missing modalities or strong coverage gaps will further clarify the operational envelope of MABLE for exploration workflows\.

Overall, MABLE provides a principled, end\-to\-end approach to learning stable geospatial graph representations that \(i\) remain well\-conditioned under augmentation and sampling artifacts, \(ii\) support both node\-level reconstruction fidelity and cross\-view node consistency together with global embedding robustness, and \(iii\) yield coherent, multi\-scale latent structure that can be operationalized as GIS\-ready layers for interpretation and targeting in the settings studied here\.

## References

- B\. Alkin, L\. Miklautz, S\. Hochreiter, and J\. Brandstetter \(2025\)MIM\-Refiner: a contrastive learning boost from intermediate pre\-trained representations\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=F01HbewbwU)Cited by:[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.SSS0.Px1.p1.1),[§3\.5](https://arxiv.org/html/2607.02990#S3.SS5.p2.1)\.
- J\. Baek, M\. Kang, and S\. J\. Hwang \(2021\)Accurate learning of graph representations with graph multiset pooling\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=JHcqXGaqiGn)Cited by:[§2\.4](https://arxiv.org/html/2607.02990#S2.SS4.p1.7)\.
- J\. Behrmann, W\. Grathwohl, R\. T\. Q\. Chen, D\. Duvenaud, and J\. Jacobsen \(2019\)Invertible residual networks\.InProceedings of the 36th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.97,Long Beach, USA,pp\. 573–582\.External Links:[Link](https://proceedings.mlr.press/v97/behrmann19a.html)Cited by:[§2\.5](https://arxiv.org/html/2607.02990#S2.SS5.p1.12)\.
- T\. Chen, S\. Kornblith, M\. Norouzi, and G\. Hinton \(2020\)A simple framework for contrastive learning of visual representations\.InProceedings of the 37th International Conference on Machine Learning,ICML’20,pp\. 1597–1607\.External Links:[Link](https://proceedings.mlr.press/v119/chen20j.html)Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1)\.
- M\. Cisse, P\. Bojanowski, É\. Grave, Y\. Dauphin, and N\. Usunier \(2017\)Parseval networks: improving robustness to adversarial examples\.InProceedings of the 34th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.70,Sydney, Australia,pp\. 854–863\.External Links:[Link](https://proceedings.mlr.press/v70/cisse17a.html)Cited by:[§2\.5](https://arxiv.org/html/2607.02990#S2.SS5.p1.12),[§2\.6](https://arxiv.org/html/2607.02990#S2.SS6.p1.1)\.
- A\. Daruna, V\. Zadorozhnyy, G\. Lukoczki, and H\. Chiu \(2024\)GFM4MPM: towards geospatial foundation models for mineral prospectivity mapping\.InProceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems,SIGSPATIAL ’24,New York, NY, USA,pp\. 565–568\.External Links:[Link](https://doi.org/10.1145/3678717.3691268),[Document](https://dx.doi.org/10.1145/3678717.3691268)Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1),[§4\.1](https://arxiv.org/html/2607.02990#S4.SS1.SSS0.Px2.p1.1)\.
- I\. Gulrajani, F\. Ahmed, M\. Arjovsky, V\. Dumoulin, and A\. Courville \(2017\)Improved training of wasserstein GANs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,Long Beach, USA,pp\. 5767–5777\.External Links:[Link](https://papers.nips.cc/paper/2017/hash/892c3b60920f561af15af1b7f4ab8af4-Abstract.html)Cited by:[§2\.5](https://arxiv.org/html/2607.02990#S2.SS5.p1.12)\.
- M\. Gutmann and A\. Hyvärinen \(2010\)Noise‑contrastive estimation: a new estimation principle for unnormalized statistical models\.InProceedings of the 13th International Conference on Artificial Intelligence and Statistics \(AISTATS\),Proceedings of Machine Learning Research, Vol\.9,Chia Laguna Resort, Sardinia, Italy,pp\. 297–304\.External Links:[Link](https://proceedings.mlr.press/v9/gutmann10a.html)Cited by:[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.p1.6)\.
- J\. Z\. HaoChen, C\. Wei, A\. Gaidon, and T\. Ma \(2021\)Provable guarantees for self\-supervised deep learning with spectral contrastive loss\.InAdvances in Neural Information Processing Systems,M\. Ranzato, A\. Beygelzimer, Y\. Dauphin, P\.S\. Liang, and J\. W\. Vaughan \(Eds\.\),Vol\.34,pp\. 5000–5011\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2021/file/27debb435021eb68b3965290b5e24c49-Paper.pdf)Cited by:[§A\.3](https://arxiv.org/html/2607.02990#A1.SS3.p1.1),[§1](https://arxiv.org/html/2607.02990#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.p2.1),[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.p2.2)\.
- K\. He, X\. Chen, S\. Xie, Y\. Li, P\. Dollár, and R\. B\. Girshick \(2022\)Masked autoencoders are scalable vision learners\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 15979–15988\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52688.2022.01551),[Link](https://doi.org/10.1109/CVPR52688.2022.01551)Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.02990#S2.SS1.SSS0.Px5.p1.8)\.
- K\. He, H\. Fan, Y\. Wu, S\. Xie, and R\. Girshick \(2019\)Momentum contrast for unsupervised visual representation learning\.CoRRabs/1911\.05722\.Note:CVPR 2020 camera\-readyExternal Links:1911\.05722,[Link](https://arxiv.org/abs/1911.05722)Cited by:[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.p1.6)\.
- R\. D\. Hjelm, A\. Fedorov, S\. Lavoie\-Marchildon, K\. Grewal, P\. Bachman, A\. Trischler, and Y\. Bengio \(2018\)Learning deep representations by mutual information estimation and maximization\.CoRRabs/1808\.06670\.External Links:1808\.06670,[Link](https://arxiv.org/abs/1808.06670)Cited by:[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.p1.6),[§2](https://arxiv.org/html/2607.02990#S2.p2.1),[§3\.2](https://arxiv.org/html/2607.02990#S3.SS2.p3.1)\.
- Z\. Hou, F\. Liu, Y\. Dong, C\. Huang, M\. Ju, Z\. Zhang, Y\. Qiao, and Y\. Lin \(2022\)GraphMAE: self\-supervised masked graph autoencoders\.InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 594–604\.External Links:[Document](https://dx.doi.org/10.1145/3534678.3539321),[Link](https://doi.org/10.1145/3534678.3539321)Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.02990#S2.SS1.SSS0.Px5.p1.8)\.
- Z\. Hou, X\. Zhang, L\. Wang, Z\. Zhu, Y\. Dong, M\. Ju, F\. Liu, Y\. Yang, and C\. Shi \(2023\)GraphMAE2: a decoding\-enhanced masked self\-supervised graph learner\.Proceedings of the ACM Web Conference 2023 Companion,pp\. 737–746\.External Links:[Document](https://dx.doi.org/10.1145/3543873.3587651),[Link](https://doi.org/10.1145/3543873.3587651)Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.02990#S2.SS1.SSS0.Px5.p1.8)\.
- N\. Kitaev, Ł\. Kaiser, and A\. Levskaya \(2020\)Reformer: the efficient transformer\.Note:arXiv preprint arXiv:2001\.04451, ICLR 2020External Links:[Link](http://arxiv.org/abs/2001.04451)Cited by:[§4\.1](https://arxiv.org/html/2607.02990#S4.SS1.SSS0.Px1.p1.1)\.
- J\. Lee, Y\. Lee, J\. Kim, A\. R\. Kosiorek, S\. Choi, and Y\. W\. Teh \(2019\)Set transformer: a framework for attention\-based permutation\-invariant neural networks\.InProceedings of the 36th International Conference on Machine Learning,pp\. 3744–3753\.External Links:[Link](https://proceedings.mlr.press/v97/lee19d.html)Cited by:[§2\.4](https://arxiv.org/html/2607.02990#S2.SS4.p1.7)\.
- J\. Li, R\. Wu, W\. Sun, L\. Chen, S\. Tian, L\. Zhu, C\. Meng, Z\. Zheng, and W\. Wang \(2023\)What’s behind the mask: understanding masked graph modeling for graph autoencoders\.InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,KDD ’23,New York, NY, USA,pp\. 1268–1279\.External Links:[Document](https://dx.doi.org/10.1145/3580305.3599546),[Link](https://doi.org/10.1145/3580305.3599546),ISBN 979\-8\-4007\-0103\-0Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1)\.
- T\. Miyato, T\. Kataoka, M\. Koyama, and Y\. Yoshida \(2018\)Spectral normalization for generative adversarial networks\.InInternational Conference on Learning Representations \(ICLR\),Vancouver, Canada\.External Links:[Link](https://openreview.net/forum?id=B1QRgziT-)Cited by:[§2\.5](https://arxiv.org/html/2607.02990#S2.SS5.p1.12),[§2\.5](https://arxiv.org/html/2607.02990#S2.SS5.p2.10),[§2\.6](https://arxiv.org/html/2607.02990#S2.SS6.p1.1)\.
- F\. M\. H\. Sihombing, R\. M\. Palin, H\. S\. R\. Hughes, and L\. J\. Robb \(2024\)Improved mineral prospectivity mapping using graph neural networks\.Ore Geology Reviews172,pp\. 106215\.External Links:[Document](https://dx.doi.org/10.1016/j.oregeorev.2024.106215),ISSN 0169\-1368Cited by:[§4\.1](https://arxiv.org/html/2607.02990#S4.SS1.SSS0.Px2.p1.1)\.
- S\. Thakoor, C\. Tallec, M\. G\. Azar, and P\. Veličković \(2022\)Bootstrapped representation learning on graphs\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=QrzVRAA49Ud)Cited by:[§2](https://arxiv.org/html/2607.02990#S2.p2.1),[§3\.1](https://arxiv.org/html/2607.02990#S3.SS1.SSS0.Px1.p1.5)\.
- C\. Tosh, A\. Krishnamurthy, and D\. J\. Hsu \(2020\)Contrastive learning, multi\-view redundancy, and linear models\.CoRRabs/2008\.10150\.External Links:2008\.10150,[Link](https://arxiv.org/abs/2008.10150)Cited by:[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.SSS0.Px1.p1.1)\.
- A\. van den Oord, Y\. Li, and O\. Vinyals \(2018\)Representation learning with Contrastive Predictive Coding\.CoRRabs/1807\.03748\.External Links:[Link](http://arxiv.org/abs/1807.03748),1807\.03748Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1),[§1](https://arxiv.org/html/2607.02990#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.p1.6)\.
- P\. Veličković, W\. Fedus, W\. L\. Hamilton, P\. Liò, Y\. Bengio, and R\. D\. Hjelm \(2019\)Deep graph infomax\.InInternational Conference on Learning Representations \(ICLR\),New Orleans, USA\.External Links:[Link](https://openreview.net/forum?id=rklz9iAcKQ)Cited by:[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.p1.6),[§2](https://arxiv.org/html/2607.02990#S2.p2.1),[§3\.2](https://arxiv.org/html/2607.02990#S3.SS2.p3.1)\.
- T\. Wang and P\. Isola \(2020\)Understanding contrastive representation learning through alignment and uniformity on the hypersphere\.InProceedings of the 37th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.119,Virtual Conference,pp\. 9929–9939\.External Links:[Link](https://proceedings.mlr.press/v119/wang20k.html)Cited by:[§2\.2](https://arxiv.org/html/2607.02990#S2.SS2.SSS0.Px1.p1.1)\.
- Y\. Wang, D\. M\. Blei, and J\. P\. Cunningham \(2021\)Posterior collapse and latent variable non\-identifiability\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.34,Virtual Conference,pp\. 5443–5455\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2021/file/2b6921f2c64dee16ba21ebf17f3c2c92-Paper.pdf)Cited by:[§2\.1](https://arxiv.org/html/2607.02990#S2.SS1.SSS0.Px5.p2.9)\.
- N\. Wu, Q\. Cao, Z\. Wang, Z\. Liu, Y\. Qi, J\. Zhang, J\. Ni, X\. Yao, H\. Ma, L\. Mu, S\. Ermon, T\. Ganu, A\. Nambi, N\. Lao, and G\. Mai \(2024\)TorchSpatial: a location encoding framework and benchmark for spatial representation learning\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 81437–81460\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/9449c2d5b0cc8c9a445752f3ff195a1c-Paper-Datasets_and_Benchmarks_Track.pdf)Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p2.1)\.
- K\. Xu, W\. Hu, J\. Leskovec, and S\. Jegelka \(2019\)How powerful are graph neural networks?\.InProceedings of the 7th International Conference on Learning Representations,ICLR ’19,New Orleans, LA, USA\.External Links:[Link](https://openreview.net/forum?id=ryGs6iA5Km)Cited by:[§2\.7](https://arxiv.org/html/2607.02990#S2.SS7.p1.3)\.
- Y\. You, T\. Chen, Y\. Sui, T\. Chen, Z\. Wang, and Y\. Shen \(2020\)Graph contrastive learning with augmentations\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 5812–5823\.External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/3fe230348e9a12c13120749e3f9fa4cd-Abstract.html)Cited by:[§2](https://arxiv.org/html/2607.02990#S2.p2.1)\.
- M\. Zaheer, S\. Kottur, S\. Ravanbakhsh, B\. Póczos, R\. Salakhutdinov, and A\. J\. Smola \(2017\)Deep sets\.InAdvances in Neural Information Processing Systems,Vol\.30\.External Links:[Link](https://proceedings.neurips.cc/paper/2017/hash/f22e4747da1aa27e363d86d40ff442fe-Abstract.html)Cited by:[§2\.1](https://arxiv.org/html/2607.02990#S2.SS1.SSS0.Px1.p4.3)\.
- Q\. Zhang, Y\. Wang, and Y\. Wang \(2022\)How mask matters: towards theoretical understandings of masked autoencoders\.InProceedings of the 36th International Conference on Neural Information Processing Systems,NeurIPS ’22,Red Hook, NY, USA\.External Links:ISBN 9781713871088Cited by:[§1](https://arxiv.org/html/2607.02990#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.02990#S2.SS1.SSS0.Px5.p1.8),[§2\.1](https://arxiv.org/html/2607.02990#S2.SS1.SSS0.Px6.p2.1),[§3\.3](https://arxiv.org/html/2607.02990#S3.SS3.p1.17)\.
- Y\. Zhu, Y\. Xu, F\. Yu, Q\. Liu, S\. Wu, and L\. Wang \(2020\)Deep graph contrastive representation learning\.InICML Workshop on Graph Representation Learning and Beyond,External Links:[Link](https://arxiv.org/abs/2006.04131)Cited by:[§2](https://arxiv.org/html/2607.02990#S2.p2.1)\.
- R\. Zuo, Y\. Xiong, Z\. Wang, J\. Wang, and O\. P\. Kreuzer \(2023\)A new generation of artificial intelligence algorithms for mineral prospectivity mapping\.Natural Resources Research32\(5\),pp\. 1859–1869\.External Links:[Document](https://dx.doi.org/10.1007/s11053-023-10237-w),[Link](https://doi.org/10.1007/s11053-023-10237-w),ISSN 1573\-8981Cited by:[§4\.1](https://arxiv.org/html/2607.02990#S4.SS1.SSS0.Px2.p1.1)\.

## Appendix AAppendix

### A\.1Proof of Proposition[1](https://arxiv.org/html/2607.02990#Thmproposition1)\- Node Feature Similarity Preservation\.

###### Proof\.

Recallh​\(u\)=g​\(eu\)h\(u\)=g\(e\_\{u\}\)witheu=π​\(f​\(u\)\)e\_\{u\}=\\pi\(f\(u\)\), and assumeggis bi\-Lipschitz: for alle1,e2e\_\{1\},e\_\{2\},

m​‖e1−e2‖≤‖g​\(e1\)−g​\(e2\)‖≤L​‖e1−e2‖\.m\\\|e\_\{1\}\-e\_\{2\}\\\|\\leq\\\|g\(e\_\{1\}\)\-g\(e\_\{2\}\)\\\|\\leq L\\\|e\_\{1\}\-e\_\{2\}\\\|\.In particular, foreu,eve\_\{u\},e\_\{v\}we have

m​‖eu−ev‖≤‖h​\(u\)−h​\(v\)‖≤L​‖eu−ev‖\.m\\\|e\_\{u\}\-e\_\{v\}\\\|\\leq\\\|h\(u\)\-h\(v\)\\\|\\leq L\\\|e\_\{u\}\-e\_\{v\}\\\|\.\(1\)By the reconstruction assumption,‖h​\(u\)−xu‖≤ε\\\|h\(u\)\-x\_\{u\}\\\|\\leq\\varepsilonand‖h​\(v\)−xv‖≤ε\\\|h\(v\)\-x\_\{v\}\\\|\\leq\\varepsilon\.

Upper bound on‖eu−ev‖\\\|e\_\{u\}\-e\_\{v\}\\\|\.By the triangle inequality,

‖h​\(u\)−h​\(v\)‖≤‖h​\(u\)−xu‖\+‖xu−xv‖\+‖xv−h​\(v\)‖≤‖xu−xv‖\+2​ε\.\\\|h\(u\)\-h\(v\)\\\|\\leq\\\|h\(u\)\-x\_\{u\}\\\|\+\\\|x\_\{u\}\-x\_\{v\}\\\|\+\\\|x\_\{v\}\-h\(v\)\\\|\\leq\\\|x\_\{u\}\-x\_\{v\}\\\|\+2\\varepsilon\.Combining with the left inequality in \([1](https://arxiv.org/html/2607.02990#A1.E1)\) gives

m​‖eu−ev‖≤‖xu−xv‖\+2​ε⟹‖eu−ev‖≤1m​‖xu−xv‖\+2​εm\.m\\\|e\_\{u\}\-e\_\{v\}\\\|\\leq\\\|x\_\{u\}\-x\_\{v\}\\\|\+2\\varepsilon\\quad\\Longrightarrow\\quad\\\|e\_\{u\}\-e\_\{v\}\\\|\\leq\\frac\{1\}\{m\}\\\|x\_\{u\}\-x\_\{v\}\\\|\+\\frac\{2\\varepsilon\}\{m\}\.
Lower bound on‖eu−ev‖\\\|e\_\{u\}\-e\_\{v\}\\\|\.Using the reverse triangle inequality \(equivalently, a two\-term triangle bound\),

‖h​\(u\)−h​\(v\)‖≥‖xu−xv‖−‖h​\(u\)−xu‖−‖h​\(v\)−xv‖≥‖xu−xv‖−2​ε\.\\\|h\(u\)\-h\(v\)\\\|\\geq\\\|x\_\{u\}\-x\_\{v\}\\\|\-\\\|h\(u\)\-x\_\{u\}\\\|\-\\\|h\(v\)\-x\_\{v\}\\\|\\geq\\\|x\_\{u\}\-x\_\{v\}\\\|\-2\\varepsilon\.Combining with the right inequality in \([1](https://arxiv.org/html/2607.02990#A1.E1)\) yields

L​‖eu−ev‖≥‖xu−xv‖−2​ε⟹‖eu−ev‖≥‖xu−xv‖−2​εL\.L\\\|e\_\{u\}\-e\_\{v\}\\\|\\geq\\\|x\_\{u\}\-x\_\{v\}\\\|\-2\\varepsilon\\quad\\Longrightarrow\\quad\\\|e\_\{u\}\-e\_\{v\}\\\|\\geq\\frac\{\\\|x\_\{u\}\-x\_\{v\}\\\|\-2\\varepsilon\}\{L\}\.Since norms are nonnegative, one can equivalently write‖eu−ev‖≥max⁡\{0,‖xu−xv‖−2​εL\}\\\|e\_\{u\}\-e\_\{v\}\\\|\\geq\\max\\\!\\left\\\{0,\\frac\{\\\|x\_\{u\}\-x\_\{v\}\\\|\-2\\varepsilon\}\{L\}\\right\\\}\.

The same argument applies to anyℓp\\ell\_\{p\}norm by replacing∥⋅∥\\\|\\cdot\\\|with∥⋅∥p\\\|\\cdot\\\|\_\{p\}and takingmmandLLto be the corresponding co\-Lipschitz/Lipschitz constants forggunderℓp\\ell\_\{p\}\. ∎

### A\.2Relationship Between the Log\-Based Surrogate and Normalized SCL

This appendix details the local relationship between the proposed log\-based surrogate and the normalized/cosine form of Spectral Contrastive Loss \(SCL\)\. Assume embeddings are compared by cosine similarity, equivalently by inner product after unit normalization, socos⁡\(z,z′\)∈\[−1,1\]\\cos\(z,z^\{\\prime\}\)\\in\[\-1,1\]\.

##### Log\-Based Surrogate and Approximation by SCL\.

We define a log\-based surrogate loss designed to explicitly target positive alignment \(cos⁡\(z,z\+\)=1\\cos\(z,z^\{\+\}\)=1\) and negative orthogonality \(cos⁡\(z,z−\)=0\\cos\(z,z^\{\-\}\)=0\)\. The loss takes the form:

ℒSurrogate=−𝔼\(z,z\+\)​\[log⁡\(PositiveArg\)\]−𝔼\(z,z−\)​\[log⁡\(NegativeArg\)\]\.\\mathcal\{L\}\_\{\\text\{Surrogate\}\}=\-\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\[\\log\(\\text\{PositiveArg\}\)\]\-\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\[\\log\(\\text\{NegativeArg\}\)\]\.To ensure the arguments are in\(0,1\]\(0,1\]and achieve the value 1 at the desired optimal cosine similarities, we choose:

PositiveArg=exp\(cos\(z,z\+\)−1\)andNegativeArg=1−12cos\(z,z−\)2\.\\text\{PositiveArg\}=\\exp\(\\cos\(z,z^\{\+\}\)\-1\)\\quad\\text\{and\}\\quad\\text\{NegativeArg\}=1\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\.The log\-based surrogate is then:

ℒSurrogate=−𝔼\(z,z\+\)\[log\(exp\(cos\(z,z\+\)−1\)\)\]−𝔼\(z,z−\)\[log\(1−12cos\(z,z−\)2\)\]\.\\mathcal\{L\}\_\{\\text\{Surrogate\}\}=\-\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\[\\log\(\\exp\(\\cos\(z,z^\{\+\}\)\-1\)\)\]\-\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\[\\log\(1\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\)\]\.Simplifying the positive term:

ℒSurrogate=−𝔼\(z,z\+\)\[cos\(z,z\+\)−1\]−𝔼\(z,z−\)\[log\(1−12cos\(z,z−\)2\)\]\.\\mathcal\{L\}\_\{\\text\{Surrogate\}\}=\-\\mathbb\{E\}\_\{\(z,z^\{\+\}\)\}\[\\cos\(z,z^\{\+\}\)\-1\]\-\\mathbb\{E\}\_\{\(z,z^\{\-\}\)\}\[\\log\(1\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\)\]\.Dropping the constant−1\-1from the positive term \(which does not affect minimization\):

ℒSurrogate′=−𝔼\[cos\(z,z\+\)\]−𝔼\[log\(1−12cos\(z,z−\)2\)\]\.\\mathcal\{L\}\_\{\\text\{Surrogate\}\}^\{\\prime\}=\-\\mathbb\{E\}\[\\cos\(z,z^\{\+\}\)\]\-\\mathbb\{E\}\[\\log\(1\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\)\]\.Lett\+=cos⁡\(z,z\+\)t^\{\+\}=\\cos\(z,z^\{\+\}\)andt−=cos⁡\(z,z−\)t^\{\-\}=\\cos\(z,z^\{\-\}\)and12​ℒSCL=−𝔼​\[t\+\]\+12​𝔼​\[\(t−\)2\]\\tfrac\{1\}\{2\}\\,\\mathcal\{L\}\_\{\\text\{SCL\}\}=\-\\mathbb\{E\}\[t^\{\+\}\]\+\\tfrac\{1\}\{2\}\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\. The positive term ofℒSurrogate′\\mathcal\{L\}\_\{\\text\{Surrogate\}\}^\{\\prime\}is exactly the positive term of12​ℒSCL\\tfrac\{1\}\{2\}\\,\\mathcal\{L\}\_\{\\text\{SCL\}\}\.

We show that the negative term ofℒSurrogate′\\mathcal\{L\}\_\{\\text\{Surrogate\}\}^\{\\prime\}is approximated by the negative term of12​ℒSCL\\tfrac\{1\}\{2\}\\,\\mathcal\{L\}\_\{\\text\{SCL\}\}around the desired negative optimumt−=0t^\{\-\}=0\. We perform a Taylor expansion of−log⁡\(1−12​t2\)\-\\log\(1\-\\frac\{1\}\{2\}t^\{2\}\)aroundt=0t=0: The Taylor expansion of−log⁡\(1−x\)\-\\log\(1\-x\)aroundx=0x=0isx\+x22\+x33\+…x\+\\frac\{x^\{2\}\}\{2\}\+\\frac\{x^\{3\}\}\{3\}\+\\dotsfor\|x\|<1\|x\|<1\. Substitutex=12​t2x=\\frac\{1\}\{2\}t^\{2\}\. Fort∈\[−1,1\]t\\in\[\-1,1\],t2∈\[0,1\]t^\{2\}\\in\[0,1\], so12​t2∈\[0,1/2\]\\frac\{1\}\{2\}t^\{2\}\\in\[0,1/2\], which means\|x\|<1\|x\|<1is satisfied\.

−log⁡\(1−12​t2\)=\(12​t2\)\+\(12​t2\)22\+\(12​t2\)33\+⋯=12​t2\+18​t4\+124​t6\+…\-\\log\(1\-\\frac\{1\}\{2\}t^\{2\}\)=\(\\frac\{1\}\{2\}t^\{2\}\)\+\\frac\{\(\\frac\{1\}\{2\}t^\{2\}\)^\{2\}\}\{2\}\+\\frac\{\(\\frac\{1\}\{2\}t^\{2\}\)^\{3\}\}\{3\}\+\\dots=\\frac\{1\}\{2\}t^\{2\}\+\\frac\{1\}\{8\}t^\{4\}\+\\frac\{1\}\{24\}t^\{6\}\+\\dotsSo, the expectation of the negative term ofℒSurrogate′\\mathcal\{L\}\_\{\\text\{Surrogate\}\}^\{\\prime\}is:

−𝔼\[log\(1−12cos\(z,z−\)2\)\]=𝔼\[12cos\(z,z−\)2\+18cos\(z,z−\)4\+124cos\(z,z−\)6\+…\]\.\-\\mathbb\{E\}\[\\log\(1\-\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\)\]=\\mathbb\{E\}\[\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\+\\frac\{1\}\{8\}\\cos\(z,z^\{\-\}\)^\{4\}\+\\frac\{1\}\{24\}\\cos\(z,z^\{\-\}\)^\{6\}\+\\dots\]\.Taking only the leading term from this expansion aroundcos⁡\(z,z−\)=0\\cos\(z,z^\{\-\}\)=0, we get𝔼\[12cos\(z,z−\)2\]\\mathbb\{E\}\[\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\]\. Thus, the negative term ofℒSurrogate′\\mathcal\{L\}\_\{\\text\{Surrogate\}\}^\{\\prime\}is approximated by the negative term of SCL aroundcos⁡\(z,z−\)=0\\cos\(z,z^\{\-\}\)=0\.

Combining the exact positive term and the approximated negative term, we see thatℒSurrogate′\\mathcal\{L\}\_\{\\text\{Surrogate\}\}^\{\\prime\}is approximated by scaled normalized SCL around the desired cosine geometry \(cos⁡\(z,z\+\)=1,cos⁡\(z,z−\)=0\\cos\(z,z^\{\+\}\)=1,\\cos\(z,z^\{\-\}\)=0\):

ℒSurrogate′≈−𝔼\[cos\(z,z\+\)\]\+𝔼\[12cos\(z,z−\)2\]=12ℒSCL\.\\mathcal\{L\}\_\{\\text\{Surrogate\}\}^\{\\prime\}\\approx\-\\mathbb\{E\}\[\\cos\(z,z^\{\+\}\)\]\+\\mathbb\{E\}\\\!\\left\[\\frac\{1\}\{2\}\\cos\(z,z^\{\-\}\)^\{2\}\\right\]=\\tfrac\{1\}\{2\}\\,\\mathcal\{L\}\_\{\\text\{SCL\}\}\.This demonstrates that normalized SCL is a computationally simpler form that captures the leading behavior of a principled log\-based surrogate near the desired cosine geometry\.

##### Conclusion\.

Through the construction of a log\-based surrogate whose terms explicitly target the desired cosine geometry, we obtain normalized SCL as the leading local quadratic approximation: positive pairs are driven toward angular alignment and unpaired embeddings toward angular decorrelation\. This is the fixed\-cosine geometry used by MABLE without requiring a learned discriminator\.

### A\.3A DV\-Style Interpretation of the Normalized/Cosine SCL Surrogate

This appendix concerns the normalized/cosine SCL\-style surrogate used in MABLE, not unconstrained raw inner\-product SCL except when embeddings are unit\-normalized or otherwise tightly norm\-controlled\. The original spectral contrastive loss ofHaoChenet al\.\[[2021](https://arxiv.org/html/2607.02990#bib.bib9)\]is written using raw inner products, whereas MABLE uses cosine similarity to remove scale ambiguity and keep scores bounded in\[−1,1\]\[\-1,1\]\. The DV\-style argument below should therefore be read as a heuristic justification for the normalized/cosine surrogate used here\.

Mutual information,I​\(Z,Z\+\)I\(Z,Z^\{\+\}\), quantifies the statistical dependence between the latent representationZ=f​\(X\)Z=f\(X\)and its augmented viewZ\+=f​\(X\+\)Z^\{\+\}=f\(X^\{\+\}\)\. A standard method to optimize MI is through variational lower bounds like the Donsker–Varadhan \(DV\) representation\. The DV bound states that for any measurable functionT​\(z,z′\)T\(z,z^\{\\prime\}\), a lower bound on MI is given by:

I​\(Z,Z\+\)≥𝔼\(Z,Z\+\)​\[T​\(Z,Z\+\)\]−log⁡𝔼Z,Z−∼p​\(Z\)2​\[eT​\(Z,Z−\)\]\.I\(Z,Z^\{\+\}\)\\geq\\mathbb\{E\}\_\{\(Z,Z^\{\+\}\)\}\\bigl\[T\(Z,Z^\{\+\}\)\\bigr\]\-\\log\\mathbb\{E\}\_\{Z,Z^\{\-\}\\sim p\(Z\)^\{2\}\}\\Bigl\[e^\{T\(Z,Z^\{\-\}\)\}\\Bigr\]\.Here,Z−Z^\{\-\}denotes an independent sample from the marginalp​\(Z\)p\(Z\)\(so the second expectation is under the product distributionp​\(Z\)​p​\(Z−\)p\(Z\)p\(Z^\{\-\}\); we writep​\(Z\)2p\(Z\)^\{2\}for brevity\)\. To relate this to the normalized/cosine SCL\-style surrogate used in MABLE, we choose the test functionT​\(z,z′\)=cos⁡\(z,z′\)T\(z,z^\{\\prime\}\)=\\cos\(z,z^\{\\prime\}\)\. Lett\+=cos⁡\(Z,Z\+\)t^\{\+\}=\\cos\(Z,Z^\{\+\}\)andt−=cos⁡\(Z,Z−\)t^\{\-\}=\\cos\(Z,Z^\{\-\}\)\. Assume the latent embeddingsZZare unit\-normalized, socos⁡\(z,z′\)∈\[−1,1\]\\cos\(z,z^\{\\prime\}\)\\in\[\-1,1\]\(and equals the dot product\), and thatp​\(Z\)p\(Z\)is approximately antipodally symmetric \(e\.g\., close to isotropic on the unit sphere𝕊d−1\\mathbb\{S\}^\{d\-1\}\), so thatt−=cos⁡\(Z,Z−\)t^\{\-\}=\\cos\(Z,Z^\{\-\}\)is approximately symmetric about0\(hence𝔼​\[t−\]≈0\\mathbb\{E\}\[t^\{\-\}\]\\approx 0and odd moments are negligible\)\. The bound becomes:

I​\(Z,Z\+\)≥𝔼​\[t\+\]−log⁡𝔼​\[et−\]\.I\(Z,Z^\{\+\}\)\\geq\\mathbb\{E\}\[t^\{\+\}\]\-\\log\\mathbb\{E\}\[e^\{t^\{\-\}\}\]\.
We analyze the second term,−log⁡𝔼​\[et−\]\-\\log\\mathbb\{E\}\[e^\{t^\{\-\}\}\], wheret−=cos⁡\(Z,Z−\)t^\{\-\}=\\cos\(Z,Z^\{\-\}\)andZ−∼p​\(Z\)Z^\{\-\}\\sim p\(Z\)is independent ofZZ\. Under this approximate symmetry, the expectation of any odd power oft−t^\{\-\}is approximately zero:𝔼​\[\(t−\)2​k\+1\]≈0\\mathbb\{E\}\[\(t^\{\-\}\)^\{2k\+1\}\]\\approx 0fork≥0k\\geq 0\.

We perform a Taylor expansion ofexe^\{x\}aroundx=0x=0:ex=∑n=0∞xnn\!e^\{x\}=\\sum\_\{n=0\}^\{\\infty\}\\frac\{x^\{n\}\}\{n\!\}\. Substitutingx=t−x=t^\{\-\}and taking the expectation:

𝔼​\[et−\]=𝔼​\[∑n=0∞\(t−\)nn\!\]=∑n=0∞𝔼​\[\(t−\)n\]n\!\.\\mathbb\{E\}\[e^\{t^\{\-\}\}\]=\\mathbb\{E\}\\left\[\\sum\_\{n=0\}^\{\\infty\}\\frac\{\(t^\{\-\}\)^\{n\}\}\{n\!\}\\right\]=\\sum\_\{n=0\}^\{\\infty\}\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{n\}\]\}\{n\!\}\.Using the property that odd moments are approximately zero:

𝔼​\[et−\]=𝔼​\[\(t−\)0\]0\!\+𝔼​\[\(t−\)1\]1\!\+𝔼​\[\(t−\)2\]2\!\+𝔼​\[\(t−\)3\]3\!\+…\\mathbb\{E\}\[e^\{t^\{\-\}\}\]=\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{0\}\]\}\{0\!\}\+\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{1\}\]\}\{1\!\}\+\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\}\{2\!\}\+\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{3\}\]\}\{3\!\}\+\\dots𝔼​\[et−\]≈1\+0\+𝔼​\[\(t−\)2\]2\+0\+⋯=1\+∑k=1∞𝔼​\[\(t−\)2​k\]\(2​k\)\!\.\\mathbb\{E\}\[e^\{t^\{\-\}\}\]\\approx 1\+0\+\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\}\{2\}\+0\+\\dots=1\+\\sum\_\{k=1\}^\{\\infty\}\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{2k\}\]\}\{\(2k\)\!\}\.
Now, we take the logarithm using the Taylor expansion oflog⁡\(1\+x\)\\log\(1\+x\)aroundx=0x=0, which isx−x22\+O​\(x3\)x\-\\frac\{x^\{2\}\}\{2\}\+O\(x^\{3\}\)\. Letx=∑k=1∞𝔼​\[\(t−\)2​k\]\(2​k\)\!=𝔼​\[\(t−\)2\]2\+𝔼​\[\(t−\)4\]24\+…x=\\sum\_\{k=1\}^\{\\infty\}\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{2k\}\]\}\{\(2k\)\!\}=\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\}\{2\}\+\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{4\}\]\}\{24\}\+\\dots\.

Assuming𝔼​\[\(t−\)2\]\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]is small \(typically the case in high dimensions under near\-uniform hyperspherical marginals\), we can approximate:

log⁡𝔼​\[et−\]=log⁡\(1\+x\)≈x−x22\\log\\mathbb\{E\}\[e^\{t^\{\-\}\}\]=\\log\(1\+x\)\\approx x\-\\frac\{x^\{2\}\}\{2\}log⁡𝔼​\[et−\]≈\(𝔼​\[\(t−\)2\]2\+𝔼​\[\(t−\)4\]24\+…\)−12​\(𝔼​\[\(t−\)2\]2\+…\)2\\log\\mathbb\{E\}\[e^\{t^\{\-\}\}\]\\approx\\left\(\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\}\{2\}\+\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{4\}\]\}\{24\}\+\\dots\\right\)\-\\frac\{1\}\{2\}\\left\(\\frac\{\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\}\{2\}\+\\dots\\right\)^\{2\}
Keeping only the leading terms:

log⁡𝔼​\[et−\]≈12​𝔼​\[\(t−\)2\]\+higher order terms\.\\log\\mathbb\{E\}\[e^\{t^\{\-\}\}\]\\approx\\frac\{1\}\{2\}\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\+\\text\{higher order terms\}\.
For the raw inner\-product choiceT​\(z,z′\)=z⊤​z′T\(z,z^\{\\prime\}\)=z^\{\\top\}z^\{\\prime\}, the same DV representation can formally be written, but the second\-order reduction to a quadratic negative term requires stronger assumptions: finite exponential moments, approximate zero mean, small variance, negligible higher\-order cumulants, and explicit norm control\. Without norm control,log⁡𝔼​\[exp⁡\(Z⊤​Z−\)\]\\log\\mathbb\{E\}\[\\exp\(Z^\{\\top\}Z^\{\-\}\)\]can be dominated by vector norms rather than angular dependence\. This is the scale ambiguity avoided by the cosine surrogate used in MABLE\.

Substituting this approximation back into the DV lower bound:

I​\(Z,Z\+\)≳𝔼​\[t\+\]−12​𝔼​\[\(t−\)2\]\.I\(Z,Z^\{\+\}\)\\gtrsim\\mathbb\{E\}\[t^\{\+\}\]\-\\frac\{1\}\{2\}\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\.
Recall the scaled normalized/cosine SCL\-style variantℒSSCL=12​ℒSCL=−𝔼​\[t\+\]\+12​𝔼​\[\(t−\)2\]\\mathcal\{L\}\_\{\\text\{SSCL\}\}=\\tfrac\{1\}\{2\}\\mathcal\{L\}\_\{\\text\{SCL\}\}=\-\\mathbb\{E\}\[t^\{\+\}\]\+\\tfrac\{1\}\{2\}\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\. MinimizingℒSSCL\\mathcal\{L\}\_\{\\text\{SSCL\}\}is equivalent to maximizing−ℒSSCL=𝔼​\[t\+\]−12​𝔼​\[\(t−\)2\]\-\\mathcal\{L\}\_\{\\text\{SSCL\}\}=\\mathbb\{E\}\[t^\{\+\}\]\-\\tfrac\{1\}\{2\}\\mathbb\{E\}\[\(t^\{\-\}\)^\{2\}\]\. Thus, we see that:

I​\(Z,Z\+\)≳−ℒSSCL\(equivalently,​I​\(Z,Z\+\)≳−12​ℒSCL​\)\.I\(Z,Z^\{\+\}\)\\gtrsim\-\\mathcal\{L\}\_\{\\text\{SSCL\}\}\\quad\\text\{\(equivalently, \}I\(Z,Z^\{\+\}\)\\gtrsim\-\\tfrac\{1\}\{2\}\\mathcal\{L\}\_\{\\text\{SCL\}\}\\text\{\)\.\}This shows that, under the stated boundedness, approximate symmetry, and small\-variance assumptions, minimizing the normalized/cosine SCL\-style surrogate used in MABLE approximately maximizes a second\-order approximation to this particular DV\-style mutual\-information objective\. It should not be read as a general claim that unconstrained raw inner\-product SCL has the same controlled DV approximation without additional norm\-control assumptions\.

### A\.4Proof of Proposition[2](https://arxiv.org/html/2607.02990#Thmproposition2)\- Lipschitz Pooling is Stable Under Optimal Matching

###### Proof\.

LetZ=\{zi\}i=1nZ=\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\}andZ′=\{zi′\}i=1nZ^\{\\prime\}=\\\{z^\{\\prime\}\_\{i\}\\\}\_\{i=1\}^\{n\}be two multisets of node embeddings, and fix an arbitrary ordering\(z1′,…,zn′\)\(z^\{\\prime\}\_\{1\},\\dots,z^\{\\prime\}\_\{n\}\)ofZ′Z^\{\\prime\}\. Letξ⋆∈Sn\\xi^\{\\star\}\\in S\_\{n\}attain the minimum in the matching distance, so that

dmatch​\(Z,Z′\)=∑i=1n‖zi−zξ⋆​\(i\)′‖\.d\_\{\\mathrm\{match\}\}\(Z,Z^\{\\prime\}\)=\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{\\xi^\{\\star\}\(i\)\}\\\|\.BecausePPis permutation\-invariant, reordering inputs does not change its output:

P​\(Z′\)=P​\(z1′,…,zn′\)=P​\(zξ⋆​\(1\)′,…,zξ⋆​\(n\)′\)\.P\(Z^\{\\prime\}\)=P\(z^\{\\prime\}\_\{1\},\\dots,z^\{\\prime\}\_\{n\}\)=P\(z^\{\\prime\}\_\{\\xi^\{\\star\}\(1\)\},\\dots,z^\{\\prime\}\_\{\\xi^\{\\star\}\(n\)\}\)\.Applying the Lipschitz property ofPPwith respect to∥⋅∥sum\\\|\\cdot\\\|\_\{\\mathrm\{sum\}\}to the ordered tuples\(z1,…,zn\)\(z\_\{1\},\\dots,z\_\{n\}\)and\(zξ⋆​\(1\)′,…,zξ⋆​\(n\)′\)\(z^\{\\prime\}\_\{\\xi^\{\\star\}\(1\)\},\\dots,z^\{\\prime\}\_\{\\xi^\{\\star\}\(n\)\}\)gives

‖P​\(Z\)−P​\(Z′\)‖=‖P​\(z1,…,zn\)−P​\(zξ⋆​\(1\)′,…,zξ⋆​\(n\)′\)‖≤L​∑i=1n‖zi−zξ⋆​\(i\)′‖=L​dmatch​\(Z,Z′\),\\\|P\(Z\)\-P\(Z^\{\\prime\}\)\\\|=\\\|P\(z\_\{1\},\\dots,z\_\{n\}\)\-P\(z^\{\\prime\}\_\{\\xi^\{\\star\}\(1\)\},\\dots,z^\{\\prime\}\_\{\\xi^\{\\star\}\(n\)\}\)\\\|\\leq L\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{\\xi^\{\\star\}\(i\)\}\\\|=L\\,d\_\{\\mathrm\{match\}\}\(Z,Z^\{\\prime\}\),which is the claimed bound\. ∎

### A\.5Proof of Proposition[3](https://arxiv.org/html/2607.02990#Thmproposition3)\- Sum, Mean, and Max Pooling are Lipschitz

###### Proof\.

Let\{zi\}i=1n\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\}and\{zi′\}i=1n\\\{z^\{\\prime\}\_\{i\}\\\}\_\{i=1\}^\{n\}be collections ofnnvectors inℝd\\mathbb\{R\}^\{d\}\. We verify the Lipschitz bounds stated in Proposition[3](https://arxiv.org/html/2607.02990#Thmproposition3)under the specified output norms and input metrics\.

Sum Pooling\.Under theℓ2\\ell\_\{2\}norm for individual vectors and the output:

‖SumPool​\(\{zi\}\)−SumPool​\(\{zi′\}\)‖ℓ2=‖∑i=1nzi−∑i=1nzi′‖ℓ2=‖∑i=1n\(zi−zi′\)‖ℓ2\.\\bigl\\\|\\text\{SumPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{SumPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\bigr\\\|\_\{\\ell\_\{2\}\}\\;=\\;\\Bigl\\\|\\sum\_\{i=1\}^\{n\}z\_\{i\}\-\\sum\_\{i=1\}^\{n\}z^\{\\prime\}\_\{i\}\\Bigr\\\|\_\{\\ell\_\{2\}\}\\;=\\;\\Bigl\\\|\\sum\_\{i=1\}^\{n\}\(z\_\{i\}\-z^\{\\prime\}\_\{i\}\)\\Bigr\\\|\_\{\\ell\_\{2\}\}\.By the triangle inequality for vector sums:

‖∑i=1n\(zi−zi′\)‖ℓ2≤∑i=1n‖zi−zi′‖ℓ2\.\\Bigl\\\|\\sum\_\{i=1\}^\{n\}\(z\_\{i\}\-z^\{\\prime\}\_\{i\}\)\\Bigr\\\|\_\{\\ell\_\{2\}\}\\;\\leq\\;\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\}\.Thus,‖SumPool​\(\{zi\}\)−SumPool​\(\{zi′\}\)‖ℓ2≤1⋅∑i=1n‖zi−zi′‖ℓ2\\\|\\text\{SumPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{SumPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\_\{\\ell\_\{2\}\}\\leq 1\\cdot\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\}\. Hence the Lipschitz constant for SumPool under theℓ2\\ell\_\{2\}norm is at most11\.

Mean Pooling\.MeanPool is defined as SumPool scaled by1/n1/n\. Under theℓ2\\ell\_\{2\}norm:

‖MeanPool​\(\{zi\}\)−MeanPool​\(\{zi′\}\)‖ℓ2=‖1n​∑i=1nzi−1n​∑i=1nzi′‖ℓ2=1n​‖∑i=1n\(zi−zi′\)‖ℓ2\.\\\|\\text\{MeanPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{MeanPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\_\{\\ell\_\{2\}\}\\;=\\;\\left\\\|\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}z\_\{i\}\-\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}z^\{\\prime\}\_\{i\}\\right\\\|\_\{\\ell\_\{2\}\}\\;=\\;\\frac\{1\}\{n\}\\Bigl\\\|\\sum\_\{i=1\}^\{n\}\(z\_\{i\}\-z^\{\\prime\}\_\{i\}\)\\Bigr\\\|\_\{\\ell\_\{2\}\}\.By the triangle inequality for vector sums:

1n​‖∑i=1n\(zi−zi′\)‖ℓ2≤1n​∑i=1n‖zi−zi′‖ℓ2\.\\frac\{1\}\{n\}\\Bigl\\\|\\sum\_\{i=1\}^\{n\}\(z\_\{i\}\-z^\{\\prime\}\_\{i\}\)\\Bigr\\\|\_\{\\ell\_\{2\}\}\\;\\leq\\;\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\}\.Thus,‖MeanPool​\(\{zi\}\)−MeanPool​\(\{zi′\}\)‖ℓ2≤1n⋅∑i=1n‖zi−zi′‖ℓ2\\\|\\text\{MeanPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{MeanPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\_\{\\ell\_\{2\}\}\\leq\\frac\{1\}\{n\}\\cdot\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\}\. Hence the Lipschitz constant for MeanPool under theℓ2\\ell\_\{2\}norm is at most1/n1/n\.

MaxPool\.Define thekk\-th coordinate of MaxPool asMaxPool\(\{zi\}\)k=max1≤i≤n\(zi\)k\\text\{MaxPool\}\(\\\{z\_\{i\}\\\}\)\_\{k\}=\\max\_\{1\\leq i\\leq n\}\(z\_\{i\}\)\_\{k\}\. We consider theℓ∞\\ell\_\{\\infty\}norm for the output of MaxPool\. The difference in outputs is:

∥MaxPool\(\{zi\}\)−MaxPool\(\{zi′\}\)∥∞=maxk\|maxi\(zi\)k−maxi\(zi′\)k\|\.\\\|\\text\{MaxPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{MaxPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\_\{\\infty\}\\;=\\;\\max\_\{k\}\\left\|\\max\_\{i\}\(z\_\{i\}\)\_\{k\}\-\\max\_\{i\}\(z^\{\\prime\}\_\{i\}\)\_\{k\}\\right\|\.A standard property of the maximum function is\|maxi⁡ai−maxi⁡bi\|≤maxi⁡\|ai−bi\|\\left\|\\max\_\{i\}a\_\{i\}\-\\max\_\{i\}b\_\{i\}\\right\|\\leq\\max\_\{i\}\\left\|a\_\{i\}\-b\_\{i\}\\right\|\. Applying this coordinate\-wise:

maxk\|maxi\(zi\)k−maxi\(zi′\)k\|≤maxk\(maxi\|\(zi\)k−\(zi′\)k\|\)\.\\max\_\{k\}\\left\|\\max\_\{i\}\(z\_\{i\}\)\_\{k\}\-\\max\_\{i\}\(z^\{\\prime\}\_\{i\}\)\_\{k\}\\right\|\\;\\leq\\;\\max\_\{k\}\\left\(\\max\_\{i\}\\left\|\(z\_\{i\}\)\_\{k\}\-\(z^\{\\prime\}\_\{i\}\)\_\{k\}\\right\|\\right\)\.We can swap the order of maximization:

maxk⁡\(maxi⁡\|\(zi\)k−\(zi′\)k\|\)=maxi⁡\(maxk⁡\|\(zi\)k−\(zi′\)k\|\)\.\\max\_\{k\}\\left\(\\max\_\{i\}\\left\|\(z\_\{i\}\)\_\{k\}\-\(z^\{\\prime\}\_\{i\}\)\_\{k\}\\right\|\\right\)\\;=\\;\\max\_\{i\}\\left\(\\max\_\{k\}\\left\|\(z\_\{i\}\)\_\{k\}\-\(z^\{\\prime\}\_\{i\}\)\_\{k\}\\right\|\\right\)\.The termmaxk⁡\|\(zi\)k−\(zi′\)k\|\\max\_\{k\}\\left\|\(z\_\{i\}\)\_\{k\}\-\(z^\{\\prime\}\_\{i\}\)\_\{k\}\\right\|is theℓ∞\\ell\_\{\\infty\}norm of the vector differencezi−zi′z\_\{i\}\-z^\{\\prime\}\_\{i\}, i\.e\.,‖zi−zi′‖ℓ∞\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{\\infty\}\}\. So the expression becomesmaxi⁡‖zi−zi′‖ℓ∞\\max\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{\\infty\}\}\. Thus,

‖MaxPool​\(\{zi\}\)−MaxPool​\(\{zi′\}\)‖∞≤maxi⁡‖zi−zi′‖ℓ∞,\\\|\\text\{MaxPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{MaxPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\_\{\\infty\}\\;\\leq\\;\\max\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{\\infty\}\},
Using‖v‖ℓ∞≤‖v‖ℓ2\\\|v\\\|\_\{\\ell\_\{\\infty\}\}\\leq\\\|v\\\|\_\{\\ell\_\{2\}\}for allv∈ℝdv\\in\\mathbb\{R\}^\{d\}andmaxi⁡ai≤∑i=1nai\\max\_\{i\}a\_\{i\}\\leq\\sum\_\{i=1\}^\{n\}a\_\{i\}forai≥0a\_\{i\}\\geq 0, we obtain

maxi⁡‖zi−zi′‖ℓ∞≤maxi⁡‖zi−zi′‖ℓ2≤∑i=1n‖zi−zi′‖ℓ2\.\\max\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{\\infty\}\}\\;\\leq\\;\\max\_\{i\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\}\\;\\leq\\;\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\}\.Combining the inequalities yields

‖MaxPool​\(\{zi\}\)−MaxPool​\(\{zi′\}\)‖∞≤∑i=1n‖zi−zi′‖ℓ2,\\\|\\text\{MaxPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{MaxPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\_\{\\infty\}\\;\\leq\\;\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\},If the output is instead measured inℓ2\\ell\_\{2\}, then using‖v‖ℓ2≤d​‖v‖ℓ∞\\\|v\\\|\_\{\\ell\_\{2\}\}\\leq\\sqrt\{d\}\\,\\\|v\\\|\_\{\\ell\_\{\\infty\}\}gives

‖MaxPool​\(\{zi\}\)−MaxPool​\(\{zi′\}\)‖ℓ2≤d​∑i=1n‖zi−zi′‖ℓ2,\\\|\\text\{MaxPool\}\(\\\{z\_\{i\}\\\}\)\-\\text\{MaxPool\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\_\{\\ell\_\{2\}\}\\;\\leq\\;\\sqrt\{d\}\\,\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{\\ell\_\{2\}\},which yields the statedd\\sqrt\{d\}bound\. ∎

### A\.6Proof of Lemma[1](https://arxiv.org/html/2607.02990#Thmlemma1)\- Bounded\-Domain Lipschitzness of Clipped Softmax Attention

###### Proof\.

LetZ=\(z1,…,zn\)Z=\(z\_\{1\},\\dots,z\_\{n\}\)andZ′=\(z1′,…,zn′\)Z^\{\\prime\}=\(z^\{\\prime\}\_\{1\},\\dots,z^\{\\prime\}\_\{n\}\)lie in𝒟B\\mathcal\{D\}\_\{B\}, and use the product metric

dsum​\(Z,Z′\):=∑i=1n‖zi−zi′‖2\.d\_\{\\mathrm\{sum\}\}\(Z,Z^\{\\prime\}\):=\\sum\_\{i=1\}^\{n\}\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{2\}\.The normalization mapNε​\(z\)=z/max⁡\(‖z‖2,ε\)N\_\{\\varepsilon\}\(z\)=z/\\max\(\\\|z\\\|\_\{2\},\\varepsilon\)is Lipschitz for fixedε\>0\\varepsilon\>0\(with a constant depending only onε\\varepsilon\)\. Sinceψ\\psiis Lipschitz, applyingψ∘Nε\\psi\\circ N\_\{\\varepsilon\}coordinate\-wise to thennnodes gives a Lipschitz map fromZZto the raw score vectors~​\(Z\)\\tilde\{s\}\(Z\)\. The maps~↦s~−maxj⁡s~j​𝟏\\tilde\{s\}\\mapsto\\tilde\{s\}\-\\max\_\{j\}\\tilde\{s\}\_\{j\}\\mathbf\{1\}is Lipschitz because the maximum function is Lipschitz, coordinate\-wise clipping to\[−c,0\]\[\-c,0\]is non\-expansive, and softmax is smooth and Lipschitz on the compact cube\[−c,0\]n\[\-c,0\]^\{n\}\. Therefore the attention\-weight vectorα​\(Z\)=\(α1​\(Z\),…,αn​\(Z\)\)∈Δn−1\\alpha\(Z\)=\(\\alpha\_\{1\}\(Z\),\\dots,\\alpha\_\{n\}\(Z\)\)\\in\\Delta^\{n\-1\}is Lipschitz on𝒟B\\mathcal\{D\}\_\{B\}; i\.e\., for some finite constantLαL\_\{\\alpha\},

‖α​\(Z\)−α​\(Z′\)‖1=∑i=1n\|αi​\(Z\)−αi​\(Z′\)\|≤Lα​dsum​\(Z,Z′\)\.\\\|\\alpha\(Z\)\-\\alpha\(Z^\{\\prime\}\)\\\|\_\{1\}=\\sum\_\{i=1\}^\{n\}\|\\alpha\_\{i\}\(Z\)\-\\alpha\_\{i\}\(Z^\{\\prime\}\)\|\\leq L\_\{\\alpha\}\\,d\_\{\\mathrm\{sum\}\}\(Z,Z^\{\\prime\}\)\.Now decompose the attention\-weighted sums:

‖A​\(Z\)−A​\(Z′\)‖2\\displaystyle\\\|A\(Z\)\-A\(Z^\{\\prime\}\)\\\|\_\{2\}=‖∑i=1nαi​\(Z\)​zi−∑i=1nαi​\(Z′\)​zi′‖2\\displaystyle=\\left\\\|\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(Z\)z\_\{i\}\-\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(Z^\{\\prime\}\)z^\{\\prime\}\_\{i\}\\right\\\|\_\{2\}≤‖∑i=1nαi​\(Z\)​\(zi−zi′\)‖2\+‖∑i=1n\(αi​\(Z\)−αi​\(Z′\)\)​zi′‖2\\displaystyle\\leq\\left\\\|\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(Z\)\(z\_\{i\}\-z^\{\\prime\}\_\{i\}\)\\right\\\|\_\{2\}\+\\left\\\|\\sum\_\{i=1\}^\{n\}\\bigl\(\\alpha\_\{i\}\(Z\)\-\\alpha\_\{i\}\(Z^\{\\prime\}\)\\bigr\)z^\{\\prime\}\_\{i\}\\right\\\|\_\{2\}≤∑i=1nαi​\(Z\)​‖zi−zi′‖2\+B​∑i=1n\|αi​\(Z\)−αi​\(Z′\)\|\\displaystyle\\leq\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(Z\)\\\|z\_\{i\}\-z^\{\\prime\}\_\{i\}\\\|\_\{2\}\+B\\sum\_\{i=1\}^\{n\}\|\\alpha\_\{i\}\(Z\)\-\\alpha\_\{i\}\(Z^\{\\prime\}\)\|≤\(1\+B​Lα\)​dsum​\(Z,Z′\)\.\\displaystyle\\leq\\bigl\(1\+BL\_\{\\alpha\}\\bigr\)\\,d\_\{\\mathrm\{sum\}\}\(Z,Z^\{\\prime\}\)\.ThusAAis Lipschitz on𝒟B\\mathcal\{D\}\_\{B\}\. Ifσmax​\(Wp​o​o​l\)≤LW\\sigma\_\{\\max\}\(W\_\{pool\}\)\\leq L\_\{W\}, then

‖Wp​o​o​l​A​\(Z\)−Wp​o​o​l​A​\(Z′\)‖2≤LW​‖A​\(Z\)−A​\(Z′\)‖2,\\\|W\_\{pool\}A\(Z\)\-W\_\{pool\}A\(Z^\{\\prime\}\)\\\|\_\{2\}\\leq L\_\{W\}\\\|A\(Z\)\-A\(Z^\{\\prime\}\)\\\|\_\{2\},soPattn=Wp​o​o​l∘AP\_\{\\mathrm\{attn\}\}=W\_\{pool\}\\circ Ais Lipschitz on the same bounded domain\. ∎

### A\.7Proof of Corollary[1](https://arxiv.org/html/2607.02990#Thmcorollary1)\- Conditioned Projection Preserves Attention\-Induced Separation

###### Proof\.

Recall that the pooling operation is

Pattn​\(\{zi\}\)=W​\(∑i=1nαi​\(\{zk\}k=1n\)​zi\),P\_\{\\mathrm\{attn\}\}\(\\\{z\_\{i\}\\\}\)\\;=\\;W\\Bigl\(\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z\_\{i\}\\Bigr\),where the matrixW∈ℝdG×dW\\in\\mathbb\{R\}^\{d\_\{G\}\\times d\}hasσmin​\(W\)≥m\>0\\sigma\_\{\\min\}\(W\)\\geq m\>0\. This condition meansWWismm\-co\-Lipschitz:

‖W​\(u\)−W​\(v\)‖≥m​‖u−v‖for all​u,v∈ℝd\.\\\|\\,W\(u\)\-W\(v\)\\\|\\;\\geq\\;m\\,\\\|\\,u\-v\\\|\\quad\\text\{for all \}u,v\\in\\mathbb\{R\}^\{d\}\.Let\{zi\}\\\{z\_\{i\}\\\}and\{zi′\}\\\{z^\{\\prime\}\_\{i\}\\\}be two distinct sets of embeddings from the data distribution\. Define their respective attention\-weighted sums as:

Δz=∑i=1nαi​\(\{zk\}k=1n\)​zi,Δz′=∑i=1nαi​\(\{zk′\}k=1n\)​zi′\.\\Delta\_\{z\}\\;=\\;\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z\_\{i\},\\quad\\Delta\_\{z^\{\\prime\}\}\\;=\\;\\sum\_\{i=1\}^\{n\}\\alpha\_\{i\}\(\\\{z^\{\\prime\}\_\{k\}\\\}\_\{k=1\}^\{n\}\)\\,z^\{\\prime\}\_\{i\}\.Letδpair=‖Δz−Δz′‖\\delta\_\{\\text\{pair\}\}=\\\|\\Delta\_\{z\}\-\\Delta\_\{z^\{\\prime\}\}\\\|denote the actual separation achieved between these weighted sums by the attention mechanismαi​\(⋅\)\\alpha\_\{i\}\(\\cdot\)for this specific pair\. The difference between the pooled outputs is

‖Pattn​\(\{zi\}\)−Pattn​\(\{zi′\}\)‖=‖W​\(Δz\)−W​\(Δz′\)‖\.\\\|\\,P\_\{\\mathrm\{attn\}\}\(\\\{z\_\{i\}\\\}\)\-P\_\{\\mathrm\{attn\}\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\\;=\\;\\Bigl\\\|W\(\\Delta\_\{z\}\)\\;\-\\;W\(\\Delta\_\{z^\{\\prime\}\}\)\\Bigr\\\|\.Applying themm\-co\-Lipschitz property ofWWwithu=Δzu=\\Delta\_\{z\}andv=Δz′v=\\Delta\_\{z^\{\\prime\}\}:

‖W​\(Δz\)−W​\(Δz′\)‖≥m​‖Δz−Δz′‖\.\\Bigl\\\|W\(\\Delta\_\{z\}\)\\;\-\\;W\(\\Delta\_\{z^\{\\prime\}\}\)\\Bigr\\\|\\;\\geq\\;m\\,\\\|\\Delta\_\{z\}\-\\Delta\_\{z^\{\\prime\}\}\\\|\.Substituting the definition ofδpair\\delta\_\{\\text\{pair\}\}, we obtain

‖Pattn​\(\{zi\}\)−Pattn​\(\{zi′\}\)‖≥m⋅δpair,\\\|\\,P\_\{\\mathrm\{attn\}\}\(\\\{z\_\{i\}\\\}\)\-P\_\{\\mathrm\{attn\}\}\(\\\{z^\{\\prime\}\_\{i\}\\\}\)\\\|\\;\\geq\\;m\\cdot\\delta\_\{\\text\{pair\}\},as desired\. This inequality holds for any pair; the corollary highlights the consequence whenδpair\>0\\delta\_\{\\text\{pair\}\}\>0\. ∎

### A\.8Ablation on feature\-slice dimensionalityded\_\{e\}

![Refer to caption](https://arxiv.org/html/2607.02990v1/figures/ablation_plot_row_v2.png)Figure 12:Embedding dimension ablation under spatial holdout \(south→\\rightarrownorth\) with spatial CV onCuEq PK Regressiondataset \(Experiment 1, Study 4\)\. Comparison of Random Forest performance using pretrained embeddings produced withde=15d\_\{e\}=15vs\.de=128d\_\{e\}=128\. Curves showemb\_onlyandpos\_plus\_embacross PCA rankkk\(log\-scale;k=128k=128denotes no PCA\), and the dashed line indicates thepos\_onlybaseline\.Table 1:Feature\-slice dimensionality ablation\.Best\-checkpoint training and validation losses for three feature\-slice widthsded\_\{e\}\(selected by minimum validation total loss\) forCuEq PK Regressiondataset \(Experiment 1, Study 4\)\. Whilede=128d\_\{e\}\{=\}128achieves the lowest validation total loss,de=15d\_\{e\}\{=\}15is effectively comparable on validation loss while reaching its best checkpoint earlier \(about∼\\sim84% of the 128d training steps in our runs\) and exhibiting a smaller train–validation gap thande=128d\_\{e\}\{=\}128, consistent with reduced overfitting\.