SAOT: Self-Supervised Continual Graph Learning with Structure-Aware Optimal Transport

# SAOT: Self-Supervised Continual Graph Learning with Structure-Aware Optimal Transport
Source: [https://arxiv.org/html/2607.00377](https://arxiv.org/html/2607.00377)
###### Abstract

Self\-supervised Continual Graph Learning \(CGL\) aims to successively learn from a graph sequence with different tasks without label supervision—a paradigm that has attracted widespread attention\. Most existing self\-supervised CGL methods rely on instance\-level consistency objectives that enforce stability of individual node \(or node\-pair\) embeddings\. Due to optimizing nodes in isolation, these methods fail to maintain global relational structure, causing inter\-node correspondences to progressively distort under continual learning\. To this end, we propose a novel Structure\-Aware Optimal Transport \(SAOT\) framework that explicitly captures and preserves relational structure within graph representations across sequential tasks\. Specifically, SAOT leverages optimal transport theory to capture global inter\-node correspondences, thereby facilitating and enhancing graph representation learning\. Simultaneously, SAOT incorporates a cross\-task knowledge distillation mechanism to preserve the previous structural knowledge\. Extensive experiments on four CGL benchmark datasets demonstrate that SAOT outperforms existing self\-supervised baselines\. In particular, SAOT achieves significant performance gains, improving average accuracy by up to 5% on CoraFull\-CL and over 15% on Products\-CL compared with state\-of\-the\-art methods in the Class\-IL setting\.

Machine Learning, ICML

## 1Introduction

Graph data are ubiquitous in real\-world applications, modeling complex systems with rich relational dependencies, such as citation networks, e\-commerce systems, and biochemical molecules\(Hamiltonet al\.,[2017](https://arxiv.org/html/2607.00377#bib.bib2); Wuet al\.,[2021](https://arxiv.org/html/2607.00377#bib.bib1)\)\. Despite the remarkable progress of graph representation learning, most existing methods are designed for static settings, where the data distribution is assumed to be stationary\(Kipf and Welling,[2017](https://arxiv.org/html/2607.00377#bib.bib3); Veličkovićet al\.,[2018](https://arxiv.org/html/2607.00377#bib.bib4)\)\. In contrast, real\-world graph data are continuously generated, with new nodes, edges, or tasks appearing over time\(Wanget al\.,[2020](https://arxiv.org/html/2607.00377#bib.bib5)\)\. For instance, papers on new research topics continuously enter citation networks, and novel molecular properties are progressively encountered in drug discovery tasks\(Liuet al\.,[2021](https://arxiv.org/html/2607.00377#bib.bib11); Zhanget al\.,[2022b](https://arxiv.org/html/2607.00377#bib.bib7)\)\. To cope with such evolving data, graph models are required to incrementally acquire new knowledge while maintaining performance on previously learned tasks, a learning paradigm known as Continual Graph Learning \(CGL\)\(Zhou and Cao,[2021](https://arxiv.org/html/2607.00377#bib.bib6); Zhanget al\.,[2022b](https://arxiv.org/html/2607.00377#bib.bib7)\)\. However, simply training models sequentially on incoming data is prone to catastrophic forgetting\(McCloskey and Cohen,[1989](https://arxiv.org/html/2607.00377#bib.bib8); Goodfellowet al\.,[2014](https://arxiv.org/html/2607.00377#bib.bib9)\), while retraining a model on all accumulated data is computationally expensive and often infeasible when historical data are unavailable\.

To mitigate catastrophic forgetting, existing CGL approaches generally are divided into three categories\. Parameter isolation methods\(Zhanget al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib16); Caiet al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib15)\)allocate task\-specific parameters or modules to avoid interference between tasks\. Regularization\-based approaches\(Kirkpatricket al\.,[2017](https://arxiv.org/html/2607.00377#bib.bib10); Liuet al\.,[2021](https://arxiv.org/html/2607.00377#bib.bib11)\)constrain parameter updates by penalizing changes to important parameters or outputs learned from previous tasks\. Replay\-based methods\(Zhou and Cao,[2021](https://arxiv.org/html/2607.00377#bib.bib6); Zhanget al\.,[2022a](https://arxiv.org/html/2607.00377#bib.bib12)\)explicitly store a small set of historical nodes or subgraphs and rehearse them during training to maintain past knowledge\. While these approaches have demonstrated effectiveness, they typically rely on explicit supervision\. However, annotating complex graph data is often time\-consuming, labor\-intensive, and sometimes even impractical, especially when graphs continuously emerge in an online manner\.

Consequently, it is crucial to advance self\-supervised continual graph learning, enabling models to acquire knowledge directly from unlabeled streaming data\. Driven by the above necessity, recent studies\(Sunet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib19); Penget al\.,[2025](https://arxiv.org/html/2607.00377#bib.bib14)\)have explored self\-supervised approaches for continual graph learning scenarios\. In particular, TRACE\(Penget al\.,[2025](https://arxiv.org/html/2607.00377#bib.bib14)\)provides a systematic empirical study of representative self\-supervised graph representation learning paradigms in continual settings, demonstrating that self\-supervised graph models\(Kipf and Welling,[2016](https://arxiv.org/html/2607.00377#bib.bib20); Youet al\.,[2020b](https://arxiv.org/html/2607.00377#bib.bib31); Bielaket al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib26)\)can learn more transferable and stable representations than their supervised counterparts in the absence of label supervision\.

However, most existing self\-supervised CGL methods rely on instance\-level consistency objectives, treating each node \(or node pair\) as an independent learning instance to stabilize individual node embeddings\(Sunet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib19); Penget al\.,[2025](https://arxiv.org/html/2607.00377#bib.bib14)\)\. Although these methods effectively promote local embedding stability, they overlook a critical characteristic of streaming graph data: long\-term knowledge is encoded not merely in isolated embeddings but in the relational structure within the representation space\. Consequently, even if individual nodes embeddings remain locally stable, the relationships among nodes can gradually distort—a phenomenon known as structural drift, which manifests as changes in relative distances or the overall organization of node clusters across sequential tasks\. Because most existing objectives do not explicitly model relational dependencies, such drift accumulates over time, ultimately impairing long\-term knowledge retention\.

To this end, we propose a novel self\-supervised continual graph learning framework based on Structure\-Aware Optimal Transport, named SAOT\. Specifically, SAOT leverages Optimal Transport \(OT\) theory to encode the relational structure among all nodes\. Within each task, SAOT constructs an optimal transport plan that jointly captures correspondences among nodes in the graph space\. The plan is used as a structural reference to guide the encoder to learn structure\-aware, transferable node representations\. To mitigate structural drift caused by sequential tasks, SAOT employs cross\-task knowledge distillation to preserve the previous structural information\. Through the coordinated operation of the above two modules, SAOT enables the model to adapt effectively to new tasks while consolidating previously learned structural knowledge, achieving an optimal balance between plasticity and stability\. In summary, the main contributions of the proposed SAOT are as follows:

- •We propose SAOT, a novel self\-supervised continual graph learning framework that leverages optimal transport to model and preserve relational structure in graph representations across sequential tasks\.
- •We design a cross\-task knowledge distillation mechanism that preserves relational structure by distilling optimal transport plans defined in the representation space across tasks, thereby effectively mitigating the problem of structure drift under continual learning\.
- •We conduct extensive experiments on four CGL benchmark datasets under both Class\-IL and Task\-IL settings\. Experimental results demonstrate that the proposed SAOT outperforms existing self\-supervised baselines\. In particular, in the Class\-IL setting, SAOT achieves an improvement of up to 5% on CoraFull\-CL and over 15% on Products\-CL in terms of average accuracy compared with state\-of\-the\-art methods\.

## 2Related Work

### 2\.1Continual Graph Learning

Inspired by recent advances in continual learning for computer vision, various methods for CGL have been proposed in recent years to address learning on streaming graph data\(Babakniyaet al\.,[2024](https://arxiv.org/html/2607.00377#bib.bib36); Leeet al\.,[2024](https://arxiv.org/html/2607.00377#bib.bib35)\)\. From a methodological perspective, existing CGL methods can be broadly grouped into three categories\. Parameter isolation methods\(Caiet al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib15); Zhanget al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib16)\)allocate task\-specific parameters or network components through specialized architectures, such as expanding networks, task\-specific modules, or gated routing mechanisms, in order to prevent interference between tasks\. Replay\-based methods\(Zhou and Cao,[2021](https://arxiv.org/html/2607.00377#bib.bib6); Zhanget al\.,[2022a](https://arxiv.org/html/2607.00377#bib.bib12); Liuet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib17); Zhanget al\.,[2024](https://arxiv.org/html/2607.00377#bib.bib18)\)mitigate catastrophic forgetting by maintaining a memory buffer that stores information from previous tasks and replaying it during the learning of new tasks\. Regularization\-based methods\(Kirkpatricket al\.,[2017](https://arxiv.org/html/2607.00377#bib.bib10); Liuet al\.,[2021](https://arxiv.org/html/2607.00377#bib.bib11); Sunet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib19)\)restrict parameter updates or representation changes across tasks by imposing consistency constraints on model parameters, node embeddings, or output predictions\.

However, most existing CGL methods rely heavily on supervisory signals, such as node or graph labels, which limits their applicability in realistic scenarios\. In contrast, research on unsupervised or self\-supervised CGL remains relatively underexplored\. More recently, a notable attempt is RieGrace\(Sunet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib19)\), which proposes a unified framework that combines GNNs with CurvNet\. It explicitly handles task\-specific adaptation in Riemannian space and employs a label\-free Lorentz distillation mechanism to mitigate catastrophic forgetting\. Inspired by the Complementary Learning Systems theory, TRACE\(Penget al\.,[2025](https://arxiv.org/html/2607.00377#bib.bib14)\)employs a dual\-system framework to mitigate catastrophic forgetting without external supervision\. Specifically, a fast\-learning system extracts key knowledge through node proxies, while a slow\-learning system consolidates memory via adaptive spaced replay\.

### 2\.2Self\-Supervised Graph Learning

Self\-supervised graph learning aims to learn expressive node or graph representations without relying on explicit labels by exploiting intrinsic structural and attribute\-based signals\. Early approaches\(Kipf and Welling,[2016](https://arxiv.org/html/2607.00377#bib.bib20); Jinet al\.,[2020](https://arxiv.org/html/2607.00377#bib.bib23)\)are primarily based on generative objectives, such as graph autoencoders and variational graph autoencoders, which reconstruct node features or graph structures from latent representations\. More recent methods\(Veličkovićet al\.,[2019](https://arxiv.org/html/2607.00377#bib.bib22); Youet al\.,[2020a](https://arxiv.org/html/2607.00377#bib.bib21); Zhuet al\.,[2020](https://arxiv.org/html/2607.00377#bib.bib24)\)adopt contrastive learning paradigms, where representations are learned by maximizing agreement between different augmented views of the same graph while contrasting them against other samples\. These approaches have demonstrated good performance on various downstream tasks\. Beyond contrastive learning, redundancy\-reduction and non\-contrastive paradigms\(Leeet al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib27); Thakooret al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib25); Bielaket al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib26)\)have been proposed to remove the reliance on negative samples\. These methods learn invariant representations across augmented views by explicitly minimizing feature redundancy or aligning cross\-view statistics\. Nevertheless, most methods are trained offline under the assumption of full data availability, which does not reflect the reality that graph data arrive sequentially over time\.

## 3Preliminaries

In this section, we briefly review optimal transport, which serve as the key technical foundation of our method\. We then formally defined the studied paradigm: self\-supervised continual graph learning\.

### 3\.1Plan and Optimal Transport

The optimal transport problem was originally proposed to study the most cost\-effective way of transporting the shape of one pile of sand into the shape of another\(Wanget al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib28)\)\. Specifically, it studies how to transform distributionμ\\muinto another distributionν\\nuwith the minimum total transportation cost \(i\.e, the optimal transportation distance\)\. This transport plan is called the optimal planπ\\pi, where the element ofπ\\pidescribes the probability of moving mass from one position to another\. Formally, given two sets of features𝐗1=\{𝐗1i\}i=1n\{\\mathbf\{X\}\_\{1\}\}=\\\{\\mathbf\{X\}\_\{1\}^\{i\}\\\}\_\{i=1\}^\{n\}and𝐗2=\{𝐗2j\}j=1m\{\\mathbf\{X\}\_\{2\}\}=\\\{\\mathbf\{X\}\_\{2\}^\{j\}\\\}\_\{j=1\}^\{m\}, wherennandmmare the number of features,μ∈ℝn\{\\mu\}\\in\\mathbb\{R\}^\{n\}andν∈ℝm\{\\nu\}\\in\\mathbb\{R\}^\{m\}are the probability distributions of the entities in the two sets, respectively\. The formulation of the OT distance is

𝒟​\(𝐗1,𝐗2\)=minπ∈Π​\(μ,ν\)​∑i∈\[\[n\]\]∑j∈\[\[m\]\]c𝒳​\(𝐗1i,𝐗2j\)⋅πi​j,\\mathcal\{D\}\(\\mathbf\{X\}\_\{1\},\\mathbf\{X\}\_\{2\}\)=\\min\_\{\\pi\\in\\Pi\(\\mu,\\nu\)\}\\sum\_\{i\\in\[\\\!\[n\]\\\!\]\}\\sum\_\{j\\in\[\\\!\[m\]\\\!\]\}c\_\{\\mathcal\{X\}\}\(\\mathbf\{X\}\_\{1\}^\{i\},\\mathbf\{X\}\_\{2\}^\{j\}\)\\cdot\\pi\_\{ij\},\(1\)abbreviated as

𝒟​\(𝐗1,𝐗2\)=minπ∈Π​\(μ,ν\)⁡⟨𝒦​\(𝐗1,𝐗2\),π⟩,\\mathcal\{D\}\(\\mathbf\{X\}\_\{1\},\\mathbf\{X\}\_\{2\}\)=\\min\_\{\\pi\\in\\Pi\(\\mu,\\nu\)\}\\langle\\mathcal\{K\}\(\\mathbf\{X\}\_\{1\},\\mathbf\{X\}\_\{2\}\),\\pi\\rangle,\(2\)
whereπ∈Π​\(μ,ν\)=\{π∈ℝn×m∣π​𝟏m=μ,π⊤​𝟏n=ν\}\\pi\\in\\Pi\(\\mu,\\nu\)=\\\{\\pi\\in\\mathbb\{R\}^\{n\\times m\}\\mid\\pi\\mathbf\{1\}\_\{m\}=\\mu,\\pi^\{\\top\}\\mathbf\{1\}\_\{n\}=\\nu\\\}denotes all the joint distributionsπ\\piwith the marginal distributionμ\\muandν\\nu\.𝒦​\(𝐗1,𝐗2\)i​j=c𝒳​\(𝐗1i,𝐗2j\)\\mathcal\{K\}\(\\mathbf\{X\}\_\{1\},\\mathbf\{X\}\_\{2\}\)\_\{ij\}=c\_\{\\mathcal\{X\}\}\(\\mathbf\{X\}\_\{1\}^\{i\},\\mathbf\{X\}\_\{2\}^\{j\}\)is the cost of moving𝐗1i\\mathbf\{X\}\_\{1\}^\{i\}to𝐗2j\\mathbf\{X\}\_\{2\}^\{j\}\. The cost can be calculated by cosine distance\. The𝟏\\mathbf\{1\}denotes a vector that elements are all 1,\[\[n\]\]=\{1,2,…,n\}\[\\\!\[n\]\\\!\]=\\\{1,2,\\dots,n\\\}, and⟨⋅,⋅⟩\\langle\\cdot,\\cdot\\rangleis the inner product operator\. Theπ∈ℝn×m\\pi\\in\\mathbb\{R\}^\{n\\times m\}is called as transport plan\. The distance𝒟​\(⋅,⋅\)\\mathcal\{D\}\\left\(\\cdot,\\cdot\\right\)is also known as Wasserstein distance\. In this work, we focus on the case of discrete distributions on graphs\.

![Refer to caption](https://arxiv.org/html/2607.00377v1/x1.png)Figure 1:Overview of SAOT\.It leverages optimal transport mechanism to learn and preserve relational structure in CGL\. When a new task arrives, the model learns structure\-aware representations by aligning optimal transport plans in the graph and embedding spaces\. Simultaneously, SAOT consolidates the learned structural knowledge from the previous model by distillation mechanism\.
### 3\.2Problem Formulation

The setting of continual graph learning assumes a sequence of disjoint tasks𝒯=\{𝒯1,…,𝒯T\}\\mathcal\{T\}=\\\{\\mathcal\{T\}\_\{1\},\\ldots,\\mathcal\{T\}\_\{T\}\\\}which will be encountered sequentially\. Each task𝒯t∈𝒯\\mathcal\{T\}\_\{t\}\\in\\mathcal\{T\}is defined on a graph𝒢t=\(𝒱t,ℰt\)\\mathcal\{G\}\_\{t\}=\(\\mathcal\{V\}\_\{t\},\\mathcal\{E\}\_\{t\}\), where𝒱t\\mathcal\{V\}\_\{t\}andℰt\\mathcal\{E\}\_\{t\}denote the sets of nodes and edges, respectively\. The node attributes are represented by the feature matrix𝐗t∈ℝ\|𝒱t\|×d\\mathbf\{X\}\_\{t\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\_\{t\}\|\\times d\}, whereddis the feature dimension\. Furthermore, the topological structure of the graph is captured by the adjacency matrix𝐀t∈\{0,1\}\|𝒱t\|×\|𝒱t\|\\mathbf\{A\}\_\{t\}\\in\\\{0,1\\\}^\{\|\\mathcal\{V\}\_\{t\}\|\\times\|\\mathcal\{V\}\_\{t\}\|\}\. In this work, we follow a decoupled representation–classifier paradigm\(Chaet al\.,[2021](https://arxiv.org/html/2607.00377#bib.bib40); Gomez\-Villaet al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib39); Sunet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib19)\)for self\-supervised CGL, in which representation learning is explicitly separated from downstream classifier training\.

Given a sequence of graph related tasks𝒯\\mathcal\{T\}, we aim at learning a shared graph encoderℱξ:𝒱t→ℝH\\mathcal\{F\}\_\{\\xi\}:\\mathcal\{V\}\_\{t\}\\rightarrow\\mathbb\{R\}^\{H\}across all task sequences, which maps nodes toHH\-dimensional representation vectors, so that the encoder is able to continuously adapt new knowledge in the current tasks without catastrophically forgetting knowledge acquired from previous tasks\. During the learning process, no external supervisory signals \(i\.e\., labels\) are involved\.

To evaluate the quality of learned representations, we follow the experimental setting in prior work on self\-supervised continual learning\(Sunet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib19); Gomez\-Villaet al\.,[2024](https://arxiv.org/html/2607.00377#bib.bib37); Penget al\.,[2025](https://arxiv.org/html/2607.00377#bib.bib14)\)\. Specifically, for the graph𝒢t\\mathcal\{G\}\_\{t\}from each task, we have training node set𝒱ttr\\mathcal\{V\}\_\{t\}^\{\\textit\{tr\}\}and testing node set𝒱tte\\mathcal\{V\}\_\{t\}^\{\\textit\{te\}\}\. Each nodevtiv\_\{t\}^\{i\}in𝒢t\\mathcal\{G\}\_\{t\}belongs to a categoryyti∈𝒴Py\_\{t\}^\{i\}\\in\\mathcal\{Y\}\_\{P\}, where𝒴P\\mathcal\{Y\}\_\{P\}denotes the label set ofPcategories involved in all tasks\. During evaluation, the parameters of the graph encoderℱξ\\mathcal\{F\}\_\{\\xi\}are kept frozen, and we provide the learned embeddings across the training node set to the linear classifier for performance evaluation\. We consider two different settings, i\.e\., Task Incremental Learning \(Task\-IL\) and Class Incremental Learning \(Class\-IL\)\. In the Task\-IL setting, the model is only required to distinguish different classes within each task without considering the classes that have already appeared from existing tasks\. In the Class\-IL setting, the model is required to distinguish among all classes from the current and previous tasks\. Obviously, Class\-IL is more realistic and challenging\.

## 4Methodology

This section describes the details of the proposed SAOT framework\. SAOT consists of two components: \(i\) Graph Representation via Optimal Transport, which learns structure\-aware representations within each task, and \(ii\) Cross\-task Knowledge Distillation, which preserves relational knowledge across sequential tasks\. The framework details are illustrated in Figure 1\.

### 4\.1Graph Representation via Optimal Transport

The primary objective of our framework is to maximize plasticity on new tasks\. Specifically, we expect to fully adapt the current input graph for acquiring high\-quality knowledge\. To this end, we design a replay\-based graph representation learning framework based on optimal transport mechanism, which explicitly aligns relational structures between the input graph and the learned representation space, enabling structure\-aware graph embeddings\.

#### Graph Augmentation\.

For task𝒯t\\mathcal\{T\}\_\{t\}, the training input data consists of the graph𝒢t=\(𝒱t,ℰt\)\\mathcal\{G\}\_\{t\}=\(\\mathcal\{V\}\_\{t\},\\mathcal\{E\}\_\{t\}\)and a set of historical subgraphs\{𝒢τsub\}τ=1t−1\\\{\\mathcal\{G\}\_\{\\tau\}^\{\\textit\{sub\}\}\\\}\_\{\\tau=1\}^\{t\-1\}retrieved from the fixed\-size memory bufferℬ\\mathcal\{B\}, whereτ∈\{1,…,t−1\}\\tau\\in\\\{1,\\ldots,t\-1\\\}indexes the prior tasks\. Each historical subgraph𝒢τsub\\mathcal\{G\}\_\{\\tau\}^\{\\textit\{sub\}\}is constructed via layer\-wise neighbor sampling to a fixed budget, ensuring bounded memory usage while preserving local structural information\. During training, the current graph and replayed subgraphs are consolidated into batched graph𝒢tbat\\mathcal\{G\}\_\{t\}^\{\\textit\{bat\}\}via the disjoint union operation\. For simplicity, we omit the task index in the following and denote the batched graph as𝒢bat\\mathcal\{G\}^\{\\textit\{bat\}\}\. For the batched graph𝒢bat\\mathcal\{G\}^\{\\textit\{bat\}\}, we construct two augmented views,𝒢1bat\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\}and𝒢2bat\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}, via a stochastic augmentation function that randomly drops edges and masks node features\. Formally, we define two views as𝒢1bat=\(𝐀1,𝐗1,μ\)\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\}=\(\\mathbf\{A\}\_\{1\},\\mathbf\{X\}\_\{1\},\\mu\)and𝒢2bat=\(𝐀2,𝐗2,ν\)\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}=\(\\mathbf\{A\}\_\{2\},\\mathbf\{X\}\_\{2\},\\nu\), whereμ\\muandν\\nuare the empirical distributions of nodes in𝒢1bat\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\}and𝒢2bat\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}, respectively\.𝐗1∈ℝn×d\\mathbf\{X\}\_\{1\}\\in\\mathbb\{R\}^\{n\\times d\}and𝐗2∈ℝm×d\\mathbf\{X\}\_\{2\}\\in\\mathbb\{R\}^\{m\\times d\}denote their feature matrices\.𝐀1∈ℝn×n\\mathbf\{A\}\_\{1\}\\in\\mathbb\{R\}^\{n\\times n\}and𝐀2∈ℝm×m\\mathbf\{A\}\_\{2\}\\in\\mathbb\{R\}^\{m\\times m\}are adjacency matrices\.

The underlying graph topology provides a valuable source of structure\-level supervision for representation learning\. However, directly leveraging the structural signal is non\-trivial\. Since node embeddings and graph topology reside in different metric spaces, it is challenging to establish comparable similarity metrics across these spaces\. Optimal transport theory offers a solution for such comparison via the transport plan\. In this work, we aim to find the optimal transport plan for two graphs by minimizing the transport cost \(i\.e\.,optimal transport distance\)\.

The Wasserstein distance \(Equation 1\) requires the two distributions to reside in the same space\. But for graphs, it is difficult to measure the cost between two nodes on different graphs without node label \(attribute\)\. Even if there is some way to get the cost between nodes, the Wasserstein distance cannot consider the edge information\. Therefore, we need to take the edge information into account as follows

𝒟GW​\(𝐀1,𝐀2\)=minπ𝒢∈Π​\(μ,ν\)​∑i,k∈\[\[n\]\]2∑j,l∈\[\[m\]\]2c𝒜​\(𝐀1i​k,𝐀2j​l\)⋅πi​k​πj​l,\\begin\{split\}\\mathcal\{D\}\_\{\\text\{GW\}\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)=\\min\_\{\\pi\_\{\\mathcal\{G\}\}\\in\\Pi\(\\mu,\\nu\)\}\\sum\_\{i,k\\in\[\\\!\[n\]\\\!\]^\{2\}\}\\sum\_\{j,l\\in\[\\\!\[m\]\\\!\]^\{2\}\}\\\\ c\_\{\\mathcal\{A\}\}\(\\mathbf\{A\}\_\{1\}^\{ik\},\\mathbf\{A\}\_\{2\}^\{jl\}\)\\cdot\\pi\_\{ik\}\\pi\_\{jl\},\\end\{split\}\(3\)abbreviated as

𝒟GW​\(𝐀1,𝐀2\)=minπ𝒢∈Π​\(μ,ν\)​⟨ℒ​\(𝐀1,𝐀2\)⊗π,π⟩,\\mathcal\{D\}\_\{\\text\{GW\}\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)=\\underset\{\\pi\_\{\\mathcal\{G\}\}\\in\\Pi\(\\mu,\\nu\)\}\{\\mathrm\{min\}\}\\left\\langle\\mathcal\{L\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)\\otimes\\pi,\\pi\\right\\rangle,\(4\)
whereℒ​\(𝐀1,𝐀2\)\\mathcal\{L\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)is 4\-D tensor andℒ​\(𝐀1,𝐀2\)i​j​k​l=c𝒜​\(𝐀1i​k,𝐀2j​l\)\\mathcal\{L\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)\_\{ijkl\}=c\_\{\\mathcal\{A\}\}\(\\mathbf\{A\}\_\{1\}^\{ik\},\\mathbf\{A\}\_\{2\}^\{jl\}\)\. The cost functionc𝒜c\_\{\\mathcal\{A\}\}is defined asc𝒜​\(𝐀1i​k,𝐀2j​l\)=\|𝐀1i​k−𝐀2j​l\|c\_\{\\mathcal\{A\}\}\(\\mathbf\{A\}\_\{1\}^\{ik\},\\mathbf\{A\}\_\{2\}^\{jl\}\)=\|\\mathbf\{A\}\_\{1\}^\{ik\}\-\\mathbf\{A\}\_\{2\}^\{jl\}\|, where\|⋅\|\|\\cdot\|denotes the absolute value operator\.\[\[n\]\]2=\[\[n\]\]×\[\[n\]\]\[\\\!\[n\]\\\!\]^\{2\}=\[\\\!\[n\]\\\!\]\\times\[\\\!\[n\]\\\!\],⊗\\otimesdenotes the tensor\-matrix multiplication\. The distance𝒟GW​\(⋅,⋅\)\\mathcal\{D\}\_\{\\text\{GW\}\}\(\\cdot,\\cdot\)is known as the Gromov\-Wasserstein distance\. Then we consider the complete graph including edge structure and node features\. The fused Gromov–Wasserstein distance between graphs𝒢1bat\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\}and𝒢2bat\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}can be defined as

𝒟FGW​\(𝒢1bat,𝒢2bat\)\\displaystyle\\mathcal\{D\}\_\{\\text\{FGW\}\}\(\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\},\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}\)=minπ𝒢∈Π​\(μ,ν\)⁡σ​∑i​jc𝒳​\(𝐗1i,𝐗2j\)⋅πi​j𝒢\\displaystyle=\\min\_\{\\pi\_\{\\mathcal\{G\}\}\\in\\Pi\(\\mu,\\nu\)\}\\sigma\\sum\_\{ij\}c\_\{\\mathcal\{X\}\}\(\\mathbf\{X\}\_\{1\}^\{i\},\\mathbf\{X\}\_\{2\}^\{j\}\)\\cdot\\pi^\{\{\\mathcal\{G\}\}\}\_\{ij\}\(5\)\+\(1−σ\)​∑i​j​k​lc𝒜​\(𝐀1i​k,𝐀2j​l\)⋅πi​j𝒢​πk​l𝒢,\\displaystyle\\quad\+\(1\-\\sigma\)\\sum\_\{ijkl\}c\_\{\\mathcal\{A\}\}\(\\mathbf\{A\}\_\{1\}^\{ik\},\\mathbf\{A\}\_\{2\}^\{jl\}\)\\cdot\\pi^\{\\mathcal\{G\}\}\_\{ij\}\\pi^\{\\mathcal\{G\}\}\_\{kl\},which is equivalent to

minπ𝒢∈Π​\(μ,ν\)​⟨σ​𝒦​\(𝐗1,𝐗2\)\+\(1−σ\)​ℒ​\(𝐀1,𝐀2\)⊗π𝒢,π𝒢⟩,\\underset\{\\pi\_\{\\mathcal\{G\}\}\\in\\Pi\(\\mu,\\nu\)\}\{\\mathrm\{min\}\}\\;\\big\\langle\\sigma\\,\\mathcal\{K\}\(\\mathbf\{X\}\_\{1\},\\mathbf\{X\}\_\{2\}\)\+\(1\-\\sigma\)\\,\\mathcal\{L\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)\\otimes\\pi\_\{\\mathcal\{G\}\},\\;\\pi\_\{\\mathcal\{G\}\}\\big\\rangle,\(6\)whereσ∈\[0;1\]\\sigma\\in\[0;1\]represents the trade\-off parameter for adjusting nodes and edges\.

#### Optimal Transport Plan\.

In this work, we aim to learn structure\-aware representations while preserving nodel\-level semantic information\. To this end, we integrate the node features and edge structure on the graph and design a new objective function to align the optimal transport plans derived from the graph space and the representation space\. Specifically, the fused optimal transport planπ𝒢∗∈ℝn×m\\pi\_\{\\mathcal\{G\}\}^\{\*\}\\in\\mathbb\{R\}^\{n\\times m\}between the two graphs𝒢1bat\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\}and𝒢2bat\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}above can be defined by minimizing the fused Gromov\-Wasserstein distance objective:

π𝒢∗=argminπ𝒢∈Π​\(μ,ν\)⟨σ𝒦\(𝐗1,𝐗2\)\+\(1−σ\)ℒ\(𝐀1,𝐀2\)⊗π𝒢,π𝒢⟩\.\\begin\{split\}\\pi^\{\*\}\_\{\\mathcal\{G\}\}=\{\\arg\\min\}\_\{\\pi\_\{\\mathcal\{G\}\}\\in\\Pi\\left\(\\mu,\\nu\\right\)\}\\langle\\sigma\\,\\mathcal\{K\}\(\\mathbf\{X\}\_\{1\},\\mathbf\{X\}\_\{2\}\)\+\\\\ \(1\-\\sigma\)\\,\\mathcal\{L\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)\\otimes\\pi\_\{\\mathcal\{G\}\},\\;\\pi\_\{\\mathcal\{G\}\}\\big\\rangle\.\\end\{split\}\(7\)By tuning the parameterσ\\sigma, we can control the deviation of the learned optimal transport plan between node features and edge structure\.

Then we use the backbone model \(e\.g\., GNNs\) to obtain the node representations of graph\. We can use Equation \(1\) directly to calculate the optimal planπ𝒵∗∈ℝn×m\\pi\_\{\\mathcal\{Z\}\}^\{\*\}\\in\\mathbb\{R\}^\{n\\times\{m\}\}as

π𝒵∗=arg⁡minπ𝒵∈Π​\(μ,ν\)⁡⟨ℛ​\(𝐙1,𝐙2\),π𝒵⟩,\\pi\_\{\\mathcal\{Z\}\}^\{\*\}=\{\\arg\\min\}\_\{\\pi\_\{\\mathcal\{Z\}\}\\in\\Pi\\left\(\\mu,\\nu\\right\)\}\\left\\langle\\mathcal\{R\}\(\\mathbf\{Z\}\_\{1\},\\mathbf\{Z\}\_\{2\}\),\\;\{\\pi\_\{\\mathcal\{Z\}\}\}\\right\\rangle,\(8\)whereℛ​\(𝐙1,𝐙2\)i​j=c𝒵​\(𝐙1i,𝐙2j\)\\mathcal\{R\}\(\\mathbf\{Z\}\_\{1\},\\mathbf\{Z\}\_\{2\}\)\_\{ij\}=c\_\{\\mathcal\{Z\}\}\\\!\\left\(\\mathbf\{Z\}^\{i\}\_\{1\},\\;\\mathbf\{Z\}^\{j\}\_\{2\}\\right\),𝐙1\\mathbf\{Z\}\_\{1\}and𝐙2\\mathbf\{Z\}\_\{2\}denote the node representations corresponding to𝒢1bat\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\}and𝒢2bat\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}, respectively\.

#### Optimal Transport Alignment\.

Obviously, if the node embeddings learned by the encoder can preserve the relational correspondences observed in the graph space, they are likely to yield more informative representations\. Therefore, we force the encoder to preserve matching relationships in graph space by aligning the optimal transport plan defined on the input graphs with that computed from their node embeddings\. We minimize the discrepancy between the optimal transport plans in the two spaces as the alignment loss as follows

ℒm​a​t=Θ​\(𝝅𝒢∗,𝝅𝒵∗\),\\mathcal\{L\}\_\{mat\}=\\Theta\(\\boldsymbol\{\\pi\}\_\{\\mathcal\{G\}\}^\{\*\},\\boldsymbol\{\\pi\}\_\{\\mathcal\{Z\}\}^\{\*\}\),\(9\)
whereΘ​\(⋅,⋅\)\\Theta\(\\cdot,\\cdot\)denotes a discrepancy measure, such as the KL divergence or the Frobenius norm\.

Additionally, to guide the encoder to learn a representation retaining the structural information of the input graph, we calibrate the cost matrixℛ​\(𝐙1,𝐙2\)\\mathcal\{R\}\(\\mathbf\{Z\}\_\{1\},\\mathbf\{Z\}\_\{2\}\), which captures the implicit relational structure between nodes in the representation space, as follows

ℒs​t​r=Θ\(σ𝒦\(𝐗1,𝐗2\)\+\(1−σ\)ℒ\(𝐀1,𝐀2\)\\displaystyle\\mathcal\{L\}\_\{str\}=\\Theta\\\!\\Big\(\\sigma\\,\\mathcal\{K\}\(\\mathbf\{X\}\_\{1\},\\mathbf\{X\}\_\{2\}\)\+\(1\-\\sigma\)\\,\\mathcal\{L\}\(\\mathbf\{A\}\_\{1\},\\mathbf\{A\}\_\{2\}\)\(10\)⊗π𝒢∗,ℛ\(𝐙1,𝐙2\)\)\.\\displaystyle\\otimes\\pi\_\{\\mathcal\{G\}\}^\{\*\},\\mathcal\{R\}\(\\mathbf\{Z\}\_\{1\},\\mathbf\{Z\}\_\{2\}\)\\Big\)\.
As a result, the overall intra\-task graph transport alignment loss is defined as

ℒg​t​a=ℒm​a​t\+α​ℒs​t​r,\\mathcal\{L\}\_\{gta\}=\\mathcal\{L\}\_\{mat\}\+\\alpha\\mathcal\{L\}\_\{str\},\(11\)whereα\\alphais the trade\-off parameter\. The alignment loss encourages the encoder to learn the relational structure of the input graph in the learned node embeddings\.

### 4\.2Cross\-task Knowledge Distillation

Continuous optimization on new tasks inevitably induces representation drift, causing the structural knowledge \(i\.e\., relational dependencies\) acquired from prior tasks to deteriorate\. To consolidate past and present knowledge, we design a cross\-task knowledge distillation mechanism, regulating the changes of relational structure\. Formally, letξt−1\\xi\_\{t\-1\}denote the parameters of the encoder after completing task𝒯t−1\\mathcal\{T\}\_\{t\-1\}, which is then frozen to serve as a reference teacher modelℱξt−1\\mathcal\{F\}\_\{\\xi\_\{t\-1\}\}for the current task𝒯t\\mathcal\{T\}\_\{t\}\. Given the two augmented graphs𝒢1bat\\mathcal\{G\}\_\{1\}^\{\\textit\{bat\}\}and𝒢2bat\\mathcal\{G\}\_\{2\}^\{\\textit\{bat\}\}, we compute a reference optimal transport planπ𝒵t−1,∗\\pi\_\{\\mathcal\{Z\}\}^\{t\-1,\*\}using the teacher modelℱξt−1\\mathcal\{F\}\_\{\\xi\_\{t\-1\}\}\. The cross\-task structural knowledge distillation loss is defined as

ℒs​k​d=Θ​\(π𝒵t−1,∗,π𝒵t,∗\),\\mathcal\{L\}\_\{skd\}=\\Theta\\\!\\left\(\\pi\_\{\\mathcal\{Z\}\}^\{t\-1,\*\},\\pi\_\{\\mathcal\{Z\}\}^\{t,\*\}\\right\),\(12\)whereΘ​\(⋅,⋅\)\\Theta\(\\cdot,\\cdot\)denotes the divergence measure between optimal transport plans\.

To this end, the overall loss function is composed of graph transport alignment lossℒg​t​a\\mathcal\{L\}\_\{gta\}and distillation lossℒs​k​d\\mathcal\{L\}\_\{skd\}as follows

ℒt​o​t​a​l=ℒg​t​a\+β​ℒs​k​d=ℒm​a​t\+α​ℒs​t​r\+β​ℒs​k​d,\\mathcal\{L\}\_\{total\}=\\mathcal\{L\}\_\{gta\}\+\\beta\\mathcal\{L\}\_\{skd\}=\\mathcal\{L\}\_\{mat\}\+\\alpha\\mathcal\{L\}\_\{str\}\+\\beta\\mathcal\{L\}\_\{skd\},\(13\)whereβ\\betacontrols the strength of cross\-task structural regularization\. Under the guidance of the overall objective function, SAOT integrates newly acquired representations with previously learned structural knowledge in the representation space, and progressively consolidates them into the model\.

Table 1:Statistics of datasets and task splittings\.

## 5Experiments

### 5\.1Datasets

Following the Continual Graph Learning Benchmark \(CGLB\)\(Zhanget al\.,[2022b](https://arxiv.org/html/2607.00377#bib.bib7)\), we conduct experiments on four public benchmark datasets: CoraFull\-CL, Arxiv\-CL, Reddit\-CL, and Products\-CL\. For each dataset, node classes are split into a sequence of disjoint tasks with a fixed order, where each task contains two classes, and the original graph is accordingly split into task\-specific subgraphs without inter\-task edges\. In the continual learning setting, an increasing number of tasks leads to more severe forgetting\. The setting with the maximum number of tasks allows us to thoroughly assess the limitations of different CGL methods\. Detailed dataset statistics and task splittings are summarized in Table 1\.

Table 2:Performance comparison under the Class\-IL setting without inter\-task edges\.Table 3:Performance comparison under the Task\-IL setting without inter\-task edges\.
### 5\.2Baselines

To demonstrate the effectiveness of SAOT, we compare with representative baselines from two distinct categories\.Self\-supervised CGL Methods:RieGrace\(Sunet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib19)\)and TRACE\(Penget al\.,[2025](https://arxiv.org/html/2607.00377#bib.bib14)\)are standard self\-supervised CGL models\. In addition, we adapt representative self\-supervised graph learning models, i\.e\., GAE\(Kipf and Welling,[2016](https://arxiv.org/html/2607.00377#bib.bib20)\), DGI\(Veličkovićet al\.,[2019](https://arxiv.org/html/2607.00377#bib.bib22)\), Graph Barlow Twins \(G\-BT\)\(Bielaket al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib26)\), and various GCN contrastive variants \(GCN\-Clu, GCN\-Par, GCN\-Comp\)\(Youet al\.,[2020b](https://arxiv.org/html/2607.00377#bib.bib31)\)to the continual learning setting by training on each task sequentially\.Supervised CGL Methods:For a comprehensive comparison, we include supervised approaches, i\.e, TWP\(Liuet al\.,[2021](https://arxiv.org/html/2607.00377#bib.bib11)\), ER\-GNN\(Zhou and Cao,[2021](https://arxiv.org/html/2607.00377#bib.bib6)\), CaT\(Liuet al\.,[2023](https://arxiv.org/html/2607.00377#bib.bib17)\), TACO\(Hanet al\.,[2024](https://arxiv.org/html/2607.00377#bib.bib29)\), and PUMA\(Liuet al\.,[2025](https://arxiv.org/html/2607.00377#bib.bib30)\)\. Additionally, we perform the joint training strategy, where the model is trained on all tasks simultaneously without replay, serving as a empirical upper bound for continual learning\.

### 5\.3Metrics and Parameter Settings

To comprehensively evaluate the performance of CGL models, we follow standard evaluation metrics widely adopted in prior works\(Jinet al\.,[2020](https://arxiv.org/html/2607.00377#bib.bib23); Zhou and Cao,[2021](https://arxiv.org/html/2607.00377#bib.bib6)\)\. We construct a performance matrix𝐌p∈ℝT×T\\mathbf\{M\}^\{p\}\\in\\mathbb\{R\}^\{T\\times T\}, whereMi,jpM^\{p\}\_\{i,j\}denotes the classification accuracy on taskjjafter the model has been trained on the firstiitasks\. Based on𝐌p\\mathbf\{M\}^\{p\}, we adopt two commonly used metrics: Average Performance \(AP, also called average accuracy\) and Average Forgetting \(AF, also called backward transfer\)\. AP measures the overall predictive performance after completing all tasks and is defined asAP=1T​∑i=1TMT,ip\\mathrm\{AP\}=\\frac\{1\}\{T\}\\sum\_\{i=1\}^\{T\}M^\{p\}\_\{T,i\}\. AF quantifies the degree of catastrophic forgetting across tasks and is computed asAF=1T−1​∑i=1T−1\(MT,ip−Mi,ip\)\\mathrm\{AF\}=\\frac\{1\}\{T\-1\}\\sum\_\{i=1\}^\{T\-1\}\\left\(M^\{p\}\_\{T,i\}\-M^\{p\}\_\{i,i\}\\right\)\. A negative AF indicates severe forgetting of previously learned tasks, whereas a positive AF suggests effective knowledge retention\. When models achieve similar AP, a higher AF is preferable\.

#### Implementation Details\.

All experiments are implemented in PyTorch and conducted on NVIDIA L40S GPUs with 48GB memory\. For baseline methods, we follow the experimental settings reported in their original papers\. For the proposed SAOT, we employ a 2\-layer GCN encoder with a hidden dimension of 512 on Arxiv\-CL, CoraFull\-CL, and Products\-CL\. We switch to a 3\-layer GAT encoder with four attention heads on Reddit\-CL due to its much higher edge density and extremely large neighborhoods\. The replay buffer size is set to 800 samples per task for Arxiv\-CL, Products\-CL, and Reddit\-CL, while a smaller budget of 200 samples per task is used for CoraFull\-CL\. The learning rate is tuned via a ”grid\-search” strategy ranging from 1e\-4 to 1e\-2, and all models are optimized by using Adam\(Kingma and Ba,[2015](https://arxiv.org/html/2607.00377#bib.bib33)\)\. All results are averaged over five independent runs with different random seeds, and we report the mean and standard deviation\.

### 5\.4Experimental Results

Table 2 and Table 3 summarize the overall performance of all methods under Class\-IL and Task\-IL settings\. The best results among self\-supervised methods are highlighted in bold, and the second\-best are underlined\. Across all datasets, SAOT consistently achieves the best AP performance while effectively mitigating catastrophic forgetting\.

#### Class\-IL Scenario\.

As shown in Table 2, under the challenging Class\-IL scenario, SAOT outperforms existing self\-supervised CGL baselines on four datasets\. Specifically, SAOT achieves an improvement of over 15% in terms of AP on the large\-scale Products\-CL dataset, where severe class imbalance and dense relational structure make cross\-task structural preservation essential\. SAOT improves the performance by 5% in terms of AP and achieves near\-zero forgetting on CoraFull\-CL\. Furthermore, SAOT attains a score of 98\.5% in terms of AP with minimal forgetting on Reddit\-CL\. Although both CoraFull\-CL and Arxiv\-CL are citation networks, the stronger semantic overlap and structural homophily in Arxiv\-CL result in more severe forgetting under continual learning\. The performance matrices in Figure 2 further demonstrate that SAOT sustains more stable accuracy over time and reduces performance degradation across most datasets\. Notably, SAOT achieves competitive performance across multiple datasets and has a lower forgetting rate than most supervised learning baselines\.

#### Task\-IL Scenario\.

In the Task\-IL setting, where task identities are provided during inference, the performance differences among methods typically narrow\. The comparison results under the Task\-IL scenario are shown in Table 3\. We can observe that SAOT outperforms self\-supervised baselines across four datasets\. Although SAOT performs slightly worse than the TRACE method in AF on the Arxiv\-CL dataset, it significantly outperforms TRACE in terms of AP\. In continual learning scenarios, a desirable objective is to achieve high overall accuracy while preserving performance on previously learned tasks, which is reflected by high AP and positive AF values, as demonstrated by SAOT\.

### 5\.5Ablation Study

We compare SAOT with its variants to validate the contribution of two components\. The designed variants are as follows: \(1\) To verify the importance of cross\-task structural knowledge distillation, we design a variant that removes the distillation, denoted by w/o Cross\-task Distillation, where only intra\-task graph transport alignment is retained\. In addition, to investigate the effect of plan\-level distillation compared to point\-wise constraints, we design two variants that replace the transport plan distillation with embedding\-level objectives based on cosine similarity \(w/ Cosine\) and mean squared error \(w/ MSE\), respectively\. \(2\) To examine the importance of optimal transport alignment, we design a variant referred to as w/o Optimal Transport\. Specifically, the variant preserves the same replay mechanism as SAOT but replaces the OT\-based structural alignment objective with the Graph Barlow Twins objective\(Bielaket al\.,[2022](https://arxiv.org/html/2607.00377#bib.bib26)\)for representation learning\.

The ablation results in Table 4 highlight two key findings: \(1\) Both cross\-task transport\-plan distillation and intra\-task OT alignment are essential, as removing either component leads to clear drops in AP and more negative AF, especially under the challenging Class\-IL setting\. \(2\) Distilling relational structure at the transport\-plan level is more effective than point\-wise feature constraints, where replacing plan\-level distillation with cosine or MSE objectives consistently degrades performance\.

Table 4:Ablation study of SAOT on CoraFull\-CL under both Class\-IL and Task\-IL settings\.
### 5\.6Parameter Sensitivity Analysis

We investigate the sensitivity of hyperparametersα\\alphaandβ\\beta, as shown in Figure 3\. Firstly, we fix the value of parameterβ\\betaand analyze the effect of parameterα\\alphaon performance\. We can see that as the parameterα\\alphaincreases, SAOT shows a clear improvement in terms of AP on CoraFull\-CL\. In contrast, SAOT remains relatively insensitive toα\\alphaon Reddit\-CL and achieves the best performance whenα=0\\alpha=0\. These experimental results indicate limited benefits from intra\-task structural calibration on large graphs\.

We further evaluateβ\\betawith the optimalα\\alphaon two datasets\. SAOT achieves the best trade\-off between AP and AF atβ=0\.6\\beta=0\.6andβ=0\.5\\beta=0\.5on CoraFull\-CL and Reddit\-CL, respectively\. These results indicate that excessively strong cross\-task regularization restricts the plasticity of learning new tasks, which in turn negatively affects stability\. Accordingly, we adoptβ=0\.6\\beta=0\.6for CoraFull\-CL andβ=0\.5\\beta=0\.5for Reddit\-CL\.

### 5\.7Complexity Analysis

The computational complexity of SAOT mainly comes from graph encoding and the structure\-aware optimal transport alignment module\. We analyze the complexity on a single taskTtT\_\{t\}with graph𝒢t=\(𝒱t,ℰt\)\\mathcal\{G\}\_\{t\}=\(\\mathcal\{V\}\_\{t\},\\mathcal\{E\}\_\{t\}\)\. The graph encoder requires𝒪​\(\|𝒱t\|​H2\+\|ℰt\|​H\)\\mathcal\{O\}\(\|\\mathcal\{V\}\_\{t\}\|H^\{2\}\+\|\\mathcal\{E\}\_\{t\}\|H\), whereHHis the embedding dimension\. For the structure\-aware optimal transport alignment, computing the pairwise transport cost over the full graph requires𝒪​\(\|𝒱t\|2​H\)\\mathcal\{O\}\(\|\\mathcal\{V\}\_\{t\}\|^\{2\}H\), while the Sinkhorn optimization introduces an additional𝒪​\(K​\|𝒱t\|2\)\\mathcal\{O\}\(K\|\\mathcal\{V\}\_\{t\}\|^\{2\}\)complexity, whereKKdenotes the number of Sinkhorn iterations\. As a result, the transport alignment module has quadratic time and memory complexity with respect to the number of graph nodes, making full\-graph alignment inefficient for large\-scale datasets such as Reddit\-CL and Products\-CL\. To improve scalability, SAOT adopts a point\-cloud sampling mechanism that restricts the number of aligned nodes to a fixed budgetMM\. The practical complexity of the transport alignment module is reduced to𝒪​\(M2​H\+K​M2\)\\mathcal\{O\}\(M^\{2\}H\+KM^\{2\}\), whereM≪\|𝒱t\|M\\ll\|\\mathcal\{V\}\_\{t\}\|in practice\. Figure 4 shows the relation between Class\-IL performance of various self\-supervised methods and the average running time per task\. SAOT achieves a favorable balance between effectiveness and efficiency\.

![Refer to caption](https://arxiv.org/html/2607.00377v1/performance_matrix_heatmap.jpg)Figure 2:Performance matrices of TRACE \(top\) and SAOT \(bottom\) on Arxiv\-CL, CoraFull\-CL, and Reddit\-CL under the Class\-IL setting\.![Refer to caption](https://arxiv.org/html/2607.00377v1/new_parameter_sensitivity_black_axis.jpg)Figure 3:Analysis of hyperparametersα\\alphaandβ\\betaon CoraFull\-CL and Reddit\-CL datasets under Class\-IL setting\.![Refer to caption](https://arxiv.org/html/2607.00377v1/x2.png)Figure 4:The trade\-off between AP scores and average running time per task on Arxiv\-CL and Reddit\-CL datasets under Class\-IL setting\.

## 6Conclusion

In this paper, we propose SAOT, a novel self\-supervised framework for CGL that leverages the optimal transport mechanism to capture and preserve relational structure among nodes\. SAOT constructs optimal transport plans as structure\-level references, guiding the encoder to learn high\-quality and structure\-aware representations\. Simultaneously, we incorporate cross\-task knowledge distillation to mitigate structural drift caused by continual learning\. Extensive experiments show that SAOT consistently outperforms existing self\-supervised baselines in terms of AP under both Task\-IL and Class\-IL settings, and achieves competitive performance compared to supervised methods\.

## Acknowledgements

This work is supported in part by National Natural Science Foundation of China \(No\. 62371340, 62322203\), Tianjin Natural Science Foundation Project \(No\. 24JCZDJC00820, 23JCYBJC00520\), Tianjin Measurement Science and Technology Project \(NO\.2024TJMT061\)\.

## Impact Statement

This work provides a novel self\-supervised learning approach for continual graph learning in real\-world scenario, and it aims to promote the development of continual learning in machine learning\. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here\.

## References

- S\. Babakniya, Z\. Fabian, C\. He, M\. Soltanolkotabi, and S\. Avestimehr \(2024\)A data\-free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks\.InAdvances in Neural Information Processing Systems,,pp\. 66408–66425\.Cited by:[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1)\.
- P\. Bielak, T\. Kajdanowicz, and N\. V\. Chawla \(2022\)Graph Barlow twins: a self\-supervised representation learning framework for graphs\.Knowledge\-Based Systems256,pp\. 109631\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1),[§5\.5](https://arxiv.org/html/2607.00377#S5.SS5.p1.1)\.
- J\. Cai, X\. Wang, C\. Guan, Y\. Tang, J\. Xu, B\. Zhong, and W\. Zhu \(2022\)Multimodal continual graph learning with neural architecture search\.InProceedings of the ACM Web Conference 2022,pp\. 1292–1300\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1)\.
- H\. Cha, J\. Lee, and J\. Shin \(2021\)Co2L: contrastive continual learning\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 9516–9525\.Cited by:[§3\.2](https://arxiv.org/html/2607.00377#S3.SS2.p1.8)\.
- A\. Gomez\-Villa, B\. Twardowski, K\. Wang, and J\. V\. de Weijer \(2024\)Plasticity\-optimized complementary networks for unsupervised continual learning\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision,Vol\.,,pp\. 1690–1700\.Cited by:[§3\.2](https://arxiv.org/html/2607.00377#S3.SS2.p3.9)\.
- A\. Gomez\-Villa, B\. Twardowski, L\. Yu, A\. D\. Bagdanov, and J\. V\. de Weijer \(2022\)Continually learning self\-supervised representations with projected functional regularization\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 3867–3877\.Cited by:[§3\.2](https://arxiv.org/html/2607.00377#S3.SS2.p1.8)\.
- I\. J\. Goodfellow, M\. Mirza, D\. Xiao, A\. Courville, and Y\. Bengio \(2014\)An empirical investigation of catastrophic forgetting in gradient\-based neural networks\.InProceedings of the 2nd International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1)\.
- W\. Hamilton, Z\. Ying, and J\. Leskovec \(2017\)Inductive representation learning on large graphs\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1)\.
- X\. Han, Z\. Feng, and Y\. Ning \(2024\)A topology\-aware graph coarsening framework for continual graph learning\.InAdvances in Neural Information Processing Systems,,pp\. 132491–132523\.Cited by:[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- W\. Jin, T\. Derr, H\. Liu, Y\. Wang, S\. Wang, Z\. Liu, and J\. Tang \(2020\)Self\-supervised learning on graphs: deep insights and new direction\.External Links:2006\.10141Cited by:[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1),[§5\.3](https://arxiv.org/html/2607.00377#S5.SS3.p1.7)\.
- D\. P\. Kingma and J\. Ba \(2015\)Adam: a method for stochastic optimization\.InProceedings of the 3rd International Conference on Learning Representations,Vol\.,San Diego, CA,pp\.\.Cited by:[§5\.3](https://arxiv.org/html/2607.00377#S5.SS3.SSS0.Px1.p1.1)\.
- T\. N\. Kipf and M\. Welling \(2016\)Variational Graph Auto\-Encoders\.External Links:1611\.07308Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- T\. N\. Kipf and M\. Welling \(2017\)Semi\-supervised classification with graph convolutional networks\.InProceedings of the 5th International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1)\.
- J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska, D\. Hassabis, C\. Clopath, D\. Kumaran, and R\. Hadsell \(2017\)Overcoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1)\.
- H\. Lee, H\. Cho, H\. Kim, D\. Kim, D\. Min, J\. Choo, and C\. Lyle \(2024\)Slow and steady wins the race: maintaining plasticity with hare and tortoise networks\.InForty\-first International Conference on Machine Learning,Vol\.,,pp\. 26416–26438\.Cited by:[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1)\.
- N\. Lee, J\. Lee, and C\. Park \(2022\)Augmentation\-free self\-supervised learning on graphs\.InProceedings of the AAAI Conference on Artificial Intelligence,pp\. 7372–7380\.Cited by:[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1)\.
- H\. Liu, Y\. Yang, and X\. Wang \(2021\)Overcoming catastrophic forgetting in graph neural networks\.InProceedings of the AAAI Conference on Artificial Intelligence,pp\. 8653–8661\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1),[§1](https://arxiv.org/html/2607.00377#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- Y\. Liu, R\. Qiu, and Z\. Huang \(2023\)CaT: balanced continual graph learning with graph condensation\.InProceedings of the 2023 IEEE International Conference on Data Mining,pp\. 1157–1162\.Cited by:[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- Y\. Liu, R\. Qiu, Y\. Tang, H\. Yin, and Z\. Huang \(2025\)PUMA: efficient continual graph learning for node classification with graph condensation\.IEEE Transactions on Knowledge and Data Engineering37\(1\),pp\. 449–461\.Cited by:[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- M\. McCloskey and N\. J\. Cohen \(1989\)Catastrophic interference in connectionist networks: the sequential learning problem\.InPsychology of Learning and Motivation,G\. H\. Bower \(Ed\.\),pp\. 109–165\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1)\.
- Z\. Peng, X\. Hua, J\. Hao, Q\. Lin, B\. Dong, and C\. Shen \(2025\)Self\-supervised continual graph learning via adaptive spaced replay on node proxies\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining,Vol\.2,pp\. 2303–2314\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p3.1),[§1](https://arxiv.org/html/2607.00377#S1.p4.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p2.1),[§3\.2](https://arxiv.org/html/2607.00377#S3.SS2.p3.9),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- L\. Sun, J\. Ye, H\. Peng, F\. Wang, and P\. S\. Yu \(2023\)Self\-supervised continual graph learning in adaptive Riemannian spaces\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.37,pp\. 4633–4642\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p3.1),[§1](https://arxiv.org/html/2607.00377#S1.p4.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p2.1),[§3\.2](https://arxiv.org/html/2607.00377#S3.SS2.p1.8),[§3\.2](https://arxiv.org/html/2607.00377#S3.SS2.p3.9),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- S\. Thakoor, C\. Tallec, M\. G\. Azar, M\. Azabou, E\. L\. Dyer, R\. Munos, P\. Veličković, and M\. Valko \(2022\)Large\-scale representation learning on graphs via bootstrapping\.InProceedings of the 10th International Conference on Learning Representations,Cited by:[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1)\.
- P\. Veličković, G\. Cucurull, A\. Casanova, A\. Romero, P\. Liò, and Y\. Bengio \(2018\)Graph attention networks\.InProceedings of the 6th International Conference on Learning Representations \(ICLR 2018\),Vancouver, Canada\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1)\.
- P\. Veličković, W\. Fedus, W\. L\. Hamilton, P\. Liò, Y\. Bengio, and R\. D\. Hjelm \(2019\)Deep graph infomax\.InProceedings of the 7th International Conference on Learning Representations \(ICLR 2019\),New Orleans, LA, USA\.Cited by:[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- J\. Wang, G\. Song, Y\. Wu, and L\. Wang \(2020\)Streaming graph neural networks via continual learning\.InProceedings of the 29th ACM International Conference on Information & Knowledge Management,pp\. 1515–1524\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1)\.
- Y\. Wang, Y\. Zhao, D\. Z\. Wang, and L\. Li \(2023\)GALOPA: graph transport learning with optimal plan alignment\.InAdvances in Neural Information Processing Systems,pp\. 9117–9130\.Cited by:[§3\.1](https://arxiv.org/html/2607.00377#S3.SS1.p1.10)\.
- Z\. Wu, S\. Pan, F\. Chen, G\. Long, C\. Zhang, and P\. S\. Yu \(2021\)A comprehensive survey on graph neural networks\.IEEE Transactions on Neural Networks and Learning Systems32\(1\),pp\. 4–24\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1)\.
- Y\. You, T\. Chen, Y\. Sui, T\. Chen, Z\. Wang, and Y\. Shen \(2020a\)Graph contrastive learning with augmentations\.InAdvances in Neural Information Processing Systems,pp\. 5812–5823\.Cited by:[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1)\.
- Y\. You, T\. Chen, Z\. Wang, and Y\. Shen \(2020b\)When does self\-supervision help graph convolutional networks?\.InInternational Conference on Machine Learning,,pp\. 10871–10880\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p3.1),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1)\.
- X\. Zhang, D\. Song, and D\. Tao \(2022a\)Sparsified subgraph memory for continual graph representation learning\.InProceedings of the 2022 IEEE International Conference on Data Mining,pp\. 1335–1340\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1)\.
- X\. Zhang, D\. Song, Y\. Chen, and D\. Tao \(2024\)Topology\-aware embedding memory for continual learning on expanding networks\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 4326–4337\.Cited by:[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1)\.
- X\. Zhang, D\. Song, and D\. Tao \(2022b\)CGLB: benchmark tasks for continual graph learning\.InAdvances in Neural Information Processing Systems,pp\. 13006–13021\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1),[§5\.1](https://arxiv.org/html/2607.00377#S5.SS1.p1.1)\.
- X\. Zhang, D\. Song, and D\. Tao \(2023\)Hierarchical prototype networks for continual graph representation learning\.IEEE Transactions on Pattern Analysis and Machine Intelligence45\(4\),pp\. 4622–4636\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1)\.
- F\. Zhou and C\. Cao \(2021\)Overcoming catastrophic forgetting in graph neural networks with experience replay\.InProceedings of the AAAI Conference on Artificial Intelligence,pp\. 4714–4722\.Cited by:[§1](https://arxiv.org/html/2607.00377#S1.p1.1),[§1](https://arxiv.org/html/2607.00377#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00377#S2.SS1.p1.1),[§5\.2](https://arxiv.org/html/2607.00377#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2607.00377#S5.SS3.p1.7)\.
- Y\. Zhu, Y\. Xu, F\. Yu, Q\. Liu, S\. Wu, and L\. Wang \(2020\)Deep graph contrastive representation learning\.External Links:2006\.04131Cited by:[§2\.2](https://arxiv.org/html/2607.00377#S2.SS2.p1.1)\.

## Appendix ADatasets Details

The four benchmark datasets for continual graph learning employed in our experiments are derived from four public data sources: OGB\-Arxiv, OGB\-Products, Reddit, and CoraFull\.

CoraFull:CoraFull is a citation network dataset extended from the original Cora dataset, covering a broader range of computer science literature\. In this article, the dataset consists of 19,793 papers \(NN\), 126,842 citations \(EE\), and 70 class categories \(CC\) based on research fields\. The attribute information of papers is extracted as 8,710\-dimensional sparse Bag\-of\-Words \(BoW\) vectors \(DD\)\. We divide the 70 classes into 35 incremental tasks \(TT\) for the continual learning experiments, with each task containing 2 classes\.

OGB\-Arxiv:OGB\-Arxiv is a directed citation network constructed from the Microsoft Academic Graph \(MAG\), representing Computer Science papers\. This dataset consists of 169,343 papers \(NN\) and 1,166,243 edges \(EE\)\. The nodes are labeled into 40 subject areas \(CC\) \(e\.g\., cs\.AI, cs\.LG\)\. The attribute information is represented by 128\-dimensional feature vectors \(DD\) obtained by averaging the Word2Vec embeddings of titles and abstracts\. We classify the papers into 20 tasks \(TT\) according to their subject areas, where each task consists of 2 classes\.

Reddit:Reddit is a social interaction graph derived from Reddit posts, where nodes represent posts and edges represent user co\-interaction\. A subset of Reddit which contains 227,853 posts \(NN\) and approximately 114\.6 million edges \(EE\) is extracted after filtering\. The original dataset contains 41 communities; we remove one class with insufficient samples to retain 40 classes \(CC\)\. The attribute information of posts is extracted as 602\-dimensional vectors \(DD\) from GloVe word embeddings and metadata\. We divide the communities into 20 tasks \(TT\) for use with experiments\.

OGB\-Products:OGB\-Products is an undirected co\-purchasing network representing products sold on Amazon\. This dataset consists of 2,449,028 products \(NN\) and 61,859,036 edges \(EE\)\. The products are categorized into 47 classes; we exclude one class \(containing only a single node\) to keep 46 classes \(CC\)\. The attribute information is 100\-dimensional \(DD\), generated by applying PCA to bag\-of\-words features from product descriptions\. We divide the products into 23 tasks \(TT\), with each task containing 2 classes\.

Data Resources:

The datasets used in this study are publicly available from the following sources:

OGB\-Arxiv:Available from https://ogb\.stanford\.edu/docs/nodeprop/\#ogbn\-arxiv\.

OGB\-Products:Available from https://ogb\.stanford\.edu/docs/nodeprop/\#ogbn\-products\.

Reddit:Available from https://archive\.org/details/FullRedditSubmissionCorpus2006ThruAugust2015\.

## Appendix BSensitivity Analysis of Replay Buffer Size

To evaluate the dependence of SAOT on the memory buffer capacity, we conduct a sensitivity analysis by varying the replay buffer size\|ℬ\|\|\\mathcal\{B\}\|across all four datasets under both Class\-IL and Task\-IL settings\. We test buffer sizes ranging from 0 \(no replay\) up to 4,000 samples \(depending on the dataset scale\), measuring both Average Performance \(AP\) and Average Forgetting \(AF\)\. The results are illustrated in Figure 4 \(Class\-IL\) and Figure 5 \(Task\-IL\)\.

![Refer to caption](https://arxiv.org/html/2607.00377v1/Class-IL_robustness.png)Figure 5:Sensitivity analysis of buffer size under the Class\-IL setting\.![Refer to caption](https://arxiv.org/html/2607.00377v1/Task-IL_robustness.png)Figure 6:Sensitivity analysis of buffer size under the Task\-IL setting\.#### Low Dependency on Explicit Memory:

In most scenarios, SAOT achieves near\-optimal or optimal performance with zero or minimal replay budgets \(e\.g\., buffer sizes of 0 or 160\)\. Especially in the Task\-IL setting, the performance curves remain remarkably flat regardless of the buffer size\. This indicates that the structural knowledge acquired via optimal transport alignment is intrinsically robust, mitigating forgetting without relying on the rehearsal of historical samples\.

#### Effectiveness of Structural Alignment over Replay:

In the Class\-IL setting, increasing the buffer size does not consistently yield performance gains; for instance, on Products\-CL, performance even fluctuates as the buffer size increases\. This observation reinforces that the effectiveness of SAOT stems from its structure\-aware design rather than rote memorization\. The framework is capable of capturing and preserving essential topological structures with minimal resources, whereas excessive replay might introduce structural noise or inductive bias from the buffered subgraphs\.