From Euclidean to Graph-Structured Data: A Survey of Collaborative Learning

# From Euclidean to Graph-Structured Data:A Survey of Collaborative Learning
Source: [https://arxiv.org/html/2609.02984](https://arxiv.org/html/2609.02984)
Rémi Bourgerieremibo@kth\.seAffiliation:School of Electrical Engineering and Computer Science, and Digital FuturesAffiliation:KTH Royal Institute of TechnologyAffiliation:Stockholm, SwedenŠarūnas Girdzijauskassarunasg@kth\.seAffiliation:School of Electrical Engineering and Computer Science, and Digital FuturesAffiliation:KTH Royal Institute of TechnologyAffiliation:Stockholm, SwedenViktoria Fodorvfodor@kth\.seAffiliation:School of Electrical Engineering and Computer Science, and Digital FuturesAffiliation:KTH Royal Institute of TechnologyAffiliation:Stockholm, Sweden

###### Abstract

The conventional approach to machine learning, that is, collecting data, training models, and performing inference in a single location, faces fundamental limitations, including scalability and privacy, that restrict its applicability\. To address these challenges, recent research has exploredcollaborative learningapproaches, includingfederated learninganddecentralized learning, where individual agents perform training and inference locally, with limited collaboration\. Most collaborative learning research focuses on Euclidean data with regular, grid\-like structure \(e\.g\., images, text\)\. However, these approaches fail to capture the relational patterns in many real\-world applications, best represented by graphs\. Learning on graphs relies on message\-passing mechanisms to propagate information between connected nodes, making it conceptually well\-suited for collaborative environments where agents must exchange information\. Yet, the opportunities and challenges of learning on graph\-structured data in collaborative settings remain largely underexplored\. This survey provides a comprehensive investigation of collaborative learning from Euclidean to graph\-structured data, aiming to consolidate this emerging field\. We begin by reviewing its foundational principles for Euclidean data, organizing them along three core dimensions: learning effectiveness, efficiency, and privacy preservation\. We then extend the discussion to graph\-structured data, introducing a taxonomy of graph distribution scenarios, characterizing associated statistical heterogeneities, and developing standardized problem formulations and algorithmic frameworks\. Finally, we systematically identify open challenges and promising research directions\. By bridging established techniques for Euclidean data with emerging methods for graph learning, our survey provides researchers and practitioners with a well\-structured foundation of collaborative learning, supporting further development across a wide range of scientific and industrial fields\.Resources are available at[https://github\.com/remibourgerie/collaborative\_gnns](https://github.com/remibourgerie/collaborative_gnns)\.

## 1Introduction

Over the past decade, Deep Learning has turned into a success story, largely due to the combination of four factors\([Goodfellow et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib241)\): \(i\) neural networks asuniversalfunction approximators\([Rosenblatt, 1958](https://arxiv.org/html/2609.02984#bib.bib296);[Rumelhart et al\., 1986](https://arxiv.org/html/2609.02984#bib.bib26)\)\(ii\)sample efficienttraining algorithms\([Bottou, 2010](https://arxiv.org/html/2609.02984#bib.bib21)\), \(iii\)parallelizationschemes that align with hardware architectures\([Hooker, 2021](https://arxiv.org/html/2609.02984#bib.bib293)\); and \(iv\) the availability ofmassiveamounts of high\-quality data\.

While the first three factors have been consolidated over the years, the fourth requirement,access to massive datasets, remains a major obstacle\. Indeed, today data ecosystem remains highly fragmented, constrained by privacy legislation\([European Union, 2016](https://arxiv.org/html/2609.02984#bib.bib144)\), data ownership concerns, integration complexity, and the inherent coordination difficulties of multi\-agent systems\. This access to data is partially addressed for LLMs by scaling onpublicweb data\([Naveed et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib407);[Baack, 2024](https://arxiv.org/html/2609.02984#bib.bib304)\), but the fundamental challenge persists: the absence of mechanisms to collect, align, and utilize data across organizational and technical boundaries\.

Collaborative learning, often termed under the umbrella offederatedordecentralized learning, provides a framework to address this challenge by allowing multiple agents to train a shared model directly on their private datasets\. In this framework, only model updates or aggregated information are exchanged between agents rather than entire datasets, drastically reducing the amount of data communicated\. Moreover, by keeping data at its source, this framework holds the promise of privacy preservation\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/tabular_to_graph.jpg)Figure 1:Collaborative Learning from Euclidean to Graph Data\.Collaborative learning refers to a system of agents that collectively train and infer ML models on their local private data\. Collaborative learning has been extensively studied for Euclidean data \(a\), comprising highly popular domains like vision, language, temporal sequences, and measurement data\. Conversely, collaborative learning on graph data \(b\) is an emerging field\. Graphs canonically represent data structured with relations, arising in domains including human behavior, physical systems, chemistry, and information systems\. Compared tocentralized learning, collaborative learning offers three main advantages: enabling training and inference on distributed datasets, communication efficiency, and the promise of privacy preservation of agents data\. Collaborative learning on graph data aligns naturally withMessage Passing Neural Networks \(MPNNs\)\.Collaborative learning solutions can be divided into two categories based on data structure:Euclidean data, which exhibits regular, grid\-like structure \(e\.g\., images as 2D grids, text as 1D sequences\), andnon\-Euclidean data, where relationships are defined by irregular structures such asgraphs\([Bronstein et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib406)\), as illustrated in Figure[1](https://arxiv.org/html/2609.02984#S1.F1)\.

For Euclidean data, typical use cases include cross\-device scenarios involving numerous agents, each possessing limited data insufficient for generating accurate models independently\. These agents may include mobile phones\([Xu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib38)\), autonomous vehicles\([Hellström et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib143)\), mobile base stations\([Zhang et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib13)\), or Internet of Things \(IoT\) devices\([Zhou et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib166)\)\. Cross\-silo applications involve large datasets held by distinct actors who cannot share data to preserve privacy, prevalent in healthcare and finance\([Li et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib332)\)\. Collaboration typically involves exchanging and aggregating model parameters during training, while inference is generally performed locally by individual agents\.

For graph\-structured data, two distinct scenarios emerge\. The first scenario is learning on a large collection of independent graph instances, typical in computational biology \(e\.g\., predicting molecular properties\)\([He et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib250)\), involves learning parameters of a graph\-based model \(e\.g\., a GNN\) using tools conceptually similar to those for Euclidean data, as each graph instance can be treated analogously to a single data sample\. The second scenario arises when agents hold a portion of asharedglobal graph, both inference and learning necessitate information exchange among agents, as the graph topology encodes critical relationships spanning agent boundaries\. Applications include collaborative autonomous agents in IoT systems\([Liu et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib318)\), autonomous vehicles\([Blumenkamp et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib264);[Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\), and privacy\-preserving learning over large\-scale social network or recommender system graphs\([He et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib14);[Dong et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib367);[Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195)\)\.

While these diverse applications demonstrate broad potential, their implementation presents significant challenges that differ fundamentally between Euclidean and non\-Euclidean settings\. Modern machine learning \(ML\) relies predominantly on iterative optimization algorithms\([Bottou et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib23)\), which have long posed difficulties for distributed systems\([Bertsekas, 2011](https://arxiv.org/html/2609.02984#bib.bib110);[Ram et al\., 2009b](https://arxiv.org/html/2609.02984#bib.bib111);[Low et al\., 2010](https://arxiv.org/html/2609.02984#bib.bib99);[Xing et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib88)\)\. These challenges are exacerbated in collaborative learning due to the substantial size of the models that need to be communicated and inherent system discrepancies, driving active research in system architecture and algorithm design\([Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18);[Daly et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib75)\)\.

For graph\-structured data, the challenges are further amplified by the non\-Euclidean nature of the data\. The graph topology is often captured byMessage Passing Neural Networks\(MPNNs\)\([Gilmer et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib3);[Hamilton et al\., 2017b](https://arxiv.org/html/2609.02984#bib.bib392)\), where the nodes iteratively exchange and aggregate messages with their neighbors\. These MPNNs are inherently distributed iterative algorithms, which makes them conceptually well\-suited for collaborative settings\. However, when each agent holds a partition of the shared global graph and cannot freely share data, the requirement for information to propagate through the graph topology creates novel challenges beyond those in Euclidean settings, leading to a rapidly expanding area of research\.

### 1\.1The Objectives of the Survey and Related Work

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/3pillars.jpg)Figure 2:Collaborative Learning on graph data and its three pillars\.Collaborative learning on graph data relies on three distinct pillars: diffusion algorithm on networks, graph representation learning, and collaborative learning\.While existing surveys have extensively coveredfederated learningon Euclidean data, and to a lesser extent on graph data, none have systematically analyzed their intersection\. This survey bridges that gap by providing a unified view that maps the design space of collaborative learning from Euclidean to graph\-structured data\. We outline how different approaches relate to each other and identify research challenges that remain unanswered for the less understood case of graphs\. Crucially, while methods for Euclidean data are well\-established, graph\-structured data introduces unique complexities that draw upon knowledge from multiple domains\. Collaborative learning for graph data stands on three foundational pillars, borrowing from distinct research areas as illustrated in Figure[2](https://arxiv.org/html/2609.02984#S1.F2):

- •Diffusion Algorithms on Networksare distributed iterative methods that compute node\-level or network\-level functions through local message passing\. These methods were initially developed for distributed decision problems in networked systems \(e\.g\., multi\-agent systems, sensor networks\)\([Tsitsiklis et al\., 1986](https://arxiv.org/html/2609.02984#bib.bib31);[Olfati\-Saber et al\., 2007](https://arxiv.org/html/2609.02984#bib.bib9)\), and later adopted in large\-scale network analytics and learning through closely related random\-walk\-based processes \(e\.g\., ranking, recommendation, semi\-supervised inference on graphs\)\([Zhou et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib11)\)\.
- •Graph Representation Learningemploys deep neural architectures to learn representations from graph\-structured data\([Kipf and Welling, 2017](https://arxiv.org/html/2609.02984#bib.bib390)\)\. Applications where relational structure is intrinsic \(e\.g\., social networks, molecular modeling, infrastructure systems\) have driven this area, which emerged at the intersection of graph signal processing and machine learning\([Bruna et al\., 2013](https://arxiv.org/html/2609.02984#bib.bib432)\)\.
- •Collaborative Learning, for examplefederated learning\([McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335)\), allows training and inference on private distributed datasets through centralized or decentralized approaches\. Collaborative learning is motivated in two key practical cases: when sensitive data cannot be shared between actors \(healthcare, finance, user data\) and when natively distributed data cannot be collected due to communication constraints \(robotics, IoT\)\. The supporting disciplines span machine learning, distributed optimization, information theory, and communication systems\.
- •Collaborative learning on Graph Data\([Zhang et al\., 2021d](https://arxiv.org/html/2609.02984#bib.bib180)\)studies learning on private distributed graph\-structured data\. It intersects the three areas above, by combining the message\-passing principles of diffusion algorithms on networks, the neural architectures of graph representation learning, and the distributed optimization frameworks of collaborative learning\.

This survey connects these three domains to examine how established results can accelerate research in emerging fields and novel applications\. Table[1](https://arxiv.org/html/2609.02984#S1.T1)presents representative surveys and foundational papers across these pillars, providing comprehensive details that were necessarily excluded to maintain focus\. The table also includes works on parallel processing of big data, an area with partial overlap with collaborative learning\. To limit the scope of the survey, we focus on supervised learning\. Interested readers may find references for collaborative unsupervised and self\-supervised learning in\([Uludag et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib456);[Zhang et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib457);[Ji et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib455)\), for multi\-agent reinforcement learning in\([Gronauer and Diepold, 2022](https://arxiv.org/html/2609.02984#bib.bib329);[Chen et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib317)\), and for reinforcement learning on graph\-structured data in\([Nie et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib458);[Liu et al\., 2024c](https://arxiv.org/html/2609.02984#bib.bib459)\)\.

TopicRepresentative WorksDiffusion Algorithms on NetworksDistributed averaging and consensus\([Bertsekas and Tsitsiklis, 2015](https://arxiv.org/html/2609.02984#bib.bib373);[Olfati\-Saber et al\., 2007](https://arxiv.org/html/2609.02984#bib.bib9)\)\([Yang et al\., 2019c](https://arxiv.org/html/2609.02984#bib.bib438)\)Label propagation\([Zhou et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib11)\)Graph Representation LearningGraph Neural Architectures\([Kipf and Welling, 2017](https://arxiv.org/html/2609.02984#bib.bib390)\)\([Wu et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib27)\)Graph Foundation Models\([Wang et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib285)\)Collaborative Learning on Euclidean DataFederated Learning\([McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335)\)\([Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18)\),\([Zhang et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib141);[Yang et al\., 2019b](https://arxiv.org/html/2609.02984#bib.bib343);[Liu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib139)\)Vertical Federated Learning\([Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\)Data heterogeneity and personalization\([Zhu et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib127);[Tan et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib338);[Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\)Privacy and security\([Lyu et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib48);[Daly et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib75)\)Parallel training\([Xing et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib88)\)Machine Learning in wireless networks\([Hellström et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib143)\)Collaborative Learning on Graph DataFederated Learning with GNNs\([Zhang et al\., 2021d](https://arxiv.org/html/2609.02984#bib.bib180)\)\([He et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib14);[Zhang et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib337)\)\([Liu et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib168);[Fu et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib197)\)Applications and use cases\([He et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib14);[Dong et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib367)\)GNNs in wireless communications\([He et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib244);[Lee et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib246)\)Parallel training for large GNNs\([Shao et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib142);[Lin et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib140);[Liu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib139)\)Table 1:Key surveys and seminal works \(in bold\) on collaborative learning on graph data, including the three pillars ofCollaborative Learning,Diffusion algorithms on Networks, andGraph Representation Learning\.
### 1\.2The Organization of the Paper

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/survey_map.jpg)Figure 3:The structure of the survey\. Our contributions reside in extending and mapping the concepts found for collaborative learning on Euclidean data to collaborative learning on graph data, thereby consolidating insights across these separate research areas\. Our analysis of the design choices focuses on three directions: improving learning and inferenceeffectiveness, improving learning and inferenceefficiency, andprivacy\-preservation techniques\.In this survey, we provide a structured overview of learning from distributed private datasets, including the basic concepts, the research challenges, and proposed solutions, starting from classical collaborative learning, to learning over distributed private graph datasets\. The skeleton of the paper is shown in Figure[3](https://arxiv.org/html/2609.02984#S1.F3)\. In the first part of the paper, we discuss collaborative learning over Euclidean data, that is, classical centralized FL and its distributed counterparts\. We formulate the learning problem, discuss how the data can be partitioned across the agents, and what heterogeneity aspects need to be considered\. Then we discuss proposed solutions \(i\) for effective learning, that is, to achieve models with high accuracy despite the various forms of heterogeneity, \(ii\) for efficiency, in terms of the use of communication and computation resources, and finally, \(iii\) solutions for privacy preservation\. Since collaborative learning over Euclidean data is an established research area, we review the key contributions in this part\.

In the second part of the paper, we follow the same structure to discuss the emerging research field of collaborative learning on graph\-structured data\. The ways data \(or information\) can be distributed are significantly more varied now than in the Euclidean case, so we introduce a taxonomy for data partition\. Then, we follow the structure of the first part to find relevant research results and questions that are still open for the design of collaborative learning systems on graph data\. This part is based on a rigorous survey of all recent papers with keywords*federated graph*and*decentralized graph learning*\. The taxonomy of solutions for effective, efficient, and privacy\-preserving collaborative learning for Euclidean and graph data is provided in Appendix[A](https://arxiv.org/html/2609.02984#A1), together with the notations and the glossary of key terminology used throughout this survey\.

## 2Collaborative Learning on Euclidean Data

The concept ofcollaborative learningemerges from multiple taxonomies, which are all cast under the umbrella term ofdistributed machine learningas outlined in Figure[4](https://arxiv.org/html/2609.02984#S2.F4)\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/taxonomy.jpg)Figure 4:Taxonomies of distributed learning and thescope of our survey\.Distributed learning\(a\) is often used as an umbrella term encompassing various scenarios and techniques\. In particular,parallel computing\(b\) aims to accelerate training and/or inference under the assumption that the data can be centralized, and includes bothdata\-parallelismandmodel\-parallelismapproaches\. Our focus, however, is on a different scenario that we termcollaborative learning\(c\), which addresses cases where the data remains distributed and private to its owner, covering bothfederated learninganddecentralized learning\.The termdistributed machine learningoriginally served as a broad label for methods designed to train and/or infer ML models on massive datasets by distributing computations across multiple machines, often in a cluster\([Xing et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib88);[Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18)\)\. Distribution is then largely associated with parallelization, combiningdata parallelism, where datasets are divided among machines, andmodel parallelism, where parts of the model are divided among machines and updated in parallel\.

The termcollaborative learningrefers specifically to scenarios where local data remains strictly private to its owner\. For clarity, we assume that local data is generated locally at multiple agents, where it persists throughout training and inference phases while remaining exclusively accessible to the data owner, a case commonly termedfederated learningordecentralized learningin the literature\. This privacy constraint necessitates precise problem formulation and distinct design methodologies to address real\-world implementation challenges, which we develop in this section\. Our focus here is onEuclidean data, defined as data where each sample is represented as a feature vector in an Euclidean space \(in practice, someℝd\\mathbb\{R\}^\{d\}with the standard inner product\)\.

### 2\.1Collaborative Learning: Problem Formulation and the Federated Learning Framework

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/FL.jpg)Figure 5:Federated learning, the basic approach to collaborative learning represented here forKKagents\. Agents possess private local datasetsSkS\_\{k\}\. In each roundtt, they train their local modelFθF\_\{\\theta\}on their local loss and communicate the model parameter updatesθk​\(t\)\\theta\_\{k\}\(t\)\. The central server aggregates in turn the local models into a global model with parametersθ⁡\(t\)\\theta\(t\), which is then shared again with the agents for another roundt\+1t\+1\.##### Problem definition\.

Consider a set of agents, denoted by𝒦=\{1,…,K\}\\mathcal\{K\}=\\\{1,\\ldots,K\\\}, as illustrated in Figure[5](https://arxiv.org/html/2609.02984#S2.F5), where the data of each agent is sampled from an unknown distribution𝒟k\\mathcal\{D\}\_\{k\}on𝒳×𝒴\\mathcal\{X\}\\times\\mathcal\{Y\}where𝒳\\mathcal\{X\}is the feature Euclidean domain and𝒴\\mathcal\{Y\}the label domain\. Given a loss functionℓ:𝒳×𝒴→ℝ\+\\ell:\\mathcal\{X\}\\times\\mathcal\{Y\}\\to\\mathbb\{R\}^\{\+\}, each agent seeks to learn a mappingF:𝒳→𝒴F:\\mathcal\{X\}\\to\\mathcal\{Y\}among a class of functionℱ\\mathcal\{F\}that minimizes the local population loss

ℒk​\(F\):=𝔼\(x,y\)∼𝒟k​ℓ​\(F⁡\(x\),y\)\.\\mathcal\{L\}\_\{k\}\(F\):=\\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\_\{k\}\}\\ell\(F\(x\),y\)\.\(1\)
If the data distributions𝒟k\\mathcal\{D\}\_\{k\}were known, agents could determine an optimal solutionFk∗∈ℱF\_\{k\}^\{\*\}\\in\\mathcal\{F\}, by directly minimizing their respective local loss \([1](https://arxiv.org/html/2609.02984#S2.E1)\) onℱ\\mathcal\{F\}\. However, in practice, the true distributions𝒟k\\mathcal\{D\}\_\{k\}are unknown\. Instead, agents rely on empirical samples forming local datasetsSkS\_\{k\}, drawn i\.i\.d\. from𝒟k\\mathcal\{D\}\_\{k\}\. These datasets allow agents to construct a surrogateℒ^k\\hat\{\\mathcal\{L\}\}\_\{k\}, which represents an empirical approximation of the true lossℒk\\mathcal\{L\}\_\{k\}based on their local data only

ℒ^k​\(F\):=1\|Sk\|​∑\(x,y\)∈Skℓ⁡\(F⁡\(x\),y\)\.\\hat\{\\mathcal\{L\}\}\_\{k\}\(F\):=\\frac\{1\}\{\|S\_\{k\}\|\}\\sum\_\{\(x,y\)\\in S\_\{k\}\}\\ell\(F\(x\),y\)\.\(2\)
This approach provides a local estimateF^k∗∈ℱ\\hat\{F\}\_\{k\}^\{\*\}\\in\\mathcal\{F\}\. However, the local datasetsSkS\_\{k\}are typically too small to accurately approximate the underlying distribution𝒟k\\mathcal\{D\}\_\{k\}\. As a result, the local estimateF^k∗\\hat\{F\}\_\{k\}^\{\*\}poorly approximates the true optimal functionFk∗F\_\{k\}^\{\*\}, leading to high estimation error\([Valiant, 1984](https://arxiv.org/html/2609.02984#bib.bib297);[Bottou and Bousquet, 2007](https://arxiv.org/html/2609.02984#bib.bib300)\)\. To address this issue, a key assumption is that although the distributions𝒟k\\mathcal\{D\}\_\{k\}may differ across agents, they share strong similarities\. Therefore, collaboration among agents enables them to benefit from the aggregated knowledge contained in the union of local datasets\{Sk\}k∈𝒦\\\{S\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}\.

In the most favorable case, when the local distributionsDkD\_\{k\}are the same, an effective solution is to collaborate on buildinga single model common to all agents\. The problem can then be framed as identifyingF∗∈ℱF^\{\*\}\\in\\mathcal\{F\}that minimizes the average of local population losses among all agents

ℒ⁡\(F\):=1K​∑k=1Kℒk​\(F\)\.\\mathcal\{L\}\(F\):=\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\mathcal\{L\}\_\{k\}\(F\)\.\(3\)Since the population lossesℒk\\mathcal\{L\}\_\{k\}are generally uncomputable, \([3](https://arxiv.org/html/2609.02984#S2.E3)\) is typically approximated using local empirical losses\. Most existing work focuses on minimizing \([4](https://arxiv.org/html/2609.02984#S2.E4)\), which is computationally feasible, rather than the true loss

ℒ^​\(F\):=1K​∑k=1Kℒ^k​\(F\)\.\\hat\{\\mathcal\{L\}\}\(F\):=\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\hat\{\\mathcal\{L\}\}\_\{k\}\(F\)\.\(4\)This line of work typically assumes thatF^∗≈F∗\\hat\{F\}^\{\*\}\\approx F^\{\*\},that is, minimizing \([4](https://arxiv.org/html/2609.02984#S2.E4)\) provides a good approximation of minimizing \([3](https://arxiv.org/html/2609.02984#S2.E3)\)\. This assumption holds only when the generalization gap\|ℒ⁡\(F^∗\)−ℒ^​\(F^∗\)\|\|\\mathcal\{L\}\(\\hat\{F\}^\{\*\}\)\-\\hat\{\\mathcal\{L\}\}\(\\hat\{F\}^\{\*\}\)\|is negligible\([Bottou et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib23)\)\.

To minimize \([4](https://arxiv.org/html/2609.02984#S2.E4)\), it can be convenient to parametrizeFθF\_\{\\theta\}withθ∈Θ\\theta\\in\\ThetawhereΘ\\Thetais the domain parameter\. Typically,FFcan be a neural network, parametrized by its learnable weightsθ\\theta\. The problem can be framed as identifyingθ∗\\theta^\{\*\}, a solution of

minθ∈Θ⁡ℒ^​\(F\)​, or equivalently,minθ∈Θ⁡1K​∑k=1Kℒ^k​\(Fθ\)\.\\min\_\{\\theta\\in\\Theta\}\\hat\{\\mathcal\{L\}\}\(F\)\\text\{, or equivalently,\}\\quad\\min\_\{\\theta\\in\\Theta\}\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\hat\{\\mathcal\{L\}\}\_\{k\}\(F\_\{\\theta\}\)\.\(5\)

##### Federated learning framework\.

The basic approach for collaborative learning on distributed datasets is theFederated Learning \(FL\)framework\([Konečný et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib397);[McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335);[Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18)\)\. The key idea of federated learning is to*federate*\([McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335)\)the training of agents through a common round\-based protocol coordinated by a central server, as shown in Figure[5](https://arxiv.org/html/2609.02984#S2.F5)\. At the end of training, each agent receives the jointly trained model, which can be used to perform inference independently on local data\.

##### Federated Averaging \(FedAvg\) protocol\.

The most popular and simplest implementation of FL isFederated Averaging \(FedAvg\)\([McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335)\)detailed in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\. It proceeds inTTcommunication rounds between a leader, referred to as the server, and the agents\. At the beginning of each roundtt, the server randomly selectsKmaxK\_\{\\max\}agents \(line[4](https://arxiv.org/html/2609.02984#alg1.l4)\) and communicates the latest global model parametersθ⁡\(t−1\)\\theta\(t\-1\)to them \(line[5](https://arxiv.org/html/2609.02984#alg1.l5)\)\. Upon reception, each selected agent initializes its temporary local modelθk​\(t−12\)\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)to the received global modelθ⁡\(t−1\)\\theta\(t\-1\)\. Then, each agentkkperformsEElocal update steps \([9](https://arxiv.org/html/2609.02984#S2.Ex1)\) on its local datasetSkS\_\{k\}to minimize its empirical loss \([2](https://arxiv.org/html/2609.02984#S2.E2)\) using theStochastic Gradient Descent \(SGD\)algorithm\([Robbins and Monro, 1951](https://arxiv.org/html/2609.02984#bib.bib22)\)\. After completing the local updates, each agent sends its resulting local modelθk​\(t\)\\theta\_\{k\}\(t\)back to the server \(line[12](https://arxiv.org/html/2609.02984#alg1.l12)\)\. The server then aggregates all received models to compute the updated global modelθ⁡\(t\)\\theta\(t\)via averaging, denoted byθ¯​\(t\)\\bar\{\\theta\}\(t\)\([14](https://arxiv.org/html/2609.02984#S2.Ex2)\)\. Under specific conditions, this iterative process converges toθ∗\\theta^\{\*\}a solution of \([5](https://arxiv.org/html/2609.02984#S2.E5)\)\([Yu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib17);[Stich, 2019](https://arxiv.org/html/2609.02984#bib.bib16)\)\.

Algorithm 1Federated Averaging \(FedAvg\) adapted from\([McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335)\)1:Set of agents

𝒦=\{1,…,K\}\\mathcal\{K\}=\\\{1,\\ldots,K\\\}, local datasets

SkS\_\{k\}, total number of communication rounds

TT, number of local training iterations

EE, number of sampled clients

Km​a​xK\_\{max\}, initial global model

θ⁡\(0\)\\theta\(0\), learning rate

η\\eta
2:foreach round

t=1,…,Tt=1,\\ldots,Tdo

3:Server:

4:Sample uniformly a random subset

𝒦⁡\(t\)⊆𝒦\\mathcal\{K\}\(t\)\\subseteq\\mathcal\{K\}of

mmclients \(

\|𝒦⁡\(t\)\|=𝒦m​a​x\|\\mathcal\{K\}\(t\)\|=\\mathcal\{K\}\_\{max\}\)

5:Distribute global model parameters

θ⁡\(t−1\)\\theta\(t\-1\)to all sampled clients

k∈𝒦tk\\in\\mathcal\{K\}\_\{t\}
6:foreach agent

k∈𝒦⁡\(t\)k\\in\\mathcal\{K\}\(t\)in paralleldo

7:Upon reception of

θ⁡\(t−1\)\\theta\(t\-1\), initialize local model parameter

θk​\(t−12\)=θ⁡\(t−1\)\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)=\\theta\(t\-1\)
8:for

e=1,…,Ee=1,\\ldots,Edo

9:Update local model using gradient descent111In practice and as proposed in the original algorithm\([McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335)\), the datasetSkS\_\{k\}is divided into batches of fixed size and the model is updated with the gradient on each batch of data, elided here for simplicity\.:

θk\(t−12\)←θk\(t−12\)−η∇\(∑\(x,y\)∈Skℓ\(Fθk​\(t−12\)\(x\),y\)\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)\\leftarrow\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)\-\\eta\\nabla\\left\(\\sum\_\{\(x,y\)\\in S\_\{k\}\}\\ell\(F\_\{\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)\}\(x\),y\\right\)
10:endfor

11:endfor

12:Communicate local model

θk​\(t\)=θk​\(t−12\)\\theta\_\{k\}\(t\)=\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)to the server

13:Server:

14:Upon reception of models

\{θk​\(t\)\}k∈𝒦⁡\(t\)\\\{\\theta\_\{k\}\(t\)\\\}\_\{k\\in\\mathcal\{K\}\(t\)\}, average local models:

θ¯​\(t\)=∑k∈𝒦⁡\(t\)nkn​θk​\(t\)\\bar\{\\theta\}\(t\)=\\sum\_\{k\\in\\mathcal\{K\}\(t\)\}\\frac\{n\_\{k\}\}\{n\}\\theta\_\{k\}\(t\)
15:Set global model:

θ​\(t\)←θ¯​\(t\)\\theta\(t\)\\leftarrow\\bar\{\\theta\}\(t\)
16:endfor

17:returnFinal global model

θ⁡\(T\)\\theta\(T\)

### 2\.2Data Partition, Heterogeneity and Privacy Challenges

The basic approach for collaborative learning, as defined in*FedAvg*\(Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\), builds on several unrealistic assumptions\. In the sequel, we discuss the characteristics of collaborative learning that need to be considered to address realistic use cases, that is,data partition,statistical imbalance,system heterogeneity, andprivacy vulnerabilities\.

#### 2\.2\.1Data Partition

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/fl_data_partition.jpg)Figure 6:The different partitions of Euclidean data for a two\-agent case, whereSkS\_\{k\}denotes the dataset of agentkk\. Compared tono partitioning\(a\),horizontal partitioning\(b\) splits the sample spaceℐ\\mathcal\{I\},vertical partitioning\(c\) splits the feature space𝒳\\mathcal\{X\}, andtransfer partitioning\(d\) splits both, with the source agentssholding most of the data\.The first deviation from the ideal assumptions outlined in Section[2\.1](https://arxiv.org/html/2609.02984#S2.SS1)concerns data\-related aspects, notably the partitioning of data across agents and the resulting statistical imbalance\.

FL assumes a significant shared knowledge among the agents through the homogeneity of the local datasets\. However, in practice, local datasets\{Sk\}k∈𝒦\\\{S\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}often vary significantly across the agents, with heterogeneity arising from the partition of the data or statistical imbalance\. An example of these partitions for two agents is shown in Figure[6](https://arxiv.org/html/2609.02984#S2.F6), with a summary provided in Table[2](https://arxiv.org/html/2609.02984#S3.T2)\. In the native scenario \(Section[2\.1](https://arxiv.org/html/2609.02984#S2.SS1)\), agents collect similar types of features and labels, represented by a common feature and label spaces𝒳×𝒴\\mathcal\{X\}\\times\\mathcal\{Y\}\. However, inconsistency in the features and labels collected is frequent in data collection practices\. While these challenges have long been recognized in centralized ML\([Pan and Yang, 2009](https://arxiv.org/html/2609.02984#bib.bib359);[Zhu, 2005](https://arxiv.org/html/2609.02984#bib.bib159)\), they are amplified in collaborative settings where the datasetSS, that would ideally be centrally available, is instead partitioned across agents in local datasets\{Sk\}k∈𝒦\\\{S\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}which stay private to each agent\. To formalize, we assume the existence of a global dataset

S=\{\(xi,yi\)\}i∈ℐ\.S=\\left\\\{\(x^\{i\},y^\{i\}\)\\right\\\}\_\{i\\in\\mathcal\{I\}\}\.whereℐ\\mathcal\{I\}is the global sample index, and each agentk∈𝒦k\\in\\mathcal\{K\}holds a private partitionSkS\_\{k\}ofSS\. Different partitions exist forSS:

- •Horizontal partition\. Under horizontal partition, the indexℐ\\mathcal\{I\}ofSSis partitioned across agents into local indexesℐk⊂ℐ\\mathcal\{I\}\_\{k\}\\subset\\mathcal\{I\}such as∪k∈𝒦ℐk=ℐ\\cup\_\{k\\in\\mathcal\{K\}\}\\mathcal\{I\}\_\{k\}=\\mathcal\{I\} Sk=\{\(xi,yi\)\}i∈ℐk\.S\_\{k\}=\\left\\\{\(x^\{i\},y^\{i\}\)\\right\\\}\_\{i\\in\\mathcal\{I\}\_\{k\}\}\.Therefore, agents possess data about distinct samples indexed byℐk\\mathcal\{I\}\_\{k\}, but with the same feature𝒳\\mathcal\{X\}and label space𝒴\\mathcal\{Y\}\. This is the default setting of FL presented in Algorithm[1](https://arxiv.org/html/2609.02984#alg1), and the most studied partition\([Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18)\)in the FL literature\. This partition further distinguishescross\-devicescenarios, where millions of agents with limited data \(e\.g\., mobile devices\) collaborate, fromcross\-siloscenarios, where a few institutions holding vast amounts of data "silos" \(e\.g\., banks, hospitals, e\-commerce\) collaborate\.
- •Vertical partition\.Under vertical partitions, the domains𝒳\\mathcal\{X\}and𝒴\\mathcal\{Y\}, usually some Euclidean vectorial spaces, are partitioned across agents into local feature𝒳k\\mathcal\{X\}\_\{k\}and label𝒴k\\mathcal\{Y\}\_\{k\}domains such that𝒳=⊕k∈𝒦𝒳k\\mathcal\{X\}=\\oplus\_\{k\\in\\mathcal\{K\}\}\\mathcal\{X\}\_\{k\}and𝒴=⊕k∈𝒦𝒴k\\mathcal\{Y\}=\\oplus\_\{k\\in\\mathcal\{K\}\}\\mathcal\{Y\}\_\{k\}\. Consequently, each sample\(xi,yi\)\(x^\{i\},y^\{i\}\)is partitioned into\(xki,yki\)∈𝒳k×𝒴k\(x\_\{k\}^\{i\},y\_\{k\}^\{i\}\)\\in\\mathcal\{X\}\_\{k\}\\times\\mathcal\{Y\}\_\{k\}to form the local datasets Sk=\{\(xki,yki\)\}i∈ℐ\.S\_\{k\}=\\left\\\{\(x\_\{k\}^\{i\},y\_\{k\}^\{i\}\)\\right\\\}\_\{i\\in\\mathcal\{I\}\}\.Therefore, agents observe the same samples but with distinct views, as represented by different feature and label domains\([Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\)\. This naturally arises in multi\-modal scenarios where agents collect features and labels specific to their modality \(e\.g\., a hospital will diagnose respiratory diseases using lung radiography techniques, while another will use respiratory tests\) A special case of vertical partition is thetransfer partition\. It occurs when one or moresourceagents \(e\.g\., agents∈𝒦s\\in\\mathcal\{K\}\) hold a dataset from one feature space𝒳s\\mathcal\{X\}\_\{s\}, while anothertargetagent \(e\.g\., agentt∈𝒦t\\in\\mathcal\{K\}\) holds a dataset from a different feature space𝒳t\\mathcal\{X\}\_\{t\}\. For simplicity, we describe the case involving two agentsssandtt ∀k∈\{s,t\},Sk=\{\(xi,yi\)∈𝒳k×𝒴k\}i∈ℐk\.\\forall k\\in\\\{s,t\\\},\\quad S\_\{k\}=\\left\\\{\(x^\{i\},y^\{i\}\)\\in\\mathcal\{X\}\_\{k\}\\times\\mathcal\{Y\}\_\{k\}\\right\\\}\_\{i\\in\\mathcal\{I\}\_\{k\}\}\.Despite differences in feature domains, the agents may share a subset of common samples ∀k∈\{s,t\},\{\(xki,yki\)\}i∈ℐs∩ℐt⊂Sk\.\\forall k\\in\\\{s,t\\\},\\quad\\left\\\{\(x\_\{k\}^\{i\},y\_\{k\}^\{i\}\)\\right\\\}\_\{i\\in\\mathcal\{I\}\_\{s\}\\cap\\mathcal\{I\}\_\{t\}\}\\subset S\_\{k\}\.These act as anchor points for transferring information between agents\. Transfer partitions often involve significant data imbalance, where\|ℐs\|≫\|ℐt\|\|\\mathcal\{I\}\_\{s\}\|\\gg\|\\mathcal\{I\}\_\{t\}\|, consistent with the idea of “transferring” substantial knowledge fromsstott\. This setting frequently arises when organizations employ different data acquisition modalities but share overlapping entities\. For example, a hospital with a large archive of traditional diagnostic records may support another institution, using a small set of shared patient samples\.

#### 2\.2\.2Statistical Imbalance

In the native model, local datasetsSkS\_\{k\}are assumed to be of the same size and sampled from a shared distribution𝒟\\mathcal\{D\}on𝒳×𝒴\\mathcal\{X\}\\times\\mathcal\{Y\}\. However, in practice, the statistical properties of local datasets can vary significantly from one agent to another, and may evolve over time as agents join or leave the system, leading to different modes of imbalance\([Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18);[Paulik et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib44)\)\.

##### Quantity imbalance\.

The most obvious phenomenon arises fromquantity imbalance\(\|𝒮k\|≫\|𝒮l\|\|\\mathcal\{S\}\_\{k\}\|\\gg\|\\mathcal\{S\}\_\{l\}\|\), where the number of samples collected locally varies significantly across agents\. For example, a small clinic might manage thousands of patient records, while a multinational institution handles millions\. This imbalance represents a natural phenomenon observed across many fields\([Nisonger, 2008](https://arxiv.org/html/2609.02984#bib.bib157)\): most of the world’s data is concentrated among a minority of data holders\. The significant imbalance has given rise to different regimes known ascross\-device, where millions of agents usually corresponding to users devices \(e\.g\., a phone\) collaborate, andcross\-silo, where a few agents holding vast silos of data \(e\.g\. an hospital\) collaborate\.

##### Distributional shift\.

In addition, statistical imbalance in local datasets can arise from the distribution itself, where the datasets are drawn\. Factors like population bias, dynamic environments, multiple modalities, and varying labeling standards can influence a shift in the marginal distribution𝒟k​\(x,y\)\\mathcal\{D\}\_\{k\}\(x,y\)from one agent to another, known asdistributional shift\([Moreno\-Torres et al\., 2012](https://arxiv.org/html/2609.02984#bib.bib122);[Quiñonero\-Candela et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib123);[Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18);[Zhu et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib127)\)\. We distinguish:

- •Covariate shiftoccurs when the feature𝒟k​\(x\)\\mathcal\{D\}\_\{k\}\(x\)or label distribution𝒟k​\(y\)\\mathcal\{D\}\_\{k\}\(y\)observed locally, differ across agents but the concept𝒟⁡\(x∣y\)\\mathcal\{D\}\(x\\mid y\)or𝒟⁡\(y∣x\)\\mathcal\{D\}\(y\\mid x\)remains common\. Covariate shift is a ubiquitous phenomenon in statistics dating back to early statistical studies\([Snow, 1855](https://arxiv.org/html/2609.02984#bib.bib156)\)and is the most studied case for FL\([Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18);[Zhu et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib127);[McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335);[Hsieh et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib124);[Li et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib125)\)\. This often arises from population selection bias or non\-stationary environments\([Moreno\-Torres et al\., 2012](https://arxiv.org/html/2609.02984#bib.bib122)\)\. This distributional shift can be observed from two different perspectives\. Firstly, differences in observed label distribution𝒟k​\(y\)\\mathcal\{D\}\_\{k\}\(y\), known aslabel shift, lead to to differences in observed features distribution𝒟k​\(x\)=𝒟⁡\(x∣y\)\.𝒟k​\(y\)\\mathcal\{D\}\_\{k\}\(x\)=\\mathcal\{D\}\(x\\mid y\)\.\\mathcal\{D\}\_\{k\}\(y\)\. For example, a hospital located in a polluted area records a higher frequency of respiratory disease diagnoses compared to one in a clean environment\. Similarly, a difference in observed feature distributionsDk​\(x\)D\_\{k\}\(x\), known asfeature shift, leads to a difference in label distribution𝒟k​\(x\)=𝒟⁡\(y∣x\)\.𝒟k​\(x\)\\mathcal\{D\}\_\{k\}\(x\)=\\mathcal\{D\}\(y\\mid x\)\.\\mathcal\{D\}\_\{k\}\(x\)\. For example, one hospital focuses on diagnosing a young population while another focuses on an older population, leading to differences in features collected, yet the diagnostic knowledge remains the same\.
- •Concept shiftoccurs when the when the conditional distribution𝒟k​\(y∣x\)\\mathcal\{D\}\_\{k\}\(y\\mid x\)\(or𝒟k​\(x∣y\)\\mathcal\{D\}\_\{k\}\(x\\mid y\)\) varies across agents\. This corresponds to cases where the understanding of the concept varies from one agent to another\. For example, the standards used for diagnosing hypertension from blood pressure differ between hospitals\. This scenario is rather rare\([Moreno\-Torres et al\., 2012](https://arxiv.org/html/2609.02984#bib.bib122)\), and challenging for most applications that assume shared mappingF∈ℱF\\in\\mathcal\{F\}from features to labels\([Huang et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib126)\)\.

#### 2\.2\.3System Heterogeneity

The native forms of FL, such asFedAvgin Algorithm[1](https://arxiv.org/html/2609.02984#alg1), work best if the capabilities and tasks of the agents are similar\. In practice, real\-world systems include a variety of devices, networks, and tasks, and these system heterogeneities affect the learning and inference performance\. We briefly characterize the main types of system heterogeneity that impact collaborative training and inference\.

##### Device and network constraints\.

Collaborative learning systems must accommodate agents with vastly different hardware and network capabilities\. These constraints manifest in two primary dimensions: computational resources and communication infrastructure\.

- •Computational heterogeneity\.Agent computations are performed on diverse physical machines ranging from IoT sensors and mobile devices to telecom base stations, cloud containers, powerful servers, or even HPC clusters\. This diversity creates order\-of\-magnitude differences in the time required to calculate local model updates\([Xie et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib382);[Even et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib389)\), and in the model sizes the devices can hold\. Computing capabilities may also fluctuate over time due to parallel workloads, varying energy availability, orquantity shifts\.
- •Communication heterogeneity\.Agents employ different network technologies such as fiber, WiFi, 5G, and low\-power IoT protocols\. These technologies determine both thecommunication topology\(typically a star configuration with a central base station or a mesh with direct peer\-to\-peer links\) and theachievable bitrate, which directly affects transmission delays\([Hellström et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib143);[Wang et al\., 2020c](https://arxiv.org/html/2609.02984#bib.bib232)\)\. For wireless communication, interference and noise introduce temporal variations that degrade link reliability\. These network characteristics evolve dynamically as agents join, leave, or move through the environment\.

These computational and communication heterogeneities create challenges at multiple levels\. At the individual level, capability variations affect resource consumption such as energy cost, computation time, and communication overhead during collaborative learning\([Wang et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib266);[Meng et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib259)\)\. At the system level, slower agents, known asstragglers, can become bottlenecks that delay the entire learning process or introduce biases affecting all participants\([Li et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib383)\)\. Dynamic populations present additional complications: when agents join or leave during training, the system must initialize newcomers and address potential distributional shifts and novel label classes\.

##### Task heterogeneity\.

The native model assumes that all agents share the same task, represented as a common mapping from the feature domain𝒳\\mathcal\{X\}to the label domain𝒴\\mathcal\{Y\}\. In practice, however, agents may use their data to perform different tasks, thereby optimizing distinct mappings\([Tan et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib338)\)\. For example, different hospitals may use the same radiography images to diagnose different diseases, resulting in heterogeneous local lossesℒk\\mathcal\{L\}\_\{k\}\. Another case of task heterogeneity emerges when models are shared among applications with compound ML models, as exemplified by certain Android implementations\([Huang et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib53)\)\. Furthermore, agents may employ different model architecturesℱ\\mathcal\{F\}due to limitations in memory or computational resources\([Park and Joe\-Wong, 2024](https://arxiv.org/html/2609.02984#bib.bib47)\)\.

#### 2\.2\.4Privacy Vulnerabilities

Collaborative learning techniques were initially motivated by their potential to enable privacy\-preserving training between agents\([McMahan et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib335);[Bonawitz et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib395)\), making them well\-suited for applications where local data cannot be pooled for regulatory reasons\([Yang et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib411);[European Union, 2016](https://arxiv.org/html/2609.02984#bib.bib144)\)\. The design of the original*FedAvg*algorithm \(Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\) supports this motivation, since it ensures minimal data collection and anonymization\([Bonawitz et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib72);[Daly et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib75)\): local datasetsSkS\_\{k\}remain with the agents while only model parametersθk​\(t\)\\theta\_\{k\}\(t\)or gradients are communicated \(line[12](https://arxiv.org/html/2609.02984#alg1.l12)\), limiting what the server learns about the underlying data; and communicated parameters are immediately aggregated intoθ¯​\(t\)\\bar\{\\theta\}\(t\)\(line[14](https://arxiv.org/html/2609.02984#alg1.l14)\) before broadcast \(line[5](https://arxiv.org/html/2609.02984#alg3.l5)\), preventing other agents from directly accessing individual contributions\. Despite these design choices, collaborative learning does not provide inherent privacy guarantees\. Even in the presence ofhonest\-but\-curiousagents or server, that is, entities that follow the protocol correctly while attempting to infer sensitive information, collaborative learning in its native form suffers from information leakage at multiple levels:

- •The fully trained model parametersθ⁡\(T\)\\theta\(T\)are subject tomodel inversion attacksthat infer training labels\([Fredrikson et al\., 2015](https://arxiv.org/html/2609.02984#bib.bib54)\)or determine membership of specific samples in the training data\([Shokri et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib440)\)\.
- •Local model updates \(θk​\(t\)−θ⁡\(t−1\)\\theta\_\{k\}\(t\)\-\\theta\(t\-1\)\), or equivalently gradients, sent to the server\([Phong et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib57)\)are vulnerable togradient inversion attacksthat infer sample properties\([Melis et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib56)\)or achieve pixel\-perfect data reconstruction through gradient matching\([Zhu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib402);[Geiping et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib441);[Zhao et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib442)\)\. While other agents can also attempt gradient inversion upon receiving the aggregated modelθ¯​\(t\)\\bar\{\\theta\}\(t\), these attacks degrade significantly with the number of agents and batch size due to aggregation\.
- •Observingthe evolution of the aggregated model\{θ¯​\(t\)\}t\\\{\\bar\{\\theta\}\(t\)\\\}\_\{t\}over training enables attacks beyond single\-round inference\. Passive observation reveals temporal information, such as when samples with specific properties first appear in the training data\([Melis et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib56)\)\. More advanced threat scenarios involve agents actively manipulating their contributions to extract information about others’ training data\([Melis et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib56);[Hitaj et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib439)\)\.

### The Collaborative Learning Trilemma

Maintaining effective and efficient learning under the privacy requirements of collaborative learning \(Section[2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4)\), while simultaneously accommodating data and system heterogeneities \(Sections[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1)–[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\), poses significant challenges\. Specifically, collaborative learning must balance three competing objectives:effective learning\(accurate model predictions\),efficient learning\(constrained communication and computation\), andprivacy preservation\(anonymizing agents data\)\. These objectives create fundamental tensions: effective learning requires extensive information exchange and complex computations that challenge efficiency, while privacy preservation requires cryptographic techniques and noise injection that decrease efficiency and effectiveness, respectively\. These trade\-offs form thetrilemmaillustrated in Figure[7](https://arxiv.org/html/2609.02984#S2.F7), which serves as the foundation for analyzing the design choices in the following subsections\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/trilemna.jpg)Figure 7:The trilemma of collaborative learning:balancing effectiveness \(Section[2\.3](https://arxiv.org/html/2609.02984#S2.SS3)\), efficiency \(Section[2\.3](https://arxiv.org/html/2609.02984#S2.SS3)\) and privacy \(Section[2\.5](https://arxiv.org/html/2609.02984#S2.SS5)\)\.
### 2\.3Improving Learning Effectiveness

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/fl_effectiveness.jpg)Figure 8:Variations of federated learning that aim at improving the learning effectiveness\.In contrast tocentralized learning\(a\), where data is collected from agents and training occurs on a central server, \(b\)Federated Averagingrepresents the baseline approach for collaborative learning systems\. Methods to improve effectiveness \(c\) focus ondata\-based\(i\) andmodel\-based\(ii\) techniques, typically applied to horizontal partitions\. An alternative to the baseline isVertical Federated Learning \(VFL\) \(iii\)where the inference becomescollaborative\.Learning algorithms are effective if the trained models manage to minimize the learning loss of the agents, according to \([1](https://arxiv.org/html/2609.02984#S2.E1)\)\. The native proposition of collaborative learning \(Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\) offers strong theoretical guarantees\([Yu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib17);[Stich, 2019](https://arxiv.org/html/2609.02984#bib.bib16)\)to convergence to the solution of \([5](https://arxiv.org/html/2609.02984#S2.E5)\), particularly when the local datasets𝒟k∈𝒦\\mathcal\{D\}\_\{k\\in\\mathcal\{K\}\}are i\.i\.d\. However, real\-world deployments often diverge from this ideal case, as outlined in Section[2\.1](https://arxiv.org/html/2609.02984#S2.SS1)\.

In such circumstances, the commonly usedFedAvgalgorithm becomes suboptimal to achieve adequate minimization of local objectives\. This motivates the exploration of alternative strategies aimed atimproving the effectiveness of the models used by agents to minimize their respective local objectives, that is,data\-based,model\-based techniqueandlearning under vertical partition, as discussed in the following sections and outlined in Figure[8](https://arxiv.org/html/2609.02984#S2.F8)\.

#### 2\.3\.1Data\-based Techniques

Federated learning employs ashared global model, computed by the server as an average of the models over some participants in \(see step \([14](https://arxiv.org/html/2609.02984#S2.Ex2)\)\), to compute an optimal solutionθ∗\\theta^\{\*\}of the optimization problem \([5](https://arxiv.org/html/2609.02984#S2.E5)\)\. This approach performs effectively in its native application of horizontal cross\-device scenarios with small, balanced, and similar datasets\([Yang et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib411);[Hard et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib398)\)\. However, in horizontal settings, under statistical data heterogeneity \(seen in Section[2\.2](https://arxiv.org/html/2609.02984#S2.SS2)\), or unbalanced agent participation \(seen in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\), the returned local modelsθk​\(t\)\\theta\_\{k\}\(t\)show high variance, which famously leads the average modelθ¯​\(t\)\\bar\{\\theta\}\(t\)to drift away fromθ∗\\theta^\{\*\}, the solution of \([5](https://arxiv.org/html/2609.02984#S2.E5)\), a phenomenon well\-known in FL asclient drift\([Karimireddy et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib128);[Li et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib125)\)\. To mitigate this client drift, a variety of methods have been developed that retain a shared global model paradigm but modify theFedAvgAlgorithm[1](https://arxiv.org/html/2609.02984#alg1)to achieve tighter convergence to an optimal solutionθ∗\\theta^\{\*\}\. These methods, collectively known asdata\-based solutions, range from regularizing the local models to influencing the underlying data distribution through data augmentation and client sampling\.

##### Local regularization\.

Regularization approaches aim to minimize the discrepancy between the local models\{θk​\(t\)\}k∈𝒦t\\\{\\theta\_\{k\}\(t\)\\\}\_\{k\\in\\mathcal\{K\}\_\{t\}\}returned to the server by modifying the local updates \([9](https://arxiv.org/html/2609.02984#S2.Ex1)\)\. Bounding the variance of local gradient updates in \([9](https://arxiv.org/html/2609.02984#S2.Ex1)\) across agents has shown theoretical guarantees\([Li et al\., 2020c](https://arxiv.org/html/2609.02984#bib.bib404)\)as well as empirical evidence\([Li et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib356)\)forFedAvgto converge to the optimal solutionθ∗\\theta^\{\*\}, also in non\-i\.i\.d\. settings\. Local regularization methods can be broadly categorized into three approaches\([Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18)\):

- •Gradient control methodsreduce the gradient variance across agents and within local iterations by incorporating variance correction terms into the local updates \([9](https://arxiv.org/html/2609.02984#S2.Ex1)\)\. These terms are usually computed from the previous iterations of the localθk​\(t−1\)\\theta\_\{k\}\(t\-1\)and globalθ⁡\(t−1\)\\theta\(t\-1\)models\. The main approaches rely on adaptive algorithms\([Karimireddy et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib128);[Khanduri et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib240);[Kim et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib239)\)or dual\-based optimization\([Fan et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib146);[Gong et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib120);[Zhou and Li, 2023](https://arxiv.org/html/2609.02984#bib.bib121)\)\.
- •Selective update strategiesreduce noise by refining which model components are updated in \([9](https://arxiv.org/html/2609.02984#S2.Ex1)\), either by restricting updates to the least noisy layers\([Park and Joe\-Wong, 2025](https://arxiv.org/html/2609.02984#bib.bib151)\)or by using local optimizers that ensure smoother gradients\([Wang et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib33)\)\.
- •Objective modification approachesmodify the local empirical objective \([2](https://arxiv.org/html/2609.02984#S2.E2)\) by penalizing divergence from the global modelθ⁡\(t−1\)\\theta\(t\-1\)\([Li et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib356);[Yao and Sun, 2020](https://arxiv.org/html/2609.02984#bib.bib136);[Li et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib357)\)or, conversely, encouraging deviation from the previous local modelθk​\(t−1\)\\theta\_\{k\}\(t\-1\)\([Li et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib357)\)\. It can also penalize divergence between predictions of the global and the local model\([Lin et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib346)\), typically using contrastive losses\([Lee et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib436)\)\. Alternatively, second\-order extensions to SGD provide smoother local updates\([Fallah et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib108);[T Dinh et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib137);[Luo et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib155)\)\.
- •Prototype\-based regularizationregularize the embedding space, that is, the hidden output of the modelFFbefore its prediction head, rather than the model space\. It is achieved using "prototypes", that is, representative embeddings for each label\([Snell et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib430)\)\. Agents construct local prototypes by aggregating embeddings of the same\-label samples ofSkS\_\{k\}, which are aggregated at the server into global prototypes\. Local training then incorporates regularization terms to align embeddings with these prototypes\([Tan et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib428)\)or contrastive losses\([Tan et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib429)\)\.

##### Data augmentation\.

Data augmentation addresses distributional shifts, including quantity, feature, and label shifts, as well as missing labels, by modifying the local datasets𝒮k\\mathcal\{S\}\_\{k\}used for local training \(see \([9](https://arxiv.org/html/2609.02984#S2.Ex1)\) in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\)\. Augmentation strategies in federated settings can be categorized as follows:

- •Server\-instructed augmentation\.A central server may augment local datasets by redistributing samples between agents to balance distributional shifts\([Zhao et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib403)\), by instructing selective down\-sampling of overrepresented classes in agents local datasets\([Duan et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib130)\), or by generating synthetic samples and communicating them to the agents\([Jeong et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib129)\)\.
- •Independent local augmentation\.Agents may augment their local datasets independently\. Techniques include traditional transformations\([Zhang et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib399)\)or learned generative models\([Wu et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib131)\)\. When labels are missing, contrastive losses can be used to train on the unlabeled local dataset, augmented with negative samples\([Li et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib138);[He et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib360)\)\.
- •Collaborative local data augmentation\.Agents may augment their local datasets by collaborating with other agents\. This is particularly salient in vertical partitioning, where common samples have to be aligned between agents\([Yang et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib363);[Kang et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib362)\)\. When not all local samples of the local datasets can be aligned, the pseudo features or labels can be estimated with generative models\([Kang et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib362)\), or by matching features across agents\([Yang et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib363)\)\.

##### Client selection\.

Client selection techniques tackle the statistical imbalance seen in horizontal partition settings by modifying the server strategy for sampling agents in line 3 of Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\. There are two approaches:

- •Total availability\.Some works assume total availability of agents\. Strategies are designed to select underperforming clients\([Li et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib132);[Cho et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib35)\), group clients with similar performance in the same round\([Chai et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib133);[Yang et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib134)\), or prioritize clients that contribute to improving the global accuracy\([Wang et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib135)\)\. These strategies can be further optimized using reinforcement learning techniques on the server side, such as Q\-learning\([Wang et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib135)\)or multi\-armed bandits\([Yang et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib134)\)\.
- •Partial availability\. Other works assume that not all agents are available at the same time\. Several participation patterns are studied, ranging from cyclical patterns\([Cho et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib41)\)to temporal participation modeled by stationary Markov chains\([Wang and Ji, 2022](https://arxiv.org/html/2609.02984#bib.bib34);[Rodio et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib42)\), or a combination of both using R\-separated Markov chains\([Sun et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib32);[Xiang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib43)\)\. Some works look at spatio\-temporal correlation using Markov chains\([Rodio et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib42)\)or assuming independent agent unavailability\([Wang and Ji, 2024](https://arxiv.org/html/2609.02984#bib.bib36)\)or handle agent turnover with departing and newly arriving agents\([Ruan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib330)\)\. The techniques involve maintaining estimate of the participation statistics to modify the sampling strategy \(line[4](https://arxiv.org/html/2609.02984#alg1.l4)in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\)\([Cho et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib35);[Wang and Ji, 2022](https://arxiv.org/html/2609.02984#bib.bib34);[Ruan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib330);[Ribero et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib37)\), the aggregation \(line[14](https://arxiv.org/html/2609.02984#alg1.l14)in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\)\([Ruan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib330);[Ribero et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib37);[Wang and Ji, 2024](https://arxiv.org/html/2609.02984#bib.bib36)\), and the local update \(line[28](https://arxiv.org/html/2609.02984#alg3.l28)in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\)\([Ruan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib330)\)accordingly to guarantee a biased convergence toθ∗\\theta^\{\*\}\. Theoretical results on correlated participation\([Rodio et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib42);[Cho et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib41)\)establish the trade\-off of fast and tight convergence toθ∗\\theta^\{\*\}\.

Data\-based techniquesaddress client drift under statistical heterogeneity while maintaining the shared global model\. They follow three complementary approaches: local regularization, data augmentation, and client selection\. These techniques intervene at different stages of the learning pipeline: regularization constrains local gradient updates during training, data augmentation modifies the effective data distribution before training, and client selection filters which distributions reach aggregation\.Trilemma\.Each intervention point trades effectiveness through variance reduction against resource efficiency\. Regularization requires computing correction terms or maintaining dual variables, data augmentation demands generative models or sample redistribution, and adaptive client selection needs participation tracking and optimized aggregation schemes\. The privacy implications vary by technique: while regularization methods preserve privacy, techniques requiring data sharing \(server\-instructed augmentation, collaborative augmentation, prototype aggregation\) expose local information and necessitate additional protection mechanisms\.

#### 2\.3\.2Model\-based Techniques

In the first place, FL aims to minimize the local objectives of \([1](https://arxiv.org/html/2609.02984#S2.E1)\) among all agents\. However, the optimal solutionsF∗F^\{\*\}of the different local objectives in \([1](https://arxiv.org/html/2609.02984#S2.E1)\) might be different between agents due to factors like distributional shifts \(Section[2\.2](https://arxiv.org/html/2609.02984#S2.SS2)\) or task heterogeneity \(Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\)\. This implies that the convenient approach of looking for a model solutionFθ∗∈ℱF\_\{\\theta\}^\{\*\}\\in\\mathcal\{F\}, common to all agents, is suboptimal\. Therefore, some efforts have been devoted to allowmodel solutions that are personalized to each agent, while retaining the main steps of the collaborative learning framework depicted in Section[2\.1](https://arxiv.org/html/2609.02984#S2.SS1)\. With personalization, the optimization problem becomes

minθ1∈Θ1,…,θK∈ΘK⁡1K​∑k=1Kℒk​\(Fθk\)\.\\min\_\{\\theta\_\{1\}\\in\\Theta\_\{1\},\\ldots,\\theta\_\{K\}\\in\\Theta\_\{K\}\}\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\mathcal\{L\}\_\{k\}\(F\_\{\\theta\_\{k\}\}\)\.\(6\)Personalization is a well\-established direction in federated learning\([Tan et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib338);[Sabah et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib147)\)\. It can be achieved using two approaches: keeping a common architecture while personalizing the model parametersθk∈Θ\\theta\_\{k\}\\in\\Theta, or personalizing the model architectures\{Θk\}k∈𝒦\\\{\\Theta\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}themselves\.

##### Parameter personalization\.

∀l≠k,θk≠θl;Θk=Θl\\forall l\\neq k,\\theta\_\{k\}\\neq\\theta\_\{l\};\\Theta\_\{k\}=\\Theta\_\{l\}Parameter personalization maintains a common domainΘ\\Thetafor the personalized models but allows models to converge to distinct parameters\. These techniques apply exclusively to horizontal partitions, where all agents maintain similar model architectures\. We distinguish:

- •Local adaptation\.Local adaptation extends Algorithm[1](https://arxiv.org/html/2609.02984#alg1)by allowing each agent to modify the received global model parametersθ⁡\(t\)\\theta\(t\)into a personalized versionθk​\(t\+12\)\\theta\_\{k\}\(t\+\\frac\{1\}\{2\}\)through additional steps in the local training round\. This adaptation can occur after the federated learning process, as inmeta\-learning, whereθ⁡\(T\)\\theta\(T\)is rapidly fine\-tuned on local data\([Finn et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib106);[Jiang et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib107)\)\. Alternatively, adaptation can be performed in every roundttby modifying the local model initialization step \(line[7](https://arxiv.org/html/2609.02984#alg1.l7)in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\), e\.g\., by interpolating betweenθ⁡\(t\)\\theta\(t\)andθk​\(t−1\)\\theta\_\{k\}\(t\-1\)\([Deng et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib349)\), using learnable local masks\([Li et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib203);[Dai et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib204);[Huang et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib205)\), or applying low\-rank adaption methods toθ⁡\(t\)\\theta\(t\)\([Sun et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib434);[Guo et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib435)\)\.
- •Hypernetwork\-based\.A hypernetwork\([Klein et al\., 2015](https://arxiv.org/html/2609.02984#bib.bib309)\), that is, a network that generates the weights of a target model, can be used to produce personalized parameters for each agent\. Given an agent representation vectorckc\_\{k\}, the server\-side hypernetworkℋα\\mathcal\{H\}\_\{\\alpha\}with trainable parametersα\\alphaoutputs the personalized modelθk​\(t\)=ℋα​\(ck\)\\theta\_\{k\}\(t\)=\\mathcal\{H\}\_\{\\alpha\}\(c\_\{k\}\)\. Instead of maintaining a shared modelFθ⁡\(t\)F\_\{\\theta\(t\)\}, all agents share acommonhypernetwork with learnable weightsα\\alphajointly trained across agents by minimizing\([Shamsian et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib308)\) 1K​∑k=1Kℒ^k​\(Fℋα​\(ck\)\)\.\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\hat\{\\mathcal\{L\}\}\_\{k\}\\big\(F\_\{\\mathcal\{H\}\_\{\\alpha\}\(c\_\{k\}\)\}\\big\)\.\(7\)Training is performed on the server, where agents transmit∇αℒ^k\\nabla\_\{\\alpha\}\\hat\{\\mathcal\{L\}\}\_\{k\}and the server updatesα\\alphavia gradient steps\. The representationckc\_\{k\}can be fixed \(e\.g\., encoding an agent type\) or updated jointly withα\\alpha\.
- •Cluster\-based\.Similar agents are divided into clusters, and learn cluster\-specific models\. Assignments can be static\([Duan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib354)\), where clusters operate independently, or dynamic, allowing clustering and learning to occur simultaneously\. A straightforward approach leverages similarities for clustering\. These methods typically alternate between clustering and local updates, framing the problem as an optimization task\([Long et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib355)\)\. Clustering optimization aims to minimize inter\-cluster similarities\([Briggs et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib340);[Werner et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib153)\)or maximize intra\-cluster similarities\([Sattler et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib339);[Briggs et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib340);[Long et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib355);[Werner et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib153)\)\. Similarities are computed between clients\([Briggs et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib340)\), clients and clusters\([Ghosh et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib352);[Mansour et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib350);[Werner et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib153)\), or clients and the global model\([Sattler et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib339)\), using losses\([Ghosh et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib352);[Sattler et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib339)\)or gradients\([Werner et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib153)\)\. Techniques include greedy assignment\([Ghosh et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib352);[Mansour et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib350)\), recursive bi\-partitioning\([Sattler et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib339)\), recursive fusion\([Briggs et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib340)\), and threshold\-based clustering robust to Byzantine settings\([Werner et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib153)\)\. Assignments can be performed server\-side\([Sattler et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib339);[Long et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib355)\)or client\-side, although this requires sending all cluster models\([Ghosh et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib352)\)\. Additionally, some studies also address the assignment of new entrants to clusters\([Duan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib354)\)\.
- •Similarities between agentscan be used to favor aggregation between the most similar agents even without hard division into clusters\. A similarity matrixΩ=\[ωk,l\]∈ℝK2\\Omega=\[\\omega\_\{k,l\}\]\\in\\mathbb\{R\}^\{K^\{2\}\}, measuring the proximity between local objectives, is maintained by the server to build personalized models\. These similarities can be computed explicitly using metrics on local modelsθk​\(t\)k∈𝒦\\theta\_\{k\}\(t\)\_\{k\\in\\mathcal\{K\}\}\([Huang et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib126);[Chen and Zhang, 2022](https://arxiv.org/html/2609.02984#bib.bib154)\)or by evaluating local empirical objectives on models of other agentsfk^​\(θl\)\(k,l\)∈𝒦2\\hat\{f\_\{k\}\}\(\\theta\_\{l\}\)\_\{\(k,l\)\\in\\mathcal\{K\}^\{2\}\}\([Onoszko et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib149);[Zec et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib150)\)\. GivenΩ\\Omega, models can be personalized by averaging the most similar ones\([Onoszko et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib149)\)or using weighted averages\([Zec et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib150);[Huang et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib126)\)\. Alternatively,Ω\\Omegacan be learned as a problem parameter by adding a regularization termℛ⁡\(Ω,\[θk,l\]\(k,l\)∈𝒦2\)\\mathcal\{R\}\(\\Omega,\[\\theta\_\{k,l\}\]\_\{\(k,l\)\\in\\mathcal\{K\}^\{2\}\}\)to \([6](https://arxiv.org/html/2609.02984#S2.E6)\):\([Smith et al\., 2017b](https://arxiv.org/html/2609.02984#bib.bib348);[Huang et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib126);[Shoham et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib39);[Chen and Zhang, 2022](https://arxiv.org/html/2609.02984#bib.bib154);[Chen et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib307)\)\. This requires alternating optimization, updatingΩ\\Omegaand model parameters in separate steps\([Smith et al\., 2017b](https://arxiv.org/html/2609.02984#bib.bib348);[Huang et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib126);[Chen et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib307)\)\. While learningΩ\\Omegaintroduces significant communication overhead, decentralized protocols\([Zec et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib150)\)and smart sampling\([Zec et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib152)\)help mitigate scaling issues\.

##### Architecture personalization\.

∀l≠k,θk≠θl;Θk≠Θl\\forall l\\neq k,\\theta\_\{k\}\\neq\\theta\_\{l\};\\Theta\_\{k\}\\neq\\Theta\_\{l\}Agents can utilize different model architectures, for example, neural networks of different sizes\. The need for architecture personalization is driven both by concept statistical imbalance \(seen in Section[2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2)\) but also by system heterogeneity constraints like device heterogeneity and application heterogeneity \(seen in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\)\. We distinguish:

- •Model decoupling\.Local models,θk\\theta\_\{k\}, are divided into a shared component,θkc=θ¯\\theta\_\{k\}^\{c\}=\\bar\{\\theta\}, trained using theFedAvgalgorithm, and a private component,θkp\\theta\_\{k\}^\{p\}, which may vary in architecture across agents\. In neural networks, this split is typicallylayer\-wise\(depth\)\([Wang et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib29);[Wang et al\., 2023a](https://arxiv.org/html/2609.02984#bib.bib51);[Liang et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib416);[Arivazhagan et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib341);[Bui et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib415)\), but may also occurwidth\-wise\([Diao et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib351)\)or in a hybrid manner\([Kang et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib52)\)\. The structure of the split directly influences communication costs\. Some strategies reduce these costs by replicating shared layers to private components, especially in CNNs\([Park and Joe\-Wong, 2024](https://arxiv.org/html/2609.02984#bib.bib47)\), or by focusing on the most impactful layers\([Park and Joe\-Wong, 2025](https://arxiv.org/html/2609.02984#bib.bib151)\)\. The level of decoupling impacts model performance: greater device heterogeneity benefits from more private neurons, while noisy local updates favor a larger shared component\([Liang et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib416);[Collins et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib145)\)\. The optimal split can be determined throughadaptive\([Vahidian et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib50)\)or evenlearnable\([Deng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib49)\)strategies\. However, incomplete aggregation schemes introduce security risks\([Park and Joe\-Wong, 2024](https://arxiv.org/html/2609.02984#bib.bib47)\)\.
- •Multi\-modal personalization\.The server maintainsMMglobal models,θ=∪m=1,…,Mθcm\\theta=\\cup\_\{m=1,\\ldots,M\}\\theta^\{c\}\_\{m\}, where modelθmc\\theta^\{c\}\_\{m\}corresponds to a data domainmm\. Agents selectively combine these models based on their local data, addressing feature shifts\. Identifying the modality can rely on prior knowledge\([Chen and Zhang, 2022](https://arxiv.org/html/2609.02984#bib.bib154)\)or statistical estimation\([Huang et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib353)\)\. Model combinations vary: agents may assign distinct models per sample\([Huang et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib353)\)or aggregate outputs from multiple models when tasks require multi\-modal fusion\([Chen and Zhang, 2022](https://arxiv.org/html/2609.02984#bib.bib154)\)\. Training often leveragesparameter personalization, using clustering\([Huang et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib353)\)or agent similarity\([Smith et al\., 2017b](https://arxiv.org/html/2609.02984#bib.bib348);[Chen and Zhang, 2022](https://arxiv.org/html/2609.02984#bib.bib154)\)to train these models jointly\.
- •Knowledge distillation\.Instead of sharing full model parameters, agents can exchange sample\-like information, such as predicted labels or embeddings, which can be shared across agents with potentially different architectures\. This information may come from a globally shared unlabeled dataset\([Li and Wang, 2019](https://arxiv.org/html/2609.02984#bib.bib344);[Bistritz et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib420);[He et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib347)\)or, for more communication\-efficient approaches, from prototypes, i\.e\., a fixed set of representatives of the same label samples built collaboratively\([Tan et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib429);[Kim et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib431)\)\. The shared sample information must be aligned across agents, typically using knowledge distillation losses\([Hinton et al\., 2014](https://arxiv.org/html/2609.02984#bib.bib396)\), which can be further improved with contrastive losses\([Tan et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib429)\)\. Exchanging these compact data instead of full models can significantly reduce communication and storage costs\([Tan et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib429)\)\. Even when a global model is not strictly required, maintaining one can be useful: it can distill knowledge to agents with smaller architectures, rapidly initialize new agents’ models from a pre\-trained model\([Li and Wang, 2019](https://arxiv.org/html/2609.02984#bib.bib344);[Kim et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib431)\), or continuously provide shared knowledge through server\-side training\([He et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib347);[Lin et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib346);[Zhu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib345)\)\.

Model personalization techniquesaddress the fundamental limitation that a single shared model cannot optimally serve all agents under heterogeneous conditions\. They follow two complementary strategies: parameter personalization and architecture personalization\. Parameter personalization maintains common architectures while allowing the model parameters to diverge through local adaptation, hypernetworks, clustering, or similarity\-based weighting\. Architecture personalization accommodates structural heterogeneity through model decoupling \(shared\-private splits\), maintaining multiple global models, or the alignment of representations via distillation and prototypes\.Trilemma\.Agent\-specific adaptation improves local performance but decreases efficiency due to increased coordination complexity \(similarity computation, cluster maintenance, hypernetwork training\) and communication overhead \(multiple model transfers, prototype exchange\)\. The techniques that operate purely on model parameters \(local adaptation, layer decoupling\) limit information leakage, while those requiring explicit agent comparison or data sharing \(similarity estimation, clustering, distillation, prototype aggregation\) expose agent\-specific information\.

#### 2\.3\.3Training and Inference under Vertical Partitioning

The techniques presented in Sections[2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1)and[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2)considerhorizontal partitioningof the data \(Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1)\)\. In many domains, however, the data are insteadvertically partitioned, with agents observing complementary feature domains of the same samples \(Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1)\)\. In this case, effective learning requires combining the information from these partitions\. There are two main lines of solutions for training under vertical partitioning, training withcollaborativeinference, or forisolatedinference\.

##### Training with collaborative inference\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/VFL.jpg)Figure 9:Training under vertical partitioning with three agents\.During inference, the computations of the global modelFθ,ψF\_\{\\theta,\\psi\}are distributed across agents: to produce a predictiony^i\\hat\{y\}^\{i\}for inputxix^\{i\}, each agentkkcommunicates its partial representationzik=ϕk,θk​\(xki\)z\_\{i\}^\{k\}=\\phi\_\{k,\\theta\_\{k\}\}\(x\_\{k\}^\{i\}\)of the sample\. Only one or a subset of the agents \(here, agent 3\) possesses the true label, which can evaluate the training lossℒ^\\hat\{\\mathcal\{L\}\}\. During training, the gradients with respect to the learnable parametersθ1,θ2,θ3,ψ3\\theta\_\{1\},\\theta\_\{2\},\\theta\_\{3\},\\psi\_\{3\}must be communicated across agents in the form of∂ℒ^∂zki\\frac\{\\partial\\hat\{\\mathcal\{L\}\}\}\{\\partial z\_\{k\}^\{i\}\}\.Models can be trained under vertical data partitions by collaboratively optimizing a collaborative inference loss\. In previous Sections[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2)and[2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1), each agent trains a local modelθk\\theta\_\{k\}used to perform inferencein isolationon its local data\. Collaboration occurs solely during training when agents periodically share their local model iterateθk​\(t\)\\theta\_\{k\}\(t\)\. However, when agents know about the same sample, as seen in the example ofvertical data partition\(Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1)\), agents might benefit from collaborating during inference as well, for building a common prediction\. This collaborative inference imposes a reevaluation of the problem definition of standard FL in \([5](https://arxiv.org/html/2609.02984#S2.E5)\), and a reevaluation of the training protocol\. Under vertical partition, input features of a sample\(xi,yi\)\(x^\{i\},y^\{i\}\)are split across the agentsk∈𝒦k\\in\\mathcal\{K\}\. Each agentkkholdsxkix\_\{k\}^\{i\}, a subset of the featuresxix^\{i\}such asxi=∪k∈𝒦xkix^\{i\}=\\cup\_\{k\\in\\mathcal\{K\}\}x\_\{k\}^\{i\}, forming the local dataset𝒮k=\{xki∈𝒳k\}i∈ℐ\\mathcal\{S\}\_\{k\}=\\\{x\_\{k\}^\{i\}\\in\\mathcal\{X\}\_\{k\}\\\}\_\{i\\in\\mathcal\{I\}\}\. For simplicity, we assume that a single agent, agentk∗k^\{\*\}has access to the labels, hence𝒮k∗=\{\(xk∗i,yi\)∈𝒳×𝒴\}i∈ℐ\\mathcal\{S\}\_\{k^\{\*\}\}=\\\{\(x\_\{k^\{\*\}\}^\{i\},y^\{i\}\)\\in\\mathcal\{X\}\\times\\mathcal\{Y\}\\\}\_\{i\\in\\mathcal\{I\}\}\. Since the features of each sample are split across agents, the global modelF∈ℱF\\in\\mathcal\{F\}, used to infer on the centrally available sample\(xi,yi\)\(x^\{i\},y^\{i\}\), can be naturally decomposed to process the features locally222A related setting, known assplit learning\([Gupta and Raskar, 2018](https://arxiv.org/html/2609.02984#bib.bib342);[Vepakomma et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib400)\), partitions the inference model across agents for computational offloading, where typically one agent holds the data while others provide model layers\. While this can be viewed as an extreme case of vertical partition \(where one agent holds all features\), we focus on scenarios where multiple agents each possess complementary private data\.\. Consider that each agentkkholds a local model componentϕk,θk\\phi\_\{k,\\theta\_\{k\}\}mapping from𝒳k\\mathcal\{X\}\_\{k\}to latent space𝒵k\\mathcal\{Z\}\_\{k\}, parameterized byθk∈Θk\\theta\_\{k\}\\in\\Theta\_\{k\}while a final predictorfψk∗f\_\{\\psi\_\{k^\{\*\}\}\}, parameterized byψk∗∈Ψk∗\\psi\_\{k^\{\*\}\}\\in\\Psi\_\{k^\{\*\}\}, is held by agentk∗\{k^\{\*\}\}\. LettingFF’s model parameters beθ=\(θ1,…,θK,ψk∗\)\\theta=\(\\theta\_\{1\},\\ldots,\\theta\_\{K\},\\psi\_\{k^\{\*\}\}\)with parameter spaceΘ=Θ1×⋯×ΘK×Ψk∗\\Theta=\\Theta\_\{1\}\\times\\cdots\\times\\Theta\_\{K\}\\times\\Psi\_\{k^\{\*\}\}, the inference ofFθF\_\{\\theta\}on a sample indexed byiican be expressed as

OPENy^i=Fθ​\(xi\)=Fθ​\(\(,,,,,\)\)=fψk∗​\(ϕ1,θ1​\(x1i\),…,ϕK,θK​\(xKi\)\)\)\.\\hat\{y\}^\{i\}=F\_\{\\theta\}\(x^\{i\}\)=F\_\{\\theta\}\(\(x\_\{1\}^\{i\},\\ldots,x\_\{K\}^\{i\}\)\)=f\_\{\\psi\_\{k^\{\*\}\}\}\\left\(\\phi\_\{1,\\theta\_\{1\}\}\(x\_\{1\}^\{i\}\),\\ldots,\\phi\_\{K,\\theta\_\{K\}\}\(x\_\{K\}^\{i\}\)\)\\right\)\.\(8\)Therefore, the partition of the global modelFFresults in acollaborative inferencedescribed in Figure[9](https://arxiv.org/html/2609.02984#S2.F9): for performing a prediction onxix^\{i\}, each agentkkcommunicates its partial representationϕk,θk​\(xki\)\\phi\_\{k,\\theta\_\{k\}\}\(x\_\{k\}^\{i\}\)of the sample\. Since collaborative inference requires agents to exchange intermediate representations and gradients, which may expose sensitive information, practical deployments typically incorporate privacy protection mechanisms to mitigate such leakage\([Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\), detailed in Section[2\.5](https://arxiv.org/html/2609.02984#S2.SS5)\.

The population loss can be defined for all agents as

ℒV​F​L​\(Fθ1,…,θK,ψk∗\):=𝔼\(\(x1,…,xK\),y\)∼𝒟​ℓ​\(Fθ1,…,θK,ψk∗​\(\(,,,,,\)\),y\)\.\\mathcal\{L\}\_\{VFL\}\(F\_\{\\theta\_\{1\},\\ldots,\\theta\_\{K\},\\psi\_\{k^\{\*\}\}\}\):=\\mathbb\{E\}\_\{\(\(x\_\{1\},\\ldots,x\_\{K\}\),y\)\\sim\\mathcal\{D\}\}\\ell\\left\(F\_\{\\theta\_\{1\},\\ldots,\\theta\_\{K\},\\psi\_\{k^\{\*\}\}\}\\left\(\(x\_\{1\},\\ldots,x\_\{K\}\)\\right\),y\\right\)\.\(9\)Since the data distribution𝒟\\mathcal\{D\}is unknown, the loss \([9](https://arxiv.org/html/2609.02984#S2.E9)\) is approximated by an empirical counterpart defined overSkS\_\{k\}

ℒ^V​F​L​\(Fθ1,…,θK,ψk∗\):=1\|ℐ\|​∑i∈ℐℓ⁡\(Fθ1,…,θK,ψk∗​\(x1i,…,xKi\),yi\)\.\\hat\{\\mathcal\{L\}\}\_\{VFL\}\(F\_\{\\theta\_\{1\},\\ldots,\\theta\_\{K\},\\psi\_\{k^\{\*\}\}\}\):=\\frac\{1\}\{\|\\mathcal\{I\}\|\}\\sum\_\{i\\in\\mathcal\{I\}\}\\ell\\left\(F\_\{\\theta\_\{1\},\\ldots,\\theta\_\{K\},\\psi\_\{k^\{\*\}\}\}\\left\(x\_\{1\}^\{i\},\\ldots,x\_\{K\}^\{i\}\\right\),y^\{i\}\\right\)\.\(10\)Notably, both \([9](https://arxiv.org/html/2609.02984#S2.E9)\) and \([10](https://arxiv.org/html/2609.02984#S2.E10)\) are common to all agents, allowing the global loss \([3](https://arxiv.org/html/2609.02984#S2.E3)\) to be equivalently expressed in this collaborative setting as

1K​∑k=1KℒV​F​L​\(F\)=ℒV​F​L​\(F\)\.\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\mathcal\{L\}\_\{VFL\}\(F\)=\\mathcal\{L\}\_\{VFL\}\(F\)\.\(11\)However, unlike in \([3](https://arxiv.org/html/2609.02984#S2.E3)\), where local losses can be computed independently by each agent, the structure of \([9](https://arxiv.org/html/2609.02984#S2.E9)\) requires collaborative inference to evaluate the loss\. As a result, computing the objective requires inter\-agent communication, and we refer \([9](https://arxiv.org/html/2609.02984#S2.E9)\) and \([10](https://arxiv.org/html/2609.02984#S2.E10)\) to ascollaborative inference loss\. Despite this difference, the learning goal remains the same: to approximate a minimizer of the population loss \([9](https://arxiv.org/html/2609.02984#S2.E9)\) by minimizing its empirical surrogate \([10](https://arxiv.org/html/2609.02984#S2.E10)\) over a parametrized hypothesis class\. This leads to the following optimization problem ofVertical Federated Learning, in direct analogy with \([5](https://arxiv.org/html/2609.02984#S2.E5)\):

minθ1∈Θ1,…,θK∈ΘK,ψk∗∈Ψk∗⁡ℒ^V​F​L​\(Fθ1,…,θK,ψk∗\)\.\\min\_\{\\theta\_\{1\}\\in\\Theta\_\{1\},\\ldots,\\theta\_\{K\}\\in\\Theta\_\{K\},\\psi\_\{k^\{\*\}\}\\in\\Psi\_\{k^\{\*\}\}\}\\hat\{\\mathcal\{L\}\}\_\{VFL\}\(F\_\{\\theta\_\{1\},\\ldots,\\theta\_\{K\},\\psi\_\{k^\{\*\}\}\}\)\.\(12\)The typical training pipeline follows two steps\([Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\):

- •Privacy\-preserving sample alignment\.The formulation in \([10](https://arxiv.org/html/2609.02984#S2.E10)\) assumes a shared understanding of sample indicesℐ\\mathcal\{I\}, their features\(xi\)\(x^\{i\}\), and labels\(yi\)\(y^\{i\}\)across agents\. However, in practice, each agent only sees its local view, making the computation of \([10](https://arxiv.org/html/2609.02984#S2.E10)\) infeasible without preliminary coordination\. To address this problem, the first step is to identify the common index setℐ\\mathcal\{I\}across the agents\. It is usually achieved through thePrivate Set Intersection \(PSI\)protocols\([Liang and Chawathe, 2004](https://arxiv.org/html/2609.02984#bib.bib234);[Pinkas et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib235)\)\. PSI is extended to multiparty setups in\([Zhou et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib236);[Lu and Ding, 2020](https://arxiv.org/html/2609.02984#bib.bib237)\)\.
- •Collaborative training\.The optimization problem in \([12](https://arxiv.org/html/2609.02984#S2.E12)\) is solved collaboratively between agents𝒦=\{1,…,K\}\\mathcal\{K\}=\\\{1,\\ldots,K\\\}, as summarized in Algorithm[2](https://arxiv.org/html/2609.02984#alg2)\. It proceeds inTTrounds, where each round consists of two phases: - –Collaborative Inference \(lines[5](https://arxiv.org/html/2609.02984#alg2.l5)–[12](https://arxiv.org/html/2609.02984#alg2.l12)\):Each agentkkcomputes intermediate representationszki=ϕk,θk​\(xki\)z\_\{k\}^\{i\}=\\phi\_\{k,\\theta\_\{k\}\}\(x\_\{k\}^\{i\}\)\(line[5](https://arxiv.org/html/2609.02984#alg2.l5)\) for each sample and sends them to the active agentk∗k^\{\*\}\(line[6](https://arxiv.org/html/2609.02984#alg2.l6)\), which aggregates them and computes the empirical lossℒ^​\(Fθ\)\\hat\{\\mathcal\{L\}\}\(F\_\{\\theta\}\)on the mini\-batch \(line[10](https://arxiv.org/html/2609.02984#alg2.l10)\), updatesψk∗\\psi\_\{k^\{\*\}\}\(line[11](https://arxiv.org/html/2609.02984#alg2.l11)\), and sends partial gradients∂ℒ^∂zki\\frac\{\\partial\\hat\{\\mathcal\{L\}\}\}\{\\partial z\_\{k\}^\{i\}\}to each agent \(line[12](https://arxiv.org/html/2609.02984#alg2.l12)\)\. - –Collaborative Model Update \(lines[15](https://arxiv.org/html/2609.02984#alg2.l15)–[16](https://arxiv.org/html/2609.02984#alg2.l16)\):Each agent applies the chain rule to compute∇θkℒ^\\nabla\_\{\\theta\_\{k\}\}\\hat\{\\mathcal\{L\}\}\(line[15](https://arxiv.org/html/2609.02984#alg2.l15)\) and updates its local parameters \(line[16](https://arxiv.org/html/2609.02984#alg2.l16)\)\.

Algorithm 2Vertical Federated Learning algorithm \(VFL\) adapted from\([Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\)1:Set of agents

𝒦=\{1,…,K\}\\mathcal\{K\}=\\\{1,\\ldots,K\\\}, shared sample index set

ℐ\\mathcal\{I\}, learning rates

η1,η2\\eta\_\{1\},\\eta\_\{2\}, local parameters

θ⁡\(0\)=\{θk​\(0\)\}k=1K∪\{ψk∗​\(0\)\}\\theta\(0\)=\\\{\\theta\_\{k\}\(0\)\\\}\_\{k=1\}^\{K\}\\cup\\\{\\psi\_\{k^\{\*\}\}\(0\)\\\}, number of communication rounds

TT, mini\-batch size

BB
2:foreach round

t=1,…,Tt=1,\\ldots,Tdo

3:Collaborative Inference:

4:foreach agent

k∈𝒦k\\in\\mathcal\{K\}in paralleldo

5:Compute local representations:

zki=ϕk,θk​\(t−1\)​\(xki\)z\_\{k\}^\{i\}=\\phi\_\{k,\\theta\_\{k\}\(t\{\-\}1\)\}\(x\_\{k\}^\{i\}\)for all333Note that, in practice, this operation can be efficiently parallelized using batching over the samples, voluntarily eluded here for clarity\.

i∈ℐi\\in\\mathcal\{I\}
6:Send

\{zki\}i∈ℐ\\\{z\_\{k\}^\{i\}\\\}\_\{i\\in\\mathcal\{I\}\}to the active agent

k∗k^\{\*\}
7:endfor

8:Active agent

k∗k^\{\*\}:

9:Aggregate

\{zki\}\\\{z\_\{k\}^\{i\}\\\}and compute empirical predictions for each

i∈ℐi\\in\\mathcal\{I\}:

\{y^i\}i∈ℐ=\{fψk∗​\(z1i,…,zKi\)\}i∈ℐ\\\{\\hat\{y\}^\{i\}\\\}\_\{i\\in\\mathcal\{I\}\}=\\left\\\{f\_\{\\psi\_\{k^\{\*\}\}\}\(z\_\{1\}^\{i\},\\ldots,z\_\{K\}^\{i\}\)\\right\\\}\_\{i\\in\\mathcal\{I\}\}
10:Compute partial empirical loss on samples of index

ℐ\\mathcal\{I\},

ℒ^V​F​L​\(Fψk∗,θ1,…,θK\)=1\|ℐ\|​∑i∈ℐℓ⁡\(y^i,yi\)\\hat\{\\mathcal\{L\}\}\_\{VFL\}\\left\(F\_\{\\psi\_\{k^\{\*\}\},\\theta\_\{1\},\\ldots,\\theta\_\{K\}\}\\right\)=\\frac\{1\}\{\|\\mathcal\{I\}\|\}\\sum\_\{i\\in\\mathcal\{I\}\}\\ell\\left\(\\hat\{y\}^\{i\},y^\{i\}\\right\)
11:Update predictor model:

ψk∗​\(t\)←ψk∗​\(t−1\)−η1​∇ψk∗ℒ^V​F​L\\psi\_\{k^\{\*\}\}\(t\)\\leftarrow\\psi\_\{k^\{\*\}\}\(t\{\-\}1\)\-\\eta\_\{1\}\\nabla\_\{\\psi\_\{k^\{\*\}\}\}\\hat\{\\mathcal\{L\}\}\_\{VFL\}
12:Compute and send partial derivatives to each agent

kk:

\{∂ℒ^∂zki\}i∈ℬt\\left\\\{\\frac\{\\partial\\hat\{\\mathcal\{L\}\}\}\{\\partial z\_\{k\}^\{i\}\}\\right\\\}\_\{i\\in\\mathcal\{B\}\_\{t\}\}
13:Collaborative Model Update:

14:foreach agent

k∈𝒦k\\in\\mathcal\{K\}in paralleldo

15:Compute gradients via chain rule:

∇θkℒ^=∑i∈ℐ∂ℒ^V​F​L∂zki⋅∂zki∂θk\\nabla\_\{\\theta\_\{k\}\}\\hat\{\\mathcal\{L\}\}=\\sum\_\{i\\in\\mathcal\{I\}\}\\frac\{\\partial\\hat\{\\mathcal\{L\}\}\_\{VFL\}\}\{\\partial z\_\{k\}^\{i\}\}\\cdot\\frac\{\\partial z\_\{k\}^\{i\}\}\{\\partial\\theta\_\{k\}\}
16:Update local model:

θk​\(t\)←θk​\(t−1\)−η2​∇θkℒ^V​F​L\\theta\_\{k\}\(t\)\\leftarrow\\theta\_\{k\}\(t\{\-\}1\)\-\\eta\_\{2\}\\nabla\_\{\\theta\_\{k\}\}\\hat\{\\mathcal\{L\}\}\_\{VFL\}
17:endfor

18:endfor

19:returnFinal parameters

θ=\(θ1​\(T\),…,θK​\(T\),ψk∗​\(T\)\)\\theta=\\left\(\\theta\_\{1\}\(T\),\\ldots,\\theta\_\{K\}\(T\),\\psi\_\{k^\{\*\}\}\(T\)\\right\)and sample predictions

\{y^i\}i∈ℐ\\\{\\hat\{y\}^\{i\}\\\}\_\{i\\in\\mathcal\{I\}\}

Algorithm[2](https://arxiv.org/html/2609.02984#alg2)can be extended in several ways to adapt to more realistic settings with:

- •Pre\-training\.To decrease the level of communication required by the collaboration, agents may pre\-train their local models on private data prior to collaborative training\([Feng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib366);[Shen et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib160)\)\. This approach leads to scalable VFL, but may introduce bias from local data distributions\([Shen et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib160)\)\.
- •Model averaging\.The VFL protocol in Algorithm[2](https://arxiv.org/html/2609.02984#alg2)assumes that each agentk∈𝒦k\\in\\mathcal\{K\}maintains a private local modelϕk,θk\\phi\_\{k,\\theta\_\{k\}\}with domain parametersΘk\\Theta\_\{k\}which are individually updated based on the collaborative model update phase \(lines[12](https://arxiv.org/html/2609.02984#alg2.l12)–[16](https://arxiv.org/html/2609.02984#alg2.l16)\)\. However, in practice, certain agents might use the same model architecture, leading to the same domain spaceΘk=Θl\\Theta\_\{k\}=\\Theta\_\{l\}\. This can occur, for example, when agents \(e\.g\., hospitals\) collect the same features related to the same samples \(e\.g\., patients\)\. In such cases, it can be beneficial to introduce a periodic averaging of the models,θk​\(t\)=θl​\(t\)=θk​\(t\)\+θl​\(t\)2\\theta\_\{k\}\(t\)=\\theta\_\{l\}\(t\)=\\frac\{\\theta\_\{k\}\(t\)\+\\theta\_\{l\}\(t\)\}\{2\}, in the collaborative model update steps\([Fu et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib197)\)\.
- •Unlabeled data\.The formulation in \([12](https://arxiv.org/html/2609.02984#S2.E12)\) assumes full access to supervision through the labelsyiy^\{i\}\. However, in practical scenarios, a significant portion of the data may be unlabeled\. To handle such cases, the empirical loss in \([10](https://arxiv.org/html/2609.02984#S2.E10)\) can be extended to include self\-supervised objectives, such as contrastive losses over unlabeled samples\([Li et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib138);[He et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib360)\)\. Additionally, the local inference modulesϕk,θk\\phi\_\{k,\\theta\_\{k\}\}can be instantiated as autoencoders, enabling self\-supervised training via reconstruction losses\([Feng, 2022](https://arxiv.org/html/2609.02984#bib.bib361)\)\. These extensions apply primarily to the collaborative inference phase \(lines[5](https://arxiv.org/html/2609.02984#alg2.l5)–[6](https://arxiv.org/html/2609.02984#alg2.l6)\) of Algorithm[2](https://arxiv.org/html/2609.02984#alg2), and they allow agents to learn useful representations even in the absence of labels\.

##### Training for isolated inference\.

Transfer techniques aim to train models capable of performing isolated inference on vertically partitioned data while achieving effectiveness comparable to collaborative inference\. This addresses the primary limitation of collaborative inference, which is the communication overhead at inference time, while still exploiting the knowledge contained in vertically partitioned datasets\. Consider a transfer partition as defined in Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1), involving asourceagentssand atargetagenttt, each aware of their common sample setℐs∩ℐt\\mathcal\{I\}\_\{s\}\\cap\\mathcal\{I\}\_\{t\}, obtained, for example, viaPSIprotocols\. Two main approaches are commonly used depending on the structure of the partition\([Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\):

- •Knowledge distillation\.When the feature spaces of the agents exhibit significant overlap, knowledge distillation \(KD\) techniques, also discussed for horizontal partitioning in Section[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2), can be adapted to vertical settings\. The objective is to transfer the knowledge of the collaborative inference model\(ϕs,θs,ϕt,θt,fψs\)\(\\phi\_\{s,\\theta\_\{s\}\},\\phi\_\{t,\\theta\_\{t\}\},f\_\{\\psi\_\{s\}\}\), pre\-trained on domains𝒳s\\mathcal\{X\}\_\{s\}and𝒳t\\mathcal\{X\}\_\{t\}by agentsssandttaccording to \([12](https://arxiv.org/html/2609.02984#S2.E12)\), to a student model\(ϕθtt,fψtt\)\(\\phi\_\{\\theta\_\{t\}^\{t\}\},f\_\{\\psi\_\{t\}^\{t\}\}\)for agenttt, that performsisolated inferenceon domain𝒳t\\mathcal\{X\}\_\{t\}\. To this end, agenttttrains its model on a distillation loss that aligns the representations ofϕθtt\\phi\_\{\\theta\_\{t\}^\{t\}\},ϕs,θs\\phi\_\{s,\\theta\_\{s\}\}, andϕt,θt\\phi\_\{t,\\theta\_\{t\}\}in the shared latent space𝒵s×𝒵t\\mathcal\{Z\}\_\{s\}\\times\\mathcal\{Z\}\_\{t\}, using aligned unlabeled samples\([Huang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib364);[Ren et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib365)\)\.
- •Transfer learning\.When the number of aligned samples is limited, and only the source agentsshas sufficient data to train an independent model, transfer learning techniques\([Liu et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib358)\)can be employed\. The goal is to adapt a model pre\-trained by agentssson its datasetSsS\_\{s\}and domain𝒳s\\mathcal\{X\}\_\{s\}to the target domain𝒳t\\mathcal\{X\}\_\{t\}, using a shared embedding space𝒵\\mathcal\{Z\}\. The core assumption in this setting is that unaligned samples fromssandttstill yield semantically similar representations in𝒵\\mathcal\{Z\}\. The process involves two steps\. First, agentsstrains a local encoderϕs,θs:𝒳s→𝒵\\phi\_\{s,\\theta\_\{s\}\}:\\mathcal\{X\}\_\{s\}\\rightarrow\\mathcal\{Z\}and a predictorfψsf\_\{\\psi\_\{s\}\}onSsS\_\{s\}, yielding representationsZs=\{zsi,i∈ℐs\}=\{ϕs,θs​\(xsi\),i∈ℐs\}Z\_\{s\}=\\\{z^\{i\}\_\{s\},i\\in\\mathcal\{I\}\_\{s\}\\\}=\\\{\\phi\_\{s,\\theta\_\{s\}\}\(x\_\{s\}^\{i\}\),i\\in\\mathcal\{I\}\_\{s\}\\\}\. Then, agentttinitializes its own encoderϕt,θt:𝒳t→𝒵\\phi\_\{t,\\theta\_\{t\}\}:\\mathcal\{X\}\_\{t\}\\rightarrow\\mathcal\{Z\}and predictorfψtf\_\{\\psi\_\{t\}\}\. The training of parametersψt\\psi\_\{t\}andθt\\theta\_\{t\}proceeds by aligningZt=\{zti,i∈ℐt\}=\{ϕt,θt​\(xti\),i∈ℐt\}Z\_\{t\}=\\\{z^\{i\}\_\{t\},i\\in\\mathcal\{I\}\_\{t\}\\\}=\\\{\\phi\_\{t,\\theta\_\{t\}\}\(x\_\{t\}^\{i\}\),i\\in\\mathcal\{I\}\_\{t\}\\\}withZsZ\_\{s\}in the latent space𝒵\\mathcal\{Z\}using a transfer loss, while keepingψs,θs\\psi\_\{s\},\\theta\_\{s\}frozen\. SinceZsZ\_\{s\}andZtZ\_\{t\}may differ in size, advanced transfer losses are typically required, such as feature matching\([Yang et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib363)\), pseudo\-label matching\([Feng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib366)\), or adversarial methods\([Kang et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib301)\)\.

Learning under vertical partitioning methodsimproves learning effectiveness when agents hold complementary features of shared samples\. They follow two paradigms based on inference requirements:collaborative inferenceandisolated inference\. Learning with collaborative inference enables joint optimization over the full feature space, while isolated inference trains models for autonomous deployment through knowledge distillation \(when feature spaces overlap\) or transfer learning \(when source agents have richer data\)\.Trilemma\.Collaborative inference decreases efficiency due to persistent coordination overhead \(representation exchange, gradient communication\) but achieves accurate models\. Isolated inference avoids the communication overhead at inference, but needs complex representation alignment\. Privacy exposure is severe in both approaches compared to horizontal methods: collaborative inference continuously shares feature\-derived embeddings during training and inference, while isolated inference requires sample alignment through PSI protocols and representation sharing during knowledge transfer, necessitating explicit privacy\-preserving mechanisms throughout\.

### 2\.4Improving Learning Efficiency

While Section[2\.3](https://arxiv.org/html/2609.02984#S2.SS3)explored techniques to enhance the effectiveness of the learning process, these approaches often overlook the constraints of the communication and computing resources introduced in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\. This section examines strategies forefficientlearning, that is, ways to achieve the fast convergence of the learning models despite the constraints given by the network topology, the heterogeneity of the devices and network connections, and the often costly and unreliable communication links\. We discussmodel aggregationstrategies, and solutions forasynchronous aggregation, and for increasedcommunication efficiency\.

#### 2\.4\.1Model Aggregation

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/aggregation_pattern.jpg)Figure 10:Different aggregation patterns for collaborative learning\.Centralized aggregation\(a\) uses a central server that collects and combines model updates from all agents\.Decentralized aggregation\(b\) allows agents to exchange and aggregate updates over a network with restricted communication links\.Hierarchical aggregation\(c\) relies on a multi\-level structure where a central server coordinates with edge servers, which in turn oversee the aggregation of updates of their associated agents\.The efficiency of collaborative learning depends primarily on the model aggregation pattern, defined by the topology and the timing of the model parameter exchange and aggregation\. This section discusses aggregation schemes in three aggregation topologies: centralized, decentralized, and hierarchical, as shown in Figure[10](https://arxiv.org/html/2609.02984#S2.F10)\.

##### Centralized aggregation\.

Centralized aggregation is the standard strategy for combining local models, as employed in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\. In this setting, a central server coordinates training: at each roundt=1,…,Tt=1,\\ldots,T, it collects local model parametersθk​\(t\)\\theta\_\{k\}\(t\), each obtained after training forEElocal epochs, aggregates them, and redistributes the result to all agents\.

The motivation for centralized aggregation originates in distributed computing, where distributing stochastic gradient computations can theoretically reduce training time by a factor proportional to the number of agentsKK\([Zinkevich et al\., 2009](https://arxiv.org/html/2609.02984#bib.bib85);[McDonald et al\., 2010](https://arxiv.org/html/2609.02984#bib.bib113)\)\. However, naive parallelization with aggregation at every epoch \(E=1E=1\) incurs significant communication overhead, which can offset the computational benefits depending on system cost constraints described in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\.

When communication overhead outweighs the gains from faster convergence, a simple workaround is*one\-shot aggregation*, where models are aggregated once after local training completes \(T=1T=1\)\([Mcdonald et al\., 2009](https://arxiv.org/html/2609.02984#bib.bib24);[Zinkevich et al\., 2010](https://arxiv.org/html/2609.02984#bib.bib15)\)\. While highly communication\-efficient\([Zhang et al\., 2012](https://arxiv.org/html/2609.02984#bib.bib112)\), this strategy relies on strong assumptions, such as statistical similarity across agents datasets, that become increasingly difficult to satisfy asKKincreases\([Li et al\., 2020c](https://arxiv.org/html/2609.02984#bib.bib404);[Khaled et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib46)\)\.

A more balanced alternative is to let agents perform multiple local updates \(E\>1E\>1\) before periodically aggregating their models\. This approach, known as*local SGD*, leverages the robustness of SGD to noise and relaxed synchronization\([Dean et al\., 2012](https://arxiv.org/html/2609.02984#bib.bib298);[Bottou et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib23)\)\. It has proven successful both in practice and theory, achieving near\-linear speedup under mild i\.i\.d\. assumptions\([Stich, 2019](https://arxiv.org/html/2609.02984#bib.bib16);[Yu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib17);[Li et al\., 2020c](https://arxiv.org/html/2609.02984#bib.bib404)\)\.

Further refinements include hybrid schemes combining one\-shot and local SGD\([Hou et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib162)\), the use of server\-side optimizers\([Reddi et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib393)\), and dual\-based methods such as ADMM\([Boyd et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib118);[Zhou and Li, 2023](https://arxiv.org/html/2609.02984#bib.bib121)\)\.

##### Decentralized aggregation\.

Decentralized aggregation is the opposite of centralized aggregation: agents aggregate models directly without a central coordinator\. It is often proposed as a more robust alternative, mitigating the single point of failure inherent to centralized schemes\. In some cases, it is also necessitated by system constraints or by governance, trust, and incentive considerations that make a central coordinator undesirable or infeasible\. Unlike centralized approaches, both training and aggregation occur over a mesh network, introducing additional challenges related to network topology\.

Distributed aggregation builds on the early results ofdistributed averaging\([DeGroot, 1974](https://arxiv.org/html/2609.02984#bib.bib93);[Tsitsiklis et al\., 1986](https://arxiv.org/html/2609.02984#bib.bib31);[Bertsekas and Tsitsiklis, 2015](https://arxiv.org/html/2609.02984#bib.bib373)\)\. For simplicity, consider agents connected according to a mesh topology modeled as a graph, and holding initial value vectorsπi​\(0\)\\pi\_\{i\}\(0\), with an average ofπ¯​\(0\)\\bar\{\\pi\}\(0\)\. At each steptt, the updates follow a linear rule:∀k∈𝒦,πk​\(t\+1\)=wk​k​πk​\(t\)\+∑l∈𝒩⁡\(k\)wk​l​\(t\)​πl​\(t\)\\forall k\\in\\mathcal\{K\},\\pi\_\{k\}\(t\+1\)=w\_\{kk\}\\pi\_\{k\}\(t\)\+\\sum\_\{l\\in\\mathcal\{N\}\(k\)\}w\_\{kl\}\(t\)\\pi\_\{l\}\(t\), where𝒩⁡\(k\)\\mathcal\{N\}\(k\)denotes the neighbors of agentkk\. These updates can be expressed with a Markov chain asΠ⁡\(t\+1\)=W⁡\(t\)​Π​\(t\)\\Pi\(t\+1\)=W\(t\)\\Pi\(t\)whereW⁡\(t\)=\[wk​l​\(t\)\]\(k,l\)∈𝒦2W\(t\)=\[w\_\{kl\}\(t\)\]\_\{\(k,l\)\\in\\mathcal\{K\}^\{2\}\}andΠ⁡\(t\)=\[πk​l​\(t\)\]\\Pi\(t\)=\[\\pi\_\{kl\}\(t\)\]\. Convergence to a consensus state occurs when all nodes reach the same value∀k∈𝒦,limt→∞πk​\(t\)=π∗\\forall k\\in\\mathcal\{K\},\\lim\_\{t\\to\\infty\}\\pi\_\{k\}\(t\)=\\pi^\{\*\}\. Average consensus is achieved if, in addition,π∗=π¯​\(0\)\\pi^\{\*\}=\\bar\{\\pi\}\(0\)\. In the literature, we find two algorithmic approaches for distributed averaging\([Denantes et al\., 2008](https://arxiv.org/html/2609.02984#bib.bib109)\):

- •Average consensusalgorithms assume that all nodes communicate and update their values simultaneously\. The problem is formulated by minimizing theDirichlet energy, expressed as 12​Π​\(t\)⊤​L​Π​\(t\)=∑i∈𝒦∑j∈𝒩⁡\(i\)\(πi​\(t\)−πj​\(t\)\)2,\\frac\{1\}\{2\}\\,\\Pi\(t\)^\{\\top\}L\\Pi\(t\)\\;=\\;\\sum\_\{i\\in\\mathcal\{K\}\}\\sum\_\{j\\in\\mathcal\{N\}\(i\)\}\\big\(\\pi\_\{i\}\(t\)\-\\pi\_\{j\}\(t\)\\big\)^\{2\},\(13\)which quantifies the quadratic disagreement between neighboring nodes, whereLLis the graph Laplacian\([Mohar et al\., 1991](https://arxiv.org/html/2609.02984#bib.bib376)\)and𝒩⁡\(i\)\\mathcal\{N\}\(i\)denotes the neighbors of nodeiiin the mesh network\. Incontinuous time, updates towards a minimum of \([13](https://arxiv.org/html/2609.02984#S2.E13)\) can be expressed as a diffusion equation asπ˙​\(t\)=−L​π​\(t\)\\dot\{\\pi\}\(t\)=\-L\\pi\(t\)in\([Cortés, 2006](https://arxiv.org/html/2609.02984#bib.bib100);[Jiang and Wang, 2009](https://arxiv.org/html/2609.02984#bib.bib101);[Olfati\-Saber et al\., 2007](https://arxiv.org/html/2609.02984#bib.bib9)\)\. Indiscrete time, the update takes a Markov chain formΠ⁡\(t\+1\)=W⁡\(t\)​Π​\(t\)\\Pi\(t\+1\)=W\(t\)\\Pi\(t\)in\([Blondel et al\., 2005](https://arxiv.org/html/2609.02984#bib.bib89);[DeGroot, 1974](https://arxiv.org/html/2609.02984#bib.bib93);[Nedic et al\., 2009](https://arxiv.org/html/2609.02984#bib.bib98);[Olshevsky and Tsitsiklis, 2009](https://arxiv.org/html/2609.02984#bib.bib90)\)\. For exampleW⁡\(t\)W\(t\)is derived directly from \([13](https://arxiv.org/html/2609.02984#S2.E13)\) with the update ruleW⁡\(t\)=I−ϵ​LW\(t\)=I\-\\epsilon L, with step sizeϵ\>0\\epsilon\>0andIIthe identity matrix in\([Saber and Murray, 2003](https://arxiv.org/html/2609.02984#bib.bib374);[Olfati\-Saber and Murray, 2004](https://arxiv.org/html/2609.02984#bib.bib7);[Olfati\-Saber et al\., 2007](https://arxiv.org/html/2609.02984#bib.bib9)\)\. For static graphs, convergence is guaranteed whenWWis doubly stochastic and connected\. The convergence rate is exponential, determined by the topology of the graph\([Fiedler, 1973](https://arxiv.org/html/2609.02984#bib.bib375)\), through the second\-largest eigenvalue of the symmetric Laplacianλ2​\(Ls\)\\lambda\_\{2\}\(L\_\{s\}\)\([Fax and Murray, 2004](https://arxiv.org/html/2609.02984#bib.bib378);[Olfati\-Saber and Murray, 2004](https://arxiv.org/html/2609.02984#bib.bib7)\)\. Average consensus algorithms have been widely applied in multi\-agent systems\([Vicsek et al\., 1995](https://arxiv.org/html/2609.02984#bib.bib91);[Jadbabaie et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib8);[Blondel et al\., 2005](https://arxiv.org/html/2609.02984#bib.bib89);[Fax and Murray, 2004](https://arxiv.org/html/2609.02984#bib.bib378)\)and adapted to address issues like dynamic topologies\([Jadbabaie et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib8);[Olshevsky and Tsitsiklis, 2009](https://arxiv.org/html/2609.02984#bib.bib90);[Moreau, 2005](https://arxiv.org/html/2609.02984#bib.bib377)\)and delays\([Blondel et al\., 2005](https://arxiv.org/html/2609.02984#bib.bib89);[Tsitsiklis et al\., 1986](https://arxiv.org/html/2609.02984#bib.bib31)\)\.
- •Gossip algorithmsperform pairwise communication\([Karp et al\., 2000](https://arxiv.org/html/2609.02984#bib.bib103);[Bawa et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib102)\)\. Upon a tick of its internal clock, nodekkcommunicates with nodellwith probabilitypk​lp\_\{kl\}, and they update their values asπk​\(t\+1\)=πl​\(t\+1\)=πk​\(t\)\+πl​\(t\)2\\pi\_\{k\}\(t\+1\)=\\pi\_\{l\}\(t\+1\)=\\frac\{\\pi\_\{k\}\(t\)\+\\pi\_\{l\}\(t\)\}\{2\}\([Boyd et al\., 2005](https://arxiv.org/html/2609.02984#bib.bib94);[Boyd et al\., 2006](https://arxiv.org/html/2609.02984#bib.bib28)\)\. The probability matrixPPmirrors the role of adjacency matrices in synchronous algorithms\. Gossip protocols are communication\-efficient, and due to their inherent randomness, they are robust to asynchronous execution and partial observability\([Jelasity et al\., 2005](https://arxiv.org/html/2609.02984#bib.bib223);[Denantes et al\., 2008](https://arxiv.org/html/2609.02984#bib.bib109)\)\. Variations include geographic routing\([Dimakis et al\., 2008](https://arxiv.org/html/2609.02984#bib.bib97);[Bénézit et al\., 2010b](https://arxiv.org/html/2609.02984#bib.bib96)\), weighted updates\([Bénézit et al\., 2010a](https://arxiv.org/html/2609.02984#bib.bib95)\), and memory\-augmented protocols\([Cao et al\., 2006](https://arxiv.org/html/2609.02984#bib.bib105);[Liu et al\., 2013](https://arxiv.org/html/2609.02984#bib.bib104)\)\.

Distributed averaging combined with local learning methods leads to decentralized aggregation\. Compared to centralized aggregation, the challenge is that both the local learning and the model aggregation processes require multiple steps \(as represented by respective indíceseeandtt\)\. Early methods combine local gradient updates with a single averaging step\([Tsitsiklis et al\., 1986](https://arxiv.org/html/2609.02984#bib.bib31);[Nedic and Ozdaglar, 2009](https://arxiv.org/html/2609.02984#bib.bib30)\)\. More advanced techniques incorporate multiple propagation steps\([Johansson et al\., 2008](https://arxiv.org/html/2609.02984#bib.bib114)\), handle link failures\([Lobel and Ozdaglar, 2008](https://arxiv.org/html/2609.02984#bib.bib115)\), enforce constraints via projections\([Nedic et al\., 2010](https://arxiv.org/html/2609.02984#bib.bib116)\), or consider non\-i\.i\.d\. data\([Wang et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib199)\)\. Gossip\-based methods\([Ram et al\., 2009a](https://arxiv.org/html/2609.02984#bib.bib117);[Ormándi et al\., 2013](https://arxiv.org/html/2609.02984#bib.bib20);[Blot et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib84)\)enhance robustness to asynchronous participation and dynamically changing agent population\([Hegedűs et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib83)\)\. Dual\-based techniques, such as ADMM\([Boyd et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib118)\), also support decentralized learning, where dual variable averaging is performed synchronously\([Nesterov, 2009](https://arxiv.org/html/2609.02984#bib.bib119);[Wei and Ozdaglar, 2012](https://arxiv.org/html/2609.02984#bib.bib45)\)or via gossip\([Vanhaesebrouck et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib25)\)\.

The methods above rely ondistributed averaging, where each agent’s valueπk​\(t\)\\pi\_\{k\}\(t\)converges to a global consensusπ∗=π¯​\(0\)\\pi^\{\*\}=\\bar\{\\pi\}\(0\)\. However, when the network topology reflects data heterogeneity, aggregation can instead balance global and local consensus\. To preserve local information, the objective \([13](https://arxiv.org/html/2609.02984#S2.E13)\) can be modified to combine global agreement with local smoothness, as in12​Π​\(t\)T​L​Π​\(t\)\+μ​‖Π⁡\(t\)−Π⁡\(0\)‖22,\\frac\{1\}\{2\}\\Pi\(t\)^\{T\}L\\Pi\(t\)\+\\mu\\\|\\Pi\(t\)\-\\Pi\(0\)\\\|\_\{2\}^\{2\},as proposed in\([Zhou et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib11)\)\. If this formulation admits a closed\-form solution, it can be computed in a decentralized way using a broad class ofdiffusion\(also known aspropagationorsmoothing\) algorithms\([Zhu and Ghahramani, 2002](https://arxiv.org/html/2609.02984#bib.bib10);[Zhou et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib11)\)\. These mechanisms integrate naturally with local learning by initializingπk​\(t\)=θk​\(t\)\\pi\_\{k\}\(t\)=\\theta\_\{k\}\(t\), as in\([Vanhaesebrouck et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib25)\)\.

##### Hierarchical aggregation\.

Both centralized and distributed aggregation schemes have limitations in scalability in terms of the number of agents and in terms of the size of the geographic area the agents are spread over\. Hierarchical architectures are proposed to overcome these limitations by introducing multiple layers of aggregation\. The lower layers typically follow a star topology with local aggregators\. In wireless networks, these are often implemented at the base stations or edge servers\. Aggregation at the higher layers may use a central server\([Luo et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib163);[Gupta et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib164);[Wu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib165);[Azimi\-Abarghouyi and Fischione, 2025](https://arxiv.org/html/2609.02984#bib.bib265)\), or be performed over a mesh structure\([Castiglia et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib417);[Zhang et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib13)\)\.

Hierarchical architectures are often motivated by retransmission constraints, where the wide geographic distribution of agents requires the model parameters to be transmitted over long transmission paths\([Gupta et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib164)\)\. In wireless networks, the hierarchical structure improves the transmission quality over wireless links\([Wen et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib167)\)and increases the efficiency of the learning process by localizing certain operations, which conserves communication resources and reduces time\([Gupta et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib164)\)\. Clustering of agents helps to manage the heterogeneity of the system, supports personalization\([Castiglia et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib417);[Wu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib165)\), and increases privacy by limiting data traffic to specific administrative regions or social groups\([Zhou et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib166)\)\.

Model aggregation techniquesdetermine how and when agents exchange and combine their models, directly impacting communication efficiency and convergence behavior\.Centralizedaggregation offers simple coordination, but creates communication bottlenecks and single points of failure\.Decentralizedaggregation eliminates central coordination through distributed synchronous or asynchronous consensus protocols, naturally handling dynamic topologies\.Hierarchicalaggregation scales to larger systems by introducing multiple aggregation layers that localize communication\.Trilemma\.Centralized aggregation allows effective learning with accurate models and fast convergence\. Under decentralized aggregation, both the convergence and the model accuracy depend on the aggregation topology, and hierarchical aggregation needs complex coordination to ensure efficient learning under non\-i\.i\.d\. data\. Beyond topology, aggregation timing trades communication efficiency against convergence speed\. The privacy exposure depends both on the number of aggregating nodes and the required model updates: fewer aggregators reduce exposure but require trust, while frequent local updates increase data leakage\.

#### 2\.4\.2Asynchronous Aggregation

The model aggregation approaches discussed above generally assume that all clients are fully synchronized in the learning process, that is, they are all aware of and, if selected, participate in a given global iterationtt\. This is often an unrealistic approach\. First, due to the high number of agents, which makes it is costly to manage a shared view in the entire system\. Second, due to the presence of stragglers, that is, agents with limited computational capacity or slow network connectivity, which can result in the server receiving the updated modelθk​\(t\)\\theta\_\{k\}\(t\)from the agents \(line[14](https://arxiv.org/html/2609.02984#alg1.l14)\) at different times\. The common concept to describe stragglers is*staleness*, defined for the server as the difference between the index of the current round global model update \(line[15](https://arxiv.org/html/2609.02984#alg1.l15)of Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\), and the indextk≤tt\_\{k\}\\leq tof the local modelθk​\(tk\)\\theta\_\{k\}\(t\_\{k\}\)received by a stragglerkk\. The issue of staleness is considered both for centralized aggregation and for decentralized learning, with the usual assumption of bounded staleness\.

##### Asynchronous centralized aggregation\.

In the centralized aggregation scheme of\([Xie et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib382)\), the global model is updated \(line[14](https://arxiv.org/html/2609.02984#alg1.l14)\) at each asynchronously arriving client update, with weights that consider the staleness of the client\. Under a non\-i\.i\.d\. data distribution, the models of straggling agents may still hold important updates for the global learning process\. Therefore,\([Li et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib383)\)suggests that late model updates from stragglers should also be weighted according to the importance of their update for the learning process\. However, updating the global model after individual client updates has two disadvantages\. It increases the communication requirements and contradicts the efforts for privacy\-preserving FL\([Nguyen et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib385)\)\. Therefore, to allow efficient model aggregation despite stragglers,\([Chai et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib384)\)introduces a hierarchical scheme, where clients are dynamically clustered according to their update speed, synchronous learning is performed within the clusters, and asynchronous updates across the clusters\. The same challenge is addressed in\([Wei et al\., 2015](https://arxiv.org/html/2609.02984#bib.bib86);[Nguyen et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib385);[Huba et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib386)\), suggesting delaying the global model update until a given number of asynchronous client updates arrive at the central parameter server\.

##### Asynchronous aggregation for decentralized learning\.

Asynchronous updates with stragglers are also addressed for decentralized learning over graph topologies\. The convergence of the learning process is proved in\([Lian et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib387)\), assuming bounded staleness, and an averaging process that does not take this staleness into account\. Instead, the weight matrix for the averaging is optimized according to the expected staleness of the neighboring nodes in\([Bornstein et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib388)\)\. The comprehensive work\([Even et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib389)\)suggests update strategies and proves convergence, considering both staleness due to computation delays and the effect of unreliable communication\.

The convergence results of the asynchronous distributed learning schemes show that under bounded staleness, the convergence rate and the accuracy of synchronous systems are preserved, though the level of staleness and delayed aggregated updates may decrease the actual convergence speed\.

Asynchronous aggregation techniquesaddress the presence of stragglers, that is, agents with delayed updates due to computational or communication constraints, by relaxing synchronization requirements in both centralized and decentralized settings\. Two aggregation strategies emerge based on update timing: immediate aggregation processes each arriving update independently, maximizing responsiveness but increasing communication overhead, while batched aggregation waits for multiple updates before aggregating, reducing communication costs\. These strategies can be enhanced through staleness\-aware weighting \(accounting for update age\), hierarchical clustering \(grouping agents by speed for mixed synchronous\-asynchronous operation\), or optimized consensus matrices in decentralized settings \(adapting weights to expected neighbor delays\)\. Theoretical analysis under bounded staleness assumptions shows preserved convergence guarantees with degraded convergence speed proportional to staleness levels, establishing the fundamental tradeoff between synchronization flexibility and optimization tightness\.Trilemma\.Immediate aggregation leads to effective learning, but compromises resource efficiency and leads to privacy exposure due to the individual updates and the frequent model exchanges\. Batched aggregation at the same time increases the convergence times, and potentially discarding valuable straggler information may decrease model accuracy\.

#### 2\.4\.3Communication Efficiency

For collaborative learning, model parameters or gradients must be communicated to and from the parameter server \(line[5](https://arxiv.org/html/2609.02984#alg1.l5)and[12](https://arxiv.org/html/2609.02984#alg1.l12)of Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\) or among the agents in a peer\-to\-peer or mesh architecture\. Since the communication links between the agents and the central server or among the agents may be heterogeneous, unreliable, and costly \(as described in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\), the transmission of the information needs to be organized by taking the state of the communication links and the cost of communication into account\. The communication requirements of collaborative learning differ from traditional data transmission, as exact, error\-free information is usually not required for learning, and the model aggregation, as \([14](https://arxiv.org/html/2609.02984#S2.Ex2)\), needs the function of the inputs, rather than the individual values\. These motivate the design of novel, FL\-specific communication schemes, including the efficient transmission of information, the organization of medium access in wireless networks, and client scheduling\([Hellström et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib143)\)\.

##### Transmission efficiency\.

Transmission efficiency measures the importance of the transmitted information versus the invested resources, such as bandwidth and transmission power\. Bandwidth usage and learning efficiency in distributed learning can be balanced through quantization, that is, the number of bits representing a model parameter, and model sparsification, that is, the number of model parameters transmitted\([Oh et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib231);[Shlezinger et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib230);[Xie et al\., 2015](https://arxiv.org/html/2609.02984#bib.bib87)\)\. Similarly, in wireless networks, power control can achieve a tradeoff between energy efficiency and learning rate\([Wang et al\., 2020c](https://arxiv.org/html/2609.02984#bib.bib232)\)\. As randomization can help the learning process, noise and wireless interference are proposed to be tuned dynamically to accelerate learning in\([Zhang et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib233)\)\. To ensure that important model updates are transmitted successfully,\([Liu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib311)\)suggests importance\-aware power and retransmission control\.

##### Wireless communication for model aggregation\.

FL often assumes that agents communicate through wireless channels\. Several works propose to utilize the superposition property of the wireless multiple access channel to allow all nodes in a wireless cell upload the model parameters at the same time, performing the averaging step of \([14](https://arxiv.org/html/2609.02984#S2.Ex2)\) over the air in the analog\([Liu et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib224);[Hellström et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib143)\)or in the digital\([Razavikia et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib225)\)domain, including hierarchical federated learning\([Azimi\-Abarghouyi and Fodor, 2024](https://arxiv.org/html/2609.02984#bib.bib226)\)\. Over\-the\-air computation ensures that the communication costs are independent of the number of nodes\. In fully decentralized learning, the broadcast nature of the medium can be used for efficient model exchange among neighbors\([Pérez Herrera et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib227)\)\.

##### Client scheduling\.

To limit wireless resource usage while ensuring learning accuracy,\([Nishio and Yonetani, 2019](https://arxiv.org/html/2609.02984#bib.bib228);[Yang et al\., 2019a](https://arxiv.org/html/2609.02984#bib.bib208)\)suggests scheduling clients with good channel quality for each global update \(line 3 in Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\)\. Recognizing that some gradient or model updates have higher importance for the learning process, combined importance and channel\-aware client selection is proposed in several works, with different importance metrics\([Goetz et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib414);[Ren et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib229);[Liu et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib312);[Leng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib313)\)\. Typically, client selection methods aim to maximize the learning accuracy in one global round, under a time constraint\. Instead, client selection for minimizing the total learning time is proposed in\([Chen et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib314);[Nishio and Yonetani, 2019](https://arxiv.org/html/2609.02984#bib.bib228)\), balancing the time of a global round and the improvements achieved\.

Summary:Communication efficiency techniques exploit the unique properties of collaborative learning, the tolerance to noise, and the aggregation\-oriented communication, to reduce transmission costs\. They cover three complementary mechanisms: transmission efficiency, wireless\-specific aggregation, and client scheduling\. These mechanisms target different bottlenecks: transmission efficiency reduces information volume through quantization and sparsification, wireless aggregation exploits channel superposition to perform computation during transmission, while client scheduling selectively activates agents based on communication conditions\.Trilemma\.Improved efficiency typically trades off learning accuracy \(e\.g\., quantization error, gradient sparsity\), system complexity \(e\.g\., synchronized wireless access, channel state information\), or fairness \(e\.g\., bias toward agents with better connectivity\)\. Wireless over\-the\-air computation achieves communication costs independent of agent count but requires careful noise management and synchronization\.

### 2\.5Privacy\-preserving Collaborative Learning

As we have seen in Section[2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4), collaborative learning in its native form does not provide formal privacy guarantees and is thus subject to privacy vulnerabilities\. To address these vulnerabilities, the literature has developed both a rigorous framework for quantifying information leakage throughdifferential privacy \(DP\)and privacy\-preserving mechanisms that operate at different stages of the collaborative learning process\. Considering*honest\-but\-curious*agents and servers, the two main lines of privacy\-preserving mechanisms areblind computations, which apply cryptographic techniques that prevent access to intermediate computations, and DP for collaborative learning, which provides statistical guarantees that limit what can be inferred from the evolution of model parameters\.

##### Blind computations\.

If training data is not shared, Algorithms[1](https://arxiv.org/html/2609.02984#alg1)and[2](https://arxiv.org/html/2609.02984#alg2)still require agents to exchange intermediate computations, which may need to be protected from information leakage\. These intermediate values, such as local models, gradients, or intermediate representations, still carry information onSkS\_\{k\}and can potentially be exploited to infer private information\([Fredrikson et al\., 2015](https://arxiv.org/html/2609.02984#bib.bib54);[Phong et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib57);[Chai et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib219)\)\. VFL scenarios are more subject to this information leakage since they require the exchange of intermediate representations of samples, whereas horizontal FL scenarios only aggregate models or gradients\([Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\)\. One approach to prevent such an attack is to run the algorithm usingblind computationtechniques\. They provide cryptographic guarantees that the operations on the private variables of the agents are performed without the possibility of reading them\. Notable techniques in the literature\([Kairouz et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib18);[Liu et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib238)\)include Trusted Execution Environments \(TEEs\)\([Mo et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib201)\), Homomorphic Encryption \(HE\)\([Chai et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib219)\), and Secure Multi\-Party Computation \(SMPC\)\([Liu et al\., 2024a](https://arxiv.org/html/2609.02984#bib.bib299)\)\.

##### Differential privacy \(DP\)\.

Even without access to raw data or intermediate computations, agents can observe the evolution of the global model parametersθ⁡\(t\)\\theta\(t\), which may reveal information aboutSkS\_\{k\}\. Prior knowledge on the training algorithm can be used to extract private information onSkS\_\{k\}, for example, by querying the final modelFθ⁡\(t\)F\_\{\\theta\(t\)\}\([Fredrikson et al\., 2015](https://arxiv.org/html/2609.02984#bib.bib54)\), or querying the collaborative learning algorithm\([Melis et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib56);[Zhu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib402)\)\. To quantify and address these risks, the concept ofdifferential privacy \(DP\)\([Dwork et al\., 2006](https://arxiv.org/html/2609.02984#bib.bib60);[Dwork, 2006](https://arxiv.org/html/2609.02984#bib.bib418)\)can be applied\. Formally, a randomized mechanismℳ\\mathcal\{M\}satisfies\(ε,δ\)\(\\varepsilon,\\delta\)\-DP if for all neighboring sample setsSSandS′S^\{\\prime\}differing in one sample and all measurable outcome setsUU, we have:

ℙ\[ℳ\(S\)∈U\]≤eεℙ\[ℳ\(S′\)∈U\]\+δ\\mathbb\{P\}\[\\mathcal\{M\}\(S\)\\in U\]\\leq e^\{\\varepsilon\}\\mathbb\{P\}\[\\mathcal\{M\}\(S^\{\\prime\}\)\\in U\]\+\\delta\(14\)whereε\\varepsilonrepresents the privacy budget andδ\\deltathe failure probability\. In other words, DP bounds the ability of an adversary to distinguish whether a specific sample was included inRR, with smallerε\\varepsilonproviding stronger guarantees andδ\\deltaallowing for rare violations of this bound\. Originally proposed for centralized databases whereSSrepresents the database records andℳ\\mathcal\{M\}a query mechanism, DP offers rigorous formal guarantees against information leakage\([Dwork and Roth, 2014](https://arxiv.org/html/2609.02984#bib.bib59)\)\. Since DP is a general\-purpose privacy framework, it can be effectively adapted to any operations that involve the release of aggregate statistics\([Duchi et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib65);[Liu et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib220)\)\. In machine learning, DP is widely used to protect the training process, whereSSrepresents the training dataset andℳ\\mathcal\{M\}the training algorithm \(e\.g\., SGD\)\. Repeated queries, such as the stochastic gradient computations, are privatized usingDifferentially Private Stochastic Gradient Descent \(DP\-SGD\)and its variants\([Song et al\., 2013](https://arxiv.org/html/2609.02984#bib.bib62);[Bassily et al\., 2014](https://arxiv.org/html/2609.02984#bib.bib63);[Abadi et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib61)\)\. These methods ensure that an adversary observing the final model parametersθ⁡\(T\)\\theta\(T\)cannot reliably infer whether any specific sample\(x,y\)\(x,y\)was included in the training datasetSS, through a sequence of randomized operations:

- •subsampling and shuffling of training data\([Kasiviswanathan et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib66);[Bassily et al\., 2014](https://arxiv.org/html/2609.02984#bib.bib63);[Abadi et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib61)\)
- •clipping or quantization per\-sample of the gradients to bound sensitivity\([Abadi et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib61);[Andrew et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib71)\)
- •adding calibrated noise scaled to the privacy budget\(ε,δ\)\(\\varepsilon,\\delta\)before model update\([Dwork et al\., 2006](https://arxiv.org/html/2609.02984#bib.bib60);[Abadi et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib61)\)

##### Differential privacy for collaborative learning\.

Applying DP to collaborative learning introduces new challenges due to the distributed nature of the data and the system\. In collaborative settings, the global datasetSSis partitioned across agents asS=∪k∈𝒦SkS=\\cup\_\{k\\in\\mathcal\{K\}\}S\_\{k\}, making it natural to protect each agent’s data as a whole rather than individual samples\. Consequently,agent\-level DP\([McMahan et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib394);[Kairouz et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib40)\), that is, where neighboring datasets differ by one agent’s entire local datasetSkS\_\{k\}rather than a single sample\([Dwork et al\., 2006](https://arxiv.org/html/2609.02984#bib.bib60)\), is typically considered\. In this setting, the randomized mechanismℳ\\mathcal\{M\}corresponds to the full collaborative learning algorithm \(e\.g\., Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\)\. To achieve this protection, privacy\-preserving primitives can be integrated at different levels depending on the privacy threat model:

- •*Local DP \(LDP\)\.*A first approach involves applying DP techniques directly to the local training loop \(lines[8](https://arxiv.org/html/2609.02984#alg1.l8)–[11](https://arxiv.org/html/2609.02984#alg1.l11)of Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\) to privatize the gradient or model parametersθk​\(t\)\\theta\_\{k\}\(t\)transmitted to the aggregator, for example, using DP\-SGD and its variants\. These techniques are designed for the case when participants aim to protect their data from an “honest\-but\-curious” central aggregator\. While LDP provides strong privacy guarantees, it has high communication\([Smith et al\., 2017a](https://arxiv.org/html/2609.02984#bib.bib78);[Duchi and Rogers, 2019](https://arxiv.org/html/2609.02984#bib.bib77)\)and computation demands\([Kasiviswanathan et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib66);[Choi et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib218)\), and leads to the degradation of model estimates\([Duchi et al\., 2013](https://arxiv.org/html/2609.02984#bib.bib76);[Duchi et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib65);[Bhowmick et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib55)\)\. This limited the practical adoption of LDP techniques\([Bonawitz et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib72)\)\.
- •*Distributed DP\.*A second approach to protecting the privacy of local data from the central coordinator is to use distributed DP techniques\. These methods combine mild LDP mechanisms with server\-side primitives for aggregating local statistics, which amplify this privacy level\([Evfimievski et al\., 2003](https://arxiv.org/html/2609.02984#bib.bib79)\)\.Privacy amplificationis typically achieved through randomized shuffling and subsampling of the received LDP records\([Bittau et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib81);[Erlingsson et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib64);[Cheu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib80);[Balle et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib69)\)\. Adapting these distributed DP principles to collaborative learning presents specific challenges\. The goal is to perform distributed training while achieving agent\-level DP\. However, naively distributing DP\-SGD is not possible, as the server lacks sampling control over local training data and local model updates\. Therefore, DP variants ofFedAvg\(Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\) have been proposed that balance effective learning, privacy, and communication efficiency\([McMahan et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib394);[Agarwal et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib58)\)\. These variants modify model aggregation \(lines[14](https://arxiv.org/html/2609.02984#alg1.l14)–[15](https://arxiv.org/html/2609.02984#alg1.l15)\) by incorporating server\-side primitives: clipping\([Andrew et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib71)\), scaling\([Geyer et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib409)\), shuffling\([Geyer et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib409);[Pihur et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib92)\), and subsampling\([Geyer et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib409);[Pihur et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib92);[McMahan et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib394);[Talwar et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib73)\)of the received gradients or model parameters\. These server\-side primitives are coupled with mild local DP mechanisms, including additive noise and gradient quantization\([McMahan et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib394)\)\. The current state\-of\-the\-art technique,Differential Privacy\-Follow The Regularized Leader \(DP\-FTRL\)\([Kairouz et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib40)\), proposes a different approach\. It avoids sampling and shuffling by combining tree aggregation\([Dwork et al\., 2010](https://arxiv.org/html/2609.02984#bib.bib68);[Chan et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib67)\)with correlated noise through online estimate regularization\([Duchi et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib82)\)\. Distributed DP techniques have been demonstrated for large\-scale federated learning deployments at Google\([Ramaswamy et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib70);[Xu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib38)\), Apple\([Apple, 2023](https://arxiv.org/html/2609.02984#bib.bib74)\), and Microsoft\([Ding et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib252)\)\.

While most existing work focuses on privacy for centralized model aggregation,\([El Mrini et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib331);[Cyffers et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib333);[Biswas et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib334)\)extend privacy mechanisms to decentralized architectures with gossip learning\. To enable DP in real systems, recent research proposes DP solutions with transparent model aggregation\([Talwar et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib73);[Daly et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib75)\)and inference\([Shumailov et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib158)\), and verifiable privacy mechanisms\([Daly et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib75)\)\.

Privacy\-preserving techniquesaddress the gap between keeping data local and achieving true privacy\. They represent two complementary paradigms:blind computationsanddifferential privacy \(DP\)\. These paradigms address distinct threat models: blind computation uses cryptographic primitives \(TEE, HE, SMPC\) to prevent adversaries from accessing intermediate values during computation, while DP performs calibrated randomization and provides formal statistical bounds on what can be inferred from observing the model itself, even under full observability of the training process\. Distributed DP techniques achieve practical agent\-level protection through server\-side amplification mechanisms that strengthen mild local protections while maintaining efficiency\.Trilemma\.Privacy protection trades off learning effectiveness \(e\.g\., noise\-induced accuracy degradation, bias from gradient clipping, reduced statistical efficiency due to subsampling\)\. It also trades off system efficiency, by increasing computational overhead \(e\.g\., cryptographic operations, agent\- and server\-side processing\), communication cost \(e\.g\., verbose or repeated noisy updates under local mechanisms\), and reducing robustness to asynchronicity \(e\.g\., complex multi\-agent protocols\)\.

## 3Collaborative Learning on Graph\-structured Data

We have reviewed the main design choices of collaborative learning on Euclidean data in Section[2](https://arxiv.org/html/2609.02984#S2)\. We now extend these concepts to the case ofgraph\-structured data\. Since graph\-structured data introduces unique characteristics and challenges, we first provide an overview of state\-of\-the\-art solutions forlearning on graph\-structured datain Section[3\.1](https://arxiv.org/html/2609.02984#S3.SS1)\. Building on this foundation, we apply the same analytical structure used forEuclidean datato thegraph domain\. We begin by formulating thecollaborative learning problemfor graph\-structured data in Section[3\.2](https://arxiv.org/html/2609.02984#S3.SS2), then examine the different types ofheterogeneitythat emerge in Section[3\.3](https://arxiv.org/html/2609.02984#S3.SS3)\. Subsequently, we structure our analysis of design choices along the sametrilemmaidentified in Section[2](https://arxiv.org/html/2609.02984#S2)and formalized in Figure[7](https://arxiv.org/html/2609.02984#S2.F7):learning effectivenessin Section[3\.4](https://arxiv.org/html/2609.02984#S3.SS4),efficiencyin Section[3\.5](https://arxiv.org/html/2609.02984#S3.SS5), andprivacy preservationin Section[3\.6](https://arxiv.org/html/2609.02984#S3.SS6)\. Table[2](https://arxiv.org/html/2609.02984#S3.T2)summarizes the papers surveyed along these three lines\.

CaseSubgraphMultiple Graph InstancesGraph typeHomogeneousHeterogeneousHomogeneous orHeterogeneousheterogeneousFeature partitionHoriz\.Vert\.Horiz\.Horiz\.Horiz\.Type partitionNot applicableNot applicableNoYesPossibleYesTopology partitionHoriz\.Horiz\.Vert\.Vert\.Horiz\.Horiz\.Horiz\.Horiz\.No partitioningVert\.GranularitySubgraphEgoedge\-fulledge\-freeSubgraphEgoSubgraphEgoNot applicableNot applicableResulting graph cutEdgeEdgeVertexVertexVertexVertexVertexVertexNot applicableNot applicableRepresentative work[436](https://arxiv.org/html/2609.02984#bib.bib180)[382](https://arxiv.org/html/2609.02984#bib.bib319)[283](https://arxiv.org/html/2609.02984#bib.bib171)[67](https://arxiv.org/html/2609.02984#bib.bib170)[158](https://arxiv.org/html/2609.02984#bib.bib195)[400](https://arxiv.org/html/2609.02984#bib.bib405)[304](https://arxiv.org/html/2609.02984#bib.bib176)[355](https://arxiv.org/html/2609.02984#bib.bib371)[162](https://arxiv.org/html/2609.02984#bib.bib250)[89](https://arxiv.org/html/2609.02984#bib.bib370)Improving Learning and Inference Effectiveness \(Section[3\.4](https://arxiv.org/html/2609.02984#S3.SS4)\)Data\-based techniques \(Section[3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1)\)Data Augmentation[436](https://arxiv.org/html/2609.02984#bib.bib180);[307](https://arxiv.org/html/2609.02984#bib.bib179);[303](https://arxiv.org/html/2609.02984#bib.bib268)[235](https://arxiv.org/html/2609.02984#bib.bib251);[138](https://arxiv.org/html/2609.02984#bib.bib19)[4](https://arxiv.org/html/2609.02984#bib.bib217);[54](https://arxiv.org/html/2609.02984#bib.bib177);[63](https://arxiv.org/html/2609.02984#bib.bib173);[249](https://arxiv.org/html/2609.02984#bib.bib186)[315](https://arxiv.org/html/2609.02984#bib.bib182);[54](https://arxiv.org/html/2609.02984#bib.bib177)[258](https://arxiv.org/html/2609.02984#bib.bib221)[148](https://arxiv.org/html/2609.02984#bib.bib187);[438](https://arxiv.org/html/2609.02984#bib.bib188);[129](https://arxiv.org/html/2609.02984#bib.bib198);[364](https://arxiv.org/html/2609.02984#bib.bib423)Local Regularization[362](https://arxiv.org/html/2609.02984#bib.bib422)[126](https://arxiv.org/html/2609.02984#bib.bib197)[64](https://arxiv.org/html/2609.02984#bib.bib175)[364](https://arxiv.org/html/2609.02984#bib.bib423);[207](https://arxiv.org/html/2609.02984#bib.bib421);[11](https://arxiv.org/html/2609.02984#bib.bib202);[258](https://arxiv.org/html/2609.02984#bib.bib221)[48](https://arxiv.org/html/2609.02984#bib.bib269)Subsampling[56](https://arxiv.org/html/2609.02984#bib.bib185);[244](https://arxiv.org/html/2609.02984#bib.bib207)[295](https://arxiv.org/html/2609.02984#bib.bib213);[91](https://arxiv.org/html/2609.02984#bib.bib183)[304](https://arxiv.org/html/2609.02984#bib.bib176)[48](https://arxiv.org/html/2609.02984#bib.bib269);[363](https://arxiv.org/html/2609.02984#bib.bib282);[340](https://arxiv.org/html/2609.02984#bib.bib305)Model\-based techniques \(Section[3\.4\.2](https://arxiv.org/html/2609.02984#S3.SS4.SSS2)\)Personalization ofembeddings or parameters[232](https://arxiv.org/html/2609.02984#bib.bib306)[158](https://arxiv.org/html/2609.02984#bib.bib195)[63](https://arxiv.org/html/2609.02984#bib.bib173);[4](https://arxiv.org/html/2609.02984#bib.bib217);[391](https://arxiv.org/html/2609.02984#bib.bib191);[316](https://arxiv.org/html/2609.02984#bib.bib206)[11](https://arxiv.org/html/2609.02984#bib.bib202);[438](https://arxiv.org/html/2609.02984#bib.bib188);[376](https://arxiv.org/html/2609.02984#bib.bib242);[410](https://arxiv.org/html/2609.02984#bib.bib189)[113](https://arxiv.org/html/2609.02984#bib.bib215);[340](https://arxiv.org/html/2609.02984#bib.bib305)ArchitecturePersonalization[269](https://arxiv.org/html/2609.02984#bib.bib211);[211](https://arxiv.org/html/2609.02984#bib.bib262)[126](https://arxiv.org/html/2609.02984#bib.bib197);[146](https://arxiv.org/html/2609.02984#bib.bib222);[258](https://arxiv.org/html/2609.02984#bib.bib221)[340](https://arxiv.org/html/2609.02984#bib.bib305);[162](https://arxiv.org/html/2609.02984#bib.bib250);[438](https://arxiv.org/html/2609.02984#bib.bib188);[113](https://arxiv.org/html/2609.02984#bib.bib215)Training and Inference under aligned partitions \(Section[3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3)\)Aggregating noderepresentations[217](https://arxiv.org/html/2609.02984#bib.bib209)[445](https://arxiv.org/html/2609.02984#bib.bib261)[283](https://arxiv.org/html/2609.02984#bib.bib171);[64](https://arxiv.org/html/2609.02984#bib.bib175)[67](https://arxiv.org/html/2609.02984#bib.bib170)[158](https://arxiv.org/html/2609.02984#bib.bib195)[400](https://arxiv.org/html/2609.02984#bib.bib405);[63](https://arxiv.org/html/2609.02984#bib.bib173);[399](https://arxiv.org/html/2609.02984#bib.bib181);[435](https://arxiv.org/html/2609.02984#bib.bib174)[315](https://arxiv.org/html/2609.02984#bib.bib182);[316](https://arxiv.org/html/2609.02984#bib.bib206);[4](https://arxiv.org/html/2609.02984#bib.bib217)[391](https://arxiv.org/html/2609.02984#bib.bib191)[67](https://arxiv.org/html/2609.02984#bib.bib170)Aggregating graphrepresentations[89](https://arxiv.org/html/2609.02984#bib.bib370)Transfer techniques[258](https://arxiv.org/html/2609.02984#bib.bib221);[258](https://arxiv.org/html/2609.02984#bib.bib221)Improving Communication and Computation Efficiency \(Section[2\.4](https://arxiv.org/html/2609.02984#S2.SS4)\)Aggregation patterns \(Section[2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1)\)Centralized \(Models\)[422](https://arxiv.org/html/2609.02984#bib.bib419)[53](https://arxiv.org/html/2609.02984#bib.bib410)[400](https://arxiv.org/html/2609.02984#bib.bib405);[4](https://arxiv.org/html/2609.02984#bib.bib217);[249](https://arxiv.org/html/2609.02984#bib.bib186);[315](https://arxiv.org/html/2609.02984#bib.bib182)[54](https://arxiv.org/html/2609.02984#bib.bib177);[304](https://arxiv.org/html/2609.02984#bib.bib176);[399](https://arxiv.org/html/2609.02984#bib.bib181);[400](https://arxiv.org/html/2609.02984#bib.bib405)[244](https://arxiv.org/html/2609.02984#bib.bib207);[414](https://arxiv.org/html/2609.02984#bib.bib216);[391](https://arxiv.org/html/2609.02984#bib.bib191);[316](https://arxiv.org/html/2609.02984#bib.bib206)[158](https://arxiv.org/html/2609.02984#bib.bib195);[435](https://arxiv.org/html/2609.02984#bib.bib174);[63](https://arxiv.org/html/2609.02984#bib.bib173)[64](https://arxiv.org/html/2609.02984#bib.bib175);[258](https://arxiv.org/html/2609.02984#bib.bib221);[304](https://arxiv.org/html/2609.02984#bib.bib176)[340](https://arxiv.org/html/2609.02984#bib.bib305);[410](https://arxiv.org/html/2609.02984#bib.bib189);[11](https://arxiv.org/html/2609.02984#bib.bib202);[376](https://arxiv.org/html/2609.02984#bib.bib242)[438](https://arxiv.org/html/2609.02984#bib.bib188);[162](https://arxiv.org/html/2609.02984#bib.bib250);[113](https://arxiv.org/html/2609.02984#bib.bib215)[129](https://arxiv.org/html/2609.02984#bib.bib198);[48](https://arxiv.org/html/2609.02984#bib.bib269);[363](https://arxiv.org/html/2609.02984#bib.bib282);[148](https://arxiv.org/html/2609.02984#bib.bib187)[67](https://arxiv.org/html/2609.02984#bib.bib170)Centralized\(Embeddings/Features\)[422](https://arxiv.org/html/2609.02984#bib.bib419)[91](https://arxiv.org/html/2609.02984#bib.bib183)[53](https://arxiv.org/html/2609.02984#bib.bib410);[283](https://arxiv.org/html/2609.02984#bib.bib171);[61](https://arxiv.org/html/2609.02984#bib.bib172)[63](https://arxiv.org/html/2609.02984#bib.bib173);[304](https://arxiv.org/html/2609.02984#bib.bib176);[399](https://arxiv.org/html/2609.02984#bib.bib181);[400](https://arxiv.org/html/2609.02984#bib.bib405)[244](https://arxiv.org/html/2609.02984#bib.bib207);[54](https://arxiv.org/html/2609.02984#bib.bib177);[4](https://arxiv.org/html/2609.02984#bib.bib217);[414](https://arxiv.org/html/2609.02984#bib.bib216)[391](https://arxiv.org/html/2609.02984#bib.bib191)DecentralizedAggregation \(Models\)[335](https://arxiv.org/html/2609.02984#bib.bib336);[128](https://arxiv.org/html/2609.02984#bib.bib263);[244](https://arxiv.org/html/2609.02984#bib.bib207);[445](https://arxiv.org/html/2609.02984#bib.bib261)[393](https://arxiv.org/html/2609.02984#bib.bib316);[150](https://arxiv.org/html/2609.02984#bib.bib200)[138](https://arxiv.org/html/2609.02984#bib.bib19);[268](https://arxiv.org/html/2609.02984#bib.bib259);[382](https://arxiv.org/html/2609.02984#bib.bib319);[292](https://arxiv.org/html/2609.02984#bib.bib148)[162](https://arxiv.org/html/2609.02984#bib.bib250)Decentralized\(Embeddings\)[429](https://arxiv.org/html/2609.02984#bib.bib249);[28](https://arxiv.org/html/2609.02984#bib.bib264);[429](https://arxiv.org/html/2609.02984#bib.bib249);[217](https://arxiv.org/html/2609.02984#bib.bib209)[316](https://arxiv.org/html/2609.02984#bib.bib206);[393](https://arxiv.org/html/2609.02984#bib.bib316);[150](https://arxiv.org/html/2609.02984#bib.bib200)[295](https://arxiv.org/html/2609.02984#bib.bib213);[277](https://arxiv.org/html/2609.02984#bib.bib260);[132](https://arxiv.org/html/2609.02984#bib.bib248);[447](https://arxiv.org/html/2609.02984#bib.bib368)[296](https://arxiv.org/html/2609.02984#bib.bib212);[394](https://arxiv.org/html/2609.02984#bib.bib266);[138](https://arxiv.org/html/2609.02984#bib.bib19);[216](https://arxiv.org/html/2609.02984#bib.bib267)[349](https://arxiv.org/html/2609.02984#bib.bib380);[131](https://arxiv.org/html/2609.02984#bib.bib247);[130](https://arxiv.org/html/2609.02984#bib.bib6);[215](https://arxiv.org/html/2609.02984#bib.bib245)[211](https://arxiv.org/html/2609.02984#bib.bib262);[133](https://arxiv.org/html/2609.02984#bib.bib243);[292](https://arxiv.org/html/2609.02984#bib.bib148);[268](https://arxiv.org/html/2609.02984#bib.bib259)[382](https://arxiv.org/html/2609.02984#bib.bib319)[355](https://arxiv.org/html/2609.02984#bib.bib371)Hierarchical \(Models\)[150](https://arxiv.org/html/2609.02984#bib.bib200)Hierarchical\(Embeddings\)[277](https://arxiv.org/html/2609.02984#bib.bib260);[295](https://arxiv.org/html/2609.02984#bib.bib213);[277](https://arxiv.org/html/2609.02984#bib.bib260)Asynchronicity \(Section[3\.5\.2](https://arxiv.org/html/2609.02984#S3.SS5.SSS2)\)Stale model aggregationStale Message Passing[349](https://arxiv.org/html/2609.02984#bib.bib380);[110](https://arxiv.org/html/2609.02984#bib.bib325);[261](https://arxiv.org/html/2609.02984#bib.bib274);[128](https://arxiv.org/html/2609.02984#bib.bib263)[394](https://arxiv.org/html/2609.02984#bib.bib266)Communication \(Section[3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3)\)Transmission Efficiency[215](https://arxiv.org/html/2609.02984#bib.bib245);[132](https://arxiv.org/html/2609.02984#bib.bib248);[382](https://arxiv.org/html/2609.02984#bib.bib319)Wireless communicationfor aggregation[28](https://arxiv.org/html/2609.02984#bib.bib264)[131](https://arxiv.org/html/2609.02984#bib.bib247);[394](https://arxiv.org/html/2609.02984#bib.bib266);[130](https://arxiv.org/html/2609.02984#bib.bib6);[216](https://arxiv.org/html/2609.02984#bib.bib267)[123](https://arxiv.org/html/2609.02984#bib.bib5)Client scheduling andtopology management[244](https://arxiv.org/html/2609.02984#bib.bib207);[429](https://arxiv.org/html/2609.02984#bib.bib249);[277](https://arxiv.org/html/2609.02984#bib.bib260);[335](https://arxiv.org/html/2609.02984#bib.bib336)[382](https://arxiv.org/html/2609.02984#bib.bib319)Privacy \(Section[3\.6](https://arxiv.org/html/2609.02984#S3.SS6)\)For model parameters[422](https://arxiv.org/html/2609.02984#bib.bib419)[249](https://arxiv.org/html/2609.02984#bib.bib186);[400](https://arxiv.org/html/2609.02984#bib.bib405);[414](https://arxiv.org/html/2609.02984#bib.bib216)For exchangedembeddings[303](https://arxiv.org/html/2609.02984#bib.bib268)[235](https://arxiv.org/html/2609.02984#bib.bib251);[295](https://arxiv.org/html/2609.02984#bib.bib213);[216](https://arxiv.org/html/2609.02984#bib.bib267)[53](https://arxiv.org/html/2609.02984#bib.bib410);[283](https://arxiv.org/html/2609.02984#bib.bib171);[61](https://arxiv.org/html/2609.02984#bib.bib172);[62](https://arxiv.org/html/2609.02984#bib.bib184)[400](https://arxiv.org/html/2609.02984#bib.bib405);[399](https://arxiv.org/html/2609.02984#bib.bib181);[315](https://arxiv.org/html/2609.02984#bib.bib182);[414](https://arxiv.org/html/2609.02984#bib.bib216)[445](https://arxiv.org/html/2609.02984#bib.bib261);[249](https://arxiv.org/html/2609.02984#bib.bib186)[438](https://arxiv.org/html/2609.02984#bib.bib188)For topology[56](https://arxiv.org/html/2609.02984#bib.bib185);[150](https://arxiv.org/html/2609.02984#bib.bib200);[163](https://arxiv.org/html/2609.02984#bib.bib193)[235](https://arxiv.org/html/2609.02984#bib.bib251);[295](https://arxiv.org/html/2609.02984#bib.bib213);[303](https://arxiv.org/html/2609.02984#bib.bib268)[283](https://arxiv.org/html/2609.02984#bib.bib171)[414](https://arxiv.org/html/2609.02984#bib.bib216);[4](https://arxiv.org/html/2609.02984#bib.bib217);[315](https://arxiv.org/html/2609.02984#bib.bib182);[399](https://arxiv.org/html/2609.02984#bib.bib181)[400](https://arxiv.org/html/2609.02984#bib.bib405);[63](https://arxiv.org/html/2609.02984#bib.bib173);[158](https://arxiv.org/html/2609.02984#bib.bib195);[249](https://arxiv.org/html/2609.02984#bib.bib186)[435](https://arxiv.org/html/2609.02984#bib.bib174);[431](https://arxiv.org/html/2609.02984#bib.bib178);[64](https://arxiv.org/html/2609.02984#bib.bib175)[438](https://arxiv.org/html/2609.02984#bib.bib188)

Table 2:The landscape of collaborative learning techniques on graph data organized by graph partition\.The table presents the design choices and techniques employed by different approaches across various graph partitioning schemes\. Each row represents a specific technique or design choice, while columns correspond to different graph partitioning strategies categorized by graph type, feature partition, and granularity\. Citations are placed in columns according to the partitioning approach used by each work, and representative works are marked in bold\.### 3\.1Machine Learning on Graphs and GNNs

##### Problem definition\.

Machine Learning on graphs aims to develop models that operate on graph\-structured data to solve tasks defined on graphs and generalize to unseen graph instances\. Graphs are a fundamental data structure for modeling relationships between entities, which arise naturally in numerous domains such as social networks, molecular chemistry, and networked systems\. Unlike traditional data residing in Euclidean spaces \(e\.g\., images, text sequences\), graphs exhibit irregular, non\-Euclidean structure requiring specialized methods to process them effectively\.

Formally, a graph is defined as𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\), where𝒱\\mathcal\{V\}is the set ofvertices, also callednodes, andℰ⊆𝒱×𝒱\\mathcal\{E\}\\subseteq\\mathcal\{V\}\\times\\mathcal\{V\}the set ofedges, also calledlinks\. Given a fixed indexing of the nodes𝒱=\{v1\.…,vN\}\\mathcal\{V\}=\\\{v\_\{1\}\.\\ldots,v\_\{N\}\\\}, the graph structure can be conveniently encoded in an equivalent444A graph𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\)corresponds to an equivalence class of adjacency matricesA∈ℝN×NA\\in\\mathbb\{R\}^\{N\\times N\}, where two matrices are equivalent if they are related by a permutation of vertex indices; that is, for any permutation matrix onΠ∈ℝN\\Pi\\in\\mathbb\{R\}^\{N\}, we haveΠ​A​Π⊤∼A\\Pi A\\Pi^\{\\top\}\\sim A\.matrix form as an adjacency matrixA∈ℝN×NA\\in\\mathbb\{R\}^\{N\\times N\}, whereai​j=1⇔\(vi,vj\)∈ℰa\_\{ij\}=1\\iff\(v\_\{i\},v\_\{j\}\)\\in\\mathcal\{E\}\.

In practice, graphs often include auxiliary information\. Most commonly, nodes are associated with features\{xv∈𝒳\}v∈𝒱\\\{x\_\{v\}\\in\\mathcal\{X\}\\\}\_\{v\\in\\mathcal\{V\}\}, organized into a feature matrixX∈ℝN×dxX\\in\\mathbb\{R\}^\{N\\times d\_\{x\}\}, wheredxd\_\{x\}denotes the feature dimension\. Depending on the task, label information may also be available: node\-level labels for classification \(e\.g\., user characteristics\), edge\-level labels for link prediction \(e\.g\., friend recommendation\), and graph\-level labels for graph classification \(e\.g\., molecule property\)\.

Agraph samplecan then be expressed as\(G,y\)\(G,y\)whereG=\(A,X\)G=\(A,X\)represents thegraph datain its matrix form andyythe label associated with the task\. When learning on graph data, we consider that we observenngraph data samples drawn from an unknown distribution𝒟\\mathcal\{D\}, which together form a datasetS=\{Gi,yi\}i=1nS=\\\{G^\{i\},y^\{i\}\\\}\_\{i=1\}^\{n\}\. The goal is then to learn a modelFFamong our class of modelsℱ\\mathcal\{F\}which minimizes the expected loss

ℒ⁡\(F\):=𝔼\(G,y\)∼𝒟​ℓ​\(F⁡\(G\),y\)\.\\mathcal\{L\}\(F\):=\\mathbb\{E\}\_\{\(G,y\)\\sim\\mathcal\{D\}\}\\ell\(F\(G\),y\)\.\(15\)

##### Graph representation learning\.

Unlike traditional data residing in Euclidean spaces \(e\.g\., images, text sequences\), graphs exhibit irregular,non\-Euclideanstructure where nodes have variable numbers of neighbors and no canonical ordering, requiring permutation equivariant \(or invariant\) representations\([Bronstein et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib406)\)\.Graph representation learningaddresses this by learning mappings that embed graph data into fixed\-dimensional Euclidean vector spacesℝdz\\mathbb\{R\}^\{d\_\{z\}\}while preserving both topological structureAAand node attributesXX\. Specifically, the goal is to learn a mapping that embeds vertices, edges, or full graphs intoℝdz\\mathbb\{R\}^\{d\_\{z\}\}such that the embeddings preserve both topological structure and node attributes \(see Figure[11](https://arxiv.org/html/2609.02984#S3.F11)\)\. This mapping typically consists of a composition of two functions\. First, a learnable node encoderϕn​o​d​e\\phi\_\{node\}maps each nodevvin graphGGto an embeddingzv:=ϕn​o​d​e​\(G,v\)∈ℝdzz\_\{v\}:=\\phi\_\{node\}\(G,v\)\\in\\mathbb\{R\}^\{d\_\{z\}\}that captures both structural and feature information about nodevv\. These node embeddings\{zv;v∈𝒱\}\\\{z\_\{v\};v\\in\\mathcal\{V\}\\\}then serve as building blocks for edge and graph embeddings, depending on the task:

- •The edge embeddings can be computed by an encoderϕl​i​n​k​\(zu,zv\)\\phi\_\{link\}\(z\_\{u\},z\_\{v\}\)\(e\.g\., a simple concatenation\), which can also be learnable \(e\.g\., via an MLP\)\.
- •The graph embedding can be computed by an encoder on the multiset of all node representationsϕg​r​a​p​h​\(\{\{zv,v∈𝒢\}\}\)\\phi\_\{graph\}\(\\\{\\\{z\_\{v\},v\\in\\mathcal\{G\}\\\}\\\}\)\(e\.g\., a mean\), which can be learnable \(e\.g\., via an attention layer\)\.

The complete encoder modelϕ\\phithus depends on the task at stake\. In most models, only the node encoder is learnable, andϕn​o​d​e\\phi\_\{node\}is parametrized by these learnable weightsθ∈Θ\\theta\\in\\Theta, expressed by the notationϕθ\\phi\_\{\\theta\}\. When labels are available, a predictorfψf\_\{\\psi\}, parameterized byψ∈Ψ\\psi\\in\\Psi, is added to the output of the encoder such thatFψ,θF\_\{\\psi,\\theta\}is the complete model\. The problem is then to identifyFψ∗,θ∗F\_\{\\psi^\{\*\},\\theta^\{\*\}\}which minimizes

ℒ⁡\(Fψ,θ\):=𝔼\(G,y\)∼𝒟​ℓ​\(Fψ,θ​\(G\),y\)\.\\mathcal\{L\}\(F\_\{\\psi,\\theta\}\):=\\mathbb\{E\}\_\{\(G,y\)\\sim\\mathcal\{D\}\}\\ell\(F\_\{\\psi,\\theta\}\(G\),y\)\.\(16\)

##### Graph Neural Networks \(GNNs\)\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/ml_graph.jpg)Figure 11:Graph representation learning\.The objective is to learn a node encoderϕn​o​d​e\\phi\_\{node\}that embed the graph dataG=\(A,X\)G=\(A,X\), consisting of the adjacency matrixA∈ℝN×NA\\in\\mathbb\{R\}^\{N\\times N\}and the feature matrixX∈ℝN×dxX\\in\\mathbb\{R\}^\{N\\times d\_\{x\}\}, into a node representation matrixZ∈ℝN×dzZ\\in\\mathbb\{R\}^\{N\\times d\_\{z\}\}\. These node representations are then used for predictive tasks at the node, link, or graph level, with the use of a predictorfψf\_\{\\psi\}and, depending on the task, a linkϕl​i​n​k\\phi\_\{link\}or graphϕg​r​a​p​h\\phi\_\{graph\}encoder\.Graph Neural Networks represent the dominant node encoderϕn​o​d​e\\phi\_\{node\}architecture\. Two architectural paradigms are commonly distinguished\([Wu et al\., 2020b](https://arxiv.org/html/2609.02984#bib.bib27)\):

- •Spectral GNNsapply spectral filters to the graph\. They operate on the full spatial domain and a truncated spectral domain\([Hammond et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib424)\)\.
- •Spatial GNNsapply spatial filters to the graph\. They operate on the truncated spatial domain and the full spectral domain\([Kipf and Welling, 2017](https://arxiv.org/html/2609.02984#bib.bib390)\)\.

Modern GNN architectures predominantly adopt theMessage Passing Neural Network \(MPNN\)framework\([Gilmer et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib3)\)\(see Figure[12](https://arxiv.org/html/2609.02984#S3.F12)\), which is a spatial approach to defineϕn​o​d​e\\phi\_\{node\}\. To encode a nodev∈𝒱v\\in\\mathcal\{V\}in a representationzvz\_\{v\}, an MPNN follows an iterative process that aggregates the information ofvv’sLL\-hop neighborhood\. This neighborhood, denoted asGvG^\{v\}, represents the so\-calledLL\-hopego\-graphcentered on nodevv\. The iterative process is as follows:

- •Feature encoding: hv\(0\)←fE​n​c​\(xv\)h\_\{v\}^\{\(0\)\}\\leftarrow f\_\{Enc\}\(x\_\{v\}\)\(17\)
- •Message Passing:for each layerl=0,…,L−1l=0,\\ldots,L\-1, mv,u\(l\+1\)\\displaystyle m\_\{v,u\}^\{\(l\+1\)\}=fM​e​s\(l\+1\)​\(hv\(l\),hu\(l\)\)\\displaystyle=f\_\{Mes\}^\{\(l\+1\)\}\(h\_\{v\}^\{\(l\)\},h\_\{u\}^\{\(l\)\}\)\\quad\(message computation\)\(18\)mv\(l\+1\)\\displaystyle m\_\{v\}^\{\(l\+1\)\}=fA​g​g\(l\+1\)​\(\{\{mv,u\(l\+1\),u∈𝒩⁡\(v\)\}\}\)\\displaystyle=f\_\{Agg\}^\{\(l\+1\)\}\(\\\{\\\{m\_\{v,u\}^\{\(l\+1\)\},u\\in\\mathcal\{N\}\(v\)\\\}\\\}\)\\quad\(message aggregation\)\(19\)hv\(l\+1\)\\displaystyle h\_\{v\}^\{\(l\+1\)\}=fU​p\(l\+1\)​\(hv\(l\),mv\(l\+1\)\)\\displaystyle=f\_\{Up\}^\{\(l\+1\)\}\(h\_\{v\}^\{\(l\)\},m\_\{v\}^\{\(l\+1\)\}\)\\quad\(state update\)\(20\)where𝒩⁡\(v\)\\mathcal\{N\}\(v\)denotes the set of11\-hop neighbours ofvvin𝒢\\mathcal\{G\},fE​n​c,fM​e​s\(l\+1\)f\_\{Enc\},f\_\{Mes\}^\{\(l\+1\)\},fU​p\(l\+1\)f\_\{Up\}^\{\(l\+1\)\}are neural functions \(e\.g\., MLPs\), andfA​g​g\(l\+1\)f\_\{Agg\}^\{\(l\+1\)\}is a permutation\-invariant operator on multisets \(e\.g\., sum, mean, attention\)\.
- •Final Embedding: zv:=hv\(L\)=ϕn​o​d​e​\(Gv,v\)\.z\_\{v\}:=h\_\{v\}^\{\(L\)\}=\\phi\_\{node\}\(G^\{v\},v\)\.\(21\)

The full encoderϕθ\\phi\_\{\\theta\}can be considered to be parametrized byθ\\theta, representing all learnable weights across message, aggregation, and update functions and across all layersl=0,…,Ll=0,\\ldots,L\. The framework is modular: different architecture choices forfM​e​sf\_\{Mes\},fA​g​gf\_\{Agg\}andfU​pf\_\{Up\}define the most famous GNN architectures \(e\.g\., GCN\([Kipf and Welling, 2017](https://arxiv.org/html/2609.02984#bib.bib390)\), GraphSAGE\([Hamilton et al\., 2017a](https://arxiv.org/html/2609.02984#bib.bib372)\), GATs\([Veličković et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib391);[Brody et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib454)\)\)\. The resulting node embeddings\{zv\}v∈𝒱\\\{z\_\{v\}\\\}\_\{v\\in\\mathcal\{V\}\}can then be fed to the predictor for solving downstream tasks\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/message_passing.jpg)Figure 12:Computations in a Message Passing Neural Network \(MPNN\)ϕθ\\phi\_\{\\theta\}\.To infer the representationzvz\_\{v\}of a nodevv, the MPNN takes as input the ego\-graphGvG^\{v\}centered onvv, illustrated in \(a\) for the case of a 2\-layer MPNN\. The inference process consists of a sequence of hierarchical steps, given by \([17](https://arxiv.org/html/2609.02984#S3.E17)\),\([18](https://arxiv.org/html/2609.02984#S3.E18)\),\([19](https://arxiv.org/html/2609.02984#S3.E19)\), and \([20](https://arxiv.org/html/2609.02984#S3.E20)\), which together form the computational tree𝒯v\\mathcal\{T\}^\{v\}shown in \(b\)\. In practice, these operations can be efficiently parallelized to infer the representationszvz\_\{v\}of a batch of nodes\.While the MPNN framework described above represents the modern approach to GNNs,spectral GNNswere the historical approach, motivating the design of architectures from spectral graph theory\([Bruna et al\., 2013](https://arxiv.org/html/2609.02984#bib.bib432)\)\. Given the spectral decomposition of the symmetric normalized graph LaplacianL~=I−D−12​A​D−12\\tilde\{L\}=I\-D^\{\-\\frac\{1\}\{2\}\}AD^\{\-\\frac\{1\}\{2\}\}asL~=U​Λ​U⊤\\tilde\{L\}=U\\Lambda U^\{\\top\}, whereU∈ℝN×NU\\in\\mathbb\{R\}^\{N\\times N\}is the orthogonal matrix of eigenbasis andΛ=diag⁡\(λ1,…,λN\)\\Lambda=\\mathrm\{diag\}\(\\lambda\_\{1\},\\ldots,\\lambda\_\{N\}\)is the diagonal matrix of eigenvalues, spectral GNNs define convolution through a spectral filtering operation∗\*given by

g∗X=U​g​\(Λ\)​U⊤​X,g\*X=Ug\(\\Lambda\)U^\{\\top\}X,\(22\)whereg:\[0,2\]→ℝg:\[0,2\]\\to\\mathbb\{R\}is a spectral filter\([Hammond et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib424)\)\. However, this requires expensive eigendecomposition and filtering operations, and the decomposition must be recomputed for graph modifications\. To address these limitations, practical spectral GNNs approximate spectral filters using learnable polynomial functions of the normalized LaplacianL~\\tilde\{L\},

g⁡\(L~\)=∑l=0Lαl​L~l,Z=g⁡\(L~\)​H\(0\),g\(\\tilde\{L\}\)=\\sum\_\{l=0\}^\{L\}\\alpha\_\{l\}\\tilde\{L\}^\{l\},\\qquad Z=g\(\\tilde\{L\}\)H^\{\(0\)\},\(23\)with learnable coefficients\{αl\}l=0L\\\{\\alpha\_\{l\}\\\}\_\{l=0\}^\{L\}\. This polynomial formulation induces iterative information propagation, with each powerL~l\\tilde\{L\}^\{l\}aggregating features fromll\-hop neighborhoods, thereby reducing to spatial\-based Message Passing Neural Networks \(MPNNs\)\. Various polynomial bases have been explored \(e\.g\., ChebNets\([Defferrard et al\., 2016](https://arxiv.org/html/2609.02984#bib.bib433);[He et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib426)\), BernNet\([He et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib425)\), APPNP\([Gasteiger et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib452)\), JacobiNet\([Wang and Zhang, 2022](https://arxiv.org/html/2609.02984#bib.bib427)\), and GPRGNN\([Chien et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib453)\)\), each exploiting the recurrence relations of its polynomial basis to efficiently computeg⁡\(L~\)g\(\\tilde\{L\}\)through 1\-hop message passing operations\.

##### GNN training\.

GNNs can be trained to approximate the solution of \([16](https://arxiv.org/html/2609.02984#S3.E16)\) on empirical samples by minimizing the loss

ℒ^​\(Fψ,θ\)=1\|S\|​∑\(G,y\)∈Sℓ⁡\(Fψ,θ​\(G\),y\)\.\\hat\{\\mathcal\{L\}\}\(F\_\{\\psi,\\theta\}\)=\\frac\{1\}\{\|S\|\}\\sum\_\{\(G,y\)\\in S\}\\ell\(F\_\{\\psi,\\theta\}\(G\),y\)\.\(24\)Minimizing \([24](https://arxiv.org/html/2609.02984#S3.E24)\) is commonly done using the SGD algorithm and neural backpropagation techniques\([Rumelhart et al\., 1986](https://arxiv.org/html/2609.02984#bib.bib26);[Hamilton et al\., 2017a](https://arxiv.org/html/2609.02984#bib.bib372)\), often augmented with self\-supervised objectives such as link prediction\([Hamilton et al\., 2017a](https://arxiv.org/html/2609.02984#bib.bib372)\)or with contrastive learning\([Veličković et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib412);[You et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib281)\)\. In practice, most approaches minimize the empirical risk \([24](https://arxiv.org/html/2609.02984#S3.E24)\) as a tractable proxy for the true risk \([16](https://arxiv.org/html/2609.02984#S3.E16)\), assuming that the resulting model generalizes well, i\.e\., that the generalization gap is small\([Jegelka, 2022](https://arxiv.org/html/2609.02984#bib.bib288)\)\.

##### Interest for distributed systems

A notable characteristic of the MPNN architecture is that both the model parameters and the computed gradients need to be propagated along the edges of the graph\. MPNNs share strong similarities with distributed message passing algorithms\([Papp and Wattenhofer, 2022](https://arxiv.org/html/2609.02984#bib.bib161)\), which iteratively aggregate messages from their neighbors, update their state, and then pass on the new states to their neighbors again\. This suggests that MPNNs are a natural fit for Machine Learning tasks in collaborative systems\.

### 3\.2Collaborative Learning on Graph\-structured Data Problem Definition

As pointed out for Euclidean data in Section[2](https://arxiv.org/html/2609.02984#S2), the term distributed ML often encapsulates two techniques: parallel training on distributed machines and training on private distributed data\. Our settings focus on the latter, where the graph data is collected locally by different agents\. For the parallel training case, we refer the reader to\([Shao et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib142)\)\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/mgi_vs_sgi.jpg)Figure 13:Two main settings of Collaborative Learning on Graph Data\.Agents hold either \(a\) multiple independent graph instances or \(b\) disjoint subgraphs of a single global graph\.We now extend the collaborative learning framework introduced in Section[2\.1](https://arxiv.org/html/2609.02984#S2.SS1)to learning on graph data\. Two main problem settings are distinguished:learning from multiple graph instancesandlearning from subgraphs, as shown in Figure[13](https://arxiv.org/html/2609.02984#S3.F13)\.

##### Learning from multiple graph instances\.

We consider a set of agents𝒦=\{1,…,K\}\\mathcal\{K\}=\\\{1,\\ldots,K\\\}, where each agentkkholds a datasetSkS\_\{k\}containingnkn\_\{k\}graphs sampled from an unknown local distribution𝒟k\\mathcal\{D\}\_\{k\}\. For example, the dataset can be made of numerous molecule graphs known by different labs\. Usually, graph\-level tasks are considered for this case, for example, predicting a property of a molecule graph\([Xie et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib189)\)\. Each agent aims to minimize the local objective

ℒk​\(Fψ,θ\)=𝔼\(G,y\)∼𝒟k​ℓ​\(Fψ,θ​\(G\),y\)\.\\mathcal\{L\}\_\{k\}\(F\_\{\\psi,\\theta\}\)=\\mathbb\{E\}\_\{\(G,y\)\\sim\\mathcal\{D\}\_\{k\}\}\\ell\(F\_\{\\psi,\\theta\}\(G\),y\)\.\(25\)Agents can compute local estimates of the local objectives

ℒ^k​\(Fψ,θ\)=1nk​∑\(G,y\)∈Skℓ⁡\(Fψ,θ​\(G\),y\)\.\\hat\{\\mathcal\{L\}\}\_\{k\}\(F\_\{\\psi,\\theta\}\)=\\frac\{1\}\{n\_\{k\}\}\\sum\_\{\(G,y\)\\in S\_\{k\}\}\\ell\(F\_\{\\psi,\\theta\}\(G\),y\)\.\(26\)One can notice that the local objectives \([25](https://arxiv.org/html/2609.02984#S3.E25)\) \([26](https://arxiv.org/html/2609.02984#S3.E26)\) are formulated in the same form as for Euclidean data in \([1](https://arxiv.org/html/2609.02984#S2.E1)\) \([2](https://arxiv.org/html/2609.02984#S2.E2)\), with the difference here that the data distribution𝒟k\\mathcal\{D\}\_\{k\}operates on graph data\. As all the other aspects are the same as in Euclidean settings, if the distributions𝒟k\\mathcal\{D\}\_\{k\}are i\.i\.d\., then theFedAvgalgorithm can be used to effectively identify the optimal GNN parameters, the solution of \([3](https://arxiv.org/html/2609.02984#S2.E3)\)\.

##### Learning from subgraphs\.

We now consider the subgraph case\. We consider a global graph𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\)where each agentsk∈𝒦k\\in\\mathcal\{K\}observes a set ofNkN\_\{k\}nodes and the edges connecting these nodesℰk=\{\(u,v\)∈ℰ\|u,v∈𝒱k\}\\mathcal\{E\}\_\{k\}=\\\{\(u,v\)\\in\\mathcal\{E\}\|u,v\\in\\mathcal\{V\}\_\{k\}\\\}𝒱k⊂𝒱\\mathcal\{V\}\_\{k\}\\subset\\mathcal\{V\}, forming a subgraph topology𝒢k=\(𝒱k,ℰk\)\\mathcal\{G\}\_\{k\}=\(\\mathcal\{V\}\_\{k\},\\mathcal\{E\}\_\{k\}\)\. Each agent collects subgraph dataGkG\_\{k\}related to its observed nodes𝒱k\\mathcal\{V\}\_\{k\}, and some labelsyyassociated with the task\. Learning for recommender systems, large knowledge graphs, social networks, or large\-scale infrastructures, but also learning in groups of interactive objects, falls into this category\. The learning tasks are typically on the node, link, or subgraph level\.

Recall that the GNN requires, for each nodevv, the data from theLL\-hop neighborhood ofvvas formulated in \([21](https://arxiv.org/html/2609.02984#S3.E21)\)\. The local empirical objective for agentkkis then expressed not on the graph dataGkG\_\{k\}, but on the graph data supported on𝒢k\\mathcal\{G\}\_\{k\}expanded by anLL\-hop neighborhood within the global graph\. We denote this expanded topology by𝒢~k\\tilde\{\\mathcal\{G\}\}\_\{k\}, and the corresponding data supported on it byG~k\\tilde\{G\}\_\{k\}

ℒ^k​\(Fψ,θ\):=ℓ⁡\(Fψ,θ​\(G~k\),y\)\.\\hat\{\\mathcal\{L\}\}\_\{k\}\(F\_\{\\psi,\\theta\}\):=\\ell\(F\_\{\\psi,\\theta\}\(\\tilde\{G\}\_\{k\}\),y\)\.\(27\)If agentkkdoes not observeG~k\\tilde\{G\}\_\{k\}, then it does not have the necessary data to minimize its local loss \([27](https://arxiv.org/html/2609.02984#S3.E27)\)\. This makes the collaboration of the agents that hold parts ofG~k\\tilde\{G\}\_\{k\}necessary, a phenomenon illustrated in Figure[14](https://arxiv.org/html/2609.02984#S3.F14)\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/collaborative_inference.jpg)Figure 14:Collaborative inference in a subgraph\.In the subgraph setting, inference becomes collaborative as described in Algorithm[3](https://arxiv.org/html/2609.02984#alg3), illustrated here for the inference of the red nodevv’s representation based on itsLL\-hop ego\-graphGvG^\{v\}\. The computational tree𝒯v\\mathcal\{T\}^\{v\}shows how computations can be distributed across multiple agents, requiring the exchange of intermediate results between them, hence the termcollaborative inference\.This need for collaborative inference resembles the setting of vertical partitions described in Section[2\.3\.3](https://arxiv.org/html/2609.02984#S2.SS3.SSS3.Px1)and leads to the following observation:collaborative learning on subgraph data borrows from the principles of Vertical Federated Learning\.If we assume that\(G~k,y\)\(\\tilde\{G\}\_\{k\},y\)are sampled from an unknown distribution𝒟k\\mathcal\{D\}\_\{k\}, the true local objective can be formulated in a collaborative form

ℒk​\(Fψ,θ\):=𝔼\(G~k​y\)∼𝒟k​ℓ​\(Fψ,θ​\(G~k\),y\)\.\\mathcal\{L\}\_\{k\}\(F\_\{\\psi,\\theta\}\):=\\mathbb\{E\}\_\{\(\\tilde\{G\}\_\{k\}y\)\\sim\\mathcal\{D\}\_\{k\}\}\\ell\\left\(\{F\}\_\{\\psi,\\theta\}\(\\tilde\{G\}\_\{k\}\),y\\right\)\.\(28\)
An effective solution to minimize the local objectives \([9](https://arxiv.org/html/2609.02984#S2.E9)\) is to collaboratively train a common GNNFψ,θF\_\{\\psi,\\theta\}to minimize the average of local objectives across agents

1K​∑k∈𝒦ℒk​\(Fψ,θ\)\.\\frac\{1\}\{K\}\\sum\_\{k\\in\\mathcal\{K\}\}\\mathcal\{L\}\_\{k\}\(F\_\{\\psi,\\theta\}\)\.\(29\)This formulation is analogous to the Horizontal FL problem formulation \([2](https://arxiv.org/html/2609.02984#S2)\) with the difference that the local lossesℒk\\mathcal\{L\}\_\{k\}are collaborative, and thuscollaborative learning on subgraph data borrows from the principles of Horizontal Federated Learning\.

A GNN can be trained to minimize \([29](https://arxiv.org/html/2609.02984#S3.E29)\) in collaborative settings by following two algorithmic directions:

- •Withisolated inference, inference is performed on the local subgraph only\. This case borrows directly from the Horizontal FL Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\.
- •Withcollaborative inference, inference requires the information exchange between the agents\. This case borrows directly from the Vertical FL Algorithm[2](https://arxiv.org/html/2609.02984#alg2)\.

A possible unified implementation of federated training from subgraph data is described in Algorithm[3](https://arxiv.org/html/2609.02984#alg3), following the concepts from\([Chen et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib185);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Du and Wu, 2022](https://arxiv.org/html/2609.02984#bib.bib183);[Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175);[Yao et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib419);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195)\)\. It is a round\-based algorithm which outputs both the model parametersψ⁡\(T\),θ⁡\(T\)\\psi\(T\),\\theta\(T\), and the node representations\{zv​\(T\)\}v∈𝒱\\\{z\_\{v\}\(T\)\\\}\_\{v\\in\\mathcal\{V\}\}\(line[37](https://arxiv.org/html/2609.02984#alg3.l37)\) at the end of the training round\. The algorithm builds on the centralized aggregation of Algorithm[1](https://arxiv.org/html/2609.02984#alg1), where model parameters iterateψ⁡\(t\)\\psi\(t\)andθk​\(t\)\\theta\_\{k\}\(t\)are aggregated by a central server \(line[33](https://arxiv.org/html/2609.02984#alg3.l33)\)\. In the subgraph setting, however, the server also assumes additional responsibilities: it stores the node representation iterateszv​\(t\)v∈𝒱\{z\_\{v\}\(t\)\}\_\{v\\in\\mathcal\{V\}\}\(line[34](https://arxiv.org/html/2609.02984#alg3.l34)\) and allows the exchange of node embeddings between agents during collaborative inference \(lines[18](https://arxiv.org/html/2609.02984#alg3.l18)and[19](https://arxiv.org/html/2609.02984#alg3.l19)\)\.

In the algorithm, for agentsk∈𝒦k\\in\\mathcal\{K\},GkG\_\{k\}is the local subgraph data defined on𝒱k\\mathcal\{V\}\_\{k\}, the set of nodes held by the agentkk\.GkvG^\{v\}\_\{k\}is the part of the ego\-graph data held by agentkkand related tovv\. We also need to define boundary nodes, that is, nodes that are connected to nodes held by different agents\. For each agentkk, the set of foreign boundary nodes is the set of nodes held by a different agent and connected to a node seen bykk:𝒱f,k=\{v;∃u∈𝒱k∣\(u,v\)∈ℰ,v∈𝒱∖𝒱k\}\\mathcal\{V\}\_\{f,k\}=\\\{v;\\exists u\\in\\mathcal\{V\}\_\{k\}\\mid\(u,v\)\\in\\mathcal\{E\},v\\in\\mathcal\{V\}\\setminus\\mathcal\{V\}\_\{k\}\\\}\. The set of private boundary nodes is the set of nodes private tokkthat are foreign nodes of other agents:𝒱p,k=⋃l∈𝒦∖k\{v∈𝒱k∣v∈𝒱f,l\}\\mathcal\{V\}\_\{p,k\}=\\bigcup\_\{l\\in\\mathcal\{K\}\\setminus k\}\\\{v\\in\\mathcal\{V\}\_\{k\}\\mid v\\in\\mathcal\{V\}\_\{f,l\}\\\}\.

A round of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)begins with the broadcast of common model parameters to all agents \(line[5](https://arxiv.org/html/2609.02984#alg3.l5)\), which then initialize their local model parameters accordingly \(line[8](https://arxiv.org/html/2609.02984#alg3.l8)\)\. Two inference modes are considered:

- •Isolated inference:each agent performs inference solely on its local subgraph dataGkG\_\{k\}\(line[12](https://arxiv.org/html/2609.02984#alg3.l12)\)\.
- •Collaborative inference:agents collectively replicate the inference process of a message\-passing neural network \(MPNN\) in a distributed fashion\. This is executed iteratively over theLLlayers of the MPNN\. At each iterationll, agents communicate \(line[18](https://arxiv.org/html/2609.02984#alg3.l18)\), and request \(line[19](https://arxiv.org/html/2609.02984#alg3.l19)\) thell\-th layer embeddings of their foreign boundary nodes𝒱f,k\\mathcal\{V\}\_\{f,k\}through the server, then compute the\(l\+1\)\(l\{\+\}1\)\-th embeddings using the MPNN inference equations \([18](https://arxiv.org/html/2609.02984#S3.E18)\), \([19](https://arxiv.org/html/2609.02984#S3.E19)\), and \([20](https://arxiv.org/html/2609.02984#S3.E20)\)\.

The remaining steps are common to both inference modes\. Node representations are sent to the server for aggregation \(lines[24](https://arxiv.org/html/2609.02984#alg3.l24)–[34](https://arxiv.org/html/2609.02984#alg3.l34)\)\. Local model parameters are updated using gradient descent based on the computed loss \(line[28](https://arxiv.org/html/2609.02984#alg3.l28)\), and subsequently sent to the server for aggregation \(line[14](https://arxiv.org/html/2609.02984#alg1.l14)\)\.

Algorithm 3Federated training of subgraph GNN1:

𝒦=\{1,…,K\}\\mathcal\{K\}=\\\{1,\\ldots,K\\\}set of agents ,

Fψ,θF\_\{\\psi,\\theta\}: a GNN parametrized by

ψ∈Ψ,θ∈Θ\\psi\\in\\Psi,\\theta\\in\\Theta, global graph topology

𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\),

\{𝒱k\}k∈𝒦\\\{\\mathcal\{V\}\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}subset of nodes

𝒱\\mathcal\{V\}held by each agent, foreign boundary nodes

\{𝒱p,k\}k∈𝒦\\\{\\mathcal\{V\}\_\{p,k\}\\\}\_\{k\\in\\mathcal\{K\}\}known by each agent, private boundary nodes

\{𝒱f,k\}k∈𝒦\\\{\\mathcal\{V\}\_\{f,k\}\\\}\_\{k\\in\\mathcal\{K\}\}known by each agent, boundary edges

\{ℰk\}k∈𝒦\\\{\\mathcal\{E\}\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}known by each agent, learning rate

η\\eta, initial model parameters

\{ψ⁡\(0\),θ⁡\(0\)\}\\\{\\psi\(0\),\\theta\(0\)\\\}and initial node embeddings

\{zv​\(0\)\}v∈𝒱\\\{z\_\{v\}\(0\)\\\}\_\{v\\in\\mathcal\{V\}\}, local graph data

\{Gk\}k∈𝒦\\\{G\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}held by each agent, number of training rounds

TT, booleanisolated inference,ollaborative inference

2:— Training loop —

3:foreach round

t=1,…,Tt=1,\\ldots,Tdo

4:Server:

5:Broadcast current model parameters

θ⁡\(t−1\),ψ⁡\(t−1\)\\theta\(t\{\-\}1\),\\psi\(t\{\-\}1\)to all agents

6:foreach agent

k∈𝒦k\\in\\mathcal\{K\}in paralleldo

7:— Local Model Initialization —

8:Initialize

Fψk​\(t−12\),θk​\(t−12\),F\_\{\\psi\_\{k\}\(t\{\-\}\\tfrac\{1\}\{2\}\),\\theta\_\{k\}\(t\{\-\}\\tfrac\{1\}\{2\}\),\}such as

θk​\(t−12\)←θ⁡\(t−1\)\\theta\_\{k\}\(t\{\-\}\\tfrac\{1\}\{2\}\)\\leftarrow\\theta\(t\{\-\}1\),

ψk​\(t−12\)←ψ⁡\(t−1\)\\psi\_\{k\}\(t\{\-\}\\tfrac\{1\}\{2\}\)\\leftarrow\\psi\(t\{\-\}1\)
9:— Inference Phase —

10:ifisolated inferencethen

11:— Local Inference on Isolated Subgraph —

12:Decompose

GkG\_\{k\}into

\{Gkv\}v∈𝒱k\\\{G\_\{k\}^\{v\}\\\}\_\{v\\in\\mathcal\{V\}\_\{k\}\}and compute

zv​\(t−12\)=ϕθk​\(t−12\)​\(Gkv,v\)z\_\{v\}\(t\-\\tfrac\{1\}\{2\}\)=\\phi\_\{\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)\}\(G\_\{k\}^\{v\},v\)for each node555In practice, this step can be efficiently computed in parallel using vectorization libraries for GNN such as PyTorch Geometric\([Fey et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib408)\)\.

v∈𝒱kv\\in\\mathcal\{V\}\_\{k\}as in \([21](https://arxiv.org/html/2609.02984#S3.E21)\)

13:endif

14:ifcollaborative inferencethen

15:— Collaborative Inference —

16:Compute

\{hv\(0\)\}v∈𝒱k\\\{h\_\{v\}^\{\(0\)\}\\\}\_\{v\\in\\mathcal\{V\}\_\{k\}\}as in \([17](https://arxiv.org/html/2609.02984#S3.E17)\) and set

∀v∈𝒱k,hv\(0\)​\(t\)←hv\(0\)\\forall v\\in\\mathcal\{V\}\_\{k\},h\_\{v\}^\{\(0\)\}\(t\)\\leftarrow h\_\{v\}^\{\(0\)\}
17:for

l=0,…,L−1l=0,\\ldots,L\-1do

18:Send

\{hv\(l\)​\(t\)\}v∈𝒱p,k\\\{h\_\{v\}^\{\(l\)\}\(t\)\\\}\_\{v\\in\\mathcal\{V\}\_\{p,k\}\}to the server

19:Request

\{hv\(l\)​\(t−1\)\}v∈𝒱f,k\\\{h\_\{v\}^\{\(l\)\}\(t\-1\)\\\}\_\{v\\in\\mathcal\{V\}\_\{f,k\}\}from the server

20:Upon reception, compute \([18](https://arxiv.org/html/2609.02984#S3.E18)\) \([19](https://arxiv.org/html/2609.02984#S3.E19)\) \([20](https://arxiv.org/html/2609.02984#S3.E20)\) for each††footnotemark:

v∈𝒱kv\\in\\mathcal\{V\}\_\{k\}
21:endfor

22:

zv​\(t−12\)←hv\(L\)z\_\{v\}\(t\-\\frac\{1\}\{2\}\)\\leftarrow h\_\{v\}^\{\(L\)\}for all

v∈𝒱kv\\in\\mathcal\{V\}\_\{k\}
23:endif

24:Send

\{zv​\(t−12\)\}v∈𝒱k\\\{z\_\{v\}\(t\-\\frac\{1\}\{2\}\)\\\}\_\{v\\in\\mathcal\{V\}\_\{k\}\}to the server

25:— Loss Computation —

26:Compute

ℒ^k​\(Fψk​\(t−12\),θk​\(t−12\)\)\\hat\{\\mathcal\{L\}\}\_\{k\}\\left\(F\_\{\\psi\_\{k\}\(t\-\\frac\{1\}\{2\}\),\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)\}\\right\)
27:— Local Model Update —

28:

θk​\(t\)←θk​\(t−12\)−η⋅∂ℒk^∂θk\(l\)​\(t−12\),ψk​\(t\)←ψk​\(t−12\)−η⋅∂ℒk^∂ψk\(l\)​\(t−12\)\\theta\_\{k\}\(t\)\\leftarrow\\theta\_\{k\}\(t\-\\tfrac\{1\}\{2\}\)\-\\eta\\cdot\\frac\{\\partial\\hat\{\\mathcal\{L\}\_\{k\}\}\}\{\\partial\\theta\_\{k\}^\{\(l\)\}\(t\-\\tfrac\{1\}\{2\}\)\},\\quad\\psi\_\{k\}\(t\)\\leftarrow\\psi\_\{k\}\(t\-\\tfrac\{1\}\{2\}\)\-\\eta\\cdot\\frac\{\\partial\\hat\{\\mathcal\{L\}\_\{k\}\}\}\{\\partial\\psi\_\{k\}^\{\(l\)\}\(t\-\\tfrac\{1\}\{2\}\)\}
29:Send

θk​\(t\),ψk​\(t\)\\theta\_\{k\}\(t\),\\psi\_\{k\}\(t\)to the server

30:endfor

31:— Server Aggregation —

32:Server:

33:

θ⁡\(t\)←∑k∈𝒦NkN​θk​\(t\),ψ⁡\(t\)←∑k∈𝒦tNkN​ψk​\(t\)\\theta\(t\)\\leftarrow\\sum\_\{k\\in\\mathcal\{K\}\}\\frac\{N\_\{k\}\}\{N\}\\theta\_\{k\}\(t\),\\psi\(t\)\\leftarrow\\sum\_\{k\\in\\mathcal\{K\}\_\{t\}\}\\frac\{N\_\{k\}\}\{N\}\\psi\_\{k\}\(t\)
34:

\{zv​\(t\)\}v∈𝒱←⋃k∈𝒦\{zv​\(t−12\)\}v∈𝒱k\\\{z\_\{v\}\(t\)\\\}\_\{v\\in\\mathcal\{V\}\}\\leftarrow\\bigcup\_\{k\\in\\mathcal\{K\}\}\\\{z\_\{v\}\(t\-\\frac\{1\}\{2\}\)\\\}\_\{v\\in\\mathcal\{V\}\_\{k\}\}
35:endfor

36:— Final Output —

37:return

ψ⁡\(T\),θ⁡\(T\)\\psi\(T\),\\theta\(T\),

\{zv​\(T\)\}v∈𝒱\\\{z\_\{v\}\(T\)\\\}\_\{v\\in\\mathcal\{V\}\}

### 3\.3Graph Data Partition, Heterogeneity and Privacy Challenges

The homogeneity of graph data collected across agents is central to the problem setup in \([25](https://arxiv.org/html/2609.02984#S3.E25)\)–\([28](https://arxiv.org/html/2609.02984#S3.E28)\)\. However, data heterogeneity, which is well\-studied for Euclidean data in Section[2\.2](https://arxiv.org/html/2609.02984#S2.SS2), also challenges collaborative learning in the graph setting\. We now describe graph\-specific heterogeneity from two angles: data partitioning and statistical heterogeneity\.

#### 3\.3\.1Graph Partition Taxonomy

A key feature of distributed learning is how the data is partitioned across agents\. We subsequently propose a characterization of the different partitions for graph data appearing in the literature\. To structure our analysis, we build on the taxonomy for Euclidean data partitions \(Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1)\) and adapt it to graph data, following prior efforts in\([Zhang et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib337);[He et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib14);[Liu et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib168)\)\. We continue to discuss the two main cases introduced in Section[3\.2](https://arxiv.org/html/2609.02984#S3.SS2): \(i\) agents holding multiple graph instances, and \(ii\) agents holding a subgraph as described in Figure[13](https://arxiv.org/html/2609.02984#S3.F13)\.

##### Multiple graph instances\.

We examine the case where agents observe full graph instances as introduced in Section[3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px1)\. To formalize, we assume the existence of a global datasetSSmade of full graph instancesS=\{\(Gi,yi\)\}i∈ℐS=\\\{\(G^\{i\},y^\{i\}\)\\\}\_\{i\\in\\mathcal\{I\}\}\. We distinguish two ways in which graph instances are typically distributed across agents, illustrated in Figure[15](https://arxiv.org/html/2609.02984#S3.F15):

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/mgi_partition.jpg)Figure 15:Illustration of the different partitioning schemes for the case of multiple graph instances\. Multiple graph instances can be partitioned \(a\) horizontally, where agents hold unrelated graph samples, or \(b\) vertically, where graph instances associated with the same label are divided among agents\.- •Horizontal topology partitionanalogous to the taxonomy ofinter\-graph FL\([Xie et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib189)\)orgraph level FL\([He et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib14)\)\. Similar to horizontal partitioning seen in Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1), the indexℐ\\mathcal\{I\}ofSSis partitioned across agents into local indexesℐk⊂ℐ\\mathcal\{I\}\_\{k\}\\subset\\mathcal\{I\}such as∪k∈𝒦ℐk=ℐ\\cup\_\{k\\in\\mathcal\{K\}\}\\mathcal\{I\}\_\{k\}=\\mathcal\{I\}, yielding datasets for agentkk: Sk=\{\(\(Ai,Xi\),yi\)\}i∈ℐk\.S\_\{k\}=\\left\\\{\(\(A^\{i\},X^\{i\}\),y^\{i\}\)\\right\\\}\_\{i\\in\\mathcal\{I\}\_\{k\}\}\.That is, agents possess data about distinct graph samples indexed byℐk\\mathcal\{I\}\_\{k\}, for example, laboratories observe distinct sets of protein graphs\([Xie et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib189);[He et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib250)\)\.
- •Vertical topology partitionFor each sample of graph dataGiG^\{i\}, the topology𝒢i\\mathcal\{G\}^\{i\}is partitioned intoKKdisconnected components𝒢ki\\mathcal\{G\}\_\{k\}^\{i\}held by different agents\. The node featuresXiX^\{i\}are split so that agentkkobserves the features of the nodes𝒢ki\\mathcal\{G\}\_\{k\}^\{i\}\. The label, usually a graph\-label for these cases, is held by agentk∗k^\{\*\}, yielding datasets: Sk=\{Gki\}i∈ℐ;Sk∗=\{Gki,yi\}i∈ℐ\.S\_\{k\}=\\\{G\_\{k\}^\{i\}\\\}\_\{i\\in\\mathcal\{I\}\};\\quad S\_\{k^\{\*\}\}=\\\{G\_\{k\}^\{i\},y^\{i\}\\\}\_\{i\\in\\mathcal\{I\}\}\.This partitioning is typical in multimodal scenarios where the graphGkiG\_\{k\}^\{i\}represents one modalitykkfor the same sampleii\. These modalities are combined to solve a task related to the same sampleGiG^\{i\}\. An example is to diagnose patients based on the combination of different physiological observations, represented as graphs and held by different hospitals\([Dong et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib370)\)\. This vertical topology partition is studied for centralized training\([Qian et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib290)\), but has not yet been implemented in distributed training settings\.

##### Subgraphs of a global graph\.

We now consider a setting where agents observe parts of a single, usually large\-scale, global graph\. In such scenarios, agents have only partial local views of the global graph, making learning over isolated graph data a central challenge\.

In the most general case, the global, possibly heterogeneous graph, is partitioned across agents, such as each agentkkobserves a local heterogeneous subgraph𝒢k=\(𝒱k,ℰk,𝒪k,ℛk\)\\mathcal\{G\}\_\{k\}=\(\\mathcal\{V\}\_\{k\},\\mathcal\{E\}\_\{k\},\\mathcal\{O\}\_\{k\},\\mathcal\{R\}\_\{k\}\)\. That is, each agent observes a subset of nodes𝒱k⊆𝒱\\mathcal\{V\}\_\{k\}\\subseteq\\mathcal\{V\}of a subset of types𝒪k⊆𝒪\\mathcal\{O\}\_\{k\}\\subseteq\\mathcal\{O\}, and linksℰk⊆ℰ\\mathcal\{E\}\_\{k\}\\subseteq\\mathcal\{E\}of subset of relation typesℛk⊆ℛ\\mathcal\{R\}\_\{k\}\\subseteq\\mathcal\{R\}\. These may induce a restricted type and relation mappingρk:𝒱k→𝒪k\\rho\_\{k\}:\\mathcal\{V\}\_\{k\}\\to\\mathcal\{O\}\_\{k\}andρk:ℰk→ℛk\\rho\_\{k\}:\\mathcal\{E\}\_\{k\}\\to\\mathcal\{R\}\_\{k\}\. For example, in a citation network, we might have object types𝒪=\{Author,Paper,Venue\}\\mathcal\{O\}=\\\{\\text\{Author\},\\text\{Paper\},\\text\{Venue\}\\\}and relation typesℛ=\{Writes,PublishedIn,Cites\}\\mathcal\{R\}=\\\{\\text\{Writes\},\\text\{PublishedIn\},\\text\{Cites\}\\\}\.

A graph is characterized by three main components: its topology, the types of nodes and relations along with their mappings, and the node features and potential labels\. Accordingly, partitions can be defined along these dimensions, giving rise to the following taxonomy of graph partitions illustrated in Figures[16](https://arxiv.org/html/2609.02984#S3.F16)–[17](https://arxiv.org/html/2609.02984#S3.F17)\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/homogeneous_graph.jpg)Figure 16:Illustration of different subgraph partitioning schemes in a homogeneous graph\.Intopology partitioning\(a\), the graph topology is divided across agents by edge cuts\. Each agent observes the complete set of node features for the nodes assigned to it\. Two cases can be distinguished based on the subgraph size:ego\-graph partition\(i\), where an agent only observes a node together with its direct neighbors, thus forming an ego\-graph; andsubgraph partition\(ii\), where an agent holds a larger portion of the graph consisting of multiple, aligned nodes\. Infeature partitioning\(b\), agents share the same set of nodes but access different subsets of features\. Two scenarios arise: infull topology knowledge\(i\), all agents know the complete set of graph links; whereas in thetopology\-freecase \(ii\), only a subset of agents has access to link information\.- •Topology partition\.The partition of the graph is done along the graph structureℋ𝒢=\(𝒱,ℰ\)\\mathcal\{H\}\_\{\\mathcal\{G\}\}=\(\\mathcal\{V\},\\mathcal\{E\}\), without considering the node types, and is achieved by : - –Edge\-cut:Nodes are divided into \(possibly overlapping\) sets⋃k∈𝒦𝒱k\\bigcup\_\{k\\in\\mathcal\{K\}\}\\mathcal\{V\}\_\{k\}, inducing local edge setsℰ𝒱k×𝒱k=\{\(u,v\)∈ℰ∣\(u,v\)∈𝒱k×𝒱k\}\\mathcal\{E\}\_\{\{\\mathcal\{V\}\_\{k\}\}\\times\{\\mathcal\{V\}\_\{k\}\}\}=\\\{\(u,v\)\\in\\mathcal\{E\}\\mid\(u,v\)\\in\{\\mathcal\{V\}\_\{k\}\}\\times\\mathcal\{V\}\_\{k\}\\\}\. For example, consider a network of IoT devices related by a global graph\. Telecom base stations monitor different geographical areas, resulting in a different set of devices observed\([Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\)\. Usually, agents also know about their foreign nodes𝒱f,k=\{v;∃u∈𝒱k∣\(u,v\)∈ℰ,v∈𝒱∖𝒱k\}\\mathcal\{V\}\_\{f,k\}=\\\{v;\\exists u\\in\\mathcal\{V\}\_\{k\}\\mid\(u,v\)\\in\\mathcal\{E\},v\\in\\mathcal\{V\}\\setminus\\mathcal\{V\}\_\{k\}\\\}and the related edges cutℰc​u​t,k:=ℰ𝒱k×𝒱f,k=\{\(u,v\)∈ℰ∣u∈𝒱k​v∈𝒱f,k\}\\mathcal\{E\}\_\{cut,k\}:=\\mathcal\{E\}\_\{\{\\mathcal\{V\}\_\{k\}\}\\times\{\\mathcal\{V\}\_\{f,k\}\}\}=\\\{\(u,v\)\\in\\mathcal\{E\}\\mid u\\in\{\\mathcal\{V\}\_\{k\}\}\\ v\\in\{\\mathcal\{V\}\_\{f,k\}\}\\\}but not their features\. - –Vertex\-cut:Edges are divided into \(possibly overlapping\) sets⋃k∈𝒦ℰk\\bigcup\_\{k\\in\\mathcal\{K\}\}\\mathcal\{E\}\_\{k\}, inducing node sets𝒱k=𝒱∣ℰk=\{u∈𝒱\|∃v∈𝒱,\(u,v\)∈ℰkor\(v,u\)∈ℰk\}\\mathcal\{V\}\_\{k\}=\\mathcal\{V\}\_\{\\mid\\mathcal\{E\}\_\{k\}\}=\\left\\\{u\\in\\mathcal\{V\}\\,\\middle\|\\,\\exists v\\in\\mathcal\{V\},\\ \(u,v\)\\in\\mathcal\{E\}\_\{k\}\\text\{ or \}\(v,u\)\\in\\mathcal\{E\}\_\{k\}\\right\\\}\. For example, a device records the items browsed by a single user in a marketplace, forming an interaction graph private to the device\([Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195)\)\. We define the set ofvertex\-cut, vertices which are shared between agents:𝒱cut,k=\{v∈𝒱k\|∃k′≠k,v∈𝒱k′\}\.\\mathcal\{V\}\_\{\\text\{cut\},k\}=\\left\\\{v\\in\\mathcal\{V\}\_\{k\}\\,\\middle\|\\,\\exists k^\{\\prime\}\\neq k,\\ v\\in\\mathcal\{V\}\_\{k^\{\\prime\}\}\\right\\\}\.The interactions represent the edges private to the agent, and some items represent the vertex cut between agents\.
- •Feature partition\.Analogous to vertical partitioning for Euclidean data seen in Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1), the feature space𝒳\\mathcal\{X\}is split among agents𝒳=⊕k∈𝒦𝒳k\\mathcal\{X\}=\\oplus\_\{k\\in\\mathcal\{K\}\}\\mathcal\{X\}\_\{k\}, resulting invertical feature partitioningof the features matrixXX\. Specifically, for each object typeo∈𝒪o\\in\\mathcal\{O\}, the feature matricesXoX\_\{o\}is partitioned across feature dimensions: for each nodevvof object typeoo,xv=\(xv,k\)k∈𝒦x\_\{v\}=\(x\_\{v,k\}\)\_\{k\\in\\mathcal\{K\}\}, wherexv,k∈𝒳kx\_\{v,k\}\\in\\mathcal\{X\}\_\{k\}is the part of vectorxvx\_\{v\}observed by agentkk\.\. Feature partition may result topology partition as well, which can be classified as a vertical topology partition, with a vertex cut\.
- •Schema partition\.In a heterogeneous graph𝒢\\mathcal\{G\}, partitioning can be defined along the network schema𝒯𝒢=\(𝒪,ℛ\)\\mathcal\{T\}\_\{\\mathcal\{G\}\}=\(\\mathcal\{O\},\\mathcal\{R\}\), by partitioning: - –The object types:Partitioning the set of object types𝒪\\mathcal\{O\}into disjoint subsets\{𝒪k\}k\\\{\\mathcal\{O\}\_\{k\}\\\}\_\{k\}, i\.e\.,𝒪=⋃k𝒪k\\mathcal\{O\}=\\bigcup\_\{k\}\\mathcal\{O\}\_\{k\}, induces a corresponding partition of the node set in the original heterogeneous graph via the inverse mapping of the type functionτ:𝒱→𝒪\\tau:\\mathcal\{V\}\\to\\mathcal\{O\}, namely𝒱k=𝒪k−1:=τ−1​\(𝒪k\)=\{v∈𝒱∣τ⁡\(v\)∈𝒪k\}\.\\mathcal\{V\}\_\{k\}=\{\\mathcal\{O\}\_\{k\}\}^\{\-1\}:=\\tau^\{\-1\}\(\\mathcal\{O\}\_\{k\}\)=\\\{v\\in\\mathcal\{V\}\\mid\\tau\(v\)\\in\\mathcal\{O\}\_\{k\}\\\}\.Object type based partition often leads to edge cuts\. - –The relation types:Similarly, partitioning the set of relation typesℛ\\mathcal\{R\}into disjoint subsets\{ℛk\}k\\\{\\mathcal\{R\}\_\{k\}\\\}\_\{k\}, i\.e\.,ℛ=⋃kℛk\\mathcal\{R\}=\\bigcup\_\{k\}\\mathcal\{R\}\_\{k\}, induces a partition of the edge set via the inverse mapping of the relation functionρ:ℰ→ℛ\\rho:\\mathcal\{E\}\\to\\mathcal\{R\}, yieldingℰk=ℛk−1:=ρ−1​\(ℛk\)=\{e∈ℰ∣ρ⁡\(e\)∈ℛk\}\.\\mathcal\{E\}\_\{k\}=\{\\mathcal\{R\}\_\{k\}\}^\{\-1\}:=\\rho^\{\-1\}\(\\mathcal\{R\}\_\{k\}\)=\\\{e\\in\\mathcal\{E\}\\mid\\rho\(e\)\\in\\mathcal\{R\}\_\{k\}\\\}\.Relation type based partition often leads to vertex cut\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/heterogeneous_graph.jpg)Figure 17:Illustration of different subgraph partitioning schemes in a heterogeneous graph\. The subgraph can be partitioned at two levels: either by dividing the node set𝒱\\mathcal\{V\}among agents, or by partitioning the schema graph, partitioning the object types𝒪\\mathcal\{O\}across agents\.Graph partition of topology, schema, or features can be combined freely, leading to a high number of possible combinations of data distribution across the agents\. In Table[2](https://arxiv.org/html/2609.02984#S3.T2)we list the combinations that are most prominent in the literature\.

- •Consideringhomogeneous graphs,topology partition, resulting horizontal feature partition, is the most frequently studied case\. Agents may hold asubgraphof the global graph due to geographic limitations, for example, in mobile networks, or due to collaborating system providers in IoT systems\([Zhang et al\., 2021d](https://arxiv.org/html/2609.02984#bib.bib180)\)\. The other typical case of topology partition is when agents hold a single node and perform inference through theego\-graph\. This is the typical case of interacting autonomous vehicles, robots, or drones\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)\. Traffic flow forecasting\([Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\), location\-based user recommendation\([Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\), vehicle motion forecasting\([Casas et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib369)\), or swarm navigation\([Zhou et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib368);[Solodova et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib380);[Gao et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib248);[Blumenkamp et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib264)\)are prominent tasks\.
- •Vertical feature partitionofhomogeneous graphsarises when agents hold partial information about the nodes, for example, spending habits of customers are known partially by e\-commerce sites\([Ni et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib171);[Chen et al\., 2024c](https://arxiv.org/html/2609.02984#bib.bib184);[He et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib193)\), and hospitals have partial records of patients\([Fu et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib197)\)\. In most scenarios,all agents hold a graph\([Ni et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib171);[Chen et al\., 2024c](https://arxiv.org/html/2609.02984#bib.bib184);[He et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib193)\), but alternatively, some agents may be topology\-free, holding only Euclidean data\([Mei et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib169);[Cheung et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib170);[Fu et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib197)\)\.
- •Key examples ofheterogeneous graphsare knowledge graphs\([Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175);[Wang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib191)\)and recommender systems\([Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195);[Liu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib186);[Qiu et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib182);[Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\)\. Heterogeneous graphs can be distributed without considering the type of nodes, resulting intopology partition\. Within that,subgraph partitionemerges when companies observe the ego\-graphs of a group of customers\([Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195)\)\. The requirement to preserve node privacy, on the other hand, may lead toego\-graphpartition, where one agent knows about the ego graph of a single customer\([Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217)\)\.
- •Type\-based partitioninheterogeneous graphsis typical in knowledge graphs, for example, when one agent holds a patient\-symptom database, and another one a symptom\-disease database\([Peng et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib176)\), with a common goal of training a patient\-to\-disease predictor, leading to agents holdingtype\-based subgraphs\. Privacy preservation under collaborating agents, for example, in health applications, can require a combination oftype\-basedandego\-graphpartition\([Sun et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib371)\)\.

#### 3\.3\.2Statistical Heterogeneity in Graphs

Section[2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2)outlines the main forms of statistical imbalance in Euclidean data,quantity skewanddistributional shift, which in turn includecovariate shiftandconcept shift\. In the Euclidean setting, data is typically drawn from a joint distribution𝒟⁡\(x,y\)\\mathcal\{D\}\(x,y\)\. In contrast, for graph data, the feature matrixXXis structured through an adjacency matrix, which together forms a graph sampleG=\(A,X\)G=\(A,X\)\. The coupling between topology, features, and labels is captured by the joint distribution𝒟⁡\(G,y\)\\mathcal\{D\}\(G,y\), and brings new dimensions in the possible forms of statistical imbalance in the data, illustrated in Figure[18](https://arxiv.org/html/2609.02984#S3.F18)\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/graph_shifts.jpg)Figure 18:Different types of statistical heterogeneity in partitioned graph data\.\(a\)Quantity shiftrefers to differences in dataset size across agents sampling from the same distribution, reflected in varying \(i\) subgraph sizes or \(ii\) numbers of graph instances\. \(b\)Covariate shiftrefers to differences in input graph distributions, reflected in \(i\) node or edge features or \(ii\) graph structures\. \(c\)Concept shiftrefers to differences in the mapping from input graphs to labels \(colored on the figure\), reflected in \(i\)feature–conceptor \(ii\)structure–conceptshifts\.Quantity shiftcan be observed in graph data when agents hold different amounts of graph information\. In the subgraph case, this can be observed when the size of subgraphs, measured by the number of edges or nodes, differs across agents\. In the full graph instances case, quantity shift can be observed when agents hold different numbers of graph samples\.

Distributional shiftsexhibit greater diversity for graph data, compared to the Euclidean case, due to the additional information of the graph structureAA, as discussed in detail in\([Zhang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib276)\)\. Below, we characterize graph\-specific distributional shifts using the global properties\(A,X,y\)\(A,X,y\)\. However, it is important to note that MPNNs operate on the collectionLL\-hop ego graphs data\{Gv\}v∈𝒱k\\\{G^\{v\}\\\}\_\{v\\in\\mathcal\{V\}\_\{k\}\}centered at each node rather than the full graph dataG=\(A,X\)G=\(A,X\), and thus experience these shifts in practice through changes in local neighborhood distributions𝒟⁡\(Gv\)\\mathcal\{D\}\(G^\{v\}\)\.

##### Covariate shift\.

Covariate shift refers to differences in the input graph distribution𝒟⁡\(G\)\\mathcal\{D\}\(G\)across agents, while the conditional label distribution𝒟⁡\(y∣G\)\\mathcal\{D\}\(y\\mid G\)remains unchanged\. Alternatively, one can view it from the label perspective as discussed in Section[2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2), where𝒟⁡\(y\)\\mathcal\{D\}\(y\)and𝒟⁡\(G∣y\)\\mathcal\{D\}\(G\\mid y\)differ across agents\. We adopt the input perspective, which is more common in the literature, and introduce graph\-specific covariate shifts,feature shiftandstructure shift\([Zhang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib276)\)\.

- •Feature shiftfor graph data is analogous tofeature shiftfor Euclidean data seen in Section[2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1)\. It occurs when, given a graph topologyAA, the node features distribution𝒟⁡\(X∣A\)\\mathcal\{D\}\(X\\mid A\)differs\([Zhang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib276)\)\. For example, in superpixel graphs of natural scenes\([Monti et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib272)\), one agent may use grayscale images while another uses color images from the same scene\. In email networks\([Bojchevski and Günnemann, 2018](https://arxiv.org/html/2609.02984#bib.bib273)\), changes in the language used across organizations can lead to different node features\.
- •Structure shiftis specific to graph data\. Real\-world graphs often exhibit structural diversity across regions and nodes\. In collaborative scnearios it means that the topology distribution𝒟⁡\(A\)\\mathcal\{D\}\(A\)may differ significantly across agents\([Liu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib275)\)\. The factors influencing structural shifts are many, often driven by hidden latent variables of the environment\([Wu et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib270);[Gui et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib257)\)or differences in data domains\([Tan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib423)\)\. These shifts can be present from the start or emerge when new agents or nodes with distinct structures or labels arrive\. While structural shifts can be characterized through the spectrum ofAA\([Tan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib423)\), three large categories are usually considered:Size shiftsoccur when graphs or subgraphs differ in the number of vertices, e\.g\., sentence graphs vary with text length, and it is the most discussed in the literature\([Yehudai et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib277)\)\.Density shiftsare present when the edge density, measured by different metrics \(node degree, clustering coefficient\), differs\. This is a structure shift that could prove to be the most critical to GNN generalization capabilities\([Bazhenov et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib258)\)\.Motif shiftsoccur when the base of the graph is common, but only a portion of the graph differs in terms of typical connection patterns\([Wu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib271)\)\.

##### Concept shift\.

Concept shift arises when the conditional distribution𝒟⁡\(y∣G\)\\mathcal\{D\}\(y\\mid G\)differs between agents\.

- •Feature concept shiftin graph data is analog toconcept shiftseen in Section[2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2)\. It refers to changes in the concept distribution𝒟⁡\(y∣X\)\\mathcal\{D\}\(y\\mid X\), assuming the topologyAAis fixed\. For example, in the case of full graph instances, some superpixel graphs might have a tight correlation between color and class \(e\.g\., blue for water\), while others do not\([Gui et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib257)\)\. In the subgraph case, an email network, agents holding emails from respectively older and younger users, may observe different vocabulary\-class relations\. These shifts are particularly challenging for GNNs since they assume stable feature\-label correlations\([Duong et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib254);[Faber et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib255)\)\.
- •Structure concept shiftfor graph refers to differences in𝒟⁡\(y∣A\)\\mathcal\{D\}\(y\\mid A\)\(or𝒟⁡\(A∣y\)\\mathcal\{D\}\(A\\mid y\)\)\([Liu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib275)\)\. This structural concept has been historically modeled using using theStochastic Block Model\([Holland et al\., 1983](https://arxiv.org/html/2609.02984#bib.bib322);[Karrer and Newman, 2011](https://arxiv.org/html/2609.02984#bib.bib321);[Abbe, 2018](https://arxiv.org/html/2609.02984#bib.bib323)\)that assumes𝒟u​\(Au​v∣\{yv,yu\}\)\\mathcal\{D\}\_\{u\}\(A\_\{uv\}\\mid\\\{y\_\{v\},y\_\{u\}\\\}\)uniform across the graph\. More commonly, structure concept shift is connected to the well\-studied principle of graph homophily\([Zhu et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib192);[Platonov et al\., 2023b](https://arxiv.org/html/2609.02984#bib.bib379);[Platonov et al\., 2023a](https://arxiv.org/html/2609.02984#bib.bib256);[Fuchsgruber et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib283)\), which states that nodes tend to connect to nodes of the same labels\. Constant homophily level is critical in centralized training, as GNN tends to perform gradually worse on heterophilous graphs, referred to ashomophily bias\([Tan et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib282)\)\. In the case of distributed graphs, differences in the graphs homophily level between agents, referred to ashomophily conflict\([Tan et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib282)\), lead to decreased learning accuracy\.

While the distinctions between feature shift, structure shift, covariate shift, and concept shift help understanding the challenges of learning, this categorization is limiting forevolvinggraphs, and requires further research\. Since MPNNs operate on overlappingLL\-hop ego graphs, topologyAAand featuresXXare inherently coupled: when nodes join or leave a network, both𝒟⁡\(A\)\\mathcal\{D\}\(A\)and𝒟⁡\(X\)\\mathcal\{D\}\(X\)change concurrently across multiple overlapping neighborhoods\([Kim et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib421)\)\. In the subgraph case, when partition boundaries evolve \(e\.g\., nodes reassigned across agents\), agents observe different local distributions despite a unified global graph\. Moreover, these shifts have heterogeneous impacts across ego graphs, since not all nodes affect the distribution equally when joining or leaving the network\. For example, highly connected nodes, nodes with rare labels, and nodes violating local homophily levels have greater effects\.

#### 3\.3\.3System Heterogeneity in Collaborative Learning over Graph Data

Collaborative learning over graphs must addressdevice,network, andtaskheterogeneity, as in the Euclidean case discussed in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3)\. However, for the inference process, both under and after learning, graph\-structured data introduces new challenges that stem from the interdependence between the system constraints and the graph topology itself\. We discuss these challenges below\.

##### Device and network constraints under collaborative inference\.

Unlike Euclidean federated learning, where inference is local \(Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\), subgraph inference is inherentlycollaborativeand requires agents to exchange node embeddings across subgraph boundaries during message passing \(lines[18](https://arxiv.org/html/2609.02984#alg3.l18)–[19](https://arxiv.org/html/2609.02984#alg3.l19)of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\)\. While collaborative inference also appears in vertical federated learning \(Algorithm[2](https://arxiv.org/html/2609.02984#alg2)\), graph\-structured data differs in that the graph topology itself determines inference dependencies between nodes and communication patterns between agents \(Figure[14](https://arxiv.org/html/2609.02984#S3.F14)\), resulting in tightly coupled system constraints\. Building on the device and network heterogeneities identified in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3), collaborative inference over graphs does not introduce entirely new aspects\. Instead, the graph topology modulates and amplifies existing computational and communication heterogeneities:

- •Computational heterogeneityis amplified by the structure of local subgraphs\. Message passing cost scales with topology:computationgrows with the number of edges in𝒢k\\mathcal\{G\}\_\{k\}times the number of layersLL, whilememorymust store boundary embeddings\{hv\(l\)\}v∈𝒱f,k,l=1,…,L\\\{h\_\{v\}^\{\(l\)\}\\\}\_\{v\\in\\mathcal\{V\}\_\{f,k\},\\,l=1,\\ldots,L\}\(line[19](https://arxiv.org/html/2609.02984#alg3.l19)\)\. Consequently, dense subgraphs or highly connected agents induce higher computational and memory loads, exacerbating eventual device heterogeneity\.
- •Communication heterogeneityis further amplified by the topology of inter\-agent boundaries\. Incoming and outgoing communication volumes scale with the number of foreign and private boundary nodes,𝒱f,k\\mathcal\{V\}\_\{f,k\}and𝒱p,k\\mathcal\{V\}\_\{p,k\}, which typically correlate with the centrality of boundary nodes\. These unbalanced loads further compound the communication heterogeneities identified in Section[2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3), where unreliable links can delay or corrupt embedding exchanges\. When subgraph boundaries span many agents, synchronizing boundary node embeddings incurs additional communication overhead and delays\.

The joint effects of communication and computing heterogeneities induce apartitioning tradeoff\. Minimizing edge cuts reduces cross\-agent communication but may distribute load unevenly, amplifying device heterogeneity, whereas distributing computation more evenly increases cross\-agent edges and consequently communication overhead\. This trade\-off becomes harder to manage as the number of agents grows and additional system constraints are introduced\. The partitioning dilemma results in system\-level bottlenecks\. Stragglers caused by uneven computation or communication delays slow both parameter updates and node representation propagation \(line[24](https://arxiv.org/html/2609.02984#alg3.l24)\)\. Late embeddings prevent message passing from completing, directly degrading inference speed and quality\.

These challenges affect the management of dynamic graphs\. When new nodes join, the inference topology changes, compounding existing system heterogeneity with the shifting local neighborhood distributions discussed in Section[3\.3\.2](https://arxiv.org/html/2609.02984#S3.SS3.SSS2)\.

##### Task heterogeneity\.

Task heterogeneity requires the learning and inference processes to handle multiple distinct tasks within the same GNN framework\([Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221);[Meng et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib211);[Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188)\)\. This heterogeneity can manifest across prediction levels \(node, edge, or graph\-level tasks\) or across different application domains \(e\.g\., citation networks and social networks\)\([Wang et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib285)\)\. The modular structure of graph representation learning, and MPNNs in particular, naturally supports this heterogeneity through separate encoding, message passing, and decoding components, as described in Section[3\.1](https://arxiv.org/html/2609.02984#S3.SS1)\. This modularity allows agents to share common encoders while maintaining task\-specific prediction heads\.

#### 3\.3\.4Privacy Vulnerabilities in Collaborative Learning over Graphs

Collaborative learning over graphs inherits the privacy vulnerabilities identified in Section[2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4)for model aggregation\. The fully trained parameters, local updates, and the evolution of aggregated models remain vulnerable to model and gradient inversion\. Additionally, it inherits the privacy vulnerabilities ofcollaborative inferenceseen for Euclidean data in Section[2\.3\.3](https://arxiv.org/html/2609.02984#S2.SS3.SSS3)\. Collaborative inference also presents additional privacy challenges, considering the information held by the topology itself, as discussed below\.

##### Privacy\-preserving collaborative inference\.

Unlike standard collaborative learning where inference is local \(Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\), collaborative inference over graphs requires agents to exchange node embeddings across subgraph boundaries during message passing \(lines[18](https://arxiv.org/html/2609.02984#alg3.l18)–[19](https://arxiv.org/html/2609.02984#alg3.l19)of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\)\. These embedding exchanges create an attack surface as observing boundary node representations enables inference about the local subgraph structure and feature distributions held by neighboring agents\. If intermediary embeddingshv\(l\)h\_\{v\}^\{\(l\)\}and featuresxvx\_\{v\}during distributed message passing as in \([18](https://arxiv.org/html/2609.02984#S3.E18)\), \([19](https://arxiv.org/html/2609.02984#S3.E19)\), \([20](https://arxiv.org/html/2609.02984#S3.E20)\)\) are not communicated, the final embeddingszv=hv\(L\)z\_\{v\}=h\_\{v\}^\{\(L\)\}shared across agents can still leak sensitive information\([Duddu et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib443)\)\. Adversaries can performinference attacksthat can extract private node featuresxvx\_\{v\}for nodes held by other agents\([Duddu et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib443)\), reconstruct edge existence\(u,v\)∈ℰj\(u,v\)\\in\\mathcal\{E\}\_\{j\}in the subgraphs of other agents𝒢j\\mathcal\{G\}\_\{j\}\([Wu et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib196);[He et al\., 2021d](https://arxiv.org/html/2609.02984#bib.bib446)\), or determine whether specific nodes\([Olatunji et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib444);[Zhang et al\., 2021e](https://arxiv.org/html/2609.02984#bib.bib445)\)or edges\([He et al\., 2021d](https://arxiv.org/html/2609.02984#bib.bib446)\)participated in the training\.

The vulnerabilities discussed above assumehonest\-but\-curiousagents that passively infer information from shared embeddings\. Beyond passive observation, malicious agents couldactivelymanipulate their local subgraph structure or features to attack inference outcomes\([Zügner et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib194);[Sun et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib450)\)or poison the training process\([Dai et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib451);[Sun et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib450)\)\. These malicious threat scenarios fall outside the scope of this study\.

### 3\.4Improving Inference and Learning Effectiveness

As per Euclidean data, methods for improving the effectiveness of learning have been proposed for graph data\. Table[2](https://arxiv.org/html/2609.02984#S3.T2)shows that the proposed solutions spread over data\-based and model\-based techniques, as well as methods for vertically aligned data\. Several solutions address learning for multiple graph instances, which is closely related to traditional FL\.

Methods to increase learning effectiveness aim at two objectives: designing a tighter convergence to a global solutionψ∗,θ∗\\psi^\{\*\},\\theta^\{\*\}of the collaborative training objective \([29](https://arxiv.org/html/2609.02984#S3.E29)\), or tighter individual convergence to the optimal solution of the agent,ψ∗,θ∗\\psi^\{\*\},\\theta^\{\*\}, according to the local objectives \([25](https://arxiv.org/html/2609.02984#S3.E25)\) \([28](https://arxiv.org/html/2609.02984#S3.E28)\)\. Effectiveness is challenged by the generalization gap \(seen in Section[3\.1](https://arxiv.org/html/2609.02984#S3.SS1)\), data isolation arising from the various graph data partitions \(seen in Section[3\.3\.1](https://arxiv.org/html/2609.02984#S3.SS3.SSS1)\), or by statistical heterogeneity \(seen in Section[3\.3\.2](https://arxiv.org/html/2609.02984#S3.SS3.SSS2)\)\. Subsequently, we classify and describe the main strategies that address effectiveness, following the classification seen for Euclidean data in Section[2\.3](https://arxiv.org/html/2609.02984#S2.SS3), that is,data\-based,model\-basedtechniques, andtraining and inference under aligned partitions\.

#### 3\.4\.1Data\-based Techniques

Data\-based techniques manipulate the training data to improve convergence toward the optimal parametersψ∗,θ∗\\psi^\{\*\},\\theta^\{\*\}of a jointly trained GNN\. For Euclidean data, these techniques typically operate on independent samples through data augmentation, controlled sampling, or local regularization to mitigate statistical heterogeneity \(Section[2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1)\)\. However, graph data is more complex than sets of independent samples\. First, topology couples nodes, features, and labels, creating high structural diversity: nodes vary by centrality and connectivity, subgraphs differ in density and spectral properties, exhibiting high statistical heterogeneity \(Section[3\.3\.2](https://arxiv.org/html/2609.02984#S3.SS3.SSS2)\), making stable learning challenging\. Second, in subgraph partitions, receptive fields extend beyond local boundaries, creating severe data isolation as neighborhoods are cut across agent boundaries \(Section[3\.3\.1](https://arxiv.org/html/2609.02984#S3.SS3.SSS1)\)\. Furthermore, depending on the data partition setting and the learning tasks, graph\-specific data\-based techniques manipulate entities of different granularity, such as node features and labels, topology, or entire graph instances\.

##### Data augmentation\.

The datasetsSkS\_\{k\}held locally by agents can be augmented to address three challenges: addressing the generalization gap, statistical heterogeneity, and data isolation\. We distinguish methods that augment a graph instance, or augment the dataset with new graph instances\.

Methods thataugment a graph instancemay manipulate the graph topology:

- •LL\-hop recoveringtechniques are applied both for edge and vertex cut, and exchange raw node and edge information between agents, so that they complete the ego\-graphs of the nodes\. This simplifies the collaborative inference phase of Algorithm[3](https://arxiv.org/html/2609.02984#alg3); however requires the use of cryptographic primitives to preserve privacy\([Qiu et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib182);[Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217)\)\. Consequently, theLL\-hop expansion can prove to be expensive and can therefore be restricted to the11\-hop neighborhood\([Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217)\)\.
- •Node rewiringmodifies the existing edge setℰk\\mathcal\{E\}\_\{k\}to yield a graph \(or subgraph\) with a connectivity that improves generalization\. This is achieved by a collaboratively trained transformer encoder\-decoder to modify the spectrum, and thus the connectivity, of each local graph instance\([Tan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib423)\), introducing new frequencies that counter covariate structure shift\.
- •Node additiontechniques aim to extend the local graph by adding new virtual nodes and their associated edges using local graph data only\. This can be done using generative models trained on the server\-side and then communicated to agents to extend local graphs\([Peng et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib179)\)\. Rewiring strategies using pairwise feature similarity have also been explored, allowing agents to infer plausible edges and labels\. These are then integrated into a global graph on the server and redistributed to other agents\([Chen et al\., 2024a](https://arxiv.org/html/2609.02984#bib.bib177)\)\.

Other ways of manipulating the graph instances are the augmentation of features and labels\. These techniques are also specific to graph data, since their objective is to decrease statistical heterogeneity among neighbors or in ego\-graphs:

- •Feature augmentationtechniques for graph data manipulate the feature matrixXX\. Averaging features of neighbors and synthetic feature generation address homophily bias and feature imbalance\([Pei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib268);[Lin et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib251)\)\. Classical methods developed for Euclidean data are used to inject controlled noise\([Gao et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib198)\)\.
- •Label augmentationhas been proposed for various learning tasks\. For link prediction, one method uses a local link prediction model to identify plausible links, which are treated as virtual edges\. These virtual edges can then be shared across agents to enhance their subgraphs in vertex\-cut partitions\([Liu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib186)\)\. For node classification, label smoothing has been applied to mitigate the effects of homophily bias\. In this setting, a single round of distributed averaging \(as introduced in Section[2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1)\) is used to propagate label information across agents\([Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221)\)\. An alternative approach uses contrastive learning, where negative node embeddings, chosen to be close to true node embeddings, are used to sharpen representation learning\. These embeddings can be sampled from other agents, such as randomly selected agents in ego\-graph datasets when the number of agents is large enough\([Giaretta and Girdzijauskas, 2023](https://arxiv.org/html/2609.02984#bib.bib19)\)\.

Other methodsaugment the dataset with new graph instances\. A graph generative model, trained on the server side, can be used to synthesize new graph instances on the agent side based on local graph data\([Guo et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib187)\)\. The existing graph instances and their node labels can be combined\([Wang et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib289)\)to generate new plausible graph and label instances\. These techniques can be applied to full graph instances or to the collection of ego graphs and node label\(Gkv,yv\)\(G^\{v\}\_\{k\},y\_\{v\}\)in the subgraph case\([Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188)\)\. Instead of generating new plausible graphs, one can focus on generating non\-plausible graphs\. This contrastive data augmentation is a frequent approach to provide good generalization capabilities to GNN\([Veličković et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib412);[You et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib281)\)\. Perturbations include node dropping, edge perturbation, attribute masking, random reindexing of the features, or subgraph sampling\([Veličković et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib412);[You et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib281)\)\. In collaborative settings, these techniques can be applied to the local graph data for better generalization as in\([Chen et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib173);[Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175);[Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221)\)\.

##### Local regularization\.

Local regularization techniques reduce discrepancies between agents in their model parameters or vertex\-cut embeddings\. For graph data, these methods exploit the topology by using graph perturbations for contrastive learning, regularizing based on structural properties \(centrality, connectivity\), or aligning shared vertex representations across partitions\.

- •Regularizing local modelsis ubiquitous for FL as described in Section[2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1)\. For graph data, it can be achieved by modifying the local loss \(line[26](https://arxiv.org/html/2609.02984#alg3.l26)\) of the agents with an additional regularization term\. This additional term can make use of the augmented data techniques described above, when integrated in a contrastive term against perturbed graphs\([Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221)\), or a loss on trimmed subgraphs\([Ceyani et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib269)\)\.
- •Prototype\-based regularizationis challenging since nodes can exhibit high structural variety yet must be mapped to a limited set of prototypes\. Solutions, therefore, leverage graph\-specific metrics to construct structurally relevant prototypes\. Prototypes can be stratified according to graph properties such as node centrality\([Tan et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib422);[Kim et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib421)\)or label influence\([Tan et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib422)\)within subgraphs, thereby mitigating noise from low\-connectivity nodes while amplifying information from rare labels\. The locally constructed prototypes are then aligned with global prototypes through contrastive losses\([Kim et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib421);[Tan et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib422)\)\. For full\-graph instances, prototype variance can be reduced via accumulation across training iterations\([Tan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib423)\)\.
- •Regularizing the vertex cut representationsis specific to the subgraph case with vertex cut, and so far discussed for KG tasks only\([Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175)\)\. Regularization terms can be added to the local loss \(line[26](https://arxiv.org/html/2609.02984#alg3.l26)\) to decrease the variance between vertex cut representations, respectively on the agent\-side \(line[22](https://arxiv.org/html/2609.02984#alg3.l22)\) and server\-side \(line[34](https://arxiv.org/html/2609.02984#alg3.l34)\), or to decrease the similarity of the consecutive updates\.

##### Subsampling strategies\.

In contrast to data augmentation, which uses more data than available locally, subsampling techniques use less\. This subsampling operates at multiple levels: agents are sampled in training rounds \(line[4](https://arxiv.org/html/2609.02984#alg1.l4)of Algorithm[1](https://arxiv.org/html/2609.02984#alg1)and line[6](https://arxiv.org/html/2609.02984#alg3.l6)of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\) while nodes are sampled during local inference \(lines[12](https://arxiv.org/html/2609.02984#alg3.l12)and[20](https://arxiv.org/html/2609.02984#alg3.l20)of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\)\. For graph data, subsampling serves dual purposes: it can improve learning effectiveness through regularization or improve efficiency by reducing message\-passing costs\. The techniques include:

- •Agent selectionfor graph data is inspired by agent selection techniques in traditional FL, seen in Section[2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1)\. It can be used to mitigate thehomophily conflictthrough server\-side weighted aggregation of the models, where the weight assigned to each agent depends on the homophily level of its local dataset\. Learning effectiveness is increased if agents with high\-homophily graphs receive higher aggregation weights\([Tan et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib282)\)\. As a form of agent selection for vertex\-cut subgraphs,\([Peng et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib176)\)suggests averagingzv​\(t\)z\_\{v\}\(t\)only if it leads to lower local loss\.
- •Graph subsamplinghas the potential to improve the model accuracy\. For effective sampling in large graphs,\([Ceyani et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib269)\)suggests the use of advanced generative models\([Bengio et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib287)\)\. Training on graphs that are perturbed using subsampling techniques like random node dropping or subgraph sampling is a classical technique that allows GNN to better generalize\([You et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib281)\)\. For collaborative learning, the same technique is applied to the local datasets in\([Chen et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib173);[Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175)\)\. Node and model parameter sampling is proposed in\([Shi et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib305)\)to decrease the computational and communication burden under training\. In the subgraph collaborative inference case, node subsampling can be applied to the local expanded graphG~k\\tilde\{G\}\_\{k\}\. For instance,\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207)\)proposes to select foreign vertex boundary nodes only if their contribution improves the local loss \([26](https://arxiv.org/html/2609.02984#S3.E26)\)\. Subsampling strategies are also designed to balance efficiency and effectiveness, using greedy heuristics\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\), reinforcement learning\([Chen et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib185)\), or Markov chain methods\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)\.

##### Open challenges of data\-based techniques\.

In collaborative settings, graph\-specific data properties create fundamental challenges for learning and inference effectiveness that extend beyond the Euclidean case\. Some challenges stem from open problems existing also for centralized graph machine learning, while others emerge specifically from the interaction between graph structure and collaborative dynamics during both training and inference\. Below we identify key research directions where graph\-specific data properties create unique obstacles for collaborative settings:

- •Towards collaborative graph rewiring\.In centralized settings, recent research has explored decoupling the observed input graph from the graph used for message passing\([Wang et al\., 2019b](https://arxiv.org/html/2609.02984#bib.bib1);[Fatemi et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib302);[Kazi et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib2);[Topping et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib381);[Gutteridge et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib291)\)\. Thesegraph rewiringapproaches augment the graph structure to use neighbor information more effectively, improving model accuracy\. Extending such rewiring to collaborative settings presents unique challenges: when a global graph is partitioned across agents, rewiring decisions have to take into account conflicting constraints\. For instance, adding edges between nodes held by different agents increases inter\-agent communication and thus adds communication costs and privacy vulnerabilities\.
- •Towards capturing statistical heterogeneities\.Collaborative GNN learning is challenged by structural diversity inherent to graph domains\. Unlike images or text, graphs span vastly different domains, from social networks to molecular structures, that create challenging statistical heterogeneity across and within agents datasets\. At the same time, graph properties are typically characterized by classical network analytics metrics \(e\.g\., centrality, degree\), which do not integrate the computational scheme of MPNN \(Figure[12](https://arxiv.org/html/2609.02984#S3.F12)\), nor the feature information\. Promising directions for more expressive characterization include adapting GNN\-specific metrics from centralized training theory to collaborative settings\. For example, moving tree distances\([Chuang and Jegelka, 2022](https://arxiv.org/html/2609.02984#bib.bib279)\)have been tied to GNN generalization capabilities both empirically and theoretically\([Maskey et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib278);[Southern et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib280)\)\. This improved characterization could enable more informed data\-sharing strategies, such as identifying which agents should collaborate based on structural similarity rather than simple topological statistics\.
- •Towards graph\-specific model optimization\.While global model optimization has been extensively studied for collaborative learning on Euclidean data \(Section[2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1)\), ensuring convergence and generalization for GNNs remains poorly understood even in centralized settings\([Jegelka, 2022](https://arxiv.org/html/2609.02984#bib.bib288);[Morris et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib315)\)\. Collaborative scenarios exacerbate this challenge: agents with topologically distinct subgraphs \(scale\-free vs\. regular networks, dense vs\. sparse regions\) may experience fundamentally different optimization landscapes\. For instance, MPNNs exhibit representation collapse along high commute\-time paths\([Di Giovanni et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib437)\), causing sensitivity to vary with network structure, a problem amplified when agents cannot share data to compensate for these structural differences\. Developing convergence guarantees and regularization strategies that account for topology\-dependent optimization dynamics in collaborative settings is a promising research direction\.
- •Towards collaborative learning on evolving graphs\.Most collaborative GNN work assumes static graphs, where topology and features remain fixed during training\. However, real\-world graphs evolve temporally: nodes and edges appear or disappear \(e\.g\., users joining social networks, devices entering IoT systems\), causing topology𝒟⁡\(A\)\\mathcal\{D\}\(A\)and features𝒟⁡\(X\)\\mathcal\{D\}\(X\)to shift concurrently \(Section[3\.3\.2](https://arxiv.org/html/2609.02984#S3.SS3.SSS2)\)\. This coupled evolution creates unique challenges for collaborative settings: when a node’s neighborhood changes, its ego graph distribution𝒟⁡\(Gv\)\\mathcal\{D\}\(G^\{v\}\)shifts, but agents holding overlapping ego graphs may observe these changes asynchronously, creating temporal misalignment during collaborative training\. Therefore, it is an essential open challenge to establish characterization methods for temporal evolution and benchmarks for dynamic collaborative graph learning\([Kim et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib421)\)\.

Data\-based techniques for graph\-structured dataadapt the techniques seen for Euclidean data \(local regularization, data augmentation, and subsampling\) to address statistical heterogeneity and data isolation in partitioned graphs\. However, in contrast to Euclidean data, many of the methods manipulate the graph itself, extending, regulizing or subsampling nodes, links, subgraphs, vertex\-cuts or the agents themselves\. Open challenges include the statistical characterization of static or dynamic subgraphs, as well the extension of emerging centralized models to collaborative settings\.Trilemma\.Efficiency trade\-offs emerge as subsampling reduces message\-passing costs but may discard informative boundary nodes, while topology augmentation increases communication overhead through cross\-agent coordination\. Privacy concerns arise as methods requiring structural coordination \(L\-hop recovery, label smoothing, prototype sharing\) expose topology and embeddings\.

#### 3\.4\.2Model\-based Techniques

Model\-based solutions withparameter personalizationandarchitecture personalizationare proposed to find agent\-optimal GNN models despite data and system heterogeneity\. Most model\-based solutions seen for Euclidean data in Section[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2)have been adapted for graph\-structured data as seen in Table[2](https://arxiv.org/html/2609.02984#S3.T2)\. However, graph\-structured data and GNNs introduce distinct considerations for model personalization\. Techniques for personalizing model parameters can address statistical heterogeneity on different levels of granularity \(nodes, edges, subgraphs, entire graphs\)\. Model personalization can utilize the layered message\-passing architecture of GNNs to decouple model components\.

##### Parameter personalization\.

Parameter personalization techniques seen in Section[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2)have been adapted for graph data, using graph\-specific quantities to personalize model parameters:

- •Local adaptationstrategies can be extended to GNNs\. In\([Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188)\), the local model is formulated as a learnable interpolation between the global and the local GNN parameters, while\([Baek et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib202);[Fang et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib215);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195)\)suggest masking the weights of the global models using learnable parameters\. A meta\-learning framework is suggested for node\-level tasks on graph data in\([Wang et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib242);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195)\), where the support and query sets are defined as subgraph partitions\.
- •Hypernetwork\-basedtechniques\([Shamsian et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib308)\)are extended for graph data in\([Liang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib306)\), that defines the representation vector of agentkkas the mean of node representations within the local subgraph:1Nk​∑v∈𝒱kzv​\(t\)\.\\frac\{1\}\{N\_\{k\}\}\\sum\_\{v\\in\\mathcal\{V\}\_\{k\}\}z\_\{v\}\(t\)\.
- •Cluster\-basedtechniques can be used to cluster agents based on gradient similarities\([Xie et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib189)\)\. If the agents observe a single ego\-graph only, then they can be clustered based on their node embeddings\. As suggested in\([Qu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib206)\), an effective way to support learning within clusters is to construct virtual local graphs\.
- •Similarities\-basedtechniques are also adapted specifically to graph data\. However, defining similarities between graph distributions is challenging\. The similarity matrixΩ=\[ωk​l\]k,l∈𝒦2\\Omega=\[\\omega\_\{kl\}\]\_\{k,l\\in\\mathcal\{K\}^\{2\}\}can be interpreted as defining a latentinter\-agent graph, distinct from the local graph data of each agent\. Similarity scores, inspired by\([Zec et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib150)\), are evaluated for different GNN models on randomly generated subgraphs in\([Baek et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib202)\)\.\([Liang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib306)\)use subgraph representationsckc\_\{k\}to compute similarities between agents, which are then diffused over the inter\-agent graph using learnablesheaf diffusion\([Bodnar et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib310)\)algorithms trained on the server\.

##### Architecture personalization\.

Architecture personalization modifies the model architecture used by each agent\. While standard techniques from Section[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2)remain applicable, GNNs compositional message\-passing architecture is particularly well\-suited for flexible architecture personalization:

- •Model decoupling\.As described in Section[3\.1](https://arxiv.org/html/2609.02984#S3.SS1), a GNNFψ,θF\_\{\\psi,\\theta\}can be decomposed into an encoderϕθ\\phi\_\{\\theta\}decomposed into sublayersfE​n​c,fM​e​s\(l\)f\_\{Enc\},f\_\{Mes\}^\{\(l\)\},fA​g​g\(l\)f\_\{Agg\}^\{\(l\)\},fU​p\(l\)f\_\{Up\}^\{\(l\)\}for l=1,…,L, and a predictorfψf\_\{\\psi\}\. This layered architecture allows for splitting the modellayer\-wiseinto different components\. The split means that one component is shared while another is personalized to the agents\([Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221)\)\. More precisely, the shared component can either be trained on the agent side and is then aggregated periodically on the server, or it is trained on the server\([Meng et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib211);[Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188)\)using alternative optimization techniques\([Meng et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib211)\)\. Decoupling can occur between the encoderϕθ\\phi\_\{\\theta\}and predictorfψf\_\{\\psi\}\([Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221);[Tan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib423)\), or between the feature encoderfE​n​cf\_\{Enc\}and subsequent layers, where either the early layers are personalized, and later ones shared\([Meng et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib211)\), or vice versa\([Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188)\)\. Local GNNs have also been prunedwidth\-wisein\([Shi et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib305)\), where each agent removes local learnable weights based on a layer\-specific, learnable threshold\. To mitigate severe performance degradation during aggressive pruning, a gradual rollback strategy is employed\. Aggregating width\-wise pruned models remains challenging and has been proposed to be performed at lower bit resolution to promote generalization across agents\.
- •Multimodal personalization\.The server can maintainMMspecialized GNNs parameters\{θmc\}m=1M\\\{\\theta^\{c\}\_\{m\}\\\}^\{M\}\_\{m=1\}for each graph or subgraph type\. The framework for multimodal personalization\([Chen and Zhang, 2022](https://arxiv.org/html/2609.02984#bib.bib154);[Smith et al\., 2017b](https://arxiv.org/html/2609.02984#bib.bib348)\)seen in Section[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2)has been applied to full graph instances associated when one agent that can collect graphs of different modalities or tasks\([He et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib250)\)\. In vertically partitioned subgraph settings, predictions from agents local models for the same nodes can be combined using a confidence\-based aggregation scheme known asknowledge voting\([Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221)\)\.
- •Knowledge distillation\.Instead of sharing knowledge through model parameters, agents can exchange node representations while remaining free to choose their local architectures\. This approach leverages a key property of GNNs: they maintain a shared representation space𝐑dz\\mathbf\{R\}^\{d\_\{z\}\}across layers, enabling meaningful comparison of node embeddings between agents\. To achieve knowledge sharing, agents add an alignment term to their local loss function\. This term measures the discrepancy between local and global representations, either for vertex\-cut nodes\([Fu et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib197)\)or for node prototypes\([Tan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib423)\)\. In vertical topology\-free partitions, this embedding alignment enables a useful form of knowledge transfer: topology\-aware agents can distill their structural knowledge to topology\-agnostic agents\([Fu et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib197)\)\. The topology\-agnostic feature encoderfE​n​cf\_\{Enc\}thereby learns to integrate topological information indirectly, avoiding communication and synchronization overhead during inference\.
- •Decoupling the message passing operations\.An architecture personalization approach specific to GNNs is the personalization of the message passing operation itself\. In\([Lei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib209)\), the message computations in \([18](https://arxiv.org/html/2609.02984#S3.E18)\) are adapted based on edge type\. Messages along edges cutℰcut\\mathcal\{E\}\_\{\\text\{cut\}\}carry only partial information, for instance, when connected to stale embeddings\. To compensate for this limited information, these cross\-boundary messages are processed using more expressive message and aggregation functions,fMesf\_\{\\text\{Mes\}\}andfAggf\_\{\\text\{Agg\}\}, designed to better capture missing dependencies\. In contrast, messages over local edges \(i\.e\., those withinGkG\_\{k\}\) rely on simpler functions\([Wu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib210)\), as they benefit from more complete and up\-to\-date local information\.

##### Open challenges of model\-based techniques\.

While collaborative GNN learning has adapted many techniques from Euclidean federated learning, recent advances in graph machine learning, from expressive architectures beyond standard MPNNs to foundation models for graphs, present new opportunities and challenges for collaborative settings\. Key research directions include:

- •Towards more advanced topological structures\.Work in collaborative settings has primarily studied standard GNN architectures \(e\.g, GCN\([Kipf and Welling, 2017](https://arxiv.org/html/2609.02984#bib.bib390)\), GAT\([Veličković et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib391)\)\)\. However, more expressive extensions of the MPNN framework have been introduced under the umbrella oftopological deep learning\([Hajij et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib413);[Papamarkou et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib4)\)orgeometric deep learning\([Bronstein et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib406)\)\. These architectures introduce new collaborative challenges, since partitioning higher\-order structures across agents creates more complex boundary interactions than standard vertex cuts\. Therefore, understanding how these structures integrate into the current collaborative learning solutions and physical network architectures is an exciting open research direction\.
- •Towards effective layer pruning\. Apart from\([Wu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib210);[Lei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib209)\), most work assumes all GNN layers are equally important for collaborative learning\. However, studies have revealed this is generally not true in deep architectures, with GNNs potentially benefiting from aggressive layer pruning strategies\([Kummer et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib284)\)\. While this concept has been exploited to alleviate communication costs in CNN\-based collaborative settings\([Park and Joe\-Wong, 2024](https://arxiv.org/html/2609.02984#bib.bib47)\), its application to GNNs remains unexplored\. In partitioned graphs, layer importance may depend explicitly on graph topology and partitioning characteristics\. Investigating which layers are essential for collaborative learning, building on\([Lei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib209)\)’s finding that boundary\-processing layers require special treatment, could enable more efficient communication strategies\.
- •Towards the utilization of foundation models\.Most works focus on training task\-specific models from scratch, or maintain separate models for different domains\. However, this use of personalized per\-domain or per\-task architectures might be challenged by the paradigm shift offoundation models \(FMs\)currently revolutionizing ML\. These generalist pre\-trained models, often with billions of parameters, can be effectively adapted to downstream tasks with a relatively small amount of new data, but the development of FMs on graphs remains an open problem\([Wang et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib285)\)\. Their integration into collaborative settings, where billion\-parameter models face computational, storage, communication, and privacy constraints, is a research avenue\.

Model\-based techniques for graph\-structured dataadapt parameter personalization and architecture personalization strategies from Euclidean collaborative learning to address data and system heterogeneity\. Parameter personalization, however, needs to consider statistical properties observed by an agent, which requires new solutions\. GNNs layered message\-passing architecture enables fine\-grained model decoupling across layers, edge types, and message\-passing operations, providing richer personalization opportunities than fully\-connected networks\. Challenging research questions include the adaptation of advanced topological structures, optimized layer pruning, and the general question of the adaptability of foundation models\.Trilemma\.Architecture decoupling improves effectiveness by allowing agents with heterogeneous model architectures to collaborate, but increases communication overhead through coordination of multiple model components\. Knowledge distillation and embedding alignment methods expose intermediate node representations and topology information, necessitating privacy\-protection mechanisms\.

#### 3\.4\.3Training and Inference under Aligned Partitioning

The vertical and vertex\-cut partitioning cases introduced in Section[3\.3\.1](https://arxiv.org/html/2609.02984#S3.SS3.SSS1)create a distinct challenge, since multiple agents hold complementary views of the same graph entities \(nodes or entire graphs\)\. Unlike horizontal partitions, where each agent observes complete but disjoint samples, these partitions split information about the same entities across agents, similarly to vertical federated learning for Euclidean data \(Section[2\.3\.3](https://arxiv.org/html/2609.02984#S2.SS3.SSS3)\)\. The presence of aligned entities across agents induces the need for aggregation mechanisms that go beyond model parameter aggregation \(Algorithm[1](https://arxiv.org/html/2609.02984#alg1)\) or cross\-boundary message passing \(Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\), borrowing instead from aggregating cross\-agent views \(Algorithm[2](https://arxiv.org/html/2609.02984#alg2)\)\.

Specifically, when multiple agents generate representations for the same node \(vertex\-cut partitions of homogeneous graphs in Figure[16](https://arxiv.org/html/2609.02984#S3.F16)and heterogeneous graphs in Figure[17](https://arxiv.org/html/2609.02984#S3.F17)\) or for the same graph instance \(vertical partitions of multiple graph instances in Figure[15](https://arxiv.org/html/2609.02984#S3.F15)\), these representations must be combined to support effective prediction\. This section, therefore, organizes existing methods into three families: techniques for aggregating node representations across agents, transfer learning strategies for cross\-agent adaptation without inference\-time alignment, and methods for combining graph\-level representations\.

##### Aggregating node representations\.

In the case of vertex\-cut partitions, node representations for the same node are generated at different agents, and therefore need to be combined\. Indeed, the server receives \(line[24](https://arxiv.org/html/2609.02984#alg3.l24)\) a collection of node representation\{zv,k​\(t−12\)\}v∈𝒱k\\\{z\_\{v,k\}\(t\-\\frac\{1\}\{2\}\)\\\}\_\{v\\in\\mathcal\{V\}\_\{k\}\}from each agentk∈𝒦k\\in\\mathcal\{K\}\. So for a node cutv∈𝒱c​u​tv\\in\\mathcal\{V\}\_\{cut\}, the different representations\{zv,k​\(t−12\)\}k∈𝒦\\\{z\_\{v,k\}\(t\-\\frac\{1\}\{2\}\)\\\}\_\{k\\in\\mathcal\{K\}\}have to be aggregated \(line[34](https://arxiv.org/html/2609.02984#alg3.l34)\)\. Different choices exist for this aggregation step, which aim to increase the quality of the final node representation\{zv​\(T\)\}v∈𝒱\\\{z\_\{v\}\(T\)\\\}\_\{v\\in\\mathcal\{V\}\}:

- •Averaging node representations\.The standard approach is to computez¯v​\(t\)\\bar\{z\}\_\{v\}\(t\), the average on𝒦\\mathcal\{K\}of the representations\{zv,k​\(t−12\),k∈𝒦\}\\\{z\_\{v,k\}\(t\-\\frac\{1\}\{2\}\),k\\in\\mathcal\{K\}\\\}from the different agents to obtain more generalizable representations\([Chen et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib173);[Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181);[Zhang et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib174);[Qu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib206);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195);[Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217);[Ni et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib171)\)\. The averaging can be restricted to agents with low local loss\([Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221)\)\. Alternatively, the embeddingszv​\(t\)z\_\{v\}\(t\)can be updated by taking a small step towardz¯v​\(t\)\\bar\{z\}\_\{v\}\(t\)using a gradient\-based formulation, which provides more stable updates throughout the training\([Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Zheng et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib261)\)\.
- •Personalizing node representations\.Personalization techniques return a different node representationzv,k​\(T\)z\_\{v,k\}\(T\)for each agent at the end of the training instead of a common onezv​\(T\)z\_\{v\}\(T\)\. The personalization techniques used for the model parametersθk​\(t−12\)\\theta\_\{k\}\(t\-\\frac\{1\}\{2\}\)in Section[2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2)can be applied to the node representationszv,k​\(t−12\)z\_\{v,k\}\(t\-\\frac\{1\}\{2\}\)to address statistical shifts for graph data and task heterogeneity\.zv,k​\(t\)z\_\{v,k\}\(t\)is constructed by combiningzv,k​\(t−12\)z\_\{v,k\}\(t\-\\frac\{1\}\{2\}\)andz¯v​\(t\)\\bar\{z\}\_\{v\}\(t\)using MLP\([Chen et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib173)\)or a learnable linear interpolation\([Wang et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib191)\)\.

##### Transfer learning\.

Rather than aggregating the node representations\{zv,k​\(t−12\)\}k∈𝒦\\\{z\_\{v,k\}\(t\-\\frac\{1\}\{2\}\)\\\}\_\{k\\in\\mathcal\{K\}\}, their difference between agents can be used to designtransfer learning\. Typically, it is assumed that a model has been trained by "source" agentssson a sufficient amount of vertically partitioned subgraph data, and this knowledge is transferred to a target agenttt\. Aligning these representations whiletttrains its local model allows to adapt the frozen model ofsstott’s domain\. This alignment can be achieved by training against an adversarial model\([Shen et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib253)\), trained to distinguish between source and target domains\([Guan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib222)\), or using a knowledge distillation loss\([Hinton et al\., 2014](https://arxiv.org/html/2609.02984#bib.bib396);[Mai et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib221)\)\.

##### Aggregating graph representations\.

Lastly, in the case of vertical partition of full graph instances \(see Figure[15](https://arxiv.org/html/2609.02984#S3.F15)\), multiple graphs held by different agents can characterize the same item or person\. To increase the generalization capabilities of graph\-level tasks,\([Dong et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib370)\)suggests averaging the graph embeddings to increase prediction accuracy, considering the specific use case of health applications\.

##### Open challenges\.

Most work on vertical graph partitions focuses on basic aggregation of aligned representations\. However, practical implementations come with open challenges, as they often include multi\-modal features on one side, and have significant communication or privacy constraints on the other side, leading:

- •Towards transfer learning for multi\-modal aligned partitions\.Current work on vertical partitions primarily uses simple averaging of aligned node representations\. However, cross\-organizational collaboration typically involves multi\-modal heterogeneity: entities align, but features and topology differ \(e\.g\., banks and social platforms sharing customer data\)\. While transfer learning\([Frasca et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib190);[Wang et al\., 2025b](https://arxiv.org/html/2609.02984#bib.bib285)\)and knowledge distillation\([Tian et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib320)\)are well\-studied for centralized GNNs, their adaptation to collaborative multi\-modal settings remains underexplored\. A key challenge is negative transfer: structural diversity across agents means transfer may not always be beneficial, requiring methods to identify when knowledge transfer is appropriate across heterogeneous collaborators\.
- •Towards efficient and private aggregation for aligned entities\.Current work merges all aligned node representations uniformly, regardless of importance\. However, vertex\-cut partitions can involve hundreds of aligned nodes, creating substantial communication overhead and privacy exposure\. For graphs, structural heterogeneity means nodes vary significantly in their contribution to learning \(centrality, boundary position\)\. Additionally, aggregating node representations leaks inference data beyond model or gradient inversion attacks\. Developing selective aggregation strategies that prioritize informative nodes while minimizing leakage remains an open challenge\.

Training and inference under aligned partitionsadapts aggregation and transfer learning strategies from vertical federated learning \(Section[2\.3\.3](https://arxiv.org/html/2609.02984#S2.SS3.SSS3)\) to merge complementary views of the same graph entities held across agents\. Unlike horizontal partitions, where agents observe complete but disjoint samples, vertical and vertex\-cut partitions split information about the same entities across agents\. For effective learning and inference, local representations are aggregated, or personalized collaboratively\. Transfer learning over alignment entities enables collaborative learning and isolated inference\. Challenging open questions include multi\-modal alignment and increasing efficiency and privacy through sampling\. Trilemma\.Merging aligned node representations creates substantial communication overhead, while transfer learning reduces inference\-time communication at the cost of potential negative transfer when structural diversity is high\. Node representation aggregation exposes topology and inference data beyond model or gradient inversion attacks, with aligned entities creating greater attack surfaces that require privacy\-potection\. The process of aligning nodes or subgraphs themselves requires privacy guarantees\.

### 3\.5Improving Inference and Learning Efficiency

While Section[2\.4](https://arxiv.org/html/2609.02984#S2.SS4)reviewed general design principles to improve efficiency in collaborative learning under realistic resource constraints for Euclidean data, we now focus on how these principles translate to graph data specifically\. The most important difference between the Euclidean and graph\-based scenarios is that due to message passing, efficiency now needs to be considered both for training and for inference, and during training, model updates with gradient backpropagation may also use communication resources\. We discussaggregation patterns,asynchronous aggregation, andcommunication efficiency\. Table[2](https://arxiv.org/html/2609.02984#S3.T2)shows that existing work concentrates primarily on aggregation patterns, where solutions depend heavily on the graph partitioning\.

#### 3\.5\.1Aggregation over a network

The centralized, decentralized, and hierarchical aggregation patterns, introduced in Section[2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1), are all applicable for learning with graph data\. They can be employed to aggregate the local modelsθk​\(t\)\\theta\_\{k\}\(t\)duringcollaborative trainingand to aggregate node embeddings in the message passing equation \([19](https://arxiv.org/html/2609.02984#S3.E19)\), duringcollaborative inference\.

Under collaborative learning with Euclidean data, the communication graphWWfor the model aggregation is determined mainly by the physical network topology, that is, by the possible communication links or paths among the agents\. This is true as well for collaborative learning over multiple graph instances\. However, in the subgraph or ego\-graph partition cases, the computational graph𝒢\\mathcal\{G\}\(or its matrix representationAA\) has to be taken into account as well, both for inference and for the local model update\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/computation_vs_communcation_graph.jpg)Figure 19:llustration of the coupling between computation and communication topologies in the subgraph case\.The computational topology, represented by graph𝒢\\mathcal\{G\}, and the communication topology, represented by matrixWW, can be: \(a\)derived fromthe partitioning of𝒢\\mathcal\{G\}between agents, such as an edge\-cut partition \(see Section[3\.3\.1](https://arxiv.org/html/2609.02984#S3.SS3.SSS1)\), or \(b\)independent from𝒢\\mathcal\{G\}, representing an arbitrary communication network\.The coupling of the computation graph𝒢\\mathcal\{G\}and the communication graph given byWWtypically follows one of two paradigms, as illustrated in Figure[19](https://arxiv.org/html/2609.02984#S3.F19)\.

- •Communication graph derived from subgraph partitioning of𝒢\\mathcal\{G\}\.In this case, the communication topology is consistent with the computational one, meaning that certain edges in𝒢\\mathcal\{G\}also represent communication links inWW\. For example, in a swarm of robots\([Blumenkamp et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib264);[Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260)\),𝒢\\mathcal\{G\}connects physically adjacent robots, thereby defining both computation and communication topologies\. In ego\-graph settings, the communication graph is identical to the computation graph\. For edge\-cut subgraph partitions, communication links correspond to edges cut between agents, as in IoT systems where agents monitor different subsets of nodes\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213);[Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260);[Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\)\. The node\-to\-agent assignments in𝒢\\mathcal\{G\}may be given as problem constraints, for example, determined by agent locations\([Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260);[Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\), or optimized given the problem constraints\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213);[Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\)\.
- •Communication and computational graph decoupled\.Here, computation and communication are decoupled\. Typically, when peer\-to\-peer communications among all agents are allowed\([Giaretta and Girdzijauskas, 2023](https://arxiv.org/html/2609.02984#bib.bib19);[Qu et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib206)\)\(WWforming a fully connected graph\), the communications graph can be further refined adaptively to accommodate learning efficiency\([Wang et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib319)\)\.

Typically, agents aggregate their model parameters and embeddings through the same topology\. However, distinct communication topologies can be employed for model and embedding aggregation, allowing the topology to differ between training and inference\([Lei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib209);[Giaretta and Girdzijauskas, 2023](https://arxiv.org/html/2609.02984#bib.bib19);[Guo et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib200);[Wang et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib319)\)\. In hierarchical architectures, centralized or decentralized aggregation patterns may occur for model updates and for inference at different levels of the hierarchy\([Guo et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib200)\)\. Due to this increased flexibility, we discuss solutions for embedding aggregation and for model aggregation separately\.

##### Embedding aggregation\.

In the case of subgraph partitions, embeddingshv\(l\)h\_\{v\}^\{\(l\)\}generated by different agents must be aggregated either to perform message passing \(line[20](https://arxiv.org/html/2609.02984#alg3.l20)of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\) or for vertex\-cut processing \(see Section[3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3)\)\. Embedding aggregation techniques aim to adapt to constrained communication topologies while reducing both the communication cost of transmitting embeddings \(line[19](https://arxiv.org/html/2609.02984#alg3.l19)\) and the computational cost of aggregating them \(line[20](https://arxiv.org/html/2609.02984#alg3.l20)\) during collaborative inference in Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\. Solutions are proposed for centralized and for decentralized aggregation topologies:

- •Centralized communicationsare the default approach inherited from FL algorithms and Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\. The server collects and redistributes embeddings and is sometimes responsible for maintaining them\([Liu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib186);[Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\)\.
- •Decentralized communications,leading to truly decentralized inference, means that embeddings are exchanged directly between the agents\. Decentralized inference is the default for edge\-cut ego\-graph partitions, where the communication pattern naturally aligns with the computation pattern\([Pan and Zhu, 2022](https://arxiv.org/html/2609.02984#bib.bib212);[Solodova et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib380);[Guo et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib200)\)However, decentralized inference also appears in subgraph edge\-cut partitions, for example, when agents represent institutions or edge servers monitoring a subset of nodes\([Lei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib209);[Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213);[Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260);[Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249);[Wang et al\., 2023c](https://arxiv.org/html/2609.02984#bib.bib316)\)\. In such cases, the communication graph can be inferred from the cut of𝒢\\mathcal\{G\}\([Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\)or from the communication coverage area of the servers\([Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260)\), as shown in Figure[19](https://arxiv.org/html/2609.02984#S3.F19)\.

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/embeddings_aggregation.jpg)Figure 20:Embedding aggregation techniques\.Methods focus on modifying: \(a\) the timing of communications, \(b\) the computation or communication topology, or \(c\) the data communicatedThe embedding aggregation techniques for increased efficiency modify the time of the aggregation, the topology, or the content of the exchanged data, as summarized in Figure[20](https://arxiv.org/html/2609.02984#S3.F20)\.

The first line of work focuses on modifying thetimingof embedding aggregation to reduce the communication cost using:

- •Pre\-communication\.Communicating sufficient information before the start of the training can avoid the need for a collaborative inference\. Typically, the expanded subgraph dataG~k\\tilde\{G\}\_\{k\}is communicated to relevant agents or the server prior to the first training round, so that agents perform isolated inference onG~k\\tilde\{G\}\_\{k\}, proposed for centralized aggregation\([Zhang et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib174);[Chen et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib173);[Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175)\)or through a decentralized protocol\([Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217);[Qiu et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib182)\)\. This uphill communication cost can prove worth it if the algorithm takes many training rounds to converge in the collaborative inference, ultimately outweighing it\.
- •Stale embeddings\.Letting node embeddingshv\(l\)​\(t\)h\_\{v\}^\{\(l\)\}\(t\), requested normally in every round in the default Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\(line[19](https://arxiv.org/html/2609.02984#alg3.l19)\), become stale reduces the communication cost of collaborative inference\. Therefore,hv​\(t\)h\_\{v\}\(t\)is proposed to be requested periodically in\([Du and Wu, 2022](https://arxiv.org/html/2609.02984#bib.bib183)\)\. To avoid affecting learning effectiveness due to this staleness,\([Du and Wu, 2022](https://arxiv.org/html/2609.02984#bib.bib183)\)derives an optimal sampling rate w\.r\.t\. the tradeoff between a communication budget and the convergence speed of training\.
- •Truncated message passing\.Alternatively, to reduce the amount of information to be communicated in each round,\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207)\)suggests communicating only a part of the embeddings, specifically only the later layersll, eliminating layers that have less contribution to the learning process\.

The literature focuses on modifying the computation and communication topology to reduce communication overhead\. During inference, the representationzvz\_\{v\}for nodevvis obtained by propagating and aggregating messages along the edges of the input graph, following the procedures in \([18](https://arxiv.org/html/2609.02984#S3.E18)\), \([19](https://arxiv.org/html/2609.02984#S3.E19)\), and \([20](https://arxiv.org/html/2609.02984#S3.E20)\)\. This sequence of computations induces a*computational tree*𝒯v\\mathcal\{T\}^\{v\}rooted invv, directly derived from its ego\-graphGvG^\{v\}\. In the case of subgraph partitioning, edge cuts in the input graph translate into inter\-agent communication links in the computational tree\. Reducing communication during collaborative inference thus often amounts to modifying𝒯v\\mathcal\{T\}\_\{v\}by:

- •Trimming the computational treeTrimming the computational tree𝒯v\\mathcal\{T\}^\{v\}can significantly reduce the communication needed\. Tree\-trimming strategies proposed in the literature\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207);[Chen et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib185);[Pan and Zhu, 2022](https://arxiv.org/html/2609.02984#bib.bib212);[Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)remove selected branches of𝒯v\\mathcal\{T\}\_\{v\}leading to nodevv, thereby reducing both computation and communication, and notably the communication when the removed branches involve inter\-agent communication\. Such strategies can be determined locally by each agent\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207)\), applied to its local expanded topology𝒢~k\\tilde\{\\mathcal\{G\}\}\_\{k\}, assumed to be known; or centrally by a server with knowledge of the global graph𝒢\\mathcal\{G\}\([Chen et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib185);[Pan and Zhu, 2022](https://arxiv.org/html/2609.02984#bib.bib212);[Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)\. The communication workload can be balanced, either at the global level, e\.g\., minimizing the maximum workload across agents\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\), or at the individual level by balancing each agent’s workload\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207);[Chen et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib172)\)\. While trimming𝒯v\\mathcal\{T\}^\{v\}reduces the computational and communication load, it also discards part of the knowledge contained in the subgraphGvG^\{v\}, potentially degrading model performance\. The impact of the computational tree depth on prediction accuracy has been demonstrated in real\-world applications such as robot swarm localization\([Zhou et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib368);[Blumenkamp et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib264)\)\. The trade\-off between efficiency and accuracy can be balanced by various techniques, including selecting nodes that contribute only marginally to loss reduction\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207)\)or using reinforcement learning to guide branch removal\([Chen et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib185)\)\. In homophilous graphs, shortcut jumps within𝒯v\\mathcal\{T\}^\{v\}can be used to reach a target nodevvin fewer communication steps\([Pan and Zhu, 2022](https://arxiv.org/html/2609.02984#bib.bib212)\)\.
- •Graph partitioning\.Another approach is to optimize graph partitioning to minimize communication cost shown by𝒯v\\mathcal\{T\}^\{v\}\. Partition refinement can be achieved by reassigning nodes dynamically to agents to reduce the number of edge cuts\([Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260)\)\. Node assignment can also account for other system\-level constraints such as data acquisition cost or the number of agents\([Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\)\.

Modifying the nature of the data exchanged can significantly reduce communication costs\. This can be achieved by employing simpler architectures than the generic MPNN described in Section[3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3), achieved with techniques such as:

- •Pre\-aggregating before communicating,Embeddings can be pre\-aggregated before being communicated, thus reducing the amount of communication\. When the expanded subgraphG~k\\tilde\{G\}\_\{k\}is transmitted for isolated inference, communicating the fullLL\-hop neighborhood may still involve a prohibitive amount of node data\. A practical solution is to leverage the properties of simple GNNs such as GCN\([Kipf and Welling, 2017](https://arxiv.org/html/2609.02984#bib.bib390)\)or SGC\([Wu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib210)\), which use non\-learnablefMesf\_\{\\text\{Mes\}\}andfAggf\_\{\\text\{Agg\}\}, to pre\-aggregate the required embeddings locally before transmission\. For example, in a GCN, the data of anll\-hop neighborhood of a private boundary nodev∈𝒱p,kv\\in\\mathcal\{V\}\_\{p,k\}can be pre\-aggregated as∑u∈𝒱k\|\(u,v\)∈ℰkxu\\sum\_\{u\\in\\mathcal\{V\}\_\{k\}\\mid\(u,v\)\\in\\mathcal\{E\}\_\{k\}\}x\_\{u\}instead of transmitting the set\{xu∣u∈𝒱k,\(u,v\)∈ℰk\}\.\\\{x\_\{u\}\\mid u\\in\\mathcal\{V\}\_\{k\},\(u,v\)\\in\\mathcal\{E\}\_\{k\}\\\}\.\([Yao et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib419)\)This reduces the cost of communicating anll\-hop neighborhood to that of an\(l−1\)\(l\-1\)\-hop neighborhood for GCN, and to that of the nodevvitself for SGC\. For SGC, the local inference cost can be further reduced by pre\-computing the non\-learnable operations\([Lei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib209)\)\.
- •Squashing the embedding space,The dimensionality of node embeddingsdxd\_\{x\}directly impacts communication cost\. In node\-level prediction tasks, an agent can locally generate pseudo\-labelsy^v\\hat\{y\}\_\{v\}by applying a feature encoderfEncf\_\{\\text\{Enc\}\}followed by a local predictorfψf\_\{\\psi\}\. Instead of transmitting full embeddings, only the scalar pseudo\-labelsy^v\\hat\{y\}\_\{v\}are sent, substantially reducing the communication load\. Although these pseudo\-labels do not initially incorporate graph topology, they can be refined via non\-learnable diffusion algorithms \(see Section[2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1)\), which propagate labels across the graph in a topology\-aware manner\. Under the homophily assumption, this allows even unlabeled nodes not involved in the collaborative inference to receive meaningful predictionsyvy\_\{v\}\([Huang et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib401)\)\. This approach has been implemented in decentralized settings for node\-label prediction, achieving minimal communication cost while maintaining competitive performance\([Krasanakis et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib262)\)\.

##### Model aggregation\.

For model aggregation, centralized, decentralized, and hierarchical architectures are proposed, very similarly to the Euclidean scenarios\. The choice, however, is largely determined by the type of graph partition and by the communication costs caused by the backpropagation of the gradients\.

- •Centralized model aggregation, inherited from Algorithms[1](https://arxiv.org/html/2609.02984#alg1)and[3](https://arxiv.org/html/2609.02984#alg3), is the dominant approach in cross\-silo scenarios where network constraints are moderate\. It is commonly used for Multiple Graph Instance settings\([He et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib250)\), Vertical Subgraph settings\([Chen et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib410)\), and, less frequently, for cross\-silo subgraph cases\([Yao et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib419)\)\. As in Euclidean FL, periodic aggregation trades off convergence speed and communication cost\. For Euclidean data, prior work typically assumed generic data distributions and derived convergence bounds from the local objectives \([2](https://arxiv.org/html/2609.02984#S2.E2)\)\([Stich, 2019](https://arxiv.org/html/2609.02984#bib.bib16);[Yu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib17)\)\. In the graph case, structural assumptions such as graph homophily allow a refined convergence characterization\([Yao et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib419)\)as a function of the input graphs\{Gk\}k∈𝒦\\\{G\_\{k\}\\\}\_\{k\\in\\mathcal\{K\}\}\. In particular,\([Yao et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib419)\)uses a stochastic random graph model\([Keriven et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib294)\)with fixed homophily for node\-level tasks to analyze convergence in the subgraph edge\-cut configuration using a GCNℱ\\mathcal\{F\}\. Their analysis covers both i\.i\.d\. and label\-shift scenarios, showing that convergence depends on the structural shift between eachGkG\_\{k\}and the global graphGG\. Higher graph homophily, lower label shift, and smaller structural shift, achieved via higher\-hop graph expansion, improve convergence\. As the number of agentsKKincreases, distributional shifts intensify, requiring proportionally larger hop expansions to sustain performance\.
- •Decentralized synchronous model aggregationis typically used in ego\-graph scenarios\([Olshevskyi et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib148)\)and has been extensively analyzed for subgraph edge\-cut cases\([Scardapane et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib336)\)\. The mixing matrixW=\[wk​l\]\(k,l\)∈𝒦2W=\[w\_\{kl\}\]\_\{\(k,l\)\\in\\mathcal\{K\}^\{2\}\}corresponds to the adjacency matrix of the communication graph derived from the subgraph edge\-cut\. Convergence proofs rely on standard FL assumptions of loss similarity and training stability\([Stich, 2019](https://arxiv.org/html/2609.02984#bib.bib16);[Yu et al\., 2019](https://arxiv.org/html/2609.02984#bib.bib17)\)\. Stability requiresWWto be doubly stochastic and connected\. Dual\-based distributed aggregation inspired by ADMM\([Boyd et al\., 2011](https://arxiv.org/html/2609.02984#bib.bib118)\)has also been proposed\([Scardapane et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib336)\)\. Consistent with Euclidean FL, frequent aggregation accelerates convergence and improves resilience to link loss\. Additional robustness to link failures is achieved in\([Gao et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib263)\)by centrally training a GNN to perform distributed averaging that tolerates both link loss and partial participation\. Other works propose to use a different mixing matrix at each round, subsampled from the original communication graph to limit communications\. This subsampling optimizes the mixing matrix at every round using an actor\-critic model, trained to minimize convergence time\. During training, backpropagating the gradient \([28](https://arxiv.org/html/2609.02984#alg3.l28)of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\) requires communication between agents for collaborative inference and local gradient aggregation\. This communication challenge can be mitigated\([Olshevskyi et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib148)\)through batching and message piggybacking: gradients are backpropagated along the graph during each sample’s backward pass with small messages, while expensive parameter consensus occurs only once per mini\-batch, amortizing the communication cost across multiple samples\.
- •Decentralized asynchronous model aggregationuses gossip\-based protocols\([Zheng et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib261);[Giaretta and Girdzijauskas, 2023](https://arxiv.org/html/2609.02984#bib.bib19)\), though the derivation of the formal convergence rates remains an open problem\.\([Giaretta and Girdzijauskas, 2023](https://arxiv.org/html/2609.02984#bib.bib19)\)propose asynchronously exchanging parameters at the layer level—aggregating different layers at different times—to enhance robustness to asynchronicity\.
- •Hierarchical aggregation, discussed in\([Guo et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib200)\), improves the scalability of learning by combining local peer\-to\-peer aggregation with occasional centralized coordination, while collaborative inference is proposed to remain peer\-to\-peer, to avoid central bottlenecks\.

Beyond topology choice, communication cost can be reduced by adjusting the timing of model aggregation:

- •Pre\-trainingperforms part of the training centrally before collaborative learning, avoiding communication during early training stages\([Krasanakis et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib262);[Gao et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib263)\)\. A hybrid approach initializes parameters from a centrally pre\-trained model\([Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175)\)before fine\-tuning collaboratively with Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\.
- •Local SGDruns several local epochs between aggregation rounds\. As in standard FL \(Section[2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1)\), this trades reduced communication for slower model synchronization\. This trade\-off has also been studied in the subgraph setting\([Yao et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib419)\)\.

##### Open challenges\.

While aggregation techniques are well\-studied, key questions remain about aligning computational and communication topologies, establishing the viability of highly decentralized inference, and understanding the role of localized learning in GNNs leading:

- •Towards separating input, computational, and communication graphs\.Current work treats computational topology𝒢\\mathcal\{G\}and communication topologyWWas independent\. However, graph rewiring, that is, separating the input graph𝒢\\mathcal\{G\}, that is, the graph observed, from the computational graph, that is, the graph defining the node computational trees𝒯v\\mathcal\{T\}^\{v\}\(Figure[12](https://arxiv.org/html/2609.02984#S3.F12)\), improves scalability in centralized settings\([Wang et al\., 2019b](https://arxiv.org/html/2609.02984#bib.bib1);[Fatemi et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib302);[Kazi et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib2);[Topping et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib381);[Gutteridge et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib291)\)\. Further research could explore therefore collaborative solutions that jointly optimize input, computational, and physical topologies to improve efficiency, for example by aligning message\-passing with the available communication links\.
- •Towards establishing viability of highly decentralized inference\.Most work assumes centralized or modestly decentralized aggregation as seen in Table[2](https://arxiv.org/html/2609.02984#S3.T2)\)\. However, highly decentralized scenarios \(millions of agents representing one graph node each\) with asynchronous, resource\-constrained environments remain largely unexplored\. Whether decentralized message\-passing can remain correct and stable under realistic conditions \(e\.g\., asynchrony, heterogeneity, partial participation, or justify its complexity over simpler alternatives, remains uncertain\([Scardapane et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib336);[Blumenkamp et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib264)\), and further research is needed to understand performance bounds and practical benefits\.
- •Towards modular frameworks with topology as post\-processing\.Current work trains end\-to\-end MPNNs coupling feature transformation with message\-passing, requiring substantial embedding communication\. However, deep MPNNs may be unnecessarily complex for certain tasks, such as node classification on homophilous graphs\([Krasanakis et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib262)\)\. Agents could train Euclidean models locally, then apply lightweight graph post\-processing\([Zhu and Ghahramani, 2002](https://arxiv.org/html/2609.02984#bib.bib10)\), substantially reducing communication\. Further research is needed to understand the limitations of this simplification\([Bechler\-Speicher et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib286)\), but also the possibilities it brings, for example, providing a way to utilize foundation models\.

Aggregation techniques for graph\-structured dataextend centralized, decentralized, and hierarchical patterns from Euclidean collaborative learning \(Section[2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1)\) to accommodate dual aggregation needs: model parameters during training and node embeddings during inference\. Unlike Euclidean settings, where communication topology depends solely on physical networks, message passing couples computational topology𝒢\\mathcal\{G\}with communication topologyWW, either derivingWWfrom edge cuts in𝒢\\mathcal\{G\}or maintaining independent topologies\. This coupling enables graph\-specific optimization: computational tree trimming removes marginally contributing branches, pre\-aggregation using simple GNNs reduces transmission to lower\-hop neighborhoods, and dimensionality reduction transmits pseudo\-labels instead of full embeddings\. Future research is needed, however, for the joint optimization of learning and network design, for the support of highly\-decentralized cases, and for resource\-efficient aggregation with modular learning frameworks\.Trilemma\.Efficiency techniques \(tree trimming, stale embeddings, pre\-aggregation\) reduce communication costs but degrade model effectiveness, with impact depending on structural properties \(homophilous level, critical branches of trees\)\. Pre\-communicating expanded subgraphs exposes neighborhood structure, embedding exchanges expose intermediate representations during inference, and decentralized aggregation distributes exposure across agents rather than centralizing it, requiring privacy preserving schemes\.

#### 3\.5\.2Asynchronous Aggregation and Delay Control

The synchronous standard Algorithms[1](https://arxiv.org/html/2609.02984#alg1)and[3](https://arxiv.org/html/2609.02984#alg3)assume synchronicity of agents for model aggregation and message aggregation\. However, in practice, communication and computing delays can differ from agent to agent and can also vary in time, VFnot only due to device heterogeneity, but also due to structural shifts in the graph topology\. This leads to stale model parameters and embeddings, which challenge the efficiency of the learning\. Therefore, solutions that address the communication delays have been explored\. Conversely, asynchronicity in the form of introduced delay has also been exploited to improve the effectiveness of models\.

##### Model aggregation under asynchronicity\.

In collaborative training of GNNs, as in Algorithms[1](https://arxiv.org/html/2609.02984#alg1)and[3](https://arxiv.org/html/2609.02984#alg3), the model aggregation is similar to traditional FL\. Consequently, techniques and convergence results from Section[2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2)apply to GNN\. With subgraph partitioning, the communication graph often matches the computation graph, making the system topology\-dependent\. This ties aggregation issues to asynchronous aggregation over graphs, as studied in\([Lian et al\., 2018](https://arxiv.org/html/2609.02984#bib.bib387)\)\.

##### Embedding aggregation

under asynchronicity\. Standard GNN inference assumes synchronous message passing between nodes \([18](https://arxiv.org/html/2609.02984#S3.E18)\),\([19](https://arxiv.org/html/2609.02984#S3.E19)\), and \([20](https://arxiv.org/html/2609.02984#S3.E20)\), which introduces delays and requires synchronization steps\. To avoid these, message passing under asynchronicity is considered, with two complementary approaches: to mitigate the effect of the delays, or conversely, to exploit delays to improve model performance\.

Staleness is typically reflected in the indexttof the node embeddingshv\(l\)​\(t\)h\_\{v\}^\{\(l\)\}\(t\), arising due to imperfect synchronization steps \(lines[19](https://arxiv.org/html/2609.02984#alg3.l19)and[20](https://arxiv.org/html/2609.02984#alg3.l20)of Algorithm[3](https://arxiv.org/html/2609.02984#alg3)\) during collaborative inference\. Some works suggest to introduce topological delays or acceleration, that changes the hop count indexllin the message aggregation \([19](https://arxiv.org/html/2609.02984#S3.E19)\), either at the level of the aggregated messagesmv,u\(l\)m^\{\(l\)\}\_\{v,u\}or in the neighborhood𝒩\(l\)​\(v\)\\mathcal\{N\}^\{\(l\)\}\(v\)\([Gutteridge et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib291);[Chen et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib324)\)\.

The main solutions with asynchronous embedding aggregation focus on:

- •Modifying the collaborative inference protocolto maintain model performance despite embedding staleness\. GNN training with stale embeddings is studied in\([Fey et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib327);[Peng et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib328)\)\. To reduce communication costs during training, these works propose to skip certain embedding transmissions and derive convergence bounds that account for the resulting staleness\. Message passing under asynchronicity is also addressed in\([Yu et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib326)\), where the communication and processing burden of stale embeddings is reduced by sequentially performing updates along pre\-defined paths in the graph, referred to as update chains\.
- •Modifying the MPNN to mitigate staleness\.Specifically, \([18](https://arxiv.org/html/2609.02984#S3.E18)\), \([19](https://arxiv.org/html/2609.02984#S3.E19)\), and \([20](https://arxiv.org/html/2609.02984#S3.E20)\) are modified to improve robustness to embedding staleness during collaborative inference\. For example, Energy GNNs\([Gu et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib12)\), presented as a learnable version of diffusion algorithms on networks \(see Section[2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1)\) allow for asymptotic inference that is provably robust to stale embeddings\([Solodova et al\., 2025](https://arxiv.org/html/2609.02984#bib.bib380)\)\. Alternatively, asynchronous MPNNs\([Faber and Wattenhofer, 2024](https://arxiv.org/html/2609.02984#bib.bib325)\)support scenarios in which a feature update at an origin node triggers asynchronous updates throughout the graph\. These models propose integrating the origin node’s state into the messages \([18](https://arxiv.org/html/2609.02984#S3.E18)\) to mitigate the effects of stale embeddings\([Faber and Wattenhofer, 2024](https://arxiv.org/html/2609.02984#bib.bib325)\)\. Another asynchronous MPNN class is proposed in\([Mathys et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib274)\), where node updates are propagated outward from the origin up to distanceLL\(flood phase\) and then back \(echo phase\), a process found to be more effective than standard MPNNs\.
- •Modifying the MPNN to improve expressiveness\.Variants of the MPNN have also been proposed to incorporate topological delays for enhanced expressiveness\. These delays can be controlled by learnable node actions \(e\.g\., listening, broadcasting, or remaining inactive\)\([Finkelshtein et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib292)\)\. Embedding updates may be delayed or accelerated based on topological considerations\([Gutteridge et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib291);[Chen et al\., 2024b](https://arxiv.org/html/2609.02984#bib.bib324)\), typically by modifying the aggregation step \([19](https://arxiv.org/html/2609.02984#S3.E19)\)\. Delays arise whenll\-hop embeddings from 1\-hop neighbors are used, whereas accelerations occur when embeddings from nodes beyond 1\-hop are included\. These adjustments help address oversmoothing\([Chen et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib295)\)and oversquashing\([Topping et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib381)\), two expressive limitations of GNNs\.

##### Open challenges\.

While asynchronous MPNN architectures are advancing, key questions remain about adapting to dynamic agent capabilities and thus dynamically changing staleness, and about handling extreme staleness regimes in prolonged\-delay environments leading:

- •Towards dynamic capability\-aware MPNNs\.Current work assumes homogeneous agents with static capabilities\. However, practical deployments involve dynamic fluctuations in computational power, memory, and bandwidth due to varying workloads and network conditions\. While prior work balances static capability constraints\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207);[Chen et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib172);[Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\), dynamically adapting message\-passing depth, embedding dimensions, or aggregation responsibilities to runtime capability changes remains unexplored\. For graphs, this requires the challenging coordination of computational tree traversal across agents operating at varying speeds\.
- •Towards aggregation strategies for extreme staleness regimes\.Current asynchronous methods handle moderate staleness\. However, prolonged delays \(satellite networks, sensor networks with intermittent connectivity\) create extreme staleness and cyclical connectivity patterns\. Key challenges include heterogeneous embedding ages across nodes, trade\-offs between recent\-but\-distant versus stale\-but\-neighbor embeddings, and exploiting predictable patterns for optimization\. Developing aggregation strategies that leverage temporal patterns when staleness varies by orders of magnitude across computational trees remains unexplored\.

Asynchronous aggregation techniques for graph\-structured dataextend asynchronicity handling from Euclidean federated learning to address staleness in both model parameters and node embeddings\. Unlike Euclidean settings, where asynchronicity affects only model synchronization, graphs require managing embedding staleness during message\-passing\. Two strategies emerge: mitigating delays through protocol modifications \(skipping transmissions, sequential updates\) or exploiting delays for effectiveness \(topological acceleration/deceleration addressing oversmoothing and/or oversquashing\)\. Future research is needed to handle dynamically changing staleness and emerging systems with extreme delays\.Trilemma\.Skipping embeddings or using stale values reduces communication but degrades effectiveness\. Specifically, deep computational trees and high\-degree nodes are more sensitive to staleness\. Asynchronous protocols expose temporal patterns \(update timing, message sequences\) and topology through sequential update paths, creating attack surfaces beyond synchronous aggregation\.

#### 3\.5\.3Communication Efficiency

For collaborative GNNs with topology partitioning, embedding vectors must be communicated among agents during the message\-passing phase\. These exchanges can occur in centralized, peer\-to\-peer, or mesh architectures and are subject to heterogeneous, unreliable, and costly communication links\. Similar to model aggregation in traditional FL, the efficiency and reliability of embedding aggregation depend on the state of the communication links and the cost of transmission\.

However, communication for message passing differs from conventional data exchange: embeddings are intermediate representations, and the downstream learning task typically depends on aggregated functions of these embeddings rather than their exact values\. This observation motivates the design of communication schemes tailored to collaborative GNNs, optimizing transmission efficiency, adapting embedding aggregation to wireless communication, and communication patterns to the network state\.

##### Transmission efficiency\.

Transmission efficiency quantifies the utility of the transmitted embeddings relative to the consumed communication resources, such as bandwidth and transmission power\. In collaborative GNNs, this is particularly critical when each agent holds only a small portion of the graph, potentially a single node, as in fully decentralized settings\. In\([Lee et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib245)\), a coding and retransmission scheme is proposed to improve the reliability of embedding aggregation, aiming to maximize inference accuracy under constrained resources\. The work in\([Wang et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib319)\)proposes a node subsampling strategy for selecting communication edges that accommodates link bandwidth constraints, to minimize convergence time during training\. Theoretical analysis for the same scenario is provided in\([Gao et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib248)\), where disruptions in embedding communication are modeled by a probability of failed message passing due to wireless impairments or temporary disconnections\. The results show that accurate inference remains possible if such stochastic failures are accounted for in the learning process\.

##### Wireless communication for embedding aggregation\.

The principles of over\-the\-air computation, originally proposed for FL model aggregation, have been extended to embedding aggregation in collaborative GNNs\([Gao and Gündüz, 2023](https://arxiv.org/html/2609.02984#bib.bib6);[Lee et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib267);[Gao and Gündüz, 2025](https://arxiv.org/html/2609.02984#bib.bib247)\)\. In this setting, the superposition property of the wireless channel is exploited to aggregate embeddings directly during transmission, reducing communication costs and allowing simultaneous updates from multiple agents\. These methods have been shown to maintain robustness against channel impairments and, in some cases, operate without channel state information\. Extensions to more complex and dynamic topological structures are explored in\([Fiorellino et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib5)\)\. Practical demonstrations, such as\([Blumenkamp et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib264)\), further highlight that variable delays and unreliable communication links must be considered in system design, while resource allocation in wireless mesh networks\([Wang et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib266)\)has been shown to significantly affect application performance\.

##### Client scheduling and topology management\.

Client scheduling and topology management for collaborative GNNs are mainly considered for subgraphs with topology partitioning in learning tasks over wide\-area networks, where the message\-passing operation is costly\. In\([Liu et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib207)\), collaborative GNNs in wide\-area networks are considered\. To reduce traffic congestion, client selection is performed based on the importance of the communicated embeddings and the communication costs\. Scheduling algorithms are designed to allow interleaved message passing and local updates\. A multi\-tier wide\-area network with edge servers is considered in\([Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249);[Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260)\)\. The topology of the GNN is given by the application, and these papers address the problem of GNN node assignment to edge servers and edge\-network topology design for communication\-efficient training, for both static\([Zeng et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib249)\)and dynamic\([Nazzal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib260)\)GNN topologies\.

Most works on communication efficiency in collaborative GNN implementations assume that the communication graph given by the possible connections between agents is identical to the computing graph given by the adjacency matrixAA\. This assumption is not necessary\. An algorithm to find the optimal communication graph for subgraphs with topology partitioning is proposed in\([Scardapane et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib336)\)\. Conversely,\([Wang et al\., 2025a](https://arxiv.org/html/2609.02984#bib.bib319)\)assumes a fully connected communication graph \(i\.e\., agents are free to communicate with anyone in a peer\-to\-peer network\) but samples a communication topology at each round\. The authors design an adaptive algorithm that samples communication topologies adaptively throughout training to accommodate bandwidth constraints while maintaining fast inference and training\.

##### Open challenges\.

Existing work has started to address the challenges of communicating embeddings and model parameters over unreliable or costly communication channels or transmission paths\. However, since both inference and learning involve communication, several interesting research questions remain open in the co\-design of communication and computing, in utilizing the broadcast and multicast capabilities of the wireless channel, and in ensuring safe learning and inference despite the communication impairments leading:

- •Towards co\-designing communication and computing\.While recent methods consider the effects of unreliable communication channels\([Lee et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib245);[Gao et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib248)\), they treat communication and computation topologies independently\. Co\-designing these topologies while explicitly accounting for channel quality represents an avenue for further efficiency improvements\. Additionally, existing embedding aggregation methods operate on static neighbor selection policies and could be extended to dynamically exploit time\-varying wireless channel quality by scheduling or selecting neighbors with temporarily favorable channel conditions\.
- •Towards communication\-native inference\.The wireless medium itself, with its superposition and broadcast properties, natively supportsmany\-to\-oneandone\-to\-manycommunications\. These properties could allow efficient embedding and model aggregation in a single operation among clusters of agents or agents connecting to the same base station or edge computing server\. While initial results exist in\([Fiorellino et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib5)\), further research is still needed to fully exploit these network environment properties through complex topological structures beyond simple graphs\.
- •Towards safe training and inference\.Training and inference on graph data require communication in both phases\. Consequently, communication impairments deteriorate inference quality even when the model itself is optimal, potentially leading to unsafe decisions\. This raises the question of whether safety requirements should be incorporated into the learning process or considered only during inference, a design choice that requires further investigation\.

Communication efficiency techniques for graph\-structured dataadapt transmission optimization and topology management from Euclidean collaborative learning, but the importance of communication efficiency is increased due to the need for both message passing for inference and for backpropagation during training\. Most results address learning effectiveness and efficiency directly, or suggest solutions that utilize the broadcast and superposition properties of the wireless channel\. Challenges remain to increase effectiveness by additional network\-specific solutions, but also to ensure safe decisions despite information loss both at inference and learning\.Trilemma\.Coding and retransmission schemes improve reliability under channel impairments but increase transmission overhead, while topology subsampling and neighbor selection reduce communication costs but risk slower convergence or degraded inference effectiveness\. Over\-the\-air computation and wireless transmission expose intermediate embeddings to any eavesdropper within radio range\.

### 3\.6Privacy Preservation Techniques for Graph Data

A high variety of privacy\-preserving techniques are proposed for distributed graph data, aiming to protect training and inference data fromhonest\-but\-curiousagents and central servers\. These techniques address the vulnerabilities identified in Section[3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4), model parameter leakage from collaborative training, embedding, and topology leakage from collaborative inference\. Anonymization has to target three main levels:\(i\)model parameters,\(ii\)exchanged node embeddings, and\(iii\)exchanged edge information\. Many of these techniques build on the blind computation methods, and LDP introduced in Section[2\.5](https://arxiv.org/html/2609.02984#S2.SS5)\. As shown in Table[2](https://arxiv.org/html/2609.02984#S3.T2), privacy preservation is an important topic for collaborative GNNs, with most of the works focusing on cross\-silo applications, with use cases handling personal information, like health data or user preferences\.

##### Anonymizing model parameters\.

To address the inherited vulnerabilities from collaborative learning \(Section[2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4)\), model parameters or gradients exchanged under GNN training can be anonymized directly with the techniques proposed for FL in Section[2\.5](https://arxiv.org/html/2609.02984#S2.SS5)\([Liu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib186);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Yao et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib419);[Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\)\. These techniques can be further refined to improve the stability–privacy trade\-off, such asadaptive noise scaling\([Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405)\), in which the injected noise magnitude is adjusted according to the gradient norm to maintain a consistent perturbation throughout training\.

##### Anonymizing exchanged embeddings\.

As discussed in Section[3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4), collaborative inference over graphs requires exchanging intermediate embeddings across subgraph boundaries, creating attack surfaces for feature extraction and membership inference\. Early works like\([Chen et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib410)\)incorrectly assumed that if local computations are performed entirely in isolation, including local model updates and isolated inference, then the raw node featuresXkX\_\{k\}would be inherently protected from privacy leakage\. However, it is now clear that even when only intermediate embeddingshv\(l\)h\_\{v\}^\{\(l\)\}are shared and aggregated, they must still be protected through some form of differential privacy\([Chen et al\., 2024c](https://arxiv.org/html/2609.02984#bib.bib184)\)\.

- •Blind computationmethods securely aggregate embeddings \(e\.g\., computingfAggf\_\{\\mathrm\{Agg\}\}in \([19](https://arxiv.org/html/2609.02984#S3.E19)\)\) or average representationszv​\(t\)z\_\{v\}\(t\)without revealing their values\. For example, secure multiparty computation can be applied to jointly perform specific GNN functions such as aggregatingfA​g​gf\_\{Agg\}\([19](https://arxiv.org/html/2609.02984#S3.E19)\) or computing matrix multiplications\([Wang et al\., 2023b](https://arxiv.org/html/2609.02984#bib.bib447)\)\. Due to their high computational cost, these methods are often applied only to a subset of layers\([Chen et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib410)\), with the remaining computations delegated to the server in a split\-learning setup\. Homomorphic encryption offers another option, allowing the server to aggregate results while keeping them encrypted end\-to\-end\([Ran et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib448);[Peng et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib449)\), using techniques such as secure multiparty computation\([Chen et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib410)\)or homomorphic encryption for server\-side aggregation\([Ni et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib171)\)\.
- •LDPtechniques can be applied directly to the communicated vectorszvz\_\{v\}orhv\(l\)h\_\{v\}^\{\(l\)\}\. The simplest approach adds Gaussian noise to the vectors\([Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188);[Lee et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib267);[Ni et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib171)\)\. More advanced methods adapt classical LDP protocols to the GNN setting\. These generally follow a pipeline of noise injection, vector clipping, dimension subsampling, and privacy budget adjustment\. Noise can follow a Laplace distribution\([Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181)\), a Gaussian distribution\([Chen et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib410)\), or a Bernoulli distribution\([Zheng et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib261);[Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)\. Clipping may be applied to vectors or gradients\([Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181);[Chen et al\., 2021a](https://arxiv.org/html/2609.02984#bib.bib410);[Liu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib186);[Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\)and can be combined with stochastic quantization\([Lin et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib251);[Zheng et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib261);[Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)\. Dimensionality reduction can be performed by random subsampling\([Lin et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib251);[Pei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib268);[Zheng et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib261);[Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)or top\-kksampling\([Chen et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib172)\)\. In some cases, these operations are unified through complex noise functions such as piecewise noise injection\([Pei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib268);[Wang et al\., 2019a](https://arxiv.org/html/2609.02984#bib.bib303)\)\. The privacy level can also be balanced against other objectives or system constraints\. The noise added in LDP is known for degrading learning effectiveness\([Lee et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib267)\), mitigated while aggregating high degree nodes\([Guo et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib200)\)\. Therefore, adaptive noise strategies propose to scale the noise by the gradient norm during training to stabilize learning\([Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405)\)\. When communication occurs over wireless channels, the trade\-off between signal\-to\-noise ratio and DP can be jointly optimized\([Lee et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib267)\)\. LDP can also be applied to node labels in transfer learning with adversarial losses to protect the training data of the source agent\([Guan et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib222)\)\.

##### Anonymizing edge \(or topology\) information\.

Beyond node feature extraction, adversaries can extract existing edges from shared embeddings \(Section[3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4)\), a threat known aslink stealing\([Pei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib268)\)\. This represents a serious privacy attack when the existence of the edge itself represents sensitive information, or when the edges encode most of the information, as is often the case in heterogeneous graphs\([Liu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib186);[Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\)\. Therefore, the edge or topology information needs to be protected in many scenarios\. The concept of*Edge Local Differential Privacy \(Edge LDP\)*is formalized in\([Wu et al\., 2022b](https://arxiv.org/html/2609.02984#bib.bib196);[Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195);[Lin et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib251)\)and is extended to heterogeneous graphs in\([Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\), providing privacy guarantees for graphs differing by a single edge\. In some cases, protection also covers statistics on the edge set, such as degree distributions\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)\. Protection strategies build both on blind computations and on LDP\.

- •Blind computationscan be employed forprivacy\-preserving subgraph expansion\.Each agent obtains itsLL\-hop expansionG~k\\tilde\{G\}\_\{k\}of the local graph using blind computation techniques\. The process begins with agents collectively running a private set intersection \(PSI\) protocol to identify the vertex cut𝒱c​u​t\\mathcal\{V\}\_\{cut\}between agents\. Given the knowledge of𝒱c​u​t∩𝒱k\\mathcal\{V\}\_\{cut\}\\cap\\mathcal\{V\}\_\{k\}, each agent collects and communicates the missing expanded subgraphs, typically via a central server\([Zhang et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib174);[Chen et al\., 2021c](https://arxiv.org/html/2609.02984#bib.bib173);[Chen et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib175);[Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217);[Qiu et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib182)\)\. Blind computation techniques can also be applied to aggregate useful graph statistics, such as node degree\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\), using methods such as secure group aggregation\([Bonawitz et al\., 2017](https://arxiv.org/html/2609.02984#bib.bib395)\)or zero\-knowledge protocols\([Goldwasser et al\., 1985](https://arxiv.org/html/2609.02984#bib.bib214)\)\. Such statistics may be required to compute auxiliary losses, for example, fairness losses\([Agrawal et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib217)\), or may be used by the local learning algorithm\([Pan et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib213)\)\.
- •Standard LDP\([Dwork and Roth, 2014](https://arxiv.org/html/2609.02984#bib.bib59)\)can be applied to edge information in set formℰk\\mathcal\{E\}\_\{k\}\([Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216);[Lin et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib251)\)or in adjacency matrix formAkA\_\{k\}\([Han et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib195);[Qiu et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib182)\)\. For example,\([Zhang et al\., 2021b](https://arxiv.org/html/2609.02984#bib.bib178)\)projectsAkA\_\{k\}into a lower\-dimensional Gaussian random subspace\. However, LDP introduces topological noise, which is known to degrade GNN performance even at low noise levels\([Zügner et al\., 2020](https://arxiv.org/html/2609.02984#bib.bib194)\)\. Excessive noise can densify the graph, leading to*oversmoothing*\([Chen et al\., 2020a](https://arxiv.org/html/2609.02984#bib.bib295)\)\. Mitigation strategies include regularization\([Lin et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib251)\), degree preservation\([Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\), or allocating part of the privacy budget to maintain adjacency sparsity\([Qiu et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib182)\)\.

Protecting edge information is often formulated alongside protecting the raw featureXkX\_\{k\}, thus protecting the local graph dataGk=\(Ak,Xk\)G\_\{k\}=\(A\_\{k\},X\_\{k\}\)\. The concept of node\-level privacy is introduced informally for the ego\-graph case\([Guo et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib200)\), consisting of protecting the raw features of nodesvvalongside the 1\-hop topology typical of ego\-graph partitions\. This concept is further extended to heterogeneous graphs, specifically focusing on protecting nodes that correspond to users\([Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\)\. Other works focus on protecting collections of ego\-graphs sampled fromGkG\_\{k\}\([Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188);[Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405)\)\. Subgraph\-level privacy extends this protection to entire subgraphsGkG\_\{k\}obtained from aggregating the private ego\-graphs of the agents\([Guo et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib200)\)\. Techniques that anonymizeAkA\_\{k\}together withXkX\_\{k\}have been proposed in the following forms:

- •Blending ego\-graphs together\.The embeddings of ego\-graphs are averaged across all ego\-graphs subsampled fromGkG\_\{k\}in the same batch, respectively between nodes at the same depth in the computational tree, to anonymize the ego\-graphs and their embeddings\([Zhang et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib188)\)\.
- •Inferring pseudo\-edges\.This common technique is used for anonymizingAkA\_\{k\}for heterogeneous graphs, usually in the vertex cut case\([Liu et al\., 2022c](https://arxiv.org/html/2609.02984#bib.bib186);[Wu et al\., 2021](https://arxiv.org/html/2609.02984#bib.bib181);[Wu et al\., 2022a](https://arxiv.org/html/2609.02984#bib.bib405);[Yan et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib216)\)\. It consists of constructing unobserved yet highly probable edges based on node representation similarity, and adding them to the observed edges inGkG\_\{k\}\.

In hierarchical setups \(e\.g\., device–edge and server–central server\)[Guo et al\. \(2024\)](https://arxiv.org/html/2609.02984#bib.bib200)considers privacy protections between higher\-rank and lower\-rank entities as well as among peers of the same rank\. The above anonymization techniques, focusing on model parameters, embeddings, and topology information, can then be selectively combined depending on the nature of the data exchanged at each hierarchy level\.

##### Open challenges\.

Privacy\-preserving techniques for collaborative GNNs operate at the intersection of two research domains: privacy mechanisms for GNNs and privacy\-preserving collaborative training and inference\. This convergence presents unique challenges and opportunities, as collaborative settings introduce multi\-agent dynamics that both complicate and potentially enhance privacy guarantees, leading:

- •Towards multi\-agent privacy amplification\.Most work applies LDP locally to individual training or inference data\. However, LDP degrades learning effectiveness by compressing communicated information\([Smith et al\., 2017a](https://arxiv.org/html/2609.02984#bib.bib78);[Bonawitz et al\., 2022](https://arxiv.org/html/2609.02984#bib.bib72)\)\. Privacy amplification \(Section[2\.5](https://arxiv.org/html/2609.02984#S2.SS5)\) offers an alternative: agents sequentially amplify protection guarantees along collaborative computation chains rather than applying hard local anonymization\. This approach fits naturally with sequential MPNN operations and could limit information loss while maintaining privacy guarantees, but is still unexplored for collaborative graph settings\.
- •Towards integrating privacy with defense mechanisms\.Current anonymization techniques protect against honest\-but\-curious agents \(Section[3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4)\) but are insufficient against byzantine malicious agents who actively manipulate graph structure or features\. Privacy mechanisms are often treated as a first\-line defense against attacks such as link stealing\([Pei et al\., 2023](https://arxiv.org/html/2609.02984#bib.bib268)\), embedding extraction\([Chen et al\., 2024c](https://arxiv.org/html/2609.02984#bib.bib184)\), and graph rewiring\([Chen et al\., 2024c](https://arxiv.org/html/2609.02984#bib.bib184)\)\. However, how privacy mechanisms can be combined with additional defenses \(e\.g\., embedding filtering\([He et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib193)\)\) against sophisticated adversaries remains unclear, since privacy guarantees often hide the very signals needed for detecting these adversarial threats, calling for unified frameworks that jointly address honest\-but\-curious and malicious threat models\. For instance, mitigating adversarial edge injection may require integrating verifiable private mechanisms\([Daly et al\., 2024](https://arxiv.org/html/2609.02984#bib.bib75)\)to ensure graph integrity without sacrificing privacy\.

Privacy\-preserving techniques for graph\-structured dataadapt blind computation and local differential privacy from Euclidean federated learning \(Section[2\.5](https://arxiv.org/html/2609.02984#S2.SS5)\), to protect not only the model parameters, but also the intermediate embeddings, and the topology information\. This three\-level protection requirement is amplified by repeated embedding exchanges across subgraph boundaries during multi\-layer message\-passing, creating cumulative exposure particularly severe for boundary nodes with many cross\-agent connections\. Additionally, graph\-specific techniques are proposed to randomize edge and subgraph data\. Since privacy preservation affects both efficiency and effectiveness heavily, further research is needed to fit the methods to the computation chains of GNNs and to effectively combine them with similarly costly security mechanisms\.Trilemma\.Blind computation preserves data locality but is computationally heavy; LDP\-based methods offer formal privacy guarantees for embeddings and edges but can degrade message\-passing quality and inference accuracy\. Edge LDP, in particular, introduces topological noise that may cause oversmoothing through graph densification, requiring mitigation strategies that can further constrain model effectiveness\.

## 4Conclusion

Collaborative learning has unlocked access to previously unavailable distributed data sources, leading to the success of federated learning and decentralized learning on Euclidean data, and more recently, to emerging solutions for graph\-structured data\. Extending collaborative learning to graph\-structured data opens up opportunities that exceed those of centralized graph ML through its ability to mine data across domains ranging from sensor networks to molecular biology\.

With this survey, we systematically map the design choices from Euclidean\-structured to graph\-structured collaborative learning\. We first consolidate the foundational principles of collaborative learning on Euclidean data \(Section[2](https://arxiv.org/html/2609.02984#S2)\), then extend this mapping to graph\-structured data \(Section[3](https://arxiv.org/html/2609.02984#S3)\)\. For both cases, we formulate the fundamental problem, discuss the challenges of data and system heterogeneity, and analyze how the trilemma of effectiveness, efficiency, and privacy preservation manifests in the design choices for addressing these challenges\. Specifically for graph\-data, we identify emerging research challenges that define its future development\.

Our analysis reveals that collaborative learning on graph\-structured data has given rise to a nascent, yet rich, landscape of problems and solutions\. This landscape can be broadly divided into two paradigms:isolated inferenceandcollaborative inference\. Notably, collaborative inference emerges as a more central paradigm for graph\-structured data due to the message passing mechanism of GNNs\. Beyond challenges that persist in Euclidean collaborative learning, combining distributed relational data with collaborative training and/or inference introduces unique constraints that require novel solutions\. This includes the joint consideration of topologies, communication, privacy, and heterogeneities\.

By systematically bridging collaborative learning on Euclidean and graph\-structured data, this survey establishes a coherent vision to support researchers in developing the next generation of collaborative graph learning methods and to enable their practical impact across scientific and industrial domains, leading to societally important breakthroughs in drug discovery, healthcare, sustainable industries, and smart cities\.

## Acknowledgement

This work was supported in part by Vinnova, Sweden´s Innovation Agency, under grant no\. 2024\-00648\.

## References

- \[1\]\(2016\)Deep learning with differential privacy\.InACM SIGSAC Conference on Computer and Communications Security,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I14.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I14.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S2.I14.i3.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.2)\.
- \[2\]E\. Abbe\(2018\)Community detection and stochastic block models: recent developments\.Journal of Machine Learning Research18\(177\),pp\. 1–86\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1)\.
- \[3\]N\. Agarwal, A\. T\. Suresh, F\. X\. X\. Yu, S\. Kumar, and B\. McMahan\(2018\)CpSGD: communication\-efficient and differentially\-private distributed SGD\.Advances in Neural Information Processing Systems31\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1)\.
- \[4\]N\. Agrawal, A\. K\. Sirohi, and S\. Kumar\(2024\)No prejudice\! Fair federated graph neural networks for personalized recommendation\.InAAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p6.1),[1st item](https://arxiv.org/html/2609.02984#S3.I12.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I24.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I8.i3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.7),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.1.1)\.
- \[5\]G\. Andrew, O\. Thakkar, B\. McMahan, and S\. Ramaswamy\(2021\)Differentially private learning with adaptive clipping\.Advances in Neural Information Processing Systems34,pp\. 17455–17466\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I14.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1)\.
- \[6\]Apple\(2023\)Private cloud compute\.Technical reportApple\.Note:[https://security\.apple\.com/blog/private\-cloud\-compute/](https://security.apple.com/blog/private-cloud-compute/)Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p3.1)\.
- \[7\]M\. G\. Arivazhagan, V\. Aggarwal, A\. K\. Singh, and S\. Choudhary\(2019\)Federated learning with personalization layers\.arXiv:1912\.00818\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[8\]S\. M\. Azimi\-Abarghouyi and C\. Fischione\(2025\)Multi\-layer hierarchical federated learning with quantization\.arXiv:2505\.08145\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p1.1)\.
- \[9\]S\. M\. Azimi\-Abarghouyi and V\. Fodor\(2024\)Scalable hierarchical over\-the\-air federated learning\.IEEE Transactions on Wireless Communications\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px2.p1.1)\.
- \[10\]S\. Baack\(2024\)A critical analysis of the largest source for generative ai training data: common crawl\.InACM Conference on Fairness, Accountability, and Transparency,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p2.1)\.
- \[11\]J\. Baek, W\. Jeong, J\. Jin, J\. Yoon, and S\. J\. Hwang\(2023\)Personalized subgraph federated learning\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I17.i1.p1.1),[4th item](https://arxiv.org/html/2609.02984#S3.I17.i4.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.10.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.10.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.1.1)\.
- \[12\]B\. Balle, P\. Kairouz, B\. McMahan, O\. Thakkar, and A\. Guha Thakurta\(2020\)Privacy amplification via random check\-ins\.Advances in Neural Information Processing Systems33,pp\. 4623–4634\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p1.1)\.
- \[13\]R\. Bassily, A\. Smith, and A\. Thakurta\(2014\)Private empirical risk minimization: efficient algorithms and tight error bounds\.InIEEE Annual Symposium on Foundations of Computer Science,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I14.i1.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.2)\.
- \[14\]M\. Bawa, H\. Garcia\-Molina, A\. Gionis, and R\. Motwani\(2003\)Estimating aggregates on a peer\-to\-peer network\.Stanford InfoLab\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[15\]G\. Bazhenov, D\. Kuznedelev, A\. Malinin, A\. Babenko, and L\. Prokhorenkova\(2023\)Evaluating robustness and uncertainty of graph models under structural distributional shifts\.Advances in Neural Information Processing Systems36,pp\. 75567–75594\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I9.i2.p1.1)\.
- \[16\]M\. Bechler\-Speicher, B\. Finkelshtein, F\. Frasca, L\. Müller, J\. Tönshoff, A\. Siraudin, V\. Zaverkin, M\. M\. Bronstein, M\. Niepert, B\. Perozzi, M\. Galkin, and C\. Morris\(2025\)Position: graph learning will lose relevance due to poor benchmarks\.InInternational Conference on Machine Learning,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I29.i3.p1.1)\.
- \[17\]F\. Bénézit, V\. Blondel, P\. Thiran, J\. Tsitsiklis, and M\. Vetterli\(2010\)Weighted gossip: distributed averaging using non\-doubly stochastic matrices\.InIEEE International Symposium on Information Theory,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[18\]F\. Bénézit, A\. G\. Dimakis, P\. Thiran, and M\. Vetterli\(2010\)Order\-optimal consensus through randomized path averaging\.IEEE Transactions on Information Theory56\(10\),pp\. 5150–5167\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[19\]Y\. Bengio, S\. Lahlou, T\. Deleu, E\. J\. Hu, M\. Tiwari, and E\. Bengio\(2023\)Gflownet foundations\.Journal of Machine Learning Research24\(210\),pp\. 1–55\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1)\.
- \[20\]D\. P\. Bertsekas\(2011\)Incremental gradient, subgradient, and proximal methods for convex optimization: a survey\.Optimization for Machine Learning2010\(1\-38\),pp\. 3\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p7.1)\.
- \[21\]D\. Bertsekas and J\. Tsitsiklis\(2015\)Parallel and distributed computation: numerical methods\.Athena Scientific\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.3.2.2.1.1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p2.1)\.
- \[22\]A\. Bhowmick, J\. Duchi, J\. Freudiger, G\. Kapoor, and R\. Rogers\(2018\)Protection against reconstruction and its applications in private federated learning\.arXiv:1812\.00984\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1)\.
- \[23\]I\. Bistritz, A\. Mann, and N\. Bambos\(2020\)Distributed distillation for on\-device learning\.Advances in Neural Information Processing Systems33,pp\. 22593–22604\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1)\.
- \[24\]S\. Biswas, D\. Frey, R\. Gaudel, A\. Kermarrec, D\. Lerévérend, R\. Pires, R\. Sharma, and F\. Taïani\(2025\)Low\-cost privacy\-preserving decentralized learning\.InProceedings on Privacy Enhancing Technologies Symposium,Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p2.1)\.
- \[25\]A\. Bittau, Ú\. Erlingsson, P\. Maniatis, I\. Mironov, A\. Raghunathan, D\. Lie, M\. Rudominer, U\. Kode, J\. Tinnes, and B\. Seefeld\(2017\)Prochlo: strong privacy for analytics in the crowd\.InSymposium on Operating Systems Principles,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p1.1)\.
- \[26\]V\. D\. Blondel, J\. M\. Hendrickx, A\. Olshevsky, and J\. N\. Tsitsiklis\(2005\)Convergence in multiagent coordination, consensus, and flocking\.InIEEE Conference on Decision and Control,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[27\]M\. Blot, D\. Picard, M\. Cord, and N\. Thome\(2016\)Gossip training for deep learning\.InConference on Neural Information Processing Systems,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[28\]J\. Blumenkamp, S\. Morad, J\. Gielis, Q\. Li, and A\. Prorok\(2022\)A framework for real\-world multi\-robot systems running decentralized GNN\-based policies\.InIEEE International Conference on Robotics and Automation,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p6.1),[1st item](https://arxiv.org/html/2609.02984#S3.I22.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I25.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I29.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.2.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.35.2)\.
- \[29\]C\. Bodnar, F\. Di Giovanni, B\. Chamberlain, P\. Lio, and M\. Bronstein\(2022\)Neural sheaf diffusion: a topological perspective on heterophily and oversmoothing in GNNs\.Advances in Neural Information Processing Systems35,pp\. 18527–18541\.Cited by:[4th item](https://arxiv.org/html/2609.02984#S3.I17.i4.p1.1)\.
- \[30\]A\. Bojchevski and S\. Günnemann\(2018\)Deep gaussian embedding of graphs: unsupervised inductive learning via ranking\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I9.i1.p1.1)\.
- \[31\]K\. Bonawitz, P\. Kairouz, B\. McMahan, and D\. Ramage\(2022\)Federated learning and privacy\.Communications of the ACM19\(5\),pp\. 87–114\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1),[§2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I36.i1.p1.1)\.
- \[32\]K\. Bonawitz, V\. Ivanov, B\. Kreuter, A\. Marcedone, H\. B\. McMahan, S\. Patel, D\. Ramage, A\. Segal, and K\. Seth\(2017\)Practical secure aggregation for privacy\-preserving machine learning\.InACM SIGSAC Conference on Computer and Communications Security,Cited by:[§2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1)\.
- \[33\]M\. Bornstein, T\. Rabbani, E\. Wang, A\. S\. Bedi, and F\. Huang\(2023\)SWIFT: rapid decentralized federated learning via wait\-free model communication\.InInternational Conference on Learning Representations,Cited by:[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px2.p1.1)\.
- \[34\]L\. Bottou and O\. Bousquet\(2007\)The tradeoffs of large scale learning\.Advances in Neural Information Processing Systems20\.Cited by:[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px1.p3.1)\.
- \[35\]L\. Bottou, F\. E\. Curtis, and J\. Nocedal\(2018\)Optimization methods for large\-scale machine learning\.SIAM Review60\(2\),pp\. 223–311\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p7.1),[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px1.p4.3),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p4.1)\.
- \[36\]L\. Bottou\(2010\)Large\-scale machine learning with stochastic gradient descent\.InProceedings of Computational Statistics,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p1.1)\.
- \[37\]S\. Boyd, A\. Ghosh, B\. Prabhakar, and D\. Shah\(2005\)Gossip algorithms: design, analysis and applications\.InIEEE Infocom Conference on Computer Communications,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[38\]S\. Boyd, A\. Ghosh, B\. Prabhakar, and D\. Shah\(2006\)Randomized gossip algorithms\.IEEE Transactions on Information Theory52\(6\),pp\. 2508–2530\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[39\]S\. Boyd, N\. Parikh, E\. Chu, B\. Peleato, and J\. Eckstein\(2011\)Distributed optimization and statistical learning via the alternating direction method of multipliers\.Foundations and Trends® in Machine Learning3\(1\),pp\. 1–122\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p5.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I27.i2.p1.1)\.
- \[40\]C\. Briggs, Z\. Fan, and P\. Andras\(2020\)Federated learning with hierarchical clustering of local updates to improve training on non\-iid data\.InIEEE International Joint Conference on Neural Networks,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I8.i3.p1.1)\.
- \[41\]S\. Brody, U\. Alon, and E\. Yahav\(2022\)How attentive are graph attention networks?\.InInternational Conference on Learning Representations,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p3.2)\.
- \[42\]M\. M\. Bronstein, J\. Bruna, T\. Cohen, and P\. Veličković\(2021\)Geometric deep learning: grids, groups, graphs, geodesics, and gauges\.arXiv:2104\.13478\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p4.1),[1st item](https://arxiv.org/html/2609.02984#S3.I19.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px2.p1.1)\.
- \[43\]J\. Bruna, W\. Zaremba, A\. Szlam, and Y\. LeCun\(2013\)Spectral networks and locally connected networks on graphs\.arXiv:1312\.6203\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S1.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.1)\.
- \[44\]D\. Bui, K\. Malik, J\. Goetz, H\. Liu, S\. Moon, A\. Kumar, and K\. G\. Shin\(2019\)Federated user representation learning\.arXiv:1909\.12535\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[45\]M\. Cao, D\. A\. Spielman, and E\. M\. Yeh\(2006\)Accelerated gossip algorithms for distributed computation\.InThe Annual Allerton Conference on Communication, Control, and Computation,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[46\]S\. Casas, C\. Gulino, R\. Liao, and R\. Urtasun\(2020\)Spagnn: spatially\-aware graph neural networks for relational behavior forecasting from sensor data\.InIEEE International Conference on Robotics and Automation,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1)\.
- \[47\]T\. Castiglia, A\. Das, and S\. Patterson\(2021\)Multi\-level local SGD for heterogeneous hierarchical networks\.InInternational Conference on Learning Representations,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p2.1)\.
- \[48\]E\. Ceyani, H\. Xie, B\. Buyukates, C\. Yang, and S\. Avestimehr\(2025\)FedGrAINS: personalized subgraph federated learning with adaptive neighbor sampling\.InSIAM International Conference on Data Mining,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I14.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.10.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.3.1)\.
- \[49\]D\. Chai, L\. Wang, K\. Chen, and Q\. Yang\(2020\)Secure federated matrix factorization\.IEEE Intelligent Systems36\(5\),pp\. 11–20\.Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px1.p1.1)\.
- \[50\]Z\. Chai, A\. Ali, S\. Zawad, S\. Truex, A\. Anwar, N\. Baracaldo, Y\. Zhou, H\. Ludwig, F\. Yan, and Y\. Cheng\(2020\)Tifl: a tier\-based federated learning system\.InInternational Symposium on High\-Performance Parallel and Distributed Computing,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I7.i1.p1.1)\.
- \[51\]Z\. Chai, Y\. Chen, A\. Anwar, L\. Zhao, Y\. Cheng, and H\. Rangwala\(2021\)FedAT: a high\-performance and communication\-efficient federated learning system with asynchronous tiers\.InInternational Conference for High Performance Computing, Networking, Storage and Analysis,Cited by:[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px1.p1.1)\.
- \[52\]T\. H\. Chan, E\. Shi, and D\. Song\(2011\)Private and continual release of statistics\.ACM Transactions on Information and System Security14\(3\),pp\. 1–24\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1)\.
- \[53\]C\. Chen, J\. Zhou, L\. Zheng, H\. Wu, L\. Lyu, J\. Wu, B\. Wu, Z\. Liu, L\. Wang, and X\. Zheng\(2021\)Vertically federated graph neural network for privacy\-preserving node classification\.InInternational Joint Conference on Artificial Intelligence,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I27.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I33.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.4)\.
- \[54\]C\. Chen, Z\. Xu, W\. Hu, Z\. Zheng, and J\. Zhang\(2024\)FedGL: federated graph learning framework with global self\-supervision\.Information Sciences657,pp\. 119976\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I12.i3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.2.1)\.
- \[55\]D\. Chen, Y\. Lin, W\. Li, P\. Li, J\. Zhou, and X\. Sun\(2020\)Measuring and relieving the over\-smoothing problem for graph neural networks from the topological view\.InAAAI Conference on Artificial Intelligence,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I30.i3.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1)\.
- \[56\]F\. Chen, P\. Li, T\. Miyazaki, and C\. Wu\(2021\)Fedgraph: federated graph learning with intelligent sampling\.IEEE Transactions on Parallel and Distributed Systems33\(8\),pp\. 1775–1786\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I25.i1.p1.1),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px2.p5.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.2)\.
- \[57\]F\. Chen, G\. Long, Z\. Wu, T\. Zhou, and J\. Jiang\(2022\)Personalized federated learning with a graph\.InInternational Joint Conference on Artificial Intelligence,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1)\.
- \[58\]J\. Chen, T\. Liao, C\. Chen, and Z\. Zheng\(2024\)Improving message\-passing GNNs by asynchronous aggregation\.InACM International Conference on Information and Knowledge Management,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I30.i3.p1.1),[§3\.5\.2](https://arxiv.org/html/2609.02984#S3.SS5.SSS2.Px2.p2.1)\.
- \[59\]J\. Chen and A\. Zhang\(2022\)Fedmsplit: correlation\-adaptive federated multi\-task learning across multimodal split networks\.InACM SIGKDD Conference on Knowledge Discovery and Data Mining,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I9.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I18.i2.p1.1)\.
- \[60\]J\. Chen, H\. Yang, Z\. Liu, and C\. Joe\-Wong\(2025\)The five Ws of multi\-agent communication: who talks to whom, when, what, and why\-a survey from MARL to emergent language and LLMs\.Authorea Preprints\.Cited by:[§1\.1](https://arxiv.org/html/2609.02984#S1.SS1.p1.2)\.
- \[61\]J\. Chen, G\. Huang, H\. Zheng, S\. Yu, W\. Jiang, and C\. Cui\(2022\)Graph\-fraudster: adversarial attacks on graph neural network\-based vertical federated learning\.IEEE Transactions on Computational Social Systems10\(2\),pp\. 492–506\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I25.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I31.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.4)\.
- \[62\]J\. Chen, X\. Zhang, and H\. Zheng\(2024\)Adversarial attacks on GNN\-based vertical federated learning\.InAttacks, Defenses and Testing for Deep Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I36.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I8.i2.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.4)\.
- \[63\]M\. Chen, W\. Zhang, Z\. Yuan, Y\. Jia, and H\. Chen\(2021\)Fede: embedding knowledge graphs in federated setting\.InInternational Joint Conference on Knowledge Graphs,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I20.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I24.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1),[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.7),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.4.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.2.1)\.
- \[64\]M\. Chen, W\. Zhang, Z\. Yuan, Y\. Jia, and H\. Chen\(2022\)Federated knowledge graph completion via embedding\-contrastive learning\.Knowledge\-Based Systems252,pp\. 109459\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I14.i3.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I24.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I28.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I8.i3.p1.1),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px2.p5.2),[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.8),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.8),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.3.1)\.
- \[65\]M\. Chen, H\. V\. Poor, W\. Saad, and S\. Cui\(2020\)Convergence time optimization for federated learning over wireless networks\.IEEE Transactions on Wireless Communications20,pp\. 2457–2471\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px3.p1.1)\.
- \[66\]A\. Cheu, A\. Smith, J\. Ullman, D\. Zeber, and M\. Zhilyaev\(2019\)Distributed differential privacy via shuffling\.InAnnual International Conference on the Theory and Applications of Cryptographic Techniques,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p1.1)\.
- \[67\]T\. Cheung, W\. Dai, and S\. Li\(2021\)Fedsgc: federated simple graph convolution for node classification\.InInternational Workshop on Federated and Transfer Learning for Data Sparsity and Confidentiality,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I8.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.5),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.4.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.5)\.
- \[68\]E\. Chien, J\. Peng, P\. Li, and O\. Milenkovic\(2021\)Adaptive universal generalized pagerank graph neural network\.InInternational Conference on Learning Representations,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.3)\.
- \[69\]Y\. J\. Cho, P\. Sharma, G\. Joshi, Z\. Xu, S\. Kale, and T\. Zhang\(2023\)On the convergence of federated averaging with cyclic client participation\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[70\]Y\. J\. Cho, J\. Wang, and G\. Joshi\(2020\)Client selection in federated learning: convergence analysis and power\-of\-choice selection strategies\.arXiv:2010\.01243\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I7.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[71\]W\. Choi, M\. Tomei, J\. R\. S\. Vicarte, P\. K\. Hanumolu, and R\. Kumar\(2018\)Guaranteeing local differential privacy on ultra\-low\-power systems\.InACM/IEEE Annual International Symposium on Computer Architecture,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1)\.
- \[72\]C\. Chuang and S\. Jegelka\(2022\)Tree mover’s distance: bridging graph metrics and stability of graph neural networks\.Advances in Neural Information Processing Systems35,pp\. 2944–2957\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I16.i2.p1.1)\.
- \[73\]L\. Collins, H\. Hassani, A\. Mokhtari, and S\. Shakkottai\(2021\)Exploiting shared representations for personalized federated learning\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[74\]J\. Cortés\(2006\)Finite\-time convergent gradient flows with applications to network consensus\.Automatica42\(11\),pp\. 1993–2000\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[75\]E\. Cyffers, A\. Bellet, and J\. Upadhyay\(2024\)Differentially private decentralized learning with random walks\.InInternational Conference on Machine Learning,Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p2.1)\.
- \[76\]H\. Dai, H\. Li, T\. Tian, X\. Huang, L\. Wang, J\. Zhu, and L\. Song\(2018\)Adversarial attack on graph structured data\.InInternational Conference on Machine Learning,Cited by:[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p2.1)\.
- \[77\]R\. Dai, L\. Shen, F\. He, X\. Tian, and D\. Tao\(2022\)DisPFL: towards communication\-efficient personalized federated learning via decentralized sparse training\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[78\]K\. Daly, H\. Eichner, P\. Kairouz, H\. B\. McMahan, D\. Ramage, and Z\. Xu\(2024\)Federated learning in practice: reflections and projections\.InIEEE International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications,Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.13.2),[§1](https://arxiv.org/html/2609.02984#S1.p7.1),[§2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p2.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I36.i2.p1.1)\.
- \[79\]J\. Dean, G\. Corrado, R\. Monga, K\. Chen, M\. Devin, M\. Mao, M\. Ranzato, A\. Senior, P\. Tucker, K\. Yang, Q\. V\. Le, and A\. Y\. Ng\(2012\)Large scale distributed deep networks\.Advances in Neural Information Processing Systems25\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p4.1)\.
- \[80\]M\. Defferrard, X\. Bresson, and P\. Vandergheynst\(2016\)Convolutional neural networks on graphs with fast localized spectral filtering\.InAdvances in Neural Information Processing Systems,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.3)\.
- \[81\]M\. H\. DeGroot\(1974\)Reaching a consensus\.Journal of the American Statistical Association69\(345\),pp\. 118–121\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p2.1)\.
- \[82\]P\. Denantes, F\. Bénézit, P\. Thiran, and M\. Vetterli\(2008\)Which distributed averaging algorithm should i choose for my sensor network?\.InIEEE Infocom Conference on Computer Communications,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p2.1)\.
- \[83\]Y\. Deng, W\. Chen, J\. Ren, F\. Lyu, Y\. Liu, Y\. Liu, and Y\. Zhang\(2022\)TailorFL: dual\-personalized federated learning under system and data heterogeneity\.InACM Conference on Embedded Networked Sensor Systems,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[84\]Y\. Deng, M\. M\. Kamani, and M\. Mahdavi\(2020\)Adaptive personalized federated learning\.arXiv:2003\.13461\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[85\]F\. Di Giovanni, L\. Giusti, F\. Barbero, G\. Luise, P\. Lio, and M\. M\. Bronstein\(2023\)On over\-squashing in message passing neural networks: the impact of width, depth, and topology\.InInternational Conference on Machine Learning,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I16.i3.p1.1)\.
- \[86\]E\. Diao, J\. Ding, and V\. Tarokh\(2021\)HeteroFL: computation and communication efficient federated learning for heterogeneous clients\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[87\]A\. D\. Dimakis, A\. D\. Sarwate, and M\. J\. Wainwright\(2008\)Geographic gossip: efficient averaging for sensor networks\.IEEE Transactions on Signal Processing56\(3\),pp\. 1205–1216\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[88\]B\. Ding, J\. Kulkarni, and S\. Yekhanin\(2017\)Collecting telemetry data privately\.Advances in Neural Information Processing Systems30\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p3.1)\.
- \[89\]G\. Dong, M\. Tang, L\. Cai, L\. E\. Barnes, and M\. Boukhechba\(2021\)Semi\-supervised graph instance transformer for mental health inference\.InIEEE International Conference on Machine Learning and Applications,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I6.i2.p1.2),[§3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.20.11),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.11)\.
- \[90\]G\. Dong, M\. Tang, Z\. Wang, J\. Gao, S\. Guo, L\. Cai, R\. Gutierrez, B\. Campbel, L\. E\. Barnes, and M\. Boukhechba\(2023\)Graph neural networks in IoT: a survey\.ACM Transactions on Sensor Networks19\(2\),pp\. 1–50\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.18.2),[§1](https://arxiv.org/html/2609.02984#S1.p6.1)\.
- \[91\]B\. Du and C\. Wu\(2022\)Federated graph learning with periodic neighbour sampling\.InIEEE/ACM International Symposium on Quality of Service,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I24.i2.p1.1),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px2.p5.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.3)\.
- \[92\]M\. Duan, D\. Liu, X\. Chen, R\. Liu, Y\. Tan, and L\. Liang\(2020\)Self\-balancing federated learning with global imbalanced data in mobile systems\.IEEE Transactions on Parallel and Distributed Systems32\(1\),pp\. 59–71\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I6.i1.p1.1)\.
- \[93\]M\. Duan, D\. Liu, X\. Ji, R\. Liu, L\. Liang, X\. Chen, and Y\. Tan\(2021\)Fedgroup: efficient federated learning via decomposed similarity\-based clustering\.InIEEE Internatioanal Conference on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I8.i3.p1.1)\.
- \[94\]J\. C\. Duchi, M\. I\. Jordan, and M\. J\. Wainwright\(2013\)Local privacy and statistical minimax rates\.InIEEE Annual Symposium on Foundations of Computer Science,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1)\.
- \[95\]J\. C\. Duchi, M\. I\. Jordan, and M\. J\. Wainwright\(2018\)Minimax optimal procedures for locally private estimation\.Journal of the American Statistical Association113\(521\),pp\. 182–201\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.2)\.
- \[96\]J\. Duchi, E\. Hazan, and Y\. Singer\(2011\)Adaptive subgradient methods for online learning and stochastic optimization\.\.Journal of Machine Learning Research12\(7\)\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1)\.
- \[97\]J\. Duchi and R\. Rogers\(2019\)Lower bounds for locally private estimation via communication complexity\.InConference on Learning Theory,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1)\.
- \[98\]V\. Duddu, A\. Boutet, and V\. Shejwalkar\(2020\)Quantifying privacy leakage in graph embedding\.InEAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services,Cited by:[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p1.1)\.
- \[99\]C\. T\. Duong, T\. D\. Hoang, H\. T\. H\. Dang, Q\. V\. H\. Nguyen, and K\. Aberer\(2019\)On node features for graph neural networks\.arXiv:1911\.08795\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I10.i1.p1.1)\.
- \[100\]C\. Dwork, F\. McSherry, K\. Nissim, and A\. Smith\(2006\)Calibrating noise to sensitivity in private data analysis\.InTheory of Cryptography,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I14.i3.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p1.1)\.
- \[101\]C\. Dwork, M\. Naor, T\. Pitassi, and G\. N\. Rothblum\(2010\)Differential privacy under continual observation\.InACM Symposium on Theory of Computing,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1)\.
- \[102\]C\. Dwork and A\. Roth\(2014\)The algorithmic foundations of differential privacy\.Foundations and Trends® in Theoretical Computer Science9\(3–4\),pp\. 211–407\.Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.2),[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1)\.
- \[103\]C\. Dwork\(2006\)Differential privacy\.InInternational Colloquium on Automata, Languages, and Programming,Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.1)\.
- \[104\]A\. El Mrini, E\. Cyffers, and A\. Bellet\(2024\)Privacy attacks in decentralized learning\.InInternational Conference on Machine Learning,Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p2.1)\.
- \[105\]Ú\. Erlingsson, V\. Feldman, I\. Mironov, A\. Raghunathan, K\. Talwar, and A\. Thakurta\(2019\)Amplification by shuffling: from local to central differential privacy via anonymity\.InAnnual ACM\-SIAM Symposium on Discrete Algorithms,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p1.1)\.
- \[106\]European Union\(2016\)General data protection regulation\.Technical reportEuropean Union\.Note:Regulation \(EU\) 2016/679Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p2.1),[§2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4.p1.1)\.
- \[107\]M\. Even, A\. Koloskova, and L\. Massoulie\(2024\)Asynchronous SGD on graphs: a unified framework for asynchronous decentralized and federated optimization\.InInternational Conference on Artificial Intelligence and Statistics,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I3.i1.p1.1),[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px2.p1.1)\.
- \[108\]A\. Evfimievski, J\. Gehrke, and R\. Srikant\(2003\)Limiting privacy breaches in privacy preserving data mining\.InACM SIGMOD\-SIGACT\-SIGART Symposium on Principles of Database Systems,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p1.1)\.
- \[109\]L\. Faber, Y\. Lu, and R\. Wattenhofer\(2021\)Should graph neural networks use features, edges, or both?\.arXiv:2103\.06857\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I10.i1.p1.1)\.
- \[110\]L\. Faber and R\. Wattenhofer\(2024\)GwAC: GNNs with asynchronous communication\.InLearning on Graphs Conference,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I30.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.32.3.2.1.1.1)\.
- \[111\]A\. Fallah, A\. Mokhtari, and A\. Ozdaglar\(2020\)Personalized federated learning: a meta\-learning approach\.arXiv:2002\.07948\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1)\.
- \[112\]Z\. Fan, H\. Fang, and M\. P\. Friedlander\(2022\)A dual approach for federated learning\.arXiv:2201\.11183\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I5.i1.p1.1)\.
- \[113\]H\. Fang, Y\. Gao, P\. Zhang, J\. Yao, H\. Chen, and H\. Wang\(2025\)Large language models enhanced personalized graph neural architecture search in federated learning\.InAAAI Conference on Artificial Intelligence,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I17.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.10.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.2.1)\.
- \[114\]B\. Fatemi, L\. El Asri, and S\. M\. Kazemi\(2021\)SLAPS: self\-supervision improves structure learning for graph neural networks\.Advances in Neural Information Processing Systems34,pp\. 22667–22681\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I16.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I29.i1.p1.1)\.
- \[115\]J\. A\. Fax and R\. M\. Murray\(2004\)Information flow and cooperative control of vehicle formations\.IEEE Transactions on Automatic Control49\(9\),pp\. 1465–1476\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[116\]S\. Feng, B\. Li, H\. Yu, Y\. Liu, and Q\. Yang\(2022\)Semi\-supervised federated heterogeneous transfer learning\.Knowledge\-Based Systems252,pp\. 109384\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I11.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I12.i2.p1.1)\.
- \[117\]S\. Feng\(2022\)Vertical federated learning\-based feature selection with non\-overlapping sample utilization\.Expert Systems with Applications208,pp\. 118097\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I11.i3.p1.1)\.
- \[118\]M\. Fey, J\. E\. Lenssen, F\. Weichert, and J\. Leskovec\(2021\)GNNAutoscale: scalable and expressive graph neural networks via historical embeddings\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I30.i1.p1.1)\.
- \[119\]M\. Fey, J\. Sunil, A\. Nitta, R\. Puri, M\. Shah, B\. Stojanovič, R\. Bendias, A\. Barghi, V\. Kocijan, Z\. Zhang, X\. He, J\. E\. Lenssen, and J\. Leskovec\(2025\)PyG 2\.0: scalable learning on real world graphs\.InKDD Worshop on Temporal Graph Learning,Cited by:[footnote 5](https://arxiv.org/html/2609.02984#footnote5)\.
- \[120\]M\. Fiedler\(1973\)Algebraic connectivity of graphs\.Czechoslovak Mathematical Journal23\(2\),pp\. 298–305\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[121\]B\. Finkelshtein, X\. Huang, M\. Bronstein, and İ\. İ\. Ceylan\(2024\)Cooperative graph neural networks\.InInternational Conference on Machine Learning,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I30.i3.p1.1)\.
- \[122\]C\. Finn, P\. Abbeel, and S\. Levine\(2017\)Model\-agnostic meta\-learning for fast adaptation of deep networks\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[123\]S\. Fiorellino, C\. Battiloro, and P\. Di Lorenzo\(2024\)Topological neural networks over the air\.InIEEE International Conference on Acoustics, Speech and Signal Processing,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I32.i2.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.35.3.2.1.2.1)\.
- \[124\]F\. Frasca, F\. Jogl, M\. Eliasof, M\. Ostrovsky, C\. Schönlieb, T\. Gärtner, and H\. Maron\(2024\)Towards foundation models on graphs: an analysis on cross\-dataset transfer of pretrained GNNs\.InNEURIPS Workshop on Symmetry and Geometry in Neural Representations,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I21.i1.p1.1)\.
- \[125\]M\. Fredrikson, S\. Jha, and T\. Ristenpart\(2015\)Model inversion attacks that exploit confidence information and basic countermeasures\.InACM SIGSAC Conference on Computer and Communications Security,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I4.i1.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px1.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.1)\.
- \[126\]X\. Fu, S\. Wang, Y\. Dong, B\. Zhang, C\. Chen, and J\. Li\(2024\)Federated graph learning with graphless clients\.Transactions on Machine Learning Research\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.17.2.2.1.2.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I11.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I18.i3.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I8.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.4)\.
- \[127\]D\. Fuchsgruber, T\. Wollschläger, J\. Bordne, and S\. Günnemann\(2025\)Uncertainty estimation for heterophilic graphs through the lens of information theory\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1)\.
- \[128\]H\. Gao, M\. Lee, G\. Yu, and Z\. Zhou\(2022\)A graph neural network based decentralized learning scheme\.Sensors22\(3\),pp\. 1030\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I27.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I28.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.2.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.32.3.2.1.1.1)\.
- \[129\]R\. Gao, Z\. Liu, C\. Jiang, Y\. Wang, S\. Wang, and P\. Wang\(2024\)BI\-fedGNN: federated graph neural networks framework based on bayesian inference\.Neural Networks169,pp\. 143–153\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I13.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.3.1)\.
- \[130\]Z\. Gao and D\. Gündüz\(2023\)Airgnns: graph neural networks over the air\.InIEEE European Signal Processing Conference,Cited by:[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.35.3.2.1.1.1)\.
- \[131\]Z\. Gao and D\. Gündüz\(2025\)Graph neural networks over the air for decentralized tasks in wireless networks\.IEEE Transactions on Signal Processing\.Cited by:[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.35.3.2.1.1.1)\.
- \[132\]Z\. Gao, E\. Isufi, and A\. Ribeiro\(2021\)Stochastic graph neural networks\.IEEE Transactions on Signal Processing69,pp\. 4428–4443\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I32.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.34.3)\.
- \[133\]Z\. Gao, Y\. Shao, D\. Gündüz, and A\. Prorok\(2023\)Decentralized channel management in WLANs with graph neural networks\.InIEEE International Conference on Communications,Cited by:[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.4.1)\.
- \[134\]J\. Gasteiger, A\. Bojchevski, and S\. Günnemann\(2019\)Predict then propagate: graph neural networks meet personalized pagerank\.InInternational Conference on Learning Representations,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.3)\.
- \[135\]J\. Geiping, H\. Bauermeister, H\. Dröge, and M\. Moeller\(2020\)Inverting gradients \- how easy is it to break privacy in federated learning?\.InAdvances in Neural Information Processing Systems,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I4.i2.p1.1)\.
- \[136\]R\. C\. Geyer, T\. Klein, and M\. Nabi\(2017\)Differentially private federated learning: a client level perspective\.InNEURIPS Workshop on Machine Learning on the Phone and Other Consumer Devices,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1)\.
- \[137\]A\. Ghosh, J\. Chung, D\. Yin, and K\. Ramchandran\(2020\)An efficient framework for clustered federated learning\.Advances in Neural Information Processing Systems33,pp\. 19586–19597\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I8.i3.p1.1)\.
- \[138\]L\. Giaretta and S\. Girdzijauskas\(2023\)Fully\-decentralized training of GNNs using layer\-wise self\-supervision\.diva2:1745180\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I13.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I22.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I27.i3.p1.1),[§3\.5\.1](https://arxiv.org/html/2609.02984#S3.SS5.SSS1.p4.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.2.1)\.
- \[139\]J\. Gilmer, S\. S\. Schoenholz, P\. F\. Riley, O\. Vinyals, and G\. E\. Dahl\(2017\)Neural message passing for quantum chemistry\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p8.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p3.1)\.
- \[140\]J\. Goetz, K\. Malik, D\. Bui, S\. Moon, H\. Liu, and A\. Kumar\(2019\)Active federated learning\.Technical reportUniversity of Michigan\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px3.p1.1)\.
- \[141\]S\. Goldwasser, S\. Micali, and C\. Rackoff\(1985\)The knowledge complexity of interactive proof\-systems\.InACM Symposium on Theory of Computing,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1)\.
- \[142\]Y\. Gong, Y\. Li, and N\. M\. Freris\(2022\)FedADMM: a robust federated deep learning framework with adaptivity to system heterogeneity\.InIEEE International Conference on Data Engineering,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I5.i1.p1.1)\.
- \[143\]I\. Goodfellow, Y\. Bengio, and A\. Courville\(2016\)Deep learning\.MIT Press\.Note:[http://www\.deeplearningbook\.org](http://www.deeplearningbook.org/)Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p1.1)\.
- \[144\]S\. Gronauer and K\. Diepold\(2022\)Multi\-agent deep reinforcement learning: a survey\.Artificiell Intelligence Review55\(2\),pp\. 895–943\.Cited by:[§1\.1](https://arxiv.org/html/2609.02984#S1.SS1.p1.2)\.
- \[145\]F\. Gu, H\. Chang, W\. Zhu, S\. Sojoudi, and L\. El Ghaoui\(2020\)Implicit graph neural networks\.Advances in Neural Information Processing Systems33,pp\. 11984–11995\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I30.i2.p1.1)\.
- \[146\]Z\. Guan, Y\. Li, Z\. Xue, Y\. Liu, H\. Gao, and Y\. Shao\(2021\)Federated graph neural network for cross\-graph node classification\.InIEEE International Conference on Cloud Computing and Intelligent Systems,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[§3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.4)\.
- \[147\]S\. Gui, X\. Li, L\. Wang, and S\. Ji\(2022\)Good: a graph out\-of\-distribution benchmark\.Advances in Neural Information Processing Systems35,pp\. 2059–2073\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I10.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I9.i2.p1.1)\.
- \[148\]J\. Guo, S\. Li, and Y\. Zhang\(2023\)An information theoretic perspective for heterogeneous subgraph federated learning\.InInternational Conference on Database Systems for Advanced Applications,Cited by:[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.3.1)\.
- \[149\]P\. Guo, S\. Zeng, Y\. Wang, H\. Fan, F\. Wang, and L\. Qu\(2025\)Selective aggregation for low\-rank adaptation in federated learning\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[150\]Z\. Guo, D\. Yao, Q\. Yang, and H\. Liu\(2024\)HiFGL: a hierarchical framework for cross\-silo cross\-device federated graph learning\.InACM SIGKDD Conference on Knowledge Discovery and Data Mining,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[4th item](https://arxiv.org/html/2609.02984#S3.I27.i4.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[§3\.5\.1](https://arxiv.org/html/2609.02984#S3.SS5.SSS1.p4.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p3.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p3.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.2.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.2.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.28.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.2)\.
- \[151\]O\. Gupta and R\. Raskar\(2018\)Distributed learning of deep neural network over multiple agents\.Journal of Network and Computer Applications116,pp\. 1–8\.Cited by:[footnote 2](https://arxiv.org/html/2609.02984#footnote2)\.
- \[152\]S\. Gupta, W\. Zhang, and F\. Wang\(2016\)Model accuracy and runtime tradeoff in distributed deep learning: a systematic study\.InIEEE International Conference on Data Mining,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p2.1)\.
- \[153\]B\. Gutteridge, X\. Dong, M\. M\. Bronstein, and F\. Di Giovanni\(2023\)Drew: dynamically rewired message passing with delay\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I16.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I29.i1.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I30.i3.p1.1),[§3\.5\.2](https://arxiv.org/html/2609.02984#S3.SS5.SSS2.Px2.p2.1)\.
- \[154\]M\. Hajij, G\. Zamzmi, T\. Papamarkou, N\. Miolane, A\. Guzmán\-Sáenz, K\. N\. Ramamurthy, T\. Birdal, T\. K\. Dey, S\. Mukherjee, S\. N\. Samaga, N\. Livesay, R\. Walters, P\. Rosen, and M\. T\. Schaub\(2022\)Topological deep learning: going beyond graph data\.arXiv:2206\.00606\.Cited by:[item Topology](https://arxiv.org/html/2609.02984#A1.I1.ix11.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I19.i1.p1.1.2)\.
- \[155\]W\. Hamilton, Z\. Ying, and J\. Leskovec\(2017\)Inductive representation learning on large graphs\.Advances in Neural Information Processing Systems30\.Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p3.2),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px4.p1.2)\.
- \[156\]W\. L\. Hamilton, R\. Ying, and J\. Leskovec\(2017\)Representation learning on graphs: methods and applications\.IEEE Data Engineering Bulletin\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p8.1)\.
- \[157\]D\. K\. Hammond, P\. Vandergheynst, and R\. Gribonval\(2011\)Wavelets on graphs via spectral graph theory\.Applied and Computational Harmonic Analysis30\(2\),pp\. 129–150\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I2.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.2)\.
- \[158\]Z\. Han, C\. Hu, T\. Li, Q\. Qi, P\. Tang, and S\. Guo\(2024\)Subgraph\-level federated graph neural network for privacy\-preserving recommendation with meta\-learning\.Neural Networks179,pp\. 106574\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p6.1),[1st item](https://arxiv.org/html/2609.02984#S3.I17.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I7.i1.I1.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I8.i3.p1.1),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px2.p5.2),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.6),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.6),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.4.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.6)\.
- \[159\]A\. Hard, K\. Rao, R\. Mathews, S\. Ramaswamy, F\. Beaufays, S\. Augenstein, H\. Eichner, C\. Kiddon, and D\. Ramage\(2018\)Federated learning for mobile keyboard prediction\.Technical reportGoogle\.Cited by:[§2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1.p1.1)\.
- \[160\]C\. He, M\. Annavaram, and S\. Avestimehr\(2020\)Group knowledge transfer: federated learning of large cnns at the edge\.Advances in Neural Information Processing Systems33,pp\. 14068–14080\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1)\.
- \[161\]C\. He, K\. Balasubramanian, E\. Ceyani, C\. Yang, H\. Xie, L\. Sun, L\. He, L\. Yang, P\. S\. Yu, Y\. Rong, P\. Zhao, J\. Huang, M\. Annavaram, and S\. Avestimehr\(2021\)Fedgraphnn: a federated learning benchmark system for graph neural networks\.InWorkshop on Distributed and Private Machine Learning,Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.17.2.2.1.1.1),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.18.2),[§1](https://arxiv.org/html/2609.02984#S1.p6.1),[1st item](https://arxiv.org/html/2609.02984#S3.I6.i1.p1.1),[§3\.3\.1](https://arxiv.org/html/2609.02984#S3.SS3.SSS1.p1.1)\.
- \[162\]C\. He, E\. Ceyani, K\. Balasubramanian, M\. Annavaram, and S\. Avestimehr\(2022\)Spreadgnn: decentralized multi\-task federated learning for graph neural networks on molecular data\.InAAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p6.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I18.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I27.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I6.i1.p1.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.10)\.
- \[163\]G\. He, Y\. Ren, J\. Jiang, G\. Feng, and X\. Zhang\(2024\)Privacy\-enhanced federated GNN inference against adversarial example attack\.IEEE Transactions on Emerging Topics in Computational Intelligence\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I36.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I8.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.2)\.
- \[164\]M\. He, Z\. Wei, and J\. Wen\(2022\)Convolutional neural networks on graphs with chebyshev approximation, revisited\.InAdvances in Neural Information Processing Systems,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.3)\.
- \[165\]M\. He, Z\. Wei, H\. Xu,et al\.\(2021\)BernNet: learning arbitrary graph spectral filters via bernstein approximation\.InAdvances in Neural Information Processing Systems,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.3)\.
- \[166\]S\. He, S\. Xiong, Y\. Ou, J\. Zhang, J\. Wang, Y\. Huang, and Y\. Zhang\(2021\)An overview on the application of graph neural networks in wireless networks\.IEEE Open Journal of the Communications Society2,pp\. 2547–2565\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.19.2)\.
- \[167\]X\. He, J\. Jia, M\. Backes, N\. Z\. Gong, and Y\. Zhang\(2021\)Stealing links from graph neural networks\.InUSENIX Security Symposium,Cited by:[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p1.1)\.
- \[168\]Y\. He, Y\. Kang, X\. Zhao, J\. Luo, L\. Fan, and Y\. Han\(2025\)A hybrid self\-supervised learning framework for vertical federated learning\.IEEE Transactions on Big Data\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I11.i3.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I6.i2.p1.1)\.
- \[169\]I\. Hegedűs, G\. Danner, and M\. Jelasity\(2019\)Gossip learning as a decentralized alternative to federated learning\.InIFIP International Conference on Distributed Applications and Interoperable Systems,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[170\]H\. Hellström, J\. M\. B\. da Silva Jr, M\. M\. Amiri, M\. Chen, V\. Fodor, H\. V\. Poor, and C\. Fischione\(2022\)Wireless for machine learning: a survey\.Foundations and Trends® in Signal Processing15\(4\),pp\. 290–399\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.15.2),[§1](https://arxiv.org/html/2609.02984#S1.p5.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I3.i2.p1.1),[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px2.p1.1),[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.p1.1)\.
- \[171\]G\. Hinton, O\. Vinyals, and J\. Dean\(2014\)Distilling the knowledge in a neural network\.InNIPS Deep Learning and Representation Learning Workshop,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1),[§3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3.Px2.p1.1)\.
- \[172\]B\. Hitaj, G\. Ateniese, and F\. Perez\-Cruz\(2017\)Deep models under the GAN: information leakage from collaborative deep learning\.InACM SIGSAC Conference on Computer and Communications Security,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I4.i3.p1.1)\.
- \[173\]P\. W\. Holland, K\. B\. Laskey, and S\. Leinhardt\(1983\)Stochastic blockmodels: first steps\.Social Networks5\(2\),pp\. 109–137\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1)\.
- \[174\]S\. Hooker\(2021\)The hardware lottery\.Communications of the ACM64\(12\),pp\. 58–65\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p1.1)\.
- \[175\]C\. Hou, K\. K\. Thekumparampil, G\. Fanti, and S\. Oh\(2022\)FedChain: chained algorithms for near\-optimal communication cost in federated learning\.InInternational Conference on Learning Representations,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p5.1)\.
- \[176\]K\. Hsieh, A\. Phanishayee, O\. Mutlu, and P\. Gibbons\(2020\)The non\-iid data quagmire of decentralized machine learning\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I2.i1.p1.1)\.
- \[177\]C\. Huang, L\. Wang, and X\. Han\(2023\)Vertical federated knowledge transfer via representation distillation for healthcare collaboration networks\.InACM Web Conference,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I12.i1.p1.1)\.
- \[178\]L\. Huang, A\. L\. Shea, H\. Qian, A\. Masurkar, H\. Deng, and D\. Liu\(2019\)Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records\.Journal of Biomedical Informatics99,pp\. 103291\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I9.i2.p1.1)\.
- \[179\]Q\. Huang, H\. He, A\. Singh, S\. Lim, and A\. R\. Benson\(2021\)Combining label propagation and simple models out\-performs graph neural networks\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I26.i2.p1.1)\.
- \[180\]T\. Huang, S\. Liu, L\. Shen, F\. He, W\. Lin, and D\. Tao\(2022\)Achieving personalized federated learning with sparse local models\.arXiv:2201\.11380\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[181\]Y\. Huang, H\. Hu, and C\. Chen\(2021\)Robustness of on\-device models: adversarial attack to deep learning models on android apps\.InIEEE/ACM International Conference on Software Engineering,Cited by:[§2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3.Px2.p1.1)\.
- \[182\]Y\. Huang, L\. Chu, Z\. Zhou, L\. Wang, J\. Liu, J\. Pei, and Y\. Zhang\(2021\)Personalized cross\-silo federated learning on non\-iid data\.InAAAI Conference on Artificial Intelligence,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I2.i2.p1.1),[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1)\.
- \[183\]D\. Huba, J\. Nguyen, K\. Malik, R\. Zhu, M\. Rabbat, A\. Yousefpour, C\. Wu, H\. Zhan, P\. Ustinov, H\. Srinivas, K\. Wang, A\. Shoumikhin, J\. Min, and M\. Malek\(2022\)PAPAYA: practical, private, and scalable federated learning\.InMachine Learning and Systems,Cited by:[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px1.p1.1)\.
- \[184\]A\. Jadbabaie, J\. Lin, and A\. S\. Morse\(2003\)Coordination of groups of mobile autonomous agents using nearest neighbor rules\.IEEE Transactions on Automatic Control48\(6\),pp\. 988–1001\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[185\]S\. Jegelka\(2022\)Theory of graph neural networks: representation and learning\.InThe International Congress of Mathematicians,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I16.i3.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px4.p1.2)\.
- \[186\]M\. Jelasity, A\. Montresor, and O\. Babaoglu\(2005\)Gossip\-based aggregation in large dynamic networks\.ACM Transactions on Computer Systems23\(3\),pp\. 219–252\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[187\]E\. Jeong, S\. Oh, H\. Kim, J\. Park, M\. Bennis, and S\. Kim\(2018\)Communication\-efficient on\-device machine learning: federated distillation and augmentation under non\-iid private data\.InNEURIPS Workshop on Machine Learning on the Phone and Other Consumer Devices,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I6.i1.p1.1)\.
- \[188\]S\. Ji, Y\. Tan, T\. Saravirta, Z\. Yang, Y\. Liu, L\. Vasankari, S\. Pan, G\. Long, and A\. Walid\(2024\)Emerging trends in federated learning: from model fusion to federated X learning\.International Journal of Machine Learning and Cybernetics15\(9\),pp\. 3769–3790\.Cited by:[§1\.1](https://arxiv.org/html/2609.02984#S1.SS1.p1.2)\.
- \[189\]F\. Jiang and L\. Wang\(2009\)Finite\-time information consensus for multi\-agent systems with fixed and switching topologies\.Physica D: Nonlinear Phenomena238\(16\),pp\. 1550–1560\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[190\]Y\. Jiang, J\. Konečný, K\. Rush, and S\. Kannan\(2019\)Improving federated learning personalization via model agnostic meta learning\.arXiv:1909\.12488\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[191\]B\. Johansson, T\. Keviczky, M\. Johansson, and K\. H\. Johansson\(2008\)Subgradient methods and consensus algorithms for solving convex optimization problems\.InIEEE Conference on Decision and Control,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[192\]P\. Kairouz, B\. McMahan, S\. Song, O\. Thakkar, A\. Thakurta, and Z\. Xu\(2021\)Practical and private \(deep\) learning without sampling or shuffling\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p1.1)\.
- \[193\]P\. Kairouz, H\. B\. McMahan, B\. Avent, A\. Bellet, M\. Bennis, A\. N\. Bhagoji, K\. Bonawitz, Z\. Charles, G\. Cormode, R\. Cummings, R\. G\. L\. D’Oliveira, H\. Eichner, S\. El Rouayheb, D\. Evans, J\. Gardner, Z\. Garrett, A\. Gascón, B\. Ghazi, P\. B\. Gibbons, M\. Gruteser, Z\. Harchaoui, C\. He, L\. He, Z\. Huo, B\. Hutchinson, J\. Hsu, M\. Jaggi, T\. Javidi, G\. Joshi, M\. Khodak, J\. Konečný, A\. Korolova, F\. Koushanfar, S\. Koyejo, T\. Lepoint, Y\. Liu, P\. Mittal, M\. Mohri, R\. Nock, A\. Özgür, R\. Pagh, M\. Raykova, H\. Qi, D\. Ramage, R\. Raskar, D\. Song, W\. Song, S\. U\. Stich, Z\. Sun, A\. T\. Suresh, F\. Tramèr, P\. Vepakomma, J\. Wang, L\. Xiong, Z\. Xu, Q\. Yang, F\. X\. Yu, H\. Yu, and S\. Zhao\(2021\)Advances and open problems in federated learning\.Foundations and Trends® in Machine Learning14\(1–2\),pp\. 1–210\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.9.2.4),[§1](https://arxiv.org/html/2609.02984#S1.p7.1),[1st item](https://arxiv.org/html/2609.02984#S2.I1.i1.p1.2),[1st item](https://arxiv.org/html/2609.02984#S2.I2.i1.p1.1),[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px2.p1.1),[§2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2.Px2.p1.1),[§2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2.p1.1),[§2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1.Px1.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.02984#S2.p2.1)\.
- \[194\]H\. Kang, S\. Cha, J\. Shin, J\. Lee, and J\. Kang\(2025\)NeFL: nested model scaling for federated learning with system heterogeneous clients\.IEEE Transactions on Mobile Computing24\(8\),pp\. 6734–6746\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[195\]Y\. Kang, Y\. He, J\. Luo, T\. Fan, Y\. Liu, and Q\. Yang\(2022\)Privacy\-preserving federated adversarial domain adaptation over feature groups for interpretability\.IEEE Transactions on Big Data10\(6\),pp\. 879–890\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I12.i2.p1.1)\.
- \[196\]Y\. Kang, Y\. Liu, and X\. Liang\(2022\)FedCVT: semi\-supervised vertical federated learning with cross\-view training\.ACM Transactions on Intelligent Systems and Technology13\(4\),pp\. 1–16\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I6.i3.p1.1)\.
- \[197\]S\. P\. Karimireddy, S\. Kale, M\. Mohri, S\. Reddi, S\. Stich, and A\. T\. Suresh\(2020\)Scaffold: stochastic controlled averaging for federated learning\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I5.i1.p1.1),[§2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1.p1.1)\.
- \[198\]R\. Karp, C\. Schindelhauer, S\. Shenker, and B\. Vocking\(2000\)Randomized rumor spreading\.InIEEE Annual Symposium on Foundations of Computer Science,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[199\]B\. Karrer and M\. E\. Newman\(2011\)Stochastic blockmodels and community structure in networks\.Physical Review E83\(1\),pp\. 016107\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1)\.
- \[200\]S\. P\. Kasiviswanathan, H\. K\. Lee, K\. Nissim, S\. Raskhodnikova, and A\. Smith\(2011\)What can we learn privately?\.SIAM Journal on Computing40\(3\),pp\. 793–826\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I14.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1)\.
- \[201\]A\. Kazi, L\. Cosmo, S\. Ahmadi, N\. Navab, and M\. M\. Bronstein\(2022\)Differentiable graph module \(dgm\) for graph convolutional networks\.IEEE Transactions on Pattern Analysis and Machine Intelligence45\(2\),pp\. 1606–1617\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I16.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I29.i1.p1.1)\.
- \[202\]N\. Keriven, A\. Bietti, and S\. Vaiter\(2020\)Convergence and stability of graph convolutional networks on large random graphs\.Advances in Neural Information Processing Systems33,pp\. 21512–21523\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I27.i1.p1.1)\.
- \[203\]A\. Khaled, K\. Mishchenko, and P\. Richtárik\(2020\)Tighter theory for local SGD on identical and heterogeneous data\.InInternational Conference on Artificial Intelligence and Statistics,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p3.1)\.
- \[204\]P\. Khanduri, P\. Sharma, H\. Yang, M\. Hong, J\. Liu, K\. Rajawat, and P\. Varshney\(2021\)Stem: a stochastic two\-sided momentum algorithm achieving near\-optimal sample and communication complexities for federated learning\.Advances in Neural Information Processing Systems34,pp\. 6050–6061\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I5.i1.p1.1)\.
- \[205\]G\. Kim, J\. Kim, and B\. Han\(2024\)Communication\-efficient federated learning with accelerated client gradient\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I5.i1.p1.1)\.
- \[206\]H\. Kim, Y\. Kwak, M\. Jung, J\. Shin, Y\. Kim, and C\. Kim\(2023\)ProtoFL: unsupervised federated learning via prototypical distillation\.InIEEE/CVF International Conference on Computer Vision,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1)\.
- \[207\]S\. Kim, Y\. Lee, Y\. Oh, N\. Lee, S\. Yun, J\. Lee, S\. Kim, C\. Yang, and C\. Park\(2025\)Subgraph federated learning for local generalization\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I14.i2.p1.1),[4th item](https://arxiv.org/html/2609.02984#S3.I16.i4.p1.1),[§3\.3\.2](https://arxiv.org/html/2609.02984#S3.SS3.SSS2.Px2.p2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.10.2.1.1.1)\.
- \[208\]T\. N\. Kipf and M\. Welling\(2017\)Semi\-supervised classification with graph convolutional networks\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S1.I1.i2.p1.1),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.6.2),[1st item](https://arxiv.org/html/2609.02984#S3.I19.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I2.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I26.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p3.2)\.
- \[209\]B\. Klein, L\. Wolf, and Y\. Afek\(2015\)A dynamic convolutional layer for short range weather prediction\.InIEEE Conference on Computer Vision and Pattern Recognition,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I8.i2.p1.1)\.
- \[210\]J\. Konečný, H\. B\. McMahan, F\. X\. Yu, P\. Richtárik, A\. T\. Suresh, and D\. Bacon\(2016\)Federated learning: strategies for improving communication efficiency\.NIPS Workshop on Private Multi\-Party Machine Learning\.Cited by:[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px2.p1.1)\.
- \[211\]E\. Krasanakis, S\. Papadopoulos, and I\. Kompatsiaris\(2022\)p2pGNN: a decentralized graph neural network for node classification in peer\-to\-peer networks\.IEEE Access10,pp\. 34755–34765\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I26.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I28.i1.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I29.i3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.4.1)\.
- \[212\]L\. Kummer, S\. Moustafa, A\. Ehrlich, F\. Bause, N\. Suess, W\. N\. Gansterer, and N\. M\. Kriege\(2025\)Weisfeiler and leman go gambling: why expressive lottery tickets win\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I19.i2.p1.1)\.
- \[213\]G\. Lee, M\. Jeong, Y\. Shin, S\. Bae, and S\. Yun\(2022\)Preservation of the global knowledge by not\-true distillation in federated learning\.InAdvances in Neural Information Processing Systems,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1)\.
- \[214\]M\. Lee, G\. Yu, H\. Dai, and G\. Y\. Li\(2022\)Graph neural networks meet wireless communications: motivation, applications, and future directions\.IEEE Wireless Communications29\(5\),pp\. 12–19\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.19.2)\.
- \[215\]M\. Lee, G\. Yu, and H\. Dai\(2021\)Decentralized inference with graph neural networks in wireless communication systems\.IEEE Transactions on Mobile Computing22\(5\),pp\. 2582–2598\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I32.i1.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.34.3)\.
- \[216\]M\. Lee, G\. Yu, and H\. Dai\(2023\)Privacy\-preserving decentralized inference with graph neural networks in wireless networks\.IEEE Transactions on Wireless Communications23\(1\),pp\. 543–558\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.35.3.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.3)\.
- \[217\]R\. Lei, P\. Wang, J\. Zhao, L\. Lan, J\. Tao, C\. Deng, J\. Feng, X\. Wang, and X\. Guan\(2023\)Federated learning over coupled graphs\.IEEE Transactions on Parallel and Distributed Systems34\(4\),pp\. 1159–1172\.Cited by:[4th item](https://arxiv.org/html/2609.02984#S3.I18.i4.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I19.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I26.i1.p1.1),[§3\.5\.1](https://arxiv.org/html/2609.02984#S3.SS5.SSS1.p4.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.2.2.1.1.1)\.
- \[218\]J\. Leng, Z\. Lin, M\. Ding, P\. Wang, D\. Smith, and B\. Vucetic\(2022\)Client scheduling in wireless federated learning based on channel and learning qualities\.IEEE Wireless Communications Letters11\(4\),pp\. 732–735\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px3.p1.1)\.
- \[219\]A\. Li, J\. Sun, X\. Zeng, M\. Zhang, H\. Li, and Y\. Chen\(2021\)Fedmask: joint computation and communication\-efficient personalized federated learning via heterogeneous masking\.InACM Conference on Embedded Networked Sensor Systems,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[220\]D\. Li and J\. Wang\(2019\)FedMD: heterogenous federated learning via model distillation\.InNEURIPS Workshop on Federated Learning for Data Privacy and Confidentiality,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1)\.
- \[221\]L\. Li, M\. Duan, D\. Liu, Y\. Zhang, A\. Ren, X\. Chen, Y\. Tan, and C\. Wang\(2021\)FedSAE: a novel self\-adaptive federated learning framework in heterogeneous systems\.InIEEE International Joint Conference on Neural Networks,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I7.i1.p1.1)\.
- \[222\]L\. Li, Y\. Fan, M\. Tse, and K\. Lin\(2020\)A review of applications in federated learning\.Computers & Industrial Engineering149,pp\. 106854\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p5.1)\.
- \[223\]Q\. Li, Y\. Diao, Q\. Chen, and B\. He\(2022\)Federated learning on non\-iid data silos: an experimental study\.InIEEE International Conference on Data Engineering,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I2.i1.p1.1),[§2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1.p1.1)\.
- \[224\]Q\. Li, B\. He, and D\. Song\(2021\)Model\-contrastive federated learning\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1)\.
- \[225\]T\. Li, A\. K\. Sahu, M\. Zaheer, M\. Sanjabi, A\. Talwalkar, and V\. Smith\(2020\)Federated optimization in heterogeneous networks\.Proceedings of Machine Learning and Systems2,pp\. 429–450\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1),[§2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1.Px1.p1.1)\.
- \[226\]W\. Li, Q\. Xia, J\. Deng, H\. Cheng, J\. Liu, K\. Xue, Y\. Cheng, and S\. Xia\(2022\)VFed\-SSD: towards practical vertical federated advertising\.InTrustworthy Federated Learning Workshop at the International Joint Conference on Artificial Intelligence,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I11.i3.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I6.i2.p1.1)\.
- \[227\]X\. Li, K\. Huang, W\. Yang, S\. Wang, and Z\. Zhang\(2020\)On the convergence of FedAvg on non\-iid data\.InInternational Conference on Learning Representations,Cited by:[§2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1.Px1.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p3.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p4.1)\.
- \[228\]X\. Li, Z\. Qu, B\. Tang, and Z\. Lu\(2022\)Stragglers are not disasters: a hybrid federated learning framework with delayed gradients\.InIEEE International Conference on Machine Learning and Applications,Cited by:[§2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3.Px1.p1.2),[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px1.p1.1)\.
- \[229\]X\. Lian, W\. Zhang, C\. Zhang, and J\. Liu\(2018\)Asynchronous decentralized parallel stochastic gradient descent\.InInternational Conference on Machine Learning,Cited by:[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px2.p1.1),[§3\.5\.2](https://arxiv.org/html/2609.02984#S3.SS5.SSS2.Px1.p1.1)\.
- \[230\]G\. Liang and S\. S\. Chawathe\(2004\)Privacy\-preserving inter\-database operations\.InInternational Conference on Intelligence and Security Informatics,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I10.i1.p1.1)\.
- \[231\]P\. P\. Liang, T\. Liu, L\. Ziyin, N\. B\. Allen, R\. P\. Auerbach, D\. Brent, R\. Salakhutdinov, and L\. Morency\(2020\)Think locally, act globally: federated learning with local and global representations\.arXiv:2001\.01523\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[232\]W\. Liang, Y\. Zhao, R\. She, Y\. Li, and W\. P\. Tay\(2024\)Fedsheafhn: personalized federated learning on graph\-structured data\.arXiv:2405\.16056\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I17.i2.p1.1),[4th item](https://arxiv.org/html/2609.02984#S3.I17.i4.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.2)\.
- \[233\]H\. Lin, M\. Yan, X\. Ye, D\. Fan, S\. Pan, W\. Chen, and Y\. Xie\(2023\)A comprehensive survey on distributed training of graph neural networks\.Proceedings of the IEEE\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.20.2)\.
- \[234\]T\. Lin, L\. Kong, S\. U\. Stich, and M\. Jaggi\(2020\)Ensemble distillation for robust model fusion in federated learning\.Advances in Neural Information Processing Systems33,pp\. 2351–2363\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1)\.
- \[235\]W\. Lin, B\. Li, and C\. Wang\(2022\)Towards private learning on decentralized graphs with local differential privacy\.IEEE Transactions on Information Forensics and Security17,pp\. 2936–2946\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I13.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.3)\.
- \[236\]D\. Liu, G\. Zhu, J\. Zhang, and K\. Huang\(2019\)Wireless data acquisition for edge learning: importance\-aware retransmission\.InIEEE International Workshop on Signal Processing Advances in Wireless Communications,Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px1.p1.1)\.
- \[237\]D\. Liu, G\. Zhu, J\. Zhang, and K\. Huang\(2021\)Data\-importance aware user scheduling for communication\-efficient edge machine learning\.IEEE Transactions on Cognitive Communications and Networking7\(1\),pp\. 265–278\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px3.p1.1)\.
- \[238\]F\. Liu, Z\. Zheng, Y\. Shi, Y\. Tong, and Y\. Zhang\(2024\)A survey on federated learning: a perspective from multi\-party computation\.Frontiers of Computer Science18\(1\),pp\. 181336\.Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px1.p1.1)\.
- \[239\]J\. Liu, B\. D\. Anderson, M\. Cao, and A\. S\. Morse\(2013\)Analysis of accelerated gossip algorithms\.Automatica49\(4\),pp\. 873–883\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I13.i2.p1.1)\.
- \[240\]J\. Liu, J\. Huang, Y\. Zhou, X\. Li, S\. Ji, H\. Xiong, and D\. Dou\(2022\)From distributed machine learning to federated learning: a survey\.Knowledge and Information Systems64\(4\),pp\. 885–917\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.10.2),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.20.2)\.
- \[241\]R\. Liu, P\. Xing, Z\. Deng, A\. Li, C\. Guan, and H\. Yu\(2025\)Federated graph neural networks: overview, techniques, and challenges\.IEEE Transactions on Neural Networks and Learning Systems36\(3\),pp\. 4279–4295\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.17.2.2.1.2.1),[§3\.3\.1](https://arxiv.org/html/2609.02984#S3.SS3.SSS1.p1.1)\.
- \[242\]S\. Liu, T\. Li, Y\. Feng, N\. Tran, H\. Zhao, Q\. Qiu, and P\. Li\(2023\)Structural re\-weighting improves graph domain adaptation\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I9.i2.p1.1)\.
- \[243\]S\. Liu, Y\. Ge, S\. Xu, Y\. Zhang, and A\. Marian\(2022\)Fairness\-aware federated matrix factorization\.InACM Conference on Recommender Systems,Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.2)\.
- \[244\]T\. Liu, P\. Li, and Y\. Gu\(2021\)Glint: decentralized federated graph learning with traffic throttling and flow scheduling\.InIEEE/ACM International Symposium on Quality of Service,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I24.i3.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I25.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I31.i1.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.2.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.36.2)\.
- \[245\]W\. Liu, X\. Zang, Y\. Li, and B\. Vucetic\(2020\)Over\-the\-air computation systems: optimization, analysis and scaling laws\.IEEE Transactions on Wireless Communications19\(8\),pp\. 5488–5502\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px2.p1.1)\.
- \[246\]Y\. Liu, Y\. Kang, C\. Xing, T\. Chen, and Q\. Yang\(2020\)A secure federated transfer learning framework\.IEEE Intelligent Systems35\(4\),pp\. 70–82\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I12.i2.p1.1)\.
- \[247\]Y\. Liu, Y\. Kang, T\. Zou, Y\. Pu, Y\. He, X\. Ye, Y\. Ouyang, Y\. Zhang, and Q\. Yang\(2024\)Vertical federated learning: concepts, advances, and challenges\.IEEE Transactions on Knowledge and Data Engineering36\(7\),pp\. 3615–3634\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.11.2),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.12.2),[2nd item](https://arxiv.org/html/2609.02984#S2.I1.i2.p1.2),[§2\.3\.3](https://arxiv.org/html/2609.02984#S2.SS3.SSS3.Px1.p1.2),[§2\.3\.3](https://arxiv.org/html/2609.02984#S2.SS3.SSS3.Px1.p2.5),[§2\.3\.3](https://arxiv.org/html/2609.02984#S2.SS3.SSS3.Px2.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px1.p1.1),[Algorithm 2](https://arxiv.org/html/2609.02984#alg2)\.
- \[248\]Z\. Liu, Y\. Jiang, J\. Gu, D\. Liu, Y\. Song, A\. Jiang, and P\. Duan\(2025\)Federated graph neural networks based on multiscale residuals in industrial internet of things\.IEEE Transactions on Computational Social Systems\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p6.1)\.
- \[249\]Z\. Liu, L\. Yang, Z\. Fan, H\. Peng, and P\. S\. Yu\(2022\)Federated social recommendation with graph neural network\.ACM Transactions on Intelligent Systems and Technology13\(4\),pp\. 1–24\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I13.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I23.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I35.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I8.i3.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px1.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.38.7),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.2.1)\.
- \[250\]Z\. Liu, J\. Zhang, E\. Shi, Z\. Liu, D\. Niyato, B\. Ai, and X\. Shen\(2024\)Graph neural network meets multi\-agent reinforcement learning: fundamentals, applications, and future directions\.IEEE Wireless Communications31\(6\),pp\. 39–47\.Cited by:[§1\.1](https://arxiv.org/html/2609.02984#S1.SS1.p1.2)\.
- \[251\]I\. Lobel and A\. Ozdaglar\(2008\)Convergence analysis of distributed subgradient methods over random networks\.InIEEE Allerton Conference on Communication, Control, and Computing,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[252\]G\. Long, M\. Xie, T\. Shen, T\. Zhou, X\. Wang, and J\. Jiang\(2023\)Multi\-center federated learning: clients clustering for better personalization\.World Wide Web26\(1\),pp\. 481–500\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I8.i3.p1.1)\.
- \[253\]Y\. Low, J\. Gonzalez, A\. Kyrola, D\. Bickson, C\. Guestrin, and J\. Hellerstein\(2010\)GraphLab: a new framework for parallel machine learning\.InConference on Uncertainty in Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p7.1)\.
- \[254\]L\. Lu and N\. Ding\(2020\)Multi\-party private set intersection in vertical federated learning\.InIEEE International Conference on Trust, Security and Privacy in Computing and Communications,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I10.i1.p1.1)\.
- \[255\]R\. Luo, S\. U\. Stich, S\. Horváth, and M\. Takáč\(2025\)Revisiting localSGD and SCAFFOLD: improved rates and missing analysis\.InInternational Conference on Artificial Intelligence and Statistics,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1)\.
- \[256\]S\. Luo, X\. Chen, Q\. Wu, Z\. Zhou, and S\. Yu\(2020\)HFEL: joint edge association and resource allocation for cost\-efficient hierarchical federated edge learning\.IEEE Transactions on Wireless Communications19\(10\),pp\. 6535–6548\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p1.1)\.
- \[257\]L\. Lyu, H\. Yu, and Q\. Yang\(2020\)Threats to federated learning: a survey\.arXiv:2003\.02133\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.13.2)\.
- \[258\]C\. Mai, T\. Liao, C\. Chen, and Z\. Zheng\(2024\)FGTL: federated graph transfer learning for node classification\.ACM Transactions on Knowledge Discovery from Data19\(1\),pp\. 1–20\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I13.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I14.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I18.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I18.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[§3\.3\.3](https://arxiv.org/html/2609.02984#S3.SS3.SSS3.Px2.p1.1),[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3),[§3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.8),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.10.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.21.5),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.8)\.
- \[259\]Y\. Mansour, M\. Mohri, J\. Ro, and A\. T\. Suresh\(2020\)Three approaches for personalization with applications to federated learning\.arXiv:2002\.10619\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I8.i3.p1.1)\.
- \[260\]S\. Maskey, R\. Paolino, F\. Jogl, G\. Kutyniok, and J\. F\. Lutzeyer\(2025\)Graph representational learning: when does more expressivity hurt generalization?\.arXiv:2505\.11298\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I16.i2.p1.1)\.
- \[261\]J\. Mathys, F\. Grötschla, K\. V\. Nadimpalli, and R\. Wattenhofer\(2024\)Rethinking message passing for algorithmic alignment\.InNEURIPS Workshop on Symmetry and Geometry in Neural Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I30.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.32.3.2.1.1.1)\.
- \[262\]R\. McDonald, K\. Hall, and G\. Mann\(2010\)Distributed training strategies for the structured perceptron\.InAnnual Conference of the North American Chapter of the Association for Computational Linguistics,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p2.1)\.
- \[263\]R\. Mcdonald, M\. Mohri, N\. Silberman, D\. Walker, and G\. Mann\(2009\)Efficient large\-scale distributed training of conditional maximum entropy models\.Advances in Neural Information Processing Systems22\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p3.1)\.
- \[264\]B\. McMahan, E\. Moore, D\. Ramage, S\. Hampson, and B\. A\. y Arcas\(2017\)Communication\-efficient learning of deep networks from decentralized data\.InArtificial Intelligence and Statistics,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S1.I1.i3.p1.1),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.9.2),[1st item](https://arxiv.org/html/2609.02984#S2.I2.i1.p1.1),[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px2.p1.1),[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px3.p1.1),[§2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4.p1.1),[Algorithm 1](https://arxiv.org/html/2609.02984#alg1),[footnote 1](https://arxiv.org/html/2609.02984#footnote1)\.
- \[265\]H\. B\. McMahan, D\. Ramage, K\. Talwar, and L\. Zhang\(2018\)Learning differentially private recurrent language models\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p1.1)\.
- \[266\]G\. Mei, Z\. Guo, S\. Liu, and L\. Pan\(2019\)Sgnn: a graph neural network based federated learning approach by hiding structure\.InIEEE International Conference on Big Data,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I8.i2.p1.1)\.
- \[267\]L\. Melis, C\. Song, E\. De Cristofaro, and V\. Shmatikov\(2019\)Exploiting unintended feature leakage in collaborative learning\.InIEEE Symposium on Security and Privacy,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I4.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S2.I4.i3.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.1)\.
- \[268\]C\. Meng, M\. Tang, M\. Setayesh, and V\. W\. Wong\(2025\)Tackling resource allocation for decentralized federated learning: a GNN\-based approach\.IEEE Transactions on Mobile Computing\.Cited by:[§2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3.Px1.p1.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.4.1)\.
- \[269\]C\. Meng, S\. Rambhatla, and Y\. Liu\(2021\)Cross\-node federated graph neural network for spatio\-temporal data modeling\.InACM SIGKDD Conference on Knowledge Discovery & Data Mining,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I18.i1.p1.1),[§3\.3\.3](https://arxiv.org/html/2609.02984#S3.SS3.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.3)\.
- \[270\]F\. Mo, H\. Haddadi, K\. Katevas, E\. Marin, D\. Perino, and N\. Kourtellis\(2021\)PPFL: privacy\-preserving federated learning with trusted execution environments\.InAnnual International Conference on Mobile Systems, Applications, and Services,Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px1.p1.1)\.
- \[271\]B\. Mohar, Y\. Alavi, G\. Chartrand, and O\. Oellermann\(1991\)The laplacian spectrum of graphs\.Graph Theory, Combinatorics, and Applications2\(871\-898\),pp\. 12\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[272\]F\. Monti, D\. Boscaini, J\. Masci, E\. Rodola, J\. Svoboda, and M\. M\. Bronstein\(2017\)Geometric deep learning on graphs and manifolds using mixture model CNNs\.InIEEE Conference on Computer Vision and Pattern Recognition,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I9.i1.p1.1)\.
- \[273\]L\. Moreau\(2005\)Stability of multiagent systems with time\-dependent communication links\.IEEE Transactions on Automatic Control50\(2\),pp\. 169–182\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[274\]J\. G\. Moreno\-Torres, T\. Raeder, R\. Alaiz\-Rodríguez, N\. V\. Chawla, and F\. Herrera\(2012\)A unifying view on dataset shift in classification\.Pattern Recognition45\(1\),pp\. 521–530\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I2.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I2.i2.p1.1),[§2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2.Px2.p1.1)\.
- \[275\]C\. Morris, F\. Frasca, N\. Dym, H\. Maron, İ\. İ\. Ceylan, R\. Levie, D\. Lim, M\. Bronstein, M\. Grohe, and S\. Jegelka\(2024\)Future directions in the theory of graph machine learning\.arXiv:2402\.02287\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I16.i3.p1.1)\.
- \[276\]H\. Naveed, A\. U\. Khan, S\. Qiu, M\. Saqib, S\. Anwar, M\. Usman, N\. Akhtar, N\. Barnes, and A\. Mian\(2023\)A comprehensive overview of large language models\.ACM Transactions on Intelligent Systems and Technology\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p2.1)\.
- \[277\]M\. Nazzal, A\. Khreishah, J\. Lee, S\. Angizi, A\. Al\-Fuqaha, and M\. Guizani\(2024\)Semi\-decentralized inference in heterogeneous graph neural networks for traffic demand forecasting: an edge\-computing approach\.IEEE Transactions on Vehicular Technology73\(12\),pp\. 19400–19416\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I22.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I25.i2.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.29.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.36.2)\.
- \[278\]A\. Nedic, A\. Olshevsky, A\. Ozdaglar, and J\. N\. Tsitsiklis\(2009\)On distributed averaging algorithms and quantization effects\.IEEE Transactions on Automatic Control54\(11\),pp\. 2506–2517\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[279\]A\. Nedic, A\. Ozdaglar, and P\. A\. Parrilo\(2010\)Constrained consensus and optimization in multi\-agent networks\.IEEE Transactions on Automatic Control55\(4\),pp\. 922–938\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[280\]A\. Nedic and A\. Ozdaglar\(2009\)Distributed subgradient methods for multi\-agent optimization\.IEEE Transactions on Automatic Control54\(1\),pp\. 48–61\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[281\]Y\. Nesterov\(2009\)Primal\-dual subgradient methods for convex problems\.Mathematical Programming120\(1\),pp\. 221–259\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[282\]J\. Nguyen, K\. Malik, H\. Zhan, A\. Yousefpour, M\. Rabbat, M\. Malek, and D\. Huba\(2022\)Federated learning with buffered asynchronous aggregation\.InInternational Conference on Artificial Intelligence and Statistics,Cited by:[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px1.p1.1)\.
- \[283\]X\. Ni, X\. Xu, L\. Lyu, C\. Meng, and W\. Wang\(2021\)A vertical federated learning framework for graph convolutional network\.arXiv:2106\.11593\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I33.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I8.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.4),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.4)\.
- \[284\]M\. Nie, D\. Chen, and D\. Wang\(2023\)Reinforcement learning on graphs: a survey\.IEEE Transactions on Emerging Topics in Computational Intelligence7\(4\),pp\. 1065–1082\.Cited by:[§1\.1](https://arxiv.org/html/2609.02984#S1.SS1.p1.2)\.
- \[285\]T\. Nishio and R\. Yonetani\(2019\)Client selection for federated learning with heterogeneous resources in mobile edge\.InIEEE International Conference on Communications,Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px3.p1.1)\.
- \[286\]T\. E\. Nisonger\(2008\)The “80/20 rule” and core journals\.The Serials Librarian55\(1\-2\),pp\. 62–84\.Cited by:[§2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2.Px1.p1.1)\.
- \[287\]Y\. Oh, Y\. Jeon, M\. Chen, and W\. Saad\(2024\)FedVQCS: federated learning via vector quantized compressed sensing\.IEEE Transactions on Wireless Communications23\(3\),pp\. 1755–1770\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px1.p1.1)\.
- \[288\]I\. E\. Olatunji, W\. Nejdl, and M\. Khosla\(2021\)Membership inference attack on graph neural networks\.InIEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications,Cited by:[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p1.1)\.
- \[289\]R\. Olfati\-Saber, J\. A\. Fax, and R\. M\. Murray\(2007\)Consensus and cooperation in networked multi\-agent systems\.IEEE Conference on Decision and Control95\(1\),pp\. 215–233\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S1.I1.i1.p1.1),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.3.2.2.1.1.1),[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[290\]R\. Olfati\-Saber and R\. M\. Murray\(2004\)Consensus problems in networks of agents with switching topology and time\-delays\.IEEE Transactions on Automatic Control49\(9\),pp\. 1520–1533\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[291\]A\. Olshevsky and J\. N\. Tsitsiklis\(2009\)Convergence speed in distributed consensus and averaging\.SIAM Journal on Control and Optimization48\(1\),pp\. 33–55\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[292\]R\. Olshevskyi, Z\. Zhao, K\. Chan, G\. Verma, A\. Swami, and S\. Segarra\(2025\)Fully distributed online training of graph neural networks in networked systems\.InIEEE International Conference on Machine Learning for Communication and Networking,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I27.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.4.1)\.
- \[293\]N\. Onoszko, G\. Karlsson, O\. Mogren, and E\. L\. Zec\(2021\)Decentralized federated learning of deep neural networks on non\-iid data\.InInternational Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1)\.
- \[294\]R\. Ormándi, I\. Hegedűs, and M\. Jelasity\(2013\)Gossip learning with linear models on fully distributed data\.Concurrency and Computation: Practice and Experience25\(4\),pp\. 556–571\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[295\]Q\. Pan, Y\. Zhu, and L\. Chu\(2023\)Lumos: heterogeneity\-aware federated graph learning over decentralized devices\.InIEEE International Conference on Data Engineering,Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p6.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I22.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I25.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.29.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.3)\.
- \[296\]Q\. Pan and Y\. Zhu\(2022\)Fedwalk: communication efficient federated unsupervised node embedding with differential privacy\.InACM SIGKDD Conference on Knowledge Discovery and Data Mining,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I25.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.2.1)\.
- \[297\]S\. J\. Pan and Q\. Yang\(2009\)A survey on transfer learning\.IEEE Transactions on Knowledge and Data Engineering22\(10\),pp\. 1345–1359\.Cited by:[§2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1.p2.1)\.
- \[298\]T\. Papamarkou, T\. Birdal, M\. Bronstein, G\. Carlsson, J\. Curry, Y\. Gao, M\. Hajij, R\. Kwitt, P\. Liò, P\. Di Lorenzo, V\. Maroulas, N\. Miolane, F\. Nasrin, K\. N\. Ramamurthy, B\. Rieck, S\. Scardapane, M\. T\. Schaub, P\. Veličković, B\. Wang, Y\. Wang, G\. Wei, and G\. Zamzmi\(2024\)Position: topological deep learning is the new frontier for relational learning\.InInternational Conference on Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I19.i1.p1.1.2)\.
- \[299\]P\. A\. Papp and R\. Wattenhofer\(2022\)An introduction to graph neural networks from a distributed computing perspective\.InInternational Conference on Distributed Computing and Internet Technology,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px5.p1.1)\.
- \[300\]J\. Park and C\. Joe\-Wong\(2024\)Federated learning with flexible architectures\.InJoint European Conference on Machine Learning and Knowledge Discovery in Databases,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1),[§2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3.Px2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I19.i2.p1.1)\.
- \[301\]J\. Park and C\. Joe\-Wong\(2025\)FedTLU: federated learning with targeted layer updates\.InIEEE International Conference on Acoustics, Speech and Signal Processing,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I5.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[302\]M\. Paulik, M\. Seigel, H\. Mason, D\. Telaar, J\. Kluivers, R\. van Dalen, C\. W\. Lau, L\. Carlson, F\. Granqvist, C\. Vandevelde, S\. Agarwal, J\. Freudiger, A\. Byde, A\. Bhowmick, G\. Kapoor, S\. Beaumont, Á\. Cahill, D\. Hughes, O\. Javidbakht, F\. Dong, R\. Rishi, and S\. Hung\(2021\)Federated evaluation and tuning for on\-device personalization: system design & applications\.Technical reportApple\.Cited by:[§2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2.p1.1)\.
- \[303\]X\. Pei, X\. Deng, S\. Tian, J\. Liu, and K\. Xue\(2023\)Privacy\-enhanced graph neural network for decentralized local graphs\.IEEE Transactions on Information Forensics and Security19,pp\. 1614–1629\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I13.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I36.i2.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.3)\.
- \[304\]H\. Peng, H\. Li, Y\. Song, V\. Zheng, and J\. Li\(2021\)Differentially private federated knowledge graphs embedding\.InACM International Conference on Information & Knowledge Management,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I15.i1.p1.1),[4th item](https://arxiv.org/html/2609.02984#S3.I8.i4.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.8),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.8),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.8)\.
- \[305\]H\. Peng, R\. Ran, Y\. Luo, J\. Zhao, S\. Huang, K\. Thorat, T\. Geng, C\. Wang, X\. Xu, W\. Wen,et al\.\(2023\)LinGCN: structural linearized graph convolutional network for homomorphically encrypted inference\.InAdvances in Neural Information Processing Systems,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I33.i1.p1.1)\.
- \[306\]J\. Peng, Z\. Chen, Y\. Shao, Y\. Shen, L\. Chen, and J\. Cao\(2022\)Sancus: staleness\-aware communication\-avoiding full\-graph decentralized training in large\-scale graph neural networks\.Proceedings of the VLDB Endowment15\(9\),pp\. 1937–1950\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I30.i1.p1.1)\.
- \[307\]L\. Peng, N\. Wang, N\. Dvornek, X\. Zhu, and X\. Li\(2022\)Fedni: federated graph learning with network inpainting for population\-based disease prediction\.IEEE Transactions on Medical Imaging42\(7\),pp\. 2032–2043\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I12.i3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.2)\.
- \[308\]D\. Pérez Herrera, Z\. Chen, and E\. G\. Larsson\(2025\)Faster convergence with less communication: broadcast\-based subgraph sampling for decentralized learning over wireless networks\.IEEE Open Journal of the Communications Society\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px2.p1.1)\.
- \[309\]L\. T\. Phong, Y\. Aono, T\. Hayashi, L\. Wang, and S\. Moriai\(2017\)Privacy\-preserving deep learning via additively homomorphic encryption\.IEEE Transactions on Information Forensics and Security13\(5\),pp\. 1333–1345\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I4.i2.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px1.p1.1)\.
- \[310\]V\. Pihur, A\. Korolova, F\. Liu, S\. Sankuratripati, M\. Yung, D\. Huang, and R\. Zeng\(2022\)Differentially\-private “draw and discard” machine learning: training distributed model from enormous crowds\.InInternational Symposium on Cyber Security, Cryptology, and Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1)\.
- \[311\]B\. Pinkas, T\. Schneider, and M\. Zohner\(2018\)Scalable private set intersection based on ot extension\.ACM Transactions on Privacy and Security21\(2\),pp\. 1–35\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I10.i1.p1.1)\.
- \[312\]O\. Platonov, D\. Kuznedelev, A\. Babenko, and L\. Prokhorenkova\(2023\)Characterizing graph datasets for node classification: homophily\-heterophily dichotomy and beyond\.Advances in Neural Information Processing Systems36,pp\. 523–548\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1)\.
- \[313\]O\. Platonov, D\. Kuznedelev, M\. Diskin, A\. Babenko, and L\. Prokhorenkova\(2023\)A critical look at the evaluation of GNNs under heterophily: are we really making progress?\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1)\.
- \[314\]C\. Qian, G\. Rattan, F\. Geerts, M\. Niepert, and C\. Morris\(2022\)Ordered subgraph aggregation networks\.Advances in Neural Information Processing Systems35,pp\. 21030–21045\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I6.i2.p1.2)\.
- \[315\]Y\. Qiu, C\. Huang, J\. Wang, Z\. Huang, and J\. Xiao\(2022\)A privacy\-preserving subgraph\-level federated graph neural network via differential privacy\.InInternational Conference on Knowledge Science, Engineering and Management,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I12.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I24.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I8.i3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.1.1)\.
- \[316\]L\. Qu, N\. Tang, R\. Zheng, Q\. V\. H\. Nguyen, Z\. Huang, Y\. Shi, and H\. Yin\(2023\)Semi\-decentralized federated ego graph learning for recommendation\.InACM Web Conference,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I17.i3.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I22.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.7),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.2.2.1.2.1)\.
- \[317\]J\. Quiñonero\-Candela, M\. Sugiyama, A\. Schwaighofer, and N\. D\. Lawrence\(2022\)Dataset shift in machine learning\.MIT Press\.Cited by:[§2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2.Px2.p1.1)\.
- \[318\]S\. S\. Ram, A\. Nedić, and V\. V\. Veeravalli\(2009\)Asynchronous gossip algorithms for stochastic optimization\.InIEEE Conference on Decision and Control,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[319\]S\. S\. Ram, A\. Nedić, and V\. V\. Veeravalli\(2009\)Incremental stochastic subgradient algorithms for convex optimization\.SIAM Journal on Optimization20\(2\),pp\. 691–717\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p7.1)\.
- \[320\]S\. Ramaswamy, O\. Thakkar, R\. Mathews, G\. Andrew, H\. B\. McMahan, and F\. Beaufays\(2020\)Training production language models without memorizing user data\.Technical reportGoogle\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p3.1)\.
- \[321\]R\. Ran, W\. Wang, Q\. Gang, J\. Yin, N\. Xu, and W\. Wen\(2022\)CryptoGCN: fast and scalable homomorphically encrypted graph convolutional network inference\.InAdvances in Neural Information Processing Systems,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I33.i1.p1.1)\.
- \[322\]S\. Razavikia, J\. M\. Barros da Silva, and C\. Fischione\(2024\)ChannelComp: a general method for computation by communications\.IEEE Transactions on Communications6\(\),pp\. 1497–1511\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px2.p1.1)\.
- \[323\]S\. Reddi, Z\. Charles, M\. Zaheer, Z\. Garrett, K\. Rush, J\. Konečný, S\. Kumar, and H\. B\. McMahan\(2021\)Adaptive federated optimization\.InInternational Conference on Learning Representations,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p5.1)\.
- \[324\]J\. Ren, Y\. He, D\. Wen, G\. Yu, K\. Huang, and D\. Guo\(2020\)Scheduling for cellular federated edge learning with importance and channel awareness\.IEEE Transactions on Wireless Communications19\(11\),pp\. 7690–7703\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px3.p1.1)\.
- \[325\]Z\. Ren, L\. Yang, and K\. Chen\(2022\)Improving availability of vertical federated learning: relaxing inference on non\-overlapping data\.ACM Transactions on Intelligent Systems and Technology13\(4\),pp\. 1–20\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I12.i1.p1.1)\.
- \[326\]M\. Ribero, H\. Vikalo, and G\. De Veciana\(2022\)Federated learning under intermittent client availability and time\-varying communication constraints\.IEEE Journal of Selected Topics in Signal Processing17\(1\),pp\. 98–111\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[327\]H\. Robbins and S\. Monro\(1951\)A stochastic approximation method\.The Annals of Mathematical Statistics,pp\. 400–407\.Cited by:[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px3.p1.1)\.
- \[328\]A\. Rodio, F\. Faticanti, O\. Marfoq, G\. Neglia, and E\. Leonardi\(2023\)Federated learning under heterogeneous and correlated client availability\.InIEEE Infocom Conference on Computer Communications,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[329\]F\. Rosenblatt\(1958\)The perceptron: a probabilistic model for information storage and organization in the brain\.\.Psychological Review65\(6\),pp\. 386\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p1.1)\.
- \[330\]Y\. Ruan, X\. Zhang, S\. Liang, and C\. Joe\-Wong\(2021\)Towards flexible device participation in federated learning\.InInternational Conference on Artificial Intelligence and Statistics,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[331\]D\. E\. Rumelhart, G\. E\. Hinton, and R\. J\. Williams\(1986\)Learning representations by back\-propagating errors\.Nature323\(6088\),pp\. 533–536\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px4.p1.2)\.
- \[332\]F\. Sabah, Y\. Chen, Z\. Yang, M\. Azam, N\. Ahmad, and R\. Sarwar\(2024\)Model optimization techniques in personalized federated learning: a survey\.Expert Systems with Applications243,pp\. 122874\.Cited by:[§2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2.p1.2)\.
- \[333\]R\. O\. Saber and R\. M\. Murray\(2003\)Consensus protocols for networks of dynamic agents\.InIEEE American Control Conference,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[334\]F\. Sattler, K\. Müller, and W\. Samek\(2020\)Clustered federated learning: model\-agnostic distributed multitask optimization under privacy constraints\.IEEE Transactions on Neural Networks and Learning Systems32\(8\),pp\. 3710–3722\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I8.i3.p1.1)\.
- \[335\]S\. Scardapane, I\. Spinelli, and P\. Di Lorenzo\(2020\)Distributed training of graph convolutional networks\.IEEE Transactions on Signal and Information Processing over Networks7,pp\. 87–100\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I27.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I29.i2.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px3.p2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.2.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.36.2)\.
- \[336\]A\. Shamsian, A\. Navon, E\. Fetaya, and G\. Chechik\(2021\)Personalized federated learning using hypernetworks\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I8.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I17.i2.p1.1)\.
- \[337\]Y\. Shao, H\. Li, X\. Gu, H\. Yin, Y\. Li, X\. Miao, W\. Zhang, B\. Cui, and L\. Chen\(2024\)Distributed graph neural network training: a survey\.ACM Computing Surveys56\(8\),pp\. 1–39\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.20.2),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.p1.1)\.
- \[338\]W\. Shen, M\. Ye, W\. Yu, and P\. C\. Yuen\(2025\)Build yourself before collaboration: vertical federated learning with limited aligned samples\.IEEE Transactions on Mobile Computing24\(7\),pp\. 6503–6516\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I11.i1.p1.1)\.
- \[339\]X\. Shen, Q\. Dai, F\. Chung, W\. Lu, and K\. Choi\(2020\)Adversarial deep network embedding for cross\-network node classification\.InAAAI Conference on Artificial Intelligence,Cited by:[§3\.4\.3](https://arxiv.org/html/2609.02984#S3.SS4.SSS3.Px2.p1.1)\.
- \[340\]Z\. Shi, G\. Wan, W\. Huang, G\. Zhang, H\. Li, C\. Yang, and M\. Ye\(2025\)EAGLES: towards effective, efficient, and economical federated graph learning via unified sparsification\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I18.i1.p2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.10.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.1.1)\.
- \[341\]N\. Shlezinger, M\. Chen, Y\. C\. Eldar, H\. V\. Poor, and S\. Cui\(2021\)UVeQFed: universal vector quantization for federated learning\.IEEE Transactions on Signal Processing69,pp\. 500–514\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px1.p1.1)\.
- \[342\]N\. Shoham, T\. Avidor, A\. Keren, N\. Israel, D\. Benditkis, L\. Mor\-Yosef, and I\. Zeitak\(2019\)Overcoming forgetting in federated learning on non\-iid data\.InNEURIPS Workshop on Federated Learning for Data Privacy and Confidentiality,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1)\.
- \[343\]R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov\(2017\)Membership inference attacks against machine learning models\.InIEEE Symposium on Security and Privacy,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I4.i1.p1.1)\.
- \[344\]I\. Shumailov, D\. Ramage, S\. Meiklejohn, P\. Kairouz, F\. Hartmann, B\. Balle, and E\. Bagdasarian\(2025\)Trusted machine learning models unlock private inference for problems currently infeasible with cryptography\.Technical reportGoogle\.Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p2.1)\.
- \[345\]A\. Smith, A\. Thakurta, and J\. Upadhyay\(2017\)Is interaction necessary for distributed private learning?\.InIEEE Symposium on Security and Privacy,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I15.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I36.i1.p1.1)\.
- \[346\]V\. Smith, C\. Chiang, M\. Sanjabi, and A\. S\. Talwalkar\(2017\)Federated multi\-task learning\.Advances in Neural Information Processing Systems30\.Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I9.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I18.i2.p1.1)\.
- \[347\]J\. Snell, K\. Swersky, and R\. Zemel\(2017\)Prototypical networks for few\-shot learning\.InAdvances in Neural Information Processing Systems,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I5.i4.p1.1)\.
- \[348\]J\. Snow\(1855\)On the mode of communication of cholera\.Edinburgh medical journal\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I2.i1.p1.1)\.
- \[349\]O\. Solodova, N\. Richardson, D\. Oktay, and R\. P\. Adams\(2025\)Graph neural networks gone hogwild\.InThe International Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I30.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.32.3.2.1.1.1)\.
- \[350\]S\. Song, K\. Chaudhuri, and A\. D\. Sarwate\(2013\)Stochastic gradient descent with differentially private updates\.InIEEE Global Conference on Signal and Information Processing,Cited by:[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.2)\.
- \[351\]J\. Southern, Y\. Eitan, G\. Bar\-Shalom, M\. M\. Bronstein, H\. Maron, and F\. Frasca\(2025\)Balancing efficiency and expressiveness: subgraph GNNs with walk\-based centrality\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I16.i2.p1.1)\.
- \[352\]S\. U\. Stich\(2019\)Local SGD converges fast and communicates little\.InInternational Conference on Learning Representations,Cited by:[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px3.p1.1),[§2\.3](https://arxiv.org/html/2609.02984#S2.SS3.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p4.1),[1st item](https://arxiv.org/html/2609.02984#S3.I27.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I27.i2.p1.1)\.
- \[353\]L\. Sun, Y\. Dou, C\. Yang, K\. Zhang, J\. Wang, P\. S\. Yu, L\. He, and B\. Li\(2022\)Adversarial attack and defense on graph data: a survey\.IEEE Transactions on Knowledge and Data Engineering35\(8\),pp\. 7693–7711\.Cited by:[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p2.1)\.
- \[354\]Y\. Sun, Z\. Li, Y\. Li, and B\. Ding\(2024\)Improving LoRA in privacy\-preserving federated learning\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I8.i1.p1.1)\.
- \[355\]Z\. Sun, H\. Yin, H\. Chen, T\. Chen, L\. Cui, and F\. Yang\(2020\)Disease prediction via graph neural networks\.IEEE Journal of Biomedical and Health Informatics25\(3\),pp\. 818–826\.Cited by:[4th item](https://arxiv.org/html/2609.02984#S3.I8.i4.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.9),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.9)\.
- \[356\]Z\. Sun, Z\. Zhang, Z\. Xu, G\. Joshi, P\. Sharma, and E\. Wei\(2025\)Debiasing federated learning with correlated client participation\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[357\]C\. T Dinh, N\. Tran, and J\. Nguyen\(2020\)Personalized federated learning with Moreau envelopes\.Advances in Neural Information Processing Systems33,pp\. 21394–21405\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1)\.
- \[358\]K\. Talwar, S\. Wang, A\. McMillan, V\. Jina, V\. Feldman, P\. Bansal, B\. Basile, A\. Cahill, Y\. S\. Chan, M\. Chatzidakis, J\. Chen, O\. Chick, M\. Chitnis, S\. Ganta, Y\. Goren, F\. Granqvist, K\. Guo, F\. Jacobs, O\. Javidbakht, A\. Liu, R\. Low, D\. Mascenik, S\. Myers, D\. Park, W\. Park, G\. Parsa, T\. Pauly, C\. Priebe, R\. Rishi, G\. Rothblum, M\. Scaria, L\. Song, C\. Song, K\. Tarbe, S\. Vogt, L\. Winstrom, and S\. Zhou\(2024\)Samplable anonymous aggregation for private federated data analysis\.InACM SIGSAC Conference on Computer and Communications Security,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p2.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px3.p2.1)\.
- \[359\]A\. Z\. Tan, H\. Yu, L\. Cui, and Q\. Yang\(2022\)Towards personalized federated learning\.IEEE Transactions on Neural Networks and Learning Systems\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.12.2),[§2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3.Px2.p1.1),[§2\.3\.2](https://arxiv.org/html/2609.02984#S2.SS3.SSS2.p1.2)\.
- \[360\]Y\. Tan, G\. Long, L\. Liu, T\. Zhou, Q\. Lu, J\. Jiang, and C\. Zhang\(2022\)FedProto: federated prototype learning across heterogeneous clients\.InAAAI Conference on Artificial Intelligence,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I5.i4.p1.1)\.
- \[361\]Y\. Tan, G\. Long, J\. Ma, L\. Liu, T\. Zhou, and J\. Jiang\(2022\)Federated learning from pre\-trained models: a contrastive learning approach\.InAdvances in Neural Information Processing Systems,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I5.i4.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1)\.
- \[362\]Z\. Tan, S\. Huang, G\. Wan, W\. Huang, H\. Li, and M\. Ye\(2025\)S2FGL: spatial spectral federated graph learning\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I14.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.2)\.
- \[363\]Z\. Tan, G\. Wan, W\. Huang, H\. Li, G\. Zhang, C\. Yang, and M\. Ye\(2025\)FedSPA: generalizable federated graph learning under homophily heterogeneity\.InComputer Vision and Pattern Recognition Conference,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I15.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.14.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.3.1)\.
- \[364\]Z\. Tan, G\. Wan, W\. Huang, and M\. Ye\(2024\)FedSSP: federated graph learning with spectral knowledge and personalized preference\.InAdvances in Neural Information Processing Systems,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I12.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I14.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I18.i1.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I18.i3.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I9.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.13.10.2.1.1.1)\.
- \[365\]Y\. Tian, S\. Pei, X\. Zhang, C\. Zhang, and N\. V\. Chawla\(2025\)Knowledge distillation on graphs: a survey\.ACM Computing Surveys57\(8\),pp\. 1–16\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I21.i1.p1.1)\.
- \[366\]J\. Topping, F\. Di Giovanni, B\. P\. Chamberlain, X\. Dong, and M\. M\. Bronstein\(2022\)Understanding over\-squashing and bottlenecks on graphs via curvature\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I16.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I29.i1.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I30.i3.p1.1)\.
- \[367\]J\. Tsitsiklis, D\. Bertsekas, and M\. Athans\(1986\)Distributed asynchronous deterministic and stochastic gradient optimization algorithms\.IEEE Transactions on Automatic Control31\(9\),pp\. 803–812\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S1.I1.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p2.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[368\]K\. Uludag, Ç\. Eroglu Erdem, and Ö\. Korçak\(2025\)Leveraging unlabeled data in federated learning: a review\.IEEE Access13,pp\. 181719–181743\.Cited by:[§1\.1](https://arxiv.org/html/2609.02984#S1.SS1.p1.2)\.
- \[369\]S\. Vahidian, M\. Morafah, and B\. Lin\(2021\)Personalized federated learning by structured and unstructured pruning under data heterogeneity\.InIEEE International Conference on Distributed Computing Systems Workshops,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[370\]L\. G\. Valiant\(1984\)A theory of the learnable\.Communications of the ACM27\(11\),pp\. 1134–1142\.Cited by:[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px1.p3.1)\.
- \[371\]P\. Vanhaesebrouck, A\. Bellet, and M\. Tommasi\(2017\)Decentralized collaborative learning of personalized models over networks\.InInternational Conference on Artificial Intelligence and Statistics,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p5.1)\.
- \[372\]P\. Veličković, G\. Cucurull, A\. Casanova, A\. Romero, P\. Liò, and Y\. Bengio\(2018\)Graph attention networks\.InInternational Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I19.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p3.2)\.
- \[373\]P\. Veličković, W\. Fedus, W\. L\. Hamilton, P\. Liò, Y\. Bengio, and R\. D\. Hjelm\(2019\)Deep graph infomax\.InInternational Conference on Learning Representations,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px4.p1.2),[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3)\.
- \[374\]P\. Vepakomma, O\. Gupta, T\. Swedish, and R\. Raskar\(2019\)Split learning for health: distributed deep learning without sharing raw patient data\.InICLR Workshop on AI for Social Good,Cited by:[footnote 2](https://arxiv.org/html/2609.02984#footnote2)\.
- \[375\]T\. Vicsek, A\. Czirók, E\. Ben\-Jacob, I\. Cohen, and O\. Shochet\(1995\)Novel type of phase transition in a system of self\-driven particles\.Physical Review Letters75\(6\),pp\. 1226\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I13.i1.p1.2)\.
- \[376\]B\. Wang, A\. Li, M\. Pang, H\. Li, and Y\. Chen\(2022\)Graphfl: a federated learning framework for semi\-supervised node classification on graphs\.InIEEE International Conference on Data Mining,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I17.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.10.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.1.1)\.
- \[377\]H\. Wang, Z\. Kaplan, D\. Niu, and B\. Li\(2020\)Optimizing federated learning on non\-iid data with reinforcement learning\.InIEEE Infocom Conference on Computer Communications,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I7.i1.p1.1)\.
- \[378\]J\. Wang, M\. Kolar, and N\. Srerbo\(2016\)Distributed multi\-task learning\.InInternational Conference on Artificial Intelligence and Statistics,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[379\]J\. Wang, Q\. Liu, H\. Liang, G\. Joshi, and H\. V\. Poor\(2020\)Tackling the objective inconsistency problem in heterogeneous federated optimization\.InAdvances in Neural Information Processing Systems,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I5.i2.p1.1)\.
- \[380\]K\. Wang, Q\. He, F\. Chen, C\. Chen, F\. Huang, H\. Jin, and Y\. Yang\(2023\)Flexifed: personalized federated learning for edge clients with heterogeneous model architectures\.InACM Web Conference,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I9.i1.p1.1)\.
- \[381\]N\. Wang, X\. Xiao, Y\. Yang, J\. Zhao, S\. C\. Hui, H\. Shin, J\. Shin, and G\. Yu\(2019\)Collecting and analyzing multidimensional data with local differential privacy\.InIEEE International Conference on Data Engineering,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1)\.
- \[382\]S\. Wang, J\. Liu, H\. Xu, C\. Tang, Q\. Ma, and L\. Huang\(2025\)Towards communication\-efficient decentralized federated graph learning over non\-iid data\.arXiv:2509\.08409\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I22.i2.p1.1),[§3\.5\.1](https://arxiv.org/html/2609.02984#S3.SS5.SSS1.p4.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px1.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px3.p2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.5.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.34.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.36.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.3)\.
- \[383\]S\. Wang and M\. Ji\(2022\)A unified analysis of federated learning with arbitrary client participation\.Advances in Neural Information Processing Systems35,pp\. 19124–19137\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[384\]S\. Wang and M\. Ji\(2024\)A lightweight method for tackling unknown participation statistics in federated averaging\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[385\]S\. Wang, Y\. Wu, M\. Xia, R\. Wang, and H\. V\. Poor\(2020\)Machine intelligence at the edge with learning centric power allocation\.IEEE Transactions on Wireless Communications21\(10\),pp\. 7926–7941\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I3.i2.p1.1),[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px1.p1.1)\.
- \[386\]S\. Wang, Y\. Zheng, and X\. Jia\(2023\)SecGNN: privacy\-preserving graph neural network training and inference as a cloud service\.IEEE Transactions on Services Computing16\(4\),pp\. 2923–2938\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I33.i1.p1.1)\.
- \[387\]X\. Wang, A\. Lalitha, T\. Javidi, and F\. Koushanfar\(2022\)Peer\-to\-peer variational federated learning over arbitrary graphs\.IEEE Journal on Selected Areas in Information Theory30\(2\),pp\. 129–150\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[388\]X\. Wang and M\. Zhang\(2022\)How powerful are spectral graph neural networks\.InInternational Conference on Machine Learning,Cited by:[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p4.3)\.
- \[389\]Y\. Wang, W\. Wang, Y\. Liang, Y\. Cai, and B\. Hooi\(2021\)Mixup for node and graph classification\.InWeb Conference,Cited by:[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3)\.
- \[390\]Y\. Wang, Y\. Sun, Z\. Liu, S\. E\. Sarma, M\. M\. Bronstein, and J\. M\. Solomon\(2019\)Dynamic graph cnn for learning on point clouds\.ACM Transactions on Graphics38\(5\),pp\. 1–12\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I16.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I29.i1.p1.1)\.
- \[391\]Y\. Wang, H\. Wang, X\. Liu, and Y\. Yan\(2024\)GFedKG: GNN\-based federated embedding model for knowledge graph completion\.Knowledge\-Based Systems301,pp\. 112290\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I20.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I8.i3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.7),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.3.1)\.
- \[392\]Z\. Wang, Z\. Liu, T\. Ma, J\. Li, Z\. Zhang, X\. Fu, Y\. Li, Z\. Yuan, W\. Song, Y\. Ma, Q\. Zeng, X\. Chen, J\. Zhao, J\. Li, M\. Jiang, P\. Lio, N\. Chawla, C\. Zhang, and Y\. Ye\(2025\)Graph foundation models: a comprehensive survey\.arXiv:2505\.15116\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.7.2),[3rd item](https://arxiv.org/html/2609.02984#S3.I19.i3.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I21.i1.p1.1),[§3\.3\.3](https://arxiv.org/html/2609.02984#S3.SS3.SSS3.Px2.p1.1)\.
- \[393\]Z\. Wang, X\. Fan, J\. Qi, H\. Jin, P\. Yang, S\. Shen, and C\. Wang\(2023\)Fedgs: federated graph\-based sampling with arbitrary client availability\.InAAAI Conference on Artificial Intelligence,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.2.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.2.2.1.2.1)\.
- \[394\]Z\. Wang, M\. Eisen, and A\. Ribeiro\(2022\)Learning decentralized wireless resource allocations with graph neural networks\.IEEE Transactions on Signal Processing70,pp\. 1850–1863\.Cited by:[§2\.2\.3](https://arxiv.org/html/2609.02984#S2.SS2.SSS3.Px1.p1.2),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.32.3.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.35.3.2.1.1.1)\.
- \[395\]E\. Wei and A\. Ozdaglar\(2012\)Distributed alternating direction method of multipliers\.InIEEE Conference on Decision and Control,Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p4.1)\.
- \[396\]J\. Wei, W\. Dai, A\. Qiao, Q\. Ho, H\. Cui, G\. R\. Ganger, P\. B\. Gibbons, G\. A\. Gibson, and E\. P\. Xing\(2015\)Managed communication and consistency for fast data\-parallel iterative analytics\.InACM Symposium on Cloud Computing,Cited by:[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px1.p1.1)\.
- \[397\]W\. Wen, Z\. Chen, H\. H\. Yang, W\. Xia, and T\. Q\. Quek\(2022\)Joint scheduling and resource allocation for hierarchical federated edge learning\.IEEE Transactions on Wireless Communications21\(8\),pp\. 5857–5872\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p2.1)\.
- \[398\]M\. Werner, L\. He, M\. Jordan, M\. Jaggi, and S\. P\. Karimireddy\(2024\)Provably personalized and robust federated learning\.Transactions on Machine Learning Research\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I8.i3.p1.1)\.
- \[399\]C\. Wu, F\. Wu, Y\. Cao, Y\. Huang, and X\. Xie\(2021\)Fedgnn: federated graph neural network for privacy\-preserving recommendation\.arXiv:2102\.04925\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I23.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I35.i2.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.1.1)\.
- \[400\]C\. Wu, F\. Wu, L\. Lyu, T\. Qi, Y\. Huang, and X\. Xie\(2022\)A federated graph neural network framework for privacy\-preserving personalization\.Nature Communications13\(1\),pp\. 3091\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I23.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I35.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I7.i1.I1.i2.p1.1),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px2.p5.2),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px1.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.38.7),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.7)\.
- \[401\]F\. Wu, Y\. Long, C\. Zhang, and B\. Li\(2022\)Linkteller: recovering private edges from graph neural networks via influence analysis\.InIEEE Symposium on Security and Privacy,Cited by:[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1)\.
- \[402\]F\. Wu, A\. Souza, T\. Zhang, C\. Fifty, T\. Yu, and K\. Weinberger\(2019\)Simplifying graph convolutional networks\.InInternational Conference on Machine Learning,Cited by:[4th item](https://arxiv.org/html/2609.02984#S3.I18.i4.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I19.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I26.i1.p1.1)\.
- \[403\]Q\. Wu, X\. Chen, T\. Ouyang, Z\. Zhou, X\. Zhang, S\. Yang, and J\. Zhang\(2023\)HiFlash: communication\-efficient hierarchical federated learning with adaptive staleness control and heterogeneity\-aware client\-edge association\.IEEE Transactions on Parallel and Distributed Systems34\(5\),pp\. 1560–1579\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p2.1)\.
- \[404\]Q\. Wu, X\. Chen, Z\. Zhou, and J\. Zhang\(2020\)Fedhome: cloud\-edge based personalized federated learning for in\-home health monitoring\.IEEE Transactions on Mobile Computing21\(8\),pp\. 2818–2832\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I6.i2.p1.1)\.
- \[405\]Y\. Wu, X\. Wang, A\. Zhang, X\. He, and T\. Chua\(2022\)Discovering invariant rationales for graph neural networks\.arXiv:2201\.12872\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I9.i2.p1.1)\.
- \[406\]Z\. Wu, B\. Ramsundar, E\. N\. Feinberg, J\. Gomes, C\. Geniesse, A\. S\. Pappu, K\. Leswing, and V\. Pande\(2018\)MoleculeNet: a benchmark for molecular machine learning\.Chemical Science9\(2\),pp\. 513–530\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I9.i2.p1.1)\.
- \[407\]Z\. Wu, S\. Pan, F\. Chen, G\. Long, C\. Zhang, and S\. Y\. Philip\(2020\)A comprehensive survey on graph neural networks\.IEEE Transactions on Neural Networks and Learning Systems32\(1\),pp\. 4–24\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.6.2),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px3.p1.1)\.
- \[408\]M\. Xiang, S\. Ioannidis, E\. Yeh, C\. Joe\-Wong, and L\. Su\(2024\)Efficient federated learning against heterogeneous and non\-stationary client unavailability\.Advances in Neural Information Processing Systems37,pp\. 104281–104328\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I7.i2.p1.1)\.
- \[409\]C\. Xie, O\. Koyejo, and I\. Gupta\(2020\)Asynchronous federated optimization\.InAnnual Workshop on Optimization for Machine Learning,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I3.i1.p1.1),[§2\.4\.2](https://arxiv.org/html/2609.02984#S2.SS4.SSS2.Px1.p1.1)\.
- \[410\]H\. Xie, J\. Ma, L\. Xiong, and C\. Yang\(2021\)Federated graph classification over non\-iid graphs\.Advances in Neural Information Processing Systems34,pp\. 18839–18852\.Cited by:[3rd item](https://arxiv.org/html/2609.02984#S3.I17.i3.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I6.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I6.i1.p1.2),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.10.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.1.1)\.
- \[411\]P\. Xie, J\. K\. Kim, Y\. Zhou, Q\. Ho, A\. Kumar, Y\. Yu, and E\. Xing\(2015\)Distributed machine learning via sufficient factor broadcasting\.arXiv:1511\.08486\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px1.p1.1)\.
- \[412\]E\. P\. Xing, Q\. Ho, P\. Xie, and D\. Wei\(2016\)Strategies and principles of distributed machine learning on big data\.Engineering2\(2\),pp\. 179–195\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.14.2),[§1](https://arxiv.org/html/2609.02984#S1.p7.1),[§2](https://arxiv.org/html/2609.02984#S2.p2.1)\.
- \[413\]Z\. Xu, Y\. Zhang, G\. Andrew, C\. A\. Choquette\-Choo, P\. Kairouz, H\. B\. McMahan, J\. Rosenstock, and Y\. Zhang\(2023\)Federated learning of gboard language models with differential privacy\.Technical reportGoogle\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p5.1),[2nd item](https://arxiv.org/html/2609.02984#S2.I15.i2.p3.1)\.
- \[414\]B\. Yan, Y\. Cao, H\. Wang, W\. Yang, J\. Du, and C\. Shi\(2024\)Federated heterogeneous graph neural network for privacy\-preserving recommendation\.InACM Web Conference,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I23.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I35.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I8.i3.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px1.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p1.1),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.7.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.38.7),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.1.1)\.
- \[415\]H\. H\. Yang, Z\. Liu, T\. Q\. Quek, and H\. V\. Poor\(2019\)Scheduling policies for federated learning in wireless networks\.IEEE Transactions on Communications68\(1\),pp\. 317–333\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px3.p1.1)\.
- \[416\]M\. Yang, X\. Wang, H\. Zhu, H\. Wang, and H\. Qian\(2021\)Federated learning with class imbalance reduction\.InIEEE European Signal Processing Conference,Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I7.i1.p1.1)\.
- \[417\]Q\. Yang, Y\. Liu, T\. Chen, and Y\. Tong\(2019\)Federated machine learning: concept and applications\.ACM Transactions on Intelligent Systems and Technology10\(2\),pp\. 1–19\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.10.2)\.
- \[418\]T\. Yang, X\. Yi, J\. Wu, Y\. Yuan, D\. Wu, Z\. Meng, Y\. Hong, H\. Wang, Z\. Lin, and K\. H\. Johansson\(2019\)A survey of distributed optimization\.Annual Reviews in Control47,pp\. 278–305\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.3.2.2.1.2.1)\.
- \[419\]T\. Yang, G\. Andrew, H\. Eichner, H\. Sun, W\. Li, N\. Kong, D\. Ramage, and F\. Beaufays\(2018\)Applied federated learning: improving Google keyboard query suggestions\.Technical reportGoogle\.Cited by:[§2\.2\.4](https://arxiv.org/html/2609.02984#S2.SS2.SSS4.p1.1),[§2\.3\.1](https://arxiv.org/html/2609.02984#S2.SS3.SSS1.p1.1)\.
- \[420\]Y\. Yang, X\. Ye, and T\. Sakurai\(2022\)Multi\-view federated learning with data collaboration\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I12.i2.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S2.I6.i3.p1.1)\.
- \[421\]X\. Yao and L\. Sun\(2020\)Continual local training for better initialization of federated models\.InIEEE International Conference on Image Processing,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I5.i3.p1.1)\.
- \[422\]Y\. Yao, W\. Jin, S\. Ravi, and C\. Joe\-Wong\(2023\)FedGCN: convergence\-communication tradeoffs in federated training of graph convolutional networks\.Advances in Neural Information Processing Systems36,pp\. 79748–79760\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I26.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I27.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I28.i2.p1.1),[§3\.2](https://arxiv.org/html/2609.02984#S3.SS2.SSS0.Px2.p5.2),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.25.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.38.2)\.
- \[423\]G\. Yehudai, E\. Fetaya, E\. Meirom, G\. Chechik, and H\. Maron\(2021\)From local structures to size generalization in graph neural networks\.InInternational Conference on Machine Learning,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I9.i2.p1.1)\.
- \[424\]Y\. You, T\. Chen, Y\. Sui, T\. Chen, Z\. Wang, and Y\. Shen\(2020\)Graph contrastive learning with augmentations\.Advances in Neural Information Processing Systems33,pp\. 5812–5823\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I15.i2.p1.1),[§3\.1](https://arxiv.org/html/2609.02984#S3.SS1.SSS0.Px4.p1.2),[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3)\.
- \[425\]H\. Yu, S\. Yang, and S\. Zhu\(2019\)Parallel restarted SGD with faster convergence and less communication: demystifying why model averaging works for deep learning\.InAAAI Conference on Artificial Intelligence,Cited by:[§2\.1](https://arxiv.org/html/2609.02984#S2.SS1.SSS0.Px3.p1.1),[§2\.3](https://arxiv.org/html/2609.02984#S2.SS3.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p4.1),[1st item](https://arxiv.org/html/2609.02984#S3.I27.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I27.i2.p1.1)\.
- \[426\]H\. Yu, Y\. Zhang, L\. He, D\. He, Q\. Li, J\. Zhao, X\. Liao, H\. Jin, L\. Gu, and H\. Liu\(2024\)CDA\-GNN: a chain\-driven accelerator for efficient asynchronous graph neural network\.InACM/IEEE Design Automation Conference,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I30.i1.p1.1)\.
- \[427\]E\. L\. Zec, E\. Ekblom, M\. Willbo, O\. Mogren, and S\. Girdzijauskas\(2022\)Decentralized adaptive clustering of deep nets is beneficial for client collaboration\.InInternational Workshop on Trustworthy Federated Learning,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1),[4th item](https://arxiv.org/html/2609.02984#S3.I17.i4.p1.1)\.
- \[428\]E\. L\. Zec, J\. Östman, O\. Mogren, and D\. Gillblad\(2024\)Efficient node selection in private personalized decentralized learning\.InNorthern Lights Deep Learning Conference,Cited by:[4th item](https://arxiv.org/html/2609.02984#S2.I8.i4.p1.1)\.
- \[429\]L\. Zeng, C\. Yang, P\. Huang, Z\. Zhou, S\. Yu, and X\. Chen\(2022\)GNN at the edge: cost\-efficient graph neural network processing over distributed edge servers\.IEEE Journal on Selected Areas in Communications41\(3\),pp\. 720–739\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I22.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I23.i2.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I25.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I31.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I7.i1.I1.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1),[§3\.5\.3](https://arxiv.org/html/2609.02984#S3.SS5.SSS3.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.2.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.36.2)\.
- \[430\]C\. Zhang, Y\. Xie, H\. Bai, B\. Yu, W\. Li, and Y\. Gao\(2021\)A survey on federated learning\.Knowledge\-Based Systems216,pp\. 106775\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.10.2)\.
- \[431\]C\. Zhang, S\. Zhang, J\. James, and S\. Yu\(2021\)FASTGNN: a topological information protected federated learning approach for traffic speed forecasting\.IEEE Transactions on Industrial Informatics17\(12\),pp\. 8464–8474\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.3.1)\.
- \[432\]H\. Zhang, M\. Cisse, Y\. N\. Dauphin, and D\. Lopez\-Paz\(2018\)Mixup: beyond empirical risk minimization\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I6.i2.p1.1)\.
- \[433\]H\. Zhang, T\. Shen, F\. Wu, M\. Yin, H\. Yang, and C\. Wu\(2021\)Federated graph learning–a position paper\.arXiv:2105\.11099\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.17.2.2.1.1.1),[§3\.3\.1](https://arxiv.org/html/2609.02984#S3.SS3.SSS1.p1.1)\.
- \[434\]J\. Zhang, L\. Yang, S\. M\. Sajjadi Mohammadabadi, and F\. Yan\(2025\)A survey on self\-supervised learning: recent advances and open problems\.Neurocomputing655,pp\. 131409\.Cited by:[§1\.1](https://arxiv.org/html/2609.02984#S1.SS1.p1.2)\.
- \[435\]K\. Zhang, Y\. Wang, H\. Wang, L\. Huang, C\. Yang, and L\. Sun\(2022\)Efficient federated learning on knowledge graphs via privacy\-preserving relation embedding aggregation\.InACL Workshop on Federated Learning for Natural Language Processing,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I24.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I34.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.7.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.7.2.1.4.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.7.2.1.3.1)\.
- \[436\]K\. Zhang, C\. Yang, X\. Li, L\. Sun, and S\. M\. Yiu\(2021\)Subgraph federated learning with missing neighbor generation\.Advances in Neural Information Processing Systems34,pp\. 6671–6682\.Cited by:[4th item](https://arxiv.org/html/2609.02984#S1.I1.i4.p1.1),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.17.2.2.1.1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.2),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.9.2)\.
- \[437\]K\. Zhang, S\. Liu, S\. Wang, W\. Shi, C\. Chen, P\. Li, S\. Li, J\. Li, and K\. Ding\(2024\)A survey of deep graph learning under distribution shifts: from graph out\-of\-distribution generalization to adaptation\.arXiv:2410\.19265\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I9.i1.p1.1),[§3\.3\.2](https://arxiv.org/html/2609.02984#S3.SS3.SSS2.Px1.p1.1),[§3\.3\.2](https://arxiv.org/html/2609.02984#S3.SS3.SSS2.p3.1)\.
- \[438\]T\. Zhang, C\. Mai, Y\. Chang, C\. Chen, L\. Shu, and Z\. Zheng\(2023\)FedEgo: privacy\-preserving personalized federated graph learning with ego\-graphs\.ACM Transactions on Knowledge Discovery from Data18\(2\),pp\. 1–27\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I17.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I18.i1.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I35.i1.p1.1),[§3\.3\.3](https://arxiv.org/html/2609.02984#S3.SS3.SSS3.Px2.p1.1),[§3\.4\.1](https://arxiv.org/html/2609.02984#S3.SS4.SSS1.Px1.p2.3),[§3\.6](https://arxiv.org/html/2609.02984#S3.SS6.SSS0.Px3.p3.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.12.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.16.10.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.17.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.24.10.2.1.2.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.10),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.40.10)\.
- \[439\]Y\. Zhang, M\. J\. Wainwright, and J\. C\. Duchi\(2012\)Communication\-efficient algorithms for statistical optimization\.Advances in Neural Information Processing Systems25\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p3.1)\.
- \[440\]Z\. Zhang, Q\. Liu, Z\. Huang, H\. Wang, C\. Lu, C\. Liu, and E\. Chen\(2021\)GraphMI: extracting private graph data from graph neural networks\.InInternational Joint Conference on Artificial Intelligence,Cited by:[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p1.1)\.
- \[441\]Z\. Zhang, G\. Zhu, R\. Wang, V\. K\. N\. Lau, and K\. Huang\(2022\)Turning channel noise into an accelerator for over\-the\-air principal component analysis\.IEEE Transactions on Wireless Communications\.Cited by:[§2\.4\.3](https://arxiv.org/html/2609.02984#S2.SS4.SSS3.Px1.p1.1)\.
- \[442\]Z\. Zhang, Z\. Gao, Y\. Guo, and Y\. Gong\(2022\)Scalable and low\-latency federated learning with cooperative mobile edge networking\.IEEE Transactions on Mobile Computing23\(1\),pp\. 812–822\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p5.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p1.1)\.
- \[443\]B\. Zhao, K\. R\. Mopuri, and H\. Bilen\(2020\)iDLG: improved deep leakage from gradients\.arXiv:2001\.02610\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I4.i2.p1.1)\.
- \[444\]Y\. Zhao, M\. Li, L\. Lai, N\. Suda, D\. Civin, and V\. Chandra\(2018\)Federated learning with non\-iid data\.arXiv:1806\.00582\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I6.i1.p1.1)\.
- \[445\]X\. Zheng, Z\. Wang, C\. Chen, J\. Qian, and Y\. Yang\(2023\)Decentralized graph neural network for privacy\-preserving recommendation\.InACM International Conference on Information and Knowledge Management,Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I20.i1.p1.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I27.i3.p1.1),[2nd item](https://arxiv.org/html/2609.02984#S3.I33.i2.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.19.3),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.26.2.2.1.1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.39.7.2.1.2.1)\.
- \[446\]D\. Zhou, O\. Bousquet, T\. Lal, J\. Weston, and B\. Schölkopf\(2003\)Learning with local and global consistency\.Advances in Neural Information Processing Systems16\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S1.I1.i1.p1.1),[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.4.2),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p5.1)\.
- \[447\]L\. Zhou, V\. D\. Sharma, Q\. Li, A\. Prorok, A\. Ribeiro, P\. Tokekar, and V\. Kumar\(2021\)Graph neural networks for decentralized multi\-robot submodular action selection\.arXiv:2105\.08601\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S3.I25.i1.p1.1),[1st item](https://arxiv.org/html/2609.02984#S3.I8.i1.p1.1),[Table 2](https://arxiv.org/html/2609.02984#S3.T2.2.1.1.1.1.1.1.1.1.27.3.2.1.1.1)\.
- \[448\]S\. Zhou and G\. Y\. Li\(2023\)Federated learning via inexact ADMM\.IEEE Transactions on Pattern Analysis and Machine Intelligence45\(8\),pp\. 9699–9708\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I5.i1.p1.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p5.1)\.
- \[449\]X\. Zhou, X\. Ye, I\. Kevin, K\. Wang, W\. Liang, N\. K\. C\. Nair, S\. Shimizu, Z\. Yan, and Q\. Jin\(2023\)Hierarchical federated learning with social context clustering\-based participant selection for internet of medical things applications\.IEEE Transactions on Computational Social Systems10\(4\),pp\. 1742–1751\.Cited by:[§1](https://arxiv.org/html/2609.02984#S1.p5.1),[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px3.p2.1)\.
- \[450\]Z\. Zhou, Y\. Tian, and C\. Peng\(2021\)Privacy\-preserving federated learning framework with general aggregation and multiparty entity matching\.Wireless Communications and Mobile Computing2021\(1\),pp\. 6692061\.Cited by:[1st item](https://arxiv.org/html/2609.02984#S2.I10.i1.p1.1)\.
- \[451\]H\. Zhu, J\. Xu, S\. Liu, and Y\. Jin\(2021\)Federated learning on non\-iid data: a survey\.Neurocomputing465,pp\. 371–390\.Cited by:[Table 1](https://arxiv.org/html/2609.02984#S1.T1.2.12.2),[1st item](https://arxiv.org/html/2609.02984#S2.I2.i1.p1.1),[§2\.2\.2](https://arxiv.org/html/2609.02984#S2.SS2.SSS2.Px2.p1.1)\.
- \[452\]J\. Zhu, Y\. Yan, L\. Zhao, M\. Heimann, L\. Akoglu, and D\. Koutra\(2020\)Beyond homophily in graph neural networks: current limitations and effective designs\.Advances in Neural Information Processing Systems33,pp\. 7793–7804\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I10.i2.p1.1)\.
- \[453\]L\. Zhu, Z\. Liu, and S\. Han\(2019\)Deep leakage from gradients\.InAdvances in Neural Information Processing Systems,Cited by:[2nd item](https://arxiv.org/html/2609.02984#S2.I4.i2.p1.1),[§2\.5](https://arxiv.org/html/2609.02984#S2.SS5.SSS0.Px2.p1.1)\.
- \[454\]X\. Zhu and Z\. Ghahramani\(2002\)Learning from labeled and unlabeled data with label propagation\.Technical reportCarnegie Mellon University\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px2.p5.1),[3rd item](https://arxiv.org/html/2609.02984#S3.I29.i3.p1.1)\.
- \[455\]X\. J\. Zhu\(2005\)Semi\-supervised learning literature survey\.CS Technical Reports University of Wisconsin\-Madision\.Cited by:[§2\.2\.1](https://arxiv.org/html/2609.02984#S2.SS2.SSS1.p2.1)\.
- \[456\]Z\. Zhu, J\. Hong, and J\. Zhou\(2021\)Data\-free knowledge distillation for heterogeneous federated learning\.InInternational Conference on Machine Learning,Cited by:[3rd item](https://arxiv.org/html/2609.02984#S2.I9.i3.p1.1)\.
- \[457\]M\. Zinkevich, J\. Langford, and A\. Smola\(2009\)Slow learners are fast\.Advances in Neural Information Processing Systems22\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p2.1)\.
- \[458\]M\. Zinkevich, M\. Weimer, L\. Li, and A\. Smola\(2010\)Parallelized stochastic gradient descent\.Advances in Neural Information Processing Systems23\.Cited by:[§2\.4\.1](https://arxiv.org/html/2609.02984#S2.SS4.SSS1.Px1.p3.1)\.
- \[459\]D\. Zügner, O\. Borchert, A\. Akbarnejad, and S\. Günnemann\(2020\)Adversarial attacks on graph neural networks: perturbations and their patterns\.ACM Transactions on Knowledge Discovery from Data14\(5\),pp\. 1–31\.Cited by:[2nd item](https://arxiv.org/html/2609.02984#S3.I34.i2.p1.1),[§3\.3\.4](https://arxiv.org/html/2609.02984#S3.SS3.SSS4.Px1.p2.1)\.

## Appendix ASurvey map, Notations and Glossary

### Organization of the solutions

![Refer to caption](https://arxiv.org/html/2609.02984v1/images/organisational_tree.png)Figure 21:The taxonomy of the solutions surveyed for effective, efficient, and privacy\-preserving collaborative learning for Euclidean and graph data\.
### Glossary

In this work, we use several terms whose definitions are either subjective or inconsistently used in the literature; we clarify them here for consistency and clarity\. These definitions are limited to the present work\.

AgentAn autonomous entity with local data, computational capabilities, and the ability to communicate with other agents\.

ServerA central coordinating unit that supervises a group of agents, possessing computational capabilities and the ability to communicate with agents\.

DistributedA property of a system in which some or all components \(e\.g\., data, computation, or control\) are spread across multiple agents\.

Data parallelismA property of a distributed system where data collected centrally is distributed across multiple agents\.

Model parallelismA property of a distributed system where parts of the model maintained centrally are distributed across multiple agents\.

Collaborative \(system\)A distributed system of agents \(either federated or decentralized\) where agents cooperate to achieve shared objectives, though their willingness to collaborate may be limited by local constraints or incentives such as, but not limited to, local objective, computational and communication capabilities, data availability, or privacy requirements\.

Federated \(system\)A distributed system of agents that are federated under the supervision of a central server coordinates\.

Decentralized \(system\)A distributed system of agents where all agents operate without a central server\.

LocalDescribes a property or object that pertains to a single agent \(e\.g\., local data, local model\)\.

GlobalDescribes a property or object shared across or computed from all or many agents\. It does not imply that each agent has a full view of this property \(e\.g\., global graph\)\.

TopologyThe discrete structure describing the communication or computational relationships between agents, typically modeled as a graph \(see\([154](https://arxiv.org/html/2609.02984#bib.bib413)\)for a formal equivalence between topology and graph\)\.

SampleA single data point drawn from a dataset\.

FeatureAn observed attribute or measurable property of a sample\.

LabelThe target property or variable of a sample that we aim to infer from its features\.

TrainingThe process where models learn patterns from data by optimizing parameters to minimize prediction errors on the training dataset\.

InferenceThe process by which the model predicts the label of a sample based on its features using learned parameters\.

LearningThe comprehensive framework encompasses various methodologies, algorithms, and theoretical foundations for training models to perform inference tasks effectively\.

EuclideanPertaining to data where each sample is represented as a feature vector inℝn\\mathbb\{R\}^\{n\}equipped with the standard Euclidean inner product\.

Non\-EuclideanPertaining to data with inherent relational structure that cannot be faithfully represented inℝn\\mathbb\{R\}^\{n\}with the Euclidean inner product\. Graphs are the most commonly used example of discrete non\-Euclidean structures\.

### Notations

Agents and Data𝒟k\\mathcal\{D\}\_\{k\},𝒟\\mathcal\{D\}Local and global data distributionsℐk\\mathcal\{I\}\_\{k\},ℐ\\mathcal\{I\},iiLocal and global index sets of samples, and sample index𝒦\\mathcal\{K\},KK,kk,k∗,s,tk^\{\*\},s,tSet of agents, number of agents, agent index, active agent, source and target agentnkn\_\{k\},nnNumber of samples in local and global datasetsΩ\\OmegaSimilarity matrix representing a latent graph between agentsΠ\\Pi,πi\\pi\_\{i\}State of a Markov chain and itsii\-th elementSkS\_\{k\},SSLocal and global datasetsW∈ℝK×KW\\in\\mathbb\{R\}^\{K\\times K\}Communication topology matrix between agentsFeatures, Labels, Representationsdxd\_\{x\},dyd\_\{y\},dzd\_\{z\}Dimensions of feature vector, label space, and embedding spacexx,𝒳\\mathcal\{X\}Feature sample and feature domainxix^\{i\},xkix^\{i\}\_\{k\}Feature of theii\-th sample, global and localxkx\_\{k\},𝒳k\\mathcal\{X\}\_\{k\}Local fraction of featurexxin local domain𝒳k\\mathcal\{X\}\_\{k\}yy,𝒴\\mathcal\{Y\}Label and label domainyiy^\{i\},y^i\\hat\{y\}^\{i\}Label of theii\-th sample and its inferred valuezz,𝒵\\mathcal\{Z\}Representation of sample\(x,y\)\(x,y\)and representation domainzkz\_\{k\},𝒵k\\mathcal\{Z\}\_\{k\}Local fraction of global representationzzin local domain𝒵k\\mathcal\{Z\}\_\{k\}Graphs and TopologyAAAdjacency matrix of𝒢\\mathcal\{G\}ℰk\\mathcal\{E\}\_\{k\},ℰ\\mathcal\{E\}Local and global sets of edgesℰc​u​t,k\\mathcal\{E\}\_\{cut,k\}Local set of cut edgesG=\(A,X\)G=\(A,X\)Graph data sample in matrix formGkG\_\{k\},GkiG\_\{k\}^\{i\}Local subgraph andii\-th local graph sampleGvG^\{v\},GkvG^\{v\}\_\{k\}Global and localLL\-hop neighborhoods of nodevv𝒢k\\mathcal\{G\}\_\{k\},𝒢\\mathcal\{G\}Local and global graph structures \(homogeneous\)G~k\\tilde\{G\}\_\{k\},𝒢~k\\tilde\{\\mathcal\{G\}\}\_\{k\}Local subgraph data and structure \(fromGkG\_\{k\}and𝒢k\\mathcal\{G\}\_\{k\}\), respectively, expanded toLL\-hop neighborhoodsℋ𝒢\\mathcal\{H\}\_\{\\mathcal\{G\}\}Homogeneous projection of heterogeneous graphNkN\_\{k\},NNNumber of vertices in local and global vertex sets𝒪k\\mathcal\{O\}\_\{k\},𝒪\\mathcal\{O\}Local and global sets of object typesℛk\\mathcal\{R\}\_\{k\},ℛ\\mathcal\{R\}Local and global sets of relation typesρk\\rho\_\{k\},ρ\\rhoLocal and global relation type mappings𝒯v\\mathcal\{T\}^\{v\}Computational tree for target nodevvτk\\tau\_\{k\},τ\\tauLocal and global node type mappings𝒱k\\mathcal\{V\}\_\{k\},𝒱\\mathcal\{V\},vvLocal and global sets of vertices \(nodes\), and vertex index𝒱f,k\\mathcal\{V\}\_\{f,k\},𝒱p,k\\mathcal\{V\}\_\{p,k\},𝒱c​u​t,k\\mathcal\{V\}\_\{cut,k\}Sets of foreign nodes, private nodes, and vertex cutNode\-Level Quantitieshv\(l\)h\_\{v\}^\{\(l\)\},hv,k\(l\)h\_\{v,k\}^\{\(l\)\}Global and local embeddings of nodevvat layerllXXFeature matrix of the graphxvx\_\{v\},xv,kx\_\{v,k\}Global and local features of nodevvzvz\_\{v\},zv,kz\_\{v,k\}Global and local node representation of nodevvModels and LearningEE,eeNumber of local iterations and local iteration indexFkF\_\{k\},ℱk\\mathcal\{F\}\_\{k\}Local model and its function spaceFF,ℱ\\mathcal\{F\}Global model and its function spaceFθF\_\{\\theta\},Fθ,ψF\_\{\\theta,\\psi\}ModelFFparametrized byθ\\thetaand by\(θ,ψ\)\(\\theta,\\psi\)fM​e​s\(l\)f\_\{Mes\}^\{\(l\)\},fA​g​g\(l\)f\_\{Agg\}^\{\(l\)\},fU​p\(l\)f\_\{Up\}^\{\(l\)\}Message, aggregation, and update functions at layerllfψf\_\{\\psi\}Predictor parametrized byψ\\psiℋ\\mathcal\{H\}Hypernetworkℓ\\ellPairwise lossLL,llNumber of layers and layer indexℒk\\mathcal\{L\}\_\{k\},ℒ\\mathcal\{L\}Local and global population lossesℒ^k\\hat\{\\mathcal\{L\}\}\_\{k\},ℒ^\\hat\{\\mathcal\{L\}\}Local and global empirical lossesϕk\\phi\_\{k\},ϕ\\phiLocal and global encodersϕk,θ\\phi\_\{k,\\theta\},ϕθ\\phi\_\{\\theta\}Local and global encoders parametrized byθ\\thetaϕn​o​d​e\\phi\_\{node\},ϕe​d​g​e\\phi\_\{edge\},ϕg​r​a​p​h\\phi\_\{graph\}Node, edge, and graph encoder functionsψ\\psi,Ψ\\PsiPredictor parameters and parameter spaceTT,ttNumber of global iterations and global iteration indexθk\\theta\_\{k\},θ\\theta,Θk\\Theta\_\{k\},Θ\\ThetaLocal and global model parameters and parameter domainsPrivacy and Differential Privacyδ\\deltaFailure probability in differential privacyε\\varepsilonPrivacy budget in differential privacyℳ\\mathcal\{M\}Randomized mechanism \(training algorithm\) for differential privacyUUSet of possible outcomes of mechanismℳ\\mathcal\{M\}

Table 3:Notations used in this paper, organized by comprehensive categories and alphabetical order\.