Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity

# Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity
Source: [https://arxiv.org/html/2605.12584](https://arxiv.org/html/2605.12584)
Sirui Zhang Haonan Wang Xunkai Li Zekai Chen Shumeng Li Hongchao Qin Rong\-Hua Li Guoren Wang Beijing Institute of Technology

###### Abstract

Recently, multimodal graph learning \(MGL\) has garnered significant attention for integrating diverse modality information and structured context to support various network applications\. However, real\-world graphs are often isolated due to data\-sharing limitations across multiple parties, and their modalities are frequently incomplete\. This highlights an urgent need to develop a robust federated approach\. However, we find that existing methods remain insufficient\. On the one hand, centralized MGL methods that handle missing modalities overlook the knowledge sharing and generalization in federated scenarios\. On the other hand, while federated MGL methods have become increasingly mature, they primarily target non\-graph data\. Based on these technologies, we identify a two\-stage pipeline wherein client\-side completion reconstructs missing modalities, and server\-side aggregation integrates the client\-updated parameters of both the modality generator and the backbone models\. Although this serves as a general solution, we identify two primary challenges in achieving greater robustness:\(1\) Topology\-Isolated Local Completion: Client\-side modality generation struggles to effectively leverage global semantics\.\(2\) Reliability\-Imbalanced Global Aggregation: Server\-side multi\-party collaboration is hindered by client updates with varying modality availability and recovery reliability\. To address these challenges, we proposeFedMPO, which utilizes topology\-aware cross\-modal generation to recover missing features using comprehensive graph context, missing\-aware expert routing to locally filter out noisy recovered signals, and reliability\-aware aggregation to appropriately down\-weight unreliable updates\. Extensive experiments on 3 tasks across 6 datasets demonstrate thatFedMPOoutperforms baselines, achieving performance gains of up to 4\.10% and 5\.65% in high\-missing and non\-IID settings\.

## 1Introduction

Multimodal graph learning models real\-world systems by jointly exploiting graph topology and heterogeneous modalities such as text and imagesWeiet al\.\([2019](https://arxiv.org/html/2605.12584#bib.bib14)\); Taoet al\.\([2020](https://arxiv.org/html/2605.12584#bib.bib15)\); Zhuet al\.\([2025](https://arxiv.org/html/2605.12584#bib.bib21)\)\. In practice, such graphs are often isolated across clients due to privacy constraints, commercial competition, or platform\-specific ownership, making centralized training infeasibleMcMahanet al\.\([2017](https://arxiv.org/html/2605.12584#bib.bib1)\); Liet al\.\([2026](https://arxiv.org/html/2605.12584#bib.bib24)\)\. Federated learning methodsMcMahanet al\.\([2017](https://arxiv.org/html/2605.12584#bib.bib1)\); Liet al\.\([2020](https://arxiv.org/html/2605.12584#bib.bib2)\)enable collaboration without sharing raw data, but modalities are frequently incomplete because of heterogeneous collection, storage limitations, privacy restrictions, or user behavior differences\. Therefore, missing\-modality robust learning for federated multimodal graphs becomes a crucial yet underexplored problem in realistic distributed graph applications\.

![Refer to caption](https://arxiv.org/html/2605.12584v1/x1.png)Figure 1:Overview of missing\-modality learning paradigms and the proposedFedMPO\.As shown in Fig\.[1](https://arxiv.org/html/2605.12584#S1.F1), existing studies provide two complementary but incomplete directions\. Centralized incomplete multimodal graph learning exploits graph topology and multimodal attributes to generate missing modalities before downstream graph learningJiaet al\.\([2023](https://arxiv.org/html/2605.12584#bib.bib16)\); Heet al\.\([2025](https://arxiv.org/html/2605.12584#bib.bib17)\); Wanet al\.\([2026](https://arxiv.org/html/2605.12584#bib.bib23)\); Wanget al\.\([2023](https://arxiv.org/html/2605.12584#bib.bib38)\); Wu and others \([2024](https://arxiv.org/html/2605.12584#bib.bib27)\)\. It uses graph context as evidence for node\-level modality generation, but assumes centralized data access and cannot benefit from cross\-client knowledge\. In contrast, federated missing\-modality learning methods aggregate locally updated modality generators on the server and broadcast global knowledge back to clientsCheet al\.\([2024](https://arxiv.org/html/2605.12584#bib.bib4)\); Nguyenet al\.\([2024](https://arxiv.org/html/2605.12584#bib.bib5)\), improving local modality generation through collaboration\. However, they mainly target non\-graph data and ignore topology\-aware graph context\. When adapted to federated multimodal graphs, they form a generation–aggregation pipeline, where clients conduct local modality generation and graph learning while the server aggregates modality\-generator and graph\-backbone updates\. Since each client only observes its own subgraph, missing features are generated from locally biased graph context and partial modality observations rather than globally consistent cross\-client structural semantics\. Thus, this pipeline remains limited by client\-isolated local generation and reliability\-skewed global aggregation\.

We identify two key challenges in this pipeline\.\(1\) Topology\-Isolated Local Completion, where modality generation is performed within client\-owned subgraphs and struggles to leverage global semantics beyond local graph context\. Although local topology provides useful graph context, sparse observations and biased client subgraphs may still generate features that are inconsistent with cross\-client structural semantics, further misleading graph message passing\.\(2\) Reliability\-Imbalanced Global Aggregation\. Since clients differ in modality availability, missingness patterns, and generation reliability, data\-size\-based aggregation may bias the shared modality generator and graph backbone toward unreliable generated signals or dominant modality patterns\.

To address these challenges, we proposeFedMPO, a unified missing\-modality robust framework for federated multimodal graphs\.FedMPOfollows a four\-stage pipeline\.\(1\) Client\-side feature and graph context encodingdistinguishes natural missingness from training\-time masking and builds neighborhood\-guided structural anchors as topology priors for modality generation \(Sec\.[3\.1](https://arxiv.org/html/2605.12584#S3.SS1)\)\.\(2\) Client\-side topology\-aware cross\-modal generationrecovers missing features by querying cross\-modal context banks and graph context, addressing topology\-isolated local completion \(Sec\.[3\.2](https://arxiv.org/html/2605.12584#S3.SS2)\)\.\(3\) Client\-side missing\-aware mixture\-of\-experts routingestimates generation uncertainty and fuses observed, generated, and structural signals to filter noisy recovered signals before local update \(Sec\.[3\.3](https://arxiv.org/html/2605.12584#S3.SS3)\)\.\(4\) Server\-side reliability\-aware aggregationaggregates client\-updated modality\-generator and graph\-backbone parameters, and uses reliability statistics to down\-weight unreliable updates, mitigating reliability\-imbalanced global aggregation \(Sec\.[3\.4](https://arxiv.org/html/2605.12584#S3.SS4)\)\.

Our contributions\.\(1\)New Problem: We study missing\-modality robust learning for federated multimodal graphs, a practical yet underexplored problem where graph\-structured multimodal data are distributed and often incomplete\. \(2\)New Method: We proposeFedMPO, which integrates topology\-aware cross\-modal generation, missing\-aware expert routing, and reliability\-aware aggregation into a unified client–server workflow to improve local modality generation reliability and global robustness\. \(3\)SOTA Performance: Experiments on three downstream tasks across six datasets under challenging missing and non\-IID conditions demonstrate the effectiveness ofFedMPO, achieving up to 4\.10% and 5\.65% gains under high\-missing and highly non\-IID settings, respectively\.

## 2Preliminaries & Related Work

### 2\.1Notations and Problem Formulation

We study missing\-modality robust learning on federated multimodal graphs\. Consider a federated system withKKclients and a central server\. Each clientkkowns a private multimodal graph𝒢k=\(𝒱k,ℰk\)\\mathcal\{G\}\_\{k\}=\(\\mathcal\{V\}\_\{k\},\\mathcal\{E\}\_\{k\}\), where nodes denote local entities and edges encode relational dependencies\. Each nodevk,i∈𝒱kv\_\{k,i\}\\in\\mathcal\{V\}\_\{k\}is associated withMMmodality\-specific attributes𝐱k,i=\{𝐱k,i\(1\),…,𝐱k,i\(M\)\}\\mathbf\{x\}\_\{k,i\}=\\\{\\mathbf\{x\}\_\{k,i\}^\{\(1\)\},\\ldots,\\mathbf\{x\}\_\{k,i\}^\{\(M\)\}\\\}, where𝐱k,i\(m\)\\mathbf\{x\}\_\{k,i\}^\{\(m\)\}denotes the feature of modalitymm\. Due to privacy constraints and data ownership, clients cannot directly exchange raw node attributes, modality contents, or local graph structures\. They may also exhibit heterogeneous feature, label, topology, and missingness distributions, leading to coupled non\-IID challenges across multimodal graph clients\.

To describe incomplete observations, we define a modality\-availability mask𝐫k,i∈\{0,1\}M\\mathbf\{r\}\_\{k,i\}\\in\\\{0,1\\\}^\{M\}, whererk,i\(m\)=1r\_\{k,i\}^\{\(m\)\}=1indicates that modalitymmis observed andrk,i\(m\)=0r\_\{k,i\}^\{\(m\)\}=0otherwise\. Thus, the accessible input of nodevk,iv\_\{k,i\}consists of its observed modalities and local graph context, denoted as𝐱~k,i=\{𝐱k,i\(m\)∣rk,i\(m\)=1\}\\widetilde\{\\mathbf\{x\}\}\_\{k,i\}=\\\{\\mathbf\{x\}\_\{k,i\}^\{\(m\)\}\\mid r\_\{k,i\}^\{\(m\)\}=1\\\}\. Missingness may occur at both the client level and the node level\. We distinguish natural missingness from training\-time masking: naturally missing modalities have no ground\-truth features, while artificially masked observed modalities provide reconstruction supervision\.

For a downstream task𝒯\\mathcal\{T\}, such as node classification, link prediction, or modality retrieval, the learning process follows three steps\.First, each client performs local learning on its private incomplete multimodal graph by exploiting available modalities and graph neighborhoods\.Second, missing modalities are recovered or represented locally, but their reliability can vary across clients because of different modality availability and structural context\.Third, the server aggregates client updates to obtain a global model and broadcasts it back for the next communication round\.

This formulation highlights two core requirements\. Locally, missing\-modality recovery should be topology\-aware, since node semantics are shaped by both its own observed modalities and its neighbors through local structural context\. Globally, federated aggregation should be reliability\-aware, since data\-size\-based aggregation may over\-emphasize client updates dominated by unreliable recovered signals\. Therefore, the goal ofFedMPOis to learn robust local representations from incomplete multimodal graphs and integrate the resulting client updates into a reliable global model without sharing raw multimodal attributes or graph structures\.

### 2\.2Related Work

Graph and multimodal graph learning\.GNNs learn node representations by aggregating neighborhood information, with representative models including GCN, GraphSAGE, GAT, and GINKipf and Welling \([2017](https://arxiv.org/html/2605.12584#bib.bib10)\); Hamiltonet al\.\([2017](https://arxiv.org/html/2605.12584#bib.bib11)\); Veličkovićet al\.\([2018](https://arxiv.org/html/2605.12584#bib.bib12)\); Xuet al\.\([2019](https://arxiv.org/html/2605.12584#bib.bib13)\)\. Multimodal graph learning further incorporates heterogeneous attributes such as text and images into graph representation learningWeiet al\.\([2019](https://arxiv.org/html/2605.12584#bib.bib14)\); Taoet al\.\([2020](https://arxiv.org/html/2605.12584#bib.bib15)\); Jiaet al\.\([2023](https://arxiv.org/html/2605.12584#bib.bib16)\); Heet al\.\([2025](https://arxiv.org/html/2605.12584#bib.bib17)\)\. In such graphs, topology not only connects related entities but also provides contextual evidence for resolving modality ambiguity and enhancing incomplete node semantics\. Recent benchmarks highlight the importance of cross\-modal alignment, modality fusion, and graph topology in multimodal graph scenariosZhuet al\.\([2025](https://arxiv.org/html/2605.12584#bib.bib21)\); Wanet al\.\([2026](https://arxiv.org/html/2605.12584#bib.bib23)\)\. However, most existing methods assume centralized data access and complete or directly usable modalities, making them unsuitable for privacy\-constrained federated settings with missing modalities\.

Federated graph and multimodal federated learning\.Federated learning enables collaborative training across distributed clients without raw data sharing, and methods such as FedAvg, FedProx, and SCAFFOLD address optimization under data heterogeneityMcMahanet al\.\([2017](https://arxiv.org/html/2605.12584#bib.bib1)\); Liet al\.\([2020](https://arxiv.org/html/2605.12584#bib.bib2)\); Karimireddyet al\.\([2020](https://arxiv.org/html/2605.12584#bib.bib3)\)\. Federated graph learning extends this paradigm to graph data, mainly focusing on structural heterogeneity, client drift, and communication efficiencyLiet al\.\([2025](https://arxiv.org/html/2605.12584#bib.bib9)\)\. Multimodal federated learning studies modality heterogeneity and partial\-modality clientsCheet al\.\([2024](https://arxiv.org/html/2605.12584#bib.bib4)\); Nguyenet al\.\([2024](https://arxiv.org/html/2605.12584#bib.bib5)\); Penget al\.\([2024b](https://arxiv.org/html/2605.12584#bib.bib6)\); Wuet al\.\([2024](https://arxiv.org/html/2605.12584#bib.bib7)\); Xieet al\.\([2024](https://arxiv.org/html/2605.12584#bib.bib8)\)\. Nevertheless, most FGL methods assume complete node features, while multimodal FL methods usually treat samples independently and ignore graph\-structured dependencies\. Recent benchmark studies further show that naive combinations of FGL and multimodal FL are insufficient for practical multimodal graph scenarios with incomplete modalitiesLiet al\.\([2026](https://arxiv.org/html/2605.12584#bib.bib24)\)\.

Missing\-modality learning\.Incomplete multimodal learning handles partially missing modalities through cross\-modal reconstruction, generative imputation, shared–specific representation learning, and missing\-aware fusionMaet al\.\([2021](https://arxiv.org/html/2605.12584#bib.bib36),[2022](https://arxiv.org/html/2605.12584#bib.bib37)\); Wanget al\.\([2023](https://arxiv.org/html/2605.12584#bib.bib38)\); Rezaet al\.\([2024](https://arxiv.org/html/2605.12584#bib.bib39)\); Wu and others \([2024](https://arxiv.org/html/2605.12584#bib.bib27)\)\. Although effective in centralized or non\-graph settings, these methods do not explicitly model neighborhood\-dependent semantics or cross\-client aggregation bias, especially under heterogeneous client\-level modality missingness\. In multimodal graphs, unreliable recovered features may be propagated through message passing and further amplified by federated aggregation\.FedMPOaddresses this gap by integrating topology\-aware completion, missing\-aware routing, and reliability\-aware aggregation into a unified federated multimodal graph learning framework\.

## 3Methodology

We presentFedMPO, a missing\-modality robust framework for federated multimodal graph learning\. Modality incompleteness affects both local graph representation learning and global federated optimization: locally, missing modalities should exploit neighborhood structural and semantic evidence rather than be recovered independently; globally, clients with severe missingness or unreliable recovery may bias model aggregation\.

As shown in Fig\.[2](https://arxiv.org/html/2605.12584#S3.F2),FedMPOuses a four\-stage pipeline\. Each client first encodes available modalities and graph context into a shared space, then generates missing modalities with cross\-modal evidence and structural anchors\. The observed, recovered, and structural signals are further fused by a missing\-aware MoE module, followed by local prediction and reliability\-aware server aggregation\.

![Refer to caption](https://arxiv.org/html/2605.12584v1/x2.png)Figure 2:The overview of the proposedFedMPO\.### 3\.1Stage 1: Feature and Graph Context Encoding

Each client owns an incomplete multimodal graph, where different nodes may have different available modalities\. For nodeviv\_\{i\}and modalitymm, we denote the natural modality availability byri\(m\)∈\{0,1\}r\_\{i\}^\{\(m\)\}\\in\\\{0,1\\\}\. Since naturally missing modalities have no ground\-truth features, they cannot be directly used as reconstruction targets\. To obtain reliable supervision for modality generation, we apply an artificial mask only to naturally observed modalities\. The effective visibility mask, artificial reconstruction indicator, and raw modality embedding are defined as

r~i\(m\)=ri\(m\)​r¯i\(m\),Δi\(m\)=ri\(m\)−r~i\(m\),𝐳i,raw\(m\)=ϕm​\(𝐱i\(m\)\)\.\\tilde\{r\}\_\{i\}^\{\(m\)\}=r\_\{i\}^\{\(m\)\}\\bar\{r\}\_\{i\}^\{\(m\)\},\\qquad\\Delta\_\{i\}^\{\(m\)\}=r\_\{i\}^\{\(m\)\}\-\\tilde\{r\}\_\{i\}^\{\(m\)\},\\qquad\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}=\\phi\_\{m\}\(\\mathbf\{x\}\_\{i\}^\{\(m\)\}\)\.\(1\)Here,r~i\(m\)\\tilde\{r\}\_\{i\}^\{\(m\)\}indicates whether modalitymmis visible to the model, andΔi\(m\)=1\\Delta\_\{i\}^\{\(m\)\}=1means that an originally observed modality is deliberately hidden for training the generator\. This design separates natural missingness from training\-time masking, so that the model learns from reliable reconstruction targets without assuming unavailable modalities are known\.

A key difference between multimodal graphs and ordinary multimodal samples is that node semantics are shaped not only by their own attributes, but also by their neighbors\. Therefore, before generating missing modalities,FedMPOfirst constructs a modality\-wise structural anchor from neighboring nodes whose modality\-mmfeatures are visible:

𝐳i,anc\(m\)=∑vj∈𝒩​\(i\)ai​j​r~j\(m\)​𝐳j,raw\(m\)∑vj∈𝒩​\(i\)ai​j​r~j\(m\)\+ϵ\.\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}=\\frac\{\\sum\_\{v\_\{j\}\\in\\mathcal\{N\}\(i\)\}a\_\{ij\}\\tilde\{r\}\_\{j\}^\{\(m\)\}\\mathbf\{z\}\_\{j,\\mathrm\{raw\}\}^\{\(m\)\}\}\{\\sum\_\{v\_\{j\}\\in\\mathcal\{N\}\(i\)\}a\_\{ij\}\\tilde\{r\}\_\{j\}^\{\(m\)\}\+\\epsilon\}\.\(2\)Here,ai​ja\_\{ij\}denotes normalized neighbor weights, implemented with degree\- or entropy\-aware normalization to suppress high\-degree or redundant neighbors\. The anchor acts as a topology\-derived prior for the missing modality\. If no valid neighbor is available, we use a learnable null token or zero vector as the fallback\. In this way, a missing modality is not inferred solely from the current node’s other modalities, but also from structurally related nodes\.

We then feed either the visible raw embedding or its structural anchor into a graph encoder with lightweight modality adapters:

𝐡i\(m\)=GNNθ​\(ψm​\(r~i\(m\)​𝐳i,raw\(m\)\+\(1−r~i\(m\)\)​𝐳i,anc\(m\)\),𝒢\)\.\\mathbf\{h\}\_\{i\}^\{\(m\)\}=\\mathrm\{GNN\}\_\{\\theta\}\\left\(\\psi\_\{m\}\\left\(\\tilde\{r\}\_\{i\}^\{\(m\)\}\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\+\(1\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\)\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\\right\),\\mathcal\{G\}\\right\)\.\(3\)The output𝐡i\(m\)\\mathbf\{h\}\_\{i\}^\{\(m\)\}is a graph\-enhanced representation for modalitymm\. For reconstructing a target modality, we further pool the representations of other modalities into a target\-exclusive context𝐡i,∖m\\mathbf\{h\}\_\{i,\\setminus m\}\. This prevents the generator from trivially copying the target modality during artificial masking and forces it to exploit complementary multimodal and structural evidence\.

### 3\.2Stage 2: Topology\-aware Cross\-modal Generation

The second stage aims to recover missing modality representations\. A naive generator usually predicts a missing modality from the remaining modalities of the same node\. However, this is insufficient for multimodal graphs because missing semantics may be better inferred from both its own observed modalities and neighboring observed modalities\. Therefore,FedMPObuilds a context bankℬi\(m\)\\mathcal\{B\}\_\{i\}^\{\(m\)\}for each target modalitymm, which contains the current node’s other visible modalities and neighboring visible modality representations\. To avoid self\-leakage during reconstruction, the target node’s own modality\-mmtoken is excluded by a diagonal attention mask𝐃i\(m\)\\mathbf\{D\}\_\{i\}^\{\(m\)\}, whose corresponding logit is set to−∞\-\\inftybefore softmax\.

To query this context bank, we construct a modality\-specific query using the target\-exclusive context, node\-level missingness pattern, and target modality identity:

𝐐i\(m\)=𝐖Q​\[𝐡i,∖m​‖𝐄mask​\(𝐫~i\)‖​𝐄mod​\(m\)\],𝐜i\(m\)=MHA​\(𝐐i\(m\),ℬi\(m\);𝐃i\(m\)\)\.\\mathbf\{Q\}\_\{i\}^\{\(m\)\}=\\mathbf\{W\}\_\{Q\}\[\\mathbf\{h\}\_\{i,\\setminus m\}\\parallel\\mathbf\{E\}\_\{\\mathrm\{mask\}\}\(\\tilde\{\\mathbf\{r\}\}\_\{i\}\)\\parallel\\mathbf\{E\}\_\{\\mathrm\{mod\}\}\(m\)\],\\qquad\\mathbf\{c\}\_\{i\}^\{\(m\)\}=\\mathrm\{MHA\}\(\\mathbf\{Q\}\_\{i\}^\{\(m\)\},\\mathcal\{B\}\_\{i\}^\{\(m\)\};\\mathbf\{D\}\_\{i\}^\{\(m\)\}\)\.\(4\)The attended evidence𝐜i\(m\)\\mathbf\{c\}\_\{i\}^\{\(m\)\}summarizes useful observed modalities and neighboring signals for recovering the target modality, making the generator topology\-aware rather than purely cross\-modal\.

The generated representation combines attention\-based cross\-modal evidence, pooled self context, and the structural anchor:

𝐳^i\(m\)=γt​\(𝐠i\(m\)⊙𝐜i\(m\)\+\(𝟏−𝐠i\(m\)\)⊙𝐖s​𝐡i,∖m\)\+\(1−γt\)​𝐖a​𝐳i,anc\(m\)\.\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}=\\gamma\_\{t\}\\left\(\\mathbf\{g\}\_\{i\}^\{\(m\)\}\\odot\\mathbf\{c\}\_\{i\}^\{\(m\)\}\+\(\\mathbf\{1\}\-\\mathbf\{g\}\_\{i\}^\{\(m\)\}\)\\odot\\mathbf\{W\}\_\{s\}\\mathbf\{h\}\_\{i,\\setminus m\}\\right\)\+\(1\-\\gamma\_\{t\}\)\\mathbf\{W\}\_\{a\}\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\.\(5\)Here,𝐠i\(m\)\\mathbf\{g\}\_\{i\}^\{\(m\)\}is a learnable gate balancing cross\-modal evidence and self context\. We use a linear warmupγt=min⁡\(1,t/Tw\)\\gamma\_\{t\}=\\min\(1,t/T\_\{w\}\), whereTwT\_\{w\}is the warmup round\. Early rounds therefore rely more on structural anchors, which act as a low\-variance topology prior and reduce unstable gradients in the non\-convex generator optimization under noisy recovered signals\. As training proceeds,FedMPOgradually shifts toward learned cross\-modal generation\.

The generation module is supervised only on artificially masked modalities:

ℒrec=∑i,mΔi\(m\)​‖𝐳^i\(m\)−sg​\(𝐳i,raw\(m\)\)‖22∑i,mΔi\(m\)\+ϵ\.\\mathcal\{L\}\_\{\\mathrm\{rec\}\}=\\frac\{\\sum\_\{i,m\}\\Delta\_\{i\}^\{\(m\)\}\\left\\\|\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\-\\mathrm\{sg\}\(\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\)\\right\\\|\_\{2\}^\{2\}\}\{\\sum\_\{i,m\}\\Delta\_\{i\}^\{\(m\)\}\+\\epsilon\}\.\(6\)Naturally missing modalities are still generated, but they are not used as direct reconstruction targets because their ground\-truth features are unavailable\. This avoids false supervision in the completion process\.

In addition to reconstruction, recovered modalities should be semantically consistent with observed modalities\. We therefore apply a cross\-modal alignment objective:

ℒalign=1\|Ω\|​∑\(i,m,m′\)∈Ω\[1−cos⁡\(𝐖al​𝐳~i\(m\),𝐖al​𝐳~i\(m′\)\)\],𝐳~i\(m\)=r~i\(m\)​𝐳i,raw\(m\)\+\(1−r~i\(m\)\)​𝐳^i\(m\)\.\\mathcal\{L\}\_\{\\mathrm\{align\}\}=\\frac\{1\}\{\|\\Omega\|\}\\sum\_\{\(i,m,m^\{\\prime\}\)\\in\\Omega\}\\left\[1\-\\cos\\left\(\\mathbf\{W\}\_\{\\mathrm\{al\}\}\\tilde\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\},\\mathbf\{W\}\_\{\\mathrm\{al\}\}\\tilde\{\\mathbf\{z\}\}\_\{i\}^\{\(m^\{\\prime\}\)\}\\right\)\\right\],\\quad\\tilde\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}=\\tilde\{r\}\_\{i\}^\{\(m\)\}\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\+\(1\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\)\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\.\(7\)Here,Ω\\Omegadenotes valid modality pairs\. This objective encourages observed and recovered modalities of the same node to lie in a coherent semantic space, reducing inconsistency before downstream fusion\.

### 3\.3Stage 3: Missing\-aware Mixture\-of\-Experts Fusion

Although Stage 2 provides candidate representations for missing modalities, not all recovered features are equally reliable\. For example, a missing modality can be recovered with high confidence when the node has rich observed modalities and many informative neighbors\. In contrast, recovery may be unreliable when both node\-level observations and neighborhood evidence are sparse\. Directly fusing observed and recovered features without distinguishing their reliability may propagate noisy semantics through graph message passing\.

To address this issue,FedMPOfirst estimates a modality\-wise recovery uncertainty and then uses it to control expert routing in a reliability\-aware manner:

ui\(m\)=\{0,r~i\(m\)=1,σ​\(fθu​\[𝐳^i\(m\)​‖𝐡i,∖m‖​𝐳i,anc\(m\)\]\),r~i\(m\)=0,u\_\{i\}^\{\(m\)\}=\\begin\{cases\}0,&\\tilde\{r\}\_\{i\}^\{\(m\)\}=1,\\\\ \\sigma\\left\(f\_\{\\theta\_\{u\}\}\[\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\\parallel\\mathbf\{h\}\_\{i,\\setminus m\}\\parallel\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\]\\right\),&\\tilde\{r\}\_\{i\}^\{\(m\)\}=0,\\end\{cases\}\(8\)\[wi,obs\(m\),wi,rec\(m\)\]=Softmax​\(fθr​\[r~i\(m\)​‖ui\(m\)‖​ρi∥ρ\]/τ\)\.\\qquad\[w\_\{i,\\mathrm\{obs\}\}^\{\(m\)\},w\_\{i,\\mathrm\{rec\}\}^\{\(m\)\}\]=\\mathrm\{Softmax\}\\left\(f\_\{\\theta\_\{r\}\}\[\\tilde\{r\}\_\{i\}^\{\(m\)\}\\parallel u\_\{i\}^\{\(m\)\}\\parallel\\rho\_\{i\}\\parallel\\rho\]/\\tau\\right\)\.\(9\)
Observed modalities are assigned zero recovery uncertainty, while generated modalities are assigned uncertainty scores based on generated features, cross\-modal context, and structural anchors\. The uncertainty head is calibrated on artificially masked observed modalities by matching uncertainty to normalized reconstruction errors\. For naturally missing modalities without error labels, the calibrated proxy is transferred through shared generator features, while missing\-ratio cues reduce heterogeneity\-induced shift\. The router further considers node\-level missing ratioρi\\rho\_\{i\}and client\-level missing ratioρ\\rho, so that routing decisions are aware of both local and client\-wise missingness patterns\.

The routed modality representation is produced by two experts: an observed expert and a recovered expert\. The observed expert focuses on reliable raw modality semantics, while the recovered expert processes generated or recovery\-related semantics:

𝐟i\(m\)=\{wi,obs\(m\)​Eobs​\(𝐳i,raw\(m\)\)\+wi,rec\(m\)​Erec​\(𝐳^i\(m\)\),r~i\(m\)=1,Erec​\(𝐳^i\(m\)\),r~i\(m\)=0\.\\mathbf\{f\}\_\{i\}^\{\(m\)\}=\\begin\{cases\}w\_\{i,\\mathrm\{obs\}\}^\{\(m\)\}E\_\{\\mathrm\{obs\}\}\(\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\)\+w\_\{i,\\mathrm\{rec\}\}^\{\(m\)\}E\_\{\\mathrm\{rec\}\}\(\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\),&\\tilde\{r\}\_\{i\}^\{\(m\)\}=1,\\\\ E\_\{\\mathrm\{rec\}\}\(\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\),&\\tilde\{r\}\_\{i\}^\{\(m\)\}=0\.\\end\{cases\}\(10\)This design avoids treating observed and recovered modalities as equally trustworthy\. For observed modalities, the model can still exploit the recovered branch as complementary regularization; for missing modalities, it relies on the recovered expert\. To avoid routing collapse, we include a load\-balancing regularizer in the routing loss, together with uncertainty calibration\.

After expert routing, modalities are fused according to their reliability\. More reliable modalities receive larger weights, while uncertain recovered modalities contribute less\. For highly incomplete nodes, a structural fallback expert is introduced to provide a graph\-only representation:

ai\(m\)\\displaystyle a\_\{i\}^\{\(m\)\}=exp⁡\(−ui\(m\)\)∑m′=1Mexp⁡\(−ui\(m′\)\)\+ϵ,𝐫i=LayerNorm​\(\(1−αfb,i\)​∑m=1Mai\(m\)​𝐟i\(m\)\+αfb,i​Estruct​\(𝐡str,i\)\)\.\\displaystyle=\\frac\{\\exp\(\-u\_\{i\}^\{\(m\)\}\)\}\{\\sum\_\{m^\{\\prime\}=1\}^\{M\}\\exp\(\-u\_\{i\}^\{\(m^\{\\prime\}\)\}\)\+\\epsilon\},\\mathbf\{r\}\_\{i\}=\\mathrm\{LayerNorm\}\\left\(\(1\-\\alpha\_\{\\mathrm\{fb\},i\}\)\\sum\_\{m=1\}^\{M\}a\_\{i\}^\{\(m\)\}\\mathbf\{f\}\_\{i\}^\{\(m\)\}\+\\alpha\_\{\\mathrm\{fb\},i\}E\_\{\\mathrm\{struct\}\}\(\\mathbf\{h\}\_\{\\mathrm\{str\},i\}\)\\right\)\.\(11\)
The fallback coefficientαfb,i\\alpha\_\{\\mathrm\{fb\},i\}increases when the node has a high missing ratio or high average uncertainty\. Thus,FedMPOcan gradually shift from multimodal fusion to structure\-only evidence when recovered modalities are unreliable, thereby reducing noisy semantic propagation\.

### 3\.4Stage 4: Task Prediction and Federated Aggregation

After obtaining the fused representation𝐫i\\mathbf\{r\}\_\{i\}, each client performs a lightweight graph refinement step and applies a task\-specific prediction head\. The task loss is selected according to the downstream task, such as cross\-entropy for node classification, pairwise prediction loss for link prediction, or contrastive ranking loss for modality retrieval\. The local objective combines task learning with reconstruction, alignment, and routing regularization:

𝐫i′=LayerNorm​\(𝐫i\+σ​\(SAGEConv​\(𝐫i,𝒢\)\)\)\\mathbf\{r\}^\{\\prime\}\_\{i\}=\\mathrm\{LayerNorm\}\\left\(\\mathbf\{r\}\_\{i\}\+\\sigma\(\\mathrm\{SAGEConv\}\(\\mathbf\{r\}\_\{i\},\\mathcal\{G\}\)\)\\right\)\(12\)𝐲^i=ftask​\(𝐫i′\),ℒk=ℒtask\(k\)\+λrec​ℒrec\(k\)\+λalign​ℒalign\(k\)\+λroute​ℒroute\(k\)\.\\hat\{\\mathbf\{y\}\}\_\{i\}=f\_\{\\mathrm\{task\}\}\(\\mathbf\{r\}^\{\\prime\}\_\{i\}\),\\ \\mathcal\{L\}\_\{k\}=\\mathcal\{L\}\_\{\\mathrm\{task\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{rec\}\}\\mathcal\{L\}\_\{\\mathrm\{rec\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{align\}\}\\mathcal\{L\}\_\{\\mathrm\{align\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{route\}\}\\mathcal\{L\}\_\{\\mathrm\{route\}\}^\{\(k\)\}\.\(13\)Here,ℒroute\\mathcal\{L\}\_\{\\mathrm\{route\}\}contains uncertainty calibration and expert load balancing\. This local objective ensures that the learned representation is not only task\-discriminative, but also completion\-aware and reliability\-aware\.

Finally, the server aggregates client updates\. Standard FedAvg weights clients mainly by local data size, which can be problematic under heterogeneous modality missingness\. A client with many nodes but unreliable recovered modalities may produce biased updates and dominate global aggregation\. To mitigate this global aggregation bias, each client uploads lightweight reliability statistics together with model parameters, including average uncertaintyu¯k\\bar\{u\}\_\{k\}, reconstruction errore¯k\\bar\{e\}\_\{k\}, and client\-level missing ratioρk\\rho\_\{k\}\. The server converts these statistics into reliability\-aware aggregation weights:

sk=exp⁡\(−ηu​u¯k−ηe​e¯k−ηρ​ρk\),ωk=\|𝒱k\|​sk∑j=1K\|𝒱j\|​sj\+ϵ,Θt\+1=∑k=1Kωk​Θkt\+1\.s\_\{k\}=\\exp\(\-\\eta\_\{u\}\\bar\{u\}\_\{k\}\-\\eta\_\{e\}\\bar\{e\}\_\{k\}\-\\eta\_\{\\rho\}\\rho\_\{k\}\),\\qquad\\omega\_\{k\}=\\frac\{\|\\mathcal\{V\}\_\{k\}\|s\_\{k\}\}\{\\sum\_\{j=1\}^\{K\}\|\\mathcal\{V\}\_\{j\}\|s\_\{j\}\+\\epsilon\},\\qquad\\Theta^\{t\+1\}=\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}\\Theta\_\{k\}^\{t\+1\}\.\(14\)When clients have similar modality completeness and recovery quality, this rule reduces to standard data\-size\-weighted FedAvg\. Otherwise, updates from clients dominated by unreliable recovered signals are down\-weighted, preventing completion errors from being repeatedly amplified across communication rounds\. Sincesks\_\{k\}remains positive for every selected client, reliability\-aware aggregation softly rescales rather than excludes client updates, preserving federated collaboration while reducing unreliable dominance\.

Overall,FedMPOaddresses topology\-unaware local completion through graph\-contextualized modality generation, and mitigates reliability\-imbalanced global aggregation through uncertainty\-guided routing and reliability\-aware federated aggregation, leading to more stable optimization\.

## 4Experiments

We conduct experiments on federated multimodal graph benchmarks under non\-IID and missing\-modality settings\. Our evaluation aims to answer four questions:Q1\(Effectiveness\): DoesFedMPOoutperform representative baselines across downstream tasks?Q2\(Interpretability\): Do its key components contribute to robust learning?Q3\(Robustness\): Is it robust to modality missingness, client heterogeneity, and hyperparameter variations? andQ4\(Efficiency\): Does it remain efficient in federated training?

Experimental setup\.We evaluateFedMPOon six federated multimodal graph datasets covering node classification, link prediction, and modality retrieval\. We report Accuracy/F1\-score, AUC/AP, and Recall@5/MRR for the three tasks, respectively\. Datasets are partitioned by Dirichlet\-based non\-IID splits, and both client\-level and node\-level modality missingness are simulated to reflect realistic cross\-client modality incompleteness\. Unless otherwise specified, the missing rate is30%30\\%and the Dirichlet concentration isα=0\.5\\alpha=0\.5\. We compare with three groups of baselines: basic FGL methods, including FedAvg\-Zero, FedGCN, and FedGraphSAGE; advanced FGL methods, including FedProto, FedPub, FedLAP, S2FGL, FedSPA, and FedIIH; and multimodal FL methods, including FedMVP and FedMAC\. All methods use the same partitions, missingness protocols, and evaluation splits\. N/A indicates that the original method is not directly applicable to the corresponding task without modifying its core learning objective; for example, prototype\-based methods are not naturally defined for link prediction or retrieval\. Details are provided in Appendix[C](https://arxiv.org/html/2605.12584#A3)\.

### 4\.1Overall Performance

Table[1](https://arxiv.org/html/2605.12584#S4.T1)summarizes the overall results\.FedMPOachieves the best performance across all datasets and metrics, showing its effectiveness under modality incompleteness and federated heterogeneity\. It consistently outperforms strong baselines on node classification, link prediction, and modality retrieval, demonstrating advantages for both graph\-centric and modality\-centric tasks\. These gains verify the joint benefits of topology\-aware completion, missing\-aware fusion, and reliability\-aware aggregation\.

Table 1:Performance comparison of our proposed method against baselines across Node Classification, Link Prediction, and Modality Retrieval tasks\.
### 4\.2Ablation study

We conduct module\-wise ablations across node classification, link prediction, and modality retrieval\. As shown in Fig\.[3](https://arxiv.org/html/2605.12584#S4.F3), removing any key component consistently degrades performance, confirming thatFedMPObenefits from the cooperation of topology\-aware completion, missing\-aware fusion, cross\-modal alignment, and reliability\-aware aggregation\. Specifically,w/o AGMGremoves the adaptive gated modality generation module, leading to the largest drop and showing that graph\-structured context is crucial for missing\-modality recovery\.w/o MoEreplaces missing\-aware expert fusion with simpler fusion, indicating that observed and recovered modalities should be treated according to their reliability\.w/o Alignremoves cross\-modal alignment, which weakens semantic consistency among modalities\. Finally,w/o RelAggreplaces reliability\-aware aggregation with standard FedAvg\. Its performance drop shows that data\-size\-weighted aggregation may over\-emphasize unreliable client updates under heterogeneous modality missingness, whileFedMPOmitigates global aggregation bias by down\-weighting clients with higher recovery uncertainty, reconstruction error, or missing ratio\.

![Refer to caption](https://arxiv.org/html/2605.12584v1/x3.png)Figure 3:Ablation study ofFedMPO\.We compare the full model with variants removing AGMG, MoE fusion, cross\-modal alignment, and reliability\-aware aggregation across three downstream tasks\. The performance drops verify the effectiveness of each component\.
### 4\.3Robustness Study

Missing rate sensitivity\.Table[2](https://arxiv.org/html/2605.12584#S4.T2)evaluates robustness under different modality missing rates on the Ele\-fashion node classification task\. Asη\\etaincreases, all methods degrade, butFedMPOremains more stable and exhibits a slower performance decay\. Whenη\\etaincreases from30%30\\%to70%70\\%,FedMPOdrops from85\.67%85\.67\\%to81\.35%81\.35\\%in Accuracy and still outperforms the best baseline by4\.104\.10percentage points\. Thus, the performance gap increases from0\.940\.94to4\.104\.10points, showing thatFedMPOis particularly advantageous under severe modality incompleteness\. This confirms that topology\-aware completion and missing\-aware fusion can better preserve useful multimodal semantics when observations become sparse\.

Client heterogeneity\.We further evaluate node classification robustness by varying the Dirichlet concentration parameterα\\alphawhile fixing the missing rate at30%30\\%\. A smallerα\\alphaindicates stronger non\-IID client partitions and more biased local data distributions\. Under the highly heterogeneous settingα=0\.1\\alpha=0\.1,FedMPOachieves82\.45%82\.45\\%Accuracy and outperforms the best baseline by5\.655\.65percentage points\. This result indicates that reliability\-aware aggregation effectively reduces the negative impact of biased and unreliable local updates, leading to more stable global optimization across heterogeneous clients\.

Table 2:Robustness Study on the Ele\-fashion dataset\.Left:Sensitivity to varying modality missing rates \(η\\eta\)\.Right:Adaptability to data heterogeneity across clients under different Dirichlet distribution parameters \(α\\alpha\)\.Hyperparameter sensitivity\.Fig\.[4](https://arxiv.org/html/2605.12584#S4.F4)\(a\) reports the sensitivity ofFedMPOtoλrec\\lambda\_\{\\mathrm\{rec\}\}andλroute\\lambda\_\{\\mathrm\{route\}\}\. The performance remains stable across a wide range of coefficient combinations, suggesting thatFedMPOdoes not rely on a narrow hyperparameter optimum\. Fig\.[4](https://arxiv.org/html/2605.12584#S4.F4)\(b\) further summarizes robustness trends under different missing rates and Dirichletα\\alphavalues, confirming thatFedMPOmaintains strong performance under both modality incompleteness and client heterogeneity\.

![Refer to caption](https://arxiv.org/html/2605.12584v1/figs/hyper-parameters.png)\(a\)Sensitivity analysis ofλrec\\lambda\_\{\\mathrm\{rec\}\}andλroute\\lambda\_\{\\mathrm\{route\}\}\.
![Refer to caption](https://arxiv.org/html/2605.12584v1/figs/fedmpo_sensitivity.png)\(b\)Robustness analysis settings\.

Figure 4:Sensitivity and robustness analysis ofFedMPO\.\(a\)Node classification ACC on Ele\-Fashion and Grocery under different combinations of reconstruction coefficientλrec\\lambda\_\{\\mathrm\{rec\}\}and routing coefficientλroute\\lambda\_\{\\mathrm\{route\}\}\.\(b\)Robustness ofFedMPOunder different Dirichletα\\alphavalues and modality missing ratesη\\eta\.
### 4\.4Efficiency Study

FedMPOintroduces topology\-aware modality generation and missing\-aware expert routing, but the overhead is moderate\. First, these modules operate on compact hidden representations rather than raw text or image inputs\. Second, reliability\-aware aggregation only requires lightweight client\-level scalar statistics, including recovery uncertainty, reconstruction error, and missing ratio, whose communication cost is negligible compared with model parameters\. Third, by filtering unreliable recovered features locally and reducing unreliable updates,FedMPOimproves training stability under missing\-modality and non\-IID settings\. Runtime and resource results are provided in Appendix[C](https://arxiv.org/html/2605.12584#A3)\. As shown in Fig\.[6](https://arxiv.org/html/2605.12584#A3.F6),FedMPOincurs only moderate runtime and memory overhead while remaining more efficient than several advanced FGL baselines\. Therefore,FedMPOachieves a favorable trade\-off between accuracy, robustness, and federated training cost\.

## 5Conclusion

We presentedFedMPO, a unified framework for missing\-modality robust learning in federated multimodal graphs\. By disentangling natural missingness from training\-time masking, performing graph\-enhanced cross\-modal completion, and adaptively integrating observed, recovered, and structural signals through missing\-aware expert fusion,FedMPOjointly addresses modality incompleteness, graph dependency, and cross\-client heterogeneity within a single federated optimization workflow\. Across node classification, link prediction, and modality retrieval tasks, experiments demonstrate consistent improvements over strong baselines, especially under severe missing rates and non\-IID client partitions across diverse federated graph settings\. The robustness gains further indicate that explicitly modeling recovery reliability is important when local updates are affected by heterogeneous modality availability\. These results show that reliable modality recovery, missing\-aware fusion, and reliability\-aware aggregation are essential for robust multimodal graph learning in federated environments\. Limitations and future directions are discussed in Appendix[E](https://arxiv.org/html/2605.12584#A5)\.

## References

- Leveraging foundation models for multi\-modal federated learning with incomplete modality\.Machine Learning and Knowledge Discovery in Databases\. Research Track,pp\. 401–417\.Cited by:[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p3.1),[§1](https://arxiv.org/html/2605.12584#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- Y\. Ektefaie, G\. Dasoulas, A\. Noori, M\. Farhat, and M\. Zitnik \(2023\)Multimodal learning with graphs\.Nature Machine Intelligence5\(4\),pp\. 340–350\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p4.1)\.
- R\. Girdhar, A\. El\-Nouby, Z\. Liu, M\. Singh, K\. V\. Alwala, A\. Joulin, and I\. Misra \(2023\)ImageBind: one embedding space to bind them all\.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 15180–15190\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p3.1)\.
- W\. L\. Hamilton, R\. Ying, and J\. Leskovec \(2017\)Inductive representation learning on large graphs\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- Y\. He, Y\. Sui, X\. He, Y\. Liu, Y\. Sun, and B\. Hooi \(2025\)UniGraph2: learning a unified embedding space to bind multimodal graphs\.Proceedings of the ACM on the Web Conference 2025,pp\. 1759–1770\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p2.1),[§1](https://arxiv.org/html/2605.12584#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- X\. Jia, M\. Jiang, Y\. Dong, F\. Zhu, H\. Lin, Y\. Xin, and H\. Chen \(2023\)Multimodal heterogeneous graph attention network\.Neural Computing and Applications35\(4\),pp\. 3357–3372\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p2.1),[§1](https://arxiv.org/html/2605.12584#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- S\. P\. Karimireddy, S\. Kale, M\. Mohri, S\. Reddi, S\. Stich, and A\. T\. Suresh \(2020\)SCAFFOLD: stochastic controlled averaging for federated learning\.InProceedings of the 37th International Conference on Machine Learning \(ICML\),PMLR, Vol\.119,pp\. 5132–5143\.Cited by:[§A\.2](https://arxiv.org/html/2605.12584#A1.SS2.p1.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- T\. N\. Kipf and M\. Welling \(2017\)Semi\-supervised classification with graph convolutional networks\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- T\. Li, A\. K\. Sahu, A\. Talwalkar, and V\. Smith \(2020\)Federated optimization in heterogeneous networks\.Proceedings of Machine Learning and Systems2,pp\. 429–450\.Cited by:[§A\.2](https://arxiv.org/html/2605.12584#A1.SS2.p1.1),[§1](https://arxiv.org/html/2605.12584#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- X\. Li, Y\. Ai, Y\. Zhu, H\. Lu, Y\. Zhang, G\. Fu, B\. Fan, Q\. Dai, R\. Li, and G\. Wang \(2026\)MM\-openfgl: a comprehensive benchmark for multimodal federated graph learning\.arXiv preprint arXiv:2601\.22416\.Cited by:[§A\.2](https://arxiv.org/html/2605.12584#A1.SS2.p5.1),[§1](https://arxiv.org/html/2605.12584#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- X\. Li, Y\. Zhu, B\. Pang, G\. Yan, Y\. Yan, Z\. Li, Z\. Wu, W\. Zhang, R\. Li, and G\. Wang \(2025\)OpenFGL: a comprehensive benchmark for federated graph learning\.Proceedings of the VLDB Endowment18,pp\. 1305–1320\.Cited by:[§A\.2](https://arxiv.org/html/2605.12584#A1.SS2.p2.1),[§A\.2](https://arxiv.org/html/2605.12584#A1.SS2.p3.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- M\. Ma, J\. Ren, L\. Zhao, D\. Testuggine, and X\. Peng \(2022\)Are multimodal transformers robust to missing modality?\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 18177–18186\.Cited by:[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p3.1)\.
- M\. Ma, J\. Ren, L\. Zhao, S\. Tulyakov, C\. Wu, and X\. Peng \(2021\)SMIL: multimodal learning with severely missing modality\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 2302–2310\.Cited by:[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p3.1)\.
- B\. McMahan, E\. Moore, D\. Ramage, S\. Hampson, and B\. A\. y\. Arcas \(2017\)Communication\-efficient learning of deep networks from decentralized data\.InProceedings of the 20th International Conference on Artificial Intelligence and Statistics \(AISTATS\),PMLR, Vol\.54,pp\. 1273–1282\.Cited by:[§A\.2](https://arxiv.org/html/2605.12584#A1.SS2.p1.1),[§1](https://arxiv.org/html/2605.12584#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- M\. D\. Nguyen, T\. T\. Nguyen, H\. H\. Pham, T\. N\. Hoang, P\. Le Nguyen, and T\. T\. Huynh \(2024\)FedMAC: tackling partial\-modality missing in federated learning with cross\-modal aggregation and contrastive regularization\.In2024 22nd International Symposium on Network Computing and Applications \(NCA\),pp\. 278–285\.Cited by:[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p3.1),[§1](https://arxiv.org/html/2605.12584#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- C\. Peng, J\. He, and F\. Xia \(2024a\)Learning on multimodal graphs: a survey\.arXiv preprint arXiv:2402\.05322\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p4.1)\.
- Y\. Peng, J\. Bian, and J\. Xu \(2024b\)FedMM: federated multimodal learning with modality heterogeneity in computational pathology\.ICASSP 2024 \- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing,pp\. 1696–1700\.Cited by:[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p3.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark, G\. Krueger, and I\. Sutskever \(2021\)Learning transferable visual models from natural language supervision\.InProceedings of the 38th International Conference on Machine Learning \(ICML\),PMLR, Vol\.139,pp\. 8748–8763\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p3.1)\.
- M\. K\. Reza, A\. Prater\-Bennette, and M\. S\. Asif \(2024\)Robust multimodal learning with missing modalities via parameter\-efficient adaptation\.IEEE Transactions on Pattern Analysis and Machine Intelligence\.Cited by:[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p3.1)\.
- Z\. Tao, Y\. Wei, X\. Wang, X\. He, X\. Huang, and T\. Chua \(2020\)MGAT: multimodal graph attention network for recommendation\.Information Processing & Management57\(5\),pp\. 102277\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p2.1),[§1](https://arxiv.org/html/2605.12584#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- P\. Veličković, G\. Cucurull, A\. Casanova, A\. Romero, P\. Liò, and Y\. Bengio \(2018\)Graph attention networks\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- C\. Wan, X\. Li, Y\. Zuo, H\. Deng, S\. Li, B\. Fan, H\. Qin, R\. Li, and G\. Wang \(2026\)OpenMAG: a comprehensive benchmark for multimodal\-attributed graph\.arXiv preprint arXiv:2602\.05576\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p3.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p4.1),[§1](https://arxiv.org/html/2605.12584#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- H\. Wang, Y\. Chen, C\. Ma, J\. Avery, L\. Hull, and G\. Carneiro \(2023\)Multi\-modal learning with missing modality via shared\-specific feature modelling\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 15878–15887\.Cited by:[§1](https://arxiv.org/html/2605.12584#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p3.1)\.
- W\. Wanget al\.\(2023\)Multimodal learning with incomplete modalities by knowledge distillation\.Pattern Recognition\.Cited by:[§A\.4](https://arxiv.org/html/2605.12584#A1.SS4.p3.1)\.
- Y\. Wei, X\. Wang, L\. Nie, X\. He, R\. Hong, and T\. Chua \(2019\)MMGCN: multi\-modal graph convolution network for personalized recommendation of micro\-video\.Proceedings of the 27th ACM International Conference on Multimedia,pp\. 1437–1445\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p2.1),[§1](https://arxiv.org/html/2605.12584#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- F\. Wu, X\. Wang, Y\. Wang, T\. Liu, L\. Su, and J\. Gao \(2024\)FIARSE: model\-heterogeneous federated learning via importance\-aware submodel extraction\.Advances in Neural Information Processing Systems37\.Cited by:[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p3.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- J\. Wuet al\.\(2024\)Incomplete multimodal learning: a survey\.arXiv preprint arXiv:2407\.19108\.Cited by:[§A\.4](https://arxiv.org/html/2605.12584#A1.SS4.p1.1),[§A\.4](https://arxiv.org/html/2605.12584#A1.SS4.p2.1),[§1](https://arxiv.org/html/2605.12584#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p3.1)\.
- L\. Xie, M\. Lin, T\. Luan, C\. Li, Y\. Fang, Q\. Shen, and Z\. Wu \(2024\)MH\-pflid: model heterogeneous personalized federated learning via injection and distillation for medical data analysis\.arXiv preprint arXiv:2405\.06822\.Cited by:[§A\.3](https://arxiv.org/html/2605.12584#A1.SS3.p3.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p2.1)\.
- K\. Xu, W\. Hu, J\. Leskovec, and S\. Jegelka \(2019\)How powerful are graph neural networks?\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.
- H\. Yan, C\. Li, Z\. Yu, J\. Yin, R\. Liu, P\. Zhang, W\. Han, M\. Li, Z\. Zeng, H\. Sun, W\. Deng, F\. Sun, Q\. Zhang, and S\. Wang \(2024\)When graph meets multimodal: benchmarking on multimodal attributed graphs learning\.arXiv preprint arXiv:2410\.09132\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p3.1)\.
- X\. Zhaoet al\.\(2023\)MMANet: missing modality aware network for incomplete multimodal learning\.Proceedings of the AAAI Conference on Artificial Intelligence\.Cited by:[§A\.4](https://arxiv.org/html/2605.12584#A1.SS4.p4.1)\.
- J\. Zhu, Y\. Zhou, S\. Qian, Z\. He, T\. Zhao, N\. Shah, and D\. Koutra \(2025\)Mosaic of modalities: a comprehensive benchmark for multimodal graph learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 14215–14224\.Cited by:[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2605.12584#A1.SS1.p3.1),[§1](https://arxiv.org/html/2605.12584#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.12584#S2.SS2.p1.1)\.

## Appendix ARelated Works in Details

### A\.1Centralized Multimodal Graph Learning

Multimodal graph learning studies how to jointly model graph topology and heterogeneous node attributes, such as text and images, in order to improve structural reasoning and semantic understandingEktefaieet al\.\[[2023](https://arxiv.org/html/2605.12584#bib.bib19)\], Penget al\.\[[2024a](https://arxiv.org/html/2605.12584#bib.bib18)\]\. Compared with unimodal graph learning, its central premise is that different modalities provide complementary semantic evidence, while graph structure offers relational context that helps propagate, refine, and align such evidence across connected nodesZhuet al\.\[[2025](https://arxiv.org/html/2605.12584#bib.bib21)\], Wanet al\.\[[2026](https://arxiv.org/html/2605.12584#bib.bib23)\]\. This joint modeling perspective has driven a growing line of work in recommendation, social analysis, and multimodal relational reasoningWeiet al\.\[[2019](https://arxiv.org/html/2605.12584#bib.bib14)\], Taoet al\.\[[2020](https://arxiv.org/html/2605.12584#bib.bib15)\], Jiaet al\.\[[2023](https://arxiv.org/html/2605.12584#bib.bib16)\], Heet al\.\[[2025](https://arxiv.org/html/2605.12584#bib.bib17)\]\.

A first line of methods extends standard graph neural networks to multimodal node representations by designing modality\-aware message passing and fusion mechanisms\. Representative examples include MMGCN and MGAT, which perform modality\-specific representation learning and then aggregate structural and cross\-modal signals through graph propagation or attention\-based fusionWeiet al\.\[[2019](https://arxiv.org/html/2605.12584#bib.bib14)\], Taoet al\.\[[2020](https://arxiv.org/html/2605.12584#bib.bib15)\]\. These methods demonstrate that multimodal semantics can substantially enrich graph representations beyond structure\-only or single\-modality baselines\. Subsequent work further explores more refined multimodal interaction mechanisms, heterogeneous attention, and unified multimodal embedding spaces, seeking to improve the alignment between topology and cross\-modal semanticsJiaet al\.\[[2023](https://arxiv.org/html/2605.12584#bib.bib16)\], Heet al\.\[[2025](https://arxiv.org/html/2605.12584#bib.bib17)\]\.

A second line of studies emphasizes the role of modality encoders and feature alignment\. Recent benchmark efforts such as MM\-GRAPH, MAGB, and OpenMAG show that multimodal graph performance is influenced not only by graph architectures, but also by how raw modalities are encoded and whether they are mapped into an aligned latent spaceZhuet al\.\[[2025](https://arxiv.org/html/2605.12584#bib.bib21)\], Yanet al\.\[[2024](https://arxiv.org/html/2605.12584#bib.bib22)\], Wanet al\.\[[2026](https://arxiv.org/html/2605.12584#bib.bib23)\]\. In particular, unified multimodal encoders such as CLIP and ImageBind often provide stronger cross\-modal consistency than independent modality encoders, while graph structure can further enhance multimodal reasoning by propagating semantic information over neighborhoodsRadfordet al\.\[[2021](https://arxiv.org/html/2605.12584#bib.bib29)\], Girdharet al\.\[[2023](https://arxiv.org/html/2605.12584#bib.bib28)\], Zhuet al\.\[[2025](https://arxiv.org/html/2605.12584#bib.bib21)\], Wanet al\.\[[2026](https://arxiv.org/html/2605.12584#bib.bib23)\]\. These observations have made encoder choice, feature quality, and alignment quality central concerns in multimodal graph learning\.

A third line of work moves beyond discriminative graph tasks and treats multimodal graphs as a more general substrate for retrieval, alignment, and generationWanet al\.\[[2026](https://arxiv.org/html/2605.12584#bib.bib23)\]\. This broader view suggests that multimodal graphs are not merely graphs with extra node features, but structured systems in which topology and heterogeneous semantics mutually constrain one another\. From this perspective, graph structure can act as a semantic scaffold, while modalities provide fine\-grained content that topology alone cannot recoverEktefaieet al\.\[[2023](https://arxiv.org/html/2605.12584#bib.bib19)\], Penget al\.\[[2024a](https://arxiv.org/html/2605.12584#bib.bib18)\]\.

Despite this progress, existing centralized multimodal graph methods usually rely on three assumptions that do not hold in our setting\. First, they assume that all multimodal node attributes and graph structures are available to a single training system\. Second, they usually assume relatively complete modality observations, or at least do not explicitly model multi\-granularity missingness as a first\-class challenge\. Third, even when they incorporate multimodal interaction and graph propagation effectively, they do not consider how such interactions behave under decentralized training and repeated federated aggregation\. As a result, while centralized multimodal graph learning provides the architectural and representational foundation for our work, it does not directly address privacy\-preserving distributed training with severe missing modalities\.

### A\.2Federated Learning and Federated Graph Learning

Federated learning enables multiple clients to collaboratively optimize a shared model without exposing raw local data, typically by alternating between local updates and server\-side aggregationMcMahanet al\.\[[2017](https://arxiv.org/html/2605.12584#bib.bib1)\]\. This framework was originally introduced for decentralized learning on Euclidean data, and later extended to heterogeneous settings through optimization strategies such as FedProx and SCAFFOLD, which aim to reduce client drift and improve stability under non\-IID data distributionsLiet al\.\[[2020](https://arxiv.org/html/2605.12584#bib.bib2)\], Karimireddyet al\.\[[2020](https://arxiv.org/html/2605.12584#bib.bib3)\]\. These methods establish the basic optimization backbone for a wide range of decentralized learning systems\.

Building on this foundation, federated graph learning extends collaborative training from independent samples to graph\-structured data\. Compared with standard federated learning, graph data introduce additional challenges because local graph topology, homophily patterns, neighborhood semantics, and label distributions can vary significantly across clients\. Recent benchmark and algorithmic studies have shown that structural heterogeneity, topology shift, and graph\-specific statistical skew substantially complicate both local training and global aggregationLiet al\.\[[2025](https://arxiv.org/html/2605.12584#bib.bib9)\]\. In this sense, federated graph learning is not a trivial application of standard federated optimization, but a setting in which graph structure becomes an additional source of heterogeneity\.

Existing federated graph learning methods mainly focus on issues such as graph structural inconsistency, personalized aggregation, graph\-specific knowledge sharing, and spectral or topology\-aware regularizationLiet al\.\[[2025](https://arxiv.org/html/2605.12584#bib.bib9)\]\. Their central concern is how to preserve the benefits of graph representation learning under decentralized settings while mitigating the mismatch between local graph distributions\. This literature has already demonstrated that direct adoption of standard FL algorithms can be suboptimal for graphs, and that graph\-aware aggregation or personalization is often necessary\.

However, most current federated graph learning methods still assume single\-modality node attributes or relatively homogeneous feature spaces\. Even when node features are high\-dimensional, they are usually treated as a single vector representation rather than a set of heterogeneous modalities with different availability patterns\. Therefore, the main challenge in standard federated graph learning is usually the mismatch of structure or labels across clients, rather than the joint interaction between graph topology and incomplete multimodal semantics\.

This limitation becomes particularly important when moving to multimodal graph data\. Once node attributes consist of multiple modalities, federated optimization must deal not only with client drift and graph heterogeneity, but also with cross\-modal inconsistency, modality\-dependent signal quality, and possible missingness across clients and nodes\. Recent benchmark work such as MM\-OpenFGL explicitly shows that multimodal federated graph learning constitutes a distinct problem setting rather than a straightforward extension of existing federated graph learning pipelinesLiet al\.\[[2026](https://arxiv.org/html/2605.12584#bib.bib24)\]\. In particular, naive adaptation of standard federated learning or federated graph learning methods often fails to reconcile cross\-modal semantic conflicts and cross\-client structural mismatch, which motivates more targeted modeling of multimodal incompleteness and fusion reliability\.

### A\.3Multimodal Federated Learning with Missing Modalities

A related but distinct research direction is multimodal federated learning, where clients collaboratively train on multimodal data without sharing raw observationsCheet al\.\[[2024](https://arxiv.org/html/2605.12584#bib.bib4)\], Nguyenet al\.\[[2024](https://arxiv.org/html/2605.12584#bib.bib5)\], Penget al\.\[[2024b](https://arxiv.org/html/2605.12584#bib.bib6)\]\. Compared with federated graph learning, the main focus here is not graph structure, but how to align, fuse, or transfer heterogeneous modalities across decentralized clients under privacy constraints\. This line of work is especially relevant to our setting because it directly addresses incomplete modalities, modality heterogeneity, and the reliability of cross\-modal information under federation\.

A key theme in this literature is that multimodal heterogeneity is often more severe than the classical non\-IID setting considered in standard federated learning\. Different clients may possess different modality subsets, different encoder backbones, or different modality quality distributions\. In such cases, the challenge is not only that data distributions differ across clients, but that the semantics represented by local observations may be structurally incomplete or asymmetric\. This leads to a family of methods that seek to improve multimodal collaboration through prototype transfer, cross\-modal aggregation, generator modules, or representation reconfiguration\.

FedMVP is a representative example that leverages foundation\-model\-driven multimodal priors to support federated learning under incomplete modalitiesCheet al\.\[[2024](https://arxiv.org/html/2605.12584#bib.bib4)\]\. Its central intuition is that powerful pretrained multimodal representations can compensate for local modality absence and provide a stronger basis for cross\-client collaboration\. FedMAC, in contrast, emphasizes cross\-modal aggregation and contrastive regularization for partial\-modality missing scenarios, aiming to transfer information across modalities while maintaining representation consistencyNguyenet al\.\[[2024](https://arxiv.org/html/2605.12584#bib.bib5)\]\. FedMM studies modality heterogeneity in federated multimodal learning from an application perspective, showing that multimodal inconsistency across clients is not merely an implementation detail but a core challenge for collaborative trainingPenget al\.\[[2024b](https://arxiv.org/html/2605.12584#bib.bib6)\]\. FIARSE and MH\-pFLID further highlight that model or modality heterogeneity requires more flexible collaboration schemes than homogeneous FL pipelinesWuet al\.\[[2024](https://arxiv.org/html/2605.12584#bib.bib7)\], Xieet al\.\[[2024](https://arxiv.org/html/2605.12584#bib.bib8)\]\.

These works provide two important insights for our problem\. First, missing or heterogeneous modalities across clients require explicit mechanism design; they cannot be handled reliably by treating missing entries as ordinary noise or padding\. Second, the value of recovered or transferred modality information depends on its reliability, not merely on its existence\. This is particularly relevant for our setting, where recovered modality representations may enter graph propagation and then be repeatedly mixed through federated aggregation\.

At the same time, existing multimodal federated learning methods remain insufficient for our task for two reasons\. The first is structural: most of them are designed for non\-graph multimodal data and therefore do not model neighborhood interaction, topology\-conditioned completion, or graph\-aware signal propagation\. The second is reliability\-related: while they address missing modalities, they typically do not study how completion errors interact with graph message passing and global aggregation over multiple communication rounds\. For this reason, multimodal federated learning with missing modalities provides an important conceptual basis for our work, but does not fully solve missing\-modality robust learning in federated multimodal graphs\.

### A\.4Incomplete Multimodal Learning and Modality Completion

Incomplete multimodal learning studies how to learn robust models when one or more modalities are absent during training or inferenceWu and others\[[2024](https://arxiv.org/html/2605.12584#bib.bib27)\]\. This problem has attracted increasing attention because missingness is common in practical multimodal systems, arising from sensor failure, acquisition cost, privacy restrictions, corruption, or long\-term unavailability\. Unlike standard multimodal learning, which assumes a complete set of inputs, incomplete multimodal learning must explicitly reason about how to compensate for absent information without overly trusting noisy or spurious substitutes\.

A major line of methods tackles this challenge through cross\-modal reconstruction or modality completion\. The core idea is to recover a missing modality from the observed ones by exploiting cross\-modal redundancy or shared latent semantics\. These methods treat different modalities as partially predictive of each other and learn reconstruction mappings so that incomplete observations can still be projected into a semantically meaningful spaceWu and others\[[2024](https://arxiv.org/html/2605.12584#bib.bib27)\]\. This perspective is closely related to our generator, but in our setting the recovered representations must also be compatible with graph structure and federated optimization\.

Another influential line of work focuses on shared–specific representation modeling\. Instead of assuming that all modalities should collapse into a single common representation, these methods explicitly separate modality\-shared semantics from modality\-specific informationWang and others\[[2023](https://arxiv.org/html/2605.12584#bib.bib25)\]\. This decomposition is useful because it acknowledges both redundancy and irreducible modality differences\. For incomplete multimodal learning, such decomposition helps prevent trivial fusion and supports more robust transfer from observed modalities to missing ones\. It also suggests that completion should not simply imitate a missing modality at the feature level, but should preserve the correct balance between shared semantics and modality\-specific variation\.

A third line of work emphasizes missing\-aware fusion and reliability\-aware integration\. MMANet and related models show that the main challenge is not only to reconstruct missing modalities, but also to determine how much the model should trust recovered information relative to observed informationZhao and others\[[2023](https://arxiv.org/html/2605.12584#bib.bib26)\]\. This perspective is particularly important in high\-missing regimes, where recovered modalities may be systematically biased or uncertain\. In such cases, blindly fusing completed features can degrade downstream performance rather than improve it\.

Although incomplete multimodal learning offers important tools and insights, most of this literature remains centralized and non\-graph\. As a result, it typically ignores two mechanisms that are essential in our setting\. The first is topology\-conditioned semantics: in graphs, a node is not interpreted only through its own modalities, but also through neighborhood context\. The second is federated error accumulation: under decentralized training, unreliable recovered information can be locally amplified through graph propagation and then globally reinforced through aggregation\. Therefore, while incomplete multimodal learning gives a natural foundation for modality completion and missing\-aware fusion, it does not by itself explain how to make such completion robust in federated multimodal graphs\.

### A\.5Position of FedMPO Compared with Existing Paradigms

FedMPO lies at the intersection of centralized multimodal graph learning, federated graph learning, multimodal federated learning, and incomplete multimodal learning, but is not reducible to any one of them\. Compared with centralized multimodal graph learning, our setting introduces privacy\-preserving distributed optimization and cross\-client heterogeneity\. Compared with federated graph learning, we explicitly study multi\-granularity modality missingness rather than assuming intact or unimodal node attributes\. Compared with multimodal federated learning under missing modalities, we further require the model to operate over graph\-structured data, where node semantics are jointly shaped by multimodal observations and neighborhood context\. Compared with incomplete multimodal learning, we must additionally consider how recovered information behaves under graph message passing and repeated federated aggregation\.

This difference changes the problem formulation in an essential way\. In centralized incomplete multimodal learning, a recovered modality mainly affects the prediction of the current sample\. In our setting, however, a recovered modality may affect both the current node and its neighbors through graph propagation, and the resulting local parameters may further influence other clients after server aggregation\. Therefore, the question is no longer only whether a missing modality can be reconstructed, but whether the reconstructed information is sufficiently reliable to remain useful after structural propagation and federated mixing\. This is why completion quality, fusion reliability, and optimization stability cannot be designed independently\.

FedMPO is built precisely around this observation\. Rather than treating missing\-modality recovery, graph reasoning, and federated optimization as separate components, it couples them into a single closed loop\. Training\-time mask disentanglement addresses the supervision mismatch between natural missingness and learnable reconstruction targets\. Graph\-enhanced completion injects structural context into cross\-modal recovery so that missing modalities are not reconstructed in isolation\. Missing\-aware mixture\-of\-experts fusion explicitly models the choice among observed modalities, recovered modalities, and structural fallback, instead of assuming that all available sources should be fused indiscriminately\. Finally, the full framework is trained under federated objectives that encourage cross\-round stability rather than relying on completion alone\.

In this sense, FedMPO should be viewed neither as a minor variant of incomplete multimodal learning nor as a direct adaptation of existing multimodal federated learning methods\. Its central contribution is to elevate*reliability\-aware missing\-modality handling under graph\-structured federated optimization*into the core design principle\. This is also the main point of departure between our work and prior paradigms: we focus not only on recovering missing modalities, but on deciding when recovered modalities should be trusted, how they should interact with graph context, and how their influence should be controlled under decentralized multimodal graph training\.

## Appendix BMethodology in Details

This section provides additional implementation details ofFedMPO\. We organize the framework into four stages consistent with Sec\. 3: feature and graph context encoding, topology\-aware cross\-modal generation, missing\-aware mixture\-of\-experts fusion, and task prediction with reliability\-aware federated aggregation\.

### B\.1Stage 1: Feature and Graph Context Encoding

Each client owns a private incomplete multimodal graph𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\)withNNnodes andMMmodalities\. For nodeviv\_\{i\}and modalitymm, the natural modality availability is denoted byri\(m\)∈\{0,1\}r\_\{i\}^\{\(m\)\}\\in\\\{0,1\\\}, whereri\(m\)=1r\_\{i\}^\{\(m\)\}=1indicates that the raw feature𝐱i\(m\)\\mathbf\{x\}\_\{i\}^\{\(m\)\}is observed\. Since naturally missing modalities have no ground\-truth features, they cannot be used as direct reconstruction targets\. Therefore, during training we only apply artificial masking to naturally observed modalities\. Letr¯i\(m\)∈\{0,1\}\\bar\{r\}\_\{i\}^\{\(m\)\}\\in\\\{0,1\\\}be an artificial keep\-mask sampled only whenri\(m\)=1r\_\{i\}^\{\(m\)\}=1\. The effective visibility mask and reconstruction indicator are

r~i\(m\)=ri\(m\)​r¯i\(m\),Δi\(m\)=ri\(m\)−r~i\(m\)\.\\tilde\{r\}\_\{i\}^\{\(m\)\}=r\_\{i\}^\{\(m\)\}\\bar\{r\}\_\{i\}^\{\(m\)\},\\qquad\\Delta\_\{i\}^\{\(m\)\}=r\_\{i\}^\{\(m\)\}\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\.\(15\)Here,r~i\(m\)=1\\tilde\{r\}\_\{i\}^\{\(m\)\}=1means that modalitymmis visible to the model, whileΔi\(m\)=1\\Delta\_\{i\}^\{\(m\)\}=1means that an originally observed modality is deliberately hidden for self\-supervised reconstruction\. This design separates natural missingness from training\-time masking and prevents the model from assuming that unavailable modalities are known\.

For observed modalities, we map raw features into a shared latent space:

𝐳i,raw\(m\)=ϕm​\(𝐱i\(m\)\),\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}=\\phi\_\{m\}\(\\mathbf\{x\}\_\{i\}^\{\(m\)\}\),\(16\)whereϕm​\(⋅\)\\phi\_\{m\}\(\\cdot\)is a modality\-specific encoder\. For naturally missing modalities,𝐳i,raw\(m\)\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}is not computed from raw input; a learnable null token or zero vector is used only as an implementation placeholder and is excluded from reconstruction supervision\.

A key difference between multimodal graphs and ordinary multimodal samples is that node semantics are shaped by both node attributes and graph neighborhoods\. Thus, for each target modalitymm, we construct a modality\-wise structural anchor from neighboring nodes whose modality\-mmrepresentations are visible:

𝐳i,anc\(m\)=∑vj∈𝒩​\(i\)ai​j​r~j\(m\)​𝐳j,raw\(m\)∑vj∈𝒩​\(i\)ai​j​r~j\(m\)\+ϵ\.\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}=\\frac\{\\sum\_\{v\_\{j\}\\in\\mathcal\{N\}\(i\)\}a\_\{ij\}\\tilde\{r\}\_\{j\}^\{\(m\)\}\\mathbf\{z\}\_\{j,\\mathrm\{raw\}\}^\{\(m\)\}\}\{\\sum\_\{v\_\{j\}\\in\\mathcal\{N\}\(i\)\}a\_\{ij\}\\tilde\{r\}\_\{j\}^\{\(m\)\}\+\\epsilon\}\.\(17\)Here,ai​ja\_\{ij\}is the normalized edge weight or attention coefficient, andϵ\\epsilonavoids numerical instability\. If no valid neighbor is available for modalitymm, a learnable null anchor is used\. This anchor provides a topology\-derived prior for later missing\-modality generation\.

The modality\-wise input to the graph encoder is formed by choosing the visible raw embedding when available and the structural anchor otherwise:

𝐡i\(m\)=GNNθ​\(ψm​\(r~i\(m\)​𝐳i,raw\(m\)\+\(1−r~i\(m\)\)​𝐳i,anc\(m\)\),𝒢\),\\mathbf\{h\}\_\{i\}^\{\(m\)\}=\\mathrm\{GNN\}\_\{\\theta\}\\left\(\\psi\_\{m\}\\left\(\\tilde\{r\}\_\{i\}^\{\(m\)\}\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\+\(1\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\)\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\\right\),\\mathcal\{G\}\\right\),\(18\)whereψm​\(⋅\)\\psi\_\{m\}\(\\cdot\)is a lightweight modality adapter\. For a target modalitymm, we further pool the graph\-enhanced representations of other modalities into a target\-exclusive context:

𝐡i,∖m=Pool​\(\{𝐡i\(m′\)∣m′≠m\}\)\.\\mathbf\{h\}\_\{i,\\setminus m\}=\\mathrm\{Pool\}\\left\(\\\{\\mathbf\{h\}\_\{i\}^\{\(m^\{\\prime\}\)\}\\mid m^\{\\prime\}\\neq m\\\}\\right\)\.\(19\)This target\-exclusive context prevents the generator from trivially copying the target modality during artificial masking and forces it to use complementary modalities and graph evidence\.

Algorithm 1Feature and Graph Context Encoding \(Local Client\)

0:Local graph

𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\), raw observed features

\{𝐱i\(m\)\}\\\{\\mathbf\{x\}\_\{i\}^\{\(m\)\}\\\}, natural modality masks

\{ri\(m\)\}\\\{r\_\{i\}^\{\(m\)\}\\\}\.

0:Effective masks

\{r~i\(m\)\}\\\{\\tilde\{r\}\_\{i\}^\{\(m\)\}\\\}, reconstruction indicators

\{Δi\(m\)\}\\\{\\Delta\_\{i\}^\{\(m\)\}\\\}, raw embeddings

\{𝐳i,raw\(m\)\}\\\{\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\\\}, anchors

\{𝐳i,anc\(m\)\}\\\{\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\\\}, graph\-enhanced representations

\{𝐡i\(m\)\}\\\{\\mathbf\{h\}\_\{i\}^\{\(m\)\}\\\}\.

1:foreach node

vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}and modality

mmdo

2://\(A\) Masking and raw embedding

3:Sample artificial keep\-mask

r¯i\(m\)\\bar\{r\}\_\{i\}^\{\(m\)\}only if

ri\(m\)=1r\_\{i\}^\{\(m\)\}=1\.

4:Construct

r~i\(m\)=ri\(m\)​r¯i\(m\)\\tilde\{r\}\_\{i\}^\{\(m\)\}=r\_\{i\}^\{\(m\)\}\\bar\{r\}\_\{i\}^\{\(m\)\}and

Δi\(m\)=ri\(m\)−r~i\(m\)\\Delta\_\{i\}^\{\(m\)\}=r\_\{i\}^\{\(m\)\}\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\.

5:if

ri\(m\)=1r\_\{i\}^\{\(m\)\}=1then

6:Compute raw embedding

𝐳i,raw\(m\)=ϕm​\(𝐱i\(m\)\)\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}=\\phi\_\{m\}\(\\mathbf\{x\}\_\{i\}^\{\(m\)\}\)\.

7:else

8:Use a null token or zero placeholder and exclude it from reconstruction supervision\.

9:endif

10:endfor

11:foreach node

vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}and modality

mmdo

12://\(B\) Structural anchor construction

13:Compute modality\-wise structural anchor:

14:

𝐳i,anc\(m\)=∑vj∈𝒩​\(i\)ai​j​r~j\(m\)​𝐳j,raw\(m\)∑vj∈𝒩​\(i\)ai​j​r~j\(m\)\+ϵ\\displaystyle\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}=\\frac\{\\sum\_\{v\_\{j\}\\in\\mathcal\{N\}\(i\)\}a\_\{ij\}\\tilde\{r\}\_\{j\}^\{\(m\)\}\\mathbf\{z\}\_\{j,\\mathrm\{raw\}\}^\{\(m\)\}\}\{\\sum\_\{v\_\{j\}\\in\\mathcal\{N\}\(i\)\}a\_\{ij\}\\tilde\{r\}\_\{j\}^\{\(m\)\}\+\\epsilon\}\.

15:Encode graph\-enhanced modality representation:

16:

𝐡i\(m\)=GNNθ​\(ψm​\(r~i\(m\)​𝐳i,raw\(m\)\+\(1−r~i\(m\)\)​𝐳i,anc\(m\)\),𝒢\)\\displaystyle\\mathbf\{h\}\_\{i\}^\{\(m\)\}=\\mathrm\{GNN\}\_\{\\theta\}\\left\(\\psi\_\{m\}\\left\(\\tilde\{r\}\_\{i\}^\{\(m\)\}\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\+\(1\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\)\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\\right\),\\mathcal\{G\}\\right\)\.

17:endfor

18:foreach node

vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}and target modality

mmdo

19:Pool non\-target modality representations to obtain

𝐡i,∖m\\mathbf\{h\}\_\{i,\\setminus m\}\.

20:endfor

21:Return

\{r~i\(m\)\}\\\{\\tilde\{r\}\_\{i\}^\{\(m\)\}\\\},

\{Δi\(m\)\}\\\{\\Delta\_\{i\}^\{\(m\)\}\\\},

\{𝐳i,raw\(m\)\}\\\{\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\\\},

\{𝐳i,anc\(m\)\}\\\{\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\\\}, and

\{𝐡i\(m\)\}\\\{\\mathbf\{h\}\_\{i\}^\{\(m\)\}\\\}\.

### B\.2Stage 2: Topology\-aware Cross\-modal Generation

The second stage recovers missing modality representations\. A naive cross\-modal generator predicts a missing modality only from the remaining modalities of the same node\. However, in multimodal graphs, a node’s missing semantics may be inferred from both its own observed modalities and structurally related neighbors\. Therefore,FedMPObuilds a topology\-aware context bank for each node and target modality\.

For target modalitymm, the context bankℬi\(m\)\\mathcal\{B\}\_\{i\}^\{\(m\)\}contains the current node’s visible non\-target modalities and neighboring visible modality representations:

ℬi\(m\)=\{𝐡i\(m′\)∣m′≠m,r~i\(m′\)=1\}∪\{𝐡j\(m′\)∣vj∈𝒩​\(i\),r~j\(m′\)=1\}\.\\mathcal\{B\}\_\{i\}^\{\(m\)\}=\\left\\\{\\mathbf\{h\}\_\{i\}^\{\(m^\{\\prime\}\)\}\\mid m^\{\\prime\}\\neq m,\\tilde\{r\}\_\{i\}^\{\(m^\{\\prime\}\)\}=1\\right\\\}\\cup\\left\\\{\\mathbf\{h\}\_\{j\}^\{\(m^\{\\prime\}\)\}\\mid v\_\{j\}\\in\\mathcal\{N\}\(i\),\\tilde\{r\}\_\{j\}^\{\(m^\{\\prime\}\)\}=1\\right\\\}\.\(20\)During reconstruction, the target node’s own modality\-mmrepresentation is excluded from the bank to avoid information leakage\.

The query for target modalitymmis built from the target\-exclusive context, the node\-level visibility pattern, and the target modality identity:

𝐐i\(m\)=𝐖Q​\[𝐡i,∖m​‖𝐄mask​\(𝐫~i\)‖​𝐄mod​\(m\)\],\\mathbf\{Q\}\_\{i\}^\{\(m\)\}=\\mathbf\{W\}\_\{Q\}\\left\[\\mathbf\{h\}\_\{i,\\setminus m\}\\parallel\\mathbf\{E\}\_\{\\mathrm\{mask\}\}\(\\tilde\{\\mathbf\{r\}\}\_\{i\}\)\\parallel\\mathbf\{E\}\_\{\\mathrm\{mod\}\}\(m\)\\right\],\(21\)where𝐫~i=\[r~i\(1\),…,r~i\(M\)\]\\tilde\{\\mathbf\{r\}\}\_\{i\}=\[\\tilde\{r\}\_\{i\}^\{\(1\)\},\\ldots,\\tilde\{r\}\_\{i\}^\{\(M\)\}\],𝐄mask​\(⋅\)\\mathbf\{E\}\_\{\\mathrm\{mask\}\}\(\\cdot\)embeds the missingness pattern, and𝐄mod​\(m\)\\mathbf\{E\}\_\{\\mathrm\{mod\}\}\(m\)is a learnable target\-modality embedding\. The cross\-modal and neighborhood evidence is obtained by multi\-head attention:

𝐜i\(m\)=MHA​\(𝐐i\(m\),ℬi\(m\)\)\.\\mathbf\{c\}\_\{i\}^\{\(m\)\}=\\mathrm\{MHA\}\\left\(\\mathbf\{Q\}\_\{i\}^\{\(m\)\},\\mathcal\{B\}\_\{i\}^\{\(m\)\}\\right\)\.\(22\)
The final generated representation combines attention\-based evidence, self context, and the structural anchor:

𝐳^i\(m\)=γt​\(𝐠i\(m\)⊙𝐜i\(m\)\+\(1−𝐠i\(m\)\)⊙𝐖s​𝐡i,∖m\)\+\(1−γt\)​𝐖a​𝐳i,anc\(m\)\.\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}=\\gamma\_\{t\}\\left\(\\mathbf\{g\}\_\{i\}^\{\(m\)\}\\odot\\mathbf\{c\}\_\{i\}^\{\(m\)\}\+\(1\-\\mathbf\{g\}\_\{i\}^\{\(m\)\}\)\\odot\\mathbf\{W\}\_\{s\}\\mathbf\{h\}\_\{i,\\setminus m\}\\right\)\+\(1\-\\gamma\_\{t\}\)\\mathbf\{W\}\_\{a\}\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\.\(23\)Here,𝐠i\(m\)\\mathbf\{g\}\_\{i\}^\{\(m\)\}is a learnable gate, andγt\\gamma\_\{t\}is a round\-dependent warmup factor\. At early communication rounds, the model relies more on stable structural anchors\. As training proceeds, it gradually shifts toward learned cross\-modal generation\.

The reconstruction objective is computed only on artificially masked modalities:

ℒrec=∑i=1N∑m=1MΔi\(m\)​‖𝐳^i\(m\)−sg​\(𝐳i,raw\(m\)\)‖22∑i=1N∑m=1MΔi\(m\)\+ϵ,\\mathcal\{L\}\_\{\\mathrm\{rec\}\}=\\frac\{\\sum\_\{i=1\}^\{N\}\\sum\_\{m=1\}^\{M\}\\Delta\_\{i\}^\{\(m\)\}\\left\\\|\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\-\\mathrm\{sg\}\\left\(\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\\right\)\\right\\\|\_\{2\}^\{2\}\}\{\\sum\_\{i=1\}^\{N\}\\sum\_\{m=1\}^\{M\}\\Delta\_\{i\}^\{\(m\)\}\+\\epsilon\},\(24\)wheresg​\(⋅\)\\mathrm\{sg\}\(\\cdot\)denotes the stop\-gradient operation\. Naturally missing modalities are generated for downstream learning but are not used as direct reconstruction targets\.

To prevent recovered modalities from drifting away from observed modalities, we use cross\-modal pairwise alignment rather than class\-prototype supervision\. Let

𝐳~i\(m\)=r~i\(m\)​𝐳i,raw\(m\)\+\(1−r~i\(m\)\)​𝐳^i\(m\)\.\\tilde\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}=\\tilde\{r\}\_\{i\}^\{\(m\)\}\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\+\(1\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\)\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\.\(25\)The alignment loss is

ℒalign=1\|Ω\|​∑\(i,m,m′\)∈Ω\[1−cos⁡\(𝐖al​𝐳~i\(m\),𝐖al​𝐳~i\(m′\)\)\],\\mathcal\{L\}\_\{\\mathrm\{align\}\}=\\frac\{1\}\{\|\\Omega\|\}\\sum\_\{\(i,m,m^\{\\prime\}\)\\in\\Omega\}\\left\[1\-\\cos\\left\(\\mathbf\{W\}\_\{\\mathrm\{al\}\}\\tilde\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\},\\mathbf\{W\}\_\{\\mathrm\{al\}\}\\tilde\{\\mathbf\{z\}\}\_\{i\}^\{\(m^\{\\prime\}\)\}\\right\)\\right\],\(26\)whereΩ\\Omegadenotes valid modality pairs of the same node\. This formulation does not rely on task labels and is therefore applicable to node classification, link prediction, and modality retrieval\.

Algorithm 2Topology\-aware Cross\-modal Generation \(Local Client\)

0:Graph\-enhanced representations

\{𝐡i\(m\)\}\\\{\\mathbf\{h\}\_\{i\}^\{\(m\)\}\\\}, raw embeddings

\{𝐳i,raw\(m\)\}\\\{\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\\\}, anchors

\{𝐳i,anc\(m\)\}\\\{\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\\\}, effective masks

\{r~i\(m\)\}\\\{\\tilde\{r\}\_\{i\}^\{\(m\)\}\\\}, reconstruction indicators

\{Δi\(m\)\}\\\{\\Delta\_\{i\}^\{\(m\)\}\\\}, training round

tt\.

0:Generated representations

\{𝐳^i\(m\)\}\\\{\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\\\}, reconstruction loss

ℒrec\\mathcal\{L\}\_\{\\mathrm\{rec\}\}, alignment loss

ℒalign\\mathcal\{L\}\_\{\\mathrm\{align\}\}\.

1:Hyper\-parameters:Round\-dependent warmup factor

γt\\gamma\_\{t\}\.

2:foreach node

vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}and target modality

mmdo

3:Build target\-exclusive context

𝐡i,∖m\\mathbf\{h\}\_\{i,\\setminus m\}from non\-target modalities\.

4:Build context bank

ℬi\(m\)\\mathcal\{B\}\_\{i\}^\{\(m\)\}from visible non\-target modalities and visible neighboring modality representations\.

5:Construct modality\-specific query:

6:

𝐐i\(m\)=𝐖Q​\[𝐡i,∖m​‖𝐄mask​\(𝐫~i\)‖​𝐄mod​\(m\)\]\\displaystyle\\mathbf\{Q\}\_\{i\}^\{\(m\)\}=\\mathbf\{W\}\_\{Q\}\[\\mathbf\{h\}\_\{i,\\setminus m\}\\parallel\\mathbf\{E\}\_\{\\mathrm\{mask\}\}\(\\tilde\{\\mathbf\{r\}\}\_\{i\}\)\\parallel\\mathbf\{E\}\_\{\\mathrm\{mod\}\}\(m\)\]\.

7:Obtain attention evidence

𝐜i\(m\)=MHA​\(𝐐i\(m\),ℬi\(m\)\)\\mathbf\{c\}\_\{i\}^\{\(m\)\}=\\mathrm\{MHA\}\(\\mathbf\{Q\}\_\{i\}^\{\(m\)\},\\mathcal\{B\}\_\{i\}^\{\(m\)\}\)\.

8:Generate modality representation:

9:

𝐳^i\(m\)=γt​\(𝐠i\(m\)⊙𝐜i\(m\)\+\(1−𝐠i\(m\)\)⊙𝐖s​𝐡i,∖m\)\+\(1−γt\)​𝐖a​𝐳i,anc\(m\)\\displaystyle\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}=\\gamma\_\{t\}\\left\(\\mathbf\{g\}\_\{i\}^\{\(m\)\}\\odot\\mathbf\{c\}\_\{i\}^\{\(m\)\}\+\(1\-\\mathbf\{g\}\_\{i\}^\{\(m\)\}\)\\odot\\mathbf\{W\}\_\{s\}\\mathbf\{h\}\_\{i,\\setminus m\}\\right\)\+\(1\-\\gamma\_\{t\}\)\\mathbf\{W\}\_\{a\}\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\.

10:endfor

11:Compute

ℒrec\\mathcal\{L\}\_\{\\mathrm\{rec\}\}only on artificially masked modalities with

Δi\(m\)=1\\Delta\_\{i\}^\{\(m\)\}=1\.

12:Form

𝐳~i\(m\)=r~i\(m\)​𝐳i,raw\(m\)\+\(1−r~i\(m\)\)​𝐳^i\(m\)\\tilde\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}=\\tilde\{r\}\_\{i\}^\{\(m\)\}\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\+\(1\-\\tilde\{r\}\_\{i\}^\{\(m\)\}\)\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\.

13:Compute

ℒalign\\mathcal\{L\}\_\{\\mathrm\{align\}\}over valid observed/recovered modality pairs

Ω\\Omega\.

14:Return

\{𝐳^i\(m\)\}\\\{\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\\\},

ℒrec\\mathcal\{L\}\_\{\\mathrm\{rec\}\}, and

ℒalign\\mathcal\{L\}\_\{\\mathrm\{align\}\}\.

### B\.3Stage 3: Missing\-aware Mixture\-of\-Experts Fusion

Although Stage 2 provides candidate representations for missing modalities, not all recovered features are equally reliable\. Recovery is usually more reliable when the node has rich observed modalities and informative neighbors, but less reliable when both node\-level observations and neighborhood evidence are sparse\. Therefore,FedMPOuses uncertainty\-aware expert routing to distinguish observed and recovered signals\.

We first estimate modality\-wise recovery uncertainty:

ui\(m\)=\{0,r~i\(m\)=1,σ​\(fθu​\[𝐳^i\(m\)​‖𝐡i,∖m‖​𝐳i,anc\(m\)\]\),r~i\(m\)=0\.u\_\{i\}^\{\(m\)\}=\\begin\{cases\}0,&\\tilde\{r\}\_\{i\}^\{\(m\)\}=1,\\\\ \\sigma\\left\(f\_\{\\theta\_\{u\}\}\\left\[\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\\parallel\\mathbf\{h\}\_\{i,\\setminus m\}\\parallel\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\\right\]\\right\),&\\tilde\{r\}\_\{i\}^\{\(m\)\}=0\.\\end\{cases\}\(27\)Here,ui\(m\)u\_\{i\}^\{\(m\)\}measures recovery uncertainty rather than raw\-modality quality\. Thus, visible observed modalities are assigned zero recovery uncertainty\. The uncertainty head is calibrated on artificially masked modalities by matching uncertainty scores to normalized reconstruction errors:

ℒunc=1∑i,mΔi\(m\)\+ϵ​∑i,mΔi\(m\)​\(ui\(m\)−NormErri\(m\)\)2\.\\mathcal\{L\}\_\{\\mathrm\{unc\}\}=\\frac\{1\}\{\\sum\_\{i,m\}\\Delta\_\{i\}^\{\(m\)\}\+\\epsilon\}\\sum\_\{i,m\}\\Delta\_\{i\}^\{\(m\)\}\\left\(u\_\{i\}^\{\(m\)\}\-\\mathrm\{NormErr\}\_\{i\}^\{\(m\)\}\\right\)^\{2\}\.\(28\)
The router considers modality visibility, recovery uncertainty, node\-level missing ratioρi\\rho\_\{i\}, and client\-level missing ratioρk\\rho\_\{k\}:

\[wi,obs\(m\),wi,rec\(m\)\]=Softmax​\(fθr​\[r~i\(m\)​‖ui\(m\)‖​ρi∥ρk\]/τ\)\.\\left\[w\_\{i,\\mathrm\{obs\}\}^\{\(m\)\},w\_\{i,\\mathrm\{rec\}\}^\{\(m\)\}\\right\]=\\mathrm\{Softmax\}\\left\(f\_\{\\theta\_\{r\}\}\\left\[\\tilde\{r\}\_\{i\}^\{\(m\)\}\\parallel u\_\{i\}^\{\(m\)\}\\parallel\\rho\_\{i\}\\parallel\\rho\_\{k\}\\right\]/\\tau\\right\)\.\(29\)The observed and recovered experts are

𝐞i,obs\(m\)=Eobs​\(𝐳i,raw\(m\)\),𝐞i,rec\(m\)=Erec​\(𝐳^i\(m\)\)\.\\mathbf\{e\}\_\{i,\\mathrm\{obs\}\}^\{\(m\)\}=E\_\{\\mathrm\{obs\}\}\\left\(\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\\right\),\\qquad\\mathbf\{e\}\_\{i,\\mathrm\{rec\}\}^\{\(m\)\}=E\_\{\\mathrm\{rec\}\}\\left\(\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\\right\)\.\(30\)The routed modality representation is

𝐟i\(m\)=\{wi,obs\(m\)​𝐞i,obs\(m\)\+wi,rec\(m\)​𝐞i,rec\(m\),r~i\(m\)=1,𝐞i,rec\(m\),r~i\(m\)=0\.\\mathbf\{f\}\_\{i\}^\{\(m\)\}=\\begin\{cases\}w\_\{i,\\mathrm\{obs\}\}^\{\(m\)\}\\mathbf\{e\}\_\{i,\\mathrm\{obs\}\}^\{\(m\)\}\+w\_\{i,\\mathrm\{rec\}\}^\{\(m\)\}\\mathbf\{e\}\_\{i,\\mathrm\{rec\}\}^\{\(m\)\},&\\tilde\{r\}\_\{i\}^\{\(m\)\}=1,\\\\ \\mathbf\{e\}\_\{i,\\mathrm\{rec\}\}^\{\(m\)\},&\\tilde\{r\}\_\{i\}^\{\(m\)\}=0\.\\end\{cases\}\(31\)
To prevent routing collapse, we use a simple load\-balancing regularizer over the average routing probabilities:

ℒbal=∑q∈\{obs,rec\}\(w¯q−12\)2,w¯q=1N​M​∑i=1N∑m=1Mwi,q\(m\)\.\\mathcal\{L\}\_\{\\mathrm\{bal\}\}=\\sum\_\{q\\in\\\{\\mathrm\{obs\},\\mathrm\{rec\}\\\}\}\\left\(\\bar\{w\}\_\{q\}\-\\frac\{1\}\{2\}\\right\)^\{2\},\\qquad\\bar\{w\}\_\{q\}=\\frac\{1\}\{NM\}\\sum\_\{i=1\}^\{N\}\\sum\_\{m=1\}^\{M\}w\_\{i,q\}^\{\(m\)\}\.\(32\)The routing loss is

ℒroute=ℒunc\+λbal​ℒbal\.\\mathcal\{L\}\_\{\\mathrm\{route\}\}=\\mathcal\{L\}\_\{\\mathrm\{unc\}\}\+\\lambda\_\{\\mathrm\{bal\}\}\\mathcal\{L\}\_\{\\mathrm\{bal\}\}\.\(33\)
After modality\-level routing, modalities are fused according to their estimated reliability:

ai\(m\)=exp⁡\(−ui\(m\)\)∑m′=1Mexp⁡\(−ui\(m′\)\)\+ϵ\.a\_\{i\}^\{\(m\)\}=\\frac\{\\exp\(\-u\_\{i\}^\{\(m\)\}\)\}\{\\sum\_\{m^\{\\prime\}=1\}^\{M\}\\exp\(\-u\_\{i\}^\{\(m^\{\\prime\}\)\}\)\+\\epsilon\}\.\(34\)For highly incomplete nodes, a structural fallback expert provides graph\-only evidence\. Letu¯i=1M​∑m=1Mui\(m\)\\bar\{u\}\_\{i\}=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}u\_\{i\}^\{\(m\)\}be the average uncertainty\. The fallback coefficient is

αfb,i=σ​\(ffb​\[ρi∥u¯i\]\)\.\\alpha\_\{\\mathrm\{fb\},i\}=\\sigma\\left\(f\_\{\\mathrm\{fb\}\}\\left\[\\rho\_\{i\}\\parallel\\bar\{u\}\_\{i\}\\right\]\\right\)\.\(35\)The final fused node representation is

𝐫i=LayerNorm​\(\(1−αfb,i\)​∑m=1Mai\(m\)​𝐟i\(m\)\+αfb,i​Estruct​\(𝐡str,i\)\),\\mathbf\{r\}\_\{i\}=\\mathrm\{LayerNorm\}\\left\(\(1\-\\alpha\_\{\\mathrm\{fb\},i\}\)\\sum\_\{m=1\}^\{M\}a\_\{i\}^\{\(m\)\}\\mathbf\{f\}\_\{i\}^\{\(m\)\}\+\\alpha\_\{\\mathrm\{fb\},i\}E\_\{\\mathrm\{struct\}\}\(\\mathbf\{h\}\_\{\\mathrm\{str\},i\}\)\\right\),\(36\)where𝐡str,i\\mathbf\{h\}\_\{\\mathrm\{str\},i\}is a structure\-only representation obtained from the local graph encoder\.

Algorithm 3Missing\-aware MoE Fusion \(Local Client\)

0:Raw embeddings

𝐳i,raw\(m\)\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}, generated features

𝐳^i\(m\)\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}, anchors

𝐳i,anc\(m\)\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}, contexts

𝐡i,∖m\\mathbf\{h\}\_\{i,\\setminus m\}, effective masks

r~i\(m\)\\tilde\{r\}\_\{i\}^\{\(m\)\}, node missing ratio

ρi\\rho\_\{i\}, client missing ratio

ρk\\rho\_\{k\}\.

0:Fused node representations

\{𝐫i\}\\\{\\mathbf\{r\}\_\{i\}\\\}, routing loss

ℒroute\\mathcal\{L\}\_\{\\mathrm\{route\}\}, uncertainty statistics\.

1:foreach node

vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}and modality

mmdo

2://\(A\) Recovery uncertainty estimation

3:if

r~i\(m\)=1\\tilde\{r\}\_\{i\}^\{\(m\)\}=1then

4:Assign zero recovery uncertainty

ui\(m\)=0u\_\{i\}^\{\(m\)\}=0\.

5:else

6:Estimate recovery uncertainty:

7:

ui\(m\)=σ​\(fθu​\[𝐳^i\(m\)​‖𝐡i,∖m‖​𝐳i,anc\(m\)\]\)\\displaystyle u\_\{i\}^\{\(m\)\}=\\sigma\\left\(f\_\{\\theta\_\{u\}\}\[\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\\parallel\\mathbf\{h\}\_\{i,\\setminus m\}\\parallel\\mathbf\{z\}\_\{i,\\mathrm\{anc\}\}^\{\(m\)\}\]\\right\)\.

8:endif

9://\(B\) Expert routing

10:Compute routing weights:

11:

\[wi,obs\(m\),wi,rec\(m\)\]=Softmax​\(fθr​\[r~i\(m\)​‖ui\(m\)‖​ρi∥ρk\]/τ\)\\displaystyle\[w\_\{i,\\mathrm\{obs\}\}^\{\(m\)\},w\_\{i,\\mathrm\{rec\}\}^\{\(m\)\}\]=\\mathrm\{Softmax\}\\left\(f\_\{\\theta\_\{r\}\}\[\\tilde\{r\}\_\{i\}^\{\(m\)\}\\parallel u\_\{i\}^\{\(m\)\}\\parallel\\rho\_\{i\}\\parallel\\rho\_\{k\}\]/\\tau\\right\)\.

12:if

r~i\(m\)=1\\tilde\{r\}\_\{i\}^\{\(m\)\}=1then

13:

𝐟i\(m\)=wi,obs\(m\)​Eobs​\(𝐳i,raw\(m\)\)\+wi,rec\(m\)​Erec​\(𝐳^i\(m\)\)\\mathbf\{f\}\_\{i\}^\{\(m\)\}=w\_\{i,\\mathrm\{obs\}\}^\{\(m\)\}E\_\{\\mathrm\{obs\}\}\(\\mathbf\{z\}\_\{i,\\mathrm\{raw\}\}^\{\(m\)\}\)\+w\_\{i,\\mathrm\{rec\}\}^\{\(m\)\}E\_\{\\mathrm\{rec\}\}\(\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\)\.

14:else

15:

𝐟i\(m\)=Erec​\(𝐳^i\(m\)\)\\mathbf\{f\}\_\{i\}^\{\(m\)\}=E\_\{\\mathrm\{rec\}\}\(\\hat\{\\mathbf\{z\}\}\_\{i\}^\{\(m\)\}\)\.

16:endif

17:endfor

18:foreach node

vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}do

19://\(C\) Reliability\-aware fusion and structural fallback

20:Compute reliability weights

ai\(m\)=exp⁡\(−ui\(m\)\)∑m′=1Mexp⁡\(−ui\(m′\)\)\+ϵa\_\{i\}^\{\(m\)\}=\\frac\{\\exp\(\-u\_\{i\}^\{\(m\)\}\)\}\{\\sum\_\{m^\{\\prime\}=1\}^\{M\}\\exp\(\-u\_\{i\}^\{\(m^\{\\prime\}\)\}\)\+\\epsilon\}\.

21:Compute average uncertainty

u¯i=1M​∑m=1Mui\(m\)\\bar\{u\}\_\{i\}=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}u\_\{i\}^\{\(m\)\}\.

22:Estimate fallback coefficient

αfb,i=σ​\(ffb​\[ρi∥u¯i\]\)\\alpha\_\{\\mathrm\{fb\},i\}=\\sigma\(f\_\{\\mathrm\{fb\}\}\[\\rho\_\{i\}\\parallel\\bar\{u\}\_\{i\}\]\)\.

23:Fuse modality experts with structural fallback:

24:

𝐫i=LayerNorm​\(\(1−αfb,i\)​∑m=1Mai\(m\)​𝐟i\(m\)\+αfb,i​Estruct​\(𝐡str,i\)\)\\displaystyle\\mathbf\{r\}\_\{i\}=\\mathrm\{LayerNorm\}\\left\(\(1\-\\alpha\_\{\\mathrm\{fb\},i\}\)\\sum\_\{m=1\}^\{M\}a\_\{i\}^\{\(m\)\}\\mathbf\{f\}\_\{i\}^\{\(m\)\}\+\\alpha\_\{\\mathrm\{fb\},i\}E\_\{\\mathrm\{struct\}\}\(\\mathbf\{h\}\_\{\\mathrm\{str\},i\}\)\\right\)\.

25:endfor

26:Compute

ℒroute\\mathcal\{L\}\_\{\\mathrm\{route\}\}from uncertainty calibration and expert load balancing\.

27:Return

\{𝐫i\}\\\{\\mathbf\{r\}\_\{i\}\\\},

ℒroute\\mathcal\{L\}\_\{\\mathrm\{route\}\}, and uncertainty statistics\.

### B\.4Stage 4: Task Prediction and Reliability\-aware Federated Aggregation

After obtaining the fused representation𝐫i\\mathbf\{r\}\_\{i\}, each client performs a lightweight post\-completion graph refinement:

𝐫i′=LayerNorm​\(𝐫i\+σ​\(SAGEConv​\(𝐑,𝒢\)i\)\),\\mathbf\{r\}^\{\\prime\}\_\{i\}=\\mathrm\{LayerNorm\}\\left\(\\mathbf\{r\}\_\{i\}\+\\sigma\\left\(\\mathrm\{SAGEConv\}\(\\mathbf\{R\},\\mathcal\{G\}\)\_\{i\}\\right\)\\right\),\(37\)where𝐑\\mathbf\{R\}is the matrix of fused node representations\. A task\-specific prediction head is then applied:

𝐲^i=ftask​\(𝐫i′\)\.\\hat\{\\mathbf\{y\}\}\_\{i\}=f\_\{\\mathrm\{task\}\}\(\\mathbf\{r\}^\{\\prime\}\_\{i\}\)\.\(38\)The task loss depends on the downstream task: cross\-entropy for node classification, pairwise prediction loss for link prediction, and contrastive ranking loss for modality retrieval\.

The local objective on clientkkis

ℒk=ℒtask\(k\)\+λrec​ℒrec\(k\)\+λalign​ℒalign\(k\)\+λroute​ℒroute\(k\)\.\\mathcal\{L\}\_\{k\}=\\mathcal\{L\}\_\{\\mathrm\{task\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{rec\}\}\\mathcal\{L\}\_\{\\mathrm\{rec\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{align\}\}\\mathcal\{L\}\_\{\\mathrm\{align\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{route\}\}\\mathcal\{L\}\_\{\\mathrm\{route\}\}^\{\(k\)\}\.\(39\)Here,ℒroute\(k\)\\mathcal\{L\}\_\{\\mathrm\{route\}\}^\{\(k\)\}contains uncertainty calibration and expert load\-balancing regularization\. This local objective jointly optimizes task prediction, self\-supervised modality generation, cross\-modal consistency, and missing\-aware routing\.

After local training, each client uploads the updated model parameters and lightweight reliability statistics to the server\. The uploaded model parameters include the trainable parameters of the generator, fusion module, graph backbone, and task head\. The reliability statistics include average recovery uncertaintyu¯k\\bar\{u\}\_\{k\}, average reconstruction errore¯k\\bar\{e\}\_\{k\}, and client\-level missing ratioρk\\rho\_\{k\}\. These statistics are scalar summaries rather than raw node features, raw modality contents, recovered modality representations, or graph structures\.

The server first converts the client reliability statistics into a reliability score:

sk=exp⁡\(−ηu​u¯k−ηe​e¯k−ηρ​ρk\),s\_\{k\}=\\exp\\left\(\-\\eta\_\{u\}\\bar\{u\}\_\{k\}\-\\eta\_\{e\}\\bar\{e\}\_\{k\}\-\\eta\_\{\\rho\}\\rho\_\{k\}\\right\),\(40\)whereηu\\eta\_\{u\},ηe\\eta\_\{e\}, andηρ\\eta\_\{\\rho\}are aggregation\-scale coefficients controlling the effects of recovery uncertainty, reconstruction error, and client\-level missingness, respectively\. These coefficients should not be confused with the modality missing rateη\\etaused in the robustness study\.

The aggregation weight of clientkkis then computed by combining its local graph size and reliability score:

ωk=\|𝒱k\|​sk∑j∈𝒮t\|𝒱j\|​sj\+ϵ,\\omega\_\{k\}=\\frac\{\|\\mathcal\{V\}\_\{k\}\|s\_\{k\}\}\{\\sum\_\{j\\in\\mathcal\{S\}^\{t\}\}\|\\mathcal\{V\}\_\{j\}\|s\_\{j\}\+\\epsilon\},\(41\)where𝒮t\\mathcal\{S\}^\{t\}denotes the set of selected clients at communication roundtt\. The global model is updated as

Θt\+1=∑k∈𝒮tωk​Θkt\+1\.\\Theta^\{t\+1\}=\\sum\_\{k\\in\\mathcal\{S\}^\{t\}\}\\omega\_\{k\}\\Theta\_\{k\}^\{t\+1\}\.\(42\)When clients have similar modality completeness and recovery reliability, the reliability scores become similar and the rule reduces to standard data\-size\-weighted FedAvg\. Otherwise, clients dominated by unreliable recovered modalities are down\-weighted, preventing completion errors from being repeatedly amplified across communication rounds\.

Algorithm 4Task Prediction and Reliability\-aware Federated Aggregation

0:Total communication rounds

TT, selected clients

𝒮t\\mathcal\{S\}^\{t\}, initialized global model

Θ0\\Theta^\{0\}, reliability scales

ηu,ηe,ηρ\\eta\_\{u\},\\eta\_\{e\},\\eta\_\{\\rho\}\.

0:Optimized global model

ΘT\\Theta^\{T\}\.

1:for

t=0,1,…,T−1t=0,1,\\dots,T\-1do

2:Server broadcasts global model

Θt\\Theta^\{t\}to selected clients

𝒮t\\mathcal\{S\}^\{t\}\.

3:foreach client

k∈𝒮tk\\in\\mathcal\{S\}^\{t\}in paralleldo

4:Initialize local model with

Θt\\Theta^\{t\}\.

5:Execute Algorithm 1 for feature and graph context encoding\.

6:Execute Algorithm 2 for topology\-aware cross\-modal generation\.

7:Execute Algorithm 3 for missing\-aware MoE fusion\.

8://\(A\) Local task prediction

9:foreach node

vi∈𝒱kv\_\{i\}\\in\\mathcal\{V\}\_\{k\}do

10:Refine fused representation:

11:

𝐫i′=LayerNorm​\(𝐫i\+σ​\(SAGEConv​\(𝐑,𝒢k\)i\)\)\\displaystyle\\mathbf\{r\}^\{\\prime\}\_\{i\}=\\mathrm\{LayerNorm\}\\left\(\\mathbf\{r\}\_\{i\}\+\\sigma\(\\mathrm\{SAGEConv\}\(\\mathbf\{R\},\\mathcal\{G\}\_\{k\}\)\_\{i\}\)\\right\)\.

12:Predict task output

𝐲^i=ftask​\(𝐫i′\)\\hat\{\\mathbf\{y\}\}\_\{i\}=f\_\{\\mathrm\{task\}\}\(\\mathbf\{r\}^\{\\prime\}\_\{i\}\)\.

13:endfor

14:Compute task loss

ℒtask\(k\)\\mathcal\{L\}\_\{\\mathrm\{task\}\}^\{\(k\)\}according to the downstream task\.

15:Compute local objective:

16:

ℒk=ℒtask\(k\)\+λrec​ℒrec\(k\)\+λalign​ℒalign\(k\)\+λroute​ℒroute\(k\)\\displaystyle\\mathcal\{L\}\_\{k\}=\\mathcal\{L\}\_\{\\mathrm\{task\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{rec\}\}\\mathcal\{L\}\_\{\\mathrm\{rec\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{align\}\}\\mathcal\{L\}\_\{\\mathrm\{align\}\}^\{\(k\)\}\+\\lambda\_\{\\mathrm\{route\}\}\\mathcal\{L\}\_\{\\mathrm\{route\}\}^\{\(k\)\}\.

17:Update local parameters and obtain

Θkt\+1\\Theta\_\{k\}^\{t\+1\}\.

18:Compute client reliability statistics

\(u¯k,e¯k,ρk\)\(\\bar\{u\}\_\{k\},\\bar\{e\}\_\{k\},\\rho\_\{k\}\)\.

19:Upload updated parameters

Θkt\+1\\Theta\_\{k\}^\{t\+1\}and statistics

\(u¯k,e¯k,ρk\)\(\\bar\{u\}\_\{k\},\\bar\{e\}\_\{k\},\\rho\_\{k\}\)to the server\.

20:endfor

21://\(B\) Server\-side reliability\-aware aggregation

22:foreach client

k∈𝒮tk\\in\\mathcal\{S\}^\{t\}do

23:Compute reliability score:

24:

sk=exp⁡\(−ηu​u¯k−ηe​e¯k−ηρ​ρk\)\\displaystyle s\_\{k\}=\\exp\\left\(\-\\eta\_\{u\}\\bar\{u\}\_\{k\}\-\\eta\_\{e\}\\bar\{e\}\_\{k\}\-\\eta\_\{\\rho\}\\rho\_\{k\}\\right\)\.

25:Compute aggregation weight:

26:

ωk=\|𝒱k\|​sk∑j∈𝒮t\|𝒱j\|​sj\+ϵ\\displaystyle\\omega\_\{k\}=\\frac\{\|\\mathcal\{V\}\_\{k\}\|s\_\{k\}\}\{\\sum\_\{j\\in\\mathcal\{S\}^\{t\}\}\|\\mathcal\{V\}\_\{j\}\|s\_\{j\}\+\\epsilon\}\.

27:endfor

28:Update global model:

29:

Θt\+1=∑k∈𝒮tωk​Θkt\+1\\displaystyle\\Theta^\{t\+1\}=\\sum\_\{k\\in\\mathcal\{S\}^\{t\}\}\\omega\_\{k\}\\Theta\_\{k\}^\{t\+1\}\.

30:endfor

31:Return

ΘT\\Theta^\{T\}\.

## Appendix CExperiment in Details

### C\.1Computation Resource

All experiments were conducted on distributed container nodes of the AutoDL cloud computing platform\. Table[3](https://arxiv.org/html/2605.12584#A3.T3)summarizes the machine configuration and software stack used in our experiments\. The environment provides sufficient GPU memory for multimodal feature processing and federated graph training\.

Table 3:Compute environment used for all experiments\.
### C\.2Datasets

We evaluateFedMPOon six multimodal graph datasets covering three downstream tasks: node classification, link prediction, and modality retrieval\. For all datasets, image or video modalities are encoded into 512\-dimensional features, while textual modalities are encoded into 768\-dimensional features\. Dataset statistics are summarized in Table[4](https://arxiv.org/html/2605.12584#A3.T4)\.

Ele\-fashionis a heterogeneous product graph constructed by merging Amazon Electronics and Fashion categories\. Nodes represent products, and edges are derived from cross\-category co\-purchasing relations\. Textual features are extracted from product titles, descriptions, and specifications, while visual features are extracted from product images\. This dataset is used for node classification, where labels correspond to high\-level product categories\.

Groceryis sourced from Amazon Grocery and Gourmet Food\. Nodes represent food or household products, and edges are built from co\-purchasing relations\. Textual attributes are extracted from product titles and descriptions, while visual attributes are extracted from packaging images\. This dataset is used for node classification with fine\-grained product categories\.

DYis sourced from a short\-video platform\. Nodes represent short videos, and edges are constructed from co\-interaction relations, such as videos liked or watched by similar users\. Textual features are extracted from captions and hashtags, while visual features are obtained from sampled video frames\. This dataset is used for link prediction\.

Bili\_Danceis collected from the dance category of Bilibili\. Nodes represent dance videos, and edges encode co\-viewing or sequential watching relations among users\. Textual attributes are derived from video descriptions and tags, while visual features are extracted from keyframes\. This dataset is used for link prediction, evaluating whether the learned representations can recover potential structural relations under missing modalities\.

Toysoriginates from Amazon Toys and Games\. Nodes represent toy products, and edges are derived from co\-purchasing relations\. Textual features are extracted from product descriptions and age recommendations, while visual features are extracted from product images\. This dataset is used for modality retrieval\.

Flickr30kis an image\-text dataset for multimodal alignment\. We construct a multimodal graph where nodes correspond to images or textual descriptions, and edges reflect semantic image\-text associations\. This dataset is used for modality retrieval, evaluating cross\-modal matching between visual and textual representations\.

Table 4:Statistics of the experimental datasets\.
### C\.3Baseline Models

We compareFedMPOwith three groups of baselines: basic federated graph learning methods, advanced federated graph learning methods, and multimodal federated learning methods\. For fair comparison, all baselines use the same multimodal features, client partitions, missing\-modality protocols, and evaluation splits\.

##### Basic federated graph learning baselines\.

FedAvg\-Zeroadapts standard FedAvg to incomplete multimodal graphs by zero\-padding missing modality features before local training\. This baseline evaluates whether simple missing\-feature filling is sufficient under federated multimodal graph settings\.

FedGCNextends graph convolutional networks to federated training\. It models local graph topology but does not explicitly recover or distinguish missing multimodal features\.

FedGraphSAGEapplies the inductive GraphSAGE architecture in a federated setting\. Although scalable for graph representation learning, it assumes directly usable node features and is vulnerable when modality observations are incomplete\.

##### Advanced federated graph learning baselines\.

FedProtomitigates statistical heterogeneity by sharing class\-level prototypes rather than only model parameters\. However, under missing modalities, locally estimated prototypes can become biased by incomplete feature observations\.

FedPubperforms personalized subgraph federated learning by estimating client similarities and adaptively aggregating model updates\. It handles subgraph heterogeneity but does not explicitly model modality missingness or completion reliability\.

FedLAPuses Laplacian\-based graph regularization to improve federated graph learning under structural heterogeneity\. It focuses on graph structure but lacks cross\-modal recovery mechanisms\.

S2FGLcombines spatial and spectral information to reduce client drift in federated graph learning\. It is designed for graph distribution shifts but does not explicitly address incomplete multimodal attributes\.

FedSPAaddresses homophily heterogeneity in federated graph learning through structure\-aware propagation and aggregation\. Its design mainly targets topology heterogeneity rather than missing multimodal features\.

FedIIHmodels inter\-client and intra\-client heterogeneity in federated subgraphs\. Although effective for heterogeneous graph distributions, it does not explicitly recover missing modalities or estimate completion reliability\.

##### Multimodal federated learning baselines\.

FedMVPleverages multimodal priors for federated learning with incomplete modalities\. It is effective for non\-graph multimodal data, but it does not explicitly use graph neighborhoods for topology\-aware modality recovery\.

FedMACaddresses partial\-modality missingness with cross\-modal aggregation and contrastive regularization\. However, it treats samples independently and does not model graph\-structured dependencies during completion\.

### C\.4Downstream Tasks

We evaluateFedMPOon graph\-centric and modality\-centric downstream tasks\. Graph\-centric tasks include node classification and link prediction, while the modality\-centric task is modality retrieval\. These tasks jointly evaluate whetherFedMPOcan learn robust representations under incomplete modalities, graph heterogeneity, and federated non\-IID partitions\.

##### Node Classification\.

Node classification evaluates whether the learned node representations are discriminative under incomplete multimodal inputs\. Given node embeddings produced by the local graph model, a task\-specific prediction head outputs class probabilities\. We evaluate node classification on Ele\-fashion and Grocery using Accuracy and F1\-score\.

##### Link Prediction\.

Link prediction evaluates whether the model can infer missing or potential edges from multimodal graph representations\. For each candidate node pair, the model computes a similarity or scoring function based on the learned embeddings\. Positive edges are sampled from observed graph links, while negative edges are generated by negative sampling\. We evaluate link prediction on DY and Bili\_Dance using AUC and AP\.

##### Modality Retrieval\.

Modality retrieval evaluates cross\-modal alignment under incomplete modalities\. Given a query from one modality, the model ranks candidate instances from another modality in the shared representation space\. This task directly reflects whether recovered and observed modalities are aligned in a retrieval setting\. We evaluate modality retrieval on Toys and Flickr30k using Recall@K and MRR\.

### C\.5Evaluation Metrics

We use standard evaluation metrics for the three downstream tasks\.

Accuracy \(Acc\)\.For node classification, Accuracy measures the proportion of correctly classified nodes:

Acc=1N​∑i=1N𝕀​\(y^i=yi\),\\mathrm\{Acc\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\(\\hat\{y\}\_\{i\}=y\_\{i\}\),\(43\)whereNNis the number of evaluated nodes,yiy\_\{i\}is the ground\-truth label, andy^i\\hat\{y\}\_\{i\}is the predicted label\.

F1\-score\.F1\-score is the harmonic mean of Precision and Recall:

F1=2⋅Precision⋅RecallPrecision\+Recall\.\\mathrm\{F1\}=2\\cdot\\frac\{\\mathrm\{Precision\}\\cdot\\mathrm\{Recall\}\}\{\\mathrm\{Precision\}\+\\mathrm\{Recall\}\}\.\(44\)It complements Accuracy when class distributions are imbalanced\.

Area Under the ROC Curve \(AUC\)\.For link prediction, AUC evaluates whether positive edges receive higher scores than negative edges:

AUC=∑i∈𝒫ranki−\|𝒫\|​\(\|𝒫\|\+1\)2\|𝒫\|×\|𝒩\|,\\mathrm\{AUC\}=\\frac\{\\sum\_\{i\\in\\mathcal\{P\}\}\\mathrm\{rank\}\_\{i\}\-\\frac\{\|\\mathcal\{P\}\|\(\|\\mathcal\{P\}\|\+1\)\}\{2\}\}\{\|\\mathcal\{P\}\|\\times\|\\mathcal\{N\}\|\},\(45\)where𝒫\\mathcal\{P\}and𝒩\\mathcal\{N\}denote positive and negative edge sets\.

Average Precision \(AP\)\.AP summarizes the precision\-recall curve:

AP=∑n\(Rn−Rn−1\)​Pn,\\mathrm\{AP\}=\\sum\_\{n\}\(R\_\{n\}\-R\_\{n\-1\}\)P\_\{n\},\(46\)wherePnP\_\{n\}andRnR\_\{n\}denote the precision and recall at thenn\-th threshold\.

Recall@K \(R@K\)\.For modality retrieval, Recall@K measures whether the correct target appears in the top\-KKretrieved candidates:

R​@​K=1\|Q\|​∑i=1\|Q\|𝕀​\(ranki≤K\),\\mathrm\{R@K\}=\\frac\{1\}\{\|Q\|\}\\sum\_\{i=1\}^\{\|Q\|\}\\mathbb\{I\}\(\\mathrm\{rank\}\_\{i\}\\leq K\),\(47\)whereQQis the query set\.

Mean Reciprocal Rank \(MRR\)\.MRR evaluates the rank position of the first correct retrieved result:

MRR=1\|Q\|​∑i=1\|Q\|1ranki\.\\mathrm\{MRR\}=\\frac\{1\}\{\|Q\|\}\\sum\_\{i=1\}^\{\|Q\|\}\\frac\{1\}\{\\mathrm\{rank\}\_\{i\}\}\.\(48\)

### C\.6Further Experimental Analysis and Discussions

#### C\.6\.1Additional Ablation Analysis

We provide a more detailed ablation analysis in Fig\.[5](https://arxiv.org/html/2605.12584#A3.F5)\. The left panel compares the fullFedMPOmodel with three structural variants: w/o AGMG, w/o MoE, and w/o Align\. Specifically, w/o AGMG removes the topology\-aware cross\-modal generation module, w/o MoE replaces missing\-aware expert routing with simple fusion, and w/o Align removes the cross\-modal alignment objective\. Across node classification, link prediction, and modality retrieval datasets, all ablated variants lead to consistent performance degradation, verifying that each component contributes to robust federated multimodal graph learning\. Among them, removing AGMG causes the largest drop in most cases, indicating that topology\-aware modality completion is essential for recovering reliable missing representations\. The degradation of w/o MoE further shows that simply fusing observed and recovered modalities is insufficient, since recovered features may contain task\-irrelevant or noisy semantics under severe modality missingness\. The performance drop of w/o Align also confirms that cross\-modal consistency regularization helps reduce semantic mismatch between observed and generated modality representations\.

The right panel further comparesFedMPOwith w/o RelAgg, where reliability\-aware aggregation is replaced by standard data\-size\-weighted FedAvg\.FedMPOconsistently achieves better performance across all six datasets, with gains ranging from 1\.6 to 3\.5 percentage points\. This demonstrates that aggregation based only on local data size can over\-emphasize unreliable client updates when clients have heterogeneous modality availability and recovery quality\. By incorporating client\-level uncertainty, reconstruction error, and missing ratio into aggregation weights, reliability\-aware aggregation improves the robustness of the global model\. Meanwhile, the cost comparison suggests thatFedMPOmaintains a favorable performance–cost trade\-off, since more reliable aggregation can stabilize federated optimization without introducing heavy communication overhead\.

#### C\.6\.2MoE Router Diagnostics and Expert Behavior

To analyze the behavior of the missing\-aware MoE module, we track routing probabilities, recovery uncertainty, expert output norms, and the similarity between recovered representations and structure\-only fallback representations\. At early communication rounds, recovered features are less reliable, and the router assigns relatively higher weights to the structural fallback\. As the topology\-aware generator becomes better trained, the router gradually increases the contribution of recovered and observed modality experts\. This trend suggests that the router learns to adaptively adjust the reliance on observed, recovered, and structural signals according to recovery reliability\.

#### C\.6\.3Extended Robustness to Varying Missing Rates

Beyond the main robustness study on node classification, we further evaluateFedMPOunder missing rates ranging from 30% to 70% on link prediction and modality retrieval tasks\. As missingness increases, all methods degrade, butFedMPOshows a slower performance drop\. This is because AGMG exploits neighborhood context to provide topology\-aware recovery, while the missing\-aware router reduces the influence of uncertain recovered modalities\. These results support the robustness ofFedMPOunder severe modality incompleteness\.

![Refer to caption](https://arxiv.org/html/2605.12584v1/x4.png)Figure 5:Ablation study ofFedMPO\.\(a\) Consistent performance drops in variants \(w/o AGMG, MoE, Align\) prove the necessity of each structural design\. \(b\) Comparisons w/ and w/o RelAgg confirm that reliability\-aware aggregation is critical for stabilizing performance and reducing training costs\.
#### C\.6\.4Round\-wise Convergence

We also compare round\-wise convergence across methods\.FedMPOreaches a stable performance region in fewer communication rounds than baselines that either ignore missing modalities or rely on standard aggregation\. The improvement mainly comes from two factors: unreliable recovered signals are filtered during local routing, and unreliable client updates are down\-weighted during server aggregation\. Therefore,FedMPOimproves not only final performance but also optimization stability under missing\-modality and non\-IID settings\.

#### C\.6\.5Runtime and Resource Usage Study

We report system\-level efficiency in Fig\.[6](https://arxiv.org/html/2605.12584#A3.F6)to evaluate the additional cost introduced by topology\-aware generation and missing\-aware routing\. Compared with FedAvg\-Zero,FedMPOintroduces computational and memory overhead \(as shown in Fig\.[6\(a\)](https://arxiv.org/html/2605.12584#A3.F6.sf1)and[6\(b\)](https://arxiv.org/html/2605.12584#A3.F6.sf2)\) due to AGMG, uncertainty estimation, and MoE routing\. However, these modules operate on compact hidden representations rather than raw image or text inputs\. Moreover, reliability\-aware aggregation only requires lightweight scalar statistics, including average recovery uncertainty, reconstruction error, and missing ratio\. As a result, the additional communication cost is small compared with transmitting model parameters\. Overall, as demonstrated by its high throughput relative to other baselines,FedMPOprovides a favorable trade\-off between performance improvement and computational overhead\.

FedAvg\-ZeroFedTADS2FGLFedSPAFedIIHOurs \(FedMPO\)

TrainAggx0\.37DistillDecoupx3\.8RepoAlignx11\.4HomophPropagx7\.2HierarVAEx9\.1BaseAggAGMGRoutx1\.410−210^\{\-2\}10−110^\{\-1\}10010^\{0\}10110^\{1\}Time \(hours\)\(a\)Runtime Efficiency\.BaseAgg7\.6 GBModelDistill13\.5 GBRepoAlign19\.2 GBHomophFeat15\.8 GBHierarVAE18\.4 GBRoutAGMGAggBase11\.7 GB0551010151520202525Memory \(GB\)\(b\)Memory Footprint\.
Figure 6:System efficiency comparison across graph federated learning baselines\. FedMPO achieves high throughput and low memory footprint\.

## Appendix DTheoretical Analysis in Details

### D\.1Convergence Analysis under Reliability\-aware Aggregation

We provide an appendix\-level convergence analysis forFedMPOunder modality missingness and reliability\-aware aggregation\. The purpose of this analysis is not to claim a stronger convergence rate than standard federated optimization, but to clarify how modality missingness affects stochastic variance and how reliability\-aware aggregation reduces the contribution of unreliable client updates\.

##### Setup\.

LetFk​\(θ\)F\_\{k\}\(\\theta\)be the local objective of clientkk, and let the standard data\-size weight bepk=\|𝒱k\|/∑j=1K\|𝒱j\|p\_\{k\}=\|\\mathcal\{V\}\_\{k\}\|/\\sum\_\{j=1\}^\{K\}\|\\mathcal\{V\}\_\{j\}\|\. The global objective isF​\(θ\)=∑k=1Kpk​Fk​\(θ\)F\(\\theta\)=\\sum\_\{k=1\}^\{K\}p\_\{k\}F\_\{k\}\(\\theta\)\. At communication roundtt, clientkkperformsEElocal steps and returns the local modelθkt\+1\\theta\_\{k\}^\{t\+1\}\. Equivalently, we denote its effective local update direction by

𝐠kt=θt−θkt\+1ηl​E,\\mathbf\{g\}\_\{k\}^\{t\}=\\frac\{\\theta^\{t\}\-\\theta\_\{k\}^\{t\+1\}\}\{\\eta\_\{l\}E\},\(49\)whereηl\\eta\_\{l\}is the local learning rate\. Following the main method, each client also reports lightweight reliability statistics, including average recovery uncertaintyu¯kt\\bar\{u\}\_\{k\}^\{t\}, reconstruction errore¯kt\\bar\{e\}\_\{k\}^\{t\}, and missing ratioρkt\\rho\_\{k\}^\{t\}\. The reliability score and aggregation weight are

skt=exp⁡\(−ηu​u¯kt−ηe​e¯kt−ηρ​ρkt\),ωkt=pk​skt∑j=1Kpj​sjt\+ϵ\.s\_\{k\}^\{t\}=\\exp\(\-\\eta\_\{u\}\\bar\{u\}\_\{k\}^\{t\}\-\\eta\_\{e\}\\bar\{e\}\_\{k\}^\{t\}\-\\eta\_\{\\rho\}\\rho\_\{k\}^\{t\}\),\\quad\\omega\_\{k\}^\{t\}=\\frac\{p\_\{k\}s\_\{k\}^\{t\}\}\{\\sum\_\{j=1\}^\{K\}p\_\{j\}s\_\{j\}^\{t\}\+\\epsilon\}\.\(50\)The server update can then be written as

θt\+1=∑k=1Kωkt​θkt\+1=θt−ηl​E​∑k=1Kωkt​𝐠kt\.\\theta^\{t\+1\}=\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}^\{t\}\\theta\_\{k\}^\{t\+1\}=\\theta^\{t\}\-\\eta\_\{l\}E\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}^\{t\}\\mathbf\{g\}\_\{k\}^\{t\}\.\(51\)For analysis, define the reliability\-weighted objective at roundttas

Fωt​\(θ\)=∑k=1Kωkt​Fk​\(θ\)\.F\_\{\\omega^\{t\}\}\(\\theta\)=\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}^\{t\}F\_\{k\}\(\\theta\)\.\(52\)

##### Assumption D\.1 \(Smoothness and lower boundedness\)\.

Each local objectiveFkF\_\{k\}isLL\-smooth, and the global objective is lower bounded byF⋆F^\{\\star\}\.

##### Assumption D\.2 \(Missing\-aware stochastic variance\)\.

The stochastic update direction is an unbiased estimator of the local update direction, and its variance is amplified by modality unreliability:

𝔼​\[𝐠kt\]=𝐠¯kt,𝔼​\[‖𝐠kt−𝐠¯kt‖2\]≤σk2​\(t\)E,\\mathbb\{E\}\[\\mathbf\{g\}\_\{k\}^\{t\}\]=\\bar\{\\mathbf\{g\}\}\_\{k\}^\{t\},\\quad\\mathbb\{E\}\\left\[\\\|\\mathbf\{g\}\_\{k\}^\{t\}\-\\bar\{\\mathbf\{g\}\}\_\{k\}^\{t\}\\\|^\{2\}\\right\]\\leq\\frac\{\\sigma\_\{k\}^\{2\}\(t\)\}\{E\},\(53\)where

σk2​\(t\)≤σ2​\(1\+γu​u¯kt\+γe​e¯kt\+γρ​ρkt\)\.\\sigma\_\{k\}^\{2\}\(t\)\\leq\\sigma^\{2\}\\left\(1\+\\gamma\_\{u\}\\bar\{u\}\_\{k\}^\{t\}\+\\gamma\_\{e\}\\bar\{e\}\_\{k\}^\{t\}\+\\gamma\_\{\\rho\}\\rho\_\{k\}^\{t\}\\right\)\.\(54\)This assumption captures the intuition that clients with higher missingness or less reliable recovery produce noisier local updates\.

##### Assumption D\.3 \(Bounded local drift\)\.

The bias between the expected local update direction and the local full gradient is bounded by

‖𝐠¯kt−∇Fk​\(θt\)‖2≤Cdr​ηl2​E2​δk2,\\left\\\|\\bar\{\\mathbf\{g\}\}\_\{k\}^\{t\}\-\\nabla F\_\{k\}\(\\theta^\{t\}\)\\right\\\|^\{2\}\\leq C\_\{\\mathrm\{dr\}\}\\eta\_\{l\}^\{2\}E^\{2\}\\delta\_\{k\}^\{2\},\(55\)whereδk\\delta\_\{k\}measures the client\-specific distribution drift andCdrC\_\{\\mathrm\{dr\}\}is a constant\.

##### Theorem D\.4 \(One\-round descent ofFedMPO\)\.

Under Assumptions D\.1–D\.3, ifηl​E≤1/L\\eta\_\{l\}E\\leq 1/L, the update ofFedMPOsatisfies

𝔼​\[Fωt​\(θt\+1\)\]\\displaystyle\\mathbb\{E\}\\left\[F\_\{\\omega^\{t\}\}\(\\theta^\{t\+1\}\)\\right\]≤Fωt​\(θt\)−ηl​E2​‖∇Fωt​\(θt\)‖2\\displaystyle\\leq F\_\{\\omega^\{t\}\}\(\\theta^\{t\}\)\-\\frac\{\\eta\_\{l\}E\}\{2\}\\left\\\|\\nabla F\_\{\\omega^\{t\}\}\(\\theta^\{t\}\)\\right\\\|^\{2\}\(56\)\+L​ηl2​E22​∑k=1K\(ωkt\)2​σk2​\(t\)E\+Cdr​L​ηl2​E2​∑k=1Kωkt​δk2\.\\displaystyle\\qquad\+\\frac\{L\\eta\_\{l\}^\{2\}E^\{2\}\}\{2\}\\sum\_\{k=1\}^\{K\}\(\\omega\_\{k\}^\{t\}\)^\{2\}\\frac\{\\sigma\_\{k\}^\{2\}\(t\)\}\{E\}\+C\_\{\\mathrm\{dr\}\}L\\eta\_\{l\}^\{2\}E^\{2\}\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}^\{t\}\\delta\_\{k\}^\{2\}\.Consequently, summing overTTrounds gives

1T​∑t=0T−1𝔼​\[‖∇Fωt​\(θt\)‖2\]\\displaystyle\\frac\{1\}\{T\}\\sum\_\{t=0\}^\{T\-1\}\\mathbb\{E\}\\left\[\\left\\\|\\nabla F\_\{\\omega^\{t\}\}\(\\theta^\{t\}\)\\right\\\|^\{2\}\\right\]≤𝒪​\(Fω0​\(θ0\)−F⋆ηl​E​T\)\\displaystyle\\leq\\mathcal\{O\}\\left\(\\frac\{F\_\{\\omega^\{0\}\}\(\\theta^\{0\}\)\-F^\{\\star\}\}\{\\eta\_\{l\}ET\}\\right\)\(57\)\+𝒪​\(L​ηl​ET​∑t=0T−1∑k=1K\(ωkt\)2​σk2​\(t\)\)\\displaystyle\\qquad\+\\mathcal\{O\}\\left\(\\frac\{L\\eta\_\{l\}E\}\{T\}\\sum\_\{t=0\}^\{T\-1\}\\sum\_\{k=1\}^\{K\}\(\\omega\_\{k\}^\{t\}\)^\{2\}\\sigma\_\{k\}^\{2\}\(t\)\\right\)\+𝒪​\(L​ηl​ET​∑t=0T−1∑k=1Kωkt​δk2\)\.\\displaystyle\\qquad\+\\mathcal\{O\}\\left\(\\frac\{L\\eta\_\{l\}E\}\{T\}\\sum\_\{t=0\}^\{T\-1\}\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}^\{t\}\\delta\_\{k\}^\{2\}\\right\)\.

##### Proof\.

Let

𝐆t=∑k=1Kωkt​𝐠kt\\mathbf\{G\}^\{t\}=\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}^\{t\}\\mathbf\{g\}\_\{k\}^\{t\}\(58\)be the aggregated update direction\. ByLL\-smoothness,

Fωt​\(θt\+1\)≤Fωt​\(θt\)−ηl​E​⟨∇Fωt​\(θt\),𝐆t⟩\+L​ηl2​E22​‖𝐆t‖2\.F\_\{\\omega^\{t\}\}\(\\theta^\{t\+1\}\)\\leq F\_\{\\omega^\{t\}\}\(\\theta^\{t\}\)\-\\eta\_\{l\}E\\left\\langle\\nabla F\_\{\\omega^\{t\}\}\(\\theta^\{t\}\),\\mathbf\{G\}^\{t\}\\right\\rangle\+\\frac\{L\\eta\_\{l\}^\{2\}E^\{2\}\}\{2\}\\\|\\mathbf\{G\}^\{t\}\\\|^\{2\}\.\(59\)Taking expectation and decomposing𝐆t\\mathbf\{G\}^\{t\}into its mean and stochastic noise yields

𝔼​\[‖𝐆t−𝔼​\[𝐆t\]‖2\]≤∑k=1K\(ωkt\)2​𝔼​\[‖𝐠kt−𝐠¯kt‖2\]≤∑k=1K\(ωkt\)2​σk2​\(t\)E\.\\mathbb\{E\}\\left\[\\\|\\mathbf\{G\}^\{t\}\-\\mathbb\{E\}\[\\mathbf\{G\}^\{t\}\]\\\|^\{2\}\\right\]\\leq\\sum\_\{k=1\}^\{K\}\(\\omega\_\{k\}^\{t\}\)^\{2\}\\mathbb\{E\}\\left\[\\\|\\mathbf\{g\}\_\{k\}^\{t\}\-\\bar\{\\mathbf\{g\}\}\_\{k\}^\{t\}\\\|^\{2\}\\right\]\\leq\\sum\_\{k=1\}^\{K\}\(\\omega\_\{k\}^\{t\}\)^\{2\}\\frac\{\\sigma\_\{k\}^\{2\}\(t\)\}\{E\}\.\(60\)Moreover, by Assumption D\.3,

‖𝔼​\[𝐆t\]−∇Fωt​\(θt\)‖2≤Cdr​ηl2​E2​∑k=1Kωkt​δk2\.\\left\\\|\\mathbb\{E\}\[\\mathbf\{G\}^\{t\}\]\-\\nabla F\_\{\\omega^\{t\}\}\(\\theta^\{t\}\)\\right\\\|^\{2\}\\leq C\_\{\\mathrm\{dr\}\}\\eta\_\{l\}^\{2\}E^\{2\}\\sum\_\{k=1\}^\{K\}\\omega\_\{k\}^\{t\}\\delta\_\{k\}^\{2\}\.\(61\)Substituting these two bounds into the smoothness inequality and usingηl​E≤1/L\\eta\_\{l\}E\\leq 1/Lgives the one\-round descent result\. Summing the descent inequality fromt=0t=0toT−1T\-1and rearranging terms yields the stated average stationarity bound\.

##### Remark\.

The bound contains two terms directly related to modality heterogeneity\. First, the stochastic variance term is scaled byσk2​\(t\)\\sigma\_\{k\}^\{2\}\(t\), which increases with recovery uncertainty, reconstruction error, and missing ratio\. Second, reliability\-aware aggregation multiplies this term by\(ωkt\)2\(\\omega\_\{k\}^\{t\}\)^\{2\}\. Sinceωkt\\omega\_\{k\}^\{t\}decreases exponentially withu¯kt\\bar\{u\}\_\{k\}^\{t\},e¯kt\\bar\{e\}\_\{k\}^\{t\}, andρkt\\rho\_\{k\}^\{t\}, unreliable clients contribute less to the aggregated update variance\. When all clients have similar reliability,skts\_\{k\}^\{t\}becomes approximately constant and the rule reduces to standard data\-size\-weighted FedAvg\.

### D\.2Robustness Analysis of Missing\-aware Routing

We next analyze why the missing\-aware routing and structural fallback mechanism can reduce the influence of unreliable recovered modalities\. The analysis is intentionally stated under simple additive\-noise assumptions to clarify the role of uncertainty\-aware weighting\.

##### Setup\.

Let𝐳i⋆\\mathbf\{z\}\_\{i\}^\{\\star\}denote the ideal latent representation of nodeviv\_\{i\}\. For modalitymm, the processed modality representation is modeled as

𝐟i\(m\)=𝐳i⋆\+ϵi\(m\),\\mathbf\{f\}\_\{i\}^\{\(m\)\}=\\mathbf\{z\}\_\{i\}^\{\\star\}\+\\boldsymbol\{\\epsilon\}\_\{i\}^\{\(m\)\},\(62\)whereϵi\(m\)\\boldsymbol\{\\epsilon\}\_\{i\}^\{\(m\)\}is a zero\-mean error term with variance

𝔼​\[‖ϵi\(m\)‖2\]=vi\(m\)\.\\mathbb\{E\}\\left\[\\\|\\boldsymbol\{\\epsilon\}\_\{i\}^\{\(m\)\}\\\|^\{2\}\\right\]=v\_\{i\}^\{\(m\)\}\.\(63\)For recovered modalities, the variancevi\(m\)v\_\{i\}^\{\(m\)\}increases with the recovery uncertaintyui\(m\)u\_\{i\}^\{\(m\)\}\. The structure\-only fallback representation is

Estruct​\(𝐡str,i\)=𝐳i⋆\+ϵstr,i,𝔼​\[‖ϵstr,i‖2\]=vstr,i\.E\_\{\\mathrm\{struct\}\}\(\\mathbf\{h\}\_\{\\mathrm\{str\},i\}\)=\\mathbf\{z\}\_\{i\}^\{\\star\}\+\\boldsymbol\{\\epsilon\}\_\{\\mathrm\{str\},i\},\\quad\\mathbb\{E\}\[\\\|\\boldsymbol\{\\epsilon\}\_\{\\mathrm\{str\},i\}\\\|^\{2\}\]=v\_\{\\mathrm\{str\},i\}\.\(64\)

##### Theorem D\.5 \(Error bound of reliability\-weighted fusion\)\.

Let the fused representation be

𝐫i=\(1−αfb,i\)​∑m=1Mai\(m\)​𝐟i\(m\)\+αfb,i​Estruct​\(𝐡str,i\),\\mathbf\{r\}\_\{i\}=\(1\-\\alpha\_\{\\mathrm\{fb\},i\}\)\\sum\_\{m=1\}^\{M\}a\_\{i\}^\{\(m\)\}\\mathbf\{f\}\_\{i\}^\{\(m\)\}\+\\alpha\_\{\\mathrm\{fb\},i\}E\_\{\\mathrm\{struct\}\}\(\\mathbf\{h\}\_\{\\mathrm\{str\},i\}\),\(65\)where

ai\(m\)=exp⁡\(−ui\(m\)\)∑m′=1Mexp⁡\(−ui\(m′\)\)\+ϵ\.a\_\{i\}^\{\(m\)\}=\\frac\{\\exp\(\-u\_\{i\}^\{\(m\)\}\)\}\{\\sum\_\{m^\{\\prime\}=1\}^\{M\}\\exp\(\-u\_\{i\}^\{\(m^\{\\prime\}\)\}\)\+\\epsilon\}\.\(66\)If the modality errors are zero\-mean and mutually uncorrelated, then

𝔼​\[‖𝐫i−𝐳i⋆‖2\]≤2​\(1−αfb,i\)2​∑m=1M\(ai\(m\)\)2​vi\(m\)\+2​αfb,i2​vstr,i\.\\mathbb\{E\}\\left\[\\\|\\mathbf\{r\}\_\{i\}\-\\mathbf\{z\}\_\{i\}^\{\\star\}\\\|^\{2\}\\right\]\\leq 2\(1\-\\alpha\_\{\\mathrm\{fb\},i\}\)^\{2\}\\sum\_\{m=1\}^\{M\}\(a\_\{i\}^\{\(m\)\}\)^\{2\}v\_\{i\}^\{\(m\)\}\+2\\alpha\_\{\\mathrm\{fb\},i\}^\{2\}v\_\{\\mathrm\{str\},i\}\.\(67\)Furthermore, if a recovered modality becomes highly unreliable, i\.e\.,ui\(m\)→∞u\_\{i\}^\{\(m\)\}\\rightarrow\\infty, thenai\(m\)→0a\_\{i\}^\{\(m\)\}\\rightarrow 0, and its contribution to the fusion error vanishes\.

##### Proof\.

Substituting the additive\-noise model gives

𝐫i−𝐳i⋆=\(1−αfb,i\)​∑m=1Mai\(m\)​ϵi\(m\)\+αfb,i​ϵstr,i\.\\mathbf\{r\}\_\{i\}\-\\mathbf\{z\}\_\{i\}^\{\\star\}=\(1\-\\alpha\_\{\\mathrm\{fb\},i\}\)\\sum\_\{m=1\}^\{M\}a\_\{i\}^\{\(m\)\}\\boldsymbol\{\\epsilon\}\_\{i\}^\{\(m\)\}\+\\alpha\_\{\\mathrm\{fb\},i\}\\boldsymbol\{\\epsilon\}\_\{\\mathrm\{str\},i\}\.\(68\)Using‖𝐚\+𝐛‖2≤2​‖𝐚‖2\+2​‖𝐛‖2\\\|\\mathbf\{a\}\+\\mathbf\{b\}\\\|^\{2\}\\leq 2\\\|\\mathbf\{a\}\\\|^\{2\}\+2\\\|\\mathbf\{b\}\\\|^\{2\}, we have

𝔼​\[‖𝐫i−𝐳i⋆‖2\]\\displaystyle\\mathbb\{E\}\[\\\|\\mathbf\{r\}\_\{i\}\-\\mathbf\{z\}\_\{i\}^\{\\star\}\\\|^\{2\}\]≤2​\(1−αfb,i\)2​𝔼​\[‖∑m=1Mai\(m\)​ϵi\(m\)‖2\]\+2​αfb,i2​𝔼​\[‖ϵstr,i‖2\]\.\\displaystyle\\leq 2\(1\-\\alpha\_\{\\mathrm\{fb\},i\}\)^\{2\}\\mathbb\{E\}\\left\[\\left\\\|\\sum\_\{m=1\}^\{M\}a\_\{i\}^\{\(m\)\}\\boldsymbol\{\\epsilon\}\_\{i\}^\{\(m\)\}\\right\\\|^\{2\}\\right\]\+2\\alpha\_\{\\mathrm\{fb\},i\}^\{2\}\\mathbb\{E\}\[\\\|\\boldsymbol\{\\epsilon\}\_\{\\mathrm\{str\},i\}\\\|^\{2\}\]\.\(69\)Because modality errors are assumed uncorrelated,

𝔼​\[‖∑m=1Mai\(m\)​ϵi\(m\)‖2\]=∑m=1M\(ai\(m\)\)2​vi\(m\)\.\\mathbb\{E\}\\left\[\\left\\\|\\sum\_\{m=1\}^\{M\}a\_\{i\}^\{\(m\)\}\\boldsymbol\{\\epsilon\}\_\{i\}^\{\(m\)\}\\right\\\|^\{2\}\\right\]=\\sum\_\{m=1\}^\{M\}\(a\_\{i\}^\{\(m\)\}\)^\{2\}v\_\{i\}^\{\(m\)\}\.\(70\)This proves the bound\. Finally, sinceai\(m\)a\_\{i\}^\{\(m\)\}is a softmax over−ui\(m\)\-u\_\{i\}^\{\(m\)\},ui\(m\)→∞u\_\{i\}^\{\(m\)\}\\rightarrow\\inftyimpliesai\(m\)→0a\_\{i\}^\{\(m\)\}\\rightarrow 0\. Therefore, highly uncertain recovered modalities are automatically suppressed in the fused representation\.

##### Corollary D\.6 \(Structural fallback under severe missingness\)\.

If most modalities are missing or highly uncertain, and the fallback gate satisfiesαfb,i→1\\alpha\_\{\\mathrm\{fb\},i\}\\rightarrow 1, then

lim sup𝔼​\[‖𝐫i−𝐳i⋆‖2\]≤2​vstr,i\.\\limsup\\mathbb\{E\}\\left\[\\\|\\mathbf\{r\}\_\{i\}\-\\mathbf\{z\}\_\{i\}^\{\\star\}\\\|^\{2\}\\right\]\\leq 2v\_\{\\mathrm\{str\},i\}\.\(71\)Thus, the structural fallback prevents the representation error from being dominated by unbounded recovery noise\.

##### Remark\.

This result explains the design of the uncertainty estimator and the missing\-aware router\. The uncertainty score controls the modality fusion weight, while the fallback gate allows the model to rely more on structure\-only information when recovered modalities are unreliable\. This does not imply that the structural representation is always superior; rather, it provides a bounded fallback when recovered features become highly uncertain\.

### D\.3Complexity and Communication Analysis

We analyze the computational and communication cost ofFedMPO\. LetN=\|𝒱\|N=\|\\mathcal\{V\}\|and\|ℰ\|\|\\mathcal\{E\}\|denote the number of nodes and edges on a local client,MMbe the number of modalities,ddbe the hidden dimension, andP=\|Θ\|P=\|\\Theta\|be the number of model parameters\. LetBBdenote the maximum size of the sampled neighborhood context used by the generator, and defineS=M\+BS=M\+Bas the maximum context\-bank size per node\. In practice,MMandBBare much smaller thanNN\.

##### Local computation\.

The graph encoder requires sparse message passing and feature projection, whose cost is

CGNN=𝒪​\(\|ℰ\|​d\+N​d2\)\.C\_\{\\mathrm\{GNN\}\}=\\mathcal\{O\}\(\|\\mathcal\{E\}\|d\+Nd^\{2\}\)\.\(72\)The topology\-aware generator performs attention over a bounded context bank rather than all nodes\. For each node and modality, the attention cost is𝒪​\(S​d2\+S2​d\)\\mathcal\{O\}\(Sd^\{2\}\+S^\{2\}d\); therefore,

CAGMG=𝒪​\(N​M​\(S​d2\+S2​d\)\)\.C\_\{\\mathrm\{AGMG\}\}=\\mathcal\{O\}\\left\(NM\(Sd^\{2\}\+S^\{2\}d\)\\right\)\.\(73\)The uncertainty estimator and missing\-aware experts operate on hidden representations\. With a constant number of experts, the routing and fusion cost is

Croute=𝒪​\(N​M​d2\)\.C\_\{\\mathrm\{route\}\}=\\mathcal\{O\}\(NMd^\{2\}\)\.\(74\)Thus, the total local computational complexity per round is

CFedMPO=𝒪​\(\|ℰ\|​d\+N​d2\+N​M​\(S​d2\+S2​d\)\)\.C\_\{\\mathrm\{FedMPO\}\}=\\mathcal\{O\}\\left\(\|\\mathcal\{E\}\|d\+Nd^\{2\}\+NM\(Sd^\{2\}\+S^\{2\}d\)\\right\)\.\(75\)WhenMMandSSare bounded constants, the complexity is linear in the local graph size\.

##### Communication cost\.

FedMPOdoes not transmit raw node features, raw graph structures, recovered modality features, or context\-bank representations\. Each client uploads model parameters together with three scalar reliability statistics: average recovery uncertainty, reconstruction error, and missing ratio\. Therefore, the per\-client communication payload is

ℬFedMPO=𝒪​\(P\+3\)=𝒪​\(P\)\.\\mathcal\{B\}\_\{\\mathrm\{FedMPO\}\}=\\mathcal\{O\}\(P\+3\)=\\mathcal\{O\}\(P\)\.\(76\)The additional reliability statistics introduce only constant\-size overhead compared with standard model\-parameter transmission\.

##### Comparison with global attention and proxy\-based paradigms\.

A centralized global cross\-modal attention model over all nodes and modalities has at least

𝒪​\(N2​M2​d\)\\mathcal\{O\}\(N^\{2\}M^\{2\}d\)\(77\)attention cost, because it constructs pairwise interactions amongN​MNMmodality tokens\. Proxy\-based federated graph methods may additionally transmit proxy node embeddings, resulting in communication cost

𝒪​\(P\+\|Vproxy\|​d\)\.\\mathcal\{O\}\(P\+\|V\_\{\\mathrm\{proxy\}\}\|d\)\.\(78\)In contrast,FedMPOkeeps graph reasoning, modality generation, and expert routing local, and only communicates model parameters and scalar reliability statistics, , as summarized in Table[5](https://arxiv.org/html/2605.12584#A4.T5)\.

Table 5:Complexity comparison of multimodal graph federated learning paradigms\.

## Appendix ELimitations, Future Work, and Broader Impacts

AlthoughFedMPOprovides a unified framework for missing\-modality robust learning on federated multimodal graphs, it still has several limitations\. First, the topology\-aware cross\-modal generation module and the missing\-aware MoE routing module introduce additional local computation compared with simple FedAvg\-style baselines\. These modules operate on compact hidden representations rather than raw image or text inputs, andFedMPOdoes not transmit raw graph structures, node attributes, recovered modality features, or context\-bank representations\. Nevertheless, the extra client\-side computation may still be non\-negligible for very large graphs or resource\-constrained clients\.

Second, the missing\-aware router relies on uncertainty estimation and load\-balancing regularization\. Although the routing objective mitigates expert collapse in our experiments, extremely heterogeneous clients or severe modality missingness may still lead to biased expert usage or suboptimal routing decisions\. This limitation suggests that more reliable router calibration, client\-adaptive routing, and uncertainty\-aware expert selection remain important future directions\.

Third, our current missingness simulation mainly considers client\-level and node\-level modality missingness under controlled missing rates and Dirichlet\-based non\-IID partitions\. Real\-world missingness can be more complex, including modality\-dependent missingness, long\-tailed missing patterns, missing\-not\-at\-random cases, and clients with nearly isolated modality spaces\. Such settings may require more specialized missingness modeling, stronger uncertainty calibration, and better mechanisms for distinguishing recoverable missing signals from inherently unreliable observations\.

Fourth, our experiments focus on three downstream tasks: node classification, link prediction, and modality retrieval\. While these tasks cover both graph\-centric and modality\-centric evaluation, the current framework has not been fully validated on graph\-level prediction, temporal graphs, dynamic user\-item graphs, or generation\-oriented multimodal graph tasks\. ExtendingFedMPOto these scenarios would further test its generality and reveal whether topology\-aware completion and reliability\-aware aggregation remain effective under more diverse deployment conditions\.

Future work can improveFedMPOalong several directions\. One direction is to design more efficient topology\-aware generators, such as lightweight sampling\-based generators, parameter\-efficient adapters, or subgraph\-level approximation strategies, to reduce local computation while preserving graph\-conditioned modality recovery\. Another direction is to enhance the missing\-aware router with meta\-learning, stochastic routing, or calibration\-aware objectives, so that it can adapt more robustly to client\-specific missingness patterns\. It would also be useful to study privacy\-preserving variants of reliability\-aware aggregation, for example by combining scalar reliability statistics with secure aggregation or differentially private noise when client\-level missingness information is sensitive\. Finally, evaluatingFedMPOon larger real\-world distributed multimodal graph datasets, dynamic graph environments, and application\-specific federated platforms would provide stronger evidence for practical deployment\.

Broader impacts\.FedMPOaims to improve federated multimodal graph learning under incomplete modalities, which may bring positive impacts to privacy\-sensitive and data\-fragmented applications\. For example, it can help different platforms, institutions, or organizations collaboratively train graph learning models without directly sharing raw multimodal data, which is useful for recommendation, information retrieval, social network analysis, and other graph\-based services where text, image, and relational data are naturally distributed\. By explicitly modeling missing modalities and unreliable recovered signals,FedMPOmay also improve the robustness of learning systems deployed in environments where data collection is incomplete, uneven, or constrained by privacy and resource limitations\.

However, the same capability may also introduce potential risks\. First, more robust federated multimodal graph learning could be misused in privacy\-sensitive profiling, targeted recommendation, or surveillance\-like applications if deployed without proper governance\. Second, althoughFedMPOdoes not transmit raw graph structures or raw multimodal attributes, model updates and client\-level reliability statistics may still contain indirect information about client data distributions or missingness patterns\. Third, modality completion may amplify existing biases if missing modalities are not missing at random, especially when some user groups, products, or entities systematically lack certain modalities\. In such cases, recovered features may create over\-confident but inaccurate representations and lead to unfair downstream predictions\.

To mitigate these concerns, practical deployments should combineFedMPOwith privacy\-preserving aggregation, access control, fairness auditing, and uncertainty\-aware decision rules\. When reliability statistics are sensitive, secure aggregation or differential privacy can be used to reduce leakage risks\. When recovered modalities are used for high\-stakes decisions, the system should expose uncertainty estimates, avoid treating generated modalities as ground\-truth observations, and include human or domain\-expert review when appropriate\. Overall,FedMPOis intended as a foundational learning framework rather than a decision\-making system by itself, and its societal impact depends strongly on the downstream application, deployment constraints, and governance mechanisms\.

## Appendix FHyperparameters in Details

In this section, we describe the hyperparameter configurations used in our experiments forFedMPOand comparative baselines\. To ensure fairness and reproducibility, we distinguish between task\-level hyperparameters, which are shared across models within the same downstream task, and model\-level hyperparameters, which are specific to individual architectures\. Unless otherwise specified, all methods use the same modality encoders, client partitions, missingness protocols, and evaluation splits\.

### F\.1Global and Model\-level Hyperparameters

To maintain a consistent comparative environment across federated learning scenarios, we standardize the basic training configuration\. All models are optimized with Adam on local clients\. Common hyperparameters such as the learning rate, local epochs, client fraction, hidden dimension, and gradient clipping norm are shared across methods whenever applicable\. The complete set of global hyperparameters is summarized in Table[6](https://arxiv.org/html/2605.12584#A6.T6)\.

For conventional federated graph baselines, including FedAvg\-Zero, FedGCN, and FedGraphSAGE, we use two graph layers and set the hidden dimensionality tod=256d=256\. For multimodal and missing\-modality baselines, we use the same input modality features asFedMPOand tune only model\-specific components following the same validation protocol\.

Table 6:Global shared hyperparameters\.
### F\.2FedMPOSpecific Hyperparameters

FedMPOcontains three main components: topology\-aware cross\-modal generation, missing\-aware multimodal routing, and reliability\-aware federated aggregation\. The corresponding hyperparameters are summarized in Table[7](https://arxiv.org/html/2605.12584#A6.T7)\. The reconstruction coefficientλrec\\lambda\_\{\\mathrm\{rec\}\}, alignment coefficientλalign\\lambda\_\{\\mathrm\{align\}\}, and routing coefficientλroute\\lambda\_\{\\mathrm\{route\}\}are tuned on validation splits and fixed for all test runs under the same task setting\.

Table 7:FedMPOspecific hyperparameters\.HyperparameterValueDescriptionTopology\-aware Cross\-modal GenerationZero impute fullTrue / 1Whether to use zero imputation when all modalities of a node are unavailable\.Use missing tokensFalseWhether to use learnable missing\-modality tokens\.Use impute then passTrueWhether missing features are imputed before graph context encoding\.Missing skipα\\alpha0\.35Residual mixing ratio between imputed and aggregated features\.Number of attention heads4Number of heads for cross\-modal multi\-head attention\.Attention warmup rounds30Number of warmup rounds for gradually enabling attention\-based generation\.Interpolationα\\alphabounds\[0\.55,0\.78\]\[0\.55,0\.78\]Dynamic coefficient bounds for combining generated features with structural priors\.Missing\-aware Routing and FusionUse conditional routingTrueWhether to enable conditional routing in missing\-aware MoE fusion\.Router hidden dimensionmax⁡\(4,dmodel/2\)\\max\(4,d\_\{\\mathrm\{model\}\}/2\)Hidden dimension for the router and uncertainty estimator\.Uncertainty clamp max0\.12Maximum clamping value for uncertainty\-based penalty\.Router temperatureτ\\tau1\.0Softmax temperature for router probabilities\.Uniform floor0\.0Minimum probability floor for routing experts\.Dropout0\.1Dropout rate before the final task prediction layer\.Reliability\-aware AggregationUncertainty coefficientηu\\eta\_\{u\}1\.0Weight of average recovery uncertainty in client reliability scoring\.Reconstruction coefficientηe\\eta\_\{e\}1\.0Weight of reconstruction error in client reliability scoring\.Missing\-ratio coefficientηρ\\eta\_\{\\rho\}1\.0Weight of client\-level missing ratio in client reliability scoring\.
### F\.3Baseline Specific Configurations

For baseline methods, we follow their standard architectural choices whenever possible and use the same hidden dimension, modality encoders, client splits, and missingness protocol asFedMPO\. Specific hyperparameter settings for representative baselines are summarized in Table[8](https://arxiv.org/html/2605.12584#A6.T8)\. For baselines that are not naturally designed for all downstream tasks, we keep their original learning objectives and report N/A when the original method is not directly applicable without changing its core formulation\.

Table 8:Baseline specific hyperparameters\.HyperparameterValueDescriptionFedGCNNumber of layers2Number of GCNConv layers\.Dropout0\.5Dropout rate applied during local training\.FedGraphSAGENumber of layers2Number of GraphSAGE layers\.Dropout0\.3Dropout rate for the local GraphSAGE encoder\.FedPubℓ1\\ell\_\{1\}coefficient1×10−31\\times 10^\{\-3\}L1L\_\{1\}regularization coefficient for the personalized subgraph mask\.Dropout0\.5Dropout rate applied after MaskedGCN layers\.Mask initializationNormalNormal distribution initialization for the personalized mask\.FedMVPNumber of attention heads4Number of heads for cross\-modal multi\-head attention\.Dropout0\.1Dropout rate applied during attention and local training\.FedMACNumber of attention heads4Number of heads for cross\-modal multi\-head attention\.Dropout0\.1Dropout rate applied during cross\-modal aggregation\.FedProtoInitial logit scale10\.0Initial scaling factor for prototype matching logits\.Logit scale bounds\[1\.0,20\.0\]\[1\.0,20\.0\]Clamping bounds for the logit scaling factor\.FedLAPRefinement mix ratio0\.7 / 0\.3Mix ratio between original features and Laplacian\-refined features\.S2FGL, FedSPA, and FedIIHDefault configurationOfficial settingWe follow the reported configurations and tune shared training parameters using the same validation protocol\.
### F\.4Task\-level Hyperparameters

Experiments are conducted across multiple communication rounds with local training iterations to handle data heterogeneity\. The task\-level hyperparameters are tailored to the requirements of node classification, link prediction, and modality retrieval\. The detailed configurations, including loss functions, loss weights, and evaluation metrics, are summarized in Table[9](https://arxiv.org/html/2605.12584#A6.T9)\.

For all tasks, raw image or video features are encoded into 512\-dimensional embeddings, while textual features are encoded into 768\-dimensional embeddings\. These modality features are then projected into the shared hidden space with dimensiond=256d=256\.

Table 9:Task specific hyperparameters\.HyperparameterValueDescriptionNode Classification \(NC\)Task loss functionCross EntropyDefault cross\-entropy loss used for node classification\.Evaluation metricsAccuracy / F1\-scoreMetrics used for node classification\.λrec\\lambda\_\{\\mathrm\{rec\}\}0\.05Weight for the reconstruction loss\.λalign\\lambda\_\{\\mathrm\{align\}\}0\.01Weight for the cross\-modal alignment loss\.λroute\\lambda\_\{\\mathrm\{route\}\}0\.01Weight for uncertainty calibration and expert load balancing\.Link Prediction \(LP\)Global rounds150Total number of federated communication rounds\.Task loss functionBCE \+ BPR \+ MarginCombination of binary classification and ranking objectives\.Evaluation metricsAUC / APMetrics used for link prediction\.Loss weight \(BCE\)1\.0Weight for Binary Cross\-Entropy loss\.Loss weight \(BPR\)0\.5Weight for Bayesian Personalized Ranking loss\.Loss weight \(Margin\)0\.3Weight for Margin Ranking loss\.Margin value0\.1Margin size for the Margin Ranking loss\.Hard negative pool scale4\.0Multiplier for scaling the hard negative sampling pool size\.Hard negative min pool256Minimum number of samples in the hard negative pool\.λrec\\lambda\_\{\\mathrm\{rec\}\}0\.05Weight for the reconstruction loss\.λalign\\lambda\_\{\\mathrm\{align\}\}0\.01Weight for the cross\-modal alignment loss\.λroute\\lambda\_\{\\mathrm\{route\}\}0\.01Weight for uncertainty calibration and expert load balancing\.Modality Retrieval \(MR\)Query modalityimageQuery modality for retrieval\.Gallery modalitytextTarget/gallery modality for retrieval\.Task loss functionInfoNCEContrastive loss used for cross\-modal retrieval\.Evaluation metricsRecall@5 / MRRMetrics used for modality retrieval\.InfoNCE temperatureτ\\tau0\.07Temperature scaling parameter for InfoNCE\.λrec\\lambda\_\{\\mathrm\{rec\}\}0\.5Weight for the reconstruction loss\.λalign\\lambda\_\{\\mathrm\{align\}\}0\.01Weight for the cross\-modal alignment loss\.λroute\\lambda\_\{\\mathrm\{route\}\}0\.01Weight for uncertainty calibration and expert load balancing\.Auxiliary classification weightλcls\\lambda\_\{\\mathrm\{cls\}\}0\.2Weight for the auxiliary classification objective when used\.