D-FROST: Decentralized Federated pRompt-tuning via Optimal tranSporT for Non-IID and Imbalanced Data

# D-FROST: Decentralized Federated pRompt-tuning via Optimal tranSporT for Non-IID and Imbalanced Data
Source: [https://arxiv.org/html/2609.01802](https://arxiv.org/html/2609.01802)
Corresponding author\.###### Abstract

Prompt tuning provides a parameter\-efficient way to adapt foundation models \(FMs\) by freezing the pretrained backbone and updating only a small set of learnable prompts\. This property makes prompt tuning especially suitable for decentralized federated learning \(DFL\), where exchanging full\-model updates can be prohibitively expensive\. However, prompt tuning in DFL introduces new challenges\. Prompt sets learned from heterogeneous local data may not be index\-wise aligned, making standard decentralized averaging unsuitable\. In addition, the algorithm should be theoretically guaranteed to achieve consensus and make progress toward the shared objective\. In this work, we provide the first study of prompt tuning in DFL\. We formulate decentralized prompt tuning as a Wasserstein\-based optimization problem over prompt measures, which captures the set\-valued structure of prompts\. We then proposeD\-FROST, an optimal\-transport\-based \(OT\-based\) decentralized prompt\-tuning algorithm that merges neighborhood prompts into compact representative prompt sets through transportation\-based matching\. We further analyzeD\-FROSTby bounding the Wasserstein consensus error across clients, and establishing convergence of the network\-level prompt barycenter to a neighborhood of stationarity\. Experiments under heterogeneous client data demonstrate the effectiveness ofD\-FROSTfor decentralized prompt tuning\.

1University of Florida, FL, USA

2Washington State University, WA, USA

Correspondence to: mythai@cise\.ufl\.edu

## 1Introductions

Foundation models \(FMs\) have become a dominant foundation for modern AI systems, but adapting them to downstream tasks remains costly when all model parameters must be fine\-tuned\. Prompt tuning provides a parameter\-efficient alternative by freezing the pretrained backbone and optimizing only a small set of learnable prompts\([Li and Liang 2021](https://arxiv.org/html/2609.01802#bib.bib4);[Lester et al\. 2021](https://arxiv.org/html/2609.01802#bib.bib5)\)\. This makes prompt tuning particularly attractive in federated learning \(FL\), where data are distributed across clients and communication costs are a major bottleneck\. Instead of transmitting full model updates, federated prompt tuning only exchanges lightweight prompt parameters\. As state\-of\-the\-art models now increasingly rely on on fine\-tuning foundation models like LLMs and Vision Transformers, prompt tuning in FL has emerged as a promising way to fine\-tune large pretrained models over distributed data without centralizing raw information\.\([Zhao et al\. 2023](https://arxiv.org/html/2609.01802#bib.bib6);[Che et al\. 2023](https://arxiv.org/html/2609.01802#bib.bib7);[Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\)\.

Decentralized federated learning \(DFL\) is a server\-free variant of FL where clients communicate only with graph neighbors\. For pre\-trained model adaptation in DFL, prompt tuning offers a natural way to reduce communication by exchanging only lightweight prompt parameters\. However, prompt tuning in DFL introduces two challenges\. First, a prompt\-tuning algorithm in DFL must ensure convergence, where local prompt states reach consensus and the network\-level model progresses toward the shared objective\. To the best of our knowledge, no prior work studies convergence of prompt tuning in DFL\. Existing convergence studies for full\-model DFL are not directly applicable because they rely on coordinate\-aligned parameter vectors and Euclidean averaging, while prompt\-tuning DFL operates on unordered prompt sets\([Yuan et al\. 2016](https://arxiv.org/html/2609.01802#bib.bib9);[Lian et al\. 2017](https://arxiv.org/html/2609.01802#bib.bib1);[Tang et al\. 2018](https://arxiv.org/html/2609.01802#bib.bib2);[Koloskova et al\. 2020](https://arxiv.org/html/2609.01802#bib.bib3)\)\. Second, prompts learned from heterogeneous data may not be index\-wise aligned, leading to a prompt misalignment issue where directly averaging prompts by index can merge unrelated prompt directions\. In literature, PFPT\([Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\)addresses this issue in centralized FL through probabilistic prompt aggregation\. However, extending this idea to decentralized communication is nontrivial, since each client only observes local neighborhood prompts rather than a global collection\.

Contributions\.To the best of our knowledge, this is the first work to study prompt tuning in decentralized federated learning\. The key contributions and insights of this work are summarized as follows:

- \(i\)We formulate*decentralized prompt tuning*as a Wasserstein\-based optimization problem over prompt measures\. This formulation preserves the standard DFL goal of learning a shared model state, while replacing Euclidean parameter consensus with Wasserstein prompt\-measure consensus\. As a result, it naturally captures the set\-valued structure of client prompts\.
- \(ii\)We proposeD\-FROST, an OT\-based decentralized prompt\-tuning algorithm\. Each client first updates its local prompts and then applies an OT\-basedMergefunction to summarize neighborhood prompts into a compact representative prompt set\. This merge operator avoids direct index\-wise averaging and addresses prompt misalignment by matching prompts according to their geometry in the prompt embedding space\.
- \(iii\)We provide a convergence analysis ofD\-FROST\. We first show that the local OT\-basedMergesolver becomes stable as the number of inner OT steps increases\. We then establish thatD\-FROSTcontrols the Wasserstein consensus error across clients and that the network\-level prompt barycenter converges to a neighborhood of stationarity for the shared prompt\-tuning objective\.
- \(iv\)We empirically evaluateD\-FROSTagainst various decentralized federated prompt\-tuning baselines based on existing DFL techniques\. Through extensive experiments across a combination of eight diverse vision datasets, our results consistently show that our method is effective in data imbalance and extremely heterogeneous scenarios in decentralized federated prompt\-tuning\.

## 2Related Works

### 2\.1Prompt Tuning and Federated Prompt Tuning

Prompt tuning aims to adapt pretrained models by optimizing a small set of learnable prompt parameters while keeping the backbone model frozen\. Early representative works include prefix tuning, which optimizes continuous prefixes for generation tasks\([Li and Liang 2021](https://arxiv.org/html/2609.01802#bib.bib4)\), and soft prompt tuning, which learns task\-specific continuous prompts and becomes competitive with full fine\-tuning as model scale increases\([Lester et al\. 2021](https://arxiv.org/html/2609.01802#bib.bib5)\)\.

Recent works extend prompt tuning to federated learning\. FedPrompt aggregates prompt parameters rather than full models to reduce communication and storage costs\([Zhao et al\. 2023](https://arxiv.org/html/2609.01802#bib.bib6)\), while PFPT uses probabilistic prompt aggregation to address non\-IID and imbalanced data\([Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\)\. However, these methods rely on centralized server aggregation and do not consider decentralized communication among graph neighbors\. Moreover, heterogeneous clients may learn unaligned prompt sets, making index\-wise averaging prone to combining mismatched prompt directions\.

### 2\.2Decentralized Federated Learning

Unlike centralized FL, decetralized FL removes the server and lets clients communicate only with their neighbors over a graph\. Early methods, such as distributed subgradient and decentralized gradient descent, combine local optimization with neighbor averaging\([Yuan et al\. 2016](https://arxiv.org/html/2609.01802#bib.bib9)\)\. Later decentralized SGD analyses established competitive convergence under suitable mixing conditions\([Lian et al\. 2017](https://arxiv.org/html/2609.01802#bib.bib1);[Tang et al\. 2018](https://arxiv.org/html/2609.01802#bib.bib2);[Koloskova et al\. 2020](https://arxiv.org/html/2609.01802#bib.bib3)\)\. DFedAvgM\([Sun et al\. 2022](https://arxiv.org/html/2609.01802#bib.bib12)\)adapted the FedAvg approach of multiple local SGD iterations to the decentralized setting\. DFedSAM\([Shi et al\. 2023](https://arxiv.org/html/2609.01802#bib.bib13)\)employed the sharpness\-aware minimization optimizer to reduce the in consistency of local models\. NTK\-DFL\([Thompson et al\. 2025](https://arxiv.org/html/2609.01802#bib.bib14)\)improves robustness to data heterogeneity through neural tangent kernel dynamics, but does not scale well to CNNs or Transformers\.

Most decentralized learning methods assume that client states are coordinate\-aligned model parameter vectors, soMergeis implemented by weighted averaging through a mixing matrix\. This assumption does not hold for decentralized prompt tuning, where prompts form unordered, potentially misaligned sets\.Thus, our work replaces parameter averaging with OT\-based merging for Wasserstein consensus\.

## 3Preliminaries

### 3\.1Decentralized Federated Learning \(DFL\)

DFL considers a network of clients that collaboratively optimize a learning objective without relying on a central server\. The clients are connected through a communication graphG=\(V,E\)G=\(V,E\), where each nodeu∈Vu\\in Vrepresents a client and each edge\(u,v\)∈E\(u,v\)\\in Eindicates direct communication\. Each clientuuowns a private datasetDuD\_\{u\}, and the data distributions can be heterogeneous across clients\.

Let𝒩⁡\(u\)=\{v∈V:\(u,v\)∈E\}\\mathcal\{N\}\(u\)=\\\{v\\in V:\(u,v\)\\in E\\\}denote the neighbor set of clientuu\. The communication topology is often represented by a mixing matrixW∈ℝm×mW\\in\\mathbb\{R\}^\{m\\times m\}, whereWu​v\>0W\_\{uv\}\>0only ifv=uv=uorv∈𝒩⁡\(u\)v\\in\\mathcal\{N\}\(u\)\. The graph connectivity is characterized byρ:=‖W−1m​𝟏𝟏⊤‖2,\\rho:=\\left\\\|W\-\\frac\{1\}\{m\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\\right\\\|\_\{2\},whereρ<1\\rho<1for a connected graph\. A smallerρ\\rhoindicates faster information mixing and stronger consensus among clients\.

A decentralized learning round consists of two steps:LocalUpdateandMerge\. Each client first updates its local state using private data, then exchanges states with neighboring clients and aggregates the received information\. In classical full\-model decentralized training, the local state is the parameter vector of a shared architecture\. Accordingly,LocalUpdatetypically performs one or more stochastic gradient steps, whileMergeapplies mixing\-matrix\-weighted averaging over neighboring parameters\([Lian et al\. 2017](https://arxiv.org/html/2609.01802#bib.bib1);[Tang et al\. 2018](https://arxiv.org/html/2609.01802#bib.bib2);[Koloskova et al\. 2020](https://arxiv.org/html/2609.01802#bib.bib3)\)\.

In this work, we study DFL with prompt tuning, where the pretrained backbone is frozen and only a small set of learnable prompt parameters is updated\. Therefore, the client state is a prompt set rather than the full model parameter vector, and bothLocalUpdateandMergetake different forms from full\-model decentralized training\.

### 3\.2Measure Space and Wasserstein Distance

In our setting, each client maintains a set of learnable prompts as its state\. A more natural view is to treat each prompt set as a distribution over the prompt embedding space\. Under this view, comparing two prompt sets becomes a problem of comparing two probability measures\.

Let𝒫⁡\(ℝd\)\\mathcal\{P\}\(\\mathbb\{R\}^\{d\}\)denote the space of probability measures onℝd\\mathbb\{R\}^\{d\}, and let𝒫2​\(ℝd\)\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\)denote the subset of probability measures with finite second moment:

𝒫2​\(ℝd\):=\{μ∈𝒫⁡\(ℝd\):∫ℝd‖x‖2​𝑑μ​\(x\)<∞\}\.\\displaystyle\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\):=\\left\\\{\\mu\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{d\}\):\\int\_\{\\mathbb\{R\}^\{d\}\}\\\|x\\\|^\{2\}d\\mu\(x\)<\\infty\\right\\\}\.
This space provides a geometric setting for studying distributions supported in a Euclidean embedding space, such as prompt embeddings\. Given two probability measuresμ,ν∈𝒫2​\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\), a coupling between them is a joint probability measureπ∈𝒫⁡\(ℝd×ℝd\)\\pi\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}^\{d\}\)whose marginals areμ\\muandν\\nu\. We denote the set of all such couplings by

Π\(μ,ν\):=\{π∈𝒫\(ℝd×ℝd\):\\displaystyle\\Pi\(\\mu,\\nu\):=\\\{\\pi\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}^\{d\}\):π⁡\(A×ℝd\)=μ⁡\(A\),\\displaystyle\\pi\(A\\times\\mathbb\{R\}^\{d\}\)=\\mu\(A\),\\;π\(ℝd×B\)=ν\(B\)\}\.\\displaystyle\\pi\(\\mathbb\{R\}^\{d\}\\times B\)=\\nu\(B\)\\\}\.The squared 2\-Wasserstein distance betweenμ\\muandν\\nuis defined as

W22​\(μ,ν\):=infπ∈Π⁡\(μ,ν\)∫ℝd×ℝd‖x−y‖2​𝑑π​\(x,y\)\.\\displaystyle W\_\{2\}^\{2\}\(\\mu,\\nu\):=\\inf\_\{\\pi\\in\\Pi\(\\mu,\\nu\)\}\\int\_\{\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}^\{d\}\}\\\|x\-y\\\|^\{2\}d\\pi\(x,y\)\.Intuitively,W22​\(μ,ν\)W\_\{2\}^\{2\}\(\\mu,\\nu\)measures the minimum transportation cost required to move the mass ofμ\\muto matchν\\nuunder the squared Euclidean cost\. For empirical measures,

μ=∑a=1Nra​δxa,ν=∑i=1Mci​δyi,\\displaystyle\\mu=\\sum\_\{a=1\}^\{N\}r\_\{a\}\\delta\_\{x\_\{a\}\},\\qquad\\nu=\\sum\_\{i=1\}^\{M\}c\_\{i\}\\delta\_\{y\_\{i\}\},wherera,ci≥0r\_\{a\},c\_\{i\}\\geq 0,∑a=1Nra=1\\sum\_\{a=1\}^\{N\}r\_\{a\}=1, and∑i=1Mci=1\\sum\_\{i=1\}^\{M\}c\_\{i\}=1, the coupling can be represented by a transport matrixP∈ℝ\+N×MP\\in\\mathbb\{R\}\_\{\+\}^\{N\\times M\}\. The feasible set is

Π\(r,c\):=\{P∈ℝ\+N×M:P𝟏M=r,P⊤𝟏N=c\}\.\\displaystyle\\Pi\(r,c\):=\\left\\\{P\\in\\mathbb\{R\}\_\{\+\}^\{N\\times M\}:P\\mathbf\{1\}\_\{M\}=r,\\;P^\{\\top\}\\mathbf\{1\}\_\{N\}=c\\right\\\}\.In this discrete case, the squared 2\-Wasserstein distance is:

W22\(μ,ν\)=minP∈Π⁡\(r,c\)∑a=1N∑i=1MPa​i∥xa−yi∥2\.\\displaystyle W\_\{2\}^\{2\}\(\\mu,\\nu\)=\\min\_\{P\\in\\Pi\(r,c\)\}\\sum\_\{a=1\}^\{N\}\\sum\_\{i=1\}^\{M\}P\_\{ai\}\\\|x\_\{a\}\-y\_\{i\}\\\|^\{2\}\.
Therefore, Wasserstein distance compares two empirical distributions by optimizing over all possible matchings between their support points, rather than assuming a fixed ordering\. This property is particularly useful for prompt sets, where the elements are not naturally ordered and can be misaligned across clients\.

## 4Decentralized Wasserstein Prompt Tuning

In this section, we first introduce the decentralized prompt tuning setup\. Then, we define the global objective as learning a shared prompt measure in the Wasserstein space\. Finally, we propose an OT\-based algorithm to approximately solve the decentralized prompt tuning problem\.

### 4\.1Setup

LetG=\(V,E\)G=\(V,E\)be an undirected communication graph with\|V\|=M\|V\|=Mclients\. Each clientu∈Vu\\in Vowns a private local datasetDuD\_\{u\}\. The clients collaboratively adapt a pretrained backbone modelFFwhile keeping the backbone parameters fixed\. Therefore, each client only maintains and updates a local prompt set\.

At communication roundtt, clientuumaintains

ωu\(t\)=\{ωu​1\(t\),…,ωu​n\(t\)\},ωu​i\(t\)∈ℝd,\\displaystyle\\omega\_\{u\}^\{\(t\)\}=\\big\\\{\\omega\_\{u1\}^\{\(t\)\},\\dots,\\omega\_\{un\}^\{\(t\)\}\\big\\\},\\qquad\\omega\_\{ui\}^\{\(t\)\}\\in\\mathbb\{R\}^\{d\},wherennis the number of prompts andddis the prompt dimension\. We view this set as an empirical probability measure

μu\(t\):=1n​∑i=1nδωu​i\(t\)\.\\displaystyle\\mu\_\{u\}^\{\(t\)\}:=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\\omega\_\{ui\}^\{\(t\)\}\}\.This representation treats the prompt state as an unordered set of support points in the prompt embedding space\.

Let𝒩⁡\(u\)=\{v∈V:\(u,v\)∈E\}\\mathcal\{N\}\(u\)=\\\{v\\in V:\(u,v\)\\in E\\\}denote the neighbor set of clientuu\. Since each client updates prompts using its own data distribution, neighboring prompt sets may become misaligned\. Wasserstein distance provides a natural way to compare such prompt measures because it compares sets through optimal transport rather than assuming index\-wise correspondence\.

### 4\.2Decentralized Wasserstein Prompt Tuning Objective

As in classical DFL, our goal is to learn one shared model state\. In our setting, this shared state is a prompt measure rather than a full model parameter vector\. Let𝒫2​\(ℝd\)\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\)denote the space of probability measures with finite second moment over the prompt embedding space\. We define the global prompt\-tuning objective as

minμ∈𝒫2​\(ℝd\)ℱ⁡\(μ\):=1M​∑u=1Mℒu​\(μ\),\\min\_\{\\mu\\in\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\)\}\\quad\\mathcal\{F\}\(\\mu\):=\\frac\{1\}\{M\}\\sum\_\{u=1\}^\{M\}\\mathcal\{L\}\_\{u\}\(\\mu\),\(1\)whereℒu​\(μ\)\\mathcal\{L\}\_\{u\}\(\\mu\)is the local prompt\-tuning loss of clientuuevaluated at prompt measureμ\\mu\.

In practice, there is no central server that directly maintains the shared prompt measureμ\\mu\. Instead, each clientuumaintains a local empirical prompt measureμu\(t\)=1n​∑i=1nδωu​i\(t\)\.\\mu\_\{u\}^\{\(t\)\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\\omega\_\{ui\}^\{\(t\)\}\}\.These local prompt measures can be viewed as decentralized approximations of the shared prompt measure in \([1](https://arxiv.org/html/2609.01802#S4.E1)\)\.

To describe the collective state of the network, we use a network\-level prompt barycenter, denoted byμavg\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\. Formally, it can be viewed as a Wasserstein barycenter of the local prompt measures:

μavg\(t\)∈arg⁡minμ∈𝒫2​\(ℝd\)​1M​∑u=1MW22​\(μ,μu\(t\)\)\.\\displaystyle\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\in\\arg\\min\_\{\\mu\\in\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\)\}\\frac\{1\}\{M\}\\sum\_\{u=1\}^\{M\}W\_\{2\}^\{2\}\\big\(\\mu,\\mu\_\{u\}^\{\(t\)\}\\big\)\.\(2\)This barycenter provides a useful analytical object for describing the collective behavior of the decentralized system\.

Under this view, the objective of decentralized prompt tuning follows the same two\-fold principle as classical DFL\. First, the decentralized trajectory should make progress toward minimizing the shared global objective in \([1](https://arxiv.org/html/2609.01802#S4.E1)\)\. Second, the local client states should achieve network consensus\. In our setting, consensus means that the local prompt measures become close in Wasserstein distance\. We therefore measure the disagreement among local prompt measures by the Wasserstein consensus error

ε\(t\):=1M​∑u=1MW22​\(μu\(t\),μavg\(t\)\)\.\\varepsilon^\{\(t\)\}:=\\frac\{1\}\{M\}\\sum\_\{u=1\}^\{M\}W\_\{2\}^\{2\}\\big\(\\mu\_\{u\}^\{\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\big\)\.\(3\)A smaller value ofε\(t\)\\varepsilon^\{\(t\)\}indicates that the decentralized local prompt measures are more tightly concentrated around the network\-level prompt barycenter, and hence that the clients have better prompt\-level consensus\.

This problem definition is natural for decentralized prompt tuning for two reasons\. First, it preserves the standard DFL objective structure\. In details, the target is still a single shared model state that minimizes the average client loss, rather than a separate personalized objective for each client\. Second, it replaces Euclidean parameter consensus with Wasserstein prompt\-measure consensus, which is more appropriate for set\-valued prompt states\. Since prompt indices across clients may not be aligned, Wasserstein distance compares prompt sets through optimal matching instead of forcing coordinate\-wise or index\-wise correspondence\.

### 4\.3OT\-Based Decentralized Algorithm

We now propose an OT\-based decentralized algorithm, namedD\-FROST, for solving the decentralized prompt\-tuning problem\. The algorithm aims to make progress on the shared objective in \([1](https://arxiv.org/html/2609.01802#S4.E1)\) while maintaining Wasserstein consensus among the local prompt measures, as measured by \([3](https://arxiv.org/html/2609.01802#S4.E3)\)\. The details are summarized in Algorithm[1](https://arxiv.org/html/2609.01802#alg1)\. In general, each communication round ofD\-FROSTconsists of two steps: a local prompt update and a neighbor merging step\. The local update makes progress on the local loss, while the neighbor merging step promotes consensus among local prompt measures\.

We first describe the local prompt update\.

ω~u\(t\)←LocalUpdate​\(F,Du,ωu\(t−1\)\)\.\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}\\leftarrow\\texttt\{LocalUpdate\}\\big\(F,D\_\{u\},\\omega\_\{u\}^\{\(t\-1\)\}\\big\)\.\(4\)This step produces the locally adapted prompt setω~u\(t\)\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}before neighbor communication\.

Next, clientuuexchangesω~u\(t\)\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}with its neighbors and forms the neighborhood prompt collection

Ωu\(t\):=ω~u\(t\)​⨄⨄v∈𝒩⁡\(u\)ω~v\(t\),Nu\(t\):=\|Ωu\(t\)\|\.\\Omega\_\{u\}^\{\(t\)\}:=\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}\\biguplus\\biguplus\_\{v\\in\\mathcal\{N\}\(u\)\}\\tilde\{\\omega\}\_\{v\}^\{\(t\)\},\\qquad N\_\{u\}^\{\(t\)\}:=\\big\|\\Omega\_\{u\}^\{\(t\)\}\\big\|\.\(5\)The collectionΩu\(t\)\\Omega\_\{u\}^\{\(t\)\}contains prompt information from clientuuand its neighbors\. This exchange supports decentralized consensus by incorporating neighborhood knowledge\. However, retaining all received prompts can increase storage and computation costs, soΩu\(t\)\\Omega\_\{u\}^\{\(t\)\}should be summarized by a compact prompt set that preserves the essential information\.

InD\-FROST, this compact set is obtained through an OT\-based prompt merging problem\. The merge step approximates the collectionΩu\(t\)\\Omega\_\{u\}^\{\(t\)\}by a compact empirical measure under a transportation\-based geometry\. Thus, the OT\-basedMergefunction summarizes received prompts while supporting Wasserstein consensus among local prompt measures\.

We now describe the OT\-based prompt merging problem\. GivenΩu\(t\)=\{za\}a=1Nu\(t\)\\Omega\_\{u\}^\{\(t\)\}=\\big\\\{z\_\{a\}\\big\\\}\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}, the goal is to construct a representative prompt set

Φ=\{ϕi\}i=1n,ϕi∈ℝd\.\\Phi=\\\{\\phi\_\{i\}\\\}\_\{i=1\}^\{n\},\\qquad\\phi\_\{i\}\\in\\mathbb\{R\}^\{d\}\.
We introduce a transport planP∈ℝ\+Nu\(t\)×nP\\in\\mathbb\{R\}\_\{\+\}^\{N\_\{u\}^\{\(t\)\}\\times n\}, wherePa​iP\_\{ai\}measures how much the neighborhood promptzaz\_\{a\}contributes to the representative promptϕi\\phi\_\{i\}\. The matching cost is

Ca​i​\(Φ\):=12​σ2​‖za−ϕi‖2,\\displaystyle C\_\{ai\}\(\\Phi\):=\\frac\{1\}\{2\\sigma^\{2\}\}\\left\\\|z\_\{a\}\-\\phi\_\{i\}\\right\\\|^\{2\},\(6\)whereσ2\\sigma^\{2\}controls the spatial scale of the cost\.

We assign uniform source mass to the neighborhood prompts:

q:=1Nu\(t\)​𝟏Nu\(t\),\\displaystyle q:=\\frac\{1\}\{N\_\{u\}^\{\(t\)\}\}\\mathbf\{1\}\_\{N\_\{u\}^\{\(t\)\}\},and impose the source marginal constraintP​𝟏n=qP\\mathbf\{1\}\_\{n\}=q\. This ensures that every neighborhood prompt participates in the merge\. We do not impose a fixed target marginal over the representative prompts, so different merged prompts can receive different amounts of mass based on the geometry ofΩu\(t\)\\Omega\_\{u\}^\{\(t\)\}\.

Clientuucomputes the merged prompt set by solving

minP≥0,Φ𝒥u​\(P,Φ\):=\\displaystyle\\min\_\{P\\geq 0,\\ \\Phi\}\\quad\\mathcal\{J\}\_\{u\}\(P,\\Phi\):=⟨P,C⁡\(Φ\)⟩⏟spatial matching\+ε​∑a=1Nu\(t\)∑i=1nPa​i​\(log⁡Pa​i−1\)⏟entropy regularization\\displaystyle\\underbrace\{\\langle P,C\(\\Phi\)\\rangle\}\_\{\\text\{spatial matching\}\}\+\\varepsilon\\underbrace\{\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}\\sum\_\{i=1\}^\{n\}P\_\{ai\}\(\\log P\_\{ai\}\-1\)\}\_\{\\text\{entropy regularization\}\}\+λ2​σ2​∑i=1n‖ϕi‖2⏟L2regularizations\.t\.P​𝟏n=q\.\\displaystyle\+\\underbrace\{\\frac\{\\lambda\}\{2\\sigma^\{2\}\}\\sum\_\{i=1\}^\{n\}\\\|\\phi\_\{i\}\\\|^\{2\}\}\_\{\\text\{$L\_\{2\}$ regularization\}\}\\quad\\text\{s\.t\.\}\\quad P\\mathbf\{1\}\_\{n\}=q\.\(7\)
Here,ε\>0\\varepsilon\>0controls the softness of the assignments, andλ\>0\\lambda\>0controls the regularization strength\. The spatial term matches neighborhood prompts to representative prompts, the entropy term produces soft transport assignments, and theL2L\_\{2\}term stabilizes the representatives\.

The objective in \([7](https://arxiv.org/html/2609.01802#S4.E7)\) is block\-wise tractable\. FixingΦ\\Phigives a closed\-form update forPP, and fixingPPgives a closed\-form update forΦ\\Phi\. Thus, each client solves the OT merge problem by alternating between transport and barycenter updates\.

##### Transport step\.

Given the current representative promptsΦ\(s−1\)\\Phi^\{\(s\-1\)\}, clientuuupdates the transport plan\. Since the source marginal constraint fixes only the row sums ofPP, the rows ofPPdecouple, yielding the closed\-form update

Pa​i\(s\)=1Nu\(t\)​exp\(−Ca​i\(Φ\(s−1\)\)/ε\)∑j=1nexp\(−Ca​j\(Φ\(s−1\)\)/ε\)\.P\_\{ai\}^\{\(s\)\}=\\frac\{1\}\{N\_\{u\}^\{\(t\)\}\}\\frac\{\\exp\\big\(\-C\_\{ai\}\(\\Phi^\{\(s\-1\)\}\)/\\varepsilon\\big\)\}\{\\sum\_\{j=1\}^\{n\}\\exp\\big\(\-C\_\{aj\}\(\\Phi^\{\(s\-1\)\}\)/\\varepsilon\\big\)\}\.\(8\)This update softly assigns each neighborhood prompt to the representative prompts\. Smallerε\\varepsilongives sharper assignments, while largerε\\varepsilongives smoother mixing\.

##### Barycenter step\.

Given the updated transport planP\(s\)P^\{\(s\)\}, clientuuupdates each representative prompt by setting the gradient of \([7](https://arxiv.org/html/2609.01802#S4.E7)\) with respect toϕi\\phi\_\{i\}to zero, yielding

ϕi\(s\)=∑a=1Nu\(t\)Pa​i\(s\)​za∑a=1Nu\(t\)Pa​i\(s\)\+λ\.\\phi\_\{i\}^\{\(s\)\}=\\frac\{\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}P\_\{ai\}^\{\(s\)\}z\_\{a\}\}\{\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}P\_\{ai\}^\{\(s\)\}\+\\lambda\}\.\(9\)Thus, each representative prompt is a regularized weighted average of the neighborhood prompts assigned to it\. This step moves the representatives toward dominant prompt directions without relying on index\-wise averaging\.

AfterSSalternating steps, clientuusets

ωu\(t\)←MergeOT​\(Ωu\(t\)\):=Φ\(S\)\.\\displaystyle\\omega\_\{u\}^\{\(t\)\}\\leftarrow\\texttt\{Merge\}\_\{\\mathrm\{OT\}\}\\big\(\\Omega\_\{u\}^\{\(t\)\}\\big\):=\\Phi^\{\(S\)\}\.This completes the OT\-basedMergestep\. The overall procedure alternates between local adaptation, which improves the client\-specific prompt loss, and OT\-based neighbor merging, which promotes consensus among prompt measures\.

Algorithm 1Decentralized Wasserstein Prompt Tuning0:Graph

G=\(V,E\)G=\(V,E\), backbone

FF, local datasets

\{Du\}u∈V\\\{D\_\{u\}\\\}\_\{u\\in V\}, rounds

TT, OT steps

SS, number of prompts

nn, entropy weight

ε\\varepsilon, regularization weight

λ\\lambda, scale

σ2\\sigma^\{2\}
1:Initialize local prompt sets

ωu\(0\)\\omega\_\{u\}^\{\(0\)\}for all

u∈Vu\\in V
2:for

t=1t=1to

TTdo

3:for all

u∈Vu\\in Vin paralleldo

4:Local update:

ω~u\(t\)←LocalUpdate​\(F,Du,ωu\(t−1\)\)\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}\\leftarrow\\texttt\{LocalUpdate\}\\big\(F,D\_\{u\},\\omega\_\{u\}^\{\(t\-1\)\}\\big\)
5:Neighbor exchange:send

ω~u\(t\)\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}to neighbors and receive

\{ω~v\(t\):v∈𝒩⁡\(u\)\}\\\{\\tilde\{\\omega\}\_\{v\}^\{\(t\)\}:v\\in\\mathcal\{N\}\(u\)\\\}
6:Neighborhood collection:

Ωu\(t\)=ω~u\(t\)​⨄⨄v∈𝒩⁡\(u\)ω~v\(t\)\\Omega\_\{u\}^\{\(t\)\}=\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}\\biguplus\\biguplus\_\{v\\in\\mathcal\{N\}\(u\)\}\\tilde\{\\omega\}\_\{v\}^\{\(t\)\}
7:Write

Ωu\(t\)=\{ωa\}a=1Nu\(t\)\\Omega\_\{u\}^\{\(t\)\}=\\\{\\omega\_\{a\}\\\}\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}
8:Initialize representative prompts

Φ\(0\)=\{ϕi\(0\)\}i=1n\\Phi^\{\(0\)\}=\\\{\\phi\_\{i\}^\{\(0\)\}\\\}\_\{i=1\}^\{n\}from

Ωu\(t\)\\Omega\_\{u\}^\{\(t\)\}or from

ωu\(t−1\)\\omega\_\{u\}^\{\(t\-1\)\}
9:for

s=1s=1to

SSdo

10:Construct cost matrix

C⁡\(Φ\(s−1\)\)C\(\\Phi^\{\(s\-1\)\}\)using \([6](https://arxiv.org/html/2609.01802#S4.E6)\)

11:Update transport plan

P\(s\)P^\{\(s\)\}using \([8](https://arxiv.org/html/2609.01802#S4.E8)\)

12:Update representative prompts

Φ\(s\)\\Phi^\{\(s\)\}using \([9](https://arxiv.org/html/2609.01802#S4.E9)\)

13:endfor

14:Merge output:set

ωu\(t\)←Φ\(S\)\\omega\_\{u\}^\{\(t\)\}\\leftarrow\\Phi^\{\(S\)\}
15:endfor

16:endfor

17:returnFinal local prompt sets

\{ωu\(T\)\}u∈V\\\{\\omega\_\{u\}^\{\(T\)\}\\\}\_\{u\\in V\}

## 5Theoretical Analysis ofD\-FROST

In this section, we analyze the theoretical properties ofD\-FROST\. First, we show that the local OT\-basedMergestep becomes stable as the number of inner OT steps increases\. Second, we analyze the global behavior ofD\-FROSTand show that the local prompt measures remain close to a network\-level barycenter, which then converges to a neighborhood of Wasserstein stationarity for the shared prompt\-tuning objective in \([1](https://arxiv.org/html/2609.01802#S4.E1)\)\. All proofs are deferred to appendix[B](https://arxiv.org/html/2609.01802#A2)\.

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/non_iid_curves_fivedataset.png)Figure 1:Test accuracy of all methods onFiveDataset\(50 clients\) under three non\-IID settings\.### 5\.1Stability of the Local OT\-Based Merge

We first analyze the local OT\-basedMergeoperator\. Recall that after clientuuforms the neighborhood prompt collectionΩu\(t\)\\Omega\_\{u\}^\{\(t\)\}, it solves the local OT problem in \([7](https://arxiv.org/html/2609.01802#S4.E7)\) by alternating between the transport update and the barycenter update\. Since the merged prompt set is used as the client state for the next communication round, the inner OT solver should produce a stable representative prompt set\.

###### Theorem 1\(Stability of the Alternating OT Solver\)\.

LetΔ\(s\)\\Delta^\{\(s\)\}be defined as above, and letμ:=min⁡\(ε,λσ2\)\>0\.\\mu:=\\min\\left\(\\varepsilon,\\frac\{\\lambda\}\{\\sigma^\{2\}\}\\right\)\>0\.AfterSSalternating OT steps, the minimum iterate difference is bounded by an𝒪⁡\(1/S\)\\mathcal\{O\}\(1/\\sqrt\{S\}\)rate:

min1≤s≤S⁡Δ\(s\)≤4​\(𝒥u​\(P\(0\),Φ\(0\)\)−𝒥∗\)μ​S\.\\displaystyle\\min\_\{1\\leq s\\leq S\}\\Delta^\{\(s\)\}\\leq\\sqrt\{\\frac\{4\\big\(\\mathcal\{J\}\_\{u\}\(P^\{\(0\)\},\\Phi^\{\(0\)\}\)\-\\mathcal\{J\}^\{\*\}\\big\)\}\{\\mu S\}\}\.

LetΔ\(s\):=‖P\(s\)−P\(s−1\)‖F\+‖Φ\(s\)−Φ\(s−1\)‖F\\Delta^\{\(s\)\}:=\\\|P^\{\(s\)\}\-P^\{\(s\-1\)\}\\\|\_\{F\}\+\\\|\\Phi^\{\(s\)\}\-\\Phi^\{\(s\-1\)\}\\\|\_\{F\}denote the discrepancy between two consecutive inner iterations\. Theorem[1](https://arxiv.org/html/2609.01802#Thmtheorem1a)provides a quantitative stability guarantee for the local OT\-basedMergestep\. It shows that among the firstSSalternating iterations, there exists an iterate whose change from the previous iterateΔ\(s\)\\Delta^\{\(s\)\}is bounded by𝒪⁡\(1/S\)\\mathcal\{O\}\(1/\\sqrt\{S\}\)\. Thus, increasing the number of inner OT steps makes the local merge output progressively more stable\. The bound also shows that stability depends on the initial objective gap𝒥u​\(P\(0\),Φ\(0\)\)−𝒥∗\\mathcal\{J\}\_\{u\}\(P^\{\(0\)\},\\Phi^\{\(0\)\}\)\-\\mathcal\{J\}^\{\*\}and the effective strong\-convexity parameterμ=min⁡\(ε,λ/σ2\)\\mu=\\min\(\\varepsilon,\\lambda/\\sigma^\{2\}\)\. This local control is crucial because the approximation error of the OT\-based merge directly affects network\-level consensus and global convergence\.

### 5\.2Global Wasserstein Consensus and Stationarity

We now analyze the global behavior ofD\-FROST\. The analysis establishes two main results\. First,D\-FROSTcontrols the consensus errorε\(t\)\\varepsilon^\{\(t\)\}defined in \([3](https://arxiv.org/html/2609.01802#S4.E3)\), meaning that the local prompt measures remain close to a network\-level barycenterμavg\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\. Second, this barycenter converges to a neighborhood of stationarity for the shared prompt\-tuning objective in \([1](https://arxiv.org/html/2609.01802#S4.E1)\)\.

For each clientuu, we represent its prompt set at roundttas the empirical measureμu\(t\)\\mu\_\{u\}^\{\(t\)\}\. We haveμavg\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}as the Wasserstein barycenter of the local prompt measures, as defined in \([2](https://arxiv.org/html/2609.01802#S4.E2)\)\. With the learning rateη\\eta, we model the local update as:

μu\+\(t\)=\(I−η​∇W2ℱu\)\#​μu\(t−1\)\.\\displaystyle\\mu\_\{u\}^\{\+\(t\)\}=\\big\(I\-\\eta\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\_\{u\}\\big\)\_\{\\\#\}\\mu\_\{u\}^\{\(t\-1\)\}\.The subsequent OT\-basedMergestep approximates the ideal neighborhood barycenter using a compact representative prompt measure\. We capture the approximation error of this finite\-prompt OT merge by the following bounded\-error:

𝔼⁡\[‖v~\(t\)−v\(t\)‖L2​\(μavg\(t−1\)\)2\]≤δn2​\(S\),\\displaystyle\\mathbb\{E\}\\left\[\\left\\\|\\tilde\{v\}^\{\(t\)\}\-v^\{\(t\)\}\\right\\\|\_\{L^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\}^\{2\}\\right\]\\leq\\delta\_\{n\}^\{2\}\(S\),wherev~\(t\)\\tilde\{v\}^\{\(t\)\}denotes the ideal displacement induced by the uncompressed barycenter,v\(t\)v^\{\(t\)\}denotes the actual displacement induced by the OT\-merged barycenter, andδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)captures the approximation error caused by the finite prompt budgetnnand the finite number of OT stepsSS\.

We use the following standard assumptions\.

###### Assumption 1\(Wasserstein Smoothness\)\.

Each local loss functionalℱu​\(μ\)\\mathcal\{F\}\_\{u\}\(\\mu\)isLL\-smooth over the 2\-Wasserstein space\. Consequently, the global functionalℱ⁡\(μ\)=1M​∑u=1Mℱu​\(μ\)\\mathcal\{F\}\(\\mu\)=\\frac\{1\}\{M\}\\sum\_\{u=1\}^\{M\}\\mathcal\{F\}\_\{u\}\(\\mu\)is alsoLL\-smooth and bounded below byℱ∗\>−∞\\mathcal\{F\}^\{\*\}\>\-\\infty\.

###### Assumption 2\(Bounded Gradient Variance\)\.

The local Wasserstein gradients have uniformly bounded second moment:

𝔼⁡\[‖∇W2ℱu​\(μ\)‖L22\]≤G2\.\\mathbb\{E\}\\left\[\\\|\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\_\{u\}\(\\mu\)\\\|\_\{L^\{2\}\}^\{2\}\\right\]\\leq G^\{2\}\.\(10\)

###### Assumption 3\(Bounded Prompt Support\)\.

There exists a constantD\>0D\>0such that for all clients, prompts, and communication rounds,

‖ωu​i\(t\)‖≤D\.\\\|\\omega\_\{ui\}^\{\(t\)\}\\\|\\leq D\.\(11\)

###### Assumption 4\(Graph Mixing\)\.

The communication matrixW∈ℝM×MW\\in\\mathbb\{R\}^\{M\\times M\}is symmetric and doubly stochastic\. Its mixing factor satisfies

ρ:=‖W−1M​𝟏𝟏⊤‖2<1\.\\rho:=\\left\\\|W\-\\frac\{1\}\{M\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\\right\\\|\_\{2\}<1\.\(12\)

The next theorem states that the local prompt measures remain close to the network\-level barycenter\.

###### Theorem 2\(Wasserstein Consensus Bound\)\.

Suppose Assumptions[1](https://arxiv.org/html/2609.01802#Thmassumption1a)–[4](https://arxiv.org/html/2609.01802#Thmassumption4a)\. the expected network consensus errorε\(t\)\\varepsilon^\{\(t\)\}converges asymptotically to a stationary bounded neighborhood\. Specifically, for anyt→∞t\\to\\infty:

ε\(t\)≤β1−α=𝒪⁡\(δn2​\(S\)1−𝒟​ρ2\+η2​G21−𝒟​ρ2\),\\varepsilon^\{\(t\)\}\\leq\\frac\{\\beta\}\{1\-\\alpha\}=\\mathcal\{O\}\\left\(\\frac\{\\delta\_\{n\}^\{2\}\(S\)\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\+\\frac\{\\eta^\{2\}G^\{2\}\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\\right\),\(13\)whereα=6​𝒟​ρ2​\(1\+η2​L2\)<1\\alpha=6\\mathcal\{D\}\\rho^\{2\}\(1\+\\eta^\{2\}L^\{2\}\)<1, andβ=6​δn2​\(S\)\+6​𝒟​ρ2​η2​G2\\beta=6\\delta\_\{n\}^\{2\}\(S\)\+6\\mathcal\{D\}\\rho^\{2\}\\eta^\{2\}G^\{2\}, and𝒟\>0\\mathcal\{D\}\>0is a metric translation constant\.

Theorem[2](https://arxiv.org/html/2609.01802#Thmtheorem2a)shows thatD\-FROSTcontrols prompt disagreement in Wasserstein space\. The consensus neighborhood has two sources\. The first term,δn2​\(S\)\\delta\_\{n\}^\{2\}\(S\), is the approximation error introduced by representing the exchanged neighborhood prompts with a compact OT\-merged prompt set\. The second term,η2​G2\\eta^\{2\}G^\{2\}, is caused by heterogeneous local updates\. The denominator depends on the graph mixing factorρ\\rho\. Specifically, better\-connected graphs have smallerρ\\rhoand therefore tighter consensus neighborhoods\.

Finally, we state the stationarity result\.

###### Theorem 3\(Convergence to a Wasserstein Stationarity Neighborhood\)\.

Suppose Assumptions[1](https://arxiv.org/html/2609.01802#Thmassumption1a)–[4](https://arxiv.org/html/2609.01802#Thmassumption4a)\. Letη≤1/L\\eta\\leq 1/Landα<1\\alpha<1\. Then afterTTcommunication rounds,D\-FROSTsatisfies

1T​∑t=1T𝔼⁡\[‖∇W2ℱ​\(μavg\(t−1\)\)‖L2​\(μavg\(t−1\)\)2\]≤2​\(ℱ⁡\(μavg\(0\)\)−ℱ∗\)η​T\\displaystyle\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{E\}\\left\[\\left\\\|\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\\left\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\\right\)\\right\\\|\_\{L^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\}^\{2\}\\right\]\\leq\\frac\{2\\left\(\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(0\)\}\)\-\\mathcal\{F\}^\{\*\}\\right\)\}\{\\eta T\}\+𝒪⁡\(η2​L2​G21−𝒟​ρ2\)\+𝒪⁡\(L2​δn2​\(S\)1−𝒟​ρ2\+δn2​\(S\)η2\)\.\\displaystyle\+\\mathcal\{O\}\\left\(\\frac\{\\eta^\{2\}L^\{2\}G^\{2\}\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\\right\)\+\\mathcal\{O\}\\left\(\\frac\{L^\{2\}\\delta\_\{n\}^\{2\}\(S\)\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\+\\frac\{\\delta\_\{n\}^\{2\}\(S\)\}\{\\eta^\{2\}\}\\right\)\.

Theorem[3](https://arxiv.org/html/2609.01802#Thmtheorem3a)shows thatD\-FROSTconverges to a neighborhood of Wasserstein stationarity\. The first term is the standard optimization term and vanishes asTTincreases\. The second term reflects the effect of local gradient variance and graph\-induced consensus error\. The third term captures the approximation error of the OT\-basedMergestep\. Thus, more accurate local merging, achieved by increasing the prompt budgetnnor using more OT stepsSS, leads to a tighter stationarity neighborhood\.

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/consensus_error_total_comparison.png)Figure 2:Total Wasserstein consensus error across all clients on FiveDataset under Dirichletα=0\.1\\alpha=0\.1\.![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/topology_performance_5topos_4methods_pool10.png)Figure 3:Topology\-aware performance across communication topologies on FiveDataset under Dirichletα=0\.1\\alpha=0\.1\.![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/dfrost_vs_others_3panels.png)Figure 4:Robustness of D\-FROST under varying network conditions on FiveDataset \(α=0\.1\\alpha=0\.1\)\.Left:increasing link drop probabilitypp\.Center:increasing the number of clients\.Right:decreasing the number of clients sampled per round\.

## 6Experiments

### 6\.1Experiment Setup

#### Dataset and Data Partition

We induce data heterogeneity by pooling classification datasets from different visual domains\.FourDataset\([Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\)combines MNIST\-M, Fashion\-MNIST, CINIC\-10, and MMAFEDB andFiveDataset\([Wang et al\. 2022](https://arxiv.org/html/2609.01802#bib.bib15)\)combines CIFAR\-10, MNIST, Fashion\-MNIST, SVHN, and notMNIST\. Their combination introduces substantial distributional heterogeneity\. We use two partition regimes\. In theDirichlet split, each client draws a class\-proportion vector𝐩u∼Dirichlet⁡\(α⋅𝟏s\)\\mathbf\{p\}\_\{u\}\\sim\\mathrm\{Dirichlet\}\(\\alpha\\cdot\\mathbf\{1\}\_\{s\}\)over itsssdomain classes; smallerα\\alphameans stronger skew, and we reportα=0\.1\\alpha=0\.1andα=0\.5\\alpha=0\.5\. In theextreme non\-IID split, each client is dominated by a single class \(99%99\\%of its samples\), with the remaining1%1\\%pooled and spread across the others\([Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\)\. Both schemes are applied within each domain and the resulting client subsets merged\.

#### Baselines

We compare our method against three popular DFL baselines: D\-PSGD\-PT\([Lian et al\. 2017](https://arxiv.org/html/2609.01802#bib.bib1)\), DFedAvgM\-PT\([Sun et al\. 2022](https://arxiv.org/html/2609.01802#bib.bib12)\), and DFedSAM\-PT\([Shi et al\. 2023](https://arxiv.org/html/2609.01802#bib.bib13)\)\. For all methods, the pretrained backbone is frozen, while only the prompts and classification head are locally updated and communicated\.

#### Implementation Details

The number of clients is set to4040forFourDatasetand5050forFiveDataset, with all clients participating in every communication round\. We use the Adam optimizer with batch size1616and an initial learning rate of10−410^\{\-4\}\. All methods run55local epochs per communication round, except D\-PSGD, which runs only11\. We use ViT\-B/32 as the frozen backbone, with 10 trainable prompt tokens of dimensiond=768d=768prepended to the patch\-embedding sequence before it enters the frozen transformer blocks\.

#### Communication Topologies

As our standard network topology, we employ a time\-varyingκ\\kappa\-regular graph whereκ=5\\kappa=5forFiveDatasetandκ=4\\kappa=4forFourDataset\. Specifically, during each communication roundtt, we sample a new random graphG\(t\)=\(V,E\(t\)\)G^\{\(t\)\}=\(V,E^\{\(t\)\}\)with a uniform degree ofκ\\kappa, meaning every clientuuconnects to exactly\|𝒩⁡\(u\)\|=κ\|\\mathcal\{N\}\(u\)\|=\\kappaneighbors\. To evaluate the algorithmic robustness across different network structures, we also benchmark performance on Ring, Grid, Erdős\-Rényi, Regular Graph, and Fully connected topologies\. More details are given in Appx\.[C](https://arxiv.org/html/2609.01802#A3)\.

### 6\.2Experimental Results

##### Performance and Convergence\.

We report the accuracy of D\-FROST and the DFL baselines on FiveDataset under the two Dirichlet splits \(α=0\.5\\alpha=0\.5,α=0\.1\\alpha=0\.1\) and the extreme non\-IID \(imbalance\) split in Figure[1](https://arxiv.org/html/2609.01802#S5.F1)\. Across all settings, D\-FROST consistently achieves the best accuracy and fastest convergence\. Specifically, it reaches81\.95%81\.95\\%,79\.36%79\.36\\%, and70\.46%70\.46\\%, yielding improvements of6\.676\.67,8\.488\.48, and18\.5718\.57points\. The performance gap widens sharply as heterogeneity increases, particularly under the extreme non\-IID split where index\-wise prompt averaging often merges misaligned prompt directions\. Detailed and additional results on FourDataset are given in Appx\.[D](https://arxiv.org/html/2609.01802#A4)\.

Beyond final accuracy, Figure[1](https://arxiv.org/html/2609.01802#S5.F1)shows that D\-FROST separates from the baselines within a few rounds and reaches target accuracy faster\. Theorem[3](https://arxiv.org/html/2609.01802#Thmtheorem3a)supports this behavior by establishing an \(𝒪⁡\(1/T\)\\mathcal\{O\}\(1/T\)\) convergence rate to a neighborhood determined by the merge error \(δn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)\) and gradient variance\. Meanwhile, index\-wise averaging baselines ignores prompt misalignment, leading to larger consensus error under severe data skew and slower convergence \(see Figure[2](https://arxiv.org/html/2609.01802#S5.F2)\)\. We further study factors that impact the consensus error in Appx\.[E](https://arxiv.org/html/2609.01802#A5)\.

##### Topology\-aware performance\.

Figure[3](https://arxiv.org/html/2609.01802#S5.F3)evaluates accuracy across five topologies, ordered by decreasing sparsity \(mixing factorρ\\rho\): Ring\>\>Grid\>\>Erdős\-Rényi \(𝔼⁡\[κ\]=5\\mathbb\{E\}\[\\kappa\]=5\)≈\\approxRegular \(κ=5\\kappa=5\)\>\>Fully\-connected\. D\-FROST consistently outperforms all baselines across every structure\. As connectivity increases \(ρ\\rhodecreases\), performance improves for all methods\. For D\-FROST, this aligns with Theorem[2](https://arxiv.org/html/2609.01802#Thmtheorem2a): a smallerρ\\rhoyields a tighter Wasserstein consensus neighborhood, enabling faster network agreement\. Despite baseline improvements in denser networks, the substantial performance gap in favor of D\-FROST persists throughout\.

##### Robustness to network conditions\.

We further stress\-test D\-FROST along three axes that degrade decentralized training: unreliable links, network scale, and partial participation\. Figure[4](https://arxiv.org/html/2609.01802#S5.F4)reports test accuracy as each factor is made harsher\. Under link dropout \(left\), where each edge fails with probabilitypp, D\-FROST declines only mildly from79\.36%79\.36\\%atp=0\.0p=0\.0to77\.29%77\.29\\%atp=0\.4p=0\.4\. As the number of clients in the network grows to 100 \(center\), D\-FROST leads over the strongest baseline by11%11\\%\. Under partial participation \(right\): with only1010of the clients active per round, D\-FROST retains64\.88%64\.88\\%, a margin of nearly1212points over DFedAvgM\.

##### Further Experiments\.

We analyze the cost of D\-FROST in Appx\.[F](https://arxiv.org/html/2609.01802#A6)and provide ablation studies that further analyze the impact of different deployment settings in Appx\.[G](https://arxiv.org/html/2609.01802#A7)\.

## 7Conclusion

We presented the first study of prompt tuning in decentralized federated learning\. We formulate it as a Wasserstein optimization over prompt measures and proposeD\-FROST, an optimal\-transport\-based algorithm that merges neighborhood prompts into a compact representative set without index\-wise averaging\. Theoretically, we proved that the local OT solver is stable, that the Wasserstein consensus error across clients stays within a bounded neighborhood, and that the network\-level prompt barycenter converges to a neighborhood of stationarity for the shared objective\. Across eight datasets, D\-FROST consistently outperforms decentralized baselines, with the largest gains under extreme non\-IID data\.

## References

- Bulatov \(2011\)Y\. BulatovNotMNIST dataset\.Note:http://yaroslavvb\.blogspot\.com/2011/09/notmnist\-dataset\.htmlCited by:[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px2.p1.1)\.
- Cheet al\.\(2023\)T\. Che, J\. Liu, Y\. Zhou, J\. Ren, jiwen zhou, V\. S\. Sheng, H\. Dai, and D\. DouFederated learning of large language models with parameter\-efficient prompt tuning and adaptive optimization\.InThe 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://openreview.net/forum?id=WuuxbObghx)Cited by:[§1](https://arxiv.org/html/2609.01802#S1.p1.1)\.
- Cuturi and Doucet \(2014\)M\. Cuturi and A\. DoucetFast computation of wasserstein barycenters\.InProceedings of the 31st International Conference on Machine Learning,E\. P\. Xing and T\. Jebara \(Eds\.\),Proceedings of Machine Learning Research, Vol\.32,Bejing, China,pp\. 685–693\.External Links:[Link](https://proceedings.mlr.press/v32/cuturi14.html)Cited by:[§B\.2](https://arxiv.org/html/2609.01802#A2.SS2.SSS0.Px4.p2.2)\.
- Cuturi \(2013\)M\. CuturiSinkhorn distances: lightspeed computation of optimal transport\.Advances in neural information processing systems26\.Cited by:[§C\.2](https://arxiv.org/html/2609.01802#A3.SS2.SSS0.Px2.p1.1),[§G\.4](https://arxiv.org/html/2609.01802#A7.SS4.p1.1)\.
- Darlowet al\.\(2018\)L\. N\. Darlow, E\. J\. Crowley, A\. Antoniou, and A\. J\. StorkeyCinic\-10 is not imagenet or cifar\-10\.arXiv preprint arXiv:1810\.03505\.Cited by:[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px1.p1.1)\.
- d’Ascoliet al\.\(2021\)S\. d’Ascoli, H\. Touvron, M\. L\. Leavitt, A\. S\. Morcos, G\. Biroli, and L\. SagunConvit: improving vision transformers with soft convolutional inductive biases\.InInternational conference on machine learning,pp\. 2286–2296\.Cited by:[§G\.2](https://arxiv.org/html/2609.01802#A7.SS2.p1.1)\.
- Foretet al\.\(2020\)P\. Foret, A\. Kleiner, H\. Mobahi, and B\. NeyshaburSharpness\-aware minimization for efficiently improving generalization\.arXiv preprint arXiv:2010\.01412\.Cited by:[§A\.2](https://arxiv.org/html/2609.01802#A1.SS2.SSS0.Px3.p1.1)\.
- Ganinet al\.\(2016\)Y\. Ganin, E\. Ustinova, H\. Ajakan, P\. Germain, H\. Larochelle, F\. Laviolette, M\. March, and V\. LempitskyDomain\-adversarial training of neural networks\.Journal of machine learning research17\(59\),pp\. 1–35\.Cited by:[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px1.p1.1)\.
- Koloskovaet al\.\(2020\)A\. Koloskova, N\. Loizou, S\. Boreiri, M\. Jaggi, and S\. StichA unified theory of decentralized SGD with changing topology and local updates\.InProceedings of the 37th International Conference on Machine Learning,H\. D\. III and A\. Singh \(Eds\.\),Proceedings of Machine Learning Research, Vol\.119,pp\. 5381–5393\.External Links:[Link](https://proceedings.mlr.press/v119/koloskova20a.html)Cited by:[§B\.2](https://arxiv.org/html/2609.01802#A2.SS2.SSS0.Px4.p5.1),[§1](https://arxiv.org/html/2609.01802#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.01802#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.01802#S3.SS1.p3.1)\.
- Krizhevskyet al\.\(2009\)A\. Krizhevsky G\. Hintonet al\.Learning multiple layers of features from tiny images\.Cited by:[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px2.p1.1)\.
- LeCun \(1998\)Y\. LeCunThe mnist database of handwritten digits\.http://yann\. lecun\. com/exdb/mnist/\.Cited by:[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px2.p1.1)\.
- Lesteret al\.\(2021\)B\. Lester, R\. Al\-Rfou, and N\. ConstantThe power of scale for parameter\-efficient prompt tuning\.InConference on Empirical Methods in Natural Language Processing,External Links:[Link](https://api.semanticscholar.org/CorpusID:233296808)Cited by:[§1](https://arxiv.org/html/2609.01802#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.01802#S2.SS1.p1.1)\.
- Li and Liang \(2021\)X\. L\. Li and P\. LiangPrefix\-tuning: optimizing continuous prompts for generation\.Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 4582–4597\.External Links:[Link](https://api.semanticscholar.org/CorpusID:230433941)Cited by:[§1](https://arxiv.org/html/2609.01802#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.01802#S2.SS1.p1.1)\.
- Lianet al\.\(2017\)X\. Lian, C\. Zhang, H\. Zhang, C\. Hsieh, W\. Zhang, and J\. LiuCan decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent\.InProceedings of the 31st International Conference on Neural Information Processing Systems,NIPS’17,Red Hook, NY, USA,pp\. 5336–5346\.External Links:ISBN 9781510860964Cited by:[§A\.2](https://arxiv.org/html/2609.01802#A1.SS2.SSS0.Px1),[§B\.2](https://arxiv.org/html/2609.01802#A2.SS2.SSS0.Px4.p5.1),[§1](https://arxiv.org/html/2609.01802#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.01802#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.01802#S3.SS1.p3.1),[§6\.1](https://arxiv.org/html/2609.01802#S6.SS1.SSSx2.p1.1)\.
- Nedic and Ozdaglar \(2009\)A\. Nedic and A\. OzdaglarDistributed subgradient methods for multi\-agent optimization\.IEEE Transactions on Automatic Control54\(1\),pp\. 48–61\.External Links:[Document](https://dx.doi.org/10.1109/TAC.2008.2009515)Cited by:[§B\.2](https://arxiv.org/html/2609.01802#A2.SS2.SSS0.Px4.p5.1)\.
- Netzeret al\.\(2011\)Y\. Netzer, T\. Wang, A\. Coates, A\. Bissacco, B\. Wu, A\. Y\. Ng,et al\.Reading digits in natural images with unsupervised feature learning\.InNIPS workshop on deep learning and unsupervised feature learning,Vol\.2011,pp\. 4\.Cited by:[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px2.p1.1)\.
- Peyré and Cuturi \(2019\)G\. Peyré and M\. CuturiComputational optimal transport: with applications to data science\.Now Foundations and Trends\.Cited by:[§C\.2](https://arxiv.org/html/2609.01802#A3.SS2.SSS0.Px2.p1.1),[§G\.4](https://arxiv.org/html/2609.01802#A7.SS4.p1.1)\.
- Schmitzer \(2019\)B\. SchmitzerStabilized sparse scaling algorithms for entropy regularized transport problems\.SIAM Journal on Scientific Computing41\(3\),pp\. A1443–A1481\.Cited by:[§C\.2](https://arxiv.org/html/2609.01802#A3.SS2.SSS0.Px2.p1.1),[§G\.4](https://arxiv.org/html/2609.01802#A7.SS4.p1.1)\.
- Shiet al\.\(2023\)Y\. Shi, L\. Shen, K\. Wei, Y\. Sun, B\. Yuan, X\. Wang, and D\. TaoImproving the model consistency of decentralized federated learning\.InInternational Conference on Machine Learning,pp\. 31269–31291\.Cited by:[§A\.2](https://arxiv.org/html/2609.01802#A1.SS2.SSS0.Px3),[§2\.2](https://arxiv.org/html/2609.01802#S2.SS2.p1.1),[§6\.1](https://arxiv.org/html/2609.01802#S6.SS1.SSSx2.p1.1)\.
- Sunet al\.\(2022\)T\. Sun, D\. Li, and B\. WangDecentralized federated averaging\.IEEE Transactions on Pattern Analysis and Machine Intelligence45\(4\),pp\. 4289–4301\.Cited by:[§A\.2](https://arxiv.org/html/2609.01802#A1.SS2.SSS0.Px2),[§2\.2](https://arxiv.org/html/2609.01802#S2.SS2.p1.1),[§6\.1](https://arxiv.org/html/2609.01802#S6.SS1.SSSx2.p1.1)\.
- Tanget al\.\(2018\)H\. Tang, X\. Lian, M\. Yan, C\. Zhang, and J\. LiuD2D^\{2\}: Decentralized training over decentralized data\.InProceedings of the 35th International Conference on Machine Learning,J\. Dy and A\. Krause \(Eds\.\),Proceedings of Machine Learning Research, Vol\.80,pp\. 4848–4856\.External Links:[Link](https://proceedings.mlr.press/v80/tang18a.html)Cited by:[§1](https://arxiv.org/html/2609.01802#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.01802#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.01802#S3.SS1.p3.1)\.
- Thompsonet al\.\(2025\)G\. Thompson, K\. Yue, C\. Wong, and H\. DaiNTK\-dfl: enhancing decentralized federated learning in heterogeneous settings via neural tangent kernel\.InInternational Conference on Machine Learning,pp\. 59470–59491\.Cited by:[§A\.2](https://arxiv.org/html/2609.01802#A1.SS2.SSS0.Px4),[§2\.2](https://arxiv.org/html/2609.01802#S2.SS2.p1.1)\.
- Touvronet al\.\(2021\)H\. Touvron, M\. Cord, M\. Douze, F\. Massa, A\. Sablayrolles, and H\. JégouTraining data\-efficient image transformers & distillation through attention\.InInternational conference on machine learning,pp\. 10347–10357\.Cited by:[§G\.2](https://arxiv.org/html/2609.01802#A7.SS2.p1.1)\.
- Wanget al\.\(2022\)Z\. Wang, Z\. Zhang, C\. Lee, H\. Zhang, R\. Sun, X\. Ren, G\. Su, V\. Perot, J\. Dy, and T\. PfisterLearning to prompt for continual learning\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 139–149\.Cited by:[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px2),[Figure 12](https://arxiv.org/html/2609.01802#A7.F12),[§G\.3](https://arxiv.org/html/2609.01802#A7.SS3.p1.1),[§6\.1](https://arxiv.org/html/2609.01802#S6.SS1.SSSx1.p1.1)\.
- Weed and Bach \(2017\)J\. Weed and F\. R\. BachSharp asymptotic and finite\-sample rates of convergence of empirical measures in wasserstein distance\.Bernoulli\.External Links:[Link](https://api.semanticscholar.org/CorpusID:51919254)Cited by:[§B\.2](https://arxiv.org/html/2609.01802#A2.SS2.SSS0.Px4.p2.2)\.
- Wenget al\.\(2024\)P\. Weng, M\. Hoang, L\. M\. Nguyen, M\. T\. Thai, T\. Weng, and T\. N\. HoangProbabilistic federated prompt\-tuning with non\-IID and imbalanced data\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=nw6ANsC66G)Cited by:[§A\.2](https://arxiv.org/html/2609.01802#A1.SS2.SSS0.Px5),[§C\.1](https://arxiv.org/html/2609.01802#A3.SS1.SSS0.Px1.1),[§G\.3](https://arxiv.org/html/2609.01802#A7.SS3.p1.1),[§1](https://arxiv.org/html/2609.01802#S1.p1.1),[§1](https://arxiv.org/html/2609.01802#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.01802#S2.SS1.p2.1),[§6\.1](https://arxiv.org/html/2609.01802#S6.SS1.SSSx1.p1.1)\.
- Yuanet al\.\(2016\)K\. Yuan, Q\. Ling, and W\. YinOn the convergence of decentralized gradient descent\.SIAM Journal on Optimization26\(3\),pp\. 1835–1854\.External Links:[Document](https://dx.doi.org/10.1137/130943170),[Link](https://doi.org/10.1137/130943170),https://doi\.org/10\.1137/130943170Cited by:[§1](https://arxiv.org/html/2609.01802#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.01802#S2.SS2.p1.1)\.
- Zhaoet al\.\(2023\)H\. Zhao, W\. Du, F\. Li, P\. Li, and G\. LiuFedPrompt: communication\-efficient and privacy\-preserving prompt tuning in federated learning\.InICASSP 2023 \- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),Vol\.,pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP49357.2023.10095356)Cited by:[§1](https://arxiv.org/html/2609.01802#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.01802#S2.SS1.p2.1)\.

## Appendix AAdditional Preliminaries

### A\.1Prompt Tuning with a Frozen Backbone

LetFθF\_\{\\theta\}denote a pretrained backbone model parameterized byθ\\theta\. In prompt tuning, the backbone parameters are kept frozen, and only a small set of prompt parameters is optimized for downstream adaptation\. This parameter\-efficient design substantially reduces the number of trainable parameters and makes prompt tuning attractive for decentralized learning, where communication and local computation are constrained\.

For each clientuu, we denote its local dataset byDuD\_\{u\}and its prompt set at communication roundttby

ωu\(t\)=\{ωu​1\(t\),…,ωu​n\(t\)\},ωu​i\(t\)∈ℝd,\\displaystyle\\omega\_\{u\}^\{\(t\)\}=\\big\\\{\\omega\_\{u1\}^\{\(t\)\},\\dots,\\omega\_\{un\}^\{\(t\)\}\\big\\\},\\qquad\\omega\_\{ui\}^\{\(t\)\}\\in\\mathbb\{R\}^\{d\},wherennis the number of prompts maintained by each client andddis the prompt dimension\. The prompts are inserted into the frozen backbone to condition the model prediction\. Given an input\-label pair\(x,y\)∈Du\(x,y\)\\in D\_\{u\}, the client\-side loss can be written as

ℓ⁡\(Fθ​\(x,ωu\(t\)\),y\),\\displaystyle\\ell\\big\(F\_\{\\theta\}\(x;\\omega\_\{u\}^\{\(t\)\}\),y\\big\),where the notationFθ​\(x,ωu\(t\)\)F\_\{\\theta\}\(x;\\omega\_\{u\}^\{\(t\)\}\)emphasizes that the prediction depends on the prompt set while the backbone parametersθ\\thetaremain fixed\.

At each communication round, clientuuperforms a local prompt update by optimizing only its prompt parameters:

ωu\+\(t\)←LocalUpdate​\(Fθ,Du,ωu\(t−1\)\)\.\\displaystyle\\omega\_\{u\}^\{\+\(t\)\}\\leftarrow\\texttt\{LocalUpdate\}\\big\(F\_\{\\theta\},D\_\{u\},\\omega\_\{u\}^\{\(t\-1\)\}\\big\)\.Equivalently, this local update approximately minimizes the empirical prompt\-tuning objective

minωu⁡ℒu​\(ωu\):=1\|Du\|​∑\(x,y\)∈Duℓ⁡\(Fθ​\(x,ωu\),y\),θ​fixed\.\\displaystyle\\min\_\{\\omega\_\{u\}\}\\;\\mathcal\{L\}\_\{u\}\(\\omega\_\{u\}\):=\\frac\{1\}\{\|D\_\{u\}\|\}\\sum\_\{\(x,y\)\\in D\_\{u\}\}\\ell\\big\(F\_\{\\theta\}\(x;\\omega\_\{u\}\),y\\big\),\\qquad\\theta\\ \\text\{fixed\}\.
On the other hand, theMergefunction in decentralized prompt tuning is fundamentally different from the standard merge operation in full\-model decentralized training\. In full\-model training, model parameters are naturally coordinate\-aligned across clients, so neighboring models can often be merged by coordinate\-wise weighted averaging\. However, prompt sets may not have a direct index\-wise correspondence across clients\. Since each client optimizes prompts using its own local data distribution, the learned prompts may drift toward client\-specific directions\. Therefore, a naive prompt\-levelMergefunction that averages prompts by index may combine semantically misaligned prompts and produce less representative local states\. This motivates a specializedMergefunction that aligns and summarizes exchanged neighborhood prompts before producing the updated local prompt set\.

### A\.2Decentralized Federated Learning Baselines

We compare D\-FROST against three representative DFL methods, each adapted to the prompt\-tuning protocol\. Under this protocol, the pretrained ViT\-B/32 backbone is kept frozen throughout training, and only the learnable prompt tokens and the task\-specific classification head are updated locally and exchanged during each communication round\. We refer to the adapted versions asD\-PSGD\-PT,DFedAvgM\-PT, andDFedSAM\-PT, respectively\.

In all three baselines, theMergestep is implemented as coordinate\-wise weighted averaging through the doubly stochastic mixing matrixWW\. Concretely, after local updates, each clientuureceives the updated prompt parametersω~v\(t\)\\tilde\{\\omega\}\_\{v\}^\{\(t\)\}from each neighborv∈𝒩⁡\(u\)v\\in\\mathcal\{N\}\(u\)and sets

ωu\(t\)←∑v∈𝒩⁡\(u\)∪\{u\}Wu​v​ω~v\(t\)\.\\omega\_\{u\}^\{\(t\)\}\\leftarrow\\sum\_\{v\\in\\mathcal\{N\}\(u\)\\cup\\\{u\\\}\}W\_\{uv\}\\,\\tilde\{\\omega\}\_\{v\}^\{\(t\)\}\.\(14\)This index\-wise average assumes that prompt tokens at the same position index across clients represent semantically comparable directions, an assumption that fails under heterogeneous data, where locally optimized prompts are free to drift into client\-specific subspaces\.

##### D\-PSGD\([Lian et al\. 2017](https://arxiv.org/html/2609.01802#bib.bib1)\)\.

D\-PSGD is a classic decentralized parallel SGD method that uses one\-step SGD to train local models in each communication round\. Each client performs one local mini\-batch update with plain SGD, followed by a neighbor\-averagingMergestep throughWW\. Following the standard protocol, the training epoch in D\-PSGD is set to11, whereas it is set to55for all other baselines and D\-FROST, so D\-PSGD performs strictly less local computation per round\.

##### DFedAvgM\([Sun et al\. 2022](https://arxiv.org/html/2609.01802#bib.bib12)\)\.

DFedAvgM extends FedAvg to the decentralized setting by allowing clients to perform multiple local SGD iterations with momentum before communicating, reducing the number of communication rounds needed for convergence compared to D\-PSGD\. After local training, theMergestep applies index\-wise weighted averaging throughWW\.

##### DFedSAM\([Shi et al\. 2023](https://arxiv.org/html/2609.01802#bib.bib13)\)\.

DFedSAM improves upon DFedAvgM by replacing local SGD with Sharpness\-Aware Minimization \(SAM\)\([Foret et al\. 2020](https://arxiv.org/html/2609.01802#bib.bib23)\), which seeks parameters that lie in flat loss neighborhoods, thereby reducing the inconsistency that arises among local models trained on heterogeneous data\. Each local update consists of a two\-step SAM procedure: a perturbation step that moves parameters toward the neighborhood of highest loss, followed by a gradient step evaluated at the perturbed point\. As with the other baselines, theMergestep performs index\-wise weighted averaging throughWW\.

Beyond these baselines, we also discuss several other related FL protocols\.

##### NTK\-DFL\([Thompson et al\. 2025](https://arxiv.org/html/2609.01802#bib.bib14)\)\.

NTK\-DFL replaces stochastic gradient updates with Neural Tangent Kernel\-based weight evolution to improve convergence under heterogeneous data, achieving4\.6×4\.6\\timesfewer communication rounds than DFedAvgM on Fashion\-MNIST\. However, the NTK linearization is currently restricted to small two\-layer MLPs and the authors explicitly acknowledge that scaling to CNNs or Transformers remains an open problem\.

##### PFPT\([Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\)\.

PFPT is a centralized federated prompt\-tuning method that addresses prompt misalignment under non\-IID and imbalanced data by treating prompt aggregation as a distributed set modeling problem, with a dynamically sized global pool maintained by a central server\. The pool expands to accommodate new, semantically distinct prompts contributed by clients with heterogeneous local distributions, while prompts that are sufficiently similar are merged to suppress redundancy and keep the prompt pool compact\. In the centralized FL setting, the prompt pool slightly expands in early rounds then stabilizes as the server merges semantically similar prompts from all clients\.

However, extending this idea to decentralized communication is nontrivial, since each client only observes local neighbor\- hood prompts rather than a global collection\. When this method is naively extended to DFL by replacing the central server with neighborhood\-level aggregation, local prompt pools grow exponentially across rounds, inflating local model size rapidly \(Figure[5](https://arxiv.org/html/2609.01802#A1.F5)\)\. As each client merges only with its neighbors’ prompts, and under high data heterogeneity these neighborhood pools share little semantic overlap, so their union rarely contracts\. The problem compounds because each client starts every round from its own diverged local pool rather than a shared global one as in PFPT\.

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/decentralized_pfpt_explosion.png)Figure 5:Average number of local prompts per client in decentralized PFPT \(κ=4\\kappa=4, 20 clients, Fashion\-MNIST\)\. The number of prompts increases exponentially across rounds\.

## Appendix BTheoretical Analysis ofD\-FROST\(More Details\)

In this section, we analyze the theoretical properties ofD\-FROST\. The analysis is organized around the two key components of the algorithm\. We first study the local OT\-based prompt merging problem and show that its alternating solver is stable\. We then analyze the global behavior ofD\-FROSTin the decentralized network\. By viewing each client prompt set as an empirical measure in the Wasserstein space, we establish that the network\-level prompt barycenter converges to a neighborhood of Wasserstein stationarity for the shared prompt\-tuning objective in \([1](https://arxiv.org/html/2609.01802#S4.E1)\)\.

### B\.1Stability of the Local OT\-Based Merge

We first analyze the local OT\-basedMergeoperator used inD\-FROST\. Recall that after clientuuforms the neighborhood prompt collectionΩu\(t\)\\Omega\_\{u\}^\{\(t\)\}, it solves the local OT problem in \([7](https://arxiv.org/html/2609.01802#S4.E7)\) by alternating between the transport update and the barycenter update\. Since the merged prompt set is used as the client state for the next communication round, the inner OT solver should produce a stable representative prompt set\. The main result of this subsection shows that the alternating OT solver becomes stable as the number of inner steps increases\.

In Lemma[1](https://arxiv.org/html/2609.01802#Thmlemma1), we first establish a lower bound on the local OT objective, which is used to control the total objective decrease across the inner iterations\.

###### Lemma 1\(Lower Bound of the Local OT Objective\)\.

For any feasible transport planPPsatisfyingP​𝟏n=qP\\mathbf\{1\}\_\{n\}=qand any representative prompt setΦ\\Phi, the local OT merge objective𝒥u​\(P,Φ\)\\mathcal\{J\}\_\{u\}\(P,\\Phi\)in \([7](https://arxiv.org/html/2609.01802#S4.E7)\) is bounded from below:

𝒥u​\(P,Φ\)≥𝒥∗:=−ε⁡\(log⁡\(n​Nu\(t\)\)\+1\)\.\\displaystyle\\mathcal\{J\}\_\{u\}\(P,\\Phi\)\\geq\\mathcal\{J\}^\{\*\}:=\-\\varepsilon\\big\(\\log\(nN\_u^\{\(t\)\}\)\+1\\big\)\.

###### Proof\.

The spatial matching term is non\-negative because

Ca​i​\(Φ\)=12​σ2​‖za−ϕi‖2≥0,\\displaystyle C\_\{ai\}\(\\Phi\)=\\frac\{1\}\{2\\sigma^\{2\}\}\\\|z\_\{a\}\-\\phi\_\{i\}\\\|^\{2\}\\geq 0,and therefore⟨P,C⁡\(Φ\)⟩≥0\\langle P,C\(\\Phi\)\\rangle\\geq 0\. TheL2L\_\{2\}regularization term is also non\-negative:

λ2​σ2​∑i=1n‖ϕi‖2≥0\.\\displaystyle\\frac\{\\lambda\}\{2\\sigma^\{2\}\}\\sum\_\{i=1\}^\{n\}\\\|\\phi\_\{i\}\\\|^\{2\}\\geq 0\.
It remains to lower\-bound the entropy term\. SinceP​𝟏n=qP\\mathbf\{1\}\_\{n\}=qandq=1Nu\(t\)​𝟏Nu\(t\)q=\\frac\{1\}\{N\_\{u\}^\{\(t\)\}\}\\mathbf\{1\}\_\{N\_\{u\}^\{\(t\)\}\}, we have

∑a=1Nu\(t\)∑i=1nPa​i=1\.\\displaystyle\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}\\sum\_\{i=1\}^\{n\}P\_\{ai\}=1\.The quantity∑a,iPa​i​log⁡Pa​i\\sum\_\{a,i\}P\_\{ai\}\\log P\_\{ai\}is minimized over the probability simplex when the mass is uniformly distributed, i\.e\.,Pa​i=1n​Nu\(t\),∀a,i\.P\_\{ai\}=\\frac\{1\}\{nN\_\{u\}^\{\(t\)\}\},\\qquad\\forall a,i\.\. Thus, we have:

ε​∑a=1Nu\(t\)∑i=1nPa​i​\(log⁡Pa​i−1\)\\displaystyle\\varepsilon\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}\\sum\_\{i=1\}^\{n\}P\_\{ai\}\(\\log P\_\{ai\}\-1\)≥ε​∑a=1Nu\(t\)∑i=1n1n​Nu\(t\)​log⁡1n​Nu\(t\)\\displaystyle\\geq\\varepsilon\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}\\sum\_\{i=1\}^\{n\}\\frac\{1\}\{nN\_\{u\}^\{\(t\)\}\}\\log\\frac\{1\}\{nN\_\{u\}^\{\(t\)\}\}−ε∑a=1Nu\(t\)∑i=1nPa​i\\displaystyle\-\\varepsilon\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}\\sum\_\{i=1\}^\{n\}P\_\{ai\}=−ε​log⁡\(n​Nu\(t\)\)−ε\\displaystyle=\-\\varepsilon\\log\(nN\_u^\{\(t\)\}\)\-\\varepsilon=−ε⁡\(log⁡\(n​Nu\(t\)\)\+1\)\.\\displaystyle=\-\\varepsilon\\big\(\\log\(nN\_u^\{\(t\)\}\)\+1\\big\)\.Combining this entropy lower bound with the non\-negativity of the spatial matching andL2L\_\{2\}terms gives the claimed lower bound\. ∎

We now use Lemma[1](https://arxiv.org/html/2609.01802#Thmlemma1)to show that the alternating solver stabilizes\. Let

Δ\(s\):=‖P\(s\)−P\(s−1\)‖F\+‖Φ\(s\)−Φ\(s−1\)‖F\\displaystyle\\Delta^\{\(s\)\}:=\\\|P^\{\(s\)\}\-P^\{\(s\-1\)\}\\\|\_\{F\}\+\\\|\\Phi^\{\(s\)\}\-\\Phi^\{\(s\-1\)\}\\\|\_\{F\}denote the discrepancy between two consecutive inner iterations\. We have Theorem[1](https://arxiv.org/html/2609.01802#Thmtheorem1a)as follows:

###### Theorem 1\(Stability of the Alternating OT Solver\)\.

LetΔ\(s\)\\Delta^\{\(s\)\}be defined as above, and letμ:=min⁡\(ε,λσ2\)\>0\.\\mu:=\\min\\left\(\\varepsilon,\\frac\{\\lambda\}\{\\sigma^\{2\}\}\\right\)\>0\.AfterSSalternating OT steps, the minimum iterate difference is bounded by an𝒪⁡\(1/S\)\\mathcal\{O\}\(1/\\sqrt\{S\}\)rate:

min1≤s≤S⁡Δ\(s\)≤4​\(𝒥u​\(P\(0\),Φ\(0\)\)−𝒥∗\)μ​S\.\\displaystyle\\min\_\{1\\leq s\\leq S\}\\Delta^\{\(s\)\}\\leq\\sqrt\{\\frac\{4\\big\(\\mathcal\{J\}\_\{u\}\(P^\{\(0\)\},\\Phi^\{\(0\)\}\)\-\\mathcal\{J\}^\{\*\}\\big\)\}\{\\mu S\}\}\.

###### Proof\.

The alternating solver consists of two exact block minimization steps\. We first consider the transport step\. WithΦ\(s−1\)\\Phi^\{\(s\-1\)\}fixed, the transport subproblem is

minP≥0⁡\{⟨P,C⁡\(Φ\(s−1\)\)⟩\+ε​∑a=1Nu\(t\)∑i=1nPa​i​\(log⁡Pa​i−1\)\}\\displaystyle\\min\_\{P\\geq 0\}\\left\\\{\\langle P,C\(\\Phi^\{\(s\-1\)\}\)\\rangle\+\\varepsilon\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}\\sum\_\{i=1\}^\{n\}P\_\{ai\}\(\\log P\_\{ai\}\-1\)\\right\\\}s\.t\.P​𝟏n=q\.\\displaystyle\\text\{s\.t\.\}\\quad P\\mathbf\{1\}\_\{n\}=q\.The entropy term makes this subproblem strongly convex inPP\. Therefore, sinceP\(s\)P^\{\(s\)\}is the exact minimizer, we obtain

𝒥u​\(P\(s−1\),Φ\(s−1\)\)−𝒥u​\(P\(s\),Φ\(s−1\)\)≥ε2​‖P\(s\)−P\(s−1\)‖F2\.\\displaystyle\\mathcal\{J\}\_\{u\}\(P^\{\(s\-1\)\},\\Phi^\{\(s\-1\)\}\)\-\\mathcal\{J\}\_\{u\}\(P^\{\(s\)\},\\Phi^\{\(s\-1\)\}\)\\geq\\frac\{\\varepsilon\}\{2\}\\\|P^\{\(s\)\}\-P^\{\(s\-1\)\}\\\|\_\{F\}^\{2\}\.\(15\)Next, consider the barycenter step\. WithP\(s\)P^\{\(s\)\}fixed, the representative prompt update solves

minΦ⁡\{⟨P\(s\),C⁡\(Φ\)⟩\+λ2​σ2​∑i=1n‖ϕi‖2\}\.\\displaystyle\\min\_\{\\Phi\}\\left\\\{\\langle P^\{\(s\)\},C\(\\Phi\)\\rangle\+\\frac\{\\lambda\}\{2\\sigma^\{2\}\}\\sum\_\{i=1\}^\{n\}\\\|\\phi\_\{i\}\\\|^\{2\}\\right\\\}\.For each representative promptϕi\\phi\_\{i\}, the Hessian of this subproblem is

1σ2​\(∑a=1Nu\(t\)Pa​i\(s\)\+λ\)​I\.\\displaystyle\\frac\{1\}\{\\sigma^\{2\}\}\\left\(\\sum\_\{a=1\}^\{N\_\{u\}^\{\(t\)\}\}P\_\{ai\}^\{\(s\)\}\+\\lambda\\right\)I\.SincePa​i\(s\)≥0P\_\{ai\}^\{\(s\)\}\\geq 0, the minimum eigenvalue is at leastλ/σ2\\lambda/\\sigma^\{2\}\. Hence, the barycenter subproblem isλ/σ2\\lambda/\\sigma^\{2\}\-strongly convex\. SinceΦ\(s\)\\Phi^\{\(s\)\}is the exact minimizer, we have

𝒥u​\(P\(s\),Φ\(s−1\)\)−𝒥u​\(P\(s\),Φ\(s\)\)≥λ2​σ2​‖Φ\(s\)−Φ\(s−1\)‖F2\.\\displaystyle\\mathcal\{J\}\_\{u\}\(P^\{\(s\)\},\\Phi^\{\(s\-1\)\}\)\-\\mathcal\{J\}\_\{u\}\(P^\{\(s\)\},\\Phi^\{\(s\)\}\)\\geq\\frac\{\\lambda\}\{2\\sigma^\{2\}\}\\\|\\Phi^\{\(s\)\}\-\\Phi^\{\(s\-1\)\}\\\|\_\{F\}^\{2\}\.\(16\)
Combining \([15](https://arxiv.org/html/2609.01802#A2.E15)\) and \([16](https://arxiv.org/html/2609.01802#A2.E16)\), and definingμ=min⁡\(ε,λσ2\),\\mu=\\min\\left\(\\varepsilon,\\frac\{\\lambda\}\{\\sigma^\{2\}\}\\right\),we obtain

𝒥u​\(P\(s−1\),Φ\(s−1\)\)−𝒥u​\(P\(s\),Φ\(s\)\)\\displaystyle\\mathcal\{J\}\_\{u\}\(P^\{\(s\-1\)\},\\Phi^\{\(s\-1\)\}\)\-\\mathcal\{J\}\_\{u\}\(P^\{\(s\)\},\\Phi^\{\(s\)\}\)≥μ2​\(‖P\(s\)−P\(s−1\)‖F2\+‖Φ\(s\)−Φ\(s−1\)‖F2\)\\displaystyle\\geq\\frac\{\\mu\}\{2\}\\left\(\\\|P^\{\(s\)\}\-P^\{\(s\-1\)\}\\\|\_\{F\}^\{2\}\+\\\|\\Phi^\{\(s\)\}\-\\Phi^\{\(s\-1\)\}\\\|\_\{F\}^\{2\}\\right\)≥μ4​\(‖P\(s\)−P\(s−1\)‖F\+‖Φ\(s\)−Φ\(s−1\)‖F\)2\\displaystyle\\geq\\frac\{\\mu\}\{4\}\\left\(\\\|P^\{\(s\)\}\-P^\{\(s\-1\)\}\\\|\_\{F\}\+\\\|\\Phi^\{\(s\)\}\-\\Phi^\{\(s\-1\)\}\\\|\_\{F\}\\right\)^\{2\}=μ4​\(Δ\(s\)\)2\.\\displaystyle=\\frac\{\\mu\}\{4\}\\big\(\\Delta^\{\(s\)\}\\big\)^\{2\}\.
Summing overs=1,…,Ss=1,\\dots,Sgives

∑s=1S\(Δ\(s\)\)2\\displaystyle\\sum\_\{s=1\}^\{S\}\\big\(\\Delta^\{\(s\)\}\\big\)^\{2\}≤4μ​∑s=1S\[𝒥u​\(P\(s−1\),Φ\(s−1\)\)−𝒥u​\(P\(s\),Φ\(s\)\)\]\\displaystyle\\leq\\frac\{4\}\{\\mu\}\\sum\_\{s=1\}^\{S\}\\left\[\\mathcal\{J\}\_\{u\}\(P^\{\(s\-1\)\},\\Phi^\{\(s\-1\)\}\)\-\\mathcal\{J\}\_\{u\}\(P^\{\(s\)\},\\Phi^\{\(s\)\}\)\\right\]=4μ​\[𝒥u​\(P\(0\),Φ\(0\)\)−𝒥u​\(P\(S\),Φ\(S\)\)\]\.\\displaystyle=\\frac\{4\}\{\\mu\}\\left\[\\mathcal\{J\}\_\{u\}\(P^\{\(0\)\},\\Phi^\{\(0\)\}\)\-\\mathcal\{J\}\_\{u\}\(P^\{\(S\)\},\\Phi^\{\(S\)\}\)\\right\]\.By Lemma[1](https://arxiv.org/html/2609.01802#Thmlemma1),𝒥u​\(P\(S\),Φ\(S\)\)≥𝒥∗\\mathcal\{J\}\_\{u\}\(P^\{\(S\)\},\\Phi^\{\(S\)\}\)\\geq\\mathcal\{J\}^\{\*\}\. Therefore,

∑s=1S\(Δ\(s\)\)2≤4μ​\[𝒥u​\(P\(0\),Φ\(0\)\)−𝒥∗\]\.\\displaystyle\\sum\_\{s=1\}^\{S\}\\big\(\\Delta^\{\(s\)\}\\big\)^\{2\}\\leq\\frac\{4\}\{\\mu\}\\left\[\\mathcal\{J\}\_\{u\}\(P^\{\(0\)\},\\Phi^\{\(0\)\}\)\-\\mathcal\{J\}^\{\*\}\\right\]\.Finally, since we have

S​min1≤s≤S​\(Δ\(s\)\)2≤∑s=1S\(Δ\(s\)\)2,\\displaystyle S\\min\_\{1\\leq s\\leq S\}\\big\(\\Delta^\{\(s\)\}\\big\)^\{2\}\\leq\\sum\_\{s=1\}^\{S\}\\big\(\\Delta^\{\(s\)\}\\big\)^\{2\},we obtain

min1≤s≤S⁡Δ\(s\)≤4​\(𝒥u​\(P\(0\),Φ\(0\)\)−𝒥∗\)μ​S\.\\displaystyle\\min\_\{1\\leq s\\leq S\}\\Delta^\{\(s\)\}\\leq\\sqrt\{\\frac\{4\\big\(\\mathcal\{J\}\_\{u\}\(P^\{\(0\)\},\\Phi^\{\(0\)\}\)\-\\mathcal\{J\}^\{\*\}\\big\)\}\{\\mu S\}\}\.∎

Theorem[1](https://arxiv.org/html/2609.01802#Thmtheorem1a)provides a quantitative stability guarantee for the local OT\-basedMergestep\. It shows that, among the firstSSalternating iterations, there exists at least one iterate whose change from the previous iterate is bounded by𝒪⁡\(1/S\)\\mathcal\{O\}\(1/\\sqrt\{S\}\)\. Therefore, increasing the number of inner OT steps makes the local merge solution progressively more stable\. The bound also makes explicit how the stability depends on the initial objective gap𝒥u​\(P\(0\),Φ\(0\)\)−𝒥∗\\mathcal\{J\}\_\{u\}\(P^\{\(0\)\},\\Phi^\{\(0\)\}\)\-\\mathcal\{J\}^\{\*\}and the effective strong\-convexity parameterμ=min⁡\(ε,λ/σ2\)\\mu=\\min\(\\varepsilon,\\lambda/\\sigma^\{2\}\)\.

This result has two implications forD\-FROST\. First, the OT\-basedMergestep does not behave as an uncontrolled heuristic\. Specifically, its alternating updates have a provable descent structure and converge toward a stable local representative prompt set\. Second, the number of inner stepsSScontrols the quality of the local merge output\. A largerSSreduces the inner solver instability, which in turn reduces the approximation error introduced when replacing the neighborhood prompt collection by the compact representative setΦ\(S\)\\Phi^\{\(S\)\}\. This local control is the basis for the subsequent network\-level analysis, where the error of the OT\-based merge affects Wasserstein consensus and the convergence of the network barycenter\.

### B\.2Global Wasserstein Consensus and Stationarity

We now analyze the global behavior ofD\-FROST\. The analysis establishes two main results\. First, the local prompt measures remain close to a network\-level barycenter, showing thatD\-FROSTcontrols the Wasserstein consensus error across clients\. Second, this network\-level barycenter makes progress toward stationarity of the shared prompt\-tuning objective in \([1](https://arxiv.org/html/2609.01802#S4.E1)\)\.

First of all, we define notions needed for our analysis\. For each clientuu, we represent its prompt set at roundttas the empirical measure

μu\(t\):=1n​∑i=1nδωu​i\(t\)\.\\mu\_\{u\}^\{\(t\)\}:=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\delta\_\{\\omega\_\{ui\}^\{\(t\)\}\}\.\(17\)Letμavg\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}denote the Wasserstein barycenter of the local prompt measures, as defined in \([2](https://arxiv.org/html/2609.01802#S4.E2)\)\.

##### Local Update\.

During local training, clientuuupdates its prompts via standard backpropagation\. In measure space, this is strictly equivalent to updating the discrete empirical measure via the push\-forward of the Wasserstein gradient:

μu\+\(t\)=\(I−η​∇W2ℱu\)\#​μu\(t−1\)\.\\mu\_\{u\}^\{\+\(t\)\}=\\big\(I\-\\eta\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\_\{u\}\\big\)\_\{\\\#\}\\mu\_\{u\}^\{\(t\-1\)\}\.\(18\)

##### Ideal Reference States\.

During communication, the network seeks consensus over the doubly stochastic graph topologyWW\. If communication were exact, the neighborhood would converge to the Wasserstein barycenter\. We define these theoretical targets for both the local neighborhood and the global network:

1. 1\.The*ideal local barycenter*νu\(t\)\\nu\_\{u\}^\{\(t\)\}: νu\(t\):=arg⁡min⁡∑v=1mν∈𝒫2​\(ℝd\)⁡Wu​v​W22​\(ν,μv\+\(t\)\)\.\\nu\_\{u\}^\{\(t\)\}:=\\arg\\min\_\{\\nu\\in\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\)\}\\sum\_\{v=1\}^\{m\}W\_\{uv\}W\_\{2\}^\{2\}\\big\(\\nu,\\mu\_\{v\}^\{\+\(t\)\}\\big\)\.\(19\)
2. 2\.The*ideal global state*νavg\(t\)\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}, tracking the exact network center of mass: νavg\(t\):=arg⁡minν∈𝒫2​\(ℝd\)​1m​∑u=1mW22​\(ν,μu\+\(t\)\)\.\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}:=\\arg\\min\_\{\\nu\\in\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\)\}\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\big\(\\nu,\\mu\_\{u\}^\{\+\(t\)\}\\big\)\.\(20\)

##### Practical Aggregation and Consensus Error\.

Because computing the exact barycenters causes the prompt support size to grow indefinitely, Algorithm[1](https://arxiv.org/html/2609.01802#alg1)applies a fixed\-budget optimal transport estimator\. We denote this optimal transport compression operator as𝒞OTn\\mathcal\{C\}\_\{\\text\{OT\}\}^\{n\}, which projects the neighborhood updates onto a strictnn\-point summarizing measure:

μu\(t\):=𝒞OTn​\(\{μv\+\(t\)\}v∈𝒩⁡\(u\)∪\{u\}\)\.\\mu\_\{u\}^\{\(t\)\}:=\\mathcal\{C\}\_\{\\text\{OT\}\}^\{n\}\\Big\(\\big\\\{\\mu\_\{v\}^\{\+\(t\)\}\\big\\\}\_\{v\\in\\mathcal\{N\}\(u\)\\cup\\\{u\\\}\}\\Big\)\.\(21\)Consequently, the*actual global state*of the network is simply the barycenter of these compressed measures:

μavg\(t\):=arg⁡minμ∈𝒫2​\(ℝd\)​1m​∑u=1mW22​\(μ,μu\(t\)\)\.\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}:=\\arg\\min\_\{\\mu\\in\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\)\}\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\big\(\\mu,\\mu\_\{u\}^\{\(t\)\}\\big\)\.\(22\)
To track the convergence of the network, we define thenetwork consensus errorat roundttas the average squared 2\-Wasserstein distance between the individual clients’ compressed states and the actual global state:

ε\(t\):=1m​∑u=1mW22​\(μu\(t\),μavg\(t\)\)\.\\varepsilon^\{\(t\)\}:=\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\big\(\\mu\_\{u\}^\{\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\big\)\.\(23\)

##### Optimal Transport Displacement and Bounded Compression\.

To rigorously isolate the algorithmic distortion of𝒞OTn\\mathcal\{C\}\_\{\\text\{OT\}\}^\{n\}and track the network’s optimization trajectory, we must geometrically map the movement of these measures\. In the 2\-Wasserstein space, the displacement between two probability measures is defined by the vector field that optimally transports one measure into the other\. For any two absolutely continuous measuresμ,ν∈𝒫2​\(ℝd\)\\mu,\\nu\\in\\mathcal\{P\}\_\{2\}\(\\mathbb\{R\}^\{d\}\), letTμ→νT\_\{\\mu\\to\\nu\}be the optimal transport map\. The optimal transport displacement is the vector fieldv∈L2​\(μ,ℝd\)v\\in L^\{2\}\(\\mu;\\mathbb\{R\}^\{d\}\)defined asv⁡\(x\):=Tμ→ν​\(x\)−xv\(x\):=T\_\{\\mu\\to\\nu\}\(x\)\-x\. We denote this mapping as the inverse exponential map:

v=expμ−1​\(ν\)\.v=\\text\{exp\}\_\{\\mu\}^\{\-1\}\(\\nu\)\.\(24\)By definition, the squaredL2​\(μ\)L^\{2\}\(\\mu\)norm of this vector field equals the squared 2\-Wasserstein distance:‖v‖L2​\(μ\)2=W22​\(μ,ν\)\\\|v\\\|\_\{L^\{2\}\(\\mu\)\}^\{2\}=W\_\{2\}^\{2\}\(\\mu,\\nu\)\.

Letv~\(t\)=expμavg\(t−1\)−1​\(νavg\(t\)\)\\tilde\{v\}^\{\(t\)\}=\\text\{exp\}\_\{\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\}^\{\-1\}\(\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\)be the ideal displacement mapping to the uncompressed global barycenter, andv\(t\)=expμavg\(t−1\)−1​\(μavg\(t\)\)v^\{\(t\)\}=\\text\{exp\}\_\{\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\}^\{\-1\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\)be the actual displacement mapping to the OT\-compressed global barycenter\. By comparing these two vector fields, the expected projection error of the𝒞OTn\\mathcal\{C\}\_\{\\text\{OT\}\}^\{n\}compressor is bounded by:

𝔼⁡\[‖v~\(t\)−v\(t\)‖L2​\(μavg\(t−1\)\)2\]≤δn2​\(S\),\\mathbb\{E\}\\left\[\\left\\\|\\tilde\{v\}^\{\(t\)\}\-v^\{\(t\)\}\\right\\\|\_\{L^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\}^\{2\}\\right\]\\leq\\delta\_\{n\}^\{2\}\(S\),\(25\)whereSSis the number of steps𝒞OTn\\mathcal\{C\}\_\{\\text\{OT\}\}^\{n\}is allowed to run\. Crucially, established theoretical results on the finite\-sample approximation of Wasserstein barycenters\([Cuturi and Doucet 2014](https://arxiv.org/html/2609.01802#bib.bib10)\)and the convergence rates of empirical measures\([Weed and Bach 2017](https://arxiv.org/html/2609.01802#bib.bib11)\)guarantee that this error is bounded above\. The approximation errorδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)monotonically decays as the client prompt budgetnnand the number of iterative solver stepsSSincrease\.

To complete the convergence framework, we introduce the standard assumptions\.

###### Assumption 1\(Wasserstein Smoothness\)\.

The local loss functionalℱu​\(μ\)\\mathcal\{F\}\_\{u\}\(\\mu\)isLL\-smooth over the 2\-Wasserstein space\. Consequently, the global functionalℱ⁡\(μ\)\\mathcal\{F\}\(\\mu\)is alsoLL\-smooth and bounded below byℱ∗\>−∞\\mathcal\{F\}^\{\*\}\>\-\\infty\.

By the mathematical definition ofLL\-smoothness in the 2\-Wasserstein space, Assumption[1](https://arxiv.org/html/2609.01802#Thmassumption1a)guarantees that for any two absolutely continuous probability measuresμ\\muandν\\nuconnected by the optimal transport displacementv=expμ−1​\(ν\)v=\\text\{exp\}\_\{\\mu\}^\{\-1\}\(\\nu\), the functional satisfies the Taylor\-type upper bound:

ℱ⁡\(ν\)≤ℱ⁡\(μ\)\+⟨∇W2ℱ​\(μ\),v⟩L2​\(μ\)\+L2​‖v‖L2​\(μ\)2\.\\mathcal\{F\}\(\\nu\)\\leq\\mathcal\{F\}\(\\mu\)\+\\langle\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\(\\mu\),v\\rangle\_\{L^\{2\}\(\\mu\)\}\+\\frac\{L\}\{2\}\\\|v\\\|\_\{L^\{2\}\(\\mu\)\}^\{2\}\.\(26\)
###### Assumption 2\(Bounded Variance\)\.

The variance of the local Wasserstein gradients is uniformly bounded:𝔼⁡\[‖∇W2ℱu​\(μ\)‖L22\]≤G2\\mathbb\{E\}\[\\\|\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\_\{u\}\(\\mu\)\\\|\_\{L^\{2\}\}^\{2\}\]\\leq G^\{2\}\.

###### Assumption 3\(Bounded Prompt Support\)\.

The support of the prompt distributions remains within a bounded domain\. Specifically, there exists a constantD\>0D\>0such that for all promptsωu​i\(t\)\\omega\_\{ui\}^\{\(t\)\},‖ωu​i\(t\)‖≤D\\\|\\omega\_\{ui\}^\{\(t\)\}\\\|\\leq D\.

###### Assumption 4\(Graph Spectral Gap\)\.

The communication matrixW∈ℝm×mW\\in\\mathbb\{R\}^\{m\\times m\}is symmetric and doubly stochastic\. Its second largest eigenvalue magnitude governs the spectral gap, defining the network contraction factor:

ρ:=‖W−1m​𝟏𝟏⊤‖2<1\.\\rho:=\\left\\\|W\-\\frac\{1\}\{m\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\\right\\\|\_\{2\}<1\.\(27\)

Assumption[4](https://arxiv.org/html/2609.01802#Thmassumption4a)dictates the standard network topology conditions in decentralized optimization literature\([Nedic and Ozdaglar 2009](https://arxiv.org/html/2609.01802#bib.bib16);[Lian et al\. 2017](https://arxiv.org/html/2609.01802#bib.bib1)\)\. The doubly stochastic property guarantees that the exact global average of the network is strictly preserved during the gossip step\. The symmetry ofWWimplies bidirectional communication channels with equal weightings\. Finally, the spectral gap conditionρ<1\\rho<1is algebraically equivalent to assuming the underlying communication graph is connected and non\-bipartite\. This geometric property ensures that information from any isolated client will eventually propagate to all other clients, providing the mathematical engine that drives the linear contraction of local states toward the global mean\([Koloskova et al\. 2020](https://arxiv.org/html/2609.01802#bib.bib3)\)\.

Step 1\. Network Consensus in the Wasserstein Space\.Before we can establish the final optimization convergence rate of the decentralized algorithm, we must first prove that the network successfully reaches a state of geometric consensus\. The central theoretical challenge is that local prompt\-tuning pulls the clients’ distributions apart, while the graph communication and Optimal Transport \(OT\) compression attempt to pull them together\.

To rigorously bound this dynamic, we decompose the network’s behavior into three fundamental mechanics:

1. 1\.Global Average Preservation \(Lemma[2](https://arxiv.org/html/2609.01802#Thmlemma2)\):We prove that the doubly stochastic graph topology strictly preserves the exact center of mass of the network\.
2. 2\.Local Dispersion \(Lemma[3](https://arxiv.org/html/2609.01802#Thmlemma3)\):We bound how far the local gradient updates drag the clients away from this global center of mass\.
3. 3\.Graph Contraction \(Lemma[4](https://arxiv.org/html/2609.01802#Thmlemma4)\):We map the distributions into a flat kernel space to prove that the communication step strictly contracts this dispersion by the graph’s spectral gap\.

By combining these three mechanics, we construct a linear recurrence relation that permanently traps the network consensus errorε\(t\)\\varepsilon^\{\(t\)\}within a bounded mathematical neighborhood\.

###### Lemma 2\(Preservation of the Global Average\)\.

Letμavg\+\(t\):=1m​∑u=1mμu\+\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}:=\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}\\mu\_\{u\}^\{\+\(t\)\}be the average of the locally updated states\. During the gossip communication step, the ideal continuous barycenter of the network exactly equals this updated average:

νavg\(t\)=μavg\+\(t\)∀t\.\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}=\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}\\qquad\\forall t\.\(28\)

###### Proof\.

By expanding the definition of the ideal global barycenterνavg\(t\)\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}and exchanging the order of summation, we obtain:

νavg\(t\)\\displaystyle\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}=1m​∑u=1mνu\(t\)=1m​∑u=1m∑v=1mWu​v​μv\+\(t\)\\displaystyle=\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}\\nu\_\{u\}^\{\(t\)\}=\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}\\sum\_\{v=1\}^\{m\}W\_\{uv\}\\mu\_\{v\}^\{\+\(t\)\}=1m​∑v=1mμv\+\(t\)​\(∑u=1mWu​v\)\.\\displaystyle=\\frac\{1\}\{m\}\\sum\_\{v=1\}^\{m\}\\mu\_\{v\}^\{\+\(t\)\}\\left\(\\sum\_\{u=1\}^\{m\}W\_\{uv\}\\right\)\.Because the communication matrixWWis column\-stochastic \(Assumption[4](https://arxiv.org/html/2609.01802#Thmassumption4a)\), the inner sum strictly equals11for allvv\. The expression immediately simplifies toμavg\+\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}\. ∎

###### Lemma 3\(Local Dispersion Bound\)\.

Under theLL\-Lipschitz smoothness and bounded gradient variance \(G2G^\{2\}\) assumptions, the geometric dispersion of the locally updated states from their global average is bounded by the previous consensus errorε\(t−1\)\\varepsilon^\{\(t\-1\)\}:

1m​∑u=1mW22​\(μu\+\(t\),μavg\+\(t\)\)≤4​\(1\+η2​L2\)​ε\(t−1\)\+4​η2​G2\.\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\left\(\\mu\_\{u\}^\{\+\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}\\right\)\\leq 4\(1\+\\eta^\{2\}L^\{2\}\)\\varepsilon^\{\(t\-1\)\}\+4\\eta^\{2\}G^\{2\}\.\(29\)

###### Proof\.

We introduce an intermediate virtual state,μmid\(t\):=\(I−η​∇W2ℱ​\(μavg\(t−1\)\)\)\#​μavg\(t−1\)\\mu\_\{\\mathrm\{mid\}\}^\{\(t\)\}:=\\big\(I\-\\eta\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\\big\)\_\{\\\#\}\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}, which represents a perfectly synchronized gradient step\. Applying the relaxed triangle inequalityW22​\(a,c\)≤2​W22​\(a,b\)\+2​W22​\(b,c\)W\_\{2\}^\{2\}\(a,c\)\\leq 2W\_\{2\}^\{2\}\(a,b\)\+2W\_\{2\}^\{2\}\(b,c\)and averaging overmmclients, we have:

1m​∑u=1mW22​\(μu\+\(t\),μavg\+\(t\)\)\\displaystyle\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\left\(\\mu\_\{u\}^\{\+\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}\\right\)≤2​W22​\(μmid\(t\),μavg\+\(t\)\)\\displaystyle\\leq 2W\_\{2\}^\{2\}\\left\(\\mu\_\{\\mathrm\{mid\}\}^\{\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}\\right\)\+2m∑u=1mW22\(μu\+\(t\),μmid\(t\)\)\.\\displaystyle\+\\frac\{2\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\left\(\\mu\_\{u\}^\{\+\(t\)\},\\mu\_\{\\mathrm\{mid\}\}^\{\(t\)\}\\right\)\.For the second term, we bound the distance between the local push\-forward map and the synchronized push\-forward map\. By adding and subtracting the local gradients evaluated at the global average, and utilizing theLL\-smoothness and variance bounds, theL2L^\{2\}mapping error is strictly bounded by\(1\+η2​L2\)​ε\(t−1\)\+η2​G2\(1\+\\eta^\{2\}L^\{2\}\)\\varepsilon^\{\(t\-1\)\}\+\\eta^\{2\}G^\{2\}\.

Applying a similar push\-forward expansion to the first term via Jensen’s inequality isolates the gradient deviations across the network\. Summing the symmetric bounds together absorbs the remaining distances\. ∎

###### Lemma 4\(Graph Contraction via MMD Equivalence\)\.

Let the prompt distributions satisfy the bounded support constraintDD\(Assumption[3](https://arxiv.org/html/2609.01802#Thmassumption3a)\)\. The gossip communication step strictly contracts the network dispersion by the graph’s spectral gapρ2\\rho^\{2\}:

1m​∑u=1mW22​\(νu\(t\),νavg\(t\)\)≤𝒟​ρ2​1m​∑u=1mW22​\(μu\+\(t\),μavg\+\(t\)\),\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\left\(\\nu\_\{u\}^\{\(t\)\},\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\right\)\\leq\\mathcal\{D\}\\rho^\{2\}\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\left\(\\mu\_\{u\}^\{\+\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}\\right\),\(30\)where𝒟\>0\\mathcal\{D\}\>0is a metric translation constant\.

###### Proof\.

Because Wasserstein space is non\-linear, we map the empirical measures into a Reproducing Kernel Hilbert Space \(RKHS\) using the kernel mean embeddingΦ⁡\(μ\)\\Phi\(\\mu\)\. Letθu:=Φ⁡\(μu\+\(t\)\)\\theta\_\{u\}:=\\Phi\(\\mu\_\{u\}^\{\+\(t\)\}\)andyu:=Φ⁡\(νu\(t\)\)y\_\{u\}:=\\Phi\(\\nu\_\{u\}^\{\(t\)\}\)\. Because the embedding is linear, the graph mixing applies directly to the RKHS vectors:yu=∑vWu​v​xvy\_\{u\}=\\sum\_\{v\}W\_\{uv\}x\_\{v\}\. By defining the mean\-centered vectorsx¯u\\bar\{x\}\_\{u\}andy¯u\\bar\{y\}\_\{u\}, standard Euclidean algebraic graph theory provides the spectral bound:

∑u=1m‖y¯u‖ℋk2≤‖W−1m​𝟏𝟏⊤‖op2​∑u=1m‖x¯u‖ℋk2=ρ2​∑u=1m‖x¯u‖ℋk2\.\\displaystyle\\sum\_\{u=1\}^\{m\}\\\|\\bar\{y\}\_\{u\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}^\{2\}\\leq\\left\\\|W\-\\frac\{1\}\{m\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\\right\\\|\_\{\\mathrm\{op\}\}^\{2\}\\sum\_\{u=1\}^\{m\}\\\|\\bar\{x\}\_\{u\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}^\{2\}=\\rho^\{2\}\\sum\_\{u=1\}^\{m\}\\\|\\bar\{x\}\_\{u\}\\\|\_\{\\mathcal\{H\}\_\{k\}\}^\{2\}\.Because the RKHS norm is precisely the Maximum Mean Discrepancy \(MMD\), this establishes the contraction in MMD:∑uMMD2​\(νu\(t\),νavg\(t\)\)≤ρ2​∑uMMD2​\(μu\+\(t\),μavg\+\(t\)\)\\sum\_\{u\}\\mathrm\{MMD\}^\{2\}\(\\nu\_\{u\}^\{\(t\)\},\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\)\\leq\\rho^\{2\}\\sum\_\{u\}\\mathrm\{MMD\}^\{2\}\(\\mu\_\{u\}^\{\+\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\+\(t\)\}\)\.

Under the bounded support constraint, MMD and 2\-Wasserstein metrics are topologically equivalent\. Consequently, there exist strict constantsCD1,CD2\>0C\_\{D\}^\{1\},C\_\{D\}^\{2\}\>0such thatCD1​W22≤MMD2≤CD2​W22C\_\{D\}^\{1\}W\_\{2\}^\{2\}\\leq\\mathrm\{MMD\}^\{2\}\\leq C\_\{D\}^\{2\}W\_\{2\}^\{2\}\. Dividing these boundary constraints yields the metric translation constant𝒟=CD2/CD1\\mathcal\{D\}=C\_\{D\}^\{2\}/C\_\{D\}^\{1\}, converting the RKHS contraction back into the Wasserstein space\. ∎

###### Theorem 2\(Rigorous Wasserstein Consensus Bound\)\.

Under the assumptions of bounded gradients, bounded prompt support, and a doubly stochastic mixing matrix, the expected network consensus errorε\(t\)\\varepsilon^\{\(t\)\}converges asymptotically to a stationary bounded neighborhood\. Specifically, for anyt→∞t\\to\\infty:

ε\(t\)≤β1−α=𝒪⁡\(δn2​\(S\)1−𝒟​ρ2\+η2​G21−𝒟​ρ2\),\\varepsilon^\{\(t\)\}\\leq\\frac\{\\beta\}\{1\-\\alpha\}=\\mathcal\{O\}\\left\(\\frac\{\\delta\_\{n\}^\{2\}\(S\)\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\+\\frac\{\\eta^\{2\}G^\{2\}\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\\right\),\(31\)whereα=6​𝒟​ρ2​\(1\+η2​L2\)<1\\alpha=6\\mathcal\{D\}\\rho^\{2\}\(1\+\\eta^\{2\}L^\{2\}\)<1, andβ=6​δn2​\(S\)\+6​𝒟​ρ2​η2​G2\\beta=6\\delta\_\{n\}^\{2\}\(S\)\+6\\mathcal\{D\}\\rho^\{2\}\\eta^\{2\}G^\{2\}\.

###### Proof\.

We expand the consensus errorε\(t\)\\varepsilon^\{\(t\)\}by chaining the discrete mapping steps through the relaxed three\-way triangle inequality\(a\+b\+c\)2≤3​\(a2\+b2\+c2\)\(a\+b\+c\)^\{2\}\\leq 3\(a^\{2\}\+b^\{2\}\+c^\{2\}\):

W22​\(μu\(t\),μavg\(t\)\)\\displaystyle W\_\{2\}^\{2\}\\big\(\\mu\_\{u\}^\{\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\big\)≤3​W22​\(μu\(t\),νu\(t\)\)\+3​W22​\(νu\(t\),νavg\(t\)\)\\displaystyle\\leq 3W\_\{2\}^\{2\}\\big\(\\mu\_\{u\}^\{\(t\)\},\\nu\_\{u\}^\{\(t\)\}\\big\)\+3W\_\{2\}^\{2\}\\big\(\\nu\_\{u\}^\{\(t\)\},\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\big\)\+3​W22​\(νavg\(t\),μavg\(t\)\)\.\\displaystyle\+3W\_\{2\}^\{2\}\\big\(\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\},\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\big\)\.
We bound each of the three segments by averaging over allmmclients\. First, the term1m​∑uW22​\(μu\(t\),νu\(t\)\)\\frac\{1\}\{m\}\\sum\_\{u\}W\_\{2\}^\{2\}\(\\mu\_\{u\}^\{\(t\)\},\\nu\_\{u\}^\{\(t\)\}\)explicitly represents the projection error of the OT compressor, which is bounded byδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)\. Second, by the joint convexity of the Wasserstein metric, the divergence between the actual compressed global average and the ideal global average,W22​\(μavg\(t\),νavg\(t\)\)W\_\{2\}^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\},\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\), is identically bounded by the average of the local compression errors, yielding anotherδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)\.

For the central graph tracking term, we sequentially apply the contraction bound from Lemma[4](https://arxiv.org/html/2609.01802#Thmlemma4)and the dispersion bound from Lemma[3](https://arxiv.org/html/2609.01802#Thmlemma3):

1m​∑u=1mW22​\(νu\(t\),νavg\(t\)\)≤2​𝒟​ρ2​\(\(1\+η2​L2\)​ε\(t−1\)\+η2​G2\)\.\\displaystyle\\frac\{1\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\\left\(\\nu\_\{u\}^\{\(t\)\},\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\\right\)\\leq 2\\mathcal\{D\}\\rho^\{2\}\\left\(\(1\+\\eta^\{2\}L^\{2\}\)\\varepsilon^\{\(t\-1\)\}\+\\eta^\{2\}G^\{2\}\\right\)\.Plugging these three bounds back into the triangle expansion collapses the dynamics into a single linear recurrence relation:

ε\(t\)≤\[6​𝒟​ρ2​\(1\+η2​L2\)\]⏟:=αε\(t−1\)\+\[6​δn2​\(S\)\+6​𝒟​ρ2​η2​G2\]⏟:=β\.\\displaystyle\\varepsilon^\{\(t\)\}\\leq\\underbrace\{\\left\[6\\mathcal\{D\}\\rho^\{2\}\(1\+\\eta^\{2\}L^\{2\}\)\\right\]\}\_\{:=\\alpha\}\\varepsilon^\{\(t\-1\)\}\+\\underbrace\{\\left\[6\\delta\_\{n\}^\{2\}\(S\)\+6\\mathcal\{D\}\\rho^\{2\}\\eta^\{2\}G^\{2\}\\right\]\}\_\{:=\\beta\}\.To strictly ensure geometric convergence \(α<1\\alpha<1\), we require a learning rate satisfyingη2<1L2​\(16​𝒟​ρ2−1\)\\eta^\{2\}<\\frac\{1\}\{L^\{2\}\}\\big\(\\frac\{1\}\{6\\mathcal\{D\}\\rho^\{2\}\}\-1\\big\)\. Unrolling the recurrence relation ast→∞t\\to\\inftyyields the infinite geometric series boundβ/\(1−α\)\\beta/\(1\-\\alpha\)\. ∎

Step 2\. Global Optimization Convergence\.With the network geometrically trapped in a tight consensus neighborhood, we can now bound the deviation of the network’s gradient trajectory from the ideal centralized trajectory\.

###### Lemma 5\(Tangent\-Space Tracking Error\)\.

Letv^\(t\)=−η​∇W2ℱ​\(μavg\(t−1\)\)\\hat\{v\}^\{\(t\)\}=\-\\eta\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)be the virtual global gradient displacement, and letv\(t\)=expμavg\(t−1\)−1​\(μavg\(t\)\)v^\{\(t\)\}=\\text\{exp\}\_\{\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\}^\{\-1\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\)be the actual optimal transport map to the true network barycenter\. Under Assumption[1](https://arxiv.org/html/2609.01802#Thmassumption1a), the expected tangent\-space tracking error is strictly bounded by:

𝔼​‖v^\(t\)−v\(t\)‖L2​\(μavg\(t−1\)\)2\\displaystyle\\mathbb\{E\}\\left\\\|\\hat\{v\}^\{\(t\)\}\-v^\{\(t\)\}\\right\\\|\_\{L^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\}^\{2\}≤2​η2​L2m​∑u=1m𝔼⁡\[W22​\(μavg\(t−1\),μu\(t−1\)\)\]\\displaystyle\\leq\\frac\{2\\eta^\{2\}L^\{2\}\}\{m\}\\sum\_\{u=1\}^\{m\}\\mathbb\{E\}\\left\[W\_\{2\}^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\},\\mu\_\{u\}^\{\(t\-1\)\}\)\\right\]\+2​δn2​\(S\)\.\\displaystyle\+2\\delta\_\{n\}^\{2\}\(S\)\.\(32\)

###### Proof\.

We bound the divergence between the virtual map and the actual map by introducing the intermediate ideal displacement fieldv~\(t\)=expμavg\(t−1\)−1​\(νavg\(t\)\)\\tilde\{v\}^\{\(t\)\}=\\text\{exp\}\_\{\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\}^\{\-1\}\(\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\), which maps to the uncompressed global barycenter\. Applying the relaxed triangle inequality in the Hilbert spaceL2​\(μavg\(t−1\)\)L^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\), we have:

‖v^\(t\)−v\(t\)‖L22≤2​‖v^\(t\)−v~\(t\)‖L22\+2​‖v~\(t\)−v\(t\)‖L22\.\\left\\\|\\hat\{v\}^\{\(t\)\}\-v^\{\(t\)\}\\right\\\|\_\{L^\{2\}\}^\{2\}\\leq 2\\left\\\|\\hat\{v\}^\{\(t\)\}\-\\tilde\{v\}^\{\(t\)\}\\right\\\|\_\{L^\{2\}\}^\{2\}\+2\\left\\\|\\tilde\{v\}^\{\(t\)\}\-v^\{\(t\)\}\\right\\\|\_\{L^\{2\}\}^\{2\}\.\(33\)For the second term,v~\(t\)\\tilde\{v\}^\{\(t\)\}points to the uncompressed barycenterνavg\(t\)\\nu\_\{\\mathrm\{avg\}\}^\{\(t\)\}, whilev\(t\)v^\{\(t\)\}points to the OT\-compressed barycenterμavg\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\. By the bounded compression property established in \([25](https://arxiv.org/html/2609.01802#A2.E25)\), this term is bounded byδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)\.

For the first term, the virtual mapv^\(t\)\\hat\{v\}^\{\(t\)\}aggregates gradients evaluated at the synchronized global stateμavg\(t−1\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}, while the ideal mapv~\(t\)\\tilde\{v\}^\{\(t\)\}aggregates gradients evaluated at the scattered local statesμu\(t−1\)\\mu\_\{u\}^\{\(t\-1\)\}\. By Jensen’s inequality and theLL\-Lipschitz property of the gradients \(Assumption[1](https://arxiv.org/html/2609.01802#Thmassumption1a)\), we have:

‖v^\(t\)−v~\(t\)‖L22\\displaystyle\\left\\\|\\hat\{v\}^\{\(t\)\}\-\\tilde\{v\}^\{\(t\)\}\\right\\\|\_\{L^\{2\}\}^\{2\}≤η2​L2m​∑u=1mW22​\(μavg\(t−1\),μu\(t−1\)\)\.\\displaystyle\\leq\\frac\{\\eta^\{2\}L^\{2\}\}\{m\}\\sum\_\{u=1\}^\{m\}W\_\{2\}^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\},\\mu\_\{u\}^\{\(t\-1\)\}\)\.\(34\)Summing these bounds and taking the expectation completes the proof\. ∎

###### Theorem 3\(Convergence to a Wasserstein stationarity neighborhood\)\.

Suppose Assumptions[1](https://arxiv.org/html/2609.01802#Thmassumption1a)–[4](https://arxiv.org/html/2609.01802#Thmassumption4a)and the bounded compression property \([25](https://arxiv.org/html/2609.01802#A2.E25)\) hold\. Let the learning rate satisfyη≤1/L\\eta\\leq 1/L\. Then afterTTcommunication rounds, the decentralized OT\-based prompt aggregation procedure satisfies

1T​∑t=1T𝔼⁡\[‖∇W2ℱ​\(μavg\(t−1\)\)‖L2​\(μavg\(t−1\)\)2\]≤2​\(ℱ⁡\(μavg\(0\)\)−ℱ∗\)η​T\\displaystyle\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbb\{E\}\\left\[\\left\\\|\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\\left\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\\right\)\\right\\\|\_\{L^\{2\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\}^\{2\}\\right\]\\leq\\frac\{2\\left\(\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(0\)\}\)\-\\mathcal\{F\}^\{\*\}\\right\)\}\{\\eta T\}\+𝒪⁡\(η2​L2​G21−𝒟​ρ2\)\+𝒪⁡\(L2​δn2​\(S\)1−𝒟​ρ2\+δn2​\(S\)η2\)\.\\displaystyle\+\\mathcal\{O\}\\left\(\\frac\{\\eta^\{2\}L^\{2\}G^\{2\}\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\\right\)\+\\mathcal\{O\}\\left\(\\frac\{L^\{2\}\\delta\_\{n\}^\{2\}\(S\)\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\+\\frac\{\\delta\_\{n\}^\{2\}\(S\)\}\{\\eta^\{2\}\}\\right\)\.

###### Proof\.

Becauseμavg\(t\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}is a Wasserstein barycenter, we expand theLL\-smooth global functionalℱ\\mathcal\{F\}around the previous stateμavg\(t−1\)\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}using \([26](https://arxiv.org/html/2609.01802#A2.E26)\):

𝔼⁡\[ℱ⁡\(μavg\(t\)\)\]≤𝔼⁡\[ℱ⁡\(μavg\(t−1\)\)\]\\displaystyle\\mathbb\{E\}\[\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\)\]\\leq\\mathbb\{E\}\[\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\]\+𝔼⁡\[⟨∇W2ℱ​\(μavg\(t−1\)\),v\(t\)⟩L2\]\\displaystyle\+\\mathbb\{E\}\\left\[\\langle\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\),v^\{\(t\)\}\\rangle\_\{L^\{2\}\}\\right\]\+L2​𝔼​\[‖v\(t\)‖L22\]\.\\displaystyle\+\\frac\{L\}\{2\}\\mathbb\{E\}\\left\[\\\|v^\{\(t\)\}\\\|\_\{L^\{2\}\}^\{2\}\\right\]\.Applying the polarization identity to the inner product with the virtual gradientv^\(t\)=−η​∇W2ℱ​\(μavg\(t−1\)\)\\hat\{v\}^\{\(t\)\}=\-\\eta\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)and utilizing the conditionη≤1/L\\eta\\leq 1/Lto discard the non\-positive‖v\(t\)‖2\\\|v^\{\(t\)\}\\\|^\{2\}coefficient, we obtain the descent inequality:

𝔼⁡\[ℱ⁡\(μavg\(t\)\)\]\\displaystyle\\mathbb\{E\}\[\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\)\]≤𝔼⁡\[ℱ⁡\(μavg\(t−1\)\)\]−η2​𝔼​‖∇W2ℱ​\(μavg\(t−1\)\)‖L22\\displaystyle\\leq\\mathbb\{E\}\[\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\]\-\\frac\{\\eta\}\{2\}\\mathbb\{E\}\\left\\\|\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\\right\\\|\_\{L^\{2\}\}^\{2\}\+12​η​𝔼​‖v^\(t\)−v\(t\)‖L22\.\\displaystyle\+\\frac\{1\}\{2\\eta\}\\mathbb\{E\}\\left\\\|\\hat\{v\}^\{\(t\)\}\-v^\{\(t\)\}\\right\\\|\_\{L^\{2\}\}^\{2\}\.Substituting the tracking error bound from Lemma[5](https://arxiv.org/html/2609.01802#Thmlemma5)and recognizing that the trailing summation1m​∑uW22\\frac\{1\}\{m\}\\sum\_\{u\}W\_\{2\}^\{2\}is exactly our asymptotic network consensus errorε\(t−1\)\\varepsilon^\{\(t\-1\)\}defined in \([23](https://arxiv.org/html/2609.01802#A2.E23)\) and bounded in Theorem[2](https://arxiv.org/html/2609.01802#Thmtheorem2a), we find:

𝔼⁡\[ℱ⁡\(μavg\(t\)\)\]\\displaystyle\\mathbb\{E\}\[\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\)\}\)\]≤𝔼⁡\[ℱ⁡\(μavg\(t−1\)\)\]−η2​𝔼​‖∇W2ℱ​\(μavg\(t−1\)\)‖L22\\displaystyle\\leq\\mathbb\{E\}\[\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\]\-\\frac\{\\eta\}\{2\}\\mathbb\{E\}\\left\\\|\\nabla\_\{W\_\{2\}\}\\mathcal\{F\}\(\\mu\_\{\\mathrm\{avg\}\}^\{\(t\-1\)\}\)\\right\\\|\_\{L^\{2\}\}^\{2\}\+𝒪⁡\(η3​L2​G21−𝒟​ρ2\+η​L2​δn2​\(S\)1−𝒟​ρ2\)\+δn2​\(S\)η\.\\displaystyle\+\\mathcal\{O\}\\left\(\\frac\{\\eta^\{3\}L^\{2\}G^\{2\}\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\+\\frac\{\\eta L^\{2\}\\delta\_\{n\}^\{2\}\(S\)\}\{1\-\\mathcal\{D\}\\rho^\{2\}\}\\right\)\+\\frac\{\\delta\_\{n\}^\{2\}\(S\)\}\{\\eta\}\.Telescoping acrossTTrounds and dividing byη​T/2\\eta T/2yields the final result\. ∎

##### Interpretation\.

Theorem[3](https://arxiv.org/html/2609.01802#Thmtheorem3a)demonstrates that the procedure converges to a neighborhood of stationarity\. The neighborhood size is determined by the gradient varianceG2G^\{2\}and the aggregation noiseδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)induced by the fixed\-budget OT constraint\. Notably, theδn2​\(S\)/η2\\delta\_\{n\}^\{2\}\(S\)/\\eta^\{2\}penalty indicates that the approximation error in the communication step sets a floor on the achievable stationarity, a common characteristic in decentralized optimization with lossy compression\.

## Appendix CAdditional Experiment Settings

### C\.1Datasets and Partitions

Our experiments are conducted on two synthetic, multi\-domain datasets constructed by pooling together several heterogeneous image classification benchmarks\. These composite datasets are designed to simulate realistic federated learning scenarios in which clients not only disagree on class distributions, but also hold data drawn from different visual domains\.

##### 4\-dataset\([Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\):

The first composite dataset combines four sub\-datasets: MNIST\-M\([Ganin et al\. 2016](https://arxiv.org/html/2609.01802#bib.bib17)\), Fashion\-MNIST, CINIC\-10\([Darlow et al\. 2018](https://arxiv.org/html/2609.01802#bib.bib18)\), and MMAFEDB \(available on Kaggle\)111https://www\.kaggle\.com/datasets/yuulind/mmafedb\-clean\. These sub\-datasets span diverse visual domains ranging from colorized digit images to fashion items, natural scene photographs, and facial expressions\. Together they constitute 37 classes \(10 \+ 10 \+ 10 \+ 7, respectively\)\. For the training partition, we sample 30,000 examples per sub\-dataset, yielding 120,000 training images in total\. For the test partition, we sample 2,500 examples per sub\-dataset, yielding 10,000 test images in total\. We simulatem=40m=40clients by assigning 10 clients to each sub\-dataset, so that each client only ever holds data from one visual domain\.

##### 5\-dataset\([Wang et al\. 2022](https://arxiv.org/html/2609.01802#bib.bib15)\):

The second composite dataset combines five sub\-datasets: CIFAR\-10\([Krizhevsky et al\. 2009](https://arxiv.org/html/2609.01802#bib.bib19)\), MNIST\([LeCun 1998](https://arxiv.org/html/2609.01802#bib.bib20)\), Fashion\-MNIST, SVHN\([Netzer et al\. 2011](https://arxiv.org/html/2609.01802#bib.bib21)\), and NotMNIST\([Bulatov 2011](https://arxiv.org/html/2609.01802#bib.bib22)\), each contributing 10 classes for a total of 50 classes\. This collection spans natural image classification, handwritten digit recognition, grayscale fashion item recognition, street\-view digit recognition, and printed character recognition, covering a broad range of low\-level statistics and label semantics\. For the training partition, we sample 20,000 examples per sub\-dataset, yielding 100,000 training images in total\. For the test partition, we sample 2,000 examples per sub\-dataset, yielding 10,000 test images in total\. We simulatem=50m=50clients by assigning 10 clients to each sub\-dataset\.

##### Heterogeneous Partition\.

We partition each sub\-dataset independently among its 10 assigned clients using aDirichlet​\(α⋅𝟏s\)\\text\{Dirichlet\}\(\\alpha\\cdot\\mathbf\{1\}\_\{s\}\)distribution over thess\-class simplex, wheressis the number of classes in that sub\-dataset\. Each client receives a proportion vector drawn from this distribution, controlling what fraction of each class is allocated to that client\. Smaller values ofα\\alphaproduce more skewed, heterogeneous distributions\. We run experiments withα=0\.1\\alpha=0\.1, which produces high heterogeneity, andα=0\.5\\alpha=0\.5, which produces moderate heterogeneity\. Because the Dirichlet draws are applied independently per sub\-dataset using a shared random seed, the resulting distributions are statistically comparable across sub\-datasets within the same run\.

##### Extreme Non\-iid Partition\.

We additionally evaluate a manual extreme\-heterogeneity setting that produces maximally imbalanced local datasets\. For each sub\-dataset, 99% of the data belonging to each class is assigned exclusively to one designated client, while the remaining 1% is distributed among non\-designated clients via a symmetric Dirichlet distribution with concentration parameterα=1\\alpha=1\. Since each sub\-dataset contains exactly 10 classes and is partitioned among exactly 10 clients, this scheme results in a bijective assignment in which every client is dominated by exactly one class, with only trace amounts of the remaining classes present in its local dataset\.

The above partitioning schemes are applied only to the training split\. Evaluation is performed globally on the full held\-out test partition of each composite dataset\.

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/network_topologies.png)Figure 6:Communication topologies used in our experiments \(m=20m=20nodes shown for clarity\)\. From left to right: Ring, Grid, Erdős\-Rényi, Regular \(κ=5\\kappa=5\), and Fully Connected\.

### C\.2Hyperparameter Settings

For details regarding Prompt\-tuning protocol and DFL baselines, please refer to Appendix[A](https://arxiv.org/html/2609.01802#A1)\.

##### Shared settings\.

All methods share the same backbone, optimizer, batch size, number of communication rounds, prompt configuration, communication topology, participation rate, and evaluation schedule; only the method\-specific parameters listed below differ\. Concretely, every method uses a frozen ViT\-B/32 backbone withn=10n=10learnable prompt tokens of dimensiond=768d=768prepended to the patch\-embedding sequence, optimized with Adam at learning rateη=10−4\\eta=10^\{\-4\}and batch size1616\. Training runs for4040communication rounds\. All clients participate in every round \(full participation\)\. The number of clients is4040for FourDataset and5050for FiveDataset, with the default communication topology being a time\-varyingκ\\kappa\-regular graph \(κ=4\\kappa=4for FourDataset,κ=5\\kappa=5for FiveDataset\)\.

##### D\-FROST\.

D\-FROST runs55local epochs per round with Adam\. The OT\-basedMergestep runsS=50S=50alternating inner iterations, with entropy regularization weightε=0\.01\\varepsilon=0\.01, spatial scaleσ2=1\.0\\sigma^\{2\}=1\.0, andL2L\_\{2\}regularization weightλ=0\.001\\lambda=0\.001\.ε=0\.01\\varepsilon=0\.01follows standard entropic\-OT practice of choosing smallε\\varepsilonfor sharp, stable transport\([Cuturi 2013](https://arxiv.org/html/2609.01802#bib.bib24);[Peyré and Cuturi 2019](https://arxiv.org/html/2609.01802#bib.bib25)\)\. We also exploreε\\varepsilonannealing\([Schmitzer 2019](https://arxiv.org/html/2609.01802#bib.bib26)\)in Appendix[G\.4](https://arxiv.org/html/2609.01802#A7.SS4)\. Due to the low computation cost of the OT\-basedMergestep \(see Appendix[F](https://arxiv.org/html/2609.01802#A6)\), we can afford a large number of inner iterationsSSto drive the merge errorδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)low and ensure the solver converges\. We found thatS=50S=50is sufficient\.

##### DFedAvgM\-PT\.

DFedAvgM\-PT runs55local epochs per round using SGD with momentumβ=0\.99\\beta=0\.99\.

##### DFedSAM\-PT\.

DFedSAM\-PT runs55local epochs per round using adaptive SAM with perturbation radiusρ=0\.01\\rho=0\.01, following prior work\.

##### D\-PSGD\-PT\.

D\-PSGD\-PT runs11local epoch per round using SGD with momentum00\.

##### Implementation Details\.

Experiments, including the runtime measurements in Appendix[F](https://arxiv.org/html/2609.01802#A6), were conducted on a Linux workstation running Ubuntu 20\.04 LTS, equipped with an Intel Xeon E5\-2697 v4 CPU @ 2\.30 GHz \(18 cores, 36 threads\), 384 GB RAM, and a NVIDIA RTX A6000 GPU \(48 GB VRAM\)\. Our implementation is based on PyTorch 2\.0 with CUDA 12\.2\.

### C\.3Network Topologies

We evaluate all methods on five undirected communication topologies of varying connectivity\. Each topology is instantiated overmmclients and represented by a symmetric doubly stochastic mixing matrixW∈ℝm×mW\\in\\mathbb\{R\}^\{m\\times m\}, whereWu​v\>0W\_\{uv\}\>0only ifu=vu=vor\(u,v\)\(u,v\)is an edge\. The spectral mixing factorρ=‖W−1m​𝟏𝟏⊤‖2\\rho=\\\|W\-\\frac\{1\}\{m\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\\\|\_\{2\}characterizes how quickly information spreads: a smallerρ\\rhoindicates faster mixing and tighter Wasserstein consensus \(Theorem 2\)\. The five topologies, ordered from sparsest to densest \(ρ\\rhodecreasing\), are as follows\.

##### Ring\.

Each client connects to exactly two neighbors arranged in a cycle\. Withmmclients, every node has degree22, making the ring the sparsest topology and the one with the largest mixing factorρ\\rho\. Node identities are randomly permuted at each communication round while the cyclic structure is preserved\.

##### Grid\.

Clients are arranged in a two\-dimensional lattice withr×c=mr\\times c=mcells, wherer=max\{k≤⌊m⌋:k∣m\}r=\\max\\\{k\\leq\\lfloor\\sqrt\{m\}\\rfloor:k\\mid m\\\}andc=m/rc=m/r\. For FiveDataset \(m=50m=50\) this yields a5×105\\times 10lattice; for FourDataset \(m=40m=40\) a5×85\\times 8lattice\. Interior nodes have degree44, boundary nodes degree33or22, and corner nodes degree22\. Each round, node identities are randomly permuted at each communication round while the lattice structure is preserved, so neighbor assignments change over time\.

##### Erdős\-Rényi\.

Each pair of clients is connected independently with probabilityp=κ/\(m−1\)p=\\kappa/\(m\-1\), matching the expected degree of theκ\\kappa\-regular topology \(κ=5\\kappa=5,p≈0\.1p\\approx 0\.1form=50m=50\)\. Unlike the regular graph, the ER graph has degree variance, so some nodes acquire fewer links than others\. A fresh ER graph is resampled each round\.

##### Regular \(default\)\.

Each client is connected to exactlyκ\\kapparandomly chosen neighbors, forming aκ\\kappa\-regular graph\. We useκ=5\\kappa=5for FiveDataset andκ=4\\kappa=4for FourDataset\. As our*standard*topology, we employ a*time\-varying*κ\\kappa\-regular graph: at each communication roundtt, a fresh randomκ\\kappa\-regular graphG\(t\)=\(V,E\(t\)\)G^\{\(t\)\}=\(V,E^\{\(t\)\}\)is independently sampled, so the neighbor set of each client changes every round\. This models realistic wireless or peer\-to\-peer networks with transient link availability\.

##### Fully Connected\.

Every pair of clients communicates directly, yielding a complete graph of degreem−1m\-1\. The mixing matrix isW=1m​𝟏𝟏⊤W=\\frac\{1\}\{m\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}, givingρ=0\\rho=0and perfect one\-hop consensus\. This topology represents an idealized upper bound on connectivity\.

##### Mixing matrix construction\.

For all topologies, the mixing matrixWWis symmetric and doubly stochastic for any undirected graph, satisfying Assumption 4\.

## Appendix DAdditional Experiment Results

Tables[1](https://arxiv.org/html/2609.01802#A4.T1)and[2](https://arxiv.org/html/2609.01802#A4.T2)report final test accuracy on FourDataset and FiveDataset under the two Dirichlet splits \(α=0\.5\\alpha=0\.5,α=0\.1\\alpha=0\.1\) and the extreme non\-IID partition, and Table[3](https://arxiv.org/html/2609.01802#A4.T3)breaks down FiveDataset \(α=0\.1\\alpha=0\.1\) across the five communication topologies\. D\-FROST achieves the best accuracy in every setting, and its margin over the strongest baseline widens as heterogeneity increases\. The gains are also consistent across all topologies, confirming that the advantage of OT\-based merging does not depend on a particular graph structure\.

Figure[7](https://arxiv.org/html/2609.01802#A4.F7)shows the performance comparison on FourDataset\. Specifically, it reaches69\.06%69\.06\\%,63\.94%63\.94\\%, and58\.24%58\.24\\%, improving over the strongest baseline \(DFedAvgM\) by5\.785\.78,4\.224\.22, and12\.1912\.19points, respectively\.

Table 1:Test Accuracy \(%\) achieved on theFourdatasetby D\-FROST and other baselines\.Table 2:Test Accuracy \(%\) achieved on theFivedatasetby D\-FROST and other baselines\.Table 3:Test accuracy \(%\) in various network topologies on FiveDataset under Dirichletα=0\.1\\alpha=0\.1\. All methods follow the prompt\-tuning \(PT\) protocol\.![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/non_iid_curves_fourdataset.png)Figure 7:Test accuracy of all methods onFourDataset\(40 clients\) under three non\-IID settings: Dirichletα=0\.5\\alpha\{=\}0\.5, Dirichletα=0\.1\\alpha\{=\}0\.1, and the extreme non\-IID partition\.
## Appendix EFactors Impacting Network Consensus Error

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/consensus_error_factors.png)Figure 8:Total Wasserstein consensus error of D\-FROST across all clients on FiveDataset under Dirichletα=0\.1\\alpha=0\.1\.Theorem[2](https://arxiv.org/html/2609.01802#Thmtheorem2a)bounds the network consensus error byε\(t\)≤β/\(1−α\)=O⁡\(δn2​\(S\)\+η2​G21−D​ρ2\)\\varepsilon^\{\(t\)\}\\leq\\beta/\(1\-\\alpha\)=O\\\!\\big\(\\tfrac\{\\delta\_\{n\}^\{2\}\(S\)\+\\eta^\{2\}G^\{2\}\}\{1\-D\\rho^\{2\}\}\\big\), whose denominator is governed by the mixing factorρ\\rho: weaker mixing \(largerρ\\rho\) enlarges the consensus neighborhood\. We probe this prediction by degrading the network along three axes: topology connectivity, link dropout, and partial participation, and tracking the total consensus error over the communication rounds \(Figure[8](https://arxiv.org/html/2609.01802#A5.F8)\)\. The curves separate exactly as the bound predicts\. The denser Grid mixes fastest and sits well below the sparse Ring at every round\. Introducing link dropout \(p=0\.2p=0\.2\) on Ring topology degrades its effective mixing and lifts its curve to the top of the plot, while partial participation \(3030of5050clients per round\) slows mixing and places it between the clean Ring and the dropout case\. The result confirms that every factor weakening graph mixing enlarges the consensus floor through the same1/\(1−D​ρ2\)1/\(1\-D\\rho^\{2\}\)mechanism\. Across all four settings, however,ε\(t\)\\varepsilon^\{\(t\)\}still contracts monotonically, showing that the OT\-based merge keeps the network converging even under sparse, unreliable, or partially participating topologies\.

## Appendix FCost Analysis of D\-FROST

We analyze the per\-round cost the OT\-basedMergestep \(Algorithm[1](https://arxiv.org/html/2609.01802#alg1)\), asymptotically and in wall\-clock time\.

##### Setup\.

At roundtt, clientuuforms the neighborhood collectionΩu\(t\)=ω~u\(t\)⊎⨄v∈𝒩⁡\(u\)ω~v\(t\)\\Omega\_\{u\}^\{\(t\)\}=\\tilde\{\\omega\}\_\{u\}^\{\(t\)\}\\uplus\\biguplus\_\{v\\in\\mathcal\{N\}\(u\)\}\\tilde\{\\omega\}\_\{v\}^\{\(t\)\}of sizeNu\(t\):=\|Ωu\(t\)\|N\_\{u\}^\{\(t\)\}:=\|\\Omega\_\{u\}^\{\(t\)\}\|\(Eq\. \([5](https://arxiv.org/html/2609.01802#S4.E5)\)\); withnnprompts per client and aκ\\kappa\-regular topology,Nu\(t\)=\(κ\+1\)​nN\_\{u\}^\{\(t\)\}=\(\\kappa\+1\)\\,n\. The representative setΦ=\{ϕi\}i=1n\\Phi=\\\{\\phi\_\{i\}\\\}\_\{i=1\}^\{n\}stays sizennthroughout, so the merged prompt\-set size is preserved\.

##### Complexity\.

Each of theSSinner iterations runs three closed\-form steps: the cost matrixC∈ℝNu\(t\)×nC\\in\\mathbb\{R\}^\{N\_\{u\}^\{\(t\)\}\\times n\}\(Eq\. \([6](https://arxiv.org/html/2609.01802#S4.E6)\)\), the transport planPP\(Eq\. \([8](https://arxiv.org/html/2609.01802#S4.E8)\)\), and the barycenter update \(Eq\. \([9](https://arxiv.org/html/2609.01802#S4.E9)\)\)\. It is dominated by the two matrix productsΩu​Φ⊤\\Omega\_\{u\}\\Phi^\{\\top\}andP⊤​ΩuP^\{\\top\}\\Omega\_\{u\}\. This gives

𝒯OT=O⁡\(S⁡\(κ\+1\)​n2​d\)\\mathcal\{T\}\_\{\\mathrm\{OT\}\}=O\\\!\\big\(S\\,\(\\kappa\+1\)\\,n^\{2\}d\\big\)\(35\)quadratic in the prompt budgetnnand linear inκ\+1\\kappa\+1,dd, andSS\. With our settings \(S=50S\{=\}50,κ=5\\kappa\{=\}5,n=10n\{=\}10,d=768d\{=\}768, soNu\(t\)=60N\_\{u\}^\{\(t\)\}\{=\}60\), the merge costs≈4\.6×107\\approx 4\.6\\times 10^\{7\}MACs, which is negligible compared to a forward/backward pass of the frozen ViT\-B/32 model\.

##### Wall\-clock overhead\.

Table[4](https://arxiv.org/html/2609.01802#A6.T4)reports per\-client, per\-round times on FiveDataset \(κ=5\\kappa\{=\}5,n=10n\{=\}10\)\. The OT merge takes0\.630\.63s versus0\.260\.26s for index\-wise averaging\. The OT merge step only accounts for≈1\.5%\\approx 1\.5\\%of round time, which is dominated by local training \(≈42\.3\\approx 42\.3s\)\.

Table 4:Per\-round wall\-clock times \(seconds\) for one client on FiveDataset, using a time\-varyingκ\\kappa\-regular graph \(κ=5\\kappa\{=\}5\)\.*Agg\.*is theMergestep only;*Round*is total \(train\+\+merge\)\.
##### Communication overhead\.

The OT cost matrixCCand transport planPPare computed locally and*never transmitted*; clients exchange only their updated prompt setsω~v\(t\)\\tilde\{\\omega\}\_\{v\}^\{\(t\)\}\(nnprompts of dimensiondd\), exactly as the parametric baselines do\. The OT\-based merge therefore incursno extra communication costover index\-wise averaging\.

##### Accuracy\-efficiency trade\-off\.

Figure[9](https://arxiv.org/html/2609.01802#A6.F9)plots final accuracy against average wall\-clock time per round per client on FiveDataset\. D\-FROST reaches the highest accuracy \(79\.36%79\.36\\%\) at42\.9342\.93s/round and lies on the Pareto frontier\. DFedSAM\-PT is the most expensive method at80\.7580\.75s/round yet reaches only66\.74%66\.74\\%\. The cheaper baselines \(D\-PSGD\-PT at8\.688\.68s/round and DFedAvgM\-PT at42\.6942\.69s/round\) run faster per round but plateau far below D\-FROST in accuracy\. Furthermore, while the OT\-based aggregation procedure introduces a marginal computational overhead compared to index\-wise averaging, it yields substantial accuracy gains without incurring any extra communication cost\. We analyze D\-FROST’s overhead in detail in Appendix[F](https://arxiv.org/html/2609.01802#A6)\.

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/speed_vs_accuracy.png)Figure 9:Accuracy versus per\-round wall\-clock per client cost on FiveDataset \(α=0\.1\\alpha=0\.1\)\.

## Appendix GAblation Studies

### G\.1Impact of Client Prompt Budgetnn

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/l2p_prompt_budget_comparison.png)Figure 10:Impact of client prompt budgetnn\.The prompt budgetnnis the number of representative prompts each client retains after the OT\-basedMergestep, i\.e\. the size ofΦ=\{ϕi\}i=1n\\Phi=\\\{\\phi\_\{i\}\\\}\_\{i=1\}^\{n\}\. It controls how faithfully the merged set summarizes the neighborhood collectionΩu\(t\)\\Omega\_\{u\}^\{\(t\)\}: a largernnlowers the merge errorδn2​\(S\)\\delta\_\{n\}^\{2\}\(S\)\(Eq\. \([25](https://arxiv.org/html/2609.01802#A2.E25)\)\), which by Theorems[2](https://arxiv.org/html/2609.01802#Thmtheorem2a)and[3](https://arxiv.org/html/2609.01802#Thmtheorem3a)tightens both the consensus and stationarity neighborhoods and thus raises attainable accuracy\.

On FiveDataset under the extreme non\-IID partition, shrinking the budget ton=5n=5drops accuracy from70\.46%70\.46\\%to63\.35%63\.35\\%\. We usen=10n=10as the default in all main experiments\.

### G\.2Impact of Different Frozen Backbones

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/l2p_backbone_comparison.png)Figure 11:Impact of different pre\-trained backbones\.D\-FROST treats the backbone as a frozen feature extractor and performs all aggregation in thedd\-dimensional prompt embedding space \(Eq\. \([6](https://arxiv.org/html/2609.01802#S4.E6)\)\)\. Since the OT\-basedMergenever inspects the backbone weights or architecture, the method transfers across ViT architectures\. Figure[11](https://arxiv.org/html/2609.01802#A7.F11)verifies this on FiveDataset \(α=0\.1\\alpha=0\.1\) with three frozen backbones: ViT\-B/32, DeiT\-B/16\([Touvron et al\. 2021](https://arxiv.org/html/2609.01802#bib.bib27)\)and ConViT\-Base\([d’Ascoli et al\. 2021](https://arxiv.org/html/2609.01802#bib.bib28)\)\.

### G\.3Comparison with Improved Baselines

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/l2p_noniid_comparison.png)Figure 12:D\-FROST versus the improved baselines \(denoted\+\+\) on FiveDataset under the extreme non\-IID partition\. Each baseline is augmented with an L2P\-style prompt\-selection mechanism\([Wang et al\. 2022](https://arxiv.org/html/2609.01802#bib.bib15)\)\.Beyond the results in the main text, we compare D\-FROST against a stronger set of baselines\. Each DFL baseline is augmented with a client\-specific prompt\-selection mechanism\([Wang et al\. 2022](https://arxiv.org/html/2609.01802#bib.bib15);[Weng et al\. 2024](https://arxiv.org/html/2609.01802#bib.bib8)\), which lets every client contextualize its local data by selecting the most relevant prompts from a shared pool rather than collapsing distinct contexts into the same prompts\. Concretely, each client maintains a prompt pool of size2020and, for each input, selects1010most relevant prompts from this pool to prepend using a query mechanism\. Setting the pool size to1010recovers the original baseline, since every input then uses all1010prompts and no selection takes place\. We denote these improved variants with a\+\+: D\-PSGD\-PT\+\+, DFedAvgM\-PT\+\+, and DFedSAM\-PT\+\+\.

Figure[12](https://arxiv.org/html/2609.01802#A7.F12)reports the comparison on FiveDataset under the extreme non\-IID partition\. Prompt selection substantially raises the baselines over their original counterparts, yet D\-FROST still outperforms all of them by a clear margin at every round: it separates within the first few rounds and converges to roughly79%79\\%, while the strongest improved baseline, DFedAvgM\+\+, plateaus near63%63\\%, followed by DFedSAM\+\+\(≈58%\\approx 58\\%\) and D\-PSGD\+\+\(≈30%\\approx 30\\%\)\.

### G\.4Entropy Value Selection Strategies

![Refer to caption](https://arxiv.org/html/2609.01802v1/Figures/l2p_entropy_comparison.png)Figure 13:Impact of Entropy Value Selection StrategiesThe entropy weightε\\varepsilonin the OT merge objective \([7](https://arxiv.org/html/2609.01802#S4.E7)\) scales the regularizerε​∑a,iPa​i​\(log⁡Pa​i−1\)\\varepsilon\\sum\_\{a,i\}P\_\{ai\}\(\\log P\_\{ai\}\-1\), the standard entropic regularization of optimal transport\([Cuturi 2013](https://arxiv.org/html/2609.01802#bib.bib24);[Peyré and Cuturi 2019](https://arxiv.org/html/2609.01802#bib.bib25)\): a smallε\\varepsilonkeeps the plan close to the exact OT solution and yields sharp assignments\. We experiment withε\\varepsilon\-scaling\([Schmitzer 2019](https://arxiv.org/html/2609.01802#bib.bib26)\), a geometric schedule that starts from a largeεinit\\varepsilon\_\{\\text\{init\}\}and anneals it down to the targetε=0\.01\\varepsilon=0\.01over theSSinner iterations\. On FiveDataset under Dirichletα=0\.1\\alpha=0\.1\(Figure[13](https://arxiv.org/html/2609.01802#A7.F13)\), annealing does not improve over the fixed schedule\. For simplicity we therefore use a fixedε=0\.01\\varepsilon=0\.01in all experiments\.