CoASFL：面向异步分割联邦学习的耦合一致性优化

# CoASFL: Coupled Consistency Optimization for Asynchronous Split Federated Learning
Source: [https://arxiv.org/html/2610.26992](https://arxiv.org/html/2610.26992)
Yunming LiaoHongli XuYang XuZuan Xie††thanks:J\. Cheng, Y\. Liao, H\. Xu, Y\. Xu, and Z\. Xie are with the School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China, 230027, and also with Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, 215123\. E\-mails: jhcheng01@mail\.ustc\.edu\.cn, ymliao98@mail\.ustc\.edu\.cn, xuhongli@ustc\.edu\.cn, xuyangcs@ustc\.edu\.cn, xz1314@mail\.ustc\.edu\.cn\.

###### Abstract

Split Federated Learning \(SFL\) has emerged as a promising paradigm for training large\-scale deep models on resource\-constrained edge devices\. Although asynchronous SFL alleviates the straggler bottleneck caused by system heterogeneity, it inevitably introduces two coupled dimensions of inconsistency: horizontal inconsistency across clients and vertical inconsistency between clients and the server\. These inconsistencies, further exacerbated by statistical heterogeneity, result in biased activation representations, model divergence, and severe client drift\. To address these challenges, we propose CoASFL, an inconsistency\-aware asynchronous SFL framework that incorporates a dynamic activation merging mechanism and a consistency\-aware model aggregation mechanism\. Specifically, CoASFL constructs more consistent activation sequences from heterogeneous clients to alleviate horizontal inconsistency, and dynamically adjusts aggregation weights according to client participation frequencies and progress gaps to mitigate vertical inconsistency\. By jointly coordinating activation\-level and model\-level consistency, CoASFL effectively suppresses inconsistency propagation during asynchronous training\. Extensive experiments on an 80\-device heterogeneous edge testbed demonstrate that CoASFL consistently outperforms state\-of\-the\-art SFL approaches, accelerating training by 1\.61×\\timesto 3\.74×\\timesand improving final model accuracy by up to 3\.26% under heterogeneous scenarios\.

###### Index Terms:

Split Federated Learning, Asynchronous Training, Edge Intelligence, Consistency Optimization\.

## IIntroduction

The rapid advancement of deep learning has driven the deployment of intelligent applications at the network edge, including autonomous systems, intelligent healthcare, and industrial Internet of Things \(IoT\)\[[1](https://arxiv.org/html/2610.26992#bib.bib19),[2](https://arxiv.org/html/2610.26992#bib.bib18)\]\. However, training increasingly large deep models on distributed edge devices remains challenging due to the limited computation capability, storage capacity, and communication resources of individual devices\[[3](https://arxiv.org/html/2610.26992#bib.bib22),[4](https://arxiv.org/html/2610.26992#bib.bib21)\]\. Federated Learning \(FL\) provides a privacy\-preserving paradigm for collaborative model training by keeping raw data locally while aggregating knowledge from distributed clients\[[5](https://arxiv.org/html/2610.26992#bib.bib15),[6](https://arxiv.org/html/2610.26992#bib.bib20),[7](https://arxiv.org/html/2610.26992#bib.bib7)\]\. Nevertheless, conventional FL requires frequent exchanges of complete model parameters between clients and the central server, resulting in substantial communication and computation overhead when training large\-scale models\[[8](https://arxiv.org/html/2610.26992#bib.bib26)\]\. Split Federated Learning \(SFL\) addresses this limitation by partitioning the model into client\-side bottom models and a server\-side top model, where clients exchange intermediate activations and split\-layer gradients rather than full model parameters\[[9](https://arxiv.org/html/2610.26992#bib.bib25),[10](https://arxiv.org/html/2610.26992#bib.bib1)\]\. By reducing the resource burden on edge devices, SFL has become an attractive solution for collaborative intelligence in heterogeneous edge environments\[[11](https://arxiv.org/html/2610.26992#bib.bib57),[12](https://arxiv.org/html/2610.26992#bib.bib59)\]\.

Despite its advantages, conventional SFL generally follows a synchronous training paradigm, where the server must wait for all clients to complete local training before updating the global model\[[13](https://arxiv.org/html/2610.26992#bib.bib4)\]\. Such synchronization barriers severely restrict training efficiency in practical edge systems, where clients often exhibit significant heterogeneity in computation capability, network conditions, and resource availability\[[14](https://arxiv.org/html/2610.26992#bib.bib50),[15](https://arxiv.org/html/2610.26992#bib.bib46),[16](https://arxiv.org/html/2610.26992#bib.bib51),[17](https://arxiv.org/html/2610.26992#bib.bib58)\]\. For instance, heterogeneous edge devices may require substantially different execution times to process identical workloads, while unstable wireless networks can further increase activation transmission delays\[[18](https://arxiv.org/html/2610.26992#bib.bib53),[19](https://arxiv.org/html/2610.26992#bib.bib17),[20](https://arxiv.org/html/2610.26992#bib.bib39)\]\. Consequently, slow clients become stragglers that postpone global updates and leave the resources of faster clients underutilized\. To address this issue, recent studies\[[21](https://arxiv.org/html/2610.26992#bib.bib9),[22](https://arxiv.org/html/2610.26992#bib.bib28),[23](https://arxiv.org/html/2610.26992#bib.bib52)\]have introduced asynchronous execution into SFL, allowing the server to continuously process available client updates without waiting for full synchronization\. Although asynchronous SFL effectively improves resource utilization and alleviates the straggler problem, it also fundamentally changes the optimization dynamics of SFL by introducing new consistency challenges\.

Specifically, asynchronous SFL suffers from two coupled inconsistency dimensions: horizontal inconsistency across clients and vertical inconsistency between clients and the server\. Horizontal inconsistency arises because heterogeneous computation and communication capabilities cause activations to arrive at different rates\. As the server updates the top model based on these arrivals, faster clients exert disproportionate influence on the optimization, and under non\-IID data this imbalance is further amplified, deviating the top model from the global objective and impairing convergence\[[24](https://arxiv.org/html/2610.26992#bib.bib54),[25](https://arxiv.org/html/2610.26992#bib.bib55)\]\. Vertical inconsistency arises because the server continuously updates the top model, while each bottom model is updated only through its own client interactions\. The resulting progress gaps distort split\-layer gradients and bottom model updates, and their accumulation further biases bottom model aggregation and impairs convergence\[[26](https://arxiv.org/html/2610.26992#bib.bib3),[27](https://arxiv.org/html/2610.26992#bib.bib5)\]\. More importantly, these two inconsistencies are not independent and may interact during asynchronous training: inconsistent activations impair the quality of server\-side optimization, whereas biased model aggregation may further amplify client drift among heterogeneous clients\. Therefore, jointly mitigating horizontal and vertical inconsistencies becomes a fundamental challenge for achieving efficient and accurate asynchronous SFL\.

Existing asynchronous SFL approaches only partially address these challenges\. Some studies primarily focus on improving execution efficiency through asynchronous parallelism\. For example, PipeSFL\[[21](https://arxiv.org/html/2610.26992#bib.bib9)\]introduces a fine\-grained pipeline execution mechanism to overlap client computation and server processing, thereby reducing idle time caused by stragglers\. However, it does not consider the inconsistency among asynchronously received intermediate activations\. GAS\[[22](https://arxiv.org/html/2610.26992#bib.bib28)\]further investigates asynchronous SFL under heterogeneous data distributions by generating intermediate activations to alleviate the impact of statistical heterogeneity, but its aggregation strategy cannot effectively handle model divergence caused by stale client updates\. Meanwhile, MergeSFL\[[28](https://arxiv.org/html/2610.26992#bib.bib2)\]improves robustness against heterogeneous data by merging intermediate activations from multiple clients, but it is designed for synchronous SFL and cannot fully exploit asynchronous execution\. Therefore, existing methods either improve system efficiency or alleviate heterogeneity issues, while lacking a unified framework that simultaneously addresses the coupled horizontal and vertical inconsistencies in asynchronous SFL\.

To address the above challenges, we propose CoASFL, an inconsistency\-aware asynchronous SFL framework that jointly suppresses inconsistency propagation during asynchronous training\. Instead of treating horizontal and vertical inconsistencies separately, CoASFL coordinates them through a unified consistency optimization framework\. Specifically, CoASFL integrates dynamic activation merging and consistency\-aware model aggregation into a coupled optimization strategy\. The dynamic activation merging coordinates activation merging at the server to improve the consistency of top model updates, while the consistency\-aware model aggregation regulates bottom model aggregation under unequal client participation and progress mismatch\. Rather than independently optimizing activation representations or aggregation reliability, the two mechanisms reinforce each other to preserve activation\-level and model\-level consistency throughout training\. As a result, CoASFL achieves both efficient convergence and high model accuracy under heterogeneous edge environments\. The main contributions of this paper are summarized as follows:

- •We identify the dual inconsistency problem in asynchronous SFL, including horizontal inconsistency across clients and vertical inconsistency between clients and the server, and analyze their impact on model convergence and accuracy\.
- •We propose CoASFL, an inconsistency\-aware asynchronous SFL framework that jointly coordinates activation\-level and model\-level consistency through dynamic activation merging and consistency\-aware model aggregation, effectively suppressing inconsistency propagation throughout asynchronous training\.
- •We implement CoASFL on an 80\-device heterogeneous edge testbed and conduct extensive experiments\. The results demonstrate that CoASFL accelerates training by 1\.61×\\timesto 3\.74×\\timesand improves final model accuracy by up to 3\.26% under heterogeneous scenarios, compared to state\-of\-the\-art SFL approaches\.

The rest of the paper is organized as follows\. Section[II](https://arxiv.org/html/2610.26992#S2)mainly introduces the impact of horizontal and vertical inconsistency\. Section[III](https://arxiv.org/html/2610.26992#S3)elaborates on the design of CoASFL\. Section[IV](https://arxiv.org/html/2610.26992#S4)presents a convergence analysis of CoASFL\. Section[V](https://arxiv.org/html/2610.26992#S5)discusses the experimental results\. Section[VI](https://arxiv.org/html/2610.26992#S6)introduces the related works\. Finally, Section[VII](https://arxiv.org/html/2610.26992#S7)concludes the paper\.

## IIBackground and Motivation

### II\-ASplit Federated Learning

Split federated learning \(SFL\) enhances FL by integrating split learning\[[10](https://arxiv.org/html/2610.26992#bib.bib1)\], providing an effective solution for training large\-scale models on resource\-constrained devices\. We consider a typical SFL system that consists of a parameter server \(PS\) andNNclients\.By default, all clients use the same fixed batch size for training\. The core concept of SFL is to introduce a split layer that divides the model𝒘\\boldsymbol\{w\}into a bottom model𝒘b\\boldsymbol\{w\}\_\{b\}retained by the clients and a top model𝒘p\\boldsymbol\{w\}\_\{p\}held by the PS,i\.e\.,𝒘=\[𝒘b,𝒘p\]\\boldsymbol\{w\}=\[\\boldsymbol\{w\}\_\{b\},\\boldsymbol\{w\}\_\{p\}\]\. The basic training process of the synchronous SFL requires cooperation between the clients and the PS\.

Let𝒘b,ih,k\\boldsymbol\{w\}^\{h,k\}\_\{b,i\}denote the bottom model of clientiiand𝒘ph,k\\boldsymbol\{w\}\_\{p\}^\{h,k\}denote the top model at iterationkkin roundhh\. Clientiiperforms forward propagation on𝒘b,ih,k\\boldsymbol\{w\}^\{h,k\}\_\{b,i\}using a local batch sampled from its datasetdid\_\{i\}, and then sends the resulting activations to the PS\. The empirical loss associated with the bottom model of clientiiis represented as:

Fb,i​\(𝒘b,ih,k\)=1\|di\|​∑x∈diℓ⁡\(x,𝒘b,ih,k,𝒘ph,k\)F\_\{b,i\}\(\\boldsymbol\{w\}\_\{b,i\}^\{h,k\}\)=\\frac\{1\}\{\|d\_\{i\}\|\}\\sum\_\{x\\in d\_\{i\}\}\\ell\(x;\\boldsymbol\{w\}\_\{b,i\}^\{h,k\},\\boldsymbol\{w\}\_\{p\}^\{h,k\}\)\(1\)
whereℓ⁡\(x,𝒘b,ih,k,𝒘ph,k\)\\ell\(x;\\boldsymbol\{w\}\_\{b,i\}^\{h,k\},\\boldsymbol\{w\}\_\{p\}^\{h,k\}\)denotes the loss function computed by the current bottom and top models using samplexx\. After receiving activations from all clients, the PS completes forward and backward propagation and updates the top model as:

𝒘ph,k\+1=𝒘ph,k−η⋅1N∑i=1N∇~Fp,i\(𝒘ph,k\)\\boldsymbol\{w\}\_\{p\}^\{h,k\+1\}=\\boldsymbol\{w\}\_\{p\}^\{h,k\}\-\\eta\\cdot\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\widetilde\{\\nabla\}F\_\{p,i\}\(\\boldsymbol\{w\}\_\{p\}^\{h,k\}\)\(2\)
whereFp,i​\(𝒘ph,k\)F\_\{p,i\}\(\\boldsymbol\{w\}\_\{p\}^\{h,k\}\)denotes the loss computed using the activations uploaded by clientii,∇~​Fp,i​\(𝒘ph,k\)\\widetilde\{\\nabla\}F\_\{p,i\}\(\\boldsymbol\{w\}\_\{p\}^\{h,k\}\)is its stochastic gradient with respect to the top model, andη\\etais the learning rate\. The PS then returns the corresponding split layer gradient to clientii, which updates its bottom model as:

𝒘b,ih,k\+1=𝒘b,ih,k−η⋅∇~​Fb,i​\(𝒘b,ih,k\)\\boldsymbol\{w\}\_\{b,i\}^\{h,k\+1\}=\\boldsymbol\{w\}\_\{b,i\}^\{h,k\}\-\\eta\\cdot\\widetilde\{\\nabla\}F\_\{b,i\}\(\\boldsymbol\{w\}\_\{b,i\}^\{h,k\}\)\(3\)where∇~​Fb,i​\(𝒘b,ih,k\)\\widetilde\{\\nabla\}F\_\{b,i\}\(\\boldsymbol\{w\}\_\{b,i\}^\{h,k\}\)is the stochastic gradient computed using the split layer gradient returned by the PS\. At the end of roundhh, the PS aggregates bottom models from all clients as:

𝒘bh\+1=∑i=1NAih​𝒘b,ih,Aih=nih∑j=1Nnjh\\boldsymbol\{w\}\_\{b\}^\{h\+1\}=\\sum\_\{i=1\}^\{N\}A\_\{i\}^\{h\}\\boldsymbol\{w\}\_\{b,i\}^\{h\},\\qquad A\_\{i\}^\{h\}=\\frac\{n\_\{i\}^\{h\}\}\{\\sum\_\{j=1\}^\{N\}n\_\{j\}^\{h\}\}\(4\)
wherenihn\_\{i\}^\{h\}denotes the number of samples processed by clientiiin roundhh,AihA\_\{i\}^\{h\}denotes its aggregation weight, and the aggregation reduces to uniform averaging when allnihn\_\{i\}^\{h\}are equal,i\.e\.,Aih=1/NA\_\{i\}^\{h\}=1/N\.

Unlike the synchronous SFL, asynchronous SFL removes the per\-iteration barrier\. The PS immediately processes each arriving activation and updates the top model without waiting for all participating clients\. Meanwhile, the client\-side bottom models are periodically aggregated according to Eq\. \([4](https://arxiv.org/html/2610.26992#S2.E4)\) to establish a shared global state\. We define the interval between two consecutive bottom model aggregations as an aggregation round\. Within each round, top model updates are performed asynchronously, whereas the bottom models are synchronized at the round boundary\. This execution pattern introduces inconsistency along two structural dimensions: horizontally across clients and vertically across the clients and server\.

\(a\)Label divergence at the PS\.\(b\)Top\-update direction deviation\.
Fig\. 1:Impact of horizontal inconsistency\.
### II\-BCharacterization of Horizontal Inconsistency

In asynchronous SFL, heterogeneous client computation and communication capabilities cause activations to arrive at the PS at different rates\. Since each arriving activation immediately triggers a top model update, faster clients participate in substantially more updates than slower ones, leading tohorizontal inconsistency\. Under IID data, this inconsistency mainly changes the relative contribution of clients\. Under non\-IID data, however, repeatedly arriving activations from the same clients skew the effective activation and label distributions observed by the PS, causing the optimization direction of the top model to gradually deviate from the global objective\.

To characterize horizontal inconsistency, we conduct a controlled experiment using DeiT on CIFAR\-100\. The model is split at the 6th layer and trained with 20 clients under both IID and non\-IID settings\. Since unequal client influence is directly reflected by their participation frequencies, we use participation imbalance as the measurable indicator in this experiment\. Letτih\\tau\_\{i\}^\{h\}denote the number of top model updates involving clientiiduring aggregation roundhh, and quantify participation imbalance as:

ℐτh=1−\(∑i=1Nτih\)2N​∑i=1N\(τih\)2\\mathcal\{I\}\_\{\\tau\}^\{h\}=1\-\\frac\{\\left\(\\sum\_\{i=1\}^\{N\}\\tau\_\{i\}^\{h\}\\right\)^\{2\}\}\{N\\sum\_\{i=1\}^\{N\}\\left\(\\tau\_\{i\}^\{h\}\\right\)^\{2\}\}\(5\)Here,ℐτh=0\\mathcal\{I\}\_\{\\tau\}^\{h\}=0indicates balanced participation, whereas a larger value represents a more imbalanced participation pattern in this controlled experiment\. To assess whether the top model learns evenly from different client contributions, we examine the divergence between the label distribution in activations at the PS and that under balanced client participation\. To quantify the bias in top model learning direction, we measure the deviation from the balanced top model update direction\. Meanwhile, we keep the model state, sampled batches, and client\-specific update directions unchanged, so the measured differences are caused solely by unequal client participation\.

As shown in Fig\.[1a](https://arxiv.org/html/2610.26992#S2.F1.sf1), asℐτh\\mathcal\{I\}\_\{\\tau\}^\{h\}increases, the distribution of labels associated with the activations processed by the PS increasingly deviates from the reference distribution under balanced client participation\. Under the same participation imbalance, non\-IID data produce nearly twice the divergence of IID data, indicating that statistical heterogeneity amplifies the distribution bias\. Fig\.[1b](https://arxiv.org/html/2610.26992#S2.F1.sf2)further shows that the update direction deviation of the top model increases from nearly zero under balanced participation to approximately23%23\\%and26%26\\%at the highest imbalance level for IID and non\-IID data, respectively\. These results demonstrate that unequal client participation systematically biases data distribution and update direction toward frequently participating clients,i\.e\., horizontal inconsistency\. As they accumulate, uneven learning across clients and biased update directions slow model convergence and reduce final accuracy, especially under non\-IID data\. Therefore, the server should coordinate arriving activations to preserve activation diversity instead of updating the top model solely according to arrival order\.

\(a\)Split layer gradient deviation\.\(b\)Model aggregation deviation\.
Fig\. 2:Impact of vertical inconsistency\.
### II\-CCharacterization of Vertical Inconsistency

While horizontal inconsistency biases server\-side optimization, asynchronous SFL also introducesvertical inconsistencybetween the bottom model on each client and the top model on the PS\. The top model advances whenever an activation is processed, whereas each bottom model progresses only through its own client interactions\. Consequently, different clients may exhibit different progress gaps between their bottom models and the current top model\. As this progress gap increases, the gradient exchanged at the split layer deviates more from that under aligned model partitions, further affecting the corresponding bottom model update\. Aggregating bottom models updated under different partition states may therefore bias the global bottom model update direction\.

Using the same experimental setup as Sec\.[II\-B](https://arxiv.org/html/2610.26992#S2.SS2), for each evaluated clientiiparticipating in roundhh, we define its average interaction progress gap as:

ζih=1τih​∑k=1τih\(ri,kh−k\)\\zeta\_\{i\}^\{h\}=\\frac\{1\}\{\\tau\_\{i\}^\{h\}\}\\sum\_\{k=1\}^\{\\tau\_\{i\}^\{h\}\}\\left\(r\_\{i,k\}^\{h\}\-k\\right\)\(6\)whereri,khr\_\{i,k\}^\{h\}denotes the top model update index corresponding to thekk\-th bottom model update of clientii, and a largerζih\\zeta\_\{i\}^\{h\}indicates that its bottom model updates are on average performed under a greater progress mismatch between the two model partitions\. To consistently quantify vertical inconsistency, we define its level as the average progress gap over the set𝒰\\mathcal\{U\}of evaluated bottom models :

ℐζh=1\|𝒰\|​∑i∈𝒰ζih\\mathcal\{I\}\_\{\\zeta\}^\{h\}=\\frac\{1\}\{\|\\mathcal\{U\}\|\}\\sum\_\{i\\in\\mathcal\{U\}\}\\zeta\_\{i\}^\{h\}\(7\)whereℐζh=0\\mathcal\{I\}\_\{\\zeta\}^\{h\}=0corresponds to fully aligned model partitions\. To characterize vertical inconsistency, we measure the split layer gradient deviation and aggregation direction deviation, which quantify the difference in update direction and gradient mismatch of the bottom model, respectively\. We varyℐζh\\mathcal\{I\}\_\{\\zeta\}^\{h\}over\{0,1,2,4,8\}\\\{0,1,2,4,8\\\}\. For the split layer gradient experiment, the same input batch is processed using bottom model states corresponding to different progress gaps, while keeping the current top model and labels fixed\. For the aggregation experiment, client groups are assigned evenly spaced progress gaps whose mean equalsℐζh\\mathcal\{I\}\_\{\\zeta\}^\{h\}, and the resulting aggregated update direction is compared with that under fully aligned model partitions\.

As shown in Fig\.[2a](https://arxiv.org/html/2610.26992#S2.F2.sf1), the split layer gradient deviation consistently increases withℐζh\\mathcal\{I\}\_\{\\zeta\}^\{h\}, reaching approximately26%26\\%under IID data and30%30\\%under non\-IID data whenℐζh=8\\mathcal\{I\}\_\{\\zeta\}^\{h\}=8\. This indicates that a larger progress mismatch causes the returned gradient, and hence the bottom model update direction, to deviate further from that under aligned model partitions\. Fig\.[2b](https://arxiv.org/html/2610.26992#S2.F2.sf2)further shows that such gradient mismatch propagates to bottom model aggregation, where the aggregation direction deviation reaches approximately29%29\\%and33%33\\%under IID and non\-IID data, respectively\. The consistently larger deviations under non\-IID data indicate that statistical heterogeneity amplifies the impact of vertical inconsistency\. As these inconsistencies accumulate across successive updates and aggregation rounds, they can cause the global model to deviate from the desired optimization trajectory, thereby slowing convergence and reducing final accuracy\.

### II\-DDiscussion

Although horizontal and vertical inconsistencies are analyzed separately above, they are tightly coupled during asynchronous SFL\. Horizontal inconsistency biases the optimization direction of the top model, while the resulting server\-side model evolution further enlarges the progress mismatch experienced by delayed clients\. In turn, bottom models updated under different partition states generate less consistent activations in subsequent rounds, further amplifying the server\-side optimization bias caused by unequal client influence\. Therefore, mitigating only one type of inconsistency cannot fundamentally stabilize asynchronous SFL\. Instead, activation\-level consistency and model\-level consistency should be jointly optimized in a unified framework\. Motivated by this observation, we propose CoASFL, which integrates dynamic activation merging and consistency\-aware model aggregation into a coupled consistency optimization framework\.

## IIISystem Design

![Refer to caption](https://arxiv.org/html/2610.26992v1/2_AsySFL_overview_20260730.png)Fig\. 3:Overview of CoASFL### III\-AOverview

As illustrated in Fig\.[3](https://arxiv.org/html/2610.26992#S3.F3), CoASFL consists of a control module and a training module that collaboratively optimize asynchronous split federated learning\. Specifically, the PS continuously monitors the training state of each client, including its participation frequency, progress gap, and runtime status throughout the training process \(1\)\. Based on the maintained client states, the PS determines a set of compatible activations from the activation buffer before each top model update \(2\)\. Then the selected activations are merged to optimize the top model \(3\), and the corresponding split layer gradients are dispatched to the associated clients for bottom model optimization \(4\)\. Meanwhile, the monitored client states are continuously updated and used to determine the contribution of each client during model synchronization \(5\)\. Finally, the PS aggregates the bottom models using the configured weights and broadcasts the updated bottom model to all clients, initiating the next aggregation round \(6\)\. By jointly coordinating activation scheduling during top model optimization and adaptive aggregation during bottom model synchronization, CoASFL simultaneously mitigates horizontal and vertical inconsistencies in asynchronous SFL\.

### III\-BControl Module

During each aggregation round, the control module of the PS monitors client states and makes the decisions required by the training module\. Specifically, it determines the activation set for each top model update and configures the bottom model aggregation weights at the round boundary\.

#### III\-B1Client State Monitoring

The control module maintains the client information required for subsequent control decisions\. Specifically, the label distributions and computation and communication capabilities of all clients are collected to support dynamic activation selection\. Meanwhile, their participation frequencies and progress gaps are continuously maintained during each round for bottom model aggregation weight configuration at the round boundary\. Let𝐕=\{vj≥0,j∈\[1,M\]\}\\mathbf\{V\}=\\\{v\_\{j\}\\geq 0,j\\in\[1,M\]\\\}denote a categorical distribution of class labels overMMclasses, where∑j=1Mvj=1\\sum\_\{j=1\}^\{M\}v\_\{j\}=1\. Since labels are transmitted alongside activations for loss computation in conventional SFL\[[10](https://arxiv.org/html/2610.26992#bib.bib1)\], the PS collects the empirical label distribution𝐕i\\mathbf\{V\}\_\{i\}of each clientiionce before training and uses it to evaluate whether a client combination approaches the balanced label distribution\. Considering that privacy leakage is an important challenge in SFL, popular privacy protection techniques,e\.g\., Differential Privacy\[[29](https://arxiv.org/html/2610.26992#bib.bib27),[30](https://arxiv.org/html/2610.26992#bib.bib47)\], Distance Correlation\[[31](https://arxiv.org/html/2610.26992#bib.bib48)\]and Homomorphic Encryption\[[32](https://arxiv.org/html/2610.26992#bib.bib49)\], can be applied to protect the raw data, activations/gradients and models, and are orthogonal to the main focus of CoASFL\.

The time\-varying computation and communication states are used to estimate how long a buffered activation may wait for a suitable combination\. As proxy metrics, we adopt the computing time of one iteration asμih\\mu^\{h\}\_\{i\}and the corresponding transmission timeβih\\beta^\{h\}\_\{i\}, which can be recorded by the clients directly during model training, to indicate the computing and communication capabilities of clientiiin roundhh, respectively\. Before starting model training in roundhh, the PS collects the latest computing timeμ^ih\\hat\{\\mu\}^\{h\}\_\{i\}and transmission timeβ^ih\\hat\{\\beta\}^\{h\}\_\{i\}from all clients\. Additionally, we utilize the moving average with historical client states to enhance the robustness of the estimation\[[33](https://arxiv.org/html/2610.26992#bib.bib29)\]\. Accordingly, the PS estimates the computing timeμih\\mu^\{h\}\_\{i\}and corresponding transmission timeβih\\beta^\{h\}\_\{i\}for clientiiin roundhhby calculating the moving average withα∈\[0,1\]\\alpha\\in\[0,1\]\(e\.g\.,0\.80\.8in our experiments\) as:

μih=α⋅μih−1\+\(1−α\)⋅μ^ih\\displaystyle\\mu^\{h\}\_\{i\}=\\alpha\\cdot\\mu^\{h\-1\}\_\{i\}\+\(1\-\\alpha\)\\cdot\\hat\{\\mu\}^\{h\}\_\{i\}\(8\)βih=α⋅βih−1\+\(1−α\)⋅β^ih\\displaystyle\\beta^\{h\}\_\{i\}=\\alpha\\cdot\\beta^\{h\-1\}\_\{i\}\+\(1\-\\alpha\)\\cdot\\hat\{\\beta\}^\{h\}\_\{i\}\(9\)Besides, it is worth noting that advancing the techniques for state estimation is not our focus, and other existing advanced estimation techniques\[[34](https://arxiv.org/html/2610.26992#bib.bib30),[35](https://arxiv.org/html/2610.26992#bib.bib31)\]can be applied\.

The training statesτih\\tau\_\{i\}^\{h\}andζih\\zeta\_\{i\}^\{h\}are continuously updated throughout each aggregation round\. The participation frequencyτih\\tau\_\{i\}^\{h\}guides dynamic activation selection toward less frequently participating clients, whileτih\\tau\_\{i\}^\{h\}andζih\\zeta\_\{i\}^\{h\}are jointly used to configure the bottom model aggregation weights at the round boundary\.

#### III\-B2Dynamic Activation Selection

As demonstrated in Sec\.[II\-B](https://arxiv.org/html/2610.26992#S2.SS2), updating the top model strictly according to activation arrival order biases the training sequence toward frequently participating clients\. To alleviate this bias, the PS buffers arriving activations and selects a subset before each top model update according to their label distributions and participation frequencies\. LetΘr\\Theta^\{r\}denote the set of clients whose activations are pending before therr\-th top model update\. Before each top model update, the control module selects a client subsetSr⊆ΘrS^\{r\}\\subseteq\\Theta^\{r\}from the buffered activations\. Exhaustively searching the subsets ofΘr\\Theta^\{r\}online for each update is, however, computationally prohibitive\. We therefore pre\-construct valid candidate combinations offline from the client label distributions before training\.

To reduce the label\-distribution bias under non\-IID data, each candidate combination should represent a label distribution close to that under balanced client participation\. Because all clients use the same fixed batch size, we define the balanced label distribution asΦ0=1N​∑i=1N𝐕i\\Phi\_\{0\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbf\{V\}\_\{i\}\. For a client combinationC⊆\{1,…,N\}C\\subseteq\\\{1,\\ldots,N\\\}, its label distribution isΦC=1\|C\|​∑i∈C𝐕i\\Phi\_\{C\}=\\frac\{1\}\{\|C\|\}\\sum\_\{i\\in C\}\\mathbf\{V\}\_\{i\}\. We use the KL divergence to measure the gap betweenΦC\\Phi\_\{C\}andΦ0\\Phi\_\{0\}as follows\[[36](https://arxiv.org/html/2610.26992#bib.bib11),[37](https://arxiv.org/html/2610.26992#bib.bib12)\]:

KL\(ΦC∥Φ0\)=∑j=1MΦC\(vj\)logΦC​\(vj\)Φ0​\(vj\)\\mathrm\{KL\}\(\\Phi\_\{C\}\\\|\\Phi\_\{0\}\)=\\sum\_\{j=1\}^\{M\}\\Phi\_\{C\}\(v\_\{j\}\)\\log\\frac\{\\Phi\_\{C\}\(v\_\{j\}\)\}\{\\Phi\_\{0\}\(v\_\{j\}\)\}\(10\)A smaller value indicates that the activations fromCCrepresent a label distribution closer to that under balanced client participation\. Consequently, a combination is regarded as valid if it satisfies the constraints:

\{KL\(ΦC∥Φ0\)≤ϵ\|C\|≤Θ0\\left\\\{\\begin\{array\}\[\]\{l\}\\mathrm\{KL\}\(\\Phi\_\{C\}\\\|\\Phi\_\{0\}\)\\leq\\epsilon\\\\ \|C\|\\leq\\Theta\_\{0\}\\end\{array\}\\right\.\(11\)whereϵ≥0\\epsilon\\geq 0is a predefined distribution threshold, andΘ0\\Theta\_\{0\}denotes the activation buffer capacity\. We setϵ=0\.25\\epsilon=0\.25andΘ0=N/5\\Theta\_\{0\}=N/5in our experiments, under which each client belongs to at least one valid candidate combination\.

Given the exponential growth of possible client combinations with respect toNN, we adopt a greedy algorithm to construct the valid candidate set𝒴\\mathcal\{Y\}\. Starting from each client as a seed, the procedure iteratively adds the unselected client that yields the smallest KL divergence after expansion\. Each intermediate combination satisfying Eq\. \([11](https://arxiv.org/html/2610.26992#S3.E11)\) is retained, and the expansion process terminates once its size reachesΘ0\\Theta\_\{0\}\. Each seed requires at mostΘ0−1\\Theta\_\{0\}\-1successive expansions, yielding an overall construction complexity ofO⁡\(N2​Θ0​M\)O\(N^\{2\}\\Theta\_\{0\}M\)rather than the exponential cost of exhaustive enumeration\. This construction is performed only once before training and produces at mostN​Θ0N\\Theta\_\{0\}candidate combinations, which are then reused during online activation selection\.

During online training, the PS obtains the available candidate set𝒜r=\{C∈𝒴∣C⊆Θr\}\\mathcal\{A\}^\{r\}=\\\{C\\in\\mathcal\{Y\}\\mid C\\subseteq\\Theta^\{r\}\\\}from the buffered activations\. When𝒜r≠∅\\mathcal\{A\}^\{r\}\\neq\\emptyset, the control module selects a combinationSrS^\{r\}by minimizing the average participation frequency:

Sr=arg⁡minC∈𝒜r​1\|C\|​∑i∈CτihS^\{r\}=\\arg\\min\_\{C\\in\\mathcal\{A\}^\{r\}\}\\frac\{1\}\{\|C\|\}\\sum\_\{i\\in C\}\\tau\_\{i\}^\{h\}\(12\)This objective prioritizes combinations containing clients with lower participation frequencies, thereby progressively balancing client influence during the round, with ties broken by the smaller KL divergence\. When no candidate is available, the buffered activations are retained for subsequent matching\.

However, this matching process may cause an activation to remain buffered for an extended duration when the other members of its candidate combinations have yet to arrive\. To prevent indefinite buffering, we introduce a client\-specific timeout threshold derived from the estimated computation and communication states\. For clientiiand a valid combinationC∈𝒴C\\in\\mathcal\{Y\}containingii, the expected matching delay is the difference between the latest expected arrival time among the members ofCCand that of clientiiitself,i\.e\.,maxj∈C⁡\(μjh\+βjh\)−\(μih\+βih\)\\max\_\{j\\in C\}\(\\mu\_\{j\}^\{h\}\+\\beta\_\{j\}^\{h\}\)\-\(\\mu\_\{i\}^\{h\}\+\\beta\_\{i\}^\{h\}\)\. Following the practice of percentile\-based tail\-latency control in large\-scale systems\[[38](https://arxiv.org/html/2610.26992#bib.bib40)\], we set the timeout thresholdλih\\lambda\_\{i\}^\{h\}to the P90 value of these delays across allC∈𝒴C\\in\\mathcal\{Y\}withi∈Ci\\in C, and update it once per aggregation round\. Letqirq\_\{i\}^\{r\}denote the elapsed waiting time of the pending activation from clientii\. When one or more activations exceed their thresholds, the client with the largest timeout excess is identified asi⋆i^\{\\star\}according to:

i⋆=argmaxi:qir≥λih\(qir−λih\)i^\{\\star\}=\\arg\\max\_\{i:q\_\{i\}^\{r\}\\geq\\lambda\_\{i\}^\{h\}\}\(q\_\{i\}^\{r\}\-\\lambda\_\{i\}^\{h\}\)\(13\)The PS then restricts the candidate set to those containingi⋆i^\{\\star\}\. The control module applies Eq\. \([12](https://arxiv.org/html/2610.26992#S3.E12)\) to the remaining candidates\. If no available candidate containsi⋆i^\{\\star\}, it greedily constructs a combination from the pending activations starting with\{i⋆\}\\\{i^\{\\star\}\\\}, each time adding the client that yields the smallest KL divergence until a valid combination is formed or the combination size reachesΘ0\\Theta\_\{0\}\. The expansion terminates when a valid combination is formed\. If validity remains unattainable, the intermediate combination with the smallest KL divergence is selected\. When the buffer reaches capacity without any valid candidate and no timeout has occurred, the longest\-waiting client is treated asi⋆i^\{\\star\}and processed through the same greedy construction procedure\.

#### III\-B3Aggregation Weight Configuration

At the end of aggregation roundhh, the bottom models of different clients have incorporated different numbers of updates, and these updates have experienced different average progress gaps\. With the same fixed batch size, the conventional sample\-size\-based aggregation weightAihA\_\{i\}^\{h\}is proportional to the participation frequencyτih\\tau\_\{i\}^\{h\}\. Although this weight reflects the amount of training completed by each client, it does not correct the contribution imbalance caused by unequal participation or account for the update deviation induced by cross\-partition progress mismatch\. The control module thus reconfigures the aggregation weights using bothτih\\tau\_\{i\}^\{h\}andζih\\zeta\_\{i\}^\{h\}\.

The two states characterize complementary properties of a bottom model\. Specifically,τih\\tau\_\{i\}^\{h\}reflects the number of updates incorporated into the bottom model, whereasζih\\zeta\_\{i\}^\{h\}reflects the average progress mismatch experienced during these updates\. Therefore, the correction should not compensate low\-participation clients indiscriminately\. Instead, participation compensation should be conditioned on the consistency of the updates incorporated into each bottom model,i\.e\., a client with fewer but better\-aligned updates receives stronger compensation, whereas a client whose updates experience a large progress mismatch should be discounted even if frequency is low\. We adopt a multiplicative form, which compresses the contribution disparity from unequal participation, decreases with the average progress gap, and reduces to one under balanced participation and fully aligned model partitions\.

Letτ¯h=1N​∑j=1Nτjh\\bar\{\\tau\}^\{h\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\\tau\_\{j\}^\{h\}denote the average participation frequency, and letRhR^\{h\}denote the total number of top model updates in roundhh\. The correction factor for clientiiis configured as:

ξih=τ¯h\+1τih\+1⋅11\+ζih/Rh\\xi\_\{i\}^\{h\}=\\frac\{\\bar\{\\tau\}^\{h\}\+1\}\{\\tau\_\{i\}^\{h\}\+1\}\\cdot\\frac\{1\}\{1\+\\zeta\_\{i\}^\{h\}/R^\{h\}\}\(14\)The first term compensates the contribution loss caused by low participation relative to the round average\. The second term acts as a consistency gate that weakens this compensation as the progress mismatch increases\. Their multiplicative combination therefore jointly considers the horizontal contribution imbalance and the vertical reliability of bottom model updates\. Normalizingζih\\zeta\_\{i\}^\{h\}byRhR^\{h\}makes the correction comparable across rounds with different numbers of top model updates\. The additive constant in the participation term avoids division by zero and smooths the correction for small participation counts, while the leading one in the progress\-gap term keeps the corresponding factor neutral whenζih=0\\zeta\_\{i\}^\{h\}=0\.

The final aggregation weight is obtained by applying the correction factor to the base weight and normalizing the results across all clients, ensuring that the configured weights remain a valid convex combination:

A~ih=Aih​ξih∑j=1NAjh​ξjh\\widetilde\{A\}\_\{i\}^\{h\}=\\frac\{A\_\{i\}^\{h\}\\xi\_\{i\}^\{h\}\}\{\\sum\_\{j=1\}^\{N\}A\_\{j\}^\{h\}\\xi\_\{j\}^\{h\}\}\(15\)This normalization guaranteesA~ih≥0\\widetilde\{A\}\_\{i\}^\{h\}\\geq 0and∑i=1NA~ih=1\\sum\_\{i=1\}^\{N\}\\widetilde\{A\}\_\{i\}^\{h\}=1\. When all clients have the same participation frequency and their bottom model updates experience no progress mismatch,ξih=1\\xi\_\{i\}^\{h\}=1for all clients, and the configured weights reduce to the conventional aggregation weights\.

### III\-CTraining Module

The training module executes model training and bottom model synchronization according to the decisions made by the control module\. It consists of activation merging, gradient dispatching, and bottom model aggregation and distribution\.

#### III\-C1Activation Merging

For therr\-th top model update, once the control module determines the activation setSrS^\{r\}, the PS retrieves the corresponding activations from the buffer and merges them into a mixed activation sequenceGr=\[gir∣i∈Sr\]G^\{r\}=\[g\_\{i\}^\{r\}\\mid i\\in S^\{r\}\]\. The PS then performs forward and backward propagation over𝑮r\\boldsymbol\{G\}^\{r\}and updates the top model as follows:

𝒘pr\+1=𝒘pr−η​∇~​Fp​\(𝒘pr,Gr\)\\boldsymbol\{w\}\_\{p\}^\{r\+1\}=\\boldsymbol\{w\}\_\{p\}^\{r\}\-\\eta\\widetilde\{\\nabla\}F\_\{p\}\(\\boldsymbol\{w\}\_\{p\}^\{r\};G^\{r\}\)\(16\)where∇~​Fp​\(𝒘pr,Gr\)\\widetilde\{\\nabla\}F\_\{p\}\(\\boldsymbol\{w\}\_\{p\}^\{r\};G^\{r\}\)denotes the gradient averaged over the samples represented byGrG^\{r\}\.

#### III\-C2Gradient Dispatching

After backward propagation, the PS obtains the mixed split layer gradientG^r\\widehat\{G\}^\{r\}corresponding toGrG^\{r\}\. Based on the client\-wise composition ofGrG^\{r\}, the PS separates it asG^r=\[g^ir∣i∈Sr\]\\widehat\{G\}^\{r\}=\[\\widehat\{g\}\_\{i\}^\{r\}\\mid i\\in S^\{r\}\]\. It then dispatches each separated gradientg^ir\\widehat\{g\}\_\{i\}^\{r\}to the corresponding client\. Upon receiving the gradient, clientiicompletes backward propagation and updates its bottom model according to Eq\. \([3](https://arxiv.org/html/2610.26992#S2.E3)\)\.

#### III\-C3Bottom Model Aggregation

Aggregation roundhhterminates once every client has completed at least one update and the predefined average local update frequency is reached\. After all pending gradients have been applied to the corresponding bottom models, the PS collects and aggregates them using the weights configured by the control module:

𝒘bh\+1=∑i=1NA~ih​𝒘b,ih\\boldsymbol\{w\}\_\{b\}^\{h\+1\}=\\sum\_\{i=1\}^\{N\}\\widetilde\{A\}\_\{i\}^\{h\}\\boldsymbol\{w\}\_\{b,i\}^\{h\}\(17\)The PS subsequently distributes𝒘bh\+1\\boldsymbol\{w\}\_\{b\}^\{h\+1\}to all clients for the next aggregation round, or be combined with the top model to form a complete model\.

## IVConvergence Analysis

Since the training objective is generally nonconvex, we analyze the mean\-square gradients of the bottom and top models overHHaggregation rounds to characterize their convergence behavior\. Following existing convergence analyses\[[7](https://arxiv.org/html/2610.26992#bib.bib7),[39](https://arxiv.org/html/2610.26992#bib.bib8),[40](https://arxiv.org/html/2610.26992#bib.bib56),[18](https://arxiv.org/html/2610.26992#bib.bib53)\], we adopt three standard assumptions\.

###### Assumption 1

\(Lipschitz Continuous Gradient\)\. The balanced objectiveFFand the bottom and top model gradients areLL\-smooth, satisfying the following inequalities:

‖∇F​\(𝒙\)−∇F​\(𝒚\)‖\\displaystyle\\\|\\nabla F\(\\boldsymbol\{x\}\)\-\\nabla F\(\\boldsymbol\{y\}\)\\\|≤L​‖𝒙−𝒚‖\\displaystyle\\leq L\\\|\\boldsymbol\{x\}\-\\boldsymbol\{y\}\\\|\(18\)‖∇Fb,i​\(𝒙\)−∇Fb,i​\(𝒚\)‖\\displaystyle\\\|\\nabla F\_\{b,i\}\(\\boldsymbol\{x\}\)\-\\nabla F\_\{b,i\}\(\\boldsymbol\{y\}\)\\\|≤L​‖𝒙−𝒚‖\\displaystyle\\leq L\\\|\\boldsymbol\{x\}\-\\boldsymbol\{y\}\\\|‖∇Fp,i​\(𝒙\)−∇Fp,i​\(𝒚\)‖\\displaystyle\\\|\\nabla F\_\{p,i\}\(\\boldsymbol\{x\}\)\-\\nabla F\_\{p,i\}\(\\boldsymbol\{y\}\)\\\|≤L​‖𝒙−𝒚‖\\displaystyle\\leq L\\\|\\boldsymbol\{x\}\-\\boldsymbol\{y\}\\\|

###### Assumption 2

\(Unbiased Local Gradient Estimator\)\. The stochastic gradients are unbiased estimators of the corresponding bottom and top model gradients, expressed as follows:

𝔼⁡\[∇~​Fb,i​\(𝒘\)\]\\displaystyle\\mathbb\{E\}\[\\widetilde\{\\nabla\}F\_\{b,i\}\(\\boldsymbol\{w\}\)\]=∇Fb,i​\(𝒘\)\\displaystyle=\\nabla F\_\{b,i\}\(\\boldsymbol\{w\}\)\(19\)𝔼⁡\[∇~​Fp,i​\(𝒘\)\]\\displaystyle\\mathbb\{E\}\[\\widetilde\{\\nabla\}F\_\{p,i\}\(\\boldsymbol\{w\}\)\]=∇Fp,i​\(𝒘\)\\displaystyle=\\nabla F\_\{p,i\}\(\\boldsymbol\{w\}\)

###### Assumption 3

\(Bounded Gradient Second Moment\)\. The stochastic gradients of both model partitions have bounded second moments, satisfying:

𝔼​‖∇~​Fb,i​\(𝒘\)‖2\\displaystyle\\mathbb\{E\}\\\|\\widetilde\{\\nabla\}F\_\{b,i\}\(\\boldsymbol\{w\}\)\\\|^\{2\}≤σb2\\displaystyle\\leq\\sigma\_\{b\}^\{2\}\(20\)𝔼​‖∇~​Fp,i​\(𝒘\)‖2\\displaystyle\\mathbb\{E\}\\\|\\widetilde\{\\nabla\}F\_\{p,i\}\(\\boldsymbol\{w\}\)\\\|^\{2\}≤σp2\\displaystyle\\leq\\sigma\_\{p\}^\{2\}

Let𝒘h=\[𝒘bh,𝒘ph,0\]\\boldsymbol\{w\}^\{h\}=\[\\boldsymbol\{w\}\_\{b\}^\{h\},\\boldsymbol\{w\}\_\{p\}^\{h,0\}\]denote the joint model state at the beginning of roundhh, and letSh,rS^\{h,r\}denote the activation set selected for itsrr\-th top model update\. According to Eqs\. \([14](https://arxiv.org/html/2610.26992#S3.E14)\)–\([17](https://arxiv.org/html/2610.26992#S3.E17)\), the normalized gradient contributions can be expressed as follows:

πih\\displaystyle\\pi\_\{i\}^\{h\}=1Rh​∑r=1Rh𝟏\{i∈Sh,r\}\|Sh,r\|\\displaystyle=\\frac\{1\}\{R^\{h\}\}\\sum\_\{r=1\}^\{R^\{h\}\}\\frac\{\\mathbf\{1\}\\\{i\\in S^\{h,r\}\\\}\}\{\|S^\{h,r\}\|\}\(21\)γh\\displaystyle\\gamma^\{h\}=∑i=1NA~ih​τih\\displaystyle=\\sum\_\{i=1\}^\{N\}\\widetilde\{A\}\_\{i\}^\{h\}\\tau\_\{i\}^\{h\}ρih\\displaystyle\\rho\_\{i\}^\{h\}=A~ih​τihγh\\displaystyle=\\frac\{\\widetilde\{A\}\_\{i\}^\{h\}\\tau\_\{i\}^\{h\}\}\{\\gamma^\{h\}\}Here,∑iπih=∑iρih=1\\sum\_\{i\}\\pi\_\{i\}^\{h\}=\\sum\_\{i\}\\rho\_\{i\}^\{h\}=1\. Let𝔼h\\mathbb\{E\}\_\{h\}denote expectation conditioned on𝒘h\\boldsymbol\{w\}^\{h\}and the execution schedule, taken to be independent of stochastic gradient sampling\.

###### Lemma 1

Under Assumptions[1](https://arxiv.org/html/2610.26992#Thmassumption1)–[3](https://arxiv.org/html/2610.26992#Thmassumption3), the expected update direction satisfies the following inequality:

𝔼h​⟨∇F​\(𝒘h\),𝒅h⟩\\displaystyle\\mathbb\{E\}\_\{h\}\\langle\\nabla F\(\\boldsymbol\{w\}^\{h\}\),\\boldsymbol\{d\}^\{h\}\\rangle\(22\)≥γh2​\[‖∇Fb​\(𝒘h\)‖2−\(Bbh\)2\]\\displaystyle\\geq\\frac\{\\gamma^\{h\}\}\{2\}\\Big\[\\\|\\nabla F\_\{b\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\-\(B\_\{b\}^\{h\}\)^\{2\}\\Big\]\+Rh2​\[‖∇Fp​\(𝒘h\)‖2−\(Bph\)2\]\\displaystyle\+\\frac\{R^\{h\}\}\{2\}\\Big\[\\\|\\nabla F\_\{p\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\-\(B\_\{p\}^\{h\}\)^\{2\}\\Big\]where𝐝h=\(𝐰h−𝐰h\+1\)/η\\boldsymbol\{d\}^\{h\}=\(\\boldsymbol\{w\}^\{h\}\-\\boldsymbol\{w\}^\{h\+1\}\)/\\eta, and∇Fb=N−1​∑i∇Fb,i\\nabla F\_\{b\}=N^\{\-1\}\\sum\_\{i\}\\nabla F\_\{b,i\}and∇Fp=N−1​∑i∇Fp,i\\nabla F\_\{p\}=N^\{\-1\}\\sum\_\{i\}\\nabla F\_\{p,i\}denote the two components of∇F\\nabla F\. The corresponding gradient deviation bounds are defined as follows:

Bph=\\displaystyle B\_\{p\}^\{h\}=\{\}‖∑i\(πih−1N\)∇Fp,i\(𝒘h\)‖\\displaystyle\\left\\\|\\sum\_\{i\}\\left\(\\pi\_\{i\}^\{h\}\-\\frac\{1\}\{N\}\\right\)\\nabla F\_\{p,i\}\(\\boldsymbol\{w\}^\{h\}\)\\right\\\|\(23\)\+L​η​\[σb​∑iπih​\(τih−1\)\+σp​\(Rh−1\)2\]\\displaystyle\+L\\eta\\left\[\\sigma\_\{b\}\\sum\_\{i\}\\pi\_\{i\}^\{h\}\(\\tau\_\{i\}^\{h\}\-1\)\+\\frac\{\\sigma\_\{p\}\(R^\{h\}\-1\)\}\{2\}\\right\]Bbh=\\displaystyle B\_\{b\}^\{h\}=\{\}‖∑i\(ρih−1N\)∇Fb,i\(𝒘h\)‖\\displaystyle\\left\\\|\\sum\_\{i\}\\left\(\\rho\_\{i\}^\{h\}\-\\frac\{1\}\{N\}\\right\)\\nabla F\_\{b,i\}\(\\boldsymbol\{w\}^\{h\}\)\\right\\\|\+L​η​\[σp​∑iρih​ζih\+σb\+σp2​∑iρih​\(τih−1\)\]\\displaystyle\+L\\eta\\left\[\\sigma\_\{p\}\\sum\_\{i\}\\rho\_\{i\}^\{h\}\\zeta\_\{i\}^\{h\}\+\\frac\{\\sigma\_\{b\}\+\\sigma\_\{p\}\}\{2\}\\sum\_\{i\}\\rho\_\{i\}^\{h\}\(\\tau\_\{i\}^\{h\}\-1\)\\right\]

###### Proof:

According to Eqs\. \([3](https://arxiv.org/html/2610.26992#S2.E3)\), \([16](https://arxiv.org/html/2610.26992#S3.E16)\), and \([17](https://arxiv.org/html/2610.26992#S3.E17)\), the aggregated update direction can be expressed as follows:

𝒅h\\displaystyle\\boldsymbol\{d\}^\{h\}=\[γh​𝒈¯bh,Rh​𝒈¯ph\]\\displaystyle=\\big\[\\gamma^\{h\}\\overline\{\\boldsymbol\{g\}\}\_\{b\}^\{h\},R^\{h\}\\overline\{\\boldsymbol\{g\}\}\_\{p\}^\{h\}\\big\]\(24\)𝒈¯bh\\displaystyle\\overline\{\\boldsymbol\{g\}\}\_\{b\}^\{h\}=∑iρihτih​∑k=1τih𝒈~b,ih,k\\displaystyle=\\sum\_\{i\}\\frac\{\\rho\_\{i\}^\{h\}\}\{\\tau\_\{i\}^\{h\}\}\\sum\_\{k=1\}^\{\\tau\_\{i\}^\{h\}\}\\widetilde\{\\boldsymbol\{g\}\}\_\{b,i\}^\{h,k\}𝒈¯ph\\displaystyle\\overline\{\\boldsymbol\{g\}\}\_\{p\}^\{h\}=1Rh​∑r=1Rh𝒈~ph,r\\displaystyle=\\frac\{1\}\{R^\{h\}\}\\sum\_\{r=1\}^\{R^\{h\}\}\\widetilde\{\\boldsymbol\{g\}\}\_\{p\}^\{h,r\}where𝒈~b,ih,k\\widetilde\{\\boldsymbol\{g\}\}\_\{b,i\}^\{h,k\}and𝒈~ph,r\\widetilde\{\\boldsymbol\{g\}\}\_\{p\}^\{h,r\}denote the corresponding stochastic gradients\. Since all clients begin each round with the same bottom model and perform sequential local updates, Assumption[3](https://arxiv.org/html/2610.26992#Thmassumption3)gives the following parameter deviation bounds:

𝔼h​‖𝒘b,ih,k−1−𝒘bh‖\\displaystyle\\mathbb\{E\}\_\{h\}\\\|\\boldsymbol\{w\}\_\{b,i\}^\{h,k\-1\}\-\\boldsymbol\{w\}\_\{b\}^\{h\}\\\|≤η​σb​\(k−1\)\\displaystyle\\leq\\eta\\sigma\_\{b\}\(k\-1\)\(25\)𝔼h​‖𝒘ph,r−1−𝒘ph,0‖\\displaystyle\\mathbb\{E\}\_\{h\}\\\|\\boldsymbol\{w\}\_\{p\}^\{h,r\-1\}\-\\boldsymbol\{w\}\_\{p\}^\{h,0\}\\\|≤η​σp​\(r−1\)\\displaystyle\\leq\\eta\\sigma\_\{p\}\(r\-1\)According to Assumptions[1](https://arxiv.org/html/2610.26992#Thmassumption1)–[2](https://arxiv.org/html/2610.26992#Thmassumption2), the expected top model gradient deviation can be bounded as follows:

‖𝔼h\[𝒈¯ph\]−∑iπih∇Fp,i\(𝒘h\)‖\\displaystyle\\left\\\|\\mathbb\{E\}\_\{h\}\[\\overline\{\\boldsymbol\{g\}\}\_\{p\}^\{h\}\]\-\\sum\_\{i\}\\pi\_\{i\}^\{h\}\\nabla F\_\{p,i\}\(\\boldsymbol\{w\}^\{h\}\)\\right\\\|\(26\)≤L​ηRh​∑r=1Rh\[σb\|Sh,r\|​∑i∈Sh,r\(kih,r−1\)\+σp​\(r−1\)\]\\displaystyle\\leq\\frac\{L\\eta\}\{R^\{h\}\}\\sum\_\{r=1\}^\{R^\{h\}\}\\left\[\\frac\{\\sigma\_\{b\}\}\{\|S^\{h,r\}\|\}\\sum\_\{i\\in S^\{h,r\}\}\(k\_\{i\}^\{h,r\}\-1\)\+\\sigma\_\{p\}\(r\-1\)\\right\]≤L​η​\[σb​∑iπih​\(τih−1\)\+σp​\(Rh−1\)2\]\\displaystyle\\leq L\\eta\\left\[\\sigma\_\{b\}\\sum\_\{i\}\\pi\_\{i\}^\{h\}\(\\tau\_\{i\}^\{h\}\-1\)\+\\frac\{\\sigma\_\{p\}\(R^\{h\}\-1\)\}\{2\}\\right\]wherekih,rk\_\{i\}^\{h,r\}denotes the local update index associated with clientiiinSh,rS^\{h,r\}\. Applying the same analysis to the bottom model gradients, we obtain the corresponding deviation bound:

‖𝔼h\[𝒈¯bh\]−∑iρih∇Fb,i\(𝒘h\)‖\\displaystyle\\left\\\|\\mathbb\{E\}\_\{h\}\[\\overline\{\\boldsymbol\{g\}\}\_\{b\}^\{h\}\]\-\\sum\_\{i\}\\rho\_\{i\}^\{h\}\\nabla F\_\{b,i\}\(\\boldsymbol\{w\}^\{h\}\)\\right\\\|\(27\)≤L​η​∑iρihτih​∑k=1τih\[σb​\(k−1\)\+σp​\(ri,kh−1\)\]\\displaystyle\\leq L\\eta\\sum\_\{i\}\\frac\{\\rho\_\{i\}^\{h\}\}\{\\tau\_\{i\}^\{h\}\}\\sum\_\{k=1\}^\{\\tau\_\{i\}^\{h\}\}\\left\[\\sigma\_\{b\}\(k\-1\)\+\\sigma\_\{p\}\(r\_\{i,k\}^\{h\}\-1\)\\right\]According to Eq\. \([6](https://arxiv.org/html/2610.26992#S2.E6)\), the average top model update index satisfies:

1τih​∑k=1τih\(ri,kh−1\)=ζih\+τih−12\\frac\{1\}\{\\tau\_\{i\}^\{h\}\}\\sum\_\{k=1\}^\{\\tau\_\{i\}^\{h\}\}\(r\_\{i,k\}^\{h\}\-1\)=\\zeta\_\{i\}^\{h\}\+\\frac\{\\tau\_\{i\}^\{h\}\-1\}\{2\}\(28\)Combining Eqs\. \([26](https://arxiv.org/html/2610.26992#S4.E26)\)–\([28](https://arxiv.org/html/2610.26992#S4.E28)\), we obtain the following expected gradient deviation bounds:

‖𝔼h​\[𝒈¯bh\]−∇Fb​\(𝒘h\)‖\\displaystyle\\\|\\mathbb\{E\}\_\{h\}\[\\overline\{\\boldsymbol\{g\}\}\_\{b\}^\{h\}\]\-\\nabla F\_\{b\}\(\\boldsymbol\{w\}^\{h\}\)\\\|≤Bbh\\displaystyle\\leq B\_\{b\}^\{h\}\(29\)‖𝔼h​\[𝒈¯ph\]−∇Fp​\(𝒘h\)‖\\displaystyle\\\|\\mathbb\{E\}\_\{h\}\[\\overline\{\\boldsymbol\{g\}\}\_\{p\}^\{h\}\]\-\\nabla F\_\{p\}\(\\boldsymbol\{w\}^\{h\}\)\\\|≤Bph\\displaystyle\\leq B\_\{p\}^\{h\}Applying the standard inner\-product inequality2​⟨𝒙,𝒚⟩≥‖𝒙‖2−‖𝒙−𝒚‖22\\langle\\boldsymbol\{x\},\\boldsymbol\{y\}\\rangle\\geq\\\|\\boldsymbol\{x\}\\\|^\{2\}\-\\\|\\boldsymbol\{x\}\-\\boldsymbol\{y\}\\\|^\{2\}to Eq\. \([29](https://arxiv.org/html/2610.26992#S4.E29)\) establishes Eq\. \([22](https://arxiv.org/html/2610.26992#S4.E22)\)\. ∎

###### Lemma 2

Under Assumption[3](https://arxiv.org/html/2610.26992#Thmassumption3), the second moment of the aggregated update satisfies the following upper bound:

𝔼h​‖𝒅h‖2≤\(γh\)2​σb2\+\(Rh\)2​σp2\\mathbb\{E\}\_\{h\}\\\|\\boldsymbol\{d\}^\{h\}\\\|^\{2\}\\leq\(\\gamma^\{h\}\)^\{2\}\\sigma\_\{b\}^\{2\}\+\(R^\{h\}\)^\{2\}\\sigma\_\{p\}^\{2\}\(30\)

###### Proof:

According to Eq\. \([24](https://arxiv.org/html/2610.26992#S4.E24)\), Jensen’s inequality, and Assumption[3](https://arxiv.org/html/2610.26992#Thmassumption3), the second moment of the aggregated update can be bounded as follows:

𝔼h​‖𝒅h‖2\\displaystyle\\mathbb\{E\}\_\{h\}\\\|\\boldsymbol\{d\}^\{h\}\\\|^\{2\}=\(γh\)2​𝔼h​‖𝒈¯bh‖2\+\(Rh\)2​𝔼h​‖𝒈¯ph‖2\\displaystyle=\(\\gamma^\{h\}\)^\{2\}\\mathbb\{E\}\_\{h\}\\\|\\overline\{\\boldsymbol\{g\}\}\_\{b\}^\{h\}\\\|^\{2\}\+\(R^\{h\}\)^\{2\}\\mathbb\{E\}\_\{h\}\\\|\\overline\{\\boldsymbol\{g\}\}\_\{p\}^\{h\}\\\|^\{2\}\(31\)≤\(γh\)2​∑iρihτih​∑k=1τih𝔼h​‖𝒈~b,ih,k‖2\\displaystyle\\leq\(\\gamma^\{h\}\)^\{2\}\\sum\_\{i\}\\frac\{\\rho\_\{i\}^\{h\}\}\{\\tau\_\{i\}^\{h\}\}\\sum\_\{k=1\}^\{\\tau\_\{i\}^\{h\}\}\\mathbb\{E\}\_\{h\}\\\|\\widetilde\{\\boldsymbol\{g\}\}\_\{b,i\}^\{h,k\}\\\|^\{2\}\+Rh∑r=1Rh𝔼h∥𝒈~ph,r∥2\\displaystyle\+R^\{h\}\\sum\_\{r=1\}^\{R^\{h\}\}\\mathbb\{E\}\_\{h\}\\\|\\widetilde\{\\boldsymbol\{g\}\}\_\{p\}^\{h,r\}\\\|^\{2\}≤\(γh\)2​σb2\+\(Rh\)2​σp2\\displaystyle\\leq\(\\gamma^\{h\}\)^\{2\}\\sigma\_\{b\}^\{2\}\+\(R^\{h\}\)^\{2\}\\sigma\_\{p\}^\{2\}∎

###### Theorem 1

If all bottom models share the same initial parameters in each round andFFis lower bounded byFinfF\_\{\\inf\}, then under Assumptions[1](https://arxiv.org/html/2610.26992#Thmassumption1)–[3](https://arxiv.org/html/2610.26992#Thmassumption3), the mean\-square gradients afterHHaggregation rounds satisfy the following convergence bound:

1H​∑h=0H−1𝔼⁡\[γh​‖∇Fb​\(𝒘h\)‖2\+Rh​‖∇Fp​\(𝒘h\)‖2\]\\displaystyle\\frac\{1\}\{H\}\\sum\_\{h=0\}^\{H\-1\}\\mathbb\{E\}\\Big\[\\gamma^\{h\}\\\|\\nabla F\_\{b\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\+R^\{h\}\\\|\\nabla F\_\{p\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\\Big\]\(32\)≤2​\[F⁡\(𝒘0\)−Finf\]η​H\\displaystyle\\leq\\frac\{2\[F\(\\boldsymbol\{w\}^\{0\}\)\-F\_\{\\inf\}\]\}\{\\eta H\}\+1H∑h=0H−1𝔼\[γh\(Bbh\)2\+Rh\(Bph\)2\]\\displaystyle\+\\frac\{1\}\{H\}\\sum\_\{h=0\}^\{H\-1\}\\mathbb\{E\}\\Big\[\\gamma^\{h\}\(B\_\{b\}^\{h\}\)^\{2\}\+R^\{h\}\(B\_\{p\}^\{h\}\)^\{2\}\\Big\]\+L​ηH∑h=0H−1𝔼\[\(γh\)2σb2\+\(Rh\)2σp2\]\\displaystyle\+\\frac\{L\\eta\}\{H\}\\sum\_\{h=0\}^\{H\-1\}\\mathbb\{E\}\\Big\[\(\\gamma^\{h\}\)^\{2\}\\sigma\_\{b\}^\{2\}\+\(R^\{h\}\)^\{2\}\\sigma\_\{p\}^\{2\}\\Big\]

###### Proof:

According to Assumption[1](https://arxiv.org/html/2610.26992#Thmassumption1)and Lemmas[1](https://arxiv.org/html/2610.26992#Thmlemma1)–[2](https://arxiv.org/html/2610.26992#Thmlemma2), the smoothness descent inequality\[[41](https://arxiv.org/html/2610.26992#bib.bib37)\]gives the following bound on the expected loss decrease:

𝔼h​\[F⁡\(𝒘h\+1\)−F⁡\(𝒘h\)\]\\displaystyle\\mathbb\{E\}\_\{h\}\[F\(\\boldsymbol\{w\}^\{h\+1\}\)\-F\(\\boldsymbol\{w\}^\{h\}\)\]\(33\)≤−η​𝔼h​⟨∇F​\(𝒘h\),𝒅h⟩\+L​η22​𝔼h​‖𝒅h‖2\\displaystyle\\leq\-\\eta\\mathbb\{E\}\_\{h\}\\langle\\nabla F\(\\boldsymbol\{w\}^\{h\}\),\\boldsymbol\{d\}^\{h\}\\rangle\+\\frac\{L\\eta^\{2\}\}\{2\}\\mathbb\{E\}\_\{h\}\\\|\\boldsymbol\{d\}^\{h\}\\\|^\{2\}≤−η2​\[γh​‖∇Fb​\(𝒘h\)‖2\+Rh​‖∇Fp​\(𝒘h\)‖2\]\\displaystyle\\leq\-\\frac\{\\eta\}\{2\}\\Big\[\\gamma^\{h\}\\\|\\nabla F\_\{b\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\+R^\{h\}\\\|\\nabla F\_\{p\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\\Big\]\+η2​\[γh​\(Bbh\)2\+Rh​\(Bph\)2\]\\displaystyle\+\\frac\{\\eta\}\{2\}\\Big\[\\gamma^\{h\}\(B\_\{b\}^\{h\}\)^\{2\}\+R^\{h\}\(B\_\{p\}^\{h\}\)^\{2\}\\Big\]\+L​η22​\[\(γh\)2​σb2\+\(Rh\)2​σp2\]\\displaystyle\+\\frac\{L\\eta^\{2\}\}\{2\}\\Big\[\(\\gamma^\{h\}\)^\{2\}\\sigma\_\{b\}^\{2\}\+\(R^\{h\}\)^\{2\}\\sigma\_\{p\}^\{2\}\\Big\]Taking the total expectation and summing Eq\. \([33](https://arxiv.org/html/2610.26992#S4.E33)\) overHHrounds, we obtain:

η2​∑h=0H−1𝔼⁡\[γh​‖∇Fb​\(𝒘h\)‖2\+Rh​‖∇Fp​\(𝒘h\)‖2\]\\displaystyle\\frac\{\\eta\}\{2\}\\sum\_\{h=0\}^\{H\-1\}\\mathbb\{E\}\\Big\[\\gamma^\{h\}\\\|\\nabla F\_\{b\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\+R^\{h\}\\\|\\nabla F\_\{p\}\(\\boldsymbol\{w\}^\{h\}\)\\\|^\{2\}\\Big\]\(34\)≤F⁡\(𝒘0\)−Finf\\displaystyle\\leq F\(\\boldsymbol\{w\}^\{0\}\)\-F\_\{\\inf\}\+η2∑h=0H−1𝔼\[γh\(Bbh\)2\+Rh\(Bph\)2\]\\displaystyle\+\\frac\{\\eta\}\{2\}\\sum\_\{h=0\}^\{H\-1\}\\mathbb\{E\}\\Big\[\\gamma^\{h\}\(B\_\{b\}^\{h\}\)^\{2\}\+R^\{h\}\(B\_\{p\}^\{h\}\)^\{2\}\\Big\]\+L​η22∑h=0H−1𝔼\[\(γh\)2σb2\+\(Rh\)2σp2\]\\displaystyle\+\\frac\{L\\eta^\{2\}\}\{2\}\\sum\_\{h=0\}^\{H\-1\}\\mathbb\{E\}\\Big\[\(\\gamma^\{h\}\)^\{2\}\\sigma\_\{b\}^\{2\}\+\(R^\{h\}\)^\{2\}\\sigma\_\{p\}^\{2\}\\Big\]Dividing both sides of Eq\. \([34](https://arxiv.org/html/2610.26992#S4.E34)\) byη​H/2\\eta H/2establishes the convergence bound in Eq\. \([32](https://arxiv.org/html/2610.26992#S4.E32)\)\. ∎According to Theorem[1](https://arxiv.org/html/2610.26992#Thmtheorem1), the gradient deviationsBbhB\_\{b\}^\{h\}andBphB\_\{p\}^\{h\}influence the convergence bound\. Let𝝁0,ch\\boldsymbol\{\\mu\}\_\{0,c\}^\{h\}and𝝁S,ch,r\\boldsymbol\{\\mu\}\_\{S,c\}^\{h,r\}denote the expected class\-cctop model gradients under the balanced client distribution and selected activation setSh,rS^\{h,r\}, respectively\. We define the corresponding conditional gradient deviation and reference gradient bound as follows:

Gclsh\\displaystyle G\_\{\\rm cls\}^\{h\}=maxc⁡‖𝝁0,ch‖\\displaystyle=\\max\_\{c\}\\\|\\boldsymbol\{\\mu\}\_\{0,c\}^\{h\}\\\|\(35\)Δcondh,r\\displaystyle\\Delta\_\{\\rm cond\}^\{h,r\}=∑c=1MΦSh,r​\(c\)​‖𝝁S,ch,r−𝝁0,ch‖\\displaystyle=\\sum\_\{c=1\}^\{M\}\\Phi\_\{S^\{h,r\}\}\(c\)\\\|\\boldsymbol\{\\mu\}\_\{S,c\}^\{h,r\}\-\\boldsymbol\{\\mu\}\_\{0,c\}^\{h\}\\\|According to Pinsker’s inequality, the label\-dependent top model gradient deviation satisfies the following bound:

‖1\|Sh,r\|​∑i∈Sh,r∇Fp,i​\(𝒘h\)−∇Fp​\(𝒘h\)‖\\displaystyle\\left\\\|\\frac\{1\}\{\|S^\{h,r\}\|\}\\sum\_\{i\\in S^\{h,r\}\}\\nabla F\_\{p,i\}\(\\boldsymbol\{w\}^\{h\}\)\-\\nabla F\_\{p\}\(\\boldsymbol\{w\}^\{h\}\)\\right\\\|\(36\)≤Gclsh​2DKL\(ΦSh,r∥Φ0\)\+Δcondh,r\\displaystyle\\leq G\_\{\\rm cls\}^\{h\}\\sqrt\{2D\_\{\\rm KL\}\(\\Phi\_\{S^\{h,r\}\}\\\|\\Phi\_\{0\}\)\}\+\\Delta\_\{\\rm cond\}^\{h,r\}Hence, the KL\-divergence constraint limits the label\-dependent component of the server\-side gradient deviation for candidate combinations satisfying Eq\. \([11](https://arxiv.org/html/2610.26992#S3.E11)\)\. For timeout\-triggered fallback combinations, Eq\. \([36](https://arxiv.org/html/2610.26992#S4.E36)\) remains applicable with their actual KL divergence\.

For the bottom model, substituting Eqs\. \([14](https://arxiv.org/html/2610.26992#S3.E14)\)–\([15](https://arxiv.org/html/2610.26992#S3.E15)\) into Eq\. \([21](https://arxiv.org/html/2610.26992#S4.E21)\), the effective gradient contribution is proportional toρih∝\(τih\)2τih\+1​11\+ζih/Rh\\rho\_\{i\}^\{h\}\\propto\\frac\{\(\\tau\_\{i\}^\{h\}\)^\{2\}\}\{\\tau\_\{i\}^\{h\}\+1\}\\frac\{1\}\{1\+\\zeta\_\{i\}^\{h\}/R^\{h\}\}\. Letaiha\_\{i\}^\{h\}denote the normalized weights proportional to\(τih\)2/\(τih\+1\)\(\\tau\_\{i\}^\{h\}\)^\{2\}/\(\\tau\_\{i\}^\{h\}\+1\), and lettih=\(1\+ζih/Rh\)−1t\_\{i\}^\{h\}=\(1\+\\zeta\_\{i\}^\{h\}/R^\{h\}\)^\{\-1\}\. Sinceρih=aih​tih/∑jajh​tjh\\rho\_\{i\}^\{h\}=a\_\{i\}^\{h\}t\_\{i\}^\{h\}/\\sum\_\{j\}a\_\{j\}^\{h\}t\_\{j\}^\{h\}, the difference in weighted progress gaps can be expressed as follows:

∑iaih​ζih−∑iρih​ζih\\displaystyle\\sum\_\{i\}a\_\{i\}^\{h\}\\zeta\_\{i\}^\{h\}\-\\sum\_\{i\}\\rho\_\{i\}^\{h\}\\zeta\_\{i\}^\{h\}\(37\)=∑i,jaih​ajh​\(ζih−ζjh\)2​tih​tjh2​Rh​∑jajh​tjh≥0\\displaystyle=\\frac\{\\sum\_\{i,j\}a\_\{i\}^\{h\}a\_\{j\}^\{h\}\(\\zeta\_\{i\}^\{h\}\-\\zeta\_\{j\}^\{h\}\)^\{2\}t\_\{i\}^\{h\}t\_\{j\}^\{h\}\}\{2R^\{h\}\\sum\_\{j\}a\_\{j\}^\{h\}t\_\{j\}^\{h\}\}\\geq 0According to Eqs\. \([36](https://arxiv.org/html/2610.26992#S4.E36)\) and \([37](https://arxiv.org/html/2610.26992#S4.E37)\), the activation selection and aggregation weights control different sources of gradient deviation\. The former bounds the label\-dependent deviation of top model updates, while the latter reduces the weighted progress mismatch of bottom model updates\. Consequently, the convergence bound reflects the combined effects of client contribution imbalance and asynchronous model progress on the overall optimization\.

## VEvaluation

### V\-AExperimental Settings

System Deployment\.We conducted a series of comprehensive experiments on an edge computing hardware prototype system\. For the PS, we employed a deep learning GPU workstation equipped with an Intel\(R\) Xeon\(R\) Platinum 8360Y CPU, four NVIDIA A100 80GB PCIe, and 503 GiB of RAM\. To create a heterogeneous computing environment, we incorporated 80 NVIDIA Jetson devices, consisting of 30 Jetson TX2, 40 Jetson NX, and 10 Jetson AGX units\. The detailed technical specifications of the Jetson TX2, NX, and AGX models are provided in Table[I](https://arxiv.org/html/2610.26992#S5.T1)\. The experimental setup utilized a software platform based on Docker Swarm\[[42](https://arxiv.org/html/2610.26992#bib.bib13)\]and the PyTorch deep learning framework\[[43](https://arxiv.org/html/2610.26992#bib.bib14)\]\. Docker Swarm, a distributed systems orchestration tool, facilitates the deployment and management of a multi\-node architecture, enabling seamless monitoring and coordination of the devices’ operational states\. PyTorch, a widely adopted deep learning framework, was used for model training across the distributed hardware nodes\. To optimize inter\-device communication, we integrated the Message Passing Interface \(MPI\)\[[44](https://arxiv.org/html/2610.26992#bib.bib41)\], a high\-performance communication protocol that provides a robust set of functions for efficient data transmission and synchronization across the system\. This combination of hardware and software enabled the evaluation of CoASFL’s scalability and performance in a heterogeneous edge computing environment\.

TABLE I:Device technical specifications\.AI PerformanceGPU TypeJetson TX26 TOPs256\-core PascalJetson NX21 TOPs384\-core VoltaJetson AGX32 TOPs512\-core VoltaCPU TypeRAMJetson TX2Denver 2 and ARM 48 GB LPDDR4Jetson NX6\-core Carmel ARM 88 GB LPDDR4xJetson AGX8\-core Carmel ARM 832 GB LPDDR4xCPU FrequencyGPU FrequencyJetson TX22\.0GHz1\.12GHzJetson NX1\.9GHz1\.1GHzJetson AGX2\.2GHz1\.37GHz

Setting of System Heterogeneity\.The following experimental setup is employed to enable clients with heterogeneous computational and communication capabilities\.

1\)Computation Setting\.The Jetson TX2, NX, and AGX devices are configurable to operate in distinct modes \(4 for TX2, 8 for NX/AGX\), each specifying a particular number of active CPU cores and CPU/GPU frequencies\. These modes enable devices to exhibit different computational performance\. For instance, the AGX in its highest\-performance mode \(Mode 0\) can achieve training speeds up to 100×\\timesfaster than the TX2 in its lowest\-performance mode \(Mode 1\)\. Furthermore, to simulate time\-varying resource availability, the device modes are randomly altered every 10 aggregation rounds\.

2\)Communication Setting\.Devices are connected to the PS via Wi\-Fi routers, with a total of 80 devices grouped into four groups of 20 devices each\. These groups are positioned at varying distances from the Wi\-Fi routers, specifically at 2m, 8m, 14m, and 20m\. Due to random channel interference and contention between devices, the bandwidth between the PS and the devices fluctuates during training, ranging from 1 Mbps to 30 Mbps, as measured by iperf3\.

Applications and Models\.To evaluate the performance of the CoASFL in real\-world scenarios, we conduct experiments on three representative applications using pre\-trained transformer models of diverse scales:

1\)Object Recognition\.CIFAR\-100\[[45](https://arxiv.org/html/2610.26992#bib.bib32)\]is a widely used benchmark dataset for object recognition, which consists of 60,000 32×\\times32 colour images \(50,000 for training and 10,000 for test\) across 100 fine\-grained classes\. We utilize a well\-known vision transformer model, Deit\-Base\[[46](https://arxiv.org/html/2610.26992#bib.bib24)\], which has a size of 475MB and comprises 12 transformer layers that process images through 16×\\times16 patch embeddings\.

2\)Natural Language Processing\.We utilize the Question\-answering Natural Language Inference \(QNLI\) dataset\[[47](https://arxiv.org/html/2610.26992#bib.bib33)\], a classification dataset consisting of question\-sentence pairs, in this application\. There are 104,473 samples for training and 5,463 samples for testing in the QNLI dataset\. We train a RoBERTa\-Base\[[48](https://arxiv.org/html/2610.26992#bib.bib23)\]model with a size of 501MB on the QNLI dataset, which is composed of 12 transformer layers\.

3\)Text Classification\.We adopt the 20 Newsgroups dataset\[[49](https://arxiv.org/html/2610.26992#bib.bib34)\], a classical text classification benchmark containing 18,846 documents \(11,314 for training and 7,532 for testing\) across 20 categories\. For this task, we employ the Qwen2\.5\-0\.5B\[[50](https://arxiv.org/html/2610.26992#bib.bib35)\], a size of 1,884MB LLM with 24 transformer layers\.

Setting of Statistical Heterogeneity\.In the experiments, training samples of each client are drawn from a Dirichlet distribution\[[51](https://arxiv.org/html/2610.26992#bib.bib16)\],i\.e\.,v∼D​i​r​\(δ​ψ\)v\\sim Dir\(\\delta\\psi\), whereψ\\psicharacterizes a prior class distribution, andδ\>0\\delta\>0is a concentration parameter controlling the identicalness among clients\. Withδ→∞\\delta\\to\\infty, all clients have identical distributions to the prior class distribution \(i\.e\., IID\); withδ→0\\delta\\to 0each client holds data samples from only one class, which indicates high degree of statistical heterogeneity\. We specify 6 values \(e\.g\.,∞\\infty,1,0\.5,0\.25,0\.2,0\.1\) forδ\\deltato generate different data distributions that cover a spectrum of identicalness, and definep=1δp=\\frac\{1\}\{\\delta\}\(e\.g\.,pp=0,1,2,4,5,10\) to quantify the non\-IID levels\. The degree of statistical heterogeneity increases asp=0p=0is a special case of IIDness\.

Baselines\.We measure the effectiveness of CoASFL through a comparison with four baselines\. All the asynchronous baselines adopt an intra\-round asynchronous and inter\-round synchronous training strategy, which is the same as CoASFL\.

1\)PipeSFL\[[21](https://arxiv.org/html/2610.26992#bib.bib9)\]addresses the challenge of system heterogeneity in computation and communication during typical SFL by proposing a novel pipeline asynchronous framework that accelerates SFL execution across heterogeneous devices\.

2\)GAS\[[22](https://arxiv.org/html/2610.26992#bib.bib28)\]represents the state\-of\-the\-art in asynchronous SFL approaches, which addresses statistical heterogeneity challenges through an innovative approach of sampling and generating intermediate activations\.

3\)SplitFed\[[10](https://arxiv.org/html/2610.26992#bib.bib1)\]is the first SFL approach, which combines FL with SL by utilizing parallel client\-side computing to accelerate model training in a synchronous setting\.

![Refer to caption](https://arxiv.org/html/2610.26992v1/time_acc_iid_deit_cifar.png)\(a\)CIFAR\-100
\(b\)QNLI\(c\)Newsgroups
Fig\. 4:Test accuracy on the three IID datasets\(a\)CIFAR\-100\(b\)QNLI\(c\)Newsgroups
Fig\. 5:Test accuracy on the three non\-IID datasets4\)MergeSFL\[[28](https://arxiv.org/html/2610.26992#bib.bib2)\]is the state\-of\-the\-art approach in synchronous SFL, introducing the activation merging mechanism with client selection to address system and statistical heterogeneity simultaneously\.

Metrics\.We adopt the following evaluation metrics:

1\)Test Accuracymeasures the global model’s predictive performance on the test set across training rounds, reporting the final converged accuracy\.

2\)Time\-to\-Accuracymeasures the wall\-clock time to reach a target accuracy achievable by all approaches\. Clients’ average waiting time is also recorded to reflect training efficiency\.

3\)Network Trafficis calculated by summing the traffic for transmitting models or activations between the PS and clients when achieving a target accuracy\.

Experimental Parameters\.All experiments are run for 100 rounds with an average local update frequency of 20\. For DeiT on CIFAR\-100, we adopt a learning rate of 0\.1, a decay of 0\.993\[[41](https://arxiv.org/html/2610.26992#bib.bib37),[52](https://arxiv.org/html/2610.26992#bib.bib36)\], and a batch size of 64\. Conversely, for RoBERTa on QNLI and Qwen2\.5 on Newsgroups, we employ a learning rate of 5e\-5, a linear scheduler with 10% warm\-up\[[53](https://arxiv.org/html/2610.26992#bib.bib38)\], a batch size of 16, and a sequence length of 128\. For the SFL approaches, DeiT and RoBERTa are split at the 6th layer, while Qwen2\.5 is split at the 3rd layer\.

### V\-BOverall Performance

Firstly, we conduct a set of experiments on the IID datasets to evaluate the convergence performance of the CoASFL and the baselines\. The overall training processes of these approaches are presented in Fig\.[4](https://arxiv.org/html/2610.26992#S5.F4)\. According to the results, all the approaches achieve a similar test accuracy on the three datasets eventually\. However, the CoASFL achieves the fastest convergence, outperforming the other approaches on all three datasets\. For instance, as illustrated in Fig\.[4a](https://arxiv.org/html/2610.26992#S5.F4.sf1), CoASFL takes only 10,493s to achieve 85% accuracy for DeiT on CIFAR\-100, while GAS, PipeSFL, SplitFed, and MergeSFL consume 15,852s, 17,774s, 21,369s, and 14,210s, respectively\. Similarly, as shown in Fig\.[4b](https://arxiv.org/html/2610.26992#S5.F4.sf2), for RoBERTa on QNLI, CoASFL can separately speed up training by about 1\.61×\\times, 2\.81×\\times, 3\.74×\\times, and 2\.21×\\times, compared to GAS, PipeSFL, SplitFed, and MergeSFL, respectively\. Moreover, for Qwen2\.5 on Newsgroups, as shown in Fig\.[4c](https://arxiv.org/html/2610.26992#S5.F4.sf3), CoASFL reduces the total training time by about 17\.6%, 55\.9%, 65\.4%, and 49\.1%, compared to the four baselines\.

![Refer to caption](https://arxiv.org/html/2610.26992v1/acc_traffic_iid_deit_cifar.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2610.26992v1/acc_traffic_iid_roberta_qnli.png)\(b\)QNLI
![Refer to caption](https://arxiv.org/html/2610.26992v1/acc_traffic_iid_qwen_newsgroup.png)\(c\)Newsgroups

Fig\. 6:Traffic consumption to achieve target accuracies\(a\)CIFAR\-100\(b\)QNLI\(c\)Newsgroups
Fig\. 7:Average waiting time on the three datasetsSecondly, we also conduct a set of experiments of these approaches on the three datasets with non\-IID levelpp=10, and the results are presented in Fig\.[5](https://arxiv.org/html/2610.26992#S5.F5)\. We observe that all the approaches suffer from varying degrees of performance degradation, especially the SplitFed\. However, CoASFL maintains a similar convergence rate as in the IID scenario, and achieves the best performance among these approaches\. For instance, by Fig\.[5a](https://arxiv.org/html/2610.26992#S5.F5.sf1), CoASFL achieves 82\.36% accuracy in 12,553s for DeiT on CIFAR\-100, while GAS, PipeSFL, SplitFed, and MergeSFL take 18,283s, 23,388s, 29,518s, and 24,849s to reach the accuracy of 80\.3%, 79\.1%, 80\.6%, and 83\.51%, respectively\. Similarly, as shown in Fig\.[5b](https://arxiv.org/html/2610.26992#S5.F5.sf2), for RoBERTa on QNLI with the same training time of 10,187s, CoASFL improves the test accuracy by about 2\.04%, 4\.68%, 4\.96%, and 1\.55%, compared to GAS, PipeSFL, SplitFed, and MergeSFL, respectively\. In addition, CoASFL with dynamic activation merging and asynchronous training tackles both statistical heterogeneity and client drift better than all the baselines\. Specifically, Fig\.[5c](https://arxiv.org/html/2610.26992#S5.F5.sf3)illustrates that CoASFL separately improves the final accuracy by about 1\.18%, 1\.62%, and 1\.31% compared to the GAS, PipeSFL, and SplitFed, respectively\. Furthermore, CoASFL speeds up training by about 1\.96×\\timescompared to MergeSFL\. Besides, CoASFL takes 13,262s to achieve 70% accuracy for Qwen2\.5 on Newsgroups, reducing the training time by about 50\.6%, 79\.8%, 81\.1%, and 67\.3% compared to GAS, PipeSFL, SplitFed, and MergeSFL, respectively\. These results demonstrate that CoASFL is effective in simultaneously tackling the system and statistical heterogeneity\.

Furthermore, to illustrate the advantage of the CoASFL in saving communication resources, we present the network traffic consumption of these approaches when achieving different target accuracies in Fig\.[6](https://arxiv.org/html/2610.26992#S5.F6)\. According to the results, the network traffic consumption of all approaches increases with the target accuracy for all three datasets\. Furthermore, CoASFL always consumes the least network traffic among all approaches\. Specifically, as shown in Fig\.[6a](https://arxiv.org/html/2610.26992#S5.F6.sf1), when achieving 85% accuracy, CoASFL, GAS, MergeSFL consume 23,774MB, 27,736MB, and 31,699MB respectively, while PipeSFL, SplitFed consume 36,541MB and 38,831MB for DeiT on CIFAR\-100\. By Fig\.[6b](https://arxiv.org/html/2610.26992#S5.F6.sf2), for training RoBERTa on QNLI to achieve 92% accuracy, CoASFL reduces the network traffic consumption by about 4,114MB, 11,066MB, 12,996MB, and 6,219MB, compared to GAS, PipeSFL, SplitFed, and MergeSFL, respectively\. Besides, as illustrated in Fig\.[6c](https://arxiv.org/html/2610.26992#S5.F6.sf3), CoASFL saves network traffic consumption by about 9\.37%, 35\.55%, 40\.81%, and 21\.62% when achieving 74% accuracy for Qwen2\.5 on Newsgroups\.

![Refer to caption](https://arxiv.org/html/2610.26992v1/noniid_level_acc_deit_cifar.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2610.26992v1/noniid_level_acc_roberta_qnli.png)\(b\)QNLI
![Refer to caption](https://arxiv.org/html/2610.26992v1/noniid_level_acc_qwen_newsgroup.png)\(c\)Newsgroups

Fig\. 8:Test accuracy under different non\-IID levels\(a\)IID\(b\)Non\-IID
Fig\. 9:Effects of DAM and CMAFinally, to demonstrate the robustness of CoASFL towards system heterogeneity, Fig\.[7](https://arxiv.org/html/2610.26992#S5.F7)presents the average waiting time of CoASFL and several baseline methods across three datasets\. As shown in Fig\.[7](https://arxiv.org/html/2610.26992#S5.F7), CoASFL consistently achieves significantly lower waiting time than GAS and MergeSFL\. Specifically, while GAS utilizes a buffering mechanism, it strictly waits for the buffer to populate without a flexible timeout, leaving it vulnerable to blocking delays caused by stragglers\. Similarly, MergeSFL relies on an edge\-node selection strategy that imposes a rigid synchronization barrier, which inevitably leaves unselected nodes idle and incurs substantial waiting overhead\. Consequently, despite outperforming schemes such as PipeSFL and SplitFed, both baselines exhibit significantly higher waiting times than CoASFL\. For example, as illustrated in Fig\.[7a](https://arxiv.org/html/2610.26992#S5.F7.sf1), when training DeiT on CIFAR\-100, the average waiting time of CoASFL is only 24s, whereas GAS and MergeSFL require 33s and 39s, respectively\. More importantly, CoASFL enables flexible feature matching through dynamic feature aggregation and a timeout mechanism, allowing it to fully utilize system resources and prevent edge nodes from waiting indefinitely\. This design significantly reduces the average waiting time\. In Fig\.[7c](https://arxiv.org/html/2610.26992#S5.F7.sf3), CoASFL achieves 54s average waiting time for training Qwen2\.5 on Newsgroups, compared with 72s, 123s, 157s, and 106s for GAS, PipeSFL, SplitFed, and MergeSFL, respectively\.

TABLE II:Training Performance of Various Buffer Capacityκ5%10%20%30%50%Accuracy\(%\)78\.1180\.282\.3682\.7583\.24Time \(s\)9,15110,73112,55316,33221,016

### V\-CEffect of Non\-IID Levels

To demonstrate the effectiveness of the CoASFL under non\-IID data, we present the test accuracy of different approaches at varying non\-IID levels in Fig\.[8](https://arxiv.org/html/2610.26992#S5.F8), in which the model accuracy of the five approaches on all the datasets decreases as the non\-IID level increases\. However, the performance of CoASFL is very close to MergeSFL and outperforms the other baselines on all datasets\. PipeSFL ignores statistical heterogeneity and exhibits the lowest accuracy\. To handle statistical heterogeneity, GAS employs an adaptive approach that samples and generates intermediate activations, while MergeSFL performs client selection via activation merging, partially mitigating non\-IID effects on training\. Specifically, as illustrated in Fig\.[8a](https://arxiv.org/html/2610.26992#S5.F8.sf1), CoASFL can achieve an improvement of test accuracy by about 2\.06%, 3\.26%, 1\.76% on CIFAR\-100 with non\-IID level ofpp=10, compared to the baselines \(i\.e\., GAS, PipeSFL, SplitFed\), and differs from MergeSFL by only 1\.15%\. Notably, by Fig\.[8b](https://arxiv.org/html/2610.26992#S5.F8.sf2), with non\-IID level ofpp=10 on QNLI, CoASFL achieves an improvement of final accuracy by about 1\.62%, 2\.44%, 1\.14%, compared to the baselines \(i\.e\., GAS, PipeSFL, SplitFed\), and differs from MergeSFL by only 0\.22%\. Besides, as shown in Fig\.[8c](https://arxiv.org/html/2610.26992#S5.F8.sf3), while transitioning from IID to non\-IID level ofpp=10 on Newsgroups, CoASFL and MergeSFL suffer from only 2\.7%, 1\.5% loss in accuracy, while the accuracy loss for GAS, PipeSFL, and SplitFed is 3\.71%, 4\.15%, and 3\.84%, respectively\. Collectively, these results demonstrate that CoASFL maintains competitive model accuracy as statistical heterogeneity increases, confirming its robustness across different non\-IID levels\.

### V\-DEffect of Key Strategies

There are two key strategies of CoASFL,i\.e\., dynamic activation merging and consistency\-aware model aggregation, being developed to enhance the performance of SFL\. Herein, we conduct several sets of experiments for training DeiT on CIFAR\-100 with IID distribution \(pp=0\) and non\-IID distribution \(pp=10\) to evaluate the effectiveness of the two critical strategies\. We adopt the CoASFL without dynamic activation merging \(CoASFL w/o DAM\) and CoASFL without consistency\-aware model aggregation \(CoASFL w/o CMA\) as the baselines\. In CoASFL w/o DAM, the PS directly updates the top model upon each activation arrival without activation selection or merging\. While in CoASFL w/o CMA, the aggregation weight of each client follows the typical assignment method,i\.e\., the more iterations a client participates in, the greater the weight assigned\. By Fig\.[9](https://arxiv.org/html/2610.26992#S5.F9), CoASFL w/o CMA converges faster than CoASFL w/o DAM on the IID and Non\-IID datasets, but CoASFL w/o DAM achieves a higher test accuracy than CoASFL w/o CMA on the Non\-IID dataset\. Specifically, CoASFL w/o CMA reduces the total training time by about 56\.1% compared to CoASFL w/o DAM, but CoASFL w/o DAM improves the final test accuracy by about 1\.27% on the non\-IID dataset, compared to CoASFL w/o CMA\. Powered by dynamic activation merging and consistency\-aware model aggregation, CoASFL can speed up training by about 2\.09×\\timescompared to CoASFL w/o DAM, and improve the final test accuracy by about 2\.1% compared to CoASFL w/o CMA, which reflects the positive roles of the two strategies\.

### V\-EEffect of Activation Buffer Capacity

We use the DeiT on CIFAR\-100 to evaluate the performance of the CoASFL under various buffer capacitiesΘ0\\Theta\_\{0\}with a non\-IID distribution \(p=10p=10\)\. The detailed results are presented in Table[II](https://arxiv.org/html/2610.26992#S5.T2)\. For ease of evaluation, we define the buffer capacity ratio as the proportion of the total number of clients,i\.e\.,κ=Θ0/N\\kappa=\\Theta\_\{0\}/N\. We observe two key trends\. First, as the ratioκ\\kapparanges from 5% to 20%, the final accuracy improves notably\. This trend demonstrates that an insufficiently large buffer limits activation diversity, thereby affecting the quality of top model updates\. However, asκ\\kappaincreases further from 20% to 50%, the accuracy plateaus, offering only a marginal gain of 0\.88%\. Conversely, the training time increases significantly with a larger buffer\. The time cost for increasingκ\\kappafrom 5% to 20% is tolerable, but it increases rapidly by approximately 68\.9% asκ\\kappagrows from 20% to 50%, substantially weakening the efficiency benefit\. A larger buffer provides more candidate combinations but may also increase activation matching and waiting delays\. Considering this trade\-off,κ=20%\\kappa=20\\%\(i\.e\.,Θ0=N/5\\Theta\_\{0\}=N/5\) provides the most favorable balance, and we use this ratio in all other experiments\.

## VIRelated Work

In the presence of pronounced heterogeneity in computation capability and communication bandwidth across clients, a common research addresses these challenges through structural adaptation of split models\. Several studies dynamically determine split points under heterogeneous resource constraints\. For example, SmartSplit\[[54](https://arxiv.org/html/2610.26992#bib.bib10)\]analyzes each client’s memory budget and runtime training capability to dynamically assign client\-specific split layers, thereby improving training efficiency while maintaining model accuracy\. Similarly, ESFL\[[55](https://arxiv.org/html/2610.26992#bib.bib42)\]allocates bottom models of different depths to heterogeneous clients and jointly optimizes server\-side computation resource allocation, enhancing training efficiency under system heterogeneity\. HFSL\[[56](https://arxiv.org/html/2610.26992#bib.bib43)\]introduces auxiliary layers to accommodate heterogeneous model partitions and leverages self\-distillation to improve training accuracy\. HeteroSFL\[[57](https://arxiv.org/html/2610.26992#bib.bib44)\]employs bottleneck layers of varying widths to accommodate heterogeneous communication capabilities and uses bidirectional knowledge sharing to mitigate non\-IID data effects\. Similarly, HASFL\[[58](https://arxiv.org/html/2610.26992#bib.bib45)\]jointly optimizes adaptive batch\-size scheduling and split\-layer selection according to client capabilities, balancing training latency and convergence performance\. MergeSFL\[[28](https://arxiv.org/html/2610.26992#bib.bib2)\]combines feature merging with batch\-size regulation to address system and statistical heterogeneity under synchronous SFL\. Although these approaches improve training efficiency under heterogeneous resource constraints, most still rely on synchronous training, where the progress of each round is constrained by the slowest participating client\. As computation and communication delays vary across clients, faster clients must wait at synchronization barriers, leaving their resources underutilized and limiting overall training efficiency\.

To alleviate this bottleneck, several studies further explore asynchronous execution, allowing the server to process available client activations without waiting for slower clients\. DT\-ASFL\[[59](https://arxiv.org/html/2610.26992#bib.bib6)\]leverages digital twins to monitor device states and support asynchronous model updates\. PipeSFL\[[21](https://arxiv.org/html/2610.26992#bib.bib9)\]adopts a priority\-based scheduling strategy at the server and enables asynchronous execution within aggregation rounds to improve training efficiency\. From another perspective, RingSFL\[[15](https://arxiv.org/html/2610.26992#bib.bib46)\]constructs a ring topology to distribute model training across heterogeneous clients and exploits overlapping layers to improve model updates\. These approaches primarily focus on alleviating straggler effects and improving execution efficiency, rather than explicitly addressing the optimization inconsistencies arising from asynchronous client participation\. GAS\[[22](https://arxiv.org/html/2610.26992#bib.bib28)\]further considers the biased top\-model updates caused by unequal activation arrivals and generates supplementary activations to balance client influence\. However, its bottom\-model aggregation does not explicitly account for the progress mismatch between the client and server model partitions\. In contrast, CoASFL mitigates both the horizontal inconsistency caused by unequal client participation and the vertical inconsistency arising from progress mismatch between client\-side bottom models and the server\-side top model through dynamic activation merging and consistency\-aware model aggregation\.

## VIIConclusion

In this paper, we characterize horizontal inconsistency across clients and vertical inconsistency between clients and the server in asynchronous SFL\. Then we propose CoASFL, an inconsistency\-aware asynchronous SFL framework that incorporates a dynamic activation merging mechanism and a consistency\-aware model aggregation mechanism to effectively suppress inconsistency propagation throughout asynchronous training\. The experimental results show that the CoASFL can speed up model training by at least 1\.61×\\timesto 3\.74×\\timesand improve final model accuracy by up to 3\.26% in heterogeneous scenarios, compared to the baselines\.

## References

- \[1\]E\. Li, L\. Zeng, Z\. Zhou, and X\. Chen\(2019\)Edge ai: on\-demand accelerating deep neural network inference via edge computing\.IEEE transactions on wireless communications19\(1\),pp\. 447–457\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[2\]X\. Wang, Y\. Han, C\. Wang, Q\. Zhao, X\. Chen, and M\. Chen\(2019\)In\-edge ai: intelligentizing mobile edge computing, caching and communication by federated learning\.Ieee Network33\(5\),pp\. 156–165\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[3\]J\. Park, S\. Samarakoon, A\. Elgabli,et al\.\(2021\)Communication\-efficient and distributed learning over wireless networks: principles and applications\.Proceedings of the IEEE109\(5\),pp\. 796–819\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[4\]C\. Niu, F\. Wu, S\. Tang, L\. Hua, R\. Jia, C\. Lv, Z\. Wu, and G\. Chen\(2020\)Billion\-scale federated learning on mobile clients: a submodel design with tunable privacy\.InProceedings of the 26th Annual International Conference on Mobile Computing and Networking,pp\. 1–14\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[5\]B\. McMahan, E\. Moore, D\. Ramage,et al\.\(2017\)Communication\-efficient learning of deep networks from decentralized data\.InArtificial intelligence and statistics,pp\. 1273–1282\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[6\]P\. Kairouz and H\. B\. McMahan\(2021\)Advances and open problems in federated learning\.Foundations and trends in machine learning14\(1\-2\),pp\. 1–210\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[7\]Y\. Liao, Y\. Xu, H\. Xu, Z\. Yao, L\. Wang, and C\. Qiao\(2023\)Accelerating federated learning with data and model parallelism in edge computing\.IEEE/ACM Transactions on Networking\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1),[§IV](https://arxiv.org/html/2610.26992#S4.p1.1)\.
- \[8\]A\. Abedi and S\. S\. Khan\(2024\)FedSL: federated split learning on distributed sequential data in recurrent neural networks\.Multimedia Tools and Applications83\(10\),pp\. 28891–28911\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[9\]R\. Pathak and M\. J\. Wainwright\(2020\)FedSplit: an algorithmic framework for fast federated optimization\.Advances in neural information processing systems33,pp\. 7057–7066\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[10\]C\. Thapaet al\.\(2022\)Splitfed: when federated learning meets split learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.36,pp\. 8485–8493\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.26992#S2.SS1.p1.1),[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p1.1),[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p13.1)\.
- \[11\]C\. Xie, Z\. Chen, W\. Yi, H\. Shin, and A\. Nallanathan\(2025\)Tackling class imbalance and client heterogeneity for split federated learning in wireless networks\.IEEE Transactions on Wireless Communications24\(6\),pp\. 4920–4936\.External Links:[Document](https://dx.doi.org/10.1109/TWC.2025.3545236)Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[12\]D\. Yan, M\. Hu, X\. Xie, Y\. Yang, and M\. Chen\(2026\)S2\{\}^\{2\}2fl: toward efficient and accurate heterogeneous split federated learning\.IEEE Transactions on Computers75\(1\),pp\. 320–334\.External Links:[Document](https://dx.doi.org/10.1109/TC.2025.3626198)Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p1.1)\.
- \[13\]Y\. Liao, Y\. Xu, H\. Xu, Z\. Yao, L\. Huang, and C\. Qiao\(2024\)ParallelSFL: a novel split federated learning framework tackling heterogeneity issues\.arXiv preprint arXiv:2410\.01256\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1)\.
- \[14\]S\. Li, D\. Yao, and J\. Liu\(2023\)FedVS: straggler\-resilient and privacy\-preserving vertical federated learning for split models\.InInternational conference on machine learning,pp\. 20296–20311\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1)\.
- \[15\]J\. Shen, N\. Cheng, X\. Wang, F\. Lyu, W\. Xu, Z\. Liu, K\. Aldubaikhy, and X\. Shen\(2024\)RingSFL: an adaptive split federated learning towards taming client heterogeneity\.IEEE Transactions on Mobile Computing23\(5\),pp\. 5462–5478\.External Links:[Document](https://dx.doi.org/10.1109/TMC.2023.3309633)Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1),[§VI](https://arxiv.org/html/2610.26992#S6.p2.1)\.
- \[16\]A\. Gupta, S\. Misra, N\. Pathak, and D\. Das\(2023\)Fedcare: federated learning for resource\-constrained healthcare devices in iomt system\.IEEE Transactions on Computational Social Systems10\(4\),pp\. 1587–1596\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1)\.
- \[17\]D\. Liang, J\. Zhang, E\. Chen, Z\. Li, R\. Li, and H\. Yang\(2026\)Towards straggler\-resilient split federated learning: an unbalanced update approach\.Advances in neural information processing systems38,pp\. 31634–31673\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1)\.
- \[18\]Y\. Xu, Y\. Liao, H\. Xu, Z\. Ma, L\. Wang, and J\. Liu\(2022\)Adaptive control of local updating and model compression for efficient federated learning\.IEEE Transactions on Mobile Computing22\(10\),pp\. 5675–5689\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1),[§IV](https://arxiv.org/html/2610.26992#S4.p1.1)\.
- \[19\]Y\. Liao, Y\. Xu, H\. Xu, L\. Wang, and C\. Qian\(2023\)Adaptive configuration for heterogeneous participants in decentralized federated learning\.InIEEE INFOCOM 2023\-IEEE Conference on Computer Communications,pp\. 1–10\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1)\.
- \[20\]Y\. Liao, Y\. Xu, H\. Xu, M\. Chen, L\. Wang, and C\. Qiao\(2024\)Asynchronous decentralized federated learning for heterogeneous devices\.IEEE/ACM Transactions on Networking\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1)\.
- \[21\]Y\. Gao, B\. Hu, M\. B\. Mashhadi, W\. Wang, and M\. Bennis\(2024\)PipeSFL: a fine\-grained parallelization framework for split federated learning on heterogeneous clients\.IEEE Transactions on Mobile Computing\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1),[§I](https://arxiv.org/html/2610.26992#S1.p4.1),[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p11.1),[§VI](https://arxiv.org/html/2610.26992#S6.p2.1)\.
- \[22\]J\. Yang and Y\. Liu\(2025\)GAS: generative activation\-aided asynchronous split federated learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 21956–21964\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1),[§I](https://arxiv.org/html/2610.26992#S1.p4.1),[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p12.1),[§VI](https://arxiv.org/html/2610.26992#S6.p2.1)\.
- \[23\]H\. Ao, H\. Tian, W\. Ni, G\. Nie, and D\. Niyato\(2025\)Semi\-asynchronous federated split learning for computing\-limited devices in wireless networks\.IEEE Transactions on Wireless Communications\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p2.1)\.
- \[24\]Y\. J\. Cho, J\. Wang, and G\. Joshi\(2022\)Towards understanding biased client selection in federated learning\.InInternational Conference on Artificial Intelligence and Statistics,pp\. 10351–10375\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p3.1)\.
- \[25\]Q\. Ma, Y\. Xu, H\. Xu, Z\. Jiang, L\. Huang, and H\. Huang\(2021\)FedSA: a semi\-asynchronous federated learning mechanism in heterogeneous edge computing\.IEEE Journal on Selected Areas in Communications39\(12\),pp\. 3654–3672\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p3.1)\.
- \[26\]C\. Xie, S\. Koyejo, and I\. Gupta\(2019\)Asynchronous federated optimization\.arXiv preprint arXiv:1903\.03934\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p3.1)\.
- \[27\]Y\. Chen, Y\. Ning, M\. Slawski, and H\. Rangwala\(2020\)Asynchronous online federated learning for edge devices with non\-iid data\.In2020 IEEE International Conference on Big Data \(Big Data\),pp\. 15–24\.Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p3.1)\.
- \[28\]Y\. Liao, Y\. Xu, H\. Xu, L\. Wang, Z\. Yao, and C\. Qiao\(2024\)MergeSFL: split federated learning with feature merging and batch size regulation\.In2024 IEEE 40th International Conference on Data Engineering \(ICDE\),Vol\.,pp\. 2054–2067\.External Links:[Document](https://dx.doi.org/10.1109/ICDE60146.2024.00164)Cited by:[§I](https://arxiv.org/html/2610.26992#S1.p4.1),[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p14.1),[§VI](https://arxiv.org/html/2610.26992#S6.p1.1)\.
- \[29\]C\. Thapa, M\. A\. P\. Chamikara, and S\. A\. Camtepe\(2021\)Advancements of federated learning towards privacy preservation: from federated learning to split learning\.Federated Learning Systems: Towards Next\-Generation AI,pp\. 79–109\.Cited by:[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p1.1)\.
- \[30\]M\. Wu, G\. Cheng, D\. Ye, J\. Kang, R\. Yu, Y\. Wu, and M\. Pan\(2023\)Federated split learning with data and label privacy preservation in vehicular networks\.IEEE Transactions on Vehicular Technology73\(1\),pp\. 1223–1238\.Cited by:[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p1.1)\.
- \[31\]P\. Vepakomma, O\. Gupta, A\. Dubey, and R\. Raskar\(2019\)Reducing leakage in distributed deep learning for sensitive health data\.arXiv preprint arXiv:1812\.005642,pp\. 8\.Cited by:[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p1.1)\.
- \[32\]Z\. Yang, Y\. Chen, H\. Huangfu, M\. Ran, H\. Wang, X\. Li, and Y\. Zhang\(2023\)Dynamic corrected split federated learning with homomorphic encryption for u\-shaped medical image networks\.IEEE Journal of Biomedical and Health Informatics27\(12\),pp\. 5946–5957\.Cited by:[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p1.1)\.
- \[33\]D\. Leroy, A\. Coucke, T\. Lavril, T\. Gisselbrecht, and J\. Dureau\(2019\)Federated learning for keyword spotting\.InICASSP 2019\-2019 IEEE international conference on acoustics, speech and signal processing \(ICASSP\),pp\. 6341–6345\.Cited by:[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p2.1)\.
- \[34\]D\. Halperin, W\. Hu, A\. Sheth, and D\. Wetherall\(2010\)Predictable 802\.11 packet delivery from wireless channel measurements\.ACM SIGCOMM computer communication review40\(4\),pp\. 159–170\.Cited by:[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p2.2)\.
- \[35\]C\. Yue, R\. Jin, K\. Suh, Y\. Qin, B\. Wang, and W\. Wei\(2017\)LinkForecast: cellular link bandwidth prediction in lte networks\.IEEE Transactions on Mobile Computing17\(7\),pp\. 1582–1594\.Cited by:[§III\-B1](https://arxiv.org/html/2610.26992#S3.SS2.SSS1.p2.2)\.
- \[36\]Goldberger and Greenspan\(2003\)An efficient image similarity measure based on approximations of kl\-divergence between two gaussian mixtures\.InProceedings Ninth IEEE International conference on computer vision,pp\. 487–493\.Cited by:[§III\-B2](https://arxiv.org/html/2610.26992#S3.SS2.SSS2.p2.1)\.
- \[37\]J\. R\. Hershey and P\. A\. Olsen\(2007\)Approximating the kullback leibler divergence between gaussian mixture models\.In2007 IEEE International Conference on Acoustics, Speech and Signal Processing \- ICASSP ’07,Vol\.4,pp\. IV–317–IV–320\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP.2007.366913)Cited by:[§III\-B2](https://arxiv.org/html/2610.26992#S3.SS2.SSS2.p2.1)\.
- \[38\]B\. Beyer, C\. Jones, J\. Petoff,et al\.\(2016\)Site reliability engineering: how google runs production systems\." O’Reilly Media, Inc\."\.Cited by:[§III\-B2](https://arxiv.org/html/2610.26992#S3.SS2.SSS2.p5.1)\.
- \[39\]D\. Han, H\. I\. Bhatti, J\. Lee, and J\. Moon\(2021\)Accelerating federated learning with split learning on locally generated losses\.InICML 2021 workshop on federated learning for user privacy and data confidentiality\. ICML Board,Cited by:[§IV](https://arxiv.org/html/2610.26992#S4.p1.1)\.
- \[40\]H\. Yang, M\. Fang, and J\. Liu\(2021\)Achieving linear speedup with partial worker participation in non\-iid federated learning\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§IV](https://arxiv.org/html/2610.26992#S4.p1.1)\.
- \[41\]R\. M\. Gower, N\. Loizou, X\. Qian, A\. Sailanbayev, E\. Shulgin, and P\. Richtárik\(2019\)SGD: general analysis and improved rates\.InInternational conference on machine learning,pp\. 5200–5209\.Cited by:[§IV](https://arxiv.org/html/2610.26992#S4.p5.5.1),[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p19.1)\.
- \[42\]N\. Naik\(2016\)Building a virtual system of systems using docker swarm in multiple clouds\.In2016 IEEE International Symposium on Systems Engineering \(ISSE\),pp\. 1–3\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p1.1)\.
- \[43\]A\. Paszke, S\. Gross, F\. Massa,et al\.\(2019\)Pytorch: an imperative style, high\-performance deep learning library\.Advances in neural information processing systems32\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p1.1)\.
- \[44\]E\. Gabriel, G\. E\. Fagg, G\. Bosilca,et al\.\(2004\)Open mpi: goals, concept, and design of a next generation mpi implementation\.InEuropean Parallel Virtual Machine/Message Passing Interface Users’ Group Meeting,pp\. 97–104\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p1.1)\.
- \[45\]A\. Krizhevsky G\. Hintonet al\.\(2009\)Learning multiple layers of features from tiny images\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p6.1)\.
- \[46\]H\. Touvron, M\. Cord, and H\. Jégou\(2022\)Deit iii: revenge of the vit\.InEuropean conference on computer vision,pp\. 516–533\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p6.1)\.
- \[47\]A\. Wang, A\. Singh, J\. Michael, F\. Hill, O\. Levy, and S\. R\. Bowman\(2018\)GLUE: a multi\-task benchmark and analysis platform for natural language understanding\.arXiv preprint arXiv:1804\.07461\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p7.1)\.
- \[48\]Y\. Liu, M\. Ott, N\. Goyal,et al\.\(2019\)Roberta: a robustly optimized bert pretraining approach\.arXiv preprint arXiv:1907\.11692\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p7.1)\.
- \[49\]K\. Lang\(1995\)Newsweeder: learning to filter netnews\.InMachine learning proceedings 1995,pp\. 331–339\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p8.1)\.
- \[50\]B\. Hui, J\. Yang, Z\. Cui,et al\.\(2024\)Qwen2\. 5\-coder technical report\.arXiv preprint arXiv:2409\.12186\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p8.1)\.
- \[51\]M\. Yurochkin, M\. Agarwal, S\. Ghosh,et al\.\(2019\)Bayesian nonparametric federated learning of neural networks\.InInternational conference on machine learning,pp\. 7252–7261\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p9.1)\.
- \[52\]Y\. Shi, Y\. Zhang, Z\. Huang, X\. Yang, L\. Shen, W\. Chen, and X\. Wang\(2024\)Heterogeneous federated learning with splited language model\.arXiv preprint arXiv:2403\.16050\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p19.1)\.
- \[53\]I\. Loshchilov and F\. Hutter\(2017\)Decoupled weight decay regularization\.arXiv preprint arXiv:1711\.05101\.Cited by:[§V\-A](https://arxiv.org/html/2610.26992#S5.SS1.p19.1)\.
- \[54\]C\. Tian, L\. Li, K\. Tam, Y\. Wu, and C\. Xu\(2024\)Breaking the memory wall for heterogeneous federated learning via model splitting\.IEEE Transactions on Parallel and Distributed Systems\.Cited by:[§VI](https://arxiv.org/html/2610.26992#S6.p1.1)\.
- \[55\]G\. Zhu, Y\. Deng, X\. Chen, H\. Zhang, Y\. Fang, and T\. F\. Wong\(2024\)ESFL: efficient split federated learning over resource\-constrained heterogeneous wireless devices\.IEEE Internet of Things Journal11\(16\),pp\. 27153–27166\.External Links:[Document](https://dx.doi.org/10.1109/JIOT.2024.3397677)Cited by:[§VI](https://arxiv.org/html/2610.26992#S6.p1.1)\.
- \[56\]N\. Wu, W\. Zhao, Y\. Chen, J\. Xiao, J\. Wang, W\. Liang, K\. Li, and N\. Sukhija\(2024\)HFSL: heterogeneity split federated learning based on client computing capabilities\.J\. Supercomput\.81\(1\)\.External Links:ISSN 0920\-8542,[Link](https://doi.org/10.1007/s11227-024-06632-6),[Document](https://dx.doi.org/10.1007/s11227-024-06632-6)Cited by:[§VI](https://arxiv.org/html/2610.26992#S6.p1.1)\.
- \[57\]X\. Chen, J\. Li, D\. Fan, and C\. Chakrabarti\(2025\)HeteroSFL: split federated learning with heterogeneous clients and non\-iid data\.IEEE Internet of Things Journal12\(15\),pp\. 30460–30474\.External Links:[Document](https://dx.doi.org/10.1109/JIOT.2025.3572393)Cited by:[§VI](https://arxiv.org/html/2610.26992#S6.p1.1)\.
- \[58\]Z\. Lin, Z\. Chen, X\. Chen, W\. Ni, and Y\. Gao\(2025\)HASFL: heterogeneity\-aware split federated learning over edge computing systems\.arXiv preprint arXiv:2506\.08426\.Cited by:[§VI](https://arxiv.org/html/2610.26992#S6.p1.1)\.
- \[59\]V\. Stephanie, I\. Khalil, and M\. Atiquzzaman\(2023\)Digital twin enabled asynchronous splitfed learning in e\-healthcare systems\.IEEE Journal on Selected Areas in Communications\.Cited by:[§VI](https://arxiv.org/html/2610.26992#S6.p2.1)\.