SWB-DM: A Calibrated Sliced-Wasserstein-Barycenter Aggregator with Delayed-Momentum Caching for Byzantine-Robust Federated Learning under Partial Participation

# SWB-DM: A Calibrated Sliced-Wasserstein-Barycenter Aggregator with Delayed-Momentum Caching for Byzantine-Robust Federated Learning under Partial Participation
Source: [https://arxiv.org/html/2609.16099](https://arxiv.org/html/2609.16099)
1Saranraj S\.Department of AIMLVel Tech Rangarajan Dr\. SagunthalaR&D Institute of Science and TechnologyChennai, Indiasaranrajsnkr@gmail\.com2Saranya M\. S\.Assistant Professor, Department of AIMLVel Tech Rangarajan Dr\. SagunthalaR&D Institute of Science and TechnologyChennai, Indiasaranyams@veltech\.edu\.in3Alex David S\.Professor, Department of AIMLVel Tech Rangarajan Dr\. SagunthalaR&D Institute of Science and TechnologyChennai, Indiaadstechlearning@gmail\.com4Ajay Kumar A\.Department of AIMLVel Tech Rangarajan Dr\. SagunthalaR&D Institute of Science and TechnologyChennai, Indiavtu24379@veltech\.edu\.in

###### Abstract

Most robust aggregation methods for federated learning are designed and benchmarked under an implicit assumption: that the clients sampled in any given round look roughly like the full population\. In practice, partial participation breaks this assumption\. Even when the overall fraction of Byzantine clients is modest, a small per\-round sample can easily end up dominated by adversaries – and that is enough to quietly violate the finite\-sample guarantees that methods like coordinate\-wise median, Krum, Bulyan, and trimmed mean rely on\. We introduce SWB\-DM, which combines two ideas\. The first, SWB, is a randomized\-slicing aggregator that treats each chunk of a client’s update vector as a one\-dimensional empirical distribution, computes a trimmed Wasserstein barycenter across clients, and then restores coordinate identity through a medoid\-based gauge\-fixing step – a heuristic we developed ourselves and make no claim it belongs to standard optimal\-transport theory\. The second is a DeMoA\-style delayed\-momentum cache that aggregates over the entire client population each round, not just whoever happened to be sampled\. Calibrating SWB’s trim ratio to the assumed corruption level turns out to be essential, not cosmetic: under\-trimming leads to collapse at corruption levels a properly calibrated version survives\. We evaluate SWB\-DM across 448 CIFAR\-10 configurations \(7 methods, 16 attack–corruption pairings, 2 participation rates, 2 seeds\), supplemented by CIFAR\-100, FEMNIST, and a 500\-client scalability experiment\. What emerges is not one failure mode but several mechanistically distinct ones: median degrades to a deterministic wrong answer when the sample size is small and even, Krum can silently violate its ownn\>2​f\+2n\>2f\+2precondition and diverge without any warning, and Bulyan’sn≥4​f\+3n\\geq 4f\+3threshold produces a sharp, reproducible pass/fail boundary\. On the attack side, IPM defeats order\-statistic defenses – including SWB – more reliably than ALIE, a finding we corroborate through delta\-space error measurements against a semi\-formal convergence bound\. SWB\-DM’s caching does come with a genuine warm\-up cost: all 64 CIFAR\-10 and all 8 CIFAR\-100 configurations improved when given more rounds, and the magnitude of this cost tracks participation rate and task difficulty exactly as the caching mechanism would predict\. To put this in perspective, we extended every baseline to the same round budget and found that SWB\-DM’s gain on CIFAR\-10 is disproportionately large – though on CIFAR\-100, FLTrust benefits even more from the extra rounds, for an entirely different reason unrelated to caching\.

###### Index Terms:

federated learning, Byzantine robustness, sliced optimal transport, delayed momentum, robust aggregation, partial participation, adversarial machine learning

††aftertitle:## IIntroduction

The promise of federated learning is that a shared model can be trained across many clients without ever pooling their raw data – but the flip side is that the server must accept gradient updates it has no way to independently verify\[[2](https://arxiv.org/html/2609.16099#bib.bib2)\]\. The standard response has been to replace FedAvg’s simple weighted average\[[1](https://arxiv.org/html/2609.16099#bib.bib1)\]with aggregation rules designed to limit how much a handful of corrupted clients can steer the result: coordinate\-wise median or trimmed mean\[[3](https://arxiv.org/html/2609.16099#bib.bib3)\], Krum’s nearest\-neighbor selection\[[2](https://arxiv.org/html/2609.16099#bib.bib2)\], Bulyan’s layered combination of both\[[4](https://arxiv.org/html/2609.16099#bib.bib4)\], or FLTrust’s cosine\-similarity scoring against a server\-held reference\[[5](https://arxiv.org/html/2609.16099#bib.bib5)\]\. Every one of these carries a precondition – an assumed upper bound on how many adversaries are present and, in some cases, a trim parameter that must exceed the true corruption fraction – and those preconditions are almost always stated in terms of the full client population, not the much smaller group that actually participates in any single round\.

Under partial participation, this distinction matters immensely\. The population\-level corruption fractionβ\\betatells us nothing about how many attackers actually end up in a single round’s samplenn, which is usually much smaller than the full populationNN\. Simple sampling noise can easily spike effective corruption well pastβ\\betain any given round—and a strategic attacker who actively skips unfavorable rounds creates an even bigger problem\.

In both cases, the aggregator’s safety precondition quietly fails, even if you configured the system with an exact population parameter\. Otsuka, Takezawa, and Yamada tackle this directly with Delayed Momentum Aggregation \(DeMoA\)\[[8](https://arxiv.org/html/2609.16099#bib.bib8)\]: by caching every client’s latest update and aggregating over the entire cache each round, the server removes selection luck from the sample size equation entirely\.

Our contributions are as follows:

1. 1\.SWB, a new aggregator that takes a rotated chunk of client updates, treats it as a one\-dimensional empirical distribution, and computes a trimmed Wasserstein barycenter across clients\. Coordinate identity – lost during the sorting step that makes the 1\-D transport tractable – is restored through a*Wasserstein\-medoid gauge\-fixing*step that we want to be upfront about: it is our own heuristic, not something borrowed from the established sliced optimal\-transport literature \(Section[IV](https://arxiv.org/html/2609.16099#S4)\)\.
2. 2\.SWB\-DM, which pairs SWB with DeMoA\-style caching and includes a demonstration that calibrating the trim ratio to the assumed corruption level is not optional – an under\-trimmed variant collapses at corruption levels the properly calibrated version handles without difficulty\.
3. 3\.A systematic 448\-configuration CIFAR\-10 study \(7 methods, 16 constrained attack–corruption pairings, 2 participation rates, 2 seeds\), complemented by CIFAR\-100 and FEMNIST generalization experiments and a 500\-client scalability trial\. Crucially, we trace per\-round diagnostics – accuracy, parameter norms, prediction\-class histograms – that let us tease apart the failure mechanisms of median, Krum, and Bulyan, rather than burying them all under one aggregate accuracy number\.
4. 4\.A semi\-formal convergence bound validated against two structurally different adaptive attacks \(ALIE and IPM\), revealing that their error profiles are qualitatively different: ALIE’s is non\-monotonic while IPM’s rises strictly, which goes a long way toward explaining why IPM breaks more defenses in our grid\.
5. 5\.An honest accounting of SWB\-DM’s caching warm\-up cost, placed in context by extending all six baselines to the same round budget\. On CIFAR\-10 SWB\-DM’s gain turns out to be disproportionately large; on CIFAR\-100, however, FLTrust benefits even more – for an entirely different, non\-caching reason\.

## IIRelated Work

FedAvg and robust aggregation\.FedAvg\[[1](https://arxiv.org/html/2609.16099#bib.bib1)\]simply averages client updates weighted by sample count, offering zero protection against even a single unbounded adversarial contribution\. Coordinate\-wise median and trimmed mean\[[3](https://arxiv.org/html/2609.16099#bib.bib3)\]limit per\-coordinate influence, but they need a certain minimum number of honest samples to do so: whennnis small and even, the median convention just picks the lower of two middle values, which is deterministic and has no meaningful robustness left\. Krum\[[2](https://arxiv.org/html/2609.16099#bib.bib2)\]selects the single update whose summed distance to itsn−f−2n\-f\-2nearest neighbors is smallest, toleratingffadversaries only whenn\>2​f\+2n\>2f\+2\. Bulyan\[[4](https://arxiv.org/html/2609.16099#bib.bib4)\]stacks iterative Krum\-style selection on top of trimmed mean, tightening the requirement ton≥4​f\+3n\\geq 4f\+3\. FLTrust\[[5](https://arxiv.org/html/2609.16099#bib.bib5)\]takes a fundamentally different approach: it scores each client by cosine similarity with a small reference update the server computes on its own data, then rescales passing clients’ updates to match the server reference norm before averaging\. Unlike the others, it does not depend on a minimum sample size to function\.

Byzantine attacks\.Label\-flipping, sign\-flipping, and additive Gaussian noise corrupt updates without adapting to the defense\. ALIE\[[6](https://arxiv.org/html/2609.16099#bib.bib6)\]computes the largest perturbation that stays inside a robust statistic’s acceptance band for a given\(n,f\)\(n,f\)\. IPM\[[7](https://arxiv.org/html/2609.16099#bib.bib7)\]negates the honest gradient direction and norm\-matches\. Despite ALIE’s analytical sophistication, IPM defeats more aggregators in our experiments\.

Delayed Momentum Aggregation\.DeMoA\[[8](https://arxiv.org/html/2609.16099#bib.bib8)\]is the most directly relevant prior work\. It maintains a server\-side cache of every client’s most recent update; only sampled clients refresh their entries each round\. The aggregator then runs on the full cache ofNNentries, keeping effective corruption near the population\-levelβ\\betaregardless of sampling luck\. This is the caching mechanism we adopt for SWB\-DM\. Our contribution is pairing it with a different inner aggregator and characterizing the resulting warm\-up dynamics\.

Sliced Wasserstein barycenters\.The formal SWB\[[10](https://arxiv.org/html/2609.16099#bib.bib10)\]computes barycenters overNNprobability distributions projected onto random 1\-D lines where optimal transport has a closed form\. Federated aggregation does not fit this framework — each client contributes a point, not a distribution\. Section[IV](https://arxiv.org/html/2609.16099#S4)describes how we work around this by slicing coordinates*within*a chunk\. Doing so creates a coordinate\-identity problem not addressed in prior OT work; we propose a solution in Section[IV](https://arxiv.org/html/2609.16099#S4)\.

## IIIThreat Model

A server coordinatesNNclients overTTcommunication rounds\. In each round, a subsetStS\_\{t\}of sizen=p​Nn=pNis sampled at participation ratepp\. A fixed fractionβ\\betaof the population is Byzantine and remains so for the entire run, though which of those adversaries happen to fall intoStS\_\{t\}varies from round to round\. For every aggregator that carries a precondition on the number of tolerable adversaries \(Krum, Bulyan\), we setf=⌊n​β⌋f=\\lfloor n\\beta\\rfloor– a fixed, population\-level estimate computed without any access to the true per\-round attacker count\. Giving a defender oracle knowledge of who is actually malicious each round would be unrealistic and would unfairly advantage methods whose guarantees depend on knowing the count\.

Fig\. 1:Architecture of the proposed SWB\-DM framework, integrating partial client participation, Byzantine threat injection, delayed\-momentum full\-population caching, and calibrated sliced\-Wasserstein\-barycenter aggregation\.Figure[1](https://arxiv.org/html/2609.16099#S3.F1)illustrates the overall architecture\. Each communication round, the global server selects a subset of clients for participation\. After local training, any Byzantine clients in the sample may corrupt their updates using label\-flipping, sign\-flipping, Gaussian noise, ALIE, or IPM attacks\. The delayed\-momentum cache then refreshes entries for participating clients while retaining the most recent updates from everyone else\. SWB processes the complete cached population through a pipeline of chunking, randomized orthogonal rotation, row\-wise sorting, calibrated trimming, medoid\-based gauge fixing, inverse rotation, and multi\-view averaging\. The resulting robust aggregate replaces the global model heading into the next round\.

## IVMethod

### IV\-ASWB: A Randomized\-Slicing, Wasserstein\-Inspired Aggregator

Givennnclient vectors\{δ1,…,δn\}⊂ℝD\\\{\\delta\_\{1\},\\ldots,\\delta\_\{n\}\\\}\\subset\\mathbb\{R\}^\{D\}— in our implementation, these are full locally\-trained model states submitted for aggregation, not deltas from the global model — we split them into chunks of sizeCC\. For each chunk, letX∈ℝn×CX\\in\\mathbb\{R\}^\{n\\times C\}be the matrix stacking thennclients’ values\. SWB then proceeds as follows:

1. 1\.Draw a random orthogonal matrixQ∈ℝC×CQ\\in\\mathbb\{R\}^\{C\\times C\}\(obtained by QR\-decomposing a Gaussian random matrix\) and project:Y=X​QY=XQ\.
2. 2\.Sort each client’s row ofYYindependently\. This is the step that makes one\-dimensional optimal transport tractable: theCCprojected values within a single client’s chunk are treated as samples from an empirical distribution\.
3. 3\.Compute a trimmed mean across clients at each sorted\-rank position, discarding the top and bottom⌊n⋅r⌋\\lfloor n\\cdot r\\rfloorvalues, to produce a barycenter rowb∈ℝCb\\in\\mathbb\{R\}^\{C\}in sorted\-rank space\.
4. 4\.Medoid gauge fixing\.The sorting in step 2 destroys each client’s original mapping from coordinates to ranks\. To undo this, we identify the medoid clientm=arg⁡mini⁡‖sort⁡\(Yi\)−b‖2m=\\arg\\min\_\{i\}\\\|\\mathrm\{sort\}\(Y\_\{i\}\)\-b\\\|\_\{2\}– the client whose sorted chunk is closest to the barycenter – and borrow that client’s rank\-to\-coordinate permutation to placebbback into the correct coordinate positions before rotating back withQ⊤Q^\{\\top\}\.

We repeat this procedure overP=2P\{=\}2independent random rotations and average the results\. The trim ratio is set asr=min⁡\(0\.45,β\+0\.05\)r=\\min\(0\.45,\\beta\+0\.05\), tying it directly to the assumed corruption level \(Section[VI](https://arxiv.org/html/2609.16099#S6)\)\.

Step 4 is specific to our setting; the standard SWB literature operates on unordered point clouds where coordinate identity is not an issue\. An alternative approach — restoring identity by resorting the barycenter according to original coordinate ranks — did not produce usable aggregates\. The medoid approach borrows the rank\-to\-coordinate permutation from the nearest real client\. It performs well empirically but lacks a formal guarantee\. We refer to SWB throughout as*inspired by*, rather than an instance of, the formal SWB framework\.

### IV\-BDelayed\-Momentum Caching

Letci\(t\)c\_\{i\}^\{\(t\)\}be the server’s cached copy of clientii’s latest locally\-trained model state at roundtt:ci\(t\)←si\(t\)c\_\{i\}^\{\(t\)\}\\leftarrow s\_\{i\}^\{\(t\)\}fori∈Sti\\in S\_\{t\}, wheresi\(t\)s\_\{i\}^\{\(t\)\}is the trained state clientiisubmits, andci\(t\)=ci\(t−1\)c\_\{i\}^\{\(t\)\}=c\_\{i\}^\{\(t\-1\)\}otherwise\. The server then*replaces*the global model with the aggregate over the full cache,w\(t\+1\)=Agg⁡\(\{c1\(t\),…,cN\(t\)\}\)w^\{\(t\+1\)\}=\\mathrm\{Agg\}\(\\\{c\_\{1\}^\{\(t\)\},\\ldots,c\_\{N\}^\{\(t\)\}\\\}\), for any base aggregatorAgg⁡\(⋅\)\\mathrm\{Agg\}\(\\cdot\)\. SWB\-DM uses SWB as that base aggregator; we also test a DelayedMomentum baseline using plain median instead, to separate what the caching contributes from what the inner statistic contributes\.

## VTheoretical Analysis

We adopt the standard robust\-mean\-estimation bound shapeE⁡\(β\)≤E0\+C​βE\(\\beta\)\\leq E\_\{0\}\+C\\sqrt\{\\beta\}\(Eq\.[1](https://arxiv.org/html/2609.16099#S5.E1)\), whereE⁡\(β\)E\(\\beta\)denotes theL2L\_\{2\}aggregation error relative to the true benign mean,E0E\_\{0\}captures the irreducible finite\-sample error whenβ=0\\beta\{=\}0, andCCreflects the spread of the benign update distribution\[[9](https://arxiv.org/html/2609.16099#bib.bib9)\]\.

E⁡\(β\)≤E0\+C​βE\(\\beta\)\\leq E\_\{0\}\+C\\sqrt\{\\beta\}\(1\)We fitCCempirically to SWB’s measured error\. This does not constitute a tight, SWB\-specific derivation; we use it to check whether the general bound shape holds against two structurally different adaptive attacks \(Section[VII\-C](https://arxiv.org/html/2609.16099#S7.SS3)\)\. A tighter analysis is left to future work\.

## VIExperimental Setup

Datasets and model\.We use CIFAR\-10 as the primary testbed, withN=20N\{=\}20clients for most experiments andN=500N\{=\}500for the scalability study\. For generalization, we also run on CIFAR\-100 and FEMNIST \(the EMNIST\-ByClass variant with 62 classes\)\. All experiments use a compact two\-convolutional\-layer CNN, trained for 2 local epochs per round with SGD \(learning rate0\.010\.01, momentum0\.90\.9\)\.

Partitioning\.Data is split across clients using a Dirichlet non\-IID partition withα=0\.5\\alpha\{=\}0\.5, and the partition seed is held fixed across all methods and random seeds so that everyone trains on exactly the same data split\. AtN=500N\{=\}500, per\-client sample counts range from 13 to 288 \(median 92\) – a22×22\\timesspread that we verified by direct measurement rather than assumption\.

Baselines\.For the main 448\-configuration grid, we compare against FedAvg, Median, Krum, Bulyan, and FLTrust, alongside our proposed SWB and SWB\-DM \(7 methods total\)\. For the 500\-client scalability study \(Section VII\-F\), we additionally include a plain DelayedMomentum baseline \(DeMoA with coordinate\-wise median\) to isolate the inner statistic’s contribution from the caching mechanism\.

Attacks\.The attack suite includes no attack \(clean\), label\-flip, sign\-flip, Gaussian noise, ALIE\[[6](https://arxiv.org/html/2609.16099#bib.bib6)\], and IPM\[[7](https://arxiv.org/html/2609.16099#bib.bib7)\]\. ALIE’s perturbation magnitude is computed analytically from the round’s\(n,f\)\(n,f\)values\.

Grid\.Corruption levels areβ∈\{0,0\.1,0\.2,0\.3\}\\beta\\in\\\{0,0\.1,0\.2,0\.3\\\}, but not every combination runs: the clean \(no\-attack\) condition only appears atβ=0\\beta\{=\}0, and the five actual attacks run only atβ∈\{0\.1,0\.2,0\.3\}\\beta\\in\\\{0\.1,0\.2,0\.3\\\}, yielding1\+5×3=161\+5\\times 3=16attack–corruption pairings per method rather than the full6×4=246\\times 4\{=\}24\. Participation rates arep∈\{0\.5,0\.1\}p\\in\\\{0\.5,0\.1\\\}, seeds are\{42,7\}\\\{42,7\\\}, and each run goes for 10 rounds\. In total, that gives7×16×2×2=4487\\times 16\\times 2\\times 2=448configurations\. For Krum and Bulyan, we setf=⌊n​β⌋f=\\lfloor n\\beta\\rfloor\. Bulyan needs a larger sample to satisfy itsn≥4​f\+3n\\geq 4f\+3requirement, so we drawn=11n\{=\}11clients per round for it \(versusn=10n\{=\}10everywhere else\) — this is enough to meet the bound atβ=0\.2\\beta\{=\}0\.2, but not atβ=0\.3\\beta\{=\}0\.3\. We leave that violation in on purpose, as a stress test \(Section[VII\-B](https://arxiv.org/html/2609.16099#S7.SS2)\)\.

## VIIResults

### VII\-AMain Grid

Table[I](https://arxiv.org/html/2609.16099#S7.T1)presents the mean CIFAR\-10 accuracy for each method across both participation rates, averaged over all six attack conditions and random seeds\.

TABLE I:Mean accuracy \(%\) as a function of corruptionβ\\beta, shown forp=0\.5p\{=\}0\.5andp=0\.1p\{=\}0\.1, averaged across all six attack conditions and 2 random seeds\.![Refer to caption](https://arxiv.org/html/2609.16099v1/theory_bound.png)

![Refer to caption](https://arxiv.org/html/2609.16099v1/theory_bound_adaptive_attacks.png)

Fig\. 2:Empirical aggregation error against corruption level underALIE\(left\) andIPM\(right\), compared with the fitted bound\.Atp=0\.1p\{=\}0\.1, SWB\-DM falls behind all other methods within 10 rounds\. Section[VII\-E](https://arxiv.org/html/2609.16099#S7.SS5)attributes this to a warm\-up effect in the caching layer\. FLTrust maintains stable accuracy across corruption levels and participation rates because its trust scoring uses a server\-held clean dataset rather than statistics derived from the participating pool\.

Krum achieves higher accuracy atp=0\.1p\{=\}0\.1than atp=0\.5p\{=\}0\.5\(43\.1043\.10vs\.34\.1134\.11atβ=0\\beta\{=\}0\)\. This reflects its single\-client selection rule: Krum selects one update and discards the rest\. Atp=0\.5p\{=\}0\.5withn=10n\{=\}10, nine updates are discarded per round\. Under non\-IID partitions where each client captures only part of the global distribution, the information loss from discarding most updates accumulates across rounds\.

Table[II](https://arxiv.org/html/2609.16099#S7.T2)counts outright collapses — configurations where accuracy fell to≤10\.5%\\leq 10\.5\\%, essentially chance\-level on CIFAR\-10\.

TABLE II:Number of configurations that collapsed \(≤10\.5%\\leq 10\.5\\%accuracy\) out of 64 tested for each method\.FLTrust never collapses across all 64 configurations\. Median is the most collapse\-prone at 10 out of 64, predominantly under IPM \(Section[VII\-B](https://arxiv.org/html/2609.16099#S7.SS2)\)\.

### VII\-BFailure\-Mode Diagnostics

At the grid’s most extreme setting \(n=2n=2: IPM withβ=0\.3\\beta=0\.3atp=0\.1p=0\.1\), a 20\-round per\-round trace \(seediagnostics\.pyin our repository,[https://github\.com/saranrajsnkr/SWB\-DM\-full\-empirical\-study](https://github.com/saranrajsnkr/SWB-DM-full-empirical-study)\) exposes two distinct failure modes\.

#### Median under IPM atn=2n\{=\}2

With two participating clients, the coordinate\-wise median returns the smaller of the two per\-coordinate values\. IPM constructs its update to point opposite the honest gradient direction on most coordinates, so the smaller value is typically the attacker’s\. Accuracy holds at exactly10\.00%10\.00\\%across all 20 rounds — chance level for ten classes — and all test samples are assigned the same label in every round\.

The parameter norm increases from9\.899\.89at round 1 to31\.5631\.56at round 20, indicating that gradient updates are applied but in directions that do not reduce the training loss\.

#### Krum atn=2n\{=\}2

Krum’s guarantee needsn\>2​f\+2n\>2f\+2\. Atn=2n=2, no value offfsatisfies that\. We built a guarded version that checks the bound before aggregating; it simply refuses to run whenn=2n=2, which is the right call\. Our main grid, though, uses the unguarded variant everywhere for consistency across configurations\. Left to compute a neighbor\-distance score over two clients, it has almost nothing to rank against\. By round 7, it picks the corrupted client as the global update\.

The damage is immediate and total:545,066545\{,\}066of the model’s545,098545\{,\}098parameters turn toNaNin that single step, and the other3232overflow toInf\. Most aggregation rules blend several clients together, so one bad update gets diluted\. Krum doesn’t blend — it copies\. Whichever client it selects*becomes*the global state, so if that client is corrupted, so is the model, permanently\.

We see the same fragility in the extended CIFAR\-100 runs \(Section[VII\-E](https://arxiv.org/html/2609.16099#S7.SS5)\), though it shows up as inconsistency rather than collapse\. One seed under label\-flip corruption drops from13\.82%13\.82\\%at round 10 to1\.96%1\.96\\%by round 25, hovering near the ~1% chance rate for 100 classes\. A second seed, same corruption, same everything else, keeps improving the whole time\. Krum’s accuracy on any given run is really a question of which two or three clients happened to get sampled\.

#### Bulyan at the precondition boundary

Bulyan needsn≥4​f\+3n\\geq 4f\+3\. Atβ=0\.2\\beta\{=\}0\.2, withn=11n\{=\}11andf=2f\{=\}2, that works out to11≥1111\\geq 11— satisfied with nothing to spare\. And the method holds:51\.7051\.70–51\.98%51\.98\\%accuracy under both ALIE and IPM, right where the guarantee says it should land\.

Pushβ\\betato0\.30\.3andffbecomes33, so the requirement jumps ton≥15n\\geq 15\. We’re still atn=11n\{=\}11\. The bound fails, and Bulyan doesn’t degrade gracefully about it — accuracy falls straight to10\.00%10\.00\\%under both attacks\. One step up inβ\\betais the difference between a working defense and pure chance, with nothing in between\. We reran the configuration a second time to make sure; the result didn’t move\.

#### Krum under IPM within the safe zone

Atβ=0\.2\\beta\{=\}0\.2,n=10n\{=\}10,f=2f\{=\}2, Krum’s preconditionn\>2​f\+2=6n\>2f\+2\{=\}6holds with margin\. Under ALIE, Krum reaches51\.15%51\.15\\%accuracy\. Under IPM with the same\(n,f\)\(n,f\), it drops to10\.00%10\.00\\%\.

IPM negates the honest gradient direction and matches its norm, producing corrupted updates that minimize the neighbor\-distance score Krum uses to rank clients\. The corrupted update receives a low distance score and is selected as the representative\. The preconditionn\>2​f\+2n\>2f\+2is necessary but not sufficient against this class of attack\.

### VII\-CBound vs\. Empirical Error, Two Attacks

Figure[2](https://arxiv.org/html/2609.16099#S7.F2)plots the delta\-spaceL2L\_\{2\}aggregation error for Median and SWB as the corruption countffranges from 0 to 4 out of 10 sampled clients, under ALIE \(left panel\) and IPM \(right panel\), with the bound from Eq\.[1](https://arxiv.org/html/2609.16099#S5.E1)fitted to SWB\.

The two attacks leave very different fingerprints on the error curve\. UnderALIE, error for both methods actually*dips*below thef=0f\{=\}0baseline atf=1f\{=\}1andf=2f\{=\}2, before it starts climbing\. SWB, specifically, goes1\.478→1\.101→1\.0871\.478\\to 1\.101\\to 1\.087before turning upward atf=3f\{=\}3andf=4f\{=\}4\. That dip isn’t noise — it’s what ALIE is built to do\. The attack tunes each poisoned update to sit just inside a robust statistic’s tolerance threshold, so a little corruption can briefly pull the aggregate*closer*to the true mean\. Add more corrupted clients, though, and the cumulative damage eventually overwhelms that effect\.

IPMdoesn’t behave this way at all\. SWB’s error climbs at every step —1\.4831\.483,1\.5291\.529,1\.7691\.769,2\.6862\.686,3\.6803\.680— with no dip anywhere\. That tracks with how IPM works: it pushes updates along the negative gradient direction and simply scales the push with each additional corrupted client\. It’s the same story we saw on the accuracy side in Section[VII\-B](https://arxiv.org/html/2609.16099#S7.SS2)and Table[II](https://arxiv.org/html/2609.16099#S7.T2)\.

One thing worth flagging: SWB’s aggregation error is worse than Median’s across the board, everyff, both attacks\. That’s the price of chunked rotation — it shows up clearly here in delta space, but end\-to\-end accuracy tends to hide it\. Put plainly, SWB wins downstream \(Sections[VII\-A](https://arxiv.org/html/2609.16099#S7.SS1)–[VII\-E](https://arxiv.org/html/2609.16099#S7.SS5)\) despite, not because of, its per\-round aggregation error\.

### VII\-DGeneralization: CIFAR\-100 and FEMNIST

Table[III](https://arxiv.org/html/2609.16099#S7.T3)presents mean FEMNIST accuracy across 62 classes, atβ=0\.2\\beta\{=\}0\.2,p=0\.5p\{=\}0\.5, over 10 rounds\.

TABLE III:FEMNIST mean accuracy \(%\),β=0\.2\\beta\{=\}0\.2,p=0\.5p\{=\}0\.5, 2\-seed average\.On FEMNIST, FedAvg takes a much steeper hit under label\-flip and sign\-flip attacks \(71\.24%71\.24\\%,74\.50%74\.50\\%\) than its CIFAR\-10 numbers at comparable corruption levels would lead you to expect\. Bulyan, FLTrust, and Median, on the other hand, all hold up well, landing comfortably in the7676–85%85\\%range\. If anything, this makes an even stronger case for robust aggregation on harder tasks, not a weaker one\.

CIFAR\-100 accuracies are lower across the board, as one would expect with five times as many classes \(Krum sits at1313–15%15\\%, FedAvg at1919–32%32\\%\), and SWB\-DM again trails at 10 rounds \(8\.438\.43–14\.49%14\.49\\%\) for the same warm\-up reason confirmed in Section[VII\-E](https://arxiv.org/html/2609.16099#S7.SS5)\.

### VII\-ESWB\-DM’s Warm\-Up Cost

Because SWB\-DM’s cache only gets refreshed for whichever clients happen to be sampled each round, it naturally takes longer to “fully warm up” when participation is low or the task is inherently harder\. We wanted to quantify this directly, so we re\-ran all 64 CIFAR\-10 configurations out to 25 rounds and did the same for 8 CIFAR\-100 configurations\. The results were unambiguous: every single configuration improved\. On CIFAR\-10, all 64 out of 64 showed gains, with a mean improvement of 13\.12 percentage points\. The improvement was larger atp=0\.1p\{=\}0\.1\(15\.25 points\) than atp=0\.5p\{=\}0\.5\(11\.00 points\) – exactly the direction you would expect if this is genuinely a warm\-up effect\. On CIFAR\-100, all 8 out of 8 improved, gaining 10\.39 points on average and moving the mean from 11\.93% up to 22\.32%, an 87% relative improvement\. Not a single configuration got worse, and none stayed flat\. The 10\-round figures in Tables[I](https://arxiv.org/html/2609.16099#S7.T1)and[III](https://arxiv.org/html/2609.16099#S7.T3)should be interpreted alongside these extended results\.

Is the warm\-up cost unique to caching?To answer this fairly, we extended all six baselines – not just SWB\-DM – to the same 25\-round budget, covering the full CIFAR\-10 grid and the CIFAR\-100 subset\. Table[IV](https://arxiv.org/html/2609.16099#S7.T4)reports the round\-10\-to\-round\-25 accuracy gain for every method\.

TABLE IV:Mean accuracy gain \(percentage points\) from round 10 to round 25, all seven methods, CIFAR\-10 \(full grid\) and CIFAR\-100 \(four\-attack subset\)\.Every method benefits from additional rounds\. On CIFAR\-10, SWB\-DM’s gain is1\.71\.7–5\.4×5\.4\\timeslarger than any baseline’s, confirming that its warm\-up cost exceeds what continued training alone accounts for\. CIFAR\-100 shows a different pattern: FLTrust’s gain \(\+14\.19\+14\.19\) exceeds SWB\-DM’s \(\+10\.39\+10\.39\)\. FLTrust’s server\-side reference model requires more training iterations to mature on a 100\-class task, which is independent of the caching mechanism\. The large warm\-up effect observed on CIFAR\-10 does not transfer uniformly to other datasets\.

Krum is the other outlier in Table[IV](https://arxiv.org/html/2609.16099#S7.T4)\. Its CIFAR\-100 gain is negligible \(\+0\.93\+0\.93\), and as discussed in Section[VII\-B](https://arxiv.org/html/2609.16099#S7.SS2), this near\-zero average conceals a divergence in one of its two seeds\. Krum’s poor CIFAR\-100 performance reflects the same structural fragility documented above, not a warm\-up cost that additional rounds can resolve\.

### VII\-FScalability at 500 Clients

Table[V](https://arxiv.org/html/2609.16099#S7.T5)reports results for a larger deployment withN=500N\{=\}500clients,p=0\.1p\{=\}0\.1\(so 50 are sampled per round\), over 10 rounds on clean data\. The final two rows also show what happens when we extend the two caching methods out to 40 rounds\.

TABLE V:500\-client results: 10\-round main run, and 40\-round extension for the two caching methods\. Wall\-clock and per\-round aggregation time \(Wall, Agg\) refer to the 10\-round run in every row; the last two rows’ accuracy column additionally reports the 40\-round endpoint\.The aggregation cost column captures each method’s computational overhead, ranging from FedAvg’s lightweight average \(0\.01s per round\) all the way to SWB\-DM’s full 500\-client chunked\-rotation pipeline \(13\.35s\)\. Both caching methods hover near chance level at 10 rounds but recover substantially by round 40, confirming that the mechanism still works at this larger scale\. That said, SWB\-DM’s recovery is*noisier and less complete*than DelayedMomentum’s \(17\.16% vs\. 20\.61%, with a dip from 18\.14% at round 35\) – the reverse of what we see at every smaller scale in this paper, where SWB’s trimmed statistic consistently beat plain median under the same cache\. We leave this as an open question: SWB’s chunked rotation may introduce per\-aggregation variance that becomes more damaging when each cache slot only refreshes on average once everyN/n=10N/n\{=\}10rounds\. Separately, Krum’s near\-chance clean\-data result \(10\.10%\) reflects the same single\-client\-selection inefficiency we have seen throughout, made worse by the22×22\\timesspread in client data sizes at this scale; extending to 40 rounds brings it to 23\.09%, but non\-monotonically \(a dip to 11\.59% at round 20\), consistent with occasionally selecting an unrepresentative, data\-poor client\.

## VIIIDiscussion and Limitations

The failure modes documented above — convergence to a fixed wrong answer \(Median\), numerical divergence \(Krum\), and sharp precondition boundaries \(Bulyan\) — are distinct in mechanism, though they can produce similar aggregate accuracy numbers\. Distinguishing them required per\-round tracking of parameter norms and prediction histograms rather than final accuracy alone\.

Limitations\.We identify two primary limitations of SWB\-DM\.

The first is the medoid gauge\-fixing step\. It held up across every configuration we tested, but we don’t have a proof that it has to\. Picture an adversarial case where the medoid client’s own permutation is corrupted — that corruption could ride straight through into the aggregate\. Nothing in our runs actually did this\. Whether it’s fragile in that specific way is just something we can’t rule out yet\.

The second is a recovery gap at 500 clients \(Section[VII\-F](https://arxiv.org/html/2609.16099#S7.SS6)\)\. Every smaller scale in this paper tells the same story: SWB’s trimmed statistic beats plain median under the same cache, consistently\. Then at 500 clients that story flips\. DelayedMomentum climbs to20\.61%20\.61\\%; SWB\-DM gets stuck at17\.16%17\.16\\%and even dips non\-monotonically around round 35\. Our best guess is that chunked rotation is the culprit — each cache slot only refreshes about once everyN/n=10N/n\{=\}10rounds at this scale, and stale entries may end up amplifying noise instead of averaging it away\. We don’t have a clean proof of this yet, so we’re leaving it as an open question rather than a claim\.

A few smaller limitations round these out\. Atp=0\.5p\{=\}0\.5with low corruption, FedAvg, Bulyan, and Median all beat SWB\-DM \(Table[I](https://arxiv.org/html/2609.16099#S7.T1)\) — the warm\-up cost from Section[VII\-E](https://arxiv.org/html/2609.16099#S7.SS5)biting harder at higher participation\. Our bound \(Eq\.[1](https://arxiv.org/html/2609.16099#S5.E1)\) borrows its functional form from robust\-mean\-estimation theory rather than being derived specifically for SWB; a tighter, SWB\-specific bound is future work\. The adaptive attacks we test assume the attacker knows the benign updates in a round but not the cache state itself — a cache\-aware attacker would be a natural next step\. Further per\-round diagnostics are in the repository \([https://github\.com/saranrajsnkr/SWB\-DM\-full\-empirical\-study](https://github.com/saranrajsnkr/SWB-DM-full-empirical-study)\)\.

## IXConclusion

We introduced SWB\-DM, combining a randomized\-slicing aggregator inspired by sliced\-Wasserstein barycenters with delayed\-momentum caching\. Across 448 CIFAR\-10 configurations, CIFAR\-100, FEMNIST, and a 500\-client scalability study, we observe that robust aggregation guarantees depend on finite\-sample assumptions that partial participation can violate\. The resulting failure modes — convergence to a fixed wrong answer, numerical divergence, sharp precondition boundaries — are distinct and require per\-round diagnostic tracing to differentiate\. SWB\-DM’s caching introduces a warm\-up cost that on CIFAR\-10 exceeds all baselines extended to the same round budget\. The 500\-client recovery gap and the lack of a formal guarantee for medoid gauge\-fixing remain open\. Designing adaptive attacks that exploit the caching mechanism directly is a direction for future work\.

## References

- \[1\]H\. B\. McMahan, E\. Moore, D\. Ramage, S\. Hampson, and B\. A\. y Arcas, “Communication\-efficient learning of deep networks from decentralized data,” in*Proc\. AISTATS*, 2017\.
- \[2\]P\. Blanchard, E\. M\. El Mhamdi, R\. Guerraoui, and J\. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” in*Proc\. NeurIPS*, 2017\.
- \[3\]D\. Yin, Y\. Chen, R\. Kannan, and P\. Bartlett, “Byzantine\-robust distributed learning: Towards optimal statistical rates,” in*Proc\. ICML*, 2018\.
- \[4\]E\. M\. El Mhamdi, R\. Guerraoui, and S\. Rouault, “The hidden vulnerability of distributed learning in Byzantium,” in*Proc\. ICML*, 2018\.
- \[5\]X\. Cao, M\. Fang, J\. Liu, and N\. Z\. Gong, “FLTrust: Byzantine\-robust federated learning via trust bootstrapping,” in*Proc\. NDSS*, 2021\.
- \[6\]G\. Baruch, M\. Baruch, and Y\. Goldberg, “A little is enough: Circumventing defenses for distributed learning,” in*Proc\. NeurIPS*, 2019\.
- \[7\]C\. Xie, O\. Koyejo, and I\. Gupta, “Fall of empires: Breaking Byzantine\-tolerant SGD by inner product manipulation,” in*Proc\. UAI*, 2020\.
- \[8\]R\. Otsuka, K\. Takezawa, and M\. Yamada, “Delayed momentum aggregation: Byzantine\-robust federated learning with partial client participation,” arXiv:2509\.02970, 2025\.
- \[9\]G\. Lugosi and S\. Mendelson, “Mean estimation and regression under heavy\-tailed distributions: A survey,”*Foundations of Computational Mathematics*, 2019\.
- \[10\]N\. Bonneel, J\. Rabin, G\. Peyré, and H\. Pfister, “Sliced and Radon Wasserstein barycenters of measures,”*Journal of Mathematical Imaging and Vision*, 2015\.