Adaptive Phase-Switching for Communication-Efficient Federated LoRA Fine-Tuning

# Adaptive Phase-Switching for Communication-Efficient Federated LoRA Fine-Tuning
Source: [https://arxiv.org/html/2609.13512](https://arxiv.org/html/2609.13512)
\{IEEEkeywords\}

Communication efficiency, federated learning, large language models, low\-rank adaptation, parameter\-efficient fine\-tuning\.

###### Abstract

Federated fine\-tuning of large language models with low\-rank adaptation reduces per\-client trainable parameters, but client\-to\-server communication remains the dominant cost\. Existing accounting for federated LoRA protocols omits the asymmetric transition round when a protocol changes aggregation mode, and reports savings that ignore grouped\-query attention shapes\. This paper measures per\-round upload and download bytes for a bidirectional B\-only federated LoRA protocol and places five methods, three from prior work, on a single communication\-quality frontier\. The frontier has a knee, which an adaptive phase\-switching aggregator, ReverseAdaptive, locates by monitoring the relative improvement in global training loss against a dimensionless threshold rather than by fixing a phase boundary in advance\. On TinyLlama\-1\.1B\-Chat with Alpaca, ReverseAdaptive attains 40\.5 percent measured round\-trip savings over FLoRA at a held\-out instruction\-following loss cost of 0\.0063\. It outperforms FFA\-LoRA, which freezes the first of the two LoRA factors at initialization, by 0\.0182 in held\-out loss, more than twenty times the largest per\-method seed standard deviation on that metric, so learning that factor before freezing it produces better adapters\. The same threshold transfers across model scales without retuning, and the quality cost of the transition is stable across the two datasets tested\.

††corresponding:Corresponding author: Jerry Adams Franklin \(email: jerry\.adamsf@gmail\.com\)\. ORCID: 0009\-0006\-8470\-8349\.††note:This work received no external funding\. The author declares no competing financial interests\. This manuscript has been submitted to the IEEE Open Journal of the Computer Society for peer review\.## 1INTRODUCTION

\\IEEEPARstart

Federated learning enables model training across distributed clients without sharing raw data, making it attractive for privacy\-sensitive applications of LLMs\[[1](https://arxiv.org/html/2609.13512#bib.bib1)\]\. Fine\-tuning with LoRA\[[2](https://arxiv.org/html/2609.13512#bib.bib2)\]substantially reduces the number of trainable parameters per client, but federated LoRA aggregation still transmits nontrivial bytes per round\. With 10 clients participating in every round across 15 rounds, the cumulative cost determines whether a protocol is deployable in bandwidth\-constrained settings\.

Existing federated LoRA methods differ in what they aggregate and when\. FedIT\[[3](https://arxiv.org/html/2609.13512#bib.bib3)\]applies FedAvg\[[4](https://arxiv.org/html/2609.13512#bib.bib4)\]directly to LoRA’sAAandBBmatrices, paying full bidirectional cost per round\. FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]freezesAAat its initialization so that clients train and transmit onlyBBfrom the first round, giving a theoretical 50% upload saving\. FLoRA\[[6](https://arxiv.org/html/2609.13512#bib.bib6)\]stacks client LoRA modules and compresses via SVD, preserving product\-space semantics at full bidirectional cost\. FlexLoRA\[[7](https://arxiv.org/html/2609.13512#bib.bib7)\]supports heterogeneous client ranks and is orthogonal to the communication protocol studied here\. These four report communication savings as parameter\-count ratios rather than measured bytes; more recent work does measure transmitted volume directly\[[8](https://arxiv.org/html/2609.13512#bib.bib8),[9](https://arxiv.org/html/2609.13512#bib.bib9)\]\. What remains uncharacterized is the asymmetric round that occurs when a protocol transitions between aggregation modes, and the use of an adaptive signal to select when that transition should happen\.

This paper addresses these gaps with three contributions\.

Bidirectional B\-only protocol with measured byte tracking\.The protocol implements both upload and download B\-only transmission for Two\-Phase and ReverseAdaptive aggregators\. Per\-round upload and download megabytes are recorded from the tensors actually transmitted\. The transition round, in which the server seeds its frozenAAfrom client uploads, is accounted for explicitly, producing a one\-round asymmetry between when download savings begin and when upload savings begin\.

ReverseAdaptive: adaptive phase\-switching aggregator\.The aggregator starts in FLoRA mode and monitors the relative per\-round improvement in global training loss\. When that relative improvement falls below a dimensionless thresholdτ\\taufor the first time after a warmup period, the aggregator switches to FFA\-LoRA permanently\. Varyingτ\\tautraverses the same communication\-quality frontier as the fixed\-KKTwo\-Phase family without requiringKKto be selected in advance\.

A five\-method frontier and its knee\.Five protocols, three from prior work, are placed on one measured frontier on Alpaca at TinyLlama\-1\.1B scale; three of them are also compared on Dolly\-15k and at LLaMA\-3\.2\-3B\. Quality is evaluated by held\-out instruction\-following loss rather than by zero\-shot benchmarks, which do not discriminate between protocols at the smaller scale\. The frontier has a knee at ReverseAdaptive: savings beyond that point cost roughly five times more quality per point\.

Headline numbers:Two\-PhaseK=8K\{=\}8saves 27\.7% \(TinyLlama\-1\.1B\) and 26\.0% \(LLaMA\-3\.2\-3B\) over FLoRA\. ReverseAdaptive saves 40\.5% and30\.0±4\.030\.0\\pm 4\.0% respectively, and FFA\-LoRA saves 61\.9% at TinyLlama scale for 0\.0182 more held\-out loss than ReverseAdaptive\. At LLaMA\-3\.2\-3B, ReverseAdaptive and Two\-PhaseK=8K\{=\}8differ in final loss by1\.25×10−51\.25\\times 10^\{\-5\}, roughly 350 times less than at 1\.1B\.

Code, configuration files, and scripts to reproduce every figure and table are publicly released; see Appendix[11](https://arxiv.org/html/2609.13512#S11)\.

## 2RELATED WORK

### 2\.1LORA AND PARAMETER\-EFFICIENT FINE\-TUNING

Parameter\-efficient fine\-tuning \(PEFT\) methods reduce the cost of adapting large pretrained models by training only a small subset of parameters\. Adapter tuning\[[10](https://arxiv.org/html/2609.13512#bib.bib10)\]inserts small trainable modules between transformer layers\. LoRA\[[2](https://arxiv.org/html/2609.13512#bib.bib2)\]decomposes weight updates into low\-rank productsB​ABA, whereBBisd×rd\\times randAAisr×kr\\times kwithrrmuch smaller thanmin⁡\(d,k\)\\min\(d,k\)\. This reduces trainable parameters substantially, enabling fine\-tuning of large models without modifying base weights\. QLoRA\[[11](https://arxiv.org/html/2609.13512#bib.bib11)\]further reduces memory by quantizing the frozen base model to 4\-bit while training LoRA adapters in 16\-bit\. The resulting LoRA adapters are compact and composable, making them a natural fit for federated settings where both storage and communication are constrained\.

### 2\.2FEDERATED LORA AGGREGATION

FedIT\[[3](https://arxiv.org/html/2609.13512#bib.bib3)\]applies standard FedAvg\[[4](https://arxiv.org/html/2609.13512#bib.bib4)\]to LoRA matrices: the globalAAandBBare computed as weighted averages of clientAAandBBmatrices\. The aggregation introduces a mismatch: the optimal global update is the productB​ABA, not a product of separately averagedBBandAA\. Averaging independently introduces aggregation error that grows with client heterogeneity\.

FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]addresses the aggregation mismatch by freezingAAat initialization and training onlyBBacross clients\. BecauseAAis constant, the server aggregatesBBmatrices directly without product\-space distortion\. The theoretical upload saving is 50%\. The original work also motivates the frozen\-AAdesign in terms of differential privacy\. A critical difference from the protocol in this paper: FFA\-LoRA freezesAAat initialization, whereas the bidirectional B\-only protocol allowsAAto be learned through a FLoRA phase before freezing, recovering expressivity at the cost of longer initial overhead\.

FLoRA\[[6](https://arxiv.org/html/2609.13512#bib.bib6)\]preserves product\-space semantics by stacking client LoRA modules horizontally and applying truncated SVD to recover a global low\-rank adapter, reducing the aggregation bias present in FedIT\. The tradeoff is communication cost: FLoRA transmits fullA\+BA\{\+\}Bpayloads in both directions every round\. FLoRA is the full\-state baseline in this paper’s experiments\. FedSA\-LoRA\[[12](https://arxiv.org/html/2609.13512#bib.bib12)\]makes the opposite choice, sharing onlyAAon the grounds thatAAcarries general knowledge whileBBis client\-specific; under the byte accounting of Section[3](https://arxiv.org/html/2609.13512#S3)\-[3\.4](https://arxiv.org/html/2609.13512#S3.SS4), sharingAAinstead ofBBtransmits 64% of the full TinyLlama state rather than 36%\.

FlexLoRA\[[7](https://arxiv.org/html/2609.13512#bib.bib7)\]extends federated LoRA to the heterogeneous\-rank setting, where different clients use different LoRA ranks based on local compute budgets\. FlexLoRA’s contribution is orthogonal to this paper’s; the bidirectional B\-only protocol applies equally to homogeneous\-rank clients\.

### 2\.3COMMUNICATION\-EFFICIENT FEDERATED LEARNING

Communication efficiency in federated learning has been a central concern since FedAvg\[[4](https://arxiv.org/html/2609.13512#bib.bib4)\]introduced local multi\-step updates to reduce communication rounds\. The broader challenge of open problems in federated learning is surveyed by Kairouz et al\.\[[1](https://arxiv.org/html/2609.13512#bib.bib1)\]\. Subsequent work has pursued per\-round payload reduction through gradient compression\. QSGD\[[13](https://arxiv.org/html/2609.13512#bib.bib13)\]reduces the bit\-width of transmitted gradients through stochastic quantization\. Deep gradient compression\[[14](https://arxiv.org/html/2609.13512#bib.bib14)\]applies aggressive sparsification, sending only 0\.1% of gradients per round with error feedback\. FedPAQ\[[15](https://arxiv.org/html/2609.13512#bib.bib15)\]combines periodic averaging with quantization, showing that 8\-bit transmission incurs negligible accuracy loss\. These techniques are orthogonal to the B\-only protocol studied here and could in principle be composed with it for additional savings\.

### 2\.4ADAPTIVE AGGREGATION AND SCHEDULING

Several federated learning methods adapt their behavior based on observed training dynamics\. FedProx\[[16](https://arxiv.org/html/2609.13512#bib.bib16)\]adds an adaptive proximal term to the local objective, controlling deviation from the global model\. FedNova\[[17](https://arxiv.org/html/2609.13512#bib.bib17)\]normalizes local gradients by effective step count, adaptively correcting for heterogeneous local training\. FedAdam and FedYogi\[[18](https://arxiv.org/html/2609.13512#bib.bib18)\]apply adaptive moment estimation at the server level, treating the aggregated pseudo\-gradient as input to Adam\-style updates\. Curriculum scheduling in federated learning adjusts the aggregation frequency over training, structurally related to ReverseAdaptive’s phase\-switching in that both methods behave differently early vs\. late in training\. To the author’s knowledge, no prior federated LoRA work uses a loss\-plateau signal to switch between distinct aggregation modes during training\.

## 3METHOD

### 3\.1BACKGROUND AND NOTATION

Each ofNNclients holds a local dataset\. A shared base model has weight matricesWW\. LoRA\[[2](https://arxiv.org/html/2609.13512#bib.bib2)\]parameterizes updates asΔ​W=B​A\\Delta W=BAwhereBBisd×rd\\times randAAisr×kr\\times k\. In each federated round, clients train their local adapters for one epoch, upload some representation to the server, and receive a global state\.

The communication cost of a round is the sum of upload bytes and download bytes\. The B\-only fraction of the full state is not the 50% a naive parameter count suggests, because grouped\-query attention fixes the ratio ofBBtoAAparameters; Section[3](https://arxiv.org/html/2609.13512#S3)\-[3\.4](https://arxiv.org/html/2609.13512#S3.SS4)derives the exact fractions of 36% for TinyLlama\-1\.1B and 40% for LLaMA\-3\.2\-3B\.

### 3\.2BIDIRECTIONAL B\-ONLY PROTOCOL

Phase 1 \(FLoRA, rounds 1 toKK\)\.The server runs FLoRA aggregation\[[6](https://arxiv.org/html/2609.13512#bib.bib6)\]: client states are stacked and compressed via SVD to produce a global\(Aglobal,Bglobal\)\(A\_\{\\text\{global\}\},B\_\{\\text\{global\}\}\)\. Both upload and download payloads are full state\.

Transition \(roundK\+1K\{\+\}1\)\.Clients still upload full state\(Ai,Bi\)\(A\_\{i\},B\_\{i\}\)because the server needs theAAmatrices to compute and cacheAfrozenA\_\{\\text\{frozen\}\}\. The server runs FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]aggregation for the first time, cachesAfrozenA\_\{\\text\{frozen\}\}, and broadcasts the full global state\.

Phase 2 steady state \(roundsK\+2K\{\+\}2onward\)\.Clients upload onlyBiB\_\{i\}; the server reconstructs the full state as\(Afrozen,Bi\)\(A\_\{\\text\{frozen\}\},B\_\{i\}\)before aggregation\. The server broadcasts onlyBglobalB\_\{\\text\{global\}\}; clients retainAfrozenA\_\{\\text\{frozen\}\}locally\.

Look\-ahead download accounting\.This produces a one\-round asymmetry: download savings begin one round before upload savings\. The experiment runner records this consistently, ensuring reported totals are accurate rather than optimistic\.

Algorithm[1](https://arxiv.org/html/2609.13512#alg1)specifies the server\-side decision logic per round\.

Algorithm 1Bidirectional B\-only protocol \(server, per round\)1:round

rr, phase boundary

KK, frozen\-

AAcache \(initially empty\)

2:upload mode, broadcast mode

3:if

r≤Kr\\leq Kthen⊳\\trianglerightFLoRA phase

4:upload

←\\leftarrowfull state

\(A,B\)\(A,B\)
5:broadcast

←\\leftarrowfull state

6:elseif

r=K\+1r=K\+1then⊳\\trianglerighttransition: full upload seedsAfrozenA\_\{\\text\{frozen\}\}

7:upload

←\\leftarrowfull state

\(A,B\)\(A,B\)
8:

Afrozen←A\_\{\\text\{frozen\}\}\\leftarrowAAfrom the first client’s uploaded state

9:broadcast

←\\leftarrowBBonly⊳\\trianglerightclients retainAAfrom roundKK

10:else⊳\\trianglerightFFA\-LoRA steady state

11:upload

←\\leftarrowBBonly⊳\\trianglerightserver prependsAfrozenA\_\{\\text\{frozen\}\}

12:broadcast

←\\leftarrowBBonly⊳\\trianglerightclients retainAfrozenA\_\{\\text\{frozen\}\}locally

13:endif

### 3\.3REVERSEADAPTIVE AGGREGATOR

ReverseAdaptive replaces the fixed boundaryKKwith an adaptive loss\-plateau signal\. After a warmup period ofWWrounds \(defaultW=5W\{=\}5\), it monitors the*relative*per\-round loss improvementρr=\(ℓr−1−ℓr\)/ℓr−1\\rho\_\{r\}=\(\\ell\_\{r\-1\}\-\\ell\_\{r\}\)/\\ell\_\{r\-1\}\. Whenρr<τ\\rho\_\{r\}<\\taufor the first time, the aggregator switches to FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]mode permanently and instantaneously, with no intermediate phase\. Becauseρr\\rho\_\{r\}is a ratio of losses,τ\\tauis a dimensionless fraction rather than a loss difference, and its interpretation does not depend on the absolute scale of the loss for a given model or dataset; Section[5](https://arxiv.org/html/2609.13512#S5)shows that this scale\-free construction is what allows a singleτ\\tauto transfer across model sizes without retuning\. A complementary stability detector reverts to FLoRA if loss increases by more than 10% after switching\. No revert occurred in any ReverseAdaptive run reported in this paper, across both datasets, both model scales, all three partition settings, and the full threshold ablation\.

The choice ofτ\\tautrades communication savings against final training loss\. Smallτ\\taudelays the switch; largeτ\\tautriggers it at the first eligible round after warmup\. Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.7](https://arxiv.org/html/2609.13512#S4.SS7)characterizes this tradeoff empirically\.

The loss\-plateau signal is chosen for its simplicity, its availability without additional instrumentation, and the fact that it tracks the quantity a practitioner optimizes\. Alternative signals such as the gradient norm ofAAor the singular value spectrum of stacked client matrices could provide more direct measures of whetherAAhas converged, and are untested here\.

Algorithm[2](https://arxiv.org/html/2609.13512#alg2)specifies the aggregator’s per\-round decision logic\.

Algorithm 2ReverseAdaptive aggregator \(per round\)1:round

rr, current loss

ℓr\\ell\_\{r\}, prior loss

ℓr−1\\ell\_\{r\-1\}, warmup

WW, threshold

τ\\tau, current mode

mm,client\_states

2:global\_state, updated mode

mm
3:

ρr←\(ℓr−1−ℓr\)/ℓr−1\\rho\_\{r\}\\leftarrow\(\\ell\_\{r\-1\}\-\\ell\_\{r\}\)\\,/\\,\\ell\_\{r\-1\}⊳\\trianglerightrelative improvement;τ\\taudimensionless

4:if

m=FLoRAm=\\text\{FLoRA\}and

r\>Wr\>Wand

ρr<τ\\rho\_\{r\}<\\tauthen

5:

m←FFA\-LoRAm\\leftarrow\\text\{FFA\-LoRA\}
6:log event

\(r,switch\_to\_ffa\)\(r,\\texttt\{switch\\\_to\\\_ffa\}\)
7:elseif

m=FFA\-LoRAm=\\text\{FFA\-LoRA\}and

ℓr\>1\.1⋅ℓr−1\\ell\_\{r\}\>1\.1\\cdot\\ell\_\{r\-1\}then⊳\\trianglerightstability revert

8:

m←FLoRAm\\leftarrow\\text\{FLoRA\}
9:reset

AfrozenA\_\{\\text\{frozen\}\}
10:log event

\(r,revert\_to\_flora\)\(r,\\texttt\{revert\\\_to\\\_flora\}\)
11:endif

12:if

m=FFA\-LoRAm=\\text\{FFA\-LoRA\}then

13:returnFFA\-LoRA\-Aggregate\(client\_states\)

14:else

15:returnFLoRA\-Aggregate\(client\_states\)

16:endif

### 3\.4IMPLEMENTATION DETAILS

Byte tracking\.Per\-round upload and download megabytes are recorded inresults\.json\. Byte counts use the dtype in which parameters are transmitted: float32 \(4 bytes/param\) for TinyLlama\-1\.1B and float16 \(2 bytes/param\) for LLaMA\-3\.2\-3B, whose LoRA parameters are trained in float32 and cast back to the base dtype for transmission\. A full TinyLlama LoRA state is 2,252,800 parameters over 22 layers, or 9,011,200 bytes per client per direction\. All models are optimized using AdamW\[[19](https://arxiv.org/html/2609.13512#bib.bib19)\]\.

Frozen\-AAprovenance\.At the transition round the server seedsAfrozenA\_\{\\text\{frozen\}\}from the first client’s uploaded state rather than from an average across clients\. Because every client begins that round from the same broadcast global state, this is the aggregated globalAAof the preceding round plus one epoch of local adaptation at learning rate10−410^\{\-4\}\. Client ordering is fixed for the duration of a run, so the choice is deterministic and reproducible\. Two\-Phase and ReverseAdaptive share this rule, since both dispatch to the same FFA\-LoRA aggregator at their boundary\. AveragingAAacross clients at the transition round is a plausible alternative that this work does not test\.

GQA B\-fraction\.For TinyLlama\-1\.1B with rankr=16r\{=\}16targeting\{q\_proj,v\_proj\}\\\{\\texttt\{q\\\_proj\},\\texttt\{v\\\_proj\}\\\}, the projections adapted in the original LoRA formulation\[[2](https://arxiv.org/html/2609.13512#bib.bib2)\], accounting for GQA in the value projection, the B\-only fraction of the full state is exactly 36% rather than the 50% a naive parameter count would suggest\. This fraction is invariant to the size of the target set, sinceq\_projando\_projshare a shape andk\_projandv\_projshare a shape under GQA; the ratio is fixed by the GQA configuration, not by how many projections are adapted\. LLaMA\-3\.2\-3B’s different GQA configuration yields exactly 40%\.

Partition mechanics\.Alpaca and Dolly\-15k carry no class labels, so non\-IID partitions apply Dirichlet skew over a task\-diversity proxy: each instructionssis assigned to one of ten buckets by character length,min⁡\(⌊\|s\|/50⌋,9\)\\min\(\\lfloor\|s\|/50\\rfloor,9\), and a Dirichlet distribution with concentrationα∈\{0\.5,0\.1\}\\alpha\\in\\\{0\.5,0\.1\\\}is applied over those buckets\. This induces heterogeneity in task form rather than task semantics; Section[6](https://arxiv.org/html/2609.13512#S6)notes that semantic heterogeneity is untested\.

## 4EXPERIMENTS AND RESULTS

### 4\.1SETUP

Table[1](https://arxiv.org/html/2609.13512#S4.T1)summarizes the experimental configuration\. The primary corpus of 34 runs was produced on a single Apple M4 Pro workstation, approximately 285 hours of compute, with no institutional cluster\. Those runs were subsequently reproduced on commercially rented NVIDIA hardware, an RTX 4090 for TinyLlama\-1\.1B and an A100 40 GB for LLaMA\-3\.2\-3B; the cross\-backend comparison is reported in Appendix[12](https://arxiv.org/html/2609.13512#S12)\. The Dolly\-15k replication and the FFA\-LoRA and FedIT baseline runs were produced on the same rented hardware\. Both TinyLlama\-1\.1B and LLaMA\-3\.2\-3B experiments use three seeds for IID training\.

Table 1:Experimental setup\.Baselines\.FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]and FedIT\[[3](https://arxiv.org/html/2609.13512#bib.bib3)\]are reimplemented within this codebase rather than obtained as released code from the original authors, and results for them should be read with that qualification\. Following Sun et al\., the FFA\-LoRA implementation freezesAAat initialization: client\-sideAAparameters are set non\-trainable before local optimization begins and are excluded from the optimizer parameter group, so clients train onlyBBfrom the first round\. Clients transmitBBonly, a measured 3,244,032 bytes per client per direction against 9,011,200 for fullA\+BA\{\+\}Btransmission\. The former is\(32768\+4096\)×22\(32768\+4096\)\\times 22parameters at 4 bytes each, exactly the B\-only payload, and could not arise ifAAwere being transmitted\. FedIT applies FedAvg independently toAAandBBand transmits full state in both directions, so its communication volume matches FLoRA’s by construction\.

Prompt format\.All examples, in training and in held\-out evaluation, are rendered with a single template:\#\#\# Instruction:followed by the instruction text, then\#\#\# Response:followed by the output, truncated and padded to 256 tokens\. Alpaca’s optionalinputfield, populated in roughly 40% of examples, is not included, and no separate template is applied to examples that carry it\. Loss is computed over the full formatted sequence including prompt tokens, with padding masked to−100\-100\. This differs from the two\-template formatting used in the original Alpaca release and shifts absolute loss values; because training and evaluation share the same code path, comparisons between methods are unaffected\. Note that if the tokenizer aliasespad\_tokentoeos\_token, the end\-of\-sequence token is masked along with padding; this is consistent across all runs but is relevant to anyone recomputing the base\-model reference values of Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.2](https://arxiv.org/html/2609.13512#S4.SS2)\.

Backend provenance\.Communication totals are protocol\-deterministic and identical across backends\. Loss values are not: Appendix[12](https://arxiv.org/html/2609.13512#S12)reports cross\-backend agreement within0\.010\.01for IID settings, which exceeds some of the between\-method differences reported below\. Comparisons in this paper are therefore made within a backend, and tables are labeled with their corpus\. The five\-method frontier \(Table[2](https://arxiv.org/html/2609.13512#S4.T2)\), the Dolly\-15k replication \(Table[3](https://arxiv.org/html/2609.13512#S4.T3)\), and all statistical tests use the CUDA corpus, since the FFA\-LoRA, FedIT, and Dolly runs exist only there\. The threshold ablation \(Table[6](https://arxiv.org/html/2609.13512#S4.T6)\) and the LLaMA\-3\.2\-3B results use MPS\. Communication values are comparable across tables; loss values are comparable only within a backend\.

### 4\.2HELD\-OUT INSTRUCTION\-FOLLOWING METRIC

Final training loss measures fit to the federated objective, and the four zero\-shot benchmarks do not discriminate between base and fine\-tuned checkpoints at this scale \(Section[5](https://arxiv.org/html/2609.13512#S5)\-[5\.4](https://arxiv.org/html/2609.13512#S5.SS4)\)\. Quality is therefore also reported as held\-out instruction\-following loss\. From each dataset we reserve the 500\-example slicetrain\[3000:3500\], disjoint from the 3000 examples partitioned across clients, and compute mean token\-level cross\-entropy under the prompt format described in Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.1](https://arxiv.org/html/2609.13512#S4.SS1)\. Each adapted checkpoint is scored against the unadapted base model, and we reportΔ​ℓheld=ℓtuned−ℓbase\\Delta\\ell\_\{\\mathrm\{held\}\}=\\ell\_\{\\mathrm\{tuned\}\}\-\\ell\_\{\\mathrm\{base\}\}, so more negative values indicate better instruction following\. Base\-model reference values are 1\.9352 \(perplexity 6\.9254\) on Alpaca and 2\.2608 \(perplexity 9\.5905\) on Dolly\-15k\. Because the base values differ by dataset, held\-out deltas are comparable within a dataset but not across datasets; the cross\-dataset comparison in Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.4](https://arxiv.org/html/2609.13512#S4.SS4)therefore compares between\-method gaps rather than levels\.

### 4\.3THE COMMUNICATION\-QUALITY FRONTIER

Table[2](https://arxiv.org/html/2609.13512#S4.T2)and Figure[1](https://arxiv.org/html/2609.13512#S4.F1)place five federated LoRA protocols on a single measured communication\-quality frontier for TinyLlama\-1\.1B\[[20](https://arxiv.org/html/2609.13512#bib.bib20)\]under IID partitioning\. The five methods occupy four distinct operating points\. FLoRA\[[6](https://arxiv.org/html/2609.13512#bib.bib6)\]and FedIT\[[3](https://arxiv.org/html/2609.13512#bib.bib3)\]coincide at 2578\.13 MB, since both transmit fullAAandBBin both directions every round\. Two\-PhaseK=8K\{=\}8reaches 1863\.13 MB \(27\.73% savings\), ReverseAdaptive atτ=0\.01\\tau\{=\}0\.01reaches 1533\.13 MB \(40\.53%\), and FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]reaches 983\.13 MB \(61\.87%\)\. Across the four distinct points communication falls and both quality metrics degrade in the same order, so the frontier is monotone on each metric independently rather than only on the metric used to construct it\.

Figure 1:Measured communication\-quality frontier on Alpaca\-3k, TinyLlama\-1\.1B, IID, CUDA corpus\. Five protocols, three from prior work, at four distinct operating points; error bars are one sample standard deviation over three seeds, and communication is identical across seeds\. Held\-outΔ\\Deltaloss is tuned minus base on the 500\-example slice defined in Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.2](https://arxiv.org/html/2609.13512#S4.SS2); more negative is better\. FLoRA and FedIT coincide at lower right\. The segment from ReverseAdaptive to FFA\-LoRA is markedly steeper than the segment preceding it, which Section[5](https://arxiv.org/html/2609.13512#S5)quantifies\.Three features of Table[2](https://arxiv.org/html/2609.13512#S4.T2)bear on the interpretation\. The two quality metrics agree on the ordering of every pair they can resolve: across the four distinct operating points the ranking is identical on final training loss and on held\-out instruction\-following loss\. The exception is the FLoRA and FedIT pair, which shares an operating point, where FedIT is nominally better on final loss and FLoRA nominally better on held\-out loss; neither difference is statistically distinguishable \(p=0\.588p\{=\}0\.588andp=0\.425p\{=\}0\.425\)\. The shape of the frontier is not an artifact of the metric chosen to construct it\.

ReverseAdaptive outperforms FFA\-LoRA by 0\.0282 on final loss and 0\.0182 on held\-out loss, against per\-method seed standard deviations of at most 0\.0016 and 0\.0008 respectively\. Both protocols end training withAAfrozen and onlyBBaggregated, and they differ only in whetherAAwas learned first\. LearningAAthrough a FLoRA phase before freezing it produces materially better adapters than freezingAAat initialization, at a cost of 550 MB in this configuration\.

FedIT lands on FLoRA’s communication volume exactly, to the byte, and within 0\.0006 on both quality metrics\. It contributes no new operating point, which is the intended result: FedIT is an independently implemented full\-state protocol, so its coincidence with FLoRA is evidence that the byte accounting measures the protocol rather than an implementation artifact\.

Loss differences between methods are small in absolute terms, 0\.0141 between FLoRA and ReverseAdaptive atτ=0\.01\\tau\{=\}0\.01, but are resolved across three seeds because the per\-seed spread is more than twenty times smaller\. Pairedtt\-tests on final loss givep<10−3p<10^\{\-3\}for every comparison against FLoRA except FedIT, which is correctly indistinguishable\. With three seeds these tests have very low power, so effect sizes should be read alongside thepp\-values, and no multiplicity correction is applied in Table[2](https://arxiv.org/html/2609.13512#S4.T2); Appendix[10](https://arxiv.org/html/2609.13512#S10)reports the complete set of paired tests on both metrics with the correction stated\.

Table 2:Measured communication\-quality frontier\. TinyLlama\-1\.1B, Alpaca\-3k, IID, CUDA corpus, three seeds, mean±\\pmone sample standard deviation\. Communication is protocol\-deterministic and identical across seeds\. Held\-outΔ\\Deltaloss is tuned minus base on the 500\-example slice of Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.2](https://arxiv.org/html/2609.13512#S4.SS2); more negative is better\. Differences quoted in the text are computed from unrounded values and may differ from the displayed cells in the final digit\.pp\-values are pairedtt\-tests against FLoRA on final loss; the complete set of paired tests on both metrics is in Appendix[10](https://arxiv.org/html/2609.13512#S10)\. Methods: FLoRA\[[6](https://arxiv.org/html/2609.13512#bib.bib6)\], FedIT\[[3](https://arxiv.org/html/2609.13512#bib.bib3)\], FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]\. The MB column is total round\-trip communication over 15 rounds\.This frontier sweeps through the fixed\-KKTwo\-Phase operating points: on the MPS corpus,τ=0\.002\\tau\{=\}0\.002lands on Two\-PhaseK=8K\{=\}8communication andτ=0\.001\\tau\{=\}0\.001onK=10K\{=\}10\. Because total communication is a step function of the discrete switch round, at 110 MB per round, this correspondence is not backend\-invariant; Appendix[12](https://arxiv.org/html/2609.13512#S12)reports that both tight\-τ\\tausettings fire one round earlier on CUDA\. The robust claim is that the adaptive method traverses the same frontier as the fixed\-KKfamily without requiringKKto be specified, rather than that it reproduces particularKKvalues byte for byte\.

### 4\.4REPLICATION ON DOLLY\-15K

Table[3](https://arxiv.org/html/2609.13512#S4.T3)repeats the three\-method comparison on Databricks Dolly\-15k\[[23](https://arxiv.org/html/2609.13512#bib.bib23)\], an instruction dataset with a different length distribution and a broader task mix than Alpaca\. Communication is identical to the Alpaca corpus, since the protocols are dataset\-independent and ReverseAdaptive fired at round 6 on every IID seed of both datasets\. The quantity of interest is therefore the quality cost of switching rather than the absolute loss\.

Held\-out deltas are not comparable across datasets, since the base model scores 1\.9352 on the Alpaca slice and 2\.2608 on the Dolly slice\. Table[3](https://arxiv.org/html/2609.13512#S4.T3)reports the gap to each dataset’s own FLoRA baseline, which is comparable\. ReverseAdaptive costs 0\.0063 held\-out loss on Alpaca and 0\.0061 on Dolly, a shift of 0\.0002, or 3\.0% of the gap’s own magnitude\. The quality cost of the adaptive transition is stable across these two datasets\.

Two\-PhaseK=8K\{=\}8costs 0\.0034 on Alpaca and 0\.0043 on Dolly, a shift of 25\.5%\. We do not read this as evidence that fixed\-KKswitching transfers less well than adaptive switching\. Three seeds on two datasets cannot resolve a difference in stability between two methods, and the claim supported here is limited to ReverseAdaptive’s own cost being stable, not to a property of the protocol family\.

A non\-IID Dolly run atα=0\.5\\alpha\{=\}0\.5switched at round 6 on all three seeds and reached the same 1533\.13 MB, matching the IID behavior on both datasets\. No Dolly FLoRA baseline was run under heterogeneity, so no quality cost is reported for that configuration\.

Table 3:Dolly\-15k replication\. TinyLlama\-1\.1B, IID, CUDA corpus, three seeds, mean±\\pmone sample standard deviation\. Gap is the held\-outΔ\\Deltaloss difference against the same dataset’s FLoRA baseline, which is the quantity comparable across datasets; held\-out levels are not, since base\-model losses differ \(1\.9352 on Alpaca, 2\.2608 on Dolly\-15k\)\. Differences quoted in the text are computed from unrounded values and may differ from the displayed cells in the final digit\. Methods: FLoRA\[[6](https://arxiv.org/html/2609.13512#bib.bib6)\], FedIT\[[3](https://arxiv.org/html/2609.13512#bib.bib3)\], FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]\. The MB column is total round\-trip communication over 15 rounds\.
### 4\.5CONVERGENCE AND SWITCHING BEHAVIOR

Figure[2](https://arxiv.org/html/2609.13512#S4.F2)shows mean±\\pmstd loss trajectories across three seeds for FLoRA, Two\-PhaseK=8K\{=\}8, and ReverseAdaptive\. All three methods converge similarly through the FLoRA phase\. Phase transitions are visible as slight slope changes but do not destabilize training\. The final∼0\.01\{\\sim\}0\.01loss gap accumulates gradually after the switch rather than appearing as a sharp discontinuity\.

Figure 2:Training loss trajectories on TinyLlama\-1\.1B IID, MPS corpus\. Mean±\\pm1 standard deviation over three seeds\. Per\-round trajectories were logged on the MPS corpus; final\-round values elsewhere in this paper are from CUDA\. Vertical dashed lines mark ReverseAdaptive’s switch round \(round 6\) and Two\-PhaseK=8K\{=\}8’s phase boundary \(round 8; the transition round, in which the upload is still full, is round 9\)\. All three methods converge similarly through the FLoRA phase; the loss gap accumulates gradually after the switch\.Figure[3](https://arxiv.org/html/2609.13512#S4.F3)shows cumulative communication in megabytes against round number for TinyLlama\-1\.1B\. FLoRA stays linear at 85\.94 MB per round per direction throughout\. Two\-PhaseK=8K\{=\}8maintains the same slope through round 9, then reduces as B\-only transmission activates\. ReverseAdaptive splits off at round 6 \(at TinyLlama\-1\.1B scale\), producing the steepest savings trajectory\.

Figure 3:Cumulative communication on TinyLlama\-1\.1B \(seed 42, deterministic across seeds\)\. Slope changes mark phase transitions: ReverseAdaptive at round 6, Two\-PhaseK=8K\{=\}8at round 9\. The look\-ahead download accounting causes the broadcast slope to change one round before the upload slope changes\.The no\-switch sanity baseline \(ReverseAdaptive withswitch\_threshold=−1\.0=\-1\.0,warmup\_rounds=999=999\) produces final loss 1\.2594342812 on seed 42 under the MPS corpus, matching the corresponding FLoRA run to 10 decimal places\. The CUDA per\-seed values in Appendix[9](https://arxiv.org/html/2609.13512#S9)differ, as expected across backends; the guarantee is a within\-backend statement\. Upload and download match to the byte across all 15 rounds\. Disabling the switch requires a negative threshold rather than a small positive one, for reasons set out in Appendix[14](https://arxiv.org/html/2609.13512#S14)\.

### 4\.6DOWNSTREAM EVALUATION

Tables[4](https://arxiv.org/html/2609.13512#S4.T4)–[5](https://arxiv.org/html/2609.13512#S4.T5)present downstream results on MMLU\[[24](https://arxiv.org/html/2609.13512#bib.bib24)\], ARC\-Easy\[[25](https://arxiv.org/html/2609.13512#bib.bib25)\], BoolQ\[[26](https://arxiv.org/html/2609.13512#bib.bib26)\], and HellaSwag\[[27](https://arxiv.org/html/2609.13512#bib.bib27)\]\. MMLU is excluded from TinyLlama\-1\.1B comparisons: base accuracy of 0\.250 on 4\-way multiple choice indicates chance\-level performance, making the benchmark uninformative at 1\.1B scale\.

Table 4:Downstream zero\-shot accuracy, TinyLlama\-1\.1B\. MMLU omitted: base accuracy is at chance for 4\-way multiple choice\.Table 5:Downstream zero\-shot accuracy, LLaMA\-3\.2\-3B\. Mean±\\pm1 standard deviation across three random seeds \(42, 123, 456\)\. 500 examples per benchmark\. Cross\-method spread is at most 0\.5 percentage points on any benchmark and the per\-method error bars overlap\.At TinyLlama\-1\.1B \(Table[4](https://arxiv.org/html/2609.13512#S4.T4)\), all federated methods cluster within 1\.0 percentage points of each other across the three informative benchmarks\. All methods regress modestly relative to the base model\. The base model’s BoolQ accuracy of 0\.626 closely matches BoolQ’s natural yes\-class rate of approximately 0\.62, suggesting the base exploits a yes\-biased prior\. Federated Alpaca\-3k instruction tuning attenuates this prior, producing regression across all methods uniformly\.

At LLaMA\-3\.2\-3B scale, the three federated methods are statistically indistinguishable on downstream benchmarks\. Mean accuracies across three seeds differ by at most 0\.5 percentage points on any benchmark, and the per\-method error bars overlap on every benchmark\. The picture differs from the TinyLlama panel in two ways\. The federated methods are flat to slightly positive relative to the base model at 3B rather than regressing as they did at 1\.1B; the LLaMA base achieves MMLU 0\.544, ARC\-Easy 0\.832, BoolQ 0\.706, and HellaSwag 0\.540, and federated methods either match or slightly exceed each of these\. The within\-method seed variance is also smaller at 3B than the within\-scale spread observed at TinyLlama, indicating that downstream benchmarks become more stable as model capacity grows\.

### 4\.7THRESHOLD ABLATION

Table[6](https://arxiv.org/html/2609.13512#S4.T6)and Figure[4](https://arxiv.org/html/2609.13512#S4.F4)characterize howτ\\taucontrols the savings\-quality tradeoff\. Sinceτ\\taubounds a relative rather than an absolute loss improvement, the values below are fractions of the previous round’s loss:τ=0\.01\\tau\{=\}0\.01requires the loss to fall by at least 1% in a round for the FLoRA phase to continue\. The method has a sensitive regime forτ≤0\.002\\tau\\leq 0\.002: atτ=0\.002\\tau\{=\}0\.002the switch occurs at round 9, matching Two\-PhaseK=8K\{=\}8; atτ=0\.001\\tau\{=\}0\.001the switch occurs at round 11, matching Two\-PhaseK=10K\{=\}10\. Forτ≥0\.005\\tau\\geq 0\.005the method saturates: the switch fires at round 6, the first round after warmup\. All rows forτ∈\{0\.005,0\.010,0\.020,0\.050,0\.100,0\.200\}\\tau\\in\\\{0\.005,0\.010,0\.020,0\.050,0\.100,0\.200\\\}produce identical results\.

Figure 4:ReverseAdaptive threshold sensitivity \(TinyLlama IID, seed 42, MPS corpus\)\. Communication is backend\-invariant; loss values are not comparable with the CUDA results in Table[2](https://arxiv.org/html/2609.13512#S4.T2)\.A practitioner seeking maximum savings can setτ\\tauanywhere in\[0\.005,0\.2\]\[0\.005,0\.2\]without precision loss\. Theτ=0\.001\\tau\{=\}0\.001andτ=0\.002\\tau\{=\}0\.002operating points demonstrate that the adaptive method genuinely tracks the loss trajectory rather than saturating trivially\.

Table 6:ReverseAdaptive threshold sensitivity \(TinyLlama IID, seed 42, MPS corpus\)\.
### 4\.8SCALE VALIDATION

Table[7](https://arxiv.org/html/2609.13512#S4.T7)and Figure[5](https://arxiv.org/html/2609.13512#S4.F5)confirm that the Pareto structure observed at TinyLlama\-1\.1B is preserved at LLaMA\-3\.2\-3B\. Three random seeds \(42, 123, 456\) were run for each of FLoRA, Two\-PhaseK=8K\{=\}8, and ReverseAdaptive, producing nine total runs\. Final\-round results \(mean±\\pmstandard deviation\): FLoRA reaches loss1\.3724±0\.00091\.3724\\pm 0\.0009with 2625\.0 MB total communication\. Two\-PhaseK=8K\{=\}8reaches loss1\.3938±0\.00111\.3938\\pm 0\.0011with 1942\.5 MB \(savings 26\.0%\)\. ReverseAdaptive reaches loss1\.3938±0\.00361\.3938\\pm 0\.0036with1837\.5±105\.01837\.5\\pm 105\.0MB \(savings30\.0±4\.030\.0\\pm 4\.0%\)\. The protocol\-deterministic communication for FLoRA and Two\-PhaseK=8K\{=\}8produces zero seed\-to\-seed variance; ReverseAdaptive’s variance reflects across\-seed differences in switch round \(7, 8, 9 across the three seeds, mean8\.0±1\.08\.0\\pm 1\.0\)\.

Figure 5:Scale validation\. Left: TinyLlama\-1\.1B, CUDA corpus \(mean±\\pm1 standard deviation over three seeds\)\. Right: LLaMA\-3\.2\-3B, MPS corpus \(mean±\\pm1 standard deviation over three seeds\)\. Both panels show the three federated methods on the same axes\. Dotted polyline connects the three method points on each panel\. The Pareto structure \(downward slope from FLoRA to ReverseAdaptive\) is preserved across scales\. Y\-axis ranges differ between panels because absolute training loss is model\-dependent; the relative Pareto shape is the invariant being validated\. ReverseAdaptive’s larger error bar at LLaMA\-3\.2\-3B reflects across\-seed variance in switch round \(7, 8, 9\)\.The communication savings are slightly lower at 3B than at 1\.1B: 26\.0% vs\. 27\.7% for Two\-PhaseK=8K\{=\}8, and 30\.0% vs\. 40\.5% for ReverseAdaptive\. The B\-only fraction of the full state is 40% at LLaMA\-3\.2\-3B \(vs\. 36% for TinyLlama\) due to different GQA configurations, meaning each B\-only round saves a smaller fraction of the full payload at 3B scale\.

At LLaMA\-3\.2\-3B, ReverseAdaptive and Two\-PhaseK=8K\{=\}8reach final losses differing by1\.25×10−51\.25\\times 10^\{\-5\}, roughly 350 times smaller than the 0\.0043 separating them at TinyLlama\-1\.1B\. A pairedtt\-test over three seeds givesp=0\.997p=0\.997with a 95% confidence interval of\[−0\.0114,0\.0114\]\[\-0\.0114,0\.0114\]\. That interval contains the 1\.1B effect size, so the test cannot distinguish no difference from a difference as large as the one observed at 1\.1B, and the result is reported as a statement about effect size rather than as evidence of equivalence\. ReverseAdaptive also saves 4\.0 percentage points more mean communication than the hand\-tuned configuration \(30\.0% against 26\.0%\), because it switches at round 7 on one of three seeds\. The same thresholdτ=0\.01\\tau\{=\}0\.01used at TinyLlama\-1\.1B produced this behavior with no scale\-specific tuning\.

Table 7:LLaMA\-3\.2\-3B IID results\. ReverseAdaptive usesτ=0\.01\\tau\{=\}0\.01\. The MB column is total round\-trip communication over 15 rounds and savings are against FLoRA\. Mean±\\pm1 standard deviation across three random seeds \(42, 123, 456\)\. ReverseAdaptive switch rounds: 7, 8, 9\. FLoRA and Two\-PhaseK=8K\{=\}8communication is protocol\-deterministic; ReverseAdaptive communication varies with switch round\.

## 5DISCUSSION

### 5\.1IMPLICATIONS FOR FEDERATED LLM DEPLOYMENT

In mobile or IoT federated settings where uplink bandwidth is scarce and metered, cutting round\-trip communication by 30 to 40% can change the economic viability of federated fine\-tuning\. At TinyLlama\-1\.1B the 40\.5% reduction costs 0\.0063 in held\-out instruction\-following loss\. Whether that trade is acceptable is a deployment decision rather than a universal one, and the frontier in Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.3](https://arxiv.org/html/2609.13512#S4.SS3)is intended to let a practitioner make it explicitly: the same measurements show that accepting a further 21\.3 points of savings costs roughly five times more quality per point\. Kairouz et al\.\[[1](https://arxiv.org/html/2609.13512#bib.bib1)\]identify communication as a fundamental open problem in federated learning at scale, and measured byte\-level reporting is a prerequisite for addressing it\.

### 5\.2CHOOSING AN OPERATING POINT

The Two\-Phase protocol requires a practitioner to selectKKbefore training begins, without knowing when the loss plateau will occur for a given model\-dataset combination\. This is a hyperparameter that must be tuned per dataset, per model, and potentially per client population: precisely the kind of manual configuration that slows deployment in production federated systems\. ReverseAdaptive replaces it by monitoring the signal thatKKwas meant to approximate\.

The criterion transfers across model scales because it is scale\-free\. Sinceτ\\taubounds a relative loss improvement, the same numerical value encodes the same stopping condition at any absolute loss level, so it carries across models whose losses differ in magnitude; an absolute improvement threshold would not, since a fixed loss delta that plateaus one model’s training would fire immediately or never on another\. Using the sameτ=0\.01\\tau\{=\}0\.01, ReverseAdaptive fires at round 6 on TinyLlama\-1\.1B and at a mean of round8\.0±1\.08\.0\\pm 1\.0on LLaMA\-3\.2\-3B, with no scale\-specific retuning\. The IID switch rounds at both scales match exactly across the MPS and CUDA backends \(see Appendix[12](https://arxiv.org/html/2609.13512#S12)\), so the cross\-scale comparison does not rest on corpora that disagree about when the switch occurred\.

Two qualifications bound this result\. At TinyLlama scale the switch fires at round 6 in every MPS run, across all three partition settings and both datasets, and round 6 is the first round after the warmup periodW=5W\{=\}5; on CUDA oneα=0\.1\\alpha\{=\}0\.1partition fires at round 7 instead, as Appendix[12](https://arxiv.org/html/2609.13512#S12)reports\. Atτ=0\.01\\tau\{=\}0\.01and this scale the adaptive rule is behaviorally equivalent to a fixedK=5K\{=\}5, and its adaptivity is visible only in the threshold ablation \(Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.7](https://arxiv.org/html/2609.13512#S4.SS7)\), whereτ≤0\.002\\tau\\leq 0\.002yields switch rounds of 9 and 11\. Across seeds, switch round varies only at LLaMA\-3\.2\-3B, where three runs give rounds 7, 8, and 9; those runs also differ in code revision \(Section[6](https://arxiv.org/html/2609.13512#S6)\), so they cannot separate training dynamics from environment\. Two model sizes and two datasets do not establish thatτ=0\.01\\tau\{=\}0\.01is a universal default; they establish that a single value carried across the two scales tested without retuning\.

At 3B the adaptive and hand\-tuned configurations reach final losses differing by1\.25×10−51\.25\\times 10^\{\-5\}, roughly 350 times smaller than the 0\.0043 that separates them at 1\.1B\. A pairedtt\-test over three seeds givesp=0\.997p=0\.997\. We report this as a statement about effect size rather than as evidence of equivalence: with three seeds the test has almost no power to detect a difference of the magnitude observed at 1\.1B, so a largepp\-value cannot establish that none exists\.

Where ReverseAdaptive sits on the frontier matters more than which fixedKKit happens to match\. Moving from FLoRA to ReverseAdaptive buys 40\.5 percentage points of communication savings at a held\-out cost of 0\.0063, or1\.56×10−41\.56\\times 10^\{\-4\}per point\. Moving onward to FFA\-LoRA buys a further 21\.3 points at a cost of 0\.0182, or8\.54×10−48\.54\\times 10^\{\-4\}per point, 5\.5 times more expensive per point saved\. The frontier has a knee and ReverseAdaptive sits at it\. A practitioner who takes the first tranche of savings and declines the second is making the trade the measured data supports, and the value of the adaptive rule is that it locates that point withoutKKbeing guessed in advance\.

### 5\.3MEASURED BYTES AND THEORETICAL PARAMETER COUNTS

Federated LoRA work commonly reports communication savings as parameter\-count ratios\[[3](https://arxiv.org/html/2609.13512#bib.bib3),[5](https://arxiv.org/html/2609.13512#bib.bib5),[6](https://arxiv.org/html/2609.13512#bib.bib6),[7](https://arxiv.org/html/2609.13512#bib.bib7)\], conflating how many parameters exist with how many bytes a protocol transmits\. The practice is not universal: FedSRD\[[8](https://arxiv.org/html/2609.13512#bib.bib8)\]and FLoRIST\[[9](https://arxiv.org/html/2609.13512#bib.bib9)\]both report measured per\-round communication in megabytes\. The contribution here is therefore not measurement as such, but the isolation of two quantities that a parameter ratio cannot express and that aggregate megabyte totals do not separate\.

Architectural choices such as GQA fix the B\-only fraction in ways a naive parameter count does not anticipate\. TinyLlama\-1\.1B’s GQA makes B\-only transmission save exactly 36% rather than the 50% a non\-GQA count suggests, and LLaMA\-3\.2\-3B’s different GQA configuration yields exactly 40%\. A theoretical 50% savings claim made for one model family does not transfer to another without architectural accounting\.

The transition round carries a cost that parameter\-ratio analyses omit\. At the FLoRA\-to\-FFA\-LoRA boundary the upload is full while the broadcast is already B\-only, so exactly one directional payload is upgraded from B\-only to full\. On TinyLlama this costs 55\.0 MB, and the figure does not depend on the switch round: totals that ignore the transition would be 928\.1, 1478\.1, and 1808\.1 MB for switch rounds 1, 6, and 9, against measured totals of 983\.1, 1533\.1, and 1863\.1\. What varies with the number of remaining rounds is the penalty’s share of the total, not its size\. A protocol upgrading both directions at the boundary would pay 110\.0 MB; the look\-ahead broadcast halves it\.

Total communication across the entire corpus is one closed\-form expression in one discrete parameter\. For a switching protocol with switch roundssoverNNrounds, uploads are full for rounds1​…​s1\\ldots sand broadcasts for rounds1​…​s−11\\ldots s\{\-\}1, so2​s−12s\-1directional payloads are full and the total is55​\(2​s−1\)\+928\.12555\(2s\-1\)\+928\.125MB on TinyLlama and52\.5​\(2​s−1\)\+105052\.5\(2s\-1\)\+1050MB on LLaMA\-3\.2\-3B\. FFA\-LoRA iss=1s\{=\}1, ReverseAdaptives=6s\{=\}6, Two\-PhaseK=8K\{=\}8iss=9s\{=\}9, and Two\-PhaseK=10K\{=\}10iss=11s\{=\}11; every measured total in this paper at full ten\-client participation lands on that lattice to the byte; the two reduced\-participation runs in Appendix[9](https://arxiv.org/html/2609.13512#S9)scale off the lattice in proportion to the active client count\. The2​s−12s\-1arises from the harvest rule rather than from the protocol: an implementation that froze the server’s existing globalAAwould upgrade no directional payload at the boundary and give110​\(s−1\)\+928\.125110\(s\-1\)\+928\.125instead\. A non\-switching protocol has all2​N2Ndirectional payloads full and is not a point on the lattice: FLoRA is the separate endpoint at55×30\+928\.125=2578\.12555\\times 30\+928\.125=2578\.125MB\. The lattice spacing of 110 MB per round is also why no continuousτ\\taureproduces a fixed\-KKtotal robustly, as Section[4](https://arxiv.org/html/2609.13512#S4)\-[4\.3](https://arxiv.org/html/2609.13512#S4.SS3)notes\.

One artifact of this implementation deserves disclosure, because it charges the strongest baseline too much\. FFA\-LoRA freezesAAat initialization, so every client’sAAequals the shared initial value at every round and never needs transmitting\. The measured 983\.1 MB nonetheless includes the 55\.0 MB seeding upload, because the server harvestsAAfrom the first client’s uploaded state rather than deriving it from the initialization \(Section[3](https://arxiv.org/html/2609.13512#S3)\-[3\.4](https://arxiv.org/html/2609.13512#S3.SS4)\)\. An implementation faithful to the design reaches the floor of 928\.1 MB, raising FFA\-LoRA’s savings from 61\.87% to 64\.00%, widening the ReverseAdaptive\-to\-FFA\-LoRA segment from 21\.3 to 23\.5 percentage points, and moving the marginal cost ratio from 5\.5 to 5\.0\. The knee conclusion is unchanged\. The same harvest rule governs ReverseAdaptive and Two\-Phase, where the seeding upload is likewise avoidable in principle, since the server already holds the aggregated globalAAit broadcast in the preceding round; freezing that value instead is an untested alternative recorded in Section[6](https://arxiv.org/html/2609.13512#S6)\.

### 5\.4THE BASE\-MODEL REGRESSION OBSERVATION

Federated Alpaca\-3k\[[22](https://arxiv.org/html/2609.13512#bib.bib22)\]instruction tuning regresses TinyLlama\-1\.1B\[[20](https://arxiv.org/html/2609.13512#bib.bib20)\]on all three informative zero\-shot benchmarks: ARC\-Easy by 2\.2 to 2\.8 percentage points depending on method, BoolQ by 6\.4 to 7\.4, and HellaSwag by 2\.8 uniformly\. At LLaMA\-3\.2\-3B\[[21](https://arxiv.org/html/2609.13512#bib.bib21)\]the same procedure is flat\-to\-positive on all four benchmarks\. The cross\-scale contrast is consistent with a model\-capacity explanation: TinyLlama\-1\.1B may lack the parameter budget to absorb instruction tuning without distributional drift on these benchmarks, though a single dataset and two scales cannot rule out other explanations\.

The same procedure improves substantially on the objective it optimizes\. On the held\-out Alpaca slice the base model scores 1\.9352 \(perplexity 6\.9254\) and every fine\-tuned checkpoint scores between 1\.3349 and 1\.3608 \(perplexity 3\.80 to 3\.90\)\. The regression is a divergence between the training objective and these benchmarks rather than a training failure, and it is why quality is reported here as held\-out instruction\-following loss\. The held\-out slice is drawn from the same distribution as the training data, so it measures instruction following on unseen examples and is not evidence of generalization to unrelated tasks\.

The regression is independent of aggregation method\. Cross\-method spread at TinyLlama scale is 0\.006 on ARC\-Easy, 0\.010 on BoolQ, and 0\.000 on HellaSwag, so the zero\-shot benchmarks do not separate protocols that the held\-out metric separates atp<0\.05p<0\.05\. These accuracies are single\-seed \(seed 42, MPS corpus\), unlike the three\-seed CUDA results in Table[2](https://arxiv.org/html/2609.13512#S4.T2), so the spreads carry no variance estimate and indicate insensitivity rather than establishing equivalence\.

The BoolQ result admits a specific account\. The base model’s accuracy of 0\.626 closely matches BoolQ’s natural yes\-class rate of approximately 0\.62, suggesting the base exploits a yes\-biased prior\. Instruction tuning attenuates that prior, which is why BoolQ moves furthest of the three\.

## 6LIMITATIONS

The experiments use two instruction\-following datasets \(Alpaca\-3k\[[22](https://arxiv.org/html/2609.13512#bib.bib22)\]and Dolly\-15k\[[23](https://arxiv.org/html/2609.13512#bib.bib23)\]\) and a single task type\. Generalization to classification tasks, longer training runs, or generation\-quality metrics is not characterized\. Quality is measured as held\-out loss on examples drawn from the same distribution as the training data, which captures instruction following on unseen examples rather than transfer to unrelated tasks\.

FFA\-LoRA and FedIT were evaluated on Alpaca only, so Table[3](https://arxiv.org/html/2609.13512#S4.T3)compares three protocols where Table[2](https://arxiv.org/html/2609.13512#S4.T2)compares five\. The cross\-dataset result therefore speaks to the cost of the adaptive transition rather than to the stability of the whole frontier across datasets\.

The four zero\-shot benchmarks do not discriminate between aggregation protocols at TinyLlama scale, where cross\-method spread is at most 1\.0 percentage point and every fine\-tuned checkpoint scores below the base model \(Section[5](https://arxiv.org/html/2609.13512#S5)\-[5\.4](https://arxiv.org/html/2609.13512#S5.SS4)\)\. Conclusions about relative protocol quality therefore rest on held\-out instruction\-following loss alone\.

The simulation assumes full client participation in every round\. Real federated deployments typically involve partial participation, where only a fraction of clients respond per round\. The effect of partial participation on transition\-round timing and on ReverseAdaptive’s plateau detection is not studied here\.

Non\-IID partitions are constructed by Dirichlet skew over instruction\-length buckets rather than over semantic labels, since neither dataset carries class labels\. This induces heterogeneity in task form\. Heterogeneity in task semantics, which is what semantic\-class partitioning models in classification settings, is untested\.

The study covers only Llama\-style decoder\-only architectures with LoRA targetingq\_projandv\_proj\. Other architectures, encoder\-decoder models, larger target sets, or heterogeneous rank configurations may produce different B\-only fractions and different savings profiles\. The reported percentages are specific to this target set and to the grouped\-query attention configurations of the two models tested\.

At the transition round the server freezesAAfrom the first client’s uploaded state\. Freezing the aggregated globalAAthat the server already holds from the preceding round would avoid the seeding upload entirely and is untested, so the 55\.0 MB transition cost reported here is a property of this implementation rather than of phase\-switching protocols in general \(Section[3](https://arxiv.org/html/2609.13512#S3)\-[3\.4](https://arxiv.org/html/2609.13512#S3.SS4)\)\.

Runs were produced across multiple code revisions with uncommitted local modifications\. The primary CUDA corpus spans two revisions, and the FLoRA and FedIT rows of Table[2](https://arxiv.org/html/2609.13512#S4.T2)sit on either side of that boundary while remaining statistically indistinguishable, which indicates that the combined algorithmic and revision difference across that boundary is roughly 0\.0006 in final loss\. The LLaMA\-3\.2\-3B results are not single\-revision: the three seeds were produced at three revisions, so the variability reported there combines seed and environment variation\.

LLaMA\-3\.2\-3B experiments use three random seeds with downstream evaluation per checkpoint\. Seed\-to\-seed evaluation noise within a single checkpoint is not separately characterized\. At TinyLlama\-1\.1B, downstream evaluation is single\-seed per checkpoint while training uses three seeds\.

No real\-world network conditions are simulated\. Byte counts are measured from the tensors actually transmitted, but network latency, packet loss, and bandwidth constraints are not modeled, so wall\-clock communication time is not predicted\.

No differential privacy analysis is included\. FFA\-LoRA\[[5](https://arxiv.org/html/2609.13512#bib.bib5)\]was originally motivated in part by DP\-friendliness; the transition\-round dynamics of the bidirectional B\-only protocol may affect DP accounting in non\-obvious ways that warrant separate study\.

Future work should address partial participation, non\-Llama architectures, semantic heterogeneity, and real network conditions to validate deployment claims more broadly\. Composing the bidirectional B\-only protocol with gradient compression techniques\[[13](https://arxiv.org/html/2609.13512#bib.bib13),[14](https://arxiv.org/html/2609.13512#bib.bib14)\]for savings beyond the B\-only floor is a natural extension\. Per\-layer adaptive phase\-switching, where different transformer layers switch at different rounds based on their individual loss\-plateau signals, is another direction\.

## 7CONCLUSION

This paper’s central contribution is a shift from theoretical parameter\-count savings to measured byte\-level reporting for federated LoRA\. Tracking per\-round upload and download megabytes, including at the asymmetric transition boundary, shows that the transition costs a fixed 55\.0 MB on TinyLlama\-1\.1B regardless of when it occurs, a quantity parameter\-count accounting cannot express\.

Those measurements place five protocols on a single communication\-quality frontier with a knee\. ReverseAdaptive sits at that knee and locates it without a phase boundary being specified in advance; the marginal cost of moving past it is roughly five times higher per point saved\.

The same threshold transfers across model scales without retuning, firing at round 6 on TinyLlama\-1\.1B and at a mean of round 8 on LLaMA\-3\.2\-3B, and the quality cost of the transition is stable across the two datasets tested\.

As open\-weight LLMs continue to grow in capability and federated deployments expand into privacy\-sensitive domains, the communication bottleneck identified by Kairouz et al\.\[[1](https://arxiv.org/html/2609.13512#bib.bib1)\]will only intensify\. Reporting what protocols actually transmit, rather than what a parameter count predicts, is a prerequisite for comparing them\.

## ACKNOWLEDGMENT

All code, configuration files, and analysis scripts are publicly available at[https://github\.com/jerryadamsfranklin/fedlora\-protocols](https://github.com/jerryadamsfranklin/fedlora-protocols)\. Adapter checkpoints are not released\. The Alpaca and Dolly\-15k datasets are publicly available from their respective sources\.

The author used Claude \(Anthropic\) for grammar and language editing only\. All technical content, analysis, and interpretations are the author’s own\.

## References

- \[1\]P\. Kairouz, H\. B\. McMahan, B\. Avent, A\. Bellet, M\. Bennis, A\. N\. Bhagoji*et al\.*, “Advances and open problems in federated learning,”*Found\. Trends Mach\. Learn\.*, vol\. 14, no\. 1–2, pp\. 1–210, 2021, doi: 10\.1561/2200000083\.
- \[2\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen, “LoRA: Low\-rank adaptation of large language models,” in*International Conference on Learning Representations \(ICLR\)*, 2022\.
- \[3\]J\. Zhang, S\. Vahidian, M\. Kuo, C\. Li, R\. Zhang, T\. Yu, G\. Wang, and Y\. Chen, “Towards building the FederatedGPT: Federated instruction tuning,” in*IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, 2024, pp\. 6915–6919, doi: 10\.1109/ICASSP48485\.2024\.10447454\.
- \[4\]B\. McMahan, E\. Moore, D\. Ramage, S\. Hampson, and B\. Agüera y Arcas, “Communication\-efficient learning of deep networks from decentralized data,” in*International Conference on Artificial Intelligence and Statistics \(AISTATS\)*, 2017\.
- \[5\]Y\. Sun, Z\. Li, Y\. Li, and B\. Ding, “Improving LoRA in privacy\-preserving federated learning,” in*International Conference on Learning Representations \(ICLR\)*, 2024, arXiv:2403\.12313\.
- \[6\]Z\. Wang, Z\. Shen, Y\. He, G\. Sun, H\. Wang, L\. Lyu, and A\. Li, “FLoRA: Federated fine\-tuning large language models with heterogeneous low\-rank adaptations,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2024, pp\. 22 513–22 533, arXiv:2409\.05976\.
- \[7\]J\. Bai, D\. Chen, B\. Qian, L\. Yao, and Y\. Li, “Federated fine\-tuning of large language models under heterogeneous tasks and client resources,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2024, pp\. 14 457–14 483, arXiv:2402\.11505\.
- \[8\]G\. Yan, L\. Xie, Q\. Shen, Y\. Fang, and Z\. Wu, “FedSRD: Sparsify\-reconstruct\-decompose for communication\-efficient federated large language models fine\-tuning,” in*Proceedings of the ACM Web Conference 2026 \(WWW ’26\)*, 2026, pp\. 5087–5098, doi: 10\.1145/3774904\.3792144\.
- \[9\]H\. Ramesh and J\. Dass, “FLoRIST: Singular value thresholding for efficient and accurate federated fine\-tuning of large language models,” in*Proceedings of the Ninth Conference on Machine Learning and Systems \(MLSys\)*, 2026, arXiv:2506\.09199v2\.
- \[10\]N\. Houlsby, A\. Giurgiu, S\. Jastrzebski, B\. Morrone, Q\. de Laroussilhe, A\. Gesmundo, M\. Attariyan, and S\. Gelly, “Parameter\-efficient transfer learning for NLP,” in*International Conference on Machine Learning \(ICML\)*, 2019\.
- \[11\]T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. Zettlemoyer, “QLoRA: Efficient finetuning of quantized LLMs,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2023\.
- \[12\]P\. Guo, S\. Zeng, Y\. Wang, H\. Fan, F\. Wang, and L\. Qu, “Selective aggregation for low\-rank adaptation in federated learning,” in*International Conference on Learning Representations \(ICLR\)*, 2025, arXiv:2410\.01463\.
- \[13\]D\. Alistarh, D\. Grubic, J\. Li, R\. Tomioka, and M\. Vojnovic, “QSGD: Communication\-efficient SGD via gradient quantization and encoding,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2017\.
- \[14\]Y\. Lin, S\. Han, H\. Mao, Y\. Wang, and W\. J\. Dally, “Deep gradient compression: Reducing the communication bandwidth for distributed training,” in*International Conference on Learning Representations \(ICLR\)*, 2018\.
- \[15\]A\. Reisizadeh, A\. Mokhtari, H\. Hassani, A\. Jadbabaie, and R\. Pedarsani, “FedPAQ: A communication\-efficient federated learning method with periodic averaging and quantization,” in*International Conference on Artificial Intelligence and Statistics \(AISTATS\)*, 2020\.
- \[16\]T\. Li, A\. K\. Sahu, M\. Zaheer, M\. Sanjabi, A\. Talwalkar, and V\. Smith, “Federated optimization in heterogeneous networks,” in*Proceedings of Machine Learning and Systems \(MLSys\)*, 2020\.
- \[17\]J\. Wang, Q\. Liu, H\. Liang, G\. Joshi, and H\. V\. Poor, “Tackling the objective inconsistency problem in heterogeneous federated optimization,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2020\.
- \[18\]S\. Reddi, Z\. Charles, M\. Zaheer, Z\. Garrett, K\. Rush, J\. Konečný, S\. Kumar, and H\. B\. McMahan, “Adaptive federated optimization,” in*International Conference on Learning Representations \(ICLR\)*, 2021\.
- \[19\]I\. Loshchilov and F\. Hutter, “Decoupled weight decay regularization,” in*International Conference on Learning Representations \(ICLR\)*, 2019\.
- \[20\]P\. Zhang, G\. Zeng, T\. Wang, and W\. Lu, “TinyLlama: An open\-source small language model,”*arXiv preprint*, 2024, arXiv:2401\.02385\.
- \[21\]A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle*et al\.*, “The Llama 3 herd of models,”*arXiv preprint*, 2024, arXiv:2407\.21783\.
- \[22\]R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. Hashimoto, “Alpaca: A strong, replicable instruction\-following model,” Stanford CRFM Blog, 2023\.
- \[23\]M\. Conover, M\. Hayes, A\. Mathur, J\. Xie, J\. Wan, S\. Shah, A\. Ghodsi, P\. Wendell, M\. Zaharia, and R\. Xin, “Free dolly: Introducing the world’s first truly open instruction\-tuned LLM,”[https://www\.databricks\.com/blog/2023/04/12/dolly\-first\-open\-commercially\-viable\-instruction\-tuned\-llm](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm), 2023\.
- \[24\]D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt, “Measuring massive multitask language understanding,” in*International Conference on Learning Representations \(ICLR\)*, 2021\.
- \[25\]P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord, “Think you have solved question answering? Try ARC, the AI2 reasoning challenge,”*arXiv preprint*, 2018, arXiv:1803\.05457\.
- \[26\]C\. Clark, K\. Lee, M\.\-W\. Chang, T\. Kwiatkowski, M\. Collins, and K\. Toutanova, “BoolQ: Exploring the surprising difficulty of natural yes/no questions,” in*Proceedings of NAACL\-HLT*, 2019, pp\. 2924–2936, doi: 10\.18653/v1/N19\-1300\.
- \[27\]R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi, “HellaSwag: Can a machine really finish your sentence?” in*Proceedings of the Association for Computational Linguistics \(ACL\)*, 2019, pp\. 4791–4800, doi: 10\.18653/v1/P19\-1472\.

\{IEEEbiographynophoto\}

Jerry Adams Franklin is an Independent Researcher in Fairport, NY 14450 USA\. He received the M\.S\. degree in data science from Northeastern University, Boston, MA, USA, in 2022\. He previously worked in machine learning engineering at Intel Corporation and Digital Currency Group\. His research interests include federated learning, parameter\-efficient fine\-tuning, and communication\-efficient distributed training\. Contact him at jerry\.adamsf@gmail\.com\.

\\appendices

## 8Hyperparameter Details

All experiments use AdamW\[[19](https://arxiv.org/html/2609.13512#bib.bib19)\]with learning rate10−410^\{\-4\},β1=0\.9\\beta\_\{1\}\{=\}0\.9,β2=0\.999\\beta\_\{2\}\{=\}0\.999\. No learning rate schedule is applied\. Gradients are clipped to a maximumℓ2\\ell\_\{2\}norm of 1\.0 at every optimizer step\. Maximum sequence length is 256 tokens\. See Table[1](https://arxiv.org/html/2609.13512#S4.T1)for the complete configuration including ReverseAdaptive defaults\.

## 9Per\-Seed Full Results

Tables in this appendix are labeled with their corpus, which follows the corresponding table in the main text\. The TinyLlama\-1\.1B method comparisons use the CUDA corpus, since the FFA\-LoRA, FedIT, and Dolly\-15k runs exist only there; the LLaMA\-3\.2\-3B results use MPS, matching Table[7](https://arxiv.org/html/2609.13512#S4.T7); and the partition\-sensitivity results retain the larger MPS seed set, where five seeds atα=0\.1\\alpha\{=\}0\.1provide better evidence than three\.

Atα=0\.5\\alpha\{=\}0\.5, mean final loss is statistically indistinguishable from IID for every method in the released analysis CSVs, with differences of 0\.001 to 0\.007 against per\-seed standard deviations near 0\.02 over three seeds\. Heterogeneity manifests instead as sensitivity to the partition draw: seed\-to\-seed standard deviation rises by roughly an order of magnitude, from0\.50\.5to1\.6×10−31\.6\\times 10^\{\-3\}under IID in Table[S1](https://arxiv.org/html/2609.13512#S9.T1)to1818to25×10−325\\times 10^\{\-3\}atα=0\.5\\alpha\{=\}0\.5across the methods in the released CSVs, which is also why the per\-seed spread atα=0\.1\\alpha\{=\}0\.1in Table[S2](https://arxiv.org/html/2609.13512#S9.T2)is wide\.

Table S1:Per\-seed TinyLlama\-1\.1B IID results, CUDA corpus, backing Table[2](https://arxiv.org/html/2609.13512#S4.T2)\. Communication is deterministic across seeds\.†\\daggerFor Two\-Phase the column shows the predetermined phase boundaryKK, not an adaptive switch event; for FFA\-LoRA,AAis frozen from initialization; for ReverseAdaptive it shows the round at which the plateau signal triggered\.Table S2:Non\-IID results, MPS corpus, five seeds atα=0\.1\\alpha\{=\}0\.1\. This table documents partition sensitivity rather than comparing methods, so the larger MPS seed set is retained; Table[S1](https://arxiv.org/html/2609.13512#S9.T1)follows the CUDA corpus because it backs a method comparison\. Seeds 42 and 1000 atα=0\.1\\alpha\{=\}0\.1had 9 and 8 active clients due to empty Dirichlet partitions, which scales communication to9/109/10and8/108/10of the full value \(1379\.81 and 1226\.50 MB against 1533\.13\)\. Cross\-backend verification \(Appendix[12](https://arxiv.org/html/2609.13512#S12)\) covers seeds 42, 123, and 456; seeds 789 and 1000 were executed on MPS only\.Table S3:Per\-seed LLaMA\-3\.2\-3B IID results, MPS corpus, backing Table[7](https://arxiv.org/html/2609.13512#S4.T7)\. ReverseAdaptive usesτ=0\.01\\tau\{=\}0\.01\. Communication is deterministic given the switch round; the three ReverseAdaptive seeds occupy three adjacent lattice points\.Table S4:Per\-seed TinyLlama\-1\.1B Dolly\-15k IID results, CUDA corpus, backing Table[3](https://arxiv.org/html/2609.13512#S4.T3)\. Held\-outΔ\\Deltais tuned minus base on the Dolly\-15k held\-out slice; more negative is better\.
## 10Statistical Tests

Pairedtt\-tests over three seeds on final\-round training loss and on held\-out instruction\-following loss\. TinyLlama\-1\.1B results use the CUDA corpus, matching Table[2](https://arxiv.org/html/2609.13512#S4.T2); LLaMA\-3\.2\-3B results use the MPS corpus\.

With three seeds these tests have very low power, so effect sizes should be read alongside thepp\-values\. No multiplicity correction is applied in the main results tables\. Applying a Bonferroni correction across the six TinyLlama\-1\.1B Alpaca IID comparisons gives a threshold of0\.00830\.0083, which the ReverseAdaptive versus Two\-PhaseK=8K\{=\}8comparison on held\-out loss \(p=0\.026p=0\.026\) does not meet\. That comparison is secondary under the frontier framing of Section[5](https://arxiv.org/html/2609.13512#S5); the load\-bearing comparison is ReverseAdaptive versus FFA\-LoRA, which holds atp<10−3p<10^\{\-3\}on both metrics with a difference roughly twenty times the largest per\-method standard deviation\.

The LLaMA\-3\.2\-3B comparison between ReverseAdaptive and Two\-PhaseK=8K\{=\}8has a 95% confidence interval of\[−0\.0114,0\.0114\]\[\-0\.0114,0\.0114\], which contains the0\.00430\.0043difference observed between the same methods at TinyLlama\-1\.1B\. The test therefore cannot distinguish equivalence from a difference of the magnitude seen at the smaller scale\.

Table S5:Pairedtt\-tests, three seeds\. TinyLlama\-1\.1B rows use the CUDA corpus; LLaMA\-3\.2\-3B rows use MPS\. PositiveΔ\\Deltaindicates the first method has higher loss\. Held\-out instruction\-following loss was not computed for the LLaMA\-3\.2\-3B corpus; quality at that scale is reported as final training loss and downstream zero\-shot accuracy\.Figure S1:Downstream zero\-shot accuracy across two model scales \(500 examples per benchmark\)\. Left: TinyLlama\-1\.1B \(seed 42 only; ARC\-Easy, BoolQ, HellaSwag; MMLU omitted because base accuracy is at chance for 4\-way multiple choice\)\. Right: LLaMA\-3\.2\-3B \(mean across three seeds; MMLU, ARC\-Easy, BoolQ, HellaSwag\)\. Federated methods cluster within 1\.0 percentage points at TinyLlama\-1\.1B, across the three informative benchmarks, and 0\.5 percentage points at LLaMA\-3\.2\-3B\. TinyLlama accuracies are single\-seed and from the MPS corpus; loss values elsewhere in this paper for the same configurations are three\-seed and from the CUDA corpus\.
## 11Reproducibility Checklist

Code:Full source code is publicly released, including all configuration YAML files, training scripts, evaluation scripts, and scripts to reproduce every figure and table from the analysis CSVs\.

Seeds:The IID seeds are listed in Table[1](https://arxiv.org/html/2609.13512#S4.T1)\. Theα=0\.1\\alpha\{=\}0\.1partition\-sensitivity runs additionally use seeds 789 and 1000, which appear only in Table[S2](https://arxiv.org/html/2609.13512#S9.T2)of this appendix\. Per\-seed results are in Appendix[9](https://arxiv.org/html/2609.13512#S9)\.

Hardware:The primary corpus of 34 runs was produced on a single Apple M4 Pro workstation with 48 GB unified memory using the MPS backend, approximately 285 hours of compute, with no institutional cluster\. Those runs were re\-executed on commercially rented NVIDIA hardware, an RTX 4090 for TinyLlama\-1\.1B and an A100 40 GB for LLaMA\-3\.2\-3B; the comparison is in Appendix[12](https://arxiv.org/html/2609.13512#S12)\. The Dolly\-15k replication and the FFA\-LoRA and FedIT baselines were produced on the rented hardware only\.

Data:Alpaca\[[22](https://arxiv.org/html/2609.13512#bib.bib22)\]viatatsu\-lab/alpacaon HuggingFace, 3000\-sample subset, and Dolly\-15k\[[23](https://arxiv.org/html/2609.13512#bib.bib23)\]viadatabricks/databricks\-dolly\-15k, matched subset\. Adapter checkpoints are not released; reproduction scripts are sufficient\.

## 12Cross\-Backend Verification

The primary corpus was produced on the MPS backend and subsequently re\-executed on CUDA\. Table[S6](https://arxiv.org/html/2609.13512#S12.T6)summarizes the comparison; Table[S7](https://arxiv.org/html/2609.13512#S12.T7)details every disagreement\.

Table S6:Cross\-backend verification summary, MPS against CUDA, 34 run pairs\.Table S7:Cross\-backend disagreements\. Every communication difference is exactly one 110 MB step of the switch\-round lattice\. Run 16 breached only the consecutive\-spike rule, with no communication or switch\-round difference\.Tolerances are setting\-aware\. IID runs require per\-round training loss to agree within0\.010\.01absolute\. Non\-IID runs allow0\.0250\.025, since heterogeneous partitions amplify per\-round variation, but additionally require the switch round to match exactly and permit no more than two consecutive rounds exceeding0\.010\.01\. Communication must agree within0\.010\.01in all settings, which is effectively exact since totals are protocol\-deterministic\.

Of 34 run pairs, 31 have a valid MPS reference\. The three Two\-PhaseK=8K\{=\}8runs atα=0\.5\\alpha\{=\}0\.5do not: their MPS executions predate the bidirectional B\-only implementation and never switched, recording 2578\.13 MB, the full FLoRA volume\. Those are absent references rather than disagreements, and CUDA is the source of truth for that configuration\. Of the 31 comparable pairs, 27 agree within tolerance and 4 do not\.

Three of the four disagreements are the same phenomenon\. Total communication is a step function of the discrete switch round with 110 MB granularity \(Section[5](https://arxiv.org/html/2609.13512#S5)\-[5\.3](https://arxiv.org/html/2609.13512#S5.SS3)\), and every observed cross\-backend communication difference is exactly one step: the MPS and CUDA totals in Table[S7](https://arxiv.org/html/2609.13512#S12.T7)are adjacent points on the same lattice\. The backends do not disagree about the protocol, only about which round the plateau criterion fires, and only where that decision sits near a boundary\. Two of the three are IID runs at tight thresholds \(τ=0\.001\\tau\{=\}0\.001andτ=0\.002\\tau\{=\}0\.002\), where the criterion is nearly satisfied in either of two adjacent rounds; one is anα=0\.1\\alpha\{=\}0\.1run, where heterogeneity produces a noisier loss trajectory\. The fourth, run 16, shows no communication or switch\-round difference at all: the backends agree on the protocol and on the switch round and differ only in the per\-round loss trajectory, which breached the consecutive\-spike rule\. This is boundary sensitivity rather than heterogeneity sensitivity\.

Run 17 was investigated separately\. Re\-executing on CUDA with the data partition seed fixed at 456 and the training seed changed to 999 reproduced switch round 7 exactly, establishing that the one\-round difference is a stable property of the backend and partition rather than run\-to\-run stochasticity\. Atα=0\.1\\alpha\{=\}0\.1, cross\-backend agreement on the discrete switch round is therefore not guaranteed: one of three partitions with CUDA counterparts \(seeds 42, 123, and 456\) fired one round later on CUDA, and CUDA is reported as primary for that setting\.

The switch round matched exactly for every IID run atτ=0\.01\\tau\{=\}0\.01and for all three LLaMA\-3\.2\-3B ReverseAdaptive runs, which is what supports the cross\-scale transfer claim in Section[5](https://arxiv.org/html/2609.13512#S5)\.

Byte accounting\.Communication is instrumented at the transport layer, countingnumeltimeselement\_sizeon the tensors actually transmitted, rather than derived from parameter counts\. Theget\_communication\_costhelper in the FFA\-LoRA aggregator is not used by this accounting\.

Code revisions\.Runs were produced across multiple code revisions with uncommitted working\-tree modifications, so no reported number is recoverable by checking out a single commit\. Reproducibility rests instead on the cross\-backend replication documented here\. The byte accounting is revision\-invariant by direct measurement: FLoRA, Two\-PhaseK=8K\{=\}8and ReverseAdaptive were each executed under two different revisions within the CUDA corpus and produced identical totals of 2578\.13, 1863\.13 and 1533\.13 MB\. Loss values carry revision uncertainty; the FLoRA and FedIT rows of Table[2](https://arxiv.org/html/2609.13512#S4.T2)span that boundary and differ by 0\.0006, against a ReverseAdaptive\-to\-FFA\-LoRA separation of 0\.0282\.

## 13Downstream Accuracy Figure

Figure[S1](https://arxiv.org/html/2609.13512#S10.F1)plots the downstream zero\-shot accuracies of Tables[4](https://arxiv.org/html/2609.13512#S4.T4)and[5](https://arxiv.org/html/2609.13512#S4.T5)\. It restates those tables graphically and introduces no additional measurements\.

## 14Disabling the ReverseAdaptive Switch

The no\-switch sanity baseline deserves specific comment\. Running ReverseAdaptive with switching disabled reproduces the matched FLoRA\[[6](https://arxiv.org/html/2609.13512#bib.bib6)\]run to 10 decimal places at every round, establishing that the wrapper is a zero\-cost modification when unused: it rules out numerical artifacts from the wrapper’s presence, not merely from its activation\. The comparison is within a single backend and a single execution environment\. Agreement across hardware is weaker than bit\-identity and is reported in Appendix[12](https://arxiv.org/html/2609.13512#S12)\.

Disabling the switch requires care\. Settingτ\\tauto a small positive value does not achieve it: any round in which the training loss fails to improve satisfiesρr<τ\\rho\_\{r\}<\\tau, so the criterion fires\. On the FLoRA trajectory underlying the threshold ablation the loss rises once, at round 11, soτ=0\\tau\{=\}0still triggers a switch at that round, and suppressing the switch entirely requiresτ\\taubelow the most negative round\-over\-round relative change observed, approximately−0\.0009\-0\.0009here\. The sanity baseline accordingly usesτ=−1\.0\\tau\{=\}\-1\.0together with a warmup exceeding the round budget\. A practitioner wanting the wrapper present but inert should disable it by one of those means rather than by choosing a conservative threshold\.