Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices

# Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices
Source: [https://arxiv.org/html/2607.06651](https://arxiv.org/html/2607.06651)
Quoc Bao Phan![[Uncaptioned image]](https://arxiv.org/html/2607.06651v1/x1.png)\{\}^\{\\href https://orcid\.org/0009\-0002\-5643\-3828\{\}\},Graduate Student Member, IEEE, and Tuy Tan Nguyen![[Uncaptioned image]](https://arxiv.org/html/2607.06651v1/x2.png)\{\}^\{\\href https://orcid\.org/0000\-0002\-9485\-7720\{\}\},Senior Member, IEEEThis research was supported by the Department of Electrical and Computer Engineering, Center for Advanced Power Systems, FAMU\-FSU College of Engineering, Florida State University\.\(Corresponding author: Tuy Tan Nguyen\.\)The authors are with the Department of Electrical and Computer Engineering, Center for Advanced Power Systems, FAMU\-FSU College of Engineering, Florida State University, Tallahassee, FL 32310, USA\. \(e\-mail: qp25c@fsu\.edu, tuy\.nguyen@fsu\.edu\)\.

###### Abstract

Federated learning \(FL\) over mobile and edge devices increasingly involves multimodal models in which clients differ in both sensing capability and computational capacity\. Existing update compression schemes typically apply uniform policies across layers and devices, without accounting for modality\-specific differences in spectral structure and compressibility\. We propose MESH\-FL, an entropy\-guided matrix product state \(MPS\) update\-compression framework for modality\-heterogeneous FL on resource\-constrained devices\. MESH\-FL estimates the spectral entropy of each layer\-wise update via truncated singular value decomposition and allocates MPS compression ranks adaptively across layers, modalities, and devices under per\-client payload budgets\. We show that higher spectral entropy necessitates a higher reconstruction rank under the majorization order on singular\-value energy distributions\. Building on this result, we prove that the proposed entropy\-guided allocation solves a convex surrogate rank\-allocation problem, preserves monotonicity under the exact payload model, and achieves convergence with an explicit compression\-dependent error term\. Experiments on a 15\-node heterogeneous Raspberry Pi 4/5 cluster with modality\-heterogeneous clients show that MESH\-FL achieves up to56\.8×56\.8\\timescompression while surpassing the uncompressed FedAvg baseline in final accuracy by up to 2\.01%, and reduces total transmitted data to reach convergence by up to66×66\\times\.

## IIntroduction

Federated learning \(FL\) has emerged as an important framework for training models across large collections of mobile and edge devices without centralizing raw data\[[20](https://arxiv.org/html/2607.06651#bib.bib21),[5](https://arxiv.org/html/2607.06651#bib.bib22)\]\. Its appeal is particularly strong in sensing applications, where data are generated continuously on user devices and are often subject to privacy, bandwidth, and storage constraints\[[17](https://arxiv.org/html/2607.06651#bib.bib2)\]\. At the same time, many edge applications are inherently multimodal\. Human activity recognition, mobile health, intelligent environments, and audio\-visual perception often rely on multiple sensing streams, while the devices participating in training may not share the same sensing hardware\[[1](https://arxiv.org/html/2607.06651#bib.bib23)\]\. As a result, practical FL deployments increasingly operate under modality heterogeneity, where clients differ not only in computation and communication resources, but also in the modalities they can observe and contribute during training\.

This setting gives rise to a communication problem that is more structured than in conventional FL\. In cross\-device systems, transmitting model updates already accounts for a large fraction of the training cost, especially when clients are resource\-constrained and communicate over wireless links\[[7](https://arxiv.org/html/2607.06651#bib.bib32)\]\. In multimodal settings, this challenge becomes more pronounced because different branches of the model can produce updates with different statistical and structural properties\. Consequently, a compression policy that is adequate for one modality may be unnecessarily aggressive or insufficient for another\. Communication efficiency in multimodal FL is therefore not only a systems issue, but also a representation issue: the compression mechanism should reflect the structure of the layer\-wise updates being transmitted\.

Existing update compression methods, including sparsification, quantization, and low\-rank approximation, have shown that substantial reductions in communication are possible in FL\[[10](https://arxiv.org/html/2607.06651#bib.bib28),[28](https://arxiv.org/html/2607.06651#bib.bib31)\]\. However, most existing schemes apply a largely uniform compression policy across layers or across clients, with limited regard for differences in update structure or device capability\[[3](https://arxiv.org/html/2607.06651#bib.bib29)\]\. In heterogeneous edge environments, such uniformity can be inefficient in two ways\. First, it can misallocate compression budget by treating updates of different complexity similarly, even though some updates admit stronger compression than others\. Second, it can overlook per\-device resource constraints, making the same compression choice suboptimal across weak and capable devices\. These limitations are particularly relevant in modality\-heterogeneous FL, where both model structure and hardware variability shape the communication bottleneck\[[4](https://arxiv.org/html/2607.06651#bib.bib30)\]\.

Our work is motivated by the observation that the spectral entropy of a layer\-wise update matrix, derived from the normalized singular\-value energy distribution, provides a compact measure of compressibility: low entropy indicates energy concentrated in a few modes, whereas high entropy reflects a more diffuse spectrum and greater resistance to low\-rank approximation\. Building on this view, we show that higher spectral entropy generally requires higher reconstruction rank under the natural majorization order on singular\-value energy distributions, which motivates entropy\-guided rank selection across layers, modalities, and devices\. Based on this principle, we developMESH\-FL, an entropy\-guided and device\-aware compression framework for modality\-heterogeneous FL that combines per\-layer entropy estimation, adaptive rank allocation under per\-client payload budgets, and matrix product state \(MPS\)\-based update compression within a unified pipeline\. Instead of enforcing a uniform compression level across all clients and model components, MESH\-FL adapts compression to both update complexity and device capability; we further analyze its optimization behavior through an explicit compression\-dependent approximation term and evaluate it on a heterogeneous edge testbed and a multimodal benchmark, where it reduces communication while surpassing the uncompressed baseline in accuracy\.

Our main contributions are summarized as follows:

- •We formulate communication\-efficient FL under joint modality and device heterogeneity, showing why uniform compression is poorly matched to heterogeneous edge settings\.
- •We introduce a spectral\-entropy\-guided view of update compression and show that higher spectral entropy implies weakly larger reconstruction rank under the majorization order\.
- •We develop MESH\-FL, an entropy\-guided MPS compression framework that allocates ranks adaptively across layers, modalities, and clients under per\-client payload budgets\.
- •We provide convergence analysis with an explicit compression\-dependent error term and validate MESH\-FL on heterogeneous Raspberry Pi hardware, demonstrating reduced communication and improved accuracy over baselines\.

The remainder of this paper is organized as follows\. Section[II](https://arxiv.org/html/2607.06651#S2)reviews related work and background on update compression and heterogeneous FL\. Section[III](https://arxiv.org/html/2607.06651#S3)defines the problem formulation\. Section[IV](https://arxiv.org/html/2607.06651#S4)presents the MESH\-FL framework and algorithm\. Section[V](https://arxiv.org/html/2607.06651#S5)provides the theoretical analysis\. Section[VI](https://arxiv.org/html/2607.06651#S6)reports the experimental results\. Section[VII](https://arxiv.org/html/2607.06651#S7)concludes the paper\.

## IIRelated Work

### II\-AUpdate Compression in Federated Learning

Reducing the per\-round communication cost in FL has been pursued through sparsification, quantization, and low\-rank approximation\. Sparsification methods transmit only a subset of update coordinates selected by magnitude threshold or random sampling\[[15](https://arxiv.org/html/2607.06651#bib.bib3),[22](https://arxiv.org/html/2607.06651#bib.bib24)\]\. With error feedback, these methods converge at rates comparable to dense communication, but they treat each coordinate independently and cannot exploit the structural redundancy within weight matrices\. Quantization reduces the bit\-width of transmitted values\. QSGD\[[2](https://arxiv.org/html/2607.06651#bib.bib26)\]provides stochastic rounding with variance\-bounded convergence guarantees; TernGrad\[[26](https://arxiv.org/html/2607.06651#bib.bib4)\]pushes quantization to three levels for extreme compression\. Both are agnostic to the spectral organization of updates and offer no mechanism to allocate precision proportionally to layer complexity\.

Low\-rank approximation methods are structurally closer to our approach\. PowerSGD\[[25](https://arxiv.org/html/2607.06651#bib.bib27)\]compresses gradient matrices via randomized power iteration and has demonstrated strong empirical compression ratios on large\-scale models\. FedPAQ\[[21](https://arxiv.org/html/2607.06651#bib.bib5)\]combines periodic averaging with quantization\. However, existing methods largely apply uniform rank or budget policies across layers and devices, without adapting to layer\-wise spectral structure or per\-client resource constraints\. MESH\-FL departs from this uniformity by using spectral entropy to allocate MPS rank adaptively across layers, modalities, and heterogeneous devices\.

### II\-BTensor Decompositions in Neural Networks and Federated Learning

Tensor decompositions have been used extensively for compressing neural network weights\[[18](https://arxiv.org/html/2607.06651#bib.bib6),[12](https://arxiv.org/html/2607.06651#bib.bib7)\]and, more recently, for compressing gradients in distributed optimization\. The MPS, also known as the tensor\-train \(TT\) decomposition\[[19](https://arxiv.org/html/2607.06651#bib.bib8)\], factorizes a reshaped weight or gradient tensor into a chain of three\-dimensional cores, enabling compression ratios that scale favorably with tensor order\. Authors in\[[24](https://arxiv.org/html/2607.06651#bib.bib9)\]applied TT decomposition to recurrent weights;\[[29](https://arxiv.org/html/2607.06651#bib.bib10)\]extended this to fully connected layers for edge deployment\. For a third\-order reshaping, the transmitted MPS payload depends directly on the selected bond rank through the core sizes, making rank selection the main mechanism that controls the communication–reconstruction tradeoff\. In FL, tensor\-based compression has been explored in server\-side aggregation and model pruning contexts, but prior work fixes decomposition rank globally and does not adapt to per\-layer spectral complexity or per\-device payload budgets\. MESH\-FL combines entropy\-guided adaptive MPS rank selection with device\-aware budget allocation for modality\-heterogeneous FL\.

### II\-CHeterogeneous Federated Learning

System heterogeneity, where clients differ in computation speed and communication bandwidth, has been addressed through asynchronous aggregation\[[27](https://arxiv.org/html/2607.06651#bib.bib11)\], partial model training\[[6](https://arxiv.org/html/2607.06651#bib.bib12)\], and proximal regularization\[[14](https://arxiv.org/html/2607.06651#bib.bib13)\]\. Statistical heterogeneity \(non\-independent and identically distributed \(non\-IID\) data distributions\) has been studied through variance reduction\[[11](https://arxiv.org/html/2607.06651#bib.bib14)\], gradient correction, and data\-sharing strategies\[[30](https://arxiv.org/html/2607.06651#bib.bib15)\]\. These works treat heterogeneity either in the computational or statistical dimension alone and do not address heterogeneous update compression under per\-client payload constraints\.

Modality heterogeneity in FL, where clients differ in their sensing hardware, is less studied\. Concurrent work has considered partial\-modality participation and modality dropout in cross\-silo settings\[[23](https://arxiv.org/html/2607.06651#bib.bib16)\], but without adaptive compression\. MESH\-FL addresses modality heterogeneity, device heterogeneity, and communication efficiency jointly within a single framework, providing layer\-wise aggregation over modality\-eligible clients and entropy\-guided rank allocation that respects device\-specific payload limits\.

## IIIProblem Formulation

### III\-AFL Objective

Consider an FL system with a central server and a set ofKKclients, indexed by𝒦=\{1,2,…,K\}\\mathcal\{K\}=\\\{1,2,\\ldots,K\\\}\. Each clientkkholds a local dataset𝒟k\\mathcal\{D\}\_\{k\}ofnkn\_\{k\}samples, and the datasets are distributed in a non\-IID manner across clients\. The global learning objectives is:

min𝐰⁡F​\(𝐰\)=∑k=1Knkn​Fk​\(𝐰\),\\min\_\{\\mathbf\{w\}\}F\(\\mathbf\{w\}\)=\\sum\_\{k=1\}^\{K\}\\frac\{n\_\{k\}\}\{n\}F\_\{k\}\(\\mathbf\{w\}\),\(1\)whereFk​\(𝐰\)=1nk​∑i∈𝒟kℓ​\(𝐰;xi\)F\_\{k\}\(\\mathbf\{w\}\)=\\frac\{1\}\{n\_\{k\}\}\\sum\_\{i\\in\\mathcal\{D\}\_\{k\}\}\\ell\(\\mathbf\{w\};x\_\{i\}\)is the empirical local objective of clientkk,n=∑k=1Knkn=\\sum\_\{k=1\}^\{K\}n\_\{k\}is the total number of samples,ℓ​\(𝐰;xi\)\\ell\(\\mathbf\{w\};x\_\{i\}\)is the per\-sample loss, and𝐰∈ℝd\\mathbf\{w\}\\in\\mathbb\{R\}^\{d\}denotes the global model parameters\. In communication roundtt, the server broadcasts the current global model𝐰t\\mathbf\{w\}^\{t\}to a selected subset𝒮t⊆𝒦\\mathcal\{S\}^\{t\}\\subseteq\\mathcal\{K\}of participating clients\. Each clientk∈𝒮tk\\in\\mathcal\{S\}^\{t\}performsEEsteps of local stochastic gradient descent starting from𝐰t\\mathbf\{w\}^\{t\}, producing a locally updated model𝐰kt\\mathbf\{w\}\_\{k\}^\{t\}\. The transmitted local update is the accumulated model delta

Δ​𝐰kt=𝐰t−𝐰kt,\\Delta\\mathbf\{w\}\_\{k\}^\{t\}=\\mathbf\{w\}^\{t\}\-\\mathbf\{w\}\_\{k\}^\{t\},\(2\)which captures the net parameter change afterEElocal steps\. The server aggregates these deltas over participating clients only:

𝐰t\+1=𝐰t−∑k∈𝒮tnkn𝒮t​Δ​𝐰kt,\\mathbf\{w\}^\{t\+1\}=\\mathbf\{w\}^\{t\}\-\\sum\_\{k\\in\\mathcal\{S\}^\{t\}\}\\frac\{n\_\{k\}\}\{n\_\{\\mathcal\{S\}^\{t\}\}\}\\Delta\\mathbf\{w\}\_\{k\}^\{t\},\(3\)wheren𝒮t=∑k∈𝒮tnkn\_\{\\mathcal\{S\}^\{t\}\}=\\sum\_\{k\\in\\mathcal\{S\}^\{t\}\}n\_\{k\}is the total number of samples held by participating clients at roundtt\. For each layerℓ\\ell, letGk\(ℓ\)∈ℝmℓ×nℓG\_\{k\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{m\_\{\\ell\}\\times n\_\{\\ell\}\}denote the layer\-wise component of the local update, defined as the difference between the global and locally updated parameters of that layer afterEEsteps\. This layer\-wise update matrix is compressed before transmission\.

### III\-BModality\-Heterogeneous Clients

We consider a setting in which clients differ in sensing capability\. Letℳ\\mathcal\{M\}denote the set of available modalities\. Each clientkkhas access to a subsetℳk⊆ℳ\\mathcal\{M\}\_\{k\}\\subseteq\\mathcal\{M\}determined by its hardware configuration\. The global model consists of modality\-specific encoder branches and a shared fusion module\. Formally,

𝐰=\{𝐰\(ℓ\)\}ℓ=1L,\\mathbf\{w\}=\\left\\\{\\mathbf\{w\}^\{\(\\ell\)\}\\right\\\}\_\{\\ell=1\}^\{L\},\(4\)whereLLis the total number of layers and each𝐰\(ℓ\)\\mathbf\{w\}^\{\(\\ell\)\}belongs either to a modality\-specific encoder or to the shared fusion network\. Clientkkcomputes local updates only for layers associated with its available modalitiesℳk\\mathcal\{M\}\_\{k\}\. Layers corresponding to absent modalities produce no update contribution from clientkk\. Letℒk\\mathcal\{L\}\_\{k\}denote the set of layers updated by clientkk\. This naturally partitions clients by modality profile, with unimodal clients contributing updates for a single encoder branch and multimodal clients contributing updates for multiple branches and the fusion module\.

### III\-CDevice Heterogeneity

Beyond modality heterogeneity, clients also differ in computation and memory capacity\. We characterize each clientkkby a compression budgetCkC\_\{k\}, defined as the maximum total number of scalar values that may be transmitted per round\. This budget serves as a unified abstraction of uplink bandwidth and on\-device memory constraints, and is consistently interpreted throughout the paper as a scalar payload limit\. We do not assume prior knowledge of the data distribution or update statistics at the server; all compression decisions are made locally at each client based on its own updates and device profile\.

### III\-DMPS Update Compression

Before transmission, each layer\-wise update matrixGk\(ℓ\)∈ℝmℓ×nℓG\_\{k\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{m\_\{\\ell\}\\times n\_\{\\ell\}\}is compressed using a matrix product state \(MPS\) decomposition\. Since MPS operates on higher\-order tensors, the update matrix is first reshaped into a third\-order tensor𝒢k\(ℓ\)∈ℝm1×m2×nℓ\\mathcal\{G\}\_\{k\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{m\_\{1\}\\times m\_\{2\}\\times n\_\{\\ell\}\}, wherem1m\_\{1\}andm2m\_\{2\}are chosen such thatm1​m2≥mℓm\_\{1\}m\_\{2\}\\geq m\_\{\\ell\}and\|m1−m2\|\|m\_\{1\}\-m\_\{2\}\|is minimized\. Whenm1​m2\>mℓm\_\{1\}m\_\{2\}\>m\_\{\\ell\}, the matrix is zero\-padded along its first dimension before reshaping, and the padded rows are discarded after reconstruction\. This tensorization scheme is the one adopted in our framework; other reshaping strategies are possible but are not considered here\. The MPS decomposition expresses this tensor as:

𝒢k\(ℓ\)​\(i1,i2,i3\)≈𝐀\(1\)​\[i1\]​𝐀\(2\)​\[i2\]​𝐀\(3\)​\[i3\],\\mathcal\{G\}\_\{k\}^\{\(\\ell\)\}\(i\_\{1\},i\_\{2\},i\_\{3\}\)\\approx\\mathbf\{A\}^\{\(1\)\}\[i\_\{1\}\]\\,\\mathbf\{A\}^\{\(2\)\}\[i\_\{2\}\]\\,\\mathbf\{A\}^\{\(3\)\}\[i\_\{3\}\],\(5\)where𝐀\(1\)∈ℝm1×1×r\\mathbf\{A\}^\{\(1\)\}\\in\\mathbb\{R\}^\{m\_\{1\}\\times 1\\times r\},𝐀\(2\)∈ℝr×m2×r\\mathbf\{A\}^\{\(2\)\}\\in\\mathbb\{R\}^\{r\\times m\_\{2\}\\times r\}, and𝐀\(3\)∈ℝr×nℓ×1\\mathbf\{A\}^\{\(3\)\}\\in\\mathbb\{R\}^\{r\\times n\_\{\\ell\}\\times 1\}, withr=rk\(ℓ\)r=r\_\{k\}^\{\(\\ell\)\}denoting the bond dimension, or MPS rank\. The total payload size of the compressed representation is

ϕ​\(mℓ,nℓ,rk\(ℓ\)\)=m1​r\+m2​r2\+r​nℓ,\\phi\\\!\\left\(m\_\{\\ell\},n\_\{\\ell\},r\_\{k\}^\{\(\\ell\)\}\\right\)=m\_\{1\}r\+m\_\{2\}r^\{2\}\+rn\_\{\\ell\},\(6\)where the dominant termm2​r2m\_\{2\}r^\{2\}arises from the middle core𝐀\(2\)\\mathbf\{A\}^\{\(2\)\}, making the payload size quadratic inrr\. The corresponding compression ratio is

ρk\(ℓ\)=mℓ​nℓϕ​\(mℓ,nℓ,rk\(ℓ\)\)\.\\rho\_\{k\}^\{\(\\ell\)\}=\\frac\{m\_\{\\ell\}n\_\{\\ell\}\}\{\\phi\(m\_\{\\ell\},n\_\{\\ell\},r\_\{k\}^\{\(\\ell\)\}\)\}\.\(7\)Thus,rk\(ℓ\)r\_\{k\}^\{\(\\ell\)\}is the primary decision variable governing the trade\-off between payload size and reconstruction fidelity\.

### III\-EProblem Statement

The central challenge is to select compression ranks\{rk\(ℓ\)\}\\\{r\_\{k\}^\{\(\\ell\)\}\\\}across clients and layers so as to minimize total update reconstruction error subject to per\-client compression budgets:

min\{rk\(ℓ\)\}​∑k∈𝒮tnkn𝒮t​∑ℓ∈ℒkεk,ℓ2​\(rk\(ℓ\)\)\\displaystyle\\min\_\{\\left\\\{r\_\{k\}^\{\(\\ell\)\}\\right\\\}\}\\sum\_\{k\\in\\mathcal\{S\}^\{t\}\}\\frac\{n\_\{k\}\}\{n\_\{\\mathcal\{S\}^\{t\}\}\}\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\varepsilon\_\{k,\\ell\}^\{2\}\\\!\\left\(r\_\{k\}^\{\(\\ell\)\}\\right\)\\quads\.t\.∑ℓ∈ℒkϕ​\(mℓ,nℓ,rk\(ℓ\)\)≤Ck,∀k∈𝒮t,\\displaystyle\\text\{s\.t\.\}\\quad\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\\\!\\left\(m\_\{\\ell\},n\_\{\\ell\},r\_\{k\}^\{\(\\ell\)\}\\right\)\\leq C\_\{k\},\\quad\\forall k\\in\\mathcal\{S\}^\{t\},\(8\)whereεk,ℓ​\(r\)=‖Gk\(ℓ\)−G^k\(ℓ\)‖F‖Gk\(ℓ\)‖F\\varepsilon\_\{k,\\ell\}\(r\)=\\frac\{\\\|G\_\{k\}^\{\(\\ell\)\}\-\\hat\{G\}\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}\}\{\\\|G\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}\}is the relative reconstruction error of layerℓ\\ellat clientkkunder rank\-rrMPS compression, andG^k\(ℓ\)\\hat\{G\}\_\{k\}^\{\(\\ell\)\}denotes the reconstructed update\. The constraint is dimensionally consistent: the left\-hand side counts the total number of transmitted scalars under MPS compression, andCkC\_\{k\}is measured in the same units\. This formulation captures both sources of heterogeneity jointly: the constraint set is client\-specific throughCkC\_\{k\}, while the objective reflects modality participation throughℒk\\mathcal\{L\}\_\{k\}\.

Solving this problem requires understanding howεk,ℓ​\(r\)\\varepsilon\_\{k,\\ell\}\(r\)depends on the intrinsic structure of the updateGk\(ℓ\)G\_\{k\}^\{\(\\ell\)\}\. The key insight developed in Section[V](https://arxiv.org/html/2607.06651#S5)is that the spectral entropy ofGk\(ℓ\)G\_\{k\}^\{\(\\ell\)\}provides a principled basis for rank selection\. In our framework, spectral entropy is estimated efficiently via a truncated singular value decomposition that retains only the topqqsingular values to estimate the singular\-value energy distribution, whereq≪min⁡\(mℓ,nℓ\)q\\ll\\min\(m\_\{\\ell\},n\_\{\\ell\}\), thereby avoiding the cost of full decomposition at runtime, as detailed in Section[IV](https://arxiv.org/html/2607.06651#S4)\.

## IVMESH\-FL Framework

MESH\-FL operates in three steps per round: each participating client estimates the spectral entropy of its layer\-wise updates via truncated SVD, allocates entropy\-guided MPS ranks under its local payload budgetCkC\_\{k\}, and compresses active updates into MPS cores for transmission\. The server reconstructs the received updates, aggregates them layer\-wise across eligible clients, and broadcasts the updated global model\. Only global parameters in modality\-specific encoders and the shared fusion module are transmitted, while local classifier and fusion\-adapter parameters remain on\-device\.

### IV\-ASpectral Entropy Estimation

For each active layerℓ∈ℒk\\ell\\in\\mathcal\{L\}\_\{k\}, clientkkestimates the spectral entropy of its update matrixGk\(ℓ\)∈ℝmℓ×nℓG\_\{k\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{m\_\{\\ell\}\\times n\_\{\\ell\}\}via truncated SVD\. Let

σ^1≥σ^2≥⋯≥σ^q\\displaystyle\\hat\{\\sigma\}\_\{1\}\\geq\\hat\{\\sigma\}\_\{2\}\\geq\\cdots\\geq\\hat\{\\sigma\}\_\{q\}\(9\)denote the topqqsingular values ofGk\(ℓ\)G\_\{k\}^\{\(\\ell\)\}, whereq≪min⁡\(mℓ,nℓ\)q\\ll\\min\(m\_\{\\ell\},n\_\{\\ell\}\)is a fixed truncation parameter\. The normalized singular\-value energy distribution over the retained components is

q^i=σ^i2∑j=1qσ^j2,i=1,…,q,\\displaystyle\\hat\{q\}\_\{i\}=\\frac\{\\hat\{\\sigma\}\_\{i\}^\{2\}\}\{\\sum\_\{j=1\}^\{q\}\\hat\{\\sigma\}\_\{j\}^\{2\}\},\\quad i=1,\\ldots,q,\(10\)and the resulting truncated spectral entropy estimator is

H^k\(ℓ\)=−∑i=1qq^i​log⁡q^i\.\\displaystyle\\hat\{H\}\_\{k\}^\{\(\\ell\)\}=\-\\sum\_\{i=1\}^\{q\}\\hat\{q\}\_\{i\}\\log\\hat\{q\}\_\{i\}\.\(11\)
We useH^k\(ℓ\)\\hat\{H\}\_\{k\}^\{\(\\ell\)\}as a computational proxy for the true spectral entropyHk\(ℓ\)H\_\{k\}^\{\(\\ell\)\}defined over the full singular\-value energy distribution\. The quality of this proxy depends on the omitted tail energy

τk\(ℓ\)=∑i\>qσi2‖Gk\(ℓ\)‖F2,\\displaystyle\\tau\_\{k\}^\{\(\\ell\)\}=\\frac\{\\sum\_\{i\>q\}\\sigma\_\{i\}^\{2\}\}\{\\\|G\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}\},\(12\)which measures the fraction of spectral energy not captured by the topqqsingular values\. Whenτk\(ℓ\)\\tau\_\{k\}^\{\(\\ell\)\}is small, the truncated entropyH^k\(ℓ\)\\hat\{H\}\_\{k\}^\{\(\\ell\)\}approximatesHk\(ℓ\)H\_\{k\}^\{\(\\ell\)\}up to a controlled deviation\. In particular, the entropy\-guided rank ordering induced byH^\\hat\{H\}is stable under small tail energy, and pairwise ordering is preserved whenever the true entropy gaps exceed the corresponding perturbation bound\.

In practice,qqis chosen so that the retained singular values capture the dominant spectral mass of each update and the resulting entropy estimates are stable across rounds\. Using truncated rather than full SVD reduces the per\-layer cost fromO​\(mℓ​nℓ​min⁡\(mℓ,nℓ\)\)O\(m\_\{\\ell\}n\_\{\\ell\}\\min\(m\_\{\\ell\},n\_\{\\ell\}\)\)toO​\(mℓ​nℓ​q\)O\(m\_\{\\ell\}n\_\{\\ell\}q\)with randomized SVD, which is practical on resource\-constrained devices\. Entropy estimation is performed independently for each active layer, allowing rank allocation to adapt to layer\-specific and modality\-specific update structure\.

### IV\-BDevice\-Aware Rank Allocation

Given the entropy estimates\{H^k\(ℓ\)\}ℓ∈ℒk\\\{\\hat\{H\}\_\{k\}^\{\(\\ell\)\}\\\}\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}and payload budgetCkC\_\{k\}, clientkkallocates MPS ranks across its active layers\. As established in Section[V](https://arxiv.org/html/2607.06651#S5), the nominal continuous allocation is monotone in spectral entropy, and under the linearized payload model admits the closed\-form scalingrk\(ℓ\)∝eH^k\(ℓ\)/2r\_\{k\}^\{\(\\ell\)\}\\propto e^\{\\hat\{H\}\_\{k\}^\{\(\\ell\)\}/2\}\. We therefore define

r¯k\(ℓ\)=αk​eH^k\(ℓ\)/2,ℓ∈ℒk,\\displaystyle\\bar\{r\}\_\{k\}^\{\(\\ell\)\}=\\alpha\_\{k\}e^\{\\hat\{H\}\_\{k\}^\{\(\\ell\)\}/2\},\\quad\\ell\\in\\mathcal\{L\}\_\{k\},\(13\)whereαk\>0\\alpha\_\{k\}\>0is a client\-specific scaling factor chosen so that the resulting allocation matches the available payload budget as closely as possible under the MPS payload model\. Since the payload function

ϕ​\(mℓ,nℓ,r\)=m1​r\+m2​r2\+r​nℓ\\displaystyle\\phi\(m\_\{\\ell\},n\_\{\\ell\},r\)=m\_\{1\}r\+m\_\{2\}r^\{2\}\+rn\_\{\\ell\}\(14\)is monotone increasing inrr, the scaling factorαk\\alpha\_\{k\}can be obtained efficiently by bisection on

∑ℓ∈ℒkϕ​\(mℓ,nℓ,r¯k\(ℓ\)\)≤Ck\.\\displaystyle\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\\\!\\left\(m\_\{\\ell\},n\_\{\\ell\},\\bar\{r\}\_\{k\}^\{\(\\ell\)\}\\right\)\\leq C\_\{k\}\.\(15\)
To obtain implementable integer ranks, the continuous allocation is projected onto device\-feasible bounds:

r~k\(ℓ\)=Πℤ∩\[rmin,rmax\(k\)\]​\(r¯k\(ℓ\)\),\\displaystyle\\tilde\{r\}\_\{k\}^\{\(\\ell\)\}=\\Pi\_\{\\mathbb\{Z\}\\cap\[r\_\{\\min\},\\,r\_\{\\max\}^\{\(k\)\}\]\}\\\!\\left\(\\bar\{r\}\_\{k\}^\{\(\\ell\)\}\\right\),\(16\)whererminr\_\{\\min\}is a global minimum rank andrmax\(k\)r\_\{\\max\}^\{\(k\)\}is a device\-specific maximum rank determined by clientkk’s available working memory\. Because integer projection may slightly violate the payload budget, we apply a lightweight feasibility correction: if

∑ℓ∈ℒkϕ​\(mℓ,nℓ,r~k\(ℓ\)\)\>Ck,\\displaystyle\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\\\!\\left\(m\_\{\\ell\},n\_\{\\ell\},\\tilde\{r\}\_\{k\}^\{\(\\ell\)\}\\right\)\>C\_\{k\},\(17\)the client iteratively decreases the ranks of the lowest\-entropy layers until the budget is satisfied; if residual budget remains, it is reassigned to the highest\-entropy layers while respectingrmax\(k\)r\_\{\\max\}^\{\(k\)\}\. The final ranks are denoted by\{rk\(ℓ\)\}ℓ∈ℒk\\\{r\_\{k\}^\{\(\\ell\)\}\\\}\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\.

This procedure preserves the entropy\-guided structure implied by the theory while ensuring exact budget feasibility under integer ranks and device bounds\. Layers with higher spectral entropy receive higher rank, layers with lower entropy are compressed more aggressively, and every client remains within its payload limit\. For bias vectors and other one\-dimensional parameter tensors, we directly assign rankrminr\_\{\\min\}, since their small size makes both entropy estimation and compression gains marginal\.

### IV\-CMPS Compression and Decompression

Given the allocated rankrk\(ℓ\)r\_\{k\}^\{\(\\ell\)\}for each active layer, clientkkcompresses the update matrixGk\(ℓ\)G\_\{k\}^\{\(\\ell\)\}using the MPS construction introduced in Section[III](https://arxiv.org/html/2607.06651#S3)\. If necessary, the matrix is zero\-padded, reshaped into a third\-order tensor, and factorized through two sequential truncated SVD steps to produce the cores𝐀\(1\)\\mathbf\{A\}^\{\(1\)\},𝐀\(2\)\\mathbf\{A\}^\{\(2\)\}, and𝐀\(3\)\\mathbf\{A\}^\{\(3\)\}\. The singular values from each intermediate decomposition are absorbed into the subsequent core so that the server can reconstruct the update directly from the transmitted cores without storing singular values separately\. The total transmitted payload for layerℓ\\ellat clientkkis therefore

ϕ​\(mℓ,nℓ,rk\(ℓ\)\)=m1​rk\(ℓ\)\+m2​\(rk\(ℓ\)\)2\+rk\(ℓ\)​nℓ\.\\phi\\\!\\left\(m\_\{\\ell\},n\_\{\\ell\},r\_\{k\}^\{\(\\ell\)\}\\right\)=m\_\{1\}r\_\{k\}^\{\(\\ell\)\}\+m\_\{2\}\\bigl\(r\_\{k\}^\{\(\\ell\)\}\\bigr\)^\{2\}\+r\_\{k\}^\{\(\\ell\)\}n\_\{\\ell\}\.\(18\)
At the server, decompression contracts the received cores sequentially to recoverG^k\(ℓ\)\\hat\{G\}\_\{k\}^\{\(\\ell\)\}, removes any padded rows, and reshapes the result to the original layer dimensions\. Since decompression is performed on the server side, its overhead is modest relative to server\-scale compute resources\. The reconstructed layer\-wise updates are then aggregated across clients as in FL, with the important distinction that modality\-specific layers are aggregated only over clients that actually contribute to them\.

### IV\-DLayer\-Wise Aggregation Under Modality Heterogeneity

Because not every client updates every layer, aggregation must be defined layer\-wise\. For each layerℓ\\ell, let𝒮ℓt=\{k∈𝒮t:ℓ∈ℒk\}\\mathcal\{S\}\_\{\\ell\}^\{t\}=\\\{k\\in\\mathcal\{S\}^\{t\}:\\ell\\in\\mathcal\{L\}\_\{k\}\\\}denote the set of participating clients that contribute an update to layerℓ\\ell, and letn𝒮ℓt=∑k∈𝒮ℓtnkn\_\{\\mathcal\{S\}\_\{\\ell\}^\{t\}\}=\\sum\_\{k\\in\\mathcal\{S\}\_\{\\ell\}^\{t\}\}n\_\{k\}denote the total number of samples represented by those clients\. The server then computes the aggregated update for layerℓ\\ellas

G^\(ℓ\)=∑k∈𝒮ℓtnkn𝒮ℓt​G^k\(ℓ\)\.\\displaystyle\\hat\{G\}^\{\(\\ell\)\}=\\sum\_\{k\\in\\mathcal\{S\}\_\{\\ell\}^\{t\}\}\\frac\{n\_\{k\}\}\{n\_\{\\mathcal\{S\}\_\{\\ell\}^\{t\}\}\}\\hat\{G\}\_\{k\}^\{\(\\ell\)\}\.\(19\)If𝒮ℓt=∅\\mathcal\{S\}\_\{\\ell\}^\{t\}=\\emptyset, no client contributes to layerℓ\\ellin roundtt, and the corresponding parameters remain unchanged\. This layer\-wise normalization avoids diluting modality\-specific updates by averaging over clients that do not possess the relevant sensing branch\.

### IV\-EComplete Protocol

Algorithm[1](https://arxiv.org/html/2607.06651#alg1)summarizes the complete MESH\-FL workflow\. In each communication round, the server broadcasts the current global model to the selected clients, and each client performsEElocal SGD steps to obtain layer\-wise updates only for the layers in its modality\-dependent active setℒk\\mathcal\{L\}\_\{k\}\. For each active layer, the client estimates the truncated spectral entropyH^k\(ℓ\)\\hat\{H\}\_\{k\}^\{\(\\ell\)\}from the top\-qqsingular\-value energy distribution and uses these estimates to allocate MPS ranks\{rk\(ℓ\)\}ℓ∈ℒk\\\{r\_\{k\}^\{\(\\ell\)\}\\\}\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}under the payload budget∑ℓ∈ℒkϕ​\(mℓ,nℓ,rk\(ℓ\)\)≤Ck\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\(m\_\{\\ell\},n\_\{\\ell\},r\_\{k\}^\{\(\\ell\)\}\)\\leq C\_\{k\}\. The resulting ranks assign higher reconstruction capacity to spectrally diffuse updates and stronger compression to low\-entropy updates\. Clients then transmit only the compressed MPS cores, while the server reconstructs the updates and aggregates each layer over the modality\-eligible clientsSℓt=k∈St:ℓ∈ℒkS\_\{\\ell\}^\{t\}=\{k\\in S^\{t\}:\\ell\\in\\mathcal\{L\}\_\{k\}\}\. Thus, compared with standard FedAvg, MESH\-FL jointly introduces entropy\-guided rank allocation, device\-aware MPS update compression, and layer\-wise aggregation under modality heterogeneity\.

Input:

𝐰0\\mathbf\{w\}^\{0\}, rounds

TT, local steps

EE, truncation

qq, rank bounds

rmin,\{rmax\(k\)\}r\_\{\\min\},\\\{r\_\{\\max\}^\{\(k\)\}\\\}, budgets

\{Ck\}\\\{C\_\{k\}\\\}
Output:

𝐰T\\mathbf\{w\}^\{T\}
1

2for*t=1,…,Tt=1,\\ldots,T*do

3Server selects

𝒮t⊆𝒦\\mathcal\{S\}^\{t\}\\subseteq\\mathcal\{K\}and broadcasts

𝐰t\\mathbf\{w\}^\{t\}
4

5foreach*k∈𝒮tk\\in\\mathcal\{S\}^\{t\}in parallel*do

6Run

EElocal SGD steps from

𝐰t\\mathbf\{w\}^\{t\}to obtain

𝐰kt\\mathbf\{w\}\_\{k\}^\{t\}
7

8foreach*ℓ∈ℒk\\ell\\in\\mathcal\{L\}\_\{k\}*do

9

Gk\(ℓ\)←𝐰t,\(ℓ\)−𝐰kt,\(ℓ\)G\_\{k\}^\{\(\\ell\)\}\\leftarrow\\mathbf\{w\}^\{t,\(\\ell\)\}\-\\mathbf\{w\}\_\{k\}^\{t,\(\\ell\)\}
10Estimate

H^k\(ℓ\)\\hat\{H\}\_\{k\}^\{\(\\ell\)\}from the top\-

qqsingular\-value energy of

Gk\(ℓ\)G\_\{k\}^\{\(\\ell\)\}
11

12end foreach

13

14Allocate

\{rk\(ℓ\)\}ℓ∈ℒk\\\{r\_\{k\}^\{\(\\ell\)\}\\\}\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}using

\{H^k\(ℓ\)\}\\\{\\hat\{H\}\_\{k\}^\{\(\\ell\)\}\\\}subject to

∑ℓ∈ℒkϕ​\(mℓ,nℓ,rk\(ℓ\)\)≤Ck\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\(m\_\{\\ell\},n\_\{\\ell\},r\_\{k\}^\{\(\\ell\)\}\)\\leq C\_\{k\}
15

16Compress and transmit MPS cores of

Gk\(ℓ\)G\_\{k\}^\{\(\\ell\)\}at rank

rk\(ℓ\)r\_\{k\}^\{\(\\ell\)\}for all

ℓ∈ℒk\\ell\\in\\mathcal\{L\}\_\{k\}
17Update

𝐰kloc\\mathbf\{w\}\_\{k\}^\{\\mathrm\{loc\}\}
18

19end foreach

20

21foreach*ℓ=1,…,L\\ell=1,\\ldots,L*do

22

𝒮ℓt←\{k∈𝒮t:ℓ∈ℒk\}\\mathcal\{S\}\_\{\\ell\}^\{t\}\\leftarrow\\\{k\\in\\mathcal\{S\}^\{t\}:\\ell\\in\\mathcal\{L\}\_\{k\}\\\}
23if*𝒮ℓt≠∅\\mathcal\{S\}\_\{\\ell\}^\{t\}\\neq\\emptyset*then

24

G^\(ℓ\)←∑k∈𝒮ℓtnkn𝒮ℓt​MPSDec⁡\(coresk\(ℓ\)\)\\hat\{G\}^\{\(\\ell\)\}\\leftarrow\\sum\_\{k\\in\\mathcal\{S\}\_\{\\ell\}^\{t\}\}\\frac\{n\_\{k\}\}\{n\_\{\\mathcal\{S\}\_\{\\ell\}^\{t\}\}\}\\operatorname\{MPSDec\}\\\!\\left\(\\operatorname\{cores\}\_\{k\}^\{\(\\ell\)\}\\right\)
25

𝐰t\+1,\(ℓ\)←𝐰t,\(ℓ\)−G^\(ℓ\)\\mathbf\{w\}^\{t\+1,\(\\ell\)\}\\leftarrow\\mathbf\{w\}^\{t,\(\\ell\)\}\-\\hat\{G\}^\{\(\\ell\)\}
26

27else

28

𝐰t\+1,\(ℓ\)←𝐰t,\(ℓ\)\\mathbf\{w\}^\{t\+1,\(\\ell\)\}\\leftarrow\\mathbf\{w\}^\{t,\(\\ell\)\}
29

30end if

31

32end foreach

33

34end for

return*𝐰T\\mathbf\{w\}^\{T\}*

Algorithm 1MESH\-FL Work Flow
### IV\-FComplexity Analysis

Relative to standard FedAvg, MESH\-FL introduces two main client\-side overheads: entropy estimation and MPS compression\. For clientkk, truncated\-SVD\-based entropy estimation has complexity

∑ℓ∈ℒkO​\(mℓ​nℓ​q\),\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}O\(m\_\{\\ell\}n\_\{\\ell\}q\),\(20\)whereq≪min⁡\(mℓ,nℓ\)q\\ll\\min\(m\_\{\\ell\},n\_\{\\ell\}\)\. The MPS compression cost is approximately

∑ℓ∈ℒkO​\(mℓ​nℓ​rk\(ℓ\)\),\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}O\(m\_\{\\ell\}n\_\{\\ell\}r\_\{k\}^\{\(\\ell\)\}\),\(21\)which is comparable to low\-rank factorization at the target rank\. The rank\-allocation step is lightweight relative to these matrix operations, since it consists of entropy\-weight computation, one\-dimensional bisection forαk\\alpha\_\{k\}, and a small feasibility correction over active layers\.

On the server side, decompression and aggregation incur

∑ℓ=1L∑k∈𝒮ℓtO​\(m1​m2​\(rk\(ℓ\)\)2\+m1​m2​rk\(ℓ\)​nℓ\),\\sum\_\{\\ell=1\}^\{L\}\\sum\_\{k\\in\\mathcal\{S\}\_\{\\ell\}^\{t\}\}O\\\!\\left\(m\_\{1\}m\_\{2\}\\bigl\(r\_\{k\}^\{\(\\ell\)\}\\bigr\)^\{2\}\+m\_\{1\}m\_\{2\}\\,r\_\{k\}^\{\(\\ell\)\}n\_\{\\ell\}\\right\),\(22\)which is modest at server scale\. The main efficiency gain remains in communication: the payload per client per round is reduced from

∑ℓ∈ℒkmℓ​nℓ\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}m\_\{\\ell\}n\_\{\\ell\}\(23\)to

∑ℓ∈ℒkϕ​\(mℓ,nℓ,rk\(ℓ\)\),\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\(m\_\{\\ell\},n\_\{\\ell\},r\_\{k\}^\{\(\\ell\)\}\),\(24\)with larger savings obtained when the allocated ranks remain small relative to the layer dimensions\.

## VTheoretical Analysis

This section provides the theoretical basis for MESH\-FL\. We first bound the reconstruction error of rank\-constrained MPS compression and express it through an entropy\-weighted spectral\-complexity coefficient\. We then relate spectral entropy to rank complexity, show how compression error enters the FL convergence bound, and prove that the proposed entropy\-guided rank allocation solves a convex surrogate problem\.

### V\-AAssumptions

We use standard conditions from non\-IID FL convergence analysis\. Each local objectiveFkF\_\{k\}is differentiable andLL\-smooth, i\.e\.,

‖∇Fk​\(𝐮\)−∇Fk​\(𝐯\)‖≤L​‖𝐮−𝐯‖,∀𝐮,𝐯∈ℝd\.\\displaystyle\\\|\\nabla F\_\{k\}\(\\mathbf\{u\}\)\-\\nabla F\_\{k\}\(\\mathbf\{v\}\)\\\|\\leq L\\\|\\mathbf\{u\}\-\\mathbf\{v\}\\\|,\\quad\\forall\\,\\mathbf\{u\},\\mathbf\{v\}\\in\\mathbb\{R\}^\{d\}\.\(25\)The uncompressed local deltaGktG\_\{k\}^\{t\}, produced afterEElocal SGD steps from𝐰t\\mathbf\{w\}^\{t\}, satisfies

𝔼​\[Gkt∣𝐰t\]\\displaystyle\\mathbb\{E\}\[G\_\{k\}^\{t\}\\mid\\mathbf\{w\}^\{t\}\]=η​E​∇Fk​\(𝐰t\),\\displaystyle=\\eta E\\nabla F\_\{k\}\(\\mathbf\{w\}^\{t\}\),\(26\)𝔼​\[‖Gkt−η​E​∇Fk​\(𝐰t\)‖2∣𝐰t\]\\displaystyle\\mathbb\{E\}\\\!\\left\[\\\|G\_\{k\}^\{t\}\-\\eta E\\nabla F\_\{k\}\(\\mathbf\{w\}^\{t\}\)\\\|^\{2\}\\mid\\mathbf\{w\}^\{t\}\\right\]≤η2​E2​σ2,\\displaystyle\\leq\\eta^\{2\}E^\{2\}\\sigma^\{2\},\(27\)whereη\\etais the step size andσ2\\sigma^\{2\}controls the stochastic local\-update variance\. Client heterogeneity is bounded by

‖∇Fk​\(𝐰\)−∇F​\(𝐰\)‖2≤δ2,∇F​\(𝐰\)=∑k=1Knkn​∇Fk​\(𝐰\)\.\\displaystyle\\\|\\nabla F\_\{k\}\(\\mathbf\{w\}\)\-\\nabla F\(\\mathbf\{w\}\)\\\|^\{2\}\\leq\\delta^\{2\},\\quad\\nabla F\(\\mathbf\{w\}\)=\\sum\_\{k=1\}^\{K\}\\frac\{n\_\{k\}\}\{n\}\\nabla F\_\{k\}\(\\mathbf\{w\}\)\.\(28\)These conditions are standard and are used only for the convergence analysis\. The following assumption is specific to the spectral approximation analysis of MESH\-FL\.

###### Assumption V\.1\(Exponential tail decay of unfolding spectra\)\.

For each unfoldings∈\{1,2\}s\\in\\\{1,2\\\}, there exist constantsAs\>0A\_\{s\}\>0andαs\>0\\alpha\_\{s\}\>0such that the normalized tail energy satisfies

∑j\>rqj\(s\)≤As​e−αs​r,r=1,2,…\.\\displaystyle\\sum\_\{j\>r\}q\_\{j\}^\{\(s\)\}\\leq A\_\{s\}e^\{\-\\alpha\_\{s\}r\},\\quad r=1,2,\\ldots\.\(29\)

Assumption[V\.1](https://arxiv.org/html/2607.06651#S5.Thmassumption1)is a sufficient technical condition for deriving a closed\-form entropy\-weighted MPS approximation bound\. It is used only to establish Proposition[V\.1](https://arxiv.org/html/2607.06651#S5.Thmproposition1)and is not required for the convergence argument in Theorem[V\.2](https://arxiv.org/html/2607.06651#S5.Thmtheorem2)or for the convexity and monotonicity of the rank\-allocation problem in Theorem[V\.3](https://arxiv.org/html/2607.06651#S5.Thmtheorem3)\. The assumption is not meant to require that every layer and every communication round exhibit exact exponential spectral decay\. In practice, the allocation rule relies on the weaker ordering principle that layers with more diffuse singular\-value energy distributions require larger reconstruction ranks\. Thus, even when empirical tail decay is slower than exponential, the1/r1/r\-type surrogate remains a tractable proxy for rank allocation, while the constants and tightness of the approximation bound may become less favorable\.

### V\-BMPS Approximation Error and Entropy Complexity

We first quantify the approximation error introduced by sequential MPS compression\. For each unfoldings∈\{1,2\}s\\in\\\{1,2\\\}, define the normalized singular\-value energy distribution and its Shannon entropy as

qi\(s\)=\(σi\(s\)\)2‖𝒢\(s\)‖F2,H\(s\)​\(G\)=−∑i≥1qi\(s\)​log⁡qi\(s\)\.\\displaystyle q\_\{i\}^\{\(s\)\}=\\frac\{\(\\sigma\_\{i\}^\{\(s\)\}\)^\{2\}\}\{\\\|\\mathcal\{G\}^\{\(s\)\}\\\|\_\{F\}^\{2\}\},\\quad H^\{\(s\)\}\(G\)=\-\\sum\_\{i\\geq 1\}q\_\{i\}^\{\(s\)\}\\log q\_\{i\}^\{\(s\)\}\.\(30\)The following proposition converts the discarded spectral energy of the two MPS unfoldings into a rank\-dependent reconstruction\-error bound\.

###### Proposition V\.1\(Entropy\-weighted MPS error bound\)\.

LetG∈ℝm×nG\\in\\mathbb\{R\}^\{m\\times n\}be reshaped into a third\-order tensor𝒢∈ℝm1×m2×n\\mathcal\{G\}\\in\\mathbb\{R\}^\{m\_\{1\}\\times m\_\{2\}\\times n\}, with first unfolding𝒢\(1\)∈ℝm1×m2​n\\mathcal\{G\}^\{\(1\)\}\\in\\mathbb\{R\}^\{m\_\{1\}\\times m\_\{2\}n\}and second unfolding𝒢\(2\)\\mathcal\{G\}^\{\(2\)\}obtained after the first rank\-rrtruncation\. Under Assumption[V\.1](https://arxiv.org/html/2607.06651#S5.Thmassumption1), the relative MPS reconstruction error of layerℓ\\ellat clientkksatisfies

εk,ℓ2​\(r\)=‖Gk\(ℓ\)−G^k,r\(ℓ\)‖F2‖Gk\(ℓ\)‖F2≤Ψk,ℓr,\\displaystyle\\varepsilon\_\{k,\\ell\}^\{2\}\(r\)=\\frac\{\\\|G\_\{k\}^\{\(\\ell\)\}\-\\widehat\{G\}\_\{k,r\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}\}\{\\\|G\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}\}\\leq\\frac\{\\Psi\_\{k,\\ell\}\}\{r\},\(31\)whereΨk,ℓ:=C1,ℓ​eHk,ℓ\(1\)\+C2,ℓ​eHk,ℓ\(2\),\\Psi\_\{k,\\ell\}:=C\_\{1,\\ell\}e^\{H\_\{k,\\ell\}^\{\(1\)\}\}\+C\_\{2,\\ell\}e^\{H\_\{k,\\ell\}^\{\(2\)\}\},Hk,ℓ\(s\):=H\(s\)​\(Gk\(ℓ\)\)H\_\{k,\\ell\}^\{\(s\)\}:=H^\{\(s\)\}\(G\_\{k\}^\{\(\\ell\)\}\)is the energy entropy of thessth unfolding, andC1,ℓ,C2,ℓ\>0C\_\{1,\\ell\},C\_\{2,\\ell\}\>0depend only on the tail parameters\(As,αs\)\(A\_\{s\},\\alpha\_\{s\}\)and the fixed tensorization of layerℓ\\ell\.

###### Proof\.

The sequential MPS construction applies two rank\-rrtruncated SVD steps\. The first truncation is performed on the unfolding𝒢\(1\)\\mathcal\{G\}^\{\(1\)\}, and by the Eckart–Young theorem it introduces an error𝐞1\\mathbf\{e\}\_\{1\}satisfying

‖𝐞1‖F2=∑j\>r\(σj\(1\)\)2\.\\displaystyle\\\|\\mathbf\{e\}\_\{1\}\\\|\_\{F\}^\{2\}=\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(1\)\}\\bigr\)^\{2\}\.\(32\)After reshaping the retained factor from the first step, the second truncated SVD introduces an error𝐞2\\mathbf\{e\}\_\{2\}satisfying

‖𝐞2‖F2≤∑j\>r\(σj\(2\)\)2\.\\displaystyle\\\|\\mathbf\{e\}\_\{2\}\\\|\_\{F\}^\{2\}\\leq\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(2\)\}\\bigr\)^\{2\}\.\(33\)Since the final reconstruction error can be written as

‖G−G^r‖F≤‖e1‖F\+‖e2‖F,\\displaystyle\\\|G\-\\widehat\{G\}\_\{r\}\\\|\_\{F\}\\leq\\\|e\_\{1\}\\\|\_\{F\}\+\\\|e\_\{2\}\\\|\_\{F\},\(34\)we have

‖G−G^r‖F2\\displaystyle\\\|G\-\\widehat\{G\}\_\{r\}\\\|\_\{F\}^\{2\}=‖𝐞1\+𝐞2‖F2\\displaystyle=\\\|\\mathbf\{e\}\_\{1\}\+\\mathbf\{e\}\_\{2\}\\\|\_\{F\}^\{2\}≤2​‖𝐞1‖F2\+2​‖𝐞2‖F2\\displaystyle\\leq 2\\\|\\mathbf\{e\}\_\{1\}\\\|\_\{F\}^\{2\}\+2\\\|\\mathbf\{e\}\_\{2\}\\\|\_\{F\}^\{2\}≤2​∑j\>r\(σj\(1\)\)2\+2​∑j\>r\(σj\(2\)\)2\.\\displaystyle\\leq 2\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(1\)\}\\bigr\)^\{2\}\+2\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(2\)\}\\bigr\)^\{2\}\.\(35\)For each unfoldings∈\{1,2\}s\\in\\\{1,2\\\}, Assumption[V\.1](https://arxiv.org/html/2607.06651#S5.Thmassumption1)gives

∑j\>r\(σj\(s\)\)2\\displaystyle\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(s\)\}\\bigr\)^\{2\}=‖𝒢\(s\)‖F2​∑j\>rqj\(s\)\\displaystyle=\\\|\\mathcal\{G\}^\{\(s\)\}\\\|\_\{F\}^\{2\}\\sum\_\{j\>r\}q\_\{j\}^\{\(s\)\}≤As​‖𝒢\(s\)‖F2​e−αs​r\.\\displaystyle\\leq A\_\{s\}\\\|\\mathcal\{G\}^\{\(s\)\}\\\|\_\{F\}^\{2\}e^\{\-\\alpha\_\{s\}r\}\.\(36\)For anyα\>0\\alpha\>0andr≥1r\\geq 1,e−α​r≤\(e​α​r\)−1e^\{\-\\alpha r\}\\leq\(e\\alpha r\)^\{\-1\}, becausex​e−x≤e−1xe^\{\-x\}\\leq e^\{\-1\}for allx\>0x\>0\. Therefore,

∑j\>r\(σj\(s\)\)2≤Ase​αs​‖𝒢\(s\)‖F2r\.\\displaystyle\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(s\)\}\\bigr\)^\{2\}\\leq\\frac\{A\_\{s\}\}\{e\\alpha\_\{s\}\}\\frac\{\\\|\\mathcal\{G\}^\{\(s\)\}\\\|\_\{F\}^\{2\}\}\{r\}\.\(37\)LetReff\(s\)​\(G\):=eH\(s\)​\(G\)R\_\{\\mathrm\{eff\}\}^\{\(s\)\}\(G\):=e^\{H^\{\(s\)\}\(G\)\}\. SinceReff\(s\)​\(G\)≥1R\_\{\\mathrm\{eff\}\}^\{\(s\)\}\(G\)\\geq 1, the bound can be relaxed into the entropy\-weighted form

∑j\>r\(σj\(s\)\)2≤C~s,ℓ​eHk,ℓ\(s\)r​‖𝒢\(s\)‖F2,\\displaystyle\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(s\)\}\\bigr\)^\{2\}\\leq\\frac\{\\widetilde\{C\}\_\{s,\\ell\}e^\{H\_\{k,\\ell\}^\{\(s\)\}\}\}\{r\}\\\|\\mathcal\{G\}^\{\(s\)\}\\\|\_\{F\}^\{2\},\(38\)whereC~s,ℓ\>0\\widetilde\{C\}\_\{s,\\ell\}\>0absorbs the tail\-decay constants and the fixed tensorization of layerℓ\\ell\. Applying \([35](https://arxiv.org/html/2607.06651#S5.E35)\) to layerℓ\\ellof clientkkgives

‖Gk\(ℓ\)−G^k,r\(ℓ\)‖F2\\displaystyle\\\|G\_\{k\}^\{\(\\ell\)\}\-\\widehat\{G\}\_\{k,r\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}≤2​∑j\>r\(σj\(1\)\)2\+2​∑j\>r\(σj\(2\)\)2\\displaystyle\\leq 2\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(1\)\}\\bigr\)^\{2\}\+2\\sum\_\{j\>r\}\\bigl\(\\sigma\_\{j\}^\{\(2\)\}\\bigr\)^\{2\}≤C1,ℓ​eHk,ℓ\(1\)r​‖𝒢\(1\)‖F2\+C2,ℓ​eHk,ℓ\(2\)r​‖𝒢\(2\)‖F2,\\displaystyle\\leq\\frac\{C\_\{1,\\ell\}e^\{H\_\{k,\\ell\}^\{\(1\)\}\}\}\{r\}\\\|\\mathcal\{G\}^\{\(1\)\}\\\|\_\{F\}^\{2\}\+\\frac\{C\_\{2,\\ell\}e^\{H\_\{k,\\ell\}^\{\(2\)\}\}\}\{r\}\\\|\\mathcal\{G\}^\{\(2\)\}\\\|\_\{F\}^\{2\},\(39\)where the constantsC1,ℓC\_\{1,\\ell\}andC2,ℓC\_\{2,\\ell\}absorb the factor22\. Since‖𝒢\(1\)‖F=‖Gk\(ℓ\)‖F\\\|\\mathcal\{G\}^\{\(1\)\}\\\|\_\{F\}=\\\|G\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}and‖𝒢\(2\)‖F≤‖Gk\(ℓ\)‖F\\\|\\mathcal\{G\}^\{\(2\)\}\\\|\_\{F\}\\leq\\\|G\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}, dividing both sides by‖Gk\(ℓ\)‖F2\\\|G\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}yields

εk,ℓ2​\(r\)\\displaystyle\\varepsilon\_\{k,\\ell\}^\{2\}\(r\)=‖Gk\(ℓ\)−G^k,r\(ℓ\)‖F2‖Gk\(ℓ\)‖F2\\displaystyle=\\frac\{\\\|G\_\{k\}^\{\(\\ell\)\}\-\\widehat\{G\}\_\{k,r\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}\}\{\\\|G\_\{k\}^\{\(\\ell\)\}\\\|\_\{F\}^\{2\}\}≤C1,ℓ​eHk,ℓ\(1\)\+C2,ℓ​eHk,ℓ\(2\)r=Ψk,ℓr\.\\displaystyle\\leq\\frac\{C\_\{1,\\ell\}e^\{H\_\{k,\\ell\}^\{\(1\)\}\}\+C\_\{2,\\ell\}e^\{H\_\{k,\\ell\}^\{\(2\)\}\}\}\{r\}=\\frac\{\\Psi\_\{k,\\ell\}\}\{r\}\.\(40\)This proves the stated bound\. ∎

The coefficientΨk,ℓ\\Psi\_\{k,\\ell\}provides a conservative entropy\-weighted measure of layer\-wise spectral complexity\. LargerΨk,ℓ\\Psi\_\{k,\\ell\}corresponds to a larger upper bound on the compression error at the same rank, motivating larger rank allocation for spectrally more complex updates\. This leads to the surrogate allocation problem

min\{rk\(ℓ\)\>0\}∑ℓ∈ℒkΨk,ℓrk\(ℓ\)s\.t\.∑ℓ∈ℒkϕℓ\(rk\(ℓ\)\)≤Ck\.\\displaystyle\\min\_\{\\\{r\_\{k\}^\{\(\\ell\)\}\>0\\\}\}\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\frac\{\\Psi\_\{k,\\ell\}\}\{r\_\{k\}^\{\(\\ell\)\}\}\\quad\\mathrm\{s\.t\.\}\\quad\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\_\{\\ell\}\\\!\\left\(r\_\{k\}^\{\(\\ell\)\}\\right\)\\leq C\_\{k\}\.\(41\)This surrogate is optimized in Section[V\-E](https://arxiv.org/html/2607.06651#S5.SS5)\. In the implemented algorithm, the truncated layer\-wise entropyH^k\(ℓ\)\\widehat\{H\}\_\{k\}^\{\(\\ell\)\}serves as a scalar proxy for the spectral\-complexity weightΨk,ℓ\\Psi\_\{k,\\ell\}\. AlthoughΨk,ℓ\\Psi\_\{k,\\ell\}is defined through the two sequential MPS unfoldings, both quantities reflect spectral diffuseness:Ψk,ℓ\\Psi\_\{k,\\ell\}increases when either unfolding entropy is large, whereasH^k\(ℓ\)\\widehat\{H\}\_\{k\}^\{\(\\ell\)\}increases when the singular\-value energy of the original matrix is diffuse\. Since reshaping preserves the Frobenius norm but does not generally preserve the singular spectrum, no exact equivalence betweenH^k\(ℓ\)\\widehat\{H\}\_\{k\}^\{\(\\ell\)\}andHk,ℓ\(1\)H^\{\(1\)\}\_\{k,\\ell\}is assumed\. Instead,H^k\(ℓ\)\\widehat\{H\}\_\{k\}^\{\(\\ell\)\}is used as a low\-cost proxy for the overall layer\-wise spectral complexity\. This approximation is most appropriate when the first truncation captures the dominant spectral mass, thereby reducing the practical influence of the second truncation error, consistent with the compressible\-spectrum regime described by Assumption[V\.1](https://arxiv.org/html/2607.06651#S5.Thmassumption1)\.

### V\-CSpectral Entropy and Rank Complexity

The preceding bound uses entropy to measure the spectral complexity of compressed updates\. We now define the corresponding matrix\-level entropy and the minimum rank needed to achieve a target reconstruction error\.

###### Definition V\.1\(Energy entropy and effective rank\)\.

LetG∈ℝm×nG\\in\\mathbb\{R\}^\{m\\times n\}have singular valuesσ1≥⋯≥σp\>0\\sigma\_\{1\}\\geq\\cdots\\geq\\sigma\_\{p\}\>0, wherep=min⁡\(m,n\)p=\\min\(m,n\)\. Define

qi=σi2‖G‖F2,H​\(G\)=−∑i=1pqi​log⁡qi,reff​\(G\)=eH​\(G\)\.\\displaystyle q\_\{i\}=\\frac\{\\sigma\_\{i\}^\{2\}\}\{\\\|G\\\|\_\{F\}^\{2\}\},\\quad H\(G\)=\-\\sum\_\{i=1\}^\{p\}q\_\{i\}\\log q\_\{i\},\\quad r\_\{\\mathrm\{eff\}\}\(G\)=e^\{H\(G\)\}\.\(42\)Forε∈\(0,1\)\\varepsilon\\in\(0,1\), define the minimumε\\varepsilon\-accurate rank as

r∗​\(ε;G\)=min⁡\{r:∑i\>rqi≤ε2\}\.\\displaystyle r^\{\*\}\(\\varepsilon;G\)=\\min\\left\\\{r:\\sum\_\{i\>r\}q\_\{i\}\\leq\\varepsilon^\{2\}\\right\\\}\.\(43\)

MESH\-FL does not compute the full spectrum during training\. Each client estimates entropy from the leading singular values only\. The next remark states that when this truncated estimator remains close to the full entropy and preserves entropy ordering\.

The following theorem provides the main ordering principle behind entropy\-guided compression\. Under the natural majorization order, a more diffuse singular\-value energy distribution has larger entropy and requires a weakly larger rank to achieve the same approximation target\.

###### Theorem V\.1\(Entropy\-rank monotonicity\)\.

Letq=\(q1,…,qp\)q=\(q\_\{1\},\\ldots,q\_\{p\}\)andq′=\(q1′,…,qp′\)q^\{\\prime\}=\(q\_\{1\}^\{\\prime\},\\ldots,q\_\{p\}^\{\\prime\}\)be two energy distributions sorted in nonincreasing order\. Ifqqmajorizesq′q^\{\\prime\}, then

H​\(q\)≤H​\(q′\),r∗​\(ε;q\)≤r∗​\(ε;q′\),∀ε∈\(0,1\)\.\\displaystyle H\(q\)\\leq H\(q^\{\\prime\}\),\\quad r^\{\*\}\(\\varepsilon;q\)\\leq r^\{\*\}\(\\varepsilon;q^\{\\prime\}\),\\quad\\forall\\,\\varepsilon\\in\(0,1\)\.\(47\)

###### Proof\.

Shannon entropy is Schur\-concave, soq≻q′q\\succ q^\{\\prime\}impliesH​\(q\)≤H​\(q′\)H\(q\)\\leq H\(q^\{\\prime\}\)\. Majorization also gives

∑i=1rqi≥∑i=1rqi′,∀r,\\displaystyle\\sum\_\{i=1\}^\{r\}q\_\{i\}\\geq\\sum\_\{i=1\}^\{r\}q\_\{i\}^\{\\prime\},\\quad\\forall\\,r,\(48\)which is equivalent to

∑i\>rqi≤∑i\>rqi′,∀r\.\\displaystyle\\sum\_\{i\>r\}q\_\{i\}\\leq\\sum\_\{i\>r\}q\_\{i\}^\{\\prime\},\\quad\\forall\\,r\.\(49\)Therefore, the smallest rank achieving tail energy at mostε2\\varepsilon^\{2\}satisfiesr∗​\(ε;q\)≤r∗​\(ε;q′\)r^\{\*\}\(\\varepsilon;q\)\\leq r^\{\*\}\(\\varepsilon;q^\{\\prime\}\)\. ∎

Proposition[V\.1](https://arxiv.org/html/2607.06651#S5.Thmproposition1), Remark[1](https://arxiv.org/html/2607.06651#Thmremark1), and Theorem[V\.1](https://arxiv.org/html/2607.06651#S5.Thmtheorem1)together provide the spectral foundation of MESH\-FL\. Proposition[V\.1](https://arxiv.org/html/2607.06651#S5.Thmproposition1)links MPS rank to reconstruction error through an entropy\-weighted complexity coefficient\. Remark[1](https://arxiv.org/html/2607.06651#Thmremark1)justifies the use of truncated entropy when the omitted tail energy is small\. Theorem[V\.1](https://arxiv.org/html/2607.06651#S5.Thmtheorem1)shows that entropy ordering is consistent with rank\-complexity ordering under majorization\.

### V\-DConvergence of MESH\-FL

The previous results characterize the approximation error introduced by MPS compression\. We now show how this error enters the optimization dynamics of MESH\-FL\.

###### Theorem V\.2\(Convergence with compression error\)\.

Under the conditions in Section[V\-A](https://arxiv.org/html/2607.06651#S5.SS1), assume full participation andη≤1/\(4​L​E\)\\eta\\leq 1/\(4LE\)\. LetG^kt=Gkt\+𝐞kt\\widehat\{G\}\_\{k\}^\{t\}=G\_\{k\}^\{t\}\+\\mathbf\{e\}\_\{k\}^\{t\}be the reconstructed update received from clientkkat roundtt, where𝐞kt\\mathbf\{e\}\_\{k\}^\{t\}is the compression error\. Then

1T​∑t=0T−1𝔼​‖∇F​\(𝐰t\)‖2≤4​\(F​\(𝐰0\)−F∗\)η​E​T\+4​L​η​E​σ2\\displaystyle\\frac\{1\}\{T\}\\sum\_\{t=0\}^\{T\-1\}\\mathbb\{E\}\\\|\\nabla F\(\\mathbf\{w\}^\{t\}\)\\\|^\{2\}\\leq\\frac\{4\(F\(\\mathbf\{w\}^\{0\}\)\-F^\{\*\}\)\}\{\\eta ET\}\+4L\\eta E\\sigma^\{2\}\+8​L​η​E​δ2\+5η2​E2​T​∑t=0T−1∑k=1Knkn​𝔼​‖𝐞kt‖2\.\\displaystyle\+8L\\eta E\\delta^\{2\}\+\\frac\{5\}\{\\eta^\{2\}E^\{2\}T\}\\sum\_\{t=0\}^\{T\-1\}\\sum\_\{k=1\}^\{K\}\\frac\{n\_\{k\}\}\{n\}\\mathbb\{E\}\\\|\\mathbf\{e\}\_\{k\}^\{t\}\\\|^\{2\}\.\(50\)If

‖𝐞kt‖2≤2​∑ℓ∈ℒkεk,ℓ2​\(rk\(ℓ\)\)​‖Gk\(ℓ\),t‖F2,\\displaystyle\\\|\\mathbf\{e\}\_\{k\}^\{t\}\\\|^\{2\}\\leq 2\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\varepsilon\_\{k,\\ell\}^\{2\}\\\!\\left\(r\_\{k\}^\{\(\\ell\)\}\\right\)\\\|G\_\{k\}^\{\(\\ell\),t\}\\\|\_\{F\}^\{2\},\(51\)then the final term in \([50](https://arxiv.org/html/2607.06651#S5.E50)\) is explicitly controlled by the layer\-wise MPS reconstruction errors\.

###### Proof\.

Letpk=nk/np\_\{k\}=n\_\{k\}/n,G¯t=∑kpk​Gkt\\overline\{G\}^\{t\}=\\sum\_\{k\}p\_\{k\}G\_\{k\}^\{t\}, and𝐞¯t=∑kpk​𝐞kt\\overline\{\\mathbf\{e\}\}^\{t\}=\\sum\_\{k\}p\_\{k\}\\mathbf\{e\}\_\{k\}^\{t\}, so the MESH\-FL update satisfies𝐰t\+1−𝐰t=−G¯t−𝐞¯t\\mathbf\{w\}^\{t\+1\}\-\\mathbf\{w\}^\{t\}=\-\\overline\{G\}^\{t\}\-\\overline\{\\mathbf\{e\}\}^\{t\}\. ByLL\-smoothness and the local\-delta mean condition \([26](https://arxiv.org/html/2607.06651#S5.E26)\),

𝔼​\[F​\(𝐰t\+1\)∣𝐰t\]\\displaystyle\\mathbb\{E\}\\\!\\left\[F\(\\mathbf\{w\}^\{t\+1\}\)\\mid\\mathbf\{w\}^\{t\}\\right\]≤F​\(𝐰t\)−η​E​‖∇F​\(𝐰t\)‖2\\displaystyle\\leq F\(\\mathbf\{w\}^\{t\}\)\-\\eta E\\\|\\nabla F\(\\mathbf\{w\}^\{t\}\)\\\|^\{2\}−⟨∇F​\(𝐰t\),𝐞¯t⟩\+L2​‖G¯t\+𝐞¯t‖2\.\\displaystyle\-\\left\\langle\\nabla F\(\\mathbf\{w\}^\{t\}\),\\overline\{\\mathbf\{e\}\}^\{t\}\\right\\rangle\+\\frac\{L\}\{2\}\\\|\\overline\{G\}^\{t\}\+\\overline\{\\mathbf\{e\}\}^\{t\}\\\|^\{2\}\.\(52\)Applying Young’s inequality to the inner product and Jensen’s inequality to‖𝐞¯t‖2\\\|\\overline\{\\mathbf\{e\}\}^\{t\}\\\|^\{2\},

−⟨∇F​\(𝐰t\),𝐞¯t⟩≤η​E4​‖∇F​\(𝐰t\)‖2\+1η​E​∑kpk​‖𝐞kt‖2\.\\displaystyle\-\\left\\langle\\nabla F\(\\mathbf\{w\}^\{t\}\),\\overline\{\\mathbf\{e\}\}^\{t\}\\right\\rangle\\leq\\frac\{\\eta E\}\{4\}\\\|\\nabla F\(\\mathbf\{w\}^\{t\}\)\\\|^\{2\}\+\\frac\{1\}\{\\eta E\}\\sum\_\{k\}p\_\{k\}\\\|\\mathbf\{e\}\_\{k\}^\{t\}\\\|^\{2\}\.\(53\)For the quadratic term, writingGkt=η​E​∇Fk​\(𝐰t\)\+𝝃ktG\_\{k\}^\{t\}=\\eta E\\nabla F\_\{k\}\(\\mathbf\{w\}^\{t\}\)\+\\boldsymbol\{\\xi\}\_\{k\}^\{t\}with𝔼​\[𝝃kt\]=0\\mathbb\{E\}\[\\boldsymbol\{\\xi\}\_\{k\}^\{t\}\]=0and𝔼​\[‖𝝃kt‖2\]≤η2​E2​σ2\\mathbb\{E\}\[\\\|\\boldsymbol\{\\xi\}\_\{k\}^\{t\}\\\|^\{2\}\]\\leq\\eta^\{2\}E^\{2\}\\sigma^\{2\}, and using the gradient\-dissimilarity bound \([28](https://arxiv.org/html/2607.06651#S5.E28)\),

𝔼​\[‖G¯t‖2∣𝐰t\]≤∑kpk​𝔼​\[‖Gkt‖2∣𝐰t\]\\displaystyle\\mathbb\{E\}\\\!\\left\[\\\|\\overline\{G\}^\{t\}\\\|^\{2\}\\mid\\mathbf\{w\}^\{t\}\\right\]\\leq\\sum\_\{k\}p\_\{k\}\\mathbb\{E\}\\\!\\left\[\\\|G\_\{k\}^\{t\}\\\|^\{2\}\\mid\\mathbf\{w\}^\{t\}\\right\]≤2​η2​E2​‖∇F​\(𝐰t\)‖2\+2​η2​E2​δ2\+η2​E2​σ2\.\\displaystyle\\leq 2\\eta^\{2\}E^\{2\}\\\|\\nabla F\(\\mathbf\{w\}^\{t\}\)\\\|^\{2\}\+2\\eta^\{2\}E^\{2\}\\delta^\{2\}\+\\eta^\{2\}E^\{2\}\\sigma^\{2\}\.\(54\)Substituting \([53](https://arxiv.org/html/2607.06651#S5.E53)\)–\([54](https://arxiv.org/html/2607.06651#S5.E54)\) into \([52](https://arxiv.org/html/2607.06651#S5.E52)\) and applyingη≤1/\(4​L​E\)\\eta\\leq 1/\(4LE\)\(so that2​L​η2​E2≤η​E/22L\\eta^\{2\}E^\{2\}\\leq\\eta E/2andL≤1/\(4​η​E\)L\\leq 1/\(4\\eta E\)\) gives

𝔼​\[F​\(𝐰t\+1\)∣𝐰t\]≤F​\(𝐰t\)−η​E4​‖∇F​\(𝐰t\)‖2\\displaystyle\\mathbb\{E\}\\\!\\left\[F\(\\mathbf\{w\}^\{t\+1\}\)\\mid\\mathbf\{w\}^\{t\}\\right\]\\leq F\(\\mathbf\{w\}^\{t\}\)\-\\frac\{\\eta E\}\{4\}\\\|\\nabla F\(\\mathbf\{w\}^\{t\}\)\\\|^\{2\}\+L​η2​E2​σ2\+2​L​η2​E2​δ2\+54​η​E​∑kpk​𝔼​\[‖𝐞kt‖2∣𝐰t\]\.\\displaystyle\+L\\eta^\{2\}E^\{2\}\\sigma^\{2\}\+2L\\eta^\{2\}E^\{2\}\\delta^\{2\}\+\\frac\{5\}\{4\\eta E\}\\sum\_\{k\}p\_\{k\}\\mathbb\{E\}\\\!\\left\[\\\|\\mathbf\{e\}\_\{k\}^\{t\}\\\|^\{2\}\\mid\\mathbf\{w\}^\{t\}\\right\]\.\(55\)Summing overt=0,…,T−1t=0,\\ldots,T\-1, taking total expectation, usingF​\(𝐰T\)≥F∗F\(\\mathbf\{w\}^\{T\}\)\\geq F^\{\*\}, and dividing byη​E​T/4\\eta ET/4yields

1T​∑t=0T−1𝔼​‖∇F​\(𝐰t\)‖2≤4​\(F​\(𝐰0\)−F∗\)η​E​T\+4​L​η​E​σ2\\displaystyle\\frac\{1\}\{T\}\\sum\_\{t=0\}^\{T\-1\}\\mathbb\{E\}\\\|\\nabla F\(\\mathbf\{w\}^\{t\}\)\\\|^\{2\}\\leq\\frac\{4\(F\(\\mathbf\{w\}^\{0\}\)\-F^\{\*\}\)\}\{\\eta ET\}\+4L\\eta E\\sigma^\{2\}\+8​L​η​E​δ2\+5η2​E2​T​∑t=0T−1∑k=1Knkn​𝔼​‖𝐞kt‖2\.\\displaystyle\\quad\+8L\\eta E\\delta^\{2\}\+\\frac\{5\}\{\\eta^\{2\}E^\{2\}T\}\\sum\_\{t=0\}^\{T\-1\}\\sum\_\{k=1\}^\{K\}\\frac\{n\_\{k\}\}\{n\}\\mathbb\{E\}\\\|\\mathbf\{e\}\_\{k\}^\{t\}\\\|^\{2\}\.\(56\)Finally, if‖𝐞kt‖2≤2​∑ℓ∈ℒkεk,ℓ2​\(rk\(ℓ\)\)​‖Gk\(ℓ\),t‖F2\\\|\\mathbf\{e\}\_\{k\}^\{t\}\\\|^\{2\}\\leq 2\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\varepsilon\_\{k,\\ell\}^\{2\}\(r\_\{k\}^\{\(\\ell\)\}\)\\\|G\_\{k\}^\{\(\\ell\),t\}\\\|\_\{F\}^\{2\}, the last term is explicitly controlled by the layer\-wise MPS reconstruction errors\. ∎

Theorem[V\.2](https://arxiv.org/html/2607.06651#S5.Thmtheorem2)shows that MESH\-FL preserves the standard nonconvex FL convergence pattern, with the additional term caused by compression\. Thus, rank allocation should reduce the average reconstruction\-error term while satisfying each client’s payload budget\.

### V\-EOptimality of Entropy\-Guided Allocation

The convergence bound identifies compression error as the additional optimization penalty\. We therefore choose ranks by minimizing the entropy\-weighted surrogate induced by Proposition[V\.1](https://arxiv.org/html/2607.06651#S5.Thmproposition1)under the per\-client payload constraint\.

###### Theorem V\.3\(Optimal monotone allocation\)\.

Fix clientkkand letψk\(ℓ\):=Ψk,ℓ\\psi\_\{k\}^\{\(\\ell\)\}:=\\Psi\_\{k,\\ell\}\. Consider the surrogate problem in \([41](https://arxiv.org/html/2607.06651#S5.E41)\) with payload model

ϕℓ​\(r\)=aℓ​r\+bℓ​r2,aℓ,bℓ≥0\.\\displaystyle\\phi\_\{\\ell\}\(r\)=a\_\{\\ell\}r\+b\_\{\\ell\}r^\{2\},\\quad a\_\{\\ell\},b\_\{\\ell\}\\geq 0\.\(57\)The problem is strictly convex and admits a unique optimizer\. Moreover, the optimal rankrk\(ℓ\)⁣∗r\_\{k\}^\{\(\\ell\)\*\}is strictly increasing inψk\(ℓ\)\\psi\_\{k\}^\{\(\\ell\)\}\. Under the linearized payload modelbℓ=0b\_\{\\ell\}=0, the optimizer is

rk\(ℓ\)⁣∗=Ck​ψk\(ℓ\)/aℓ∑ℓ′∈ℒkaℓ′​ψk\(ℓ′\)\.\\displaystyle r\_\{k\}^\{\(\\ell\)\*\}=\\frac\{C\_\{k\}\\sqrt\{\\psi\_\{k\}^\{\(\\ell\)\}/a\_\{\\ell\}\}\}\{\\sum\_\{\\ell^\{\\prime\}\\in\\mathcal\{L\}\_\{k\}\}\\sqrt\{a\_\{\\ell^\{\\prime\}\}\\psi\_\{k\}^\{\(\\ell^\{\\prime\}\)\}\}\}\.\(58\)Under the exact quadratic payload model, the optimizer satisfies

ψk\(ℓ\)\(rk\(ℓ\)⁣∗\)2=λk​\(aℓ\+2​bℓ​rk\(ℓ\)⁣∗\),\\displaystyle\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{\(r\_\{k\}^\{\(\\ell\)\*\}\)^\{2\}\}=\\lambda\_\{k\}\\left\(a\_\{\\ell\}\+2b\_\{\\ell\}r\_\{k\}^\{\(\\ell\)\*\}\\right\),\(59\)whereλk\>0\\lambda\_\{k\}\>0enforces the payload constraint with equality\.

###### Proof\.

The objective

∑ℓ∈ℒkψk\(ℓ\)rk\(ℓ\)\\displaystyle\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{r\_\{k\}^\{\(\\ell\)\}\}\(60\)is strictly convex onrk\(ℓ\)\>0r\_\{k\}^\{\(\\ell\)\}\>0, because each termψk\(ℓ\)/rk\(ℓ\)\\psi\_\{k\}^\{\(\\ell\)\}/r\_\{k\}^\{\(\\ell\)\}has second derivative2​ψk\(ℓ\)/\(rk\(ℓ\)\)3\>02\\psi\_\{k\}^\{\(\\ell\)\}/\(r\_\{k\}^\{\(\\ell\)\}\)^\{3\}\>0\. The feasible set

\{\{rk\(ℓ\)\}:∑ℓ∈ℒkϕℓ​\(rk\(ℓ\)\)≤Ck,rk\(ℓ\)\>0\}\\displaystyle\\left\\\{\\\{r\_\{k\}^\{\(\\ell\)\}\\\}:\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\_\{\\ell\}\\\!\\left\(r\_\{k\}^\{\(\\ell\)\}\\right\)\\leq C\_\{k\},\\;r\_\{k\}^\{\(\\ell\)\}\>0\\right\\\}\(61\)is convex because eachϕℓ​\(r\)=aℓ​r\+bℓ​r2\\phi\_\{\\ell\}\(r\)=a\_\{\\ell\}r\+b\_\{\\ell\}r^\{2\}is convex foraℓ,bℓ≥0a\_\{\\ell\},b\_\{\\ell\}\\geq 0\. Hence, the surrogate allocation problem is strictly convex and admits a unique optimizer\.

The Lagrangian is

𝒥=∑ℓ∈ℒkψk\(ℓ\)rk\(ℓ\)\+λk​\(∑ℓ∈ℒkϕℓ​\(rk\(ℓ\)\)−Ck\),\\displaystyle\\mathcal\{J\}=\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{r\_\{k\}^\{\(\\ell\)\}\}\+\\lambda\_\{k\}\\left\(\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\phi\_\{\\ell\}\\\!\\left\(r\_\{k\}^\{\(\\ell\)\}\\right\)\-C\_\{k\}\\right\),\(62\)withλk≥0\\lambda\_\{k\}\\geq 0\. Since the objective is strictly decreasing in eachrk\(ℓ\)r\_\{k\}^\{\(\\ell\)\}, the budget constraint is active at the optimum, soλk\>0\\lambda\_\{k\}\>0\. The KKT stationarity condition for each layerℓ\\ellis

−ψk\(ℓ\)\(rk\(ℓ\)\)2\+λk​\(aℓ\+2​bℓ​rk\(ℓ\)\)=0\.\\displaystyle\-\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{\(r\_\{k\}^\{\(\\ell\)\}\)^\{2\}\}\+\\lambda\_\{k\}\\left\(a\_\{\\ell\}\+2b\_\{\\ell\}r\_\{k\}^\{\(\\ell\)\}\\right\)=0\.\(63\)For fixedλk\>0\\lambda\_\{k\}\>0, define

fℓ​\(r\)=−ψk\(ℓ\)r2\+λk​\(aℓ\+2​bℓ​r\)\.\\displaystyle f\_\{\\ell\}\(r\)=\-\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{r^\{2\}\}\+\\lambda\_\{k\}\(a\_\{\\ell\}\+2b\_\{\\ell\}r\)\.\(64\)The functionfℓf\_\{\\ell\}is strictly increasing and continuous onr\>0r\>0, withfℓ​\(r\)→−∞f\_\{\\ell\}\(r\)\\to\-\\inftyasr→0\+r\\to 0^\{\+\}\. Ifbℓ\>0b\_\{\\ell\}\>0, thenfℓ​\(r\)→\+∞f\_\{\\ell\}\(r\)\\to\+\\inftyasr→∞r\\to\\infty; ifbℓ=0b\_\{\\ell\}=0, thenfℓ​\(r\)→λk​aℓ\>0f\_\{\\ell\}\(r\)\\to\\lambda\_\{k\}a\_\{\\ell\}\>0asr→∞r\\to\\infty\. Therefore, there exists a unique positive rootrk\(ℓ\)⁣∗r\_\{k\}^\{\(\\ell\)\*\}satisfying \([63](https://arxiv.org/html/2607.06651#S5.E63)\)\. For fixedλk\\lambda\_\{k\}, increasingψk\(ℓ\)\\psi\_\{k\}^\{\(\\ell\)\}shiftsfℓf\_\{\\ell\}downward, so the unique root must increase to restore equality\. Hence,rk\(ℓ\)⁣∗r\_\{k\}^\{\(\\ell\)\*\}is strictly increasing inψk\(ℓ\)\\psi\_\{k\}^\{\(\\ell\)\}\.

Whenbℓ=0b\_\{\\ell\}=0, \([63](https://arxiv.org/html/2607.06651#S5.E63)\) reduces to

−ψk\(ℓ\)\(rk\(ℓ\)\)2\+λk​aℓ=0,\\displaystyle\-\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{\(r\_\{k\}^\{\(\\ell\)\}\)^\{2\}\}\+\\lambda\_\{k\}a\_\{\\ell\}=0,\(65\)which gives

rk\(ℓ\)⁣∗=ψk\(ℓ\)λk​aℓ\.\\displaystyle r\_\{k\}^\{\(\\ell\)\*\}=\\sqrt\{\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{\\lambda\_\{k\}a\_\{\\ell\}\}\}\.\(66\)Substituting \([66](https://arxiv.org/html/2607.06651#S5.E66)\) into the active budget constraint gives

∑ℓ∈ℒkaℓ​rk\(ℓ\)⁣∗=1λk​∑ℓ∈ℒkaℓ​ψk\(ℓ\)=Ck\.\\displaystyle\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}a\_\{\\ell\}r\_\{k\}^\{\(\\ell\)\*\}=\\frac\{1\}\{\\sqrt\{\\lambda\_\{k\}\}\}\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\sqrt\{a\_\{\\ell\}\\psi\_\{k\}^\{\(\\ell\)\}\}=C\_\{k\}\.\(67\)Therefore,

1λk=Ck∑ℓ∈ℒkaℓ​ψk\(ℓ\)\.\\displaystyle\\frac\{1\}\{\\sqrt\{\\lambda\_\{k\}\}\}=\\frac\{C\_\{k\}\}\{\\sum\_\{\\ell\\in\\mathcal\{L\}\_\{k\}\}\\sqrt\{a\_\{\\ell\}\\psi\_\{k\}^\{\(\\ell\)\}\}\}\.\(68\)Substituting \([68](https://arxiv.org/html/2607.06651#S5.E68)\) into \([66](https://arxiv.org/html/2607.06651#S5.E66)\) gives

rk\(ℓ\)⁣∗=Ck​ψk\(ℓ\)/aℓ∑ℓ′∈ℒkaℓ′​ψk\(ℓ′\)\.\\displaystyle r\_\{k\}^\{\(\\ell\)\*\}=\\frac\{C\_\{k\}\\sqrt\{\\psi\_\{k\}^\{\(\\ell\)\}/a\_\{\\ell\}\}\}\{\\sum\_\{\\ell^\{\\prime\}\\in\\mathcal\{L\}\_\{k\}\}\\sqrt\{a\_\{\\ell^\{\\prime\}\}\\psi\_\{k\}^\{\(\\ell^\{\\prime\}\)\}\}\}\.\(69\)Under the quadratic payload modelbℓ\>0b\_\{\\ell\}\>0, the optimizer is characterized by the KKT condition \([63](https://arxiv.org/html/2607.06651#S5.E63)\), equivalently

ψk\(ℓ\)\(rk\(ℓ\)⁣∗\)2=λk​\(aℓ\+2​bℓ​rk\(ℓ\)⁣∗\),\\displaystyle\\frac\{\\psi\_\{k\}^\{\(\\ell\)\}\}\{\(r\_\{k\}^\{\(\\ell\)\*\}\)^\{2\}\}=\\lambda\_\{k\}\\left\(a\_\{\\ell\}\+2b\_\{\\ell\}r\_\{k\}^\{\(\\ell\)\*\}\\right\),\(70\)whereλk\>0\\lambda\_\{k\}\>0is chosen so that the payload constraint holds with equality\. ∎

Theorem[V\.3](https://arxiv.org/html/2607.06651#S5.Thmtheorem3)establishes the monotone allocation structure used by MESH\-FL: for fixed payload coefficients, layers with larger spectral\-complexity weight receive larger rank under the client budget\. The practical rulerk\(ℓ\)∝eH^k\(ℓ\)/2r\_\{k\}^\{\(\\ell\)\}\\propto e^\{\\widehat\{H\}\_\{k\}^\{\(\\ell\)\}/2\}is obtained by usingeH^k\(ℓ\)e^\{\\widehat\{H\}\_\{k\}^\{\(\\ell\)\}\}as a low\-cost online proxy for the spectral\-complexity ordering in \([58](https://arxiv.org/html/2607.06651#S5.E58)\), followed by budget scaling via bisection and integer feasibility correction\.

## VIExperimental Results

### VI\-AExperimental Setup

The experiments were conducted on a 15\-node heterogeneous cluster consisting of 5 Raspberry Pi 4 Model B \(quad\-core Cortex\-A72 at 1\.8 GHz, 8 GB LPDDR4, 64 GB storage, batch size 16\) and 10 Raspberry Pi 5 \(quad\-core Cortex\-A76 at 2\.4 GHz, 16 GB LPDDR4X, 128 GB storage, batch size 32\), all connected to a central server via a local Ethernet switch\. Pi 4 nodes were assigned higher compression targets than Pi 5 nodes to reflect their tighter uplink capacity\.

We used AV\-MNIST, a dataset combining MNIST images\[[13](https://arxiv.org/html/2607.06651#bib.bib18)\]with spoken\-digit recordings from FSDD\[[9](https://arxiv.org/html/2607.06651#bib.bib19)\]\. Each sample consists of a28×2828\\times 28grayscale image and a 1,000\-dimensional MFCC feature vector \(20 coefficients, 8 kHz, hop length 160\)\. The dataset includes 60,000 training and 10,000 test pairs across 10 classes\. The dataset is partitioned non\-IID using a Dirichlet distribution with concentration parameterα=0\.1\\alpha=0\.1, following the protocol introduced in\[[8](https://arxiv.org/html/2607.06651#bib.bib33)\]for controlling label\-distribution skew in FL\.

Each client was randomly assigned a modality profile \(image\-only, audio\-only, or multimodal\), resulting in heterogeneous participation patterns across training rounds\. Image\-only clients updated only the image encoder, audio\-only clients updated the audio encoder, and multimodal clients updated both encoders and the shared fusion module\.

![Refer to caption](https://arxiv.org/html/2607.06651v1/x3.png)Figure 1:Global test accuracy and mean train loss \(±\\pmstd across clients\) over 50 rounds on AV\-MNIST \(α=0\.1\\alpha=0\.1\)\.The global model consisted of a CNN image encoder, an MLP audio encoder, and a shared two\-layer fusion module producing a 10\-class output\. The experiments ran forT=50T=50rounds with full participation andE=5E=5local SGD steps \(lr = 0\.01\)\. Three compression levels were evaluated: light compression targeting22–10×10\\timesfor Pi 5 and55–10×10\\timesfor Pi 4; moderate compression targeting3030–50×50\\timesfor Pi 5 and4040–60×60\\timesfor Pi 4; and heavy compression targeting4040–60×60\\timesfor Pi 5 and6060–80×80\\timesfor Pi 4\. These levels produced fleet\-averaged compression ratios of9\.36×9\.36\\times,40\.52×40\.52\\times, and56\.82×56\.82\\times, respectively\. The entropy truncation parameter was fixed atq=10q=10\. Baselines included standard FedAvg\[[16](https://arxiv.org/html/2607.06651#bib.bib17)\]without compression, TopK sparsification \(s=1%s=1\\%\) with error feedback\[[15](https://arxiv.org/html/2607.06651#bib.bib3)\], QSGD with 4\-bit quantization\[[2](https://arxiv.org/html/2607.06651#bib.bib26)\], and PowerSGD with rank 4\[[25](https://arxiv.org/html/2607.06651#bib.bib27)\]\.

### VI\-BCommunication Efficiency and Learning Performance

Fig\.[1](https://arxiv.org/html/2607.06651#S6.F1)demonstrates that all three MESH\-FL compression levels converge faster and achieve higher final accuracy than the uncompressed FedAvg baseline\. Specifically, light compression reaches 97\.73%, surpassing FedAvg \(95\.72%\) by 2\.01%, while moderate and heavy compression reach 96\.47% and 96\.20%, respectively\. This result stems from the interaction between entropy\-guided MPS compression and the non\-IID data distribution \(α=0\.1\\alpha=0\.1\)\. By projecting each layer\-wise update onto its dominant spectral subspace, MESH\-FL filters high\-frequency, client\-specific gradient components amplified in non\-IID settings, acting as implicit gradient regularization\. The loss curves confirm this: FedAvg shows larger cross\-client variance throughout training, while compressed variants maintain tighter agreement from round 10 onward\.

![Refer to caption](https://arxiv.org/html/2607.06651v1/x4.png)Figure 2:Final global accuracy and mean compression ratio\.The accuracy\-compression tradeoff is illustrated in Fig\.[2](https://arxiv.org/html/2607.06651#S6.F2)\. Notably, the uncompressed baseline \(1\.0×1\.0\\times\) provides the lowest accuracy \(95\.72%\), while every MESH\-FL compression level improves both accuracy and communication efficiency simultaneously\. Light compression achieves the best accuracy \(97\.73%\) at9\.36×9\.36\\timescompression, while moderate and heavy compression trade a modest accuracy reduction \(1\.26 and 1\.53% below light\) for substantially higher compression ratios \(40\.52×40\.52\\timesand56\.82×56\.82\\times\)\.

![Refer to caption](https://arxiv.org/html/2607.06651v1/x5.png)Figure 3:Communication rounds required to reach 80%, 90%, and 95% accuracy thresholds under each compression level\.Fig\.[3](https://arxiv.org/html/2607.06651#S6.F3)shows that all compressed variants reach accuracy thresholds faster than FedAvg\. At 95% accuracy, FedAvg requires 47 rounds versus 31, 40, and 40 for light, moderate, and heavy compression\. The 16\-round reduction for light compression, combined with its9\.36×9\.36\\timeslower per\-round payload, translates into a dramatic reduction in total bytes transmitted to reach convergence, as shown in Fig\.[4](https://arxiv.org/html/2607.06651#S6.F4)\(b\)\.

![Refer to caption](https://arxiv.org/html/2607.06651v1/fig_savings.png)Figure 4:Per\-client mean transmitted data per round and total cumulative savings over 50 rounds, broken down by device type\.Fig\.[4](https://arxiv.org/html/2607.06651#S6.F4)illustrates per\-client payload and cumulative savings across compression levels\. Multimodal clients transmit more data than unimodal clients due to their larger active layer sets\. Moderate and heavy compression reduce per\-client payload to 1\.25 MB and 0\.88 MB per round, respectively, compared with 48\.85 MB under no compression\. Heavy compression saves approximately 35,900 MB \(≈36\\approx 36GB\) relative to the uncompressed baseline across all 15 clients over 50 rounds, with Pi 5 nodes contributing the larger share due to their higher modality participation rate\.

### VI\-CComparison with Baselines

TABLE I:Accuracy and compression on AV\-MNIST \(α=0\.1\\alpha=0\.1\)\.MethodAcc\. \(%\)Ratio \(×\\times\)Rnd@95%FedAvg\[[16](https://arxiv.org/html/2607.06651#bib.bib17)\]95\.721\.0047TopK \(s=1%s\{=\}1\\%\)\[[15](https://arxiv.org/html/2607.06651#bib.bib3)\]93\.410\.248QSGD \(4\-bit\)\[[2](https://arxiv.org/html/2607.06651#bib.bib26)\]92\.18\.0\>50\{\>\}50PowerSGD \(r=4r\{=\}4\)\[[25](https://arxiv.org/html/2607.06651#bib.bib27)\]95\.812\.143MESH\-FL Light97\.739\.3631MESH\-FL Moderate96\.4740\.5240MESH\-FL Heavy96\.2056\.8240TABLE II:System efficiency on AV\-MNIST; per\-round values averaged across 15 clients\.Table[I](https://arxiv.org/html/2607.06651#S6.T1)compares accuracy and compression efficiency across all methods\. MESH\-FL light achieves the highest accuracy \(97\.73%\) at9\.36×9\.36\\timescompression, outperforming even the uncompressed FedAvg baseline\. PowerSGD is the strongest baseline at 95\.8% accuracy, but reaches only12\.1×12\.1\\timescompression and still falls 1\.93% below MESH\-FL light\. Both QSGD \(92\.1%\) and TopK \(93\.4%\) underperform FedAvg under the strongly non\-IID regime: QSGD suffers from quantization noise that compounds with gradient dissimilarity, while TopK transmits only 1% of gradient coordinates per round, disrupting layer\-wise gradient structure across heterogeneous modality branches\.

![Refer to caption](https://arxiv.org/html/2607.06651v1/fig_ablation_bar.png)Figure 5:Entropy\-guided and uniform\-rank MPS under identical moderate budgets\.Table[II](https://arxiv.org/html/2607.06651#S6.T2)provides the full runtime breakdown\. The Comp\. column captures client\-side SVD and MPS overhead, consistent with the𝒪​\(∑ℓmℓ​nℓ​q\)\\mathcal\{O\}\(\\sum\_\{\\ell\}m\_\{\\ell\}n\_\{\\ell\}q\)complexity of Section[IV](https://arxiv.org/html/2607.06651#S4)\-F\. Although MESH\-FL adds 0\.22–0\.38 s of computation per round compared to the near\-zero overhead of TopK and QSGD, it more than compensates through transmission savings\. The net per\-round wall time \(Comp\.\+\+Trans\.\) is 0\.78 s, 0\.43 s, and 0\.47 s for light, moderate, and heavy compression, all well below FedAvg \(4\.90 s\) and below TopK \(0\.50 s\) and QSGD \(0\.63 s\) at the moderate and heavy levels\. Most critically, the total data transmitted \(Total TX\) column shows that MESH\-FL heavy requires only 0\.51 GB to reach 95% accuracy versus 33\.63 GB for FedAvg \(66×66\\timesreduction\) and 2\.54 GB for the next\-best PowerSGD \(5×5\\timesreduction\)\.

### VI\-DAblation Study

To isolate the impact of entropy\-guided rank allocation, Fig\.[5](https://arxiv.org/html/2607.06651#S6.F5)compares MESH\-FL moderate with a uniform\-rank MPS baseline under identical per\-client payload budgets\. MESH\-FL reaches 96\.47% final accuracy, outperforming uniform\-rank MPS \(94\.88%\) by 1\.59% and FedAvg \(95\.72%\) by 0\.75%, despite using the same40\.52×40\.52\\timescompression budget\. It also reaches 95% accuracy within 40 rounds\. In contrast, uniform\-rank MPS fails to reach this threshold within 50 rounds, reducing Total TX to 0\.73 GB compared with at least 0\.92 GB for uniform\-rank MPS and 33\.63 GB for FedAvg\. Since both MPS variants have nearly identical per\-round wall\-clock times, namely 0\.43 s for MESH\-FL and 0\.41 s for uniform\-rank MPS, the improvement cannot be attributed to computational differences\. The gain is most visible during rounds 5–20 underα=0\.1\\alpha=0\.1, where entropy guidance allocates higher ranks to diffuse fusion layers spectrally and fewer ranks to low\-entropy encoder layers\. This ablation confirms that the gain results from more effective layer\-wise rank allocation, not additional communication or computation\.

## VIIConclusion

In this paper, we presented MESH\-FL, an entropy\-guided MPS update\-compression framework for FL under joint modality and device heterogeneity\. By projecting layer\-wise updates onto dominant spectral subspaces, MESH\-FL allocates compression ranks according to spectral entropy, solving a convex surrogate rank\-allocation problem while preserving the standard non\-convex FL convergence rate\. Experiments on a 15\-node Raspberry Pi 4/5 cluster showed up to56\.8×56\.8\\timescompression, up to a 2\.01% accuracy improvement over uncompressed FedAvg under strongly non\-IID data \(α=0\.1\\alpha=0\.1\), and up to a66×66\\timesreduction in transmitted data to convergence\. These results demonstrate that entropy\-guided rank allocation can reduce communication cost while maintaining, and in some cases improving, model accuracy\. The main limitations are that the exponential tail\-decay assumption has not been verified across all layers and training regimes, and the observed accuracy gain lacks a formal theoretical explanation\. Future work will extend the three\-core MPS design to higher\-order tensor\-train decompositions, combine entropy\-guided compression with asynchronous aggregation, and study when low\-rank gradient compression improves generalization in non\-IID FL\.

## References

- \[1\]\(2025\)Survey of multimodal federated learning: exploring data integration, challenges, and future directions\.IEEE Open Journal of the Communications Society6\(\),pp\. 2510–2538\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p1.1)\.
- \[2\]D\. Alistarh, D\. Grubic, J\. Li, R\. Tomioka, and M\. Vojnovic\(2017\)QSGD: communication\-efficient SGD via gradient quantization and encoding\.Advances in neural information processing systems30\.Cited by:[§II\-A](https://arxiv.org/html/2607.06651#S2.SS1.p1.1),[§VI\-A](https://arxiv.org/html/2607.06651#S6.SS1.p4.19),[TABLE I](https://arxiv.org/html/2607.06651#S6.T1.5.3.2)\.
- \[3\]X\. Cao, T\. Başar, S\. Diggavi, Y\. C\. Eldar, K\. B\. Letaief, H\. V\. Poor, and J\. Zhang\(2023\-04\)Communication\-efficient distributed learning: an overview\.IEEE Journal on Selected Areas in Communications41\(4\),pp\. 851–873\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p3.1)\.
- \[4\]L\. Che, J\. Wang, Y\. Zhou, and F\. Ma\(2023\)Multimodal federated learning: a survey\.Sensors23\(15\)\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p3.1)\.
- \[5\]G\. Chen, Y\. Jia, H\. Wang, C\. Pang, and W\. Xu\(2026\-06\)FedGSE: gradient\-based submodel extraction for resource\-constrained federated learning\.IEEE Transactions on Mobile Computing25\(6\),pp\. 7735–7747\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p1.1)\.
- \[6\]E\. Diao, J\. Ding, and V\. Tarokh\(2021\-May 3–7,\)HeteroFL: computation and communication efficient federated learning for heterogeneous clients\.InProceedings of the 9th International Conference on Learning Representations \(ICLR\),Cited by:[§II\-C](https://arxiv.org/html/2607.06651#S2.SS3.p1.1)\.
- \[7\]G\. Gad, A\. Farrag, A\. Aboulfotouh, K\. Bedda, Z\. Md\. Fadlullah, and M\. M\. Fouda\(2024\-05\)Joint self\-organizing maps and knowledge\-distillation\-based communication\-efficient federated learning for resource\-constrained UAV\-IoT systems\.IEEE Internet of Things Journal11\(9\),pp\. 15504–15522\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p2.1)\.
- \[8\]T\. H\. Hsu, H\. Qi, and M\. Brown\(2019\)Measuring the effects of non\-identical data distribution for federated visual classification\.arXiv preprint arXiv:1909\.06335\.Cited by:[§VI\-A](https://arxiv.org/html/2607.06651#S6.SS1.p2.2)\.
- \[9\]Z\. Jackson, C\. Souza, J\. Flaks, Y\. Nicholas, H\. Kaiser, and A\. Thite\(2018\)Free spoken digit dataset \(FSDD\)\.Note:GitHub repositoryAvailable:[https://github\.com/Jakobovski/free\-spoken\-digit\-dataset](https://github.com/Jakobovski/free-spoken-digit-dataset)Cited by:[§VI\-A](https://arxiv.org/html/2607.06651#S6.SS1.p2.2)\.
- \[10\]N\. Jia, Z\. Qu, B\. Ye, Y\. Wang, S\. Hu, and S\. Guo\(2025\-12\)A comprehensive survey on communication\-efficient federated learning in mobile edge environments\.IEEE Communications Surveys & Tutorials27\(6\),pp\. 3710–3741\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p3.1)\.
- \[11\]S\. P\. Karimireddy, S\. Kale, M\. Mohri, S\. Reddi, S\. Stich, and A\. T\. Suresh\(2020\-Jul\. 13–18,\)SCAFFOLD: stochastic controlled averaging for federated learning\.InProceedings of the International Conference on Machine Learning \(ICML\),pp\. 5132–5143\.Cited by:[§II\-C](https://arxiv.org/html/2607.06651#S2.SS3.p1.1)\.
- \[12\]V\. Lebedev, Y\. Ganin, M\. Rakhuba, I\. Oseledets, and V\. Lempitsky\(2014\)Speeding\-up convolutional neural networks using fine\-tuned CP\-decomposition\.arXiv preprint arXiv:1412\.6553\.Cited by:[§II\-B](https://arxiv.org/html/2607.06651#S2.SS2.p1.1)\.
- \[13\]Y\. LeCun, L\. Bottou, Y\. Bengio, and P\. Haffner\(1998\)Gradient\-based learning applied to document recognition\.Proceedings of the IEEE86\(11\),pp\. 2278–2324\.Cited by:[§VI\-A](https://arxiv.org/html/2607.06651#S6.SS1.p2.2)\.
- \[14\]T\. Li, A\. K\. Sahu, M\. Zaheer, M\. Sanjabi, A\. Talwalkar, and V\. Smith\(2020\-Mar\. 2–4,\)Federated optimization in heterogeneous networks\.InProceedings of Machine Learning and Systems \(MLSys\),Vol\.2,Austin, TX, USA,pp\. 429–450\.Cited by:[§II\-C](https://arxiv.org/html/2607.06651#S2.SS3.p1.1)\.
- \[15\]Y\. Lin, S\. Han, H\. Mao, Y\. Wang, and W\. J\. Dally\(2018\-April 30–May 3,\)Deep gradient compression: reducing the communication bandwidth for distributed training\.InProceedings of the International Conference on Learning Representations \(ICLR\),Vancouver, BC, Canada\.Cited by:[§II\-A](https://arxiv.org/html/2607.06651#S2.SS1.p1.1),[§VI\-A](https://arxiv.org/html/2607.06651#S6.SS1.p4.19),[TABLE I](https://arxiv.org/html/2607.06651#S6.T1.4.2.1)\.
- \[16\]B\. McMahan, E\. Moore, D\. Ramage, S\. Hampson, and B\. A\. y\. Arcas\(2017\-Apr\. 20–22,\)Communication\-efficient learning of deep networks from decentralized data\.InProceedings of the 20th International Conference on Artificial Intelligence and Statistics \(AISTATS\),Vol\.54,Fort Lauderdale, FL, USA,pp\. 1273–1282\.Cited by:[§VI\-A](https://arxiv.org/html/2607.06651#S6.SS1.p4.19),[TABLE I](https://arxiv.org/html/2607.06651#S6.T1.6.5.1.1)\.
- \[17\]Z\. Niu, H\. Dong, A\. K\. Qin, T\. Gu, and P\. Zhang\(2026\-06\)Energy and memory\-efficient federated learning with ordered layer freezing\.IEEE Transactions on Mobile Computing25\(6\),pp\. 7657–7671\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p1.1)\.
- \[18\]A\. Novikov, D\. Podoprikhin, A\. Osokin, and D\. P\. Vetrov\(2015\)Tensorizing neural networks\.Advances in neural information processing systems28\.Cited by:[§II\-B](https://arxiv.org/html/2607.06651#S2.SS2.p1.1)\.
- \[19\]I\. V\. Oseledets\(2011\)Tensor\-train decomposition\.SIAM Journal on Scientific Computing33\(5\),pp\. 2295–2317\.Cited by:[§II\-B](https://arxiv.org/html/2607.06651#S2.SS2.p1.1)\.
- \[20\]Q\. B\. Phan and T\. T\. Nguyen\(2026\-06\)A secure multi\-modal federated transfer learning system for ECG classification\.Neurocomputing681,pp\. 133402\.External Links:ISSN 0925\-2312Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p1.1)\.
- \[21\]A\. Reisizadeh, A\. Mokhtari, H\. Hassani, A\. Jadbabaie, and R\. Pedarsani\(2020\-August 26–28\)FedPAQ: a communication\-efficient federated learning method with periodic averaging and quantization\.InProceedings of the 23rd International Conference on Artificial Intelligence and Statistics,pp\. 2021–2031\.Cited by:[§II\-A](https://arxiv.org/html/2607.06651#S2.SS1.p2.1)\.
- \[22\]S\. U\. Stich, J\. Cordonnier, and M\. Jaggi\(2018\)Sparsified SGD with memory\.Advances in neural information processing systems31\.Cited by:[§II\-A](https://arxiv.org/html/2607.06651#S2.SS1.p1.1)\.
- \[23\]G\. Sun, M\. Mendieta, A\. Dutta, X\. Li, and C\. Chen\(2024\-Sep\. 29–Oct\. 4,\)Towards multi\-modal transformers in federated learning\.InProceedings of the 18th European Conference on Computer Vision \(ECCV\),Milan, Italy,pp\. 229–246\.Cited by:[§II\-C](https://arxiv.org/html/2607.06651#S2.SS3.p2.1)\.
- \[24\]A\. Tjandra, S\. Sakti, and S\. Nakamura\(2017\-May 14–19\)Compressing recurrent neural network with tensor train\.InProceedings of the 2017 International Joint Conference on Neural Networks \(IJCNN\),Anchorage, AK, USA,pp\. 4451–4458\.Cited by:[§II\-B](https://arxiv.org/html/2607.06651#S2.SS2.p1.1)\.
- \[25\]T\. Vogels, S\. P\. Karimireddy, and M\. Jaggi\(2019\)PowerSGD: practical low\-rank gradient compression for distributed optimization\.Advances in Neural Information Processing Systems32\.Cited by:[§II\-A](https://arxiv.org/html/2607.06651#S2.SS1.p2.1),[§VI\-A](https://arxiv.org/html/2607.06651#S6.SS1.p4.19),[TABLE I](https://arxiv.org/html/2607.06651#S6.T1.6.4.1)\.
- \[26\]W\. Wen, C\. Xu, F\. Yan, C\. Wu, Y\. Wang, Y\. Chen, and H\. Li\(2017\)TernGrad: ternary gradients to reduce communication in distributed deep learning\.Advances in neural information processing systems30\.Cited by:[§II\-A](https://arxiv.org/html/2607.06651#S2.SS1.p1.1)\.
- \[27\]C\. Xie, S\. Koyejo, and I\. Gupta\(2020\-Dec\. 11–12,\)Asynchronous federated optimization\.InProceedings of the 12th Annual Workshop on Optimization for Machine Learning at NeurIPS,Cited by:[§II\-C](https://arxiv.org/html/2607.06651#S2.SS3.p1.1)\.
- \[28\]W\. Xie, J\. Ma, T\. Lu, Y\. Li, J\. Lei, L\. Fang, and Q\. Du\(2025\)Distributed deep learning with gradient compression for big remote sensing image interpretation\.IEEE Transactions on Neural Networks and Learning Systems36\(8\),pp\. 14328–14342\.Cited by:[§I](https://arxiv.org/html/2607.06651#S1.p3.1)\.
- \[29\]Y\. Yang, D\. Krompass, and V\. Tresp\(2017\-August 6–11,\)Tensor\-train recurrent neural networks for video classification\.InProceedings of the 34th International Conference on Machine Learning \(PMLR\),Vol\.70,Sydney, NSW, Australia,pp\. 3891–3900\.Cited by:[§II\-B](https://arxiv.org/html/2607.06651#S2.SS2.p1.1)\.
- \[30\]Y\. Zhao, M\. Li, L\. Lai, N\. Suda, D\. Civin, and V\. Chandra\(2018\)Federated learning with non\-IID data\.arXiv preprint arXiv:1806\.00582\.Cited by:[§II\-C](https://arxiv.org/html/2607.06651#S2.SS3.p1.1)\.