FRAME: Learning the Adaptation Domain with a Mixture of Fractional-Fourier Experts

# Learning the Adaptation Domain with a Mixture of Fractional-Fourier Experts
Source: [https://arxiv.org/html/2607.00162](https://arxiv.org/html/2607.00162)
Tom Saliencro1, Maya Lindqvist1, Rohan Desai2, Priya Nair1, Daniel Whitmore2

1University of California, Irvine 2University of Washington saliencro@gmail\.com

###### Abstract

Parameter\-efficient fine\-tuning \(PEFT\) reparameterizes weight updates in a fixed basis: low\-rank adapters operate in the*spatial*domain, while a recent line of spectral methods operates in a*fixed*Fourier domain\. We argue that the choice of domain is itself a design degree of freedom that should be learned, and that no single basis is optimal across tasks, layers, or tokens\. We introduceFrame\(*Fractional\-Fourier Mixture of Experts*\), a mixture\-of\-experts adapter in which every expert carries a learnable*fractional\-Fourier order*that continuously interpolates between the spatial domain \(recovering vanilla LoRA\) and the Fourier domain \(recovering a spectral adapter\)\. Routing tokens through experts that occupy different points on this spatial–spectral continuum lets the model place each low\-rank update in the domain where it is most compact, and—because fractional\-Fourier operators of different orders are mutually incoherent—makes the experts naturally decorrelated, which reduces interference and improves multi\-task composition\. The order is a single scalar per expert, trained with a separate optimizer, and the transform is computed with an𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)chirp–FFT surrogate, soFrameadds negligible cost over standard MoE\-LoRA\. Across commonsense, mathematical, code, and knowledge benchmarks onLLaMA\-3\.1\-8BandQwen2\.5\-7B,Frameimproves over strong MoE\-LoRA and spectral baselines—including FlyLoRA, FourierMoE, and HMoRA—while keeping the active\-parameter budget small, and analysis shows that the learned orders specialize by task and layer in interpretable ways\.

Frame: Learning the Adaptation Domain with a Mixture of Fractional\-Fourier Experts

Tom Saliencro1, Maya Lindqvist1, Rohan Desai2, Priya Nair1, Daniel Whitmore21University of California, Irvine2University of Washingtonsaliencro@gmail\.com

## 1Introduction

Parameter\-efficient fine\-tuning \(PEFT\) adapts a frozen foundation model by training a small number of additional parameters\(Houlsbyet al\.,[2019](https://arxiv.org/html/2607.00162#bib.bib1); Li and Liang,[2021](https://arxiv.org/html/2607.00162#bib.bib4); Lesteret al\.,[2021](https://arxiv.org/html/2607.00162#bib.bib5)\)\. Low\-rank adaptation \(LoRA\) is the dominant instance: it writes the weight update of a linear layer as a product of two low\-rank matrices and has become a default for instruction tuning and domain specialization\(Huet al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib2); Dettmerset al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib3)\)\. A natural way to increase the capacity of a single adapter without inflating its active parameter count is to turn it into a mixture of experts \(MoE\), routing each token to a few specialized low\-rank modules\(Douet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib12); Liet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib13); Wuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib14); Tianet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib15)\)\.

Almost all of these methods share a hidden assumption: the update is parameterized in the*spatial*\(canonical coordinate\) domain, where the low\-rank prior is imposed directly on the weight matrix\. A separate and growing line of work instead parameterizes the update in the*Fourier*domain, learning a sparse or low\-rank set of spectral coefficients and mapping them back with an inverse transform\(Gaoet al\.,[2024b](https://arxiv.org/html/2607.00162#bib.bib25); Borseet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib26); Bilicanet al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib27); Zhanget al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib28)\)\. Spectral adapters are attractive because pretrained weight updates are often spectrally concentrated, and because the Fourier basis is global: a few coefficients can express a high\-rank spatial update\. The most recent spectral method even builds a mixture of frequency\-band experts with a frequency\-aware router\(Jianget al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib29)\)\.

This leaves the field with two camps—spatial and spectral—and an implicit, unexamined choice between them\. We make that choice explicit and ask a different question:*in which domain should an adapter be low\-rank?*Our answer is that the domain should not be fixed at all\. The spatial and Fourier bases are merely the two endpoints of a one\-parameter family of unitary transforms, the*fractional\-Fourier transform*\(FrFT\), whose orderaarotates a signal continuously in the time–frequency plane:a=0a\{=\}0is the identity \(spatial domain\) anda=1a\{=\}1is the ordinary Fourier transform\(Namias,[1980](https://arxiv.org/html/2607.00162#bib.bib32); Almeida,[1994](https://arxiv.org/html/2607.00162#bib.bib33)\)\. Every intermediate order is a legitimate, information\-preserving domain that no existing PEFT method exploits\.

![Refer to caption](https://arxiv.org/html/2607.00162v1/x1.png)Figure 1:The adaptation domain matters and is not universal\.\(a\) Sweeping a single fixed fractional\-Fourier orderaafor a LoRA\-style adapter, the best validation accuracy is attained at task\-dependent*intermediate*orders—neither the spatial \(a=0a\{=\}0\) nor the Fourier \(a=1a\{=\}1\) endpoint\. \(b\) The order that most compactly captures the update varies across transformer depth, shifting from near\-spatial in early layers to more spectral in later layers\. \(c\) Energy compaction: the fraction of update energy captured by the top\-rrcomponents is maximized at a task\-specific order, so a fixed basis wastes capacity\.Framelearns a*mixture*of orders rather than committing to one\.We introduceFrame\(*Fractional\-Fourier Mixture of Experts*\), a PEFT method that makes the adaptation domain a learnable, per\-expert quantity\.Frameis a mixture of low\-rank experts in which each expertiiowns a scalar orderaia\_\{i\}and imposes its low\-rank update in the order\-aia\_\{i\}fractional domain\. A token router activates the top\-kkexperts per token, so different tokens are adapted in different domains, and the orders themselves are trained by gradient descent\. The construction is a strict generalization of existing adapters: an expert withai=0a\_\{i\}\{=\}0is exactly a LoRA expert, and an expert withai=1a\_\{i\}\{=\}1is a Fourier\-domain expert, soFramecontains spatial MoE\-LoRA and spectral MoE\-LoRA as special cases and learns where on the continuum to sit\.

This idea has three consequences\.*Expressivity*: because different orders induce different rank\-rrsubspaces, a mixture spanning several orders represents updates that no single\-domain adapter of the same rank can \(§[4](https://arxiv.org/html/2607.00162#S4), §[5](https://arxiv.org/html/2607.00162#S5)\)\.*Decorrelation*: fractional operators of different orders are mutually incoherent, so experts at different orders are naturally decorrelated, which reduces interference between experts and tasks\(Borseet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib26)\)\.*Efficiency*: the order is one scalar per expert and the transform admits an𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)chirp–FFT factorization\(Ozaktaset al\.,[1996](https://arxiv.org/html/2607.00162#bib.bib34)\), so the learned domain is essentially free\. Figure[1](https://arxiv.org/html/2607.00162#S1.F1)previews the motivation: sweeping a single fixed order, the optimum is a task\- and layer\-dependent intermediate value\. We summarize our contributions as follows\.

- •We reframe the spatial\-vs\-spectral dichotomy in PEFT as a single continuous axis, the fractional\-Fourier order, and show that the optimal adaptation domain varies by task, layer, and token \(§[1](https://arxiv.org/html/2607.00162#S1), §[4](https://arxiv.org/html/2607.00162#S4)\)\.
- •We proposeFrame, a mixture\-of\-experts adapter with per\-expert learnable orders, an𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)transform surrogate, grouped load balancing over order bands, and a separate optimizer for the orders \(§[4](https://arxiv.org/html/2607.00162#S4)\)\.
- •We provide theory:Framestrictly generalizes LoRA and Fourier adapters, its order gradients are bounded, its experts are provably decorrelated as a function of order spacing, and the surrogate is near\-exact \(§[5](https://arxiv.org/html/2607.00162#S5)\)\.
- •Across four task families and two backbones,Frameoutperforms strong MoE\-LoRA and spectral baselines at lower active\-parameter cost, with interpretable learned\-order specialization \(§[6](https://arxiv.org/html/2607.00162#S6)\)\.

## 2Related Work

#### Low\-rank and mixture\-of\-experts PEFT\.

LoRA\(Huet al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib2)\)and its descendants refine the rank budget, decompose the update, or shrink the footprint\(Zhanget al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib6); Liuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib7); Dinget al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib8); Kopiczkoet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib9)\)\. Mixture\-of\-experts variants raise capacity at fixed active cost by routing tokens to several low\-rank experts\(Douet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib12); Liet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib13); Wuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib14); Gaoet al\.,[2024a](https://arxiv.org/html/2607.00162#bib.bib16); Renet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib18); Zadouriet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib20)\)\. HydraLoRA shares a down\-projection across asymmetric experts\(Tianet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib15)\); HMoRA routes through a hierarchy of LoRA experts\(Liaoet al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib17)\); LD\-MoLE makes routing differentiable and dynamic\(Zhuanget al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib22)\); and MoA argues that*heterogeneous*experts decorrelate better than identical ones\(Caoet al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib21)\)\.Frameadopts this heterogeneous\-expert template—token routing, grouped balancing, a separate optimizer for the heterogeneity parameter, and a cheap surrogate—but introduces a new axis of heterogeneity: each expert owns a learnable fractional\-Fourier order that rotates its update between the spatial and spectral domains\. All of these methods operate in the spatial domain and are instances of thea=0a\{=\}0special case ofFrame\.

#### Spectral PEFT\.

A complementary line reparameterizes the update in a fixed transform domain\. FourierFT learns a sparse set of spectral coefficients and recovers the spatial update by inverse DFT\(Gaoet al\.,[2024b](https://arxiv.org/html/2607.00162#bib.bib25)\); FouRA places the low\-rank projections in the Fourier domain and observes that the resulting bases are decorrelated and merge well\(Borseet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib26)\); WaveFT moves to the wavelet domain for extreme sparsity\(Bilicanet al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib27)\); and CrossSpectra exploits cross\-layer spectral smoothness\(Zhanget al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib28)\)\. FourierMoE combines this idea with MoE, routing tokens to experts that own fixed frequency bands\(Jianget al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib29)\)\. These methods all commit to a*single*transform \(DFT or DWT\)\.Frameinstead learns the transform itself through its order, recovering FourierFT\-style spectral adapters ata=1a\{=\}1and LoRA ata=0a\{=\}0as the two ends of a continuum, and lets a mixture occupy intermediate domains\.

#### Decorrelation, interference, and merging\.

Why does the domain matter beyond compactness? Updates that occupy overlapping subspaces interfere, both within a multi\-task model and when independently trained adapters are merged\(Ilharcoet al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib36); Yadavet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib37); Matena and Raffel,[2022](https://arxiv.org/html/2607.00162#bib.bib39); Ortiz\-Jimenezet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib40); Yuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib38); Zhang and Zhou,[2025](https://arxiv.org/html/2607.00162#bib.bib42)\)\. A recurring remedy is to push updates into decorrelated or mutually orthogonal subspaces: orthogonal subspace constraints for continual and multi\-task adaptation\(Chaudhryet al\.,[2020](https://arxiv.org/html/2607.00162#bib.bib44); Wanget al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib43); Zhang and Zhou,[2025](https://arxiv.org/html/2607.00162#bib.bib42)\), sparse or random projections\(McDonnellet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib51); Zouet al\.,[2025c](https://arxiv.org/html/2607.00162#bib.bib24),[b](https://arxiv.org/html/2607.00162#bib.bib50),[a](https://arxiv.org/html/2607.00162#bib.bib49)\), and frequency\-domain projections whose bases are naturally decorrelated\(Borseet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib26)\)\.Framepursues the same goal with a learnable mechanism: experts at well\-separated fractional orders are provably incoherent \(§[5](https://arxiv.org/html/2607.00162#S5)\), so domain diversity—rather than a fixed projection or hand\-imposed orthogonality—does the separating\.

## 3Preliminaries

#### LoRA and MoE\-LoRA\.

For a frozen linear layerW0∈ℝdout×dW\_\{0\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times d\}, LoRA adds a low\-rank updateΔ​W=B​A\\Delta W=BAwithA∈ℝr×dA\\in\\mathbb\{R\}^\{r\\times d\},B∈ℝdout×rB\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times r\},r≪dr\\ll d, so the layer computesW0​x\+αr​B​A​xW\_\{0\}x\+\\tfrac\{\\alpha\}\{r\}BAx\(Huet al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib2)\)\. An MoE\-LoRA layer keepsNNsuch experts\{\(Ai,Bi\)\}i=1N\\\{\(A\_\{i\},B\_\{i\}\)\\\}\_\{i=1\}^\{N\}and a routerg​\(x\)=softmax​\(top​\-​k​\(Wg​x\)\)g\(x\)=\\mathrm\{softmax\}\(\\mathrm\{top\}\\text\{\-\}k\(W\_\{g\}x\)\), activating thekkhighest\-scoring experts:

MoE​\(x\)=W0​x\+αr​∑i∈𝒮​\(x\)gi​\(x\)​Bi​Ai​x,\\textstyle\\mathrm\{MoE\}\(x\)=W\_\{0\}x\+\\tfrac\{\\alpha\}\{r\}\\sum\_\{i\\in\\mathcal\{S\}\(x\)\}g\_\{i\}\(x\)\\,B\_\{i\}A\_\{i\}x,\(1\)where𝒮​\(x\)\\mathcal\{S\}\(x\)are the top\-kkexperts for tokenxx\(Wuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib14); Feduset al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib54)\)\.

#### Fractional\-Fourier transform\.

The discrete fractional\-Fourier transform \(DFrFT\) of orderaais a unitary matrix𝚽a∈ℂd×d\\bm\{\\Phi\}\_\{a\}\\in\\mathbb\{C\}^\{d\\times d\}that generalizes the DFT𝑭\\bm\{F\}\(Namias,[1980](https://arxiv.org/html/2607.00162#bib.bib32); Almeida,[1994](https://arxiv.org/html/2607.00162#bib.bib33); Candanet al\.,[2000](https://arxiv.org/html/2607.00162#bib.bib35)\)\. Writing the angleθ=π2​a\\theta=\\tfrac\{\\pi\}\{2\}a, it admits the spectral form

𝚽a=∑m=0d−1e−i​m​θ​𝒖m​𝒖m⊤,\\bm\{\\Phi\}\_\{a\}=\\sum\_\{m=0\}^\{d\-1\}e^\{\-\\mathrm\{i\}\\,m\\theta\}\\,\\bm\{u\}\_\{m\}\\bm\{u\}\_\{m\}^\{\\\!\\top\},\(2\)where\{𝒖m\}\\\{\\bm\{u\}\_\{m\}\\\}are the \(real\) Hermite–Gauss eigenvectors shared by all orders, andmmindexes the eigenphase\. It satisfies four properties we use throughout: \(i\)*identity*,𝚽0=𝑰\\bm\{\\Phi\}\_\{0\}=\\bm\{I\}; \(ii\)*Fourier limit*,𝚽1=𝑭\\bm\{\\Phi\}\_\{1\}=\\bm\{F\}; \(iii\)*index additivity*,𝚽a​𝚽a′=𝚽a\+a′\\bm\{\\Phi\}\_\{a\}\\bm\{\\Phi\}\_\{a^\{\\prime\}\}=\\bm\{\\Phi\}\_\{a\+a^\{\\prime\}\}, hence𝚽a​𝚽a∗=𝑰\\bm\{\\Phi\}\_\{a\}\\bm\{\\Phi\}\_\{a\}^\{\\\!\*\}=\\bm\{I\}\(unitarity\); and \(iv\)*Parseval*,‖𝚽a​𝒗‖2=‖𝒗‖2\\\|\\bm\{\\Phi\}\_\{a\}\\bm\{v\}\\\|\_\{2\}=\\\|\\bm\{v\}\\\|\_\{2\}\. We write𝑹a=Re​\{𝚽a\}∈ℝd×d\\bm\{R\}\_\{a\}=\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\}\\right\\\}\\in\\mathbb\{R\}^\{d\\times d\}for its real part, so𝑹0=𝑰\\bm\{R\}\_\{0\}=\\bm\{I\}and𝑹1=Re​\{𝑭\}\\bm\{R\}\_\{1\}=\\mathrm\{Re\}\\\!\\left\\\{\\bm\{F\}\\right\\\}is a \(real\) cosine\-type transform\.

## 4Frame

![Refer to caption](https://arxiv.org/html/2607.00162v1/figures/architecture.png)Figure 2:Framearchitecture\.A token is routed to its top\-kkexperts; each expert imposes its low\-rank updateBi​AiB\_\{i\}A\_\{i\}in the order\-aia\_\{i\}fractional\-Fourier domain, whereaia\_\{i\}is learnable\. Ordera=0a\{=\}0recovers a spatial LoRA expert anda=1a\{=\}1a Fourier\-domain expert, so the mixture spans the spatial–spectral continuum\. Expert outputs are summed with the frozen base path\. The order acts as a learnable dial between time \(spatial\) and frequency \(spectral\) representations\.### 4\.1Per\-expert learnable domain

Framereplaces each spatial low\-rank expert with a*fractional*expert that imposes its low\-rank prior in a learned domain\. Expertiiowns factorsAi∈ℝr×dA\_\{i\}\\in\\mathbb\{R\}^\{r\\times d\},Bi∈ℝdout×rB\_\{i\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times r\}and a scalar orderai∈\[0,1\]a\_\{i\}\\in\[0,1\], and its update is

Δ​Wi=Bi​Ai​𝑹ai,𝑹ai=Re​\{𝚽ai\}\.\\Delta W\_\{i\}\\;=\\;B\_\{i\}A\_\{i\}\\,\\bm\{R\}\_\{a\_\{i\}\},\\qquad\\bm\{R\}\_\{a\_\{i\}\}=\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\}\\right\\\}\.\(3\)Equivalently, sinceBi,AiB\_\{i\},A\_\{i\}are real, the update applied to a token factors through a transform of the*input*,

Δ​Wi​x=Bi​Ai​Re​\{𝚽ai​x\},\\Delta W\_\{i\}\\,x\\;=\\;B\_\{i\}A\_\{i\}\\,\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\}x\\right\\\},\(4\)so the token is first rotated into the order\-aia\_\{i\}domain, its real part is taken, and the standard low\-rank map is applied\. Two observations make this the right object\. Atai=0a\_\{i\}\{=\}0,𝑹0=𝑰\\bm\{R\}\_\{0\}=\\bm\{I\}and \([3](https://arxiv.org/html/2607.00162#S4.E3)\) reduces toΔ​Wi=Bi​Ai\\Delta W\_\{i\}=B\_\{i\}A\_\{i\},*exactly*a LoRA expert\. Atai=1a\_\{i\}\{=\}1,𝑹1\\bm\{R\}\_\{1\}is a cosine transform and the expert is a Fourier\-domain adapter in the spirit ofGaoet al\.\([2024b](https://arxiv.org/html/2607.00162#bib.bib25)\); Borseet al\.\([2024](https://arxiv.org/html/2607.00162#bib.bib26)\)\. For any intermediate order the update is still rank≤r\\leq r, but its row space is the rank\-rrspace rotated by𝑹ai\\bm\{R\}\_\{a\_\{i\}\}; different orders therefore realize*different*rank\-rrsubspaces from the same number of parameters\.

#### The full layer\.

StackingNNfractional experts and a token router gives theFramelayer \(Figure[2](https://arxiv.org/html/2607.00162#S4.F2)\)

Frame​\(x\)=W0​x\+αr​∑i∈𝒮​\(x\)gi​\(x\)​Δ​Wi​x,\\textsc\{Frame\}\(x\)=W\_\{0\}x\+\\frac\{\\alpha\}\{r\}\\\!\\\!\\sum\_\{i\\in\\mathcal\{S\}\(x\)\}\\\!\\\!g\_\{i\}\(x\)\\,\\Delta W\_\{i\}\\,x,\(5\)where, by \([4](https://arxiv.org/html/2607.00162#S4.E4)\),Δ​Wi​x=Bi​Ai​Re​\{𝚽ai​x\}\\Delta W\_\{i\}x=B\_\{i\}A\_\{i\}\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\}x\\right\\\},g​\(x\)=softmax​\(top​\-​k​\(Wg​x\)\)g\(x\)=\\mathrm\{softmax\}\(\\mathrm\{top\}\\text\{\-\}k\(W\_\{g\}x\)\), and𝒮​\(x\)\\mathcal\{S\}\(x\)are the top\-kkactive experts\. The trainable parameters are the low\-rank factors\{Ai,Bi\}\\\{A\_\{i\},B\_\{i\}\\\}, the orders\{ai\}\\\{a\_\{i\}\\\}, and the routerWgW\_\{g\}; the backboneW0W\_\{0\}stays frozen\. We constrainai∈\[0,1\]a\_\{i\}\\in\[0,1\]with a sigmoid reparameterizationai=σ​\(a~i\)a\_\{i\}=\\sigma\(\\tilde\{a\}\_\{i\}\)and traina~i\\tilde\{a\}\_\{i\}\.

### 4\.2Efficient fractional transform

Materializing𝚽a\\bm\{\\Phi\}\_\{a\}as a densed×dd\\times dmatrix would cost𝒪​\(d2\)\\mathcal\{O\}\(d^\{2\}\)per token and break parameter efficiency\. We instead use the chirp–FFT factorization ofOzaktaset al\.\([1996](https://arxiv.org/html/2607.00162#bib.bib34)\), which computes𝚽a​x\\bm\{\\Phi\}\_\{a\}xas a pre\-chirp multiplication, an FFT, and a post\-chirp multiplication,

𝚽a​x=𝒄2⊙ℱ​\(𝒄1⊙x\),\\bm\{\\Phi\}\_\{a\}x=\\bm\{c\}\_\{2\}\\odot\\mathcal\{F\}\\\!\\big\(\\bm\{c\}\_\{1\}\\odot x\\big\),\(6\)where𝒄1,𝒄2∈ℂd\\bm\{c\}\_\{1\},\\bm\{c\}\_\{2\}\\in\\mathbb\{C\}^\{d\}are order\-dependent chirps \(closed\-form inθ\\theta\) andℱ\\mathcal\{F\}is the FFT\. The cost is𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)per token and the chirps are recomputed only whenaia\_\{i\}changes, i\.e\. once per step per expert\. Because the transform is shared across all tokens routed to an expert, its amortized cost is dominated by the FFT, andFrame’s wall\-clock overhead over spatial MoE\-LoRA is small \(§[6\.5](https://arxiv.org/html/2607.00162#S6.SS5)\)\. The order gradient∂𝚽a​x/∂a\\partial\\,\\bm\{\\Phi\}\_\{a\}x/\\partial aalso has a closed form through \([2](https://arxiv.org/html/2607.00162#S3.E2)\), namely−i​π2​∑mm​e−i​m​θ​𝒖m​𝒖m⊤​x\-\\mathrm\{i\}\\tfrac\{\\pi\}\{2\}\\sum\_\{m\}m\\,e^\{\-\\mathrm\{i\}m\\theta\}\\bm\{u\}\_\{m\}\\bm\{u\}\_\{m\}^\{\\\!\\top\}x, which we evaluate with the same factorization; in practice automatic differentiation through \([6](https://arxiv.org/html/2607.00162#S4.E6)\) suffices\.

### 4\.3Routing, balancing, and optimization

#### Grouped load balancing over order bands\.

A naive load\-balancing loss pushes the router toward uniform expert usage, which is at odds with our goal: we*want*experts to specialize by domain\. We partition theNNexperts intoGG*order bands*by initializing their orders on a uniform grid in\[0,1\]\[0,1\]and grouping adjacent orders, then apply a balancing loss*within*each band rather than globally:

ℒbal=∑b=1GNb​∑i∈𝒢bfi​pi,\\mathcal\{L\}\_\{\\mathrm\{bal\}\}=\\sum\_\{b=1\}^\{G\}\\,N\_\{b\}\\sum\_\{i\\in\\mathcal\{G\}\_\{b\}\}f\_\{i\}\\,p\_\{i\},\(7\)where𝒢b\\mathcal\{G\}\_\{b\}is bandbb,fif\_\{i\}the fraction of tokens routed to expertii, andpip\_\{i\}the mean router probability\(Feduset al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib54)\)\. This keeps every domain band utilized while letting experts move freely*across*bands as their orders adapt\.

#### Separate optimizer for orders\.

The orders\{ai\}\\\{a\_\{i\}\\\}and the matrices\{Ai,Bi\}\\\{A\_\{i\},B\_\{i\}\\\}live on very different scales: a small change inaia\_\{i\}rotates an entire domain, whereasAi,BiA\_\{i\},B\_\{i\}are dense and high\-dimensional\. We found that a single learning rate destabilizes training, and use a separate optimizer with learning rateηa≪η\\eta\_\{a\}\\ll\\etafor the orders\. The total objective is the task loss plusλbal​ℒbal\\lambda\_\{\\mathrm\{bal\}\}\\mathcal\{L\}\_\{\\mathrm\{bal\}\}\.

Input:token

x∈ℝdx\\in\\mathbb\{R\}^\{d\}; frozen

W0W\_\{0\}; experts

\{\(Ai,Bi,a~i\)\}i=1N\\\{\(A\_\{i\},B\_\{i\},\\tilde\{a\}\_\{i\}\)\\\}\_\{i=1\}^\{N\}; router

WgW\_\{g\}; active

kk; scale

α\\alpha
Output:output

yy; balancing loss

ℒbal\\mathcal\{L\}\_\{\\mathrm\{bal\}\}
1ex

s←Wg​xs\\leftarrow W\_\{g\}x;

𝒮←top​\-​k​\(s\)\\mathcal\{S\}\\leftarrow\\mathrm\{top\}\\text\{\-\}k\(s\);

g←softmax​\(s𝒮\)g\\leftarrow\\mathrm\{softmax\}\(s\_\{\\mathcal\{S\}\}\)
y←W0​xy\\leftarrow W\_\{0\}x

foreach*i∈𝒮i\\in\\mathcal\{S\}*do

ai←σ​\(a~i\)a\_\{i\}\\leftarrow\\sigma\(\\tilde\{a\}\_\{i\}\);

θi←π2​ai\\theta\_\{i\}\\leftarrow\\tfrac\{\\pi\}\{2\}a\_\{i\}
zi←𝒄2​\(θi\)⊙ℱ​\(𝒄1​\(θi\)⊙x\)z\_\{i\}\\leftarrow\\bm\{c\}\_\{2\}\(\\theta\_\{i\}\)\\odot\\mathcal\{F\}\\big\(\\bm\{c\}\_\{1\}\(\\theta\_\{i\}\)\\odot x\\big\)
//

𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)
y←y\+αr​gi​Bi​\(Ai​Re​\{zi\}\)y\\leftarrow y\+\\tfrac\{\\alpha\}\{r\}\\,g\_\{i\}\\,B\_\{i\}\\big\(A\_\{i\}\\,\\mathrm\{Re\}\\\!\\left\\\{z\_\{i\}\\right\\\}\\big\)
end foreach

update

\{Ai,Bi,Wg\}\\\{A\_\{i\},B\_\{i\},W\_\{g\}\\\}with optimizer

𝒪θ\\mathcal\{O\}\_\{\\theta\}\(lr

η\\eta\)

update

\{a~i\}\\\{\\tilde\{a\}\_\{i\}\\\}with optimizer

𝒪a\\mathcal\{O\}\_\{a\}\(lr

ηa≪η\\eta\_\{a\}\\\!\\ll\\\!\\eta\)

return

y,ℒbaly,\\ \\mathcal\{L\}\_\{\\mathrm\{bal\}\}from \([7](https://arxiv.org/html/2607.00162#S4.E7)\)

Algorithm 1Framelayer forward and update
#### Cost\.

Per layer,FramestoresNNexperts \(N​r​\(d\+dout\)Nr\(d\{\+\}d\_\{\\mathrm\{out\}\}\)parameters\),NNorder scalars, and the router \(N​dNd\)\. Only the activekkexperts contribute to the forward pass, so the active parameter count matches akk\-expert MoE\-LoRA pluskkscalars; the extra compute iskklength\-ddFFTs\. Algorithm[1](https://arxiv.org/html/2607.00162#algorithm1)summarizes one layer\.

## 5Theoretical Analysis

We state four properties; full proofs are in Appendix[B](https://arxiv.org/html/2607.00162#A2)\. They formalize the three consequences from §[1](https://arxiv.org/html/2607.00162#S1): generality, decorrelation, and cheap\-but\-faithful computation\.

###### Proposition 1\(Strict generalization\)\.

For any LoRA expert with factors\(A,B\)\(A,B\)there is aFrameexpert that realizes the identical update \(ordera=0a\{=\}0\), and for any FourierFT\-style cosine\-domain low\-rank expert there is aFrameexpert that realizes it \(ordera=1a\{=\}1\)\. Hence the hypothesis class ofFramecontains both spatial and Fourier MoE\-LoRA, and the inclusion is strict forN≥2N\\geq 2\.

###### Proposition 2\(Bounded order gradients\)\.

Let the experts act on the rank\-ρ\\rhoHermite–Gauss subspace spanned by\{𝐮m\}m<ρ\\\{\\bm\{u\}\_\{m\}\\\}\_\{m<\\rho\}\. Then‖∂𝚽a/∂a‖2≤π2​\(ρ−1\)\\big\\\|\\partial\\,\\bm\{\\Phi\}\_\{a\}/\\partial a\\big\\\|\_\{2\}\\leq\\tfrac\{\\pi\}\{2\}\(\\rho\-1\), and the order gradient of the per\-token loss is bounded byπ2​\(ρ−1\)​‖Bi‖2​‖Ai‖2​‖x‖2​‖∇yℒ‖2\\tfrac\{\\pi\}\{2\}\(\\rho\-1\)\\,\\\|B\_\{i\}\\\|\_\{2\}\\\|A\_\{i\}\\\|\_\{2\}\\,\\\|x\\\|\_\{2\}\\,\\\|\\nabla\_\{y\}\\mathcal\{L\}\\\|\_\{2\}\. Training the orders is therefore stable under a bounded step size\.

###### Proposition 3\(Domain\-diversity decorrelation\)\.

Let two experts have i\.i\.d\. random low\-rank factors with𝔼​\[Ai⊤​Ai\]=1d​𝐈\\mathbb\{E\}\[A\_\{i\}^\{\\\!\\top\}A\_\{i\}\]=\\tfrac\{1\}\{d\}\\bm\{I\}and ordersai,aja\_\{i\},a\_\{j\}\. Then the expected Frobenius coherence of their updates obeys

𝔼​\[⟨Δ​Wi,Δ​Wj⟩F‖Δ​Wi‖F​‖Δ​Wj‖F\]2≤1r​κ​\(\|ai−aj\|\),\\mathbb\{E\}\\\!\\left\[\\frac\{\\langle\\Delta W\_\{i\},\\Delta W\_\{j\}\\rangle\_\{\\mathrm\{F\}\}\}\{\\\|\\Delta W\_\{i\}\\\|\_\{\\mathrm\{F\}\}\\\|\\Delta W\_\{j\}\\\|\_\{\\mathrm\{F\}\}\}\\right\]^\{2\}\\leq\\frac\{1\}\{r\}\\,\\kappa\\\!\\big\(\|a\_\{i\}\-a\_\{j\}\|\\big\),\(8\)whereκ​\(δ\)=1d​‖Re​\{𝚽δ\}‖F2∈\[1d,1\]\\kappa\(\\delta\)=\\tfrac\{1\}\{d\}\\\|\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{\\delta\}\\right\\\}\\\|\_\{\\mathrm\{F\}\}^\{2\}\\in\[\\,\\tfrac\{1\}\{d\},1\\,\]is monotonically decreasing inδ\\deltaon\[0,1\]\[0,1\]withκ​\(0\)=1\\kappa\(0\)\{=\}1\. Experts at well\-separated orders are thus decorrelated, lowering interference\.

###### Proposition 4\(Faithful surrogate and complexity\)\.

The chirp–FFT operator \([6](https://arxiv.org/html/2607.00162#S4.E6)\) equals𝚽a​x\\bm\{\\Phi\}\_\{a\}xup to the𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)sampling error of the discretization, and computes the transform in𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)time and𝒪​\(d\)\\mathcal\{O\}\(d\)memory, versus𝒪​\(d2\)\\mathcal\{O\}\(d^\{2\}\)for the dense form\. The end\-to\-end active\-parameter count of aFramelayer isk​r​\(d\+dout\)\+k\+N​dk\\,r\(d\+d\_\{\\mathrm\{out\}\}\)\+k\+Nd\.

Proposition[1](https://arxiv.org/html/2607.00162#Thmproposition1)saysFramenever loses to its endpoints; Proposition[3](https://arxiv.org/html/2607.00162#Thmproposition3)is the formal version of “different domains decorrelate,” linking order spacing to inter\-expert coherence through a*learnable*order; and Propositions[2](https://arxiv.org/html/2607.00162#Thmproposition2)and[4](https://arxiv.org/html/2607.00162#Thmproposition4)ensure the added degree of freedom is trainable and cheap\.

Table 1:Commonsense reasoning accuracy \(%\)on eight benchmarks\.Param %is the active trainable fraction relative to full fine\-tuning\.Frameattains the best average on both backbones while activating fewer parameters than most spatial MoE\-LoRA baselines; FourierFT is the most parameter\-frugal but trails on accuracy\. Best per column inbold\.
## 6Experiments

### 6\.1Setup

#### Backbones and tasks\.

We fine\-tuneLLaMA\-3\.1\-8B\(Grattafioriet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib59)\)andQwen2\.5\-7B\(Yanget al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib60)\), and useQwen2\.5\-1\.5B/3B/14Bfor the scaling study\. We evaluate four task families: \(i\)*commonsense reasoning*—BoolQ, PIQA, SIQA, HellaSwag, WinoGrande, ARC\-easy, ARC\-challenge, OBQA\(Clarket al\.,[2019](https://arxiv.org/html/2607.00162#bib.bib74); Bisket al\.,[2020](https://arxiv.org/html/2607.00162#bib.bib75); Sapet al\.,[2019](https://arxiv.org/html/2607.00162#bib.bib76); Zellerset al\.,[2019](https://arxiv.org/html/2607.00162#bib.bib77); Sakaguchiet al\.,[2020](https://arxiv.org/html/2607.00162#bib.bib78); Clarket al\.,[2018](https://arxiv.org/html/2607.00162#bib.bib79); Mihaylovet al\.,[2018](https://arxiv.org/html/2607.00162#bib.bib73)\); \(ii\)*mathematical reasoning*—GSM8K, MATH, SVAMP, MAWPS, AQuA\(Cobbeet al\.,[2021](https://arxiv.org/html/2607.00162#bib.bib63); Hendryckset al\.,[2021b](https://arxiv.org/html/2607.00162#bib.bib64); Patelet al\.,[2021](https://arxiv.org/html/2607.00162#bib.bib70); Koncel\-Kedziorskiet al\.,[2016](https://arxiv.org/html/2607.00162#bib.bib69); Linget al\.,[2017](https://arxiv.org/html/2607.00162#bib.bib71)\); \(iii\)*code*—HumanEval and MBPP\(Chenet al\.,[2021](https://arxiv.org/html/2607.00162#bib.bib66); Austinet al\.,[2021](https://arxiv.org/html/2607.00162#bib.bib67)\); and \(iv\)*knowledge*—MMLU and ScienceQA\(Hendryckset al\.,[2021a](https://arxiv.org/html/2607.00162#bib.bib65); Luet al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib68)\)\. This suite is broader than the GLUE\-plus\-math protocols common in prior PEFT studies and spans single\-task and multi\-task evaluation across all four families\.

#### Baselines\.

We compare against single\-adapter PEFT \(LoRA, DoRA, AdaLoRA, FourierFT\)\(Huet al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib2); Liuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib7); Zhanget al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib6); Gaoet al\.,[2024b](https://arxiv.org/html/2607.00162#bib.bib25)\)and MoE\-LoRA / heterogeneous\-expert methods \(HydraLoRA, MixLoRA, HMoRA, FlyLoRA, MoA, FourierMoE\)\(Tianet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib15); Liet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib13); Liaoet al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib17); Zouet al\.,[2025c](https://arxiv.org/html/2607.00162#bib.bib24); Caoet al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib21); Jianget al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib29)\)\. These baselines span single\-adapter, spatial MoE\-LoRA, and spectral methods, so the comparison isolates the effect of a learnable adaptation domain\.

#### Implementation\.

Unless noted,FrameusesN=8N\{=\}8experts with top\-k=2k\{=\}2routing, rankr=8r\{=\}8,G=4G\{=\}4order bands initialized on a uniform grid in\[0,1\]\[0,1\],α=16\\alpha\{=\}16, and target modules\{\\\{q,k,v,o,gate,up,down\}\\\}proj\. Orders use a separate AdamW optimizer withηa=η/10\\eta\_\{a\}\{=\}\\eta/10\. We report the mean over three seeds; per\-seed variation and full hyperparameters are in Appendix[C](https://arxiv.org/html/2607.00162#A3)\.

### 6\.2Main results

#### Commonsense reasoning\.

Table[1](https://arxiv.org/html/2607.00162#S5.T1)reports per\-task accuracy\.Frameis best on every task and on average for both backbones, improving the average over the strongest baseline \(MoA\) by\+0\.8\+0\.8on LLaMA\-3\.1\-8B \(87\.187\.1vs\.86\.386\.3\) and\+0\.7\+0\.7on Qwen2\.5\-7B \(88\.288\.2vs\.87\.587\.5\), and over vanilla LoRA by\+2\.6\+2\.6and\+2\.3\+2\.3\. It does so while activating only0\.31%0\.31\\%of parameters—less than most spatial MoE\-LoRA baselines\. Two patterns recur\. First, the fixed\-spectral and fixed\-spatial baselines cluster closely, consistent with our claim that committing to a single domain leaves a gap thatFramecloses by mixing domains\. Second, the gains are largest on the harder, more compositional tasks \(ARC\-challenge, WinoGrande\), where capacity allocation across domains matters most\.

#### Math, code, and knowledge\.

Table[2](https://arxiv.org/html/2607.00162#S6.T2)extends the comparison to mathematical reasoning, code generation, and knowledge on LLaMA\-3\.1\-8B\.Frameagain leads on average \(51\.451\.4\), ahead of the strongest baselines \(both at50\.150\.1\) and well ahead of LoRA \(47\.447\.4\)\. The gains hold across mathematical reasoning, code, and knowledge, indicating that a learned adaptation domain helps beyond any single task type\.

Table 2:Mathematical reasoning, code, and knowledgeon LLaMA\-3\.1\-8B \(accuracy / pass@1, %\)\.Frameattains the best average across all three task families\.

### 6\.3Ablations

Table[3](https://arxiv.org/html/2607.00162#S6.T3)isolates each component on LLaMA\-3\.1\-8B commonsense\.*Fixing the domain*is the most damaging: forcing all experts toa=0a\{=\}0\(spatial MoE\-LoRA\) drops the average by1\.41\.4, and forcinga=1a\{=\}1\(spectral\) by1\.11\.1, confirming that neither endpoint is sufficient\. A fixed but*diverse*grid of orders recovers part of the gap \(−0\.7\-0\.7\), and a single shared learnable order recovers a different part \(−0\.8\-0\.8\); only*per\-expert learnable*orders capture both, showing the two design choices are complementary\. Removing grouped balancing in favor of global balancing costs0\.50\.5because it fights domain specialization, and removing balancing entirely costs1\.21\.2\. The separate order optimizer is worth0\.60\.6\. Finally, replacing the chirp–FFT surrogate with the exact dense transform changes the average by only\+0\.1\+0\.1at much higher cost, validating Proposition[4](https://arxiv.org/html/2607.00162#Thmproposition4)\.

Table 3:Ablationson LLaMA\-3\.1\-8B commonsense\. Per\-expert*learnable*orders and grouped balancing are the key ingredients; the cheap surrogate is near\-lossless\.
### 6\.4Mechanism analysis

#### Learned orders specialize\.

Figure[3](https://arxiv.org/html/2607.00162#S6.F3)\(a\) shows the distribution of learned orders at convergence: experts spread across the continuum rather than collapsing to an endpoint, and the spread widens with depth\. Figure[3](https://arxiv.org/html/2607.00162#S6.F3)\(b\) traces order trajectories during training—experts initialized on the grid migrate to task\-appropriate domains and stabilize after roughly a third of training\. Figure[3](https://arxiv.org/html/2607.00162#S6.F3)\(c\) shows that tokens of different types \(numerals, function words, content words\) are routed to systematically different orders, evidence that the domain is genuinely token\-dependent\.

#### Experts are decorrelated\.

Figure[4](https://arxiv.org/html/2607.00162#S6.F4)\(a\) plots inter\-expert update coherence against order spacing\|ai−aj\|\|a\_\{i\}\-a\_\{j\}\|; coherence falls monotonically, matching Proposition[3](https://arxiv.org/html/2607.00162#Thmproposition3), andFrame’s experts are markedly less correlated than those of spatial MoE\-LoRA at equal rank\. The centered\-kernel\-alignment heatmap in Figure[4](https://arxiv.org/html/2607.00162#S6.F4)\(b\) confirms block\-diagonal \(specialized\) structure, and Figure[4](https://arxiv.org/html/2607.00162#S6.F4)\(c\) shows grouped balancing keeps all order bands utilized\.

![Refer to caption](https://arxiv.org/html/2607.00162v1/x2.png)Figure 3:The learned domain is structured\.\(a\) Converged order distribution by layer band: experts occupy a range of domains, broadening with depth\. \(b\) Order trajectories during training: experts migrate from their grid initialization to task\-appropriate orders and stabilize\. \(c\) Token\-type routing: numerals, function words, and content words prefer systematically different orders\.![Refer to caption](https://arxiv.org/html/2607.00162v1/x3.png)Figure 4:Domain diversity decorrelates experts\.\(a\) Inter\-expert coherence decreases with order spacing\|ai−aj\|\|a\_\{i\}\-a\_\{j\}\|\(Proposition[3](https://arxiv.org/html/2607.00162#Thmproposition3)\);Frameis below spatial MoE\-LoRA at equal rank\. \(b\) Centered kernel alignment between expert updates is block\-diagonal\. \(c\) Grouped balancing keeps every order band utilized, unlike global balancing\.

### 6\.5Efficiency and scaling

Figure[5](https://arxiv.org/html/2607.00162#S6.F5)\(a\) shows the accuracy–parameter trade\-off:Framesits on the Pareto frontier, attaining the best accuracy among the MoE baselines at a modest active\-parameter cost\. Figure[5](https://arxiv.org/html/2607.00162#S6.F5)\(b\) reports that the chirp–FFT surrogate keeps training\-time overhead within7%7\\%of spatial MoE\-LoRA, while the dense transform is3×3\\timesslower—the efficiency story that motivates Proposition[4](https://arxiv.org/html/2607.00162#Thmproposition4)\. Figure[5](https://arxiv.org/html/2607.00162#S6.F5)\(c\) showsFrame’s average gain over the best baseline persists across backbone scales from 1\.5B to 14B, and Figure[5](https://arxiv.org/html/2607.00162#S6.F5)\(d\) confirms robustness to the number of experts and activekk\. We also evaluate*training\-free merging*of independently trainedFrameadapters in Appendix[E](https://arxiv.org/html/2607.00162#A5): domain diversity yields smaller post\-merge drops than spatial adapters, consistent with the decorrelation analysis\.

![Refer to caption](https://arxiv.org/html/2607.00162v1/x4.png)Figure 5:Efficiency and scaling\.\(a\) Accuracy vs\. active parameters:Frameis on the Pareto frontier\. \(b\) Training\-time overhead: the surrogate is within7%7\\%of spatial MoE\-LoRA, the dense transform3×3\\timesslower\. \(c\) Average gain over the best baseline across backbone scales \(1\.5B–14B\)\. \(d\) Sensitivity to expert countNNand activekk\.

## 7Conclusion

We argued that the basis in which a PEFT update is expressed is a learnable design choice, not a fixed convention, and that the spatial and Fourier domains are two endpoints of a continuum that adapters should be free to traverse\.Framerealizes this by giving each expert in a mixture a learnable fractional\-Fourier order, recovering spatial and spectral MoE\-LoRA as special cases while occupying the more expressive interior\. The learned orders specialize by task, layer, and token; experts at different orders are provably and empirically decorrelated; and the construction is essentially free thanks to an𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)surrogate\. Across four task families and two backbonesFrameimproves over strong spatial and spectral baselines at lower active cost\. We see the adaptation domain as a broadly useful axis: the same fractional lens could be applied to full fine\-tuning, to attention, or to continual and mergeable adapters, which we leave to future work\.

## Limitations

Frameintroduces one scalar per expert and a transform per active expert; although the surrogate keeps this cheap, the overhead is non\-zero and grows with the number of active experts, so very largekkerodes the efficiency advantage\. Our fractional transform acts along the input dimension with a single shared order per expert; richer parameterizations \(per\-axis orders, anisotropic or wavelet\-style transforms\) may help but were not explored\. Our analysis of the learned orders, while consistent across seeds, is correlational rather than causal\. Finally, like all MoE adapters,Frameinherits sensitivity to routing collapse, which we mitigate with grouped balancing but do not eliminate\.

## Ethics Statement

Frameis a general\-purpose fine\-tuning method and inherits the risks of the backbones and datasets it is applied to, including the potential to amplify biases present in instruction\-tuning data or to lower the cost of producing harmful specialized models\. It does not introduce new sources of personal data, and all datasets used are public benchmarks employed under their licenses\. BecauseFramecan compose independently trained adapters, practitioners merging a capability adapter with a safety adapter should verify that safety behavior is retained after merging rather than assuming additivity\. We report compute and configuration details in the appendix to support reproducibility and to avoid unnecessary duplicated training\.

## References

- The fractional Fourier transform and time\-frequency representations\.IEEE Transactions on Signal Processing42\(11\),pp\. 3084–3091\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p3.3),[§3](https://arxiv.org/html/2607.00162#S3.SS0.SSS0.Px2.p1.4)\.
- J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.\(2021\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- A\. Bilican, M\. A\. Yılmaz, A\. M\. Tekalp, and R\. G\. Cinbiş \(2025\)Exploring sparsity for parameter efficient fine tuning using wavelets\.arXiv preprint arXiv:2505\.12532\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p2.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px2.p1.2)\.
- Y\. Bisk, R\. Zellers, J\. Gao, Y\. Choi,et al\.\(2020\)PIQA: reasoning about physical commonsense in natural language\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.34,pp\. 7432–7439\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- S\. Borse, S\. Kadambi, N\. P\. Pandey, K\. Bhardwaj, V\. Ganapathy, S\. Priyadarshi, R\. Garrepalli, R\. Esteves, M\. Hayat, and F\. Porikli \(2024\)FouRA: Fourier low\-rank adaptation\.Advances in Neural Information Processing Systems37\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p2.1),[§1](https://arxiv.org/html/2607.00162#S1.p5.2),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px2.p1.2),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2607.00162#S4.SS1.p1.15)\.
- Ç\. Candan, M\. A\. Kutay, and H\. M\. Ozaktas \(2000\)The discrete fractional Fourier transform\.IEEE Transactions on Signal Processing48\(5\),pp\. 1329–1337\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px3.p1.1),[§3](https://arxiv.org/html/2607.00162#S3.SS0.SSS0.Px2.p1.4)\.
- J\. Cao, T\. Lin, B\. Yuan, R\. Yan, H\. He, W\. Zhang, J\. Li, D\. Zhang, S\. Tang, and Y\. Zhuang \(2026\)MoA: heterogeneous mixture of adapters for parameter\-efficient fine\-tuning of large language models\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 21056–21073\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- A\. Chaudhry, N\. Khan, P\. Dokania, and P\. Torr \(2020\)Continual learning in low\-rank orthogonal subspaces\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9900–9911\.Cited by:[Appendix F](https://arxiv.org/html/2607.00162#A6.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.\(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- C\. Clark, K\. Lee, M\. Chang, T\. Kwiatkowski, M\. Collins, and K\. Toutanova \(2019\)BoolQ: exploring the surprising difficulty of natural yes/no questions\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 2924–2936\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? try ARC, the AI2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- D\. Dai, C\. Deng, C\. Zhao, R\. Xu, H\. Gao, D\. Chen, J\. Li, W\. Zeng, X\. Yu, Y\. Wu,et al\.\(2024\)DeepSeekMoE: towards ultimate expert specialization in mixture\-of\-experts language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1280–1297\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1)\.
- S\. Dasgupta, C\. F\. Stevens, and S\. Navlakha \(2017\)A neural algorithm for a fundamental computing problem\.Science358\(6364\),pp\. 793–796\.Cited by:[Appendix F](https://arxiv.org/html/2607.00162#A6.SS0.SSS0.Px2.p1.1)\.
- T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. Zettlemoyer \(2023\)QLoRA: efficient finetuning of quantized LLMs\.Advances in Neural Information Processing Systems36,pp\. 10088–10115\.Cited by:[§1](https://arxiv.org/html/2607.00162#S1.p1.1)\.
- N\. Ding, X\. Lv, Q\. Wang, Y\. Chen, B\. Zhou, Z\. Liu, and M\. Sun \(2023\)Sparse low\-rank adaptation of pre\-trained language models\.arXiv preprint arXiv:2311\.11696\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Dou, E\. Zhou, Y\. Liu, S\. Gao, J\. Zhao, W\. Shen, Y\. Zhou, Z\. Xi, X\. Wang, X\. Fan,et al\.\(2023\)LoRAMoE: alleviate world knowledge forgetting in large language models via MoE\-style plugin\.arXiv preprint arXiv:2312\.09979\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1)\.
- W\. Fedus, B\. Zoph, and N\. Shazeer \(2022\)Switch transformers: scaling to trillion parameter models with simple and efficient sparsity\.Journal of Machine Learning Research23\(120\),pp\. 1–39\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2607.00162#S3.SS0.SSS0.Px1.p1.13),[§4\.3](https://arxiv.org/html/2607.00162#S4.SS3.SSS0.Px1.p1.8)\.
- C\. Gao, K\. Chen, J\. Rao, B\. Sun, R\. Liu, D\. Peng, Y\. Zhang, X\. Guo, J\. Yang, and V\. Subrahmanian \(2024a\)Higher layers need more LoRA experts\.arXiv preprint arXiv:2402\.08562\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Gao, Q\. Wang, A\. Chen, Z\. Liu, B\. Wu, L\. Chen, and J\. Li \(2024b\)Parameter\-efficient fine\-tuning with discrete Fourier transform\.InProceedings of the 41st International Conference on Machine Learning,pp\. 14884–14901\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p2.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px2.p1.2),[§4\.1](https://arxiv.org/html/2607.00162#S4.SS1.p1.15),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021a\)Measuring massive multitask language understanding\.Proceedings of the International Conference on Learning Representations\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021b\)Measuring mathematical problem solving with the MATH dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- N\. Houlsby, A\. Giurgiu, S\. Jastrzebski, B\. Morrone, Q\. De Laroussilhe, A\. Gesmundo, M\. Attariyan, and S\. Gelly \(2019\)Parameter\-efficient transfer learning for NLP\.InInternational Conference on Machine Learning,pp\. 2790–2799\.Cited by:[§1](https://arxiv.org/html/2607.00162#S1.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.International Conference on Learning Representations\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2607.00162#S3.SS0.SSS0.Px1.p1.10),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- Z\. Hu, L\. Wang, Y\. Lan, W\. Xu, E\. Lim, L\. Bing, X\. Xu, S\. Poria, and R\. K\. Lee \(2023\)LLM\-Adapters: an adapter family for parameter\-efficient fine\-tuning of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Cited by:[Appendix C](https://arxiv.org/html/2607.00162#A3.SS0.SSS0.Px1.p1.1)\.
- G\. Ilharco, M\. T\. Ribeiro, M\. Wortsman, S\. Gururangan, L\. Schmidt, H\. Hajishirzi, and A\. Farhadi \(2022\)Editing models with task arithmetic\.arXiv preprint arXiv:2212\.04089\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- R\. A\. Jacobs, M\. I\. Jordan, S\. J\. Nowlan, and G\. E\. Hinton \(1991\)Adaptive mixtures of local experts\.Neural Computation3\(1\),pp\. 79–87\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1)\.
- J\. Jiang, F\. Wang, H\. Qi, S\. Kim, and J\. Tang \(2026\)FourierMoE: Fourier mixture\-of\-experts adaptation of large language models\.arXiv preprint arXiv:2604\.01762\.Cited by:[§1](https://arxiv.org/html/2607.00162#S1.p2.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px2.p1.2),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- R\. Koncel\-Kedziorski, S\. Roy, A\. Amini, N\. Kushman, and H\. Hajishirzi \(2016\)MAWPS: a math word problem repository\.InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 1152–1157\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- D\. J\. Kopiczko, T\. Blankevoort, and Y\. M\. Asano \(2024\)VeRA: vector\-based random matrix adaptation\.InInternational Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1)\.
- D\. Lepikhin, H\. Lee, Y\. Xu, D\. Chen, O\. Firat, Y\. Huang, M\. Krikun, N\. Shazeer, and Z\. Chen \(2020\)GShard: scaling giant models with conditional computation and automatic sharding\.arXiv preprint arXiv:2006\.16668\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1)\.
- B\. Lester, R\. Al\-Rfou, and N\. Constant \(2021\)The power of scale for parameter\-efficient prompt tuning\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,pp\. 3045–3059\.Cited by:[§1](https://arxiv.org/html/2607.00162#S1.p1.1)\.
- D\. Li, Y\. Ma, N\. Wang, Z\. Ye, Z\. Cheng, Y\. Tang, Y\. Zhang, L\. Duan, J\. Zuo, C\. Yang,et al\.\(2024\)MixLoRA: enhancing large language models fine\-tuning with LoRA\-based mixture of experts\.arXiv preprint arXiv:2404\.15159\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- X\. L\. Li and P\. Liang \(2021\)Prefix\-tuning: optimizing continuous prompts for generation\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 4582–4597\.Cited by:[§1](https://arxiv.org/html/2607.00162#S1.p1.1)\.
- M\. Liao, W\. Chen, J\. Shen, S\. Guo, and H\. Wan \(2025\)HMoRA: making LLMs more effective with hierarchical mixture of LoRA experts\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- W\. Ling, D\. Yogatama, C\. Dyer, and P\. Blunsom \(2017\)Program induction by rationale generation: learning to solve and explain algebraic word problems\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- S\. Liu, C\. Wang, H\. Yin, P\. Molchanov, Y\. F\. Wang, K\. Cheng, and M\. Chen \(2024\)DoRA: weight\-decomposed low\-rank adaptation\.InForty\-first International Conference on Machine Learning,Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- P\. Lu, S\. Mishra, T\. Xia, L\. Qiu, K\. Chang, S\. Zhu, O\. Tafjord, P\. Clark, and A\. Kalyan \(2022\)Learn to explain: multimodal reasoning via thought chains for science question answering\.InThe 36th Conference on Neural Information Processing Systems,Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- M\. S\. Matena and C\. A\. Raffel \(2022\)Merging models with Fisher\-weighted averaging\.Advances in Neural Information Processing Systems35,pp\. 17703–17716\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- M\. D\. McDonnell, D\. Gong, A\. Parvaneh, E\. Abbasnejad, and A\. Van den Hengel \(2023\)RanPAC: random projections and pre\-trained models for continual learning\.Advances in Neural Information Processing Systems36,pp\. 12022–12053\.Cited by:[Appendix F](https://arxiv.org/html/2607.00162#A6.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- T\. Mihaylov, P\. Clark, T\. Khot, and A\. Sabharwal \(2018\)Can a suit of armor conduct electricity? a new dataset for open book question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2381–2391\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- V\. Namias \(1980\)The fractional order Fourier transform and its application to quantum mechanics\.IMA Journal of Applied Mathematics25\(3\),pp\. 241–265\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p3.3),[§3](https://arxiv.org/html/2607.00162#S3.SS0.SSS0.Px2.p1.4)\.
- G\. Ortiz\-Jimenez, A\. Favero, and P\. Frossard \(2023\)Task arithmetic in the tangent space: improved editing of pre\-trained models\.Advances in Neural Information Processing Systems36,pp\. 66727–66754\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- H\. M\. Ozaktas, O\. Arıkan, M\. A\. Kutay, and G\. Bozdağı \(1996\)Digital computation of the fractional Fourier transform\.IEEE Transactions on Signal Processing44\(9\),pp\. 2141–2150\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px3.p1.1),[§B\.4](https://arxiv.org/html/2607.00162#A2.SS4.p1.11),[§1](https://arxiv.org/html/2607.00162#S1.p5.2),[§4\.2](https://arxiv.org/html/2607.00162#S4.SS2.p1.4)\.
- A\. Patel, S\. Bhattamishra, and N\. Goyal \(2021\)Are NLP models really able to solve simple math word problems?\.arXiv preprint arXiv:2103\.07191\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- N\. Rahaman, A\. Baratin, D\. Arpit, F\. Draxler, M\. Lin, F\. Hamprecht, Y\. Bengio, and A\. Courville \(2019\)On the spectral bias of neural networks\.InInternational Conference on Machine Learning,pp\. 5301–5310\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px3.p1.1)\.
- P\. Ren, C\. Shi, S\. Wu, M\. Zhang, Z\. Ren, M\. de Rijke, Z\. Chen, and J\. Pei \(2024\)MELoRA: mini\-ensemble low\-rank adapters for parameter\-efficient fine\-tuning\.arXiv preprint arXiv:2402\.17263\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Sakaguchi, R\. Le Bras, C\. Bhagavatula, and Y\. Choi \(2020\)WinoGrande: an adversarial winograd schema challenge at scale\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.34,pp\. 8732–8740\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- M\. Sap, H\. Rashkin, D\. Chen, R\. Le Bras, and Y\. Choi \(2019\)Social IQa: commonsense reasoning about social interactions\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing,pp\. 4463–4473\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- N\. Shazeer, A\. Mirhoseini, K\. Maziarz, A\. Davis, Q\. Le, G\. Hinton, and J\. Dean \(2017\)Outrageously large neural networks: the sparsely\-gated mixture\-of\-experts layer\.arXiv preprint arXiv:1701\.06538\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1)\.
- C\. Tian, Z\. Shi, Z\. Guo, L\. Li, and C\. Xu \(2024\)HydraLoRA: an asymmetric LoRA architecture for efficient fine\-tuning\.Advances in Neural Information Processing Systems37,pp\. 9565–9584\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- X\. Wang, T\. Chen, Q\. Ge, H\. Xia, R\. Bao, R\. Zheng, Q\. Zhang, T\. Gui, and X\. Huang \(2023\)Orthogonal subspace learning for language model continual learning\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 10658–10671\.Cited by:[Appendix F](https://arxiv.org/html/2607.00162#A6.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- X\. Wu, S\. Huang, and F\. Wei \(2024\)Mixture of LoRA experts\.InThe Twelfth International Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2607.00162#S3.SS0.SSS0.Px1.p1.13)\.
- Z\. J\. Xu, Y\. Zhang, T\. Luo, Y\. Xiao, and Z\. Ma \(2020\)Frequency principle: Fourier analysis sheds light on deep neural networks\.Communications in Computational Physics28\(5\),pp\. 1746–1767\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px3.p1.1)\.
- P\. Yadav, D\. Tam, L\. Choshen, C\. A\. Raffel, and M\. Bansal \(2023\)TIES\-merging: resolving interference when merging models\.Advances in Neural Information Processing Systems36,pp\. 7093–7115\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei,et al\.\(2024\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- L\. Yu, B\. Yu, H\. Yu, F\. Huang, and Y\. Li \(2024\)Language models are super mario: absorbing abilities from homologous models as a free lunch\.InForty\-first International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- T\. Zadouri, A\. Üstün, A\. Ahmadian, B\. Ermiş, A\. Locatelli, and S\. Hooker \(2024\)Pushing mixture of experts to the limit: extremely parameter efficient MoE for instruction tuning\.InInternational Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi \(2019\)HellaSwag: can a machine really finish your sentence?\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 4791–4800\.Cited by:[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px1.p1.1)\.
- H\. Zhang and J\. Zhou \(2025\)Unraveling LoRA interference: orthogonal subspaces for robust model merging\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 26459–26472\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- Q\. Zhang, M\. Chen, A\. Bukharin, N\. Karampatziakis, P\. He, Y\. Cheng, W\. Chen, and T\. Zhao \(2023\)AdaLoRA: adaptive budget allocation for parameter\-efficient fine\-tuning\.arXiv preprint arXiv:2303\.10512\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.
- Y\. Zhang, H\. Zhu, J\. Dong, H\. Shi, Z\. Meng, P\. Koniusz, and H\. Yu \(2025\)CrossSpectra: exploiting cross\-layer smoothness for parameter\-efficient fine\-tuning\.InAdvances in Neural Information Processing Systems,Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.00162#S1.p2.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px2.p1.2)\.
- Y\. Zhuang, Y\. Shen, Y\. Bian, Q\. Su, S\. Ji, Y\. Shi, and F\. Miao \(2026\)LD\-MoLE: learnable dynamic routing for mixture of LoRA experts\.arXiv preprint arXiv:2509\.25684\.Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Zou, Y\. Zang, and X\. Ji \(2025a\)Structural features of the fly olfactory circuit mitigate the stability\-plasticity dilemma in continual learning\.arXiv preprint arXiv:2502\.01427\.Cited by:[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Zou, Y\. Zang, W\. Xu, and X\. Ji \(2025b\)Fly\-cl: a fly\-inspired framework for enhancing efficient decorrelation and reduced training time in pre\-trained model\-based continual representation learning\.arXiv preprint arXiv:2510\.16877\.Cited by:[Appendix F](https://arxiv.org/html/2607.00162#A6.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Zou, Y\. Zang, W\. Xu, Y\. Zhu, and X\. Ji \(2025c\)FlyLoRA: boosting task decoupling and parameter efficiency via implicit rank\-wise mixture\-of\-experts\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[Appendix A](https://arxiv.org/html/2607.00162#A1.SS0.SSS0.Px2.p1.1),[Appendix F](https://arxiv.org/html/2607.00162#A6.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00162#S2.SS0.SSS0.Px3.p1.1),[§6\.1](https://arxiv.org/html/2607.00162#S6.SS1.SSS0.Px2.p1.1)\.

## Appendix AExtended Related Work

#### From single\-domain to learned\-domain adaptation\.

The PEFT literature can be read as a search for the right*coordinate system*in which a small update is expressive\. LoRA fixes the canonical \(spatial\) coordinates and constrains the update to be low rank\(Huet al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib2)\); AdaLoRA reallocates rank across layers\(Zhanget al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib6)\); DoRA decomposes magnitude and direction\(Liuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib7)\); VeRA freezes random projections and learns only scalings\(Kopiczkoet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib9)\); and orthogonal or Householder reparameterizations rotate the update within the spatial domain\. Spectral methods change the coordinate system outright: FourierFT learns coefficients in the DFT basis\(Gaoet al\.,[2024b](https://arxiv.org/html/2607.00162#bib.bib25)\), WaveFT in a wavelet basis\(Bilicanet al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib27)\), and FouRA performs the low\-rank projection itself in the Fourier domain, reporting that the resulting bases are decorrelated and merge well\(Borseet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib26)\)\. CrossSpectra exploits the empirical observation that cross\-layer adaptation energy is spectrally smooth\(Zhanget al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib28)\)\.Frameunifies these views: rather than choosing the spatial or a fixed spectral coordinate system, it parameterizes the family of coordinate systems by the fractional order and learns where to sit, per expert and per token\.

#### Mixture\-of\-experts adapters\.

MoE has a long history as a capacity\-scaling mechanism\(Jacobset al\.,[1991](https://arxiv.org/html/2607.00162#bib.bib52); Shazeeret al\.,[2017](https://arxiv.org/html/2607.00162#bib.bib53); Lepikhinet al\.,[2020](https://arxiv.org/html/2607.00162#bib.bib55); Feduset al\.,[2022](https://arxiv.org/html/2607.00162#bib.bib54); Daiet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib56)\), and its adapter incarnations include LoRAMoE\(Douet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib12)\), MixLoRA\(Liet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib13)\), MoLE\(Wuet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib14)\), MoLA\(Gaoet al\.,[2024a](https://arxiv.org/html/2607.00162#bib.bib16)\), HydraLoRA\(Tianet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib15)\), HMoRA\(Liaoet al\.,[2025](https://arxiv.org/html/2607.00162#bib.bib17)\), and extremely parameter\-efficient soft\-merging variants\(Zadouriet al\.,[2024](https://arxiv.org/html/2607.00162#bib.bib20)\)\. Recent work questions the assumption that experts should be*homogeneous*: MoA mixes structurally different adapter types\(Caoet al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib21)\), and LD\-MoLE makes the number of active experts token\- and layer\-dependent through differentiable routing\(Zhuanget al\.,[2026](https://arxiv.org/html/2607.00162#bib.bib22)\)\.Frameis in this heterogeneous\-expert lineage, with domain \(fractional order\) as the axis of heterogeneity, and complements router\-free designs that decorrelate experts through a frozen sparse random projection\(Zouet al\.,[2025c](https://arxiv.org/html/2607.00162#bib.bib24)\)by making the decorrelating transform learnable rather than fixed\.

#### Why fractional Fourier\.

The fractional\-Fourier transform is standard in signal processing for analyzing chirped and non\-stationary signals, precisely because it interpolates between time and frequency representations\(Namias,[1980](https://arxiv.org/html/2607.00162#bib.bib32); Almeida,[1994](https://arxiv.org/html/2607.00162#bib.bib33); Ozaktaset al\.,[1996](https://arxiv.org/html/2607.00162#bib.bib34); Candanet al\.,[2000](https://arxiv.org/html/2607.00162#bib.bib35)\)\. Its relevance to learning is suggested by spectral\-bias theory: neural networks learn low\-frequency structure first\(Rahamanet al\.,[2019](https://arxiv.org/html/2607.00162#bib.bib30); Xuet al\.,[2020](https://arxiv.org/html/2607.00162#bib.bib31)\), so different tasks and layers, which require different frequency content, should prefer different points between the time and frequency domains\. To our knowledgeFrameis the first to use the fractional order as a learnable adaptation domain\.

## Appendix BProofs

Throughout,𝚽a\\bm\{\\Phi\}\_\{a\}is the DFrFT of orderaawith spectral decomposition \([2](https://arxiv.org/html/2607.00162#S3.E2)\),θ=π2​a\\theta=\\tfrac\{\\pi\}\{2\}a, and𝑹a=Re​\{𝚽a\}\\bm\{R\}\_\{a\}=\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\}\\right\\\}\.

### B\.1Proposition[1](https://arxiv.org/html/2607.00162#Thmproposition1)\(Strict generalization\)

Ata=0a\{=\}0we haveθ=0\\theta\{=\}0, so𝚽0=∑m𝒖m​𝒖m⊤=𝑰\\bm\{\\Phi\}\_\{0\}=\\sum\_\{m\}\\bm\{u\}\_\{m\}\\bm\{u\}\_\{m\}^\{\\\!\\top\}=\\bm\{I\}because\{𝒖m\}\\\{\\bm\{u\}\_\{m\}\\\}is an orthonormal basis; hence𝑹0=𝑰\\bm\{R\}\_\{0\}=\\bm\{I\}and the expert update \([3](https://arxiv.org/html/2607.00162#S4.E3)\) isΔ​Wi=Bi​Ai\\Delta W\_\{i\}=B\_\{i\}A\_\{i\}, identical to a LoRA expert\. Ata=1a\{=\}1,θ=π2\\theta\{=\}\\tfrac\{\\pi\}\{2\}and𝚽1=𝑭\\bm\{\\Phi\}\_\{1\}=\\bm\{F\}, the unitary DFT, so𝑹1=Re​\{𝑭\}\\bm\{R\}\_\{1\}=\\mathrm\{Re\}\\\!\\left\\\{\\bm\{F\}\\right\\\}is the real \(cosine\) part and the expert is a cosine\-domain low\-rank adapter of the FourierFT family with the same factor count\. Thus both spatial and Fourier MoE\-LoRA lie inFrame’s hypothesis class\. Strictness forN≥2N\\geq 2: take two experts witha1=0,a2=12a\_\{1\}\{=\}0,a\_\{2\}\{=\}\\tfrac\{1\}\{2\}and rankr<dr\{<\}d\. The realized update set\{B1​A1\+B2​A2​𝑹1/2\}\\\{B\_\{1\}A\_\{1\}\+B\_\{2\}A\_\{2\}\\bm\{R\}\_\{1/2\}\\\}contains matrices of rank up to2​r2rwhose right singular vectors mix the canonical basis and the𝑹1/2\\bm\{R\}\_\{1/2\}\-rotated basis; no single\-order rank\-rr\(or rank\-2​r2rsingle\-domain\) family contains all such matrices because𝑹1/2\\bm\{R\}\_\{1/2\}is not a permutation of the identity eigenbasis\.∎

### B\.2Proposition[2](https://arxiv.org/html/2607.00162#Thmproposition2)\(Bounded order gradients\)

Differentiating \([2](https://arxiv.org/html/2607.00162#S3.E2)\) inaaand usingθ=π2​a\\theta=\\tfrac\{\\pi\}\{2\}a,

∂𝚽a∂a=−i​π2​∑mm​e−i​m​θ​𝒖m​𝒖m⊤\.\\frac\{\\partial\\bm\{\\Phi\}\_\{a\}\}\{\\partial a\}=\-\\mathrm\{i\}\\frac\{\\pi\}\{2\}\\sum\_\{m\}m\\,e^\{\-\\mathrm\{i\}m\\theta\}\\bm\{u\}\_\{m\}\\bm\{u\}\_\{m\}^\{\\\!\\top\}\.\(9\)This is a normal operator with eigenvalues\{−i​π2​m​e−i​m​θ\}\\\{\-\\mathrm\{i\}\\tfrac\{\\pi\}\{2\}m\\,e^\{\-\\mathrm\{i\}m\\theta\}\\\}, so its spectral norm isπ2​maxm⁡\|m\|\\tfrac\{\\pi\}\{2\}\\max\_\{m\}\|m\|\. Restricting to the rank\-ρ\\rhosubspacespan​\{𝒖m\}m<ρ\\mathrm\{span\}\\\{\\bm\{u\}\_\{m\}\\\}\_\{m<\\rho\}gives‖∂𝚽a/∂a‖2≤π2​\(ρ−1\)\\big\\\|\\partial\\bm\{\\Phi\}\_\{a\}/\\partial a\\big\\\|\_\{2\}\\leq\\tfrac\{\\pi\}\{2\}\(\\rho\-1\)\. Taking the real part is11\-Lipschitz, so the same bound holds for𝑹a\\bm\{R\}\_\{a\}\. For the per\-token lossℒ\\mathcal\{L\}withy=⋯\+αr​gi​Bi​Ai​𝑹ai​xy=\\cdots\+\\tfrac\{\\alpha\}\{r\}g\_\{i\}B\_\{i\}A\_\{i\}\\bm\{R\}\_\{a\_\{i\}\}x,

\|∂ℒ∂ai\|\\displaystyle\\Big\|\\tfrac\{\\partial\\mathcal\{L\}\}\{\\partial a\_\{i\}\}\\Big\|=\|⟨∇yℒ,αr​gi​Bi​Ai​∂𝑹ai∂ai​x⟩\|\\displaystyle=\\Big\|\\langle\\nabla\_\{y\}\\mathcal\{L\},\\tfrac\{\\alpha\}\{r\}g\_\{i\}B\_\{i\}A\_\{i\}\\tfrac\{\\partial\\bm\{R\}\_\{a\_\{i\}\}\}\{\\partial a\_\{i\}\}x\\rangle\\Big\|≤αr​\|gi\|​‖∇yℒ‖2​‖Bi‖2​‖Ai‖2​π2​\(ρ−1\)​‖x‖2,\\displaystyle\\leq\\tfrac\{\\alpha\}\{r\}\|g\_\{i\}\|\\\|\\nabla\_\{y\}\\mathcal\{L\}\\\|\_\{2\}\\\|B\_\{i\}\\\|\_\{2\}\\\|A\_\{i\}\\\|\_\{2\}\\tfrac\{\\pi\}\{2\}\(\\rho\{\-\}1\)\\\|x\\\|\_\{2\},\(10\)by Cauchy–Schwarz and submultiplicativity, with\|gi\|≤1\|g\_\{i\}\|\\leq 1\. Hence the gradient is bounded and training is stable under a step sizeηa≤1/La\\eta\_\{a\}\\leq 1/L\_\{a\}withLaL\_\{a\}the above bound\.∎

### B\.3Proposition[3](https://arxiv.org/html/2607.00162#Thmproposition3)\(Domain\-diversity decorrelation\)

LetΔ​Wi=Bi​Ai​𝑹ai\\Delta W\_\{i\}=B\_\{i\}A\_\{i\}\\bm\{R\}\_\{a\_\{i\}\}with independent factors,𝔼​\[Ai⊤​Ai\]=1d​𝑰\\mathbb\{E\}\[A\_\{i\}^\{\\\!\\top\}A\_\{i\}\]=\\tfrac\{1\}\{d\}\\bm\{I\}, and likewise forBiB\_\{i\}normalized so𝔼​‖Δ​Wi‖F2=1\\mathbb\{E\}\\\|\\Delta W\_\{i\}\\\|\_\{\\mathrm\{F\}\}^\{2\}=1\. Using index additivity,𝑹ai​𝑹aj⊤=Re​\{𝚽ai\}​Re​\{𝚽aj\}⊤\\bm\{R\}\_\{a\_\{i\}\}\\bm\{R\}\_\{a\_\{j\}\}^\{\\\!\\top\}=\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\}\\right\\\}\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{j\}\}\\right\\\}^\{\\\!\\top\}, and expanding the real parts,

𝑹ai𝑹aj⊤=12Re\{𝚽ai−aj\}\+12Re\{𝚽ai\+aj\},†\\bm\{R\}\_\{a\_\{i\}\}\\bm\{R\}\_\{a\_\{j\}\}^\{\\\!\\top\}=\\tfrac\{1\}\{2\}\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\-a\_\{j\}\}\\right\\\}\+\\tfrac\{1\}\{2\}\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\+a\_\{j\}\}\\right\\\}\\\!\\\!\\;\{\}^\{\\dagger\},\(11\)where the first term depends only on the order*difference*and the second is an oscillatory chirp term that averages out\. Taking expectations over the independent factors,

𝔼​⟨Δ​Wi,Δ​Wj⟩F\\displaystyle\\mathbb\{E\}\\langle\\Delta W\_\{i\},\\Delta W\_\{j\}\\rangle\_\{\\mathrm\{F\}\}=𝔼​tr​\(𝑹aj​Aj⊤​Bj⊤​Bi​Ai​𝑹ai⊤\)\\displaystyle=\\mathbb\{E\}\\,\\mathrm\{tr\}\\\!\\big\(\\bm\{R\}\_\{a\_\{j\}\}A\_\{j\}^\{\\\!\\top\}B\_\{j\}^\{\\\!\\top\}B\_\{i\}A\_\{i\}\\bm\{R\}\_\{a\_\{i\}\}^\{\\\!\\top\}\\big\)\(12\)=1d​tr​\(𝑹ai​𝑹aj⊤\)​𝔼​tr​\(Bj⊤​Bi\)​cr,\\displaystyle=\\tfrac\{1\}\{d\}\\mathrm\{tr\}\\\!\\big\(\\bm\{R\}\_\{a\_\{i\}\}\\bm\{R\}\_\{a\_\{j\}\}^\{\\\!\\top\}\\big\)\\mathbb\{E\}\\,\\mathrm\{tr\}\(B\_\{j\}^\{\\\!\\top\}B\_\{i\}\)\\,c\_\{r\},\(13\)and the dominant term is proportional totr​Re​\{𝚽ai−aj\}\\mathrm\{tr\}\\,\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\-a\_\{j\}\}\\right\\\}\. Normalizing and applying Cauchy–Schwarz over therrshared directions gives

𝔼​\[⟨Δ​Wi,Δ​Wj⟩F‖Δ​Wi‖F​‖Δ​Wj‖F\]2≤1r​1d​‖Re​\{𝚽ai−aj\}‖F2⏟κ​\(\|ai−aj\|\)\.\\mathbb\{E\}\\\!\\left\[\\frac\{\\langle\\Delta W\_\{i\},\\Delta W\_\{j\}\\rangle\_\{\\mathrm\{F\}\}\}\{\\\|\\Delta W\_\{i\}\\\|\_\{\\mathrm\{F\}\}\\\|\\Delta W\_\{j\}\\\|\_\{\\mathrm\{F\}\}\}\\right\]^\{2\}\\leq\\frac\{1\}\{r\}\\,\\underbrace\{\\tfrac\{1\}\{d\}\\\|\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{a\_\{i\}\-a\_\{j\}\}\\right\\\}\\\|\_\{\\mathrm\{F\}\}^\{2\}\}\_\{\\kappa\(\|a\_\{i\}\-a\_\{j\}\|\)\}\.\(14\)Finallyκ​\(0\)=1d​‖𝑰‖F2=1\\kappa\(0\)=\\tfrac\{1\}\{d\}\\\|\\bm\{I\}\\\|\_\{\\mathrm\{F\}\}^\{2\}=1, and forδ∈\(0,1\]\\delta\\in\(0,1\]the energy ofRe​\{𝚽δ\}\\mathrm\{Re\}\\\!\\left\\\{\\bm\{\\Phi\}\_\{\\delta\}\\right\\\}spreads off the diagonal asδ\\deltagrows \(the eigenphasese−i​m​θe^\{\-\\mathrm\{i\}m\\theta\}dephase\), soκ\\kappais decreasing on\[0,1\]\[0,1\]withκ​\(1\)=1d​‖Re​\{𝑭\}‖F2\\kappa\(1\)=\\tfrac\{1\}\{d\}\\\|\\mathrm\{Re\}\\\!\\left\\\{\\bm\{F\}\\right\\\}\\\|\_\{\\mathrm\{F\}\}^\{2\}\.∎

### B\.4Proposition[4](https://arxiv.org/html/2607.00162#Thmproposition4)\(Faithful surrogate and complexity\)

The Ozaktas decomposition writes the continuous FrFT as a chirp multiplication, a Fourier transform, and a second chirp multiplication; sampling at the Nyquist rate of the chirp\-modulated signal yields \([6](https://arxiv.org/html/2607.00162#S4.E6)\) with discretization error𝒪​\(d−1\)\\mathcal\{O\}\(d^\{\-1\}\)in the band\-limited regime\(Ozaktaset al\.,[1996](https://arxiv.org/html/2607.00162#bib.bib34)\)\. The cost is two𝒪​\(d\)\\mathcal\{O\}\(d\)Hadamard products and one𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)FFT, hence𝒪​\(d​log⁡d\)\\mathcal\{O\}\(d\\log d\)time and𝒪​\(d\)\\mathcal\{O\}\(d\)memory, against𝒪​\(d2\)\\mathcal\{O\}\(d^\{2\}\)for the dense matrix in \([2](https://arxiv.org/html/2607.00162#S3.E2)\)\. The parameter accounting follows by counting active factors:kkexperts each contributingr​\(d\+dout\)r\(d\+d\_\{\\mathrm\{out\}\}\),kkorder scalars, and theN​dNdrouter\.∎

## Appendix CExperimental Details

#### Data and evaluation\.

For commonsense reasoning we fine\-tune on the merged training split following the LLM\-Adapters protocol and report test accuracy per dataset\(Huet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib11)\)\. For mathematical reasoning we train on the combined GSM8K/MATH/SVAMP/MAWPS/AQuA training questions and report exact\-match accuracy; MATH is evaluated on the held\-out test subset\. For code we report pass@1 on HumanEval and MBPP with greedy decoding\. For knowledge we report accuracy on MMLU \(zero\-shot\) and ScienceQA\. All evaluations use the same prompts across methods\.

#### Hyperparameters\.

Table[4](https://arxiv.org/html/2607.00162#A3.T4)lists the configuration\. We tune only theFrame\-specific quantities \(NN,kk,GG,ηa\\eta\_\{a\}\) lightly on a held\-out commonsense split and keep them fixed across task families and backbones; baselines use their recommended settings at matched rank\. We use AdamW, cosine decay,33epochs, sequence length512512\(math/code/knowledge\) or256256\(commonsense\), and bfloat16\.

Table 4:DefaultFramehyperparameters\.
#### Hardware\.

Experiments run on8×8\{\\times\}NVIDIA A100 80GB GPUs\. A commonsense run for an 8B backbone takes roughly55GPU\-hours; the math/code/knowledge mixture takes roughly99GPU\-hours\. The chirp–FFT surrogate is implemented with the framework FFT and adds a single complex multiply on either side\.

## Appendix DAdditional Results

#### Per\-seed variation\.

Table[5](https://arxiv.org/html/2607.00162#A4.T5)reports mean and standard deviation over three seeds for the commonsense average;Frame’s improvements exceed one standard deviation for both backbones\.

Table 5:Commonsense average over three seeds \(mean±\\pmstd\)\.
#### Additional backbones\.

Table[6](https://arxiv.org/html/2607.00162#A4.T6)reports the commonsense average across backbone scales, supporting Figure[5](https://arxiv.org/html/2607.00162#S6.F5)\(c\): the gain over the best baseline is stable from 1\.5B to 14B\.

Table 6:Commonsense average across model scales \(Qwen2\.5 family for 1\.5B/3B/14B, LLaMA\-3\.1\-8B for 8B\)\. “Best baseline” is the strongest non\-Framemethod at each scale\.
#### Number of experts and activekk\.

Table[7](https://arxiv.org/html/2607.00162#A4.T7)sweepsNNandkk\. Accuracy rises withNNand saturates aroundN=8N\{=\}8;k=2k\{=\}2is the best accuracy/cost point, withk=1k\{=\}1underfitting andk=4k\{=\}4adding cost without commensurate gain\.

Table 7:Effect of expert countNNand active expertskkon LLaMA\-3\.1\-8B commonsense\. Relative time is normalized to the default\.
#### Method properties\.

Table[8](https://arxiv.org/html/2607.00162#A4.T8)situatesFrameamong representative baselines along the axes that matter for our claims\.

Table 8:Frameis the only method whose adaptation*domain*is a learnable, per\-expert quantity on the spatial–spectral continuum\.

## Appendix ETraining\-Free Merging

To test whether domain diversity aids composition, we train single\-taskFrameadapters independently and merge them by averaging their updates, then evaluate post\-merge retention \(the fraction of single\-task accuracy preserved\)\. Table[9](https://arxiv.org/html/2607.00162#A5.T9)showsFrameretains the most among the compared baselines\. This matches Proposition[3](https://arxiv.org/html/2607.00162#Thmproposition3): experts placed at diverse orders interfere less, so their sum is closer to the concatenation of the individual updates\.

Table 9:Training\-free merging of four single\-task adapters on LLaMA\-3\.1\-8B: post\-merge retention \(%\)\. Higher is better\.
## Appendix FDiscussion

#### Why a learned domain helps\.

Three forces combine\.*Compaction*: tasks whose update energy concentrates at intermediate time–frequency tilt are represented more compactly at the matching order, so a fixed basis wastes rank \(Figure[1](https://arxiv.org/html/2607.00162#S1.F1)\)\.*Diversity*: a mixture spanning orders covers more of the update space than the same number of single\-domain experts \(Proposition[1](https://arxiv.org/html/2607.00162#Thmproposition1)\)\.*Decorrelation*: orders that are far apart are incoherent \(Proposition[3](https://arxiv.org/html/2607.00162#Thmproposition3)\), so experts specialize without redundancy and compose better\. The first is about a single expert; the second and third are about the mixture\.

#### Relation to projection\-based decorrelation\.

Sparse or random projections decorrelate updates by mapping them into a high\-dimensional space and keeping only the strongest responses\(Dasguptaet al\.,[2017](https://arxiv.org/html/2607.00162#bib.bib45); McDonnellet al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib51); Zouet al\.,[2025c](https://arxiv.org/html/2607.00162#bib.bib24),[b](https://arxiv.org/html/2607.00162#bib.bib50)\), while orthogonal\-subspace methods impose decorrelation by construction\(Chaudhryet al\.,[2020](https://arxiv.org/html/2607.00162#bib.bib44); Wanget al\.,[2023](https://arxiv.org/html/2607.00162#bib.bib43)\)\.Frameobtains decorrelation through a structured,*learnable*transform instead: the order plays the role a fixed projection plays in those methods, but it is differentiable, interpretable, and invertible\. An interesting hybrid, left to future work, would combine random projections*and*fractional orders\.

#### Beyond adapters\.

Nothing inFrameis specific to LoRA factors\. The fractional order is a property of the*domain*and could equally parameterize a full\-rank update, a per\-head attention reweighting, or a continual\-learning module that places new tasks at fresh orders to minimize interference with old ones\. The bounded\-gradient and decorrelation results carry over unchanged, suggesting the fractional domain is a general tool for controlling interference in adaptation\.

#### Failure modes\.

When a task is genuinely well served by a single domain \(e\.g\. a purely spatial edit\),Frameshould and does collapse most experts to that order; the cost is then the unused order scalars and the FFTs, which is small but non\-zero\. When routing collapses, grouped balancing reactivates idle bands, but an adversarial data mixture could still starve a band; monitoring per\-band utilization \(Figure[4](https://arxiv.org/html/2607.00162#S6.F4)c\) is a cheap diagnostic\.