MorphStrata: Layer-Specific Perturbations for Generating Morphence Students in Time-Series Moving Target Defense

# Layer-Specific Perturbations for Generating Morphence Students in Time-Series Moving Target Defense
Source: [https://arxiv.org/html/2606.17435](https://arxiv.org/html/2606.17435)
Arnav DoshiAnusri NagarajanThanh Quynh Nhu TaMohammad MasumRobert ChunJaydip SenSaptarshi Sengupta

###### Abstract

Time\-series forecasting models remain vulnerable to gradient\-based adversarial attacks, while existing defense mechanisms typically incur a trade\-off in robustness for bounded response and compute cost\. The problem is pronounced in Moving Target Defense, where maintaining multiple randomized model instances substantially exacerbates the training overhead\. In this work, we introduce MorphStrata, a student generation strategy with selective, layer\-specific stochastic noise injection that extends the traditional Morphence defense\. MorphStrata uses a Transformer backbone as the teacher and perturbs randomly selected architectural blocks to create structured heterogeneity across student models in response to varied data distributions and threat models\. We evaluate against vanilla Transformer and Morphence backbones on a suite of benchmarks including the Jena Climate \(JENA\), Electricity Load Diagrams \(ECL\), and Appliances Energy Prediction \(AEP\) using FGSM, BIM and PGD attacks across multiple attack strengths\. Across datasets and attack regimes, the proposed ensemble maintains comparable adversarial RMSE\. Specifically, for high entropy, periodic datasets as in the case of the AEP data, MorphStrata achieves the lowest RMSE across all attacks and perturbation budgets, improving over the static baseline by up to 24\.11% and 97\.97% under FGSM and BIM respectively at an epsilon value of 0\.5 over 30 randomized trials\. Targeting the layers to generate MorphStrata students accounts for less than 1% increase in train\-times over the Morphence MTD baseline for most of the experiments, while accounting for double digit gains in adversarial RMSE reduction\. From the experiments, we also observe a positive correlation between higher pairwise L2 distance \(among generated students\) and overall defense effectiveness\. In summary, MorphStrata maintains adversarial robustness as an MTD defense at marginal cost deltas when compared to existing baselines\.

Moving Target Defense, Adversarial Robustness, Time\-Series Forecasting, Transformers, Machine Learning

## 1Introduction

Modern time\-series forecasting \(TSF\) increasingly relies on high\-capacity Transformer architectures to model complex temporal dependencies\(Vaswaniet al\.,[2017](https://arxiv.org/html/2606.17435#bib.bib14); Wenet al\.,[2022](https://arxiv.org/html/2606.17435#bib.bib17); Nieet al\.,[2022](https://arxiv.org/html/2606.17435#bib.bib15)\)\. These models remain vulnerable to gradient\-based adversarial perturbations; small input changes can produce large forecasting errors\(Goodfellowet al\.,[2015](https://arxiv.org/html/2606.17435#bib.bib1); Madryet al\.,[2018](https://arxiv.org/html/2606.17435#bib.bib3); Govindarajuluet al\.,[2023](https://arxiv.org/html/2606.17435#bib.bib12)\)\. In regression settings like TSF, errors can accumulate across the prediction horizon rather than corrupting a single label, making this threat especially consequential in energy forecasting and industrial monitoring\(Siddiquiet al\.,[2020](https://arxiv.org/html/2606.17435#bib.bib13); Liuet al\.,[2022](https://arxiv.org/html/2606.17435#bib.bib23)\)\.

Moving Target Defense \(MTD\) reduces attacker reliability by swapping the exposed model dynamically, preventing an adversary from optimizing against a fixed set of parameters\(Amich and Eshete,[2021](https://arxiv.org/html/2606.17435#bib.bib10)\)\. Morphence instantiates this idea for deep networks by generating a pool of student models from a trained base through controlled Gaussian perturbation, then sampling among them at inference\. Its student generation, however, is architecture\-agnostic\. For Transformer\-based forecasting this raises a natural question: does the*location*of parameter perturbation matter for robustness, statistical heterogeneity, and cost?

MorphStrata answers this by introducing layer\-specific student generation inspired by Morphence’s MTD formulation\. Rather than perturbing all parameters indiscriminately, MorphStrata targets distinct Transformer components \(attention, feed\-forward, normalization\) through binary masking, creating structured heterogeneity across the student pool\. The goal is not to assert uniform superiority; the comparison with the Vanilla Ensemble is deliberately dataset\-dependent and attack\-dependent\. Instead, we treat perturbation location as a design lever that reshapes the robustness\-cost trade\-off in ways that aggregate perturbation cannot\.

Contributions\.

- •We introduce MorphStrata, a layer\-specific MTD framework for Transformer\-based time\-series forecasting\.
- •We conduct a trade\-off focused evaluation covering adversarial RMSE, statistical ensemble heterogeneity \(pairwise L2 distance and differential immunity\(Senguptaet al\.,[2019](https://arxiv.org/html/2606.17435#bib.bib35)\)\), and computational overhead across FGSM, BIM, and PGD attacks\.
- •We show that higher pairwise L2 distance among students correlates positively with defense effectiveness, particularly on AEP, where MorphStrata achieves the largest weight diversity gains\.
- •Layer targeting adds less than 1\.1% wall clock overhead over the Vanilla Ensemble across all nine dataset\-attack conditions, making MorphStrata a nearly zero\-overhead extension for deployments already committed to MTD\.

## 2Related Work

Adversarial attacks on TSF have been studied through both untargeted and targeted formulations\(Govindarajuluet al\.,[2023](https://arxiv.org/html/2606.17435#bib.bib12); Siddiquiet al\.,[2020](https://arxiv.org/html/2606.17435#bib.bib13); Liuet al\.,[2022](https://arxiv.org/html/2606.17435#bib.bib23); Krishanet al\.,[2024](https://arxiv.org/html/2606.17435#bib.bib33)\)\. FGSM\(Goodfellowet al\.,[2015](https://arxiv.org/html/2606.17435#bib.bib1)\)and its iterative variants BIM\(Kurakinet al\.,[2017](https://arxiv.org/html/2606.17435#bib.bib2)\)and PGD\(Madryet al\.,[2018](https://arxiv.org/html/2606.17435#bib.bib3)\)are the dominant white\-box threat models\. Defenses span adversarial training\(Goodfellowet al\.,[2015](https://arxiv.org/html/2606.17435#bib.bib1); Madryet al\.,[2018](https://arxiv.org/html/2606.17435#bib.bib3)\), detection\(Meng and Chen,[2017](https://arxiv.org/html/2606.17435#bib.bib6); Zhaoet al\.,[2018](https://arxiv.org/html/2606.17435#bib.bib5)\), and ensemble\-based inference\(Lakshminarayananet al\.,[2017](https://arxiv.org/html/2606.17435#bib.bib29); Gal and Ghahramani,[2016](https://arxiv.org/html/2606.17435#bib.bib28); Cohenet al\.,[2019](https://arxiv.org/html/2606.17435#bib.bib27)\)\. MTD\-based defenses use model switching to degrade attack transferability; Morphence\(Amich and Eshete,[2021](https://arxiv.org/html/2606.17435#bib.bib10)\)and its extension Morphence 2\.0\(Awadet al\.,[2022](https://arxiv.org/html/2606.17435#bib.bib11)\)are the closest prior work\. MTDeep\(Senguptaet al\.,[2019](https://arxiv.org/html/2606.17435#bib.bib35)\)formalizes differential immunity as a measure of transferability resistance in MTD ensembles; we adapt this metric to the regression setting\. Stronger forecasting accuracy does not imply adversarial robustness\(Chenget al\.,[2024](https://arxiv.org/html/2606.17435#bib.bib24); Zhanget al\.,[2025](https://arxiv.org/html/2606.17435#bib.bib25)\), motivating defense\-specific evaluation that goes beyond clean RMSE\.

## 3Preliminaries

Forecasting\.Let𝐱1:T\\mathbf\{x\}\_\{1:T\}be a multivariate time series withddvariables\. A Transformerfθ:ℝT×d→ℝH×d′f\_\{\\theta\}:\\mathbb\{R\}^\{T\\times d\}\\to\\mathbb\{R\}^\{H\\times d^\{\\prime\}\}maps a lookback window to a future horizonHH\. Performance is measured by RMSE\.

Threat model\.A white\-box attacker constructs𝐱adv=𝐱\+𝜹\\mathbf\{x\}\_\{\\mathrm\{adv\}\}=\\mathbf\{x\}\+\\boldsymbol\{\\delta\}with‖𝜹‖∞≤ϵ\\\|\\boldsymbol\{\\delta\}\\\|\_\{\\infty\}\\leq\\epsilonto maximize forecasting loss\. We evaluate FGSM, BIM, and PGD acrossϵ∈\{0\.1,0\.2,0\.3,0\.5\}\\epsilon\\in\\\{0\.1,0\.2,0\.3,0\.5\\\}\.

MTD formulation\.A static model exposes one fixedfθf\_\{\\theta\}at every inference call\. MTD instead samples𝐲^=fθt​\(𝐱\),θt∼p​\(θ\)\\hat\{\\mathbf\{y\}\}=f\_\{\\theta\_\{t\}\}\(\\mathbf\{x\}\),\\ \\theta\_\{t\}\\sim p\(\\theta\), so the attacker must optimize in expectation over the pool:

𝜹∗=arg⁡max‖𝜹‖∞≤ϵ⁡𝔼θt∼p​\(θ\)​\[ℒ​\(fθt​\(𝐱\+𝜹\),𝐲\)\]\.\\boldsymbol\{\\delta\}^\{\*\}=\\arg\\max\_\{\\\|\\boldsymbol\{\\delta\}\\\|\_\{\\infty\}\\leq\\epsilon\}\\,\\mathbb\{E\}\_\{\\theta\_\{t\}\\sim p\(\\theta\)\}\\\!\\left\[\\mathcal\{L\}\(f\_\{\\theta\_\{t\}\}\(\\mathbf\{x\}\+\\boldsymbol\{\\delta\}\),\\mathbf\{y\}\)\\right\]\.\(1\)Morphence baseline\.Given base parametersθ\\theta, Morphence generates studentkkasθ\(k\)=θ\+𝜼\(k\)\\theta^\{\(k\)\}=\\theta\+\\boldsymbol\{\\eta\}^\{\(k\)\},𝜼\(k\)∼𝒩​\(𝟎,σ2​𝐈\)\\boldsymbol\{\\eta\}^\{\(k\)\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\sigma^\{2\}\\mathbf\{I\}\), perturbing the full parameter vector without architectural awareness\.

BaseTransformerClean trainingon time\-series dataInput ProjectionInput embedding / projectionAttentionQ, K, V projectionsFFNFeed\-forward weightsLayerNormScale / shift parametersOutput HeadForecast regression headMasked Gaussianperturbation𝜼\(i\)∼𝒩​\(𝟎,σ2​𝐈\)\\boldsymbol\{\\eta\}^\{\(i\)\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\sigma^\{2\}\\mathbf\{I\}\)θi=θ\+𝐦ki⊙𝜼\(i\)\\theta\_\{i\}=\\theta\+\\mathbf\{m\}\_\{k\_\{i\}\}\\odot\\boldsymbol\{\\eta\}^\{\(i\)\}MorphStratastudent poolℱMS=\{fθi\}i=1Ns\\mathcal\{F\}\_\{\\mathrm\{MS\}\}=\\\{f\_\{\\theta\_\{i\}\}\\\}\_\{i=1\}^\{N\_\{s\}\}Attack\-specificstudent trainingFGSM / BIM / PGDadversarial trainingMTD inferenceSampleMMstudentsfrom trained poolEnsemble prediction𝐲^t=1M​∑m=1Mfθ\(im\)​\(𝐗t\)\\hat\{\\mathbf\{y\}\}\_\{t\}=\\dfrac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}f\_\{\\theta^\{\(i\_\{m\}\)\}\}\(\\mathbf\{X\}\_\{t\}\)Forecast &evaluationRMSE, statistical heterogeneity,stability, resource metrics

Figure 1:MorphStrata pipeline\. A cleanly trained base Transformer is used to generate student models by applying masked Gaussian perturbations to selected parameter strata, including input projection, attention, feed\-forward, LayerNorm, and output\-head components\. The resulting student pool is adversarially trained under FGSM, BIM, and PGD\. During moving target defense \(MTD\) inference,MMstudents are stochastically sampled from the trained pool, their predictions are averaged, and forecasting performance is evaluated using RMSE, statistical heterogeneity, stability, and resource metrics\.
## 4Method

### 4\.1MorphStrata Student Generation

Let\{𝒮k\}k=1K\\\{\\mathcal\{S\}\_\{k\}\\\}\_\{k=1\}^\{K\}denote parameter strata corresponding to distinct Transformer components: self\-attention projections, feed\-forward weights, normalization parameters, input projection, and output head\. For studentii, MorphStrata selects stratum𝒮ki\\mathcal\{S\}\_\{k\_\{i\}\}and applies a binary maskmki,j=𝟙​\[j∈𝒮ki\]m\_\{k\_\{i\},j\}=\\mathbbm\{1\}\[j\\in\\mathcal\{S\}\_\{k\_\{i\}\}\], giving:

θms\(i\)=θ\+𝐦ki⊙𝜼\(i\),𝜼\(i\)∼𝒩​\(𝟎,σ2​𝐈\)\.\\theta^\{\(i\)\}\_\{\\mathrm\{ms\}\}=\\theta\+\\mathbf\{m\}\_\{k\_\{i\}\}\\odot\\boldsymbol\{\\eta\}^\{\(i\)\},\\quad\\boldsymbol\{\\eta\}^\{\(i\)\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\sigma^\{2\}\\mathbf\{I\}\)\.\(2\)Confining noise to a single functional region produces structurally distinct students; each one is perturbed in a different part of the computational graph\. The vanilla baseline applies the same Gaussian noise globally with no mask, soθvan\(i\)=θ\+𝜼\(i\)\\theta^\{\(i\)\}\_\{\\mathrm\{van\}\}=\\theta\+\\boldsymbol\{\\eta\}^\{\(i\)\}\.

### 4\.2Adversarial Training and Inference

Both vanilla and MorphStrata students are adversarially fine\-tuned after generation:

minθs⁡𝔼\(𝐗,𝐲\)∼𝒟​\[ℒ​\(fθs​\(𝐗adv\),𝐲\)\]\.\\min\_\{\\theta\_\{s\}\}\\,\\mathbb\{E\}\_\{\(\\mathbf\{X\},\\mathbf\{y\}\)\\sim\\mathcal\{D\}\}\\\!\\left\[\\mathcal\{L\}\(f\_\{\\theta\_\{s\}\}\(\\mathbf\{X\}^\{\\mathrm\{adv\}\}\),\\mathbf\{y\}\)\\right\]\.\(3\)We maintain a fixed pool \(no repeated pool regeneration, unlike Morphence\), which isolates the effect of generation strategy\. At inference,MMstudents are sampled uniformly from the pool and their predictions are averaged:

𝐲^t=1M​∑f∈ℰtf​\(𝐗t\),ℰt⊂ℱ,\|ℰt\|=M\.\\hat\{\\mathbf\{y\}\}\_\{t\}=\\frac\{1\}\{M\}\\sum\_\{f\\in\\mathcal\{E\}\_\{t\}\}f\(\\mathbf\{X\}\_\{t\}\),\\quad\\mathcal\{E\}\_\{t\}\\subset\\mathcal\{F\},\\quad\|\\mathcal\{E\}\_\{t\}\|=M\.\(4\)

### 4\.3Algorithmic Summary

Algorithm[1](https://arxiv.org/html/2606.17435#alg1)summarizes MorphStrata student generation and evaluation\. The strata\{𝒮k\}k=1K\\\{\\mathcal\{S\}\_\{k\}\\\}\_\{k=1\}^\{K\}correspond to Transformer parameter groups such as attention projections, feed\-forward layers, normalization parameters, input projections, and output heads\. Students are generated from a common base model, adversarially fine\-tuned, and evaluated as a fixed pool without repeated regeneration\.

Algorithm 1MorphStrata Student Generation and Evaluation0:Base model

fθf\_\{\\theta\}, strata

\{𝒮k\}k=1K\\\{\\mathcal\{S\}\_\{k\}\\\}\_\{k=1\}^\{K\}, noise scale

σ\\sigma, attacks

𝒜\\mathcal\{A\}, student count

NsN\_\{s\}, ensemble size

MM
1:

ℱ0←∅\\mathcal\{F\}\_\{0\}\\leftarrow\\emptyset
2:for

i=1,…,Nsi=1,\\dots,N\_\{s\}do

3:Select stratum

𝒮ki\\mathcal\{S\}\_\{k\_\{i\}\}
4:Sample

𝜼\(i\)∼𝒩​\(𝟎,σ2​𝐈\)\\boldsymbol\{\\eta\}^\{\(i\)\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\sigma^\{2\}\\mathbf\{I\}\)
5:Define mask

mki,j=𝟙​\[j∈𝒮ki\]m\_\{k\_\{i\},j\}=\\mathbbm\{1\}\[j\\in\\mathcal\{S\}\_\{k\_\{i\}\}\]
6:Initialize

θi←θ\+𝐦ki⊙𝜼\(i\)\\theta\_\{i\}\\leftarrow\\theta\+\\mathbf\{m\}\_\{k\_\{i\}\}\\odot\\boldsymbol\{\\eta\}^\{\(i\)\}
7:

ℱ0←ℱ0∪\{fθi\}\\mathcal\{F\}\_\{0\}\\leftarrow\\mathcal\{F\}\_\{0\}\\cup\\\{f\_\{\\theta\_\{i\}\}\\\}
8:endfor

9:for

a∈𝒜a\\in\\mathcal\{A\}do

10:

ℱa←∅\\mathcal\{F\}\_\{a\}\\leftarrow\\emptyset
11:foreach

fθi∈ℱ0f\_\{\\theta\_\{i\}\}\\in\\mathcal\{F\}\_\{0\}do

12:Fine\-tune

fθif\_\{\\theta\_\{i\}\}using adversarial examples generated by attack

aa
13:Add trained student

fθiaf\_\{\\theta\_\{i\}^\{a\}\}to

ℱa\\mathcal\{F\}\_\{a\}
14:endfor

15:foreach test input

𝐗t\\mathbf\{X\}\_\{t\}do

16:Sample

ℰt⊂ℱa\\mathcal\{E\}\_\{t\}\\subset\\mathcal\{F\}\_\{a\}uniformly, with

\|ℰt\|=M\|\\mathcal\{E\}\_\{t\}\|=M
17:

𝐲^t←M−1​∑f∈ℰtf​\(𝐗t\)\\hat\{\\mathbf\{y\}\}\_\{t\}\\leftarrow M^\{\-1\}\\sum\_\{f\\in\\mathcal\{E\}\_\{t\}\}f\(\\mathbf\{X\}\_\{t\}\)
18:endfor

19:Compute RMSE, statistical heterogeneity, stability, transferability, and resource metrics for attack

aa
20:endfor

## 5Experiments

Datasets\.We evaluate on three multivariate forecasting benchmarks: Jena Climate \(JENA, 60\-min weather\), Electricity Load Diagrams \(ECL, 15\-min load\), and Appliances Energy Prediction \(AEP, 10\-min residential energy\)\. All splits are chronological; scaling is fit on the training partition only\. Detailed pipelines and temporal structure analysis are in Appendix[B](https://arxiv.org/html/2606.17435#A2)\.

Models\.Three families are compared: a static base Transformer \(no defense\), the Vanilla Ensemble with global Gaussian perturbation, and MorphStrata\. All share the same Transformer architecture and adversarial training procedure; only the student generation strategy differs\. Models are implemented in PyTorch\.

Evaluation\.Each configuration is evaluated over 30 randomized trials\. RMSE is the primary metric, reported as mean±\\pmstd\. We additionally track pairwise weight L2 distance as a statistical heterogeneity proxy and differential immunity\(Senguptaet al\.,[2019](https://arxiv.org/html/2606.17435#bib.bib35)\)as a measure of attack non\-transferability across the pool\. Wall clock time per pipeline stage is measured by a per\-sample monitoring system recording peak CPU RAM, peak VRAM, and stage elapsed time\.

## 6Results

![Refer to caption](https://arxiv.org/html/2606.17435v1/x1.png)Figure 2:RMSE under adversarial attacks on JENA\.![Refer to caption](https://arxiv.org/html/2606.17435v1/x2.png)Figure 3:RMSE under adversarial attacks on ECL\.![Refer to caption](https://arxiv.org/html/2606.17435v1/x3.png)Figure 4:RMSE under adversarial attacks on AEP\.![Refer to caption](https://arxiv.org/html/2606.17435v1/x4.png)Figure 5:RMSE under adversarial attacks on the Synthetic High Entropy Periodic dataset\.![Refer to caption](https://arxiv.org/html/2606.17435v1/x5.png)Figure 6:RMSE under adversarial attacks on the Synthetic Low Entropy Periodic dataset\. Full numerical results for all radar charts provided in Appendix[G](https://arxiv.org/html/2606.17435#A7)\.### 6\.1Adversarial RMSE

Figures[2](https://arxiv.org/html/2606.17435#S6.F2)–[6](https://arxiv.org/html/2606.17435#S6.F6)summarize all dataset\-attack conditions; full RMSE tables with standard deviations are in Appendix[G](https://arxiv.org/html/2606.17435#A7)\.

Both MTD ensembles substantially reduce adversarial RMSE over the static base model, particularly under BIM and PGD where the undefended baseline degrades catastrophically with increasingϵ\\epsilon\. On AEP under BIM atϵ=0\.5\\epsilon=0\.5, the base model reaches RMSE 3\.865 while MorphStrata holds at 0\.079 \(a 97\.97% reduction\), providing the strongest empirical evidence that MTD yields substantial returns against iterative threats\.

The relative performance of MorphStrata versus the Vanilla Ensemble is dataset\-dependent: the Vanilla Ensemble leads on JENA under FGSM and PGD; the two pipelines are within the margin of experimental variance on ECL; and MorphStrata achieves the lowest RMSE across all attacks and perturbation budgets on AEP, with the largest margin under FGSM \(full AEP rows in Appendix[G](https://arxiv.org/html/2606.17435#A7)\)\.

### 6\.2Statistical Heterogeneity and Robustness

The results show a positive correlation between higher pairwise L2 distance among students and defense effectiveness\. AEP is the only dataset where MorphStrata consistently produces larger pairwise weight L2 than the Vanilla Ensemble across all three attacks: increases of\+34\.2%\+34\.2\\%under FGSM,\+17\.9%\+17\.9\\%under BIM, and\+9\.0%\+9\.0\\%under PGD, and is also the only dataset where MorphStrata achieves the lowest RMSE in every condition\. Full statistical heterogeneity and differential immunity results are in Appendix[H](https://arxiv.org/html/2606.17435#A8)\.

### 6\.3Temporal Structure and Behavioral Patterns

The dataset\-dependent behavior traces back to temporal structure: JENA has the lowest normalized spectral entropy \(0\.33\) and near\-unit lag\-1 autocorrelation \(0\.9996\), while AEP has the highest spectral entropy \(0\.82\) and short memory \(decay at 260 min\)\. On high\-memory, low\-entropy datasets like JENA, perturbing a subset of components can disrupt learned long\-range dependencies; on AEP, structured layer\-specific heterogeneity appears more beneficial\. Full temporal analysis, autocorrelation decay curves, and power spectral density plots are in Appendix[C](https://arxiv.org/html/2606.17435#A3)\.

### 6\.4Computational Cost

Layer targeting adds less than 1\.1% wall clock overhead over the Vanilla Ensemble across all nine conditions\. MTD itself ranges from6×6\\timesto91×91\\timesthe base training cost, but that cost is shared by both pipelines\. For any deployment that has already accepted MTD, MorphStrata incurs negligible marginal cost\. Full stage\-level breakdowns are in Appendix[I](https://arxiv.org/html/2606.17435#A9)\.

## 7Conclusion

MorphStrata introduces layer\-specific student generation into the Morphence MTD framework for Transformer\-based time\-series forecasting\. MTD ensembles substantially outperform the static base model across all conditions, most strikingly on AEP under iterative attacks where the base collapses while both ensembles remain stable\. MorphStrata’s advantage over the Vanilla Ensemble is conditioned on temporal structure: it consistently achieves lower RMSE on AEP, is broadly competitive on ECL, and underperforms on JENA under single\-step attacks\. Layer targeting adds under 1\.1% training overhead at negligible marginal inference cost\. Future work should examine temporal\-structure\-aware perturbation scaling, adaptive student selection, and weight diversity under adaptive attacks\.

## 8Impact Statement

This paper presents work whose goal is to advance the field of Machine Learning\. There are many potential societal consequences of our work, none of which we feel must be highlighted here\.

## References

- A\. Amich and B\. Eshete \(2021\)Morphence: moving target defense against adversarial examples\.InAnnual Computer Security Applications Conference \(ACSAC\),pp\. 61–75\.Note:arXiv:2108\.13952External Links:[Document](https://dx.doi.org/10.1145/3485832.3485899),[Link](https://dl.acm.org/doi/10.1145/3485832.3485899)Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p2.1),[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- Z\. Awad, A\. Amich, and B\. Eshete \(2022\)Morphence 2\.0: evasion resilient moving target defense powered by out\-of\-distribution detection\.arXiv preprint arXiv:2206\.07321\.Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- H\. Cheng, Q\. Wen, Y\. Liu, L\. Sun, J\. Che, and Z\. Wang \(2024\)RobustTSF: towards theory and design of robust time series forecasting under anomalies\.arXiv preprint arXiv:2402\.02032\.Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- J\. M\. Cohen, E\. Rosenfeld, and Z\. Kolter \(2019\)Certified adversarial robustness via randomized smoothing\.InInternational Conference on Machine Learning \(ICML\),pp\. 1310–1320\.External Links:[Link](https://arxiv.org/abs/1902.02918)Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- Y\. Gal and Z\. Ghahramani \(2016\)Dropout as a bayesian approximation: representing model uncertainty in deep learning\.InInternational Conference on Machine Learning \(ICML\),pp\. 1050–1059\.External Links:[Link](https://arxiv.org/abs/1506.02142)Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- I\. J\. Goodfellow, J\. Shlens, and C\. Szegedy \(2015\)Explaining and harnessing adversarial examples\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1412.6572)Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1),[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- Y\. Govindarajulu, A\. Amballa, P\. Kulkarni, and M\. Parmar \(2023\)Targeted attacks on timeseries forecasting\.arXiv preprint arXiv:2301\.11544\.External Links:[Link](https://arxiv.org/abs/2301.11544)Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1),[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- P\. Krishan, R\. Mohapatra, S\. Das, and S\. Sengupta \(2024\)Adversarial attacks and defenses in multivariate time\-series forecasting for smart and connected infrastructures\.InProceedings of the Annual Conference of the Prognostics and Health Management Society,Vol\.16\.External Links:[Link](https://papers.phmsociety.org/index.php/phmconf/article/view/4082),[Document](https://dx.doi.org/10.36001/phmconf.2024.v16i1.4082)Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- A\. Kurakin, I\. Goodfellow, and S\. Bengio \(2017\)Adversarial machine learning at scale\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1611.01236)Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- B\. Lakshminarayanan, A\. Pritzel, and C\. Blundell \(2017\)Simple and scalable predictive uncertainty estimation using deep ensembles\.InAdvances in Neural Information Processing Systems \(NeurIPS\),pp\. 6405–6416\.External Links:[Link](https://arxiv.org/abs/1612.01474)Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- L\. Liu, Y\. Park, T\. N\. Hoang, H\. Hasson, and J\. Huan \(2022\)Towards robust multivariate time\-series forecasting: adversarial attacks and defense mechanisms\.InProceedings of the 8th SIGKDD Workshop on Mining and Learning from Time Series \(MileTS\),pp\. 1–9\.External Links:[Link](https://arxiv.org/abs/2207.09572)Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1),[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- A\. Madry, A\. Makelov, L\. Schmidt, D\. Tsipras, and A\. Vladu \(2018\)Towards deep learning models resistant to adversarial attacks\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1706.06083)Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1),[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- D\. Meng and H\. Chen \(2017\)MagNet: a two\-pronged defense against adversarial examples\.InProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security \(CCS\),pp\. 135–147\.External Links:[Document](https://dx.doi.org/10.1145/3133956.3134057),[Link](https://dl.acm.org/doi/10.1145/3133956.3134057)Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- Y\. Nie, N\. H\. Nguyen, P\. Sinthong, and J\. Kalagnanam \(2022\)A time series is worth 64 words: long\-term forecasting with transformers\.arXiv preprint arXiv:2211\.14730\.Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1)\.
- S\. Sengupta, T\. Chakraborti, and S\. Kambhampati \(2019\)MTDeep: boosting the security of deep neural nets against adversarial attacks with moving target defense\.InDecision and Game Theory for Security,pp\. 479–491\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-32430-8%5F28),1705\.07213Cited by:[Appendix H](https://arxiv.org/html/2606.17435#A8.p2.3),[2nd item](https://arxiv.org/html/2606.17435#S1.I1.i2.p1.1),[§2](https://arxiv.org/html/2606.17435#S2.p1.1),[§5](https://arxiv.org/html/2606.17435#S5.p3.1)\.
- S\. A\. Siddiqui, A\. Dengel, and S\. Ahmed \(2020\)Benchmarking adversarial attacks and defenses for time\-series data\.arXiv preprint arXiv:2008\.13261\.External Links:[Link](https://arxiv.org/abs/2008.13261)Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1),[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30\.External Links:[Link](https://arxiv.org/abs/1706.03762)Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1)\.
- Q\. Wen, T\. Zhou, C\. Zhang, W\. Chen, Z\. Ma, J\. Yan, and L\. Sun \(2022\)Transformers in time series: a survey\.arXiv preprint arXiv:2202\.07125\.Cited by:[§1](https://arxiv.org/html/2606.17435#S1.p1.1)\.
- J\. Zhang, Z\. Zhang, S\. Zheng, X\. Wen, J\. Li, and J\. Bian \(2025\)Are time series foundation models deployment\-ready? a systematic study of adversarial robustness across domains\.arXiv preprint arXiv:2505\.19397\.Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.
- P\. Zhao, Z\. Fu, O\. Wu, Q\. Hu, and J\. Wang \(2018\)Detecting adversarial examples via key\-based network\.arXiv preprint arXiv:1806\.00580\.External Links:[Link](https://arxiv.org/abs/1806.00580)Cited by:[§2](https://arxiv.org/html/2606.17435#S2.p1.1)\.

## Appendix AAppendix

This appendix contains dataset pipelines, temporal structure analysis, synthetic dataset experiments and generation methodology, full RMSE tables, statistical heterogeneity and differential immunity data, computational cost breakdowns, and memory footprint measurements\. The main paper summarizes key findings; all numerical claims cited in the main body are supported here\.

## Appendix BDataset Pipelines

All datasets are processed chronologically to prevent temporal leakage\. Scaling parameters are fit exclusively on the training partition and then applied to validation and test splits\. Each dataset is converted into fixed\-length sliding\-window forecasting samples before training\.

### B\.1Jena Climate

The Jena Climate dataset supports weather forecasting at hourly resolution\. The target variable is temperature\. Raw measurements are resampled to 60\-minute intervals, forward\-filled, and backfilled where necessary\. Five input features are selected \(pressure, temperature, potential temperature, relative humidity, wind speed\); the sequence is split 80/20 chronologically, scaled with a MinMax scaler fit on training only, and converted into lookback windows of length 24\.

### B\.2Electricity Load Diagrams \(ECL\)

ECL captures electricity load at 15\-minute resolution for a single meter\. The task is multi\-step ahead forecasting over a long historical context\. Because the input history is long, input patching is applied before Transformer encoding to compress the sequence into a manageable length while preserving coarse temporal structure\.

### B\.3Appliances Energy Prediction \(AEP\)

AEP captures residential appliance energy usage at 10\-minute intervals alongside indoor and outdoor environmental covariates\. Two synthetic random variables included in the original dataset are excluded as nuisance features\. The target is appliance energy consumption\. Separate MinMax scalers are fit for inputs and target on the training split\. Input patching is applied as in ECL\.

## Appendix CDataset Temporal Analysis

We characterize each dataset’s temporal structure using normalized spectral entropy \(Welch PSD,nperseg=512, normalized bylog⁡\(257\)\\log\(257\)\), lag\-1 autocorrelation, and autocorrelation decay measured as the first lag where\|ACF\|<0\.1\|\\text\{ACF\}\|<0\.1\.

Table 1:Temporal and spectral characteristics\. Memory classes: Long \(≥\\geq1000 min\), Medium \(200–999 min\), Short \(<<200 min\)\.†Synthetic Low \(periodic\) ACF oscillates due to sinusoidal interference; the 340\-min crossing underestimates true memory\.DatasetGroupNorm SEAC lag\-1Decay \(min\)MemoryJENA ClimateReal0\.32850\.999620000\+LongECLReal0\.81410\.915230000\+LongAEPReal0\.81520\.7532260MediumSyn\. Low Periodic†Synthetic0\.42950\.9884340MediumSyn\. High PeriodicSynthetic0\.95780\.021810ShortJENA has the strongest persistence and lowest spectral entropy: a highly structured, periodic signal with very long memory\. ECL shares the long\-memory property but with higher volatility and higher spectral entropy\. AEP decays much faster \(260 min\) and has the highest spectral entropy among real datasets, reflecting diffuse, locally driven energy consumption patterns\. Spectral entropy alone does not separate AEP from ECL \(both near 0\.81\); memory length is the clean differentiator\.

This structure helps interpret the RMSE results\. On JENA, the dominant long\-range temporal pattern is strong enough that confining perturbation to a subset of Transformer components can disrupt learned periodicity; global perturbation is comparatively gentler\. On AEP, where no single dominant frequency anchors the signal, layer\-specific statistical heterogeneity appears beneficial\. ECL, sitting between JENA and AEP on both axes, produces a mixed result with neither pipeline consistently ahead\.

### C\.1Autocorrelation Decay

![Refer to caption](https://arxiv.org/html/2606.17435v1/figures/acf_individual_panels.png)Figure 7:Autocorrelation decay for AEP, Jena Climate, and ECL\. AEP drops below the 0\.1 threshold at 260 minutes; Jena and ECL remain strongly autocorrelated over the full measured window\.
### C\.2Power Spectral Density

![Refer to caption](https://arxiv.org/html/2606.17435v1/figures/psd_all_datasets.png)Figure 8:Normalized power spectral density for AEP, Jena Climate, and ECL\. Spectral entropy is highest for AEP \(diffuse\) and lowest for Jena \(concentrated periodic structure\)\.
### C\.3Target Distribution

![Refer to caption](https://arxiv.org/html/2606.17435v1/figures/target_distribution_cv.png)Figure 9:Target distribution and coefficient of variation\. ECL has the highest normalized volatility; AEP has a right\-skewed residential energy distribution; Jena has a smooth unimodal temperature distribution\.

## Appendix DSynthetic Dataset Experiments

Two synthetic datasets test whether the behavioral differences observed across JENA, ECL, and AEP generalize to controlled temporal structure variations\.

### D\.1Generation Methodology

Both datasets share the following global parameters:N=19,735N=19\{,\}735samples \(matching AEP\), 5 features, 10\-minute sampling interval \(Δ​t=600\\Delta t=600s\), timestamps from 2009\-01\-01 to 2023\-10\-14, and random seed 42 \(numpy\.random\.default\_rng\(42\)\)\.

Synthetic\-Low \(Slow Decay, Periodic\)\.Each featurek∈\{0,1,2,3,4\}k\\in\\\{0,1,2,3,4\\\}is generated as a function of sample indext∈\{0,1,…,N−1\}t\\in\\\{0,1,\\ldots,N\-1\\\}:

xk​\(t\)=∑j=14Aj​sin⁡\(2​π​tPj\+ϕj\)⏟periodic component\+ϵk​\(t\)⏟AR\(1\) noise\+τk​\(t\)⏟trend\+2​kx\_\{k\}\(t\)=\\underbrace\{\\sum\_\{j=1\}^\{4\}A\_\{j\}\\sin\\\!\\left\(\\frac\{2\\pi t\}\{P\_\{j\}\}\+\\phi\_\{j\}\\right\)\}\_\{\\text\{periodic component\}\}\+\\underbrace\{\\epsilon\_\{k\}\(t\)\}\_\{\\text\{AR\(1\) noise\}\}\+\\underbrace\{\\tau\_\{k\}\(t\)\}\_\{\\text\{trend\}\}\+2k\(5\)wheret∈\{0,1,…,N−1\}t\\in\\\{0,1,\\ldots,N\{\-\}1\\\}is the integer sample index andPjP\_\{j\}is expressed in samples\. The periodic component uses four dominant sinusoids with fixed periods and amplitudes:

jjPeriodPjP\_\{j\}\(samples\)Period \(real time\)AmplitudeAjA\_\{j\}114424 hours1\.0027212 hours0\.603366 hours0\.304183 hours0\.15Phase offsetsϕj∼Uniform​\(0,2​π\)\\phi\_\{j\}\\sim\\text\{Uniform\}\(0,2\\pi\)are drawn independently per feature\. The AR\(1\) noise uses a slow\-decaying process withϕ=0\.97\\phi=0\.97:

ϵk​\(t\)=0\.97⋅ϵk​\(t−1\)\+ξt,ξt∼𝒩​\(0,0\.152\)\\epsilon\_\{k\}\(t\)=0\.97\\cdot\\epsilon\_\{k\}\(t\-1\)\+\\xi\_\{t\},\\qquad\\xi\_\{t\}\\sim\\mathcal\{N\}\(0,\\,0\.15^\{2\}\)\(6\)This produces very long autocorrelation memory \(lag\-10≈0\.78\\approx 0\.78\)\. Trend:τk​\(t\)=linspace​\(0,0\.5​\(k\+1\),N\)\\tau\_\{k\}\(t\)=\\text\{linspace\}\(0,\\,0\.5\(k\{\+\}1\),\\,N\); offset:2​k2k\. Measured properties: spectral entropy \(mean across features\)=2\.36=2\.36, AC lag\-1=0\.989=0\.989, AC lag\-10=0\.782=0\.782\.

Synthetic\-High \(Fast Decay, Diffuse\)\.Each featurekkis generated as:

xk​\(t\)=∑j=140aj​sin⁡\(2​π​tpj\+ϕj\)⏟broadband component\+ηk​\(t\)⏟AR\(1\) noise\+wk​\(t\)⏟random walk\+2​kx\_\{k\}\(t\)=\\underbrace\{\\sum\_\{j=1\}^\{40\}a\_\{j\}\\sin\\\!\\left\(\\frac\{2\\pi t\}\{p\_\{j\}\}\+\\phi\_\{j\}\\right\)\}\_\{\\text\{broadband component\}\}\+\\underbrace\{\\eta\_\{k\}\(t\)\}\_\{\\text\{AR\(1\) noise\}\}\+\\underbrace\{w\_\{k\}\(t\)\}\_\{\\text\{random walk\}\}\+2k\(7\)wherettis again the integer sample index\. The broadband component uses 40 sinusoids with randomly drawn periods and weak amplitudes spread uniformly across the frequency spectrum:

pj∼Uniform​\(5,9867\),aj∼Uniform​\(0\.02,0\.15\),ϕj∼Uniform​\(0,2​π\)p\_\{j\}\\sim\\text\{Uniform\}\(5,\\,9867\),\\quad a\_\{j\}\\sim\\text\{Uniform\}\(0\.02,\\,0\.15\),\\quad\\phi\_\{j\}\\sim\\text\{Uniform\}\(0,\\,2\\pi\)\(8\)The upper boundN/2=9,867N/2=9\{,\}867spans the full Nyquist range\. This distributes spectral energy broadly rather than concentrating it at dominant frequencies, which is the defining property of high spectral entropy\. AR\(1\) noise uses fast decay withϕ=0\.35\\phi=0\.35:

ηk​\(t\)=0\.35⋅ηk​\(t−1\)\+ζt,ζt∼𝒩​\(0,1\.02\)\\eta\_\{k\}\(t\)=0\.35\\cdot\\eta\_\{k\}\(t\-1\)\+\\zeta\_\{t\},\\qquad\\zeta\_\{t\}\\sim\\mathcal\{N\}\(0,\\,1\.0^\{2\}\)\(9\)Random walk:wk​\(t\)=cumsum​\(ε\)w\_\{k\}\(t\)=\\text\{cumsum\}\(\\varepsilon\),ε∼𝒩​\(0,0\.032\)\\varepsilon\\sim\\mathcal\{N\}\(0,\\,0\.03^\{2\}\), adding non\-stationarity and long\-range drift\. Offset:2​k2k\. Measured properties: spectral entropy=5\.35=5\.35, AC lag\-1=0\.734=0\.734, AC lag\-10=0\.587=0\.587\.

Spectral entropy was computed using Welch’s method \(nperseg=512\), normalized to a probability distribution before applying Shannon entropy\. Both datasets are fully reproducible via the fixed random seed; generation code and CSVs are provided in supplementary materials\.

### D\.2Results

On Synthetic\-High and Synthetic\-Low, MorphStrata is broadly comparable to vanilla, consistent with the AEP finding that layer\-specific heterogeneity benefits diffuse or mixed\-entropy signals\. Full RMSE tables are in Appendix[G](https://arxiv.org/html/2606.17435#A7); radar chart comparisons are in Figures[5](https://arxiv.org/html/2606.17435#S6.F5)and[6](https://arxiv.org/html/2606.17435#S6.F6)\.

## Appendix EModel and Training Details

All experiments use a shared Transformer architecture: input projection todmodel=128d\_\{\\text\{model\}\}=128, 4 attention heads, 4 encoder layers, feed\-forward dimension 256, pre\-norm \(norm\-first\) configuration, dropout 0\.1\. The same architecture is used for the base model, vanilla students, and MorphStrata students across all three datasets, ensuring the comparison is driven solely by the student generation strategy\.

The base model is trained on clean data, and the best checkpoint by validation loss is frozen as the teacher for student generation\. Vanilla and MorphStrata students are initialized from this checkpoint and then adversarially fine\-tuned\. The student pool is fixed after fine\-tuning; no repeated pool regeneration is performed\.

## Appendix FAttack Implementation

All attacks maximize the MSE forecasting loss with respect to the input under anℓ∞\\ell\_\{\\infty\}budgetϵ∈\{0\.1,0\.2,0\.3,0\.5\}\\epsilon\\in\\\{0\.1,0\.2,0\.3,0\.5\\\}\. Perturbed inputs are clipped to the valid scaled range\[0,1\]\[0,1\]\.

FGSM\.A randomized single\-step variant is used: the input is first perturbed by uniform noise𝒰​\(−ϵ,ϵ\)\\mathcal\{U\}\(\-\\epsilon,\\epsilon\), then one gradient\-sign step of sizeα=0\.02\\alpha=0\.02is applied\.

BIM\.Ten projected gradient\-sign iterations with step sizeα=ϵ/10\\alpha=\\epsilon/10, starting from the clean input\. After each step, the perturbation is clipped to theϵ\\epsilon\-ball and the result is clipped to\[0,1\]\[0,1\]\.

PGD\.Identical to BIM but initialized from a random point inside theϵ\\epsilon\-ball\.

## Appendix GFull RMSE Tables

Table 2:Jena Climate RMSE \(mean±\\pmstd, 30 runs\)\. Bold = best per row\.Attackϵ\\boldsymbol\{\\epsilon\}BaseVanillaMorphStrataFGSM0\.10\.07265±0\.000230\.07265\\pm 0\.000230\.04645±0\.00021\\mathbf\{0\.04645\\pm 0\.00021\}0\.10828±0\.000340\.10828\\pm 0\.000340\.20\.09764±0\.000390\.09764\\pm 0\.000390\.05272±0\.00019\\mathbf\{0\.05272\\pm 0\.00019\}0\.19944±0\.000510\.19944\\pm 0\.000510\.30\.11911±0\.000650\.11911\\pm 0\.000650\.05870±0\.00028\\mathbf\{0\.05870\\pm 0\.00028\}0\.25986±0\.000840\.25986\\pm 0\.000840\.50\.15934±0\.000850\.15934\\pm 0\.000850\.07182±0\.00040\\mathbf\{0\.07182\\pm 0\.00040\}0\.30254±0\.001110\.30254\\pm 0\.00111BIM0\.10\.12817±0\.000220\.12817\\pm 0\.000220\.14268±0\.000750\.14268\\pm 0\.000750\.14125±0\.00063\\mathbf\{0\.14125\\pm 0\.00063\}0\.20\.21979±0\.000270\.21979\\pm 0\.000270\.14284±0\.000640\.14284\\pm 0\.000640\.14124±0\.00045\\mathbf\{0\.14124\\pm 0\.00045\}0\.30\.30222±0\.000320\.30222\\pm 0\.000320\.14300±0\.000800\.14300\\pm 0\.000800\.14140±0\.00058\\mathbf\{0\.14140\\pm 0\.00058\}0\.50\.43441±0\.000610\.43441\\pm 0\.000610\.14297±0\.000680\.14297\\pm 0\.000680\.14139±0\.00062\\mathbf\{0\.14139\\pm 0\.00062\}PGD0\.10\.12817±0\.000220\.12817\\pm 0\.000220\.09456±0\.00087\\mathbf\{0\.09456\\pm 0\.00087\}0\.09859±0\.001210\.09859\\pm 0\.001210\.20\.21979±0\.000270\.21979\\pm 0\.000270\.13286±0\.00116\\mathbf\{0\.13286\\pm 0\.00116\}0\.14070±0\.000880\.14070\\pm 0\.000880\.30\.30222±0\.000320\.30222\\pm 0\.000320\.14624±0\.00158\\mathbf\{0\.14624\\pm 0\.00158\}0\.15425±0\.001390\.15425\\pm 0\.001390\.50\.43441±0\.000610\.43441\\pm 0\.000610\.15710±0\.00247\\mathbf\{0\.15710\\pm 0\.00247\}0\.16521±0\.001560\.16521\\pm 0\.00156Table 3:ECL RMSE \(mean±\\pmstd, 30 runs\)\. Bold = best per row\.Attackϵ\\boldsymbol\{\\epsilon\}BaseVanillaMorphStrataFGSM0\.10\.16916±0\.000460\.16916\\pm 0\.000460\.14172±0\.00096\\mathbf\{0\.14172\\pm 0\.00096\}0\.14299±0\.000970\.14299\\pm 0\.000970\.20\.23045±0\.000850\.23045\\pm 0\.000850\.13827±0\.00091\\mathbf\{0\.13827\\pm 0\.00091\}0\.13907±0\.000920\.13907\\pm 0\.000920\.30\.27207±0\.000680\.27207\\pm 0\.000680\.14609±0\.00121\\mathbf\{0\.14609\\pm 0\.00121\}0\.14837±0\.001230\.14837\\pm 0\.001230\.50\.29805±0\.000720\.29805\\pm 0\.000720\.15870±0\.001280\.15870\\pm 0\.001280\.15040±0\.00141\\mathbf\{0\.15040\\pm 0\.00141\}BIM0\.10\.24913±0\.000550\.24913\\pm 0\.000550\.19305±0\.00072\\mathbf\{0\.19305\\pm 0\.00072\}0\.20067±0\.000450\.20067\\pm 0\.000450\.20\.33875±0\.000460\.33875\\pm 0\.000460\.19668±0\.00066\\mathbf\{0\.19668\\pm 0\.00066\}0\.20547±0\.000510\.20547\\pm 0\.000510\.30\.36828±0\.000430\.36828\\pm 0\.000430\.20080±0\.00060\\mathbf\{0\.20080\\pm 0\.00060\}0\.21164±0\.000630\.21164\\pm 0\.000630\.50\.39611±0\.000370\.39611\\pm 0\.000370\.24511±0\.00116\\mathbf\{0\.24511\\pm 0\.00116\}0\.24527±0\.001330\.24527\\pm 0\.00133PGD0\.10\.24921±0\.000530\.24921\\pm 0\.000530\.19259±0\.000510\.19259\\pm 0\.000510\.18979±0\.00049\\mathbf\{0\.18979\\pm 0\.00049\}0\.20\.33865±0\.000450\.33865\\pm 0\.000450\.19457±0\.000570\.19457\\pm 0\.000570\.19113±0\.00072\\mathbf\{0\.19113\\pm 0\.00072\}0\.30\.36821±0\.000350\.36821\\pm 0\.000350\.19894±0\.000490\.19894\\pm 0\.000490\.19563±0\.00073\\mathbf\{0\.19563\\pm 0\.00073\}0\.50\.39596±0\.000450\.39596\\pm 0\.000450\.21664±0\.00069\\mathbf\{0\.21664\\pm 0\.00069\}0\.21967±0\.000960\.21967\\pm 0\.00096Table 4:AEP RMSE \(mean±\\pmstd; 30 runs\)\. Bold = best per row\.Attackϵ\\boldsymbol\{\\epsilon\}BaseVanillaMorphStrataFGSM0\.10\.10398±0\.000030\.10398\\pm 0\.000030\.08386±0\.000190\.08386\\pm 0\.000190\.08093±0\.00039\\mathbf\{0\.08093\\pm 0\.00039\}0\.20\.10430±0\.000040\.10430\\pm 0\.000040\.08384±0\.000050\.08384\\pm 0\.000050\.08095±0\.00024\\mathbf\{0\.08095\\pm 0\.00024\}0\.30\.10491±0\.000050\.10491\\pm 0\.000050\.08410±0\.000120\.08410\\pm 0\.000120\.08085±0\.00034\\mathbf\{0\.08085\\pm 0\.00034\}0\.50\.10670±0\.000100\.10670\\pm 0\.000100\.08441±0\.000070\.08441\\pm 0\.000070\.08097±0\.00028\\mathbf\{0\.08097\\pm 0\.00028\}BIM0\.10\.30552±0\.000230\.30552\\pm 0\.000230\.07841±0\.000080\.07841\\pm 0\.000080\.07805±0\.00004\\mathbf\{0\.07805\\pm 0\.00004\}0\.21\.03908±0\.001171\.03908\\pm 0\.001170\.07852±0\.000060\.07852\\pm 0\.000060\.07819±0\.00006\\mathbf\{0\.07819\\pm 0\.00006\}0\.32\.05804±0\.002412\.05804\\pm 0\.002410\.07859±0\.000080\.07859\\pm 0\.000080\.07830±0\.00006\\mathbf\{0\.07830\\pm 0\.00006\}0\.53\.86461±0\.007003\.86461\\pm 0\.007000\.07877±0\.000090\.07877\\pm 0\.000090\.07855±0\.00012\\mathbf\{0\.07855\\pm 0\.00012\}PGD0\.10\.30552±0\.000230\.30552\\pm 0\.000230\.07909±0\.000170\.07909\\pm 0\.000170\.07835±0\.00004\\mathbf\{0\.07835\\pm 0\.00004\}0\.21\.03908±0\.001181\.03908\\pm 0\.001180\.07971±0\.000220\.07971\\pm 0\.000220\.07854±0\.00005\\mathbf\{0\.07854\\pm 0\.00005\}0\.32\.05807±0\.002412\.05807\\pm 0\.002410\.08044±0\.000430\.08044\\pm 0\.000430\.07868±0\.00005\\mathbf\{0\.07868\\pm 0\.00005\}0\.53\.86453±0\.006923\.86453\\pm 0\.006920\.08268±0\.000890\.08268\\pm 0\.000890\.07887±0\.00006\\mathbf\{0\.07887\\pm 0\.00006\}Table 5:Synthetic High Entropy Periodic RMSE \(mean±\\pmstd, 30 runs\)\. Bold = best per row\.Attackϵ\\boldsymbol\{\\epsilon\}BaseVanillaMorphStrataFGSM0\.10\.14495±0\.000020\.14495\\pm 0\.000020\.13405±0\.000300\.13405\\pm 0\.000300\.13219±0\.00011\\mathbf\{0\.13219\\pm 0\.00011\}0\.20\.14525±0\.000040\.14525\\pm 0\.000040\.13406±0\.000260\.13406\\pm 0\.000260\.13219±0\.00009\\mathbf\{0\.13219\\pm 0\.00009\}0\.30\.14559±0\.000050\.14559\\pm 0\.000050\.13409±0\.000230\.13409\\pm 0\.000230\.13210±0\.00009\\mathbf\{0\.13210\\pm 0\.00009\}0\.50\.14565±0\.000070\.14565\\pm 0\.000070\.13414±0\.000330\.13414\\pm 0\.000330\.13206±0\.00011\\mathbf\{0\.13206\\pm 0\.00011\}BIM0\.10\.24846±0\.000070\.24846\\pm 0\.000070\.13230±0\.00014\\mathbf\{0\.13230\\pm 0\.00014\}0\.13356±0\.000310\.13356\\pm 0\.000310\.20\.42712±0\.000260\.42712\\pm 0\.000260\.13237±0\.00015\\mathbf\{0\.13237\\pm 0\.00015\}0\.13362±0\.000280\.13362\\pm 0\.000280\.30\.59956±0\.000490\.59956\\pm 0\.000490\.13239±0\.00016\\mathbf\{0\.13239\\pm 0\.00016\}0\.13363±0\.000250\.13363\\pm 0\.000250\.50\.88182±0\.000730\.88182\\pm 0\.000730\.13246±0\.00020\\mathbf\{0\.13246\\pm 0\.00020\}0\.13366±0\.000360\.13366\\pm 0\.00036PGD0\.10\.24846±0\.000070\.24846\\pm 0\.000070\.13031±0\.00014\\mathbf\{0\.13031\\pm 0\.00014\}0\.13071±0\.000160\.13071\\pm 0\.000160\.20\.42712±0\.000260\.42712\\pm 0\.000260\.13042±0\.00014\\mathbf\{0\.13042\\pm 0\.00014\}0\.13082±0\.000170\.13082\\pm 0\.000170\.30\.59956±0\.000490\.59956\\pm 0\.000490\.13045±0\.00012\\mathbf\{0\.13045\\pm 0\.00012\}0\.13098±0\.000160\.13098\\pm 0\.000160\.50\.88182±0\.000730\.88182\\pm 0\.000730\.13054±0\.00016\\mathbf\{0\.13054\\pm 0\.00016\}0\.13128±0\.000270\.13128\\pm 0\.00027Table 6:Synthetic Low Entropy Periodic RMSE \(mean±\\pmstd, 30 runs\)\. Bold = best per row\.Attackϵ\\boldsymbol\{\\epsilon\}BaseVanillaMorphStrataFGSM0\.10\.16885±0\.000060\.16885\\pm 0\.000060\.13929±0\.000700\.13929\\pm 0\.000700\.13079±0\.00092\\mathbf\{0\.13079\\pm 0\.00092\}0\.20\.16804±0\.000120\.16804\\pm 0\.000120\.13942±0\.000550\.13942\\pm 0\.000550\.13087±0\.00085\\mathbf\{0\.13087\\pm 0\.00085\}0\.30\.16640±0\.000240\.16640\\pm 0\.000240\.13982±0\.000790\.13982\\pm 0\.000790\.13114±0\.00093\\mathbf\{0\.13114\\pm 0\.00093\}0\.50\.16121±0\.000230\.16121\\pm 0\.000230\.14127±0\.000530\.14127\\pm 0\.000530\.13248±0\.00082\\mathbf\{0\.13248\\pm 0\.00082\}BIM0\.10\.31245±0\.000090\.31245\\pm 0\.000090\.13074±0\.00115\\mathbf\{0\.13074\\pm 0\.00115\}0\.17690±0\.002830\.17690\\pm 0\.002830\.20\.43343±0\.000150\.43343\\pm 0\.000150\.14705±0\.00100\\mathbf\{0\.14705\\pm 0\.00100\}0\.18475±0\.003100\.18475\\pm 0\.003100\.30\.52030±0\.000300\.52030\\pm 0\.000300\.16800±0\.00105\\mathbf\{0\.16800\\pm 0\.00105\}0\.19289±0\.003930\.19289\\pm 0\.003930\.50\.63804±0\.000440\.63804\\pm 0\.000440\.19577±0\.00079\\mathbf\{0\.19577\\pm 0\.00079\}0\.20963±0\.003720\.20963\\pm 0\.00372PGD0\.10\.31245±0\.000090\.31245\\pm 0\.000090\.14318±0\.00132\\mathbf\{0\.14318\\pm 0\.00132\}0\.23813±0\.008840\.23813\\pm 0\.008840\.20\.43343±0\.000150\.43343\\pm 0\.000150\.16091±0\.00136\\mathbf\{0\.16091\\pm 0\.00136\}0\.24864±0\.010540\.24864\\pm 0\.010540\.30\.52030±0\.000300\.52030\\pm 0\.000300\.17005±0\.00130\\mathbf\{0\.17005\\pm 0\.00130\}0\.25912±0\.011160\.25912\\pm 0\.011160\.50\.63804±0\.000440\.63804\\pm 0\.000440\.18485±0\.00179\\mathbf\{0\.18485\\pm 0\.00179\}0\.28165±0\.014790\.28165\\pm 0\.01479
## Appendix HStatistical Heterogeneity and Differential Immunity

We report two metrics for statistical heterogeneity\. Pairwise weight L2 distance is the Euclidean norm between flattened parameter vectors for each pair of students in the pool \):

L2​\(θi,θj\)=\(∑ℓ\(θi,ℓ−θj,ℓ\)2\)1/2\.L\_\{2\}\(\\theta\_\{i\},\\theta\_\{j\}\)=\\left\(\\sum\_\{\\ell\}\(\\theta\_\{i,\\ell\}\-\\theta\_\{j,\\ell\}\)^\{2\}\\right\)^\{1/2\}\.\(10\)Higher L2 indicates that students occupy more separated regions of parameter space, which tends to reduce attack transferability across the pool\.

Differential immunityδ\\deltais adapted from MTDeep\(Senguptaet al\.,[2019](https://arxiv.org/html/2606.17435#bib.bib35)\), originally proposed for classification\. For a fixed attackeruuand budgetϵ\\epsilon, we define:

δ​\(u,ϵ\)=maxn⁡RMSE​\(n,u\)−minn⁡RMSE​\(n,u\)maxn⁡RMSE​\(n,u\),\\delta\(u,\\epsilon\)=\\frac\{\\max\_\{n\}\\mathrm\{RMSE\}\(n,u\)\-\\min\_\{n\}\\mathrm\{RMSE\}\(n,u\)\}\{\\max\_\{n\}\\mathrm\{RMSE\}\(n,u\)\},\(11\)wherennindexes defender students\. A highδ\\deltameans the pool’s best defender performs much better than its worst defender against that attacker, i\.e\., the attack does not transfer uniformly\. We report worst\-caseδ\\deltaacross all attackers and allϵ\\epsilonvalues per cell\. The adaptation to regression is direct: RMSE replaces classification error, and the ratio preserves the same scale\-free interpretation as the original formulation\.

Table 7:Pairwise weight L2 distance and worst\-case differential immunityδ\\delta\. Bold indicates MorphStrata improves over Vanilla\.Δ​L2\\Delta L\_\{2\}andΔ​δ\\Delta\\deltaare absolute changes \(MorphStrata minus Vanilla\)\.DatasetAttackWeight L2δ\\deltaworstΔ​L2\\Delta L\_\{2\}Δ​δ\\Delta\\deltaVanillaMorphStrataVanillaMorphStrataJENAFGSM27\.2426\.470\.00410\.0397−0\.77\-0\.77\(−2\.8%\-2\.8\\%\)\+0\.0356\+0\.0356BIM12\.5813\.530\.03050\.0819\+0\.95\+0\.95\(\+7\.5%\+7\.5\\%\)\+0\.0514\+0\.0514PGD73\.1073\.680\.12740\.0525\+0\.58\+0\.58\(\+0\.8%\+0\.8\\%\)−0\.0750\-0\.0750ECLFGSM16\.0114\.810\.01580\.1497−1\.20\-1\.20\(−7\.5%\-7\.5\\%\)\+0\.1339\+0\.1339BIM9\.468\.400\.14180\.0834−1\.06\-1\.06\(−11\.2%\-11\.2\\%\)−0\.0584\-0\.0584PGD9\.018\.960\.06350\.0216−0\.05\-0\.05\(−0\.6%\-0\.6\\%\)−0\.0418\-0\.0418AEPFGSM30\.4840\.890\.01330\.0180\+10\.41\+10\.41\(\+34\.2%\+34\.2\\%\)\+0\.0047\+0\.0047BIM38\.6345\.530\.03840\.0085\+6\.91\+6\.91\(\+17\.9%\+17\.9\\%\)−0\.0299\-0\.0299PGD37\.0140\.350\.00900\.0216\+3\.34\+3\.34\(\+9\.0%\+9\.0\\%\)\+0\.0126\+0\.0126Several patterns are worth noting\. AEP is the only dataset where MorphStrata consistently produces larger pairwise L2 than Vanilla across all three attacks; it is also the only dataset where MorphStrata achieves the lowest RMSE in every condition\. This alignment supports the observed positive L2\-robustness correlation\. On JENA and ECL, L2 changes are small in absolute terms and bidirectional; MorphStrata’s RMSE behavior is correspondingly mixed on these datasets\.

Differential immunity is less monotone\. MorphStrata improvesδ\\deltain 5 of 9 cells, with the largest gains on JENA\-FGSM and ECL\-FGSM, both single\-step attack conditions\. The JENA\-PGD cell is the clearest anomaly: the Vanilla Ensemble already produces pairwise L2 of 73\.10, the largest in the entire table, leaving no room for MorphStrata to widen the pool further\. Accordingly, the Vanilla Ensemble beats MorphStrata on JENA\-PGD in the RMSE tables\.

## Appendix IComputational Cost

Resource metrics are produced by a per\-stage monitoring system running on all three dataset pipelines\. It samples every 2 seconds and records peak CPU RAM, peak VRAM, and stage wall\-clock time\. Per\-sample inference latency is not included in the monitoring stack\. All experiments were conducted on an NVIDIA L4 GPU \(Ada Lovelace, 22\.5 GB VRAM\) via Google Colab\.

### I\.1Layer Targeting Overhead over the Vanilla Ensemble

The extra cost of MorphStrata over the Vanilla Ensemble is computed as:

Extra cost=MorphStrata total−Vanilla totalVanilla total×100%\.\\text\{Extra cost\}=\\frac\{\\text\{MorphStrata total\}\-\\text\{Vanilla total\}\}\{\\text\{Vanilla total\}\}\\times 100\\%\.\(12\)
Table 8:Wall\-clock cost comparison: Vanilla Ensemble vs\. MorphStrata\. Times are total pipeline duration in minutes; Extra cost is the percentage overhead of MorphStrata over Vanilla\.DatasetAttackVanilla \(minutes\)MorphStrata \(minutes\)Extra costJENAFGSM9\.549\.64\+1\.05%\+1\.05\\%BIM41\.4841\.59\+0\.27%\+0\.27\\%PGD41\.4541\.62\+0\.41%\+0\.41\\%ECLFGSM29\.2029\.27\+0\.24%\+0\.24\\%BIM148\.43148\.85\+0\.28%\+0\.28\\%PGD148\.34149\.08\+0\.50%\+0\.50\\%AEPFGSM29\.4929\.55\+0\.20%\+0\.20\\%BIM159\.36159\.47\+0\.07%\+0\.07\\%PGD159\.36159\.44\+0\.05%\+0\.05\\%Across all nine conditions, the extra cost of layer targeting stays under 1\.1%\. The masking operation during student initialization is the only additional computation; adversarial training and inference are identical between the two pipelines\.

### I\.2MTD Overhead over the Base Model

For deployments deciding whether to adopt MTD at all, the relevant comparison is the full MorphStrata pipeline against a single base Transformer trained on clean data:

MTD overhead=MorphStrata totalBase training time\.\\text\{MTD overhead\}=\\frac\{\\text\{MorphStrata total\}\}\{\\text\{Base training time\}\}\.\(13\)
Table 9:MorphStrata pipeline overhead relative to undefended base training\. Vanilla/Base and MorphStrata/Base are multiplicative overheads; MorphStrata/Vanilla is the marginal overhead of MorphStrata over the Vanilla Ensemble\.DatasetAttackBase \(minutes\)Vanilla \(minutes\)MorphStrata \(minutes\)Vanilla / BaseMorphStrata / BaseMorphStrata / VanillaJENAFGSM1\.169\.549\.648\.22×8\.22\\times8\.31×8\.31\\times1\.01×1\.01\\timesBIM1\.1641\.4841\.5935\.76×35\.76\\times35\.85×35\.85\\times1\.00×1\.00\\timesPGD1\.1641\.4541\.6235\.73×35\.73\\times35\.88×35\.88\\times1\.00×1\.00\\timesECLFGSM5\.0329\.2029\.275\.81×5\.81\\times5\.82×5\.82\\times1\.00×1\.00\\timesBIM5\.03148\.43148\.8529\.51×29\.51\\times29\.59×29\.59\\times1\.00×1\.00\\timesPGD5\.03148\.34149\.0829\.49×29\.49\\times29\.64×29\.64\\times1\.00×1\.00\\timesAEPFGSM1\.7529\.4929\.5516\.85×16\.85\\times16\.89×16\.89\\times1\.00×1\.00\\timesBIM1\.75159\.36159\.4791\.06×91\.06\\times91\.13×91\.13\\times1\.00×1\.00\\timesPGD1\.75159\.36159\.4491\.06×91\.06\\times91\.11×91\.11\\times1\.00×1\.00\\timesMTD is expensive in absolute terms, ranging from5\.81×5\.81\\timesto91\.13×91\.13\\timesthe base training cost\. The overhead is dominated by adversarial fine\-tuning of multiple students, which scales with the number of attack iterations and the number of epsilon values in the training sweep\. The marginal cost of layer targeting on top of this is negligible: MorphStrata/Vanilla never exceeds1\.01×1\.01\\timesacross all nine conditions\. For any deployment that has accepted MTD, MorphStrata incurs negligible marginal cost; the robustness gains on AEP at high perturbation budgets provide the clearest justification\.

## Appendix JMemory Footprint

Peak VRAM and peak CPU RAM are recorded by the monitoring system\. All values are in MB\. TheBase traincolumn shows memory during undefended base model training, which is identical for both pipelines\.VanillaandMorphStratacolumns show peak memory during adversarial fine\-tuning for each pipeline\.

Table 10:Peak VRAM and CPU RAM during training stages on an NVIDIA L4 GPU \(Ada Lovelace\)\.Base trainis shared across both pipelines\.VRAM \(MB\)CPU RAM \(MB\)DatasetAttackBase trainVanillaMorphStrataBase trainVanillaMorphStrataJENAFGSM7679771,0351,7881,8221,836BIM7679771,0351,7881,8221,836PGD7679771,0351,7881,8231,836ECLFGSM8931,2851,5911,6621,7441,861BIM8931,2851,5931,6621,7461,864PGD8931,2851,5931,6621,7471,865AEPFGSM1,2772,8713,2797,5837,7817,863BIM1,2772,8713,2797,5837,7897,859PGD1,2772,8593,2577,5837,7887,865MorphStrata requires modestly more VRAM than Vanilla during adversarial training, peaking at a 24% increase for ECL \(1,285 MB to 1,591 MB\)\. CPU RAM increases are under 7% across all conditions\. Adversarial training consumes roughly2×2\\timesto3×3\\timesthe VRAM of clean base training because the attack loop holds perturbation tensors and gradients in memory simultaneously\. AEP CPU RAM is substantially higher than JENA or ECL \(≈\\approx7\.5 GB vs\. 1\.8 GB\) because the AEP notebook retains the full preprocessed dataset in CPU memory throughout training\.

## Appendix KExperimental Platform

All experiments were conducted using the hardware summarized in Table[11](https://arxiv.org/html/2606.17435#A11.T11)\. The GPU is an NVIDIA L4 \(Ada Lovelace architecture, compute capability 8\.9\), Its 22\.5 GB of VRAM was sufficient to hold all student models and their adversarial perturbation tensors in memory simultaneously, with no gradient checkpointing required\. The 12\-thread Intel Xeon CPU and 52 GB of system RAM meant that even the AEP pipeline, which retains its full preprocessed dataset in CPU memory, was not memory\-constrained\. Framework versions are fixed at PyTorch 2\.10\.0 with CUDA 12\.8; all reported results are reproducible under these versions\.

Table 11:Experimental platform specifications\.ComponentSpecificationGPUNVIDIA L4 \(Ada Lovelace, compute cap\. 8\.9\)GPU VRAM22\.5 GB \(23,034 MiB\)CPUIntel Xeon @ 2\.20 GHzCPU cores / threads6 cores, 12 threads \(2 threads/core\)System RAM52 GB