EVOTS: Evolutionary Transformer Search for Time Series Forecasting

# Evolutionary Transformer Search for Time Series Forecasting
Source: [https://arxiv.org/html/2607.00154](https://arxiv.org/html/2607.00154)
AbdElRahman ElSaidDamir Pulatovelsaida@uncw\.edupulatovd@uncw\.eduUniversity of North Carolina WilmingtonWilmington, North Carolina, USA

###### Abstract

Evolutionary neural architecture design for multivariate time\-series forecasting remains under\-explored, with most approaches relying on fixed Transformer architectures despite substantial variation across tasks and forecasting settings\. This paper introduces an evolutionary neural architecture search framework for discovering task\-adaptive Transformer\-like models for time\-series forecasting \(EvoTS\)\. Architectures are encoded using a modular genome representation that enables flexible composition of attention, feed\-forward, and projection components, while a repair mechanism enforces structural validity throughout the evolutionary process\. This formulation allows effective exploration of a diverse architecture space without relying on hand\-crafted design rules\.

The proposed approach is evaluated on four benchmark datasets from the ETT family \(ETTh1, ETTh2, ETTm1, and ETTm2\) under multiple forecasting settings, including univariate\-to\-univariate, multivariate\-to\-univariate, and multivariate\-to\-multivariate prediction, with horizons of 96, 192, 336, and 720\. In the multivariate\-to\-multivariate setting, the evolved architectures achieve competitive and, in several cases, improved mean squared error relative to a strong Transformer\-based baseline\. Additional analyses examine performance differences across forecasting settings and report wall\-clock training time to provide a coarse indication of computational cost\.

Overall, the results demonstrate that evolutionary search can effectively discover flexible and high\-performing Transformer\-like architectures for multivariate time\-series forecasting within practical runtime constraints\.

## 1Introduction

Accurate time\-series forecasting is a critical component of many real\-world systems, including energy management, industrial monitoring, and large\-scale infrastructure control\. Recent advances in Transformer\-based architectures have significantly improved forecasting performance by enabling flexible modeling of long\-range temporal dependencies and complex cross\-variable interactions\. As a result, Transformers and their variants have become a dominant paradigm in multivariate time\-series forecasting\.

Despite their success, most Transformer\-based forecasting models rely on carefully hand\-designed architectures\. Choices such as tokenization strategy, attention formulation, block composition, and network depth are typically fixed a priori and tuned through extensive manual experimentation\. While effective, this process is labor\-intensive and often yields architectures that perform well on a narrow range of tasks or prediction horizons, limiting adaptability across datasets and forecasting regimes\.

Evolutionary computation offers a principled alternative to manual architecture design\. By treating neural architectures as evolvable structures and optimizing them through population\-based search, evolutionary methods can explore diverse architectural compositions without committing to a single design template\. This paradigm is particularly well\-suited to neural architecture search problems where the design space is highly structured and the cost of evaluation is substantial\.

In this work, we proposeEvoTS111[https://github\.com/a\-elsaid/EVOTS\.git](https://github.com/a-elsaid/EVOTS.git), an evolutionary architecture search framework for Transformer\-like time\-series forecasting models\. The framework represents architectures as modular genomes composed of Transformer\-inspired processing blocks and evolves their composition using a steady\-state evolutionary algorithm with weight inheritance\. Unlike approaches that introduce new attention mechanisms or training objectives, our method focuses on automating architectural composition under a fixed and widely adopted training protocol\. All experiments follow the evaluation settings introduced in the iTransformer study\[[17](https://arxiv.org/html/2607.00154#bib.bib1)\], enabling direct and fair comparison to previous Transformer\-based baselines\.

We evaluate the proposed approach on four widely used ETT benchmark datasets \(ETTh1, ETTh2, ETTm1, ETTm2\) across multiple prediction horizons\. The evolved architectures consistently achieve strong performance in the multivariate\-to\-multivariate forecasting setting and match or exceed the strongest results reported in prior work under identical experimental conditions\. Performance gains are particularly pronounced at longer prediction horizons, where fixed architectural designs are most challenged\.

The main contributions of this paper are as follows:i\)We introduce a modular evolutionary architecture search framework for Transformer\-like time\-series forecasting models,ii\)we demonstrate that evolutionary search can automatically discover hybrid architectural compositions that outperform hand\-designed Transformer variants on standard forecasting benchmarks, andiii\)we provide empirical analysis showing that the benefits of evolutionary architecture search are amplified in long\-horizon and multivariate forecasting regimes\.

## 2Related Work

#### Transformer\-Based Time\-Series Forecasting\.

Transformer architectures have been widely adopted for time\-series forecasting due to their ability to model long\-range temporal dependencies and complex interactions among variables\[[27](https://arxiv.org/html/2607.00154#bib.bib19),[33](https://arxiv.org/html/2607.00154#bib.bib20)\]\. Early approaches primarily apply self\-attention over temporal tokens, treating each time step as a token that aggregates all variables\[[15](https://arxiv.org/html/2607.00154#bib.bib21),[33](https://arxiv.org/html/2607.00154#bib.bib20)\]\. Subsequent models introduce architectural modifications tailored to time\-series structure\. Autoformer incorporates series decomposition and replaces self\-attention with auto\-correlation mechanisms to emphasize periodic patterns\[[30](https://arxiv.org/html/2607.00154#bib.bib22)\]\. Crossformer introduces segmented embeddings that jointly preserve temporal and cross\-variable information\[[32](https://arxiv.org/html/2607.00154#bib.bib10)\]\. More recently, iTransformer proposes inverting the attention mechanism by treating variables as tokens, enabling direct modeling of cross\-variable relationships\[[17](https://arxiv.org/html/2607.00154#bib.bib1)\]\.

Following the success of large pretrained foundation models which rely on Transformer based architectures, another research direction is to build a foundation model that tackles time series instead of natural language\. The advantage of such approaches is that pretrained models can work out of the box on any time series data and no training is required by end users\. Research into pretrained models for time series ranges from simply prompting\[[12](https://arxiv.org/html/2607.00154#bib.bib37),[31](https://arxiv.org/html/2607.00154#bib.bib38)\]and fine\-tuning\[[34](https://arxiv.org/html/2607.00154#bib.bib39),[14](https://arxiv.org/html/2607.00154#bib.bib40)\]existing general\-purpose foundation models to training specialized models in which model architectures are modified to accommodate time series\[[22](https://arxiv.org/html/2607.00154#bib.bib41),[11](https://arxiv.org/html/2607.00154#bib.bib42),[4](https://arxiv.org/html/2607.00154#bib.bib43),[29](https://arxiv.org/html/2607.00154#bib.bib44),[2](https://arxiv.org/html/2607.00154#bib.bib35),[1](https://arxiv.org/html/2607.00154#bib.bib36)\]\. One of the drawbacks of such approaches is the limited amount of data available for training models of such a scale\. Pretrained models such as Chronos\[[2](https://arxiv.org/html/2607.00154#bib.bib35)\]resort to data augmentation and the use of synthetic datasets in addition to real\-world datasets while models such as ForecastPFN\[[5](https://arxiv.org/html/2607.00154#bib.bib45)\]and TabPFN\-v2\[[13](https://arxiv.org/html/2607.00154#bib.bib46)\]resort to training models purely on synthetic data\.

These works highlight the importance of architectural choices such as tokenization strategy, attention formulation, and block composition\. However, such choices are typically fixed a priori and manually tuned\[[9](https://arxiv.org/html/2607.00154#bib.bib30)\]for specific datasets and prediction horizons, limiting adaptability across forecasting regimes\.

#### Neural Architecture Search and Neuroevolution\.

Neural Architecture Search \(NAS\) seeks to automate the design of neural network architectures and has been explored using a variety of optimization paradigms\[[9](https://arxiv.org/html/2607.00154#bib.bib30)\]\. Evolutionary and neuroevolutionary approaches, including NEAT\[[26](https://arxiv.org/html/2607.00154#bib.bib3)\], EXAMM\[[20](https://arxiv.org/html/2607.00154#bib.bib23)\], EXA\-GP\[[19](https://arxiv.org/html/2607.00154#bib.bib24)\], ANTS\[[8](https://arxiv.org/html/2607.00154#bib.bib25)\], CANTS\[[7](https://arxiv.org/html/2607.00154#bib.bib27),[6](https://arxiv.org/html/2607.00154#bib.bib26)\], and NSGA\-Net\[[18](https://arxiv.org/html/2607.00154#bib.bib28)\], represent architectures as mutable structures and optimize them through population\-based search\. These methods are particularly well\-suited to discrete and structured design spaces, supporting architectural variation, reuse, and incremental refinement through mechanisms such as mutation, recombination, and inheritance\[[26](https://arxiv.org/html/2607.00154#bib.bib3),[20](https://arxiv.org/html/2607.00154#bib.bib23),[23](https://arxiv.org/html/2607.00154#bib.bib29)\]\.

Alternative NAS paradigms include reinforcement\-learning–based methods, such as controller\-driven and weight\-sharing approaches exemplified by ENAS\[[21](https://arxiv.org/html/2607.00154#bib.bib31)\], as well as gradient\-based techniques like DARTS\[[16](https://arxiv.org/html/2607.00154#bib.bib32)\]that rely on continuous relaxations of discrete architectural choices\. While these methods offer improved search efficiency, they typically operate over constrained architectural templates and are less naturally suited to search spaces involving heterogeneous modules and discrete structural variation\.

#### NAS for Transformer Architectures\.

NAS has also been applied to Transformer architectures, primarily in natural language processing\[[24](https://arxiv.org/html/2607.00154#bib.bib33)\]and computer vision\. Prior work such as the Evolved Transformer and subsequent vision\-oriented\[[3](https://arxiv.org/html/2607.00154#bib.bib34)\]approaches restrict the search space to variations of a fixed Transformer template, focusing on parameters such as embedding dimension, number of attention heads, or layer depth\. Other approaches expand the search space to include both self\-attention and convolutional operations\[[10](https://arxiv.org/html/2607.00154#bib.bib14)\], enabling hybrid designs while still enforcing predefined stage structures and tokenization schemes\.

These studies demonstrate the feasibility of automating Transformer design, but generally emphasize hyperparameter optimization or selection among predefined modules, rather than evolving the internal composition of Transformer\-like architectures under a fixed training and evaluation protocol\[[24](https://arxiv.org/html/2607.00154#bib.bib33),[10](https://arxiv.org/html/2607.00154#bib.bib14)\]\.

#### Positioning of This Work\.

In contrast to prior NAS and Transformer\-NAS approaches, this work focuses on evolutionary search over the internal composition of Transformer\-like architectures for time\-series forecasting\. Architectures are represented as modular genomes\[[26](https://arxiv.org/html/2607.00154#bib.bib3),[20](https://arxiv.org/html/2607.00154#bib.bib23)\]composed of Transformer\-inspired blocks, including multiple attention variants and convolutional token\-mixing operations, and are evolved using a steady\-state\[[28](https://arxiv.org/html/2607.00154#bib.bib4)\]evolutionary algorithm with weight inheritance\[[23](https://arxiv.org/html/2607.00154#bib.bib29),[20](https://arxiv.org/html/2607.00154#bib.bib23)\]\. Rather than introducing new attention mechanisms or task\-specific heuristics, the proposed framework operates under a fixed and widely adopted training protocol\[[17](https://arxiv.org/html/2607.00154#bib.bib1)\], enabling direct comparison to existing Transformer\-based forecasting models while automating architectural composition\.

## 3Methodology

This section describes theEvoTSframework, including genome representation, evolutionary dynamics, and evaluation\.

### 3\.1Problem Setting

We study time\-series forecasting under three standard settings: univariate\-to\-univariate \(S→\\rightarrowS\), multivariate\-to\-univariate \(M→\\rightarrowS\), and multivariate\-to\-multivariate \(M→\\rightarrowM\) prediction\. Given a historical input window of lengthLL, the objective is to predict future values over a forecasting horizonH∈\{96,192,336,720\}H\\in\\\{96,192,336,720\\\}\. Model performance is evaluated using mean squared error \(MSE\)\.

While all three settings are considered for completeness, we primarily emphasize the multivariate\-to\-multivariate setting, as it reflects the standard evaluation protocol adopted by recent Transformer\-based time\-series forecasting benchmarks\.

### 3\.2Evolutionary Architecture Search Framework

We propose an evolutionary neural architecture search framework for discovering Transformer\-like architectures tailored to time\-series forecasting tasks\. Rather than relying on a fixed, manually designed architecture, the framework explores a structured space of neural architectures using population\-based evolutionary optimization, following established neuroevolutionary principles\[[25](https://arxiv.org/html/2607.00154#bib.bib2)\]\.

The evolutionary process follows a steady\-state paradigm\[[28](https://arxiv.org/html/2607.00154#bib.bib4)\], in which new candidate architectures are generated, evaluated, and immediately considered for inclusion in the population\. Unlike generational evolution, steady\-state evolution allows continuous refinement of the population and incremental improvement of candidate solutions, making it particularly well suited to computationally expensive evaluation settings such as neural architecture search\.

### 3\.3Genome Representation

Each individual in the population is represented by a modular genome encoding a Transformer\-like forecasting architecture\. Modular and indirect genome representations have been shown to support scalable neuroevolution of deep neural networks\[[26](https://arxiv.org/html/2607.00154#bib.bib3)\]\. In our framework, the genome specifies \(i\) an input tokenization strategy, \(ii\) an ordered sequence of processing blocks, and \(iii\) an output projection head\.

Architectural hyperparameters–including embedding dimensionality, number of attention heads, convolutional kernel sizes, and block ordering–are explicitly encoded in the genome, enabling fine\-grained structural variation during evolution\.

#### Evolution of Input and Output Heads

Both the input tokenization module and the output projection head are encoded as part of the genome and are optimized through evolution\. The input head determines how raw time\-series observations are mapped into a shared token representation, while the output head defines how the learned token sequence is projected to the forecasting targets\.

Evolution controls the*type*of head selected as well as its internal architectural parameters, including encoder variants, kernel sizes, pooling strategies, and projection dimensionalities\. However, heads are subject to structural constraints: exactly one tokenization module is applied at the input of the network, and exactly one output projection is applied at the end\. Their position and multiplicity are fixed, and they are not interleaved with processing blocks\.

Importantly, the internal composition of the heads is not evolved independently of the genome\. Instead, head variants are selected deterministically based on compatibility with the tokenization strategy and the processing blocks specified by the genome\. For example, frequency\-domain encoders are instantiated only when required by frequency\-based tokenization or processing blocks\. This design enforces architectural consistency while allowing evolution to indirectly influence head structure through upstream design choices\.

#### Tokenization

Tokenization is an architectural choice applied once at model construction time\. The tokenizer defines the token axis and initial representation on which all subsequent processing blocks operate\. The tokenizer is selected from a fixed pool comprising time\-wise, variable\-wise, patch\-based, and grouped cross\-variable tokenization schemes\.

All tokenization modules map raw time\-series inputs to a shared tensor interface of shape\[B,T,D\]\[B,T,D\], whereBBdenotes batch size,TTthe number of tokens, andDDthe embedding dimension\. This shared interface ensures compatibility across heterogeneous processing blocks\.

Tokenization modules may optionally incorporate skip connections from the raw input to preserve low\-level information during encoding\. These skip connections are local to the tokenization stage and are not dynamically inserted between processing blocks\.

#### Architectural Constraints

To bound the search space while preserving architectural flexibility, genomes are constrained to form architectures consisting of a single tokenization stage followed by a sequence of processing blocks and a final output projection\. All processing blocks operate on the shared token representation produced by the tokenizer\.

Dimensional compatibility between blocks is enforced by construction, and projection layers are restricted to appear only at the input and output boundaries of the architecture\. Intermediate retokenization, token\-space transitions, or dynamic changes in token structure are reserved for future work\.

#### Supported Block Types

The modular genome supports a heterogeneous set of Transformer\-inspired and token\-mixing processing blocks\. Attention\-based blocks include standard multi\-head self\-attention as well as inverted and cross\-variable attention variants commonly used in time\-series forecasting\. In addition to attention mechanisms, the search space includes convolutional token\-mixing blocks that operate along the token dimension, providing an alternative means of capturing local temporal dependencies\.

Multiple block types may coexist within a single genome\. Compatibility between heterogeneous blocks is achieved by enforcing a common token interface rather than by introducing intermediate tokenization or projection layers\. Residual \(skip\) connections are applied within blocks as part of their standard definitions to support optimization stability and effective gradient flow\.

Although frequency\-domain processing blocks and encoders are available within the architectural vocabulary, they are instantiated only when explicitly required by the genome\-defined tokenization or block types\. As a result, frequency\-domain encoders are not universally present and may be entirely absent from evolved architectures if evolution does not favor such representations\.

Output heads support both single\-target and multi\-target forecasting through flexible projection and pooling mechanisms\. While the current block vocabulary reflects components commonly used in Transformer\-based forecasting models, the genome design is block\-agnostic and readily extensible to additional block types\.

Figure[1](https://arxiv.org/html/2607.00154#S3.F1)provides a high\-level overview of the modular architecture search space\.

![Refer to caption](https://arxiv.org/html/2607.00154v1/figs/evots_schem.png)Figure 1:High\-level schematic of the modular architecture search space\.

### 3\.4Population Initialization

The initial population is constructed using a combination of randomly sampled genomes and a small number of seed architectures\. Seed genomes correspond to simple, valid Transformer\-like configurations and serve to stabilize the early stages of evolutionary search\. Randomly initialized individuals are included to promote architectural diversity and reduce bias toward any single design pattern\.

### 3\.5Structural Repair Mechanism

Evolutionary variation operators may produce invalid or ill\-formed architectures\. To ensure executability, a deterministic repair mechanism is applied after mutation or recombination\. The repair procedure enforces dimensional compatibility between modules, ensures the presence of required components such as tokenization and output projections, and maintains valid computational connectivity\.

Repair does not introduce additional stochasticity and does not alter the evolutionary objective, enabling unrestricted exploration of the search space while guaranteeing that all evaluated individuals correspond to valid neural networks\.

### 3\.6Variation Operators

New architectures are generated using evolutionary variation operators adapted to the modular genome representation\. Mutation operators include block insertion, removal, replacement, and parameter perturbation\. Recombination operators combine architectural components from two parent genomes\.

When architectural compatibility permits, offspring architectures may optionally reuse learned parameters from parent models\. Such parameter reuse can improve sample efficiency in evolutionary architecture search, particularly when training deep neural networks from scratch\. In the experiments reported in this work, weight inheritance was not active during search; this capability is included in the framework for future use\.

### 3\.7Evolutionary Search Dynamics

The evolutionary search begins by initializing a population of candidate architectures sampled from the defined genome space\. At each evolutionary step, parent individuals are selected using tournament selection\. Offspring architectures are generated through mutation and recombination, followed by deterministic repair to ensure structural validity\.

Each offspring is trained and evaluated to obtain its fitness value\. In the steady\-state setting, evaluated offspring are immediately compared against existing population members and replace inferior individuals if they achieve superior fitness\. This process continues until a predefined termination criterion is met, such as a maximum number of architecture evaluations\.

Algorithm[1](https://arxiv.org/html/2607.00154#alg1)provides a concise summary of the steady\-state evolutionary architecture search procedure used in this work\.

### 3\.8Fitness Evaluation

Each candidate architecture is trained using a fixed training protocol and evaluated on a validation set to compute its fitness\. Fitness is defined as the validation mean squared error for the corresponding forecasting task and horizon\. Training is performed for a fixed number of epochs with early stopping based on validation performance\.

Evaluations of individuals are independent and can be executed in parallel, enabling efficient utilization of available computational resources\. The evolutionary algorithm itself remains agnostic to the underlying hardware configuration\.

### 3\.9Selection and Replacement

Selection is performed using tournament selection, favoring individuals with lower validation error while maintaining population diversity\. In the steady\-state setting, newly evaluated offspring replace the worst\-performing individuals in the population if they achieve superior fitness\.

### 3\.10Final Model Selection

The architecture with the lowest validation MSE observed during evolutionary search is selected, and all reported results correspond to this model\.

Algorithm 1Steady\-State Evolutionary Architecture Search0:Population size

NN, maximum evaluations

EmaxE\_\{\\max\}
1:Initialize population

𝒫=\{g1,…,gN\}\\mathcal\{P\}=\\\{g\_\{1\},\\dots,g\_\{N\}\\\}with random valid genomes

2:for all

g∈𝒫g\\in\\mathcal\{P\}do

3:Train model defined by

ggand evaluate fitness

f​\(g\)f\(g\)
4:endfor

5:

e←Ne\\leftarrow N
6:while

e<Emaxe<E\_\{\\max\}do

7:Select parent\(s\)

gpg\_\{p\}from

𝒫\\mathcal\{P\}via tournament selection

8:Generate offspring genome

gog\_\{o\}via mutation and/or recombination

9:Apply repair operator to

gog\_\{o\}to ensure structural validity

10:ifarchitectural compatibility permitsthen

11:Initialize

gog\_\{o\}with inherited parameters from

gpg\_\{p\}
12:endif

13:Train model defined by

gog\_\{o\}and evaluate fitness

f​\(go\)f\(g\_\{o\}\)
14:

e←e\+1e\\leftarrow e\+1
15:Identify worst individual

gw∈𝒫g\_\{w\}\\in\\mathcal\{P\}
16:if

f​\(go\)<f​\(gw\)f\(g\_\{o\}\)<f\(g\_\{w\}\)then

17:Replace

gwg\_\{w\}with

gog\_\{o\}in

𝒫\\mathcal\{P\}
18:endif

19:endwhile

20:returnbest genome

g∗∈𝒫g^\{\*\}\\in\\mathcal\{P\}

## 4Experimental Setup

Unless otherwise stated, all datasets, input lengths, prediction horizons, training procedures, and evaluation protocols follow the experimental configuration described in the iTransformer study\[[17](https://arxiv.org/html/2607.00154#bib.bib1)\]\. This ensures direct comparability between our results and previously reported Transformer\-based baselines\.

### 4\.1Datasets

We evaluate the proposed method on four widely used benchmarks from the Electricity Transformer Temperature \(ETT\) dataset family: ETTh1, ETTh2, ETTm1, and ETTm2\. These datasets consist of multivariate time\-series measurements collected from electricity transformers, including load, oil temperature, and related operational variables\. Following standard practice, ETTh datasets are sampled at an hourly resolution, while ETTm datasets are sampled at a 15\-minute resolution\.

All datasets are split chronologically into training, validation, and test sets using the standard partitioning protocol adopted by prior Transformer\-based forecasting studies\. No future information is used during training or validation\.

### 4\.2Forecasting Settings

Experiments are conducted under three forecasting settings:

- •Univariate\-to\-univariate \(S→\\rightarrowS\): a single input variable is used to predict the same variable in the future\.
- •Multivariate\-to\-univariate \(M→\\rightarrowS\): all input variables are used to predict a single target variable\.
- •Multivariate\-to\-multivariate \(M→\\rightarrowM\): all input variables are jointly predicted\.

For all settings, forecasting horizons ofH∈\{96,192,336,720\}H\\in\\\{96,192,336,720\\\}are considered\. The multivariate\-to\-multivariate setting is emphasized, as it aligns with the standard evaluation protocol used by recent Transformer\-based forecasting models\.

### 4\.3Data Preprocessing

Input time series are normalized using per\-variable z\-score normalization, where statistics are computed from the training set only and applied consistently to validation and test data\. Sliding windows with fixed input length \(L=384L=384\) are used to construct training samples, following established practice in Transformer\-based forecasting\.

### 4\.4Baselines

We compare the proposed evolutionary approach against iTransformer, a strong Transformer\-based baseline for multivariate time\-series forecasting\. iTransformer results are taken directly from the original publication and reported under identical datasets, prediction horizons, and evaluation metrics\. Although the iTransformer paper reports several competitive architectures, iTransformer is used here as a representative Transformer baseline for direct comparison\.

### 4\.5Training and Evolution Protocol

All candidate architectures are trained using a uniform optimization procedure to ensure fair comparison\. Models are optimized with AdamW using a fixed learning rate of1×10−41\\times 10^\{\-4\}, weight decay3×10−43\\times 10^\{\-4\}, and batch size 32\. Early stopping is enabled based on validation MSE with a patience of 10\. Each evolutionary evaluation consists of a short initial training phase \(20 epochs\)\. Fitness is defined as validation MSE\.

The evolutionary search follows a steady\-state strategy with a population size of 20 and a maximum budget of 300 architecture evaluations\. Tournament selection \(size 2\) is used, with mutation and crossover rates of 0\.3 and 0\.7, respectively\. All experiments use a fixed random seed to ensure reproducibility\. The initial population includes a small number of shallow seed architectures corresponding to standard Transformer variants \(e\.g\., iTransformer\- and PatchTST\-style configurations with 4 layers\), while the remaining individuals are randomly initialized to preserve architectural diversity\.

### 4\.6Runtime Measurement and Hardware

To provide a coarse indication of computational cost, we report wall\-clock training time for individual model evaluations\. Each candidate architecture is trained and validated on a single GPU\. Evolutionary evaluations are executed in parallel using three independent processes, each spawning and training one model at a time\.

All experiments were conducted on a Linux\-based system equipped with three NVIDIA RTX A6000 GPUs \(48 GB memory each\) and an AMD Ryzen Threadripper PRO 5995WX CPU\. Runtime measurements reported in Section[5](https://arxiv.org/html/2607.00154#S5)reflect wall\-clock performance under this hardware configuration\.

## 5Results

### 5\.1Multivariate\-to\-Multivariate Forecasting Performance

Table[1](https://arxiv.org/html/2607.00154#S5.T1)reports mean squared error \(MSE\) for the multivariate\-to\-multivariate \(M→\\rightarrowM\) forecasting setting on four ETT benchmark datasets \(ETTh1, ETTh2, ETTm1, ETTm2\) with prediction horizons of 96, 192, 336, and 720\. Results for iTransformer are taken directly from the original publication and correspond to identical datasets, horizons, and evaluation protocols\. As is standard in NAS, validation MSE serves as both the fitness criterion during search and the reported performance metric\. Figure[2](https://arxiv.org/html/2607.00154#S5.F2)visualizes the distribution of forecasting errors across horizons and datasets for the same experimental setting\.

Across all evaluated datasets and prediction horizons, the architectures discovered by the proposed evolutionary framework achieve MSE that is lower than or comparable to the strongest results reported for the ETT benchmarks in the iTransformer study\[[17](https://arxiv.org/html/2607.00154#bib.bib1)\], under identical datasets, horizons, and evaluation protocols\. Performance gains are modest at shorter horizons and become more pronounced as the prediction horizon increases\. For example, on ETTh1, MSE is reduced from 0\.376 to 0\.3224 at horizon 96 and from 0\.481 to 0\.3724 at horizon 720, corresponding to relative error reductions of 14\.2% and 22\.6%, respectively\. Similar trends are observed on ETTh2, ETTm1, and ETTm2, indicating that the benefits of evolutionary architecture search are particularly evident in longer\-horizon forecasting scenarios\.

The iTransformer study\[[17](https://arxiv.org/html/2607.00154#bib.bib1)\]evaluates a range of Transformer\-based forecasting models, including Autoformer, FEDformer, Informer, and Crossformer, and reports that different architectures achieve the best performance depending on dataset and prediction horizon\. In particular, iTransformer does not uniformly dominate all baselines across the ETT benchmarks but serves as a strong and representative modern Transformer baseline under a consistent evaluation protocol\.

In this context, our comparison against iTransformer should be interpreted as a controlled reference rather than a comparison to the strongest possible model in all settings\. The proposed evolutionary approach consistently improves upon iTransformer across datasets and horizons, and the distributional comparisons shown in Figure[2](https://arxiv.org/html/2607.00154#S5.F2)indicate that these gains are robust across short\- and long\-range forecasting regimes\. While direct numerical comparison with all baselines reported in\[[17](https://arxiv.org/html/2607.00154#bib.bib1)\]is beyond the scope of this study, the observed improvements relative to iTransformer suggest that the evolved architectures are competitive with, and in many cases comparable to, the best\-performing Transformer variants reported in prior work\.

Table 1:MSE comparison under multivariate\-to\-multivariate \(M→\\rightarrowM\) forecasting\. Lower is better\.![Refer to caption](https://arxiv.org/html/2607.00154v1/x1.png)\(a\)ETTh1
![Refer to caption](https://arxiv.org/html/2607.00154v1/x2.png)\(b\)ETTh2
![Refer to caption](https://arxiv.org/html/2607.00154v1/x3.png)\(c\)ETTm1
![Refer to caption](https://arxiv.org/html/2607.00154v1/x4.png)\(d\)ETTm2

Figure 2:Distribution of MSE across forecasting horizons \(9696,192192,336336,720720\) forM→MM\\rightarrow Mforecasting\. Each box summarizes performance variability across horizons for a given model\.
### 5\.2Univariate and Multivariate\-to\-Univariate Performance

For completeness, Table[2](https://arxiv.org/html/2607.00154#S5.T2)reports results for the univariate\-to\-univariate \(S→\\rightarrowS\) and multivariate\-to\-univariate \(M→\\rightarrowS\) forecasting settings\. In the univariate setting, the evolved architectures achieve consistently strong performance across all datasets and horizons, confirming that the search space remains effective even in simple forecasting regimes\.

In contrast, performance in the multivariate\-to\-univariate setting exhibits higher variance and, in several cases, degradation relative to the multivariate\-to\-multivariate setting for the same evolved architectures\. This behavior suggests that the current architectural design and output\-head configurations are better suited to joint multivariate prediction, where shared representations across variables can be exploited more effectively\. These results are therefore presented as diagnostic evidence rather than primary benchmarks\.

Table 2:MSE results for univariate\-to\-univariate \(S→\\rightarrowS\) and multivariate\-to\-univariate \(M→\\rightarrowS\) forecasting\.
### 5\.3Computational Cost and Runtime Analysis

To assess computational feasibility, we measure wall\-clock training time and CPU utilization for evolutionary evaluations\. We report approximate average CPU parallelism, estimated as

avg\_cores≈user\+sysreal\.\\text\{avg\\\_cores\}\\approx\\frac\{\\text\{user\}\+\\text\{sys\}\}\{\\text\{real\}\}\.
Aggregated runtime statistics \(Table[3](https://arxiv.org/html/2607.00154#S5.T3)\) show that average CPU utilization remains stable at approximately 3\.6–3\.9 cores across most configurations, largely independent of prediction horizon or dataset\. This indicates that the computational cost of the evolutionary procedure scales predictably and does not introduce excessive overhead beyond model training\.

Longer prediction horizons are occasionally associated with reduced wall\-clock time, which may reflect smoother long\-term dynamics and faster convergence in some settings\. However, several multi\-output configurations at horizon 720 exhibit increased runtime, reflecting the higher computational cost of producing large output tensors and reduced gradient signal\-to\-noise during optimization\. Overall, these results demonstrate that the proposed evolutionary search remains computationally tractable for realistic time\-series forecasting workloads\.

Table 3:Runtime statistics summary across all datasets \(hours\)\.
### 5\.4Architecture Analysis of Evolved Models

Table 4:Summary of architectural characteristics of best evolved models across datasets and prediction horizons \(M→MM\\rightarrow M\)\.To gain insight into the structures discovered by the evolutionary search, we analyze the best fine\-tuned genomes produced across datasets and prediction horizons\. Rather than presenting full genome specifications, we summarize key architectural characteristics of the evolved models, including tokenizer choice, block composition, and the configuration of input and output heads\. Representative architectural patterns are summarized in Table[4](https://arxiv.org/html/2607.00154#S5.T4), where values report mean and range across four datasets per horizon\.

Across all datasets and horizons, the evolved architectures consistently combine multiple Transformer\-inspired processing mechanisms within a shared token space\. Inverted\-attention, standard self\-attention, and convolutional token\-mixing blocks frequently co\-exist within the same model, indicating that hybrid designs are systematically favored over architectures relying on a single inductive bias\. This suggests that the evolutionary process benefits from balancing global dependency modeling \(attention\-based blocks\) with local pattern extraction \(convolutional blocks\)\.

Tokenizer selection generally aligns with established architectural families\. The majority of best\-performing models adopt variable\-wise tokenization, consistent with iTransformer\-style designs, while patch\-based or cross\-variable tokenization does not appear in the top\-performing single\-stage architectures for the multivariate\-to\-multivariate setting considered here\. Nevertheless, within a fixed tokenizer, substantial diversity is observed in block ordering and composition, highlighting the importance of internal architectural arrangement beyond high\-level family selection\.

All architectures analyzed in this study operate under a single\-stage constraint, with a fixed tokenization applied once at the input\. The present experiments operate under a single\-stage constraint\. Multistage architectures with intermediate retokenization are not included in the current search space and are left for future work\.

Input and output heads are configured to match the selected tokenization scheme and are not evolved independently of the block structure\. In particular, variable\-wise tokenization is consistently paired with a variable head, while cross\-variable heads are not activated in the reported models\. FFT\-based encoder variants appear only sporadically within input heads at shorter horizons and are absent at longer horizons, suggesting that frequency\-domain preprocessing is not universally beneficial under the fixed training protocol used in this study\.

Overall, the architectural analysis indicates that evolutionary search favors compact, single\-stage Transformer\-like models that flexibly combine attention and convolutional token\-mixing mechanisms under a shared token interface\. These results highlight the role of internal block composition and ordering–rather than architectural depth or token\-space transitions–as key drivers of performance in the explored forecasting regimes\.

## 6Discussion

The results presented in this work demonstrate that evolutionary architecture search provides a practical and effective alternative to manual Transformer design for time\-series forecasting\. Across four ETT benchmarks and multiple prediction horizons, the proposed framework consistently discovers architectures that achieve strong performance relative to established Transformer\-based references evaluated under identical experimental protocols\.

A key empirical observation is that performance gains increase with the prediction horizon\. At short horizons, most modern Transformerbased models already perform well, as forecasting is dominated by local temporal continuity and short\-range dependencies\. As the prediction horizon grows, long\-range temporal structure and cross\-variable interactions become increasingly important, exposing limitations of fixed, hand\-designed architectures\. In this regime, the flexibility of evolutionary search becomes more advantageous, enabling the discovery of architectural compositions that better balance global and local modeling requirements\.

Architectural analysis reveals that the best\-performing evolved models do not converge to a single canonical Transformer design\. Instead, they frequently integrate multiple Transformer\-inspired processing mechanisms within a shared token representation\. Inverted attention, standard self\-attention, and convolutional token\-mixing blocks often co\-exist within the same architecture\. This hybrid composition suggests that no single inductive bias is sufficient across datasets and horizons, and that combining complementary mechanisms enables more robust forecasting behavior\. The evolutionary process naturally favors such compositions by retaining only those components that contribute to fitness improvements\.

Another consistent finding is the dominance of variable\-wise tokenization in the multivariate\-to\-multivariate forecasting setting\. All best\-performing architectures analyzed in this study operate on variable tokens and employ a single\-stage processing pipeline\. The present experiments operate under a single\-stage constraint\. Multistage architectures with intermediate retokenization are not included in the current search space and are left for future work\. The absence of patch\-based or cross\-variable tokenization in the top\-performing models should therefore be interpreted in the context of the imposed single\-stage design rather than as a general limitation of such tokenization strategies\.

The strongest performance gains are observed in the multivariate\-to\-multivariate forecasting regime\. Joint prediction of all variables provides a richer learning signal and allows architectures to exploit shared representations across correlated time series\. In contrast, univariate and multivariate\-to\-univariate settings impose stronger output constraints, reducing the potential benefits of architectural adaptation\. These findings indicate that evolutionary architecture search is particularly well\-suited to forecasting problems with high structural complexity and strong cross\-variable dependencies\.

From a computational perspective, the steady\-state evolutionary design enables efficient use of parallel resources while avoiding disruptive population turnover\. Runtime measurements indicate stable CPU utilization across datasets and horizons, with increased cost primarily associated with larger output dimensionality at long horizons\. Despite the computational expense of evolutionary evaluation, the results demonstrate that the approach remains tractable for realistic forecasting workloads and benefits from weight inheritance and asynchronous evaluation\.

Overall, the results suggest that the primary advantage of evolutionary search in this context lies not in increasing model size or depth, but in automating the discovery of effective architectural compositions\. By removing the need to commit to a fixed design template, the framework adapts Transformer\-like architectures to the statistical structure of the forecasting task and prediction horizon\.

### Future Work

Several directions for future research naturally follow from this study\. First, extending the evolutionary search to multistage architectures with intermediate retokenization would allow exploration of transitions between variable\-wise, patch\-based, and cross\-variable token spaces within a single model, which may benefit data with heterogeneous temporal scales\.

Second, expanding the block vocabulary to include additional Transformer\-derived and non\-Transformer modules could further enrich the search space and enable discovery of novel hybrid architectures\. This includes more expressive convolutional operators, frequency\-domain processing blocks \(e\.g\., FFT\-based transformations\), and adaptive attention mechanisms, enabling joint exploration of time\- and frequency\-domain representations\.

Third, incorporating multi\-objective fitness criteria that jointly optimize forecasting accuracy, computational cost, and model complexity would allow the evolutionary process to balance performance and efficiency, which is particularly relevant for resource\-constrained deployment\. Finally, evaluating the framework on a broader range of real\-world datasets would provide deeper insight into its generality and robustness\.

## References

- \[1\]A\. F\. Ansari, O\. Shchur, J\. Küken, A\. Auer, B\. Han, P\. Mercado, S\. S\. Rangapuram, H\. Shen, L\. Stella, X\. Zhang, M\. Goswami, S\. Kapoor, D\. C\. Maddix, P\. Guerron, T\. Hu, J\. Yin, N\. Erickson, P\. M\. Desai, H\. Wang, H\. Rangwala, G\. Karypis, Y\. Wang, and M\. Bohlke\-Schneider\(2025\)Chronos\-2: from univariate to universal forecasting\.arXiv preprint arXiv:2510\.15821\.External Links:[Link](https://arxiv.org/abs/2510.15821)Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[2\]A\. F\. Ansari, L\. Stella, C\. Turkmen, X\. Zhang, P\. Mercado, H\. Shen, O\. Shchur, S\. S\. Rangapuram, S\. Pineda Arango, S\. Kapoor, J\. Zschiegner, D\. C\. Maddix, M\. W\. Mahoney, K\. Torkkola, A\. Gordon Wilson, M\. Bohlke\-Schneider, and Y\. Wang\(2024\)Chronos: learning the language of time series\.Transactions on Machine Learning Research\.External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=gerNCVqqtR)Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[3\]\(2021\)Searching the search space of vision transformer\.Advances in Neural Information Processing Systems34,pp\. 8714–8726\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px3.p1.1)\.
- \[4\]A\. Das, W\. Kong, R\. Sen, and Y\. Zhou\(2024\)A decoder\-only foundation model for time\-series forecasting\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[5\]S\. Dooley, G\. S\. Khurana, C\. Mohapatra, S\. V\. Naidu, and C\. White\(2023\)ForecastPFN: synthetically\-trained zero\-shot forecasting\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[6\]A\. ElSaid and T\. Desell\(2025\)CG\-cants\-n: a versatile graph\-based framework for scalable and adaptive problem solving across domains\.pp\. 263–266\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1)\.
- \[7\]A\. ElSaid, J\. Karns, Z\. Lyu, A\. G\. Ororbia, and T\. Desell\(2021\)Continuous ant\-based neural topology search\.pp\. 291–306\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1)\.
- \[8\]A\. ElSaid, A\. G\. Ororbia, and T\. J\. Desell\(2020\)Ant\-based neural topology search \(ants\) for optimizing recurrent networks\.pp\. 626–641\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1)\.
- \[9\]T\. Elsken, J\. H\. Metzen, and F\. Hutter\(2019\)Neural architecture search: a survey\.Journal of Machine Learning Research20\(55\),pp\. 1–21\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p3.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1)\.
- \[10\]C\. Gong and D\. Wang\(2022\)Nasvit: neural architecture search for efficient vision transformers with gradient conflict\-aware supernet training\.ICLR Proceedings 2022\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px3.p2.1)\.
- \[11\]M\. Goswami, K\. Szafer, A\. Choudhry, Y\. Cai, S\. Li, and A\. Dubrawski\(2024\)Moment: a family of open time\-series foundation models\.arXiv preprint arXiv:2402\.03885\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[12\]N\. Gruver, M\. Finzi, S\. Qiu, and A\. G\. Wilson\(2023\)Large language models are zero\-shot time series forecasters\.Advances in Neural Information Processing Systems36,pp\. 19622–19635\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[13\]S\. B\. Hoo, S\. Müller, D\. Salinas, and F\. Hutter\(2025\)From tables to time: how tabpfn\-v2 outperforms specialized time series forecasting models\.External Links:2501\.02945,[Link](https://arxiv.org/abs/2501.02945)Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[14\]M\. Jin, S\. Wang, L\. Ma, Z\. Chu, J\. Y\. Zhang, X\. Shi, P\. Chen, Y\. Liang, Y\. Li, S\. Pan,et al\.\(2023\)Time\-llm: time series forecasting by reprogramming large language models\.arXiv preprint arXiv:2310\.01728\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[15\]B\. Lim, S\. Ö\. Arık, N\. Loeff, and T\. Pfister\(2021\)Temporal fusion transformers for interpretable multi\-horizon time series forecasting\.International journal of forecasting37\(4\),pp\. 1748–1764\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p1.1)\.
- \[16\]H\. Liu, K\. Simonyan, and Y\. Yang\(2018\)Darts: differentiable architecture search\.arXiv preprint arXiv:1806\.09055\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p2.1)\.
- \[17\]Y\. Liu, T\. Hu, H\. Zhang, H\. Wu, S\. Wang, L\. Ma, and M\. Long\(2024\)ITransformer: inverted transformers are effective for time series forecasting\.InInternational Conference on Representation Learning,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 11116–11140\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/2ea18fdc667e0ef2ad82b2b4d65147ad-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2607.00154#S1.p4.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2607.00154#S4.p1.1),[§5\.1](https://arxiv.org/html/2607.00154#S5.SS1.p2.1),[§5\.1](https://arxiv.org/html/2607.00154#S5.SS1.p3.1),[§5\.1](https://arxiv.org/html/2607.00154#S5.SS1.p4.1)\.
- \[18\]Z\. Lu, I\. Whalen, V\. Boddeti, Y\. Dhebar, K\. Deb, E\. Goodman, and W\. Banzhaf\(2019\)Nsga\-net: neural architecture search using multi\-objective genetic algorithm\.pp\. 419–427\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1)\.
- \[19\]J\. Murphy, D\. Kar, J\. Karns, and T\. Desell\(2024\)EXA\-gp: unifying graph\-based genetic programming and neuroevolution for explainable time series forecasting\.pp\. 523–526\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1)\.
- \[20\]A\. Ororbia, A\. ElSaid, and T\. Desell\(2019\)Investigating recurrent neural network memory structures using neuro\-evolution\.pp\. 446–455\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px4.p1.1)\.
- \[21\]H\. Pham, M\. Guan, B\. Zoph, Q\. Le, and J\. Dean\(2018\)Efficient neural architecture search via parameters sharing\.pp\. 4095–4104\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p2.1)\.
- \[22\]K\. Rasul, A\. Ashok, A\. R\. Williams, A\. Khorasani, G\. Adamopoulos, R\. Bhagwatkar, M\. Biloš, H\. Ghonia, N\. Hassen, A\. Schneider,et al\.\(2023\)Lag\-llama: towards foundation models for time series forecasting\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[23\]E\. Real, A\. Aggarwal, Y\. Huang, and Q\. V\. Le\(2019\)Regularized evolution for image classifier architecture search\.pp\. 4780–4789\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px4.p1.1)\.
- \[24\]D\. So, Q\. Le, and C\. Liang\(2019\)The evolved transformer\.pp\. 5877–5886\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px3.p2.1)\.
- \[25\]K\. O\. Stanley, J\. Clune, J\. Lehman, and R\. Miikkulainen\(2019\)Designing neural networks through neuroevolution\.Nature Machine Intelligence1\(1\),pp\. 24–35\.Cited by:[§3\.2](https://arxiv.org/html/2607.00154#S3.SS2.p1.1)\.
- \[26\]K\. O\. Stanley and R\. Miikkulainen\(2002\)Evolving neural networks through augmenting topologies\.Evolutionary computation10\(2\),pp\. 99–127\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px4.p1.1),[§3\.3](https://arxiv.org/html/2607.00154#S3.SS3.p1.1)\.
- \[27\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.Advances in neural information processing systems30\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p1.1)\.
- \[28\]M\. D\. Vose\(1999\-08\)The simple genetic algorithm: foundations and theory\.The MIT Press\.External Links:ISBN 9780262285643,[Document](https://dx.doi.org/10.7551/mitpress/6229.001.0001),[Link](https://doi.org/10.7551/mitpress/6229.001.0001)Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px4.p1.1),[§3\.2](https://arxiv.org/html/2607.00154#S3.SS2.p2.1)\.
- \[29\]G\. Woo, C\. Liu, A\. Kumar, C\. Xiong, S\. Savarese, and D\. Sahoo\(2024\)Unified training of universal time series forecasting transformers, 2024\.URL https://arxiv\. org/abs/2402\.025927\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[30\]H\. Wu, J\. Xu, J\. Wang, and M\. Long\(2021\)Autoformer: decomposition transformers with auto\-correlation for long\-term series forecasting\.Advances in neural information processing systems34,pp\. 22419–22430\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p1.1)\.
- \[31\]H\. Xue and F\. D\. Salim\(2023\)Promptcast: a new prompt\-based learning paradigm for time series forecasting\.IEEE Transactions on Knowledge and Data Engineering36\(11\),pp\. 6851–6864\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.
- \[32\]Y\. Zhang and J\. Yan\(2023\)Crossformer: transformer utilizing cross\-dimension dependency for multivariate time series forecasting\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=vSVLM2j9eie)Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p1.1)\.
- \[33\]H\. Zhou, S\. Zhang, J\. Peng, S\. Zhang, J\. Li, H\. Xiong, and W\. Zhang\(2021\)Informer: beyond efficient transformer for long sequence time\-series forecasting\.pp\. 11106–11115\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p1.1)\.
- \[34\]T\. Zhou, P\. Niu, L\. Sun, R\. Jin,et al\.\(2023\)One fits all: power general time series analysis by pretrained lm\.Advances in neural information processing systems36,pp\. 43322–43355\.Cited by:[§2](https://arxiv.org/html/2607.00154#S2.SS0.SSS0.Px1.p2.1)\.