StateFlow: Dual-State Recurrent Modeling for Long-Horizon Time Series Forecasting

# StateFlow: Dual-State Recurrent Modeling for Long-Horizon time series Forecasting
Source: [https://arxiv.org/html/2607.00197](https://arxiv.org/html/2607.00197)
Haroon Gharwi1\*Yue Dai2\*Kai Shu3\+ \*Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA \+Department of Computer Science, Emory University, Atlanta, GA, USA 1hgharwi@hawk\.illinoistech\.edu2ydai21@illinoistech\.edu3kai\.shu@emory\.edu

###### Abstract

Long\-horizon multivariate time series forecasting \(LTSF\) remains challenging due to non\-stationarity, regime shifts, and error accumulation\. The Variability\-Aware Recursive Neural Network \(VARNN\) was recently introduced to track such variability by maintaining a residual\-memory state driven by one\-step prediction errors\. However, its original formulation is limited to one\-step sequence regression and does not directly support multi\-step forecasting\. In this work, we extend VARNN to long\-horizon forecasting and introduce StateFlow, a recurrent forecasting framework that uses VARNN as a dual\-state recurrent backbone to capture two complementary signals from the lookback sequence: a hidden\-state trajectory representing primary temporal dynamics, including trend, seasonality, level changes, and recurring patterns, and a residual\-memory trajectory representing structured local prediction deviations, driven from a nonlinear recurrent transformation of errors between one\-step base predictions and observed values\. A chunk\-based decoder separately summarizes these trajectories and maps them to the future horizon for direct multi\-step forecasting\. We further employ a two\-stage optimization strategy that first trains the VARNN encoder through a one\-step base prediction objective to optimize the internal representations over the lookback sequence, and then trains a horizon\-specific decoder for direct multi\-step forecasting\. Experiments on standard LTSF benchmarks show that StateFlow achieves competitive performance against strong linear, recurrent, convolutional, and Transformer\-based baselines while preserving linear recurrent encoding and a compact model design\.

Keywords—Time Series Forecasting, non\-stationarity, residual learning, recurrent neural networks, distribution shift\.

## 1Introduction

Long\-horizon multivariate time series forecasting \(LTSF\) is a fundamental problem in applications such as energy demand prediction, traffic flow analysis, financial modeling, and environmental monitoring\(Hyndman and Athanasopoulos,[2021](https://arxiv.org/html/2607.00197#bib.bib1); Brockwell and Davis,[2002](https://arxiv.org/html/2607.00197#bib.bib2); De Gooijer and Hyndman,[2006](https://arxiv.org/html/2607.00197#bib.bib10)\)\. LTSF requires modeling extended temporal dependencies while remaining robust to non\-stationarity, distribution shift, and evolving seasonal structure\(Lim and Zohren,[2021](https://arxiv.org/html/2607.00197#bib.bib43); Baidya and Lee,[2024](https://arxiv.org/html/2607.00197#bib.bib46)\)\. As the prediction horizon increases, modeling errors tend to accumulate, making stable long\-term extrapolation particularly challenging\(Limet al\.,[2019](https://arxiv.org/html/2607.00197#bib.bib29); Oreshkinet al\.,[2020](https://arxiv.org/html/2607.00197#bib.bib36); Benidiset al\.,[2023](https://arxiv.org/html/2607.00197#bib.bib38)\)\.

Recent progress in LTSF has been largely driven by Transformer\-based architectures that leverage self\-attention to capture long\-range dependencies\. Patch\-based tokenization strategies and channel\-wise attention mechanisms have demonstrated strong empirical performance on standard benchmarks\(Nieet al\.,[2023](https://arxiv.org/html/2607.00197#bib.bib33); Liuet al\.,[2024](https://arxiv.org/html/2607.00197#bib.bib56)\)\. However, attention\-based models generally exhibit quadratic complexity with respect to input length and often rely on global interaction patterns that may introduce computational overhead when long lookback windows are required\.

Recurrent architectures, in contrast, scale linearly with sequence length and provide a natural causal structure for temporal modeling\. Classical recurrent models such as RNNs, LSTMs, and GRUs\(Elman,[1990](https://arxiv.org/html/2607.00197#bib.bib54); Hochreiter and Schmidhuber,[1997](https://arxiv.org/html/2607.00197#bib.bib8); Choet al\.,[2014](https://arxiv.org/html/2607.00197#bib.bib9)\)have been widely studied for sequence regression\. Nevertheless, their performance in long\-horizon forecasting settings has often been observed to lag behind modern Transformer\-based methods\(Zhouet al\.,[2021](https://arxiv.org/html/2607.00197#bib.bib30); Zenget al\.,[2023](https://arxiv.org/html/2607.00197#bib.bib48)\)\. One potential limitation is that conventional recurrence primarily captures content evolution through a single hidden state and does not explicitly model systematic deviation\. These local deviations may reflect drift, bias, or other non\-stationary changes in the observed sequence\.

Recent work introduced the Variability\-Aware Recursive Neural Network \(VARNN\) for one\-step sequence regressionGharwi and Shu \([2025](https://arxiv.org/html/2607.00197#bib.bib49)\)\. VARNN explicitly uses one\-step prediction residuals to update a dedicated nonlinear residual\-memory state, allowing structured local prediction deviations to be modeled separately from primary temporal dynamics\. However, its original formulation is limited to one\-step sequence regression, and its applicability to long\-horizon time series forecasting \(LTSF\) remains unexplored\.

To address this gap, we extend VARNN to the LTSF setting and introduce the StateFlow framework\. StateFlow uses VARNN as a dual\-state recurrent backbone to capture two complementary signals from the lookback sequence: a hidden\-state trajectory representing primary temporal dynamics, including trend, seasonality, level changes, and recurring patterns, and a residual\-memory trajectory representing structured local prediction deviations, driven from recurrent nonlinear transformation of errors between one\-step base predictions and observed lookback values\. A chunk\-based forecasting decoder separately summarizes these trajectories before mapping them to the future horizon\. This design preserves the distinction between temporal and deviation dynamics while reducing the parameter growth associated with directly flattening the full encoder trajectories\.

![Refer to caption](https://arxiv.org/html/2607.00197v1/varnn-LHF.png)Figure 1:Architecture of the proposed StateFlow framework with VARNN as the recurrent backbone\. VARNN encodes the lookback sequence into hidden\-state and residual\-memory trajectories, which are then separately summarized by a chunk\-based decoder and projected to the future horizon for direct multi\-step forecasting\.We further employ a two\-stage optimization strategy to bridge one\-step residual learning and long\-horizon forecasting\. In the first stage, the VARNN encoder is trained using a one\-step prediction objective over the lookback window, giving the residual\-memory state an explicit learning signal\. In the second stage, the pretrained encoder is frozen and a horizon\-specific decoder is trained for direct multi\-step forecasting\. This separation stabilizes optimization and allows the same pretrained VARNN encoder to be reused across multiple prediction horizons for a fixed dataset and lookback length, while a separate horizon\-specific decoder is trained for each horizon\.

The proposed StateFlow model preserves linear computational complexity with respect to input length through its recurrent encoder\. Our empirical results on standard LTSF benchmarks show that StateFlow achieves competitive performance against strong linear, convolutional, recurrent, and Transformer\-based baselines, while avoiding quadratic temporal self\-attention\. These results suggest that incorporating deviation dynamics into recurrence provides an efficient alternative to attention\-based LTSF models for long\-horizon forecasting\. Our contributions are summarized as follows:

- •We investigate whether recurrence can remain competitive for LTSF when deviation dynamics are explicitly incorporated into the state transition\.
- •We extend VARNN from one\-step sequence regression to long\-horizon time\-series forecasting by using its hidden and residual\-memory trajectories as dual\-state forecasting representations\.
- •We introduce a StateFlow framework, a recurrent trajectory forecasting framework with a chunk\-based horizon decoder that separately summarizes hidden dynamics and residual\-memory dynamics before direct multi\-step prediction\.
- •We employ a two\-stage optimization strategy that first learns residual\-aware recurrent representations through one\-step prediction and then trains horizon\-specific decoders for long\-horizon forecasting\.
- •We provide systematic experiments and ablations showing that residual\-memory recurrence is a competitive and parameter\-efficient inductive bias for long\-horizon forecasting\.

## 2Related Work

### 2\.1Statistical and Linear Models

Classical forecasting methods, including ARIMA and exponential smoothing, provide principled frameworks for modeling temporal structure under linearity and stationarity assumptions\(Boxet al\.,[2015](https://arxiv.org/html/2607.00197#bib.bib57); Hyndman and Athanasopoulos,[2021](https://arxiv.org/html/2607.00197#bib.bib1)\)\. Although effective in many settings, these methods may be limited in capturing complex nonlinear and non\-stationary dynamics\. Recent work has revisited lightweight linear architectures for long\-horizon time\-series forecasting\. LTSF\-Linear and DLinear demonstrate that carefully structured linear mappings, combined with suitable decomposition or normalization strategies, can outperform several more complex Transformer\-based models on standard benchmarks\(Zenget al\.,[2023](https://arxiv.org/html/2607.00197#bib.bib48)\)\. RLinear further highlights the importance of reversible normalization and channel\-independent modeling in the performance of linear forecasting methods\(Liet al\.,[2023](https://arxiv.org/html/2607.00197#bib.bib53)\)\. These findings suggest that carefully chosen temporal inductive biases can be as important as architectural complexity in long\-horizon forecasting\.

### 2\.2Transformer\-Based Models

Transformer architectures\(Vaswaniet al\.,[2017](https://arxiv.org/html/2607.00197#bib.bib55)\)have been widely adopted for time series forecasting due to their ability to capture long\-range dependencies through self\-attention mechanisms\. Informer\(Zhouet al\.,[2021](https://arxiv.org/html/2607.00197#bib.bib30)\)improves efficiency using sparse attention, while Autoformer and FEDformer incorporate decomposition and frequency\-domain modeling\(Wuet al\.,[2021](https://arxiv.org/html/2607.00197#bib.bib31); Zhouet al\.,[2022](https://arxiv.org/html/2607.00197#bib.bib32)\), PatchTST introduces patch\-based tokenization with channel\-independent forecasting strategies\(Nieet al\.,[2023](https://arxiv.org/html/2607.00197#bib.bib33)\), and iTransformer applies attention across variables by inverting the temporal and variate dimensions\(Liuet al\.,[2024](https://arxiv.org/html/2607.00197#bib.bib56)\)\. These approaches have achieved strong empirical performance on long\-horizon benchmarks such as ETT, Weather, and Electricity datasets\. Despite their effectiveness, attention\-based models typically retain quadratic complexity in sequence length or require additional approximations to reduce this cost\. Moreover, their reliance on global interaction patterns can introduce additional overhead when long lookback windows are used\. This motivates exploring alternative sequential inductive biases that preserve efficiency while remaining competitive\.

### 2\.3Recurrent Neural Networks

Recurrent neural networks, including vanilla RNNsElman \([1990](https://arxiv.org/html/2607.00197#bib.bib54)\), long short\-term memory networks \(LSTMs\)Hochreiter and Schmidhuber \([1997](https://arxiv.org/html/2607.00197#bib.bib8)\), and gated recurrent units \(GRUs\)Choet al\.\([2014](https://arxiv.org/html/2607.00197#bib.bib9)\)model nonlinear autoregressive temporal dynamics through hidden\-state transitions\. They provide a natural causal structure for sequence modeling and scale linearly with input length\. However, these conventional recurrent models typically compress temporal information into a single hidden state that integrates past information implicitly, which may limit their ability to explicitly track systematic prediction deviations under non\-stationarity\.

Recently, the Variability\-Aware Recursive Neural Network \(VARNN\)Gharwi and Shu \([2025](https://arxiv.org/html/2607.00197#bib.bib49)\)was introduced to explicitly model such deviations in one\-step sequence regression with covariate inputs\. Unlike conventional recurrent models that encode the covariate sequence only through latent states, VARNN additionally leverages the historical ground\-truth of observed sequence and computes the residual between each one\-step base prediction and its corresponding observed value, then transforms these residuals via a nonlinear recurrent update into a dedicated residual\-memory trajectory that conditions the subsequent step\. Prior results suggest that residuals can exhibit structured nonlinear dependencies that provide useful information for modeling temporal deviations\. However, VARNN was originally formulated for one\-step regression, and its effectiveness for long\-horizon time\-series forecasting has not been systematically studied\. In this work, we extend VARNN to the LTSF setting by using its hidden\-state and residual\-memory trajectories as forecasting representations and incorporating a chunk\-based horizon decoder with two\-stage optimization for direct multi\-step forecasting\.

## 3Methodology

### 3\.1Problem Formulation

Let𝐗1:L=\[𝐱1,𝐱2,…,𝐱L\]∈ℝL×D\\mathbf\{X\}\_\{1:L\}=\[\\mathbf\{x\}\_\{1\},\\mathbf\{x\}\_\{2\},\\ldots,\\mathbf\{x\}\_\{L\}\]\\in\\mathbb\{R\}^\{L\\times D\}denote a multivariate input sequence with lookback lengthLLandDDvariables, where𝐱t∈ℝD\\mathbf\{x\}\_\{t\}\\in\\mathbb\{R\}^\{D\}is the observation at time steptt\. The objective of long\-horizon forecasting is to predict the future sequence𝐘L\+1:L\+H=\[𝐱L\+1,𝐱L\+2,…,𝐱L\+H\]∈ℝH×D,\\mathbf\{Y\}\_\{L\+1:L\+H\}=\[\\mathbf\{x\}\_\{L\+1\},\\mathbf\{x\}\_\{L\+2\},\\ldots,\\mathbf\{x\}\_\{L\+H\}\]\\in\\mathbb\{R\}^\{H\\times D\},whereHHis the forecasting horizon\.

### 3\.2Model Architecture

The proposed StateFlow architecture consists of three main components: an instance normalization layer, a dual\-state recurrent encoder, and a horizon forecasting decoder\. To improve scalability across multivariate datasets, the model follows a channel\-independent processing strategy\. Each variable is processed as an independent univariate sequence while sharing the same model parameters across variables\. We split the multivariate time series input𝐗∈ℝL×D\\mathbf\{X\}\\in\\mathbb\{R\}^\{L\\times D\}, toDDunivariate series𝐗\(𝐢\)1:L=\[𝐱\(𝐢\)1,𝐱\(𝐢\)2,…,𝐱\(𝐢\)L\]∈ℝ1×L\\mathbf\{X^\{\(i\)\}\}\_\{1:L\}=\[\\mathbf\{x^\{\(i\)\}\}\_\{1\},\\mathbf\{x^\{\(i\)\}\}\_\{2\},\\ldots,\\mathbf\{x^\{\(i\)\}\}\_\{L\}\]\\in\\mathbb\{R\}^\{1\\times L\}, where i = 1,…,D\. Accordingly, the model will predict𝐘^\(𝐢\)L\+1:L\+H=\[𝐱\(𝐢\)L\+1,𝐱\(𝐢\)L\+2,…,𝐱\(𝐢\)L\+H\]∈ℝ1×H\.\\mathbf\{\\hat\{Y\}^\{\(i\)\}\}\_\{L\+1:L\+H\}=\[\\mathbf\{x^\{\(i\)\}\}\_\{L\+1\},\\mathbf\{x^\{\(i\)\}\}\_\{L\+2\},\\ldots,\\mathbf\{x^\{\(i\)\}\}\_\{L\+H\}\]\\in\\mathbb\{R\}^\{1\\times H\}\.After forecasting, the output is reshaped back to𝐘^∈ℝH×D\\hat\{\\mathbf\{Y\}\}\\in\\mathbb\{R\}^\{H\\times D\}\.

#### 3\.2\.1Instance Normalization Layer

To reduce window\-level distribution shifts between training and test in non\-stationary time series, we apply instance normalization to each input sequence𝐗1:L\\mathbf\{X\}\_\{1:L\}before encoder processingUlyanovet al\.\([2016](https://arxiv.org/html/2607.00197#bib.bib50)\); Kimet al\.\([2022](https://arxiv.org/html/2607.00197#bib.bib45)\)\. For each variable, the mean and standard deviation are computed over the lookback window, producing the normalized sequence𝐗¯1:L\\bar\{\\mathbf\{X\}\}\_\{1:L\}\. The VARNN encoder and horizon decoder operate in the normalized space, and the stored statistics are used to transform the predicted horizon𝐘¯^L\+1:L\+H\\hat\{\\bar\{\\mathbf\{Y\}\}\}\_\{L\+1:L\+H\}back to the original scale𝐘^L\+1:L\+H\\hat\{\\mathbf\{Y\}\}\_\{L\+1:L\+H\}\.

#### 3\.2\.2Dual\-State Recurrent Encoder

The encoder extracts temporal features from the past input window\. We use VARNNGharwi and Shu \([2025](https://arxiv.org/html/2607.00197#bib.bib49)\)as the encoder that maintains two complementary recurrent states: a conventional hidden state and a residual\-memory state\. The hidden state captures the main temporal evolution of the sequence, while the residual\-memory state tracks local deviations between one\-step base predictions and observed values\.

##### VARNN encoder\.

The normalized sequence𝐗¯1:L\\bar\{\\mathbf\{X\}\}\_\{1:L\}is processed sequentially by the VARNN encoder\. For each variableiiand time steptt, the encoder maintains a hidden state𝐡t\(i\)∈ℝK\\mathbf\{h\}^\{\(i\)\}\_\{t\}\\in\\mathbb\{R\}^\{K\}and a residual\-memory state𝐞t\(i\)∈ℝJ\\mathbf\{e\}^\{\(i\)\}\_\{t\}\\in\\mathbb\{R\}^\{J\}\. The hidden state is updated using the current observation together with the previous hidden and residual\-memory states:

𝐡t\(i\)=ϕh​\(𝐖h​\[x¯t\(i\);𝐡t−1\(i\);𝐞t−1\(i\)\]\+𝐛h\),\\mathbf\{h\}\_\{t\}^\{\(i\)\}=\\phi\_\{h\}\\left\(\\mathbf\{W\}\_\{h\}\\left\[\\bar\{x\}\_\{t\}^\{\(i\)\};\\mathbf\{h\}\_\{t\-1\}^\{\(i\)\};\\mathbf\{e\}\_\{t\-1\}^\{\(i\)\}\\right\]\+\\mathbf\{b\}\_\{h\}\\right\),\(1\)
whereϕh\\phi\_\{h\}is a nonlinear activation function\. The updated hidden state produces a one\-step base prediction:

x¯^t\+1base,\(i\)=𝐰o⊤​𝐡t\(i\)\+bo\.\\hat\{\\bar\{x\}\}\_\{t\+1\}^\{\\mathrm\{base\},\(i\)\}=\\mathbf\{w\}\_\{o\}^\{\\top\}\\mathbf\{h\}\_\{t\}^\{\(i\)\}\+b\_\{o\}\.\(2\)The corresponding local prediction residual is then computed as

rt\(i\)=x¯t\+1\(i\)−x¯^t\+1base,\(i\)\.r\_\{t\}^\{\(i\)\}=\\bar\{x\}\_\{t\+1\}^\{\(i\)\}\-\\hat\{\\bar\{x\}\}\_\{t\+1\}^\{\\mathrm\{base\},\(i\)\}\.\(3\)VARNN explicitly transforms this residual, together with the previous residual\-memory state, through a nonlinear recurrent update:

𝐞t\(i\)=ϕe​\(𝐖e​\[rt\(i\);𝐞t−1\(i\)\]\+𝐛e\),\\mathbf\{e\}\_\{t\}^\{\(i\)\}=\\phi\_\{e\}\\left\(\\mathbf\{W\}\_\{e\}\\left\[r\_\{t\}^\{\(i\)\};\\mathbf\{e\}\_\{t\-1\}^\{\(i\)\}\\right\]\+\\mathbf\{b\}\_\{e\}\\right\),\(4\)
whereϕe\\phi\_\{e\}is a nonlinear activation function\. Thus,𝐡t\(i\)\\mathbf\{h\}\_\{t\}^\{\(i\)\}captures the primary temporal dynamics of the input sequence, while𝐞t\(i\)\\mathbf\{e\}\_\{t\}^\{\(i\)\}explicitly tracks structured deviations between one\-step base predictions and observed values\. Unlike conventional recurrent models, which may encode such deviations implicitly within a single latent state, VARNN preserves them as a distinct recurrent signal that influences the subsequent hidden\-state transition\.

##### Encoder Temporal representation

After processing the input window𝐗¯1:L\\bar\{\\mathbf\{X\}\}\_\{1:L\}, the encoder produces two temporal state trajectories for each variable:

𝐇\(i\)=\[𝐡1\(i\),𝐡2\(i\),…,𝐡L−1\(i\)\],∈𝐑L−1×K\\mathbf\{H\}^\{\(i\)\}=\[\\mathbf\{h\}\_\{1\}^\{\(i\)\},\\mathbf\{h\}\_\{2\}^\{\(i\)\},\\ldots,\\mathbf\{h\}\_\{L\-1\}^\{\(i\)\}\],\\in\\mathbf\{R\}^\{L\-1\\times K\}\(5\)𝐄\(i\)=\[𝐞1\(i\),𝐞2\(i\),…,𝐞L−1\(i\)\],∈𝐑L−1×J\\mathbf\{E\}^\{\(i\)\}=\[\\mathbf\{e\}\_\{1\}^\{\(i\)\},\\mathbf\{e\}\_\{2\}^\{\(i\)\},\\ldots,\\mathbf\{e\}\_\{L\-1\}^\{\(i\)\}\],\\in\\mathbf\{R\}^\{L\-1\\times J\}\(6\)
The hidden\-state trajectory𝐇\(i\)\\mathbf\{H\}^\{\(i\)\}captures the primary temporal dynamics of the input sequence, including trend, seasonality, level changes, and recurring temporal patterns\. In contrast, the residual\-memory trajectory captures structured local prediction deviations that may reflect drift, bias, or other non\-stationary changes\. These complementary trajectories are subsequently provided to the horizon forecasting decoder\.

### 3\.3Horizon Forecasting Decoder

The horizon forecasting decoder maps the encoder state sequences to the future prediction horizon\. Given the hidden\-state sequence𝐇\(i\)∈ℝ\(L−1\)×k\\mathbf\{H\}^\{\(i\)\}\\in\\mathbb\{R\}^\{\(L\-1\)\\times k\}and the residual\-memory sequence𝐄\(i\)∈ℝ\(L−1\)×J\\mathbf\{E\}^\{\(i\)\}\\in\\mathbb\{R\}^\{\(L\-1\)\\times J\}for variableii, the decoder processes the two sequences through separate temporal summarization branches\. This design allows the model to preserve the distinct roles of the hidden and residual\-memory states before combining them for long\-horizon prediction𝐘^L\+1:L\+H\\hat\{\\mathbf\{Y\}\}\_\{L\+1:L\+H\}\.

##### Chunk Summarization Layer\.

The chunk summarization layer compresses the encoder state sequences into local temporal summaries before horizon projection\. This reduces the parameter growth of direct flattening while preserving local patterns in the hidden\-state and residual\-memory sequences\. In the main model, the hidden\-state sequence𝐇\(i\)\\mathbf\{H\}^\{\(i\)\}and the residual\-memory sequence𝐄\(i\)\\mathbf\{E\}^\{\(i\)\}are processed by separate chunk branches, allowing the decoder to preserve their distinct roles\. Given a chunk window sizewwand stridess, the starting index of themm\-th chunk is defined asam=1\+\(m−1\)​s,a\_\{m\}=1\+\(m\-1\)s,form=1,2,…,M,m=1,2,\\ldots,M,where the total number of chunks isM=⌊\(L−1\)−ws⌋\+1\.M=\\left\\lfloor\\frac\{\(L\-1\)\-w\}\{s\}\\right\\rfloor\+1\.For the hidden\-state sequence, themm\-th chunk is𝐂h,m\(i\)=𝐇am:am\+w−1\(i\),\\mathbf\{C\}\_\{h,m\}^\{\(i\)\}=\\mathbf\{H\}\_\{a\_\{m\}:a\_\{m\}\+w\-1\}^\{\(i\)\},and for the residual\-memory sequence, the corresponding chunk is𝐂e,m\(i\)=𝐄am:am\+w−1\(i\)\.\\mathbf\{C\}\_\{e,m\}^\{\(i\)\}=\\mathbf\{E\}\_\{a\_\{m\}:a\_\{m\}\+w\-1\}^\{\(i\)\}\.Each chunk is flattened and projected into a fixed\-dimensional embedding:

𝐪h,m\(i\)=ϕc,h\(𝐖c,hvec\(𝐂h,m\(i\)\)\+𝐛c,h\),∈𝐑T\\mathbf\{q\}\_\{h,m\}^\{\(i\)\}=\\phi\_\{c,h\}\\left\(\\mathbf\{W\}\_\{c,h\}\\mathrm\{vec\}\(\\mathbf\{C\}\_\{h,m\}^\{\(i\)\}\)\+\\mathbf\{b\}\_\{c,h\}\\right\),\\in\\mathbf\{R\}^\{T\}\(7\)𝐪e,m\(i\)=ϕc,e\(𝐖c,evec\(𝐂e,m\(i\)\)\+𝐛c,e\),∈𝐑I\\mathbf\{q\}\_\{e,m\}^\{\(i\)\}=\\phi\_\{c,e\}\\left\(\\mathbf\{W\}\_\{c,e\}\\mathrm\{vec\}\(\\mathbf\{C\}\_\{e,m\}^\{\(i\)\}\)\+\\mathbf\{b\}\_\{c,e\}\\right\),\\in\\mathbf\{R\}^\{I\}\(8\)
The resulting chunk embeddings are collected as𝐐h\(i\)=\[𝐪h,1\(i\),…,𝐪h,M\(i\)\],𝐐e\(i\)=\[𝐪e,1\(i\),…,𝐪e,M\(i\)\]\.\\mathbf\{Q\}\_\{h\}^\{\(i\)\}=\[\\mathbf\{q\}\_\{h,1\}^\{\(i\)\},\\ldots,\\mathbf\{q\}\_\{h,M\}^\{\(i\)\}\],\\mathbf\{Q\}\_\{e\}^\{\(i\)\}=\[\\mathbf\{q\}\_\{e,1\}^\{\(i\)\},\\ldots,\\mathbf\{q\}\_\{e,M\}^\{\(i\)\}\]\.These summarized hidden and residual\-memory representations are then passed to the forecasting head\.

##### Forecasting Head

The forecasting head maps the summarized hidden and residual\-memory representations to the target prediction horizon\. The chunk embeddings from the hidden and residual branches are first concatenated𝐐\(i\)=\[𝐐h\(i\),𝐐e\(i\)\]\\mathbf\{Q\}^\{\(i\)\}=\[\\mathbf\{Q\}\_\{h\}^\{\(i\)\},\\mathbf\{Q\}\_\{e\}^\{\(i\)\}\]and flattened , then passed through a linear projection to generate the normalized forecast for variableii:

𝐘¯^L\+1:L\+H\(i\)=𝐖o​vec​\(𝐐\(i\)\)\+𝐛o\.\\hat\{\\bar\{\\mathbf\{Y\}\}\}\_\{L\+1:L\+H\}^\{\(i\)\}=\\mathbf\{W\}\_\{o\}\\mathrm\{vec\}\(\\mathbf\{Q\}^\{\(i\)\}\)\+\\mathbf\{b\}\_\{o\}\.\(9\)Finally, The resulting output is later transformed back to the original scale using the inverse instance normalization step:𝐘^L\+1:L\+H\(i\)∈ℝ1×H\\hat\{\\mathbf\{Y\}\}\_\{L\+1:L\+H\}^\{\(i\)\}\\in\\mathbb\{R\}^\{1\\times H\}\.

#### 3\.3\.1Training Objective and Optimization

The proposed model is optimized using a two\-stage training procedure\. In the first stage, the VARNN encoder is pretrained using one\-step base prediction over the lookback lengthLL\. Given the one\-step ahead base𝐱¯^t\+1\\hat\{\\bar\{\\mathbf\{x\}\}\}\_\{t\+1\}and ground truth𝐱¯t\+1\\bar\{\\mathbf\{x\}\}\_\{t\+1\}, the encoder parameters are optimized using the mean squared error for each variable and averaged over allDDvariables:

ℒenc=1D​∑i=1D‖𝐱¯^2:Lb​a​s​e,\(i\)−𝐱¯2:L\(i\)‖22\.\\mathcal\{L\}\_\{\\mathrm\{enc\}\}=\\frac\{1\}\{D\}\\sum\_\{i=1\}^\{D\}\\left\\\|\\hat\{\\bar\{\\mathbf\{x\}\}\}^\{base,\(i\)\}\_\{2:L\}\-\\bar\{\\mathbf\{x\}\}^\{\(i\)\}\_\{2:L\}\\right\\\|\_\{2\}^\{2\}\.\(10\)
After pretraining, the encoder is frozen and a horizon decoder is trained to predict the future horizon sequence𝐘^L\+1:L\+H\\hat\{\\mathbf\{Y\}\}\_\{L\+1:L\+H\}\. The forecasting objective is

ℒforecast=1D​∑i=1D‖𝐱^L\+1:L\+H\(i\)−𝐱L\+1:L\+H\(i\)‖22\.\\mathcal\{L\}\_\{\\mathrm\{forecast\}\}=\\frac\{1\}\{D\}\\sum\_\{i=1\}^\{D\}\\left\\\|\\hat\{\\mathbf\{x\}\}^\{\(i\)\}\_\{L\+1:L\+H\}\-\\mathbf\{x\}^\{\(i\)\}\_\{L\+1:L\+H\}\\right\\\|\_\{2\}^\{2\}\.\(11\)
The two\-stage strategy separates representation learning from horizon forecasting, provides an explicit training signal for the residual\-memory state, and enables a pretrained encoder to be reused across multiple forecasting horizons for the same dataset and lookback length\.

### 3\.4Computational Complexity

LetLLbe the lookback length,DDthe number of variables,HHthe forecasting horizon, andd=dh\+ded=d\_\{h\}\+d\_\{e\}the combined encoder\-state dimension\. Given chunk windowwwand stridess, the number of chunks isM=⌊L−ws⌋\+1\.M=\\left\\lfloor\\frac\{L\-w\}\{s\}\\right\\rfloor\+1\.Sinceww,dhd\_\{h\}, andded\_\{e\}are fixed model hyperparameters, the dominant forward\-pass complexity of VARNN can be summarized as

𝒪​\(D​L​d2\+D​M​w​\(dh2\+de2\)\+D​M​d​H\)\.\\mathcal\{O\}\\left\(DLd^\{2\}\+DMw\(d\_\{h\}^\{2\}\+d\_\{e\}^\{2\}\)\+DMdH\\right\)\.\(12\)The first term corresponds to the recurrent encoder over the lookback window, while the second term corresponds to the chunk summarization layers and the last to the horizon projection head\. Thus, VARNN scales linearly with the lookback lengthLLand linearly with the forecasting horizonHH\. Compared with Transformer\-based forecasters, VARNN differs mainly in the representation encoder\. Standard temporal self\-attention has complexity𝒪​\(L2​d\)\\mathcal\{O\}\(L^\{2\}d\)per layer, PatchTST reduces this to𝒪​\(P2​d\)\\mathcal\{O\}\(P^\{2\}d\)usingPPpatches, and iTransformer applies attention over variables with complexity𝒪​\(D2​d\)\\mathcal\{O\}\(D^\{2\}d\)\. Similar to these direct forecasting models, VARNN also has a horizon\-dependent projection head\. However, VARNN replaces attention\-based representation learning with compact recurrent updates and local chunk summarization, yielding linear scaling with the lookback lengthLL\.

## 4Experiment Settings

This section evaluates the proposed StateFlow, VARNN\-based long\-horizon forecasting framework, on standard multivariate time\-series forecasting benchmarks\. The experiments are designed to answer three main questions: \(i\) whether the proposed model is competitive with established long\-horizon forecasting baselines, \(ii\) whether the hidden and residual\-memory encoder sequences provide useful forecasting representations, and \(iii\) whether the chunk\-based decoder improves the accuracy–complexity tradeoff compared with direct flattening\.

##### Datasets

We thoroughly evaluate the proposed StateFlow model on seven standard long\-horizon forecasting benchmarks: ETTh1, ETTh2, ETTm1, ETTm2, Weather, ECL, and Traffic\. The ETT datasets provide electricity transformer temperature measurements at hourly and minute\-level resolutionsZhouet al\.\([2021](https://arxiv.org/html/2607.00197#bib.bib30)\)\. Weather consists of multivariate meteorological observations, ECL contains electricity consumption records from multiple clients, and Traffic includes road occupancy measurements collected from freeway sensors\. These datasets are widely adopted in LTSF evaluation protocolsWuet al\.\([2021](https://arxiv.org/html/2607.00197#bib.bib31)\); Nieet al\.\([2023](https://arxiv.org/html/2607.00197#bib.bib33)\); Liuet al\.\([2024](https://arxiv.org/html/2607.00197#bib.bib56)\)\.

##### Baselines

We compare StateFlow with ten state\-of\-the\-art deep forecasting models covering three major LTSF model families\. The Transformer\-based baselines include iTransformer, PatchTST, Crossformer, and AutoformerLiuet al\.\([2024](https://arxiv.org/html/2607.00197#bib.bib56)\); Nieet al\.\([2023](https://arxiv.org/html/2607.00197#bib.bib33)\); Wuet al\.\([2021](https://arxiv.org/html/2607.00197#bib.bib31)\)\. The convolutional baselines include TimesNet and SCINetWuet al\.\([2023](https://arxiv.org/html/2607.00197#bib.bib34)\)\. The linear baselines include DLinearZenget al\.\([2023](https://arxiv.org/html/2607.00197#bib.bib48)\)and RLinearLiet al\.\([2023](https://arxiv.org/html/2607.00197#bib.bib53)\)\. This set of baselines provides a broad comparison against attention\-based, patch\-based, convolutional, and lightweight linear forecasting architectures\.

##### Experimental Setup

We use the standard LTSF setting with lookback lengthL=96L=96and prediction horizonsH∈\{96,192,336,720\}H\\in\\\{96,192,336,720\\\}\. VARNN is implemented in a channel\-independent manner, where each variable is modeled as a univariate sequence with shared parameters across variables\. The experiment is implemented in PyTorch, with random seed 2026\. The main model uses hidden dimensionk=32k=32, residual\-memory dimensionJ=16J=16, chunk summarizer dimensionsT=32,I=16T=32,I=16, chunk window sizew=5w=5, and strides=2s=2\. We use ReLU activation for the hidden state and chunk summarizer, and tanh activation for the residual\-memory state\. Instance normalization is applied before the encoder, followed by inverse normalization after prediction\. The model is trained in two stages\. In stage 1, the encoder is pretrained with one\-step base prediction loss\. In Stage 2, the pretrained encoder is frozen, and the horizon forecasting decoder is trained using multi\-step forecasting loss\. We report MSE and MAE on the test set\.

## 5Results

Table 1:Benchmark results \(MSE/MAE\) across datasets and horizons\.Note: The baseline results are taken from the standardized benchmark table reported by iTransformerLiuet al\.\([2024](https://arxiv.org/html/2607.00197#bib.bib56)\), and implemented under the same standard training protocols: input lengthL=96L=96, prediction horizonsH∈\{96,192,336,720\}H\\in\\\{96,192,336,720\\\}, and MSE/MAE evaluation\.

Overall performanceTable[1](https://arxiv.org/html/2607.00197#S5.T1)shows the multivariate long term forecasting results\. A lower MSE or MAE indicates better forecasting performance\. Overall, StateFlow achieves competitive performance compared with strong linear, convolutional, and Transformer\-based models and obtains the largest number of first\-place results, with 15 MSE wins and 14 MAE wins across 28 dataset\-horizon settings\. The results suggested that the proposed residual\-aware recurrent encoder can provide effective long\-horizon forecasting representations, despite using a structurally different design from patch\-based and attention\-based architectures\. In particular, the hidden and residual\-memory state sequences allow the decoder to exploit both primary temporal dynamics and structured prediction deviations when generating the future horizon\. On high\-dimensional datasets such as ECL and Traffic, iTransformer remains stronger in several settings\. These high\-dimensional datasets may benefit more from explicit cross\-variate dependency modeling, whereas StateFlow adopts a channel\-independent design and focuses primarily on temporal modeling within each variable\. Overall, the results indicate that residual\-memory recurrence is an effective and efficient inductive bias for long\-horizon forecasting, while incorporating stronger cross\-variate interactions may further improve performance on high\-dimensional multivariate datasets\.

### 5\.1Ablations

#### 5\.1\.1Effect of VARNN Encoder Representation

Table[2](https://arxiv.org/html/2607.00197#S5.T2)compares four encoder representations under the same chunk\-decoder setting: the dual\-state representation\(h,e\)\(h,e\), the hidden\-state sequencehh, the residual\-memory sequenceee, and the base\-prediction with corresponding residual scalar representation\(𝐱¯^b​a​s​e,r\)\(\\hat\{\\bar\{\\mathbf\{x\}\}\}^\{base\},r\)\. Overall, the dual\-state representation\(h,e\)\(h,e\)provides the most stable and effective forecasting representation, achieving the best results on most horizons, especially on ETTh1, ETTm1, and ETTm2\. This shows that using both the residual\-conditioned hidden state and the residual\-memory state gives the decoder richer information for long\-horizon prediction\. The individual representations clarify the role of each state\. The hidden sequencehhremains competitive, indicating that it captures strong recurrent dynamics conditioned on past residual\-memory information\. The residual\-memory sequenceeeperforms particularly well on ETTh2 and is often second\-best on ETTm2, suggesting that structured prediction deviations can provide useful forecasting signals\. In contrast,\(𝐱¯^b​a​s​e,r\)\(\\hat\{\\bar\{\\mathbf\{x\}\}\}^\{base\},r\)is generally weaker than the learned state representations, showing that the nonlinear recurrent encoding of prediction errors is more effective than directly passing raw base predictions and residuals\. These results support the main StateFlow design, whereas hidden and residual\-memory states provide complementary information, and their dual\-state representation improves robustness across datasets and horizons\.

##### Dual\-State Fusion Strategy

To examine how the dual\-state representation should be fused, we compare separate and joint chunk summarization in Table[3](https://arxiv.org/html/2607.00197#S5.T3)\. The separate chunking\(h,e\)\(h,e\)summarizes the hidden trajectoryhhand residual\-memory trajectoryeeusing independent chunk summarizers before concatenation, while the joint concatenates\[h,e\]\[h,e\]before chunk summarization\. Separate chunking achieves lower average MSE and MAE in most datasets and horizons, and uses fewer parameters\. This suggests that preserving the distinction between hidden dynamics and residual\-memory dynamics before temporal compression is beneficial\. Full results are reported in Appendix[A\.1](https://arxiv.org/html/2607.00197#A1.SS1)\.

Table 2:Ablation study of encoder representations under the fixed chunk\-decoder setting\. Lower values are better\. The best result is shown in bold and the second\-best result is underlined\.Table 3:Effect of dual\-state fusion strategy under the chunk decoder\. Lower values are better\.
##### Chunk vs Direct Decoder

Figure[2](https://arxiv.org/html/2607.00197#S5.F2)compares the chunk decoder with direct flattening under the same hidden–residual representation\. The chunk decoder achieves comparable or better MSE on most datasets, especially on ETTm1 and ETTh2, while direct decoding is only slightly better in some long\-horizon ETTh1 cases\. More importantly, the chunk decoder substantially reduces the model size by nearly half compared with direct decoding\. These results show that the chunk summarization layer achieves similar or better accuracy with dramatically fewer parameters, so it is used as the default decoder in StateFlow\.

![Refer to caption](https://arxiv.org/html/2607.00197v1/x1.png)Figure 2:Effect of chunk versus direct decoding on theherepresentation\. The left panel reports test MSE across prediction horizons, while the right panel compares total parameters\.

### 5\.2Effect of Recurrent Encoder and Optimization Design

##### Recurrent Encoder Choice

We evaluate whether the proposed dual\-state VARNN encoder provides advantages over standard recurrent encoders by comparing VARNN with RNN, GRU, and LSTM under the same recurrent encoder–decoder framework\. All models use the same normalization, lookback length, chunk decoder, forecasting horizons\. and two\-stage optimization protocol\. To control decoder capacity, we use a matched\-representation setting: RNN, GRU, and LSTM use a 48\-dimensional hidden state, while VARNN uses a 32\-dimensional hidden state and a 16\-dimensional residual\-memory state, giving all models the same 48\-dimensional decoder input\.

The results show that VARNN provides the best accuracy–efficiency tradeoff\. Across the four ETT datasets and four horizons, VARNN achieves the lowest average error among recurrent encoders, with0\.3640\.364MSE, compared with LSTM0\.3740\.374, RNN0\.3760\.376, and GRU0\.3770\.377\. In terms of encoder complexity, VARNN uses the fewest encoder parameters:1,9221\{,\}922compared with2,4972\{,\}497for RNN,7,3937\{,\}393for GRU, and9,8419\{,\}841for LSTM\. Thus, VARNN reduces encoder parameters by23\.03%23\.03\\%compared to RNN and80\.47%80\.47\\%compared with LSTM\. These findings suggest that the residual\-memory state improves recurrent forecasting by providing a more parameter\-efficient temporal representation than standard gated recurrence\.

##### Optimization Strategy

We further evaluate whether the proposed two\-stage optimization contributes to the observed gains\. In one\-stage training, the encoder and horizon decoder are optimized jointly using the multi\-step forecasting loss\. In two\-stage training, the encoder is first pretrained using one\-step prediction and then frozen while the horizon decoder is trained\. VARNN achieves the best average MSE under both settings, obtaining0\.3710\.371with one\-stage training and0\.3640\.364with two\-stage training strategy\. The two\-stage strategy also improves RNN \(0\.378→0\.3760\.378\\rightarrow 0\.376\) and LSTM \(0\.377→0\.3740\.377\\rightarrow 0\.374\), while GRU shows a small degradation \(0\.376→0\.3770\.376\\rightarrow 0\.377\)\. The largest improvement is observed for VARNN, suggesting that one\-step pretraining is particularly effective for learning residual\-memory dynamics before long\-horizon decoding\. Moreover, since Stage 1 depends only on the dataset and lookback length, the pretrained encoder can be reused across multiple forecasting horizons, avoiding redundant encoder training and enabling horizon\-specific decoders to be learned from a shared temporal representation\. Full results are reported in Appendix[A\.2](https://arxiv.org/html/2607.00197#A1.SS2), Table[7](https://arxiv.org/html/2607.00197#A1.T7)\.

Table 4:Two\-stage recurrent encoder results under the matched\-representation setting\. Lower MSE is better\.Table 5:One\-stage versus two\-stage optimization performance under the matched\-representation setting\. Lower Avg MSE is better\.

![[Uncaptioned image]](https://arxiv.org/html/2607.00197v1/x2.png)Figure 3:Encoder\-only parameter comparison under the matched\-representation setting\. VARNN uses the smallest encoder\.

## 6Ethics Statement

This work develops supervised learning methods for long\-horizon time\-series forecasting\. Ethical risks are low\. However, forecasting models can affect downstream decisions in domains such as energy, transportation, finance, and healthcare\. When applied to sensitive domains, ensure appropriate consent procedures, privacy safeguards, fairness audits, and domain\-specific human oversight\.

## 7Conclusion and Future Work

This paper proposes StateFlow, a residual\-aware recurrent framework for long\-horizon time\-series forecasting\. The model extends VARNN by using hidden\-state and residual\-memory sequences as forecasting representations, enabling the decoder to leverage both primary temporal dynamics and structured residual patterns to predict the future horizon\. Experiments on standard LTSF benchmarks show that StateFlow is competitive with strong linear, convolutional, and Transformer\-based baselines\. StateFlow provides an alternative to Transformer\-based forecasting models by avoiding quadratic temporal self\-attention\. Its recurrent encoder scales linearly with the lookback length, while the chunk\-based decoder reduces the parameter growth associated with directly flattening the full encoder state sequence\. The results suggest that residual\-aware recurrent modeling provides a promising direction for long\-horizon forecasting\. Future work will investigate cross\-variate dependency modeling and more time\-series analysis tasks\.

## References

- R\. Baidya and S\. Lee \(2024\)Addressing the non\-stationarity and complexity of time series data for long\-term forecasts\.Applied Sciences14\(11\),pp\. 4436\.External Links:[Document](https://dx.doi.org/10.3390/app14114436)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1)\.
- K\. Benidis, S\. S\. Rangapuram, V\. Flunkert, Y\. Wang,et al\.\(2023\)Deep learning for time series forecasting: a tutorial and literature survey\.ACM Computing Surveys\.External Links:[Document](https://dx.doi.org/10.1145/3533382),[Link](https://arxiv.org/abs/2004.10240)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1)\.
- G\. E\. P\. Box, G\. M\. Jenkins, G\. C\. Reinsel, and G\. M\. Ljung \(2015\)Time series analysis: forecasting and control\.5 edition,Wiley\.Cited by:[§2\.1](https://arxiv.org/html/2607.00197#S2.SS1.p1.1)\.
- P\. J\. Brockwell and R\. A\. Davis \(2002\)Introduction to time series and forecasting\.2 edition,Springer,New York, NY, USA\.External Links:[Document](https://dx.doi.org/10.1007/b97391)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1)\.
- K\. Cho, B\. van Merriënboer, D\. Bahdanau, and Y\. Bengio \(2014\)On the properties of neural machine translation: encoder–decoder approaches\.InProceedings of SSST\-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation,Doha, Qatar,pp\. 103–111\.External Links:[Link](https://aclanthology.org/W14-4012)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.00197#S2.SS3.p1.1)\.
- J\. G\. De Gooijer and R\. J\. Hyndman \(2006\)25 years of time series forecasting\.International Journal of Forecasting22\(3\),pp\. 443–473\.External Links:[Document](https://dx.doi.org/10.1016/j.ijforecast.2006.01.001)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1)\.
- J\. L\. Elman \(1990\)Finding structure in time\.Cognitive Science14\(2\),pp\. 179–211\.External Links:[Document](https://dx.doi.org/10.1207/s15516709cog1402%5F1)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.00197#S2.SS3.p1.1)\.
- H\. Gharwi and K\. Shu \(2025\)Variability aware recursive neural network \(VARNN\): a residual\-memory model for capturing temporal deviation in sequence regression modeling\.arXiv preprint arXiv:2510\.08944\.External Links:[Link](https://arxiv.org/abs/2510.08944)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p4.1),[§2\.3](https://arxiv.org/html/2607.00197#S2.SS3.p2.1),[§3\.2\.2](https://arxiv.org/html/2607.00197#S3.SS2.SSS2.p1.1)\.
- S\. Hochreiter and J\. Schmidhuber \(1997\)Long short\-term memory\.Neural Computation9\(8\),pp\. 1735–1780\.External Links:[Document](https://dx.doi.org/10.1162/neco.1997.9.8.1735)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.00197#S2.SS3.p1.1)\.
- R\. J\. Hyndman and G\. Athanasopoulos \(2021\)Forecasting: principles and practice\.3 edition,OTexts,Melbourne, Australia\.Note:Accessed 2025\-09\-11External Links:[Link](https://otexts.com/fpp3/)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.00197#S2.SS1.p1.1)\.
- T\. Kim, J\. Kim, I\. Ohn, and S\. Kim \(2022\)Reversible instance normalization for accurate time\-series forecasting against distribution shift\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2103\.16900Cited by:[§3\.2\.1](https://arxiv.org/html/2607.00197#S3.SS2.SSS1.p1.4)\.
- Z\. Li, S\. Qi, Y\. Li, and Z\. Xu \(2023\)Revisiting long\-term time series forecasting: an investigation on linear mapping\.arXiv preprint arXiv:2305\.10721\.External Links:[Link](https://arxiv.org/abs/2305.10721)Cited by:[§2\.1](https://arxiv.org/html/2607.00197#S2.SS1.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px2.p1.1)\.
- B\. Lim, S\. O\. Arik, N\. Loeff, and T\. Pfister \(2019\)Temporal fusion transformers for interpretable multi\-horizon time series forecasting\.arXiv preprint arXiv:1912\.09363\.External Links:[Link](https://arxiv.org/abs/1912.09363)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1)\.
- B\. Lim and S\. Zohren \(2021\)Time\-series forecasting with deep learning: a survey\.Philosophical Transactions of the Royal Society A379\(2194\),pp\. 20200209\.External Links:[Document](https://dx.doi.org/10.1098/rsta.2020.0209)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1)\.
- Y\. Liu, T\. Hu, H\. Zhang, H\. Wu, S\. Wang, L\. Ma, and M\. Long \(2024\)iTransformer: inverted transformers are effective for time series forecasting\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=JePfAI8fah)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.00197#S2.SS2.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2607.00197#S5.T1.2.2)\.
- Y\. Nie, N\. H\. Nguyen, P\. Sinthong, and J\. Kalagnanam \(2023\)A time series is worth 64 words: long\-term forecasting with transformers\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Jbdc0vTOcol)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.00197#S2.SS2.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px2.p1.1)\.
- B\. N\. Oreshkin, D\. Carpov, N\. Chapados, and Y\. Bengio \(2020\)N\-beats: neural basis expansion analysis for time series forecasting\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=r1ecqn4YwB)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p1.1)\.
- D\. Ulyanov, A\. Vedaldi, and V\. Lempitsky \(2016\)Instance normalization: the missing ingredient for fast stylization\.arXiv preprint arXiv:1607\.08022\.External Links:[Link](https://arxiv.org/abs/1607.08022)Cited by:[§3\.2\.1](https://arxiv.org/html/2607.00197#S3.SS2.SSS1.p1.4)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30\.External Links:[Link](https://papers.nips.cc/paper/7181-attention-is-all-you-need)Cited by:[§2\.2](https://arxiv.org/html/2607.00197#S2.SS2.p1.1)\.
- H\. Wu, T\. Hu, Y\. Liu, H\. Zhou, J\. Wang, and M\. Long \(2023\)TimesNet: temporal 2d\-variation modeling for general time series analysis\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ju_Uqw384Oq)Cited by:[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px2.p1.1)\.
- H\. Wu, J\. Xu, J\. Wang, and M\. Long \(2021\)Autoformer: decomposition transformers with auto\-correlation for long\-term series forecasting\.InAdvances in Neural Information Processing Systems,Vol\.34\.Cited by:[§2\.2](https://arxiv.org/html/2607.00197#S2.SS2.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px2.p1.1)\.
- A\. Zeng, M\. Chen, L\. Zhang, and Q\. Liu \(2023\)Are transformers effective for time series forecasting?\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.37,pp\. 11121–11128\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v37i9.26317),[Link](https://doi.org/10.1609/aaai.v37i9.26317)Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.00197#S2.SS1.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px2.p1.1)\.
- H\. Zhou, S\. Zhang, J\. Peng, S\. Zhang, J\. Li, H\. Xiong, and W\. Zhang \(2021\)Informer: beyond efficient transformer for long sequence time\-series forecasting\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 11106–11115\.Cited by:[§1](https://arxiv.org/html/2607.00197#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.00197#S2.SS2.p1.1),[§4](https://arxiv.org/html/2607.00197#S4.SS0.SSS0.Px1.p1.1)\.
- T\. Zhou, Z\. Ma, Q\. Wen, X\. Wang, L\. Sun, and R\. Jin \(2022\)FEDformer: frequency enhanced decomposed transformer for long\-term series forecasting\.InInternational Conference on Machine Learning,Cited by:[§2\.2](https://arxiv.org/html/2607.00197#S2.SS2.p1.1)\.

## Appendix AMORE RESULTS ON ABLATION STUDY

### A\.1Full Dual\-State Fusion Strategy Results

Table[6](https://arxiv.org/html/2607.00197#A1.T6)reports the full dataset\-horizon comparison between separate and joint dual\-state fusion under the chunk decoder\. The separate variant applies independent chunk summarizers tohhandeebefore concatenation, while the joint variant concatenates\[h,e\]\[h,e\]before chunk summarization\. Overall, separate chunking provides lower error in most settings while also using fewer parameters\.

Table 6:Effect of dual\-state fusion strategy under the fixed chunk\-decoder setting\. Lower MSE/MAE and fewer parameters are better\. Extra Params reports the additional parameters used by Joint HE relative to Sep\. HE\.DatasetHSep\. HE\(h,e\)\(h,e\)Joint HE\[h,e\]\[h,e\]Extra ParamsMSEMAEParamsMSEMAEParamsJoint–Sep\.ETTh1960\.3640\.394220\.4K0\.3680\.395225\.6K\+5\.1K1920\.4150\.421432\.5K0\.4180\.425437\.6K\+5\.1K3360\.4520\.440750\.6K0\.4650\.445755\.7K\+5\.1K7200\.4740\.4631598\.9K0\.4690\.4611604\.0K\+5\.1KETTh2960\.2890\.344220\.4K0\.2910\.345225\.6K\+5\.1K1920\.3710\.395432\.5K0\.3820\.401437\.6K\+5\.1K3360\.4190\.430750\.6K0\.4150\.428755\.7K\+5\.1K7200\.4320\.4481598\.9K0\.4330\.4471604\.0K\+5\.1KETTm1960\.3160\.359220\.4K0\.3140\.357225\.6K\+5\.1K1920\.3510\.378432\.5K0\.3510\.379437\.6K\+5\.1K3360\.3820\.400750\.6K0\.3820\.402755\.7K\+5\.1K7200\.4450\.4381598\.9K0\.4450\.4391604\.0K\+5\.1KETTm2960\.1750\.262220\.4K0\.1760\.261225\.6K\+5\.1K1920\.2410\.304432\.5K0\.2410\.305437\.6K\+5\.1K3360\.3000\.342750\.6K0\.3020\.343755\.7K\+5\.1K7200\.3960\.4011598\.9K0\.4010\.4031604\.0K\+5\.1KOverallAvg0\.3640\.389750\.6K0\.3660\.390755\.7K\+5\.1KOverallWins12/1611/16–4/165/16––
### A\.2Recurrent Encoder Choice and Optimization Strategies Results

This appendix provides the full recurrent encoder and optimization ablation under the matched\-representation setting\. RNN, GRU, and LSTM use a 48\-dimensional hidden state, while VARNN uses a 32\-dimensional hidden state and a 16\-dimensional residual\-memory state, giving all models the same 48\-dimensional decoder representation\. All models use the same normalization, chunk decoder, lookback length, forecasting horizons, and evaluation protocol\.

Table[7](https://arxiv.org/html/2607.00197#A1.T7)reports MSE for each dataset and forecasting horizon under both one\-stage and two\-stage optimization\. In one\-stage training, the encoder and decoder are optimized jointly using the horizon forecasting loss\. In two\-stage training, the encoder is first pretrained using one\-step prediction and then frozen while the horizon decoder is trained\. The results show that VARNN achieves the best overall average under both optimization settings\. The two\-stage setting provides the strongest VARNN performance, supporting the role of one\-step encoder pretraining in learning useful residual\-memory dynamics before long\-horizon decoding\.

Table 7:Full recurrent encoder and optimization ablation under the matched\-representation setting\. Results are reported as MSE\. Lower values are better\. The best result is shown in bold and the second\-best result is underlined\.One\-stageTwo\-stageDatasetHVARNNLSTMRNNGRUVARNNLSTMRNNGRUETTh1960\.3780\.3920\.3740\.3850\.3640\.3800\.3820\.3841920\.4290\.4440\.4230\.4410\.4150\.4330\.4210\.4373360\.4840\.4780\.4710\.5000\.4520\.4650\.4700\.4947200\.4690\.5030\.5260\.4890\.4740\.4970\.5080\.513Avg0\.4400\.4540\.4480\.4540\.4260\.4440\.4450\.457ETTh2960\.2910\.2940\.3050\.3000\.2890\.3000\.2970\.3031920\.3740\.3710\.3720\.3690\.3710\.3740\.3750\.3733360\.4290\.4270\.4160\.4200\.4190\.4170\.4460\.4827200\.4370\.4540\.4640\.4520\.4320\.4450\.4440\.424Avg0\.3830\.3860\.3890\.3850\.3780\.3840\.3910\.396ETTm1960\.3170\.3310\.3440\.3270\.3160\.3270\.3350\.3231920\.3610\.3650\.3600\.3680\.3510\.3670\.3640\.3593360\.3860\.3950\.3900\.3880\.3820\.3890\.3930\.3847200\.4410\.4670\.4520\.4490\.4450\.4510\.4500\.446Avg0\.3760\.3890\.3870\.3830\.3730\.3840\.3850\.378ETTm2960\.1810\.1760\.1820\.1790\.1750\.1790\.1770\.1751920\.2430\.2390\.2490\.2420\.2410\.2490\.2430\.2403360\.3050\.3010\.3040\.3010\.3000\.3020\.3050\.3007200\.4090\.3950\.4030\.3990\.3960\.3970\.3990\.395Avg0\.2840\.2780\.2840\.2800\.2780\.2820\.2810\.277OverallAvg0\.3710\.3770\.3770\.3760\.3640\.3730\.3750\.377OverallWins645112105