How Good Can Linear Models Be for Time-Series Forecasting?
Summary
This paper demonstrates that careful preprocessing—especially context length selection, normalization, and regularization—can make simple linear models like Ridge regression competitive with or superior to large Transformer, MLP, and CNN models on time-series forecasting benchmarks.
View Cached Full Text
Cached at: 06/30/26, 03:33 AM
Paper page - How Good Can Linear Models Be for Time-Series Forecasting?
Source: https://huggingface.co/papers/2606.27282
Abstract
Research demonstrates that preprocessing optimizations, particularly in context length, normalization, and regularization, can significantly improve time-series forecasting accuracy more effectively than scaling model architectures.
Time-series forecastingresearch has been moving steadily toward larger architectures, from specializedtransformersto general-purposefoundation models, on the assumption that capacity is what unlocks accuracy. We take the opposite position: most of the gap can be closed at far lower cost by tuning preprocessing rather than scaling models. We useRidge regressionas the testbed, since it has a closed-form solution and interpretable weights, which let the optimalhyperparametersbe read off the search directly. We search overcontext length,local normalization,regularization, andaugmentationon eight standard benchmarks and find three patterns. (1) Optimal lookback is strongly series-specific and often non-monotonic inforecast horizon, with fitted power-law exponents ranging from +0.46 on ETTm2 to -0.19 on Exchange and Traffic, challenging the convention that longer horizons need longer history. (2) Normalizing over a learned trailing fraction of the context, rather than its entirety, is almost universally preferred. (3) Series within the same dataset often disagree onhyperparameters; the optimal degree ofcross-series sharingvaries from fully shared to fully per-series. The resulting models beat prior linear forecasters on most dataset-horizon entries and exceed Transformer, MLP, and CNN baselines on six of eight benchmarks. The optimizedhyperparametersalso serve as a diagnostic on the data itself, revealing structures that larger models absorb silently into their learned parameters.
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2606\.27282
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.27282 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.27282 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.27282 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
The unreasonable difficulty of time series forecasting
The article explores why time series forecasting is uniquely challenging compared to other machine learning tasks, presenting benchmark results showing that simple statistical models and zero-shot foundation models often outperform sophisticated deep learning models on many series.
Retrieval Mechanisms Surpass Long-Context Scaling in Time Series Forecasting
This academic paper challenges the effectiveness of long-context scaling in time series forecasting, demonstrating that retrieval-based methods outperform standard architectures like PatchTST and foundation models such as Chronos and Moirai.
The Simulacrum: Decision-Theoretic Pretraining for Near-Optimal Time-Series Forecasting and Inference
This paper introduces a unified decision-theoretic pretraining framework for neural network-based time series estimators, trained on stratified simulations to approximate near-optimal decision rules. Experiments show that the resulting estimators outperform traditional methods like maximum likelihood estimation on both synthetic and real-world benchmarks.
How does a 102M-parameter transformer forecast multivariate time series?
This article provides a visual walkthrough of t0-alpha, a 101.6M-parameter foundation model for multivariate time-series forecasting that separates time attention from cross-variable group attention, achieving competitive CRPS scores on GIFT-Eval compared to larger models like TimesFM 2.5 and Chronos-2.
Assessing the Operational Viability of Foundation Models for Time Series Forecasting
This paper presents an applied evaluation of foundation models for time series forecasting compared to supervised approaches across four operational domains, and proposes a Complexity Router to selectively assign series to the optimal model class for balancing accuracy and inference cost.