SAGE: Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting

# Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting
Source: [https://arxiv.org/html/2608.26829](https://arxiv.org/html/2608.26829)
###### Abstract

Time series forecasting models operate on raw numerical sequences, lacking the semantic knowledge that domain experts implicitly leverage, such as the physical meaning of each variable, its statistical behavior, and its temporal dynamics\. Recent efforts to bridge this gap fall into two camps\. Some rely on large language models at inference time, which is computationally expensive\. Others apply uniform textual prompts at the dataset level, ignoring the heterogeneous semantics across individual variates\. We proposeSAGE\(Seeing andAugmenting withGroundedEncoding\), an end\-to\-end CLIP\-based framework that jointly models temporal, cross\-variable, textual, and visual information\. The CLIP text encoder processes frequency\-enhanced patches and variable tokens, while gated residual paths inject variable\-specific descriptions and statistical descriptors\. In parallel, the frozen CLIP vision encoder aligns rendered series with temporal representations through a training\-only contrastive objective\. This dual use of CLIP adds complementary semantic and visual supervision without placing an LLM in the forecasting loop\. Across eight long\-term benchmarks and M4, SAGE achieves state\-of\-the\-art accuracy\. Ablations confirm complementary gains from multimodal alignment and variable\-level knowledge\.

1Shanghai Jiao Tong University

Shanghai, China

sharp\-ro@sjtu\.edu\.cn, lexinyi@sjtu\.edu\.cn

## 1Introduction

Time series forecasting is a fundamental task in data mining and machine learning, with wide\-ranging applications in energy management, transportation planning, weather prediction, and financial analysis\([Zhou et al\. 2020](https://arxiv.org/html/2608.26829#bib.bib26);[Wu et al\. 2021](https://arxiv.org/html/2608.26829#bib.bib10)\)\. The advent of Transformer\-based architectures has brought significant progress\([Zhou et al\. 2022](https://arxiv.org/html/2608.26829#bib.bib27);[Liu et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib12)\), while the provocative finding that simple linear models can match or surpass many Transformer variants\([Zeng et al\. 2023](https://arxiv.org/html/2608.26829#bib.bib28)\)has spurred the community to rethink what inductive biases truly matter for temporal modeling\.

Figure 1:Representative 96\-step forecasts on Weather, Electricity, and ETTm1\. SAGE better captures regime changes by combining numerical history with multimodal knowledge\. Each inset reports window\-level MSE for SAGE and the strongest displayed baseline\.More recently, two parallel trends have reshaped this landscape\. The first islarge\-scale pre\-training: foundation models such as TimesFM\([Das et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib15)\), Chronos\([Ansari et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib16)\), and Time\-MoE\([Shi et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib18)\)demonstrate that pre\-training on billions of time points yields strong zero\-shot and few\-shot forecasting capabilities\. The second iscross\-modal knowledge transfer: methods such as GPT4TS\([Zhou et al\. 2023](https://arxiv.org/html/2608.26829#bib.bib1)\), Time\-LLM\([Jin et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib3)\), and TEST\([Sun et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib4)\)repurpose pre\-trained language models for time series tasks, exploiting their rich semantic and sequential knowledge\. However, the LLM component in many of these methods can be replaced by simple attention layers without degrading performance\([Tan et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib30)\), raising questions about whether language models truly contribute temporal understanding or merely serve as general\-purpose feature extractors\.

Meanwhile, an emerging body of work exploresvision\-language models \(VLMs\)for time series\. A recent survey\([Ni et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib31)\)highlights the natural compatibility between 2D visual representations and multi\-scale temporal patterns, an insight previously demonstrated by TimesNet\([Wu et al\. 2022](https://arxiv.org/html/2608.26829#bib.bib13)\)\. Time\-VLM\([Zhong et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib6)\)fuses retrieval, vision, and text modalities for forecasting; Aurora\([Wu et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib7)\)builds the first multimodal time series foundation model; and OccamVTS\([Lyu et al\. 2026](https://arxiv.org/html/2608.26829#bib.bib19)\)shows that distilling vision models to 1% of their parameters can match full\-model forecasting performance\. The CiK benchmark\([Williams et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib9)\)further establishes that textual context can be essential for accurate prediction in many real\-world scenarios\.

Despite these advances, a fundamental gap remains: existing multimodal methods typically use language models as either frozen feature extractors or auxiliary prompt generators, without fully exploiting the*dual structure*of vision\-language models\. In CLIP\([Radford et al\. 2021](https://arxiv.org/html/2608.26829#bib.bib29)\), the text and vision encoders are jointly trained to produce aligned representations in a shared embedding space\. This dual structure naturally suits time series\. The text encoder can serve as a sequential backbone, while the vision encoder provides complementary supervision through rendered time series images\. Moreover, existing knowledge\-injection schemes\([Jin et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib3);[Pan et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib5)\)may require expensive LLM inference or reduce numerical sequences to limited textual representations\. Figure[1](https://arxiv.org/html/2608.26829#S1.F1)previews the payoff of closing this gap: across real test windows from three different datasets, SAGE tracks the periodic and directional dynamics that strong Transformer forecasters collapse toward the recent mean\.

In this paper, we propose SAGE, a multimodal semantic alignment framework for time series forecasting that addresses these limitations through three key contributions:

1. 1\.End\-to\-End CLIP Forecasting\.We make CLIP part of the forecasting architecture rather than an external feature service\. Its text encoder is adapted as the temporal backbone, and its frozen vision encoder provides bidirectional contrastive supervision during training\. This design achieves higher forecasting accuracy with a compact trainable backbone instead of relying on a billion\-parameter model in the forecasting loop\.
2. 2\.Template\-based Knowledge Injection with Variable\-wise Gating\.A lightweight offline pipeline constructs semantic, behavioral, and relational descriptions from metadata and training\-set statistics\. An LLM may assist this preparation step, but it is not called inside training or forecasting\. Gated cross\-attention controls knowledge injection independently for each variable, while a separately gated statistical bypass preserves hard numerical cues\.
3. 3\.Grounding Across Time, Variables, and Modalities\.A frequency\-enhanced patch embedding combines time\-domain and spectral evidence\. Cross\-variate attention then incorporates dependencies among variables, while vision\-language contrastive alignment regularizes the temporal representations\. Together, these components integrate complementary temporal, relational, textual, and visual signals in one forecasting model\.

## 2Related Work

### 2\.1Multimodal and Knowledge\-Enhanced Time Series Forecasting

Early work demonstrated that pre\-trained language models can serve as effective time series backbones: GPT4TS\([Zhou et al\. 2023](https://arxiv.org/html/2608.26829#bib.bib1)\)fine\-tunes only the layer normalization and positional embeddings of a frozen GPT\-2, while LLMTime\([Gruver et al\. 2023](https://arxiv.org/html/2608.26829#bib.bib2)\)encodes numerical series as text strings for zero\-shot forecasting\. Subsequent methods introduce explicit cross\-modal alignment: Time\-LLM\([Jin et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib3)\)reprograms input patches into text prototypes guided by declarative prompts, and TEST\([Sun et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib4)\)aligns time series embeddings with interpretable text prototypes via contrastive losses\. A recurring limitation is that the language component often acts as a generic feature extractor rather than a genuine source of temporal knowledge\([Tan et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib30)\)\.

More recently, the community has moved toward true multimodal fusion\. Time\-VLM\([Zhong et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib6)\)couples retrieval\-, vision\-, and text\-augmented learners with frozen VLMs; Aurora\([Wu et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib7)\)injects text and image knowledge via modality\-guided attention and flow matching for zero\-shot generative forecasting; and VLM4TS\([He et al\. 2026](https://arxiv.org/html/2608.26829#bib.bib8)\)applies VLMs to zero\-shot anomaly detection through visual screening and multimodal verification\. Despite these advances, most existing methods treat the language model as either a frozen feature extractor or an auxiliary prompt generator, without fully exploiting both branches of a vision\-language model in a unified training framework\.

### 2\.2Pre\-trained Models for Time Series

In the supervised regime, Autoformer\([Wu et al\. 2021](https://arxiv.org/html/2608.26829#bib.bib10)\)pioneers deep decomposition with auto\-correlation mechanisms; PatchTST\([Nie et al\. 2022](https://arxiv.org/html/2608.26829#bib.bib11)\)establishes a strong patch\-based channel\-independent baseline; iTransformer\([Liu et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib12)\)treats each variate as a token to capture multivariate correlations; and TimesNet\([Wu et al\. 2022](https://arxiv.org/html/2608.26829#bib.bib13)\)transforms 1D series into 2D tensors, enabling vision backbones for temporal modeling\. More recent architectures continue to refine this line: FredFormer\([Piao et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib35)\)debiases frequency\-domain representations, DUET\([Qiu et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib37)\)jointly clusters temporal and channel dependencies, Amplifier\([Fei et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib36)\)amplifies energy\-scarce components, and SRSNet\([Wu et al\. 2026](https://arxiv.org/html/2608.26829#bib.bib38)\)learns a selective representation space; these are the strong recent baselines against which we benchmark SAGE\. In the foundation model regime, TimesFM\([Das et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib15)\)trains a decoder\-only Transformer on 100B real\-world time points; Chronos\([Ansari et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib16)\)tokenizes values into a discrete vocabulary for T5\-family models; MOMENT\([Goswami et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib14)\)pre\-trains on the Time\-series Pile via masked reconstruction; Moirai\([Woo et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib17)\)introduces any\-variate attention trained on 27B observations; and Time\-MoE\([Shi et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib18)\)scales to 2\.4B parameters via sparse mixture\-of\-experts\. These efforts demonstrate the effectiveness of scale, yet they require curating massive time series corpora\. An alternative direction, exemplified by OccamVTS\([Lyu et al\. 2026](https://arxiv.org/html/2608.26829#bib.bib19)\), suggests that pre\-trained vision\-language representations already contain transferable temporal knowledge that can be leveraged without billion\-scale pre\-training\.

### 2\.3Contrastive Learning for Time Series

Contrastive learning has proven effective for learning transferable time series representations: TS\-TCC\([Eldele et al\. 2021](https://arxiv.org/html/2608.26829#bib.bib20)\)combines temporal and contextual contrasting over augmented views; TS2Vec\([Yue et al\. 2022](https://arxiv.org/html/2608.26829#bib.bib21)\)performs hierarchical contrastive learning at instance and temporal levels; CoST\([Woo et al\. 2022](https://arxiv.org/html/2608.26829#bib.bib22)\)disentangles seasonal and trend components via time\- and frequency\-domain losses; TF\-C\([Zhang et al\. 2022](https://arxiv.org/html/2608.26829#bib.bib23)\)enforces time\-frequency consistency in a joint embedding space; Soft\-CL\([Lee et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib24)\)mitigates false negatives through soft assignments; and FACL\([Wang and Zhang 2026](https://arxiv.org/html/2608.26829#bib.bib25)\)designs frequency\-aware augmentations that respect spectral structure\. All of these operate within the time series modality, constructing positive pairs from augmented views of the same data\. A natural extension is cross\-modal contrastive learning, where positive pairs couple temporal representations with their visual renderings, providing a complementary supervisory signal beyond intra\-modal augmentation\.

## 3Methodology

We propose a multimodal framework for multivariate time series forecasting\. Given an input multivariate time series𝐗∈ℝT×N\\mathbf\{X\}\\in\\mathbb\{R\}^\{T\\times N\}withTTtime steps andNNvariates, the model outputs a forecast𝐘^∈ℝH×N\\hat\{\\mathbf\{Y\}\}\\in\\mathbb\{R\}^\{H\\times N\}over a horizon ofHHsteps\. As illustrated in Figure[2](https://arxiv.org/html/2608.26829#S3.F2), the normalized input is processed by two cooperating streams that share a single pretrained CLIP text encoder\. A*numerical stream*converts each series into patch tokens and variate tokens and passes them through the CLIP text encoder, producing temporal and inter\-variate representations that are further enriched with frequency information and with per\-variate natural\-language priors drawn from an offline textual knowledge bank\. A*visual stream*, active only for low\-dimensional datasets, renders each series as an image, encodes it with the frozen CLIP vision encoder, and contrastively aligns it with the temporal representation during training\. A forecast head then fuses these representations and de\-normalizes them to produce the prediction\. Concretely, the framework comprises five components that we describe in turn: a Frequency\-Enhanced Language Module, an Inter\-Variate Dependency Module, a Multi\-View Textual Semantic Fusion Module, a Vision\-Language Contrastive Alignment Module, and a Forecast Generator\.

![Refer to caption](https://arxiv.org/html/2608.26829v1/output_pipeline.png)Figure 2:Overview ofSAGE\. Shared CLIP text branches encode frequency\-enhanced temporal tokens and cross\-variable context\. Variable\-specific text views use gated attention, while hard statistics follow a separate residual bypass\. During training, the frozen CLIP vision encoder provides contrastive supervision\. Fused representations are decoded and inverse\-normalized\. The lower callout traces a real ETTh1/HUFL knowledge example\.### 3\.1Frequency\-Enhanced Language Module

This module treats time series patches as tokens for the pretrained CLIP text Transformer\.

We first apply reversible instance normalization to mitigate distribution shift\. For variableii, RevIN stores its meanμ\(i\)\\mu^\{\(i\)\}and standard deviationσ\(i\)\\sigma^\{\(i\)\}for output de\-normalization\. We omit the batch index in per\-variable equations for clarity\. The normalized sequence𝐱\(i\)∈ℝT\\mathbf\{x\}^\{\(i\)\}\\in\\mathbb\{R\}^\{T\}is divided intoLpL\_\{p\}overlapping patches of lengthPPand strideSS\. Let𝐩j\(i\)∈ℝP\\mathbf\{p\}^\{\(i\)\}\_\{j\}\\in\\mathbb\{R\}^\{P\}denote patchjj\. A learnable tokenizer𝐖tok\\mathbf\{W\}\_\{\\mathrm\{tok\}\}maps it to the shared embedding widthDD:

𝐳j\(i\)=𝐖tok​𝐩j\(i\)\.\\mathbf\{z\}^\{\(i\)\}\_\{j\}=\\mathbf\{W\}\_\{\\mathrm\{tok\}\}\\mathbf\{p\}^\{\(i\)\}\_\{j\}\.
Time\-domain tokens alone may inadequately represent periodic patterns\. We apply a Hann\-windowed FFT to every patch and project its real and imaginary components into a frequency token𝐟j\(i\)∈ℝD\\mathbf\{f\}^\{\(i\)\}\_\{j\}\\in\\mathbb\{R\}^\{D\}\. The time token is the query, and the frequency token provides the key and value\. A learnable scalarα\\alpha, initialized at zero, controls this fusion:

𝐳j\(i\)←𝐳j\(i\)\+α​CrossAttn⁡\(𝐳j\(i\),𝐟j\(i\),𝐟j\(i\)\),\\mathbf\{z\}^\{\(i\)\}\_\{j\}\\leftarrow\\mathbf\{z\}^\{\(i\)\}\_\{j\}\+\\alpha\\,\\operatorname\{CrossAttn\}\\bigl\(\\mathbf\{z\}^\{\(i\)\}\_\{j\},\\mathbf\{f\}^\{\(i\)\}\_\{j\},\\mathbf\{f\}^\{\(i\)\}\_\{j\}\\bigr\),whereCrossAttn⁡\(Q,K,V\)\\operatorname\{CrossAttn\}\(Q,K,V\)denotes cross\-attention with queryQQ, keyKK, and valueVV\.

A learnable \[CLS\] token is prepended, and learnable positional embeddings are added to the patch sequence\. The sequence then enters the CLIP text Transformer without using its word embedding layer\. We replace the original positional embeddings with embeddings matched to the patch sequence\. The Transformer blocks and final LayerNorm are fine\-tuned at a reduced learning rate, while unused projection layers remain frozen\. The encoder outputs patch representations𝐑∈ℝB×N×\(Lp\+1\)×D\\mathbf\{R\}\\in\\mathbb\{R\}^\{B\\times N\\times\(L\_\{p\}\+1\)\\times D\}and temporal summaries𝐜text∈ℝB×N×D\\mathbf\{c\}\_\{\\text\{text\}\}\\in\\mathbb\{R\}^\{B\\times N\\times D\}from the \[CLS\] positions\.

### 3\.2Cross\-Variate Context Modeling Module

The temporal encoder processes each variable independently, but multivariate forecasting also requires dependencies across variables\.

Each full normalized sequence𝐱\(i\)\\mathbf\{x\}^\{\(i\)\}is mapped to the shared embedding width by a linear tokenizer and augmented with a learnable variable identifier\. TheNNresulting tokens pass through the same CLIP text encoder used above\. Its output𝐇∈ℝB×N×D\\mathbf\{H\}\\in\\mathbb\{R\}^\{B\\times N\\times D\}contains contextual representations of all variables\. Thus, the shared encoder processes patch sequences to capture temporal structure and variable sequences to capture cross\-variable structure\.

The temporal summaries query𝐇\\mathbf\{H\}through cross\-attention\. Layer normalizationLN\\operatorname\{LN\}and a feed\-forward networkFFN\\operatorname\{FFN\}form residual updates:

𝐯′=LN⁡\(𝐜text\+CrossAttn⁡\(LN⁡\(𝐜text\),LN⁡\(𝐇\),LN⁡\(𝐇\)\)\),\\mathbf\{v\}^\{\\prime\}=\\operatorname\{LN\}\\\!\\left\(\\mathbf\{c\}\_\{\\text\{text\}\}\+\\operatorname\{CrossAttn\}\\bigl\(\\operatorname\{LN\}\(\\mathbf\{c\}\_\{\\text\{text\}\}\),\\operatorname\{LN\}\(\\mathbf\{H\}\),\\operatorname\{LN\}\(\\mathbf\{H\}\)\\bigr\)\\right\),𝐯′=LN⁡\(𝐯′\+FFN⁡\(𝐯′\)\)\.\\mathbf\{v\}^\{\\prime\}=\\operatorname\{LN\}\\\!\\left\(\\mathbf\{v\}^\{\\prime\}\+\\operatorname\{FFN\}\(\\mathbf\{v\}^\{\\prime\}\)\\right\)\.
The result𝐯′∈ℝB×N×D\\mathbf\{v\}^\{\\prime\}\\in\\mathbb\{R\}^\{B\\times N\\times D\}combines each variable’s temporal summary with information selected from all variables\.

### 3\.3Multi\-View Textual Semantic Fusion Module

This module is the central contribution of our work\. It injects external semantic knowledge into the forecasting pipeline through three complementary mechanisms\.

Table 1:Offline variable\-specific knowledge resources\. A real ETTh1/HUFL example is visualized in Figure[2](https://arxiv.org/html/2608.26829#S3.F2)\.For each variable, we construct descriptions from domain metadata and training\-split statistics\. Table[1](https://arxiv.org/html/2608.26829#S3.T1)summarizes the knowledge sources, while Figure[2](https://arxiv.org/html/2608.26829#S3.F2)shows representative HUFL examples\. The CLIP text encoder maps the descriptions to the shared embedding space\. We also compute a 12\-dimensional statistical vector𝐬\(i\)∈ℝ12\\mathbf\{s\}^\{\(i\)\}\\in\\mathbb\{R\}^\{12\}for variableii\. All descriptions, embeddings, and statistical vectors are cached before model training\.

#### Template\-based Construction Pipeline\.

Each human\-readable template combines domain metadata with data\-driven features\. The metadata include the variable name, unit, and domain role from the dataset documentation\. Training\-set features include distributional statistics, trend slope, dominant FFT period, and Pearson correlations with other variables\. The templates may be authored or refined with optional LLM assistance during offline preparation\. The relational view names the most correlated variables, restoring context that a channel\-independent encoder would otherwise omit\. Table[5](https://arxiv.org/html/2608.26829#S4.T5)evaluates five enhancement modes that control which feature groups are verbalized\.

Let𝐄mv∈ℝN×K×D\\mathbf\{E\}\_\{\\mathrm\{mv\}\}\\in\\mathbb\{R\}^\{N\\times K\\times D\}containKKcached views for each variable\. Variable\-wise cross\-attention uses𝐜text\(b,i\)\\mathbf\{c\}\_\{\\text\{text\}\}^\{\(b,i\)\}as its query and produces an aggregated embedding𝐞\(b,i\)∈ℝD\\mathbf\{e\}^\{\(b,i\)\}\\in\\mathbb\{R\}^\{D\}\. When time\-dependent text is available, the input timestamp selects the corresponding slot embedding and appends it as an additional view\.

The aggregated embedding is fused with the cross\-variate representation through a learnable gateg\(i\)g^\{\(i\)\}for variableii:

𝐯text′\(b,i\)\\displaystyle\\mathbf\{v\}\_\{\\text\{text\}\}^\{\\prime\\,\(b,i\)\}=𝐯′\(b,i\)\\displaystyle=\{\}\\mathbf\{v\}^\{\\prime\\,\(b,i\)\}\+g\(i\)​CrossAttn⁡\(LN⁡\(𝐯′\(b,i\)\),LN⁡\(𝐞\(b,i\)\),𝐞\(b,i\)\)\\displaystyle\+g^\{\(i\)\}\\operatorname\{CrossAttn\}\\\!\\left\(\\operatorname\{LN\}\(\\mathbf\{v\}^\{\\prime\\,\(b,i\)\}\),\\operatorname\{LN\}\(\\mathbf\{e\}^\{\(b,i\)\}\),\\mathbf\{e\}^\{\(b,i\)\}\\right\)
The gate is initialized to a small positive value, allowing every variable to learn its own reliance on textual priors\. Statistical features bypass attention\. A two\-layer networkMLPstat:ℝ12→ℝD\\operatorname\{MLP\}\_\{\\mathrm\{stat\}\}:\\mathbb\{R\}^\{12\}\\rightarrow\\mathbb\{R\}^\{D\}and a separate learnable gateγstat\\gamma\_\{\\mathrm\{stat\}\}inject them after text attention:

𝐯^′\(b,i\)=𝐯text′\(b,i\)\+γstat​MLPstat⁡\(𝐬\(i\)\)\.\\hat\{\\mathbf\{v\}\}^\{\\prime\\,\(b,i\)\}=\\mathbf\{v\}\_\{\\text\{text\}\}^\{\\prime\\,\(b,i\)\}\+\\gamma\_\{\\mathrm\{stat\}\}\\,\\operatorname\{MLP\}\_\{\\mathrm\{stat\}\}\\\!\\left\(\\mathbf\{s\}^\{\(i\)\}\\right\)\.This bypass complements soft textual semantics with hard numerical evidence without consuming attention capacity\.

We also align temporal and aggregated textual representations:

ℒtext=1−1B​N​∑b=1B∑i=1Ncos⁡\(𝐜text\(b,i\),𝐞\(b,i\)\)\.\\mathcal\{L\}\_\{\\text\{text\}\}=1\-\\frac\{1\}\{BN\}\\sum\_\{b=1\}^\{B\}\\sum\_\{i=1\}^\{N\}\\cos\\\!\\left\(\\mathbf\{c\}\_\{\\text\{text\}\}^\{\(b,i\)\},\\mathbf\{e\}^\{\(b,i\)\}\\right\)\.

### 3\.4Vision\-Language Contrastive Alignment Module

WhenN≤50N\\leq 50, an auxiliary vision\-language objective provides additional training supervision\. Each normalized variable sequence is rendered as a color\-coded line chart, encoded by the frozen CLIP vision encoder, and mapped to the shared width by a learnable projection𝐖proj\\mathbf\{W\}\_\{\\mathrm\{proj\}\}:

𝐜img\(b,i\)=𝐖proj​CLIP​\-​ViT⁡\(Render⁡\(𝐱\(b,i\)\)\),𝐜img\(b,i\)∈ℝD\\mathbf\{c\}\_\{\\text\{img\}\}^\{\(b,i\)\}=\\mathbf\{W\}\_\{\\mathrm\{proj\}\}\\,\\operatorname\{CLIP\\text\{\-\}ViT\}\\\!\\left\(\\operatorname\{Render\}\(\\mathbf\{x\}^\{\(b,i\)\}\)\\right\),\\quad\\mathbf\{c\}\_\{\\text\{img\}\}^\{\(b,i\)\}\\in\\mathbb\{R\}^\{D\}
Alignment is performed*independently for each variate*\. For variateii, let𝐒img\(i\),𝐒text\(i\)∈ℝB×D\\mathbf\{S\}\_\{\\text\{img\}\}^\{\(i\)\},\\mathbf\{S\}\_\{\\text\{text\}\}^\{\(i\)\}\\in\\mathbb\{R\}^\{B\\times D\}stack the L2\-normalized visual and temporal summaries of theBBsamples in the batch\. A bidirectional InfoNCE loss matches each sample to its own rendering, using the remainingB−1B\-1same\-variate samples as in\-batch negatives, and the result is averaged over theNNvariates:

ℒalign=1N∑i=1N12\[\\displaystyle\\mathcal\{L\}\_\{\\text\{align\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\frac\{1\}\{2\}\\Big\[CE⁡\(τ​𝐒img\(i\)​𝐒text\(i\)⊤,𝐈B\)\\displaystyle\\operatorname\{CE\}\\\!\\left\(\\tau\\,\\mathbf\{S\}\_\{\\text\{img\}\}^\{\(i\)\}\\mathbf\{S\}\_\{\\text\{text\}\}^\{\(i\)\\top\},\\mathbf\{I\}\_\{B\}\\right\)\+CE\(τ𝐒text\(i\)𝐒img\(i\)⊤,𝐈B\)\],\\displaystyle\+\\operatorname\{CE\}\\\!\\left\(\\tau\\,\\mathbf\{S\}\_\{\\text\{text\}\}^\{\(i\)\}\\mathbf\{S\}\_\{\\text\{img\}\}^\{\(i\)\\top\},\\mathbf\{I\}\_\{B\}\\right\)\\Big\],
where𝐈B\\mathbf\{I\}\_\{B\}denotes the batch\-level matching\-pair targets \(the identity assignment over theBBsamples\) andτ=exp⁡\(s\)\\tau=\\exp\(s\)is the learnable logit scale\.

A curriculum schedule gradually increases the concentration of hard negatives\. This module is active only during training\. ForN\>50N\>50, rendering hundreds of variable\-wise images is impractical, so the vision loss is disabled and textual knowledge fusion remains active\.

### 3\.5Forecast Generator and Training Objective

The augmented representation𝐯^′\\hat\{\\mathbf\{v\}\}^\{\\prime\}is added to the final position of𝐑\\mathbf\{R\}\. This operation injects cross\-variable and external knowledge without replacing earlier patch representations\. The resulting token sequence is flattened and mapped toTpredT\_\{\\mathrm\{pred\}\}forecast steps by a two\-layer MLP with GELU activation\. RevIN then restores the original scale\.

ForN\>50N\>50, the full variable\-wise patch path is replaced by a lightweight decoder for computational practicality\. Cross\-variate context modeling and textual semantic fusion remain active, and a linear layer maps each augmented variable representation toTpredT\_\{\\mathrm\{pred\}\}outputs\. When time\-dependent descriptions are available, the selected slot embedding supplies an additional gated correction at each forecast step\.

The total objective combines forecast MSE with the two alignment losses:

ℒ=ℒMSE\+λalign​ℒalign\+λtext​ℒtext,\\mathcal\{L\}=\\mathcal\{L\}\_\{\\text\{MSE\}\}\+\\lambda\_\{\\text\{align\}\}\\mathcal\{L\}\_\{\\text\{align\}\}\+\\lambda\_\{\\text\{text\}\}\\mathcal\{L\}\_\{\\text\{text\}\},whereℒMSE\\mathcal\{L\}\_\{\\text\{MSE\}\}is the error between𝐘^\\hat\{\\mathbf\{Y\}\}and the ground\-truth future sequence\.

The coefficientsλalign\\lambda\_\{\\text\{align\}\}andλtext\\lambda\_\{\\text\{text\}\}weight vision\-language and text alignment, respectively\. The CLIP text encoder is optimized at a reduced learning rate to preserve pretrained knowledge while allowing task\-specific adaptation\.

## 4Experiments

### 4\.1Setup

#### Datasets\.

We evaluate SAGE on 8 widely\-used long\-term forecasting benchmarks and the M4 short\-term forecasting competition\. The long\-term benchmarks include: ETTh1, ETTh2, ETTm1, ETTm2 \(electricity transformer temperature,N=7N\{=\}7\)\([Zhou et al\. 2020](https://arxiv.org/html/2608.26829#bib.bib26)\), Electricity \(ECL,N=321N\{=\}321\), Traffic \(N=862N\{=\}862\), Weather \(N=21N\{=\}21\)\([Wu et al\. 2021](https://arxiv.org/html/2608.26829#bib.bib10)\), and Exchange \(N=8N\{=\}8\)\([Lai et al\. 2017](https://arxiv.org/html/2608.26829#bib.bib33)\)\. For long\-term forecasting, we use a fixed lookback windowT=96T\{=\}96and prediction horizonsTpred∈\{96,192,336,720\}T\_\{\\mathrm\{pred\}\}\\in\\\{96,192,336,720\\\}\. For short\-term forecasting, we evaluate on the M4 dataset\([Makridakis et al\. 2020](https://arxiv.org/html/2608.26829#bib.bib32)\)comprising 100,000 time series across six frequencies \(Yearly, Quarterly, Monthly, Weekly, Daily, Hourly\)\.

Table 2:Long\-term benchmark characteristics and SAGE pathways\. All datasets useT=96T=96and prediction horizons in\{96,192,336,720\}\\\{96,192,336,720\\\}\.Table[2](https://arxiv.org/html/2608.26829#S4.T2)highlights the diversity of the evaluation suite\. The four ETT datasets measure electricity\-transformer behavior at hourly and 15\-minute resolutions; ECL and Traffic stress scalability with hundreds of variables; Weather provides densely sampled meteorological signals; and Exchange contains daily financial series\.

#### Metrics\.

Long\-term forecasting is evaluated with Mean Squared Error \(MSE\) and Mean Absolute Error \(MAE\)\. Short\-term forecasting follows the M4 competition protocol with SMAPE, MASE, and Overall Weighted Average \(OWA\)\.

Table 3:Dataset\-averaged MSE / MAE comparison across prediction horizons\{96,192,336,720\}\\\{96,192,336,720\\\}with lookbackT=96T\{=\}96\.Bold: best;underline: second best\.
#### Implementation\.

SAGE fine\-tunes the CLIP ViT\-B/32 text encoder and keeps its vision encoder frozen\. RevIN\-normalized series use frequency\-enhanced patches of length 16 and stride 8\([Kim et al\. 2021](https://arxiv.org/html/2608.26829#bib.bib34)\); offline descriptions are fused through per\-variable gates\. ForN\>50N\{\>\}50, a lightweight path bypasses patching\. Training uses Adam, cosine scheduling, a reduced learning rate for the CLIP text encoder, and bidirectional InfoNCE alignment\. We evaluate 651 configurations across all datasets, horizons, and five text\-enhancement modes\.

#### Environment and Computational Cost\.

All experiments are conducted on an Ubuntu 22\.04 server with AMD EPYC 9654 processors, 768 GB RAM, and four NVIDIA A800 GPUs\. Each run uses one GPU\. The software stack comprises Python 3\.11, PyTorch 2\.5\.1, and CUDA 12\.1\. Training cost depends on dataset dimensionality and the active pathway\. One epoch takes roughly 17 s on ETTh1, 21 s on Exchange, 130 s on ECL, 250 s on Weather, and 230–290 s on Traffic\. Early stopping typically yields convergence within 10 epochs, ranging from under 3 minutes on ETT datasets to under 45 minutes on Traffic\. The complete 651\-run sweep consumed approximately 1,000 single\-GPU hours and was parallelized across the four GPUs\. Offline text construction takes only a few seconds per dataset and is amortized across all runs\.

### 4\.2Main Results

As shown in Table[3](https://arxiv.org/html/2608.26829#S4.T3), SAGE obtains the best average MSE on 7 of 8 datasets and the best average MAE on 6 of 8\. Its overall average MSE is 0\.331, a 4\.1% reduction from the 0\.345 achieved by iTransformer\([Liu et al\. 2024](https://arxiv.org/html/2608.26829#bib.bib12)\)\.

Amplifier\([Fei et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib36)\)and DUET\([Qiu et al\. 2025](https://arxiv.org/html/2608.26829#bib.bib37)\)remain strongest on ETTm2, each reaching 0\.280 MSE and 0\.331 MAE\. SRSNet\([Wu et al\. 2026](https://arxiv.org/html/2608.26829#bib.bib38)\)obtains the best MAE on ETTh1\. Across the full table, SAGE maintains an overall average MSE of 0\.331 and MAE of 0\.348 with the fixed lookbackT=96T\{=\}96\. The evaluated datasets range from 8 to 862 variables, showing that the gains extend across very different dimensionalities\. Text resources are prepared offline, so no LLM is invoked in the training or forecasting loop\.The appendix provides MSE and MAE for all 32 dataset\-horizon combinations\.

Table 4:M4 short\-term forecasting\. Lower is better; bold and underline mark the best and second\-best results\.On the M4 short\-term benchmark in Table[4](https://arxiv.org/html/2608.26829#S4.T4), SAGE leads all three metrics with an OWA of 0\.834\. TimesNet and N\-HiTS obtain OWA values of 0\.851 and 0\.861, respectively\. The result extends SAGE beyond long\-horizon forecasting: frequency\-aware text descriptions complement patch representations at short horizons as well\.

### 4\.3Text Enhancement Analysis

We conduct a systematic ablation over 5 text enhancement modes across all 8 datasets and 4 prediction horizons\. For each dataset\-horizon pair, all architectural and text\-mode hyperparameters are selected using validation MSE\. The selected configuration is then evaluated once on the held\-out test split\. The 5 modes progressively enrich the text description\.\+Statadds mean, variance, skewness, and kurtosis\.\+Dynadds moving averages and trend slopes\.\+Stat\+Dyn, abbreviated as \+S\+D, combines both groups\.\+Dyn\+Freqadds FFT\-derived spectral features to dynamic descriptions\.\+Fullincludes statistical, dynamic, spectral, and cross\-variable features\.

Table 5:Average text\-enhancement gains\. “Mode” is the best enhancement family andΔ\\Deltais the relative MSE reduction\.As shown in Table[5](https://arxiv.org/html/2608.26829#S4.T5), text enhancement improves forecasting in 31 of 32 dataset\-horizon combinations, with an average MSE reduction of 2\.8%\. Among the modes, \+Stat is the most reliable \(78\.1% win rate across all experiments, selected as the best mode in 45\.2% of combinations\), followed by \+Dyn \(22\.6%\)\. The combined mode \+Stat\+Dyn achieves the highest peak improvement \(6\.3% on Exchange\), suggesting that statistical and dynamic features provide complementary information\. Interestingly, modes incorporating frequency features \(\+Dyn\+Freq, \+Full\) are less consistently beneficial, likely because the frequency\-enhanced patch embedding already captures spectral structure\.

Text gains tend to be larger on lower\-dimensional datasets, although the relationship is not strictly monotonic\.\(Table[5](https://arxiv.org/html/2608.26829#S4.T5), columnNNvs\.Δ\\Delta\)\. Exchange \(N=8N\{=\}8\) achieves 6\.3% improvement, Weather \(N=21N\{=\}21\) achieves 3\.9%, ECL \(N=321N\{=\}321\) achieves 1\.2%, and Traffic \(N=862N\{=\}862\) achieves 0\.9%\. This pattern reflects two complementary factors: \(1\) the per\-variate gate can more precisely modulate text absorption when the number of channels is small, and \(2\) low\-dimensional datasets have fewer cross\-channel redundancies, making external text knowledge a proportionally larger information source\. The per\-variable gates can suppress unhelpful textual information, which limits degradation on high\-dimensional datasets\.

### 4\.4Vision Enhancement Analysis

We isolate the contribution of visual supervision by retraining SAGE withλalign=0\\lambda\_\{\\mathrm\{align\}\}=0, while keeping the model architecture, textual knowledge, optimization settings, and all other loss terms unchanged\. Because the visual branch is enabled only whenN≤50N\\leq 50, the ablation is conducted on the six eligible datasets\. Table[6](https://arxiv.org/html/2608.26829#S4.T6)reports the test MSE averaged over the four prediction horizons\.

Vision–temporal alignment improves forecasting on all six datasets, reducing the average MSE from 0\.348 to 0\.343, or 1\.6% relatively\. The largest gain is obtained on ETTm2 at 2\.5%, followed by ETTh1 and ETTm1 at 1\.9% and 1\.8%\. The ETT and Exchange series generally exhibit visually identifiable trends and periodic structures, allowing the rendered views to provide shape information complementary to the numerical encoder\. These consistent gains indicate that the frozen CLIP vision encoder acts as an effective contrastive regularizer, encouraging temporal representations to preserve global trends, periodic geometry, and regime changes\.

The smaller 0\.4% gain on Weather can be related to its dataset characteristics\. Weather contains 21 heterogeneous and strongly coupled indicators sampled every 10 minutes\([Wu et al\. 2021](https://arxiv.org/html/2608.26829#bib.bib10)\); thus, a lookback of 96 covers only 16 hours rather than a complete daily cycle\. Moreover, rendering each variable independently omits interactions among temperature, humidity, pressure, wind, and radiation, while similar meteorological channels may act as false negatives in the identity\-based InfoNCE objective\. Consequently, the visual branch provides less additional information beyond the frequency\-enhanced and cross\-variate representations\.

Importantly, the visual branch is used only during training and does not provide image features directly to the forecast head\. Therefore, the observed improvements come from representation regularization rather than additional inference inputs, and introduce no visual computation at forecasting time\.

Table 6:Average vision\-enhancement gains\.Δ\\Deltais the relative MSE reduction\.

## 5Conclusion

We present SAGE, a vision\-language framework that augments time series forecasting with template\-based textual knowledge\. SAGE repurposes a pre\-trained CLIP ViT\-B/32 as a dual\-use backbone, freezing the vision encoder as a contrastive anchor while fine\-tuning the text encoder\. On this backbone, it injects statistical, dynamic, and spectral descriptions into the forecasting pipeline, with no language model in the forecasting loop\. Three design choices are central to the framework: \(1\) a dual\-use CLIP backbone, with a trainable text encoder and a frozen vision encoder for training\-only contrastive supervision; \(2\) variable\-specific multi\-view knowledge injection with independent gates and a statistical bypass; and \(3\) frequency\-enhanced temporal encoding combined with cross\-variable context\.

SAGE obtains the best average MSE on 7 of 8 long\-term datasets, the best average MAE on 6 of 8, and the best M4 OWA of 0\.834\. Text enhancement improves 31 of 32 dataset\-horizon settings\. These results show that aligned temporal, relational, textual, and visual evidence can improve a compact forecasting backbone\.

#### Limitations and Future Work\.

Several directions remain open\. First, the current text generation relies on hand\-crafted templates; learning to compose descriptions end\-to\-end, potentially via a small language model distilled from domain experts, could further improve text quality and domain adaptability\. Second, the text benefit diminishes on high\-dimensional datasets, suggesting that the per\-variate gate may benefit from group\-wise or hierarchical designs that share text representations across correlated channels\. Third, the current descriptions are constructed from dataset metadata and training\-split statistics, without incorporating external event or ontology information could unlock additional gains, particularly for datasets where external context drives regime changes\. Finally, extending SAGE to multivariate\-to\-multivariate generation and probabilistic forecasting would broaden its applicability to real\-world decision\-making scenarios\.

## References

- Ansariet al\.\(2024\)A\. F\. Ansari, L\. Stella, C\. Turkmen, X\. Zhang, P\. Mercado, H\. Shen, O\. Shchur, S\. S\. Rangapuram, S\. P\. Arango, S\. Kapoor,et al\.Chronos: learning the language of time series\.arXiv preprint arXiv:2403\.07815\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1)\.
- Daset al\.\(2024\)A\. Das, W\. Kong, R\. Sen, and Y\. ZhouA decoder\-only foundation model for time\-series forecasting\.InForty\-first international conference on machine learning,Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1)\.
- Eldeleet al\.\(2021\)E\. Eldele, M\. Ragab, Z\. Chen, M\. Wu, C\. K\. Kwoh, X\. Li, and C\. GuanTime\-series representation learning via temporal and contextual contrasting\.arXiv preprint arXiv:2106\.14112\.Cited by:[§2\.3](https://arxiv.org/html/2608.26829#S2.SS3.p1.1)\.
- Feiet al\.\(2025\)J\. Fei, K\. Yi, W\. Fan, Q\. Zhang, and Z\. NiuAmplifier: bringing attention to neglected low\-energy components in time series forecasting\.InProceedings of the AAAI conference on artificial intelligence,Vol\.39,pp\. 11645–11653\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.6.1.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[§4\.2](https://arxiv.org/html/2608.26829#S4.SS2.p2.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.6)\.
- Goswamiet al\.\(2024\)M\. Goswami, K\. Szafer, A\. Choudhry, Y\. Cai, S\. Li, and A\. DubrawskiMoment: a family of open time\-series foundation models\.arXiv preprint arXiv:2402\.03885\.Cited by:[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1)\.
- Gruveret al\.\(2023\)N\. Gruver, M\. Finzi, S\. Qiu, and A\. G\. WilsonLarge language models are zero\-shot time series forecasters\.Advances in neural information processing systems36,pp\. 19622–19635\.Cited by:[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p1.1)\.
- Heet al\.\(2026\)Z\. He, S\. Alnegheimish, and M\. ReimherrHarnessing vision\-language models for time series anomaly detection\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 21690–21698\.Cited by:[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p2.1)\.
- Jinet al\.\(2024\)M\. Jin, S\. Wang, L\. Ma, Z\. Chu, J\. Zhang, X\. Shi, P\. Chen, Y\. Liang, Y\. Li, S\. Pan,et al\.Time\-llm: time series forecasting by reprogramming large language models\.InInternational conference on learning representations,Vol\.2024,pp\. 23857–23880\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p2.1),[§1](https://arxiv.org/html/2608.26829#S1.p4.1),[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p1.1)\.
- Kimet al\.\(2021\)T\. Kim, J\. Kim, Y\. Tae, C\. Park, J\. Choi, and J\. ChooReversible instance normalization for accurate time\-series forecasting against distribution shift\.InInternational conference on learning representations,Cited by:[§4\.1](https://arxiv.org/html/2608.26829#S4.SS1.SSS0.Px3.p1.1)\.
- Laiet al\.\(2017\)G\. Lai, W\. Chang, Y\. Yang, and H\. LiuModeling long\-and short\-term temporal patterns with deep neural networks\.arXiv preprint arXiv:1703\.07015\.Cited by:[§4\.1](https://arxiv.org/html/2608.26829#S4.SS1.SSS0.Px1.p1.1)\.
- Leeet al\.\(2024\)S\. Lee, T\. Park, and K\. LeeSoft contrastive learning for time series\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 46815–46839\.Cited by:[§2\.3](https://arxiv.org/html/2608.26829#S2.SS3.p1.1)\.
- Liuet al\.\(2024\)Y\. Liu, T\. Hu, H\. Zhang, H\. Wu, S\. Wang, L\. Ma, and M\. LongItransformer: inverted transformers are effective for time series forecasting\.InInternational conference on learning representations,Vol\.2024,pp\. 11116–11140\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.5.1.1),[§1](https://arxiv.org/html/2608.26829#S1.p1.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[§4\.2](https://arxiv.org/html/2608.26829#S4.SS2.p1.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.5)\.
- Lyuet al\.\(2026\)S\. Lyu, S\. Zhong, W\. Ruan, Q\. Liu, Q\. Wen, H\. Xiong, and Y\. LiangOccamVTS: distilling vision models to 1% parameters for time series forecasting\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 24216–24225\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1)\.
- Makridakiset al\.\(2020\)S\. Makridakis, E\. Spiliotis, and V\. AssimakopoulosThe m4 competition: 100,000 time series and 61 forecasting methods\.International Journal of Forecasting36\(1\),pp\. 54–74\.Cited by:[§4\.1](https://arxiv.org/html/2608.26829#S4.SS1.SSS0.Px1.p1.1)\.
- Niet al\.\(2025\)J\. Ni, Z\. Zhao, C\. Shen, H\. Tong, D\. Song, W\. Cheng, D\. Luo, and H\. ChenHarnessing vision models for time series analysis: a survey\.arXiv preprint arXiv:2502\.08869\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p3.1)\.
- Nieet al\.\(2022\)Y\. Nie, N\. H\. Nguyen, P\. Sinthong, and J\. KalagnanamA time series is worth 64 words: long\-term forecasting with transformers\.arXiv preprint arXiv:2211\.14730\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.3.1.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.3)\.
- Panet al\.\(2024\)Z\. Pan, Y\. Jiang, S\. Garg, A\. Schneider, Y\. Nevmyvaka, and D\. Songs2s^\{2\}IP\-llm: semantic space informed prompt learning with llm for time series forecasting\.InForty\-first International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p4.1)\.
- Piaoet al\.\(2024\)X\. Piao, Z\. Chen, T\. Murayama, Y\. Matsubara, and Y\. SakuraiFredformer: frequency debiased transformer for time series forecasting\.InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining,pp\. 2400–2410\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.4.1.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.4)\.
- Qiuet al\.\(2025\)X\. Qiu, X\. Wu, Y\. Lin, C\. Guo, J\. Hu, and B\. YangDuet: dual clustering enhanced multivariate time series forecasting\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 1,pp\. 1185–1196\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.7.1.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[§4\.2](https://arxiv.org/html/2608.26829#S4.SS2.p2.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.7)\.
- Radfordet al\.\(2021\)A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark,et al\.Learning transferable visual models from natural language supervision\.InInternational conference on machine learning,pp\. 8748–8763\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p4.1)\.
- Shiet al\.\(2025\)X\. Shi, S\. Wang, Y\. Nie, D\. Li, Z\. Ye, Q\. Wen, and M\. JinTime\-moe: billion\-scale time series foundation models with mixture of experts\.InInternational conference on learning representations,Vol\.2025,pp\. 34635–34667\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1)\.
- Sunet al\.\(2024\)C\. Sun, H\. Li, Y\. Li, and S\. HongTEST: text prototype aligned embedding to activate llm’s ability for time series\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 37854–37881\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p1.1)\.
- Tanet al\.\(2024\)M\. Tan, M\. A\. Merrill, V\. Gupta, T\. Althoff, and T\. HartvigsenAre language models actually useful for time series forecasting?\.NeurIPS37,pp\. 60162–60191\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p1.1)\.
- Wang and Zhang \(2026\)Z\. Wang and W\. ZhangFrequency\-aware adaptive contrastive learning for sequential recommendation\.arXiv preprint arXiv:2601\.17057\.Cited by:[§2\.3](https://arxiv.org/html/2608.26829#S2.SS3.p1.1)\.
- Williamset al\.\(2024\)A\. R\. Williams, A\. Ashok, É\. Marcotte, V\. Zantedeschi, J\. Subramanian, R\. Riachi, J\. Requeima, A\. Lacoste, I\. Rish, N\. Chapados,et al\.Context is key: a benchmark for forecasting with essential textual information\.arXiv preprint arXiv:2410\.18959\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p3.1)\.
- Wooet al\.\(2024\)G\. Woo, C\. Liu, A\. Kumar, C\. Xiong, S\. Savarese, and D\. SahooUnified training of universal time series forecasting transformers\.InForty\-first International Conference on Machine Learning,Cited by:[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1)\.
- Wooet al\.\(2022\)G\. Woo, C\. Liu, D\. Sahoo, A\. Kumar, and S\. HoiCost: contrastive learning of disentangled seasonal\-trend representations for time series forecasting\.arXiv preprint arXiv:2202\.01575\.Cited by:[§2\.3](https://arxiv.org/html/2608.26829#S2.SS3.p1.1)\.
- Wuet al\.\(2022\)H\. Wu, T\. Hu, Y\. Liu, H\. Zhou, J\. Wang, and M\. LongTimesnet: temporal 2d\-variation modeling for general time series analysis\.arXiv preprint arXiv:2210\.02186\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.2.1.1),[§1](https://arxiv.org/html/2608.26829#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.2)\.
- Wuet al\.\(2021\)H\. Wu, J\. Xu, J\. Wang, and M\. LongAutoformer: decomposition transformers with auto\-correlation for long\-term series forecasting\.Advances in neural information processing systems34,pp\. 22419–22430\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p1.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.26829#S4.SS1.SSS0.Px1.p1.1),[§4\.4](https://arxiv.org/html/2608.26829#S4.SS4.p3.1)\.
- Wuet al\.\(2025\)X\. Wu, J\. Jin, W\. Qiu, P\. Chen, Y\. Shu, B\. Yang, and C\. GuoAurora: towards universal generative multimodal time series forecasting\.arXiv preprint arXiv:2509\.22295\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p2.1)\.
- Wuet al\.\(2026\)X\. Wu, X\. Qiu, H\. Cheng, Z\. Li, J\. Hu, C\. Guo, and B\. YangEnhancing time series forecasting through selective representation spaces: a patch perspective\.Advances in Neural Information Processing Systems38,pp\. 23328–23354\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.8.1.1),[§2\.2](https://arxiv.org/html/2608.26829#S2.SS2.p1.1),[§4\.2](https://arxiv.org/html/2608.26829#S4.SS2.p2.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.8)\.
- Yueet al\.\(2022\)Z\. Yue, Y\. Wang, J\. Duan, T\. Yang, C\. Huang, Y\. Tong, and B\. XuTs2vec: towards universal representation of time series\.InProceedings of the AAAI conference on artificial intelligence,Vol\.36,pp\. 8980–8987\.Cited by:[§2\.3](https://arxiv.org/html/2608.26829#S2.SS3.p1.1)\.
- Zenget al\.\(2023\)A\. Zeng, M\. Chen, L\. Zhang, and Q\. XuAre transformers effective for time series forecasting?\.InProceedings of the AAAI conference on artificial intelligence,Vol\.37,pp\. 11121–11128\.Cited by:[Table 7](https://arxiv.org/html/2608.26829#A0.T7.5.1.2.1.1.1),[§1](https://arxiv.org/html/2608.26829#S1.p1.1),[Table 3](https://arxiv.org/html/2608.26829#S4.T3.5.1.2.1)\.
- Zhanget al\.\(2022\)X\. Zhang, Z\. Zhao, T\. Tsiligkaridis, and M\. ZitnikSelf\-supervised contrastive pre\-training for time series via time\-frequency consistency\.Advances in neural information processing systems35,pp\. 3988–4003\.Cited by:[§2\.3](https://arxiv.org/html/2608.26829#S2.SS3.p1.1)\.
- Zhonget al\.\(2025\)S\. Zhong, W\. Ruan, M\. Jin, H\. Li, Q\. Wen, and Y\. LiangTime\-vlm: exploring multimodal vision\-language models for augmented time series forecasting\.arXiv preprint arXiv:2502\.04395\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p2.1)\.
- Zhouet al\.\(2020\)H\. Zhou, S\. Zhang, J\. Peng, S\. Zhang, J\. Li, H\. Xiong, and W\. ZhangInformer: beyond efficient transformer for long sequence time\-series forecasting\.arXiv preprint arXiv:2012\.07436\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.26829#S4.SS1.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2022\)T\. Zhou, Z\. Ma, Q\. Wen, X\. Wang, L\. Sun, and R\. JinFedformer: frequency enhanced decomposed transformer for long\-term series forecasting\.InInternational conference on machine learning,pp\. 27268–27286\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p1.1)\.
- Zhouet al\.\(2023\)T\. Zhou, P\. Niu, L\. Sun, R\. Jin,et al\.One fits all: power general time series analysis by pretrained lm\.Advances in neural information processing systems36,pp\. 43322–43355\.Cited by:[§1](https://arxiv.org/html/2608.26829#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.26829#S2.SS1.p1.1)\.

Appendix A: Full Per\-Horizon Forecasting Results

Table 7:Full per\-horizon long\-term forecasting results \(MSE↓\\downarrow/ MAE↓\\downarrow\)\. We compare with representative methods spanning 2023–2025 under lookbackT=96T\{=\}96\.Bold: best;underline: second best\. “\-”: not available from the source\.