Lotus：通过学习预测来理解时间序列

# Lotus: Learning to Understand Time-series by Learning to Forecast
Source: [https://arxiv.org/html/2610.18877](https://arxiv.org/html/2610.18877)
Huu Hiep Nguyen\*Minh Hoang NguyenManh NguyenHung LeAffiliation:Deakin’s Applied Artificial Intelligence Initiative, AustraliaAffiliation:\{v\.do, huu\.n, s223669184, manh\.nguyen, thai\.le\}@deakin\.edu\.au

###### Abstract

Time\-series language models typically align numerical signals with language using labels, captions, or question–answer pairs\. Although such supervision encourages linguistic representations, it does not explicitly require them to preserve information about how the underlying time\-series signal evolves\. We introduce*forecast grounding*, a pretraining principle that requires language\-aligned time\-series representations to remain predictive of the future evolution before they are adapted to downstream tasks\. We instantiate this principle inLotus, which divides each time\-series channel into non\-overlapping patches and maps them into the LLM embedding space\. It then predicts the next patch from the LLM hidden states of the time\-series tokens, encouraging the learned representation to retain temporal information\. To capture both temporal changes and periodic patterns, we also propose a*temporal–spectral patch representation*that combines each raw time\-series patch with its spectral representation before mapping it into the LLM embedding space\. Across three language model backbones and six downstream evaluation datasets,Lotusachieves the best average performance with minimal trainable parameters, reaching59\.91%59\.91\\%accuracy and outperforming the strongest matched baseline by3\.36%3\.36\\%\. Our representation analyses further show thatLotususes the numerical input and preserves temporal structure in its learned patch representations\. These results establish forecast grounding as a promising pretraining principle, enabling compact time\-series language models to outperform models 25–50× their size on TSRBench\.

\*\*footnotetext:Equal contribution\.## 1Introduction

Figure 1:Six\-task average exact match accuracy versus trainable parameters \(LoRA\)\.G and Q denote Gemma3 and Qwen3 backbones, respectively\.Time series describe processes that evolve over time, making both the observed values and their temporal dynamics important for understanding the underlying signal\. Time\-series foundation models exploit this structure through objectives such as forecasting and reconstruction to learn transferable numerical representations\([Ansari et al\., 2024](https://arxiv.org/html/2610.18877#bib.bib20);[Das et al\., 2024](https://arxiv.org/html/2610.18877#bib.bib2);[Woo et al\., 2024](https://arxiv.org/html/2610.18877#bib.bib3);[Nie et al\., 2023](https://arxiv.org/html/2610.18877#bib.bib28)\)\. At the same time, many practical tasks require understanding time series through natural language, such as identifying when an abnormal event occurred, comparing changes across variables, or answering questions about a physiological signal\. Recent time\-series language models address this setting by mapping numerical observations into representations that pretrained large language models \(LLMs\) can process\([Li et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib22);[Cheng et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib17);[Xie et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib9);[Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8)\)\. The effectiveness of this approach therefore depends on what information these time\-series representations preserve\.

Existing time\-series language models are primarily aligned through language supervision, including class labels, captions, instructions, and question–answer pairs\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8);[Xie et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib9)\)\. These objectives encourage the representation to preserve information needed to produce the correct language output, but they do not explicitly require it to retain how the underlying signal may evolve\. For example, two motion segments may both correspond to*walking*while differing in speed, local dynamics, and likely continuation\. The same language target may therefore correspond to temporal states with different future behavior\. We argue that forecasting provides a natural additional constraint for learning such representations\. Based on this idea, we introduceforecast grounding: during pretraining, a language\-aligned time\-series representation is also required to predict the future evolution of its source signal\. If two observations share the same language target but lead to different futures, the forecasting objective provides an additional learning signal that distinguishes them\. Forecasting therefore serves as a pretraining constraint that encourages the representation supplied to the LLM to preserve future\-relevant temporal information, rather than as an additional downstream task\.

We instantiate this principle inLotus\. Each channel is divided into non\-overlapping patches and mapped by an encoder and projector to soft tokens in the LLM embedding space\. Temporal and channel positions identify the structure of multivariate inputs, while the causal LLM provides interaction across signal patches and text\. During pretraining, language examples train the model to generate answers, while forecasting examples train it to predict the next patch from the LLM states of the signal tokens\. As a result, the forecasting loss directly shapes the representation the language model uses\. After pretraining, the model is jointly adapted to different downstream time\-series tasks\. Beyond the pretraining objective, we also consider how local time\-series information is represented before entering the LLM\. We propose a*temporal–spectral patch representation*that combines the raw temporal patch with its Fourier transform before mapping it into the LLM embedding space\. The temporal view preserves local values and changes, while the spectral view exposes frequency content within each patch\. Longer\-range dependencies across patches are subsequently modeled by the language model\. We evaluateLotuswith three language backbones on six downstream splits covering activity recognition, sleep staging, ECG question answering, and general time\-series question answering\. As shown in Figure[1](https://arxiv.org/html/2610.18877#S1.F1),Lotusachieves the best average accuracy in every matched backbone group while using only a small number of trainable parameters\.

Our contributions are threefold:

1. 1\.We proposeforecast grounding, which uses future prediction during pretraining to encourage language\-aligned time\-series representations to preserve future\-relevant temporal information\.
2. 2\.We propose atemporal–spectral patch representationthat combines the raw time\-series patch with its spectral representation before mapping it into the LLM embedding space\. This allows the model to capture both local temporal changes and frequency patterns\.
3. 3\.We evaluateLotuswith three language model backbones on six downstream splits\.Lotusachieves the best average accuracy in every matched backbone group, demonstrating the effectiveness of combining forecast grounding with temporal–spectral patch representations for time\-series language modeling\.

## 2Related work

### 2\.1Time\-series language models and reasoning

Recent work extends pretrained language models to numerical time series and studies their ability to perform classification, interpretation, and reasoning over signals\. SensorLLM aligns motion signals with textual trend descriptions before activity\-recognition tuning, while InstructTime formulates time\-series classification through multimodal language modeling\([Li et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib22);[Cheng et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib17)\)\. ChatTS pairs synthetic multivariate signals with attribute descriptions and instructions for reasoning over temporal attributes and relationships, while Time\-MQA develops large\-scale question–answer data for numerical analysis and open\-ended interaction\([Xie et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib9);[Kong et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib23)\)\. In physiological domains, OpenTSLM studies activity recognition, sleep staging, and ECG question answering, while GEM grounds language generation in measurable ECG characteristics\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8);[Lan et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib18)\)\. TS\-Reasoner instead incorporates representations from a pretrained time\-series foundation model and aligns them with an LLM through instruction tuning\([Yu et al\., 2026b](https://arxiv.org/html/2610.18877#bib.bib24)\)\.

Despite this progress, reasoning directly over numerical signals remains difficult\.[Merrill et al\. \(2024\)](https://arxiv.org/html/2610.18877#bib.bib21)show that strong LLMs remain unreliable when time series are represented directly as text\. Existing approaches address this problem mainly by introducing stronger language supervision, task\-specific instruction data, or more informative numerical features\. These directions improve what the language model is taught or what numerical information it receives, but place less emphasis on how the underlying time\-series representation is shaped before downstream reasoning\.Lotusfocuses on this representation\. We encode each patch using both its temporal values and spectral information, providing the language model with complementary views of local signal behavior\. We further use forecasting during pretraining to encourage the resulting representation to retain information about future signal evolution\. The goal is therefore not to introduce another reasoning objective, but to learn a stronger numerical representation on which later language understanding and reasoning can operate\.

### 2\.2Time\-series–language alignment

A central challenge in time\-series language modeling is mapping continuous numerical signals into representations that pretrained language models can effectively process\. Existing methods adopt several alignment mechanisms\. Projection\-based approaches map encoded time\-series segments directly into the LLM embedding space\([Li et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib22);[Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8)\), while cross\-attention architectures retain separate signal representations that language states can attend to\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8);[Lan et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib18)\)\. TS\-Reasoner instead projects representations from a pretrained time\-series foundation model into the language\-model space\([Yu et al\., 2026b](https://arxiv.org/html/2610.18877#bib.bib24)\)\. These approaches establish effective connections between numerical signals and language, but the resulting representations are primarily shaped through language or downstream task supervision\.

A related line of work uses the time series itself as a learning signal\. PatchTST learns representations through masked\-patch reconstruction\([Nie et al\., 2023](https://arxiv.org/html/2610.18877#bib.bib28)\), while SimMTM reconstructs the original series from multiple masked views to preserve temporal variation\([Dong et al\., 2023](https://arxiv.org/html/2610.18877#bib.bib29)\)\. Prediction has also been explored in language\-model\-based time\-series methods\. Time\-LLM reprograms time\-series patches using text prototypes before passing them to a frozen LLM for forecasting\([Jin et al\., 2024](https://arxiv.org/html/2610.18877#bib.bib5)\), while TEST aligns time\-series embeddings with a text\-prototype space for prediction with a frozen language model\([Sun et al\., 2024](https://arxiv.org/html/2610.18877#bib.bib6)\)\. More closely related to our predictive objective, aLLM4TS uses causal next\-patch and multi\-patch prediction to learn transferable time\-series representations with an LLM\([Bian et al\., 2024](https://arxiv.org/html/2610.18877#bib.bib27)\)\. These methods demonstrate that reconstruction and prediction can provide useful supervision for learning temporal representations, but they are developed primarily for forecasting or conventional time\-series tasks\.Lotusbrings predictive supervision into time\-series–language alignment for downstream language tasks throughforecast grounding\. During pretraining, we predict future patches from the post\-LLM signal states, so the forecasting objective directly constrains the representations that are later used for language understanding and reasoning\. Together with the temporal–spectral patch representation, this encourages the language model to operate on signal representations that capture both local temporal dynamics and frequency\-domain structure while retaining information relevant to future signal evolution\.

## 3Methodology

Figure[2](https://arxiv.org/html/2610.18877#S3.F2)summarizesLotusin three parts\. First, the architecture divides each time\-series channel into patches and maps each patch to a soft time\-series token in the language\-model embedding space\. This design allows the representation of each patch to be contextualized jointly with other patches and language\. We use*patch*to refer to a segment of the original time series and*soft time\-series token*to refer to its projected representation supplied to the language model\. Second, forecast grounding applies predictive supervision to the language\-model states corresponding to these soft time\-series tokens\. We hypothesize that requiring the same representations used for language processing to retain information about future time series provides a complementary training signal for downstream time\-series understanding\. Third, downstream adaptation removes the next\-patch decoder and continues to adapt the time\-series encoder and LLM using the language objective alone\.

![Refer to caption](https://arxiv.org/html/2610.18877v1/main_fig.png)Figure 2:Overview ofLotus\.\(a\) Architecture:The architecture maps time\-series patches to soft tokens that are processed jointly with text by the pretrained language model\.\(b\) Pretraining:During forecast grounding pretraining, language supervision trains textual outputs while next\-patch prediction constrains LLM’s hidden states to retain future\-relevant information\.\(c\) Downstream Adaptation:During downstream adaptation, the next\-patch decoder is removed and the model is trained with language supervision only\.### 3\.1LotusArchitecture

#### Patchification\.

Let a multivariate time series be𝐒∈ℝC×L\\mathbf\{S\}\\in\\mathbb\{R\}^\{C\\times L\}, whereCCis the number of channels andLLis the sequence length\. Each channel is normalized to zero mean and unit variance over its observed, non\-padded extent, zero\-padded to a multiple of the patch lengthPP, and divided into non\-overlapping patches

𝐱c,n∈ℝP,c∈\{1,…,C\},n∈\{1,…,N\},\\mathbf\{x\}\_\{c,n\}\\in\\mathbb\{R\}^\{P\},\\qquad c\\in\\\{1,\\ldots,C\\\},\\quad n\\in\\\{1,\\ldots,N\\\},\(1\)whereN=⌈L/P⌉N=\\lceil L/P\\rceil\. The input therefore yieldsC​NCNpatch positions\. The encoder operates on each patch independently, while interaction across patches and channels is handled later by the pretrained language model\. We represent each patch in both temporal and Fourier coordinates to expose complementary views of the same local segment\. The temporal view preserves the raw within\-patch evolution, while the Fourier view provides an explicit frequency\-aware parameterization that may facilitate learning periodic structure\.

#### Time\-Series Encoder\.

While the raw temporal patch directly preserves local values and short\-term changes, patterns may be less explicit in the time domain\. We therefore propose a*temporal\-spectral patch representation*which represents each patch using temporal and spectral views of its local dynamics\. Specifically, for a patch𝐱c,n\\mathbf\{x\}\_\{c,n\}, we use the raw patch as the temporal view\. For the spectral view, letℱ⁡\(𝐱c,n\)∈ℂP/2\+1\\mathcal\{F\}\(\\mathbf\{x\}\_\{c,n\}\)\\in\\mathbb\{C\}^\{P/2\+1\}denote its real FFT\. We represent the complex spectrum by concatenating its real and imaginary components,

𝐟c,n=\[Re⁡ℱ⁡\(𝐱c,n\);Im⁡ℱ⁡\(𝐱c,n\)\]∈ℝ2​\(P/2\+1\)\.\\mathbf\{f\}\_\{c,n\}=\\left\[\\operatorname\{Re\}\\mathcal\{F\}\(\\mathbf\{x\}\_\{c,n\}\);\\operatorname\{Im\}\\mathcal\{F\}\(\\mathbf\{x\}\_\{c,n\}\)\\right\]\\in\\mathbb\{R\}^\{2\(P/2\+1\)\}\.\(2\)The temporal and spectral views are embedded separately by learnable projectionsWtW\_\{t\}andWfW\_\{f\}, and fused byWf​u​s​eW\_\{fuse\}

𝐡c,nt=Wt​𝐱c,n,𝐡c,nf=Wf​𝐟c,n,\\mathbf\{h\}^\{t\}\_\{c,n\}=W\_\{t\}\\mathbf\{x\}\_\{c,n\},\\qquad\\mathbf\{h\}^\{f\}\_\{c,n\}=W\_\{f\}\\mathbf\{f\}\_\{c,n\},\(3\)𝐡c,n=Wfuse​\[𝐡c,nt;𝐡c,nf\]\.\\mathbf\{h\}\_\{c,n\}=W\_\{\\mathrm\{fuse\}\}\\left\[\\mathbf\{h\}^\{t\}\_\{c,n\};\\mathbf\{h\}^\{f\}\_\{c,n\}\\right\]\.\(4\)where𝐡c,nt∈ℝdt\\mathbf\{h\}^\{t\}\_\{c,n\}\\in\\mathbb\{R\}^\{d\_\{t\}\}and𝐡c,nf∈ℝdf\\mathbf\{h\}^\{f\}\_\{c,n\}\\in\\mathbb\{R\}^\{d\_\{f\}\}are the temporal and spectral representations of patch\(c,n\)\(c,n\)\. Their concatenation is projected byWfuseW\_\{\\mathrm\{fuse\}\}into a shared fused representation𝐡c,n∈ℝdh\\mathbf\{h\}\_\{c,n\}\\in\\mathbb\{R\}^\{d\_\{h\}\}\. The fused representation is first normalized\. To preserve the structure of the multivariate input, we add a two\-dimensional positional encoding that identifies both the channel and temporal location of each patch\. Specifically, we use a channel embedding𝐩cc​h\\mathbf\{p\}^\{ch\}\_\{c\}and a patch\-position embedding𝐩np​o​s\\mathbf\{p\}^\{pos\}\_\{n\}, giving

𝐡~c,n=LN⁡\(𝐡c,n\)\+𝐩cc​h\+𝐩np​o​s\.\\widetilde\{\\mathbf\{h\}\}\_\{c,n\}=\\operatorname\{LN\}\(\\mathbf\{h\}\_\{c,n\}\)\+\\mathbf\{p\}^\{ch\}\_\{c\}\+\\mathbf\{p\}^\{pos\}\_\{n\}\.\(5\)Here, LN denotes layer normalization\. A transformer decoder then maps the patch representation to the language\-model widthdmodeld\_\{\\mathrm\{model\}\}\. We write the complete Time\-series Encoder module in Figure[2](https://arxiv.org/html/2610.18877#S3.F2)\(a\), including the projector, as

𝐳c,n=gϕ​\(𝐱c,n,c,n\)∈ℝdmodel,\\mathbf\{z\}\_\{c,n\}=g\_\{\\phi\}\(\\mathbf\{x\}\_\{c,n\},c,n\)\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{model\}\}\},\(6\)whereϕ\\phidenotes all trainable parameters of the time\-series encoder\. The vectors𝐳c,n\\mathbf\{z\}\_\{c,n\}are theC​NCNsoft time\-series tokens shown in the figure\.

#### Token Sequence\.

The encoded time series are inserted into the language\-model input as soft tokens while preserving the original ordering of the multimodal input\. Each time series𝐒\\mathbf\{S\}is mapped bygϕ​\(𝐒\)g\_\{\\phi\}\(\\mathbf\{S\}\)to a sequence of soft tokens and enclosed by learned⟨t​s⟩\\langle ts\\rangleand⟨/ts⟩\\langle/ts\\rangledelimiters\. When an input contains multiple time series, each series is represented by its own delimited soft\-token sequence, with any intervening text retained between them\. Thus, text, captions, and time\-series representations are interleaved in the same order as they appear in the original example\. The target answerAAis appended during teacher\-forced training\. The resulting multimodal token sequence, denoted by𝐄\\mathbf\{E\}, is processed by the pretrained causal language model,

𝐇=L​L​MΘ​\(𝐄\),\\mathbf\{H\}=LLM\_\{\\Theta\}\(\\mathbf\{E\}\),\(7\)where𝐇\\mathbf\{H\}denotes the output hidden representations for all positions in the sequence\.

### 3\.2Phase 1: Forecast Grounding Pretraining

Forecast grounding pretraining uses a mixture of forecasting, captioning, and time\-series question\-answering data, as shown in Figure[2](https://arxiv.org/html/2610.18877#S3.F2)\(b\)\. The time\-series encoder and projector are trained jointly with the next\-patch decoder, while the language model is adapted through LoRA\. We denote the encoder parameters byϕ\\phi, the next\-patch decoder parameters byψ\\psi, and the LoRA parameters byη\\eta\.

#### Language Objective\.

For language generation tasks like captioning and question\-answering examples, the supervision is a standard autoregressive language objective\. Let𝒜\\mathcal\{A\}denote the target answer positions, andata\_\{t\}denote the target token at positiontt\. We optimize:

ℒlang=−∑t∈𝒜logpϕ,η\(at∣𝐄<t\)\.\\mathcal\{L\}\_\{\\mathrm\{lang\}\}=\-\\sum\_\{t\\in\\mathcal\{A\}\}\\log p\_\{\\phi,\\eta\}\\left\(a\_\{t\}\\mid\\mathbf\{E\}\_\{<t\}\\right\)\.\(8\)This objective jointly updates the time\-series encoder’s parametersϕ\\phiand the LoRA parametersη\\etathrough the language\-model computation\.

#### Forecasting Objective\.

Forecasting examples provide the predictive component of pretraining\. For patch𝐱c,n\\mathbf\{x\}\_\{c,n\}, letu⁡\(c,n\)u\(c,n\)denote the corresponding time series token position\. The next\-patch decoderdψd\_\{\\psi\}predicts the normalized succeeding patch from the final hidden states of the LLM:

𝐱^c,n\+1=dψ​\(𝐇u⁡\(c,1\),…,𝐇u⁡\(c,n\)\)n\.\\widehat\{\\mathbf\{x\}\}\_\{c,n\+1\}=d\_\{\\psi\}\\\!\\left\(\\mathbf\{H\}\_\{u\(c,1\)\},\\ldots,\\mathbf\{H\}\_\{u\(c,n\)\}\\right\)\_\{n\}\.\(9\)In implementation,dψd\_\{\\psi\}projects the language\-model hidden states back to the time\-series representation space and applies a causally masked Transformer decoder, followed by a linear prediction head that outputs thePPvalues of the succeeding patch\. The causal Transformer allows the prediction at positionnnto condition on𝐇u⁡\(c,n\)\\mathbf\{H\}\_\{u\(c,n\)\}as well as preceding states from the same channel\.

Let𝒱=\{\(c,n\):1≤c≤C,1≤n<N\}\\mathcal\{V\}=\\left\\\{\(c,n\):1\\leq c\\leq C,\\;1\\leq n<N\\right\\\}denote the set of positions with an observed successor patch in the same channel\. Sequence\-padding patches and positions without a successor are excluded\. The forecasting loss is

ℒpred=1\|𝒱\|​∑\(c,n\)∈𝒱‖𝐱^c,n\+1−𝐱c,n\+1‖22\.\\mathcal\{L\}\_\{\\mathrm\{pred\}\}=\\frac\{1\}\{\|\\mathcal\{V\}\|\}\\sum\_\{\(c,n\)\\in\\mathcal\{V\}\}\\left\\\|\\widehat\{\\mathbf\{x\}\}\_\{c,n\+1\}\-\\mathbf\{x\}\_\{c,n\+1\}\\right\\\|\_\{2\}^\{2\}\.\(10\)
The decoder therefore operates on the LLM’s hidden state\. Gradients fromℒpred\\mathcal\{L\}\_\{\\mathrm\{pred\}\}propagate through the language\-model computation and jointly update the time\-series encoder’s parametersϕ\\phi, the LoRA parametersη\\eta, and the next\-patch decoder’s parametersψ\\psi\.

#### Optimization\.

We train on separate forecasting and language batches, so each optimization step uses eitherℒlang\\mathcal\{L\}\_\{\\mathrm\{lang\}\}orℒpred\\mathcal\{L\}\_\{\\mathrm\{pred\}\}\. LetDl​a​n​gD\_\{lang\}andDp​r​e​dD\_\{pred\}denote the language and forecasting data distributions, respectively\. The overall pretraining objective is

ℒPG=𝔼b∼𝒟lang​\[ℒlang​\(b\)\]\+𝔼b∼𝒟pred​\[ℒpred​\(b\)\]\.\\mathcal\{L\}\_\{\\mathrm\{PG\}\}=\\mathbb\{E\}\_\{b\\sim\\mathcal\{D\}\_\{\\mathrm\{lang\}\}\}\[\\mathcal\{L\}\_\{\\mathrm\{lang\}\}\(b\)\]\+\\mathbb\{E\}\_\{b\\sim\\mathcal\{D\}\_\{\\mathrm\{pred\}\}\}\[\\mathcal\{L\}\_\{\\mathrm\{pred\}\}\(b\)\]\.\(11\)Because the Phase\-1 corpora differ greatly in size, size\-proportional sampling would cause the largest forecasting corpus to dominate training\. We therefore upsample smaller sources and downsample larger ones according to predefined mixture weights, making the language–forecasting balance independent of corpus size\. Language batches updateϕ\\phiandη\\etathrough Eq\.[8](https://arxiv.org/html/2610.18877#S3.E8); forecasting batches updateϕ\\phi,η\\eta, andψ\\psithrough Eq\.[10](https://arxiv.org/html/2610.18877#S3.E10)\.

### 3\.3Phase 2: Downstream Adaptation

After pretraining, we remove the next\-patch decoder and jointly adapt one model across downstream tasks, fully fine\-tuning the encoder and projector while applying LoRA to the LLM:

minϕ,η⁡𝔼b∼𝒟down​\[ℒlang​\(b\)\]\.\\min\_\{\\phi,\\eta\}\\mathbb\{E\}\_\{b\\sim\\mathcal\{D\}\_\{\\mathrm\{down\}\}\}\[\\mathcal\{L\}\_\{\\mathrm\{lang\}\}\(b\)\]\.\(12\)We balance the five corpora by samplingB=40,000B=40\{,\}000examples from each per epoch\. For a corpus of sizeNkN\_\{k\}, we repeat the full corpusqk=⌊B/Nk⌋q\_\{k\}=\\lfloor B/N\_\{k\}\\rfloortimes, then fill the remainingrk=B−qk​Nkr\_\{k\}=B\-q\_\{k\}N\_\{k\}slots with examples selected without replacement using a seeded shuffle\. WhenNk\>BN\_\{k\}\>B, no full repetition is needed; we select a fixed subset ofBBexamples and reuse it across epochs\. The resulting200,000200\{,\}000\-example mixture gives each corpus equal representation, preventing larger datasets from dominating the sample count \(more details in Appendix[F](https://arxiv.org/html/2610.18877#A6)\)\.

## 4Experiments

### 4\.1Datasets and Implementation Details

#### Phase 1\.

Our pretraining phase uses four datasets: half\-hourly electricity demand from Monash\([Godahewa et al\., 2021](https://arxiv.org/html/2610.18877#bib.bib15)\), our synthetic multivariate corpus \(details in Appendix[E](https://arxiv.org/html/2610.18877#A5)\), TSQA multiple\-choice questions on trend, volatility, seasonality, and outliers\([Kong et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib23)\), and M4 series with generated captions\([Makridakis et al\., 2020](https://arxiv.org/html/2610.18877#bib.bib14)\)\. We retain OpenTSLM’s language pretraining data\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8)\), adding Monash and our synthetic as forecasting pretraining datasets\.

#### Phase 2\.

The downstream adaptation phase finetunes the checkpoints of Phase 1 on five datasets: HAR\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8)\), PAMAP2\([Reiss and Stricker, 2012](https://arxiv.org/html/2610.18877#bib.bib12)\), Sleep\-EDF\([Kemp et al\., 2000](https://arxiv.org/html/2610.18877#bib.bib13)\), ECG\-QA\([Oh et al\., 2023](https://arxiv.org/html/2610.18877#bib.bib10);[Wagner et al\., 2020](https://arxiv.org/html/2610.18877#bib.bib11)\), and TimeMQA\([Kong et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib23)\), whose MCQ and T/F splits are evaluated separately\. Unless otherwise stated, we finetune our method and all baselines on those datasets jointly \(as mentioned in Section[3\.3](https://arxiv.org/html/2610.18877#S3.SS3)\)\. Each corpus receives a budget of40,00040\{,\}000examples so that none dominates the mixture\. Details on phase 2 datasets are in Appendix[F](https://arxiv.org/html/2610.18877#A6)\.

#### Training and Evaluation Protocols\.

For training, during Phase 1, the encoder, projector, decoder, and LLM LoRA adapters are trained jointly\. During Phase 2, the next\-patch decoder is removed, and the time\-series interface and LLM LoRA adapters are trained using the language objective alone\. Appendix[A\.1](https://arxiv.org/html/2610.18877#A1.SS1)provides the complete optimization and LoRA configurations\. For evaluation, every baseline is evaluated under greedy decoding: the model writes a rationale and an explicit answer, which is parsed and compared with the reference\. We report classification accuracy on HAR, PAMAP2, Sleep\-EDF, and TimeMQA, and generative accuracy under normalization on ECG\-QA\. Appendix[A\.2](https://arxiv.org/html/2610.18877#A1.SS2)details answer normalization and the held\-out benchmarks\.

Table 1:Accuracy on downstream tasks\.Avg\. is the mean over the six splits; \# Params counts trainable parameters in downstream adaptation\. Per backbone block,boldis best andunderlinesecond best per column;†\\daggermarks OpenTSLM’s original evaluation protocol\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8)\)\.

### 4\.2Baselines

We include a set of well\-known baselines with different model sizes, backbones and input modalities inReference, including fine\-tuned Gemma3\-4B\([Gemma Team, 2025](https://arxiv.org/html/2610.18877#bib.bib1)\)and fine\-tuned TS\-Reasoner\([Yu et al\., 2026b](https://arxiv.org/html/2610.18877#bib.bib24)\)\. We then match backbones and sizes acrossText\-tokenized fine\-tuning, using digit strings and LoRA\([Gruver et al\., 2023](https://arxiv.org/html/2610.18877#bib.bib7)\);OpenTSLM\-SP, using soft prompts; andOpenTSLM\-Flamingo, using gated cross\-attention\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8)\)\. Each uses Gemma3\-270M, Gemma3\-1B\([Gemma Team, 2025](https://arxiv.org/html/2610.18877#bib.bib1)\), and Qwen3\-0\.6B\([Yang et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib30)\), sharingLotus’s Phase 1 source corpora with method\-specific supervision and Phase 2 downstream datasets\. OpenTSLM rows marked withsequential†follow the original sequential fine\-tuning recipe\([Langer et al\., 2026](https://arxiv.org/html/2610.18877#bib.bib8)\)\.

### 4\.3Results

#### Main Results\.

Figure 3:TSRBench overall accuracy\. Per\-subset results are in Appendix’s Table[4](https://arxiv.org/html/2610.18877#A3.T4)\.We present the main results in Table[1](https://arxiv.org/html/2610.18877#S4.T1)\.Lotusachieves the best six\-task average among the compared methods at each backbone size, and performs best in every dataset except HAR at the smallest Gemma model and ECG\-QA on Qwen3\. We achieve these results with far fewer trainable parameters than other baselines: on Qwen3\-0\.6B we train a third of what OpenTSLM\-SP does and roughly a seventieth of what OpenTSLM\-Flamingo does\. Our smallest model,Lotuson Gemma\-270M comes within0\.140\.14% of a fully fine\-tuned TS\-Reasoner\-77B system \(55\.19%55\.19\\%vs\.55\.33%55\.33\\%\) whose backbone and trainable parameters are an order of magnitude larger\. Overall,Lotuson Qwen3\-0\.6B achieves the best average result of59\.9159\.91%,4\.584\.58% above TS\-Reasoner\-7B\-FT, while being14 times smaller in backbone size\.

#### Sequential adaptation substantially degrades earlier\-task performance\.

Thesequential†baselines in Table[1](https://arxiv.org/html/2610.18877#S4.T1)follow OpenTSLM’s original sequential fine\-tuning recipe and dataset order \(HAR→\\rightarrowSleep→\\rightarrowECG\), with PAMAP2 and TimeMQA appended\. Compared with balanced joint\-mixture training, sequential adaptation lowers average accuracy by17\.21%17\.21\\%–35\.14%35\.14\\%\. Performance declines mainly on datasets encountered earlier, including eight settings with0%0\\%accuracy\. In contrast, accuracy on TimeMQA’s MCQ and T/F splits, trained last, is largely preserved and sometimes matches the jointly trained counterparts\. This pattern suggests that adapting to later tasks can compromise performance on earlier ones\. We therefore use joint adaptation for the main comparison, exposing the shared encoder and projector to all datasets throughout training\.

#### Zero\-shot Results On TSRBench\.

We evaluate zero\-shot reasoning ability on TSRBench\([Yu et al\., 2026a](https://arxiv.org/html/2610.18877#bib.bib19)\), using our Gemma3\-270M checkpoint\. As Figure[3](https://arxiv.org/html/2610.18877#S4.F3)shows,Lotus\-270M scores highest among the thirteen baselines, ahead ofbackbones 25–50×\\timeslargerlike TS\-Reasoner\-7B and ChatTS\-14B\. The per\-subset results in Table[4](https://arxiv.org/html/2610.18877#A3.T4)\.Lotusachieves the highest overall TSRBench accuracy among the evaluated models, with strong performance in event prediction and comparative analysis\. From the results, we believeLotustransferable strength shows most in reasoning from signal evidence to a conclusion\. Detailed per\-task results are reported in Appendix[C](https://arxiv.org/html/2610.18877#A3)\.

## 5Analysis

### 5\.1Component ablation

\(a\)Component ablation\.\(b\)Frequency sensitivity\.
Figure 4:Component and input analyses on Gemma3\-270M\.\(a\)Six\-task average under ablations of pretraining and the frequency branch\.\(b\)Sleep\-EDF accuracy after removing individual frequency bands from the test signal\. In\-bar labels show percentage\-point drops\.Figure[4\(a\)](https://arxiv.org/html/2610.18877#S5.F4.sf1)comparesLotuswith variants that remove pretraining or frequency information\.No Phase 1fine\-tunes directly in Phase 2 with a randomly initialized encoder and projector, lowering average accuracy from55\.2%55\.2\\%to43\.5%43\.5\\%\.Lang\-only Phase 1uses TSQA and M4 at the same pretraining step budget and scores3\.4%3\.4\\%below fullLotuson average\. Removing the frequency branch in Phase 2 alone or both phases lowers performance by3\.8%3\.8\\%and4\.7%4\.7\\%relative to fullLotus, respectively, with the largest losses on ECG\-QA and PAMAP2\. In these variants, each time\-series patch is embedded using only the temporal projectionWt​𝐱c,nW\_\{t\}\\mathbf\{x\}\_\{c,n\}\.

### 5\.2Frequency Analysis

Figure[4\(b\)](https://arxiv.org/html/2610.18877#S5.F4.sf2)removes one frequency band at a time from the Sleep\-EDF test signal\. Suppressing delta \(0\.50\.5–44Hz\) lowers accuracy by16\.0%16\.0\\%,3\.0%3\.0\\%for theta \(44–88Hz\),1\.6%1\.6\\%for spindles \(1111–1616Hz\), and0\.3%0\.3\\%for alpha \(88–1212Hz\)\. The largest drop follows delta suppression, indicating that the trained model is particularly sensitive to this low\-frequency content in Sleep\-EDF\. Appendix[B](https://arxiv.org/html/2610.18877#A2)provides complementary evidence of signal dependence: shuffling patches, replacing the signal with noise, or zeroing it reduces HAR accuracy to26\.9%26\.9\\%,26\.5%26\.5\\%and32\.2%32\.2\\%, respectively\. These results show that signal information contributes to predictions beyond the accompanying text\. The frequency\-branch ablations examine how explicitly providing Fourier coordinates affects learning: removing the branch lowers six\-task average accuracy by3\.8%3\.8\\%in Phase 2 alone and4\.7%4\.7\\%in both phases\.

### 5\.3Representation Analysis

![Refer to caption](https://arxiv.org/html/2610.18877v1/representation.png)Figure 5:Representation analysis\.\(a\)Pairwise patch similarities\.Greenboxes highlight patterns shared byLotusand the raw signal;redboxes mark corresponding regions where these patterns are less apparent with cross\-entropy \(CE\)\-only training\.\(b\)Patch colors indicate cosine distance from the window\-average representation\. Boxes highlight spans selected for comparison\.We compare patch\-pair similarities in raw signals and learned representations over held\-out Sleep\-EDF windows of4747patches\. For patch vectorsuiu\_\{i\}anduju\_\{j\}, we compute cosine similarity assi​j=ui⊤​uj/\(‖ui‖2​‖uj‖2\)s\_\{ij\}=u\_\{i\}^\{\\top\}u\_\{j\}/\(\\\|u\_\{i\}\\\|\_\{2\}\\\|u\_\{j\}\\\|\_\{2\}\)and centre each panel on its off\-diagonal mean\.Figure[5](https://arxiv.org/html/2610.18877#S5.F5)auses patch indices on both axes, with blocks identifying groups of similar patches\.Lotus\(second panel\) reproduces prominent raw\-signal blocks, highlighted bygreenboxes, suggesting that patches with similar signal content retain similar representations\. Without the frequency branch, the third panel shows a more uniform pattern with less visible correspondence to these structures\. The cross\-entropy \(CE\)\-only representation retains varied patterns \(fourth panel\), but those marked byredboxes align less closely with the raw signal\.Figure[5](https://arxiv.org/html/2610.18877#S5.F5)bshows a Sleep\-EDF EEG signal and patch\-wise cosine distancesdi=1−ui⊤​u¯/\(‖ui‖2​‖u¯‖2\)d\_\{i\}=1\-u\_\{i\}^\{\\top\}\\bar\{u\}/\(\\\|u\_\{i\}\\\|\_\{2\}\\\|\\bar\{u\}\\\|\_\{2\}\)over time in seconds, whereu¯=1N​∑j=1Nuj\\bar\{u\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}u\_\{j\}is the window\-average representation\. Brighter colours indicate greater deviation from the window average; the raw\-signal row is separately rescaled\. In the highlighted spans \(red boxes\) for comparison,Lotusvaries alongside the signal, whereas the no\-frequency variant remains relatively flat and CE\-only changes align less closely\. Together, these findings suggest thatLotuspreserves both relationships between patches and local variation within the signal\.

## 6Conclusion

We introduce LOTUS, a time\-series language model built on*forecast grounding*: pretraining language\-aligned representations to retain information about future signal evolution\. LOTUS combines next\-patch prediction from LLM hidden states with a temporal–spectral patch representation\. Across three language\-model backbones and six downstream splits, LOTUS achieves the best average performance among matched baselines\. Ablations show gains from forecast grounding and frequency information, while representation analyses suggest that predictive supervision helps preserve temporal structure\. These findings support forecasting as a pretraining objective for time\-series language models, even when downstream tasks require only language outputs\.

### AI Use Statement

Large language models were not involved in the design of our approach\. We used them solely to polish the writing of this paper \(grammar and style\)\. All methods, experiments, and analyses were designed and run by the authors, who take responsibility for the content\.

### Ethics Statement

This work uses existing time\-series datasets and synthetic data\. Physiological and activity signals raise privacy concerns, and model predictions may reflect dataset biases or produce misleading explanations\. LOTUS is evaluated as a research system; its results do not establish suitability for clinical decision\-making or other high\-stakes applications\.

### Reproducibility Statement

We describe the architecture and training objectives in Section[3](https://arxiv.org/html/2610.18877#S3)and provide training configurations and evaluation procedures in Appendix[A](https://arxiv.org/html/2610.18877#A1)\. Appendices[E](https://arxiv.org/html/2610.18877#A5)and[F](https://arxiv.org/html/2610.18877#A6)document data construction, splits, and sampling, while Appendices[H](https://arxiv.org/html/2610.18877#A8)and[I](https://arxiv.org/html/2610.18877#A9)report computational resources and prompt templates to support reproduction of our experiments\.

## References

- Ansariet al\.\(2025\)A\. F\. Ansari, O\. Shchur, J\. Küken, A\. Auer, B\. Han, P\. Mercado, S\. S\. Rangapuram, H\. Shen, L\. Stella, X\. Zhang, M\. Goswami, S\. Kapoor, D\. C\. Maddix, P\. Guerron, T\. Hu, J\. Yin, N\. Erickson, P\. Mutalik Desai, H\. Wang, H\. Rangwala, G\. Karypis, Y\. Wang, and M\. Bohlke\-SchneiderChronos\-2: from univariate to universal forecasting\.External Links:2510\.15821Cited by:[§E\.2](https://arxiv.org/html/2610.18877#A5.SS2.SSS0.Px3.p1.1)\.
- Ansariet al\.\(2024\)A\. F\. Ansari, L\. Stella, C\. Turkmen, X\. Zhang, P\. Mercado, H\. Shen, O\. Shchur, S\. S\. Rangapuram, S\. P\. Arango, S\. Kapoor, J\. Zschiegner, D\. C\. Maddix, H\. Wang, M\. W\. Mahoney, K\. Torkkola, A\. G\. Wilson, M\. Bohlke\-Schneider, and Y\. WangChronos: learning the language of time series\.Transactions on Machine Learning Research\.External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=gerNCVqqtR)Cited by:[§E\.2](https://arxiv.org/html/2610.18877#A5.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.18877#S1.p1.1)\.
- Bianet al\.\(2024\)Y\. Bian, X\. Ju, J\. Li, Z\. Xu, D\. Cheng, and Q\. XuMulti\-patch prediction: adapting language models for time series representation learning\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),External Links:[Link](https://arxiv.org/abs/2402.04852)Cited by:[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p2.1)\.
- Chenget al\.\(2025\)M\. Cheng, Y\. Chen, Q\. Liu, Z\. Liu, Y\. Luo, and E\. ChenInstructime: advancing time series classification with multimodal language modeling\.InProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining,pp\. 792–800\.Cited by:[§1](https://arxiv.org/html/2610.18877#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p1.1)\.
- Daset al\.\(2024\)A\. Das, W\. Kong, R\. Sen, and Y\. ZhouA decoder\-only foundation model for time\-series forecasting\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),PMLR, Vol\.235,pp\. 10148–10167\.External Links:[Link](https://proceedings.mlr.press/v235/das24c.html)Cited by:[§1](https://arxiv.org/html/2610.18877#S1.p1.1)\.
- Donget al\.\(2023\)J\. Dong, H\. Wu, H\. Zhang, L\. Zhang, J\. Wang, and M\. LongSimMTM: a simple pre\-training framework for masked time\-series modeling\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://arxiv.org/abs/2302.00861)Cited by:[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p2.1)\.
- Gemma Team \(2025\)Gemma TeamGemma 3 technical report\.External Links:2503\.19786,[Link](https://arxiv.org/abs/2503.19786)Cited by:[§4\.2](https://arxiv.org/html/2610.18877#S4.SS2.p1.1)\.
- Godahewaet al\.\(2021\)R\. Godahewa, C\. Bergmeir, G\. I\. Webb, R\. J\. Hyndman, and P\. Montero\-MansoMonash time series forecasting archive\.InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks,Vol\.1\.Cited by:[§E\.1](https://arxiv.org/html/2610.18877#A5.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px1.p1.1)\.
- Gruveret al\.\(2023\)N\. Gruver, M\. Finzi, S\. Qiu, and A\. G\. WilsonLarge language models are zero\-shot time series forecasters\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.36,pp\. 19622–19635\.Cited by:[§4\.2](https://arxiv.org/html/2610.18877#S4.SS2.p1.1)\.
- Jinet al\.\(2024\)M\. Jin, S\. Wang, L\. Ma, Z\. Chu, J\. Y\. Zhang, X\. Shi, P\. Chen, Y\. Liang, Y\. Li, S\. Pan, and Q\. WenTime\-LLM: time series forecasting by reprogramming large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p2.1)\.
- Kempet al\.\(2000\)B\. Kemp, A\. H\. Zwinderman, B\. Tuk, H\. A\. C\. Kamphuisen, and J\. J\. L\. OberyeAnalysis of a sleep\-dependent neuronal feedback loop: the slow\-wave microcontinuity of the EEG\.IEEE Transactions on Biomedical Engineering47\(9\),pp\. 1185–1194\.Cited by:[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px2.p1.1)\.
- Konget al\.\(2025\)Y\. Kong, Y\. Yang, Y\. Hwang, W\. Du, S\. Zohren, Z\. Wang, M\. Jin, and Q\. WenTime\-MQA: time series multi\-task question answering with context enhancement\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 29736–29753\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1437),[Link](https://aclanthology.org/2025.acl-long.1437/)Cited by:[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px2.p1.1)\.
- Lanet al\.\(2025\)X\. Lan, F\. Wu, K\. He, Q\. Zhao, S\. Hong, and M\. FengGEM: empowering MLLM for grounded ECG understanding with time series and images\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=idtZwmjakN)Cited by:[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p1.1)\.
- Langeret al\.\(2026\)P\. Langer, T\. Kaar, M\. Rosenblattl, M\. A\. Xu, W\. Chow, M\. Maritsch, R\. Jakob, N\. Wang, J\. Liu, A\. Verma, B\. Han, D\. S\. Kim, H\. Chubb, S\. Ceresnak, A\. Zahedivash, A\. T\. S\. Sandhu, F\. Rodriguez, D\. McDuff, E\. Fleisch, O\. Aalami, F\. Barata, and P\. SchmiedmayerOpenTSLM: time\-series language models for reasoning over multivariate medical text\- and time\-series data\.InProceedings of the 43rd International Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2610.18877#S1.p1.1),[§1](https://arxiv.org/html/2610.18877#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2610.18877#S4.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.18877#S4.T1)\.
- Liet al\.\(2025\)Z\. Li, S\. Deldari, L\. Chen, H\. Xue, and F\. D\. SalimSensorLLM: aligning large language models with motion sensors for human activity recognition\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 354–379\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.19/)Cited by:[§1](https://arxiv.org/html/2610.18877#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p1.1)\.
- Loshchilov and Hutter \(2019\)I\. Loshchilov and F\. HutterDecoupled weight decay regularization\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[Table 2](https://arxiv.org/html/2610.18877#A1.T2.2.7.2.1.1)\.
- Makridakiset al\.\(2020\)S\. Makridakis, E\. Spiliotis, and V\. AssimakopoulosThe M4 competition: 100,000 time series and 61 forecasting methods\.International Journal of Forecasting36\(1\),pp\. 54–74\.Cited by:[§E\.1](https://arxiv.org/html/2610.18877#A5.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px1.p1.1)\.
- Merrillet al\.\(2024\)M\. A\. Merrill, M\. Tan, V\. Gupta, T\. Hartvigsen, and T\. AlthoffLanguage models still struggle to zero\-shot reason about time series\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Miami, Florida, USA,pp\. 3512–3533\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.201),[Link](https://aclanthology.org/2024.findings-emnlp.201/)Cited by:[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p2.1)\.
- Nieet al\.\(2023\)Y\. Nie, N\. H\. Nguyen, P\. Sinthong, and J\. KalagnanamA time series is worth 64 words: long\-term forecasting with transformers\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=Jbdc0vTOcol)Cited by:[§1](https://arxiv.org/html/2610.18877#S1.p1.1),[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p2.1)\.
- Ohet al\.\(2023\)J\. Oh, G\. Lee, S\. Bae, J\. Kwon, and E\. ChoiECG\-QA: a comprehensive question answering dataset combined with electrocardiogram\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 66277–66288\.Cited by:[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px2.p1.1)\.
- Reiss and Stricker \(2012\)A\. Reiss and D\. StrickerIntroducing a new benchmarked dataset for activity monitoring\.In2012 16th International Symposium on Wearable Computers \(ISWC\),pp\. 108–109\.Cited by:[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px2.p1.1)\.
- Sunet al\.\(2024\)C\. Sun, H\. Li, Y\. Li, and S\. HongTEST: text prototype aligned embedding to activate LLM’s ability for time series\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p2.1)\.
- Tagaet al\.\(2025\)E\. O\. Taga, M\. E\. Ildiz, and S\. OymakTimePFN: effective multivariate time series forecasting with synthetic data\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39\.Cited by:[§E\.2](https://arxiv.org/html/2610.18877#A5.SS2.SSS0.Px2.p1.1)\.
- Wagneret al\.\(2020\)P\. Wagner, N\. Strodthoff, R\. Bousseljot, D\. Kreiseler, F\. I\. Lunze, W\. Samek, and T\. SchaeffterPTB\-XL, a large publicly available electrocardiography dataset\.Scientific Data7\(1\),pp\. 154\.Cited by:[§4\.1](https://arxiv.org/html/2610.18877#S4.SS1.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025\)W\. Wang, K\. Wu, Y\. B\. Li, D\. Wang, and X\. ZhangSynthetic series\-symbol data generation for time series foundation models\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=xB1ZNgq0Xp)Cited by:[§E\.2](https://arxiv.org/html/2610.18877#A5.SS2.SSS0.Px4.p1.1)\.
- Wooet al\.\(2024\)G\. Woo, C\. Liu, A\. Kumar, C\. Xiong, S\. Savarese, and D\. SahooUnified training of universal time series forecasting transformers\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),PMLR, Vol\.235,pp\. 53140–53164\.External Links:[Link](https://proceedings.mlr.press/v235/woo24a.html)Cited by:[§1](https://arxiv.org/html/2610.18877#S1.p1.1)\.
- Xieet al\.\(2025\)Z\. Xie, Z\. Li, X\. He, L\. Xu, X\. Wen, T\. Zhang, J\. Chen, R\. Shi, and D\. PeiChatTS: aligning time series with LLMs via synthetic data for enhanced understanding and reasoning\.Proceedings of the VLDB Endowment18\(8\),pp\. 2385–2398\.Cited by:[§1](https://arxiv.org/html/2610.18877#S1.p1.1),[§1](https://arxiv.org/html/2610.18877#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.2](https://arxiv.org/html/2610.18877#S4.SS2.p1.1)\.
- Yuet al\.\(2026a\)F\. Yu, X\. Guo, L\. Yuan, H\. Kang, H\. Zhao, L\. Qin, F\. Huang, B\. Hu, and T\. ZhouTSRBench: a comprehensive multi\-task multi\-modal time series reasoning benchmark for generalist models\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=S9G3yu6vHl)Cited by:[Appendix C](https://arxiv.org/html/2610.18877#A3.SS0.SSS0.Px1.p1.1),[§4\.3](https://arxiv.org/html/2610.18877#S4.SS3.SSS0.Px3.p1.1)\.
- Yuet al\.\(2026b\)F\. Yu, H\. Zhao, and T\. ZhouTS\-Reasoner: aligning time series foundation models with LLM reasoning\.Transactions on Machine Learning Research\.External Links:[Link](https://arxiv.org/abs/2510.03519)Cited by:[§2\.1](https://arxiv.org/html/2610.18877#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.18877#S2.SS2.p1.1),[§4\.2](https://arxiv.org/html/2610.18877#S4.SS2.p1.1)\.

## Appendix Contents For Lotus: Learning to Understand Time\-series by Learning to Forecast

## Appendix AExperiment Details

### A\.1Training Configuration

Table[2](https://arxiv.org/html/2610.18877#A1.T2)summarizes the recipe shared across the reported runs\. Phase 1 uses source\-homogeneous batches, so each update applies either the language or prediction loss\. Phase 2 draws the same fixed example budget from each corpus and applies only the language loss\.

### A\.2Evaluation Details

All matched systems use greedy decoding\. Each response contains a rationale followed by an explicit answer field\. The parser extracts that field, applies case folding and whitespace normalization, and maps dataset\-specific textual variants to the task’s canonical label\. An output is counted as incorrect when no label can be recovered\. ECG\-QA uses the released answer normalization\. The headline metric is generative accuracy, followed by an unweighted mean across the four non\-overlapping datasets; it is not a token\-level language\-model metric\.

### A\.3Training Details

We provide the detailed configs for both phases in Table[2](https://arxiv.org/html/2610.18877#A1.T2)\.

Table 2:Configuration of the two phases\. Settings shared by both phases are listed first\.
### A\.4Full Training Data Ablation Results

In this section, we provide the detailed results for removing elements from our training data\. Table[3](https://arxiv.org/html/2610.18877#A1.T3)gives the per\-task results behind Figure[4\(a\)](https://arxiv.org/html/2610.18877#S5.F4.sf1)\. Language\-only pretraining lowers every signal task \(HAR−4\.0\-4\.0, Sleep−4\.1\-4\.1, ECG\-QA−2\.3\-2\.3, PAMAP2−6\.4\-6\.4\) while leaving MCQ unchanged\. Removing the frequency branch is felt most on ECG\-QA \(−7\.4\-7\.4in Phase 2 only,−7\.9\-7\.9in both phases\) and PAMAP2 \(−13\.7\-13\.7/−10\.7\-10\.7\); HAR is the one task that improves without it \(\+3\.2\+3\.2in the Phase\-2\-only variant\)\.

Table 3:Ablations on Gemma3\-270M, per task\.Each variant changes one ingredient and is otherwise trained and evaluated as in Table[1](https://arxiv.org/html/2610.18877#S4.T1)\.Boldis best per column\.

## Appendix BInput Diagnostics

We test whether predictions can be recovered from the text prompt or label prior alone\. Figure[6](https://arxiv.org/html/2610.18877#A2.F6)evaluates the Gemma3\-270MLotuscheckpoint of Table[1](https://arxiv.org/html/2610.18877#S4.T1)on the full test splits \(8,2228\{,\}222HAR and930930Sleep\-EDF items\) after shuffling the signal, replacing it with Gaussian noise, or setting it to zero\. HAR accuracy falls from58\.4%58\.4\\%to26\.526\.5–32\.2%32\.2\\%, bracketing its empirical majority\-class rate of28\.3%28\.3\\%; Sleep\-EDF falls from70\.5%70\.5\\%to45\.2%45\.2\\%under every corruption, its majority\-class rate being45\.3%45\.3\\%\. Thus, the model’s predictions depend materially on the numerical input\.

Figure 6:Input diagnostics\.Accuracy after corrupting the numerical input, on the Gemma3\-270M checkpoint and the full test splits \(HARn=8,222n\{=\}8\{,\}222, Sleep\-EDFn=930n\{=\}930\)\.
## Appendix CPer\-task TSRBench Results

Figure[3](https://arxiv.org/html/2610.18877#S4.F3)reports the aggregate; the full per\-subset breakdown behind it is given in Table[4](https://arxiv.org/html/2610.18877#A3.T4)\.

#### Evaluation protocol\.

We follow the benchmark’s evaluation\([Yu et al\., 2026a](https://arxiv.org/html/2610.18877#bib.bib19)\)and use its released scorer unmodified: it extracts the option letter from the generated text, drops responses from which no letter can be recovered, and micro\-averages accuracy over subsets\.

Table 4:Per\-task TSRBench accuracy \(%\) over 14 subsets\. Columns are grouped into perception, reasoning, prediction, and decision\-making\.

## Appendix DQualitative Comparisons

In this section, we provide qualitative comparisons betweenLotusand the baselines on Sleep\-EDF, HAR, ECG\-QA, and PAMAP2\. Figures[7](https://arxiv.org/html/2610.18877#A4.F7)–[10](https://arxiv.org/html/2610.18877#A4.F10)each show one test item with every system built on Gemma3\-270M\. Each figure shows the input series as the encoder receives it, the question \(instructions omitted for clarity; see Appendix[I](https://arxiv.org/html/2610.18877#A9)\), and each model’s rationale up to its first answer; any text generated after that answer is truncated\. Highlights were checked against measurements on the input, reported under each series: red marks a wrong answer or a claim the measurements contradict, and green marks the correct answer and the claim it rests on\. Claims the measurements support are left unmarked, including in wrong answers\.

The examples show two recurring failure modes\. Some baselines describe the signal correctly but answer a different question, as on ECG\-QA, where two baselines report a q wave or ST depression that is present in the recording instead of the queried T\-wave finding\. Others misread the signal itself, as when OpenTSLM\-SP describes the PAMAP2 window as stationary despite three large excursions\. In all four examples, OpenTSLM\-SP and the tokenized baseline also keep generating after their first answer, repeating tokens or restating the answer\.

Figure 7:Qualitative example, Sleep\-EDF\. All three baselines read the epoch as wakefulness;Lotusplaces it in a lighter sleep stage, consistent with its spectral centroid\.Figure 8:Qualitative example, HAR\.Lotusand OpenTSLM\-SP share their first 360 characters verbatim and differ only in the final label; OpenTSLM\-Flamingo describes a stepping motion that the Z\-axis spectrum does not show\.Figure 9:Qualitative example, ECG\-QA\. The q wave and ST depression cited by two baselines are present in the recording but are not the queried finding;Lotusidentifies the reduced T\-wave amplitude in lead I\.Figure 10:Qualitative example, PAMAP2\. OpenTSLM\-SP misses the three shaded excursions and answers sitting\. Per\-channel normalization removes the gravity reference, so ascent and descent cannot be distinguished from the input; the example illustrates the missed motion rather than the direction\.
## Appendix EPhase 1 Training Data

### E\.1Overview

Phase 1 draws source\-homogeneous batches from four corpora with fixed sampling weights1:2:1:11\{:\}2\{:\}1\{:\}1\(Section[4\.1](https://arxiv.org/html/2610.18877#S4.SS1)\)\. The two forecasting sources therefore account for60%60\\%of optimisation steps and the two language sources for the remaining40%40\\%\. These proportions are determined by the sampling weights rather than by corpus size\.

Two corpora provide next\-patch supervision\. The first contains half\-hourly electricity demand for five Australian states from the Monash archive\([Godahewa et al\., 2021](https://arxiv.org/html/2610.18877#bib.bib15)\)\. We split each series chronologically into the first80%80\\%for training, the next10%10\\%for validation, and the final10%10\\%for testing\. The second is the synthetic corpus described below\. Because each row is generated independently rather than extracted from a shared history, we randomly partition complete rows in the same80/10/1080/10/10ratio; the same generated series never appears in more than one split\. The two remaining corpora provide language supervision:38\.438\.4K multiple\-choice items concerning trend, volatility, seasonality, and outliers, and8080K M4 series\([Makridakis et al\., 2020](https://arxiv.org/html/2610.18877#bib.bib14)\)paired with generated captions\.

### E\.2Synthetic Multivariate Corpus

To obtain large\-scale multivariate supervision without training on the downstream evaluation series, we construct a synthetic corpus of844,275844\{,\}275series\. It combines four generator families \(Table[5](https://arxiv.org/html/2610.18877#A5.T5)\), none of which contributes more than37%37\\%of the data\. Two families impose cross\-channel structure explicitly, KernelSynth contributes univariate diversity, and SymTime contributes series generated from symbolic processes\.

Table 5:Composition of the synthetic training corpus\. Variate and length ranges are observed after collection and filtering\.#### KernelSynth\.

The univariate generator introduced by Chronos\([Ansari et al\., 2024](https://arxiv.org/html/2610.18877#bib.bib20)\)randomly composes a Gaussian\-process \(GP\) covariance\. For each series, it drawsj∼𝒰​\{1,…,5\}j\\sim\\mathcal\{U\}\\\{1,\\dots,5\\\}kernels with replacement from a bank of3333configurations\. The bank contains2121ExpSineSquaredconfigurations spanning several seasonal scales, together with parameterisedDotProduct,RBF,RationalQuadratic, andWhiteKernelentries and oneConstantKernel\. The sampled kernels are recursively combined using addition or multiplication with equal probability, and the series is drawn from the resulting GP prior on a fixed grid\. Additive compositions superpose components, whereas multiplicative compositions allow one component to modulate another\.

#### Linear Model of Coregionalization\.

Independent KernelSynth draws do not provide cross\-channel dependence\. The linear model of coregionalization \(LMC\) generator from TimePFN\([Taga et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib26)\)instead expresses each channel as a mixture of shared latent processes\. For a series withD∼𝒰​\{3,…,10\}D\\sim\\mathcal\{U\}\\\{3,\\dots,10\\\}channels, we drawwwfrom a Weibull distribution with shape22and setQQtoround⁡\(10​w\+1\)\\operatorname\{round\}\(10w\+1\), clipped to\[max⁡\(2,⌊D/20⌋\),D\]\[\\max\(2,\\lfloor D/20\\rfloor\),D\]\. Each of theQQlatents receives an independently composed KernelSynth covariance and is sampled from the corresponding GP prior\. The observed channels are convex mixturesZ=W​YZ=WY, with each row ofW∈ℝD×QW\\in\\mathbb\{R\}^\{D\\times Q\}drawn fromDir⁡\(α​𝟏Q\)\\operatorname\{Dir\}\(\\alpha\\mathbf\{1\}\_\{Q\}\)andα∼𝒰⁡\(0\.1,2\.0\)\\alpha\\sim\\mathcal\{U\}\(0\.1,2\.0\)\. Small values ofα\\alphaproduce sparse mixtures dominated by a few latents, whereas larger values distribute weight more evenly across the shared latents\. Samplingα\\alphaindependently for each series yields a range of cross\-channel dependence patterns\.

#### Chronos\-2 Multivariatizer\.

The largest family follows the synthetic\-data design described in the Chronos\-2 technical report\([Ansari et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib4)\); because its implementation is not released, we reimplement the procedure from the report\. Each latent univariate series is drawn uniformly from four generators: KernelSynth; a stationary AR\(pp\) process withp∼𝒰​\{1,…,5\}p\\sim\\mathcal\{U\}\\\{1,\\dots,5\\\}, constructed by mapping partial autocorrelations in\(−0\.95,0\.95\)\(\-0\.95,0\.95\)to AR coefficients through the Levinson recursion; a trend–seasonality–irregularity model combining a random polynomial trend, one to three sinusoids, and Gaussian noise; or an additive Holt–Winters state\-space simulator with random smoothing constants and seasonal period\. After standardising the latent series, we combineKKof them using one of two multivariatizers, selected with equal probability\. The*cotemporaneous*multivariatizer computesZ=f⁡\(A​Y\)Z=f\(AY\), whereA∈ℝD×KA\\in\\mathbb\{R\}^\{D\\times K\}has i\.i\.d\.𝒩⁡\(0,1/K\)\\mathcal\{N\}\(0,1/K\)entries\. With probability0\.50\.5,ffis the identity; otherwise it is sampled uniformly from\{tanh,sin,sgn⁡\(x\)​\|x\|\}\\\{\\tanh,\\sin,\\operatorname\{sgn\}\(x\)\\sqrt\{\|x\|\}\\\}\. This branch produces either linear or nonlinear instantaneous dependence within a series\. The*sequential*multivariatizer instead computesZ⁡\[d,t\]=∑kad,k​Yk​\[t−τd,k\]Z\[d,t\]=\\sum\_\{k\}a\_\{d,k\}Y\_\{k\}\[t\-\\tau\_\{d,k\}\], with independently sampled lagsτd,k∼𝒰⁡\{0,…,32\}\\tau\_\{d,k\}\\sim\\mathcal\{U\}\\\{0,\\dots,32\\\}and zero padding at the leading boundary, thereby introducing lead–lag relationships\. Mixing shared non\-stationary latents in the cotemporaneous branch can also produce cointegrated channels; we do not use a separate cointegration generator\. Finally, with probability0\.30\.3, between one and⌊D/2⌋\\lfloor D/2\\rfloorchannels are designated as known covariates, yielding a covariate\-informed forecasting item\.

#### SymTime\.

SymTime\([Wang et al\., 2025](https://arxiv.org/html/2610.18877#bib.bib25)\)provides a qualitatively different source of variation through its series–symbol generator, which samples a symbolic expression and then realises the corresponding series\. Although the generator emits matched expression–series pairs, we retain only the series because Phase 1 applies next\-patch rather than symbolic\-recovery supervision\.

#### Unification\.

The four generators emit different container formats\. During collection, we cast each item to afloat32array with shape\(variates, length\)and stream it to a unified Arrow shard\. We reject items containing non\-finite values; this filter removes1,6681\{,\}668SymTime items and no items from the other families\. The retained corpus occupies3030GB in1616Arrow files\. At training time, all files share one corpus label so that the sampler treats them as a single mixture component\. Per\-item seeds make the generated corpora reproducible under the same software environment\.

### E\.3Dataset Statistics

Table[6](https://arxiv.org/html/2610.18877#A5.T6)summarises the corpus with five scale\-invariant descriptors, alongside the real forecasting corpus used in the same phase\. Two things are visible\. First, the families occupy different regions rather than being four parameterisations of one process: KernelSynth has the lowest spectral entropy, its composed kernels concentrating spectral power and leaving the series comparatively predictable, whereas SymTime has the highest together with by far the weakest periodicity, its symbolic realisations lying closer to noise\. Second, the two multivariatizers impose different amounts of cross\-channel dependence, the coregionalization mixture producing the more strongly coupled channels, which is what those mechanisms are for\.

The real corpus falls inside the synthetic range on spectral entropy and trend, but outside it on both dependence measures: its half\-hourly demand is more persistent than any synthetic family, and its five states are more tightly coupled than any generator produces\. The synthetic corpus is, on the other hand, much more varied, its per\-descriptor spread being an order of magnitude wider, since the real reference is a single system observed repeatedly whereas each synthetic draw resamples its own generating process\. Pretraining on this mixture therefore buys breadth rather than fidelity to any one real corpus\.

Table 6:Phase 1 dataset statistics, by generator family\. Mean±\\pmstandard deviation over sampled series; real series are cut into2,0482\{,\}048\-point windows\. Spectral entropy is spectral flatness \(00a pure tone,11white noise\); seasonal peak the largest\|ACF\|\|\\text\{ACF\}\|over lags\[2,L/4\]\[2,L/4\]; trendR2R^\{2\}the variance share of a least\-squares line; cross\-channel the mean\|\|correlation\|\|over variate pairs, undefined when univariate\.

## Appendix FPhase\-2 Training Data

Phase 2 uses the five chain\-of\-thought corpora in Table[7](https://arxiv.org/html/2610.18877#A6.T7), each rendered as question–rationale–answer triples: HAR \(three\-axis accelerometry\), Sleep\-EDF \(single\-channel EEG\), ECG\-QA \(twelve\-lead ECG\), PAMAP2 \(household and exercise activities from wearable accelerometry\), and TimeMQA \(open\-ended questions over univariate series\)\. TimeMQA is one training corpus but two evaluation columns, separated by itsquestion\_formatfield, which is why five corpora yield the six columns of Table[1](https://arxiv.org/html/2610.18877#S4.T1)\.

Table 7:Phase\-2 corpora\.Natural split sizes and the balanced per\-epoch count; multiplicity is how often an example is seen per epoch, with corpora larger than the target truncated to a fixed subset\.### F\.1Balanced Mixture\.

Each corpus is resized to40,00040\{,\}000examples per epoch \(200,000200\{,\}000in total\): smaller corpora are repeated in full with a seeded shuffled remainder, larger ones are shuffled once and truncated to a subset that is reused in every epoch\. Under the natural sizes, ECG\-QA and HAR would supply83%83\\%of the data and PAMAP21\.5%1\.5\\%; after balancing, each PAMAP2 example is seen nine to ten times per epoch while three quarters of ECG\-QA is never used\.

### F\.2Evaluation Subsets\.

Sleep\-EDF and PAMAP2 have validation and test splits of equal size \(930930and234234\)\. For ECG\-QA we score the first5,0005\{,\}000test items, pinned by a manifest keyed on a hash of the gold answer so that every system is evaluated on the same subset\.

## Appendix GComputational Cost

How much of the backbone’s context a series occupies decides both what it costs to read that series at inference and what it costs to train on it\. Everything in this section is measured on one GPU and one backbone \(Gemma3\-270m\) so that the baselines can be compared directly; the resources actually consumed by our training runs are reported in Appendix[H](https://arxiv.org/html/2610.18877#A8)\.

### G\.1Reading A Series

Figure[11](https://arxiv.org/html/2610.18877#A7.F11)measures what follows from this on one backbone\.Lotusis the cheapest interface at every length we tested and handles100100k samples in2\.22\.2GB\. OpenTSLM\-SP stops at4,0964\{,\}096samples, because its encoder uses a learned positional table of fixed length: longer inputs are not expensive, they are unrepresentable\. OpenTSLM\-Flamingo holds its token count nearly constant through a perceiver, but carries a fixed overhead of about2\.22\.2GB, four times ours at short lengths\.

Figure 11:Inference scaling with series length, all baselines on Gemma3\-270m \(single series, one1010\-token generation per point; peak allocated memory\)\. Left:Lotusis cheapest at every length; OpenTSLM\-SP’s line terminates at its architectural length ceiling\. Right: context positions consumed,L/32L/32againstL/4L/4\.
### G\.2Training on A Series

Token count also sets the cost of a training step\. Every number in Table[8](https://arxiv.org/html/2610.18877#A7.T8)is measured at batch size11, the Phase\-2 batch, with each interface built at the geometry of the checkpoint we report and optimising its own trainable set; a step is one forward, backward and optimiser pass, averaged over1212steps after warmup\.

Table 8:Phase\-2 cost per example on one GPU\(Gemma3\-270m, batch11\)\.*tok*is the context the series occupies, constant for Flamingo, whose perceiver compresses to6464latents;*s/ex*is one training step and*GB*peak training memory\.What Table[8](https://arxiv.org/html/2610.18877#A7.T8)shows is a scaling property\. Going from one channel of1,5001\{,\}500samples to twelve channels of1,0001\{,\}000multiplies our step time by1\.051\.05and our peak memory by1\.031\.03\. The same change costs OpenTSLM\-SP4\.4×4\.4\\timesin time and6\.9×6\.9\\timesin memory, because a patch length of four turns twelve leads into3,0003\{,\}000soft tokens where we use384384\. On the twelve\-lead column we therefore train3\.1×3\.1\\timesfaster than OpenTSLM\-SP in1/261/26of its memory, and13\.6×13\.6\\timesfaster than the serialized interface in1/371/37of its memory\. OpenTSLM\-SP is faster on the short single\-channel column \(0\.190\.19s against0\.260\.26s\), and that ordering reverses as soon as the input is multichannel, which is the regime the interface exists for\. OpenTSLM\-Flamingo is input\-invariant too, since its perceiver fixes the token count, but it pays14×14\\timesour training memory and two orders of magnitude more trainable parameters\. The serialized baseline looks invariant only because it throws data away: its harness decimates to1,2001\{,\}200values across all channels, so the twelve\-lead sample reaches the model with90%90\\%of its samples gone and still costs23\.823\.8GB to train on\.

Generation latency separates the baselines far less, and we report it so the efficiency claim is not over\-read\. At a200200\-token budget every interface falls between8\.28\.2and11\.111\.1seconds per sample, and the spread does not change between the two input geometries: decoding is dominated by200200sequential steps, so a7,2397\{,\}239\-token prompt and a4747\-token prompt finish within15%15\\%of each other\. The interface decides memory and reach at inference, not latency\.

## Appendix HComputational Resources

Table[9](https://arxiv.org/html/2610.18877#A8.T9)reports what the Phase 1 predictive\-grounding runs consumed\.

Table 9:Phase 1 training runs\. Wall\-clock time, allocation, and GPU\-hours are from scheduler records\. An epoch is one pass through the weighted sample drawn from the four corpora, not through every available example\. Peak VRAM is the per\-device maximum from a single\-process V100 run at the listed per\-device batch size\.BackboneEncoderGPUsBatchPeakWall clockGPU\-hEpochs/deviceeff\.VRAMGemma3\-270MTransformer Decoder2×42\\times 4161612812831\.331\.3GB2626h2102100\.70\.7Qwen3\-0\.6BTransformer Decoder2×42\\times 48864649\.19\.1GB5\.05\.0d9609602\.12\.1Gemma3\-1BTransformer Decoder4×44\\times 48812812815\.215\.2GB10\.010\.0d3,8403\{,\}8406\.26\.2Total5,0105\{,\}010### H\.1Hardware and Precision

All runs used NVIDIA V100 accelerators in four\-GPU nodes, with data parallelism only, so each device held a complete model replica\. V100s have no native bfloat16 arithmetic, so we used fp16 automatic mixed precision with gradient scaling\.

### H\.2Training Time

Each Phase 1 run is trained for at most ten days under the same early\-stopping protocol\. Validation loss is evaluated at regular intervals, and training stops when it does not improve for five consecutive validation checks\. We retain the checkpoint with the lowest validation loss rather than the final checkpoint\. Table[9](https://arxiv.org/html/2610.18877#A8.T9)reports the actual wall\-clock time and number of optimizer updates\.

### H\.3Vocabulary\-dependent Memory

Peak VRAM is not monotonic in backbone size: Gemma3\-270M uses31\.331\.3GB where Gemma3\-1B uses15\.215\.2GB\. The cross\-entropy logit tensor explains almost all of it, since its shape isbatch×length×\|𝒱\|\\text\{batch\}\\times\\text\{length\}\\times\|\\mathcal\{V\}\|\. The two Gemma backbones share a262,144262\{,\}144\-token vocabulary but run at per\-device batches of1616and88, so the smaller one materialises twice the logits at a given sequence length, while Qwen3\-0\.6B pairs batch88with a smaller vocabulary of151,936151\{,\}936\. Taking Qwen3\-0\.6B as the reference,batch×\|𝒱\|\\text\{batch\}\\times\|\\mathcal\{V\}\|predicts1:1\.73:3\.451\{:\}1\.73\{:\}3\.45for Qwen3\-0\.6B, Gemma3\-1B, and Gemma3\-270M against measured ratios of1:1\.67:3\.441\{:\}1\.67\{:\}3\.44\. Weights and activations contribute as well, but vocabulary size and per\-device batch dominate\. These single\-process measurements omit the gradient buckets that data parallelism introduces, and the Gemma3\-270M measurement enables the two\-dimensional position\-ID path that the original run predates, whose position tensor is negligible beside the logits\.

### H\.4Configurations Across Backbones

Gemma3\-270M uses the time series encoder with LoRA adapters, training3\.173\.17M parameters\. Gemma3\-1B and Qwen3\-0\.6B use the transformer encoder with LoRA adapters, training14\.7314\.73M and12\.6512\.65M, of which6\.526\.52M and5\.055\.05M are adapter weight, and halve the per\-device batch to88\. Gemma3\-1B uses twice the replicas of Qwen3\-0\.6B, which restores its effective batch to128128\. These totals include the next\-patch decoder, which Phase 2 discards; the Params column of Table[1](https://arxiv.org/html/2610.18877#S4.T1)reports the Phase\-2 trainable set and is correspondingly smaller\.

## Appendix IPrompt Templates

### I\.1Phase 1

In phase 1, we use the caption on the channel block \(Figure[12](https://arxiv.org/html/2610.18877#A9.F12)\), giving the modality, the number of channels and time steps, and the mean and standard deviation of the raw series\. Training uses the next\-patch regression of Eq\.[10](https://arxiv.org/html/2610.18877#S3.E10), so no token in this prompt is a target\. A forecasting batch therefore produces no cross\-entropy term, which is why each Phase 1 batch is drawn from a single source\.

Forecasting \(electricity demand and the synthetic corpus\)pre\-prompt
\(empty\)
channel block
This is a univariate time series with 1 channel and 1024 time steps\. It has mean 6084\.2410 and std 1174\.1908\.<ts\> … </ts\>
post\-prompt
<eos\>Figure 12:Phase\-1 forecasting prompt, shown for a1,0241\{,\}024\-step window of electricity demand\. A multivariate item from the synthetic corpus takes the same form withmultivariate, its channel count, and one block per channel\. The answer field is the end\-of\-sequence token; the training signal is the next\-patch regression rather than any token in this prompt\.The two language corpora in Phase 1 follow the three\-part schema of §[I\.2](https://arxiv.org/html/2610.18877#A9.SS2)\. Time\-series QA places the item’s question in the pre\-prompt andPredict the \{task\} Answer:in the post\-prompt, with the answer letter as the target\. M4 captioning usesYou are an expert in time series analysis\.followed byPlease generate a detailed caption for this time\-series, describing it as accurately as possible\., with the reference caption as the target\.

### I\.2Phase 2

Every corpus uses the same three\-part schema: a*pre\-prompt*stating the task and instructions, one captioned block per channel, and a*post\-prompt*fixing the answer format\. InLotusand the soft\-prompt baselines each block holds the encoder’s soft tokens, delimited by<ts\>…</ts\>; in the serialized baseline the same position holds the decimated values as text\. Each caption names the channel and gives its mean and standard deviation, so normalisation statistics reach every interface as text\. Figures[13](https://arxiv.org/html/2610.18877#A9.F13)–[17](https://arxiv.org/html/2610.18877#A9.F17)reproduce the templates verbatim; ellipses mark truncation only\.

HARpre\-prompt
You are given accelerometer data in all three dimensions\. Your task is to classify the activity based on analysis of the data\.
Instructions:
\- Begin by analyzing the time series without assuming a specific label\.
\- Think step\-by\-step about what the observed patterns suggest regarding movement intensity and behavior\.
\- Write your rationale as a single, natural paragraph \-\-\- do not use bullet points, numbered steps, or section headings\.
\- Do \*\*not\*\* mention any class label until the final sentence\.
Possible activity labels are:
biking, lying, running, sitting, standing, walking, walking\_down, walking\_up\.
\- Make sure that your last word is the answer\. You MUST end your response with "Answer: "
channel blocks \(one per axis\)
The following is the accelerometer data on the x\-axis, it has mean \-3\.2434 and std 0\.0474:<ts\> … </ts\>
The following is the accelerometer data on the y\-axis, it has mean 2\.3132 and std 0\.0550:<ts\> … </ts\>
The following is the accelerometer data on the z\-axis, it has mean 9\.2017 and std 0\.0639:<ts\> … </ts\>
post\-prompt
Rationale:Figure 13:Prompt template for HAR\.PAMAP2pre\-prompt
You are given accelerometer data in all three dimensions\. Your task is to classify the activity based on analysis of the data\.
Instructions:
\- Begin by analyzing the time series without assuming a specific label\.
\- Think step\-by\-step about what the observed patterns suggest regarding movement intensity and behavior\.
\- Write your rationale as a single, natural paragraph \-\-\- do not use bullet points, numbered steps, or section headings\.
\- Do \*\*not\*\* mention any class label until the final sentence\.
Possible activity labels are:
lying, sitting, standing, walking, running, cycling, nordic walking, watching TV, computer work, car driving, ascending stairs, descending stairs, vacuum cleaning, ironing, folding laundry, house cleaning, playing soccer, rope jumping\.
\- Make sure that your last word is the answer\. You MUST end your response with "Answer: "
channel blocks \(one per axis\)
The following is the accelerometer data on the x\-axis, it has mean \-3\.2434 and std 0\.0474:<ts\> … </ts\>
The following is the accelerometer data on the y\-axis, it has mean 2\.3132 and std 0\.0550:<ts\> … </ts\>
The following is the accelerometer data on the z\-axis, it has mean 9\.2017 and std 0\.0639:<ts\> … </ts\>
post\-prompt
Rationale:Figure 14:Prompt template for PAMAP2\. The template matches HAR except for the eighteen\-way label set\.Sleep\-EDFpre\-prompt
You are given a 30\-second EEG time series segment\. Your task is to classify the sleep stage based on analysis of the data\.
Instructions:
\- Analyze the data objectively without presuming a particular label\.
\- Reason carefully and methodically about what the signal patterns suggest regarding sleep stage\.
\- Write your reasoning as a single, coherent paragraph\. Do not use bullet points, lists, or section headers\.
\- Only reveal the correct class at the very end\.
\- Never state that you are uncertain or unable to classify the data\. You must always provide a rationale and a final answer\.
channel block
This is the EEG signal, it has mean 0\.0021 and std 0\.9873:<ts\> … </ts\>
post\-prompt
Possible sleep stages are:
Wake, Non\-REM stage 1, Non\-REM stage 2, Non\-REM stage 3, Non\-REM stage 4, REM sleep
\- Please now write your rationale\. Make sure that your last word is the answer\. You MUST end your response with "Answer: "Figure 15:Prompt template for Sleep\-EDF\. Unlike the accelerometer corpora, the class list is placed in the post\-prompt\.ECG\-QApre\-prompt
You are an expert cardiologist analyzing an ECG \(electrocardiogram\)\.
Clinical Context: 82\-year\-old male patient\. 12\-lead ECG\. clinical recording\. recorded with AT\-6 C 5\.8\. Signal quality: baseline drift noted, static noise present, burst noise present, electrode artifacts present\. extra beats detected during recording\. pacemaker present\.
Your task is to examine the ECG signal and answer the following medical question:
Question: Does this ECG show burst noise?
Instructions:
\- Begin by analyzing the time series without assuming a specific answer\.
\- Think step\-by\-step about what the observed patterns suggest regarding the cardiac condition\.
\- Write your rationale as a single, natural paragraph \-\-\- do not use bullet points, numbered steps, or section headings\.
\- Do \*\*not\*\* mention any final answer until the very end\.
\- Consider the ECG morphology, intervals, and any abnormalities that relate to the question\.
channel blocks \(one per lead, twelve in total\)
This is ECG Lead I, it has mean \-0\.0063 and std 0\.1198:<ts\> … </ts\>
… \(Leads II, III, aVR, aVL, aVF, V1\-\-V6\)
post\-prompt
Based on your analysis of the ECG data, select your answer from the following options:
no, yes
\- Make sure that your last word is the answer\. You MUST end your response with "Answer: "Figure 16:Prompt template for ECG\-QA\. The clinical\-context paragraph is assembled from the record’s metadata; the answer set in the post\-prompt is that of the question template\.TimeMQApre\-prompt
You are given a time series from the Healthcare domain\. Answer the following question\.
Which data segment shows more volatility? A\) <the provided time series\> or B\) \[0\.71, 0\.72, 0\.73, 0\.73, 0\.73, 0\.73, 0\.72, 0\.72, 0\.71, 0\.7, 0\.68, 0\.67\]
channel block
This is the time series, mean 0\.5300 std 0\.1352\.<ts\> … </ts\>
post\-prompt
Answer:Figure 17:Prompt template for TimeMQA\. The corpus is open\-ended, so the post\-prompt is a bareAnswer:\. In comparison questions the first segment enters through the time\-series interface while the second is inlined as digits, so the model compares a soft\-token series against a textual one\.
### I\.3TSRBench Evaluation

TSRBench is evaluated zero\-shot, so the prompts are the benchmark’s own rather than ours: no template is tuned forLotusand every system sees the same text\. All fourteen subsets are multiple choice and share one answer instruction, which fixes the output format and separates the rationale from the graded answer; only the way the signal is introduced differs between the three templates below\. Series enter at the<ts\> … </ts\>placeholders exactly as in Phase 2, so the interface is unchanged from training\. Parsing takes the single letter inside<answer\>; an item with no recoverable letter is scored incorrect\.

Standard multiple choice \(twelve of the fourteen subsets\)question and channel blocks
\{question\} Here are the time series ’\{name1\}’:<ts\> … </ts\>\. ’\{name2\}’:<ts\> … </ts\>\. …
answer instruction \(shared by all subsets\)
Select from the options below:
A\. \{choice A\}
B\. \{choice B\}
…
Output your reasoning process in <think\> tags and final answer as a single letter in <answer\> tags\. Format:
<think\>Your reasoning here \(less than 2048 tokens\)</think\>
<answer\>A</answer\>Figure 18:Standard TSRBench template\. Options are labelledA–Gin the order the benchmark stores them\.Temporal relation reasoningas above, but with the index channel removed
\{question\} Here are the time series ’\{name1\}’:<ts\> … </ts\>\. …
Any channel named “Time Stamps” is dropped before the placeholders are emitted, so only measured channels are encoded; the answer instruction is unchanged\.Figure 19:Temporal relation reasoning\. The subset ships an explicit timestamp channel, which would otherwise be encoded as though it were a signal\.Abductive reasoningpre\-prompt
You are an expert in basketball game analysis\. Your task is to perform abductive reasoning\.
Given a sequence of past events, future events, and corresponding time series data from a game, determine the most plausible event that occurred in between to link them\.
context
\-\-\- CONTEXT \-\-\-
Past Events \(History\):
\- \{time\}: \{event\} …
…\[A CRITICAL EVENT HAPPENED HERE\] …
Future Events:
\- \{time\}: \{event\} …
Time Series Data \(Win Probability\):
Time \| Team A Win Prob \| Team B Win Prob
…
task, channel blocks, then the shared answer instruction
\-\-\- TASK \-\-\-
\{question\} Here are the time series ’Team A Win Probability’:<ts\> … </ts\>\. ’Team B Win Probability’:<ts\> … </ts\>\.Figure 20:Abductive reasoning\. A window of ten steps either side of the critical moment is kept\. The win\-probability channels are given twice, once serialised in the table and once through the encoder, which is the benchmark’s own design and the one setting where our interface receives the signal redundantly\.