ConceptTS: LLM-Guided Concept Bottlenecks for Interpretable Multivariate Time-Series Forecasting

# ConceptTS: LLM-Guided Concept Bottlenecks for Interpretable Multivariate Time-Series Forecasting
Source: [https://arxiv.org/html/2608.21277](https://arxiv.org/html/2608.21277)
1stYichen Jiang1Thanks:1Yichen Jiang joined this VIA Lab project at UC Davis while affiliated with Stanford University and later completed his work as a remote research intern under Dongyu Liu’s supervision\.Affiliation:Department of Electrical Engineering Stanford University Stanford, CA, USA ycjiang@stanford\.edu2ndYueqiao ChenAffiliation:Department of Computer Science University of California, Davis Davis, CA, USA yeqchen@ucdavis\.edu3rdDongyu LiuAffiliation:Department of Computer Science University of California, Davis Davis, CA, USA dyuliu@ucdavis\.edu

###### Abstract

State\-of\-the\-art multivariate time\-series forecasters can model complex temporal and cross\-variable dependencies, yet their opaque representations provide limited insight into why a particular forecast is produced\. This lack of transparency restricts their use in settings where practitioners must understand and assess the factors underlying a prediction\. We introduce ConceptTS, an interpretable forecasting framework that organizes its predictions around named, human\-readable concepts\. ConceptTS uses a large language model to propose task\-relevant concepts and generate executable labeling rules, translating the language model’s domain knowledge into direct supervision without costly manual concept annotation\. The proposed concepts are organized into three complementary bottlenecks that describe the historical context, local forecast intervals, and the full forecast horizon\. A shared decoder combines representations derived from their predicted activations to construct the forecast, making the model’s decision process explicit and supporting direct concept\-level interventions\. Experiments on the Beijing Multi\-Site Air Quality dataset show that ConceptTS achieves accuracy competitive with strong black\-box baselines while producing semantically meaningful concept activations\.

###### Index Terms:

concept bottleneck models, explainable artificial intelligence, interpretable machine learning, large language models, multivariate time series forecasting

## IIntroduction

Multivariate time series forecasting predicts a target trajectory from historical observations and correlated covariates\. Across healthcare, environmental monitoring, energy, and industry, such continuously collected data demand both scalable modeling and interpretable temporal analysis\[[11](https://arxiv.org/html/2608.21277#bib.bib7),[7](https://arxiv.org/html/2608.21277#bib.bib6)\]\. Modern recurrent, convolutional, graph, linear/MLP, and Transformer architectures can capture nonlinear temporal dynamics and cross\-variable dependencies with strong accuracy\[[21](https://arxiv.org/html/2608.21277#bib.bib4)\]\. Yet their predictive progress has outpaced interpretability: a forecast does not reveal which temporal conditions or variable interactions shaped it, or whether the model used plausible evidence\. This opacity is consequential in high\-stakes, data\-intensive settings, where practitioners must understand potential failure modes before trusting a prediction\.

Interpreting multivariate time series is particularly challenging because predictive evidence is distributed across variables and temporal scales, and its relevance can change throughout the forecast horizon\[[20](https://arxiv.org/html/2608.21277#bib.bib1),[26](https://arxiv.org/html/2608.21277#bib.bib2),[16](https://arxiv.org/html/2608.21277#bib.bib3)\]\. A forecast may depend jointly on a long\-term trend in one channel, a short\-lived event in another, and their evolving interaction\. Attribution or saliency methods can identify influential channels and timestamps, but these low\-level importance scores do not necessarily explain the semantic condition represented by the selected evidence\. Concept\-based models offer a promising alternative by organizing predictions around human\-understandable intermediate variables\. Concept bottleneck models \(CBMs\) make these variables inspectable and directly intervenable\[[9](https://arxiv.org/html/2608.21277#bib.bib12)\], while recent time\-series methods provide complementary directions: ProtoTS introduces hierarchical prototype reasoning for forecasting\[[17](https://arxiv.org/html/2608.21277#bib.bib11)\], and TimeX\+\+ learns label\-preserving explanatory instances through an information bottleneck\[[13](https://arxiv.org/html/2608.21277#bib.bib5)\]\. Nevertheless, conventional CBMs depend on expert\-defined concepts and per\-example annotations; ProtoTS learns latent prototypes whose meanings must be inferred after training; and TimeX\+\+ explains an existing predictor rather than requiring forecasts to pass through named semantic evidence\. Thus, it remains difficult to obtain an accurate forecaster whose internal evidence is semantically explicit, supervised, and tied to the prediction pathway without incurring substantial annotation cost\.

We address this gap withConceptTS, an LLM\-guided concept\-bottleneck framework for interpretable multivariate time\-series forecasting\. Given domain context, dataset statistics, and representative temporal regimes, an offline large language model proposes concept names, natural\-language definitions, and executable predicates that automatically label training segments\. ConceptTS organizes these concepts into complementary bottlenecks describing the historical context, local intervals within the forecast horizon, and the horizon as a whole\. A structure\-preserving encoder predicts their activations, and a shared decoder constructs the forecast from the resulting concept representations\. Input\-independent concept embeddings limit information leakage around the displayed activation pathway, while a small regularized residual channel accommodates continuous magnitude information not fully represented by discrete concepts and exposes an explicit accuracy–interpretability trade\-off\. The LLM is used only during concept construction and is not required at inference time\. We evaluate ConceptTS on three sites from the Beijing Multi\-Site Air Quality dataset under both future\-aware and look\-back\-only settings\. The model remains competitive with strong black\-box forecasters, including an MAE within5%5\\%of the best tested look\-back\-only baseline\. Beyond predictive accuracy, concept\-count and residual\-channel ablations, qualitative case studies, counterfactual concept sweeps, and activation interventions examine how the concepts are learned and used\. In particular, replacing the activations with inverted ground\-truth concept labels increases test MAE from15\.37115\.371to99\.66199\.661, providing direct evidence that the decoder relies on the concept\-activation channel in the tested configuration\.

Our contributions are threefold:

- •LLM\-guided concept construction and supervision\.We introduce a pipeline that converts domain context and dataset summaries into named temporal concepts, natural\-language definitions, and executable labeling rules, avoiding manual concept annotation for individualtraining segments\.
- •A multiscale, intervention\-ready forecasting architecture\.We organize concepts across historical, local\-forecast, and global\-forecast scopes and design a prediction pathway that preserves temporal and variable structure while limiting information leakage around the concept bottlenecks\.
- •Evaluation of both forecasting and interpretability\.Experiments across three monitoring sites and two input regimes establish competitive forecasting performance, while ablations, case studies, concept sweeps, and controlled interventions evaluate concept quality, model reliance, and the accuracy–interpretability trade\-off\.

## IIRelated Work

### II\-AExplainability for Time\-Series Models

Explaining time\-series predictions requires more than assigning importance to independent features\. Predictive evidence may lie in an event, its duration, a trend over a window, or an interaction among channels; moreover, the relevant evidence may change across the forecast horizon\. Surveys therefore distinguish post\-hoc attribution, perturbation, and example\-based explanations from intrinsically interpretable architectures, and assess explanations along dimensions such as local versus global scope, faithfulness, stability, and human comprehensibility\[[20](https://arxiv.org/html/2608.21277#bib.bib1),[26](https://arxiv.org/html/2608.21277#bib.bib2),[16](https://arxiv.org/html/2608.21277#bib.bib3)\]\. These requirements are particularly demanding for multivariate forecasting: correlated variables make individual attribution ambiguous, temporal aggregation can hide short\-lived events, and masking may create out\-of\-distribution inputs\[[13](https://arxiv.org/html/2608.21277#bib.bib5)\]\.

Several forecasting methods expose which parts of an input appear influential\. Series Saliency jointly learns a forecast and saliency over multivariate input regions, while the Temporal Fusion Transformer provides variable\-selection weights and temporal attention patterns\[[15](https://arxiv.org/html/2608.21277#bib.bib16),[10](https://arxiv.org/html/2608.21277#bib.bib15)\]\. TimeX\+\+ addresses limitations of conventional masking by learning in\-distribution, label\-preserving explanatory instances through an information\-bottleneck objective\[[13](https://arxiv.org/html/2608.21277#bib.bib5)\]\. These approaches provide useful localization and model diagnostics, but their explanations are primarily expressed as importance over variables and timestamps or as selected input instances\. A practitioner must still infer the semantic condition represented by that evidence—for example, whether a joint change denotes a pollution episode, a weather transition, or a recurring daily regime\. In addition, an explanation selected for an existing predictor is not necessarily the intermediate representation through which its forecast is computed\. Our work instead targets an intrinsic forecasting pathway whose evidence is expressed in a stable vocabulary of named temporal concepts and can be directly intervened on\.

### II\-BConcept\-Based and Language\-Guided Models

Concept bottleneck models were originally developed for image classification, where a model predicts human\-defined visual attributes before using them to infer the target label\[[9](https://arxiv.org/html/2608.21277#bib.bib12)\]\. Because the task prediction is expressed through these intermediate concepts, users can inspect and intervene on the model’s reasoning\. Concept embedding models increase this bottleneck’s capacity by representing the active and inactive states of each concept with learned embeddings\[[5](https://arxiv.org/html/2608.21277#bib.bib17)\]\. Adapting this paradigm to multivariate time series is nontrivial: unlike static visual attributes, temporal concepts may describe trends, durations, cross\-variable relationships, and patterns at different time scales\. Moreover, defining these concepts and labeling every sequence normally requires substantial domain expertise\. Our work adapts the concept\-bottleneck paradigm to forecasting while reducing this supervision burden\.

Language\-guided methods offer a way to reduce this supervision burden\. LaBo uses a language model to propose candidate concepts and grounds them through a vision\-language model for interpretable image classification, while concept\-bottleneck LLMs organize language\-model decisions around human\-readable intermediate concepts\[[23](https://arxiv.org/html/2608.21277#bib.bib13),[19](https://arxiv.org/html/2608.21277#bib.bib14)\]\. These studies demonstrate that language models can assist concept discovery, but they primarily address image or text tasks\. Continuous multivariate sequences additionally require operational definitions that specify when a temporal concept is active, accommodate interactions among variables, and distinguish patterns occurring at different temporal scopes\.

ProtoTS provides a related concept\-like approach to interpretable time\-series forecasting by learning hierarchical latent prototypes associated with canonical forecast curves\[[17](https://arxiv.org/html/2608.21277#bib.bib11)\]\. Although these prototypes support inspection, they are unnamed, and forecasts can draw on learned information beyond the displayed prototype matches\. Users must therefore infer their semantics after training, while the matches alone do not fully account for a prediction\. In contrast, ConceptTS uses an offline LLM to propose named temporal concepts and executable labeling rules, then organizes the supervised concepts across historical context, local forecast intervals, and the full forecast horizon\. This design gives the forecasting pathway a stable semantic vocabulary that supports direct inspection and intervention without manual segment\-level annotation\.

## IIIMethod

![Refer to caption](https://arxiv.org/html/2608.21277v1/ConceptTS_structure_v2.png)Fig\. 1:Overall architecture of ConceptTS for a single forecast sub\-window\. An LLM proposes three concept sets covering the lookback window, each forecast sub\-window, and the global forecast horizon\. A ModernTCN\-style encoder embeds the input; the temporal slices of the embedding drive the concept activation probabilities, which combine with two static concept embeddings to form concept\-context vectors\. These are concatenated with a positional embedding, cross\-sub\-window BiGRU output, and a residual channel, and fed into a stacked residual decoder that refines the forecast in a coarse\-to\-fine manner\.### III\-AFormulation and Framework

#### III\-A1Problem Formulation

In this project, we consider the multivariate time series forecasting problem in which a group of exogenous features is used as context information to help forecast an endogenous \(target\) feature\. Given a look\-back target sequence𝐘1:L∈ℝL\\mathbf\{Y\}\_\{1:L\}\\in\\mathbb\{R\}^\{L\}, look\-back covariates𝐗1:L∈ℝL×C\\mathbf\{X\}\_\{1:L\}\\in\\mathbb\{R\}^\{L\\times C\}, and covariates available over the forecast horizon𝐗L\+1:L\+H∈ℝH×C\\mathbf\{X\}\_\{L\+1:L\+H\}\\in\\mathbb\{R\}^\{H\\times C\}, our modelℱ\\mathcal\{F\}predicts𝐘^L\+1:L\+H=ℱ\(𝐘1:L,𝐗1:L,𝐗L\+1:L\+H\)\\widehat\{\\mathbf\{Y\}\}\_\{L\+1:L\+H\}=\\mathcal\{F\}\(\\mathbf\{Y\}\_\{1:L\},\\mathbf\{X\}\_\{1:L\},\\mathbf\{X\}\_\{L\+1:L\+H\}\)\. In addition to this future\-aware setup, we also evaluate a look\-back\-only setting that removes access to𝐗L\+1:L\+H\\mathbf\{X\}\_\{L\+1:L\+H\}and forces the model to forecast from𝐘1:L\\mathbf\{Y\}\_\{1:L\}and𝐗1:L\\mathbf\{X\}\_\{1:L\}alone\.

#### III\-A2Framework Overview

Fig\.[1](https://arxiv.org/html/2608.21277#S3.F1)illustrates the ConceptTS pipeline\. An offline LLM proposer generates three concept sets — for the lookback window, forecast sub\-windows, and the full forecast horizon — each paired with an executable predicate that labels every training segment\. A ModernTCN\-style encoder preserves per\-channel and temporal structure, producing embeddings that drive concept activations in three parallel CEM bottlenecks\. A shared residual sub\-decoder assembles the concept contexts, cross\-sub\-window BiGRU output, and a regularized residual channel into the final forecast, trained end\-to\-end with the hybrid loss\.

### III\-BLLM\-driven Concept Proposer

Instead of relying on human\-authored concept sets and per\-segment labels \(the standard CBM/CEM requirement\), we provide an offline LLM with dataset statistics and ask it to return three concept sets along with executable Python predicates that label every training segment\.

#### III\-B1Per\-Segment Statistical Summary

For each input segmentsis\_\{i\}, we first extract the structured statistical summary from these four aspects:

- •Look\-back statistics:\{mean, std, slope, range\} per channel; plus FFT\-derived \{dominant period, 24h amplitude, spectral entropy, low\-frequency ratio\} to expose cyclic patterns\.
- •Forecast statistics:\{mean, std, slope, range\} per non\-target feature channel over the horizon\.
- •Sub\-window statistics:\{mean, std, slope, range\} per sub\-window for finer\-grained descriptions\.
- •Cross\-channel and time context:pairwise Pearson correlations between key channel pairs; look\-back/horizon start\-end timestamps; season tag \(spring/summer/autumn/winter\)\.

Combined together, these information gives the LLM a comprehensive view of what each segment looks like, facilitating its following concept proposal and per\-concept activation labeling\.

#### III\-B2Summary Vector Clustering and Profiling

To provide the LLM with realistic guidance about the dataset’s regime structure, the summary vectors are clustered intoNcN\_\{c\}groups viaKK\-means\. To prevent leakage, this clustering and all per\-cluster and global statistics referenced below are computed on the training split only\. For each clustercc, we compute a compact profile containing \(i\) its size and fraction of the training set, \(ii\) for each continuous channeljj, the z\-score\(μc,j−μglobal,j\)/σglobal,j\(\\mu\_\{c,j\}\-\\mu\_\{\\mathrm\{global\},j\}\)/\\sigma\_\{\\mathrm\{global\},j\}of the cluster’s mean against the global \(training\) distribution, and \(iii\) for each categorical channeljj, the dominant value and its purity \(fraction of cluster members sharing that value\)\. TheNcN\_\{c\}profiles are included in the LLM prompt alongside global per\-channel statistics, providing the LLM with data\-based local statistics to facilitate its concept generation without extending the prompt too long\.

#### III\-B3LLM Concept Proposal Generation

For the initial concept generation, we provide the LLM with a comprehensive prompt containing background information, channel descriptions, global statistical summaries, cluster profiles, the segment scheme, the task specification, and the output format\. A key design choice is that we ask the LLM to generate three types of concept sets, each targeting a different input aspect:

- •look\-back Conceptsdescribe semantic patterns observed in the look\-back window, providing history context and general trends information\. Examples: “is look\-back window hot on average?”, “is history rainy in general?”
- •Forecast Sub\-window Conceptsdescribe fine\-grained local details of the context features within the sub\-window over the forecasting horizon \(e\.g\., “is the sub\-window in morning rush period?”, “is the sub\-window wind speed rising?”\)
- •Global Forecast Conceptsdescribe general characteristics of the forecasting horizon \(e\.g\., “is forecasting horizon observing extreme temperature change?”, “is forecasting horizon experiencing large pressure swing?”\)\. Such global forecast concepts serve as a supplement to forecast sub\-window concepts, extracting macro patterns over the full forecasting horizon\.

For both Forecast Sub\-window and Global Forecast concepts, the prompt explicitly instructs the LLM to build the predicates from context covariates only and never from the target variable itself, since the target values in the forecast horizon are unavailable to the model at inference time\.

Unlike similar work in other domains that prompt the LLM only for natural\-language descriptions of concepts, we explicitly require the language model to return an executable Python predicate that evaluates whether a segment matches the concept, using the precomputed statistics as input\. The LLM proposer itself is invoked only once to generate these predicates; the concept labels for every training segment are then produced automatically by executing each predicate over the segment’s precomputed statistics, avoiding costly per\-segment LLM calls\.

#### III\-B4Concept Filtering

After proposingNi​n​i​t​i​a​l=Nl​o​o​k−b​a​c​k\+Ns​u​b−w​i​n​d​o​w\+Nf​o​r​e​c​a​s​tN\_\{initial\}=N\_\{look\-back\}\+N\_\{sub\-window\}\+N\_\{forecast\}concepts, we compute each concept’s positive raterir\_\{i\}over the training set and filter out concepts that are not useful by retaining only concepts withri∈\[rmin,rmax\]r\_\{i\}\\in\[r\_\{\\min\},r\_\{\\max\}\]\. After filtering,Nc​o​n​c​e​p​tN\_\{concept\}remain\. During the concept evaluating process, the binary labels derived for each segment,yi∈\{0,1\}Nc​o​n​c​e​p​ty\_\{i\}\\in\\\{0,1\\\}^\{N\_\{concept\}\}, become the training target of the concept\-supervision losses\.

### III\-CConvolution\-based Encoder

ProtoTS’s feature fusion and temporal mean pooling form compact prototype representations, while our multiscale concepts require variable\-specific local evidence, leading us to adopt ModernTCN’s variable\-independent embedding to preserve channel identity and its large\-kernel depthwise convolution to capture broad temporal context\[[14](https://arxiv.org/html/2608.21277#bib.bib18)\]\.

Given an input of shape\(B,T,C\)\(B,T,C\), each variable channel would be encoded separately and be projected into the same embedding dimension, D, and all embeddings are concatenated along the variable axis to form a tensor of shape\(B,T,C,D\)\(B,T,C,D\)\. Next, a depthwise convolution with stride S is used to patchify the tensor into shape\(B,C,D,N\)\(B,C,D,N\), withN=TSN=\\frac\{T\}\{S\}\. Each ModernTCN block combines a large\-kernel depthwise temporal convolution with two grouped pointwise inverted bottlenecks that mix information along the feature axis on a per\-variable basis and along the variable axis on a per\-feature basis, respectively\. Each block is wrapped in a residual connection\. A final linear projection over the patch axis maps theNNpatch positions back to the originalTTtimesteps, yielding the output embedding𝐄∈ℝB×T×Denc\\mathbf\{E\}\\in\\mathbb\{R\}^\{B\\times T\\times D\_\{\\text\{enc\}\}\}\. We preserve the time axis to provide more comprehensive information to subsequent concept bottleneck module\. Overall, the encoder structure produces embedding that is rich in semantic meaning along both feature and temporal axes\.

In our default setting, we use two parallel encoders to encode the look\-back window𝐘1:L,𝐗1:L\\mathbf\{Y\}\_\{1:L\},\\mathbf\{X\}\_\{1:L\}and the forecasting horizon𝐗L\+1:L\+H\\mathbf\{X\}\_\{L\+1:L\+H\}separately\. We do this to manually set an information barrier between the two time slots\.

### III\-DParallel Concept Embedding Bottleneck Module

After encoding the input segments, the embedding is then passed into three parallel concept embedding bottlenecks: a look\-back concept bottleneck summarizingwhat the recent history context looks like,NswN\_\{\\mathrm\{sw\}\}forecasting sub\-window concept bottlenecks each capturingwhat key patterns the other covariates display in this portion of prediction horizon, and a global forecasting concept bottleneck tellingwhat macro trends and behaviors the context feature channels expose in the forecasting window\. By feeding in these three concept embeddings into the decoder, the model can make accurate forecasting, and the prediction is highly interpretable by checking the activations of the concepts in each of the three concept bottleneck modules\.

#### III\-D1Attention\-weighted Context Pooling

As mentioned in the last section, the embeddings coming out from the encoder are denoted by𝐄𝐥𝐨𝐨𝐤−𝐛𝐚𝐜𝐤∈ℝB×Tl​o​o​k−b​a​c​k×Denc\\mathbf\{E\_\{look\-back\}\}\\in\\mathbb\{R\}^\{B\\times T\_\{look\-back\}\\times D\_\{\\text\{enc\}\}\}and𝐄𝐟𝐨𝐫𝐞𝐜𝐚𝐬𝐭∈ℝB×Tf​o​r​e​c​a​s​t×Denc\\mathbf\{E\_\{forecast\}\}\\in\\mathbb\{R\}^\{B\\times T\_\{forecast\}\\times D\_\{\\text\{enc\}\}\}\. In order to make three concept bottlenecks summarizing the correct temporal portions, we first use three attention\-weighted pooling modules to convert the embeddings into three corresponding context vectors:

𝐳lb=AttentionPool\(𝐄l​o​o​k−b​a​c​k\[1:Tl​o​o​k−b​a​c​k\]\)\\mathbf\{z\}\_\{\\text\{lb\}\}=\\text\{AttentionPool\}\(\\mathbf\{E\}\_\{look\-back\}\\bigl\[1:T\_\{look\-back\}\\bigr\]\),

𝐳sw\(k\)=AttentionPool\(𝐄f​o​r​e​c​a​s​t\[\(k−1\)Tsw\+1:kTsw\]\)\\mathbf\{z\}\_\{\\text\{sw\}\}^\{\(k\)\}=\\text\{AttentionPool\}\(\\mathbf\{E\}\_\{forecast\}\\bigl\[\(k\-1\)\\,T\_\{\\text\{sw\}\}\+1\\,:\\,k\\,T\_\{\\text\{sw\}\}\\bigr\]\),

𝐳gfc=AttentionPool\(𝐄f​o​r​e​c​a​s​t\[1:Tf​o​r​e​c​a​s​t\]\)\\mathbf\{z\}\_\{\\text\{gfc\}\}=\\text\{AttentionPool\}\(\\mathbf\{E\}\_\{forecast\}\\bigl\[1:T\_\{forecast\}\\bigr\]\)

in whichTl​o​o​k−b​a​c​kT\_\{look\-back\},Tf​o​r​e​c​a​s​tT\_\{forecast\}, andTswT\_\{\\text\{sw\}\}represent the number of time steps in the look\-back window, the number of time steps in the forecasting horizon, and the number of time steps in each sub\-window respectively\.

For any temporal slice𝐄∈ℝB×T′×D\\mathbf\{E\}\\in\\mathbb\{R\}^\{B\\times T^\{\{\}^\{\\prime\}\}\\times D\}, theAttentionPool​\(⋅\)\\text\{AttentionPool\}\(\\cdot\)computes an additive\-attention score per timestep, softmaxes them along the time axis, and returns the weighted sum:

𝜶=softmax\(𝐖2tanh\(𝐖1𝐄\)\),𝐳=∑t=1T′αt𝐄t∈ℝD,\\boldsymbol\{\\alpha\}=\\mathrm\{softmax\}\\\!\\left\(\\mathbf\{W\}\_\{2\}\\,\\tanh\(\\mathbf\{W\}\_\{1\}\\mathbf\{E\}\)\\right\),\\qquad\\mathbf\{z\}=\\sum\_\{t=1\}^\{T^\{\{\}^\{\\prime\}\}\}\\alpha\_\{t\}\\,\\mathbf\{E\}\_\{t\}\\in\\mathbb\{R\}^\{D\},\(1\)where𝐖1\\mathbf\{W\}\_\{1\}and𝐖2\\mathbf\{W\}\_\{2\}are the two layers of a small attention MLP\.

We use additive attention instead of the time\-axis mean pooling adopted in ProtoTS: mean pooling weights every timestep equally and dilutes any temporally\-localized signal into a single average, whereas additive attention learns content\-dependent weights that emphasize the semantically important timesteps and suppress the rest, giving the concept bottlenecks a sharper temporal summary\.

#### III\-D2Concept Embedding Module

We adapt the full concept embedding model \(CEM\) structures in our model, which is an alternative option to the original concept bottleneck model \(CBM\)\[[5](https://arxiv.org/html/2608.21277#bib.bib17)\]\. In CEM, the expressive capacity of each concept is enhanced by two high dimensional embeddings, one representswhen the concept is active\(𝐞c\+\\mathbf\{e\}\_\{c\}^\{\+\}\) and the other one representswhen the concept is absent\(𝐞c−\\mathbf\{e\}\_\{c\}^\{\-\}\)\. One important innovation is we design𝐞c\+\\mathbf\{e\}\_\{c\}^\{\+\}and𝐞c−\\mathbf\{e\}\_\{c\}^\{\-\}to beinput\-independent, meaning all concept embeddings are shaped through learning and kept static during inference stage\. Such design is to make sure there’s no strong information leakage that undermines the strength of interpretability\.

A small scoring headhch\_\{c\}takes the encoder output embedding𝐳\\mathbf\{z\}and produces a scalar logit, which gives the concept activation probability after a sigmoid activation function:

pc​\(𝐳\)=σ⁡\(hc​\(𝐳\)\)\.p\_\{c\}\(\\mathbf\{z\}\)=\\sigma\\\!\\left\(h\_\{c\}\(\\mathbf\{z\}\)\\right\)\.\(2\)
The final concept context vector, for each concept, is the corresponding weighted mixture of the two concept embeddings:

𝐜c​\(𝐳\)=pc​\(𝐳\)⋅𝐞c\+\+\(1−pc​\(𝐳\)\)⋅𝐞c−∈ℝdc\.\\mathbf\{c\}\_\{c\}\(\\mathbf\{z\}\)=p\_\{c\}\(\\mathbf\{z\}\)\\cdot\\mathbf\{e\}\_\{c\}^\{\+\}\+\\left\(1\-p\_\{c\}\(\\mathbf\{z\}\)\\right\)\\cdot\\mathbf\{e\}\_\{c\}^\{\-\}\\in\\mathbb\{R\}^\{d\_\{c\}\}\.\(3\)Stacking across allNNconcepts within the bottleneck yields the bottleneck output:

𝐂⁡\(𝐳\)=\[𝐜1​\(𝐳\);𝐜2​\(𝐳\);…;𝐜N​\(𝐳\)\]∈ℝN⋅dc\.\\mathbf\{C\}\(\\mathbf\{z\}\)=\\left\[\\mathbf\{c\}\_\{1\}\(\\mathbf\{z\}\);\\,\\mathbf\{c\}\_\{2\}\(\\mathbf\{z\}\);\\,\\ldots;\\,\\mathbf\{c\}\_\{N\}\(\\mathbf\{z\}\)\\right\]\\in\\mathbb\{R\}^\{N\\cdot d\_\{c\}\}\.\(4\)

#### III\-D3Sequence\-aware Sub\-window Connection

Since forecast sub\-windows are temporally correlated \(e\.g\., the effect of high wind speed depends on the humidity of the preceding sub\-window\), we make the decoder sub\-window\-aware\. Each sub\-window’s concept context𝐂sw\(k\)\\mathbf\{C\}\_\{\\text\{sw\}\}^\{\(k\)\}is projected to a summary vector𝐬\(k\)=𝐖summ​𝐂sw\(k\)\\mathbf\{s\}^\{\(k\)\}=\\mathbf\{W\}\_\{\\text\{summ\}\}\\,\\mathbf\{C\}\_\{\\text\{sw\}\}^\{\(k\)\}and passed through a bidirectional GRU:

\[𝐮\(1\),…,𝐮\(Nsw\)\]=BiGRU⁡\(\[𝐬\(1\),…,𝐬\(Nsw\)\]\)\.\\left\[\\mathbf\{u\}^\{\(1\)\},\\ldots,\\mathbf\{u\}^\{\(N\_\{\\text\{sw\}\}\)\}\\right\]=\\mathrm\{BiGRU\}\\\!\\left\(\\left\[\\mathbf\{s\}^\{\(1\)\},\\ldots,\\mathbf\{s\}^\{\(N\_\{\\text\{sw\}\}\)\}\\right\]\\right\)\.\(5\)
The output𝐮\(k\)\\mathbf\{u\}^\{\(k\)\}is concatenated into the sub\-decoder’s input at sub\-windowkk, providing cross\-sub\-window context\. Using the concept bottleneck output𝐂sw\(k\)\\mathbf\{C\}\_\{\\text\{sw\}\}^\{\(k\)\}\(rather than the encoder embedding\) as the BiGRU input prevents information from leaking around the bottleneck\.

TABLE I:Forecasting performance in the*future\-aware*setting across three Beijing monitoring stations\. Values are mean±\\pmstd inμ\\mug/m3across three training seeds\. Cells shadedlight redmark a baseline that outperforms ConceptTS by more than10%10\\%; cells shadedlight greenmark a baseline that underperforms ConceptTS by more than10%10\\%\. Unshaded cells fall within±10%\\pm 10\\%of ConceptTS\. Lower is better\.TABLE II:Forecasting performance in the*look\-back\-only*setting\. Cell are colored the same way as TABLE[I](https://arxiv.org/html/2608.21277#S3.T1)
#### III\-D4Residual Sub\-decoder Stack

Eventually, the forecast is produced by a shared sub\-decoder that takes in a concatenation of the concept bottleneck output embeddings of all three types, the cross\-sub\-window context embedding, a positional sub\-window id embedding, and a narrow residual bypass:

𝐝\(k\)=\[𝐂lb;𝐂sw\(k\);𝐂gfc;𝐮\(k\);𝐞sw\(k\);𝐫lb;𝐫sw\(k\)\],\\mathbf\{d\}^\{\(k\)\}=\\left\[\\mathbf\{C\}\_\{\\mathrm\{lb\}\}\\,;\\,\\mathbf\{C\}\_\{\\mathrm\{sw\}\}^\{\(k\)\}\\,;\\,\\mathbf\{C\}\_\{\\mathrm\{gfc\}\}\\,;\\mathbf\{u\}^\{\(k\)\}\\,;\\,\\mathbf\{e\}\_\{\\mathrm\{sw\}\}^\{\(k\)\}\\,;\\,\\mathbf\{r\}\_\{\\mathrm\{lb\}\}\\,;\\,\\mathbf\{r\}\_\{\\mathrm\{sw\}\}^\{\(k\)\}\\right\],\(6\)with𝐞sw\(k\)=Embedding⁡\(Nsw,did\)​\(k\)\\mathbf\{e\}\_\{\\mathrm\{sw\}\}^\{\(k\)\}=\\mathrm\{Embedding\}\(N\_\{\\mathrm\{sw\}\},d\_\{\\mathrm\{id\}\}\)\(k\), a positional\-only embedding that carries no data\-derived signal\.𝐂lb\\mathbf\{C\}\_\{\\mathrm\{lb\}\}and𝐂gfc\\mathbf\{C\}\_\{\\mathrm\{gfc\}\}represent the eventual output from the look\-back concept bottleneck and the global forecast concept bottleneck, respectively\. The last two terms form an optional residual bypass, with𝐫lb=𝐖lb​𝐳lb\\mathbf\{r\}\_\{\\mathrm\{lb\}\}=\\mathbf\{W\}\_\{\\mathrm\{lb\}\}\\mathbf\{z\}\_\{\\mathrm\{lb\}\}shared across all sub\-windows and𝐫sw\(k\)=𝐖sw​𝐳sw\(k\)\\mathbf\{r\}\_\{\\mathrm\{sw\}\}^\{\(k\)\}=\\mathbf\{W\}\_\{\\mathrm\{sw\}\}\\mathbf\{z\}\_\{\\mathrm\{sw\}\}^\{\(k\)\}computed per sub\-window, both linear projections of the pooled look\-back and forecast sub\-window encoder features into a low\-dimensional spaceℝr\\mathbb\{R\}^\{r\}\. This bypass carries the continuous information that the soft scalar concept activations cannot fully express\. It is kept low\-dimensional and regularized by a dedicated loss term to discourage the model from relying too much on the residual connection; the weightαres\\alpha\_\{\\mathrm\{res\}\}of this regularization is a direct knob for trading off forecast accuracy against how much of the prediction is routed through the concept bottlenecks\.

Inspired by NHiTS, the decoder is a total ofNd​e​cN\_\{dec\}stacked MLP blocks, with each block refining the output taken from its previous block\[[2](https://arxiv.org/html/2608.21277#bib.bib19)\]\.

𝐲^\(k\)=∑b=1Bdec𝚫b\(k\),𝚫b\(k\)=Blockb​\(𝐢b\(k\)\)\.\\hat\{\\mathbf\{y\}\}^\{\(k\)\}=\\sum\_\{b=1\}^\{B\_\{\\mathrm\{dec\}\}\}\\boldsymbol\{\\Delta\}\_\{b\}^\{\(k\)\},\\qquad\\boldsymbol\{\\Delta\}\_\{b\}^\{\(k\)\}=\\mathrm\{Block\}\_\{b\}\\\!\\left\(\\mathbf\{i\}\_\{b\}^\{\(k\)\}\\right\)\.\(7\)We also use decoder feedback structure, meaning blockb\>1b\>1additionally sees the running forecast so far:𝐢1\(k\)=𝐝\(k\)\\mathbf\{i\}\_\{1\}^\{\(k\)\}=\\mathbf\{d\}^\{\(k\)\}and𝐢b\(k\)=\[𝐝\(k\);flatten⁡\(∑b′<b𝚫b′\(k\)\)\]\\mathbf\{i\}\_\{b\}^\{\(k\)\}=\\left\[\\mathbf\{d\}^\{\(k\)\}\\,;\\,\\mathrm\{flatten\}\\\!\\left\(\\sum\_\{b^\{\\prime\}<b\}\\boldsymbol\{\\Delta\}\_\{b^\{\\prime\}\}^\{\(k\)\}\\right\)\\right\]forb\>1b\>1\. Essentially, the stacked sub\-decoder module is constantly using the information summarized by the three types of concept bottlenecks along with the neighboring sub\-windows’ contexts to improve its forecasting, from initial coarse forecasting to eventual high\-quality prediction\.

The shared decoder outputs forecasting𝐲^\(k\)∈ℝB×Tsw×1\\hat\{\\mathbf\{y\}\}^\{\(k\)\}\\in\\mathbb\{R\}^\{B\\times T\_\{\\mathrm\{sw\}\}\\times 1\}for thekk\-th sub\-window\. Then, all partial predictions are concatenated along time axis to form the final forecasting curve\.

### III\-ETraining Pipeline

The two parallel encoders, the three CEM modules, and the decoder are trained jointly end\-to\-end\. We use a hybrid loss function to make sure different parts of the model work as expected:

ℒ=αf​ℒforecast\+αlb​ℒconceptlb\+αfc​ℒconceptfc\+αgfc​ℒconceptgfc\+αbd​ℒboundary\+αcc​ℒcollapse\+αresidual​ℒresidual,\\begin\{split\}\\mathcal\{L\}=\{\}&\\alpha\_\{\\mathrm\{f\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{forecast\}\}\+\\alpha\_\{\\mathrm\{lb\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{concept\}\}^\{\\mathrm\{lb\}\}\+\\alpha\_\{\\mathrm\{fc\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{concept\}\}^\{\\mathrm\{fc\}\}\\\\ &\+\\alpha\_\{\\mathrm\{gfc\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{concept\}\}^\{\\mathrm\{gfc\}\}\+\\alpha\_\{\\mathrm\{bd\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{boundary\}\}\\\\ &\+\\alpha\_\{\\mathrm\{cc\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{collapse\}\}\+\\alpha\_\{\\mathrm\{residual\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{residual\}\},\\end\{split\}\(8\)
The full objective combines seven weighted loss terms: \(i\) anL1L\_\{1\}forecast loss on the target; \(ii\)–\(iv\) three BCE concept losses on the LLM\-derived binary labels for the lookback, forecast sub\-window, and global forecast bottlenecks, forcing the predicted activation score of each concept to be close to the LLM\-derived labels; \(v\) anL1L\_\{1\}boundary\-smoothness penalty across sub\-window boundaries, preventing discontinuous values in the final forecast output; \(vi\) a variance floor that keeps each concept’s activation from collapsing to a near\-constant \(and therefore uninformative\) value across inputs; and \(vii\) anL2L\_\{2\}regularizer on the residual channel that directly controls the accuracy\-interpretability trade\-off\.

## IVExperiments

### IV\-AModel Performance

#### IV\-A1Dataset

We choose to use Beijing Multi\-Site Air Quality dataset in this project, which is an open\-source multivariate time series dataset provided by UC Irvine Machine Learning Repository\[[3](https://arxiv.org/html/2608.21277#bib.bib20)\]\. This dataset contains 6 pollutants concentrations and 5 relevant meteorological context features at 12 air quality monitoring sites in Beijing, and the time spans from March 1st, 2013 to February 28th, 2017, at a one\-hour recording interval\. We choose to use PM2\.5 concentration as our target feature, and the context features include five other pollutants \(PM10, SO2, NO2, CO, O3\) and five meteorological measurements \(temperature, pressure, dew point, precipitation, wind speed\)\. The missing data in each feature channel are filled using linear interpolation\.

We benchmark overall forecasting performance on three monitoring sites —Aotizhongxin,Dingling, andTiantan— and useAotizhongxinas the representative site for all subsequent ablation and case\-study experiments\.

#### IV\-A2Evaluation Metrics

We report the model performance using two standard metrics in time series forecasting: mean absolute error \(MAE\) and root mean squared error \(RMSE\)\.

The two metrics are complementary: MAE shows on average how much the model’s prediction deviates from the ground\-truth value at each time step, while RMSE punishes large errors more heavily, displaying how well the model can handle the challenging portions of the forecasting like the air pollutant spikes\. The metrics are reported in rawμ​g/m3\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}, computed by inverting the training\-set z\-scoring\.

#### IV\-A3Experiment Settings

We implement two experiment settings to reflect two natural data availability regimes that are both common in multivariate time series forecasting domain\. In thefuture\-awaresetting \(the default setting clarified in section 3\.1\.1\), the exogenous features are assumed to be known beforehand for the forecasting horizon, which is realistic when the context features come from an upstream prediction system\. This is often the case in forecasting tasks related to meteorological factors like wind speed, precipitation, and temperatures\. In thelook\-back\-onlysetting, the model only has accessibility to all features in the history\{𝐘1:L,𝐗1:L\}\\\{\\mathbf\{Y\}\_\{1:L\},\\mathbf\{X\}\_\{1:L\}\\\}, matching the regular setting that Time\-Series\-Library uses for its benchmark models\.

The two settings share the same encoder\-pool\-CEM\-decoder model structure, but the concept bottleneck types are different\. In future\-aware setting, the LLM proposer proposes three concept sets for look\-back window, forecasting sub\-window, and global forecasting horizon respectively\. In the look\-back\-only setting, there is only one encoder that encodes the look\-back window \(since there is no future data to encode\), and the proposer instead proposes two concept sets, one for global look\-back window, and one for look\-back sub\-window since now the model additionally partitions the look\-back window intoNlbswN\_\{\\text\{lbsw\}\}sub\-windows to provide more interpretability and higher prediction accuracy\.

Because most Time\-Series\-Library baselines only consume the look\-back window, we compare against a different baseline set in each setting\.

#### IV\-A4Model Evaluation

Under future\-aware setting, as shown in Table[I](https://arxiv.org/html/2608.21277#S3.T1), our model is competitive with state\-of\-the\-art benchmark time forecasters\. TakingAotizhongxinas example, our model achieves a12\.71​μ​g/m312\.71\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}MAE, outperforming Informer \(28\.62​μ​g/m328\.62\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[27](https://arxiv.org/html/2608.21277#bib.bib8)\], Random Forest \(26\.73​μ​g/m326\.73\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[1](https://arxiv.org/html/2608.21277#bib.bib21)\], DeepAR \(13\.97​μ​g/m313\.97\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[18](https://arxiv.org/html/2608.21277#bib.bib24)\], and NHiT \(16\.94​μ​g/m316\.94\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[2](https://arxiv.org/html/2608.21277#bib.bib19)\], and matches performances of XGBoost \(12\.34​μ​g/m312\.34\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[4](https://arxiv.org/html/2608.21277#bib.bib22)\], TFT \(12\.44​μ​g/m312\.44\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[10](https://arxiv.org/html/2608.21277#bib.bib15)\], and LightGBM \(12\.46​μ​g/m312\.46\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[8](https://arxiv.org/html/2608.21277#bib.bib23)\], while providing concept\-level explanations these models cannot offer\. Consequently, our model contributes to the multivariate time series forecasting by achieving SOTA accuracy while directly incorporating human\-like concept\-based decision making process into the forecasting pipeline\.

For the look\-back\-only setting, we compare our model with several Time\-Series\-Library benchmarks and the results are shown in Table[II](https://arxiv.org/html/2608.21277#S3.T2)\. The results illustrate that our model’s prediction accuracy is also competitive with SOTA time series forecasters even only taking in historical context\. For the stationAotizhongxin, ConceptTS \(55\.59​μ​g/m355\.59\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}MAE\) beats TimeXer \(57\.43​μ​g/m357\.43\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[22](https://arxiv.org/html/2608.21277#bib.bib25)\], iTransformer \(59\.16​μ​g/m359\.16\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[12](https://arxiv.org/html/2608.21277#bib.bib10)\], DLinear \(56\.27​μ​g/m356\.27\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[24](https://arxiv.org/html/2608.21277#bib.bib9)\], and LSTM \(55\.91​μ​g/m355\.91\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\[[6](https://arxiv.org/html/2608.21277#bib.bib27)\]\. Notably, although the other two models \(Informer, Crossformer\) yield lower MAE, our model remains highly competitive, underperforming them by only0\.55​μ​g/m30\.55\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}and0\.81​μ​g/m30\.81\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}respectively\[[27](https://arxiv.org/html/2608.21277#bib.bib8)\]\[[25](https://arxiv.org/html/2608.21277#bib.bib26)\]\.

In both cases, the gaps on RMSE are somewhat larger, suggesting ConceptTS handles typical timesteps well but is slightly less sensitive to extreme pollution spikes, a limitation we leave to future work\.

### IV\-BAblation Study

We conduct experiments to study how different structure designs contribute to the performance of the model\. Specifically, we study \(i\) how the number of forecast sub\-window concepts affect forecast accuracy, and \(ii\) how the residual\-channel weightαr​e​s​i​d​u​a​l\\alpha\_\{residual\}trades off accuracy against interpretability\.

#### IV\-B1Number of Concepts

To study the influence of the number of concepts on forecasting accuracy, we fix both the look\-back and global forecast concept counts at 10\. This gives the model enough baseline capacity to forecast accurately without letting it depend too heavily on these two bottlenecks\. Then, we sweep the forecast sub\-window concept number to see how the MAE on the test set changes\. In this experiment, we disable the residual connection so all forecasting must go through the concept bottleneck entirely\.

![Refer to caption](https://arxiv.org/html/2608.21277v1/concept_num_vs_mae.png)Fig\. 2:Effect of the number of forecast sub\-window concepts on forecast MAE\. Thexx\-axis reports the number of forecast sub\-window concepts\(with the number of look\-back and global forecast concepts fixed at1010each\)\.For each sub\-window concept countNswN\_\{\\text\{sw\}\}, we run three LLM proposals with different random seeds and train the model separately for each; the reported test MAE is the average over these three runs\. Such design mitigates the influence of individual LLM\-proposal quality and lets us observe the general trend\. As shown in Fig[2](https://arxiv.org/html/2608.21277#S4.F2), the results indicate that MAE decreases monotonically asNswN\_\{\\text\{sw\}\}increases\. This confirms that the model genuinely relies on combinations of concepts to forecast, and it does hurt the model’s performance significantly when the available concepts are limited\.

#### IV\-B2Trade\-off between Accuracy and Interpretability

In Method section, we introduced the residual channel and its weightαr​e​s​i​d​u​a​l\\alpha\_\{residual\}; this section studies empirically howαr​e​s​i​d​u​a​l\\alpha\_\{residual\}balances forecast accuracy and interpretability\.

We use the same group of concept sets proposed by LLM to make the comparison fair, then we sweep theαresidual\\alpha\_\{\\mathrm\{residual\}\}from 0\.01 to 1\.0\. Table[III](https://arxiv.org/html/2608.21277#S4.T3)reports the results\. A largerαresidual\\alpha\_\{\\mathrm\{residual\}\}penalizes the residual connection more strongly, forcing the decoder to rely more on the concept bottleneck\. Asαresidual\\alpha\_\{\\mathrm\{residual\}\}decreases, the full\-model MAE improves \(15\.35→12\.7115\.35\\rightarrow 12\.71\), but the gap between full\-model and concept\-only MAE — the “MAE improvement” column — widens from0\.5%0\.5\\%to161\.0%161\.0\\%, indicating that the decoder increasingly bypasses the bottleneck via the residual path\.αresidual\\alpha\_\{\\mathrm\{residual\}\}thus provides a direct knob for trading accuracy against interpretability\.

TABLE III:Ablation on the residual\-channel weightαresidual\\alpha\_\{\\mathrm\{residual\}\}\. MAE values are inμ\\mug/m3; all values are mean±\\pmstd across three training seeds\. Improvement is the relative gap between concept\-only and full\-model MAE; larger values indicate the decoder relies more on the residual channel\.![Refer to caption](https://arxiv.org/html/2608.21277v1/residual_connection_weight.jpg)Fig\. 3:Comparison of full model forecasting versus concept\-only forecasting under differentαresidual\\alpha\_\{\\mathrm\{residual\}\}\. Each column corresponds to one value ofαresidual\\alpha\_\{\\mathrm\{residual\}\}\.The forecasting curves on the test set are shown in Figure[3](https://arxiv.org/html/2608.21277#S4.F3)\. The two rows display two randomly chosen test windows, and the three columns show the forecasting results atαresidual=0\.01,0\.1,and​1\.0\\alpha\_\{\\mathrm\{residual\}\}=0\.01,\\ 0\.1,\\ \\text\{and\}\\ 1\.0\. Each plot contains three curves: the ground\-truth values, the full\-model prediction, and the concept\-only prediction\. As can be seen from the figure, asαresidual\\alpha\_\{\\mathrm\{residual\}\}increases, the concept\-only curve gradually converges to the full\-model curve, indicating that the decoder increasingly bases its prediction on the concept bottlenecks\. Another point worth noticing is that, despite the relatively large MAE of the concept\-only prediction whenαresidual\\alpha\_\{\\mathrm\{residual\}\}is very small \(e\.g\.,αresidual=0\.01\\alpha\_\{\\mathrm\{residual\}\}=0\.01\), the concept\-only curve still captures the correct shape of the ground truth at both the macro and micro scales\. This shows that even whenαresidual\\alpha\_\{\\mathrm\{residual\}\}is small and the model relies heavily on the residual connection for accurate forecasting, the concept bottlenecks can still provide meaningful interpretability\.

### IV\-CCase Study

![Refer to caption](https://arxiv.org/html/2608.21277v1/case1_best_w6406_v2.png)Fig\. 4:Prediction result of our model on a randomly chosen test window, together with the top\-5 most activated concepts in each of the three concept sets\. The black dotted line marks the boundary between the look\-back window and the forecast horizon; the red shaded region indicates the sub\-window randomly spotlighted for the case study\.For the case studies below we use 20/30/20 concepts for the lookback/forecast sub\-window/forecast\-global bottlenecks and disable the residual channel to inspect the concept pathway in isolation\.

#### IV\-C1Concepts Activation

The forecast for one of the test windows is shown in Fig\.[4](https://arxiv.org/html/2608.21277#S4.F4), together with the meteorological context features \(other pollutants are omitted due to space constraints\)\. On the right, we list the top\-5 most activated concepts for each of the three concept sets; the gray text in parentheses under each concept is the natural\-language description generated by the LLM proposer\.

From the figure and the concept lists, we can see that the model correctly activates the concepts that describe the input\. To be more specific, the chosen sample is a cold window in early February, with air temperature oscillating around0∘​C0\\,^\{\\circ\}\\mathrm\{C\}and dew point constantly below−8∘​C\-8\\,^\{\\circ\}\\mathrm\{C\}\. These features are correctly captured across all three bottlenecks: each of them assigns very high confidence to concepts describing the window as “cold” and “dry”, and the forecast sub\-window bottleneck additionally identifies the sub\-window as a “winter block”\. More interestingly, the look\-back bottleneck also fires on a concept named “is\_history\_wind\_anticorrelated”, which describes the pattern where PM2\.5 is negatively correlated with wind speed\. This pattern is clearly visible in the look\-back window: the wind speed stays near zero for the first 25 hours while PM2\.5 is high, and as the wind speed rises over the next 50 hours, the PM2\.5 level drops sharply to near zero\. This ability to expose a learned cross\-channel relationship in a human\-readable way is exactly what previous black\-box models cannot offer — they may implicitly rely on the same wind–PM2\.5 anticorrelation to make their predictions, but they cannot tell the user that they are doing so\.

#### IV\-C2Concept Sweep Study

![Refer to caption](https://arxiv.org/html/2608.21277v1/case2_sweep_fc_is_calm_block.png)Fig\. 5:Forecast curves generated when the activation probability of a single concept is manually swept from00to11\. We choose two test windows to illustrate: one with a low overall pollutant level \(left\) and one with a high pollutant level \(right\)\.To further explore the model’s interpretability, we conduct a counterfactual concept sweep experiment: we manually sweep the activation probability of one concept from00to11, while leaving the other concepts’ predicted probabilities untouched\. This enables us to directly observe how turning one concept on and off influences the model’s prediction\.

One such example is shown in Fig\.[5](https://arxiv.org/html/2608.21277#S4.F5)\. The chosen concept is the sub\-window concept “is\_calm\_block”, which describes whether the wind speed stays consistently low during the sub\-window\. We show two contrasting scenarios — a low\-pollutant window \(ground truth mean≈12​μ​g/m3\\approx 12\\,\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\) and a high\-pollutant window \(ground truth mean≈332​μ​g/m3\\approx 332\\,\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\. In both scenarios the concept is positively related to the forecast: as the activation probability of “is\_calm\_block” increases from00to11, the overall predicted PM2\.5 level rises accordingly\. This matches physical intuition, since calm air prevents pollutant particles from dispersing and therefore leads to a higher local pollution concentration\.

Together, these two case studies illustrate two complementary interpretability properties of our model: the concept activations faithfully describe the input, and intervening on a concept’s activation can help a domain expert intuitively understand how the concept contributes to the forecast\.

#### IV\-C3Concept Intervention Diagnostic

To further verify that our model genuinely relies on the concept activation channel to make forecasts, we conduct a concept intervention diagnostic on the test set\. At inference time, we hold the model’s weights unchanged but replace every concept’s predicted activation probabilitypcp\_\{c\}with one of four regimes:

- •baseline: the model’s own predicted activation probabilities \(no change\)\.
- •oracle: each concept activation is replaced with its ground\-truth binary label\.
- •flip: each concept activation is replaced with1−1\-ground\-truth label \(purely inverse the original prediction\)\.
- •random: each concept activation is replaced with a sample fromBernoulli⁡\(0\.5\)\\mathrm\{Bernoulli\}\(0\.5\)\(“random guess”\)\.

The idea is straightforward\. If the decoder relies mainly on the concept activation channel to make forecasts, then flipping the activations to their opposites should dramatically hurt the forecast quality\. On the other hand, if the decoder mostly bypasses the concepts and reads information off the CEM’s±\\pmembeddings, then flipping the activations should have little effect on the forecast\.

TABLE IV:Concept intervention diagnostic on the test set\.The results are shown in Table[IV](https://arxiv.org/html/2608.21277#S4.T4)\. Theflipintervention increases the test MAE from15\.37115\.371to99\.661​μ​g/m399\.661~\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\(a degradation of84\.990​μ​g/m384\.990~\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\), and therandomintervention increases it to60\.818​μ​g/m360\.818~\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\(↓45\.446​μ​g/m3\\downarrow 45\.446~\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\)\. Both degradations are significant\. This sharp separation confirms that the concept activation channel carries the majority of the predictive signal — flipping the concepts fundamentally changes the forecast, which would not happen if the decoder were bypassing the bottleneck\. The slight degradation brought by providing the model with real ground\-truth concept labels \(oracle\) is also noticeable\. It demonstrates that the soft activation probability assignment learned by the model carries more information than the hard binary category\. The model learns to use continuous probability assignment to represent different degree of a concept \(“very cold”, “somewhat cold”, “slightly cold”\)\.

#### IV\-C4Failure Mode

A concept\-level AUROC analysis reveals that shape\-dependent concepts — those describing temporal trends like monotonic rises or falls — are consistently the hardest for the model to learn, a limitation we trace to the statistics\-based label pipeline\. Additionally, the framework is better suited to tasks whose target is driven mainly by context features than to those dominated by strong long\-term cycles\.

## VConclusion

We introduced ConceptTS, an interpretable multivariate time\-series forecaster that adapts concept bottlenecks from image classification\. An offline LLM constructs named temporal concepts and executable labeling rules, reducing manual annotation and requiring no LLM at inference time\. Across three air\-quality sites and two settings, ConceptTS achieves competitive accuracy while supporting concept\-level inspection and intervention\. Inverting ground\-truth concept labels increased MAE about ninefold, showing strong reliance on the concept pathway\.

## References

- \[1\]\(2001\)Random forests\.Machine Learning45\(1\),pp\. 5–32\.External Links:[Document](https://dx.doi.org/10.1023/A%3A1010933404324)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p1.1)\.
- \[2\]C\. Challu, K\. G\. Olivares, B\. N\. Oreshkin, F\. Garza Ramirez, M\. Mergenthaler Canseco, and A\. Dubrawski\(2023\)NHITS: neural hierarchical interpolation for time series forecasting\.Proceedings of the AAAI Conference on Artificial Intelligence37\(6\),pp\. 6989–6997\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v37i6.25854)Cited by:[§III\-D4](https://arxiv.org/html/2608.21277#S3.SS4.SSS4.p2.1),[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p1.1)\.
- \[3\]S\. Chen\(2017\)Beijing Multi\-Site Air Quality\.Note:UCI Machine Learning RepositoryExternal Links:[Document](https://dx.doi.org/10.24432/C5RK5G),[Link](https://archive.ics.uci.edu/dataset/501/beijingmultisiteairqualitydata)Cited by:[§E\-A](https://arxiv.org/html/2608.21277#A5.SS1.p1.1),[§IV\-A1](https://arxiv.org/html/2608.21277#S4.SS1.SSS1.p1.1)\.
- \[4\]T\. Chen and C\. Guestrin\(2016\)XGBoost: a scalable tree boosting system\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,New York, NY, USA,pp\. 785–794\.External Links:[Document](https://dx.doi.org/10.1145/2939672.2939785)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p1.1)\.
- \[5\]M\. Espinosa Zarlenga, P\. Barbiero, G\. Ciravegna, G\. Marra, F\. Giannini, M\. Diligenti, Z\. Shams, F\. Precioso, S\. Melacci, A\. Weller, P\. Lió, and M\. Jamnik\(2022\)Concept embedding models: beyond the accuracy\-explainability trade\-off\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 21400–21413\.External Links:[Document](https://dx.doi.org/10.52202/068431-1555)Cited by:[§II\-B](https://arxiv.org/html/2608.21277#S2.SS2.p1.1),[§III\-D2](https://arxiv.org/html/2608.21277#S3.SS4.SSS2.p1.1)\.
- \[6\]S\. Hochreiter and J\. Schmidhuber\(1997\)Long short\-term memory\.Neural Computation9\(8\),pp\. 1735–1780\.External Links:[Document](https://dx.doi.org/10.1162/neco.1997.9.8.1735)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p2.1)\.
- \[7\]Y\. Huang, J\. Chen, D\. Liu, and K\. Ma\(2026\)SigTime: learning and visually explaining time series signatures\.IEEE Transactions on Visualization and Computer Graphics32\(2\),pp\. 2099–2113\.External Links:[Document](https://dx.doi.org/10.1109/TVCG.2025.3644956),[Link](https://doi.org/10.1109/TVCG.2025.3644956)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p1.1)\.
- \[8\]G\. Ke, Q\. Meng, T\. Finley, T\. Wang, W\. Chen, W\. Ma, Q\. Ye, and T\. Liu\(2017\)LightGBM: a highly efficient gradient boosting decision tree\.InAdvances in Neural Information Processing Systems,Vol\.30,pp\. 3146–3154\.Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p1.1)\.
- \[9\]P\. W\. Koh, T\. Nguyen, Y\. S\. Tang, S\. Mussmann, E\. Pierson, B\. Kim, and P\. Liang\(2020\)Concept bottleneck models\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 5338–5348\.External Links:[Link](https://proceedings.mlr.press/v119/koh20a.html)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p2.1),[§II\-B](https://arxiv.org/html/2608.21277#S2.SS2.p1.1)\.
- \[10\]B\. Lim, S\. Ö\. Arık, N\. Loeff, and T\. Pfister\(2021\)Temporal fusion transformers for interpretable multi\-horizon time series forecasting\.International Journal of Forecasting37\(4\),pp\. 1748–1764\.External Links:[Document](https://dx.doi.org/10.1016/j.ijforecast.2021.03.012)Cited by:[§II\-A](https://arxiv.org/html/2608.21277#S2.SS1.p2.1),[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p1.1)\.
- \[11\]D\. Liu, S\. Alnegheimish, A\. Zytek, and K\. Veeramachaneni\(2022\)MTV: visual analytics for detecting, investigating, and annotating anomalies in multivariate time series\.Proceedings of the ACM on Human\-Computer Interaction6\(CSCW1\),pp\. 103:1–103:30\.External Links:[Document](https://dx.doi.org/10.1145/3512950),[Link](https://doi.org/10.1145/3512950)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p1.1)\.
- \[12\]Y\. Liu, T\. Hu, H\. Zhang, H\. Wu, S\. Wang, L\. Ma, and M\. Long\(2024\)iTransformer: inverted transformers are effective for time series forecasting\.InInternational Conference on Learning Representations,pp\. 11116–11140\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/2ea18fdc667e0ef2ad82b2b4d65147ad-Abstract-Conference.html)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p2.1)\.
- \[13\]Z\. Liu, T\. Wang, J\. Shi, X\. Zheng, Z\. Chen, L\. Song, W\. Dong, J\. Obeysekera, F\. Shirani, and D\. Luo\(2024\)TimeX\+\+: learning time\-series explanations with information bottleneck\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 32062–32082\.External Links:[Link](https://proceedings.mlr.press/v235/liu24bl.html)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p2.1),[§II\-A](https://arxiv.org/html/2608.21277#S2.SS1.p1.1),[§II\-A](https://arxiv.org/html/2608.21277#S2.SS1.p2.1)\.
- \[14\]D\. Luo and X\. Wang\(2024\)ModernTCN: a modern pure convolution structure for general time series analysis\.InInternational Conference on Learning Representations,pp\. 31728–31770\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/86b1437c1e4c3b3c4debff98234a67e7-Abstract-Conference.html)Cited by:[§III\-C](https://arxiv.org/html/2608.21277#S3.SS3.p1.1)\.
- \[15\]Q\. Pan, W\. Hu, and N\. Chen\(2021\)Two birds with one stone: series saliency for accurate and interpretable multivariate time series forecasting\.InProceedings of the Thirtieth International Joint Conference on Artificial Intelligence,Z\. Zhou \(Ed\.\),pp\. 2884–2891\.External Links:[Document](https://dx.doi.org/10.24963/ijcai.2021/397)Cited by:[§II\-A](https://arxiv.org/html/2608.21277#S2.SS1.p2.1)\.
- \[16\]Y\. Park, A\. Tong, S\. Lee, J\. Seong, Q\. Xie, and J\. Choi\(2026\)Towards transparent time series analysis: exploring methods and enhancing interpretability\.ACM Computing Surveys58\(9\),pp\. 240:1–240:36\.External Links:[Document](https://dx.doi.org/10.1145/3794839),[Link](https://doi.org/10.1145/3794839)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p2.1),[§II\-A](https://arxiv.org/html/2608.21277#S2.SS1.p1.1)\.
- \[17\]Z\. Peng, S\. Ren, X\. Gu, L\. Yang, X\. Wang, and L\. Sun\(2026\)ProtoTS: learning hierarchical prototypes for explainable time series forecasting\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=IbcdVwzLrp)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p2.1),[§II\-B](https://arxiv.org/html/2608.21277#S2.SS2.p3.1)\.
- \[18\]D\. Salinas, V\. Flunkert, J\. Gasthaus, and T\. Januschowski\(2020\)DeepAR: probabilistic forecasting with autoregressive recurrent networks\.International Journal of Forecasting36\(3\),pp\. 1181–1191\.External Links:[Document](https://dx.doi.org/10.1016/j.ijforecast.2019.07.001)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p1.1)\.
- \[19\]C\. Sun, T\. Oikarinen, B\. Ustun, and T\. Weng\(2025\)Concept bottleneck large language models\.InInternational Conference on Learning Representations,pp\. 89371–89411\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/de4ce91dfe56b919ee1c228d6a78f866-Abstract-Conference.html)Cited by:[§II\-B](https://arxiv.org/html/2608.21277#S2.SS2.p2.1)\.
- \[20\]A\. Theissler, F\. Spinnato, U\. Schlegel, and R\. Guidotti\(2022\)Explainable AI for time series classification: a review, taxonomy and research directions\.IEEE Access10,pp\. 100700–100724\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2022.3207765),[Link](https://doi.org/10.1109/ACCESS.2022.3207765)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p2.1),[§II\-A](https://arxiv.org/html/2608.21277#S2.SS1.p1.1)\.
- \[21\]Y\. Wang, H\. Wu, J\. Dong, Y\. Liu, C\. Wang, M\. Long, and J\. Wang\(2026\)Deep time series models: a comprehensive survey and benchmark\.IEEE Transactions on Pattern Analysis and Machine Intelligence,pp\. 1–20\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2026.3690845),[Link](https://doi.org/10.1109/TPAMI.2026.3690845)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p1.1)\.
- \[22\]Y\. Wang, H\. Wu, J\. Dong, G\. Qin, H\. Zhang, Y\. Liu, Y\. Qiu, J\. Wang, and M\. Long\(2024\)TimeXer: empowering transformers for time series forecasting with exogenous variables\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 469–498\.External Links:[Document](https://dx.doi.org/10.52202/079017-0015)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p2.1)\.
- \[23\]Y\. Yang, A\. Panagopoulou, S\. Zhou, D\. Jin, C\. Callison\-Burch, and M\. Yatskar\(2023\)Language in a bottle: language model guided concept bottlenecks for interpretable image classification\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 19187–19197\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52729.2023.01839)Cited by:[§II\-B](https://arxiv.org/html/2608.21277#S2.SS2.p2.1)\.
- \[24\]A\. Zeng, M\. Chen, L\. Zhang, and Q\. Xu\(2023\)Are Transformers effective for time series forecasting?\.Proceedings of the AAAI Conference on Artificial Intelligence37\(9\),pp\. 11121–11128\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v37i9.26317)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p2.1)\.
- \[25\]Y\. Zhang and J\. Yan\(2023\)Crossformer: transformer utilizing cross\-dimension dependency for multivariate time series forecasting\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=vSVLM2j9eie)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p2.1)\.
- \[26\]Z\. Zhao, Y\. Shi, S\. Wu, F\. Yang, W\. Song, and N\. Liu\(2023\)Interpretation of time\-series deep models: a survey\.Note:arXiv preprint arXiv:2305\.14582External Links:2305\.14582,[Document](https://dx.doi.org/10.48550/arXiv.2305.14582),[Link](https://arxiv.org/abs/2305.14582)Cited by:[§I](https://arxiv.org/html/2608.21277#S1.p2.1),[§II\-A](https://arxiv.org/html/2608.21277#S2.SS1.p1.1)\.
- \[27\]H\. Zhou, S\. Zhang, J\. Peng, S\. Zhang, J\. Li, H\. Xiong, and W\. Zhang\(2021\)Informer: beyond efficient transformer for long sequence time\-series forecasting\.Proceedings of the AAAI Conference on Artificial Intelligence35\(12\),pp\. 11106–11115\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v35i12.17325)Cited by:[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p1.1),[§IV\-A4](https://arxiv.org/html/2608.21277#S4.SS1.SSS4.p2.1)\.

## Appendix AProtoTS Failure Mode

![Refer to caption](https://arxiv.org/html/2608.21277v1/score_distribution_protoTS_RE_EW0.001.png)Fig\. 6:Top\-1 win counts \(top\) and mean activation scores \(bottom\) of the 12 root prototypes under ProtoTS’s default entropy\-regularization weight\.proto\_1wins the argmax on essentially every test segment, yet the mean activation scores across all prototypes remain tightly clustered around the uniform baseline1/P=0\.08331/P=0\.0833\. This gap between argmax dominance and near\-uniform activations is the hallmark of thevalue\_biasbypass: the softmax activations carry almost no per\-input information, and the actual forecasting signal flows through thevalue\_bias×\\timesprototype\_valuechannel that reads directly from the encoder output\.### A\-AProtoTS Structure

ProtoTS is a two\-stage, hierarchically\-prototypical time\-series forecaster that takes the target channel plus a set of context channels as input\.

Stage 1: Encoder pretraining\.Each input channel is first embedded independently — continuous features \(e\.g\., skin temperature\) through an MLP, and discrete features \(e\.g\., hour\-of\-day\) through a lookup table\. The per\-channel embeddings are then passed through a stack of residual MLP\-mixer blocks that mix information along both the time and feature axes\. In this stage, the encoder is trained to produce both the embedding and the forecast curve, forcing it to learn representations that already carry the information needed for the downstream task\.

Stage 2: Prototype layer\.A hierarchical prototype layer is trained jointly with the pretrained encoder\. Each prototype consists of two randomly initialized parts: a*prototype\_embedding*\(the characteristics of the prototype\) and a*prototype\_value*\(the canonical forecast curve of the prototype\)\. An adapter module projects the input window’s embedding into a lower\-dimensional “key”, which is compared against every prototype’s*prototype\_embedding*via softmax routing, and the final forecast is the score\-weighted sum of all*prototype\_value*curves\.

A core design choice of ProtoTS is that the prototypes form a tree\. Training starts with 6–12 root prototypes, and a splitting mechanism is triggered whenever the validation loss stops improving for too long\. At that point, the highest\-error prototypes — ranked by cumulative loss normalized by activation count — each split into several children, so that each child specializes on one aspect of the parent’s more complex scenario\.

### A\-BMotivations from ProtoTS

Building on ProtoTS’s design, our analysis identifies three aspects that motivate the design choices in ConceptTS:

\(1\) Randomly initialized prototypes\.Because prototypes are initialized randomly and shaped by gradient descent, their semantic meaning is not specified by the design itself\. Interpreting a trained prototype typically requires a domain expert to inspect its activation patterns and the corresponding*prototype\_value*shape after training, which can be time\-consuming and sometimes ambiguous\.

\(2\) Encoder that mixes time and feature axes\.ProtoTS’s encoder blends information along both the time and feature axes before the prototype module\. This design is effective for capturing joint temporal\-channel patterns, but it makes it harder to isolate specific cross\-channel semantics — a concept such as“extremely low temperature leads to an electricity\-load spike”would be difficult to recover from a single learned prototype through post\-hoc inspection alone\.

\(3\) Auxiliary path through value\_bias \(most impactful for our design\)\.In addition to the softmax\-normalized activation score — which represents“how similar the input window is to this prototype”— ProtoTS multiplies each prototype’s contribution by a*value\_bias*scalar produced by a separate MLP that reads from the encoder output embedding\. The final forecast is therefore

y^=∑prototypesactivation\_score×value\_bias×prototype\_value\.\\hat\{y\}=\\sum\_\{\\text\{prototypes\}\}\\text\{activation\\\_score\}\\times\\text\{value\\\_bias\}\\times\\text\{prototype\\\_value\}\.In our experiments we observed that, since the encoder is co\-trained with the prototype layer, the model tends to converge toward a configuration in which prototype activation scores become nearly uniform across input windows, while most of the input\-dependent forecasting signal is carried by the*value\_bias×\\timesprototype\_value*channel\. Because*value\_bias*reads directly from the encoder output, this pathway does not preserve the same interpretability structure as the activation channel\. ProtoTS includes an entropy\-based regularization term designed to keep activation scores non\-uniform; we found that its default weight had limited effect on our data, and when we raised the weight enough for the entropy term to actively shape training, forecast accuracy dropped noticeably — suggesting that in this configuration the trained model’s accuracy depends in part on the auxiliary pathway remaining active\.

Fig[6](https://arxiv.org/html/2608.21277#A1.F6)illustrates such problem intuitively\.

### A\-CFrom Prototypes to Concepts

In the early stage of this project we tried to adapt ProtoTS’s structure directly to improve on the three aspects above, but every attempt failed to maintain accuracy while restoring interpretability\. In retrospect, this makes sense: constraining each prototype to a single canonical forecast curve is too restrictive for multivariate time\-series forecasting, where a “prototype” would need to jointly represent a target curve*and*every meaningful cross\-channel condition that could produce it\.

We also considered attaching multiple*prototype\_value*curves to each prototype to increase expressive capacity, but that raises a new problem: how many prototypes are needed to cover every realistic scenario? Even a broad prototype such as“summer night”has many variants along orthogonal axes —“rainy or not”,“off\-work rush hour or late night”,“uncomfortably hot or tolerably hot”— and enumerating every combination scales exponentially\. ProtoTS’s tree\-splitting mechanism partially addresses this by growing new prototypes on demand, but each split adds parameters and complicates training, and the required prototype count still grows combinatorially with the number of feature axes\.

This bottleneck is what led us to the concept\-based framework\. The seemingly unbounded number of prototypes required for a real multivariate task can be replaced by a much smaller set of concepts whose on/off combinations effectively span the same space\. Each concept focuses on one or two feature channels, which makes both training and post\-hoc inspection substantially easier, while expressive capacity remains high: even 20–30 well\-chosen concepts can combine to describe tens of thousands of distinct scenarios — the same coverage prototype\-based methods can only achieve through uncontrolled tree growth\.

## Appendix BConcept Failure Mode Analysis

TABLE V:Per\-concept test\-set AUROC across the three ConceptTS bottlenecks\. Concepts are sorted by AUROC within each bottleneck\. Forecast sub\-window AUROC is averaged over all sub\-window\.Boldentries mark the three concepts with the lowest AUROC in each bottleneck\.ConceptAUROCConceptAUROCLookback concepts \(19\)is\_history\_winter1\.000is\_history\_cold1\.000is\_history\_summer1\.000is\_history\_humid0\.999is\_history\_high\_so20\.999is\_history\_dry0\.998is\_history\_high\_co0\.998is\_history\_high\_ozone0\.998is\_history\_rainy0\.997is\_history\_high\_no20\.994is\_history\_pm25\_rising0\.992is\_history\_low\_ozone0\.992is\_history\_low\_wind0\.987is\_history\_high\_pm250\.985is\_history\_pm25\_volatile0\.985is\_history\_high\_pressure0\.966is\_history\_wind\_anticorrelated0\.855is\_history\_pm25\_noisy\_spectrum0\.797is\_history\_pressure\_rising0\.676Forecast sub\-window concepts \(47\), AUROC averaged over 16 sub\-windowsis\_cold\_block1\.000is\_very\_humid\_block1\.000is\_humid\_block0\.999is\_dry\_block0\.992is\_high\_co\_block0\.989is\_high\_so2\_block0\.988is\_hot\_block0\.987is\_high\_no2\_block0\.982is\_low\_pm10\_block0\.981is\_high\_pm10\_block0\.979is\_low\_so2\_block0\.978is\_winter\_block0\.977is\_low\_co\_block0\.975is\_summer\_block0\.970is\_low\_ozone\_block0\.970is\_low\_no2\_block0\.960is\_high\_pressure\_block0\.959is\_high\_ozone\_block0\.957is\_late\_night\_block0\.957is\_nighttime\_block0\.955is\_daytime\_block0\.951is\_midday\_block0\.951is\_evening\_rush\_block0\.949is\_very\_high\_pressure\_block0\.941is\_low\_pressure\_block0\.931is\_windy\_block0\.925is\_warming\_block0\.924is\_rainy\_block0\.903is\_spring\_block0\.893is\_calm\_block0\.864is\_ozone\_rising\_block0\.848is\_ozone\_falling\_block0\.844is\_very\_calm\_block0\.829is\_moderate\_temp\_block0\.808is\_cooling\_block0\.789is\_autumn\_block0\.783is\_co\_rising\_block0\.769is\_pm10\_rising\_block0\.763is\_humidity\_falling\_block0\.761is\_no2\_rising\_block0\.756is\_pressure\_rising\_block0\.749is\_stable\_wind\_block0\.716is\_humidity\_rising\_block0\.713is\_pressure\_falling\_block0\.704is\_wind\_rising\_block0\.700is\_stable\_temp\_block0\.688is\_wind\_falling\_block0\.684Global\-forecast concepts \(19\)is\_forecast\_cold\_regime1\.000is\_forecast\_humid\_regime1\.000is\_forecast\_dry\_regime0\.999is\_forecast\_high\_pm10\_regime0\.999is\_forecast\_high\_co\_regime0\.999is\_forecast\_high\_ozone\_regime0\.999is\_forecast\_high\_no2\_regime0\.997is\_forecast\_humidity\_rising0\.987is\_forecast\_hot\_regime0\.979is\_forecast\_high\_pressure\_regime0\.967is\_forecast\_wind\_rising0\.896is\_forecast\_has\_rain0\.893is\_forecast\_wind\_falling0\.850is\_forecast\_stable\_pressure0\.784is\_forecast\_cooling0\.743is\_forecast\_temp\_large\_swing0\.713is\_forecast\_pressure\_falling0\.704is\_forecast\_pressure\_rising0\.667is\_forecast\_warming0\.599Table[V](https://arxiv.org/html/2608.21277#A2.T5)reports all concepts proposed by the LLM proposer and their test\-set AUROC\. AUROC measures the probability that a randomly chosen positive example is assigned a higher predicted probability than a randomly chosen negative example, and ranges from0\.50\.5\(random guessing\) to1\.01\.0\(perfect separation\)\. In each of the three bottlenecks, the three concepts with the lowest AUROC are marked using bold texts\.

A clear pattern emerges: the model consistently struggles with concepts that describe*temporal trends or fluctuation patterns*\. Concepts such asis\_forecast\_warming,is\_forecast\_pressure\_rising, andis\_stable\_temp\_blockall fall well below the AUROC of level\-based concepts \(e\.g\.,is\_history\_dry,is\_cold\_block, which routinely score above0\.90\.9\)\.

We attribute this failure mode to a fundamental limitation of our concept\-labeling pipeline\. Recall that the ground\-truth binary label for each concept is produced at LLM\-proposal time by applying a Python predicate over per\-segment summary statistics \(mean, standard deviation, slope, range, etc\.\)\. While these statistics are sufficient for describing concepts based on absolute levels or spreads \(e\.g\., “*cold window*”, “*high humidity*”\), they cannot cleanly express the temporal shape of a curve — whether it monotonically rises, monotonically falls, fluctuates around a mean, or follows a more complex trajectory such as a rise\-then\-fall pattern\. The ground\-truth labels for these shape\-dependent concepts are therefore internally inconsistent, giving the model a noisier and more challenging learning target than for the statistically well\-defined concepts\.

A natural remedy, which we leave to future work, is to enrich the per\-segment statistical summary with shape descriptors so that the LLM\-generated Python predicates can express shape\-based conditions as precisely as level\-based ones\.

## Appendix CModernTCN Encoder Structure

Each ModernTCN block consists of the following modules:

- •𝐃𝐖𝐂𝐨𝐧𝐯𝐨𝐥𝐮𝐭𝐢𝐨𝐧\\mathbf\{DWConvolution\}: the tensor is reshaped to\(B,C×D,N\)\(B,C\\times D,N\), and a depthwise convolution layerConv1D​\(𝐶𝐷,𝐶𝐷,kernel=κ,padding=κ/2,groups=𝐶𝐷\)\\text\{Conv1D\}\(\\mathit\{CD\},\\mathit\{CD\},\\text\{kernel\}=\\kappa,\\text\{padding\}=\\kappa/2,\\text\{groups\}=\\mathit\{CD\}\)is applied with a large kernel sizeκ\\kappa\(κ\\kappa= 51 in our setting\), performing a temporal mixing per \(variable, feature\)\. This step aims to capture the temporal patterns and also preserves the semantic meaning\.
- •𝐏𝐖𝐂𝐨𝐧𝐯𝐨𝐥𝐮𝐭𝐢𝐨𝐧𝟏\\mathbf\{PWConvolution\_\{1\}\}: a grouped pointwise inverted\-bottleneckConv1D​\(𝐶𝐷,r⋅𝐶𝐷,1,groups=C\)→GELU→Conv1D​\(r⋅𝐶𝐷,𝐶𝐷,1,groups=C\)\\text\{Conv1D\}\(\\mathit\{CD\},r\\cdot\\mathit\{CD\},1,\\text\{groups\}=C\)\\rightarrow\\text\{GELU\}\\rightarrow\\text\{Conv1D\}\(r\\cdot\\mathit\{CD\},\\mathit\{CD\},1,\\text\{groups\}=C\), aiming to mix information along feature axis on a per\-variable basis
- •𝐏𝐖𝐂𝐨𝐧𝐯𝐨𝐥𝐮𝐭𝐢𝐨𝐧𝟐\\mathbf\{PWConvolution\_\{2\}\}: a grouped pointwise inverted\-bottleneckConv1D​\(𝐷𝐶,r⋅𝐷𝐶,1,groups=D\)→GELU→Conv1D​\(r⋅𝐷𝐶,𝐷𝐶,1,groups=D\)\\text\{Conv1D\}\(\\mathit\{DC\},r\\cdot\\mathit\{DC\},1,\\text\{groups\}=D\)\\rightarrow\\text\{GELU\}\\rightarrow\\text\{Conv1D\}\(r\\cdot\\mathit\{DC\},\\mathit\{DC\},1,\\text\{groups\}=D\), aiming to mix information along variable axis on a per\-feature basis

## Appendix DLoss Function Composition

We describe each of the seven loss terms in Eq\. \(8\) below:

##### Forecast Loss\.

ℒforecast\\mathcal\{L\}\_\{\\mathrm\{forecast\}\}is the meanL1L\_\{1\}error between𝐘^L\+1:L\+H\\widehat\{\\mathbf\{Y\}\}\_\{L\+1:L\+H\}and𝐘L\+1:L\+H\\mathbf\{Y\}\_\{L\+1:L\+H\}on the z\-scored target\. This is the standard loss term that drives the model’s forecast curve towards the ground\-truth target\.

##### Concept Loss\.

ℒconceptlb\\mathcal\{L\}\_\{\\mathrm\{concept\}\}^\{\\mathrm\{lb\}\},ℒconceptfc\\mathcal\{L\}\_\{\\mathrm\{concept\}\}^\{\\mathrm\{fc\}\}, andℒconceptgfc\\mathcal\{L\}\_\{\\mathrm\{concept\}\}^\{\\mathrm\{gfc\}\}denote the concept losses for the look\-back, forecast sub\-window, and global forecast bottlenecks, respectively\. Each is the binary cross\-entropy between the predicted concept logits and the LLM\-derived binary labels, averaged over concepts \(and, for the sub\-window bottleneck, over theNswN\_\{\\mathrm\{sw\}\}sub\-window positions\)\.

##### Boundary Smoothness Loss\.

Since the decoder predicts the target sub\-window by sub\-window and concatenates the results, the transitions at sub\-window boundaries can be discontinuous\. We penalize theL1L\_\{1\}difference between adjacent forecast timesteps at theNsw−1N\_\{\\mathrm\{sw\}\}\-1boundary positions:

ℒboundary=1Nsw−1​∑k=1Nsw−1\|y^L\+k​Tsw−y^L\+k​Tsw−1\|\.\\mathcal\{L\}\_\{\\mathrm\{boundary\}\}=\\frac\{1\}\{N\_\{\\mathrm\{sw\}\}\-1\}\\sum\_\{k=1\}^\{N\_\{\\mathrm\{sw\}\}\-1\}\\left\|\\hat\{y\}\_\{L\+kT\_\{\\mathrm\{sw\}\}\}\-\\hat\{y\}\_\{L\+kT\_\{\\mathrm\{sw\}\}\-1\}\\right\|\.\(9\)

##### Concept Collapse Loss\.

The CEM gate𝐜^c=pc​𝐞c\+\+\(1−pc\)​𝐞c−\\hat\{\\mathbf\{c\}\}\_\{c\}=p\_\{c\}\\,\\mathbf\{e\}\_\{c\}^\{\+\}\+\(1\-p\_\{c\}\)\\,\\mathbf\{e\}\_\{c\}^\{\-\}degenerates ifpcp\_\{c\}becomes near\-constant across the batch: the resulting𝐜^c\\hat\{\\mathbf\{c\}\}\_\{c\}is nearly identical for every input, so the concept carries no per\-sample information and the decoder cannot use it\. Although the concept losses already penalize wrong probability assignments, we add an explicit variance\-based penalty to prevent this collapse:

ℒcollapse=∑cmax⁡\(0,σmin−Varbatch​\(pc\)\),\\mathcal\{L\}\_\{\\mathrm\{collapse\}\}=\\sum\_\{c\}\\max\\\!\\left\(0,\\,\\sigma\_\{\\min\}\-\\mathrm\{Var\}\_\{\\mathrm\{batch\}\}\(p\_\{c\}\)\\right\),\(10\)whereσmin\\sigma\_\{\\min\}is the minimum variance threshold\. Any concept whose batch\-wise activation variance falls belowσmin\\sigma\_\{\\min\}incurs a positive penalty proportional to how far below the threshold it is\.

##### Residual Regularization Loss\.

As discussed in the model architecture section, a low\-dimensional residual channel is added alongside the concept bottleneck to compensate for the loss of expressive capacity introduced by the static concept embeddings\. To prevent the decoder from relying too heavily on this channel and bypassing the concept bottleneck, we add anL2L\_\{2\}penalty on the residual vector:

ℒresidual=‖𝐫‖22,\\mathcal\{L\}\_\{\\mathrm\{residual\}\}=\\left\\lVert\\mathbf\{r\}\\right\\rVert\_\{2\}^\{\\,2\},\(11\)where𝐫\\mathbf\{r\}is the residual channel output\. The weightαres\\alpha\_\{\\mathrm\{res\}\}directly controls the trade\-off between forecast accuracy and interpretability\.

## Appendix EExperimental Setup Details

This appendix specifies the details in all the experiments introduced in Experiment Section\.

### E\-ADataset and Chronological Split

We use the Beijing Multi\-Site Air Quality dataset\[[3](https://arxiv.org/html/2608.21277#bib.bib20)\]at three monitoring stations:Aotizhongxin,Dingling, andTiantan\. Each station is treated as an independent dataset and the model is trained separately per station\. The raw data spans 1 March 2013 through 28 February 2017 at hourly resolution\.

We adopt a strictly chronological split to prevent leakage from future to past, with training set, validation set, and test set each containing 70%, 10%, and 20% of the data\. Missing values are filled via linear interpolation, applied across the full data before splitting\.

Global normalization statistics \(μ\\mu,σ\\sigma\) used for z\-scoring the target and covariates are estimated only from the training split\. The per\-cluster statistics used to condition the LLM proposer and the concept positive\-rate filter are also computed on the training split only\.

Sliding windows are extracted with look\-back lengthL=96L=96hours and forecast horizonH=96H=96hours, using a stride of 6 hours\.

Although training uses a sliding\-window stride of 1 hour, the LLM concept\-labeling predicates are evaluated at a coarser stride of 6 hours to make the entire concept proposing stage more efficient\. Concretely, for any group of six consecutive training segments whose look\-back windows begin within the same 6\-hour block, we compute the binary concept labels once and reuse them across all six segments\. This design has two motivations\. First, it reduces the number of predicate evaluations by6×6\\times, keeping the offline labeling pass tractable when the training set contains hundreds of thousands of windows\. Second, and more importantly, it matches the natural temporal coherence of the proposed concepts: most of them \(e\.g\.,is\_history\_cold,is\_forecast\_humid,is\_winter\_block\) describe conditions that persist for at least several hours, so their true binary states rarely flip within a 6\-hour window\.

### E\-BModel Architecture Hyperparameters

- •Encoder\(ModernTCN\-style\): per\-channel embedding dimensionD=32D=32; patchification patch sizeP=4P=4and strideS=4S=4; depthwise\-convolution kernel sizeκ=51\\kappa=51; ConvFFN inverted\-bottleneck expansion ratior=2r=2; number of ModernTCN blocks=4=4; post\-projection output embedding dimensionDenc=128D\_\{\\mathrm\{enc\}\}=128\. Two independent encoders with identical hyperparameters are used, one over the look\-back window and one over the forecast horizon\.
- •Attention pool: additive\-attention MLP hidden dimension=32=32\.
- •CEM bottlenecks: concept embedding dimensiondc=16d\_\{c\}=16; each concept’s scoring head is a single linear projection \(no hidden layer\) fromDencD\_\{\\mathrm\{enc\}\}to a scalar logit, followed by a sigmoid\. Concept counts per bottleneck used in the main results are: lookbackKlb=20K\_\{\\mathrm\{lb\}\}=20; forecast sub\-windowKsw=50K\_\{\\mathrm\{sw\}\}=50\(separate predictor weights per sub\-window, shared concept embeddings across sub\-windows\); global forecastKgfc=20K\_\{\\mathrm\{gfc\}\}=20\.
- •BiGRU: input summary dimensiondsumm=64d\_\{\\mathrm\{summ\}\}=64; hidden dimension per direction=32=32\(concatenated output dimension=64=64\); number of layers=1=1\.
- •Sub\-decoder:Bdec=3B\_\{\\mathrm\{dec\}\}=3residual MLP blocks with feedback of the running forecast into blocksb\>1b\>1; per\-block MLP hidden dimension=512=512; sub\-window positional embedding dimensiondid=16d\_\{\\mathrm\{id\}\}=16; residual channel dimension=4=4, projected from the look\-back encoder’s global\-pooled feature by a single learned linear layer and concatenated into the decoder input\.

### E\-CLoss Weights and Training Procedure

The seven weighting coefficients in Eq\. \(8\) are set as follows:αf=1\.0\\alpha\_\{f\}=1\.0,αlb=1\.0\\alpha\_\{\\mathrm\{lb\}\}=1\.0,αfc=1\.0\\alpha\_\{\\mathrm\{fc\}\}=1\.0,αgfc=1\.0\\alpha\_\{\\mathrm\{gfc\}\}=1\.0,αbd=0\.01\\alpha\_\{\\mathrm\{bd\}\}=0\.01,αcc=0\.05\\alpha\_\{\\mathrm\{cc\}\}=0\.05\(with variance thresholdσmin=0\.05\\sigma\_\{\\min\}=0\.05\), andαres=0\.01\\alpha\_\{\\mathrm\{res\}\}=0\.01\(default; swept over\{0\.01,0\.1,1\.0\}\\\{0\.01,0\.1,1\.0\\\}in Table III\)\. All runs use the Adam optimizer with initial learning rate3×10−43\\times 10^\{\-4\}, weight decay1×10−41\\times 10^\{\-4\}, batch size1616, and no learning\-rate scheduler \(constant learning rate throughout training\)\. Models are trained for up to6060epochs with early stopping on validation MAE \(patience=5=5epochs\); the checkpoint with the lowest validation MAE is reported\.

### E\-DBaseline Implementations

Time\-Series\-Library baselines \(Informer, iTransformer, TimeXer, Crossformer, DLinear\) use the official implementations from Time\-Series\-Library with their recommended default hyperparameters, adjusted only for our look\-back lengthL=96L=96and forecast horizonH=96H=96\. Tree\-based baselines \(Random Forest, XGBoost, LightGBM\) use the scikit\-learn, xgboost, and lightgbm library\. TFT, DeepAR, and NHiT use the PyTorch Forecasting implementation\. LSTM is a 2\-layer, hidden dimension 128 baseline implemented from scratch\. All baselines are given identical inputs to ConceptTS in each setting: full future covariates in the future\-aware setting, look\-back inputs only in the look\-back\-only setting, in order to make the comparison fair\.

### E\-EExperimental Configuration

For the main forecasting\-performance experiments, we set the number of sub\-windows toNsw=24N\_\{\\mathrm\{sw\}\}=24, which we found offers the best trade\-off between forecast accuracy and model parameter count\. At each station, every model \(ConceptTS and all baselines\) is trained three times with different random seeds, so we can report both mean performance and run\-to\-run stability\. Concept proposals are generated once per station, and the same concept sets are reused across all three ConceptTS training runs at that station\.

For the residual\-weight ablation \(αres\\alpha\_\{\\mathrm\{res\}\}\), we fix both the concept set and the three random seeds\. This ensures that any variation across values ofαres\\alpha\_\{\\mathrm\{res\}\}reflects the ablation itself rather than concept\-proposal or seed noise\.

For the case study subsection, we reduce the number of sub\-windows toNsw=16N\_\{\\mathrm\{sw\}\}=16\. Fewer sub\-windows means each sub\-window covers more timesteps, which makes the per\-sub\-window forecast curves and concept activations easier to read and analyze\.

## Appendix FAdditional Ablation Studies

![Refer to caption](https://arxiv.org/html/2608.21277v1/subwindow_vs_mae.png)\(a\)MAE
![Refer to caption](https://arxiv.org/html/2608.21277v1/subwindow_vs_rmse.png)\(b\)RMSE

Fig\. 7:Effect of the number of forecast sub\-windows on forecasting performance\. Both metrics decrease monotonically as the forecast horizon is partitioned into more sub\-windows, with diminishing returns beyondNsw=16N\_\{\\mathrm\{sw\}\}=16\.### F\-AConcept Bottleneck Contribution

To understand how much the forecast depends on each concept bottleneck, we design an ablation that measures the contribution of the three bottlenecks—the lookback bottleneck, the forecast sub\-window bottleneck, and the global forecast bottleneck— directly from a single trained model\. We first train the model in the standard setting, then evaluate it on the test set four times: once in the normal configuration, and three additional times, each of which “blocks out” one bottleneck\. To block out a bottleneck, we replace the activation score of every concept in it with that concept’s mean activation over the test set\. Since these values are constant across inputs, the corresponding concept context no longer carries any input\-dependent information, so the bottleneck is effectively removed from the forecast while the rest of the model, including the other two bottlenecks, is left unchanged\. Using the per\-concept mean rather than an arbitrary constant such as zero keeps the decoder input within its normal range, so the measured change in accuracy reflects the loss of concept information rather than the effect of feeding the model an unusual input\.

We adopt this test\-time procedure, instead of retraining the model with a bottleneck removed each time, for a specific reason\. If a bottleneck were removed before training, the model could compensate by re\-routing the same information through the remaining concepts, which would mask the true role of the removed bottleneck\. By intervening only at test time on a fixed trained model, we avoid such compensation and obtain a direct estimate of how much each bottleneck contributes to the final prediction\. We report the test error under each configuration in Table[VI](https://arxiv.org/html/2608.21277#A6.T6): a large increase in error when a bottleneck is blocked indicates that the model relies heavily on it, whereas a small change indicates that the model’s forecast doesn’t get influenced significantly by it\.

TABLE VI:Ablation study of the three concept bottlenecks in ConceptTS on Aotizhongxin\.Δ\\DeltaMAE reports the absolute performance degradation, in terms of MAE, when compared to the full model\. All values are inμ​g/m3\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}\.The results reveal a clear ordering of importance among the three bottlenecks\. Blocking the lookback bottleneck degrades the forecast only marginally \(↓0\.037​μ​g/m3\\downarrow 0\.037\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}MAE\), removing the global forecast bottleneck causes a larger but still moderate drop \(↓1\.412​μ​g/m3\\downarrow 1\.412\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}MAE\), and blocking the forecast sub\-window bottleneck is by far the most damaging \(↓43\.193​μ​g/m3\\downarrow 43\.193\\mu\\mathrm\{g\}/\\mathrm\{m\}^\{3\}MAE\), leaving the model with almost no forecasting ability\.

This ordering is consistent with our setting and the nature of the task\. Because we adopt a future\-aware configuration, the model has access to the true covariates over the forecast window, and therefore predicts mainly from what is known about the future rather than from the past\. This effect is reinforced by the fact that PM2\.5 lacks the strong, regular periodicity of series such as electricity load, so the recent history is only a weak predictor of future values; the lookback concepts are thus used merely as auxiliary information that slightly refines the forecast, which explains their small contribution\. The dominant contribution comes from the per\-sub\-window concept activations: these concepts describe the conditions within each individual sub\-window, and the model needs such fine\-grained, temporally resolved information to generate the forecast at every step of the horizon\. The global forecast bottleneck plays an intermediate role — rather than resolving each sub\-window, it summarizes the overall regime of the entire forecast window and provides a coarse context that refines the prediction on top of the sub\-window details, which is consistent with its moderate but non\-negligible effect\.

### F\-BNumber of Sub\-windows

We further analyze how the number of forecast sub\-windows,NswN\_\{\\mathrm\{sw\}\}, affects the overall performance of ConceptTS\. To isolate the effect of this hyperparameter, we disable the residual channel so that all forecasting must flow through the concept bottlenecks\. To keep the comparison fair, we reuse the same LLM\-proposed concept sets \(look\-back, global forecast, and forecast sub\-window\) across every training run, and for each value ofNswN\_\{\\mathrm\{sw\}\}we train the model twice with different random seeds and report the average\.

The results are shown in Fig\.[7](https://arxiv.org/html/2608.21277#A6.F7)\. Both MAE and RMSE decrease monotonically asNswN\_\{\\mathrm\{sw\}\}increases: MAE drops from25\.1125\.11to15\.00​μ​g/m315\.00\\ \\mu\\mathrm\{g/m\}^\{3\}asNswN\_\{\\mathrm\{sw\}\}grows from 2 to 32, and RMSE drops from41\.3741\.37to24\.03​μ​g/m324\.03\\ \\mu\\mathrm\{g/m\}^\{3\}over the same range\. This trend is intuitive — a largerNswN\_\{\\mathrm\{sw\}\}shortens each sub\-window, which lets the corresponding forecast sub\-window concepts describe finer\-grained local trends and events\. However, the improvement is not free: the number of concept\-scoring heads and CEM embeddings grows linearly withNswN\_\{\\mathrm\{sw\}\}, increasing both training time and total parameter count\. Based on this trade\-off, we setNsw=24N\_\{\\mathrm\{sw\}\}=24in our main performance experiments, which sits in the region where the accuracy curve has largely flattened but the parameter count remains manageable\.

## Appendix GLLM Concept\-Proposal Prompt

This appendix reproduces the full prompt template that ConceptTS sends to the concept\-proposing LLM, instantiated for the Beijing PM2\.5 forecasting task used throughout the paper\. Task\-dependent portions are marked with double braces — e\.g\.,`\{\{input\_length\}\}`,`\{\{n\_subwindows\}\}`,`\{\{channel\_descriptions\}\}`— and are substituted at runtime with the concrete values used for a given task\. The template is sent verbatim to the LLM once per proposal call; the JSON output is then parsed into executable Python predicates that label every training segment\.

Youareadomainexpertinurbanair\-qualityforecasting

\(PM2\.5\)forBeijing\.ThedominantdriversofPM2\.5

concentrationinthisregionarelocalemissionsplus

meteorologicalconditionsthateitherdisperseortrap

pollutants\-\-lowwindspeed\(nohorizontaldispersion\),

highpressureandinversions\(noverticalmixing\),high

humidity\(hygroscopicparticlegrowth\),precipitation

\(wetscavenging\),andchemicalinteractionswithco\-

emittedpollutants\(PM10,SO2,NO2,CO,O3\)\.Youare

helpingdesignaCONCEPTBOTTLENECKMODELfor

interpretablePM2\.5air\-qualityforecasting\.

Themodelsupportstwotrainingmodes,bothusinga

singlesharedLOOKBACKbottleneckbutwithdifferent

sub\-windowconceptsets:

\*FUTURE\-AWAREmodeuses2\+Nbottlenecks:

\-1LOOKBACKbottleneck\(K\_lbconceptssummarising

thewholehistorywindow\)\.

\-NFORECAST\-SUB\-WINDOWbottlenecks:K\_fcconcepts

appliedateachoftheNforecastsub\-windows,

sharingthesamestaticconceptembeddingsbut

withper\-sub\-windowpredictorheads\.

\-1GLOBAL\-FORECASTbottleneck\(K\_gfcconcepts

spanningthewholeforecasthorizonasasingle

unit\)\.

\*LOOK\-BACK\-ONLYmodeuses1\+N\_lb\_swbottlenecks:

\-1LOOKBACKbottleneck\(sameasabove\)\.

\-N\_lb\_swLOOKBACK\-SUB\-WINDOWbottlenecks:K\_lb\_sw

conceptsappliedateachoftheN\_lb\_swlookback

sub\-windows\.

YouwillproposeALLFOURconceptsetsinasingleJSON

response\.Eachtrainingrunuseseithertheforecast

side\(FUTURE\-AWAREmode\)ORthelookback\-sub\-window

side\(LOOK\-BACK\-ONLYmode\);thewhole\-historyLOOKBACK

setisalwaysused\.Emittingallfourinoneshotmeans

oneLLMrunsupportsbothtrainingmodes\.

\#\#TaskContext

\-Dataset:\{\{dataset\_name\}\}

\-Target:\{\{target\_variable\}\}\(\{\{target\_description\}\}\)

\-Timeresolution:\{\{time\_resolution\}\}

\-Lookbackwindow:\{\{input\_length\}\}steps

\-Forecastwindow:\{\{output\_length\}\}stepssplitinto

\{\{n\_subwindows\}\}sub\-windowsof\{\{subwindow\_length\}\}

stepseach

\-Totaltrainingsegments:\{\{total\_segments\}\}

\#\#AvailableFeatureChannels

\{\{channel\_descriptions\}\}

\#\#GlobalStatistics\(acrossalltrainingsegments\)

\{\{global\_statistics\}\}

\#\#ClusterSummaries\(\{\{n\_clusters\}\}K\-meansregimes\)

Usethesetopickthresholdsthatactuallysplitthe

dataintotheregimesyouseebelow\.

\{\{cluster\_summaries\}\}

\#\#Schema1\-\-segmentdict\(forLOOKBACKconcepts\)

defmatches\(seg\)\-\>boolreceivesadictwith:

\-lookback\_stats:per\-channel\{mean,median,std,

min,max,range,skewness,kurtosis,slope,

delta\_half,volume\}plusFFT\-derived

\{fft\_dom\_period\_h,fft\_dom\_amp,fft\_spec\_entropy,

fft\_lowfreq\_ratio,fft\_amp\_p\{24,12,8,6\}h\}\.

\-cross\_channel\_stats:target\_corr\_\{PM10,WSPM,

TEMP,DEWP,PRES,RAIN\}\.

\-time\_context:lookback\_start/lookback\_end/

season\.

\#\#Schema2\-\-sub\-windowdict\(forFORECASTand

\#\#LOOKBACK\-SUB\-WINDOWconcepts\)

defmatches\(sub\_seg\)\-\>boolreceivesadictwith:

\-subwindow\_idx:int\(0\.\.N\-1\)\.

\-stats:per\-channelshapeidenticalto

lookback\_statsbutWITHOUTFFTfields\.For

FORECAST\-SUB\-WINDOWandFORECAST\-GLOBALconcepts,

PM2\.5isMASKEDandMUSTNOTbereferenced;

LOOKBACK\-SUB\-WINDOWconceptsMAYusePM2\.5

becausethelookbackhistoryisobserved\.

\-time\_context:subwindow\_idx,start,end,season\.

\-cross\_channel\_stats:inheritedfromtheparent

segment\.

Per\-sub\-windowconcepts\(FORECAST\-SUB\-WINDOWand

LOOKBACK\-SUB\-WINDOW\)fire\{\{n\_subwindows\}\}timesper

segment,oncepersub\-window\.GLOBAL\-FORECAST

conceptsfireONCEpersegment;theirsub\_seg\.stats

andsub\_seg\.time\_contextcovertheWHOLE

\{\{output\_length\}\}\-stepforecastwindowratherthana

singlesub\-window\.

\#\#YourTask

\*\*\*CRITICALCONSTRAINT\*\*\*

PM2\.5ISTHEPREDICTIONTARGET\.Theforecastwindow’s

PM2\.5valuesaremasked/unavailableatinferencetime\.

\-LOOKBACKandLOOKBACK\-SUB\-WINDOWconceptsMAY

referencePM2\.5\(thelookbackhistoryisobserved\)\.

\-FORECAST\-SUB\-WINDOWconceptsMUSTNOTreference

PM2\.5\.

\-FORECAST\-GLOBALconceptsMUSTNOTreferencePM2\.5\.

Forecast\-sideconceptsmustbebuiltEXCLUSIVELYfrom

meteorology\(TEMP,PRES,DEWP,RAIN,WSPM\),other

pollutants\(PM10,SO2,NO2,CO,O3\),andcalendar

\(year,month,day,hour,season\)\.

ProposeFOURconceptsets:

\*Exactly\{\{n\_lookback\_concepts\}\}LOOKBACKconcepts\.

\*Exactly\{\{n\_lookback\_sub\_window\_concepts\}\}

LOOKBACK\-SUB\-WINDOWconcepts\(eachrulefires

\{\{n\_lookback\_subwindows\}\}timespersegment,once

perlookbacksub\-window\)\.

\*Exactly\{\{n\_forecast\_concepts\}\}FORECAST\-SUB\-WINDOW

concepts\(eachrulefires\{\{n\_subwindows\}\}times

persegment,onceperforecastsub\-window\)\.

\*Exactly\{\{n\_forecast\_global\_concepts\}\}

FORECAST\-GLOBALconcepts\(eachrulefiresONCEper

segmentonthewhole\{\{output\_length\}\}\-step

horizon\)\.

Hardrequirementsforeveryfunction:

1\.Namedexactly‘matches‘\.Singlepositionalarg\.

2\.ATOMIC:bodyisonereturnstatementoverasingle

booleanexpression;helperlookupsallowed\.No

loops,noif/else\.

3\.Defensive\.get\(\.\.\.\)access;missingkeysmust

returnFalse\.

4\.Noimports\.StandardPythononly\.

5\.Targetpositiveratein\[5%,60%\]acrossthe

\{\{total\_segments\}\}segments\(oversegment\*sub\-

windowpairsforsub\-windowrules\)\.

6\.Diversecategories:season,time\_of\_day,calendar,

weather\_temperature,weather\_other,

weather\_dynamics,pollutant\_chemistry,

cross\_channel,regime,frequency\(lookbackonly\),

aggregate/threshold/persistence/trend\(good

fortheFORECAST\-GLOBALset\),misc\.

7\.Nocompounds\(theCBMcombinesconceptsatthe

embeddinglevel\)\.

8\.NOTARGETLEAKAGE:forecast\-siderulesmustnot

touchPM2\.5inanyway\(name,description,or

code\);suchruleswillbeREJECTED\.

9\.MANDATORYSELF\-CHECK:scaneveryforecast\-side

entrybeforeemittingJSON;deleteandreplace

anyrulethatreferencesPM2\.5\.

\#\#Sub\-windowtrendrules

Shortsub\-windows\(\{\{subwindow\_hours\}\}hours\)have

systematically~0OLSslopeforcyclicchannels\(TEMP,

DEWP,wind,pollutants\)\.For"isXrising/falling

inthisblock"concepts\(bothFORECAST\-SUB\-WINDOWand

LOOKBACK\-SUB\-WINDOW\),usestats\[X\]\[’delta\_half’\]

\(mean\(secondhalf\)\-mean\(firsthalf\)\)insteadof

stats\[X\]\[’slope’\]\.UseslopeonlyfortheLOOKBACK

bottleneck\(whole96\-hourwindow\)andFORECAST\-GLOBAL

concepts\(wholehorizon\)\.

\#\#OutputFormat

ReturnasingleJSONobjectoftheshape:

\{

"lookback\_concepts":

\[\.\.\.\{\{n\_lookback\_concepts\}\}entries\.\.\.\],

"lookback\_sub\_window\_concepts":

\[\.\.\.entries\.\.\.\],

"forecast\_concepts":

\[\.\.\.entries\.\.\.\],

"forecast\_global\_concepts":

\[\.\.\.entries\.\.\.\]

\}

Eachentryhasfields\{id,name,description,category,

function\},where"function"istheexecutablePython

sourcefor‘defmatches\(\.\.\.\)‘\.IDsrun0\.\.K\-1

independentlyperarray\.ReturnONLYtheJSONobject,

noprosebeforeorafter\.

## Appendix HTest on Electricity Load Dataset

TABLE VII:Forecasting performance on LOF\-PC dataset\. Values are computed on thezz\-scored target \(dimensionless\)We also evaluate our model on the LOF dataset, an electric load forecasting dataset released by the ProtoTS authors\. LOF contains electricity consumption records from four regions of a northern Chinese province at 15\-minute resolution, together with 22 exogenous features covering weather measurements, temporal indicators, and calendar information\. We train ConceptTS on the regionPCand compare it against the same set of baseline models used in the experiment section\. Following the original ProtoTS paper, we use the future\-aware setting so that all models have access to the future exogenous features at inference time\.

Table[VII](https://arxiv.org/html/2608.21277#A8.T7)reports the results\. Unlike the Beijing PM2\.5 task, ConceptTS trails every baseline model on LOF\. We attribute this drop to two factors\. First, the LOF dataset shows a noticeable distribution shift between the training and testing sets, and our model handles this shift less gracefully than the tree\-based and Transformer baselines\. Second, and more importantly, electricity load has strong, clear temporal patterns on both daily and yearly cycles, and our concept bottleneck design is not well suited to capture such long\-term periodicities\. Every concept in our vocabulary describes short\-term local behavior \(e\.g\.,“the sub\-window is in the evening peak”,“the last hour was hot”\), and none of them can directly express a seasonal or annual pattern\. This also explains why ConceptTS remains competitive on the PM2\.5 task: air quality forecasting depends mainly on short\-term cross\-channel context — wind speed, other pollutant concentrations, dew point — which is exactly what our concept vocabulary is designed to expose\. Such scope limitation — concepts describing only short\-term local behavior — is a genuine constraint of our current design\. Consequently, equipping ConceptTS with concepts at longer temporal scales \(daily, seasonal, annual\) is a natural direction for future work\.