面向时间序列决策的系统一模型基准评测：Jev 案例研究

# Benchmarking System One Models for Time-Series Decision Making: A Case Study of Jev
Source: [https://arxiv.org/html/2610.33410](https://arxiv.org/html/2610.33410)
Kexin ZhangJiayin XiongHao Xue††thanks:Corresponding author\.Affiliation:The Hong Kong University of Science and Technology \(Guangzhou\)

###### Abstract

Time\-series tasks require both accurate predictions and fast decisions\. Advances in large models have introduced decision\-oriented models that directly output answers and their probabilities, which we call Structured System 1 models\. Their applicability to time\-series tasks remains underexplored\. Using Jev 1\.13\.0 as a case study, we evaluate time\-series classification, event type prediction, oil temperature direction and stock movement prediction, converting continuous forecasting targets into discrete decisions to fit the model interface\. We establish a unified evaluation framework using accuracy and AUC\. Results show stronger performance on selected explicit temporal patterns and event type prediction, but near\-chance discrimination on many numerical sequence classification tasks and stock movement prediction\. Some datasets also reveal a gap between ranking quality and final decision accuracy\. These findings motivate improvements in temporal representation, domain adaptation, and decision\-score alignment\. Standardized sample construction, model calls, and result recording support extending the evaluation to future models\.

## 1Introduction

Time series record how a system changes over time and support applications across diverse domains: wearable sensors enable human activity recognition\[[19](https://arxiv.org/html/2610.33410#bib.bib19)\], physiological measurements support patient assessment and deterioration prediction\[[12](https://arxiv.org/html/2610.33410#bib.bib12)\], operational signals enable equipment fault detection and diagnosis\[[16](https://arxiv.org/html/2610.33410#bib.bib16)\], and financial observations support market movement prediction\[[18](https://arxiv.org/html/2610.33410#bib.bib18)\]\. These applications require time\-series analysis to balance accuracy and processing speed\. Traditional machine learning methods, including support vector machines, random forests, and XGBoost\[[7](https://arxiv.org/html/2610.33410#bib.bib7),[5](https://arxiv.org/html/2610.33410#bib.bib5),[6](https://arxiv.org/html/2610.33410#bib.bib6)\], are typically trained using task\-specific features, and their predictive performance depends on feature representations and training data\.

Recent advances in large language models and time\-series foundation models have introduced new approaches to time\-series analysis\. Time\-LLM maps temporal patches into the representation space of a frozen language model, Chronos quantizes time\-series values for pretraining across domains, and Chronos\-2 further supports zero\-shot multivariate and covariate\-informed forecasting\[[14](https://arxiv.org/html/2610.33410#bib.bib14),[2](https://arxiv.org/html/2610.33410#bib.bib2),[3](https://arxiv.org/html/2610.33410#bib.bib3)\]\. Some studies use natural\-language instructions to guide models in making time\-series predictions\[[17](https://arxiv.org/html/2610.33410#bib.bib17)\]\.

Although complex temporal dependencies may require stronger modeling and reasoning capabilities, many time\-series tasks can be naturally formulated as direct decision problems rather than continuous\-valued forecasting\. A unified, direct, and structured decision\-making approach also offers potential advantages in computational efficiency and reduced generation overhead\. These considerations motivate evaluating whether Structured System 1 can effectively interpret time\-series information and produce reliable decisions\. Inspired by dual\-process theories in psychology, related studies use System 1 to describe fast, direct judgments and System 2 to describe deliberative reasoning involving multiple steps\[[11](https://arxiv.org/html/2610.33410#bib.bib11),[25](https://arxiv.org/html/2610.33410#bib.bib25)\], as shown in Figure[1](https://arxiv.org/html/2610.33410#S1.F1)\(a\)\.

Existing studies differ in how they distinguish System 1 from System 2\.*Distilling System 2 into System 1*categorizes Chain\-of\-Thought \(CoT\) and other methods that generate intermediate reasoning steps as System 2\[[25](https://arxiv.org/html/2610.33410#bib.bib25)\]\. The time\-series forecasting benchmark ReC4TS instead treats CoT, self\-consistency, and self\-correction as “System 1 reasoning enhancements,” while categorizing models that undergo dedicated reasoning post\-training as System 2\[[17](https://arxiv.org/html/2610.33410#bib.bib17)\]\.

This paper adopts an operational distinction based on the inference procedure\. As shown in Figure[1](https://arxiv.org/html/2610.33410#S1.F1)\(b\), System 1 produces final answers without generating an intermediate reasoning sequence and is divided into*Structured System 1*and*Generative System 1*according to the output mechanism\. The former directly computes decisions, while the latter generates answers through text decoding, with implications for response speed and output stability\. System 2 uses additional inference\-time computation for intermediate derivation, search, or reflection through*Built\-in Reasoning*or*Test\-Time Enhanced*approaches, which can be combined\. Existing time\-series research has focused primarily on System 2 and Generative System 1, with fewer systematic evaluations of Structured System 1\.

Structured System 1 models score predefined alternatives, labels, or ordered levels conditioned on a state, a natural\-language question, and candidate answers, returning decisions and probabilities through a unified interface\. This output format supports time\-series classification, anomaly detection, and discretized forecasting\. A typical implementation uses an encoder to obtain contextual representations and a classification or scoring head to map them into the target output space, reducing sequential decoding and answer\-parsing overhead\. BERT combines a pretrained encoder with task\-specific output layers for classification and natural language inference, while DistilBERT explores smaller and faster language representation models, providing a foundation for this approach\[[10](https://arxiv.org/html/2610.33410#bib.bib10),[20](https://arxiv.org/html/2610.33410#bib.bib20)\]\.

Jev is a System One model developed by TypeSafe AI, characterized by low latency and low API cost\[[21](https://arxiv.org/html/2610.33410#bib.bib21)\]\. Independent research has evaluated its decision quality and invocation cost on text classification, natural language inference, reading comprehension, and content moderation\[[9](https://arxiv.org/html/2610.33410#bib.bib9)\], while its performance on time\-series tasks awaits systematic evaluation\. We use Jev as a case study, organizing temporal observations into a unified state\-and\-question representation and evaluating decision quality using accuracy \(ACC\) and the area under the receiver operating characteristic curve \(AUC\) to assess the applicability of Structured System 1 across time\-series tasks\.

Our contributions are threefold:

- –We define the task scope and decision mappings for time\-series classification, anomaly detection, and adapted forecasting, mapping continuous future values to explicit decision targets\.
- –We evaluate Jev 1\.13\.0 through binary and multiclass decision interfaces, reporting ACC and AUC and analyzing performance across datasets and task types\.
- –We establish an extensible evaluation framework that standardizes sample construction, model calls, and result recording, enabling subsequent Structured System 1 models to be evaluated under the same task definitions and evaluation protocols\.

![Refer to caption](https://arxiv.org/html/2610.33410v1/catamap.png)Figure 1:\(a\) Schematic of System 1 and System 2, illustrating intuitive judgments and deliberative reasoning in dual\-process theory\. \(b\) Model taxonomy under our operational definitions: System 1 provides direct answers through structured outputs or text generation, while System 2 uses intermediate reasoning through test\-time enhancement or built\-in reasoning\.
## 2Problem formulation

### 2\.1Time\-series inputs and decision outputs

Following established time\-series definitions\[[13](https://arxiv.org/html/2610.33410#bib.bib13),[15](https://arxiv.org/html/2610.33410#bib.bib15)\], let

𝐗=\(𝐱1,…,𝐱T\)∈ℝT×D,𝐱t∈ℝD,\\mathbf\{X\}=\(\\mathbf\{x\}\_\{1\},\\ldots,\\mathbf\{x\}\_\{T\}\)\\in\\mathbb\{R\}^\{T\\times D\},\\qquad\\mathbf\{x\}\_\{t\}\\in\\mathbb\{R\}^\{D\},\(1\)whereTTis the number of ordered observations,DDis the number of channels, and𝐱t\\mathbf\{x\}\_\{t\}contains the values at timett\. A univariate series is the special caseD=1D=1\. For a decision at timett, an observed window of lengthLLis

𝐗\(t\)=𝐗t−L\+1:t=\(𝐱t−L\+1,…,𝐱t\)∈ℝL×D,L≤t≤T\.\\mathbf\{X\}^\{\(t\)\}=\\mathbf\{X\}\_\{t\-L\+1:t\}=\(\\mathbf\{x\}\_\{t\-L\+1\},\\ldots,\\mathbf\{x\}\_\{t\}\)\\in\\mathbb\{R\}^\{L\\times D\},\\qquad L\\leq t\\leq T\.\(2\)The right endpoint isttand values afterttcannot be used in the input to a future\-oriented decision\.

Some observations are events at irregular times\. We write their history as

ℋt=\{\(τi,mi\):τi≤t,i=1,…,nt\},\\mathcal\{H\}\_\{t\}=\\\{\(\\tau\_\{i\},m\_\{i\}\):\\tau\_\{i\}\\leq t,\\ i=1,\\ldots,n\_\{t\}\\\},\(3\)whereτi\\tau\_\{i\}is the event time,mim\_\{i\}is its categorical mark, andntn\_\{t\}is the number of observed events by timett\. This representation retains the event times instead of imposing a regular sampling interval\. We useℐ\\mathcal\{I\}for the task\-specific input, whether a complete series, a window, or an event history\.

Lets=σ⁡\(ℐ\)s=\\sigma\(\\mathcal\{I\}\)be the state supplied to the model, withσ\\sigmadenoting a fixed representation procedure\. Given a questionqqand a candidate set𝒴=\{1,…,K\}\\mathcal\{Y\}=\\\{1,\\ldots,K\\\}, a decision model returns

𝐩θ​\(s,q,𝒴\)=\(pθ​\(k∣s,q,𝒴\)\)k=1K,∑k=1Kpθ​\(k∣s,q,𝒴\)=1,\\mathbf\{p\}\_\{\\theta\}\(s,q,\\mathcal\{Y\}\)=\\bigl\(p\_\{\\theta\}\(k\\mid s,q,\\mathcal\{Y\}\)\\bigr\)\_\{k=1\}^\{K\},\\qquad\\sum\_\{k=1\}^\{K\}p\_\{\\theta\}\(k\\mid s,q,\\mathcal\{Y\}\)=1,\(4\)with nonnegative components\. Hereθ\\thetadenotes model parameters andKKis the number of admissible decisions\. The predicted answer isy^=arg⁡maxk∈𝒴​pθ​\(k∣s,q,𝒴\)\\hat\{y\}=\\arg\\max\_\{k\\in\\mathcal\{Y\}\}p\_\{\\theta\}\(k\\mid s,q,\\mathcal\{Y\}\)\. The three tasks below differ in what is observed, what the label denotes, and when it becomes known\.

### 2\.2Time\-series classification

For sampleii, let𝐗\(i\)∈ℝTi×D\\mathbf\{X\}^\{\(i\)\}\\in\\mathbb\{R\}^\{T\_\{i\}\\times D\}be an observed series of lengthTiT\_\{i\}and letyi∈\{1,…,K\}y\_\{i\}\\in\\\{1,\\ldots,K\\\}be its class\. The goal is to estimate

pθ​\(yi=k∣σ⁡\(𝐗\(i\)\),q,𝒴\),y^i=arg⁡maxk∈𝒴​pθ​\(yi=k∣σ⁡\(𝐗\(i\)\),q,𝒴\)\.p\_\{\\theta\}\\\!\\left\(y\_\{i\}=k\\mid\\sigma\(\\mathbf\{X\}^\{\(i\)\}\),q,\\mathcal\{Y\}\\right\),\\qquad\\hat\{y\}\_\{i\}=\\arg\\max\_\{k\\in\\mathcal\{Y\}\}p\_\{\\theta\}\\\!\\left\(y\_\{i\}=k\\mid\\sigma\(\\mathbf\{X\}^\{\(i\)\}\),q,\\mathcal\{Y\}\\right\)\.\(5\)The indexiiidentifies a sample,TiT\_\{i\}its length, andDDthe number of channels within its dataset\. A segment cut from a continuous activity record can itself be a classification sample\. ForK=2K=2the task is binary, forK\>2K\>2it is multiclass\. The label describes the observed segment rather than an unobserved future\.

### 2\.3Time\-series anomaly detection

Letat∈\{0,1\}a\_\{t\}\\in\\\{0,1\\\}indicate whether timettis anomalous, with11denoting an anomaly\. Using the historical window ending attt, define the anomaly score

rt=pθ\(at=1∣σ\(𝐗\(t\)\),q,\{0,1\}\),a^t=𝕀\[rt≥η\],r\_\{t\}=p\_\{\\theta\}\\\!\\left\(a\_\{t\}=1\\mid\\sigma\(\\mathbf\{X\}^\{\(t\)\}\),q,\\\{0,1\\\}\\right\),\\qquad\\hat\{a\}\_\{t\}=\\mathbb\{I\}\[r\_\{t\}\\geq\\eta\],\(6\)whereη∈\[0,1\]\\eta\\in\[0,1\]is a prespecified decision threshold and𝕀⁡\[⋅\]\\mathbb\{I\}\[\\cdot\]is the indicator function\. If anomalies are annotated as intervals\[uj,vj\]\[u\_\{j\},v\_\{j\}\], the corresponding point label isat=𝕀\[t∈⋃j\[uj,vj\]\]a\_\{t\}=\\mathbb\{I\}\[t\\in\\bigcup\_\{j\}\[u\_\{j\},v\_\{j\}\]\]\. The window supplies evidence, whereas its right endpoint is the object being labeled\. This distinction prevents the label “the window contains an anomaly” from silently replacing the label “timettis anomalous\.” The continuous scorertr\_\{t\}can also be used to compute threshold\-independent ranking measures\.

### 2\.4Decision\-oriented time\-series forecasting

Conventional forecasting estimates future continuous observations\. LetHHbe the forecast horizon andDyD\_\{y\}the number of forecast target channels\. With𝐲t\+h∈ℝDy\\mathbf\{y\}\_\{t\+h\}\\in\\mathbb\{R\}^\{D\_\{y\}\}, the future target is

𝐘\(t\)=\(𝐲t\+1,…,𝐲t\+H\)∈ℝH×Dy,𝐘^\(t\)=fθ​\(𝐗\(t\)\),\\mathbf\{Y\}^\{\(t\)\}=\(\\mathbf\{y\}\_\{t\+1\},\\ldots,\\mathbf\{y\}\_\{t\+H\}\)\\in\\mathbb\{R\}^\{H\\times D\_\{y\}\},\\qquad\\widehat\{\\mathbf\{Y\}\}^\{\(t\)\}=f\_\{\\theta\}\(\\mathbf\{X\}^\{\(t\)\}\),\(7\)where1≤h≤H1\\leq h\\leq Hdenotes a future step andfθf\_\{\\theta\}is a numeric forecasting model\. For example, squared\-error training minimizes a sum of‖𝐘\(t\)−fθ​\(𝐗\(t\)\)‖F2\\\|\\mathbf\{Y\}^\{\(t\)\}\-f\_\{\\theta\}\(\\mathbf\{X\}^\{\(t\)\}\)\\\|\_\{F\}^\{2\}over training examples, with∥⋅∥F\\\|\\cdot\\\|\_\{F\}denoting the Frobenius norm\.

Our reformulation fixes a label mapϕ\\phibefore evaluating the model:

zt=ϕ⁡\(𝐘\(t\),λ\)∈\{1,…,K\},z^t=arg⁡maxk∈𝒴​pθ​\(zt=k∣σ⁡\(𝐗\(t\)\),q,𝒴\)\.z\_\{t\}=\\phi\(\\mathbf\{Y\}^\{\(t\)\};\\lambda\)\\in\\\{1,\\ldots,K\\\},\\quad\\hat\{z\}\_\{t\}=\\arg\\max\_\{k\\in\\mathcal\{Y\}\}p\_\{\\theta\}\\\!\\left\(z\_\{t\}=k\\mid\\sigma\(\\mathbf\{X\}^\{\(t\)\}\),q,\\mathcal\{Y\}\\right\)\.\(8\)Hereλ\\lambdacollects task parameters, such as a horizon or threshold\. The future values generate the ground\-truth labelztz\_\{t\}, but are never included in the input state\. The model directly answers a question about an event in the future\.

For a concrete example, letPt\>0P\_\{t\}\>0be a stock’s closing price at timett, and define thehh\-step return, forh≤Hh\\leq H, by

Rt,h=Pt\+h−PtPt\.R\_\{t,h\}=\\frac\{P\_\{t\+h\}\-P\_\{t\}\}\{P\_\{t\}\}\.\(9\)The binary upward\-movement target is

zt=𝕀\[Rt,h\>0\]=𝕀\[Pt\+h\>Pt\]\.z\_\{t\}=\\mathbb\{I\}\[R\_\{t,h\}\>0\]=\\mathbb\{I\}\[P\_\{t\+h\}\>P\_\{t\}\]\.\(10\)The equivalence follows because dividing by the positivePtP\_\{t\}does not change the sign ofPt\+h−PtP\_\{t\+h\}\-P\_\{t\}\. The requested quantity is thusPr⁡\(Pt\+h\>Pt∣𝐗\(t\)\)\\Pr\(P\_\{t\+h\}\>P\_\{t\}\\mid\\mathbf\{X\}^\{\(t\)\}\), rather than a point estimate ofPt\+hP\_\{t\+h\}\. A three\-class version, with a fixed toleranceδ\>0\\delta\>0, uses

zt=\{down,Rt,h<−δ,flat,−δ≤Rt,h≤δ,up,Rt,h\>δ\.z\_\{t\}=\\begin\{cases\}\\text\{down\},&R\_\{t,h\}<\-\\delta,\\\\ \\text\{flat\},&\-\\delta\\leq R\_\{t,h\}\\leq\\delta,\\\\ \\text\{up\},&R\_\{t,h\}\>\\delta\.\\end\{cases\}\(11\)The thresholdδ\\deltadetermines the width of the flat class and must remain fixed across systems being compared\. Stock movement prediction has previously been posed as a directional task\[[23](https://arxiv.org/html/2610.33410#bib.bib23)\], the example here illustrates the general conversion of a continuous forecast target into an explicit decision event\.

The reformulation changes the statistical object being estimated\. For a scalar future priceU=Pt\+hU=P\_\{t\+h\}and a squared\-error point forecastcc, differentiating its conditional loss gives

∂∂c​𝔼​\[\(U−c\)2∣𝐗\(t\)\]=2​\(c−𝔼⁡\[U∣𝐗\(t\)\]\),c∗=𝔼⁡\[U∣𝐗\(t\)\]\.\\frac\{\\partial\}\{\\partial c\}\\mathbb\{E\}\[\(U\-c\)^\{2\}\\mid\\mathbf\{X\}^\{\(t\)\}\]=2\\bigl\(c\-\\mathbb\{E\}\[U\\mid\\mathbf\{X\}^\{\(t\)\}\]\\bigr\),\\quad c^\{\*\}=\\mathbb\{E\}\[U\\mid\\mathbf\{X\}^\{\(t\)\}\]\.\(12\)In contrast, the binary decision depends onPr⁡\(U\>Pt∣𝐗\(t\)\)\\Pr\(U\>P\_\{t\}\\mid\\mathbf\{X\}^\{\(t\)\}\)\. A conditional mean and a probability mass above a threshold summarize different aspects of the future distribution\. The same historical window can therefore lead to different optimal numerical and decision outputs\.

### 2\.5Mapping tasks to typed answers

Equation \([4](https://arxiv.org/html/2610.33410#S2.E4)\) gives a common output form\. Sequence classification uses the class of the observed sample, anomaly detection uses the status of a specified endpoint, and decision\-oriented forecasting uses a future\-derived class\. Binary questions are evaluated through Jev’s Noul primitive, which returns the probability that a statement is true\. Choice selects from a fixed candidate set and returns option probabilities, supporting both binary and multiclass decisions\. The label construction and the evaluation unit remain task\-specific even though the external decision format is shared\.

![Refer to caption](https://arxiv.org/html/2610.33410v1/method.png)Figure 2:One possible architecture for a Structured System 1 model, illustrated using BERT\. Embedding, encoding, and classification map the input to class probabilities and a predicted class\.

## 3Method

Jev is a commercial model whose full internal architecture is not publicly documented\[[21](https://arxiv.org/html/2610.33410#bib.bib21)\]\. Figure[2](https://arxiv.org/html/2610.33410#S2.F2)illustrates one possible architecture for Structured System 1 decision making using a BERT\-based classifier\[[10](https://arxiv.org/html/2610.33410#bib.bib10)\]\. We use this example to explain the main computational stages and evaluate Jev through its public decision interface\.

### 3\.1Input Embedding

Given an observed time seriesℐ\\mathcal\{I\}, a representation procedure constructs the input states=σ⁡\(ℐ\)s=\\sigma\(\\mathcal\{I\}\)\. The state records observations chronologically and can include channel names and timestamps\. Together with the task questionqq, it is serialized into a token sequence:

𝐮=Tokenize⁡\(\[CLS\]​s​\[SEP\]​q​\[SEP\]\)\.\\mathbf\{u\}=\\operatorname\{Tokenize\}\\left\(\[\\mathrm\{CLS\}\]\\,s\\,\[\\mathrm\{SEP\}\]\\,q\\,\[\\mathrm\{SEP\}\]\\right\)\.\(13\)Here,\[CLS\]\[\\mathrm\{CLS\}\]is the classification token and\[SEP\]\[\\mathrm\{SEP\}\]separates the state and question\. Numerical observations are represented through the tokens produced by the tokenizer\.

BERT combines token, position, and segment embeddings to construct the initial representation of tokenuju\_\{j\}:

𝐡j\(0\)=𝐞tok​\(uj\)\+𝐞pos​\(j\)\+𝐞seg​\(aj\),\\mathbf\{h\}\_\{j\}^\{\(0\)\}=\\mathbf\{e\}\_\{\\mathrm\{tok\}\}\(u\_\{j\}\)\+\\mathbf\{e\}\_\{\\mathrm\{pos\}\}\(j\)\+\\mathbf\{e\}\_\{\\mathrm\{seg\}\}\(a\_\{j\}\),\(14\)whereaja\_\{j\}identifies whether the token belongs to the state or question\. ForNNtokens and hidden dimensiondd, these vectors form𝐇\(0\)∈ℝN×d\\mathbf\{H\}^\{\(0\)\}\\in\\mathbb\{R\}^\{N\\times d\}\. Token embeddings represent the input content, position embeddings distinguish token locations, and segment embeddings distinguish the two input components\[[10](https://arxiv.org/html/2610.33410#bib.bib10)\]\.

### 3\.2Contextual Encoder

The embedding sequence passes through a stack of bidirectional Transformer encoder layers:

𝐇\(ℓ\)=Encoderℓ\(𝐇\(ℓ−1\)\),ℓ=1,…,Lenc,\\mathbf\{H\}^\{\(\\ell\)\}=\\operatorname\{Encoder\}\_\{\\ell\}\\left\(\\mathbf\{H\}^\{\(\\ell\-1\)\}\\right\),\\qquad\\ell=1,\\ldots,L\_\{\\mathrm\{enc\}\},\(15\)whereLencL\_\{\\mathrm\{enc\}\}is the number of encoder layers\. Each layer uses multi\-head self\-attention and a feed\-forward network, together with residual connections and layer normalization\[[22](https://arxiv.org/html/2610.33410#bib.bib22)\]\.

For one attention head, the input representations are projected into queries, keys, and values:

Q=𝐇​WQ,K=𝐇​WK,V=𝐇​WV\.Q=\\mathbf\{H\}W\_\{Q\},\\qquad K=\\mathbf\{H\}W\_\{K\},\\qquad V=\\mathbf\{H\}W\_\{V\}\.\(16\)The attention output is

Attention⁡\(Q,K,V\)=softmax⁡\(Q​K⊤dk\)​V,\\operatorname\{Attention\}\(Q,K,V\)=\\operatorname\{softmax\}\\left\(\\frac\{QK^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}\\right\)V,\(17\)wheredkd\_\{k\}is the key dimension\. This operation allows each token to incorporate information from other tokens in the observed input\. In the time\-series setting, it provides a mechanism for relating observations at different positions and conditioning their interpretation on the task question\.

The final classification\-token representation,

𝐡CLS=𝐇CLS\(Lenc\)∈ℝd,\\mathbf\{h\}\_\{\\mathrm\{CLS\}\}=\\mathbf\{H\}^\{\(L\_\{\\mathrm\{enc\}\}\)\}\_\{\\mathrm\{CLS\}\}\\in\\mathbb\{R\}^\{d\},\(18\)provides a contextual representation for the decision head\. Bidirectional encoding operates over the supplied observations, which contain only the history available for the task\.

### 3\.3Classification and Decision Probabilities

For a fixed answer set𝒴=\{1,…,K\}\\mathcal\{Y\}=\\\{1,\\ldots,K\\\}, a linear classification head maps the contextual representation to class logits:

ℓ=W​𝐡CLS\+𝐛∈ℝK,\\boldsymbol\{\\ell\}=W\\mathbf\{h\}\_\{\\mathrm\{CLS\}\}\+\\mathbf\{b\}\\in\\mathbb\{R\}^\{K\},\(19\)whereW∈ℝK×dW\\in\\mathbb\{R\}^\{K\\times d\}and𝐛∈ℝK\\mathbf\{b\}\\in\\mathbb\{R\}^\{K\}are learned parameters\. A softmax converts these logits into a probability distribution:

pθ​\(y=k∣s,q,𝒴\)=exp⁡\(ℓk\)∑j=1Kexp⁡\(ℓj\),y^=arg⁡maxk∈𝒴​pθ​\(y=k∣s,q,𝒴\)\.p\_\{\\theta\}\(y=k\\mid s,q,\\mathcal\{Y\}\)=\\frac\{\\exp\(\\ell\_\{k\}\)\}\{\\sum\_\{j=1\}^\{K\}\\exp\(\\ell\_\{j\}\)\},\\qquad\\hat\{y\}=\\arg\\max\_\{k\\in\\mathcal\{Y\}\}p\_\{\\theta\}\(y=k\\mid s,q,\\mathcal\{Y\}\)\.\(20\)
For binary decisions, a single logitv=𝐰⊤​𝐡CLS\+bv=\\mathbf\{w\}^\{\\top\}\\mathbf\{h\}\_\{\\mathrm\{CLS\}\}\+bcan instead be converted into the positive\-class probability:

pθ​\(y=1∣s,q\)=11\+e−v,pθ​\(y=0∣s,q\)=1−pθ​\(y=1∣s,q\)\.p\_\{\\theta\}\(y=1\\mid s,q\)=\\frac\{1\}\{1\+e^\{\-v\}\},\\qquad p\_\{\\theta\}\(y=0\\mid s,q\)=1\-p\_\{\\theta\}\(y=1\\mid s,q\)\.\(21\)
The fixed\-label head illustrates standard BERT classification\. To accommodate changing candidate sets, an alternative is to encode each candidate together with the state and question, apply a shared scoring head, and normalize the resulting scores across candidates\. This makes the output depend explicitly on the supplied alternatives\.

In both cases, the encoder representations are mapped directly to decision probabilities without generating an answer token by token\. The predicted class supports ACC evaluation, while the probabilities support AUC evaluation\. In our Jev experiments, these outputs are obtained through Noul for binary questions and Choice for multiclass questions\.

## 4Experiments

### 4\.1Datasets And Evaluation Metrics

This study selects datasets suitable for evaluating System 1 models, covering time series classification, event type prediction, time series anomaly detection, and forecasting\. These datasets include univariate and multivariate numerical sequences, as well as irregular event sequences consisting of timestamps and event types\. We evaluate Jev on these datasets using accuracy \(ACC\) and the area under the receiver operating characteristic curve \(AUC\)\.

#### \(1\) Time series classification: UCR and UEA archives\.

The UCR Archive is a representative collection of datasets for univariate time series classification\. Its 2018 release contains 128 datasets, covering sensor recordings, physiological signals such as electrocardiograms, motion data, and sequences derived from image outlines\[[8](https://arxiv.org/html/2610.33410#bib.bib8)\]\. Each sample consists of a univariate sequence and its class label\. The number of samples, sequence lengths, and numbers of classes vary substantially across datasets\. The task is thereforeunivariate time series classification: the model reads an entire sequence and predicts its class, under either a binary or multiclass setting\. This study uses the archive to examine Jev’s ability to recognize temporal patterns in a single channel\.

The UEA Archive focuses onmultivariate time series classification\. Its 2018 release contains 30 datasets covering activity recognition, motion recognition, physiological signals, and other settings involving multiple observation channels\[[4](https://arxiv.org/html/2610.33410#bib.bib4)\]\. Whereas each time step in UCR contains a single observation, each UEA sample consists of multiple channels evolving over time and is associated with a sequence\-level class label\. The model must identify temporal changes within individual channels and exploit relationships between channels\. For example, activity recognition may require combining recordings from several motion sensors to distinguish actions with similar local patterns\. UCR and UEA thus provide sequence classification evaluations under univariate and multivariate input settings, respectively\.

#### \(2\) Event type prediction: EasyTPP\.

EasyTPP is a benchmarking framework for temporal point processes that provides event sequence datasets, model implementations, and standardized evaluation procedures\[[24](https://arxiv.org/html/2610.33410#bib.bib24)\]\. The five real\-world datasets used in its original paper are Amazon, Retweet, Taxi, Taobao, and StackOverflow\. They contain user reviews, social reposts, taxi pick\-up and drop\-off events, e\-commerce clicks, and user badge awards, respectively\. Each sequence consists of events ordered by occurrence time, with each event containing a timestamp and a discrete event type\. The intervals between events are generally irregular\.

#### \(3\) Direction prediction: ETT and stock market data\.

The Electricity Transformer Temperature \(ETT\) dataset contains transformer monitoring records used in the Informer study\[[26](https://arxiv.org/html/2610.33410#bib.bib26)\]\. Its ETT\-small collection includes ETTh1, ETTh2, ETTm1, and ETTm2, with hourly or 15\-minute observations of six electrical load variables and oil temperature \(OT\)\. The stock market data comprise daily records for AAPL, BA, and CAT released by Al Ridhawi et al\.\[[1](https://arxiv.org/html/2610.33410#bib.bib1)\]\. These datasets cover two forecasting settings: temperature changes in electrical equipment and price movements in financial markets\.

This study reformulates continuous\-valued forecasting on both datasets asbinary direction prediction: the model reads a historical observation window and predicts whether the target value at a specified future horizon exceeds its current value, yielding “increase” and “not increase\.” For ETT, the target is OT at a 24\-hour horizon, corresponding to 24 hourly steps or 96 15\-minute steps\. We evaluate univariate \(S\) and multivariate \(MS\) input settings, using OT alone or OT together with the six electrical load variables, respectively\. For the stock data, the target is the closing price at a fixed number of trading steps ahead\. Together, these datasets examine Jev’s ability to use temporal observations to anticipate directional changes, providing information relevant to transformer operational monitoring and directional investment decisions\.

### 4\.2Implementation Details

Each sample is represented by an input state, a task question, candidate answers, and a ground\-truth label\. Observations are arranged chronologically using consistent input settings within each experiment\.

We evaluate Jev 1\.13\.0 with a fixed question template for each task\. Binary tasks use Noul to estimate the probability of the positive event, while multiclass tasks use Choice to return a selected answer and candidate probabilities\. Predictions and probabilities are recorded alongside ground\-truth labels for evaluation\.

Table 1:Overall results, averaged equally across datasets with available scores\.Table 2:Selected UCR results used in the analysis\.
### 4\.3Experimental Results and Analysis

Table[1](https://arxiv.org/html/2610.33410#S4.T1)summarizes Jev’s performance across the evaluated dataset groups\. Dataset\-level results supporting the analysis are presented in Tables[2](https://arxiv.org/html/2610.33410#S4.T2)–[6](https://arxiv.org/html/2610.33410#S4.T6)\. \(1\) Numerical sequence classification shows limited overall discrimination: the mean AUC is 0\.5341 for binary UCR tasks, 0\.5187 for multiclass UCR tasks, and 0\.5172 for UEA tasks\. Among the 86 multiclass UCR datasets, 63 have AUC values between 0\.45 and 0\.55, with a median of 0\.5047\. \(2\) Strong performance is concentrated in a small subset\. SyntheticControl achieves an ACC of 0\.8100 and an AUC of 0\.9978, while SmoothSubspace achieves 0\.9000 and 0\.9815\. Excluding these two datasets reduces the mean multiclass UCR AUC to 0\.5075\. These results show that success on selected temporal patterns has not translated into consistent performance across the archive\.

The dataset\-level results reveal differences associated with temporal structure, target granularity, and signal type\. \(1\) Several stronger results involve recognizable trends or shapes\. SyntheticControl contains classes defined by cycles, trends, and level shifts, while SmoothSubspace distinguishes the position of a continuous segment\[[8](https://arxiv.org/html/2610.33410#bib.bib8)\]\. BME and CBF also achieve AUC values of 0\.7434 and 0\.7019\. Chinatown and ItalyPowerDemand reach 0\.6657 and 0\.6597, suggesting some ability to recognize patterns in everyday activity and demand\. However, Wafer reaches 0\.8056 despite its specialized industrial setting, whereas the synthetic TwoPatterns dataset reaches only 0\.3224\. The results therefore suggest that accessibility of the discriminative pattern matters alongside domain familiarity\. \(2\) Target granularity also affects performance: DodgerLoopWeekend achieves an AUC of 0\.6953 when distinguishing weekdays from weekends, compared with 0\.5056 for the seven\-class DodgerLoopDay task\. This contrast is consistent with stronger recognition of broad activity differences than fine distinctions within the same application domain\. \(3\) Physiological signals remain difficult\. ECG200, ECGFiveDays, and TwoLeadECG have AUC values of 0\.4696, 0\.4434, and 0\.4629, while the EEG dataset FingerMovements reaches 0\.4138\. Limited domain\-specific knowledge and insufficient representation of discriminative signal features are possible explanations\. \(4\) Some scores retain useful ranking information despite weak final decisions\. GunPoint achieves an AUC of 0\.7215 with an ACC of 0\.5200, and BasicMotions achieves an AUC of 0\.7671 with an ACC of 0\.3000\. This gap motivates examining binary thresholds and the comparability of multiclass candidate scores\.

Table 3:Selected UEA results used in the analysis\.Table 4:Event type prediction results on EasyTPP datasets\.Table 5:Oil temperature direction prediction results\.Note:S denotes univariate input \(oil temperature only\), while MS denotes multivariate input \(oil temperature and six electrical load variables\)\.

Table 6:Binary stock movement prediction results\.Future\-oriented tasks show a further contrast between event categories and numerical movement directions\. \(1\) Event type prediction produces consistently higher AUC values in the reported experiments\. Across the five EasyTPP datasets, AUC ranges from 0\.6466 to 0\.8241\. Taobao achieves an ACC of 0\.6343 and an AUC of 0\.8241, while Taxi achieves 0\.6740 and 0\.7282\. These results suggest that discrete event histories provide a useful input form for Jev, potentially because event categories expose predictive relationships more directly than raw numerical sequences\. \(2\) Stock movement prediction remains close to chance\-level discrimination\. AAPL, BA, and CAT achieve AUC values of 0\.4950, 0\.5042, and 0\.5129, respectively, with a mean ACC of 0\.5034\. Thus, converting continuous price forecasting into binary movement prediction makes the task compatible with the decision interface but yields little predictive discrimination in these experiments\. Oil temperature direction prediction also shows modest discriminative ability, with a mean ACC of 0\.5137 and a mean AUC of 0\.5900 across the eight ETT settings \(Table[5](https://arxiv.org/html/2610.33410#S4.T5)\)\. Multivariate input achieves higher AUC on ETTh2, ETTm1, and ETTm2, while univariate input performs better on ETTh1\. However, ACC remains close to 0\.5 in most settings, suggesting a gap between probability ranking and final classification decisions\. Together, the results identify discrete event prediction and selected explicit temporal patterns as the stronger settings, while numerical direction prediction and specialized signal classification remain challenging\.

The results suggest that future System 1 models should extend their recognition of explicit trends and simple shapes to finer temporal patterns and cross\-channel relationships\. Strong performance on SyntheticControl and SmoothSubspace contrasts with weaker results on physiological signals and most evaluated UEA datasets, motivating broader domain\-specific training data and improved temporal and channel representations\. These developments could strengthen the recognition of local waveforms and interactions between variables\.

The higher AUC but lower ACC on GunPoint and BasicMotions also motivates better probability calibration, threshold selection, and alignment of multiclass scores\. Meanwhile, the contrast between EasyTPP and stock movement prediction highlights the need for training that explicitly targets future decisions\. Future models should improve the reliability of these decisions while retaining direct, structured outputs\.

## 5Conclusion

This study evaluates Jev 1\.13\.0 as a Structured System 1 model for time\-series decisions through a unified evaluation framework\. The results reveal uneven performance: Jev recognizes selected explicit temporal patterns and achieves relatively consistent discrimination in event type prediction, while performance on many numerical sequence classification tasks and stock movement prediction remains close to chance\-level discrimination\. Several datasets also show useful ranking information that does not translate into accurate final decisions\. These findings highlight temporal representation, domain adaptation, and decision\-score alignment as priorities for future Structured System 1 models\. The framework supports extending this evaluation to additional models under consistent task definitions\.

## References

- \[1\]Mohammad Al Ridhawi, Mahtab Haj Ali, and Hussein Al Osman\.Daily OHLCV data for 20 S&P 500 equities and the CBOE volatility index \(1982–2025\), 2026\.URL[https://doi\.org/10\.5281/zenodo\.20192822](https://doi.org/10.5281/zenodo.20192822)\.Dataset\.
- \[2\]Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C\. Maddix, Hao Wang, Michael W\. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke\-Schneider, and Yuyang Wang\.Chronos: Learning the language of time series\.*Transactions on Machine Learning Research*, 2024\.URL[https://arxiv\.org/abs/2403\.07815](https://arxiv.org/abs/2403.07815)\.
- \[3\]Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C\. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, and Michael Bohlke\-Schneider\.Chronos\-2: From univariate to universal forecasting, 2025\.URL[https://arxiv\.org/abs/2510\.15821](https://arxiv.org/abs/2510.15821)\.
- \[4\]Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh\.The UEA multivariate time series classification archive, 2018, 2018\.URL[https://arxiv\.org/abs/1811\.00075](https://arxiv.org/abs/1811.00075)\.
- \[5\]Leo Breiman\.Random forests\.*Machine Learning*, 45\(1\):5–32, 2001\.doi:10\.1023/A:1010933404324\.
- \[6\]Tianqi Chen and Carlos Guestrin\.XGBoost: A scalable tree boosting system\.In*Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, pages 785–794\. Association for Computing Machinery, 2016\.doi:10\.1145/2939672\.2939785\.
- \[7\]Corinna Cortes and Vladimir Vapnik\.Support\-vector networks\.*Machine Learning*, 20\(3\):273–297, 1995\.doi:10\.1007/BF00994018\.
- \[8\]Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin\-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh\.The UCR time series archive\.*IEEE/CAA Journal of Automatica Sinica*, 6\(6\):1293–1305, 2019\.doi:10\.1109/JAS\.2019\.1911747\.
- \[9\]Tobias Deußer, Lorenz Sparrenberg, and Rafet Sifa\.Evaluating and benchmarking the system one model Jev, 2026\.URL[https://arxiv\.org/abs/2609\.37647](https://arxiv.org/abs/2609.37647)\.
- \[10\]Jacob Devlin, Ming\-Wei Chang, Kenton Lee, and Kristina Toutanova\.BERT: Pre\-training of deep bidirectional transformers for language understanding\.In*Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\)*, pages 4171–4186\. Association for Computational Linguistics, 2019\.doi:10\.18653/v1/N19\-1423\.URL[https://aclanthology\.org/N19\-1423/](https://aclanthology.org/N19-1423/)\.
- \[11\]Jonathan St\. B\. T\. Evans and Keith E\. Stanovich\.Dual\-process theories of higher cognition: Advancing the debate\.*Perspectives on Psychological Science*, 8\(3\):223–241, 2013\.doi:10\.1177/1745691612460685\.
- \[12\]Hrayr Harutyunyan, Hrant Khachatrian, David C\. Kale, Greg Ver Steeg, and Aram Galstyan\.Multitask learning and benchmarking with clinical time series data\.*Scientific Data*, 6\(1\):96, 2019\.doi:10\.1038/s41597\-019\-0103\-9\.
- \[13\]Ming Jin, Huan Yee Koh, Qingsong Wen, Daniele Zambon, Cesare Alippi, Geoffrey I\. Webb, Irwin King, and Shirui Pan\.A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection\.*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 46\(12\):10466–10485, 2024a\.doi:10\.1109/TPAMI\.2024\.3443141\.
- \[14\]Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y\. Zhang, Xiaoming Shi, Pin\-Yu Chen, Yuxuan Liang, Yuan\-Fang Li, Shirui Pan, and Qingsong Wen\.Time\-LLM: Time series forecasting by reprogramming large language models\.In*International Conference on Learning Representations*, 2024b\.URL[https://proceedings\.iclr\.cc/paper\_files/paper/2024/hash/680b2a8135b9c71278a09cafb605869e\-Abstract\-Conference\.html](https://proceedings.iclr.cc/paper_files/paper/2024/hash/680b2a8135b9c71278a09cafb605869e-Abstract-Conference.html)\.
- \[15\]Ming Jin, Yaxuan Kong, Yuxuan Liang, Chaoli Zhang, Siqiao Xue, Xue Wang, James Zhang, Yi Wang, Haifeng Chen, Xiao\-Li Li, Vincent S\. Tseng, Yu Zheng, Lei Chen, Hui Xiong, Shirui Pan, and Qingsong Wen\.Large models for time series and spatio\-temporal data: A survey and outlook\.*ACM Computing Surveys*, 58\(14\), 2026\.doi:10\.1145/3821637\.
- \[16\]Christian Lessmeier, James Kuria Kimotho, Detmar Zimmer, and Walter Sextro\.Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: A benchmark data set for data\-driven classification\.In*Proceedings of the European Conference of the PHM Society*, 2016\.doi:10\.36001/phme\.2016\.v3i1\.1577\.
- \[17\]Haoxin Liu, Zhiyuan Zhao, Shiduo Li, and B\. Aditya Prakash\.Evaluating system 1 vs\. 2 reasoning approaches for zero\-shot time series forecasting: A benchmark and insights, 2025\.URL[https://arxiv\.org/abs/2503\.01895](https://arxiv.org/abs/2503.01895)\.
- \[18\]Jigar Patel, Sahil Shah, Priyank Thakkar, and Ketan Kotecha\.Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques\.*Expert Systems with Applications*, 42\(1\):259–268, 2015\.doi:10\.1016/j\.eswa\.2014\.07\.040\.
- \[19\]Attila Reiss and Didier Stricker\.Introducing a new benchmarked dataset for activity monitoring\.In*2012 16th International Symposium on Wearable Computers*, pages 108–109\. IEEE, 2012\.doi:10\.1109/ISWC\.2012\.13\.
- \[20\]Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf\.DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter, 2019\.URL[https://arxiv\.org/abs/1910\.01108](https://arxiv.org/abs/1910.01108)\.
- \[21\]TypeSafe AI\.Introducing system one models and Jev\.[https://typesafe\.ai/blog/introducing\-system\-one\-models\-and\-jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev), 2026\.Accessed 2026\-10\-21\.
- \[22\]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N\. Gomez, Łukasz Kaiser, and Illia Polosukhin\.Attention is all you need\.In*Advances in Neural Information Processing Systems*, volume 30, 2017\.URL[https://proceedings\.neurips\.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa\-Abstract\.html](https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html)\.
- \[23\]Yumo Xu and Shay B\. Cohen\.Stock movement prediction from tweets and historical prices\.In*Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 1970–1979\. Association for Computational Linguistics, 2018\.doi:10\.18653/v1/P18\-1183\.
- \[24\]Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Hongyan Hao, Fan Zhou, Caigao Jiang, Chen Pan, James Y\. Zhang, Qingsong Wen, Jun Zhou, and Hongyuan Mei\.EasyTPP: Towards open benchmarking temporal point processes\.In*International Conference on Learning Representations*, 2024\.URL[https://proceedings\.iclr\.cc/paper\_files/paper/2024/hash/e66233a208ef32f56df6312263239fa0\-Abstract\-Conference\.html](https://proceedings.iclr.cc/paper_files/paper/2024/hash/e66233a208ef32f56df6312263239fa0-Abstract-Conference.html)\.
- \[25\]Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov\.Distilling system 2 into system 1, 2024\.URL[https://arxiv\.org/abs/2407\.06023](https://arxiv.org/abs/2407.06023)\.
- \[26\]Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang\.Informer: Beyond efficient transformer for long sequence time\-series forecasting\.*Proceedings of the AAAI Conference on Artificial Intelligence*, 35\(12\):11106–11115, 2021\.doi:10\.1609/aaai\.v35i12\.17325\.

## Appendix AComplete UCR and UEA Results

Tables[7](https://arxiv.org/html/2610.33410#A1.T7)–[9](https://arxiv.org/html/2610.33410#A1.T9)present the complete UCR and UEA classification results\.

Table 7:Complete UCR binary results recorded in the source file\. Missing scores are shown as –\.
Table 8:UCR results\.
Table 9:UEA results