Towards Verifiable Agentic Data Science: Solving Irregular TSQA Via Tool-Grounded Reasoning

# Solving Irregular TSQA via Tool-Grounded Reasoning
Source: [https://arxiv.org/html/2606.15107](https://arxiv.org/html/2606.15107)
## Towards Verifiable Agentic Data Science: Solving Irregular TSQA via Tool\-Grounded Reasoning

Sanhorn Chen, Xiaoyang Chen, Boyu Liu, Roy Zhao University of Illinois Urbana Champaign \{sanhorn2, xc52\}@illinois\.edu

###### Abstract

Time series data in real\-world deployments is overwhelmingly irregular\. Observations are asynchronous, missing values are informative rather than random, and sampling frequencies vary across sensors and operational windows\. However, existing Time Series Question Answering \(TSQA\) benchmarks mostly assume regularly sampled inputs, leaving a fundamental gap in understanding how large language models \(LLMs\) and AI agents perform under irregular conditions\. To bridge this gap, we introduce*IRTS\-ToolBench*, a benchmark of 1,700 questions spanning 10 task types across 13 domains\.*IRTS\-ToolBench*is designed to be used independently by any researcher working on LLM\-based irregular time series analysis, providing standardized inputs and a reproducible evaluation protocol\. Code can be found in[https://github\.com/SanhornC/IRTS\-ToolBench](https://github.com/SanhornC/IRTS-ToolBench)\.

## 1Introduction

Time series data is among the most common forms of information in real\-world systems\(Changet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib1)\)\. The rapid development of Large Language Models \(LLMs\) and AI agents for complex reasoning tasks have made Time Series Question Answering \(TSQA\) an increasingly important evaluation paradigm\. Recent benchmarks\(Jinget al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib2); Konget al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib3); Wuet al\.,[2026a](https://arxiv.org/html/2606.15107#bib.bib15); Yuet al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib14); Yinet al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib16)\)have demonstrated that LLMs can perform temporal reasoning over structured time series inputs with contextual information as supplement\. Concurrently, agentic frameworks\(Wuet al\.,[2026b](https://arxiv.org/html/2606.15107#bib.bib4); Liuet al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib5); Zhaoet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib6)\)have shown that tool\-augmented and multi\-agents outperform direct LLM inference on complex time series tasks\. However, these works share a critical limitation: they operate exclusively on regular time series while assumption that time series data arrives at regular is systematically violated in practice\. For instance, ICU monitoring systems record vital signs when a nurse or physician deems it clinically necessary, and industrial sensors log readings only when anomalies are detected\. As a result, the absence of a rigorous benchmark for irregular time series constitutes a fundamental gap in our evaluation infrastructure\.

Existing synthetic irregularization methods, such as MCAR random dropout and sparse mask resampling, apply transformations without understanding the domain semantics or the reason behind the missingness, resulting in data that are statistically irregular but semantically implausible\. TIME\-IMM\(Changet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib1)\)identifies this gap, demonstrating that real\-world irregularity is organized into nine cause\-driven types across three categories \(Trigger\-Based, Constraint\-Based, and Artifact\-Based\) none of which are captured by random dropout\. To the best of our knowledge, no prior work has proposed pipelines for semantically grounded regular\-to\-irregular time series transformation\.

We introduce*IRTS\-ToolBench*, a benchmark specifically designed to evaluate LLMs and AI agents on irregular univariate TSQA via tool\-grounded reasoning\. Our contributions are threefold\. \(1\) We propose a novel LLM\-Guided Irregular Transformation Pipeline for semantically grounded regular\-to\-irregular time series transformation\. \(2\) The*IRTS\-ToolBench*consists of 1,700 questions spanning 10 task types across 13 domains\. \(3\) We collected a 30\-tool library with 7 irregularity operators and 23 analytical tools, enabling the benchmark to support evaluation of both LLMs and AI agents\.

The remainder of this paper is organized as follows\. Sec\.[2](https://arxiv.org/html/2606.15107#S2)situates IRTS\-ToolBench within related work\. Sec\.[3](https://arxiv.org/html/2606.15107#S3)describes the benchmark design in detail, including the transformation pipeline, task descriptions, and tool library\. Sec\.[4](https://arxiv.org/html/2606.15107#S4)defines the evaluation protocol\. Sec\.[5](https://arxiv.org/html/2606.15107#S5)discusses benchmark validation\. Sec\.[6](https://arxiv.org/html/2606.15107#S6)concludes with limitations and future directions\.

## 2Related Work

TSQA Benchmarks\.TSAQA\(Jinget al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib2)\)provides a hierarchical taxonomy of six task types, including anomaly detection, classification, characterization, comparison, data transformation, and temporal relationship, across 60 datasets\. Its multi\-LLM consensus question generation mechanism directly inspired our own pipeline\. Time\-MQA\(Konget al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib3)\)extends TSQA to multi\-task settings with context enhancement\. ITFormer\(Wanget al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib7)\)introduces a aero\-engine QA dataset pairing time series with natural language for cross\-modal alignment tasks\. Despite their contributions, all three benchmarks assume clean, regularly sampled time series as input\.

Irregular Time Series: Taxonomy and Transformation Methods\.TIME\-IMM\(Changet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib1)\)introduces a nine\-type irregularity taxonomy organized into Trigger\-Based, Constraint\-Based, and Artifact\-Based categories, each anchored to a real\-world dataset, and critiques existing benchmarks for assuming regular sampling\. Our transformation pipeline uses this taxonomy as its decision space\.\(Duet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib8)\)provide implementations of MCAR, MAR, MNAR, block missing, and sequence missing functions, forming the execution layer of our pipeline\. Physiome\-ODE\(klötergens2025physiomeodebenchmarkirregularlysampled\)provides an irregular multivariate forecasting benchmark based on biological ODEs\. Critically, none of these works employ LLMs to guide the irregularization process\.

Agentic Time Series Frameworks\.TimeART\(Wuet al\.,[2026b](https://arxiv.org/html/2606.15107#bib.bib4)\)proposes a ReAct\-style agent with curated tools for time series reasoning, establishing the template for tool\-augmented agentic evaluation that our benchmark extends to the irregular domain\. TS\-Agent\(Liuet al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib5)\)focuses on statistical insight gathering for time series tasks\. TimeSeriesScientist\(Zhaoet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib6)\)proposed agent with different roles optimized for forecasting workflows\.

## 3Benchmark Design

### 3\.1Task Domain, Scope, and Descriptions

IRTS\-ToolBench draws from two source datasets\(Jinget al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib2); Konget al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib3)\)spanning 13 domains \(e\.g\. finance, healthcare\)\. All collected regular TS samples are reconstructed through a three\-layer pipeline \(Sec\.[3\.2](https://arxiv.org/html/2606.15107#S3.SS2)\) into univariate irregular TS in one ofmultiple\_choices \(MC\)ortrue\_or\_false \(TF\)format\. The benchmark is organized into*10*task types grouped into*3*categories \(Details of Tasks Descriptions in Table[1](https://arxiv.org/html/2606.15107#A1.T1)\): \(1\)Standard Reasoning: This category of questions require models to perform simple time series reasoning primitives over irregular time series inputs and test whether models can identify differences between irregular and regular TS\. Specifically, we introduce*4*tasks:*Anomaly Detection*,*Classification*,*Regular vs\. Irregualr Discrimination*, and*Regularity Recovery*\. \(2\)Irregularity\-Specific Reasoning: In this category, we evaluate whether models can reason over irregular TS inputs by identifying their temporal characteristics, inferring relationships among asynchronous observations, and attributing the underlying causes of irregular sampling patterns\. Specifically, we introduce*3*tasks:*Characterization*,*Temporal Relationship*, and*Irregularity Cause Attribution*\. \(3\)Regularity–Irregularity Interface Reasoning: These tasks require models to reason across the boundary between regular and irregular representations, and test whether models can reason at the interface between regularity and irregularity\. Specifically, we introduce*3*tasks*Missingness Reasoning*,*Irregularity Severity Estimation*, and*Forecasting*\. These three categories are designed to form a progressive reasoning hierarchy, moving from basic irregular sampling, to explicit understanding of irregular temporal structures, and finally to reasoning across the interface between regular and irregular representations\. For more details regarding task descriptions, please refer to Appendix[A](https://arxiv.org/html/2606.15107#A1)\. Additionally, task examples can be found in Appendix[E](https://arxiv.org/html/2606.15107#A5)\.

### 3\.2Benchmark Construction Pipeline

Our benchmark draws from 2 source datasets\(Jinget al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib2); Konget al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib3)\)\. Specifically, each collected sample is processed through a three\-layer transformation pipeline\. We describe each pipeline stage below\.

Irregular Time Series Transformation\.Our transformation pipeline, visualized in Figure[1](https://arxiv.org/html/2606.15107#A2.F1)leverages the irregularity types from\(Changet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib1)\)as a decision space for selecting the appropriate transformation mechanism for each sample\. Specifically, the pipeline follows a unified three\-stage design\. \(1\)Context Enrichment: We leverage a LLM to generate an enriched context description capturing the domain, statistical features, and signal characteristics for each time series sequence\. \(2\)Taxonomy Selection: Given the enriched context and time series statistics, an LLM selects the most appropriate irregularity type and outputs a transformation plan with associated anomaly annotations along with a self\-evaluated confidence score\. \(3\)Parameter Generation: An LLM translates the transformation plan into validated numerical parameters for the relevant transformation functions\. Transformation execution then applies the generated parameters, producing an irregular time series with realistic missingness patterns and optionally jittered timestamps\. More information can be found in Appendix[B\.1](https://arxiv.org/html/2606.15107#A2.SS1)\.

Question Generation Pipeline\.For each task type, a dedicated question generation pipeline produces questions conditioned on the transformed irregular time series and its associated anomaly annotations\. The generation procedure, visualized in Figure[2](https://arxiv.org/html/2606.15107#A2.F2), is as follows: \(1\)Primary Generation: GPT\-5\.1 is prompted with task\-specific prompts to generate the*meta\_information*for each sample\. Depending on the task design, the corresponding*question*\(MCorTF\) and answer are either generated by GPT\-5\.1 or constructed using task\-specific deterministic functions\. \(2\)Multi\-LLM Consensus: Then, three independent LLMs \(GPT\-5\.1, Claude Sonnet 4\.5, Gemini 2\.5 Flash\) evaluate whether the generated*meta\_information*,*question*, and*answer*are clear and answerable from the provided time series \(e\.g\. score\>\>threshold\)\. More details in Appendix[B\.2](https://arxiv.org/html/2606.15107#A2.SS2)\.

Golden Tool Set Construction\.For each finalized benchmark sample, a golden tool set specifies the minimum set of tools that a model must invoke to correctly answer the question\. The construction protocol, visualized in Figure[3](https://arxiv.org/html/2606.15107#A2.F3), proceeds as follows: \(1\)Independent Proposal: Three LLMs \(GPT\-5\.1, Claude Sonnet 4\.5, Gemini 2\.5 Flash\) independently propose the required tool sequence for the question, together with a self\-confidence score and an explanation for each tool’s necessity\. \(2\)Consensus Gold Set: The Gold Set is constructed by*Majority Voting*with a*Union*fallback, which tools proposed by at least two LLMs are retained, while if no tool receives majority support, the union of all proposed tools is used\. More details in Appendix[B\.3](https://arxiv.org/html/2606.15107#A2.SS3)\.

#### 3\.2\.1Tool Library

IRTS\-ToolBench provides a 30\-tool library, as shown in Table[3](https://arxiv.org/html/2606.15107#A4.T3), that enables the benchmark to support evaluation of both LLMs \(via tool\-augmented prompting\) and AI agents \(via agentic tool\-use frameworks\)\. The library is organized into two layers: \(1\)Irregularity Operators: These tools provide the operations for handling irregular time series sequences\. \(2\)Advanced Analytical Tools: This layer consists of 23 tools, providing time series analysis primitives, such as summary statistics and trend and seasonality detection\. More details in Appendix[D](https://arxiv.org/html/2606.15107#A4)\.

## 4Evaluation Protocol

*IRTS\-ToolBench*is designed to be evaluated under a standardized protocol\. Specifically, the model output is parsed and compared with the ground\-truth, both in eitherMCorTFquestion format\. We define three primary metrics and specify how they should be computed, aggregated, and reported\.

Metrics\.The primary metric is*Overall Accuracy*:Accuracy=1N​∑i=1Nsi\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}s\_\{i\}, where N is the total number of evaluated examples\. For bothMCandTFquestions, the model output must exactly match the ground\-truth answer to receive credit\. We also report*Task\-Level Accuracy*for each task typett:Accuracy​\(t\)=1\|𝒟t\|​∑i∈𝒟tsi\\text\{Accuracy\}\(t\)=\\frac\{1\}\{\|\\mathcal\{D\}\_\{t\}\|\}\\sum\_\{i\\in\\mathcal\{D\}\_\{t\}\}s\_\{i\}, whereDtD\_\{t\}is the set of examples belonging to tasktt\. In addition, when tool evaluation is enabled, we compare the set of tools called by the model with the proposed golden tool set\. Each example is assigned one label: \(1\)*Exact Match*, where the called tool set matches the golden tool set; \(2\)*Partial Match*, where the called tool set partially matches the golden tool set; \(3\)*Complete Mismatch*, where the called tool set doesn’t match the golden tool set\. For each task type, we report these three rates\. These rates are mutually exclusive and sum to 1 within each task, and they are not used to compute overall accuracy and task\-level accuracy\.

Scoring and Aggregation\.All questions are scored using binary scoring\. Each example receives11if the model answer matches the ground\-truth, and0otherwise\.*Overall accuracy*is the average score across all samples, while*Task\-Level Accuracy*is the average score within each task type\. Tool\-match rates are aggregated separately by task type and used as indicators of tool\-use behavior\.

Reporting Requirements\.For each model, we report*Overall Accuracy*,*Task\-Level Accuracy*across the1010task types, and, when applicable, task\-level*Tool\-Match Rates*\. We also specify whether the model is evaluated in tool\-augmented mode\.

## 5Benchmark Validation

Task Quality Verification\.Task quality is controlled in both*Question Generation*and*Golden Tool Set Construction*pipeline through a multi\-LLM evaluator layer\. For*Question Generation*pipeline, each generated QA pair is reviewed by GPT\-5\.1, Claude Sonnet 4\.5, and Gemini 2\.5 Flash using the same evaluation prompt\. Evaluators assign confidence scores based on question clarity, answer support from the provided time series, and the validity of distractors\. Most tasks are retained when all evaluators score greater or equal to a threshold\. For instance, Anomaly Detection task requires all evaluators to score at least 0\.85\. For*Golden Tool Set Construction*pipeline, given each QA pair and the proposed tool library, GPT\-5\.1, Claude Sonnet 4\.5, and Gemini 2\.5 Flash independently propose tools required to solve the question\. A tool is collected if it appears in at least two model proposals\. If no tool reaches majority support, we use the union of all proposed tools as a fallback\.

Human Peer Review and Human Baseline\.We further conduct human peer\-review testing on a randomly sampled 2% subset of the benchmark, stratified by task type\. Two undergraduate reviewers independently answer the sampled questions and flag ambiguous or underspecified items\. This evaluation serves two purposes: validating task clarity and providing a simple human baseline\. The two reviewers achieve overall accuracies of 80% and 78%, respectively\. Overall uncertainty rates are low, although*Regularity Recovery*and*Missingness Reasoning*show higher uncertainty and lower agreement than the other tasks\. These results suggest that most benchmark questions are understandable to human reviewers, while the harder irregularity\-specific tasks remain challenging\.

Evaluation Results\.We evaluate zero\-shot performance of \(1\) commercial LLMs: Claude\-Opus\-4\.7 \(without thinking, with thinking, with tool\-calling\) \(2\) open\-source LLMs: Qwen3\.5\-4B\(Qwen Team,[2026a](https://arxiv.org/html/2606.15107#bib.bib13)\), Qwen3\.6\-27B\(Qwen Team,[2026b](https://arxiv.org/html/2606.15107#bib.bib11)\), and DeepSeek\-V4\-Flash\(DeepSeek\-AI,[2026](https://arxiv.org/html/2606.15107#bib.bib12)\)\(with and without tool\-calling\)\. Our*Overall Results*, as shown in Table[2](https://arxiv.org/html/2606.15107#A3.T2), suggest that Qwen3\.6\-27B \(78\.59\) outperforms the other three models in all settings, while commerical model Claude\-Opus\-4\.7 receive a consistent overall accuracy in between 74 to 77 percent\. For*Task\-Level Results*, tool calling provides the most visible improvements on several tasks\. For open\-source models, Qwen3\.6\-27B improves from 96\.80 to 99\.60 on anomaly detection and reaches 100\.00 on classification with tools, while DeepSeek\-V4\-Flash shows a particularly large gain on irregularity severity estimation task, increasing from 31\.33 to 98\.67, and also improves on regularity recovery from 64\.67 to 89\.33\. On the other hand, Claude\-Opus\-4\.7 is generally strong on irregularity cause attribution and missingness reasoning task\. However, some tasks remain difficult even with tool use, especially temporal relationship reasoning and regular\-vs\-irregular discrimination, where performance is less consistent across models\. Overall, these results suggest that for irregular time series QA, tool calling is beneficial for tasks requiring explicit numerical analysis, while higher\-level temporal reasoning remains challenging\. For more details, please refer to Appendix[C](https://arxiv.org/html/2606.15107#A3)\.

## 6Conclusion

We proposed*IRTS\-ToolBench*, a benchmark of 1,700 questions across 10 task types and 13 domains for evaluating LLMs and AI agents on irregular univariate time\-series question answering\. By combining semantically grounded irregular time\-series construction, task\-level diagnostic, and a 30\-tool library with golden tool sets,*IRTS\-ToolBench*provides a standardized evaluation for studying both answer correctness and tool\-grounded reasoning behavior\. Our evaluation results show that current LLMs already exhibit non\-trivial reasoning ability over irregular time series, especially when contextual information and tools are available, but they still struggle with higher\-level temporal reasoning and require strong scaffolding for reliable agentic tool use\. A key limitation is that our three\-layer LLM\-based generation pipeline may inherit sensitivity from LLM prompting, consensus quality, and transformation choices, so future work will strengthen pipeline robustness through broader validation and more naturally irregular source data\. We also plan to extend the benchmark toward more complex multi\-hop TSQA settings and incorporate additional modalities such as visual plots to better reflect real\-world time\-series analysis scenarios\.

## References

- Time\-imm: a dataset and benchmark for irregular multimodal multivariate time series\.External Links:2506\.10412,[Link](https://arxiv.org/abs/2506.10412)Cited by:[§A\.2](https://arxiv.org/html/2606.15107#A1.SS2.p2.1),[§B\.1](https://arxiv.org/html/2606.15107#A2.SS1.p1.1),[§1](https://arxiv.org/html/2606.15107#S1.p1.1),[§1](https://arxiv.org/html/2606.15107#S1.p2.1),[§2](https://arxiv.org/html/2606.15107#S2.p2.1),[§3\.2](https://arxiv.org/html/2606.15107#S3.SS2.p2.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-v4: towards highly efficient million\-token context intelligence\.Cited by:[Appendix C](https://arxiv.org/html/2606.15107#A3.p1.1),[§5](https://arxiv.org/html/2606.15107#S5.p3.1)\.
- W\. Du, Y\. Yang, L\. Qian, J\. Wang, and Q\. Wen \(2025\)PyPOTS: a python toolkit for machine learning on partially\-observed time series\.External Links:2305\.18811,[Link](https://arxiv.org/abs/2305.18811)Cited by:[§2](https://arxiv.org/html/2606.15107#S2.p2.1)\.
- B\. Jing, S\. Chen, L\. Zheng, B\. Liu, Z\. Li, J\. Zou, T\. Wei, Z\. Liu, Z\. Zeng, R\. Qiu, X\. Lin, Y\. Yan, D\. Fu, J\. Ni, J\. He, and H\. Tong \(2026\)TSAQA: time series analysis question and answering benchmark\.External Links:2601\.23204,[Link](https://arxiv.org/abs/2601.23204)Cited by:[Appendix B](https://arxiv.org/html/2606.15107#A2.p1.1),[§1](https://arxiv.org/html/2606.15107#S1.p1.1),[§2](https://arxiv.org/html/2606.15107#S2.p1.1),[§3\.1](https://arxiv.org/html/2606.15107#S3.SS1.p1.1),[§3\.2](https://arxiv.org/html/2606.15107#S3.SS2.p1.1)\.
- Y\. Kong, Y\. Yang, Y\. Hwang, W\. Du, S\. Zohren, Z\. Wang, M\. Jin, and Q\. Wen \(2025\)Time\-mqa: time series multi\-task question answering with context enhancement\.External Links:2503\.01875,[Link](https://arxiv.org/abs/2503.01875)Cited by:[Appendix B](https://arxiv.org/html/2606.15107#A2.p1.1),[§1](https://arxiv.org/html/2606.15107#S1.p1.1),[§2](https://arxiv.org/html/2606.15107#S2.p1.1),[§3\.1](https://arxiv.org/html/2606.15107#S3.SS1.p1.1),[§3\.2](https://arxiv.org/html/2606.15107#S3.SS2.p1.1)\.
- P\. Liu, E\. Fons, A\. Vapsi, M\. Ghassemi, S\. Vyetrenko, D\. Borrajo, V\. K\. Potluru, and M\. Veloso \(2026\)TS\-agent: understanding and reasoning over raw time series via iterative insight gathering\.External Links:2510\.07432,[Link](https://arxiv.org/abs/2510.07432)Cited by:[§1](https://arxiv.org/html/2606.15107#S1.p1.1),[§2](https://arxiv.org/html/2606.15107#S2.p3.1)\.
- Qwen Team \(2026a\)Qwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[Appendix C](https://arxiv.org/html/2606.15107#A3.p1.1),[§5](https://arxiv.org/html/2606.15107#S5.p3.1)\.
- Qwen Team \(2026b\)Qwen3\.6\-27B: flagship\-level coding in a 27B dense model\.External Links:[Link](https://qwen.ai/blog?id=qwen3.6-27b)Cited by:[Appendix C](https://arxiv.org/html/2606.15107#A3.p1.1),[§5](https://arxiv.org/html/2606.15107#S5.p3.1)\.
- Y\. Wang, P\. Lei, J\. Song, Y\. Hao, T\. Chen, Y\. Zhang, L\. Jia, Y\. Li, and Z\. Wei \(2025\)ITFormer: bridging time series and natural language for multi\-modal qa with large\-scale multitask dataset\.External Links:2506\.20093,[Link](https://arxiv.org/abs/2506.20093)Cited by:[§2](https://arxiv.org/html/2606.15107#S2.p1.1)\.
- W\. Wu, Z\. Zhang, L\. Liu, X\. Xu, J\. Zhuang, K\. Fan, Q\. Lv, J\. Liu, C\. Zhang, Z\. Yuan, S\. Hou, T\. Lin, K\. Chen, B\. Zhou, and C\. Zhang \(2026a\)SciTS: scientific time series understanding and generation with llms\.External Links:2510\.03255,[Link](https://arxiv.org/abs/2510.03255)Cited by:[§1](https://arxiv.org/html/2606.15107#S1.p1.1)\.
- X\. Wu, J\. Lu, Z\. Li, X\. Qiu, J\. Hu, C\. Guo, C\. S\. Jensen, and B\. Yang \(2026b\)TimeART: towards agentic time series reasoning via tool\-augmentation\.External Links:2601\.13653,[Link](https://arxiv.org/abs/2601.13653)Cited by:[§1](https://arxiv.org/html/2606.15107#S1.p1.1),[§2](https://arxiv.org/html/2606.15107#S2.p3.1)\.
- Y\. Yin, Z\. Xiao, M\. Li, Y\. Liu, S\. Nan, Y\. He, R\. Wang, Z\. Zhang, and Y\. Gu \(2026\)MMTS\-bench: a comprehensive benchmark for multimodal time series understanding and reasoning\.External Links:[Link](https://openreview.net/forum?id=PMKpyXk0FO)Cited by:[§1](https://arxiv.org/html/2606.15107#S1.p1.1)\.
- F\. Yu, X\. Guo, L\. Yuan, H\. Kang, H\. Zhao, L\. Qin, F\. Huang, B\. Hu, and T\. Zhou \(2026\)TSRBench: a comprehensive multi\-task multi\-modal time series reasoning benchmark for generalist models\.External Links:2601\.18744,[Link](https://arxiv.org/abs/2601.18744)Cited by:[§1](https://arxiv.org/html/2606.15107#S1.p1.1)\.
- H\. Zhao, X\. Zhang, J\. Wei, Y\. Xu, Y\. He, S\. Sun, and C\. You \(2025\)TimeSeriesScientist: a general\-purpose ai agent for time series analysis\.External Links:2510\.01538,[Link](https://arxiv.org/abs/2510.01538)Cited by:[§1](https://arxiv.org/html/2606.15107#S1.p1.1),[§2](https://arxiv.org/html/2606.15107#S2.p3.1)\.

## Appendix ATask Design

GroupTaskDescriptionQuestion TypeNumber of SamplesStandard ReasoningAnomaly DetectionTests whether the model can identify abnormal observations in irregular time series given contextual metadata\.MC250ClassificationTests coarse\-grained pattern recognition under irregular sampling conditions\. There are several areas of questions that can be asked such as classifying the input’s domain\.MC, TF150Regular vs\. IrregularDiscriminationTests the model’s ability to identify whether a given sequence is regularly or irregularly sampled\.MC, TF400Regularity RecoveryTests whether the model can identify the most plausible regular reconstruction of an irregular input\.MC150Irregularity\-SpecificReasoningCharacterizationTests the model’s ability to describe temporal properties \(trend, seasonality, stationarity\) of an irregular sequence\.MC, TF100Temporal RelationshipTests reasoning about ordering, causality, and lag relationships across asynchronous observations\.MC150Irregularity CauseAttributionTests whether the model can identify the underlying sampling mechanism \(Trigger\-Based / Artifact\-Based / Constraint\-Based\) of a given irregular sequence\.MC100Regularity–IrregularityInterface ReasoningMissingness ReasoningTests whether the model can distinguish informative gaps from non\-informative gaps in an irregular sequence\.MC100Irregularity SeverityEstimationTests whether the model can determine the interval of irregularity severity for a given sequence\.MC150Forecasting / ImputationTests forward\-looking prediction and missing\-value estimation under irregular conditions\.MC150Table 1:Task Summarization\. TF and MC denote true\_or\_false and multiple\_choices\.### A\.1Task Descriptions

We describe all1010tasks organized in three categories as shown in Table[1](https://arxiv.org/html/2606.15107#A1.T1)\. For each task we specify the input format, expected output, success criteria, and the primary challenge it poses for models and agents\.

#### A\.1\.1Standard Reasoning

Anomaly Detection\.Model is given an irregular univariate time series together with domain metadata and anomaly annotations\. The question asks whether a specific anomaly exists, where it occurs, or how many anomalies are present\. This task is formulated asMCquestion type\. This task is challenging because anomaly evidence may be embedded in sparse, non\-uniformly spaced observations, requiring models to reason about the absolute timestamps, and apply appropriate missingness handling before pattern analysis\.

Classification\.Given an irregular univariate sequence with meta context, the model must classify which domain the input time series comes from a given set of concurrent choices\. The primary challenge is that standard classification heuristics are not directly applicable to irregular input\. Models must adapt analytical reasoning to non\-uniform observation spacing or invoke appropriate resampling tools before classification\. Questions are formulated as eitherMCorTFformat\.

Regular vs\. Irregular Discrimination\.The model is given a time series, either a regular sequence or an irregular sequence transformed by our three\-layer pipeline, and asked to determine whether it is regularly or irregularly sampled\. ForMCquestions, the model selects from options such as ’regular \(uniform spacing\)’, ’irregular \(random missingness\)’, ’irregular \(event\-driven\)’, or ’irregular \(jittered timestamps\)’\. ForTFquestions, the model judges a single claim about the sampling regularity\. This task is challenging because even a visually irregular\-looking sequence may exhibit near\-regular spacing after mild jitter, requiring models to compute inter\-observation interval statistics rather than relying on surface\-level appearance\.

Regularity Recovery\.The model is given an irregular time series and asked to select the most plausible regular reconstruction fromMCoptions that differ in resampling frequency, interpolation method, or imputed value ranges\. Ground truth is derived from the original regular sequence prior to transformation\. This task requires models to first identify the appropriate regular grid, then determine which interpolation or imputation strategy best recovers the original signal, and finally evaluate candidate reconstructions against this criterion\.

### A\.2Irregularity\-Specific Reasoning

Characterization\.The model is asked to describe temporal properties of an irregular sequence, including trend direction \(upward / downward / flat\), seasonality presence and approximate period, and stationarity status\. Questions are eitherMCorTF\. This task is challenging because, in the irregular setting, models must either apply interpolation or use tools specifically designed for irregular sequence characterization before making a judgment\.

Irregularity Cause Attribution\.The model is given an irregular time series together with a domain description and asked to identify which of the nine irregularity category\(Changet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib1)\)best explains the observed missingness pattern\. Questions areMC\. Ground truth is the irregularity type assigned by our transformation pipeline\. This task is novel and challenging because distinguishing irregularity requires integrating domain knowledge with structural pattern analysis\.

Temporal Relationship\.This task evaluates the model’s ability to infer the temporal structure among irregular time series patches\. This task is formulated asMCquestion format\. Given the first chronological patch𝐱\\mathbf\{x\}, anMCquestion asks the model to choose the correct next patch from candidates\[𝐲1,𝐲2,𝐲3,𝐲4\]\[\\mathbf\{y\}\_\{1\},\\mathbf\{y\}\_\{2\},\\mathbf\{y\}\_\{3\},\\mathbf\{y\}\_\{4\}\]\. The false candidates are randomly sampled from the full dataset, but from sequences different from that of𝐱\\mathbf\{x\}\.

#### A\.2\.1Regularity–Irregularity Interface Reasoning

Missingness Reasoning\.The model is given an irregular time series and asked to distinguish whether a specific gap or pattern of missing values constitutes an informative signal or a non\-informative gap\. Questions are formulated asMCformat\. The ground truth is derived from the type of irregularity and the domain context used in the transformation\. This task is challenging because models must invoke tools to reason about gap structure before rendering a judgment\.

Irregularity Severity Estimation\.The model is given an irregular time series and asked to estimate which observation rate interval describes the sequence’s missingness density\. Question is formulated asMCformat\. Ground truth is the observation rate computed during transformation\. This task directly tests a model’s perception of missingness density\. It is challenging because models without explicit tool invocation must estimate missingness density from the raw sequence representation alone, which requires counting observed and missing entries across potentially long sequences without making positional errors\.

Forecasting\.The model is given an irregular time series up to a specified observation boundary and asked to predict the value or range of the target variable at a future timestamp\. The question is presented inMCformat with quantized answer buckets to enable exact\-match evaluation\. Ground truth is derived from the original regular series\. This task requires models to reason about the series’ trend and local dynamics while accounting for the irregular observation spacing leading up to the forecast horizon\.

## Appendix BBenchmark Construction Pipeline

Our benchmark draws from 2 source datasets\(Jinget al\.,[2026](https://arxiv.org/html/2606.15107#bib.bib2); Konget al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib3)\)\. Specifically, each collected sample is processed through a three\-layer transformation pipeline in three dimensions: \(1\) All time series sequences are transformed from regular into irregular format; \(2\) All questions are synthetically generated based on the transformed sequences and their anomaly annotations; and \(3\) Golden tool sets are constructed via a separate multi\-LLM consensus protocol\.

### B\.1Irregular Time Series Transformation

![Refer to caption](https://arxiv.org/html/2606.15107v1/figs/TSQA_transformation_diagram.png)Figure 1:Irregular Time Series Transformation\.Our transformation pipeline leverages the irregularity types from TIME\-IMM\(Changet al\.,[2025](https://arxiv.org/html/2606.15107#bib.bib1)\)as a principled decision space for selecting the appropriate transformation mechanism for each sample\. Specifically, the pipeline follows a unified three\-stage design as shown in figure[1](https://arxiv.org/html/2606.15107#A2.F1)\. \(1\)Context Enrichment: We leverage a LLM to generate an enriched context description capturing the domain, statistical features, and signal characteristics for each time series sequence\. \(2\)Taxonomy Selection: Given the enriched context and time series statistics, an LLM selects the most appropriate irregularity type and outputs a transformation plan with associated anomaly annotations along with a self\-evaluated confidence score\. \(3\)Parameter Generation: An LLM translates the transformation plan into validated numerical parameters for the relevant transformation functions\. Transformation execution then applies the generated parameters, producing an irregular time series with realistic missingness patterns and optionally jittered timestamps\.

### B\.2Question Generation Pipeline

![Refer to caption](https://arxiv.org/html/2606.15107v1/figs/question_generation.png)Figure 2:Question Generation Pipeline\.For each task type, a dedicated question generation pipeline, as shown in Figure[2](https://arxiv.org/html/2606.15107#A2.F2), produces questions conditioned on the transformed irregular time series and its associated anomaly annotations\. The generation procedure is as follows: \(1\)Primary Generation: GPT\-5\.1 is prompted with task\-specific prompts to generate the*meta\_information*for each sample\. Depending on the task design, the corresponding*question*\(MCorTF\) and answer are either generated by GPT\-5\.1 or constructed using task\-specific deterministic functions\. \(2\)Multi\-LLM Consensus: Then, three independent LLMs \(GPT\-5\.1, Claude Sonnet 4\.5, Gemini 2\.5 Flash\) evaluate whether the generated*meta\_information*,*question*, and*answer*are clear and answerable from the provided time series \(e\.g\. score\>\>threshold\)\.

### B\.3Golden Tool Set Construction

For each finalized benchmark sample, a golden tool set, as shown in Figure[3](https://arxiv.org/html/2606.15107#A2.F3), specifies the minimum set of tools that a model must invoke to correctly answer the question\. The construction protocol proceeds as follows: \(1\)Independent Proposal: Three LLMs \(GPT\-5\.1, Claude Sonnet 4\.5, Gemini 2\.5 Flash\) independently propose the required tool sequence for the question, together with a self\-confidence score and an explanation for each tool’s necessity\. \(2\)Consensus Gold Set: The Gold Set is constructed by*Majority Voting*with a*Union*fallback, which tools proposed by at least two LLMs are retained, while if no tool receives majority support, the union of all proposed tools is used\.

![Refer to caption](https://arxiv.org/html/2606.15107v1/figs/golden.png)Figure 3:Golden Tool Set Construction\.

## Appendix CEvaluation Results

We evaluate zero\-shot performance of \(1\) commercial LLMs: Claude\-Opus\-4\.7 \(without thinking, with thinking, with tool\-calling\) \(2\) open\-source LLMs: Qwen3\.5\-4B\(Qwen Team,[2026a](https://arxiv.org/html/2606.15107#bib.bib13)\), Qwen3\.6\-27B\(Qwen Team,[2026b](https://arxiv.org/html/2606.15107#bib.bib11)\), and DeepSeek\-V4\-Flash\(DeepSeek\-AI,[2026](https://arxiv.org/html/2606.15107#bib.bib12)\)\(with and without tool\-calling\)\.

Overall Results\.Our evaluation result, as shown in Table[2](https://arxiv.org/html/2606.15107#A3.T2)suggests that Qwen3\.6\-27B \(78\.59\) outperforms the other three models in all settings, while commerical model Claude\-Opus\-4\.7 receive a consistent overall accuracy in between 74 to 77 percent\.

Task\-Level Results\.For standard tasks such as anomaly detection and classification, tool calling leads to clear gains for several models\. In particular, Qwen3\.6\-27B improves from 96\.80 to 99\.60 on anomaly detection and reaches 100\.00 on classification with tool use, while Claude\-Opus\-4\.7 also achieves strong performance on both tasks\. However, smaller open\-source models remain less stable without tools, especially on anomaly detection, where Qwen3\.5\-4B obtains only 42\.40 accuracy\. This suggests that irregular time\-series questions still require reliable numerical reasoning even for seemingly conventional task types\.

Tasks that require deeper interpretation of irregular patterns show larger performance gaps across models\. Claude\-Opus\-4\.7 generally perform strongly on irregularity cause attribution and missingness reasoning\. In contrast, open\-source models show more uneven behavior: Qwen3\.5\-4B performs poorly on irregularity severity estimation, while larger models such as Qwen3\.6\-27B and DeepSeek\-V4\-Flash benefit substantially from tool calling\. For example, DeepSeek\-V4\-Flash improves from 31\.33 to 98\.67 on ISE and from 64\.67 to 89\.33 on RR when tools are enabled\.

TypeModelToolA\.D\.Char\.CLSFCAICAISEMRRvIRRTROverallCommercialClaude\-Opus\-4\.7 \(w/o thinking\)No88\.4094\.5040\.00–99\.0084\.6791\.00–37\.3362\.0074\.88Claude\-Opus\-4\.7 \(thinking\)No87\.6095\.0038\.00–97\.0095\.3391\.00–49\.3358\.6776\.72Claude\-Opus\-4\.7 \(thinking\)Yes86\.4098\.00100\.0064\.67100\.0079\.3392\.0079\.2574\.6752\.0076\.00Open\-sourceWithout tool callingQwen3\.5\-4BNo42\.4093\.0098\.6735\.3392\.0010\.0036\.0067\.2538\.0046\.0055\.18Qwen3\.6\-27BNo96\.8097\.0099\.3356\.00100\.0058\.0091\.0081\.7558\.0048\.0078\.59DeepSeek\-V4\-FlashNo59\.2092\.0090\.6751\.3384\.0031\.3366\.0052\.0064\.6746\.6760\.29With tool callingQwen3\.6\-27BYes99\.6094\.00100\.0046\.6796\.0081\.3388\.0061\.5060\.0040\.0074\.41DeepSeek\-V4\-FlashYes96\.4079\.0092\.0047\.3386\.0098\.6780\.0058\.0089\.3332\.6774\.00Table 2:Evaluation results on IRTS\-ToolBench\. Tool indicates whether tool calling is enabled\. A\.D\. denotes anomaly detection\. CLS denotes classification\. FCA denotes forecasting\. ICA denotes irregularity cause attribution\. ISE denotes irregularity severity estimation\. MR denotes missingness reasoning\. RvI denotes regular\-vs\-irregular discrimination\. RR denotes regularity recovery\. TR denotes temporal relationship\.
## Appendix DTool Library

IRTS\-ToolBench provides a 30\-tool library, shown in Table[3](https://arxiv.org/html/2606.15107#A4.T3)that enables the benchmark to support evaluation of both LLMs \(via tool\-augmented prompting\) and AI agents \(via agentic tool\-use frameworks\)\. The library is organized into two layers: \(1\)Irregularity Operators: These tools provide the operations for handling irregular time series sequences\. \(2\)Advanced Analytical Tools: This layer consists of 23 tools, providing time series analysis primitives, such as summary statistics and trend and seasonality detection\. The full tool list with descriptions is shown in Table[3](https://arxiv.org/html/2606.15107#A4.T3)\.

Tool NameDescriptionIrregularity OperatorsALIGN\_ASOFAsynchronous\-of join: aligns features observed at different timestamps onto a common time axisMISSING\_POLICYDetermines and applies an imputation or masking strategy based on missingness type \(MCAR / MAR / MNAR\)RESAMPLEResamples irregular timestamps onto a regular grid at a specified frequencyIRREGULARITY\_SCOREComputes an irregularity degree measure for a sequence \(gap variance, observation rate, timestamp CV\)GAP\_DETECTIdentifies gap segments in a time series and characterizes their length distributionEVENT\_MAPMaps irregular observations to known event timestamps for event\-driven alignmentTIMESTAMP\_JITTER\_CORRECTEstimates and corrects timestamp offsets caused by scheduling jitter or clock driftAdvanced Analytical ToolsSERIES\_INFOReturns metadata: sequence length, channel count, missing value statisticsDATAPOINT\_VALUEReturns all channel values at a specified time index or timestampSUMMARY\_STATSComputes mean, sum, max, min, std over a specified index rangeROLLING\_STATComputes rolling statistics \(mean, sum, max, min, std\) with a sliding windowQUANTILE\_VALUEReturns empirical quantile values \(e\.g\., q=0\.5 for median\) per channelVOLATILITYComputes rolling volatility \(std of first differences\) over a given windowAUTOCORRComputes autocorrelation coefficient at a specified lag per channelTREND\_CLASSIFIERClassifies trend as up / down / flat; supports global and window\-based segment analysisSEASONALITY\_DETECTORDetects periodic patterns and returns estimated period with strength indicatorCHANGE\_POINT\_DETECTORDetects structural breaks in mean or variance and returns change point indicesSTATIONARITY\_TESTTests stationarity via ADF or KPSS; returns status and test statisticsSPIKE\_DETECTORDetects spikes or dips using amplitude threshold and minimum separationNOISE\_PROFILELabels noise type \(white / red\) based on autocorrelation testsCHANNEL\_CORRELATIONComputes Pearson or Spearman correlation between two channels with optional lagCROSS\_CORRELATIONComputes cross\-correlation across multiple lags to find optimal time alignmentDTW\_DISTANCEMeasures similarity between two channels using Dynamic Time WarpingSHAPE\_SIMILARITYMeasures shape similarity between channels, invariant to amplitude scalingGRANGER\_CAUSALITYTests statistical predictability between channels within a specified lagANOMALY\_DETECTIONZero\-shot anomaly detection via reconstruction\-error\-based model \(DADA\)FORECASTERZero\-shot multivariate forecasting via lightweight foundation model \(LightGTS\)INTERPOLATEPerforms linear, spline, or forward\-fill imputation for missing value handlingDIFF\_TRANSFORMComputes first\-order differencing for stationarity analysis or data transformation tasksWAVELET\_TRANSFORMApplies wavelet decomposition for multi\-scale characterization tasks

Table 3:Full list of tools of Tool Library\.
## Appendix EExamples

In this section, we select one representative task example from each reasoning level\. For Standard Reasoning, we select*Regular vs\. Irregular Discrimination*task as example\. For Irregularity\-Specific Reasoning, we select*Irregularity Cause Attribution*task as example\. For Regularity\-Irregularity Interface Reasoning, we select*Missingness Reasoning*task as example\. Each example will consist of three inputs: \(1\) meta\_info \(optional depending on task\), \(2\) irregular time series sequence, and \(3\) question\.

IRTS\-ToolBench — Regular vs\. Irregular DiscriminationTime Series Info![[Uncaptioned image]](https://arxiv.org/html/2606.15107v1/figs/rid.png)Question Type:TFQuestion & AnswerQuestion:Based on the given time series input, is the given input an irregular time series? Respond ONLY with the letter of the correct choice \(T or F\)\.Choices:T:True\.F:False\.Answer:T

IRTS\-ToolBench — Irregularity Cause AttributionTime Series Info![[Uncaptioned image]](https://arxiv.org/html/2606.15107v1/figs/ica.png)meta\_info: Human Activity Recognition wearable motion\-sensor stream\. A single sample is absent at the second timestamp, annotated as collection\_missing\_point / nan\_insertion\_point, attributed to temporary physical detachment or sensor buffering…Question Type:MCQuestion & AnswerQuestion:Identify the most likely irregularity category for this data collection mechanism\.Choices:A:Constraint\-Based\.B:Artifact\-Based\.C:Trigger\-BasedAnswer:B

IRTS\-ToolBench — Missingness ReasoningTime Series Info![[Uncaptioned image]](https://arxiv.org/html/2606.15107v1/figs/mr.png)meta\_info: Short wearable\-sensor signals capturing human movement\. Partial observations arise from duty cycling, wireless packet loss, buffering delays, off\-body periods, or app pauses…Question Type:MCQuestion & AnswerQuestion:Identify the most likely reason for the 2nd anomalous or missing observation\.Choices:A:Resampling bug at 14\.0\.B:Wrist Impact Saturation\.C:Electromagnetic Interference\.D:Sensor Buffer Dropped a Frame at 14\.0\.Answer:D