TuiML: Machine Learning for AI Agents

# TuiML: Machine Learning for AI Agents
Source: [https://arxiv.org/html/2609.17984](https://arxiv.org/html/2609.17984)
Nilesh Verma nilesh\.verma@waikato\.ac\.nzNick Lim nick\.lim@waikato\.ac\.nzAffiliation:AI Institute, University of WaikatoHamilton 3216, New ZealandAlbert Bifet albert\.bifet@waikato\.ac\.nzAffiliation:AI Institute, University of WaikatoHamilton 3216, New ZealandAffiliation:LTCI, Télécom Paris, Institut Polytechnique de Paris19 place Marguerite Perey, 91120 Palaiseau, FranceBernhard Pfahringer bernhard\.pfahringer@waikato\.ac\.nzAffiliation:AI Institute, University of WaikatoHamilton 3216, New Zealand

###### Abstract

Machine\-learning libraries such as Weka and scikit\-learn were designed for human programmers\. Language\-model agents now use these same libraries by recalling APIs from memory and writing code, an approach that hides what a library offers, delays errors until runtime, and loses experimental state between turns\. We presentTuiML, a self\-contained machine\-learning library built for AI agents, with native algorithms across supervised, unsupervised, time\-series, data handling, tuning, and evaluation tasks\. Every component describes itself through machine\-readable metadata and parameter schemas, so an agent can search the library, inspect components, compose validated workflows, and register new ones that become discoverable in turn\. Every call is validated, seeded, and traced, and sessions export as runnable notebooks, making experiments reproducible by construction\. One specification layer drives the Model Context Protocol \(MCP\), agent\-framework adapters, aPythonAPI, a CLI, and local model serving, while data and models never leave the machine\. Benchmarks showTuiMLremains predictively competitive with scikit\-learn and Weka\. While looking like a conventional library to a human user,TuiMLis designed for agents first, allowing them to read, extend, and operate machine learning autonomously\.TuiMLis open source, with documentation at[https://tuiml\.ai](https://tuiml.ai/)\.

††shortheadings:TuiML: Machine Learning for AI Agents / Verma, Lim, Bifet, and Pfahringer††firstpage:1††editor:N/A###### keywords

AI agents, machine learning library, MCP, agentic ML workflows

## 1Introduction

Machine\-learning libraries have always been written with a particular user in mind, namely a programmer who reads the documentation, learns the API, and assembles scripts by hand\. Weka, scikit\-learn, and their successors all share this assumption\. However, language\-model agents operating these same libraries break this assumption, and the failures are measurable\. The strongest system reported in MLE\-bench reaches a Kaggle bronze\-medal threshold on only 16\.9% of competitions\([Chan et al\., 2025](https://arxiv.org/html/2609.17984#bib.bib7)\), MLAgentBench identifies hallucination and long\-horizon planning as persistent failure modes\([Huang et al\., 2024](https://arxiv.org/html/2609.17984#bib.bib6)\), and agents asked to recall large interfaces invent plausible but non\-existent calls, a problem that motivated retrieval over live documentation\([Patil et al\., 2024](https://arxiv.org/html/2609.17984#bib.bib12)\)\. The mismatch is structural, since agents are expected to hold an interface in memory that was never designed to be held that way\.

We presentTuiML, a comprehensive machine\-learning library designed from the ground up for AI agents, addressing this mismatch at the level of the library rather than the agent\. Instead of asking an agent to generate an entire program,TuiMLexposes machine\-learning operations as typed, discoverable actions, guided by the simple principle that an agent need not memorize a library when the library describes itself\. Every algorithm is a schema\-described action an agent can call directly\. A queryable registry makes components discoverable by task, data shape, or constraint\. A validated execution layer with structured errors, recorded seeds, and replayable notebook cells makes agent\-driven experiments trustworthy and reproducible by construction\. The Model Context Protocol \(MCP\) standardizes tool discovery and schema\-based invocation\([Anthropic, 2024](https://arxiv.org/html/2609.17984#bib.bib17)\), andTuiMLsupplies the machine\-learning semantics, persistent state, and local runtime behind those calls\. The design builds on the interface discipline of scikit\-learn\([Pedregosa et al\., 2011](https://arxiv.org/html/2609.17984#bib.bib1);[Buitinck et al\., 2013](https://arxiv.org/html/2609.17984#bib.bib10)\), Weka\([Hall et al\., 2009](https://arxiv.org/html/2609.17984#bib.bib4)\), MOA\([Bifet et al\., 2010](https://arxiv.org/html/2609.17984#bib.bib13)\), and River\([Montiel et al\., 2021](https://arxiv.org/html/2609.17984#bib.bib2)\)\. It supports run\-time discovery, machine\-readable parameter schemas, and task\-level execution\. Unlike approaches that consolidate agent actions into executable code\([Wang et al\., 2024](https://arxiv.org/html/2609.17984#bib.bib11)\),TuiMLconstrains actions to validated calls, eliminating silent argument errors while retaining aPythoninterface through which a code\-acting agent can import the components\.

## 2Design and Architecture

Figure 1:TuiMLseparates agent planning from machine\-learning execution through one typed contract, while data and fitted models remain local\. Solid arrows show the forward path, where the agent discovers and invokes MCP tools that the execution layer validates and runs locally\. Dashed arrows show the return path, where metrics, model identifiers, traces, and replayable notebook cells flow back to the agent\.Every component inTuiMLis addressed by a registry name and a dictionary of constructor parameters, a single scheme shared by algorithms, transformations, and metrics\. The registry resolves each name to an implementation and exposes its metadata and a JSON Schema for its parameters, extending the uniform estimator protocol of conventional libraries\([Buitinck et al\., 2013](https://arxiv.org/html/2609.17984#bib.bib10)\)with run\-time discovery\. Because discovery and execution consult the same registry, a newly registered component becomes searchable and agent\-callable immediately, and invalid names, misplaced parameters, and unsupported estimators are rejected with structured errors rather than silently substituted\. As shown in Figure[1](https://arxiv.org/html/2609.17984#S2.F1), an agent connects through an MCP client, discovers the availableTuiMLtools with their schemas, and submits workflow specifications covering the full experimental cycle, from data inspection and preprocessing through training, tuning, evaluation, serving, and notebook export\. The agent decides what to try, andTuiMLdecides how the experiment is instantiated and recorded\.

Figure 2:An agent session againstTuiML\. One request resolves into typed calls that discover, benchmark, tune, and serve a model, withTuiMLvalidating, seeding, and recording each step\.Figure[2](https://arxiv.org/html/2609.17984#S2.F2)traces a representative session, in which the agent queries the registry for candidate classifiers, benchmarks them under cross\-validation, tunes the winner, and serves the fitted model, with every argument validated before any model is fitted\. Reproducibility follows from the same contract\.TuiMLrecords each successful call with its seed and model lineage and exports the sequence as executable notebook cells, so every session yields a runnable artifact by construction\([Pineau et al\., 2021](https://arxiv.org/html/2609.17984#bib.bib16)\)\. Compact JSON Lines traces retain timestamps, arguments, duration, and status without copying large arrays\. Datasets and fitted models never leave the machine, with the agent receiving schemas and structured summaries rather than raw data\.

## 3Implementation and Evaluation

TuiMLis aPython3\.10\+ library built onNumPy\([Harris et al\., 2020](https://arxiv.org/html/2609.17984#bib.bib3)\), with performance\-critical kernels compiled fromC\+\+throughpybind11\. It provides classification, regression, clustering, association mining, anomaly detection, and time\-series modeling, together with preprocessing, feature engineering, metrics, statistical tests, tuning, and reporting\. Optional scikit\-learn\([Pedregosa et al\., 2011](https://arxiv.org/html/2609.17984#bib.bib1)\)and CapyMOA\([Gomes et al\., 2025](https://arxiv.org/html/2609.17984#bib.bib14)\)learners are exposed through namespaced wrappers\. Unlike AutoML systems that prescribe a search policy\([Feurer et al\., 2015](https://arxiv.org/html/2609.17984#bib.bib5);[Trirat et al\., 2025](https://arxiv.org/html/2609.17984#bib.bib8)\),TuiMLsupplies a stable action space on which such systems can be built\. To evaluate the runtime, we compare the thirteen algorithms shared byTuiML, scikit\-learn, and Weka on 51 TabArena v0\.1 datasets\([Erickson et al\., 2025](https://arxiv.org/html/2609.17984#bib.bib9)\)hosted on OpenML\([Vanschoren et al\., 2014](https://arxiv.org/html/2609.17984#bib.bib15)\), with hyperparameters and ten\-fold splits aligned across frameworks and runs executed in isolated, single\-threaded processes\. As shown in Figure[3](https://arxiv.org/html/2609.17984#S3.F3),TuiMLachieves accuracy comparable to scikit\-learn and Weka while being competitive in runtime and memory consumption, thus demonstrating that an agent\-facing interface orchestrate a competitive runtime\.

Figure 3:Matched results on 51 TabArena datasets, averaged over 3,318 runs\. Accuracy covers the 2,587 classification runs, and Weka memory includes its JVM baseline\.
## 4Conclusion

TuiMLprovides a self\-describing machine\-learning library in which components are discoverable, calls are validated, and experiments are traceable, backed by native, performance\-oriented implementations across classification, regression, clustering, anomaly detection, association mining, and time\-series modeling\. Documentation, tutorials, contribution guides, and the full benchmark protocol are available at[https://tuiml\.ai](https://tuiml.ai/)\. The library ships with a comprehensive test suite and cross\-platform builds for Linux, macOS, and Windows\.TuiMLremains alpha software, and a syntactically valid call can still encode a statistically inappropriate experiment, so results warrant the usual scrutiny\. Future work includes a controlled comparison of schema\-guided calls against free\-form code under matched models and budgets, broader algorithm coverage, and measurement of tool\-selection accuracy and token cost\. We expectTuiMLto evolve as a comprehensive tool for research and real\-world applications as agent interfaces mature\.

###### acknowledgments\-disclosure\-of\-funding\.

TuiMLwas developed at Te Ipu o te Mahara, the Artificial Intelligence Institute of the University of Waikato, and builds on the tradition of open machine\-learning software established there by Weka\. This work was supported in part by the TAIAO programme \(Time\-Evolving Data Science and Artificial Intelligence for Advanced Open Environmental Science\), funded by the New Zealand Ministry of Business, Innovation and Employment\.

## References

- Anthropic \(2024\)AnthropicIntroducing the Model Context Protocol\.Note:[https://www\.anthropic\.com/news/model\-context\-protocol](https://www.anthropic.com/news/model-context-protocol)Accessed 12 August 2026Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p2.1)\.
- Bifetet al\.\(2010\)A\. Bifet, G\. Holmes, R\. Kirkby, and B\. PfahringerMOA: massive online analysis\.Journal of Machine Learning Research11,pp\. 1601–1604\.Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p2.1)\.
- Buitincket al\.\(2013\)L\. Buitinck, G\. Louppe, M\. Blondel, F\. Pedregosa, A\. Mueller, O\. Grisel, V\. Niculae, P\. Prettenhofer, A\. Gramfort, J\. Grobler, R\. Layton, J\. VanderPlas, A\. Joly, B\. Holt, and G\. VaroquauxAPI design for machine learning software: experiences from the scikit\-learn project\.InECML PKDD Workshop: Languages for Data Mining and Machine Learning,pp\. 108–122\.Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p2.1),[§2](https://arxiv.org/html/2609.17984#S2.p1.1)\.
- Chanet al\.\(2025\)J\. S\. Chan, N\. Chowdhury, O\. Jaffe, J\. Aung, D\. Sherburn, E\. Mays, G\. Starace, K\. Liu, L\. Maksin, T\. Patwardhan, A\. Madry, and L\. WengMLE\-bench: evaluating machine learning agents on machine learning engineering\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p1.1)\.
- Ericksonet al\.\(2025\)N\. Erickson, L\. Purucker, A\. Tschalzev, D\. Holzmüller, P\. Desai, D\. Salinas, and F\. HutterTabArena: a living benchmark for machine learning on tabular data\.InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track,Cited by:[§3](https://arxiv.org/html/2609.17984#S3.p1.1)\.
- Feureret al\.\(2015\)M\. Feurer, A\. Klein, K\. Eggensperger, J\. T\. Springenberg, M\. Blum, and F\. HutterEfficient and robust automated machine learning\.InAdvances in Neural Information Processing Systems,Vol\.28\.Cited by:[§3](https://arxiv.org/html/2609.17984#S3.p1.1)\.
- Gomeset al\.\(2025\)H\. M\. Gomes, A\. Lee, N\. Gunasekara, Y\. Sun, G\. W\. Cassales, J\. J\. Liu, M\. Heyden, V\. Cerqueira, M\. Bahri, Y\. S\. Koh, B\. Pfahringer, and A\. BifetCapyMOA: efficient machine learning for data streams in Python\.arXiv preprint arXiv:2502\.07432\.Cited by:[§3](https://arxiv.org/html/2609.17984#S3.p1.1)\.
- Hallet al\.\(2009\)M\. Hall, E\. Frank, G\. Holmes, B\. Pfahringer, P\. Reutemann, and I\. H\. WittenThe WEKA data mining software: an update\.SIGKDD Explorations11\(1\),pp\. 10–18\.External Links:[Document](https://dx.doi.org/10.1145/1656274.1656278)Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p2.1)\.
- Harriset al\.\(2020\)C\. R\. Harris, K\. J\. Millman, S\. J\. van der Walt,et al\.Array programming with NumPy\.Nature585\(7825\),pp\. 357–362\.External Links:[Document](https://dx.doi.org/10.1038/s41586-020-2649-2)Cited by:[§3](https://arxiv.org/html/2609.17984#S3.p1.1)\.
- Huanget al\.\(2024\)Q\. Huang, J\. Vora, P\. Liang, and J\. LeskovecMLAgentBench: evaluating language agents on machine learning experimentation\.InProceedings of the 41st International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p1.1)\.
- Montielet al\.\(2021\)J\. Montiel, M\. Halford, S\. M\. Mastelini, G\. Bolmier, R\. Sourty, R\. Vaysse, A\. Zouitine, H\. M\. Gomes, J\. Read, T\. Abdessalem, and A\. BifetRiver: machine learning for streaming data in Python\.Journal of Machine Learning Research22\(110\),pp\. 1–8\.Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p2.1)\.
- Patilet al\.\(2024\)S\. G\. Patil, T\. Zhang, X\. Wang, and J\. E\. GonzalezGorilla: large language model connected with massive APIs\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 126544–126565\.Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p1.1)\.
- Pedregosaet al\.\(2011\)F\. Pedregosa, G\. Varoquaux, A\. Gramfort, V\. Michel, B\. Thirion, O\. Grisel, M\. Blondel, P\. Prettenhofer, R\. Weiss, V\. Dubourg, J\. Vanderplas, A\. Passos, D\. Cournapeau, M\. Brucher, M\. Perrot, and E\. DuchesnayScikit\-learn: machine learning in Python\.Journal of Machine Learning Research12,pp\. 2825–2830\.Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p2.1),[§3](https://arxiv.org/html/2609.17984#S3.p1.1)\.
- Pineauet al\.\(2021\)J\. Pineau, P\. Vincent\-Lamarre, K\. Sinha, V\. Larivière, A\. Beygelzimer, F\. d’Alché\-Buc, E\. Fox, and H\. LarochelleImproving reproducibility in machine learning research \(a report from the NeurIPS 2019 reproducibility program\)\.Journal of Machine Learning Research22\(164\),pp\. 1–20\.Cited by:[§2](https://arxiv.org/html/2609.17984#S2.p2.1)\.
- Triratet al\.\(2025\)P\. Trirat, W\. Jeong, and S\. J\. HwangAutoML\-Agent: a multi\-agent LLM framework for full\-pipeline AutoML\.InProceedings of the 42nd International Conference on Machine Learning,Cited by:[§3](https://arxiv.org/html/2609.17984#S3.p1.1)\.
- Vanschorenet al\.\(2014\)J\. Vanschoren, J\. N\. van Rijn, B\. Bischl, and L\. TorgoOpenML: networked science in machine learning\.ACM SIGKDD Explorations Newsletter15\(2\),pp\. 49–60\.External Links:[Document](https://dx.doi.org/10.1145/2641190.2641198)Cited by:[§3](https://arxiv.org/html/2609.17984#S3.p1.1)\.
- Wanget al\.\(2024\)X\. Wang, Y\. Chen, L\. Yuan, Y\. Zhang, Y\. Li, H\. Peng, and H\. JiExecutable code actions elicit better LLM agents\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 50208–50232\.Cited by:[§1](https://arxiv.org/html/2609.17984#S1.p2.1)\.