MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model
Summary
MLE-World: a world model finetuned from Qwen3.5-4B that predicts ML experiment outcomes (error traces, logs, validation scores) without execution, enabling affordable test-time scaling for AI research agents. It matches real terminal outputs on 77.9% of held-out programs and boosts mean performance by up to 22.1% on MLE-bench Lite tasks.
View Cached Full Text
Cached at: 10/28/26, 05:25 AM
Paper page - MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model
Source: https://huggingface.co/papers/2610.37175
Abstract
A world model predicts machine learning program outcomes to enable affordable test-time scaling for AI research agents, improving performance with fewer real executions.
AI research agents can propose, execute, and improve programs that solve machine learning tasks. However, executing each candidate program requires training and evaluating ML models, limiting how many candidates an agent can explore at each step. We introduce MLE-World, aworld modelfinetuned fromQwen3.5-4Bthat predicts a program’s terminal output (error traces, training logs, and validation scores) from its code and data description without executing it, together with aconfidence scoreestimating the prediction’s correctness. Because a prediction costs one inference call rather than a training run, MLE-World makestest time scaling(TTS) affordable for these agents. We build five TTS policies, guided by MLE-World, that compare several candidates per step by their predicted outputs:Sequential revision,Best-of-N,Pairwise judging,Fusion, and aUCBstyle rule that also adds an exploration bonus from thisconfidence score. Since predictions can be wrong, we run a candidate for real only when itsconfidence scoreis low. On 17MLE-bench Litetasks, MLE-World matches the real terminal output on 77.9% of held out programs, versus 41.1% for much larger prompted models. MLE-World guided TTS achieves up to 22.1% higher observed mean performance than the agent without aworld model, with less real execution. Project webpage: https://mle-world.github.io/.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2610\.37175
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.37175 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.37175 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.37175 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modeling
WMLLM is a self-evolving optimization-agent framework that uses predict-then-act world modeling with large language models to improve sample efficiency in black-box optimization, achieving state-of-the-art results in multi-objective molecular optimization.
MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
提出了 MLExpert，一个 35B 混合专家（MoE）智能体模型，通过分层监督微调和窗口化交互强化学习（WIRL）自主优化长周期机器学习工程工作流，在 MLE-Bench Lite 上将基模型奖牌率从 37.9% 提升至 72.7%，媲美更大的前沿模型。
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
OpenAI introduces MLE-bench, a benchmark of 75 Kaggle ML competitions to evaluate AI agents on real-world ML engineering tasks. The best setup, o1-preview with AIDE scaffolding, achieves at least a Kaggle bronze medal in 16.9% of competitions.
Language Models Learn to Improve Language Models
LM2LM trains LLMs with reinforcement learning to propose effective language-model improvements, using a frozen research world model as a scalable reward signal with periodic real executions for reliability, enabling Qwen3.8-27B to outperform GPT-6-sol at 33% lower data-generation cost.
Q-Learning With World Models
This paper introduces QWM, a framework that integrates world models with Q-learning to enhance sample efficiency in reinforcement learning by using imagined trajectories for action selection without compromising training on real data. It demonstrates significant improvements over state-of-the-art methods on manipulation benchmarks.