experiment-prediction

MLE-World: a world model finetuned from Qwen3.5-4B that predicts ML experiment outcomes (error traces, logs, validation scores) without execution, enabling affordable test-time scaling for AI research agents. It matches real terminal outputs on 77.9% of held-out programs and boosts mean performance by up to 22.1% on MLE-bench Lite tasks.

该论文构建了包含2882个LLM提出的ML实验想法的数据集，评估五个前沿大模型预测实验结果的准确性，发现最佳模型Claude Opus 5.5仅解释约一半方差（R²=0.49），且所有模型都系统性低估了变化幅度，表明当前LLM对ML想法的直觉有用但有限。