Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station
Summary
该论文提出 Station 系统,让 AI 代理在禁用网络访问的条件下自主进行开放式科学发现,在基于三篇 ICLR 口头报告论文构建的任务上平均恢复了 62.7% 的预定义子发现,远超 AI Scientist-v2(20.6%)和 Codex Multiagent-v2(15.4%),并开源了代码与完整研究记录。
View Cached Full Text
Cached at: 10/09/26, 06:24 PM
Paper page - Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station
Source: https://huggingface.co/papers/2610.08927 Can AI agents make open-ended scientific discovery?
We built open-ended research tasks from three ICLR oral papers, giving agents the research questions and experimental setups while withholding the original findings and disabling web access.
Station recovered 62.7% of the predefined sub-discoveries on average, compared with 20.6% for the strongest tested AI Scientist-v2 configuration and 15.4% for Codex Multiagent-v2. The comparison matches cumulative experiment time; token budgets and model teams differ.
Agents independently choose research directions, conduct experiments, and build on reviewed findings in a shared knowledge base. Supervisor guidance and periodic Meta Reflection encourage sustained investigation instead of early pivots to easier questions.
On two additional exploratory tasks, agents also produced findings matching concurrent human research. In subliminal learning, they found that restricting LoRA training to early layers could restore failed trait transfer.
We are releasing the code and full research records so others can inspect the discoveries—and the failures. A key remaining challenge is scientific judgment: agents still spend substantial effort on questions human researchers find uninteresting.
Paper:https://arxiv.org/abs/2610.08927 Code:https://github.com/dualverse-ai/station-open-reseach Explore the research records:https://dualverse-ai.github.io/station-open-reseach_data/
Similar Articles
[2610.08927] Can AI Agents Make Open-Ended Scientific Discovery? Evidence from Station
该论文提出Station——一个多智能体开源环境,用于评估AI代理进行开放式科学发现的能力。通过添加Supervisor和Meta Reflection机制,Station在重现ICLR oral论文发现的基准上平均达到62.7%,显著超越Codex Multiagent-v2(15.4%)和AI Scientist-v2(14.4-20.6%)。
@dair_ai: Banger paper from Meta on AI research agents. (bookmark it) If you build AI-scientist systems, this one is worth your t…
Meta 的 IdeaScientist 论文将科研构思拆解为缺口发现、创新与报告撰写三个角色并分别用强化学习训练,基于 Qwen3.6-27B 在 2.77M 理念语料库上检索,超越最强开源 autoresearch 基线 14.0%,并优于 Claude Code SDK 与 Codex SDK 方案最多 5.9%。
AI agents built a scientific literature together. That literature led to novel discoveries on 5 of 12 mathematical problems.
The Station is an open-world multi-agent environment where AI agents autonomously build a scientific literature, leading to novel discoveries on five mathematical problems out of twelve evaluated.
Autonomous Mathematical Discovery in an Open-World Multi-Agent Environment
The paper presents theStation, an open-world multi-agent environment where AI agents autonomously collaborate on mathematical research, achieving novel results on several open problems and releasing all dialogues, proofs, and code for transparency.
AI Coding Agents Can Reproduce Social Science Findings
This paper introduces SocSci-Repro-Bench, a benchmark of 221 tasks to evaluate AI coding agents' ability to reproduce social science findings from original data and code. It finds that frontier agents like Claude Code and Codex can reproduce a large share of results, with Claude substantially outperforming Codex, and that results are not primarily driven by memorization.