Stop obsessing over agent strategies? Tsinghua’s EurekAgent finds: the real bottleneck for autonomous scientific research is environment design, not smarter agents.

This overturns the mainstream view — as model capabilities increase, the reliability, cost control, and scalability of agents have become bottlenecked by environment engineering. The paper systematically addresses this through four dimensions: permissions engineering for fine-grained control of operational scope and isolated evaluation to prevent reward hacking; artifact engineering using filesystem + Git for collaboration and experiment management; budget engineering to allocate compute/API budgets balancing depth and cost; and human-in-the-loop engineering to design low-friction supervision interfaces. It solved a new optimal 26-circle packing arrangement for less than $11.

Agent Environment Engineering is All You Need for Autonomous Scientific Discovery

Source: https://arxiv.org/html/2606.13662 Amy Xin1, Jiening Siow1, Junjie Wang1, Zijun Yao1,Fanjin Zhang2, Jian Song1, Lei Hou1, Juanzi Li11Department of Computer Science and Technology, Tsinghua University2School of Information, Renmin University of China{xin-x25, xiaojn25}@mails.tsinghua.edu.cn

Abstract

LLM-based agents have shown increasing potential in automating scientific discovery. Given an optimizable metric and an execution environment, they can propose, validate, and iterate scientific solutions, and have produced results that outperform human-designed approaches. As model capabilities continue to improve, we argue that the bottleneck for autonomous scientific discovery is shifting from prescribing agent workflows to designing agent environments—the resources, constraints, and interfaces that shape agent behavior. We frame this asenvironment engineering: building environments that amplify productive behaviors, such as open-ended exploration, systematic artifact management, and inter-agent collaboration, while suppressing harmful behaviors, such as reward hacking and high-friction human oversight. We presentEurekAgent, an environment-engineered agent system for metric-driven autonomous scientific discovery.EurekAgentengineers the environment along four dimensions: permissions engineering for bounded agent execution and isolated evaluation; artifact engineering for filesystem and Git-based collaboration; budget engineering for budget-aware exploration; and human-in-the-loop engineering for easy human supervision and intervention.EurekAgentsets new state-of-the-art results on multiple mathematics, kernel engineering, and machine learning tasks, including new state-of-the-art 26-circle packing results discovered with less than $11 in total API cost. We open-source our code and results111https://github.com/THU-Team-Eureka/EurekAgent, and call for environment engineering as a core research direction for developing reliable autonomous research agents.

Refer to captionFigure 1:EurekAgentscore evolution progress on the 26-circle packing problem.MathematicsKernel Eng.Machine LearningCircle Packing (↑\uparrow)Erdős’ Min. Overlap (↓\downarrow)1st Autocorr. Ineq. (↓\downarrow)TriMul (↓\downarrow)MLE-Bench (↑\uparrow)Prev. Best Human∼\sim2.634[5 (https://arxiv.org/html/2606.13662#bib.bib3)]0.380927[7 (https://arxiv.org/html/2606.13662#bib.bib1)]1.509730[18 (https://arxiv.org/html/2606.13662#bib.bib2)]2096.04μs2096.04\,\mu\mathrm{s}N/APrev. Best AI2.635986[27 (https://arxiv.org/html/2606.13662#bib.bib4)]0.380876[30 (https://arxiv.org/html/2606.13662#bib.bib5)]1.502863[30 (https://arxiv.org/html/2606.13662#bib.bib5)]2247.78μs2247.78\,\mu\mathrm{s}[30 (https://arxiv.org/html/2606.13662#bib.bib5)]71.43%[31 (https://arxiv.org/html/2606.13662#bib.bib6)]EurekAgent2.6359990.3808701.5028612005.03 μs85.71%

Table 1:An overview ofEurekAgent’s performance on metric-driven research tasks across mathematics, kernel engineering, and machine learning.EurekAgentsets new state-of-the-art across all mathematics and kernel engineering tasks, and ranks first on the evaluated MLE-Bench subset. (↑\uparrow) denotes higher is better, while (↓\downarrow) denotes lower is better.Prev. Best HumanandPrev. Best AIdenote the best published human and AI results beforeEurekAgent.## 1Introduction

Large language models are increasingly transforming scientific discovery from manual trial-and-error to computational exploration: in domains where research progress can be measured by an optimizable metric, LLM-based agents can autonomously propose hypotheses, run experiments, observe feedback, and iterate solutions, reducing human effort in method tuning while largely expanding the scale of exploration. LLM-based agents have already produced strong results in tasks across domains such as mathematics, algorithms, kernel engineering, and machine learning engineering(Jianget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib7); Novikovet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib8); Langeet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib9); Wanget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib4); Yuksekgonulet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib5); Ouyanget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib10); Chanet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib11); Yanget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib12); Zhanget al.,2026 (https://arxiv.org/html/2606.13662#bib.bib6); Liet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib20)). We envision this as an emerging paradigm shift in scientific research: humans increasingly focus on selecting valuable directions, formulating meaningful metrics, and supervising validity, while agents execute large volumes of methodological exploration.

Most existing autonomous research systems realize this vision by prescribing research-specific agentic workflows. Evolutionary systems such as AlphaEvolve explicitly maintain populations of candidate programs and use evaluator feedback to guide mutation and selection(Novikovet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib8); Langeet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib9); Liuet al.,2026b (https://arxiv.org/html/2606.13662#bib.bib13)). Machine learning systems such as AIDE organize exploration around solution trees, feedback loops, and role-specialized agents(Jianget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib7); Yanget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib12)). More recent systems introduce structured debate, periodic self-review, and self-learning modules(Liuet al.,2026a (https://arxiv.org/html/2606.13662#bib.bib17); Quet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib18)). While these designs can be effective, they also encode strong assumptions about how research should proceed. As general-purpose coding agents like Claude Code and Codex become stronger, recent evidence suggests that much of the useful capability may already reside in the base agent: given a clear research task and an optimizable metric, these agents can already discover new state-of-the-art scientific solutions(Liuet al.,2026c (https://arxiv.org/html/2606.13662#bib.bib15); Karpathy,2026 (https://arxiv.org/html/2606.13662#bib.bib16)). On ResearchClawBench(Xuet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib19)), a benchmark of 40 research tasks across 10 diverse domains, both Claude Code and Codex, used as standalone general-purpose agents, outperform all evaluated research-specific agent systems.

However, task performance alone does not make reliable autonomous researchers. Scientific discovery requires rigor, reproducibility, and inspectability, yet agents may contaminate evaluations, manipulate artifacts, or fail to follow procedural constraints. Such reward-hacking and observability failures have already been reported in agentic research systems(Luoet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib21); Kokoromyti,2026 (https://arxiv.org/html/2606.13662#bib.bib22); Anthropic,2026 (https://arxiv.org/html/2606.13662#bib.bib23)). Therefore, trusting agents without environmental constraints can lead to impressive but unreliable results.

These observations suggest that as general-purpose agents become more capable, the bottleneck for autonomous scientific discovery is shifting from prescribing agent behavior through detailed workflows to engineering the environments in which agents operate. We frame this asenvironment engineering. This echoes Gibson’s theory of affordances in ecological psychology: an environment shapes the possibilities for action available to an actor, “either for good or ill”(Gibson,1979 (https://arxiv.org/html/2606.13662#bib.bib47)). For scientific discovery, a well-engineered environment should suppress harmful affordances such as evaluation tampering and artifact manipulation, while amplifying productive affordances such as free exploration, accurate rewards, inter-agent coordination, and easy human supervision. The analogy is a capable PhD student: productivity comes not from minute-by-minute instructions, but from accountability, research autonomy, accurate feedback, peer collaboration, and mentor supervision.

We presentEurekAgent, an agent system for autonomous scientific discovery that coordinates off-the-shelf CLI agents through four environment engineering dimensions: (1) permissions engineering, to expose useful capabilities and resources while preventing research-integrity violations; (2) artifact engineering, to structure solutions, logs, and evaluation results as shared progress memory; (3) budget engineering, to enable budget-aware exploration with runtime and compute boundaries; and (4) human-in-the-loop engineering, to support easy human supervision and intervention. Within this environment, the agent remains free to select its own research workflows and strategies.

We evaluateEurekAgenton metric-driven research tasks spanning mathematics, kernel engineering, and machine learning engineering. Using off-the-shelf CLI agents and environment-level design,EurekAgentachieves new state-of-the-art results across all mathematics and kernel engineering tasks, and ranks first on the evaluated MLE-Bench subset. Furthermore, with Claude Code as the CLI agent and GLM-5.1 as the base model,EurekAgentachieves new state-of-the-art results on the three mathematics tasks with an average API cost below $17, where the 26-circle packing task achieves the lowest API cost of $11. We call for environment engineering as a core research direction for building capable, efficient, and responsible autonomous research agents.

2Related Work

2.1Agents for Scientific Discovery

Autonomous research agents have attracted growing interest to accelerate scientific with large-scale computational exploration. Systems such as The AI Scientist aim to automate scientific research in an end-to-end manner, covering stages such as idea generation, experimentation, and paper writing(Luet al.,2024 (https://arxiv.org/html/2606.13662#bib.bib26)). Within this broader vision, one especially concrete direction is scientific discovery with verifiable objectives and optimizable metrics, where agents autonomously explore and evolve solutions through evaluator feedback. In machine learning engineering, systems such as AIDE, R&D-Agent, AIBuildAI, MLE-STAR, and ML-Master formulate progress as iterative code development guided by validation scores(Jianget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib7); Yanget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib12); Zhanget al.,2026 (https://arxiv.org/html/2606.13662#bib.bib6); Namet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib36); Zhuet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib37)). In algorithmic and mathematical discovery, training-free solution evolution methods such as FunSearch, AlphaEvolve, ShinkaEvolve, EvoX, AdaEvolve, and OpenEvolve use LLMs to propose or mutate candidate programs under evaluator-guided selection(Romera-Paredeset al.,2024 (https://arxiv.org/html/2606.13662#bib.bib25); Novikovet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib8); Langeet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib9); Liuet al.,2026b (https://arxiv.org/html/2606.13662#bib.bib13); Cemriet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib41); Sharma,2025 (https://arxiv.org/html/2606.13662#bib.bib42)). More recently, test-time training systems such as ThetaEvolve and TTT-Discover further use the optimizable metric as a reward signal to update the model during exploration(Wanget al.,2025 (https://arxiv.org/html/2606.13662#bib.bib4); Yuksekgonulet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib5)). These systems demonstrate the power of evaluator-guided discovery, but they typically use fixed workflows to prescribe core agent behaviors such as proposal, mutation, selection, or reflection.EurekAgentinstead uses strong general-purpose CLI agents as basic nodes, and focuses on engineering an environment that lets agents exercise their own capabilities reliability.

2.2Agent Environments and Research Integrity

As agents become more autonomous, the surrounding environment becomes a central determinant of reliability. Some recent systems have begun to recognize the importance of environment reliability and introduce safeguards for specific failure modes. For example, MLE-STAR adds leakage checking for machine learning pipelines, and CORAL hides grader code behind an evaluation interface(Namet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib36); Quet al.,2026 (https://arxiv.org/html/2606.13662#bib.bib18)). At the same time, analyses of real reward-hacking incidents show that agents can exploit weak evaluation protocols, contaminate evidence, or violate procedural assumptions(Luoet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib21); Kokoromyti,2026 (https://arxiv.org/html/2606.13662#bib.bib22); Anthropic,2026 (https://arxiv.org/html/2606.13662#bib.bib23)). Instruction-following failures in complex agentic settings further suggest that reliability cannot be delegated entirely to prompt engineering(Qiet al.,2025 (https://arxiv.org/html/2606.13662#bib.bib24)). Some existing work therefore explores environment design to avoid common failures, but these are usually introduced as task-specific safeguards.EurekAgentmakes environment engineering the central design objective: it organizes permissions, artifacts, budgets, and human oversight as first-class mechanisms for supporting open-ended agent exploration while preserving evaluator integrity, traceability, and reproducibility.

Refer to captionFigure 2:Overview ofEurekAgent. Given task inputs and budgets,EurekAgentexecutes a prepare stage followed by repeated propose and parallel implement stages, while the environment engineering layer provides secure evaluation, artifact memory, budget control, and human oversight.

3EurekAgent

In this section, we present the system design ofEurekAgent. Figure2 (https://arxiv.org/html/2606.13662#S2.F2)summarizes the overall architecture. We first overview the overall system loop (3.1 (https://arxiv.org/html/2606.13662#S3.SS1)), then detail the environment engineering designs (3.2 (https://arxiv.org/html/2606.13662#S3.SS2)).

3.1System Overview

EurekAgentis an environment-engineered agent system for metric-driven research tasks. Given a problem description, a hidden evaluation script, a submission-format specification document, optional initial code, and time and API cost budgets,EurekAgentcoordinates multiple sessions of off-the-shelf CLI agents to autonomously propose and iterate high-scoring solutions. Instead of prescribing a detailed research workflow,EurekAgentengineers an outer environment that organizes agent activity through a simple three-stage loop:

Prepare→[Proposer→{Implementr,p}p=1Pr]r=1R,Pr≤P,\textsc{Prepare}\rightarrow\left[\textsc{Propose}_{r}\rightarrow\{\textsc{Implement}_{r,p}\}_{p=1}^{P_{r}}\right]_{r=1}^{R},\quad P_{r}\leq P,whereRRis the maximum number of iteration rounds andPPis the maximum number of parallel implementation sessions per implement stage, both adjustable by the user. Each round consists of one proposal session followed by up toPPparallel implementation sessions. Across stages and rounds, the env