The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements
Summary
The paper introduces the Resolution Hypothesis, arguing that agent evolution stalls because noisy evaluations fail to reliably identify true improvements rather than because proposals are poor. Across seven evolution methods and six models (10M+ LLM calls), it shows candidate selection performs no better than random and published method rankings often fall within run-to-run variance.
View Cached Full Text
Cached at: 11/19/26, 03:39 AM
Paper page - The Resolution Hypothesis: Agent Evolution’s Key Is Identifying Improvements
Source: https://huggingface.co/papers/2611.23068
Abstract
Agent evolution is often bottlenecked by noisy evaluations that cannot reliably identify better candidate artifacts or rank methods, so progress requires making true improvements detectable and accumulable.
Agent evolutionfollows a simple loop: aproposerwritescandidate artifacts(prompts,skills,memories) andselectionaccepts those that score higher. This loop has shown promise, yet practitioners face a puzzle: methods trade wins across papers, and evolution often fails to bootstrap on certain tasks. We trace this puzzle to theResolution Hypothesis:agent evolutionstalls not because methods fail to propose better artifacts, but becausenoisy evaluationscannot reliably identify them. Across seven evolution methods and six models (over 10MLLM calls), we find support for the hypothesis at both levels where evaluation scores are used. At the candidate level (choosing artifacts), helpfulcandidate artifactsexist, butselectionis no better than random at telling them from harmful ones, so errors accumulate across rounds. At the method level (ranking methods), typical margins of a few percentage points are smaller than the difference between two independent runs of the same method, so published rankings often rest on luck. Together, these findings indicate thatagent evolutionis often bottlenecked by reliably identifying improvements. This calls for a shift in how we develop and evaluateself-improving agents, from designing ever more candidate improvement systems to making true improvements detectable and accumulable.
View arXiv pageView PDFProject pageAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2611.23068 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2611.23068 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2611.23068 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Evolution or Illusion? Rethinking Evaluation in LLM Evolutionary Search
This paper critiques single-point evaluations for LLM evolutionary search, showing that optimal performance depends on varying seeds and iterations, and proposes a budget-grid evaluation protocol.
Rethinking the Evaluation of Harness Evolution for Agents
This paper rethinks how automatic harness evolution for agents should be evaluated, showing that gains may be due to increased compute rather than genuine improvements, and that evolved harnesses transfer poorly to unseen tasks.
@rohanpaul_ai: Better self-improving agents need better solvers, not bigger update-writing models. This challenges the common habit of…
This paper disentangles the roles of evolver and agent in self-improving LLM agents, showing that a small evolver can write sufficiently good updates, while a mid-tier agent benefits most from using them. It recommends using the strongest model as the task executor, not the update writer.
Rethinking the Evaluation of Harness Evolution for Agents
This paper re-evaluates the methodology of automatic harness evolution for LLM agents, highlighting that its gains may stem from additional test-time search rather than improved harness design, and that evaluation on the same benchmark risks overfitting. Experiments show that harness evolution does not consistently outperform simpler test-time scaling methods.
MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution
MetaEvo proposes a two-stage framework for continual evolution of LLM-based agents, using preference-based optimization to enhance principle abstraction and modular architecture for experience reuse, outperforming strong baselines on reasoning benchmarks.