The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements

Hugging Face Daily Papers Papers

Summary

The paper introduces the Resolution Hypothesis, arguing that agent evolution stalls because noisy evaluations fail to reliably identify true improvements rather than because proposals are poor. Across seven evolution methods and six models (10M+ LLM calls), it shows candidate selection performs no better than random and published method rankings often fall within run-to-run variance.

Agent evolution follows a simple loop: a proposer writes candidate artifacts (prompts, skills, memories) and selection accepts those that score higher. This loop has shown promise, yet practitioners face a puzzle: methods trade wins across papers, and evolution often fails to bootstrap on certain tasks. We trace this puzzle to the Resolution Hypothesis: agent evolution stalls not because methods fail to propose better artifacts, but because noisy evaluations cannot reliably identify them. Across seven evolution methods and six models (over 10M LLM calls), we find support for the hypothesis at both levels where evaluation scores are used. At the candidate level (choosing artifacts), helpful candidate artifacts exist, but selection is no better than random at telling them from harmful ones, so errors accumulate across rounds. At the method level (ranking methods), typical margins of a few percentage points are smaller than the difference between two independent runs of the same method, so published rankings often rest on luck. Together, these findings indicate that agent evolution is often bottlenecked by reliably identifying improvements. This calls for a shift in how we develop and evaluate self-improving agents, from designing ever more candidate improvement systems to making true improvements detectable and accumulable.
Original Article
View Cached Full Text

Cached at: 11/19/26, 03:39 AM

Paper page - The Resolution Hypothesis: Agent Evolution’s Key Is Identifying Improvements

Source: https://huggingface.co/papers/2611.23068

Abstract

Agent evolution is often bottlenecked by noisy evaluations that cannot reliably identify better candidate artifacts or rank methods, so progress requires making true improvements detectable and accumulable.

Agent evolutionfollows a simple loop: aproposerwritescandidate artifacts(prompts,skills,memories) andselectionaccepts those that score higher. This loop has shown promise, yet practitioners face a puzzle: methods trade wins across papers, and evolution often fails to bootstrap on certain tasks. We trace this puzzle to theResolution Hypothesis:agent evolutionstalls not because methods fail to propose better artifacts, but becausenoisy evaluationscannot reliably identify them. Across seven evolution methods and six models (over 10MLLM calls), we find support for the hypothesis at both levels where evaluation scores are used. At the candidate level (choosing artifacts), helpfulcandidate artifactsexist, butselectionis no better than random at telling them from harmful ones, so errors accumulate across rounds. At the method level (ranking methods), typical margins of a few percentage points are smaller than the difference between two independent runs of the same method, so published rankings often rest on luck. Together, these findings indicate thatagent evolutionis often bottlenecked by reliably identifying improvements. This calls for a shift in how we develop and evaluateself-improving agents, from designing ever more candidate improvement systems to making true improvements detectable and accumulable.

View arXiv pageView PDFProject pageAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2611.23068 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2611.23068 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2611.23068 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Rethinking the Evaluation of Harness Evolution for Agents

Hugging Face Daily Papers

This paper rethinks how automatic harness evolution for agents should be evaluated, showing that gains may be due to increased compute rather than genuine improvements, and that evolved harnesses transfer poorly to unseen tasks.

Rethinking the Evaluation of Harness Evolution for Agents

arXiv cs.AI

This paper re-evaluates the methodology of automatic harness evolution for LLM agents, highlighting that its gains may stem from additional test-time search rather than improved harness design, and that evaluation on the same benchmark risks overfitting. Experiments show that harness evolution does not consistently outperform simpler test-time scaling methods.