@rohanpaul_ai: Better self-improving agents need better solvers, not bigger update-writing models. This challenges the common habit of…
Summary
This paper disentangles the roles of evolver and agent in self-improving LLM agents, showing that a small evolver can write sufficiently good updates, while a mid-tier agent benefits most from using them. It recommends using the strongest model as the task executor, not the update writer.
View Cached Full Text
Cached at: 06/05/26, 01:16 PM
Better self-improving agents need better solvers, not bigger update-writing models.
This challenges the common habit of putting the strongest model in the evolver seat.
The usual intuition was: put the strongest model in the evolver seat, because a better model should write better prompts, memories, tools, and skills.
This paper cuts that intuition in half.
It separates two jobs that are usually blurred together: writing useful harness updates, and benefiting from those updates during task execution.
The paper says the cheaper model can often write good enough prompt, memory, or skill updates. So a small Qwen3.5-9B evolver can create updates that help about as much as Claude Opus 4.6.
The expensive model is more useful as the agent that actually solves the task with those updates.
i.e. using the updates is very model-dependent, because weak models often fail to load the right skill or load it and then stop following it during a long task.
Strong models can use the harness, but they may already be close enough to their ceiling that the update has less room to help.
The sweet spot is the mid-tier model: capable enough to invoke and follow the new procedure, but not so capable that the harness has nothing left to teach.
Link – arxiv. org/abs/2605.30621
Title: “Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents”
Similar Articles
@dair_ai: // MetaSkill-Evolve // Great paper on self-improving agents. Most self-improving agents rewrite what the agent does and…
MetaSkill-Evolve introduces a recursive two-timescale framework for LLM agents to evolve both task skills and the improvement procedure itself, achieving notable accuracy gains on OfficeQA, SealQA, and ALFWorld benchmarks.
@omarsar0: Very good advice on self-improving agents. (bookmark it) This is something I am seeing in my own experiments with codin…
Tweet discussing advice on self-improving agents, with personal observations from experiments on coding agents for long-horizon tasks, noting that stronger models don't always yield better agents.
@qinzytech: https://x.com/qinzytech/status/2066585405479371092
A technical analysis of two approaches to building self-evolving AI agents: model-based (via architecture like SSMs or transformer with fast-weight updates, and training methods) and harness-based (via memory or meta harness that can rewrite itself). The author provides practical recommendations for different audiences.
@rohanpaul_ai: Stronger agents will not come only from larger models, but from better systems around them. The problem is that many AI…
This tweet discusses the paper 'From Model Scaling to System Scaling' which argues that stronger AI agents require better system design (harness) including context control, memory, and routing, not just larger models.
@HuggingPapers: Self-Improvements in Modern Agentic Systems A survey of 239 papers on how AI agents self-improve — by updating the mode…
A survey of 239 papers analyzing how AI agents self-improve by updating the model itself or the scaffold (prompts, memory, tools).