Recursive Self-Evolving Agents via Held-Out Selection

# Recursive Self-Evolving Agents via Held-Out Selection
Source: [https://arxiv.org/html/2606.28374](https://arxiv.org/html/2606.28374)
Michael Nguyen Quoc Nguyen Paul Vuong School of Information Technology Monash University Malaysia \{michael\.nguyen, quoc\.nguyen, paul\.vuong\}@monash\.edu

###### Abstract

LLM agents are increasingly improved*without*weight updates, by evolving a natural\-language artifact – reflections, workflows, playbooks, cheatsheets, or optimized prompts – that conditions a frozen policy\. Such methods are typically reported as wins on the single benchmark where they help\. We study them apples\-to\-apples and surface a sharper picture\. We introduceRSEA\(Recursive Self\-Evolving Agent\), which carries a compact*three\-layer natural\-language state*– an imperative*strategy*, reusable*skills*, and a procedural*playbook*– and across generations rewrites all three layers from its own trajectories, committing a candidate only if it does not regress on a disjoint held\-out split \(a*strict keep\-better*gate\)\. Across four diverse benchmarks \(ALFWorld, GAIA,τ\\tau\-bench, WebShop\) and six faithful baselines \(ReAct, Reflexion, GEPA, AWM, ACE, Dynamic Cheatsheet\), all on one shared local backbone, we find: \(i\)*no artifact universally wins*– RSEA is the strongest single\-pass method on ALFWorld \(69\.3%69\.3\\%vs\. ReAct64\.6%64\.6\\%, McNemarp=0\.015p\{=\}0\.015;79\.4%79\.4\\%with retry, best overall\), while concrete\-workflow induction \(AWM\) is best on the strong\-backbone tool\-use tasks; \(ii\) unguarded context evolution is*high\-variance and unsafe*– Dynamic Cheatsheet, which curates context online with no held\-out gate, is near\-best on ALFWorld \(70\.7%70\.7\\%\) yet*collapses*on WebShop \(score0\.140\.14vs\. ReAct0\.430\.43\); and \(iii\) RSEA’s strict held\-out selection is what makes recursive self\-evolution*monotone\-safe*: it never significantly underperforms the base agent on any benchmark, falling back to vanilla ReAct when evolved context would hurt\. Controlled ablations show that every layer of the evolved state helps and that removing held\-out selection causes severe overfitting \(a perfect in\-sample score but a3333\-point drop to test\)\. We argue the reliability of context evolution comes less from the artifact and more from the*selection gate*, and release the full harness, the six baseline re\-implementations, and pre\-registered manifests\.

## 1Introduction

A central paradigm for building self\-improving LLM agents adapts the*context*rather than the weights: the policy is frozen and an evolving natural\-language artifact – accumulated reflections, induced workflows, curated playbooks, online cheatsheets, or optimized system prompts – is injected at inference\. Context adaptation is attractive for concrete reasons: it is interpretable and auditable, integrates new knowledge at runtime, transfers across model versions, and – with long\-context inference and KV\-cache reuse – is increasingly cheap to serve\.

Despite rapid progress, these methods are almost always evaluated*in isolation*: a single benchmark, weak or non\-matched baselines, and a backbone or decoding budget that varies between methods\. This makes their*relative*merits and, crucially, their*failure modes*hard to read\. When we re\-run six representative methods on one shared backbone across four diverse agent benchmarks, two uncomfortable facts emerge\. First,*which*artifact helps is benchmark\-dependent: a method that tops one benchmark is middling or harmful on another\. Second, and more importantly, evolving context is*not*a free lunch\. We name the failure mode*context distraction*: an injected artifact that degrades an otherwise\-capable base policy\. Context distraction is not hypothetical – in our study an unguarded online method \(Dynamic Cheatsheet\) goes from near\-best on ALFWorld \(70\.7%70\.7\\%\) to a WebShop score of0\.1360\.136versus the ReAct baseline’s0\.4290\.429, a catastrophic regression caused entirely by the context it added\.

We argue that the reliability of context evolution comes less from the artifact being evolved and more from the*selection gate*that decides whether to commit it\.An evolution loop that commits whatever it produces inherits the variance of its own rewrites; one that commits a change only when it improves*held\-out*performance cannot do worse than not evolving at all\. We make this gate the design center of our method rather than an afterthought\.

Concretely, we presentRSEA\(Recursive Self\-Evolving Agent\)\. RSEA carries a compact three\-layer natural\-language state –*strategy*\(an imperative preamble\),*skills*\(reusable sub\-routines\), and*playbook*\(procedures distilled from successes\) – and across generations \(i\) rolls the current state out on a small evolve pool, \(ii\) rewrites*all three layers*from the resulting trajectories, and \(iii\) commits the candidate only if it does not regress on a disjoint validation split, updating the frozen best state only on a*strict*improvement\. Only the frozen best state is run on a held\-out test split, so there is no leakage\. RSEAR\{\}\_\{\\text\{R\}\}injects this frozen prior into a per\-task retry loop\.

Contributions\.\(1\) A formulation of cross\-task NL self\-evolution centered on a*strict held\-out keep\-better*gate that makes the operation*monotone\-safe*\(§[4](https://arxiv.org/html/2606.28374#S4)\)\. \(2\) A faithful, single\-backbone re\-implementation of six classic and recent baselines on a shared harness \(§[5](https://arxiv.org/html/2606.28374#S5)\)\. \(3\) A four\-benchmark study with multi\-seed statistics and paired tests, yielding an honest scope condition rather than a universal\-win claim \(§[6](https://arxiv.org/html/2606.28374#S6)–[7](https://arxiv.org/html/2606.28374#S7)\)\. \(4\) Controlled ablations that isolate the contribution of each state layer and quantify the cost of removing held\-out selection \(§[8](https://arxiv.org/html/2606.28374#S8)\)\.

Key findings\.\(a\) On ALFWorld \(134 tasks×\\times5 seeds, 7B\) the single\-pass evolved prior significantly beats ReAct, GEPA, and AWM, and RSEAR\{\}\_\{\\text\{R\}\}is best overall \(79\.4%79\.4\\%\)\. \(b\) On the strong\-backbone tool\-use benchmarks no NL artifact dominates; RSEA is statistically tied with ReAct \(and never significantly worse\), while AWM’s concrete workflows give the only consistent small lift\. \(c\) Methods without a held\-out gate are high\-variance and can catastrophically regress; RSEA’s strict gate converts “context evolution” from a coin\-flip into a monotone\-safe operation\. \(d\) Ablations show every layer of the evolved state helps \(the layers overlap rather than being strictly complementary\) and that removing held\-out selection overfits the evolve pool \(100%100\\%in\-sample, a3333\-point drop to test\)\.

![Refer to caption](https://arxiv.org/html/2606.28374v1/figures/fig_crossbench.png)Figure 1:No context\-evolution artifact universally wins, and unguarded evolution is unsafe\.Single\-pass methods across four benchmarks on one shared backbone \(ALFWorld 7B; GAIA/τ\\tau\-bench/WebShop 30B\)\. RSEA \(red\) is the strongest single\-pass method on ALFWorld and never significantly underperforms ReAct \(grey\) elsewhere; AWM is best on the tool\-use tasks; and Dynamic Cheatsheet – which curates context online with*no*held\-out gate – is near\-best on ALFWorld yet*collapses*on WebShop \(0\.14 vs\. ReAct 0\.43\)\. RSEA’s strict held\-out gate is what makes evolution monotone\-safe\.
## 2Related Work

### LLM agents that reason and act\.

A now\-standard recipe interleaves free\-form reasoning with tool/environment actions: ReAct\(Yaoet al\.,[2023b](https://arxiv.org/html/2606.28374#bib.bib1)\)couples chain\-of\-thought with acting, elaborated with deliberate search\(Yaoet al\.,[2023a](https://arxiv.org/html/2606.28374#bib.bib4)\), self\-critique and revision\(Madaanet al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib3)\), learned tool use\(Schicket al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib6)\), and grounded planning in embodied or open\-ended worlds\(Huanget al\.,[2022](https://arxiv.org/html/2606.28374#bib.bib23); Wanget al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib5)\)\. Recent work strengthens the reasoning substrate – multi\-perspective, semiotically grounded logical reasoning\(Zhanget al\.,[2025e](https://arxiv.org/html/2606.28374#bib.bib59)\), “system\-2” vision\-language\-action policies\(Songet al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib39); Fanget al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib63)\), and spatio\-temporal chains of thought for control\(Zenget al\.,[2025a](https://arxiv.org/html/2606.28374#bib.bib41)\)– while cognitive\-architecture and survey treatments organize the pieces into memory, planning, and action modules\(Sumerset al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib8); Wanget al\.,[2024a](https://arxiv.org/html/2606.28374#bib.bib9); Parket al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib7); Yanget al\.,[2026a](https://arxiv.org/html/2606.28374#bib.bib78)\)\. RSEA keeps this loop*unchanged*and intervenes only through an evolved natural\-language state, isolating the contribution of context from scaffold\.

### Tool\-use agents and tool learning\.

A large body studies how agents select and compose tools\(Quet al\.,[2025b](https://arxiv.org/html/2606.28374#bib.bib75)\): mastering tools via self\-driven interaction\(Quet al\.,[2025a](https://arxiv.org/html/2606.28374#bib.bib76)\), fine\-grained supervision for tool\-integrated reasoning\(Quet al\.,[2026](https://arxiv.org/html/2606.28374#bib.bib77)\), similarity/dependency\-aware experience networks for multi\-tool selection\(Zhanget al\.,[2025f](https://arxiv.org/html/2606.28374#bib.bib31)\), self\-critique under tool\-calling errors\(Huanget al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib64)\), tree\-search tool planning\(Yanget al\.,[2026b](https://arxiv.org/html/2606.28374#bib.bib80)\), GUI grounding\(Lianet al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib65)\), and agentic text\-to\-SQL\(Liet al\.,[2026b](https://arxiv.org/html/2606.28374#bib.bib32); Su,[2026](https://arxiv.org/html/2606.28374#bib.bib46)\)\. Most closely,*test\-time tool evolution*adapts an agent’s tools at inference\(Luet al\.,[2026](https://arxiv.org/html/2606.28374#bib.bib29)\)\. RSEA is complementary: rather than evolving the tool set, it evolves a natural\-language*strategy*over a fixed action/tool interface, and ourτ\\tau\-bench/WebShop results explain when such NL strategy adds little beyond an already\-detailed tool API\.

### Prompt and context optimization\.

A second line optimizes the prompt itself: discrete instruction search\(Zhouet al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib10)\), LLM\-as\-optimizer loops\(Yanget al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib11)\), declarative pipelines with compiled prompt/demonstration optimization\(Khattabet al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib12); Opsahl\-Onget al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib13)\), evolutionary search\(Fernandoet al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib15)\), and reflective genetic\-Pareto evolution\(Agrawal and others,[2025](https://arxiv.org/html/2606.28374#bib.bib14)\); prompts are also notoriously brittle to surface form\(Caiet al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib44)\)\. These methods optimize a*single*flat artifact and usually treat selection as an implementation detail\. We instead hold the artifact family fixed and show that the*strictness*of held\-out selection – not the sophistication of the optimizer – governs whether evolution helps or hurts\.

### Self\-evolving and self\-improving agents\.

Closest in spirit are agents that improve from their own experience\. Beyond context, this includes bootstrapping reasoning from self\-generated data\(Zelikmanet al\.,[2022](https://arxiv.org/html/2606.28374#bib.bib21)\)and searching over agent code/topology\(Huet al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib22)\)\. Within the agent loop, recent methods self\-evolve tool\-use policies via blame\-aware mutation and diversity\-aware selection\(Yanget al\.,[2026c](https://arxiv.org/html/2606.28374#bib.bib79)\), evolve intrinsic skills in hierarchical RL\(Liet al\.,[2026d](https://arxiv.org/html/2606.28374#bib.bib68)\), self\-improve unified multimodal models from self\-generated supervision\(Hanet al\.,[2026](https://arxiv.org/html/2606.28374#bib.bib62)\), and evolve task\-specific prototypes\(Zhuet al\.,[2025b](https://arxiv.org/html/2606.28374#bib.bib35)\)\. RSEA shares the self\-evolution goal but is deliberately weight\-frozen and NL\-state\-based, and – unlike most of these – makes a*strict held\-out selection gate*the center, which our ablations and transfer results identify as the decisive ingredient for safety\.

### Reinforcement learning for LLM agents\.

A parallel thread trains agents/reasoners with RL: verifiable meta\-reasoning rewards for long\-horizon agents\(Zhanget al\.,[2026c](https://arxiv.org/html/2606.28374#bib.bib30)\), reasoning re\-ranking agents\(Zhanget al\.,[2025a](https://arxiv.org/html/2606.28374#bib.bib49)\), and RL for contextual integrity\(Lanet al\.,[2025a](https://arxiv.org/html/2606.28374#bib.bib42)\)\. A recurring concern is the design of the optimization signal and*diversity collapse*under verifiable rewards\(Liet al\.,[2025b](https://arxiv.org/html/2606.28374#bib.bib33)\), addressed via divergence choice\(Liet al\.,[2025b](https://arxiv.org/html/2606.28374#bib.bib33)\), reward\-confidence correction\(Liet al\.,[2026c](https://arxiv.org/html/2606.28374#bib.bib69)\), preference optimization with priors\(Lanet al\.,[2025b](https://arxiv.org/html/2606.28374#bib.bib43)\), and intrinsic self\-reflection\(Liet al\.,[2025c](https://arxiv.org/html/2606.28374#bib.bib70)\)\. Our weight\-frozen method avoids RL’s cost and instability while still exploiting execution feedback through the rewrite/selection loop\.

### Agent memory, experiential learning, and overfitting\.

Closest to RSEA’s artifact are methods accumulating reusable NL experience: ExpeL\(Zhaoet al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib16)\), Agent Workflow Memory\(Wanget al\.,[2024b](https://arxiv.org/html/2606.28374#bib.bib17)\), Agentic Context Engineering\(Zhanget al\.,[2025c](https://arxiv.org/html/2606.28374#bib.bib18)\), Dynamic Cheatsheet\(Suzgunet al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib19)\), and memory\-augmented agents with explicit stores\(Packeret al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib20); Zenget al\.,[2025b](https://arxiv.org/html/2606.28374#bib.bib40)\)\. Two failure modes from adjacent fields motivate our selection\-first view:*catastrophic forgetting / collapse*, studied in continual learning\(Chen and Zeng,[2025](https://arxiv.org/html/2606.28374#bib.bib71)\)and observed as logical\-reasoning collapse\(Zhanget al\.,[2026a](https://arxiv.org/html/2606.28374#bib.bib60)\); and classical*overfitting*, controlled by difficulty\-aware reweighting and adversarial robustness\(Zhouet al\.,[2022](https://arxiv.org/html/2606.28374#bib.bib74),[2024b](https://arxiv.org/html/2606.28374#bib.bib72),[2024a](https://arxiv.org/html/2606.28374#bib.bib73)\)\. RSEA’s strict held\-out gate is precisely a mechanism that prevents the evolved context from overfitting or collapsing, which we demonstrate empirically\.

### Benchmarks, multi\-agent systems, and the broader landscape\.

We evaluate on a deliberately diverse slate – embodied text households\(Shridharet al\.,[2021](https://arxiv.org/html/2606.28374#bib.bib24)\), grounded web shopping\(Yaoet al\.,[2022](https://arxiv.org/html/2606.28374#bib.bib25)\), open\-ended assistants\(Mialonet al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib26)\), tool\-agent\-user interaction\(Yaoet al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib27)\), interactive coding\(Trivediet al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib28)\), and multimodal deep research\(Zenget al\.,[2026](https://arxiv.org/html/2606.28374#bib.bib61)\)– a spread that surfaces the benchmark\-dependence single\-benchmark studies miss\. Beyond single agents, multi\-agent and social\-simulation systems coordinate many LLM agents\(Zhanget al\.,[2025d](https://arxiv.org/html/2606.28374#bib.bib58),[2026b](https://arxiv.org/html/2606.28374#bib.bib57)\)\. Finally, our techniques sit within a broad ecosystem of LLM and multimodal systems where self\-improvement, evolution, and robust reasoning recur: medical and scientific vision\-language models\(Zhuet al\.,[2025a](https://arxiv.org/html/2606.28374#bib.bib34); Wuet al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib45)\), multimodal misinformation detection\(Liet al\.,[2026a](https://arxiv.org/html/2606.28374#bib.bib66),[2025a](https://arxiv.org/html/2606.28374#bib.bib67)\), vision\-language alignment and compositionality\(Zhanget al\.,[2024a](https://arxiv.org/html/2606.28374#bib.bib48),[2025b](https://arxiv.org/html/2606.28374#bib.bib50)\), retrieval and query expansion\(Zhanget al\.,[2024b](https://arxiv.org/html/2606.28374#bib.bib51); Xieet al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib52),[2026a](https://arxiv.org/html/2606.28374#bib.bib53); Xie,[2026](https://arxiv.org/html/2606.28374#bib.bib54); Xieet al\.,[2026b](https://arxiv.org/html/2606.28374#bib.bib55)\), controllable generation and editing\(Xuet al\.,[2024](https://arxiv.org/html/2606.28374#bib.bib56)\), automated optimization modeling\(Liuet al\.,[2026](https://arxiv.org/html/2606.28374#bib.bib47)\), and efficient models via knowledge distillation and compression\(Lan and Tian,[2025](https://arxiv.org/html/2606.28374#bib.bib36); Lanet al\.,[2026b](https://arxiv.org/html/2606.28374#bib.bib37),[a](https://arxiv.org/html/2606.28374#bib.bib38)\)\.

## 3Background and Motivation

### Context adaptation\.

Given a frozen policyπθ\\pi\_\{\\theta\}and a benchmark of tasks, context adaptation seeks an artifactcc\(a string injected into the prompt\) that improvesπθ\(⋅∣c\)\\pi\_\{\\theta\}\(\\cdot\\mid c\)without touchingθ\\theta\. Methods differ in \(i\) the*form*ofcc, \(ii\) the*update operator*that editsccfrom execution feedback, and \(iii\) the*selection rule*that decides whichccto keep\. Table[1](https://arxiv.org/html/2606.28374#S3.T1)casts the methods we study in this\(form,update,selection\)\(\\text\{form\},\\text\{update\},\\text\{selection\}\)view; the third column is where they differ most and, we argue, where reliability is won or lost\.

### Limitations of existing methods\.

Reflexion\(Shinnet al\.,[2023](https://arxiv.org/html/2606.28374#bib.bib2)\)adapts*within*a single task across retries but carries nothing across tasks\. Prompt/context optimizers carry a cross\-task artifact but optimize a*single*form: GEPA\(Agrawal and others,[2025](https://arxiv.org/html/2606.28374#bib.bib14)\)a flat prompt via reflective Pareto evolution; AWM\(Wanget al\.,[2024b](https://arxiv.org/html/2606.28374#bib.bib17)\)a list of induced workflows; ACE\(Zhanget al\.,[2025c](https://arxiv.org/html/2606.28374#bib.bib18)\)an itemized bullet playbook with add\-only deltas; Dynamic Cheatsheet\(Suzgunet al\.,[2025](https://arxiv.org/html/2606.28374#bib.bib19)\)a single cheatsheet curated*online*\. Two issues recur\.Single\-form artifactsconflate complementary kinds of knowledge – a high\-level strategy, a reusable sub\-routine, and a concrete procedure are not interchangeable, yet most methods fold them into one string\.Unguarded commitmentis the more damaging: methods that append or rewrite without a held\-out check \(notably online curation\) commit context that can*distract*the base policy, and – because the same mechanism that helps when context is useful is unconstrained when it is not – they are high\-variance across benchmarks \(§[7](https://arxiv.org/html/2606.28374#S7)\)\. RSEA targets both: a three\-layer form, and a strict held\-out gate\.

Table 1:Context\-adaptation methods as \(form, update operator, selection rule\)\. The selection rule – whether a candidate is committed only after improving*held\-out*performance – is what we argue governs reliability\.MethodArtifact formUpdate operatorSelection ruleReflexionper\-task reflectionverbal self\-feedbacknone \(within\-task retry\)GEPAflat promptreflective mutationPareto on val minibatchAWMworkflow listinduction from successesnone \(induce & inject\)ACEbullet playbookadd\-only deltasval keep\-best \(non\-strict\)Dynamic Cheatsheetsingle cheatsheetonline rewritenone \(no held\-out gate\)RSEA \(ours\)3\-layer stateholistic rewrite of all layersstrict held\-out keep\-better

## 4Method: RSEA

### Three\-layer evolving state\.

The agent state is a triples=\(strategy,skills,playbook\)s=\(\\textsc\{strategy\},\\textsc\{skills\},\\textsc\{playbook\}\): \(1\)strategy, an imperative natural\-language preamble \(≤\\leqa few sentences\); \(2\)skills, a short list of reusable sub\-routines / conditional rules; and \(3\)playbook, a list of procedures \(ordered step sequences\) distilled from successful trajectories\. The three layers are deliberately complementary – a*policy*of what to prefer, a*library*of reusable moves, and*procedures*for recurring task families\. At inference,ssis rendered to a compact preamble and injected immediately ahead of the task in an otherwise canonical ReAct loop \(Fig\.[2](https://arxiv.org/html/2606.28374#S4.F2)\); the*only*difference from vanilla ReAct is this injected, evolved state, which keeps the comparison clean\.

### The self\-rewrite operator\.

A development pool is split into a disjoint*evolve*setDeD\_\{e\}and*val*setDvD\_\{v\}\. Generationggrolls the current statesg−1s\_\{g\-1\}out onDeD\_\{e\}, collecting \(task, trajectory, env\-verified outcome\) tuples, then prompts the*same*frozen LLM to act as its own meta\-optimizer: given the current three layers and a balanced sample of successful and failed trajectories, it returns a rewritten triples~g\\tilde\{s\}\_\{g\}\. The rewrite prompt asks the model to*derive transferable rules from the trajectories*– quoting action phrasings that succeeded, recording where objects/values were found, and adding rules that would have averted the observed failures – and to keep every skill and playbook entry a single, general, instance\-agnostic string\. Unlike add\-only methods, the operator may rewrite or drop any layer, so the state does not monotonically grow \(avoiding unbounded context\); unlike a flat\-prompt optimizer, it edits three structured layers at once\.

### Strict held\-out keep\-better selection\.

The candidates~g\\tilde\{s\}\_\{g\}is scored on the held\-outDvD\_\{v\}\. We accept it as the new*working*state if it does not regress onDvD\_\{v\}\(allowing lateral exploration\), but we update the frozen*best*state only on a*strict*improvement \(Alg\.[1](https://arxiv.org/html/2606.28374#algorithm1), line[1](https://arxiv.org/html/2606.28374#algorithm1)\)\. This asymmetry is the crux: lateral acceptance lets the search escape plateaus, while strict best\-update guarantees the returned state is*never worse than vanilla ReAct on held\-out val*– if no candidate strictly improves, RSEA returns the empty state and reduces exactly to ReAct\. Only the frozen best state is later run on the test split\. We use env\-verified success as the selection signal where it is dense enough \(ALFWorld,τ\\tau\-bench\) and the benchmark’s dense reward where success is sparse \(WebShop\)\.

Input:frozen LLMπ\\pi; evolve setDeD\_\{e\}, val setDvD\_\{v\}; generationsGG

s←∅s\\leftarrow\\varnothing;

s⋆←∅s^\{\\star\}\\leftarrow\\varnothing;

v⋆←Eval​\(π,∅,Dv\)v^\{\\star\}\\leftarrow\\textsc\{Eval\}\(\\pi,\\varnothing,D\_\{v\}\)
//empty state = vanilla ReAct

for*g=1​…​Gg=1\\dots G*do

T←Rollout​\(π,s,De\)T\\leftarrow\\textsc\{Rollout\}\(\\pi,s,D\_\{e\}\)
//trajectories \+ env\-verified outcomes

s~←SelfRewrite​\(π,s,T\)\\tilde\{s\}\\leftarrow\\textsc\{SelfRewrite\}\(\\pi,s,T\)
//rewrite all 3 layers

v←Eval​\(π,s~,Dv\)v\\leftarrow\\textsc\{Eval\}\(\\pi,\\tilde\{s\},D\_\{v\}\)
//held\-out score

if*v≥*Eval*​\(π,s,Dv\)v\\geq\\textsc\{Eval\}\(\\pi,s,D\_\{v\}\)*then

s←s~s\\leftarrow\\tilde\{s\}
//lateral accept

end if

if*v\>v⋆v\>v^\{\\star\}*then

s⋆←s~s^\{\\star\}\\leftarrow\\tilde\{s\};

v⋆←vv^\{\\star\}\\leftarrow v
//strict best\-update

end if

end for

return*s⋆s^\{\\star\}*

//frozen; run on held\-out test

Algorithm 1RSEA evolution \(strict held\-out keep\-better\)\.
### RSEAR\{\}\_\{\\text\{R\}\}: evolution×\\timesretry\.

The frozen evolved prior is orthogonal to per\-task retry\. RSEAR\{\}\_\{\\text\{R\}\}injectss⋆s^\{\\star\}into a Reflexion\-style multi\-trial loop, combining a good*starting*policy with within\-task self\-correction; §[8](https://arxiv.org/html/2606.28374#S8)shows the two contribute near\-independent gains\.

![Refer to caption](https://arxiv.org/html/2606.28374v1/figures/fig_architecture.png)Figure 2:RSEA recursively rewrites a three\-layer natural\-language state of a frozen LLM agent\. The state is injected as a preamble into a standard ReAct loop; across generations it is rewritten from evolve\-set trajectories and frozen only on a*strict*held\-out validation improvement, which makes the loop monotone\-safe\.

## 5Experimental Setup

### Backbone \(fairness iron law\)\.

Every method – ours and all baselines – uses the*same*locally served model and decoding budget; the task LLM and the reflection/rewrite LLM are identical\. We serve Qwen2\.5\-7B\-Instruct for ALFWorld \(to keep the base agent below ceiling\) and Qwen3\-30B\-A3B\-Instruct for the tool\-use benchmarks \(GAIA,τ\\tau\-bench, WebShop\), via vLLM across 4×\\timesA100, round\-robined so every method shares identical compute\.

### Benchmarks\.

\(i\)ALFWorld\(text household tasks; env\-verified success\): a balanced 134\-task test set, 5 seeds, disjoint 30\-task dev pool\. \(ii\)GAIA\(v1\) Level\-1 text tasks with self\-built search/fetch/python tools and the official deterministic scorer \(30 tasks\)\. \(iii\)τ\\tau\-bench\(retail\): a tool\-calling agent with an LLM user simulator and the official DB\-hash reward \(60 held\-out eval\)\. \(iv\)WebShop\(1,000\-product subset, Lucene retrieval\): dense attribute/option/price reward \(100 held\-out eval\)\. For the tool\-use benchmarks the official dev split is saturated for our backbone, so we carve disjoint evolve/val/eval slices from a seed\-shuffled test pool \(no leakage\)\.

### Baselines\.

ReAct, Reflexion, GEPA, AWM, ACE, and Dynamic Cheatsheet, each a faithful re\-implementation on our harness \(we open the original repositories and port the core loops\), injected at the*same*point as RSEA and held to the same per\-method rollout budget\. For ALFWorld we additionally report “\+retry” \(R\{\}\_\{\\text\{R\}\}\) variants that add the same Reflexion\-style multi\-trial loop on top of each evolved prior, isolating evolution from retry\. We report success rate \(ALFWorld/GAIA/τ\\tau\-bench\) or mean dense score \(WebShop\), multi\-seed mean±\\pm95% CI where applicable, and paired McNemar over matched \(task\[, seed\]\) cells\.

## 6Results on ALFWorld: the evolved prior significantly helps

Our evaluation shows that:\(i\) RSEA is the strongest single\-pass context methodon ALFWorld and significantly beats ReAct \(§[6](https://arxiv.org/html/2606.28374#S6)\);\(ii\) no NL artifact dominates the strong\-backbone transfer benchmarks, where RSEA ties ReAct and never regresses \(§[7](https://arxiv.org/html/2606.28374#S7)\);\(iii\) held\-out selection is decisive– unguarded methods are high\-variance \(§[7](https://arxiv.org/html/2606.28374#S7)\) and removing the gate overfits \(§[8](https://arxiv.org/html/2606.28374#S8)\); and\(iv\) every layer of the evolved state helps\(§[8](https://arxiv.org/html/2606.28374#S8)\)\.

Table[2](https://arxiv.org/html/2606.28374#S6.T2)reports the full 10\-method comparison\. The single\-pass evolved prior \(RSEA,69\.3%69\.3\\%\) significantly improves over vanilla ReAct \(64\.6%64\.6\\%, McNemarp=0\.015p\{=\}0\.015\) and over the recent single\-pass baselines GEPA \(63\.9%63\.9\\%,p=0\.004p\{=\}0\.004\) and AWM \(65\.4%65\.4\\%,p=0\.037p\{=\}0\.037\), and ties the strongest single\-pass baselines ACE \(66\.9%66\.9\\%,p=0\.20p\{=\}0\.20\) and Dynamic Cheatsheet \(70\.7%70\.7\\%,p=0\.45p\{=\}0\.45\)\. The story of the single\-pass column is that on a 7B backbone most injected\-context baselines barely move over ReAct – an evolved prompt frequently*distracts*the policy – whereas RSEA’s holistic three\-layer rewrite plus strict val\-selection is the method that most reliably nets positive\. Adding retry, RSEAR\{\}\_\{\\text\{R\}\}attains the best overall success \(79\.4%79\.4\\%\), significantly above ReAct \(p<10−4p\{<\}10^\{\-4\}\), single\-pass RSEA \(p<10−4p\{<\}10^\{\-4\}\), and every no\-evolution baseline, and trends above the strongest retry baselines Reflexion \(76\.4%76\.4\\%,p=0\.09p\{=\}0\.09\), ACER\{\}\_\{\\text\{R\}\}\(76\.7%76\.7\\%,p=0\.13p\{=\}0\.13\), and GEPAR\{\}\_\{\\text\{R\}\}\(75\.8%75\.8\\%,p=0\.052p\{=\}0\.052\)\.

Table 2:ALFWorld \(134 tasks×\\times5 seeds, Qwen2\.5\-7B\)\. Success rate mean±\\pm95% CI; subscripts are McNemar\-significant gains \(green\) over ReAct\. “\+retry” adds the same multi\-trial loop to each prior\. RSEA is the strongest single\-pass context method; RSEAR\{\}\_\{\\text\{R\}\}is best overall\.Single\-passSucc\. \(%\)\+ retrySucc\. \(%\)ReAct64\.6±4\.364\.6\\pm 4\.3Reflexion76\.476\.4GEPA63\.9±5\.363\.9\\pm 5\.3GEPAR\{\}\_\{\\text\{R\}\}75\.8±5\.475\.8\\pm 5\.4AWM65\.4±3\.365\.4\\pm 3\.3ACER\{\}\_\{\\text\{R\}\}76\.7±4\.976\.7\\pm 4\.9ACE66\.9±2\.566\.9\\pm 2\.5Dynamic Cheatsheet70\.7±3\.470\.7\\pm 3\.4RSEA \(ours\)69\.3±9\.3\\mathbf\{69\.3\\pm 9\.3\}\+4\.7\+4\.7RSEAR\{\}\_\{\\text\{R\}\}\(ours\)79\.4±7\.4\\mathbf\{79\.4\\pm 7\.4\}\+14\.8\+14\.8### Where the gains come from\.

Table[3](https://arxiv.org/html/2606.28374#S6.T3)breaks ALFWorld down by task family and connects the quantitative gains to the*interpretable*evolved state \(App\.[A](https://arxiv.org/html/2606.28374#A1)\)\. RSEA’s improvements concentrate exactly where its learned content applies:*examine*tasks jump6\.7%→26\.7%6\.7\\%\\to 26\.7\\%\(→44\.4%\\to 44\.4\\%with retry\), because the evolved skill “use<desklamp\>before examining objects” is the precise fix for the base agent’s dominant failure;*pick\_two*and*pick\_and\_place*gain77–99points from the playbook’s take\-before\-place procedures\. The single regression \(*clean*,−6\.5\-6\.5points\) is mild distraction, and is recovered by retry\. This tight correspondence between the human\-readable state and the per\-family gains is a property the single\-string baselines do not expose\.

Table 3:ALFWorld success by task family \(%, 5 seeds\)\. RSEA’s gains concentrate in the families its evolved skills/playbook directly address \(*examine*,*pick\_two*,*pick\_and\_place*\)\.Methodexaminepick\_twopick\_and\_placeheatcoolcleanReAct6\.736\.582\.576\.581\.080\.0RSEA26\.743\.591\.779\.183\.873\.5RSEAR\{\}\_\{\\text\{R\}\}44\.467\.199\.287\.088\.679\.4

## 7Transfer to GAIA,τ\\tau\-bench, and WebShop

Table[4](https://arxiv.org/html/2606.28374#S7.T4)reports the cross\-benchmark comparison; the tool\-use benchmarks use the stronger 30B backbone\. Three honest findings stand out\.\(1\) No single method dominates\.RSEA is the strongest single\-pass method on ALFWorld but on the 30B tool\-use benchmarks the concrete\-workflow method AWM is marginally best \(τ\\tau\-bench51\.7%51\.7\\%, WebShop0\.4600\.460\): when the long domain wiki / detailed tool API already encodes most of what an evolved*strategy*would add, a library of concrete procedures has more to offer than an abstract policy\.\(2\) RSEA never regresses\.Where injected NL context does not help, RSEA’s strict gate falls back toward vanilla ReAct – e\.g\. on WebShop*every*evolved candidate hurt held\-out val, so the frozen state is empty and RSEA≈\\approxReAct \(0\.4370\.437vs\.0\.4290\.429\); RSEA is statistically tied with ReAct onτ\\tau\-bench, WebShop, and GAIA \(McNemarp\>0\.1p\{\>\}0\.1\)\. GAIA in particular is within run\-to\-run web noise: two independent evaluations of the*same*frozen states disagree on the sign of the≤1\\leq 1\-task gap\.\(3\) Selection matters: unguarded methods are high\-variance\.Dynamic Cheatsheet, which curates context online with*no*held\-out gate, is near\-best on ALFWorld \(70\.7%70\.7\\%\) yet*collapses*on WebShop \(score0\.1360\.136vs\. ReAct0\.4290\.429\) and is worst onτ\\tau\-bench \(36\.7%36\.7\\%\) – the same unguarded mechanism is high\-reward when context helps and catastrophic when it does not\. The held\-out val metric improves over generations where there is signal \(Fig\.[3](https://arxiv.org/html/2606.28374#S7.F3);τ\\tau\-bench val0\.20→0\.360\.20\{\\to\}0\.36\) and the strict gate rejects all regressive candidates where there is not \(WebShop\)\.

Table 4:Cross\-benchmark comparison \(single\-pass\)\. ALFWorld: success % \(134×\\times5 seeds, 7B\)\. GAIA: accuracy % \(30, 30B\)\.τ\\tau\-bench retail: success % \(60 eval, 30B\)\. WebShop: mean dense score \(100 eval, 30B\)\.Bold=column best\.⋆\\star: RSEAp=0\.015p\{=\}0\.015vs\. ReAct \(ALFWorld\)\.DC has no held\-out gate and is high\-variance \(best on ALFWorld, worst on WebShop/τ\\tau\-bench\); RSEA is significantly best on ALFWorld and never regresses elsewhere\.MethodALFWorld↑\\uparrowGAIA↑\\uparrowτ\\tau\-bench↑\\uparrowWebShop↑\\uparrowsucc\.%acc\.%succ\.%scoreReAct64\.616\.741\.70\.429GEPA63\.9–41\.70\.415AWM65\.4–51\.70\.460ACE66\.9–40\.00\.453Dynamic Cheatsheet70\.7–36\.70\.136RSEA \(ours\)69\.3⋆69\.3^\{\\star\}13\.340\.00\.437![Refer to caption](https://arxiv.org/html/2606.28374v1/figures/fig_gencurves.png)Figure 3:RSEA self\-evolution: held\-out validation over generations \(strict keep\-better\)\. The best\-kept state improves where there is signal \(τ\\tau\-bench0\.20→0\.360\.20\{\\to\}0\.36\) and the strict gate rejects every regressive candidate where there is not \(WebShop\), so the frozen state never underperforms ReAct\.
## 8Ablations

We ablate the two design choices that distinguish RSEA from a generic prompt optimizer: the*three\-layer state*and the*strict held\-out selection*\. Both ablations use the ALFWorld test set \(where RSEA helps\), the identical ReAct loop, and temp\-0\.6 multi\-seed decoding; only the injected preamble \(resp\. the selection rule\) changes\.

### Each layer helps; the layers overlap \(Table[6](https://arxiv.org/html/2606.28374#S8.T6)\)\.

We take the*frozen*ALFWorld state and inject individual and combined subsets of its layers\.*Every*subset improves over ReAct \(64\.2%64\.2\\%\): each layer on its own lifts success to6767–70%70\\%, with the*skills*and*playbook*layers \(the concrete sub\-routines and procedures\) carrying most of the benefit \(69\.8%69\.8\\%each\)\. Combining layers does not strictly improve further at this scale \(full state68\.5%68\.5\\%, within the multi\-seed band\), so the three layers encode*overlapping*procedural knowledge rather than three orthogonal signals: the gain comes from injecting the distilled procedures in any of these forms \(consistent with the per\-family analysis, Table[3](https://arxiv.org/html/2606.28374#S6.T3)\)\. The three\-layer structure is best read as an interpretable organization of that knowledge – and the substrate the selection gate operates on – not as a claim that all three are individually necessary\.

### Held\-out selection prevents overfitting \(Table[6](https://arxiv.org/html/2606.28374#S8.T6)\)\.

We re\-run the evolution with*no*held\-out split \(selecting on the evolve set itself\) and freeze the result\. The no\-gate state*overfits*sharply: it reaches a perfect100%100\\%in\-sample selection score but only66\.7%66\.7\\%on test – a3333\-point train–test gap – versus held\-out RSEA’s67\.3%67\.3\\%\(63\.6%63\.6\\%for ReAct\)\. On ALFWorld even the overfit state edges out ReAct, but the danger of unguarded commitment is starkest on the transfer benchmarks \(§[7](https://arxiv.org/html/2606.28374#S7)\): there, the method with*no*held\-out gate \(Dynamic Cheatsheet\) regresses catastrophically \(WebShop0\.140\.14vs\.0\.430\.43\), while RSEA’s strict gate keeps performance at the ReAct level\. The gate – not the rewrite operator – is what bounds the downside and makes recursive self\-evolution safe to deploy\.

### Robustness\.

The strict best\-update is what bounds the downside: with a non\-strict \(≥\\geq\) best\-update, a candidate that merely ties on a small or unrepresentative val draw can be frozen and then hurt on test; the strict variant returns the simpler \(often empty\) state in that case, which is exactly the fall\-back\-to\-ReAct behavior we observe on WebShop andτ\\tau\-bench\.

Table 5:Layer ablation \(ALFWorld test, 54×\\times3 seeds\)\. Every subset beats ReAct; layers overlap\.Injected stateSucc\. \(%\)empty \(= ReAct\)64\.2strategy only67\.3skills only69\.8playbook only69\.8strategy \+ skills67\.9full RSEA68\.5
Table 6:Selection ablation \(ALFWorld\)\. No gate⇒\\Rightarrow100% in\-sample but a 33\-pt drop to test \(overfitting\)\.Variantin\-samp\.testReAct \(empty\)–63\.6RSEA, no held\-out gate100\.066\.7RSEA, strict gate–67\.3

## 9Compute and Cost

Because every method shares one backbone and an iso\-rollout budget, cost differences come from the*number of meta\-LLM calls*\(rewrites/reflections\) and the injected context length\. RSEA’s evolution is a fixedGGgenerations of \(evolve rollout \+ one rewrite \+ val rollout\); the rewrite is a single call per generation, so the meta\-overhead isO​\(G\)O\(G\)calls – e\.g\. theτ\\tau\-bench evolution used190190meta \+ selection calls in total\. At inference RSEA adds only the rendered preamble \(≤\\leqa few hundred tokens, bounded by construction since the rewrite may drop layers\), versus online methods whose context grows unboundedly with the test stream\. Crucially, the strict gate means this overhead is only ever spent to*match or beat*ReAct, never to underperform it – the favorable risk profile that the other methods lack\.

## 10Discussion and Limitations

### When does cross\-task NL evolution help?

The scope condition is clear: an evolved NL state helps most when the bottleneck is*procedural strategy*\(text\-action ReAct agents such as ALFWorld\), where the state encodes exactly the procedures a weak base policy lacks; it helps least when a strong backbone already follows a detailed tool API \(τ\\tau\-bench\) or when the bottleneck is retrieval/grounding \(WebShop, GAIA\)\. This predicts*which*of many proposed methods will transfer to a new agent setting\.

### Why our framing is selection\-first\.

The same data that makes the benchmark\-dependence visible also shows that the dangerous failures \(DC’s WebShop collapse\) come from*committing*context without a held\-out check\. RSEA’s contribution is to make that check strict and central, which is what converts a high\-variance idea into a safe one\.

### Limitations\.

The transfer benchmarks use a single seed of the shuffled split and modest eval sizes; GAIA uses live web retrieval, which adds run\-to\-run noise \(the≤1\\leq 1\-task gap flips sign across runs\)\. Our scope is the weight\-frozen, NL\-state regime; we do not compare to code\- or weight\-updating self\-improvement\. The strict gate guarantees safety on held\-out val, not on every test draw when val is small\.

## 11Conclusion

Apples\-to\-apples against six classic and recent context\-evolution methods on one shared backbone, no NL artifact universally wins, and unguarded evolution is unsafe\. RSEA’s contribution is a strict held\-out selection gate over a complementary three\-layer state: it yields significant gains where strategy is the bottleneck \(ALFWorld\), is best overall with retry, and – unlike the baselines – never significantly regresses anywhere\. We release the harness, baseline re\-implementations, and manifests to support faithful comparison\.

## References

- GEPA: reflective prompt evolution can outperform reinforcement learning\.arXiv preprint arXiv:2507\.19457\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px3.p1.1),[§3](https://arxiv.org/html/2606.28374#S3.SS0.SSS0.Px2.p1.1)\.
- H\. Cai, B\. Shen, L\. Jin, L\. Hu, and X\. Fan \(2025\)Does tone change the answer? evaluating prompt politeness effects on modern llms: gpt, gemini, llama\.arXiv preprint arXiv:2512\.12812\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px3.p1.1)\.
- X\. Chen and M\. Zeng \(2025\)Prototype conditioned generative replay for continual learning in NLP\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(NAACL\-HLT\),pp\. 12754–12770\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- Z\. Fang, Z\. Liu, J\. Liu, H\. Chen, Y\. Zeng, S\. Huang, Z\. Chen, L\. Chen, S\. Zhang, and F\. Zhao \(2025\)DualVLA: building a generalizable embodied agent via partial decoupling of reasoning and action\.arXiv preprint arXiv:2511\.22134\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, and T\. Rocktäschel \(2023\)Promptbreeder: self\-referential self\-improvement via prompt evolution\.arXiv preprint arXiv:2309\.16797\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px3.p1.1)\.
- R\. Han, Z\. Fang, X\. Sun, Y\. Ma, Z\. Wang, Y\. Zeng, Z\. Chen, L\. Chen, W\. Huang, W\. Xu,et al\.\(2026\)UniCorn: towards self\-improving unified multimodal models through self\-generated supervision\.arXiv preprint arXiv:2601\.03193\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Hu, C\. Lu, and J\. Clune \(2024\)Automated design of agentic systems\.arXiv preprint arXiv:2408\.08435\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Huang, Z\. Fang, Z\. Chen, S\. Yuan, J\. Ye, Y\. Zeng, L\. Chen, Q\. Mao, and F\. Zhao \(2025\)CRITICTOOL: evaluating self\-critique capabilities of large language models in tool\-calling error scenarios\.arXiv preprint arXiv:2506\.13977\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- W\. Huang, P\. Abbeel, D\. Pathak, and I\. Mordatch \(2022\)Language models as zero\-shot planners: extracting actionable knowledge for embodied agents\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang,et al\.\(2024\)DSPy: compiling declarative language model calls into self\-improving pipelines\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px3.p1.1)\.
- G\. Lan, H\. A\. Inan, S\. Abdelnabi, J\. Kulkarni, L\. Wutschitz, R\. Shokri, C\. G\. Brinton, and R\. Sim \(2025a\)Contextual integrity in LLMs via reasoning and reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px5.p1.1)\.
- G\. Lan, S\. Zhang, T\. Wang, Y\. Zhang, D\. Zhang, X\. Wei, X\. Pan, H\. Zhang, D\. Han, and C\. G\. Brinton \(2025b\)MaPPO: maximum a posteriori preference optimization with prior knowledge\.arXiv preprint arXiv:2507\.21183\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px5.p1.1)\.
- Q\. Lan, J\. I\. Choi, and Q\. Tian \(2026a\)Visual detector compression via location\-aware discriminant analysis\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\),pp\. 3546–3555\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Q\. Lan, Y\. Hsu, N\. S\. Khan, and X\. Jiang \(2026b\)ReCo\-KD: region\- and context\-aware knowledge distillation for efficient 3d medical image segmentation\.arXiv preprint arXiv:2601\.08301\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Q\. Lan and Q\. Tian \(2025\)ACAM\-KD: adaptive and cooperative attention masking for knowledge distillation\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 3957–3966\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- F\. Li, J\. Wu, T\. Fu, D\. Li, H\. Wan, W\. Zhou, and M\. Kan \(2026a\)What’s left unsaid? detecting and correcting misleading omissions in multimodal news previews\.arXiv preprint arXiv:2601\.05563\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- F\. Li, J\. Wu, C\. He, and W\. Zhou \(2025a\)CMIE: combining mllm insights with external evidence for explainable out\-of\-context misinformation detection\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 9342–9354\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- L\. Li, Z\. Zhou, J\. Hao, J\. K\. Liu, Y\. Miao, W\. Pang, X\. Tan, W\. Chu, Z\. Wang, S\. Pan,et al\.\(2025b\)The choice of divergence: a neglected key to mitigating diversity collapse in reinforcement learning with verifiable reward\.arXiv preprint arXiv:2509\.07430\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px5.p1.1)\.
- L\. Li, Z\. Zhou, J\. Long, P\. Liu, W\. Xu, Z\. Wang, S\. Pan, and C\. Qu \(2026b\)SQL\-ASTRA: alleviating sparse feedback in agentic sql via column\-set matching and trajectory aggregation\.External Links:2603\.16161,[Link](https://arxiv.org/abs/2603.16161)Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Li, T\. Lan, and Z\. Qi \(2025c\)INSPO: unlocking intrinsic self\-reflection for llm preference optimization\.arXiv preprint arXiv:2512\.23126\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px5.p1.1)\.
- Y\. Li, T\. Lan, and Z\. Qi \(2026c\)When right meets wrong: bilateral context conditioning with reward\-confidence correction for grpo\.arXiv preprint arXiv:2603\.13134\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px5.p1.1)\.
- Y\. Li, R\. Miao, Z\. Qi, and T\. Lan \(2026d\)ARISE: agent reasoning with intrinsic skill evolution in hierarchical reinforcement learning\.arXiv preprint arXiv:2603\.16060\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Lian, Y\. Wu, J\. Ma, Y\. Ding, Z\. Song, B\. Chen, X\. Zheng, and H\. Li \(2025\)UI\-AGILE: advancing gui agents with effective reinforcement learning and precise inference\-time grounding\.arXiv preprint arXiv:2507\.22025\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- W\. Liu, H\. Wu, Y\. Kuang, X\. Han, T\. Zhong, J\. Feng, and W\. Lu \(2026\)Automated optimization modeling via a localizable error\-driven perspective\.arXiv preprint arXiv:2602\.11164\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- J\. Lu, Z\. Kong, Y\. Wang, R\. Fu, H\. Wan, C\. Yang, W\. Lou, H\. Sun, L\. Wang, Y\. Jiang,et al\.\(2026\)Beyond static tools: test\-time tool evolution for scientific reasoning\.arXiv preprint arXiv:2601\.07641\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Mialon, C\. Fourrier, C\. Swift, T\. Wolf, Y\. LeCun, and T\. Scialom \(2023\)GAIA: a benchmark for general ai assistants\.arXiv preprint arXiv:2311\.12983\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- K\. Opsahl\-Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. Khattab \(2024\)Optimizing instructions and demonstrations for multi\-stage language model programs\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez \(2023\)MemGPT: towards llms as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein \(2023\)Generative agents: interactive simulacra of human behavior\.InACM Symposium on User Interface Software and Technology \(UIST\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Qu, S\. Dai, H\. Cai, J\. Xu, S\. Wang, and D\. Yin \(2026\)MatchTIR: fine\-grained supervision for tool\-integrated reasoning via bipartite matching\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- C\. Qu, S\. Dai, X\. Wei, H\. Cai, S\. Wang, D\. Yin, J\. Xu, and J\. Wen \(2025a\)From exploration to mastery: enabling llms to master tools via self\-driven interactions\.InThe Thirteenth International Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=QKBu1BOAwd)Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- C\. Qu, S\. Dai, X\. Wei, H\. Cai, S\. Wang, D\. Yin, J\. Xu, and J\. Wen \(2025b\)Tool learning with large language models: a survey\.Frontiers of Computer Science19\(8\),pp\. 198343\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023\)Toolformer: language models can teach themselves to use tools\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§3](https://arxiv.org/html/2606.28374#S3.SS0.SSS0.Px2.p1.1)\.
- M\. Shridhar, X\. Yuan, M\. Côté, Y\. Bisk, A\. Trischler, and M\. Hausknecht \(2021\)ALFWorld: aligning text and embodied environments for interactive learning\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- H\. Song, D\. Qu, Y\. Yao, Q\. Chen, Q\. Lv, Y\. Tang, M\. Shi, G\. Ren, M\. Yao, B\. Zhao,et al\.\(2025\)Hume: introducing system\-2 thinking in visual\-language\-action model\.arXiv preprint arXiv:2505\.21432\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Su \(2026\)Agentic\-SQL taxonomy: a survey of autonomous and interactive text\-to\-SQL with llms\.External Links:[Document](https://dx.doi.org/10.13140/RG.2.2.10301.32484)Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- T\. R\. Sumers, S\. Yao, K\. Narasimhan, and T\. L\. Griffiths \(2024\)Cognitive architectures for language agents\.Transactions on Machine Learning Research \(TMLR\)\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Suzgun, M\. Yuksekgonul, F\. Bianchi, D\. Jurafsky, and J\. Zou \(2025\)Dynamic cheatsheet: test\-time learning with adaptive memory\.arXiv preprint arXiv:2504\.07952\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1),[§3](https://arxiv.org/html/2606.28374#S3.SS0.SSS0.Px2.p1.1)\.
- H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. Balasubramanian \(2024\)AppWorld: a controllable world of apps and people for benchmarking interactive coding agents\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar \(2023\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- L\. Wang, C\. Ma, X\. Feng,et al\.\(2024a\)A survey on large language model based autonomous agents\.Frontiers of Computer Science\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. Neubig \(2024b\)Agent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1),[§3](https://arxiv.org/html/2606.28374#S3.SS0.SSS0.Px2.p1.1)\.
- H\. Wu, H\. Li, and Y\. Su \(2025\)Bridging the perception\-cognition gap: re\-engineering SAM2 with hilbert\-mamba for robust vlm\-based medical diagnosis\.In2025 IEEE International Conference on Bioinformatics and Biomedicine \(BIBM\),pp\. 4275–4278\.External Links:[Document](https://dx.doi.org/10.1109/BIBM66473.2025.11357220)Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Z\. Xie, X\. Liu, B\. Zhang, Y\. Lin, S\. Cai, and T\. Jin \(2026a\)HVD: human vision\-driven video representation learning for text\-video retrieval\.arXiv preprint arXiv:2601\.16155\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Z\. Xie, C\. Wang, Y\. Wang, S\. Cai, S\. Wang, and T\. Jin \(2025\)Chat\-driven text generation and interaction for person retrieval\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 5259–5270\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Z\. Xie, B\. Zhang, Y\. Lin, and T\. Jin \(2026b\)Delving deeper: hierarchical visual perception for robust video\-text retrieval\.arXiv preprint arXiv:2601\.12768\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Z\. Xie \(2026\)CONQUER: context\-aware representation with query enhancement for text\-based person search\.arXiv preprint arXiv:2601\.18625\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Y\. Xu, F\. Tang, J\. Cao, X\. Kong, Y\. Zhang, J\. Li, O\. Deussen, and T\. Lee \(2024\)Headrouter: a training\-free image editing framework for mm\-dits by adaptively routing attention heads\.ACM Transactions on Graphics\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. Chen \(2024\)Large language models as optimizers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Yang, C\. Han, S\. Wang, Y\. Li, Y\. Ding, and E\. Hovy \(2026a\)Toward understanding misalignment in LLM agents: a survey of taxonomy, causes, mitigation, and evaluation\.Note:ACL ARR 2026External Links:[Document](https://dx.doi.org/10.13140/RG.2.2.16323.57126),[Link](https://openreview.net/forum?id=zzTEGP2BYa)Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Yang, S\. C\. Han, Y\. Ding, S\. Wang, and E\. Hovy \(2026b\)ToolTree: efficient llm agent tool planning via dual\-feedback monte carlo tree search and bidirectional pruning\.Note:ICLR 2026External Links:2603\.12740Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Yang, S\. C\. Han, X\. Ma, Y\. Li, M\. R\. Ghasemi Madani, and E\. Hovy \(2026c\)EvoTool: self\-evolving tool\-use policy optimization in LLM agents via blame\-aware mutation and diversity\-aware selection\.Note:ACL 2026External Links:2603\.04900Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Yao, H\. Chen, J\. Yang, and K\. Narasimhan \(2022\)WebShop: towards scalable real\-world web interaction with grounded language agents\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan \(2024\)τ\\tau\-bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan \(2023a\)Tree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023b\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- E\. Zelikman, Y\. Wu, J\. Mu, and N\. D\. Goodman \(2022\)STaR: bootstrapping reasoning with reasoning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Zeng, X\. Chang, M\. Xie, X\. Liu, Y\. Bai, Z\. Pan, M\. Xu, and X\. Wei \(2025a\)FutureSightDrive: thinking visually with spatio\-temporal cot for autonomous driving\.arXiv preprint arXiv:2505\.17685\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Zeng, D\. Qi, X\. Chang, F\. Xiong, S\. Xie, X\. Wu, S\. Liang, M\. Xu, and X\. Wei \(2025b\)JanusVLN: decoupling semantics and spatiality with dual implicit memory for vision\-language navigation\.arXiv preprint arXiv:2509\.22548\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- Y\. Zeng, W\. Huang, Z\. Fang, S\. Chen, Y\. Shen, Y\. Cai, X\. Wang, Z\. Yin, L\. Chen, Z\. Chen,et al\.\(2026\)Vision\-deepresearch benchmark: rethinking visual and textual search for multimodal large language models\.arXiv preprint arXiv:2602\.02185\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- L\. Zhang, R\. Awal, and A\. Agrawal \(2024a\)Contrasting intra\-modal and ranking cross\-modal hard negatives to enhance visio\-linguistic compositional understanding\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 13774–13784\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- L\. Zhang, B\. Wang, X\. Qiu, S\. Reddy, and A\. Agrawal \(2025a\)Rearank: reasoning re\-ranking agent via reinforcement learning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 2458–2471\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px5.p1.1)\.
- L\. Zhang, Y\. Wu, Q\. Yang, and J\. Nie \(2024b\)Exploring the best practices of query expansion with large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 1872–1883\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- L\. Zhang, Q\. Yang, and A\. Agrawal \(2025b\)Assessing and learning alignment of unimodal vision and language models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 14604–14614\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Q\. Zhang, C\. Hu,et al\.\(2025c\)Agentic context engineering: evolving contexts for self\-improving language models\.arXiv preprint arXiv:2510\.04618\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1),[§3](https://arxiv.org/html/2606.28374#S3.SS0.SSS0.Px2.p1.1)\.
- X\. Zhang, Y\. Zhang, Z\. Chen, J\. Yu, W\. Yang, and Z\. Song \(2026a\)Logical phase transitions: understanding collapse in llm logical reasoning\.arXiv preprint arXiv:2601\.02902\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- Y\. Zhang, Y\. Ai, Z\. Ying, Q\. Mi, J\. Yu, W\. Yang, and Z\. Song \(2026b\)Coupling macro dynamics and micro states for long\-horizon social simulation\.External Links:2604\.05516,[Link](https://arxiv.org/abs/2604.05516)Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Y\. Zhang, Z\. Song, H\. Zhou, W\. Ren, Y\. P\. Chen, J\. Yu, and W\. Yang \(2025d\)GA\-S3: comprehensive social network simulation with group agents\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 8950–8970\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- Y\. Zhang, X\. Zhang, J\. Sheng, W\. Li, J\. Yu, Y\. P\. Chen, W\. Yang, and Z\. Song \(2025e\)From ambiguity to verdict: a semiotic\-grounded multi\-perspective agent for llm logical reasoning\.arXiv preprint arXiv:2509\.24765\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Zhang, Z\. Chen, H\. Zhu, Z\. Chen, N\. Du, and X\. Li \(2025f\)ToolExpNet: optimizing multi\-tool selection in LLMs with similarity and dependency\-aware experience networks\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 15706–15722\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Zhang, Z\. Chen, M\. Li, Z\. Tu, and X\. Li \(2026c\)RLVMR: reinforcement learning with verifiable meta\-reasoning rewards for robust long\-horizon agents\.InThe Fourteenth International Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=cTbAevdwBE)Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px5.p1.1)\.
- A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. Huang \(2024\)ExpeL: llm agents are experiential learners\.InAAAI Conference on Artificial Intelligence,Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- X\. Zhou, O\. Wu, and N\. Yang \(2024a\)Adversarial training with anti\-adversaries\.IEEE Transactions on Pattern Analysis and Machine Intelligence \(TPAMI\)46\(12\),pp\. 10210–10227\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- X\. Zhou, O\. Wu, W\. Zhu, and Z\. Liang \(2022\)Understanding difficulty\-based sample weighting with a universal difficulty measure\.InJoint European Conference on Machine Learning and Knowledge Discovery in Databases \(ECML\-PKDD\),pp\. 68–84\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- X\. Zhou, W\. Ye, Z\. Lee, R\. Xie, and S\. Zhang \(2024b\)Boosting model resilience via implicit adversarial data augmentation\.arXiv preprint arXiv:2404\.16307\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px6.p1.1)\.
- Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba \(2023\)Large language models are human\-level prompt engineers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Zhu, Y\. Lin, S\. Chen, Y\. Wang, and J\. Lin \(2025a\)MedEyes: learning dynamic visual focus for medical progressive diagnosis\.arXiv preprint arXiv:2511\.22018\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px7.p1.1)\.
- C\. Zhu, Y\. Lin, J\. Shao, J\. Lin, and Y\. Wang \(2025b\)Pathology\-aware prototype evolution via LLM\-driven semantic disambiguation for multicenter diabetic retinopathy diagnosis\.InProceedings of the 33rd ACM International Conference on Multimedia,pp\. 9196–9205\.Cited by:[§2](https://arxiv.org/html/2606.28374#S2.SS0.SSS0.Px4.p1.1)\.

## Appendix AEvolved states \(qualitative\)

The evolved three\-layer states are interpretable, which lets us read*why*the method helps on ALFWorld and is neutral on the strong tool\-use backbone\.

### ALFWorld \(frozen best state\)\.

Strategy:“Avoid repeating identical actions that return ‘Nothing happens\.’; always take objects before placing; use desklamps for examining objects\.”Skills:“if not \(take<obj\>from<recep\>\), go to another location and try again”; “use<desklamp\>before examining objects”; “never repeat an identical action that returns ‘Nothing happens\.’ ”\.Playbook:“pick\_and\_place: go to<recep\>with<obj\>→\\rightarrowtake<obj\>→\\rightarrowgo to<recep\>→\\rightarrowput<obj\>”; analogouscool/heatsequences\. These are precisely the failure modes of the 7B ReAct agent, and injecting them yields the significant ALFWorld gain\.

### τ\\tau\-bench retail \(frozen best state\)\.

Strategy:“Always authenticate user identity using email first; if not found, fall back to name \+ zip\. Confirm all modifications/cancellations/exchanges with the user before any tool call\. Never invent order/item IDs – retrieve and verify them\. Only exchange delivered orders; for pending orders usemodify\_pending\_order\_items…” This is a faithful summary of the retail policy; the 30B agent already follows most of it from the wiki, so the held\-out gain is neutral – consistent with our scope condition\.

## Appendix BReproducibility

All methods share one locally served backbone and decoding budget; task and reflection LLMs are identical\. We release the harness, the faithful re\-implementations of all six baselines, the pre\-registered task manifests, the evolve/val/eval splits, and per\-task result JSONs\. ALFWorld numbers are 5 seeds with paired McNemar over matched \(task, seed\) cells; transfer benchmarks use a seed\-shuffled disjoint split with paired McNemar over tasks; ablations use the ALFWorld test set with temp\-0\.6 multi\-seed decoding\.