APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering

# APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering
Source: [https://arxiv.org/html/2609.02253](https://arxiv.org/html/2609.02253)
Rui Sun11footnotemark:1Affiliation:University of Science and Technology of ChinaXinyuan ZhangAffiliation:ByteDance Inc\.\{jieding25,rrsun\}@mail\.ustc\.edu\.cn,xliu@gia\.cas\.cnZeyu ZhangAffiliation:Chinese Academy of SciencesXin Liu††thanks:Corresponding author\.Affiliation:Chinese Academy of Sciences

###### Abstract

Deep research agents augment large language models with external tools to answer complex, long\-horizon questions through multi\-turn reasoning\. Learning from prior experience is crucial for continual improvement, yet existing methods either retrieve verbose task\-specific traces that burden decision\-making, or distill procedural skills that remain decoupled from downstream policy adaptation\. We proposeAPEx, a hierarchical experience utilization framework that organizes interaction history into instance\-level trajectory memories and category\-level procedural skills, and couples them through a closed\-loop architecture of Executor, Distiller, and Planner\. The three modules are optimized via a three\-stage alternating GRPO training paradigm, enabling reward\-guided skill distillation rather than fixed\-prompt generation\. At test time, distilled skills serve as procedural priors for online Planner adaptation through skill\-guided test\-time reinforcement learning, allowing ground\-truth\-free self\-improvement with skill\-alignment regularization to prevent policy drift\. Experiments on 7 benchmarks demonstrate thatAPExachieves state\-of\-the\-art performance, surpassing GPT\-5\.4 by 14\.7 points and the strongest memory\-augmented baseline by 3\.0 points\. Code is available at[https://github\.com/J\-Ding519/APEx](https://github.com/J-Ding519/APEx)\.

## 1Introduction

Deep research agents \(DRAs\) augment large language models \(LLMs\) with external tools to tackle complex, long\-horizon, and open\-ended questions through multi\-turn answering[Sun et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib3);[Sun et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib58);[Li et al\. \(2026b\)](https://arxiv.org/html/2609.02253#bib.bib5);[Zhu et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib6)\. These tasks often involve multi\-step tool orchestration[Wu et al\. \(2025a\)](https://arxiv.org/html/2609.02253#bib.bib4);[Li et al\. \(2026a\)](https://arxiv.org/html/2609.02253#bib.bib29), iterative information retrieval[Jin et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib9);[Wu et al\. \(2025b\)](https://arxiv.org/html/2609.02253#bib.bib8), and multi\-source synthesis[Chen et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib30), making learning from prior experience essential\. Without effective experience utilization, DRAs risk repeating past mistakes instead of building on prior successes[Wang et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib31);[Wang and Chen \(2025\)](https://arxiv.org/html/2609.02253#bib.bib32)\. This raises a central question: how can deep research agents continually improve through long\-term interaction experience?

![Refer to caption](https://arxiv.org/html/2609.02253v1/moti.png)Figure 1:Comparison between existing memory\-based agent methods and our adaptive procedural experience learning framework\.Recent work has attempted to answer this question by equipping agents with memory mechanisms that store, retrieve, and reuse prior experience[Zhou et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib33);[Qiao et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib13)\. Episodic memory methods store and retrieve past trajectories or task instances[Zhong et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib17);[Zhao et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib15), but retrieved experiences are often lengthy, task\-specific, and noisy, forcing agents to infer transferable strategies at decision time\. Reflective memory methods compress feedback into textual summaries[Shinn et al\. \(2023\)](https://arxiv.org/html/2609.02253#bib.bib34);[Tan et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib36), yet these summaries are typically generated by fixed prompts or optimized for memory personalization, rather than optimized for their downstream utility in planning and execution\. Procedural memory methods further represent experience as reusable skills, plans, or instructions[Zhou et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib14);[Ma et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib35);[Xia et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib22), but such skills are often treated as external knowledge artifacts rather than adaptive priors that directly optimize the agent policy during inference\.

These suggest that experience\-driven improvement in DRAs should be formulated as a closed loop rather than a one\-way memory reuse process[Zhou et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib33)\. Long\-term interaction experience should be distilled into structured, reusable skills[Ma et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib35); the generation of these skills should be optimized by downstream reinforcement learning signals rather than fixed prompts[Xia et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib22); and the resulting skills should serve as procedural priors for test\-time reinforcement learning \(TTRL\), where the agent updates its parameters during inference to adapt to new tasks[Qiao et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib13)\. As TTRL improves the agent, the agent can generate higher\-quality trajectories, which in turn support better skill distillation and further model improvement\.

To realize experience\-driven improvement, we proposeAPEx\(AdaptiveProceduralExperience Learning\)\. It is characterized by the following key features: ❶Hierarchical Experience Memory\.APExorganizes long\-term trajectories into instance\-level memories and category\-level skills, preserving concrete task\-solving traces while abstracting reusable procedural knowledge for planning\. ❷Experience\-to\-Action Agent Loop\.Built on this hierarchy,APExintegrates an Executor, a Distiller, and a Planner into an experience\-to\-action loop, where tool\-use trajectories are collected, abstracted into skills, and reused for strategic planning\. This design aligns execution, abstraction, and planning for continual agent improvement\. ❸Skill\-guided Test\-time Adaptation\.During deployment,APExuses retrieved skills and memories as procedural priors for TTRL, allowing the Planner to adapt on unlabeled queries and consolidate new trajectories into future memories and skills\. Our contributions are as follows\.

- ❶Framework\.We proposeAPEx, a hierarchical experience utilization architecture that couples instance\-level memories with category\-level skills, enabling the Distiller, Planner, and Executor to transform concrete trajectories into generalizable procedural guidance\.
- ❷Training\.We introduce a three\-stage alternating GRPO training paradigm for the Executor, Distiller, and Planner, which stabilizes cross\-module credit assignment and makes skill distillation a reward\-guided learning process rather than fixed prompting\.
- ❸Reasoning\.We develop Skill\-guided TTRL, which uses distilled skills to regularize online Planner adaptation on unlabeled queries\. Coupled with no\-ground\-truth rewards, it forms a feedback loop that turns improved trajectories into stronger skills for future adaptation\.
- ❹Experiment\.Extensive experiments on diverse benchmarks demonstrate thatAPExachieves SOTA performance \(64\.4% avg\), surpassing GPT\-5\.4 by 14\.7 points and the strongest memory\-augmented baseline by 3\.0 points, while the skill\-only variant reduces injected memory tokens, validating the context efficiency of hierarchical experience utilization over flat memory retrieval\.

## 2Methodology Overview

![Refer to caption](https://arxiv.org/html/2609.02253v1/frame.png)Figure 2:Framework ofAPEx, which comprises 4 parts: memory, framework, optimization, and TTRL\.Figure[2](https://arxiv.org/html/2609.02253#S2.F2)illustrates the overall workflow ofAPEx, a hierarchical experience utilization framework for deep research agents\.APExmaintains long\-term experience at two complementary levels: instance\-level memories that preserve concrete execution trajectories, and category\-level skills that summarize reusable procedural knowledge across related tasks\. Built upon this experience hierarchy,APExcouples planning, execution, skill distillation, and TTRL into a closed\-loop improvement process\.

#### APExAgents Workflow\.

Given an original queryqq,APExprocesses it through a closed\-loop data flow\. The queryqqis first routed to a category–modality bucket, from which relevant instance\-level memoriesℛM​\(q\)\\mathcal\{R\}\_\{M\}\(q\)and the corresponding category\-level skillℛS​\(q\)\\mathcal\{R\}\_\{S\}\(q\)are retrieved\. These experiences are organized into a structured context:

E⁡\(q\)=ℛS​\(q\)⊕ℛM​\(q\),E\(q\)=\\mathcal\{R\}\_\{S\}\(q\)\\oplus\\mathcal\{R\}\_\{M\}\(q\),\(1\)where⊕\\oplusdenotes structured prompt concatenation\. The retrieval method is detailed in Appendix[A](https://arxiv.org/html/2609.02253#A1)\. Conditioned onqqandE⁡\(q\)E\(q\), the Planner generates a research plan:

p=πψplan​\(q,E⁡\(q\)\)\.p=\\pi\_\{\\psi\}^\{\\mathrm\{plan\}\}\(q,E\(q\)\)\.\(2\)The Executor then follows the plan to perform multi\-turn tool interactions and produce an execution trajectoryτ\\tauwith the final answery^\\hat\{y\}:

τ,y^=πθexec​\(q,p∣𝒯\),\\tau,\\hat\{y\}=\\pi\_\{\\theta\}^\{\\mathrm\{exec\}\}\(q,p\\mid\\mathcal\{T\}\),\(3\)where𝒯\\mathcal\{T\}denotes the available tool set\. After execution, the trajectory is summarized and stored as a new instance\-level memory, while the corresponding category\-level skillsc,ds\_\{c,d\}is updated when sufficient new experience has accumulated:

ℳt\+1\\displaystyle\\mathcal\{M\}^\{t\+1\}=ℳt∪Consolidate⁡\(q,τ,y^\),\\displaystyle=\\mathcal\{M\}^\{t\}\\cup\\operatorname\{Consolidate\}\(q,\\tau,\\hat\{y\}\),\(4\)sc,dt\+1\\displaystyle s\_\{c,d\}^\{t\+1\}=ϕskill​\(ℳc,dt\+1,sc,dt\),\\displaystyle=\\phi\_\{\\mathrm\{skill\}\}\(\\mathcal\{M\}\_\{c,d\}^\{t\+1\},s\_\{c,d\}^\{t\}\),whereℳc,dt∈ℳt\\mathcal\{M\}\_\{c,d\}^\{t\}\\in\\mathcal\{M\}^\{t\}andℳt\\mathcal\{M\}^\{t\}is the instance\-level memory bank at timestampttandϕskill\\phi\_\{\\mathrm\{skill\}\}is the distillation operation of Distiller\.

#### Group Relative Policy Optimization \(GRPO\)\.

GRPO[Shao et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib27)optimizes the policy with intra\-group relative rewards\. In our work Distiller, Planner and Executor are optimized with GRPO\. For each inputxx, the policy samples a group ofGGresponses\{oi\}i=1G\\\{o\_\{i\}\\\}\_\{i=1\}^\{G\}and computes the group\-normalized advantage asA^i=\(R⁡\(oi\)−meanj⁡\[R⁡\(oj\)\]\)/\(stdj⁡\[R⁡\(oj\)\]\+ϵ\)\\hat\{A\}\_\{i\}=\(R\(o\_\{i\}\)\-\\operatorname\{mean\}\_\{j\}\[R\(o\_\{j\}\)\]\)/\(\\operatorname\{std\}\_\{j\}\[R\(o\_\{j\}\)\]\+\\epsilon\)\. Following the GRPO implementation, the policy objective for the currently optimized moduleπη\\pi\_\{\\eta\}is:

ℒGRPO\(η\)=−𝔼x,\{oi\}\[1G∑i=1G1\|oi\|∑t=1\|oi\|\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}\(\\eta\)=\{\}\-\\mathbb\{E\}\_\{x,\\\{o\_\{i\}\\\}\}\\Bigg\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\frac\{1\}\{\|o\_\{i\}\|\}\\sum\_\{t=1\}^\{\|o\_\{i\}\|\}\(5\)min\(ri,t\(η\)A^i,clip\(ri,t\(η\),1−ϵ,1\+ϵ\)A^i\)\],\\displaystyle\\min\\Big\(r\_\{i,t\}\(\\eta\)\\hat\{A\}\_\{i\},\\operatorname\{clip\}\\big\(r\_\{i,t\}\(\\eta\),1\-\\epsilon,1\+\\epsilon\\big\)\\hat\{A\}\_\{i\}\\Big\)\\Bigg\],whereri,t​\(η\)=πη​\(oi,t∣x,oi,<t\)/πηold​\(oi,t∣x,oi,<t\)r\_\{i,t\}\(\\eta\)=\\pi\_\{\\eta\}\(o\_\{i,t\}\\mid x,o\_\{i,<t\}\)/\\pi\_\{\\eta\_\{\\mathrm\{old\}\}\}\(o\_\{i,t\}\\mid x,o\_\{i,<t\}\)is the token\-level probability ratio\.

#### Test\-time Reinforcement Learning \(TTRL\)\.

Unlike standard TTRL based on majority voting[Zuo et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib55),APExconstructs rewards using a multi\-judge LLM\-as\-Judge mechanism[Zheng et al\. \(2023\)](https://arxiv.org/html/2609.02253#bib.bib56)\. In our implementation, we update the Planner at test time\. Given a test queryqq, the Planner first retrieves the corresponding skillℛS​\(q\)\\mathcal\{R\}\_\{S\}\(q\)and memoryℛM​\(q\)\\mathcal\{R\}\_\{M\}\(q\), generates a planpp, and the Executor produces an execution trajectoryτ\\tauand final answery^\\hat\{y\}\. The Planner’s parametersψ\\psiare updated online by GRPO using the final rewardRfinalR\_\{\\mathrm\{final\}\}:

ψt\+1=GRPO⁡\(ψt,Rfinal\)\.\\psi^\{t\+1\}=\\operatorname\{GRPO\}\\left\(\\psi^\{t\},R\_\{\\mathrm\{final\}\}\\right\)\.\(6\)

## 3Methodology

### 3\.1Hierarchical Experience Memory

To enable efficient experience utilization,APExorganizes experience into a hierarchical memory structure, where instance\-level memories preserve concrete task\-solving traces and category\-level skills summarize reusable procedural knowledge\.

#### Instance\-level Trajectories Memory\.

Given theii\-th original queryqiq\_\{i\}, the agent performs multi\-turn tool interactions and produces an execution trajectoryτi\\tau\_\{i\}\. Each instance\-level memory is represented asmi=\(qi,ξi,yi,wi\)m\_\{i\}=\(q\_\{i\},\\xi\_\{i\},y\_\{i\},w\_\{i\}\), whereξi\\xi\_\{i\}is the workflow summary,yi∈\{correct,incorrect\}y\_\{i\}\\in\\\{\\mathrm\{correct\},\\mathrm\{incorrect\}\\\}denotes the execution outcome, andwi=success​\_​counti/usage​\_​countiw\_\{i\}=\\operatorname\{success\\\_count\}\_\{i\}/\\operatorname\{usage\\\_count\}\_\{i\}is the memory reuse win rate\. All instance\-level memories are organized into a memory bankℳ\\mathcal\{M\}according to task category and input modality:

ℳ=⋃c∈𝒞⋃d∈𝒟ℳc,d,\\mathcal\{M\}=\\bigcup\_\{c\\in\\mathcal\{C\}\}\\bigcup\_\{d\\in\\mathcal\{D\}\}\\mathcal\{M\}\_\{c,d\},\(7\)where𝒞\\mathcal\{C\}is the set of predefined task categories and𝒟\\mathcal\{D\}is the set of modalities\.

#### Category\-level Skills Memory\.

While instance\-level memories provide concrete task\-solving cases, they may contain task\-specific details\. To extract reusable procedural knowledge, the Distiller synthesizes the memories within each category–modality bucket into a single evolving skill documentsc,ds\_\{c,d\}:

sc,d=ϕskill​\(ℳc,d\),s\_\{c,d\}=\\phi\_\{\\mathrm\{skill\}\}\(\\mathcal\{M\}\_\{c,d\}\),\(8\)whereϕskill\\phi\_\{\\mathrm\{skill\}\}denotes the distillation operation of Distiller\. A skill entrysc,ds\_\{c,d\}is defined assc,d=\(Pc,d,Fc,d,wc,d,nc,d,descc,d\)s\_\{c,d\}=\(P\_\{c,d\},F\_\{c,d\},w\_\{c,d\},n\_\{c,d\},\\operatorname\{desc\}\_\{c,d\}\), wherePc,dP\_\{c,d\}denotes recommended planning steps,Fc,dF\_\{c,d\}denotes common failure modes,nc,dn\_\{c,d\}denotes the number of supporting memories,wc,d=correct​\_​countc,d/nc,dw\_\{c,d\}=\\operatorname\{correct\\\_count\}\_\{c,d\}/n\_\{c,d\}denotes the skill\-level win rate, anddescc,d\\operatorname\{desc\}\_\{c,d\}provides a natural\-language description\. Finally, the Skill memory bank𝒮\\mathcal\{S\}is defined as:

𝒮=\{sc,d∣c∈𝒞,d∈𝒟\}\.\\mathcal\{S\}=\\\{s\_\{c,d\}\\mid c\\in\\mathcal\{C\},d\\in\\mathcal\{D\}\\\}\.\(9\)

### 3\.2Three\-Stage Alternating RL Training

To optimize planning, execution, and experience distillation,APExadopts a three\-stage alternating reinforcement learning framework\. The framework contains three trainable modules: the Executorπθexec\\pi\_\{\\theta\}^\{\\mathrm\{exec\}\}, the Distillerπϕdist\\pi\_\{\\phi\}^\{\\mathrm\{dist\}\}, and the Plannerπψplan\\pi\_\{\\psi\}^\{\\mathrm\{plan\}\}\.

#### Executor Training\.

In the first stage, the Planner and Distiller are frozen, and the Executor is optimized to perform multi\-turn tool interaction under a given plan\. Given a queryqqand a planpp, the Executor generates a trajectoryτ=πθexec​\(q,p∣𝒯\)\\tau=\\pi\_\{\\theta\}^\{\\mathrm\{exec\}\}\(q,p\\mid\\mathcal\{T\}\), where𝒯\\mathcal\{T\}denotes the available tool set\.

The Executor reward combines answer correctness, tool\-use effectiveness, and format validity:

Rexec​\(τ\)=\\displaystyle R^\{\\mathrm\{exec\}\}\(\\tau\)=λ1​Racc​\(τ\)\+λ2​Rtool​\(τ\)\\displaystyle\\lambda\_\{1\}R\_\{\\mathrm\{acc\}\}\(\\tau\)\+\\lambda\_\{2\}R\_\{\\mathrm\{tool\}\}\(\\tau\)\(10\)\+λ3​Rfmt​\(τ\),\\displaystyle\+\\lambda\_\{3\}R\_\{\\mathrm\{fmt\}\}\(\\tau\),whereλ1\\lambda\_\{1\},λ2\\lambda\_\{2\}, andλ3\\lambda\_\{3\}are hyperparameters\. The accuracy rewardRacc​\(τ\)=𝒥⁡\(Extract⁡\(τ\),y∗\)R\_\{\\mathrm\{acc\}\}\(\\tau\)=\\mathcal\{J\}\(\\operatorname\{Extract\}\(\\tau\),y^\{\*\}\)is computed by an LLM judge\. The tool rewardRtool\(τ\)=1\|𝒱\|∑v∈𝒱𝕀\[∃tvalid∈v\]R\_\{\\mathrm\{tool\}\}\(\\tau\)=\\frac\{1\}\{\|\\mathcal\{V\}\|\}\\sum\_\{v\\in\\mathcal\{V\}\}\\mathbb\{I\}\\left\[\\exists\\,t\_\{\\mathrm\{valid\}\}\\in v\\right\]encourages each execution segment to contain valid tool usetv​a​l​i​dt\_\{valid\}, where𝒱\\mathcal\{V\}denotes the segments split by replan boundaries\. The format rewardRfmt​\(τ\)∈\{0,1\}R\_\{\\mathrm\{fmt\}\}\(\\tau\)\\in\\\{0,1\\\}checks if the output follows the required structure\.

#### Distiller Training\.

In the second stage, the Executor and Planner are frozen, and the Distiller is optimized to transform instance\-level memories into reusable category\-level skills\. Given a memory bucketℳc,dt\\mathcal\{M\}\_\{c,d\}^\{t\}and an optional previous skillsc,dts\_\{c,d\}^\{t\}, the Distiller outputs an operation typeooand an updated skillsc,dt\+1s\_\{c,d\}^\{t\+1\}, i\.e\.,\(o,sc,dt\+1\)=πϕdist​\(ℳc,dt,sc,dt\)\(o,s\_\{c,d\}^\{t\+1\}\)=\\pi\_\{\\phi\}^\{\\mathrm\{dist\}\}\(\\mathcal\{M\}\_\{c,d\}^\{t\},s\_\{c,d\}^\{t\}\)\. The operationoois selected from\{synthesize,refine,create,skip\}\\\{\\mathrm\{synthesize\},\\mathrm\{refine\},\\mathrm\{create\},\\mathrm\{skip\}\\\}\. We define the Distiller reward as:

Rdist​\(o,sc,dt\+1\)=\\displaystyle R^\{\\mathrm\{dist\}\}\(o,s\_\{c,d\}^\{t\+1\}\)=μ1​Rquality​\(sc,dt\+1\)\+μ2​Rfmt​\(sc,dt\+1\)\\displaystyle\\mu\_\{1\}R\_\{\\mathrm\{quality\}\}\(s\_\{c,d\}^\{t\+1\}\)\+\\mu\_\{2\}R\_\{\\mathrm\{fmt\}\}\(s\_\{c,d\}^\{t\+1\}\)\(11\)\+μ3​Revolve​\(o,sc,dt\+1,sc,dt\),\\displaystyle\+\\mu\_\{3\}R\_\{\\mathrm\{evolve\}\}\(o,s\_\{c,d\}^\{t\+1\},s\_\{c,d\}^\{t\}\),whereμ1\\mu\_\{1\},μ2\\mu\_\{2\}, andμ3\\mu\_\{3\}are hyperparameters andRevolveR\_\{\\mathrm\{evolve\}\}measures if the operationoois rational\. Each reward is judged by an LLM judger𝒥skill\\mathcal\{J\}\_\{\\mathrm\{skill\}\}\.

#### Planner Training\.

In the third stage, the Executor and Distiller are frozen, and the Planner is trained through a plan–execute–evaluate–replan loop\. Given queryqq, retrieved skill guidanceℛS​\(q\)\\mathcal\{R\}\_\{S\}\(q\), and retrieved memoriesℛM​\(q\)\\mathcal\{R\}\_\{M\}\(q\), the Planner first generates an initial plan:

p1=πψplan​\(q,ℛS​\(q\),ℛM​\(q\)\)\.p\_\{1\}=\\pi\_\{\\psi\}^\{\\mathrm\{plan\}\}\\left\(q,\\mathcal\{R\}\_\{S\}\(q\),\\mathcal\{R\}\_\{M\}\(q\)\\right\)\.\(12\)The frozen Executor executesp1p\_\{1\}and produces the first answery^1\\hat\{y\}\_\{1\}\. The Planner then observesTrace⁡\(τ\)\\operatorname\{Trace\}\(\\tau\)and predicts a binary decisiond∈\{0,1\}d\\in\\\{0,1\\\}indicating whether re\-planning is needed\. Ifd=1d=1, the Planner generates a revised planp2p\_\{2\}and obtains a second answery^2\\hat\{y\}\_\{2\}\. Ifd=0d=0, we sety^2=y^1\\hat\{y\}\_\{2\}=\\hat\{y\}\_\{1\}\. The Planner reward is:

Rplan=\\displaystyle R^\{\\mathrm\{plan\}\}=α1​Racc​\(y^2\)\+α2​Racc​\(y^1\)\\displaystyle\\alpha\_\{1\}R\_\{\\mathrm\{acc\}\}\(\\hat\{y\}\_\{2\}\)\+\\alpha\_\{2\}R\_\{\\mathrm\{acc\}\}\(\\hat\{y\}\_\{1\}\)\(13\)\+α3​Rfmt\+α4​Rdec,\\displaystyle\+\\alpha\_\{3\}R\_\{\\mathrm\{fmt\}\}\+\\alpha\_\{4\}R\_\{\\mathrm\{dec\}\},whereα1\\alpha\_\{1\},α2\\alpha\_\{2\},α3\\alpha\_\{3\}, andα4\\alpha\_\{4\}are hyperparameters\. The decision rewardRdecR\_\{\\mathrm\{dec\}\}encourages the Planner to stop when the first execution is correct and to trigger re\-planning when the initial execution fails\.

### 3\.3Skill\-guided TTRL

After the training stage,APExfurther performs Skill\-guided TTRL to adapt the Planner on unlabeled test queries\. Since ground\-truth answers are unavailable at test time,APExconstructs a no\-ground\-truth reward with multiple LLM judges\. Meanwhile, the learned category\-level skills are used as regularization priors to prevent the Planner from drifting away from previously verified procedural knowledge\.

#### No\-Ground\-Truth Reward\.

Without ground\-truth answers,APExestimates answer quality through a multi\-judge LLM\-as\-Judge mechanism\. Specifically, three judges evaluate the execution from different aspects:E1​\(q,τ,y^\)E\_\{1\}\(q,\\tau,\\hat\{y\}\)checks reasoning consistency,E2​\(q,τ,y^\)E\_\{2\}\(q,\\tau,\\hat\{y\}\)checks whether the final answer is supported by retrieved evidence in the trajectory, andE3​\(q,y^\)E\_\{3\}\(q,\\hat\{y\}\)checks whether the response is valid and complete\. Their judgments are aggregated by an arbiter𝒜\\mathcal\{A\}:

Rnogt=\\displaystyle R\_\{\\mathrm\{nogt\}\}=λn​𝒜​\(E1​\(⋅\),E2​\(⋅\),E3​\(⋅\)\)\\displaystyle\\lambda\_\{n\}\\mathcal\{A\}\\left\(E\_\{1\}\(\\cdot\),E\_\{2\}\(\\cdot\),E\_\{3\}\(\\cdot\)\\right\)\(14\)\+\(1−λn\)​Rfmt\\displaystyle\+\(1\-\\lambda\_\{n\}\)R\_\{\\mathrm\{fmt\}\}
The aggregated no\-ground\-truth reward is also used as a pseudo\-supervision signal for memory consolidation\.

#### Skill\-Alignment Regularization\.

Relying only on no\-ground\-truth rewards may cause unstable online updates\. Therefore,APExuses the retrieved skill as a knowledge anchor\. For a category–modality bucket\(c,d\)\(c,d\), the skillsc,ds\_\{c,d\}contains procedurePc,dP\_\{c,d\}and pitfallsFc,dF\_\{c,d\}\. The alignment reward evaluates whether the generated plan follows the recommended procedure and avoids known pitfalls:

Ralign=𝒥align​\(p,Pc,d,Fc,d\)\.R\_\{\\mathrm\{align\}\}=\\mathcal\{J\}\_\{\\mathrm\{align\}\}\\left\(p,P\_\{c,d\},F\_\{c,d\}\\right\)\.\(15\)The final reward combines the original no\-ground\-truth reward and the skill\-alignment reward:

Rfinal=\(1−λs\)​Rnogt\+λs​Ralign,R\_\{\\mathrm\{final\}\}=\(1\-\\lambda\_\{s\}\)R\_\{\\mathrm\{nogt\}\}\+\\lambda\_\{s\}R\_\{\\mathrm\{align\}\},\(16\)The weightλs=λbase⋅wc,d⋅min⁡\(nc,dNthreshold,1\)\\lambda\_\{s\}=\\lambda\_\{\\mathrm\{base\}\}\\cdot w\_\{c,d\}\\cdot\\min\\left\(\\frac\{n\_\{c,d\}\}\{N\_\{\\mathrm\{threshold\}\}\},1\\right\)is adaptively determined by the confidence of the retrieved skill\.

## 4Experiment

Table 1:Main results across in\-domain and out\-of\-domain benchmarks\. The best results are highlighted inbold, and the second\-best results areunderlined\.In\-DomainOut\-of\-DomainMethodFVQA\-testHotpotQA2WikiSimpleVQALiveVQAInfoSeekMMSearchAvg\.Naive\-Large Language ModelGPT\-5\.451\.360\.264\.955\.922\.845\.647\.449\.7Gemini\-3\-Flash68\.466\.268\.472\.325\.766\.567\.962\.2Qwen2\.5\-VL\-7B20\.712\.620\.530\.28\.323\.67\.217\.6Qwen2\.5\-VL\-32B24\.316\.924\.039\.818\.525\.815\.723\.6Search AgentQwen2\.5\-VL\-7B\+ReAct34\.223\.731\.235\.410\.628\.121\.326\.4Qwen2\.5\-VL\-32B\+ReAct50\.931\.437\.048\.324\.738\.127\.536\.8MMSearch\-R158\.339\.248\.355\.228\.649\.543\.346\.1Deepeyes260\.6\-\-59\.4\-51\.163\.7\-Memory\-Augmented Search AgentRAG58\.647\.856\.260\.731\.955\.954\.352\.2Mem054\.949\.354\.956\.924\.348\.243\.147\.4A\-Mem38\.546\.856\.251\.622\.836\.141\.141\.9ExpeL64\.256\.362\.962\.534\.158\.861\.357\.2Memento66\.155\.264\.361\.736\.757\.361\.457\.5MIA65\.364\.071\.663\.839\.764\.760\.661\.4OursAPEx68\.767\.875\.266\.342\.466\.963\.864\.4

### 4\.1Experimental Setups

Training Settings:Our training framework is built on veRL[Sheng et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib37)and follows a three\-stage alternating optimization paradigm\. The Executor employs Qwen2\.5\-VL\-7B[Qwen Team \(2025\)](https://arxiv.org/html/2609.02253#bib.bib38)as its backbone and is trained on FVQA\-train[Wang et al\. \(2017\)](https://arxiv.org/html/2609.02253#bib.bib39)\. The Distiller, based on Qwen3\-8B[Yang et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib40), is subsequently trained on FVQA\-train to distill accumulated execution memories into structured skill documents\. The Planner, also based on Qwen3\-8B, is trained in the final stage on a mixture of FVQA\-train \(with images discarded\) and MATPO[Mo et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib41)\. The Executor is equipped with offline text\-to\-text and offline image\-to\-image search tools\. A Qwen3\-32B model serves as the LLM Judger for reward computation\. More details are provided in Appendix[C](https://arxiv.org/html/2609.02253#A3)\.

Test Settings:To evaluateAPEx, we conduct experiments on a diverse set of image\-text and text\-only benchmarks\. For image\-text evaluation, we use FVQA\-test, SimpleVQA[Cheng et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib44), LiveVQA[Fu et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib45), InfoSeek[Chen et al\. \(2023\)](https://arxiv.org/html/2609.02253#bib.bib46), and MMSearch[Jiang et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib47)\. For text\-only evaluation, we use HotpotQA[Yang et al\. \(2018\)](https://arxiv.org/html/2609.02253#bib.bib48)and 2WikiMultiHopQA[Ho et al\. \(2020\)](https://arxiv.org/html/2609.02253#bib.bib49)\. Detailed settings are provided in Appendix[D](https://arxiv.org/html/2609.02253#A4)\.

Baselines\.We compareAPExagainst a wide range of deep research methods spanning three categories: ❶Naive\-Large Language Models: GPT\-5\.4[OpenAI \(2026\)](https://arxiv.org/html/2609.02253#bib.bib50), Gemini\-3\-Flash[Google DeepMind \(2025\)](https://arxiv.org/html/2609.02253#bib.bib51), Qwen2\.5\-VL\-7B[Qwen Team \(2025\)](https://arxiv.org/html/2609.02253#bib.bib38), and Qwen2\.5\-VL\-32B[Qwen Team \(2025\)](https://arxiv.org/html/2609.02253#bib.bib38), which are directly prompted to answer questions without agentic workflow; ❷Search Agents: Qwen2\.5\-VL\-7B\+ReAct[Yao et al\. \(2022\)](https://arxiv.org/html/2609.02253#bib.bib52), Qwen2\.5\-VL\-32B\+ReAct, MMSearch\-R1[Wu et al\. \(2025b\)](https://arxiv.org/html/2609.02253#bib.bib8), and Deepeyes2[Zheng et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib53), which augment LLMs with external search tools via multi\-step reasoning but maintain no cross\-task memory; and ❸Memory\-Augmented Search Agents: RAG, Mem0[Chhikara et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib54), A\-Mem[Xu et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib12), ExpeL[Zhao et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib15), Memento[Zhou et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib33), and MIA[Qiao et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib13), which additionally incorporate memory mechanisms to store and reuse past experience across tasks\. More details on baseline settings are provided in Appendix[F](https://arxiv.org/html/2609.02253#A6)\.

### 4\.2Main Results

As shown in Table[1](https://arxiv.org/html/2609.02253#S4.T1),APExachieves the best average performance across 7 benchmarks, with an average score of 64\.4\. We highlight the following key observations:

❶Substantial improvement over standalone LLMs\.APEximproves by 14\.7 points compared to GPT\-5\.4 and slightly surpasses Gemini\-3\-Flash in average accuracy\. Notably,APExoutperforms its own backbone model Qwen2\.5\-VL\-7B by 46\.8 points in absolute terms, demonstrating that structured skill knowledge and memory\-augmented planning can effectively compensate for model scale, enabling a small open\-source model to rival closed\-source counterparts\.

❷Consistent gains over search agents\.Equipping LLMs with search tools through ReAct substantially improves performance \(e\.g\., Qwen2\.5\-VL\-7B: 17\.6→\\rightarrow26\.4\)\. Compared with the strong search agent MMSearch\-R1,APExachieves an improvement of 18\.3 points in average accuracy\. This gap indicates that tool access alone is insufficient for complex research tasks — effective utilization of accumulated experience through skill distillation is critical for guiding multi\-step tool orchestration\.

❸Superiority over memory\-augmented agents\.Among memory\-augmented baselines, MIA achieves the strongest performance \(61\.4 avg\) by combining a Manager\-Planner\-Executor architecture with instance\-level memory retrieval\.APExfurther improves upon MIA by 3\.0 points on average\. This improvement is largely associated with the Distiller module, which distills instance\-level memories into category\-level skills that provide structured procedural guidance to the Planner, rather than relying solely on raw memory retrieval at each decision point\.

### 4\.3Skill Sufficiency Analysis

We evaluateAPEx\(Skill Only\), which removes instance\-level memory retrieval from the Planner context while retaining trajectories for skill evolution\. As shown in Table[2](https://arxiv.org/html/2609.02253#S4.T2), this variant incurs an average drop of 1\.8 points compared to the full pipeline, yet outperforms the strongest baseline MIA by 1\.5 points\. This confirms that the Distiller successfully distills category\-level knowledge from raw trajectories into structured skills that are dynamically modified at test time, reducing the Planner’s reliance on verbose instance\-level memories\.

### 4\.4Context Efficiency Analysis

Figure[3](https://arxiv.org/html/2609.02253#S4.F3)compares different memory\-augmented reasoning methods in terms of accuracy, injected memory tokens per inference step, and total token consumption\. Across both FVQA\-test and 2Wiki,APEx\(Skill only\) achieves favorable context\-efficiency trade\-off\. Compared with MIA,APEximproves accuracy by 1\.7 points\. Meanwhile, it reduces injected memory tokens by 45\.6% and total token consumption by 36\.8% on average\. These results demonstrate thatAPExachieves better accuracy with a smaller memory footprint and lower token cost, highlighting its advantage in context\-efficient memory utilization for reasoning\.

Table 2:Skill sufficiency analysis\.Avg\. is computed over the four datasets shown in this table\. Best inbold, second\-bestunderlined\.Image\-TextText\-OnlyMethodFVQASimpleVQAHotpotQA2WikiAvg\.BaselineMem054\.956\.949\.354\.954\.0MMSearch\-R158\.355\.239\.248\.350\.3ExpeL64\.262\.556\.362\.961\.5Memento66\.161\.755\.264\.361\.8MIA65\.363\.864\.071\.666\.2OursAPEx\(Skill Only\)67\.464\.865\.772\.967\.7APEx68\.766\.367\.875\.269\.5

![Refer to caption](https://arxiv.org/html/2609.02253v1/cost.png)Figure 3:Token\-efficiency comparison on FVQA\-test and 2Wiki\. X\-axis denotes task accuracy\. Y\-axis denotes the number of memory tokens injected at each reasoning step\. The bubble size represents the overall token cost of a full inference process\.
### 4\.5Training Analysis

In Figure[5](https://arxiv.org/html/2609.02253#A6.F5), we analyze the GRPO training dynamics of each module to characterize their convergence behavior and identify the factors underlying their distinct learning patterns\. All modules are trained alternately with only one module updated at a time\. TheExecutorexhibits the most stable improvement, with its reward increasing steadily and converging to a high level\. TheDistilleralso converges smoothly and rapidly, suggesting that the skill distillation objective provides a relatively well\-defined optimization signal\. In contrast, thePlannershows larger fluctuations, although its reward still follows an overall upward trend\. This is expected since planning is evaluated indirectly through downstream execution and is therefore more sensitive to variations in retrieved memories, generated skills, and executor behavior\.

### 4\.6Generalization Analysis

To assess the generalization ability ofAPEx, we replace the originally trained open\-source executor model with stronger closed\-source executor models without any additional training, and compare it with ReAct on HotpotQA and LiveVQA\. As shown in Figure[4](https://arxiv.org/html/2609.02253#S4.F4),APExconsistently outperforms ReAct across all closed\-source executors, achieving an average relative improvement of 11\.0% on HotpotQA and 7\.0% on LiveVQA\. This demonstrates thatAPExis not tied to a specific trained executor, but can generalize to unseen closed\-source executors in a plug\-and\-play manner\.

Figure 4:Performance comparison between ReAct andAPExusing different closed\-source executor models on HotpotQA and LiveVQA\.
### 4\.7Ablation Study

We conduct three groups of ablation studies to evaluate each component inAPEx\.

❶Experience Utilization\.We remove different experience sources during inference\.w/o Mem\.disables the retrieval of concrete execution memories,w/o Skillremoves distilled procedural skills, andw/o Bothremoves both\. These variants reduce the average accuracy by 1\.8, 3\.1, and 10\.1 points, respectively\. This suggests that category\-level skills provide general templates for similar tasks, while instance\-level memories offer concrete problem\-solving examples; their complementarity enables more effective utilization of past experience\.

Table 3:Ablation study ofAPEx\. Accuracy is reported on image\-text and text\-only tasks\. Performance drops\(↓\\downarrow\)relative to the full model are shown in subscripts\.MethodImage\-TextText\-OnlyAvg\.FVQASimpleVQAHotpotQA2WikiAPEx68\.766\.367\.875\.269\.5w/o Experience Utilizationw/o Mem\.67\.4↓\\downarrow1\.364\.8↓\\downarrow1\.565\.7↓\\downarrow2\.172\.9↓\\downarrow2\.367\.7↓\\downarrow1\.8w/o Skill64\.9↓\\downarrow3\.862\.7↓\\downarrow3\.665\.6↓\\downarrow2\.272\.3↓\\downarrow2\.966\.4↓\\downarrow3\.1w/o Both58\.6↓\\downarrow10\.155\.0↓\\downarrow11\.358\.4↓\\downarrow9\.465\.5↓\\downarrow9\.759\.4↓\\downarrow10\.1w/o Post\-training Optimizationw/o Dis\.66\.4↓\\downarrow2\.363\.7↓\\downarrow2\.665\.8↓\\downarrow2\.072\.7↓\\downarrow2\.567\.2↓\\downarrow2\.3w/o Plan\.66\.8↓\\downarrow1\.963\.9↓\\downarrow2\.466\.2↓\\downarrow1\.674\.0↓\\downarrow1\.267\.7↓\\downarrow1\.8w/o Exec\.61\.0↓\\downarrow7\.758\.0↓\\downarrow8\.358\.2↓\\downarrow9\.667\.2↓\\downarrow8\.061\.1↓\\downarrow8\.4w/o Closed\-loop Adaptationw/o TTRL64\.6↓\\downarrow4\.160\.8↓\\downarrow5\.563\.5↓\\downarrow4\.372\.4↓\\downarrow2\.865\.3↓\\downarrow4\.2w/o Iter\.66\.0↓\\downarrow2\.764\.1↓\\downarrow2\.264\.7↓\\downarrow3\.173\.3↓\\downarrow1\.967\.0↓\\downarrow2\.5

❷Post\-training Optimization\.We evaluate module\-wise GRPO post\-training\. For each ablation, the pipeline structure is kept unchanged, while the corresponding GRPO\-trained module is replaced with its non\-trained base model, such as Qwen2\.5\-VL\-7B or Qwen3\-8B\. Removing GRPO from the Distiller, Planner, and Executor reduces the average accuracy by 2\.3, 1\.8, and 8\.4 points respectively\. The larger degradation ofw/o Exec\.indicates that executor optimization is particularly important for reliable multi\-step tool interaction, while optimizing the Distiller and Planner also benefits skill construction and strategy generation\.

❸Closed\-loop Adaptation\.Finally, we analyze the closed\-loop adaptation mechanism\.w/o TTRLperforms inference without test\-time reinforcement learning, whilew/o Iter\.distills skills only once without further updates from improved trajectories\. These variants reduce the average accuracy by 4\.2 and 2\.5 points, respectively\. Together, TTRL and iterative skill distillation form a closed\-loop feedback mechanism at test time, where improved trajectories are distilled into better skills, thereby progressively enhancing model performance\.

## 5Related Work

### 5\.1Deep Research Agents

Deep Research Agents \(DRAs\) are designed to answer complex, multi\-turn information\-seeking questions by combining dynamic reasoning, adaptive tool\-mediated search, and compact experience retrieval[Huang et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib1);[Zhang et al\. \(2025b\)](https://arxiv.org/html/2609.02253#bib.bib2);[Xu and Peng \(2025\)](https://arxiv.org/html/2609.02253#bib.bib28)\. Recent studies have improved text\-only and multimodal long\-horizon search by curating reasoning trajectories[Sun et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib3);[Li et al\. \(2026b\)](https://arxiv.org/html/2609.02253#bib.bib5);[Zhu et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib6), supervised fine\-tuning[Li et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib7);[Wu et al\. \(2025a\)](https://arxiv.org/html/2609.02253#bib.bib4)or agentic reinforcement learning[Zheng et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib10);[Jin et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib9);[Wu et al\. \(2025b\)](https://arxiv.org/html/2609.02253#bib.bib8);[Narayan et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib11)to internalize planning and searching\. To reuse prior interactions, emerging work abstracts past trajectories into reusable memories, enabling agents to dynamically organize memory, maintain compact interaction states, and optimize memory\-aware search or planning policies across similar long\-horizon tasks[Zhao et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib15);[Xu et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib12);[Zhou et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib14);[Qiao et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib13)\. Our work focuses on constructing instance\-level memories, organizing category\-level skills, retrieving and ranking reusable experiences, and distilling them into Distiller\-Planner\-Executor training\.

### 5\.2Memory Systems for Agents

Recent memory systems for LLM agents have evolved from retrieval\-centric external memory to adaptive experience management\. Early RAG methods treat external documents as non\-parametric memory to ground generation with retrieved evidence[Lewis et al\. \(2020\)](https://arxiv.org/html/2609.02253#bib.bib16);[Zhong et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib17);[Edge et al\. \(2024\)](https://arxiv.org/html/2609.02253#bib.bib18)\. Building on these foundations, recent studies shift the focus from simply retrieving past information to organizing, compressing, and updating agent memories[Fang et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib20);[Liu et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib19)\. More recent work further treats memory as a trainable and reusable experience space where agents learn what to store[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib24);[Shen et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib23), when to retrieve or discard[Yuan et al\. \(2025\)](https://arxiv.org/html/2609.02253#bib.bib25), and how to abstract trajectories into procedural guidance or skills[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.02253#bib.bib26);[Zhang et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib21);[Xia et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib22)\. These trends indicate a transition from passive knowledge retrieval toward self\-evolving memory systems that support long\-horizon planning, experience reuse, and strategy generalization\. In contrast,APExdistills trajectories within each category–modality group into reusable procedural skills and uses their confidence\-weighted alignment as a regularization signal for online Planner updates, coupling procedural skill abstraction with downstream parametric adaptation in a closed experience\-to\-action loop\.

## 6Conclusion

In this paper, we presentAPEx, a hierarchical experience utilization framework for deep research agents\.APExorganizes accumulated interactions into instance\-level memories and category\-level skills, enabling reusable procedural knowledge across tasks\. Through GRPO\-based alternating optimization of the Executor, Distiller, and Planner,APExforms a closed\-loop process spanning execution, skill abstraction, and adaptive planning\. At test time, distilled skills serve as procedural priors for skill\-guided TTRL, supporting ground\-truth\-free online adaptation while reducing drift through skill\-alignment regularization\. Experiments on 7 benchmarks show thatAPExachieves state\-of\-the\-art performance\. Future work will extendAPExto broader tool ecosystems and explore more efficient skill evolution and cross\-agent skill transfer\.

## Limitations

WhileAPExachieves the best average performance across seven benchmarks and demonstrates strong generalization to out\-of\-domain settings, our current framework is built upon small open\-source models\. AlthoughAPExenables a 7B model to surpass closed\-source systems, we have not explored howAPExscales when applied to more powerful backbone models\. Additionally, our three\-stage alternating GRPO training paradigm, despite its effectiveness in stabilizing cross\-module credit assignment and producing consistently improving rewards, requires sequential optimization of the Executor, Distiller, and Planner, which may not fully exploit the synergies achievable through end\-to\-end joint training\. Finally, our skill\-guided TTRL mechanism currently operates online at test time without ground\-truth supervision, and while it already yields significant improvements through self\-improvement, incorporating minimal supervision could further amplify the quality of distilled skills and accelerate the closed\-loop adaptation process\.

## Ethical Considerations

Our framework builds upon large language and vision\-language models that may produce inaccurate, biased, or harmful outputs due to biases inherent in their pre\-training corpora; such behaviors are not by design\. SinceAPExretrieves and processes open\-web content at inference time, retrieved results may inadvertently contain personal identifiable information \(PII\) or offensive material, and we advise deployers to apply appropriate content filtering\. The open\-source Qwen models used for training are released under their respective licenses\. Proprietary models used only for baseline evaluation or executor generalization are accessed through their APIs and are subject to their providers’ terms of use\. All evaluation benchmarks are publicly available with documented provenance\. Any code or weights we release will be accompanied by usage guidelines that explicitly prohibit malicious applications such as misinformation generation or unauthorized surveillance\. We report training procedures, hyperparameters, and computational costs to ensure reproducibility and transparency\. Finally, AI assistants were employed for grammatical error correction and rephrasing\.

## References

- Chenet al\.\(2023\)Y\. Chen, H\. Hu, Y\. Luan, H\. Sun, S\. Changpinyo, A\. Ritter, and M\. ChangCan pre\-trained vision and language models answer visual information\-seeking questions?\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 14948–14968\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p2.1)\.
- Chenet al\.\(2025\)Z\. Chen, K\. Liu, Q\. Wang, J\. Liu, W\. Zhang, K\. Chen, and F\. ZhaoMindsearch: mimicking human minds elicits deep ai searcher\.InInternational Conference On Learning Representations,Vol\.2025,pp\. 90007–90029\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1)\.
- Chenget al\.\(2025\)X\. Cheng, W\. Zhang, S\. Zhang, J\. Yang, X\. Guan, X\. Wu, X\. Li, G\. Zhang, J\. Liu, Y\. Mai, Y\. Zeng, Z\. Wen, K\. Jin, B\. Wang, W\. Zhou, Y\. Lu, T\. Li, W\. Huang, and Z\. LiSimpleVQA: multimodal factuality evaluation for multimodal large language models\.External Links:2502\.13059,[Link](https://arxiv.org/abs/2502.13059)Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p2.1)\.
- Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: building production\-ready ai agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1)\.
- Edgeet al\.\(2024\)D\. Edge, H\. Trinh, N\. Cheng, J\. Bradley, A\. Chao, A\. Mody, S\. Truitt, D\. Metropolitansky, R\. O\. Ness, and J\. LarsonFrom local to global: a graph rag approach to query\-focused summarization\.arXiv preprint arXiv:2404\.16130\.Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Fanget al\.\(2026\)J\. Fang, X\. Deng, H\. Xu, Z\. Jiang, Y\. Tang, Z\. Xu, S\. Deng, Y\. Yao, M\. Wang, S\. Qiao, H\. Chen, and N\. ZhangLightMem: lightweight and efficient memory\-augmented generation\.InInternational Conference on Learning Representations,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),Vol\.2026,pp\. 98706–98729\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/file/a05b72653ec5b473732129829ae04195-Paper-Conference.pdf)Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Fuet al\.\(2025\)M\. Fu, Y\. Peng, B\. Liu, Y\. Wan, and D\. ChenLivevqa: live visual knowledge seeking\.arXiv e\-prints,pp\. arXiv–2504\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p2.1)\.
- Google DeepMind \(2025\)Google DeepMindGemini 3 flash model card\.Note:Model card[https://storage\.googleapis\.com/deepmind\-media/Model\-Cards/Gemini\-3\-Flash\-Model\-Card\.pdf](https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-3-Flash-Model-Card.pdf)Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1)\.
- Hoet al\.\(2020\)X\. Ho, A\. D\. Nguyen, S\. Sugawara, and A\. AizawaConstructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.InProceedings of the 28th International Conference on Computational Linguistics,pp\. 6609–6625\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p2.1)\.
- Huanget al\.\(2025\)Y\. Huang, Y\. Chen, H\. Zhang, K\. Li, H\. Zhou, M\. Fang, L\. Yang, X\. Li, L\. Shang, S\. Xu, J\. Hao, K\. Shao, and J\. WangDeep research agents: a systematic examination and roadmap\.External Links:2506\.18096,[Link](https://arxiv.org/abs/2506.18096)Cited by:[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Jianget al\.\(2024\)D\. Jiang, R\. Zhang, Z\. Guo, Y\. Wu, J\. Lei, P\. Qiu, P\. Lu, Z\. Chen, C\. Fu, G\. Song, P\. Gao, Y\. Liu, C\. Li, and H\. LiMMSearch: benchmarking the potential of large models as multi\-modal search engines\.External Links:2409\.12959,[Link](https://arxiv.org/abs/2409.12959)Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p2.1)\.
- Jinet al\.\(2025\)B\. Jin, H\. Zeng, Z\. Yue, J\. Yoon, S\. Arik, D\. Wang, H\. Zamani, and J\. HanSearch\-r1: training llms to reason and leverage search engines with reinforcement learning\.arXiv preprint arXiv:2503\.09516\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Kanget al\.\(2026\)J\. Kang, C\. Li, D\. Chen, B\. Tang, F\. Xiong, and Z\. LiMemReader: from passive to active extraction for long\-term agent memory\.arXiv preprint arXiv:2604\.07877\.Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Karpukhinet al\.\(2020\)V\. Karpukhin, B\. Oguz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. YihDense passage retrieval for open\-domain question answering\.InProceedings of the 2020 conference on empirical methods in natural language processing \(EMNLP\),pp\. 6769–6781\.Cited by:[Appendix C](https://arxiv.org/html/2609.02253#A3.p1.1)\.
- Kwonet al\.\(2023\)W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. StoicaEfficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[Appendix C](https://arxiv.org/html/2609.02253#A3.SS0.SSS0.Px1.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. KielaRetrieval\-augmented generation for knowledge\-intensive nlp tasks\.InProceedings of the 34th International Conference on Neural Information Processing Systems,NIPS ’20,Red Hook, NY, USA\.External Links:ISBN 9781713829546Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Liet al\.\(2025\)K\. Li, Z\. Zhang, H\. Yin, L\. Zhang, L\. Ou, J\. Wu, W\. Yin, B\. Li, Z\. Tao, X\. Wang, W\. Shen, J\. Zhang, D\. Zhang, X\. Wu, Y\. Jiang, M\. Yan, P\. Xie, F\. Huang, and J\. ZhouWebSailor: navigating super\-human reasoning for web agent\.External Links:2507\.02592,[Link](https://arxiv.org/abs/2507.02592)Cited by:[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Liet al\.\(2026a\)X\. Li, J\. Jin, G\. Dong, H\. Qian, Y\. Wu, J\. Wen, Y\. Zhu, and Z\. DouWebthinker: empowering large reasoning models with deep research capability\.Advances in Neural Information Processing Systems38,pp\. 120091–120131\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1)\.
- Liet al\.\(2026b\)Z\. Li, D\. Jiang, X\. Ma, H\. Zhang, P\. Nie, Y\. Zhang, K\. Zou, J\. Xie, Y\. Zhang, and W\. ChenOpenresearcher: a fully open pipeline for long\-horizon deep research trajectory synthesis\.arXiv preprint arXiv:2603\.20278\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Liuet al\.\(2026\)J\. Liu, Y\. Su, P\. Xia, S\. Han, Z\. Zheng, C\. Xie, M\. Ding, and H\. YaoSimplemem: efficient lifelong memory for llm agents\.arXiv preprint arXiv:2601\.02553\.Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Maet al\.\(2026\)Z\. Ma, S\. Yang, Y\. Ji, X\. Wang, Y\. Wang, Y\. Hu, T\. Huang, and X\. ChuSkillclaw: let skills evolve collectively with agentic evolver\.arXiv preprint arXiv:2604\.08377\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1),[§1](https://arxiv.org/html/2609.02253#S1.p3.1)\.
- Moet al\.\(2025\)Z\. Mo, X\. Li, Y\. Chen, and L\. BingMulti\-agent tool\-integrated policy optimization\.arXiv preprint arXiv:2510\.04678\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p1.1)\.
- Narayanet al\.\(2025\)K\. Narayan, Y\. Xu, T\. Cao, K\. Nerella, V\. M\. Patel, N\. Shiee, P\. Grasch, C\. Jia, Y\. Yang, and Z\. GanDeepmmsearch\-r1: empowering multimodal llms in multimodal web search\.arXiv preprint arXiv:2510\.12801\.Cited by:[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- OpenAI \(2026\)OpenAIGPT\-5\.4 Thinking System Card\.Note:[https://openai\.com/index/gpt\-5\-4\-thinking\-system\-card/](https://openai.com/index/gpt-5-4-thinking-system-card/)Published March 5, 2026Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1)\.
- Qiaoet al\.\(2026\)J\. Qiao, W\. Meng, Y\. Cheng, Z\. Lin, Z\. Zhang, X\. Tan, J\. Gong, K\. Shao, and Y\. XieMemory intelligence agent\.arXiv preprint arXiv:2604\.04503\.Cited by:[Appendix C](https://arxiv.org/html/2609.02253#A3.p1.1),[§1](https://arxiv.org/html/2609.02253#S1.p2.1),[§1](https://arxiv.org/html/2609.02253#S1.p3.1),[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Qwen Team \(2025\)Qwen TeamQwen2\.5\-vl\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5-vl/)Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§2](https://arxiv.org/html/2609.02253#S2.SS0.SSS0.Px2.p1.1)\.
- Shenet al\.\(2026\)Z\. Shen, Z\. Wu, F\. Lai, S\. Lian, and Y\. RaoMembuilder: reinforcing llms for long\-term memory construction via attributed dense rewards\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 27868–27887\.Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Shenget al\.\(2025\)G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. WuHybridflow: a flexible and efficient rlhf framework\.InProceedings of the Twentieth European Conference on Computer Systems,pp\. 1279–1297\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.Advances in neural information processing systems36,pp\. 8634–8652\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1)\.
- Sunet al\.\(2026\)R\. Sun, J\. Ding, C\. Gong, T\. Gu, Y\. Jiang, J\. Zhang, L\. Pan, and L\. LüTopodim: one\-shot topology generation of diverse interaction modes for multi\-agent systems\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 4252–4269\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1)\.
- Sunet al\.\(2025\)S\. Sun, H\. Song, Y\. Wang, R\. Ren, J\. Jiang, J\. Zhang, F\. Bai, J\. Deng, W\. X\. Zhao, Z\. Liu, L\. Fang, Z\. Wang, and J\. WenSimpleDeepSearcher: deep information seeking via web\-powered reasoning trajectory synthesis\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 13705–13720\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.739/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.739),ISBN 979\-8\-89176\-335\-7Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Tanet al\.\(2025\)Z\. Tan, J\. Yan, I\. Hsu, R\. Han, Z\. Wang, L\. Le, Y\. Song, Y\. Chen, H\. Palangi, G\. Lee, A\. R\. Iyer, T\. Chen, H\. Liu, C\. Lee, and T\. PfisterIn prospect and retrospect: reflective memory management for long\-term personalized dialogue agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 8416–8439\.External Links:[Link](https://aclanthology.org/2025.acl-long.413/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.413),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1)\.
- Wanget al\.\(2022\)L\. Wang, N\. Yang, X\. Huang, B\. Jiao, L\. Yang, D\. Jiang, R\. Majumder, and F\. WeiText embeddings by weakly\-supervised contrastive pre\-training\.arXiv preprint arXiv:2212\.03533\.Cited by:[Appendix C](https://arxiv.org/html/2609.02253#A3.p1.1)\.
- Wanget al\.\(2017\)P\. Wang, Q\. Wu, C\. Shen, A\. Dick, and A\. Van Den HengelFvqa: fact\-based visual question answering\.IEEE transactions on pattern analysis and machine intelligence40\(10\),pp\. 2413–2427\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p1.1)\.
- Wang and Chen \(2025\)Y\. Wang and X\. ChenMirix: multi\-agent memory system for llm\-based agents\.arXiv preprint arXiv:2507\.07957\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1)\.
- Wanget al\.\(2024\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1)\.
- Wuet al\.\(2025a\)J\. Wu, B\. Li, R\. Fang, W\. Yin, L\. Zhang, Z\. Wang, Z\. Tao, D\. Zhang, Z\. Xi, R\. Tang, Y\. Jiang, P\. Xie, F\. Huang, and J\. ZhouWebDancer: towards autonomous information seeking agency\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38, Main Conference,pp\. 120957–120985\.External Links:[Document](https://dx.doi.org/10.52202/085713-4035),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/af043aee9cb137785d93195c6cf4cd96-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Wuet al\.\(2025b\)J\. Wu, Z\. Deng, W\. Li, Y\. Liu, B\. You, B\. Li, Z\. Ma, and Z\. LiuMmsearch\-r1: incentivizing lmms to search\.arXiv preprint arXiv:2506\.206702\(4\)\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1),[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Xiaet al\.\(2026\)P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen, Z\. Zheng, C\. Xie, and H\. YaoSkillRL: evolving agents via recursive skill\-augmented reinforcement learning\.External Links:2602\.08234,[Link](https://arxiv.org/abs/2602.08234)Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1),[§1](https://arxiv.org/html/2609.02253#S1.p3.1),[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Xu and Peng \(2025\)R\. Xu and J\. PengA comprehensive survey of deep research: systems, methodologies, and applications\.arXiv preprint arXiv:2506\.12594\.Cited by:[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Xuet al\.\(2026\)W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. ZhangA\-mem: agentic memory for llm agents\.Advances in Neural Information Processing Systems38,pp\. 17577–17604\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p1.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 conference on empirical methods in natural language processing,pp\. 2369–2380\.Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p2.1)\.
- Yaoet al\.\(2022\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReact: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[Appendix F](https://arxiv.org/html/2609.02253#A6.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1)\.
- Yuanet al\.\(2025\)Q\. Yuan, J\. Lou, Z\. Li, J\. Chen, Y\. Lu, H\. Lin, L\. Sun, D\. Zhang, and X\. HanMemsearcher: training llms to reason, search and manage memory via end\-to\-end reinforcement learning\.arXiv preprint arXiv:2511\.02805\.Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Zhanget al\.\(2025a\)G\. Zhang, H\. Ren, C\. Zhan, Z\. Zhou, J\. Wang, H\. Zhu, W\. Zhou, and S\. YanMemevolve: meta\-evolution of agent memory systems\.arXiv preprint arXiv:2512\.18746\.Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Zhanget al\.\(2026\)H\. Zhang, Q\. Long, J\. Bao, T\. Feng, W\. Zhang, H\. Yue, and W\. WangMemskill: learning and evolving memory skills for self\-evolving agents\.arXiv preprint arXiv:2602\.02474\.Cited by:[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Zhanget al\.\(2025b\)W\. Zhang, X\. Li, Y\. Zhang, P\. Jia, Y\. Wang, H\. Guo, Y\. Liu, and X\. ZhaoDeep research: a survey of autonomous research agents\.arXiv preprint arXiv:2508\.12752\.Cited by:[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpel: llm agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1),[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing, H\. Zhang, J\. Gonzalez, and I\. StoicaJudging llm\-as\-a\-judge with mt\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 46595–46623\.External Links:[Document](https://dx.doi.org/10.52202/075280-2020),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/91f18a1287b398d378ef22505bf41832-Paper-Datasets_and_Benchmarks.pdf)Cited by:[§2](https://arxiv.org/html/2609.02253#S2.SS0.SSS0.Px3.p1.1)\.
- Zhenget al\.\(2025\)Y\. Zheng, D\. Fu, X\. Hu, X\. Cai, L\. Ye, P\. Lu, and P\. LiuDeepresearcher: scaling deep research via reinforcement learning in real\-world environments\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 414–431\.Cited by:[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Zhenget al\.\(2026\)Z\. Zheng, M\. Yang, J\. Hong, C\. Zhao, G\. Xu, L\. Yang, C\. Shen, and XingYuDeepEyes: incentivizing "thinking with images" via reinforcement learning\.InInternational Conference on Learning Representations,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),Vol\.2026,pp\. 126775–126798\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/file/cdb347d7516d52a9280ea9d6708911f6-Paper-Conference.pdf)Cited by:[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1)\.
- Zhonget al\.\(2024\)W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. WangMemorybank: enhancing large language models with long\-term memory\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 19724–19731\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1),[§5\.2](https://arxiv.org/html/2609.02253#S5.SS2.p1.1)\.
- Zhouet al\.\(2025\)H\. Zhou, Y\. Chen, S\. Guo, X\. Yan, K\. H\. Lee, Z\. Wang, K\. Y\. Lee, G\. Zhang, K\. Shao, L\. Yang, and J\. WangMemento: fine\-tuning llm agents without fine\-tuning llms\.External Links:2508\.16153,[Link](https://arxiv.org/abs/2508.16153)Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1),[§1](https://arxiv.org/html/2609.02253#S1.p3.1),[§4\.1](https://arxiv.org/html/2609.02253#S4.SS1.p3.1)\.
- Zhouet al\.\(2026\)H\. Zhou, S\. Guo, A\. Liu, Z\. Yu, Z\. Gong, B\. Zhao, Z\. Chen, M\. Zhang, Y\. Chen, J\. Li, R\. Yang, Q\. Liu, X\. Yu, J\. Zhou, N\. Wang, C\. Sun, and J\. WangMemento\-skills: let agents design agents\.External Links:2603\.18743,[Link](https://arxiv.org/abs/2603.18743)Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p2.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Zhuet al\.\(2026\)B\. Zhu, Q\. Jia, T\. Lan, J\. Ren, F\. Gu, F\. Jiang, L\. Wang, Z\. Xu, and W\. LuoMarco deepresearch: unlocking efficient deep research agents via verification\-centric design\.arXiv preprint arXiv:2603\.28376\.Cited by:[§1](https://arxiv.org/html/2609.02253#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.02253#S5.SS1.p1.1)\.
- Zuoet al\.\(2025\)Y\. Zuo, K\. Zhang, L\. Sheng, S\. Qu, G\. Cui, X\. Zhu, H\. Li, y\. zhang, X\. Long, E\. Hua, B\. Qi, Y\. Sun, Z\. Ma, L\. Yuan, N\. Ding, and B\. ZhouTTRL: test\-time reinforcement learning\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38, Main Conference,pp\. 131459–131483\.External Links:[Document](https://dx.doi.org/10.52202/085713-4376),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/be690ea16f005c174f6c4102a5970e67-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.02253#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix AMemory and Skill Retrieval

For each candidate memorymi∈ℳ⁡\(q\)m\_\{i\}\\in\\mathcal\{M\}\(q\),APExcomputes a dual\-channel similarity score\. The first channel compares the current query with the stored query, while the second compares caption\-level or contextual representations:

sim⁡\(q,mi\)\\displaystyle\\operatorname\{sim\}\(q,m\_\{i\}\)=βcos\(ϕq\(q\),ϕq\(qi\)\)\\displaystyle=\\beta\\cos\\big\(\\phi\_\{q\}\(q\),\\phi\_\{q\}\(q\_\{i\}\)\\big\)\(17\)\+\(1−β\)cos\(ϕc\(q\),ϕc\(mi\)\),\\displaystyle\+\(1\-\\beta\)\\cos\\big\(\\phi\_\{c\}\(q\),\\phi\_\{c\}\(m\_\{i\}\)\\big\),whereϕq​\(⋅\)\\phi\_\{q\}\(\\cdot\)denotes the question embedding function,ϕc​\(⋅\)\\phi\_\{c\}\(\\cdot\)denotes the caption or contextual embedding function\. When no caption is available, the similarity degenerates to the question\-only similarity\. The similarity scoresim⁡\(q,mi\)\\operatorname\{sim\}\(q,m\_\{i\}\)is min–max normalized within the candidate bucket and denoted assim~​\(q,mi\)\\widetilde\{\\operatorname\{sim\}\}\(q,m\_\{i\}\)\. The final retrieval score combines normalized semantic similarity with the historical reuse win rate:

f⁡\(q,mi\)=α​sim~​\(q,mi\)\+\(1−α\)​wi,f\(q,m\_\{i\}\)=\\alpha\\,\\widetilde\{\\operatorname\{sim\}\}\(q,m\_\{i\}\)\+\(1\-\\alpha\)\\,w\_\{i\},\(18\)
APExretrieves top\-kksuccessful and failed memories separately according to the retrieval score:

ℛM\(q\)=\{TopKmi∈ℳ\+​\(q\)⁡f⁡\(q,mi\),TopKmi∈ℳ−​\(q\)⁡f⁡\(q,mi\)\.\\mathcal\{R\}\_\{M\}\(q\)=\\left\\\{\\begin\{aligned\} &\\operatorname\{TopK\}\_\{m\_\{i\}\\in\\mathcal\{M\}^\{\+\}\(q\)\}f\(q,m\_\{i\}\),\\\\ &\\operatorname\{TopK\}\_\{m\_\{i\}\\in\\mathcal\{M\}^\{\-\}\(q\)\}f\(q,m\_\{i\}\)\.\\end\{aligned\}\\right\.\(19\)
In terms of category\-level memory, each skillsc,ds\_\{c,d\}is assigned a confidence score that combines win rate with evidence sufficiency:

γc,d=wc,d⋅min⁡\(nc,dNthreshold,1\),\\gamma\_\{c,d\}=w\_\{c,d\}\\cdot\\min\\\!\\left\(\\frac\{n\_\{c,d\}\}\{N\_\{\\mathrm\{threshold\}\}\},\\,1\\right\),\(20\)whereNthresholdN\_\{\\mathrm\{threshold\}\}is the evidence threshold\. A skill is injected into the Planner only when its confidence exceeds a minimum thresholdℛS​\(q\)=Inject⁡\(sc^,d^\)\\mathcal\{R\}\_\{S\}\(q\)=\\operatorname\{Inject\}\(s\_\{\\hat\{c\},\\hat\{d\}\}\), where

Inject⁡\(sc,d\)=\{sc,d,γc,d\>ϵs,∅,γc,d≤ϵs,\\operatorname\{Inject\}\(s\_\{c,d\}\)=\\begin\{cases\}s\_\{c,d\},&\\gamma\_\{c,d\}\>\\epsilon\_\{s\},\\\\ \\varnothing,&\\gamma\_\{c,d\}\\leq\\epsilon\_\{s\},\\end\{cases\}\(21\)withϵs=0\.1\\epsilon\_\{s\}=0\.1andNthreshold=10N\_\{\\mathrm\{threshold\}\}=10\.

Finally,APExcombines category\-level skills and instance\-level memories into a structured planner context:

E⁡\(q\)=ℛS​\(q\)⊕ℛM​\(q\),\\displaystyle E\(q\)=\\mathcal\{R\}\_\{S\}\(q\)\\oplus\\mathcal\{R\}\_\{M\}\(q\),\(22\)where⊕\\oplusdenotes structured prompt concatenation rather than set union\. The Planner then generates a research plan conditioned on it:

p=πψplan​\(q,E⁡\(q\)\),p=\\pi\_\{\\psi\}^\{\\mathrm\{plan\}\}\\left\(q,E\(q\)\\right\),\(23\)which enables the Planner to combine case\-based guidance with abstract strategies for new research tasks\.

## Appendix BSupplement of TTRL

#### Skill\-Gated Update\.

For highly reliable skills,APExoptionally skips test\-time updates to protect already learned knowledge\. A skill is considered reliable if it has high confidence, sufficient historical support, and an available procedural prior\. When the gate is activated, all rollouts receive the same reward, equal to the skill confidence\. Under GRPO, this produces zero group\-normalized advantages\(A^i=0\)\(\\hat\{A\}\_\{i\}=0\), so no effective parameter update is performed\. This avoids unnecessary computation and prevents reliable skills from being degraded by noisy test\-time rewards\.

#### Closed\-loop Adaptation\.

Skill\-guided TTRL creates a positive adaptation loop in which skills and the Planner improve each other\. Skills guide the Planner to produce better plans, which lead the Executor to generate higher\-quality trajectories\. These trajectories update the Planner through GRPO and are further consolidated into memories to distill stronger skills for future queries\.

After each batch, selected successful and failed trajectories are consolidated into the memory bank:

ℳt\+1=ℳt∪Consolidate⁡\(q,τ,y^\)\.\\mathcal\{M\}^\{t\+1\}=\\mathcal\{M\}^\{t\}\\cup\\operatorname\{Consolidate\}\(q,\\tau,\\hat\{y\}\)\.\(24\)The Distiller then uses recent memories to synthesize or selectively update the corresponding skill:

sc,dt\+1=ϕskill​\(ℳc,dt\+1,sc,dt\)\.s\_\{c,d\}^\{t\+1\}=\\phi\_\{\\mathrm\{skill\}\}\\left\(\\mathcal\{M\}\_\{c,d\}^\{t\+1\},s\_\{c,d\}^\{t\}\\right\)\.\(25\)
During online TTRL, only the Planner parameters are updated\. Memory consolidation and skill evolution occur after each batch to inform subsequent queries\.

## Appendix CTraining Setups\.

This appendix provides comprehensive training configurations for the three\-stage alternating GRPO optimization \(Section 3\.2\) and the Skill\-guided TTRL \(Section 3\.3\)\. All modules are trained using the veRL framework with GRPO\. We use offline search tool during training\. For text\-to\-text retrieval, we use an offline retrieval service based on E5\-base\-v2[Wang et al\. \(2022\)](https://arxiv.org/html/2609.02253#bib.bib42)over a FAISS\-indexed wiki25[Karpukhin et al\. \(2020\)](https://arxiv.org/html/2609.02253#bib.bib43)corpus\. For image\-to\-image retrieval, we use a pre\-built local cache of image search results collected via Serper by prior work[Qiao et al\. \(2026\)](https://arxiv.org/html/2609.02253#bib.bib13)\.

#### General Training Infrastructure

All experiments are conducted on a single node equipped with 8 NVIDIA A100 GPUs\. We employ Fully Sharded Data Parallel \(FSDP\) with both parameter and optimizer offloading to enable training of 7B/8B models within GPU memory constraints\. The rollout engine uses asynchronous generation with chunked prefill enabled\. A Qwen3\-32B model served via vLLM[Kwon et al\. \(2023\)](https://arxiv.org/html/2609.02253#bib.bib57)with tensor parallelism of 2 acts as the LLM Judge𝒥\\mathcal\{J\}for reward computation across all stages\.

#### Stage 1: Executor Training

The Executorπθexec\\pi^\{\\mathrm\{exec\}\}\_\{\\theta\}is initialized from Qwen2\.5\-VL\-7B and trained on FVQA\-train with multi\-turn tool interactions\. Table[4](https://arxiv.org/html/2609.02253#A3.T4)summarizes the hyperparameters\.

Table 4:Executor GRPO training hyperparameters\.HyperparameterValueBase modelQwen2\.5\-VL\-7BTraining dataFVQA\-trainLearning rate1×10−61\\times 10^\{\-6\}Batch size128Group sizeGG\(rollouts per query\)8Max prompt length16,384 tokensMax response length16,384 tokensMax tool response length4,096 tokensMax assistant turns10Sampling temperature1\.0KL coefficientβ\\beta0\.0Entropy coefficient0\.0PPO clip rangeϵ\\epsilon0\.2Training epochs8Number of GPUs8OptimizerAdamWFor the reward of executor, we setλ1=0\.7\\lambda\_\{1\}=0\.7,λ2=0\.2\\lambda\_\{2\}=0\.2, andλ3=0\.1\\lambda\_\{3\}=0\.1, respectively\.

#### Stage 2: Distiller Training

As shown in Table[5](https://arxiv.org/html/2609.02253#A3.T5), the Distillerπϕdist\\pi^\{\\mathrm\{dist\}\}\_\{\\phi\}is initialized from Qwen3\-8B and trained to transform instance\-level memories into structured category\-level skill documents\. Unlike the Executor and Planner, the Distiller does not perform multi\-turn tool interaction; it generates skill documents in a single turn\. For Distiller reward, we setμ1=0\.5\\mu\_\{1\}=0\.5,μ2=0\.1\\mu\_\{2\}=0\.1,μ3=0\.4\\mu\_\{3\}=0\.4\.

Table 5:Distiller GRPO training hyperparameters\.HyperparameterValueBase modelQwen3\-8BTraining dataFVQA\-trainLearning rate1×10−61\\times 10^\{\-6\}Batch size64Group sizeGG\(rollouts per query\)8Max prompt length4,096 tokensMax response length2,048 tokensMulti\-turn interactionDisabledSampling temperature1\.0KL coefficientβ\\beta0\.0Entropy coefficient0\.0PPO clip rangeϵ\\epsilon0\.2Training epochs8Number of GPUs8OptimizerAdamWThe quality rewardRqualityR\_\{\\mathrm\{quality\}\}is evaluated by the LLM Judge𝒥skill\\mathcal\{J\}\_\{\\mathrm\{skill\}\}, assessing whether the generated skill contains well\-structured procedural steps and meaningful failure modes\. The evolution rewardRevolveR\_\{\\mathrm\{evolve\}\}checks whether the selected operation \(synthesize, refine, create, or skip\) is rational given the current memory state\.

#### Stage 3: Planner Training

The Plannerπψplan\\pi^\{\\mathrm\{plan\}\}\_\{\\psi\}is initialized from Qwen3\-8B and trained on a mixture of FVQA\-train \(with images discarded\) and MATPO data through a plan–execute–evaluate–replan loop, as shown in Table[6](https://arxiv.org/html/2609.02253#A3.T6)\. For the Planner reward, we setα1=0\.7\\alpha\_\{1\}=0\.7,α2=0\.2\\alpha\_\{2\}=0\.2,α3=0\.05\\alpha\_\{3\}=0\.05andα4=0\.05\\alpha\_\{4\}=0\.05\. When the Planner decides not to replan \(d=0d=0\), we sety^2=y^1\\hat\{y\}\_\{2\}=\\hat\{y\}\_\{1\}\.

Table 6:Planner GRPO training hyperparameters\.HyperparameterValueBase modelQwen3\-8BTraining dataFVQA\-train, MATPOLearning rate1×10−61\\times 10^\{\-6\}Batch size48Group sizeGG\(rollouts per query\)4Max prompt length24,576 tokensMax response length8,192 tokensMax tool response length8,192 tokensMax assistant turns10Sampling temperature1\.0KL coefficientβ\\beta0\.0Entropy coefficient0\.0PPO clip rangeϵ\\epsilon0\.2Training epochs5Number of GPUs8OptimizerAdamW

## Appendix DTest Setups\.

At test time, the Planner is further adapted via Skill\-guided TTRL on unlabeled test queries\. Table[7](https://arxiv.org/html/2609.02253#A4.T7)lists the TTRL configuration\. For text\-to\-text search, we use wiki25 for HotpotQA, 2WikiMultiHopQA, and SimpleVQA, while using Serper for other benchmarks\. For image\-to\-image search, we use Serper for all multimodal benchmarks\.

Table 7:Skill\-guided TTRL hyperparameters\.HyperparameterValueAdapted modulePlanner \(πψplan\\pi^\{\\mathrm\{plan\}\}\_\{\\psi\}\)Learning rate2×10−72\\times 10^\{\-7\}Batch size8Group sizeGG\(rollouts per query\)4Max prompt length24,576 tokensMax response length8,192 tokensMax tool response length8,192 tokensMax assistant turns10Max LLM calls per run20Sampling temperature1\.0KL coefficientβ\\beta0\.0Entropy coefficient0\.0PPO clip rangeϵ\\epsilon0\.2Number of GPUs8Epochs \(over test set\)1OptimizerAdamW#### TTRL Reward Formulation\.

The final reward for Skill\-guided TTRL \(Eq\.[16](https://arxiv.org/html/2609.02253#S3.E16)\) combines the no\-ground\-truth reward and skill\-alignment reward:

Rfinal=\(1−λs\)⋅Rnogt\+λs⋅Ralign,R\_\{\\mathrm\{final\}\}=\(1\-\\lambda\_\{s\}\)\\cdot R\_\{\\mathrm\{nogt\}\}\+\\lambda\_\{s\}\\cdot R\_\{\\mathrm\{align\}\},\(26\)whereRnogt=0\.9⋅𝒜\+0\.1⋅RfmtR\_\{\\mathrm\{nogt\}\}=0\.9\\cdot\\mathcal\{A\}\+0\.1\\cdot R\_\{\\mathrm\{fmt\}\}\. The adaptive weightλs\\lambda\_\{s\}is computed as:

λs=λbase⋅wc,d⋅min⁡\(nc,dNthreshold,1\),\\lambda\_\{s\}=\\lambda\_\{\\mathrm\{base\}\}\\cdot w\_\{c,d\}\\cdot\\min\\\!\\left\(\\frac\{n\_\{c,d\}\}\{N\_\{\\mathrm\{threshold\}\}\},\\,1\\right\),\(27\)withλbase=0\.1\\lambda\_\{\\mathrm\{base\}\}=0\.1andNthreshold=10N\_\{\\mathrm\{threshold\}\}=10\. This ensures that skills with higher win ratewc,dw\_\{c,d\}and more supporting evidencenc,dn\_\{c,d\}exert stronger regularization, while low\-confidence or newly created skills have minimal influence on the policy update\.

## Appendix EDataset Description

We conduct comprehensive evaluation across seven benchmark datasets spanning both multimodal and text\-only scenarios\. Table[8](https://arxiv.org/html/2609.02253#A5.T8)provides an overview of all evaluation datasets\.

Table 8:Summary of evaluation datasets used in our experiments\.DatasetModality\# ExamplesSourceFVQA\-testImage\-text1,800MMSearch\-R1InfoSeekImage\-text2,000MMSearch\-R1SimpleVQAImage\-text1,013MMSearch\-R1LiveVQAImage\-text2,384Public VersionMMSearchImage\-text171MMSearch\-R1HotpotQAText\-only7,405Public Version2WikiText\-only12,576Public Version

## Appendix FBaselines Settings

![Refer to caption](https://arxiv.org/html/2609.02253v1/train.png)Figure 5:The left column reports the mean reward of each module across training batches, while the right column shows the corresponding mean response length\.We compare against a broad spectrum of baselines organized into three categories: direct answering, search\-augmented agents, and memory\-based search agents\.

#### Direct Answer\.

In this setting, models are prompted to produce concise answers relying solely on their internal parametric knowledge, without accessing any external tools or retrieval mechanisms\.

#### Search Agent\.

These baselines perform multi\-turn tool calling under the ReAct paradigm\([Yao et al\., 2022](https://arxiv.org/html/2609.02253#bib.bib52)\), iteratively querying external search tools and reasoning over retrieved results\. The results of Deepeyes2 are cited from their respective technical reports\.

#### Memory\-based Search Agent\.

To ensure a fair comparison among memory\-augmented approaches, we train Executor variants sharing identical training configurations but differing in how memory is incorporated\. The Executor is trained under either workflow memory prompt or plan prompt\.

## Appendix GTraining Analysis

As discussed in Section[4](https://arxiv.org/html/2609.02253#S4), we further visualize the training process in Figure[5](https://arxiv.org/html/2609.02253#A6.F5)\. In each subplot, the light\-colored curve denotes the raw training records, while the dark\-colored curve denotes the batch\-averaged smoothed trajectory\.

## Appendix HPrompt Details

This appendix presents the complete prompts\. Each module operates with a dedicated prompt that governs its behavior within the multi\-agent pipeline:

#### Executor Prompt\.

The Executor receives the following prompt to guide its multi\-turn tool\-calling behavior\. Below is the text\-only variant; the image variant appends<image\>after the question\.

Executor Prompt \(Text\-Only\)[⬇](data:text/plain;base64,WW91IG11c3QgZm9sbG93IHRoZXNlIHN0ZXBzIGluIG9yZGVyLiBJbiBldmVyeSBjb252ZXJzYXRpb24gdHVybiwgeW91IHN0YXJ0IGZyb20gU3RlcCAxLgoKKipTdGVwIDE6IFRoaW5rKioKKiAqKlRoaXMgaXMgdGhlIHN0YXJ0aW5nIHBvaW50IGZvciBldmVyeSB0dXJuLioqCiogQW5hbHl6ZSB0aGUgdXNlcidzIHF1ZXJ5IGFuZCBhbGwgYXZhaWxhYmxlIGluZm9ybWF0aW9uIChpbmNsdWRpbmcgcHJldmlvdXMgb2JzZXJ2YXRpb25zKSBjYXJlZnVsbHkuCiogKipFdmFsdWF0ZSB0aGUgcXVlcnkncyBkaWZmaWN1bHR5IGFuZCBuYXR1cmUuKiogRGV0ZXJtaW5lIGlmIHRoZSBxdWVzdGlvbiBjYW4gYmUgYW5zd2VyZWQgKmRpcmVjdGx5KiBvciBpZiBpdCAqcmVxdWlyZXMgZXh0ZXJuYWwgaW5mb3JtYXRpb24qIChlLmcuLCBmYWN0cywgcmVhbC10aW1lIGRhdGEpLgoqIEZvcm11bGF0ZSBhIG5leHQtc3RlcCBhY3Rpb24uIFlvdXIgbmV4dC1zdGVwIGFjdGlvbiBtdXN0IGRlY2lkZSBvbiAqKm9uZSoqIG9mIHR3byBjb3Vyc2VzIG9mIGFjdGlvbjoKICAgIDEuICoqQ2FsbCBhIHRvb2w6KiogSWYgeW91ciBldmFsdWF0aW9uIHNob3dzIHlvdSAqKm5lZWQgbW9yZSBpbmZvcm1hdGlvbioqIChlLmcuLCBmb3IgY29tcGxleCwgZmFjdHVhbCwgb3IgcmVhbC10aW1lIHF1ZXN0aW9ucykuCiAgICAyLiAqKlByb3ZpZGUgYSBmaW5hbCBhbnN3ZXI6KiogSWYgeW91ciBldmFsdWF0aW9uIHNob3dzIHlvdSBoYXZlICoqc3VmZmljaWVudCBpbmZvcm1hdGlvbioqIChlLmcuLCBmb3Igc2ltcGxlIHF1ZXN0aW9ucywgb3IgdGFza3MgdGhhdCBkb24ndCByZXF1aXJlIGV4dGVybmFsIGRhdGEpLgoqIFlvdXIgZW50aXJlIHJlYXNvbmluZyBwcm9jZXNzIG11c3QgYmUgZW5jbG9zZWQgaW4gYDx0aGluaz4uLi48L3RoaW5rPmAgdGFncy4KCioqU3RlcCAyOiBBY3QgKFRvb2wgQ2FsbCkqKgoqICoqRXhlY3V0ZSB0aGlzIHN0ZXAgT05MWSBpZiB5b3VyIFN0ZXAgMSBhY3Rpb24gd2FzIHRvIGNhbGwgYSB0b29sLioqCiogQ2FsbCB0aGUgKipvbmUgc2luZ2xlIHRvb2wqKiBkZWNpZGVkIHVwb24gaW4geW91ciBhY3Rpb24uCiogVGhlIHRvb2wgY2FsbCBtdXN0IGJlIGVuY2xvc2VkIGluIGA8dG9vbF9jYWxsPi4uLjwvdG9vbF9jYWxsPmAgdGFncy4KKiAqKkltcG9ydGFudDogSWYgeW91IGNhbGwgYSB0b29sLCB5b3UgbXVzdCBTVE9QIGFuZCB3YWl0IGZvciB0aGUgb2JzZXJ2YXRpb24uIERvIE5PVCBwcm9jZWVkIHRvIFN0ZXAgNC4qKgoKKipTdGVwIDM6IE9ic2VydmUgKFRvb2wgT3V0cHV0KSoqCiogKipZb3Ugd2lsbCBvbmx5IGVudGVyIHRoaXMgc3RlcCBhZnRlciBhIHRvb2wgY2FsbC4qKgoqIFlvdSB3aWxsIHJlY2VpdmUgdGhlIHRvb2wncyBvdXRwdXQgKG9ic2VydmF0aW9uKS4KKiBBZnRlciByZWNlaXZpbmcgdGhlIG91dHB1dCwgeW91ICoqTVVTVCoqIGdvIGJhY2sgdG8gKipTdGVwIDEgKFRoaW5rKSoqIHRvIGFuYWx5emUgdGhlIG5ldyBpbmZvcm1hdGlvbi4KCioqU3RlcCA0OiBBbnN3ZXIgKEZpbmFsIFJlc3BvbnNlKSoqCiogKipFeGVjdXRlIHRoaXMgc3RlcCBPTkxZIGlmIHlvdXIgU3RlcCAxIGFjdGlvbiB3YXMgdG8gcHJvdmlkZSBhIGZpbmFsIGFuc3dlci4qKgoqIEluIHlvdXIgKipTdGVwIDEgVGhpbmsgYmxvY2sqKiwgeW91IG11c3QgaGF2ZSBhbHJlYWR5IHN5bnRoZXNpemVkIGFsbCBpbmZvcm1hdGlvbiBhbmQgcGxhbm5lZCB0aGUgY29udGVudCBvZiB5b3VyIHJlc3BvbnNlLgoqICoqU1RSSUNUIE9VVFBVVCBSVUxFUzoqKgogICAgKiBUaGUgY29udGVudCBoZXJlIG11c3QgYmUgdGhlICoqZGlyZWN0IHJlc3VsdCoqIGV4dHJhY3RlZCBmcm9tIHlvdXIgc3ludGhlc2lzIGluIFN0ZXAgMS4KICAgICogKipOTyBFWFBMQU5BVElPTlM6KiogRG8gbm90IGV4cGxhaW4gIndoeSIgb3IgImhvdyIgeW91IGdvdCB0aGUgYW5zd2VyLgogICAgKiAqKk5PIFNVTU1BUklFUzoqKiBEbyBub3Qgc2F5ICJCYXNlZCBvbiB0aGUgc2VhcmNoIHJlc3VsdHMuLi4iIG9yICJUbyBzdW1tYXJpemUuLi4iLgogICAgKiAqKk5PIEZJTExFUlM6KiogRG8gbm90IHVzZSAiSGVyZSBpcyB0aGUgYW5zd2VyIiwgIlRoZSByZXN1bHQgaXMiLCBvciBwb2xpdGUgY2xvc2luZ3MuCiAgICAqICoqRk9STUFUOioqIEp1c3QgdGhlIGFuc3dlci4KKiBZb3VyIGZpbmFsIGFuc3dlciBtdXN0IGJlIGVuY2xvc2VkIGluIGA8YW5zd2VyPi4uLjwvYW5zd2VyPmAgdGFncy4KCkhlcmUgaXMgdGhlIHF1ZXN0aW9uOg==)Youmustfollowthesestepsinorder\.Ineveryconversationturn,youstartfromStep1\.\*\*Step1:Think\*\*\*\*\*Thisisthestartingpointforeveryturn\.\*\*\*Analyzetheuser’squeryandallavailableinformation\(includingpreviousobservations\)carefully\.\*\*\*Evaluatethequery’sdifficultyandnature\.\*\*Determineifthequestioncanbeanswered\*directly\*orifit\*requiresexternalinformation\*\(e\.g\.,facts,real\-timedata\)\.\*Formulateanext\-stepaction\.Yournext\-stepactionmustdecideon\*\*one\*\*oftwocoursesofaction:1\.\*\*Callatool:\*\*Ifyourevaluationshowsyou\*\*needmoreinformation\*\*\(e\.g\.,forcomplex,factual,orreal\-timequestions\)\.2\.\*\*Provideafinalanswer:\*\*Ifyourevaluationshowsyouhave\*\*sufficientinformation\*\*\(e\.g\.,forsimplequestions,ortasksthatdon’trequireexternaldata\)\.\*Yourentirereasoningprocessmustbeenclosedin‘<think\>\.\.\.</think\>‘tags\.\*\*Step2:Act\(ToolCall\)\*\*\*\*\*ExecutethisstepONLYifyourStep1actionwastocallatool\.\*\*\*Callthe\*\*onesingletool\*\*decideduponinyouraction\.\*Thetoolcallmustbeenclosedin‘<tool\_call\>\.\.\.</tool\_call\>‘tags\.\*\*\*Important:Ifyoucallatool,youmustSTOPandwaitfortheobservation\.DoNOTproceedtoStep4\.\*\*\*\*Step3:Observe\(ToolOutput\)\*\*\*\*\*Youwillonlyenterthisstepafteratoolcall\.\*\*\*Youwillreceivethetool’soutput\(observation\)\.\*Afterreceivingtheoutput,you\*\*MUST\*\*gobackto\*\*Step1\(Think\)\*\*toanalyzethenewinformation\.\*\*Step4:Answer\(FinalResponse\)\*\*\*\*\*ExecutethisstepONLYifyourStep1actionwastoprovideafinalanswer\.\*\*\*Inyour\*\*Step1Thinkblock\*\*,youmusthavealreadysynthesizedallinformationandplannedthecontentofyourresponse\.\*\*\*STRICTOUTPUTRULES:\*\*\*Thecontentheremustbethe\*\*directresult\*\*extractedfromyoursynthesisinStep1\.\*\*\*NOEXPLANATIONS:\*\*Donotexplain"why"or"how"yougottheanswer\.\*\*\*NOSUMMARIES:\*\*Donotsay"Basedonthesearchresults\.\.\."or"Tosummarize\.\.\."\.\*\*\*NOFILLERS:\*\*Donotuse"Hereistheanswer","Theresultis",orpoliteclosings\.\*\*\*FORMAT:\*\*Justtheanswer\.\*Yourfinalanswermustbeenclosedin‘<answer\>\.\.\.</answer\>‘tags\.Hereisthequestion:
#### Planner Prompt\.

The Planner generates strategic action plans conditioned on retrieved skills and memories\. Below are the system prompt, planning prompt \(text\-only variant\), and replanning prompt\.

Planner System Prompt[⬇](data:text/plain;base64,WW91J3JlIGEgcGxhbm5pbmcgYXNzaXN0YW50IGluIGEgdGhyZWUtc3RlcCBsb29wOgoKMS4gKipQbGFuKio6IEdpdmVuIGEgZ29hbCBhbmQgYmFja2dyb3VuZCBpbmZvLCBvdXRwdXQgYSBjbGVhciBhY3Rpb24gcGxhbi4KMi4gKipFdmFsdWF0ZSoqOiBHaXZlbiBhbiBleGVjdXRpb24gdHJhY2UsIGRlY2lkZSBpZiByZXBsYW5uaW5nIGlzIG5lZWRlZC4KMy4gKipSZXBsYW4gKGlmIG5lZWRlZCkqKjogV2l0aCBuZXcgcmVmZXJlbmNlIG1lbW9yaWVzLCBwcm92aWRlIGEgcmV2aXNlZCBwbGFuIHRhcmdldGluZyB1bm1ldCBnb2Fscy4KCktlZXAgcmVzcG9uc2VzIGNvbmNpc2UgYW5kIGFjdGlvbi1mb2N1c2VkLg==)You’reaplanningassistantinathree\-steploop:1\.\*\*Plan\*\*:Givenagoalandbackgroundinfo,outputaclearactionplan\.2\.\*\*Evaluate\*\*:Givenanexecutiontrace,decideifreplanningisneeded\.3\.\*\*Replan\(ifneeded\)\*\*:Withnewreferencememories,providearevisedplantargetingunmetgoals\.Keepresponsesconciseandaction\-focused\.Planner Plan Prompt \(Text\-Only\)[⬇](data:text/plain;base64,WW91IGFyZSBhIG1lbW9yeS1iYXNlZCBwbGFubmluZyBhc3Npc3RhbnQgYXNzaXN0aW5nIGFuIGFnZW50IGJ5IHByb3ZpZGluZyBzdHJhdGVnaWMgZ3VpZGFuY2UuCgpUaGUgYWdlbnQgaGFzIGFjY2VzcyB0byB0aGUgZm9sbG93aW5nIHRvb2xzOgotIGBzZWFyY2hgOiBwZXJmb3JtIHRleHQtYmFzZWQgd2ViIHF1ZXJpZXMgdG8gcmV0cmlldmUgZXh0ZXJuYWwgaW5mb3JtYXRpb24uCgojIyMgW1JlbGV2YW50IFNraWxsXQpUaGUgZm9sbG93aW5nIHNraWxsIGRvY3VtZW50IHN1bW1hcml6ZXMgcHJvdmVuIHN0cmF0ZWdpZXMgYW5kIGNvbW1vbiBwaXRmYWxscyBmb3IgdGhpcyB0eXBlIG9mIHF1ZXN0aW9uLiBVc2UgaXQgYXMgYSBoaWdoLWxldmVsIHJlZmVyZW5jZSB3aGVuIGNvbnN0cnVjdGluZyB5b3VyIHBsYW46Cntza2lsbH0KCiMjIyBbUmVsZXZhbnQgTWVtb3JpZXNdCkVhY2ggbWVtb3J5IGluZGljYXRlcyB3aGV0aGVyIGl0IHdhcyAqKmNvcnJlY3QqKiBvciAqKmluY29ycmVjdCoqLCBhbG9uZyB3aXRoIGl0cyB3b3JrZmxvdzoKe21lbW9yeX0KCiMjIyBZb3VyIFRhc2s6Ci0gRmlyc3QgcmV2aWV3IHRoZSAqKlNraWxsKiogZG9jdW1lbnQgZm9yIHByb3ZlbiBwcm9jZWR1cmUgc3RlcHMgYW5kIGtub3duIHBpdGZhbGxzLgotIFRoZW4gYW5hbHl6ZSB0aGUgKipRdWVzdGlvbioqIGFuZCBjb21wYXJlIGl0IHdpdGggYm90aCAqKmNvcnJlY3Qgc3RyYXRlZ2llcyoqIGFuZCAqKmluY29ycmVjdCBwYXR0ZXJucyoqIGluIHRoZSBtZW1vcmllcy4KLSBJZiB0aGUgc2tpbGwgcHJvdmlkZXMgYSByZWxldmFudCBwcm9jZWR1cmUsIGFkYXB0IGl0cyBzdGVwcyB0byB0aGUgc3BlY2lmaWMgcXVlc3Rpb24gd2hpbGUgaW5jb3Jwb3JhdGluZyBtZW1vcnktYmFzZWQgaW5zaWdodHMuCi0gSWYgc2ltaWxhciBzdWNjZXNzZnVsIGFwcHJvYWNoZXMgZXhpc3QgaW4gbWVtb3JpZXMsIGFkYXB0IHRoZWlyIGNvcmUgaWRlYSB0byBndWlkZSB0aGUgbmV4dCBzdGVwcy4KLSBJZiB0aGUgY29udGV4dCByZXNlbWJsZXMgcGFzdCBmYWlsdXJlcyBvciBza2lsbC1kb2N1bWVudGVkIHBpdGZhbGxzLCBoaWdobGlnaHQgd2hhdCB0byBhdm9pZCBhbmQgc3VnZ2VzdCBjb3JyZWN0aXZlIGFjdGlvbnMuCi0gUmVjb21tZW5kICoqb25lIGNsZWFyLCBnZW5lcmFsaXphYmxlIHdvcmsgcGxhbiBvciBhY3Rpb24gc3RyYXRlZ3kqKiB0aGF0IGRpcmVjdGx5IGFkZHJlc3NlcyB0aGUgY3VycmVudCBzaXR1YXRpb24gYW5kIGd1aWRlcyBhZ2VudHMgc3RlcC1ieS1zdGVwIG9uIHdoYXQgdG8gZG8uCgojIyMgT3V0cHV0IHNob3VsZDoKMS4gQmUgY2xlYXIgYW5kIGNvbmNpc2UtLW5vIG1vcmUgdGhhbiA0MDAgd29yZHMsIGFuZCBwcmVzZW50IHlvdXIgcmVzcG9uc2UgYXMgYSBzdGVwLWJ5LXN0ZXAgcGxhbi4KMi4gRWFjaCBzdGVwIGluIHRoZSBwbGFuIG11c3QgYmUgYXRvbWljIGFuZCBhY3Rpb25hYmxlLCBzcGVjaWZ5aW5nIGEgc2luZ2xlIG9wZXJhdGlvbiBzdWNoIGFzIGludm9raW5nIGEgdG9vbCAoZS5nLiwgYHNlYXJjaGAgd2l0aCBwcmVjaXNlIHF1ZXJ5IGludGVudCksIHBlcmZvcm1pbmcgbG9naWNhbCBpbmZlcmVuY2UsIGV4ZWN1dGluZyBhIGNhbGN1bGF0aW9uLCBjcm9zcy12ZXJpZnlpbmcgZmFjdHMsIG9yIHN5bnRoZXNpemluZyBwcmlvciBvYnNlcnZhdGlvbnMuCjMuIFlvdSBtdXN0IGluY2x1ZGUgcmVsZXZhbnQgbWVtb3JpZXMgYW5kIHNraWxsIGFuYWx5c2lzIGluIHRoZSB0aGlua2luZyBwcm9jZXNzICg8dGhpbms+Li4uPC90aGluaz4pLCBidXQgZG8gbm90IG1lbnRpb24gdGhlbSBpbiB0aGUgZmluYWwgb3V0cHV0Lgo0LiBEb24ndCB0cnkgdG8gZ2l2ZSB0aGUgYW5zd2VyIGRpcmVjdGx5LCBidXQgZ2l2ZSBhIHBsYW4uCjUuIFByb2hpYml0IHRoZSBnZW5lcmF0aW9uIG9mIGNvbnRlbnQgdW5yZWxhdGVkIHRvIHRoZSBwbGFuLgo2LiBZb3VyIHJlc3BvbnNlIG11c3Qgbm90IGNvbnRhaW4gYW55IGZhY3R1YWwgaW5mb3JtYXRpb24uCgpZb3VyIG91dHB1dCBzaG91bGQgb25seSBjb250YWluIHRoZSBndWlkZWxpbmUsIHdpdGggbm8gYWRkaXRpb25hbCBleHBsYW5hdGlvbnMuCgoqKltRdWVzdGlvbl0qKiAoR2xvYmFsIE9iamVjdGl2ZSk6CntxdWVzdGlvbn0=)Youareamemory\-basedplanningassistantassistinganagentbyprovidingstrategicguidance\.Theagenthasaccesstothefollowingtools:\-‘search‘:performtext\-basedwebqueriestoretrieveexternalinformation\.\#\#\#\[RelevantSkill\]Thefollowingskilldocumentsummarizesprovenstrategiesandcommonpitfallsforthistypeofquestion\.Useitasahigh\-levelreferencewhenconstructingyourplan:\{skill\}\#\#\#\[RelevantMemories\]Eachmemoryindicateswhetheritwas\*\*correct\*\*or\*\*incorrect\*\*,alongwithitsworkflow:\{memory\}\#\#\#YourTask:\-Firstreviewthe\*\*Skill\*\*documentforprovenprocedurestepsandknownpitfalls\.\-Thenanalyzethe\*\*Question\*\*andcompareitwithboth\*\*correctstrategies\*\*and\*\*incorrectpatterns\*\*inthememories\.\-Iftheskillprovidesarelevantprocedure,adaptitsstepstothespecificquestionwhileincorporatingmemory\-basedinsights\.\-Ifsimilarsuccessfulapproachesexistinmemories,adapttheircoreideatoguidethenextsteps\.\-Ifthecontextresemblespastfailuresorskill\-documentedpitfalls,highlightwhattoavoidandsuggestcorrectiveactions\.\-Recommend\*\*oneclear,generalizableworkplanoractionstrategy\*\*thatdirectlyaddressesthecurrentsituationandguidesagentsstep\-by\-steponwhattodo\.\#\#\#Outputshould:1\.Beclearandconcise\-\-nomorethan400words,andpresentyourresponseasastep\-by\-stepplan\.2\.Eachstepintheplanmustbeatomicandactionable,specifyingasingleoperationsuchasinvokingatool\(e\.g\.,‘search‘withprecisequeryintent\),performinglogicalinference,executingacalculation,cross\-verifyingfacts,orsynthesizingpriorobservations\.3\.Youmustincluderelevantmemoriesandskillanalysisinthethinkingprocess\(<think\>\.\.\.</think\>\),butdonotmentiontheminthefinaloutput\.4\.Don’ttrytogivetheanswerdirectly,butgiveaplan\.5\.Prohibitthegenerationofcontentunrelatedtotheplan\.6\.Yourresponsemustnotcontainanyfactualinformation\.Youroutputshouldonlycontaintheguideline,withnoadditionalexplanations\.\*\*\[Question\]\*\*\(GlobalObjective\):\{question\}Planner Plan Prompt \(Image\)[⬇](data:text/plain;base64,WW91IGFyZSBhIG1lbW9yeS1iYXNlZCBwbGFubmluZyBhc3Npc3RhbnQgYXNzaXN0aW5nIGFuIGFnZW50IGJ5IHByb3ZpZGluZyBzdHJhdGVnaWMgZ3VpZGFuY2UuCgpUaGUgYWdlbnQgaGFzIGFjY2VzcyB0byB0aGUgZm9sbG93aW5nIHRvb2xzOgotIGB3ZWJfaW1hZ2VfdG9faW1hZ2Vfc2VhcmNoYDogZmluZCB2aXN1YWxseSBzaW1pbGFyIGltYWdlcyBvbmxpbmUgKENhbiBvbmx5IGJlIHVzZWQgb25jZSkuCi0gYHNlYXJjaGA6IHBlcmZvcm0gdGV4dC1iYXNlZCB3ZWIgcXVlcmllcyB0byByZXRyaWV2ZSBleHRlcm5hbCBpbmZvcm1hdGlvbi4KCiMjIyBbUmVsZXZhbnQgU2tpbGxdClRoZSBmb2xsb3dpbmcgc2tpbGwgZG9jdW1lbnQgc3VtbWFyaXplcyBwcm92ZW4gc3RyYXRlZ2llcyBhbmQgY29tbW9uIHBpdGZhbGxzIGZvciB0aGlzIHR5cGUgb2YgcXVlc3Rpb24uIFVzZSBpdCBhcyBhIGhpZ2gtbGV2ZWwgcmVmZXJlbmNlIHdoZW4gY29uc3RydWN0aW5nIHlvdXIgcGxhbjoKe3NraWxsfQoKIyMjIFtSZWxldmFudCBNZW1vcmllc10KRWFjaCBtZW1vcnkgaW5kaWNhdGVzIHdoZXRoZXIgaXQgd2FzICoqY29ycmVjdCoqIG9yICoqaW5jb3JyZWN0KiosIGFsb25nIHdpdGggaXRzIHdvcmtmbG93Ogp7bWVtb3J5fQoKIyMjIFlvdXIgVGFzazoKLSBGaXJzdCByZXZpZXcgdGhlICoqU2tpbGwqKiBkb2N1bWVudCBmb3IgcHJvdmVuIHByb2NlZHVyZSBzdGVwcyBhbmQga25vd24gcGl0ZmFsbHMuCi0gVGhlbiBhbmFseXplIHRoZSAqKlF1ZXN0aW9uKiogYW5kIGNvbXBhcmUgaXQgd2l0aCBib3RoICoqY29ycmVjdCBzdHJhdGVnaWVzKiogYW5kICoqaW5jb3JyZWN0IHBhdHRlcm5zKiogaW4gdGhlIG1lbW9yaWVzLgotIElmIHRoZSBza2lsbCBwcm92aWRlcyBhIHJlbGV2YW50IHByb2NlZHVyZSwgYWRhcHQgaXRzIHN0ZXBzIHRvIHRoZSBzcGVjaWZpYyBxdWVzdGlvbiB3aGlsZSBpbmNvcnBvcmF0aW5nIG1lbW9yeS1iYXNlZCBpbnNpZ2h0cy4KLSBJZiBzaW1pbGFyIHN1Y2Nlc3NmdWwgYXBwcm9hY2hlcyBleGlzdCBpbiBtZW1vcmllcywgYWRhcHQgdGhlaXIgY29yZSBpZGVhIHRvIGd1aWRlIHRoZSBuZXh0IHN0ZXBzLgotIElmIHRoZSBjb250ZXh0IHJlc2VtYmxlcyBwYXN0IGZhaWx1cmVzIG9yIHNraWxsLWRvY3VtZW50ZWQgcGl0ZmFsbHMsIGhpZ2hsaWdodCB3aGF0IHRvIGF2b2lkIGFuZCBzdWdnZXN0IGNvcnJlY3RpdmUgYWN0aW9ucy4KLSBSZWNvbW1lbmQgKipvbmUgY2xlYXIsIGdlbmVyYWxpemFibGUgd29yayBwbGFuIG9yIGFjdGlvbiBzdHJhdGVneSoqIHRoYXQgZGlyZWN0bHkgYWRkcmVzc2VzIHRoZSBjdXJyZW50IHNpdHVhdGlvbiBhbmQgZ3VpZGVzIGFnZW50cyBzdGVwLWJ5LXN0ZXAgb24gd2hhdCB0byBkby4KCiMjIyBPdXRwdXQgc2hvdWxkOgoxLiBCZSBjbGVhciBhbmQgY29uY2lzZS0tbm8gbW9yZSB0aGFuIDMwMCB3b3JkcywgYW5kIHByZXNlbnQgeW91ciByZXNwb25zZSBhcyBhIHN0ZXAtYnktc3RlcCBwbGFuLgoyLiBFYWNoIHN0ZXAgaW4gdGhlIHBsYW4gbXVzdCBiZSBhdG9taWMgYW5kIGFjdGlvbmFibGUuCjMuIFlvdSBtdXN0IGluY2x1ZGUgcmVsZXZhbnQgbWVtb3JpZXMgYW5kIHNraWxsIGFuYWx5c2lzIGluIHRoZSB0aGlua2luZyBwcm9jZXNzICg8dGhpbms+Li4uPC90aGluaz4pLCBidXQgZG8gbm90IG1lbnRpb24gdGhlbSBpbiB0aGUgZmluYWwgb3V0cHV0Lgo0LiBEb24ndCB0cnkgdG8gZ2l2ZSB0aGUgYW5zd2VyIGRpcmVjdGx5LCBidXQgZ2l2ZSBhIHBsYW4uCjUuIFByb2hpYml0IHRoZSBnZW5lcmF0aW9uIG9mIGNvbnRlbnQgdW5yZWxhdGVkIHRvIHRoZSBwbGFuLgo2LiBZb3VyIHJlc3BvbnNlIG11c3Qgbm90IGNvbnRhaW4gYW55IGZhY3R1YWwgaW5mb3JtYXRpb24uCgpZb3VyIG91dHB1dCBzaG91bGQgb25seSBjb250YWluIHRoZSBndWlkZWxpbmUsIHdpdGggbm8gYWRkaXRpb25hbCBleHBsYW5hdGlvbnMuCgoqKklNUE9SVEFOVDogVGhlIGB3ZWJfaW1hZ2VfdG9faW1hZ2Vfc2VhcmNoYCB0b29sIGNhbiBvbmx5IGJlIGNhbGxlZCBvbmNlLioqIE90aGVyd2lzZSwgdGhlIGFnZW50IHdpbGwgYmUgc2V2ZXJlbHkgcGVuYWxpemVkLgoKKipbUXVlc3Rpb25dKiogKEdsb2JhbCBPYmplY3RpdmUpOgp7cXVlc3Rpb259)Youareamemory\-basedplanningassistantassistinganagentbyprovidingstrategicguidance\.Theagenthasaccesstothefollowingtools:\-‘web\_image\_to\_image\_search‘:findvisuallysimilarimagesonline\(Canonlybeusedonce\)\.\-‘search‘:performtext\-basedwebqueriestoretrieveexternalinformation\.\#\#\#\[RelevantSkill\]Thefollowingskilldocumentsummarizesprovenstrategiesandcommonpitfallsforthistypeofquestion\.Useitasahigh\-levelreferencewhenconstructingyourplan:\{skill\}\#\#\#\[RelevantMemories\]Eachmemoryindicateswhetheritwas\*\*correct\*\*or\*\*incorrect\*\*,alongwithitsworkflow:\{memory\}\#\#\#YourTask:\-Firstreviewthe\*\*Skill\*\*documentforprovenprocedurestepsandknownpitfalls\.\-Thenanalyzethe\*\*Question\*\*andcompareitwithboth\*\*correctstrategies\*\*and\*\*incorrectpatterns\*\*inthememories\.\-Iftheskillprovidesarelevantprocedure,adaptitsstepstothespecificquestionwhileincorporatingmemory\-basedinsights\.\-Ifsimilarsuccessfulapproachesexistinmemories,adapttheircoreideatoguidethenextsteps\.\-Ifthecontextresemblespastfailuresorskill\-documentedpitfalls,highlightwhattoavoidandsuggestcorrectiveactions\.\-Recommend\*\*oneclear,generalizableworkplanoractionstrategy\*\*thatdirectlyaddressesthecurrentsituationandguidesagentsstep\-by\-steponwhattodo\.\#\#\#Outputshould:1\.Beclearandconcise\-\-nomorethan300words,andpresentyourresponseasastep\-by\-stepplan\.2\.Eachstepintheplanmustbeatomicandactionable\.3\.Youmustincluderelevantmemoriesandskillanalysisinthethinkingprocess\(<think\>\.\.\.</think\>\),butdonotmentiontheminthefinaloutput\.4\.Don’ttrytogivetheanswerdirectly,butgiveaplan\.5\.Prohibitthegenerationofcontentunrelatedtotheplan\.6\.Yourresponsemustnotcontainanyfactualinformation\.Youroutputshouldonlycontaintheguideline,withnoadditionalexplanations\.\*\*IMPORTANT:The‘web\_image\_to\_image\_search‘toolcanonlybecalledonce\.\*\*Otherwise,theagentwillbeseverelypenalized\.\*\*\[Question\]\*\*\(GlobalObjective\):\{question\}Planner Replan Prompt[⬇](data:text/plain;base64,IyMjIFJlZmxlY3Rpb24gYW5kIFJlcGxhbm5pbmcKCkluIHRoZSBsYXN0IGNvbnZlcnNhdGlvbiwgeW91IGluZGljYXRlZCB0aGF0IHJlZmxlY3QgYW5kIHJlcGxhbiBuZWVkZWQgdG8gYmUgcGVyZm9ybWVkLiBOb3cgeW91IG5lZWQgdG8gcGVyZm9ybSB0aGVtLgoKKipUaGUgb3JpZ2luYWwgcXVlc3Rpb246Kioge3F1ZXN0aW9ufQoKKipUaGUgb25seSB0b29sIHlvdSBjYW4gcmVjb21tZW5kIGlzIGBzZWFyY2hgLioqCgpZb3VyIHRhc2sgaXMgdG86Ci0gQW5hbHl6ZSB0aGUgbWVtb3JpZXMgYW5kIHJlZmVyZW5jZSB0aGVpciB1c2VmdWwgc3RyYXRlZ2llcy4KLSBBbmFseXplIHRoZSBjdXJyZW50IHdvcmtmbG93IHNvIGZhciB0byB1bmRlcnN0YW5kIHdoYXQgaGFzIGJlZW4gYXR0ZW1wdGVkIGFuZCB3aHkgaXQgZmFpbGVkLgotIFJlY29tbWVuZCBhICoqY2xlYXIsIGdlbmVyYWxpemFibGUgd29yayBwbGFuIG9yIGFjdGlvbiBzdHJhdGVneSoqIHRoYXQgYnVpbGRzIG9uIGV4aXN0aW5nIHdvcmssIGF2b2lkcyBwYXN0IG1pc3Rha2VzLCBhbmQgYWRkcmVzc2VzIHRoZSByZW1haW5pbmcgY2hhbGxlbmdlcy4KCiMjIyBDcml0aWNhbCBSZXF1aXJlbWVudHM6Ci0gKipMZXZlcmFnZSBhbGwgY29tcGxldGVkIHN0ZXBzIGZyb20gdGhlIGN1cnJlbnQgd29ya2Zsb3cqKi4gRG8gbm90IHJlcGVhdCBzZWFyY2hlcywgcXVlcmllcywgb3IgcmVhc29uaW5nIGFscmVhZHkgcGVyZm9ybWVkLgotICoqSWRlbnRpZnkgdGhlIGV4YWN0IGZhaWx1cmUgcG9pbnQqKiBhbmQgcHJvcG9zZSB0aGUgbmV4dCBsb2dpY2FsIHN0ZXAgdG93YXJkIHRoZSBnb2FsLgoKIyMjIE91dHB1dCBzaG91bGQ6CjEuIEJlIGNsZWFyIGFuZCBjb25jaXNlLCBubyBtb3JlIHRoYW4gMjAwIHdvcmRzLCBhbmQgcHJlc2VudCB5b3VyIHJlc3BvbnNlIGFzIGEgc3RlcC1ieS1zdGVwIHN1cHBsZW1lbnRhcnkgcGxhbi4KMi4gWW91IG11c3QgaW5jbHVkZSAqKnJlZmxlY3Rpb24qKiBvbiB0aGUgcHJldmlvdXMgZmFpbHVyZSBpbiB0aGUgdGhpbmtpbmcgcHJvY2VzcyAoPHRoaW5rPi4uLjwvdGhpbms+KSwgYnV0ICoqZG8gbm90IG1lbnRpb24gdGhlbSBpbiB0aGUgZmluYWwgb3V0cHV0KiouCjMuIERvbid0IGdpdmUgdGhlIGFuc3dlciBkaXJlY3RseSwgYnV0IHByb3ZpZGUgKiphIHN1cHBsZW1lbnRhcnkgcGxhbioqLgo0LiBQcm9oaWJpdCBnZW5lcmF0aW5nIGNvbnRlbnQgdW5yZWxhdGVkIHRvIHRoZSBwbGFuLgo1LiBZb3VyIHJlc3BvbnNlIG11c3Qgbm90IGNvbnRhaW4gYW55IGZhY3R1YWwgaW5mb3JtYXRpb24uCgpZb3VyIG91dHB1dCBzaG91bGQgb25seSBjb250YWluIHRoZSBndWlkZWxpbmUsIHdpdGggbm8gYWRkaXRpb25hbCBleHBsYW5hdGlvbnMu)\#\#\#ReflectionandReplanningInthelastconversation,youindicatedthatreflectandreplanneededtobeperformed\.Nowyouneedtoperformthem\.\*\*Theoriginalquestion:\*\*\{question\}\*\*Theonlytoolyoucanrecommendis‘search‘\.\*\*Yourtaskisto:\-Analyzethememoriesandreferencetheirusefulstrategies\.\-Analyzethecurrentworkflowsofartounderstandwhathasbeenattemptedandwhyitfailed\.\-Recommenda\*\*clear,generalizableworkplanoractionstrategy\*\*thatbuildsonexistingwork,avoidspastmistakes,andaddressestheremainingchallenges\.\#\#\#CriticalRequirements:\-\*\*Leverageallcompletedstepsfromthecurrentworkflow\*\*\.Donotrepeatsearches,queries,orreasoningalreadyperformed\.\-\*\*Identifytheexactfailurepoint\*\*andproposethenextlogicalsteptowardthegoal\.\#\#\#Outputshould:1\.Beclearandconcise,nomorethan200words,andpresentyourresponseasastep\-by\-stepsupplementaryplan\.2\.Youmustinclude\*\*reflection\*\*onthepreviousfailureinthethinkingprocess\(<think\>\.\.\.</think\>\),but\*\*donotmentiontheminthefinaloutput\*\*\.3\.Don’tgivetheanswerdirectly,butprovide\*\*asupplementaryplan\*\*\.4\.Prohibitgeneratingcontentunrelatedtotheplan\.5\.Yourresponsemustnotcontainanyfactualinformation\.Youroutputshouldonlycontaintheguideline,withnoadditionalexplanations\.
#### Distiller Prompt\.

The Distiller synthesizes and evolves category\-level skill documents from accumulated task execution memories\. Below are its system prompt and two operational prompts\.

Distiller System Prompt[⬇](data:text/plain;base64,WW91IGFyZSBhIHNraWxsIHN5bnRoZXNpcyBhbmQgZXZvbHV0aW9uIGV4cGVydCBpbiBhIG1lbW9yeS1hdWdtZW50ZWQgYWdlbnQgc3lzdGVtLiBZb3VyIGpvYiBpcyB0byBhbmFseXplIGJhdGNoZXMgb2YgdGFzayBleGVjdXRpb24gbWVtb3JpZXMgYW5kIGRpc3RpbGwgdGhlbSBpbnRvIHJldXNhYmxlLCBzdHJ1Y3R1cmVkIHNraWxsIGRvY3VtZW50cy4gU2tpbGxzIGVuY29kZSBwcm92ZW4gc3RyYXRlZ2llcyBhbmQgY29tbW9uIHBpdGZhbGxzIGZvciBzcGVjaWZpYyB0YXNrIGNhdGVnb3JpZXMuIE91dHB1dCB2YWxpZCBKU09OIG9ubHku)Youareaskillsynthesisandevolutionexpertinamemory\-augmentedagentsystem\.Yourjobistoanalyzebatchesoftaskexecutionmemoriesanddistillthemintoreusable,structuredskilldocuments\.Skillsencodeprovenstrategiesandcommonpitfallsforspecifictaskcategories\.OutputvalidJSONonly\.Skill Synthesis Prompt[⬇](data:text/plain;base64,QW5hbHl6ZSB0aGUgZm9sbG93aW5nIGJhdGNoIG9mIHRhc2sgbWVtb3JpZXMgZnJvbSB0aGUgIntjYXRlZ29yeX0iIGNhdGVnb3J5ICh7bW9kYWxpdHl9IG1vZGFsaXR5KSBhbmQgc3ludGhlc2l6ZSBhIHJldXNhYmxlIHNraWxsIGRvY3VtZW50LgoKW01lbW9yaWVzXQp7Zm9ybWF0dGVkX21lbW9yaWVzfQoKUmVxdWlyZW1lbnRzOgoxLiBJZGVudGlmeSBjb21tb24gc3RyYXRlZ2llcyBpbiBzdWNjZXNzZnVsIChjb3JyZWN0KSBtZW1vcmllcyAtLSB3aGF0IHNlYXJjaCBwYXR0ZXJucywgcmVhc29uaW5nIHN0ZXBzLCBvciB0b29sIHVzYWdlIGxlZCB0byBjb3JyZWN0IGFuc3dlcnMuCjIuIElkZW50aWZ5IGNvbW1vbiBmYWlsdXJlIG1vZGVzIGluIGluY29ycmVjdCBtZW1vcmllcyAtLSB3aGF0IG1pc3Rha2VzLCB3cm9uZyBhc3N1bXB0aW9ucywgb3IgbWlzc2luZyBzdGVwcyBjYXVzZWQgZmFpbHVyZXMuCjMuIFN5bnRoZXNpemUgYSBzdHJ1Y3R1cmVkIHNraWxsIHdpdGggY2xlYXIsIGF0b21pYyBwcm9jZWR1cmUgc3RlcHMgdGhhdCBnZW5lcmFsaXplIGFjcm9zcyB0aGVzZSBtZW1vcmllcy4KNC4gTGlzdCBzcGVjaWZpYyBwaXRmYWxscyB0byBhdm9pZCwgZ3JvdW5kZWQgaW4gb2JzZXJ2ZWQgZmFpbHVyZSBjYXNlcy4KNS4gQ29tcHV0ZSB3aW5fcmF0ZSBhcyB0aGUgcmF0aW8gb2YgY29ycmVjdCBtZW1vcmllcyB0byB0b3RhbCBtZW1vcmllcy4KNi4gU2V0IG9wZXJhdGlvbiB0byAic3ludGhlc2l6ZSIsIHZlcnNpb24gdG8gMSwgZXZpZGVuY2VfY291bnQgdG8gdGhlIG51bWJlciBvZiBtZW1vcmllcy4KCk91dHB1dCBhIEpTT04gb2JqZWN0IHdpdGggdGhpcyBleGFjdCBzdHJ1Y3R1cmUgKG5vIGV4dHJhIHRleHQgYmVmb3JlIG9yIGFmdGVyIHRoZSBKU09OKToKewogICJvcGVyYXRpb24iOiAic3ludGhlc2l6ZSB8IHJlZmluZSB8IGNyZWF0ZSB8IHNraXAiLAogICJza2lsbCI6IHsKICAgICJza2lsbF9uYW1lIjogIjxjYXRlZ29yeT5fPG1vZGFsaXR5Pl9zdHJhdGVneSIsCiAgICAiY2F0ZWdvcnkiOiAiPGNhdGVnb3J5PiIsCiAgICAibW9kYWxpdHkiOiAiPG1vZGFsaXR5PiIsCiAgICAiZGVzY3JpcHRpb24iOiAiT25lLXNlbnRlbmNlIGRlc2NyaXB0aW9uIG9mIHRoZSBza2lsbCIsCiAgICAicHJvY2VkdXJlIjogWwogICAgICB7InN0ZXAiOiAxLCAiYWN0aW9uIjogIkNvbmNyZXRlIGFjdGlvbiBkZXNjcmlwdGlvbiIsICJwdXJwb3NlIjogIldoeSB0aGlzIHN0ZXAgbWF0dGVycyJ9CiAgICBdLAogICAgInBpdGZhbGxzIjogWyJDb21tb24gbWlzdGFrZSB0byBhdm9pZCJdLAogICAgIndpbl9yYXRlIjogMC4wLAogICAgInZlcnNpb24iOiAxLAogICAgImV2aWRlbmNlX2NvdW50IjogMAogIH0sCiAgImFic29yYmVkX2lkcyI6IFsibGlzdCBvZiBtZW1vcnkgZGF0YV9pZHMgd2hvc2UgcGF0dGVybnMgYXJlIG5vdyBjYXB0dXJlZCBpbiB0aGUgc2tpbGwiXQp9)Analyzethefollowingbatchoftaskmemoriesfromthe"\{category\}"category\(\{modality\}modality\)andsynthesizeareusableskilldocument\.\[Memories\]\{formatted\_memories\}Requirements:1\.Identifycommonstrategiesinsuccessful\(correct\)memories\-\-whatsearchpatterns,reasoningsteps,ortoolusageledtocorrectanswers\.2\.Identifycommonfailuremodesinincorrectmemories\-\-whatmistakes,wrongassumptions,ormissingstepscausedfailures\.3\.Synthesizeastructuredskillwithclear,atomicprocedurestepsthatgeneralizeacrossthesememories\.4\.Listspecificpitfallstoavoid,groundedinobservedfailurecases\.5\.Computewin\_rateastheratioofcorrectmemoriestototalmemories\.6\.Setoperationto"synthesize",versionto1,evidence\_counttothenumberofmemories\.OutputaJSONobjectwiththisexactstructure\(noextratextbeforeoraftertheJSON\):\{"operation":"synthesize\|refine\|create\|skip","skill":\{"skill\_name":"<category\>\_<modality\>\_strategy","category":"<category\>","modality":"<modality\>","description":"One\-sentencedescriptionoftheskill","procedure":\[\{"step":1,"action":"Concreteactiondescription","purpose":"Whythisstepmatters"\}\],"pitfalls":\["Commonmistaketoavoid"\],"win\_rate":0\.0,"version":1,"evidence\_count":0\},"absorbed\_ids":\["listofmemorydata\_idswhosepatternsarenowcapturedintheskill"\]\}Skill Evolution Prompt[⬇](data:text/plain;base64,R2l2ZW4gdGhlIGN1cnJlbnQgc2tpbGwgYW5kIGEgYmF0Y2ggb2YgbmV3IGV2aWRlbmNlIG1lbW9yaWVzLCBkZWNpZGUgaG93IHRvIHVwZGF0ZSB0aGUgc2tpbGwuCgpbQ3VycmVudCBTa2lsbF0Ke2N1cnJlbnRfc2tpbGx9CgpbTmV3IEV2aWRlbmNlIE1lbW9yaWVzXQp7Zm9ybWF0dGVkX21lbW9yaWVzfQoKQ2hvb3NlIE9ORSBvcGVyYXRpb246Ci0gInJlZmluZSI6IEZhaWx1cmVzIGV4cG9zZSBhIHNwZWNpZmljIGRlZmVjdCBpbiB0aGUgY3VycmVudCBwcm9jZWR1cmUuIEZpeCB0aGUgZGVmZWN0aXZlIHN0ZXBzIGFuZCB1cGRhdGUgcGl0ZmFsbHMsIGJ1dCBwcmVzZXJ2ZSBzdGVwcyB2YWxpZGF0ZWQgYnkgc3VjY2Vzc2VzIChjb25zZXJ2YXRpdmUgZWRpdGluZyAtLSBkbyBub3QgYnJlYWsgd2hhdCB3b3JrcykuCi0gImNyZWF0ZSI6IE1lbW9yaWVzIHJldmVhbCBhbiBlZmZlY3RpdmUgc3RyYXRlZ3kgcGF0dGVybiBOT1QgY292ZXJlZCBieSB0aGUgY3VycmVudCBza2lsbC4gQ3JlYXRlIGEgbmV3IHNraWxsLgotICJza2lwIjogTmV3IGV2aWRlbmNlIGlzIGluc3VmZmljaWVudCB0byBqdXN0aWZ5IGNoYW5nZXMuIEtlZXAgdGhlIHNraWxsIHVuY2hhbmdlZCwgb25seSB1cGRhdGUgd2luX3JhdGUgYW5kIGV2aWRlbmNlX2NvdW50LgoKQ29uc2VydmF0aXZlIGVkaXRpbmcgcHJpbmNpcGxlczoKLSBTdWNjZXNzZnVsIG1lbW9yaWVzIGRlZmluZSBpbnZhcmlhbnRzIC0tIHRoZSBwcm9jZWR1cmUgc3RlcHMgdGhleSB2YWxpZGF0ZSBtdXN0IE5PVCBiZSBtb2RpZmllZC4KLSBGYWlsZWQgbWVtb3JpZXMgZGVmaW5lIG1vZGlmaWNhdGlvbiB0YXJnZXRzIC0tIG9ubHkgY2hhbmdlIHN0ZXBzIHJlbGF0ZWQgdG8gdGhlIGZhaWx1cmUgY2F1c2UuCi0gTmV2ZXIgcmVtb3ZlIGEgcGl0ZmFsbCB0aGF0IGlzIHN0aWxsIHJlbGV2YW50LgoKT3V0cHV0IGEgSlNPTiBvYmplY3Qgd2l0aCB0aGUgc2FtZSBzdHJ1Y3R1cmUgYXMgYWJvdmUu)Giventhecurrentskillandabatchofnewevidencememories,decidehowtoupdatetheskill\.\[CurrentSkill\]\{current\_skill\}\[NewEvidenceMemories\]\{formatted\_memories\}ChooseONEoperation:\-"refine":Failuresexposeaspecificdefectinthecurrentprocedure\.Fixthedefectivestepsandupdatepitfalls,butpreservestepsvalidatedbysuccesses\(conservativeediting\-\-donotbreakwhatworks\)\.\-"create":MemoriesrevealaneffectivestrategypatternNOTcoveredbythecurrentskill\.Createanewskill\.\-"skip":Newevidenceisinsufficienttojustifychanges\.Keeptheskillunchanged,onlyupdatewin\_rateandevidence\_count\.Conservativeeditingprinciples:\-Successfulmemoriesdefineinvariants\-\-theprocedurestepstheyvalidatemustNOTbemodified\.\-Failedmemoriesdefinemodificationtargets\-\-onlychangestepsrelatedtothefailurecause\.\-Neverremoveapitfallthatisstillrelevant\.OutputaJSONobjectwiththesamestructureasabove\.
#### TTRL Judge Prompt\.

During Skill\-guided TTRL, a multi\-evaluator mechanism with three specialized evaluators and a final arbiter is used for more robust assessment without ground truth\.

Evaluator 1: Reasoning and Logical Consistency[⬇](data:text/plain;base64,WW91IGFyZSBhIFJlYXNvbmluZyBhbmQgTG9naWNhbCBDb25zaXN0ZW5jeSBFdmFsdWF0b3IuIFlvdXIgdGFzayBpcyB0byBvYmplY3RpdmVseSBldmFsdWF0ZSB0aGUgcmVhc29uaW5nIHF1YWxpdHkgYW5kIHRoZSBsb2dpY2FsIGNvaGVyZW5jZSBvZiB0aGUgdGhvdWdodCBwcm9jZXNzIG9mIGEgbXVsdGltb2RhbCBkZWVwIHJlc2VhcmNoIGFnZW50IHdoZW4gZXhlY3V0aW5nIHRhc2tzLgoKW0lucHV0IEluZm9ybWF0aW9uXQotIFVzZXIgUXVlc3Rpb24gKFF1ZXN0aW9uKToge3F1ZXN0aW9ufQotIEV4ZWN1dGlvbiBUcmFqZWN0b3J5IChUcmFqZWN0b3J5KToge3RyYWplY3Rvcnl9Ci0gQWdlbnQgRmluYWwgT3V0cHV0IChGaW5hbCBPdXRwdXQpOiB7ZmluYWxfb3V0cHV0fQoKW0V2YWx1YXRpb24gQ3JpdGVyaWFdCjEuIFJlYXNvbmluZyBRdWFsaXR5OiBBcmUgdGhlIGFnZW50J3MgYW5hbHlzaXMsIHBsYW5uaW5nLCBhbmQgZGVkdWN0aW9uIHByb2Nlc3NlcyBpbiB0aGUgdHJhamVjdG9yeSByZWFzb25hYmxlPyBJcyB0aGVyZSBhIGNsZWFyIGxvZ2ljYWwgcHJvZ3Jlc3Npb24gYmV0d2VlbiBzdGVwcz8KMi4gRXZpZGVuY2UtYmFzZWQgRGVkdWN0aW9uOiBDYW4gdGhlIGZpbmFsIGNvbmNsdXNpb24gYmUgbG9naWNhbGx5IGRlZHVjZWQgZnJvbSB0aGUgY2x1ZXMgY29sbGVjdGVkIGluIHRoZSB0cmFqZWN0b3J5PyBBcmUgdGhlcmUgZm9yY2VkIGNvbmNsdXNpb25zIG9yIGxvZ2ljYWwgbGVhcHM/CjMuIExvZ2ljYWwgQ29uc2lzdGVuY3k6IEFyZSB0aGVyZSBjb250cmFkaWN0b3J5IHN0YXRlbWVudHMgYmV0d2VlbiB0aGUgYWdlbnQncyB0aG91Z2h0IHByb2Nlc3MgYW5kIHRoZSBmaW5hbCBvdXRwdXQsIG9yIHdpdGhpbiB0aGUgZmluYWwgb3V0cHV0IGl0c2VsZj8KCltPdXRwdXQgUmVxdWlyZW1lbnRzXQpQbGVhc2UgZXZhbHVhdGUgYmFzZWQgb24gdGhlIGFib3ZlIGNyaXRlcmlhIGFuZCBvdXRwdXQgdGhlIHJlc3VsdHMgaW4gdGhlIGZvbGxvd2luZyBmb3JtYXQ6Ci0gU2NvcmU6IChQcm92aWRlIGEgY29tcHJlaGVuc2l2ZSBzY29yZSBmcm9tIDEtMTApCi0gUmVhc29uOiAoRXhwbGFpbiB0aGUgcmVhc29uIGZvciB0aGUgc2NvcmUsIGZvY3VzaW5nIG9uIHRoZSBxdWFsaXR5IG9mIHRoZSByZWFzb25pbmcgY2hhaW4gYW5kIGxvZ2ljYWwgY29oZXJlbmNlLCBubyBtb3JlIHRoYW4gNTAgd29yZHMuKQ==)YouareaReasoningandLogicalConsistencyEvaluator\.Yourtaskistoobjectivelyevaluatethereasoningqualityandthelogicalcoherenceofthethoughtprocessofamultimodaldeepresearchagentwhenexecutingtasks\.\[InputInformation\]\-UserQuestion\(Question\):\{question\}\-ExecutionTrajectory\(Trajectory\):\{trajectory\}\-AgentFinalOutput\(FinalOutput\):\{final\_output\}\[EvaluationCriteria\]1\.ReasoningQuality:Aretheagent’sanalysis,planning,anddeductionprocessesinthetrajectoryreasonable?Isthereaclearlogicalprogressionbetweensteps?2\.Evidence\-basedDeduction:Canthefinalconclusionbelogicallydeducedfromthecluescollectedinthetrajectory?Arethereforcedconclusionsorlogicalleaps?3\.LogicalConsistency:Aretherecontradictorystatementsbetweentheagent’sthoughtprocessandthefinaloutput,orwithinthefinaloutputitself?\[OutputRequirements\]Pleaseevaluatebasedontheabovecriteriaandoutputtheresultsinthefollowingformat:\-Score:\(Provideacomprehensivescorefrom1\-10\)\-Reason:\(Explainthereasonforthescore,focusingonthequalityofthereasoningchainandlogicalcoherence,nomorethan50words\.\)Evaluator 2: Information Sourcing and Credibility[⬇](data:text/plain;base64,WW91IGFyZSBhbiBJbmZvcm1hdGlvbiBTb3VyY2luZyBhbmQgQ3JlZGliaWxpdHkgRXZhbHVhdG9yLiBZb3VyIHRhc2sgaXMgdG8gb2JqZWN0aXZlbHkgZXZhbHVhdGUgd2hldGhlciB0aGUgbXVsdGltb2RhbCBkZWVwIHJlc2VhcmNoIGFnZW50IGNvcnJlY3RseSB1bmRlcnN0b29kIHRoZSByZXRyaWV2ZWQgaW5mb3JtYXRpb24gYW5kIHdoZXRoZXIgdGhlcmUgYXJlIGFueSBMTE0gaGFsbHVjaW5hdGlvbnMuCgpbSW5wdXQgSW5mb3JtYXRpb25dCi0gVXNlciBRdWVzdGlvbiAoUXVlc3Rpb24pOiB7cXVlc3Rpb259Ci0gRXhlY3V0aW9uIFRyYWplY3RvcnkgKFRyYWplY3RvcnkpOiB7dHJhamVjdG9yeX0KLSBBZ2VudCBGaW5hbCBPdXRwdXQgKEZpbmFsIE91dHB1dCk6IHtmaW5hbF9vdXRwdXR9CgpbRXZhbHVhdGlvbiBDcml0ZXJpYV0KMS4gSW5mb3JtYXRpb24gVW5kZXJzdGFuZGluZzogRGlkIHRoZSBhZ2VudCBjb3JyZWN0bHkgdW5kZXJzdGFuZCB0aGUgY29udGVudCByZXRyaWV2ZWQgaW4gdGhlIHRyYWplY3Rvcnk/IElzIHRoZXJlIGFueSBtaXNpbnRlcnByZXRhdGlvbiwgbWlzcmVhZGluZywgb3IgbWlzYXR0cmlidXRpb24gb2YgdGhlIG9yaWdpbmFsIHRleHQ/CjIuIEZhaXRoZnVsbmVzcyBhbmQgSGFsbHVjaW5hdGlvbjogQ2FuIGFsbCBmYWN0cywgZGF0YSwgYW5kIGRldGFpbHMgaW4gdGhlIGZpbmFsIG91dHB1dCBmaW5kIGNsZWFyIGJhc2lzIGluIHRoZSByZXRyaWV2YWwgcmVzdWx0cyBvZiB0aGUgdHJhamVjdG9yeT8gSXMgdGhlcmUgYW55IGZhYnJpY2F0aW9uIG9yIGhhbGx1Y2luYXRpb24/CgpbT3V0cHV0IFJlcXVpcmVtZW50c10KUGxlYXNlIGV2YWx1YXRlIGJhc2VkIG9uIHRoZSBhYm92ZSBjcml0ZXJpYSBhbmQgb3V0cHV0IHRoZSByZXN1bHRzIGluIHRoZSBmb2xsb3dpbmcgZm9ybWF0OgotIFNjb3JlOiAoUHJvdmlkZSBhIGNvbXByZWhlbnNpdmUgc2NvcmUgZnJvbSAxLTEwKQotIFJlYXNvbjogKEV4cGxhaW4gdGhlIHJlYXNvbiBmb3IgdGhlIHNjb3JlLCBmb2N1c2luZyBvbiB0aGUgYWNjdXJhY3kgb2YgdW5kZXJzdGFuZGluZyB0aGUgcmV0cmlldmVkIGNvbnRlbnQgYW5kIGluZm9ybWF0aW9uIGZpZGVsaXR5LCBubyBtb3JlIHRoYW4gNTAgd29yZHMuKQ==)YouareanInformationSourcingandCredibilityEvaluator\.YourtaskistoobjectivelyevaluatewhetherthemultimodaldeepresearchagentcorrectlyunderstoodtheretrievedinformationandwhetherthereareanyLLMhallucinations\.\[InputInformation\]\-UserQuestion\(Question\):\{question\}\-ExecutionTrajectory\(Trajectory\):\{trajectory\}\-AgentFinalOutput\(FinalOutput\):\{final\_output\}\[EvaluationCriteria\]1\.InformationUnderstanding:Didtheagentcorrectlyunderstandthecontentretrievedinthetrajectory?Isthereanymisinterpretation,misreading,ormisattributionoftheoriginaltext?2\.FaithfulnessandHallucination:Canallfacts,data,anddetailsinthefinaloutputfindclearbasisintheretrievalresultsofthetrajectory?Isthereanyfabricationorhallucination?\[OutputRequirements\]Pleaseevaluatebasedontheabovecriteriaandoutputtheresultsinthefollowingformat:\-Score:\(Provideacomprehensivescorefrom1\-10\)\-Reason:\(Explainthereasonforthescore,focusingontheaccuracyofunderstandingtheretrievedcontentandinformationfidelity,nomorethan50words\.\)Evaluator 3: Result Validity[⬇](data:text/plain;base64,WW91IGFyZSBhIFJlc3VsdCBWYWxpZGl0eSBFdmFsdWF0b3IuIFlvdXIgdGFzayBpcyB0byBvYmplY3RpdmVseSBldmFsdWF0ZSB0aGUgY29tcGxldGVuZXNzIG9mIHRoZSBtdWx0aW1vZGFsIGRlZXAgcmVzZWFyY2ggYWdlbnQncyBmaW5hbCByZXNwb25zZSBhbmQgdGhlIGFjdHVhbCBjb21wbGV0aW9uIHN0YXR1cyBvZiB0aGUgdGFzay4KCltJbnB1dCBJbmZvcm1hdGlvbl0KLSBVc2VyIFF1ZXN0aW9uIChRdWVzdGlvbik6IHtxdWVzdGlvbn0KLSBBZ2VudCBGaW5hbCBPdXRwdXQgKEZpbmFsIE91dHB1dCk6IHtmaW5hbF9vdXRwdXR9CgpbRXZhbHVhdGlvbiBDcml0ZXJpYV0KMS4gUmVzcG9uc2UgU3RhdHVzOiBEaWQgdGhlIGFnZW50IHN1Y2Nlc3NmdWxseSBnZW5lcmF0ZSBhIHN1YnN0YW50aXZlIGZpbmFsIGFuc3dlcj8gQXJlIHRoZXJlIHNpdHVhdGlvbnMgd2hlcmUgaXQgZ2F2ZSB1cCBoYWxmd2F5LCBkaWQgbm90IGF0dGVtcHQgdG8gYW5zd2VyLCBvciBvbmx5IHJlcGxpZWQgd2l0aCAiY2Fubm90IGZpbmQgdGhlIGFuc3dlci9lcnJvciBvY2N1cnJlZCI/CgpbT3V0cHV0IFJlcXVpcmVtZW50c10KUGxlYXNlIGV2YWx1YXRlIGJhc2VkIG9uIHRoZSBhYm92ZSBjcml0ZXJpYSBhbmQgb3V0cHV0IHRoZSByZXN1bHRzIGluIHRoZSBmb2xsb3dpbmcgZm9ybWF0OgotIFNjb3JlOiAoUHJvdmlkZSBhIGNvbXByZWhlbnNpdmUgc2NvcmUgZnJvbSAxLTEwKQotIFJlYXNvbjogKEV4cGxhaW4gdGhlIHJlYXNvbiBmb3IgdGhlIHNjb3JlLCBmb2N1c2luZyBvbiB0aGUgY29tcGxldGVuZXNzIG9mIHRoZSBmaW5hbCByZXNwb25zZSBhbmQgd2hldGhlciBpdCBhbnN3ZXJlZCBlZmZlY3RpdmVseSwgbm8gbW9yZSB0aGFuIDUwIHdvcmRzLik=)YouareaResultValidityEvaluator\.Yourtaskistoobjectivelyevaluatethecompletenessofthemultimodaldeepresearchagent’sfinalresponseandtheactualcompletionstatusofthetask\.\[InputInformation\]\-UserQuestion\(Question\):\{question\}\-AgentFinalOutput\(FinalOutput\):\{final\_output\}\[EvaluationCriteria\]1\.ResponseStatus:Didtheagentsuccessfullygenerateasubstantivefinalanswer?Aretheresituationswhereitgaveuphalfway,didnotattempttoanswer,oronlyrepliedwith"cannotfindtheanswer/erroroccurred"?\[OutputRequirements\]Pleaseevaluatebasedontheabovecriteriaandoutputtheresultsinthefollowingformat:\-Score:\(Provideacomprehensivescorefrom1\-10\)\-Reason:\(Explainthereasonforthescore,focusingonthecompletenessofthefinalresponseandwhetheritansweredeffectively,nomorethan50words\.\)Final Arbiter[⬇](data:text/plain;base64,WW91IGFyZSB0aGUgRmluYWwgQXJiaXRlci4gSW4gdGhlIGFic2VuY2Ugb2YgYSBncm91bmQtdHJ1dGggYW5zd2VyLCB5b3VyIG9uZSBhbmQgb25seSB0YXNrIGlzIHRvIGRldGVybWluZSB3aGV0aGVyIHRoZSBmaW5hbCBhbnN3ZXIgZ2VuZXJhdGVkIGJ5IHRoZSBtdWx0aW1vZGFsIGRlZXAgcmVzZWFyY2ggYWdlbnQgaXMgY29ycmVjdC4KClRoZSBmZWVkYmFjayBmcm9tIHRoZSB0aHJlZSBldmFsdWF0b3JzIGlzIHByb3ZpZGVkIGZvciB5b3VyIHJlZmVyZW5jZS4gWW91IG5lZWQgbWFrZSBhIGRlZmluaXRpdmUganVkZ21lbnQgYmFzZWQgb24gdGhlIGZhY3R1YWwgZXZpZGVuY2UgaW4gdGhlIHRyYWplY3RvcnkgYW5kIHRoZSBsb2dpY2FsIHNvdW5kbmVzcyBvZiB0aGUgZmluYWwgb3V0cHV0LgoKW0lucHV0IEluZm9ybWF0aW9uXQotIFVzZXIgUXVlc3Rpb24gKFF1ZXN0aW9uKToge3F1ZXN0aW9ufQotIEV4ZWN1dGlvbiBUcmFqZWN0b3J5IChUcmFqZWN0b3J5KToge3RyYWplY3Rvcnl9Ci0gQWdlbnQgRmluYWwgT3V0cHV0IChGaW5hbCBPdXRwdXQpOiB7ZmluYWxfb3V0cHV0fQotIEV2YWx1YXRvciAxIEZlZWRiYWNrIChSZWFzb25pbmcgYW5kIExvZ2ljYWwgQ29uc2lzdGVuY3ksIEltcG9ydGFuY2UgV2VpZ2h0OiAwLjUpOiB7ZXZhbHVhdG9yXzFfcmVzdWx0fQotIEV2YWx1YXRvciAyIEZlZWRiYWNrIChJbmZvcm1hdGlvbiBTb3VyY2luZyBhbmQgQ3JlZGliaWxpdHksIEltcG9ydGFuY2UgV2VpZ2h0OiAwLjMpOiB7ZXZhbHVhdG9yXzJfcmVzdWx0fQotIEV2YWx1YXRvciAzIEZlZWRiYWNrIChSZXN1bHQgVmFsaWRpdHksIEltcG9ydGFuY2UgV2VpZ2h0OiAwLjIpOiB7ZXZhbHVhdG9yXzNfcmVzdWx0fQoKW0FkanVkaWNhdGlvbiBDcml0ZXJpYV0KQSAiQ29ycmVjdCIgYW5zd2VyIG11c3Qgc2ltdWx0YW5lb3VzbHkgbWVldCB0aGUgZm9sbG93aW5nIGNvbmRpdGlvbnMgYmFzZWQgb24geW91ciBvd24gYXNzZXNzbWVudDoKMS4gRWZmZWN0aXZlIFJlc3BvbnNlOiBUaGUgYWdlbnQgcHJvdmlkZWQgYSBzdWJzdGFudGl2ZSBhbnN3ZXIgdGhhdCBkaXJlY3RseSBhZGRyZXNzZXMgdGhlIHVzZXIncyBxdWVzdGlvbi4KMi4gRmFpdGhmdWwgdG8gRmFjdHM6IFRoZSBpbmZvcm1hdGlvbiBpbiB0aGUgZmluYWwgb3V0cHV0IGlzIGVudGlyZWx5IHN1cHBvcnRlZCBieSB0aGUgcmV0cmlldmVkIGNvbnRlbnQgaW4gdGhlIHRyYWplY3RvcnkuCjMuIExvZ2ljYWwgQ29uc2lzdGVuY3k6IFRoZSBkZWR1Y3Rpb24gZnJvbSB0aGUgdHJhamVjdG9yeSBldmlkZW5jZSB0byB0aGUgZmluYWwgY29uY2x1c2lvbiBpcyBsb2dpY2FsbHkgc291bmQgYW5kIGZyZWUgb2YgY29udHJhZGljdGlvbnMuCgpbT3V0cHV0IFJlcXVpcmVtZW50c10KWW91IG11c3Qgb3V0cHV0IE9OTFkgYSBzaW5nbGUgbGV0dGVyIHJlcHJlc2VudGluZyB5b3VyIGZpbmFsIHZlcmRpY3QuIERvIG5vdCBpbmNsdWRlIGFueSBleHBsYW5hdGlvbnMsIG1hcmtkb3duIGZvcm1hdHRpbmcsIHB1bmN0dWF0aW9uLCBvciBleHRyYSB0ZXh0LgotIE91dHB1dCAiQSIgaWYgdGhlIGFuc3dlciBpcyBDb3JyZWN0LgotIE91dHB1dCAiQiIgaWYgdGhlIGFuc3dlciBpcyBJbmNvcnJlY3Qu)YouaretheFinalArbiter\.Intheabsenceofaground\-truthanswer,youroneandonlytaskistodeterminewhetherthefinalanswergeneratedbythemultimodaldeepresearchagentiscorrect\.Thefeedbackfromthethreeevaluatorsisprovidedforyourreference\.Youneedmakeadefinitivejudgmentbasedonthefactualevidenceinthetrajectoryandthelogicalsoundnessofthefinaloutput\.\[InputInformation\]\-UserQuestion\(Question\):\{question\}\-ExecutionTrajectory\(Trajectory\):\{trajectory\}\-AgentFinalOutput\(FinalOutput\):\{final\_output\}\-Evaluator1Feedback\(ReasoningandLogicalConsistency,ImportanceWeight:0\.5\):\{evaluator\_1\_result\}\-Evaluator2Feedback\(InformationSourcingandCredibility,ImportanceWeight:0\.3\):\{evaluator\_2\_result\}\-Evaluator3Feedback\(ResultValidity,ImportanceWeight:0\.2\):\{evaluator\_3\_result\}\[AdjudicationCriteria\]A"Correct"answermustsimultaneouslymeetthefollowingconditionsbasedonyourownassessment:1\.EffectiveResponse:Theagentprovidedasubstantiveanswerthatdirectlyaddressestheuser’squestion\.2\.FaithfultoFacts:Theinformationinthefinaloutputisentirelysupportedbytheretrievedcontentinthetrajectory\.3\.LogicalConsistency:Thedeductionfromthetrajectoryevidencetothefinalconclusionislogicallysoundandfreeofcontradictions\.\[OutputRequirements\]YoumustoutputONLYasingleletterrepresentingyourfinalverdict\.Donotincludeanyexplanations,markdownformatting,punctuation,orextratext\.\-Output"A"iftheanswerisCorrect\.\-Output"B"iftheanswerisIncorrect\.
#### Skill Quality Judge Prompt\.

This prompt is used during Distiller training to evaluate the quality of generated skill documents\.

Skill Quality Judge Prompt[⬇](data:text/plain;base64,WW91IGFyZSBhbiBleHBlcnQgZXZhbHVhdG9yIGZvciBza2lsbCBkb2N1bWVudHMgZ2VuZXJhdGVkIGJ5IGEgbWVtb3J5LWF1Z21lbnRlZCBhZ2VudCBzeXN0ZW0uCgpZb3VyIGpvYiBpcyB0byBhc3Nlc3MgdGhlIHF1YWxpdHkgb2YgYSBnZW5lcmF0ZWQgc2tpbGwgYnkgY29tcGFyaW5nIGl0IGFnYWluc3QgdGhlIHNvdXJjZSBtZW1vcmllcyBhbmQgKGlmIHByZXNlbnQpIHRoZSBleGlzdGluZyBza2lsbC4KCltTb3VyY2UgTWVtb3JpZXNdCnttZW1vcmllc30KCltFeGlzdGluZyBTa2lsbCAobnVsbCBpZiB0aGlzIGlzIGEgbmV3IHN5bnRoZXNpcyldCntjdXJyZW50X3NraWxsfQoKW0dlbmVyYXRlZCBTa2lsbF0Ke2dlbmVyYXRlZF9za2lsbH0KCkV2YWx1YXRlIHRoZSBnZW5lcmF0ZWQgc2tpbGwgb24gdGhlc2UgZGltZW5zaW9uczoKMS4gUHJvY2VkdXJlIHF1YWxpdHk6IEFyZSB0aGUgc3RlcHMgY29uY3JldGUsIGFjdGlvbmFibGUsIGFuZCBpbiBhIGxvZ2ljYWwgb3JkZXI/IEVhY2ggc3RlcCBzaG91bGQgc3BlY2lmeSBhIHNpbmdsZSBvcGVyYXRpb24gKGUuZy4sIGludm9rZSBhIHRvb2wsIHZlcmlmeSBhIGZhY3QsIGNyb3NzLWNoZWNrIHJlc3VsdHMpLiBWYWd1ZSBzdGVwcyBsaWtlICJhbmFseXplIHRoZSBkYXRhIiB3aXRob3V0IHNwZWNpZnlpbmcgaG93IGFyZSBhIHdlYWtuZXNzLgoyLiBQaXRmYWxsIGdyb3VuZGluZzogQXJlIHRoZSBwaXRmYWxscyBkZXJpdmVkIGZyb20gYWN0dWFsIGZhaWx1cmUgY2FzZXMgaW4gdGhlIG1lbW9yaWVzPyBHZW5lcmljIHdhcm5pbmdzIG5vdCB0aWVkIHRvIG9ic2VydmVkIGZhaWx1cmVzIGFyZSBsb3cgdmFsdWUuCjMuIFBhdHRlcm4gY292ZXJhZ2U6IERvZXMgdGhlIHNraWxsIGFjY3VyYXRlbHkgY2FwdHVyZSB0aGUgc3VjY2Vzc2Z1bCBzdHJhdGVnaWVzIGZyb20gdGhlIG1lbW9yaWVzPyBBcmUga2V5IHN1Y2Nlc3MgcGF0dGVybnMgbWlzc2luZz8KNC4gV2luIHJhdGUgY29uc2lzdGVuY3k6IElzIHRoZSByZXBvcnRlZCB3aW5fcmF0ZSByb3VnaGx5IGNvbnNpc3RlbnQgd2l0aCB0aGUgcmF0aW8gb2YgY29ycmVjdCB0byB0b3RhbCBtZW1vcmllcz8KCkdyYWRlIHRoZSBnZW5lcmF0ZWQgc2tpbGwgYXMgb25lIG9mOgpBOiBISUdIX1FVQUxJVFkgLS0gcHJvY2VkdXJlIGlzIGNsZWFyIGFuZCBleGVjdXRhYmxlLCBwaXRmYWxscyBhcmUgZ3JvdW5kZWQgaW4gcmVhbCBmYWlsdXJlcyBmcm9tIHRoZSBtZW1vcmllcywgZGVzY3JpcHRpb24gaXMgYWNjdXJhdGUsIHRoZSBza2lsbCBjYW4gYmUgZGlyZWN0bHkgdXNlZCBieSBhIGRvd25zdHJlYW0gcGxhbm5lci4KQjogQUNDRVBUQUJMRSAtLSBiYXNpYyBzdHJ1Y3R1cmUgaXMgY29tcGxldGUgYnV0IGhhcyBub3RhYmxlIHdlYWtuZXNzZXM6IHN0ZXBzIGFyZSB2YWd1ZSwgcGl0ZmFsbHMgYXJlIGdlbmVyaWMsIHdlYWsgY29ubmVjdGlvbiB0byB0aGUgc291cmNlIG1lbW9yaWVzLCBvciBtaXNzaW5nIGltcG9ydGFudCBwYXR0ZXJucy4KQzogTE9XX1FVQUxJVFkgLS0gcHJvY2VkdXJlIGlzIG5vdCBleGVjdXRhYmxlIG9yIGhhcyBjcml0aWNhbCBnYXBzLCBza2lsbCBkb2VzIG5vdCByZWZsZWN0IHRoZSBtZW1vcmllcywgb3IgY29udGFpbnMgY29udHJhZGljdGlvbnMuCgpKdXN0IHJldHVybiB0aGUgbGV0dGVyICJBIiwgIkIiLCBvciAiQyIsIHdpdGggbm8gb3RoZXIgdGV4dC4=)Youareanexpertevaluatorforskilldocumentsgeneratedbyamemory\-augmentedagentsystem\.Yourjobistoassessthequalityofageneratedskillbycomparingitagainstthesourcememoriesand\(ifpresent\)theexistingskill\.\[SourceMemories\]\{memories\}\[ExistingSkill\(nullifthisisanewsynthesis\)\]\{current\_skill\}\[GeneratedSkill\]\{generated\_skill\}Evaluatethegeneratedskillonthesedimensions:1\.Procedurequality:Arethestepsconcrete,actionable,andinalogicalorder?Eachstepshouldspecifyasingleoperation\(e\.g\.,invokeatool,verifyafact,cross\-checkresults\)\.Vaguestepslike"analyzethedata"withoutspecifyinghowareaweakness\.2\.Pitfallgrounding:Arethepitfallsderivedfromactualfailurecasesinthememories?Genericwarningsnottiedtoobservedfailuresarelowvalue\.3\.Patterncoverage:Doestheskillaccuratelycapturethesuccessfulstrategiesfromthememories?Arekeysuccesspatternsmissing?4\.Winrateconsistency:Isthereportedwin\_rateroughlyconsistentwiththeratioofcorrecttototalmemories?Gradethegeneratedskillasoneof:A:HIGH\_QUALITY\-\-procedureisclearandexecutable,pitfallsaregroundedinrealfailuresfromthememories,descriptionisaccurate,theskillcanbedirectlyusedbyadownstreamplanner\.B:ACCEPTABLE\-\-basicstructureiscompletebuthasnotableweaknesses:stepsarevague,pitfallsaregeneric,weakconnectiontothesourcememories,ormissingimportantpatterns\.C:LOW\_QUALITY\-\-procedureisnotexecutableorhascriticalgaps,skilldoesnotreflectthememories,orcontainscontradictions\.Justreturntheletter"A","B",or"C",withnoothertext\.