EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds

# An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds
Source: [https://arxiv.org/html/2610.20786](https://arxiv.org/html/2610.20786)
Jingqi ShiAffiliation:Harbin Institute of TechnologyBaoxin WangHonghong ZhaoAffiliation:iFLYTEKHaoyuan XuAffiliation:Harbin Institute of TechnologyYueqing SunAffiliation:MeituanJiafeng LiangAffiliation:Harbin Institute of TechnologyTao HeAffiliation:Harbin Institute of TechnologyDayong WuAffiliation:iFLYTEKQi GuShijin WangAffiliation:iFLYTEKMing LiuAffiliation:Harbin Institute of TechnologyBing QinAffiliation:Harbin Institute of TechnologyAffiliation:Pengcheng Laboratory\* Equal Contribution†Corresponding Author\{rxliu,mliu\}@ir\.hit\.edu\.cn,bxwang2@iflytek\.com,guqi03@meituan\.com

###### Abstract

Multi\-modal agents have made significant progress in interacting with open worlds\. However, their ability to self\-evolve remains limited\. Supervised fine\-tuning enables imitation but not active experience organization, while outcome\-based reinforcement learning promotes exploration yet underuses intermediate trajectory information\. Therefore, we proposeEvolvingBrain, which uses an adaptive skill system as the brain for agent self\-evolution, consisting of two following components\.Stratified Skill Self\-Evolution: extracts Procedural Patterns and Problematic Patterns from model\-specific trajectories, summarizes experience by generating skills, and evolves through multiple interactions with the open world, ultimately forming a Stratified Skill Architecture consisting of Domain, Plan, and Action/Trap Skills, which provides the agent with a multi\-granularity knowledge repository and a novel evolution approach\.State\-Aware Adaptive Execution: summarizes long\-horizon interactions into Structured Abstractive Memory and uses it to guide the State\-Aware Skill Fusion Strategy, enabling the agent to utilize knowledge according to its current state\. Experiments on VAB\-Minecraft and DiscoveryWorld demonstrate that EvolvingBrain improves success rates on open\-world tasks and exhibits generalization capabilities\. Our code is provided in the supplementary materials and will be publicly released in the camera\-ready version\.

## 1Introduction

Multimodal agents have made substantial progress in interacting with open worlds\. They can translate natural\-language instructions into actions, plan long\-horizon behaviors, and reuse procedural knowledge from prior experience\([Lifshitz et al\., 2023](https://arxiv.org/html/2610.20786#bib.bib1);[Wang et al\., 2024a](https://arxiv.org/html/2610.20786#bib.bib2);[Park et al\., 2025](https://arxiv.org/html/2610.20786#bib.bib3);[Zhang et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib4)\)\.

However, their ability to self\-evolve from interaction experience remains limited\. Supervised fine\-tuning teaches agents to imitate demonstrated behaviors but does not enable them to actively organize and consolidate experience\([Zeng et al\., 2024](https://arxiv.org/html/2610.20786#bib.bib30)\)\. Reinforcement learning driven by outcome\-based rewards can improve agents’ exploration capabilities but often underutilizes the rich intermediate information contained in interaction trajectories\([Uesato et al\., 2022](https://arxiv.org/html/2610.20786#bib.bib32);[Choudhury, 2025](https://arxiv.org/html/2610.20786#bib.bib31)\)\.

An adaptive skill system offers a alternative for agent self\-evolution: it can actively organize intermediate trajectory information into reusable knowledge and continuously refine this knowledge through interaction\. However, existing skill\-based methods primarily distill successful trajectories into procedural guidance\([Che et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib5)\), leaving three key issues unresolved: how to learn from both successful and failed trajectories, how to validate and evolve the extracted knowledge through environmental feedback, and how to organize and retrieve knowledge at different granularities according to the current state\. These limitations motivate EvolvingBrain\.

![Refer to caption](https://arxiv.org/html/2610.20786v1/introduction.png)Figure 1:Inspired by human experiential learning, EvolvingBrain extracts, evolves, organizes, and adaptively utilizes knowledge from open\-world interactions\.As illustrated in Figure[1](https://arxiv.org/html/2610.20786#S1.F1), humans improve by examining local procedures and failure modes, testing possible adjustments, and abstracting recurring experience into reusable knowledge\([Kolb, 1984](https://arxiv.org/html/2610.20786#bib.bib6)\)\. Inspired by this process, we proposeEvolvingBrain, which uses an adaptive skill system as the brain for agent self\-evolution\. EvolvingBrain consists of two components:Stratified Skill Self\-Evolution, which constructs and evolves a multi\-granularity knowledge repository, andState\-Aware Adaptive Execution, which utilizes this knowledge according to the current state\.

Stratified Skill Self\-EvolutionextractsProcedural PatternsandProblematic Patternsfrom model\-specific successful and failed trajectories\. Procedural Patterns capture reusable procedures, whereas Problematic Patterns characterize behaviors and obstacles that should be avoided or resolved\. These patterns are summarized asPlan Skillsfor task\-level planning,Action Skillsfor fine\-grained multimodal operations, andTrap Skillsfor problem identification, avoidance, and recovery\. The skills are applied and evolved through multiple interactions with the open world, while shared knowledge across tasks is further abstracted into coarse\-grainedDomain Skills\. Together, they form aStratified Skill Architecturethat provides the agent with a multi\-granularity knowledge repository and an evolution approach\.

DuringState\-Aware Adaptive Execution, long\-horizon interaction histories are summarized intoStructured Abstractive Memory, which records confirmed progress, current obstacles, and ineffective attempts\. Guided by this memory, theState\-Aware Skill Fusion Strategyretrieves and integrates relevant skills according to the current state, allowing the agent to utilize knowledge at an appropriate level of granularity\.

We evaluate EvolvingBrain using three multimodal backbones on VAB\-Minecraft\([Liu et al\., 2025](https://arxiv.org/html/2610.20786#bib.bib7)\)and DiscoveryWorld\([Jansen et al\., 2024](https://arxiv.org/html/2610.20786#bib.bib8)\)\. Averaged across the three backbones, EvolvingBrain improves success rates over direct agents by 11\.64 and 8\.44 percentage points, respectively\. The learned skills also transfer to held\-out DiscoveryWorld domains\. Further analyses demonstrate the complementary contributions of procedural and problematic knowledge, the relationship between backbone capability and skill granularity, and the effectiveness of feedback\-driven skill evolution\.

Our main contributions are as follows:

- •We proposeEvolvingBrain, an adaptive skill system that enables multimodal agents to actively organize, evolve, and reuse knowledge from their own open\-world interactions\.
- •We introduceStratified Skill Self\-Evolutionfor constructing and evolving a multi\-granularity skill architecture, together withState\-Aware Adaptive Executionfor retrieving and integrating knowledge according to the current interaction state\.
- •We systematically evaluate EvolvingBrain across three backbones on VAB\-Minecraft and DiscoveryWorld, demonstrating consistent performance improvements, skill\-granularity adaptation, and generalization to unseen domains\.

## 2Methods

![Refer to caption](https://arxiv.org/html/2610.20786v1/framework.png)Figure 2:Overview of EvolvingBrain\.### 2\.1Problem Formulation and Method Overview

Given a frozen multimodal agentMMand an open\-world environmentℰ\\mathcal\{E\}, the agent observesoto\_\{t\}, takes actionata\_\{t\}, and receives environmental feedbackftf\_\{t\}at each steptttoward a task goalgg\. The resulting trajectory is represented as

τ=\(g,\{ot,at,ft\}t=1T,y\),\\tau=\\left\(g,\\\{o\_\{t\},a\_\{t\},f\_\{t\}\\\}\_\{t=1\}^\{T\},y\\right\),\(1\)wherey∈\{0,1\}y\\in\\\{0,1\\\}, withy=1y=1denoting successful completion andy=0y=0denoting failure\.

Without updating model parameters, EvolvingBrain accumulates interaction experience in a persistent, model\-specific external*Skill Brain*:

ℬM\(k\)=\(𝒮D\(k\),𝒮P\(k\),𝒮A\(k\),𝒮T\(k\),𝒢\(k\)\),\\mathcal\{B\}\_\{M\}^\{\(k\)\}=\\left\(\\mathcal\{S\}\_\{D\}^\{\(k\)\},\\mathcal\{S\}\_\{P\}^\{\(k\)\},\\mathcal\{S\}\_\{A\}^\{\(k\)\},\\mathcal\{S\}\_\{T\}^\{\(k\)\},\\mathcal\{G\}^\{\(k\)\}\\right\),\(2\)where𝒮D\\mathcal\{S\}\_\{D\},𝒮P\\mathcal\{S\}\_\{P\},𝒮A\\mathcal\{S\}\_\{A\}, and𝒮T\\mathcal\{S\}\_\{T\}denote the Domain, Plan, Action, and Trap Skill sets, respectively, and𝒢\\mathcal\{G\}organizes their relationships\. Here,k∈\{0,…,K\}k\\in\\\{0,\\ldots,K\\\}indexes the self\-evolution rounds\.

Through iterative self\-evolution, EvolvingBrain transforms the initial Skill BrainℬM\(0\)\\mathcal\{B\}\_\{M\}^\{\(0\)\}intoℬM\(K\)\\mathcal\{B\}\_\{M\}^\{\(K\)\}, enabling the agent to improve task completion and reduce inefficient behaviors\.

As shown in Figure[2](https://arxiv.org/html/2610.20786#S2.F2), EvolvingBrain comprises two components\. During*Stratified Skill Self\-Evolution*, it extracts Procedural Patterns and Problematic Patterns from model\-specific successful and failed trajectories, evolves skills through repeated interactions with the open world, and organizes them into a multi\-granularity architecture of Domain, Plan, Action, and Trap Skills\. During*State\-Aware Adaptive Execution*, the evolved skill architecture is frozen, while Structured Abstractive Memory summarizes long\-horizon interactions and guides the dynamic retrieval and fusion of relevant skills according to the current state\.

Table 1:Relationship between trajectory types and pattern types\.
### 2\.2Stratified Skill Self\-Evolution

#### 2\.2\.1Model\-Specific Initial Trajectory Collection

Before skill evolution, each frozen target modelMMindependently interacts withℰ\\mathcal\{E\}and collects an initial trajectory set𝒯M\(0\)\\mathcal\{T\}\_\{M\}^\{\(0\)\}, where each trajectory follows Eq\.[1](https://arxiv.org/html/2610.20786#S2.E1)and\(0\)\(0\)denotes the collection stage before skill evolution\. Both successful and failed trajectories are retained\.

Skill construction forMMuses only𝒯M\(0\)\\mathcal\{T\}\_\{M\}^\{\(0\)\}and trajectories subsequently generated by the same model, without sharing a predefined trajectory pool or skill library\. The resulting skills therefore reflect the model’s capability boundary, failure distribution, and problems encountered in the open world\.

#### 2\.2\.2Procedural and Problematic Patterns

Rather than directly learning skills from successful and failed trajectories, we specify what should be learned by defining two complementary types of trajectory knowledge:

- •Procedural patterns:Reusable regularities that advance a task goal under particular task contexts and environmental conditions\.
- •Problematic patterns:Failure\-related conditions and behaviors that may cause ineffective actions, redundant steps, goal deviation, or task failure, along with structured records oftrigger conditions, erroneous behaviors, consequences, corresponding interaction steps, and resolution evidence\. Especially for multimodal problematic patterns, images from the relevant rounds are inspected to verify and record the cross\-modal cause\.

Success and failure are trajectory\-level outcomes, whereas procedural and problematic patterns are properties of trajectory segments\. Therefore, either trajectory type may contain both patterns, as summarized in Table[1](https://arxiv.org/html/2610.20786#S2.T1)\.

Trap Skills derived from problematic patterns can be consolidated into higher\-level skills, which represents transforming problematic patterns into procedural patterns\. EvolvingBrain also incorporates dedicated multimodal mechanisms, detailed in the following subsections\.

#### 2\.2\.3Constructing Trap Skill

Self\-Evolution Mechanism\.Firstly, EvolvingBrain extracts problematic patterns from trajectories\. For each selected pattern or pattern set, EvolvingBrain searches for existing Trap Skills through a two\-stage procedure\. It first queries a compact index containing each skill’s summary, problem description, root cause, and impact to shortlist up to three candidates, and then loads their complete contents for strict semantic verification\.

A skill is reused only when its problem mechanism, root cause, and repair steps match every selected pattern, including its multimodal cause when applicable\. Otherwise, a new Trap Skill is created, containingapplicable conditions, problem diagnosis, execution steps, actions to avoid, fallback strategies, and success criteria\. The Trap Skill schema and generation prompt are provided in Figures[10](https://arxiv.org/html/2610.20786#A5.F10)and[13](https://arxiv.org/html/2610.20786#A5.F13), respectively\.

The model then retries the task from its initial state with the accumulated Trap Skills\. Persistent problems trigger skill revision, while newly exposed problems initiate another iteration\. Failed and successful trajectories share the same mechanism, but differ in their specific implementations\. We introduce them separately below\.

Processing Failed Trajectory\.EvolvingBrain first selects the failed trajectories:

𝒯M,−\(0\)=\{τM,i\(0\)∈𝒯M\(0\)∣yM,i=0\}\.\\mathcal\{T\}\_\{M,\-\}^\{\(0\)\}=\\left\\\{\\tau\_\{M,i\}^\{\(0\)\}\\in\\mathcal\{T\}\_\{M\}^\{\(0\)\}\\mid y\_\{M,i\}=0\\right\\\}\.\(3\)Failure recovery focuses on root\-cause patterns that directly block task progress\. When multiple independent root causes exist, EvolvingBrain selects the minimum sufficient repair set and iteratively revises the corresponding Trap Skills until the task succeeds, the retry budget is exhausted, or no actionable root cause remains\.

The first successful retry is retained as a recovered trajectory\. All recovered trajectories form𝒯M,rec\\mathcal\{T\}\_\{M,\\mathrm\{rec\}\}and are combined with the initial successful trajectories:

𝒯M,succ=𝒯M,\+\(0\)∪𝒯M,rec\.\\mathcal\{T\}\_\{M,\\mathrm\{succ\}\}=\\mathcal\{T\}\_\{M,\+\}^\{\(0\)\}\\cup\\mathcal\{T\}\_\{M,\\mathrm\{rec\}\}\.\(4\)
Processing Successful Trajectory\.Task success does not guarantee high\-quality execution\. EvolvingBrain therefore inspects𝒯M,succ\\mathcal\{T\}\_\{M,\\mathrm\{succ\}\}for ineffective actions, repeated attempts, and incorrect steps\.

EvolvingBrain processes all problematic patterns rather than only root causes\. The patterns are handled in their order of occurrence\. Each retry uses the Trap Skills associated with the current and previously processed patterns, including those obtained during failure recovery\.

After all known problems are processed, EvolvingBrain audits the latest trajectory for task success, recurring problems, and newly emerged problems\. Any unresolved or new problem is returned to the iterative process\. A trajectory is accepted only when the task succeeds, no known problem recurs, and no new problem is detected\.

The accepted trajectories form𝒯M,HQ\\mathcal\{T\}\_\{M,\\mathrm\{HQ\}\}, while the accumulatedTrap Skillsform𝒮T\\mathcal\{S\}\_\{T\}\.

#### 2\.2\.4Constructing Plan and Action Skills

Taking𝒯M,HQ\\mathcal\{T\}\_\{M,\\mathrm\{HQ\}\}as input, EvolvingBrain distills procedural patterns into two levels of skills:Plan Skillsdescribe overall task procedures, whileAction Skillsdescribe concrete procedures for achieving local subgoals\.

For each trajectory, EvolvingBrain first generates a candidate Plan Skill containingthe task goal, applicable conditions, prerequisites, and end\-to\-end execution steps\.

It is compared with the current Plan Skill library based on the task goal\. A matched skill is reused if it already covers the candidate information, or updated if the candidate provides new prerequisites, step ordering, or a more reliable procedure\. Otherwise, a new Plan Skill is created\. The Plan Skill schema and generation prompt are provided in Figures[8](https://arxiv.org/html/2610.20786#A5.F8)and[11](https://arxiv.org/html/2610.20786#A5.F11), respectively\.

EvolvingBrain then divides the trajectory into ordered and contiguous segments, each representing a local state transition from an observable initial state, through actions toward a local goal, to a verifiable final state\. Only segments containing reusable procedural knowledge are retained\.

Their action sequences and multimodal observations are used to construct Action Skills that specifythe local goal, applicable conditions, action steps, visual cues, stopping conditions, and success criteria\. Reference images are retained only when they provide useful visual evidence\. The Action Skill schema and generation prompt are provided in Figures[9](https://arxiv.org/html/2610.20786#A5.F9)and[12](https://arxiv.org/html/2610.20786#A5.F12), respectively\.

Each candidate Action Skill is globally compared with the Action Skill library and is reused, updated, or newly created according to whether a matching skill exists and whether the candidate contributes new information\.

Processing all high\-quality trajectories produces the Plan Skill set𝒮P\\mathcal\{S\}\_\{P\}and Action Skill set𝒮A\\mathcal\{S\}\_\{A\}, whose structural relationships are established during Stratified Skill Architecture construction\.

#### 2\.2\.5Constructing Domain Skills and the Stratified Skill Architecture

After obtaining the Plan, Action, and Trap Skills, EvolvingBrain further derivesDomain Skills\.

It extracts candidate domain\-level principles from the three skill types\. Plan Skills provide task prerequisites, subgoal organization, and overall decision procedures; Action Skills supplement local state transitions and effect verification; and Trap Skills provide common problems, execution constraints, and recovery strategies\.

Semantically equivalent candidates are merged, while instance\-specific information is removed\. The resulting Domain Skills form𝒮D\\mathcal\{S\}\_\{D\}\.

The Domain Skill schema and synthesis prompt are provided in Figures[7](https://arxiv.org/html/2610.20786#A5.F7)and[14](https://arxiv.org/html/2610.20786#A5.F14), respectively\.

EvolvingBrain then organizes the four types of skills into aStratified Skill Architecture\. Each Domain Skill is connected to multiple Plan Skills within its domain, forming a one\-to\-many relationship\. A Plan Skill is connected to the Action Skills that achieve its local subgoals and to the Trap Skills learned from or reused in its associated trajectories\. Both types of relationships are many\-to\-many\.

Action Skills and Trap Skills are stored as atomic skills in the global skill library\. Therefore, a Plan Skill can compose multiple atomic skills, while the same atomic skill can be reused by multiple Plan Skills\. This shared structure improves skill generalization\.

EvolvingBrain thus forms a three\-level skill architecture consisting of Action / Trap, Plan, and Domain Skills, with the granularity increasing from bottom to top\. This architecture provides the structural foundation for subsequent process\.

### 2\.3State\-Aware Adaptive Execution

#### 2\.3\.1Structured Abstractive Memory

Long\-horizon interaction in open\-world produces a mass of history, in which early error feedback and critical information can be buried by redundant context\. To address this issue, EvolvingBrain independently maintains a persistent Structured Abstractive Memory for each task, denoted asℳt\\mathcal\{M\}\_\{t\}\. Instead of directly preserving the complete interaction history, it abstracts previous interactions into structured information\.

The memory structure containstask\-relevant states, confirmed progress, unresolved feedback, ineffective attempts, current blockers, active constraints, invalidated assumptions, and retry conditions\.

The memory is initialized as empty at the beginning of each task and updated before skill retrieval in every interaction round\. Each update takes the task goal, previous memory, pending action, and latest environmental feedback as input, while using the Domain Skill as domain\-level guidance\. The memory is maintained through add, update, resolve, supersede, or no\-op operations, and each memory item is grounded in corresponding environmental evidence\. The structured memory update prompt is provided in Figure[15](https://arxiv.org/html/2610.20786#A5.F15)\.

The memory retains only textual information, while the current image is passed directly to downstream stages\. Under a fixed token budget, EvolvingBrain constructs compact stage\-specific memory views for state\-aware skill retrieval and fusion\.

Table 2:Main results on VAB\-Minecraft and DiscoveryWorld, averaged over three runs\.
#### 2\.3\.2State\-Aware Skill Fusion Strategy

The frozen Skill Brain provides reusable knowledge, but it may not match the agent’s current state\. To address this issue, EvolvingBrain dynamically fuses relevant skills, the Structured Abstractive Memory, and the current multimodal observation into temporary Active Guidance\. Active Guidance contains only the execution guidance required at the current stage\. It is not written back into the Skill Brain and can be reused, updated, or cleared as the state changes\.

At each interaction round, we prompt LLMs to determines whether skill guidance is needed based on current observation and memory\.noneindicates that no guidance is required,reuseretains the existing Active Guidance, andretrieveindicates a new knowledge gap\. Whenretrieveis selected, the system further determines whether Plan, Action, or Trap Skills are required\. Trap Skills are retrieved only when explicit evidence of failure or lack of progress exists\. The corresponding state\-aware gate prompt is provided in Figure[16](https://arxiv.org/html/2610.20786#A5.F16)\.

Plan Skills are retrieved at the level of relevant sections\. For Action and Trap Skills, EvolvingBrain first searches the graph neighbors of the current Plan Skills and then expands to the global atomic skill library if their coverage is insufficient\. The prompt is Figure[17](https://arxiv.org/html/2610.20786#A5.F17)\. The retrieved knowledge is fused according to the current state rather than directly concatenated\. The prompt is Figure[18](https://arxiv.org/html/2610.20786#A5.F18)\.

This hierarchical skill representation and adaptive retrieval mechanism provide skills at suitable levels of granularity according to the size of models at different scales, thereby improving their effectiveness in long\-horizon task execution\.

## 3Experiments

### 3\.1Experimental Setup

##### Benchmarks\.

We evaluate EvolvingBrain on VAB\-Minecraft and DiscoveryWorld, two open\-world benchmarks requiring long\-horizon multimodal interaction\. For VAB\-Minecraft, we follow its official split\. For DiscoveryWorld, we construct our own training, test, and out\-of\-domain splits\. We report task success rate and progress score\. Details are provided in Appendix[A](https://arxiv.org/html/2610.20786#A1)\.

Figure 3:Ablation results on skill components\.Figure 4:Success\-rate loss of fixed\-granularity retrieval relative to adaptive retrieval\. Lower is better\.
##### Baselines\.

We compare against direct agents based on Qwen3\.8\-27B \(xhigh\), Kimi\-K3 \(xhigh\), and GPT\-5\.6\-Sol \(high\), spanning a range of model sizes\. They use the same interface and interaction budget but no external skill library\. We also compare with MMSkills on VAB\-Minecraft; it is omitted on DiscoveryWorld because its public implementation does not provide a complete pipeline\. Details are provided in Appendix[B](https://arxiv.org/html/2610.20786#A2)\.

### 3\.2Overall Performance

Compared with direct agents, EvolvingBrain improves success rate by an average of 11\.64 percentage points on VAB\-Minecraft and 8\.44 percentage points on DiscoveryWorld, with corresponding score gains of 0\.0945 and 0\.0607\. On VAB\-Minecraft, EvolvingBrain also outperforms MMSkills by 6\.97 percentage points in success rate and 0\.0604 in average score\. EvolvingBrain with Kimi\-K3 achieves the best overall results, reaching 79\.37% success on VAB\-Minecraft and 74\.07% on DiscoveryWorld\.

A closer examination of the VAB\-Minecraft results reveals a phenomenon\. MMSkills improves success rate by 12\.17 percentage points with Qwen3\.8\-27B, but by only 0\.53 and 1\.32 points with Kimi\-K3 and GPT\-5\.6\-Sol, respectively\. In contrast, EvolvingBrain maintains gains of 9\.53 and 10\.05 points on the two stronger backbones\. These results suggest that the plan skills learned by MMSkills from successful trajectories become less complementary as backbone capability increases\. In contrast, EvolvingBrain provides additional knowledge beyond high\-level planning, and we investigate the source of this advantage through the subsequent ablations\.

### 3\.3Ablation Studies

##### Contributions of the Skill Components\.

We compare EvolvingBrain with a variant that disables Trap Skill while retaining Domain, Plan, and Action Skills\. The variant use the same adaptive retrieval mechanism\.

As shown in Figure[3](https://arxiv.org/html/2610.20786#S3.F3), the variant without Trap Skills already provides substantial gains over the direct backbone\. Restoring Trap Skill retrieval yields further average success\-rate gains of 3\.49 and 3\.37 percentage points on VAB\-Minecraft and DiscoveryWorld, respectively, together with score gains of 0\.0253 and 0\.0243\. These consistent improvements demonstrate the effectiveness of the skill components in EvolvingBrain\.

##### Capability–Granularity Alignment\.

We investigate whether the appropriate procedural\-skill granularity depends on backbone capability\. Under coarse procedural retrieval, the agent receives Domain Skills without descending to lower\-level procedural skills\. Under fine\-grained procedural retrieval, the agent is required to retrieve lower\-level Plan, Action, and Trap Skills\.

Figure 5:Self\-evolution gains and out\-of\-domain generalization\.Figure[4](https://arxiv.org/html/2610.20786#S3.F4)reveals a clear pattern\. For Qwen3\.8\-27B, restricting retrieval to coarse procedural skills reduces success rate by 6\.64 and 4\.32 percentage points on VAB\-Minecraft and DiscoveryWorld, respectively\. In contrast, forcing Kimi\-K3 to use fine\-grained procedural skills causes losses of 5\.19 and 5\.55 points\. Adaptive retrieval achieves the best performance across both benchmarks and metrics\. These results indicate that greater skill detail is not universally beneficial: less capable backbones benefit from explicit lower\-level guidance, whereas stronger backbones can better exploit compact high\-level abstractions\.

##### Effect of Self\-Evolution\.

We evaluate self\-evolution mechanism by reducing the maximum number of evolution iterations from three to one\. The single\-iteration setting performs one\-pass skill construction without subsequent refinement, while all other components remain unchanged\.

As shown in Figure[5](https://arxiv.org/html/2610.20786#S3.F5)\(a\)–\(b\), allowing up to three evolution iterations consistently improves every backbone\. Averaged across backbones, success rate increases by 3\.89 and 3\.60 percentage points on VAB\-Minecraft and DiscoveryWorld, respectively, while the corresponding score gains are 0\.0309 and 0\.0320\. These results indicate that feedback\-driven refinement improves the evolved skill library beyond one\-pass construction\.

### 3\.4Out\-of\-Domain Generalization

We construct the skill library exclusively from the six DiscoveryWorld training themes and evaluate it without modification on all 30 tasks from the two held\-out themes\. No out\-of\-domain trajectories are used for skill construction or evolution\.

As shown in Figure[5](https://arxiv.org/html/2610.20786#S3.F5)\(c\)–\(d\), EvolvingBrain consistently improves performance on unseen domains across all backbones\. It increases success rate by 10\.00, 11\.11, and 10\.00 percentage points for Qwen3\.8\-27B, Kimi\-K3, and GPT\-5\.6\-Sol, respectively, yielding an average gain of 10\.37 points and 0\.0688 in progress score\. Moreover, the average in\-domain\-to\-out\-of\-domain success\-rate degradation is smaller for EvolvingBrain than for direct agents, at 12\.42 versus 14\.36 points\. These results indicate that the learned skills transfer beyond the domains from which they were constructed\. Complete numerical results are provided in Appendix[D\.1](https://arxiv.org/html/2610.20786#A4.SS1)–[D\.4](https://arxiv.org/html/2610.20786#A4.SS4)\.

## 4Related Work

##### Open\-World Agents\.

Open\-world agents learn from demonstrations, external knowledge, world models, planning, and memory\([Baker et al\., 2022](https://arxiv.org/html/2610.20786#bib.bib13);[Lifshitz et al\., 2023](https://arxiv.org/html/2610.20786#bib.bib1);[Yang et al\., 2024](https://arxiv.org/html/2610.20786#bib.bib9);[Li et al\., 2025](https://arxiv.org/html/2610.20786#bib.bib14);[Fan et al\., 2022](https://arxiv.org/html/2610.20786#bib.bib12);[Hafner et al\., 2023](https://arxiv.org/html/2610.20786#bib.bib10);[Yuan et al\., 2023](https://arxiv.org/html/2610.20786#bib.bib11);[Wang et al\., 2024a](https://arxiv.org/html/2610.20786#bib.bib2);[Park et al\., 2025](https://arxiv.org/html/2610.20786#bib.bib3)\)\. EvolvingBrain instead converts a frozen multimodal agent’s own interactions into persistent skills without parameter updates\.

##### Experience\-Based Skill Evolution\.

Agents distill reflections, manuals, workflows, and skills from interaction\([Shinn et al\., 2023](https://arxiv.org/html/2610.20786#bib.bib15);[Zhao et al\., 2024](https://arxiv.org/html/2610.20786#bib.bib16);[Chen et al\., 2024](https://arxiv.org/html/2610.20786#bib.bib17);[Wang et al\., 2024b](https://arxiv.org/html/2610.20786#bib.bib18);[Wang et al\., 2023](https://arxiv.org/html/2610.20786#bib.bib19);[Zheng et al\., 2025](https://arxiv.org/html/2610.20786#bib.bib20);[Xia et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib21);[Yan et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib22);[Yang et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib23)\)\. MMSkills uses public trajectories, while MMG2Skill starts from external guides\([Zhang et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib4);[Che et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib5)\)\. EvolvingBrain builds a model\-specific Skill Brain from the target agent’s own trajectories\. It extracts Procedural and Problematic Patterns from both successful and failed interactions, validates Trap Skills through environmental retries, and distills them into Domain, Plan, and Action Skills\.

##### Hierarchical and State\-Aware Skills\.

Hierarchical RL motivates temporal abstraction\([Sutton et al\., 1999](https://arxiv.org/html/2610.20786#bib.bib24);[Dietterich, 2000](https://arxiv.org/html/2610.20786#bib.bib25)\); recent systems explore skill structures, multi\-granularity retrieval, and explicit execution states\([Xia et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib21);[Pu et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib27);[Hao et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib28);[Miao et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib26);[Badhe et al\., 2026](https://arxiv.org/html/2610.20786#bib.bib29)\)\. EvolvingBrain organizes Domain, Plan, Action, and Trap Skills\. Structured Abstractive Memory tracks progress and blockers, enabling state\-aware retrieval and fusion of only the skills required by the current state\.

## 5Conclusion

EvolvingBrain enables multimodal agents to self\-evolve by distilling successful and failed interactions into a hierarchical, model\-specific Skill Brain and retrieving guidance according to execution state\. Across three backbones, it consistently improves performance on VAB\-Minecraft and DiscoveryWorld and generalizes to held\-out domains\.

## Limitations

Future work will evaluate EvolvingBrain in broader real\-world settings and improve the efficiency of online skill evolution\.

## References

- Badheet al\.\(2026\)S\. Badhe, P\. Tiwari, and J\. ChungSKILL\.state: scalable long\-horizon agent skills\.arXiv preprint arXiv:2608\.26263\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px3.p1.1)\.
- Bakeret al\.\(2022\)B\. Baker, I\. Akkaya, P\. Zhokov, J\. Huizinga, J\. Tang, A\. Ecoffet, B\. Houghton, R\. Sampedro, and J\. CluneVideo pretraining \(vpt\): learning to act by watching unlabeled online videos\.Advances in Neural Information Processing Systems35,pp\. 24639–24654\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Cheet al\.\(2026\)X\. Che, J\. Xiong, Y\. Ge, X\. Lei, S\. Li, H\. Yan, H\. Li, Y\. Zhang, Z\. Bai, J\. Hao,et al\.MMG2Skill: can agents distill in\-the\-wild guides into self\-evolving skills?\.arXiv preprint arXiv:2606\.01993\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p3.1),[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2024\)M\. Chen, Y\. Li, Y\. Yang, S\. Yu, B\. Lin, and X\. HeAutomanual: constructing instruction manuals by llm agents via interactive environmental learning\.Advances in Neural Information Processing Systems37,pp\. 589–631\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Choudhury \(2025\)S\. ChoudhuryProcess reward models for llm agents: practical framework and directions\.arXiv preprint arXiv:2502\.10325\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p2.1)\.
- Dietterich \(2000\)T\. G\. DietterichHierarchical reinforcement learning with the maxq value function decomposition\.Journal of artificial intelligence research13,pp\. 227–303\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px3.p1.1)\.
- Fanet al\.\(2022\)L\. Fan, G\. Wang, Y\. Jiang, A\. Mandlekar, Y\. Yang, H\. Zhu, A\. Tang, D\. Huang, Y\. Zhu, and A\. AnandkumarMinedojo: building open\-ended embodied agents with internet\-scale knowledge\.Advances in Neural Information Processing Systems35,pp\. 18343–18362\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Hafneret al\.\(2023\)D\. Hafner, J\. Pašukonis, J\. Ba, and T\. P\. LillicrapMastering diverse domains through world models\.ArXivabs/2301\.04104\.External Links:[Link](https://api.semanticscholar.org/CorpusID:255569874)Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Haoet al\.\(2026\)Y\. Hao, J\. Cai, Q\. Zhang, Y\. Li, Z\. Zhang, C\. Shi, and C\. YangHiSkill: empowering llm agents with hierarchical skill graphs\.arXiv preprint arXiv:2607\.25853\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px3.p1.1)\.
- Jansenet al\.\(2024\)P\. Jansen, M\. Côté, T\. Khot, E\. Bransom, B\. Dalvi Mishra, B\. P\. Majumder, O\. Tafjord, and P\. ClarkDiscoveryWorld: a virtual environment for developing and evaluating automated scientific discovery agents\.InAdvances in Neural Information Processing Systems,Vol\.37\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p7.1)\.
- Kolb \(1984\)D\. A\. KolbExperiential learning: experience as the source of learning and development\.Prentice Hall\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p4.1)\.
- Liet al\.\(2025\)M\. Li, Z\. Wang, K\. He, X\. Ma, and Y\. LiangJarvis\-vla: post\-training large\-scale vision language models to play visual games with keyboards and mouse\.arXiv preprint arXiv:2503\.16365\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Lifshitzet al\.\(2023\)S\. Lifshitz, K\. Paster, H\. Chan, J\. Ba, and S\. McIlraithSteve\-1: a generative model for text\-to\-behavior in minecraft\.Advances in Neural Information Processing Systems36,pp\. 69900–69929\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p1.1),[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2025\)X\. Liu, T\. Zhang, Y\. Gu, I\. L\. Iong, Y\. Xu, X\. Song, S\. Zhang, H\. Lai, X\. Liu, H\. Zhao, J\. Sun, X\. Yang, Y\. Yang, Z\. Qi, S\. Yao, X\. Sun, S\. Cheng, Q\. Zheng, H\. Yu, H\. Zhang, W\. Hong, M\. Ding, L\. Pan, X\. Gu, A\. Zeng, Z\. Du, C\. H\. Song, Y\. Su, Y\. Dong, and J\. TangVisualAgentBench: towards large multimodal models as visual foundation agents\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p7.1)\.
- Miaoet al\.\(2026\)Y\. Miao, Z\. Yu, L\. Zhao, B\. Zhu, and H\. HaqueSkillLens: adaptive multi\-granularity skill reuse for cost\-efficient llm agents\.arXiv preprint arXiv:2605\.08386\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px3.p1.1)\.
- Parket al\.\(2025\)J\. Park, J\. Cho, and S\. AhnMrSteve: instruction\-following agents in minecraft with what\-where\-when memory\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 16922–16960\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p1.1),[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Puet al\.\(2026\)H\. Pu, X\. Song, and L\. ZhaoSkillOps: managing llm agent skill libraries as self\-maintaining software ecosystems\.arXiv preprint arXiv:2605\.13716\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px3.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning, 2023\.URL https://arxiv\. org/abs/2303\.113661,pp\. 5\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Suttonet al\.\(1999\)R\. S\. Sutton, D\. Precup, and S\. SinghBetween mdps and semi\-mdps: a framework for temporal abstraction in reinforcement learning\.Artificial intelligence112\(1\-2\),pp\. 181–211\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px3.p1.1)\.
- Uesatoet al\.\(2022\)J\. Uesato, N\. Kushman, R\. Kumar, F\. Song, N\. Siegel, L\. Wang, A\. Creswell, G\. Irving, and I\. HigginsSolving math word problems with process\-and outcome\-based feedback\.arXiv preprint arXiv:2211\.14275\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p2.1)\.
- Wanget al\.\(2023\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2024a\)Z\. Wang, S\. Cai, A\. Liu, Y\. Jin, J\. Hou, B\. Zhang, H\. Lin, Z\. He, Z\. Zheng, Y\. Yang,et al\.Jarvis\-1: open\-world multi\-task agents with memory\-augmented multimodal language models\.IEEE Transactions on Pattern Analysis and Machine Intelligence47\(3\),pp\. 1894–1907\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p1.1),[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024b\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Xiaet al\.\(2026\)P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen,et al\.Skillrl: evolving agents via recursive skill\-augmented reinforcement learning\.arXiv preprint arXiv:2602\.08234\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px3.p1.1)\.
- Yanet al\.\(2026\)Z\. Yan, D\. Song, H\. Zhang, W\. Liang, Y\. Zhang, Y\. Dai, L\. He, P\. S\. Yu, R\. Xu, X\. Li,et al\.OpenSkill: open\-world self\-evolution for llm agents\.arXiv preprint arXiv:2606\.06741\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2026\)Y\. Yang, Z\. Gong, W\. Huang, Q\. Yang, Z\. Zhou, Z\. Huang, Y\. Li, X\. Gao, Q\. Dai, B\. Liu,et al\.Skillopt: executive strategy for self\-evolving agent skills\.arXiv preprint arXiv:2605\.23904\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2024\)Y\. Yang, T\. Zhou, K\. Li, D\. Tao, L\. Li, L\. Shen, X\. He, J\. Jiang, and Y\. ShiEmbodied multi\-modal agent trained by an llm from a parallel textworld\.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 26265–26275\.External Links:[Link](https://api.semanticscholar.org/CorpusID:265466250)Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Yuanet al\.\(2023\)H\. Yuan, C\. Zhang, H\. Wang, F\. Xie, P\. Cai, H\. Dong, and Z\. LuSkill reinforcement learning and planning for open\-world long\-horizon tasks\.arXiv preprint arXiv:2303\.16563\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px1.p1.1)\.
- Zenget al\.\(2024\)A\. Zeng, M\. Liu, R\. Lu, B\. Wang, X\. Liu, Y\. Dong, and J\. TangAgenttuning: enabling generalized agent abilities for llms\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 3053–3077\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p2.1)\.
- Zhanget al\.\(2026\)K\. Zhang, S\. Shao, Q\. Li, J\. Lin, L\. Fu, S\. Wang, W\. Jiao, Y\. Lu, W\. Liu, W\. Zhang,et al\.Mmskills: towards multimodal skills for general visual agents\.arXiv preprint arXiv:2605\.13527\.Cited by:[§1](https://arxiv.org/html/2610.20786#S1.p1.1),[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpel: llm agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.
- Zhenget al\.\(2025\)B\. Zheng, M\. Y\. Fatemi, X\. Jin, Z\. Z\. Wang, A\. Gandhi, Y\. Song, Y\. Gu, J\. Srinivasa, G\. Liu, G\. Neubig,et al\.Skillweaver: web agents can self\-improve by discovering and honing skills\.arXiv preprint arXiv:2504\.07079\.Cited by:[§4](https://arxiv.org/html/2610.20786#S4.SS0.SSS0.Px2.p1.1)\.

## Appendix AExperimental Setup Details

Table 3:Our DiscoveryWorld data split\.### A\.1Benchmark Splits

##### VAB\-Minecraft\.

We follow the official VAB\-Minecraft split\. The released training trajectories cover 90 unique task goals and are used only for skill construction\. Evaluation is conducted on the 116 official test tasks, with no task overlap between the two sets\.

##### DiscoveryWorld\.

DiscoveryWorld contains eight themes, three difficulty levels \(*Easy*,*Normal*, and*Challenge*\), and five task variants indexed from 0 to 4, resulting in 120 instances\.

We use Archaeology Dating, Combinatorial Chemistry, Plant Nutrients, Proteomics, Reactor Lab, and Space Sick as the six training themes\. Across these themes and all difficulty levels, variants 0–2 form the training set, while variants 3–4 form the in\-domain test set\. The remaining two themes, Lost in Translation and It’s \(not\) Rocket Science\!, are entirely held out, with all their variants used for out\-of\-domain evaluation\.

The training set is used exclusively for skill construction and does not involve updating the parameters of the underlying models\. The resulting skill library is frozen before evaluation, and test trajectories are not used to update it\.

### A\.2Evaluation Metrics

We report task success rate and environment\-defined progress score\. ForNNevaluation tasks, success rate is computed as

Success=1N∑i=1N𝟏\[yi=1\],\\mathrm\{Success\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbf\{1\}\[y\_\{i\}=1\],\(5\)whereyiy\_\{i\}denotes whether the environment reports taskiias successfully completed\.

The average progress score is

Score=1N​∑i=1Nsi,\\mathrm\{Score\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}s\_\{i\},\(6\)wheresis\_\{i\}is determined by the native scoring function of each benchmark\.

For VAB\-Minecraft, success is determined by whether the required item and quantity are present in the agent’s inventory\. The progress score is defined as

si=\{1,if task​i​is successful,max⁡\(0,rifinal\),otherwise,s\_\{i\}=\\begin\{cases\}1,&\\text\{if task \}i\\text\{ is successful\},\\\\ \\max\(0,r\_\{i\}^\{\\mathrm\{final\}\}\),&\\text\{otherwise\},\\end\{cases\}\(7\)whererifinalr\_\{i\}^\{\\mathrm\{final\}\}is the environment reward accumulated from obtaining task\-relevant intermediate items\.

For DiscoveryWorld, success is determined by the environment’scompletedSuccessfullyflag\. Its progress score is the normalized score

si=scoreimaxScorei,s\_\{i\}=\\frac\{\\texttt\{score\}\_\{i\}\}\{\\texttt\{maxScore\}\_\{i\}\},\(8\)which measures completion of the procedural subgoals defined by the environment\. Success rate is used as the primary metric, while the progress score captures partial completion when a task is not fully solved\.

## Appendix BImplementation and Baseline Details

##### Evaluation protocol\.

Within each backbone, EvolvingBrain and its corresponding direct baseline use the same task instructions, observation and action interfaces, inference settings, and interaction budget\. The direct baseline receives no retrieved skills, while EvolvingBrain augments the same base agent with skills retrieved from its hierarchical skill library\.

We allow at most 60 agent interaction rounds on VAB\-Minecraft\. For DiscoveryWorld, we follow its difficulty\-dependent interaction limits: 100 steps for*Easy*and*Normal*tasks and 200 steps for*Challenge*tasks\. Unless otherwise specified, the skill\-evolution process is capped at three feedback iterations\.

##### Backbones\.

We evaluate direct agents based on Qwen3\.8\-27B, Kimi\-K3, and GPT\-5\.6\-Sol\. Qwen3\.8\-27B and Kimi\-K3 use thexhighinference setting, while GPT\-5\.6\-Sol uses thehighsetting\. Model parameters remain frozen throughout skill construction and evaluation\.

##### MMSkills\.

We evaluate MMSkills on VAB\-Minecraft using its publicly released skills\. We do not reconstruct its skill\-generation process\. MMSkills is not evaluated on DiscoveryWorld because its public release does not provide the complete skill\-generation and clustering pipeline required to construct skills for a new environment\. We therefore report its DiscoveryWorld results as N/A\.

## Appendix CCase Study

To make the operation of the adaptive skill system concrete, we analyze a held\-out VAB\-Minecraft instance whose goal is to obtain a piston\. We select this trajectory as a*diagnostic case*: the direct agent and EvolvingBrain diverge under the same evaluation protocol, while the EvolvingBrain trace contains several grounded state changes that exercise different levels of the Skill Brain\. The example is intended to illustrate the mechanism rather than provide statistical evidence, which is supplied by the aggregate experiments in the main paper\.

##### Controlled comparison\.

Both agents use Kimi\-K3 with the same task instruction, inference setting, observation and action interfaces, and 60\-round interaction budget\. They also begin from the same environment state: 64 units ofredstone, 16oak\_log, and position\(431\.5,65\.0,−158\.5\)\(431\.5,65\.0,\-158\.5\)\. The direct agent receives no external skill knowledge, whereas EvolvingBrain uses the frozen model\-specific Skill Brain constructed from training trajectories\. Thus, no test trajectory is used to update the library\.

![Refer to caption](https://arxiv.org/html/2610.20786v1/piston_case_study.png)Figure 6:State\-aware recovery on a held\-out VAB\-Minecraft piston task\. Both agents use Kimi\-K3 and start from the same state under a 60\-round interaction budget\. The direct agent repeatedly reformulates its search and terminates without a piston\. EvolvingBrain first retrieves Plan guidance, retrieves and fuses frozen Trap guidance after reaching bedrock without iron, refreshes its guidance after the observed inventory changes, and then reuses that guidance to craft and verify the piston\. The Skill Brain is frozen throughout evaluation; the trajectory therefore illustrates state\-aware retrieval and recovery rather than test\-time skill learning\. Here, Rkkdenotes the state observed afterkkactions, so R0 is the initial state and R27 is the terminal state after 27 actions\.
##### Direct agent: search\-reformulation loop\.

The direct agent first identifies the piston recipe and constructs the basic tool chain\. It subsequently fails to obtain iron and repeatedly changes the wording or location of its search, including ore\-, cave\-, ravine\-, village\-, and mountain\-oriented commands\. Across the episode it issues 40executecalls and eight teleports before exhausting all 60 rounds\. It does not obtain a piston and receives a VAB\-Minecraft progress score of0\.31140\.3114\.

##### EvolvingBrain: state\-aware recovery\.

EvolvingBrain completes the same task in 27 rounds\. At round 0, the absence of existing guidance and the unresolved multi\-stage dependency chain cause the Gate to retrieve Plan knowledge\. At round 11, an ore\-search action times out after moving the agent fromY=28Y=28toY=4Y=4without increasing its iron inventory\. Structured Abstractive Memory records the missing state progress and the bedrock\-level blocker, and the Gate retrieves Trap knowledge rather than allowing another unchanged mining attempt\. The memory mechanism also rejects nominally successful tool feedback when the reported inventory and coordinates do not change\.

The trajectory later acquires two units ofiron\_ore, invalidating the premise that further mining is required\. A subsequent teleport returns the agent to the surface but drops all 64 units ofredstone; the current image shows a candidate dropped item directly ahead\. Because both the task plan and the current local transition have changed, the Gate requests Plan, Action, and Trap knowledge together\. Fusion converts the selected knowledge into temporary ActiveGuidance that prioritizes bounded item recovery instead of another mining cycle\. After the stack is confirmed and recovered, the remaining guidance is still applicable and is reused through smelting and the final piston craft\. Across the 27 action rounds, four rounds invoke retrieval and construct or refresh ActiveGuidance: its initial construction at R0 and subsequent refreshes at R11, R17, and R18\. We abbreviate these four events as “refreshes” in the figure\. The remaining 23 rounds reuse the current guidance without invoking retrieval again\.

##### Offline evolution and test\-time retrieval\.

The purple callout in Figure[6](https://arxiv.org/html/2610.20786#A3.F6)summarizes the principal Trap Skill retrieved at R11; the retriever also returns complementary Trap Skills, which Fusion combines into ActiveGuidance\. This representative skill is an offline\-evolved library entry consolidated from ten source trajectories spanning nine training tasks\. Its portable safeguards are to verify position and inventory changes, stop at bedrock, and switch the prompt family after grounded evidence of no progress\. The Skill Brain is frozen before evaluation: the held\-out trajectory can retrieve and reuse this knowledge but cannot modify it\. The example does not suggest that EvolvingBrain eliminates execution errors\. Rather, it shows how grounded memory and adaptive retrieval prevent a local error from becoming a persistent failure loop\.

## Appendix DAdditional Experimental Results

### D\.1Skill\-System Ablation

Table[4](https://arxiv.org/html/2610.20786#A4.T4)reports the complete numerical results corresponding to Figure[3](https://arxiv.org/html/2610.20786#S3.F3)\. The w/o Trap Skills variant disables Trap Skill retrieval while retaining Domain, Plan, and Action Skills\. Both skill\-enabled variants use the same adaptive retrieval mechanism\.

Table 4:Ablation of Trap Skill retrieval on VAB\-Minecraft and DiscoveryWorld, averaged over three runs\.
### D\.2Adaptive Skill Retrieval

Table[5](https://arxiv.org/html/2610.20786#A4.T5)reports the complete results corresponding to Figure[4](https://arxiv.org/html/2610.20786#S3.F4)\. Coarse procedural retrieval stops at Domain Skills, whereas fine\-grained procedural retrieval requires the agent to descend to Plan and Action Skills\. Trap Skill retrieval remains unchanged in all variants\.

Table 5:Complete results for the adaptive retrieval analysis\. Trap Skill retrieval is retained in all variants, while only procedural retrieval depth is varied\.
### D\.3Effect of Self\-Evolution

Table[6](https://arxiv.org/html/2610.20786#A4.T6)reports the complete results for the self\-evolution ablation\. The maximum number of evolution iterations is varied while the skill hierarchy, retrieval mechanism, and evaluation settings remain unchanged\.

Table 6:Effect of the maximum number of self\-evolution iterations\.
### D\.4Out\-of\-Domain Generalization

Table[7](https://arxiv.org/html/2610.20786#A4.T7)compares in\-domain and out\-of\-domain performance on DiscoveryWorld\. The out\-of\-domain evaluation contains all 30 tasks from the two themes held out during skill construction\.

Table 7:In\-domain and out\-of\-domain results on DiscoveryWorld\.

## Appendix ESkill Templates and Core Prompts

This section provides the concrete skill schemas and the principal prompts used by EvolvingBrain\. The four templates instantiate the hierarchy described in the main paper: Domain Skills encode environment\-level regularities, Plan Skills organize reusable task procedures, Action Skills ground individual interaction steps, and Trap Skills record failure conditions and recovery strategies\. We then report the eight core prompts that construct, maintain, retrieve, and fuse these skills\. The displayed prompts retain the operational instructions and structured input–output constraints while omitting implementation\-only wrappers\.

### E\.1Four Skill Templates

Figures[7](https://arxiv.org/html/2610.20786#A5.F7)–[10](https://arxiv.org/html/2610.20786#A5.F10)show the canonical representations stored in the hierarchical skill library\. Together, they separate broadly reusable environment knowledge from task\-level procedures, executable actions, and failure\-aware guidance\.

### E\.2Core Prompts

The eight prompts below cover the complete skill lifecycle\. Figures[11](https://arxiv.org/html/2610.20786#A5.F11)–[14](https://arxiv.org/html/2610.20786#A5.F14)describe offline skill construction; Figure[15](https://arxiv.org/html/2610.20786#A5.F15)maintains a compact structured state during interaction; and Figures[16](https://arxiv.org/html/2610.20786#A5.F16)–[18](https://arxiv.org/html/2610.20786#A5.F18)implement adaptive online retrieval and guidance composition\.

![Refer to caption](https://arxiv.org/html/2610.20786v1/domain_skill.png)Figure 7:Domain Skill template\. Domain Skills summarize environment\-level mechanics, conventions, and reusable knowledge shared across tasks\.![Refer to caption](https://arxiv.org/html/2610.20786v1/plan_skill.png)Figure 8:Plan Skill template\. Plan Skills encode task\-oriented procedures, including prerequisites, ordered stages, and completion criteria\.![Refer to caption](https://arxiv.org/html/2610.20786v1/action.png)Figure 9:Action Skill template\. Action Skills ground reusable operations in observable preconditions, executable actions, and expected outcomes\.![Refer to caption](https://arxiv.org/html/2610.20786v1/trap_skill.png)Figure 10:Trap Skill template\. Trap Skills capture recurrent failure patterns, diagnostic cues, preventive constraints, and recovery actions\.![Refer to caption](https://arxiv.org/html/2610.20786v1/paln_skill_generation.png)Figure 11:Plan Skill generation prompt\. It abstracts successful trajectories into reusable task procedures with explicit applicability conditions and ordered stages\.![Refer to caption](https://arxiv.org/html/2610.20786v1/action_skill_generation.png)Figure 12:Action Skill generation prompt\. It converts interaction evidence into grounded operations with observable preconditions and verifiable effects\.![Refer to caption](https://arxiv.org/html/2610.20786v1/trap_skill_generation.png)Figure 13:Trap Skill generation prompt\. It extracts recurrent failure modes and records their triggers, diagnoses, avoidance rules, and recovery procedures\.![Refer to caption](https://arxiv.org/html/2610.20786v1/domain_skill_synthesis.png)Figure 14:Domain Skill synthesis prompt\. It consolidates cross\-task evidence into environment\-level knowledge while removing task\-specific details\.![Refer to caption](https://arxiv.org/html/2610.20786v1/memory_update.png)Figure 15:Structured memory update prompt\. It compresses the latest observation and action outcome into a bounded state representation used by subsequent retrieval decisions\.![Refer to caption](https://arxiv.org/html/2610.20786v1/state-aware_gate.png)Figure 16:State\-aware gate prompt\. It decides whether additional procedural or failure\-aware knowledge is needed under the current task state\.![Refer to caption](https://arxiv.org/html/2610.20786v1/skill_retrieval.png)Figure 17:Hierarchical Skill retrieval prompt\. It selects the appropriate retrieval depth and returns the most relevant Domain, Plan, Action, and Trap Skills\.![Refer to caption](https://arxiv.org/html/2610.20786v1/skill_fusion.png)Figure 18:State\-aware Skill fusion prompt\. It reconciles retrieved procedural and failure\-aware guidance into a concise, state\-conditioned instruction for the acting agent\.