Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost

# Programmatic SkillLearning Best Reduces Agent Cost
Source: [https://arxiv.org/html/2608.11338](https://arxiv.org/html/2608.11338)
## Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost

Xiheng WangAffiliation:Equal contributionAndrew WangWilliam JurayjBernal Jiménez GutiérrezDaniel KhashabiAffiliation:Corresponding authorsNicholas AndrewsJohns Hopkins University\{zhuang60, awang116, danielk, noa\}@jhu\.eduAffiliation:Corresponding authors

###### Abstract

Recently, the practice of augmenting LLM agent capability with*skills*has gained prevalence\. We explore the cost effective adaptation of agents to novel domains by means of learning skills\. Existing works focus on performance gain over cost effectiveness\. As a result, little is known about what skill learning strategies save cost\. We argue that among all the different skill learning methods, those that view skills as programs can achieve the best cost reduction\. By executing sequences of actions deterministically, a program\-augmented agent can reliably and cheaply achieve goals that would otherwise require trial and error and risk degenerate behavior over long horizons\. An agent can learn at inference time by incrementally discovering these programs and equipping them for future tasks\. We hypothesize that past trajectories contain enough signal to guide skill learning, even without replay or validation, provided the agent can learn to analyze them\. To test our claims, we proposeSpeedRunner, a coding agent that analyzes trajectories and refactors skills for better performance on future tasks\. Across three different embodied environments, we show thatSpeedRunnerconsistently achieves the frontier in learning and cost reduction while remaining robust against distribution shifts and environmental randomness\.

## 1Introduction

Figure 1:Performance vs average cost per trace for different approaches with GPT 5\.4\-mini on 3 benchmarks using learned programmatic skills\.SpeedRunnerconsistently dominates baselines in cost and performance, highlighting the potential advantages of programmatic skills\.As LLM agents become more prevalent, they are increasingly deployed in complex real world environments\. For instance, such agents now run a real world coffee shop in Stockholm\.111[https://andonlabs\.com/blog/ai\-cafe\-stockholm](https://andonlabs.com/blog/ai-cafe-stockholm)Such deployments rarely present a fixed task distribution: new menu items and unfamiliar requests accumulate over time, so a static policy degrades unless the agent can adapt on the job\. In these settings, we expect agents to learn*online*from their environment without catastrophically forgetting their pre\-existing knowledge\. We view*skill learning*as a promising paradigm\. Here an agent reflects on its trajectories and updates its policy with environment\-specific capabilities called*skills*\(§[2](https://arxiv.org/html/2608.11338#S2)\)\. The manner in which the policy is updated, the representation of skills, and the degree of online vs offline learning, vary between methods\.

The predominant representation of skills are natural language descriptions\([40](https://arxiv.org/html/2608.11338#bib.bib26);[16](https://arxiv.org/html/2608.11338#bib.bib28);[17](https://arxiv.org/html/2608.11338#bib.bib21);[29](https://arxiv.org/html/2608.11338#bib.bib29);[35](https://arxiv.org/html/2608.11338#bib.bib30)\), following the release of Anthropic’s agent skills \. However, this shift has overlooked a key consideration for the average user:cost\. Existing works focus on performance gains, with cost savings as an afterthought\. Thus the effect of different skill representation and learning strategies on cost remains unknown\.We argue that the modality of code can be a more cost effective way to represent skills, compared to natural language\.Rather than reasoning about the same routine over and over again, an agent learns to offload that reasoning to a cheaper Turing machine\. Hence, the more reasoning successfully translated into code, the more cost saved\.

However, the key question is how well agents can learn programmatic skills from experience, and then successfully use them to accomplish tasks\. Existing approaches fall short in multiple ways\. First, the prevailing method of learning from experience has been to prompt an LLM with past trajectories\. This method is critically bottlenecked by trajectory length; the number that fit in context decreases as they grow longer, until none fit at all\. Second, unrealistic assumptions are made to facilitate learning, the most glaring of which assumes the ability to manipulate the environment to the extent of undoing and rewinding states\. This often manifests as a set of held\-out episodes used to gather heuristics for hillclimbing that are replayed repeatedly\. Together, these shortcomings prevent the practical deployment of programmatic skill learning, especially in multi\-turn agentic settings\.

We hypothesize that the agent trajectories themselves are signal rich, and that simply analyzing them properly resolves these problems\.At a high level, we frame the process of extracting signal as an agentic coding task\.Given a history of trajectories, an agent with access to a code execution environment can programmatically identify recurring behavior patterns, skill usage and success metrics, performance regression, among others\. Unlike natural language feedback\([1](https://arxiv.org/html/2608.11338#bib.bib9);[19](https://arxiv.org/html/2608.11338#bib.bib10)\)or handcrafted tools\([25](https://arxiv.org/html/2608.11338#bib.bib6)\), a coding agent can avoid needle\-in\-a\-haystack retrieval over long trace histories, support more structured cross\-trajectory analysis, and mitigate the context\-window bottleneck that arises in long\-horizon environments with 200\-step solutions\([9](https://arxiv.org/html/2608.11338#bib.bib13)\)\.

To test our hypothesis, we proposeSpeedRunner, a simple yet effective way to induce programmatic skills using coding agents to analyze observed trajectories\. Our goal is to optimize a stochastic policy parameterized by a library of skills, which, in practice, is an LLM agent equipped with a set of callable functions\. The search for optimal skills cycles between two phases\. The stochastic policy receives tasks and generates trajectories\. Then, based on the trajectories,SpeedRunnerupdates the skills\.

We evaluateSpeedRunneron ScienceWorld\([31](https://arxiv.org/html/2608.11338#bib.bib12)\), Crafter\([9](https://arxiv.org/html/2608.11338#bib.bib13)\), and BabyAI\([4](https://arxiv.org/html/2608.11338#bib.bib14)\)—a diverse set of embodied simulations\. Each domain features vastly different dynamics, physical constraints, and action spaces\. Nevertheless, we find consistent trends across all domains\. \(1\) Programmatic skills can be learned online, without replay buffers or continual validation\. \(2\) Their primary advantage is cost reduction without compromising performance\. \(3\) Programmatic trajectory analysis enables skill induction for long\-horizon agents\. We also find that programmatic skill learning can be robust to environmental randomness and to distribution shifts\.

## 2Preliminaries

Problem formulation\.We assume a partially observed Markov decision process \(POMDP\) and a set of valid initial states, where task*instructions*if available are part of the initial state\. The POMDP we refer to as the*environment*and the set of valid initial states as*tasks*\(𝒯\\mathcal\{T\}\)\. Tasks are sampled one by one for a stochastic policyπ\\pito complete\. A rollout results in a sequence of observations and actions ending in task reward\(o1,a1,o2,a2,…,oT,aT,R\)\(o\_\{1\},a\_\{1\},o\_\{2\},a\_\{2\},\.\.\.,o\_\{T\},a\_\{T\},R\)\. We refer to this tuple as a*trajectory*\(τ\\tau\)\.

Pure Online\-ness\.In order to accommodate real world settings, we aim to require as little as possible from the environment\. Therefore, we do not assume the ability to undo and redo environment actions, as many settings such as customer service or high frequency trading preclude such possibility\. This decision leaves us with “purely\-online” learning, where tasks are ephemeral and cannot be played again with an updated policy\. All approaches and baselines we evaluate are made to conform to this standard\.

Defining skills\.Prior works have treated skills as prompt strategies\([34](https://arxiv.org/html/2608.11338#bib.bib1)\), executable functions\([25](https://arxiv.org/html/2608.11338#bib.bib6);[33](https://arxiv.org/html/2608.11338#bib.bib7);[39](https://arxiv.org/html/2608.11338#bib.bib8)\), or a mixture of both\.222[https://platform\.claude\.com/docs/en/agents\-and\-tools/agent\-skills/overview](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview)We focus on executable functions because code is expressive, precise, reliably executed, and structurally analyzable through call graphs and abstract syntax trees\. We define a*skill*as an executable function paired with documentation, operating on the atomic actions defined by the POMDP or other existing skills\. We refer to*skill*when we mean a function and its documentation, and*function*when we mean the code alone\.

## 3SpeedRunner: Effective Trajectory Analysis With Coding Agents

![Refer to caption](https://arxiv.org/html/2608.11338v1/methods-figure2.png)Figure 2:Overview ofSpeedRunner\. A stochastic policyπL\{\\color\[rgb\]\{0,0\.4453,0\.6992\}\\pi\}\_\{\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\}\(the actor\) is equipped with a skill libraryL\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}, while a proposal functiong\{\\color\[rgb\]\{0\.4922,0\.2383,0\.4063\}g\}\(the inducer\) updates the library using trajectory historyH\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\. During the wake phase, the actor generates a batch of trajectories\. During the sleep phase, the inducer examines the trajectories and edits the library\. In this example, the inducer notices repeated “not found” responses fromtravel\_to\_resource, checks the function’s usage frequency, and introduces\_bounded\_exploration, which searches the actor’s local neighborhood for the desired resource\.We parameterize a stochastic policy by askill libraryL\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}, writing the resulting policy asπL\{\\color\[rgb\]\{0,0\.4453,0\.6992\}\\pi\}\_\{\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\}\. Learning alternates between two phases\. During the*wake*phase, theactorπL\{\\color\[rgb\]\{0,0\.4453,0\.6992\}\\pi\}\_\{\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\}generates a batch of trajectoriesBB, which is appended to thehistoryH\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\. During the*sleep*phase, theinducerg:L×H→L\{\\color\[rgb\]\{0\.4922,0\.2383,0\.4063\}g\}:\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\\times\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\\rightarrow\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}edits the library\. In our experiments, the actor is an LLM agent equipped with callable skills, and the inducer is a coding agent\.

SpeedRunnerfocuses on the inducer: how shouldg\{\\color\[rgb\]\{0\.4922,0\.2383,0\.4063\}g\}extract learning signal fromH\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}? Trajectory\-level rewards are available but too sparse to reliably guide library edits from only a few online episodes\. Richer signal lies inside trajectories as*environment responses*: errors, unexpected outcomes, and other feedback from the actor’s actions\. These signals are unstructured and often buried in long\-horizon traces, making full\-context reading expensive and summarization lossy\.

Algorithm 1SpeedRunner: Wake–Sleep Skill Library Learning1:initial library

L0\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\_\{0\}\}, actor

πL\{\\color\[rgb\]\{0,0\.4453,0\.6992\}\\pi\}\_\{\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\}, inducer

g:L×H→L\{\\color\[rgb\]\{0\.4922,0\.2383,0\.4063\}g\}:\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\\times\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\\to\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}, environment

EE, task distribution

𝒯\\mathcal\{T\}, total episodes

NN, minibatch size

kk
2:

H←∅\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\\leftarrow\\emptyset
3:

L←L0\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\\leftarrow\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\_\{0\}\}
4:for

n=1,…,Nn=1,\\dots,Nby

kkdo

5:

⊳\\trianglerightwake phase

6:sample tasks

\{x1,…,xk\}∼𝒯\\\{x\_\{1\},\\dots,x\_\{k\}\\\}\\sim\\mathcal\{T\}
7:for all

xi∈\{x1,…,xk\}x\_\{i\}\\in\\\{x\_\{1\},\\dots,x\_\{k\}\\\}do

8:

τi∼πL​\(xi\)\\tau\_\{i\}\\sim\{\\color\[rgb\]\{0,0\.4453,0\.6992\}\\pi\}\_\{\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\}\(x\_\{i\}\)⊳\\trianglerightrollout in

EEstarting from task

xix\_\{i\}
9:endfor

10:

B←\{τ1,…,τk\}B\\leftarrow\\\{\\tau\_\{1\},\\dots,\\tau\_\{k\}\\\}
11:

H←H∪\{\(B,L\)\}\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\\leftarrow\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\\cup\\\{\(B,\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\)\\\}⊳\\trianglerighttag trajectories with the generating library

12:

⊳\\trianglerightsleep phase

13:

L←g⁡\(L,H\)\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}\\leftarrow\{\\color\[rgb\]\{0\.4922,0\.2383,0\.4063\}g\}\(\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\},\{\\color\[rgb\]\{0\.6523,0\.3555,0\.0938\}H\}\)⊳\\trianglerightinducer inspects trajectories and past libraries in

HH; diagnoses failures/regressions; edits skills; sets public/private visibility

14:endfor

15:return

L\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}

Intuition\.The solution is to make the inducer a coding agent\. A coding agent has access to a code execution environment and can add, edit, and delete library skills\. If trajectories are persisted to memory or disk, the inducer can inspect failures, aggregate statistics across episodes, and test hypotheses about recurring patterns\. This lets it analyze traces surgically without placing whole trajectories in context\. Because the raw trajectories are still saved, this avoids the signal loss introduced by summary\-based compression\.

Harness\.The inducer acts as a programmer examining history and deciding what logic to implement\. It must understand the environment, diagnose failures, assign credit to skills, and check for regressions\. To support these analyses, we store both trajectories and past library versions\. Each trajectory is indexed by theL\{\\color\[rgb\]\{0\.1484,0\.4375,0\.3789\}L\}version that generated it \(line 10 of Algorithm[1](https://arxiv.org/html/2608.11338#alg1)\), allowing the inducerg\{\\color\[rgb\]\{0\.4922,0\.2383,0\.4063\}g\}to analyze how functions evolve or regress over time\. We also augment trajectories with call stacks for invoked skills: when a skill expands into a sequence of primitive actions, we record those intermediate actions and their outputs\. These call stacks help diagnose silent skill failures\.

Finally, we control library size\. If the library grows without bound, the actor considers too many functions in context and performance degrades\([26](https://arxiv.org/html/2608.11338#bib.bib11)\)\. We therefore introduce public and private access modifiers\. The inducer is instructed to make helper functions private and main functions public; private functions remain callable by other skills but are hidden from the actor\.

## 4Experimental Setup

### 4\.1Baselines

We compare four methods that differ in how they accumulate experience—none, prose, or code—while standardizing the primitive actions available to both the actor and inducer across all baselines\.

ReActThis baseline serves as a control with no learning between episodes\. The actor uses only the benchmark’s atomic action interface and whatever context is available within the current episode\([36](https://arxiv.org/html/2608.11338#bib.bib15)\); it does not update any persistent strategy notes or reusable skills from past experience\.

Online Prompt Optimization \(OPO\)\.We implement an online version of prompt optimization methods such as[1](https://arxiv.org/html/2608.11338#bib.bib9);[37](https://arxiv.org/html/2608.11338#bib.bib18)\. After each training batch, a sleep\-phase LLM reviews a concatenation of recent trajectories and updates the actor prompt\.

ASI\.Where supported, we also compare against Agent Skill Induction \(ASI\)\([33](https://arxiv.org/html/2608.11338#bib.bib7)\), a recent code\-based baseline that induces skills from successful trajectories\. UnlikeSpeedRunner, ASI is append\-only and verifies newly induced skills through replay\-style checking on the inducing task instance\. To satisfy our no\-replay constraint, we remove this verification step\. Appendix[E](https://arxiv.org/html/2608.11338#A5)details this adaptation, and Appendix[G\.3](https://arxiv.org/html/2608.11338#A7.SS3)reports an ablation of its effect\.

Voyager\.For Crafter, we additionally compare against Voyager\([28](https://arxiv.org/html/2608.11338#bib.bib3)\), a code\-based lifelong\-learning agent that learns through an automatic curriculum and an append\-only executable skill library\. We evaluate Voyager only on Crafter because our online protocol removes the replay\-and\-retry behavior available in Voyager’s original setting: once a rollout ends, the agent cannot reset to the same state, preserve inventory or other environment progress, and retry the same attempted skill\. Under this constraint, Voyager’s curriculum is well defined only when the agent can continue pursuing a relatively stable objective across rollouts\.

Crafter satisfies this requirement\. Although each rollout starts in a new world, the high\-level objective remains fixed: make progress in the survival environment\. Voyager can therefore continue proposing and refining intermediate exploration goals across rollouts without the target distribution changing after every episode\. ScienceWorld and BabyAI do not have this property\. Their goals are externally specified and vary from episode to episode, so Voyager’s curriculum would be optimizing against a moving target\. Evaluating Voyager on these benchmarks would require additional modification on the benchmark itself, making the comparison less faithful to the original algorithm\. We therefore report Voyager only on Crafter, where a small adaptation is sufficient to fit our online setting; details are provided in Appendix[F](https://arxiv.org/html/2608.11338#A6)\.

### 4\.2Selected Benchmarks

We evaluate on three text\-based embodied benchmarks—ScienceWorld, BabyAI, and Crafter\. Each defines its own physics, action space, and procedural demands, so success across all three is stronger evidence of generality than success across benchmarks that share a single underlying medium \(e\.g\., compute\-use benchmarks where the primitives reduce to file and terminal operations regardless of the task\)\. We select subtasks where ReAct is neither trivial nor saturated, allowing us to isolate gains from skill induction\.

#### 4\.2\.1ScienceWorld

ScienceWorld\([31](https://arxiv.org/html/2608.11338#bib.bib12)\)is a text\-based interactive environment grounded in elementary school science curricula with 10 task categories and 25 typed atomic actions \(full list in Appendix[B](https://arxiv.org/html/2608.11338#A2)\)\. An episode succeeds when the simulator score reaches1\.01\.0; partial scores are binarized to failure\.

Since skill reuse happens within category, we evaluate one category at a time\. We chooseElectricity\(task 3\) andClassification\(task 4\) because they are \(a\) solvable but not saturated by the out\-of\-the\-box agent and \(b\) have enough tasks to measure learning over time\.

#### 4\.2\.2BabyAI

BabyAI\([4](https://arxiv.org/html/2608.11338#bib.bib14)\)is a procedurally generated 2D grid\-world for instruction following under partial observability\. We use the BabyAI\-Text wrapper\([3](https://arxiv.org/html/2608.11338#bib.bib19)\)as integrated in BALROG\([18](https://arxiv.org/html/2608.11338#bib.bib20)\), which surfaces text observations\. The agent acts through 6 primitives; episodes are scored as binary success\.

Of BALROG’s 5 subtasks, we evaluate onpick\_up\_seq\_go\_to, which requires sequentially picking up an object and navigating to a goal in the correct order\. The ReAct baseline already saturates the other four, leaving no headroom to measure gains from skill induction\. We learn from 200 training episodes \(full details in Appendix[C](https://arxiv.org/html/2608.11338#A3)\)\.

#### 4\.2\.3Crafter

Crafter\([9](https://arxiv.org/html/2608.11338#bib.bib13)\)is an open\-ended survival benchmark inspired by Minecraft, where agents gather resources, craft tools, and survive hazards in procedurally generated worlds\. We expose 18 primitive actions and score each episode by the fraction of 22 predefined achievements unlocked before death or the 2,000\-action budget\. Unlike the BALROG\([18](https://arxiv.org/html/2608.11338#bib.bib20)\)text wrapper, we expose full episode history and a richer textual view of the surroundings \(full details in Appendix[D](https://arxiv.org/html/2608.11338#A4)\)\.

### 4\.3Evaluation Setup

All experiments usegpt\-5\.4\-minifor both the actor and inducer\. Each run consists of200200online rollouts with a sleep cycle every1010rollouts\. We impose a 30\-minute wall\-clock limit on each rollout to prevent nonterminating or pathologically long executions, such as loops inside generated skills\. When the limit is reached, the rollout is terminated and scored using the environment state at termination; its token usage and partial task progress are retained in the reported metrics\. All settings are repeated across33random seeds; we report means and±1\\pm 1standard deviation across seed\-level results to reflect run\-to\-run variability\. A fixed held\-out test set of3030episodes per benchmark is shared across all methods and checkpoints; evaluation is performed every5050training rollouts\. We evaluate along two axes: task progression \(benchmark\-specific success or achievement rate\) and efficiency \(output tokens per episode\)\. Full implementation details are provided in Appendix[A](https://arxiv.org/html/2608.11338#A1)\.

## 5Results

Figure 3:Performance and output\-token cost across Crafter, ScienceWorld, and BabyAI\. Our method and all baselines use GPT\-5\.4\-mini as their backbone LLM\. When the library overflows the model context limit, as happened for ASI, OPO and Voyager in Crafter, the sleep phase produces no changes and the process continues\.SpeedRunnersignificantly outperforms all baselines \(§[4\.1](https://arxiv.org/html/2608.11338#S4.SS1)\) across benchmarks in terms of performance and cost except for OPO in ScienceWorld on performance, per two\-sided paired t\-tests\.Across all three benchmarks,SpeedRunnerimproves the performance–cost tradeoff of online skill induction\. The clearest and most consistent effect is efficiency:SpeedRunnersubstantially reduces cost on every benchmark\. Performance also improves, but less uniformly, suggesting that the value of executable skill induction depends on how much reusable behavior the environment exposes\. All results below use GPT\-5\.4\-mini, however, our cross\-model experiments using Gemini\-3\-Flash and Qwen\-3\.5\-27B in Appendix[G\.2](https://arxiv.org/html/2608.11338#A7.SS2)reveal that our method’s performance–cost tradeoff improvements hold in most settings\.

Performance\.SpeedRunnerachieves the strongest final performance on all three benchmarks, with the largest margins in domains where learned routines transfer reliably across episodes\. On BabyAI,SpeedRunnerclimbs from the∼\\sim67%67\\%ReAct baseline to near\-perfect performance, a regime no other method reaches\. On ScienceWorld,SpeedRunnerties OPO at the top and substantially outperforms ASI\. Crafter is the harder case \(every method ends below30%30\\%mean progression\) butSpeedRunnerstill leads\. Appendix[G\.4](https://arxiv.org/html/2608.11338#A7.SS4)discusses why the magnitude of these gains varies with benchmark structure\.

Efficiency\.SpeedRunneris the only method whose cost*decreases*over training, and the reduction holds across all three benchmarks — most dramatically on BabyAI, where final usage falls to roughly an eighth of the ReAct baseline\. This drop indicates that the library compresses repeated behavior into callable routines: the actor invokes high\-level skills rather than re\-deriving primitive sequences\. ASI moves in the opposite direction on every benchmark, with cost growing as the library accumulates\. OPO stays close to the ReAct baseline on ScienceWorld and BabyAI but grows substantially on Crafter, suggesting that prose memory cannot compress recurring patterns into reusable structure when the environment is high\-entropy\.

Figure 4:AblatingSpeedRunnercomponents on the BabyAI environment\. Individual components have only small effects onSpeedRunner’s performance, however, removing the code interpreter degrades efficiency considerably\.
## 6Discussion

### 6\.1Ablation Study

We ablate three components ofSpeedRunneron BabyAI: the goal\-oriented inducer prompt, the public/private function split, and the inducer’s code interpreter\. The first ablation uses an ASI\-style prompt that abstracts observed behavior rather than directly optimizing task solution; the second exposes all helper functions to the actor; the third replaces code\-assisted trace analysis with in\-context trajectory inspection\. Due to experiment cost, full ablations are limited to BabyAI\. Further discussion on the value of the code interpreter across environments can be found in Appendix[G\.1](https://arxiv.org/html/2608.11338#A7.SS1)\.

Figure[4](https://arxiv.org/html/2608.11338#S5.F4)shows that all three ablations reduce performance, increase token usage, and introduce instability\. Removing the code interpreter has the largest efficiency cost, suggesting that programmatic trace analysis is especially important for compression\.

### 6\.2Code Complexity Analysis

#### 6\.2\.1Quantitative Analysis

We summarize the main quantitative findings here and defer the full structural statistics to Appendix[H](https://arxiv.org/html/2608.11338#A8)\. For each final codebook, we construct a directed call graphG=\(V,E\)G=\(V,E\), where nodes are induced functions and edges denote function calls\. We focus on two call\-graph measures in Table[1](https://arxiv.org/html/2608.11338#S6.T1): maximum depth and density\. Maximum depth is the length of the longest directed path inGG, measuring the deepest abstraction chain in the library\. Density is\|E\|/\(\|V\|​\(\|V\|−1\)\)\|E\|/\(\|V\|\(\|V\|\-1\)\), excluding self\-edges, and measures how often induced functions call one another rather than only primitive actions\.

##### Deeper and denser call graphs\.

Table[1](https://arxiv.org/html/2608.11338#S6.T1)shows thatSpeedRunnerlearns more structurally organized codebooks than the baselines\. Across benchmark settings,SpeedRunnergenerally produces deeper and denser call graphs than ASI\. This indicates that its learned skills are not merely independent wrappers around primitive actions, but are arranged into reusable hierarchical routines\.

##### Reuse rather than accumulation\.

The contrast is especially clear on Crafter, where all methods are available\. Voyager reaches maximum depth5\.75\.7, but its density is only0\.00050\.0005, indicating little reuse among induced functions\. In comparison,SpeedRunnerreaches greater maximum depth \(8\.78\.7\) and much higher density \(0\.1490\.149\)\. Thus,SpeedRunnerdoes not improve by accumulating many scenario\-specific functions; it factors repeated behavior into shared abstractions that later skills can call\.

Overall, the quantitative signature ofSpeedRunneris compositional organization: its functions call one another more often and form deeper abstraction hierarchies\.

Table 1:Main call\-graph statistics for final skill libraries, using GPT\-5\.4\-mini with mean±\\pmsd over three seeds\. The ScienceWorld column averages task 3 and task 4; reported means and standard deviations are averaged across the two tasks\. We report maximum call\-graph depth \(Max d\.\) and call\-graph density\.SpeedRunnerlearns deeper and denser call graphs\.

#### 6\.2\.2Qualitative Analysis

To understand whereSpeedRunner’s gains come from, we now qualitatively compare our best\-trial libraries against those of our baselines and ablations\. Two patterns recur across all four environments; we summarize them here and defer per\-environment evidence to Appendix[I](https://arxiv.org/html/2608.11338#A9)\.

Code\-enabled trace filtering\.The inducer’s code interpreter lets it query specific slices of actor trajectories before committing a function\. In BabyAI, candidate mission parsers were executed against historical mission strings to reveal that reversed\-order phrasings \(*“go to X after you pick up Y”*\) invert execution order—a failure mode that the ablation’s LLM\-only inducer misses on 6\.7% of episodes\. In Crafter, the inducer noticed a high rate ofnot\_foundreturns fromtravel\_to\_visible\_resource, queried trace history to confirm the function was being called hundreds of times per batch while still failing, and introduced\_bounded\_explorationto sweep the local neighborhood for the desired resource before giving up\. Without this loop, fixes accrue reactively one failure at a time\.

Higher\-level skills\.The code interpreter enables abstractions that span high\-level goals rather than a single primitive, a category of function essentially unique toSpeedRunner\. ASI and Voyager produce primitives templated from individual traces \(fixed motion sequences, per\-resource collectors\), with no high level goal logic\. In contrast,SpeedRunnergenerates functions such assolve\_sequential\_mission\(BabyAI\),\_find\_visible\_name\_candidates\(ScienceWorld\), and\_choose\_progress\_target\(Crafter\), each encoding a complex decision hierarchy in one place\. The actor can often complete episodes with one or two high\-level calls, explainingSpeedRunner’s 2–8×\\timestoken reduction over ASI\.

### 6\.3Effect of External Randomness \(Crafter\)

A central concern with code\-based skill induction is that induced functions are*deterministic*by construction and therefore potentially brittle to environmental noise: each function commits to a fixed sequence of primitive actions, while real environments rarely repeat themselves exactly\. We refer to this as the*external randomness gap*\. Crafter lets us isolate this factor by varying the zombie spawn frequency —0x\(no zombies\),1x\(standard\),2x\(doubled\) — while holding all other mechanics fixed\.

Figure 5:Crafter final performance vs*%Combat*, the fraction of induced functions whose body references a creature/combat keyword under different zombie settings \(GPT\-5\.4\-mini, 200 training rollouts\)\.Figure[5](https://arxiv.org/html/2608.11338#S6.F5)\(bars\) evaluates the robustness of all three methods to external randomness, measured against the ReAct baseline\. Prose memory \(OPO\) remains useful under low randomness, but collapses once the environment becomes sufficiently stochastic\. Append\-only code \(Voyager\) shows a similar trend\. In contrast, onlySpeedRunnerimproves over the baseline in every condition\.

The %Combat overlay \(Figure[5](https://arxiv.org/html/2608.11338#S6.F5), dotted lines\) characterizes the learned codebooks\. As zombie pressure increases,SpeedRunnerdoes not simply expand its library; it reallocates representational budget based on the noisy signal, becoming deeper and more combat\-aware by revising existing skills in place\. This allows a single defensive abstraction to be amortized across the call graph\. Voyager, by contrast, repeatedly reintroduces the same combat logic as duplicated branches inside newly generated, encounter\-specific functions, without consolidating it into a general abstraction \(Appendix[J](https://arxiv.org/html/2608.11338#A10)gives the detailed per\-condition structural breakdown and examples\)\.

Together, these results suggest thatSpeedRunneris better able to extract useful learning signal from stochastic traces\. Rather than treating external randomness as irreducible noise, it identifies recurring failure modes and quickly incorporates the corresponding adaptations into reusable abstractions\.

### 6\.4Adaptation under Distribution Shift \(ScienceWorld\)

Figure 6:ScienceWorld distribution shift: performance \(left\) and cost \(right\) when switching from task 3 to task 4 midway through training\.SpeedRunnerachieves the best performance and strongest token compression across both tasks\.Agents deployed in long\-running environments must continually adapt to changing task distributions\. A central challenge is to acquire new capabilities while retaining previously acquired ones\. We study this setting by training on ScienceWorld task 3 \(Electricity\) for 200 rollouts before switching to task 4 \(Classification\), carrying the library over unchanged \(Figure[6](https://arxiv.org/html/2608.11338#S6.F6); GPT\-5\.4\-mini, three seeds\)\.

At the distribution shift, all methods’ performance and cost approach ReAct’s behavior because the two tasks require largely disjoint procedural knowledge\. Throughout learning,SpeedRunnerachieves the best performance and strongest token compression on both tasks\. By the end of task 4\. it refines a compact library of approximately 40 functions, compared to ASI which accumulates approximately 160 functions and plateaus at lower performance\.

To evaluate knowledge retention, we re\-evaluate task 3 after task 4 training\. Despite actively revising its library,SpeedRunnerloses only 3\.3 percentage points on task 3 while reducing inference cost by 27\.1%, making it the only method that becomes more efficient on the original task after adapting to a new distribution\. In contrast, ASI improves task 3 performance by 4\.4 percentage points but increases inference cost by 40\.6% because of its append\-only nature: it preserves all learned knowledge, at the cost of unbounded library growth\. OPO loses 5\.6 percentage points on task 3 while increasing inference cost by 6\.8%, performing worse thanSpeedRunnerin both retention and efficiency\.

## 7Related Work

Our work builds on the broader literature on continual learning\([11](https://arxiv.org/html/2608.11338#bib.bib31);[12](https://arxiv.org/html/2608.11338#bib.bib32);[30](https://arxiv.org/html/2608.11338#bib.bib33);[22](https://arxiv.org/html/2608.11338#bib.bib34)\)and self\-evolving agents\([7](https://arxiv.org/html/2608.11338#bib.bib38);[8](https://arxiv.org/html/2608.11338#bib.bib37)\), which aim to build agents that improve continuously through interaction with their environment\. Existing approaches broadly follow two lines: adapting the model’s weights\([21](https://arxiv.org/html/2608.11338#bib.bib27);[24](https://arxiv.org/html/2608.11338#bib.bib39);[32](https://arxiv.org/html/2608.11338#bib.bib40)\)or augmenting a frozen model with externally stored knowledge\([23](https://arxiv.org/html/2608.11338#bib.bib35);[28](https://arxiv.org/html/2608.11338#bib.bib3);[38](https://arxiv.org/html/2608.11338#bib.bib36)\)\. We study the latter, specifically*skill learning*, which distills experience into reusable skills loaded into context to help complete a task\. Prior skill\-learning methods primarily aim to improve task performance and adaptation, while treating inference cost as a secondary outcome\. We instead study skill learning through the lens of inference efficiency, where the central design choice is how skills are represented\.

Most methods represent skills in natural language\. A line of work distills trajectories into reusable natural\-language skill documents temporarily loaded into context when relevant\([38](https://arxiv.org/html/2608.11338#bib.bib36);[34](https://arxiv.org/html/2608.11338#bib.bib1);[14](https://arxiv.org/html/2608.11338#bib.bib2);[27](https://arxiv.org/html/2608.11338#bib.bib23);[13](https://arxiv.org/html/2608.11338#bib.bib25);[17](https://arxiv.org/html/2608.11338#bib.bib21);[2](https://arxiv.org/html/2608.11338#bib.bib24)\)\. Some methods adopt skill folders which bundle textual instructions and helper scripts\([13](https://arxiv.org/html/2608.11338#bib.bib25);[2](https://arxiv.org/html/2608.11338#bib.bib24)\), but the primary optimization target remains the natural\-language artifact\. While natural\-language skills are flexible and generalize well, the agent must reread and re\-follow the instructions on every reuse, so recurring procedures keep incurring inference cost\.

The second line of work represents reusable skills as executable code\. Foundational works used Bayesian program induction to synthesize formal representations through bottom\-up composition of primitives\([5](https://arxiv.org/html/2608.11338#bib.bib4);[6](https://arxiv.org/html/2608.11338#bib.bib5)\)\. Recent work extends this paradigm to LLM agents: Voyager grows a skill library through open\-ended Minecraft exploration\([28](https://arxiv.org/html/2608.11338#bib.bib3)\), while a wave of systems induce and refine skills from agent trajectories\([25](https://arxiv.org/html/2608.11338#bib.bib6);[33](https://arxiv.org/html/2608.11338#bib.bib7);[39](https://arxiv.org/html/2608.11338#bib.bib8);[20](https://arxiv.org/html/2608.11338#bib.bib22)\)\. Programmatic skills replace repeated reasoning with delegated computation and thus provide a more efficient representation for recurring procedures\. However, previous work primarily studies programmatic skills to improve agent capability\. Many methods further rely on experience replay for validating candidate skills to guard against regression, yet these assumptions are difficult to satisfy in deployed environments where interactions are irreversible\.

Our work similarly represents skills as executable code but differs in both objective and methodology\. Rather than treating programmatic skills primarily as a means of improving capability, we highlight their capacity to minimize inference cost by maximizing reusable computation\. We further show that such skills can be induced entirely online without replay by keeping the accumulated trajectories available as a growing dataset for programmatic analysis, enabling continual refinement of the skill library while mitigating regression\.

## 8Conclusion

In this work, we demonstrate that representing skills as code can achieve the best cost effectiveness while retaining performance\. We observe that this trend is remarkably consistent across distribution shifts, randomness, different domains, and even different models\. We further demonstrate that this trend exists in a purely\-online setting, without a held\-out set of episodes to evaluate skill update quality for hillclimbing\. Not all programmatic skill learning approaches reduce cost to the same extent\. We are only able to do so by framing the process of extracting signal from past trajectories as an agentic coding task\.

LimitationsWe were unable to benchmark flagship models due to cost limitations\. For the same reason, we were unable to evaluate on more expensive benchmarks such as SWE\-Bench\([10](https://arxiv.org/html/2608.11338#bib.bib16)\)or Terminal\-Bench\([15](https://arxiv.org/html/2608.11338#bib.bib17)\)\. Nevertheless, we believe that the diversity and controllability of our evaluations are sufficient to support our general conclusions\. Skill learning at test time suffers from instability, as demonstrated by the large error bars across all methods we evaluated; stabilizing online skill induction remains an open problem\. Lastly, we do not study meta\-learning of the inducer itself—in principle, the inducer could learn to perform better analyses over time, but we leave this to future work\.

## References

- Agrawalet al\.\(2026\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. G\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. KhattabGEPA: reflective prompt evolution can outperform reinforcement learning\.External Links:2507\.19457,[Link](https://arxiv.org/abs/2507.19457)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p4.1),[§4\.1](https://arxiv.org/html/2608.11338#S4.SS1.p3.1)\.
- Alzubiet al\.\(2026\)S\. Alzubi, N\. Provenzano, J\. Bingham, W\. Chen, and T\. VuEvoSkill: automated skill discovery for multi\-agent systems\.External Links:2603\.02766,[Link](https://arxiv.org/abs/2603.02766)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p2.1)\.
- Cartaet al\.\(2023\)T\. Carta, C\. Romac, T\. Wolf, S\. Lamprier, O\. Sigaud, and P\. OudeyerGrounding large language models in interactive environments with online reinforcement learning\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.Cited by:[§4\.2\.2](https://arxiv.org/html/2608.11338#S4.SS2.SSS2.p1.1)\.
- Chevalier\-Boisvertet al\.\(2019\)M\. Chevalier\-Boisvert, D\. Bahdanau, S\. Lahlou, L\. Willems, C\. Saharia, T\. H\. Nguyen, and Y\. BengioBabyAI: first steps towards grounded language learning with a human in the loop\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rJeXCo0cYX)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p6.1),[§4\.2\.2](https://arxiv.org/html/2608.11338#S4.SS2.SSS2.p1.1)\.
- Elliset al\.\(2018\)K\. Ellis, L\. Morales, M\. Sablé\-Meyer, A\. Solar\-Lezama, and J\. B\. TenenbaumLibrary learning for neurally\-guided Bayesian program induction\.InProceedings of the 32nd International Conference on Neural Information Processing Systems,NIPS’18,Red Hook, NY, USA,pp\. 7816–7826\.External Links:[Link](https://dl.acm.org/doi/10.5555/3327757.3327878)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p3.1)\.
- Elliset al\.\(2021\)K\. Ellis, C\. Wong, M\. Nye, M\. Sablé\-Meyer, L\. Morales, L\. Hewitt, L\. Cary, A\. Solar\-Lezama, and J\. B\. TenenbaumDreamCoder: bootstrapping inductive program synthesis with wake\-sleep library learning\.InProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation,PLDI 2021,New York, NY, USA,pp\. 835–850\.External Links:ISBN 978\-1\-4503\-8391\-2,[Link](https://dl.acm.org/doi/10.1145/3453483.3454080),[Document](https://dx.doi.org/10.1145/3453483.3454080)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p3.1)\.
- Fanget al\.\(2025\)J\. Fang, Y\. Peng, X\. Zhang, Y\. Wang, X\. Yi, G\. Zhang, Y\. Xu, B\. Wu, S\. Liu, Z\. Li, Z\. Ren, N\. Aletras, X\. Wang, H\. Zhou, and Z\. MengA comprehensive survey of self\-evolving ai agents: a new paradigm bridging foundation models and lifelong agentic systems\.External Links:2508\.07407,[Link](https://arxiv.org/abs/2508.07407)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Gaoet al\.\(2026\)H\. Gao, J\. Geng, W\. Hua, M\. Hu, X\. Juan, H\. Liu, S\. Liu, J\. Qiu, X\. Qi, Y\. Wu, H\. Wang, H\. Xiao, Y\. Zhou, S\. Zhang, J\. Zhang, J\. Xiang, Y\. Fang, Q\. Zhao, D\. Liu, Q\. Ren, C\. Qian, Z\. Wang, M\. Hu, H\. Wang, Q\. Wu, H\. Ji, and M\. WangA survey of self\-evolving agents: what, when, how, and where to evolve on the path to artificial super intelligence\.External Links:2507\.21046,[Link](https://arxiv.org/abs/2507.21046)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Hafner \(2022\)D\. HafnerBenchmarking the spectrum of agent capabilities\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=1W0z96MFEoH)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p4.1),[§1](https://arxiv.org/html/2608.11338#S1.p6.1),[§4\.2\.3](https://arxiv.org/html/2608.11338#S4.SS2.SSS3.p1.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. NarasimhanSWE\-bench: can language models resolve real\-world github issues?\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VTF8yNQM66)Cited by:[§8](https://arxiv.org/html/2608.11338#S8.p2.1)\.
- Kirkpatricket al\.\(2017\)J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska, D\. Hassabis, C\. Clopath, D\. Kumaran, and R\. HadsellOvercoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.External Links:ISSN 1091\-6490,[Link](http://dx.doi.org/10.1073/pnas.1611835114),[Document](https://dx.doi.org/10.1073/pnas.1611835114)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Li and Hoiem \(2017\)Z\. Li and D\. HoiemLearning without forgetting\.External Links:1606\.09282,[Link](https://arxiv.org/abs/1606.09282)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Liuet al\.\(2026\)X\. Liu, X\. Luo, L\. Li, G\. Huang, J\. Liu, and H\. QiaoSkillForge: forging domain\-specific, self\-evolving agent skills in cloud technical support\.External Links:2604\.08618,[Document](https://dx.doi.org/https%3A//doi.org/10.1145/3805712.3808466),[Link](https://arxiv.org/abs/2604.08618)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p2.1)\.
- Maet al\.\(2026\)Z\. Ma, S\. Yang, Y\. Ji, X\. Wang, Y\. Wang, Y\. Hu, T\. Huang, and X\. ChuSkillClaw: Let Skills Evolve Collectively with Agentic Evolver\.arXiv\.Note:arXiv:2604\.08377 \[cs\]External Links:[Link](http://arxiv.org/abs/2604.08377),[Document](https://dx.doi.org/10.48550/arXiv.2604.08377)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p2.1)\.
- Merrillet al\.\(2026\)M\. A\. Merrill, A\. G\. Shaw, N\. Carlini, B\. Li, H\. Raj, I\. Bercovich, L\. Shi, J\. Y\. Shin, T\. Walshe, E\. K\. Buchanan, J\. Shen, G\. Ye, H\. Lin, J\. Poulos, M\. Wang, M\. Nezhurina, D\. Lu, O\. M\. Mastromichalakis, Z\. Xu, Z\. Chen, Y\. Liu, R\. Zhang, L\. L\. Chen, A\. Kashyap, J\. Uslu, J\. Li, J\. Wu, M\. Yan, S\. Bian, V\. Sharma, K\. Sun, S\. Dillmann, A\. Anand, A\. Lanpouthakoun, B\. Koopah, C\. Hu, E\. K\. Guha, G\. H\. S\. Dreiman, J\. Zhu, K\. Krauth, L\. Zhong, N\. Muennighoff, R\. K\. Amanfu, S\. Tan, S\. Pimpalgaonkar, T\. Aggarwal, X\. Lin, X\. Lan, X\. Zhao, Y\. Liang, Y\. Wang, Z\. Wang, C\. Zhou, D\. Heineman, H\. Liu, H\. Trivedi, J\. Yang, J\. Lin, M\. Shetty, M\. Yang, N\. Omi, N\. Raoof, S\. Li, T\. Y\. Zhuo, W\. Lin, Y\. Dai, Y\. Wang, W\. Chai, S\. Zhou, D\. Wahdany, Z\. She, J\. Hu, Z\. Dong, Y\. Zhu, S\. Cui, A\. Saiyed, A\. Kolbeinsson, C\. M\. Rytting, R\. Marten, Y\. Wang, J\. Jitsev, A\. Dimakis, A\. Konwinski, and L\. SchmidtTerminal\-bench: benchmarking agents on hard, realistic tasks in command line interfaces\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=a7Qa4CcHak)Cited by:[§8](https://arxiv.org/html/2608.11338#S8.p2.1)\.
- Miet al\.\(2026\)Q\. Mi, Z\. Ma, M\. Yang, H\. Li, Y\. Wang, H\. Zhang, and J\. WangSkill\-pro: learning reusable skills from experience via non\-parametric PPO for LLM agents\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=9kJQjx2B80)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p2.1)\.
- Niet al\.\(2026\)J\. Ni, Y\. Liu, X\. Liu, Y\. Sun, M\. Zhou, P\. Cheng, D\. Wang, E\. Zhao, X\. Jiang, and G\. JiangTrace2Skill: distill trajectory\-local lessons into transferable agent skills\.External Links:2603\.25158,[Link](https://arxiv.org/abs/2603.25158)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p2.1),[§7](https://arxiv.org/html/2608.11338#S7.p2.1)\.
- Paglieriet al\.\(2025\)D\. Paglieri, B\. Cupiał, S\. Coward, U\. Piterbarg, M\. Wolczyk, A\. Khan, E\. Pignatelli, Ł\. Kuciński, L\. Pinto, R\. Fergus, J\. N\. Foerster, J\. Parker\-Holder, and T\. RocktäschelBALROG: benchmarking agentic LLM and VLM reasoning on games\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=fp6t3F669F)Cited by:[§4\.2\.2](https://arxiv.org/html/2608.11338#S4.SS2.SSS2.p1.1),[§4\.2\.3](https://arxiv.org/html/2608.11338#S4.SS2.SSS3.p1.1)\.
- Pryzantet al\.\(2023\)R\. Pryzant, D\. Iter, J\. Li, Y\. T\. Lee, C\. Zhu, and M\. ZengAutomatic prompt optimization with "gradient descent" and beam search\.External Links:2305\.03495,[Link](https://arxiv.org/abs/2305.03495)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p4.1)\.
- Qiuet al\.\(2026\)L\. Qiu, Z\. Gao, J\. Chen, Y\. Ye, W\. Huang, X\. Xue, W\. Qiu, and S\. TangAutoRefine: from trajectories to reusable expertise for continual llm agent refinement\.External Links:2601\.22758,[Link](https://arxiv.org/abs/2601.22758)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p3.1)\.
- Quet al\.\(2025\)Y\. Qu, A\. Singh, Y\. Lee, A\. Setlur, R\. Salakhutdinov, C\. Finn, and A\. KumarRLAD: training llms to discover abstractions for solving reasoning problems\.External Links:2510\.02263,[Link](https://arxiv.org/abs/2510.02263)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Shiet al\.\(2024\)H\. Shi, Z\. Xu, H\. Wang, W\. Qin, W\. Wang, Y\. Wang, Z\. Wang, S\. Ebrahimi, and H\. WangContinual learning of large language models: a comprehensive survey\.External Links:2404\.16789,[Link](https://arxiv.org/abs/2404.16789)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.External Links:2303\.11366,[Link](https://arxiv.org/abs/2303.11366)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Songet al\.\(2024\)Y\. Song, D\. Yin, X\. Yue, J\. Huang, S\. Li, and B\. Y\. LinTrial and error: exploration\-based trajectory optimization for llm agents\.External Links:2403\.02502,[Link](https://arxiv.org/abs/2403.02502)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Stengel\-Eskinet al\.\(2024\)E\. Stengel\-Eskin, A\. Prasad, and M\. BansalReGAL: Refactoring Programs to Discover Generalizable Abstractions\.arXiv\.Note:arXiv:2401\.16467 \[cs\]External Links:[Link](http://arxiv.org/abs/2401.16467),[Document](https://dx.doi.org/10.48550/arXiv.2401.16467)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p4.1),[§2](https://arxiv.org/html/2608.11338#S2.p3.1),[§7](https://arxiv.org/html/2608.11338#S7.p3.1)\.
- Wanget al\.\(2025a\)A\. Wang, S\. Hager, A\. Asija, D\. Khashabi, and N\. AndrewsHell or high water: evaluating agentic recovery from external failures\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=Zk224WPT42)Cited by:[§3](https://arxiv.org/html/2608.11338#S3.p5.1)\.
- Wanget al\.\(2026a\)C\. Wang, Z\. Yu, X\. Xie, W\. Yao, R\. Fang, S\. Qiao, K\. Cao, G\. Zheng, X\. Qi, P\. Zhang, and S\. DengSkillX: automatically constructing skill knowledge bases for agents\.External Links:2604\.04804,[Link](https://arxiv.org/abs/2604.04804)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p2.1)\.
- Wanget al\.\(2023\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: An Open\-Ended Embodied Agent with Large Language Models\.Transactions on Machine Learning Research\(en\)\.External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=ehfRiF0R3a)Cited by:[Appendix F](https://arxiv.org/html/2608.11338#A6.p1.1),[§4\.1](https://arxiv.org/html/2608.11338#S4.SS1.p5.1),[§7](https://arxiv.org/html/2608.11338#S7.p1.1),[§7](https://arxiv.org/html/2608.11338#S7.p3.1)\.
- Wanget al\.\(2026b\)H\. Wang, G\. Wang, H\. Xiao, Y\. Zhou, Y\. Pan, J\. Wang, K\. Xu, Y\. Wen, X\. Ruan, X\. Chen, and H\. QiSkill\-sd: skill\-conditioned self\-distillation for multi\-turn llm agents\.External Links:2604\.10674,[Link](https://arxiv.org/abs/2604.10674)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p2.1)\.
- Wanget al\.\(2024\)L\. Wang, X\. Zhang, H\. Su, and J\. ZhuA comprehensive survey of continual learning: theory, method and application\.External Links:2302\.00487,[Link](https://arxiv.org/abs/2302.00487)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Wanget al\.\(2022\)R\. Wang, P\. Jansen, M\. Côté, and P\. AmmanabroluScienceWorld: is your agent smarter than a 5th grader?\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 11279–11298\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.775/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.775)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p6.1),[§4\.2\.1](https://arxiv.org/html/2608.11338#S4.SS2.SSS1.p1.1)\.
- Wanget al\.\(2025b\)Z\. Wang, K\. Wang, Q\. Wang, P\. Zhang, L\. Li, Z\. Yang, X\. Jin, K\. Yu, M\. N\. Nguyen, L\. Liu, E\. Gottlieb, Y\. Lu, K\. Cho, J\. Wu, L\. Fei\-Fei, L\. Wang, Y\. Choi, and M\. LiRAGEN: understanding self\-evolution in llm agents via multi\-turn reinforcement learning\.External Links:2504\.20073,[Link](https://arxiv.org/abs/2504.20073)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1)\.
- Wanget al\.\(2025c\)Z\. Z\. Wang, A\. Gandhi, G\. Neubig, and D\. FriedInducing programmatic skills for agentic tasks\.External Links:2504\.06821,[Link](https://arxiv.org/abs/2504.06821)Cited by:[Appendix E](https://arxiv.org/html/2608.11338#A5.p1.1),[§2](https://arxiv.org/html/2608.11338#S2.p3.1),[§4\.1](https://arxiv.org/html/2608.11338#S4.SS1.p4.1),[§7](https://arxiv.org/html/2608.11338#S7.p3.1)\.
- Wanget al\.\(2025d\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent Workflow Memory\.\(en\)\.External Links:[Link](https://openreview.net/forum?id=NTAhi2JEEE)Cited by:[§2](https://arxiv.org/html/2608.11338#S2.p3.1),[§7](https://arxiv.org/html/2608.11338#S7.p2.1)\.
- Yanget al\.\(2026\)Y\. Yang, J\. Li, Q\. Pan, B\. Zhan, Y\. Cai, L\. Du, J\. Zhou, K\. Chen, Q\. Chen, X\. Li, B\. Zhang, and L\. HeAutoSkill: experience\-driven lifelong learning via skill self\-evolution\.External Links:2603\.01145,[Link](https://arxiv.org/abs/2603.01145)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p2.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§4\.1](https://arxiv.org/html/2608.11338#S4.SS1.p2.1)\.
- Yuksekgonulet al\.\(2025\)M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, P\. Lu, Z\. Huang, C\. Guestrin, and J\. ZouOptimizing generative ai by backpropagating language model feedback\.Nature639,pp\. 609–616\.Cited by:[§4\.1](https://arxiv.org/html/2608.11338#S4.SS1.p3.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: llm agents are experiential learners\.External Links:2308\.10144,[Link](https://arxiv.org/abs/2308.10144)Cited by:[§7](https://arxiv.org/html/2608.11338#S7.p1.1),[§7](https://arxiv.org/html/2608.11338#S7.p2.1)\.
- Zhenget al\.\(2025\)B\. Zheng, M\. Y\. Fatemi, X\. Jin, Z\. Z\. Wang, A\. Gandhi, Y\. Song, Y\. Gu, J\. Srinivasa, G\. Liu, G\. Neubig, and Y\. SuSkillWeaver: web agents can self\-improve by discovering and honing skills\.External Links:2504\.07079,[Link](https://arxiv.org/abs/2504.07079)Cited by:[§2](https://arxiv.org/html/2608.11338#S2.p3.1),[§7](https://arxiv.org/html/2608.11338#S7.p3.1)\.
- Zhonget al\.\(2026\)S\. Zhong, Y\. Lu, J\. Ning, Y\. Wan, L\. Feng, Y\. Ao, L\. F\. R\. Ribeiro, M\. Dreyer, S\. Ammirati, and C\. XiongSkillLearnBench: benchmarking continual learning methods for agent skill generation on real\-world tasks\.External Links:2604\.20087,[Link](https://arxiv.org/abs/2604.20087)Cited by:[§1](https://arxiv.org/html/2608.11338#S1.p2.1)\.

## Appendix AImplementation Details

### A\.1Models and Decoding

All results in the main paper usegpt\-5\.4\-minias the underlying LLM for every role\. Decoding settings differ by role:

- •*Actor agent:*reasoning\_effort = low,parallel\_tool\_calls = false, request limit100100per episode\.
- •*Inducer agent:*reasoning\_effort = medium,parallel\_tool\_calls = false, request limit100100per episode\.

Libraries are reset to empty at the start of training for each task family; only the primitives carry over\. During the sleep cycle, the inducer is allowed to add, edit, or remove library entries between sleep cycles\.

### A\.2Training and Evaluation Schedule

Each training run consists of200200online rollouts\. The inducer is invoked every1010rollouts \(one sleep cycle per1010training episodes\), and the resulting library is evaluated on the3030\-episode test set every5050training rollouts, giving five evaluation checkpoints per run \(zero\-shot at rollout00, plus rollouts50,100,150,20050,100,150,200\)\. Each setting is repeated over33random seeds\. At each checkpoint, we first compute each seed\-level metric over the3030test episodes, then report the mean and±1,s\.d\.\\pm 1,\\text\{s\.d\.\}across the three seed\-level values\.

### A\.3Cost Accounting

Per\-episode inference cost is computed as

cost=pin​Ninuncached\+pcache​Nincached\+pout​Nout,\\mathrm\{cost\}=p\_\{\\mathrm\{in\}\}N\_\{\\mathrm\{in\}\}^\{\\mathrm\{uncached\}\}\+p\_\{\\mathrm\{cache\}\}N\_\{\\mathrm\{in\}\}^\{\\mathrm\{cached\}\}\+p\_\{\\mathrm\{out\}\}N\_\{\\mathrm\{out\}\},whereNinuncachedN\_\{\\mathrm\{in\}\}^\{\\mathrm\{uncached\}\},NincachedN\_\{\\mathrm\{in\}\}^\{\\mathrm\{cached\}\}, andNoutN\_\{\\mathrm\{out\}\}are the uncached input, cached input, and output tokens summed over all LLM calls in the episode\. This includes actor calls during rollout and sleep\-time inducer calls amortized over the rollouts in the corresponding inducer batch\. We use the published API prices as of May 6, 2026\. Forgpt\-5\.4\-mini, we set

pin=$0\.75,pcache=$0\.075,pout=$4\.50p\_\{\\mathrm\{in\}\}=\\$0\.75,\\qquad p\_\{\\mathrm\{cache\}\}=\\$0\.075,\\qquad p\_\{\\mathrm\{out\}\}=\\$4\.50per million tokens\.

Forgemini\-3\-flash\-preview, we set

pin=$0\.35,pcache=$0\.0875,pout=$1\.05p\_\{\\mathrm\{in\}\}=\\$0\.35,\\qquad p\_\{\\mathrm\{cache\}\}=\\$0\.0875,\\qquad p\_\{\\mathrm\{out\}\}=\\$1\.05per million tokens\.

We use this dollar\-cost accounting in Figure[3](https://arxiv.org/html/2608.11338#S5.F3)\.

## Appendix BScienceWorld Details

### B\.1Chosen Tasks

We evaluate on two ScienceWorld task families:Electricity\(family 3\) andClassification\(family 4\)\. Each family contains multiple task names \(sub\-tasks\); episodes are addressed by*\(task\_name, variation\_idx\)*pairs\. We use ScienceWorld’s built\-in train/dev/test variation partitions \(get\_variations\_train/dev/test\), which are disjoint by construction\. For each family we sample200train variations and30test variations with a fixed seed \(4242\); the same train/test sets are reused across all three training seeds, so only the rollout stochasticity differs across runs\.

### B\.2Action Interface

The agent acts through a fixed set of 25 primitive Python functions that wrap ScienceWorld’s text parser\. These primitives cover navigation \(go,look\_around,look\_at,look\_in\), object manipulation \(pick\_up,put\_down,move,pour,dunk\), state changes \(open,close,activate,deactivate,connect,disconnect,mix\), measurement and inspection \(focus\_on,read,use,inventory,task\_description\), and auxiliary actions \(eat,flush,wait,disambiguate\)\. Each primitive takes typed string arguments \(e\.g\.connect\(obj\_a, obj\_b\)\); this is the interface the induced library composes against\. We do not expose ScienceWorld’sget\_valid\_actionslist to the agent\. In our setting, the agent’s effective action space includes both primitives and any library functions available under the current method; surfacing only the primitive\-level valid\-action set would be misleading, since it omits the higher\-level abstractions the agent is expected to prefer\.

### B\.3Observations and Episode Termination

Observations are plain text: on each step the agent receives the room description and the parser’s acknowledgement of the previous action\. The full observation history is kept in context \(no sliding window\)\. The initial prompt contains the task description returned by ScienceWorld’staskdescription\(\)and the opening room observation; no oracle hints or gold trajectories are provided\.

An episode ends when \(i\) the actor agent calls thesubmit\_answertool to declare the task complete, or \(ii\) the actor agent exceeds its budget of100LLM calls per episode\. We set this budget at100100because additional tool calls no longer meaningfully improve the agent’s success rate, so further extending the budget would inflate cost without changing the headline results\. In both cases, success is determined by checking ScienceWorld’s internal score: the episode is scored as a success iff the score has reached its maximum value of1\.01\.0, and as a failure otherwise\.

## Appendix CBabyAI Details

### C\.1Chosen Tasks

We evaluate on thepick\_up\_seq\_go\_tosubtask of BabyAI\-Text as exposed by BALROG\. Episodes are addressed by environment seed\. We sample200train seeds and30test seeds with a fixed top\-level seed \(4242\); train and test seed sets are disjoint by construction\. The same train/test sets are reused across all three training seeds, so only the rollout stochasticity differs across runs\.

### C\.2Action Interface

The agent acts through a fixed set of 6 primitive Python functions that wrap BabyAI\-Text’s command interface: navigation \(turn\_left,turn\_right,go\_forward\), object manipulation \(pick\_up,drop\), and state change \(toggle\)\. Each primitive takes no arguments and operates on the cell directly in front of the agent; this is the interface the induced library composes against\. We do not expose BALROG’s valid\-action list to the agent\. In our setting, the agent’s effective action space includes both primitives and any library functions available under the current method; surfacing only the primitive\-level valid\-action set would be misleading, since it omits the higher\-level abstractions the agent is expected to prefer\.

### C\.3Observations and Episode Termination

Observations are plain text descriptions of the agent’s egocentric view, as produced by the BabyAI\-Text wrapper\. The full observation history is kept in context \(no sliding window\)\. The initial prompt contains the synthetic natural\-language instruction sampled by BabyAI’s Baby Language grammar and the opening observation\.

An episode ends when \(i\) the simulator emits a terminal signal indicating the instruction has been satisfied, or \(ii\) the actor agent exceeds its budget of30LLM calls per episode\. We set this budget at3030because additional tool calls no longer meaningfully improve the agent’s success rate, so further extending the budget would inflate cost without changing the headline results\. Like ScienceWorld, success is determined entirely by the environment’s binary task\-completion signal: the episode is scored as a success if the instruction is fully satisfied within the budget, and as a failure otherwise\.

## Appendix DCrafter Details

### D\.1Episode Sampling

Each Crafter episode is a freshly generated world keyed by a single integer seed\. We use three independent runs, each with200training worlds and30test worlds, but with different top\-level seeds to ensure diversity\. All methods are run three times using the same training and test worlds, so comparisons are matched by world seed\. Unless stated otherwise, reported means and uncertainty bands are computed over these three matched runs\.

### D\.2Action Interface

The agent acts through a fixed set of 18 primitive Python functions wrapping the Crafter engine: 17 are game\-advancing primitives and one is a non\-advancing read action\. These primitives cover navigation \(move\_north,move\_south,move\_east,move\_west\), interaction \(do– a multi\-use action that collects material, drinks from a lake, or hits the creature in front;sleep;noop– advance one tick without acting\), structure placement \(place\_stone,place\_table,place\_furnace,place\_plant\), and tool crafting \(make\_wood\_pickaxe,make\_stone\_pickaxe,make\_iron\_pickaxe,make\_wood\_sword,make\_stone\_sword,make\_iron\_sword\)\. All 17 game\-advancing primitives are zero\-argument and returnNone; each advances the Crafter world by exactly one tick\. Theget\_current\_observation\(\)primitive returns the current textual observation without advancing the world, and is provided so that library functions can re\-inspect the world between primitive calls\. Crafter also has nosubmit\_answeraction; episodes terminate only through the environment itself \(see below\)\.

### D\.3Observations

Observations are plain text rendered from the agent’s egocentric9×99\\times 9local view, with three blocks shown each step: vital status, inventory contents, and visible objects \(with the object directly in front of the agent reported separately\)\. Each observation is wrapped between<<OBSERVATION\_BEGIN\>\>and<<OBSERVATION\_END\>\>delimiters and substituted once into the\{obs\}slot of the user prompt template at the start of the rollout\. It will also be returned upon agent calling theget\_current\_observation\(\)primitive\.

A representative mid\-episode observation looks as follows:

<<OBSERVATION\_BEGIN\>\>

Yourstatus:

\-health:7/9

\-food:5/9

\-drink:6/9

\-energy:8/9

Yourinventory:

\-wood:3

\-stone:1

\-wood\_pickaxe:1

Yousee:

\-grass1stepnorth

\-tree2stepsnorthand1stepeast

\-stone3stepseast

\-zombie4stepsnorthand2stepswest

\-water5stepssouthand3stepseast

Youfacegrassatyourfront\.

<<OBSERVATION\_END\>\>

When the agent’s inventory is empty, the inventory block collapses to the line “*You have nothing in your inventory\.*”; when nothing other than the player is visible in the9×99\\times 9window, the visible\-objects block collapses to “*You see nothing away from you\.*” Sleeping or dead states are surfaced by prefixing the status block with a single sentence \(e\.g\., “*You are sleeping, and will not be able to take actions until energy is full\.*”\)\.

### D\.4Episode Termination

An episode ends when \(i\) the player’s health reaches zero, or \(ii\) the agent exhausts its budget of2,000primitive actions\. Each episode is scored as the fraction of the 22 achievements unlocked at termination\. The actor agent’s per\-episode LLM\-call budget \(request\_limit\) is also set to2,0002\{,\}000to ensure that code baselines are not disadvantaged by being cut off early\.\. In practice, we rarely see any rollout exceeding this budget\.

### D\.5Achievements

The agent is told about the 22 Crafter achievements in the initial system prompt, in a fixed display order: Collect Wood, Place Table, Eat Cow, Collect Sapling, Collect Drink, Make Wood Pickaxe, Make Wood Sword, Place Plant, Defeat Zombie, Collect Stone, Place Stone, Eat Plant, Defeat Skeleton, Make Stone Pickaxe, Make Stone Sword, Wake Up, Place Furnace, Collect Coal, Collect Iron, Make Iron Pickaxe, Make Iron Sword, Collect Diamond\.

### D\.6Zombie\-Frequency Variants

The external\-randomness analysis in §[6\.3](https://arxiv.org/html/2608.11338#S6.SS3)sweeps a single environment knob,zombie\_frequency\(ff\), while holding all other settings fixed\. The knob enters the upstream Crafter engine in two places\. First, at world reset: whenf=0f=0, every zombie generated by terrain initialization is explicitly removed before the first step\. Second, at every tick during world balancing: zombies spawn with probabilitymin⁡\(1\.0,0\.3​f\)\\min\(1\.0,\\ 0\.3f\)and despawn with probabilitymin⁡\(1\.0,0\.4​f\)\\min\(1\.0,\\ 0\.4f\)per eligible chunk, and the per\-chunk target population is scaled byff\. With this scaling,f=1\.0f=1\.0recovers the upstream Crafter defaults,f=0\.0f=0\.0removes zombies entirely, andf=2\.0f=2\.0doubles both spawn and despawn pressures \(each clipped at1\.01\.0\) and doubles the target population\. No other engine parameters are changed across variants\.

For the external randomness experiments, we instantiate this knob at three levels labeled0x\(f=0\.0f=0\.0\),1x\(f=1\.0f=1\.0, default\), and2x\(f=2\.0f=2\.0\), and run each of the three comparison methods at each level: OPO, Voyager, andSpeedRunner\. For a given method, the 1x configuration is the same as the one used in the main results; therefore we directly reused the results from the main experiments; the 0x and 2x configurations differ only inzombie\_frequency\.

## Appendix EASI Adaptation Details

We adapt ASI\([33](https://arxiv.org/html/2608.11338#bib.bib7)\)to our*purly*\-online setting while preserving its programmatic skill induction mechanism\. In the original algorithm, each newly induced skill undergoes a*skill verification*step, in which the agent attempts to re\-solve the inducing task using the new skill and retains the skill only if verification succeeds\. Because this step requires additional task interaction beyond the original online trajectory, we remove it to satisfy our no\-replay constraint\. All other components of ASI are left unchanged\.

## Appendix FVoyager Adaptation Details

Given its relevance to our work, we make three task\-level adaptations to preserve Voyager’s\([28](https://arxiv.org/html/2608.11338#bib.bib3)\)core ingredients while aligning its training protocol with our online constraints\. First, the curriculum agent is informed about the Crafter’s 22 achievements as the final optimization target when proposing the next intermediate subgoals\. Second, we remove Voyager’s respawn\-style continuation after death: when the agent dies or the episode terminates, training restarts from a fresh Crafter world with no inventory carryover, matching the irreversible online setting used by all other methods\. Third, evaluation is performed through the same rollout interface used by the other baselines, so that learned skills are assessed under the same acting protocol rather than through a bespoke execution regime\.

## Appendix GAdditional Main Results

### G\.1Code Interpreter Ablations

Our results in Figure[7](https://arxiv.org/html/2608.11338#A7.F7)show that removing the code interpreter degrades final performance in two out of three environments and increases the amount of output tokens required substantially in all three\. This demonstrates the importance of this component forSpeedRunner’s gains in performance and especially efficiency for programmatic skill induction\.

Figure 7:ExtraSpeedRunnerablations on the Crafter and ScienceWorld environments\. As in BabyAI, removing access to the code interpreter has only small effects on performance but contributes substantially to improvements in efficiency\.
### G\.2Cross\-Model Results

Figure 8:Main results using GPT\-5\.4\-mini with tokens instead of cost for easy comparison across models\.Figure 9:Gemini\-3\-Flash performance and output\-token cost across Crafter, ScienceWorld, and BabyAI\. As with GPT\-5\.4\-mini,SpeedRunnerreduces output tokens across the board, however, performance improvements are much less consistent than with GPT\-5\.4\-mini\.Figure 10:Qwen\-3\.5\-27B performance and output\-token cost across Crafter, ScienceWorld, and BabyAI\. As with GPT\-5\.4\-mini and Gemini\-3\-Flash,SpeedRunnergenerally matches or improves final task performance while reducing output tokens\.In order to test whetherSpeedRunner’s trends depend on the backbone LLM, we repeat the main comparison with Gemini\-3\-Flash and Qwen\-3\.5\-27B\. As shown in Figures[9](https://arxiv.org/html/2608.11338#A7.F9)and[10](https://arxiv.org/html/2608.11338#A7.F10),SpeedRunnerretains the best output\-token efficiency across the board and the best performance–efficiency tradeoff in most instances\. More specifically, using the same two\-sided pair t\-test from our main results, we find that our method is significantly more efficient and better performing than our code\-based baselines in all settings with every models except for Crafter, where they are much more competitive\. Additionally, our method’s performance against OPO, our non\-code baseline, reveals an interesting pattern\.SpeedRunnerachieves significantly worse final success rate than OPO with Qwen in ScienceWorld and Gemini in all benchmarks, despite achieving much lower token usage\. We highlight this result because it shows that aggressive code delegation is not always performance\-improving when the base actor is already strong\. Not all performance can be transferred into code\.

In an attempt to understand this behavior, we note that both Gemini\-3\-Flash and Qwen\-3\.5\-27B start from higher performance levels than GPT\-5\.4\-mini in all benchmarks where OPO beatsSpeedRunner, indicating stronger raw model capacity in these environments\. We believe this behavior is driven by compression aggressiveness\.SpeedRunner’s iterative induction produces hierarchical libraries that delegate a large fraction of decisions to code; in Crafter, for example, Gemini\-3\-Flash reduces LLM calls by 94% over training\. This yields very low inference cost, but it can also make the policy too rigid in stochastic environment\. Code induced from earlier trajectories may encode procedures that are efficient on average but less able to adapt to later situations requiring reactive judgment\. Thus, these results expose a real tradeoff:SpeedRunnercan over\-compress a capable actor, improving cost while sacrificing some performance\.

Nevertheless, as illustrated in Figure[11](https://arxiv.org/html/2608.11338#A7.F11), our approach succeeds in*consistent*learning and efficiency performance\. Of the 9 settings, our approach dominates in 5 and is on the frontier in the remainder, a feat that is unmatched by any of the other baselines\.

Figure 11:Performance vs output tokens per trace for all approaches and benchmarks for GPT\-5\.4\-mini \(top\), Gemini\-3\-Flash \(middle\) and Qwen\-3\.5\-27B \(bottom\) for easy comparison\. This figure demonstrates that SpeedRunner dominates in most settings or remains at the frontier in all experiments we conducted\.
### G\.3Offline Continual Learning with ASI

Figure 12:Effect of ASI’s replay\-based skill verification on ScienceWorld task 3\. Restoring replay\-based verification improves ASI’s task performance, particularly early in training\.SpeedRunnerachieves the highest final performance while using substantially fewer output tokens than both ASI variants\.We further study the effect of experience replay by comparing our online adaptation and the original offline version of ASI\. As described in Appendix[E](https://arxiv.org/html/2608.11338#A5), the original algorithm verifies each new skill by re\-executing the inducing task, whereas our online adaptation removes this step\.

Figure[12](https://arxiv.org/html/2608.11338#A7.F12)compare ASI, ASI \(replay\), andSpeedRunneron ScienceWorld task 3\. Restoring the verification step improves ASI’s mean task performance, particularly early in training,because it filters out broken skills that lead to regression on a previous task\. However, it does not reduce token cost beyond the adapted online version\. After 200 training traces,SpeedRunnerachieves higher mean task performance than both ASI variants while using substantially fewer output tokens per episode \(approximately 65% fewer than both ASI variants\)\. These results indicate that removing ASI’s replay\-based verification step makes continual skill induction more challenging\.

### G\.4Why Performance Gains Vary by Benchmark

SpeedRunner’s main advantage comes from converting repeated behavior into compact executable routines, and the value of this conversion depends on the structure of the environment\. ScienceWorld and BabyAI contain many reusable procedures that can be invoked repeatedly once discovered, soSpeedRunnerimproves performance while dramatically compressing output length\. The effect is sharpest on BabyAI, where final per\-eval\-epoch output reaches roughly an*eighth*of the ReAct baseline — approaching an order\-of\-magnitude compression ratio that, combined with near\-perfect final accuracy, indicates that the actor is almost entirely delegating decisions to a small set of induced abstractions rather than re\-deriving primitive sequences\. BabyAI’s high zero\-shot baseline \(∼\\sim67%67\\%\) limits how much headroom remains for raw performance gains, but this compression is itself the evidence that the code delegation pays off in reliability*and*efficiency\.

Crafter is more challenging: the environment is longer\-horizon, stochastic, and reactive, so a routine that is useful in one state can become inappropriate when immediate survival needs arise \(e\.g\. a zombie attack\)\. This shrinks the room for raw performance gains, but the compression advantage persists —SpeedRunner’s end\-of\-training output tokens are more than3×3\\timeslower than OPO’s and ASI’s, even though all four methods begin from the same ReAct cost baseline and ASI and OPO actually grow rather than shrink\. We analyze the structural reorganization that letsSpeedRunnerretain this advantage as environmental randomness rises in §[6\.3](https://arxiv.org/html/2608.11338#S6.SS3)\.

## Appendix HFull Quantitative Codebook Analysis

We provide the full structural analysis of the learned codebooks here\. The main text summarizes the two most diagnostic call\-graph measures, maximum depth and density; this appendix reports the complete set of statistics, including library size, average depth, and cyclomatic complexity\.

Recall that we characterize the codebooks learned by each method using five structural measures, computed from the final codebook at the end of training\. We first construct a directed call graphG=\(V,E\)G=\(V,E\)for each codebook, where each nodev∈Vv\\in Vis an induced function and each edge\(u,v\)∈E\(u,v\)\\in Eindicates that functionuucalls functionvv\. Letn=\|V\|n=\|V\|\.

- •Library size\(\#Fns\) is the number of induced functions, including both public and private functions\.
- •Maximum depth\(max d\.\) is the length of the longest directed path inGG, measuring the height of the deepest abstraction built by the inducer\.
- •Average depth\(avg d\.\) is the mean depth across all induced functions inGG\.
- •Densityis\|E\|/\(n⁡\(n−1\)\)\|E\|/\(n\(n\-1\)\), excluding self\-edges, and measures how often induced functions call other induced functions rather than relying only on the primitive action set\.
- •Cyclomatic complexity\(cyclo\.\) is the standard McCabe complexity averaged across functions, capturing whether individual skills encode fixed action sequences or branch over multiple cases\.

Table 2:Structural measures of each method’s final skill library across four benchmark settings, using GPT\-5\.4\-mini with mean±\\pmsd over three seeds\. Shaded rows mark the fullSpeedRunnersystem\. Bold marks the most compact library for \#Fns and the highest value for the remaining structural measures within each benchmark\.Table[2](https://arxiv.org/html/2608.11338#A8.T2)shows thatSpeedRunnerlearns compact but substantially more structured libraries than the baselines\. Relative to ASI,SpeedRunnerconsistently increases call\-graph density, maximum depth, and average depth, indicating that its skills are not merely shallow wrappers around primitive actions but reusable routines with hierarchical dependencies\. Relative to Voyager on Crafter,SpeedRunnerachieves much higher density with nearly two orders of magnitude fewer functions, suggesting that it factors repeated behavior into shared helpers instead of accumulating scenario\-specific code\.

Although Voyager has higher cyclomatic complexity on Crafter, this complexity is concentrated inside individual functions rather than distributed through reusable library structure\. Its density is only0\.00050\.0005, compared with0\.1490\.149forSpeedRunner, meaning that its functions rarely call one another\. Thus, Voyager’s library is large and internally branchy, but weakly compositional: each function tends to encode a complete scenario rather than decomposing behavior into smaller reusable components\.

The code\-interpreter ablation preserves much ofSpeedRunner’s organization, but the full method is generally deeper, denser, or more decision\-rich\. On Crafter and both ScienceWorld settings,SpeedRunnerhas higher maximum and average depth thanSpeedRunnerw/o CI\. On BabyAI, where the tasks are shorter and the learned libraries are naturally small,SpeedRunnerinstead has the highest density and cyclomatic complexity\. Overall,SpeedRunner’s advantage is not larger codebooks, but better organized ones: compact libraries whose skills are more reusable, modular, and compositional\.

## Appendix IPer\-Environment Qualitative Analysis

This appendix expands the three patterns identified in the main paper with concrete examples from each environment\. Each subsection contrasts the key abstraction discovered by ASI, the code\-interpreter\-free ablation \(SpeedRunnerw/o CI\), andSpeedRunner\.

### I\.1BabyAI: Compositional Mission Parsing

The PickUpSeqGoTo task requires parsing a natural\-language mission \(e\.g\.,*“go to the red box after you pick up the blue ball”*\) and executing it in a partially\-observable grid world\. Some phrasings reverse grammatical order relative to execution order, which is the decisive challenge\.

Table 3:Key abstractions per method in the BabyAI environment\. The task hinges on parsing missions where grammatical order can reverse execution order\. ASI templates motion sequences from traces and never parses the mission;SpeedRunnerw/o CI patches navigation failures reactively but still defers mission parsing to the LLM; onlySpeedRunnerderives an explicit priority\-ordered parser by executing candidate patterns against historical mission strings\.A naïve left\-to\-right parser inverts execution order on reversed\-phrasing missions\.SpeedRunnerw/o CI has no parser and leaves disambiguation to the LLM, which occasionally misorders goals \(its 6\.7% error\)\.SpeedRunner’s coding agent iterated over historical mission strings insideexecute\_code, observed the inversion, and added reordering entries to a priority\-ordered pattern list \(matching*“go to X after you pick up Y”*and*“pick up X after you go to Y”*before the fallback\)\. The gap betweenSpeedRunnerw/o CI andSpeedRunneris entirely attributable to this one structural decision\.

### I\.2Crafter

Table 4:Crafter: key abstractions per method, showing how each handles the tech\-tree progression and resource\-lookup failures characteristic of the environment\. ASI crafts unconditionally with no inventory checks; Voyager appends a new function per episode and never consolidates, yielding 483 near\-duplicate variants;SpeedRunnerw/o CI builds reactive guards one failure at a time;SpeedRunnerencodes the full progression hierarchy in a single function and diagnoses systematic lookup failures via trace queries to introduce\_bounded\_exploration\. The Voyager row illustrates that observation\-reading alone is insufficient—consolidation is a necessary co\-condition\.Table 5:ScienceWorld T3 \(conductivity\): key abstractions per method\. The task requires wiring an unknown substance into a battery\-and\-bulb circuit and placing it in the correct box based on whether the bulb lights\. The decisive ASI–ablation gap is programmatic result reading: ASI places the substance after a fixed wait regardless of bulb state, whileSpeedRunnerw/o CI gates placement on a regex over the bulb observation\.SpeedRunner’s additional contribution is consolidation—hoisting per\-function ambiguity handling into a single wrapper, cutting library size by 22%\.SpeedRunner’s\_choose\_progress\_targetencodes the full tech tree in one place: higher\-tier ores are gated behind the corresponding pickaxe tier, and stone is de\-prioritized once the stockpile is sufficient\. Neither ASI norSpeedRunnerw/o CI has an analogue—ASI crafts unconditionally, andSpeedRunnerw/o CI’s guards are reactive and incomplete\.

The second decisive abstraction,\_bounded\_exploration, illustrates the inducer’s diagnostic loop\. After observing repeatednot\_foundreturns fromtravel\_to\_visible\_resource, the inducer queried the last batch of traces and printed every top\-levelnot\_foundalongside its target \(wood, tree, cow\)\. It then walked the trace tree to count call frequencies per batch and observed thattravel\_to\_visible\_resourcewas being invoked 116–250 times per batch in later rounds—i\.e\., the agent was repeatedly trying and failing to find resources in view\. Diagnosing this as a visibility problem rather than a navigation bug, the inducer added\_bounded\_exploration, which sweeps the local neighborhood around the agent for the desired resource before falling back to anot\_found\.

Voyager is the counterfactual to consolidation: 483 functions, no shared progression model, boilerplate duplicated hundreds of times, and the lowest score of any method \(14\.7%, below ASI’s 20\.5%\)\. Reading observations is not sufficient; consolidation is a necessary co\-condition\.

### I\.3ScienceWorld: Programmatic Verification

ScienceWorld T3 \(conductivity\) requires wiring an unknown substance into a battery\-and\-bulb circuit and placing it in the correct box\. T4 \(find category\) requires locating an object of a given category across rooms\.

The decisive gap between ASI andSpeedRunnerw/o CI is programmatic result reading: ASI moves to the target box after a fixed wait regardless of bulb state;SpeedRunnerw/o CI gates placement onre\.search\("which is on"\)of the bulb observation\.SpeedRunner’s additional contribution is consolidation—inSpeedRunnerw/o CI, ambiguity handling was added ad\-hoc inside each function as the error appeared;SpeedRunnerhoisted it into a single wrapper, cutting library size 22% \(27→\\to21\) while applying the fix uniformly\.

Table 6:ScienceWorld T4 \(find category\): key abstractions per method, the one setting whereSpeedRunnerw/o CI outperformsSpeedRunner\. Both methods program category classification—a clear advance over ASI’s full delegation to the LLM—but with different strategies:SpeedRunnerw/o CI maintains targeted exclusion lists for observed false positives \(“painting,” “egg”\), whileSpeedRunnerderives general normalization rules viaexecute\_code\. The result suggests pre\-commit testing helps most when the failure\-mode space is combinatorially large; for finite enumerable false\-positive sets, reactive per\-failure refinement is sufficient\.T4 is the one setting whereSpeedRunnerw/o CI outperformsSpeedRunner\. Both implement programmatic category classification—a clear advance over ASI’s delegation—but with different strategies:SpeedRunnerw/o CI uses targeted exclusion lists \(skip “painting,” “egg”\), whileSpeedRunneruses general normalization rules\. ThatSpeedRunnerw/o CI still wins suggests pre\-commit testing helps most when the failure\-mode space is*combinatorially large*—as in BabyAI’s mission ordering or Crafter’s tech tree—rather than a finite enumerable set of false\-positive patterns, where reactive per\-failure refinement is sufficient\.

## Appendix JExternal Randomness: Additional Analysis

Table[7](https://arxiv.org/html/2608.11338#A10.T7)reports the full structural measures \(definition in §[H](https://arxiv.org/html/2608.11338#A8)\) that the figure abstracts over\.

The mechanism behindSpeedRunner’s %Combat rise is concretely visible in the 2x library: of its∼17\\sim 17combat\-aware functions, eleven are*existing*harvest and movement routines \(e\.g\.,harvest\_visible,clear\_front\_blocker,opening\_turn\) that have been edited to accept aninclude\_combatorallow\_combatkeyword argument — one kwarg flip and every caller gets the defensive variant\. Voyager, unable to edit, instead spawns a fresh bespoke function for each new encounter \(collect\_wood\_log\_avoid\_front\_zombie,collect\_one\_log\_avoiding\_east\_zombie,step\_adjacent\_to\_skeleton, …\): the same combat logic thatSpeedRunneramortizes once across its call graph is paid in duplicated branches inside every Voyager skill, which is why Voyager’s per\-skill cyclomatic remains the highest of any method even as its density stays near zero\.

Table 7:Final\-library structure across the three randomness conditions \(GPT\-5\.4\-mini\)\. Columns are the structural measures from §[H](https://arxiv.org/html/2608.11338#A8);*%Combat*is the fraction of induced functions whose body references a creature/combat keyword\.
## Appendix KPost\-hoc Library Truncation on Voyager

A natural reading of Voyager’s regression under standard \(1×1\\times\) and doubled \(2×2\\times\) zombie pressure \(Table[7](https://arxiv.org/html/2608.11338#A10.T7)\) is that its append\-only library simply grows past the point where the rollout agent can usefully attend to it: by the end of training, the GPT\-5\.4\-mini seeds carry libraries of\{615,909,1707\}\\\{615,909,1707\\\}public skills, and one of them already overflows the model’s context window outright\. If volume alone were the issue, then keeping only the most recently induced skills should recover most of the gap to the no\-codebook baseline\. We therefore ask the following question: can a simple last\-NNtruncation of Voyager’s final library rescue its1×1\\times\-zombie performance?

##### Setup\.

We take the final codebook checkpoint of one1×1\\times\-zombie Voyager seed and re\-evaluate it on the same3030\-episode Crafter test set used in the main results \(§[5](https://arxiv.org/html/2608.11338#S5)\), holding the rollout agent and seeds fixed\. For eachN∈\{50,100,200,400,800\}N\\in\\\{50,100,200,400,800\\\}, we present the rollout agent with only the*last*NNskills—those induced most recently—as callable tools and as entries in the instruction manual; the earlier909−N909\-Nskills are hidden from the prompt but remain in the execution namespace, so any visible skill whose body invokes an earlier one still runs correctly\. This isolates the variable of interest: the size of the*agent\-visible*library, with the underlying behavior of each visible skill held fixed\. We bracket the sweep with two reference points reused from the training\-time periodic evaluation of the same run:N=0N=0corresponds to the no\-codebook ReAct baseline \(the actor runs with primitives only\), andN=909N=909corresponds to the full final library\.

Table 8:Post\-hoc truncation of Voyager’s final1×1\\times\-zombie library \(909909induced skills\)\. The rollout agent is shown only the most recently inducedNNskills; earlier skills remain callable when invoked from inside a visible one\. Mean Progression is the fraction of the2222Crafter achievements unlocked per episode, averaged over the3030\-episode held\-out test set\.N=0N=0andN=909N=909are the no\-codebook and full\-library reference points taken from the training\-time periodic evaluation of the same run\.
##### Truncation does not structurally remedy Voyager\.

Three observations from Table[8](https://arxiv.org/html/2608.11338#A11.T8)point against volume being the binding constraint\. First, the truncation curve is*non\-monotonic*inNN: there is no clear rising or falling trend as the visible library shrinks\. Second,*no truncation meaningfully exceeds the no\-codebook baseline*: the closest setting,N=400N=400, matches it within noise \(a gap of one achievement unlocked on one episode\), and every other setting falls below\. No choice ofNN—including the smallest visible surface we test—turns Voyager’s library into a net asset relative to running with primitives only\. Third, the full library also underperforms the no\-codebook baseline on this seed, consistent with the regression pattern reported in Table[7](https://arxiv.org/html/2608.11338#A10.T7)\.

Taken together, these results indicate that Voyager’s regression under standard zombie density is not a pure context\-bloat phenomenon; the*composition*of the induced library matters more than the headcount of skills exposed at rollout time\.

## Appendix LUse of AI Assistants

We used ChatGPT and Claude as assistants for writing \(grammar correction, clarity edits, and suggestions on presentation\) and coding \(implementation and debugging of experimental infrastructure\)\. All scientific ideas, experimental designs, analyses, claims, and final text were reviewed and approved by the authors\.