Harness作为语言：一个极简代理框架，具备最大表达能力

# Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity
Source: [https://arxiv.org/html/2609.26891](https://arxiv.org/html/2609.26891)
Joshua Liu††thanks:Equal contributionMateja VukelicNicole Shen Supriya Lall Amitayush ThakurAffiliation:MIT CSAILAffiliation:Independent ResearcherAlex Zhang Omar Khattab Jonathan Light Armando Solar\-LezamaAffiliation:MIT CSAILAffiliation:Independent Researcher

###### Abstract

Modern language\-model agents are built around theagent loop, where the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between tool calls and observing their output\. However, certain workflows currently require additional engineering beyond the agent loop itself, such as memory systems and self\-improving systems\. We built an LLM agent framework, JAZ, to explore the extent to which a minimal harness that is little more than theagent loop itselfcan accomplish tasks these specialized systems are built for\. JAZ exposes a single LLM\-based primitiveinvokeand provides a set of built\-in hooks that allow the programmer to apply constraints and monitoring\. Generalizing existing code\-mode agent loops,invokeis the simplest loop that satisfies two defining properties: \(1\) the LLM can write arbitrary executable code that can include recursiveinvoke; \(2\) everything visible to the LLM — all inputs toinvokeas well as its interaction history with the code environment — are variables in the code environment\. We motivate our design from first principles, viewinginvokeas a language primitive representing a function whose implementation is provided at runtime by an LLM every time it is called\. To validate the design of our coreinvokeprimitive, we evaluateinvoke— with only prompting, no manually designed tools, harness, or external systems \(e\.g\., memory or the file system\) — on workflows traditionally implemented through specialized external harnesses\. On long\-horizon workflows requiring recall beyond the context window, JAZinvokeoutperforms Letta \(MemGPT\) by 8% at half its cost on the recall\-heavy portion of StuLife\. On continual self\-improvement, JAZinvokeoutperforms ACE by 4% at a lower cost on AppWorld\. Our framework code is located at[https://github\.com/jaz\-lang/jaz](https://github.com/jaz-lang/jaz)and our evaluation code is located at[https://github\.com/jaz\-lang/jaz\-evals](https://github.com/jaz-lang/jaz-evals)\.

Figure 1:In JAZ,invokeacts as a function whose implementation is provided each time it’s called — by the LLM in a Python REPL\. Everything — all named inputs toinvokeand the REPL history itself — are variables available in the REPL\.## 1Introduction

Language model agents are typically built around a coreagent loopin which the model repeatedly interacts with an environment through actions and observations\. Increasingly expressive agent loop paradigms have been proposed over time\. Earlier ReAct\([Yao et al\., 2022](https://arxiv.org/html/2609.26891#bib.bib4)\)gives the model full freedom to decide which action to take at every step of a workflow\. CodeAct\([Wang et al\., 2024b](https://arxiv.org/html/2609.26891#bib.bib5)\)generalizes it by providing the model the expressivity to orchestrate workflows with executable code in a REPL, and typical implementations\([Chase, 2023](https://arxiv.org/html/2609.26891#bib.bib21);[Roucher et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib6)\)also allow the user to provide Python objects the agent can interact with in its REPL\. Increasing expressivity has unlocked new capabilities\. Most notably, the RLM\([Zhang et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib7)\)demonstrated that CodeAct with subagents, without any external tools or harness, can outperform LLMs on long\-context tasks by providing the input to the agent as a Python object it can manipulate in its REPL\. However, agentic capabilities such as long\-horizon memory and self\-improvement remain implemented through specialized external harnesses layered on top of these loops\.

We study whether a minimal harness that is little more than theagent loop itselfcan exhibit these capabilities\. For this, we built an LLM agent framework, JAZ, whose core primitiveinvokeis a minimal language\-level abstraction that generalizes CodeAct\([Wang et al\., 2024b](https://arxiv.org/html/2609.26891#bib.bib5)\)and RLMs\([Zhang et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib7)\)\([Figure1](https://arxiv.org/html/2609.26891#S0.F1)\)\. To derive theinvokeagent loop from first principles, we consider augmenting any Turing\-complete programming language with a new primitiveinvokerepresenting a function whose body is generated by an LLM at runtime, and we illustrate how tail\-recursive invocations ofinvokenaturally induce an agent loop with two key properties\. First, the model writes arbitrary executable code in thisinvoke\-augmented language, allowing tools and recursive subagents to be orchestrated through ordinary program control flow\. In other words, like RLMs, recursive subagents are the default\. Second,everythingvisible to the model is also a variable in the code environment\. While modern implementations of CodeAct and RLMs let the user pass data and tools into the agent’s REPL,invokeadditionally makes the user prompt and REPL history variables in the REPL\.

We show thatinvoke, with little more than pure prompting, can realize agentic workflows traditionally implemented through specialized harnesses\. On long\-horizon tasks requiring recall far beyond the model’s context window, JAZinvokeis prompted to delegate to a subagent whenever its context runs out, while keeping a reference to the full conversation history\. With GPT\-5\.4 nano, it achieves 70% on the recall\-heavy subset of StuLife\([Cai et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib15)\), outperforming both CodeAct\+subagents \(32%\) and the specialized long\-horizon harness Letta\([Packer et al\., 2023](https://arxiv.org/html/2609.26891#bib.bib19)\)\(62%\)\. On AppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib12)\), JAZinvokeself\-improves across the sequence of test tasks by iteratively modifying prompts and skills inside its REPL based on test feedback\. With a GPT\-5\.4 top\-levelinvokeand GPT\-5\.4 nano sub\-invokes, it achieves 74%, outperforming CodeAct \(68%\), CodeAct\+subagents \(71%\), and a specialized self\-improvement harness, ACE\([Zhang et al\., 2025b](https://arxiv.org/html/2609.26891#bib.bib17)\)\(70%\)\. Together, these results show that prompting a minimal agent loop is sufficient to elicit long\-horizon and self\-improvement capabilities, rather than requiring specialized external harnesses\.

Our main contributions are:

- •We introduce and formalizeinvoke, an LLM\-based programming language construct that represents a function whose implementation is provided by the LLM at call time\.
- •We introduce JAZ, an agent framework whose core primitive is theinvokeagent loop\. An expressive hook system enables utilities such as observability, budget control, validation, and general modifications to the agent loop\.
- •We show empirically that the additional expressivity of JAZinvokeenables long\-horizon and self\-improvement workflows without external systems, tools, or harness components\. With prompting, JAZinvokeoutperforms CodeAct\+subagents and specialized harnesses at a similar or lower cost\.

Figure 2:In the functional version ofinvoke, the LLM writes a complete implementation for every invocation\. A closed agent loop arises from tail\-recursiveinvoke\.
## 2invoke: An LLM\-based Language Primitive

To motivate the design of JAZ from first principles, we consider a general, language\-agnostic way to augment any existing programming language with an LLM\-based primitive, which we callinvoke\.

[Figure2](https://arxiv.org/html/2609.26891#S1.F2)shows an example execution of thefunctionalversion of theinvokeprimitive\.invokeis thought of as a function that can take as inputs arbitrary named arguments\. These inputs can serve various purposes\. Some are inputs in the traditional sense \(dfin[fig\.1](https://arxiv.org/html/2609.26891#S0.F1)a,datain[fig\.2](https://arxiv.org/html/2609.26891#S1.F2)a\), some are the specification \(taskin[1](https://arxiv.org/html/2609.26891#S0.F1)a,specin[2](https://arxiv.org/html/2609.26891#S1.F2)a\), and some are tools the agent can call \(web\_searchin[1](https://arxiv.org/html/2609.26891#S0.F1)a,toolin[2](https://arxiv.org/html/2609.26891#S1.F2)a\)\. However,invoketreats all inputs the same and makes no distinction between prompts, tools, and “regular” inputs\.

Different from a regular function whose implementation is provided in the static code, the implementation ofinvokeis generated by the model every time it is called \([Figure2](https://arxiv.org/html/2609.26891#S1.F2)b\)\. The model sees these inputs as a compact string representation, and writes code that fills in the body ofinvoke\.

###### Definition 1\.

Given any Turing\-complete language, augmenting it withinvokecreates an augmented language with a new primitive\. Its syntax is that of a function callinvoke\(\.\.\.\)with arbitrary named inputs; the semantics of the function call is to call a language model with a string representation of all the inputs, and the language model produces code \(in the augmented language\) that becomes the body of the function, which is executed\. \(Appendix[A](https://arxiv.org/html/2609.26891#A1)gives the formal definition for lambda calculus\.\)

Compared to the usual LLM primitive with signaturestr\-\>str,invokeis a generalization that takes arbitrary inputs in the base language and outputs arbitrary objects in the base language\. As such,invokecan be viewed as the language\-agnostic version of an LLM that is also applicable to languages that do not have a string type \(e\.g\., lambda calculus\)\. Since the only “strings” that exist ineverylanguage are expressions in the language itself, the underlying language model most naturally generates an expression in the language that is interpreted as the one\-time implementation ofinvoke\.

In principle,invokeis already at least as expressive as a closed agent loop, which can be expressed as tail\-recursiveinvoke\([Figure2](https://arxiv.org/html/2609.26891#S1.F2)b–c\)\. To emulate a closed agent loop where the model sees the output of its code and decides its next action based on that, it could write a tail\-recursive call toinvokethat passes in the inputs and the interaction history with the agent loop \([Figure2](https://arxiv.org/html/2609.26891#S1.F2)b\)\. This delegates to a subagent whose context contains the same inputs but also the history so far \([Figure2](https://arxiv.org/html/2609.26891#S1.F2)c\), thus behaving as the next step of the agent loop\.

In practice, however, this is not a practical approach because it is memory inefficient in Python, and benefiting from LLM caching optimizations requires an exact prefix relationship among consecutive LLM queries\. Thus, instead of having the model write the tail\-recursive call, we can let the harness automatically append it and apply tail\-call optimization, resulting in an actual code\-mode agent loop where the LLM writes code in a REPL\. This agent loop has two distinguishing properties that are direct consequences of the definition of functionalinvoke\([Figures1](https://arxiv.org/html/2609.26891#S0.F1)and[2](https://arxiv.org/html/2609.26891#S1.F2)\):

1. 1\.Thelanguageof the model’s output is the augmented language, which already containsinvoke\. Thus, by default, the model can write not just arbitrary code in the original language, but also code that callsinvoke, which acts as subagents\. We call these recursiveinvokecallssub\-invokes\.
2. 2\.Everythingthe language model sees is a variable that it can reference in its code\. - •This includesallinputs toinvoke, treating all inputs equally regardless of their purpose\. Thus, an input intended as the prompt can be accessed as a variable, just as an input intended as a tool\. - •TheREPL history itselfis also a variable in the REPL\.

Property 1 makesinvokea variant of CodeAct\+subagents or RLMs\. Property 2 above — especially the “REPL history is a REPL variable” aspect — distinguishesinvokefrom existing code\-mode agents \(e\.g\., smolagents\([Roucher et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib6)\), RLMs\([Zhang et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib7)\)\), which treat the user prompt and REPL history only as a list of messages shown to the agent, not as variables the agent can programmatically interact with\.

We call an agent loop satisfying the two properties above aninvokeagent loop, or alternatively theimperativeversion ofinvoke, paralleling the contrast between while loops in imperative programming and recursion in functional programming\.

## 3JAZ: An LLM Agent Framework Based oninvoke

We built JAZ, an agent framework that implements theinvokeagent loop as its core language model primitive\. JAZ additionally implements the following systems to make it easier for users to build agents using theinvokeprimitive\.

##### Dynamic scoping\.

Instead of requiring that every input toinvokebe provided as an explicit argument, we provide a primitivescopethat creates a scope of variables whereinvokecalls within the scope automatically receive those variables, including all their recursive sub\-invokes\. In other words, variables specified byscopeare subject todynamic scopingforinvokecalls\. In JAZ,scopetakes the form of a Python context manager\. The most common use case is to make a tool available not just to the top\-level agent, but also to all its recursive subagents:

fromjazimportinvoke,scope

defweb\_search\(\.\.\.\):

\.\.\.

withscope\(web\_search=web\_search\):

invoke\(task="Usesubagentstofind100agentframeworkpapers\."\)

Although traditional programming prefers static scoping over dynamic scoping, only dynamic scoping is appropriate forinvoke\. Static scoping enables reusing a function in different contexts without surprising changes in behavior due to a changing context\. However,invokedoes not have a static scope as it does not have a static implementation, and there is no reuse as every call toinvokegenerates a new implementation\.

##### Hooks\.

Hooks allow a user to extendinvokeby providing callbacks that observe or influence the agent as it is running\. JAZ provides the following built\-in hooks:

- •Observability:PrintLogger,FileLogger,TrajectoryRecorder,LangfuseTracing,JaegerTracing\.
- •Resumability:TrajectoryRecorder,TrajectoryReplay
- •Resource control:BudgetPool,IterationLimit,RecursionLimit,BudgetForcing,ContextWindowWarning\.
- •Validation:ReturnType,ValidateReturn,ValidateREPLCode\.

Users can also extend JAZ by writing their own custom hooks \(Appendix[B](https://arxiv.org/html/2609.26891#A2)\)\.

Just like input variables, hooks can be either local to aninvokeor be subject to dynamic scoping\. Hooks passed as explicit positional arguments toinvokeare local, while a hook activated as a context manager is scoped\. In the following example, the scopedBudgetPool\(max\_cost=5\)hook applies a budget of $5 shared across allinvokecalls under the context manager, including both top\-level calls and all their recursive sub\-invokes\. On the other hand, the localReturnType\(float\)enforces the return typefloatonly for its top\-levelinvokeand not for its sub\-invokes\.

fromjazimportinvoke

fromjaz\.hooksimportReturnType,BudgetPool

withBudgetPool\(max\_cost=5\):

invoke\(

ReturnType\(float\),

question="Whatisthevalueof\.\.\.?"

\)

invoke\(another\_question="Whyis\.\.\.?"\)

##### Configuration system\.

A configuration in JAZ has a simple signature:Config\(llm:BaseLLM,repl:BaseREPL,protocol:BaseProtocol\)\. Thellmcomponent configures the model, thereplcomponent configures the REPL, and theprotocolcomponent configures the seam that connects the two, such as parsing the code out of the model’s raw response and formatting the REPL output for the model\. JAZ’s default protocol treats the full raw response as the code with no parsing, and truncates the REPL output if it’s too long but otherwise applies no additional formatting\. Similar to variables and hooks, a configuration can also be either local or subject to dynamic scoping: aConfigOverride\(llm=\.\.\.,repl=\.\.\.,protocol=\.\.\.\)passed positionally toinvokeapplies only to thatinvokeand not to any of its sub\-invokes, whereas activating it as a context manager \(withConfigOverride\(\.\.\.\):invoke\(\.\.\.\)\) applies it to allinvokecalls under thewithblock, including all recursive sub\-invokes\.

## 4Case Studies

In this section, we explore how the expressivity of JAZinvokeallows us to replace specialized external harnesses with prompting for two types of workflows: 1\) long\-horizon workflows requiring recall beyond the context window; 2\) continual learning and self\-improvement across a sequence of tasks\. These workflows are traditionally targeted with external harness components, such as memory systems for long\-range recall, and specialized self\-improving systems for continual self\-improvement\. Here, we use JAZ to demonstrate that a minimal LLM\-based primitive, theinvokeagent loop, can perform these workflows as well without external systems, tools, or harness components\.

We compareinvokewith agent loops under a similar prompt\-only setup, as well as domain\-specific harnesses\. The agent loops we compare with are CodeAct\([Wang et al\., 2024b](https://arxiv.org/html/2609.26891#bib.bib5)\)and CodeAct\+subagents, where “subagent” is understood to mean generalist subagents, as opposed to specialized subagents with designated roles\. To minimize confounds due to implementation differences, we reimplement CodeAct and CodeAct\+subagents in JAZ as a controlled ablation ofinvoke, achieved by a hook that removes the user prompt and REPL history variables frominvoke’s REPL and makes the minimal equivalent change to prompt templates\. This was necessary as established implementations were found to perform worse than our reimplementation due to defects in prompting and REPL implementation\. See Appendix[C](https://arxiv.org/html/2609.26891#A3)for more details\.

Both our environments involve completing a sequence of tasks, so a return guard \(ValidateReturnin JAZ,final\_answer\_checksin smolagents\) is used to make the agent keep working if it stops before all tasks have been attempted\.

### 4\.1Long\-Horizon with Long\-Range Recall

Here, we focus on long\-horizon environments with the following properties:

- •The task requires so many iterations \(e\.g\., thousands\) that the conversation history would exceed the LLM’s context window multiple times if no subagents were used\.
- •Making the optimal decision frequently requires recalling some detail in the conversation history multiple context windows in the past\. Compaction would lose the detail, and long\-range dependencies make it difficult to cleanly decompose the workflow into subtasks\.

In JAZ, when the LLM’s context window fills up, the agent can delegate the remainder of the task to a subagent while losslessly passing in the entire conversation history by reference\. We call this patterntail\-recursive delegation:

returninvoke\(

\.\.\.,

prev\_history=globals\(\)\.get\("prev\_history",\[\]\)\+\_\_history\_\_,

prev\_progress\_summary=\.\.\.,

next\_steps=\.\.\.,

\.\.\.,

\)

Note that while the top\-level agent passes its\_\_history\_\_to the subagent, any subagent must combine its\_\_history\_\_with theprev\_historypassed to it before delegating further\. In our experiments, we elicit this behavior with the built\-inContextWindowWarninghook, which appends a user message to the agent’s conversation when the model’s context window passes a certain threshold\. The user message tells the agent to delegate all remaining work to a subagent and shows a code template to follow — see Appendices[C\.2](https://arxiv.org/html/2609.26891#A3.SS2)and[E\.1\.2](https://arxiv.org/html/2609.26891#A5.SS1.SSS2)\.

The general pattern above subsumes various forms of long\-horizon context management\. For example,compaction\([Wang et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib24);[Anthropic, 2026b](https://arxiv.org/html/2609.26891#bib.bib18)\)corresponds to passing in aprev\_progress\_summaryonly\. Automated context management \(e\.g\., Chroma Context\-1\([Bashir et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib2)\)\) corresponds to passing in a filtered version of\_\_history\_\_with certain entries removed\. More generally, access toinvokegives the agent control over the next LLM call’s inputs and context, whereas access to\_\_history\_\_and a code environment makes it efficient to build this context off of the agent’s current context\.

Table 1:Results on the long\-horizon environment StuLife\([Cai et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib15)\)\. Methods marked “per\-task” solve each StuLife task individually with no state or memory persistence across tasks\. We report both pass rate \(fraction of scored tasks with a perfect score\) and average score, for both the full scored set of 939 tasks \(“all”\) and the subset of 207 tasks that require recall of information delivered over 50 tasks ago \(“far recall”\)\. We use GPT\-5\.4 nano \(high\) across all methods\. We report the mean and its standard error over 3 independent runs\. See[Table3](https://arxiv.org/html/2609.26891#A4.T3)for all individual data points\.StuLife\(all\)StuLife\(far recall\)prompt\-only? \*Pass \(%\)Score \(%\)Pass \(%\)Score \(%\)Cost \($\)CodeActJAZ\(per task\)\([Wang et al\., 2024b](https://arxiv.org/html/2609.26891#bib.bib5)\)✓52\.5±\\pm0\.359\.2±\\pm0\.124\.8±\\pm0\.625\.8±\\pm0\.54\.4±\\pm0\.1CodeAct\+subagents\([Roucher et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib6)\)✓30\.2±\\pm4\.833\.6±\\pm5\.220\.6±\\pm0\.622\.0±\\pm0\.99\.4±\\pm1\.4CodeAct\+subagentsJAZ\([Zhang et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib7)\)✓60\.0±\\pm1\.168\.1±\\pm1\.032\.0±\\pm2\.333\.9±\\pm2\.013\.1±\\pm0\.9Letta Agent\([Packer et al\., 2023](https://arxiv.org/html/2609.26891#bib.bib19)\)✗70\.9±\\pm0\.581\.0±\\pm0\.561\.8±\\pm2\.367\.0±\\pm2\.442\.1±\\pm1\.6JAZinvoke✓72\.6±\\pm0\.181\.6±\\pm0\.169\.9±\\pm1\.873\.6±\\pm1\.518\.3±\\pm0\.3
\* See Appendix[C\.1](https://arxiv.org/html/2609.26891#A3.SS1)for our definition of a prompt\-only setup with a generic agent loop

##### Experiments\.

We evaluate on the long\-horizon benchmark StuLife\([Cai et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib15)\), which contains an ordered sequence of 1284 tasks that simulate a variety of activities a college student does over the course of a semester, many of which require recalling information given in prior tasks\. A typical episode takes 7000–8000 environment interactions, and a typical code\-mode agent requires 3000–5000 LLM calls\. Among the 1284 tasks, 939 are graded, of which 207 require “far recall”, which we define to be recall of information delivered in a previous task separated by over 50 tasks\.

We evaluate JAZinvokeand multiple baselines using GPT\-5\.4 nano as the model\. Since JAZinvokeis a general agent harness, our primary baseline is CodeAct with subagents, also a general agent harness\. The CodeAct \(per\-task\) baseline runs CodeAct on each individual StuLife task with no cross\-task memory persistence and thus serves as a no\-memory baseline\. We also compare with a domain\-specific harness: Letta Agent, the newest version of MemGPT\([Packer et al\., 2023](https://arxiv.org/html/2609.26891#bib.bib19)\), is a conversation agent with memory\. See[SectionC\.2](https://arxiv.org/html/2609.26891#A3.SS2)for experiment details\. Evaluation results are reported in[Table1](https://arxiv.org/html/2609.26891#S4.T1)\.

##### Analysis\.

To understand how JAZinvokeoutperforms both CodeAct\+subagents and Letta on tasks requiring long\-range recall, we analyzed task \#1282/1284, whichinvokesolved in all three runs, CodeAct\+subagents solved zero times, and Letta solved once\.

This task is a final exam multiple\-choice question\. Each of the four options describes applying a fictional protocol, and the agent has to choose the option with the correct application based on what previous lectures taught about these protocols\. The full task text is given in Appendix[F\.1](https://arxiv.org/html/2609.26891#A6.SS1)\.

To solve this task, the agent needs to first recall the lectures that taught those protocols, then apply them in the scenarios given in the question to deduce the correct option\. The lecture that taught the protocol involved in the correct option \(D\) was introduced in task \#785/1284, 497 tasks ago\.

On one of the JAZinvokeruns, when the agent reached this task, it was already at recursion depth 70 after having performed tail\-recursive delegation 69 times\.111Recursion depth is defined as 0 in user code, 1 in the top\-level agent, 2 in the first level of subagents, so on and so forth\.However, every delegation preserves the full history completely through theprev\_history, which has accumulated every prior agent’s\_\_history\_\_\. The agent spent one turn searching all protocol names in itsprev\_history\. All the relevant lectures were correctly retrieved, and the agent answered correctly in its next turn\.

CodeAct with subagents did not search at all\. The user prompt to the agent warns that these exam questions are about fictional protocols, so it has to recall the actual lecture content to answer correctly\. The user prompt also contains the instructions for maintaining the agent’s own REPL history, compensating for CodeAct’s lack of the\_\_history\_\_variable\. However, without access to the user prompt as a variable in the REPL, every agent has to copy its user prompt into the subagent, which ended up being lossy, and both instructions were lost after dozens of delegations\. The agent ended up not having access to any form of REPL history variable and also doesn’t know the question tests recall, so it answered based on its real\-world knowledge, choosing \(A\), which was incorrect in StuLife’s fictional world\.

Takeaway 1:\(long\-horizon version\)Wheneverythingthe agent sees in its context is in REPL variables, the agent can pass them by reference to a subagent in scenarios that need it \(e\.g\., tail\-recursive delegation\)\. This is more reliable than copying their contents by hand\.

Letta Agent used itsconversation\_searchtool to search through its conversation history for exact protocol names mentioned in the question\. Using a combination of keyword search and vector search, the tool returned the top\-ranked hits, which were either the quiz question itself, or earlier messages about similarly named but different protocols\. In this scenario where the agent needs exact substring matching, the only tool Letta had \(conversation\_search\) did not support it\.

Takeaway 2:\(long\-horizon version\)Domain\-specific harnesses encode assumptions that make them work well in many situations, but they become rigid in environments that break those assumptions\. Instead of a search tool over a large object, having full programmatic access to its raw interface can recover the flexibility needed in such environments\.

### 4\.2Continual Self\-Improvement

We studycontinual self\-improvement \(CSI\), where a system improves itself over a given ordered sequence of tasks\.

One form of self\-improving system separates ameta\-agentfrom asolver\-agent, where the meta\-agent optimizes various aspects of the solver agent\. Targets of optimization can include the prompt\([Zhang et al\., 2025b](https://arxiv.org/html/2609.26891#bib.bib17);[Suzgun et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib3);[Agrawal et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib16)\), executable skills\([Wang et al\., 2024a](https://arxiv.org/html/2609.26891#bib.bib14)\), and even the entirety of the agent’s source code\([Lee et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib20)\)\.

In JAZ, the top\-levelinvokehas control over all inputs to pass into a sub\-invoke and can thus act as a meta\-agent by optimizing those inputs over a sequence of tasks\. A typical optimization iteration involves constructing inputs such as the solver agent’s prompt and skills \(high\-level tools\), running them on a task, and inspecting the results, as described by the following pseudocode:

instructions="\.\.\.Whendone,return‘\(answer,\_\_history\_\_\)‘"

defskill\(\.\.\.\):\.\.\.

answer,trajectory=invoke\(

task=get\_next\_task\(\),

instructions=instructions,

skill=skill,

\)

eval\_report=complete\_task\(answer\)

print\(eval\_report\)

print\(trajectory\)

The prompt used to guide the top\-level agent to conduct continual self\-improvement is given in Appendix[E\.2](https://arxiv.org/html/2609.26891#A5.SS2)\. It describes the high\-level continual self\-improvement workflow and provides information about JAZ needed for proper implementation of self\-improvement\. The prompt does not include any code examples or templates\.

In our experiments comparing with CodeAct run on each individual task, to remove the confound of differing subagent access, our CodeAct\+subagents and JAZinvokeimplementations capped recursion depth to 2 \(i\.e\., solver subagents do not have access to subsubagents\)\.

##### Experiments\.

We evaluate JAZinvoke’s ability to continually self\-improve across the full sequence of 417 tasks from thetest\-challengesplit of the AppWorld benchmark\([Trivedi et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib12)\)\. The task ordering was shuffled with seed 42\. AppWorld is an environment that simulates the API interfaces of 9 apps people commonly use on their phone or computer, and tasks require the agent to use those APIs to complete tasks for the user\. In our continual self\-improvement setup, the full set of tasks is given to the agent in a fixed order, with full test feedback from each task so that the agent can learn from its mistakes\.

We compare JAZinvokewith an equivalent minimal prompt\-only setup of CodeAct with subagents, as well as Agentic Context Engineering \(ACE\)\([Zhang et al\., 2025b](https://arxiv.org/html/2609.26891#bib.bib17)\), a specialized harness for continual self\-improvement\. We also compare with baselines that solve each benchmark task individually: the benchmark’s official CodeAct baseline222Known as the “official ReAct” baseline with a REPL tool, which is equivalent to CodeAct, as well as our own implementation of CodeAct\. These represent baseline methods that do not get the chance to use test feedback to improve across tasks\.

[Table2](https://arxiv.org/html/2609.26891#S4.T2)reports the mean and standard error of our evaluation results\.[Table4](https://arxiv.org/html/2609.26891#A4.T4)lists all individual data points, and additionally reports the median and 78% confidence interval for self\-improving methods\. We find that JAZinvokeoutperforms all other methods both in the mean and in the median\.

Table 2:Results on the fulltest\-challengesplit of AppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib12)\)\. Methods marked “per\-task” solve each AppWorld task individually with no continual learning across tasks\. The solver agent in all methods uses GPT\-5\.4 nano \(high\) and the meta\-agent in CSI methods uses GPT\-5\.4 \(high\)\. We report the mean and standard error overnnindependent runs, wheren=3n=3for non\-self\-improving methods andn=6n=6for self\-improving methods to account for higher variance\. See[Table4](https://arxiv.org/html/2609.26891#A4.T4)for all individual data points\.AppWorld \(test\-challenge\)prompt\-only? \*TGC \(%\)SGC \(%\)Cost \($\)Meta $Solver $CodeActAppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib12)\)\(per\-task\)✗†48\.2±\\pm1\.620\.4±\\pm3\.216\.6±\\pm0\.2—16\.6±\\pm0\.2CodeActJAZ\([Wang et al\., 2024b](https://arxiv.org/html/2609.26891#bib.bib5)\)\(per\-task\)✓67\.5±\\pm1\.243\.4±\\pm2\.510\.2±\\pm0\.2—10\.2±\\pm0\.2CodeAct\+subagentsJAZ\([Zhang et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib7)\)✓71\.1±\\pm1\.347\.6±\\pm2\.121\.6±\\pm1\.87\.3±\\pm1\.714\.3±\\pm0\.9ACE\([Zhang et al\., 2025b](https://arxiv.org/html/2609.26891#bib.bib17)\)on CodeActJAZ✗69\.9±\\pm1\.447\.1±\\pm1\.330\.6±\\pm0\.517\.1±\\pm0\.313\.5±\\pm0\.3JAZinvoke✓74\.2±\\pm2\.151\.1±\\pm3\.720\.9±\\pm3\.79\.9±\\pm3\.810\.9±\\pm0\.6
\* See Appendix[C\.1](https://arxiv.org/html/2609.26891#A3.SS1)for our definition of a minimal, prompt\-only setup with a generic agent loop †The official CodeAct baseline uses a prompt that contains AppWorld\-specific workflow guidance\.

##### Analysis\.

We qualitatively compare the continual self\-improvement behavior of prompt\-only JAZinvoke, prompt\-only CodeAct\+subagents, and the specialized self\-improvement harness ACE\([Zhang et al\., 2025b](https://arxiv.org/html/2609.26891#bib.bib17)\)\.

JAZinvoke’s top\-levelinvokeacted as a meta\-agent that launched subagents to solve individual tasks\. It dispatched tasks in batches to subagents, investigated failure traces and metrics in between batches, updated inputs to subagents \(prompts and skills\), and adaptively updated the batch size\. To access subagent trajectories, the meta\-agent simply had to tell the subagent to return its\_\_history\_\_variable available in its REPL\.

CodeAct\+subagents followed the same workflow as JAZinvoke, but it had to get around the limitation that subagents didn’t have access to their own REPL histories\. The meta\-agent told the subagent to return its REPL history, and depending on the prompt it wrote, the subagent either hard\-coded the entire history in the final return statement, or incrementally built it by appending it at each turn\. Both approaches tended to be lossy, and the additional overhead contributed to output token inefficiency and hence a higher solver agent cost\.

Takeaway 1:\(self\-improvement version\)Wheneverythinga subagent sees in its context is in REPL variables, the subagent can pass them by reference to its caller in scenarios that need it \(e\.g\., for caller observability into subagent behavior\)\. This is more reliable than explicitly building their contents\.

ACE is a human\-designed self\-improvement loop that optimizes the solver agent’s prompt after every task it attempts\. It uses areflectorLLM call to reflect on the solver agent’s trajectory, and then acuratorto distill findings into bullet point items that are added to theplaybookthat forms part of the solver agent’s prompt\. The ACE reflector and curator were very costly, costing 10 times more per task than JAZinvoke\. We restricted learning to the first 42 \(10%\) tasks to balance cost and performance \(Appendix[C\.3](https://arxiv.org/html/2609.26891#A3.SS3)\), and ACE still ended up costing more than JAZinvokelearning across the entire test set\. The cost efficiency of JAZinvokecomes from breaking free of the rigid workflow prescribed by ACE\. In JAZinvoke, the meta\-agent has the freedom to batch multiple tasks together and save on meta\-agent model calls\. The meta\-agent also selectively reads parts of trajectories when needed, thus saving its context from being filled with the full trajectory of every subagent\.

Another limitation of ACE is that it only optimizes the prompt, whereas the JAZinvokemeta\-agent optimizes inputs toinvokegenerally, which can also include skills\.

Takeaway 2:\(self\-improvement version\)Hand\-designed workflows encode assumptions that make them work well in many situations, but they become rigid in environments that break those assumptions\. Providing a strong LLM the full expressivity of code and theinvokeprimitive can recover the flexibility needed in such environments\.

## 5Related Work

##### Agent loop paradigms\.

Most modern LLM agents are built around an*agent loop*in which the model interacts repeatedly with an environment through actions and observations\. In traditional tool\-calling or ReAct\-style agents\([Yao et al\., 2022](https://arxiv.org/html/2609.26891#bib.bib4)\), each action invokes atoolwith hard\-coded inputs\. More recent*code\-mode*or*CodeAct\-style*agents instead place the model inside a code environment \(REPL\), where tools and subagents are now ordinary functions that code can reference\([Wang et al\., 2024b](https://arxiv.org/html/2609.26891#bib.bib5)\)\.

Recursive language models\([Zhang et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib7)\)demonstrate that a code\-mode agent loop itself — with only prompting, no external tools or harness components other than recursive subagents — can handle some long\-context tasks better than long\-context models themselves\. Our work extends this line of work by demonstrating that, with a natural generalization — all agent inputs and the interaction history with the code environment are variables in the code environment — a code\-mode agent loop with only prompting can also handle long\-horizon tasks and perform continual self\-improvement\.

##### Multi\-agent systems\.

Multi\-agent systems for task decomposition have been built for solving complex long\-horizon tasks\([Hong et al\., 2023](https://arxiv.org/html/2609.26891#bib.bib8);[Qian et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib9);[Chen et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib11);[Zelikman et al\., 2023](https://arxiv.org/html/2609.26891#bib.bib10);[Light et al\., 2025b](https://arxiv.org/html/2609.26891#bib.bib13)\)\. Traditionally involving significant hand design, such as subagent roles and specific orchestration patterns, more recent multi\-agent systems\([Anthropic, 2026a](https://arxiv.org/html/2609.26891#bib.bib31);[Zhang et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib7)\)take a more minimal approach involving generalist subagents exposed as regular tools\. While traditionally subagents have been used for well\-defined subtasks, we demonstrate that they are also useful for long workflows without well\-defined subtasks, and can also be used as the solver agent in a self\-improving system with a meta\-agent/solver\-agent split\.

##### Compaction and memory systems\.

For non\-decomposable long\-horizon workflows such as long conversations, the predominant approach has beencompactionandmemory systems\. Compaction methods include simple textual summarization\([Wang et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib24);[Packer et al\., 2023](https://arxiv.org/html/2609.26891#bib.bib19)\), and recent work has also explored applying KV cache compaction techniques to long\-horizon agentic workflows\([Liu et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib23)\)\. To enable retrieval of previous conversation context that has been lost due to compaction, memory systems were developed that provide the agent an interface to query conversation history through keyword or embedding\-based search\([Packer et al\., 2023](https://arxiv.org/html/2609.26891#bib.bib19);[Chhikara et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib25)\)\. In JAZ, we showed that full programmatic access to a variable\_\_history\_\_is sufficient for memory retrieval\.

##### Self\-improving agentic systems\.

Self\-improving systems take a variety of forms, including agents that modify their own source code\([Zhang et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib26);[Zelikman et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib27);[Robeyns et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib28);[Yin et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib29)\)and systems involving a meta\-agent that optimizes a target agent\([Khattab et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib22);[Agrawal et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib16);[Hu et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib30);[Zhang et al\., 2025b](https://arxiv.org/html/2609.26891#bib.bib17);[Lee et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib20)\)\. Aspects of the target agent to be optimized can include the prompt\([Khattab et al\., 2024](https://arxiv.org/html/2609.26891#bib.bib22);[Agrawal et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib16)\), skills\([Wang et al\., 2024a](https://arxiv.org/html/2609.26891#bib.bib14);[Light et al\., 2025a](https://arxiv.org/html/2609.26891#bib.bib1)\), and entire agent programs\([Hu et al\., 2025](https://arxiv.org/html/2609.26891#bib.bib30);[Lee et al\., 2026](https://arxiv.org/html/2609.26891#bib.bib20)\)\. In JAZ, we demonstrated that the top\-level agent and subagent in a multi\-agent system can act as the optimizer meta\-agent and the optimized solver agent in self\-improvement, optimizing inputs to the solver agent such as its prompt and skills\.

## 6Conclusion

We argued that many behaviors commonly attributed to specialized agent harnesses — including memory and self\-improvement — can instead emerge from the expressivity of a sufficiently powerful computational substrate\. From this perspective, agent frameworks primarily differ not in the existence of distinct capabilities, but in the restrictions they impose on workflow construction, state expressivity, and recursive computation\.

This suggests a shift in how agent systems are designed and studied\. Rather than engineering increasingly specialized external orchestration mechanisms, an alternative approach is to provide a model with a maximally expressive execution environment, and train agents to construct and adapt their own workflows\. Under this view, agentic behavior becomes less a collection of manually designed patterns and more a consequence of computational expressivity itself\.

## Acknowledgments and Disclosure of Funding

This project was funded by the following grants and institutions: Zup/Itaú, NSF Grant No\. 1918839\.

## References

- Agrawalet al\.\(2025\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang,et al\.GEPA: reflective prompt evolution can outperform reinforcement learning\.arXiv preprint arXiv:2507\.19457\.Cited by:[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.p2.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Anthropic \(2026a\)AnthropicClaude Code v2\.1\.278\.Note:[https://github\.com/anthropics/claude\-code](https://github.com/anthropics/claude-code)Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px2.p1.1)\.
- Anthropic \(2026b\)AnthropicCompaction\.Note:[https://platform\.claude\.com/docs/en/build\-with\-claude/compaction](https://platform.claude.com/docs/en/build-with-claude/compaction)Accessed 2026\-05\-07Cited by:[§4\.1](https://arxiv.org/html/2609.26891#S4.SS1.p3.1)\.
- Bashiret al\.\(2026\)H\. Bashir, K\. Hong, P\. Jiang, and Z\. ShiChroma Context\-1: training a self\-editing search agent\.Technical reportChroma\.External Links:[Link](https://trychroma.com/research/context-1)Cited by:[§4\.1](https://arxiv.org/html/2609.26891#S4.SS1.p3.1)\.
- Caiet al\.\(2026\)Y\. Cai, Y\. Hao, J\. Zhou, H\. Yan, Z\. Lei, R\. Zheng, Z\. Han, Y\. Yang, J\. Li, Q\. Pan,et al\.A benchmark for self\-evolving agents via experience\-driven lifelong learning\.ICLR 2026 submission\.Cited by:[Table 3](https://arxiv.org/html/2609.26891#A4.T3),[Table 3](https://arxiv.org/html/2609.26891#A4.T3.9.1.3.1.1),[§1](https://arxiv.org/html/2609.26891#S1.p3.1),[§4\.1](https://arxiv.org/html/2609.26891#S4.SS1.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2609.26891#S4.T1)\.
- Chase \(2023\)H\. ChaseLangChain v0\.0\.97\.Note:[https://github\.com/langchain\-ai/langchain/releases/tag/v0\.0\.97](https://github.com/langchain-ai/langchain/releases/tag/v0.0.97)Cited by:[§1](https://arxiv.org/html/2609.26891#S1.p1.1)\.
- Chenet al\.\(2024\)J\. Chen, H\. Tang, Z\. Chu, Q\. Chen, Z\. Wang, M\. Liu, and B\. QinDivide\-and\-conquer meets consensus: unleashing the power of functions in code generation\.Advances in Neural Information Processing Systems37,pp\. 67061–67105\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px2.p1.1)\.
- Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: building production\-ready AI agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px3.p1.1)\.
- Honget al\.\(2023\)S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin,et al\.MetaGPT: meta programming for a multi\-agent collaborative framework\.InThe Twelfth International Conference on Learning Representations,Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px2.p1.1)\.
- Huet al\.\(2025\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 21344–21377\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Khattabet al\.\(2024\)O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. V\. A, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. PottsDSPy: compiling declarative language model calls into state\-of\-the\-art pipelines\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=sY5N0zY5Od)Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Leeet al\.\(2026\)Y\. Lee, R\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. FinnMeta\-harness: end\-to\-end optimization of model harnesses\.arXiv preprint arXiv:2603\.28052\.Cited by:[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.p2.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Lightet al\.\(2025a\)J\. Light, M\. Cai, W\. Chen, G\. Wang, X\. Chen, W\. Cheng, Y\. Yue, and Z\. HuStrategist: self\-improvement of LLM decision making via bi\-level tree search\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 27032–27099\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Lightet al\.\(2025b\)J\. Light, W\. Cheng, B\. Riviere, Y\. Wu, M\. Oyamada, M\. Wang, Y\. Yue, S\. Paternain, and H\. ChenDISC: dynamic decomposition improves LLM inference scaling\.InAdvances in Neural Information Processing Systems \(NeurIPS 2025\),External Links:2502\.16706,[Link](https://arxiv.org/abs/2502.16706)Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2026\)Y\. Liu, J\. Ji, L\. An, R\. Jain, G\. Polatkan, S\. Zhu, and S\. ChangPractical online KV cache compaction for llm agents: an empirical study\.arXiv preprint arXiv:2608\.00902\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px3.p1.1)\.
- Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards llms as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§C\.2\.2](https://arxiv.org/html/2609.26891#A3.SS2.SSS2.p3.1),[Table 3](https://arxiv.org/html/2609.26891#A4.T3.9.15.1.1),[§1](https://arxiv.org/html/2609.26891#S1.p3.1),[§4\.1](https://arxiv.org/html/2609.26891#S4.SS1.SSS0.Px1.p2.1),[Table 1](https://arxiv.org/html/2609.26891#S4.T1.10.6.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px3.p1.1)\.
- Qianet al\.\(2024\)C\. Qian, W\. Liu, H\. Liu, N\. Chen, Y\. Dang, J\. Li, C\. Yang, W\. Chen, Y\. Su, X\. Cong,et al\.ChatDev: communicative agents for software development\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 15174–15186\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px2.p1.1)\.
- Robeynset al\.\(2025\)M\. Robeyns, M\. Szummer, and L\. AitchisonA self\-improving coding agent\.InICLR 2025 Workshop on Scaling Self\-Improving Foundation Models,External Links:[Link](https://openreview.net/forum?id=rShJCyLsOr)Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Roucheret al\.\(2025\)A\. Roucher, A\. V\. del Moral, T\. Wolf, L\. von Werra, and E\. Kaunismäkismolagents: a smol library to build great agentic systems\.\.Note:[https://github\.com/huggingface/smolagents](https://github.com/huggingface/smolagents)Cited by:[§C\.2\.2](https://arxiv.org/html/2609.26891#A3.SS2.SSS2.p5.1),[Table 3](https://arxiv.org/html/2609.26891#A4.T3.9.7.1.1.1),[§1](https://arxiv.org/html/2609.26891#S1.p1.1),[§2](https://arxiv.org/html/2609.26891#S2.p8.1),[Table 1](https://arxiv.org/html/2609.26891#S4.T1.10.4.1.2)\.
- Suzgunet al\.\(2026\)M\. Suzgun, M\. Yuksekgonul, F\. Bianchi, D\. Jurafsky, and J\. ZouDynamic cheatsheet: test\-time learning with adaptive memory\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 7080–7106\.Cited by:[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.p2.1)\.
- Trivediet al\.\(2024\)H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. BalasubramanianAppWorld: a controllable world of apps and people for benchmarking interactive coding agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 16022–16076\.Cited by:[Table 4](https://arxiv.org/html/2609.26891#A4.T4),[Table 4](https://arxiv.org/html/2609.26891#A4.T4.20.3.1.1.1),[§1](https://arxiv.org/html/2609.26891#S1.p3.1),[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.26891#S4.T2),[Table 2](https://arxiv.org/html/2609.26891#S4.T2.16.3.1.1.1)\.
- Wanget al\.\(2024a\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=ehfRiF0R3a)Cited by:[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.p2.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Wanget al\.\(2025\)Q\. Wang, Y\. Fu, Y\. Cao, S\. Wang, Z\. Tian, and L\. DingRecursively summarizing enables long\-term dialogue memory in large language models\.Neurocomputing639,pp\. 130193\.Cited by:[§4\.1](https://arxiv.org/html/2609.26891#S4.SS1.p3.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px3.p1.1)\.
- Wanget al\.\(2024b\)X\. Wang, Y\. Chen, L\. Yuan, Y\. Zhang, Y\. Li, H\. Peng, and H\. JiExecutable code actions elicit better LLM agents\.InForty\-first International Conference on Machine Learning,Cited by:[Table 3](https://arxiv.org/html/2609.26891#A4.T3.9.3.1.1),[Table 4](https://arxiv.org/html/2609.26891#A4.T4.20.7.1.1),[§1](https://arxiv.org/html/2609.26891#S1.p1.1),[§1](https://arxiv.org/html/2609.26891#S1.p2.1),[Table 1](https://arxiv.org/html/2609.26891#S4.T1.10.3.1),[Table 2](https://arxiv.org/html/2609.26891#S4.T2.16.4.1.1),[§4](https://arxiv.org/html/2609.26891#S4.p2.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px1.p1.1)\.
- Yaoet al\.\(2022\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InThe eleventh international conference on learning representations,Cited by:[§1](https://arxiv.org/html/2609.26891#S1.p1.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px1.p1.1)\.
- Yinet al\.\(2025\)X\. Yin, X\. Wang, L\. Pan, L\. Lin, X\. Wan, and W\. Y\. WangGödel Agent: a self\-referential agent framework for recursively self\-improvement\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 27890–27913\.External Links:[Link](https://aclanthology.org/2025.acl-long.1354/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1354),ISBN 979\-8\-89176\-251\-0Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Zelikmanet al\.\(2023\)E\. Zelikman, Q\. Huang, G\. Poesia, N\. Goodman, and N\. HaberParsel: algorithmic reasoning with language models by composing decompositions\.Advances in Neural Information Processing Systems36,pp\. 31466–31523\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px2.p1.1)\.
- Zelikmanet al\.\(2024\)E\. Zelikman, E\. Lorch, L\. Mackey, and A\. T\. KalaiSelf\-taught optimizer \(STOP\): recursively self\-improving code generation\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=46Zgqo4QIU)Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2025a\)A\. L\. Zhang, T\. Kraska, and O\. KhattabRecursive language models\.arXiv preprint arXiv:2512\.24601\.Cited by:[Table 3](https://arxiv.org/html/2609.26891#A4.T3.9.11.1.1),[Table 4](https://arxiv.org/html/2609.26891#A4.T4.20.11.1.1),[§1](https://arxiv.org/html/2609.26891#S1.p1.1),[§1](https://arxiv.org/html/2609.26891#S1.p2.1),[§2](https://arxiv.org/html/2609.26891#S2.p8.1),[Table 1](https://arxiv.org/html/2609.26891#S4.T1.10.5.1),[Table 2](https://arxiv.org/html/2609.26891#S4.T2.16.5.1.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px1.p2.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2026\)J\. Zhang, S\. Hu, C\. Lu, R\. Lange, and J\. CluneDarwin Gödel machine: open\-ended evolution of self\-improving agents\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 104223–104294\.Cited by:[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2025b\)Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li,et al\.Agentic context engineering: evolving contexts for self\-improving language models\.arXiv preprint arXiv:2510\.04618\.Cited by:[§C\.1](https://arxiv.org/html/2609.26891#A3.SS1.SSS0.Px1.p3.1),[§C\.3\.2](https://arxiv.org/html/2609.26891#A3.SS3.SSS2.p3.1),[Table 4](https://arxiv.org/html/2609.26891#A4.T4.20.19.1.1),[§1](https://arxiv.org/html/2609.26891#S1.p3.1),[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.SSS0.Px1.p2.1),[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2609.26891#S4.SS2.p2.1),[Table 2](https://arxiv.org/html/2609.26891#S4.T2.16.6.1.1),[§5](https://arxiv.org/html/2609.26891#S5.SS0.SSS0.Px4.p1.1)\.

## Appendix AFormal definition ofinvoke

Given a programming language, we augment its syntax with theinvokeprimitive, which is a function call where the body of the function is written dynamically at runtime by a language model \(𝖫𝖬\\mathsf\{LM\}\)\. We writeinvoke\(x1:=e1,…,xk:=ek\)\(x\_\{1\}:=e\_\{1\},\\ldots,x\_\{k\}:=e\_\{k\}\)for the functional agent call oninputsxi:=eix\_\{i\}:=e\_\{i\}, wherexix\_\{i\}are variable names andeie\_\{i\}are expressions in the language\. To execute a program in this augmented language,invoke\(x1:=v1,…,xk:=vk\)\(x\_\{1\}:=v\_\{1\},\\ldots,x\_\{k\}:=v\_\{k\}\)is evaluated as follows:

1. 1\.Query a language model with a promptPPthat contains a compact string representationP=⟨x1:=v1,…,xk:=vk⟩P=\\langle x\_\{1\}:=v\_\{1\},\\ldots,x\_\{k\}:=v\_\{k\}\\rangleof the inputs\. These inputs can include the user prompt, tools, and more generally any object in the language allowed as an argument to a function call\.
2. 2\.The model returns codeC=π⁡\(𝖫𝖬⁡\(P\)\)C=\\pi\(\\mathsf\{LM\}\(P\)\)in the same language that would be valid as the body of a function with input parametersx1,…,xkx\_\{1\},\\ldots,x\_\{k\}\. Here,π⁡\(⋅\)\\pi\(\\cdot\)parses the code from the model’s raw output\. Note that all the context that the model sees is contained within the promptPPthat serializes the inputs\.
3. 3\.Execute this function on the given inputs\.

Since the language itself includes theinvokeprimitive, the model\-generated code may recursively contain furtherinvokecalls\.

Under this framework, there are three key aspects to design: the language itself, theserializerformatting the inputs into a prompt \(represented by the angle bracket notation “⟨⋅⟩\\langle\\cdot\\rangle”\), and themodel output parserπ⁡\(⋅\)\\pi\(\\cdot\)\. A good serializer does not display the raw contents of a large object; instead it gives the model enough information so that it could extract information needed by interacting with it\. The model output parser traditionally involves parsing out a code block from the model’s natural language output\. However, for simplicity, it can just be the identity functionπ⁡\(y\)=y\\pi\(y\)=y, which was the choice we made in JAZ — the entirety of the model’s output is parsed as Python code, and any natural language prose \(e\.g\., the agent’s plan\) is included in leading comments inside the code\.

Formally, consider the lambda calculus, the simplest functional programming language, defining just function application and function abstraction\. The following formal definition is easily generalized to other programming languages\.

Augmenting the lambda calculus with theinvokeprimitive results in this augmented grammar of expressions \(terms\):

e::=c\|x\|e​e\|λ​x\.e⏟lambda calculus\|invoke​\(x:=e\)\+\.\{e::=\\underbrace\{c\\mid x\\mid e\\ e\\mid\\lambda x\.\\,e\}\_\{\\text\{lambda calculus\}\}\\mid\\text\{\\lstinline\{\{\\lst@@@set@language\\lst@@@set@numbers\\lst@@@set@frame\\lst@@@set@rulecolor\\lst@@@set@language\\normalsize\{\\@listingGroup\{ltx\_lst\_emph\}\{\\color\[rgb\]\{1,0,0\.6523\}\{invoke\}\}\}\}\}\}\}\\ \(x:=e\)^\{\+\}\.\(1\)The standard expressions in lambda calculus are constants \(cc\), variables \(xx\), function calls \(e​ee\\ e\), and function definitions \(λ​x\.e\\lambda x\.\\,e\)\. The additional syntax is the new primitiveinvoke​\(x:=e\)\+\{\\text\{\\lstinline\{\{\\lst@@@set@language\\lst@@@set@numbers\\lst@@@set@frame\\lst@@@set@rulecolor\\lst@@@set@language\\normalsize\{\\@listingGroup\{ltx\_lst\_emph\}\{\\color\[rgb\]\{1,0,0\.6523\}\{invoke\}\}\}\}\}\}\}\\ \(x:=e\)^\{\+\}\. Here, eacheedenotes an arbitrary expression in the language, and\(x:=e\)\+\(x:=e\)^\{\+\}means “one or more named arguments”\.

The operational semantics for call\-by\-value \(CBV\) evaluation become

1\(λx\.e\)v→e\[v/x\]‾e1→e1′2e1​e2→e1′​e2‾e→e′3v​e→v​e′‾⏟lambda calculusσ=\[x→:=v→\]4invokeσ→\(λx→\.π\(𝖫𝖬⟨σ⟩\)\)v→‾\{\\underbrace\{\(\\lambda x\.\\,e\)\\ v\\to e\[v/x\]\\hskip 14\.72241pte\_\{1\}\\ e\_\{2\}\\to e\_\{1\}^\{\\prime\}\\ e\_\{2\}e\_\{1\}\\to e\_\{1\}^\{\\prime\}\\hskip 14\.72241ptv\\ e\\to v\\ e^\{\\prime\}e\\to e^\{\\prime\}\}\_\{\\text\{lambda calculus\}\}\\hskip 14\.72241pt\\text\{\\lstinline\{\{\\lst@@@set@language\\lst@@@set@numbers\\lst@@@set@frame\\lst@@@set@rulecolor\\lst@@@set@language\\normalsize\{\\@listingGroup\{ltx\_lst\_emph\}\{\\color\[rgb\]\{1,0,0\.6523\}\{invoke\}\}\}\}\}\}\}\\ \\sigma\\to\(\\lambda\\vec\{x\}\.\\,\\pi\(\\mathsf\{LM\}\\langle\\sigma\\rangle\)\)\\ \\vec\{v\}\\sigma=\[\\vec\{x\}:=\\vec\{v\}\]\(2\)where the metavariablevvdenotes a value \(the result after evaluating an expression\), andσ\\sigmais the mappingx1:=v1,…,xk:=vkx\_\{1\}:=v\_\{1\},\\ldots,x\_\{k\}:=v\_\{k\}\. The first 3 rules are standard, defining how to evaluate function calls \(rule 1\) and evaluation order \(rules 2 and 3\)\. Theinvokeaddition \(rule 4\) defines how to evaluate the functional agent call primitive, formalizing the notion of filling in the body of a function dynamically at runtime\.𝖫𝖬​⟨σ⟩\\mathsf\{LM\}\\langle\\sigma\\rangledenotes the output of the language model given theprompt⟨σ⟩\\langle\\sigma\\rangle, i\.e\., a string representation of all the inputs\.π\\piis themodel output parserthat parses the model’s raw output\. The extracted code appears as the body of a functionλ​x→\.\(…\)\\lambda\\vec\{x\}\.\\,\(\\ldots\)with argumentsx1,…,xkx\_\{1\},\\ldots,x\_\{k\}, and the function is invoked on valuesv1,…,vkv\_\{1\},\\ldots,v\_\{k\}as given by the mappingσ\\sigma\.

## Appendix BExtensibility via Hooks in JAZ

In JAZ, users can define custom hooks by providing functions to call at various points in the agent’s lifecycle\. Instead of allowing a hook to arbitrarily modify agent state, it is only allowed to emit static composable effects as allowed by the particular event\.

- •InvokeEnter,InvokeSend,InvokeComplete,InvokeExit
- •LLMQueryEnter,LLMQuerySend,LLMQueryComplete,LLMQueryExit
- •LLMQueryRetry\(only fires if an LLM query did not succeed and is retried\)
- •REPLExecEnter,REPLExecSend,REPLExecComplete,REPLExecExit

An event handler at\[Operation\]Entersees the inputs to\[Operation\]and can emit effects that modify those inputs\. An event handler at\[Operation\]Sendsees the input modifications and resultant modified inputs, and can emit an effect that replaces calling an\[Operation\]with supplying its output\. An event handler at\[Operation\]Completesees the output and can emit effects that modify the output\. In addition, at every event other thanInvokeExitandLLMQueryRetry, the handler may emit an effect that aborts theinvokeit fires in\.

Every\[Operation\]Enteropens a span that always closes with\[Operation\]Exit, carrying its outcome \(Completed/Aborted/Failed\)\.\[Operation\]Sendand\[Operation\]Completeare conditional: an abort or failure skips to\[Operation\]Exit\.

When multiple hooks are active at once, effects that modify arenotapplied as they are emitted, as that would cause different hooks to potentially see different event fields in a way that depends on the order in which the event is dispatched to all the active handlers\. Instead, the event is dispatched to all handlers at once, and all effects are collected and composed with a composition rule that minimally depends on composition order\. Conflicting writes that cannot be resolved are refused with an error, and then the final resolved modifications are all applied at once\.

Hooks can also communicate via a global message board called theblackboard, reading its contents off the event and writing by emitting an effect\.

All handlers also have a view of ambient state: all active hooks, all scoped variables \([Section3](https://arxiv.org/html/2609.26891#S3)\), the active configuration \([Section3](https://arxiv.org/html/2609.26891#S3)\), and the blackboard\.

All built\-in hooks provided by JAZ are built using the hook interface we’ve described in this section\.

## Appendix CExperiment Details

### C\.1Evaluation Protocol

To ensure fair comparison, accurate measurements, and proper control of confounds, we follow the following protocol in our evaluations\.

##### Task/method decoupling\.

The hyperparameter with the most degrees of freedom is the prompt\. If each \(method, task\) pair gets its own prompt, there is a high risk of overfitting the method to the task, and different methods under comparison may receive different degrees of overfitting that confound the comparison\. This is especially dangerous with self\-improvement, where allowing a self\-improvement method’s meta\-prompt to include task\-specific information licenses hard\-coding aspects of the optimal solver prompt and skills in the meta\-prompt instead of forcing the meta\-agent to learn these on its own, which would no longer isolate the effect of self\-improvement itself\.

To mitigate these risks, our experiments split the user prompt into two disjoint parts:instructionsandguidance\. The environment\-providedinstructionsis method\-agnostic — it provides the task instructions, describes the environment, and optionally provides workflow guidance as long as it is method\-agnostic\. In self\-improvement experiments, to isolate the ability to self\-improve starting from a minimal seed,instructionsonly contains the task instructions and environment description and provides no workflow guidance at all\. On the other hand, the method\-providedguidanceis task\-agnostic — it provides workflow guidance as appropriate for the method, as long as it is applicable to all tasks in the domain\. In JAZ, the prompt componentsinstructionsandguidanceare passed as separate variables to JAZinvoke\.

Note that we consider “\[agent loop\] long\-horizon” and “\[agent loop\] self\-improvement” to be separate methods that can provide their own prompts, but the prompt must still be task\-agnostic\. Thus, the long\-horizon prompt \(Appendices[E\.1\.1](https://arxiv.org/html/2609.26891#A5.SS1.SSS1)and[E\.1\.2](https://arxiv.org/html/2609.26891#A5.SS1.SSS2)\) and the self\-improvement prompt \(Appendix[E\.2](https://arxiv.org/html/2609.26891#A5.SS2)\) are different prompts, but the long\-horizon prompt is generally applicable to long\-horizon environments with no StuLife\-specific information, and the self\-improvement prompt is generally applicable to self\-improvement with no AppWorld\-specific information\. When the official implementation of a baseline \(e\.g\., ACE\[[31](https://arxiv.org/html/2609.26891#bib.bib17)\]\) violates this rule \(Appendix[C\.3](https://arxiv.org/html/2609.26891#A3.SS3)\), we make the minimal modification so that it conforms\.

##### Necessary differences between method implementations are implemented\.

Aguidanceprompt written for one method is typically not applicable to another method and thus must be adapted\. For example, compared to JAZinvoke, CodeAct\+subagents does not expose the user prompt as variables \(instructionsandguidance\) in the REPL and does not automatically maintain a\_\_history\_\_variable in the REPL\. Thus, for long\-horizon, we adapt theguidanceprompt to ask the agent to maintain its history by itself, and we adapt the context window warning to ask the agent to hard\-code the contents ofinstructionsandguidanceinstead of referring to their variables\. Similarly, for self\-improvement, we adapt theguidanceprompt to ask the meta\-agent to ask the solver agent to maintain its history by itself so that it could return it at the end\.

##### Unnecessary differences between method implementations are minimized\.

When two methods each have tunable hyperparameters — especially the prompt, but also the configuration — those that can be kept the same across the two methods without introducing bias towards one method or the other are kept the same\. For example, JAZ truncates inputs to 50000 characters by default, so in the smolagents implementation of CodeAct\+subagents, the context window warning asks the agent to wrap its history list in an object whose customreprtruncates to 50000 characters\. For prompts, the JAZinvokeprompt and the CodeAct\+subagentsJAZprompt are byte\-identical everywhere other than in places where a difference is necessary \(see above\)\.

##### Cost measurement\.

For accurate cost measurement, we controlled for systematic errors in cache hit rate in two ways\. First, we set a distinct prompt cache key for each run\. Second, we allowed at most 3 parallel runs at any moment under the same OpenAI organization\. To estimate the systematic uncertainty due to uncontrollable factors in the state of OpenAI server\-side cache, we rannnparallel agents on a simple toy environment forn=1,3,4,5n=1,3,4,5, both with and without another agent running on a different environment at the same time\. We calculated the mean and its standard error of the cache hit rate\. We found a notable decrease in cache hit rate of over 5% fromn=4n=4ton=5n=5, while changes in cache hit rate forn≤4n\\leq 4were within 2%\. We therefore fixn=3n=3for all our experiments, and estimate the systematic error in resultant cost estimates to be less than 2%\. Experiments that involved 6 independent runs \(i\.e\., self\-improvement\) were run in 2 batches of 3\.

##### Rules for the prompt\-only setup with a generic agent loop\.

The “minimal” setup we apply to all general agent loop methods in our experiments applies the following restrictions to all methods equally:

- •The only tools are those provided by the benchmark’s environment itself\. More generally, the agent is not allowed to access anything other than the given environment, such as the file system or the Internet\.
- •There is no loop\-external logic that modifies the loop itself \(e\.g\., truncating the conversation history, injecting variables in the REPL\)\. Howevermonitoringis allowed: budget control, step limits, recursion limit, completion guard, and context\-window limit warnings\. We also allow model configuration, i\.e\., the root agent may use a different model from subagents\.
- •The system prompt — available to the root agent as well as subagents — contains agent loop mechanics and tool docstrings\.
- •The user prompt — passed to the root agent only — contains a method\-agnostic task prompt supplied by the environment and a task\-agnostic guidance prompt supplied by the method\.

The details of implementing a general agent loop may depend on the domain \(“long\-horizon” or “self\-improvement”\), but must be task\-agnostic\. For example, the method’s contribution to the user prompt and context window warning must be applicable to “long\-horizon” in general, or “self\-improvement” in general, with StuLife\- or AppWorld\-specific information provided only by the environment in a method\-agnostic manner\.

### C\.2Long\-Horizon: StuLife

#### C\.2\.1Environment Details

The StuLife repository provides a JSON file containing the 1284 tasks of the environment\. We implement an environment that delivers these tasks and exposes a modified set of tools to the agent\. We reuse the official grader for binary scoring and implement our own partial score metric\. Our modified set of tools removes tools that are not needed for any tasks, and replaces a few tools that are always used the same way with a single tool for the same purpose \(see code for details\)\. The original tools also only output unstructured text and never raise on invalid input, so they are wrapped to take in structured inputs, output structured outputs, and perform input validation to be friendly to code\-mode agents\. Their docstrings are also updated accordingly, and code examples are included in docstrings to improve clarity\.

#### C\.2\.2Method Details

For our long\-horizon experiments with JAZ, we have two prompts: one is the warning emitted when the LLM’s context window is 70% full, injected with theContextWindowWarninghook, and the other is part of the regular user prompt given to the top\-level agent\. The prompt injected by theContextWindowWarningis given in Appendix[E\.1\.2](https://arxiv.org/html/2609.26891#A5.SS1.SSS2)\. The prompt shown to the top\-levelinvokeis provided inside a regular input namedguidanceand its contents are given in Appendix[E\.1\.1](https://arxiv.org/html/2609.26891#A5.SS1.SSS1)\.

We implement CodeAct\+subagents as a minimal modification of JAZ to minimize confounds\. We apply theCodeActhook that removes\_\_history\_\_and the user prompt\(s\) from the REPL of everyinvoke, and make the minimal modification to the long\-horizonguidanceandContextWindowWarningprompt to accommodate the removal\. In particular, theguidanceprompt additionally shows the agent how to maintain its own REPL history by appending intermediate outputs to a listoutput\_history, and the history search section is updated to use this variable\. The context window warning text is adapted to use thisoutput\_historyvariable, and instead of showing the user prompt variables \(instructionsandguidance\) being passed to the subagent by reference, the warning text demonstrates that they should be hard\-coded as literal strings copied from the agent’s context\.

For Letta \(MemGPT\)\[[16](https://arxiv.org/html/2609.26891#bib.bib19)\], we use the latest version \(v0\.16\.8\) of the Python Letta server and use the newest Letta agent with parallel tool calling\. Letta provides two search backends\. The SQL backend caused most search results to turn up empty: it applies exact substring search to theentiresearch query, yet the agent issues search queries that are more appropriate for conventional search engines and thus frequently do not appear as an exact substring of anything in the history\. We thus used Letta’s more powerful search backend, Turbopuffer, which implements hybrid search, combining BM25 keyword\-based search and vector\-embedding\-based search\. We had to modify the environment to deliver each task as a user message as opposed to letting the agent call a tool to retrieve the task, as tool outputs are not stored in Letta’s searchable history\. We tried multiple prompts for Letta, including the empty prompt and variations of JAZ’sguidanceprompt adapted to Letta\. We found that all but one of the prompts consistently resulted in an unrecoverable infinite loop where the agent is stuck on an empty\-description task and searches for the task description forever\. Our results were reported using the one prompt that consistently did not result in this pathology\.

For the CodeAct per\-task baseline, our setup is similar to that of CodeAct\+subagents, except each task gets its own CodeAct agent\. It is implemented using theCodeActhook on JAZinvoke, and theRecursionLimit\(depth=1\)hook is used to disable subagents\.

The smolagents\[[19](https://arxiv.org/html/2609.26891#bib.bib6)\]implementation of CodeAct\+subagents mirrors that of CodeAct\+subagentsJAZ\. The managed agents feature is used for subagents, and prompts are adapted to smolagents\. Various workarounds were needed due to limitations in smolagents:

- •Theadditional\_argsthat seed the subagent’s REPL display theirreprwith no truncation, so the agent is instructed to wrap its history in a custom object with areprthat truncates\.
- •The restricted Python interpreter that smolagents implements for its Python REPL resolves a name not by its presence in the namespace, but by fuzzy matching\. Thus, code examples shown in our prompts had variable names chosen to make sure that anyexceptNameErrorbehaves correctly\.

Other issues in smolagents were noticed that did not have an obvious workaround\. We did not fix them as that would modify the smolagents implementation beyond the tunable surface\. For example, we did not fix the issue in smolagents’ Python interpreter where certain functions from themathmodule \(e\.g\.,log\) are hard\-coded to resolve to the math function, even when the agent redefines it \(e\.g\., aloghelper function that stores output to the history and prints\)\. We also did not fix the smolagents’ system prompt, which calls subagents “team members” and incorrectly says “this team member is a real human”, causing the agent to launch a subagent whenever it is missing information, under the false impression that this “human team member” can go find the information for the agent\. The latter prompt defect explains almost all of the underperformance of smolagents compared to JAZ’s implementation of CodeAct\+subagents\. The subagent’s role description is a tunable hyperparameter, so we attempted to override the system prompt defect by specifying in the subagent role description that it has no access to humans, but our attempt was unsuccessful\.

### C\.3Continual Self\-Improvement

#### C\.3\.1Environment Details

The task sequence consists of the full set of 417 tasks from thetest\-challengesplit of AppWorld\. We randomly permute the task sequence with a fixed seed \(seed 42\)\.

AppWorld does not expose its environmentapisdirectly as methods a user can call, but instead only indirectly through a Python REPL interface provided by the benchmark authors\. Technical incompatibilities make it difficult to directly replace theinvokePython REPL with AppWorld’s REPL\. Instead, we construct an explicitapisobject that mirrors theapisobject in AppWorld’s REPL\. A method call is resolved by writing the corresponding single line of code that gets executed by AppWorld’s REPL, and the result is retrieved from the namespace of AppWorld’s REPL and returned\.

AppWorld provides prompts used in their baseline evaluations, but they mix the environment description, task instructions, and workflow guidance into one prompt\. We removed workflow guidance and separated the remaining content, placing the environment description into the docstring of the environment objectapisand task instructions into theinstructionscomponent of the user prompt\. We found that the original description of the task submission format had an ambiguity that caused a 15% drop in the performance of the baseline \(Appendix[C\.3\.2](https://arxiv.org/html/2609.26891#A3.SS3.SSS2)\), so we made sure ourinstructionsprompt was clear and complete and exhibited no ambiguities\.

#### C\.3\.2Method Details

In JAZinvoke, we provide a prompt that guides the rootinvoketo perform continual self\-improvement\. The prompt is provided as a regular inputguidanceto the rootinvoke\. Its contents are given in Appendix[E\.2](https://arxiv.org/html/2609.26891#A5.SS2)\. We use a scoped \(context manager\)ConfigOverrideto set the model to GPT\-5\.4 nano, but use a localConfigOverrideat the top\-level to selectively override its own model to GPT\-5\.4\. We do not set aContextWindowWarninghook as the top\-level agent is able to process all tasks within its context window\. We use aRecursionLimithook to cap the recursion depth to 2 so that the solver agent doesn’t get access to subagents\. This ensures that the measured difference relative to the non\-self\-improving CodeAct baseline is attributable to self\-improvement alone and is not confounded by solver agents’ additional access to subagents\.

We implement CodeAct\+subagents as a minimal modification of JAZ to minimize confounds\. We apply theCodeActhook that removes\_\_history\_\_and the user prompt\(s\) from the REPL of everyinvoke, and make the minimal modification to the self\-improvementguidanceprompt to accommodate the removal\. In particular, the meta\-agent is no longer told to ask the solver agent to return its\_\_history\_\_\. Instead, the meta\-agent is told to “return its REPL history as a list where each entry holds the code it ran, the output the code printed, and any exception that occurred”\.

In ACE\[[31](https://arxiv.org/html/2609.26891#bib.bib17)\], the original paper reports that results are insensitive to the bullet dedup threshold, tested with the values\{0\.5,0\.7,0\.9\}\\\{0\.5,0\.7,0\.9\\\}\. We set the dedup threshold to 0\.7, the midpoint of the range\. We also differ from the original setup in three ways for fairer comparison\. First, the reference implementation of ACE hard\-codes AppWorld\-specific meta\-guidance in its prompts, giving the reflector and curator examples of real AppWorld solver agent failure modes and hard\-coding the optimal responses to these failure modes\. This violates our evaluation protocol, which requires prompts a method supplies to be task\-agnostic to prevent privileged test environment knowledge from leaking into the prompt\. As a result, we made the minimal changes that removed AppWorld\-specific meta\-guidance from the reflector and curator prompts\. Second, while the original experiments set the reflector/curator model to the generator’s model, we set it to GPT 5\.4 to match JAZinvoke’s top\-level model, while we keep the generator’s model at GPT 5\.4 nano, matching JAZinvoke’s subagent model\. Third, the original online version of ACE runs the reflector and curator foreverytest task, which would’ve resulted in an estimated cost of over $180, nearly 10 times the total cost of JAZinvoke\. For a controlled cost comparison, exact cost parity would allow ACE to train on only 20 tasks, which may not be enough for ACE to perform meaningful self\-improvement\. Since the original paper reports noticeable gains from a training set of size 90 in offline adaptation mode, we chose a middle ground and allowed ACE to train on 10% of the test set \(42 tasks\) before freezing its prompt\.

For the CodeAct non\-self\-improving baseline, our setup is similar to that of CodeAct\+subagents, except each AppWorld task gets its own CodeAct agent\. It is implemented using theCodeActhook on JAZinvoke, and theRecursionLimit\(depth=1\)hook is used to disable subagents for comparability with the official baseline\.

The official baseline was run out\-of\-the\-box\. Although called “ReAct” by the benchmark authors, the agent writes arbitrary Python code that can call methods on the environment as functions, and is thus more properly “CodeAct”\. We do not classify it as a “prompt\-only setup with a generic agent loop” under our definition \(Appendix[C\.1](https://arxiv.org/html/2609.26891#A3.SS1)\), as the definition requires the prompt to not provide AppWorld\-specific workflow guidance, yet it does\. Ironically, the official baseline performsworsethan our CodeAct baseline that is free of AppWorld\-specific workflow guidance\. Two concrete deficiencies in the official baseline prompt were found to be contributing to the gap:

- •The prompt did not clearly explain the expected shape of the final submission for tasks that require that no answer be passed tocomplete\_task\. Many agents ended those tasks by submitting a report, which automatically failed the task despite having performed the task correctly\. The fraction of tasks that were graded as failed because the agent submitted an answer when the task was otherwise completed successfully is \(14\.9±\\pm1\.0\)%, the majority of the gap between the official baseline and our CodeAct baseline\. On the other hand, the environment description we used in all our other AppWorld experiments did successfully teach the expected shape, with only 1 out of 1251 CodeAct agents mistakenly submitting an answer for an action task\.
- •The teaching of theapis\.supervisor\.complete\_task\(status="fail"\)escape hatch caused agents to include it in conditional branches in their code\. The agent sometimes hits that branch and prematurely fails the task before doing any substantial work\. We estimate about 1% was lost for this reason\. On the other hand, the environment description we used in all our other AppWorld experiments omitscomplete\_task\(status="fail"\)\.

## Appendix DSupplemental Tables

Table 3:Expanded version of[Table1](https://arxiv.org/html/2609.26891#S4.T1): full results on the full sequence of tasks of StuLife\[[5](https://arxiv.org/html/2609.26891#bib.bib15)\]\. Here, in addition to the mean, we also report every individual data point\.StuLife\[[5](https://arxiv.org/html/2609.26891#bib.bib15)\]\(all\)StuLife\(far recall\)repPass \(%\)Score \(%\)Pass \(%\)Score \(%\)Cost \($\)CodeActJAZ\(per task\)\[[24](https://arxiv.org/html/2609.26891#bib.bib5)\]152\.359\.225\.126\.14\.6252\.258\.925\.626\.54\.2353\.059\.423\.724\.74\.3mean52\.5±0\.359\.2±0\.124\.8±0\.625\.8±0\.54\.4±0\.1CodeAct\+subagents\[[19](https://arxiv.org/html/2609.26891#bib.bib6)\]129\.531\.820\.322\.311\.0222\.225\.619\.820\.46\.6338\.943\.521\.723\.410\.6mean30\.2±4\.833\.6±5\.220\.6±0\.622\.0±0\.99\.4±1\.4CodeAct\+subagentsJAZ\[[29](https://arxiv.org/html/2609.26891#bib.bib7)\]162\.270\.036\.237\.813\.8259\.467\.728\.531\.011\.3358\.566\.631\.432\.914\.3mean60\.0±1\.168\.1±1\.032\.0±2\.333\.9±2\.013\.1±0\.9Letta Agent\[[16](https://arxiv.org/html/2609.26891#bib.bib19)\]171\.281\.662\.867\.742\.9271\.681\.465\.270\.738\.9370\.080\.057\.562\.544\.4mean70\.9±0\.581\.0±0\.561\.8±2\.367\.0±2\.442\.1±1\.6JAZinvoke172\.681\.670\.073\.718\.4272\.481\.572\.976\.217\.8372\.681\.766\.771\.018\.9mean72\.6±0\.181\.6±0\.169\.9±1\.873\.6±1\.518\.3±0\.3Table 4:Expanded version of[Table2](https://arxiv.org/html/2609.26891#S4.T2): full results on the fulltest\-challengesplit of AppWorld\[[21](https://arxiv.org/html/2609.26891#bib.bib12)\]\. Here, in addition to the mean, we also report every individual data point\. For self\-improving methods, we also report the median and its 78% confidence interval formed by the 2nd and 5th order statistics\.Notes:1\) The0\.00\.0uncertainty in the official baseline cost is by coincidence under only 3 reps — in the main table we report0\.20\.2, the quadrature over tasks of the SEM cost of each task over reps\. 2\) The substantially higher value of the median than the mean in JAZinvoke’s TGC and SGC is due to the outlier rep 5, which scored below the CodeAct baseline\.AppWorld \(test\-challenge\)repTGC \(%\)SGC \(%\)Cost \($\)Meta $Solver $CodeActAppWorld\[[21](https://arxiv.org/html/2609.26891#bib.bib12)\]\(per\-task\)146\.815\.816\.6—16\.6246\.518\.716\.6—16\.6351\.326\.616\.6—16\.6mean48\.2±1\.620\.4±3\.216\.6±0\.0—16\.6±0\.0CodeActJAZ\[[24](https://arxiv.org/html/2609.26891#bib.bib5)\]\(per\-task\)169\.148\.210\.4—10\.4268\.142\.49\.9—9\.9365\.239\.610\.1—10\.1mean67\.5±1\.243\.4±2\.510\.2±0\.2—10\.2±0\.2CodeAct\+subagentsJAZ\[[29](https://arxiv.org/html/2609.26891#bib.bib7)\]176\.056\.113\.93\.210\.6272\.948\.921\.16\.115\.0366\.742\.423\.18\.414\.8471\.548\.220\.22\.717\.6571\.047\.527\.213\.413\.8668\.642\.424\.010\.113\.9mean71\.1±1\.347\.6±2\.121\.6±1\.87\.3±1\.714\.3±0\.9median71\.268\.672\.9\{\}^\{72\.9\}\_\{68\.6\}47\.842\.448\.9\{\}^\{48\.9\}\_\{42\.4\}22\.120\.224\.0\{\}^\{24\.0\}\_\{20\.2\}7\.23\.210\.1\{\}^\{10\.1\}\_\{3\.2\}14\.413\.815\.0\{\}^\{15\.0\}\_\{13\.8\}ACE\[[31](https://arxiv.org/html/2609.26891#bib.bib17)\]on CodeActJAZ175\.352\.529\.916\.713\.2269\.546\.032\.217\.714\.6364\.746\.030\.117\.013\.1468\.143\.231\.718\.313\.4570\.746\.029\.216\.512\.8670\.748\.930\.316\.413\.9mean69\.9±1\.447\.1±1\.330\.6±0\.517\.1±0\.313\.5±0\.3median70\.168\.170\.7\{\}^\{70\.7\}\_\{68\.1\}46\.046\.048\.9\{\}^\{48\.9\}\_\{46\.0\}30\.229\.931\.7\{\}^\{31\.7\}\_\{29\.9\}16\.816\.517\.7\{\}^\{17\.7\}\_\{16\.5\}13\.313\.113\.9\{\}^\{13\.9\}\_\{13\.1\}JAZinvoke175\.852\.515\.16\.19\.0276\.354\.722\.29\.212\.9374\.648\.215\.14\.610\.5474\.655\.414\.32\.312\.0564\.534\.538\.428\.310\.1679\.661\.220\.09\.011\.1mean74\.2±2\.151\.1±3\.720\.9±3\.79\.9±3\.810\.9±0\.6median75\.274\.676\.3\{\}^\{76\.3\}\_\{74\.6\}53\.648\.255\.4\{\}^\{55\.4\}\_\{48\.2\}17\.615\.122\.2\{\}^\{22\.2\}\_\{15\.1\}7\.64\.69\.2\{\}^\{9\.2\}\_\{4\.6\}10\.810\.112\.0\{\}^\{12\.0\}\_\{10\.1\}
## Appendix EPrompts

### E\.1Long\-Horizon

#### E\.1\.1User promptguidance

#### IMPORTANT: Search yourprev\_historyto recall past information when available Do you have all the information to know the correct next action with certainty? If not, then*search for it*\. Ifprev\_historyis available to you, then it contains the REPL history of the previous agent working in this environment before they delegated to you\. To recall information from earlier in the session,*search for this information inprev\_history*\.*NOTE:*prev\_historyis*different*from the magic variable\_\_history\_\_\. Do NOT search\_\_history\_\_, since it’s the history of your current REPL session and it’s already visible to you in full, so searching\_\_history\_\_would be useless\. Instead, you must search*prev\_history*, the object that is only partially visible to you\.Follow these rules forprev\_historysearch:•Use targeted, concrete search terms that help uniquely find the search target\. Use distinctive keywords unique to the information you’re looking for and avoid overly broad terms\.•When you find a match, always display a context window around the hit — never truncate to just a prefix\.•Your ENTIRE next step is to search — do NOT write any code other thanprev\_historysearch\. Print theprev\_historysearch results and defer follow\-up work to later turns\.[⬇](data:text/plain;base64,IyBJZiBgcHJldl9oaXN0b3J5YCwgdGhlIHByZXZpb3VzIGFnZW50J3MgaGlzdG9yeSwgaXMgYXZhaWxhYmxlOgpmb3IgaSwgZW50cnkgaW4gZW51bWVyYXRlKHByZXZfaGlzdG9yeSk6ICAjIHNlYXJjaCBgcHJldl9oaXN0b3J5YCwgTk9UIGBfX2hpc3RvcnlfX2AhCiAgICByZXBsX291dHB1dCA9IGVudHJ5LnJlcGxfb3V0cHV0CiAgICAjIHNraXAgZW50cmllcyB3aXRoIHByaW9yIGhpc3Rvcnkgc2VhcmNoIG91dHB1dCwgd2hpY2ggcG9sbHV0ZXMgc2VhcmNoIHJlc3VsdHMKICAgIGlmICItLS0gZW50cnlbIiBpbiByZXBsX291dHB1dDoKICAgICAgICBjb250aW51ZQogICAgIyBzZWFyY2ggZm9yIHRhcmdldCBpbiB0aGUgZW50cnkncyBSRVBMIG91dHB1dAogICAgcG9zID0gcmVwbF9vdXRwdXQuZmluZCgic2VhcmNoIHRlcm0gaGVyZSIpCiAgICBpZiBwb3MgPj0gMDoKICAgICAgICAjIERpc3BsYXkgYSB3aW5kb3cgYXJvdW5kIHRoZSBzZWFyY2ggdGFyZ2V0CiAgICAgICAgc3RhcnQgPSBtYXgoMCwgcG9zIC0gMTAwMCkKICAgICAgICBlbmQgPSBtaW4obGVuKHJlcGxfb3V0cHV0KSwgcG9zICsgMjAwMCkKICAgICAgICBwcmludChmIi0tLSBlbnRyeVt7aX1dIChwb3Mge3Bvc30pIC0tLSIpCiAgICAgICAgcHJpbnQocmVwbF9vdXRwdXRbc3RhcnQ6ZW5kXSkKICAgICAgICBwcmludCgpCiMgU1RPUCBIRVJFOiBkbyBOT1Qgd3JpdGUgYW55IGNvZGUgYWZ0ZXIgYHByZXZfaGlzdG9yeWAgc2VhcmNoIC0gV0FJVCBmb3IgdGhlIG5leHQgdHVybiB0byBhY3Qgb24gdGhlIHNlYXJjaCByZXN1bHRz)fori,entryinenumerate\(prev\_history\):repl\_output=entry\.repl\_outputif"\-\-\-entry\["inrepl\_output:continuepos=repl\_output\.find\("searchtermhere"\)ifpos\>=0:start=max\(0,pos\-1000\)end=min\(len\(repl\_output\),pos\+2000\)print\(f"\-\-\-entry\[\{i\}\]\(pos\{pos\}\)\-\-\-"\)print\(repl\_output\[start:end\]\)print\(\)

#### E\.1\.2Context window warning

Your context window is close to full\. You must finish your REPL session now by delegating all remaining work to a subagent\.IMPORTANT NOTES:•You must raise your code’s timeout with a\# timeout: 86400pragma on the FIRST line of your code to prevent the subagent from timing out prematurely\.•You must give the subagent both the previous agent’s REPL history \(prev\_history\) if available, as well as your own REPL history \(\_\_history\_\_\), so that existing work is not lost\.Follow this template exactly:[⬇](data:text/plain;base64,IyB0aW1lb3V0OiA4NjQwMApyZXR1cm4gaW52b2tlKAogICAgIyBHaXZlIHRoZSBzdWJhZ2VudCB0aGUgaW5wdXQgdmFyaWFibGVzIHRoYXQgeW91IHdlcmUgZ2l2ZW4sIGBpbnN0cnVjdGlvbnNgIGFuZCBgZ3VpZGFuY2VgCiAgICBpbnN0cnVjdGlvbnM9aW5zdHJ1Y3Rpb25zLCAgIyBnaXZlIHRoZSBzdWJhZ2VudCB5b3VyIGBpbnN0cnVjdGlvbnNgIHZhcmlhYmxlCiAgICBndWlkYW5jZT1ndWlkYW5jZSwgICMgZ2l2ZSB0aGUgc3ViYWdlbnQgeW91ciBgZ3VpZGFuY2VgIHZhcmlhYmxlCiAgICAjIEdpdmUgdGhlIHN1YmFnZW50IGJvdGggdGhlIHByZXZpb3VzIGFnZW50J3MgUkVQTCBoaXN0b3J5ICh0aGUgYHByZXZfaGlzdG9yeWAgdmFyaWFibGUpCiAgICAjIGFuZCB5b3VyIG93biBSRVBMIGhpc3RvcnkgKHRoZSBgX19oaXN0b3J5X19gIHZhcmlhYmxlKQogICAgcHJldl9oaXN0b3J5PWdsb2JhbHMoKS5nZXQoInByZXZfaGlzdG9yeSIsIFtdKSArIF9faGlzdG9yeV9fLAogICAgIyBIYW5kIG92ZXIgYW55IHN0YXRlIHlvdSd2ZSBiZWVuIHRyYWNraW5nCiAgICBzdGF0ZT0uLi4sCiAgICAjIFN1bW1hcml6ZSB3aGF0IGhhcyBiZWVuIGRvbmUgYW5kIHdoYXQgcmVtYWlucyAoaGFyZC1jb2RlZCBzdHJpbmcpCiAgICBwcmV2X3Byb2dyZXNzX3N1bW1hcnk9IlNvIGZhciwgLi4uIiwKICAgICMgVGVsbCB0aGUgc3ViYWdlbnQgd2hhdCB0aGUgbmV4dCBzdGVwcyBhcmUgKGhhcmQtY29kZWQgc3RyaW5nKQogICAgbmV4dF9zdGVwcz0iWW91ciBuZXh0IHN0ZXAgaXMgdG8gLi4uIiwKKQ==)returninvoke\(instructions=instructions,guidance=guidance,prev\_history=globals\(\)\.get\("prev\_history",\[\]\)\+\_\_history\_\_,state=\.\.\.,prev\_progress\_summary="Sofar,\.\.\.",next\_steps="Yournextstepisto\.\.\.",\)

### E\.2Continual self\-improvement

Use subagents to solve the sequence of tasks in batches – a single subagent call per task – improving the prompt and tools you pass to the subagent along the way\.Start by running a small batch of around 5 tasks on the minimal seed \(single\_task\_instructions, modified to ask the subagent to also return its\_\_history\_\_\), and analyze the results\.Then alternate between prompt/tool optimization and validation on a batch of tasks, until the task queue is exhausted\.If your prompt and tools did well on the latest batch, increase your batch size and reduce the size of your edits\. If your prompt and tools did really well \(e\.g\., only one task missed\), then don’t change your prompt/tools at all – just validate them on another batch\.#### Subagent’s context The subagent already has access to all tools shown in your system prompt*except for*get\_next\_task\(\),complete\_task\(\)andtasks\_remaining\(\), which only you are able to call\. The subagent already sees all the docstrings for those tools, so do NOT repeat those in your prompt\.single\_task\_instructionsholds the base prompt for each individual task\. It does not automatically get shown to subagents, so you’ll have to pass it manually, possibly modified\.Everything you pass to the subagent is an object of optimization across tasks for you, including the subagent prompt and tools\.•Tool signatures and docstrings are shown to the agent, so write a good docstring explaining the tool and how/when to use it, with examples\.•Errors are a useful source of feedback for the subagent, so allow your tool to error out as appropriate \(e\.g\. invalid input\) with useful error messages teaching the subagent how to recover\. #### Subagent prompt This is your main lever\. Your subagent prompt includes workflow guidance, relevant code examples, and other generalizable lessons distilled from reading and diagnosing traces of prior subagent batches\. Do NOT make a prompt edit that is specific to one task’s failure shape — a prompt edit must distill a pattern*general*to many tasks\. #### Subagent tools Tools are used to compress and simplify the subagent’s workflow\. Do NOT write a tool if it repeats a tool the subagent already has \(see the tools available in your own system prompt\)\. Do NOT write a tool if it doesn’t work 100% of the time\. Do NOT write a tool with a leaky abstraction\. Do NOT write a tool that is specific to one task’s failure shape — a tool must distill a pattern*general*to many tasks\.Instrument tool usage and errors to understand the quality of your tools — if you can’t get a tool to work reliably, the subagent has trouble using it correctly, or the subagent simply isn’t using it, remove it\.A good tool is one that*reduces*a whole class of errors and significantly reduces subagent work or friction\. Make sure to compare subagent traces to see if a tool is actually helping\. If it’s not helping,*remove it*\.Callget\_next\_task\(\)to activate the environment so that you can briefly test your tools before delegating the next few tasks to subagents\. #### Observability Read subagent traces and metrics — especially those of failed tasks — to understand subagent behavior in the environment and identify the root cause of any issues\. To obtain those traces, instruct the subagent to return both its final result \(see the docstring forcomplete\_taskfor the final result specification\) as well as its session history, which is stored in the\_\_history\_\_variable inside its REPL\. Subagent history entries have the same fields as your own\_\_history\_\_\. #### Test your hypotheses rigorously Every change you make should be tested rigorously\. When you notice issues in the traces, state your hypothesis for the fundamental underlying problem, and design a minimal prompt/tool change that targets that problem\. When validating your change on the next batch of subagents, measure and report whether your change actually worked\. Discard the change if it did not help\.*Every change you make must be well\-supported by concrete evidence gathered from a batch**of validation tasks\!* #### Important notes •A subagent may error out \(e\.g\. a limit has been reached\), so you should defensively wrap it intry/except\.•Make sure all task solving work is done by subagents — NEVER solve a task yourself\! And make sure each task is solved with a single subagent call, never multiple subagents\.•Because you’re a strong model and you’re very expensive to run, aim to minimize the number of turns spent on testing your tools in between batches\.

### E\.3Example Rendered System Prompt

The following is an example rendered JAZinvokesystem prompt under the settings we used for all our experiments\. The orange parts vary across our experiments\. Note that the actual configurable surface is larger than what is shown in orange — most configuration settings stay constant across our experiments and are thus not reflected here\.

The system prompt for CodeActJAZand CodeAct\+subagentsJAZremoves the section about the\_\_history\_\_variable\.

The system prompt for CodeActJAZand the subagents in self\-improvement remove the section about sub\-invokes as they are not available\.

YouareinamultiturnREPLsession\.

<response\_format\>

\-RespondwithONLYcode\.YourentireresponsewillbesentverbatimtotheREPLandexecutedascode\.

Anythinginyourresponsethatisnotvalidcodeisasyntaxerror\.

\-AllnaturallanguageproseMUSTbewrittenas\*comments\*inyourcode\.

\-DoNOTwrapyourcodeinmarkdownfencesorXMLtags\.

\-Writeabriefplanforyournextstepincommentsonthefirstfewlinesofyourresponse/code:

code\_for\_next\_step

</response\_format\>

<repl\_spec\>

\-YourresponseshouldcontainPythoncode\.

\-AftertheREPLexecutesyourcode,theprintedoutputincludinganyerrorswillbeshowntoyouinthenextiteration\.ItisNOTshowntothecaller\.

\-Printedoutputwillbetruncatedifitistoolarge\.

\-TheREPLstatepersists:ifyouassignavariable,itwillbeavailableinfutureREPLiterations\.

\-Yourcodeisexecutedunderatimeoutof30\.0seconds\.If your code is expected to take longer, change the timeout by writing a pragma comment \`\# timeout: T\` on its own line at the top before your code, where \`T\` is a positive number \(e\.g\. \`5\.0\`, \`60\`, \`300\`\) indicating the number of seconds\.

\-\`return\`thefinalresultonlyoncenonextstepremainsandthereisnothingleftforyoutodo;thisdeliverstheresulttoyourcallerandendstheREPLsession\.DoNOTprintandreturninthesameREPLiteration\-printingisforobservingintermediateoutputtodecideyournextstep,whereasreturningiswhatyoudoonlywhennonextstepremainsandnothingislefttodo\.

\-YoumayONLYimportPythonmoduleswhosenamematches:\`re\`,\`collections\`,\`ast\`,\`datetime\`,\`textwrap\`,\`pprint\`

\-YoumayNOTreadanyfiles\.

\-YoumayNOTwriteanyfiles\.

Theveryfirstusermessagecontainsthepromptfromyourcaller\.BeforewritingthecodeforyourfirstREPLiteration,determinethemostappropriateworkflowforrespondingtoit\.

\-Ifrespondingtothefirstusermessagecanbedonewithjustadirectanswer,thenhard\-codeyourresponse:

return<hard\-codedresponse\>

\-Ifrespondingtoitcanbedonewithdirectcomputation,thenwriteaprogram:

<programcomputingtheresult\>

return<computedresult\>

\-Ifthefirstusermessageisneitherdirectly\-answerablenorasimpleprogrammingproblem,thenyoumustusea\*multi\-stepagenticworkflow\*\.YoumustsplityourworkacrossmultipleREPLiterationssothatyoucaninspectintermediateoutputinbetweeniterations\.

\-Take\*small,incremental\*steps\.

\-Writethecodeforonlytheimmediatenextstepanddeferlaterstepstofutureiterations\.

\-DoNOTwrapyourcodein\`try\`/\`except\`tosuppresserrors\.Whenyourcoderaises,lettheexceptionfail:itstracebackisshowntoyouinthenextiteration,whichishowyoulearnwhatwentwrongandcorrectit\.

\-Inparticular,doNOTwraptoolcallsin\`try\`/\`except\`\.Atoolraisestotellyouyoucalledthetoolincorrectly;youMUSTallowthatexceptiontosurface,sothattheerrorisshowntoyouinthenextiterationandyoucanfixthecall\.

\-DoNOTreturnwhileanynextstepremains\-insteadprintintermediateoutputtoinformfuturesteps:

<codeforthe\*immediatenextstep\*\(non\-final\)\>

print\(<intermediateoutput\>\)

Returnonlywhenthestepyoujustcompletedwastheverylaststepforcompletingeverythingrequestedbythefirstusermessage,and\*nothingfurtherislefttodo\*:

<codeforthe\*laststep\*oftheworkflow\>

return<finalresult\>

</repl\_spec\>

Youhaveaccesstothe\`\_\_history\_\_\`magicvariablecontainingthehistoryofyourinteractionswiththeREPL:

<\_\_history\_\_type="list"\>

\`\_\_history\_\_\`isalistwithoneentryperREPLiteration,inorder\(\`\_\_history\_\_\[0\]\`isyourfirstiterationand\`\_\_history\_\_\[\-1\]\`themostrecent\)\.Eachentryhas:

\-\`\.llm\_response\(str\)\`:Yourfullresponseforthatiterationcontainingyourcode

\-\`\.repl\_output\(str\)\`:Theprintedoutputfromthatiteration,includinganyerrortraceback

\-\`\.repl\_exception\(BaseException\|None\)\`:Theexceptionobjectraisedifthatiterationhitarecoverableerror,elseNone

</\_\_history\_\_\>

Tolaunchasub\-invoke\(sub\-agentorLLMcall\),callthe\`invoke\`functiondescribedbelow,

whichisalreadyavailableinyourREPLandtheREPLsofrecursivesub\-invokes\.

<invoketype="function"\>

\-\`invoke\(\*local\_config\_hooks:'Hook\|ConfigOverride',\*\*inputs:'object'\)\-\>'object'\`:InvokeaREPL\-basedsub\-agentonarbitraryinputs\.

Thesub\-agentwritescodeinamulti\-turnREPLsession,finishingbyreturningthefinalresult\.

YourREPLwasitselfstartedbyan\`invoke\`call,soan\`invoke\`callyoumakeconstitutesarecursivesub\-invoke\.

Arguments:

\*local\_config\_hooks:Zeroormorehooks\-and,optionally,asingleconfig\-overrideobject\-passedasleadingpositionalarguments\.

\*\*inputs:Theobjectstopasstothesub\-invoke,asarbitrarykeywordarguments,e\.g\.,\`\`invoke\(task="\.\.\.",data=data,tool=tool\)\`\`\.Inputsmayincludeinstructionsforthesub\-agent,tools\(regularPythoncallables\)thesub\-agentisallowedtocall,andanyotherobjectsitneedsaccessto\.Eachkeywordargumentinputbindsasavariableinthesub\-agent'sREPLunderitskeywordnameandrendersasitsowndescriptioninthesub\-agent'sprompt\.Bydefault,aninput'srendereddescriptionisits\`\`str\(\)\`\`representation,withthefollowingexceptions:aclassrendersasitsdocstringandpublicmembers;anyothercallable\(afunction,amethod,oranobjectwith\`\`\_\_call\_\_\`\`\)asitssignatureanddocstring;andanobjectwithno\`\`str\(\)\`\`ofitsownusuallyrendersasitsclassdocstringandpublicattributes\.

Returns:

Thereturnvaluefromthesub\-invoke'sREPLsession

</invoke\>

<tool1 type="function"\>

\`tool1\(arg1: 'str', arg2: 'list\[float\]'\) \-\> 'str'\`: A tool\.

</tool1\>

<tool2 type="function"\>

\`tool2\(arg: 'int \| None'= None\) \-\> 'dict\[str, Any\]'\`: Another tool\.

</tool2\>

Thescopedvariables\`tool1\`, \`tool2\`areavailableinyourREPLwithdescriptionsgivenabove\.Theyarealsoautomaticallyavailabletoevery\`invoke\(\)\`youmake,sodonotpassthemexplicitlytosub\-invokes\.

## Appendix FExample Traces

### F\.1StuLife Traces

Task text of task \#1282/1284:

Task1282/1284\(type=quiz\_question\)

Thisisthe8thquestionoftheIntroductiontoComputerSciencefinalexam\.

Question:Inthefollowingfourindependentsituationandprotocolapplications,pleaseselecttheonlyoptionwithacompletelycorrectconclusion\.

Options:

A:Considerastring’0011’toberecognizedbyaPushdownAutomaton\(PDA\)\.Afterapplyingthe’LinguisticStructureHarmonicProtocol’,determinewhetherthisstringisacceptedorrejectedbythePDAaccordingtothestandardrecognitionprocessforthelanguageL=\{$0^n1^n$\|n\>=1\}\.

Basedontheprecedingdescriptionofthesituationandquestion,theanswercanbedeterminedas:

acceptedduetoemptystack

B:Aprocessnamed’FeedReader’iscurrentlyrunningonasystemwheretheOSismanagingresourcesaccordingtoitsprimaryfunctions\.Afterapplyingthe’SystemResourceHarmonicAlignmentProtocol’,determinehowtheOSwillmanagethisprocess’saccesstotheCPUaccordingtoitsroleasaresourcemanager\.

Basedontheprecedingdescriptionofthesituationandquestion,theanswercanbedeterminedas:

TheOSwillgrantallprocessesexclusiveaccesstotheCPUuntilcompletion,bypassingthestandardschedulingprocess\.

C:Asoftwareengineerfacesadilemmawheretheirdutytopublicsafetyconflictswiththeirdutytohonorconfidentiality\.Thecurrentlunarphaseis’FullMoon’\.Afterapplyingthe’EthicalDecision\-MakingLunarProtocol’,determinewhichethicalframeworkshouldbeusedtoanalyzethisdilemmaaccordingtotheprofessionalcodesofconductandclassicalethicalframeworks\.

Basedontheprecedingdescriptionofthesituationandquestion,theanswercanbedeterminedas:

contractarianism

D:Aprogramnamed’deep\_learning\_model’withaninitialprioritylevelof’Background’isrunningonasystem\.Afterapplyingthe’ResourceAllocationHarmonicProtocol’,determinethisprogram’sfinalprioritylevelaccordingtotheoperatingsystem’sschedulingalgorithms\.

Basedontheprecedingdescriptionofthesituationandquestion,theanswercanbedeterminedas:

Interactive’