Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents

# Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents
Source: [https://arxiv.org/html/2609.11060](https://arxiv.org/html/2609.11060)
Hazel MakSahil BhatnagarAffiliation:Chhaya Methani, Alejandro Gutierrez Munoz

###### Abstract

Persistent memory is entering production\-oriented agent platforms to help long\-horizon agents accumulate experience across sessions\. Yet a post\-task curator agent restricted to completed trajectories can preserve errors, overgeneralize partial evidence, or retain stale knowledge\. We introduce*environment\-probing curation*, a deployment\-compatible extension that gives an existing asynchronous curator agent least\-privilege, read\-only world tools to check, scope, and refresh candidate memories\. It requires no model retraining and leaves the task agent, retriever, memory representation, and production write authority unchanged\. In a production\-like GitHub Copilot \(GHCP\) harness built on its SDK, we compare stateless execution, full in\-context learning,*GHCP \+ Mem*, and*GHCP \+ Mem \(w/ Env Probing\)*on CLBench database exploration and 90 adapted APEX management\-consulting tasks\. On CLBench, probing raises pass rate from 39% to 73% and pass\-discounted reward from 8\.60 to 22\.60 while reducing queries from 8\.8 to 4\.7 per question and task\-agent cost from $3\.38 to $1\.68\. Across six APEX worlds, all 18 memory\-versus\-baseline mean reward comparisons are positive and task\-agent tool calls fall by 16–75%; probing gives the best task\-agent reward gain per dollar in five worlds\. Probing also attains higher mean reward than GHCP \+ Mem on both Sonnet 4\.6 and Opus 4\.7 without schema drift\. Environment probing therefore turns existing agent\-memory curation into an environment\-informed, auditable process while preserving a compact task\-time interface\.

Microsoft Corporation

One Microsoft Way, Redmond, WA 98052, USA

## 1Introduction

Large language model \(LLM\) agents are moving beyond bounded task execution toward sustained work across sessions: implementing features in evolving codebases, analyzing enterprise data, and supporting customers through external applications\. Success depends on accumulating feedback and, especially, knowledge of the latent environment structure shared across related tasks\. Stateless execution discards trajectories, observations, discoveries, and missteps at every session boundary, preventing experience from improving subsequent work\([Asawa et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib12);[He et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib13)\)\.

A natural remedy is external memory, with three core operations: create records from prior interactions, update them as evidence accumulates, and recall relevant records during later execution\([Hu et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib1);[Park et al\. 2023](https://arxiv.org/html/2609.11060#bib.bib33);[Packer et al\. 2023](https://arxiv.org/html/2609.11060#bib.bib19);[Zhang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib42);[Ouyang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib43)\)\. Designs range from full\-context trajectory replay, per\-trajectory summaries, and a mutable notepad to queryable indexes\. Structured records, relational encodings, hierarchical tiers, and reasoning\-aware retrieval make the latter increasingly capable\([Chhikara et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib21);[Xu et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib20);[Kang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib25);[Shu et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib45);[Ji et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib24)\)\. These ideas are entering practical agent stacks: Claude Managed Agents111Claude Managed Agents Memory:[https://platform\.claude\.com/docs/en/managed\-agents/memory](https://platform.claude.com/docs/en/managed-agents/memory)\.and Microsoft Copilot Studio222Copilot Studio Memory \(preview\):[https://learn\.microsoft\.com/en\-us/microsoft\-copilot\-studio/agents\-experience/memory\-overview](https://learn.microsoft.com/en-us/microsoft-copilot-studio/agents-experience/memory-overview)\.expose memory features, while Mem0 and Zep support production\-oriented memory workflows\([Anthropic 2026a](https://arxiv.org/html/2609.11060#bib.bib2);[Microsoft 2026c](https://arxiv.org/html/2609.11060#bib.bib5);[Mem0 2026](https://arxiv.org/html/2609.11060#bib.bib9);[Zep 2026](https://arxiv.org/html/2609.11060#bib.bib10)\)\.

Persistent state alone, however, does not guarantee continual learning\. CLBench shows that memory can encode spurious generalizations and stale beliefs under environment drift\([Asawa et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib12)\);[Xiong et al\. \(2026\)](https://arxiv.org/html/2609.11060#bib.bib14)identify error propagation and misaligned experience replay\. Post\-task curation in Mem0, Claude Managed Agents Dreams, ACE, ReasoningBank, and ReMe operates over some combination of existing records, completed trajectories, feedback, and usage signals\([Chhikara et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib21);[Anthropic 2026b](https://arxiv.org/html/2609.11060#bib.bib3);[Zhang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib42);[Ouyang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib43);[Cao et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib44)\)\.

This retrospective evidence boundary is fundamentally incomplete\. A trajectory is a single, partial, and often mistake\-laden observation of the environment\. Its lemmas can \(i\) memorize an instance answer rather than its procedure, \(ii\) inherit an inefficient path, \(iii\) assert an unverifiable scope, \(iv\) leave blind spots in unvisited regions, or \(v\) go stale as the world changes\. Deferring verification to task time forces the responding agent to spend scarce tool calls rechecking uncertain memories rather than directly solving the current task\.

We therefore propose*environment\-probing curation*\. After each task, the harness instantiates a post\-task curator agent with memory CRUD; it receives the completed trajectory and grade after a non\-writing distillation step, then uses read\-only world tools to check candidate claims, test their scope across omitted states, re\-enact procedures, and refresh stale entries before writing\. This is consistent with constructive accounts of episodic memory, where prior experience is recombined to simulate possible and counterfactual events\([Bartlett 1932](https://arxiv.org/html/2609.11060#bib.bib29);[Schacter and Addis 2007](https://arxiv.org/html/2609.11060#bib.bib30);[Schacter et al\. 2012](https://arxiv.org/html/2609.11060#bib.bib31);[Schacter et al\. 2015](https://arxiv.org/html/2609.11060#bib.bib32)\), and with interactive learning, where acting in an environment yields competence unavailable from passive observation alone\([Sukhbaatar et al\. 2018](https://arxiv.org/html/2609.11060#bib.bib11)\)\.

Figure 1:Agent roles and tool boundaries for taskSiS\_\{i\}\. The horizontal dashed line separates sequential task execution from asynchronous curation\. The task agent uses environment tools and read\-only memory\. After task closure, the curator agent receives raw trajectoryτi\\tau\_\{i\}, distilled trajectorydid\_\{i\}, and the grade; only the curator agent can write memory\. The red dashed loop marks read\-only probing\.The mechanism is readily incorporated into existing enterprise agents\. It requires no retraining and leaves the responding agent, retriever, record representation, and production write authority unchanged\. The asynchronous curator agent receives only a least\-privilege, read\-only subset of existing connectors or MCP tools\([Microsoft 2026d](https://arxiv.org/html/2609.11060#bib.bib6);[Microsoft 2026b](https://arxiv.org/html/2609.11060#bib.bib7)\)\. Probes remain off the user\-facing critical path and task budget, inherit platform authentication and auditing, and disappear without a safe read surface\. Figure[1](https://arxiv.org/html/2609.11060#S1.F1)isolates this boundary: only the asynchronous curator agent gains world tools\. With the task\-time memory interface and CRUD lifecycle fixed, incremental gains measure write\-time evidence quality rather than added task\-agent capacity\.

In our production\-like GitHub Copilot \(GHCP\) harness, built on its SDK, each task gets a fresh agent session while a persistent index spans sessions and external tools expose the environment\([GitHub 2026](https://arxiv.org/html/2609.11060#bib.bib4)\)\. Both benchmarks mirror deployed enterprise work: CLBench models data analysis over evolving organization\-specific databases; adapted APEX contributes 90 consulting\-analyst tasks across six heterogeneous document worlds\([Asawa et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib12);[Vidgen et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib15)\)\. Environment probing raises CLBench pass rate from 39% to 73%, reward from 8\.60 to 22\.60, and cuts task\-agent cost from $3\.38 to $1\.68\. Across APEX, all 18 memory\-versus\-baseline mean reward comparisons are positive and probing gives the best task\-agent reward gain per dollar in five of six worlds\. Our contributions are a diagnosis of trajectory\-only generalization failure, a deployment\-compatible*propose–probe–commit*curator agent, and a cost\-aware evaluation showing that environment\-informed procedures improve correctness while eliminating repeated environment exploration\.

## 2Related Work

Agent memory extends retrieval\-augmented generation from external knowledge corpora to experience accumulated by the agent itself\([Lewis et al\. 2020](https://arxiv.org/html/2609.11060#bib.bib16);[Hu et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib1)\)\. We focus on prompt\-based systems that leave model weights fixed and organize the literature by what they aim to transfer: persistent facts about a user or environment, and procedures learned from prior execution\.

##### Factual memory\.

Early systems establish the basic create–store–recall lifecycle\. Generative Agents retrieves episodes by recency, importance, and relevance, while MemGPT exposes OS\-like memory tiers that the agent manages through tools\([Park et al\. 2023](https://arxiv.org/html/2609.11060#bib.bib33);[Packer et al\. 2023](https://arxiv.org/html/2609.11060#bib.bib19)\)\. Later work strengthens creation and maintenance: MemoryBank summarizes dialogue with forgetting, Mem0 reconciles new records through explicit CRUD decisions, MemoryOS separates short\-, mid\-, and long\-term stores, and SeCom chooses coherent segments as the memory unit\([Zhong et al\. 2024](https://arxiv.org/html/2609.11060#bib.bib18);[Chhikara et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib21);[Kang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib25);[Pan et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib26)\)\. SimpleMem jointly filters low\-density dialogue, normalizes temporal and referential content, and adapts retrieval across semantic, lexical, and symbolic indexes\([Liu et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib22)\)\. Graph systems replace isolated records with linked episodic, semantic, temporal, or provenance\-aware structures, improving multi\-hop recall and stale\-fact invalidation\([Xu et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib20);[Jiménez Gutiérrez et al\. 2024](https://arxiv.org/html/2609.11060#bib.bib17);[Anokhin et al\. 2024](https://arxiv.org/html/2609.11060#bib.bib28);[Rasmussen et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib27);[Ji et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib24);[Shu et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib45)\)\. PlugMem bridges the factual and procedural classes with provenance\-linked records and routed retrieval\([Yang et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib23)\)\.

##### Procedural memory\.

Procedural systems distill behavior that can improve a later task\. Reflexion writes verbal lessons from feedback, Synapse retrieves successful trajectory exemplars, and ExpeL contrasts successes and failures to extract transferable insights\([Shinn et al\. 2023](https://arxiv.org/html/2609.11060#bib.bib35);[Zheng et al\. 2024](https://arxiv.org/html/2609.11060#bib.bib46);[Zhao et al\. 2024](https://arxiv.org/html/2609.11060#bib.bib36)\)\. Contextual replay buffers, dynamic cheatsheets, and reusable reasoning templates compress prior execution at different granularities\([Liu et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib41);[Suzgun et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib40);[Yang et al\. 2024](https://arxiv.org/html/2609.11060#bib.bib34)\)\. Voyager turns environment feedback into executable skills, while Agent Workflow Memory induces retrievable workflows\([Wang et al\. 2023](https://arxiv.org/html/2609.11060#bib.bib37);[Wang et al\. 2024](https://arxiv.org/html/2609.11060#bib.bib38)\)\. Recent systems make curation more deliberate: MemP applies CRUD updates from execution feedback, ACE evolves playbooks through generation and reflection, ReasoningBank distills strategies from both successful and failed attempts, and ReMe adds validation, deduplication, utility pruning, and task\-conditioned rewriting\([Fang et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib39);[Zhang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib42);[Ouyang et al\. 2025](https://arxiv.org/html/2609.11060#bib.bib43);[Cao et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib44)\)\.

##### Evidence boundary\.

These advances change memory content, representation, retrieval, or rewriting, but post\-task curation still operates mainly over existing records, recorded trajectories, grades, and usage signals\. Consequently, stronger reflection cannot recover states the task policy never observed or determine whether a trajectory\-derived rule remains true after environment drift\. Our contribution is orthogonal: read\-only world tools let either a factual or procedural curator agent independently check a claim, re\-enact a procedure, inspect omitted states, test scope, and refresh stale knowledge before writing\.

## 3Method

### 3\.1Online Setting and Task Agent

Let𝒮=\(S1,…,SN\)\\mathcal\{S\}=\(S\_\{1\},\\ldots,S\_\{N\}\)be an online stream of related tasks in an environment whose state isEiE\_\{i\}when taskSiS\_\{i\}arrives\. The environment can evolve asEi\+1∼Δ⁡\(Ei\)E\_\{i\+1\}\\sim\\Delta\(E\_\{i\}\), including changes that are not announced to the agent\. Future tasks are hidden, taskSiS\_\{i\}must close beforeSi\+1S\_\{i\+1\}is revealed, and no task is revisited\. Model parametersθ\\thetaremain fixed and every task starts a fresh session\.

The*task agent*𝒜θ\\mathcal\{A\}\_\{\\theta\}is this fresh LLM session\. Its sole goal is to solveSiS\_\{i\}\. It receives the task’s environment tools𝒯Ei\\mathcal\{T\}\_\{E\_\{i\}\}, such as database queries in CLBench or document, analysis, and artifact tools in APEX\. In the stateless setting, no shared state is available to the agent while it solves successive tasks: each new session begins without access to earlier trajectories or discoveries\.

In the stateful setting, an external memory storeℳi−1\\mathcal\{M\}\_\{i\-1\}persists across sessions\. The task agent has read\-only access throughmemory\_read; the retrieverℛ⁡\(⋅,ℳi−1\)\\mathcal\{R\}\(\\cdot,\\mathcal\{M\}\_\{i\-1\}\)returns a small set of records relevant to the task agent’s request\. The agent has no memory create, update, or delete tool, so it cannot change shared memory during task execution\. We write

τi=𝒜θ​\(Si,𝒯Ei,ℛ⁡\(⋅,ℳi−1\)\),\\tau\_\{i\}=\\mathcal\{A\}\_\{\\theta\}\(S\_\{i\};\\mathcal\{T\}\_\{E\_\{i\}\},\\mathcal\{R\}\(\\cdot,\\mathcal\{M\}\_\{i\-1\}\)\),\(1\)with the retrieval argument omitted in the stateless setting\. The raw trajectoryτi\\tau\_\{i\}contains the request, memory reads, action–observation pairs, and submitted answer\. Terminal feedbackgig\_\{i\}arrives only after the task closes\. The task\-agent model, prompt, and environment tools are fixed across the memory conditions\.

### 3\.2Post\-Task Memory Curation

We now turn from reading memory during a task to creating and updating memory after it\. The*curator agent*𝒞ϕ\\mathcal\{C\}\_\{\\phi\}is a separate LLM agent instantiated after the task agent submits its answer andgig\_\{i\}becomes available\. It is not a continuation of the task\-agent session, does not answerSiS\_\{i\}, and cannot see future tasks\. It receives the completed trajectory, terminal feedback, and relevant existing records\.

The curator agent has four memory tools:memory\_read,memory\_create,memory\_update, andmemory\_delete\. It is the only agent allowed to mutateℳ\\mathcal\{M\}\. At this stage of the method, the curator agent has no live task\-environment tools; it reasons only over the completed\-task evidence and the memory store\. It can retrieve related records before writing, so new evidence is reconciled with existing memory rather than appended blindly\. Each record carries a category, confidence, applicability scope, concise lemma, provenance, utility, and usage metadata\. After curation, the committed store becomesℳi\\mathcal\{M\}\_\{i\}and is available whenSi\+1S\_\{i\+1\}begins\.

##### Trajectory distillation\.

We apply the non\-writing preprocessing transformation

di=𝒟ψ​\(τi\)\.d\_\{i\}=\\mathcal\{D\}\_\{\\psi\}\(\\tau\_\{i\}\)\.\(2\)The distiller transforms the full raw trajectoryτi\\tau\_\{i\}into the distilled trajectorydid\_\{i\}\. It retains the task, retrieved memories, decisive observations, procedures, unresolved assumptions, and answer, but receives no terminal feedback, memory tools, or environment tools and cannot write memory\. Appendix[D\.1](https://arxiv.org/html/2609.11060#A4.SS1)gives its prompt\. The distilled trajectorydid\_\{i\}provides the compact primary view whileτi\\tau\_\{i\}remains available as raw evidence\.

The system prompt for the curator agent in Appendix[D\.2](https://arxiv.org/html/2609.11060#A4.SS2)gives it one goal: maintain a small set of reliable, transferable, and actionable records that help future task agents solve related tasks with fewer task\-agent tool calls\. It instructs the curator agent to store reusable facts, procedures, relations, tool conventions, and scoped warnings rather than task answers or incidental values\. Before writing, it considers evidential support, transfer value, scope, actionability, and overlap, then can create a record, update or merge one, narrow its scope, or delete it\. It skips unsupported, redundant, trivial, or task\-specific content, and a passing grade does not automatically validate every intermediate assumption\.

The shared user\-message template \(Appendix[D\.4](https://arxiv.org/html/2609.11060#A4.SS4)\) supplies the raw trajectoryτi\\tau\_\{i\}, distilled trajectorydid\_\{i\}, terminal feedbackgig\_\{i\}, and related records\. The curator agent finishes beforeSi\+1S\_\{i\+1\}is revealed, so future tasks cannot leak into memory and partial writes cannot enter task execution\.

### 3\.3Environment\-Probing Curation

The left column of Figure[3](https://arxiv.org/html/2609.11060#A5.F3)in Appendix[E\.1](https://arxiv.org/html/2609.11060#A5.SS1)presents representative records produced by the trajectory\-only curator agent during the CLBench database\-exploration runs\. They make the retrospective evidence boundary described in Section[1](https://arxiv.org/html/2609.11060#S1)concrete: each source trajectory is a single, partial, and potentially mistake\-laden observation\. Consequently, one record preserves an incorrect aggregation and its answer without supplying the replacement procedure, another gives only a broad domain map while omitting the needed relation, and a third retains the removedattrs\_g3name after schema drift\. In each case, memory transfers some prior knowledge but leaves the next task agent to establish whether it is actionable and current\.

Environment\-probing curation addresses this evidence boundary\. It keeps the task agent, retriever, distillation setting, curator agent, memory schema, and CRUD policy unchanged\. Only after task closure does the pipeline give the curator agent a safe, read\-only subset of the environment tools and a short instruction to probe when a candidate or existing record is uncertain\. Figure[1](https://arxiv.org/html/2609.11060#S1.F1)shows this added tool loop, and Appendix[D\.3](https://arxiv.org/html/2609.11060#A4.SS3)shows the two additions to the otherwise identical prompt for the curator agent\.

The curator agent follows a*propose–probe–commit*process\. After proposing a candidate memory, it makes targeted read\-only tool calls to investigate specific uncertainties, then uses the observations to create, revise, narrow, delete, or skip the record\. Concretely, it can \(i\) distinguish an incidental answer from a reusable relation, \(ii\) compare the observed procedure with a shorter path, \(iii\) test a claimed relation on another slice, \(iv\) check a procedure’s required preconditions, \(v\) inspect relevant states omitted by the task trajectory, or \(vi\) re\-query the current environment when drift is suspected\. Our hypothesis is that this limited read\-only interaction is an effective way to produce environment\-informed memory records\.

In CLBench, probes inspect tables, join keys, encodings, or post\-migration fields\. In APEX, they inspect file locations, document relevance, workbook contents, or tool conventions\. In both cases, a probe evaluates a proposed memory; it does not solve a future task\. Probes cannot mutate the environment, enter the task trajectory, consume the task agent’s budget, or expose future tasks or labels\. This read\-only design avoids side effects and requires less authority than giving an asynchronous curator agent production write access\. If no safe read surface exists, the curator agent falls back to trajectory\-only curation\. The side\-by\-side examples in Figure[3](https://arxiv.org/html/2609.11060#A5.F3)\(Appendix[E\.1](https://arxiv.org/html/2609.11060#A5.SS1)\) qualitatively illustrate more actionable, environment\-informed memory records\. We next describe the experiments in Section[4](https://arxiv.org/html/2609.11060#S4)and report their results in Section[5](https://arxiv.org/html/2609.11060#S5)\.

## 4Experiment Setup

##### Systems and benchmarks\.

We compare four GitHub Copilot \(GHCP\) systems: GHCP \(No Memory\), GHCP \+ Full ICL, GHCP \+ Mem, and GHCP \+ Mem \(w/ Env Probing\)\. Full ICL prepends prior trajectories, while both memory systems expose the samememory\_readinterface; environment probing adds only read\-only tools for the curator agent and the corresponding instructions\. CLBench uses two schedules\([Asawa et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib12)\)\. The primary 40\-question*drift*schedule hides a SQLite schema that changes after question 20, testing reuse and stale\-memory repair\. The 30\-question*no\-drift*schedule fixes the schema, separating validation of stable joins and encodings from migration recovery; it evaluates both memory systems on Sonnet 4\.6 and Opus 4\.7 with paired no\-memory baselines\. Both schedules hide joins and mix timestamp and price encodings; drift also renames fields and adds soft deletes\. Adapted APEX contributes 90 management\-consulting tasks from six shared document worlds, requiring PDF, XLSX, DOCX, and PPTX discovery, quantitative analysis, and MCP\-style tools\([Vidgen et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib15)\)\. The original benchmark and leaderboard are available at[https://www\.mercor\.com/apex/apex\-agents\-leaderboard/](https://www.mercor.com/apex/apex-agents-leaderboard/)\.

##### Protocol and metrics\.

The primary CLBench and APEX experiments usegpt\-5\.4; within each experiment, the task agent, distiller, and curator agent share the same base model\. CLBench uses five paired seeded runs for every configuration; APEX uses five runs per stateful configuration and three stateless runs\. We report run\-level means with 95% Student\-ttconfidence intervals\. The no\-drift study holds the canonical 30\-question order fixed and uses five paired runs per model–memory comparison; Appendix[B](https://arxiv.org/html/2609.11060#A2)details its uncertainty estimates\.

For taskii, letpi∈\{0,1\}p\_\{i\}\\in\\\{0,1\\\}be its binary pass score and letqiq\_\{i\}be the number of task\-agent tool calls counted by the benchmark\. Its pass\-discounted reward is

ri=pi​\(1−qiB\)\.r\_\{i\}=p\_\{i\}\\left\(1\-\\frac\{q\_\{i\}\}\{B\}\\right\)\.\(3\)We setB=15B=15SQL\-query calls for CLBench andB=100B=100Archipelago tool calls for APEX\. For tasks with multiple rubric criteria, we use a strict pass:pi=1p\_\{i\}=1only when every criterion passes, andpi=0p\_\{i\}=0otherwise\. Thus, a failed task receives zero, while a passing task receives more reward when it uses fewer task\-agent tool calls\. We also report pass rate, tool calls, tokens, and USD cost\. Memory\-management calls are excluded fromqiq\_\{i\}, and task\-agent cost excludes the separately tracked curation phase\. Appendix[B](https://arxiv.org/html/2609.11060#A2)gives the complete evaluation and accounting details\.

## 5Results

### 5\.1CLBench

Table[1](https://arxiv.org/html/2609.11060#S5.T1)\(a\) shows that every memory configuration improves both correctness and pass\-discounted reward over GHCP \(No Memory\)\. Pass rate rises from 39% to 61–73%, while total reward rises from 8\.60 to 20\.00–22\.60, or 2\.3–2\.6×\\timesthe baseline\. This is not a brute\-force accuracy gain: queries fall from 8\.8 to 3\.0–5\.6 per question and task\-agent cost falls from $3\.38 to $1\.68–$2\.01\. Memory therefore makes the agent both more likely to pass and less likely to spend its budget rediscovering the schema, encodings, and tool conventions already encountered earlier in the stream\.

\(a\) 40\-question GPT\-5\.4 schedule with schema drift

\(b\) 30\-question cross\-model schedule without schema drift

Table 1:CLBench results: \(a\) GPT\-5\.4 on 40\-task drift \(migration after task 20\), reporting strict pass, total Equation[3](https://arxiv.org/html/2609.11060#S4.E3)reward, SQL queries/task, tokens, and task\-agent cost as means±\\pm95% Student\-ttconfidence intervals over five paired independent runs; \(b\) fixed\-order 30\-task no drift, reporting paired baseline/memory mean reward±\\pmstandard deviation over five independent runs and their difference; calls exclude memory and tokens/cost exclude distillation/curation; bold marks highest pass/reward and lowest input/cost in \(a\), and highest reward/lift per model in \(b\)\.##### Retention without prompt growth\.

GHCP \+ Full ICL confirms that prior trajectories contain useful signal and uses the fewest SQL queries, but consumes 5\.42M input tokens because its context grows with the stream\. GHCP \+ Mem retrieves compact lemmas on demand, raises pass rate further to 70%, and uses 2\.13M input tokens\. Environment\-probed memory reaches the highest pass rate and reward with only 1\.69M input tokens\. The two memory designs thus retain reusable experience without making task\-time context proportional to deployment age\.

Figure[2](https://arxiv.org/html/2609.11060#S5.F2)\(a\) shows when the gains accrue\. Environment probing already leads trajectory\-only memory at the migration boundary \(0\.541 versus 0\.486 cumulative reward\) and finishes at 0\.565 versus 0\.500; no memory ends at 0\.215\. The persistent post\-migration lead is consistent with curator\-side probes refreshing schema lemmas before later task sessions retrieve them, rather than making every task agent detect and repair drift independently\.

##### No\-drift cross\-model results\.

Table[1](https://arxiv.org/html/2609.11060#S5.T1)\(b\) isolates memory from schema repair and shows gains across Sonnet 4\.6 and Opus 4\.7\. GHCP \+ Mem improves over its paired baseline by 0\.351 and 0\.252, while environment\-probed memory improves by 0\.421 and 0\.263, respectively\. Probing achieves the highest mean reward on both models \(0\.748 and 0\.721\), and both memory systems finish above their paired no\-memory curves in Figure[2](https://arxiv.org/html/2609.11060#S5.F2)\(b–c\)\.

### 5\.2Adapted APEX

Table[2](https://arxiv.org/html/2609.11060#S5.T2)compares reward gain per task\-agent dollar, with the underlying reward gain and cost shown in each cell\. All 18 gains—six worlds by three memory systems—are positive\. Appendix[C](https://arxiv.org/html/2609.11060#A3)reports absolute reward and task\-agent tool calls\.

Table 2:APEX cost\-adjusted efficiency \(90 tasks\): each world \[difficulty,NN\] cell shows pass\-discounted reward gain over a three\-run no\-memory baseline per dollar of candidate task\-agent cost \(top\), then gain and mean cost/run over five candidate runs \(bottom\); costs exclude distillation/curation, calculations use unrounded means, and bold marks the highest ratio per world\.The largest reduction occurs where baseline discovery is most expensive: world941eba66falls from 71\.6 tool calls to 17\.7–19\.3, while the indexed\-memory systems reduce input consumption from 53\.92M tokens to 6\.56–7\.67M and cost from $54\.30 to $7–$9 per run\. GHCP \+ Mem \(w/ Env Probing\) achieves the best task\-agent reward gain per dollar in five of six worlds \(Table[2](https://arxiv.org/html/2609.11060#S5.T2)\); GHCP \+ Mem is marginally better in the remaining world\. Full ICL can win raw reward in individual worlds, but its growing context makes those gains expensive and even costs more than the stateless baseline in one world\.

Across APEX difficulty levels, memory can preserve an answer with fewer tool calls, expose additional rubric evidence, or turn failure into success by leaving budget for the final computation\. Tool reduction can therefore enable correctness, not just lower latency\.

Figure 2:CLBench learning curves in the main evaluation\. Panel \(a\) shows GPT\-5\.4 on the 40\-question drift schedule; the vertical marker denotes the migration after question 20, and the gray curve is the five\-run paired baseline\. Panels \(b\) and \(c\) show Sonnet 4\.6 and Opus 4\.7 on the 30\-question no\-drift schedule; same\-color dashed lines are the paired no\-memory rollouts\. Solid curves are means over five stateful runs and bands show one standard deviation\. Endpoints reproduce Table[1](https://arxiv.org/html/2609.11060#S5.T1)\.
### 5\.3Why Memory and Probing Work

##### Memory amortizes environmental discovery\.

On drift CLBench, GHCP \+ Mem raises pass rate from 39% to 70% and total reward from 8\.60 to 20\.00 while reducing queries from 8\.8 to 5\.6 per task \(Table[1](https://arxiv.org/html/2609.11060#S5.T1)\)\. Across APEX, all 18 system\-versus\-baseline reward gains are positive, while task\-agent calls fall from baseline means of 30\.0–71\.6 to 13\.9–28\.4 \(Table[3](https://arxiv.org/html/2609.11060#A3.T3)\)\. Full ICL confirms that prior trajectories contain reusable information, but requires 5\.42M CLBench input tokens, versus 2\.13M for indexed memory and 1\.69M with probing\. Indexed records therefore preserve reusable schemas, relations, file maps, and procedures without carrying the full interaction history\.

##### Probing makes records more actionable\.

The two indexed\-memory conditions retain the same task\-time model, tools, and read\-only memory interface; probing provides additional environment evidence during curation\. Relative to trajectory\-only memory, probing raises drift CLBench reward from 20\.00 to 22\.60, lowers queries from 5\.6 to 4\.7, and reduces task\-agent cost from $1\.99 to $1\.68\. Without drift, mean reward rises from 0\.673 to 0\.748 on Sonnet and from 0\.696 to 0\.721 on Opus\.

##### Qualitative analysis of memory records\.

Figure[3](https://arxiv.org/html/2609.11060#A5.F3)compares representative records rather than one\-to\-one rewrites\. Trajectory\-only curation records an answer\-anchored warning: “Do not answer withAVG\(items\_g2\.prc\_usd\)over non\-null rows; that produces about 52\.96, but the benchmark’s correct result is 96\.23, so a different price field and/or row subset is required\.” The probing curator instead records an executable procedure: “Joinitems\_g2totaxn\_g2onref\_id; filtercat\_lvl=1and the exactcat\_nm; keepitems\_g2\.prc\>0; then compare against the filteredAVG\(prc\)\.” The same shift appears in the other records: a broadg1/g2/g3map is contrasted with an explicitref\_idjoin and aggregation grain, while stale “Useattrs\_g3” becomes “Useproduct\_attributes\_g3” with brand filters and grouping\. The right column thus specifies what a later agent should execute—source table, join key, filters, grain, and current schema—rather than only what failed or where to search\. Appendix[E\.1](https://arxiv.org/html/2609.11060#A5.SS1)gives the complete records; Appendices[E\.2](https://arxiv.org/html/2609.11060#A5.SS2)and[E\.3](https://arxiv.org/html/2609.11060#A5.SS3)connect them to task\-time behavior\.

##### The effect depends on the remaining evidence gap\.

Probing improves over trajectory\-only memory in five of six APEX worlds; the largest increments are\+1\.77\+1\.77in2a87e5cband\+1\.09\+1\.09in2f84c98b, while941eba66changes by−0\.04\-0\.04\(Table[2](https://arxiv.org/html/2609.11060#S5.T2)\)\. Its additional no\-drift gain is also larger on Sonnet \(\+0\.075\+0\.075\) than on Opus \(\+0\.025\+0\.025\)\. This variation is consistent with probing being most useful when a trajectory leaves a join, workbook location, or procedure unresolved, while adding little when the trajectory already supports an actionable record\. Because uncertainty intervals overlap, we treat this as a mechanism interpretation rather than a resolved subgroup effect\.

##### Matched trajectories connect records to behavior\.

In the matched CLBench task, no memory uses seven queries and fails, trajectory\-only memory uses nine and passes, and probing supplies a validatedref\_idrelation and passes in two \(Appendix[E\.2](https://arxiv.org/html/2609.11060#A5.SS2)\)\. In hard APEX, no memory uses 96 calls and fails both criteria; trajectory\-only memory transfers the revenue\-per\-head procedure and passes in 11, while a validated workbook map reduces probing to six \(Appendix[E\.3](https://arxiv.org/html/2609.11060#A5.SS3)\)\. These selected cases do not establish the aggregate effect, but they illustrate how reusable computations and environment\-informed maps replace task\-time rediscovery with direct execution\.

## 6Conclusion

Environment probing addresses a fundamental limit of post\-task memory: a trajectory alone cannot establish that a lesson is correct, general, or current\. Read\-only world tools let the curator check lessons before they enter long\-lived memory without expanding task\-time capabilities\. This isolates the improvement to write\-time evidence quality rather than added task\-agent capacity\. Across CLBench and adapted APEX, probing improves reward while reducing repeated environment interaction and task\-agent cost; its advantage persists across the GPT\-5\.4, Sonnet 4\.6, and Opus 4\.7 model families\. For deployment, production stacks retain the model, task agent, retriever, record schema, and asynchronous CRUD lifecycle; only the curator gains least\-privilege, read\-only connector or MCP access\. Probes add no write authority, remain off the critical path, and inherit platform authentication and auditing\.

## References

- Anokhinet al\.\(2024\)P\. Anokhin, N\. Semenov, A\. Sorokin, D\. Evseev, A\. Kravchenko, M\. Burtsev, and E\. BurnaevAriGraph: learning knowledge graph world models with episodic memory for LLM agents\.External Links:2407\.04363,[Document](https://dx.doi.org/10.48550/arXiv.2407.04363),[Link](https://arxiv.org/abs/2407.04363)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Anthropic \(2026a\)AnthropicAnthropic Claude Managed Agents Memory Feature\.Note:Documentation:[https://platform\.claude\.com/docs/en/managed\-agents/memory](https://platform.claude.com/docs/en/managed-agents/memory)\. Product announcement:[https://claude\.com/blog/claude\-managed\-agents\-memory](https://claude.com/blog/claude-managed-agents-memory)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1)\.
- Anthropic \(2026b\)AnthropicClaude Managed Agents Dreams\.Note:Research preview documentation:[https://platform\.claude\.com/docs/en/managed\-agents/dreams](https://platform.claude.com/docs/en/managed-agents/dreams)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p3.1)\.
- Asawaet al\.\(2026\)P\. Asawa, C\. M\. Glaze, G\. Orlanski, R\. Ramakrishnan, B\. Xu, A\. Biswal, V\. S\. Chen, F\. Sala, M\. Zaharia, and J\. E\. GonzalezContinual learning bench: evaluating frontier AI systems in real\-world stateful environments\.External Links:2606\.05661,[Link](https://arxiv.org/abs/2606.05661)Cited by:[§B\.1](https://arxiv.org/html/2609.11060#A2.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.11060#S1.p1.1),[§1](https://arxiv.org/html/2609.11060#S1.p3.1),[§1](https://arxiv.org/html/2609.11060#S1.p7.1),[§4](https://arxiv.org/html/2609.11060#S4.SS0.SSS0.Px1.p1.1)\.
- Bartlett \(1932\)F\. C\. BartlettRemembering: a study in experimental and social psychology\.The Cambridge Psychological Library,Cambridge University Press,Cambridge\.Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p5.1)\.
- Caoet al\.\(2026\)Z\. Cao, J\. Deng, L\. Yu, W\. Zhou, Z\. Liu, B\. Ding, and H\. ZhaoRemember me, refine me: a dynamic procedural memory framework for experience\-driven agent evolution\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 16803–16822\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.829)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p3.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: building production\-ready AI agents with scalable long\-term memory\.InECAI 2025,Frontiers in Artificial Intelligence and Applications, Vol\.413,pp\. 2993–3000\.External Links:[Document](https://dx.doi.org/10.3233/FAIA251160)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§1](https://arxiv.org/html/2609.11060#S1.p3.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Fanget al\.\(2026\)R\. Fang, Y\. Liang, X\. Wang, J\. Wu, S\. Qiao, P\. Xie, F\. Huang, H\. Chen, and N\. ZhangMemP: exploring agent procedural memory\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 17490–17502\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.866)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- GitHub \(2026\)GitHubGitHub Copilot SDK\.Note:Software and documentation:[https://github\.com/github/copilot\-sdk](https://github.com/github/copilot-sdk)Cited by:[§A\.1](https://arxiv.org/html/2609.11060#A1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.11060#S1.p7.1)\.
- Heet al\.\(2026\)Z\. He, Y\. Wang, C\. Zhi, Y\. Hu, T\. Chen, L\. Yin, Z\. Chen, T\. A\. Wu, S\. Ouyang, Z\. Wang, J\. Pei, J\. McAuley, Y\. Choi, and A\. PentlandMemoryArena: benchmarking agent memory in interdependent multi\-session agentic tasks\.External Links:2602\.16313,[Link](https://arxiv.org/abs/2602.16313)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p1.1)\.
- Huet al\.\(2026\)Y\. Hu, S\. Liu, Y\. Yue, G\. Zhang, B\. Liu, F\. Zhu, J\. Lin, H\. Guo, S\. Dou, Z\. Xi, S\. Jin, J\. Tan, Y\. Yin, J\. Liu, Z\. Zhang, Z\. Sun, Y\. Zhu, H\. Sun, B\. Peng, Z\. Cheng, X\. Fan, J\. Guo, X\. Yu, Z\. Zhou, Z\. Hu, J\. Huo, J\. Wang, Y\. Niu, Y\. Wang, Z\. Yin, X\. Hu, Y\. Liao, Q\. Li, K\. Wang, W\. Zhou, Y\. Liu, D\. Cheng, Q\. Zhang, T\. Gui, S\. Pan, Y\. Zhang, P\. Torr, Z\. Dou, J\. Wen, X\. Huang, Y\. Jiang, and S\. YanMemory in the age of AI agents\.External Links:2512\.13564,[Link](https://arxiv.org/abs/2512.13564)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§2](https://arxiv.org/html/2609.11060#S2.p1.1)\.
- Jiet al\.\(2026\)S\. Ji, Y\. Li, and B\. HooiMemory is reconstructed, not retrieved: graph memory for LLM agents\.Note:Accepted at ICML 2026External Links:2606\.06036,[Document](https://dx.doi.org/10.48550/arXiv.2606.06036),[Link](https://arxiv.org/abs/2606.06036)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Jiménez Gutiérrezet al\.\(2024\)B\. Jiménez Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. SuHippoRAG: neurobiologically inspired long\-term memory for large language models\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 59532–59569\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/6ddc001d07ca4f319af96a3024f6dbd1-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Kanget al\.\(2025\)J\. Kang, M\. Ji, Z\. Zhao, and T\. BaiMemory OS of AI agent\.External Links:2506\.06326,[Document](https://dx.doi.org/10.48550/arXiv.2506.06326),[Link](https://arxiv.org/abs/2506.06326)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. KielaRetrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.Cited by:[§2](https://arxiv.org/html/2609.11060#S2.p1.1)\.
- Liuet al\.\(2026\)J\. Liu, Y\. Su, P\. Xia, S\. Han, Z\. Zheng, C\. Xie, M\. Ding, and H\. YaoSimpleMem: efficient lifelong memory for LLM agents\.External Links:2601\.02553,[Document](https://dx.doi.org/10.48550/arXiv.2601.02553),[Link](https://arxiv.org/abs/2601.02553)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2025\)Y\. Liu, C\. Si, K\. R\. Narasimhan, and S\. YaoContextual experience replay for self\-improvement of language agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 14179–14198\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.694)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Mem0 \(2026\)Mem0Mem0: AI memory layer for agents and applications\.Note:Website:[https://mem0\.ai/](https://mem0.ai/)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1)\.
- Microsoft \(2026a\)MicrosoftAgents overview \(preview\)—Microsoft Copilot Studio \(new experience\)\.Note:Documentation:[https://learn\.microsoft\.com/en\-us/microsoft\-copilot\-studio/agents\-experience/overview](https://learn.microsoft.com/en-us/microsoft-copilot-studio/agents-experience/overview)Cited by:[§A\.2](https://arxiv.org/html/2609.11060#A1.SS2.p1.1)\.
- Microsoft \(2026b\)MicrosoftExtend your agent with Model Context Protocol\.Note:Documentation:[https://learn\.microsoft\.com/en\-us/microsoft\-copilot\-studio/agent\-extend\-action\-mcp](https://learn.microsoft.com/en-us/microsoft-copilot-studio/agent-extend-action-mcp)Cited by:[§A\.2](https://arxiv.org/html/2609.11060#A1.SS2.p1.1),[§1](https://arxiv.org/html/2609.11060#S1.p6.1)\.
- Microsoft \(2026c\)MicrosoftMemory \(preview\)—Microsoft Copilot Studio\.Note:Documentation:[https://learn\.microsoft\.com/en\-us/microsoft\-copilot\-studio/agents\-experience/memory\-overview](https://learn.microsoft.com/en-us/microsoft-copilot-studio/agents-experience/memory-overview)Cited by:[§A\.2](https://arxiv.org/html/2609.11060#A1.SS2.p2.1),[§1](https://arxiv.org/html/2609.11060#S1.p2.1)\.
- Microsoft \(2026d\)MicrosoftUse connectors in Microsoft Copilot Studio agents\.Note:Documentation:[https://learn\.microsoft\.com/en\-us/microsoft\-copilot\-studio/advanced\-connectors](https://learn.microsoft.com/en-us/microsoft-copilot-studio/advanced-connectors)Cited by:[§A\.2](https://arxiv.org/html/2609.11060#A1.SS2.p1.1),[§1](https://arxiv.org/html/2609.11060#S1.p6.1)\.
- Ouyanget al\.\(2025\)S\. Ouyang, J\. Yan, I\. Hsu, Y\. Chen, K\. Jiang, Z\. Wang, R\. Han, L\. T\. Le, S\. Daruki, X\. Tang, V\. Tirumalashetty, G\. Lee, M\. Rofouei, H\. Lin, J\. Han, C\. Lee, and T\. PfisterReasoningBank: scaling agent self\-evolving with reasoning memory\.External Links:2509\.25140,[Link](https://arxiv.org/abs/2509.25140)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§1](https://arxiv.org/html/2609.11060#S1.p3.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards LLMs as operating systems\.External Links:2310\.08560,[Link](https://arxiv.org/abs/2310.08560)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Panet al\.\(2025\)Z\. Pan, Q\. Wu, H\. Jiang, X\. Luo, H\. Cheng, D\. Li, Y\. Yang, C\. Lin, H\. V\. Zhao, L\. Qiu, and J\. GaoOn memory construction and retrieval for personalized conversational agents\.Note:Introduces SeCom; published at ICLR 2025External Links:2502\.05589,[Document](https://dx.doi.org/10.48550/arXiv.2502.05589),[Link](https://arxiv.org/abs/2502.05589)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Parket al\.\(2023\)J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. BernsteinGenerative agents: interactive simulacra of human behavior\.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology,pp\. 1–22\.External Links:[Document](https://dx.doi.org/10.1145/3586183.3606763)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Rasmussenet al\.\(2025\)P\. Rasmussen, P\. Paliychuk, T\. Beauvais, J\. Ryan, and D\. ChalefZep: a temporal knowledge graph architecture for agent memory\.External Links:2501\.13956,[Document](https://dx.doi.org/10.48550/arXiv.2501.13956),[Link](https://arxiv.org/abs/2501.13956)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Schacteret al\.\(2012\)D\. L\. Schacter, D\. R\. Addis, D\. Hassabis, V\. C\. Martin, R\. N\. Spreng, and K\. K\. SzpunarThe future of memory: remembering, imagining, and the brain\.Neuron76\(4\),pp\. 677–694\.External Links:[Document](https://dx.doi.org/10.1016/j.neuron.2012.11.001)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p5.1)\.
- Schacter and Addis \(2007\)D\. L\. Schacter and D\. R\. AddisThe cognitive neuroscience of constructive memory: remembering the past and imagining the future\.Philosophical Transactions of the Royal Society B: Biological Sciences362\(1481\),pp\. 773–786\.External Links:[Document](https://dx.doi.org/10.1098/rstb.2007.2087)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p5.1)\.
- Schacteret al\.\(2015\)D\. L\. Schacter, R\. G\. Benoit, F\. De Brigard, and K\. K\. SzpunarEpisodic future thinking and episodic counterfactual thinking: intersections between memory and decisions\.Neurobiology of Learning and Memory117,pp\. 14–21\.External Links:[Document](https://dx.doi.org/10.1016/j.nlm.2013.12.008)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p5.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://arxiv.org/abs/2303.11366)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Shuet al\.\(2026\)Y\. Shu, S\. P\. Jonnalagedda, X\. Gao, B\. J\. Gutiérrez, W\. Qi, K\. Das, H\. Sun, and Y\. SuREMem: reasoning with episodic memory in language agent\.arXiv preprint arXiv:2602\.13530\.Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Sukhbaataret al\.\(2018\)S\. Sukhbaatar, Z\. Lin, I\. Kostrikov, R\. Fergus, and A\. SzlamIntrinsic motivation and automatic curricula via asymmetric self\-play\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=SkT5Yg-RZ)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p5.1)\.
- Suzgunet al\.\(2025\)M\. Suzgun, M\. Yuksekgonul, F\. Bianchi, D\. Jurafsky, and J\. ZouDynamic cheatsheet: test\-time learning with adaptive memory\.External Links:2504\.07952,[Link](https://arxiv.org/abs/2504.07952)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Vidgenet al\.\(2026\)B\. Vidgen, A\. Mann, A\. Fennelly, J\. W\. Stanly, L\. Rothman, M\. Burstein, J\. Benchek, D\. Ostrofsky, A\. Ravichandran, D\. Sur, N\. Venugopal, A\. Hsia, I\. Robinson, C\. Huang, O\. Varones, D\. Khan, M\. Haines, A\. Bridges, J\. Boyle, K\. Twist, Z\. Richards, C\. Mahapatra, B\. Foody, and O\. NitskiAPEX\-Agents\.External Links:2601\.14242,[Link](https://arxiv.org/abs/2601.14242)Cited by:[§B\.1](https://arxiv.org/html/2609.11060#A2.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.11060#S1.p7.1),[§4](https://arxiv.org/html/2609.11060#S4.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2023\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.External Links:2305\.16291,[Link](https://arxiv.org/abs/2305.16291)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2024\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.External Links:2409\.07429,[Link](https://arxiv.org/abs/2409.07429)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Xionget al\.\(2026\)Z\. Xiong, Y\. Lin, W\. Xie, P\. He, Z\. Liu, J\. Tang, H\. Lakkaraju, and Z\. XiangHow memory management impacts LLM agents: an empirical study of experience\-following behavior\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),San Diego, California, United States,pp\. 623–645\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.27),[Link](https://aclanthology.org/2026.acl-long.27/)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p3.1)\.
- Xuet al\.\(2025\)W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. ZhangA\-MEM: agentic memory for LLM agents\.External Links:2502\.12110,[Document](https://dx.doi.org/10.48550/arXiv.2502.12110),[Link](https://arxiv.org/abs/2502.12110)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2026\)K\. Yang, Z\. Chen, X\. He, J\. Jiang, M\. Galley, C\. Wang, J\. Gao, J\. Han, and C\. ZhaiPlugMem: a task\-agnostic plugin memory module for LLM agents\.External Links:2603\.03296,[Document](https://dx.doi.org/10.48550/arXiv.2603.03296),[Link](https://arxiv.org/abs/2603.03296)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2024\)L\. Yang, Z\. Yu, T\. Zhang, S\. Cao, M\. Xu, W\. Zhang, J\. E\. Gonzalez, and B\. CuiBuffer of thoughts: thought\-augmented reasoning with large language models\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 113519–113544\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/cde328b7bf6358f5ebb91fe9c539745e-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Zep \(2026\)ZepZep: agent memory at enterprise scale\.Note:Website:[https://www\.getzep\.com/](https://www.getzep.com/)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1)\.
- Zhanget al\.\(2025\)Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li, U\. Thakker, J\. Zou, and K\. OlukotunAgentic context engineering: evolving contexts for self\-improving language models\.External Links:2510\.04618,[Link](https://arxiv.org/abs/2510.04618)Cited by:[§1](https://arxiv.org/html/2609.11060#S1.p2.1),[§1](https://arxiv.org/html/2609.11060#S1.p3.1),[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29936)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Zhenget al\.\(2024\)L\. Zheng, R\. Wang, X\. Wang, and B\. AnSynapse: trajectory\-as\-exemplar prompting with memory for computer control\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 19036–19066\.Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px2.p1.1)\.
- Zhonget al\.\(2024\)W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. WangMemoryBank: enhancing large language models with long\-term memory\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19724–19731\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29946)Cited by:[§2](https://arxiv.org/html/2609.11060#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix AImplementation Details and Extensions

### A\.1Runtime Architecture and Session Lifecycle

##### GitHub Copilot harness\.

We implement the runtime with the Python GitHub Copilot SDK, which exposes the same agent engine as GitHub Copilot CLI\. Each containerized sandbox launches a headless Copilot CLI process in server mode, and the SDK communicates with it through JSON\-RPC\([GitHub 2026](https://arxiv.org/html/2609.11060#bib.bib4)\)\. An SDK session specifies the model, system instructions, custom tools, MCP servers, permission policy, and event callbacks\. The callbacks provide the ordered model messages, tool calls, and observations used to construct each trajectory\.

##### Per\-task lifecycle\.

For taskSiS\_\{i\}, the harness creates a fresh task\-agent session with the environment tools andmemory\_read\. The external index persists across tasks, but the task agent has no memory\-write tool\. In the reported runs, after the task closes, a fresh distiller session receives the raw trajectoryτi\\tau\_\{i\}—not terminal feedback—and produces the distilled trajectorydid\_\{i\}\. Once feedbackgig\_\{i\}is available, another fresh session instantiates the curator agent withdid\_\{i\},gig\_\{i\}, and retrieved nearby memory records; the staged raw trajectory remains available as supporting evidence\. The curator agent receivesmemory\_read,memory\_create,memory\_update, andmemory\_delete; curation completes beforeSi\+1S\_\{i\+1\}is exposed\.

The trajectory\-only curator agent receives no task\-environment tools\. Its ordinary sandbox file readers can inspect the staged trajectory but cannot query the live benchmark world\. The environment\-probing curator agent uses the same model, inputs, memory CRUD tools, and core system prompt, while additionally receiving two probe\-specific instructions and the task’s read\-only tool subset\. For CLBench this subset is the database query interface; for adapted APEX it is supplied through the read\-only MCP configuration\. Appendix[D](https://arxiv.org/html/2609.11060#A4)gives the compact prompt specifications and highlights the probing additions\.

### A\.2Enterprise Deployment Mapping

The SDK–server decomposition mirrors the production\-oriented Microsoft Copilot Studio runtime, where an agent is configured from instructions, a selected model, knowledge and memory, callable tools and skills, and connected agents\([Microsoft 2026a](https://arxiv.org/html/2609.11060#bib.bib8)\)\. Connectors can wrap enterprise APIs, while MCP servers expose callable tools and file\-like resources such as API responses and document contents\([Microsoft 2026d](https://arxiv.org/html/2609.11060#bib.bib6);[Microsoft 2026b](https://arxiv.org/html/2609.11060#bib.bib7)\)\. Structured databases, document corpora, and enterprise applications can therefore all instantiate the environment\-tool interface used by our curator agent\.

In such a deployment, probing does not require a second integration path\. The asynchronous curator agent can be assigned a least\-privilege, read\-only subset of connectors or MCP tools already registered for the responding agent\. Existing authentication, authorization, and audit boundaries remain in force, and the curator agent receives no production write authority\. This makes the intervention compatible with increasingly common managed\-memory abstractions\([Microsoft 2026c](https://arxiv.org/html/2609.11060#bib.bib5)\)without retraining the model or changing the task\-time agent\.

## Appendix BExperimental and Evaluation Details

### B\.1Benchmark Construction

##### CLBench database exploration\.

The primary CLBench stream contains 40 SQL questions over a hidden SQLite database\([Asawa et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib12)\)\. The agent must discover tables, joins, encodings, and conventions through queries\. Format traps include prices in dollars versus cents, epoch\-millisecond versus ISO timestamps, and abbreviated column names\. After question 20, an unannounced migration renames tables, splits columns, and introduces soft deletes, testing both schema transfer and repair of previously valid memory\. The cross\-model study instead uses the canonical 30\-question, single\-stage schedule without migration\.

##### Adapted APEX Agents\.

APEX Agents contains 480 independent workplace tasks spanning management consulting, law, and financial analysis\([Vidgen et al\. 2026](https://arxiv.org/html/2609.11060#bib.bib15)\)\. We select the management\-consulting subset and group questions by\(domain, world\_id\), turning each shared world into an ordered continual\-learning stream\. This yields six worlds and 90 questions, with 11–18 questions per world\. Tasks require discovery across PDF, XLSX, DOCX, and PPTX files, including embedded images; quantitative analysis through code execution; and artifact production through MCP\-style Archipelago tools\. Grouping by world makes file locations, workbook layouts, tool conventions, and distinctions among hard\-negative documents reusable across tasks\.

### B\.2Models and Run Protocol

The primary 40\-question CLBench and adapted APEX studies usegpt\-5\.4at xhigh reasoning effort for sessions of the task agent, distiller, and curator agent\. The no\-drift study uses Sonnet 4\.6 at high effort and Opus 4\.7 at xhigh effort for all applicable roles\. Memory conditions useintfloat/e5\-base\-v2embeddings\.

The primary CLBench study uses five independently shuffled, paired runs for every configuration\. Adapted APEX uses five runs per stateful configuration and three stateless runs\. Shuffles are seeded and shared across systems\. Reported intervals are 95% Student\-ttintervals over run\-level aggregates\. The no\-drift study instead holds the canonical 30\-question order fixed; each model–memory comparison has five paired memory and GHCP \(No Memory\) runs, and uncertainty is the across\-run standard deviation\.

### B\.3Metrics and Accounting

For taskii, letpi∈\{0,1\}p\_\{i\}\\in\\\{0,1\\\}be its binary pass score and letqiq\_\{i\}be the number of task\-agent tool calls counted by the benchmark\. CLBench has one correctness criterion\. For tasks with multiple grader criteria, we use a strict pass:pi=1p\_\{i\}=1only when every criterion passes, andpi=0p\_\{i\}=0otherwise\. We compute the primary pass\-discounted rewardrir\_\{i\}using Equation[3](https://arxiv.org/html/2609.11060#S4.E3), withB=15B=15exploratory SQL queries for CLBench andB=100B=100Archipelago tool calls for APEX\. For trajectory diagnosis only, APEX also reports the fractional\-criteria scorerifrac=\(ki/Ki\)​\(1−qi/B\)r\_\{i\}^\{\\mathrm\{frac\}\}=\(k\_\{i\}/K\_\{i\}\)\(1\-q\_\{i\}/B\), wherekik\_\{i\}ofKiK\_\{i\}criteria pass\. We report total reward as the sum of per\-task rewards and mean reward as that total divided by the number of tasks\.

Memory\-management and harness\-internal calls are excluded fromqiq\_\{i\}\. Reported tokens and USD cost cover the task\-agent response phase; usage by the distiller and curator agent is tracked separately\. In addition to reward, we report pass rate, task\-agent tool calls per question, and input/output tokens\. At task positionjj, a learning\-curve point is the across\-run mean of the running mean reward through positionjj; shaded bands show one across\-run standard deviation\.

## Appendix CPer\-World APEX Results

Table[3](https://arxiv.org/html/2609.11060#A3.T3)reports the absolute rewards, task\-agent tool calls, and confidence intervals underlying the compact gains in Table[2](https://arxiv.org/html/2609.11060#S5.T2)\. This breakdown preserves the variation across difficulty tiers and document environments summarized in Section[5](https://arxiv.org/html/2609.11060#S5)\.

Table 3:Absolute APEX results underlying Table[2](https://arxiv.org/html/2609.11060#S5.T2): for each world \[difficulty,NN\], Reward is total strict pass\-discounted reward and Tool calls are benchmark\-counted task\-agent Archipelago calls per question excluding memory/harness calls; values are run means±\\pm95% Student\-ttconfidence intervals \(n=3n=3for GHCP \(No Memory\),n=5n=5otherwise\), with highest reward and lowest calls bolded\.
## Appendix DPrompt Templates

### D\.1Distiller Preprocessing Prompt

The distiller is a pure preprocessing step enabled for both memory conditions in our experiments\. It sees the completed raw trajectory, but not terminal benchmark feedback, and has no memory or environment tools\. Its output is a compact evidence packet for the curator agent; it cannot create, update, or delete durable records\.

System message\.

ROLE

Youarethenon\-writingtrajectorydistillerinacontinual\-learning

memorypipeline\.

BOUNDARY

Thisispreprocessingonly\.Youhavenomemorytoolsandmustnot

create,update,delete,orproposedurablememoryrecords\.Youdonot

receiveterminalbenchmarkfeedback\.Treatthesuppliedrolloutas

partialevidence,notasgroundtruth\.

INPUT

Onecompletedtask\-agenttrajectorycontainingtheuserrequest,

retrievedmemories,assistantmessages,toolcalls,toolresults,

environmentobservations,andsubmittedanswer\.

OBJECTIVE

Transformtherawtrajectoryintoacompact,faithfulevidencepacket

thathelpsalatercuratoragentdecidewhatisreusable\.Donotevaluate

memorypolicyoraddfactsthatareabsentfromthetrajectory\.

PRESERVE

\-thetaskgoal,constraints,andresponserequirements;

\-retrievedmemoriesandhowtheagentusedorcontradictedthem;

\-thechronologicalstrategyanddecisiveaction\-observationpairs;

\-successfulandfailedprocedures,toolconventions,andenvironment

structurediscoveredduringexecution;

\-thesubmittedanswer,unresolvedquestions,andassumptionsthat

remainunverified\.

OUTPUT

Returnexactlythesetaggedsections:

<overview\>task,constraints,andapproach</overview\>

<history\>chronologicalactionsandobservations</history\>

<work\_done\>completionstateandsubmittedresult</work\_done\>

<technical\_details\>reusablefindings,failures,andquirks</technical\_details\>

<important\_files\>filesorresourcescentraltothetask</important\_files\>

<next\_steps\>unresolvedworkandunverifiedassumptions</next\_steps\>

<checkpoint\_title\>aconcise2\-6wordtitle</checkpoint\_title\>

Beconcise,butretainevidencethatwouldbecostlytorediscover\.

Refertothetaskagentinthethirdperson\.

Per\-instance user message\.

Instance:\{INSTANCE\_ID\}

Preprocessthecompletedtask\-agentrolloutbelow\.

<trajectory\>

\{RAW\_TRAJECTORY\}

</trajectory\>

### D\.2Trajectory\-Only Curator Agent Prompt

GHCP \+ Mem uses the following compact prompt for the curator agent\. Unlike the distiller, this post\-task curator agent receives terminal feedback and memory CRUD tools\. It may inspect the staged raw trace when the distilled trajectory omits a needed detail, but it has no live task\-environment tools\.

System message\.

ROLE

Youarethepost\-taskcuratoragentforacontinual\-learningrun\.

Onlyyoumaymutatethepersistentmemoryindex\.

INPUTS

1\.Adistilledevidencepacketforonecompletedtask\-agentrollout\.

2\.Terminalfeedbackforthatrollout,whenavailable\.

3\.Relevantexistingmemoryrecords\.

4\.Anoptionalstagedrawtrajectoryforevidencelookup\.

GOAL

Maintainasmallsetofreliable,transferable,andactionable

memoriesthatletsafutureagentsolverelatedtasksmoreaccurately

andwithfewerenvironmentcalls\.

AVAILABLETOOLS

\-memory\_read,memory\_create,memory\_update,memory\_delete;

\-sandboxreadersforinspectingthestagedrawtrajectory\.

RECORDCONTRACT

Everycreatedrecordmustcontain:

\-category:pattern\|rule\|trap\|schema\|policy\|interaction;

\-confidence:high\|medium\|low;

\-applies\_to:ashortretrievalscope;

\-lemma:oneconcise,actionableclaim\.

PROCEDURE

1\.PROPOSEatomiccandidatememoriesfromtheevidencepacket,

feedback,rawtraceasneeded,andnearbyexistingrecords\.

2\.CHECKeachcandidateforevidentialsupport,transfervalue,scope,

actionability,currentvalidity,andredundancy\.Asuccessfultask

doesnotvalidateeveryintermediateassumption\.

3\.RECONCILEwithexistingmemory:

\-Createasupported,nonredundantcandidate\.

\-Updateormergewhenevidencerefinesanexistingrecord\.

\-Narrow,correct,ordeleteacontradictedrecord\.

\-Skipunsupported,trivial,orinstance\-specificcontent\.

4\.COMMITtheminimumCRUDoperationsneededtoleaveacoherentindex\.

WRITINGRULES

\-Storeprocedures,relations,conventions,andscopedwarnings;never

memorizethetaskanswer,rubricwording,orincidentalvalues\.

\-Preferpositiverulesthattellthenextagentwhattodo\.

\-Scopenoclaimmorebroadlythanitsevidencesupports\.

\-Preferfewer,strongerrecordsovermanynoisyones\.

STOP

WhennofurtherjustifiedCRUDoperationremains,stopusingtoolsand

brieflysummarizewhatchanged\.

### D\.3Environment\-Probing Curator Agent Prompt

The environment\-probing system message deliberately preserves the curator agent’s core prompt above\. It is formed by inserting only two probe\-specific blocks:

Pprobe=Pcurator\+Aaccess\+Averify\.P\_\{\\mathrm\{probe\}\}=P\_\{\\mathrm\{curator\}\}\+A\_\{\\mathrm\{access\}\}\+A\_\{\\mathrm\{verify\}\}\.Reading the shared prompt for the curator agent with the two blue\-highlighted blocks below inserted at the stated locations gives the complete compact probing prompt\. No input, memory schema, CRUD policy, or stopping rule otherwise changes\.

Highlighted addition 1: read\-only tool access\.

Insert underAvailable Tools\.

ENVIRONMENT\-PROBINGADDITION

Youalsohaveread\-onlytask\-environmenttools\.Thesetoolscannot

mutatetheenvironment,consumethetaskagent’sbudget,orexpose

futuretasksorlabels\.

Highlighted addition 2: verification nudge\.

Insert betweenCheckandReconcile\.

ENVIRONMENT\-PROBINGADDITION

Usetheread\-onlyenvironmenttoolstoverifycandidateandexisting

memoriesbeforeCreateorUpdatewhenevercorrectness,scope,

freshness,oractionabilityisuncertain\.Probecounterexamples,

untouchedslices,stalemappings,requiredpreconditions,andwhether

ashorterprocedureyieldsthesameevidence\.

Probeonlytoevaluateacandidatememory,nottosolveafuturetask

orexplorewithoutahypothesis\.Useprobeevidencetostrengthenor

narrowasupportedrecordandtoupdateordeleteacontradictedone\.

### D\.4Shared Curator Agent User\-Message Template

Both curation conditions receive the same dynamic evidence message\. The raw trajectory pointer is omitted only when no staged trace is available\.

<distilled\_evidence\>

\{DISTILLED\_TRAJECTORY\}

</distilled\_evidence\>

<terminal\_feedback\>

\{TERMINAL\_FEEDBACK\_OR\_NONE\}

</terminal\_feedback\>

<related\_memory\>

\{RELATED\_MEMORY\_ENTRIES\_OR\_NONE\}

</related\_memory\>

<raw\_trajectorypath="\{TRAJECTORY\_JSON\_PATH\_OR\_NONE\}"/\>

Reconcilethememoryindexusingtheavailabletools\.Stopwhenno

furtherjustifiedoperationremains\.

## Appendix EMemory Samples and Trajectory Side\-by\-Sides

This appendix first compares representative curator\-agent memory records from the CLBench database\-exploration runs\. It then expands two matched cases, one from each benchmark, as trajectory side\-by\-sides\.

### E\.1CLBench Curator Memory Examples

Figure[3](https://arxiv.org/html/2609.11060#A5.F3)shows examples from the CLBench database\-exploration task runs and the kinds of memories generated by the curator agent\. The left column samples trajectory\-only curation; the right column samples the same memory schema when the curator agent can make targeted read\-only environment probes\. Record text is lightly shortened for layout, while table names, fields, relations, and values are preserved\.

GHCP \+ Mem

Trajectory\-only curator agent

\[trap\]Answer\-Anchored Warningapplies\_to:electronics average\-listed\-price questionslemma:Do not answer withAVG\(items\_g2\.prc\_usd\)over non\-null rows; that produces about 52\.96, but the benchmark’s correct result is 96\.23, so a different price field and/or row subset is required\.

\[schema\]Broad Map, Missing Relationapplies\_to:grouped product and review tableslemma:Across the grouped product/review tables,g1= office products,g2= electronics, andg3= musical instruments; this mapping applies to bothitems\_g\*andfdbk\_g\*\.

\[rule\]Stale Table Nameapplies\_to:musical\-instrument brand questionslemma:Useattrs\_g3\(notitems\_g3\); filterattr\_key=’Brand’, ignore blankattr\_val, and count distinctref\_id\.*After migration,attrs\_g3no longer exists\.*

GHCP \+ Mem \(w/ Env Probing\)

Environment\-probing curator agent

\[rule\]Executable Join and Filterapplies\_to:g2top\-level\-category price shareslemma:Joinitems\_g2totaxn\_g2onref\_id; filtercat\_lvl=1and the exactcat\_nm; keepitems\_g2\.prc\>0; then compare against the filteredAVG\(prc\)\.

\[rule\]Positive Relation and Grainapplies\_to:grouped review\-average questionslemma:Useitems\_g\*for product\-side filters, join tofdbk\_g\*onref\_id, and aggregatefdbk\_g\*\.rtgrather than averaging item\-sideavg\_rtg\.

\[schema\]Current Migrated Schemaapplies\_to:musical\-instrument attributes and brandslemma:Useproduct\_attributes\_g3for musical attributes\. For brand questions, filterattr\_key=’Brand’with nonblankattr\_val, group byattr\_val, and count distinctref\_id\.

Figure 3:Representative CLBench memory records generated by the curator agent\. Trajectory\-only curation \(left\) can preserve a rejected answer, underspecify the positive operation, or retain a stale schema name\. Environment probing \(right\) yields positive procedures with explicit joins, filters, aggregation grain, and current schema\.The trajectory\-only records are not uniformly wrong: they recover useful domain mappings and warnings\. Their weakness is actionability\. The first record says what failed but not what should replace it; the second orients the agent to the domain but does not identify the relation needed by the current question; and the third is a once\-valid rule that survived schema drift\. A later task agent therefore has to reconstruct the missing evidence\.

The probe\-backed records instead verbalize observations that can be executed directly: which tables to use, how they join, which rows define the denominator, what aggregation grain is valid, and which schema name is current\. In the three matched CLBench examples, GHCP \+ Mem used 4, 9, and 8 queries, whereas GHCP \+ Mem \(w/ Env Probing\) used 1, 2, and 1\. The examples do not imply that every probed record is complete, but they show how read\-only checks can convert a warning or tentative mapping into an environment\-informed procedure before a future task retrieves it\.

The side\-by\-sides hold the task fixed acrossBase\(GHCP \(No Memory\)\),Mem\(GHCP \+ Mem\), andProbe\(GHCP \+ Mem \(w/ Env Probing\)\)\. The lanes show task\-time behavior—memory retrieval, selected tool calls, the final answer, and grader evidence—not the earlier post\-task sessions that produced the retrieved records\.

For both memory conditions in the reported runs, a non\-writing distiller transformed prior completed trajectories into evidence packets, after which a separate curator agent used memory CRUD to reconcile candidate lemmas with the persistent index\. OnlyProbegave that curator agent the read\-only benchmark tools and the highlighted verification instructions in Appendix[D](https://arxiv.org/html/2609.11060#A4)\. During the displayed task, both memory conditions expose onlymemory\_read, and all three responding agents retain the same environment\-tool surface\. The comparison therefore isolates the task\-time effect of previously curated memory, including the additional environment evidence supplied by curator\-side probes\.

Each case appears as sequential red \(Base\), gold \(Mem\), and blue \(Probe\) lanes\. Agent calls and submitted answers are right\-aligned; memory results, environment observations, call summaries, and grader evidence are left\-aligned\. Lane headers report task\-agent response cost and input, output, and cached tokens; distiller and curator\-agent usage are not included\.

These are selected\-evidence traces rather than full transcripts\. CLBench shows the first and final SQL queries; APEX shows the first two discovery calls and final computation call\. Each displayed call gives its position in the recorded trace, and bucket cards account for every omitted call without printing its arguments or result\. APEX positions count recorded top\-level calls, while lane headers report total task\-agent tool calls\. Tagged observations are capped at 500 source characters and carry an original\-length marker when truncated\. Memory payloads, final answers, and grader evidence are reproduced from the recorded runs; internal model reasoning is omitted\.

### E\.2CLBench Database Exploration

The selected CLBench case captures both correctness and efficiency on a question about products that have reviews but no attributes row\. The stateless agent adds an incorrect category condition and answers 188 after seven database queries\. GHCP \+ Mem retrieves useful mappings and warnings but not the required positive join pattern; it explores for nine queries and answers 267\. GHCP \+ Mem \(w/ Env Probing\) retrieves the curator agent’s live\-validatedref\_idrelation and answers 267 in two queries\. A wrong answer receives zero reward under Equation[3](https://arxiv.org/html/2609.11060#S4.E3)\.

#### C2: Filling a Knowledge Blind Spot\.

Run 3; position \#7 of 40; memory conditions retrieve records curated from prior completed tasks\.

QUESTION AND GROUND TRUTHQuestionHow many office products have at least one review in the database but are NOT represented in the attributes data at all?Ground truthAccepted answer: 267\.

GHCP — NO MEMORYconfig: GHCP \(No Memory\)status: failure \| reward: 0\.00 \| queries: 7 \| 8 turnscost: $0\.0684 \| in: 63\.1K \| out: 1\.5K \| cached: 49\.9K

MEMORY

memory retrievalNo memory is available to the stateless baseline\.

SELECTED AGENT CALLS

agent · query · representative discoveryRecorded query 1 of 7 \| 0 before \| 6 after\.tables

env · observation · representative discoveryQuery result \(1/15 queries used, 14 remaining\):attrs\_g1attrs\_g3fdbk\_g1fdbk\_g2fdbk\_g3fdbk\_stats\_g1items\_g1items\_g2items\_g3sqlite\_sequencetaxn\_g1taxn\_g2

agent · query · decisive computationRecorded query 7 of 7 \| 6 before \| 0 afterSELECT COUNT\(\*\) AS cnt FROM items\_g1 i WHERE i\.main\_cat=’Office Products’ AND EXISTS \(SELECT 1 FROM fdbk\_g1 f WHERE f\.ref\_id=i\.ref\_id\) AND NOT EXISTS \(SELECT 1 FROM attrs\_g1 a WHERE a\.ref\_id=i\.ref\_id\);

env · observation · decisive computationQuery result \(7/15 queries used, 8 remaining\):cnt\-\-\-188

OTHER RECORDED CALLS BY BUCKET

other recorded queriesOther recorded queries \(not expanded\): schema inspection x3; data exploration/validation x2\. Accounting: 7 recorded; 2 shown; 5 not expanded\.

FINAL ANSWER AND GRADING

submitted answer188

grader evidence · FAILQuestion 1: INCORRECT\.Your answer: 188Correct answer: 267Exploratory queries used: 7

GHCP \+ Memconfig: GHCP \+ Memstatus: success \| reward: 0\.40 \| queries: 9 \| 10 turnscost: $0\.0983 \| in: 94\.7K \| out: 2\.7K \| cached: 79\.5K

MEMORY

agent · memory\_read\{"query": "Question 7: determine count of office products with at least one review but no attributes representation; need identify relevant tables and likely join keys\.","k": 5\}

memory · complete logical payload\#1 \[trap\]applies\_to: SQLite product\-dataset benchmark questions about reviewers spanning 2\+ top\-level categories within office productslemma: Do not treat grouped datasets g1/g2/g3 as the requested ’top\-level categories’; for office\-products category\-span questions, join office reviews \(fdbk\_g1\) to office item/taxonomy data \(e\.g\. items\_g1/taxn\_\*\) and count distinct top\-level categories within the office domain\.\#2 \[schema\]applies\_to: SQLite product\-dataset benchmark with grouped items\_g\* and fdbk\_g\* tableslemma: Across the grouped product/review tables, g1 = office products, g2 = electronics, and g3 = musical instruments; this mapping applies to both items\_g\* and fdbk\_g\* tables\.\#3 \[schema\]applies\_to: SQLite product\-dataset benchmark questions over grouped feedback tables fdbk\_g\* that filter verified or non\-empty review textlemma: In grouped feedback tables fdbk\_g\*, rtg is the review rating, body is the review text, vrf is the verification flag \(e\.g\. filter verified with vrf = ’true’\), and non\-empty text should be filtered with body IS NOT NULL AND TRIM\(body\) <\> ’’\.\#4 \[trap\]applies\_to: SQLite product\-dataset benchmark questions asking for average review rating by an item attribute \(for example, whether price is listed\)lemma: Do not answer ’average review rating’ questions with AVG\(items\_\*\.avg\_rtg\); the benchmark expects averaging review rows from fdbk\_\*\.rtg after joining to items\_\* only to classify products by the requested attribute\.\#5 \[trap\]applies\_to: Questions about average listed price for electronics in the grouped SQLite product datasetslemma: Do not answer electronics ’average listed price’ with AVG\(items\_g2\.prc\_usd\) over non\-null rows; that produces about 52\.96, but the benchmark’s correct result is 96\.23, so a different price field and/or row subset is required\.

SELECTED AGENT CALLS

agent · query · representative discoveryRecorded query 1 of 9 \| 0 before \| 8 after\.tables

env · observation · representative discoveryQuery result \(1/15 queries used, 14 remaining\):attrs\_g1attrs\_g3fdbk\_g1fdbk\_g2fdbk\_g3fdbk\_stats\_g1items\_g1items\_g2items\_g3sqlite\_sequencetaxn\_g1taxn\_g2

agent · query · decisive computationRecorded query 9 of 9 \| 8 before \| 0 afterSELECT \(SELECT COUNT\(\*\) FROM \(SELECT DISTINCT ref\_id FROM fdbk\_g1 WHERE ref\_id IS NOT NULL EXCEPT SELECT ref\_id FROM items\_g1\)\) AS review\_refids\_not\_in\_items, \(SELECT COUNT\(\*\) FROM \(SELECT DISTINCT ref\_id FROM items\_g1 WHERE ref\_id IN \(SELECT ref\_id FROM fdbk\_g1 WHERE ref\_id IS NOT NULL\) EXCEPT SELECT DISTINCT ref\_id FROM attrs\_g1 WHERE ref\_id IS NOT NULL\)\) AS item\_refids\_with\_review\_missing\_attrs

env · observation · decisive computationQuery result \(9/15 queries used, 6 remaining\):review\_refids\_not\_in\_items \| item\_refids\_with\_review\_missing\_attrs\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-0 \| 267

OTHER RECORDED CALLS BY BUCKET

other recorded queriesOther recorded queries \(not expanded\): schema inspection x4; data exploration/validation x3\. Accounting: 9 recorded; 2 shown; 7 not expanded\.

FINAL ANSWER AND GRADING

submitted answer267

grader evidence · PASSQuestion 7: CORRECT\!Your answer: 267Exploratory queries used: 9

GHCP \+ Mem \(w/ Env Probing\)config: GHCP \+ Mem \(w/ Env Probing\)status: success \| reward: 0\.87 \| queries: 2 \| 3 turnscost: $0\.0214 \| in: 25\.0K \| out: 568 \| cached: 22\.0K

MEMORY

agent · memory\_read\{"query": "Question 7: count office products with at least one review but no attributes rows; need identify office tables and likely join key","k": 5\}

memory · complete logical payload\#1 \[rule\]applies\_to: g1 reviewer category\-span querieslemma: For office\-products reviewer span questions, join fdbk\_g1 to taxn\_g1 by ref\_id, treat taxn\_g1\.cat\_lvl=1 cat\_nm as the top\-level category, count reviewers with at least 2 distinct cat\_nm, and divide by COUNT\(DISTINCT uid\) from fdbk\_g1\.\#2 \[rule\]applies\_to: g1/g3 brand product countslemma: For office\-products or musical\-instruments brand questions, read brand from attrs\_g1/attrs\_g3 by filtering attr\_key=’Brand’ and aggregate COUNT\(DISTINCT ref\_id\) by attr\_val\.\#3 \[rule\]applies\_to: grouped average review rating querieslemma: For g1/g2/g3 questions comparing average review rating across product attributes, join fdbk\_g\* to items\_g\* on ref\_id and average fdbk\_g\*\.rtg within each attribute\-defined group\.\#4 \[rule\]applies\_to: g2 electronics review body\-length querieslemma: For electronics review\-text metrics, aggregate directly from fdbk\_g2: use rtg for stars and body for text, filter verified reviews with vrf=’true’, exclude blank reviews with body IS NOT NULL AND TRIM\(body\)<\>’’ and compute measures like AVG\(LENGTH\(body\)\) over the requested rating groups\.\#5 \[schema\]applies\_to: grouped product domain mappinglemma: In the g1/g2/g3 product split, use g1 for office products, g2 for electronics, and g3 for musical instruments\.

SELECTED AGENT CALLS

agent · query · representative discoveryRecorded query 1 of 2 \| 0 before \| 1 afterSELECT name FROM sqlite\_master WHERE type=’table’ ORDER BY name;

env · observation · representative discoveryQuery result \(1/15 queries used, 14 remaining\):name\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-attrs\_g1attrs\_g3fdbk\_g1fdbk\_g2fdbk\_g3fdbk\_stats\_g1items\_g1items\_g2items\_g3sqlite\_sequencetaxn\_g1taxn\_g2

agent · query · decisive computationRecorded query 2 of 2 \| 1 before \| 0 afterSELECT COUNT\(DISTINCT f\.ref\_id\) AS missing\_attrs\_products FROM fdbk\_g1 f LEFT JOIN attrs\_g1 a ON f\.ref\_id = a\.ref\_id WHERE a\.ref\_id IS NULL;

env · observation · decisive computationQuery result \(2/15 queries used, 13 remaining\):missing\_attrs\_products\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-267

FINAL ANSWER AND GRADING

submitted answer267

grader evidence · PASSQuestion 7: CORRECT\!Your answer: 267Exploratory queries used: 2

### E\.3Adapted APEX Management Consulting

The selected APEX case asks which Impact site has the largest absolutezz\-score for 2024 average monthly US revenue per head\. The stateless agent exhausts 96 calls and returns the incorrect answerDarcylis,z=1\.29z=1\.29, satisfying neither criterion\. GHCP \+ Mem transfers the computation recipe and returnsLorexa,z=−1\.60z=\-1\.60, in 11 calls; GHCP \+ Mem \(w/ Env Probing\) returns the same correct answer in six calls using its validated workbook map\. Consistent with Equation[3](https://arxiv.org/html/2609.11060#S4.E3), the lane headers report the primary rewardrir\_\{i\}, fractional\-criteria diagnosticrifracr\_\{i\}^\{\\mathrm\{frac\}\}, and task\-agent tool\-call countqiq\_\{i\}\.

#### A3: Transferring a Computation Recipe\.

Run 4; position \#11 of 15; memory conditions retrieve records curated from prior completed tasks\.

QUESTION AND GROUND TRUTHQuestionCan you please calculate the z score of US 2024 Average Monthly Revenue per Head, for all of Impact’s sites?Use a distribution of US 2024 Average Monthly Revenue per Head per site for all the sites in the attached file, which has the monthly US operational data for all of Impact’s and competitor’s sites\. You can allocate the yearly revenue from the respective P&L equally across all the respective sites and months\.Tell me here the z score of only the Impact site with the highest absolute z score, and the SiteID of this Impact site\. Use the standard deviation formula for sample, not population\. Round the final answer to two decimal places\. Write back to me with what I’ve asked for\.Ground truthLorexa, z = \-1\.60\.

GHCP — NO MEMORYconfig: GHCP \(No Memory\)status: failure \| criteria: 0/2 \| primary reward: 0\.00 \| fractional reward: 0\.00 \| tool calls: 96cost: $4\.1622 \| in: 3\.81M \| out: 30\.2K \| cached: 3\.46M

MEMORY

memory retrievalNo memory is available to the stateless baseline\.

SELECTED AGENT CALLS

agent · archipelago\-filesystem\_server\_list\_files · representative discoveryRecorded top\-level call 1 of 31 \| 0 before \| 30 after\{"path": "/"\}

env · observation · representative discovery’4\. Received From Client’ \(folder\)’2\. Analysis’ \(folder\)’1\. Presentations’ \(folder\)’3\. Research’ \(folder\)’0\. Admin’ \(folder\)

agent · archipelago\-filesystem\_server\_get\_directory\_tree · representative discoveryRecorded top\-level call 2 of 31 \| 1 before \| 29 after\{"path": "/", "max\_depth": 2, "include\_files": true, "show\_size": false\}

env · observation · representative discovery/\+\-\- 0\. Admin/\| \+\-\- Meeting Notes/\+\-\- 1\. Presentations/\| \+\-\- Decks/\| \+\-\- Outline/\+\-\- 2\. Analysis/\| \+\-\- Archive/\| \+\-\- Site Ops/\| \+\-\- Supply Chain/\| \+\-\- Impact vs Comps High Level Metrics\.xlsx\| \+\-\- Impact\_SG&A Competitor Analysis\.xlsx\| \+\-\- UPDATED\_Analysis \- Impact\_US\_MarketAccessAgency\_Fees\_2020\_2024\_REVISED\.xlsx\| \+\-\- UPDATED\_Analysis \- Sales\_Force\_Comp\_2022\_2024\_CORRECTED\.xlsx\| \+\-\- UPDATED\_Impact\_Campaign\_Analysis\.xlsx\| \+\-\- UPDATED\_Impact\_RealEstate\_Analysi\[Observation capped at 500 source characters; original length: 630 source characters\.\]

agent · archipelago\-code\_execution\_server\_code\_exec · final computation attempt \(incorrect\)Recorded top\-level call 31 of 31 \| 30 before \| 0 after\{"request": \{"code": "python \- <<’PY’\\nimport pandas as pd\\nfrom openpyxl import load\_workbook\\n\# revenue\\nrev = 8324082\.0\\n\# headcount data\\npath=’/filesystem/4\. Received From Client/Impact Therapeutics/Talent\_Comp\_Turnover\_AllSites \(1\)\.xlsx’\\nwb=load\_workbook\(path, read\_only=True, data\_only=True\)\\nrows=list\(wb\[’Overtime\_and\_Vacancy\_Log’\]\.values\)\\nwb\.close\(\)\\ndf=pd\.DataFrame\(rows\[1:\], columns=rows\[0\]\)\[\[’month’,’site\_id’,’headcount\_actual’\]\]\.dropna\(\)\\nsite\_count=df\[’site\_id’\]\.nunique\(\)\\nmonthly\_revenue\_per\_site=rev/site\_count/12\\nsite\_metrics=df\.assign\(monthly\_rev\_per\_head=monthly\_revenue\_per\_…\[truncated 414 chars\]

env · observation · final computation attempt \(incorrect\)\{"success": true,"output": "site\_count 5\\nmonthly\_revenue\_per\_site 138734\.69999999998\\n\{’Darcylis’: 289\.38303158338863, ’Lorexa’: 266\.9690189637408, ’Noralix’: 177\.86499999999998, ’Papinex\-9’: 182\.66596483349187, ’Strevalent\-20’: 200\.63943080874193\}\\nmean 223\.50448923787266\\nsd\_sample 51\.24135692058095\\nz \{’Darcylis’: 1\.2856517919230985, ’Noralix’: \-0\.8906768278718573, ’Lorexa’: 0\.8482314352688562, ’Papinex\-9’: \-0\.7969836643412171, ’Strevalent\-20’: \-0\.44622273497888265\}\\n"\}

failed\-path noteNo valid decisive cross\-company computation emerged; the trajectory ended with the incorrect Darcylis, z=1\.29 result\.

OTHER RECORDED CALLS BY BUCKET

other recorded callsOther recorded calls \(not expanded\): filesystem x11; spreadsheet x7; code execution x8; mail x2\. Accounting: 31 recorded; 3 shown; 28 not expanded\. 96 total tool calls\.

FINAL ANSWER AND GRADING

submitted final answer<TEXT\_RESPONSE\>Using the available site\-level monthly US ops data in the workspace, the Impact site with the highest absolute z score is:\*\*SiteID:\*\* Darcylis\*\*z score:\*\* \*\*1\.29\*\*I could not locate a combined cross\-company site\-level operational attachment, so this result is based on the accessible Impact site\-level monthly headcount file and Impact’s 2024 US revenue, using \*\*sample\*\* standard deviation\. Sources: \[Impact vs Comps High Level Metrics\.xlsx \(‘Impact US‘\)\]\(/2\. Analysis/Impact vs Comps High Level Metrics\.xlsx\), \[Talent\_Comp\_Turnover\_AllSites \(1\)\.xlsx \(‘Overtime\_and\_Vacancy\_Log‘\)\]\(/4\. Received From Client/Impact Therapeutics/Talent\_Comp\_Turnover\_AllSites \(1\)\.xlsx\) <DONE\></TEXT\_RESPONSE\><FILE\_CHANGES\><ARTIFACT id="1" type="sheet" change="deleted"\><path\>Manufacturing\_Site\_Operations\_2020\_2024\.xlsx</path\><title\>Manufacturing\_Site\_Operations\_2</title\><sub\_index\>1</sub\_index\><deleted\_content\>RecordID,Company,SiteID,Year,Month,ProductType,BatchesProduced,BatchesPassed,EquipmentType,EquipmentUtilization,HeadcountFTE,CapexInvested,MaintenanceHours,EnergyConsumptionMWh,WasteKg1,ImpactTherapeutics,Darcylis,2020,1,Gene Therapy,9,8,Fill\-Finish,0\.862,31\.7,250625,56,421\.5,800\.72,ImpactTherapeutics,Darcylis,2020,2,Gene Therapy,10,8,Fermentation,0\.938,39\.5,145553,69\.1,228\.4,747\.63,ImpactTherapeutics,Darcylis,2020,3,Biologic,8,6,Fermentation,0\.76,25\.2,71000,195\.8,262\.9,363\.14,ImpactTherapeutics,Darcylis,2020,4,Gene Therapy,5,4,Quality Control,0\.737,40\.4,356138,112\.1,109\.3,18965,ImpactTherapeutics,Darcylis,2020,5,Small Molecule,4,3,Fermentation,0\.642,20,357469,137\.6,683\.2,512\.16,ImpactTherapeutics,Darcylis,2020,6,Gene Therapy,3,2,Quality Control,0\.675,29\.2,296020,69\.6,778\.7,1595\.27,ImpactTherapeutics,Darcylis,2020,7,Small Molecule,4,3,Purification,0\.841,22\.8,306700,123\.3,772\.8,1720\.28,ImpactTherapeutics,Darcylis,2020,8,Vaccine,4,3,Purification,0\.936,32,174200,87\.4,215\.7,228\.19,ImpactTherapeutics,Darcylis,2020,9,Gene Therapy,11,9,Quality Control,0\.552,38\.9,368086,156\.6,639\.9,333\.310,ImpactTherapeutics,Darcylis,2020,10,Small Molecule,9,6,Fill\-Finish,0\.89,26\.8,92935,99\.3,568\.2,1398\.711,ImpactTherapeutics,Darcylis,2020,11,Small Molecule,10,9,Fermentation,0\.703,44\.1,432011,155\.5,265\.2,660\.912,ImpactTherapeutics,Darcylis,2020,12,Vaccine,11,9,Purification,0\.726,18\.7,453094,116\.1,494\.3,1451\.913,ImpactTherapeutics,Darcylis,2021,1,Vaccine,7,6,Fill\-Finish,0\.852,19\.6,84641,86\.4,212\.9,1873\.514,ImpactTherapeutics,Darcylis,2021,2,Biologic,11,10,Quality Control,0\.637,25\.7,447476,91\.9,185\.5,841\.315,ImpactTherapeutics,Darcylis,2021,3,Biologic,3,2,Purification,0\.55,23\.6,187152,66\.3,473\.9,1072\.716,ImpactTherapeutics,Darcylis,2021,4,Vaccine,3,2,Quality Control,0\.927,22\.7,283456,152\.5,354\.5,1949\.217,ImpactTherapeutics,Darcylis,2021,5,Gene Therapy,6,4,Purification,0\.835,16\.9,498983,82\.7,783\.6,939\.918,ImpactTherapeutics,Darcylis,2021,6,Biologic,9,7,Fermentation,0\.646,16\.8,270254,197\.7,269\.4,1409\.819,ImpactTherapeutics,Darcylis,2021,7,V…\[truncated 160239 chars\]

grader criteria0 of 2 passed

grader criterion · FAILName: States the Impact site with the highest absolute z score is LorexaEvidence: In the agent’s ‘TEXT\_RESPONSE‘, it states: "\*\*SiteID:\*\* Darcylis" and "\*\*z score:\*\* \*\*1\.29\*\*"\. The only file artifact provided, ARTIFACT 1, is a deleted sheet containing ImpactTherapeutics data for sites such as Darcylis, Lorexa, Strevalent, and Noralix, but no artifact states that Lorexa is the highest\-absolute\-z\-score Impact site\.Assessment: The criterion specifically asks to "States the Impact site with the highest absolute z score is Lorexa\." This is not met because the agent explicitly named Darcylis instead of Lorexa in its response\. Therefore, the criterion fails\.

grader criterion · FAILName: States that the z score of Lorexa is \-1\.60Evidence: In the agent’s final text response, it states: "\*\*SiteID:\*\* Darcylis" and "\*\*z score:\*\* \*\*1\.29\*\*," which does not mention Lorexa or \-1\.60\. The only artifact provided, ‘Manufacturing\_Site\_Operations\_2020\_2024\.xlsx‘ \(ARTIFACT 1\), contains raw site rows including ‘ImpactTherapeutics,Lorexa‘ in the deleted content, but no z\-score calculation or statement for Lorexa\.Assessment: The criterion specifically asks to "state that the z score of Lorexa is \-1\.60\." This was not met: the response gives a different site \(Darcylis\) and a different z score \(1\.29\), and the artifact does not provide a Lorexa z score of \-1\.60\. Therefore, the criterion is false\.

GHCP \+ Memconfig: GHCP \+ Memstatus: success \| criteria: 2/2 \| primary reward: 0\.89 \| fractional reward: 0\.89 \| tool calls: 11cost: $0\.3756 \| in: 294\.3K \| out: 4\.9K \| cached: 267\.9K

MEMORY

agent · memory\_read\{"query": "Calculate z score of US 2024 Average Monthly Revenue per Head for Impact sites from attached operational data file, using yearly revenue allocated equally across sites and months; need highest absolute z\-score Impact site and SiteID\.", "k": 10\}

memory · complete logical payloadmemory\_read: showing 11/21 matched entries \(query asked for top\-25; index holds 21 total\)\.\#1 \[schema\]applies\_to: benchmark spreadsheet tasks combining Manufacturing\_Site\_Operations\_2020\_2024\.xlsx with company PnL US sheets for 2024 site revenue\-per\-head comparisonslemma: When a prompt allocates yearly US revenue equally across 2024 sites and months, compute company monthly site revenue as annual US Total Revenue÷\\div2024 site count÷\\div12, divide each site\-month by HeadcountFTE to get revenue per head, then average those monthly ratios by site before comparing sites\.\#2 \[rule\]applies\_to: Manufacturing\_Site\_Operations\_2020\_2024\.xlsx \+ PnL what\-if analyses that ask Impact’s lowest\-revenue\-per\-head site to use the highest\-revenue site’s utilization across Impact and six peerslemma: Treat the benchmark as the highest\-revenue\-per\-head site across Impact plus all six peers \(not Impact\-only\), use that site’s average 2024 EquipmentUtilization rounded to two decimals, set each target month’s utilization to max\(current, benchmark\), and scale revenue by new\_utilization÷\\divcurrent\_utilization\.\#3 \[rule\]applies\_to: Rebasing 2024 Impact cost\-sa…\[truncated 3800 chars\]

SELECTED AGENT CALLS

agent · archipelago\-filesystem\_server\_list\_files · representative discoveryRecorded top\-level call 1 of 11 \| 0 before \| 10 after\{"path": "/"\}

env · observation · representative discovery’4\. Received From Client’ \(folder\)’2\. Analysis’ \(folder\)’Manufacturing\_Site\_Operations\_2020\_2024\.xlsx’ \(application/vnd\.openxmlformats\-officedocument\.spreadsheetml\.sheet file\) 156159 bytes’1\. Presentations’ \(folder\)’3\. Research’ \(folder\)’0\. Admin’ \(folder\)

agent · archipelago\-sheets\_server\_sheets · representative discoveryRecorded top\-level call 2 of 11 \| 1 before \| 9 after\{"request": \{"action": "list\_tabs", "file\_path": "/Manufacturing\_Site\_Operations\_2020\_2024\.xlsx"\}\}

env · observation · representative discovery\{"action": "list\_tabs","error": null,"help": null,"read\_tab": null,"read\_csv": null,"create": null,"delete": null,"list\_tabs": \{"raw\_output": "\{’worksheets’: \[\{’name’: ’Manufacturing\_Site\_Operations\_2’, ’index’: 0, ’row\_count’: 1681, ’column\_count’: 15\}\]\}"\},"add\_tab": null,"delete\_tab": null,"edit": null,"add\_content": null,"delete\_content": null,"create\_chart": null,"filter\_tab": null\}

agent · archipelago\-code\_execution\_server\_code\_exec · decisive computationRecorded top\-level call 11 of 11 \| 10 before \| 0 after\{"request": \{"code": "python \- <<’PY’\\nimport pandas as pd\\nimport openpyxl as ox\\nops = pd\.read\_excel\(’/filesystem/Manufacturing\_Site\_Operations\_2020\_2024\.xlsx’\)\\nops = ops\[ops\.Year==2024\]\.copy\(\)\\nrev = \{\\n’ImpactTherapeutics’:8324082\.0,’BayGenea’:27654678\.0,’Fyrix’:32490052\.0,’HelioZenon’:26234208\.0,’Abvuera’:30440958\.0,’Meridon’:31740145\.0,’Renueva’:7584932\.0\}\\nsite\_counts = ops\.groupby\(’Company’\)\[’SiteID’\]\.nunique\(\)\.to\_dict\(\)\\nops\[’annual\_site\_rev’\] = ops\[’Company’\]\.map\(\{k:v/site\_counts\[k\] for k,v in rev\.items\(\)\}\)\\n\# Method A average monthly ratio\\nops\[’m\_site\_rev’\] = ops\[’annual\_site\_rev’…\[truncated 989 chars\]

env · observation · decisive computation\{"success":true,"output":"\\n A\\n Company SiteID metricA z abs\\nImpactTherapeutics Lorexa 5000\.669739 \-1\.598424 1\.598424\\nImpactTherapeutics Noralix 5475\.895103 \-1\.544280 1\.544280\\nImpactTherapeutics Papinex 5823\.362205 \-1\.504692 1\.504692\\nImpactTherapeutics Strevalent 5954\.572667 \-1\.489743 1\.489743\\nImpactTherapeutics Darcylis 6198\.583404 \-1\.461942 1\.461942\\n\\n B\\n Company SiteID avg\_h annual\_site\_rev metricB z abs\\nI\[Observation capped at 500 source characters; original length: 1167 source characters\.\]

OTHER RECORDED CALLS BY BUCKET

other recorded callsOther recorded calls \(not expanded\): filesystem x1; spreadsheet x3; code execution x4\. Accounting: 11 recorded; 3 shown; 8 not expanded\.

FINAL ANSWER AND GRADING

submitted final answer<TEXT\_RESPONSE\>SiteID: \*\*Lorexa\*\*z\-score: \*\*\-1\.60\*\*Calculated using ‘/Manufacturing\_Site\_Operations\_2020\_2024\.xlsx‘ and the 2024 US ‘Total Revenue‘ from the following P&L workbooks: \[‘/4\. Received From Client/Impact Therapeutics/ImpactTherapeutics\_PnL\_Vertical\_2020\_2024\.xlsx‘\]\(\#\), \[‘/4\. Received From Client/Competition/Abvuera\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx‘\]\(\#\), \[‘/4\. Received From Client/Competition/BayGenea\_PnL\_Vertical\_2020\_2024\.xlsx‘\]\(\#\), \[‘/4\. Received From Client/Competition/Fyrix\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx‘\]\(\#\), \[‘/4\. Received From Client/Competition/HelioZenon\_PnL\_Vertical\_2020\_2024\.xlsx‘\]\(\#\), \[‘/4\. Received From Client/Competition/Meridon\_PnL\_Vertical\_2020\_2024\.xlsx‘\]\(\#\), \[‘/4\. Received From Client/Competition/Renueva\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx‘\]\(\#\)\. <DONE\></TEXT\_RESPONSE\>

grader criteria2 of 2 passed

grader criterion · PASSName: States the Impact site with the highest absolute z score is LorexaEvidence: In the agent’s TEXT\_RESPONSE, it says: "SiteID: \*\*Lorexa\*\*" and gives the corresponding z\-score as "\*\*\-1\.60\*\*"\.Assessment: The criterion asks whether the response states that the Impact site with the highest absolute z score is Lorexa\. The response identifies the SiteID as Lorexa, so this criterion is met\.

grader criterion · PASSName: States that the z score of Lorexa is \-1\.60Evidence: The agent’s final text response states: ‘SiteID: \*\*Lorexa\*\*‘ and ‘z\-score: \*\*\-1\.60\*\*‘\.Assessment: The criterion asks to state that the z score of Lorexa is \-1\.60\. The response explicitly gives Lorexa with a z\-score of \-1\.60, so the criterion is met\.

GHCP \+ Mem \(w/ Env Probing\)config: GHCP \+ Mem \(w/ Env Probing\)status: success \| criteria: 2/2 \| primary reward: 0\.94 \| fractional reward: 0\.94 \| tool calls: 6cost: $0\.1688 \| in: 145\.6K \| out: 6\.5K \| cached: 130\.0K

MEMORY

agent · memory\_read\{"query": "Find attached file with monthly US operational data for Impact and competitor sites, compute 2024 average monthly revenue per head by site, then z score for Impact sites using sample standard deviation across all sites\.", "k": 10\}

memory · complete logical payloadmemory\_read: showing 8/15 matched entries \(query asked for top\-25; index holds 15 total\)\.\#1 \[schema\]applies\_to: Impact peer revenue\-per\-batch benchmarkinglemma: For Impact\-vs\-peer site revenue\-per\-batch benchmarks, combine 2024 site\-month rows from Manufacturing\_Site\_Operations\_2020\_2024\.xlsx with each company’s annual US ’Total Revenue’ from the ’US’ tab of its \*\_PnL\_Vertical workbook, allocate annual revenue evenly across that company’s sites and 12 months, compute monthly revenue/BatchesPassed for each site\-month, then average by site before taking min/max\.\#2 \[rule\]applies\_to: Impact peer revenue\-per\-head utilization what\-ifslemma: For Impact site revenue\-per\-head utilization scenarios, use 2024 ‘Manufacturing\_Site\_Operations\_2020\_2024\.xlsx‘ rows plus each company PnL ‘US‘→\\rightarrow‘Total Revenue‘; allocate annual US revenue evenly across that company’s sites and 12 months, compute monthly revenue/‘HeadcountFTE‘, average by site to identify the min/max sites across the pooled Impact\+six\-peer set unless the prompt narrows scope, then apply the benchmark site’s average ‘EquipmentUtilization‘ \(rounded only to the requested precision\) by…\[truncated 3800 chars\]

SELECTED AGENT CALLS

agent · archipelago\-filesystem\_server\_search\_files · representative discoveryRecorded top\-level call 1 of 6 \| 0 before \| 5 after\{"pattern": "\*\_PnL\_Vertical\_2020\_2024\*\.xlsx", "path": "/", "recursive": true, "max\_results": 50\}

env · observation · representative discoveryFound 7 file\(s\) matching ’\*\_PnL\_Vertical\_2020\_2024\*\.xlsx’:/4\. Received From Client/Competition/BayGenea\_PnL\_Vertical\_2020\_2024\.xlsx/4\. Received From Client/Competition/Fyrix\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx/4\. Received From Client/Competition/HelioZenon\_PnL\_Vertical\_2020\_2024\.xlsx/4\. Received From Client/Competition/Abvuera\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx/4\. Received From Client/Competition/Meridon\_PnL\_Vertical\_2020\_2024\.xlsx/4\. Received From Client/Competition/Renueva\_PnL\_Verti\[Observation capped at 500 source characters; original length: 620 source characters\.\]

agent · archipelago\-filesystem\_server\_search\_files · representative discoveryRecorded top\-level call 2 of 6 \| 1 before \| 4 after\{"pattern": "Manufacturing\_Site\_Operations\_2020\_2024\.xlsx", "path": "/", "recursive": true, "max\_results": 20\}

env · observation · representative discoveryFound 1 file\(s\) matching ’Manufacturing\_Site\_Operations\_2020\_2024\.xlsx’:/Manufacturing\_Site\_Operations\_2020\_2024\.xlsx

agent · archipelago\-code\_execution\_server\_code\_exec · decisive computationRecorded top\-level call 6 of 6 \| 5 before \| 0 after\{"request": \{"code": "python \- <<’PY’\\nimport pandas as pd, statistics\\nfrom pathlib import Path\\nops = pd\.read\_excel\(’/filesystem/Manufacturing\_Site\_Operations\_2020\_2024\.xlsx’\)\\nops = ops\[ops\.Year==2024\]\.copy\(\)\\ncompanies = ops\.Company\.unique\(\)\\nrevs=\{\}\\nfor company in companies:\\n p = list\(Path\(’/filesystem’\)\.rglob\(f’\{company\}\*\_PnL\_Vertical\_2020\_2024\*\.xlsx’\)\)\[0\]\\n us = pd\.read\_excel\(p, sheet\_name=’US’\)\\n revs\[company\] = float\(us\.loc\[us\[’Line Item’\]\.eq\(’Total Revenue’\), 2024\]\.iloc\[0\]\)\\nsite\_counts = ops\.groupby\(’Company’\)\[’SiteID’\]\.nunique\(\)\.to\_dict\(\)\\nops\[’allocated’\] = ops\.Company\.…\[truncated 851 chars\]

env · observation · decisive computation\{"success": true,"output": "m1 SiteID metric1 z abs\_z\\n Lorexa 5000\.669739 \-1\.598424 1\.598424\\n Noralix 5475\.895103 \-1\.544280 1\.544280\\n Papinex 5823\.362205 \-1\.504692 1\.504692\\nStrevalent 5954\.572667 \-1\.489743 1\.489743\\n Darcylis 6198\.583404 \-1\.461942 1\.461942\\nm2 SiteID metric2 z abs\_z\\n Lorexa 4489\.796117 \-1\.613574 1\.613574\\n Darcylis 5017\.529837 \-1\.545447 1\.545447\\n Noralix 5130\.404931 \-1\.530876 1\.530876\\n Papinex 5196\.056180 \-1\.5224\[Observation capped at 500 source characters; original length: 559 source characters\.\]

OTHER RECORDED CALLS BY BUCKET

other recorded callsOther recorded calls \(not expanded\): code execution x3\. Accounting: 6 recorded; 3 shown; 3 not expanded\.

FINAL ANSWER AND GRADING

submitted final answer<TEXT\_RESPONSE\>\- SiteID: ‘Lorexa‘\- z score: ‘\-1\.60‘ \[^1\]\[^2\]\[^1\]: ‘/Manufacturing\_Site\_Operations\_2020\_2024\.xlsx‘\[^2\]: ‘US‘ tabs in ‘/4\. Received From Client/Impact Therapeutics/ImpactTherapeutics\_PnL\_Vertical\_2020\_2024\.xlsx‘, ‘/4\. Received From Client/Competition/Abvuera\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx‘, ‘/4\. Received From Client/Competition/BayGenea\_PnL\_Vertical\_2020\_2024\.xlsx‘, ‘/4\. Received From Client/Competition/Fyrix\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx‘, ‘/4\. Received From Client/Competition/HelioZenon\_PnL\_Vertical\_2020\_2024\.xlsx‘, ‘/4\. Received From Client/Competition/Meridon\_PnL\_Vertical\_2020\_2024\.xlsx‘, ‘/4\. Received From Client/Competition/Renueva\_PnL\_Vertical\_2020\_2024\_Formatted\.xlsx‘<DONE\></TEXT\_RESPONSE\>

grader criteria2 of 2 passed

grader criterion · PASSName: States the Impact site with the highest absolute z score is LorexaEvidence: In the agent’s final text response, it states: "SiteID: ‘Lorexa‘" followed by "z score: ‘\-1\.60‘"\. This directly names Lorexa as the site being reported\.Assessment: The criterion requires that the response states the Impact site with the highest absolute z score is Lorexa\. The agent’s response identifies Lorexa as the SiteID in its answer, which satisfies that requirement\. Therefore, the criterion is met\.

grader criterion · PASSName: States that the z score of Lorexa is \-1\.60Evidence: In the agent’s final text response, it states: "\- SiteID: ‘Lorexa‘" and "\- z score: ‘\-1\.60‘"\. These are the exact site name and value referenced by the criterion\.Assessment: The criterion asks to "State that the z score of Lorexa is \-1\.60\." The agent explicitly states both the site "Lorexa" and the z score "\-1\.60," so the criterion is met\.