Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline

# Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline
Source: [https://arxiv.org/html/2606.04315](https://arxiv.org/html/2606.04315)
Zhikai Chen1Jialiang Gu211footnotemark:1Junyu Yin211footnotemark:1Xianxuan Long1Shenglai Zeng1 Xiaoze Liu3Kai Guo1Keren Zhou2Jiliang Tang1

1Michigan State University2George Mason University3Purdue University

###### Abstract

LLM agents accumulate histories that outgrow their context windows, motivating a growing literature on memory systems\. Yet most existing designs are tuned to a single scenario \(multi\-session chat or a single trajectory format\), and there is little evidence that they generalize across the heterogeneous trajectories agents encounter in deployment\. We revisit eight memory systems plus an agentic harness for search problems, on five scenarios: single\-turn QA, multi\-session chat, agentic\-trajectory QA, memory stress tests, and long\-horizon agentic tasks\. The harness, which self\-manages flat text\-file storage via tool calls, achieves the best cross\-task ranking, suggesting that memory performance hinges on giving the agent active control over storage and retrieval rather than on a passive store behind a fixed pipeline\. We instantiate this insight inAutoMEM, an agentic memory harness with a self\-managed tool interface that achieves the best cross\-scenario generality among the systems we evaluate\.

Exploring Cross\-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline

Zhikai Chen1††thanks:Equal contribution\.Jialiang Gu211footnotemark:1Junyu Yin211footnotemark:1Xianxuan Long1Shenglai Zeng1Xiaoze Liu3Kai Guo1Keren Zhou2Jiliang Tang11Michigan State University2George Mason University3Purdue University

## 1Introduction

Large language models \(LLMs\) have evolved from single\-turn chatbots\(Ouyang et al\.,[2022](https://arxiv.org/html/2606.04315#bib.bib25)\)into general\-purpose agents that plan, invoke tools, and execute long\-horizon workflows\(Yao et al\.,[2022](https://arxiv.org/html/2606.04315#bib.bib49); Schick et al\.,[2023](https://arxiv.org/html/2606.04315#bib.bib31); Wang et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib36)\)\. Across these regimes, models accumulate historical information: knowledge corpora to ground responses\(Lewis et al\.,[2020](https://arxiv.org/html/2606.04315#bib.bib18)\), codebases to navigate\(Jiménez et al\.,[2023](https://arxiv.org/html/2606.04315#bib.bib14); Heule et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib9)\), multi\-step workflows to plan\(Xie et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib41)\), and user state to maintain\(Salemi et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib30)\)\.Memory, the mechanism that selects and organizes past information, is what lets an LLM act on more than the immediately available prompt\.

Many memory systems have been proposed\(Hu et al\.,[2026b](https://arxiv.org/html/2606.04315#bib.bib13)\), yet whether the added complexity helps is increasingly contested\(Pollertlam and Kornsuwannawit,[2026](https://arxiv.org/html/2606.04315#bib.bib28); He et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib8)\)\. The central problem is thatmost existing memory systems are scenario\-narrow: each is tuned around a single regime, typically multi\-session chat\(Wu et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib39); Maharana et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib24)\)or one agentic\-trajectory format, and the characteristics those regimes reward differ sharply\(Zhao et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib54)\)\. Winning on one does not imply winning on others, and little evidence shows which designs generalize\.

We therefore revisit existing designs through across\-scenario generalitylens: a practical memory system must work across the heterogeneous trajectories agents encounter in deployment, such as multi\-session chat, code interactions, and browser logs\. We also track token cost and latency to ensure generality is not bought at impractical cost\(Yang et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib47)\)\.

We evaluate eight representative memory systems plus anagent harness\(Li et al\.,[2026c](https://arxiv.org/html/2606.04315#bib.bib21)\)designed for search across five task families: single\-turn QA, multi\-session QA, agentic\-trajectory QA, memory stress tests, and long\-horizon agentic tasks\. These systems span long\-context, note\-based, multi\-store hierarchical, graph\-based, and RL\-trained designs\. The main observation is that even this off\-the\-shelf harness achieves the best cross\-scenario generality: structured memories remain useful as tools, but on their own are too scenario\-narrow to span real agentic trajectories\. Spanning that diversity requires anagentic memory harnessthat self\-manages memory through tool calls\. We instantiate this inAutoMEM, which achieves the best cross\-scenario generality across the systems we evaluate\. Our key contributions:

1. \(1\)A cross\-scenario evaluation of eight memory systems plus an agent harness across five task families, with token cost and latency tracked alongside accuracy\.
2. \(2\)Empirical finding: existing memory systems struggle on agentic trajectories through two failure modes, arepresentation\-levelfailure where build\-time schemas drop step\- and action\-level evidence, and aretrieval\-levelfailure where passive retrieval cannot surface evidence the storage retains; an agent harness that defers retrieval to query time achieves the best generality\.
3. \(3\)AutoMEM, an agent harness with a memory tool\-calling interface, achieving49\.6%49\.6\\%improvement over the original agent harness on LoCoMo and24\.4%24\.4\\%in overall ranking\.

## 2Related Work

### 2\.1Memory Design

Memory designs for LLM agents pursue two complementary goals: improvingquality\(accuracy, capability, generality\) and improvingefficiency\(token cost, build cost, latency\)\.

#### Quality\.

Memory design reduces to two coupled questions: how to store and update past information, and how to retrieve it\. On the storage side, the central choice is granularity\. The baseline stores raw chunked passages and retrieves them by similarity search\(Lewis et al\.,[2020](https://arxiv.org/html/2606.04315#bib.bib18); Karpukhin et al\.,[2020](https://arxiv.org/html/2606.04315#bib.bib16)\), on top of which three richer designs target different access patterns: atomic notes\(Chhablani et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib3); Zhong et al\.,[2023](https://arxiv.org/html/2606.04315#bib.bib55); Xu et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib43)\)make each fact an addressable, editable record, fitting evolving stable\-fact workloads; OS\-style tiered memory\(Packer et al\.,[2023](https://arxiv.org/html/2606.04315#bib.bib26); Qian et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib29); Liu et al\.,[2026b](https://arxiv.org/html/2606.04315#bib.bib23)\)pages information through tool calls between bounded working and long\-term tiers, fitting multi\-session agents; and graph\-based storage\(Edge et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib4); Gutiérrez et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib7)\)makes entity\-relation edges first\-class, fitting multi\-hop composition\. On agentic tasks, however, single\-granularity stores prove insufficient, and hierarchical or mixed\-granularity stores\(Yang et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib47); Yue et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib51); Shu et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib34); Zhao et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib54)\)are adopted at the cost of higher build overhead and token cost\. Retrieval typically mirrors the storage form: graph traversal for graphs, semantic search for notes and passages, and tool\-call paging for tiers\. Beyond this default, two directions stand out: training an LLM tool\-policy with RL to manage retrieval\(Yue et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib51); Zhang et al\.,[2026b](https://arxiv.org/html/2606.04315#bib.bib53); Yan et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib45); Zhou et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib56); Wang et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib38); Yu et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib50)\)shifts memory work to post\-training time at the cost of training compute, while multi\-stage retrieval, the retrieval\-side analog of mixed\-granularity storage, ensembles multiple retrieval paths \(e\.g\., graph traversal plus semantic search\) and aggregates their candidates to recover what any single path misses, at the cost of more retrieval calls per query\(Yang et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib47); Zhao et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib54)\)\.

#### Efficiency\.

Cost\-focused work groups into three patterns\.Compressionreduces content via gating, coreference resolution, and summarization\(Liu et al\.,[2026a](https://arxiv.org/html/2606.04315#bib.bib22); Xu et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib44); Li et al\.,[2026a](https://arxiv.org/html/2606.04315#bib.bib19)\)\.Offline delegationshifts work to build time so retrieval becomes non\-LLM\(Tian et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib35); Gutiérrez et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib7); Yang et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib47)\)\.Cost\-tier routingpicks the cheapest module meeting a quality threshold\(Zhang et al\.,[2026a](https://arxiv.org/html/2606.04315#bib.bib52)\)\. All three are validated mainly on conversational QA, so transfer to agentic workloads is untested\. We cover the first and leave the others as future work\.

### 2\.2Memory Benchmark

Memory benchmarks fall into three categories\.QA benchmarkstest memory through QA over conversation history or long documents\(Maharana et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib24); Li et al\.,[2026b](https://arxiv.org/html/2606.04315#bib.bib20); Wu et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib39); Lee et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib17); Hu et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib12); Ai et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib1); Pang et al\.,[2021](https://arxiv.org/html/2606.04315#bib.bib27); Bai et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib2); Hsieh et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib10)\)\.Agentic QA benchmarksask questions about an agent’s trajectory: AMA\-Bench\(Zhao et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib54)\)uses machine\-generated trajectories with causal dependencies; SAGE\(Hu et al\.,[2026a](https://arxiv.org/html/2606.04315#bib.bib11)\)targets retrieval in deep research agents\.Real agentic task benchmarkscouple memory to actions so memory quality affects task completion; MemoryArena\(He et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib8)\)runs memory\-agent\-environment loops\. Existing benchmarks typically target a single category and rarely report token cost or latency, leaving no principled comparison along the cross\-task, cost, and latency axes\.

## 3Evaluation Design

We measure each memory system on three axes: cross\-scenario generality, token cost, and latency\.

#### Tasks\.

Table[1](https://arxiv.org/html/2606.04315#S3.T1)lists the five scenarios, which span the main deployment regimes for memory in agentic systems\.

Table 1:The five task scenarios\.
#### Memory baselines\.

We evaluate eight memory systems plus an agent harness baseline\(Li et al\.,[2026c](https://arxiv.org/html/2606.04315#bib.bib21)\)originally designed for search \(Table[2](https://arxiv.org/html/2606.04315#S3.T2)\), covering the architectural design space of §[2\.1](https://arxiv.org/html/2606.04315#S2.SS1)\. For agentic tasks we use a ReAct agent\(Yao et al\.,[2022](https://arxiv.org/html/2606.04315#bib.bib49)\)with task\-tuned prompts\.

Table 2:Memory baselines grouped by architectural category\.
#### Metrics\.

For LoCoMo, HotpotQA, AMABench, and MemoryAgentBench we report a Qwen3\-32B LLM\-judge score\(Zhao et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib54)\), since token F1 mis\-scores long\-form answers on verbose benchmarks \(Appendix[B\.1](https://arxiv.org/html/2606.04315#A2.SS1)\)\. For agentic tasks we report environment outcomes: success rate on ALFWorld; process score on MA\-Shop \(fraction of subtasks solved\); and c\-sPS on MA\-Travel, a corrected MemoryArena sPS\(He et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib8)\)that only scores slots the user asks about in the current session\.

#### Backbones\.

The default backbone is Qwen3\-32B\(Yang et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib46)\)for both construction and answering, with Qwen2\.5\-7B\-Instruct on ALFWorld \(Qwen3\-32B solves most episodes trivially\)\. The embedding model is Qwen3\-Embedding\-4B throughout\. A Qwen3\-4B\-Instruct ablation is in Appendix[D](https://arxiv.org/html/2606.04315#A4)\.

## 4Experimental Results

### 4\.1Overview

We report performance across benchmarks in Table[3](https://arxiv.org/html/2606.04315#S4.T3)and per\-method preprocessing vs\. inference cost in Fig\.[1](https://arxiv.org/html/2606.04315#S4.F1)\. The following high\-level patterns emerge:

1. \(1\)No method dominates\.Every index\-based method \(one that pre\-builds a structured store such as a graph, summary notes, or multi\-store cache\) lags long context on at least one benchmark; DCI\-Lite holds the best generality rank\. The shortfall of index\-based methods is most visible on agentic\-trajectory QA \(§[4\.2](https://arxiv.org/html/2606.04315#S4.SS2)\)\.
2. \(2\)Long context is stronger than commonly assumed\(Hu et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib12); Zhao et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib54)\)and remains cost\-competitive: heavy\-index methods such as HippoRAG cannot recover preprocessing cost unless many future queries hit the same store, while lighter alternatives such as LightMem keep the build cheap but score lower than long context\. Neither Pareto\-dominates the baseline\.
3. \(3\)On dynamic agentic tasks, most methods convergewithin sampling variance\.
4. \(4\)Token efficiency≠\\neqsystem efficiency\.HippoRAG and AMA\-Agent issue tokens as long chains of small serial LLM calls with KV\-cache misses, inflating wall\-clock and GPU contention far beyond what raw token counts suggest\. This reflects a memory system’s infrastructure affinity, which token counts alone do not capture; per\-method numbers are in Appendix[E](https://arxiv.org/html/2606.04315#A5)\.

The rest of §[4](https://arxiv.org/html/2606.04315#S4)follows one thread: when to commit memory structure at build time versus query time\. §[4\.2](https://arxiv.org/html/2606.04315#S4.SS2)diagnoses why early commitment fails on agentic QA; §[4\.3](https://arxiv.org/html/2606.04315#S4.SS3)shows the agent harness wins by deferring that commitment to query time; §[4\.4](https://arxiv.org/html/2606.04315#S4.SS4)characterizes when the build\-time cost still pays off; §[4\.5](https://arxiv.org/html/2606.04315#S4.SS5)closes with the scope limit: on dynamic agentic tasks, no memory design closes the gap\.

Table 3:Main results: per\-method accuracy across all benchmarks; best per column inbold\. A\-ALF/A\-Web/A\-SQL are AMABench domains; HQA is HotpotQA; AR/TTL/LRU/CR are MemoryAgentBench\(Hu et al\.,[2025](https://arxiv.org/html/2606.04315#bib.bib12)\)subtasks; ALFW is ALFWorld; MA\-S/MA\-T are MemoryArena shopping/travel\. The Gen\. column is the mean fractional rank across benchmarks, with MemoryAgentBench counted once by averaging AR/TTL/LRU/CR\. The Variance row reports the sampling\-noise floor in percentage points\.![Refer to caption](https://arxiv.org/html/2606.04315v1/x1.png)Figure 1:Per\-method preprocessing vs\. inference cost per question, averaged across sub\-benchmarks within each task category\.
### 4\.2Where index\-based memory fails on agentic QA

Raw trajectoryscrollStep 5scrollStep 6go\_backStep 7HippoRAG:black case$11\.99has\_pricestep/action: lostAMA\-Agent:e@5e@6e@7scrollscrollgo\_backstep/action: keptFigure 2:Schema loss on an AMABench\-Web question \(“which step performed the return from the product page, and how many scrolls happened before it?”\)\. HippoRAG keeps entity\-relation facts but discards step indices and actions; AMA\-Agent’s turn\-indexed graph keeps both\.Indexing\-based methods perform poorly on agentic QA \(Table[3](https://arxiv.org/html/2606.04315#S4.T3), AMABench columns\), with several scoring less than half the long\-context baseline despite building richer structures\. To localize the failure, we ablate the pipeline and ask whether the answer is still recoverable before the normal retrieval interface is applied\. HippoRAG2 extracts an entity\-fact KG; AMA\-Agent attaches per\-turn state reports to a turn\-indexed causality graph with action\-labeled edges; PlugMem keeps three parallel stores \(episodic raw trace, semantic entity facts, procedural subgoal summaries\) behind a mode\-routing retriever\. We sample 60 questions across the three AMABench domains and use Codex with gpt\-5\.5 \(extra high reasoning\) to check this oracle storage answerability\. Methods designed for personal\-chat or Wikipedia\-style assistants hit a hard representational ceiling \(Table[4](https://arxiv.org/html/2606.04315#S4.T4)\); AMA\-Agent and PlugMem show a different pattern: the answer is often still present somewhere in storage, but the deployed retrieval interface fails to surface it reliably\.

Table 4:Oracle storage answerability \(%\)\. The probe asks whether the answer is still recoverable from the method’s stored representation, or from the raw episodic trace when the method keeps one; it is therefore an upper bound on what the retrieval interface can deliver, not an end\-to\-end retrieval score\. Question types are AMABench’s: A state\-value lookup, B causal inference, C causal\-chain, D constraint comparison\.A deeper investigation reveals two failure modes\. The first is arepresentation\-levelfailure \(Figure[2](https://arxiv.org/html/2606.04315#S4.F2)\): HippoRAG’s step indices and action labels never enter the entity\-relation KG because they are not entity\-relation facts, so the data is gone before retrieval and no answerer can recover it; the only fix is to widen the schema at build time\.

The second is aretrieval\-levelfailure: storage isn’t where the answer is lost\. Table[4](https://arxiv.org/html/2606.04315#S4.T4)shows that PlugMem keeps over98%98\\%of the answer information \(its episodic store holds the dense raw trace\), so an oracle reading the whole memory could answer almost any question\. What fails ispassive retrieval: a router picks one branch by question\-embedding similarity and returns a compact note; when the answer lives in step\-level detail, the note never carries, the raw trace stays in storage, and the answerer has no second chance to ask for it\. An active agent\-driven retrieval interface would close this gap: when the returned evidence is not enough, the agent can issue a follow\-up call and fetch what is missing\.

A further issue isinflexibility: index\-based memory commits to a schema at build time, so the designer must anticipate which relation types future questions will probe\. To accommodate agentic trajectories, AMA\-Agent and PlugMem must add more complexity to the graph and include more node and relation types, which is unlikely to scale as agentic scenarios grow more diverse\. Letting an agent actively write the query partially addresses this; the intrinsic problems of index\-based memory for agentic use cases are left for future study\.

### 4\.3When the agent harness wins

Both failures in §[4\.2](https://arxiv.org/html/2606.04315#S4.SS2)stem from committing memory structure at build time\. We now turn to the design that defers commitment to query time: DCI\-Lite, an agent harness that keeps evidence in raw text and lets the LLM compose a per\-question retrieval plan viagrepandread, holds the best generality rank in our suite\. Compared to index\-based methods that commit a schema at build time and pay for that choice with the inflexibility of §[4\.2](https://arxiv.org/html/2606.04315#S4.SS2), the harness inverts the binding order: the LLM sees the actual question first, then decides at runtime which slice of raw evidence to read and how to aggregate it, shifting the schema\-commitment cost from a one\-time frozen choice at build time to a per\-question choice at query time informed by the question itself\. Runtime overhead is comparable to LongCtx since both route the retrieved evidence through an answer LLM; what DCI\-Lite buys with its grep–read loop is generality across regimes, not raw token savings on any single bench\. The cost win is clearer against heavy\-index methods: HippoRAG pays a∼130\{\\sim\}130K\-per\-question amortized build on Long\-context QA versus DCI\-Lite’s near\-zero build \(Fig\.[1](https://arxiv.org/html/2606.04315#S4.F1)\)\.

Table[5](https://arxiv.org/html/2606.04315#S4.T5)reports three signals behind DCI’s lead, with one probe per signal while pinning the other two\.Predicate complexity:HippoRAG indexes Wikipedia entities by name, and each HotpotQA question hinges on two such entities\. When the question names both entities, the index lookup is trivial; when it refers to them indirectly, the index must first recover the entity names before it can do anything\. Structured indexing tends to win in the first case, DCI in the second\.Corpus–window regime:the harness becomes valuable when a single read of the corpus is impractical\.Schema expressiveness:the index has a hard representational ceiling that no amount of retrieval quality can lift\.

Table 5:Three controlled probes for DCI’s generality lead\.Δ\\Deltais DCI minus the named baseline\.
### 4\.4When indexing pays off

We now study when an index\-based method is still worth its build cost, along two axes\. From theefficiencyside, we ask when the per\-question token cost of an index drops below the no\-build baselines once enough queries amortize the build\. From theperformanceside, we ask which task properties make index\-based methods a good fit in the first place\. The combined picture is that an index pays off only when both axes are satisfied: the workload amortizes the build,andthe task structure rewards build\-time aggregation\.

#### Efficiency axis: the amortization effect\.

An index\-based method pays its build cost once and amortizes it across all questions that hit the same store\. As the number of shared questionsNNgrows, its per\-question cost drops; no\-build methods \(LongCtx, DCI\-Lite\) stay flat\. Fig\.[3](https://arxiv.org/html/2606.04315#S4.F3)makes this concrete on AMABench: LightMem crosses DCI\-Lite atN≈2N\{\\approx\}2, and even HippoRAG, despite its much larger build, catches up atN≈32N\{\\approx\}32\.

![Refer to caption](https://arxiv.org/html/2606.04315v1/x2.png)Figure 3:Per\-question token cost vs\. amortisationNNon AMABench\. LightMem crosses DCI\-Lite atN≈2N\{\\approx\}2; HippoRAG atN≈32N\{\\approx\}32\.
#### Performance axis: when does indexing help?

Being cheap is not the same as being useful: Table[3](https://arxiv.org/html/2606.04315#S4.T3)shows index\-based methods are dominated in accuracy on most benchmarks, even where the build cost is amortized away by enough queries\. We first characterize what an index does at storage time, then identify the workload property that makes that work valuable\. The index\-based methods in our suite come in two flavors\.Graph\-basedmethods like HippoRAG2 encodeexplicitrelations between stored units \(entity\-fact triples, turn\-indexed causal edges, or multi\-store mode tags\) at build time\.Semantic note\-basedmethods like SimpleMem compress multiple turns into LLM\-summarized notes and embed the notes, encoding cross\-unit contentimplicitlyin the summary text\. The two flavors look different at the storage layer, but they share one defining property: both pay a build\-time cost toaggregate information across raw units, so a downstream query can answer with a single retrieval rather than reconstruct the relation on the fly\. This shared property, not the specific mechanism, controls when indexing pays off\.

Aggregation only pays off when the workload demands it\. We measure that demand by thestructural rate: the fraction of questions whose answer cannot be recovered from any single retrieval unit \(passage, dialog turn, or trajectory step\) under flat retrieval \(grep, BM25, or dense top\-55\), and therefore requires combining content across units\. It upper\-bounds the room build\-time aggregation that has to help\.

Table 6:Structural rate predicts the head\-to\-head between indexed methods and the no\-index harness DCI\-Lite\. AccuracyΔ\\Deltain pp, averaged over methods in each flavor \(graph: HippoRAG, PlugMem, AMA\-Agent, Mem\-T; semantic: LightMem, SimpleMem\)\.
#### Observation and implication\.

Two corpus\-side properties jointly gate when indexing pays off, and they play distinct roles\.Schema expressiveness\(§[4\.3](https://arxiv.org/html/2606.04315#S4.SS3)\) gates whether indexingcanhelp at all: if the schema cannot encode the predicates the questions probe, no amount of structural demand translates into accuracy gains \(the AMABench\-ALF failure mode\)\.Structural rate\(this section\) gates whether indexing isneeded: even with a fitting schema, indexing only adds value when many questions require cross\-unit aggregation a flat retriever cannot do\. Both must hold for indexed methods to beat the no\-index harness\.

### 4\.5Scope: dynamic agentic tasks are policy\-limited

The preceding subsections assume static\-style QA, where the right memory design can improve accuracy\. On dynamic agentic tasks this assumption breaks: no memory choice in our suite closes the gap, regardless of when it commits\. On ALFWorld, no memory method consistently improves the long\-context baseline, while parameter\-level post\-training \(GRPO\(Shao et al\.,[2024](https://arxiv.org/html/2606.04315#bib.bib32)\)on Qwen2\.5\-7B\) does\. To localize where memory is blocked, we hand\-craft agolden procedural memoryby distilling the rules that win ALFWorld games from a small set of expert trajectories into a single static rule sheet injected via the system prompt, and compare four conditions \(Table[7](https://arxiv.org/html/2606.04315#S4.T7)\)\. The oracle closes only part of the gap to the trained actor\.

Table 7:ALFWorld success rate with Qwen2\.5\-7B as the actor backbone\.†Strongest of all memory methods on the ALFWorld column of Table[3](https://arxiv.org/html/2606.04315#S4.T3)\(SimpleMem / LightMem at 43\.3%\)\.The gap splits into two pieces: one that the golden memory can close, and one that it cannot\.

Gap 1: over\-entropic action posterior\.Without a rule in context, the actor wanders across too many candidate actions and runs out of the 50\-step budget \(all 32 failed trajectories hit the cap\)\. Golden memory fixes this by spelling out the recipe for each task family, and the lift concentrates on tasks whose solution can be written down as a short procedure\. Memory systems fail to deliver the same lift because reflections written from past trajectories capture episode\-specific events rather than reusable procedures\. An expert is needed to distill these memories from trajectories\.

Gap 2: state\-dependent decisions text rules cannot encode\.The remaining 17 of 32 failures persist even with the golden recipe in context\. Inspecting them, the correct next action depends on the current environment state \(e\.g\., which receptacles are currently open, what the agent is already holding\) rather than on a fixed recipe step\. A text rule cannot enumerate the right branch for every possible state, so the actor still picks the wrong action at these decision points\. The rule can also actively mislead: when its vocabulary surface\-matches the task description, the actor mechanically follows it and triggers unnecessary action sequences\. Closing this gap requires reshaping the actor’s behavior at exactly those state\-conditional decision points\.

These two gaps connect to two open lines on memory\-based agent self\-improvement\.Gap 1explains why an agent struggles to self\-summarize its own procedural memory: a weaker actor cannot reliably distill canonical recipes from its own trajectories, so the propositional content must be sourced from a higher\-capability LLM oracle and distilled into the actor \(as in SkillRL\(Xia et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib40)\)\) before it can produce the lift\.Gap 2delineates the capability ceiling of self\-evolving agents whose only update channel is non\-parametric memory: such agents cannot move past behavior already implicit in the actor’s prior\. Going beyond this ceiling requires parametric updates such as the RL\-based self\-evolution in OpenClaw\-RL\(Wang et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib37)\)\. Mem\-T and MemRL sit on the non\-parametric side: they post\-train a memory controller on top of an unchanged actor, so they remain bounded by Gap 2\.

## 5AutoMEM: A Design Inspired by Our Findings

The experiments in §[4](https://arxiv.org/html/2606.04315#S4)suggest that the agent harness performs well on most benchmarks and only loses ground when an answer must be pieced together from many records, where an index can close the gap if its schema matches the questions\. The natural design is to migrate the indexed memory component into the harness\.

We instantiate this as an agentic loop \(Algorithm[1](https://arxiv.org/html/2606.04315#alg1)\)\. At each step, the agent \(1\) picks one action from a four\-item menu:rgfor regex over raw records,readfor fetching a record by identifier,dumpfor sending a short corpus to the answer LLM, andindex\_queryfor a typed query against a pre\-built index\. \(2\) A planner LLM writes the call, an executor runs it, and the evidence is appended to a running trace\. \(3\) An LLM judge inspects the trace; on sufficiency, an answerer composes the final answer, on insufficiency, the judge returns a hint naming what is missing, and the planner is re\-invoked\. One LLM backbone serves all roles\. By default,index\_queryruns against a Cypher graph\(Francis et al\.,[2018](https://arxiv.org/html/2606.04315#bib.bib6)\)built from LightMem’s write\-time summary index with edgesSAID\_BY,OF\_TOPIC,FOLLOWS; a PlugMem\-graph variant uses PlugMem’s tag/semantic/episodic stores with edgesHAS\_TAG,MENTIONS,SHARES\_TAG\.

Algorithm 1AutoMEM: agentic memory\-access loop1:memory store

MM, question

qq, step budget

KmaxK\_\{\\max\}
2:action set

𝒜=\{rg,read,dump,index\_query\}\\mathcal\{A\}=\\\{\\texttt\{rg\},\\,\\texttt\{read\},\\,\\texttt\{dump\},\\,\\texttt\{index\\\_query\}\\\}
3://𝑡𝑟𝑎𝑐𝑒\\mathit\{trace\}: evidence so far\.ℎ𝑖𝑛𝑡\\mathit\{hint\}: judge feedback\.

4:

𝑡𝑟𝑎𝑐𝑒←\[\]\\mathit\{trace\}\\leftarrow\[\\,\];

ℎ𝑖𝑛𝑡←∅\\mathit\{hint\}\\leftarrow\\varnothing
5:for

k=1,…,Kmaxk=1,\\dots,K\_\{\\max\}do

6:// Plan: 1 LLM call, emits one action∈𝒜\\in\\mathcal\{A\}\.

7:

ak←Plan​\(q,𝑡𝑟𝑎𝑐𝑒,ℎ𝑖𝑛𝑡\)a\_\{k\}\\leftarrow\\textsc\{Plan\}\(q,\\,\\mathit\{trace\},\\,\\mathit\{hint\}\)
8:// Execute: tool call onMM, no LLM\.

9:

ok←Execute​\(M,ak\)o\_\{k\}\\leftarrow\\textsc\{Execute\}\(M,\\,a\_\{k\}\)
10:

𝑡𝑟𝑎𝑐𝑒←𝑡𝑟𝑎𝑐𝑒∥\(ak,ok\)\\mathit\{trace\}\\leftarrow\\mathit\{trace\}\\;\\\|\\;\(a\_\{k\},o\_\{k\}\)
11:// Judge: 1 LLM call\. Skipped whenKmax=1K\_\{\\max\}\{=\}1\(single\-pass\)\.

12:

v←Judge​\(q,𝑡𝑟𝑎𝑐𝑒\)v\\leftarrow\\textsc\{Judge\}\(q,\\,\\mathit\{trace\}\)
13:if

v\.𝑠𝑢𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑡v\.\\mathit\{sufficient\}thenbreak

14:else

ℎ𝑖𝑛𝑡←v\.𝑟𝑒𝑎𝑠𝑜𝑛\\mathit\{hint\}\\leftarrow v\.\\mathit\{reason\}
15:endif

16:endfor

17:// Answer: 1 LLM call composes the final answer\.

18:return

Answer​\(q,𝑡𝑟𝑎𝑐𝑒\)\\textsc\{Answer\}\(q,\\,\\mathit\{trace\}\)

#### Results\.

AutoMEM beats DCI\-Lite and long context on LoCoMo \(Table[8](https://arxiv.org/html/2606.04315#S5.T8)\); the LightMem\-graph default leads the PlugMem\-graph variant by1\.31\.3pp\. On LoCoMo, AutoMEM \(67\.367\.3\) improves over DCI\-Lite \(45\.045\.0\) by\+22\.3\+22\.3pp, over long context \(61\.561\.5\) by\+5\.8\+5\.8pp, and edges the previously strongest method Mem\-T \(66\.566\.5\) by\+0\.8\+0\.8pp\. AutoMEM also tops the suite on average rank \(3\.103\.10, vs\.4\.104\.10for DCI\-Lite once inserted into Table[3](https://arxiv.org/html/2606.04315#S4.T3)\); the largest single\-column lift sits on LoCoMo, the benchmark with the highest structural rate\.

Table 8:LoCoMo accuracy andaverage fractional rank across all twelve task columns of Table[3](https://arxiv.org/html/2606.04315#S4.T3)\. LoCoMo is one of the benchmarks where DCI\-Lite itself underperforms \(45\.0, below the long\-context baseline at 61\.5\)\. Each AutoMEM row’s rank is computed by inserting that variant as the new method into Table[3](https://arxiv.org/html/2606.04315#S4.T3)\.
#### General algorithm\.

Algorithm[1](https://arxiv.org/html/2606.04315#alg1)defines the canonical AutoMEM design: a multi\-step plan–execute–judge loop that can re\-query memory when the judge finds the trace insufficient\. This iterative form is what delivers the accuracy in Table[8](https://arxiv.org/html/2606.04315#S5.T8)\. On top of long context’s single LLM call, the loop issues three to four calls per question \(planner, optional dump, judge, answerer\), and the outer loop can issue further calls when the judge requests more evidence \(Table[9](https://arxiv.org/html/2606.04315#S5.T9)\)\.

#### Further efficiency improvements\.

On top of Algorithm[1](https://arxiv.org/html/2606.04315#alg1), we layer two latency\-oriented optimizations that retain most of its accuracy at a fraction of the token cost, and apply them identically across the three QA benchmarks\. \(i\)KV\-cache\-friendly chat layout— amortise prefill across the multi\-call loop\. The stable instruction block lives in the system message; the user message places the corpus catalog and graph\-schema hint before the per\-question delta\. On Qwen3\-32B / vLLM, this lifts the in\-run prefix\-cache hit rate from5\.4%5\.4\\%to over50%50\\%, so most multi\-call overhead resolves to cache reads\. \(ii\)Architectural single\-pass cap— replace the loop with one parallel pass\. As a deployment ablation of Algorithm[1](https://arxiv.org/html/2606.04315#alg1), we fixKmax=1K\_\{\\max\}\{=\}1, disable thedumpbranch, and skip the judge–replan step; the planner issues a single call, runsrg,read, andindex\_queryin parallel, and returns\. At a small accuracy cost \(−3\.8\-3\.8pp on LoCoMo\), single\-pass cuts query\-time tokens by6666–72%72\\%across the three QA benchmarks \(full per\-benchmark numbers in Table[9](https://arxiv.org/html/2606.04315#S5.T9)\)\. At that price, it is usually cheaper than long context and within±1\.6×\\pm 1\.6\\timesof DCI\-Lite’s single\-call cost\.

Table 9:Per\-question LLM token cost; preprocessing is amortised the same way as Fig\.[1](https://arxiv.org/html/2606.04315#S4.F1)\.†AutoMEM’s build equals LightMem\-graph’s preprocessing; the PlugMem\-graph variant substitutes PlugMem’s \(24\.224\.2K,52\.252\.2K,41\.541\.5K\)\.

## 6Conclusion

We revisit memory systems for LLM agents through a cross\-scenario generality and cost lens and draw: \(1\) no existing memory system wins across all five task families, while an off\-the\-shelf agent harness achieves the best generality; \(2\) index\-based methods fail on agentic\-trajectory QA in two ways: build\-time schemas drop step\- and action\-level evidence \(storage\), and passive retrieval cannot surface evidence the storage retains \(retrieval\); \(3\) reaching the cost–accuracy frontier requires combining the agent harness with selective indexing, which we instantiate in AutoMEM\.

We see two potential future directions: \(i\)a memory DSL that automates schema design— a DSL over memory primitives could replace verbose Cypher and let the schema be meta\-learned per workload\(Xiong et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib42)\); \(ii\)dedicated infrastructure for memory\-bound agents— serial prefill across plan/judge/answer dominates wall\-clock \(§[E](https://arxiv.org/html/2606.04315#A5)\), and a program\-aware serving stack that batches these calls and reuses prefix caches\(Kang et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib15)\)could close most of the latency gap to long\-context QA\.

## Limitations

#### No tiered budget routing\.

AutoMEM runs plan, judge and answer composition on the same backbone, but the three steps differ in difficulty: planning over a small corpus catalog and judging a candidate answer’s sufficiency are short, low\-entropy decisions, while answer composition is the step that needs the strong model’s reasoning\. A per\-step router that sends plan and judge to a cheap tier and reserves the strong backbone for answer composition would, by the call\-count breakdown in our efficiency table, cut the AutoMEM per\-question LLM cost by roughly22to33times\. We do not implement this router and report uniform\-backbone numbers throughout, so the reported cost of AutoMEM should be read as an upper bound\.

#### Judge protocol variance\.

On conversational QA, lenient judge prompts can reverse method rankings\. We therefore use a strict factual judge throughout; absolute scores under a more permissive rubric would be higher but the relative ordering should be read with the judge protocol in mind\.

## Ethical Considerations

#### Data and licensing\.

All datasets used in this work \(HotpotQA, LoCoMo, MemoryAgentBench, AMABench, ALFWorld, MemoryArena\) are publicly released research benchmarks under their original licenses, and we use them only for evaluation\. We do not collect new human\-subject data and we do not redistribute the underlying corpora; only derived metrics, judged\-answer files, and run configurations are released through our HuggingFace artifact repository\.

#### Models\.

Our backbones \(Qwen3\-32B,Qwen3\-Embedding\-4B,Qwen2\.5\-7B,Qwen3\-4B\-Instruct\) are open\-weight models released by their authors under permissive licenses\. All results are from inference\-time use through standard serving stacks \(sglang, vLLM\)\.

#### Risks and intended use\.

This paper studies the cost and generality of memory systems for LLM agents in benchmark settings\. It does not deploy agents in user\-facing products and does not store, retrieve, or operate on personal data\. The agentic memory harness recipe we recommend, AutoMEM, gives the LLM tool\-calling control over a memory store\. In a deployment setting where the memory contains user data, the same surface would need standard safeguards \(access control, content filtering, audit logging\) that are out of scope for this paper\.

#### Compute and environmental cost\.

Our experiment runs on a mix of GPUs \(NVIDIA B200, H200, RTX PRO 6000\)\. We report per\-question token and wallclock costs in the main paper so that practitioners can estimate compute before reproducing\. All cost analysis is conducted on an isolated B200 instance\.

## References

- Ai et al\. \(2025\)Qingyao Ai, Yichen Tang, Changyue Wang, Jianming Long, Weihang Su, and Yiqun Liu\. 2025\.[MemoryBench: A benchmark for memory and continual learning in LLM systems](https://arxiv.org/abs/2510.17281)\.*ArXiv preprint*, abs/2510\.17281\.
- Bai et al\. \(2024\)Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li\. 2024\.[LongBench v2: Towards deeper understanding and reasoning on realistic long\-context multitasks](https://arxiv.org/abs/2412.15204)\.*ArXiv preprint*, abs/2412\.15204\.
- Chhablani et al\. \(2024\)Gunjan Chhablani, Deshraj Khanna, and Singh Taranjeet\. 2024\.[Mem0: The memory layer for AI agents](https://github.com/mem0ai/mem0)\.GitHub\.
- Edge et al\. \(2024\)Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson\. 2024\.[From local to global: A graph RAG approach to query\-focused summarization](https://arxiv.org/abs/2404.16130)\.*ArXiv preprint*, abs/2404\.16130\.
- Fang et al\. \(2025\)Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang\. 2025\.[LightMem: Lightweight and efficient memory\-augmented generation](https://arxiv.org/abs/2510.18866)\.*ArXiv preprint*, abs/2510\.18866\.
- Francis et al\. \(2018\)Nadime Francis, Alastair Green, Paolo Guagliardo, Leonid Libkin, Tobias Lindaaker, Victor Marsault, Stefan Plantikow, Mats Rydberg, Petra Selmer, and Andrés Taylor\. 2018\.Cypher: An evolving query language for property graphs\.In*Proceedings of the 2018 International Conference on Management of Data \(SIGMOD\)*, pages 1433–1445\.
- Gutiérrez et al\. \(2025\)Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su\. 2025\.[From RAG to memory: Non\-parametric continual learning for large language models](https://arxiv.org/abs/2502.14802)\.*ArXiv preprint*, abs/2502\.14802\.
- He et al\. \(2026\)Zexue He, Yu Wang, Churan Zhi, Yuanzhe Hu, Tzu\-Ping Chen, Lang Yin, Ze Chen, Tong Arthur Wu, Siru Ouyang, Zihan Wang, Jiaxin Pei, Julian McAuley, Yejin Choi, and Alex Pentland\. 2026\.[MemoryArena: Benchmarking agent memory in interdependent multi\-session agentic tasks](https://arxiv.org/abs/2602.16313)\.*ArXiv preprint*, abs/2602\.16313\.
- Heule et al\. \(2025\)Stefan Heule, Emily Jia, and Naman Jain\. 2025\.[Improving agent with semantic search](https://cursor.com/blog/semsearch)\.Cursor Blog\.Published November 6, 2025\.
- Hsieh et al\. \(2024\)Cheng\-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg\. 2024\.[RULER: What’s the real context size of your long\-context language models?](https://arxiv.org/abs/2404.06654)*ArXiv preprint*, abs/2404\.06654\.
- Hu et al\. \(2026a\)Tiansheng Hu, Yilun Zhao, Canyu Zhang, Arman Cohan, and Chen Zhao\. 2026a\.[SAGE: Benchmarking and improving retrieval for deep research agents](https://arxiv.org/abs/2602.05975)\.*ArXiv preprint*, abs/2602\.05975\.
- Hu et al\. \(2025\)Yuanzhe Hu, Yu Wang, and Julian McAuley\. 2025\.[MemoryAgentBench: Evaluating memory in LLM agents via incremental multi\-turn interactions](https://arxiv.org/abs/2507.05257)\.*ArXiv preprint*, abs/2507\.05257\.
- Hu et al\. \(2026b\)Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yanbin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, and 28 others\. 2026b\.[Memory in the age of AI agents](https://arxiv.org/abs/2512.13564)\.*ArXiv preprint*, abs/2512\.13564\.
- Jiménez et al\. \(2023\)Carlos E\. Jiménez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan\. 2023\.[SWE\-bench: Can language models resolve real\-world GitHub issues?](https://arxiv.org/abs/2310.06770)*ArXiv preprint*, abs/2310\.06770\.
- Kang et al\. \(2026\)Hao Kang, Ziyang Li, Xinyu Yang, Weili Xu, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, and Simran Arora\. 2026\.[ThunderAgent: A simple, fast and program\-aware agentic inference system](https://arxiv.org/abs/2602.13692)\.*ArXiv preprint*, abs/2602\.13692\.
- Karpukhin et al\. \(2020\)Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen\-tau Yih\. 2020\.[Dense passage retrieval for open\-domain question answering](https://doi.org/10.18653/v1/2020.emnlp-main.550)\.In*Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, pages 6769–6781, Online\. Association for Computational Linguistics\.
- Lee et al\. \(2025\)Dong\-Ho Lee, Adyasha Maharana, Jay Pujara, Xiang Ren, and Francesco Barbieri\. 2025\.[REALTALK: A 21\-day real\-world dataset for long\-term conversation](https://arxiv.org/abs/2502.13270)\.*ArXiv preprint*, abs/2502\.13270\.
- Lewis et al\. \(2020\)Patrick S\. H\. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela\. 2020\.[Retrieval\-augmented generation for knowledge\-intensive NLP tasks](https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html)\.In*Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, virtual*\.
- Li et al\. \(2026a\)Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, and Jie Tan\. 2026a\.[TiMem: Temporal\-hierarchical memory consolidation for long\-horizon conversational agents](https://arxiv.org/abs/2601.02845)\.*ArXiv preprint*, abs/2601\.02845\.
- Li et al\. \(2026b\)Yifei Li, Weidong Guo, Lingling Zhang, Rongman Xu, Muye Huang, Hui Liu, Lijiao Xu, Yu Xu, and Jun Liu\. 2026b\.[Locomo\-plus: Beyond\-factual cognitive memory evaluation framework for LLM agents](https://arxiv.org/abs/2602.10715)\.*ArXiv preprint*, abs/2602\.10715\.
- Li et al\. \(2026c\)Zhuofeng Li, Haoxiang Zhang, Cong Wei, Pan Lu, Ping Nie, Yi Lu, Yuyang Bai, Shangbin Feng, Hangxiao Zhu, Ming Zhong, Yuyu Zhang, Jianwen Xie, Yejin Choi, James Zou, Jiawei Han, Wenhu Chen, Jimmy Lin, Dongfu Jiang, and Yu Zhang\. 2026c\.[Beyond semantic similarity: Rethinking retrieval for agentic search via direct corpus interaction](https://arxiv.org/abs/2605.05242)\.*ArXiv preprint*, abs/2605\.05242\.
- Liu et al\. \(2026a\)Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao\. 2026a\.[SimpleMem: Efficient lifelong memory for LLM agents](https://arxiv.org/abs/2601.02553)\.*ArXiv preprint*, abs/2601\.02553\.
- Liu et al\. \(2026b\)Xiaoyuan Liu, Tian Liang, Dongyang Ma, Deyu Zhou, Haitao Mi, Pinjia He, and Yan Wang\. 2026b\.[The pensieve paradigm: Stateful language models mastering their own context](https://arxiv.org/abs/2602.12108)\.*ArXiv preprint*, abs/2602\.12108\.
- Maharana et al\. \(2024\)Adyasha Maharana, Dong\-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang\. 2024\.[Evaluating very long\-term conversational memory of LLM agents](https://arxiv.org/abs/2402.17753)\.*ArXiv preprint*, abs/2402\.17753\.
- Ouyang et al\. \(2022\)Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L\. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F\. Christiano, Jan Leike, and Ryan Lowe\. 2022\.Training language models to follow instructions with human feedback\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Packer et al\. \(2023\)Charles Packer, Vivian Fang, Shishir G\. Patil, Kevin Lin, Sarah Wooders, and Joseph E\. Gonzalez\. 2023\.[MemGPT: Towards LLMs as operating systems](https://arxiv.org/abs/2310.08560)\.*ArXiv preprint*, abs/2310\.08560\.
- Pang et al\. \(2021\)Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, and Samuel R\. Bowman\. 2021\.[QuALITY: Question answering with long input texts, yes\!](https://arxiv.org/abs/2112.08608)*ArXiv preprint*, abs/2112\.08608\.
- Pollertlam and Kornsuwannawit \(2026\)Natchanon Pollertlam and Witchayut Kornsuwannawit\. 2026\.[Beyond the context window: A cost\-performance analysis of fact\-based memory vs\. long\-context LLMs for persistent agents](https://arxiv.org/abs/2603.04814)\.*ArXiv preprint*, abs/2603\.04814\.
- Qian et al\. \(2024\)Hongjin Qian, Zheng Liu, Peitian Zhang, Kelong Mao, Defu Lian, Zhicheng Dou, and Tiejun Huang\. 2024\.[MemoRAG: Boosting long context processing with global memory\-enhanced retrieval augmentation](https://arxiv.org/abs/2409.05591)\.*ArXiv preprint*, abs/2409\.05591\.
- Salemi et al\. \(2024\)Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani\. 2024\.[LaMP: When large language models meet personalization](https://doi.org/10.18653/v1/2024.acl-long.399)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 7370–7392\. Association for Computational Linguistics\.
- Schick et al\. \(2023\)Timo Schick, Jane Dwivedi\-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom\. 2023\.[Toolformer: Language models can teach themselves to use tools](https://arxiv.org/abs/2302.04761)\.*ArXiv preprint*, abs/2302\.04761\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y\.K\. Li, Y\. Wu, and Daya Guo\. 2024\.[DeepSeekMath: Pushing the limits of mathematical reasoning in open language models](https://arxiv.org/abs/2402.03300)\.*ArXiv preprint*, abs/2402\.03300\.
- Shridhar et al\. \(2020\)Mohit Shridhar, Xingdi Yuan, Marc\-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht\. 2020\.[ALFWorld: Aligning text and embodied environments for interactive learning](https://arxiv.org/abs/2010.03768)\.*ArXiv preprint*, abs/2010\.03768\.
- Shu et al\. \(2026\)Yiheng Shu, Saisri Padmaja Jonnalagedda, Xiang Gao, Bernal Jiménez Gutiérrez, Weijian Qi, Kamalika Das, Huan Sun, and Yu Su\. 2026\.[REMem: Reasoning with episodic memory in language agent](https://arxiv.org/abs/2602.13530)\.*ArXiv preprint*, abs/2602\.13530\.
- Tian et al\. \(2026\)Anxin Tian, Yiming Li, Xing Li, Hui\-Ling Zhen, Lei Chen, Xianzhi Yu, Zhenhua Dong, and Mingxuan Yuan\. 2026\.[SwiftMem: Fast agentic memory via query\-aware indexing](https://arxiv.org/abs/2601.08160)\.*ArXiv preprint*, abs/2601\.08160\.
- Wang et al\. \(2024\)Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji\-Rong Wen\. 2024\.[A survey on large language model based autonomous agents](https://doi.org/10.1007/s11704-024-40231-1)\.*Frontiers of Computer Science*, 18\(6\):186345\.
- Wang et al\. \(2026\)Yinjie Wang, Xuyang Chen, Xiaolong Jin, Mengdi Wang, and Ling Yang\. 2026\.[OpenClaw\-RL: Train any agent simply by talking](https://arxiv.org/abs/2603.10165)\.*ArXiv preprint*, abs/2603\.10165\.
- Wang et al\. \(2025\)Yu Wang, Ryuichi Takanobu, Zhiqi Liang, Yuzhen Mao, Yuanzhe Hu, Julian McAuley, and Xiaojian Wu\. 2025\.[Mem\-α\\alpha: Learning memory construction via reinforcement learning](https://arxiv.org/abs/2509.25911)\.*ArXiv preprint*, abs/2509\.25911\.
- Wu et al\. \(2024\)Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai\-Wei Chang, and Dong Yu\. 2024\.[LongMemEval: Benchmarking chat assistants on long\-term interactive memory](https://arxiv.org/abs/2410.10813)\.*ArXiv preprint*, abs/2410\.10813\.
- Xia et al\. \(2026\)Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Cihang Xie, and Huaxiu Yao\. 2026\.[SkillRL: Evolving agents via recursive skill\-augmented reinforcement learning](https://arxiv.org/abs/2602.08234)\.*ArXiv preprint*, abs/2602\.08234\.
- Xie et al\. \(2024\)Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su\. 2024\.[TravelPlanner: A benchmark for real\-world planning with language agents](https://arxiv.org/abs/2402.01622)\.*ArXiv preprint*, abs/2402\.01622\.
- Xiong et al\. \(2026\)Yiming Xiong, Shengran Hu, and Jeff Clune\. 2026\.[Learning to continually learn via meta\-learning agentic memory designs](https://arxiv.org/abs/2602.07755)\.*ArXiv preprint*, abs/2602\.07755\.
- Xu et al\. \(2025\)Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang\. 2025\.[A\-MEM: Agentic memory for LLM agents](https://arxiv.org/abs/2502.12110)\.*ArXiv preprint*, abs/2502\.12110\.
- Xu et al\. \(2026\)Xiucheng Xu, Bingbing Xu, Xueyun Tian, Zihe Huang, Rongxin Chen, Yunfan Li, and Huawei Shen\. 2026\.[Chain\-of\-memory: Lightweight memory construction with dynamic evolution for LLM agents](https://arxiv.org/abs/2601.14287)\.*ArXiv preprint*, abs/2601\.14287\.
- Yan et al\. \(2025\)Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Jinhe Bi, Kristian Kersting, Jeff Z\. Pan, Hinrich Schütze, Volker Tresp, and Yunpu Ma\. 2025\.[Memory\-R1: Enhancing large language model agents to manage and utilize memories via reinforcement learning](https://arxiv.org/abs/2508.19828)\.*ArXiv preprint*, abs/2508\.19828\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others\. 2025\.[Qwen3 technical report](https://arxiv.org/abs/2505.09388)\.*ArXiv preprint*, abs/2505\.09388\.
- Yang et al\. \(2026\)Ke Yang, Zixi Chen, Xuan He, Jize Jiang, Michel Galley, Chenglong Wang, Jianfeng Gao, Jiawei Han, and ChengXiang Zhai\. 2026\.[PlugMem: A task\-agnostic plugin memory module for LLM agents](https://arxiv.org/abs/2603.03296)\.*ArXiv preprint*, abs/2603\.03296\.
- Yang et al\. \(2018\)Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D\. Manning\. 2018\.[HotpotQA: A dataset for diverse, explainable multi\-hop question answering](https://doi.org/10.18653/v1/D18-1259)\.In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, pages 2369–2380, Brussels, Belgium\. Association for Computational Linguistics\.
- Yao et al\. \(2022\)Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao\. 2022\.[ReAct: Synergizing reasoning and acting in language models](https://arxiv.org/abs/2210.03629)\.*ArXiv preprint*, abs/2210\.03629\.
- Yu et al\. \(2025\)Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya\-Qin Zhang, Wei\-Ying Ma, Jingjing Liu, Mingxuan Wang, and Hao Zhou\. 2025\.[MemAgent: Reshaping long\-context LLM with multi\-conv RL\-based memory agent](https://arxiv.org/abs/2507.02259)\.*ArXiv preprint*, abs/2507\.02259\.
- Yue et al\. \(2026\)Yanwei Yue, Boci Peng, Xuanbo Fan, Jiaxin Guo, Qiankun Li, and Yan Zhang\. 2026\.[Mem\-T: Densifying rewards for long\-horizon memory agents](https://arxiv.org/abs/2601.23014)\.*ArXiv preprint*, abs/2601\.23014\.
- Zhang et al\. \(2026a\)Haozhen Zhang, Haodong Yue, Tao Feng, Quanyu Long, Jianzhu Bao, Bowen Jin, Weizhi Zhang, Xiao Li, Jiaxuan You, Chengwei Qin, and Wenya Wang\. 2026a\.[Learning query\-aware budget\-tier routing for runtime agent memory](https://arxiv.org/abs/2602.06025)\.*ArXiv preprint*, abs/2602\.06025\.
- Zhang et al\. \(2026b\)Shengtao Zhang, Jiaqian Wang, Ruiwen Zhou, Junwei Liao, Yuchen Feng, Zhuo Li, Yujie Zheng, Weinan Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Yutao Qi, Bo Tang, and Muning Wen\. 2026b\.[MemRL: Self\-evolving agents via runtime reinforcement learning on episodic memory](https://arxiv.org/abs/2601.03192)\.*ArXiv preprint*, abs/2601\.03192\.
- Zhao et al\. \(2026\)Yujie Zhao, Boqin Yuan, Junbo Huang, Haocheng Yuan, Zhongming Yu, Haozhou Xu, Lanxiang Hu, Abhilash Shankarampeta, Zimeng Huang, Wentao Ni, Yuandong Tian, and Jishen Zhao\. 2026\.[AMA\-Bench: Evaluating long\-horizon memory for agentic applications](https://arxiv.org/abs/2602.22769)\.*ArXiv preprint*, abs/2602\.22769\.
- Zhong et al\. \(2023\)Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang\. 2023\.[MemoryBank: Enhancing large language models with long\-term memory](https://arxiv.org/abs/2305.10250)\.*ArXiv preprint*, abs/2305\.10250\.
- Zhou et al\. \(2025\)Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang\. 2025\.[MEM1: Learning to synergize memory and reasoning for efficient long\-horizon agents](https://arxiv.org/abs/2506.15841)\.*ArXiv preprint*, abs/2506\.15841\.

## Appendix AUse of AI Writing Assistance

In line with the policy on AI Writing Assistance, we use AI assistants for code development, language polishing, and grammar of the manuscript\. All scientific claims, experimental design, results, and analyses are authored, reviewed, and verified by the human authors\.

## Appendix BDataset Details and Selection Criteria

Running all methods on the full release of each benchmark would be prohibitively expensive: a single full sweep would cost hundreds of GPU\-hours of LLM inference plus tens of millions of judge tokens\. We therefore evaluate every method on a fixed, deterministic subset per benchmark, chosen to preserve question\-type coverage\. Table[10](https://arxiv.org/html/2606.04315#A2.T10)lists the subset configuration for each task\.

Table 10:Benchmark subsets used in all main experiments\. Subsets are fixed across methods and runs to make accuracy and cost numbers directly comparable\.### B\.1Why We Use the LLM Judge as the Primary QA Metric

We use the LLM judge rather than token F1 on every QA benchmark because F1 is sensitive to answer verbosity rather than correctness, and LoCoMo is the cleanest illustration in our suite\. LoCoMo answers are naturally verbose: questions probe multi\-session personal context, so models respond in full sentences rather than entity spans\. For a question whose gold answer is “Sweden”, the prediction “Sweden” scores F11\.001\.00while “The country is Sweden” scores0\.400\.40, a2\.5×2\.5\\timesgap for the same underlying fact\. The effect compounds at the system level: holding the model fixed \(long\-context GPT\-4o\-mini\) and changing only the answering prompt produces a3434\-point swing in average LoCoMo F1, driven entirely by output verbosity rather than retrieval or reasoning quality \(Table[11](https://arxiv.org/html/2606.04315#A2.T11)\)\. HotpotQA gold answers are short entity spans \(“Eiffel Tower”, “1887”, “Stephen King”\), so the gap is smaller, but it does not vanish\. We therefore use the LLM judge uniformly across QA benchmarks\.

Table 11:LoCoMo token F1 under three answering prompts, holding the model fixed to long\-context GPT\-4o\-mini\.

## Appendix CBaseline Implementation Details

We integrate the official implementation of each memory baseline into our code repository as a git submodule, so that results can be reproduced against the upstream code without modification\. Two exceptions:

#### AMA\-Agent\.

The released open\-source code has gaps relative to the system described in the original paper\(Zhao et al\.,[2026](https://arxiv.org/html/2606.04315#bib.bib54)\)\. We adapt the implementation to match the paper’s description\.

#### MemoryArena\.

The release provides the dataset but not the simulated environment\. We implement an approximated environment ourselves and release it alongside the rest of our reproducibility artifacts\.

## Appendix DResults on Other Backbones

Our goal here is to investigate how the quality of the LLM backbone influences our conclusions\. We selectQwen3\-4B\-Instructsince Mem\-T also adopts this backbone\. We find that the instruct model works much better than its thinking counterpart\. All other components \(the embedding model, the 32B strict factual judge, prompts, subsets, and per\-method configs\) are held fixed to the main protocol\. ALFW, MA\-S, and MA\-T are omitted because Qwen3\-4B falls below the capability threshold required by these benchmarks: every method collapses to≈0\\approx 0on them, meaning the columns do not differentiate between methods\. Based on these results, we find that the effectiveness of DCI\-Lite on MemoryAgentBench decreases, which can be attributed to the smaller model’s weaker capability to generate appropriate queries\. Moreover, we find that for smaller models, index\-based memory systems actually hold a greater advantage over long\-context methods\. One interesting phenomenon we observe is that across a series of tasks, MemRL’s performance is actually better with the 4B model backbone, which may stem from a distinction between thinking and instruction modes\. Generally, these results align with our intuition: historically, when LLM capabilities were limited, index\-based memory dominated since it could improve both performance and efficiency\. However, as LLM capabilities grow, we have crossed an inflection point where memory management paradigms should innovate to become LLM\-led\. While the Qwen3\-32B model used in our main text is a mid\-tier LLM, we believe that active management will present even more pronounced advantages for frontier LLMs\.

Table 12:Qwen3\-4B\-Instruct\-2507 backbone ablation\. Mem\-T is ignored here since the result in Table[3](https://arxiv.org/html/2606.04315#S4.T3)is already taken from a post\-trained 4B model\.
## Appendix EToken Efficiency vs\. System Efficiency

In §[3](https://arxiv.org/html/2606.04315#S3)we track per\-question token cost alongside accuracy\. However, token counts alone do not capture deployed cost: wall\-clock latency, GPU utilization, and KV\-cache reuse all depend on how the tokens are issued, not just how many\. This appendix unpacks that gap in two steps\. First, Table[13](https://arxiv.org/html/2606.04315#A5.T13)reports the serving\-stack measurements reporting alongside accuracy and tokens, taken from Qwen3\-32B on sglang on a single B200\. Second, Table[14](https://arxiv.org/html/2606.04315#A5.T14)classifies each system\-efficiency finding as either an implementation gap that engineering can close or a structural tradeoff tied to the method’s design\.

Table 13:Serving\-stack metrics per method, Qwen3\-32B on sglang on a single B200\. HotpotQA’s numbers are recorded using a 2000\-corpus subset\.Three patterns stand out\. First, the methods with the lowest stable\-prefix fraction \(HippoRAG, AMA\-Agent, SimpleMem\) are exactly the methods where the wall\-vs\-token gap is largest; the prefix column predicts the gap better than the LLM\-calls\-per\-question column does\. Second, the average in\-flight requests is∼\\sim11across every structured\-memory method we audited, regardless of stated design: the orchestrator dispatches one request per chunk, so continuous batching has nothing to batch against, and decode MFU lands in the lower range\. Third, decode MFU correlates negatively with prefill\-to\-decode ratio: methods that send long prompts and decode short outputs leave the decode kernels idle even when their token totals look low\. Long context and DCI\-Lite sit at the top of every column because they neither break the prefix nor serialize the orchestration\.

The takeaway for future memory\-system reports is that per\-question token counts can hide order\-of\-magnitude differences in wallclock and dollar cost\. We recommend publishing the columns in Table[13](https://arxiv.org/html/2606.04315#A5.T13)alongside accuracy and token totals so that the comparison cannot be dominated by orchestrator choices\.

#### Case\-by\-case: implementation problems vs\. inherent tradeoffs\.

Not every system\-efficiency finding is closeable by engineering: some are structurally tied to what the method is designed to do\. Table[14](https://arxiv.org/html/2606.04315#A5.T14)classifies each finding as either an implementation problem \(an engineering pass would close most of the gap without changing the method\), an inherent tradeoff \(closing the gap would require redesigning the method\), or a mixed case\. The reading: most of HippoRAG’s gap is closable by async fan\-out and prompt\-shape changes, AMA\-Agent’s gap is largely structural \(the tool loop and mutating state are the method’s premise\), and SimpleMem sits in between\.

Table 14:Classification of every system\-efficiency finding in this appendix\. impl\.: closeable by engineering without changing the method\. tradeoff: structurally tied to the method’s design\. mixed: one component closeable, another structural\.