Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents

# Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents
Source: [https://arxiv.org/html/2608.15008](https://arxiv.org/html/2608.15008)
Wooseong Yang Yiwei Yang Henry Peng Zou Hanrong Zhang Ying Nian WuHaolun Wu Kai\-Wei Chang Philip S\. Yu Xue Liu Aylin Caliskan\[3pt\]University of Illinois Chicago University of Washington McGill University\[1pt\]MBZUAI University of California, Los Angeles\[1pt\]Corresponding authors

###### Abstract

Memory is becoming core infrastructure for long\-horizon LLM agents, yet existing evaluations offer limited guidance on which memory substrate, namely the underlying medium in which memory is represented and stored, should be used under different operating regimes\. We present a controlled harness evaluation of memory substrates for memory\-augmented agents, covering dense and sparse indices, text records, structural stores, hierarchical stores, refinement\-based memories, parametric updates, and activation\-compatible context mechanisms\. Across three backbone models and four benchmark suites spanning user\-centric question answering and agent\-centric decision\-making, we instrument 26 performance and efficiency metrics under a unified harness\. Our results show that no single substrate consistently dominates: broad retrieval benefits long\-context factual QA, while excessive retrieval can harm sequential decision\-making by shifting attention away from action\-critical context\. Scalability introduces a further routing axis, as substrates that perform well at moderate history lengths can become costly or brittle at longer horizons\. These findings motivate substrate routing as a necessary component of adaptive agent memory systems and provide empirical guidance for designing efficient, reliable, and regime\-aware long\-term memory for LLM agents\. Code will be made available upon acceptance\.

## 1Introduction

Foundation model agents are rapidly advancing from single\-turn assistants into long\-horizon settings that demand persistent memory: coding assistants accumulating project context across pull requests\[[81](https://arxiv.org/html/2608.15008#bib.bib1),[31](https://arxiv.org/html/2608.15008#bib.bib5)\], web agents learning browsing patterns through repeated navigation\[[83](https://arxiv.org/html/2608.15008#bib.bib34)\], personal companions tracking evolving user preferences across hundreds of sessions\[[82](https://arxiv.org/html/2608.15008#bib.bib15)\], and scientific discovery agents refining hypotheses over iterative cycles\[[41](https://arxiv.org/html/2608.15008#bib.bib3)\]\. Despite spanning vastly different domains, these agents share a common challenge: writing new experience into a persistent store, reading relevant knowledge at decision time, and managing an ever\-growing memory under finite compute and storage budgets\[[69](https://arxiv.org/html/2608.15008#bib.bib82)\]\. No single memory design excels at all three along the axes of accuracy, efficiency, and cost\[[27](https://arxiv.org/html/2608.15008#bib.bib7),[85](https://arxiv.org/html/2608.15008#bib.bib8)\]: a vector index writes cost\-effectively but struggles with multi\-hop retrieval\[[9](https://arxiv.org/html/2608.15008#bib.bib40)\]; a knowledge graph reasons relationally but requires orders of magnitude more LLM calls to construct\[[33](https://arxiv.org/html/2608.15008#bib.bib83)\]; a KV cache integrates tightly with the model but sacrifices scalability\[[26](https://arxiv.org/html/2608.15008#bib.bib23)\]\. This diversity of trade\-offs motivates the long\-term goal of auniversal, adaptive memory systemthat dynamically selects or composes substrates, namely the underlying media in which memory is represented and stored, depending on the query, the task, and the deployment constraints\.

Designing such a system requires understanding when each substrate is preferable and, critically, under what operating regime\. Recent work on agent harnesses, the scaffolding around a model that decomposes tasks, manages context across sessions, and evaluates outputs, has shown that this scaffolding is often the primary lever for agent performance beyond raw model capability\[[37](https://arxiv.org/html/2608.15008#bib.bib80),[2](https://arxiv.org/html/2608.15008#bib.bib81),[54](https://arxiv.org/html/2608.15008#bib.bib84)\]\. The memory substrate is the infrastructure layer of any such harness: its design constrains every other harness decision, from write cost and read latency to whether stored knowledge is inspectable and how the system degrades as histories grow\[[71](https://arxiv.org/html/2608.15008#bib.bib59)\]\. Yet the same substrate can be the quality leader on one task and a strictly dominated choice on another, so a universal harness must not commit to a single substrate at design time\[[78](https://arxiv.org/html/2608.15008#bib.bib85)\]\. It must route between them per regime\. Producing the empirical signal for that routing requires*evaluating the harness itself*: isolating the memory substrate as a controlled variable while holding the rest of the scaffolding fixed, across regimes that stress fundamentally different memory operations\.

Figure 1:Evaluation landscape of 52 memory\-augmented LLM systems \(2023–2026\)\.\(a\)Benchmark adoption: 62% of usage concentrates on LoCoMo and LongMemEval\(b\)Metric coverage: every system reports accuracy but only 21% report any efficiency metric; 81% of systems use GPT\-family backbones\.\(c\)Breadth and efficiency: half of systems use a single benchmark with no efficiency metrics; no system simultaneously achieves broad coverage and comprehensive efficiency reporting\.Yet current evaluations provide almost no such signal\. As shown in Figure[1](https://arxiv.org/html/2608.15008#S1.F1), a landscape analysis of 52 recent memory\-augmented systems exposes three limitations that leave the path toward universal agent memory empirically ungrounded\. First, evaluations are*benchmark\-concentrated*: 62% of all benchmark and system pairs draw from just two dialogue\-centric datasets, LoCoMo\[[30](https://arxiv.org/html/2608.15008#bib.bib9)\]and LongMemEval \(LME\)\[[65](https://arxiv.org/html/2608.15008#bib.bib2)\], while agentic tasks remain almost unexamined \(Figure[1](https://arxiv.org/html/2608.15008#S1.F1)a\)\. Second,*metric and backbone coverage is narrow*: every system reports accuracy, but only 21% measure any efficiency dimension; management cost and compression ratio are unreported\. Furthermore, 81% of systems use the GPT\-family as their sole backbone, making it impossible to disentangle substrate effects from model\-specific behavior \(Figure[1](https://arxiv.org/html/2608.15008#S1.F1)b\)\. Third,*evaluation scope is shallow*: half of all systems test on a single benchmark with zero efficiency metrics, and no system simultaneously achieves broad coverage and comprehensive efficiency reporting \(Figure[1](https://arxiv.org/html/2608.15008#S1.F1)c\)\. The full per\-system survey, inclusion criteria, and aggregate statistics are reported in Appendix[A](https://arxiv.org/html/2608.15008#A1)\. As a result, we know which system tops a given leaderboard in terms of accuracy, but not under what operating regime a different substrate would be preferable, precisely the routing signal that any universal memory harness requires\.

We close this gap with a controlled harness evaluation that isolates the memory substrate as the sole experimental variable\. Adopting the taxonomy of\[[15](https://arxiv.org/html/2608.15008#bib.bib11)\], we partition the design space intoexternal memory\(information in data structures outside the model\) andinternal memory\(information encoded in model weights or activations\), and implement 11 methods spanning all seven substrate families, as shown in Figure[2](https://arxiv.org/html/2608.15008#S1.F2)\. All substrates are evaluated on identical interaction histories under three backbones and four benchmarks covering both user\-centric question answering and agent\-centric decision\-making\.

The harness evaluation yields three findings\. First, the optimal substrate*reverses*between regimes: structural graphs lead dialogue QA but are Pareto\-dominated on agentic tasks, while refinement memories lead embodied planning but trail on QA \(Section[5\.1](https://arxiv.org/html/2608.15008#S5.SS1)\)\. Second, retrieving more entries helps QA but hurts agentic decisions, and an attention probe explains why: as retrieval depth grows, the model’s attention shifts away from the task context toward the retrieved block, which is exactly where the answer lives in QA but not in agentic tasks, where the policy needs to attend to the current observation instead \(Section[5\.2](https://arxiv.org/html/2608.15008#S5.SS2)\)\. Third, a scalability study shows that refinement memories scale gracefully while structural and full\-context substrates face deployment limits at long horizons \(Section[5\.3](https://arxiv.org/html/2608.15008#S5.SS3)\)\. Together, these findings yield one design rule,*trade read breadth for write depth*\(retrieve fewer entries per query and invest more in distilling and structuring memory at write time\), and a broader implication: no single substrate can serve every regime, so universal agent memory must be realized as amulti\-substrate or multi\-agent systemin which heterogeneous substrates are composed and routed per regime, each handling the operating conditions it is suited for\. Our contributions are summarized as follows:

- •The first holistic harness evaluation of agent memory\.A controlled evaluation isolating the memory substrate across 11 methods and seven substrate families of the external and internal taxonomy, three backbones, and four benchmarks spanning user\-centric and agent\-centric regimes, instrumented with 26 metrics that make deployment\-critical costs visible for the first time\.
- •A cross\-regime performance–latency analysis of memory substrates\.The first controlled comparison that maps each method and substrate family onto a unified performance–latency landscape across user\-centric and agent\-centric tasks, identifying which substrate families are dominated under which operating regime, providing the routing signal that any multi\-substrate or multi\-agent memory system needs\.
- •A diagnostic study of retrieval scaling and context scalability\.A retrieval\-depth sweep paired with attention probing that diagnoses how retrieval breadth reshapes attention allocation across regimes, together with a scalability analysis that characterizes how each substrate family scales with input growth, yielding actionable design rules for regime\-aware memory\.

![Refer to caption](https://arxiv.org/html/2608.15008v1/submem_mem.png)Figure 2:Overview of the harness evaluation\. 11 memory methods \(M1–M11\) spanning seven substrate families are evaluated on identical interaction histories across four benchmarks, instrumented with 26 metrics in two families \(performance and efficiency\)\.
## 2Memory Substrates and Configuration

Memory system configurations are highly complex, with many tunable parameters and modular design choices, so rather than asking which method is inherently superior, we examine which configurations yield specific benefits and trade\-offs under different task settings and operating regimes, the empirical prerequisite for any universal memory system that must select substrates dynamically\.

#### Substrate overview\.

We study eleven memory substrates arranged into seven families \([Table4](https://arxiv.org/html/2608.15008#A2.T4)in Appendix[B](https://arxiv.org/html/2608.15008#A2)\)\. On the external side, the eight methods span a structure spectrum\.Flat Index: M1 Dense Vector and M2 Sparse Vector\[[68](https://arxiv.org/html/2608.15008#bib.bib12),[43](https://arxiv.org/html/2608.15008#bib.bib86)\]require no LLM calls and establish the retrieval floor\.Text Record: M3 Gist Index\[[23](https://arxiv.org/html/2608.15008#bib.bib16)\]adds write\-side gisting and read\-side page selection\.Structuralintroduces relational links: M4 Evolving Notes\[[70](https://arxiv.org/html/2608.15008#bib.bib6)\]writes interlinked notes with evolution\-triggered rewrites, and M5 Dual\-Level Graph\[[8](https://arxiv.org/html/2608.15008#bib.bib48)\]builds a dual\-level entity\-relation graph\.Hierarchical: M6 Hierarchical Tree\[[45](https://arxiv.org/html/2608.15008#bib.bib19),[49](https://arxiv.org/html/2608.15008#bib.bib18)\]recursively clusters and summarizes memory into a multi\-level hierarchy retrieved via collapsed\-tree dense search\.Refinementdistills experience rather than accumulating it: M7 Distilled Strategies\[[35](https://arxiv.org/html/2608.15008#bib.bib49)\]judges trajectories and extracts reusable strategies, and M8 Skill Bundles\[[79](https://arxiv.org/html/2608.15008#bib.bib21)\]clusters experiences into skill bundles\. On the internal side, theWeightfamily, M9 Adapter Tuning\[[80](https://arxiv.org/html/2608.15008#bib.bib22)\], bakes knowledge into weights via fine\-tuning, while theActivationfamily includes M10 Full Context and M11 Episode\-Clustered Re\-prefill\[[20](https://arxiv.org/html/2608.15008#bib.bib24)\], which clusters turns into episodes and re\-prefills selected episodes at read time\. We report Zep\[[40](https://arxiv.org/html/2608.15008#bib.bib17)\], Mem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\], and MemGPT\[[36](https://arxiv.org/html/2608.15008#bib.bib14)\]in an auxiliary\-cost analysis \(Appendix[B\.3](https://arxiv.org/html/2608.15008#A2.SS3)\) rather than the main controlled comparison because their production pipelines introduce substantially different auxiliary\-LLM budgets\.

## 3Experiment Design

Our goal is to isolate the implemented memory substrate family under a shared harness, backbone, prompting, and auxiliary\-LLM setting\. We ask two questions: \(1\) how much of an agent’s behavior, in terms of task performance and efficiency, is attributable to the substrate alone, and \(2\) whether that attribution is stable across operating regimes\. We test every substrate on both user\-centric benchmarks, where the agent retrieves facts from long conversational histories, and agent\-centric benchmarks, where the agent must execute precise action sequences\. Three configurations are excluded by design: M7 only applies to trajectory\-bearing agent\-centric tasks \(its strategy\-distillation pipeline has no analog on dialogue inputs\), M9 is incompatible with the A4B MoE routing inGemma\-4\-26B\-A4B\-IT, and M10 \(full context\) is omitted from both agent\-centric benchmarks \(ALFWorld and BigCodeBench\-Hard\) because the cumulative context exceeds every tested context window\.

#### Benchmarks\.

We select four benchmarks that collectively span the user\-centric and agent\-centric divide\. On the user\-centric side,LoCoMo\[[30](https://arxiv.org/html/2608.15008#bib.bib9)\]provides ten multi\-session dialogues with1,9861\{,\}986questions across five categories, andMemoryAgentBench \(MAB\)\[[12](https://arxiv.org/html/2608.15008#bib.bib28)\]factors long\-context memory into four capabilities, Accurate Retrieval \(AR, instantiated on the LongMemEval\-S∗subset\[[65](https://arxiv.org/html/2608.15008#bib.bib2)\]\), Long\-Range Understanding \(LRU\), Test\-Time Learning \(TTL\), and Conflict Resolution \(CR\); we additionally use CR as our scalability probe by sweeping content length\. In the result tables we surface AR under its LongMemEval\-S \(LME\-S\) label as a separate column alongside the LRU, TTL, and CR capabilities\. Together these benchmarks stress recall\-oriented capabilities, the regime where broad retrieval helps\. On the agentic side,ALFWorld\[[47](https://arxiv.org/html/2608.15008#bib.bib31)\]provides134134valid\-unseen embodied\-planning tasks evaluated under both within\-episode \(accumulation\) and cross\-episode \(recovery from a pre\-built bank\) regimes, andBigCodeBench\-Hard\[[86](https://arxiv.org/html/2608.15008#bib.bib32)\]provides148148code tasks graded with retrieval over a cross\-task pool of successes and failures\. These benchmarks stress the opposite regime, where retrieval noise is toxic and the substrate must distill experience into compact, high\-precision representations\. Full benchmark setups, capability definitions, and pool construction details are in Appendix[C](https://arxiv.org/html/2608.15008#A3)\.

#### Models, inference, and metrics\.

The backbone models areQwen3\-8B,Qwen3\-32B\-AWQ, andGemma\-4\-26B\-A4B\-IT, tested on all four benchmarks using vLLM on4×4\{\\times\}H200 GPUs\. For consistency, the auxiliary LLM is fixed togpt\-4o\-miniacross all methods that require one for memory writing, management, or input organization\. LLM\-as\-a\-judge evaluation also usesgpt\-4o\-miniwith a shared judging template\. We instrument every run with the 26 metrics defined in Table[6](https://arxiv.org/html/2608.15008#A4.T6)in Appendix[D](https://arxiv.org/html/2608.15008#A4), organized into task\-performance metrics and efficiency metrics, including benchmark\-native primary metrics such as Pass@1 on BCB, task success on ALFWorld, and SubEM on MAB, as well as storage, latency, token, call, total\-time, and cost measurements\.

#### Ablations\.

We isolate two design axes while holding the substrate implementation, backbone, and prompt template fixed\. First, a top\-kkretrieval\-breadth sweep on every substrate exposing akkknob:k∈\{1,2,5,10,20\}k\\in\\\{1,2,5,10,20\\\}on LoCoMo andk∈\{1,2,3,4,5\}k\\in\\\{1,2,3,4,5\\\}on ALFWorld, with a no\-memory baseline that receives the same template with an empty retrieved block, so the curves attribute quality to retrieval rather than scaffolding\. Second, a context\-window stress on MAB CR under three conversation\-length regimes \(6K, 32K, 262K tokens\) with the question pool, gold answers, and judge held constant\. Together these ablations separate substrates whose advantage comes from retrieval breadth from those driven by recency\-aware update semantics\.

## 4Results

We organize the results along the user\-centric and agent\-centric divide that defines our harness\. Table[1](https://arxiv.org/html/2608.15008#S4.T1)reports user\-centric results on LoCoMo, LME\-S, and MAB, where the substrate must surface relevant facts from accumulated dialogue history; Table[2](https://arxiv.org/html/2608.15008#S4.T2)reports agent\-centric results on ALFWorld and BigCodeBench\-Hard, where the substrate must inform sequential decisions without diluting attention over action\-critical context\. Full per\-metric breakdowns across all 26 metrics are in Appendix[E](https://arxiv.org/html/2608.15008#A5)\.

Table 1:Memory substrates onLoCoMo,LME\-S, andMAB\(LRU, TTL, CR\)\. P4↑\\uparrow: LLM\-judge score;E15E\_\{15\}↓\\downarrow: per\-query latency \(s\)\. Within each model band, green and red mark the per\-column best and worst;boldmarks the per\-model P4 winner on each benchmark\.LoCoMoLME\-SMAB LRUMAB TTLMAB CRModelSub\.FamilyP4↑\\uparrow𝑬𝟏𝟓E\_\{15\}↓\\downarrowP4↑\\uparrow𝑬𝟏𝟓E\_\{15\}↓\\downarrowP4↑\\uparrow𝑬𝟏𝟓E\_\{15\}↓\\downarrowP4↑\\uparrow𝑬𝟏𝟓E\_\{15\}↓\\downarrowP4↑\\uparrow𝑬𝟏𝟓E\_\{15\}↓\\downarrow![[Uncaptioned image]](https://arxiv.org/html/2608.15008v1/Qwen.png)Qwen3\-8BExternalM1Flat0\.5400\.770\.4787\.300\.5789\.940\.5600\.920\.2701\.65M2Flat0\.4700\.330\.4801\.750\.6203\.230\.4700\.330\.2701\.39M3Text0\.5624\.180\.49555\.580\.563146\.650\.64027\.150\.2402\.30M4Struct0\.4356\.360\.430225\.670\.493274\.060\.48012\.650\.2701\.50M5Struct0\.64827\.840\.537187\.210\.578313\.180\.54023\.700\.4406\.01M6Hier0\.5561\.390\.50719\.960\.49359\.980\.2903\.240\.2903\.30M8Refine0\.5093\.030\.490102\.850\.592197\.090\.5601\.020\.3402\.07InternalM9Weight0\.3791\.760\.25051\.500\.30085\.000\.2208\.500\.2601\.95M10Act0\.5894\.550\.1206\.610\.6766\.580\.4802\.280\.2701\.64M11Act0\.3072\.030\.09014\.610\.63421\.300\.5600\.560\.2401\.76![[Uncaptioned image]](https://arxiv.org/html/2608.15008v1/Qwen.png)Qwen3\-32B\-AWQExternalM1Flat0\.5822\.590\.54812\.000\.67632\.390\.8101\.810\.4604\.33M2Flat0\.4872\.370\.4977\.310\.66223\.140\.7801\.200\.4104\.03M3Text0\.5866\.470\.56067\.420\.775477\.970\.85094\.290\.3403\.94M4Struct0\.4528\.700\.490234\.000\.535278\.930\.71017\.180\.1901\.73M5Struct0\.68329\.220\.627194\.080\.578333\.950\.63026\.020\.3809\.38M6Hier0\.5735\.200\.59320\.270\.57773\.160\.3303\.770\.4108\.70M8Refine0\.5335\.530\.533106\.780\.592228\.660\.8101\.930\.4403\.39InternalM9Weight0\.3994\.730\.290171\.700\.422253\.630\.25022\.150\.2805\.50M10Act0\.66916\.250\.19022\.980\.62027\.430\.7602\.430\.3803\.74M11Act0\.2744\.170\.11352\.390\.62025\.000\.8101\.450\.3002\.65![[Uncaptioned image]](https://arxiv.org/html/2608.15008v1/Gemma.png)Gemma\-4\-26B\-A4B\-ITExternalM1Flat0\.5740\.930\.5158\.490\.67616\.500\.77014\.500\.4604\.09M2Flat0\.5550\.490\.5133\.370\.6629\.180\.7300\.200\.4603\.88M3Text0\.5994\.810\.53031\.780\.720238\.780\.75036\.210\.4303\.67M4Struct0\.4766\.610\.523230\.390\.535267\.000\.72016\.120\.4603\.97M5Struct0\.71925\.950\.667190\.930\.451332\.940\.72025\.790\.5508\.23M6Hier0\.5921\.420\.59317\.190\.50762\.760\.25022\.000\.4006\.50M8Refine0\.5543\.220\.537103\.800\.535228\.160\.7701\.680\.5002\.82InternalM10Act0\.6885\.170\.1106\.910\.57710\.380\.4601\.360\.4504\.24M11Act0\.3302\.160\.11518\.500\.31876\.000\.4801\.500\.1801\.48

### 4\.1User\-centric Benchmark

No single substrate wins every capability, ruling out single\-substrate universal memory by design\.Table[1](https://arxiv.org/html/2608.15008#S4.T1)shows that the best substrate changes with the capability being tested, and this pattern is broadly visible across backbones despite model\-specific interactions\. On LoCoMo and LME\-S, M5 performs best across model bands, suggesting that combining entity\-level graph traversal with chunk\-level vector search is useful for fact retrieval over long dialogue histories\. On MAB LRU, less lossy access mechanisms often perform better: M10 leadsQwen3\-8B, while M3 leadsQwen3\-32B\-AWQ\. On MAB TTL, M3 and M8 dominate, whereas the M9 adapter\-tuning implementation performs poorly because newly acquired facts are not directly inspectable or selectively queried\. On MAB CR, M5 leads two of three models, consistent with the value of explicit update semantics for recency\-sensitive conflicts\. Cost separates otherwise competitive methods: M2 is consistently cheap, while M5 can be 10–100×\\timesslower and is only worthwhile when its structural mechanism matches the task bottleneck\.

Table 2:Memory substrates evaluated onALFWorld\-unseenandBigCodeBench\-Hard\. Within each model band, green marks the best value on a metric, red marks the worst\.PavgP\_\{\\mathrm\{avg\}\}is the mean per\-task partial\-success score\.Boldmarks the per\-model performance winner \(TSR for ALFWorld, Pass@1 for BCB\)\. M10 is omitted from both agent\-centric benchmarks because the cumulative context exceeds the tested context windows; M9 is omitted from theGemma\-4band because it is incompatible with the A4B MoE routing\.ALFWorld\-unseenBigCodeBench\-HardModelSub\.FamilyTSR↑\\uparrow𝐒𝐭𝐞𝐩𝐬\|𝓢\\mathrm\{Steps\}\\\!\\mid\\\!\\mathcal\{S\}↓\\downarrow𝑷𝐚𝐯𝐠P\_\{\\mathrm\{avg\}\}↑\\uparrow𝑬𝟐𝐩𝟗𝟎E\_\{2\}^\{\\mathrm\{p90\}\}↓\\downarrow𝑬𝟏𝟓E\_\{15\}↓\\downarrowPass@1↑\\uparrow𝑷𝐚𝐯𝐠P\_\{\\mathrm\{avg\}\}↑\\uparrow𝑬𝟐𝐚𝐯𝐠E\_\{2\}^\{\\mathrm\{avg\}\}↓\\downarrow𝑬𝟏𝟓E\_\{15\}↓\\downarrow![[Uncaptioned image]](https://arxiv.org/html/2608.15008v1/Qwen.png)Qwen3\-8B\-NoMem5\.712\.213\.513\.24888\.112\.43\.84M1Flat5\.25\.96\.616\.16729\.520\.15\.05M2Flat6\.76\.09\.415\.764710\.111\.94\.95M3Text5\.28\.38\.416\.36538\.112\.45\.35M4Struct7\.57\.98\.514\.860510\.120\.44\.65M5Struct9\.017\.412\.515\.762015\.517\.128\.328M6Hier4\.515\.08\.315\.665612\.222\.24\.87M7Refine7\.513\.79\.616\.062912\.216\.94\.617M8Refine8\.218\.59\.515\.362014\.924\.05\.119M9Weight3\.019\.53\.222\.09217\.44\.612\.713M11Act11\.914\.214\.513\.555313\.523\.616\.817![[Uncaptioned image]](https://arxiv.org/html/2608.15008v1/Qwen.png)Qwen3\-32B\-AWQ\-NoMem22\.411\.323\.720\.358317\.625\.63\.94M1Flat27\.69\.616\.231\.11\.08k18\.224\.25\.05M2Flat21\.610\.815\.430\.81\.09k19\.621\.85\.05M3Text26\.99\.921\.830\.395018\.224\.55\.48M4Struct26\.912\.424\.328\.681617\.622\.44\.920M5Struct23\.111\.423\.027\.485116\.216\.631\.231M6Hier23\.99\.922\.826\.881716\.216\.84\.47M7Refine32\.111\.129\.927\.271617\.613\.54\.321M8Refine22\.412\.421\.328\.088613\.523\.04\.622M9Weight29\.912\.127\.421\.461214\.215\.514\.615M11Act26\.912\.324\.722\.078313\.513\.821\.321![[Uncaptioned image]](https://arxiv.org/html/2608.15008v1/Gemma.png)Gemma\-4\-26B\-A4B\-IT\-NoMem7\.511\.317\.316\.467114\.219\.519\.820M1Flat10\.415\.016\.321\.678618\.226\.421\.922M2Flat9\.711\.517\.221\.475319\.627\.020\.621M3Text10\.414\.814\.920\.578114\.219\.523\.423M4Struct8\.214\.513\.922\.594218\.926\.922\.423M5Struct4\.58\.59\.419\.974920\.928\.445\.646M6Hier10\.411\.418\.121\.174118\.926\.024\.328M7Refine8\.210\.117\.821\.178816\.925\.124\.137M8Refine8\.212\.518\.520\.176318\.226\.321\.226M11Act9\.011\.720\.117\.171016\.924\.031\.632

### 4\.2Agent\-centric Benchmark

The same retrieval operation that sharpens code generation starves embodied planning, splitting the agent\-centric regime in two\.On ALFWorld, the substrates that win are those that*denoise*accumulated experience before it reaches the policy: M11 doubles NoMem onQwen3\-8Bby re\-prefilling only the matched episode rather than the full trajectory buffer, and M7 achieves the table’s peak TSR of32\.1%32\.1\\%onQwen3\-32B\-AWQby distilling trajectories into compact, reusable strategies\. Both substrates share a common operation: they suppress task\-irrelevant tokens before retrieval, either by clustering raw turns into episodes \(M11\) or by abstracting trajectories into reasoning templates \(M7\)\. Substrates that lack this denoising step show inconsistent gains across backbones: onQwen3\-32B\-AWQ, flat retrievers range from slightly below NoMem \(M2, 21\.6 vs\. 22\.4\) to well above it \(M1, 27\.6\), and text and structural stores \(M3 26\.9, M4 26\.9, M5 23\.1\) likewise match or exceed NoMem, yet none close the gap to the refinement winner M7 \(32\.1\)\. OnQwen3\-8B, however, flat retrievers straddle NoMem \(M1 5\.2 below, M2 6\.7 above 5\.7\), confirming that the value of raw\-text retrieval on embodied planning is backbone\-dependent rather than universally harmful\. On embodied planning, retrieved trajectories compete for attention with the current observation and admissible\-action list, so the substrate’s value lies in noise removal rather than recall: only memories that explicitly compress raw experience into actionable abstractions help, and the rest dilute the cue the policy depends on\.

On BigCodeBench\-Hard the picture inverts because retrieved code snippets do not compete with the task prompt, they extend it: a working solution to a related problem is reusable scaffolding rather than a distractor\. Retrieval therefore helps broadly, with M5 leadingQwen3\-8BandGemma\-4\(15\.5%15\.5\\%and20\.9%20\.9\\%Pass@1\) by surfacing structurally related code through its graph and vector hybrid, and even M2 lifting Pass@1 above NoMem on every backbone\. Yet the cost trade\-off is steep: onQwen3\-32B\-AWQM2 \(19\.6%19\.6\\%\) overtakes M5 \(16\.2%16\.2\\%\) at roughly1/61/6the latency, and across backbones M5 pays 2 to 6 times the inference cost of flat substrates, making it a strictly dominated choice when latency matters\. The two benchmarks therefore stress opposite memory operations: ALFWorld rewards substrates that abstract noise away before retrieval, while BigCodeBench rewards substrates that retrieve broadly without paying for abstraction, an asymmetry whose attention\-level mechanism we trace in Section[5\.2](https://arxiv.org/html/2608.15008#S5.SS2)\.

Figure 3:Performance vs\. latency across four benchmarks onQwen3\-32B\-AWQ\. \(a\) LoCoMo and \(b\) LongMemEval\-S: absoluteP4P\_\{4\}\. \(c\) ALFWorld\-unseen and \(d\) BigCodeBench\-Hard:Δ\\Delta\-performance vs\. latency overhead anchored at NoMem \(No memory baseline\)\.![Refer to caption](https://arxiv.org/html/2608.15008v1/subMem_K.png)Figure 4:Retrieval breadthkkscales oppositely on long\-context QA versus sequential decision\-making\. Top: LoCoMo \(Qwen3\-8B,n=1986n\{=\}1986,k∈\{1,2,5,10,20\}k\{\\in\}\\\{1,2,5,10,20\\\}\)\. Bottom: ALFWorld\-unseen \(Qwen3\-32B\-AWQ,n=134n\{=\}134,k∈\{1,2,3,4,5\}k\{\\in\}\\\{1,2,3,4,5\\\}\)\.greenwhen the trend is in the preferred direction,redotherwise\. No Mem stands for no memory baseline\.

## 5Discussion

### 5\.1Performance–Latency Tradeoffs Across Substrate Families

Figure[3](https://arxiv.org/html/2608.15008#S4.F3)plots every memory substrate against its per\-query latency under a shared Qwen3\-32B\-AWQ backbone\. Panels \(a, b\) report absoluteP4P\_\{4\}on long\-form dialog QA; panels \(c, d\) re\-anchor at NoMem so the y\-axis becomes the gain over no\-memory and the x\-axis the latency overheadE15/E15NoMemE\_\{15\}/E\_\{15\}^\{\\mathrm\{NoMem\}\}\.

Disjoint Pareto frontiers across regimes\.The QA frontier \(a, b\) is occupied by structural and hierarchical memories \(M5, M6\): M5 wins at1010to30×30\\timesthe latency of the baselines, while M6’s plotted latency is dominated by a one\-off tree build absorbed by the first query \(Appendix[E](https://arxiv.org/html/2608.15008#A5)\); its steady\-state per\-query cost is comparable to the Flat baselines, while the agentic frontier \(c, d\) is occupied by a disjoint set: flat retrieval \(M2 on BCB\-Hard,\+2\.0\+2\.0% at1\.28×1\.28\\timesoverhead\) and refinement\-based distillation \(M7 on ALFWorld,\+9\.7\+9\.7% at1\.23×1\.23\\times\)\. That the two frontiers share*no*substrate confirms the divide of Section[4\.2](https://arxiv.org/html/2608.15008#S4.SS2): the regimes reward fundamentally different operations, recall over a stored corpus versus concise procedural cues, so a substrate engineered for one provides no advantage at the other\.

Cost is heavy\-tailed and frequently unrewarded\.Additional latency rarely buys quality monotonically: on LME\-S, M4 at234234s underperforms M2 at7\.37\.3s, and M9 at172172s buys little quality \(P4=0\.29P\_\{4\}\{=\}0\.29\)\. The asymmetry sharpens on agentic tasks, where M5, M9, and M11 incur3\.73\.7to8\.08\.0times overhead on BCB\-Hard while landing strictly below NoMem \(Δ\\DeltaPass@1 between−1\.4\-1\.4% and−4\.1\-4\.1%\)\. Substrate cost therefore reflects mechanism complexity rather than expected gain: heavy machinery earns its overhead only when its mechanism aligns with the task bottleneck, and turns into pure tax otherwise\. Family labels predict performance only weakly \(Flat is mediocre on QA yet leads BCB\-Hard; Refine spans M7 best on ALFWorld and M8 near the bottom on two panels\), ruling out family\-level routing and motivating the per\-substrate analysis we develop next\.

### 5\.2Retrieval Scaling Reverses Across Regimes

Figure[4](https://arxiv.org/html/2608.15008#S4.F4)sweeps retrieval depthkkon LoCoMo withQwen3\-8Band on ALFWorld withQwen3\-32B\-AWQ\. On LoCoMo,P4P\_\{4\}rises monotonically withkkacross all substrates: each retrieval raises the chance of surfacing a gold evidence turn, and surplus context is passively ignorable because factual QA only requires the model to*find and extract*\. On ALFWorld the sign flips: task success*falls*withkk\(M7 drops from32\.1%32\.1\\%atk=1k\{=\}1to∼25%\{\\sim\}25\\%atk=5k\{=\}5, while the flat retrievers M1 and M2 fall below the no\-memory baseline of22\.4%22\.4\\%\), and steps\-to\-goal climbs toward the cap \(panel f\), confirming that over\-retrieved agents do not simply fail, they wander\. Because the cross\-substrate trend \(black band\) flips sign between panels, retrieval breadth itself is the lever, not any individual substrate\.

![Refer to caption](https://arxiv.org/html/2608.15008v1/attn_region_heatmap.png)Figure 5:Response\-token attention under M1 retrievalfor \(a\) LoCoMo \(b\) ALFWorld\.Attention dilution is universal, but its consequence depends on where the answer lives\.As shown in Figure[5](https://arxiv.org/html/2608.15008#S5.F5), we probe the last prompt token ofQwen3\-8B\(eager attention, upper\-half layers, head\-summed mass\) under M1 retrieval and bin attention into four regions: System, Retrieved, task Context \(transcript, trajectory, observation, admissible actions\), and Cue \(question and next\-action prompt\)\. On*both*tasks, growingkkfunnels mass out of Context into Retrieved \(LoCoMo:→0\.100\.34\\\!\\to\\\!0\.10vs\.→0\.660\.05\\\!\\to\\\!0\.66; ALFWorld:→0\.170\.19\\\!\\to\\\!0\.17vs\.→0\.440\.34\\\!\\to\\\!0\.44\): the mechanism is identical, what differs is which region carries the answer\. On LoCoMo the load\-bearing facts live in the retrieved utterances, so siphoning attention toward them is precisely the desired behavior andP4P\_\{4\}rises\. On ALFWorld the load\-bearing region is the observation and admissible\-action list, so the same shift starves the decision and TSR falls; Cue mass also loses≈4\\approx 4% betweenk=1k\{=\}1andk=5k\{=\}5, enough to misroute the next action when the cue encodes the only valid move\. The implication for memory design is that retrieval breadth is not a universal hyperparameter but a regime\-conditioned one: the routing decision must be informed by whether the prompt’s answer lives in retrieved memory or in the task context window\.

### 5\.3Scalability Under Growing Input Context

Figure 6:MAB CR scalability onQwen3\-32B\-AWQ:P4P\_\{4\}\(bars, left\) and per\-query latency \(lines, right, log\) at 6K, 32K, 262K tokens\.Scalability is a third routing axis: substrates that win at moderate scale can become infeasible at long horizons, and vice versa\.Figure[6](https://arxiv.org/html/2608.15008#S5.F6)sweeps content length on MAB Conflict Resolution from 6K to 262K tokens\. Most external substrates improve with scale: M8 climbs from0\.320\.32to0\.510\.51and M5 from0\.280\.28to0\.480\.48, because longer histories give their write\-side abstraction and entity\-relation updates more material to disambiguate which fact is current\. But the cost dynamics decompose the substrates into two classes\. The refinement family couples strong quality scaling with moderate latency growth \(M8:∼1\{\\sim\}1s→∼10\{\\to\}\{\\sim\}10s\) because its stored representation is a fixed\-size skill bundle whose read cost is independent of input length\. The structural family \(M4, M5\) pays a steep latency price at 262K as graph rebuild and entity extraction scale with input size, eroding the quality lead M5 enjoyed at moderate scale\. The internal substrates diverge sharply along the same axis: M10 scales well in quality \(0\.27→0\.470\.27\{\\to\}0\.47\) because longer context gives the model more evidence to resolve contradictions in\-place but pays linear read\-time cost; M9 remains flat at 0\.18–0\.22, lacking any mechanism for selective temporal update and so poorly suited to recency\-sensitive conflict resolution\.Universal memory must route on history depth, not only on the current query\.Beyond regime \(Section[5\.1](https://arxiv.org/html/2608.15008#S5.SS1)\) and retrieval depth \(Section[5\.2](https://arxiv.org/html/2608.15008#S5.SS2)\), substrates that amortize write cost into compact representations \(M7, M8\) hold their quality\-per\-second as histories grow, while those whose read or rebuild cost scales with input length \(M4, M5, M10\) face hard deployment limits\. A substrate optimal for short sessions can thus become a liability over a long\-running agent’s lifetime\.

No Substrate Wins Alone: Composition Is the Path to Universal MemoryThe three diagnostic axes of Sections[5\.1](https://arxiv.org/html/2608.15008#S5.SS1)–[5\.3](https://arxiv.org/html/2608.15008#S5.SS3)converge: no single substrate can simultaneously serve recall over long histories, action selection under attention pressure, and graceful scaling to long horizons, because these regimes reward qualitatively different operations\. The path forward is not a better single substrate but a*multi\-substrate or multi\-agent memory system*composed by role —refinementsubstrates curate*abstractions*,structuralandtextsubstrates hold*facts*,activationandflatsubstrates preserve raw*experience*— with the harness routing each query to the substrate whose representation aligns with the load\-bearing region of its prompt\.

## 6Related Work

#### Memory\-augmented LLM agents\.

Agent memory systems span a broad substrate design space, from external stores \(vector indices, text buffers, knowledge graphs, hierarchies, distilled skill memories\[[39](https://arxiv.org/html/2608.15008#bib.bib4),[36](https://arxiv.org/html/2608.15008#bib.bib14),[3](https://arxiv.org/html/2608.15008#bib.bib13),[40](https://arxiv.org/html/2608.15008#bib.bib17),[17](https://arxiv.org/html/2608.15008#bib.bib25),[45](https://arxiv.org/html/2608.15008#bib.bib19),[49](https://arxiv.org/html/2608.15008#bib.bib18),[79](https://arxiv.org/html/2608.15008#bib.bib21)\]\) to internal stores \(parametric updates\[[80](https://arxiv.org/html/2608.15008#bib.bib22)\]and KV\-cache activations\[[26](https://arxiv.org/html/2608.15008#bib.bib23),[20](https://arxiv.org/html/2608.15008#bib.bib24)\]\), with hybrids combining several\[[27](https://arxiv.org/html/2608.15008#bib.bib7),[22](https://arxiv.org/html/2608.15008#bib.bib26),[63](https://arxiv.org/html/2608.15008#bib.bib27)\]\. Most systems commit to a fixed configuration at design time; we instead isolate the substrate as the experimental variable and identify when each is preferable\.

#### Memory evaluation benchmarks and practices\.

Existing benchmarks cover user\-centric\[[30](https://arxiv.org/html/2608.15008#bib.bib9),[65](https://arxiv.org/html/2608.15008#bib.bib2),[12](https://arxiv.org/html/2608.15008#bib.bib28)\]and agent\-centric settings\[[47](https://arxiv.org/html/2608.15008#bib.bib31),[86](https://arxiv.org/html/2608.15008#bib.bib32),[18](https://arxiv.org/html/2608.15008#bib.bib33)\], but the two regimes are typically evaluated in isolation, and evaluations emphasize end\-task accuracy while omitting deployment\-critical dimensions such as write cost, latency, storage, and management overhead\. Our harness addresses both gaps; details are in Appendix[F](https://arxiv.org/html/2608.15008#A6)\.

## 7Conclusion

We presented a controlled harness evaluation of agent memory across 11 substrates, three backbones, four benchmarks, and 26 metrics\. The optimal substrate*reverses*between long\-form QA and agentic decision\-making and shifts again with history depth, making substrate routing necessary rather than optional\.

## References

- \[1\]L\. C\. Borro, L\. A\. Macarini, G\. Tindall, M\. Montero, and A\. B\. Struck\(2026\)Memori: a persistent memory layer for efficient, context\-aware llm agents\.arXiv preprint arXiv:2603\.19935\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.25.8)\.
- \[2\]N\. D\. Bui\(2026\)Building effective ai coding agents for the terminal: scaffolding, harness, context engineering, and lessons learned\.arXiv preprint arXiv:2603\.05344\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p2.1)\.
- \[3\]P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav\(2025\)Mem0: building production\-ready ai agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.2.8),[§B\.3](https://arxiv.org/html/2608.15008#A2.SS3.p1.1),[Table 5](https://arxiv.org/html/2608.15008#A2.T5.2.3.2.1),[§D\.2](https://arxiv.org/html/2608.15008#A4.SS2.SSS0.Px4.p1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px3.p2.1),[Appendix G](https://arxiv.org/html/2608.15008#A7.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[4\]P\. Daftari, K\. Patel, S\. Kapale, J\. George, and S\. Surendira\(2026\)Cognis: context\-aware memory for conversational ai agents\.arXiv preprint arXiv:2604\.19771\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.26.1)\.
- \[5\]Y\. Du, B\. Wang, Y\. Xiang, Z\. Wang, W\. Huang, B\. Xue, B\. Liang, X\. Zeng, F\. Mi, H\. Bai,et al\.\(2025\)Memory\-t1: reinforcement learning for temporal reasoning in multi\-session agents\.arXiv preprint arXiv:2512\.20092\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px3.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.9.8)\.
- \[6\]J\. Fang, X\. Deng, H\. Xu, Z\. Jiang, Y\. Tang, Z\. Xu, S\. Deng, Y\. Yao, M\. Wang, S\. Qiao,et al\.\(2025\)Lightmem: lightweight and efficient memory\-augmented generation\.arXiv preprint arXiv:2510\.18866\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.22.1)\.
- \[7\]Z\. Fountas, M\. Benfeghoul, A\. Oomerjee, F\. Christopoulou, G\. Lampouras, H\. B\. Ammar, and J\. Wang\(2025\)Human\-inspired episodic memory for infinite context llms\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px1.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.9.1)\.
- \[8\]Z\. Guo, L\. Xia, Y\. Yu, T\. Ao, and C\. Huang\(2024\)Lightrag: simple and fast retrieval\-augmented generation\.arXiv preprint arXiv:2410\.057792\(3\)\.Cited by:[§B\.1](https://arxiv.org/html/2608.15008#A2.SS1.SSS0.Px5.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.7.2.1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1)\.
- \[9\]B\. J\. Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. Su\(2024\)Hipporag: neurobiologically inspired long\-term memory for large language models\.Advances in neural information processing systems37,pp\. 59532–59569\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px3.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.10.1),[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[10\]B\. J\. Gutiérrez, Y\. Shu, W\. Qi, S\. Zhou, and Y\. Su\(2025\)From rag to memory: non\-parametric continual learning for large language models\.InInternational Conference on Machine Learning,pp\. 21497–21515\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.21.1)\.
- \[11\]C\. Hu, X\. Gao, Z\. Zhou, D\. Xu, Y\. Bai, X\. Li, H\. Zhang, T\. Li, C\. Zhang, L\. Bing,et al\.\(2026\)EverMemOS: a self\-organizing memory operating system for structured long\-horizon reasoning\.arXiv preprint arXiv:2601\.02163\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.25.1)\.
- \[12\]Y\. Hu, Y\. Wang, and J\. McAuley\(2025\)Evaluating memory in llm agents via incremental multi\-turn interactions\.arXiv preprint arXiv:2507\.05257\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2608.15008#S3.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px2.p1.1)\.
- \[13\]Y\. Hu, J\. Liu, J\. Tan, Y\. Zhu, and Z\. Dou\(2026\)Memory matters more: event\-centric memory as a logic map for agent searching and reasoning\.arXiv preprint arXiv:2601\.04726\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.26.8)\.
- \[14\]Y\. Hu, S\. Liu, Y\. Yue, G\. Zhang, B\. Liu, F\. Zhu, J\. Lin, H\. Guo, S\. Dou, Z\. Xi,et al\.\(2025\)Memory in the age of ai agents\.arXiv preprint arXiv:2512\.13564\.Cited by:[Appendix A](https://arxiv.org/html/2608.15008#A1.p1.1)\.
- \[15\]W\. Huang, W\. Zhang, Y\. Liang, Y\. Bei, Y\. Chen, T\. Feng, X\. Pan, Z\. Tan, Y\. Wang, T\. Wei,et al\.\(2026\)Rethinking memory mechanisms of foundation agents in the second half\.arXiv preprint arXiv:2602\.06052\.Cited by:[Appendix A](https://arxiv.org/html/2608.15008#A1.p1.1),[§1](https://arxiv.org/html/2608.15008#S1.p4.1)\.
- \[16\]Z\. Huang, Z\. Tian, Q\. Guo, F\. Zhang, Y\. Zhou, D\. Jiang, Z\. Xie, and X\. Zhou\(2025\)Licomemory: lightweight and cognitive agentic memory for efficient long\-term reasoning\.arXiv preprint arXiv:2511\.01448\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.23.1)\.
- \[17\]D\. Jiang, Y\. Li, G\. Li, and B\. Li\(2026\)MAGMA: a multi\-graph based agentic memory architecture for ai agents\.arXiv preprint arXiv:2601\.03236\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.27.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[18\]C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan\(2023\)Swe\-bench: can language models resolve real\-world github issues?\.arXiv preprint arXiv:2310\.06770\.Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p2.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px2.p1.1)\.
- \[19\]J\. Kang, M\. Ji, Z\. Zhao, and T\. Bai\(2025\)Memory os of ai agent\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 25972–25981\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.7.8)\.
- \[20\]M\. Kim, A\. Kundu, H\. Kim, R\. Dixit, and M\. Cho\(2025\)Epicache: episodic kv cache management for long conversational question answering\.arXiv preprint arXiv:2509\.17396\.Cited by:[§B\.2](https://arxiv.org/html/2608.15008#A2.SS2.SSS0.Px3.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.14.2.1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[21\]N\. Kim, K\. T\. Ong, Y\. Hwang, M\. Kang, I\. Jihn, G\. Kim, M\. Kim, and J\. Yeo\(2025\)PRINCIPLES: synthetic strategy memory for proactive dialogue agents\.InThe 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP 2025\),Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.14.8)\.
- \[22\]C\. Latimer, N\. Boschi, A\. Neeser, C\. Bartholomew, G\. Srivastava, X\. Wang, and N\. Ramakrishnan\(2025\)Hindsight is 20/20: building agent memory that retains, recalls, and reflects\.arXiv preprint arXiv:2512\.12818\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.20.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[23\]K\. Lee, X\. Chen, H\. Furuta, J\. Canny, and I\. Fischer\(2024\)A human\-inspired reading agent with gist memory of very long contexts\.InForty\-first International Conference on Machine Learning,Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.14.1),[§B\.1](https://arxiv.org/html/2608.15008#A2.SS1.SSS0.Px3.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.5.2.1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1)\.
- \[24\]H\. Li, C\. Yang, A\. Zhang, Y\. Deng, X\. Wang, and T\. Chua\(2025\)Hello again\! llm\-powered personalized agent for long\-term dialogue\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 5259–5276\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.11.1)\.
- \[25\]Q\. Li, S\. Dou, K\. Shao, C\. Chen, and H\. Hu\(2025\)Evaluating scoring bias in llm\-as\-a\-judge\.arXiv preprint arXiv:2506\.22316\.Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px3.p2.1)\.
- \[26\]Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. Chen\(2024\)Snapkv: llm knows what you are looking for before generation\.Advances in Neural Information Processing Systems37,pp\. 22947–22970\.Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1),[§1](https://arxiv.org/html/2608.15008#S1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[27\]Z\. Li, C\. Xi, C\. Li, D\. Chen, B\. Chen, S\. Song, S\. Niu, H\. Wang, J\. Yang, C\. Tang,et al\.\(2025\)Memos: a memory os for ai system\.arXiv preprint arXiv:2507\.03724\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.5.8),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1),[§1](https://arxiv.org/html/2608.15008#S1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[28\]K\. Lin, C\. Snell, Y\. Wang, C\. Packer, S\. Wooders, I\. Stoica, and J\. E\. Gonzalez\(2025\)Sleep\-time compute: beyond inference scaling at test\-time\.arXiv preprint arXiv:2504\.13171\.Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1)\.
- \[29\]J\. Lu, S\. An, M\. Lin, G\. Pergola, Y\. He, D\. Yin, X\. Sun, and Y\. Wu\(2023\)Memochat: tuning llms to use memos for consistent long\-range open\-domain conversation\.arXiv preprint arXiv:2308\.08239\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.4.1)\.
- \[30\]A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang\(2024\)Evaluating very long\-term conversational memory of llm agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 13851–13870\.Cited by:[§A\.1](https://arxiv.org/html/2608.15008#A1.SS1.p2.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2608.15008#S1.p3.1),[§3](https://arxiv.org/html/2608.15008#S3.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px2.p1.1)\.
- \[31\]C\. Miao, H\. P\. Zou, Y\. Li, Y\. Chen, Y\. Wang, F\. Wang, Y\. Li, W\. Yang, B\. He, X\. Zhang,et al\.\(2025\)Recode\-h: a benchmark for research code development with interactive human feedback\.arXiv preprint arXiv:2510\.06186\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[32\]J\. Nan, W\. Ma, W\. Wu, and Y\. Chen\(2025\)Nemori: self\-organizing agent memory inspired by cognitive science\.arXiv preprint arXiv:2508\.03341\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.12.8)\.
- \[33\]Y\. Ning and H\. Liu\(2024\)Urbankgent: a unified large language model agent framework for urban knowledge graph construction\.Advances in Neural Information Processing Systems37,pp\. 123127–123154\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[34\]K\. T\. Ong, N\. Kim, M\. Gwak, H\. Chae, T\. Kwon, Y\. Jo, S\. Hwang, D\. Lee, and J\. Yeo\(2025\)Towards lifelong dialogue agents via timeline\-based memory management\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 8631–8661\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.15.1)\.
- \[35\]S\. Ouyang, J\. Yan, I\. Hsu, Y\. Chen, K\. Jiang, Z\. Wang, R\. Han, L\. T\. Le, S\. Daruki, X\. Tang,et al\.\(2025\)Reasoningbank: scaling agent self\-evolving with reasoning memory\.arXiv preprint arXiv:2509\.25140\.Cited by:[§B\.1](https://arxiv.org/html/2608.15008#A2.SS1.SSS0.Px7.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.9.2.1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1)\.
- \[36\]C\. Packer, V\. Fang, S\. Patil, K\. Lin, S\. Wooders, and J\. Gonzalez\(2023\)MemGPT: towards llms as operating systems\.\.ArXiv\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.3.1),[§B\.3](https://arxiv.org/html/2608.15008#A2.SS3.p1.1),[Table 5](https://arxiv.org/html/2608.15008#A2.T5.2.2.2.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p1.1),[Appendix G](https://arxiv.org/html/2608.15008#A7.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[37\]L\. Pan, L\. Zou, S\. Guo, J\. Ni, and H\. Zheng\(2026\)Natural\-language agent harnesses\.arXiv preprint arXiv:2603\.25723\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p2.1)\.
- \[38\]Z\. Pan, Q\. Wu, H\. Jiang, X\. Luo, H\. Cheng, D\. Li, Y\. Yang, C\. Lin, H\. V\. Zhao, L\. Qiu,et al\.\(2025\)SeCom: on memory construction and retrieval for personalized conversational agents\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.19.8)\.
- \[39\]J\. S\. Park, J\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein\(2023\)Generative agents: interactive simulacra of human behavior\.InProceedings of the 36th annual acm symposium on user interface software and technology,pp\. 1–22\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.2.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[40\]P\. Rasmussen, P\. Paliychuk, T\. Beauvais, J\. Ryan, and D\. Chalef\(2025\)Zep: a temporal knowledge graph architecture for agent memory\.arXiv preprint arXiv:2501\.13956\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.22.8),[§B\.3](https://arxiv.org/html/2608.15008#A2.SS3.p1.1),[Table 5](https://arxiv.org/html/2608.15008#A2.T5.2.4.2.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px3.p1.1),[Appendix G](https://arxiv.org/html/2608.15008#A7.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[41\]S\. Ren, P\. Jian, Z\. Ren, C\. Leng, C\. Xie, and J\. Zhang\(2025\)Towards scientific intelligence: a survey of llm\-based scientific agents\.ArXivabs/2503\.24047\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[42\]A\. Rezazadeh, Z\. Li, W\. Wei, and Y\. Bao\(2024\)From isolated conversations to hierarchical schemas: dynamic tree memory representation for llms\.arXiv preprint arXiv:2410\.14052\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.12.1)\.
- \[43\]S\. Robertson and H\. Zaragoza\(2009\)The probabilistic relevance framework: BM25 and beyond\.Foundations and Trends in Information Retrieval3,pp\. 333–389\.Cited by:[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.4.2.1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1)\.
- \[44\]S\. Sarin, L\. Singh, B\. Sarmah, and D\. Mehta\(2025\)Memoria: a scalable agentic memory framework for personalized conversational ai\.In2025 5th International Conference on AI\-ML\-Systems \(AIMLSystems\),pp\. 32–39\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px2.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.6.8)\.
- \[45\]P\. Sarthi, S\. Abdullah, A\. Tuli, S\. Khanna, A\. Goldie, and C\. D\. Manning\(2024\)Raptor: recursive abstractive processing for tree\-organized retrieval\.InThe Twelfth International Conference on Learning Representations,Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.13.1),[§B\.1](https://arxiv.org/html/2608.15008#A2.SS1.SSS0.Px6.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.8.2.1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[46\]Y\. Shen, D\. Pei, Y\. Guo, J\. Wang, Y\. Guo, Z\. Zhang, Q\. Jia, J\. Zhou, and G\. Zhai\(2026\)EvolMem: a cognitive\-driven benchmark for multi\-session dialogue memory\.arXiv preprint arXiv:2601\.03543\.Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p1.1)\.
- \[47\]M\. Shridhar, X\. Yuan, M\. Côté, Y\. Bisk, A\. Trischler, and M\. Hausknecht\(2020\)Alfworld: aligning text and embodied environments for interactive learning\.arXiv preprint arXiv:2010\.03768\.Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p2.1),[§3](https://arxiv.org/html/2608.15008#S3.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px2.p1.1)\.
- \[48\]M\. Su, Y\. Guo, Z\. Hou, L\. Bai, Z\. Li, Y\. Zhang, G\. Yin, W\. Lin, X\. Jin, J\. Guo,et al\.\(2026\)Beyond dialogue time: temporal semantic memory for personalized llm agents\.arXiv preprint arXiv:2601\.07468\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.28.8)\.
- \[49\]H\. Sun, S\. Zeng, and B\. Zhang\(2026\)H\-mem: hierarchical memory for high\-efficiency long\-term reasoning in llm agents\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 341–350\.Cited by:[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.8.2.1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[50\]H\. Tan, Z\. Zhang, C\. Ma, X\. Chen, Q\. Dai, and Z\. Dong\(2025\)Membench: towards more comprehensive evaluation on the memory of llm\-based agents\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 19336–19352\.Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p1.1)\.
- \[51\]Z\. Tan, J\. Yan, I\. Hsu, R\. Han, Z\. Wang, L\. Le, Y\. Song, Y\. Chen, H\. Palangi, G\. Lee,et al\.\(2025\)In prospect and retrospect: reflective memory management for long\-term personalized dialogue agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8416–8439\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.18.8)\.
- \[52\]A\. Tian, Y\. Li, X\. Li, H\. Zhen, L\. Chen, X\. Yu, Z\. Dong, and M\. Yuan\(2026\)SwiftMem: fast agentic memory via query\-aware indexing\.arXiv preprint arXiv:2601\.08160\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.27.8)\.
- \[53\]A\. Tian, Y\. Lu, X\. Fan, C\. Wang, L\. Zhou, Y\. Zhang, and Y\. Liu\(2025\)Rgmem: renormalization group\-based memory evolution for language agent user profile\.arXiv preprint arXiv:2510\.16392\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.17.8)\.
- \[54\]G\. Vijayaraghavan, P\. Jayachandran, A\. Murthy, S\. Govindan, and V\. Subramanian\(2026\)If you want coherence, orchestrate a team of rivals: multi\-agent models of organizational intelligence\.arXiv preprint arXiv:2601\.14351\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p2.1)\.
- \[55\]B\. Wang, X\. Liang, J\. Yang, H\. Huang, Z\. Wu, S\. Wu, Z\. Ma, and Z\. Li\(2025\)Scm: enhancing large language model with self\-controlled memory framework\.InInternational Conference on Database Systems for Advanced Applications,pp\. 188–203\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.7.1)\.
- \[56\]J\. Wang and J\. Han\(2025\)Proprag: guiding retrieval with beam search over proposition paths\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 6223–6238\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px3.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.15.8)\.
- \[57\]J\. Wang, R\. Zhao, W\. Wei, Y\. Wang, M\. Yu, J\. Zhou, J\. Xu, and L\. Xu\(2026\)Comorag: a cognitive\-inspired memory\-organized rag for stateful long narrative reasoning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 33557–33565\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.17.1)\.
- \[58\]P\. Wang, M\. Tian, J\. Li, Y\. Liang, Y\. Wang, Q\. Chen, T\. Wang, Z\. Lu, J\. Ma, Y\. E\. Jiang,et al\.\(2025\)O\-mem: omni memory system for personalized, long horizon, self\-evolving agents\.arXiv preprint arXiv:2511\.13593\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.13.8)\.
- \[59\]Q\. Wang, Y\. Fu, Y\. Cao, S\. Wang, Z\. Tian, and L\. Ding\(2025\)Recursively summarizing enables long\-term dialogue memory in large language models\.Neurocomputing639,pp\. 130193\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.6.1)\.
- \[60\]S\. Wang, E\. Yu, O\. Love, T\. Zhang, T\. Wong, S\. Scargall, and C\. Fan\(2026\)MemMachine: a ground\-truth\-preserving memory system for personalized ai agents\.arXiv preprint arXiv:2604\.04853\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px4.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.24.8)\.
- \[61\]T\. Wang, M\. Tao, R\. Fang, H\. Wang, S\. Wang, Y\. E\. Jiang, and W\. Zhou\(2024\)Ai persona: towards life\-long personalization of llms\.arXiv preprint arXiv:2412\.13103\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.8.1)\.
- \[62\]X\. Wang, S\. Wang, Y\. Zhu, and B\. Liu\(2025\)R3mem: bridging memory retention and retrieval via reversible compression\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 4541–4557\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px3.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.16.8)\.
- \[63\]Y\. Wang and X\. Chen\(2025\)Mirix: multi\-agent memory system for llm\-based agents\.arXiv preprint arXiv:2507\.07957\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.10.8),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[64\]Y\. Wang, R\. Takanobu, Z\. Liang, Y\. Mao, Y\. Hu, J\. McAuley, and X\. Wu\(2025\)Mem\-α\\alpha: learning memory construction via reinforcement learning\.arXiv preprint arXiv:2509\.25911\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px1.p1.1),[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px3.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.3.8)\.
- \[65\]D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu\(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2608.15008#A1.SS1.p2.1),[1st item](https://arxiv.org/html/2608.15008#A3.I1.i1.p1.1),[§D\.2](https://arxiv.org/html/2608.15008#A4.SS2.SSS0.Px4.p1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2608.15008#S1.p3.1),[§3](https://arxiv.org/html/2608.15008#S3.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px2.p1.1)\.
- \[66\]Y\. Wu, Y\. Zhang, S\. Liang, and Y\. Liu\(2025\)Sgmem: sentence graph memory for long\-term conversational agents\.arXiv preprint arXiv:2509\.21212\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.20.8)\.
- \[67\]Z\. Xi, Y\. Ding, W\. Chen, B\. Hong, H\. Guo, J\. Wang, X\. Guo, D\. Yang, C\. Liao, W\. He,et al\.\(2025\)Agentgym: evaluating and training large language model\-based agents across diverse environments\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 27914–27961\.Cited by:[§C\.1](https://arxiv.org/html/2608.15008#A3.SS1.SSS0.Px3.p1.1),[§C\.2](https://arxiv.org/html/2608.15008#A3.SS2.SSS0.Px1.p1.1)\.
- \[68\]L\. Xiong, C\. Xiong, Y\. Li, K\. Tang, J\. Liu, P\. N\. Bennett, J\. Ahmed, and A\. Overwijk\(2021\)Approximate nearest neighbor negative contrastive learning for dense text retrieval\.InInternational Conference on Learning Representations,Cited by:[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.3.2.1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1)\.
- \[69\]Z\. Xiong, Y\. Lin, W\. Xie, P\. He, Z\. Liu, J\. Tang, H\. Lakkaraju, and Z\. Xiang\(2025\)How memory management impacts llm agents: an empirical study of experience\-following behavior\.arXiv preprint arXiv:2505\.16067\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[70\]W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang\(2025\)A\-mem: agentic memory for llm agents\.arXiv preprint arXiv:2502\.12110\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.16.1),[§B\.1](https://arxiv.org/html/2608.15008#A2.SS1.SSS0.Px4.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.6.2.1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1)\.
- \[71\]B\. Yan, C\. Li, H\. Qian, S\. Lu, and Z\. Liu\(2025\)General agentic memory via deep research\.arXiv preprint arXiv:2511\.18423\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.18.1),[§1](https://arxiv.org/html/2608.15008#S1.p2.1)\.
- \[72\]S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan,et al\.\(2025\)Memory\-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.arXiv preprint arXiv:2508\.19828\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px3.p1.1),[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px4.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.8.8)\.
- \[73\]S\. Yao, N\. Shinn, P\. Razavi, and K\. R\. Narasimhan\(2025\)τ\\tau\-bench: a benchmark for\\\\backslashunderline\{\\\{t\}\\\}ool\-\\\\backslashunderline\{\\\{a\}\\\}gent\-\\\\backslashunderline\{\\\{u\}\\\}ser interaction in real\-world domains\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p1.1)\.
- \[74\]Z\. Ye, J\. Huang, W\. Chen, and Y\. Zhang\(2026\)H\-mem: hybrid multi\-dimensional memory management for long\-context conversational agents\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 7756–7775\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.19.1)\.
- \[75\]H\. Yu, T\. Chen, J\. Feng, J\. Chen, W\. Dai, Q\. Yu, Y\. Zhang, W\. Ma, J\. Liu, M\. Wang,et al\.\(2025\)Memagent: reshaping long\-context llm with multi\-conv rl\-based memory agent\.arXiv preprint arXiv:2507\.02259\.Cited by:[§A\.2](https://arxiv.org/html/2608.15008#A1.SS2.SSS0.Px3.p1.1),[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.4.8)\.
- \[76\]Y\. Yu, L\. Yao, Y\. Xie, Q\. Tan, J\. Feng, Y\. Li, and L\. Wu\(2026\)Agentic memory: learning unified long\-term and short\-term memory management for large language model agents\.arXiv preprint arXiv:2601\.01885\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.24.1)\.
- \[77\]G\. Zhang, B\. Wang, Y\. Ma, D\. Zhao, and Z\. Yu\(2025\)Multiple memory systems for enhancing the long\-term memory of agent\.arXiv e\-prints,pp\. arXiv–2508\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.11.8)\.
- \[78\]G\. Zhang, H\. Ren, C\. Zhan, Z\. Zhou, J\. Wang, H\. Zhu, W\. Zhou, and S\. Yan\(2025\)Memevolve: meta\-evolution of agent memory systems\.arXiv preprint arXiv:2512\.18746\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p2.1)\.
- \[79\]H\. Zhang, Q\. Long, J\. Bao, T\. Feng, W\. Zhang, H\. Yue, and W\. Wang\(2026\)MemSkill: learning and evolving memory skills for self\-evolving agents\.arXiv preprint arXiv:2602\.02474\.Cited by:[§B\.1](https://arxiv.org/html/2608.15008#A2.SS1.SSS0.Px8.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.10.2.1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[80\]L\. Zhang, L\. Zhang, S\. Shi, X\. Chu, and B\. Li\(2023\)Lora\-fa: memory\-efficient low\-rank adaptation for large language models fine\-tuning\.arXiv preprint arXiv:2308\.03303\.Cited by:[§B\.2](https://arxiv.org/html/2608.15008#A2.SS2.SSS0.Px1.p1.1),[Table 4](https://arxiv.org/html/2608.15008#A2.T4.18.12.2.1.1),[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2608.15008#S2.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px1.p1.1)\.
- \[81\]Y\. Zhang, H\. Ruan, Z\. Fan, and A\. Roychoudhury\(2024\)Autocoderover: autonomous program improvement\.InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis,pp\. 1592–1604\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[82\]W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang\(2024\)Memorybank: enhancing large language models with long\-term memory\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 19724–19731\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.5.1),[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[83\]S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried,et al\.\(2024\)WebArena: a realistic web environment for building autonomous agents\.InThe Twelfth International Conference on Learning Representations,Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[84\]S\. Zhou and J\. Han\(2025\)A simple yet strong baseline for long\-term conversational memory of llm agents\.arXiv preprint arXiv:2511\.17208\.Cited by:[Table 3](https://arxiv.org/html/2608.15008#A1.T3.12.1.21.8)\.
- \[85\]Z\. Zhou, A\. Qu, Z\. Wu, S\. Kim, A\. Prakash, D\. Rus, J\. Zhao, B\. K\. H\. Low, and P\. P\. Liang\(2025\)Mem1: learning to synergize memory and reasoning for efficient long\-horizon agents\.arXiv preprint arXiv:2506\.15841\.Cited by:[§1](https://arxiv.org/html/2608.15008#S1.p1.1)\.
- \[86\]T\. Y\. Zhuo, V\. M\. Chien, J\. Chim, H\. Hu, W\. Yu, R\. Widyasari, I\. N\. B\. Yusuf, H\. Zhan, J\. He, I\. Paul,et al\.\(2025\)BigCodeBench: benchmarking code generation with diverse function calls and complex instructions\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Appendix F](https://arxiv.org/html/2608.15008#A6.SS0.SSS0.Px2.p2.1),[§3](https://arxiv.org/html/2608.15008#S3.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2608.15008#S6.SS0.SSS0.Px2.p1.1)\.

## Appendix ALandscape of LLM Memory\-Augmented Systems

This appendix expands the motivation in Section[1](https://arxiv.org/html/2608.15008#S1)with the full survey of52 memory\-augmented LLM systemsreleased between early 2023 and April 2026\. We document not the mechanisms of individual systems, which are catalogued by the recent surveys of agentic memory\[[15](https://arxiv.org/html/2608.15008#bib.bib11),[14](https://arxiv.org/html/2608.15008#bib.bib37)\], but how the current work evaluates them\. The result is shown in Table[3](https://arxiv.org/html/2608.15008#A1.T3)\.

### A\.1Inclusion criteria and data collection

From this collection, we included papers that \(a\) propose a novel memory method with an associated substrate or architecture, \(b\) report evaluation on at least one memory benchmark, and \(c\) have a publicly available preprint\. We excluded survey papers, benchmark\-only papers, and systems assessed only qualitatively\. For each candidate, we independently verified the arXiv identifier, title, publication year, and evaluation setting against primary sources, discarding entries whose benchmarks or backbone models could not be reliably confirmed\.

For each system we record four evaluation dimensions\. \(i\)Benchmarks, grouped into six canonical categories:LC=LoCoMo\[[30](https://arxiv.org/html/2608.15008#bib.bib9)\],LME=LongMemEval\[[65](https://arxiv.org/html/2608.15008#bib.bib2)\],MH=multi\-hop QA \(HotPotQA, MuSiQue, 2WikiMQA\),DS=DialSim,DMR=DMR,Other=other \(MSC, Carecall, Conversation Chronicles, MT\-Bench\+, Time\-Dialog, RULER, NarrativeQA, PopQA, NQ, LongBench,∞\\infty\-Bench, ESConv, PersuasionForGood, MemoryAgentBench, PersonaMem, custom evaluations, …\)\. \(ii\)Performance metrics\(A=accuracy,P=per\-type breakdown\)\. \(iii\)Efficiency metrics\(T=token usage,L=latency,R=runtime,M=memory footprint,K=API\-call count\); only numeric reports count\. The five\-dimension scheme used here is the most that any prior system reports; the harness in Section[3](https://arxiv.org/html/2608.15008#S3)extends this to 15 efficiency metrics \(E1E\_\{1\}–E15E\_\{15\}\) covering write, read, and management phases separately, together with aggregate token, call, and latency totals\. \(iv\)Base LLM family:Gfor GPT\-series \(GPT\-3\.5/4/4o/4\.1/5, including*\-mini*variants\) orOfor non\-GPT open\-source models \(Llama, Qwen, DeepSeek, …\)\.

### A\.2Aggregate statistics

#### Benchmark concentration\.

Of 76 benchmark–system pairs in the 52 curated papers, LoCoMo accounts for 29 \(38%\) and LongMemEval for 18 \(24%\), together covering 62% of adopted benchmarks\. Adding DMR \(2\) and DialSim \(1\) pushes the dialogue\-centric share to 66%\. The multi\-hop QA cluster contributes 4 pairs, drawn mainly from the HippoRAG/PropRAG lineage\. The residual “Other” category \(22\) is heterogeneous but, on closer inspection, remains overwhelmingly dialogue\-based: MSC, Carecall, Conversation Chronicles, MT\-Bench\+, and PersonaMem are all multi\-session dialogue benchmarks; MemoryAgentBench\[[12](https://arxiv.org/html/2608.15008#bib.bib28)\], adopted as a primary evaluation by Mem\-α\\alpha\[[64](https://arxiv.org/html/2608.15008#bib.bib38)\], is likewise a multi\-turn dialogue memory benchmark that re\-packages QA, classification, and summarization over incremental context\. Only ReadAgent, RAPTOR, and EM\-LLM\[[7](https://arxiv.org/html/2608.15008#bib.bib39)\]target non\-dialogue long\-document settings \(QuALITY, NarrativeQA, LongBench,∞\\infty\-Bench\)\. In consequence, the field has no evaluation data point from agent\-centric task execution, the gap our harness fills with ALFWorld and BigCodeBench\-Hard\.

#### Metric coverage\.

Every system \(100%\) reports end\-task accuracy and 35 \(67%\) also report a per\-type breakdown\. By contrast, only 11 \(21%\) report any efficiency metric: latency \(12%\), token consumption \(10%\), runtime \(4%\), memory footprint \(4%\), and API\-call count \(2%\), with zero reports of management cost or compression ratio\. LightMem\[[6](https://arxiv.org/html/2608.15008#bib.bib10)\]is the sole system to report four efficiency dimensions simultaneously \(token, latency, runtime, and API\-call count\); Mem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\]and Memoria\[[44](https://arxiv.org/html/2608.15008#bib.bib43)\]each report two; the remainder report one or none\.

#### Base\-model concentration\.

42 of 52 systems \(81%\) use a GPT\-family backbone as their primary experimental setting, typically GPT\-4o\-mini for extraction combined with GPT\-4o or GPT\-4\.1\-mini for generation\. The non\-GPT systems span at least four lineages: the HippoRAG / PropRAG / R3Mem family on multi\-hop QA\[[9](https://arxiv.org/html/2608.15008#bib.bib40),[56](https://arxiv.org/html/2608.15008#bib.bib71),[62](https://arxiv.org/html/2608.15008#bib.bib42)\], the RL line including Memory\-R1 and Memory\-T1 on Llama\-3\.1\-8B and Qwen\-2\.5\[[72](https://arxiv.org/html/2608.15008#bib.bib44),[5](https://arxiv.org/html/2608.15008#bib.bib45)\], Mem\-α\\alphaon Qwen3\-4B\[[64](https://arxiv.org/html/2608.15008#bib.bib38)\], and MemAgent on Qwen\-2\.5 with RL\[[75](https://arxiv.org/html/2608.15008#bib.bib46)\]\.

#### Joint coverage\.

Figure[1](https://arxiv.org/html/2608.15008#S1.F1)\(c\) projects each system onto the*benchmark breadth*×\\times*efficiency depth*plane\. 26 of 52 systems \(50%\) occupy the origin cell\(1,0\)\(1,0\), a single benchmark with no efficiency metric\. Only two systems\[[72](https://arxiv.org/html/2608.15008#bib.bib44),[60](https://arxiv.org/html/2608.15008#bib.bib47)\]evaluate on three or more benchmarks, and only LightMem reports three or more efficiency dimensions\. No system occupies the upper\-right region\(≥3,≥3\)\(\{\\geq\}3,\{\\geq\}3\)that would indicate broad coverage together with comprehensive efficiency reporting\.

### A\.3Trends over time

The 2026 cohort does not close the evaluation gap\. Only 2 of 9 \(22%\) report any efficiency metric, statistically indistinguishable from the pre\-2026 cohort\. Per\-type breakdowns, however, have become near\-universal: 9 of 9 2026 systems \(100%\) report them, up from 60% in pre\-2026 work\. The field has converged on fine\-grained accuracy analysis as a standard but treats efficiency as an afterthought, an asymmetry we identify as the single largest obstacle to principled substrate routing of the kind required by universal agent memory\.

Table 3:Landscape of 52 memory\-augmented LLM systems \(2023–2026\)\.Benchmark codes: LC = LoCoMo, LME = LongMemEval, MH = Multi\-hop QA, DS = DialSim, DMR = DMR\. Metric codes: A = accuracy, P = per\-type, T = token, L = latency, R = runtime, M = memory size, K = API calls\.\#B: distinct benchmarks used\.\#E: efficiency dimensions reported \(bold≥2\{\\geq\}2\)\.Base: G = GPT\-series, O = open\-source\.Summary: 81% GPT\-family; 62% of benchmark pairs on LC \+ LME; only 21% report any efficiency metric; no system reaches≥3\{\\geq\}3benchmarks×\\times≥3\{\\geq\}3efficiency dimensions\.SystemYearBench\.Metrics\#B\#EBaseSystemYearBench\.Metrics\#B\#EBaseGenerative Agents\[[39](https://arxiv.org/html/2608.15008#bib.bib4)\]2023OtherA10GMem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\]2025LCA, P, T, L12GMemGPT\[[36](https://arxiv.org/html/2608.15008#bib.bib14)\]2023DMRA10GMem\-α\\alpha\[[64](https://arxiv.org/html/2608.15008#bib.bib38)\]2025OtherA, P, M11OMemoChat\[[29](https://arxiv.org/html/2608.15008#bib.bib50)\]2023OtherA10GMemAgent\[[75](https://arxiv.org/html/2608.15008#bib.bib46)\]2025OtherA10OMemoryBank\[[82](https://arxiv.org/html/2608.15008#bib.bib15)\]2023OtherA10GMemOS\[[27](https://arxiv.org/html/2608.15008#bib.bib7)\]2025LC, LMEA, P20GRSum\[[59](https://arxiv.org/html/2608.15008#bib.bib51)\]2023OtherA10GMemoria\[[44](https://arxiv.org/html/2608.15008#bib.bib43)\]2025LMEA, T, L12GSCM\[[55](https://arxiv.org/html/2608.15008#bib.bib52)\]2023OtherA10GMemoryOS\[[19](https://arxiv.org/html/2608.15008#bib.bib66)\]2025LCA, P10GAI PERSONA\[[61](https://arxiv.org/html/2608.15008#bib.bib53)\]2024OtherA10GMemory\-R1\[[72](https://arxiv.org/html/2608.15008#bib.bib44)\]2025LC, LME, OtherA, P30OEM\-LLM\[[7](https://arxiv.org/html/2608.15008#bib.bib39)\]2024OtherA, P10OMemory\-T1\[[5](https://arxiv.org/html/2608.15008#bib.bib45)\]2025LC, OtherA, P20OHippoRAG\[[9](https://arxiv.org/html/2608.15008#bib.bib40)\]2024MHA, P10OMIRIX\[[63](https://arxiv.org/html/2608.15008#bib.bib27)\]2025LC, OtherA, P, M21GLD\-Agent\[[24](https://arxiv.org/html/2608.15008#bib.bib54)\]2024OtherA, P10GMMS\[[77](https://arxiv.org/html/2608.15008#bib.bib67)\]2025LCA, P10GMemTree\[[42](https://arxiv.org/html/2608.15008#bib.bib55)\]2024OtherA10GNemori\[[32](https://arxiv.org/html/2608.15008#bib.bib68)\]2025LC, LMEA, P20GRAPTOR\[[45](https://arxiv.org/html/2608.15008#bib.bib19)\]2024OtherA10GO\-Mem\[[58](https://arxiv.org/html/2608.15008#bib.bib69)\]2025LC, LMEA, P20GReadAgent\[[23](https://arxiv.org/html/2608.15008#bib.bib16)\]2024OtherA10GPRINCIPLES\[[21](https://arxiv.org/html/2608.15008#bib.bib70)\]2025OtherA10GTHEANINE\[[34](https://arxiv.org/html/2608.15008#bib.bib56)\]2024OtherA10GPropRAG\[[56](https://arxiv.org/html/2608.15008#bib.bib71)\]2025MH, OtherA, P20OA\-Mem\[[70](https://arxiv.org/html/2608.15008#bib.bib6)\]2025LC, DSA, P, T21GR3Mem\[[62](https://arxiv.org/html/2608.15008#bib.bib42)\]2025OtherA10OComoRAG\[[57](https://arxiv.org/html/2608.15008#bib.bib57)\]2025OtherA10GRGMem\[[53](https://arxiv.org/html/2608.15008#bib.bib72)\]2025LC, OtherA, P20GGAM\[[71](https://arxiv.org/html/2608.15008#bib.bib59)\]2025LCA10GRMM\[[51](https://arxiv.org/html/2608.15008#bib.bib73)\]2025LCA, P10GH\-Mem\[[74](https://arxiv.org/html/2608.15008#bib.bib58)\]2025LCA, P, R11OSeCom\[[38](https://arxiv.org/html/2608.15008#bib.bib74)\]2025LCA, P10GHindsight\[[22](https://arxiv.org/html/2608.15008#bib.bib26)\]2025LC, LMEA, P20GSGMem\[[66](https://arxiv.org/html/2608.15008#bib.bib75)\]2025LC, LMEA, P20GHippoRAG 2\[[10](https://arxiv.org/html/2608.15008#bib.bib41)\]2025MHA, P10OEMem\[[84](https://arxiv.org/html/2608.15008#bib.bib76)\]2025LC, LMEA, P20GLightMem\[[6](https://arxiv.org/html/2608.15008#bib.bib10)\]2025LC, LMEA, P, T, L, R, K24GZep\[[40](https://arxiv.org/html/2608.15008#bib.bib17)\]2025DMR, LMEA, P, L21GLiCoMemory\[[16](https://arxiv.org/html/2608.15008#bib.bib60)\]2025LC, LMEA, P, L21GAgentic Memory\[[76](https://arxiv.org/html/2608.15008#bib.bib61)\]2026LCA, P10GMemMachine\[[60](https://arxiv.org/html/2608.15008#bib.bib47)\]2026LC, LME, MHA, P, T31GEverMemOS\[[11](https://arxiv.org/html/2608.15008#bib.bib62)\]2026LC, LMEA, P20GMemori\[[1](https://arxiv.org/html/2608.15008#bib.bib78)\]2026LCA, P10GCognis\[[4](https://arxiv.org/html/2608.15008#bib.bib63)\]2026LC, LMEA, P20GMMM\[[13](https://arxiv.org/html/2608.15008#bib.bib77)\]2026LC, LMEA, P20GMAGMA\[[17](https://arxiv.org/html/2608.15008#bib.bib25)\]2026LC, LMEA, P20GSwiftMem\[[52](https://arxiv.org/html/2608.15008#bib.bib64)\]2026LC, LMEA, P, L21GTSM\[[48](https://arxiv.org/html/2608.15008#bib.bib65)\]2026LC, LMEA, P20G

## Appendix BMemory Substrate Implementation Details

Table 4:Memory substrate configurations\.TF: training\-free \(✓\) or requires weight modification \(✗\)\.LLM: auxiliary LLM required \(✓\) beyond final\-answer generation\.‡attention\-guided utterance selection with re\-prefill \(Qwen3 hybrid\-attention adaptation\)\.§zero\-shot LLM controller in place of PPO\.FamilyMethodTFLLMWriteReadMgmtExternal MemoryFlat IndexM1Dense Vector\[[68](https://arxiv.org/html/2608.15008#bib.bib12)\]✓✗Encode; add to indexDense top\-kk—M2Sparse Vector\[[43](https://arxiv.org/html/2608.15008#bib.bib86)\]✓✗Tokenize; append to BM25Sparse top\-kk—Text RecordM3Gist Index\[[23](https://arxiv.org/html/2608.15008#bib.bib16)\]✓✓LLM gist per page; index gistsLLM selects pages to expandText de\-dupStructuralM4Evolving Notes\[[70](https://arxiv.org/html/2608.15008#bib.bib6)\]✓✓LLM note per write; build linksANN\+\+link expansionEvolution\-triggered neighbour rewritesM5Dual\-Level Graph\[[8](https://arxiv.org/html/2608.15008#bib.bib48)\]✓✓LLM extracts entities\+\+relations; build KGKG\+\+vector hybrid \(mix mode\)Lazy batch indexing; cached graphHierarchicalM6Hierarchical Tree\[[45](https://arxiv.org/html/2608.15008#bib.bib19),[49](https://arxiv.org/html/2608.15008#bib.bib18)\]✓✓LLM assigns four levelsCollapsed\-tree top\-kk; all levelsTop\-level de\-dup; decayRefinementM7Distilled Strategies\[[35](https://arxiv.org/html/2608.15008#bib.bib49)\]✓✓LLM judges; distills strategies; MaTTSDense over strategies; top\-1Trajectory judge; strategy de\-dupM8Skill Bundles§\[[79](https://arxiv.org/html/2608.15008#bib.bib21)\]✓✓Cluster experiences into bundlesZero\-shot LLM; all bundles prefilledMerge and prune on overflowInternal MemoryWeightM9Adapter Tuning\[[80](https://arxiv.org/html/2608.15008#bib.bib22)\]✗✓LLM QA pairs; LoRA answer\-only lossDirect generation on base\+\+adapter—ActivationM10Full Context✓✗Append to bufferConcatenate; generateTruncate at windowM11Episode\-Clustered Re\-prefill‡\[[20](https://arxiv.org/html/2608.15008#bib.bib24)\]✓✗Cluster into episodes; keep 25%Match to centroid; re\-prefill episodeEpisodic clustering; budgetingThis appendix provides full implementation details for each of the eleven memory substrates summarized in Section[2](https://arxiv.org/html/2608.15008#S2)and listed in[Table4](https://arxiv.org/html/2608.15008#A2.T4)\.

### B\.1External Substrates

#### M1: Dense Vector \(Flat Index\)\.

Each incoming utterance is encoded into a 1536\-dimensional embedding via OpenAItext\-embedding\-3\-smalland appended to a flat index\. At read time, the query is embedded with the same encoder and the top\-kknearest neighbours are returned by cosine similarity\. No LLM is invoked during write or read\. No management operation is defined; the index grows monotonically\.

#### M2: Sparse Vector \(Flat Index\)\.

Each utterance is tokenized and appended to a BM25 inverted index\. At read time, the query is tokenized and scored against all entries via BM25\. No LLM is invoked\. No management operation is defined\.

#### M3: Gist Index \(Text Record\)\.

Implements ReadAgent\[[23](https://arxiv.org/html/2608.15008#bib.bib16)\]\. At write time, an auxiliary LLM generates a one\-sentence gist summary for each page of the input and indexes these gists\. At read time, the same LLM reads all gists and selects which pages to expand to their full original text; the expanded pages are concatenated as context for the final answer\. Management consists of text\-level deduplication over stored gists\. The two\-stage read pipeline \(gist filter then page expand\) means read cost scales with the number of stored pages, which becomes expensive on long histories\.

#### M4: Evolving Notes \(Structural\)\.

Implements A\-Mem\[[70](https://arxiv.org/html/2608.15008#bib.bib6)\]\. At write time, an auxiliary LLM generates a structured note \(with contextual description, keywords, and tags\) for each incoming message, then computes cosine similarity between the new note’s dense embedding and existing notes to establish semantic links\. At read time, the query embedding is matched against stored notes via cosine similarity, returning the top\-k most relevant notes; the result set is further expanded by following the note\-level links established at write time\. Management is triggered by an evolution detector: when a new note significantly updates an existing one, the LLM rewrites the affected neighbours to maintain consistency\.

#### M5: Dual\-Level Graph \(Structural\)\.

Implements LightRAG\[[8](https://arxiv.org/html/2608.15008#bib.bib48)\]\. At write time, an auxiliary LLM extracts entities and relations from the input and builds a knowledge graph \(KG\) alongside a chunk\-level vector index\. At read time, queries are routed through a hybrid mode that combines KG traversal \(entity and relation matching\) with dense vector search over the chunk store, merging results via reciprocal rank fusion\. Management uses lazy batch indexing: the KG and vector index are rebuilt periodically rather than incrementally, with a cached graph serving queries between rebuilds\.

#### M6: Hierarchical Tree \(Hierarchical\)\.

Implements RAPTOR\[[45](https://arxiv.org/html/2608.15008#bib.bib19)\]\. Incoming entries are embedded at write time; on the first read call, entries are recursively clustered and each cluster is summarised by an auxiliary LLM to form the next level of the tree, with clustering and summarisation repeated until a fixed maximum depth is reached\. The resulting multi\-level tree contains the raw entries at the leaves and progressively more abstract summaries at higher internal nodes\. At read time, we use RAPTOR’s collapsed\-tree variant: nodes from every level are flattened into a single dense index and the query embedding selects the top\-kkmost relevant nodes regardless of depth, so fine\-grained leaves and coarse summaries compete for the same retrieval slots\. In our implementation, clustering and summarisation are deferred to the first read call \(lazy construction\);write\(\)only embeds and appends entries, and the one\-off build cost is therefore absorbed by the first query \(see Tab\.[11](https://arxiv.org/html/2608.15008#A5.T11)\)\. Management performs top\-level deduplication and temporal decay, removing or demoting summary nodes superseded by more recent entries\.

#### M7: Distilled Strategies \(Refinement\)\.

Implements ReasoningBank\[[35](https://arxiv.org/html/2608.15008#bib.bib49)\]\. At write time, an auxiliary LLM judges each completed trajectory \(success or failure\), then distills the judgment into a structured, reusable reasoning strategy via memory\-aware test\-time scaling \(MaTTS\)\. Strategies are indexed via dense embedding\. At read time, the query is embedded and the most relevant strategy is retrieved\. Management performs trajectory\-level judging and strategy deduplication: redundant or superseded strategies are merged or pruned\.

#### M8: Skill Bundles \(Refinement\)\.

Implements MemSkill\[[79](https://arxiv.org/html/2608.15008#bib.bib21)\]with one deviation\. We keep the online skill\-evolution pipeline \(clustering, summarisation, merging, pruning\), so M8 incurs the write\- and management\-side LLM cost reported in the efficiency tables\. We replace only the PPO\-trained controller that originally selects a single skill bundle at read time with a zero\-shot LLM: all refined skill bundles are prefilled into the prompt, and the model attends to the full skill set when generating the answer for the current query\. We make this swap because no other substrate involves RL training, and keeping M8 RL\-driven would make its training cost dominate the efficiency axis\.

### B\.2Internal Substrates

#### M9: Adapter Tuning \(Weight\)\.

Implements LoRA\-FA\[[80](https://arxiv.org/html/2608.15008#bib.bib22)\]\. At write time, an auxiliary LLM synthesizes question\-answer pairs from the incoming content \(e\.g\., Q: “Where does the user live?” A: “San Francisco”\)\. A LoRA adapter \(r=8r\{=\}8\) is then fine\-tuned on these pairs using answer\-only loss on the base model\. At read time, the query is passed directly to the base model with the adapter applied; no explicit retrieval step occurs\.

#### M10: Full Context \(Activation\)\.

The full\-history concatenation baseline\. At write time, each incoming message is appended verbatim to a running buffer\. At read time, the entire buffer is concatenated and passed as context to the model for generation\. Management consists solely of truncation: when the buffer exceeds the model’s context window, the oldest entries are dropped\. This substrate establishes the “raw history retention” reference point against which all other substrates are compared\. We omit M10 from ALFWorld evaluation because cumulative trajectory text exceeds the context window of every backbone we test\.

#### M11: Episode\-Clustered Re\-prefill \(Activation\)\.

Implements EpiCache\[[20](https://arxiv.org/html/2608.15008#bib.bib24)\]with one deviation \(marked‡in[Table4](https://arxiv.org/html/2608.15008#A2.T4)\)\. At write time, past conversational turns are clustered into episodic groups, and a token budget of 25% is applied \(only the top quarter of tokens per episode are retained\)\. At read time, the query is encoded and matched to episode centroids; the best\-matching episode is re\-prefilled into the KV cache for generation\.

Deviation from reference\.The original EpiCache edits the KV cache in place using custom attention kernels\. Qwen3’s hybrid attention architecture \(mixing full attention and sliding\-window attention across layers\) is not supported by the authors’ released kernels\. Our implementation preserves the paper’s attention\-based episode scoring but applies it at the level of whole utterances: the selected episode’s utterances are re\-prefilled into the context rather than surgically inserted into the KV cache\. This changes the read\-time mechanism \(re\-prefill vs\. in\-place edit\) but preserves the core design principle of episodic selection with budget constraints\.

### B\.3Memory Substrate and Method Exclusion

We considered three additional substrates, MemGPT\[[36](https://arxiv.org/html/2608.15008#bib.bib14)\], Mem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\], and Zep\[[40](https://arxiv.org/html/2608.15008#bib.bib17)\], which are widely cited as production\-grade memory systems but which we ultimately excluded from our analysis on budget grounds\.[Table5](https://arxiv.org/html/2608.15008#A2.T5)reports the auxiliary\-LLM cost each substrate incurs on a single end\-to\-end LoCoMo run \(ten conversations, 1,986 queries, GPT\-4o\-mini aux model\), measured with the same harness used for every substrate in[Table4](https://arxiv.org/html/2608.15008#A2.T4)\. Embedding calls and embedding latency are excluded because they are served by a local FAISS index shared across runs\.

Table 5:Auxiliary\-LLM cost on a full LoCoMo run \(ten conversations, 1,986 queries; GPT\-4o\-mini aux model, embedding excluded\)\. Costs are one to two orders of magnitude above the lightest retrieval baselines \(e\.g\. M2 BM25 issues zero aux\-LLM calls and finishes in roughly ten minutes\)\.SubstrateMethodAuxiliary LLM callsExecution timeText RecordMemGPT\[[36](https://arxiv.org/html/2608.15008#bib.bib14)\]2,73916\.3 hText RecordMem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\]8,98411\.8 hStructuralZep\[[40](https://arxiv.org/html/2608.15008#bib.bib17)\]7,62423\.1 hThese three systems are LLM\-call\-heavy by design: every write triggers multiple auxiliary calls \(entity extraction, fact deduplication, conflict resolution, temporal invalidation, agent tool dispatch\), and every read fires further reranker, summary, or agent\-loop calls\. On a single LoCoMo run this manifests as roughly 2,700 to 9,000 auxiliary LLM calls per substrate and wall\-clock budgets of 11 to 23 hours, one to two orders of magnitude above the lighter retrieval baselines\.

Because the goal of our evaluation is a controlled cross\-substrate ablation rather than a production\-cost benchmark, and because all three substrates’ core retrieval ideas are already represented in[Table4](https://arxiv.org/html/2608.15008#A2.T4)by lighter, paper\-faithful approximations \(Mem0’s per\-input text distillation is loosely approximated by M3; Zep’s temporal knowledge graph overlaps with the entity/relation indexing in M5; MemGPT’s tiered buffer corresponds to the record store semantics in M3\), we report these systems here as a “production cost” reference and exclude them from the main comparison\. Their inclusion would not change any cross\-substrate ranking reported in the main text, but would dominate the wall\-clock and token\-count columns by an order of magnitude, obscuring the architectural differences our evaluation is designed to surface\.

## Appendix CBenchmark Setups and Bank Alignment

This appendix expands the benchmark descriptions in Section[3](https://arxiv.org/html/2608.15008#S3)with full capability definitions, evaluation splits, and pool construction details, and audits the semantic alignment between the offline retrieval banks and their evaluation pools for the two agent\-centric benchmarks\.

### C\.1Benchmark Setups and Capability Definitions

#### LoCoMo\.

The benchmark provides ten multi\-session dialogues with1,9861\{,\}986questions across five categories, serving as an aggregate quality indicator that mixes hardness dimensions within a single benchmark\.

#### MemoryAgentBench \(MAB\)\.

MAB factors long\-context memory into four capabilities:

- •Accurate Retrieval \(AR\), measured on the LongMemEval\-S∗subset\[[65](https://arxiv.org/html/2608.15008#bib.bib2)\]\(300300queries,∼355\{\\sim\}355K tokens each\), requires the substrate to locate relevant information within a massive history\.
- •Long\-Range Understanding \(LRU\)tests whether the substrate preserves signal as content grows\.
- •Test\-Time Learning \(TTL\)requires the agent to classify newly acquired patterns, implicitly demanding that stored knowledge be inspectable, a capability that parametric substrates fundamentally lack\.
- •Conflict Resolution \(CR\)presents contradictory facts at different time points and tests whether the substrate surfaces the most recent version\. We additionally use CR as our scalability probe by sweeping content length\.

#### ALFWorld\.

The benchmark provides134134valid\-unseen embodied\-planning tasks under a binary environment reward\. We report both a within\-episode regime \(memory grows step by step, stressing accumulation\) and a cross\-episode regime in which every method receives the same offline bank consisting of100100AgentGym successes\[[67](https://arxiv.org/html/2608.15008#bib.bib79)\]combined with100100Qwen3\-32B\-AWQreal failures \(stressing recovery, as the agent must bootstrap from a pre\-built memory\)\.

#### BigCodeBench\-Hard\.

The benchmark provides148148code tasks graded under realpytestexecution\. Each method is populated once from an offline pool of cross\-task solutions and harvested failures, with Hard\-set overlap removed before any run\. Retrieved entries are rendered as labelledSuccessfulandFailedblocks prepended to the official instruction prefix\. BigCodeBench stresses both accumulation \(encoding a diverse solution pool without drowning retrieval signal\) and staleness \(distinguishing current successes from outdated failures\)\.

### C\.2Bank–Test Semantic Alignment for ALFWorld and BCB\-Hard

A retrieval\-memory benchmark is meaningful only if its offline bank is*semantically related*to its evaluation pool: if the two come from disjoint distributions, every substrate is reduced to retrieving unrelated text and the rankings degenerate to base\-model noise\. We verify here that the offline banks we use for ALFWorld and BCB\-Hard both carry strong, statistically distinguishable semantic alignment with their evaluation pools\.

We embed every goal/instruction withtext\-embedding\-3\-small\(15361536\-D,ℓ2\\ell\_\{2\}\-normalized\) and report the nearest\-neighbor cosine similarity from each test item to the corresponding bank\. As a null baseline we replace the bank with i\.i\.d\. Gaussian unit\-norm vectors and recompute nearest cosines; this is the similarity that random retrieval would surface\.

#### ALFWorld\.

The bank holds200200trajectories \(100100AgentGym successes\[[67](https://arxiv.org/html/2608.15008#bib.bib79)\]\+\+100100Qwen3\-32B\-AWQreal failures, balanced across the six task types\)\. The evaluation pool is the canonicaln=134n\{=\}134valid\_unseensplit\. Nearest\-bank cosine for the test goals:

- •mean=0\.879=\\mathbf\{0\.879\}, median=0\.881=0\.881, min=0\.568=0\.568;
- •top\-55mean cosine=0\.761=0\.761\(the test goal is close to a*neighbourhood*of bank items, not a single outlier\);
- •distribution:30\.6%30\.6\\%in\[0\.95,1\.00\]\[0\.95,1\.00\],35\.8%35\.8\\%in\[0\.85,0\.95\)\[0\.85,0\.95\),22\.4%22\.4\\%in\[0\.75,0\.85\)\[0\.75,0\.85\),6\.0%6\.0\\%in\[0\.65,0\.75\)\[0\.65,0\.75\),5\.2%5\.2\\%in\[0\.00,0\.65\)\[0\.00,0\.65\)\.
- •Random\-bank control: mean=0\.068=0\.068, an order of magnitude below the real bank\.

Every test goal therefore has a structurally analogous example in the bank, typically the same task type with a different target object/receptacle\. The prefix filter prevents the bank from containing the actualvalid\_unseenenvironment instance, so the alignment is template\-level rather than instance\-level: substrates can borrow*strategy*\(e\.g\. “go to drawer, open it, take key, go to desk”\), not the answer\.

#### BigCodeBench\-Hard\.

The cross\-task BCB pool \(cross\-task solution snippets harvested from non\-Hard tasks plus failures collected during preliminary runs\) provides retrieval items forn=148n\{=\}148Hard instructions\. The Hard\-set overlap is removed before any run \(Sec\.[3](https://arxiv.org/html/2608.15008#S3)\)\. Nearest\-bank cosine on the Hard split:

- •mean=0\.594=\\mathbf\{0\.594\}, median=0\.586=0\.586, min=0\.303=0\.303;
- •top\-55mean cosine=0\.526=0\.526;
- •25%25\\%of Hard tasks have a bank example with cos≥0\.65\\geq 0\.65\(concrete reusable scaffolding such as “download files via FTP and parse”→\\to“connect to S3 and parse”\),40%40\\%in\[0\.55,0\.65\)\[0\.55,0\.65\)\(shared library or pattern, e\.g\.requests/json/pandasidioms\), and35%35\\%below0\.550\.55\(genuinely out\-of\-pool\)\.
- •Random\-bank control: mean=0\.061=0\.061, again an order of magnitude below the real bank\.

BCB\-Hard’s lower mean cosine vs\. ALFWorld is expected: code instructions span a much larger surface area \(different libraries, data sources, signatures\) than the templated goals of ALFWorld, so a diverse cross\-task pool can match each Hard task at most along an abstract dimension \(library family, control\-flow pattern\)\. Crucially the alignment is still ten\-times the random null and concentrated in the\[0\.55,0\.85\]\[0\.55,0\.85\]regime that semantic encoders typically reserve for “same topic, different surface form”\.

#### Summary\.

Both banks pass the basic sanity check that a memory benchmark requires: the retrieval target distribution overlaps the evaluation distribution well above chance, so substrate differences in Tab\.[2](https://arxiv.org/html/2608.15008#S4.T2)reflect their use of*available*signal rather than their luck against an unrelated bank\.ALFWorldprovides high\-density alignment \(0\.880\.88mean cosine,30%30\\%near\-duplicate templates\), exposing pure strategy\-transfer ability;BCB\-Hardprovides moderate, broadly distributed alignment \(0\.590\.59mean cosine\), stressing diverse\-pool encoding and cross\-task generalisation\. The two benchmarks therefore probe complementary memory regimes within the same evaluation framework\.

## Appendix DMetrics Definition and Assessment

This appendix expands Table[6](https://arxiv.org/html/2608.15008#A4.T6)with full mathematical definitions, evaluation protocols, and applicability notes for every metric we report\. We organize the discussion by the two families introduced in the main text:*Performance*\(§[D\.2](https://arxiv.org/html/2608.15008#A4.SS2)\) and*Efficiency*\(§[D\.3](https://arxiv.org/html/2608.15008#A4.SS3)\)\. Where a metric is benchmark\-specific, we state the reporting convention used by that benchmark; where a metric is substrate\-level \(Allin the table\), we describe how we compute it uniformly across the four benchmarks\.

Table 6:Metric taxonomy for evaluating memory substrates\.Two families:Performance\(P1–P6, P8–P12\) captures answer accuracy and retrieval quality;Efficiency\(E1–E15\) captures latency, storage, LLM call costs, and aggregate totals\. TheBenchmarkcolumn records which benchmarks officially report each metric\. Abbreviations:LC= LoCoMo,MA= MemoryAgentBench,ALF= ALFWorld,BC= BigCodeBench\-Hard;Alldenotes substrate\-level metrics applicable across all four\. Arrows indicate the preferred direction\.IDMetricDefinitionBenchmarkPerformance: answer accuracy and retrieval qualityP1↑\\uparrowExact Match𝟙\[𝚗𝚘𝚛𝚖\(y^\)=𝚗𝚘𝚛𝚖\(y\)\]\\mathbb\{1\}\[\\mathtt\{norm\}\(\\hat\{y\}\)=\\mathtt\{norm\}\(y\)\]; lowercase, strip articles, punctuation, whitespaceLCP2↑\\uparrowToken F1Harmonic mean of token level precision and recall between𝚗𝚘𝚛𝚖⁡\(y^\)\\mathtt\{norm\}\(\\hat\{y\}\)and𝚗𝚘𝚛𝚖⁡\(y\)\\mathtt\{norm\}\(y\)LCP3↑\\uparrowBLEU 1Clipped unigram precision with brevity penalty:BP⋅∑wmin⁡\(cwy^,cwy\)/\|y^\|\\mathrm\{BP\}\\cdot\\sum\_\{w\}\\min\(c\_\{w\}^\{\\hat\{y\}\},c\_\{w\}^\{y\}\)/\|\\hat\{y\}\|LCP4↑\\uparrowLLM JudgeLLM labels the answer CORRECT/WRONG given\(q,y,y^\)\(q,y,\\hat\{y\}\); fraction correct in\[0,1\]\[0,1\]LC,MAP5Compression Ratio\|input tokens\|/\|stored tokens\|\|\\text\{input tokens\}\|/\|\\text\{stored tokens\}\|; larger values indicate more lossy compression \(unsigned\)AllP6↑\\uparrowRecall@kk\|𝒢∩ℛ\|/\|𝒢\|\|\\mathcal\{G\}\\cap\\mathcal\{R\}\|/\|\\mathcal\{G\}\|; coverage of gold evidencedia\_idsLC,MAP8↑\\uparrowTask SuccessFraction of episodes where all goal conditions are satisfied \(binary per episode\); ALFWorld primary metricALFP9↑\\uparrowGC SuccessFraction of individual sub goal conditions satisfied \(ALFRED inherited\); reported in parentheses alongside P8ALFP10↓\\downarrowSteps to GoalEnvironment actions per successful episode; auxiliary efficiency signal, not officially reported by ALFWorldALFP11↑\\uparrowPass@1Unit test pass rate under greedy decoding; BigCodeBench primary metricBCP12↑\\uparrowSubEM𝟙\[𝚗𝚘𝚛𝚖\(y\)⊆𝚗𝚘𝚛𝚖\(y^\)\]\\mathbb\{1\}\[\\mathtt\{norm\}\(y\)\\subseteq\\mathtt\{norm\}\(\\hat\{y\}\)\]; substring exact match\. MA primary metric for Accurate Retrieval and Conflict ResolutionMAEfficiency: latency, storage, LLM call costs, and aggregate totalsE1↓\\downarrowMemory SizeMemory unit’s data structures in bytes; for parametric methods \(M9\), includes edited weights or adapter sizeAllE2↓\\downarrowInference TimeTotal per query wall time:read\(\)\+\+generate\(\); mean, p50, p90 \(ms\)AllE3↓\\downarrowWrite LatencyPer messagewrite\(\)wall time; mean, p50, p90 \(ms\)AllE4↓\\downarrowRetrieval LatencyPer queryread\(\)wall time; mean, p50, p90 \(ms\)AllE5↓\\downarrowWrite TokensLLM tokens consumed during the write phase \(input\+\+output\)AllE6↓\\downarrowWrite CallsNumber of LLM calls during the write phaseAllE7Retrieved TokensTokens surfaced by memory to support answer generationAllE8↓\\downarrowRead TokensLLM tokens consumed during read phase calls \(input\+\+output\)AllE9↓\\downarrowRead CallsNumber of LLM calls during the read phaseAllE10↓\\downarrowMgmt\. TokensLLM tokens for memory management \(dedup, update, delete, conflict resolution\)AllE11↓\\downarrowMgmt\. CallsNumber of LLM calls for memory managementAllE12↓\\downarrowTotal TokensAggregate auxiliary\-LLM tokens across phases:E5\+E8\+E10E\_\{5\}\+E\_\{8\}\+E\_\{10\}\(excludes retrieved tokensE7E\_\{7\}\)AllE13↓\\downarrowTotal CallsAggregate LLM calls across phases:E6\+E9\+E11E\_\{6\}\+E\_\{9\}\+E\_\{11\}AllE14↓\\downarrowTotal Wall\-clockEnd\-to\-end run time across the benchmark \(s\); includes write, read, and management phasesAllE15↓\\downarrowPer\-query LatencyE14/nqE\_\{14\}/n\_\{q\}, the principal cost reported in the main tables \(s/query\)All
### D\.1Notation

Throughout we letqqdenote a query,yythe gold answer, andy^\\hat\{y\}the predicted answer\. For retrieval based substrates, we writeℛ=\{r1,…,rk\}\\mathcal\{R\}=\\\{r\_\{1\},\\ldots,r\_\{k\}\\\}for the set of items returned byread\(q\)\(q\)at retrieval depthkk, and𝒢\\mathcal\{G\}for the set of gold evidence identifiers \(in LoCoMo, dialoguedia\_ids; in MemoryAgentBench, source chunk ids\)\. The text normalisation operator𝚗𝚘𝚛𝚖⁡\(⋅\)\\mathtt\{norm\}\(\\cdot\)lowercases the input, strips a fixed list of articles \(\{\\\{a, an, the\}\\\}\), removes punctuation, and collapses whitespace; this is the SQuAD style normaliser shared by LoCoMo and MemoryAgentBench\.

A*conversation*\(LoCoMo\) or*stream*\(MemoryAgentBench\) generates a sequence ofwritecalls followed by a batch ofreadcalls\. We refer to the union ofwritecalls as the*write phase*and to the per queryread→\\togenerateloop as the*read phase*\. Substrates that perform asynchronous maintenance \(for example, A Mem evolution, MemSkill clustering, or LoRA optimiser steps\) accumulate this work in a third*management phase*that we account for separately under E10 and E11\.

### D\.2Performance metrics

#### P1: Exact Match\.

EM\(y^,y\)=𝟙\[𝚗𝚘𝚛𝚖\(y^\)=𝚗𝚘𝚛𝚖\(y\)\]\\textrm\{EM\}\(\\hat\{y\},y\)=\\mathbb\{1\}\[\\mathtt\{norm\}\(\\hat\{y\}\)=\\mathtt\{norm\}\(y\)\]\. We average over all queries in the benchmark and report a single scalar in\[0,1\]\[0,1\]\. EM is the classical short answer metric; it is brittle to paraphrase and therefore complementary to P2, P3, and P4\.

#### P2: Token F1\.

LetT⁡\(⋅\)T\(\\cdot\)tokenise on whitespace after𝚗𝚘𝚛𝚖\\mathtt\{norm\}\. WithP=\|T⁡\(y^\)∩T⁡\(y\)\|/\|T⁡\(y^\)\|P=\|T\(\\hat\{y\}\)\\cap T\(y\)\|/\|T\(\\hat\{y\}\)\|andR=\|T⁡\(y^\)∩T⁡\(y\)\|/\|T⁡\(y\)\|R=\|T\(\\hat\{y\}\)\\cap T\(y\)\|/\|T\(y\)\|, F1=2​P​R/\(P\+R\)=2PR/\(P\+R\)\. We use multiset intersection so that repeated tokens contribute proportionally\. EmptyT⁡\(y^\)T\(\\hat\{y\}\)contributes F1=0=0\.

#### P3: BLEU 1\.

BLEU​1=BP⋅∑w∈T⁡\(y^\)min⁡\(cwy^,cwy\)/\|T⁡\(y^\)\|\\textrm\{BLEU\}\\,1=\\mathrm\{BP\}\\cdot\\sum\_\{w\\in T\(\\hat\{y\}\)\}\\min\(c\_\{w\}^\{\\hat\{y\}\},c\_\{w\}^\{y\}\)/\|T\(\\hat\{y\}\)\|, wherecwxc\_\{w\}^\{x\}counts occurrences ofwwinxxand the brevity penalty isBP=min⁡\(1,exp⁡\(1−\|T⁡\(y\)\|/\|T⁡\(y^\)\|\)\)\\mathrm\{BP\}=\\min\(1,\\exp\(1\-\|T\(y\)\|/\|T\(\\hat\{y\}\)\|\)\)\. P3 differs from P2 in two ways: it ignores recall, and it applies BP to discourage truncated answers\.

#### P4: LLM Judge\.

A judge modelJJreceives the queryqq, the gold answeryy, and the model’s answery^\\hat\{y\}and labels the answer as CORRECT or WRONG\. P4 reports the fraction of CORRECT labels across the evaluation set\. We usegpt\-4o\-miniat temperature00with deterministic templates that vary only by benchmark family: LoCoMo and MemoryAgentBench use the binary rubric introduced by Mem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\]; LongMemEval uses the original authors’ per\-question\-type rubric, which dispatches to one of four variants \(standard,temporal\-reasoning,knowledge\-update,abstention\) based on the question’s type label\[[65](https://arxiv.org/html/2608.15008#bib.bib2)\]\. The full templates are reproduced verbatim in the released code repository\. The judge sees only\(q,y,y^\)\(q,y,\\hat\{y\}\), not the retrieved evidence or memory trace, so P4 measures answer correctness, not retrieval grounding\.

#### P5: Compression Ratio\.

P5=Ninput/Nstored\\textrm\{P5\}=N\_\{\\text\{input\}\}/N\_\{\\text\{stored\}\}, whereNinputN\_\{\\text\{input\}\}is the total tokens passed towriteacross the whole benchmark andNstoredN\_\{\\text\{stored\}\}is the total tokens materialised in the substrate’s persistent store after writes complete\. Larger P5 means the substrate discards more raw text in favour of summaries or compressed structures\. P5=1=1means lossless caching; P5<1<1means the substrate*expands*the raw stream \(for example, via gist annotations\)\. P5 is unsigned: which direction is “better” depends on the downstream use, hence no arrow in the table\.

#### P6: Recall@kk\.

Recall@​k=\|𝒢∩ℛ\|/\|𝒢\|\\textrm\{Recall@\}k=\|\\mathcal\{G\}\\cap\\mathcal\{R\}\|/\|\\mathcal\{G\}\|\. For LoCoMo we resolveℛ\\mathcal\{R\}to golddia\_ids by finding, for each retrieved chunk, the smallest dialogue message span that covers it\. For MemoryAgentBench we use the published chunk ids directly\. Queries with\|𝒢\|=0\|\\mathcal\{G\}\|=0\(for example, adversarial or unanswerable items in LoCoMo\) are excluded\.

#### P8: ALFWorld Task Success\.

Per ALFWorld convention, an episode is successful iff every goal condition specified by the PDDL goal predicate is satisfied at the final step or earlier\. P8 is the unweighted mean over the 134\-episodevalid\_unseentest split\. We follow the official evaluator \(alfworld\.agents\.environment\.AlfredTWEnv\) and use amax\_steps=50cap\.

#### P9: ALFWorld GC Success\.

For each episode, GC Success is the fraction of individual sub goal predicates achieved \(numerator\), divided by the total predicates \(denominator\)\. The episode level GC contribution can therefore be non zero even when P8 is zero\. We report the macro average over episodes, matching ALFRED’s reporting convention\.

#### P10: Steps to Goal\.

For each successful episode \(P8=1=1\), we record the number of environment steps; P10 is the mean over successes, with the count of successes reported alongside for context\. Failed episodes are*excluded*, not penalised, so P10 is a conditional efficiency measure and should not be compared across substrates with very different P8\.

#### P11: BigCodeBench Pass@1\.

We sample one greedy completion \(T=0T=0\) per problem and execute it against the per problem unit test suite inside the officialbigcodebench\.evalsandbox\. P11 is the fraction of problems that pass all tests\. We use theinstructvariant of BigCodeBench Hard and thecalibrated=Falseevaluator so that imports are the substrate’s responsibility\.

#### P12: SubEM\.

SubEM\(y^,y\)=𝟙\[𝚗𝚘𝚛𝚖\(y\)⊆𝚗𝚘𝚛𝚖\(y^\)\]\\textrm\{SubEM\}\(\\hat\{y\},y\)=\\mathbb\{1\}\[\\mathtt\{norm\}\(y\)\\subseteq\\mathtt\{norm\}\(\\hat\{y\}\)\], where⊆\\subseteqis contiguous substring containment after normalisation\. SubEM is MemoryAgentBench’s primary metric for the Accurate Retrieval \(AR\) and Conflict Resolution \(CR\) capabilities; it is more lenient than EM for tasks where the gold is a short fact embedded in a longer free form answer\.

### D\.3Efficiency metrics

All wall clock measurements are taken on4×4\{\\times\}H200 GPUs unless otherwise noted; tokens are counted with the substrate’s own tokenizer \(for example, the Qwen3 BPE for Qwen3 substrates, orcl100k\_basefor OpenAI API substrates\)\. Latencies are reported as themean / p50 / p90triple; the table prints the mean for brevity\.

#### E1: Memory Size\.

The disk or RAM resident bytes occupied by the substrate’s data structures after the write phase has completed\. For RAG style substrates this is the size of the vector index plus any auxiliary metadata; for graph substrates we add the on disk DB \(neo4jorkuzu\) directory size\. For parametric substrates we report the trainable parameter footprint: the LoRA adapter size for M9\. E1 is comparable across families because all substrates eventually have to store their state on disk to support out of process evaluation\.

#### E2: Inference Time\.

Per queryread\(q\)→\(q\)\\togeneratewall time, end to end, including any context assembly\. We exclude the model load time \(amortised\) and any benchmark side overhead\.

#### E3: Write Latency\.

Perwritecall\. For substrates that batch writes \(for example, ReadAgent’s gisting\), we attribute the batch wall time to the constituent calls in proportion to their input length so that the distribution is well defined\.

#### E4: Retrieval Latency\.

Perreadcall, excluding the downstreamgeneratecall\. For substrates with no retrieval \(full context, parametric\), E4 is undefined and excluded from aggregates\.

#### E5: Write Tokens\.

Sum of input plus output tokens across all LLM calls issued during the write phase\. Embedding only substrates \(M1, M2\) have E5=0=0\.

#### E6: Write Calls\.

Number of LLM completions issued during the write phase\. We count each chat completion request as one call regardless of streaming\.

#### E7: Retrieved Tokens\.

The token count of the context block surfaced byreadto the generator\. This is informational rather than directional \(no arrow\): larger E7 buys more recall but inflates E8 downstream\.

#### E8: Read Tokens\.

LLM tokens consumed during the read phase, summed across the generator call and any reader side LLM calls \(for example, ReadAgent’s lookup or RAPTOR’s re ranking\)\. Excludes embedding tokens\.

#### E9: Read Calls\.

LLM call count for the read phase, by the same convention as E6\.

#### E10: Management Tokens\.

LLM tokens consumed by maintenance work that is neither write nor read attributable: A Mem evolution, MemSkill re clustering, and similar\. Substrates without maintenance loops have E10=0=0\.

#### E11: Management Calls\.

LLM call count for the management phase, by the same convention as E6\.

### D\.4Aggregation and reporting conventions

Per query versus per conversation\.P1 to P4, P6, and P12 are per query metrics; we report the unweighted mean over all queries in the benchmark\. P8 to P11 are per episode or per problem; we report the unweighted mean over episodes or problems\. E2 to E4 latencies are per event; we report the empirical mean \(the table prints mean only; p50 and p90 are in the per substrate result files\)\.

Sub domain breakdowns\.For LoCoMo we additionally report the per question type means \(single hop, multi hop, temporal, open domain, adversarial\) for P1 and P2; the table in the main text shows the aggregate\. For MemoryAgentBench we report each capability \(AR, TTL, LRU, CR\) separately because the capabilities target different sub skills\.

Cross substrate comparability\.A small number of metric×\\timessubstrate cells are undefined: P6 and E4 are not defined for full context or parametric substrates \(no retrieval set\); E5, E6, E10, and E11 are zero by construction for embedding only substrates\. We render undefined cells as “n/a” in the result tables rather than imputing zeros, since a zero would bias direction aware aggregates\.

Confidence intervals\.We report point estimates in the main tables for legibility\. Because the benchmark sample sizes are large \(n=1986n\{=\}1986for LoCoMo,n=300n\{=\}300for LME\-S,n=134n\{=\}134for ALFWorld,n=148n\{=\}148for BCB\-Hard\), the cross\-substrate gaps we highlight \(≥0\.05\\geq 0\.05absolute onP4P\_\{4\}or task success\) are unlikely to reverse under resampling\.

## Appendix EFull Table

### E\.1Full Table for the Locomo Benchmark

#### Substrate family vs\. accuracy and cost\.

The three appendix tables sort the ten substrates into a clear ordering by family\. The Flat family \(M1, M2\) requires no auxiliary LLM at any phase, stores the raw utterance set \(P5=1\.00P\_\{5\}\{=\}1\.00\), and lands at the cheap end of the curve: M2 at 0\.33 s/query, M1 at 0\.77 with the dense encoder advantage of\+0\.07\+0\.07P4\. The Text and Structural families \(M3, M4, M5\) all push auxiliary LLM tokens into the pipeline, but where they spend that budget matters more than how much\. M3 concentrates LLM work in the per\-query page\-select call \(E8=8\.2E\_\{8\}\{=\}8\.2M, one call per query\) and recovers M1\-level P4 with a5\.3×5\.3\\timesstore compression\. M4 spends mostly on note generation at write \(E5=4\.6E\_\{5\}\{=\}4\.6M\) and on the evolution loop at management \(E10=3\.2E\_\{10\}\{=\}3\.2M\), giving one of the smallest stores in the family \(178K\) at the cost of one of the lowest P6 Recall@k \(0\.465\)\. M5 is the most LLM\-intensive substrate by a wide margin \(E12=51E\_\{12\}\{=\}51M,E15=28E\_\{15\}\{=\}28s/query\) yet posts the table’s top P4 \(0\.648\) by raising the LLM Judge score, not retrieval recall\. The Hierarchical and Refinement families invert this trade\-off: M6 pays modest write\-time tree summarisation \(E5=382E\_\{5\}\{=\}382K\) and zero read\-phase LLM yet lands the highest P6 Recall@k in the table \(0\.797\), because the collapsed tree exposes nodes at every layer for the embedding search; M8 sits between M6 and M5 in cost but trails M6 by 4 to 5 P4 points because prefilling every skill bundle dilutes the model’s attention over the one relevant to the query\.

#### Backbone dependence and internal\-memory families\.

The relative ordering on P4 is mostly preserved across the three backbones, and the leading substrate’s margin over the flat baselines is robust to model size\. M5’s\+0\.11\+0\.11P4 lead over M1 onQwen3\-8Bholds at\+0\.10\+0\.10onQwen3\-32B\-AWQand widens to\+0\.15\+0\.15on Gemma\-4\-26B; M3’s parity with M1 holds across all three; M2 trails M1 on every backbone\. The internal\-memory families illustrate the limits of backbone\-only solutions\. The Weight family \(M9\) collapses on LoCoMo regardless of host \(P1 EM<<0\.025\): single\-pass adapter tuning over QA pairs is not enough for the recall\-style questions LoCoMo asks\. The Activation family \(M10, M11\) shows the sharpest backbone dependence in the table\. M10 reaches the highest P4 among the Activation family onQwen3\-32B\-AWQ\(0\.669\), just behind structural M5 \(0\.683\), by paying55to10×10\\timesthe latency of M1, but the gain over M1 narrows to\+0\.05\+0\.05onQwen3\-8Bbecause the smaller model under\-utilises long contexts, and onGemma\-4the cost\-benefit balance moves further against M10\. M11 is the only substrate that exhibits clear inverse scaling, with P4 dropping from 0\.307 onQwen3\-8Bto 0\.274 onQwen3\-32B\-AWQ, consistent with episode\-clustering tuned forQwen3\-8B’s KV statistics mis\-clustering underQwen3\-32B\-AWQ’s grouped\-query attention\. Together, these results suggest that the recall\-heavy structure of LoCoMo rewards retrieval substrates that pair a faithful textual store \(P5P\_\{5\}near 1\) with broad retrieval coverage, and that auxiliary LLM cost translates into accuracy only when the LLM work targets the actual retrieval bottleneck rather than synthesis quality on top of it\.

Table 7:Table for different substrate and model LoCoMo Performance\.ModelSub\.FamilyP1 EMP2 F1P3 BLEUP4 JudgeP6 Recall@kQwen3\-8BM1Flat0\.2730\.4660\.2850\.5400\.696M2Flat0\.2830\.4360\.2410\.4700\.595M3Text0\.1900\.3660\.2690\.5620\.620M4Struct0\.2260\.3550\.2160\.4350\.465M5Struct0\.1960\.3810\.2520\.6480\.455M6Hier0\.2510\.4360\.2760\.5560\.797M8Refine0\.3010\.4560\.2520\.5090\.615M9Weight0\.0190\.0990\.0710\.379–M10Act0\.1590\.3420\.2850\.589–M11Act0\.0660\.1610\.1380\.307–Qwen3\-32B\-AWQM1Flat0\.2080\.4080\.3280\.5820\.696M2Flat0\.1870\.3520\.2720\.4870\.595M3Text0\.1730\.3680\.2890\.5860\.620M4Struct0\.1650\.3200\.2400\.4520\.465M5Struct0\.1340\.3290\.2860\.6830\.455M6Hier0\.1920\.3870\.3070\.5730\.797M8Refine0\.1900\.3650\.2810\.5330\.615M9Weight0\.0240\.1010\.0820\.399–M10Act0\.1900\.4120\.3370\.669–M11Act0\.0350\.1180\.0950\.274–Gemma\-4\-26BM1Flat0\.3290\.5130\.3110\.5740\.696M2Flat0\.3020\.4510\.2570\.5550\.595M3Text0\.3030\.4840\.3000\.5990\.620M4Struct0\.3100\.4050\.2320\.4760\.465M5Struct0\.2030\.4510\.2710\.7190\.455M6Hier0\.3100\.4830\.2920\.5920\.797M8Refine0\.3350\.4820\.2410\.5540\.615M10Act0\.3000\.5150\.3080\.688–M11Act0\.0850\.1810\.1490\.330–Table 8:Table for different substrate and model LoCoMo Latency & Quality\.ModelSub\.Family𝑬𝟐E\_\{2\}\(ms/q\)𝑬𝟑E\_\{3\}\(ms/wr\)𝑬𝟒E\_\{4\}\(ms/rd\)𝑬𝟏𝟒E\_\{14\}\(s\)𝑬𝟏𝟓E\_\{15\}\(s/q\)𝑷𝟓P\_\{5\}Qwen3\-8BM1Flat5221992261,5340\.771\.00M2Flat302016600\.331\.00M3Text2,2921,6637818,2934\.185\.29M4Struct1696,210912,6386\.360\.71M5Struct20,6805,1169,61055,29027\.840\.44M6Hier6836411952,7581\.390\.62M8Refine4312,5701796,0193\.031\.09M9Weight3351,400–3,4981\.76–M10Act4,132––9,0274\.55–M11Act1,848––4,0372\.03–Qwen3\-32B\-AWQM1Flat2,1741946485,1422\.591\.00M2Flat2,153014,7032\.371\.00M3Text4,3791,6231,10012,8526\.475\.19M4Struct2,2896,3581217,2698\.700\.72M5Struct21,9495,2219,58658,02529\.220\.43M6Hier2,65065361010,3275\.200\.60M8Refine2,7072,60016610,9915\.531\.11M9Weight8603,800–9,3864\.73–M10Act14,774––32,27516\.25–M11Act3,790––8,2804\.17–Gemma\-4\-26BM1Flat6672001881,8500\.931\.00M2Flat445019720\.491\.00M3Text2,8701,6641,0189,5554\.815\.37M4Struct3946,1451113,1296\.610\.73M5Struct18,9825,1359,44451,54425\.950\.43M6Hier7126261742,8221\.420\.61M8Refine6002,5721666,3883\.221\.06M10Act4,698––10,2635\.17–M11Act1,967––4,2972\.16–Table 9:Table for different substrate LoCoMo Storage, Tokens & Calls\.Sub\.Family𝑬𝟏E\_\{1\}𝑬𝟓E\_\{5\}𝑬𝟔E\_\{6\}𝑬𝟕E\_\{7\}𝑬𝟖E\_\{8\}𝑬𝟗E\_\{9\}𝑬𝟏𝟎E\_\{10\}𝑬𝟏𝟏E\_\{11\}𝑬𝟏𝟐E\_\{12\}𝑬𝟏𝟑E\_\{13\}M1Flat3\.59M00889\.8K000000M2Flat173\.7K00806\.6K000000M3Text257\.4K311\.8K39011\.52M8\.23M2,040008\.54M2,430M4Struct177\.6K4\.59M3,26896\.2K003\.16M2,3127\.75M5,580M5Struct167\.8K33\.0M9,8606\.00M17\.8M3,9800050\.8M13,840M6Hier4\.33M381\.8K9101\.87M00127\.8K190509\.6K1,100M8Refine3\.40M1\.46M481806\.2K2\.94M2,037242\.9K514\.64M2,569M9Weighthost\-dep–––––––––M10Act101\.2K––60\.9M––––––M11Act100\.7K––15\.3M––––––

### E\.2Full Tables for the LongMemEval\-S and MAB Benchmarks

#### Substrate family vs\. accuracy and cost\.

The three appendix tables sort the ten substrates evaluated on LongMemEval\-S \(LME\-S\) and the three MAB sub\-benchmarks — LRU \(detective\_qa, long\-range understanding\), TTL \(icl\_banking77\_5900shot\_balance, test\-time learning\) and CR \(factconsolidation\_sh\_6k, conflict resolution\) — into a clear ordering by family\. The Flat family \(M1, M2\) requires no auxiliary LLM at any phase, stores the raw conversation slots \(P5=1\.00P\_\{5\}\{=\}1\.00\), and lands at the cheap end of every column: M2 finishes LME\-S at 1\.75 s/query and TTL at sub\-second per query while M1 carries the dense\-encoder advantage of at least\+0\.05\+0\.05P12 on LME\-S, widening on the larger backbones\. The Text family \(M3\) flips the trade\-off: gist generation paysE5=437E\_\{5\}\{=\}437K tokens at write and another5\.05\.0M retrieved tokens at read time, but on TTL banking77 — where the answer is a single class id and the gist matches almost surface\-form — it climbs to a high TTL score onQwen3\-8B\(P12==0\.840, still below M8’s 0\.960\) and stays there across stronger backbones \(0\.860 / 0\.798\)\. The Structural family splits cleanly: M4 concentrates LLM work in note evolution at write \(E5=4\.4E\_\{5\}\{=\}4\.4M,E6=1310E\_\{6\}\{=\}1310calls\) and trails M1 on LME\-S P12 by about0\.080\.08despite a6\.2×6\.2\\timesstore compression, and stalls on CR where the evolved notes do not preserve the temporal serial\-number cue the benchmark relies on \(M4 CR P12==0\.080\); M5 is the most LLM\-intensive substrate by a wide margin \(E5=10\.3E\_\{5\}\{=\}10\.3M,E2≈8\.5E\_\{2\}\{\\approx\}8\.5s/query onQwen3\-8B\) and only matches the flat M1 on LME\-S SubEM \(P12\), its higher LLM\-judge score not reflecting better retrieval recall\. The Hierarchical \(M6\) and Refinement families invert this trade\-off: M6 pays modest write\-time tree summarisation \(E5=280E\_\{5\}\{=\}280K\) and posts one of the highest LME\-S P2 scores \(0\.1760\.176onQwen3\-32B\-AWQ, just behind M1’s0\.1800\.180\); M8 sits between M6 and M5 in cost but takes the TTL P12 lead \(0\.96 / 0\.94 / 0\.95 across backbones\), confirming that skill bundles transfer cleanly when the downstream task is itself few\-shot ICL\.

#### Backbone dependence and internal\-memory families\.

The relative ordering on P12 is mostly preserved across the three backbones\. M5’s LME\-S advantage over M1 shows up only on the LLM\-judge score, growing from\+0\.06\+0\.06onQwen3\-8Bto\+0\.15\+0\.15onGemma\-4, while on SubEM the two stay within0\.020\.02\. On TTL, by contrast, M3’s lead over the Flat baselines is largest onQwen3\-8B\(\+0\.27\+0\.27over M1\) and narrows on stronger backbones as the flat stores themselves improve, while M8 keeps a commanding TTL P12 score across all three \(0\.96 / 0\.94 / 0\.95\)\. The internal\-memory families illustrate the limits of backbone\-only solutions on dialog QA\. The Weight family \(M9\) collapses on LME\-S regardless of host \(P12≤0\.10\\leq 0\.10\): single\-pass adapter tuning over QA pairs is not enough for the recall\-style preference questions LME\-S asks, and the same weakness reappears on CR \(P12=0\.16=0\.16on both Qwen backbones\) where conflicts hinge on serial\-number\-encoded recency that the adapter cannot represent\. The Activation family \(M10, M11\) shows the sharpest*task*dependence in the table: both substrates collapse on LME\-S long contexts \(M10 P12≤0\.137\\leq 0\.137, M11 P12≤0\.234\\leq 0\.234\) because compressed KV pools cannot reconstruct distant single\-utterance evidence, but on TTL classification — where the relevant ICL examples are recent — M11 climbs back to within0\.100\.10of the Flat substrates \(its P12 reaches0\.8200\.820onQwen3\-32B\-AWQ\)\. M9, in contrast, gains nothing from scale on CR — its P12 stays flat at0\.160\.16across both Qwen backbones while every external substrate except M4 gains 0\.10–0\.40 fromQwen3\-8BtoGemma\-4— because the LoRA adapter encodes a fixed\-schema mapping that does not benefit from the larger backbone’s better in\-context routing\. Together, these results suggest that LME\-S rewards substrates with a faithful, well\-covered textual store, that TTL is dominated by prompt\-format alignment rather than retrieval recall, and that auxiliary LLM cost \(M5, M8\) translates into accuracy only on benchmarks \(TTL, CR\) whose answer surface aligns with the substrate’s generative format\.

#### Storage and latency trade\-offs\.

Tab\.[11](https://arxiv.org/html/2608.15008#A5.T11)confirms three structural patterns\. First,E1E\_\{1\}partitions the substrates into three storage tiers that are nearly model\-independent:lightstores \(≤80\\leq 80KB\) for the Activation family \(M10, M11\), which keep only episode centroids or chunk pointers;midstores \(1\.6–9\.7 MB\) for the Flat and Text families and M4; andheavystores \(∼\{\\sim\}22–32 MB, plus M9’s 80–320 MB adapter\) for M5, the Hierarchical and Refinement families, and Weight\. Second,E2E\_\{2\}shifts with backbone size — roughly22–4×4\\timesup fromQwen3\-8BtoQwen3\-32B\-AWQ, then about0\.5×0\.5\\timesdown onGemma\-4— so the*substrate\-induced*latency overhead \(the within\-band spread ofE2E\_\{2\}\) is what differentiates the substrates: M5 pays22to5×5\\timesthe per\-query inference of M1/M2 because its read path interleaves entity\-graph traversal and chunk\-level vector search with the generator\. M6 is the one apparent exception: our implementation defers tree construction to the firstread\(\)call, so the first query absorbs the entire one\-off build cost, inflatingE2,meanE\_\{2,\\mathrm\{mean\}\},E14E\_\{14\}, andE15E\_\{15\}on every backbone — M6 is the only substrate withE2,p90<E2,meanE\_\{2,\\mathrm\{p90\}\}<E\_\{2,\\mathrm\{mean\}\}— while its steady\-stateE2,p90E\_\{2,\\mathrm\{p90\}\}is comparable to the Flat baselines\. Third,E14E\_\{14\}separates substrates whose total cost is dominated by*write*\(M4 at∼\{\\sim\}68–70K s and M5 at∼\{\\sim\}56–58K s across backbones, driven by per\-write note evolution and entity extraction\) from those dominated by*read*\(M1, M2, and M10, mostly under 5K s\); since write phases are amortisable across model swaps but read phases are not, the per\-query inference timeE2E\_\{2\}is the more reliable cross\-substrate efficiency signal\.

#### Token accounting and memory quality\.

Tab\.[12](https://arxiv.org/html/2608.15008#A5.T12)cleanly separates the three substrate families by their LLM\-token profile\.Flat\(M1, M2\) substrates record only the retrieved\-token channel \(E7E\_\{7\}\) and invoke no auxiliary LLM at write or management time, soE5=E6=E10=E11=0E\_\{5\}=E\_\{6\}=E\_\{10\}=E\_\{11\}=0andE13=0E\_\{13\}=0; theActivationsubstrates \(M10, M11\) likewise use no auxiliary LLM and are shown as ‘–’ in the token\-phase columns\.Refinement / Structural / Textsubstrates push 0\.4–10\.4M tokens through the construct phase \(E5E\_\{5\}\): M5 dominates at 10\.3M with 2,371 calls because it runs both entity\-extraction and relation\-extraction passes per chunk, while M6 is the cheapest of the auxiliary\-LLM substrates at 280K / 80 calls thanks to log\-depth tree clustering\. TheP5P\_\{5\}column exposes the storage\-vs\-compression trade\-off:P5P\_\{5\}ranges from 1\.00 \(M1/M2 — verbatim stores\) through 2\.80 \(M6, mild summarisation\) to 12\.50 \(M8, highly abstracted skill bundles\)\. The dashedP5P\_\{5\}entries for M9 / M10 / M11 reflect that these substrates have no textual store on which compression can be defined — a structural limitation, not missing data\.

Table 10:P12 \(SubEM\), P2 \(F1\), P3 \(BLEU\-1\) per \(substrate, generator, sweep\) for LongMemEval\-S and MemoryAgentBench\.LME\-SMAB LRUMAB TTLMAB CRModelIDFamilyP12P2P3P12P2P3P12P2P3P12P2P3Qwen3\-8BM1Flat0\.3900\.1700\.1130\.5920\.4910\.4170\.5700\.4760\.4050\.3300\.2110\.171M2Flat0\.3400\.1700\.1140\.6060\.5270\.4480\.4800\.3990\.3400\.2800\.1880\.155M3Text0\.1230\.0470\.0290\.5490\.4790\.4070\.8400\.5440\.4620\.2700\.1920\.163M4Struct0\.3030\.1450\.0940\.4650\.4190\.3560\.4900\.4680\.3970\.2900\.1210\.103M5Struct0\.3800\.1600\.1030\.5490\.4910\.4170\.5800\.4680\.3970\.5000\.3620\.308M6Hier0\.3570\.1620\.1110\.5210\.4190\.3560\.2900\.2460\.2100\.2700\.1710\.138M8Refine0\.3730\.1590\.1010\.5920\.5030\.4270\.9600\.5700\.4840\.3100\.2720\.231M9Weight0\.0830\.0820\.0580\.2390\.1920\.1630\.1700\.1670\.1420\.1600\.1080\.089M10Act0\.0900\.0750\.0530\.6480\.5750\.4880\.1400\.1300\.1110\.2700\.1610\.124M11Act0\.2020\.1110\.0650\.5920\.5390\.4580\.5600\.4680\.3970\.0500\.0370\.032Qwen3\-32B\-AWQM1Flat0\.4430\.1800\.1220\.6340\.5750\.4880\.8200\.6890\.5850\.2800\.1340\.093M2Flat0\.3770\.1630\.1090\.6200\.5630\.4780\.7900\.6630\.5640\.2700\.1360\.091M3Text0\.2250\.1240\.0720\.7040\.6580\.5600\.8600\.7220\.6140\.2800\.1580\.120M4Struct0\.3570\.1600\.1060\.5210\.4550\.3870\.6800\.6660\.5660\.1400\.0720\.049M5Struct0\.4600\.1590\.1040\.5920\.4910\.4170\.6200\.5100\.4330\.4400\.1910\.127M6Hier0\.4130\.1760\.1170\.5770\.6230\.5290\.3400\.2810\.2380\.2800\.1350\.090M8Refine0\.4030\.1660\.1110\.5770\.5030\.4270\.9400\.7900\.6720\.3100\.1620\.115M9Weight0\.1000\.0980\.0650\.4080\.3590\.3050\.3240\.3180\.2700\.1600\.0860\.059M10Act0\.1370\.0870\.0610\.6060\.5270\.4480\.7600\.6460\.5490\.2700\.1280\.085M11Act0\.0800\.0800\.0560\.5920\.5150\.4380\.8200\.6890\.5850\.2300\.1220\.086Gemma\-4\-26BM1Flat0\.4330\.2000\.1350\.6620\.5750\.4880\.7700\.6290\.5350\.5200\.2670\.223M2Flat0\.3570\.1800\.1220\.6620\.6730\.5720\.7300\.6200\.5270\.5200\.2480\.202M3Text0\.2230\.1230\.0710\.3520\.2750\.2340\.7980\.6160\.5240\.5200\.3040\.259M4Struct0\.3530\.1660\.1110\.5210\.4550\.3870\.6900\.6160\.5240\.0800\.0680\.058M5Struct0\.4530\.1920\.1290\.5350\.3830\.3260\.7200\.6120\.5200\.6600\.2940\.214M6Hier0\.3970\.1750\.1180\.4930\.4310\.3660\.2500\.2120\.1810\.4800\.2280\.182M8Refine0\.3730\.1640\.1080\.5490\.4550\.3870\.9500\.8070\.6860\.4700\.3270\.278M10Act0\.0630\.0450\.0390\.5770\.5790\.4920\.4600\.3910\.3320\.5000\.2630\.222M11Act0\.2340\.1290\.0750\.5720\.4430\.3760\.7020\.6160\.5240\.0500\.0500\.043

Table 11:LongMemEval\-S storage and latency per substrate\.ModelIDFamily𝑬𝟏E\_\{1\}𝑬𝟐,𝐦𝐞𝐚𝐧E\_\{2,\\mathrm\{mean\}\}𝑬𝟐,𝐩𝟗𝟎E\_\{2,\\mathrm\{p90\}\}𝑬𝟏𝟒E\_\{14\}𝑬𝟏𝟓E\_\{15\}Qwen3\-8BM1Flat9\.7M2,0754,6022,1917\.30M2Flat2\.7M1,7473,5805251\.75M3Text4\.5M4,5009,00016,67455\.58M4Struct1\.6M1,8284,23367,702225\.67M5Struct32\.0M8,50018,00056,162187\.21M6Hier27\.5M19,9565,1955,98719\.96M8Refine22\.7M1,8694,39230,855102\.85M9Weight80\.0M2,4005,00015,45151\.50M10Act60K3,7445,5391,9836\.61M11Act80K1,9004,2004,38214\.61Qwen3\-32B\-AWQM1Flat9\.7M7,41318,1003,60012\.00M2Flat2\.7M7,30718,8242,1937\.31M3Text4\.5M16,07632,15320,22667\.42M4Struct1\.6M7,07018,01970,200234\.00M5Struct32\.0M18,00035,00058,225194\.08M6Hier27\.5M20,27117,8166,08220\.27M8Refine23\.4M6,14915,19232,034106\.78M9Weight320M8,50019,00051,511171\.70M10Act60K14,65028,0226,89422\.98M11Act80K7,00017,00015,71752\.39Gemma\-4\-26BM1Flat9\.7M4,1498,5502,5468\.49M2Flat2\.7M3,3726,5011,0123\.37M3Text4\.5M8,99817,9969,53431\.78M4Struct1\.6M3,8059,02769,116230\.39M5Struct32\.0M12,00023,00057,280190\.93M6Hier27\.5M17,1899,0075,15717\.19M8Refine21\.7M3,1246,54031,140103\.80M10Act60K3,3403,9452,0736\.91M11Act80K––5,55018\.50Table 12:LongMemEval\-S token usage \(E5E\_\{5\}–E13E\_\{13\}\)\.ModelIDFamily𝑬𝟓E\_\{5\}𝑬𝟔E\_\{6\}𝑬𝟕E\_\{7\}𝑬𝟏𝟐E\_\{12\}𝑬𝟏𝟑E\_\{13\}𝑷𝟓P\_\{5\}Qwen3\-8BM1Flat00101K001\.00M2Flat00153K001\.00M3Text437K6715\.0M5\.3M7314\.50M4Struct4\.4M1310108K4\.4M13106\.20M5Struct10\.3M2371375K10\.4M24313\.80M6Hier280K8046K280K802\.80M8Refine1\.8M175221K2\.78M350412\.50M9Weight––––––M10Act––1\.9M–––M11Act––250K–––Qwen3\-32B\-AWQM1Flat00101K001\.00M2Flat00153K001\.00M3Text437K6715\.0M5\.3M7314\.50M4Struct4\.4M1310108K4\.4M13106\.20M5Struct10\.4M2369385K10\.4M24293\.80M6Hier280K8046K280K802\.80M8Refine1\.8M175220K2\.78M350412\.50M9Weight––––––M10Act––1\.9M–––M11Act––250K–––Gemma\-4\-26BM1Flat00104K001\.00M2Flat00158K001\.00M3Text437K6715\.0M5\.3M7314\.50M4Struct4\.4M1310111K4\.4M13106\.20M5Struct10\.4M2377379K10\.4M24373\.80M6Hier280K8046K280K802\.80M8Refine1\.9M175223K2\.98M350412\.50M9Weight––––––M10Act––422K–––M11Act––250K–––

### E\.3Full Table for AlfWorld

#### Substrate family vs\. task success and cost\.

The three appendix tables sort the ten substrates evaluated on ALFWorld into a clear ordering by family\. The Flat family \(M1, M2\) shows that retrieval\-only memory is backbone\-dependent on ALFWorld: M1 trails NoMem onQwen3\-8B\(5\.2 vs\. 5\.7 TSR\) but edges above it onQwen3\-32B\-AWQ\(27\.6 vs\. 22\.4\) andGemma\-4\(10\.4 vs\. 7\.5\), and M2 lands close to NoMem for free\. The Text and Structural families \(M3, M4, M5\) recover modest gains onQwen3\-32B\-AWQ\(M3 26\.9, M4 26\.9, M5 23\.1 vs\. NoMem 22\.4\) by paying auxiliary LLM tokens during write: M3 and M4 each spend 0\.4 to 1\.6 million write tokens on gist or note generation, while M5 runs an entity extraction loop \(1\.4 to 1\.6 million tokens, around 400 calls\)\. OnQwen3\-8BandGemma\-4the same substrates fail to translate write cost into TSR gain, suggesting the bottleneck is not retrieval recall but action planning\. The Hierarchical and Refinement families behave like specialised performers on the strongest backbone: M7 is the clear winner onQwen3\-32B\-AWQ\(top TSR 32\.1\) because its skill\-distillation prompts encode the goal decomposition structure that ALFWorld rewards\. The Refinement M8 trails M7 by 9\.7 TSR points despite carrying nearly the same write cost, which we attribute to prefilling every skill bundle diluting the model’s attention over the goal\-relevant one\.

#### Backbone dependence and internal\-memory families\.

Backbone size dominates the absolute TSR:Qwen3\-32B\-AWQ’s strongest external substrate \(M7 32\.1\) more than triplesQwen3\-8B’s \(M5 9\.0\), whileGemma\-4sits between \(M1, M3, M6 each at 10\.4\)\. The internal\-memory families illustrate the limits of backbone\-only solutions\. The Weight family \(M9\) is among the substrates that materially exceed NoMem onQwen3\-32B\-AWQ\(29\.9 vs\. 22\.4, second only to M7\) because adapter tuning on demonstration trajectories captures action grounded patterns that pure text retrieval cannot\. The same M9 collapses onQwen3\-8B\(3\.0 TSR\) where the smaller backbone overfits the LoRA pairs, and it is excluded onGemma\-4because it is incompatible with the A4B MoE routing\. The Activation family \(M11\) exceeds NoMem on every backbone, most sharply onQwen3\-8B\(11\.9 vs\. 5\.7, roughly double\), because the episodic key value cache compression preserves the action policy while keeping the persistent store small \(around 6\.9 thousand bytes\)\.We omit M10 from the ALFWorld evaluation because the cumulative trajectory text across the 134 task suite exceeds the context window of every backbone we test\.Naively concatenating prior trajectories drives input length past 128 thousand tokens by task 30 to 40 and forces aggressive truncation, which destroys the comparison: any reported M10 number on ALFWorld would reflect a truncation policy rather than the substrate\. The lighter M11 sidesteps this problem by re\-prefilling only the matched episode \(around 120 thousand retrieved tokens across the run\), which fits inside a single context window per task\. Together, these results suggest ALFWorld is dominated by action policy grounding rather than retrieval recall, and that within a fixed backbone the strongest gains come from the refinement \(M7\) and weight \(M9\) substrates on the strongest backbone; the remaining substrates are useful as targeted policy aids on strong backbones \(M7, M3\) but offer little on weaker ones\.

Table 13:Table for different substrate and model ALFWorld\-unseen Performance\.ModelSub\.FamilyP8 TSRStepsP10 Steps∣𝒮\\mid\\mathcal\{S\}PavgP\_\{\\mathrm\{avg\}\}Qwen3\-8B–NoMem5\.747\.012\.213\.5M1Flat5\.247\.75\.96\.6M2Flat6\.747\.06\.09\.4M3Text5\.247\.88\.38\.4M4Struct7\.546\.97\.98\.5M5Struct9\.047\.117\.412\.5M6Hier4\.548\.415\.08\.3M7Refine7\.547\.313\.79\.6M8Refine8\.247\.418\.59\.5M9Weight3\.049\.119\.53\.2M11Act11\.945\.714\.214\.5Qwen3\-32B\-AWQ–NoMem22\.441\.611\.323\.7M1Flat27\.639\.69\.616\.2M2Flat21\.641\.410\.815\.4M3Text26\.939\.49\.921\.8M4Struct26\.939\.512\.424\.3M5Struct23\.141\.111\.423\.0M6Hier23\.940\.29\.922\.8M7Refine32\.137\.511\.129\.9M8Refine22\.441\.612\.421\.3M9Weight29\.938\.712\.127\.4M11Act26\.939\.512\.324\.7Gemma\-4\-26B–NoMem7\.547\.111\.317\.3M1Flat10\.446\.315\.016\.3M2Flat9\.746\.311\.517\.2M3Text10\.446\.314\.814\.9M4Struct8\.247\.114\.513\.9M5Struct4\.548\.18\.59\.4M6Hier10\.446\.011\.418\.1M7Refine8\.246\.710\.117\.8M8Refine8\.246\.912\.518\.5M11Act9\.046\.611\.720\.1Table 14:Table for different substrate and model ALFWorld\-unseen Latency & Storage\.ModelSub\.Family𝑬𝟏E\_\{1\}𝑬𝟐E\_\{2\}𝑬𝟐,p90E\_\{2,\\text\{p90\}\}𝑬𝟑E\_\{3\}𝑬𝟒E\_\{4\}𝑬𝟏𝟒E\_\{14\}𝑬𝟏𝟓E\_\{15\}Qwen3\-8B–NoMem012,72813,200––65,392488M1Flat2\.17M14,08616,10418822090,048672M2Flat1\.49M13,74715,6920186,698647M3Text1\.43M13,65516,3181,62075487,502653M4Struct942K12,90314,8436,134981,070605M5Struct532K13,16715,7025,0219,41283,080620M6Hier25\.1M13,53915,64162419187,904656M7Refine3\.90M13,29715,9882,51917584,286629M8Refine5\.27M13,06915,3002,52117383,080620M9Weight15\.3M18,75722,0021,400–123,414921M11Act6\.9K12,72813,500––74,102553Qwen3\-32B\-AWQ–NoMem020,10020,300––78,122583M1Flat2\.17M20,50031,100192644144,7201,080M2Flat1\.49M20,30030,80001146,0601,090M3Text1\.43M20,90030,3001,6031,087127,300950M4Struct942K20,60028,6006,28412109,344816M5Struct532K20,73627,4005,1549,477114,034851M6Hier25\.1M20,40026,800642600109,478817M7Refine3\.90M20,20027,2002,57516595,944716M8Refine5\.27M20,50028,0002,571164118,724886M9Weight41\.9M15,82521,4003,800–82,008612M11Act6\.9K20,10022,000––104,922783Gemma\-4\-26B–NoMem016,06016,400––89,914671M1Flat2\.17M16,80021,600195184105,324786M2Flat1\.49M16,50021,40001100,902753M3Text1\.43M16,85720,4661,6491,009104,654781M4Struct942K20,00722,5176,08911126,228942M5Struct532K15,56419,9065,0899,358100,366749M6Hier25\.1M16,11421,09961917399,294741M7Refine3\.90M16,86621,1072,542161105,592788M8Refine5\.27M16,26920,1482,544159102,242763M11Act6\.9K16,03117,100––95,140710Table 15:Table for different substrate ALFWorld\-unseen Token Accounting\.Sub\.Family𝑬𝟓E\_\{5\}𝑬𝟔E\_\{6\}𝑬𝟕E\_\{7\}𝑬𝟖E\_\{8\}𝑬𝟗E\_\{9\}𝑬𝟏𝟎E\_\{10\}𝑬𝟏𝟏E\_\{11\}𝑬𝟏𝟐E\_\{12\}𝑬𝟏𝟑E\_\{13\}M1Flat00391K000000M2Flat003\.66M000000M3Text372K2008\.14M1\.84M1,340002\.21M1,540M4Struct1\.62M1995\.05M00480K3802\.10M579M5Struct1\.42M3852611\.21M770002\.63M1,155M6Hier357K460235K0084\.6K92442K552M7Refine398K200183K0000398K200M8Refine931K57713\.5K00277K3801\.21M957M9Weight––8\.7K––––––M11Act––118K––––––Table 16:Table for different substrate and model BigCodeBench\-Hard Performance & Quality\.ModelSub\.FamilyPass@1↑\\uparrow𝑷𝐚𝐯𝐠P\_\{\\mathrm\{avg\}\}↑\\uparrow𝑷𝟓P\_\{5\}Qwen3\-8B–NoMem8\.112\.4–M1Flat9\.520\.11\.000M2Flat10\.111\.91\.000M3Text8\.112\.41\.964M4Struct10\.120\.40\.982M5Struct15\.517\.10\.817M6Hier12\.222\.20\.820M7Refine12\.216\.95\.705M8Refine14\.924\.04\.432M9Weight7\.44\.6–M11Act13\.523\.6–Qwen3\-32B\-AWQ–NoMem17\.625\.6–M1Flat18\.224\.21\.000M2Flat19\.621\.81\.000M3Text18\.224\.51\.995M4Struct17\.622\.40\.910M5Struct16\.216\.60\.786M6Hier16\.216\.80\.799M7Refine17\.613\.52\.261M8Refine13\.523\.03\.575M9Weight14\.215\.5–M11Act13\.513\.8–Gemma\-4\-26B–NoMem14\.219\.5–M1Flat18\.226\.41\.000M2Flat19\.627\.01\.000M3Text14\.219\.52\.444M4Struct18\.926\.90\.982M5Struct20\.928\.40\.813M6Hier18\.926\.00\.813M7Refine16\.925\.11\.102M8Refine18\.226\.34\.471M11Act16\.924\.0–Table 17:Table for different substrate and model BigCodeBench\-Hard Latency & Storage\.ModelSub\.Family𝑬𝟏E\_\{1\}𝑬𝟐E\_\{2\}𝑬𝟐,p90E\_\{2,\\text\{p90\}\}𝑬𝟑E\_\{3\}𝑬𝟒E\_\{4\}𝑬𝟏𝟒E\_\{14\}𝑬𝟏𝟓E\_\{15\}Qwen3\-8B–NoMem150K3,8004,940––5924M1Flat1\.32M4,9766,6311991837765\.2M2Flat245K4,8536,6000\.137184\.9M3Text305K5,3006,8906,853467845\.3M4Struct150K4,6426,4321371387475\.0M5Struct239K28,27336,7008,77912,4324,18428\.3M6Hier3\.42M4,7646,2341,4441849946\.7M7Refine990K4,6386,2569,0291832,51917\.0M8Refine1\.22M5,0536,82510,4401972,83619\.2M9Weight14\.6M12,70016,510––1,92413M11Act151K16,80021,840––2,51617Qwen3\-32B\-AWQ–NoMem150K3,9005,070––5924M1Flat1\.32M5,0006,5002562257405M2Flat245K5,0006,5000\.137405M3Text296K5,4007,0209,3091,8431,1848M4Struct150K4,9006,37012,4603762,96020M5Struct249K31,22128,2397,88814,7234,62131\.2M6Hier3\.50M4,4005,7201,9952201,0367M7Refine1\.93M4,3005,59012,9941953,10821M8Refine1\.14M4,6005,98012,4471863,25622M9Weight38\.0M14,60018,980––2,22015M11Act161K21,30027,690––3,15221\.3Gemma\-4\-26B–NoMem150K19,80025,740––2,96020M1Flat1\.32M21,91628,5332132893,28622\.2M2Flat245K20,60227,8600\.023,04920\.6M3Text272K23,40030,42018,378303,46323\.4M4Struct150K22,37531,073173923,37222\.8M5Struct240K45,59547,4368,19413,0876,74845\.6M6Hier3\.47M24,31033,6142,7172514,14128\.0M7Refine3\.13M24,14931,5138,9213175,46036\.9M8Refine894K21,18629,8323,5933123,85426\.0M11Act115K31,60041,080––4,73632Table 18:Table for different substrate and model BigCodeBench\-Hard Token Accounting\.ModelSub\.Family𝑬𝟓E\_\{5\}𝑬𝟔E\_\{6\}𝑬𝟕E\_\{7\}𝑬𝟖E\_\{8\}𝑬𝟗E\_\{9\}𝑬𝟏𝟎E\_\{10\}𝑬𝟏𝟏E\_\{11\}𝑬𝟏𝟐E\_\{12\}𝑬𝟏𝟑E\_\{13\}Qwen3\-8B–NoMem––4\.75M––––––M1Flat002\.29M000000M2Flat002\.26M000000M3Text55\.9K362\.80M2\.84M148002\.90M184M4Struct002\.16M000000M5Struct0040\.7K116K14800116K148M6Hier44\.7K821\.14M0011\.5K1656\.2K98M7Refine244K200593K0000244K200M8Refine308K200550K00146K200454K400M9Weight––0––––––M11Act––1\.44M––––––Qwen3\-32B\-AWQ–NoMem––4\.75M––––––M1Flat002\.29M000000M2Flat002\.26M000000M3Text55\.6K362\.76M2\.81M148002\.86M184M4Struct524K199286K0000524K199M5Struct0041\.7K121K14800121K148M6Hier45\.1K86575K0013\.5K2458\.6K110M7Refine245K20091\.6K0000245K200M8Refine293K20084\.8K00148K200441K400M9Weight––0––––––M11Act––1\.48M––––––Gemma\-4\-26B–NoMem––4\.75M––––––M1Flat002\.29M000000M2Flat002\.26M000000M3Text52\.2K362\.26M2\.30M148002\.35M184M4Struct002\.16M000000M5Struct0040\.1K177K14800177K148M6Hier44\.4K821\.14M0012\.6K2457\.0K106M7Refine161K200901K0000161K200M8Refine204K200705K00129K200333K400M11Act––1\.30M––––––

## Appendix FAdditional Related Work

#### Memory\-augmented LLM agents\.

The design space of agent memory has expanded rapidly along multiple substrate axes\. On the retrieval side, early systems such as Generative Agents\[[39](https://arxiv.org/html/2608.15008#bib.bib4)\]and MemGPT\[[36](https://arxiv.org/html/2608.15008#bib.bib14)\]store raw observations in flat vector indices or paged text buffers and retrieve them via embedding similarity or explicit memory paging\. These designs provide a simple and general interface for long\-term recall, but they often rely on shallow similarity matching and provide limited structure for multi\-hop reasoning, conflict handling, or memory maintenance\. Subsequent work shifts toward more structured representations\. Mem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\]distills conversations into atomic facts, Zep\[[40](https://arxiv.org/html/2608.15008#bib.bib17)\]and MAGMA\[[17](https://arxiv.org/html/2608.15008#bib.bib25)\]build temporal or multi\-graph knowledge stores, and RAPTOR\[[45](https://arxiv.org/html/2608.15008#bib.bib19)\]and H\-MEM\[[49](https://arxiv.org/html/2608.15008#bib.bib18)\]organize memory into multi\-level hierarchies\. These systems improve inspectability and abstraction, but they also introduce additional design choices around write\-time summarization, graph construction, entity resolution, hierarchy depth, and retrieval traversal\.

More recently, refinement\-based and internal\-memory substrates have emerged\. Sleep\-time compute\[[28](https://arxiv.org/html/2608.15008#bib.bib20)\]uses offline computation to pre\-generate useful future responses, while MemSkill\[[79](https://arxiv.org/html/2608.15008#bib.bib21)\]abstracts procedural experience into reusable skill entries for self\-evolving agents\. On the internal\-memory side, SnapKV\[[26](https://arxiv.org/html/2608.15008#bib.bib23)\]and EpiCache\[[20](https://arxiv.org/html/2608.15008#bib.bib24)\]manage KV\-cache activations through attention\-guided eviction or episodic clustering, while adapter\-based approaches update parametric memory by fine\-tuning LoRA modules on accumulated sessions\[[80](https://arxiv.org/html/2608.15008#bib.bib22)\]\. Several recent systems combine multiple substrates into broader memory operating systems or multi\-agent memory architectures\[[27](https://arxiv.org/html/2608.15008#bib.bib7),[22](https://arxiv.org/html/2608.15008#bib.bib26),[63](https://arxiv.org/html/2608.15008#bib.bib27)\]\. Despite this diversity, most systems commit to a fixed substrate configuration at design time and evaluate that configuration in a narrow task regime\. Our work complements these efforts by isolating the substrate as the experimental variable and providing a controlled empirical comparison of when each substrate family is preferable\.

#### Memory evaluation benchmarks\.

Existing benchmarks span both user\-centric\[[50](https://arxiv.org/html/2608.15008#bib.bib29),[46](https://arxiv.org/html/2608.15008#bib.bib30)\]and agent\-centric\[[83](https://arxiv.org/html/2608.15008#bib.bib34),[73](https://arxiv.org/html/2608.15008#bib.bib35)\]memory demands, but they remain largely siloed within each regime\. On the user\-centric side, LoCoMo\[[30](https://arxiv.org/html/2608.15008#bib.bib9)\]evaluates long\-term conversational memory across five question types in dialogues averaging roughly 9K tokens over up to 35 sessions\. LongMemEval\[[65](https://arxiv.org/html/2608.15008#bib.bib2)\]scales memory evaluation to very long interaction histories and tests abilities such as information seeking, knowledge updates, abstention, and temporal reasoning\. MemoryAgentBench\[[12](https://arxiv.org/html/2608.15008#bib.bib28)\]frames memory evaluation as incremental multi\-turn interaction, covering accurate retrieval, long\-range understanding, test\-time learning, and conflict resolution\. Together, these benchmarks stress whether a memory system can preserve and retrieve relevant user information over long histories\.

On the agent\-centric side, ALFWorld\[[47](https://arxiv.org/html/2608.15008#bib.bib31)\]requires multi\-step household planning from text\-based observations and admissible actions, BigCodeBench\[[86](https://arxiv.org/html/2608.15008#bib.bib32)\]evaluates code generation with compositional function calls and realistic execution\-based grading, and SWE\-bench\[[18](https://arxiv.org/html/2608.15008#bib.bib33)\]tests repository\-level bug fixing where memory of codebase structure and previous patches may be useful\. These benchmarks stress a different regime: the agent must execute precise action or code\-generation decisions, and retrieved information can be harmful when it distracts from the current observation, instruction, or execution context\. This user\-centric/agent\-centric divide exposes fundamentally different retrieval dynamics\. User\-centric tasks often benefit from returning more candidate facts, while agent\-centric tasks can degrade when imprecise retrieval dilutes attention over action\-critical context\. However, existing evaluations rarely test the same set of memory substrates across both regimes under controlled conditions\. Our work bridges this gap by evaluating the same substrate configurations across benchmarks from both sides of the spectrum\.

#### Empirical evaluation practices in agent memory\.

The metrics reported by existing memory systems provide an incomplete picture of substrate behavior\. Nearly all systems report some form of end\-task accuracy, token F1, exact match, task success, pass rate, or LLM\-as\-a\-judge score, but the dimensions that govern deployment feasibility are often omitted\. Write cost, including the number of LLM calls and tokens consumed during memory construction, is rarely measured systematically\. Management cost, including the overhead of deduplication, conflict resolution, summarization, graph maintenance, and memory rewriting, is likewise underreported, even in systems such as A\-Mem\[[70](https://arxiv.org/html/2608.15008#bib.bib6)\]and Zep\[[40](https://arxiv.org/html/2608.15008#bib.bib17)\], whose architectures explicitly include management operations\. Compression ratio, which quantifies how much information each stored token preserves, has also not been consistently measured across substrates\.

Among efficiency dimensions that are occasionally reported, coverage remains sparse\. For example, Mem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\]reports token consumption and p95 latency but does not provide a complete account of storage footprint or write overhead, while Zep reports latency and token cost without a full cross\-substrate accounting of memory construction cost\. This metric narrowness means that a substrate consuming a large number of auxiliary LLM calls can appear indistinguishable from a substrate requiring no auxiliary calls when both are evaluated only by downstream accuracy\. In addition, even the accuracy metrics that are commonly reported may be less stable than assumed\. Recent work\[[25](https://arxiv.org/html/2608.15008#bib.bib36)\]shows that reordering score rubrics or altering reference answer quality in LLM\-as\-a\-judge prompts can disrupt scoring stability across judge models, yet published memory evaluations often use different judge prompts and scoring configurations\. Our work fills these gaps by measuring task performance, computational efficiency, and memory quality across all substrate configurations under a single controlled setup\.

## Appendix GLimitations

#### Excluded production\-grade systems\.

We exclude three widely cited production\-grade memory systems, MemGPT\[[36](https://arxiv.org/html/2608.15008#bib.bib14)\], Mem0\[[3](https://arxiv.org/html/2608.15008#bib.bib13)\], and Zep\[[40](https://arxiv.org/html/2608.15008#bib.bib17)\], from the main cross\-substrate comparison because their auxiliary\-LLM cost \(2,700–9,000 calls and 11–23 hours per LoCoMo run\) is one to two orders of magnitude above the lighter substrates, which would dominate the wall\-clock and token\-count columns and obscure architectural differences\. Their core retrieval ideas are already represented in the main comparison by lighter, paper\-faithful substrates \(M3, M5\); see Appendix[B\.3](https://arxiv.org/html/2608.15008#A2.SS3)\.

#### Implementation deviations from reference papers\.

Two substrates deviate from their reference implementations\. M8 replaces MemSkill’s PPO\-trained controller with a zero\-shot LLM that consumes all refined skill bundles prefilled into context, because no other substrate involves RL training and keeping M8 RL\-driven would confound the cross\-substrate comparison\. M11 replaces EpiCache’s in\-place KV\-cache edits with utterance\-level re\-prefill, because Qwen3’s hybrid attention architecture is not supported by the authors’ released kernels\. Both deviations preserve the core design principle of each method; full details are in Appendix[B](https://arxiv.org/html/2608.15008#A2)\.

## Appendix HDeclaration of LLM Usage

LLMs were used by the authors only for routine writing assistance, such as grammar polishing, phrasing refinement, and LaTeX formatting\. They were not used to generate research ideas, design experiments, analyze results, or produce any of the empirical findings reported in this paper\. All scientific contributions, including the harness design, substrate implementations, metric taxonomy, and analysis, are the original work of the authors\.