Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents

arXiv cs.AI Papers

Summary

This paper presents HORMA, a hierarchical organize-and-retrieve memory agent that organizes agent experiences into a file-system-like structure for efficient retrieval, improving performance on long-horizon tasks while reducing token usage.

arXiv:2606.11680v1 Announce Type: new Abstract: Large language model (LLM) agents struggle with long-horizon tasks due to their inherent statelessness, requiring all task-relevant information to be encoded in growing input contexts. The resulting degraded reasoning quality, increased inference cost, and higher latency necessitate efficient working memory mechanisms. However, existing approaches either rely on lossy compression or similarity-based retrieval, which often fail to capture temporal structure and causal dependencies required for multi-step agentic tasks. In this work, we present HORMA, a Hierarchical Organize-and-Retrieve Memory Agent that organizes experience into a file-system-like hierarchical structure, where summarized entities are linked to the corresponding raw trajectories, enabling efficient access without losing detailed information. HORMA decomposes working memory into two stages: structured memory construction and navigation-based retrieval. The construction module iteratively refines how experiences are structured by distinguishing between failures caused by missing information and those caused by misleading or overloaded context. The navigation module retrieves task-relevant context by traversing the hierarchy using a lightweight agent trained with reinforcement learning to select minimal yet sufficient context, thereby reducing latency along the critical execution path. Across ALFWorld, LoCoMo, and LongMemEval, HORMA improves task performance under constrained context budgets while requiring at most 22.17% of the baseline token usage in long conversation tasks. Compared to existing methods, it consistently achieves better efficiency-performance trade-offs and generalizes effectively to unseen tasks.
Original Article
View Cached Full Text

Cached at: 06/11/26, 01:48 PM

# Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents
Source: [https://arxiv.org/html/2606.11680](https://arxiv.org/html/2606.11680)
Hao\-Lun Hsu1, Nikki Lijing Kuang2, Boyi Liu2, Zhewei Yao2, Yuxiong He2 1Duke University2Snowflake AI Research

###### Abstract

Large language model \(LLM\) agents struggle with long\-horizon tasks due to their inherent statelessness, requiring all task\-relevant information to be encoded in growing input contexts\. The resulting degraded reasoning quality, increased inference cost, and higher latency necessitate efficient working memory mechanisms\. However, existing approaches either rely on lossy compression or similarity\-based retrieval, which often fail to capture temporal structure and causal dependencies required for multi\-step agentic tasks\. In this work, we present HORMA, aHierarchicalOrganize\-and\-RetrieveMemoryAgent that organizes experience into a file\-system\-like hierarchical structure, where summarized entities are linked to the corresponding raw trajectories, enabling efficient access without losing detailed information\. HORMA decomposes working memory into two stages: structured memory construction and navigation\-based retrieval\. The construction module iteratively refines how experiences are structured by distinguishing between failures caused by missing information and those caused by misleading or overloaded context\. The navigation module retrieves task\-relevant context by traversing the hierarchy using a lightweight agent trained with reinforcement learning to select minimal yet sufficient context, thereby reducing latency along the critical execution path\. Across ALFWorld, LoCoMo, and LongMemEval, HORMA improves task performance under constrained context budgets while requiring at most 22\.17% of the baseline token usage in long conversation tasks\. Compared to existing methods, it consistently achieves better efficiency\-performance trade\-offs and generalizes effectively to unseen tasks\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2606.11680v1/x1.png)Figure 1:Overview of HORMA Framework\. The system aims to solve the long\-horizon problem \(0\) and explicitly decouples working memory into two specialized modules \(1\) & \(2\), accompanied by its dedicated retrieval training and verification benchmarks \(3 & 4\): \(1\) Hierarchical Management Agent, which organizes raw trajectories into structured, linked notes within a file\-system workspace using recursive skill refinement; \(2\) Hierarchical Retrieval Agent, which navigates this hierarchy using Bash tools and terminal actions to select task\-relevant context\.In agentic systems, working memory functions as a short\-term workspace that allows the agent to maintain task\-relevant information in complex long\-horizon tasks\. Existing approaches suffer from two key limitations: agents either act as history hoarders \(see Figure[1](https://arxiv.org/html/2606.11680#S1.F1)\(0\)\), retaining large amounts of history\[[40](https://arxiv.org/html/2606.11680#bib.bib65),[49](https://arxiv.org/html/2606.11680#bib.bib66)\], leading to context overload\[[1](https://arxiv.org/html/2606.11680#bib.bib62)\], information dilution\[[20](https://arxiv.org/html/2606.11680#bib.bib53)\], prohibitive latency and high inference cost\[[14](https://arxiv.org/html/2606.11680#bib.bib30)\], or rely on lossy compression mechanisms\[[8](https://arxiv.org/html/2606.11680#bib.bib24),[9](https://arxiv.org/html/2606.11680#bib.bib25),[18](https://arxiv.org/html/2606.11680#bib.bib64),[52](https://arxiv.org/html/2606.11680#bib.bib63)\], including summarization\[[21](https://arxiv.org/html/2606.11680#bib.bib41),[37](https://arxiv.org/html/2606.11680#bib.bib39),[42](https://arxiv.org/html/2606.11680#bib.bib29)\]and context folding\[[34](https://arxiv.org/html/2606.11680#bib.bib31),[51](https://arxiv.org/html/2606.11680#bib.bib32)\], which irreversibly discard fine\-grained information necessary for downstream reasoning\[[16](https://arxiv.org/html/2606.11680#bib.bib38),[17](https://arxiv.org/html/2606.11680#bib.bib11),[25](https://arxiv.org/html/2606.11680#bib.bib37),[27](https://arxiv.org/html/2606.11680#bib.bib51),[43](https://arxiv.org/html/2606.11680#bib.bib18)\]\.

To address these limitations, recent work has delegated working memory to explicit external storage systems\[[13](https://arxiv.org/html/2606.11680#bib.bib47),[24](https://arxiv.org/html/2606.11680#bib.bib49),[3](https://arxiv.org/html/2606.11680#bib.bib43),[46](https://arxiv.org/html/2606.11680#bib.bib44),[47](https://arxiv.org/html/2606.11680#bib.bib5),[63](https://arxiv.org/html/2606.11680#bib.bib13)\]\. Despite improving storage scalability, existing external memory architectures typically organize experience as flat collections of independent entries retrieved through semantic similarity\[[3](https://arxiv.org/html/2606.11680#bib.bib43),[15](https://arxiv.org/html/2606.11680#bib.bib50),[28](https://arxiv.org/html/2606.11680#bib.bib19),[46](https://arxiv.org/html/2606.11680#bib.bib44)\]\. Such designs fail to capture temporal hierarchies and causal dependencies accumulated over long interaction horizons\. As a result, retrieval often degenerates into shallow semantic matching that surfaces temporally inconsistent or contextually irrelevant information\[[66](https://arxiv.org/html/2606.11680#bib.bib12),[47](https://arxiv.org/html/2606.11680#bib.bib5),[60](https://arxiv.org/html/2606.11680#bib.bib3),[4](https://arxiv.org/html/2606.11680#bib.bib4)\]\. Effective long\-horizon memory therefore requires not only selective retention, but also hierarchical organization of accumulated experience into reusable and semantically coherent structures\[[13](https://arxiv.org/html/2606.11680#bib.bib47),[45](https://arxiv.org/html/2606.11680#bib.bib14),[57](https://arxiv.org/html/2606.11680#bib.bib20)\], thereby improving downstream task performance\.

To support such structured long\-horizon memory, most existing memory systems treat memory construction and retrieval as a monolithic system that is jointly optimized within a unified framework\[[4](https://arxiv.org/html/2606.11680#bib.bib4),[60](https://arxiv.org/html/2606.11680#bib.bib3),[65](https://arxiv.org/html/2606.11680#bib.bib46),[53](https://arxiv.org/html/2606.11680#bib.bib36),[34](https://arxiv.org/html/2606.11680#bib.bib31)\]\. However, memory construction and retrieval serve fundamentally different functional roles and admit distinct optimization strategies\. Memory construction determines how experiences are abstracted and structurally organized over time\. Its impact often manifests only after extended interactions, making its quality difficult to assess through immediate task outcomes\. Furthermore, modern proprietary LLMs already demonstrate strong capabilities for semantic abstraction and hierarchical structuring\[[10](https://arxiv.org/html/2606.11680#bib.bib27),[32](https://arxiv.org/html/2606.11680#bib.bib28),[12](https://arxiv.org/html/2606.11680#bib.bib26)\], suggesting that effective memory structures can often be induced directly from their existing capabilities\. In contrast, memory retrieval determines which information is exposed to the agent at inference time and therefore directly influences downstream decisions\. Consequently, retrieval is naturally more amenable to explicit optimization\.

This distinction becomes particularly problematic in reinforcement learning \(RL\)\-based memory systems\. Jointly optimizing memory construction and retrieval through sparse task\-level rewards introduces a severecredit assignment gap\[[33](https://arxiv.org/html/2606.11680#bib.bib7)\]: when an agent fails a long\-horizon task, it becomes unclear whether the failure originates from poor memory organization, inaccurate retrieval, or downstream reasoning\[[55](https://arxiv.org/html/2606.11680#bib.bib40),[59](https://arxiv.org/html/2606.11680#bib.bib67),[35](https://arxiv.org/html/2606.11680#bib.bib22)\]\. As a result, sparse outcome rewards provide weak and entangled supervision signals for both components\. Existing attempts to mitigate this issue through intermediate or multi\-level rewards partially alleviate the optimization difficulty, but they often require carefully engineered reward designs\[[4](https://arxiv.org/html/2606.11680#bib.bib4),[39](https://arxiv.org/html/2606.11680#bib.bib45)\]and generalize poorly beyond conversational settings\[[4](https://arxiv.org/html/2606.11680#bib.bib4)\]\.

Motivated by these observations, we propose HORMA, aHierarchicalOrganize\-and\-RetrieveMemoryAgent that explicitly decouples memory construction from retrieval within a shared hierarchical file\-system workspace \(Figure[1](https://arxiv.org/html/2606.11680#S1.F1)\)\. Both modules are implemented as tool\-using agents that interact with the workspace through executable file\-system operations and Bash tools, while serving distinct functional roles\. The memory construction module is responsible for maintaining semantically organized memory structures that provide stable abstractions for long\-horizon reasoning\. Rather than optimizing memory construction directly through unstable long\-horizon RL, HORMA treats memory construction as a continual management skill acquisition process\. We initialize a domain\-agnostic construction policy using proprietary LLMs with strong hierarchical reasoning capabilities\[[10](https://arxiv.org/html/2606.11680#bib.bib27),[32](https://arxiv.org/html/2606.11680#bib.bib28)\], and iteratively refine this policy through contrastive analysis between successful and failed trajectories\. Over time, the construction module accumulates reusable memory management skills\[[2](https://arxiv.org/html/2606.11680#bib.bib55)\]that transfer across tasks without relearning memory construction from scratch\.

In contrast, the retrieval module operates directly on the inference path and is responsible for efficiently extracting task\-relevant context from the hierarchical workspace\. Instead of relying on flat semantic retrieval, the retrieval agent actively navigates the organized memory structure through dedicated Bash tools, enabling more temporally consistent and causally grounded access to historical information\[[19](https://arxiv.org/html/2606.11680#bib.bib15),[45](https://arxiv.org/html/2606.11680#bib.bib14)\]\. We further introduce two executable actions,select,done, that allow the agent to iteratively verify retrieved memory and uncover missing contextual details\[[48](https://arxiv.org/html/2606.11680#bib.bib8)\]\. To enable retrieval\-specific optimization beyond sparse task\-level supervision, we introduce an auxiliary learning signal \(i\.e\., evidence\-grounded retrieval reward\) based on overlap between retrieved context and task\-relevant ground\-truth evidence\. This provides direct, fine\-grained feedback on retrieval quality that is decoupled from downstream reasoning performance\. Leveraging this signal, we optimize the retrieval policy using RL on a lightweight backbone, enabling efficient context extraction under constrained context budgets while reducing computational overhead\.

We evaluate HORMA on three challenging long\-horizon benchmarks\. On ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\], HORMA achieves higher success rates under both small and large context limits while improving Pareto efficiency between interaction steps and token usage\. On long\-conversation benchmarks, HORMA significantly reduces context consumption, using only 3\.07%–22\.17% of the tokens required by different baselines on LoCoMo\[[22](https://arxiv.org/html/2606.11680#bib.bib23)\]and 1\.24%–16\.19% on LongMemEval\[[41](https://arxiv.org/html/2606.11680#bib.bib2)\]\. Notably, the learned lightweight retrieval agent exhibits strong out\-of\-distribution generalization on LongMemEval, outperforming all baselines, including those without context constraints\. Overall, these results demonstrate that explicitly decoupling memory management and retrieval yields a more efficient, interpretable, and scalable mechanism for working memory under strict context limits\.

## 2Related Work

#### Working Memory in LLM\-Based Agents\.

Working memory approaches differ in whether they emphasize compression and structuring prior to context entry or dynamic, policy\-driven maintenance during execution, but both aim to mitigate context saturation while preserving task\-relevant information for reasoning\[[7](https://arxiv.org/html/2606.11680#bib.bib35)\]\. One line of work focuses on pre\- or in\-context state formation, compressing or restructuring interaction history before or as it enters the active context\. Methods such as ReSum\[[42](https://arxiv.org/html/2606.11680#bib.bib29)\]and ACON\[[14](https://arxiv.org/html/2606.11680#bib.bib30)\]perform learned compression of trajectories into compact reasoning states, while hierarchical folding\[[34](https://arxiv.org/html/2606.11680#bib.bib31),[51](https://arxiv.org/html/2606.11680#bib.bib32)\]and subgoal\-based methods\[[6](https://arxiv.org/html/2606.11680#bib.bib42),[38](https://arxiv.org/html/2606.11680#bib.bib17)\]introduce restructuring to organize long\-horizon interactions into manageable abstractions\. A second line of work addresses online maintenance of working memory during execution, directly operating on the evolving context under fixed budgets\. Approaches\[[53](https://arxiv.org/html/2606.11680#bib.bib36),[55](https://arxiv.org/html/2606.11680#bib.bib40),[61](https://arxiv.org/html/2606.11680#bib.bib34)\]use recurrent updates to maintain compact states, while policy\-based methods\[[3](https://arxiv.org/html/2606.11680#bib.bib43),[47](https://arxiv.org/html/2606.11680#bib.bib5)\]treat memory operations as actions that decide what to store, update, or discard during interaction\.

#### RL for LLMs\.

Reinforcement learning \(RL\) has become a core technique for improving performance in LLMs\[[29](https://arxiv.org/html/2606.11680#bib.bib10),[30](https://arxiv.org/html/2606.11680#bib.bib60),[26](https://arxiv.org/html/2606.11680#bib.bib9)\]\. RL enables the emergence of reasoning\-centric models such as DeepSeek\-R1\[[5](https://arxiv.org/html/2606.11680#bib.bib59)\]and Search\-R1\[[11](https://arxiv.org/html/2606.11680#bib.bib58)\]\. However, these approaches typically rely on retaining the entire interaction trajectory, leading to scalability and efficiency limitations in long\-horizon settings\. Recent work has begun exploring memory construction and management through RL\. Early approaches\[[65](https://arxiv.org/html/2606.11680#bib.bib46),[53](https://arxiv.org/html/2606.11680#bib.bib36)\]train models to maintain lightweight text\-based memories\. Subsequent methods introduce richer memory representations together with simplified memory tool interfaces\[[47](https://arxiv.org/html/2606.11680#bib.bib5),[62](https://arxiv.org/html/2606.11680#bib.bib57),[58](https://arxiv.org/html/2606.11680#bib.bib56)\]\. In contrast to prior end\-to\-end memory\-augmented RL approaches, we formulate memory retrieval as a navigation problem and purely train a dedicated retrieval agent with RL, mitigating the credit assignment challenges\.

#### Memory and Skill Evolution\.

The ability to abstract complex experiences into reusable skills is fundamental to self\-improving agents\[[2](https://arxiv.org/html/2606.11680#bib.bib55)\], enabling memory\-guided decision\-making\. Prior work uses RL to select or refine skills within an agent’s repertoire\. MemSkill\[[59](https://arxiv.org/html/2606.11680#bib.bib67)\]treats memory operations as learnable skills and trains a controller via RL to select appropriate memory behaviors\. SkillRL\[[44](https://arxiv.org/html/2606.11680#bib.bib69)\]jointly evolves the agent policy and a SkillBank by distilling successful trajectories into reusable strategies\. Representing skills as executable code can further improve precision and reusability\. PolySkill\[[54](https://arxiv.org/html/2606.11680#bib.bib54)\]separates high\-level skill abstractions from site\-specific implementations to transfer skills across different web interfaces, while skill library\-integrated GRPO\[[36](https://arxiv.org/html/2606.11680#bib.bib68)\]learns reusable action\-sequence skills from long\-horizon task chains, albeit with high training costs\. While effective, these parametric approaches often incur expensive training and reduced generalization\. In contrast, non\-parametric methods improve agent behavior at inference time without updating model parameters\. Skill\-Pro\[[23](https://arxiv.org/html/2606.11680#bib.bib70)\]learns reusable procedural skills from interaction experience, and MCE\[[50](https://arxiv.org/html/2606.11680#bib.bib52)\]evolves skills through anagentic crossovermechanism that recombines successful past behaviors\. Similarly, our HORMA framework models memory management skills as non\-parametric updates, enabling continual adaptation without modifying the underlying LLM\.

## 3Preliminaries

We consider long\-horizon decision\-making settings in which an LLM agent must solve a task specified by a natural language queryqqthrough multi\-step interaction with an environment\. At each stept∈\[T\]t\\in\[T\], the agent receives an observationoto\_\{t\}and produces an actionata\_\{t\}, forming an interaction trajectory𝐇t−1=\(o0,a0,o1,…,ot−1,at−1\)\\mathbf\{H\}\_\{t\-1\}=\(o\_\{0\},a\_\{0\},o\_\{1\},\\dots,o\_\{t\-1\},a\_\{t\-1\}\)\. The primary agent is modeled as an LLM\-based policyMθM\_\{\\theta\}with frozen parameters:

Mθ​\(at∣ot,𝐇t−1,q;𝒫main\),\\displaystyle M\_\{\\theta\}\(a\_\{t\}\\mid o\_\{t\},\\mathbf\{H\}\_\{t\-1\},q;\\mathcal\{P\}\_\{\\text\{main\}\}\),\(1\)where𝒫main\\mathcal\{P\}\_\{\\text\{main\}\}specifies the prompting context, including environment descriptions, tool specifications, output formats, and few\-shot demonstrations\.

#### The Context Bottleneck\.

Whileqqand𝒫main\\mathcal\{P\}\_\{\\text\{main\}\}remain fixed, the interaction history𝐇t\\mathbf\{H\}\_\{t\}grows with trajectory length\. Under a finite context window of sizeWW, tokens exceeding the limit must be truncated, resulting in loss of long\-range dependencies\. Moreover, long histories introduce substantial computational overhead and information dilution, where task\-relevant signals become increasingly obscured by irrelevant context\. As trajectories grow, the agent must not only retain information under strict context budgets, but also organize and retrieve relevant information efficiently across long temporal horizons\.

## 4Hierarchical Organize\-and\-Retrieve Memory Agent

We present HORMA \(HierarchicalOrganize\-and\-RetrieveMemoryAgent\), a framework that augments a primary LLM agentMθM\_\{\\theta\}with an external working memory system\. HORMA is motivated by the observation that memory construction and memory retrieval operate at fundamentally different temporal and functional scales\. Memory construction shapes the long\-term structure of stored information and induces delayed effects on downstream reasoning, whereas retrieval directly affects per\-step inference quality on the execution path\. We therefore explicitly decouple these processes into two specialized modules: a memory managerMmM\_\{m\}responsible for organizing information and a retrieval agentMrM\_\{r\}responsible for selecting task\-relevant context\.

Both modules are implemented as tool\-using agents that interact with a shared hierarchical memory workspace exclusively through executable file\-system operations and Bash tools\. This shared grounded interface enables interpretable memory manipulation, explicit provenance tracking, and modular optimization of memory management and retrieval behaviors\. The overall architecture is illustrated in Figure[1](https://arxiv.org/html/2606.11680#S1.F1)\(1\) & \(2\)\.

### 4\.1Memory\-Augmented Agent Policy

To address the limitations of growing interaction histories, HORMA externalizes working memory into a persistent hierarchical workspace that evolves alongside agent interaction\. Rather than treating memory as a flat sequence of tokens, the workspace maintains structured and navigable representations of past experience, enabling memory construction and retrieval to operate independently from the primary agent’s context window\. We formalize the framework using a Memory\-augmented Markov Decision Process \(M\-MDP\)\[[64](https://arxiv.org/html/2606.11680#bib.bib61)\], defined as\(𝒮,𝒪,𝒜,𝒯,ℛ,ℱ\)\(\\mathcal\{S\},\\mathcal\{O\},\\mathcal\{A\},\\mathcal\{T\},\\mathcal\{R\},\\mathcal\{F\}\)\. Here,𝒮\\mathcal\{S\},𝒪\\mathcal\{O\}, and𝒜\\mathcal\{A\}denote state, observation, and action spaces;𝒯\\mathcal\{T\}defines environment dynamics;ℛ\\mathcal\{R\}provides a binary task rewardRT∈\{0,1\}R\_\{T\}\\in\\\{0,1\\\}at the final stepTT, andℱt\\mathcal\{F\}\_\{t\}denotes the external memory state at timett\.

The memory workspaceℱt\\mathcal\{F\}\_\{t\}consists of structured files and directories \(e\.g\., entity logs, event summaries, state trackers\) that evolve through a memory transition operator:

ℱt\+1=𝒯ℱ​\(ℱt,at,ot\)\.\\displaystyle\\mathcal\{F\}\_\{t\+1\}=\\mathcal\{T\}\_\{\\mathcal\{F\}\}\(\\mathcal\{F\}\_\{t\},a\_\{t\},o\_\{t\}\)\.\(2\)
Unlike interaction history stored directly in the context window,ℱt\\mathcal\{F\}\_\{t\}persists externally and can scale with task complexity\. While𝒯ℱ\\mathcal\{T\}\_\{\\mathcal\{F\}\}could be implemented using hand\-crafted heuristics such as fixed summarization or rule\-based file updates, such approaches are often brittle and fail to generalize across domains requiring complex management and retrieval strategies\. To overcome these limitations, HORMA operationalizes this transition by framing memory construction as an agentic management task driven by a memory managerMmM\_\{m\}, while decomposing downstream per\-step action generation into localized retrieval and execution:

π​\(at∣ot,ℱt,q\)=Mr​\(𝐂t∣ℱt,q\)​Mθ​\(at∣ot,𝐂t,q;𝒫main\),\\pi\(a\_\{t\}\\mid o\_\{t\},\\mathcal\{F\}\_\{t\},q\)=M\_\{r\}\(\\mathbf\{C\}\_\{t\}\\mid\\mathcal\{F\}\_\{t\},q\)\\;M\_\{\\theta\}\(a\_\{t\}\\mid o\_\{t\},\\mathbf\{C\}\_\{t\},q;\\mathcal\{P\}\_\{\\text\{main\}\}\),where the retrieval moduleMrM\_\{r\}selects context𝐂t⊆ℱt\\mathbf\{C\}\_\{t\}\\subseteq\\mathcal\{F\}\_\{t\}to ground the primary agentMθM\_\{\\theta\}\.

#### Generalization to Long\-Horizon Conversations\.

Although formulated for interactive environments, the framework naturally extends to long\-horizon conversational settings such as long\-form QA and dialogue memory benchmarks\[[22](https://arxiv.org/html/2606.11680#bib.bib23),[41](https://arxiv.org/html/2606.11680#bib.bib2)\]\. In this setting, the interaction history becomes

𝐇=\(u0,r0,u1,r1,…,uT,rT\),\\mathbf\{H\}=\(u\_\{0\},r\_\{0\},u\_\{1\},r\_\{1\},\\dots,u\_\{T\},r\_\{T\}\),whereutu\_\{t\}andrtr\_\{t\}denote user and assistant turns\. Standard approaches directly condition on the entire dialogue history:

Mθ​\(a∣𝐇,q;𝒫main\),\\displaystyle M\_\{\\theta\}\(a\\mid\\mathbf\{H\},q;\\mathcal\{P\}\_\{\\text\{main\}\}\),\(3\)which becomes increasingly inefficient as conversations grow\. HORMA instead retrieves compact task\-relevant context from external memory:

π​\(a∣ℱ,q\)=Mr​\(𝐂∣ℱ,q\)​Mθ​\(a∣𝐂,q;𝒫main\),\\pi\(a\\mid\\mathcal\{F\},q\)=M\_\{r\}\(\\mathbf\{C\}\\mid\\mathcal\{F\},q\)\\;M\_\{\\theta\}\(a\\mid\\mathbf\{C\},q;\\mathcal\{P\}\_\{\\text\{main\}\}\),enabling scalable reasoning over long conversational histories under strict context limits\.

### 4\.2The Grounded Workspace: Hierarchy and Provenance

HORMA organizes memoryℱt\\mathcal\{F\}\_\{t\}as a hierarchical file system rather than a flat memory buffer\[[47](https://arxiv.org/html/2606.11680#bib.bib5)\]\. This design enables memory construction to operate over semantically meaningful structures while supporting efficient retrieval through directory navigation and localized search\. For each interaction\(at,ot\)\(a\_\{t\},o\_\{t\}\)or dialogue turn\(ut,rt\)\(u\_\{t\},r\_\{t\}\), the memory managerMmM\_\{m\}first archives the raw trajectory into a timestamped directory\. It then selectively synthesizes structured notes based on task relevance\. Each synthesized note stores compact task\-relevant abstractions together with temporal metadata and references to underlying raw trajectories, enabling efficient retrieval without sacrificing provenance or recoverability\.

### 4\.3Memory Management via Skill Evolution

The memory managerMmM\_\{m\}maintains the workspaceℱt\\mathcal\{F\}\_\{t\}by issuing executable file\-system operations through Bash commands \(e\.g\.,mkdir,nano,mv\)\. Its role is to transform raw trajectories into semantically organized memory structures that support long\-horizon reasoning\. Because memory construction operates over long temporal horizons and induces delayed structural effects on downstream reasoning, directly optimizing memory structure through sparse task rewards is highly unstable\. We therefore treat memory construction as a structure induction problem\. We initializeMmM\_\{m\}with a domain\-agnostic prompt𝒫m\(0\)\\mathcal\{P\}\_\{m\}^\{\(0\)\}that specifies high\-level organizational principles such as entity tracking, event abstraction, and relation grouping\. This initialization leverages the strong hierarchical reasoning and abstraction capabilities already exhibited by frontier LLMs\.

#### Recursive Skill Refinement\.

To improve memory construction, we iteratively refine𝒫m\\mathcal\{P\}\_\{m\}using task trajectories\. We identify failure modes by comparing performance using raw history𝐇\\mathbf\{H\}\(unconstrained baseline\) versus managed context𝐇′\\mathbf\{H^\{\\prime\}\}\(HORMA\)\. We categorize failures based on whether structured memory helps or hinders performance relative to unstructured history, enabling us to distinguish cases where memory construction removes information versus cases where it improves reasoning by filtering noise\. We define two contrastive subsets:

- •𝒟exo\\mathcal\{D\}\_\{\\text\{exo\}\}\(Exogenous set\): tasks where𝐇\\mathbf\{H\}succeeds but𝐇′\\mathbf\{H^\{\\prime\}\}fails, indicating information loss during memory construction;
- •𝒟end\\mathcal\{D\}\_\{\\text\{end\}\}\(Endogenous set\): tasks where𝐇′\\mathbf\{H^\{\\prime\}\}succeeds but𝐇\\mathbf\{H\}fails, indicating that structured memory mitigates issues such as hallucination orlost\-in\-the\-middleeffects\[[20](https://arxiv.org/html/2606.11680#bib.bib53)\]\.

For each task, we generate natural language feedback via contrastive analysis:

Feedbacki=LLM​\(Feedback Instruction,𝐇,𝐇′\)\.\\text\{Feedback\}\_\{i\}=\\text\{LLM\}\(\\text\{Feedback Instruction\},\\mathbf\{H\},\\mathbf\{H^\{\\prime\}\}\)\.\(4\)
We aggregate feedback across tasks to iteratively refine the memory management policy with additional memory management skills:

𝒫m\(k\+1\)=LLM​\(Skill Augmentation Instruction,𝒫m\(k\),\{Feedbacki\}i=1n\),\\mathcal\{P\}\_\{m\}^\{\(k\+1\)\}=\\text\{LLM\}\(\\text\{Skill Augmentation Instruction\},\\mathcal\{P\}\_\{m\}^\{\(k\)\},\\\{\\text\{Feedback\}\_\{i\}\\\}\_\{i=1\}^\{n\}\),\(5\)which can be viewed as a form of textual gradient descent\[[56](https://arxiv.org/html/2606.11680#bib.bib16)\]\. This process yields a growing library of domain\-specific memory management skills \(exogenous and endogenous\)\. The overall memory management pipeline is illustrated in Figure[1](https://arxiv.org/html/2606.11680#S1.F1)\(1\)\.

### 4\.4Memory Retrieval via Reinforcement Learning

The retrieval agentMrM\_\{r\}navigates the memory workspaceℱt\\mathcal\{F\}\_\{t\}to construct task\-relevant context𝐂t\\mathbf\{C\}\_\{t\}\. Unlike similarity\-based retrieval, which may retrieve temporally inconsistent or causally irrelevant information,MrM\_\{r\}exploits explicit structural signals such as directory hierarchy, temporal organization, and provenance metadata to efficiently locate relevant content, illustrated in Figure[1](https://arxiv.org/html/2606.11680#S1.F1)\(2\)\.

Retrieval decisions lie directly on the execution path and admit localized behavioral feedback, making retrieval naturally amenable to sequential policy optimization\. We therefore formulate retrieval as a grounded decision\-making process over executable file\-system operations\.

#### Grounded Action Space\.

The retrieval agent interacts with the workspace using Bash commands such asls,grep,cd, andcat\. We further augment the action space with two terminal actions:

\{select,done\}\.\\\{\\texttt\{select\},\\texttt\{done\}\\\}\.
Theselectaction adds verified content to the retrieved context𝐂t\\mathbf\{C\}\_\{t\}, whiledoneterminates retrieval once sufficient evidence has been collected\. The primary agent acts conditioned on retrieved context:

Mθ​\(at∣ot,𝐂t,q;𝒫main\)\.\\displaystyle M\_\{\\theta\}\(a\_\{t\}\\mid o\_\{t\},\\mathbf\{C\}\_\{t\},q;\\mathcal\{P\}\_\{\\text\{main\}\}\)\.\(6\)
This design improves efficiency by allowing retrieval to operate over compact structured notes rather than full trajectories while selectively expanding into raw interaction traces only when necessary\.

#### RL\-based Policy Optimization\.

To improve retrieval reliability under strict context constraints, we optimize the retrieval policyMrM\_\{r\}using Group Relative Policy Optimization \(GRPO\)\[[30](https://arxiv.org/html/2606.11680#bib.bib60)\]\.

To encourage precise yet compact context construction, we define an evidence\-grounded retrieval reward based on overlap between retrieved context𝐂t\\mathbf\{C\}\_\{t\}and ground\-truth evidenceEE:

J​\(𝐂t,E\)=\|𝐂t∩E\|\|𝐂t∪E\|\.J\(\\mathbf\{C\}\_\{t\},E\)=\\frac\{\|\\mathbf\{C\}\_\{t\}\\cap E\|\}\{\|\\mathbf\{C\}\_\{t\}\\cup E\|\}\.\(7\)
This reward encourages retrieval of relevant evidence while penalizing irrelevant or redundant context\. Over time, the retrieval agent learns efficient navigation strategies such as hierarchical exploration, recovery from failed commands, and refinement of search trajectories, enabling robust and lightweight retrieval under limited context budgets\.

## 5Experiments

### 5\.1Experimental Setup

#### Benchmarks\.

We evaluate our methods on three benchmarks: ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\], LoCoMo\[[22](https://arxiv.org/html/2606.11680#bib.bib23)\], and LongMemEval\[[41](https://arxiv.org/html/2606.11680#bib.bib2)\]\. ALFWorld is an embodied interactive task benchmark, where we evaluate134134tasks across66task categories\. LoCoMo and LongMemEval are long\-horizon conversational benchmarks designed to test memory construction from extended dialogue histories\. LoCoMo contains1010conversations; we evaluate on519519question\-answering instances drawn from33conversations, while the remaining77conversations are used to train the lightweight retrieval agent with Qwen 3\.5 4B\. We evaluate on LongMemEval with367367instances spanning diverse question types\. For the main results \(Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2)and Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2)\), we use Claude Sonnet 4\.5 as the backbone model for reasoning, memory management, and retrieval across all compared methods\. Variants using Qwen\-based GRPO retrievers are analyzed separately in Section[5\.3](https://arxiv.org/html/2606.11680#S5.SS3)and Table[3](https://arxiv.org/html/2606.11680#S5.T3)\. Additional training details, experimental setups, and hyper\-parameters are provided in Appendix[B](https://arxiv.org/html/2606.11680#A2)\.

#### Baselines and Metrics\.

We compare against representative context management baselines, including static memory methods such as truncation, sliding window, and embedding\-based similarity, as well as dynamic memory approaches such as ReSum\[[42](https://arxiv.org/html/2606.11680#bib.bib29)\]and Acon\[[14](https://arxiv.org/html/2606.11680#bib.bib30)\]\. For ALFWorld, we additionally include context folding methods, whereFoldaggregates action\-observation trajectories into their preceding reasoning steps within the ReAct framework\[[49](https://arxiv.org/html/2606.11680#bib.bib66)\], as well as HIAGENT\[[6](https://arxiv.org/html/2606.11680#bib.bib42)\]\. For conversational benchmarks\[[22](https://arxiv.org/html/2606.11680#bib.bib23),[41](https://arxiv.org/html/2606.11680#bib.bib2)\], we further compare against external memory systems including A\-MEM\[[46](https://arxiv.org/html/2606.11680#bib.bib44)\]and Mem0\[[3](https://arxiv.org/html/2606.11680#bib.bib43)\], along with an embedding\-based similarity retrieval baseline built on our structured note representations\. We evaluate both task performance and memory efficiency\. On ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\], we report success rate, along with the average number of interaction steps per task and the average input tokens per step, where total token usage is their product\. On conversational benchmarks\[[22](https://arxiv.org/html/2606.11680#bib.bib23),[41](https://arxiv.org/html/2606.11680#bib.bib2)\], we report LLM\-as\-a\-judge \(L\-J\) scores using Claude Sonnet 4\.5, with F1 scores provided in Appendix[A](https://arxiv.org/html/2606.11680#A1), as well as total input token usage per question\-answering instance\.

![Refer to caption](https://arxiv.org/html/2606.11680v1/x2.png)\(a\)ALFWorld Pareto Efficiency
![Refer to caption](https://arxiv.org/html/2606.11680v1/x3.png)\(b\)Conversation Benchmark Token Usage

Figure 2:Efficiency\-Performance Trade\-offs Across Benchmarks: \(a\) Comparison of average interaction steps versus tokens per step under Small \(1950\) and Large \(2200\) context limits; \(b\) Total input tokens consumed on LoCoMo \(x\-axis\) and LongMemEval \(y\-axis\) on a logarithmic scale\.Table 1:ALFWorld Performance with Claude Sonnet 4\.5 as primary agent’s backbone under different context windows: Small \(19501950context input token limit\) and Large \(22002200context input token limit\)\. We report Success Rate \(%\\%\)\. Subscripts denote differences relative to Truncation withimprovementanddegradation\. The best results are highlighted inboldand our methods are inblue\.Table 2:We report LoCoMo \(1010K context input token limit\) and LongMemEval \(5050K context input token limit\) performance with Claude Sonnet 4\.5 as primary agent’s backbone\. We reportL\-Jscores \(↑\\uparrow\) on varying task types and Overall splits\. Subscripts denote differences relative to Truncation baseline withimprovementanddegradation\. The best results except forNo limitare highlighted inboldand our methods are inblue\. Note that SS refers to single\-session and KU refers to Knowledge Update\.Table 3:Ablation of skill usage in management and RL \(e\.g\., GRPO\) in retrieval indicated by ✓/✗\. We report task performance \(↑\\uparrow\) \(success rate for ALFWorld and L\-J scores for conversational benchmarks\) and the number of LLM retrieval callsNcall\{N\}\_\{\\text\{call\}\}\(↓\\downarrow\) per interactive step or question\-answering instance\. The primary agent and memory manager use Claude Sonnet 4\.5\. Best results are inbold\. Relative improvements over the no\-skill, no\-RL baseline are shown ingreen\.RetrieverSkillRLALFWorld \(Large\)LoCoMoLongMemEvalPerformanceNc​a​l​l\{N\}\_\{\{call\}\}PerformanceNc​a​l​l\{N\}\_\{\{call\}\}PerformanceNc​a​l​l\{N\}\_\{\{call\}\}Claude Sonnet 4\.5×\\times×\\times51\.54\.4742\.24\.9843\.65\.47✓\\checkmark×\\times73\.9\+22\.4\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+22\.4\}\}\}4\.43\-0\.04\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.04\}\}\}51\.6\+9\.4\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+9\.4\}\}\}4\.73\-0\.25\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.25\}\}\}55\.9\+12\.3\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+12\.3\}\}\}5\.26\-0\.21\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.21\}\}\}Qwen 3\.5 4B×\\times×\\times35\.85\.2127\.05\.4630\.85\.53✓\\checkmark×\\times40\.3\+4\.5\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+4\.5\}\}\}5\.08\-0\.13\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.13\}\}\}32\.6\+5\.6\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+5\.6\}\}\}5\.33\-0\.13\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.13\}\}\}40\.6\+9\.8\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+9\.8\}\}\}5\.41\-0\.12\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.12\}\}\}✓\\checkmark✓\\checkmark64\.9\+29\.1\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+29\.1\}\}\}4\.58\-0\.63\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.63\}\}\}42\.2\+15\.2\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+15\.2\}\}\}5\.12\-0\.54\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.54\}\}\}58\.0\+27\.2\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\+27\.2\}\}\}5\.22\-0\.31\{\}\_\{\{\\color\[rgb\]\{0,0\.5,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0\.5,0\}\\scalebox\{1\.0\}\{\\tiny\-0\.31\}\}\}

![Refer to caption](https://arxiv.org/html/2606.11680v1/x4.png)\(c\)Memory construction bottleneck across varying memory manager
![Refer to caption](https://arxiv.org/html/2606.11680v1/x5.png)\(d\)Error Attribution for temporally sensitive tasks
![Refer to caption](https://arxiv.org/html/2606.11680v1/x6.png)\(e\)Skill library growth and scaling

Figure 3:Analysis of Retrieval Reliability and Skill Acquisition: \(a\) Hashed bars as the gain when replacing native retrievers with a stronger retriever \(Sonnet 4\.5\) in LoCoMo; \(b\) Comparison of failure modes for temporally sensitive tasks between similarity\-based retrieval and HORMA’s navigated retrieval in LongMemEval; \(c\) The iterative expansion of the skill library over four refinement rounds in LongMemEval\.
### 5\.2Main Results

#### Interactive Benchmark\.

On ALFWorld, we evaluate all methods under two context window settings\. We analyze Pareto efficiency in terms of interaction steps and input tokens per step in Figure[2\(a\)](https://arxiv.org/html/2606.11680#S5.F2.sf1)\. Across methods, larger context windows generally reduce the number of interaction steps, as more information can be incorporated at each decision step\. Our method consistently achieves both fewer interaction steps and lower token usage under both settings\. Incorporating memory management skills further improves efficiency\. Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2)reports task success rates under both settings\. Notably, theFoldbaseline underperforms sliding window despite preserving reasoning traces, likely because retained reasoning significantly increases context cost and reduces actionable capacity\. Overall, HORMA \(with skill\) achieves the best performance, reaching 56\.7% and 73\.9% success rate under small and large context limits, respectively\. These results demonstrate the effectiveness of structured memory and navigation\-based retrieval under strict context constraints\.

#### Conversational Benchmarks\.

On conversational benchmarks, we evaluate settings where input lengths exceed2020K tokens in LoCoMo and100100K tokens in LongMemEval, stressing long\-context retrieval under extreme context constraints\. We analyze token efficiency across all methods in Figure[2\(b\)](https://arxiv.org/html/2606.11680#S5.F2.sf2), including HORMA variants with different retrievers \(Claude, Qwen, and Qwen\-GRPO\), as well asEmbedding Retrievalaugmented with our agentic memory management\. All HORMA variants andEmbedding Retrievalconsistently operate within10001000tokens per query, demonstrating that the efficiency gains from our memory management are robust across retrieval backbones\.

In downstream task performance in Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2), replacing standard embedding as internal static memory withEmbedding Retrievalenhanced by our structured note representation improves results on both benchmarks, indicating that memory management alone strengthens even simple retrieval methods\. Together with Figure[2\(b\)](https://arxiv.org/html/2606.11680#S5.F2.sf2), this suggests that our memory management generalizes beyond HORMA to alternative retrieval paradigms\. We further report HORMA’s performance using a single representative configuration of HORMA \(Claude\-based retriever\) in Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2)while analyses of other HORMA variants are deferred to Section[5\.3](https://arxiv.org/html/2606.11680#S5.SS3)\. HORMA on LoCoMo achieves performance approaching the no\-context\-limit setting and outperforms all baselines\. On LongMemEval, where information dilution andlost\-in\-the\-middleeffects are pronounced, several baselines even outperform no\-limit counterparts due to implicit filtering of irrelevant context\. HORMA follows this trend and achieves the best overall performance, demonstrating stronger robustness to long\-context degradation\.

### 5\.3Analysis

#### Ablation Studies\.

We conduct ablation studies to analyze the contributions of \(i\) self\-evolving memory management skills and \(ii\) the lightweight retriever trained with RL post\-training\. Results are reported in Table[3](https://arxiv.org/html/2606.11680#S5.T3)\. HORMA without skill evolution already achieves competitive performance against the baselines in Tables[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2)and[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2), suggesting that the unified memory management prompt alone generalizes effectively across tasks\. Incorporating self\-evolving memory skills further improves task performance while consistently reducing the number of LLM retrieval calls across retriever backbones and benchmarks\. Using Claude Sonnet 4\.5 for all modules \(primary agent, memory manager, and retriever\) achieves the strongest performance on ALFWorld and LoCoMo, highlighting the effectiveness of structured command\-based memory management and retrieval within a unified framework\. We further evaluate an RL\-trained lightweight retriever based on Qwen 3\.5 4B, trained only on the LoCoMo training split\. The learned retriever improves LoCoMo performance from 32\.6% to 42\.2% while also exhibiting strong zero\-shot transfer to ALFWorld and LongMemEval, improving both task performance and retrieval efficiency by reducing unnecessary retrieval calls\. Notably, despite being trained solely on conversational data, the retriever generalizes across domains without modification and achieves the best overall performance on LongMemEval \(58%\), surpassing the Claude Sonnet 4\.5\-based retrieval configuration\.

#### Memory Construction Bottleneck\.

To test the hypothesis that effective memory construction requires high\-level semantic reasoning\[[10](https://arxiv.org/html/2606.11680#bib.bib27),[32](https://arxiv.org/html/2606.11680#bib.bib28)\], we evaluate HORMA across varying backbones for both management and retrieval\. As shown in Figure[2\(c\)](https://arxiv.org/html/2606.11680#S5.F2.sf3), while proprietary models establish a high performance ceiling, the smaller Qwen 3\.5 4B lags significantly\. Crucially, when we replace each model’s native retriever with Claude Sonnet 4\.5, the performance gains are non\-uniform\. The improvement for Qwen 3\.5 remains marginal even when enhanced by the superior retrieval capabilities of Sonnet 4\.5\. This demonstrates that flawed memory organization cannot be compensated for by high\-quality retrieval\. If the manager fails to induce a coherent structure, even an optimal navigation policy is restricted by the deficiencies of the underlying workspace\. These results empirically justify the necessity of high\-capacity LLMs for the memory management role\. The full cross\-backbone performance on LoCoMo can be found in Table[7](https://arxiv.org/html/2606.11680#A2.T7)in Appendix\.

#### Impact of Agentic Retrieval on Temporal Reasoning\.

Semantic similarity\-based retrieval is widely used\[[46](https://arxiv.org/html/2606.11680#bib.bib44),[47](https://arxiv.org/html/2606.11680#bib.bib5)\], with extensions such as two\-stage retrieval\[[60](https://arxiv.org/html/2606.11680#bib.bib3)\]and RL\-based temporal\-aware retrieval\[[4](https://arxiv.org/html/2606.11680#bib.bib4)\]\. Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2)shows that HORMA consistently outperforms baselines across categories\. We further analyze its impact on temporally sensitive tasks \(e\.g\., Knowledge Update and Temporal\) in LongMemEval\. For a controlled comparison, we evaluate bothEmbedding Retrievaland HORMA on top of the same memory management framework\. We collect inference trajectories and categorize failure cases into \(i\) reasoning errors, where retrieved context is correct but not properly utilized, and \(ii\) non\-reasoning errors, including temporal staleness and irrelevant retrieval\. As shown in Figure[2\(d\)](https://arxiv.org/html/2606.11680#S5.F2.sf4), HORMA significantly reduces non\-reasoning errors compared to embedding\-based retrieval, while both methods share the same primary agent and thus similar reasoning capability\. This indicates that improvements primarily arise from more accurate retrieval of temporally relevant information\.

#### Skill Library Growth\.

Figure[2\(e\)](https://arxiv.org/html/2606.11680#S5.F2.sf5)illustrates the evolution of the skill library over multiple refinement rounds on LongMemEval\. Starting from an empty set, the library expands to6363skills after four rounds\. Task performance improves steadily as the skill set grows, indicating that accumulated skills provide increasingly effective guidance for memory management\. Examples of agent\-generated memory management skills are provided in Tables[9](https://arxiv.org/html/2606.11680#A3.T9),[10](https://arxiv.org/html/2606.11680#A3.T10), and[11](https://arxiv.org/html/2606.11680#A3.T11)in Appendix[C](https://arxiv.org/html/2606.11680#A3)\. Importantly, our framework starts from a domain\-agnostic memory management prompt and progressively augments it with domain\-specific skills learned through interaction, enabling adaptation across tasks\.

## 6Conclusion

We introduced HORMA, a hierarchical organize\-and\-retrieve memory agent that decouples working memory into a high\-level memory manager and a low\-level retriever operating over a structured file\-system workspace\. By separating asynchronous memory organization from per\-step retrieval, HORMA improves credit assignment, context efficiency, and scalability in long\-horizon reasoning\. The memory manager acquires organizational skills through recursive trajectory refinement, while the retriever is optimized with RL to navigate hierarchical memory via executable file\-system operations\. HORMA achieves consistent gains across three benchmarks while substantially reducing context usage and retrieval overhead, and the learned retrieval policy generalizes effectively across domains\. Future work will extend the current evidence\-based retrieval training framework to fully online interaction\-driven learning while preserving HORMA’s modular design\.

## References

- \[1\]\(2025\)Why does the effective context length of LLMs fall short?\.InProceedings of the International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[2\]Anthropic\(2024\)The Claude 3 model family: Opus, Sonnet, Haiku\.Note:[https://www\.anthropic\.com/news/claude\-3\-family](https://www.anthropic.com/news/claude-3-family)Accessed: 2026\-05\-12Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p5.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px3.p1.1)\.
- \[3\]P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav\(2025\)Mem0: building production\-ready AI agents with scalable long\-term memory\.InProceedings of the European Conference on Artificial Intelligence \(ECAI\),Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[4\]Y\. Du, B\. Wang, Y\. Xiang, Z\. Wang, W\. Huang, B\. Xue, B\. Liang, X\. Zeng, F\. Mi, H\. Bai, L\. Shang, J\. Z\. Pan, Y\. Jiang, and K\. Wong\(2026\)MEMORY\-t1: reinforcement learning for temporal reasoning in multi\-session agents\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1),[§1](https://arxiv.org/html/2606.11680#S1.p3.1),[§1](https://arxiv.org/html/2606.11680#S1.p4.1),[§5\.3](https://arxiv.org/html/2606.11680#S5.SS3.SSS0.Px3.p1.1)\.
- \[5\]D\. Guo, D\. Yang, H\. Zhang,et al\.\(2025\)DeepSeek\-r1 incentivizes reasoning in LLMs through reinforcement learning\.Nature645,pp\. 633–638\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[6\]M\. Hu, T\. Chen, Q\. Chen, Y\. Mu, W\. Shao, and P\. Luo\(2025\-07\)HiAgent: hierarchical working memory management for solving long\-horizon agent tasks with large language model\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics,pp\. 32779–32798\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[7\]W\. Huang, W\. Zhang, Y\. Liang, Y\. Bei, Y\. Chen,et al\.\(2026\)Rethinking memory mechanisms of foundation agents in the second half: a survey\.arXiv preprint arXiv:2602\.06052\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1)\.
- \[8\]H\. Jiang, Q\. Wu, C\. Lin, Y\. Yang, and L\. Qiu\(2023\-12\)LLMLingua: compressing prompts for accelerated inference of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 13358–13376\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[9\]H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. Qiu\(2024\-08\)LongLLMLingua: accelerating and enhancing LLMs in long context scenarios via prompt compression\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 1658–1677\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[10\]Z\. Jiang, P\. Wu, Z\. Liang, P\. Q\. Chen, X\. Yuan, Y\. Jia, J\. Tu, C\. Li, P\. H\. F\. Ng, and Q\. Li\(2025\)HiBench: benchmarking LLMs capability on hierarchical structure reasoning\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining \(KDD ’25\),Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p3.1),[§1](https://arxiv.org/html/2606.11680#S1.p5.1),[§5\.3](https://arxiv.org/html/2606.11680#S5.SS3.SSS0.Px2.p1.1)\.
- \[11\]B\. Jin, H\. Zeng, Z\. Yue, J\. Yoon, S\. O\. Arik, D\. Wang, H\. Zamani, and J\. Han\(2025\)Search\-r1: training LLMs to reason and leverage search engines with reinforcement learning\.InSecond Conference on Language Modeling,Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[12\]M\. Jin, W\. Luo, S\. Cheng, X\. Wang, W\. Hua, R\. Tang, W\. Y\. Wang, and Y\. Zhang\(2025\-07\)Disentangling memory and reasoning ability in large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 1681–1701\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p3.1)\.
- \[13\]J\. Kang, M\. Ji, Z\. Zhao, and T\. Bai\(2025\)Memory os of AI agent\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1)\.
- \[14\]M\. Kang, W\. Chen, D\. Han, H\. A\. Inan, L\. Wutschitz, Y\. Chen, R\. Sim, and S\. Rajmohan\(2025\)ACON: optimizing context compression for long\-horizon LLM agents\.arXiv preprint arXiv:2510\.00615\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[15\]V\. Karpukhin, B\. Oguz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih\(2020\-11\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),B\. Webber, T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\. 6769–6781\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1)\.
- \[16\]P\. Laban, H\. Hayashi, Y\. Zhou, and J\. Neville\(2026\)LLMs get lost in multi\-turn conversation\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[17\]A\. K\. Lampinen, M\. Engelcke, Y\. Li, A\. Chaudhry, and J\. L\. McClelland\(2025\)Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences\.arXiv preprint arXiv:2509\.16189\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[18\]Y\. Li, B\. Dong, C\. Lin, and F\. Guerin\(2023\)Compressing context to enhance inference efficiency of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[19\]Z\. Li, H\. Zhang, C\. Wei, P\. Lu, P\. Nie, Y\. Lu, Y\. Bai, S\. Feng, H\. Zhu, M\. Zhong, Y\. Zhang, J\. Xie, Y\. Choi, J\. Zou, J\. Han, W\. Chen, J\. Lin, D\. Jiang, and Y\. Zhang\(2026\)Beyond semantic similarity: rethinking retrieval for agentic search via direct corpus interaction\.arXiv preprint arXiv:2605\.05242\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p6.1)\.
- \[20\]N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang\(2024\)Lost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1),[2nd item](https://arxiv.org/html/2606.11680#S4.I1.i2.p1.3)\.
- \[21\]M\. Lu, W\. Sun, W\. Du, Z\. Ling, X\. Yao, K\. Liu, and J\. Chen\(2025\)Scaling LLM multi\-turn RL with end\-to\-end summarization\-based context management\.arXiv preprint arXiv:2510\.06727\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[22\]A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang\(2024\)Evaluating very long\-term conversational memory of LLM agents\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§A\.2](https://arxiv.org/html/2606.11680#A1.SS2.p1.3),[Table 4](https://arxiv.org/html/2606.11680#A1.T4),[Table 4](https://arxiv.org/html/2606.11680#A1.T4.6.2),[Appendix B](https://arxiv.org/html/2606.11680#A2.p1.2),[Table 10](https://arxiv.org/html/2606.11680#A3.T10),[Table 10](https://arxiv.org/html/2606.11680#A3.T10.3.2),[§1](https://arxiv.org/html/2606.11680#S1.p7.1),[§4\.1](https://arxiv.org/html/2606.11680#S4.SS1.SSS0.Px1.p1.3),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px1.p1.7),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[23\]Q\. Mi, Z\. Ma, M\. Yang, H\. Li, Y\. Wang,et al\.\(2026\)Skill\-pro: learning reusable skills from experience via non\-parametric ppo for LLM agents\.arXiv preprint arXiv:2602\.01869\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px3.p1.1)\.
- \[24\]C\. Packer, V\. Fang, S\. G\. Patil, K\. Lin, S\. Wooders, and J\. E\. Gonzalez\(2023\)MemGPT: towards LLMs as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1)\.
- \[25\]Z\. Pan, Q\. Wu, H\. Jiang, X\. Luo, H\. Cheng, D\. Li, Y\. Yang, C\. Lin, H\. V\. Zhao, L\. Qiu, and J\. Gao\(2025\)SeCom: on memory construction and retrieval for personalized conversational agents\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[26\]R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn\(2023\)Direct preference optimization: your language model is secretly a reward model\.InThirty\-seventh Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[27\]M\. Ravaut, A\. Sun, N\. Chen, and S\. Joty\(2024\-08\)On context utilization in summarization with large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 2764–2781\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[28\]R\. Salama, J\. Cai, M\. Yuan, A\. Currey, M\. Sunkara, Y\. Zhang, and Y\. Benajiba\(2025\-11\)MemInsight: autonomous memory augmentation for LLM agents\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 33136–33152\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1)\.
- \[29\]J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov\(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[30\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2606.11680#S4.SS4.SSS0.Px2.p1.1)\.
- \[31\]M\. Shridhar, X\. Yuan, M\. Côté, Y\. Bisk, A\. Trischler, and M\. Hausknecht\(2021\)ALFWorld: aligning text and embodied environments for interactive learning\.International Conference on Learning Representation\.Cited by:[§A\.1](https://arxiv.org/html/2606.11680#A1.SS1.SSS0.Px1.p1.1),[§A\.2](https://arxiv.org/html/2606.11680#A1.SS2.SSS0.Px2.p1.2),[Table 4](https://arxiv.org/html/2606.11680#A1.T4),[Table 4](https://arxiv.org/html/2606.11680#A1.T4.6.2),[Table 5](https://arxiv.org/html/2606.11680#A1.T5),[Table 5](https://arxiv.org/html/2606.11680#A1.T5.3.2),[Table 9](https://arxiv.org/html/2606.11680#A3.T9),[Table 9](https://arxiv.org/html/2606.11680#A3.T9.3.2),[§1](https://arxiv.org/html/2606.11680#S1.p7.1),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px1.p1.7),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[32\]W\. Son and H\. Kim\(2026\)Content\-based file classification and organization system using LLMs\.Electronics15\(7\),pp\. 1524\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p3.1),[§1](https://arxiv.org/html/2606.11680#S1.p5.1),[§5\.3](https://arxiv.org/html/2606.11680#S5.SS3.SSS0.Px2.p1.1)\.
- \[33\]C\. Sun, X\. Chen, J\. Luo, D\. Zhang, and X\. Li\(2025\)Beyond heuristics: a decision\-theoretic framework for agent memory management\.arXiv preprint arXiv:2512\.21567\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p4.1)\.
- \[34\]W\. Sun, M\. Lu, Z\. Ling, K\. Liu, X\. Yao, Y\. Yang, and J\. Chen\(2025\)Scaling long\-horizon LLM agent via context\-folding\.arXiv preprint arXiv:2510\.11967\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1),[§1](https://arxiv.org/html/2606.11680#S1.p3.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1)\.
- \[35\]H\. Tan, X\. Yang, H\. Chen, J\. Shao, Y\. Wen, Y\. Shen, W\. Luo, X\. Du, L\. Guo, and Y\. Li\(2026\)Hindsight credit assignment for long\-horizon LLM agents\.arXiv preprint arXiv:2603\.08754\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p4.1)\.
- \[36\]J\. Wang, Q\. Yan, Y\. Wang, Y\. Tian, S\. S\. Mishra,et al\.\(2025\)Reinforcement learning for self\-improving agent with skill library\.arXiv preprint arXiv:2512\.17102\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px3.p1.1)\.
- \[37\]Q\. Wang, Y\. Fu, Y\. Cao, S\. Wang, Z\. Tian, and L\. Ding\(2025\)Recursively summarizing enables long\-term dialogue memory in large language models\.Neurocomputing639\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[38\]T\. Wang, S\. Gooding, F\. Hartmann, O\. Riva, and E\. Grefenstette\(2026\)A subgoal\-driven framework for improving long\-horizon LLM agents\.arXiv preprint arXiv:2603\.19685\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1)\.
- \[39\]Y\. Wang, R\. Takanobu, Z\. Liang, Y\. Mao, Y\. Hu,et al\.\(2025\)Mem\-α\\alpha: learning memory construction via reinforcement learning\.arXiv preprint arXiv:2509\.25911\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p4.1)\.
- \[40\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[41\]D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu\(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InInternational Conference on Learning Representations,Cited by:[§A\.3](https://arxiv.org/html/2606.11680#A1.SS3.p1.6),[Table 4](https://arxiv.org/html/2606.11680#A1.T4),[Table 4](https://arxiv.org/html/2606.11680#A1.T4.6.2),[Table 11](https://arxiv.org/html/2606.11680#A3.T11),[Table 11](https://arxiv.org/html/2606.11680#A3.T11.3.2),[§1](https://arxiv.org/html/2606.11680#S1.p7.1),[§4\.1](https://arxiv.org/html/2606.11680#S4.SS1.SSS0.Px1.p1.3),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px1.p1.7),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[42\]X\. Wu, K\. Li, Y\. Zhao, L\. Zhang, L\. Ou, H\. Yin, Z\. Zhang, X\. Yu, D\. Zhang, Y\. Jiang, P\. Xie, F\. Huang, M\. Cheng, S\. Wang, H\. Cheng, and J\. Zhou\(2025\)ReSum: unlocking long\-horizon search intelligence via context summarization\.arXiv preprint arXiv:2509\.13313\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[43\]Y\. Wu, T\. Lin, Y\. Zhou, F\. Zhang, Q\. Guo, X\. Zhou, S\. Wang, X\. Liu, Y\. Ma, and Y\. Fang\(2026\)Memory in the LLM era: modular architectures and strategies in a unified framework\.arXiv preprint arXiv:2604\.01707\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[44\]P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen,et al\.\(2026\)SkillRL: evolving agents via recursive skill\-augmented reinforcement learning\.arXiv preprint arXiv:2602\.08234\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px3.p1.1)\.
- \[45\]B\. Xu, Y\. Chen, J\. Fang, R\. Zhong, Y\. Yao, Y\. Zhu, L\. Du, and S\. Deng\(2026\)StructMem: structured memory for long\-horizon behavior in LLMs\.arXiv preprint arXiv:2604\.21748\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1),[§1](https://arxiv.org/html/2606.11680#S1.p6.1)\.
- \[46\]W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang\(2025\)A\-mem: agentic memory for LLM agents\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1),[§5\.3](https://arxiv.org/html/2606.11680#S5.SS3.SSS0.Px3.p1.1)\.
- \[47\]S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan, H\. Schütze, V\. Tresp, and Y\. Ma\(2026\)Memory\-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2606.11680#S4.SS2.p1.4),[§5\.3](https://arxiv.org/html/2606.11680#S5.SS3.SSS0.Px3.p1.1)\.
- \[48\]C\. Yang, Z\. Sun, W\. Wei, and W\. Hu\(2026\)Beyond static summarization: proactive memory extraction for LLM agents\.arXiv preprint arXiv:2601\.04463\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p6.1)\.
- \[49\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.International Conference on Learning Representation\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1),[§5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2.p1.1)\.
- \[50\]H\. Ye, X\. He, V\. Arak, H\. Dong, and G\. Song\(2026\)Meta context engineering via agentic skill evolution\.arXiv preprint arXiv:2601\.21557\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px3.p1.1)\.
- \[51\]R\. Ye, Z\. Zhang, K\. Li, H\. Yin, Z\. Tao, Y\. Zhao, L\. Su, L\. Zhang, Z\. Qiao, X\. Wang, X\. Wu, X\. Yu, Y\. Jiang, D\. Zhang, H\. Cheng, and J\. Zhou\(2025\)AgentFold: long\-horizon web agents with proactive context management\.arXiv preprint arXiv:2510\.24699\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2510.24699)Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1)\.
- \[52\]C\. Yoon, T\. Lee, H\. Hwang, M\. Jeong, and J\. Kang\(2024\)Compact: compressing retrieved documents actively for question answering\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p1.1)\.
- \[53\]H\. Yu, T\. Chen, J\. Feng, J\. Chen, W\. Dai, Q\. Yu, Y\. Zhang, W\. Ma, J\. Liu, M\. Wang, and H\. Zhou\(2026\)MemAgent: reshaping long\-context LLM with multi\-conv RL\-based memory agent\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p3.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[54\]S\. Yu, G\. Li, W\. Shi, and P\. Qi\(2026\)PolySkill: learning generalizable skills through polymorphic abstraction for continual learning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px3.p1.1)\.
- \[55\]Q\. Yuan, J\. Lou, Z\. Li, J\. Chen, Y\. Lu, H\. Lin, L\. Sun, D\. Zhang, and X\. Han\(2025\)MemSearcher: training LLMs to reason, search and manage memory via end\-to\-end reinforcement learning\.arXiv preprint arXiv:2511\.02805\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p4.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1)\.
- \[56\]M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, P\. Lu, Z\. Huang, C\. Guestrin, and J\. Zou\(2025\)Optimizing generative ai by backpropagating language model feedback\.Nature639\(8055\),pp\. 609–616\.Cited by:[§4\.3](https://arxiv.org/html/2606.11680#S4.SS3.SSS0.Px1.p3.2)\.
- \[57\]R\. Zeng, J\. Fang, S\. Liu, and Z\. Meng\(2024\)On the structural memory of LLM agents\.arXiv preprint arXiv:2412\.15266\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1)\.
- \[58\]D\. Zhang, L\. Chen, S\. Zhang, H\. Xu, Z\. Zhao, and K\. Yu\(2023\)Large language models are semi\-parametric reinforcement learning agents\.InThirty\-seventh Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[59\]H\. Zhang, Q\. Long, J\. Bao, T\. Feng, W\. Zhang, H\. Yue, and W\. Wang\(2026\)MemSkill: learning and evolving memory skills for self\-evolving agents\.arXiv preprint arXiv:2602\.02474\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p4.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px3.p1.1)\.
- \[60\]S\. Zhang, J\. Wang, R\. Zhou, J\. Liao, Y\. Feng,et al\.\(2026\)MemRL: self\-evolving agents via runtime reinforcement learning on episodic memory\.arXiv preprint arXiv:2601\.03192\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1),[§1](https://arxiv.org/html/2606.11680#S1.p3.1),[§5\.3](https://arxiv.org/html/2606.11680#S5.SS3.SSS0.Px3.p1.1)\.
- \[61\]Y\. Zhang, J\. Shu, Y\. Ma, X\. Lin, S\. Wu, and J\. Sang\(2025\)Memory as action: autonomous context curation for long\-horizon agentic tasks\.arXiv preprint arXiv:2510\.12635\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px1.p1.1)\.
- \[62\]Z\. Zhang, Q\. Dai, R\. Li, X\. Bo, X\. Chen, and Z\. Dong\(2025\)Learn to memorize: optimizing LLM\-based agents with adaptive memory framework\.arXiv preprint arXiv:2508\.16629\.Cited by:[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[63\]W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang\(2024\)MemoryBank: enhancing large language models with long\-term memory\.InProceedings of the AAAI Conference on Artificial Intelligence,pp\. 19724–19731\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1)\.
- \[64\]H\. Zhou, Y\. Chen, S\. Guo, X\. Yan, K\. H\. Lee, Z\. Wang, K\. Y\. Lee, G\. Zhang, K\. Shao, L\. Yang, and J\. Wang\(2025\)Memento: fine\-tuning LLM agents without fine\-tuning LLMs\.arXiv preprint arXiv:2508\.16153\.Cited by:[§4\.1](https://arxiv.org/html/2606.11680#S4.SS1.p1.10)\.
- \[65\]Z\. Zhou, A\. Qu, Z\. Wu, S\. Kim, A\. Prakash, D\. Rus, J\. Zhao, B\. K\. H\. Low, and P\. P\. Liang\(2025\)MEM1: learning to synergize memory and reasoning for efficient long\-horizon agents\.arXiv preprint arXiv:2506\.15841\.Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p3.1),[§2](https://arxiv.org/html/2606.11680#S2.SS0.SSS0.Px2.p1.1)\.
- \[66\]L\. Zhuang, S\. Chen, Y\. Xiao, H\. Zhou, Y\. Zhang, H\. Chen, Q\. Zhang, and X\. Huang\(2026\)LinearRAG: linear graph retrieval augmented generation on large\-scale corpora\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.11680#S1.p2.1)\.

## Appendix ADataset and Experiment Setup

Table[4](https://arxiv.org/html/2606.11680#A1.T4)summarizes the benchmarks and dataset splits used in our experiments\. Additional details for each benchmark are provided below\.

Table 4:Summary of benchmarks and data splits used in our experiments\. We evaluate HORMA across embodied interaction \(ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\]\) and extended conversational settings \(LoCoMo\[[22](https://arxiv.org/html/2606.11680#bib.bib23)\]andLongMemEval\[[41](https://arxiv.org/html/2606.11680#bib.bib2)\]\) to test long\-context reasoning, context efficiency, and out\-of\-distribution \(OOD\) generalization\.### A\.1ALFWorld

#### Environment\.

ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\]is a text\-based interactive environment built on TextWorld, where agents operate in household settings via natural language\. At each step, the agent issues an action and receives textual feedback from the environment\. The goal is to complete high\-level tasks \(e\.g\., placing an object in a specified location\) within a fixed horizon of5050steps\. Tasks often require long\-horizon reasoning\. Consequently, agents must perform effective planning, maintain subgoal structure, and explore systematically\.

#### Tasks and Action Space\.

The action space of the interactive agent is summarized in Table[5](https://arxiv.org/html/2606.11680#A1.T5)\. We evaluate on the standard out\-of\-distribution \(OOD\) split consisting of134134tasks across six categories: Pick & Place \(2424\), Clean & Place \(3131\), Heat & Place \(2323\), Cool & Place \(2121\), Examine in Light \(1818\), and Pick Two & Place \(1717\)\.

#### Context Budget\.

To study performance under constrained context windows, we first establish a near\-upper\-bound by deploying an unconstrained agent based on Claude Sonnet 4\.5\. This setting achieves a97\.0%97\.0\\%success rate\. The per\-episode token usage ranges from15881588to34353435tokens \(median:21722172, mean:2207\.82207\.8\)\. Guided by this distribution, we define two context budgets,19501950and22002200tokens, to simulate realistic memory constraints while preserving performance sensitivity\. These budgets enable controlled evaluation of how different methods trade off context efficiency and task success\.

Table 5:Action space for ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\], where \(object\) refers to manipulable objects and \(receptacle\) refers to receptacles or locations in the environment\.Table 6:LoCoMo \(1010K context input token limit\) and LongMemEval \(5050K context input token limit\) performance with Claude Sonnet 4\.5 as primary agent’s backbone\. We reportF1scores \(↑\\uparrow\) on varying task types and Overall splits\. Subscripts denote differences relative to Truncation baseline withimprovementanddegradation\. The best results except forNo limitare highlighted inboldand our methods are inblue\. Note that SS refers to single\-session and KU refers to Knowledge Update\.### A\.2LoCoMo

LoCoMo\[[22](https://arxiv.org/html/2606.11680#bib.bib23)\]consists of long, multi\-session dialogues designed to evaluate memory and reasoning over extended contexts\. The benchmark contains1010conversations, each comprising1919–3232sessions, where each session includes multiple dialogue turns\. We focus on question\-answering tasks that probe short\-term and compositional reasoning, specifically from three categories: \(i\) Single\-hop, where answers are grounded in a single session; \(ii\) Temporal, which require reasoning over temporal relationships and tracking time\-dependent cues across sessions; and \(iii\) Adversarial, which are constructed to induce incorrect responses, requiring the agent to recognize unanswerable or misleading queries\.

#### Data Split\.

We use the first77conversations for training, yielding10891089question\-answering instances, and evaluate on519519instances from the remaining33conversations\. This split prevents exposure to evaluation dialogues during training, ensuring a strict separation of conversational context\.

#### Context Budget\.

Following the setup in ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\], we first analyze the unconstrained setting\. The average token length per instance exceeds2020K tokens, reflecting the long\-horizon nature of the benchmark\. Based on this distribution, we impose a1010K token context budget to evaluate different methods under constrained memory, enabling systematic comparison of context efficiency and reasoning performance\.

### A\.3LongMemEval

To assess out\-of\-domain \(OOD\) generalization, we adopt LongMemEval\[[41](https://arxiv.org/html/2606.11680#bib.bib2)\], a benchmark designed to evaluate memory retrieval and reasoning over long, multi\-session conversations\. We evaluate on367367question\-answering instances spanning diverse task types\. The benchmark includes: \(1\) Single\-session\-user \(7070\) and Single\-session\-assistant \(5656\), which test the ability to recall information introduced by the user or assistant within a single session; \(2\) Single\-session\-preference \(3030\), which evaluates whether the model can leverage user\-specific information to generate personalized responses; \(3\) Knowledge Update \(KU\) \(7878\), which requires tracking changes in user state and updating stored memory accordingly; and \(4\) Temporal Reasoning \(TR\) \(133133\), which involves reasoning over both metadata timestamps and explicit temporal references\.

#### Context Budget\.

We analyze the unconstrained setting by deploying an answering agent based on Claude Sonnet 4\.5\. The average context length exceeds100100K tokens per instance, reflecting the substantial memory demands of the benchmark\. To enable controlled evaluation, we impose a5050K token context budget, allowing us to compare methods in terms of both context efficiency and reasoning performance under realistic constraints\.

## Appendix BImplementation Details

In Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2)and Table[5\.1](https://arxiv.org/html/2606.11680#S5.SS1.SSS0.Px2), all methods, including HORMA, use Claude Sonnet 4\.5 as the primary agent to ensure fair comparison\. Unless otherwise specified, HORMA additionally employs Claude Sonnet 4\.5 as both the memory manager and retrieval agent\. We further investigate lightweight open\-source retrievers based on Qwen 3\.5 4B, including RL post\-training, as reported in Table[3](https://arxiv.org/html/2606.11680#S5.T3)\. Table[8](https://arxiv.org/html/2606.11680#A2.T8)summarizes the hyper\-parameters used for GRPO training\. Our lightweight retrieval agent is trained on the LoCoMo training split, which consists of77conversations and10891089question\-answering tasks\. We evaluate the trained agent across all benchmarks to assess cross\-domain generalization\. The evidence set used for computing the evidence\-grounded reward \(defined in Section[4\.4](https://arxiv.org/html/2606.11680#S4.SS4)\) is directly derived from the original dataset\[[22](https://arxiv.org/html/2606.11680#bib.bib23)\]\. We also study the impact of varying memory management and retrieval backbones in Table[7](https://arxiv.org/html/2606.11680#A2.T7)\. All RL\-based retriever training experiments were conducted using 4 NVIDIA H200 GPUs\. Inference with closed\-source models was performed on a CPU\-based virtual machine equipped with an Intel Xeon Platinum 8488C processor with 48 physical cores \(96 logical CPUs\)\.

Table 7:Cross\-Backbone Performance on LoCoMo\. We evaluate the impact of different LLM backbones for memory management and retrieval strategies\.Table 8:Hyper\-parameters for GRPO training on memory retrieval agent\.
## Appendix CMemory Management Skill Examples

In Table[9](https://arxiv.org/html/2606.11680#A3.T9)through Table[11](https://arxiv.org/html/2606.11680#A3.T11), we provide representative examples of endogenous and exogenous memory management skills across all three benchmarks, detailing their IDs, titles, and underlying principles\. The specific logic for inducing these skills is defined by the contrastive discovery protocols in Prompt[D](https://arxiv.org/html/2606.11680#A4)\(Endogenous\) and Prompt[D](https://arxiv.org/html/2606.11680#A4)\(Exogenous\)\. These prompts are further structured by the JSON output requirements defined in the skill fields specification in Prompt[D](https://arxiv.org/html/2606.11680#A4)\.

These skills are generated to refine domain\-specific memory management guidelines via the contrastive analysis of past trajectories\. For example, the end\_002 Failed\-Action Loop Detection in Table[9](https://arxiv.org/html/2606.11680#A3.T9)addresses theNothing happensenvironment response, a failure mode unique to ALFWorld\. We also observe notable consistency in skill acquisition across similar domains\. Because LoCoMo and LongMemEval are both multi\-session conversational benchmarks, they share identical endogenous skills such as end\_001 Temporal Precision Anchoring and end\_002 Verbatim Quote Preservation in Table[10](https://arxiv.org/html/2606.11680#A3.T10)and Table[11](https://arxiv.org/html/2606.11680#A3.T11)\.

Table 9:Examples of generated endogenous and exogenous skills for ALFWorld\[[31](https://arxiv.org/html/2606.11680#bib.bib1)\]\.Table 10:Examples of generated endogenous and exogenous skills for LoCoMo\[[22](https://arxiv.org/html/2606.11680#bib.bib23)\]\.Table 11:Examples of generated endogenous and exogenous skills for LongMemEval\[[41](https://arxiv.org/html/2606.11680#bib.bib2)\]\.
## Appendix DPrompt Template

D\.1: ALFWorld Interactive PromptYou are an intelligent agent in a household environment and your target is to perform actions \(aligns with subgoal\) to complete the task goal\. At the beginning of your interactions, you will be given the detailed description of the current environment and your goal to accomplish\.For each of your turn, you will be given the useful context for current turn\. Your output must strictly follow this format:<your next action\>\.The available actions are:1\.go to \(receptacle2\.take \(object\) from \(receptacle\)3\.use \(object\)4\.move \(object\) to \(receptacle\)5\.open \(receptacle\)6\.close \(receptacle\)7\.toggle \(object\) \(receptacle\)8\.clean \(object\) with \(receptacle\)9\.heat \(object\) with \(receptacle\)10\.cool \(object\) with \(receptacle\)11\.think: \(your thought\), where \(object\) refers to manipulable objects and \(receptacle\) refers to receptacles or locations in the environment\.1\.If the environment output: Nothing happens, that means the previous action is invalid and you should try more options\.2\.You can only hold one object at a time\. Before taking a new object, make sure you have placed down any object you are currently holding\.3\.You should not assume or anticipate the feedback\.4\.Even if you have planned multiple steps ahead, you should only execute one action at a time, which aligns with subgoal\.5\.Do not proceed with any further exploration or actions until you receive the feedback from the environment after your action\.6\.Do not keep thinking\.Your response should use one of the following formats:<your next action\>think: <your thoughts\>\.Here are two examples\.<example1\><example2\>Here is the task <task\>\.

D\.2: LoCoMo Answer PromptBased on the following conversation, answer the question with a short, precise answer\. Use the date/time information derived or concluded from the conversation sessions if it is temporal question\. Pay close attention to who said what\.Conversation: \{context\}Question: \{question\}Answer:"""

D\.3: LongMemEval Answer PromptBased on the following conversation, answer the question with a short, precise answer\. Use the date/time information derived or concluded from the conversation sessions if it is temporal question\. Pay close attention to who said what\.Conversation: \{context\}Question: \{question\}Answer:"""

D\.4: Dynamic Endogenous Skill DiscoveryYou are an expert in contrastive analysis of interactive agents with different memory setups\.EXPERIMENTAL SETTING \(READ CAREFULLY\) We compare two runs of thesametask\_id\.•BASELINE simulatesno practical context limit: the agent’s policy model can use thefull raw interaction trace\(steps or dialogue turns as exported\) in context \(no llm\_file / external folder memory\)\.•OUR METHOD: anexternal memory folderupdated and read viaBash\(files, notes, summaries\)\. The live policy prompt is not carrying the entire raw trace\.Outcome here: OUR METHOD succeeded\. BASELINE failed\. original memory management promptTASKTask ID: IDTask Category: taskInitial Observation: initial observationSuccess Trajectory: OUR METHOD’s trajectoryFailed Trajectory: BASELINE’s trajectoryYOUR JOB — ENDOGENOUS SKILLS \(PRIMARY HYPOTHESIS\)Your main hypothesis is that baseline failed because super\-long raw history in context causes hallucination, lost\-in\-the\-middle effects, or attention dilution, not because the task is impossible\. Endogenous skills are advantages of compressed /structured external memory that avoid relying on an ever\-growing raw trace\.For each skill, tie it to why raw\-history overload hurts and how OUR METHOD sidesteps it\. Do not reduce this to generic “better planning” unless you anchor it in context\-length /memory\-structure mechanisms\.<Skill Fields Specification\>

D\.5: Dynamic Exogenous Skill DiscoveryYou are an expert in contrastive analysis of interactive agents with different memory setups\.EXPERIMENTAL SETTING \(READ CAREFULLY\) We compare two runs of thesametask\_id\.•BASELINE simulatesno practical context limit: the agent’s policy model can use thefull raw interaction trace\(steps or dialogue turns as exported\) in context \(no llm\_file / external folder memory\)\.•OUR METHOD: anexternal memory folderupdated and read viaBash\(files, notes, summaries\)\. The live policy prompt is not carrying the entire raw trace\.Outcome here: OUR METHOD failed\. BASELINE succeeded\. original memory management promptTASKTask ID: IDTask Category: taskInitial Observation: initial observationSuccess Trajectory: BASELINE’s trajectoryFailed Trajectory: OUR METHOD’s trajectoryYOUR JOB — EXOGENOUS SKILLS \(PRIMARY HYPOTHESIS\)Your main hypothesis is information loss in the Bash\-managed llm file pipeline, e\.g, facts never written, overwritten summaries, wrong file read, grep misses, truncation\. Exogenous skills are concrete gaps or failure modes in external memory that this contrast exposes\. If the trace includes Bash/file/memory tool strings, use them as evidence\.<Skill Fields Specification\>

D\.6: Skill Fields SpecificationSkill fields \(every skill\):•skill\_id: end\_001, end\_002,… in order \(unique within this JSON\)\.•title: 3\-5 words only \(no trailing punctuation\)\.•principle: 1\-2 sentences stating the core transferable idea\.•memory\_prompt\_improvement: 1\-3 sentences \- how to change the memory\-writer / extraction prompt or schema so future runs better encode this skill \(or "" if not applicable\)\.Return only valid JSON \(no markdown fences, no extra text\) with this schema:\{"task\_id": \.\.\.,"task\_category": "\.\.\.","comparison\_type": "endogenous" or "exogenous","winning\_method": "ours\_llm\_file" or "baseline\_full\_raw\_history","losing\_method": "baseline\_full\_raw\_history" or "ours\_llm\_file","hypothesis\_alignment": \.\.\.,"divergence": \{"first\_divergence\_step": <int or null\>,"summary": "<when trajectories diverge\>"\},"skills": \[ \{"skill\_id": "end\_001" or "exo\_001","title": "<exactly three to five words\>","principle": "<one or two sentences\>","evidence": \.\.\.,"how\_to\_reinforce/acquire": \.\.\.,"memory\_prompt\_improvement": \.\.\.\} \],"root\_cause\_summary": "<2\-4 sentences\>"\}

D\.7: LLM Judge Prompt for Conversational TasksYou are an expert judge evaluating and labeling an answer to a question as ’CORRECT’ or ’WRONG’\. You will be given the following data:•a question•a gold \(ground truth\) answer•a generated answer, which you will score as CORRECT or WRONG\.The point of the question is to ask about something based on two users’ conversations\. The gold answer will usually be a concise and short answer that includes the referenced topic, for example:Question: Do you remember what I got the last time I went to Hawaii?Gold answer: A shell necklaceThe generated answer might be longer, but you should be generous with your grading, as long as it touches on the same topic as the gold answer, it should be counted as CORRECT\.For time\-related questions, the gold answer will be a specific date, month, or year\. The generated answer might include relative references \(e\.g\., last Tuesday\), but you should be generous if it refers to the same time period as the gold answer, mark it CORRECT, even if the format differs \(e\.g\., "May 7th" vs\. "7 May"\)\.Now it’s time for the real question:Question: questionGold answer: gold\_answerGenerated answer: generated\_answerFirst, provide a short \(one sentence\) explanation of your reasoning, then finish with CORRECT or WRONG\.Do NOT include both CORRECT and WRONG in your response, or it will break the evaluation script\.Return the label in JSON format with the key as "label"\.

## Appendix EMemory Management: Hierarchical Workspace Examples

The following excerpts illustrate the structured memory representations generated by the memory manager using Claude Sonnet 4\.5 as the backbone\. These examples, drawn from the LoCoMo benchmark, demonstrate how raw dialogue is transformed into a navigable, file\-centric hierarchy that preserves temporal anchors and provenance\.

People/Caroline\.mdLGBTQ Support Group Attendance\-Attended an LGBTQ support group on May 7, 2023 \(the day before the conversation\) \(D1:3\)\-Found the experience "powerful" \(D1:3\)\-Was inspired by transgender stories shared at the group \(D1:5\)\-The group made her feel accepted and gave her courage \(D1:7\)Career Plans\-Interested in counseling or mental health work \(D1:11\)\-Wants to support people with similar issues \(D1:11\)\-Planning to continue education and explore career options \(D1:9\)

Dates/May2023\.mdMay 7, 2023\-Caroline attended an LGBTQ support group \(D1:3\)\-She found the experience "powerful" \(D1:3\)May 8, 2023\-Caroline and Melanie had their first conversation session at 1:56 pm \(Session 1\)\-Caroline shared about her experience at the LGBTQ support group from the previous day \(D1:3\)

Topics/LGBTQ/support group\.mdCaroline’s Attendance\-Caroline attended the group on May 7, 2023 \(the day before their conversation\) \(D1:3\)\-"I went to a LGBTQ support group yesterday and it was so powerful\." \(D1:3\)Impact on Caroline\-She found transgender stories shared at the group inspiring \(D1:5\)\-"The support group has made me feel accepted and given me courage to embrace myself\." \(D1:7\)\-The experience may have influenced her interest in counseling or mental health work \(D1:11\)

Similar Articles

Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability

arXiv cs.CL

This paper presents the first systematic exploration of filesystem-based memory for LLM agents, formalizing roles of management, search, and execution agents around a shared memory store. It finds that organization primarily reduces retrieval cost but does not yet improve answer quality, and that tooling choices affect store shape as much as model selection.

An open source natural temporal memory for claude code, hermes and openclaw agent

Reddit r/ArtificialInteligence

agentmemory is an open-source library that provides natural temporal memory for AI agents like Claude Code, Hermes, and OpenClaw. It uses a three-tier architecture with hybrid retrieval (BM25, vector, knowledge graph) and Ebbinghaus decay, achieving ~92% fewer tokens and 200x more tool calls before context limits.