Learning User-Aware Recall: Personalized Retrieval in Long-Term Conversational Memory

# Personalized Retrieval in Long-Term Conversational Memory
Source: [https://arxiv.org/html/2607.00017](https://arxiv.org/html/2607.00017)
## Learning User\-Aware Recall: Personalized Retrieval in Long\-Term Conversational Memory

ZhiShu Jiang1, Haibo Liu1, Xin Shen1,2, Guanqiang Qi1, Chenxi Miao1, Weikang Li1,†\\dagger, Liwei Qian1, Xin Pei1, Jizhou Huang1

1Baidu Inc\.2The University of Queensland jiangzhishu@bajtu\.edu\.cn, wavejkd@pku\.edu\.cn

###### Abstract

Long\-term conversational agents are expected to remember past interactions, but memory is useful only when the right evidence is recalled for the right user\. Existing memory\-augmented LLM agents have made progress in building compact memory banks, yet retrieval is still often driven by query\-centered similarity or fixed ranking rules, leaving user\-conditioned relevance underexplored\. To address this gap, we proposeProfile\-guidedPersonalizedRetrievalOptimization \(PPRO\), a retrieval\-centric framework that makes memory retrieval both user\-aware and optimizable\. PPRO builds episodic and semantic memory banks from dialogue histories and derives a user profile from accumulated memories\. The profile serves as an explicit personalized prior in memory ranking, allowing retrieval to account for stable user attributes, preferences, and relationships\. PPRO further trains a query rewriter with Group Relative Policy Optimization, using both evidence retrieval quality and downstream answer quality as feedback while keeping the memory banks and answer model fixed\. Experiments on LoCoMo and LongMemEval\-S show consistent gains over training\-free memory systems and training\-based baselines\. Ablation studies further show that both profile\-guided ranking and retrieval\-oriented rewriting contribute substantially to performance, highlighting retrieval optimization as a key factor in personalized long\-term memory use\.

Learning User\-Aware Recall: Personalized Retrieval in Long\-Term Conversational Memory

ZhiShu Jiang1, Haibo Liu1, Xin Shen1,2, Guanqiang Qi1, Chenxi Miao1,Weikang Li1,†\\dagger, Liwei Qian1, Xin Pei1, Jizhou Huang11Baidu Inc\.2The University of Queenslandjiangzhishu@bajtu\.edu\.cn, wavejkd@pku\.edu\.cn††thanks:†\\daggerCorresponding author

![Refer to caption](https://arxiv.org/html/2607.00017v1/x1.png)Figure 1:Motivating example of profile\-guided personalized memory retrieval\.Query\-only retrieval may return surface\-matching evidence, while profile\-guided retrieval uses the user’s preferences, plans, and relationships to produce a more personalized answer\.## 1Introduction

Large language model \(LLM\) agents are increasingly expected to support long\-term personalized interaction, where useful knowledge must be accumulated across extended dialogue histories and reused in later conversations\(Maharanaet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib10); Packeret al\.,[2023](https://arxiv.org/html/2607.00017#bib.bib21); Wuet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib60)\)\. Recent memory\-augmented systems have made substantial progress in reducing the cost of long\-context inputs by extracting, compressing, and organizing historical interactions into external memory banks\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib14); Xuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib16); Liuet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib20); Kanget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib15); Lewiset al\.,[2020](https://arxiv.org/html/2607.00017#bib.bib49); Shenet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib61); Liaoet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib59)\)\. As these systems mature, a central challenge moves from memory construction alone to memory use at inference time: given a user query, the agent must retrieve evidence that is not only semantically relevant to the query, but also appropriate for the user behind the queryShenet al\.\([2021c](https://arxiv.org/html/2607.00017#bib.bib62)\)\. In long\-term dialogue, different users accumulate distinct memories that reflect their relationships, topical interests, and life contextsShenet al\.\([2021a](https://arxiv.org/html/2607.00017#bib.bib63),[b](https://arxiv.org/html/2607.00017#bib.bib64)\)\. The same query may therefore correspond to different information needs depending on who is asking\. As illustrated in Figure[1](https://arxiv.org/html/2607.00017#S0.F1), personalized memory retrieval requires going beyond surface query matching: the retriever must identify evidence that is jointly relevant to the query and to the user’s long\-term profile\.

Existing memory systems have not fully addressed this user\-conditioned retrieval problem\. Most work focuses on memory construction and management, including extraction, compression, hierarchical organization, graph\-based indexing, and read\-write operations\(Liuet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib20); Xuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib16); Chhikaraet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib14)\)\. These methods improve the coverage and efficiency of stored memories, but retrieval is often treated as a fixed downstream step based on semantic matching or heuristic scoring\(Gutiérrezet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib25); Kim and Jang,[2025](https://arxiv.org/html/2607.00017#bib.bib38); Karpukhinet al\.,[2020](https://arxiv.org/html/2607.00017#bib.bib50)\)\. Personalized dialogue systems, on the other hand, model user\-specific information\(Zhuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib40); Lianget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib39); Zhanget al\.,[2018](https://arxiv.org/html/2607.00017#bib.bib54); Salemiet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib56)\), but usually focus on constructing personalized memory content or using profiles during generation rather than injecting user\-level priors into retrieval ranking\. As a result, current memory pipelines may retrieve memories that match the query surface while failing to prioritize evidence that is more informative under the user’s stable attributes, preferences, and relationships\.

To address these issues, we proposeProfile\-guidedPersonalizedRetrievalOptimization \(PP RO\), a retrieval\-centric framework for personalized long\-term conversational memory\. PPRO first constructs a hierarchical memory representation from dialogue histories\. It extracts episodic memories as fine\-grained factual statements, aggregates related episodic memories into semantic memories, and summarizes stable user attributes, preferences, and relationships into a user profile\. During inference, PPRO performs dual\-path retrieval over both episodic and semantic memories\. More importantly, the user profile is not only provided to the answer model as context, but also injected into the retrieval score as an embedding\-level personalized prior\. This design allows the retriever to rank memories by considering both query relevance and user\-level relevance, making memory retrieval more sensitive to personalized information needs\.

PPRO further optimizes retrieval through a trainable query rewriter\. Because retrieving discrete memory items is non\-differentiable, gradients cannot flow from answer quality back to the retriever\. To bypass this, we train a query rewriter with Group Relative Policy Optimization \(GRPO\)\(Shaoet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib12); Schulmanet al\.,[2017](https://arxiv.org/html/2607.00017#bib.bib53)\): multiple rewritten queries are sampled and rewarded based on both evidence retrieval quality and final answer quality, while the memory banks, embedding model, and answer model remain frozen\.

We evaluate PPRO on LoCoMo\(Maharanaet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib10)\)and LongMemEval\-S\(Wuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib43)\), two benchmarks for long\-term conversational question answering\. PPRO consistently outperforms both training\-free memory systems and training\-based baselines across both benchmarks, and ablation studies confirm the contribution of user\-profile guidance, dual\-path retrieval, retrieval dimensions, and GRPO\-based query rewriting\. The results show that personalized retrieval optimization is an effective complement to memory construction\. Overall, the contributions of our work are threefold:

- •We formulate personalized memory retrieval as a key problem in long\-term conversational agents, emphasizing that retrieved evidence should be relevant to both the input query and the target user\.
- •We propose PPRO, combining profile\-guided retrieval and GRPO\-based query rewriting for personalized memory recall\.
- •Experiments on LoCoMo and LongMemEval\-S show consistent gains over both training\-free and training\-based baselines\.

## 2Related Work

### 2\.1Long\-Term Conversational Memory

As the field progresses, research on long\-term conversational memory has evolved from memory construction and organization, to retrieval mechanism design, and more recently to learning\-based optimization\(Sumerset al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib27); Jianget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib30)\)\.

Memory Construction and Organization\.Effective memory organization is fundamental to long\-term conversational agents\. MemGPT\(Packeret al\.,[2023](https://arxiv.org/html/2607.00017#bib.bib21)\)introduces OS\-inspired virtual context management\. A\-Mem\(Xuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib16)\)employs Zettelkasten\-inspired dynamic indexing, Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib14)\)builds graph\-based memory banks, and MemoryOS\(Kanget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib15)\)proposes a multi\-level memory operating system\. SimpleMem\(Liuet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib20)\)achieves strong performance through semantic lossless compression with multi\-view indexing\.

Memory Retrieval Mechanisms\.Beyond better memory organization, how to accurately retrieve relevant memories given a query is equally critical\. HippoRAG\(Gutiérrezet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib25)\)draws from hippocampal indexing theory for multi\-hop retrieval\. THEANINE\(Baeet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib37)\)retrieves via temporal\-causal graph traversal, SynapticRAG\(Kim and Jang,[2025](https://arxiv.org/html/2607.00017#bib.bib38)\)applies biologically\-inspired synaptic propagation for temporal scoring, Zep\(Rasmussenet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib26)\)leverages temporal knowledge graphs, and CDMem\(Wang and others,[2025](https://arxiv.org/html/2607.00017#bib.bib42)\)applies context\-dependent indexing conditioned on task state\.

RL\-Based Memory Optimization\.More recently, reinforcement learning has been applied to optimize memory systems end\-to\-end\. Memory\-R1\(Yanet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib11)\)trains policies for memory CRUD operations via GRPO\. MEM1\(Zhouet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib19)\)optimizes the think\-search\-answer loop, MemAgent\(Luet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib22)\)trains segmented reading for long\-context compression, and Mem\-T\(Yueet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib31)\)employs memory\-of\-thought augmented GRPO\. However, RL in these methods targets memory management or reasoning without being tightly coupled with the retrieval process\. Our work unifies reinforcement learning with user\-aware personalized retrieval, optimizing the retrieval process with reward signals derived from both evidence retrieval quality and answer quality\.

### 2\.2User\-Aware Retrieval

User\-aware retrieval focuses on leveraging user\-specific information to improve retrieval relevance\. In recommendation, recent work enhances user\-aware retrieval through personalized representations and preference modeling\(Tanet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib32); Konget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib33); Chenet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib34); Zhanget al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib35); Jiaet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib36)\)\. In conversation, PRIME\(Zhuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib40)\)maps cognitive dual\-memory theory onto LLM personalization, RMM\(Lianget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib39)\)introduces reflective memory management with RL\-refined retrieval, InsideOut\(Zhaoet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib9)\)evolves user\-centric core memory trees, and EMG\-RAG\(Lee and others,[2024](https://arxiv.org/html/2607.00017#bib.bib41)\)crafts personalized agents through editable memory graphs\. Our work injects user profile embeddings directly into the retrieval scoring function as a personalized ranking prior, making user\-aware retrieval end\-to\-end optimizable\.

## 3Methodology

We present Profile\-guided Personalized Retrieval Optimization \(PPRO\), a retrieval\-centric framework for personalized long\-term conversational memory\. As shown in Figure[2](https://arxiv.org/html/2607.00017#S3.F2), PPRO constructs episodic memories, semantic memories, and user profiles from dialogue histories, and then uses the profile as a personalized prior to guide dual\-path retrieval\. A GRPO\-trained query rewriter further optimizes retrieval using evidence retrieval quality and answer quality as feedback\.

![Refer to caption](https://arxiv.org/html/2607.00017v1/x2.png)Figure 2:Overview of the PPRO framework\. The upper panel shows offline memory construction, while the lower panel shows online retrieval, answer generation, and GRPO optimization\. PPRO builds episodic memories, semantic memories, and user profiles from dialogue histories, then uses profile\-guided dual\-path retrieval to recall memories\. A GRPO\-trained query rewriter further optimizes retrieval with evidence\-level and answer\-level feedback\.### 3\.1Problem Formulation

Given a target useruu, we assume access to the user’s long\-term dialogue historyDu=\{d1,…,dT\}D\_\{u\}=\\\{d\_\{1\},\\ldots,d\_\{T\}\\\}, where each dialoguedtd\_\{t\}contains a sequence of timestamped utterances from multiple speakers\. Given a history\-grounded queryqq, the task is to generate an answeryythat matches the reference answery∗y^\{\*\}\.

We focus on personalized memory retrieval as the key intermediate step\. PPRO representsDuD\_\{u\}with an episodic memory bankℳuE\\mathcal\{M\}^\{E\}\_\{u\}, a semantic memory bankℳuS\\mathcal\{M\}^\{S\}\_\{u\}, and a user profile𝒫u\\mathcal\{P\}\_\{u\}\. At inference time, the retriever selects memories from the episodic and semantic memory banks that are both query\-relevant and user\-relevant\. These retrieved memories, together with the user profile, are then provided to a frozen answer model𝒜\\mathcal\{A\}to generate the final answer\.

### 3\.2Offline Memory Construction

The offline phase converts raw dialogue histories into persistent user\-specific memory resources\. For each target user, PPRO builds three complementary components: episodic memories for fine\-grained factual evidence, semantic memories for cross\-dialogue aggregation, and a user profile for stable personalized information\. We use the same instruction\-following language model𝒢\\mathcal\{G\}across all offline construction stages, with task\-specific prompts\.

Episodic Memory\.For each dialoguedtd\_\{t\},𝒢\\mathcal\{G\}converts dialogue utterances into a set of atomic episodic memories\. Each memory uniteeis a factual textual statement paired with retrieval dimensions:

ℳu,tE=𝒢​\(dt,u\),e=\(c,δ\)∈ℳu,tE,\\mathcal\{M\}\_\{u,t\}^\{E\}=\\mathcal\{G\}\(d\_\{t\},u\),\\quad e=\(c,\\delta\)\\in\\mathcal\{M\}\_\{u,t\}^\{E\},\(1\)whereccis the textual content andδ\\deltadenotes the retrieval dimensions attached to the memory, including tense, subjectivity, and certainty\. These retrieval dimensions provide structured cues for consistent memory extraction and are later used as dimension\-aware matching signals in retrieval scoring\. The full episodic memory bank for useruuisℳuE=⋃t=1Tℳu,tE\\mathcal\{M\}^\{E\}\_\{u\}=\\bigcup\_\{t=1\}^\{T\}\\mathcal\{M\}^\{E\}\_\{u,t\}, providing fine\-grained factual evidence for downstream retrieval\.

Semantic Memory\.Episodic memories are atomic and dialogue\-local, making it difficult to answer queries that require cross\-dialogue synthesis\. To capture higher\-level patterns, PPRO introduces a semantic memory bankℳuS\\mathcal\{M\}\_\{u\}^\{S\}by clustering episodic memories based on cosine similarity:

sim​\(ei,ej\)=cos⁡\(ℰ​\(ei\),ℰ​\(ej\)\),\\mathrm\{sim\}\(e\_\{i\},e\_\{j\}\)=\\cos\(\\mathcal\{E\}\(e\_\{i\}\),\\mathcal\{E\}\(e\_\{j\}\)\),\(2\)whereℰ\\mathcal\{E\}denotes the embedding model\. Episodic memories whose similarity exceeds a thresholdτc\\tau\_\{c\}are grouped into clustersCgC\_\{g\}\. For each resulting clusterCgC\_\{g\}, the language model𝒢\\mathcal\{G\}summarizes its episodic memories into a semantic memory unit:

s=𝒢​\(Cg\),s∈ℳuS\.s=\\mathcal\{G\}\(C\_\{g\}\),\\quad s\\in\\mathcal\{M\}\_\{u\}^\{S\}\.\(3\)
User Profile\.While episodic memories preserve local facts and semantic memories capture cross\-dialogue patterns, many personalized queries benefit from a high\-level representation of a user’s stable characteristics, including attributes, preferences, and relationships\. Inspired by personalized recommendation in recommender systemsNguyenet al\.\([2026](https://arxiv.org/html/2607.00017#bib.bib8)\), we propose aggregating user\-level profiles to enhance memory retrieval during the online stage\. PPRO constructs a textual user profile𝒫u\\mathcal\{P\}\_\{u\}by summarizing the user’s semantic memory bank:

𝒫u=𝒢​\(ℳuS\),𝐩u=ℰ​\(𝒫u\),\\mathcal\{P\}\_\{u\}=\\mathcal\{G\}\(\\mathcal\{M\}^\{S\}\_\{u\}\),\\quad\\mathbf\{p\}\_\{u\}=\\mathcal\{E\}\(\\mathcal\{P\}\_\{u\}\),\(4\)where𝒢\\mathcal\{G\}summarizes long\-term preferences, occupations, relationships, habits, and recurring interests, andℰ\\mathcal\{E\}is the shared embedding model\. The profile is stored both as text𝒫u\\mathcal\{P\}\_\{u\}and as an embedding vector𝐩u\\mathbf\{p\}\_\{u\}, which later serves as a personalized prior for online retrieval\.

### 3\.3Online Inference and Optimization

At the online stage, PPRO first rewrites the input query into a retrieval\-oriented form, then retrieves complementary evidence from episodic and semantic memory banks, and finally generates an answer from the assembled memory context\. Personalization is introduced by injecting the user profile as a prior into episodic memory scoring, while the query rewriter is further optimized with GRPO using retrieval and answer\-quality feedback\. The complete procedure is summarized in Algorithm[1](https://arxiv.org/html/2607.00017#alg1)in the Appendix\.

Retrieval\-Oriented Query Rewriting\.The rewriter modelfθf\_\{\\theta\}transforms the original queryqq, conditioned on dialogue contextCC, into a retrieval\-oriented queryq′q^\{\\prime\}and retrieval dimensionsδq\\delta\_\{q\}:

\(q′,δq\)=fθ​\(q,C\)\.\(q^\{\\prime\},\\delta\_\{q\}\)=f\_\{\\theta\}\(q,C\)\.\(5\)The rewritten queryq′q^\{\\prime\}is used for memory retrieval, whileδq\\delta\_\{q\}provides dimension\-level signals that complement semantic similarity in episodic retrieval\.

Dual\-path Memory Retrieval\.PPRO retrieves from episodic and semantic memory banks in parallel\. The episodic path focuses on fine\-grained factual evidence, while the semantic path provides compact cross\-dialogue summaries\.

Episodic Retrieval\.PPRO scores each candidateeeby fusing query relevance with a profile\-based personalized prior, augmented by a retrieval\-dimension boost:

σ​\(e\)=λ​σq​\(e\)\+\(1−λ\)​σp​\(e\)\+bδ​\(e\),\\sigma\(e\)=\\lambda\\,\\sigma\_\{q\}\(e\)\+\(1\-\\lambda\)\\,\\sigma\_\{p\}\(e\)\+b\_\{\\delta\}\(e\),\(6\)whereσq​\(e\)=cos⁡\(ℰ​\(q′\),ℰ​\(e\)\)\\sigma\_\{q\}\(e\)=\\cos\(\\mathcal\{E\}\(q^\{\\prime\}\),\\mathcal\{E\}\(e\)\)measures query\-memory relevance,σp​\(e\)=cos⁡\(𝐩u,ℰ​\(e\)\)\\sigma\_\{p\}\(e\)=\\cos\(\\mathbf\{p\}\_\{u\},\\mathcal\{E\}\(e\)\)captures user\-profile relevance via the profile embedding𝐩u\\mathbf\{p\}\_\{u\}, andbδ​\(e\)b\_\{\\delta\}\(e\)is the number of matching retrieval dimensions betweenδq\\delta\_\{q\}andδe\\delta\_\{e\}multiplied by a fixed bonus \(0\.05 per match across three dimensions: tense, subjectivity, and certainty\)\. The hyperparameterλ\\lambdacontrols the trade\-off between query\-driven and profile\-driven retrieval\.

Semantic Retrieval\.In parallel, PPRO retrieves fromℳuS\\mathcal\{M\}^\{S\}\_\{u\}using the same rewritten queryq′q^\{\\prime\}\. Because semantic memories are already aggregated from user\-specific episodic memories, they provide compact high\-level context and are ranked directly by query similarity\. Semantic memories are therefore ranked by standard cosine similaritycos⁡\(ℰ​\(q′\),ℰ​\(s\)\)\\cos\(\\mathcal\{E\}\(q^\{\\prime\}\),\\mathcal\{E\}\(s\)\)\.

Answer Generation\.The retrieved memories are concatenated into a unified memory context𝒞m=\[𝒫u;ℛE;ℛS\]\\mathcal\{C\}\_\{m\}=\[\\mathcal\{P\}\_\{u\};\\mathcal\{R\}^\{E\};\\mathcal\{R\}^\{S\}\], and the answer model𝒜\\mathcal\{A\}generates the answer:

y=𝒜​\(q,𝒞m\)\.y=\\mathcal\{A\}\(q,\\mathcal\{C\}\_\{m\}\)\.\(7\)

### 3\.4GRPO\-based Optimization

The quality of memory\-augmented QA depends heavily on whether the rewritten query can retrieve the correct evidence\. Following recent work that uses multi\-aspect feedback to train query rewritersWanget al\.\([2024](https://arxiv.org/html/2607.00017#bib.bib13)\); Maet al\.\([2023](https://arxiv.org/html/2607.00017#bib.bib51)\); Wanget al\.\([2023](https://arxiv.org/html/2607.00017#bib.bib58)\), we optimize the rewriterfθf\_\{\\theta\}through the online inference pipeline using Group Relative Policy Optimization \(GRPO\)Shaoet al\.\([2024](https://arxiv.org/html/2607.00017#bib.bib12)\)\. Instead of optimizing rewriting quality in isolation,fθf\_\{\\theta\}is trained with feedback from both evidence retrieval quality and final answer quality\.

Policy and Environment\.The rewriterfθf\_\{\\theta\}is treated as the policyπθ\\pi\_\{\\theta\}\. For each training data, the state is the original queryqqwith its dialogue contextCC, and the policy samples a group ofNNactions, corresponding to rewritten queries\{q1′,…,qN′\}\\\{q^\{\\prime\}\_\{1\},\\ldots,q^\{\\prime\}\_\{N\}\\\}fromπθ\(⋅∣q,C\)\\pi\_\{\\theta\}\(\\cdot\\mid q,C\)\. For each sampled rewriteqi′q^\{\\prime\}\_\{i\}, PPRO executes the online inference pipeline to obtain retrieved episodic memoriesℛiE\\mathcal\{R\}^\{E\}\_\{i\}, semantic memoriesℛiS\\mathcal\{R\}^\{S\}\_\{i\}, and a generated answeryiy\_\{i\}\.

Reward Design\.The reward combines a retrieval reward and an answer reward:

Ri=α​R~a​n​s,i\+\(1−α\)​R~r​e​t,i,R\_\{i\}=\\alpha\\widetilde\{R\}\_\{ans,i\}\+\(1\-\\alpha\)\\widetilde\{R\}\_\{ret,i\},\(8\)whereα\\alphacontrols the trade\-off between the two signals\. The retrieval rewardRr​e​t,iR\_\{ret,i\}is the F1 score between the source turns of retrieved episodic memories and ground\-truth evidence annotations\. The answer rewardRa​n​s,i=BLEU​\-​1​\(yi,y∗\)R\_\{ans,i\}=\\mathrm\{BLEU\\text\{\-\}1\}\(y\_\{i\},y^\{\*\}\)measures unigram precision between the generated answer and the referencePapineniet al\.\([2002](https://arxiv.org/html/2607.00017#bib.bib24)\)\. Both rewards are normalized via EMA\-based adaptive scaling before being combined\.

Group\-relative Advantage and Optimization\.After executing the pipeline for each of theNNrewrites, the group\-relative advantage is:

A^i=Ri−mean​\(\{Rj\}j=1N\)std​\(\{Rj\}j=1N\)\+ϵ\.\\hat\{A\}\_\{i\}=\\frac\{R\_\{i\}\-\\mathrm\{mean\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{N\}\)\}\{\\mathrm\{std\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{N\}\)\+\\epsilon\}\.\(9\)This eliminates the need for a separately trained critic and directly compares alternative rewrites for the same query\. The rewriter is then optimized with the standard GRPO clipped policy\-gradient objective with a KL penalty \(full formulation inAppendix[A](https://arxiv.org/html/2607.00017#A1)\)\.

## 4Experiment

Table 1:Token\-level F1 and BLEU\-1 results on LoCoMo\.Upper: comparison with training\-free baselines on the full benchmark\.Lower: comparison with training\-based baselines the test split\.Bold= best,underline= second best within each group\.### 4\.1Dataset

We evaluate on two benchmarks that test long\-term conversational memory:LoCoMoMaharanaet al\.\([2024](https://arxiv.org/html/2607.00017#bib.bib10)\)andLongMemEval\-SWuet al\.\([2025](https://arxiv.org/html/2607.00017#bib.bib43)\)\. LoCoMo contains multi\-session dyadic conversations paired with QA annotations spanning four question categories: single\-hop factual recall, temporal reasoning, open\-domain, and multi\-hop\. Following the conversation\-level split protocol ofYanet al\.\([2025](https://arxiv.org/html/2607.00017#bib.bib11)\), we partition the ten conversations into a 1:1:8 train/validation/test split, yielding 152/81/1,307 QA pairs respectively \(detailed statistics in Table[5](https://arxiv.org/html/2607.00017#A6.T5)\)\. The training and validation sets are used exclusively for optimizing the query rewriter; all main results are reported on the held\-out test set\. LongMemEval\-SWuet al\.\([2025](https://arxiv.org/html/2607.00017#bib.bib43)\)features exceptionally long interaction histories that require precise answer localization\. We evaluate on five categories:Single\-Session\-User\(S\-U\),Single\-Session\-Preference\(S\-P\),Temporal\-Reasoning\(Tmp\.\),Knowledge\-Update\(K\-U\.\), andMulti\-Session\(Mlt\.\)\.

### 4\.2Baselines

We compare PPRO against two families of baselines \(detailed descriptions inAppendix[B\.4](https://arxiv.org/html/2607.00017#A2.SS4)\)\.Training\-free baselinesinclude Full\-Context, MemGPT\(Packeret al\.,[2023](https://arxiv.org/html/2607.00017#bib.bib21)\), A\-Mem\(Xuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib16)\), LightMem\(Fanget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib17)\), Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib14)\), and SimpleMem\(Liuet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib20)\); we adopt results reported byLiuet al\.\([2026](https://arxiv.org/html/2607.00017#bib.bib20)\)under identical evaluation metrics\.Training\-based baselinesinclude MEM1\(Zhouet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib19)\)and MemAgent\(Luet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib22)\); we deploy their official Qwen2\.5\-7B checkpoints and evaluate on our test split\.

### 4\.3Evaluation Metrics

We use three metrics across the two benchmarks\.Token\-level F1Rajpurkaret al\.\([2016](https://arxiv.org/html/2607.00017#bib.bib23)\)measures the overlap between predicted and reference answer tokens; it is our primary metric on LoCoMo\.BLEU\-1Papineniet al\.\([2002](https://arxiv.org/html/2607.00017#bib.bib24)\)serves as a secondary metric that rewards precision and penalizes irrelevant tokens in the prediction\.LLM\-as\-Judge Accuracyis used for LongMemEval\-S: followingLiuet al\.\([2026](https://arxiv.org/html/2607.00017#bib.bib20)\), we employgpt\-4\.1\-minito produce binaryCorrect/Wronglabels and report accuracy\.

Table 2:LLM\-as\-Judge Accuracy on LongMemEval\-S \(weighted by category size\)\. Baseline results are fromLiuet al\.\([2026](https://arxiv.org/html/2607.00017#bib.bib20)\)\.
Table 3:Ablation study on the LoCoMo test split with Qwen2\.5\-7B backbone\. Per\-category breakdown is in Table[6](https://arxiv.org/html/2607.00017#A6.T6)\.

### 4\.4Experiment Setting

To ensure fair comparison, we control the backbone of the answer model in PPRO to be identical to that of the baselines in each evaluation group\. For LoCoMo, we evaluate with GPT\-4o, Qwen2\.5\-3B, and Qwen3\-8B as backbones for the comparison with training\-free baselines, and Qwen2\.5\-7B for the comparison with training\-based baselines\. For LongMemEval\-S, we usegpt\-4\.1\-minias the backbone followingLiuet al\.\([2026](https://arxiv.org/html/2607.00017#bib.bib20)\)\. The rewriter model shares the same backbone as the answer model in each setting\. Detailed hyperparameter settings, offline memory construction, and embedding models are provided inAppendix[B\.3](https://arxiv.org/html/2607.00017#A2.SS3)\.

![Refer to caption](https://arxiv.org/html/2607.00017v1/x3.png)Figure 3:Hyperparameter sensitivity on the LoCoMo test split\. \(a\) Query\-profile trade\-offλ\\lambda\(optimal at 0\.8\)\. \(b\) Reward signal trade\-offα\\alpha\(optimal at 0\.2\)\.
### 4\.5Main Results

Results on LoCoMo\.Table[1](https://arxiv.org/html/2607.00017#S4.T1)presents the main results on LoCoMo\. In the comparison with training\-free baselines, PPRO achieves the best overall F1 across all three backbones, consistently outperforming the previous best SimpleMem by 7–19 points\. The gains are particularly pronounced on Temporal and MultiHop categories, confirming the effectiveness of our retrieval dimensions and profile\-guided personalization\. On GPT\-4o, Full\-Context achieves the highest SingleHop F1 as its strong long\-context capability suffices for single\-hop recall\. On Qwen3\-8B, SimpleMem outperforms PPRO on SingleHop due to its adaptive retrieval size that avoids context overload for smaller models\.

Compared to training\-based baselines, PPRO achieves the best overall F1, outperforming MemAgent by more than 5 points, with the largest gains in Temporal and SingleHop\. MemAgent achieves a higher MultiHop F1, likely because it is trained on HotpotQA which directly reinforces multi\-hop evidence chaining\.

Results on LongMemEval\-S\.Table[3](https://arxiv.org/html/2607.00017#S4.T3)shows that PPRO achieves the best overall accuracy, outperforming all baselines\. The advantage is most evident onTemporal\-ReasoningandMulti\-Session, demonstrating that our hierarchical memory generalizes to longer interaction histories\. PPRO also achieves the bestSingle\-Session\-Preferenceaccuracy, validating that the user profile effectively captures personalized preferences\.

### 4\.6Ablation Study

We ablate each component by: \(1\)w/o Episodic Memory: removing episodic memory; \(2\)w/o Semantic Memory: removing semantic memory; \(3\)w/o Profile Injection: disabling profile\-guided retrieval and removing profile context from answer generation; \(4\)w/o Retrieval Dimensions: disabling the retrieval dimensions in both rewriting and retrieval boosting; \(5\)w/o GRPO Training: removing GRPO training for the query rewriter\.

As shown in Table[3](https://arxiv.org/html/2607.00017#S4.T3), all components contribute positively, with removing episodic memory causing the largest drop as it provides the primary evidence source\. Removing semantic memory also leads to a substantial decline, confirming its complementary role in providing cross\-dialogue aggregated context\. Profile injection, retrieval dimensions, and GRPO training each provide consistent gains, demonstrating that they collectively enhance online retrieval quality through personalization, dimension matching, and learned query rewriting\.

![Refer to caption](https://arxiv.org/html/2607.00017v1/x4.png)Figure 4:Evidence Recall with and without profile\-guided retrieval\. Profile guidance yields increasing gains at larger retrieval sizes\.Case Study: Resolving Temporally Confounded Charity MemoriesQuestionWhat was the main goal of the money raised from the charity tournament organized by John and his friends in May 2022?Gold AnswerRaise money for a dog shelter\.EvidenceJohn: “Our main goal was to raise money for a dog shelter, which is not far from the street where I live\. And we did it\!”SettingKey Retrieved SignalModel PredictionFailure ModeF1No Memory–No information availableNo evidence0\.00\+ EpisodicChildren’s hospital, Oct\. 2022To support a children’s hospitalTemporal confusion0\.18\+ SemanticDog shelter; secondary action: homelessTo raise money for charity and help the homelessPrimary/secondary goal confusion0\.36\+ User ProfileCharity/community\-impact priorTo raise money for charityMissing specific target0\.40\+ GRPO\(Full\)May 2022 tournament matched to dog shelterRaised money for a dog shelter near his streetCorrect0\.67Table 4:Case study on temporally confounded memory retrieval\.
### 4\.7Discussion

Hyperparameter Sensitivity\.We analyze the sensitivity of PPRO to two key hyperparameters\.

Query\-profile trade\-offλ\\lambda\.Figure[3](https://arxiv.org/html/2607.00017#S4.F3)\(a\) shows how the balance between query and profile embeddings affects retrieval quality\. The query remains the dominant retrieval signal as performance generally increases withλ\\lambda, yet injecting a small proportion of profile embedding \(λ=0\.8\\lambda=0\.8\) achieves the best F1, surpassing query\-only retrieval \(λ=1\\lambda=1\)\.

Reward signal trade\-offα\\alpha\.Figure[3](https://arxiv.org/html/2607.00017#S4.F3)\(b\) shows how the balance between retrieval and answer rewards affects downstream performance\. The optimalα=0\.2\\alpha=0\.2indicates that the retrieval reward should dominate the training signal, as it directly guides the rewriter toward better evidence coverage\. The answer reward, while contributing a smaller proportion, still provides complementary guidance by ensuring the rewritten queries ultimately lead to better answer generation\.

![Refer to caption](https://arxiv.org/html/2607.00017v1/x5.png)Figure 5:Contour plot of Overall F1 \(%\) over episodic and semantic memory retrieval sizes\.Retrieval Analysis\.We further analyze retrieval quality from two perspectives\.

Profile\-Guided Retrieval Quality\.We measure Evidence Recall—the fraction of ground\-truth evidence turns covered by retrieved episodic memories at varying retrieval sizes \(formal definition inAppendix[A\.5](https://arxiv.org/html/2607.00017#A1.SS5)\)\. Figure[4](https://arxiv.org/html/2607.00017#S4.F4)compares profile\-guided retrieval against query\-only retrieval on episodic memory\. At small sizes, the two methods perform comparably, but as the retrieval size grows, profile\-guided retrieval increasingly outperforms the baseline\. This confirms that the profile embedding acts as a relevance prior that helps rank relevant memories higher when the candidate pool is large\.

Memory Layer Interaction\.Figure[5](https://arxiv.org/html/2607.00017#S4.F5)shows the joint effect of episodic and semantic memory retrieval sizes\. Both layers contribute positively with clear diminishing returns\. Performance changes more rapidly when increasing the semantic retrieval size in the low\-to\-moderate range, indicating that semantic memory provides greater marginal benefit per retrieved item due to its higher information density from cross\-dialogue aggregation\. Moderate retrieval sizes achieve near\-optimal performance without excessive context length\.

### 4\.8Case Study

To qualitatively illustrate how each component contributes to the final answer, Table[4](https://arxiv.org/html/2607.00017#S4.T4)presents a representative example where PPRO progressively resolves a temporally confounded query\. As components are added, each addresses a distinct failure mode, and only the full system produces the correct answer\. This demonstrates that profile\-guided retrieval and retrieval\-oriented rewriting are complementary—the profile resolves user\-level ambiguity, while the trained rewriter resolves temporal and factual specificity\.

## 5Conclusion

We presented PPRO, a retrieval\-centric framework that addresses user\-agnostic ranking and task\-agnostic optimization in memory\-augmented conversational agents\. PPRO constructs a hierarchical memory system and injects user profile embeddings as a personalized prior into retrieval scoring\. A query rewriter trained with GRPO using evidence retrieval quality and answer quality as reward signals closes the optimization loop between query formulation and downstream performance without modifying memory banks or the answer model\. Experiments on LoCoMo and LongMemEval\-S demonstrate consistent improvements over both training\-free and training\-based baselines, confirming personalized retrieval optimization as a key factor in effective long\-term memory use\.

## Limitations

Although PPRO achieves effective personalized retrieval through hierarchical memory and profile\-guided optimization, several limitations remain\.

Static Evaluation Setting\.PPRO is evaluated on static benchmarks where all dialogues are given in advance\. We do not evaluate in streaming scenarios where memories must be incrementally updated during ongoing interaction\.

Supervised Reward Signal\.GRPO training relies on ground\-truth evidence annotations for the retrieval reward\. Extending the optimization to unsupervised settings where such annotations are unavailable remains to be explored\.

Single\-User Scope\.The current framework models each user independently\. In multi\-party or community settings, shared context and inter\-user relationships could further inform retrieval but are not captured by our per\-user profile design\.

Future Work:We plan to extend PPRO to streaming incremental memory updates, explore self\-supervised reward signals to remove annotation dependency, and investigate cross\-user profile sharing for multi\-party conversations\.

## References

- N\. Bae, J\. Oh, Y\. Kim, J\. Park, and G\. Kim \(2025\)Towards lifelong dialogue agents via timeline\-based memory management\.InProceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/2025.naacl-long.435/)Cited by:[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p3.1)\.
- S\. Bird, E\. Klein, and E\. Loper \(2009\)Natural language processing with python\.O’Reilly Media\.Cited by:[§B\.2](https://arxiv.org/html/2607.00017#A2.SS2.p4.1)\.
- Y\. Chen, J\. Tan, A\. Zhang, Z\. Yang, L\. Sheng, E\. Zhang, X\. Wang, and T\. Chua \(2024\)On softmax direct preference optimization for recommendation\.External Links:2406\.09215,[Link](https://arxiv.org/abs/2406.09215)Cited by:[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- P\. Chhikara, P\. Khant, S\. Aryan, T\. Singh, and D\. Yadav \(2025\)Mem0: building production\-ready AI agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.External Links:[Link](https://arxiv.org/abs/2504.19413)Cited by:[§B\.4](https://arxiv.org/html/2607.00017#A2.SS4.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p2.1),[§4\.2](https://arxiv.org/html/2607.00017#S4.SS2.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-v3\.2: pushing the frontier of open large language models\.Cited by:[§B\.3](https://arxiv.org/html/2607.00017#A2.SS3.p2.2)\.
- X\. Fang, J\. Huang, Z\. Liu, Z\. Zhang, X\. Zhang, Y\. Zhang, N\. Chen, and H\. Hu \(2025\)LightMem: cutting token costs with efficient memory augmentation for LLM agents\.arXiv preprint arXiv:2505\.24845\.External Links:[Link](https://arxiv.org/abs/2505.24845)Cited by:[§B\.4](https://arxiv.org/html/2607.00017#A2.SS4.p1.1),[§4\.2](https://arxiv.org/html/2607.00017#S4.SS2.p1.1)\.
- B\. J\. Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. Su \(2025\)HippoRAG: neurobiologically inspired long\-term memory for large language models\.External Links:2405\.14831,[Link](https://arxiv.org/abs/2405.14831)Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p3.1)\.
- J\. Jia, Y\. Wang, Y\. Li, H\. Chen, X\. Bai, Z\. Liu, J\. Liang, Q\. Chen, H\. Li, P\. Jiang, and K\. Gai \(2024\)LEARN: knowledge adaptation from large language model to recommendation for practical industrial application\.External Links:2405\.03988,[Link](https://arxiv.org/abs/2405.03988)Cited by:[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- X\. Jiang, F\. Li, H\. Zhao, J\. Qiu, J\. Wang, J\. Shao, S\. Xu, S\. Zhang, W\. Chen, X\. Tang, Y\. Chen, M\. Wu, W\. Ma, M\. Wang, and T\. Chen \(2025\)Long term memory: the foundation of ai self\-evolution\.External Links:2410\.15665,[Link](https://arxiv.org/abs/2410.15665)Cited by:[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p1.1)\.
- Z\. Kang, X\. Si, Z\. Zhang, H\. Chen, P\. Li, Z\. Li, W\. Jiao, and Z\. Tu \(2025\)MemoryOS: a memory operating system for AI system\.arXiv preprint arXiv:2506\.06326\.External Links:[Link](https://arxiv.org/abs/2506.06326)Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p2.1)\.
- V\. Karpukhin, B\. Oguz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih \(2020\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,pp\. 6769–6781\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p2.1)\.
- S\. Kim and E\. Jang \(2025\)SynapticRAG: enhancing temporal memory retrieval in large language models through synaptic mechanisms\.InFindings of the Association for Computational Linguistics: ACL 2025,External Links:[Link](https://arxiv.org/abs/2410.13553)Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p3.1)\.
- X\. Kong, J\. Wu, A\. Zhang, L\. Sheng, H\. Lin, X\. Wang, and X\. He \(2025\)Customizing language models with instance\-wise lora for sequential recommendation\.External Links:2408\.10159,[Link](https://arxiv.org/abs/2408.10159)Cited by:[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- Z\. Leeet al\.\(2024\)Crafting personalized agents through retrieval\-augmented generation on editable memory graphs\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2024.emnlp-main.281/)Cited by:[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1)\.
- Z\. Liang, H\. Yang, J\. Li, and R\. Xu \(2025\)In prospect and retrospect: reflective memory management for long\-term personalized dialogue agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/2025.acl-long.413/)Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- M\. Liao, Y\. Wan, C\. Miao, X\. Shen, W\. Li, Y\. Li, D\. Xia, J\. Huang,et al\.\(2026\)Stay in character, stay safe: dual\-cycle adversarial self\-evolution for safety role\-playing agents\.arXiv preprint arXiv:2602\.13234\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1)\.
- J\. Liu, Y\. Su, P\. Xia, S\. Han, Z\. Zheng, C\. Xie, M\. Ding, and H\. Yao \(2026\)SimpleMem: efficient lifelong memory for LLM agents\.arXiv preprint arXiv:2601\.02553\.Note:ICLR 2026 WorkshopExternal Links:[Link](https://arxiv.org/abs/2601.02553)Cited by:[§B\.4](https://arxiv.org/html/2607.00017#A2.SS4.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p2.1),[§4\.2](https://arxiv.org/html/2607.00017#S4.SS2.p1.1),[§4\.3](https://arxiv.org/html/2607.00017#S4.SS3.p1.1),[§4\.4](https://arxiv.org/html/2607.00017#S4.SS4.p1.1),[Table 3](https://arxiv.org/html/2607.00017#S4.T3.fig1)\.
- Y\. Lu, Z\. Zhang, J\. Chen, T\. Wang, and S\. Li \(2025\)MemAgent: reshaping long\-context LLM with multi\-conv RL based memory agent\.arXiv preprint arXiv:2507\.02259\.External Links:[Link](https://arxiv.org/abs/2507.02259)Cited by:[§B\.4](https://arxiv.org/html/2607.00017#A2.SS4.p2.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p4.1),[§4\.2](https://arxiv.org/html/2607.00017#S4.SS2.p1.1)\.
- X\. Ma, Y\. Gong, P\. He, H\. Zhao, and N\. Duan \(2023\)Query rewriting in retrieval\-augmented large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 5303–5315\.Cited by:[§3\.4](https://arxiv.org/html/2607.00017#S3.SS4.p1.2)\.
- A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang \(2024\)Evaluating very long\-term conversational memory of LLM agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 13851–13870\.External Links:[Link](https://aclanthology.org/2024.acl-long.747/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.747)Cited by:[§B\.1](https://arxiv.org/html/2607.00017#A2.SS1.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p5.1),[§4\.1](https://arxiv.org/html/2607.00017#S4.SS1.p1.1)\.
- M\. Nguyen, H\. Kieu, and D\. D\. Le \(2026\)AMEM4Rec: leveraging cross\-user similarity for memory evolution in agentic LLM recommenders\.arXiv preprint arXiv:2602\.08837\.Cited by:[§3\.2](https://arxiv.org/html/2607.00017#S3.SS2.p4.1)\.
- C\. Packer, V\. Fang, S\. G\. Patil, K\. Lin, S\. Wooders, and J\. E\. Gonzalez \(2023\)MemGPT: towards LLMs as operating systems\.arXiv preprint arXiv:2310\.08560\.External Links:[Link](https://arxiv.org/abs/2310.08560)Cited by:[§B\.4](https://arxiv.org/html/2607.00017#A2.SS4.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p2.1),[§4\.2](https://arxiv.org/html/2607.00017#S4.SS2.p1.1)\.
- K\. Papineni, S\. Roukos, T\. Ward, and W\. Zhu \(2002\)BLEU: a method for automatic evaluation of machine translation\.InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics,pp\. 311–318\.Cited by:[§B\.2](https://arxiv.org/html/2607.00017#A2.SS2.p2.1),[§3\.4](https://arxiv.org/html/2607.00017#S3.SS4.p3.3),[§4\.3](https://arxiv.org/html/2607.00017#S4.SS3.p1.1)\.
- P\. Rajpurkar, J\. Zhang, K\. Lopyrev, and P\. Liang \(2016\)SQuAD: 100,000\+ questions for machine comprehension of text\.InProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing,pp\. 2383–2392\.Cited by:[§4\.3](https://arxiv.org/html/2607.00017#S4.SS3.p1.1)\.
- P\. Rasmussen, P\. Paliychuk, T\. Beauvais, J\. Ryan, and D\. Chalef \(2025\)Zep: a temporal knowledge graph architecture for agent memory\.External Links:2501\.13956,[Link](https://arxiv.org/abs/2501.13956)Cited by:[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p3.1)\.
- A\. Salemi, S\. Mysore, M\. Bendersky, and H\. Zamani \(2024\)LaMP: when large language models meet personalization\.arXiv preprint arXiv:2304\.11406\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p2.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p4.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.External Links:[Link](https://arxiv.org/abs/2402.03300)Cited by:[§A\.3](https://arxiv.org/html/2607.00017#A1.SS3.p1.4),[§1](https://arxiv.org/html/2607.00017#S1.p4.1),[§3\.4](https://arxiv.org/html/2607.00017#S3.SS4.p1.2)\.
- L\. Shen, H\. Zhan, X\. Shen, H\. Chen, X\. Zhao, and X\. Zhu \(2021a\)Identifying untrustworthy samples: data filtering for open\-domain dialogues with bayesian optimization\.InProceedings of the 30th ACM International Conference on Information & Knowledge Management,pp\. 1598–1608\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1)\.
- L\. Shen, H\. Zhan, X\. Shen, and Y\. Feng \(2021b\)Learning to select context in a hierarchical and global perspective for open\-domain dialogue generation\.InICASSP 2021\-2021 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 7438–7442\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1)\.
- L\. Shen, H\. Zhan, X\. Shen, Y\. Song, and X\. Zhao \(2021c\)Text is not enough: integrating visual impressions into open\-domain dialogue generation\.InProceedings of the 29th ACM International Conference on Multimedia,pp\. 4287–4296\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1)\.
- X\. Shen, Z\. Jiang, J\. Yang, H\. Liu, Y\. Wan, J\. Zhang, T\. Dai, L\. Xu, S\. Wu, G\. Qi,et al\.\(2026\)DuCCAE: a hybrid engine for immersive conversation via collaboration, augmentation, and evolution\.arXiv preprint arXiv:2603\.19248\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1)\.
- G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. Wu \(2024\)HybridFlow: a flexible and efficient rlhf framework\.arXiv preprint arXiv:2409\.19256\.Cited by:[§B\.2](https://arxiv.org/html/2607.00017#A2.SS2.p4.1)\.
- T\. R\. Sumers, S\. Yao, K\. Narasimhan, and T\. L\. Griffiths \(2024\)Cognitive architectures for language agents\.External Links:2309\.02427,[Link](https://arxiv.org/abs/2309.02427)Cited by:[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p1.1)\.
- J\. Tan, S\. Xu, W\. Hua, Y\. Ge, Z\. Li, and Y\. Zhang \(2024\)IDGenRec: llm\-recsys alignment with textual id learning\.External Links:2403\.19021,[Link](https://arxiv.org/abs/2403.19021)Cited by:[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- L\. Wang, N\. Yang, and F\. Wei \(2023\)Query2doc: query expansion with large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 9414–9423\.Cited by:[§3\.4](https://arxiv.org/html/2607.00017#S3.SS4.p1.2)\.
- Y\. Wanget al\.\(2025\)An efficient context\-dependent memory framework for LLM\-centric agents\.InProceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Industry Track,External Links:[Link](https://aclanthology.org/2025.naacl-industry.80/)Cited by:[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p3.1)\.
- Y\. Wang, H\. Zhang, L\. Pang, B\. Guo, H\. Zheng, and Z\. Zheng \(2024\)MaFeRw: query rewriting with multi\-aspect feedbacks for retrieval\-augmented large language models\.arXiv preprint arXiv:2408\.17072\.External Links:[Link](https://arxiv.org/abs/2408.17072)Cited by:[§3\.4](https://arxiv.org/html/2607.00017#S3.SS4.p1.2)\.
- D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu \(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=pZiyCaVuti)Cited by:[§B\.1](https://arxiv.org/html/2607.00017#A2.SS1.p2.1),[§1](https://arxiv.org/html/2607.00017#S1.p5.1),[§4\.1](https://arxiv.org/html/2607.00017#S4.SS1.p1.1)\.
- S\. Wu, Z\. Jiang, J\. Yang, X\. Shen, H\. Liu, Y\. Wan, C\. Miao, G\. Qi, T\. Dai, J\. Zhang,et al\.\(2026\)True\-to\-role, tailored\-to\-you: a survey of llm\-based role\-playing agents\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p1.1)\.
- S\. Xiao, Z\. Liu, P\. Zhang, N\. Muennighoff, D\. Lian, and J\. Nie \(2024\)C\-pack: packed resources for general chinese embeddings\.arXiv preprint arXiv:2309\.07597\.Cited by:[§B\.2](https://arxiv.org/html/2607.00017#A2.SS2.p4.1)\.
- W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang \(2025\)A\-Mem: agentic memory for LLM agents\.InAdvances in Neural Information Processing Systems,Cited by:[§B\.4](https://arxiv.org/html/2607.00017#A2.SS4.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p1.1),[§1](https://arxiv.org/html/2607.00017#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p2.1),[§4\.2](https://arxiv.org/html/2607.00017#S4.SS2.p1.1)\.
- S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan, H\. Schütze, V\. Tresp, and Y\. Ma \(2025\)Memory\-R1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.arXiv preprint arXiv:2508\.19828\.External Links:[Link](https://arxiv.org/abs/2508.19828)Cited by:[§B\.1](https://arxiv.org/html/2607.00017#A2.SS1.p1.1),[Table 5](https://arxiv.org/html/2607.00017#A6.T5),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p4.1),[§4\.1](https://arxiv.org/html/2607.00017#S4.SS1.p1.1)\.
- Y\. Yue, B\. Peng, X\. Fan, J\. Guo, Q\. Li, and Y\. Zhang \(2026\)Mem\-t: densifying rewards for long\-horizon memory agents\.External Links:2601\.23014,[Link](https://arxiv.org/abs/2601.23014)Cited by:[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p4.1)\.
- A\. Zhang, Y\. Chen, L\. Sheng, X\. Wang, and T\. Chua \(2024\)On generative agents in recommendation\.External Links:2310\.10108,[Link](https://arxiv.org/abs/2310.10108)Cited by:[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- S\. Zhang, E\. Dinan, J\. Urbanek, A\. Szlam, D\. Kiela, and J\. Weston \(2018\)Personalizing dialogue agents: I have a dog, do you have pets too?\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2204–2213\.Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p2.1)\.
- J\. Zhao, D\. Chen, Z\. Fan, K\. Xu, M\. Hu, B\. Tang, F\. Xiong, and Z\. Li \(2026\)Inside out: evolving user\-centric core memory trees for long\-term personalized dialogue systems\.arXiv preprint arXiv:2601\.05171\.Cited by:[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.
- H\. Zhou, Z\. Guo, J\. Zheng, Y\. Lu, J\. Chang, Z\. Wang, G\. Zhang, T\. Xie, Y\. Feng, X\. Wang, C\. Cheng, K\. Wu, J\. Chen, A\. Cherif, P\. Talukdar, W\. Xu, L\. Kong, and Z\. Yan \(2025\)MEM1: learning to synergize memory and reasoning for efficient long\-horizon agents\.arXiv preprint arXiv:2509\.16170\.External Links:[Link](https://arxiv.org/abs/2509.16170)Cited by:[§B\.4](https://arxiv.org/html/2607.00017#A2.SS4.p2.1),[§2\.1](https://arxiv.org/html/2607.00017#S2.SS1.p4.1),[§4\.2](https://arxiv.org/html/2607.00017#S4.SS2.p1.1)\.
- Y\. Zhu, K\. Wang, and Z\. Yang \(2025\)PRIME: language model personalization with cognitive memory and thought processes\.arXiv preprint arXiv:2507\.04607\.External Links:[Link](https://arxiv.org/abs/2507.04607)Cited by:[§1](https://arxiv.org/html/2607.00017#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.00017#S2.SS2.p1.1)\.

## Appendix

This appendix is organized as follows:

- •Method details \(Section[A](https://arxiv.org/html/2607.00017#A1)\)
- •Experiment setup \(Section[B](https://arxiv.org/html/2607.00017#A2)\)
- •Detailed ablation study \(Section[C](https://arxiv.org/html/2607.00017#A3)\)
- •Additional case studies \(Section[D](https://arxiv.org/html/2607.00017#A4)\)
- •Prompts \(Section[E](https://arxiv.org/html/2607.00017#A5)\)
- •LLM usage statement \(Section[F](https://arxiv.org/html/2607.00017#A6)\)

## Appendix AMethod Details

### A\.1Retrieval Reward

The retrieval rewardRr​e​t,iR\_\{ret,i\}measures how well the retrieved episodic memories cover the ground\-truth evidence\. Let𝒱q\\mathcal\{V\}\_\{q\}denote the annotated evidence turn set for queryqq, and let𝒱^i=⋃e∈ℛiEsrc​\(e\)\\hat\{\\mathcal\{V\}\}\_\{i\}=\\bigcup\_\{e\\in\\mathcal\{R\}^\{E\}\_\{i\}\}\\mathrm\{src\}\(e\)be the set of source turns from the retrieved episodic memoriesℛiE\\mathcal\{R\}^\{E\}\_\{i\}\. The retrieval reward is defined as:

Rr​e​t,i=F1​\(𝒱^i,𝒱q\)=2​Pi​ReciPi\+Reci,R\_\{ret,i\}=F\_\{1\}\(\\hat\{\\mathcal\{V\}\}\_\{i\},\\,\\mathcal\{V\}\_\{q\}\)=\\frac\{2P\_\{i\}\\mathrm\{Rec\}\_\{i\}\}\{P\_\{i\}\+\\mathrm\{Rec\}\_\{i\}\},\(10\)wherePi=\|𝒱^i∩𝒱q\|/\|𝒱^i\|P\_\{i\}=\|\\hat\{\\mathcal\{V\}\}\_\{i\}\\cap\\mathcal\{V\}\_\{q\}\|/\|\\hat\{\\mathcal\{V\}\}\_\{i\}\|andReci=\|𝒱^i∩𝒱q\|/\|𝒱q\|\\mathrm\{Rec\}\_\{i\}=\|\\hat\{\\mathcal\{V\}\}\_\{i\}\\cap\\mathcal\{V\}\_\{q\}\|/\|\\mathcal\{V\}\_\{q\}\|\.

### A\.2EMA\-based Reward Normalization

Because the retrieval rewardRr​e​tR\_\{ret\}and answer rewardRa​n​sR\_\{ans\}can have different scales and learning dynamics, we apply EMA\-based adaptive normalization before combining them:

R~∗,i=R∗,imax⁡\(μ∗,ϵ\),\\widetilde\{R\}\_\{\*,i\}=\\frac\{R\_\{\*,i\}\}\{\\max\(\\mu\_\{\*\},\\epsilon\)\},\(11\)whereμ∗\\mu\_\{\*\}denotes the exponential moving average of each respective reward stream \(∗∈\{ret,ans\}\*\\in\\\{ret,ans\\\}\)\. This normalization prevents one signal from dominating merely due to scale differences\.

### A\.3GRPO Objective

The GRPO objective optimizes the rewriter with a clipped policy\-gradient surrogate and a KL penalty against a frozen reference policyπr​e​f\\pi\_\{ref\}:

ℒG​R​P​O\(θ\)=−𝔼i,t\[min\(ri,tA^i,clip\(ri,t,1−ε,1\+ε\)A^i\)−μDK​L\],\\begin\{split\}\\mathcal\{L\}\_\{GRPO\}\(\\theta\)=\-\\mathbb\{E\}\_\{i,t\}\\big\[&\\min\\big\(r\_\{i,t\}\\hat\{A\}\_\{i\},\\;\\mathrm\{clip\}\(r\_\{i,t\},1\\\!\-\\\!\\varepsilon,\\\\ 1\\\!\+\\\!\\varepsilon\)\\hat\{A\}\_\{i\}\\big\)&\-\\mu D\_\{KL\}\\big\],\\end\{split\}\(12\)where𝔼i,t\\mathbb\{E\}\_\{i,t\}averages over allNNsampled rewrites and their token positions, and the importance ratio is

ri,t=πθ​\(qi,t′∣qi,<t′,q,C\)πθold​\(qi,t′∣qi,<t′,q,C\)\.r\_\{i,t\}=\\frac\{\\pi\_\{\\theta\}\(q^\{\\prime\}\_\{i,t\}\\mid q^\{\\prime\}\_\{i,<t\},q,C\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(q^\{\\prime\}\_\{i,t\}\\mid q^\{\\prime\}\_\{i,<t\},q,C\)\}\.\(13\)The KL divergence is estimated token\-wise following DeepSeekMathShaoet al\.\([2024](https://arxiv.org/html/2607.00017#bib.bib12)\)\. The answer model, embedding model, and memory banks remain frozen; gradients are applied only to the rewriter\.

### A\.4Retrieval Dimensions

Each episodic memory and each rewritten query is annotated with three retrieval dimensions:

- •Tense:present,past, orfuture— indicates the temporal orientation of the statement\.
- •Subjectivity:objectiveorsubjective— distinguishes factual statements from opinions or preferences\.
- •Certainty:certainorspeculative— differentiates confirmed facts from plans or hypotheticals\.

During episodic retrieval, each dimension of the rewritten queryδq\\delta\_\{q\}is compared against the corresponding dimension of each candidate memoryδe\\delta\_\{e\}\. A fixed bonus of0\.050\.05is added to the retrieval score for each matching dimension, yielding a maximum boost of0\.150\.15when all three dimensions align\.

### A\.5Evidence Recall

Evidence Recall measures how well the retrieved memories cover the ground\-truth evidence for a given query\. Each queryqqis annotated with a set of evidence turn identifiers𝒱q\\mathcal\{V\}\_\{q\}\. Each retrieved memoryeeis associated with its source turn\(s\)src​\(e\)\\mathrm\{src\}\(e\)via provenance tracking\. Given the retrieved episodic memoriesℛ\\mathcal\{R\}at a given retrieval size:

EvidenceRecall=\|𝒱q∩⋃e∈ℛsrc​\(e\)\|\|𝒱q\|\\mathrm\{EvidenceRecall\}=\\frac\{\|\\mathcal\{V\}\_\{q\}\\cap\\bigcup\_\{e\\in\\mathcal\{R\}\}\\mathrm\{src\}\(e\)\|\}\{\|\\mathcal\{V\}\_\{q\}\|\}\(14\)

### A\.6Algorithm

Algorithm 1PPRO: Inference and Training0:Query

qq, dialogue context

CC, user

uu, memory banks

ℳuE\\mathcal\{M\}^\{E\}\_\{u\},

ℳuS\\mathcal\{M\}^\{S\}\_\{u\}, profile

𝒫u\\mathcal\{P\}\_\{u\}
0:Answer

yy
1:// Phase 1: Online Inference

2:

\(q′,δq\)←fθ​\(q,C\)\(q^\{\\prime\},\\delta\_\{q\}\)\\leftarrow f\_\{\\theta\}\(q,C\)\{Query rewriting\}

3:

𝐩u←ℰ​\(𝒫u\)\\mathbf\{p\}\_\{u\}\\leftarrow\\mathcal\{E\}\(\\mathcal\{P\}\_\{u\}\)\{Profile embedding\}

4:

ℛE←Retrieve​\(ℳuE,q′,𝐩u,δq,λ\)\\mathcal\{R\}^\{E\}\\leftarrow\\mathrm\{Retrieve\}\(\\mathcal\{M\}^\{E\}\_\{u\},q^\{\\prime\},\\mathbf\{p\}\_\{u\},\\delta\_\{q\},\\lambda\)\{Episodic\}

5:

ℛS←Retrieve​\(ℳuS,q′\)\\mathcal\{R\}^\{S\}\\leftarrow\\mathrm\{Retrieve\}\(\\mathcal\{M\}^\{S\}\_\{u\},q^\{\\prime\}\)\{Semantic\}

6:

𝒞m←\[𝒫u;ℛE;ℛS\]\\mathcal\{C\}\_\{m\}\\leftarrow\[\\mathcal\{P\}\_\{u\};\\;\\mathcal\{R\}^\{E\};\\;\\mathcal\{R\}^\{S\}\]
7:

y←𝒜​\(q,𝒞m\)y\\leftarrow\\mathcal\{A\}\(q,\\mathcal\{C\}\_\{m\}\)\{Answer generation\}

8:

9:// Phase 2: GRPO Training

10:Sample

NNrewrites

\{qi′\}i=1N∼πθ\(⋅∣q,C\)\\\{q^\{\\prime\}\_\{i\}\\\}\_\{i=1\}^\{N\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q,C\)
11:for

i=1i=1to

NNdo

12:Execute Phase 1 with

qi′q^\{\\prime\}\_\{i\}to obtain

yiy\_\{i\}
13:Compute

Rr​e​t,iR\_\{ret,i\}and

Ra​n​s,iR\_\{ans,i\}
14:

Ri←α​R~a​n​s,i\+\(1−α\)​R~r​e​t,iR\_\{i\}\\leftarrow\\alpha\\widetilde\{R\}\_\{ans,i\}\+\(1\-\\alpha\)\\widetilde\{R\}\_\{ret,i\}
15:endfor

16:

A^i←\(Ri−mean​\(\{Rj\}\)\)/\(std​\(\{Rj\}\)\+ϵ\)\\hat\{A\}\_\{i\}\\leftarrow\(R\_\{i\}\-\\mathrm\{mean\}\(\\\{R\_\{j\}\\\}\)\)/\(\\mathrm\{std\}\(\\\{R\_\{j\}\\\}\)\+\\epsilon\)
17:Update

θ\\thetavia

ℒG​R​P​O​\(θ\)\\mathcal\{L\}\_\{GRPO\}\(\\theta\)

## Appendix BExperiment Setup

### B\.1Dataset Split Statistics

LoCoMo\.The LoCoMo dataset\(Maharanaet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib10)\)contains 10 multi\-session dyadic conversations, each spanning 19–35 sessions between two speakers\. Questions are annotated into five categories: SingleHop \(single\-session factual recall\), Temporal \(temporal reasoning requiring date/order understanding\), OpenDomain \(requiring external or commonsense knowledge\), MultiHop \(cross\-session evidence synthesis\), and Adversarial \(unanswerable questions\)\. Following the conversation\-level split protocol ofYanet al\.\([2025](https://arxiv.org/html/2607.00017#bib.bib11)\), we assign conversations to train/validation/test splits at the conversation level \(1:1:8 ratio\) to prevent information leakage across splits\. Category 5 \(Adversarial\) questions are excluded from all evaluations, as their gold answer is uniformly “unanswerable” and they do not test memory retrieval capability\. The resulting statistics are shown in Table[5](https://arxiv.org/html/2607.00017#A6.T5)\.

LongMemEval\-S\.LongMemEval\(Wuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib43)\)provides exceptionally long interaction histories \(up to 500\+ turns\) and tests fine\-grained answer localization\. We use its short\-form QA subset \(LongMemEval\-S\) which contains 5 question categories:Single\-Session\-User\(user\-stated facts within one session\),Single\-Session\-Preference\(user preferences expressed in one session\),Temporal\-Reasoning\(temporal ordering and date\-based questions\),Knowledge\-Update\(facts that evolve over time\), andMulti\-Session\(evidence scattered across multiple sessions\)\. We evaluate on this benchmark in a zero\-shot manner without any fine\-tuning on its data\.

### B\.2Evaluation Metrics

Token\-level F1\.Let𝒯p\\mathcal\{T\}\_\{p\}and𝒯r\\mathcal\{T\}\_\{r\}denote the multisets of whitespace\-tokenized tokens in the prediction and reference respectively\. Precision is\|𝒯p∩𝒯r\|/\|𝒯p\|\|\\mathcal\{T\}\_\{p\}\\cap\\mathcal\{T\}\_\{r\}\|/\|\\mathcal\{T\}\_\{p\}\|, recall is\|𝒯p∩𝒯r\|/\|𝒯r\|\|\\mathcal\{T\}\_\{p\}\\cap\\mathcal\{T\}\_\{r\}\|/\|\\mathcal\{T\}\_\{r\}\|, and F1 is their harmonic mean\.

BLEU\-1\.Unigram BLEU with a brevity penalty, followingPapineniet al\.\([2002](https://arxiv.org/html/2607.00017#bib.bib24)\)\. It complements Token\-level F1 by rewarding precision and penalizing irrelevant tokens in the prediction\.

LLM\-as\-Judge Accuracy\.For LongMemEval\-S,gpt\-4\.1\-minievaluates each predicted answer against the ground\-truth reference, producing a binaryCorrect/Wronglabel based on semantic equivalence and temporal consistency\. Accuracy is the percentage ofCorrectlabels\.

Implementation details\.Tokenization and BLEU\-1 computation use NLTK 3\.8\(Birdet al\.,[2009](https://arxiv.org/html/2607.00017#bib.bib46)\)withword\_tokenizeandsentence\_bleu\(smoothing method 1\)\. The BLEU\-1 reward is computed with NLTK’ssentence\_bleu\(smoothing method 1\)\. Sentence embeddings usebge\-large\-en\(Xiaoet al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib47)\)via the FlagEmbedding library\. GRPO training is implemented with the VERL framework\(Shenget al\.,[2024](https://arxiv.org/html/2607.00017#bib.bib48)\)\.

### B\.3Hyperparameter Settings

Retrieval\.We usebge\-large\-enas the embedding model\. Episodic retrieval top\-KK=25, semantic retrieval top\-KK=7, similarity threshold 0\.6, query\-profile mixing weightλ\\lambda=0\.8, and dimension boostbδb\_\{\\delta\}=0\.05\.

Offline memory construction\.All offline stages \(episodic extraction, semantic aggregation, and profile generation\) use DeepSeek\-V3\.2\(DeepSeek\-AI,[2025](https://arxiv.org/html/2607.00017#bib.bib44)\)as the shared instruction\-following model𝒢\\mathcal\{G\}\. The semantic memory clustering threshold isτc=0\.85\\tau\_\{c\}=0\.85\.

GRPO training\.We use the VERL framework with 2×\\timesNVIDIA A800 80GB GPUs\. Learning rate1×10−61\\times 10^\{\-6\}, batch size 64, 30 epochs, group sizeNN=8\. Clip ratioε\\varepsilon=0\.2, KL coefficientμ\\mu=0\.001, entropy coefficient 0\.001, weight decay 0\.01, gradient clipping 1\.0\. Answer reward weightα\\alpha=0\.2, retrieval reward weight\(1−α\)\(1\-\\alpha\)=0\.8\. Loss aggregation uses token\-mean mode\. We validate before training and select the best checkpoint based on validation F1\.

Computational resources\.The rewriter model shares the same backbone as the answer model in each evaluation setting\. The embedding model isbge\-large\-en\(335M parameters\)\. Offline memory construction usesqwen3\-8b\(8B parameters\)\. GRPO training is conducted on 2×\\timesNVIDIA A800 80GB GPUs; each training run takes approximately 15 hours \(∼\\sim30 GPU hours in total\)\.

### B\.4Baseline Descriptions

Training\-free baselines\.Full\-Contextfeeds the complete dialogue history to the answer model without explicit memory construction\.MemGPT\(Packeret al\.,[2023](https://arxiv.org/html/2607.00017#bib.bib21)\)manages memory through an operating\-system\-inspired virtual context mechanism\.A\-Mem\(Xuet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib16)\)stores memories as structured notes connected through dynamic indexing and memory evolution\.LightMem\(Fanget al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib17)\)separates lightweight online memory use from offline consolidation\.Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib14)\)dynamically extracts and consolidates salient information with graph\-based memory representations\.SimpleMem\(Liuet al\.,[2026](https://arxiv.org/html/2607.00017#bib.bib20)\)applies semantic structured compression, online synthesis, and intent\-aware retrieval planning\.

Training\-based baselines\.MEM1\(Zhouet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib19)\)learns to maintain a compact internal memory state via PPO\-trained iterative think\-search\-answer loops\.MemAgent\(Luet al\.,[2025](https://arxiv.org/html/2607.00017#bib.bib22)\)uses DAPO to train a recurrent memory compression agent that processes long contexts through sequential chunk\-level updates\. Both methods are trained on HotpotQA\.

## Appendix CDetailed Ablation Study

Table[6](https://arxiv.org/html/2607.00017#A6.T6)presents the per\-category breakdown of the ablation study\. The trends are consistent with the overall results in the main text\. Removing episodic memory causes the largest degradation, especially on MultiHop \(−12\.23\-12\.23F1\) where multiple fine\-grained facts must be combined\. Removing semantic memory impacts SingleHop and Temporal most, as aggregated timelines and counts directly serve these question types\. Profile injection contributes most to OpenDomain \(−4\.85\-4\.85F1\), where user\-specific relational context is needed for disambiguation\. Retrieval dimensions show the largest impact on Temporal \(−4\.88\-4\.88F1\), confirming that tense matching effectively guides temporal retrieval\. GRPO training benefits OpenDomain and Temporal most, as these categories require more specific query reformulation\.

## Appendix DAdditional Case Studies

Tables[7](https://arxiv.org/html/2607.00017#A6.T7)and[8](https://arxiv.org/html/2607.00017#A6.T8)present two additional case studies following the same progressive format as Table[4](https://arxiv.org/html/2607.00017#S4.T4)in the main text\. Each table shows how incrementally adding PPRO components resolves a distinct retrieval challenge\.

Case 2: Distinguishing Motivation from Outcome\(Table[7](https://arxiv.org/html/2607.00017#A6.T7)\)\. The question askswhyAudrey makes recycled jewelry, but the memory bank contains both the stated motivation \(“creativity and sustainability”\) and a downstream outcome \(“donate profits to an animal shelter”\)\. Without the full system, retrieval conflates these two aspects\. Episodic retrieval alone surfaces the more salient outcome memory \(donation\), while semantic memory partially captures sustainability but merges it with other environmental actions\. The GRPO\-trained rewriter learns to reformulate the query to specifically target stated motivations rather than behavioral consequences, surfacing the exact “creativity and sustainability” memory\.

Case 3: Stated Reason vs\. Inferred Motivation\(Table[8](https://arxiv.org/html/2607.00017#A6.T8)\)\. James explicitly stated he “wanted to learn something new,” but his memory bank also contains cooking\-related social events \(cooking for friends\) and self\-improvement themes\. The challenge is distinguishing what the useractually saidfrom what could beinferred\. Episodic retrieval without GRPO surfaces socially\-oriented cooking memories; semantic memory picks up the self\-improvement theme but drifts toward inference \(“challenge himself”\)\. The profile further reinforces self\-improvement but introduces paraphrase drift\. Only the GRPO\-trained rewriter generates a query specific enough to retrieve the verbatim stated reason, demonstrating that retrieval\-oriented optimization helps the system distinguish explicit statements from plausible inferences\.

## Appendix EPrompts

Figures[9](https://arxiv.org/html/2607.00017#A6.F9)–[9](https://arxiv.org/html/2607.00017#A6.F9)show the simplified prompts used in the offline memory construction stages and the online query rewriting stage\.

## Appendix FLLM Usage Statement

Large Language Models \(LLMs\) such as ChatGPT are used as general\-purpose tools to improve readability and clarity of the manuscript, e\.g\., for grammar checking, LaTeX formatting, and restructuring sentences\. No parts of the research idea, dataset design, or experimental results are generated or influenced by LLMs\. All technical contributions and conclusions are solely those of the authors\.

Table 5:LoCoMo data split statistics following the conversation\-level protocol ofYanet al\.\([2025](https://arxiv.org/html/2607.00017#bib.bib11)\)\. Category 5 \(adversarial\) questions are excluded\. “Avg\. Evidence” denotes the mean number of evidence turns per question\. The training and validation sets are used exclusively for GRPO optimization of the query rewriter\.Table 6:Full ablation study with per\-category breakdown on the LoCoMo test split with Qwen2\.5\-7B backbone\.Case Study: Distinguishing Motivation from OutcomeQuestionWhy does Audrey make jewelry out of recycled objects?Gold AnswerTo show love for creativity and sustainability\.EvidenceAudrey: “Oh yes\! I love making jewelry out of recycled stuff\. It’s a great way to show my love of creativity and sustainability\.”SettingKey Retrieved SignalModel PredictionFailure ModeF1No Memory–No information availableNo evidence0\.00\+ EpisodicDonation profits; animal shelterTo donate the profits to an animal shelterWrong aspect0\.12\+ SemanticWaste reduction; recycled materialsTo reduce waste and support a good causePartial motivation0\.24\+ User ProfileSustainability prior boostedTo make a difference by reducing wasteIncomplete dual framing0\.20\+ GRPO\(Full\)“creativity and sustainability” memory surfacedTo show her love of creativity and sustainabilityCorrect0\.80Table 7:Case study: distinguishing motivation from outcome\.Case Study: Stated Reason vs\. Inferred MotivationQuestionWhy did James sign up for a cooking class?Gold AnswerHe wanted to learn something new\.EvidenceJames: “I never liked cooking, but I felt that I wanted to learn something new\.”SettingKey Retrieved SignalModel PredictionFailure ModeF1No Memory–No information availableNo evidence0\.00\+ EpisodicCooking events; social goalTo cook for his friend JohnWrong aspect0\.18\+ Semantic“Never liked cooking” contrastBecause he wanted to challenge himselfInferred not stated0\.29\+ User Profile“Improve himself” boostedTo improve himself by trying something newParaphrase drift0\.50\+ GRPO\(Full\)“wanted to learn something new” surfacedHe felt he wanted to learn something newCorrect0\.92Table 8:Case study: stated reason vs\. inferred motivation\.![Refer to caption](https://arxiv.org/html/2607.00017v1/x6.png)Figure 6:Prompt for episodic memory extraction\.
![Refer to caption](https://arxiv.org/html/2607.00017v1/x7.png)Figure 7:Prompt for user profile generation\.
![Refer to caption](https://arxiv.org/html/2607.00017v1/x8.png)Figure 8:Prompt for semantic memory aggregation\.
![Refer to caption](https://arxiv.org/html/2607.00017v1/x9.png)Figure 9:Prompt for profile\-aware query rewriting\.