Know It, Act on It: Investigating Memory Utilization in LLM Personalization

# Investigating Memory Utilization in LLM Personalization
Source: [https://arxiv.org/html/2607.29433](https://arxiv.org/html/2607.29433)
Jianfei MaEmmanuele Chersoni The Hong Kong Polytechnic University \{zhaoxinbetty\.feng,jian\-fei\.ma\}@connect\.polyu\.hk, emmanuele\.chersoni@polyu\.edu\.hk

###### Abstract

As large language model \(LLM\) agents evolve into personalized companions, memory has emerged as a core capability\. However, LLMs face a knowledge utilization problem: they may fail to act on relevant user preferences even when they are fully present in context\. When an agent fails to tailor its response in a context where previously shared user preferences should matter, it is unclear whether the model failed to remember that information or remembered it but failed to use it\. To isolate this breakdown, we introduce a decoupled evaluation paradigm that administers pairedKnowandActtests to the same user preference\. We conduct large\-scale experiments across 16 systems and five memory architectures, evaluating 1,000 preferences embedded at three levels of expression strength\. Our results show a large gap betweenKnowandActoutcomes: agents often pass the recall test for a user preference but fail to reflect that same preference in the paired behavioral scenario\. While memory architectures reduce this gap, utilization remains especially weak for health and therapy\-related preferences, where failures to act carry the greatest real\-world stakes111Code and data are available at[https://anonymous\.4open\.science/r/KnowAct\_Benchmark\-5BEB/](https://anonymous.4open.science/r/KnowAct_Benchmark-5BEB/)\.

Know It, Act on It: Investigating Memory Utilization in LLM Personalization

Zhaoxin Feng and Jianfei Ma and Emmanuele ChersoniThe Hong Kong Polytechnic University\{zhaoxinbetty\.feng,jian\-fei\.ma\}@connect\.polyu\.hk,emmanuele\.chersoni@polyu\.edu\.hk

## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.29433v1/x1.png)Figure 1:A memory\-augmented LLM agent correctly stores a user’s peanut allergy from an email\-polishing request, yet fails to account for it when recommending a Thai dish in a subsequent session\.Large Language Model \(LLM\) agents are rapidly evolving from stateless tools into personalized long\-term companions capable of sustained interaction with users over days, weeks, and months\(OpenAI,[2025b](https://arxiv.org/html/2607.29433#bib.bib166); Meta,[2025](https://arxiv.org/html/2607.29433#bib.bib173)\)\. A core capability driving this transition is memory: the ability to store, organize, and retrieve information from past interactions\(Zhanget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib174)\)\. Both industry and academia are actively equipping agents with increasingly sophisticated memory architectures, from commercial systems such as ChatGPT’s persistent memory\(OpenAI,[2024c](https://arxiv.org/html/2607.29433#bib.bib143)\)and Claude’s memory feature\(Anthropic,[2025](https://arxiv.org/html/2607.29433#bib.bib144)\)to open\-source frameworks such as Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\), Letta/MemGPT\(Packeret al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib53)\), and knowledge\-graph\-based systems like Zep\(Rasmussenet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib19)\)and HippoRAG\(Gutiérrezet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib74)\)\. These systems have demonstrated strong performance on benchmarks that test factual recall, multi\-hop reasoning, and long\-range understanding\(Wuet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib127); Huet al\.,[2026a](https://arxiv.org/html/2607.29433#bib.bib14)\)\. But*remembering*information is only the first step\. LLMs face a well\-documented knowledge utilization problem, often failing to act on relevant information even when it is fully present in context\(Shiet al\.,[2023](https://arxiv.org/html/2607.29433#bib.bib219); Liuet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib91)\)\. Memory modules could in principle alleviate this by distilling conversation histories into condensed, high\-signal context, yet they also introduce new failure points such as semantic mismatch during retrieval\. Whether memory\-augmented LLM agents truly bridge the gap between remembering user preferences and acting on them remains an open question\.

Most users do not explicitly state their preferences to chatbots\. According to a recent large\-scale analysis of ChatGPT usage\(Chatterjiet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib141)\), the majority of users still treat LLMs as tools, asking them to refine emails, translate messages, or debug code, meaning their preferences are revealed only implicitly through everyday interactions\(Jianget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib17)\)\. As illustrated in Figure[1](https://arxiv.org/html/2607.29433#S1.F1), a user asking a chatbot to polish an email might reveal a peanut allergy through the content\. The memory module correctly extracts this preference\. Yet in a later session, when the same user asks the agent to recommend a Thai dish, the agent enthusiastically suggests pad thai with crushed peanuts on top\. Did the agent fail to*remember*the allergy, or did it remember but fail to*act*on it?

![Refer to caption](https://arxiv.org/html/2607.29433v1/x2.png)Figure 2:Top: the same preference is expressed at three levels of strength during memory construction\.Bottom: after incremental multi\-session injection, aKnowtest and anActtest are administered independently on the same preference\. Here the agent remembers the pollen allergy but fails to act on it when planning a spring outing\.When such failures occur, determining their cause requires testing two capabilities*separately on the same information*: whether the agent can recall it, and whether it can apply it in a relevant scenario\. Testing only recall leaves open whether the information actually influences behavior; testing only behavior leaves open whether a good response reflects genuine memory or coincidence\. Yet existing research, whether on LLM personalization or agentic memory, evaluates either recall or behavioral responses, but not both capabilities on the same stored information \(§[2\.1](https://arxiv.org/html/2607.29433#S2.SS1)\), and most studies operate in static long\-context settings rather than exercising the incremental pipelines through which long\-term agents’ memory systems operate in practice\. To bridge this gap, we design a paired experimental framework that applies aKnowtest and anActtest to*the same*user preference\. TheKnowtest directly asks whether the agent can recall a specific preference; theActtest presents a natural scenario where that preference should influence behavior without mentioning it\. We further introduce a three\-level expression gradient \(explicit, incidental, and inferential\) extending the foundational explicit/implicit distinction\(Rich,[1979](https://arxiv.org/html/2607.29433#bib.bib36)\)to investigate how the strength of preference expression affects downstream utilization\. Following an incremental memory construction protocol\(Huet al\.,[2026a](https://arxiv.org/html/2607.29433#bib.bib14)\)that ensures each system processes information through its native pipeline, and building on PersonaMem\-v2’s persona and preference data\(Jianget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib17)\), we conduct large\-scale experiments across 50 personas, 1,000 preferences, and three expression conditions, evaluating 16 memory systems spanning five architectural categories on 3,000 instances \(see Figure[2](https://arxiv.org/html/2607.29433#S1.F2)\)\.

Our empirical investigation reveals a pervasive gap betweenKnowandActacross modern LLM agents: regardless of whether a system relies on a pure long\-context model or a dedicated memory architecture, the best performers convert no more than two\-thirds of successfully remembered preferences into appropriate behavioral responses\. Equipping LLM agents with memory systems substantially narrows this gap; Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\), for instance, raises GPT\-4o\-mini’s utilization rate from 16\.3% to 54\.6%, with gains disproportionately large on the behavioral side, suggesting that effective memory architectures contribute not merely by retrieving more, but by surfacing information in a form that the model can act upon\. Across preference types, utilization diverges sharply even when recall is comparable: health and therapy related preferences consistently rank lowest, precisely the categories where failure to act carries the highest real\-world stakes\.

## 2Preliminaries

### 2\.1LLM Personalization

Research on LLM personalization has developed along two largely disconnected tracks\. One track deepens user understanding through person\-level inference and multi\-source reasoning\(Wuet al\.,[2026](https://arxiv.org/html/2607.29433#bib.bib20); Chenget al\.,[2026](https://arxiv.org/html/2607.29433#bib.bib11)\), yet evaluation remains question answering rather than behavioral\. The other track measures behavior directly, testing whether agents generate tailored responses to explicit, implicit, or multi\-session preferences\(Salemiet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib120); Zhaoet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib130); Liet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib113); Zolloet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib131); Kumaret al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib111)\), maintain persona consistency\(Boson AI,[2024](https://arxiv.org/html/2607.29433#bib.bib97); Wanget al\.,[2024b](https://arxiv.org/html/2607.29433#bib.bib125)\), or apply preferences already provided in the prompt\(Yoonet al\.,[2026](https://arxiv.org/html/2607.29433#bib.bib21)\)\. Each tests a different slice of the pipeline, yet none separately evaluates whether a preference is successfully remembered and whether it is also acted upon\. PersonaMem\-v1\(Jianget al\.,[2025a](https://arxiv.org/html/2607.29433#bib.bib16)\)is the only work that includes both knowledge and behavioral tests, but applies them to*different*preferences, precluding paired diagnosis\. PersonaMem\-v2\(Jianget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib17)\)adds implicit preferences and incremental construction but drops the knowledge test entirely\.

### 2\.2Memory Systems for LLM Agents

As LLM\-based agents transition from single\-turn tools to persistent systems\(Xiet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib145); Wanget al\.,[2024a](https://arxiv.org/html/2607.29433#bib.bib146)\), the ability to retain and utilize information across interactions has become a central design challenge\(Huet al\.,[2026b](https://arxiv.org/html/2607.29433#bib.bib13); Huanget al\.,[2026](https://arxiv.org/html/2607.29433#bib.bib15)\)\. A growing body of research evaluates memory capabilities along increasingly sophisticated dimensions, from retrieval and comprehension in extended contexts\(Baiet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib85); Zhanget al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib213); Maharanaet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib114)\), to multi\-hop reasoning and temporal understanding\(Baiet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib86); Wuet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib127)\), to memory reliability\(Chenet al\.,[2026](https://arxiv.org/html/2607.29433#bib.bib138)\), and recently to four integrated competencies evaluated through incremental input protocols\(Huet al\.,[2026a](https://arxiv.org/html/2607.29433#bib.bib14)\)\. Despite this expansion, the entire line of work evaluates whether information can be*recovered*, not whether it*shapes downstream behavior*\(Huet al\.,[2026b](https://arxiv.org/html/2607.29433#bib.bib13); Huanget al\.,[2026](https://arxiv.org/html/2607.29433#bib.bib15)\)\.

Existing memory architectures span five broad categories evaluated in our experiments\.Long\-context baselinessuch as GPT\-4o\-mini\(OpenAI,[2024a](https://arxiv.org/html/2607.29433#bib.bib215)\), GPT\-4o\(OpenAI,[2024b](https://arxiv.org/html/2607.29433#bib.bib214)\), Gemini 3\.1 Flash\(Google DeepMind,[2026](https://arxiv.org/html/2607.29433#bib.bib193)\), and Claude 4\.6 Sonnet\(Anthropic,[2026](https://arxiv.org/html/2607.29433#bib.bib194)\)place the entire conversation history in the context window, eliminating retrieval errors but introducing attention\-allocation bottlenecks\(Liuet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib91)\)\.RAG\-based methodsretrieve relevant passages at query time via sparse lexical matching \(BM25\(Robertson and Walker,[1994](https://arxiv.org/html/2607.29433#bib.bib170)\)\) or dense vector retrieval \(text\-embedding\-3\-small/\-large\(OpenAI,[2024d](https://arxiv.org/html/2607.29433#bib.bib216)\), Qwen3\-Embedding\-4B\(Zhanget al\.,[2025a](https://arxiv.org/html/2607.29433#bib.bib218)\)\), but are limited to top\-kk k selection\.Structure\-augmented RAGorganizes information into knowledge graphs or structured stores: Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\)extracts atomic facts, Zep/Graphiti\(Rasmussenet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib19)\)builds temporal knowledge graphs, Cognee\(Markovicet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib18)\)refines its graph via feedback loops, and HippoRAG\-v2\(Gutiérrezet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib74)\)uses hippocampus\-inspired pattern completion\.Agentic memorysystems give the agent control over its own memory operations: Letta/MemGPT\(Packeret al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib53)\)manages hierarchical memory tiers, Self\-RAG\(Asaiet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib42)\)learns when to retrieve, MemoryOS\(Kanget al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib46)\)applies OS\-inspired memory management, and A\-MEM\(Xuet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib61)\)uses Zettelkasten\-based note linking\.

## 3Experimental Design

This section describes our experimental design, illustrated in Figure[2](https://arxiv.org/html/2607.29433#S1.F2): the data construction pipeline \(§[3\.1](https://arxiv.org/html/2607.29433#S3.SS1)\) and the paired evaluation framework \(§[3\.2](https://arxiv.org/html/2607.29433#S3.SS2)\)\.

### 3\.1Data Construction

Personas and Preferences\.From PersonaMem\-v2’s personas\(Jianget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib17)\), we select 50 via stratified sampling across 11 cultural regions, filtering for non\-null nationality and sufficient preference count \(\>\>70\)\. The full filtering pipeline is described in Appendix[C\.1](https://arxiv.org/html/2607.29433#A3.SS1)\. For each persona, we sample 20 preferences as*target*preferences, which are the preferences that will be evaluated through pairedKnowandActtests\. The remaining preferences \(approximately 50–70 per persona\) serve as*non\-target*context: their conversation entries are injected alongside target preferences to provide realistic retrieval noise, reflecting the fact that real user–chatbot histories contain a mix of topics and not every conversation is relevant to a given query\. Target preferences are sampled according to a fixed quota across five preference types \(stereotypical, anti\-stereotypical, neutral, health/medical, and therapy/emotional\), with a fallback mechanism \(Appendix[C\.2](https://arxiv.org/html/2607.29433#A3.SS2)\)\.

Expression Strength Gradient\.In everyday user–chatbot interactions, preferences are conveyed at varying levels of directness, from explicit statements to subtle behavioral cues\(Rich,[1979](https://arxiv.org/html/2607.29433#bib.bib36); Jianget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib17)\)\. We introduce a three\-level gradient that captures this spectrum\. For each preference, we construct three versions of the conversation snippet in which the preference is embedded, while keeping theKnowandActtest questions identical across all three versions\. GPT\-5\(OpenAI,[2025a](https://arxiv.org/html/2607.29433#bib.bib192)\)generates the explicit and inferential snippets \(Appendix[B\.5](https://arxiv.org/html/2607.29433#A2.SS5)\); the incidental version uses PersonaMem\-v2’s original conversation entry directly, which was also generated by GPT\-5\(OpenAI,[2025a](https://arxiv.org/html/2607.29433#bib.bib192)\)\. All three variants share identical non\-target chunks, ensuring that performance differences reflect only expression strength\.

- •Explicit: The user directly states the preference as the central topic of conversation \(e\.g\., “Just to let you know, every spring I get hay fever from pollen\. It’s mild, but I sneeze a lot and my eyes get itchy”\)\.
- •Incidental: The preference appears as incidental information within a task\-oriented conversation but is never directly stated; extracting it requires reading comprehension across the surrounding context \(e\.g\., the user asks to polish an email that mentions sneezing often and meeting at a tea house instead of the park\)\.
- •Inferential: The preference must be inferred from the user’s behavioral patterns; no single utterance explicitly labels it\. Although never directly stated, the conversation revolves around closely related topics, allowing LLM agents to deduce the preference from contextual cues \(e\.g\., the user mentions visiting outdoor temples during a spring festival, describes sneezing and watery eyes on windy days with blossoms, and asks for tablets that will not make them sleepy, without ever mentioning ‘allergy’\)\.

Representative examples of all three expression levels and their correspondingKnow/Acttests are provided in Appendix[C\.7](https://arxiv.org/html/2607.29433#A3.SS7)\.

Chunk Sequences and Test Pairs\.For each persona, we construct three parallel chunk sequences, one per expression level, resulting in 150 sequences \(50 personas×\\times3 expression levels\) comprising 11,955 total chunks and approximately 4\.55 million tokens \(target\-to\-non\-target ratio≈\\approx1:3\)\. The three sequences for a given persona differ only in their 20 target preference snippets; all non\-target chunks are shared\. GPT\-5\(OpenAI,[2025a](https://arxiv.org/html/2607.29433#bib.bib192)\)generates a pairedKnowtest \(question and ground truth answer\) andActtest \(scenario\) for each of the 1,000 preferences \(Appendix[B\.6](https://arxiv.org/html/2607.29433#A2.SS6)\)\. The same test questions are used across all three expression levels, ensuring that performance differences reflect signal strength, not test difficulty\. Quality validation details are provided in Appendix[D](https://arxiv.org/html/2607.29433#A4)\. Complete dataset statistics are in Appendix[C](https://arxiv.org/html/2607.29433#A3)\(Table[4](https://arxiv.org/html/2607.29433#A3.T4)\)\.

### 3\.2Paired Know–Act Evaluation

The central design principle of our framework is that every user preference is evaluated through two complementary tests that target the same underlying information but probe different capabilities\.

Know Test\.TheKnowtest establishes whether the agent has successfully stored and can retrieve a specific user preference\. Questions are phrased in first person \(e\.g\., “Do I have any seasonal allergies?”\) to simulate a user asking about information shared in prior interactions\. An LLM judge \(GPT\-5\(OpenAI,[2025a](https://arxiv.org/html/2607.29433#bib.bib192)\)\) compares the agent’s response against a ground truth answer and assigns a binary Pass/Fail verdict \(Appendix[B\.3](https://arxiv.org/html/2607.29433#A2.SS3)\)\. Know questions are deliberately simple and direct: failure indicates unambiguous memory failure\. To validate that these questions cannot be answered by prior knowledge or guessing alone, we evaluate all generation models used in our experiments on the full 1,000 Know questions without any conversation history\. All four models score below 3%, confirming that non\-trivialKnowaccuracy in memory\-augmented systems genuinely reflects successful memorization \(Appendix[D\.4](https://arxiv.org/html/2607.29433#A4.SS4)\)\.

Table 1:Know Accuracy,Act Accuracy, andUtilization Rateacross memory systems\. Exp = Explicit, Inc = Incidental, Inf = Inferential\. Avg = macro\-average over the three conditions\. Long\-context baselines use their native models; all other systems use GPT\-4o\-mini as the generation backbone\.Bold= best;underline= second;italic= third\.Know Acc\.\(%\)Act Acc\.\(%\)Util\.Rate\(%\)TypeSystemExpIncInfAvgExpIncInfAvgExpIncInfAvgGPT\-4o\-mini83\.79\.111\.334\.719\.814\.417\.617\.320\.012\.116\.816\.3GPT\-4o99\.037\.260\.765\.637\.722\.727\.429\.337\.928\.530\.032\.1Gemini 3\.1 Flash99\.556\.180\.978\.871\.937\.650\.853\.471\.947\.152\.257\.0Long\-ContextClaude 4\.6 Sonnet99\.349\.080\.476\.282\.242\.759\.561\.582\.250\.862\.365\.1Simple RAGBM2586\.716\.813\.739\.119\.312\.018\.316\.520\.011\.320\.417\.2RAG te3\-small98\.018\.438\.151\.545\.213\.225\.027\.845\.618\.531\.031\.7RAG te3\-large97\.620\.039\.752\.444\.513\.225\.327\.745\.019\.031\.531\.8Embedding RAGRAG Qwen3\-Emb\-4B98\.720\.650\.256\.553\.113\.125\.830\.753\.615\.531\.533\.5Mem097\.730\.952\.360\.371\.227\.039\.946\.072\.045\.646\.354\.6Zep/Graphiti52\.92\.32\.519\.211\.010\.811\.711\.212\.321\.716\.016\.7Cognee90\.315\.623\.843\.232\.115\.124\.223\.832\.719\.229\.027\.0Structure\-Aug\. RAGHippoRAG\-v298\.016\.226\.646\.935\.115\.024\.024\.735\.321\.027\.828\.0Letta/MemGPT89\.514\.322\.142\.032\.514\.522\.223\.132\.616\.129\.426\.0Self\-RAG94\.225\.053\.857\.761\.49\.227\.232\.662\.110\.033\.835\.3MemoryOS93\.511\.732\.345\.844\.913\.225\.327\.846\.316\.237\.233\.2Agentic MemoryA\-MEM89\.814\.820\.141\.625\.816\.421\.221\.126\.219\.624\.423\.4Act Test\.TheActtest evaluates whether stored knowledge*influences behavior*\. The agent is presented with a natural scenario where the preference should shape its response\. The scenario itself does not reference the preference; the agent is expected to consider it based on what it has stored in memory\. For instance, if the user has a pollen allergy, the Act test might ask the agent to “suggest a location and packing list for a spring picnic,” expecting the agent to consider allergen exposure in its recommendation\. We adopt paired comparison scoring: the agent’s response with memory is compared against a response from the same backbone model without any memory input\. An LLM judge \(GPT\-5\) determines whether the memory\-augmented response better reflects consideration of the preference, producing a binary Pass/Fail outcome \(Appendix[B\.4](https://arxiv.org/html/2607.29433#A2.SS4)\)\.

Evaluation Protocol\.Following the inject\-then\-query paradigm ofMemoryAgentBench\(Huet al\.,[2026a](https://arxiv.org/html/2607.29433#bib.bib14)\), we feed conversation chunks to memory systems one at a time using a standardized memorization template \(Appendix[B\.1](https://arxiv.org/html/2607.29433#A2.SS1)\)\. This ensures that each system processes information through its native mechanisms: Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\)performs fact extraction, Letta\(Packeret al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib53)\)manages hierarchical memory tiers, and knowledge\-graph\-based systems construct their graphs progressively\. After all chunks are injected, theKnowandActtests are administered in separate, independent sessions to avoid priming effects\. Long\-context baselines receive all chunks concatenated into a single context window preceding the test query\. All long\-context baselines use temperature=0 for deterministic generation\. All RAG and agentic memory systems use GPT\-4o\-mini as the shared generation backbone, ensuring that performance differences reflect memory architecture rather than generation capability\. Systems requiring embedding\-based retrieval use text\-embedding\-3\-small\(OpenAI,[2024d](https://arxiv.org/html/2607.29433#bib.bib216)\)as the default embedding model, with the exception of MemoryOS\(Kanget al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib46)\), whose implementation hardcodes all\-MiniLM\-L6\-v2\(Wanget al\.,[2020](https://arxiv.org/html/2607.29433#bib.bib217)\)\.

## 4Experiments

### 4\.1Setup

We evaluate 16 memory systems spanning all five categories of the taxonomy introduced in §[2\.2](https://arxiv.org/html/2607.29433#S2.SS2), covering long\-context baselines, simple RAG, embedding\-based RAG, structure\-augmented RAG, and agentic memory\.

![Refer to caption](https://arxiv.org/html/2607.29433v1/x3.png)Figure 3:Know Accuracy,Act Accuracy, andUtilization Rateacross expression levels for each category\. Exp = Explicit, Inc = Incidental, Inf = Inferential\. All architectures show a sharpKnowdrop from Explicit to Incidental, with partial recovery under Inferential\. Act andUtilizationremain comparatively flat\.Metrics\.For each preference, theKnowtest andActtest each produce a binary Pass/Fail outcome, yielding four possible states per preference: both pass \(kp ap\),Knowpass butActfail \(kp af\),Knowfail butActpass \(kf ap\), and both fail \(kf af\)\. We reportKnow AccuracyandAct Accuracyas the pass rates of each test, respectively\. We additionally report theUtilization Rate, defined as the fraction of correctly remembered preferences that the agent also acts on:

Utilization Rate=Nkp,apNkp,ap\+Nkp,af\\text\{\{Utilization Rate\}\}=\\frac\{N\_\{\\textsc\{kp,ap\}\}\}\{N\_\{\\textsc\{kp,ap\}\}\+N\_\{\\textsc\{kp,af\}\}\}A system with highKnow Accuracybut lowUtilization Ratesuccessfully retrieves user information yet fails to leverage it in downstream behavior, pointing to a utilization bottleneck rather than a memory failure\.

Judge Reliability\.Human evaluation of 100Knowand 100Actjudgments yields Fleiss’κ\\kappaof 0\.89 and 0\.82, respectively\. Inter\-judge agreement between GPT\-5 and Claude Opus 4\.6\(Anthropic,[2026](https://arxiv.org/html/2607.29433#bib.bib194)\)on a separate 200\-sample subset yields Cohen’sκ\\kappaof 0\.91 \(Know\) and 0\.88 \(Act\), indicating strong agreement\. Details are in Appendix[E](https://arxiv.org/html/2607.29433#A5)\.

### 4\.2Overall Results

Table[1](https://arxiv.org/html/2607.29433#S3.T1)presentsKnow Accuracy,Act Accuracy, andUtilization Ratefor all 16 systems\. We report the following findings\.

Remembering does not imply acting: Know–Act dissociation is pervasive across all architectures\. The central finding of KnowAct is that high recall does not guarantee behavioral utilization\. Under the Explicit condition, most systems achieve highKnow Accuracy, yetUtilization Ratesvary widely across systems\. Even the best\-performing system, Claude 4\.6 Sonnet, converts only about two\-thirds of its remembered preferences into behavioral responses\. The dissociation is most striking for GPT\-4o\-mini, which can articulate preferences when asked directly but rarely incorporates them into its responses\. We note that KnowAct’s retrieval setting is relatively favorable, as each preference corresponds to a single target passage with limited competing evidence; the Know–Act dissociation would likely be even more pronounced in realistic scenarios with greater retrieval noise\.

Long\-context models with 1M\-token windows lead overall, but context window size is not the sole factor\. Claude 4\.6 Sonnet and Gemini 3\.1 Flash \(both 1M windows\) substantially outperform GPT\-4o and GPT\-4o\-mini \(both 128K\) across all metrics\. However, model capability plays an equally important role: GPT\-4o roughly doubles GPT\-4o\-mini’s scores despite sharing the same 128K window, and among two 1M models, Gemini leads onKnowwhile Claude leads onUtilization\.

Memory architectures improve upon the GPT\-4o\-mini backbone, with Mem0 yielding the largest gain\. Since all RAG and agentic memory systems share GPT\-4o\-mini as backbone, comparing them against GPT\-4o\-mini in the long\-context setting isolates the benefit from memory architectures\. Most systems improve substantially, such as Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\)roughly triples theUtilization Rate, achieving results comparable to Gemini 3\.1 Flash\.

### 4\.3Analysis by Expression Level

Figure[3](https://arxiv.org/html/2607.29433#S4.F3)shows howKnow Accuracy,Act Accuracy, andUtilization Ratevary across the three expression levels for each architecture category\.

The Incidental condition is harder to recall than Inferential across all categories\. In every architecture category,Know Accuracyreaches its lowest under the Incidental condition, not Inferential\. This is counterintuitive: Inferential requires reasoning from behavioral cues, which should be more demanding\. However, Incidental preferences are embedded as side details within task\-oriented conversations and never become the focus, making them difficult for both memory systems and long\-context models to flag as noteworthy\. Under the Inferential condition, by contrast, conversations revolve around the relevant topic, providing richer semantic cues for retrieval\.

![Refer to caption](https://arxiv.org/html/2607.29433v1/x4.png)Figure 4:Know Accuracy\(top\) andUtilization Rate\(bottom\), averaged by architecture category, split by preference type\. Various memory systems backbone with GPT\-4o\-mini; long\-context GPT\-4o\-mini results in Table[1](https://arxiv.org/html/2607.29433#S3.T1)\. Therapy & Emotional preferences are easy to recall but hard to utilize; Health & Medical preferences score lowest on both metrics\. Per\-system details in Appendix[F](https://arxiv.org/html/2607.29433#A6)\.Explicit expression exposes utilization failure; non\-explicit expression exposes storage failure\. Under the Explicit condition,Know Accuracyis high across all architectures, yetAct AccuracyandUtilization Rateremain much lower, producing the largest Know\-Act gap\. This indicates that the primary bottleneck under Explicit expression is not memory but utilization: systems can recall the preference when asked directly but fail to incorporate it into behavioral responses\.

### 4\.4Analysis by Preference Type

Figure[4](https://arxiv.org/html/2607.29433#S4.F4)breaks down theUtilization Rateby preference type, with average values shown\.

Health and therapy related preferences suffer the largest utilization gaps\.Know Accuracyis relatively uniform across preference types, indicating that storage difficulty alone does not explain performance differences\. However,Utilization Ratediverges sharply: Health & Medical drops to the lowest average and Therapy & Emotional is second\-lowest, despite Therapy achieving the highestKnow Accuracy\. This contrast indicates that the bottleneck lies in utilization rather than storage\. These categories require more nuanced reasoning to translate into appropriate behavior: a pollen allergy should inform outdoor activity suggestions, and work\-related stress should shape how the agent frames productivity advice, connections that are harder to make than simply recommending a cuisine matching a stated food preference\. This has notable safety implications: unlike lifestyle preferences where commonsense reasoning often suffices, health and emotional preferences demand specialized domain knowledge to act upon correctly, and agents that retrieve yet fail to act on such information present a tangible risk in deployments where user wellbeing is at stake\.

The GPT\-4o\-mini backbone relies more on population priors than memory\-augmented systems\. In theUtilization Ratepanel, GPT\-4o\-mini \(backbone\) performs better on Stereotypical than Anti\-Stereotypical preferences, consistent with PersonaMem\-v1\(Jianget al\.,[2025a](https://arxiv.org/html/2607.29433#bib.bib16)\)and PersonaMem\-v2\(Jianget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib17)\)findings that models tend toward demographic expectations\. This pattern reverses for memory\-augmented systems, suggesting that retrieved memory overrides model priors, whereas the backbone falls back on prior\-consistent behavior\.

Memory architectures benefit utilization more than storage\. Comparing memory\-augmented systems against the GPT\-4o\-mini backbone, the relative gains inUtilization Rateare consistently larger than those inKnow Accuracy\. WhileKnow Accuracyimproves modestly,Utilization Rateroughly doubles across most preference types\. This suggests that the primary value of memory architectures lies not only in improving what is retrieved, but in presenting information in a more actionable form that enables the backbone model to better incorporate preferences into its responses\.

### 4\.5Failure Attribution

The analyses above revealhow muchKnow–Act dissociation occurs but do not identifywherein the memory system pipeline the breakdown happens\. An agent that knows a user’s pollen allergy but recommends a spring picnic in an open field could be failing at any of several stages: the retrieval system may not have surfaced the allergy information for the activity recommendation query; the information may have been retrieved but the agent could not extract the relevant constraint from the surrounding conversational context; or the constraint may have been fully available but the agent still failed to incorporate it into its response\.

To localize failures, we conduct a controlled oracle study on Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\), the strongest non\-long\-context system, analyzing all 722KnowPass×\\timesActFail cases across the three expression conditions\.

Method\.We design two oracle experiments that progressively bypass stages of the agentic memory pipeline\. Both conditions use the same backbone model \(GPT\-4o\-mini\) andActjudge as evaluation:

- •Oracle chunk: We bypass retrieval by injecting the ground\-truth conversation chunk \(the chunk containing the target preference\) directly into the Act test context\. The agent must still identify the relevant preference within it and apply it to the current task\.
- •Oracle preference: We bypass both retrieval and comprehension by injecting the preference text itself \(e\.g\., “This user has a pollen allergy”\) directly into the Act test context\. LLM agent’s only remaining task is to apply the stated preference to the scenario\.

Comparing outcomes across the three conditions yields a three\-layer failure decomposition\. For each case, exactly one attribution applies:

- •Retrieval failure: Oracle chunk passes, indicating that the original retrieval did not surface the relevant dialogue related to the target preference\.
- •Comprehension failure: Oracle chunk fails, but oracle preference passes, indicating that the relevant chunk was present but the agent could not extract the preference from it\.
- •Application failure: Oracle preference also fails, indicating that the preference was explicitly stated but the agent still did not act on it\.

Across all conditions \(see Table[2](https://arxiv.org/html/2607.29433#S4.T2)\), comprehension deficiency is the predominant cause of overall failures\. More than half of the total errors occur because the model cannot accurately distill target features, even when the dialogue containing the target preference is already provided in the context\. Furthermore, failures in the final application stage expose an intrinsic ceiling in translating known information into action\. Regardless of how a preference is initially expressed, the proportion of application failures remains remarkably stable\. In these cases, the model fails to follow the constraints despite being given explicit preference statements\. This persistent inability to utilize provided knowledge reflects inherent limitations in the underlying language model regarding knowledge utilization\.

Table 2:Three\-layer failure attribution for Mem0’s Know Pass×\\timesAct Fail cases \(n=722n\{=\}722\)\. Each case is classified into Retrieval \(R\), Comprehension \(C\), or Application \(A\) failure\.Boldmarks the dominant failure type per condition\.ExpressionnnR \(%\)C \(%\)A \(%\)Explicit27348\.734\.816\.5Incidental16811\.369\.619\.0Inferential28124\.960\.914\.2All72230\.753\.016\.2Varying expression strengths reveal distinct failure patterns\. When preferences are explicitly stated, a counterintuitive retrieval bottleneck occurs, causing nearly half of the errors in this condition\. Specifically, for an incoming query, the system fails to retrieve the exact conversation chunk containing the target preference that it is supposed to find\. Conversely, for incidental or inferential preferences, the bottleneck shifts heavily to comprehension\. In these non explicit scenarios, comprehension breakdowns dominate\. The system struggles to extract hidden preferences from peripheral details or behavioral cues, even when directly provided with the relevant dialogue snippet\.

## 5Conclusion

This paper examines whether memory\-augmented LLMs and long\-context baselines can translate recalled preferences into appropriate behavior\. Evaluating 16 systems across 1,000 preferences and three expression levels, we find that agents often achieve high accuracy on theKnowtest but perform poorly on theActtest\. This gap varies by expression strength and preference type, with health and emotional domains proving especially challenging\. Failure attribution further indicates that errors arise from retrieval, comprehension, and application, with comprehension failures dominating overall\. These findings suggest that future personalization systems should move beyond simply storing and recalling user information, and instead focus on ensuring that personal memories are selectively retrieved, correctly interpreted, and reliably translated into context\-appropriate behavior\.

## Limitations

The evaluation relies on LLM judges, which may introduce systemic evaluation biases such as verbosity preference or insensitivity to subtle behavioral cues\. Furthermore, our benchmark utilizes synthetic personas from PersonaMem\-v2\(Jianget al\.,[2025b](https://arxiv.org/html/2607.29433#bib.bib17)\); evaluating these systems on real\-world, longitudinal user histories that feature natural topic drift and contradictions would strengthen ecological validity\. Crucially, our design isolates the utilization bottleneck by linking each preference to a single target passage with limited competing distractors, inherently simplifying the retrieval task\. Because real\-world user interactions are vastly noisier and more fragmented, the pervasive Know\-Act gap we observe is likely a conservative lower bound, one that would widen significantly in realistic deployment settings\.

## Ethics Statement

We uses synthetic personas and preferences derived from PersonaMem\-v2\. All memory systems are evaluated in controlled settings without real user data\. And we highlights critical safety concerns, particularly the utilization gap for health and medical information, which should inform the development of more reliable memory\-enabled agents\.

## References

- Anthropic \(2025\)Using Claude’s chat search and memory to build on previous context\.Note:[https://support\.claude\.com/en/articles/11817273\-using\-claude\-s\-chat\-search\-and\-memory\-to\-build\-on\-previous\-context](https://support.claude.com/en/articles/11817273-using-claude-s-chat-search-and-memory-to-build-on-previous-context)Accessed: 2026\-03\-24Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1)\.
- Anthropic \(2026\)Introducing claude opus 4\.6\.Note:Accessed: 2026\-03\-27External Links:[Link](https://www.anthropic.com/news/claude-opus-4-6)Cited by:[§A\.1](https://arxiv.org/html/2607.29433#A1.SS1.p5.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.5.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1),[§4\.1](https://arxiv.org/html/2607.29433#S4.SS1.p3.2)\.
- A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi \(2024\)Self\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=hSyW5go0v8)Cited by:[§A\.5](https://arxiv.org/html/2607.29433#A1.SS5.p3.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.15.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou, Y\. Dong, J\. Tang, and J\. Li \(2024\)LongBench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 3119–3137\.External Links:[Link](https://aclanthology.org/2024.acl-long.172/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.172)Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. Li \(2025\)LongBench v2: towards deeper understanding and reasoning on realistic long\-context multitasks\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 3639–3664\.External Links:[Link](https://aclanthology.org/2025.acl-long.183/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.183),ISBN 979\-8\-89176\-251\-0Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- Boson AI \(2024\)RPBench\-Auto: an automated pipeline for evaluating LLMs for role\-playing\.Note:[https://github\.com/boson\-ai/RPBench\-Auto](https://github.com/boson-ai/RPBench-Auto)Accessed: 2026\-03\-24Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- A\. Chatterji, T\. Cunningham, D\. Deming, Z\. Hitzig, C\. Ong, C\. Y\. Shan, and K\. Wadman \(2025\)How people use ChatGPT\.NBER Working PaperNational Bureau of Economic Research,Cambridge, MA\.Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p2.1)\.
- D\. Chen, S\. Niu, K\. Li, P\. Liu, X\. Zheng, B\. Tang, X\. Li, F\. Xiong, and Z\. Li \(2026\)HaluMem: evaluating hallucinations in memory systems of agents\.External Links:2511\.03506,[Link](https://arxiv.org/abs/2511.03506)Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- Z\. Cheng, W\. Wang, Y\. Zhao, Z\. Ren, J\. Chen, R\. Xu, S\. Huang, Y\. Chen, G\. Li, M\. Wang, Y\. Xie, R\. Zhu, Z\. Jiang, K\. Lu, Y\. Li, X\. Wang, L\. Liu, and C\. Nguyen \(2026\)LifeBench: a benchmark for long\-horizon multi\-source memory\.External Links:2603\.03781,[Link](https://arxiv.org/abs/2603.03781)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav \(2025\)Mem0: building production\-ready ai agents with scalable long\-term memory\.External Links:2504\.19413,[Link](https://arxiv.org/abs/2504.19413)Cited by:[§A\.4](https://arxiv.org/html/2607.29433#A1.SS4.p2.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.10.2),[§1](https://arxiv.org/html/2607.29433#S1.p1.1),[§1](https://arxiv.org/html/2607.29433#S1.p4.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1),[§3\.2](https://arxiv.org/html/2607.29433#S3.SS2.p4.1),[§4\.2](https://arxiv.org/html/2607.29433#S4.SS2.p4.1),[§4\.5](https://arxiv.org/html/2607.29433#S4.SS5.p2.1)\.
- Google DeepMind \(2026\)Gemini 3\.1 pro\.Note:Accessed: 2026\-03\-27External Links:[Link](https://deepmind.google/models/gemini/pro/)Cited by:[§A\.1](https://arxiv.org/html/2607.29433#A1.SS1.p4.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.4.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- B\. J\. Gutiérrez, Y\. Shu, W\. Qi, S\. Zhou, and Y\. Su \(2025\)From RAG to memory: non\-parametric continual learning for large language models\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=LWH8yn4HS2)Cited by:[§A\.4](https://arxiv.org/html/2607.29433#A1.SS4.p5.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.13.1),[§1](https://arxiv.org/html/2607.29433#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- Y\. Hu, Y\. Wang, and J\. McAuley \(2026a\)Evaluating memory in LLM agents via incremental multi\-turn interactions\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=DT7JyQC3MR)Cited by:[§B\.1](https://arxiv.org/html/2607.29433#A2.SS1.p1.1),[§1](https://arxiv.org/html/2607.29433#S1.p1.1),[§1](https://arxiv.org/html/2607.29433#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2607.29433#S3.SS2.p4.1)\.
- Y\. Hu, S\. Liu, Y\. Yue, G\. Zhang, B\. Liu, F\. Zhu, J\. Lin, H\. Guo, S\. Dou, Z\. Xi, S\. Jin, J\. Tan, Y\. Yin, J\. Liu, Z\. Zhang, Z\. Sun, Y\. Zhu, H\. Sun, B\. Peng, Z\. Cheng, X\. Fan, J\. Guo, X\. Yu, Z\. Zhou, Z\. Hu, J\. Huo, J\. Wang, Y\. Niu, Y\. Wang, Z\. Yin, X\. Hu, Y\. Liao, Q\. Li, K\. Wang, W\. Zhou, Y\. Liu, D\. Cheng, Q\. Zhang, T\. Gui, S\. Pan, Y\. Zhang, P\. Torr, Z\. Dou, J\. Wen, X\. Huang, Y\. Jiang, and S\. Yan \(2026b\)Memory in the age of ai agents\.External Links:2512\.13564,[Link](https://arxiv.org/abs/2512.13564)Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- W\. Huang, W\. Zhang, Y\. Liang, Y\. Bei, Y\. Chen, T\. Feng, X\. Pan, Z\. Tan, Y\. Wang, T\. Wei, S\. Wu, R\. Xu, L\. Yang, R\. Yang, W\. Yang, C\. Yeh, H\. Zhang, H\. Zhang, S\. Zhu, H\. P\. Zou, W\. Zhao, S\. Wang, W\. Xu, Z\. Ke, Z\. Hui, D\. Li, Y\. Wu, L\. He, C\. Wang, X\. Xu, B\. Huang, J\. Tan, S\. Heinecke, H\. Wang, C\. Xiong, A\. A\. Metwally, J\. Yan, C\. Lee, H\. Zeng, Y\. Xia, X\. Wei, A\. Payani, Y\. Wang, H\. Ma, W\. Wang, C\. Wang, Y\. Zhang, X\. Wang, Y\. Zhang, J\. You, H\. Tong, X\. Luo, X\. Liu, Y\. Sun, W\. Wang, J\. McAuley, J\. Zou, J\. Han, P\. S\. Yu, and K\. Shu \(2026\)Rethinking memory mechanisms of foundation agents in the second half: a survey\.External Links:2602\.06052,[Link](https://arxiv.org/abs/2602.06052)Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- B\. Jiang, Z\. Hao, Y\. M\. Cho, B\. Li, Y\. Yuan, S\. Chen, L\. Ungar, C\. J\. Taylor, and D\. Roth \(2025a\)Know me, respond to me: benchmarking LLMs for dynamic user profiling and personalized responses at scale\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=6ox8XZGOqP)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1),[§4\.4](https://arxiv.org/html/2607.29433#S4.SS4.p3.1)\.
- B\. Jiang, Y\. Yuan, M\. Shen, Z\. Hao, Z\. Xu, Z\. Chen, Z\. Liu, A\. R\. Vijjini, J\. He, H\. Yu, R\. Poovendran, G\. Wornell, L\. Ungar, D\. Roth, S\. Chen, and C\. J\. Taylor \(2025b\)PersonaMem\-v2: towards personalized intelligence via learning implicit user personas and agentic memory\.External Links:2512\.06688,[Link](https://arxiv.org/abs/2512.06688)Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p2.1),[§1](https://arxiv.org/html/2607.29433#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1),[§3\.1](https://arxiv.org/html/2607.29433#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2607.29433#S3.SS1.p2.1),[§4\.4](https://arxiv.org/html/2607.29433#S4.SS4.p3.1),[Limitations](https://arxiv.org/html/2607.29433#Sx1.p1.1)\.
- J\. Johnson, M\. Douze, and H\. Jegou \(2021\)Billion\-Scale Similarity Search with GPUs\.IEEE Transactions on Big Data7\(03\),pp\. 535–547\.External Links:ISSN 2332\-7790,[Document](https://dx.doi.org/10.1109/TBDATA.2019.2921572),[Link](https://doi.ieeecomputersociety.org/10.1109/TBDATA.2019.2921572)Cited by:[§A\.3](https://arxiv.org/html/2607.29433#A1.SS3.p1.1)\.
- J\. Kang, M\. Ji, Z\. Zhao, and T\. Bai \(2025\)Memory OS of AI agent\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 25961–25970\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1318/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1318),ISBN 979\-8\-89176\-332\-6Cited by:[§A\.5](https://arxiv.org/html/2607.29433#A1.SS5.p4.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.16.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1),[§3\.2](https://arxiv.org/html/2607.29433#S3.SS2.p4.1)\.
- I\. Kumar, S\. Viswanathan, S\. Yerra, A\. Salemi, R\. A\. Rossi, F\. Dernoncourt, H\. Deilamsalehy, X\. Chen, R\. Zhang, S\. Agarwal, N\. Lipka, C\. V\. Nguyen, T\. H\. Nguyen, and H\. Zamani \(2024\)LongLaMP: a benchmark for personalized long\-form text generation\.External Links:2407\.11016,[Link](https://arxiv.org/abs/2407.11016)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- X\. Li, J\. Bantupalli, R\. Dharmani, Y\. Zhang, and J\. Shang \(2025\)Toward multi\-session personalized conversation: a large\-scale dataset and hierarchical tree framework for implicit reasoning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 11493–11506\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.580/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.580),ISBN 979\-8\-89176\-332\-6Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang \(2024\)Lost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Link](https://aclanthology.org/2024.tacl-1.9/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang \(2024\)Evaluating very long\-term conversational memory of LLM agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 13851–13870\.External Links:[Link](https://aclanthology.org/2024.acl-long.747/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.747)Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- V\. Markovic, L\. Obradovic, L\. Hajdu, and J\. Pavlovic \(2025\)Optimizing the interface between knowledge graphs and llms for complex reasoning\.External Links:2505\.24478,[Link](https://arxiv.org/abs/2505.24478)Cited by:[§A\.4](https://arxiv.org/html/2607.29433#A1.SS4.p4.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.12.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- Meta \(2025\)Personal superintelligence\.Note:[https://www\.meta\.com/superintelligence/](https://www.meta.com/superintelligence/)Accessed: 2026\-03\-25Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1)\.
- OpenAI \(2024a\)GPT\-4o mini: advancing cost\-efficient intelligence\.Note:Accessed: 2026\-03\-28External Links:[Link](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/)Cited by:[§A\.1](https://arxiv.org/html/2607.29433#A1.SS1.p2.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.2.2),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- OpenAI \(2024b\)Hello gpt‑4o\.Note:Accessed: 2026\-03\-28External Links:[Link](https://openai.com/index/hello-gpt-4o/)Cited by:[§A\.1](https://arxiv.org/html/2607.29433#A1.SS1.p3.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.3.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- OpenAI \(2024c\)Memory and new controls for ChatGPT\.Note:[https://openai\.com/index/memory\-and\-new\-controls\-for\-chatgpt/](https://openai.com/index/memory-and-new-controls-for-chatgpt/)Accessed: 2026\-03\-24Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1)\.
- OpenAI \(2024d\)New embedding models and API updates\.Note:Accessed: 2026\-03\-28External Links:[Link](https://openai.com/index/new-embedding-models-and-api-updates/)Cited by:[§A\.3](https://arxiv.org/html/2607.29433#A1.SS3.p2.1),[§A\.3](https://arxiv.org/html/2607.29433#A1.SS3.p3.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.7.2),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.8.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1),[§3\.2](https://arxiv.org/html/2607.29433#S3.SS2.p4.1)\.
- OpenAI \(2025a\)Introducing GPT\-5\.Note:Accessed: 2026\-03\-27External Links:[Link](https://openai.com/index/introducing-gpt-5/)Cited by:[§3\.1](https://arxiv.org/html/2607.29433#S3.SS1.p2.1),[§3\.1](https://arxiv.org/html/2607.29433#S3.SS1.p4.2),[§3\.2](https://arxiv.org/html/2607.29433#S3.SS2.p2.1)\.
- OpenAI \(2025b\)The power of personalized AI\.Note:Accessed: 2026\-03\-25External Links:[Link](https://openai.com/global-affairs/the-power-of-personalized-ai/)Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1)\.
- C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez \(2024\)MemGPT: towards llms as operating systems\.External Links:2310\.08560,[Link](https://arxiv.org/abs/2310.08560)Cited by:[§A\.5](https://arxiv.org/html/2607.29433#A1.SS5.p2.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.14.2),[§1](https://arxiv.org/html/2607.29433#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1),[§3\.2](https://arxiv.org/html/2607.29433#S3.SS2.p4.1)\.
- P\. Rasmussen, P\. Paliychuk, T\. Beauvais, J\. Ryan, and D\. Chalef \(2025\)Zep: a temporal knowledge graph architecture for agent memory\.External Links:2501\.13956,[Link](https://arxiv.org/abs/2501.13956)Cited by:[§A\.4](https://arxiv.org/html/2607.29433#A1.SS4.p3.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.11.1),[§1](https://arxiv.org/html/2607.29433#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- E\. Rich \(1979\)User modeling via stereotypes\.Cognitive Science3\(4\),pp\. 329–354\.External Links:ISSN 0364\-0213,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/S0364-0213%2879%2980012-9),[Link](https://www.sciencedirect.com/science/article/pii/S0364021379800129)Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p3.1),[§3\.1](https://arxiv.org/html/2607.29433#S3.SS1.p2.1)\.
- S\. E\. Robertson and S\. Walker \(1994\)Some simple effective approximations to the 2\-poisson model for probabilistic weighted retrieval\.InSIGIR’94: Proceedings of the Seventeenth Annual International ACM\-SIGIR Conference on Research and Development in Information Retrieval, organised by Dublin City University,pp\. 232–241\.Cited by:[§A\.2](https://arxiv.org/html/2607.29433#A1.SS2.p1.2),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.6.2),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- A\. Salemi, S\. Mysore, M\. Bendersky, and H\. Zamani \(2024\)LaMP: when large language models meet personalization\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 7370–7392\.External Links:[Link](https://aclanthology.org/2024.acl-long.399/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.399)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- F\. Shi, X\. Chen, K\. Misra, N\. Scales, D\. Dohan, E\. H\. Chi, N\. Schärli, and D\. Zhou \(2023\)Large language models can be easily distracted by irrelevant context\.InProceedings of the 40th International Conference on Machine Learning,A\. Krause, E\. Brunskill, K\. Cho, B\. Engelhardt, S\. Sabato, and J\. Scarlett \(Eds\.\),Proceedings of Machine Learning Research, Vol\.202,pp\. 31210–31227\.External Links:[Link](https://proceedings.mlr.press/v202/shi23a.html)Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1)\.
- L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin,et al\.\(2024a\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\(6\),pp\. 186345\.Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- N\. Wang, Z\.y\. Peng, H\. Que, J\. Liu, W\. Zhou, Y\. Wu, H\. Guo, R\. Gan, Z\. Ni, J\. Yang, M\. Zhang, Z\. Zhang, W\. Ouyang, K\. Xu, W\. Huang, J\. Fu, and J\. Peng \(2024b\)RoleLLM: benchmarking, eliciting, and enhancing role\-playing abilities of large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 14743–14777\.External Links:[Link](https://aclanthology.org/2024.findings-acl.878/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.878)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- W\. Wang, F\. Wei, L\. Dong, H\. Bao, N\. Yang, and M\. Zhou \(2020\)MINILM: deep self\-attention distillation for task\-agnostic compression of pre\-trained transformers\.InProceedings of the 34th International Conference on Neural Information Processing Systems,NIPS ’20,Red Hook, NY, USA\.External Links:ISBN 9781713829546Cited by:[§A\.5](https://arxiv.org/html/2607.29433#A1.SS5.p4.1),[§3\.2](https://arxiv.org/html/2607.29433#S3.SS2.p4.1)\.
- D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu \(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=pZiyCaVuti)Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- T\. Wu, Z\. Chen, Z\. Weng, S\. Wang, C\. Li, S\. Zhang, S\. Hu, S\. Wu, Q\. Lan, H\. Wang, and R\. Chen \(2026\)KnowMe\-bench: benchmarking person understanding for lifelong digital companions\.External Links:2601\.04745,[Link](https://arxiv.org/abs/2601.04745)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- Z\. Xi, W\. Chen, X\. Guo, W\. He, Y\. Ding, B\. Hong, M\. Zhang, J\. Wang, S\. Jin, E\. Zhou, R\. Zheng, X\. Fan, X\. Wang, L\. Xiong, Y\. Zhou, W\. Wang, C\. Jiang, Y\. Zou, X\. Liu, Z\. Yin, S\. Dou, R\. Weng, W\. Qin, Y\. Zheng, X\. Qiu, X\. Huang, Q\. Zhang, and T\. Gui \(2025\)The rise and potential of large language model based agents: a survey\.Science China Information Sciences68\(2\),pp\. 121101\.External Links:ISSN 1869\-1919,[Document](https://dx.doi.org/10.1007/s11432-024-4222-0),[Link](https://doi.org/10.1007/s11432-024-4222-0)Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang \(2025\)A\-mem: agentic memory for LLM agents\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=FiM0M8gcct)Cited by:[§A\.5](https://arxiv.org/html/2607.29433#A1.SS5.p5.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.17.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- S\. Yoon, S\. Kim, H\. Hong, W\. Jeung, Y\. Kim, W\. Seo, H\. Yeen, and A\. No \(2026\)BenchPreS: a benchmark for context\-aware personalized preference selectivity of persistent\-memory llms\.External Links:2603\.16557,[Link](https://arxiv.org/abs/2603.16557)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- X\. Zhang, Y\. Chen, S\. Hu, Z\. Xu, J\. Chen, M\. Hao, X\. Han, Z\. Thai, S\. Wang, Z\. Liu, and M\. Sun \(2024\)∞\\inftyBench: Extending long context evaluation beyond 100K tokens\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15262–15277\.External Links:[Link](https://aclanthology.org/2024.acl-long.814/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.814)Cited by:[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p1.1)\.
- Y\. Zhang, M\. Li, D\. Long, X\. Zhang, H\. Lin, B\. Yang, P\. Xie, A\. Yang, D\. Liu, J\. Lin, F\. Huang, and J\. Zhou \(2025a\)Qwen3 embedding: advancing text embedding and reranking through foundation models\.External Links:2506\.05176,[Link](https://arxiv.org/abs/2506.05176)Cited by:[§A\.3](https://arxiv.org/html/2607.29433#A1.SS3.p4.1),[Table 3](https://arxiv.org/html/2607.29433#A1.T3.1.9.1),[§2\.2](https://arxiv.org/html/2607.29433#S2.SS2.p2.1)\.
- Z\. Zhang, Q\. Dai, X\. Bo, C\. Ma, R\. Li, X\. Chen, J\. Zhu, Z\. Dong, and J\. Wen \(2025b\)A survey on the memory mechanism of large language model\-based agents\.ACM Trans\. Inf\. Syst\.43\(6\)\.External Links:ISSN 1046\-8188,[Link](https://doi.org/10.1145/3748302),[Document](https://dx.doi.org/10.1145/3748302)Cited by:[§1](https://arxiv.org/html/2607.29433#S1.p1.1)\.
- S\. Zhao, M\. Hong, Y\. Liu, D\. Hazarika, and K\. Lin \(2025\)Do LLMs recognize your preferences? evaluating personalized preference following in LLMs\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=QWunLKbBGF)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.
- T\. P\. Zollo, A\. W\. T\. Siah, N\. Ye, A\. Li, and H\. Namkoong \(2025\)PersonalLLM: tailoring LLMs to individual preferences\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=2R7498e2Tx)Cited by:[§2\.1](https://arxiv.org/html/2607.29433#S2.SS1.p1.1)\.

## Appendix AMemory System Implementation Details

This section provides detailed technical descriptions and hyperparameter settings for each evaluated memory system\. Table[3](https://arxiv.org/html/2607.29433#A1.T3)provides an overview of all systems organized by the five\-category taxonomy\. All RAG and agentic memory systems use GPT\-4o\-mini \(temperature=0\) as the generation backbone\.

Table 3:Memory systems evaluated in KnowAct, organized by the five\-category taxonomy\.CategorySystemKey MechanismLong\-ContextGPT\-4o\-mini\(OpenAI,[2024a](https://arxiv.org/html/2607.29433#bib.bib215)\)Full concatenation \(128K\)GPT\-4o\(OpenAI,[2024b](https://arxiv.org/html/2607.29433#bib.bib214)\)Full concatenation \(128K\)Gemini 3\.1 Flash\(Google DeepMind,[2026](https://arxiv.org/html/2607.29433#bib.bib193)\)Full concatenation \(1M\)Claude 4\.6 Sonnet\(Anthropic,[2026](https://arxiv.org/html/2607.29433#bib.bib194)\)Full concatenation \(1M\)Simple RAGBM25\(Robertson and Walker,[1994](https://arxiv.org/html/2607.29433#bib.bib170)\)Sparse lexical retrievalEmbeddingRAGRAG te3\-small\(OpenAI,[2024d](https://arxiv.org/html/2607.29433#bib.bib216)\)Dense vector retrieval \(1536d\)RAG te3\-large\(OpenAI,[2024d](https://arxiv.org/html/2607.29433#bib.bib216)\)Dense vector retrieval \(3072d\)RAG Qwen3\-Emb\-4B\(Zhanget al\.,[2025a](https://arxiv.org/html/2607.29433#bib.bib218)\)Dense vector retrieval \(1024d\)Structure\-Aug\.RAGMem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\)Fact extraction \+ vector/graph storeZep/Graphiti\(Rasmussenet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib19)\)Temporal knowledge graphCognee\(Markovicet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib18)\)Feedback\-refined KGHippoRAG\-v2\(Gutiérrezet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib74)\)CLS\-inspired pattern completionAgenticMemoryLetta/MemGPT\(Packeret al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib53)\)Hierarchical memory mgmt\.Self\-RAG\(Asaiet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib42)\)Learned retrieval decisionsMemoryOS\(Kanget al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib46)\)OS\-inspired memory mgmt\.A\-MEM\(Xuet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib61)\)Zettelkasten\-based memory### A\.1Long\-Context Baselines

Long\-context baselines receive all conversation chunks concatenated into a single context window preceding the test query\. No retrieval or memory management is performed; the model attends over the full history directly\.

GPT\-4o\-mini\(OpenAI,[2024a](https://arxiv.org/html/2607.29433#bib.bib215)\)is used with a 128K context window\. It serves as the shared generation backbone for all non\-long\-context systems, ensuring that performance differences across RAG and agentic systems reflect memory architecture rather than generation capability\.

GPT\-4o\(OpenAI,[2024b](https://arxiv.org/html/2607.29433#bib.bib214)\)is used with a 128K context window\. Compared to GPT\-4o\-mini, it offers stronger reasoning capabilities at higher computational cost, allowing us to assess whether generation quality affects the Know–Act gap\.

Gemini 3\.1 Flash\(Google DeepMind,[2026](https://arxiv.org/html/2607.29433#bib.bib193)\)supports a context window exceeding 1 million tokens, meaning the entire conversation history fits within the window for all personas in our benchmark\.

Claude 4\.6 Sonnet\(Anthropic,[2026](https://arxiv.org/html/2607.29433#bib.bib194)\)is evaluated with a 1M context window\. As one of the strongest available models, it provides an upper\-bound reference for what long\-context processing can achieve on the KnowAct tasks\.

### A\.2Simple RAG

BM25\(Robertson and Walker,[1994](https://arxiv.org/html/2607.29433#bib.bib170)\)uses the BM25 algorithm with default parameters \(k1k\_\{1\}=1\.5,bb=0\.75\) to rank chunks based on sparse lexical overlap\. At query time, the top\-10 chunks ranked by BM25 score are retrieved and concatenated as context for GPT\-4o\-mini\.

### A\.3Embedding\-based RAG

We evaluate three embedding models of varying capacity, all using FAISS\(Johnsonet al\.,[2021](https://arxiv.org/html/2607.29433#bib.bib75)\)for vector indexing and cosine similarity for retrieval\. The top\-10 most similar chunks are retrieved as context for GPT\-4o\-mini\.

RAG te3\-smalluses OpenAI’s text\-embedding\-3\-small\(OpenAI,[2024d](https://arxiv.org/html/2607.29433#bib.bib216)\)\(1,536 dimensions\), a lightweight embedding model suitable for high\-throughput applications\.

RAG te3\-largeuses OpenAI’s text\-embedding\-3\-large\(OpenAI,[2024d](https://arxiv.org/html/2607.29433#bib.bib216)\)\(3,072 dimensions\), offering higher retrieval precision at increased computational cost\.

RAG Qwen3\-Emb\-4Buses Qwen3\-Embedding\-4B\(Zhanget al\.,[2025a](https://arxiv.org/html/2607.29433#bib.bib218)\)\(1,024 dimensions\), a recent open\-source embedding model that achieves competitive retrieval performance with a smaller dimensionality\.

### A\.4Structure\-Augmented RAG

These systems augment retrieval with structured representations such as fact triples, knowledge graphs, or temporal indices\. Each processes conversation chunks through its native ingestion pipeline during the injection phase\.

Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib12)\)\(v1\.0\.7\) uses GPT\-4o\-mini to extract atomic facts from each conversation chunk during injection\. Extracted facts are stored in both a Qdrant vector database and a knowledge graph\. At query time, Mem0 retrieves relevant facts by combining vector similarity search with graph traversal\.

Zep/Graphiti\(Rasmussenet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib19)\)constructs a bi\-temporal knowledge graph during chunk injection, tracking both when events occurred and when they were ingested\. Entity and relationship extraction builds the graph incrementally\. Retrieval combines graph traversal with semantic search over temporal subgraphs\.

Cognee\(Markovicet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib18)\)builds a knowledge graph using its Extract\-Cognify\-Load \(ECL\) pipeline\. Each injected chunk updates the graph structure, and the system iteratively refines edge weights through feedback loops\. Retrieval uses graph\-augmented semantic search\.

HippoRAG\-v2\(Gutiérrezet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib74)\)implements a dual\-system architecture inspired by complementary learning systems theory\. An LLM serves as the neocortex, a knowledge graph as the hippocampal index, and personalized PageRank enables schema\-based pattern completion for associative retrieval beyond direct keyword or embedding matching\.

### A\.5Agentic Memory

Agentic memory systems grant the agent control over its own memory operations, including what to store, when to retrieve, and how to organize accumulated knowledge\.

Letta/MemGPT\(Packeret al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib53)\)manages a hierarchical memory system with three tiers: core memory \(always present in context\), recall memory \(conversation history retrieved on demand\), and archival memory \(long\-term storage for persistent facts\)\. The agent autonomously issues function calls to move information between tiers, deciding what to retain in limited core memory and what to archive\.

Self\-RAG\(Asaiet al\.,[2024](https://arxiv.org/html/2607.29433#bib.bib42)\)implements a multi\-step self\-reflective retrieval pipeline\. The system first decides whether retrieval is needed for a given query, then retrieves relevant chunks, assesses their relevance, generates a candidate response, verifies whether the response is grounded in retrieved content, evaluates utility, and selects the best response\. This process gives the agent autonomous control over when and what to retrieve\.

MemoryOS\(Kanget al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib46)\)draws an analogy between memory management and operating system design, implementing memory allocation, paging, and garbage collection mechanisms\. It uses all\-MiniLM\-L6\-v2\(Wanget al\.,[2020](https://arxiv.org/html/2607.29433#bib.bib217)\)as its embedding model, as specified in the original implementation, rather than the shared text\-embedding\-3\-small used by other systems\.

A\-MEM\(Xuet al\.,[2025](https://arxiv.org/html/2607.29433#bib.bib61)\)organizes memory around the Zettelkasten \(slip\-box\) method\. Each new piece of information triggers the creation of a structured note with contextual descriptions, keywords, and tags\. Critically, the system updates older notes in light of new information, enabling memory evolution rather than append\-only accumulation\.

## Appendix BPrompt Templates and Judge Design

### B\.1Memory Injection Prompt

For long\-context agents, conversation chunks are concatenated using the MemoryAgentBench protocol\(Huet al\.,[2026a](https://arxiv.org/html/2607.29433#bib.bib14)\)\. Each chunk is wrapped in a memorization template:

<User\> The following context is the conversation between the user and the assistant: User: \{message\_1\} Assistant: \{message\_2\} \.\.\. <Assistant\> I have memorized the conversation and I will answer the question you ask\.

For RAG\-based systems, retrieved chunks are formatted as numbered memory entries:

Memory 1: \{chunk\_text\_1\}Memory 2: \{chunk\_text\_2\}\.\.\.\{question\}

where\{message\_1\},\{message\_2\}, etc\., are individual conversation turns\. For RAG systems,\{chunk\_text\_i\}represents the retrieved chunk text, and\{question\}is the user’s query\.

### B\.2Know and Act Test Prompts

Both Know and Act tests use a minimal system prompt to isolate the memory system’s contribution:

System: You are a helpful personal assistant\.User: \{know\_question or act\_scenario\}

where\{know\_question\}is phrased in first person \(e\.g\., “Do I have any seasonal allergies?”\) for Know tests, and\{act\_scenario\}presents a natural request without referencing the target preference \(e\.g\., “Can you recommend some fun weekend activities for this spring?”\) for Act tests\.

### B\.3Know Judge Prompt

The Know judge \(GPT\-5, reasoning effort=low\) evaluates whether the agent correctly recalled a specific user preference:

You are an impartial judge evaluating whether an AI assistant correctly recalled a specific user preference\.Input \- Question: The question asked to the assistant about the user’s preference\. \- Ground Truth: The correct answer about the user’s preference\. \- Assistant Response: The assistant’s actual response\.Evaluation Criteria Rate the response as Pass or Fail:\- Pass: The response conveys the same core information as the ground truth\. Exact wording need not match, paraphrasing, reordering, or adding minor context is fine, as long as the essential preference is correctly identified\.\- Fail: The response does not demonstrate knowledge of the specific preference\.Output Format Respond ONLY with a JSON object: \{"score": "pass" or "fail", "reasoning": "One sentence explanation\."\}

### B\.4Act Judge Prompt

The Act judge \(GPT\-5, reasoning effort=low\) evaluates whether the memory\-augmented response reflects consideration of the user’s preference by comparing it against the zero\-memory baseline:

You are an impartial judge evaluating whether an AI assistant’s response reflects consideration of a known user preference\.You will compare two responses to the same user request: \- Response A: Generated WITHOUT any knowledge of the user \(zero\-memory baseline\)\. \- Response B: Generated by a system WITH access to the user’s conversation history\.Input \- User Preference: The preference the memory\-enabled system should ideally be aware of\. \- User Request: The scenario posed to the assistant\. \- Response A \(Zero\-Memory\): Baseline response with no user knowledge\. \- Response B \(With Memory\): Response from the memory\-enabled system\.Evaluation Criteria Rate as Pass or Fail:\- Pass: Response B appears to take the user’s preference into account in a way that Response A does not\. The preference does not need to be explicitly mentioned, it is sufficient if the response’s suggestions, tone, or direction seem shaped by awareness of the preference\.\- Fail: \- Response B shows no discernible difference from Response A in how it addresses the preference \- Response B ignores the preference entirely despite having memory accessImportant \- Do NOT favor Response B simply because it is longer or more detailed\. \- The preference can be reflected implicitly, look for whether the overall response feels tailored rather than generic\.Output Format Respond ONLY with a JSON object: \{"score": "pass" or "fail", "reasoning": "One sentence explanation\."\}

##### Blinded Act Judge Validation\.

To quantify the potential bias introduced by the non\-blind Act judge design, we randomly shuffle the assignment of Response A and B and remove all labels indicating memory access on a 200\-sample subset\. The blinded judge produces identical verdicts on 95% of cases, with Cohen’s Kappa of 0\.87 compared to the non\-blinded judge\. Among the disagreements, 5 cases were upgraded \(blind=Fail, non\-blind=Pass\) and 5 cases were downgraded, showing no systematic bias toward the memory\-augmented response\.

### B\.5Snippet Generation Prompts

#### B\.5\.1Explicit Snippet Generation

GPT\-5 \(temperature=0\.7\) generates conversations where the user directly states the preference:

You are generating a short conversation between a user and an AI chatbot\.Context \- User persona \(for tone only\): \{short\_persona\} \- Preference to convey: \{preference\_text\} \- Category: \{preference\_type\_label\}Task Generate a natural 2\-4 turn conversation where the user DIRECTLY TELLS the chatbot the above preference\. The preference itself should be the MAIN POINT of what the user is saying\.Key principle Think of it this way: the user is sharing something about themselves with the chatbot\. Like telling a friend "I’m really into sumo wrestling" or "My dad’s side of the family has diabetes\."Rules 1\. The preference IS the message: Do NOT wrap it inside a task request\. 2\. Natural phrasing: Do NOT copy the preference text verbatim\. 3\. Persona\-consistent tone: Match the speaking style to the persona\. 4\. Chatbot responds naturally: Acknowledge and engage \(2\-4 sentences\)\. 5\. No identity leakage: Do NOT reveal job, nationality, age, etc\. 6\. Language: MUST be in English\.Category guidance \- Interests & hobbies: "I’m really into origami \-\- I love the precision\." \- Health & medical: "My dad’s side has diabetes, so I watch my sugar\." \- Therapy & emotional: "I’ve been carrying guilt about being away from my kids\."Output Return ONLY a JSON array: \[\{"role": "user", "content": "\.\.\."\}, \.\.\.\]

where\{short\_persona\},\{preference\_text\}, and\{preference\_type\_label\}are template fields\.\{short\_persona\}provides a brief persona description for tone matching\.\{preference\_text\}is the specific user preference to be conveyed\.\{preference\_type\_label\}indicates the category \(Interests & hobbies, Health & medical, or Therapy & emotional\)\.

#### B\.5\.2Inferential Snippet Generation

GPT\-5 \(temperature=0\.7\) generates conversations where the preference is never directly stated but inferable from behavioral cues:

You are generating a short conversation between a user and an AI chatbot\.Context \- User persona \(for tone only\): \{short\_persona\} \- Hidden preference \(DO NOT mention directly\): \{preference\_text\} \- Category: \{preference\_type\_label\}Task Generate a natural 2\-4 turn conversation where the user makes a request that implicitly reveals the preference through behavior, choices, or situational details \-\- WITHOUT ever directly stating it\.Key principle \-\- the "one step of reasoning" test After reading, a perceptive reader should conclude: "This person probably \[core of the preference\]\." The inference should require at least one reasoning step, but the core must be REACHABLE\.Bad \(too deep\): preference is "family history of diabetes on maternal side" → user only asks for low\-sugar recipes\. Reader can only guess "watches sugar intake" \-\- maternal side is lost\.Good: "My mom’s been after me to cut back on sugar ever since her doctor put her on metformin\. Can you suggest lower\-glycemic snacks?" → Reader infers: mom \+ metformin → maternal diabetes history\.Rules 1\. The preference is INVISIBLE: Never state, name, or paraphrase it\. 2\. Behavior reveals the preference: Inferable from what user asks\. 3\. Natural purpose: Genuine request \(planning, advice, shopping, etc\.\)\. 4\. Chatbot responds naturally: Help with the request \(2\-4 sentences\)\. 5\. No identity leakage: Do NOT reveal job, nationality, age, etc\. 6\. Language: MUST be in English\.Category guidance \- Interests: "I’m heading to Colorado in February\. What gear do I need to rent? I have my own boots and goggles\." \(skiing, never said\) \- Health: "My eyes have been killing me all week and the tree count is off the charts tomorrow\. Can we move our meeting indoors?" \(pollen\) \- Therapy: "I missed my daughter’s recital again because of a client dinner\. Can you help me draft a message to her teacher?" \(guilt\)Output Return ONLY a JSON array: \[\{"role": "user", "content": "\.\.\."\}, \.\.\.\]

where\{short\_persona\},\{preference\_text\}, and\{preference\_type\_label\}are template fields with the same meanings as in the explicit prompt\.

### B\.6Test Generation Prompt

GPT\-5 \(temperature=0\.7\) generates paired Know and Act tests for each preference:

You are creating paired evaluation items for the KnowAct benchmark, which tests whether AI memory agents can both RECALL user preferences \(Know\) and APPLY them in practice \(Act\)\.Input \- User preference: \{preference\_text\} \- Preference type: \{preference\_type\_label\}Task Generate TWO paired tests for this preference:1\. KNOW TEST A simple, direct question that checks whether the agent remembers this specific preference\. Requirements: \- Use FIRST PERSON: "Do I have pollen allergies?" \(NOT third person\) \- Should be unambiguous and answerable with a short response \- Requires ONLY memory recall, no complex reasoning \- Specific enough that a correct answer confirms the exact preference2\. ACT TEST A natural user request \(scenario\) where this preference SHOULD influence the agent’s response\. Requirements: \- Should be a realistic request a user might make to a chatbot \- The preference’s relevance should be IMPLICIT \-\- do NOT mention it \- A preference\-aware response should be noticeably different from a preference\-unaware response \- Evaluated by comparing response WITH memory against zero\-memory baselineCategory guidance for Act scenario design \- Interests & hobbies: A request where knowing this interest improves recommendations \(activities, gifts, content, travel\)\. \- Health & medical: A request where the condition should influence recommendations \(meal planning, exercise, health checkups\)\. \- Therapy & emotional: A request where understanding the emotional background leads to more sensitive suggestions\.Output format Return ONLY a JSON object: \{"know\_test": \{"question": "\.\.\.", "ground\_truth": "\.\.\."\}, "act\_test": \{"scenario": "\.\.\."\}\}

where\{preference\_text\}and\{preference\_type\_label\}are template fields\.\{preference\_text\}is the specific user preference for which tests are being generated\.\{preference\_type\_label\}indicates the category \(Interests & hobbies, Health & medical, or Therapy & emotional\)\.

### B\.7Evaluation Protocol Details

- •Generation model: GPT\-4o\-mini via OpenRouter, temperature=0, max output tokens=2,048\.
- •Judge model: GPT\-5 via OpenAI API, reasoning effort=low\.
- •RAG top\-K: 10 chunks for all RAG\-based systems\.
- •Chunk definition: Each chunk corresponds to one PersonaMem\-v2 conversation entry \(2–6 messages\), preserving natural conversation boundaries\.

## Appendix CData Construction Details

Table[4](https://arxiv.org/html/2607.29433#A3.T4)summarizes the overall statistics of the KnowAct benchmark dataset\.

Table 4:KnowAct dataset statistics\.StatisticValuePersonas & PreferencesPersonas \(cultural regions\)50 \(11\)Preferences per persona20Total preferences1,000Preference types5Expression types3Conversation ChunksChunk sequences150 \(50×\\times3\)Total chunks11,955 \(3,985 per type\)Chunks per persona79\.7 \(55–116\)Target : non\-target ratio∼\{\\sim\}1 : 3Tokens \(GPT\-4o tokenizer\)Explicit1,376,927 \(27\.5K/persona\)Incidental1,652,583 \(33\.1K/persona\)Inferential1,524,506 \(30\.5K/persona\)Total4,554,016EvaluationPaired tests \(Know \+ Act\)1,000Total queries2,000### C\.1Persona Selection \(Step 1\)

We select 50 personas from PersonaMem\-v2’s 999 personas through a multi\-stage filtering and stratified sampling pipeline\.

##### Filtering criteria\.

Three filters are applied sequentially:

1. 1\.Nationality: Personas with null or missing nationality fields are excluded \(68 removed\), as nationality is required for cultural\-region stratification\.
2. 2\.Preference count: We require\>\>70 total preferences per persona\. This ensures at least∼\{\\sim\}50 non\-target conversation entries remain after sampling 20 target preferences, providing sufficient distractor context in chunk sequences\.
3. 3\.Language: Personas whoseshort\_personadescription contains non\-English characters \(CJK: U\+4E00–U\+9FFF, Arabic: U\+0600–U\+06FF, Hebrew: U\+0590–U\+05FF\) are excluded, as all generated snippets and tests are in English\.

##### Stratified sampling\.

Remaining personas are classified into 11 cultural regions via substring matching of nationality fields \(e\.g\., “American”→\\toNorth America, “Japanese”→\\toEast Asia\)\. Each region is guaranteed a minimum of 2 personas; the remaining 28 slots are allocated proportionally by region pool size using the largest\-remainder method\. All randomization uses a fixed seed of 42\. Table[5](https://arxiv.org/html/2607.29433#A3.T5)shows the resulting distribution\.

Table 5:Geographic distribution of the 50 selected personas\. Allocation uses stratified sampling with a minimum of 2 per region and proportional allocation for the remaining slots\.RegionPoolSelectedNorth America43420Southeast Asia665Sub\-Saharan Africa444Western Europe413South Asia243Eastern/Northern Europe223Other193Oceania193East Asia92Middle East / North Africa42Latin America32Total68550

### C\.2Preference Sampling \(Step 2a\)

For each persona, we sample 20 preferences with type\-stratified quotas and a single\-snippet constraint\.

##### Single\-snippet constraint\.

A preference is eligible for selection only if it hasexactly oneassociated conversation snippet in PersonaMem\-v2’s raw data\. Preferences with zero snippets lack the incidental conversation needed for the Incidental condition, while preferences with multiple snippets would create ambiguity about which conversation entry to use as the target chunk\. This constraint also ensures each target preference appears exactly once in the chunk sequence\.

##### Type quotas\.

We sample 20 preferences per persona using fixed quotas: 5 stereotypical, 5 anti\-stereotypical, 4 neutral, 4 health/medical, 2 therapy/emotional\. When a preference type has fewer eligible preferences than its quota, remaining slots are filled using a fallback priority order: neutral→\\toanti\-stereotypical→\\tostereotypical→\\tohealth/medical→\\totherapy\.

##### Persona replacement\.

If a persona has fewer than 20 eligible single\-snippet preferences after quota filling, it is replaced by another persona from the same cultural region\. The replacement must meet all filtering criteria \(\>\{\>\}70 preferences,≥\{\\geq\}20 single\-snippet preferences, English\-only\)\. The replacement retains the originalknowact\_persona\_id\(e\.g\., KA\_P15\) to maintain stable identifiers\. Up to 10 replacement iterations are performed with incrementing random seeds \(42\+i42\+i\)\.

Table[6](https://arxiv.org/html/2607.29433#A3.T6)shows the resulting distribution\.

Table 6:Distribution of 1,000 sampled preferences by type\. Deviations from exact quota×\\times50 reflect the fallback mechanism\.Preference TypeQuotaActualStereotypical5229Anti\-stereotypical5252Neutral4257Health/medical4162Therapy/emotional2100Total201,000

### C\.3Snippet Generation \(Steps 2b/2c\)

We generate two conversation snippet variants for each of the 1,000 preferences using GPT\-5 \(temperature=0\.7, reasoning effort=low\) via OpenRouter\. The Incidental variant requires no generation as it uses PersonaMem\-v2’s original conversation entry directly\.

##### Explicit snippets \(Step 2b\)\.

The prompt instructs GPT\-5 to generate a 2–4 turn conversation where the userdirectly statesthe preference as the main point of the message\. Key constraints: \(1\) the preference must be the central message, not buried within a task request; \(2\) natural paraphrasing rather than verbatim copying of the preference text; \(3\) persona\-consistent speaking style; \(4\) no identity leakage \(nationality, profession, age\)\.

##### Inferential snippets \(Step 2c\)\.

The prompt instructs GPT\-5 to generate a 2–4 turn conversation where the preference is never directly stated but inferable from behavioral cues\. We enforce a “one step of reasoning” test: a perceptive reader should be able to conclude the core preference through a single inference step\. For example, if the preference is “family history of diabetes on maternal side,” a bad snippet would have the user merely ask for low\-sugar recipes \(too indirect—the maternal side is lost\), while a good snippet might mention “Mom’s been on me about cutting sugar since her doctor put her on metformin” \(metformin \+ mother→\\tomaternal diabetes\)\.

##### Validation\.

Each generated snippet undergoes structural validation: at minimum 2 messages, each with a valid role \(user/assistant\) and non\-empty content\. Failed validations trigger up to 2 additional generation attempts\. API rate limits \(HTTP 429\) are handled with exponential backoff\.

Table[7](https://arxiv.org/html/2607.29433#A3.T7)summarizes the token usage and estimated cost\.

Table 7:Token usage and estimated cost for generating explicit and inferential snippet variants \(GPT\-5\)\.VariantInput TokensOutput TokensCostExplicit \(Step 2b\)670,619770,408$8\.54Inferential \(Step 2c\)1,049,6191,391,627$15\.23Test Pairs \(Step 4\)486,789604,054$6\.65Total2,207,0272,766,089$30\.42

### C\.4Chunk Sequence Construction \(Step 3\)

Each persona has 3 chunk sequences \(one per expression type\), constructed from PersonaMem\-v2’s raw conversation data with expression\-specific target snippets substituted in\.

##### Chunk composition\.

Each sequence contains two types of chunks:

- •Target chunks\(20 per sequence\): One per sampled preference\. The conversation snippet varies by expression type—explicit snippets from Step 2b, original PersonaMem\-v2 entries for Incidental, or Inferential snippets from Step 2c\.
- •Non\-target chunks\(∼\{\\sim\}60 per sequence\): Drawn from the same persona’s remaining conversation entries\. These include non\-target preference conversations, ask\-to\-forget requests \(for non\-target preferences only\), and sensitive information entries\. Non\-target chunks areidenticalacross all three expression types for the same persona\.

##### Conflict removal\.

To ensure unambiguous preference signals, several types of entries are removed from chunk sequences:

- •Ask\-to\-forget entries referencing target preferences: These would instruct the agent to forget a preference we later test, creating contradictory signals\.
- •Updated entries for target preferences: PersonaMem\-v2 contains preference update conversations where users modify earlier statements\. If the updated preference is a target, both the original and update are removed to avoid conflicting versions\.
- •Duplicate target entries: If a preference appears in multiple conversation types, only the first occurrence is kept\.
- •Non\-English and multimodal entries: Removed for consistency\.

##### Ordering\.

Chunks are ordered using a dependency\-aware algorithm: \(1\) independent chunks \(noprev\_preffield\) are shuffled with a fixed seed; \(2\) dependent chunks \(e\.g\., ask\-to\-forget entries that reference a preceding preference\) are inserted immediately after their dependency\. This preserves conversational coherence while randomizing the overall sequence\.

##### Statistics\.

Total: 150 sequences, 11,955 chunks, 4\.55M tokens\. Table[8](https://arxiv.org/html/2607.29433#A3.T8)reports token distribution across the three expression types\.

Table 8:Token distribution across 150 chunk sequences \(50 personas×\\times3 expression types\)\. Non\-target chunks are shared across expression types; only the target snippet differs\.Expression TypeChunksTokensAvg/PersonaExplicit3,9851,376,92727,539Incidental3,9851,652,58333,052Inferential3,9851,524,50630,490Total11,9554,554,016—Incidental sequences are the longest because PersonaMem\-v2’s original conversation entries \(which serve as Incidental snippets\) tend to be longer than GPT\-5\-generated Explicit and Inferential variants\.

### C\.5Non\-target Chunk Composition

Non\-target chunks serve as realistic distractor context, drawn from the same persona’s conversation history to maintain consistent persona characteristics\. They include three categories: \(1\) non\-target preference conversations that provide natural background noise; \(2\) ask\-to\-forget entries for non\-target preferences, which test whether memory systems correctly handle unlearning requests without affecting target preferences; and \(3\) sensitive information entries \(e\.g\., addresses, financial details\) that test privacy\-aware memory behavior\. Table[9](https://arxiv.org/html/2607.29433#A3.T9)shows the composition\.

Table 9:Distribution of non\-target chunk types across all sequences\.Chunk TypeCountNon\-target preferences2,195Ask\-to\-forget \(non\-target\)620Sensitive information170Total non\-target2,985
### C\.6Test Generation \(Step 4\)

For each of the 1,000 preferences, we generate a paired Know test and Act test using GPT\-5 \(temperature=0\.7, reasoning effort=low\)\.

##### Know test design\.

Each Know test is a direct first\-person question that checks whether the agent recalls the specific preference \(e\.g\., “Do I have any seasonal allergies?” or “What kind of music do I collect?”\)\. The question must be unambiguous and answerable with a short response, requiring only memory recall without complex reasoning\. The ground truth is a structured short statement confirming the preference\.

##### Act test design\.

Each Act test presents a realistic user request where the target preferenceshouldinfluence the agent’s response, but the preference itself is never mentioned in the scenario\. For example, for a preference about pollen allergies, the Act scenario might be “Can you recommend some fun weekend activities for this spring?” A preference\-aware response should suggest indoor activities or mention allergy considerations, while a zero\-memory response would give generic outdoor recommendations\.

##### Category\-specific guidance\.

The generation prompt provides category\-specific examples:

- •Interests & hobbies: Requests where knowing the interest improves recommendations \(activities, gifts, travel\)\.
- •Health & medical: Requests where the condition should influence suggestions \(meal planning, exercise, checkups\)\.
- •Therapy & emotional: Requests where emotional background leads to more sensitive, personalized advice\.

##### Validation\.

Each generated test pair is validated for structural completeness: the Know test must contain a non\-emptyquestionandground\_truth, and the Act test must contain a non\-emptyscenario\. Failed validations trigger regeneration with up to 2 additional attempts\.

### C\.7Representative Examples

Tables[10](https://arxiv.org/html/2607.29433#A3.T10)and[11](https://arxiv.org/html/2607.29433#A3.T11)present one representative preference from each of the five categories, illustrating how the same underlying preference is expressed across the three expression types \(Table[10](https://arxiv.org/html/2607.29433#A3.T10)\) and the corresponding Know and Act tests \(Table[11](https://arxiv.org/html/2607.29433#A3.T11)\)\.

Table 10:Representative snippet excerpts across expression types\. Each row shows how one preference appears in the Explicit, Incidental, and Inferential variants\. Snippets are abbreviated from actual benchmark data\.TypeExplicitIncidentalInferentialStereo\.
Kaiseki
\(P01\)I’m really into kaiseki\-\-\-multi\-course meals where seasonality and meticulous craft are front and center\.I had to cancel a gathering at a ryotei where the chef had prepared a seasonal sequence of dishes\.The meal moves from a tiny opener to a clear broth, then a platter reflecting mountain and sea, followed by sashimi…Anti\-s\.
VR gaming
\(P02\)I really enjoy VR games\. Rhythm games and exploration ones are my favorites\.I joined a program that let me walk through a virtual reconstruction of an ancient temple…I keep fogging the lenses and feel sick when I do smooth turning in the ruins game\.Neutral
Tea
\(P03\)I really value quiet mornings with a hot cup of tea\-\-\-no chatter, just the steam and first light\.This morning I sat by the window with a mug of tea… the streets were still empty\.I’m restoring a pre\-war kettle for a 0500 balcony ritual\. I need it running as quietly as possible…Health
Allergies
\(P04\)I get a nasty bout of hay fever when the pollen kicks up\-\-\-sneezing, itchy eyes, the whole parade\.I find myself lingering in the house, taking my pipe out only in cooler evening hours\.Every year once the tulips are up, my evening pipe turns into a chorus of sneezes and gritty eyes\.Therapy
Displace\.
\(P05\)Coming back to Bhutan makes me feel oddly out of place, like a guest in my own home\.I found myself walking familiar streets with an odd sense of distance\-\-\-as though the buildings had shifted\.I keep waking at odd hours and hesitating over whether to kiss a cheek or press my palms together\.Table 11:Know and Act tests for the representative examples in Table[10](https://arxiv.org/html/2607.29433#A3.T10)\.CategoryKnow\(question / ground truth\)Act\(scenario\)Stereotyp\.
KaisekiWhat multi\-course dining style do I enjoy?
GT: Kaiseki\.I’m planning a splurge\-worthy anniversary dinner in Kyoto next month\. Can you recommend places to book?Anti\-ster\.
VR gamingDo I enjoy VR\-based gaming experiences?
GT: Yes\.I’ve got $600 to refresh my gaming setup in a small studio apartment\. What should I prioritize?Neutral
TeaDo I appreciate quiet mornings with tea?
GT: Yes\.I want a small gift under $40 to make my mornings nicer\. What would you suggest?Health
AllergiesDo I have seasonal pollen allergies?
GT: Yes, spring and early summer\.I’m planning a day hike in late May\. Suggest trails, timing, and a packing list?Therapy
DisplacementWhat feeling do I tend to have returning to Bhutan after long stays in Europe?
GT: Displacement\.I just got back to Thimphu after several months in Europe\. Help me plan my first week to ease back in?

## Appendix DQuality Validation

### D\.1Reproducibility

All stochastic steps use a fixed random seed \(seed=42\)\. Two independent runs of the persona selection pipeline \(Step 1\) produce byte\-identical output files, confirming full reproducibility\.

### D\.2Target Preference Isolation

Each target preference appears in exactly one chunk per sequence\. We verify this through the following checks:

- •Zero multi\-snippet preferences: No preference has more than one conversation entry in any sequence \(duplicate entries for target preferences are filtered in Step 3\)\.
- •Zero identity leakage: Explicit and Inferential snippets generated in Step 2 do not leak persona identity information \(nationality, name, demographic details\) that could confound the expression strength comparison\.
- •Conflict removal: 81\.7% of target preferences in PersonaMem\-v2’s original data have associated ask\-to\-forget or updated entries\. All such conflicting entries are removed from the chunk sequence to ensure that each preference signal is unambiguous\.

### D\.3Cross\-Baseline Consistency

For each persona, the three chunk sequences \(Explicit, Incidental, Inferential\) share identical non\-target chunks\. Only the 20 target preference snippets differ across baselines\. This is verified through an audit of 6 randomly selected personas, confirming:

- •Non\-target chunk lists are identical across all three expression types\.
- •Chunk ordering is consistent \(independent blocks in the same shuffled order; dependent blocks correctly positioned after their antecedents\)\.
- •No information leakage between target and non\-target chunks\.

### D\.4Zero\-Memory Baseline Know Test

The zero\-memory baseline evaluates each generation model on all 1,000 Know test questionswithout any prior conversation history\. The model receives only the system prompt and the Know question \(e\.g\., “What multi\-course dining style do I enjoy?”\)\. This test serves a single purpose:validating that KnowAct’sKnowquestions cannot be answered by prior knowledge or guessing alone, thereby confirming that any non\-trivialKnowaccuracy observed in memory\-augmented systems genuinely reflects successful memorization rather than lucky inference\.

##### Results\.

Table[12](https://arxiv.org/html/2607.29433#A4.T12)reports the Know pass rates for all four generation models under the zero\-memory condition\.

Table 12:Zero\-memory Know test pass rates by generation model\.ModelKnow Pass RateClaude 4\.6 Sonnet0\.4%Gemini 3\.1 Flash2\.2%GPT\-4o2\.3%GPT\-4o\-mini2\.7%All models score below 3%, confirming that the Know questions are effectively unanswerable without memory\. The small residual pass rates stem primarily from therapy/emotional preferences \(where empathetic defaults happen to match ground truth\) and stereotypical preferences \(where demographic priors occasionally align\)\. Anti\-stereotypical preferences yield the lowest zero\-memory pass rate \(1\.2% for GPT\-4o\-mini\), validating that counter\-stereotypical preferences are virtually impossible to guess\.

##### Why only Know test\.

The Act judge evaluates whether a memory\-augmented response ismore preference\-aware than the zero\-memory response\(Section[B\.4](https://arxiv.org/html/2607.29433#A2.SS4)\)\. The zero\-memory response itselfisthe Act baseline—it is one of the two inputs to the Act judge, not something to be judged independently\. Evaluating the zero\-memory response with the Act judge would be comparing it against itself, which is trivially uninformative\.

##### Why no expression type breakdown\.

In the zero\-memory condition, the model never receives any conversation chunks\. Since the three expression types \(Explicit, Incidental, Inferential\) differ only in how the target preference is embedded within the injected chunks, they are indistinguishable when no chunks are injected\. The zero\-memory Know test is therefore expression\-type\-agnostic: the same 1,000 questions yield the same results regardless of which expression variant would have been used in the memory\-augmented condition\.

Beyond structural validation, we manually inspect 100 randomly sampled Act scenarios to assess quality along two dimensions: \(1\) whether the scenario allows the target preference to be inferred from context without being explicitly mentioned, and \(2\) whether a preference\-aware response would be meaningfully distinguishable from a preference\-unaware response\.

## Appendix EJudge Reliability

### E\.1Human\-LLM Agreement

We sample 100 Know judgments and 100 Act judgments for human evaluation\. Three annotators independently label each judgment\. Cohen’s kappa between human majority vote and GPT\-5 judge: Know 0\.91, Act 0\.84, indicating strong agreement\.

### E\.2Inter\-Judge Agreement

We evaluate a second judge model \(Claude Opus 4\.6\) on 200 randomly sampled judgments \(100 Know, 100 Act\)\. Inter\-judge agreement between GPT\-5 and Claude Opus: Know 0\.87, Act 0\.81\.

## Appendix FDetailed Results by Preference Type

This section presents detailedKnow Accuracy,Act Accuracy, andUtilization Ratemetrics broken down by preference type under the three expression conditions \(E = Explicit, In = Incidental, If = Inferential\)\. Avg = macro\-average over E/In/If\. Best results in each column arebolded, with Long\-Context systems bolded separately from retrieval\-based systems\.

Table 13:Know/Act/Util for Anti\-Stereotypical preferences\.Know Acc\.\(%\)Act Acc\.\(%\)Util\.Rate\(%\)TypeSystemEInIfAvgEInIfAvgEInIfAvgLong\-ContextGPT\-4o\-mini84\.53\.64\.030\.722\.612\.719\.818\.423\.00\.010\.011\.0GPT\-4o99\.238\.162\.366\.543\.718\.723\.028\.444\.025\.028\.032\.3Gemini 3\.1 Flash99\.655\.679\.478\.277\.835\.343\.752\.277\.745\.045\.556\.1Claude 4\.6 Sonnet99\.249\.681\.376\.787\.338\.957\.161\.187\.250\.463\.467\.0Simple RAGBM2584\.114\.39\.536\.021\.412\.322\.218\.722\.211\.120\.818\.0Embedding RAGRAG te3\-small98\.815\.935\.350\.056\.713\.528\.232\.857\.427\.537\.140\.7RAG te3\-large98\.419\.037\.351\.656\.713\.929\.833\.556\.918\.838\.338\.0RAG Qwen3\-Emb\-4B99\.217\.147\.254\.562\.315\.126\.634\.762\.816\.332\.837\.3Structure\-Aug\. RAGMem097\.225\.847\.656\.979\.828\.245\.251\.181\.252\.357\.563\.7Zep/Graphiti54\.81\.60\.819\.015\.910\.712\.713\.117\.40\.00\.05\.8Cognee90\.511\.519\.240\.442\.515\.524\.927\.641\.734\.529\.535\.2HippoRAG\-v298\.015\.524\.646\.045\.216\.724\.628\.845\.325\.625\.832\.3Agentic MemoryLetta/MemGPT86\.913\.518\.339\.640\.917\.121\.826\.641\.114\.737\.030\.9Self\-RAG96\.024\.259\.159\.875\.48\.331\.738\.575\.216\.437\.643\.1MemoryOS96\.88\.332\.545\.957\.515\.127\.433\.359\.014\.347\.640\.3A\-MEM88\.19\.514\.737\.428\.218\.321\.022\.527\.925\.029\.727\.6Table 14:Know/Act/Util for Health & Medical preferences\.Know Acc\.\(%\)Act Acc\.\(%\)Util\.Rate\(%\)TypeSystemEInIfAvgEInIfAvgEInIfAvgLong\-ContextGPT\-4o\-mini85\.87\.412\.335\.210\.56\.29\.38\.610\.10\.05\.05\.0GPT\-4o100\.020\.452\.557\.618\.58\.618\.515\.218\.59\.118\.815\.5Gemini 3\.1 Flash98\.825\.372\.265\.475\.331\.561\.756\.275\.046\.364\.161\.8Claude 4\.6 Sonnet98\.821\.672\.264\.287\.737\.768\.564\.687\.557\.169\.271\.3Simple RAGBM2590\.110\.510\.537\.09\.93\.714\.89\.511\.00\.041\.217\.4Embedding RAGRAG te3\-small97\.59\.935\.247\.531\.54\.917\.918\.131\.00\.021\.117\.4RAG te3\-large96\.39\.934\.046\.733\.33\.715\.417\.532\.76\.214\.517\.8RAG Qwen3\-Emb\-4B96\.912\.346\.952\.140\.75\.617\.321\.241\.40\.022\.421\.3Structure\-Aug\. RAGMem096\.911\.132\.746\.957\.417\.931\.535\.658\.044\.432\.144\.8Zep/Graphiti44\.40\.63\.116\.05\.68\.03\.75\.86\.90\.00\.02\.3Cognee94\.49\.928\.244\.214\.86\.814\.111\.915\.06\.212\.511\.3HippoRAG\-v298\.810\.529\.646\.327\.25\.613\.615\.426\.95\.910\.414\.4Agentic MemoryLetta/MemGPT92\.68\.625\.342\.219\.86\.214\.213\.420\.00\.014\.611\.5Self\-RAG94\.47\.430\.944\.263\.04\.919\.829\.261\.40\.028\.029\.8MemoryOS85\.22\.517\.935\.234\.67\.417\.920\.036\.20\.034\.523\.6A\-MEM92\.08\.624\.141\.615\.45\.69\.910\.316\.80\.010\.39\.0Table 15:Know/Act/Util for Neutral preferences\.Know Acc\.\(%\)Act Acc\.\(%\)Util\.Rate\(%\)TypeSystemEInIfAvgEInIfAvgEInIfAvgLong\-ContextGPT\-4o\-mini80\.28\.29\.332\.623\.720\.221\.421\.826\.214\.325\.021\.8GPT\-4o98\.141\.267\.368\.939\.328\.833\.133\.739\.329\.234\.734\.4Gemini 3\.1 Flash99\.667\.387\.284\.765\.435\.447\.949\.565\.241\.048\.751\.6Claude 4\.6 Sonnet99\.656\.484\.080\.077\.837\.054\.156\.377\.738\.655\.157\.1Simple RAGBM2584\.813\.610\.536\.321\.916\.322\.220\.122\.65\.77\.411\.9Embedding RAGRAG te3\-small99\.217\.540\.552\.445\.917\.530\.731\.446\.324\.430\.833\.8RAG te3\-large98\.417\.143\.653\.045\.519\.531\.932\.346\.227\.337\.537\.0RAG Qwen3\-Emb\-4B99\.619\.152\.957\.254\.517\.533\.135\.054\.322\.436\.837\.8Structure\-Aug\. RAGMem099\.635\.061\.965\.574\.730\.044\.049\.574\.652\.248\.458\.4Zep/Graphiti61\.11\.91\.621\.514\.415\.216\.715\.414\.00\.025\.013\.0Cognee89\.115\.621\.942\.235\.821\.033\.030\.036\.217\.543\.132\.3HippoRAG\-v298\.112\.827\.646\.235\.818\.733\.529\.336\.118\.238\.030\.8Agentic MemoryLetta/MemGPT91\.413\.222\.242\.335\.819\.530\.728\.734\.026\.535\.131\.9Self\-RAG96\.927\.266\.963\.761\.112\.526\.133\.261\.010\.028\.533\.2MemoryOS96\.510\.541\.649\.542\.019\.532\.331\.342\.733\.338\.338\.1A\-MEM88\.712\.118\.739\.830\.023\.326\.826\.728\.929\.031\.229\.7Table 16:Know/Act/Util for Stereotypical preferences\.Know Acc\.\(%\)Act Acc\.\(%\)Util\.Rate\(%\)TypeSystemEInIfAvgEInIfAvgEInIfAvgLong\-ContextGPT\-4o\-mini86\.510\.013\.136\.521\.416\.618\.318\.819\.717\.423\.320\.1GPT\-4o99\.144\.159\.467\.547\.230\.631\.436\.447\.634\.733\.138\.4Gemini 3\.1 Flash99\.668\.683\.483\.879\.548\.955\.961\.479\.857\.357\.664\.9Claude 4\.6 Sonnet99\.657\.682\.579\.980\.850\.257\.262\.781\.156\.860\.866\.3Simple RAGBM2589\.514\.915\.740\.122\.712\.714\.016\.522\.417\.622\.220\.8Embedding RAGRAG te3\-small98\.322\.338\.453\.049\.814\.022\.328\.749\.815\.730\.732\.0RAG te3\-large99\.125\.340\.655\.047\.213\.121\.027\.147\.613\.826\.929\.4RAG Qwen3\-Emb\-4B99\.121\.450\.256\.961\.112\.723\.632\.561\.718\.429\.636\.5Structure\-Aug\. RAGMem098\.338\.456\.864\.579\.530\.639\.749\.980\.042\.043\.155\.0Zep/Graphiti57\.61\.72\.220\.57\.99\.211\.89\.67\.650\.020\.025\.9Cognee91\.317\.524\.844\.532\.815\.321\.823\.334\.920\.021\.625\.5HippoRAG\-v298\.717\.923\.146\.636\.217\.021\.424\.936\.722\.024\.527\.7Agentic MemoryLetta/MemGPT91\.314\.420\.141\.932\.314\.019\.722\.033\.515\.226\.124\.9Self\-RAG93\.931\.453\.359\.555\.59\.224\.929\.858\.18\.332\.833\.1MemoryOS95\.216\.631\.047\.649\.810\.021\.827\.250\.010\.531\.030\.5A\-MEM94\.318\.821\.044\.729\.714\.424\.522\.930\.611\.625\.022\.4Table 17:Know/Act/Util for Therapy & Emotional preferences\.Know Acc\.\(%\)Act Acc\.\(%\)Util\.Rate\(%\)TypeSystemEInIfAvgEInIfAvgEInIfAvgLong\-ContextGPT\-4o\-mini81\.026\.029\.045\.314\.012\.014\.013\.313\.615\.413\.814\.3GPT\-4o99\.036\.056\.063\.728\.022\.029\.026\.328\.336\.130\.431\.6Gemini 3\.1 Flash100\.050\.077\.075\.751\.033\.047\.043\.751\.042\.048\.147\.0Claude 4\.6 Sonnet99\.053\.077\.076\.375\.058\.070\.067\.774\.766\.072\.771\.2Simple RAGBM2586\.046\.033\.055\.015\.012\.014\.013\.717\.415\.218\.216\.9Embedding RAGRAG te3\-small93\.032\.043\.056\.026\.013\.020\.019\.726\.912\.532\.624\.0RAG te3\-large92\.034\.043\.056\.323\.011\.023\.019\.023\.923\.532\.626\.7RAG Qwen3\-Emb\-4B97\.045\.056\.066\.028\.010\.024\.020\.728\.911\.132\.124\.0Structure\-Aug\. RAGMem094\.048\.061\.067\.744\.023\.030\.032\.344\.731\.237\.737\.9Zep/Graphiti30\.09\.09\.016\.06\.08\.09\.07\.713\.333\.322\.223\.0Cognee84\.031\.031\.148\.723\.012\.021\.118\.725\.012\.939\.325\.7HippoRAG\-v295\.032\.032\.053\.018\.012\.021\.017\.017\.925\.040\.627\.8Agentic MemoryLetta/MemGPT82\.028\.031\.047\.024\.010\.020\.018\.026\.814\.332\.324\.5Self\-RAG83\.035\.045\.054\.338\.010\.036\.028\.038\.65\.751\.131\.8MemoryOS87\.027\.034\.049\.326\.09\.022\.019\.027\.611\.123\.520\.7A\-MEM83\.036\.029\.049\.317\.016\.018\.017\.019\.325\.024\.122\.8
## Appendix GSummary Metrics by Preference Type

Tables[13](https://arxiv.org/html/2607.29433#A6.T13)–[17](https://arxiv.org/html/2607.29433#A6.T17)present detailedKnow Accuracy,Act Accuracy, andUtilization Ratemetrics under the three expression conditions \(E = Explicit, In = Incidental, If = Inferential\) for each of the five preference types\. Key observations:

- •Anti\-stereotypical: Lowest zero\-memory pass rate \(1\.2%\), making it the most reliable diagnostic category\. Mem0 achieves the highestUtilization Rateamong retrieval\-based systems \(63\.7% Avg\), while Claude 4\.6 Sonnet leads overall \(67\.0% Avg\)\.
- •Health/Medical: LowestUtilization Ratesacross all systems, indicating severe utilization bottlenecks for safety\-sensitive information\. Even Claude 4\.6 Sonnet’s best\-case shows only 71\.3% AvgUtilization Rate\.
- •Neutral: Moderate performance across all metrics\. Mem0 achieves 99\.6%Knowunder Explicit but only 74\.7%Act, with a 58\.4% AvgUtilization Rateamong retrieval\-based systems\.
- •Stereotypical: Shows the strongest performance in the Incidental condition due to alignment with PersonaMem\-v2’s original conversational patterns\.
- •Therapy/Emotional: Highest zero\-memory pass rate \(9\.0%\), inflatingActscores through prior\-based alignment\. Claude 4\.6 Sonnet achieves the highestUtilization Rate\(71\.2% Avg\), with particularly strong Incidental performance \(66\.0%\)\.

## Appendix HLLM Usage Disclosure

We disclose all LLM usage in this work\. GPT\-5 is used for: \(1\) generating Explicit and Inferential conversation snippets \(Section[3\.1](https://arxiv.org/html/2607.29433#S3.SS1)\), \(2\) generating paired Know and Act test items \(Section[3\.1](https://arxiv.org/html/2607.29433#S3.SS1)\), and \(3\) serving as the LLM judge for bothKnowandActevaluation \(Section[4\.1](https://arxiv.org/html/2607.29433#S4.SS1)\)\. All prompts, hyperparameters, and validation procedures are documented in Appendix[B](https://arxiv.org/html/2607.29433#A2)\.