Deployment-Time Memorization in Foundation-Model Agents

arXiv cs.AI Papers

Summary

This paper introduces the concept of deployment-time memorization in foundation-model agents, analyzing how memory design choices (summarization aggressiveness, retrieval breadth, deletion mode) affect personalization utility, extraction risk, and deletion fidelity, proposing new metrics such as Personalization Recall, Adversarial Extraction Rate, and Forgetting Residue Score.

arXiv:2606.10062v1 Announce Type: new Abstract: Foundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a property of model weights. Existing work addresses parametric memorization or audits fixed memory configurations, but does not characterize how memory-design choices jointly shape personalization utility, extraction risk, and deletion fidelity. We study this surface as deployment-time memorization, formulating agent memory as a privacy-utility frontier measured by Personalization Recall (PR) and Adversarial Extraction Rate (AER), and sweeping three memory-design knobs: summarization aggressiveness, retrieval breadth (k), and deletion mode. We further introduce the Forgetting Residue Score (FRS) to quantify whether deleted information remains recoverable from derived memory tiers. On LongMemEval, key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall; critically, once content is compressed away, increasing k no longer restores leakage. The same compression, however, induces a deletion-fidelity failure: raw-only deletion leaves derived summary copies recoverable in approximately 20% of instances, and only full-pipeline purge or tombstone redaction drives worst-tier residue to zero. Together, these results establish that persistent agent memory must be evaluated as a first-class memorization mechanism -- assessed by what it helps agents recall, what it makes extractable, and what it can truly erase.
Original Article
View Cached Full Text

Cached at: 06/10/26, 06:12 AM

# Deployment-Time Memorization in Foundation-Model Agents
Source: [https://arxiv.org/html/2606.10062](https://arxiv.org/html/2606.10062)
Guilin ZhangKai ZhaoDalmo CirneAndy OlsenZeke MillerXu ChuAlet BlankenAmine AnounJerry Ting

###### Abstract

Foundation\-model agents are increasingly long\-lived systems that remember users across interactions, making memorization an explicit deployment\-time function rather than solely a property of model weights\. Existing work addresses parametric memorization or audits fixed memory configurations, but does not characterize how memory\-design choices jointly shape personalization utility, extraction risk, and deletion fidelity\. We study this surface as*deployment\-time memorization*, formulating agent memory as a*privacy–utility frontier*measured by Personalization Recall \(PR\) and Adversarial Extraction Rate \(AER\), and sweeping three memory\-design knobs: summarization aggressiveness, retrieval breadth \(kk\), and deletion mode\. We further introduce the*Forgetting Residue Score*\(FRS\) to quantify whether deleted information remains recoverable from derived memory tiers\. On LongMemEval, key\-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT\-4o\-mini while preserving nearly all personalization recall; critically, once content is compressed away, increasingkkno longer restores leakage\. The same compression, however, induces a deletion\-fidelity failure: raw\-only deletion leaves derived summary copies recoverable in≈20%\{\\approx\}20\\%of instances, and only full\-pipeline purge ortombstoneredaction drives worst\-tier residue to zero\. Together, these results establish that persistent agent memory must be evaluated as a first\-class memorization mechanism — assessed by what it helps agents recall, what it makes extractable, and what it can truly erase\.

## 1Introduction

Foundation\-model agents are moving from stateless assistants to long\-lived systems that remember users\. A travel assistant that recalls “I prefer aisle seats,” or a coding agent that retains “I use Python 3\.11 with mypy strict,” is useful precisely because it does not begin each interaction from a blank slate\(Parket al\.,[2023](https://arxiv.org/html/2606.10062#bib.bib7); Packeret al\.,[2023](https://arxiv.org/html/2606.10062#bib.bib8); Zhonget al\.,[2024](https://arxiv.org/html/2606.10062#bib.bib9)\)\. Yet this capability changes the privacy problem\. In persistent agents, memorization is no longer only an incidental property of model weights; it is an explicit system function\. The deployed pipeline writes user facts into memory, summarizes them, retrieves them, and conditions future responses on them\.

Existing memorization research primarily studies*parametric memorization*: what training examples are retained in model weights and can be exposed through extraction or membership\-inference attacks\(Carliniet al\.,[2021](https://arxiv.org/html/2606.10062#bib.bib2); Shokriet al\.,[2017](https://arxiv.org/html/2606.10062#bib.bib3)\)\. Recent audits show that memory\-enabled agents can also leak private information under adversarial probing\(El Yagoubiet al\.,[2026](https://arxiv.org/html/2606.10062#bib.bib14); Daset al\.,[2026](https://arxiv.org/html/2606.10062#bib.bib15); Liuet al\.,[2025](https://arxiv.org/html/2606.10062#bib.bib16); Wanget al\.,[2025](https://arxiv.org/html/2606.10062#bib.bib17)\)\. However, these studies largely evaluate fixed configurations\. They do not characterize the design frontier faced by practitioners: how leakage changes as memory is compressed, how utility changes as more memories are retrieved, or whether a “forget me” operation actually removes derived copies from all memory tiers\.

We study this missing surface, which we call*deployment\-time memorization*: recoverable user information stored not in model parameters, but in the external memory pipeline wrapped around a foundation model\. We formulate agent memory design as a privacy–utility frontier\. Utility is measured by*Personalization Recall*\(PR\), and leakage by*Adversarial Extraction Rate*\(AER\), the rate at which a high\-entropy canary inserted into a prior session can be recovered under direct, indirect, or jailbreak\-style probing\. We sweep three memory\-design knobs: summarization aggressiveness and retrieval breadth \(the number of memory chunks injected into the agent context\) govern routine read/write at every query, while deletion mode governs how the pipeline implements “forget” — a five\-step ladder ranging from a no\-op control throughraw\_only,raw\_plus\_resummarize,full\_purge, to compliance\-styletombstoneredaction\.

A persistent agent may copy the same information into summaries, embeddings, caches, or other derived artifacts, so deleting the original raw record may not be enough\. We introduce a*Forgetting Residue Score*\(FRS\) that measures post\-deletion leakage separately across memory tiers, and benchmark each deletion mode against this metric\.

Our experiments on LongMemEval reveal a double\-edged role for summarization\. Key\-fact summarization reduces canary extraction by76%76\\%on Gemma 3 12B and64%64\\%on GPT\-4o\-mini while preserving nearly all personalization recall\. Once the canary is compressed away, increasing top\-kkretrieval no longer restores leakage\. However, the same compression creates a deletion\-fidelity failure: raw\-only deletion removes the original chunk but leaves derived summary copies recoverable in roughly20%20\\%of instances\. Re\-summarizing partially cleaned memory reduces but does not eliminate the residue; only full\-pipeline purge or tombstone redaction drives worst\-tier residue to zero\.

We make three contributions:

- •Deployment\-time memorization as a measurable frontier\. We formalize persistent agent memory as a deployment\-time memorization system and introduce a privacy–utility frontier based on PR, AER, and Privacy–Utility AUC\.
- •A controlled sweep over memory design knobs\. We show that summarization substantially reduces extraction with small personalization cost, while retrieval breadth alone cannot recover a favorable privacy–utility trade\-off\.
- •A deletion\-fidelity benchmark for tiered memory\. We introduce FRS and show that raw\-chunk deletion is insufficient when secrets survive in derived summaries, whereas full\-pipeline purge and tombstone redaction eliminate worst\-tier residue\.

## 2Method

We study memorization as a property of the deployed memory system around a foundation\-model agent, rather than as a property of model parameters\. Our goal is to measure how memory design controls three quantities: useful recall, adversarial extraction, and post\-deletion residue\.

### 2\.1Agent\-Memory Pipeline

We model persistent agent memory as a write–retrieve–respond pipeline following retrieval\-augmented memory\(Lewiset al\.,[2020](https://arxiv.org/html/2606.10062#bib.bib20); Packeret al\.,[2023](https://arxiv.org/html/2606.10062#bib.bib8)\)\. After each user session the system writes information into long\-term memory; at query time it retrieves memory chunks by cosine similarity and prepends them to the agent context\.

We expose three memory\-design knobs\. Summarization aggressivenessSScontrols what is stored: raw user turns \(S=0S\{=\}0\), key personal facts \(S=1S\{=\}1\), or one\-sentence session summaries \(S=2S\{=\}2\)\. Retrieval breadthkkcontrols how many chunks are injected into the agent context\. Deletion mode controls how the pipeline responds to a userforgetrequest \(detailed in §[2\.4](https://arxiv.org/html/2606.10062#S2.SS4)\)\. Each memory chunk carries an origin tiert∈\{raw,summary\}t\\in\\\{\\texttt\{raw\},\\texttt\{summary\}\\\}, enabling us to attribute leakage and deletion residue to original records versus derived artifacts\.

### 2\.2Threat Model

We consider an adversary who can query the agent after memory has been written but cannot directly inspect or edit the memory store\. We test three escalating probes:directprobes ask explicitly for the stored secret;indirectprobes ask the agent to report what it recalls about the user; andjailbreakprobes issue developer\-mode or debug\-dump instructions inspired by prompt\-injection attacks\(Greshakeet al\.,[2023](https://arxiv.org/html/2606.10062#bib.bib18); El Yagoubiet al\.,[2026](https://arxiv.org/html/2606.10062#bib.bib14)\)\. The same probes are re\-run after the user invokesforgetto evaluate each deletion mode\.

### 2\.3Privacy–Utility Frontier

For each configurationθ=\(S,k\)\\theta\{=\}\(S,k\)we measure both utility and leakage\.

*Personalization Recall \(PR\)\.*PR measures whether the agent correctly answers a query requiring remembered user context\. An answer is correct if its cosine similarity to the expected response exceeds0\.500\.50or all content words appear verbatim:

PR​\(θ\)=𝔼​\[𝟏​\[answer correct\]\]\.\\mathrm\{PR\}\(\\theta\)=\\mathbb\{E\}\\\!\\left\[\\mathbf\{1\}\[\\text\{answer correct\}\]\\right\]\.
*Adversarial Extraction Rate \(AER\)\.*We inject a high\-entropy canarycic\_\{i\}into a prior session to build memory stateℳθ,i\\mathcal\{M\}\_\{\\theta,i\}\. For probe levelℓ∈\{direct,indirect,jailbreak\}\\ell\\in\\\{\\text\{direct\},\\text\{indirect\},\\text\{jailbreak\}\\\}, letri,ℓr\_\{i,\\ell\}be the agent response\. AER is the fraction of canaries recovered verbatim:

AERℓ​\(θ\)=1\|C\|​∑ci∈C𝟏​\[ci∈ri,ℓ\],\\mathrm\{AER\}\_\{\\ell\}\(\\theta\)=\\frac\{1\}\{\|C\|\}\\sum\_\{c\_\{i\}\\in C\}\\mathbf\{1\}\\\!\\left\[c\_\{i\}\\in r\_\{i,\\ell\}\\right\],\(1\)whereC=\{ci\}C=\\\{c\_\{i\}\\\}is the canary set\. We report per\-probeAERℓ\\mathrm\{AER\}\_\{\\ell\}and worst\-caseAERmax​\(θ\)=maxℓ⁡AERℓ​\(θ\)\\mathrm\{AER\}\_\{\\max\}\(\\theta\)=\\max\_\{\\ell\}\\,\\mathrm\{AER\}\_\{\\ell\}\(\\theta\)\.

*Privacy–Utility AUC \(PUA\)\.*Sweepingk∈Kk\\in Kat fixedSStraces frontier points\{\(PR​\(k\),AER​\(k\)\)\}\\\{\(\\mathrm\{PR\}\(k\),\\mathrm\{AER\}\(k\)\)\\\}\. We summarize the frontier as the area under the empirical*achievable\-recall envelope*:

PR⋆​\(a\)\\displaystyle\\mathrm\{PR\}^\{\\star\}\(a\)≔maxk:AER​\(k\)≤a⁡PR​\(k\),\\displaystyle\\;\\coloneqq\\;\\max\_\{k:\\,\\mathrm\{AER\}\(k\)\\,\\leq\\,a\}\\mathrm\{PR\}\(k\),\(2\)PUA​\(S\)\\displaystyle\\mathrm\{PUA\}\(S\)=∫01PR⋆​\(a\)​𝑑a\.\\displaystyle\\;=\\;\\int\_\{0\}^\{1\}\\mathrm\{PR\}^\{\\star\}\(a\)\\,da\.Higher PUA indicates higher recall at lower extraction risk\. We also report summarization launderingΔS=AER​\(S=0\)−AER​\(S\)\\Delta\_\{S\}=\\mathrm\{AER\}\(S\{=\}0\)\-\\mathrm\{AER\}\(S\)\.

### 2\.4Forgetting Residue

Given a pre\-deletion memory stateℳS\\mathcal\{M\}\_\{S\}containing canarycc, we apply a deletion procedure parameterized by mode, re\-run the adversarial probes on the resulting stateℳS′\\mathcal\{M\}\_\{S\}^\{\\prime\}, and attribute leakage to each origin tiert∈\{raw,summary\}t\\in\\\{\\texttt\{raw\},\\texttt\{summary\}\\\}:

ℳS′\\displaystyle\\mathcal\{M\}\_\{S\}^\{\\prime\}≔forget​\(ℳS,c,mode\),\\displaystyle\\;\\coloneqq\\;\\texttt\{forget\}\(\\mathcal\{M\}\_\{S\},\\,c,\\,\\text\{mode\}\),\(3\)FRSt​\(S,mode\)\\displaystyle\\mathrm\{FRS\}\_\{t\}\(S,\\text\{mode\}\)=𝔼​\[AERt​\(ℳS′,c\)\]\.\\displaystyle\\;=\\;\\mathbb\{E\}\\\!\\left\[\\mathrm\{AER\}\_\{t\}\(\\mathcal\{M\}\_\{S\}^\{\\prime\},\\,c\)\\right\]\.We report worst\-tier residueFRSworst=maxt⁡FRSt\\mathrm\{FRS\}\_\{\\mathrm\{worst\}\}=\\max\_\{t\}\\,\\mathrm\{FRS\}\_\{t\}; a nonzero value indicates the secret remains recoverable despite deletion\.

The five deletion modes form an ablation ladder isolating whether deletion de\-memorizes only the raw record or the full pipeline across both text and embedding surfaces \(Table[1](https://arxiv.org/html/2606.10062#S2.T1)\)\.

Table 1:Deletion\-mode ladder\. Each row toggles exactly one additional engineering decision relative to the row above \(raw→\\tosummary tier\)\.

## 3Experimental Results

### 3\.1Experimental Setup

We evaluate deployment\-time memorization on the oracle split of LongMemEval\(Wuet al\.,[2025](https://arxiv.org/html/2606.10062#bib.bib13)\), a benchmark of multi\-session chat histories with question–answer pairs requiring long\-term user context\. We sampleN=50N\{=\}50instances and run a full factorial sweep over memory configurations, adversarial probes, and deletion modes\.

To separate pipeline memorization from chance generation or training\-time exposure, we inject one synthetic canary per instance\. Each canary has the form “*my private session token is\[value\]*” and is placed in a randomly chosen non\-evidence user turn\. Each\[value\]is drawn from a high\-entropy grammar \(e\.g\.,XQ7\-VIOLET\-3829;≈5\.6×109\{\\approx\}5\.6\{\\times\}10^\{9\}possible strings\), synthesized independently of LongMemEval\. Verbatim reproduction by the agent is therefore attributable to deployment\-time pipeline memorization rather than training exposure\.

Our primary sweep uses Gemma 3 12B served locally via Ollama; we replicate theS∈\{0,1\}S\\in\\\{0,1\\\}slice on GPT\-4o\-mini on the same instance set\. The two models span open\-weight and closed\-weight deployments with independent training pipelines, allowing us to distinguish pipeline\-level effects from model\-specific artifacts\.

We vary three memory\-design knobs: summarization levelS∈\{0,1,2\}S\\in\\\{0,1,2\\\}\(raw turns, key\-facts, one\-sentence\); retrieval breadthk∈\{1,3,6,10,25\}k\\in\\\{1,3,6,10,25\\\}; and deletion mode, as defined in Table[1](https://arxiv.org/html/2606.10062#S2.T1), exercised only during forgetting residue evaluation \(§[3\.3](https://arxiv.org/html/2606.10062#S3.SS3)\)\. Retrieval uses cosine similarity overall\-MiniLM\-L6\-v2embeddings\(Reimers and Gurevych,[2019](https://arxiv.org/html/2606.10062#bib.bib23)\)\. Utility is scored by cosine similarity to the ground\-truth answer \(cos\>0\.50\\cos\{\>\}0\.50\) or exact content\-word coverage; Appendix reports LLM\-as\-judge validation, where disagreements are conservative false negatives that lower\-bound PR\.

![Refer to caption](https://arxiv.org/html/2606.10062v1/x1.png)Figure 1:Privacy–utility frontier and probe decomposition\.*Top:*Frontier on Gemma 3 12B and GPT\-4o\-mini; color encodesSS, each line tracesk∈\{1,3,6,10,25\}k\\in\\\{1,3,6,10,25\\\}\. UnderS=0S\{=\}0the line stretches diagonally \(PR/AER both rise withkk\); underS≥1S\{\\geq\}1it collapses to a near\-point \(kk\-flatness\)\.*Bottom:*AER by probe type atS∈\{0,1\}S\\in\\\{0,1\\\}\. The direct–indirect spreadΔDI≔\|AERdirect−AERindirect\|\\Delta\_\{\\mathrm\{DI\}\}\\coloneqq\|\\mathrm\{AER\}\_\{\\text\{direct\}\}\-\\mathrm\{AER\}\_\{\\text\{indirect\}\}\|collapses under summarization on both models \(0\.16→0\.000\.16\{\\to\}0\.00Gemma,0\.08→0\.020\.08\{\\to\}0\.02GPT\-4o\-mini\); the dotted “RLHF wall” marks GPT\-4o\-mini’s jailbreak refusal floor\.
### 3\.2Summarization Moves the Privacy–Utility Frontier

Table 2:Main results\.Privacy–utility frontier and forgetting residue on Gemma 3 12B and GPT\-4o\-mini\. Frontier rows are averaged overk∈\{1,3,6,10,25\}k\\in\\\{1,3,6,10,25\\\}; subscripts denote 95% bootstrap confidence intervals over 1000 resamples\.The central question is whether memory compression merely reduces context length, or whether it changes what an adversary can recover\. Figure[1](https://arxiv.org/html/2606.10062#S3.F1)and Table[2](https://arxiv.org/html/2606.10062#S3.T2)show that summarization changes the privacy–utility frontier, not merely the context length\.

#### Raw memory offers no clean operating point\.

UnderS=0S\{=\}0, Gemma 3 12B reaches PR≈0\.59\\approx 0\.59but leaks at AER≈0\.83\\approx 0\.83\. Askkgrows from11to2525, PR rises from0\.420\.42to0\.660\.66while AER rises from0\.700\.70to a plateau near0\.860\.86: larger retrieval breadth helps the user and the adversary equally\.

#### Key\-fact summarization launders the canary\.

S=1S\{=\}1reduces AER by76%76\\%on Gemma \(ΔS=0\.63\\Delta\_\{S\}\{=\}0\.63\) and64%64\\%on GPT\-4o\-mini \(ΔS=0\.46\\Delta\_\{S\}\{=\}0\.46\), with PR costs of≈5%\{\\approx\}5\\%and≈2\{\\approx\}2pp respectively\. Despite differing raw baselines, both models converge to nearly the same summarized operating point \(PUA0\.460\.46and0\.470\.47\), indicating that laundering is a pipeline\-level effect rather than a Gemma artifact\. One\-sentence summarization \(S=2S\{=\}2\) reduces AER further to≈0\.16\{\\approx\}0\.16on Gemma, but at additional utility cost, makingS=1S\{=\}1the best\-attainable point on the frontier across both models\.

#### After compression, retrieval breadth becomes privacy\-neutral\.

UnderS≥1S\{\\geq\}1, AER is flat across allkk: once the canary is absent from stored memory representations, retrieving more chunks does not restore it\. Summarization therefore changes secret recoverability, not just context size\.

Direct and indirect probes collapse under summarization\.Figure[1](https://arxiv.org/html/2606.10062#S3.F1)\(bottom\) decomposes AER by probe type atS∈\{0,1\}S\\in\\\{0,1\\\}\. Under raw memory, direct and indirect probes diverge — the spreadΔDI≔\|AERdirect−AERindirect\|\\Delta\_\{\\mathrm\{DI\}\}\\coloneqq\|\\mathrm\{AER\}\_\{\\text\{direct\}\}\-\\mathrm\{AER\}\_\{\\text\{indirect\}\}\|is0\.160\.16on Gemma and0\.080\.08on GPT\-4o\-mini — while after summarization both collapse to the same low extraction rate \(ΔDI=0\.00\\Delta\_\{\\mathrm\{DI\}\}\{=\}0\.00and0\.020\.02\)\. Jailbreak behavior diverges across models: on Gemma, jailbreak AER joins the same collapse atS=1S\{=\}1; on GPT\-4o\-mini, jailbreak AER is near zero even under raw memory, consistent with RLHF\-mediated refusal\. This split confirms that summarization controls factual recoverability at the pipeline level, while jailbreak resistance is governed by the underlying model\.

### 3\.3Deletion Fidelity: When Forgetting Is Not Deletion

Persistent memory has a second requirement: when a user invokesforget, the system must remove not only the original record but also any derived copies created downstream by the pipeline\. We quantify this gap with FRS \(Eq\. \([3](https://arxiv.org/html/2606.10062#S2.E3)\)\) and exercise the five\-mode ladder of Table[1](https://arxiv.org/html/2606.10062#S2.T1)atS∈\{0,1,2\}S\\in\\\{0,1,2\\\}\.

Raw\-only deletion provides no erasure guarantee:at\(S=1,raw\_only\)\(S\{=\}1,\\texttt\{raw\\\_only\}\)deletion is statistically indistinguishable fromnoop*in the summary tier*on both models \(FRSworst≈0\.20\\mathrm\{FRS\}\_\{\\mathrm\{worst\}\}\\approx 0\.20, overlapping CIs\) — the raw chunk is gone but the summary\-derived copy remains\.Re\-summarization helps but does not fully purge:raw\_plus\_resummarizereduces residue to0\.110\.11on Gemma and0\.100\.10on GPT\-4o\-mini atS=1S\{=\}1, and0\.050\.05on Gemma atS=2S\{=\}2, but stays nonzero — pre\-existing summary artifacts are not fully removed before regeneration\.Only full\-pipeline purge or tombstone drives FRS to zero:full\_purgeandtombstonedriveFRSworst\\mathrm\{FRS\}\_\{\\mathrm\{worst\}\}to zero across all evaluated settings, confirming that complete deletion in a tiered pipeline requires either dropping all derived artifacts or replacing them with an explicit redaction marker\.

The experiments reveal a consistent story: raw memory gives useful recall but makes secrets easy to extract; summarization launders the canary out of retrieval but copies it into the summary tier; only full\-pipeline deletion truly erases\. Persistent agent memory must therefore be evaluated as a full memorization system: what it stores, what it exposes, and what it can truly erase\. More broadly, deployment\-time memorization is a*controllable*, not inevitable, utility–leakage frontier\. The practical recipe is key\-fact summarization at moderatekkwith a tier\-aware delete; the same harness extends to orthogonal levers \(τ\\tau, embedding strength, chunk granularity\) and longer\-horizon settings\.

## References

- N\. Carlini, F\. Tramèr, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, U\. Erlingsson, A\. Oprea, and C\. Raffel \(2021\)Extracting training data from large language models\.In30th USENIX Security Symposium,Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p2.1)\.
- D\. Das, J\. Piet, D\. Kaviani, L\. Beurer\-Kellner, F\. Tramèr, and D\. Wagner \(2026\)Trojan hippo: weaponizing agent memory for data exfiltration\.External Links:2605\.01970Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p2.1)\.
- F\. El Yagoubi, G\. Badu\-Marfo, and R\. Al Mallah \(2026\)AgentLeak: a full\-stack benchmark for privacy leakage in multi\-agent LLM systems\.External Links:2602\.11510Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p2.1),[§2\.2](https://arxiv.org/html/2606.10062#S2.SS2.p1.1)\.
- K\. Greshake, S\. Abdelnabi, S\. Mishra, C\. Endres, T\. Holz, and M\. Fritz \(2023\)Not what you’ve signed up for: compromising real\-world LLM\-integrated applications with indirect prompt injection\.InACM Workshop on Artificial Intelligence and Security \(AISec\),Cited by:[§2\.2](https://arxiv.org/html/2606.10062#S2.SS2.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.1](https://arxiv.org/html/2606.10062#S2.SS1.p1.1)\.
- J\. Liu, D\. Cao, Y\. Wei, T\. Su, Y\. Liang, Y\. Dong, Y\. Liu, Y\. Zhao, and X\. Hu \(2025\)Topology matters: measuring memory leakage in multi\-agent LLMs\.External Links:2512\.04668Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p2.1)\.
- C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez \(2023\)MemGPT: towards LLMs as operating systems\.External Links:2310\.08560Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.10062#S2.SS1.p1.1)\.
- J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein \(2023\)Generative agents: interactive simulacra of human behavior\.InACM Symposium on User Interface Software and Technology \(UIST\),Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p1.1)\.
- N\. Reimers and I\. Gurevych \(2019\)Sentence\-BERT: sentence embeddings using siamese BERT\-networks\.InConference on Empirical Methods in Natural Language Processing,Cited by:[§3\.1](https://arxiv.org/html/2606.10062#S3.SS1.p4.3)\.
- R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov \(2017\)Membership inference attacks against machine learning models\.InIEEE Symposium on Security and Privacy,Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p2.1)\.
- B\. Wang, W\. He, S\. Zeng, Z\. Xiang, Y\. Xing, J\. Tang, and P\. He \(2025\)Unveiling privacy risks in LLM agent memory\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p2.1)\.
- D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu \(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InInternational Conference on Learning Representations,Note:arXiv:2410\.10813Cited by:[§3\.1](https://arxiv.org/html/2606.10062#S3.SS1.p1.1)\.
- W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang \(2024\)MemoryBank: enhancing large language models with long\-term memory\.InAAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2606.10062#S1.p1.1)\.

Similar Articles

Agent memory is not just RAG over user facts

Reddit r/AI_Agents

The article argues that simple RAG-based agent memory systems fail in production due to issues like stale preferences, missed keywords, and prompt injection, and advocates for a layered memory architecture with active selection, deterministic fallback, governance, and testing.