有些记忆必须淡去:模型参数中用户记忆的持久性与衰减

arXiv cs.LG 论文

摘要

该论文提出了首个直接在大语言模型权重中管理用户记忆的系统,采用固定容量的 LoRA 适配器,对各秩分量施加时间衰减,并配备一个仅含 200 万参数的轻量级控制器,用于决定记住、更新或遗忘哪些内容。在长期用户轨迹评估中,与持续学习(CL)和检索增强生成(RAG)基线相比,该系统达到了理想预知模型性能的 86%。

arXiv:2611.25235v1 Announce Type: new Abstract: Parametric memory allows Large Language Models to internalize knowledge directly into their weights, offering a potentially powerful alternative to retrieval-based memory (RAG). However, existing approaches to parametric memory are difficult to deploy as they largely mirror the Continual Learning (CL) objective of retaining information indefinitely. This is neither sustainable nor sufficient for a memory system, which must govern knowledge across its lifecycle by retaining stable facts, reinforcing recurring observations, updating changed beliefs, and allowing transient information to fade. We propose a system for parametric memory management consisting of a memory substrate (handling information storage and forgetting) and a learned policy (what to remember, how deeply, and for how long). Our substrate is a fixed-capacity LoRA adapter that consists of k-hot rank combinations which store facts without catastrophic interference and maintain an independently controlled temporal decay, enabling outdated information to fade while long-term facts persist. On this substrate, we train a lightweight controller that governs each incoming user event to ensure the memory system reflects the user's true, updated belief state. To evaluate how memory systems manage user memory, we introduce a controlled dataset of long-horizon user trajectories with varying persistence of facts based on changing belief states and user-behavioral shifts, and probe queries that test recall while information is valid and abstention after it expires. Our policy learns to effectively manage user memory over its entire lifecycle, balancing recall and abstention to achieve 86% of oracle performance, unlike CL and RAG baselines that primarily default to one of the two behaviors. By providing the first system to manage memory directly in model weights, this work establishes a foundation for long-horizon parametric memory.
查看原文
查看缓存全文

缓存时间: 2026/11/20 09:16

# Some Memories Must Fade: Persistence and Decay of User Memories in Model Parameters
Source: [https://arxiv.org/html/2611.25235](https://arxiv.org/html/2611.25235)
###### Abstract

Parametric memory allows Large Language Models \(LLMs\) to internalize knowledge directly into their weights, offering a potentially powerful alternative to retrieval\-based memory \(RAG\)\. However, existing approaches to parametric memory are difficult to deploy as they largely mirror the Continual Learning \(CL\) objective of retaining information indefinitely\. This is neither sustainable nor sufficient for a memory system, which must govern knowledge across its lifecycle by retaining stable facts, reinforcing recurring observations, updating changed beliefs, and allowing transient information to fade\. We propose a system for parametric memory management consisting of a memory substrate \(handling information storage and forgetting\) and a learned policy \(what to remember, how deeply, and for how long\)\. Our substrate is a fixed\-capacity LoRA adapter that stores memories in sparsekk\-hot combinations of adapter ranks to limit interference\. This allows each rank and its stored information to maintain an independently controlled temporal decay, enabling outdated information to fade while long\-term facts persist\. On this substrate, we train a lightweight 2M\-parameter controller that governs each incoming user event to ensure the memory system reflects the user’s true, updated belief state\. To evaluate how memory systems manage user memory, we introduce a controlled dataset of long\-horizon user trajectories with varying persistence of facts based on changing belief states and user\-behavioral shifts, with probe queries that test recall while information is valid and abstention after it expires\. Our policy learns to effectively manage user memory over its entire lifecycle, balancing recall and abstention to achieve 86% of oracle performance, unlike CL and RAG baselines that primarily default to one of the two behaviors\. By providing the first system to manage memory directly in model weights, this work establishes a foundation for long\-horizon parametric memory\.

## 1Introduction

Large Language Models \(LLMs\) increasingly operate over long horizons as agents and personalized assistants, necessitating memory systems to match evolving user preferences\([Ding et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib2);[Liu et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib3)\)\. Currently, retrieval\-based memory is widely adopted for its simplicity—storing information externally and injecting it into the context when relevant\([Lewis et al\., 2021](https://arxiv.org/html/2611.25235#bib.bib4);[Kang et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib5)\)\. Parametric memory instead encodes information directly into the model’s weights, allowing LLMs to leverage memory implicitly without cluttering the prompt context\([Wang et al\., 2025b](https://arxiv.org/html/2611.25235#bib.bib6);[Behrouz et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib7);[Su et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib45)\)\. However, this approach remains difficult to deploy in practice due to its rigidity in adapting to the evolving nature of user memory\. Current approaches largely inherit the objective of continual learning \(CL\), focusing on how to acquire new knowledge without forgetting\([McCloskey and Cohen, 1989](https://arxiv.org/html/2611.25235#bib.bib8);[Kirkpatrick et al\., 2017](https://arxiv.org/html/2611.25235#bib.bib9)\)\. We demonstrate that this objective is insufficient for long\-horizon assistants\. Memory management instead asks which knowledge should be retained, reinforced, overwritten, or forgotten, taking downstream use into account\. For instance,I had a bagel for breakfastshould not remain valid even the next day, whileI work as a teachershould likely be relevant information for years\. External stores expose lifecycle operations directly through addition, update, and deletion, but model parameters do not\([Yan et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib10);[Yu et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib12)\)\. Consequently, in our evaluation, no CL method is able to represent outdated information even though they store values well \(0\.82 retrieval with PCGrad\([Yu et al\., 2020](https://arxiv.org/html/2611.25235#bib.bib11)\)\)\.

We hypothesize that usable parametric memory requires separating the storage dynamics \(how information is stored, and isolated in the substrate\) from its policy \(what to store, how deeply, and for how long\)\. For the substrate dynamics, we design a fixed\-size LoRA substrate\([Hu et al\., 2021](https://arxiv.org/html/2611.25235#bib.bib13)\)where each fact is routed to akk\-hot combination of adapter ranks\([Zou et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib14)\)\. This bounds gradient interference between unrelated facts, while an independent temporal decay rate on each rank allows stale information to fade\([Ebbinghaus, 1885](https://arxiv.org/html/2611.25235#bib.bib15)\)without explicit unlearning\. For the policy, we train a lightweight 2M\-parameter controller that decides, per incoming event, whether to write, at what gradient depth, and with what lifespan\. Stable information receives deep durable writes, repeated routines see accumulating writes, while transient events see short or no lifespans\.

Evaluating these lifecycle decisions requires ground truth that existing memory benchmarks do not provide at the event level\. We therefore construct909909controlled, long\-horizon user trajectories with lifecycle annotations capturing validity, persistence, reinforcement, and contradiction, along with programmatically injected queries that probe per\-event management decisions\. On this testbed, our learned management system reaches0\.560\.56class\-balanced accuracy, compared with0\.420\.42for the strongest CL baseline \(PCGrad\) and0\.500\.50for a compute\-matched retrieval\-based memory manager, recovering 86% of oracle performance\. We study two questions: \(1\) Can a fixed\-capacity parametric substrate support reliable information persistence and decay with varying per\-fact dynamics? \(2\) Given such a substrate, can a policy infer from online observations how each fact should be managed? Our contributions are:

- •Task and controlled benchmark\(§2\): We formalize parametric memory management around four requirements—valid retention, selective expiration, reinforcement and contradiction, and user\-specific dynamics—and construct a dataset of long\-horizon user trajectories with event\-level lifecycle supervision and diagnostic probes\.
- •Memory substrate\(§3\): We propose a fixed\-capacity LoRA substrate combining sparsekk\-hot rank allocation with per\-rank temporal decay, providing controllable mechanisms for selective persistence, reinforcement, overwrite, and calibrated expiration\.
- •Learned management\(§4\): We define a\(binary gate,depth,lifespan\)\(\\text\{binary gate\},\\text\{depth\},\\text\{lifespan\}\)action space over the substrate and train a 2M\-parameter controller that recovers 86\-92% of oracle performance, demonstrating that effective lifecycle decisions can be inferred from the available online signals\.

## 2Task: Parametric Memory Management

### 2\.1Memory Management Requirements

We study parametric memory management for lifelong user\-facing agents, where rather than preserving all written information, the system must manage each fact’s lifecycle as user knowledge evolves\. We identify four requirements:

1. 1\.Valid retention:Recall stored information while it remains valid\.
2. 2\.Selective expiration:Once information is no longer valid, abstain\.
3. 3\.Reinforcement and contradiction:Repeated observations should strengthen an existing belief, while contradictory observations should weaken or overwrite it\.
4. 4\.User\-specific dynamics:Persistence and decay must be inferred from individual history, as the same type of information can be long\-term for one user but transient for another\.

Success requires controlling when and how individual memories persist, accumulate, change, or expire\. Existing long\-term memory benchmarks evaluate several related capabilities, including recall and belief revision, but none evaluate these lifecycle requirements at the event level \(Table[1](https://arxiv.org/html/2611.25235#S2.T1)\)\.

### 2\.2Controlled Long\-Horizon Trajectories

Evaluating these requirements requires ground truth for each fact at different stages in its lifecycle\. Real long\-term user histories generally do not provide this supervision\. Privacy constraints limit collection, management events such as contradictions and temporary shifts are sparse, and the true validity of individual memories is generally unknown even when long interaction histories are available\([Uddin et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib16)\)\.

We therefore generate a benchmark of 909 controlled, multi\-year user trajectories \(spanning 700–1,500 days\) with dense event\-level supervision from Gemini 3\.7 Flash\. Each trajectory is constructed via a five\-stage pipeline: \(1\) sampling a behavioral motif \(e\.g\., frequent routine changes\) and a persona, \(2\) generating a multi\-year backstory, \(3\) converting the backstory into a chronological event stream, \(4\) masking entities and injecting ground\-truth labels for lifespan and stability, and \(5\) validating chronological consistency\.

![Refer to caption](https://arxiv.org/html/2611.25235v1/visual1_decay_vs_stability.png)Figure 1:Distribution of the five memory classes by ground\-truth stability score and validity horizon\.To prevent the model from relying on pretrained world knowledge, we replace concrete entities with typed placeholders inside fixed templates \(e\.g\., replacing ”apple” withFRUIT\_4in the templateMy favorite fruit is \[MASK\]\)\. Both writes and queries use this templated form as input in our main experiments and the model must output the valid value that should be in place of \[MASK\] or abstain if expired\. By stripping away real\-world priors, we force the controller to learn generalizable patterns that are not entity\-specific or data generation artifacts \(not memorizing typical lifespans for entities\)\. The event\-level annotations support both evaluation and training\. They provide dense supervision over memory validity, lifespans, reinforcement, and contradiction, enabling supervised or imitation learning, dense reward construction, and end\-to\-end reinforcement learning for memory management \(Appendix[C](https://arxiv.org/html/2611.25235#A3)\)\.111We also provide naturalistic, non\-templated versions of all inputs for evaluating retrieval\-based methods\. §[4\.7](https://arxiv.org/html/2611.25235#S4.SS7)runs the full learned system on these natural\-language \(NL\) inputs\.

### 2\.3Memory Dynamics

Table 1:Coverage of memory\-management properties \(✓ full, ◐ partial, ✗ absent\)\. Ours evaluates belief revision, time\-based expiry, reinforcement, and user\-conditioned lifespans while providing dense lifecycle supervision\. Further details are in Appendix[C\.4](https://arxiv.org/html/2611.25235#A3.SS4)\.The benchmark requirements are informed by memory behaviors emphasized across prior work on long\-term memory and personalization\([Maharana et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib17);[Wu et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib18);[Jiang et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib19);[Uddin et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib16);[McClelland et al\., 1995](https://arxiv.org/html/2611.25235#bib.bib22);[Park and Bak, 2024](https://arxiv.org/html/2611.25235#bib.bib41)\)\. Following generation, the facts are grouped into five management classes based on their annotated validity horizon, stability, reinforcement history, and contradictions:

1. 1\.STABLE \(26%\):Long\-lived \(90–3650 days\), typically introduced once \(e\.g\., profession\)\. Requires durable retention over long horizons\.
2. 2\.ROUTINE \(42%\):Short\-lived \(2–21 days\) observations that repeatedly recur with the same value\. Requires evidence to accumulate rather than treating each observation independently\.
3. 3\.CONTESTED \(20%\):Information whose value is revised over time \(e\.g\., current project\)\. Requires updating the belief state to the latest value and superseding previous state\.
4. 4\.TRANSIENT \(7%\):Very short\-lived \(1–10 days\), unrepeated information \(e\.g\., today’s lunch\)\. Requires expiration and eventual abstention even without a subsequent contradiction\.
5. 5\.ONESHOT \(4%\):Unrepeated information with an intermediate validity horizon \(15–90 days\)\. Requires temporary retention without either immediate loss or indefinite lifespans\.

### 2\.4Query Evaluation

We evaluate management using queries placed at controlled points in each fact’s lifecycle\. A*retrieval query \(VAL\)*occurs while the fact is valid and is scored on recovering the current value\. An*abstention query \(ABS\)*occurs after expiry and is scored on answeringUnknown\. For CONTESTED facts, queries target the currently valid value\. Repeated ROUTINE events test whether consistent observations accumulate into stronger retention\. From Figure[1](https://arxiv.org/html/2611.25235#S2.F1), we can see that some classes have distinctive ground\-truth patterns \(STABLE vs TRANSIENT\) but others can be overlapping \(ROUTINE vs CONTESTED vs TRANSIENT\)\. A valid management policy must therefore identify the user\-level signals that segregate overlapping classes\. Per\-fact query budgets are reweighted to ensure that important but less frequent behaviors, such as expiration, receive sufficient evaluation mass despite write imbalances \(Appendix[C](https://arxiv.org/html/2611.25235#A3)\)\. The resulting query distribution is therefore designed as a controlled stress test of memory\-management capabilities rather than an estimate of natural query frequency\. Each class isolates a distinct challenge: long\-horizon retention \(STABLE\), reinforcement accumulation \(ROUTINE\), belief revision \(CONTESTED\), principled expiration \(TRANSIENT\), and intermediate\-horizon retention \(ONESHOT\)\. We report class\-balanced \(CB\) accuracy so that performance is not dominated by the most frequent lifecycle class\.

## 3Memory Substrate

Figure 2:Overview of the parametric memory management architecture\.An incoming user memory is mapped by the router to its storage in the LoRA adapter, consisting of akk\-hot set of ranks in the up\-projection \(B\) matrix\. Based on the available online features including features from these ranks, the controller decides whether and how the information should be memorized\.Prior work shows that LoRA adapters can serve as compact parametric stores for information\([Xu et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib23);[Back et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib24)\)\. Standard LoRA, however, is not well suited to lifelong memory management as consecutive writes update the same shared parameter space, causing interference between unrelated memories, while information written once has no mechanism to expire selectively\([Yang et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib26)\)\. We therefore design a fixed\-capacity substrate on a Qwen 3 1\.7B backbone with three properties required for parametric memory management in the attached LoRA adapter:isolated writes,controllable lifespans, andcalibrated expiration\. Together, these provide the memory dynamics over which the management policy in §[4](https://arxiv.org/html/2611.25235#S4)operates \(Figure[2](https://arxiv.org/html/2611.25235#S3.F2)\)\.

### 3\.1Sparse Combinatorial Memory Locations

We treat theRRrank columns of the adapter’sBBmatrix as an addressable memory bank\. Each write is routed to akk\-hot subset of columns while all others remain frozen and retrieval loads the same subset\([Zou et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib14)\)\. TheAAmatrix is orthonormally initialized and frozen throughout, as freezingAAyields a more predictable, approximately proportional relationship between write depth and memorization strength\([Zhang et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib25)\)\. WithR=32R=32andk=2k=2as our main configuration, this yields\(322\)=496\\binom\{32\}\{2\}=496addressable locations from3232shared basis columns\. Largerkkraises recall at the expense of abstention \(Appendix[A\.1](https://arxiv.org/html/2611.25235#A1.SS1)\)\. This value\-abstain trade\-off dictates the limits of the management policies analyzed in §[4](https://arxiv.org/html/2611.25235#S4)\.

This routing provides two core properties\. First,interference is boundedas writes to disjoint masks share no gradient path, and contention scales with mask overlap\|m1∩m2\|\|m\_\{1\}\\cap m\_\{2\}\|only\. Second,repeated writes accumulateas five writes at depthd=20d=20\(20 gradient descent steps at fixed learning rate\) achieve comparable retention to a singled=100d=100write \(Appendix[A\.2](https://arxiv.org/html/2611.25235#A1.SS2)\), allowing policies to reinforce facts via incremental updates\. A key component of this architecture is the router that assigns related facts and queries to the same k\-hot ranks for reliable updates and retrieval\. We evaluate two routers\. Enabled by our templated setting, the*perfect router*deterministically hashes templates to masks, upper\-bounding routing\. The*learned router*assigns slots using cosine similarity on the fact’s whitened embedding\([Su et al\., 2021](https://arxiv.org/html/2611.25235#bib.bib27)\)with existing slots, allocating new masks when similarity is low \(Appendix[A](https://arxiv.org/html/2611.25235#A1)\)\. Routing is a separable, independently tuned module\. By evaluating both perfect and learned variants, we decouple the impact of storage from management\.

### 3\.2Controllable Temporal Lifespans

To support selective expiration, each rank columniicarries an independent decay constantτi\\tau\_\{i\}\. Between events separated byΔ​t\\Delta tdays,B\[:,i\]←B\[:,i\]⋅exp\(−Δt/τi\)B\[:,i\]\\leftarrow B\[:,i\]\\cdot\\exp\(\-\\Delta t/\\tau\_\{i\}\)\. Each write overwritesτi\\tau\_\{i\}for its columns, letting the policy assign different lifespan horizons to different information\. We discretizeτ\\tauinto five temporal scales using empirical decay timelines, from short\-lived to effectively permanent \(values in §[4\.1](https://arxiv.org/html/2611.25235#S4.SS1)\)\. Per\-location decay is structurally necessary because stable and transient information coexist in the same fixed\-capacity adapter\. A single global rate either expires stable facts prematurely or preserves transient ones indefinitely \(validated in §[4\.5](https://arxiv.org/html/2611.25235#S4.SS5), where a globalτ\\taureduces abstention on all volatile classes\)\. Independent decay makes forgetting a controllable operation where information fades and its capacity is reclaimed without an explicit unlearning step\. However, when two facts are routed to overlapping columns, the older fact’s decay rate is overwritten by the newer fact’s decay rate\. We analyze the impact of this shared\-column limitation in §[4\.6](https://arxiv.org/html/2611.25235#S4.SS6)\.

### 3\.3Calibrated Retrieval and Abstention

A memory substrate must also represent absence\. A weak or expired trace should produce abstention rather than a hallucinated value\. Base models tend to fabricate values from weak parametric traces, and prior work treats this behavior as an intrinsic threshold of parametric memorization\([Xu et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib23)\)\. We show that abstention from weak traces is instead a resolvable, calibration problem\.

A separate rank\-3232*retrieval adapter*operates jointly with the memory adapter during inference, translating memory traces into expected outputs\. Instead of memorizing facts itself, the retrieval adapter is trained to read the underlying substrate’s state using depth\-conditioned labels\. If a routed slot is empty, it is trained to outputUnknown\. If a fact was written to the memory adapter at depthdd, the retrieval adapter’s target isUnknownwith label probabilitypunk​\(d\)p\_\{\\text\{unk\}\}\(d\)and the correct value otherwise \(e\.g\.punk​\(20\)=0\.9p\_\{\\text\{unk\}\}\(20\)=0\.9andpunk​\(150\)=0p\_\{\\text\{unk\}\}\(150\)=0\)\.

The shape of the monotonically decreasing curvepunk​\(d\)p\_\{\\text\{unk\}\}\(d\)\(Appendix[A\.3](https://arxiv.org/html/2611.25235#A1.SS3)\) governs behavior under weak writes\. A curve that drops too soon may retrieve hallucinated values from weak traces, while one that drops too late may collapse all queries to abstention\. This probabilistic mapping yields a graded storage mechanism, rather than a binary retrievable threshold, giving the controller meaningful control over retention strength viadd\. Crucially, because the retrieval adapter uses the state of the substrate to generate answers rather than the input, retrieval is separated from memory management\. The same substrate can be paired with alternative adapters, such as one trained for free\-form natural language instead of our templates, without altering the underlying mechanics\. \(§[4\.7](https://arxiv.org/html/2611.25235#S4.SS7), Appendix[D](https://arxiv.org/html/2611.25235#A4)\)\.

### 3\.4Why Continual Learning Is Insufficient

Table 2:CL protects stored information but cannot represent selective expiration\. Per\-class breakdowns are in the Appendix[A\.4](https://arxiv.org/html/2611.25235#A1.SS4)\.Do these properties emerge from applying standard continual learning \(CL\) to a shared LoRA store? We test EWC\([Kirkpatrick et al\., 2017](https://arxiv.org/html/2611.25235#bib.bib9)\)and PCGrad with sequential fixed\-depth LoRA writes, selectingd=15d=15\(optimal selection fromd=10,15,20,40,60d=10,15,20,40,60\)\. Both reduce interference by protecting previously written information but provide no mechanism for selective expiration\. Consequently, every CL baseline achieves0\.000\.00ABS despite PCGrad reaching0\.820\.82VAL \(Table[2](https://arxiv.org/html/2611.25235#S3.T2)\)\. Adding uniform exponential decay changes accuracy by<0\.01<0\.01, since dense shared parameters cannot assign different lifespans to different memories\. Our substrate resolves this structurally as specific ranks decay to physically represent forgetting, which trigger the retrieval adapter to abstain\. Sparse locations, independent decay, and calibrated retrieval provide this control\.

## 4Learned Memory Management

The substrate defines the storage of parametric memory\. Here, we investigate \(1\) what actions are required to manage this substrate, \(2\) whether the information available online is sufficient to infer effective management decisions, and \(3\) what trade\-offs remain even under effective management due to the substrate’s finite capacity\.

### 4\.1Memory Management Actions

For each incoming event, the controller emits an action\(g,d,τ\)\(g,d,\\tau\):

- •Gate\(g∈0,1g\\in\{0,1\}\): whether to write the event at all\. Declining leaves the substrate unchanged\.
- •Depth\(d∈0,20,30,40,60,80,110,150d\\in\{0,20,30,40,60,80,110,150\}SGD steps\): how strongly to write\. Higherddyields stronger retention but a larger gradient footprint on shared mask columns\.
- •Lifespan\(τ∈25,100,400,1600,20000\\tau\\in\{25,100,400,1600,20000\}\): the decay timescale assigned to the routed columns, controlling how long the resulting trace persists\.

The three actions are predicted by independent discretized heads, allowing each to be conditioned and bias\-initialized separately \(§[4\.4](https://arxiv.org/html/2611.25235#S4.SS4)\)\. Discretization also constrains the policy search space and simplifies optimization\. We find two design choices to be important\. First, the action space distinguishes*rejecting*information \(g=0g=0\) from*briefly retaining*it through a shallow write with short lifespan\. Different volatile events may require either behavior, motivating independent gate and lifespan controls\. Second, routing is separated from the management policy\. The router in Section[3\.1](https://arxiv.org/html/2611.25235#S3.SS1)determines which ranks an event is written in, while the controller determines how/whether it is stored through gate, depth, and lifespan\. We evaluate both perfect and learned routing to separate allocation errors from management errors\.

### 4\.2State and Policy Architecture

The 2M\-parameter controller \(<0\.4%<0\.4\\%of the base model\) conditions on three sources of online information: \(1\) theincoming eventrepresented by its semantic embedding, \(2\)trajectory historyrepresented by a GRU state\([Cho et al\., 2014](https://arxiv.org/html/2611.25235#bib.bib29)\), and \(3\)substrate state, consisting of per\-rank ages \(time since last write for each rank\), per\-rank norms \(to capture stored information\), and token\-level prediction signals from a forward pass that indicate whether the incoming event reinforces or contradicts the existing memory\. A shared neural trunk feeds independent decision heads for gate, depth, and lifespan \(Appendix[B\.4](https://arxiv.org/html/2611.25235#A2.SS4)\)\. Persistence cannot be determined from event content alone\. Because identical facts exhibit different stabilities across users, relying solely on semantics leads to a prediction ceiling of onlyR2=0\.36R^\{2\}=0\.36\(Appendix[B\.13](https://arxiv.org/html/2611.25235#A2.SS13)\)\. The controller addresses this ambiguity by grounding its decisions in the user’s trajectory history\. Finally, to stabilize early training, the action heads are initialized with empirical biases derived from an expert policy\.

### 4\.3Pipeline Workflow

Figure 3:End\-to\-end workflow of parametric memory management \(PMM\)Components\.The system is composed of a frozen Qwen3\-1\.7B backbone, two additive LoRA adapters \(Memory and Retrieval\), and two management components \(a router and a controller\)\. Both adapters are attached toq\_projandv\_projin every layer, so a layer’s output ish=W​x\+Bread​Aread​x\+Bmem​Amem​xh=Wx\+B\_\{\\text\{read\}\}A\_\{\\text\{read\}\}x\+B\_\{\\text\{mem\}\}A\_\{\\text\{mem\}\}x\. The base model is never updated in the process \(Figure[3](https://arxiv.org/html/2611.25235#S4.F3)\)\.

Retrieval\-adapter training \(before deployment\)\.We simulate writes of random facts at random depths and train the retrieval adapter to output either the stored value orUnknown, according to the depth\-conditioned label probabilitypunk​\(d\)p\_\{\\text\{unk\}\}\(d\)\(§[3\.3](https://arxiv.org/html/2611.25235#S3.SS3)\)\. It is then frozen for all subsequent steps\.

Write \(per incoming fact\)\.The router maps the fact to an existing rank mask, or allocates a new one\. The mask is specific to a fact family and shared across all layers \(we usek=2k=2columns per family\)\. The controller reads the fact embedding, the memory\-adapter state, and the user history \(through its GRU\), and outputs whether to write the fact, a depthd∈\[20,150\]d\\in\[20,150\], and a lifetimeτ∈\[25,20000\]\\tau\\in\[25,20000\]days\. If written to the Memory adapter, the write isddSGD steps on the selected columns ofBmemB\_\{\\text\{mem\}\}only, and those columns take decay rateτ\\tau\.AmemA\_\{\\text\{mem\}\}is orthonormal and frozen throughout\. The retrieval adapter is not active during writes\.

Decay \(between events\)\.Each column decays asBi←Biexp\(−Δt/τi\)B\_\{i\}\\leftarrow B\_\{i\}\\exp\(\-\\Delta t/\\tau\_\{i\}\), making a fact expire\.

Read \(per query\)\.The router selects the query’s mask, and ranks outside it are zeroed\. The Memory and Retrieval adapters are both active, and the Retrieval adapter decides from the memory state whether to emit the stored value orUnknown\. The controller is not used\.

### 4\.4Learning from a Metadata Oracle

We train the controller by imitation from a manually configured*causal metadata oracle*\. The oracle observes the same online history as the controller together with ground\-truth lifecycle metadata \(stability, time to decay, and reinforcement status\) and computes a target write depth based on a deficit\. The deficit is the estimated depth required for a fact to remain retrievable over its target horizon, minus any depth already accumulated through prior writes, capped at 150\. The oracle has no access to future events, but receives lifecycle metadata that the learned controller must infer from its observable state features\. Crucially, this supervision adapts to the student’s actual substrate state under DAgger\([Ross et al\., 2011](https://arxiv.org/html/2611.25235#bib.bib30)\)\. When the student deviates from the oracle, the target depth is recomputed from what the student has actually written rather than assuming ideal previous actions\. Replacing this state\-aware supervision with fixed target depths derived solely from ground\-truth metadata reduces class\-balanced accuracy from0\.560\.56to0\.490\.49\(Appendix[B\.7](https://arxiv.org/html/2611.25235#A2.SS7)\)\.

#### Training\.

The training pipeline has three stages: \(A\) stability\-head MSE training, \(B\) behavior cloning against the oracle via DAgger, and \(C\) optional PPO refinement\([Schulman et al\., 2017](https://arxiv.org/html/2611.25235#bib.bib31)\)\(Appendix[B\.15](https://arxiv.org/html/2611.25235#A2.SS15)\)\. Stage B reaches near\-oracle performance and is our primary configuration\. We compare our parametric memory management \(PMM\) controllers trained with perfect routing \(PMM\-PR\) and learned routing \(PMM\-LR\) to separate the learned policy impact from allocation quality\.

### 4\.5Results

We evaluate on1515held\-out user trajectories \(∼600\\sim 600events per user\) with sequential LoRA writes and queries\. Results are reported per lifecycle class and per query type \(VAL, ABS\), aggregated using class\-balanced \(CB\) accuracy\. Table[3](https://arxiv.org/html/2611.25235#S4.T3)answers three questions\. First, can conventional parametric memorization solve the lifecycle problem? No, CL baselines and unmanaged\-write on our substrate retrieve values well but fail to expire outdated information\. Second, is lifecycle management learnable? Yes, PMM\-LR recovers 86% of the metadata\-oracle ceiling\. Third, what limits performance? The fully learned PMM\-LR reaches0\.560\.56CB accuracy, while perfect routing raises performance to0\.600\.60against a metadata\-oracle ceiling of0\.650\.65\. This hierarchy separates distinct sources of error\. The PMM\-LR\-to\-PMM\-PR gap reflects semantic allocation, the PMM\-PR\-to\-oracle gap primarily reflects lifecycle inference, and the oracle ceiling reflects limitations of the substrate itself, including shared\-column interference between distinct memories andτ\\tauassignment \(analyzed below\)\. Improving routing can close the first gap, richer online management signals can close the second, and changes to allocation or capacity are required to raise the substrate ceiling\.

Table 3:Evaluation results onn=15n=15held\-out users,45844584queries\. RAG\-MM refers to RAG with a write\-time memory\-management architecture \(Appendix[B\.3](https://arxiv.org/html/2611.25235#A2.SS3)\)\. Naive/EWC continual\-learning variants \(≤0\.23\\leq 0\.23CB\), oracle\-metadata RAG upper bounds, and no\-τ\\tauunmanaged baselines are reported in Appendix[B\.6](https://arxiv.org/html/2611.25235#A2.SS6)\. NL refers to natural\-language interfacing of the same trajectories\.95%95\\%percentile bootstrap CIs clustered by user \(users resampled, 2,000 resamples\)\.Both controllers outperform all CL and unmanaged\-write baselines, as well as the retrieval\-based memory managers tested at compute parity\. PMM\-LR reaches0\.560\.56versus0\.500\.50for compute\-matched RAG\-MM and also exceeds RAG\-MM with a substantially larger 122B management model \(0\.530\.53\)\. A trained manager in RAG\-MM matches performance, though it often struggles to synthesize correct answers even with improved management\. Retrieval surpasses our controllers when its management pass is replaced with ground\-truth lifecycle information \(0\.640\.64at1\.71\.7B and0\.820\.82at122122B §[4\.6](https://arxiv.org/html/2611.25235#S4.SS6)\), highlighting the importance of the management signal independent of the underlying memory substrate\. Once information is written, queries require no retrieval over an external history and no context growth with trajectory length\.

### 4\.6Per\-Class Trade\-offs and Failure Modes

Figure 4:Pareto positions in value\-retrieval \(VAL\) versus abstention \(ABS\) accuracy\. Four regimes emerge: \(1\) continual learning and unmanaged writes at high VAL and near\-zero ABS, \(2\) compute\-parity retrieval at high ABS and low VAL, \(3\) augmented retrieval using greater compute or privileged lifecycle information reaching the top\-right, and \(4\) our controllers occupying a balanced regime that no compute\-parity baseline reaches\. Breakdown in Table[12](https://arxiv.org/html/2611.25235#A2.T12)\.#### Memorization without management fails at abstention\.

Fixed\-depth writes demonstrate a clear trend\. Atd=20d=20, VAL ranges from0\.600\.60\-0\.900\.90across classes while ABS remains only0\.010\.01\-0\.220\.22\. Increasing depth tod=80d=80raises VAL to0\.850\.85\-1\.001\.00while driving ABS to≤0\.01\\leq 0\.01\(per\-class VAL/ABS splits in Appendix[B\.1](https://arxiv.org/html/2611.25235#A2.SS1)\)\. Addingτ\\tauwithout event\-specific management does not improve this trade\-off \(Appendix[B\.2](https://arxiv.org/html/2611.25235#A2.SS2)\)\.

#### Retrieval\-based memory also requires explicit management\.

Compute\-matched simple RAG fails due to frequently defaulting toUnknownunder ambiguous context \(ABS0\.750\.75–0\.870\.87\) while rarely committing to values \(VAL0\.160\.16–0\.300\.30on most classes\)\. RAG\-MM’s write\-time management pass—resolving reinforcement, contradiction, and expiry as events arrive \(Appendix[B\.3](https://arxiv.org/html/2611.25235#A2.SS3)\)—shifts this Pareto position\. Replacing the learned management pass with ground\-truth lifecycle information raises RAG\-MM to0\.640\.64at1\.71\.7B and0\.820\.82at122122B \(Appendix[B\.6](https://arxiv.org/html/2611.25235#A2.SS6)\), providing an upper bound when management is ideal and not inferred \(Figure[4](https://arxiv.org/html/2611.25235#S4.F4)\)\.

#### Management matters where events require different lifespans\.

Our controllers occupy a balanced regime that other compute\-parity baseline do not reach\. For TRANSIENT facts, no unmanaged fixed\-depth policy exceeds0\.260\.26accuracy because writing them strongly enough for immediate retrieval causes them to remain accessible after expiry\. PMM\-LR and PMM\-PR reach0\.440\.44and0\.590\.59, respectively, by assigning short lifespans to volatile information\. Conversely, on STABLE facts, unmanagedd=80d=80reaches0\.650\.65while PMM reaches0\.390\.39–0\.400\.40, pairing moderate VAL \(0\.330\.33–0\.340\.34\) with strong ABS \(0\.780\.78–0\.810\.81\)\.

#### The STABLE gap is due toτ\\tauoverwrites on shared columns\.

Failures in retrieval of STABLE facts occur primarily at longer query horizons and under greater foreign\-write interference, rather than from policy failures like declined writes or insufficient initial depth \(Appendix[B\.14](https://arxiv.org/html/2611.25235#A2.SS14)\)\. Because each write overwritesτ\\tauon the columns it touches, a STABLE fact can inherit the shorter timescale of a later volatile write sharing the same column over its long horizon\. Preserving the maximumτ\\taurecovers∼13\{\\sim\}13points of STABLE VAL but collapses ABS on volatile classes, reducing CB accuracy to0\.450\.45\. This exposes a lifespan\-compression trade\-off in the shared substrate\. Reinforcement partially mitigates the effect as STABLE facts with3\+3\+writes achieve0\.680\.68retrieval success as compared to0\.510\.51for single\-write facts\. High STABLE retrieval performance can be achieved by routing them to non\-overlapping ranks, at the cost of compression \(Table[4](https://arxiv.org/html/2611.25235#S4.T4)\)\.

Table 4:Dedicated columns for long\-lived facts improve their retention\. MB refers to size of adapter\.
#### Per\-rankτ\\tauenables joint optimization\.

Removing per\-rank lifespan control and forcing a globalτ\\taudrops PMM\-PR from0\.600\.60to0\.480\.48\. Without the ability to selectively expire information, the controller maintains retrieval only by sacrificing abstention\. VAL remains≥0\.79\\geq 0\.79across volatile classes, while TRANSIENT ABS falls to0\.090\.09, ROUTINE ABS to0\.060\.06, and CONTESTED ABS to0\.040\.04\. This establishes independent lifespan control as a necessary mechanism for supporting both retention and expiration in this substrate\.

Table 5:Ablations on controller input features\.
#### Ablations\.

Table[5](https://arxiv.org/html/2611.25235#S4.T5)reports input\-feature ablations for PMM\-PR\. Removing semantic features or configured priors substantially reduces aggregate accuracy \(0\.470\.47and0\.520\.52, respectively\), confirming that both provide non\-redundant signals during training\. History and substrate state are partially redundant because both reflect previous observations \(through trajectory or through substrate state impact\)\. However, class\-specific failures highlight the distinct mechanistic role of each feature\. Without trajectory history, ONESHOT VAL drops by 24 points because these first\-time events lack a pre\-existing trace to provide a redundant signal\. Conversely, without substrate state, ROUTINE ABS drops by 9 points\. Blind to how strongly a recurring fact is already embedded, the controller over\-relies on historical frequency and over\-writes the trace, preventing it from properly expiring\. These results show that effective lifecycle management depends on event semantics, the user’s trajectory, and the current state of memory\. Full per\-class results are provided in Appendix[B\.16](https://arxiv.org/html/2611.25235#A2.SS16)\.

### 4\.7Natural Language Transfer

The experiments above use templated writes and queries to decouple management from memorization\. Here we run the full learned system on the naturalistic version of the same trajectories, with the same architecture and training recipe\. Natural\-language writes behave differently \(recall plateaus after about6060SGD steps\), so we re\-measured write strength and decay once, and refit the controller targets and the retrieval adapter on these measurements \(Appendix[D](https://arxiv.org/html/2611.25235#A4)\)\. As seen in Table[3](https://arxiv.org/html/2611.25235#S4.T3), every controller lands within0\.020\.02CB of its templated counterpart\.

## 5Related Work

### 5\.1Parametric Memory and Lifelong Knowledge Editing

A growing body of work stores information in model parameters or latent states rather than external text\([Sun et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib35);[Ba et al\., 2016](https://arxiv.org/html/2611.25235#bib.bib34)\)\. Recent systems divide into latent\-state memory, maintaining pools of hidden\-state memory tokens\([Wang et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib32);[Wang et al\., 2025a](https://arxiv.org/html/2611.25235#bib.bib33)\), and parametric memory, writing into weights through context distillation\([Wang et al\., 2025b](https://arxiv.org/html/2611.25235#bib.bib6)\), dedicated memory modules\([Berges et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib36)\), or test\-time weight updates\([Behrouz et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib7)\)\. In parallel, lifelong knowledge editing develops efficient, localized modification of model knowledge\([Meng et al\., 2023](https://arxiv.org/html/2611.25235#bib.bib37);[Hartvigsen et al\., 2023](https://arxiv.org/html/2611.25235#bib.bib38);[Yu et al\., 2023](https://arxiv.org/html/2611.25235#bib.bib39)\)\. Both lines study whether information can be written, retrieved, or updated while preserving unrelated knowledge\. We instead study memory lifecycle management which explores how strongly information should be stored, how long it should persist, and when new observations should reinforce or replace existing beliefs\.

### 5\.2Sparse and Modular Parameter Updates

Sparse and modular weight updates reduce interference during updates\. O\-LoRA\([Wang et al\., 2023](https://arxiv.org/html/2611.25235#bib.bib40)\)uses near\-orthogonal task subspaces, N\-LoRA\([Yang et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib26)\), and FlyLoRA\([Zou et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib14)\)allocate sparse adapter ranks to increase capacity and limit overlap\. Our substrate similarly uses sparse combinations of adapter ranks, but as reusable memory locations rather than task\-specific modules\. Further, parametric overlap is not uniformly undesirable as unrelated writes should stay isolated, while overlap between related information supports reinforcement and overwrite\. We further provide a controlled temporal decay for each rank, making persistence an explicit property of the memory rather than a byproduct of the memorization\.

### 5\.3Memory Management and Continual Learning

To our knowledge, no prior work attempts to govern the parametric memory lifecycle—retention, expiration, reinforcement, and overwrite—as a learned management problem\. Retrieval\-based memory management systems organize external stores through explicit hierarchies and paging\([Packer et al\., 2024](https://arxiv.org/html/2611.25235#bib.bib1);[Kang et al\., 2025](https://arxiv.org/html/2611.25235#bib.bib5)\), learned memory operations\([Yu et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib12);[Yan et al\., 2026](https://arxiv.org/html/2611.25235#bib.bib10)\), or importance\- and lifespan\-based retention\([Park and Bak, 2024](https://arxiv.org/html/2611.25235#bib.bib41)\), demonstrating that what is retained or forgotten can be explicitly managed\. These managers, however, operate over explicit entries that can be directly added, updated, or deleted\. Our controller instead governs the formation and lifespan of information encoded in parameters—deciding whether, how strongly, and for how long to write—so retention and forgetting emerge from the dynamics of the substrate itself\.

Continual learning\([Zeng et al\., 2019](https://arxiv.org/html/2611.25235#bib.bib42);[Yu et al\., 2020](https://arxiv.org/html/2611.25235#bib.bib11);[Saha et al\., 2021](https://arxiv.org/html/2611.25235#bib.bib43)\)addresses the complementary problem of acquiring new knowledge while minimizing interference with the old\. Our experiments \(§[3\.4](https://arxiv.org/html/2611.25235#S3.SS4)\) show memorization alone is insufficient for long\-horizon memory, where some information must persist while other information must become inaccessible\.

## 6Conclusion

We present parametric memorization and parametric memory management as distinct problems, often conflated by prior work\. Long\-horizon memory requires deciding what to retain, reinforce, overwrite, or let expire\. Existing parametric memory mechanisms do not explicitly expose or control these lifecycle decisions\. Separating memory into its substrate dynamics—a fixed\-capacity substrate with sparse, addressable locations and controllable temporal decay—and its policy—a lightweight controller governing the substrate through a defined action space—enables the system to balance retrieval of valid information against abstention on information that should no longer be accessible\. Our controlled environment supports two central findings: \(1\) a fixed\-capacity parametric substrate can support varying memory lifecycles, and \(2\) the decisions required to manage those lifecycles can be inferred from online observations\.

#### Limitations\.

Moving to a new input format type requires re\-measuring write strength and decay \(§[4\.7](https://arxiv.org/html/2611.25235#S4.SS7), Appendix[D](https://arxiv.org/html/2611.25235#A4)\)\. We also assume memory identification, determining what information constitutes a memory event, is given, and our controlled trajectories may not cover all real\-world user dynamics\. Dedicated columns for long\-lived facts increase per\-user memory \(§[4\.6](https://arxiv.org/html/2611.25235#S4.SS6)\)\.

#### Future work\.

The learned\-routing, controller, and oracle gaps motivate better semantic addressing, richer write\-time signals, and improved capacity allocation\. RL over long\-term memory utility may further improve the controller, and different types of information like workflows can be stored in the substrate\.

Overall, our results suggest that reliable parametric memory requires progress beyond how models memorize\. It requires controlling how information evolves after it has been written\. By showing that a 2M\-parameter policy operating on online signals recovers 86\-92% of a metadata\-privileged oracle, we establish parametric memory management as a distinct and learnable problem and provide a foundation for lifecycle\-aware memory in model parameters\.

## References

- Allen\-Zhu and Li \(2024\)Z\. Allen\-Zhu and Y\. LiPhysics of language models: part 3\.1, knowledge storage and extraction\.External Links:2309\.14316,[Link](https://arxiv.org/abs/2309.14316)Cited by:[§D\.2](https://arxiv.org/html/2611.25235#A4.SS2.p1.1)\.
- Baet al\.\(2016\)J\. Ba, G\. Hinton, V\. Mnih, J\. Z\. Leibo, and C\. IonescuUsing fast weights to attend to the recent past\.External Links:1610\.06258,[Link](https://arxiv.org/abs/1610.06258)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Backet al\.\(2026\)S\. Back, D\. Lee, N\. Kang, T\. Lee, S\. K\. Hong, Y\. Gwon, and S\. AhnUnderstanding lora as knowledge memory: an empirical analysis\.External Links:2603\.01097,[Link](https://arxiv.org/abs/2603.01097)Cited by:[§3](https://arxiv.org/html/2611.25235#S3.p1.1)\.
- Behrouzet al\.\(2024\)A\. Behrouz, P\. Zhong, and V\. MirrokniTitans: learning to memorize at test time\.External Links:2501\.00663,[Link](https://arxiv.org/abs/2501.00663)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1),[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Bergeset al\.\(2024\)V\. Berges, B\. Oğuz, D\. Haziza, W\. Yih, L\. Zettlemoyer, and G\. GhoshMemory layers at scale\.External Links:2412\.09764,[Link](https://arxiv.org/abs/2412.09764)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Choet al\.\(2014\)K\. Cho, B\. van Merrienboer, C\. Gulcehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. BengioLearning phrase representations using rnn encoder\-decoder for statistical machine translation\.External Links:1406\.1078,[Link](https://arxiv.org/abs/1406.1078)Cited by:[§4\.2](https://arxiv.org/html/2611.25235#S4.SS2.p1.1)\.
- Dinget al\.\(2026\)T\. Ding, A\. Nannapaneni, B\. Liu, and L\. ZhangAlways\-onagents:a survey of persistent memory, state, and governance in llmagents\.External Links:2606\.30306,[Link](https://arxiv.org/abs/2606.30306)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1)\.
- Ebbinghaus \(1885\)H\. EbbinghausMemory: a contribution to experimental psychology\.Duncker & Humblot,Leipzig, Germany\.Note:English translation published 1913 by Teachers College, Columbia UniversityCited by:[§1](https://arxiv.org/html/2611.25235#S1.p2.1)\.
- Guet al\.\(2026\)R\. Gu, Z\. Zhang, and W\. WangForgetBench: benchmarking forgetting dynamics of long\-term parametric memory in language models\.External Links:2607\.26455,[Link](https://arxiv.org/abs/2607.26455)Cited by:[Table 1](https://arxiv.org/html/2611.25235#S2.T1.2.2.1)\.
- Hartvigsenet al\.\(2023\)T\. Hartvigsen, S\. Sankaranarayanan, H\. Palangi, Y\. Kim, and M\. GhassemiAging with grace: lifelong model editing with discrete key\-value adaptors\.External Links:2211\.11031,[Link](https://arxiv.org/abs/2211.11031)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Huet al\.\(2021\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.External Links:2106\.09685,[Link](https://arxiv.org/abs/2106.09685)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p2.1)\.
- Huet al\.\(2026\)Y\. Hu, Y\. Wang, and J\. McAuleyEvaluating memory in llm agents via incremental multi\-turn interactions\.External Links:2507\.05257,[Link](https://arxiv.org/abs/2507.05257)Cited by:[Table 1](https://arxiv.org/html/2611.25235#S2.T1.2.5.1)\.
- Jianget al\.\(2025\)B\. Jiang, Z\. Hao, Y\. Cho, B\. Li, Y\. Yuan, S\. Chen, L\. Ungar, C\. J\. Taylor, and D\. RothKnow me, respond to me: benchmarking llms for dynamic user profiling and personalized responses at scale\.External Links:2504\.14225,[Link](https://arxiv.org/abs/2504.14225)Cited by:[§2\.3](https://arxiv.org/html/2611.25235#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2611.25235#S2.T1.2.4.1)\.
- Kanget al\.\(2025\)J\. Kang, M\. Ji, Z\. Zhao, and T\. BaiMemory os of ai agent\.External Links:2506\.06326,[Link](https://arxiv.org/abs/2506.06326)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1),[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p1.1)\.
- Kirkpatricket al\.\(2017\)J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska, D\. Hassabis, C\. Clopath, D\. Kumaran, and R\. HadsellOvercoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.External Links:ISSN 1091\-6490,[Link](http://dx.doi.org/10.1073/pnas.1611835114),[Document](https://dx.doi.org/10.1073/pnas.1611835114)Cited by:[§A\.4](https://arxiv.org/html/2611.25235#A1.SS4.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2611.25235#S1.p1.1),[§3\.4](https://arxiv.org/html/2611.25235#S3.SS4.p1.1)\.
- Lewiset al\.\(2021\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. KielaRetrieval\-augmented generation for knowledge\-intensive nlp tasks\.External Links:2005\.11401,[Link](https://arxiv.org/abs/2005.11401)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1)\.
- Liuet al\.\(2025\)J\. Liu, Z\. Qiu, Z\. Li, Q\. Dai, W\. Yu, J\. Zhu, M\. Hu, M\. Yang, T\. Chua, and I\. KingA survey of personalized large language models: progress and future directions\.External Links:2502\.11528,[Link](https://arxiv.org/abs/2502.11528)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1)\.
- Maharanaet al\.\(2024\)A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. FangEvaluating very long\-term conversational memory of llm agents\.External Links:2402\.17753,[Link](https://arxiv.org/abs/2402.17753)Cited by:[§2\.3](https://arxiv.org/html/2611.25235#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2611.25235#S2.T1.2.3.1)\.
- McClellandet al\.\(1995\)J\. L\. McClelland, B\. L\. McNaughton, and R\. C\. O’ReillyWhy there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory\.Psychological Review102\(3\),pp\. 419–457\.Cited by:[§2\.3](https://arxiv.org/html/2611.25235#S2.SS3.p1.1)\.
- McCloskey and Cohen \(1989\)M\. McCloskey and N\. J\. CohenCatastrophic interference in connectionist networks: the sequential learning problem\.G\. H\. Bower \(Ed\.\),Psychology of Learning and Motivation, Vol\.24,pp\. 109–165\.External Links:ISSN 0079\-7421,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/S0079-7421%2808%2960536-8),[Link](https://www.sciencedirect.com/science/article/pii/S0079742108605368)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1)\.
- Mecklenburget al\.\(2024\)N\. Mecklenburg, Y\. Lin, X\. Li, D\. Holstein, L\. Nunes, S\. Malvar, B\. Silva, R\. Chandra, V\. Aski, P\. K\. R\. Yannam, T\. Aktas, and T\. HendryInjecting new knowledge into large language models via supervised fine\-tuning\.External Links:2404\.00213,[Link](https://arxiv.org/abs/2404.00213)Cited by:[§D\.2](https://arxiv.org/html/2611.25235#A4.SS2.p1.1)\.
- Menget al\.\(2023\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in gpt\.External Links:2202\.05262,[Link](https://arxiv.org/abs/2202.05262)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Ovadiaet al\.\(2024\)O\. Ovadia, M\. Brief, M\. Mishaeli, and O\. ElishaFine\-tuning or retrieval? comparing knowledge injection in LLMs\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 237–250\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.15/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.15)Cited by:[§D\.2](https://arxiv.org/html/2611.25235#A4.SS2.p1.1)\.
- Packeret al\.\(2024\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards llms as operating systems\.External Links:2310\.08560,[Link](https://arxiv.org/abs/2310.08560)Cited by:[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p1.1)\.
- Park and Bak \(2024\)S\. Park and J\. BakMemoria: resolving fateful forgetting problem through human\-inspired memory architecture\.External Links:2310\.03052,[Link](https://arxiv.org/abs/2310.03052)Cited by:[§2\.3](https://arxiv.org/html/2611.25235#S2.SS3.p1.1),[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p1.1)\.
- Rosset al\.\(2011\)S\. Ross, G\. J\. Gordon, and J\. A\. BagnellA reduction of imitation learning and structured prediction to no\-regret online learning\.External Links:1011\.0686,[Link](https://arxiv.org/abs/1011.0686)Cited by:[§B\.5](https://arxiv.org/html/2611.25235#A2.SS5.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2611.25235#S4.SS4.p1.1)\.
- Sahaet al\.\(2021\)G\. Saha, I\. Garg, and K\. RoyGradient projection memory for continual learning\.External Links:2103\.09762,[Link](https://arxiv.org/abs/2103.09762)Cited by:[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p2.1)\.
- Schulmanet al\.\(2017\)J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.External Links:1707\.06347,[Link](https://arxiv.org/abs/1707.06347)Cited by:[§4\.4](https://arxiv.org/html/2611.25235#S4.SS4.SSS0.Px1.p1.1)\.
- Suet al\.\(2021\)J\. Su, J\. Cao, W\. Liu, and Y\. OuWhitening sentence representations for better semantics and faster retrieval\.External Links:2103\.15316,[Link](https://arxiv.org/abs/2103.15316)Cited by:[§3\.1](https://arxiv.org/html/2611.25235#S3.SS1.p2.1)\.
- Suet al\.\(2025\)W\. Su, Y\. Tang, Q\. Ai, J\. Yan, C\. Wang, H\. Wang, Z\. Ye, Y\. Zhou, and Y\. LiuParametric retrieval augmented generation\.External Links:2501\.15915,[Link](https://arxiv.org/abs/2501.15915)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1)\.
- Sunet al\.\(2025\)Y\. Sun, X\. Li, K\. Dalal, J\. Xu, A\. Vikram, G\. Zhang, Y\. Dubois, X\. Chen, X\. Wang, S\. Koyejo, T\. Hashimoto, and C\. GuestrinLearning to \(learn at test time\): rnns with expressive hidden states\.External Links:2407\.04620,[Link](https://arxiv.org/abs/2407.04620)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Uddinet al\.\(2026\)M\. N\. Uddin, K\. Shubham, E\. Blanco, C\. Baral, and G\. WangFrom recall to forgetting: benchmarking long\-term memory for personalized agents\.External Links:2604\.20006,[Link](https://arxiv.org/abs/2604.20006)Cited by:[§2\.2](https://arxiv.org/html/2611.25235#S2.SS2.p1.1),[§2\.3](https://arxiv.org/html/2611.25235#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2611.25235#S2.T1.2.7.1)\.
- Wanget al\.\(2023\)X\. Wang, T\. Chen, Q\. Ge, H\. Xia, R\. Bao, R\. Zheng, Q\. Zhang, T\. Gui, and X\. HuangOrthogonal subspace learning for language model continual learning\.External Links:2310\.14152,[Link](https://arxiv.org/abs/2310.14152)Cited by:[§5\.2](https://arxiv.org/html/2611.25235#S5.SS2.p1.1)\.
- Wanget al\.\(2024\)Y\. Wang, Y\. Gao, X\. Chen, H\. Jiang, S\. Li, J\. Yang, Q\. Yin, Z\. Li, X\. Li, B\. Yin, J\. Shang, and J\. McAuleyMEMORYLLM: towards self\-updatable large language models\.External Links:2402\.04624,[Link](https://arxiv.org/abs/2402.04624)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Wanget al\.\(2025a\)Y\. Wang, D\. Krotov, Y\. Hu, Y\. Gao, W\. Zhou, J\. McAuley, D\. Gutfreund, R\. Feris, and Z\. HeM\+: extending memoryllm with scalable long\-term memory\.External Links:2502\.00592,[Link](https://arxiv.org/abs/2502.00592)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Wanget al\.\(2025b\)Y\. Wang, X\. Liu, X\. Chen, S\. O’Brien, J\. Wu, and J\. McAuleySelf\-updatable large language models by integrating context into model parameters\.External Links:2410\.00487,[Link](https://arxiv.org/abs/2410.00487)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1),[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Wuet al\.\(2025\)D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. YuLongMemEval: benchmarking chat assistants on long\-term interactive memory\.External Links:2410\.10813,[Link](https://arxiv.org/abs/2410.10813)Cited by:[§2\.3](https://arxiv.org/html/2611.25235#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2611.25235#S2.T1.2.6.1)\.
- Xuet al\.\(2026\)Z\. Xu, H\. Hong, L\. Yu, B\. Cui, L\. Huang, H\. Xue, and N\. ZhangHow lora remembers? a parametric memory law for llm finetuning\.External Links:2605\.30260,[Link](https://arxiv.org/abs/2605.30260)Cited by:[§3\.3](https://arxiv.org/html/2611.25235#S3.SS3.p1.1),[§3](https://arxiv.org/html/2611.25235#S3.p1.1)\.
- Yanet al\.\(2026\)S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan, H\. Schütze, V\. Tresp, and Y\. MaMemory\-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.External Links:2508\.19828,[Link](https://arxiv.org/abs/2508.19828)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1),[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p1.1)\.
- Yanget al\.\(2024\)S\. Yang, K\. Ning, Y\. Liu, J\. Yao, Y\. Tian, Y\. Song, and L\. YuanIs parameter collision hindering continual learning in llms?\.External Links:2410\.10179,[Link](https://arxiv.org/abs/2410.10179)Cited by:[§3](https://arxiv.org/html/2611.25235#S3.p1.1),[§5\.2](https://arxiv.org/html/2611.25235#S5.SS2.p1.1)\.
- Yuet al\.\(2023\)L\. Yu, Q\. Chen, J\. Zhou, and L\. HeMELO: enhancing model editing with neuron\-indexed dynamic lora\.External Links:2312\.11795,[Link](https://arxiv.org/abs/2312.11795)Cited by:[§5\.1](https://arxiv.org/html/2611.25235#S5.SS1.p1.1)\.
- Yuet al\.\(2020\)T\. Yu, S\. Kumar, A\. Gupta, S\. Levine, K\. Hausman, and C\. FinnGradient surgery for multi\-task learning\.External Links:2001\.06782,[Link](https://arxiv.org/abs/2001.06782)Cited by:[§A\.4](https://arxiv.org/html/2611.25235#A1.SS4.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2611.25235#S1.p1.1),[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p2.1)\.
- Yuet al\.\(2026\)Y\. Yu, L\. Yao, Y\. Xie, Q\. Tan, J\. Feng, Y\. Li, and L\. WuAgentic memory: learning unified long\-term and short\-term memory management for large language model agents\.External Links:2601\.01885,[Link](https://arxiv.org/abs/2601.01885)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p1.1),[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p1.1)\.
- Zenget al\.\(2019\)G\. Zeng, Y\. Chen, B\. Cui, and S\. YuContinual learning of context\-dependent processing in neural networks\.Nature Machine Intelligence1\(8\),pp\. 364–372\.External Links:ISSN 2522\-5839,[Link](http://dx.doi.org/10.1038/s42256-019-0080-x),[Document](https://dx.doi.org/10.1038/s42256-019-0080-x)Cited by:[§5\.3](https://arxiv.org/html/2611.25235#S5.SS3.p2.1)\.
- Zhanget al\.\(2026\)L\. Zhang, L\. Zhang, S\. Shi, X\. Chu, and B\. LiLoRA\-fa: efficient and effective low rank representation fine\-tuning\.External Links:2308\.03303,[Link](https://arxiv.org/abs/2308.03303)Cited by:[§3\.1](https://arxiv.org/html/2611.25235#S3.SS1.p1.1)\.
- Zouet al\.\(2025\)H\. Zou, Y\. Zang, W\. Xu, Y\. Zhu, and X\. JiFlyLoRA: boosting task decoupling and parameter efficiency via implicit rank\-wise mixture\-of\-experts\.External Links:2510\.08396,[Link](https://arxiv.org/abs/2510.08396)Cited by:[§1](https://arxiv.org/html/2611.25235#S1.p2.1),[§3\.1](https://arxiv.org/html/2611.25235#S3.SS1.p1.1),[§5\.2](https://arxiv.org/html/2611.25235#S5.SS2.p1.1)\.

## Appendix ASubstrate Details

This appendix expands the substrate measurements, routing mechanics, retrieval adapter training, and continual\-learning comparison summarized in Section[3](https://arxiv.org/html/2611.25235#S3)\.

### A\.1Rank andkkSelection Data

Table[6](https://arxiv.org/html/2611.25235#A1.T6)shows the full metadata oracle selection acrossRRandkkwith per\-class VAL/ABS breakdown\. All configurations use real substrate with per\-rankτ\\tauand each measurement is over55held\-out users \(14521452queries\)\.

Table 6:Full metadata oracle selection across substrate rankRRand mask sizekk, with VAL/ABS split per class\. The value\-abstain trade\-off is visible in both dimensions\. Increasingkkgenerally shifts volatile and revision\-heavy classes toward higher VAL at the cost of ABS, while increasingRRredistributes between classes without changing aggregate substantially aboveR=32R=32\.The per\-class VAL/ABS decomposition surfaces two mechanisms:

#### Increasingkkshifts the value\-abstention trade\-off\.

CONTESTED VAL climbs from0\.850\.85atk=2k=2to1\.001\.00atk=4k=4and above, but CONTESTED ABS falls from0\.270\.27to0\.060\.06\. The same pattern is observed for TRANSIENT \(VAL0\.46→0\.970\.46\\rightarrow 0\.97, ABS0\.80→0\.090\.80\\rightarrow 0\.09\)\. Largerkkmeans each write touches more columns, so facts are more likely to persist under any query \(raising VAL\) but also less likely to expire correctly \(lowering ABS\)\. Atk=16k=16, the substrate is essentially always writing to overlapping columns and per\-factτ\\tauhas been almost entirely overwritten\.

#### IncreasingRRprimarily improves STABLE\.

STABLE VAL climbs from0\.520\.52atR=16R=16to0\.820\.82atR=128R=128, a3030\-point improvement\. This is exactly the mechanism we expect as more rank columns means each STABLE fact is less likely to share columns with subsequently\-written short\-τ\\taufacts\. However, ROUTINE and CONTESTED VAL both drop asRRgrows because their routing dispersion increases \(the router allocates similar facts to more distinct columns\)\. The trade\-off produces near\-flat aggregate accuracy aboveR=32R=32, at which pointR=32R=32is preferred for its22–4×4\\timessmaller inner\-loop compute per write\.

#### Dynamic router mechanics\.

The dynamic router projects each fact’s routing embedding through a learned whitening transform \(fitted to dimension256256\), then compares it by cosine similarity against the accumulated signature of every currently allocated mask\. A mask’s signature is maintained as anL2L\_\{2\}\-normalized exponential moving average \(α=0\.5\\alpha=0\.5\) of the embeddings previously routed there\. If the highest similarity exceeds a threshold of0\.850\.85, the write is routed to that mask and its signature is updated, otherwise a new mask is allocated, with up to\(Rk\)\\binom\{R\}\{k\}locations total\.

### A\.2Substrate Dynamics Measurements

#### Retrieval after first write, full curve\.

Table[7](https://arxiv.org/html/2611.25235#A1.T7)extends Figure/Table 2 \(main text\) to all tested depths\.

Table 7:Retrieval after first writeP⁡\(value\)P\(\\text\{value\}\)per depth\.
#### Accumulation under repeated writes\.

Table[8](https://arxiv.org/html/2611.25235#A1.T8)supports the constructive\-accumulation claim of §[3\.1](https://arxiv.org/html/2611.25235#S3.SS1)\. Five writes atd=20d=20\(P≥0\.93P\\geq 0\.93\) substantially exceed a single write at the same depth \(P≈0P\\approx 0atd=20d=20per the table above\), and approach the retention of a single deep write in thed=80d=80–110110range\.

Table 8:Cumulative retentionP⁡\(value\)P\(\\text\{value\}\)afternnwrites at depthd=20d=20with minimal intervening foreign traffic\.
#### Contradict\-primed overwrite\.

Table[9](https://arxiv.org/html/2611.25235#A1.T9)characterizes overwrite behavior \(§[3\.4](https://arxiv.org/html/2611.25235#S3.SS4)\): a contradicting value written atd≥40d\\geq 40dominates retrieval under both routing configurations\.

Table 9:Contradict overwrite\. Initial fact is written atd=40d=40with valuev1v\_\{1\}\. The overwrite is a subsequent write with valuev2v\_\{2\}at varying depths\.P⁡\(v2\)P\(v\_\{2\}\)is the retrieval probability of the overwrite\.
#### Priming and decay validation\.

To validate that per\-rank exponential decay behaves as designed, we run a simple test\. A fact is written at depthddwithτ=1000\\tau=1000, decayed forΔ​t\\Delta tdays, and retrieved\. Measured retention matches predictedP\(cold,d\)⋅exp\(−Δt/τ\)P\(\\text\{cold\},d\)\\cdot\\exp\(\-\\Delta t/\\tau\)within measurement noise ford∈\{40,80,110\}d\\in\\\{40,80,110\\\}andΔ​t∈\{10,100,300\}\\Delta t\\in\\\{10,100,300\\\}days\.

#### Slot norm regression details\.

The slot norm regression takes as input a five\-dimensional standardized feature vector: self steps, foreign steps,log\(1\+self steps\)\\log\(1\+\\text\{self steps\}\),log⁡\(1\+foreign steps\)\\log\(1\+\\text\{foreign steps\}\), and time since last write\. Cross\-validatedR2R^\{2\}is0\.870\.87perfect routing and0\.750\.75on the learned router\. The dynamic router’s lower predictability reflects that its slot signatures drift with fact allocation, adding noise the regression cannot capture\. This regression is what allows offline controller training to substitute predicted slot norm for a measured one \(§[4\.2](https://arxiv.org/html/2611.25235#S4.SS2)\), at a15×15\\timesspeedup\.

#### Forward pass test\.

Before each write, we computeP⁡\(value∣current slot state\)−P⁡\(value∣clean slot\)P\(\\text\{value\}\\mid\\text\{current slot state\}\)\-P\(\\text\{value\}\\mid\\text\{clean slot\}\)This provides a substrate\-derived signal of whether the incoming value is already represented in the routed slot\. The difference is positive for reinforcement events and near zero for first\-write events\. AUC against the ground\-truth reinforcement labels is0\.990\.99on both routers\.

#### Cost and latency\.

Table[10](https://arxiv.org/html/2611.25235#A1.T10)compares PMM with RAG\-MM, measured on55users\. Reads take a constant9494ms regardless of how many facts are stored, versus360360ms for RAG\-MM, whose cost grows with the retrieved context\. Writes are more expensive and depend on depth \(1\.81\.8s atd=20d=20to13\.113\.1s atd=150d=150, versus2\.42\.4s\), but they can run asynchronously, off the query path\. Over a full user trajectory, PMM costs702702s against617617s \(\+14%\+14\\%\), and is cheaper for22of the55users\.

Table 10:End\-to\-end timing \(55users\)\.

### A\.3Retrieval Adapter Training

We train a rank\-3232retrieval adapter jointly loaded with the memory adapter during both writes and reads\. The training data is a mixed recipe: some facts get shallow writes \(d∈\{10,15,20\}d\\in\\\{10,15,20\\\}\), some medium \(d∈\{30,40\}d\\in\\\{30,40\\\}\), some deep \(d∈\{60,80\}d\\in\\\{60,80\\\}\), some very deep \(d∈\{100,130\}d\\in\\\{100,130\\\}\), and some slots are left empty\. For each written fact, we sample the target asUnknownwith probabilitypunk​\(d\)p\_\{\\text\{unk\}\}\(d\)or the correct value otherwise\.

Table[11](https://arxiv.org/html/2611.25235#A1.T11)shows the final noise curve alongside the implied target retrieval probability\. The constraintpunk​\(d\)≤0\.55p\_\{\\text\{unk\}\}\(d\)\\leq 0\.55prevents the retrieval from collapsing to always\-abstain at any depth\.

Table 11:Retrieval adapter noise curve\. Each depth targets a specificP⁡\(value\)P\(\\text\{value\}\)during SFT\.
### A\.4Continual Learning Baseline Details

We evaluate two continual\-learning methods against our substrate, all applied to LoRA writes at fixed depthd=15d=15\(chosen fromd∈\{15,30,60\}d\\in\\\{15,30,60\\\}for each method\)\.d=15d=15maximized each method’s aggregate accuracy\. All CL baselines use the same base model and LoRA \(rank3232onqprojq\_\{\\text\{proj\}\},vprojv\_\{\\text\{proj\}\}\) but with fixed depth and no per\-rankτ\\tau\.

#### Naive fine\-tuning\.

Direct backprop on the LoRA parameters at each write event, no regularization\. Serves as the floor baseline for LoRA\-based approaches\.

#### EWC \(Elastic Weight Consolidation\)\.

Adds a Fisher\-weighted penalty term to write losses:ℒtotal=ℒwrite\+λ​∑iFi​\(θi−θi∗\)2\\mathcal\{L\}\_\{\\text\{total\}\}=\\mathcal\{L\}\_\{\\text\{write\}\}\+\\lambda\\sum\_\{i\}F\_\{i\}\(\\theta\_\{i\}\-\\theta\_\{i\}^\{\\ast\}\)^\{2\}whereFiF\_\{i\}is the diagonal Fisher information at parameteriiestimated from the calibration data,θi∗\\theta\_\{i\}^\{\\ast\}is the pre\-write parameter, andλ=1000\\lambda=1000\(from\([Kirkpatrick et al\., 2017](https://arxiv.org/html/2611.25235#bib.bib9)\)\)\. Fisher is updated with exponential moving average,α=0\.9\\alpha=0\.9\.

#### PCGrad \(Projected Conflicting Gradients\)\.

Maintains a running memory of thek=5k=5most recent write gradients\. Each new write gradient is projected to be orthogonal to any stored gradient with which it has negative cosine similarity, following\([Yu et al\., 2020](https://arxiv.org/html/2611.25235#bib.bib11)\)\. This is designed to prevent new writes from destroying prior ones\.

#### Retrieval adapter compatibility\.

CL baselines cannot use our retrieval adapter directly because the resoute was trained assuming thekk\-hot mask retrieval\. For fair comparison, we retrained a compatible retrieval adapter for each CL baseline that reads from the full LoRA weight rather than a masked subset\. This ensures accuracy differences reflect substrate design, not reading calibration mismatches\.

## Appendix BLearned Memory Management Details

This appendix expands the controller architecture, expert policy, training details, per\-class breakdowns, retrieval baseline architecture, and negative results summarized in Section[4](https://arxiv.org/html/2611.25235#S4)\.

### B\.1Full VAL/ABS Breakdown

Table[12](https://arxiv.org/html/2611.25235#A2.T12)shows the per\-pattern accuracy split by target type \(VAL versus ABS\) for the systems in Table[3](https://arxiv.org/html/2611.25235#S4.T3)and Appendix[B\.6](https://arxiv.org/html/2611.25235#A2.SS6)\.

Table 12:Per\-pattern accuracy split into VAL and ABS query types\.n=15n=15held\-out users\.
### B\.2Unmanaged Baselines Withoutτ\\tau

For completeness, we ran the unmanaged baseline depth selections both with per\-rankτ\\tau\(main table\) and withoutτ\\tau\(this table\)\.

Table 13:Unmanaged baselines withoutτ\\tauare nearly identical to their with\-τ\\taucounterparts\.The near\-identical aggregate numbers \(all within±0\.01\\pm 0\.01CI overlap\) demonstrate an important point thatτ\\taualone is not what enables our controller’s performance\. The controller’s benefit comes from combining per\-factτ\\taupolicy with per\-fact depth and gate decisions\.

### B\.3Retrieval Baseline Details

All retrieval baselines share three components: an external text store containing the naturalistic realization of each event, a BGE\-small embedder \(3333M parameters\) for indexing and query\-time retrieval, and a generator\. Because the data is an event stream, no text chunking is required and facts are embedded directly as individual, timestamped strings \(e\.g\.,\[Clock: 0045\] I cook dinner\)\. At query time, the system retrieves the topk=5k=5most similar entries\. The generator prompt explicitly instructs the model to read the retrieved timestamped entries and outputUnknownif the memory does not clearly support an answer or appears outdated\.

The variants differ only in how the store is maintained:

- •RAG:the standard baseline\. Events are appended to the store as they arrive and no management is applied\. Retrieval at query time returns the top\-kkmost similar entries\.
- •RAG\-MM:the store is actively managed at write time by an LLM management pass\. As each event arrives, the pass resolves its relationship to existing entries—merging reinforcements of the same fact, superseding contradicted values, expiring entries past their inferred validity, and deduplicating\. It receives the new fact and retrieved candidates, and outputs a JSON decision to either ADD, UPDATE \(soft\-deleting the old entry\), REINFORCE \(merging and extending lifespan\), or IGNORE\. It also assigns an estimated decay in days \(e\.g\., Transient:7\.07\.0, Routine:30\.030\.0, Stable:3650\.03650\.0\)\.
- •RAG \+ oracle metadata:identical to RAG\-MM except the LLM management pass is replaced by the ground\-truth lifecycle policy, using each event’s generation\-time metadata \(validity horizon, contradiction and reinforcement status\) to maintain the store perfectly\. This removes lifecycle\-management error while retaining the same retrieval and generation pipeline, providing an upper bound on retrieval\-side memory management under privileged lifecycle information\.

#### Generators\.

Compute\-matched rows \(labeled1\.71\.7B\) use Qwen3\-1\.71\.7B, the same base model as our substrate\. Large\-generator rows \(labeled122122B\) use Qwen 3\.5122122B A10B\.

### B\.4Controller Architecture Details

The learned management system is a 2M\-parameter neural network consisting of two primary modules: a State Representation Engine \(which maintains trajectory context and extracts semantic features\) and the Action Policy \(which predicts lifecycle management decisions\)\. All internal representations use a hidden dimension ofd=256d=256\.

#### 1\. State Representation Engine

The representation engine processes the raw outputs of the base language model into dense, controller\-ready features\. Letxt∈ℝ2048x\_\{t\}\\in\\mathbb\{R\}^\{2048\}be the last\-token embedding of the incoming event template from the base model\.

1. 1\.Semantic Projection:The raw embedding is normalized via a fitted input statistics layer and projected to a dense representation to decouple it from lexical noise:zt=MLPsem​\(xt\)∈ℝ256z\_\{t\}=\\text\{MLP\}\_\{\\text\{sem\}\}\(x\_\{t\}\)\\in\\mathbb\{R\}^\{256\}\.
2. 2\.Trajectory GRU:To capture user\-specific behavioral context, a Gated Recurrent Unit \(GRU\) maintains a hidden stateht∈ℝ256h\_\{t\}\\in\\mathbb\{R\}^\{256\}\. At each event, the GRU consumes the dense embeddingztz\_\{t\}and the token\-level forward pass result \(probability of fact before writing\)phit∈\[0,1\]p\_\{\\text\{hit\}\}\\in\[0,1\]\(indicating if the model already knows the fact\): ht=GRUCell​\(xin=\[zt⊕phit\],ht−1\)h\_\{t\}=\\text\{GRUCell\}\(x\_\{\\text\{in\}\}=\[z\_\{t\}\\oplus p\_\{\\text\{hit\}\}\],\\,h\_\{t\-1\}\)

#### 2\. Auxiliary State Heads

Before computing actions, the network predicts the fact’s structural properties using one auxiliary head attached to the representation engine\. Because this property depend only on semantics \(not the substrate\), its input is detached from the computational graph to prevent auxiliary gradients from corrupting the shared representation trunk\.

- •Stability Head:A scalar prediction of the fact’s true stability score, optimized via Mean Squared Error \(MSE\):s^t=σ⁡\(Linear​\(zt⊕ht\)\)∈\(0,1\)\\hat\{s\}\_\{t\}=\\sigma\(\\text\{Linear\}\(z\_\{t\}\\oplus h\_\{t\}\)\)\\in\(0,1\)\.

#### 3\. Action Policy and Shared Trunk

The main controller determines the action tuple\(gate,depth,τ\)\(\\text\{gate\},\\text\{depth\},\\tau\)by observing both the semantic trajectory and the state of the underlying LoRA substrate\. The input to the shared policy trunk is the concatenation of six feature families:

spolicy=\[zt⊕ht⊕A⊕‖Wslot‖2⊕s^t⊕Frank⊕phit\]s\_\{\\text\{policy\}\}=\\Big\[z\_\{t\}\\oplus h\_\{t\}\\oplus A\\oplus\|\|W\_\{\\text\{slot\}\}\|\|\_\{2\}\\oplus\\hat\{s\}\_\{t\}\\oplus F\_\{\\text\{rank\}\}\\oplus p\_\{\\text\{hit\}\}\\Big\]whereA∈ℝRA\\in\\mathbb\{R\}^\{R\}are the rank ages,‖Wslot‖2\|\|W\_\{\\text\{slot\}\}\|\|\_\{2\}is the predicted or measuredL2L\_\{2\}norm of the active LoRA mask, andFrankF\_\{\\text\{rank\}\}are the substrate interference features \(foreign and self steps\)\.

#### 4\. Decision Heads and Priors

The policy statespolicys\_\{\\text\{policy\}\}is passed through independent linear heads to produce the final action logits\. The model utilizes empirical priors to stabilize training and guide the depth decision:

- •Gate Head \(g∈\{0,1\}g\\in\\\{0,1\\\}\):Outputs a single logitlgatel\_\{\\text\{gate\}\}\. To prevent the gate from collapsing to00early in training, the logit is initialized with an empirical biasbgateb\_\{\\text\{gate\}\}derived from the training set’s marginal write rate:P⁡\(write\)=σ⁡\(lgate\+bgate\)P\(\\text\{write\}\)=\\sigma\(l\_\{\\text\{gate\}\}\+b\_\{\\text\{gate\}\}\)\.
- •Persistence Head \(τ∈\{25,100,400,1600,20000\}\\tau\\in\\\{25,100,400,1600,20000\\\}\):Outputs a 5\-dimensional logit vectorlτl\_\{\\tau\}, shifted by empirical class frequency biases to account for the heavy skew toward shorter decay horizons\.
- •Depth Head \(d∈𝒟\>0d\\in\\mathcal\{D\}\_\{\>0\}\):Outputs a 7\-dimensional logit vectorldepthl\_\{\\text\{depth\}\}over the available non\-zero write depths\{20,30,40,60,80,110,150\}\\\{20,30,40,60,80,110,150\\\}\. The depth logits are directly augmented by the auxiliary head, acting as a learned prior\. ldepthfinal=ldepth\+bempirical\+α1​\(s^t​Vstab\)l\_\{\\text\{depth\}\}^\{\\text\{final\}\}=l\_\{\\text\{depth\}\}\+b\_\{\\text\{empirical\}\}\+\\alpha\_\{1\}\(\\hat\{s\}\_\{t\}V\_\{\\text\{stab\}\}\)whereVstabV\_\{\\text\{stab\}\}is a vector pushing high\-stability predictions toward deeper bins\.

### B\.5Training Dynamics and Loss Formulation

Training a shared representation trunk alongside multiple action heads presents significant optimization challenges\. To stabilize Behavioral Cloning \(Phase 1\) and ensure smooth convergence, we employ three specific training techniques\.

#### 1\. Ordinal Soft Targets for Depth Classification

Standard cross\-entropy loss heavily penalizes any deviation from the target class\. However, write depthddis an ordinal variable where adjacent bins represent similar retention strengths\. To reflect this, the magnitude head is trained via Kullback\-Leibler \(KL\) divergence against ordinal soft targets\. If the expert deficit calculation targets depthd∗d^\{\*\}, the target distribution smears probability mass across adjacent available bins according to proximity, rather than placing a one\-hot1\.01\.0on the nearest bin\. This ensures that near\-miss predictions \(e\.g\., predictingd=80d=80when the target is110110\) are penalized less than catastrophic under\-writes\.

#### 2\. DAgger Expert Annealing

During Phase 1, we use DAgger\([Ross et al\., 2011](https://arxiv.org/html/2611.25235#bib.bib30)\)to supervise the policy over the substrate\. The probabilityβ\\betaof executing the expert’s target action versus the student policy’s sampled action follows a linear decay schedule\. For the first5%5\\%of training \(warmup\),β=0\.95\\beta=0\.95, ensuring the substrate remains sound while the trunk learns\. Over the remainder of training,β\\betadecays linearly to0\.300\.30, forcing the student to increasingly manage the consequences of its own imperfect writes\.

#### Total Phase 1 Loss Formulation

The complete multi\-task objective during Behavioral Cloning is:

ℒtotal=\\displaystyle\\mathcal\{L\}\_\{\\text\{total\}\}=\{\}wctrl​\(λg​ℒBCE​\(gate\)\+λd​ℒKL​\(depth\)\+λτ​ℒCE​\(τ\)\)\\displaystyle w\_\{\\text\{ctrl\}\}\\Big\(\\lambda\_\{g\}\\mathcal\{L\}\_\{\\text\{BCE\}\}\(\\text\{gate\}\)\+\\lambda\_\{d\}\\mathcal\{L\}\_\{\\text\{KL\}\}\(\\text\{depth\}\)\+\\lambda\_\{\\tau\}\\mathcal\{L\}\_\{\\text\{CE\}\}\(\\tau\)\\Big\)\+λs​ℒMSE​\(stability\)\\displaystyle\+\\lambda\_\{s\}\\mathcal\{L\}\_\{\\text\{MSE\}\}\(\\text\{stability\}\)wherewctrlw\_\{\\text\{ctrl\}\}is the warmup schedule scaler, andλ\\lambdaare static hyperparameter weights balancing the relative entropy of the action spaces\.

### B\.6Full Results: Additional Baselines

Table[14](https://arxiv.org/html/2611.25235#A2.T14)gives the remaining systems omitted from Table[3](https://arxiv.org/html/2611.25235#S4.T3)for space\.

Table 14:Additional baselines omitted from Table[3](https://arxiv.org/html/2611.25235#S4.T3)for space: weaker continual\-learning variants and retrieval given ground\-truth lifecycle metadata in place of a learned or LLM\-managed store\.
### B\.7Expert Policy Details

The deficit\-based expert \(capped\_deficit\) computes, for each write event, a scalar deficit and returns an action tuple\.

Algorithm 1Deficit\-based expert policy1:Inputs:stability score

ss,is\_reinforcementflag

rr, base decay

bb, self\-steps on mask

SS, foreign\-steps on mask

FF, depth bins

\{di\}\\\{d\_\{i\}\\\}, foreign threshold

F∗F^\{\\ast\}, tolerance

ϵ\\epsilon, per\-event cap

C=150C=150\.

2:Compute target retention depth

D∗=D∗​\(s,b,r\)D^\{\\ast\}=D^\{\\ast\}\(s,b,r\)based on substrate curves\.

3:Compute effective accumulated depth

Dacc=S−γ​FD\_\{\\text\{acc\}\}=S\-\\gamma Fwith foreign discount

γ=min⁡\(1,F/F∗\)\\gamma=\\min\(1,F/F^\{\\ast\}\)\.

4:Compute deficit

Δ=D∗−Dacc\\Delta=D^\{\\ast\}\-D\_\{\\text\{acc\}\}\.

5:if

Δ≤ϵ\\Delta\\leq\\epsilonthen

6:return

\(g=0,d=0,τ=0\)\(g=0,d=0,\\tau=0\)\{decline\}

7:endif

8:Set write depth

d∗=min⁡\(Δ,C\)d^\{\\ast\}=\\min\(\\Delta,C\)\.

9:Snap

d∗d^\{\\ast\}to nearest available bin\.

10:Set

τ∗=arg⁡minb​\|log⁡τb−log⁡\(24\.5⋅b\)\|\\tau^\{\\ast\}=\\arg\\min\_\{b\}\|\\log\\tau\_\{b\}\-\\log\(24\.5\\cdot b\)\|\.

11:return

\(g=1,d=d∗,τ=τ∗\)\(g=1,d=d^\{\\ast\},\\tau=\\tau^\{\\ast\}\)\.

The foreign thresholdF∗F^\{\\ast\}is calibrated per machine as the7575th percentile ofFFover training\-set write events, producing roughly25%25\\%of writes triggering non\-trivial foreign discount\.

The target depth functionD∗​\(s,b,r\)D^\{\\ast\}\(s,b,r\)interpolates the substrate retrieval curve to determine what accumulated depth is needed for a fact of stabilityssand base decaybbto remain retrievable through its expected query horizon\. Reinforcement events use a slightly higherD∗D^\{\\ast\}than first\-write events to preserve accumulation opportunity for future queries\.

### B\.8State Features

The controller conditions on six feature families summarized in Section[4\.2](https://arxiv.org/html/2611.25235#S4.SS2)\. Full details:

#### Dense semantic embedding\.

A projection of the fact template’s last\-token representation from the base model, passed through an input normalization and a small MLP \(sem\_proj\) that decorrelates raw LM features from lexical surface form\. Dimension256256\.

#### GRU hidden state\.

A recurrent summary of the trajectory so far, updated per event\. Takes the current fact’s dense embedding, event type indicator, and time delta since the previous event\. Provides temporal context that a bag\-of\-features representation would lose\. Dimension256256\.

#### Rank ages\.

For each of theRRrank columns, the time in days since that column was last written\. Long\-inactive columns are candidates for aggressive writes because their existing content has already decayed; recently touched columns carry current content that will be interfered with\. DimensionRR\.

#### Slot norm\.

TheL2L\_\{2\}norm of the mask\-restricted LoRA weights, either measured directly on the substrate \(during online training and evaluation\) or predicted from ledger state via the fitted regression from Appendix[A\.2](https://arxiv.org/html/2611.25235#A1.SS2)\(during offline training, for a15×15\\timesspeedup\)\. Dimension11\.

#### Predicted stability\.

A scalar prediction from a dedicated head trained via MSE against the ground\-truth stability score\. Enables the controller to condition depth and gate decisions on predicted fact durability without needing symbolic access to metadata at inference\.

#### Mask\-level substrate features and forward\-pass signal\.

For the specific mask that would receive this write, mean rank age within the mask, mean foreign\-step count within the mask, and the forward pass test hit \(Appendix[A\.2](https://arxiv.org/html/2611.25235#A1.SS2)\)\. The forward pass provides a substrate\-derived signal of whether the incoming value is already represented in the current slot, complementing the rank\-age and interference features\.AUC against ground\-truth reinforcement labels reaches 0\.99\. DimensionNrank feats\+1N\_\{\\text\{rank feats\}\}\+1\.

### B\.9Tau Head

Theτ\\tauhead is a five\-way classifier over\{25,100,400,1600,20000\}\\\{25,100,400,1600,20000\\\}days\. For a fact with base decaybb, the target lifespan class is

τ∗​\(b\)=arg⁡minτj∈𝒯​\|log⁡τj−log⁡\(24\.5​b\)\|,\\tau^\{\\ast\}\(b\)=\\arg\\min\_\{\\tau\_\{j\}\\in\\mathcal\{T\}\}\\left\|\\log\\tau\_\{j\}\-\\log\(24\.5\\,b\)\\right\|,
where𝒯=\{25,100,400,1600,20000\}\\mathcal\{T\}=\\\{25,100,400,1600,20000\\\}days\.

#### Class weighting\.

Theτ\\tauhead uses inverse\-frequency class weights since the target distribution is more skewed\. Bin00\(τ=25\\tau=25\) receives roughly46%46\\%of events, bin11\(τ=100\\tau=100\) receives10%10\\%, bin33\(τ=1600\\tau=1600\) receives2%2\\%\. Inverse\-frequency weighting maintains recall on rare bins at the cost of noisier training signal\.

### B\.10Empirical Bias Initialization

Each action head’s output bias is initialized tolog⁡\(f\)\\log\(f\)whereffis the marginal frequency of each action in the expert policy on a calibration set of150150users\. Table[15](https://arxiv.org/html/2611.25235#A2.T15)shows the initialized values\.

Table 15:Empirical bias initialization from expert marginals on150150\-user calibration set\.The depth bias is heavily negative on rare bins \(e\.g\.,d=30d=30at−4\.47\-4\.47reflects that only1%1\\%of expert writes use this depth\)\. The controller learns to adjust these biases per fact context and the initialization ensures early training does not have to first recover the marginal distribution\.

### B\.11Lifecycle\-Class Depth Prior

The prior matrixW∈ℝ5×7W\\in\\mathbb\{R\}^\{5\\times 7\}maps predicted lifecycle\-class probabilities to depth\-preference adjustments\.

Table 16:Pattern\-conditional depth prior matrixWW\. Each row is mean\-centered so the prior does not shift aggregate write rate\. The prior is combined with the pattern head’s softmax asαpattern⋅ppattern⋅W\\alpha\_\{\\text\{pattern\}\}\\cdot p\_\{\\text\{pattern\}\}\\cdot Wand added to the depth head’s raw logits\.A parallel stability\-conditional depth prior encourages the depth head toward deep bins when predicted stability is high, using a seven\-dimensional vector\[−1,−1,−0\.5,0,0\.5,1\.5,1\.5\]\[\-1,\-1,\-0\.5,0,0\.5,1\.5,1\.5\]over the depth bins, scaled byαstab\\alpha\_\{\\text\{stab\}\}and the predicted stability scalar\.

### B\.12Per\-Router Hyperparameters

Building on the bias initialization \(Appendix[B\.10](https://arxiv.org/html/2611.25235#A2.SS10)\) and depth priors \(Appendix[B\.11](https://arxiv.org/html/2611.25235#A2.SS11)\) above, this section gives the remaining per\-model hyperparameters\. The prior scales differ between routers due to different substrate retrieval curves\.

Table 17:Hyperparameters for the two model configurations\. PMM\-LR uses smaller prior scales because dynamic routing’s permissive shallow\-bin dynamics would otherwise push the controller to over\-invest in deep writes\.
### B\.13Stability Head Pretraining

The stability head is pretrained via MSE regression against ground\-truth stability scores over template\-stability pairs from the training user pool\. On920920unseen templates \(held\-out from3434users not seen during training\), Stage A pretraining reachesR2=0\.22R^\{2\}=0\.22at epoch1010, peaking slightly before plateauing\.

The template\-only prediction ceiling \(best any model could achieve from the template embedding alone\) is approximately0\.360\.36\. The gap from 0\.22 to 0\.36 is due to residual variance the head cannot extract from the embedding alone\. Meanwhile, the gap from 0\.36 to 1\.00 represents the limit of this feature as identical fact templates have different true stabilities across different users depending on their personal patterns\. This limit flows downstream to theτ\\tauhead, which reads from the stability prediction\.

### B\.14STABLE Analysis Details

Per\-query diagnostics on STABLE VAL outcomes were collected on5656STABLE VAL queries across a55\-user analysis sample using the PMM\-LR controller\.

Semantic GenerationPhysics & EnforcementPhase 1: LLM Gen\.Motif Sampling & MaskingRaw Masked LedgersLLM PruningStorageD​e​c​a​ye​f​f=D0×1\.8s​t​r​e​a​kDecay\_\{eff\}=D\_\{0\}\\times 1\.8^\{streak\}Trajectory ExpansionConstructive AccumulationQuery InjectorPlacement via Decay CurveFinal DatasetFresh:t<De​f​ft<D\_\{eff\}⟹\\impliesVALUE TargetStale:t\>De​f​ft\>D\_\{eff\}⟹\\impliesABSTAIN TargetFigure 5:Overall dataset generation pipeline\. Phase 1 grounds the data via LLM generation, while Phase 2 strictly enforces temporal dynamics, constructive accumulation, and delay\-calibrated query injection\.Table 18:Per\-query diagnostics on STABLE VAL\. Successes and failures similar on writing behavior \(depth, prior writes\) but differ dramatically on query horizon and foreign\-to\-self ratio at query time\.The distinguishing features between success and failure are query horizon and foreign\-to\-self ratio at query time\. Successful STABLE queries fire on average18\.518\.5days after last write; failed queries fire536\.7536\.7days after\. Failed queries suffered from a2\.7×2\.7\\timeshigher foreign\-to\-self gradient ratio\. This was not caused by encountering more absolute foreign traffic \(means 1205 vs\. 1128\), but by the stable fact receiving far fewer reinforcing writes to defend its own trace against that traffic \(57 vs\. 124\)\.

#### Max\-τ\\tauablation\.

An alternativeτ\\tauupdate rule that preserves the longestτ\\tauany fact has assigned to a column \(rather than overwriting\) protects STABLE facts on shared columns from subsequent shorter\-τ\\tauwrites\. This variant achieves class\-balanced accuracy0\.450\.45\(55\-user eval\), with STABLE VAL improving by∼13\\sim 13points but ABS accuracy collapsing across TRANSIENT, ROUTINE, CONTESTED, and ONESHOT \(each below0\.050\.05\)\. TRANSIENT facts written to shared columns accidentally inherit the longτ\\tauof older STABLE facts\. Consequently, these volatile facts never decay, causing the system to provide outdated answers instead of correctly abstaining during stale queries\. This shows that the trade\-off cannot be resolved by this simple max\-τ\\tauupdate rule and reflects the coupling introduced by shared\-column compression\.

#### Reinforcement effect\.

STABLE facts written multiple times over their trajectory re\-assert longτ\\tauon their columns and refresh self\-investment against accumulated foreign traffic\. STABLE facts with3\+3\+writes hit at0\.680\.68versus0\.510\.51for single\-write STABLE\. The controller does not independently trigger reinforcement and it is driven by the upstream event stream identifying repeated observations of the same fact\.

### B\.15Phase 2 PPO Details

Phase 2 refines the phase 1 controller via PPO\. Reward is only outcome\-based for correct VAL and ABS queries\. The phase 1 checkpoint serves as the reference policy for KL regularization\.

KL divergence to reference is penalized with weight0\.100\.10on the store head and0\.020\.02on depth\. Batch size is44users per PPO iteration, with33PPO epochs and minibatch size3232\. Value coefficient0\.50\.5, auxiliary loss coefficient0\.30\.3, entropy coefficient0\.0150\.015on depth and0\.0050\.005on the store head\.

In our experiments, phase 2 provided marginal improvement over phase11\(roughly±1\\pm 1point class\-balanced\), primarily by shifting the policy’s VAL\-ABS balance without meaningfully improving aggregate\. We attribute this to phase 1 already being close to the metadata oracle \(∼86%\{\\sim\}86\\%\), leaving little room for refinement via reward signal\. We treat phase 2 as an optional stage\.

### B\.16Input Feature Ablations

To understand which information is strictly necessary for parametric memory management, we performed a set of input feature ablations\. Because the controller uses a shared representation trunk that feeds multiple heads, simply removing an architectural head could confound the results\. Instead, we ablate by zeroing out the specific input sources entirely before they reach the controller, directly testing the information bottleneck\. We evaluate four ablations against the PMM\-PR baseline:

1. 1\.No Semantic:Zeroes out the dense text embedding\.
2. 2\.No Priors:Removes the Stage A stability warm\-start and the mathematical depth\-bias\.
3. 3\.No History:Zeroes out the GRU trajectory hidden state\.
4. 4\.No Substrate:Zeroes out substrate readings \(probe\_hit,slot\_norm,foreign\_steps\)\.

Table 19:Input feature ablations\. Losing semantic features or priors causes substantial failure\. History and Substrate act as a dual\-redundancy system, but isolating them reveals specific, severe capability collapses \(e\.g\., losing history crashes ONESHOT VAL by 24 points\)\.The per\-class VAL/ABS splits \(Table[19](https://arxiv.org/html/2611.25235#A2.T19)\) reveal exactly how the controller uses its inputs, surfacing three core findings:

#### 1\. Semantic embeddings and priors provide substantial management signal\.

Without semantic embeddings, the model cannot map an incoming fact to a behavioral pattern\. Without this information, the policy falls back toward less differentiated write behavior, reducing its ability to separate persistent and volatile events\. Removing semantic features lowers CB to0\.470\.47, while removing the substrate\-derived priors lowers it to0\.520\.52\.

#### 2\. History and substrate state provide complementary information\.

The*No History*\(CB 0\.58\) and*No Substrate*\(CB 0\.58\) ablations survive in aggregate because the GRU and the LoRA state provide overlapping signals for reinforcement\. If the GRU is removed, the substrate’sprobe\_hitstill tells the controller a fact is recurring\. If the substrate is removed, the GRU’s temporal memory knows it has seen the embedding multiple times\. However, the class\-wise splits show they are not perfectly redundant:

- •History disambiguates lifecycle from trajectory context:Without the GRU, the model is blind to the timeline\. When a ONESHOT fact arrives, the model cannot confidently assign it moderate depth because it lacks the behavioral context to separate it from the first write of a ROUTINE fact\. Consequently, it hedges its bets, and ONESHOT VAL crashes by2424points \(0\.82→0\.580\.82\\rightarrow 0\.58\)\.
- •Substrate state exposes the consequences of prior writes:Without the substrate, the model cannot see collision metrics \(foreign\_steps,slot\_norm\)\. Blind to overlap, it overwrites frequently\. This leaves stale traces persisting when they should have decayed, dropping abstention \(ABS\) accuracy on volatile, highly\-reinforced classes \(ROUTINE ABS drops0\.64→0\.550\.64\\rightarrow 0\.55; CONTESTED ABS drops0\.34→0\.280\.34\\rightarrow 0\.28\)\.

#### 3\. Higher abstention under ablation reflects more conservative writing\.

STABLE and ONESHOT ABS actually improve under these ablations \(climbing to 0\.91\-0\.95\), but this is not a genuine gain as the controller compensates for information loss by defaulting to shallower writes, which decay faster and inflate ABS at the cost of VAL\.

## Appendix CDataset and Trajectory Generation Details

This appendix provides extended statistics, motif distributions, and qualitative examples for the controlled long\-horizon user trajectories introduced in Section 2\.

### C\.1Dataset Statistics

The dataset consists of909909user trajectories spanning multi\-year horizons\. Table[20](https://arxiv.org/html/2611.25235#A3.T20)summarizes the aggregate dataset statistics and the class balance across the five memory classes\. Figure[6](https://arxiv.org/html/2611.25235#A3.F6)visualizes how identical fact properties map to different classes, while Figure[7](https://arxiv.org/html/2611.25235#A3.F7)illustrates the query injection density across a fact’s decay threshold\.

Table 20:At\-a\-glance dataset statistics \(top\) and class balance distributions \(bottom\)\. Queries are intentionally reweighted to ensure critical behaviors like expiration \(TRANSIENT\) are adequately represented despite write imbalances\. Query horizon is separate from write events horizon of 700\-1500 days\.Figure 6:Two facts can have very similar properties such as stability and number of writes, yet belong to very different classes based on user history and semantic meaning\.Figure 7:Fresh versus stale query injection density\. Retrieval \(VAL\) and abstention \(ABS\) queries are placed relative to each fact’s validity horizon\.
### C\.2Motif Distribution and Cross\-User Variance

To ensure heterogeneous, user\-level memory dynamics, user personas were generated using a variety of underlying temporal motifs governing their behavior \(Table[21](https://arxiv.org/html/2611.25235#A3.T21)\)\. Because these motifs dictate behavior at the user level, the same fact type \(e\.g\., a hobby or a job\) can be stable for one user but volatile for another\. Figure[8](https://arxiv.org/html/2611.25235#A3.F8)illustrates the resulting cross\-user variance in routine predictability\.

Table 21:Distribution of the underlying temporal motifs used to govern user memory dynamics during dataset generation\.Figure 8:Cross\-user variance in routine prevalence\. The variation across users illustrates why lifespans cannot be determined solely from fact type and must also depend on individual trajectory context\.
### C\.3Qualitative Example

Table[22](https://arxiv.org/html/2611.25235#A3.T22)provides a qualitative snippet of a user trajectory demonstrating how writes, updates, and queries are interleaved\. Entities are masked to decouple semantic world knowledge from parametric retention\.

Table 22:A narrative snippet of a user trajectory demonstrating all five memory lifecycles\. The example illustrates the distinct target behaviors: TRANSIENT information is queried after its validity horizon, CONTESTED information is queried after a value revision, and STABLE information must remain retrievable across long intervening histories\. The non\-query inputs also follow the same \[MASK\]\-Output format to memorize as the query inputs but are presented in a readable form here with outputs values inserted in their respective \[MASK\] positions\.
### C\.4Benchmark Comparison Criteria

Table[1](https://arxiv.org/html/2611.25235#S2.T1)compares prior long\-term memory benchmarks along the following properties:

#### Belief revision\.

Whether later observations can supersede an earlier value and evaluation requires recovering the currently valid state\.

#### Time\-based expiry\.

Whether previously valid information can become invalid solely because its validity horizon has elapsed, without an explicit replacement, correction, or deletion event\.

#### Reinforcement\.

Whether repeated consistent observations affect subsequent memory behavior, such as strengthening retention, rather than being treated as independent events\.

#### User\-conditioned lifespan\.

Whether the appropriate lifespan of a memory depends on the individual user’s observed history rather than only on the semantic type of the information\.

#### Lifecycle supervision\.

Whether the benchmark provides event\-level ground truth describing how a memory’s validity or state evolves over time\.

We mark a property as*full*when it is explicitly constructed and evaluated,*partial*when related behavior is present but not directly isolated, and*absent*otherwise\.

### C\.5Dataset Schema and Comprehensive Trajectory Example

Because the full dataset consists of multi\-year trajectories with hundreds of events per user, we provide a detailed example here to illustrate the exact inputs, metadata, and query formats utilized by both our parametric controllers and the retrieval\-based baselines\.

Each user in the dataset is represented as a JSON object containing afacts\_metadatadictionary \(the ground\-truth for each fact\) and aneventsarray \(the chronological stream of writes and queries\)\.

#### 1\. Ground\-Truth Metadata Dictionary

Thefacts\_metadatadictionary maps each semantic fact type \(e\.g\.,favorite\_coffee\_shop\) to its ground\-truth lifecycle properties which represents a user’s belief states\. This metadata is strictly hidden from the controller at inference time but is used by the DAgger oracle during Phase 1 training to compute target depths\.

```
"facts_metadata": {
  "user_profession": {
    "write_pattern": "STABLE",
    "stability_score": 0.92,
    "base_decay_days": 3650.0
  },
  "morning_coffee_spot": {
    "write_pattern": "ROUTINE",
    "stability_score": 0.45,
    "base_decay_days": 14.0
  },
  "current_project": {
    "write_pattern": "CONTESTED",
    "stability_score": 0.65,
    "base_decay_days": 90.0
  },
  "todays_lunch": {
    "write_pattern": "TRANSIENT",
    "stability_score": 0.12,
    "base_decay_days": 2.0
  }
}
```

#### 2\. Chronological Event Stream

Theeventsarray represents the timeline\. Each event contains a timestamp, the event type \(INTRODUCE,UPDATE,CONTRADICT, orQUERY\), and the event\.

Below is an illustrative sequence showing how themorning\_coffee\_spot\(a ROUTINE fact\) andtodays\_lunch\(a TRANSIENT fact\) are processed over a 20\-day window\.

Event 1: First observation of a routine \(Day 0\)

```
{
  "clock_days": 0.0,
  "event_type": "INTRODUCE",
  "fact_type": "morning_coffee_spot",
  "value": "CAFE_4",
  "is_reinforcement": false,
  "template_input": "I ordered my morning coffee from [MASK]",
  "natural_language": "[Clock: 0000] I ordered my morning coffee
  from CAFE_4."
}
```

Event 2: A transient, unrepeated event \(Day 2\)

```
{
  "clock_days": 2.5,
  "event_type": "INTRODUCE",
  "fact_type": "todays_lunch",
  "value": "RESTAURANT_9",
  "is_reinforcement": false,
  "template_input": "I grabbed a quick lunch with Sarah at [MASK]",
  "natural_language": "[Clock: 0002] I grabbed a quick lunch with
  Sarah at RESTAURANT_9."
}
```

Event 3: Reinforcement of the routine \(Day 5\)

```
{
  "clock_days": 5.0,
  "event_type": "UPDATE",
  "fact_type": "morning_coffee_spot",
  "value": "CAFE_4",
  "is_reinforcement": true,
  "template_input": "I ordered my morning coffee from [MASK]",
  "natural_language": "[Clock: 0005] I ordered my morning coffee
  from CAFE_4."
}
```

Event 4: Stale Abstention Query for the Transient fact \(Day 15\) Becausetodays\_lunchhas a base decay of 2\.0 days, and 12\.5 days have passed since Event 2, the ground\-truth target strictly demands abstention\.

```
{
  "clock_days": 15.0,
  "event_type": "QUERY",
  "target_type": "ABSTAIN",
  "fact_type": "todays_lunch",
  "target_value": "Unknown",
  "template_input": "I grabbed a quick lunch with Sarah at [MASK]",
  "natural_language": "Where did you grab lunch with Sarah?"
}
```

Event 5: Value Retrieval Query for the Routine fact \(Day 18\) Because the routine was reinforced at Day 5, the controller should have accumulated depth, keeping the memory valid for this query 13 days later\.

```
{
  "clock_days": 18.0,
  "event_type": "QUERY",
  "target_type": "VALUE",
  "fact_type": "morning_coffee_spot",
  "target_value": "CAFE_4",
  "template_input": "I ordered my morning coffee from [MASK]",
  "natural_language": "Question: Where do you usually
  order your morning coffee?"
}
```

## Appendix DNatural\-Language Memory Management End to End

The main paper isolates memory management with a templated interface\. Here we run the*complete*system with natural\-language \(NL\) writes and queries: the same substrate, controller architecture, action space and training recipe \(§[4](https://arxiv.org/html/2611.25235#S4)\)\. Only quantities that depend on the input format are re\-measured\. On the same held\-out users, NL performance is within0\.020\.02class\-balanced \(CB\) accuracy of templated performance for the oracle, PMM\-PR and PMM\-LR \(Table[3](https://arxiv.org/html/2611.25235#S4.T3)\)\.

### D\.1Setup

Each write is expressed as1010question–answer pairs about the fact \(e\.g\.,Where did I take my dog?→\\rightarrowthe park\)\. This means that natural language interfacing requires an additional question\-answer generation module to ensure correct writes\. Model editing approaches may further help with the write step and do away with the question generation module\. A depth\-ddwrite performsddSGD steps on the routed columns, split evenly across the pairs, with the loss on answer tokens\. Queries are free\-form questions that do not repeat the write wording\. The substrate mechanics are format\-independent, but write strength and its survival under decay are not\. Table[23](https://arxiv.org/html/2611.25235#A4.T23)measures both for NL writes\. Two properties differ from the templated setting: recall plateaus early \(reliable fromd≈60d\\approx 60\), and decay acts as a cliff rather than a gradual fade \(recall is0\.880\.88–0\.960\.96atf=0\.9f=0\.9, but at most0\.190\.19atf=0\.7f=0\.7and zero atf=0\.5f=0\.5\)\.

Table 23:Recall of a single NL fact vs\. write depth and remaining decay factor \(question–answer writes, matched retrieval adapter,4848facts\)\.
### D\.2Relation to Prior Work

Extracting knowledge written into weights with free\-form questions depends on how it was written\.[Allen\-Zhu and Li \(2024\)](https://arxiv.org/html/2611.25235#bib.bib46)show that extractability requires diverse formulations of each fact during training, and[Mecklenburg et al\. \(2024\)](https://arxiv.org/html/2611.25235#bib.bib44)and[Ovadia et al\. \(2024\)](https://arxiv.org/html/2611.25235#bib.bib28)find that fine\-tuned question answering improves with paraphrase augmentation\. Our lifecycle management \(when to write, how deeply, for how long\) transfers to natural language without change to the substrate, or controller training recipe\.

相似文章

长时程语言智能体中的重尾记忆轨迹

arXiv cs.AI

该论文对长时程语言智能体的记忆使用模式进行了审计,发现其检索轨迹呈现重尾(核心-尾部)分布,并提出了 Core-Tail World Model(CTWM)——一种基于排序的记忆控制器。在 LongMemEval 上,该控制器在保持准确率不变的情况下,最多可将提示词 token 数削减 24.48%。