HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents

# Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents
Source: [https://arxiv.org/html/2609.30797](https://arxiv.org/html/2609.30797)
Zihong He1Junxiao Shen2Chen Liang1,∗Hai\-Ning Liang1,3,∗

1The Hong Kong University of Science and Technology \(Guangzhou\)

2University of Bristol

3The Hong Kong University of Science and Technology

zhe154@connect\.hkust\-gz\.edu\.cn,Junxiao\.shen@bristol\.ac\.uk chenliang2@hkust\-gz\.edu\.cn,hainingliang@hkust\-gz\.edu\.cn ∗Corresponding authors

###### Abstract

Text\-based memory and context compression support reuse of past interactions\. Resizing continuous memory changes the input to a frozen LLM, coupling capacity allocation with readout\. We propose Hard\-Origin Adaptively Softened Memory \(HasMem\)\. Frozen hard\-prompt embeddings provide a verifiable initial state\. A controller adjusts memory widths, a Writer re\-encodes resized entries, and Reader and Global provide readout adaptation and cross\-turn state\. On all535535questions in a reconstruction probe derived from the Multi\-Session Chat \(MSC\) development split, the main configuration achieves lexical F1 of95\.395\.3\(\+4\.4\+4\.4percentage points\) at93\.6%93\.6\\%of the hard reference’s framed memory positions\. With approximately matched per\-question target body budgets, six configurations at mean per\-entry retention around0\.830\.83–0\.910\.91exceed rule\-based re\-encoding by8\.08\.0–23\.623\.6exact\-match \(EM\) percentage points\. With fixed model parameters and rule target width ratio0\.750\.75, Global’s EM gain passes a user\-level exact paired test with Bonferroni correction over eight comparisons\. On all500500LongMemEval\-S questions, local lexical F1 rises from the hard reference’s3\.43\.4to8\.98\.9, and answer negative log\-likelihood \(NLL\) falls from12\.25712\.257to5\.2745\.274\. F1 gains accompany lower EM on both evaluations\.

## 1Introduction

Long\-term LLM agents retain information to answer later questions\[[57](https://arxiv.org/html/2609.30797#bib.bib1),[49](https://arxiv.org/html/2609.30797#bib.bib2)\]\. Text\-based systems manage history as notes or paged context\[[7](https://arxiv.org/html/2609.30797#bib.bib6),[37](https://arxiv.org/html/2609.30797#bib.bib5)\]\. Continuous compressors produce compact slots\[[14](https://arxiv.org/html/2609.30797#bib.bib12)\], and token compressors retain selected text\[[38](https://arxiv.org/html/2609.30797#bib.bib15)\]\. Adaptive compression selects budgets per input\[[59](https://arxiv.org/html/2609.30797#bib.bib38)\]\.

A further challenge for persistent continuous memory is revising stored records’ widths, measured in continuous positions, as interactions proceed\. Resizing changes the frozen LLM’s input, coupling capacity allocation, re\-encoding, and readout\. The frozen LLM’s token embeddings provide a verifiable hard\-prompt starting point for gradual per\-record width adaptation\.

We therefore propose*Hard\-Origin Adaptively Softened Memory*\(HasMem\), which maintains these representations in a*Soft Bank*outside the frozen LLM parametersW0W\_\{0\}\. A Controller chooses KEEP, SHRINK, or EXPAND, and a Writer re\-encodes resized entries\. KEEP preserves the slot\. Reader provides low\-rank readout adaptation, and a recurrent Global state conditions re\-encoding and readout\. A causal gate preserves hard\-prompt equivalence for all\-KEEP trajectories under identical retrieval and input construction \(Section[3\.2](https://arxiv.org/html/2609.30797#S3.SS2)\)\. Subsequent question answering supervises training, with a penalty on cumulative memory positions\. Evaluation updates use observed history and stored state before receiving the question\.

The main configuration uses frozen Qwen2\.5\-7B,k=8k=8\. We evaluate all535535reconstruction questions derived from the Multi\-Session Chat \(MSC\) development split\[[52](https://arxiv.org/html/2609.30797#bib.bib19)\]and all500500LongMemEval\-S questions\[[48](https://arxiv.org/html/2609.30797#bib.bib22)\]\. On LongMemEval\-S, local lexical F1 improves by5\.65\.6percentage points and answer NLL falls by6\.9836\.983\(Table[2](https://arxiv.org/html/2609.30797#S4.T2)\)\. Exact match \(EM\) is lower than the hard reference on both datasets\.

We make three contributions\.

1. 1\.Continuous memory with a verifiable hard origin\.HasMem initializes records from frozen LLM token embeddings and adapts their widths, preserving all\-KEEP hard\-prompt equivalence \(Section[3\.2](https://arxiv.org/html/2609.30797#S3.SS2)\)\. On MSC\-derived, the main configuration improves lexical F1 by4\.44\.4percentage points at93\.6%93\.6\\%of the hard reference’s framed positions \(Table[1](https://arxiv.org/html/2609.30797#S4.T1)\)\.
2. 2\.Learned width decisions and re\-encoding\.A controller selects record widths, and a re\-encoding module updates resized entries\. Sharing trained parameters and the re\-encoder at approximately matched per\-question target body budgets, six configurations with mean per\-entry retention around0\.830\.83–0\.910\.91exceed rule\-based re\-encoding by8\.08\.0–23\.623\.6EM percentage points \(Table[4](https://arxiv.org/html/2609.30797#S4.T4)\)\.
3. 3\.Readout adaptation with cross\-turn context\.We introduce low\-rank adaptation \(Reader\) and recurrent conditioning of re\-encoding and readout \(Global\)\. With model parameters and per\-question positions fixed, Global readout at rule targetρ=0\.75\\rho=0\.75raises question\-weighted MSC\-derived EM by2\.242\.24percentage points \(owner\-level exact paired test\[[47](https://arxiv.org/html/2609.30797#bib.bib20)\], Bonferroni\-adjustedp=0\.0146p=0\.0146over eight comparisons\[[10](https://arxiv.org/html/2609.30797#bib.bib21)\], Table[6](https://arxiv.org/html/2609.30797#S4.T6)\)\.

## 2Related Work

### 2\.1Learning\-Based Memory Management

RAG\[[26](https://arxiv.org/html/2609.30797#bib.bib23)\]and RETRO\[[1](https://arxiv.org/html/2609.30797#bib.bib24)\]generate from retrieved text\. HippoRAG\[[15](https://arxiv.org/html/2609.30797#bib.bib25)\]uses graph retrieval, and ReadAgent\[[24](https://arxiv.org/html/2609.30797#bib.bib26)\]combines gist memory with selective rereading\. Generative Agents\[[39](https://arxiv.org/html/2609.30797#bib.bib3)\]retrieves by importance, recency, and relevance\. MemoryBank\[[60](https://arxiv.org/html/2609.30797#bib.bib4)\]applies time\-dependent forgetting\. MemGPT\[[37](https://arxiv.org/html/2609.30797#bib.bib5)\]pages context, while Mem0\[[7](https://arxiv.org/html/2609.30797#bib.bib6)\]and A\-MEM\[[53](https://arxiv.org/html/2609.30797#bib.bib7)\]extract and link notes\. Memory\-R1\[[54](https://arxiv.org/html/2609.30797#bib.bib8)\]learns text\-memory operations from task outcomes\. Larimar\[[9](https://arxiv.org/html/2609.30797#bib.bib27)\]supports one\-shot episodic\-memory edits\. HasMem controls continuous width per record\. Appendix[A](https://arxiv.org/html/2609.30797#A1)evaluates external adaptations with shared data and lexical scoring\.

### 2\.2Persistent Memory in Continuous Contexts

Prompt Tuning\[[25](https://arxiv.org/html/2609.30797#bib.bib28)\]learns continuous input prompts\. Prefix\-Tuning\[[27](https://arxiv.org/html/2609.30797#bib.bib9)\]and P\-Tuning v2\[[31](https://arxiv.org/html/2609.30797#bib.bib29)\]learn multilayer continuous prefixes\. Adapters\[[16](https://arxiv.org/html/2609.30797#bib.bib30)\]and LoRA\[[17](https://arxiv.org/html/2609.30797#bib.bib31)\]add trainable parameters to frozen backbones\. Generative Adapter\[[4](https://arxiv.org/html/2609.30797#bib.bib42)\]maps context to low\-rank adapters\. xRAG\[[5](https://arxiv.org/html/2609.30797#bib.bib43)\]bridges retrieval embeddings to LLMs\. Transformer\-XL\[[8](https://arxiv.org/html/2609.30797#bib.bib32)\]and Recurrent Memory Transformer\[[2](https://arxiv.org/html/2609.30797#bib.bib40)\]carry states or memory vectors across segments\. Compressive Transformer\[[40](https://arxiv.org/html/2609.30797#bib.bib33)\]compresses older states\. LongMem\[[44](https://arxiv.org/html/2609.30797#bib.bib37)\]adds a memory network beside a frozen backbone\. The∞\\infty\-former\[[33](https://arxiv.org/html/2609.30797#bib.bib34)\]uses continuous\-space attention\. Memorizing Transformers\[[50](https://arxiv.org/html/2609.30797#bib.bib35)\]and Focused Transformer\[[43](https://arxiv.org/html/2609.30797#bib.bib36)\]retrieve stored keys and values\.

Gist Tokens\[[34](https://arxiv.org/html/2609.30797#bib.bib10)\], AutoCompressors\[[6](https://arxiv.org/html/2609.30797#bib.bib11)\], and ICAE\[[14](https://arxiv.org/html/2609.30797#bib.bib12)\]compress contexts into vectors\. MemoryLLM\[[45](https://arxiv.org/html/2609.30797#bib.bib13)\]and M\+\[[46](https://arxiv.org/html/2609.30797#bib.bib14)\]maintain layerwise shared pools\. Matryoshka Representation Learning\[[23](https://arxiv.org/html/2609.30797#bib.bib39)\]supports nested embedding dimensions\. FlexComp\[[59](https://arxiv.org/html/2609.30797#bib.bib38)\]trains one compressor across memory\-token budgets and predicts write\-time budgets from context\. HasMem starts each record from hard\-prompt embeddings and continually revises its width\.

### 2\.3Token and Memory Pruning

LLMLingua\[[19](https://arxiv.org/html/2609.30797#bib.bib45)\]and Selective Context\[[28](https://arxiv.org/html/2609.30797#bib.bib46)\]prune by perplexity or self\-information\. LongLLMLingua\[[20](https://arxiv.org/html/2609.30797#bib.bib16)\]ranks documents and prunes tokens using questions\. LLMLingua\-2\[[38](https://arxiv.org/html/2609.30797#bib.bib15)\]learns task\-agnostic extractive compression\. Rate–distortion analysis characterizes question\-conditioned retention at fixed budgets\[[35](https://arxiv.org/html/2609.30797#bib.bib18)\]\. SDTP\[[42](https://arxiv.org/html/2609.30797#bib.bib47)\]learns layerwise pruning from gradient saliency\. SimpleMem\[[30](https://arxiv.org/html/2609.30797#bib.bib17)\]compresses conversations into text memories\. MemRefine\[[22](https://arxiv.org/html/2609.30797#bib.bib48)\]compresses memory banks through question\-independent deletion, merging, and retention to meet storage budgets\. HasMem retains entries and adapts their continuous widths\.

StreamingLLM\[[51](https://arxiv.org/html/2609.30797#bib.bib49)\]retains attention sinks and recent tokens\. H2O\[[58](https://arxiv.org/html/2609.30797#bib.bib50)\]and Scissorhands\[[32](https://arxiv.org/html/2609.30797#bib.bib51)\]evict KV entries by attention importance\. SnapKV\[[29](https://arxiv.org/html/2609.30797#bib.bib52)\]selects positions from observation\-window attention\. PyramidKV\[[3](https://arxiv.org/html/2609.30797#bib.bib53)\]allocates across layers, Ada\-KV\[[11](https://arxiv.org/html/2609.30797#bib.bib54)\]across heads\. DMC\[[36](https://arxiv.org/html/2609.30797#bib.bib41)\]learns KV append/merge operations\. CCM\[[21](https://arxiv.org/html/2609.30797#bib.bib44)\]recursively compresses states for online interaction\. At inference time, HasMem allocates persistent record widths before questions arrive\. Table[4](https://arxiv.org/html/2609.30797#S4.T4)compares learned trajectories with rule re\-encoding at approximately matched per\-question target body budgets\.

## 3Method

Figure[1](https://arxiv.org/html/2609.30797#S3.F1)summarizes HasMem’s architecture, inference procedure, and training\.

Figure 1:HasMem architecture\. Records begin as frozen token embeddings and are maintained as variable\-width continuous slots\. The Controller selects width changes, the Writer re\-encodes affected slots, and Reader and Global provide readout increments\. Global also maintains a recurrent state across records\.### 3\.1Architecture

HasMem maintains a variable\-width*Soft Bank*and cross\-turn*Global*state outside the frozen backboneW0W\_\{0\}\. For useruu, stept=0t=0writes the first record\. Subsequent records or maintenance incrementtt\. In stateℳu,t=\(gu,t,ℬu,t\)\\mathcal\{M\}\_\{u,t\}=\(g\_\{u,t\},\\mathcal\{B\}\_\{u,t\}\),gu,t∈ℝmg\_\{u,t\}\\in\\mathbb\{R\}^\{m\}is the Global state\. Useruu’s Soft Bankℬu,t\\mathcal\{B\}\_\{u,t\}at stepttcontainsNu,tN\_\{u,t\}entries in arrival order and is defined by

ℬu,t=\{\(cipre,Si,t,cisuf\)\}i=1Nu,t\.\\mathcal\{B\}\_\{u,t\}=\\bigl\\\{\(c\_\{i\}^\{\\mathrm\{pre\}\},\\,S\_\{i,t\},\\,c\_\{i\}^\{\\mathrm\{suf\}\}\)\\bigr\\\}\_\{i=1\}^\{N\_\{u,t\}\}\.\(1\)Entryiihas a body slotSi,t∈ℝKi,t×dS\_\{i,t\}\\in\\mathbb\{R\}^\{K\_\{i,t\}\\times d\}of widthKi,tK\_\{i,t\}and backbone hidden dimensiondd, with fixed role\-framing token sequencesciprec\_\{i\}^\{\\mathrm\{pre\}\}andcisufc\_\{i\}^\{\\mathrm\{suf\}\}\. We omituuwhere unambiguous\.

Controller, Writer, and Global share a position\-wise encoder\. For a sequenceV=\(v1,…,v\|V\|\)V=\(v\_\{1\},\\ldots,v\_\{\|V\|\}\)withvj∈ℝdv\_\{j\}\\in\\mathbb\{R\}^\{d\}, its mean\-pooled feature is

ϕ⁡\(V\)=1\|V\|​∑j=1\|V\|GELU⁡\(We​LN​\(vj\)\)\.\\phi\(V\)=\\frac\{1\}\{\|V\|\}\\sum\_\{j=1\}^\{\|V\|\}\\mathrm\{GELU\}\\bigl\(W\_\{e\}\\,\\mathrm\{LN\}\(v\_\{j\}\)\\bigr\)\.\(2\)HereWe∈ℝm×dW\_\{e\}\\in\\mathbb\{R\}^\{m\\times d\}is learned\. LN and GELU denote layer normalization and the Gaussian error linear unit\. Writer retains position\-wise features\. Equations omit affine biases\.

#### 3\.1\.1Controller

Controller selects an action from𝒜=\{KEEP,SHRINK,EXPAND\}\\mathcal\{A\}=\\\{\\mathrm\{KEEP\},\\mathrm\{SHRINK\},\\mathrm\{EXPAND\}\\\}, indexed0,1,20,1,2, for scheduled entryii\. The inputzi,t∈ℝ2​m\+4z\_\{i,t\}\\in\\mathbb\{R\}^\{2m\+4\}combines slot features, the latest record’s mean\-pooled embedding featureϕt\\phi\_\{t\}, and metadata as

zi,t=\[\\displaystyle z\_\{i,t\}=\\bigl\[ϕ⁡\(Si,t\);ϕt;log⁡\(1\+t−τi\);\\displaystyle\\phi\(S\_\{i,t\}\);\\ \\phi\_\{t\};\\ \\log\(1\+t\-\\tau\_\{i\}\);\(3\)Ki,t/Li;log⁡\(1\+t\);\\displaystyle K\_\{i,t\}/L\_\{i\};\\ \\log\(1\+t\);log\(1\+min\(st,smax\)\)\]\.\\displaystyle\\log\(1\+\\min\(s\_\{t\},s\_\{\\max\}\)\)\\,\\bigr\]\.HereLiL\_\{i\}counts original body tokens\.τi\\tau\_\{i\}is the arrival step,sts\_\{t\}is the preceding KEEP streak, andsmaxs\_\{\\max\}is its clipping limit\. Semicolons concatenate features\. Scalars encode age, retention, step, and streak\. Maintenance reusesϕt\\phi\_\{t\}\.

A two\-layer multilayer perceptron \(MLP\)fθf\_\{\\theta\}with hyperbolic tangent \(Tanh\) activation predicts costs\(ρ~1,ρ~2\)=fθ​\(zi,t\)\(\\tilde\{\\rho\}\_\{1\},\\tilde\{\\rho\}\_\{2\}\)=f\_\{\\theta\}\(z\_\{i,t\}\)relative to KEEP, withρ~0≡0\\tilde\{\\rho\}\_\{0\}\\equiv 0, giving

ai,t=arg⁡mina∈\{0,1,2\}⁡ρ~a\.a\_\{i,t\}=\\arg\\min\_\{a\\in\\\{0,1,2\\\}\}\\tilde\{\\rho\}\_\{a\}\.\(4\)The final layer is zero\-initialized, and ties select KEEP\. For adjustment fractionη∈\(0,1\)\\eta\\in\(0,1\)and minimum positive integer widthKminK\_\{\\min\}, setΔ​Ki,t=⌈η​Ki,t⌉\\Delta K\_\{i,t\}=\\lceil\\eta K\_\{i,t\}\\rceil, where⌈⋅⌉\\lceil\\cdot\\rceilrounds up\. Widths follow

Ki,t\+1\\displaystyle K\_\{i,t\+1\}=\{Ki,t,KEEP/bounded,max⁡\(Kmin,Ki,t−Δ​Ki,t\),SHRINK,min⁡\(Li,Ki,t\+Δ​Ki,t\),EXPAND\.\\displaystyle=\\begin\{cases\}K\_\{i,t\},&\\text\{KEEP/bounded\},\\\\ \\max\(K\_\{\\min\},K\_\{i,t\}\-\\Delta K\_\{i,t\}\),&\\text\{SHRINK\},\\\\ \\min\(L\_\{i\},K\_\{i,t\}\+\\Delta K\_\{i,t\}\),&\\text\{EXPAND\}\.\\end\{cases\}\(5\)SHRINK is bounded whenKi,t≤KminK\_\{i,t\}\\leq K\_\{\\min\}\. Otherwise the second case applies\. Unchanged widths count as KEEP\. Widths for records withLi≥KminL\_\{i\}\\geq K\_\{\\min\}stay within\[Kmin,Li\]\[K\_\{\\min\},L\_\{i\}\]\. Shorter records retain their initial width\.

#### 3\.1\.2Soft Bank and Writer

Writer alone changes slot contents whenKi,t\+1≠Ki,tK\_\{i,t\+1\}\\neq K\_\{i,t\}, resampling positions independently per hidden channel using

Ai,t=\{AvgPool⁡\(Si,t,Ki,t\+1\),Ki,t\+1<Ki,t,Interp⁡\(Si,t,Ki,t\+1\),Ki,t\+1\>Ki,t\.A\_\{i,t\}=\\begin\{cases\}\\mathrm\{AvgPool\}\(S\_\{i,t\};K\_\{i,t\+1\}\),&K\_\{i,t\+1\}<K\_\{i,t\},\\\\ \\mathrm\{Interp\}\(S\_\{i,t\};K\_\{i,t\+1\}\),&K\_\{i,t\+1\}\>K\_\{i,t\}\.\\end\{cases\}\(6\)AvgPool and Interp denote adaptive average pooling and linear interpolation to the second argument’s target width\. A learned up\-projectionU∈ℝd×mU\\in\\mathbb\{R\}^\{d\\times m\}adds a Global\-conditioned residual at positionjj, yielding

Si,t\+1​\[j\]=Ai,t​\[j\]\+U⁡\(CLOSE\\displaystyle S\_\{i,t\+1\}\[j\]=A\_\{i,t\}\[j\]\+U\\Bigl\(GELU⁡\(We​LN​\(Ai,t​\[j\]\)\)\\displaystyle\\mathrm\{GELU\}\\bigl\(W\_\{e\}\\mathrm\{LN\}\(A\_\{i,t\}\[j\]\)\\bigr\)\(7\)OPEN\+gt\)\.\\displaystyle\+g\_\{t\}\\Bigr\)\.UUis zero\-initialized\. KEEP preserves slots\. SHRINK and EXPAND re\-encode current states withgtg\_\{t\}\. EXPAND is bounded byLiL\_\{i\}\.

#### 3\.1\.3Reader

Question\-answering \(QA\) supervision trains Reader to read softened slots through low\-rank adaptation\[[17](https://arxiv.org/html/2609.30797#bib.bib31)\]of selected attention output projections\. For layerℓ\\ellwith frozen projectionWo\(ℓ\)W\_\{o\}^\{\(\\ell\)\}and inputh∈ℝdh\\in\\mathbb\{R\}^\{d\}, Reader adds

ΔR\(ℓ\)​\(h\)=Br\(ℓ\)​Ar\(ℓ\)​h\.\\Delta\_\{\\mathrm\{R\}\}^\{\(\\ell\)\}\(h\)=B\_\{r\}^\{\(\\ell\)\}A\_\{r\}^\{\(\\ell\)\}h\.\(8\)Learned down\- and up\-projectionsAr\(ℓ\)∈ℝr×dA\_\{r\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{r\\times d\}andBr\(ℓ\)∈ℝd×rB\_\{r\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{d\\times r\}use adaptation rankrr\.Br\(ℓ\)B\_\{r\}^\{\(\\ell\)\}is zero\-initialized\. Parameters remain fixed at inference\.

#### 3\.1\.4Global

A gated recurrent unit \(GRU\) updates Global in record order fromg−1=𝟎g\_\{\-1\}=\\mathbf\{0\}, gated byβt=σ⁡\(wr⊤​ϕt\+br\)\\beta\_\{t\}=\\sigma\(w\_\{r\}^\{\\top\}\\phi\_\{t\}\+b\_\{r\}\)with sigmoid functionσ\\sigma, learned weightwr∈ℝmw\_\{r\}\\in\\mathbb\{R\}^\{m\}, and biasbrb\_\{r\}\. The update is

gt=gt−1\+βt​\(GRU⁡\(ϕt,gt−1\)−gt−1\)\.g\_\{t\}=g\_\{t\-1\}\+\\beta\_\{t\}\\bigl\(\\mathrm\{GRU\}\(\\phi\_\{t\},g\_\{t\-1\}\)\-g\_\{t\-1\}\\bigr\)\.\(9\)wrw\_\{r\}is zero\-initialized\. Maintenance leavesgt=gt−1g\_\{t\}=g\_\{t\-1\}\. In Reader’s attention layers, an affine hypernetworkBg\(ℓ\):ℝm→ℝd×rB\_\{g\}^\{\(\\ell\)\}:\\mathbb\{R\}^\{m\}\\to\\mathbb\{R\}^\{d\\times r\}generates a state\-conditioned up\-projection used in

ΔG\(ℓ\)​\(h,gt\)=Bg\(ℓ\)​\(gt\)​Ag\(ℓ\)​h\.\\Delta\_\{\\mathrm\{G\}\}^\{\(\\ell\)\}\(h;g\_\{t\}\)=B\_\{g\}^\{\(\\ell\)\}\(g\_\{t\}\)A\_\{g\}^\{\(\\ell\)\}h\.\(10\)The down\-projectionAg\(ℓ\)∈ℝr×dA\_\{g\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{r\\times d\}is learned\. Hypernetwork parameters are zero\-initialized and fixed after training\.gtg\_\{t\}conditions the generated projection and Writer\.

### 3\.2Inference Procedure

##### Writing and hard initialization\.

The tokenizer and chat template separate body tokensxix\_\{i\}of lengthLi=\|xi\|L\_\{i\}=\|x\_\{i\}\|from role framing\. With frozen embedding lookupE⁡\(⋅\)E\(\\cdot\), record features update Global, after which the new slot is initialized as

Si,0=E⁡\(xi\)∈ℝLi×d\.S\_\{i,0\}=E\(x\_\{i\}\)\\in\\mathbb\{R\}^\{L\_\{i\}\\times d\}\.\(11\)Subscript00denotes local initialization, withKi,0=LiK\_\{i,0\}=L\_\{i\}\. Inspired by input\-level continuous prompts\[[25](https://arxiv.org/html/2609.30797#bib.bib28)\], slots preserve token order, role boundaries, and full width on chronological arrival\.

##### Softening\.

Each new\-record step visits at most one entry\. With positive\-integer round\-robin intervalcrevc\_\{\\mathrm\{rev\}\}andt\>0t\>0, its zero\-based index isjt=⌊\(t−1\)/crev⌋modNtj\_\{t\}=\\lfloor\(t\-1\)/c\_\{\\mathrm\{rev\}\}\\rfloor\\bmod N\_\{t\}\. Preset post\-history maintenance continues this schedule without adding records or updating Global\. Arrival order and maintenance duration determine visit counts\.

##### Retrieval\.

For question tokensqq, entryiiof widthKi,tK\_\{i,t\}, anddd\-dimensional position vectorsSi,t​\[j\]S\_\{i,t\}\[j\]andE​\(q\)​\[j\]E\(q\)\[j\], the retrieval score is the cosine similarity of position\-wise means

sim⁡\(i,q\)=cos⁡\(1Ki,t​∑j=1Ki,tSi,t​\[j\],1\|q\|​∑j=1\|q\|E⁡\(q\)​\[j\]\),\\mathrm\{sim\}\(i,q\)=\\cos\\left\(\\frac\{1\}\{K\_\{i,t\}\}\\sum\_\{j=1\}^\{K\_\{i,t\}\}S\_\{i,t\}\[j\],\\frac\{1\}\{\|q\|\}\\sum\_\{j=1\}^\{\|q\|\}E\(q\)\[j\]\\right\),\(12\)Letℛk​\(q\)\\mathcal\{R\}\_\{k\}\(q\)index the top\-scoring, at mostkkentries in arrival order\. With system\-prompt tokenscsysc^\{\\mathrm\{sys\}\}and chat\-templated question tokenscqc^\{q\}, the input is

Pt\(q\)=\[\\displaystyle P\_\{t\}\(q\)=\\bigl\[E⁡\(csys\);\\displaystyle E\(c^\{\\mathrm\{sys\}\}\);\(13\)OPEN\{E⁡\(cipre\),Si,t,E⁡\(cisuf\)\}i∈ℛk​\(q\);E⁡\(cq\)\]\.\\displaystyle\\\{E\(c\_\{i\}^\{\\mathrm\{pre\}\}\),S\_\{i,t\},E\(c\_\{i\}^\{\\mathrm\{suf\}\}\)\\\}\_\{i\\in\\mathcal\{R\}\_\{k\}\(q\)\};E\(c^\{q\}\)\\bigr\]\.Semicolons concatenate positions\. Slot\-derived retrieval keys allow softening to change the selected set\. Compared strategies share retrieval andkk\. Paired evaluations record realized sets and budgets\.

##### State\-conditioned readout\.

Readout increments activate after the first width change through the gate

γt=𝟏\[∃i,0≤s≤t:ai,s≠KEEP\]\.\\gamma\_\{t\}=\\mathbf\{1\}\\bigl\[\\exists i,\\ 0\\leq s\\leq t:\\ a\_\{i,s\}\\neq\\mathrm\{KEEP\}\\bigr\]\.\(14\)The indicator𝟏​\[⋅\]\\mathbf\{1\}\[\\cdot\]equals11when its condition holds\.ssindexes preceding steps\. Unchanged\-width actions count as KEEP, giving the effective projection

W~o\(ℓ\)​\(gt\)\\displaystyle\\widetilde\{W\}\_\{o\}^\{\(\\ell\)\}\(g\_\{t\}\)=Wo\(ℓ\)\+γt​Br\(ℓ\)​Ar\(ℓ\)\\displaystyle=W\_\{o\}^\{\(\\ell\)\}\+\\gamma\_\{t\}B\_\{r\}^\{\(\\ell\)\}A\_\{r\}^\{\(\\ell\)\}\(15\)\+γt​Bg\(ℓ\)​\(gt\)​Ag\(ℓ\)\.\\displaystyle\+\\gamma\_\{t\}B\_\{g\}^\{\(\\ell\)\}\(g\_\{t\}\)A\_\{g\}^\{\(\\ell\)\}\.At inference, parameters are fixed andgtg\_\{t\}evolves\.

##### Hard equivalence\.

An all\-KEEP trajectory preservesSi,t=E⁡\(xi\)S\_\{i,t\}=E\(x\_\{i\}\)andγt=0\\gamma\_\{t\}=0\. Under identical retrieval, ordering, tokenization, chat templates, positional encoding, attention masks, and inference settings, HasMem and the hard reference have identical inputs, frozen operators, and logits\.

##### Position budgets\.

Total bank positions, including role framing, are

Ct=∑i=1Nt\(Ki,t\+\|cipre\|\+\|cisuf\|\)\.C\_\{t\}=\\sum\_\{i=1\}^\{N\_\{t\}\}\\bigl\(K\_\{i,t\}\+\|c\_\{i\}^\{\\mathrm\{pre\}\}\|\+\|c\_\{i\}^\{\\mathrm\{suf\}\}\|\\bigr\)\.\(16\)Framing lengths count tokens\. Top\-kkinput length depends on retrieved entries\.RallR\_\{\\mathrm\{all\}\}is the bank total divided by the hard reference’s total\. Each table specifies aggregation\. Withbs,bg,bcb\_\{s\},b\_\{g\},b\_\{c\}bytes per slot\-vector, Global\-state, and framing\-ID element, persistent numerical payload is

bs​d​∑i=1NtKi,t\+bg​m\+bc​∑i=1Nt\(\|cipre\|\+\|cisuf\|\)\.b\_\{s\}d\\sum\_\{i=1\}^\{N\_\{t\}\}K\_\{i,t\}\+b\_\{g\}m\+b\_\{c\}\\sum\_\{i=1\}^\{N\_\{t\}\}\\bigl\(\|c\_\{i\}^\{\\mathrm\{pre\}\}\|\+\|c\_\{i\}^\{\\mathrm\{suf\}\}\|\\bigr\)\.\(17\)Storage also includes metadata, containers, and module parameters\. Position ratios measure representation length\. Latency and GPU memory depend on implementation\.

For Table[4](https://arxiv.org/html/2609.30797#S4.T4),ℛadaptive​\(q\)\\mathcal\{R\}\_\{\\mathrm\{adaptive\}\}\(q\)andℛhard​\(q\)\\mathcal\{R\}\_\{\\mathrm\{hard\}\}\(q\)are the retrieval sets of HasMem and the hard reference\.KiK\_\{i\}is answer\-time width\. The target body ratio is

Rbody​\(q\)=∑i∈ℛadaptive​\(q\)Ki∑i∈ℛhard​\(q\)Li\.R\_\{\\mathrm\{body\}\}\(q\)=\\frac\{\\sum\_\{i\\in\\mathcal\{R\}\_\{\\mathrm\{adaptive\}\}\(q\)\}K\_\{i\}\}\{\\sum\_\{i\\in\\mathcal\{R\}\_\{\\mathrm\{hard\}\}\(q\)\}L\_\{i\}\}\.\(18\)Equally weighting evaluation questions𝒬eval\\mathcal\{Q\}\_\{\\mathrm\{eval\}\}and their retrieved entriesℛ⁡\(q\)\\mathcal\{R\}\(q\)gives the mean per\-entry retentionr¯\\bar\{r\}

r¯=1\|𝒬eval\|​∑q∈𝒬eval1\|ℛ⁡\(q\)\|​∑i∈ℛ⁡\(q\)KiLi\.\\bar\{r\}=\\frac\{1\}\{\|\\mathcal\{Q\}\_\{\\mathrm\{eval\}\}\|\}\\sum\_\{q\\in\\mathcal\{Q\}\_\{\\mathrm\{eval\}\}\}\\frac\{1\}\{\|\\mathcal\{R\}\(q\)\|\}\\sum\_\{i\\in\\mathcal\{R\}\(q\)\}\\frac\{K\_\{i\}\}\{L\_\{i\}\}\.\(19\)

### 3\.3Training

Training uses QA supervision from each user’s \(owner’s\) history, withW0W\_\{0\}frozen\. Warm\-up updates all added modules\. Policy training updates Controller, the shared encoder, and Writer’s output layer\. Reader\- and Global\-specific parameters retain warm\-up values while Global state evolves\.

##### Quality and position cost\.

For an owner’s training questionsQQ, letℓa,q\\ell\_\{a,q\}andℓh,q\\ell\_\{h,q\}be answer NLLs for action branchaaand the hard reference\. Weighting NLL increases byλdamage\\lambda\_\{\\mathrm\{damage\}\}gives the quality objective

Qa=1\|Q\|​∑q∈Q\(ℓa,q\+λdamage​\[ℓa,q−ℓh,q\]\+\),Q\_\{a\}=\\frac\{1\}\{\|Q\|\}\\sum\_\{q\\in Q\}\\left\(\\ell\_\{a,q\}\+\\lambda\_\{\\mathrm\{damage\}\}\[\\ell\_\{a,q\}\-\\ell\_\{h,q\}\]\_\{\+\}\\right\),\(20\)Here\[x\]\+=max⁡\(x,0\)\[x\]\_\{\+\}=\\max\(x,0\), andcac\_\{a\}is cumulative framed bank positions across writing and maintenance divided by the hard reference’s rollout total\. With position\-cost weightλcost\\lambda\_\{\\mathrm\{cost\}\}, the branch score is

Ja=Qa\+λcost​ca\.J\_\{a\}=Q\_\{a\}\+\\lambda\_\{\\mathrm\{cost\}\}c\_\{a\}\.\(21\)Warm\-up minimizes meanQaQ\_\{a\}across KEEP, SHRINK, and EXPAND and trains Controller on stopped\-gradient regression targets\(J1−J0,J2−J0\)\(J\_\{1\}\-J\_\{0\},J\_\{2\}\-J\_\{0\}\)\.

##### Policy supervision\.

Bounded search on training QA finds sequences containing SHRINK that reduce cumulative positions, meet per\-question NLL tolerance against matched KEEP, and improve the search objective\. The best sequence supplies its first non\-KEEP label\. Without a qualifying sequence, a scheduled probability permits a cost\-reducing SHRINK weak label, with KEEP used when no weak label is obtained\. Appendix[F](https://arxiv.org/html/2609.30797#A6)specifies search and weighting\.

For inputz=zi,tz=z\_\{i,t\}and predicted costsρ~=\(0,ρ~1,ρ~2\)\\tilde\{\\rho\}=\(0,\\tilde\{\\rho\}\_\{1\},\\tilde\{\\rho\}\_\{2\}\), action probabilities are

πθ​\(a∣z\)=softmax⁡\(−ρ~\)a\.\\pi\_\{\\theta\}\(a\\mid z\)=\\operatorname\{softmax\}\(\-\\tilde\{\\rho\}\)\_\{a\}\.\(22\)Combining weighted action classificationℒπ\\mathcal\{L\}\_\{\\pi\}, a non\-KEEP frequency penaltyℒband\\mathcal\{L\}\_\{\\mathrm\{band\}\}outside a scheduled target band, Writer’s answer\-NLL damage hingeℒhinge\\mathcal\{L\}\_\{\\mathrm\{hinge\}\}, and a KEEP\-streak penaltyℒstreak\\mathcal\{L\}\_\{\\mathrm\{streak\}\}on qualifying positives gives

ℒpolicy=ℒπ\+ℒband\+ℒhinge\+ℒstreak\.\\mathcal\{L\}\_\{\\mathrm\{policy\}\}=\\mathcal\{L\}\_\{\\pi\}\+\\mathcal\{L\}\_\{\\mathrm\{band\}\}\+\\mathcal\{L\}\_\{\\mathrm\{hinge\}\}\+\\mathcal\{L\}\_\{\\mathrm\{streak\}\}\.\(23\)Losses average equally over owners\. Appendix[F](https://arxiv.org/html/2609.30797#A6)defines the losses, curriculum, and supervision\. Section[4\.1](https://arxiv.org/html/2609.30797#S4.SS1)gives experimental settings\.

## 4Experiments

Table 1:Reconstruction and position cost on MSC\-derived 535 \(Qwen2\.5\-7B\)\. Cells give HasMem / hard reference\. F1 and EM are percentages\.RallR\_\{\\mathrm\{all\}\}includes role framing\.BackbonekkF1↑\\uparrowHasMem / hardEM↑\\uparrowHasMem / hardNLL↓\\downarrowHasMem / hardRall↓R\_\{\\mathrm\{all\}\}\\downarrowHasMem / hardQwen2\.5\-7B286\.7 / 87\.669\.6 / 84\.70\.463 / 0\.4160\.900 / 1\.000Qwen2\.5\-7B695\.4 / 91\.065\.7 / 88\.20\.154 / 0\.0740\.879 / 1\.000Qwen2\.5\-7B895\.3 / 90\.984\.3 / 88\.10\.115 / 0\.0740\.936 / 1\.000Qwen2\.5\-7B1292\.3 / 90\.977\.4 / 88\.10\.173 / 0\.0740\.921 / 1\.000Table 2:Answer quality on all 500 LongMemEval\-S questions\. Cells give HasMem / hard reference\. Metrics except NLL are percentages\. The local judge is Qwen\-judge\. Training settings appear in Table[E1](https://arxiv.org/html/2609.30797#A5.T1)\.BackbonekkF1↑\\uparrowHasMem / hardEM↑\\uparrowHasMem / hardNLL↓\\downarrowHasMem / hardCover↑\\uparrowHasMem / hardBrief↑\\uparrowHasMem / hardLocal judge↑\\uparrowHasMem / hardQwen2\.5\-0\.5B85\.4 / 5\.60\.00 / 1\.004\.663 / 3\.6498\.4 / 8\.80\.6 / 8\.410\.8 / 10\.6Qwen2\.5\-1\.5B47\.0 / 3\.20\.80 / 1\.205\.058 / 4\.9629\.8 / 7\.43\.4 / 7\.48\.6 / 7\.6Qwen2\.5\-1\.5B81\.3 / 3\.60\.40 / 1\.405\.931 / 4\.9111\.4 / 8\.21\.0 / 8\.25\.0 / 8\.4Qwen2\.5\-3B86\.8 / 3\.60\.00 / 0\.204\.842 / 10\.41315\.4 / 10\.00\.4 / 7\.69\.8 / 10\.6Qwen2\.5\-7B20\.5 / 1\.20\.00 / 0\.208\.805 / 13\.2000\.0 / 7\.60\.0 / 7\.02\.2 / 7\.6Qwen2\.5\-7B42\.0 / 2\.70\.00 / 1\.205\.328 / 12\.9855\.4 / 9\.02\.0 / 8\.43\.8 / 9\.2Qwen2\.5\-7B68\.8 / 3\.11\.00 / 1\.404\.759 / 12\.53612\.4 / 10\.25\.8 / 9\.011\.6 / 10\.8Qwen2\.5\-7B88\.9 / 3\.40\.60 / 1\.805\.274 / 12\.25714\.2 / 10\.65\.6 / 9\.813\.4 / 10\.8Qwen2\.5\-7B127\.5 / 4\.70\.00 / 2\.405\.491 / 11\.53416\.4 / 12\.83\.4 / 10\.212\.6 / 13\.0Qwen2\.5\-14B88\.0 / 2\.80\.00 / 0\.006\.413 / 17\.19316\.2 / 12\.63\.0 / 9\.412\.4 / 13\.2Mistral\-7B88\.1 / 8\.11\.00 / 0\.804\.029 / 5\.5438\.2 / 13\.05\.0 / 5\.212\.0 / 12\.8### 4\.1Experimental Setup

##### Implementation and comparisons\.

The default frozen backbone is Qwen2\.5\-7B\-Instruct\[[55](https://arxiv.org/html/2609.30797#bib.bib55)\]\. Cross\-family runs use Mistral\-7B\-Instruct\[[18](https://arxiv.org/html/2609.30797#bib.bib56)\]\. Training uses field QA from the MSC training split and an approximately six\-hour wall\-clock budget, followed by external evaluation without further training\. Tables[G1](https://arxiv.org/html/2609.30797#A7.T1)and[E1](https://arxiv.org/html/2609.30797#A5.T1)give defaults and actual update counts\. Tables[1](https://arxiv.org/html/2609.30797#S4.T1)–[2](https://arxiv.org/html/2609.30797#S4.T2)use the full curriculum and seed20260913312026091331, withk=8k=8for the main run\. Each hard reference preserves unsoftened token embeddings and shares HasMem’s backbone, records, retrieval function, and maximum retrieved\-entry countkk\. LongMemEval\-S covers all1111configurations\. MSC\-derived covers the trained 7B models atk=2,6,8,12k=2,6,8,12\. The notationfulldenotes the complete curriculum and modules atk=8k=8, andfull−X\-Xspecifies a component change\.

##### MSC\-derived reconstruction probe\.

MSC supplies multi\-session conversations and speaker profiles\[[52](https://arxiv.org/html/2609.30797#bib.bib19)\]\. To assess retention and reconstruction of written information, deterministic templates convert profile and dialogue values into tagged records\. Questions identify tags and require the stored values as answers\. Preparation preserves user grouping and record order \(filters and target selection in Appendix[G\.2](https://arxiv.org/html/2609.30797#A7.SS2)\)\. Filtering the272272development owners yields268268owners,646646events, and535535questions, with two questions per owner except one and at most77records per owner\. Thusk=8k=8retrieves all records\. Tag lookup and value copying attain F1/EM of100\.0100\.0\. Configuration and checkpoint analyses use this development probe\. The test split remains held out\.

##### External evaluation\.

LongMemEval\-S’s500500questions test cross\-session factual recall, knowledge updates, temporal reasoning, and information synthesis\[[48](https://arxiv.org/html/2609.30797#bib.bib22)\]\. Memory is maintained per owner for MSC\-derived and independently per question for LongMemEval\-S\. External\-component protocols appear in Appendix[A](https://arxiv.org/html/2609.30797#A1)\.

##### Metrics\.

Lexical F1 and EM use SQuAD normalization\[[41](https://arxiv.org/html/2609.30797#bib.bib57)\]\. MSC\-derived EM requires the complete normalized record value\. Teacher\-forced answer NLL averages negative natural log\-probabilities over gold tokens, excluding the end token\. Within each backbone, paired scores share gold tokens and scoring positions\. Cross\-backbone NLL depends on tokenization and vocabulary\. MSC\-derived scores are owner\-weighted unless specified\. LongMemEval\-S scores are question\-weighted\. Differences and ratios use unrounded values\. Cover applies local answer\-coverage rules\. Brief additionally limits normalized prediction length tomax⁡\(12,3​ngold\)\\max\(12,3n\_\{\\mathrm\{gold\}\}\)words, wherengoldn\_\{\\mathrm\{gold\}\}counts normalized gold\-answer words\. Qwen\-judge consistently uses official LongMemEval yes/no prompts with local Qwen2\.5\-7B\-Instruct\. Scoring details appear in Appendix[G\.2](https://arxiv.org/html/2609.30797#A7.SS2)\.

##### Paired inference and training variation\.

Fixed\-model readout effects are question\-weighted EM differences\. Treating owners as independent units, two\-sided exact tests flip all paired outcomes within each owner\[[47](https://arxiv.org/html/2609.30797#bib.bib20)\]\. Intervals use owner\-clustered paired bootstrap\[[12](https://arxiv.org/html/2609.30797#bib.bib60)\]\. The eight rule\-width and two learned\-trajectory comparisons form separate Bonferroni families \(adjustedp<0\.05p<0\.05\)\[[10](https://arxiv.org/html/2609.30797#bib.bib21)\]\. Pointwise95%95\\%bootstrap intervals are unadjusted\. Appendix[G\.2](https://arxiv.org/html/2609.30797#A7.SS2)gives calculation details\. Eight fixed seeds with common training budgets and checkpoint selection \(Table[C1](https://arxiv.org/html/2609.30797#A3.T1)\) capture variation in training, selection, and realized budgets\.

### 4\.2Results

Table 3:Target\-band penalty sweep \(MSC\-derived 535,k=8k=8\)\. Cells give owner\-weighted EM \(percent\) /RallR\_\{\\mathrm\{all\}\}\. Single\-seed runs target54655465updates\. 1\.5B atλband=0\\lambda\_\{\\mathrm\{band\}\}=0and 14B at0\.60\.6complete51325132and43864386, respectively\. Full metrics appear in Table[B1](https://arxiv.org/html/2609.30797#A2.T1)\.λband\\lambda\_\{\\mathrm\{band\}\}0\.5BEM↑\\uparrow/Rall↓R\_\{\\mathrm\{all\}\}\\downarrow1\.5BEM↑\\uparrow/Rall↓R\_\{\\mathrm\{all\}\}\\downarrow3BEM↑\\uparrow/Rall↓R\_\{\\mathrm\{all\}\}\\downarrow7BEM↑\\uparrow/Rall↓R\_\{\\mathrm\{all\}\}\\downarrow14BEM↑\\uparrow/Rall↓R\_\{\\mathrm\{all\}\}\\downarrow0\.00\.012\.3 / 0\.6709\.0 / 0\.60277\.2 / 0\.88975\.2 / 0\.88745\.1 / 0\.8140\.60\.6\(default\)7\.1 / 0\.60417\.0 / 0\.66572\.0 / 0\.89238\.4 / 0\.78849\.8 / 0\.8141\.51\.59\.9 / 0\.63228\.2 / 0\.71477\.8 / 0\.91580\.0 / 0\.91784\.3 / 0\.9163\.03\.09\.9 / 0\.63749\.4 / 0\.90957\.8 / 0\.90990\.5 / 0\.99293\.3 / 1\.0006\.06\.063\.2 / 0\.92479\.9 / 0\.95780\.6 / 0\.97386\.0 / 0\.97091\.8 / 0\.974Hard reference51\.1 / 1\.00080\.8 / 1\.00074\.6 / 1\.00088\.1 / 1\.00093\.3 / 1\.000Table 4:Learned versus rule\-based re\-encoding at approximately matched per\-question target body budgets \(MSC\-derived 535, frozen Qwen2\.5\-7B,k=8k=8unless specified\)\. Scores are question\-weighted\. The hard reference has F1/EM of90\.9/88\.090\.9/88\.0\. Rows follow learned retentionr¯\\bar\{r\}\. The main run is Table[1](https://arxiv.org/html/2609.30797#S4.T1)’s seed20260913312026091331, final update54655465, with a wall\-clock curriculum\. The position gap is learned minus rule\-based body positions as a percentage of the hard reference’s body positions \(question mean / maximum\)\.Δ\\DeltaEM is learned minus rule\-based, before rounding\.Trained configurationLearnedr¯\\bar\{r\}LearnedF1 / EMRule\-basedr¯\\bar\{r\}Rule\-basedF1 / EMΔ\\DeltaEMlearned−\-rule\-basedPosition gap \(%\)mean / maximumSeed 20260917030\.9520\.95293\.7 / 79\.60\.9520\.95293\.8 / 79\.8−0\.2\-0\.20\.05 / 2\.40Seed 20260914010\.9460\.94696\.1 / 82\.20\.9460\.94696\.5 / 82\.6−0\.4\-0\.40\.05 / 2\.40Main run0\.9110\.91195\.5 / 84\.50\.9100\.91093\.9 / 66\.5\+17\.9\+17\.91\.06 / 5\.41Seed 20260917020\.8840\.88493\.6 / 79\.30\.8830\.88394\.3 / 65\.4\+13\.8\+13\.81\.35 / 5\.95Seed 20260917010\.8830\.88393\.7 / 77\.60\.8820\.88292\.1 / 54\.0\+23\.6\+23\.61\.22 / 5\.41full\(fixed steps\)0\.8470\.84794\.1 / 75\.00\.8410\.84193\.9 / 55\.5\+19\.4\+19\.41\.64 / 5\.13full,k=6k=60\.8400\.84095\.5 / 65\.80\.8380\.83894\.7 / 56\.6\+9\.2\+9\.20\.68 / 4\.69Gaussian\-input\-trained0\.8330\.83393\.7 / 58\.50\.8320\.83288\.8 / 50\.5\+8\.0\+8\.00\.60 / 4\.69full, width step20%20\\%0\.7650\.76587\.2 / 39\.10\.7600\.76088\.8 / 40\.6−1\.5\-1\.51\.20 / 4\.48full−\-target band0\.7610\.76191\.9 / 48\.00\.7590\.75990\.2 / 44\.5\+3\.6\+3\.60\.84 / 4\.11full, final target30%30\\%0\.7450\.74583\.9 / 34\.60\.7410\.74182\.5 / 31\.6\+3\.0\+3\.00\.41 / 4\.29full−\-forced softening0\.6830\.68384\.0 / 30\.70\.6780\.67881\.6 / 32\.9−2\.2\-2\.20\.99 / 4\.00#### 4\.2\.1Comparison with Hard\-Prompt References

##### Reconstruction and position cost\.

The main configuration uses93\.6%93\.6\\%of the hard reference’s framed positions \(Rbody≈0\.921R\_\{\\mathrm\{body\}\}\\approx 0\.921,r¯≈0\.911\\bar\{r\}\\approx 0\.911\), with F14\.44\.4percentage points higher, EM3\.73\.7percentage points lower, and NLL0\.0410\.041higher\. Under identical instructions, the hard reference and HasMem output UNKNOWN on34/1234/12questions\. Their answered subsets have question\-weighted F197\.0/97\.797\.0/97\.7and EM94\.0/86\.494\.0/86\.4\. F1 gains accompany fewer abstentions and lower sentence agreement\.

##### Cross\-session QA\.

On LongMemEval\-S, the main configuration improves F1/Cover by5\.6/3\.65\.6/3\.6percentage points and reduces NLL by6\.9836\.983\. Qwen\-judge scores are13\.413\.4versus the hard reference’s10\.810\.8\. Brief/EM decline by4\.2/1\.24\.2/1\.2percentage points\. Answers satisfying coverage while exceeding the length limit constitute8\.6%8\.6\\%of questions for HasMem versus0\.8%0\.8\\%for the hard reference\.

##### Configuration dependence\.

NLL improves in eight of eleven configurations\. Local judge scores improve in four and decline in seven\. MSC\-derived EM decreases at every retrieval width\. Runs complete29072907–58795879updates within the common wall\-clock budget \(Table[E1](https://arxiv.org/html/2609.30797#A5.T1)\)\. Comparisons therefore reflect backbone, retrieval width, and update\-count differences\.

#### 4\.2\.2Configuration and Strategy Comparisons

##### Target\-band penalty\.

Table[3](https://arxiv.org/html/2609.30797#S4.T3)variesλband∈\{0,0\.6,1\.5,3,6\}\\lambda\_\{\\mathrm\{band\}\}\\in\\\{0,0\.6,1\.5,3,6\\\}across five Qwen2\.5 backbones with other planned settings fixed \(Appendix[G](https://arxiv.org/html/2609.30797#A7)\)\. Each backbone’s highest EM occurs atλband≥3\.0\\lambda\_\{\\mathrm\{band\}\}\\geq 3\.0and relative budgets0\.9240\.924–1\.0001\.000\. EM is nonmonotonic for four backbones\. The 14B model atλband=3\.0\\lambda\_\{\\mathrm\{band\}\}=3\.0selects KEEP throughout and matches the hard reference\.

##### Training variation\.

Eight seeds yield mean F1/EM94\.2/82\.694\.2/82\.6and EM range77\.477\.4–88\.188\.1\(Appendix[C](https://arxiv.org/html/2609.30797#A3)\)\. Tables[E2](https://arxiv.org/html/2609.30797#A5.T2)–[E4](https://arxiv.org/html/2609.30797#A5.T4)cover hyperparameters, action policies, and training recipes\. Under common deployment, all four recipes increase LongMemEval\-S F1 and decrease its NLL and MSC\-derived EM relative to the hard reference\. Objectives, input distributions, and realized budgets vary jointly\.

##### Width trajectories at approximately matched target budgets\.

Table[4](https://arxiv.org/html/2609.30797#S4.T4)shares trained parameters, Writer, and hard initialization across strategies\. The rule shrinks to each learned trajectory’s targetRbodyR\_\{\\mathrm\{body\}\}or below, leaving discrete residuals\. Learned re\-encoding occurs during writing and maintenance\. Rule\-based re\-encoding follows the completed history\. Twelve configurations span retention levels, including settings selected for compression depth\. All elevenk=8k=8configurations retrieve every record\. Thek=6k=6pair has different retrieved length sequences on22questions\.

Learned EM gains are8\.08\.0–23\.623\.6percentage points for six configurations atr¯≈0\.83\\bar\{r\}\\approx 0\.83–0\.910\.91, versus−0\.2/−0\.4\-0\.2/\-0\.4near0\.950\.95and−2\.2\-2\.2to\+3\.6\+3\.6at0\.680\.68–0\.770\.77\. Mean learned\-minus\-rule position gaps are0\.05%0\.05\\%–1\.64%1\.64\\%of the hard reference’s body positions \(per\-question maximum5\.95%5\.95\\%\)\. The main run’s equal\-position subset \(236236questions,118118owners\) retains an EM gain of8\.058\.05percentage points, paired95%95\\%interval\[3\.81,12\.29\]\[3\.81,12\.29\]\. This deployment\-selected subset preserves the original maintenance timing \(Appendix[D](https://arxiv.org/html/2609.30797#A4)\)\.

Table 5:Training\-component ablations \(MSC\-derived 535, frozen Qwen2\.5\-7B,k=8k=8\)\. Trained variants use one seed and the final checkpoint\. Updates are actual counts\. F1/EM \(percent\) andRallR\_\{\\mathrm\{all\}\}are owner\-weighted\. Interventions and NLL appear in Table[E5](https://arxiv.org/html/2609.30797#A5.T5)\.ConfigurationUpdatesRallR\_\{\\mathrm\{all\}\}F1EMFull HasMem54650\.9360\.93695\.384\.3No forced softening or target band60680\.7520\.75282\.427\.1No forced softening57030\.7600\.76083\.830\.6No target band53190\.8190\.81991\.747\.9Frozen Writer without hinge68680\.8110\.81187\.034\.9No Writer hinge67241\.0001\.00090\.988\.1No KEEP\-wrong weighting55720\.9140\.91496\.181\.7No Reader50850\.9170\.91796\.482\.1No Global readout51810\.8900\.89093\.975\.6Hard referenceNo training1\.0001\.00090\.988\.1
Table 6:Readout gains on MSC\-derived 535\.Δ\\DeltaEM is full readout minus the indicated disabled pathway \(question\-weighted percentage points\)\.RallR\_\{\\mathrm\{all\}\}is owner\-weighted\.ρ\\rhois the rule target width ratio\. Owner\-level exactppvalues use separate Bonferroni families of two learned and eight rule comparisons \(bold indicates adjustedp<0\.05p<0\.05\)\. Full results appear in Tables[G2](https://arxiv.org/html/2609.30797#A7.T2)–[G3](https://arxiv.org/html/2609.30797#A7.T3)\.ReaderGlobalAllocationRallR\_\{\\mathrm\{all\}\}Δ\\DeltaEMppΔ\\DeltaEMppLearned0\.9360\.936\+0\.37\+0\.371\.0000\+1\.12\+1\.120\.1406Rule\-basedρ=0\.91\\rho=0\.910\.9090\.909\+1\.87\+1\.870\.1702\+1\.87\+1\.870\.2471Rule\-basedρ=0\.85\\rho=0\.850\.8260\.826\+1\.12\+1\.121\.0000\+1\.12\+1\.121\.0000Rule\-basedρ=0\.75\\rho=0\.750\.7750\.775\+1\.87\+1\.870\.0508\+2\.24\+2\.240\.0146Rule\-basedρ=0\.60\\rho=0\.600\.7520\.752\+0\.00\+0\.001\.0000\+0\.00\+0\.001\.0000

#### 4\.2\.3Ablation and Mechanism Analysis

##### Training components\.

Table[6](https://arxiv.org/html/2609.30797#S4.T6)ablates the curriculum, Writer training, and readout components\. Variants without forced softening or the target band have lower F1, EM, and realized budgets\. Removing Writer hinge yields all KEEP and matches the hard reference\. The main MSC\-derived trajectory contains14701470KEEP,516516SHRINK, and no EXPAND actions\.

##### Readout pathways\.

Table[6](https://arxiv.org/html/2609.30797#S4.T6)holds model parameters and per\-question positions fixed within each allocation condition, disabling Reader or Global increments during answering\. Learned allocation preserves Controller actions and Bank/Global trajectories\. Rule\-based allocation uses four target widths\. Global updates and Writer conditioning remain active\.

Global at rule targetρ=0\.75\\rho=0\.75gains2\.242\.24percentage points in EM and passes correction \(p=0\.0146p=0\.0146\)\. Reader gains1\.871\.87percentage points \(p=0\.0508p=0\.0508\)\. All other comparisons have adjustedp≥0\.05p\\geq 0\.05\. Disabling either pathway increases NLL throughout\. EM gains depend on allocation and deployment\.

## 5Limitations

MSC\-derived uses development\-split sentence reconstruction with at most77records per owner and a rule\-based ceiling of100\.0100\.0\. Longer histories and independent testing are needed to assess accumulation, interference, scaling, and generalization\. Higher F1 accompanies fewer abstentions, lower EM, and higher NLL than the hard reference\.

In Table[4](https://arxiv.org/html/2609.30797#S4.T4), discrete position residuals and maintenance timing limit attribution to width allocation\. Cross\-training comparisons additionally vary in seeds, updates, and checkpoints\. Storage, GPU memory, and latency depend on vector precision, module parameters, caches, and implementation overhead as well as position counts\. Table[E4](https://arxiv.org/html/2609.30797#A5.T4)evaluates Gaussian\-input\-trained and rule\-trained models under hard\-input, Controller\-driven deployment\. Gaussian\-input and rule\-action deployment remain untested\.

Fixed\-model readout interventions estimate answering\-stage pathway effects\. Retrained module effects across seeds, configuration–module interactions, and EXPAND’s independent contribution require controlled evaluation\. Owner\-clustered intervals capture evaluation\-sample variability and assume independent owners representative of the target population\.

The local LongMemEval\-S judge requires validation against the reference evaluation configuration\. Different realized budgets and integration protocols limit equal\-resource comparisons with external systems\.

## 6Conclusion

HasMem learns per\-record width adjustment, re\-encoding, and readout around a frozen LLM, starting from verifiable token\-aligned embeddings\. All\-KEEP trajectories preserve hard\-prompt equivalence under identical retrieval and input construction\. At approximately matched per\-question target body budgets, six configurations at mean per\-record retention0\.830\.83–0\.910\.91exceed rule\-based re\-encoding by8\.08\.0–23\.623\.6EM percentage points\. The main configuration improves MSC\-derived F1 by4\.44\.4percentage points at93\.6%93\.6\\%of the hard reference’s framed positions\. LongMemEval\-S F1 improves by5\.65\.6percentage points and NLL decreases by6\.9836\.983\. These gains coexist with lower EM\. Width\-allocation and readout benefits depend on budget and deployment conditions, supporting further study of continual softening and adapted readout\.

## Generative Artificial Intelligence \(AI\) Usage Disclosure

The authors determined the research question, core framework, and methodological design\. Generative AI tools assisted with research ideation, parts of experimental refinement, code implementation and experiment execution, parts of result analysis and checks of formal statements, selected literature searches and preliminary categorization, and drafting portions of the paper, translation, and language editing\. The authors screened, verified, and integrated the literature; reviewed and revised the plans, code, analyses, and manuscript; and take responsibility for the final research judgments, citations, and content\.

## References

- \[1\]S\. Borgeaud, A\. Mensch, J\. Hoffmann, T\. Cai, E\. Rutherford, K\. Millican, G\. B\. Van Den Driessche, J\. Lespiau, B\. Damoc, A\. Clark,et al\.\(2022\)Improving language models by retrieving from trillions of tokens\.InInternational conference on machine learning,pp\. 2206–2240\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[2\]\(2022\)Recurrent memory transformer\.Advances in neural information processing systems35,pp\. 11079–11091\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[3\]Z\. Cai, Y\. Zhang, B\. Gao, Y\. Liu, Y\. Li, T\. Liu, K\. Lu, W\. Xiong, Y\. Dong, J\. Hu,et al\.\(2024\)Pyramidkv: dynamic kv cache compression based on pyramidal information funneling\.arXiv preprint arXiv:2406\.02069\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[4\]T\. Chen, H\. Fang, P\. Xia, X\. Liu, B\. Van Durme, L\. Zettlemoyer, J\. Gao, and H\. Cheng\(2025\)Generative adapter: contextualizing language models in parameters with a single forward pass\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 27450–27470\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[5\]X\. Cheng, X\. Wang, X\. Zhang, T\. Ge, S\. Chen, F\. Wei, H\. Zhang, and D\. Zhao\(2024\)Xrag: extreme context compression for retrieval\-augmented generation with one token\.Advances in Neural Information Processing Systems37,pp\. 109487–109516\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[6\]A\. Chevalier, A\. Wettig, A\. Ajith, and D\. Chen\(2023\)Adapting language models to compress contexts\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 3829–3846\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p2.1)\.
- \[7\]P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav\(2025\)Mem0: building production\-ready ai agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[§A\.1](https://arxiv.org/html/2609.30797#A1.SS1.p5.1),[§1](https://arxiv.org/html/2609.30797#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[8\]Z\. Dai, Z\. Yang, Y\. Yang, J\. G\. Carbonell, Q\. Le, and R\. Salakhutdinov\(2019\)Transformer\-xl: attentive language models beyond a fixed\-length context\.InProceedings of the 57th annual meeting of the association for computational linguistics,pp\. 2978–2988\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[9\]P\. Das, S\. Chaudhury, E\. Nelson, I\. Melnyk, S\. Swaminathan, S\. Dai, A\. Lozano, G\. Kollias, V\. Chenthamarakshan, S\. Dan,et al\.\(2024\)Larimar: large language models with episodic memory control\.arXiv preprint arXiv:2403\.11901\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[10\]O\. J\. Dunn\(1961\)Multiple comparisons among means\.Journal of the American statistical association56\(293\),pp\. 52–64\.Cited by:[§G\.2](https://arxiv.org/html/2609.30797#A7.SS2.SSS0.Px4.p1.1),[item 3](https://arxiv.org/html/2609.30797#S1.I1.i3.p1.1),[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px5.p1.1)\.
- \[11\]Y\. Feng, J\. Lv, Y\. Cao, X\. Xie, and S\. K\. Zhou\(2026\)Ada\-kv: optimizing kv cache eviction by adaptive budget allocation for efficient llm inference\.Advances in Neural Information Processing Systems38,pp\. 113152–113188\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[12\]C\. A\. Field and A\. H\. Welsh\(2007\)Bootstrapping clustered data\.Journal of the Royal Statistical Society Series B: Statistical Methodology69\(3\),pp\. 369–390\.Cited by:[§G\.2](https://arxiv.org/html/2609.30797#A7.SS2.SSS0.Px4.p1.1),[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px5.p1.1)\.
- \[13\]J\. Fu, S\. K\. Ng, Z\. Jiang, and P\. Liu\(2024\)Gptscore: evaluate as you desire\.InProceedings of the 2024 conference of the north american chapter of the association for computational linguistics: Human language technologies \(volume 1: Long papers\),pp\. 6556–6576\.Cited by:[§G\.2](https://arxiv.org/html/2609.30797#A7.SS2.SSS0.Px2.p1.1)\.
- \[14\]T\. Ge, J\. Hu, L\. Wang, X\. Wang, S\. Chen, and F\. Wei\(2023\)In\-context autoencoder for context compression in a large language model\.arXiv preprint arXiv:2307\.06945\.Cited by:[§A\.3](https://arxiv.org/html/2609.30797#A1.SS3.p1.1),[§1](https://arxiv.org/html/2609.30797#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p2.1)\.
- \[15\]B\. J\. Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. Su\(2024\)Hipporag: neurobiologically inspired long\-term memory for large language models\.Advances in neural information processing systems37,pp\. 59532–59569\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[16\]N\. Houlsby, A\. Giurgiu, S\. Jastrzebski, B\. Morrone, Q\. De Laroussilhe, A\. Gesmundo, M\. Attariyan, and S\. Gelly\(2019\)Parameter\-efficient transfer learning for nlp\.InInternational conference on machine learning,pp\. 2790–2799\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[17\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2021\)Lora: low\-rank adaptation of large language models\.arXiv preprint arXiv:2106\.09685\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1),[§3\.1\.3](https://arxiv.org/html/2609.30797#S3.SS1.SSS3.p1.1)\.
- \[18\]A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier,et al\.\(2023\)Mistral 7b\.External Links:[Link](https://arxiv.org/abs/2310.06825)Cited by:[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px1.p1.1)\.
- \[19\]H\. Jiang, Q\. Wu, C\. Lin, Y\. Yang, and L\. Qiu\(2023\)Llmlingua: compressing prompts for accelerated inference of large language models\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 13358–13376\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[20\]H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. Qiu\(2024\)Longllmlingua: accelerating and enhancing llms in long context scenarios via prompt compression\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1658–1677\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[21\]J\. Kim, J\. Yeom, S\. Yun, and H\. O\. Song\(2024\)Compressed context memory for online language model interaction\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 35316–35334\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[22\]M\. Kim, J\. Baek, S\. Jeong, and S\. J\. Hwang\(2026\)MemRefine: llm\-guided compression for long\-term agent memory\.arXiv preprint arXiv:2606\.13177\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[23\]A\. Kusupati, G\. Bhatt, A\. Rege, M\. Wallingford, A\. Sinha, V\. Ramanujan, W\. Howard\-Snyder, K\. Chen, S\. Kakade, P\. Jain,et al\.\(2022\)Matryoshka representation learning\.Advances in Neural Information Processing Systems35,pp\. 30233–30249\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p2.1)\.
- \[24\]K\. Lee, X\. Chen, H\. Furuta, J\. Canny, and I\. Fischer\(2024\)A human\-inspired reading agent with gist memory of very long contexts\.arXiv preprint arXiv:2402\.09727\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[25\]B\. Lester, R\. Al\-Rfou, and N\. Constant\(2021\)The power of scale for parameter\-efficient prompt tuning\.InProceedings of the 2021 conference on empirical methods in natural language processing,pp\. 3045–3059\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2609.30797#S3.SS2.SSS0.Px1.p1.2)\.
- \[26\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[27\]X\. L\. Li and P\. Liang\(2021\)Prefix\-tuning: optimizing continuous prompts for generation\.InProceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing \(volume 1: Long papers\),pp\. 4582–4597\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[28\]Y\. Li, B\. Dong, F\. Guerin, and C\. Lin\(2023\)Compressing context to enhance inference efficiency of large language models\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 6342–6353\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[29\]Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. Chen\(2024\)Snapkv: llm knows what you are looking for before generation\.Advances in Neural Information Processing Systems37,pp\. 22947–22970\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[30\]J\. Liu, Y\. Su, P\. Xia, S\. Han, Z\. Zheng, C\. Xie, M\. Ding, and H\. Yao\(2026\)Simplemem: efficient lifelong memory for llm agents\.arXiv preprint arXiv:2601\.02553\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[31\]X\. Liu, K\. Ji, Y\. Fu, W\. Tam, Z\. Du, Z\. Yang, and J\. Tang\(2022\)P\-tuning: prompt tuning can be comparable to fine\-tuning across scales and tasks\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 61–68\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[32\]Z\. Liu, A\. Desai, F\. Liao, W\. Wang, V\. Xie, Z\. Xu, A\. Kyrillidis, and A\. Shrivastava\(2023\)Scissorhands: exploiting the persistence of importance hypothesis for llm kv cache compression at test time\.Advances in Neural Information Processing Systems36,pp\. 52342–52364\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[33\]P\. H\. Martins, Z\. Marinho, and A\. F\. Martins\(2022\)∞\-Former: infinite memory transformer\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 5468–5485\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[34\]J\. Mu, X\. Li, and N\. Goodman\(2023\)Learning to compress prompts with gist tokens\.Advances in Neural Information Processing Systems36,pp\. 19327–19352\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p2.1)\.
- \[35\]A\. Nagle, A\. Girish, M\. Bondaschi, M\. Gastpar, A\. V\. Makkuva, and H\. Kim\(2024\)Fundamental limits of prompt compression: a rate\-distortion framework for black\-box language models\.Advances in Neural Information Processing Systems37,pp\. 94934–94970\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[36\]P\. Nawrot, A\. Łańcucki, M\. Chochowski, D\. Tarjan, and E\. M\. Ponti\(2024\)Dynamic memory compression: retrofitting llms for accelerated inference\.arXiv preprint arXiv:2403\.09636\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[37\]C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez\(2023\)Memgpt: towards llms as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§1](https://arxiv.org/html/2609.30797#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[38\]Z\. Pan, Q\. Wu, H\. Jiang, M\. Xia, X\. Luo, J\. Zhang, Q\. Lin, V\. Rühle, Y\. Yang, C\. Lin,et al\.\(2024\)Llmlingua\-2: data distillation for efficient and faithful task\-agnostic prompt compression\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 963–981\.Cited by:[§A\.1](https://arxiv.org/html/2609.30797#A1.SS1.p5.1),[§1](https://arxiv.org/html/2609.30797#S1.p1.1),[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[39\]J\. S\. Park, J\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein\(2023\)Generative agents: interactive simulacra of human behavior\.InProceedings of the 36th annual acm symposium on user interface software and technology,pp\. 1–22\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[40\]J\. W\. Rae, A\. Potapenko, S\. M\. Jayakumar, and T\. P\. Lillicrap\(2019\)Compressive transformers for long\-range sequence modelling\.arXiv preprint arXiv:1911\.05507\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[41\]P\. Rajpurkar, J\. Zhang, K\. Lopyrev, and P\. Liang\(2016\)Squad: 100,000\+ questions for machine comprehension of text\.InProceedings of the 2016 conference on empirical methods in natural language processing,pp\. 2383–2392\.Cited by:[§G\.2](https://arxiv.org/html/2609.30797#A7.SS2.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px4.p1.1)\.
- \[42\]Y\. Tao, Y\. Tang, Y\. Wang, M\. Zhu, H\. Hu, and Y\. Wang\(2025\)Saliency\-driven dynamic token pruning for large language models\.arXiv preprint arXiv:2504\.04514\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p1.1)\.
- \[43\]S\. Tworkowski, K\. Staniszewski, M\. Pacek, Y\. Wu, H\. Michalewski, and P\. Miłoś\(2023\)Focused transformer: contrastive training for context scaling\.Advances in neural information processing systems36,pp\. 42661–42688\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[44\]W\. Wang, L\. Dong, H\. Cheng, X\. Liu, X\. Yan, J\. Gao, and F\. Wei\(2023\)Augmenting language models with long\-term memory\.Advances in neural information processing systems36,pp\. 74530–74543\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[45\]Y\. Wang, Y\. Gao, X\. Chen, H\. Jiang, S\. Li, J\. Yang, Q\. Yin, Z\. Li, X\. Li, B\. Yin,et al\.\(2024\)Memoryllm: towards self\-updatable large language models\.arXiv preprint arXiv:2402\.04624\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p2.1)\.
- \[46\]Y\. Wang, D\. Krotov, Y\. Hu, Y\. Gao, W\. Zhou, J\. McAuley, D\. Gutfreund, R\. Feris, and Z\. He\(2025\)M\+: extending memoryllm with scalable long\-term memory\.arXiv preprint arXiv:2502\.00592\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p2.1)\.
- \[47\]A\. M\. Winkler, G\. R\. Ridgway, M\. A\. Webster, S\. M\. Smith, and T\. E\. Nichols\(2014\)Permutation inference for the general linear model\.Neuroimage92,pp\. 381–397\.Cited by:[§G\.2](https://arxiv.org/html/2609.30797#A7.SS2.SSS0.Px4.p1.1),[item 3](https://arxiv.org/html/2609.30797#S1.I1.i3.p1.1),[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px5.p1.1)\.
- \[48\]D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu\(2024\)Longmemeval: benchmarking chat assistants on long\-term interactive memory\.arXiv preprint arXiv:2410\.10813\.Cited by:[§1](https://arxiv.org/html/2609.30797#S1.p4.1),[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px3.p1.1)\.
- \[49\]Y\. Wu, S\. Liang, C\. Zhang, Y\. Wang, Y\. Zhang, H\. Guo, R\. Tang, and Y\. Liu\(2025\)From human memory to ai memory: a survey on memory mechanisms in the era of llms\.arXiv preprint arXiv:2504\.15965\.Cited by:[§1](https://arxiv.org/html/2609.30797#S1.p1.1)\.
- \[50\]Y\. Wu, M\. N\. Rabe, D\. Hutchins, and C\. Szegedy\(2022\)Memorizing transformers\.arXiv preprint arXiv:2203\.08913\.Cited by:[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p1.1)\.
- \[51\]G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. Lewis\(2024\)Efficient streaming language models with attention sinks\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 21875–21895\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[52\]J\. Xu, A\. Szlam, and J\. Weston\(2022\)Beyond goldfish memory: long\-term open\-domain conversation\.InProceedings of the 60th annual meeting of the association for computational linguistics \(volume 1: long papers\),pp\. 5180–5197\.Cited by:[§1](https://arxiv.org/html/2609.30797#S1.p4.1),[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px2.p1.1)\.
- \[53\]W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang\(2026\)A\-mem: agentic memory for llm agents\.Advances in Neural Information Processing Systems38,pp\. 17577–17604\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[54\]S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan,et al\.\(2026\)Memory\-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 12805–12825\.Cited by:[§A\.1](https://arxiv.org/html/2609.30797#A1.SS1.p5.1),[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.
- \[55\]A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei,et al\.\(2025\)Qwen2\.5 technical report\.External Links:[Link](https://arxiv.org/abs/2412.15115)Cited by:[§4\.1](https://arxiv.org/html/2609.30797#S4.SS1.SSS0.Px1.p1.1)\.
- \[56\]W\. Yuan, G\. Neubig, and P\. Liu\(2021\)Bartscore: evaluating generated text as text generation\.Advances in neural information processing systems34,pp\. 27263–27277\.Cited by:[§G\.2](https://arxiv.org/html/2609.30797#A7.SS2.SSS0.Px2.p1.1)\.
- \[57\]Z\. Zhang, Q\. Dai, X\. Bo, C\. Ma, R\. Li, X\. Chen, J\. Zhu, Z\. Dong, and J\. Wen\(2025\)A survey on the memory mechanism of large language model\-based agents\.ACM Transactions on Information Systems43\(6\),pp\. 1–47\.Cited by:[§1](https://arxiv.org/html/2609.30797#S1.p1.1)\.
- \[58\]Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett,et al\.\(2023\)H2o: heavy\-hitter oracle for efficient generative inference of large language models\.Advances in neural information processing systems36,pp\. 34661–34710\.Cited by:[§2\.3](https://arxiv.org/html/2609.30797#S2.SS3.p2.1)\.
- \[59\]K\. Zhao, Z\. Miao, A\. Aizawa, and Y\. Tsuruoka\(2026\)FlexComp: one model for every ratio in context compression\.arXiv preprint arXiv:2609\.11192\.Cited by:[§1](https://arxiv.org/html/2609.30797#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.30797#S2.SS2.p2.1)\.
- \[60\]W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang\(2024\)Memorybank: enhancing large language models with long\-term memory\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 19724–19731\.Cited by:[§2\.1](https://arxiv.org/html/2609.30797#S2.SS1.p1.1)\.

## Appendix ASupplementary Diagnostics of External Memory Systems

### A\.1Answering and Paraphrasing Sensitivity on MSC\-derived

Table A1:Diagnostic results of external\-system adaptations on MSC\-derived 535 with frozen Qwen2\.5\-7B and full\-sentence answers\.ConfigurationMSC F1MSC EMWriting and readoutVerbatim window93\.0486\.19Verbatim textBM25 retrieval \(k=8k=8\)87\.9178\.73Verbatim textHard embedding retrieval87\.8677\.99Verbatim textA\-MEM adaptation85\.5775\.75One write per eventMem0 adaptation5\.650\.00One write per eventLLMLingua\-2 \(official, single pass\)19\.670\.00One token\-pruning pass before readout at nominal retention0\.330\.33This appendix reports the reconstruction performance and integration protocols of external\-system adaptations on MSC\-derived 535\. The probe is sensitive to the preservation of identifiers and original wording\. Its scores reflect memory construction, retrieval, and answer generation jointly\.

Table[A1](https://arxiv.org/html/2609.30797#A1.T1)reports averages with equal weight for each of the268268owners\. BM25 and hard embedding retrieval usek=8k=8\. Mem0 and A\-MEM receive individual events but construct memories at different granularities\. LLMLingua\-2 compresses text before readout\. Paired results for hard embedding retrieval and LLMLingua\-2 with the same backbone appear in Table[A3](https://arxiv.org/html/2609.30797#A1.T3)\.

MSC\-derived uses the complete stored record value as the gold answer, whereas a short answer may return only one span\. For example, “I go to concerts every weekend\.” and “concerts” convey related information but differ under full\-sentence lexical scoring\.

In Table[A1](https://arxiv.org/html/2609.30797#A1.T1), verbatim\-text references achieve higher F1 and EM than the evaluated external memory adaptations\. Semantic question answering and long\-term memory management require corresponding task\-specific evaluations\.

Mem0\[[7](https://arxiv.org/html/2609.30797#bib.bib6)\]and LLMLingua\-2\[[38](https://arxiv.org/html/2609.30797#bib.bib15)\]obtain lexical EM of00on this field\-based question\-answering probe\. Identifier preservation, paraphrasing, and integration choices may contribute to these scores\. The available comparisons do not isolate their effects\. The local Memory\-R1\[[54](https://arxiv.org/html/2609.30797#bib.bib8)\]adaptation lacks a configuration trained with reinforcement learning and is excluded from this table\.

### A\.2Complete Supplementary Results on LongMemEval\-S

Table A2:Descriptive comparisons on all 500 LongMemEval\-S questions with frozen Qwen2\.5\-7B\.MethodCoverBriefLexical F1ProtocolHard reference, same retrieval width10\.609\.803\.39Retrieval settings shared with HasMemHasMem \(main\)14\.205\.608\.94Learned softening with the configuration in Table[2](https://arxiv.org/html/2609.30797#S4.T2)LLMLingua\-2 \(official, single pass\)26\.201\.809\.23Whole\-context compression at nominal ratio0\.330\.33Table[A2](https://arxiv.org/html/2609.30797#A1.T2)summarizes the available comparisons on all500500questions\. The Mem0 and Memory\-R1 runs share state across questions and do not satisfy the question\-level isolation protocol\. Complete results with independent state for each question are unavailable for these two adaptations\. MSC\-derived uses each owner as the memory\-isolation unit, as specified in Table[A1](https://arxiv.org/html/2609.30797#A1.T1)\.

All three rows use frozen Qwen2\.5\-7B, the same question set, greedy decoding with a6464\-token answer budget, and the same rule\-based scoring\. HasMem and its hard reference share the controlled retrieval settings\. LLMLingua\-2 uses whole\-context compression, with input construction and realized memory budgets that differ from the other two methods\.

LLMLingua\-2 obtains F1 of9\.239\.23, compared with8\.948\.94for HasMem\. These are descriptive results under their respective deployment protocols\. LLMLingua\-2 has higher Cover and lower Brief\. These metrics measure answer coverage and coverage subject to a length constraint, respectively\. Sentence reconstruction on MSC\-derived and external question answering on LongMemEval\-S address different evaluation objectives\.

### A\.3Sentence Reconstruction withHasMemand External Compressors

Table A3:Sentence reconstruction by HasMem and external compressors on MSC\-derived 535\. Differences are relative to the hard reference within each group\.MethodBackboneMemoryformPositionratioMSC F1Δ\\DeltaF1MSC EMΔ\\DeltaEMHard reference \(HasMem\)Qwen2\.5\-7BToken embeddings1\.00×1\.00\\times90\.87—88\.06—HasMem \(main\)Qwen2\.5\-7BContinuous slots0\.936×0\.936\\times95\.29\+4\.42\+4\.4284\.33−3\.73\-3\.73Hard embedding retrievalQwen2\.5\-7BText tokens1\.00×1\.00\\times87\.86—77\.99—LLMLingua\-2 \(official, single pass\)Qwen2\.5\-7BText tokens0\.33×0\.33\\times\(nominal\)19\.67−68\.19\-68\.190\.00−77\.99\-77\.99Hard embedding retrievalMistral\-7BText tokens1\.00×1\.00\\times72\.11—31\.34—ICAE v2, one bank per contextMistral\-7BContinuous slots1\.48×1\.48\\times28\.45−43\.66\-43\.6610\.26−21\.08\-21\.08ICAE v2, one bank per recordMistral\-7BContinuous slots7\.13×7\.13\\times0\.62−71\.48\-71\.480\.00−31\.34\-31\.34Table[A3](https://arxiv.org/html/2609.30797#A1.T3)compares HasMem with two released compressors, pairing each with a hard reference that uses the same backbone and covers the same original events\. On the token side, LLMLingua\-2 applies its official single\-pass, whole\-context compression procedure at the nominal ratio of0\.330\.33\. On the continuous\-vector side, ICAE\[[14](https://arxiv.org/html/2609.30797#bib.bib12)\]uses official v2 weights and fixed slot banks\. The ICAE comparison uses Mistral\-7B\-Instruct\-v0\.2, the backbone corresponding to its released weights, together with a separate text reference for that backbone\. These comparisons evaluate each complete compression pipeline on sentence reconstruction\.

Within each integration group, the method and its reference share the answer instruction,6464\-token generation budget, and lexical scoring, with owner\-weighted aggregation\. The HasMem group uses the main configuration and input construction of Table[1](https://arxiv.org/html/2609.30797#S4.T1)\. External compressors use their respective integration protocols\. ICAE shares its reference’s top\-kkretrieval\. LLMLingua\-2 compresses the whole context in event\-arrival order, whereas its reference orders events by question similarity\. Both cover the same events with different input orders\. Instructions and questions remain in text form\.

The evaluation uses the same268268owners and535535questions as Table[E3](https://arxiv.org/html/2609.30797#A5.T3), with their records processed using the Qwen tokenizer\. The memory\-position ratio divides each method’s unrounded owner\-weighted mean position count by that of its hard reference on the same questions\. Ratios for HasMem and ICAE are measured\. The LLMLingua\-2 pipeline does not record the post\-compression token count, so its official nominal ratio is reported and marked accordingly\. The Qwen text reference and LLMLingua\-2 F1 and EM scores also appear in Table[A1](https://arxiv.org/html/2609.30797#A1.T1)\.

Both external compressors score below their paired hard references on verbatim reconstruction \(Table[A3](https://arxiv.org/html/2609.30797#A1.T3)\)\. LLMLingua\-2 targets task\-agnostic textual fidelity and produces no exact matches here\. ICAE uses1\.48×1\.48\\timesand7\.13×7\.13\\timesthe hard reference’s positions\. Its scores reflect fixed slot counts, bank concatenation, prompt formats, and training\-distribution adaptation jointly\.

Retrieval chunks contain128128tokens, and each memory manager constructs entries according to its own protocol\. Each table specifies the complete question set and state\-isolation boundary used for evaluation\.

## Appendix BTarget\-Band Penalty Sweep

Table B1:Target\-band penalty sweep on MSC\-derived 535, with a target of54655465training updates per configuration\.Backbone\(k=8k=8\)λband\\lambda\_\{\\mathrm\{band\}\}F1EMNLLRallR\_\{\\mathrm\{all\}\}Hard referenceF1 / EM / NLLΔ\\DeltaF1 /Δ\\DeltaEM0\.5B0\.00\.069\.812\.31\.0650\.6700\.67059\.4 / 51\.1 / 0\.187\+10\.4\+10\.4/−38\.8\-38\.80\.5B0\.60\.658\.97\.11\.5980\.6040\.60459\.4 / 51\.1 / 0\.187−0\.4\-0\.4/−44\.0\-44\.00\.5B1\.51\.562\.59\.91\.3490\.6320\.63259\.4 / 51\.1 / 0\.187\+3\.1\+3\.1/−41\.2\-41\.20\.5B3\.03\.061\.19\.91\.4170\.6370\.63759\.4 / 51\.1 / 0\.187\+1\.7\+1\.7/−41\.2\-41\.20\.5B6\.06\.091\.563\.20\.1970\.9240\.92459\.4 / 51\.1 / 0\.187\+32\.1\+32\.1/\+12\.1\+12\.11\.5B0\.00\.063\.09\.01\.3620\.6020\.60290\.4 / 80\.8 / 0\.039−27\.4\-27\.4/−71\.8\-71\.81\.5B0\.60\.674\.917\.00\.9240\.6650\.66590\.4 / 80\.8 / 0\.039−15\.5\-15\.5/−63\.8\-63\.81\.5B1\.51\.582\.328\.20\.6750\.7140\.71490\.4 / 80\.8 / 0\.039−8\.1\-8\.1/−52\.6\-52\.61\.5B3\.03\.086\.549\.40\.2410\.9090\.90990\.4 / 80\.8 / 0\.039−4\.0\-4\.0/−31\.3\-31\.31\.5B6\.06\.093\.479\.90\.0890\.9570\.95790\.4 / 80\.8 / 0\.039\+2\.9\+2\.9/−0\.9\-0\.93B0\.00\.094\.377\.20\.1720\.8890\.88984\.7 / 74\.6 / 0\.167\+9\.5\+9\.5/\+2\.6\+2\.63B0\.60\.692\.272\.00\.2470\.8920\.89284\.7 / 74\.6 / 0\.167\+7\.5\+7\.5/−2\.6\-2\.63B1\.51\.594\.577\.80\.1440\.9150\.91584\.7 / 74\.6 / 0\.167\+9\.8\+9\.8/\+3\.2\+3\.23B3\.03\.094\.257\.80\.1860\.9090\.90984\.7 / 74\.6 / 0\.167\+9\.4\+9\.4/−16\.8\-16\.83B6\.06\.091\.480\.60\.1270\.9730\.97384\.7 / 74\.6 / 0\.167\+6\.7\+6\.7/\+6\.0\+6\.07B0\.00\.093\.475\.20\.1740\.8870\.88790\.9 / 88\.1 / 0\.074\+2\.5\+2\.5/−12\.9\-12\.97B0\.60\.687\.538\.40\.4670\.7880\.78890\.9 / 88\.1 / 0\.074−3\.4\-3\.4/−49\.6\-49\.67B1\.51\.595\.580\.00\.1150\.9170\.91790\.9 / 88\.1 / 0\.074\+4\.7\+4\.7/−8\.0\-8\.07B3\.03\.092\.690\.50\.0630\.9920\.99290\.9 / 88\.1 / 0\.074\+1\.7\+1\.7/\+2\.4\+2\.47B6\.06\.094\.686\.00\.0940\.9700\.97090\.9 / 88\.1 / 0\.074\+3\.7\+3\.7/−2\.1\-2\.114B0\.00\.091\.745\.10\.3330\.8140\.81497\.4 / 93\.3 / 0\.045−5\.7\-5\.7/−48\.1\-48\.114B0\.60\.691\.949\.80\.3050\.8140\.81497\.4 / 93\.3 / 0\.045−5\.5\-5\.5/−43\.5\-43\.514B1\.51\.597\.184\.30\.0980\.9160\.91697\.4 / 93\.3 / 0\.045−0\.3\-0\.3/−9\.0\-9\.014B3\.03\.097\.493\.30\.0451\.0001\.00097\.4 / 93\.3 / 0\.0450\.00\.0/0\.00\.014B6\.06\.098\.091\.80\.0430\.9740\.97497\.4 / 93\.3 / 0\.045\+0\.6\+0\.6/−1\.5\-1\.5Table[B1](https://arxiv.org/html/2609.30797#A2.T1)sweepsλband∈\{0,0\.6,1\.5,3,6\}\\lambda\_\{\\mathrm\{band\}\}\\in\\\{0,0\.6,1\.5,3,6\\\}across five backbones, using all535535MSC\-derived questions and a target of54655465training updates\. The target\-band half\-width isβ=0\.12\\beta=0\.12, the forced\-softening sampling probability decreases from75%75\\%to15%15\\%, and the default penalty strength is0\.60\.6\. Other training, checkpoint\-selection, and evaluation settings are shared\.

The relative position ratioRallR\_\{\\mathrm\{all\}\}is the number of memory positions, including role framing, divided by that of the hard reference under the same configuration\. The hard references use101\.1101\.1vectors for every backbone\. Differences are relative to the hard reference with the same backbone and retrieval width\. Two of the twenty\-five configurations finish below the target update count, completing51325132updates for 1\.5B withλband=0\\lambda\_\{\\mathrm\{band\}\}=0and43864386for 14B withλband=0\.6\\lambda\_\{\\mathrm\{band\}\}=0\.6\. The 14B configuration withλband=3\.0\\lambda\_\{\\mathrm\{band\}\}=3\.0has relative budget1\.0001\.000\. The Controller selects KEEP throughout, the causal gate remains closed, and all four metrics match the hard reference exactly\.

Each configuration uses one seed\. The eight\-seed results in Table[C1](https://arxiv.org/html/2609.30797#A3.T1)provide a descriptive reference for 7B \(F190\.990\.9–96\.196\.1, EM77\.477\.4–88\.188\.1\)\. Corresponding cross\-seed ranges are unavailable for the other backbones\.

The relationship between penalty strength and EM varies by backbone\. EM increases monotonically for 1\.5B, peaks atλband=3\.0\\lambda\_\{\\mathrm\{band\}\}=3\.0and then decreases for 14B, improves at the largest strength for 0\.5B, and is nonmonotonic for 3B and 7B\.

The default\-strength 14B configuration in Table[B1](https://arxiv.org/html/2609.30797#A2.T1)completes43864386updates, while the independent training run used in Table[2](https://arxiv.org/html/2609.30797#S4.T2)completes37523752\. In this sweep, default\-strength EM is43\.543\.5percentage points below the hard reference\. Atλband=1\.5\\lambda\_\{\\mathrm\{band\}\}=1\.5, F1 approaches the hard reference at a relative budget of0\.9160\.916, but EM remains9\.09\.0points lower and answer NLL is0\.0980\.098, compared with0\.0450\.045for the hard reference\.

Realized budget and EM vary jointly\. The seven configurations with budgets below0\.720\.72have EM of7\.17\.1–28\.228\.2, the ten configurations with budgets between0\.810\.81and0\.920\.92have EM of45\.145\.1–84\.384\.3, and the six configurations with budgets above0\.950\.95have EM of79\.979\.9–93\.393\.3\. The highest\-EM configurations for the five backbones have relative budgets of0\.9240\.924,0\.9570\.957,0\.9730\.973,0\.9920\.992, and1\.0001\.000, respectively\. Performance can also vary at similar budgets\. The five 3B configurations span budgets of0\.8890\.889–0\.9730\.973and an EM range of22\.822\.8points\.

Relative to their respective hard references, both F1 and EM improve for 0\.5B atλband=6\.0\\lambda\_\{\\mathrm\{band\}\}=6\.0and for 3B atλband=0,1\.5,6\.0\\lambda\_\{\\mathrm\{band\}\}=0,1\.5,6\.0\. The 3B configurations atλband=1\.5,6\.0\\lambda\_\{\\mathrm\{band\}\}=1\.5,6\.0also have lower NLL \(0\.1440\.144and0\.1270\.127\) than the hard reference \(0\.1670\.167\)\. NLL is slightly higher than the hard reference for 0\.5B atλband=6\.0\\lambda\_\{\\mathrm\{band\}\}=6\.0and 3B atλband=0\\lambda\_\{\\mathrm\{band\}\}=0\. For 7B,λband=3\.0\\lambda\_\{\\mathrm\{band\}\}=3\.0yields EM of90\.590\.5at relative budget0\.9920\.992, compared with88\.188\.1for the hard reference\.

## Appendix CCross\-Seed Variation in Training and Checkpoint Selection

Table C1:Eight\-seed evaluation on MSC\-derived 535 \(7B,k=8k=8\)\. Each run completes54655465updates, including50815081policy updates\. Checkpoints are selected by in\-band training loss\.SeedSelectedupdateRallR\_\{\\mathrm\{all\}\}F1EMAnswer NLL202609133120260913313851\.0001\.00090\.988\.10\.074202609140120260914014110\.9640\.96496\.182\.30\.1102026091701202609170149060\.9160\.91693\.577\.40\.1612026091702202609170238010\.9180\.91893\.679\.30\.134202609170320260917034170\.9680\.96893\.779\.70\.095202609171120260917114220\.9660\.96696\.086\.60\.0722026091712202609171249220\.9350\.93594\.682\.80\.124202609171320260917134260\.9660\.96695\.584\.50\.094Mean—0\.9540\.95494\.282\.60\.108Eight independent retraining runs use fixed random seeds, a curriculum indexed by update count, the same training budget, and the same checkpoint\-selection rule based on in\-band training loss\. Each run completes54655465updates and is evaluated at its selected checkpoint\. Variation in the selected update and realized position budget reflects this training and selection procedure jointly\.

The hard reference at the same retrieval width has F1/EM/NLL of90\.9/88\.1/0\.07490\.9/88\.1/0\.074\. Selected update counts include384384warm\-up updates\. The relative budget is the total number of memory positions, including role framing, relative to the hard reference under the same configuration\.

For seed20260913312026091331, the in\-band rule selects update385385, whose trajectory is all KEEP and whose results match the hard reference\. The sample standard deviations of F1, EM, answer NLL, and relative budget are1\.731\.73percentage points,3\.703\.70percentage points,0\.0310\.031, and0\.0290\.029, respectively\.

## Appendix DPaired Sensitivity Analysis of Budget Residuals

Table D1:Paired EM differences and equal\-body\-position subsets for all configurations in Table[4](https://arxiv.org/html/2609.30797#S4.T4)\. Differences are learned minus rule\-based EM, in percentage points\.ConfigurationAll questionsΔ\\DeltaEM \[95% interval\]Subsetquestions / ownersEqual\-position subsetΔ\\DeltaEM \[95% interval\]Seed 2026091703\-0\.19 \[\-1\.12, 0\.75\]511 / 256\+0\.00 \[\-0\.59, 0\.59\]Seed 2026091401\-0\.37 \[\-1\.12, 0\.37\]512 / 256\-0\.20 \[\-0\.59, 0\.00\]Original configuration\+17\.94 \[14\.55, 21\.39\]236 / 118\+8\.05 \[3\.81, 12\.29\]Seed 2026091702\+13\.83 \[10\.09, 17\.54\]248 / 124\+6\.45 \[1\.61, 11\.29\]Seed 2026091701\+23\.55 \[19\.66, 27\.43\]253 / 127\+17\.79 \[12\.20, 23\.62\]full\(fixed update count\)\+19\.44 \[15\.54, 23\.26\]177 / 89\+12\.43 \[5\.17, 19\.66\]full,k=6k=6\+9\.16 \[6\.16, 12\.31\]350 / 176\+4\.29 \[1\.15, 7\.45\]Gaussian\-input training\+8\.04 \[5\.23, 10\.86\]366 / 183\+5\.74 \[3\.01, 8\.47\]full, width adjustment20%20\\%\-1\.50 \[\-5\.41, 2\.43\]221 / 111\+0\.90 \[\-5\.00, 6\.82\]full−\-target band\+3\.55 \[0\.56, 6\.54\]291 / 146\+1\.37 \[\-2\.74, 5\.17\]full, final target30%30\\%\+2\.99 \[0\.75, 5\.23\]447 / 224\+1\.12 \[\-0\.67, 3\.12\]full−\-forced softening\-2\.24 \[\-5\.23, 0\.75\]247 / 124\-0\.40 \[\-4\.47, 4\.03\]For the original configuration in Table[4](https://arxiv.org/html/2609.30797#S4.T4), the rule\-based trajectory uses fewer body positions on299299questions\. Across all535535questions, it uses0\.940\.94fewer positions on average and at most55fewer positions on any question\.

Table[D1](https://arxiv.org/html/2609.30797#A4.T1)uses question\-weighted differences and the owner\-clustered paired bootstrap from Section[4\.1](https://arxiv.org/html/2609.30797#S4.SS1)\. Every full\-set comparison contains535535questions from268268owners\. The equal\-position subset retains questions for which the two trajectories use exactly the same number of body positions, preserving their original predictions and maintenance trajectories\. Intervals are descriptive, pointwise 95% intervals\. Subsets are selected from deployment outcomes and therefore change the estimand\.

Differences on equal\-position subsets describe the two existing trajectories under that condition\. Online maintenance and re\-encoding after the complete history still differ in timing\. Isolating the effect of the width\-allocation rule requires joint control of the maintenance trajectory and realized budget\.

## Appendix ETraining Settings and Supplementary Experiments

### E\.1Training Settings for Backbone and Retrieval\-Width Configurations

Table E1:Completed training updates for the backbone and retrieval\-width configurations in Tables[1](https://arxiv.org/html/2609.30797#S4.T1)and[2](https://arxiv.org/html/2609.30797#S4.T2)\.BackbonekkTraining updatesQwen2\.5\-0\.5B85683Qwen2\.5\-1\.5B42907Qwen2\.5\-1\.5B84949Qwen2\.5\-3B85065Qwen2\.5\-7B25879Qwen2\.5\-7B45825Qwen2\.5\-7B65476Qwen2\.5\-7B85465Qwen2\.5\-7B125295Qwen2\.5\-14B83752Mistral\-7B85046The configurations in Table[E1](https://arxiv.org/html/2609.30797#A5.T1)use the MSC\-derived training set, the same wall\-clock budget, and the main seed20260913312026091331\. Table[2](https://arxiv.org/html/2609.30797#S4.T2)reports all configurations on LongMemEval\-S 500\. Table[1](https://arxiv.org/html/2609.30797#S4.T1)reports the Qwen2\.5\-7B configurations withk=2,6,8,12k=2,6,8,12on MSC\-derived 535\.

### E\.2Seed and Hyperparameter Configurations

Table E2:Seed and hyperparameter configurations on MSC\-derived 535 \(Qwen2\.5\-7B,k=8k=8, unless specified\)\. Updates are listed as evaluated checkpoint / completed training\.ConfigurationSeedUpdatesevaluated / completedF1EMNLLRallR\_\{\\mathrm\{all\}\}full\+ always\-on readout gate20260913315465 / 546595\.180\.00\.1350\.9140\.914Additional hard anchoring2026091401426 / 546594\.381\.00\.1000\.9660\.966Additional hard anchoring2026091701428 / 546592\.578\.20\.1140\.9590\.959Additional hard anchoring2026091702422 / 546597\.089\.70\.0590\.9670\.967Additional hard anchoring2026091703433 / 546593\.782\.60\.0910\.9680\.968Final target frequency10%10\\%20260917014896 / 546592\.784\.00\.0960\.9620\.962Final target frequency10%10\\%20260917024287 / 546591\.180\.40\.1740\.9480\.948Final target frequency20%20\\%20260913315121 / 512195\.881\.20\.1220\.9100\.910Final target frequency30%30\\%20260913315363 / 536383\.734\.50\.6380\.7990\.799Stronger band, 3B20260913315317 / 531790\.880\.60\.1440\.9760\.976Stronger band, 14B20260913314107 / 410796\.969\.80\.1080\.9090\.909Stronger band, 7Bk=4k=420260913315979 / 597991\.786\.20\.1350\.9710\.971Llama\-3\.1\-8B, default band20260913315024 / 502491\.147\.60\.3470\.7890\.789Stronger band, Llama\-3\.1\-8B20260913315782 / 578298\.686\.40\.0440\.9480\.948Final target frequency0\.320\.3220260914015461 / 546594\.088\.60\.0780\.9720\.972Final target frequency0\.320\.3220260917015283 / 546593\.384\.70\.1210\.9490\.949Final target frequency0\.320\.3220260917025357 / 546597\.090\.50\.0620\.9680\.968Final target frequency0\.320\.3220260917035328 / 546595\.089\.70\.0760\.9760\.976Width adjustment20%20\\%20260913314809 / 480984\.734\.50\.5130\.6880\.688Final target frequency10%10\\%20260913314921 / 492195\.380\.00\.1480\.8990\.899Default settings \(last update\)20260914015274 / 527493\.378\.50\.2170\.9080\.908Temporal weak\-teacher selection20260913315040 / 504095\.478\.50\.1610\.9160\.916Final forced\-sampling probability5%5\\%20260913315887 / 588796\.986\.20\.0840\.9480\.948Additional hard anchoring adds a loss with weight0\.50\.5whenever the original Writer hinge is computed\. For each training question, it takes the positive part of the SHRINK\-path gold\-answer NLL minus the hard\-path NLL and averages across questions\. Gradients do not propagate through the hard path\. The stronger\-band setting usesλband=3\.0\\lambda\_\{\\mathrm\{band\}\}=3\.0,β=0\.08\\beta=0\.08, and\(f0,f1\)=\(0\.40,0\.05\)\(f\_\{0\},f\_\{1\}\)=\(0\.40,0\.05\), with the target frequency unchanged at0\.50→0\.220\.50\\to 0\.22\. The final\-target\-frequency0\.320\.32group also usesλband=3\.0\\lambda\_\{\\mathrm\{band\}\}=3\.0andβ=0\.08\\beta=0\.08, while retaining the default forced\-sampling probabilities of0\.75→0\.150\.75\\to 0\.15\. These targets constrain the exponential moving average \(EMA\) of the predicted non\-KEEP frequency\. Realized position retention is determined by the trajectory\. The temporal weak teacher prioritizes candidates that first execute SHRINK earlier, breaking ties by answer degradation and then position savings\.

Update counts in Table[E2](https://arxiv.org/html/2609.30797#A5.T2)include384384warm\-up updates\. The additional\-hard\-anchoring group, the final\-target\-frequency0\.320\.32group, and the two final\-10%10\\%runs that complete54655465updates select the checkpoint with lowest training loss among those with non\-KEEP prediction EMA in\[0\.22,0\.40\]\[0\.22,0\.40\]\. Other rows use the final update\. The eight\-seed results under default settings appear in Table[C1](https://arxiv.org/html/2609.30797#A3.T1)\. All metrics are owner\-weighted, and relative budget denotesRallR\_\{\\mathrm\{all\}\}\.

Each configuration is compared with the hard reference using the same backbone andkk\. The hard reference’s F1/EM/NLL is90\.9/88\.1/0\.07490\.9/88\.1/0\.074for the default 7B configuration,84\.7/74\.6/0\.16784\.7/74\.6/0\.167for 3B,97\.4/93\.3/0\.04597\.4/93\.3/0\.045for 14B,91\.1/88\.4/0\.09691\.1/88\.4/0\.096for 7B withk=4k=4, and95\.8/92\.9/0\.03295\.8/92\.9/0\.032for Llama\-3\.1\-8B\. The default\-settings row for seed20260914012026091401is an independent run completing52745274updates\. The run with the same seed in Table[C1](https://arxiv.org/html/2609.30797#A3.T1)completes54655465and selects update411411\. The fixed\-update, final\-10%10\\%group contains two independent runs\.

The four\-seed mean EM of the additional\-hard\-anchoring group is82\.982\.9, below the hard reference’s88\.188\.1\. The final\-target\-frequency0\.320\.32group has mean EM88\.488\.4\. Three of its four runs match or exceed the hard reference, at relative budgets of0\.9490\.949–0\.9760\.976\. Under the stronger band, 3B improves both F1 and EM relative to its hard reference\. Llama\-3\.1\-8B and 7B withk=4k=4improve F1 but have lower EM, and 14B has lower scores on both metrics\. The stronger band changes penalty strength, band half\-width, and the forced\-sampling schedule jointly\. These results reflect training settings, completed updates, and realized position budgets\.

### E\.3Action\-Policy Comparisons

Table E3:Action\-policy comparisons on MSC\-derived 535 with frozen Qwen2\.5\-7B andk=8k=8\.MethodAction ruleRallR\_\{\\mathrm\{all\}\}F1EMHasMem \(original main run\)Controller\-selected actions0\.9360\.93695\.384\.3Hard referenceOriginal token embeddings1\.0001\.00090\.988\.1FixedSHRINK at every step0\.750\.7580\.022\.9RandomUniform sampling over three actions0\.980\.9890\.650\.4The comparisons in Table[E3](https://arxiv.org/html/2609.30797#A5.T3)share the answer instruction and scoring protocol\. F1 and EM are owner\-weighted\. Relative budget is the ratioRallR\_\{\\mathrm\{all\}\}of all memory positions, including role framing, to the hard reference\. Table[4](https://arxiv.org/html/2609.30797#S4.T4)compares allocation strategies under approximately matched per\-question target body budgets\.

The random\-action condition shares the trained parameters and Writer and samples uniformly from\{KEEP,SHRINK,EXPAND\}\\\{\\mathrm\{KEEP\},\\mathrm\{SHRINK\},\\mathrm\{EXPAND\}\\\}\. HasMem has EM gains of34\.034\.0and61\.461\.4percentage points over random and fixed actions, respectively\. These differences jointly reflect action decisions, final widths, and state trajectories\. The all\-KEEPnoshrinkpath shares the hard reference’s implementation branch, so its results follow from their equivalence\.

### E\.4Common\-Deployment Evaluation of Training Recipes

Table E4:Common\-deployment evaluation of four training recipes with frozen Qwen2\.5\-7B andk=8k=8\.ConfigurationTraining and deploymentMSCF1MSCEMMSCNLLMemoryvectorsLME\-SF1LME\-SNLLHard referenceOriginal token embeddings, without softening90\.988\.10\.074101\.13\.412\.257full\(fixed update count\)HasMem,54655465updates including50815081policy updates, final checkpoint94\.074\.80\.17289\.89\.85\.260Gaussian\-input trainingGaussian inputs during training and hard initialization at evaluation93\.658\.40\.25087\.98\.75\.481Fixed\-ratio rule training \(0\.910\.91\)Rule\-based training with action imitation and Controller at evaluation98\.578\.40\.06791\.99\.14\.794Retention\-based rule training \(0\.910\.91\)Highest\-retention entry selection during training and Controller at evaluation97\.773\.70\.09991\.99\.55\.065Table[E4](https://arxiv.org/html/2609.30797#A5.T4)evaluates four training recipes on MSC\-derived 535 and LongMemEval\-S 500\. All four use the final checkpoint after54655465training updates, including50815081policy updates\. Fixed\-ratio and retention\-based rule training additionally disable hinge, forced softening, and the target band, and use an action\-imitation loss\. Every model is evaluated from hard token embeddings with Controller\-selected actions under the same deployment protocol\.

The same trained models are evaluated on both complete question sets\. Memory vectors denote the owner\-weighted total number of positions, including role framing, on MSC\-derived 535\. LongMemEval\-S columns report lexical F1 and gold\-answer NLL, measuring generated\-text overlap and the conditional likelihood of the gold answer, respectively\.

All four softening configurations have lower MSC\-derived EM, higher LongMemEval\-S lexical F1, and lower LongMemEval\-S answer NLL than the hard reference\. The Gaussian\-input\-trained model experiences an input\-distribution shift when deployed from hard embeddings\. The two rule\-trained models also use more memory positions than the full\-curriculum model\.

Under the same answer instruction, the fixed\-ratio rule\-trained model never outputs UNKNOWN on the535535MSC\-derived questions\. Its question\-weighted F1/EM is98\.5/78\.398\.5/78\.3\. Abstention counts and scores on answered subsets for the main configuration and the hard reference appear in Section[4\.2\.1](https://arxiv.org/html/2609.30797#S4.SS2.SSS1)\.

### E\.5Retraining Ablations

Table E5:Training interventions and answer NLL for the ablations in Table[6](https://arxiv.org/html/2609.30797#S4.T6)\(MSC\-derived 535, frozen Qwen2\.5\-7B,k=8k=8\)\. NLL is owner\-weighted\. Configurations follow the order in Table[6](https://arxiv.org/html/2609.30797#S4.T6), which reports training updates,RallR\_\{\\mathrm\{all\}\}, F1, and EM\.ConfigurationTraining interventionNLLFull HasMemFull curriculum and all modules0\.115No forced softening or target bandRemove forced softening and the target\-band penalty0\.621No forced softeningRemove forced softening0\.571No target bandRemove the target\-band penalty on non\-KEEP prediction frequency0\.285Frozen Writer without hingeFreeze the warmed\-up Writer and disable Writer hinge, retaining forward re\-encoding0\.488No Writer hingeDisable Writer hinge, with an all\-KEEP evaluation trajectory0\.074No KEEP\-wrong weightingRemove KEEP\-wrong weighting0\.114No ReaderRemove low\-rank readout adaptation0\.089No Global readoutDisable Global readout during training, retaining state updates and Writer state input0\.193Hard referenceOriginal token embeddings without softening0\.074MSC\-derived ablation details appear in Table[E5](https://arxiv.org/html/2609.30797#A5.T5)\.

##### LongMemEval\-S retraining ablations\.

On all500500questions with frozen Qwen2\.5\-7B andk=8k=8, disabling Soft Bank, disabling the Global readout increment, and removing Reader yield lexical F1 of2\.8/2\.72\.8/2\.7,5\.2/4\.45\.2/4\.4, and7\.3/7\.27\.3/7\.2, respectively, listed in seed order2026091331/20260914012026091331/2026091401\. The full model trained with a fixed update count \(Table[E4](https://arxiv.org/html/2609.30797#A5.T4), seed20260913312026091331\) achieves9\.89\.8\. Completed training updates differ across these groups\. Their results jointly reflect module configuration, memory widths, and state trajectories\.

## Appendix FTraining Objectives and Supervision

This appendix specifies warm\-up regression, policy\-label search, curriculum penalties, and sample weighting\. Hyperparameter values are reported with the experimental settings in Appendix[G](https://arxiv.org/html/2609.30797#A7)\.

##### Warm\-up\.

For each owner, intervention branchesa=0,1,2a=0,1,2correspond to KEEP, SHRINK, and EXPAND\. Their qualityQaQ\_\{a\}and cumulative\-position scoreJaJ\_\{a\}are defined in Equations[20](https://arxiv.org/html/2609.30797#S3.E20)–[21](https://arxiv.org/html/2609.30797#S3.E21), using the owner’s question setQQand answer NLLsℓa,q\\ell\_\{a,q\}andℓh,q\\ell\_\{h,q\}\. The representation and readout modules minimize meanQaQ\_\{a\}across the three branches\. Controller regresses the relative targets\(J1−J0,J2−J0\)\(J\_\{1\}\-J\_\{0\},J\_\{2\}\-J\_\{0\}\)with stopped gradients using a Smooth L1 loss\. Writingstopgrad\\operatorname\{stopgrad\}for an operator that preserves forward values and blocks gradients, the per\-owner warm\-up objective is

ℒwarm\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{warm\}\}=13​∑a=02Qa\\displaystyle=\\frac\{1\}\{3\}\\sum\_\{a=0\}^\{2\}Q\_\{a\}\(24\)\+SmoothL1\(\(ρ~1,ρ~2\),\\displaystyle\+\\operatorname\{SmoothL1\}\\Bigl\(\(\\tilde\{\\rho\}\_\{1\},\\tilde\{\\rho\}\_\{2\}\),OPENstopgrad⁡\(J1−J0,J2−J0\)\)\.\\displaystyle\\operatorname\{stopgrad\}\(J\_\{1\}\-J\_\{0\},J\_\{2\}\-J\_\{0\}\)\\Bigr\)\.

##### Policy supervision\.

After warm\-up, we select action windows from training owners’ histories, search candidate sequences with preset depth and beam width, and score them using each owner’s training QA\. A candidate sequencebbreceives scoreJbJ\_\{b\}from the same quality and cumulative\-position objectives, with answer loss evaluated after the complete rollout\. Candidates are ranked byJb\+λstreak​min⁡\(sb/ssat,1\)J\_\{b\}\+\\lambda\_\{\\mathrm\{streak\}\}\\min\(s\_\{b\}/s\_\{\\mathrm\{sat\}\},1\), wheresbs\_\{b\}is the KEEP streak at the window’s end,λstreak\\lambda\_\{\\mathrm\{streak\}\}is its penalty weight, andssat\>0s\_\{\\mathrm\{sat\}\}\>0is the saturation length\. LetϵNLL≥0\\epsilon\_\{\\mathrm\{NLL\}\}\\geq 0be the per\-question NLL tolerance\. A qualifying branch must contain an actual SHRINK, keep each question’s NLL no more thanϵNLL\\epsilon\_\{\\mathrm\{NLL\}\}above that of the KEEP branch sharing its prefix, strictly lower cumulative position cost, and strictly improve the search objective including the streak term\. We select the lowest\-scoring qualifying branch and use its first actual departure from KEEP as labela⋆a^\{\\star\}\. If no branch qualifies, the curriculum samples a weak label with probabilityf⁡\(p\)f\(p\)by selecting, among evaluated SHRINK branches that reduce cumulative position cost, the branch with the smallest maximum per\-question damage and labeling its first SHRINK\. KEEP is used when no weak label is obtained\.

For supervised inputz=zi,tz=z\_\{i,t\}, action probabilitiesπθ​\(a∣z\)\\pi\_\{\\theta\}\(a\\mid z\)follow Equation[22](https://arxiv.org/html/2609.30797#S3.E22)\. For targeta⋆a^\{\\star\}and sample weightwwdefined below, the classification loss is

ℒπ=−w​log⁡πθ​\(a⋆∣z\)\.\\mathcal\{L\}\_\{\\pi\}=\-w\\log\\pi\_\{\\theta\}\(a^\{\\star\}\\mid z\)\.\(25\)

##### Forced\-softening rate and target band\.

Policy progressppis defined by elapsed wall\-clock time\. LetTTbe the current time,TstartT\_\{\\mathrm\{start\}\}the policy\-phase start time, andTpolicyT\_\{\\mathrm\{policy\}\}its available duration after reserving time for completion\. Then

p=clip⁡\(T−TstartTpolicy,0,1\)\.p=\\operatorname\{clip\}\\left\(\\frac\{T\-T\_\{\\mathrm\{start\}\}\}\{T\_\{\\mathrm\{policy\}\}\},0,1\\right\)\.\(26\)The operatorclip⁡\(x,a,b\)\\operatorname\{clip\}\(x,a,b\)clipsxxto\[a,b\]\[a,b\]\. The forced sampling probability for weak labels isf⁡\(p\)=f0​\(1−p\)\+f1​pf\(p\)=f\_\{0\}\(1\-p\)\+f\_\{1\}p, with initial and final probabilitiesf0f\_\{0\}andf1f\_\{1\}\. The target\-band centerτ⁡\(p\)=τ0​\(1−p\)\+τ1​p\\tau\(p\)=\\tau\_\{0\}\(1\-p\)\+\\tau\_\{1\}pspecifies the target frequency of non\-KEEP predictions, with endpointsτ0\\tau\_\{0\}andτ1\\tau\_\{1\}\.

Letnnindex owner samples in the policy phase, with supervised inputznz\_\{n\}and current predictiona^n=arg⁡maxa​πθ​\(a∣zn\)\\hat\{a\}\_\{n\}=\\arg\\max\_\{a\}\\pi\_\{\\theta\}\(a\\mid z\_\{n\}\)\. We track the exponential moving average \(EMA\)u¯n\\bar\{u\}\_\{n\}of non\-KEEP prediction frequency, initialized atu¯0=0\\bar\{u\}\_\{0\}=0\. For smoothing coefficientαEMA∈\[0,1\)\\alpha\_\{\\mathrm\{EMA\}\}\\in\[0,1\),

u¯n=αEMAu¯n−1\+\(1−αEMA\)𝟏\[a^n≠KEEP\]\.\\bar\{u\}\_\{n\}=\\alpha\_\{\\mathrm\{EMA\}\}\\bar\{u\}\_\{n\-1\}\+\(1\-\\alpha\_\{\\mathrm\{EMA\}\}\)\\mathbf\{1\}\[\\hat\{a\}\_\{n\}\\neq\\mathrm\{KEEP\}\]\.\(27\)The current sample usesu¯n−1\\bar\{u\}\_\{n\-1\}to compute the band penalty before updating the EMA\. Letδn\\delta\_\{n\}be the distance ofu¯n−1\\bar\{u\}\_\{n\-1\}from\[τ⁡\(p\)−β,τ⁡\(p\)\+β\]\[\\tau\(p\)\-\\beta,\\tau\(p\)\+\\beta\], whereβ\\betais the band half\-width andλband\\lambda\_\{\\mathrm\{band\}\}the penalty weight\. Then

ℒband\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{band\}\}=λband​min⁡\(1,δn/β\)\\displaystyle=\\lambda\_\{\\mathrm\{band\}\}\\min\(1,\\delta\_\{n\}/\\beta\)\(28\)×\{πθ​\(KEEP∣zn\),u¯n−1<τ⁡\(p\)−β,1−πθ​\(KEEP∣zn\),u¯n−1\>τ⁡\(p\)\+β,0,otherwise\.\\displaystyle\\times\\begin\{cases\}\\pi\_\{\\theta\}\(\\mathrm\{KEEP\}\\mid z\_\{n\}\),&\\bar\{u\}\_\{n\-1\}<\\tau\(p\)\-\\beta,\\\\ 1\-\\pi\_\{\\theta\}\(\\mathrm\{KEEP\}\\mid z\_\{n\}\),&\\bar\{u\}\_\{n\-1\}\>\\tau\(p\)\+\\beta,\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}Forced labels supply supervision for rare non\-KEEP actions, while the band penalty discourages deviations from the prescribed prediction frequency\.

##### Quality constraints and sample weights\.

Letℓshrink,q\\ell\_\{\\mathrm\{shrink\},q\}andℓkeep,q\\ell\_\{\\mathrm\{keep\},q\}be answer NLLs for the selected SHRINK branch and its matched KEEP branch\. With Writer quality\-constraint weightλhinge\\lambda\_\{\\mathrm\{hinge\}\}, the hinge loss is

ℒhinge=λhinge\|Q\|∑q∈Q\[\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{hinge\}\}=\\frac\{\\lambda\_\{\\mathrm\{hinge\}\}\}\{\|Q\|\}\\sum\_\{q\\in Q\}\\Bigl\[ℓshrink,q\\displaystyle\\ell\_\{\\mathrm\{shrink\},q\}\(29\)−stopgrad\(ℓkeep,q\)−ϵNLL\]\+\.\\displaystyle\-\\operatorname\{stopgrad\}\(\\ell\_\{\\mathrm\{keep\},q\}\)\-\\epsilon\_\{\\mathrm\{NLL\}\}\\Bigr\]\_\{\+\}\.Both branches share the history preceding intervention\. A SHRINK branch from the winning sequence is preferred\. If none is available, we select the least\-damaging evaluated candidate that lowers cumulative position cost and whose maximum per\-question damage is at mostϵcand\\epsilon\_\{\\mathrm\{cand\}\}\. The term is zero when no candidate is available\. This constraint trains Writer through answer cross\-entropy \(CE\)\.

Letn\+n^\{\+\}andn−n^\{\-\}count qualifying and nonqualifying samples, including the current sample\.w\+maxw\_\{\+\}^\{\\max\}caps the base weight of qualifying positives, andwweak​\(p\)w\_\{\\mathrm\{weak\}\}\(p\)is the weak\-label weight, which decreases linearly with progress\. The base sample weight is

w0=\{clip⁡\(n−/max⁡\(1,n\+\),1,w\+max\),qualifying,wweak​\(p\),weak label,1,otherwise\.w\_\{0\}=\\begin\{cases\}\\operatorname\{clip\}\\bigl\(n^\{\-\}/\\max\(1,n^\{\+\}\),1,w\_\{\+\}^\{\\max\}\\bigr\),&\\text\{qualifying\},\\\\ w\_\{\\mathrm\{weak\}\}\(p\),&\\text\{weak label\},\\\\ 1,&\\text\{otherwise\}\.\\end\{cases\}\(30\)If the hard reference fails to predict all gold\-answer tokens under teacher forcing, including the answer\-ending token used in training, or the visited entry’s tag is absent from the targets of all training questions for that owner, we setw=min⁡\(wmax,κw​w0\)w=\\min\(w\_\{\\max\},\\kappa\_\{w\}w\_\{0\}\)\. Hereκw\\kappa\_\{w\}is a weight multiplier andwmaxw\_\{\\max\}the final weight cap\. Otherwise,w=w0w=w\_\{0\}\. Letssbe the number of consecutive KEEP actions preceding the supervised decision, corresponding tosts\_\{t\}in Section[3\.1\.1](https://arxiv.org/html/2609.30797#S3.SS1.SSS1)\. Qualifying positives also receive the streak penalty

ℒstreak=\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{streak\}\}=\{\}λstreak​𝟏​\[qualifying positive\]\\displaystyle\\lambda\_\{\\mathrm\{streak\}\}\\mathbf\{1\}\[\\text\{qualifying positive\}\]\(31\)×min⁡\(\(s\+1\)/ssat,1\)​πθ​\(KEEP∣z\)\.\\displaystyle\\times\\min\(\(s\+1\)/s\_\{\\mathrm\{sat\}\},1\)\\pi\_\{\\theta\}\(\\mathrm\{KEEP\}\\mid z\)\.The four terms form the policy objective in Equation[23](https://arxiv.org/html/2609.30797#S3.E23)\. Batch losses average equally over owners\. Evaluation runs the trained Controller, Writer, Reader, and Global through forward state updates and width decisions\.

## Appendix GExperimental Settings and Statistical Details

### G\.1Default Configuration

Table G1:Default architecture, maintenance, and training settings\. Progressppfollows the training schedule in Appendix[F](https://arxiv.org/html/2609.30797#A6)\. Fixed\-update variants are specified in Appendix[G\.2](https://arxiv.org/html/2609.30797#A7.SS2)\.CategorySettingDefaultArchitectureGlobal/shared feature dimensionmmand Controller MLP hidden dimension6464and6464Reader and Global readout adaptationAttention output projections in final44layers, rankr=4r=4Added parameters and policy\-phase trainable parameters4,402,6274\{,\}402\{,\}627and478,210478\{,\}210MaintenanceWidth adjustmentη\\etaand minimum body widthKminK\_\{\\min\}0\.10\.1\(10%10\\%before rounding\) and44KEEP\-streak clippingsmaxs\_\{\\max\}and round\-robin intervalcrevc\_\{\\mathrm\{rev\}\}88and22Post\-write maintenance steps1212for Qwen2\.5\-0\.5B/1\.5B,66for Qwen2\.5\-3B/7B/14B and Mistral\-7BGlobal write\-gate biasbrb\_\{r\}−4\-4, givingσ⁡\(−4\)≈0\.018\\sigma\(\-4\)\\approx 0\.018Hard\-equivalence tolerance for embeddings and logits10−410^\{\-4\}OptimizationWarmup and effective batch size384384updates and88ownersOptimizer, learning rate, and weight decayAdamW,10−410^\{\-4\}, and00Gradient\-norm clipping and wall\-clock budget11and approximately66hoursTime reserved before the job deadline2020minutesSupervisionSearch depth and beam width22and22Damage / position\-cost weightsλdamage\\lambda\_\{\\mathrm\{damage\}\},λcost\\lambda\_\{\\mathrm\{cost\}\}44/0\.050\.05Streak weight / saturationλstreak\\lambda\_\{\\mathrm\{streak\}\},ssats\_\{\\mathrm\{sat\}\}0\.020\.02/33NLL toleranceϵNLL\\epsilon\_\{\\mathrm\{NLL\}\}10−410^\{\-4\}Forced\-sampling endpoints\(f0,f1\)\(f\_\{0\},f\_\{1\}\)\(0\.75,0\.15\)\(0\.75,0\.15\)Non\-KEEP target endpoints\(τ0,τ1\)\(\\tau\_\{0\},\\tau\_\{1\}\)\(0\.50,0\.22\)\(0\.50,0\.22\)EMA coefficientαEMA\\alpha\_\{\\mathrm\{EMA\}\}0\.950\.95Band half\-width / weightβ\\beta,λband\\lambda\_\{\\mathrm\{band\}\}0\.120\.12/0\.60\.6Writer hinge weight / candidate damage limitλhinge\\lambda\_\{\\mathrm\{hinge\}\},ϵcand\\epsilon\_\{\\mathrm\{cand\}\}0\.50\.5/0\.20\.2Positive base\-weight capw\+maxw\_\{\+\}^\{\\max\}1616Weak\-label weightwweak​\(p\)w\_\{\\mathrm\{weak\}\}\(p\)0\.45​\(1−p\)\+0\.15​p0\.45\(1\-p\)\+0\.15pSample\-weight multiplier / capκw\\kappa\_\{w\},wmaxw\_\{\\max\}44/3232Table[G1](https://arxiv.org/html/2609.30797#A7.T1)gives the defaults used unless an experiment specifies a variant\. Embeddings and logits satisfy the listed hard\-equivalence tolerance at initialization and on trained all\-KEEP trajectories\.

### G\.2Data, Scoring, and Statistical Procedures

Table G2:Readout\-pathway ablations under rule\-based allocation \(MSC\-derived 535, frozen Qwen2\.5\-7B,k=8k=8\)\. All rows share the main model parameters\. Per\-question position counts are identical within each ratio\. F1, EM, NLL, and budget are owner\-weighted\. Wrong→\\toright / right→\\towrong count questions whose EM changes after disabling a pathway\.Δ\\DeltaEM is the question\-weighted gain from enabling the pathway\. Intervals use paired owner bootstrap\. Bold marks adjustedp<0\.05p<0\.05\. Exactppvalues are Bonferroni\-adjusted over eight comparisons \(Section[4\.1](https://arxiv.org/html/2609.30797#S4.SS1)\)\.Rule targetratioρ\\rhoRallR\_\{\\mathrm\{all\}\}ReadoutF1EMNLLWrong→\\toright /right→\\towrongΔ\\DeltaEM \[95% interval\]Adjustedpp0\.910\.910\.9090\.909All enabled95\.860\.10\.182Reference——0\.910\.910\.9090\.909No Reader95\.058\.20\.22444/1414\+1\.87\+1\.87\[0\.56,3\.360\.56,3\.36\]0\.17020\.910\.910\.9090\.909No Global95\.058\.20\.23455/1515\+1\.87\+1\.87\[0\.37,3\.380\.37,3\.38\]0\.24710\.850\.850\.8260\.826All enabled91\.141\.60\.438Reference——0\.850\.850\.8260\.826No Reader90\.740\.50\.56544/1010\+1\.12\+1\.12\[−0\.19,2\.43\-0\.19,2\.43\]1\.00000\.850\.850\.8260\.826No Global90\.540\.50\.58855/1111\+1\.12\+1\.12\[−0\.19,2\.44\-0\.19,2\.44\]1\.00000\.750\.750\.7750\.775All enabled85\.034\.10\.651Reference——0\.750\.750\.7750\.775No Reader84\.832\.30\.81711/1111\+1\.87\+1\.87\[0\.75,3\.180\.75,3\.18\]0\.05080\.750\.750\.7750\.775No Global84\.631\.90\.84811/1313\+2\.24\+2\.24\[0\.93,3\.730\.93,3\.73\]0\.01460\.600\.600\.7520\.752All enabled80\.022\.90\.799Reference——0\.600\.600\.7520\.752No Reader80\.122\.90\.98244/44\+0\.00\+0\.00\[−1\.12,0\.94\-1\.12,0\.94\]1\.00000\.600\.600\.7520\.752No Global79\.922\.91\.01766/66\+0\.00\+0\.00\[−1\.31,1\.31\-1\.31,1\.31\]1\.0000Table G3:Readout\-pathway ablations under learned allocation \(MSC\-derived 535\)\. All rows share the main parameters, Controller actions, Bank / Global state trajectories, and per\-question budgets, with relative budget0\.9360\.936\. Metrics are question\-weighted, whereas Table[1](https://arxiv.org/html/2609.30797#S4.T1)is owner\-weighted\. Change counts andΔ\\DeltaEM follow Table[G2](https://arxiv.org/html/2609.30797#A7.T2)\. Exact owner\-levelppvalues are Bonferroni\-adjusted over two comparisons\.Readout at inferenceF1EMNLLWrong→\\toright /right→\\towrongΔ\\DeltaEM \[95% interval\]AdjustedppAll enabled95\.4684\.490\.1123Reference——No Reader95\.4684\.110\.127422/44\+0\.37\+0\.37\[−0\.56,1\.31\-0\.56,1\.31\]1\.0000No Global95\.2583\.360\.131511/77\+1\.12\+1\.12\[0\.19,2\.240\.19,2\.24\]0\.1406##### Record construction and filtering\.

Source values are whitespace\-normalized and capped at220220characters for profiles and260260for dialogue\. The evaluation loader excludes records longer than384384tokens and values longer than4848tokens, and excludes histories with fewer than two eligible records\. Question targets use the first and last retained record tags, with the latest retained value serving as the answer for each target tag\. The resulting MSC\-derived development probe contains all535535questions from the268268retained owners\. Under question weighting, one correct answer contributes approximately0\.190\.19EM percentage points\. Under owner weighting, its contribution depends on the owner’s question count\.

##### Lexical scoring and likelihood\.

SQuAD normalization lowercases predictions and gold answers, removes punctuation and the articles*a*,*an*, and*the*, and collapses whitespace\[[41](https://arxiv.org/html/2609.30797#bib.bib57)\]\. EM assigns11to an exact normalized\-string match\. F1 measures bag\-of\-words overlap\. Answer NLL concatenates the prompt and gold answer, runs a teacher\-forced forward pass, and averages the negative natural logarithm of each gold token’s conditional probability over answer tokens, excluding the end token\. Conditional log\-probability also underlies generation metrics such as BARTScore\[[56](https://arxiv.org/html/2609.30797#bib.bib58)\]and GPTScore\[[13](https://arxiv.org/html/2609.30797#bib.bib59)\]\. Differences, ranges, and budget ratios are calculated from unrounded values before display rounding\.

##### Coverage and local judging\.

LongMemEval Cover applies local rules by question type\. Ordinary answers require a normalized substring match or the presence of all answer words\. Multipart answers must satisfy every part\. Temporal questions with a single integer answer allow a tolerance of±1\\pm 1\. Preference questions require at least half of the answer words longer than two characters, and abstention questions use abstention\-phrase rules\. Brief requires Cover andnpred≤max⁡\(12,3​ngold\)n\_\{\\mathrm\{pred\}\}\\leq\\max\(12,3n\_\{\\mathrm\{gold\}\}\), where the two counts denote normalized prediction and gold\-answer words\. The local Qwen judge uses the official LongMemEval yes/no prompts with Qwen2\.5\-7B\-Instruct\. For500500questions and accuracy near0\.110\.11, the binomial standard error of a single accuracy estimate is approximately1\.41\.4percentage points\.

##### Paired statistics\.

For owneruuamong the268268owners, letDuD\_\{u\}sum the differences in binary EM indicators between full readout and the disabled pathway over that owner’s questions\. The question\-weighted effect in EM percentage points is100535​∑u=1268Du\\frac\{100\}\{535\}\\sum\_\{u=1\}^\{268\}D\_\{u\}\. The two\-sided exact test enumerates sign flips of theDuD\_\{u\}, assuming independent owners and invariance under swapping each owner’s complete paired outcomes under the null\[[47](https://arxiv.org/html/2609.30797#bib.bib20)\]\. Pointwise95%95\\%percentile intervals use100,000100\{,\}000owner\-clustered paired bootstrap resamples with seed2026092320260923\[[12](https://arxiv.org/html/2609.30797#bib.bib60)\]\. The four rule\-width ratios and two pathways define one family of eight comparisons\. The two learned\-trajectory comparisons define a separate family\. Significance requires Bonferroni\-adjusted exact\-testp<0\.05p<0\.05\[[10](https://arxiv.org/html/2609.30797#bib.bib21)\]\. Bootstrap intervals estimate effect uncertainty without multiplicity adjustment\. Complete readout results appear in Tables[G2](https://arxiv.org/html/2609.30797#A7.T2)and[G3](https://arxiv.org/html/2609.30797#A7.T3)\.

##### Training variants and notation\.

Letspolicys\_\{\\mathrm\{policy\}\}denote completed policy\-phase updates\. The sweep in Table[B1](https://arxiv.org/html/2609.30797#A2.T1)and training in Table[E4](https://arxiv.org/html/2609.30797#A5.T4)usep=clip⁡\(spolicy/5081,0,1\)p=\\operatorname\{clip\}\(s\_\{\\mathrm\{policy\}\}/5081,0,1\)and a total target of384\+5081=5465384\+5081=5465updates\. Actual counts accompany the corresponding results\. The fixed\-ratio and retention\-rule training arms in Table[E4](https://arxiv.org/html/2609.30797#A5.T4)optimize answer NLL plus imitation of rule actions, with forced softening, the target band, and Writer hinge disabled\. The completefullcurriculum includes forced softening, the target band, Writer hinge, and KEEP\-wrong sample weighting\. Parameter variants specifyX=vX=valongsidefull\. The hard reference has zero softening\. The all\-KEEPnoshrinkpath is equivalent to the hard reference\. Unless specified otherwise, an em dash denotes an unmeasured or undefined quantity\.

##### External\-component protocols\.

Mem0 and A\-MEM use public memory components\. LLMLingua\-2 calls the officialcompress\_promptinterface once per complete history\. Appendix[A](https://arxiv.org/html/2609.30797#A1)specifies backbones, integration protocols, and state\-isolation boundaries\. The reconstruction probe depends on identifier lookup, wording, and answer format\. For*Stable candidate P2\-0: I’m a man\.*, questions identify P2\-0 and expect its record value\.*User is a man\.*omits this identifier and changes the normalized answer\.

### G\.3Complete Fixed\-Model Readout Ablations

Tables[G2](https://arxiv.org/html/2609.30797#A7.T2)–[G3](https://arxiv.org/html/2609.30797#A7.T3)give the full measurements underlying Table[6](https://arxiv.org/html/2609.30797#S4.T6)\. Absolute metric values under rule\-based allocation are owner\-weighted\. Those under learned allocation and all paired EM differences are question\-weighted\. Interventions within each allocation condition share per\-question memory positions and model parameters\. Global interventions retain state updates and Writer conditioning\.

Under rule allocation atρ=0\.60\\rho=0\.60, disabling Reader changes four answers from wrong to right and four from right to wrong, yielding zero net EM change\. Under learned allocation, Global’s pointwise bootstrap interval is\[0\.19,2\.24\]\[0\.19,2\.24\], while its multiplicity\-adjusted exact\-testppis0\.14060\.1406\. The intervals are unadjusted for multiple comparisons\. Significance follows the exact\-test procedure in Section[4\.1](https://arxiv.org/html/2609.30797#S4.SS1)\.