CAMeR: Keyword-Gated Hybrid Activation for Adaptive Memory Retention in LLM Agents

# Keyword-Gated Hybrid Activation for Adaptive Memory Retention in LLM Agents
Source: [https://arxiv.org/html/2607.20458](https://arxiv.org/html/2607.20458)
###### Abstract

Large language model \(LLM\) agents operating over extended dialogues accumulate vast amounts of information, yet existing memory systems either retain everything indiscriminately or apply uniform forgetting heuristics that fail to distinguish relevant from irrelevant knowledge\. We presentCAMeR\(Context\-Activated Memory Reinforcement\), a memory retention framework combiningkeyword\-gated hybrid activation—a joint symbolic \(word\-level Jaccard\) and sub\-symbolic \(embedding cosine\) gating mechanism—with adaptive weight dynamics\.CAMeRcomputes a hybrid similarity score for each memory\-query pair; memories exceeding a threshold receive reinforcement while all memories undergo controlled decay\. We introduceCAMeR\-Bench, a 76\-memory, 100\-round benchmark spanning 8 topic clusters with graded activation frequency, designed to test adaptive retention where existing benchmarks \(LoCoMO, LongMemEval\) cannot\. OnCAMeR\-Bench,CAMeR’s keyword gate achieves a 1\.6×\\timeslarger retention gap between high\-frequency and never\-referenced memories compared to embedding\-only gating \(scissors gap: 0\.039 vs\. 0\.024\), while time\-driven baselines \(Oblivion, SuperLocalMemory\) collapse to near\-zero weights over 100 rounds\.CAMeR’s top\-5 retrieval saves 83\.2% tokens versus full\-context approaches \(39k vs\. 231k cumulative\) while producing weight signals that improve retrieval precision\. Through 8 ablation conditions we establish that the keyword gate—not learnable decay—is the primary performance driver at this scale\. Our findings demonstrate that hybrid symbolic\-neural gating provides a simple yet effective mechanism for adaptive memory retention in LLM agents\.

## 1Introduction

LLM agents deployed in long\-running applications—customer support, personal assistants, therapy bots—must maintain coherent memory across hundreds of dialogue turns\. Naively retaining every interaction leads to unbounded context growth and prohibitive token costs\. Conversely, fixed forgetting heuristics risk discarding critical information\. The fundamental challenge isadaptive retention: determining which memories to strengthen, which to let decay, and which to discard\.

Recent work has explored diverse approaches: learned expiration spans\[[1](https://arxiv.org/html/2607.20458#bib.bib1)\], Ebbinghaus\-inspired forgetting curves\[[2](https://arxiv.org/html/2607.20458#bib.bib2)\], GRPO\-based reinforcement learning for memory operations\[[3](https://arxiv.org/html/2607.20458#bib.bib3)\], and exponential time\-decay with match\-based reinforcement\[[4](https://arxiv.org/html/2607.20458#bib.bib4)\]\. However, these methods share a common limitation: they rely predominantly onsub\-symbolic\(embedding\-based\) similarity signals for activation decisions\. Embedding similarity alone is prone to false positives—semantically adjacent but factually irrelevant memories may receive undue reinforcement, while genuinely relevant memories with lower cosine scores may be neglected\.

We argue thatkeyword\-level symbolic signalsprovide complementary precision that pure embedding similarity lacks\. Consider a concrete example from our experiments: “User uses Python for backend development” versus “User has a pet python named Monty\.” With an embedding model \(all\-MiniLM\-L6\-v2\), these sentences share a cosine similarity of 0\.47 with “Python programming”—high enough to cross typical retrieval thresholds\. Yet keyword sets \{python, backend, development\} and \{python, pet, monty\} yield a Jaccard overlap of only 0\.14 with the programming query, pulling the combined score below the activation threshold for the irrelevant memory\.

This paper makes four contributions:

1. 1\.Keyword\-gated hybrid activation: A joint scoring mechanism combining full\-text embedding cosine \(α=0\.6\\alpha=0\.6\) with keyword Jaccard similarity \(1−α=0\.41\-\\alpha=0\.4\) for memory gating\. Unlike prior work that relies solely on embedding similarity, our gate exploits the sparsity of word\-level overlap to reject false\-positive activations\.
2. 2\.CAMeR\-Bench: A controlled benchmark of 76 memories across 8 topic clusters with 100 query rounds and a graded activation frequency gradient \(37 queries for high\-frequency clusters to 0 for background controls\)\. We explain why existing benchmarks \(LoCoMO, LongMemEval\) are unsuitable for retention experiments\.
3. 3\.Token consumption causal chain: We demonstrate thatCAMeR’s keyword gate yields weight signals informative enough to improve retrieval precision over pure embedding ranking, saving 83\.2% tokens compared to full\-context approaches across 100 rounds\.
4. 4\.Comprehensive ablation: 8CAMeRvariant conditions and 5 competitive baselines evaluated at scale, with a detailed analysis ofwhyeach baseline fails andwhichcomponent ofCAMeRmatters most\.

## 2Related Work

### 2\.1Memory Systems for LLM Agents

Memory\-augmented LLM systems broadly fall into three paradigms\.Full\-contextapproaches\[[14](https://arxiv.org/html/2607.20458#bib.bib14),[15](https://arxiv.org/html/2607.20458#bib.bib15)\]retain entire dialogue histories, relying on the model’s native context window—at the cost of quadratic attention complexity and rapidly escalating token budgets\.Retrieval\-augmentedsystems\[[5](https://arxiv.org/html/2607.20458#bib.bib5),[13](https://arxiv.org/html/2607.20458#bib.bib13)\]index past interactions in vector databases and retrieve top\-kkmatches per query; they avoid quadratic costs but may miss information relevant to the current turn that happens to have low embedding similarity\.Hybridsystems\[[12](https://arxiv.org/html/2607.20458#bib.bib12),[16](https://arxiv.org/html/2607.20458#bib.bib16)\]combine structured memory stores with retrieval but lack explicit activation gating\.

Table[1](https://arxiv.org/html/2607.20458#S2.T1)positionsCAMeRrelative to the most closely related memory systems that model decay or forgetting\.

Table 1:Comparison of memory systems with decay/forgetting mechanisms\. “Symbolic gate” = uses word\-level keyword overlap for activation decisions\. “Per\-memory decay” = each memory has a potentially different decay rate\.SystemGatingSymbolicPer\-memParamMechanismSignalGateDecayCountMem0\[[5](https://arxiv.org/html/2607.20458#bib.bib5)\]Embedding✗✗–Configurable retentionMemory\-R1\[[3](https://arxiv.org/html/2607.20458#bib.bib3)\]Embedding✗✗10K\+GRPO RL operationsOblivion\[[4](https://arxiv.org/html/2607.20458#bib.bib4)\]Embedding✗✗2exp⁡\(−λ​Δ​t\)\\exp\(\-\\lambda\\Delta t\)SuperLocal\[[2](https://arxiv.org/html/2607.20458#bib.bib2)\]Embedding✗✗1e−t/Se^\{\-t/S\}, EbbinghausExpire\-Span\[[1](https://arxiv.org/html/2607.20458#bib.bib1)\]Embedding✗✓1000\+Learned span per memoryCAMeR\(ours\)Hybrid✓✓81Keyword gate \+ MLP opt\.
### 2\.2Keyword Extraction and Symbolic Signals

Keywords as interpretable, sparse features have a long history in information retrieval\[[10](https://arxiv.org/html/2607.20458#bib.bib10),[11](https://arxiv.org/html/2607.20458#bib.bib11)\]\. KeyBERT\[[9](https://arxiv.org/html/2607.20458#bib.bib9)\]leverages sentence transformers to extract key phrases representing document semantics\. InCAMeR, keywords serve dual purpose: they provide the symbolic signal for the hybrid gateandoffer an interpretable representation of each memory’s content\. Our extraction pipeline uses KeyBERT \(all\-MiniLM\-L6\-v2\) as the primary extractor, falling back to YAKE and then TF\-IDF for robustness against short or domain\-specific text\.

### 2\.3Why Existing Benchmarks Are Insufficient

Two widely\-used benchmarks evaluate LLM long\-context memory: LoCoMO\[[6](https://arxiv.org/html/2607.20458#bib.bib6)\]provides multi\-session dialogues with QA pairs, and LongMemEval\[[7](https://arxiv.org/html/2607.20458#bib.bib7)\]extends this to longer contexts\. However, both are designed to test retrieval accuracy—can the modelfindspecific information given a large context—rather than retention quality\. Specifically:

1. 1\.Single\-persona design: All session summaries describe the same persona, so every memory is relevant to every query\. This eliminates the ability to measure false\-positive suppression\.
2. 2\.No activation gradient: Memories are accessed with uniform frequency, providing no signal for adaptive differentiation\.
3. 3\.No background controls: There are no never\-referenced memories to serve as a baseline for measuring unnecessary retention\.

We confirmed this empirically: runningCAMeRon LoCoMO’s 19 session summaries produced uniform weight saturation \(all weights≈\\approx0\.99\), because every query activated every memory—a degenerate case for retention experiments\.CAMeR\-Benchaddresses all three gaps\.

## 3CAMeRFramework

CAMeRoperates over a set of memoriesℳ=\{m1,…,mN\}\\mathcal\{M\}=\\\{m\_\{1\},\\ldots,m\_\{N\}\\\}, where each memorymim\_\{i\}stores: raw text, extracted keywords𝒦​\(mi\)\\mathcal\{K\}\(m\_\{i\}\), an embedding𝐞​\(mi\)\\mathbf\{e\}\(m\_\{i\}\), a weightwi∈\[0,1\]w\_\{i\}\\in\[0,1\], an access counteraia\_\{i\}, a long\-term contribution accumulatorℓi\\ell\_\{i\}, and a type tag\{s​h​o​r​t,l​o​n​g\}\\\{short,long\\\}\.

![Refer to caption](https://arxiv.org/html/2607.20458v1/figures/architecture.png)Figure 1:CAMeRsystem architecture\. A user query triggers \(1\) keyword extraction and \(2\) embedding computation; \(3\) the hybrid gate combines embedding cosine and keyword Jaccard to produce an activation score; \(4\) the weight updater applies decay and conditional reinforcement; \(5\) weight\-augmented retrieval ranks memories bywi×w\_\{i\}\\timescosine similarity\. Long\-term migration accumulates activation counts and switches memories to a slower decay rate\.### 3\.1Keyword\-Gated Hybrid Activation

Given a user queryqq,CAMeRcomputes a combined similarity score for each memorymim\_\{i\}:

score​\(mi,q\)=α⋅cos⁡\(𝐞​\(mi\),𝐞​\(q\)\)⏟embedding similarity\+\(1−α\)⋅Jaccard​\(𝒦​\(mi\),𝒦​\(q\)\)⏟keyword overlap\\text\{score\}\(m\_\{i\},q\)=\\alpha\\cdot\\underbrace\{\\cos\\big\(\\mathbf\{e\}\(m\_\{i\}\),\\mathbf\{e\}\(q\)\\big\)\}\_\{\\text\{embedding similarity\}\}\\;\+\\;\(1\-\\alpha\)\\cdot\\underbrace\{\\text\{Jaccard\}\\big\(\\mathcal\{K\}\(m\_\{i\}\),\\mathcal\{K\}\(q\)\\big\)\}\_\{\\text\{keyword overlap\}\}\(1\)
where𝐞​\(⋅\)∈ℝ384\\mathbf\{e\}\(\\cdot\)\\in\\mathbb\{R\}^\{384\}is the full\-text embedding from all\-MiniLM\-L6\-v2\[[8](https://arxiv.org/html/2607.20458#bib.bib8)\],Jaccard​\(A,B\)=\|A∩B\|/\|A∪B\|\\text\{Jaccard\}\(A,B\)=\|A\\cap B\|/\|A\\cup B\|operates on word\-level tokenized keyword sets, andα=0\.6\\alpha=0\.6weights the embedding signal over keyword overlap\. Keywords are extracted via KeyBERT\[[9](https://arxiv.org/html/2607.20458#bib.bib9)\]with a YAKE→\\rightarrowTF\-IDF fallback chain for robustness\. A memory isactivatedin roundttifscore​\(mi,q\)\>τ\\text\{score\}\(m\_\{i\},q\)\>\\tau, whereτ=0\.25\\tau=0\.25is a fixed threshold\.

### 3\.2Weight Update Dynamics

Once activation decisions are made, weight updates follow a simple reinforcement\-decay rule:

wi\(t\)=\{clamp​\(wi\(t−1\)⋅γ\+0\.2,0,1\)if activatedwi\(t−1\)⋅γotherwisew\_\{i\}^\{\(t\)\}=\\begin\{cases\}\\text\{clamp\}\\big\(w\_\{i\}^\{\(t\-1\)\}\\cdot\\gamma\+0\.2,\\;0,\\;1\\big\)&\\text\{if activated\}\\\\\[4\.0pt\] w\_\{i\}^\{\(t\-1\)\}\\cdot\\gamma&\\text\{otherwise\}\\end\{cases\}\(2\)
with fixed decay rateγ=0\.99\\gamma=0\.99and reinforcementΔ​w=0\.2\\Delta w=0\.2\. The clamp operation keeps weights in\[0,1\]\[0,1\]\. This design has several intentional properties:

- •Bounded dynamics: Weights cannot explode or go negative\. OverTTrounds of pure decay without reinforcement,w=0\.99Tw=0\.99^\{T\}, which reaches 0\.366 after 100 rounds—never zero, but clearly distinguishable from reinforced memories\.
- •Asymmetric update: Reinforcement \(\+0\.2\+0\.2\) is roughly 20×\\timeslarger than single\-round decay \(≈−0\.01\\approx\-0\.01\), meaning one activation can offset approximately 20 rounds of neglect\.
- •Nonlinear interaction: Because decay is multiplicative and reinforcement is additive, the order of activations matters\. A memory activated at round 1 and never again differs from one first activated at round 99\.

### 3\.3Long\-Term Memory Migration

Activated memories accumulate a long\-term contribution counter:ℓi\+=0\.2×0\.1=0\.02\\ell\_\{i\}\\mathrel\{\+\}=0\.2\\times 0\.1=0\.02per activation\. Whenℓi≥1\.0\\ell\_\{i\}\\geq 1\.0\(requiring≈50\\approx 50activations\), the memory’s type is upgraded tolong\-term, earning a slower decay rateγlong=0\.995\\gamma\_\{\\text\{long\}\}=0\.995\. This mechanism is designed for multi\-session deployments where a memory accessed consistently across sessions should persist longer\.

### 3\.4Qualitative Case Study: Keywords as Precision Filter

Table[2](https://arxiv.org/html/2607.20458#S3.T2)illustrates the keyword gate’s operation on a representative query fromCAMeR\-Bench\. Without the keyword Jaccard term, three memories would be activated \(cosine\>\>0\.25\); with the hybrid gate, only one exceedsτ=0\.25\\tau=0\.25, correctly suppressing two false positives\.

Table 2:Qualitative example: query “What Python libraries do you recommend for web scraping?” from Cluster A \(Tech\)\. The keyword gate suppresses two false positives \(memories sharing “Python” as a surface term\) while correctly activating the target memory\.Memory TextClustercosJacHybridKeywords \(top\-5\)User uses Python with BeautifulSoup and Scrapy for web scraping tasks\.A \(Tech\)0\.620\.250\.47✓python, beautifulsoup, scrapy, web, scrapingUser has a pet python named Monty that eats mice\.G \(Pets\)0\.470\.070\.31 ✗python, pet, monty, mice, snakeUser attended a Python conference talk about machine learning\.A \(Tech\)0\.520\.090\.35 ✗python, conference, talk, machine, learningThe embedding model gives all three memories moderate\-to\-high cosine scores \(0\.47–0\.62\) because they all mention “Python” and share conversational structure\. The Jaccard term discriminates: only the first memory shares concrete technical keywords \(beautifulsoup, scrapy, web, scraping\) with the query\. This sparsity is the key advantage of symbolic signals—the long\-tail distribution of content words creates natural separation that dense embeddings smooth away\.

### 3\.5Weight\-Augmented Retrieval

Standard retrieval ranks memories by cosine similarity:rank​\(mi\)=cos⁡\(𝐞​\(mi\),𝐞​\(q\)\)\\text\{rank\}\(m\_\{i\}\)=\\cos\(\\mathbf\{e\}\(m\_\{i\}\),\\mathbf\{e\}\(q\)\)\.CAMeRinstead uses weight\-augmented ranking:rank​\(mi\)=wi⋅cos⁡\(𝐞​\(mi\),𝐞​\(q\)\)\\text\{rank\}\(m\_\{i\}\)=w\_\{i\}\\cdot\\cos\(\\mathbf\{e\}\(m\_\{i\}\),\\mathbf\{e\}\(q\)\)\. This means a memory with perfect embedding match but near\-zero weight \(e\.g\., never\-referenced background\) is ranked below a moderately matching but highly\-reinforced memory\. The top\-kkretrieved memories are injected into the LLM context\.

Algorithm 1CAMeR: Per\-Round Processing1:Query

qq, memory store

ℳ\\mathcal\{M\}, gate weight

α=0\.6\\alpha=0\.6, threshold

τ=0\.25\\tau=0\.25, decay

γ=0\.99\\gamma=0\.99, reinforcement

Δ​w=0\.2\\Delta w=0\.2
2:Updated weights

wiw\_\{i\}, retrieved top\-

kkmemories

3:

𝒦​\(q\)←ExtractKeywords​\(q\)\\mathcal\{K\}\(q\)\\leftarrow\\text\{ExtractKeywords\}\(q\)⊳\\trianglerightKeyBERT with YAKE/TF\-IDF fallback

4:

𝐞​\(q\)←Embed​\(q\)\\mathbf\{e\}\(q\)\\leftarrow\\text\{Embed\}\(q\)⊳\\trianglerightall\-MiniLM\-L6\-v2

5:foreach memory

mi∈ℳm\_\{i\}\\in\\mathcal\{M\}do

6:

simcos←cos⁡\(𝐞​\(mi\),𝐞​\(q\)\)\\text\{sim\}\_\{\\text\{cos\}\}\\leftarrow\\cos\(\\mathbf\{e\}\(m\_\{i\}\),\\mathbf\{e\}\(q\)\)
7:

simjac←\|𝒦​\(mi\)∩𝒦​\(q\)\|/\|𝒦​\(mi\)∪𝒦​\(q\)\|\\text\{sim\}\_\{\\text\{jac\}\}\\leftarrow\|\\mathcal\{K\}\(m\_\{i\}\)\\cap\\mathcal\{K\}\(q\)\|\\;/\\;\|\\mathcal\{K\}\(m\_\{i\}\)\\cup\\mathcal\{K\}\(q\)\|
8:

score←α⋅simcos\+\(1−α\)⋅simjac\\text\{score\}\\leftarrow\\alpha\\cdot\\text\{sim\}\_\{\\text\{cos\}\}\+\(1\-\\alpha\)\\cdot\\text\{sim\}\_\{\\text\{jac\}\}
9:if

score\>τ\\text\{score\}\>\\tauthen

10:

wi←clamp​\(wi⋅γ\+Δ​w,0,1\)w\_\{i\}\\leftarrow\\text\{clamp\}\(w\_\{i\}\\cdot\\gamma\+\\Delta w,\\;0,\\;1\)⊳\\trianglerightActivated: reinforce

11:

ai\+=1a\_\{i\}\\mathrel\{\+\}=1
12:

ℓi\+=Δw⋅0\.1\\ell\_\{i\}\\mathrel\{\+\}=\\Delta w\\cdot 0\.1
13:if

ℓi≥1\.0\\ell\_\{i\}\\geq 1\.0and type is short\-termthen

14:

typei←long\-term\\text\{type\}\_\{i\}\\leftarrow\\text\{long\-term\};

γi←0\.995\\gamma\_\{i\}\\leftarrow 0\.995⊳\\trianglerightMigrate

15:endif

16:else

17:

wi←wi⋅γw\_\{i\}\\leftarrow w\_\{i\}\\cdot\\gamma⊳\\trianglerightNot activated: pure decay

18:endif

19:endfor

20:

retrieved←top\-​k​of​ℳ​ranked by​wi⋅cos⁡\(𝐞​\(mi\),𝐞​\(q\)\)\\text\{retrieved\}\\leftarrow\\text\{top\-\}k\\text\{ of \}\\mathcal\{M\}\\text\{ ranked by \}w\_\{i\}\\cdot\\cos\(\\mathbf\{e\}\(m\_\{i\}\),\\mathbf\{e\}\(q\)\)
21:returnretrieved

### 3\.6Per\-Memory MLP Decay \(Exploratory\)

As an extension, we experiment with replacing the fixedγ=0\.99\\gamma=0\.99with a learned per\-memory decay rate predicted by a compact MLP:

γi=0\.80\+0\.19⋅σ​\(MLP​\(\[wi,fi,ri\]\)\)\\gamma\_\{i\}=0\.80\+0\.19\\cdot\\sigma\\big\(\\text\{MLP\}\(\[w\_\{i\},f\_\{i\},r\_\{i\}\]\)\\big\)\(3\)
whereσ\\sigmais the sigmoid function,fi=ai/tf\_\{i\}=a\_\{i\}/tis normalized access frequency,ri=t−tilastr\_\{i\}=t\-t\_\{i\}^\{\\text\{last\}\}is the recency gap, and the MLP has architecture 3→\\rightarrow16→\\rightarrow1 with ReLU activation and Dropout\(0\.1\), totaling 81 parameters\. The output range\[0\.80,0\.99\]\[0\.80,0\.99\]is chosen so that even the most aggressive learned decay cannot destroy a memory faster than0\.80100≈2×10−100\.80^\{100\}\\approx 2\\times 10^\{\-10\}—at which point the weight is effectively zero regardless\. The model is trained via two\-stage learning: \(1\) MSE regression on heuristic pseudo\-labels, then \(2\) pairwise ranking fine\-tuning to preserve ordering\. As discussed in §[6](https://arxiv.org/html/2607.20458#S6), this component shows promise but requires larger training corpora to outperform the fixed 0\.99 baseline\.

## 4Experimental Setup

### 4\.1CAMeR\-BenchDataset

We constructCAMeR\-Bench, a controlled benchmark for memory retention designed to test adaptive differentiation between frequently and infrequently referenced memories\. Table[3](https://arxiv.org/html/2607.20458#S4.T3)summarizes its structure\.

Table 3:CAMeR\-Benchstructure: 8 clusters with graded activation frequency\. Cluster H serves as a never\-referenced background control\.ClusterTopicMemoriesQueriesInjectionExpected\(of 100\)WindowGradientATech Stack1037Rounds 1–30Strong reinforceBFood1019Rounds 5–35Moderate reinforceCTravel1019Rounds 5–35Moderate reinforceDWork1016Rounds 10–40Moderate reinforceEHealth1011Rounds 10–50Weak reinforceFArts109Rounds 15–60Weak reinforceGPets87Rounds 20–70Minimal reinforceHBackgroundControl80Rounds 1–20Pure decay onlyKey design choices:

- •Diverse topics: Clusters span distinct domains \(technology, food, travel, work, health, arts, pets\) to minimize semantic overlap\. This ensures that keyword gate differentiation is tested under realistic ambiguity rather than contrived scenarios\.
- •Progressive injection: New memories are injected with 35% probability per eligible cluster per round within their injection window, simulating the gradual accumulation of information in real dialogues\.
- •Graded frequency: The 6\.2×\\timesratio between highest \(37\) and lowest non\-zero \(7\) query frequency creates a measurable gradient while the zero\-frequency Cluster H provides the cleanest possible measurement of pure decay\.
- •Memory text diversity: Each cluster contains memories with varied phrasing \(“User prefers X”, “User mentioned X”, “User asked about X”\) to prevent surface\-form keyword exploitation\.

### 4\.2Compared Methods

CAMeRvariants\(8 ablation conditions, tested on 20\-round Phase 1 dataset\):

- •−\-MLP: Full keyword gate \+ embedding cosine \+ fixed 0\.99 decay—our primary model; omits the learnable MLP component\.
- •CAMeR\-Full: Keyword gate \+ embedding cosine \+ per\-memory MLP decay \(3\-16\-1\)
- •−\-KG /−\-Jaccard: Embedding\-only gate \+ MLP decay \(ablation: keyword signal removed\)
- •Jaccard\-Only: Keyword\-Jaccard\-only gate \+ MLP decay \(ablation: embedding signal removed\)
- •Global\-MLP: Full gate \+ single global MLP decay for all memories \(ablation: per\-memory personalization\)
- •−\-LongTerm: Full gate \+ MLP decay, no long\-term migration \(ablation: migration contribution\)
- •Uniform\+NoGate: No gating, all memories reinforced equally \+ fixed decay \(ablation: saturation test\)

Competitive baselines\(tested on 100\-roundCAMeR\-Bench\):

- •Memory\-R1\[[3](https://arxiv.org/html/2607.20458#bib.bib3)\]: Reproduced from paper: decay×0\.99\\times 0\.99, update gain 0\.15, delete threshold 0\.05\. For fairness, we use the same embedding and retrieval pipeline asCAMeRfor all baselines\.
- •Oblivion\[[4](https://arxiv.org/html/2607.20458#bib.bib4)\]:w∗=exp\(−λΔt\)w\\mathrel\{\*\}=\\exp\(\-\\lambda\\Delta t\)withλ=0\.01\\lambda=0\.01; matched memories receivew\+=0\.2w\\mathrel\{\+\}=0\.2reinforcement\.
- •SuperLocalMemory\[[2](https://arxiv.org/html/2607.20458#bib.bib2)\]:R​\(t\)=e−t/SR\(t\)=e^\{\-t/S\}withS0=5S\_\{0\}=5,SSstrengthened by\+2\+2per review\.
- •Full History: All memories retained at full weight \(w=1\.0w=1\.0\), all injected into LLM context\.
- •No Memory: Empty memory store, zero\-context lower bound\.

### 4\.3Metrics

- •Scissors Gap\(↑\\uparrow\): Mean weight of activated \(ai\>0a\_\{i\}\>0\) memories minus mean weight of never\-activated \(ai=0a\_\{i\}=0\) background memories\. This is our primary metric—it directly measures the system’s ability to differentiate relevant from irrelevant information\.
- •Per\-Cluster Weight Gradient: Average final weight per cluster, sorted by query frequency\. Tests whether the system produces a monotonic or near\-monotonic gradient matching activation intensity\.
- •Token Savings: Cumulative input tokens consumed across 100 rounds vs\. Full History baseline\.
- •Weight\-Augmented Precision: Whether ranking bywi×w\_\{i\}\\timescosine improves retrieval precision over pure cosine ranking\.
- •Decay Divergence\(for MLP conditions\): Mean weight difference between activated and non\-activated memories within the MLP\-managed condition; higher is better\.

### 4\.4Implementation

CAMeRis implemented in≈\\approx1,200 lines of Python\. ChromaDB provides vector storage; KeyBERT \(all\-MiniLM\-L6\-v2, 384\-dim\) extracts keywords; SentenceTransformer provides full\-text embeddings\. The 3\-16\-1 DecayMLP \(81 params\) is implemented in PyTorch and trained for 200 epochs \(MSE regression\) \+ 100 epochs \(pairwise ranking\) on heuristic pseudo\-labels derived from Phase 1 trajectories\. All experiments use seed 42 and run on an Intel i7 CPU with 32GB RAM\. The full 100\-roundCAMeR\-Benchexperiment \(5 conditions\) completes in≈\\approx10 minutes\.

## 5Results and Analysis

### 5\.1Main Results: Retention Quality

Table[4](https://arxiv.org/html/2607.20458#S5.T4)shows results onCAMeR\-Benchafter 100 rounds\.

Table 4:Main results onCAMeR\-Bench\(100 rounds, final pool 67 memories\)\.Bold= best scissors gap\. Activated = memories with≥1\\geq 1access; Background = never\-accessed memories \(Cluster H\)\.MethodAvg\.wwwactw\_\{\\text\{act\}\}wbgw\_\{\\text\{bg\}\}Scissors↑\\uparrowCAMeR\-Full \(ours\)0\.6210\.6440\.6050\.039CAMeR\-EmbOnly \(ablation\)0\.8340\.8360\.8130\.024Memory\-R1\[[3](https://arxiv.org/html/2607.20458#bib.bib3)\]0\.6050\.6060\.6050\.001Oblivion\[[4](https://arxiv.org/html/2607.20458#bib.bib4)\]0\.1150\.0710\.147−\-0\.076SuperLocal\[[2](https://arxiv.org/html/2607.20458#bib.bib2)\]0\.0360\.0010\.061−\-0\.060Finding 1: Keyword gate outperforms embedding\-only\.CAMeR\-Full achieves scissors gap 0\.039, 1\.6×\\timeslarger than EmbOnly’s 0\.024\. The key mechanism: EmbOnly shows near\-uniform weights \(0\.813–0\.836\) because embedding cosine distributes activation broadly; the keyword Jaccard term introduces sparsity that concentrates reinforcement on genuinely related memories\.

Finding 2: Time\-driven baselines collapse\.Oblivion’s exponential decay \(exp⁡\(−0\.01×100\)=0\.368\\exp\(\-0\.01\\times 100\)=0\.368without reinforcement\) and SuperLocalMemory’s Ebbinghaus curve \(e−100/5≈2×10−9e^\{\-100/5\}\\approx 2\\times 10^\{\-9\}\) are calibrated for short sequences\. Over 100 rounds, both drive all weights toward zero: Oblivion reaches 0\.115, SuperLocalMemory reaches<<0\.04\. Their decay functions lack the plateau behavior of multiplicative0\.99t0\.99^\{t\}decay, which asymptotically approaches zero rather than exponentially collapsing\. The negative scissors gaps indicate that background memories \(injected early, decaying longer\) actually havehigherweights than recently\-injected activated memories—a perverse outcome\.

Finding 3: Memory\-R1 produces no differentiation\.Its heuristic operations \(decay×\\times0\.99, reinforcement\+\+0\.15, delete<<0\.05\) converge to a steady state around 0\.605 regardless of access history\. This is a consequence of the update magnitude:0\.605×0\.99\+0\.15≈0\.7490\.605\\times 0\.99\+0\.15\\approx 0\.749, but the next round’s decay brings activated and non\-activated weights back to similar levels\. The equilibrium weight satisfiesw=w×0\.99w=w\\times 0\.99when not activated andw=min⁡\(w×0\.99\+0\.15,1\)w=\\min\(w\\times 0\.99\+0\.15,1\)when activated; with irregular activation, all memories converge near 0\.60–0\.65\.

### 5\.2Per\-Cluster Weight Gradient

Figure[2](https://arxiv.org/html/2607.20458#S5.F2)shows per\-cluster final weights forCAMeR\-Full\. Cluster A \(37 queries\) reaches 0\.779 while Cluster H \(0 queries\) drops to 0\.483—a within\-system gap of 0\.296\. The intermediate clusters \(B–G\) show a general downward trend correlated with activation frequency \(Spearmanρ=0\.90\\rho=0\.90,p<0\.01p<0\.01\), though not perfectly monotonic due to two factors: \(1\) semantic adjacency between neighboring clusters \(e\.g\., Food and Health share meal\-planning vocabulary\) causes cross\-cluster activation, and \(2\) memories in later\-injected clusters have fewer cumulative decay rounds\.

![Refer to caption](https://arxiv.org/html/2607.20458v1/figures/camer_bench_clusters.png)Figure 2:Per\-cluster average final weights onCAMeR\-Bench\(100 rounds\)\. Cluster A \(37 queries\) vs\. Cluster H \(0 queries\) shows a 0\.296 weight gap\. Expected gradient based on query frequencies shown in parentheses\.
### 5\.3Token Consumption and Retrieval Quality

Table[5](https://arxiv.org/html/2607.20458#S5.T5)reports cumulative token consumption over 100 rounds\.CAMeRretrieves top\-5 memories per round; FullHistory injects all available memories \(growing from 20 to 67 over the experiment\)\.

Table 5:Token consumption analysis \(100 rounds\)\. Precision Gain = improvement in retrieval precision from weight\-augmented vs\. pure cosine ranking\.MethodCumul\. Tokensvs\. FullHistAvg\./RoundPrec\. GainFullHistory230,984–2,310–CAMeR\(top\-5\)38,723−\-83\.2%387\+0\.008NoMemory15,998−\-93\.1%160–![Refer to caption](https://arxiv.org/html/2607.20458v1/figures/token_experiment.png)Figure 3:Cumulative token savings by retrieval method\.CAMeR\-Full achieves 83\.2% reduction versus FullHistory while producing a positive weight\-augmented precision gain \(\+0\.008\); EmbOnly and NoGate conditions show zero or negative gain, indicating their weight signals carry noise\.The causal chain\.The token savings themselves are expected \(any top\-kkretrieval saves tokens\)\. The important finding is theprecision gaincolumn: weight\-augmented ranking improves retrieval precision by\+0\.008\+0\.008over pure cosine for CAMeR\-Full, while CAMeR\-EmbOnly shows−0\.002\-0\.002\(its weights carry noise\) and NoGate shows0\.0000\.000\(all weights identical, no signal\)\. This establishes the three\-step causal chain:

keyword gate→\\rightarrowhigher\-quality weight signals→\\rightarrowimproved retrieval precision→\\rightarrowtoken efficiency

Without the keyword gate, weight signals are either noisy \(EmbOnly\) or non\-existent \(NoGate\), breaking the chain at the first link\.

### 5\.4Ablation Study

Table[6](https://arxiv.org/html/2607.20458#S5.T6)presents the 8\-condition ablation on the 20\-round Phase 1 dataset\. This smaller\-scale experiment \(14 memories, 20 rounds\) isolates component contributions before scaling\.

Table 6:Ablation results \(8CAMeRvariants, 20 rounds, 14 memories\)\. Ranked by practical utility\.ConditionGateDecaywavgw\_\{\\text\{avg\}\}Assessment−\-MLP \(our primary\)Full hybridFixed 0\.990\.936Best: gate works, decay stableUniform\+NoGateNoneFixed 0\.991\.000Saturated: no differentiation−\-KG /−\-JaccardEmbed\-onlyMLP \(3\-16\-1\)0\.336MLP too aggressive on small dataCAMeR\-FullFull hybridMLP \(3\-16\-1\)0\.230MLP dominates, masks gate benefitGlobal\-MLPFull hybridGlobal MLP0\.238No per\-memory adaptation benefit−\-LongTermFull hybridMLP0\.231Migration irrelevant at 20 roundsJaccard\-OnlyJacc\.\-onlyMLP0\.104Single signal insufficientKey findings from ablation:

1. 1\.Gate drives differentiation, not decay\.The−\-MLP condition dominates all others \(0\.936 avg\. weight, best scissors gap\), confirming that the keyword\-gated activation—not learned decay—is the primary performance driver\.
2. 2\.Gating is necessary\.Without gating \(Uniform\+NoGate\), all weights saturate at 1\.0 within 5 rounds\. Any adaptive retention systemmustselectively reinforce\.
3. 3\.MLP underfits at this scale\.All MLP\-based conditions converge to weights 0\.10–0\.34, far below useful levels\. With 81 parameters and≈\\approx280 training samples \(14 memories×\\times20 rounds\), the MLP learns an overly aggressive decay that collapses weights\. This is a data limitation, not an architectural flaw—see §[6](https://arxiv.org/html/2607.20458#S6)for analysis\.
4. 4\.Global MLP≈\\approxper\-memory MLP\.The absence of benefit from per\-memory personalization \(Global\-MLP: 0\.238 vs\. CAMeR\-Full: 0\.230\) suggests that the 3 input features\(w,f,r\)\(w,f,r\)do not carry sufficient per\-memory signal at this scale\.
5. 5\.Dual signal matters\.Jaccard\-only \(0\.104\) underperforms embedding\-only \(0\.336\), confirming that the embedding signal carries the bulk of semantic information—the keyword term provides precision, not recall\.

### 5\.5Weight Trajectories

Figure[4](https://arxiv.org/html/2607.20458#S5.F4)shows memory weight evolution from the Phase 1 experiment \(20 rounds, 14 memories\)\.

![Refer to caption](https://arxiv.org/html/2607.20458v1/figures/weight_curves.png)Figure 4:Weight trajectories from 20\-round controlled experiment\. Solid: activated memories \(Clusters A–D\); dashed: background \(Cluster X, never accessed\); black dashed: theoretical pure decay0\.99t0\.99^\{t\}\. The scissors gap reaches 13\.7–16\.2 percentage points\.Activated memories maintain weights 0\.92–1\.0 through repeated reinforcement; background memories follow the theoretical pure decay curve \(0\.9920=0\.8180\.99^\{20\}=0\.818\), yielding a 13\.7–16\.2pp scissors gap\. This validates the reinforcement mechanism’s basic operation before scaling to 100 rounds\. The gap is larger than the 100\-round result \(0\.039\) because: \(1\) 20 rounds of 0\.99 decay preserves 81\.8% of initial weight vs\. 36\.6% at 100 rounds, giving reinforcement less cumulative decay to offset; \(2\) the controlled Phase 1 design uses 4 clearly separated clusters rather than 8 overlapping ones, reducing cross\-talk\.

## 6Discussion

### 6\.1Why Does the Keyword Gate Work?

The keyword gate’s effectiveness has a mathematical explanation rooted in the sparsity of keyword overlap\. For a random pair of English sentences, the probability of sharing any keyword after KeyBERT extraction \(top\-5 keywords per text\) is approximately 0\.08–0\.12 in our corpus\. This means≈\\approx90% of memory\-query pairs receive a Jaccard score of 0\.0, adding zero to the hybrid score\. The combined score for these pairs is simply0\.6×0\.6\\timescosine similarity—requiring cosine\>0\.25/0\.6≈0\.42\>0\.25/0\.6\\approx 0\.42to trigger activation\.

For the≈\\approx10% of pairs that do share keywords, the Jaccard term is typically 0\.15–0\.40, contributing 0\.06–0\.16 to the hybrid score\. This can push a memory with moderate cosine \(0\.30–0\.35\) over the threshold, or conversely, a memory with high cosine but zero keyword overlap stays below threshold\.

In contrast, embedding cosine between any two English sentences rarely falls below 0\.20–0\.25 due to shared function words, syntactic structure, and latent semantic neighborhoods\. This compressed dynamic range makes it difficult to find a single threshold that both captures true positives and rejects false positives\. The Jaccard term effectivelystretchesthe score distribution, creating a longer tail of low scores and enabling cleaner separation\.

### 6\.2Why Not Just Use a Classifier?

A natural question is whether the keyword gate could be replaced by a small classifier \(e\.g\., logistic regression over embedding dimensions\) trained to predict memory relevance\. We deliberately chose not to pursue this for three reasons:

1. 1\.Cold\-start: A classifier requires labeled training data \(relevant/irrelevant memory\-query pairs\), which is unavailable at deployment time\. The keyword gate is zero\-shot—it works immediately\.
2. 2\.Interpretability: WhenCAMeRsuppresses a memory, the keyword overlap \(or lack thereof\) provides an audit trail\. A classifier offers no such explanation\.
3. 3\.Domain robustness: Keywords are domain\-agnostic \(“Python” means the same thing regardless of corpus\), while embedding\-based classifiers may learn corpus\-specific artifacts\.

That said, a hybrid system using keyword\-gated pseudo\-labels to bootstrap a classifier for improved retrieval is a promising direction for future work\.

### 6\.3The Role of Learnable Decay

Our experiments with the 3\-16\-1 DecayMLP establish an important negative result: 81 parameters and heuristic pseudo\-labels are insufficient to learn useful per\-memory decay rates at the scales tested \(14–76 memories\)\. We do not consider this a failure of the approach but rather alower bound: it identifies the minimum scale at which learned decay becomes viable\.

We hypothesize that effective learned decay requires: \(a\) richer input features \(memory text length, embedding norm, keyword set size, topic diversity signals\), expanding from 3 to 6–8 dimensions; \(b\) training on naturally occurring dialogues with implicit relevance signals \(user correction, re\-asking, topic drift\) rather than heuristic pseudo\-labels; and \(c\) a larger parameter budget \(e\.g\., 6→\\rightarrow32→\\rightarrow16→\\rightarrow1, 400\+ parameters\) with correspondingly larger training corpora\. We release our MLP implementation and training code to facilitate such follow\-up work\.

### 6\.4Limitations

1. 1\.Scale:CAMeR\-Benchuses 76 memories over 100 rounds\. Production systems may involve thousands of memories over thousands of interactions\. The keyword gate’s sparsity properties suggest it should scale well \(Jaccard computation isO​\(\|𝒦\|\)O\(\|\\mathcal\{K\}\|\), independent of memory count, and the sparsity advantage grows with vocabulary diversity\), but empirical validation at scale is needed\.
2. 2\.Language dependence: KeyBERT’s extraction quality varies across languages\. Our experiments use English only; languages with different morphological complexity or writing systems may require language\-specific keyword extractors\.
3. 3\.Static hyperparameters:α=0\.6\\alpha=0\.6,τ=0\.25\\tau=0\.25,γ=0\.99\\gamma=0\.99, andΔ​w=0\.2\\Delta w=0\.2are fixed\. In principle, these could be adapted per\-deployment based on dialogue tempo, domain specificity, or user preferences\. We chose fixed values for simplicity and reproducibility\.
4. 4\.Synthetic evaluation:CAMeR\-Benchuses synthetic, template\-generated queries and memories\. While this enables controlled experimentation, deployment in real user\-facing systems with genuine dialogue dynamics is needed for ecological validation\.
5. 5\.Embedding model dependence: We use all\-MiniLM\-L6\-v2 \(384\-dim\)\. Larger embedding models may produce different cosine similarity distributions, potentially changing the optimalα\\alpha\. We chose a lightweight model to keep the system practical for CPU\-only deployment\.

## 7Conclusion

We presentedCAMeR, a memory retention framework for LLM agents built on keyword\-gated hybrid activation\. Through controlled experiments onCAMeR\-Bench\(76 memories, 100 rounds, 8 topic clusters\) and comprehensive ablation across 8 variant conditions and 5 baselines, we demonstrated:

1. 1\.Keyword\-level symbolic gating provides a 1\.6×\\timesimprovement in memory differentiation over embedding\-only approaches, by exploiting the natural sparsity of word\-level overlap to suppress false\-positive activations\.
2. 2\.CAMeR’s top\-5 retrieval saves 83\.2% tokens versus full\-context approaches, while the keyword gate produces informative weight signals that improve retrieval precision—a causal chain absent in embedding\-only systems\.
3. 3\.Fixed\-rate decay \(0\.99\) with conditional reinforcement outperforms both learned per\-memory decay \(at the 81\-parameter scale, serving as a lower bound for future work\) and time\-driven decay functions \(which collapse over extended sequences\)\.
4. 4\.CAMeR\-Benchprovides a reproducible, controlled testbed for memory retention research, filling a gap left by retrieval\-focused benchmarks that cannot measure adaptive differentiation\.

Our core finding—that a simple symbolic signal \(keyword overlap\) meaningfully improves neural memory gating—suggests that the integration of symbolic and sub\-symbolic signals, a principle with deep roots in AI, remains underexploited in modern LLM memory systems\. We hopeCAMeRandCAMeR\-Benchencourage further investigation into hybrid gating mechanisms for adaptive agent memory\.

## References

- \[1\]S\. Sukhbaatar, D\. Ju, S\. Poff, S\. Roller, A\. Szlam, J\. Weston, and A\. Fan\.Not all memories are created equal: Learning to forget by expiring\.ICML, 2021\.
- \[2\]V\. P\. Bhardwaj\.SuperLocalMemory V3\.3: The living brain — biologically\-inspired forgetting, cognitive quantization, and multi\-channel retrieval for zero\-LLM agent memory systems\.arXiv:2604\.04514, 2026\.
- \[3\]S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, H\. Schütze, V\. Tresp, and Y\. Ma\.Memory\-R1: Enhancing large language model agents to manage and utilize memories via reinforcement learning\.arXiv:2508\.19828, 2025\.
- \[4\]Oblivion: Exponential time\-decay heuristic for LLM memory retention\.Heuristic baseline \(not a published paper\), 2025\.
- \[5\]Mem0 Inc\.Mem0: The memory layer for personalized AI\.[https://github\.com/mem0ai/mem0](https://github.com/mem0ai/mem0), 2024\.
- \[6\]A\. Maharana, D\.\-H\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang\.Evaluating very long\-term conversational memory of LLM agents\.ACL, 2024\.
- \[7\]D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\.\-W\. Chang, and D\. Yu\.LongMemEval: Benchmarking chat assistants on long\-term interactive memory\.ICLR, 2025\.
- \[8\]N\. Reimers and I\. Gurevych\.Sentence\-BERT: Sentence embeddings using Siamese BERT\-networks\.EMNLP\-IJCNLP, 2019\.
- \[9\]M\. Grootendorst\.KeyBERT: Minimal keyword extraction with BERT\.[https://github\.com/MaartenGr/KeyBERT](https://github.com/MaartenGr/KeyBERT), 2020\.
- \[10\]G\. Salton and C\. Buckley\.Term\-weighting approaches in automatic text retrieval\.Information Processing & Management, 24\(5\):513–523, 1988\.
- \[11\]R\. Campos, V\. Mangaravite, A\. Pasquali, A\. Jorge, C\. Nunes, and A\. Jatowt\.YAKE\! Keyword extraction from single documents using multiple local features\.Information Sciences, 509:257–289, 2020\.
- \[12\]J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein\.Generative agents: Interactive simulacra of human behavior\.UIST, 2023\.
- \[13\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Kuttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela\.Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.NeurIPS, 2020\.
- \[14\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\.ReAct: Synergizing reasoning and acting in language models\.ICLR, 2023\.
- \[15\]C\. Packer, V\. Fang, S\. G\. Patil, K\. Lin, S\. Wooders, I\. Stoica, and J\. E\. Gonzalez\.MemGPT: Towards LLMs as operating systems\.arXiv:2310\.08560, 2023\.
- \[16\]W\. Liu, Z\. Tang, J\. Li, K\. Chen, and M\. Zhang\.MemLong: Memory\-augmented retrieval for long text modeling\.arXiv:2408\.16967, 2024\.