MemoryAthena: Adaptive Routing over Latent and Generated Memories

# MemoryAthena: Adaptive Routing over Latent and Generated Memories
Source: [https://arxiv.org/html/2609.25853](https://arxiv.org/html/2609.25853)
Guangsheng YuAffiliation:University of Technology SydneyJuyuan ZhangAffiliation:University of Science and Technology of ChinaXu WangAffiliation:University of Technology SydneyZhibo ManAffiliation:ELLIS Institute of FinlandAffiliation:University of TurkuHaonan ZhangAffiliation:Shanghai Jiao Tong UniversityShaoxiong JiAffiliation:ELLIS Institute of FinlandAffiliation:University of Turku

###### Abstract

Learned\-memory methods store information in an explicit table and consume it through a separate reader, allowing addressing, storage, and reading to be modified independently\. Prior work on cross\-model memory transfer exploits this separation to reuse learned memory across frozen backbones through an adapted reader, while the representation consumed by the model still originates from stored memory\. This raises a natural question: must useful memory always be retrieved from storage, or can it also be generated? We investigate this question withMemoryAthena, a memory interface with three pathways: direct Engram retrieval \(E\), generation from retrieved Engram cues \(GE\), and generation from causal backbone states without consulting the memory table \(GH\)\. Generated memory is not uniformly better than direct retrieval: it can complement E in one context but interfere with it in another\.MemoryAthenatherefore treats E as an explicit anchor and learns when a generated representation should intervene\. With the backbone, memory, generators, and readers frozen, a lightweight causal routing head is trained from counterfactual future\-token likelihood advantages of GE and GH relative to E\. At inference time, an admitted candidate modifies the E residual through bounded interpolation, while rejection recovers the direct pathway exactly\. On question answering,MemoryAthenaraises the five\-task average from 37\.65 to 39\.28 over the direct pathway of the same checkpoint, while the six\-task general\-NLP average increases from 76\.73 to 79\.13\. The complete memory\-side system contains approximately 201M parameters, excluding the frozen backbone\. Further analyses show that the utility of E, GE, and GH varies across tasks and inputs, while gold\-label oracles reveal additional complementarity among the three pathways\. These results support generated memory as a selective correction to direct retrieval rather than a universal replacement, and highlight routing when, which, and how strongly to intervene as the central challenge\.

## 1Introduction

Retrieval\-augmented generation supplies a language model with external text\([11](https://arxiv.org/html/2609.25853#bib.bib4)\), nearest\-neighbor language models supply examples drawn from a non\-parametric datastore\([9](https://arxiv.org/html/2609.25853#bib.bib5)\), and learned\-memory approaches supply trainable representations read at inference time\([21](https://arxiv.org/html/2609.25853#bib.bib1),[2](https://arxiv.org/html/2609.25853#bib.bib3)\)\. All three pass a stored item to the model unchanged\. Engram makes that structure explicit by keeping memory outside the backbone, storing information in an addressable table and consuming it through a lightweight neural interface\([2](https://arxiv.org/html/2609.25853#bib.bib3)\)\. Prior work treats such a memory as a reusable artifact across language\-model backbones and separates a memory system into three roles\([13](https://arxiv.org/html/2609.25853#bib.bib2)\)\.*Addressing*determines where to access,*memory*stores the representations, and*reading*transforms a retrieved representation into a form the backbone can consume\. None of the systems above asks whether a memory representation must be retrieved from a stored table, or whether useful memory can instead be generated\.

Treating memory as reconstruction rather than literal readout makes the question tractable\. An addressable memory supplies an index or a cue, from which a neural model can reconstruct a richer internal representation conditioned on its current context\. The stored memory then need not be the final representation injected into the model, and is instead a substrate from which a latent memory representation is generated\. Generation also need not start from an external lookup, because the model’s own causal hidden states may already carry enough contextual evidence to construct a useful latent representation\. Two forms of generation therefore accompany direct memory access, one conditioned on retrieved memory cues and one conditioned on the model’s causal context\.

Building on the addressing, memory, and reader decomposition of prior work\([13](https://arxiv.org/html/2609.25853#bib.bib2)\), this paper considers three memory pathways\. Direct Engram retrieval \(E\) follows the conventional interface and reads an addressable memory entry unchanged\. Generation from retrieved Engram cues \(GE\) treats those cues as conditions for a latent memory representation that the backbone consumes in its place\. Generation from causal backbone states \(GH\) forms that representation without reading the external memory table\. The three differ in how far the representation entering the model departs from what storage holds, ranging from*direct retrieval*through*retrieval\-conditioned generation*to*context\-conditioned generation*\. GE and GH are alternative memory\-side representations attached to the same frozen backbone rather than additional language models\.

Generated memories are not uniformly better than direct retrieval\. A generated representation can be highly useful for one input yet unnecessary or even harmful for another\. This heterogeneity is precisely what makes generated memory a routing problem rather than a replacement problem: if GE or GH were consistently superior to E, one could simply replace the direct pathway\. Instead, the useful regime is selective intervention, where a strong direct\-memory pathway is retained and generated memory is introduced only when it is expected to add value\. The resulting decision is asymmetric\. Conventional conditional routing chooses among several equivalent experts\([18](https://arxiv.org/html/2609.25853#bib.bib6)\), whereas here E already provides a strong direct\-memory reference\. The model must therefore determine whether a generated memory should intervene, which generated pathway should be used, and how strongly it should modify the direct representation\. Learning these decisions from general text rather than downstream task labels makes the problem particularly challenging\.

This paper proposesMemoryAthena, an E\-anchored framework for integrating direct and generated memories\. Rather than routing symmetrically among E, GE, and GH,MemoryAthenatreats E as an explicit reference\. A lightweight causal head predicts the E\-relative advantage and confidence of each generated candidate, which is admitted only when both exceed the routing criteria\. The resulting memory residual is

r=e\+α​\(g−e\),0≤α≤1,r=e\+\\alpha\(g\-e\),\\qquad 0\\leq\\alpha\\leq 1,\(1\)whereα\\alphacontrols the intervention strength\. If no generated candidate is admitted, the model recovers the direct E pathway exactly\. Generated memory is thus treated as a candidate correction rather than a replacement for direct retrieval\.

To train the router without downstream labels, we freeze the backbone, memory, generators, and readers and compare the E, GE, and GH endpoints under teacher forcing\. For a generated sourcess, we define its token\-level advantage over E as

as,t=log⁡ps​\(xt\+1∣x≤t\)−log⁡pE​\(xt\+1∣x≤t\),a\_\{s,t\}=\\log p\_\{s\}\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\)\-\\log p\_\{E\}\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\),\(2\)and aggregate these differences over multiple future horizons to supervise the routing head\. Future tokens are used only to construct training targets; at inference time, routing remains causal\. Memory selection therefore becomes an E\-relative utility prediction problem\. Our contributions are threefold:

- •From memory retrieval to memory generation\.Building on the decomposition of external memory into addressing, storage, and reading\([13](https://arxiv.org/html/2609.25853#bib.bib2)\), we introduce a three\-path memory interface spanning direct retrieval \(E\), generation from retrieved Engram cues \(GE\), and generation from causal backbone states \(GH\)\. We use it to investigate whether the representation consumed by a language model must be explicitly stored, or can instead be generated from memory cues or contextual states\.
- •Routing under heterogeneous memory utility\.We formulate generated memory as a conditional intervention problem: GE and GH can complement direct retrieval, but neither is uniformly preferable\.MemoryAthenatherefore retains E as an explicit reference and learns when, which, and how strongly a generated representation should intervene, using bounded interpolation with exact fallback to the direct pathway\.
- •Counterfactual advantage distillation without downstream supervision\.We construct routing targets from future\-token likelihood differences among frozen memory pathways and distill them into a lightweight causal head\. This separates learning*how to construct*memory representations from learning*when to use*them\.

We evaluateMemoryAthenaon question answering and general NLP tasks\. It improves all five QA summary metrics and five of six NLP tasks over the direct\-memory pathway of the same checkpoint\. Analyses of the individual endpoints reveal complementary successful predictions across E, GE, and GH\.

## 2Background and Problem Setup

External and learned memory\.A language model can reach information its backbone parameters do not hold, by retrieval or by learned memory\. Retrieval\-augmented generation conditions the output on retrieved text\([11](https://arxiv.org/html/2609.25853#bib.bib4)\), whilekkNN\-LMs interpolate neural predictions with a distribution drawn from a nearest\-neighbor datastore\([9](https://arxiv.org/html/2609.25853#bib.bib5)\)\. Learned\-memory approaches move the memory into trained representations instead, so that MLP Memory learns a parametric memory module\([21](https://arxiv.org/html/2609.25853#bib.bib1)\)and Engram introduces an addressable conditional memory based on causalNN\-gram lookup\([2](https://arxiv.org/html/2609.25853#bib.bib3)\)\. Engram is the direct memory substrate throughout, and the one thing varied here is how the representation consumed by the backbone is constructed from that stored memory or built beside it\.

Reusable memory and target\-side reading\.Cross\-model memory transfer separates a memory system into three roles\([13](https://arxiv.org/html/2609.25853#bib.bib2)\)\.*Addressing*determines what memory is accessed,*memory*holds the reusable representations, and*reading*adapts a retrieved representation to the target backbone\. A memory learned with one language model can then remain frozen and be reused by another backbone through an adapted target\-side reader\. The stored representation and the representation the model finally consumes therefore need not be identical, which is the property this paper builds on\.

Letx1:Tx\_\{1:T\}be a token sequence andfθf\_\{\\theta\}a frozen autoregressive backbone\. An addressable memoryMϕM\_\{\\phi\}retrieves

mt=Mϕ​\[canon⁡\(x≤t\)\],m\_\{t\}=M\_\{\\phi\}\[\\operatorname\{canon\}\(x\_\{\\leq t\}\)\],\(3\)wherecanon⁡\(⋅\)\\operatorname\{canon\}\(\\cdot\)denotes the canonical addressing rule\. A reader then maps the retrieved memory and the current hidden statehtℓh\_\{t\}^\{\\ell\}into a residual contribution that is injected as

htℓ←htℓ\+rtℓ\.h\_\{t\}^\{\\ell\}\\leftarrow h\_\{t\}^\{\\ell\}\+r\_\{t\}^\{\\ell\}\.\(4\)MemoryAthenastarts from this addressing, memory and reader view\.

From memory reading to memory generation\.If the representation consumed by the backbone is already produced through a learned interface, it need not be obtained by reading the stored memory directly, and three alternatives follow\. TheEpathway reads the retrieved Engram representation directly and produces a residualetℓe\_\{t\}^\{\\ell\}\. TheGEpathway generates a latent memory representation conditioned on retrieved Engram cues, producinggGE,tℓg\_\{\\mathrm\{GE\},t\}^\{\\ell\}\. TheGHpathway generates a latent memory representation from causal backbone states without consulting the external memory table, producinggGH,tℓg\_\{\\mathrm\{GH\},t\}^\{\\ell\}\. All three are residual representations in the same target hidden\-state space, attached to the same frozen backbone\.

Fors∈\{E,GE,GH\}s\\in\\\{\\mathrm\{E\},\\mathrm\{GE\},\\mathrm\{GH\}\\\}, we denote bypsp\_\{s\}the endpoint distribution obtained when pathwayssis used throughout the configured memory\-injection sites, and these endpoints are the common reference against which direct and generated memory representations are compared\.

Conditional routing over memory representations\.Mixture\-of\-experts methods learn input\-dependent combinations of expert outputs\([18](https://arxiv.org/html/2609.25853#bib.bib6),[3](https://arxiv.org/html/2609.25853#bib.bib22)\), and memory\-augmented language models have used learned selection mechanisms\([16](https://arxiv.org/html/2609.25853#bib.bib20)\)\. Our setting is asymmetric\. E is already a usable direct\-memory pathway, whereas GE and GH are candidate modifications to that reference\. The decision is therefore whether a generated representation provides additional utility over E, which candidate should intervene, and how strongly it should modify the direct residual\. Section[3](https://arxiv.org/html/2609.25853#S3)develops the routing mechanism for this E\-relative decision\.

## 3METHOD

MemoryAthenaseparates memory construction from memory selection\. Training proceeds in three stages\. First, an addressable memory is learned under causal language\-modeling supervision while the source backbone is frozen\. Second, the memory and target backbone are fixed, and the memory\-side interfaces are adapted to construct the direct pathway E and the generated pathways GE and GH\. Third, all memory pathways are frozen and only a lightweight routing head is trained to predict the E\-relative utility of GE and GH from counterfactual future\-token supervision\. The model therefore first learns*how to construct candidate memory representations*and then learns*when and how strongly a generated representation should modify the direct memory*\. Detailed objectives, architectures, and optimization settings are provided in Appendix[C](https://arxiv.org/html/2609.25853#A3)\.

![Refer to caption](https://arxiv.org/html/2609.25853v1/overview.png)Figure 1:Overview ofMemoryAthena\. The framework extends direct Engram retrieval \(E\) with two generated\-memory pathways, generation from retrieved Engram cues \(GE\) and generation from causal backbone states \(GH\)\. A lightweight E\-anchored router predicts the relative advantage of GE and GH, conditionally admits a generated candidate, and controls its contribution through bounded interpolation with exact fallback to E\. The routing head is trained from counterfactual future\-token likelihood differences while the memory pathways remain frozen\.### 3\.1Stage1: Direct and Generated Memory Pathways

We consider a frozen autoregressive backbone augmented with three memory\-side pathways that differ in how the representation injected into the backbone is constructed\. Lethtℓh\_\{t\}^\{\\ell\}denote the backbone hidden state at positionttand injection layerℓ\\ell, and letmtm\_\{t\}denote the representation retrieved from the addressable Engram memory\. The three pathways produce residual contributions in the same target hidden space:

etℓ\\displaystyle e\_\{t\}^\{\\ell\}=REℓ​\(htℓ,mt\),\\displaystyle=R\_\{\\mathrm\{E\}\}^\{\\ell\}\\left\(h\_\{t\}^\{\\ell\},m\_\{t\}\\right\),\(5\)gGE,tℓ\\displaystyle g\_\{\\mathrm\{GE\},t\}^\{\\ell\}=RGEℓ​\(htℓ,zE,tℓ\),\\displaystyle=R\_\{\\mathrm\{GE\}\}^\{\\ell\}\\left\(h\_\{t\}^\{\\ell\},z\_\{\\mathrm\{E\},t\}^\{\\ell\}\\right\),\(6\)gGH,tℓ\\displaystyle g\_\{\\mathrm\{GH\},t\}^\{\\ell\}=RGHℓ​\(htℓ,zH,tℓ\)\.\\displaystyle=R\_\{\\mathrm\{GH\}\}^\{\\ell\}\\left\(h\_\{t\}^\{\\ell\},z\_\{\\mathrm\{H\},t\}^\{\\ell\}\\right\)\.\(7\)
TheEpathway directly reads the retrieved Engram representation\. TheGEpathway first generates a latent memory representationzE,tℓz\_\{\\mathrm\{E\},t\}^\{\\ell\}conditioned on retrieved Engram cues and then maps it into the backbone hidden space\. TheGHpathway instead generateszH,tℓz\_\{\\mathrm\{H\},t\}^\{\\ell\}from causal backbone states obtained in a separate pass with memory injection disabled\.

GE and GH are memory\-side representations rather than independent language models, and all three pathways operate around the same frozen backbone\.

### 3\.2Stage2: E\-Relative Advantage Distillation

Generated memory is not uniformly preferable to direct memory\. We therefore treat the direct E pathway as an explicit reference and learn whether each generated candidate is expected to improve upon it\. After the memory pathways have been learned, we freeze the backbone, memory, generators and readers\. For each endpoints∈\{E,GE,GH\}s\\in\\\{\\mathrm\{E\},\\mathrm\{GE\},\\mathrm\{GH\}\\\}, we obtain a next\-token distributionpsp\_\{s\}by using that pathway alone under teacher forcing\. For a generated sources∈\{GE,GH\}s\\in\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}, we define its token\-level advantage relative to E as

as,t=log⁡ps​\(xt\+1∣x≤t\)−log⁡pE​\(xt\+1∣x≤t\)\.a\_\{s,t\}=\\log p\_\{s\}\\left\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\\right\)\-\\log p\_\{\\mathrm\{E\}\}\\left\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\\right\)\.\(8\)A positive value indicates that the generated pathway assigns greater likelihood to the observed future token than the direct\-memory pathway\. Token\-level advantages are aggregated over multiple future positions to obtain a smoother training target, denoted byAs,tA\_\{s,t\}\. A lightweight routing head predicts an advantage and a confidence scorecs,tc\_\{s,t\}for each generated candidate:

\(A^s,tℓ,cs,tℓ\)=qℓ​\(stopgrad⁡\(ϕs,tℓ\)\),\\left\(\\widehat\{A\}\_\{s,t\}^\{\\ell\},c\_\{s,t\}^\{\\ell\}\\right\)=q^\{\\ell\}\\left\(\\operatorname\{stopgrad\}\\left\(\\phi\_\{s,t\}^\{\\ell\}\\right\)\\right\),\(9\)whereϕs,tℓ\\phi\_\{s,t\}^\{\\ell\}denotes current\-state features available up to positiontt, including the backbone hidden state and statistics of the direct and generated memory residuals\. The scalarA^s,tℓ\\widehat\{A\}\_\{s,t\}^\{\\ell\}predicts how much pathwayssis expected to improve over E, whilecs,tℓc\_\{s,t\}^\{\\ell\}provides an additional confidence signal for admission\. Thestopgrad\\operatorname\{stopgrad\}operator keeps the routing loss from updating the backbone or the memory pathways\. Onlyqℓq^\{\\ell\}is trained\. Future tokens enter only the E\-relative advantage targets during training, and at inference time the router relies on current\-state features alone\. Appendix[C](https://arxiv.org/html/2609.25853#A3)specifies the feature setϕ\\phiand the routing objective\.

Algorithm 1MemoryAthenaInput:Direct residual

ee; generated residuals

gGEg\_\{\\mathrm\{GE\}\}and

gGHg\_\{\\mathrm\{GH\}\}; routing head

qq; thresholds

τ\\tauand

ρ\\rho; maximum scale

amaxa\_\{\\max\}; temperature

TαT\_\{\\alpha\}
Output:Memory residual

rr
//// Predict E\-relative advantage

\(A^s,cs\)s∈\{GE,GH\}←q⁡\(current\-state features\)\(\\widehat\{A\}\_\{s\},c\_\{s\}\)\_\{s\\in\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}\}\\leftarrow q\(\\text\{current\-state features\}\);

//// Admit only beneficial generated memories

𝒞←\{s∈\{GE,GH\}:A^s\>τ∧σ​\(cs\)≥ρ\}\\mathcal\{C\}\\leftarrow\\\{\\,s\\in\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}:\\widehat\{A\}\_\{s\}\>\\tau\\;\\wedge\\;\\sigma\(c\_\{s\}\)\\geq\\rho\\,\\\};

if*𝒞=?\\mathcal\{C\}=\\varnothing*then

returnee\[exact E fallback\];

//// Select the best generated candidate

s⋆←arg⁡maxs∈𝒞⁡A^ss^\{\\star\}\\leftarrow\\displaystyle\\arg\\max\_\{s\\in\\mathcal\{C\}\}\\widehat\{A\}\_\{s\};

//// Determine intervention strength

α←amax​clip⁡\(A^s⋆−τTα,0,1\)​σ​\(cs⋆\)\\alpha\\leftarrow a\_\{\\max\}\\operatorname\{clip\}\\left\(\\frac\{\\widehat\{A\}\_\{s^\{\\star\}\}\-\\tau\}\{T\_\{\\alpha\}\},0,1\\right\)\\sigma\(c\_\{s^\{\\star\}\}\);

//// Apply an E\-anchored correction

r←e\+α​\(gs⋆−e\)r\\leftarrow e\+\\alpha\\bigl\(g\_\{s^\{\\star\}\}\-e\\bigr\);

return

rr;

### 3\.3Stage3: E\-Anchored Memory Routing

At inference time, GE and GH are treated as candidate corrections to the direct E pathway rather than as symmetric experts\.

For each generated sources∈\{GE,GH\}s\\in\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}, we test whether its predicted advantage and confidence satisfy the admission criteria:

𝒞tℓ=\{s∈\{GE,GH\}\|A^s,tℓ\>τ,σ\(cs,tℓ\)≥ρ\},\\mathcal\{C\}\_\{t\}^\{\\ell\}=\\left\\\{s\\in\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}\\;\\middle\|\\;\\widehat\{A\}\_\{s,t\}^\{\\ell\}\>\\tau,\\quad\\sigma\\\!\\left\(c\_\{s,t\}^\{\\ell\}\\right\)\\geq\\rho\\right\\\},\(10\)whereτ\\tauandρ\\rhoare the advantage and confidence thresholds\. If at least one generated source is eligible, the router selects the candidate with the largest predicted advantage:s⋆=arg⁡maxs∈𝒞tℓ⁡A^s,tℓ\.s^\{\\star\}=\\arg\\max\_\{s\\in\\mathcal\{C\}\_\{t\}^\{\\ell\}\}\\widehat\{A\}\_\{s,t\}^\{\\ell\}\.The selected generated representation modifies the direct residual through bounded interpolation:

rtℓ=etℓ\+αtℓ​\(gs⋆,tℓ−etℓ\),0≤αtℓ≤1\.r\_\{t\}^\{\\ell\}=e\_\{t\}^\{\\ell\}\+\\alpha\_\{t\}^\{\\ell\}\\left\(g\_\{s^\{\\star\},t\}^\{\\ell\}\-e\_\{t\}^\{\\ell\}\\right\),\\qquad 0\\leq\\alpha\_\{t\}^\{\\ell\}\\leq 1\.\(11\)
The interpolation strengthαtℓ\\alpha\_\{t\}^\{\\ell\}increases with the predicted utility and confidence of the selected candidate\. Algorithm[1](https://arxiv.org/html/2609.25853#alg1)uses maximum scaleamax=1a\_\{\\max\}=1and temperatureTα=0\.15T\_\{\\alpha\}=0\.15\. If no generated source is admitted, we setαtℓ=0\\alpha\_\{t\}^\{\\ell\}=0and thenrtℓ=etℓr\_\{t\}^\{\\ell\}=e\_\{t\}^\{\\ell\}\.

The resulting residual is injected into the backbone ashtℓ←htℓ\+rtℓ\.h\_\{t\}^\{\\ell\}\\leftarrow h\_\{t\}^\{\\ell\}\+r\_\{t\}^\{\\ell\}\.E therefore holds a privileged role\. Generated memory modifies the direct\-memory contribution only when it is predicted to be useful, and rejection recovers the direct E pathway exactly at the corresponding injection site\.

## 4Experiments

We evaluate whether generated memory improves a direct reader, whether its pathways provide complementary answers, and whether E\-relative admission and the reader interface explain the gains\. The primary backbone is Mistral\-7B\-v0\.3\([7](https://arxiv.org/html/2609.25853#bib.bib8)\), with memory injected at layers 2 and 10 through a four\-branch reader following[Li et al\. \(13\)](https://arxiv.org/html/2609.25853#bib.bib2)\. Llama\-2\-7B\([20](https://arxiv.org/html/2609.25853#bib.bib21)\)supplies the imported source memory and is the target backbone in the cross\-backbone transfer row\. We compare frozen inference rules within a shared checkpoint on five QA benchmarks and six classification tasks\. Dataset definitions, scoring, sample counts, and training budgets appear in Appendices[A](https://arxiv.org/html/2609.25853#A1)and[D](https://arxiv.org/html/2609.25853#A4)\.

Table 1:QA comparison, routing\-policy analysis, and cross\-backbone transfer on Natural Questions \(NQ\), WebQuestions \(WebQA\), TriviaQA, TruthfulQA and HotpotQA\. Average is the unweighted mean of four F1 scores and one TruthfulQA multiple\-choice \(MC\) summary\.Table 2:Six\-task NLP accuracy \(%\) on SST2, MR, CR, RT, AG News \(AGN\) and Yahoo Answers\. Small signed values are percentage\-point differences from the Full\-choice dCPMI Base\.### 4\.1RQ1: When does generated memory improve a strong direct\-memory pathway?

Table[1](https://arxiv.org/html/2609.25853#S4.T1)reports the QA results together with literature baselines, individual pathways, alternative routing rules and cross\-backbone transfer, and Table[2](https://arxiv.org/html/2609.25853#S4.T2)reports the six general NLP tasks\.

Compared routing strategies\.*E only*,*GE only*, and*GH only*force one memory pathway throughout inference\.*Ordinary hard routing*treats the three pathways as symmetric candidates and selects a single pathway, while*ordinary soft fusion*combines their representations using learned routing weights\. The*subset hard*and*subset soft*variants restrict routing to a learned subset of sources\. The hard variant makes a discrete routing decision within that subset, whereas the soft variant fuses the subset with learned weights\.MemoryAthenainstead treats E as the reference pathway throughout\. GE or GH modifies E only when the predicted E\-relative advantage and confidence satisfy the admission criteria, and the selected representation is combined with E through bounded interpolation\. Appendix[C](https://arxiv.org/html/2609.25853#A3)defines the subset\-routing controls\.

QA performance\.The individual pathways show that generated memory is useful but not uniformly better than direct retrieval\. GE improves WebQA from 33\.35 to 35\.24 but is weaker than E on TriviaQA and HotpotQA, and GH is weaker than E on all five QA summary metrics\. This makes unconditional replacement of E undesirable\.

The alternative routing rules lead to the same conclusion\. Ordinary hard routing reaches an average of 35\.83, while ordinary soft fusion falls to 28\.62\. The stronger subset\-hard control reaches 37\.74, but remains belowMemoryAthenaat 39\.28\. Relative to the same\-checkpoint E pathway,MemoryAthenaimproves all five QA metrics, by 4\.74 points on NQ, 1\.25 on WebQA, 1\.34 on TriviaQA, 0\.49 on TruthfulQA, and 0\.30 on HotpotQA\. The average increases from 37\.65 to 39\.28\. These results indicate that the benefit comes from conditionally modifying a strong direct\-memory pathway rather than simply combining all available representations\.

After transferring the memory interface from Mistral to Llama, the resulting system reaches an average of 37\.87 and remains competitive with the same\-checkpoint Mistral E pathway\. In particular, WebQA increases to 36\.40\. Because the target backbone and adaptation history differ, this row shows that the memory interface remains usable after transfer rather than a matched gain over a bare Llama model\.

General NLP performance\.Across the six NLP tasks,MemoryAthenaimproves over E by 3\.90 points on SST2, 3\.70 on MR, 1\.70 on CR, 1\.50 on RT, and 3\.71 on AGN\. For these five tasks, we use the default admission thresholdτ=0\\tau=0\. For Yahoo, we use the more conservative settingτ=1\\tau=1, under which the router reaches 57\.43 compared with 57\.51 for E\. With these task\-specific inference settings, the six\-task average increases from 76\.73 to 79\.13\.

Takeaway\.The benefit of generated memory is heterogeneous across tasks and inputs: neither GE nor GH uniformly dominates direct Engram retrieval\. This is precisely the regime targeted byMemoryAthena, which retains E as a stable reference and selectively admits generated memories only when they are predicted to help\.

### 4\.2RQ2: What drives the gains from the memory interface?

We examine whether the gains arise from the memory interface itself, its initialization, or the learned routing policy\. We therefore compareMemoryAthenawith retrained architectural controls, a from\-scratch memory variant, and a random\-router control\.

Table 3:QA ablations \(%\): four open\-QA F1 scores and TruthfulQA MC mean\. Base is the reported Vanilla Mistral from[Li et al\. \(13\)](https://arxiv.org/html/2609.25853#bib.bib2)\. Average is the unweighted five\-task mean\.Architectural controls\.The full model outperforms the no\-gate and affine\-stitch variants on four of five QA metrics and the parameter\-matched FFN on all five\. Removing the gate reduces the five\-task average from 39\.28 to 33\.35, while replacing the interface with a parameter\-matched FFN reduces it further to 24\.92\. These results support the importance of the learned memory interface rather than parameter count alone\.

Memory initialization\.Training the memory system from scratch reaches an average of 40\.16, slightly above the pretrained\-memory configuration at 39\.28\. The pretrained memory initialization is therefore not required for the observed gains in this setup; the learned interface can recover strong performance when trained jointly from scratch\.

Learned routing\.The random\-router control provides a direct test of whether exposing the model to multiple memory pathways is sufficient without learning when to use them\. Its five\-task average is 31\.67, compared with 39\.28 forMemoryAthena, a decrease of 7\.61 points\. The degradation is especially large on TriviaQA, where F1 falls from 70\.68 to 49\.18, and is also substantial on NQ and HotpotQA\. Thus, the gain cannot be explained simply by making E, GE, and GH available: learning a selective routing policy is critical for exploiting their complementary behavior\.

Complete ablation metrics are reported in Appendix[G](https://arxiv.org/html/2609.25853#A7)\.

Takeaway\.The gains depend strongly on the learned memory interface and routing policy\. Random routing reduces the five\-task average from 39\.28 to 31\.67, showing that access to multiple memory pathways alone is insufficient\. At the same time, training from scratch slightly exceeds the pretrained\-memory configuration, suggesting that the main benefit comes from learning how to construct and select useful memory representations rather than from a specific pretrained memory initialization\.

### 4\.3RQ3: How doesMemoryAthenause complementary memory pathways?

\(a\)Downstream memory\-pathway contribution\.\(b\)Gold\-label oracle headroom\.
Figure 2:Left:effective downstream contribution of E, GE, and GH across tasks\.Right:comparison between deployedMemoryAthenaand gold\-label source oracles, showing the remaining routing headroom\.The previous results establish that generated memory can improve downstream performance, but not how the router combines the three pathways\. We therefore analyze the downstream routing behavior and, separately, use a label\-informed oracle to estimate how much complementary information remains unexploited\.

Downstream routing behavior\.When a generated pathways⋆s^\{\\star\}is admitted, the effective memory contribution can be written as

\(wE,wGE,wGH\)=\(1−α,α𝟏\[s⋆=GE\],α𝟏\[s⋆=GH\]\)\.\(w\_\{\\mathrm\{E\}\},w\_\{\\mathrm\{GE\}\},w\_\{\\mathrm\{GH\}\}\)=\\left\(1\-\\alpha,\\;\\alpha\\mathbf\{1\}\[s^\{\\star\}=\\mathrm\{GE\}\],\\;\\alpha\\mathbf\{1\}\[s^\{\\star\}=\\mathrm\{GH\}\]\\right\)\.\(12\)We aggregate these effective weights over downstream inference to characterize how much each pathway contributes to the injected memory representation\.

Figure[2\(a\)](https://arxiv.org/html/2609.25853#S4.F2.sf1)shows task\-dependent routing\. On the available QA traces, GH provides the majority of the memory contribution, 74\.15% on NQ, 76\.08% on WebQA, and 76\.77% on TruthfulQA, whereas GE contributes 3\.83–6\.23%\. A similar preference for GH appears on several classification tasks, including SST2 \(77\.99%\), MR \(75\.50%\), AGN \(89\.03%\), and Yahoo \(71\.86%\)\.

CR is the counterexample\. There GE receives 44\.60% of the contribution, against 34\.04% for GH and 21\.36% for E, and RT also retains a larger direct\-memory component \(28\.42%\)\. The mean interpolation strength ranges from0\.7160\.716to0\.8930\.893\. An admitted generated memory therefore typically makes a large correction to the direct representation\. These percentages are effective interpolation weights\. Because E retains the\(1−α\)\(1\-\\alpha\)component whenever GE or GH is admitted, a source’s contribution differs from the fraction of tokens routed exclusively to it\.

Oracle headroom\.The routing statistics describe what the deployed router does, whereas a post\-hoc source oracle estimates the gain that was attainable\. It observes the downstream label and selects the highest\-scoring endpoint among the available pathways\. The three\-source oracle improves over the best E\-containing two\-source oracle on every task, with gains of 2\.32 points on NQ, 3\.37 on WebQA, 1\.79 on TriviaQA, 0\.73 on TruthfulQA, and 2\.35 on HotpotQA \(Figure[2\(b\)](https://arxiv.org/html/2609.25853#S4.F2.sf2)\)\. This means that GE and GH provide successful predictions on examples that are not captured by a single generated pathway\. A gap remains between the label\-informed oracle and the deployed causal router\. The three\-source oracle exceedsMemoryAthenaby 5\.31 points on NQ, 9\.07 on WebQA, 5\.94 on TriviaQA, 3\.04 on TruthfulQA, and 7\.74 on HotpotQA\. Diverse memory representations are already available, and much of their potential therefore depends on identifying*when*a generated memory is useful and*which*generated pathway should intervene\.

Takeaway\.The utility of E, GE, and GH is strongly task\-dependent, confirming that no single memory pathway is uniformly preferable\. Gold\-label oracles exceed the deployed router by 3\.04–9\.07 points, showing that substantial complementarity among the pathways remains unexploited\. The key challenge is therefore to better identify when a generated memory should intervene and which pathway is most useful\.

## 5Conclusion

MemoryAthenastarts from the observation that generated memory is conditionally useful rather than uniformly superior to direct retrieval\. Instead of replacing E, the method treats GE and GH as candidate corrections and learns when and how strongly they should intervene\. This converts heterogeneous pathway quality into consistent gains on the QA summary metrics and improvements across the evaluated NLP setting\. The remaining oracle gap suggests that better utility estimation and admission, rather than universally stronger generated memories, is a central direction for future work\.

#### Reproducibility statement

The appendices specify every pathway, freezing boundary, objective, admission rule and control, together with sample counts and checkpoint selection, and the accompanying evidence ledger maps each table to its artifacts\. Available hashes support traceability but do not substitute for an immutable launch environment or contamination audit\. Missing uncertainty estimates or functional coding scores are not inferred\.

#### AI assistance

An AI assistant assisted drafting, organization, and consistency checks against experiment records\. The claims, experimental validity, and final submission remain the authors’ responsibility\.

## References

- \[1\]J\. Berant, A\. Chou, R\. Frostig, and P\. Liang\(2013\)Semantic parsing on Freebase from question\-answer pairs\.InProceedings of the 2013 Conference on Empirical Methods in Natural Language Processing,D\. Yarowsky, T\. Baldwin, A\. Korhonen, K\. Livescu, and S\. Bethard \(Eds\.\),Seattle, Washington, USA,pp\. 1533–1544\.External Links:[Link](https://aclanthology.org/D13-1160/)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p2.1)\.
- \[2\]X\. Cheng, R\. Tian, W\. Zeng, D\. Dai, Q\. Chen, B\. Wang, Z\. Xie, K\. Huang, X\. Yu, C\. Deng, S\. Zhou, C\. Zhao, Z\. Hao, Y\. Li, H\. Zhang, Z\. Zhang, Y\. Wei, M\. Y\. Xu, H\. Zhang, D\. Zhao, and W\. Liang\(2026\)Conditional memory via scalable lookup: a new axis of sparsity for large language models\.External Links:2601\.07372,[Link](https://arxiv.org/abs/2601.07372)Cited by:[§1](https://arxiv.org/html/2609.25853#S1.p1.1),[§2](https://arxiv.org/html/2609.25853#S2.p1.1)\.
- \[3\]W\. Fedus, B\. Zoph, and N\. Shazeer\(2022\)Switch transformers: scaling to trillion parameter models with simple and efficient sparsity\.Journal of Machine Learning Research23\(120\),pp\. 1–39\.External Links:[Link](http://jmlr.org/papers/v23/21-0998.html)Cited by:[§2](https://arxiv.org/html/2609.25853#S2.p6.1)\.
- \[4\]F\. Hamborg, N\. Meuschke, C\. Breitinger, and B\. Gipp\(2017\)News\-please: a generic news crawler and extractor\.InProceedings of the 15th International Symposium of Information Science,pp\. 218–223\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.4120316)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p1.1)\.
- \[5\]A\. Holtzman, P\. West, V\. Shwartz, Y\. Choi, and L\. Zettlemoyer\(2021\)Surface form competition: why the highest probability answer isn’t always right\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 7038–7051\.External Links:[Link](https://aclanthology.org/2021.emnlp-main.564/),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.564)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p3.1)\.
- \[6\]M\. Hu and B\. Liu\(2004\)Mining and summarizing customer reviews\.InProceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining,KDD04,pp\. 168–177\.External Links:[Link](https://doi.org/10.1145/1014052.1014073),[Document](https://dx.doi.org/10.1145/1014052.1014073)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p3.1)\.
- \[7\]A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. Sayed\(2023\)Mistral 7B\.External Links:2310\.06825,[Link](https://arxiv.org/abs/2310.06825)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p1.1),[§4](https://arxiv.org/html/2609.25853#S4.p1.1)\.
- \[8\]M\. Joshi, E\. Choi, D\. Weld, and L\. Zettlemoyer\(2017\)TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),R\. Barzilay and M\. Kan \(Eds\.\),Vancouver, Canada,pp\. 1601–1611\.External Links:[Link](https://aclanthology.org/P17-1147/),[Document](https://dx.doi.org/10.18653/v1/P17-1147)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p2.1)\.
- \[9\]U\. Khandelwal, O\. Levy, D\. Jurafsky, L\. Zettlemoyer, and M\. Lewis\(2020\)Generalization through memorization: nearest neighbor language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=HklBjCEKvH)Cited by:[§1](https://arxiv.org/html/2609.25853#S1.p1.1),[§2](https://arxiv.org/html/2609.25853#S2.p1.1)\.
- \[10\]T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. Petrov\(2019\)Natural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 452–466\.External Links:[Link](https://aclanthology.org/Q19-1026/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00276)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p2.1)\.
- \[11\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela\(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 9459–9474\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2609.25853#S1.p1.1),[§2](https://arxiv.org/html/2609.25853#S2.p1.1)\.
- \[12\]J\. Li, X\. Cheng, X\. Zhao, J\. Nie, and J\. Wen\(2023\)HaluEval: a large\-scale hallucination evaluation benchmark for large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 6449–6464\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.397/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.397)Cited by:[Appendix J](https://arxiv.org/html/2609.25853#A10.SS0.SSS0.Px1.p3.1)\.
- \[13\]M\. Li, G\. Yu, X\. Wang, and S\. Ji\(2026\)Cross\-model memory transfer via target\-side reader adaptation\.External Links:2608\.17050,[Link](https://arxiv.org/abs/2608.17050)Cited by:[1st item](https://arxiv.org/html/2609.25853#S1.I1.i1.p1.1),[§1](https://arxiv.org/html/2609.25853#S1.p1.1),[§1](https://arxiv.org/html/2609.25853#S1.p3.1),[§2](https://arxiv.org/html/2609.25853#S2.p2.1),[Table 3](https://arxiv.org/html/2609.25853#S4.T3),[Table 3](https://arxiv.org/html/2609.25853#S4.T3.4),[§4](https://arxiv.org/html/2609.25853#S4.p1.1)\.
- \[14\]S\. Lin, J\. Hilton, and O\. Evans\(2022\)TruthfulQA: measuring how models mimic human falsehoods\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 3214–3252\.External Links:[Link](https://aclanthology.org/2022.acl-long.229/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.229)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p2.1)\.
- \[15\]A\. L\. Maas, R\. E\. Daly, P\. T\. Pham, D\. Huang, A\. Y\. Ng, and C\. Potts\(2011\)Learning word vectors for sentiment analysis\.InProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies,D\. Lin, Y\. Matsumoto, and R\. Mihalcea \(Eds\.\),Portland, Oregon, USA,pp\. 142–150\.External Links:[Link](https://aclanthology.org/P11-1015/)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p1.1)\.
- \[16\]S\. Merity, C\. Xiong, J\. Bradbury, and R\. Socher\(2017\)Pointer sentinel mixture models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Byj72udxe)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p1.1),[§2](https://arxiv.org/html/2609.25853#S2.p6.1)\.
- \[17\]B\. Pang and L\. Lee\(2005\)Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales\.InProceedings of the 43rd Annual Meeting of the Association for Computational Linguistics \(ACL’05\),K\. Knight, H\. T\. Ng, and K\. Oflazer \(Eds\.\),Ann Arbor, Michigan,pp\. 115–124\.External Links:[Link](https://aclanthology.org/P05-1015/),[Document](https://dx.doi.org/10.3115/1219840.1219855)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p3.1)\.
- \[18\]N\. Shazeer, A\. Mirhoseini, K\. Maziarz, A\. Davis, Q\. Le, G\. Hinton, and J\. Dean\(2017\)Outrageously large neural networks: the sparsely\-gated mixture\-of\-experts layer\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=B1ckMDqlg)Cited by:[§1](https://arxiv.org/html/2609.25853#S1.p4.1),[§2](https://arxiv.org/html/2609.25853#S2.p6.1)\.
- \[19\]R\. Socher, A\. Perelygin, J\. Wu, J\. Chuang, C\. D\. Manning, A\. Ng, and C\. Potts\(2013\)Recursive deep models for semantic compositionality over a sentiment treebank\.InProceedings of the 2013 Conference on Empirical Methods in Natural Language Processing,D\. Yarowsky, T\. Baldwin, A\. Korhonen, K\. Livescu, and S\. Bethard \(Eds\.\),Seattle, Washington, USA,pp\. 1631–1642\.External Links:[Link](https://aclanthology.org/D13-1170/)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p3.1)\.
- \[20\]H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. C\. Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. Scialom\(2023\)Llama 2: open foundation and fine\-tuned chat models\.External Links:2307\.09288,[Link](https://arxiv.org/abs/2307.09288)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p1.1),[§4](https://arxiv.org/html/2609.25853#S4.p1.1)\.
- \[21\]R\. Wei, J\. Cao, J\. Wang, J\. Kai, Q\. Guo, B\. Zhou, and Z\. Lin\(2025\)MLP memory: a retriever\-pretrained memory for large language models\.External Links:2508\.01832,[Link](https://arxiv.org/abs/2508.01832)Cited by:[§1](https://arxiv.org/html/2609.25853#S1.p1.1),[§2](https://arxiv.org/html/2609.25853#S2.p1.1)\.
- \[22\]Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning\(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,E\. Riloff, D\. Chiang, J\. Hockenmaier, and J\. Tsujii \(Eds\.\),Brussels, Belgium,pp\. 2369–2380\.External Links:[Link](https://aclanthology.org/D18-1259/),[Document](https://dx.doi.org/10.18653/v1/D18-1259)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p2.1)\.
- \[23\]X\. Zhang, J\. Zhao, and Y\. LeCun\(2015\)Character\-level convolutional networks for text classification\.InAdvances in Neural Information Processing Systems,C\. Cortes, N\. Lawrence, D\. Lee, M\. Sugiyama, and R\. Garnett \(Eds\.\),Vol\.28\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2015/file/250cf8b51c773f3f8dc8b4be867a9a02-Paper.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.25853#A1.p1.1),[Appendix A](https://arxiv.org/html/2609.25853#A1.p3.1)\.

## Appendix AEvaluation datasets and protocol

The primary backbone is Mistral\-7B\-v0\.3\([7](https://arxiv.org/html/2609.25853#bib.bib8)\)\. The QA router uses Wikipedia\-2021 text and imports a learned Llama\-2 source memory\([20](https://arxiv.org/html/2609.25853#bib.bib21)\)\. General NLP uses an equal\-token mixture of WikiText\-103\([16](https://arxiv.org/html/2609.25853#bib.bib20)\), Amazon Polarity review text\([23](https://arxiv.org/html/2609.25853#bib.bib19)\), CC\-News\([4](https://arxiv.org/html/2609.25853#bib.bib23)\), and IMDB text\([15](https://arxiv.org/html/2609.25853#bib.bib18)\)\. All parameters are frozen downstream\.

QA covers Natural Questions \(NQ\)\([10](https://arxiv.org/html/2609.25853#bib.bib13)\), WebQuestions \(WebQA\)\([1](https://arxiv.org/html/2609.25853#bib.bib14)\), TriviaQA\([8](https://arxiv.org/html/2609.25853#bib.bib12)\), TruthfulQA\([14](https://arxiv.org/html/2609.25853#bib.bib11)\), and HotpotQA\([22](https://arxiv.org/html/2609.25853#bib.bib10)\)\. We report exact match \(EM\) and token F1 for open QA, and the arithmetic mean of three recorded multiple\-choice metrics for TruthfulQA\. Sample counts are 3,609, 2,032, 17,944, 817, and 7,405\. Exact metrics are retained in Appendix[E](https://arxiv.org/html/2609.25853#A5)\.

For general NLP, SST2\([19](https://arxiv.org/html/2609.25853#bib.bib15)\), MR and RT \(movie\-review sentiment benchmarks, see[17](https://arxiv.org/html/2609.25853#bib.bib16)\), and CR\([6](https://arxiv.org/html/2609.25853#bib.bib17)\)test sentiment, while AG News \(AGN\) and Yahoo Answers test topic classification\([23](https://arxiv.org/html/2609.25853#bib.bib19)\)\. SST2 uses its development split, MR and CR use the provided test files, and RT uses the Rotten Tomatoes test split\. The exact release identifiers and preprocessing lineage of these MR/RT files remain to be verified, and the citation identifies the benchmark family\. Our next\-token scoring adapts domain\-conditional pointwise mutual information\([5](https://arxiv.org/html/2609.25853#bib.bib7)\)\. For label\-token setVyV\_\{y\},

S⁡\(y,x\)=∑v∈Vy\[log⁡p⁡\(v∣C⁡\(x\)\)−log⁡p⁡\(v∣Cdomain\)\],y^=arg⁡maxy⁡S⁡\(y,x\)\.S\(y;x\)=\\sumop\\displaylimits\_\{v\\in V\_\{y\}\}\[\\log p\(v\\mid C\(x\)\)\-\\log p\(v\\mid C\_\{\\mathrm\{domain\}\}\)\],\\qquad\\widehat\{y\}=\\arg\\max\_\{y\}S\(y;x\)\.\(13\)Verbalizers map to their first valid token and are deduplicated following the implementation\. This sums log\-score differences token by token rather than forming a log\-sum\-exp or a full\-label likelihood\. The six tasks contain 872, 2,000, 2,000, 1,066, 7,600, and 60,000 examples\. We report accuracy and an unweighted six\-task mean\.

Our primary comparison keeps the memory and expert checkpoints fixed and changes only the inference rule, providing the cleanest assessment of routing\. Retrained ablations are reported separately because they also change the learned components\. The routing models are trained on general text without downstream labels\.

## Appendix BLimitations and Future Work

AlthoughMemoryAthenalearns when to admit GE or GH from unlabeled text, the overall routing system still relies on memory pathways, routing features, training objectives, and admission hyperparameters\. It therefore does not yet provide a fully automatic mechanism for discovering which memory representation should be constructed and used for a given input\. The substantial gap between the deployed router and the gold\-label source oracle further shows that the available memory pathways contain useful complementary information that the current router does not fully exploit\. Developing more automated routing objectives that can jointly discover useful memory candidates, calibrate their utility, and approach oracle\-level selection without downstream labels is an important direction for future work\.

## Appendix CArchitecture and routing implementation

### C\.1Direct and generated pathways

E directly reads retrieved memory\. GE generates a small latent representation from a causal window of Engram cues\. GH generates from clean causal backbone states, obtained without memory injection:

etℓ\\displaystyle e\_\{t\}^\{\\ell\}=REℓ​\(htℓ,mt\),\\displaystyle=R\_\{E\}^\{\\ell\}\(h\_\{t\}^\{\\ell\},m\_\{t\}\),zE,tℓ\\displaystyle z\_\{E,t\}^\{\\ell\}=Gℓ\(Mt−w\+1:t;E\),\\displaystyle=G^\{\\ell\}\(M\_\{t\-w\+1:t\};\\mathrm\{E\}\),zH,tℓ\\displaystyle z\_\{H,t\}^\{\\ell\}=Gℓ\(Ht−w\+1:t0,ℓ;H\),\\displaystyle=G^\{\\ell\}\(H^\{0,\\ell\}\_\{t\-w\+1:t\};H\),\(14\)gGE,tℓ\\displaystyle g\_\{\\mathrm\{GE\},t\}^\{\\ell\}=RG​Eℓ​\(htℓ,zE,tℓ\),\\displaystyle=R\_\{GE\}^\{\\ell\}\(h\_\{t\}^\{\\ell\},z\_\{E,t\}^\{\\ell\}\),gGH,tℓ\\displaystyle g\_\{\\mathrm\{GH\},t\}^\{\\ell\}=RG​Hℓ​\(htℓ,zH,tℓ\)\.\\displaystyle=R\_\{GH\}^\{\\ell\}\(h\_\{t\}^\{\\ell\},z\_\{H,t\}^\{\\ell\}\)\.Reader notation subsumes projections, branch aggregation, and output gates\. Generated paths share components and use source embeddings and low\-rank adaptations\. GE is grounded in retrieved cues, whereas GH can propose a representation when the table is unhelpful\. This creates potential complementarity, but also makes strong performance with corrupted memory possible\.

The evaluated Mistral configuration injects memory at layers 2 and 10\. Memory dimension is 512 and the reader has four branches\. Each generator produces four latents from a three\-position causal window, with width 256, two layers, and four attention heads\. Source\-adaptation rank is 16\. GH requires a clean backbone computation in the current implementation, and freezing its parameters does not remove this inference cost\.

### C\.2Memory learning and reader adaptation

Training comprises three optimization stages followed by frozen evaluation\. Stage 1 learns the table and source adaptor with the source backbone fixed:

ℒmem\(ϕ,ψsrc\)=−∑tlogpθsrc,ϕ,ψsrc\(xt\+1∣x≤t\)\.\\mathcal\{L\}\_\{\\mathrm\{mem\}\}\(\\phi,\\psi\_\{\\mathrm\{src\}\}\)=\-\\sumop\\displaylimits\_\{t\}\\log p\_\{\\theta\_\{\\mathrm\{src\}\},\\phi,\\psi\_\{\\mathrm\{src\}\}\}\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\)\.\(15\)The table and canonicalization configuration can be reused, but the source adaptor need not be compatible with another backbone\.

Stage 2 freezes the table and target backbone and trains the generators and target\-side readers\. In the later general\-NLP and coding pipeline, each endpoint receives equal\-weight causal language\-modeling supervision:

ℒexperts=−13∑s∈\{E,GE,GH\}∑tlogps\(xt\+1∣x≤t\)\.\\mathcal\{L\}\_\{\\mathrm\{experts\}\}=\-\\frac\{1\}\{3\}\\sumop\\displaylimits\_\{s\\in\\\{\\mathrm\{E\},\\mathrm\{GE\},\\mathrm\{GH\}\\\}\}\\sumop\\displaylimits\_\{t\}\\log p\_\{s\}\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\)\.\(16\)This makes the endpoints usable before their relative advantages are distilled\. It does not optimize downstream labels\.

The QA checkpoint also contains auxiliary routing modules used to construct the baseline inference rules in Table[1](https://arxiv.org/html/2609.25853#S4.T1)\.*Ordinary hard routing*treats E, GE, and GH as three symmetric candidates and selects the single pathway with the highest routing score\.*Ordinary soft fusion*instead applies learned softmax weights over all three pathways and combines their residuals continuously\. The*subset*variants restrict routing to learned subsets of the available pathways\.*Subset hard routing*makes a discrete choice within the selected subset, whereas*subset soft fusion*uses the learned mixture weights within that subset\. These auxiliary routers are trained as part of the QA reader checkpoint and are used only as comparison baselines\. In contrast, the finalMemoryAthenarouter operates over E, GE, and GH using E\-relative advantage prediction: E is kept as the anchor, and GE or GH modifies it only when the generated pathway is predicted to be beneficial\.

### C\.3Counterfactual future\-advantage supervision

Stage 3 freezes the backbone, memory, generators, and readers\. Under teacher forcing, the advantage of sourcessover E is

as,t=log⁡ps​\(xt\+1∣x≤t\)−log⁡pE​\(xt\+1∣x≤t\),aE,t=0\.a\_\{s,t\}=\\log p\_\{s\}\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\)\-\\log p\_\{E\}\(x\_\{t\+1\}\\mid x\_\{\\leq t\}\),\\qquad a\_\{E,t\}=0\.\(17\)A positive advantage corresponds to a lower next\-token loss than E\. Each endpoint pass forces the same source at all injection sites\. The advantage is therefore a full\-path counterfactual and cannot be attributed to one layer\.

We average over future horizonsℋ=\{1,4,8,16,32\}\\mathcal\{H\}=\\\{1,4,8,16,32\\\}\. Letvjv\_\{j\}indicate a valid target andℋt\\mathcal\{H\}\_\{t\}contain horizons with at least one valid target:

As,t=1\|ℋt\|​∑k∈ℋt∑j=0k−1vt\+j​as,t\+j∑j=0k−1vt\+j\.A\_\{s,t\}=\\frac\{1\}\{\|\\mathcal\{H\}\_\{t\}\|\}\\sumop\\displaylimits\_\{k\\in\\mathcal\{H\}\_\{t\}\}\\frac\{\\sumop\\displaylimits\_\{j=0\}^\{k\-1\}v\_\{t\+j\}a\_\{s,t\+j\}\}\{\\sumop\\displaylimits\_\{j=0\}^\{k\-1\}v\_\{t\+j\}\}\.\(18\)Out\-of\-range targets are excluded\. Future text constructs offline supervision only, so the deployed head sees neither future tokens nor these targets\.

At each injection layer, the routing head predictsA^s,tℓ\\widehat\{A\}\_\{s,t\}^\{\\ell\}and confidence logitcs,tℓc\_\{s,t\}^\{\\ell\}from detached causal features\. Features include the hidden state, reader\-gate strength, residual magnitudes, pairwise similarities, active\-source indicators, and projected semantic features\. E’s predicted advantage is fixed at zero\. The head preserves the E stream while exposing generated alternatives as detached features\.

Fors∈\{GE,GH\}s\\in\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}, defineus,t=clip⁡\(\|As,t\|,0\.01,2\)u\_\{s,t\}=\\operatorname\{clip\}\(\|A\_\{s,t\}\|,0\.01,2\)\. The per\-position loss is

Ls,tℓ\\displaystyle L\_\{s,t\}^\{\\ell\}=SmoothL1⁡\(A^s,tℓ,clip⁡\(As,t,−2,2\)\)\\displaystyle=\\operatorname\{SmoothL1\}\\left\(\\widehat\{A\}\_\{s,t\}^\{\\ell\},\\operatorname\{clip\}\(A\_\{s,t\},\-2,2\)\\right\)\+0\.25​BCEWithLogits⁡\(cs,tℓ,σ​\(As,t/0\.15\)\)\.\\displaystyle\\quad\+0\.25\\,\\operatorname\{BCEWithLogits\}\\left\(c\_\{s,t\}^\{\\ell\},\\sigma\(A\_\{s,t\}/0\.15\)\\right\)\.\(19\)We take a valid\-position,us,tu\_\{s,t\}\-weighted average and then average injection\-layer heads\. Only these heads are optimized, amounting to 534,924 trainable parameters in the audited Mistral runs\. Confidence is trained against an advantage\-derived soft target, and it is not established to be a calibrated probability of downstream correctness\.

### C\.4E\-Anchored Causal Inference

At each position and injection layer, a generated pathways∈\{GE,GH\}s\\in\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}is considered only if its predicted E\-relative advantage exceedsτ\\tauand its confidence exceedsρ\\rho\. If both generated pathways are eligible, we select the one with the larger predicted advantage\. For the selected pathways⋆s^\{\\star\}, the intervention strength is

αtℓ=amax​clip⁡\(A^s⋆,tℓ−τTα,0,1\)​σ​\(cs⋆,tℓ\),\\alpha\_\{t\}^\{\\ell\}=a\_\{\\max\}\\operatorname\{clip\}\\left\(\\frac\{\\widehat\{A\}\_\{s^\{\\star\},t\}^\{\\ell\}\-\\tau\}\{T\_\{\\alpha\}\},0,1\\right\)\\sigma\(c\_\{s^\{\\star\},t\}^\{\\ell\}\),\(20\)and the memory residual is

rtℓ=\(1−αtℓ\)​etℓ\+αtℓ​gs⋆,tℓ\.r\_\{t\}^\{\\ell\}=\(1\-\\alpha\_\{t\}^\{\\ell\}\)e\_\{t\}^\{\\ell\}\+\\alpha\_\{t\}^\{\\ell\}g\_\{s^\{\\star\},t\}^\{\\ell\}\.\(21\)
If no generated pathway is admitted,αtℓ=0\\alpha\_\{t\}^\{\\ell\}=0and the model exactly recovers the E residual at that injection site\. We useτ=0\\tau=0,ρ=0\.5\\rho=0\.5,Tα=0\.15T\_\{\\alpha\}=0\.15, andamax=1a\_\{\\max\}=1by default\.

The fallback guarantees recovery of the same\-checkpoint E pathway when all generated candidates are rejected, but it does not guarantee that every admitted intervention improves downstream performance because the router can make incorrect predictions\. For analysis, the corresponding effective pathway weights are

\(wE,wGE,wGH\)=\(1−α,α𝟏\[s⋆=GE\],α𝟏\[s⋆=GH\]\)\.\(w\_\{\\mathrm\{E\}\},w\_\{\\mathrm\{GE\}\},w\_\{\\mathrm\{GH\}\}\)=\\left\(1\-\\alpha,\\,\\alpha\\mathbf\{1\}\[s^\{\\star\}=\\mathrm\{GE\}\],\\,\\alpha\\mathbf\{1\}\[s^\{\\star\}=\\mathrm\{GH\}\]\\right\)\.\(22\)These weights measure contribution to the injected residual and should not be confused with discrete source\-selection frequencies\.

## Appendix DTraining Configuration and Model Architecture

### D\.1Training Configuration

We use a common budget of 20M processed input positions for each optimization stage\. The three stages separately learn the memory, adapt the generated\-memory interfaces, and train the final routing head\. During downstream evaluation, all model parameters are frozen\.

Table 4:Training stages and token budgets\. Each optimization stage uses at most 20M processed input positions\.For the QA experiments, the learned source memory is reused rather than retrained during the final routing run\. Thus, the 20M budget in Table[4](https://arxiv.org/html/2609.25853#A4.T4)describes the budget of each optimization stage in the full pipeline, not an additional 20M\-token memory\-training phase for every downstream experiment\.

Table 5:Router training and inference configuration for the evaluated Mistral\-7B\-v0\.3 models\.The final checkpoint does not have to coincide with the end of training\. Router checkpoints are selected using held\-out causal\-text validation rather than downstream task accuracy\. The QA router selects the checkpoint at 8\.192M processed positions, while the general\-NLP router selects the checkpoint at 16\.384M positions\. The separate Yahoo threshold adjustment is reported explicitly because it uses downstream test performance for model selection\.

### D\.2Architecture Specification

Table[6](https://arxiv.org/html/2609.25853#A4.T6)summarizes the memory interface, generated\-memory modules, and routing head used with Mistral\-7B\-v0\.3\.

Table 6:Architecture and parameterization of the generated\-memory pathways and E\-anchored router used with Mistral\-7B\-v0\.3\.ComponentConfigurationValueMemory interfaceInjection layersTarget backbone layers\{2,10\}\\\{2,10\\\}Memory dimensionRetrieved / latent memory width512Memory tableFrozen Engram parameters33,554,432Reader branchesParallel branches per injection layer4Direct E readerParameters per layer / two layers10,506,244 / 21,012,488Generated\-memory pathways \(GE/GH\)Generator contextCausal input window3 positionsGenerated latentsLatent representations per position4Generator hidden widthInternal representation size256Generator depthTransformer layers2Generator attentionAttention heads4Source\-specific adaptationLow\-rank output adapterr=16r=16Output\-adapter shapeBottleneck projection256→16→dmodel256\\rightarrow 16\\rightarrow d\_\{\\mathrm\{model\}\}GE conditioningRetrieved Engram cues–GH conditioningClean causal backbone states–Generated readerProjection, branch aggregation, and output gating–E\-anchored routerRouter architectureMLP hidden width64Candidate pathwaysGenerated candidates\{GE,GH\}\\\{\\mathrm\{GE\},\\mathrm\{GH\}\\\}Reference pathwayFixed routing anchorERouter outputsE\-relative advantage and confidence2 per candidateAdvantage thresholdτ\\tau0\.0Confidence thresholdρ\\rho0\.5Maximum interpolationamaxa\_\{\\max\}1\.0Interpolation temperatureTαT\_\{\\alpha\}0\.15Parameter summaryShared generatorParameters per layer / two layers3,288,576 / 6,577,152Generated readersParameters per layer / two layers69,625,352 / 139,250,704Advantage routerParameters per layer / two layers267,462 / 534,924Full tri\-path adaptorAll adaptor and routing parameters per layer83,960,284Two\-layer adaptorLayers 2 and 10, excluding backbone167,920,568Total memory systemMemory table \+ two\-layer adaptor, excluding backbone201,475,000##### Reader and generator training\.

Stage 2 freezes the memory table and target backbone and optimizes the generated\-memory modules and target\-side readers\. For general NLP and coding, the three endpoints E, GE, and GH are trained with the equal\-weight language\-modeling objective in Eq\.[16](https://arxiv.org/html/2609.25853#A3.E16)\. This stage contains 167,364,632 trainable parameters\.

The QA reader checkpoint uses the same three endpoints but additionally trains auxiliary pair\-restricted and subset\-fusion modules used by the hard\- and soft\-routing baselines\. These auxiliary modules are not additional memory sources in the final system: the deployedMemoryAthenarouter operates only over E, GE, and GH\. The QA reader/generator stage contains 167,385,644 trainable parameters and uses routing distillation with coefficient 0\.5\.

##### Training time\.

The recorded reader/generator training times are approximately 31\.70 hours for QA, 16\.01 hours for general NLP, and 16\.41 hours for coding\. Router training takes approximately 11\.19, 10\.58, and 10\.94 hours, respectively\. These values are wall\-clock measurements from the corresponding runs\. Because router training evaluates multiple counterfactual endpoints, and GH additionally uses a clean\-backbone forward pass, processed\-token count alone does not represent the total computational cost\.

##### General\-NLP training data\.

The general\-NLP training corpus uses an equal\-token mixture of WikiText\-103, Amazon Polarity, CC\-News, and IMDB\. No downstream labels are used for training the memory pathways or the router\. We nevertheless do not assume that this guarantees benchmark decontamination: review corpora may overlap with downstream review benchmarks, and the CC\-News training and validation streams are sampled from the same underlying split rather than from explicitly document\-disjoint partitions\.

## Appendix EComplete QA Metrics

Figure[3](https://arxiv.org/html/2609.25853#A5.F3)summarizes the complete QA comparison using F1 for open\-QA tasks and the mean multiple\-choice score for TruthfulQA\. The same\-checkpoint rows provide the cleanest comparison because they share the same memory and expert checkpoints\. Standalone Engram and Mistral→\\rightarrowLlama serve as additional references but have different training histories\.

Figure 3:Complete QA comparison\. Open\-QA tasks report F1, and TruthfulQA reports the mean of MC1, MC2, and MC3\. Same\-checkpoint rows isolate the effect of the inference rule more cleanly than Standalone Engram or Mistral→\\rightarrowLlama\.The E\+GE and E\+GH pair controls are learned two\-source fusion baselines\. Their mixing weights depend on the input, so they should be interpreted as complete learned\-fusion systems rather than as isolated measurements of the contribution of GE or GH alone\.

To make the answer\-level behavior explicit, Figure[4](https://arxiv.org/html/2609.25853#A5.F4)compares EM and F1 between same\-checkpoint E andMemoryAthena\. NQ and WebQA improve on both metrics, but TriviaQA and HotpotQA show a different pattern: F1 increases while EM decreases\. Thus, generated\-memory routing can improve average answer overlap without necessarily increasing exact\-match accuracy\.

Figure 4:EM/F1 comparison between same\-checkpoint E andMemoryAthenaon the open\-QA tasks\. Numbers above bars show the change from E toMemoryAthena\. TriviaQA and HotpotQA exhibit higher F1 but lower EM\.For TruthfulQA, Table[7](https://arxiv.org/html/2609.25853#A5.T7)reports the three multiple\-choice metrics separately\.

Table 7:TruthfulQA multiple\-choice metrics \(%\)\. The summary is the mean of MC1, MC2, and MC3\.Finally, Figure[5](https://arxiv.org/html/2609.25853#A5.F5)reports paired F1 changes relative to same\-checkpoint E\. Most examples remain unchanged, while the numbers of improved and degraded examples are of similar order\. Positive mean F1 gains therefore arise from the magnitudes of the changes rather than from uniformly one\-sided answer flips\.

Figure 5:Paired F1 changes relative to same\-checkpoint E\. Bars show the fraction of evaluation examples whose F1 improves, degrades, or remains unchanged underMemoryAthena\. The right margin reports the mean F1 change\. These counts are descriptive only and are not significance tests\.Paired improve/degrade counts alone do not determine the mean score change, because the sizes of the answer\-level changes can differ substantially\. We therefore report these counts as descriptive diagnostics only\. TruthfulQA is excluded from this paired F1 analysis because it does not use token\-overlap F1\.

## Appendix FSource\-Oracle Analysis

Figure[2\(b\)](https://arxiv.org/html/2609.25853#S4.F2.sf2)evaluates the complementarity of the three memory pathways using a gold\-label oracle\. For each example, we run the E, GE, and GH endpoints separately and select the endpoint that obtains the highest score against the gold answer\. This oracle is used only for analysis and is not available at inference time\.

Adding more candidate endpoints cannot reduce oracle performance, since the oracle can always retain the best previously available endpoint\. Therefore, the improvement from the two\-source to the three\-source oracle indicates that the additional pathway is useful on some examples, but does not imply that a deployable router can always identify those examples\.

Table 8:Number and percentage of examples for which each endpoint is selected by the gold\-label oracle\. Ties are resolved in the order E, GE, GH\.E is the most frequent oracle winner, although its count is increased by our tie\-breaking rule: examples on which multiple endpoints receive the same score are assigned to E first\. Importantly, these counts describe which complete endpoint performs best on each example; they are not routing frequencies ofMemoryAthena\. The deployed router mixes pathways locally across tokens and layers, so its average pathway weights measure a different quantity\. This does not contradict the GH\-dominated average routing weights reported elsewhere: the two statistics measure different quantities\. Oracle counts identify the best complete endpoint per example, whereas routing weights measure the local contribution of each pathway within the deployed mixed trajectory\.

## Appendix GAblations and Architectural Controls

Figure[6](https://arxiv.org/html/2609.25853#A7.F6)summarizes the main architectural ablations relative toMemoryAthena\. For open\-QA tasks, we report F1; for TruthfulQA, we report the mean of the three multiple\-choice metrics\. Positive values indicate an improvement over the full system, and negative values indicate a degradation\.

![Refer to caption](https://arxiv.org/html/2609.25853v1/memoryathena_ablation_delta_heatmap.png)Figure 6:Performance changes relative to the fullMemoryAthenasystem\. Open\-QA tasks use F1 and TruthfulQA uses the mean multiple\-choice score\. The final column reports the change in the five\-task average\.Removing the output gate causes the largest degradation among the memory\-interface ablations, reducing the five\-task average by 5\.92 points\. Replacing the learned interface with a parameter\-matched FFN is substantially weaker, with an average drop of 14\.36 points\. The affine\-stitch variant also underperforms the full system by 1\.46 points on average\.

Permuting the memory keys has a comparatively small effect on the aggregate score, decreasing the five\-task average by 0\.70 points\. In contrast, training the memory system from scratch reaches a slightly higher average than the pretrained\-memory configuration \(\+0\.89 points\), with improvements on NQ, TriviaQA, TruthfulQA, and HotpotQA but a small decrease on WebQA\.

These results suggest that the learned memory interface and its gating mechanism are important for performance, whereas the advantage of the pretrained memory initialization is less consistent across the evaluated QA tasks\.

## Appendix HThreshold Sensitivity

We study the sensitivity of the router to the advantage thresholdτ\\tauon Yahoo Answers by varyingτ\\tauwhile keeping the remaining inference configuration fixed\. As shown in Table[9](https://arxiv.org/html/2609.25853#A8.T9), performance improves steadily as the threshold becomes more conservative, with the largest changes occurring betweenτ=0\.3\\tau=0\.3andτ=0\.9\\tau=0\.9\. Performance then largely stabilizes aroundτ=0\.9\\tau=0\.9–1\.01\.0, indicating that routing behavior is sensitive to the admission threshold but becomes relatively stable in the high\-threshold regime\.

Table 9:Sensitivity of Yahoo Answers accuracy to the routing advantage thresholdτ\\tau\(N=60,000N=60\{,\}000\)\.These results show that the admission threshold can have a substantial effect on downstream classification performance\. A largerτ\\taumakes the router more selective about when generated memory is allowed to intervene, suggesting that conservative routing is particularly important on Yahoo Answers\.

As an additional classification result, the router reaches 72\.56% accuracy on RTE, compared with 71\.48% for the E pathway, over 277 evaluation examples\.

## Appendix IInterpreting Routing Statistics

Table 10:Average interpolation mass on the validation corpora \(%\)\. These values measure the contribution of each pathway to the injected residual and should not be interpreted as discrete source\-selection frequencies\.Table[10](https://arxiv.org/html/2609.25853#A9.T10)reports the average effective contribution of E, GE, and GH to the injected memory residual on the corresponding validation corpora\. The statistic is averaged over token positions, layers, and batches\. Because the deployed update has the form

r=\(1−α\)​e\+α​g,r=\(1\-\\alpha\)e\+\\alpha g,E retains a contribution of1−α1\-\\alphaeven when a generated pathway is admitted\. Thus, interpolation mass and source\-selection frequency measure different aspects of the routing behavior\.

The QA checkpoint places most of its interpolation mass on GH, whereas the NLP and coding checkpoints retain substantially more mass on E\. GE receives a smaller average contribution in all three settings\. This variation suggests that the learned balance among the three pathways depends strongly on the training domain and checkpoint\.

For comparison, the calibration teacher stream records generated\-pathway admission rates of 99\.97% for QA and 98\.07% for NLP\. These high admission rates are compatible with non\-trivial E mass because an admitted generated pathway can still be interpolated with E usingα<1\\alpha<1\. Admission rate and interpolation mass therefore should not be conflated\.

A more detailed downstream analysis could additionally report, for each task, the fraction of positions admitting GE or GH, the exact\-E fallback rate, the conditional mean interpolation coefficientα\\alpha, and the resulting effective pathway weights\.

## Appendix JTransfer and secondary evaluations

##### Transfer\.

Mistral\-to\-Llama QA scores are 29\.98, 36\.40, 67\.39, 30\.41, and 25\.17\. WebQA exceeds the source Mistral router, and the other four are lower\. On six NLP tasks, the target router mean is 40\.48 versus 38\.23 for target E, with Yahoo at 10%\. These results demonstrate target\-side adaptation only\. A matched advantage over bare Llama would require its same\-scorer baseline, which is missing \(Appendix[J](https://arxiv.org/html/2609.25853#A10)\)\.

Table 11:Target\-Llama NLP accuracy \(%\)\. Without matched bare\-Llama scores, these compare target memory endpoints only\.The router improves MR, CR, RT, and AGN, decreases SST2, and leaves Yahoo unchanged\. Low absolute accuracies make these interface\-transfer diagnostics rather than evidence of broadly successful NLP transfer\. Without the bare target model, positive transfer and avoidance of negative transfer are not established\.

Table 12:Performance on the HaluEval benchmark for question answering and summarization\. Results report accuracy \(%\)\. Small signed values denote percentage\-point differences from Mistral\-7B\-v0\.3\. Average is the unweighted mean of QA and Summarization\. RAG is not evaluated on summarization because this task requires only the source document\.HaluEval evaluates hallucination recognition\([12](https://arxiv.org/html/2609.25853#bib.bib9)\)\. Summarization improves by 20\.01 points, while dialogue changes modestly and QA decreases\. Confusion matrices and class\-balance checks are needed before attributing the summarization gain to improved detection rather than response bias\. Classification on this benchmark is not a direct measurement of hallucinations in free\-form generation\.

## Appendix KScaling Analysis

We study two complementary scaling dimensions:*model scaling*, where the backbone and the complete memory\-side system are enlarged jointly, and*training\-token scaling*, where the architecture is fixed and only the optimization budget is increased\. We report only completed runs\.

##### Model\-scaling setup\.

We use GPT\-2 Small, Medium, Large, and XL as the backbone family\. As the backbone grows, the Engram table, generated\-memory modules, readers, and routing head are scaled jointly rather than varying the memory table in isolation\. Table[13](https://arxiv.org/html/2609.25853#A11.T13)summarizes the corresponding architectures and parameter counts\.

Table 13:Model\-scaling configurations\. “Gen\.” reports generator width/depth/number of generated latents\. Memory\-side total includes the Engram table, generated\-memory modules, readers, and router, but excludes the backbone\.The Engram addressing structure is kept fixed across scales, with maximumnn\-gram order 3 and four heads per order\. Generator width increases from 256 to 896, generator depth from 2 to 4 layers, the number of generated latents from 4 to 12, the source\-adaptation rank from 16 to 56, and the router hidden width from 64 to 224\. The resulting memory\-side system grows from 37\.6M parameters with GPT\-2 Small to 472\.9M with GPT\-2 XL\.

The routing head itself remains comparatively small\. Its parameter count grows from approximately 0\.054M, 0\.119M, and 0\.233M to 0\.413M across the four scales, while most memory\-side capacity is allocated to the Engram table and the generated\-memory interface\.

##### Training procedure\.

We consider two corpora for model scaling\. The WikiText setting uses a maximum budget of 100M processed tokens per optimization stage, while the general\-text mixture uses up to 600M tokens per stage\.

Training follows the same staged procedure as the main experiments\. The memory is first learned with the source backbone fixed\. The generated\-memory modules and readers are then optimized while the backbone and memory table are fixed\. Finally, these components are frozen and only the routing head is trained from E\-relative advantage supervision\. The no\-memory results are obtained by directly evaluating the corresponding pretrained GPT\-2 backbones and are used only as reference points\.

##### Model scaling\.

Figure[7](https://arxiv.org/html/2609.25853#A11.F7)shows the completed model\-scaling experiments\. Across all completed runs, the same ordering is observed:

Router<Memory<No memory,\\text\{Router\}<\\text\{Memory\}<\\text\{No memory\},where lower perplexity is better\.

\(a\)WikiText\.\(b\)General\-text mixture\.
Figure 7:Model scaling across GPT\-2 backbones\. Lower perplexity is better\. The routed system consistently improves over the corresponding memory\-only configuration at every completed scale\.On WikiText, perplexity decreases from 30\.841 to 24\.033 to 23\.372 for GPT\-2 Small, from 22\.569 to 17\.797 to 17\.291 for Medium, and from 19\.342 to 14\.955 to 14\.545 for Large, corresponding to the no\-memory, memory\-only, and routed systems\.

The completed general\-mixture runs exhibit the same pattern\. For GPT\-2 Small, perplexity decreases from 36\.752 to 32\.300 and then to 31\.772; for Medium, it decreases from 27\.958 to 24\.445 and then to 24\.181\. Thus, the routing benefit persists as the backbone and memory\-side system are jointly scaled\. The current results support persistence of the gain across scale rather than an increasing routing advantage with model size\.

Table 14:Exact perplexities for the completed model\-scaling runs\. Lower is better\.
##### Training\-token scaling\.

We next isolate the effect of training budget while holding model capacity fixed\. These experiments use GPT\-2 XL with the same 405\.5M\-parameter Engram table and identical generator, reader, and router architectures\. Only the number of processed training tokens per stage is varied\.

Figure[8](https://arxiv.org/html/2609.25853#A11.F8)a shows that increasing the training budget from 10M to 30M and 100M tokens monotonically reduces perplexity for both systems\. Memory\-only perplexity decreases from 20\.714 to 20\.369 and 19\.783, while routed\-memory perplexity decreases from 20\.222 to 19\.889 and 19\.462\. The routed model therefore remains better than memory alone at every completed training budget\.

Figure[8](https://arxiv.org/html/2609.25853#A11.F8)b provides the corresponding parameter breakdown across model scales\. Most of the memory\-side capacity is allocated to the Engram table, followed by the generated\-memory modules and readers, while the routing head contributes only a small fraction of the total parameter count\.

\(a\)Training\-token scaling with GPT\-2 XL fixed\.\(b\)Memory\-side parameter scaling\.
Figure 8:Training and capacity scaling\. Left: increasing the per\-stage training budget improves both memory\-only and routed systems while routing remains consistently better\. Right: breakdown of memory\-side parameters as the complete memory system is scaled with the backbone\.Table 15:Training\-token scaling with the GPT\-2 XL architecture fixed\. Only the per\-stage optimization budget changes\.Overall, the completed scaling experiments show two consistent trends\. First, the benefit of adaptive routing is preserved as the backbone and memory\-side architecture are jointly enlarged\. Second, increasing the training budget improves both memory\-only and routed systems, while the routing gain remains present throughout the evaluated range\.

## Appendix LCase Study

Figure[9](https://arxiv.org/html/2609.25853#A12.F9)shows two representative downstream examples\. In both cases, none of the standalone pathways \(EE,G​EGE, orG​HGH\) yields the correct final answer, whereasMemoryAthenadoes\.

In the first example, the task is to identify which of two events occurred earlier\. AlthoughG​HGHreceives most of the routed source mass, its standalone answer is still incorrect, while the routed system recovers the correct year, 1907\. In the second example, the question asks for the winner of the 1992 Spengler Cup\. The standalone outputs are either noisy or incomplete, but the routed system returns the exact answer,HC Davos\. These examples illustrate that the benefit of routing is not simply selecting the best single endpoint\. Instead,MemoryAthenacan exploit complementary information across memory pathways and transform imperfect endpoint predictions into a correct final answer\.

![Refer to caption](https://arxiv.org/html/2609.25853v1/figures/memoryathena_case_studies.png)Figure 9:Two downstream case studies ofMemoryAthena\. The routed system is correct in both examples although all individual endpoints \(EE,G​EGE, andG​HGH\) are incorrect\. The route traces show the effective source mass assigned to each pathway\.
## Appendix MComputational Cost

We analyze the computational cost ofMemoryAthenafrom two perspectives: the additional optimization required to learn the generated memory interface and routing policy, and the runtime overhead introduced during inference\. The routing head itself is parameter\-light, but the complete system must additionally evaluate generated\-memory components and, in the current implementation, obtain clean causal backbone states for the GH pathway\.

### M\.1Training Cost

Training follows the three\-stage procedure described in Appendix[D](https://arxiv.org/html/2609.25853#A4)\. The memory is learned first, the generated\-memory modules and readers are then optimized with the backbone and memory table frozen, and the final stage trains only the E\-relative routing heads\.

The reader/generator stage contains approximately 167\.4M trainable parameters, whereas the final router contains only 534,924 trainable parameters\. Thus, the routing head accounts for only a small fraction of the trainable memory\-interface parameters\.

Table 16:Recorded wall\-clock training time for the reader/generator and routing stages\. The router contains 0\.535M trainable parameters, compared with approximately 167\.4M in the reader/generator stage\.The recorded reader/generator and router stages together take approximately 42\.89 hours for QA, 26\.59 hours for general NLP, and 27\.35 hours for coding\. These numbers describe the recorded stages rather than the complete lifetime cost of the reusable source memory\.

Although only a small routing head is optimized in the final stage, routing training still requires non\-trivial computation\. Counterfactual supervision is constructed by evaluating multiple frozen endpoints under teacher forcing, and the GH pathway additionally requires clean backbone states\. Consequently, trainable parameter count alone is not a direct measure of total training compute\.

### M\.2Inference Microbenchmark

We additionally benchmark the inference overhead ofMemoryAthenarelative to the direct E\-only pathway across GPT\-2 Small, Medium, Large, and XL\.

All measurements are performed on an AMD MI250X GPU using BF16 precision\. Each run uses a fixed synthetic sequence of 338 input tokens and generates 16 output tokens\. We perform one warmup iteration followed by three measured iterations\. We report end\-to\-end latency, total\-token throughput, and peak reserved GPU memory\.

The E\-only configuration executes the direct Engram pathway\. In contrast,MemoryAthenaadditionally evaluates the generated\-memory interface, routing features, and the clean causal backbone states required by GH in the current implementation\.

\(a\)End\-to\-end latency\.\(b\)Total\-token throughput\.\(c\)Peak reserved GPU memory\.
Figure 10:Inference microbenchmark on an AMD MI250X using BF16, with a fixed 338\-token input and 16 generated tokens\.Left:latency, where annotations show the speed advantage of E\-only inference\.Middle:total\-token throughput\.Right:peak reserved GPU memory, with annotations showing the relative memory overhead ofMemoryAthena\. The relative runtime and memory overhead decrease as the backbone scales\.##### Latency and throughput\.

E\-only inference is faster at all four evaluated scales\. For GPT\-2 Small, latency increases from 146\.7 ms for E\-only to 276\.0 ms forMemoryAthena, corresponding to a 1\.88×\\timesspeed advantage for E\-only\. The relative gap decreases with model size, to 1\.50×\\timesfor Medium, 1\.43×\\timesfor Large, and 1\.40×\\timesfor XL\.

The corresponding total\-token throughput decreases from 2412\.4 to 1282\.9 tokens/s at Small, from 1352\.0 to 898\.4 tokens/s at Medium, from 942\.7 to 658\.4 tokens/s at Large, and from 702\.6 to 500\.7 tokens/s at XL\. Thus, the absolute cost of both systems increases with model scale, while the relative overhead of the routed system becomes smaller\.

##### Memory overhead\.

Peak reserved GPU memory increases from 0\.57 to 0\.93 GiB at Small, 1\.60 to 2\.28 GiB at Medium, 3\.63 to 4\.90 GiB at Large, and 7\.21 to 9\.56 GiB at XL\. These correspond to relative overheads of approximately 63\.2%, 42\.5%, 35\.0%, and 32\.6%, respectively\.

The memory results therefore exhibit the same qualitative trend as latency: althoughMemoryAthenarequires additional runtime state, this additional cost represents a smaller fraction of the overall system footprint as the backbone becomes larger\.

##### Where does the overhead come from?

The additional cost should not be attributed primarily to the routing MLP\. The routing head contains only 534,924 parameters\. Instead, the main runtime overhead comes from evaluating the generated\-memory pathways and maintaining their intermediate states\. In particular, GH currently requires a separate memory\-disabled backbone computation to obtain its clean causal conditioning states\.

This distinction is important:MemoryAthenais*parameter\-efficient*as a routing mechanism, but it is not a zero\-overhead inference method\.

##### Scope of the benchmark\.

The experiment is a controlled microbenchmark using a fixed synthetic prompt, generation length, precision, and hardware configuration\. Its purpose is to measure relative systems overhead under matched conditions\. The reported numbers should therefore not be interpreted as full downstream\-task throughput, which can vary with sequence length, batch size, generation length, routing behavior, and hardware utilization\.

Overall,MemoryAthenatrades additional computation for adaptive use of generated memory\. The inference overhead is measurable at all evaluated scales, but its relative cost decreases with backbone size: the E\-only latency advantage falls from 1\.88×\\timesat Small to 1\.40×\\timesat XL, while the reserved\-memory overhead decreases from approximately 63% to 33%\.

## Appendix NLanguage\-model diagnostics and incomplete extensions

Table 17:General\-NLP within\-run validation perplexity\. Endpoints and downstream accuracy need not rank identically\.EGEGHRouter9\.072608\.790918\.783588\.83063The NLP router improves E perplexity but not either generated endpoint\. QA likewise records lower validation perplexity for ordinary soft fusion \(7\.07310\) and hard routing \(7\.09269\) than for the advantage router \(7\.14538\), with E at 8\.57136\. Corpus perplexity alone does not establish better QA performance\.

##### Coding\.

Completed Nemotron\-CC\-Code expert records give perplexities 3\.10842 \(E\), 2\.99503 \(GE\), and 2\.98369 \(GH\), with arithmetic mean 3\.02853\. Routing records give 2\.99563 after 19,998,720 input positions which is lower than tri\-experts mean\.