Towards Evolving Context Parameterization for Large Language Models

# Towards Evolving Context Parameterization for Large Language Models
Source: [https://arxiv.org/html/2609.14168](https://arxiv.org/html/2609.14168)
Zherui LiAffiliation:Nanyang Technological UniversityYiming JiangAffiliation:Nanyang Technological UniversityKun WangAffiliation:Nanyang Technological UniversityYufei GuoAffiliation:Peking University

###### Abstract

Context parameterization enables large language models \(LLMs\) to internalize contexts into reusable model parameters, avoiding repeated processing across subsequent queries\. However, existing methods typically assume static contexts and lack explicit mechanisms for distinguishing validity states under continual updates\. To study this real\-world scenario, we formalized theMemoryUpdating withSequentialEvolution\(MUSE\)task and constructedMUSE\-Benchto evaluate update incorporation and unaffected\-information preservation\. The resulting challenge requires preserving the global state while adjusting the contribution of memory evidence\. Motivated by this, we proposedPLUME, a training\-free method that constructs a global update representation, activates memory evidence to form a local parameter view, and adaptively integrates their predictions during decoding\. Comprehensive evaluation onMUSE\-BenchdemonstratedPLUME’s effectiveness in sequential evolution settings, yielding relative improvements of29\.9%in average ROUGE\-L Recall and54\.9%in LLM\-as\-a\-Judge\. Our codes are available at:[https://github\.com/xiaobingshi\-LLM/PLUME](https://github.com/xiaobingshi-LLM/PLUME)\.

## 1Introduction

Large Language Models \(LLMs\) demonstrated remarkable general\-purpose capabilities in question answering, reasoning, and text generation, advancing natural language processing\([Brown et al\., 2020](https://arxiv.org/html/2609.14168#bib.bib4);[Wei et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib45);[Guo et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib15)\)\. Many practical tasks are context\-intensive, requiring models to efficiently locate and reuse information from long contexts across successive queries\([Bai et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib2);[Bai et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib3);[Li et al\., 2025a](https://arxiv.org/html/2609.14168#bib.bib25)\)\. Existing retrieval\-augmented generation \(RAG\)\([Lewis et al\., 2020](https://arxiv.org/html/2609.14168#bib.bib24);[Guu et al\., 2020](https://arxiv.org/html/2609.14168#bib.bib16)\)and long\-context methods\([Ding et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib11);[Chen et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib8)\)incur substantial inference overhead from repeated context processing\([Gao et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib13)\), whereas context parameterization decouples this process from subsequent queries, enabling efficient reuse\([Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40);[Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)\. Prior research followed two lines: ① one updated parametric knowledge throughlocalized parameter modificationswithout retraining the model, as in knowledge editing\([Mitchell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib34);[Meng et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib32);[Meng et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib33)\); ② the other internalized contextual information intoreusable parameter representations, as in context distillation, Doc\-to\-LoRA, which compiled contexts into lightweight representations for subsequent queries\([Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40);[Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)\.

Although these methods enabled effective, efficient context reuse, none addressedparameterization under evolving information validity\([Liu et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib30);[Tian et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib42)\)\. In long\-running agents and interactive systems, information continuously accumulates into persistent memory, making full context reconstruction impractical\([Park et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib37);[Packer et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib36);[Zhong et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib50)\), motivating parameterized states that incorporate new information while preserving prior\([Jang et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib20)\)\. However, under continual updates, some information remains valid while other information is supplemented, replaced, or invalidated\([Wallat et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib43)\), and jointly retaining information with different validity states can cause reliance on invalid information or inconsistent responses\([Marjanović et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib31);[Wallat et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib43)\)\. Incorporating continual updates into a parameterized state while adhering to valid information thus remains an open challenge, as illustrated in Figure[1](https://arxiv.org/html/2609.14168#S1.F1): after an update, the parameterized state mustincorporate updates while preserving what remains unaffected\.

To study this continual\-update setting, we formalizedMemory Updating with Sequential Evolution \(MUSE\)and designed a data construction pipeline preserving the original question and context structures, updating only the target and coupled information, and appending the updated context to the original\. This construction produces a context history with explicit update order and validity changes that supports the updated answer while no longer supporting the previous one\. Using this pipeline, we constructedMUSE\-Benchfrom five datasets, covering QA and reasoning\. Unlike prior work on conflicts between external context and model priors\([Xie et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib46);[Xu et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib47);[Kortukov et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib23)\),MUSE\-Benchstudies continual updates wherevalid and invalid information coexist within a parameterized state\. Experiments showed that parameterizing the full context into a unified representation substantially degraded performance, indicating severe interference among information with different validity states\.

![Refer to caption](https://arxiv.org/html/2609.14168v1/intro.png)Figure 1:Illustration of context parameterization under continual updates\.While existing methods perform well in conventional settings, they may conflate obsolete information with subsequent updates, whereasPLUMEfollows the currently valid information\.Building onMUSE\-Benchevaluation, we examined whether interference reflectedinformation loss or merely suppressed activationby computing token\-level ROUGE\-L Recall under teacher forcing for top\-1 and top\-5 candidate tokens at each step\. Top\-5 recall reached71\.79%, substantially exceeding top\-1’s45\.90%, indicating thatvalid information was largely retained in the output distribution but often outranked by invalid candidates\. This finding motivatesPLUME’s design: since the signal remains present but underweighted, we amplify the global update representation\. However, suppression varies across queries, and uniform amplification does not consistently identify which candidates require reweighting, motivating memory evidence as a local signal for targeted, per\-query correction\. Memory evidence exhibits reduced reliability under phrasing divergence and often captures only partial information; we therefore incorporate it as a complement to the global representation rather than as a standalone signal\.PLUMEtherefore adaptively integrates both during decoding—the global representation as a stable priorandmemory evidence as a targeted correction\.

Experiments spanned multiple model families, includingQwen3andGemma2\([Yang et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib48);[Team et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib41)\), onMUSE\-Bench, comparingPLUMEwith knowledge editing\([Jiang et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib21)\), context distillation\([Askell et al\., 2021](https://arxiv.org/html/2609.14168#bib.bib1);[Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40)\), Doc\-to\-LoRA\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\), and other representative baselines\. Averaged acrossMUSE\-Bench,PLUMEimproved ROUGE\-L Recall from44\.46\(D2L\) to57\.76\(29\.91%↑\\uparrow\), and LLM\-as\-a\-Judge from34\.49to53\.41\(54\.86%↑\\uparrow\), while consistently outperforming all baselines across models and task settings\. These results demonstrate thatPLUMEeffectively mitigates parametric interference among information with different validity states while improving adherence to currently valid information throughout sequential evolution without sacrificing efficient context reuse\.

## 2Background

We study context parameterization under continual updates, internalizing information into a parameterized state reused across queries asCCgrows monotonically\. LetCC,qq, andyydenote the context, query, and answer, andpθ\(⋅∣⋅\)p\_\{\\theta\}\(\\cdot\\mid\\cdot\)the conditional distribution of aθ\\theta\-parameterized language model\. Conventional long\-context and RAG methodssupply context at inference time—concatenating context into the input window or retrieving passages—viapθ​\(y∣C,q\)p\_\{\\theta\}\(y\\mid C,q\)\([Lewis et al\., 2020](https://arxiv.org/html/2609.14168#bib.bib24);[Guu et al\., 2020](https://arxiv.org/html/2609.14168#bib.bib16);[Ding et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib11)\), incurring reprocessing costs scaling with\|C\|\|C\|, compounding across queries\([Gao et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib13)\), and suffering position bias, underutilizing evidence away from the input’s boundaries\([Liu et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib29);[Hsieh et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib18)\)\. These costs motivate context parameterization internalizingCCinto parameters decoupled from per\-query inference\.

Context Distillation \(CD\) instead transfers contextual information from explicit input into model parameters\. GivenCCand its query distribution𝒬C\\mathcal\{Q\}\_\{C\}, CD uses a teacher conditioned onCCto optimize a context\-specific parameter stateθC\\theta\_\{C\}:

minθC𝔼q∼𝒬C\[DKL\(pθ\(⋅∣C,q\)∥pθC\(⋅∣q\)\)\]\.\\min\_\{\\theta\_\{C\}\}\\mathbb\{E\}\_\{q\\sim\\mathcal\{Q\}\_\{C\}\}\\\!\\left\[D\_\{\\mathrm\{KL\}\}\\\!\\left\(p\_\{\\theta\}\(\\cdot\\mid C,q\)\\,\\\|\\,p\_\{\\theta\_\{C\}\}\(\\cdot\\mid q\)\\right\)\\right\]\.\(1\)The teacher has direct access toCC, whereas the student is optimized to reproduce the teacher distribution using only the query\. In this way, information originally supplied through the input context is encoded into the resulting parameter state\. After distillation, the model can answer queries without explicitly receivingCCduring inference, allowing the same parameterized context to be reused across multiple queries\([Askell et al\., 2021](https://arxiv.org/html/2609.14168#bib.bib1);[Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40)\)\. However, CD requirescontext\-specific distillation data and iterative optimizationfor each new context, making repeated updates costly\.

Doc\-to\-LoRA \(D2L\) amortizes this per\-context optimization using a hypernetworkHϕH\_\{\\phi\}that maps a context directly to a LoRA weight delta\([Hu et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib19);[Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\):

Δ​WC=Hϕ​\(C\),θC=θ⊕Δ​WC\.\\Delta\\mathrm\{W\}\_\{C\}=H\_\{\\phi\}\(C\),\\qquad\\theta\_\{C\}=\\theta\\oplus\\Delta\\mathrm\{W\}\_\{C\}\.\(2\)Here,Δ​WC=BC​AC\\Delta\\mathrm\{W\}\_\{C\}=B\_\{C\}A\_\{C\}denotes the effective low\-rank weight update induced by the generated LoRA factors\(AC,BC\)\(A\_\{C\},B\_\{C\}\), and⊕\\oplusdenotes applying this update to the frozen base model\. Instead of optimizing a separate parameter state for each context, D2L learns a generalizable mapping from contextual information to parameter updates across a collection of training contexts\. Once trained,HϕH\_\{\\phi\}generates a reusable context\-specific adapter ina single forward pass, thereby substantially reducing the parameterization cost for previously unseen contexts\. The generated adapter can be independently applied to or removed from the frozen base model, providing a lightweight parametric representation of the corresponding context\. However, D2L parameterizes contexts without explicitly modeling continual updates to existing parameterized states\.

![Refer to caption](https://arxiv.org/html/2609.14168v1/method.png)Figure 2:The overall framework of our proposedPLUME\.PLUMEconstructs a global update representation from the full context state, activates memory evidence according to the query to provide a local parameter view, and adaptively integrates the global and local predictions during decoding\.
## 3MUSE\-Bench

To systematically evaluate context parameterization under continual updates, we defined theMUSE\(Memory Updating with Sequential Evolution\)task and constructedMUSE\-Bench, focusing on incorporating currently valid information while preserving unaffected information\.

### 3\.1Motivation and Task Formulation

##### Motivation\.

Existing context parameterization methods primarily assume static, internally consistent contexts, without explicitly considering conflicts arising as contexts evolve\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)\. Prior evaluations primarily assessed either static context internalization\([Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40);[Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)or targeted factual modifications\([Meng et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib32);[Meng et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib33)\), leaving the incorporation of continual updates and preservation of unaffected information in parameterized representations insufficiently evaluated\.MUSE\-Benchis designed to address this gap in a unified setting\.

##### Task Formulation\.

We formulate sequential evolution as a series of state transitions\. At update steptt, letC\(t−1\)C^\{\(t\-1\)\}denote the context history preceding transitiontt, and letCu\(t\)C\_\{u\}^\{\(t\)\}denote the update context\. The resulting context history is

C\(t\)=C\(t−1\)\|Cu\(t\),C^\{\(t\)\}=C^\{\(t\-1\)\}\\\|C\_\{u\}^\{\(t\)\},\(3\)whereCu\(t\)C\_\{u\}^\{\(t\)\}updates randomly sampled related information inC\(t−1\)C^\{\(t\-1\)\}, while unaffected information remains valid\. The latest correction notice marks the state\-transition boundary\. This formulation applies recursively: after steptt,C\(t\)C^\{\(t\)\}becomes the pre\-update history for stept\+1t\+1\. Let𝒜\\mathcal\{A\}map pre\-update and current histories to a context\-specific parameter state, optionally query\-conditioned:

θ\(t\)​\(q\)=𝒜⁡\(θ,C\(t−1\),C\(t\),q\),\\theta^\{\(t\)\}\(q\)=\\mathcal\{A\}\\\!\\left\(\\theta,C^\{\(t\-1\)\},C^\{\(t\)\};q\\right\),\(4\)whereθ\\thetadenotes the base model parameters\. Context affects predictions only through the parameterized stateθ\(t\)​\(q\)\\theta^\{\(t\)\}\(q\)\. The target language model receives onlyqqas input, with no context text included in its input\. The query may condition the construction or selection of the parameterized state\.

y^\(t\)​\(q\)=arg​maxy⁡pθ\(t\)​\(q\)​\(y∣q\)\.\\hat\{y\}^\{\(t\)\}\(q\)=\\operatorname\*\{arg\\,max\}\_\{y\}p\_\{\\theta^\{\(t\)\}\(q\)\}\(y\\mid q\)\.\(5\)Lety⋆\(t\)​\(q\)y^\{\\star\(t\)\}\(q\)denote the reference answer supported by valid information inC\(t\)C^\{\(t\)\}\. At each transition,𝒬upd\(t\)\\mathcal\{Q\}\_\{\\mathrm\{upd\}\}^\{\(t\)\}contains queries with update\-affected answers, while𝒬keep\(t\)\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}^\{\(t\)\}contains queries with unchanged answers\. Both categories are evaluated underC\(t\)C^\{\(t\)\}after parameterization\.MUSEevaluates whether the parameterized state incorporates the latest update while preserving unaffected information\.

### 3\.2Benchmark Construction

To instantiate the transition\-level formulation above,MUSE\-Benchdesignates one update transition per example and usesCoC\_\{o\},CuC\_\{u\}, andCfC\_\{f\}as shorthand forC\(t−1\)C^\{\(t\-1\)\},Cu\(t\)C\_\{u\}^\{\(t\)\}, andC\(t\)C^\{\(t\)\}, respectively\. It comprises five datasets: SQuAD, ROPES, 2WikiMultihopQA, MultiFieldQA\-en, and QASPER\([Rajpurkar et al\., 2016](https://arxiv.org/html/2609.14168#bib.bib38);[Lin et al\., 2019](https://arxiv.org/html/2609.14168#bib.bib28);[Ho et al\., 2020](https://arxiv.org/html/2609.14168#bib.bib17);[Bai et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib2);[Dasigi et al\., 2021](https://arxiv.org/html/2609.14168#bib.bib10)\), with approximately3,000 contextsand13,000 question\-answer pairs\. Given an original contextCo=c1,c2,…,cnC\_\{o\}=\{c\_\{1\},c\_\{2\},\\ldots,c\_\{n\}\}, we randomly sampled evidence units supporting a target answer and updated the corresponding facts while preserving the original question and context structures\. When other facts had semantic or reasoning dependencies on the target facts, weupdated them consistentlyto construct an internally coherent update contextCuC\_\{u\}, then appended it toCoC\_\{o\}to form the full context historyCfC\_\{f\}\. The resulting context history supports the currently valid answers for update\-affected queries while preserving the original answers for unaffected queries\.

Candidate samples were generated withGPT\-seriesmodels for fact modification, dependency identification, and question\-answer annotation, primarily usingGPT\-5\.5\([OpenAI, 2026](https://arxiv.org/html/2609.14168#bib.bib35)\)andGPT\-5\.6\-Solthroughout the benchmark construction process\. We further usedGPT\-5\.6\-Solto conduct 26 rounds of quality verification, with a subset of the annotations manually inspected in each round checking: \(1\) whetherCfC\_\{f\}supports the current correct answers; \(2\) whether coupled information is consistently updated; and \(3\) whether unaffected answers remain supported byCfC\_\{f\}\. Detailed construction procedures, annotation prompts, and quality control protocols are provided in Appendix[E](https://arxiv.org/html/2609.14168#A5)\.

### 3\.3Evaluation Protocol

MUSE\-Benchevaluates model outputs from four complementary perspectives of answer quality and update robustness\.D1 Answer Coverageuses token\-level ROUGE\-L Recall\([Lin, 2004](https://arxiv.org/html/2609.14168#bib.bib27)\)to measure reference\-answer coverage\.D2 Semantic Equivalenceevaluates whether the generated answer is semantically equivalent to the reference answer\.D3 Final\-Answer Agreementmeasures whether the final answer agrees with the reference answer\.D4 Localitymeasures whether the model preserves correct answers for queries unaffected by the update\. Together, these metrics evaluate both effective incorporation of valid information and reliable preservation of unaffected information\.

All metrics are computed at the query level and then aggregated within each dataset for overall performance comparison\. For D2 and D3, we usedGPT\-5\.6\-Solas the LLM Judge with unified evaluation prompts and consistent evaluation criteria\. Detailed evaluation settings and complete evaluation rubrics are provided in Appendix[B\.4](https://arxiv.org/html/2609.14168#A2.SS4)\.

## 4Method

We proposedParametricLoRAUpdates withMemoryEvidence \(PLUME\), which combines a global update representation, query\-activated memory evidence, and adaptive decoding for context parameterization under continual updates\. Figure[2](https://arxiv.org/html/2609.14168#S2.F2)illustrates the overall framework\.

### 4\.1Global Update Representation

Throughout sequential evolution, a parameterized context state can simultaneously entangle currently valid, invalidated, and unaffected information without an explicit mechanism for disentangling them\. Given the full context historyCfullC\_\{\\mathrm\{full\}\}and its pre\-update historyColdC\_\{\\mathrm\{old\}\}, we parameterize both separately with the pretrained D2L hypernetworkHϕH\_\{\\phi\}introduced in Section[2](https://arxiv.org/html/2609.14168#S2), obtainingΔ​Wold=Hϕ​\(Cold\)\\Delta\\mathrm\{W\}\_\{\\mathrm\{old\}\}=H\_\{\\phi\}\(C\_\{\\mathrm\{old\}\}\)andΔ​Wfull=Hϕ​\(Cfull\)\\Delta\\mathrm\{W\}\_\{\\mathrm\{full\}\}=H\_\{\\phi\}\(C\_\{\\mathrm\{full\}\}\)\. The global update representation is accordingly defined to jointly integrate these two parameterized states as

Δ​Wg=α​Δ​Wfull\+β⁡\(Δ​Wfull−Δ​Wold\),\\Delta\\mathrm\{W\}\_\{g\}=\\alpha\\Delta\\mathrm\{W\}\_\{\\mathrm\{full\}\}\+\\beta\\left\(\\Delta\\mathrm\{W\}\_\{\\mathrm\{full\}\}\-\\Delta\\mathrm\{W\}\_\{\\mathrm\{old\}\}\\right\),\(6\)whereα,β≥0\\alpha,\\beta\\geq 0are weighting coefficients: the first termretains the complete context state, and the secondselectively amplifies the parameter shift induced by the latest updatewithout discarding it\. Operationally, we implement Eq\.[6](https://arxiv.org/html/2609.14168#S4.E6)by concatenating the LoRA factors along the rank dimension, yielding a global adapter with rankrg=rfull\+roldr\_\{g\}=r\_\{\\mathrm\{full\}\}\+r\_\{\\mathrm\{old\}\}\. Appendix[B\.5\.1](https://arxiv.org/html/2609.14168#A2.SS5.SSS1)provides the exact construction and its extension to chunked contexts\. The full\-context component provides the broader context required for answering queries, including information unaffected by the update, while the difference term emphasizes changes between the pre\-update and full parameterized states\. Using the difference term alone would omit this explicit source of contextual support, since the parameter shift need not encode all background information required to interpret and answer a query\. We treatα\\alphaandβ\\betaas tunable weighting coefficients that favora supplementary rather than substitutive rolefor the update\-shift term, a design choice empirically validated in Section[5](https://arxiv.org/html/2609.14168#S5)\. At generation steptt, its predictive distribution is

pg,t​\(v\)=pθ⊕Δ​Wg​\(v∣q,y<t\),p\_\{g,t\}\(v\)=p\_\{\\theta\\oplus\\Delta\\mathrm\{W\}\_\{g\}\}\\left\(v\\mid q,y\_\{<t\}\\right\),\(7\)wherevvdenotes a candidate token,y<ty\_\{<t\}is the generated prefix, and⊕\\oplusdenotes applying the corresponding parameter update to the frozen base model\.

### 4\.2Query\-Activated Memory Evidence

While the global update representation aggregates the entire context history into a parameterized state, this compression can attenuate fine\-grained detail in long, information\-dense contexts\. We therefore constructa complementary local parameter viewby segmentingCfullC\_\{\\mathrm\{full\}\}into memory units and scoring lexical relevance via query\-term coverage and saturated term\-frequency matching\. Letsmaxs\_\{\\max\}denote the highest score among candidate units\. Among units whose scores fall within a marginδ\\deltaofsmaxs\_\{\\max\}, we activate the temporally latest as local memory evidencem⋆​\(q\)m^\{\\star\}\(q\), whereδ≥0\\delta\\geq 0controls how close in score two units must be for temporal order to take precedence, so thatrecency breaks ties only among comparably relevant candidatesrather than overriding a clearly stronger lexical match\. Temporal order follows each unit’s position withinCfullC\_\{\\mathrm\{full\}\}, withCupdateC\_\{\\mathrm\{update\}\}units treated as later thanColdC\_\{\\mathrm\{old\}\}units\. The memory evidence is parameterized as

Δ​We​\(q\)=Hϕ​\(m⋆​\(q\)\),\\Delta\\mathrm\{W\}\_\{e\}\(q\)=H\_\{\\phi\}\\left\(m^\{\\star\}\(q\)\\right\),\(8\)with the corresponding predictive distribution

pe,t​\(v\)=pθ⊕Δ​We​\(q\)​\(v∣q,y<t\)\.p\_\{e,t\}\(v\)=p\_\{\\theta\\oplus\\Delta\\mathrm\{W\}\_\{e\}\(q\)\}\\left\(v\\mid q,y\_\{<t\}\\right\)\.\(9\)Formed independently of the compressed global representation,Δ​We​\(q\)\\Delta\\mathrm\{W\}\_\{e\}\(q\)internalizes a localized memory state that surfaces under\-represented detail, reinforcing rather than replacing it\. The query determines which evidence is selected, butthe hypernetwork receives only the selected evidence text; the dependence ofΔ​We​\(q\)\\Delta\\mathrm\{W\}\_\{e\}\(q\)onqqtherefore arises through evidence selection\. During generation, this activated memory evidence influences predictions through the resulting parameter adapter\.

MethodSQuADRecall↑\\uparrowLLM↑\\uparrowROPESRecall↑\\uparrowLLM↑\\uparrow2WikiRecall↑\\uparrowLLM↑\\uparrowMFQARecall↑\\uparrowLLM↑\\uparrowQASPERRecall↑\\uparrowLLM↑\\uparrow\\SetRowht=abovesep=belowsep=Qwen\-3\\SetHline1\-110\.5pt,solid Base w/ Context \(oracle\)95\.8691\.6997\.8295\.9160\.1152\.3564\.5060\.0861\.7553\.95CD \(oracle\)94\.9391\.7598\.3197\.5268\.5165\.4466\.2461\.1762\.4655\.81D2L \(oracle\)76\.7764\.9285\.5780\.3943\.1338\.3328\.8020\.3422\.9815\.80\\SetHline1\-110\.5pt,solid Base w/o Context24\.6115\.1982\.8864\.9931\.7524\.6722\.4811\.331\.281\.89AnyEdit15\.7712\.4476\.4367\.4816\.2515\.0013\.239\.257\.404\.02CD40\.5727\.8574\.6967\.0036\.1530\.6730\.7823\.3316\.789\.57D2L54\.9637\.2786\.9981\.3336\.3030\.8427\.1512\.0016\.8811\.02\\SetRowfont=PLUME80\.6576\.8194\.6190\.2647\.2441\.4837\.7634\.0128\.5324\.48\\SetRowht=abovesep=belowsep=Gemma\-2\\SetHline1\-110\.5pt,solid Base w/ Context \(oracle\)90\.7384\.6584\.7380\.1736\.3928\.2131\.8127\.4952\.4844\.72CD \(oracle\)89\.9284\.7785\.8179\.3842\.2334\.3935\.1130\.8452\.4645\.40D2L \(oracle\)70\.8157\.4658\.8450\.4322\.1116\.1514\.757\.7219\.1012\.58\\SetHline1\-110\.5pt,solid Base w/o Context22\.0712\.5444\.4425\.9514\.766\.495\.023\.360\.951\.17AnyEdit16\.248\.9351\.8141\.7210\.378\.645\.234\.060\.540\.46CD32\.3020\.1954\.6345\.2120\.8214\.489\.726\.7612\.945\.23D2L49\.1731\.3657\.5348\.6921\.0915\.9611\.446\.5514\.227\.96\\SetRowfont=PLUME74\.8068\.6062\.7055\.7829\.8524\.8223\.2919\.4224\.5120\.35\\SetHline1\-112pt,solid

Table 1:A Comprehensive Comparison of Different Methods\.ROUGE\-L Recall and LLM\-as\-a\-Judge are reported on MUSE\-Bench datasets, whereboldandunderlinedenote the best and second\-best values respectively\.
### 4\.3Adaptive Memory Evidence Decoding

Here,pg,tp\_\{g,t\}characterizes the global update state, whilepe,tp\_\{e,t\}captures the query\-activated memory evidence\. Since the relative reliability of each context can vary unpredictably across generation steps depending on its alignment with the query, we adaptively reweight the evidence term based on its token\-level divergence from the global prediction, formally quantified via the JS divergence:

dt=DJS\(pe,t∥pg,t\),d\_\{t\}=D\_\{\\mathrm\{JS\}\}\\left\(p\_\{e,t\}\\parallel p\_\{g,t\}\\right\),\(10\)A largerdtd\_\{t\}indicates that the activated memory evidence departs substantially from the global prediction at steptt, suggesting complementary information worth emphasizing\. We therefore mapdtd\_\{t\}to an adaptive gating weight that increases monotonically with divergence and saturates atλmax\\lambda\_\{\\max\}:

λt=λmax​dtdt\+τ,\\lambda\_\{t\}=\\lambda\_\{\\max\}\\frac\{d\_\{t\}\}\{d\_\{t\}\+\\tau\},\(11\)whereλmax\\lambda\_\{\\max\}bounds the influence of the memory evidence, preventing the model from degenerating into a purely local decoder under sharp divergence, andτ\>0\\tau\>0controls the sensitivity ofλt\\lambda\_\{t\}todtd\_\{t\}\. Guided by this gate, weretain the global prediction as decoding backboneand fuse in this evidence through

St​\(v\)=log⁡pg,t​\(v\)\+λt​log⁡pe,t​\(v\),S\_\{t\}\(v\)=\\log p\_\{g,t\}\(v\)\+\\lambda\_\{t\}\\log p\_\{e,t\}\(v\),\(12\)with the final token selected asyt=arg⁡maxv​St​\(v\)y\_\{t\}=\\arg\\max\_\{v\}S\_\{t\}\(v\)at each generation step\. Asλt\\lambda\_\{t\}approaches zero, the fused scores approach the global log\-probabilities, making token selection increasingly governed by the global prediction\. Increasingλt\\lambda\_\{t\}gives greater weight to the local evidence’s relative token preferences, allowing it to more strongly reshape the ranking of candidate tokens\. Given its sensitivity to phrasing variation and partial coverage, memory evidence serves as a complementary signal to the global representation, preventing any single source from dominating the fused prediction\.

## 5Experiments

We evaluatedPLUMEonMUSE\-Bench, cross\-task generalization, efficiency and information retention, component ablations, and representative cases\.

### 5\.1Experimental Setup

##### Datasets and Models\.

We evaluated on the fiveMUSE\-Benchdatasets and used GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2609.14168#bib.bib9)\)and CRUXEval\([Gu et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib14)\)for cross\-task generalization, withQwen3andGemma2as model backbones\. Both D2L andPLUMEused the official pretrained D2L hypernetwork checkpoint for context parameterization\.

##### Baselines\.

We compared against Base Model \(w/o and w/ Context\), D2L\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\), CD\([Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40)\), and AnyEdit\([Jiang et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib21)\), further reporting CD and D2L oracles as reference points\. Detailed baseline configurations are in Appendix[B\.3](https://arxiv.org/html/2609.14168#A2.SS3)for reproducibility\.

##### Evaluation Metrics\.

We evaluated performance using ROUGE\-L Recall, LLM\-as\-a\-Judge, and Locality, while additionally reporting update and generation latency along with peak memory usage as efficiency metrics, with human evaluation detailed in the Appendix E\. All methods used identical generation settings unless otherwise specified\.

### 5\.2Main Results

As shown in Table[1](https://arxiv.org/html/2609.14168#S4.T1),PLUMEperformed best among practical parameterized methods across all five datasets, improving average ROUGE\-L Recall / LLM\-as\-a\-Judge from 44\.46/34\.49 to 57\.76/53\.41 \(∼\\sim29\.92%↑\\uparrow/∼\\sim54\.84%↑\\uparrow\) over D2L\. On SQuAD,PLUMEimproves ROUGE\-L Recall and LLM\-as\-a\-Judge from 54\.96/37\.27 to80\.65/76\.81\(∼\\sim46\.74%↑\\uparrow/∼\\sim106\.10%↑\\uparrow\)\. Meanwhile, on the longer, more challenging 2Wiki, MFQA, and QASPER datasets, it maintains consistent improvements \(e\.g\., 36\.30/30\.84 to 47\.24/41\.48 on 2Wiki,∼\\sim30\.14%↑\\uparrow/∼\\sim34\.50%↑\\uparrow\)\. By pairing a global update representation with query\-activated memory evidence,PLUMEbetter preserves currently valid information without providing the full context history to the target model\. Under continual updates, Recall declines by only 2\.09 points from 3 to 20 updates \(Appendix[D\.1](https://arxiv.org/html/2609.14168#A4.SS1.SSS0.Px1)\)\.

### 5\.3Efficiency and Information Retention

##### Computational Efficiency\.

We evaluated update and generation efficiency on SQuAD and 2Wiki in terms of latency and peak memory usage\.PLUMEmaintained update efficiency on par with lightweight context\-parameterization methods while incurring only modest additional overhead during generation\. CD and the parameter\-editing baselines, in contrast, required substantially greater optimization or update costs across both benchmarks\. These results collectively demonstrated thatPLUMEimproved robustness to continual updates without sacrificing the efficiency benefits characteristic of reusable parameterized contexts\. Detailed experimental results are provided in Appendix[D\.6](https://arxiv.org/html/2609.14168#A4.SS6)\.

##### Locality\.

Locality evaluates whether the model preserves answers to queries outside the scope of the update, thereby quantifying interference with previously retained information\. As shown in Figure[3](https://arxiv.org/html/2609.14168#S5.F3),PLUMEattains the strongest Locality among parameterized methods, surpassing D2L and CD by 10\.25 \(∼\\sim13\.85%↑\\uparrow\) and20\.30\(∼\\sim31\.75%↑\\uparrow\) points, respectively\. This result indicates thatPLUMEmore effectively confines the interference introduced by parameter updates, thereby better preserving the established context state\.

Figure 3:Locality under updates\.Higher scores indicate better unaffected information preservation\.MethodGSM8KCRUXEvalRecallLLMRecallLLMQwen3w/ Context \(oracle\)54\.2152\.7668\.4355\.37CD \(oracle\)51\.5450\.8365\.0753\.20D2L \(oracle\)36\.2330\.2444\.0920\.50w/o Context15\.217\.8237\.754\.62AnyEdit16\.991\.9044\.4110\.50CD36\.4026\.3151\.5213\.75D2L23\.1816\.4239\.4813\.63PLUME45\.3941\.4762\.3150\.62Gemma2w/ Context \(oracle\)42\.9039\.8647\.3733\.23CD \(oracle\)41\.9639\.9247\.0931\.81D2L \(oracle\)22\.2315\.2934\.3612\.83w/o Context10\.124\.4118\.072\.96AnyEdit10\.671\.2634\.356\.77CD23\.0312\.5230\.5510\.74D2L20\.0411\.7429\.9511\.44PLUME33\.6929\.6540\.5226\.12

Table 2:Cross\-task generalization results\.ROUGE\-L Recall and LLM\-as\-a\-Judge are reported\.

### 5\.4Generalization and Overall Comparison

We further extend the sequential evolution setting to GSM8K and CRUXEval\. As shown in Table[2](https://arxiv.org/html/2609.14168#S5.T2),PLUMEconsistently outperforms D2L across tasks: ① on GSM8K, the average scores improve from 23\.18/16\.42 to 45\.39/41\.47 \(∼\\sim95\.82%↑\\uparrow/∼\\sim152\.56%↑\\uparrow\); ② on CRUXEval, they increase from 39\.48/13\.63 to 62\.31/50\.62 \(∼\\sim57\.83%↑\\uparrow/∼\\sim271\.39%↑\\uparrow\)\. These results demonstrate thatPLUMEgeneralizes beyond the QA settingofMUSE\-Bench\. Figure[4](https://arxiv.org/html/2609.14168#S5.F4)further shows balanced improvements across evaluation dimensions, together with strong efficiency and robust information retention under continual updates\.

Figure 4:Overall comparison of effectiveness and efficiency\.Normalized metrics include answer quality, locality, and update/generation efficiency\.Figure 5:Ablation ofPLUMEcomponents\.The memory\-only and global\-only variants each underperformPLUME, demonstrating the complementary contributions of the global update representation and query\-activated memory evidence\.![Refer to caption](https://arxiv.org/html/2609.14168v1/case_study_bg.png)Figure 6:Representative case under a context update\.D2L conflates information with different validity states and produces an incorrect answer, whereasPLUMEfollows the currently valid update\.
### 5\.5Ablation Study

We comparedPLUMEwith two component ablations: a memory\-only variant and a global\-only variant\. As shown in Figure[5](https://arxiv.org/html/2609.14168#S5.F5),PLUMEachieved an average normalized ROUGE\-L Recall score of81\.36, compared with 57\.28without the global update representationand 63\.95without memory evidence, corresponding to improvements of24\.09\(∼\\sim42\.05%↑\\uparrow\) and17\.41\(∼\\sim27\.22%↑\\uparrow\) points\. For normalized Locality,PLUMEreached87\.37, exceeding the two ablations by11\.03\(∼\\sim14\.45%↑\\uparrow\) and4\.80\(∼\\sim5\.81%↑\\uparrow\) points\.The larger degradation without the global update representation highlights its role in maintaining the current context state, while memory evidence provides complementary gains in answer quality and preserving unaffected information\.

### 5\.6Case Study

Representative cases contained valid, invalidated, and unaffected information within the same parameterized state, creating a substantial challenge for reliable validity\-state disambiguation\. As shown in Figure[6](https://arxiv.org/html/2609.14168#S5.F6), D2L became confused by competing information with different validity states and generated an incorrect answer, whereasPLUMEbetter captured the global update representation and emphasized the latest valid evidence\. This behavior is consistent with the validity\-state interference observed in the aggregate results and further reinforces our quantitative findings\.

## 6Related Work

##### Context Internalization\.

RAG\([Gao et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib13)\)and long\-context models\([Ding et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib11)\)supply context at inference time, incurring repeated overhead\. Context parameterization instead internalizes context into reusable representations\. CD transfers context\-induced behavior into model parameters\([Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40)\), while D2L compiles context into lightweight parameterized representations\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)\. Both, however, presuppose static, internally consistent contexts, leaving validity changes under updates unaddressed\.

##### Knowledge Editing and Continual Updates\.

Knowledge editing modifies factual knowledge in language models without full retraining\([Yao et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib49)\), with representative methods including ROME, MEMIT, and AlphaEdit\([Meng et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib32);[Meng et al\., 2023](https://arxiv.org/html/2609.14168#bib.bib33);[Fang et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib12)\)\. Recent work has further examined sequential editing, interference, and reliable preservation of prior knowledge\([Wang et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib44);[Chen et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib7);[Jiang et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib22);[Li et al\., 2025b](https://arxiv.org/html/2609.14168#bib.bib26)\)\. Our setting instead studies continual updates to parameterized contexts with factual dependencies and unaffected information\.

##### Knowledge Conflicts\.

Prior work has primarily examined conflicts between parametric knowledge and external context\([Xie et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib46);[Shi et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib39)\), or conflicts among retrieved contexts\([Cattan et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib5);[Xu et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib47)\)\. These settings expose conflicting evidence directly to the model at inference time\. In contrast, our setting concerns currently valid and invalidated information already jointly internalized within parameters, without direct access to the original context\.MUSE\-BenchandPLUMEare designed for this setting\.

## 7Conclusion

We introducedMUSEandMUSE\-Benchto study context parameterization under continual updates, jointly evaluating update incorporation and preservation of unaffected knowledge\. Our training\-free method,PLUME, combines a global update representation with query\-activated memory evidence through adaptive decoding\. Experiments across datasets and backbones demonstrated improvements in answer quality and locality, with further gains on mathematical and code reasoning tasks\. Ablations supported the complementary roles of global representations and local evidence\. These findings support coordinating global updates with selective evidence access to build reliable, reusable parameterized memory\.

## Limitations

While we believe thatPLUMEoffers a valuable contribution to context parameterization and utilization for continually updated knowledge, several limitations should be acknowledged\. First,PLUMEis built entirely atop D2L, whose hypernetwork compresses a context into a compact set of LoRA parameters;this parameterization is inherently lossy, and long or information\-dense contexts may not be faithfully preserved during encoding, which in turn upper\-bounds the overall performance ofPLUMEby the fidelity and capacity of the underlying parameterization method\. Second,PLUMEis deliberately training\-free, a design choice that preserves its flexibility, low memory footprint, and low latency, but that also confines its interventions to parameter construction and decoding time,without directly enhancing the hypernetwork’s intrinsic ability to discriminate valid updates from invalidated information\. Our future work will therefore focus on jointly improving the fidelity of context parameterization and equipping the hypernetwork itself with update\-aware, training\-based mechanisms, aiming to provide continued valuable insights for the development of truly reliable context\-adaptive systems\.

## References

- Askell et al\. \(2021\)Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, and 1 others\. 2021\.A general language assistant as a laboratory for alignment\.*arXiv preprint arXiv:2112\.00861*\.
- Bai et al\. \(2024\)Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, and 1 others\. 2024\.Longbench: A bilingual, multitask benchmark for long context understanding\.In*Proceedings of the 62nd annual meeting of the association for computational linguistics \(volume 1: Long papers\)*, pages 3119–3137\.
- Bai et al\. \(2025\)Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, and 1 others\. 2025\.Longbench v2: Towards deeper understanding and reasoning on realistic long\-context multitasks\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 3639–3664\.
- Brown et al\. \(2020\)Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others\. 2020\.Language models are few\-shot learners\.*Advances in neural information processing systems*, 33:1877–1901\.
- Cattan et al\. \(2025\)Arie Cattan, Alon Jacovi, Ori Ram, Jonathan Herzig, Roee Aharoni, Sasha Goldshtein, Eran Ofek, Idan Szpektor, and Avi Caciularu\. 2025\.Dragged into conflicts: Detecting and addressing conflicting sources in search\-augmented llms\.*arXiv preprint arXiv:2506\.08500*\.
- Charakorn et al\. \(2026\)Rujikorn Charakorn, Edoardo Cetin, Shinnosuke Uesaka, and Robert Tjarko Lange\. 2026\.Doc\-to\-lora: Learning to instantly internalize contexts\.*arXiv preprint arXiv:2602\.15902*\.
- Chen et al\. \(2024\)Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, and 1 others\. 2024\.Lifelong knowledge editing for llms with retrieval\-augmented continuous prompt learning\.In*Proceedings of the 2024 conference on empirical methods in natural language processing*, pages 13565–13580\.
- Chen et al\. \(2023\)Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian\. 2023\.Extending context window of large language models via positional interpolation\.*arXiv preprint arXiv:2306\.15595*\.
- Cobbe et al\. \(2021\)Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others\. 2021\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*\.
- Dasigi et al\. \(2021\)Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner\. 2021\.A dataset of information\-seeking questions and answers anchored in research papers\.In*Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4599–4610\.
- Ding et al\. \(2023\)Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng, and Furu Wei\. 2023\.Longnet: Scaling transformers to 1,000,000,000 tokens\.*arXiv preprint arXiv:2307\.02486*\.
- Fang et al\. \(2025\)Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat\-Seng Chua\. 2025\.Alphaedit: Null\-space constrained knowledge editing for language models\.In*International Conference on Learning Representations*, volume 2025, pages 16366–16396\.
- Gao et al\. \(2023\)Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang\. 2023\.Retrieval\-augmented generation for large language models: A survey\.*arXiv preprint arXiv:2312\.10997*\.
- Gu et al\. \(2024\)Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar\-Lezama, Gabriel Synnaeve, and Sida I Wang\. 2024\.Cruxeval: A benchmark for code reasoning, understanding and execution\.*arXiv preprint arXiv:2401\.03065*\.
- Guo et al\. \(2025\)Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, and 1 others\. 2025\.Deepseek\-r1: Incentivizing reasoning capability in llms via reinforcement learning\.*arXiv preprint arXiv:2501\.12948*\.
- Guu et al\. \(2020\)Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang\. 2020\.Retrieval augmented language model pre\-training\.In*International conference on machine learning*, pages 3929–3938\. PMLR\.
- Ho et al\. \(2020\)Xanh Ho, Anh\-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa\. 2020\.Constructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.In*Proceedings of the 28th International Conference on Computational Linguistics*, pages 6609–6625\.
- Hsieh et al\. \(2024\)Cheng\-Yu Hsieh, Yung\-Sung Chuang, Chun\-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen\-Yu Lee, Ranjay Krishna, and 1 others\. 2024\.Found in the middle: Calibrating positional attention bias improves long context utilization\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 14982–14995\.
- Hu et al\. \(2022\)Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\. 2022\.[LoRA: Low\-rank adaptation of large language models](https://openreview.net/forum?id=nZeVKeeFYf9)\.In*International Conference on Learning Representations*\.
- Jang et al\. \(2022\)Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, and Minjoon Seo\. 2022\.Temporalwiki: A lifelong benchmark for training and evaluating ever\-evolving language models\.In*Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, pages 6237–6250\.
- Jiang et al\. \(2025\)Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Mingyang Wan, Guojun Ma, Xiang Wang, Xiangnan He, and Tat\-Seng Chua\. 2025\.[Anyedit: Edit any knowledge encoded in language models](https://openreview.net/forum?id=aJIoBur0Ef)\.In*Forty\-second International Conference on Machine Learning*\.
- Jiang et al\. \(2024\)Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, and 1 others\. 2024\.Learning to edit: Aligning llms with knowledge editing\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 4689–4705\.
- Kortukov et al\. \(2024\)Evgenii Kortukov, Alexander Rubinstein, Elisa Nguyen, and Seong Joon Oh\. 2024\.[Studying large language model behaviors under context\-memory conflicts with real documents](https://openreview.net/forum?id=xm8zYRfrqE)\.In*First Conference on Language Modeling*\.
- Lewis et al\. \(2020\)Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, and 1 others\. 2020\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.*Advances in neural information processing systems*, 33:9459–9474\.
- Li et al\. \(2025a\)Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, and 1 others\. 2025a\.Scbench: A kv cache\-centric analysis of long\-context methods\.In*International Conference on Learning Representations*, volume 2025, pages 66063–66093\.
- Li et al\. \(2025b\)Zherui Li, Houcheng Jiang, Hao Chen, Baolong Bi, Zhenhong Zhou, Fei Sun, Junfeng Fang, and Xiang Wang\. 2025b\.[Reinforced lifelong editing for language models](https://openreview.net/forum?id=1jUXprrfcb)\.In*Forty\-second International Conference on Machine Learning*\.
- Lin \(2004\)Chin\-Yew Lin\. 2004\.Rouge: A package for automatic evaluation of summaries\.In*Text summarization branches out*, pages 74–81\.
- Lin et al\. \(2019\)Kevin Lin, Oyvind Tafjord, Peter Clark, and Matt Gardner\. 2019\.Reasoning over paragraph effects in situations\.In*Proceedings of the 2nd Workshop on Machine Reading for Question Answering*, pages 58–62\.
- Liu et al\. \(2024\)Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang\. 2024\.Lost in the middle: How language models use long contexts\.*Transactions of the association for computational linguistics*, 12:157–173\.
- Liu et al\. \(2025\)Wei Liu, Haomei Xu, Bingqing Liu, Zhiying Deng, Haozhao Wang, Jun Wang, Ruixuan Li, Yee Whye Teh, and Wee Sun Lee\. 2025\.Is model editing built on sand? revealing its illusory success and fragile foundation\.*arXiv preprint arXiv:2510\.00625*\.
- Marjanović et al\. \(2024\)Sara Vera Marjanović, Haeun Yu, Pepa Atanasova, Maria Maistro, Christina Lioma, and Isabelle Augenstein\. 2024\.Dynamicqa: Tracing internal knowledge conflicts in language models\.In*Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 14346–14360\.
- Meng et al\. \(2022\)Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov\. 2022\.Locating and editing factual associations in gpt\.*Advances in neural information processing systems*, 35:17359–17372\.
- Meng et al\. \(2023\)Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau\. 2023\.[Mass\-editing memory in a transformer](https://openreview.net/forum?id=MkbcAHIYgyS)\.In*The Eleventh International Conference on Learning Representations*\.
- Mitchell et al\. \(2022\)Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning\. 2022\.[Fast model editing at scale](https://openreview.net/forum?id=0DcZxeWfOPt)\.In*International Conference on Learning Representations*\.
- OpenAI \(2026\)OpenAI\. 2026\.GPT\-5\.5 system card\.[https://openai\.com/index/gpt\-5\-5\-system\-card/](https://openai.com/index/gpt-5-5-system-card/)\.Published April 23, 2026; updated April 24, 2026\.
- Packer et al\. \(2023\)Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G Patil, Ion Stoica, and Joseph E Gonzalez\. 2023\.Memgpt: Towards llms as operating systems, 2024\.*URL https://arxiv\. org/abs/2310\.08560*, 7\.
- Park et al\. \(2023\)Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein\. 2023\.Generative agents: Interactive simulacra of human behavior\.In*Proceedings of the 36th annual acm symposium on user interface software and technology*, pages 1–22\.
- Rajpurkar et al\. \(2016\)Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang\. 2016\.Squad: 100,000\+ questions for machine comprehension of text\.In*Proceedings of the 2016 conference on empirical methods in natural language processing*, pages 2383–2392\.
- Shi et al\. \(2024\)Dan Shi, Renren Jin, Tianhao Shen, Weilong Dong, Xinwei Wu, and Deyi Xiong\. 2024\.Ircan: Mitigating knowledge conflicts in llm generation via identifying and reweighting context\-aware neurons\.*Advances in Neural Information Processing Systems*, 37:4997–5024\.
- Snell et al\. \(2022\)Charlie Snell, Dan Klein, and Ruiqi Zhong\. 2022\.Learning by distilling context\.*arXiv preprint arXiv:2209\.15189*\.
- Team et al\. \(2024\)Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others\. 2024\.Gemma 2: Improving open language models at a practical size\.*arXiv preprint arXiv:2408\.00118*\.
- Tian et al\. \(2026\)Bowen Tian, Caixue He, Jiemin Wu, Jingying Wang, Wenshuo Chen, Zexi Li, and Yutao Yue\. 2026\.[Anyedit\+\+: Adaptive long\-form knowledge editing via bayesian surprise](https://openreview.net/forum?id=W6qfbvysDh)\.In*Forty\-third International Conference on Machine Learning*\.
- Wallat et al\. \(2026\)Jonas Wallat, Wolfgang Nejdl, and Sandipan Sikdar\. 2026\.When facts change: Temporal knowledge conflict resolution in llms\.In*Findings of the Association for Computational Linguistics: ACL 2026*, pages 2154–2184\.
- Wang et al\. \(2024\)Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li\. 2024\.Knowledge editing for large language models: A survey\.*ACM Computing Surveys*, 57\(3\):1–37\.
- Wei et al\. \(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others\. 2022\.Chain\-of\-thought prompting elicits reasoning in large language models\.*Advances in neural information processing systems*, 35:24824–24837\.
- Xie et al\. \(2024\)Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su\. 2024\.Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts\.In*International Conference on Learning Representations*, volume 2024, pages 35623–35646\.
- Xu et al\. \(2024\)Rongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu\. 2024\.Knowledge conflicts for llms: A survey\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 8541–8565\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others\. 2025\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*\.
- Yao et al\. \(2023\)Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang\. 2023\.Editing large language models: Problems, methods, and opportunities\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 10222–10240\.
- Zhong et al\. \(2024\)Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang\. 2024\.Memorybank: Enhancing large language models with long\-term memory\.In*Proceedings of the AAAI conference on artificial intelligence*, volume 38, pages 19724–19731\.

## Appendix ALLM Usage Statement

We utilized Large Language Models to assist with the writing and polishing of the manuscript, including improvements to grammar, clarity, conciseness, and word choice\. As described in Section[3](https://arxiv.org/html/2609.14168#S3), LLMs were also used for data construction, quality verification, and automatic evaluation\. However, they were not used to formulate the core research ideas or design the proposed methodology\.

## Appendix BDetailed Setup

In this section, we provide additional details regarding the models, benchmark, datasets, and baseline methods used in our experiments\.

### B\.1Models

We used two base language models and their corresponding pretrained D2L hypernetworks:

- •Qwen3\-4B\-Instruct\-2507\.Qwen3\-4B\-Instruct\-2507is a 4B\-parameter instruction\-tuned autoregressive language model from theQwen3family\([Yang et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib48)\)\. It served as our primary backbone and provided a relatively strong instruction\-following capability at a moderate model scale\. We downloaded theQwen/Qwen3\-4B\-Instruct\-2507checkpoint from Hugging Face\.
- •Gemma\-2\-2B\-it\.Gemma\-2\-2B\-itis a 2B\-parameter instruction\-tuned autoregressive language model from theGemma 2family\([Team et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib41)\)\. Compared withQwen3\-4B\-Instruct\-2507, it provided a smaller and architecturally distinct backbone for evaluating the robustness of our method across model families and scales\. We downloaded thegoogle/gemma\-2\-2b\-itcheckpoint from Hugging Face\.
- •Qwen\-4BD2L hypernetwork\.TheQwen\-4BD2L hypernetwork is a pretrained context\-to\-parameter model that maps textual context directly into LoRA parameters conditioned on the target backbone,Qwen3\-4B\-Instruct\-2507\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)\. It enables long\-form context information to be compressed into a lightweight parametric representation without directly modifying the frozen backbone parameters\. We used theqwen\_4b\_d2l/checkpoint\-20000checkpoint after 20K training steps\.
- •Gemma\-2BD2L hypernetwork\.TheGemma\-2BD2L hypernetwork performs the same context\-to\-LoRA parameterization for the smallerGemma\-2\-2B\-itbackbone\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)\. Its backbone\-specific parameter generation allowed us to evaluate D2L consistently across model families and at a reduced parameter scale\. We used thegemma\_2b\_d2l/checkpoint\-20000checkpoint, matching the identical training budget used forQwen\.

SincePLUMEis a training\-free approach, both the base models and the D2L hypernetworks remained frozen throughout evaluation, without requiring any additional parameter updates\.

### B\.2Benchmark and Datasets

##### Benchmark\.

Table[3](https://arxiv.org/html/2609.14168#A2.T3)summarizes the detailed statistics ofMUSE\-Bench, which comprises2,944 contextsand12,932 QA pairs\. Averaged across the five constituent datasets and weighted by context frequency, the mean length of the full interaction history is3,811\.9 tokens, as measured by theQwen3\-4B\-Instruct\-2507tokenizer\. Per\-dataset averages vary considerably, ranging from 328\.3 tokens on SQuAD to 18,539\.8 tokens on 2WikiMultihopQA, underscoring the substantial diversity in context length spanned by the benchmark\.

##### Datasets\.

In total, we employed seven datasets spanning factual and reasoning\-intensive settings\. The first five constituteMUSE\-Bench, while GSM8K and CRUXEval were additionally incorporated to examine whetherPLUME, together with the compared baseline and oracle approaches, generalizes beyond factual QA to more challenging mathematical and code\-reasoning tasks\. For all datasets, we constructed and annotated samples under our context\-update setting, thereby ensuring that every method was evaluated under an identical protocol across domains\.

- •SQuAD\.The Stanford Question Answering Dataset \(SQuAD\)\([Rajpurkar et al\., 2016](https://arxiv.org/html/2609.14168#bib.bib38)\)is a widely used extractive question\-answering benchmark comprising questions posed over Wikipedia passages, with answers grounded in corresponding textual spans\. Following our benchmark construction procedure, we annotated selected samples with context updates and their corresponding question\. We adopted therajpurkar/squaddataset from Hugging Face as the underlying data source for this construction\. A representative annotated example is illustrated in Figure[7](https://arxiv.org/html/2609.14168#A2.F7)\.
- •ROPES\.Reasoning Over Paragraph Effects in Situations \(ROPES\)\([Lin et al\., 2019](https://arxiv.org/html/2609.14168#bib.bib28)\)evaluates whether models can apply causal or qualitative relationships described in background passages to reason about novel, previously unseen situations\. We annotated selected samples by updating answer\-relevant information and its associated dependencies, while preserving overall contextual consistency\. We adopted theallenai/ropesdataset from Hugging Face as the underlying data source\. A representative annotated example is illustrated in Figure[8](https://arxiv.org/html/2609.14168#A2.F8)\.
- •2WikiMultihopQA\.2WikiMultihopQA\([Ho et al\., 2020](https://arxiv.org/html/2609.14168#bib.bib17)\)is a multi\-hop question\-answering dataset that requires compositional reasoning over multiple supporting facts drawn from interlinked Wikipedia passages\. We annotated selected samples by modifying answer\-supporting facts and consistently propagating the corresponding updates throughout the associated reasoning dependencies\. We adopted thexanhho/2WikiMultihopQAdataset from Hugging Face as the underlying data source\. A representative annotated example is illustrated in Figure[10](https://arxiv.org/html/2609.14168#A2.F10)\.
- •MultiFieldQA\-en\.MultiFieldQA\-en is a single\-document question\-answering dataset fromLongBench\([Bai et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib2)\), containing long\-form contexts collected from diverse domains and disciplines\. We annotated selected samples with context updates while preserving the remaining information and structure of the original long context\. We used themultifieldqa\_ensubset ofTHUDMLongBenchfrom Hugging Face\. A representative annotated example is shown in Figure[9](https://arxiv.org/html/2609.14168#A2.F9)\.
- •QASPER\.QASPER\([Dasigi et al\., 2021](https://arxiv.org/html/2609.14168#bib.bib10)\)is a question\-answering dataset built upon scientific papers, where answering a question may require synthesizing evidence distributed across multiple sections\. We annotated selected samples by updating answer\-relevant evidence and its dependent information while preserving the overall structure and coherence of the original context\. We used theallenai/qasperdataset from Hugging Face\. A representative annotated example is shown in Figure[11](https://arxiv.org/html/2609.14168#A2.F11)\.
- •GSM8K\.Grade School Math 8K \(GSM8K\)\([Cobbe et al\., 2021](https://arxiv.org/html/2609.14168#bib.bib9)\)contains linguistically diverse grade\-school mathematical word problems that typically require multi\-step arithmetic reasoning to solve correctly\. We annotated selected samples under our context\-update setting to evaluate whether the same update mechanism generalized beyond question answering to mathematical reasoning\. We used theopenai/gsm8kdataset from Hugging Face\.
- •CRUXEval\.CRUXEval\([Gu et al\., 2024](https://arxiv.org/html/2609.14168#bib.bib14)\)is a code reasoning benchmark consisting of short Python functions paired with input–output examples, covering both input and output prediction tasks\. We annotated selected samples under our context\-update setting to evaluate generalization to structured code reasoning\. We used thecruxeval\-org/cruxevaldataset from Hugging Face\.

![Refer to caption](https://arxiv.org/html/2609.14168v1/1-squad.png)Figure 7:RepresentativeMUSE\-Benchexample derived from SQuAD\.The example illustrates the original context, its subsequent update, and the associated question–answer annotations\.![Refer to caption](https://arxiv.org/html/2609.14168v1/2-ropes.png)Figure 8:RepresentativeMUSE\-Benchexample derived from ROPES\.The example illustrates a context update while preserving the causal reasoning structure of the original instance\.![Refer to caption](https://arxiv.org/html/2609.14168v1/3-mfqa.png)Figure 9:RepresentativeMUSE\-Benchexample derived from MultiFieldQA\-en\.The example illustrates an update within a long\-form context while retaining its original document structure\.![Refer to caption](https://arxiv.org/html/2609.14168v1/4-2wiki.png)Figure 10:RepresentativeMUSE\-Benchexample derived from 2WikiMultihopQA\.The example illustrates a context update with consistently revised multi\-hop dependencies\.![Refer to caption](https://arxiv.org/html/2609.14168v1/5-qasper.png)Figure 11:RepresentativeMUSE\-Benchexample derived from QASPER\.The example illustrates an update to evidence distributed across a scientific document\.Table[3](https://arxiv.org/html/2609.14168#A2.T3)reports the number of contexts, the number of question–answer pairs, and the average full\-context length for the five datasets constitutingMUSE\-Bench\. Context length is measured using theQwen3\-4B\-Instruct\-2507tokenizer\.

\\SetRowbg=black\!6DatasetContextsQA pairsAvg\. context tokensSQuAD2,06710,570328\.3ROPES2031,688434\.62WikiMultihopQA30030018,539\.8MultiFieldQA\-en15015014,068\.8QASPER22422412,424\.7MUSE\-Bench2,94412,9323,811\.9GSM8K1,3191,319123\.0CRUXEval80080086\.3Table 3:Dataset statistics forMUSE\-Benchand the cross\-task evaluation sets\.We report the numbers of contexts and question–answer pairs and the mean full\-history length measured with theQwen3\-4B\-Instruct\-2507tokenizer\.

### B\.3Baselines

We comparedPLUMEwith the following baselines:

- •Base Model w/o Context\.The base model receives only the query and generates the response without access to any contextual information\. This setting measures the model’s parametric knowledge and serves as a lower\-bound reference for context\-dependent tasks\.
- •Base Model w/ Context \(oracle\)\.The base model receives the complete context history together with the query at inference time\. Unlike parameterized methods, it directly accesses the textual context during generation and therefore serves as a full\-context reference\.
- •D2L\.Doc\-to\-LoRA \(D2L\)\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)parameterizes textual context into LoRA adapters using a pretrained hypernetwork\. We used the official pretrained hypernetwork checkpoint corresponding to each backbone\. For contexts longer than the supported input length, we divided the full context history into chunks of at most 8,192 tokens and followed the original iterative layer\-wise adapter generation procedure\. The generated rank\-8 LoRA adapters were applied to the MLP down\-projection layer of each Transformer block\. The hypernetwork is trained across contexts with the same teacher–student principle as context distillation: a context\-conditioned teacher supplies the target behavior, while the generated adapter enables the student to reproduce that behavior without receiving the context at query time\. Architecturally, D2L aggregates variable\-length context representations with a Perceiver\-style module and uses layer\-specific output heads to generate the LoRA factors\. This amortizes the per\-context optimization required by CD into a single hypernetwork pass for a previously unseen context\.
- •Context Distillation \(CD\)\.Context Distillation \(CD\)\([Snell et al\., 2022](https://arxiv.org/html/2609.14168#bib.bib40)\)transfers information from a context\-conditioned teacher into model parameters through supervised distillation\. For each context, we generated 20 auxiliary question–answer pairs in four rounds of five pairs and used them as distillation examples\. We optimized a newly initialized rank\-8 LoRA adapter with LoRA alpha 16 for 300 epochs using a learning rate of10−410^\{\-4\}\.
- •Other oracle references\.CD \(oracle\) uses the target evaluation query as its sole distillation query and the response generated by the full\-context teacher as its target\. A fresh query\-specific rank\-8 adapter is optimized for 300 iterations with the same LoRA alpha and learning rate as CD, and is then evaluated on that same query without textual context\. D2L \(oracle\) parameterizes only the gold supporting context associated with the target query rather than the complete accumulated history\. Both settings use query\-specific information unavailable to the non\-oracle methods and are therefore upper\-bound references, not directly comparable deployment settings\.
- •AnyEdit\.AnyEdit\([Jiang et al\., 2025](https://arxiv.org/html/2609.14168#bib.bib21)\)is a knowledge\-editing baseline that directly modifies model parameters to incorporate updated information\. We used its AlphaEdit\-ARE configuration\. Edits were applied to the MLP down\-projection layers of Transformer blocks 4–8\. We divided the full context history into chunks of at most 8,192 tokens and used an edit\-window size of 50 without overlap, 25 gradient steps per edit window, a learning rate of0\.50\.5, a weight decay of0\.0010\.001, a clamp\-norm factor of44, and anL2L\_\{2\}coefficient of1010\.

Except for Base Model w/ Context \(oracle\), all parameterized methods answered queries without receiving the full textual context again at inference time\. Unless otherwise specified, all methods used the same base model and generation configuration\.

### B\.4Evaluation

Following Section[3\.3](https://arxiv.org/html/2609.14168#S3.SS3), we evaluated answer coverage, semantic correctness, final\-answer agreement, and locality\. All answer\-quality metrics were first computed for individual queries and then averaged within each dataset\. When reporting an overall result across datasets, we used themacro\-average of the corresponding per\-dataset scoresso that datasets with more question–answer pairs did not dominate the comparison\.

##### ROUGE\-L Overlap Metrics\.

For each queryii, letri=\(ri,1,…,ri,mi\)r\_\{i\}=\(r\_\{i,1\},\\ldots,r\_\{i,m\_\{i\}\}\)be the tokenized reference answer, and lety^i=\(y^i,1,…,y^i,ni\)\\hat\{y\}\_\{i\}=\(\\hat\{y\}\_\{i,1\},\\ldots,\\hat\{y\}\_\{i,n\_\{i\}\}\)be the tokenized model output\. Denote the length of their longest common subsequence by

Li=LCS⁡\(ri,y^i\)\.L\_\{i\}=\\operatorname\{LCS\}\(r\_\{i\},\\hat\{y\}\_\{i\}\)\.\(13\)ROUGE\-L Recall measures how much of the reference answer is covered by the model output, whereas ROUGE\-L Precision measures how much of the model output is supported by the reference answer:

RiL=Limi,PiL=Lini\.R\_\{i\}^\{\\mathrm\{L\}\}=\\frac\{L\_\{i\}\}\{m\_\{i\}\},\\qquad P\_\{i\}^\{\\mathrm\{L\}\}=\\frac\{L\_\{i\}\}\{n\_\{i\}\}\.\(14\)Their harmonic mean gives ROUGE\-L F1:

F1,iL=2​PiL​RiLPiL\+RiL,F\_\{1,i\}^\{\\mathrm\{L\}\}=\\frac\{2P\_\{i\}^\{\\mathrm\{L\}\}R\_\{i\}^\{\\mathrm\{L\}\}\}\{P\_\{i\}^\{\\mathrm\{L\}\}\+R\_\{i\}^\{\\mathrm\{L\}\}\},\(15\)where the score is defined as zero when the denominator is zero\. These overlap metrics are computed with therouge\-scoreimplementation and stemming enabled\. Because the longest common subsequence preserves token order without requiring matched tokens to be contiguous, these metrics accommodate short intervening phrases while still rewarding structurally consistent answer overlap\. Among the three, ROUGE\-L Recall serves as our primary lexical metric, as reference answers are generally concise and answer coverage is more important than matching the reference length or wording exactly\. This choice is further motivated by a systematic mismatch between instruction\-tuned models and span\-style references: models such asQwenandGemmatypically produce complete, fluent sentences rather than terse answer spans, so even semantically correct responses tend to be substantially longer than the reference\. Under this mismatch, Precision and F1 disproportionately penalize verbose yet correct generations, making them less reliable as standalone indicators of answer quality\. We nonetheless retain Precision and F1 as complementary diagnostics: Precision additionally penalizes unrelated or unnecessarily long generations, while F1 summarizes the balance between coverage and conciseness\. Together, these variants allow us to distinguish omission of reference content from the inclusion of extraneous material\. For a datasetddcontaining query set𝒬d\\mathcal\{Q\}\_\{d\}, each metricM∈\{RL,PL,F1L\}M\\in\\\{R^\{\\mathrm\{L\}\},P^\{\\mathrm\{L\}\},F\_\{1\}^\{\\mathrm\{L\}\}\\\}is aggregated as

Md=1\|𝒬d\|​∑i∈𝒬dMi\.M\_\{d\}=\\frac\{1\}\{\|\\mathcal\{Q\}\_\{d\}\|\}\\sum\_\{i\\in\\mathcal\{Q\}\_\{d\}\}M\_\{i\}\.\(16\)

##### Locality\.

Locality measures whether incorporating an update damages information that should remain unchanged\. For datasetdd, let𝒰d⊆𝒬d\\mathcal\{U\}\_\{d\}\\subseteq\\mathcal\{Q\}\_\{d\}denote the queries whose correct answers are unaffected by the update\. Because their original and current reference answers are identical, any reduction in answer coverage after context parameterization indicates interference with unaffected information\. We computed Locality as the mean ROUGE\-L Recall on this unaffected\-query subset:

Localityd=1\|𝒰d\|​∑i∈𝒰dRiL\.\\operatorname\{Locality\}\_\{d\}=\\frac\{1\}\{\|\\mathcal\{U\}\_\{d\}\|\}\\sum\_\{i\\in\\mathcal\{U\}\_\{d\}\}R\_\{i\}^\{\\mathrm\{L\}\}\.\(17\)Update\-affected queries are excluded from𝒰d\\mathcal\{U\}\_\{d\}\. Our locality evaluation uses 50 annotated SQuAD contexts containing 754 queries in total\. Exactly one answer is changed in each context, leaving 704 unaffected queries for computing Eq\.[17](https://arxiv.org/html/2609.14168#A2.E17)\. Thus, a higher Locality score means that the method more faithfully preserves answers supported by facts that were not modified, while a lower score indicates collateral interference introduced by context parameterization, parameter editing, or decoding\. Locality is reported on a00–100100scale in the figures\.

For D2 Semantic Equivalence and D3 Final\-Answer Agreement, we usedGPT\-5\.6\-Solunder unified evaluation prompts and criteria\. For efficiency, we separately measured latency and peak GPU memory during context parameterization and query generation, as detailed in Appendix[D\.6](https://arxiv.org/html/2609.14168#A4.SS6)\.

##### Evaluation Rubric\.

We usedGPT\-5\.6\-Solto evaluate each generated answer under two independent rubrics corresponding to D2 Semantic Equivalence and D3 Final\-Answer Agreement\. Each evaluation instance contained the query, the current reference answer determined by the full context history, and the model output\. The judge did not receiveColdC\_\{\\mathrm\{old\}\},CupdateC\_\{\\mathrm\{update\}\}, orCfullC\_\{\\mathrm\{full\}\}\. We used a temperature of 0, allowed at most 300 output tokens, and required a JSON object\. Each rubric was evaluated in a separate call\. The complete evaluation prompts are provided in Appendix[I](https://arxiv.org/html/2609.14168#A9)\.

##### Judgment Output and Aggregation\.

The two rubrics were evaluated independently, each targeting a distinct facet of answer quality\. For each rubric, the judge returned a binaryyes/nodecision, a brief rationale justifying the verdict\. A query received an overall LLM\-as\-a\-Judge score of 1only when both D2 Semantic Equivalence and D3 Final\-Answer Agreement were simultaneously satisfied; otherwise, it received a score of 0\. We report the mean binary score across queries within each dataset\.

### B\.5Implementation Details

All experiments were conducted on two NVIDIA A800 GPUs using bfloat16 precision\.

#### B\.5\.1Exact LoRA Composition and Rank

As defined in Section[2](https://arxiv.org/html/2609.14168#S2), each context adapter represents an effective updateΔ​WC=BC​AC\\Delta\\mathrm\{W\}\_\{C\}=B\_\{C\}A\_\{C\}, whereAC∈ℝrC×dinA\_\{C\}\\in\\mathbb\{R\}^\{r\_\{C\}\\times d\_\{\\mathrm\{in\}\}\}andBC∈ℝdout×rCB\_\{C\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times r\_\{C\}\}\. Writingcfull=α\+βc\_\{\\mathrm\{full\}\}=\\alpha\+\\betaandcold=−βc\_\{\\mathrm\{old\}\}=\-\\beta, Eq\.[6](https://arxiv.org/html/2609.14168#S4.E6)becomes

Δ​Wg=cfull​Bfull​Afull\+cold​Bold​Aold\.\\Delta\\mathrm\{W\}\_\{g\}=c\_\{\\mathrm\{full\}\}B\_\{\\mathrm\{full\}\}A\_\{\\mathrm\{full\}\}\+c\_\{\\mathrm\{old\}\}B\_\{\\mathrm\{old\}\}A\_\{\\mathrm\{old\}\}\.\(18\)We represent this sum exactly with the concatenated factors

Ag\\displaystyle A\_\{g\}=\[AfullAold\],\\displaystyle=\\begin\{bmatrix\}A\_\{\\mathrm\{full\}\}\\\\ A\_\{\\mathrm\{old\}\}\\end\{bmatrix\},\(19\)Bg\\displaystyle B\_\{g\}=\[cfull​Bfullcold​Bold\]\.\\displaystyle=\\begin\{bmatrix\}c\_\{\\mathrm\{full\}\}B\_\{\\mathrm\{full\}\}&c\_\{\\mathrm\{old\}\}B\_\{\\mathrm\{old\}\}\\end\{bmatrix\}\.Consequently,Bg​AgB\_\{g\}A\_\{g\}is exactly Eq\.[18](https://arxiv.org/html/2609.14168#A2.E18), without element\-wise subtraction of LoRA factors, dense\-weight merging, or recompression to rank 8\. The stored factorization width is thereforerg=rfull\+roldr\_\{g\}=r\_\{\\mathrm\{full\}\}\+r\_\{\\mathrm\{old\}\}; when both inputs are single rank\-8 adapters,rg=16r\_\{g\}=16\. The algebraic matrix rank satisfiesrank⁡\(Δ​Wg\)≤rg\\operatorname\{rank\}\(\\Delta\\mathrm\{W\}\_\{g\}\)\\leq r\_\{g\}and may be lower, sorgr\_\{g\}refers to the implemented LoRA factorization width rather than a claim that the effective matrix necessarily has full rank\.

For a contextCCdivided intonCn\_\{C\}chunks, D2L first generates one rank\-r0r\_\{0\}adapter per chunk \(r0=8r\_\{0\}=8in our experiments\) and aggregates the chunk updates by the same rank\-wise concatenation\. Thus,rC=nC​r0r\_\{C\}=n\_\{C\}r\_\{0\}and the global factorization width isrg=\(nfull\+nold\)​r0r\_\{g\}=\(n\_\{\\mathrm\{full\}\}\+n\_\{\\mathrm\{old\}\}\)r\_\{0\}\. If the D2L checkpoint enables its learned rank\-r0r\_\{0\}LoRA bias block, that block is appended once to each context adapter, givingrC=\(nC\+1\)​r0r\_\{C\}=\(n\_\{C\}\+1\)r\_\{0\}andrg=\(nfull\+nold\+2\)​r0r\_\{g\}=\(n\_\{\\mathrm\{full\}\}\+n\_\{\\mathrm\{old\}\}\+2\)r\_\{0\}instead\. These concatenated factors are applied directly during the LoRA forward pass; the corresponding dense matrices are never materialized\. Accordingly, the adapter storage and LoRA multiplication cost scale linearly with the implemented factorization widthrgr\_\{g\}\.

## Appendix CHyperparameters

We separated the backbone and checkpoint configuration, the context\-parameterization execution mode, and thePLUME\-specific inference hyperparameters\. We reported inference settings only; the training configuration of the released D2L hypernetworks was not included\. All models and hypernetworks remained frozen and were loaded in bfloat16 precision\. Table[4](https://arxiv.org/html/2609.14168#A3.T4)summarizes the backbone and D2L checkpoint configurations\.

\\SetRowbg=black\!6Base modelD2L checkpointBase rankTargetQwen3\-4BQwen D2L\-20k8MLP downGemma\-2\-2BGemma D2L\-20k8MLP downTable 4:Backbone and D2L checkpoint configurations\.All models use bfloat16 precision, and the generated rank\-8 LoRA adapters are applied to the MLP down\-projection layers\.##### Context parameterization\.

For contexts exceeding the supported input length, we preserved the complete context history by dividing it into near\-equal contiguous chunks of at most 8,192 tokens\. We used the Iterative D2L inference mode for all reported experiments\. Table[5](https://arxiv.org/html/2609.14168#A3.T5)also describes the available Batched mode for completeness; no experimental metrics are reported for that mode\. InIterativemode, the official layer\-wise procedure processed the Transformer\-layer representations sequentially\. InBatchedmode, the layer dimension was folded into the batch dimension and the corresponding representations were processed jointly\. This execution\-mode choice was independent of context chunking: both modes support the same complete\-context chunks and generate rank\-8 LoRA adapters with the same structure\. Here, 8 denotes the base rank of each chunk adapter, not the width of the final composed adapter\. Chunk aggregation and global update composition increase the factorization width as detailed in Appendix[B\.5\.1](https://arxiv.org/html/2609.14168#A2.SS5.SSS1)\. ForPLUME,the query was used only to activate memory evidence and was never passed to the D2L hypernetwork\.

\\SetRowbg=black\!6SettingIterativeBatchedContext coverageComplete historyComplete historyMaximum chunk length8,1928,192Chunk constructionNear\-equal, contiguousNear\-equal, contiguousLayer executionSequentialJoint batched passBase LoRA rank88Target moduledown\_projdown\_projHypernetwork inputContext/evidence onlyContext/evidence onlyTable 5:D2L inference modes for context parameterization\.Iterative and batched execution differ only in how Transformer\-layer representations are processed; both support complete\-context chunking and the same base\-rank configuration\. Only Iterative results are reported\.
##### PLUMEand generation settings\.

Table[6](https://arxiv.org/html/2609.14168#A3.T6)summarizes the method\-specific and generation hyperparameters\.PLUMEseparately parameterizesColdC\_\{\\mathrm\{old\}\}andCfullC\_\{\\mathrm\{full\}\}and constructs the global update LoRA withα=1\\alpha=1andβ=0\.75\\beta=0\.75in Eq\.[6](https://arxiv.org/html/2609.14168#S4.E6)\. Memory units are formed independently of the 8,192\-token full\-context chunks while preserving sentence and paragraph boundaries whenever possible\.PLUMEactivates one memory unit using the relevance score defined below and a recency margin ofδ=0\.5\\delta=0\.5\. We usedλmax=1\\lambda\_\{\\max\}=1andτ=0\.3\\tau=0\.3for adaptive memory evidence decoding, greedy decoding with a temperature of 0, and at most 256 new tokens for both backbones and all comparison methods\.

For the component ablations in Figure[5](https://arxiv.org/html/2609.14168#S5.F5), thememory\-onlyvariant removed the global update representation while retaining the same memory segmentation, query\-dependent evidence activation, and hypernetwork parameterizationΔ​We​\(q\)=Hϕ​\(m⋆​\(q\)\)\\Delta\\mathrm\{W\}\_\{e\}\(q\)=H\_\{\\phi\}\(m^\{\\star\}\(q\)\); it generated predictions using the resulting evidence adapter alone\. Theglobal\-onlyvariant retainedΔ​Wg\\Delta\\mathrm\{W\}\_\{g\}while removing evidence activation, evidence parameterization, and adaptive fusion\. In both variants, as in the complete method, no memory\-evidence text was provided to the target language model\.

For lexical routing, text is lowercased and tokenized with the regular expression\[a\-z0\-9\]\+\. LetQQbe the set of query terms,ci,tc\_\{i,t\}the count of termttin memory unitmim\_\{i\}, andwtw\_\{t\}its document\-level inverse\-frequency weight\. Withk1=1\.5k\_\{1\}=1\.5andW=∑t∈Qwt\\mathrm\{W\}=\\sum\_\{t\\in Q\}w\_\{t\}, the lexical score is

slex​\(q,mi\)\\displaystyle s\_\{\\mathrm\{lex\}\}\(q,m\_\{i\}\)=12∑t:ci,t\>0wtW\+12FiFi\+W,\\displaystyle=\\frac\{1\}\{2\}\\frac\{\\sum\_\{t:c\_\{i,t\}\>0\}w\_\{t\}\}\{\\mathrm\{W\}\}\+\\frac\{1\}\{2\}\\frac\{F\_\{i\}\}\{F\_\{i\}\+\\mathrm\{W\}\},\(20\)Fi\\displaystyle F\_\{i\}=∑t:ci,t\>0wtci,t​\(k1\+1\)ci,t\+k1\.\\displaystyle=\\sum\_\{t:c\_\{i,t\}\>0\}w\_\{t\}\\frac\{c\_\{i,t\}\(k\_\{1\}\+1\)\}\{c\_\{i,t\}\+k\_\{1\}\}\.Letsmax=maxi⁡slex​\(q,mi\)s\_\{\\max\}=\\max\_\{i\}s\_\{\\mathrm\{lex\}\}\(q,m\_\{i\}\)\. We form the candidate set

ℐδ=\{i\|slex​\(q,mi\)≥smax−δ\}\.\\mathcal\{I\}\_\{\\delta\}=\\left\\\{i\\,\\middle\|\\,s\_\{\\mathrm\{lex\}\}\(q,m\_\{i\}\)\\geq s\_\{\\max\}\-\\delta\\right\\\}\.\(21\)We useδ=0\.5\\delta=0\.5and activate the temporally latest unit inℐδ\\mathcal\{I\}\_\{\\delta\}\. Temporal order follows each unit’s position inCfullC\_\{\\mathrm\{full\}\}\. Thus, recency takes precedence only among units whose relevance scores are withinδ\\deltaof the maximum\. If the query\-term set is empty, all units receive a score of zero and the temporally latest unit is activated\.

\\SetRowbg=black\!6SettingPLUMEMaximum context\-chunk length8,192Evidence activationRelevance with recency marginNumber of activated units1Recency marginδ\\delta0\.5α\\alpha/β\\beta1 / 0\.75λmax\\lambda\_\{\\max\}/τ\\tau1 / 0\.3Maximum new tokens256Temperature0DecodingGreedyTable 6:PLUME\-specific and generation hyperparameters\.Memory units are constructed from context structure, so their number varies with the length and organization of each context\.

## Appendix DMore Experiments

This section presents supplementary discussion of the main results together with analyses ofPLUME’s hyperparameter sensitivity and computational efficiency\.

### D\.1Additional Analysis of the Main Results

##### Continual\-update performance\.

We further evaluatedPLUMEas the number of continual updates increased\. Figure[12](https://arxiv.org/html/2609.14168#A4.F12)reports ROUGE\-L Recall of 79\.27, 78\.72, 78\.05, 77\.74, and 77\.18 after 3, 5, 10, 15, and 20 updates, respectively\. The gradual decline of 2\.09 points from 3 to 20 updates suggests thatPLUMEremains effective as successive revisions accumulate, although longer update histories still introduce additional difficulty\. These results extend the single\-transition evaluation in the main benchmark and support the applicability of the method to continual context evolution\.

Figure 12:Performance under continual updates\.ROUGE\-L Recall decreases gradually as the number of updates increases from 3 to 20\.
##### Cross\-task generalization\.

GSM8K and CRUXEval differ from the fiveMUSE\-BenchQA datasets in both output structure and reasoning process: the former requires multi\-step arithmetic, whereas the latter requires deterministic program reasoning\.PLUME’s gains on both tasks and both backbones show that its benefit is not restricted to a particular extractive or document\-QA format, but extends to broader continual context\-update scenarios\.

##### Overall comparison and component roles\.

The overall comparison jointly considers ROUGE\-L Recall, LLM\-as\-a\-Judge, update/generation latency and memory, and Locality\.PLUME’s advantage is distributed across these dimensions rather than being driven by a single answer\-quality metric\. The component analysis further clarifies the division of labor: the global update representation preserves the full context state while emphasizing the latest change, and query\-activated memory evidence provides a local parameter view that reduces interference from information irrelevant to the current query\. Their combination accounts for the stronger balance between update adoption and preservation of unaffected knowledge\.

##### Radar\-chart normalization\.

For Figure[4](https://arxiv.org/html/2609.14168#S5.F4), letℳ\\mathcal\{M\}contain the four plotted methods:PLUME, CD, CD \(oracle\), and AnyEdit\. Letxm,jx\_\{m,j\}denote methodmm’s value on axisjj, with dataset\-specific axes normalized separately\. Let𝒥\+\\mathcal\{J\}\_\{\+\}contain the answer\-quality and Locality metrics and𝒥−\\mathcal\{J\}\_\{\-\}contain the latency and memory metrics\. We first orient each metric so that larger values are better:

um,j=\{xm,j,j∈𝒥\+,xm,j−1,j∈𝒥−\.u\_\{m,j\}=\\begin\{cases\}x\_\{m,j\},&j\\in\\mathcal\{J\}\_\{\+\},\\\\ x\_\{m,j\}^\{\-1\},&j\\in\\mathcal\{J\}\_\{\-\}\.\\end\{cases\}\(22\)The plotted value is then

zm,j=um,jmaxk∈ℳ⁡uk,j\.z\_\{m,j\}=\\frac\{u\_\{m,j\}\}\{\\max\_\{k\\in\\mathcal\{M\}\}u\_\{k,j\}\}\.\(23\)Thus, the best of the four methods receives 1 on each axis\. For the positive latency and memory measurements, this is equivalent to dividing the smallest value by the method’s value\.

##### Ablation normalization\.

Figure[5](https://arxiv.org/html/2609.14168#S5.F5)uses a different reference: each score is divided by the corresponding Base Model w/ Context \(oracle\) score on the same dataset and backbone, then multiplied by 100\. ForPLUME, this gives SQuAD Recall100×80\.65/95\.86=84\.13100\\times 80\.65/95\.86=84\.13, 2Wiki Recall100×47\.24/60\.11=78\.59100\\times 47\.24/60\.11=78\.59, and SQuAD Locality100×84\.24/96\.42=87\.37100\\times 84\.24/96\.42=87\.37\. The reported average normalized ROUGE\-L Recall of 81\.36 is the mean of the two normalized Recall scores; the same averaging rule yields 57\.28 and 63\.95 for the memory\-only and global\-only variants, respectively\.

### D\.2Candidate\-Rank Analysis

To determine whether continual\-update interference erases valid information or merely suppresses it, we extended the teacher\-forced token analysis in Section[1](https://arxiv.org/html/2609.14168#S1)from top\-1 to top\-10 candidates\. Figure[13](https://arxiv.org/html/2609.14168#A4.F13)shows that ROUGE\-L Recall rises from45\.90%at top\-1 to67\.73%at top\-3 and71\.79%at top\-5, then increases more gradually to 74\.93% and 75\.86% at top\-8 and top\-10\. The large early gains, followed by saturation, indicate thatmuch of the currently valid answer remains near the top of the output distribution but is outranked by interfering candidates\. This result supports targeted reweighting rather than treating the missing top\-1 prediction as complete information loss\.

Figure 13:Teacher\-forced candidate\-rank analysis\.Token\-level ROUGE\-L Recall increases sharply from top\-1 to top\-5 and then begins to saturate through top\-10\.
### D\.3Sensitivity toβ\\beta

The coefficientβ\\betacontrols how strongly the latest\-update direction is amplified within the global representation\. As shown in Figure[14](https://arxiv.org/html/2609.14168#A4.F14), increasingβ\\betafrom 0\.125 to 0\.75 raises ROUGE\-L Recall from 71\.94 to80\.65and Locality from 78\.63 to84\.24\. Both metrics then decline as amplification becomes excessive, reaching 70\.62 and 75\.48 atβ=3\\beta=3\. We therefore useβ=0\.75\\beta=0\.75\. Importantly, this optimum is obtained by strengthening the update direction inside the full\-context representation, rather than replacing that representation with query\-local evidence\. The result supports the intended division of labor: global context remains the primary state, while local evidence only corrects query\-specific suppression\.

Figure 14:Sensitivity to the global update coefficientβ\\beta\.Recall and Locality achieve their best joint result atβ=0\.75\\beta=0\.75and decrease when the update direction is under\- or over\-amplified\.
### D\.4Sensitivity toτ\\tau

The parameterτ\\taudetermines how quickly the adaptive evidence weight grows with predictive divergence\. Figure[15](https://arxiv.org/html/2609.14168#A4.F15)shows that ROUGE\-L Recall reaches its maximum of80\.65atτ=0\.3\\tau=0\.3, while Locality reaches 84\.24\. The neighboring settingsτ=0\.2\\tau=0\.2andτ=0\.4\\tau=0\.4yield lower Recall values of 80\.10 and 80\.24, respectively; beyond this region, performance generally weakens, with Recall/Locality falling to 78\.77/81\.26 atτ=1\\tau=1\. We therefore selectτ=0\.3\\tau=0\.3as the best operating point for answer recovery while retaining strong unaffected\-information preservation\.

Figure 15:Sensitivity to the adaptive\-gating parameterτ\\tau\.The selected valueτ=0\.3\\tau=0\.3maximizes Recall while maintaining strong Locality\.
### D\.5Sensitivity toλmax\\lambda\_\{\\max\}

The capλmax\\lambda\_\{\\max\}directly limits the maximum contribution of query\-relevant memory evidence\. Figure[16](https://arxiv.org/html/2609.14168#A4.F16)shows the strongest joint result atλmax=1\\lambda\_\{\\max\}=1, where Recall and Locality reach80\.65and84\.24, respectively\. Recall remains near 80 over a moderate neighborhood, but increasing the cap to 5 reduces Recall/Locality to 79\.21/82\.52\. Thus, allowing local evidence to dominate does not improve the model; it instead erodes the preservation supplied by the global context state\. Together with theβ\\betaanalysis, this directly addresses the concern that a query\-dependent method might collapse into local\-evidence decoding: the empirically preferred configuration strengthens the full\-context update direction and explicitly caps local evidence at a complementary level\.

Figure 16:Sensitivity to the maximum evidence weightλmax\\lambda\_\{\\max\}\.Recall and Locality achieve their strongest joint balance atλmax=1\\lambda\_\{\\max\}=1and decline when local evidence is assigned excessive influence\.MethodSQuAD2WikiMem\. \(GiB\)↓\\downarrowLat\. \(s/document\)↓\\downarrowMem\. \(GiB\)↓\\downarrowLat\. \(s/document\)↓\\downarrowCD \(oracle\)3\.56856\.60022\.376950\.085AnyEdit6\.33218\.75455\.0284099\.473CD48\.708451\.82751\.956489\.059D2L0\.4380\.46611\.5361\.841PLUME0\.4380\.44711\.5361\.832Table 7:Update efficiency on SQuAD and 2Wiki\.Memory is the peak additional CUDA memory allocated during context parameterization, and latency is the mean update time per document\. The two base\-model references are omitted because they incur no update\-stage cost\. Oracle methods are excluded from ranking; bold and underlined values denote the best and second\-best non\-oracle results, respectively\. Lower values are better, and tied best values are both bolded\.MethodSQuAD2WikiMem\. \(GiB\)↓\\downarrowLat\. \(s/query\)↓\\downarrowMem\. \(GiB\)↓\\downarrowLat\. \(s/query\)↓\\downarroww/ Context \(oracle\)0\.2081\.9721\.7491\.194CD \(oracle\)0\.0571\.6630\.0534\.709w/o Context0\.0476\.4670\.0420\.924AnyEdit0\.0424\.7890\.0464\.709CD0\.0261\.9060\.0325\.718D2L0\.0391\.5770\.0455\.079PLUME0\.0591\.5920\.0685\.152Table 8:Generation efficiency on SQuAD and 2Wiki\.Memory is the peak additional CUDA memory allocated during generation, and latency is the mean generation time per query\. Oracle methods are excluded from ranking; bold and underlined values denote the best and second\-best non\-oracle results, respectively\. Lower values are better\.
### D\.6Efficiency Measurement

We followed the efficiency protocol of D2L\([Charakorn et al\., 2026](https://arxiv.org/html/2609.14168#bib.bib6)\)and separately measured context update and query generation\.Update Latencyis the mean wall\-clock time required to transform one full context history into the reusable parameterized state of a method\. It includes all method\-specific context processing and parameter construction performed before query answering\. Sequential components of the same update were summed for each context before averaging across contexts\.Generation Latencyis the mean wall\-clock time required to generate an answer after the update stage is complete\. The total generation time was divided by the number of evaluated queries\. ForPLUME, query\-dependent evidence activation, generation of the corresponding evidence LoRA, and adaptive memory evidence decoding were included in this stage\. Base Model w/ Context \(oracle\) re\-encoded the full context history for every query, whereas parameterized methods provided only the query to the target language model during generation\.

We synchronized CUDA immediately before starting and after completing each measured phase to include all asynchronous GPU operations in the elapsed time\. We did not discard separate warm\-up instances because the official D2L protocol does not specify a warm\-up exclusion\. All methods used an evaluation batch size of one in the efficiency experiments, greedy decoding, and the same maximum generation length\.

Tables[7](https://arxiv.org/html/2609.14168#A4.T7)and[8](https://arxiv.org/html/2609.14168#A4.T8)report update and generation efficiency, respectively, on SQuAD and 2Wiki under this unified protocol\. Together, the tables separate the one\-time context\-update cost from the cost incurred while answering queries, allowing direct comparison among full\-context inference, context parameterization, context distillation, parameter editing, andPLUME\.

Update MemoryandGeneration Memorymeasure thepeak additional CUDA memory allocatedduring their respective phases\. For a phases∈\{upd,gen\}s\\in\\\{\\mathrm\{upd\},\\mathrm\{gen\}\\\}, we compute

Ms=Mspeak−Msstart,M\_\{s\}=M^\{\\mathrm\{peak\}\}\_\{s\}\-M^\{\\mathrm\{start\}\}\_\{s\},\(24\)whereMsstartM^\{\\mathrm\{start\}\}\_\{s\}is the allocated CUDA memory immediately before the phase begins andMspeakM^\{\\mathrm\{peak\}\}\_\{s\}is the maximum allocated CUDA memory observed at any point during that phase\. This additional\-memory definition deliberately excludes model parameters and other allocations already resident at the phase boundary, so as to isolate the incremental cost attributable to the phase itself\. For methods with multiple sequential update components, we reported the largest additional peak observed among the components rather than summing their individual peaks, since the components did not coexist in memory at the same time\. We took the maximum phase\-level memory value across all evaluated contexts or queries to reflect the worst\-case footprint\. In multi\-GPU experiments, measurements were collected independently on each GPU, and the maximum across workers was reported as the representative value\.

Memory was measured usingtorch\.cuda\.memory\_allocated\(\)andtorch\.cuda\.max\_memory\_allocated\(\), rather than relying on reserved memory, which can overstate actual usage\. We additionally sampled process\-level GPU memory withnvidia\-smievery 200 ms in order to independently verify the end\-to\-end absolute peak, but used the phase\-level additional allocated memory as the basis for the reported Update Memory and Generation Memory\. Under this protocol, the generation memory of Base Model w/ Context \(oracle\) included the additional activations and KV cache induced by the full context history, whereas parameterized methods instead measured only the additional memory required to answer the query using their previously constructed parameterized states\.

## Appendix EData Annotation Guidelines

This section describes the annotation and verification protocols used to constructMUSE\-Benchand the cross\-task evaluation data\. The protocol follows the task definition in Section[3](https://arxiv.org/html/2609.14168#S3): the complete history is retained in chronological order, later information supersedes only the corresponding earlier information, and information outside the update scope remains valid\. Accordingly, annotation is performed at the level of a context and all of its associated questions, rather than by independently editing isolated question–answer pairs\.

### E\.1Task Semantics

For an original contextCoC\_\{o\}and updateCuC\_\{u\},MUSEretains the chronological historyCf=Co\|CuC\_\{f\}=C\_\{o\}\\\|C\_\{u\}rather than replacing or deleting earlier text\. The later context determines the current value only for information within its update scope; all unrelated information inCoC\_\{o\}remains valid\. Methods concatenateColdC\_\{\\mathrm\{old\}\}andCupdateC\_\{\\mathrm\{update\}\}to formCfullC\_\{\\mathrm\{full\}\}; the hypernetwork parameterizesColdC\_\{\\mathrm\{old\}\},CfullC\_\{\\mathrm\{full\}\}, or memory units derived fromCfullC\_\{\\mathrm\{full\}\}, but does not parameterizeCupdateC\_\{\\mathrm\{update\}\}separately\. The target language model receives only the query as textual input\.

Each query is assigned to𝒬upd\\mathcal\{Q\}\_\{\\mathrm\{upd\}\}if its reference answer changes under the update, or to𝒬keep\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}if its answer should remain unchanged\. Both sets are evaluated against the currently valid state defined by the full history\. Success therefore requires simultaneously adopting revised information for𝒬upd\\mathcal\{Q\}\_\{\\mathrm\{upd\}\}and preserving unaffected information for𝒬keep\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}; improving one set by indiscriminately overwriting the other does not satisfy the task\.

##### Recursive state evolution\.

The two\-context notation above describes one transition within a recursive sequence\. Starting fromC\(0\)C^\{\(0\)\}, updateCu\(t\)C\_\{u\}^\{\(t\)\}is appended to the complete history available before steptt:

C\(t\)\\displaystyle C^\{\(t\)\}=C\(t−1\)\|Cu\(t\),\\displaystyle=C^\{\(t\-1\)\}\\\|C\_\{u\}^\{\(t\)\},\(25\)=C\(0\)\|Cu\(1\)​‖⋯‖​Cu\(t\)\.\\displaystyle=C^\{\(0\)\}\\\|C\_\{u\}^\{\(1\)\}\\\|\\cdots\\\|C\_\{u\}^\{\(t\)\}\.Consequently, the output history of one transition is the input history of the next: at stept\+1t\+1,Cold=C\(t\)C\_\{\\mathrm\{old\}\}=C^\{\(t\)\},Cupdate=Cu\(t\+1\)C\_\{\\mathrm\{update\}\}=C\_\{u\}^\{\(t\+1\)\}, andCfull=C\(t\+1\)C\_\{\\mathrm\{full\}\}=C^\{\(t\+1\)\}\. No earlier text is removed from the accumulated history\. If several updates concern the same fact or dependency chain, the latest applicable statement determines its current value, while earlier versions remain in the history as invalidated evidence\. Information outside the union of the update scopes retains its most recent valid value\. The correction notices delimit successive transitions and make this chronological precedence explicit\.

##### Recursive parameterization and evaluation\.

At every step, the method parameterizes both the pre\-update history and the newly accumulated history,

Δ​Wold\(t\)\\displaystyle\\Delta\\mathrm\{W\}\_\{\\mathrm\{old\}\}^\{\(t\)\}=Hϕ​\(C\(t−1\)\),\\displaystyle=H\_\{\\phi\}\\\!\\left\(C^\{\(t\-1\)\}\\right\),\(26\)Δ​Wfull\(t\)\\displaystyle\\Delta\\mathrm\{W\}\_\{\\mathrm\{full\}\}^\{\(t\)\}=Hϕ​\(C\(t\)\)\.\\displaystyle=H\_\{\\phi\}\\\!\\left\(C^\{\(t\)\}\\right\)\.and constructs the step\-specific global update representation from their difference as defined in Eq\.[6](https://arxiv.org/html/2609.14168#S4.E6)\. Query\-activated memory evidence is selected from units in the current complete historyC\(t\)C^\{\(t\)\}\. Thus, each transition must resolve the latest update against every version retained so far, rather than treatingCu\(t\)C\_\{u\}^\{\(t\)\}as an isolated document\. After constructing the current parameterized state, queries are answered without supplying the textual history to the target model\.

The query partition and references are also transition\-specific\. A query belongs to𝒬upd\(t\)\\mathcal\{Q\}\_\{\\mathrm\{upd\}\}^\{\(t\)\}when its correct answer changes from stept−1t\-1to steptt, and to𝒬keep\(t\)\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}^\{\(t\)\}when the answer remains valid across that transition\. A query unaffected at one step may become update\-affected later, or vice versa\. In every case, the referencey⋆\(t\)​\(q\)y^\{\\star\(t\)\}\(q\)is determined by the latest valid evidence inC\(t\)C^\{\(t\)\}\. Evaluation at steptttherefore measures both adoption of the new state and retention of all information that should survive that particular transition\.

##### Benchmark instantiation\.

Each standardMUSE\-Benchexample instantiates one transition, usingCoC\_\{o\},CuC\_\{u\}, andCfC\_\{f\}as shorthand forC\(t−1\)C^\{\(t\-1\)\},Cu\(t\)C\_\{u\}^\{\(t\)\}, andC\(t\)C^\{\(t\)\}\. This transition\-level design supports controlled comparison across methods while preserving the recursive task definition\. The continual\-update experiment in Appendix[D\.1](https://arxiv.org/html/2609.14168#A4.SS1.SSS0.Px1)explicitly chains multiple transitions, repeatedly carryingC\(t\)C^\{\(t\)\}forward as the pre\-update history and evaluating the resulting state after increasing numbers of accumulated updates\.

### E\.2MUSE\-Bench

##### Context and version structure\.

For every instance, the original contextColdC\_\{\\mathrm\{old\}\}is placed first andpreserved verbatim, including sentence order, punctuation, spacing, and any source\-specific formatting\. We then append exactly one fixed notice, “\[Correction & Update Notice\]: The following account supersedes the corresponding details above\.”, followed by the update contextCupdateC\_\{\\mathrm\{update\}\}; adjacent components are separated by a single line break\. This producesCfull=Cold\|CupdateC\_\{\\mathrm\{full\}\}=C\_\{\\mathrm\{old\}\}\\\|C\_\{\\mathrm\{update\}\}while making the temporal precedence relation explicit\. The notice applies only to overlapping details: facts not addressed by the update remain valid and continue to support𝒬keep\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}\.

##### Selecting the update scope\.

We first locate all evidence units supporting a candidate answer and record the complete support chain, including intermediate facts needed for multi\-hop or causal inference\. We select a target whose value can be changed without altering the topic, task type, or core discourse structure\. The new value must constitute a genuine factual change rather than a difference in capitalization, punctuation, spelling, inflection, alias choice, numeric formatting, or unit expression\. It must also preserve the answer type requested by the query \(e\.g\., person, location, date, scalar, list, or yes/no\)\. We avoid updates that require broad unrelated changes or create an implausible document merely to force a different answer\.

##### Structure\-preserving construction\.

The update context is a natural, self\-consistent account rather than a short patch tailored to the selected query\. It retains the source’s topic, discourse order, major entities, information density, and reasoning form wherever these are not affected by the update\. Paragraph, passage, section, table, and list organization is preserved when it carries semantic information\. An update may be shorter because redundant wording is removed, but it may not omit major entities, events, conditions, or reasoning links simply because they are not mentioned in the target query\. Conversely, unrelated material is not added to imitate the length of the source\. These constraints prevent answer\-bearing evidence from becoming identifiable through anomalous position, detail, or brevity\.

##### Dependency closure\.

After changing the target fact, we compute itsdependency closure within the documentand synchronously revise every affected statement\. This includes repeated or paraphrased mentions; aliases, abbreviations, and coreference; forward and inverse relations; intermediate multi\-hop nodes; temporal order, age, duration, and date relations; quantities, totals, percentages, rankings, and unit conversions; causal consequences and conditional outcomes; and summaries, captions, tables, discussions, or conclusions derived from the target fact\. All mechanically checkable relations are recalculated\. No statement after the notice may preserve a direct or indirect path that makes the old answer currently valid, and the revised statements may not introduce a second plausible answer\.

##### Affected and unaffected questions\.

Consistent with Section[3](https://arxiv.org/html/2609.14168#S3), every question is assigned to either𝒬upd\\mathcal\{Q\}\_\{\\mathrm\{upd\}\}or𝒬keep\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}\. Forq∈𝒬updq\\in\\mathcal\{Q\}\_\{\\mathrm\{upd\}\}, the reference answer must differ semantically from the original answer, and the update context must explicitly state the complete evidence needed to obtain the new answer under the supersession rule\. Forq∈𝒬keepq\\in\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}, the original answer is retained unchanged, and neither the target update nor any dependent revision may alter its supporting evidence or otherwise render it ambiguous\. Thus,CfullC\_\{\\mathrm\{full\}\}must simultaneously support all current answers in both sets; a document is rejected outright if fixing an affected query happens to invalidate an intended unaffected query\.

##### Question and answer preservation\.

Questions are kept verbatimwhenever they remain well formed and coherent under the new state\. A question is changed only if it explicitly contains a replaced entity, value, relation, condition, or false premise, and then only the smallest coupled span necessary is revised; its intent, difficulty, answer type, and position within the set are all preserved\. Duplicate questions and their relative ordering are likewise retained without modification\. Answers remain concise and consistently follow the source dataset’s granularity and data type\. Every entity, number, qualifier, and list member appearing in an answer must be supported by the currently valid evidence, with correct event association, temporal scope, geographic level, unit, and set boundary\. Mere surface occurrence of an answer string is insufficient unless the surrounding context clearly establishes the queried relation\.

##### Naturalness and leakage prevention\.

Apart from the fixed notice,CupdateC\_\{\\mathrm\{update\}\}contains only ordinary declarative prose in the style of the source\. It cannot reproduce or closely paraphrase the query, introduce question–answer formatting, list responses in query order, state “the answer is,” refer to prompts or annotation, or describe how an earlier answer was changed\. Supporting facts are inserted at their natural discourse locations\. For inference\-oriented examples, the update provides the necessary premises but does not append a query\-specific conclusion that performs the intended reasoning for the model\. The update must be understandable without external knowledge, while unrelated valid information may still be inherited chronologically fromColdC\_\{\\mathrm\{old\}\}\.

##### Dataset\-specific constraints\.

ForSQuAD, we preserve the main narrative, entity roles, and event organization, and verify identity, date, location, quantity, causality, condition, and enumeration relations separately\. ForROPES, we preferentially preserve the background scientific or commonsense principle and modify the concrete scenario’s conditions; the principle itself is changed only when no coherent scenario\-level update can change the target answer\. The resulting premises must still require the same type of reasoning rather than directly stating the comparison outcome\. For2WikiMultihopQA, all source passages and their order are retained, every hop connecting the question entity to the new answer is explicit, and reciprocal family or relational statements are updated together\. ForMultiFieldQA\-en, the long document’s organization and topical coverage are maintained, with repeated evidence checked throughout the document\. ForQASPER, the paper\-like structure is preserved and modified facts are propagated across the abstract, method, experiments, tables or captions, results, discussion, conclusion, glossary, and appendix when present; numerical totals, subsets, percentages, and method–metric–conclusion relations are recomputed for consistency\.

##### Quality verification and rejection criteria\.

Each candidate first undergoes deterministic checks for parseability, schema and field types, record count, identifier preservation, exact retention ofColdC\_\{\\mathrm\{old\}\}, notice uniqueness and boundary formatting, question–answer alignment, and preservation of duplicate items\. Lexical checks flag copied questions, annotation meta\-language, unchanged answers in𝒬upd\\mathcal\{Q\}\_\{\\mathrm\{upd\}\}, and residual old answers including aliases, abbreviations, Unicode variants, equivalent numbers, and converted units\. These checks serve only as filters: semantic verification must additionally confirm the subject–relation–object match, completeness of lists and multi\-hop paths, causal and temporal attribution, dependency closure, uniqueness of the current answer, and preservation of every𝒬keep\\mathcal\{Q\}\_\{\\mathrm\{keep\}\}answer\. As stated in Section[3\.2](https://arxiv.org/html/2609.14168#S3.SS2),GPT\-5\.6\-Solperforms 26 rounds of verification over candidate samples\. A sample is returned for revision or discarded if any affected answer lacks complete support, any obsolete evidence path remains valid after the update, any dependent fact is inconsistent, any unaffected answer changes, or the update exhibits leakage, meta\-narration, structural truncation, or internal contradiction\.Only samples passing all checks are retained\.

### E\.3Human Evaluation of Main Results

To assess the automatic metrics used in the main results, we randomly sampled 50 contexts comprising 1,037 question–answer pairs and manually evaluated the corresponding model responses\. On this subset, ROUGE\-L Recall, LLM\-as\-a\-Judge, and human evaluation scored 78\.17, 72\.86, and 73\.22, respectively, on a 0–100 scale\. The automatic scores were close to the aggregate human assessment, supporting their use as evaluation indicators; LLM\-as\-a\-Judge was more closely aligned, differing by only 0\.36 points, compared with 4\.95 points for ROUGE\-L Recall\.

### E\.4Cross\-Task Data Construction

We apply the same chronological, structure\-preserving principle to GSM8K and CRUXEval without treating them as additionalMUSE\-Benchsubsets\. ForGSM8K, the final question is split into a declarative problem context and a complete query; conditional clauses and output instructions belonging to the query remain there verbatim\. The fixed notice states that the revised problem statement supersedes overlapping details\. We preserve the problem type and reasoning topology, change one or more numerical conditions, and recompute every dependent intermediate quantity, total, unit, and final answer\. The revised problem must remain solvable, unambiguous, and comparable in difficulty, and neither the old nor revised query is duplicated inside the accumulated context\. ForCRUXEval, the fixed notice states that the revised code supersedes overlapping details\. We preserve the original prediction task and program format while making a substantive, deterministic change to the program state or computation\. The revised code must be syntactically valid and executable, and the reference output is obtained from the revised program and input rather than from textual resemblance or the obsolete execution trace\. For both tasks,CfullC\_\{\\mathrm\{full\}\}is the only source of the currently valid problem or program state, and the query retains the form and output requirements of the original task\.

## Appendix FPLUMEAlgorithm

The completePLUMEpipeline is formalized in Algorithm[1](https://arxiv.org/html/2609.14168#algorithm1), where the notation directly follows Section[4](https://arxiv.org/html/2609.14168#S4)\. The global update LoRA is constructed once for each context history and reused across queries, whereas memory evidence activation and adaptive decoding are performed for each query\.

Algorithm 1Detailed inference workflow ofPLUME\.The algorithm constructs the global update representation, activates query\-relevant memory evidence, and adaptively combines their token distributions during decoding\.Input:

Cold,Cupdate,qC\_\{\\mathrm\{old\}\},C\_\{\\mathrm\{update\}\},q;

fθ,Hϕf\_\{\\theta\},H\_\{\\phi\};

α,β,δ,λmax,τ,Tmax\\alpha,\\beta,\\delta,\\lambda\_\{\\max\},\\tau,T\_\{\\max\}
Output:Generated response

y=\(y1,…,yT\)y=\(y\_\{1\},\\ldots,y\_\{T\}\)
/\*

■\\blacksquareGlobal Update LoRA\*/

1

Cfull←Cold\|CupdateC\_\{\\mathrm\{full\}\}\\leftarrow C\_\{\\mathrm\{old\}\}\\\|C\_\{\\mathrm\{update\}\};

/\*Parameterize the pre\-update and full context states \*/

2

Δ​Wold←Hϕ​\(Cold\)\\Delta\\mathrm\{W\}\_\{\\mathrm\{old\}\}\\leftarrow H\_\{\\phi\}\(C\_\{\\mathrm\{old\}\}\),

Δ​Wfull←Hϕ​\(Cfull\)\\Delta\\mathrm\{W\}\_\{\\mathrm\{full\}\}\\leftarrow H\_\{\\phi\}\(C\_\{\\mathrm\{full\}\}\);

3Δ​Wg←α​Δ​Wfull\+β⁡\(Δ​Wfull−Δ​Wold\)\\Delta\\mathrm\{W\}\_\{g\}\\leftarrow\\alpha\\Delta\\mathrm\{W\}\_\{\\mathrm\{full\}\}\+\\beta\(\\Delta\\mathrm\{W\}\_\{\\mathrm\{full\}\}\-\\Delta\\mathrm\{W\}\_\{\\mathrm\{old\}\}\)⊳\\trianglerightEq\.[6](https://arxiv.org/html/2609.14168#S4.E6);

/\*

∙\\bulletQuery\-Activated Memory Evidence\*/

4

\{mi\}i=1n←Segment⁡\(Cfull\)\\\{m\_\{i\}\\\}\_\{i=1\}^\{n\}\\leftarrow\\operatorname\{Segment\}\(C\_\{\\mathrm\{full\}\}\);

5

si←slex​\(q,mi\)s\_\{i\}\\leftarrow s\_\{\\mathrm\{lex\}\}\(q,m\_\{i\}\)for each

i∈\{1,…,n\}i\\in\\\{1,\\ldots,n\\\};

6

smax←maxi⁡sis\_\{\\max\}\\leftarrow\\max\_\{i\}s\_\{i\};

7

ℐδ←\{i:smax−si≤δ\}\\mathcal\{I\}\_\{\\delta\}\\leftarrow\\\{i:s\_\{\\max\}\-s\_\{i\}\\leq\\delta\\\};

8i⋆←max⁡ℐδi^\{\\star\}\\leftarrow\\max\\mathcal\{I\}\_\{\\delta\}⊳\\trianglerightlatest unit within the recency margin;

9

m⋆←mi⋆m^\{\\star\}\\leftarrow m\_\{i^\{\\star\}\};

/\*The hypernetwork receives only the activated evidence \*/

10Δ​We​\(q\)←Hϕ​\(m⋆\)\\Delta\\mathrm\{W\}\_\{e\}\(q\)\\leftarrow H\_\{\\phi\}\(m^\{\\star\}\)⊳\\trianglerightEq\.[8](https://arxiv.org/html/2609.14168#S4.E8);

/\*

▶\\blacktrianglerightAdaptive Memory Evidence Decoding\*/

11

y<1←∅y\_\{<1\}\\leftarrow\\varnothing;

12for*t←1t\\leftarrow 1toTmaxT\_\{\\max\}*do

/\*Compute the global and evidence predictions \*/

13pg,t←pθ⊕Δ​Wg\(⋅∣q,y<t\)p\_\{g,t\}\\leftarrow p\_\{\\theta\\oplus\\Delta\\mathrm\{W\}\_\{g\}\}\(\\cdot\\mid q,y\_\{<t\}\)⊳\\trianglerightEq\.[7](https://arxiv.org/html/2609.14168#S4.E7);

14pe,t←pθ⊕Δ​We​\(q\)\(⋅∣q,y<t\)p\_\{e,t\}\\leftarrow p\_\{\\theta\\oplus\\Delta\\mathrm\{W\}\_\{e\}\(q\)\}\(\\cdot\\mid q,y\_\{<t\}\)⊳\\trianglerightEq\.[9](https://arxiv.org/html/2609.14168#S4.E9);

15dt←DJS\(pe,t∥pg,t\)d\_\{t\}\\leftarrow D\_\{\\mathrm\{JS\}\}\(p\_\{e,t\}\\parallel p\_\{g,t\}\)⊳\\trianglerightEq\.[10](https://arxiv.org/html/2609.14168#S4.E10);

16λt←λmax​dt/\(dt\+τ\)\\lambda\_\{t\}\\leftarrow\\lambda\_\{\\max\}\\,d\_\{t\}/\(d\_\{t\}\+\\tau\)⊳\\trianglerightEq\.[11](https://arxiv.org/html/2609.14168#S4.E11);

/\*Fuse token scores with the adaptive evidence weight \*/

17foreach*candidate tokenvv*do

18St​\(v\)←log⁡pg,t​\(v\)\+λt​log⁡pe,t​\(v\)S\_\{t\}\(v\)\\leftarrow\\log p\_\{g,t\}\(v\)\+\\lambda\_\{t\}\\log p\_\{e,t\}\(v\)⊳\\trianglerightEq\.[12](https://arxiv.org/html/2609.14168#S4.E12);

19

yt←arg⁡maxv​St​\(v\)y\_\{t\}\\leftarrow\\arg\\max\_\{v\}S\_\{t\}\(v\);

20

y<t\+1←y<t\|yty\_\{<t\+1\}\\leftarrow y\_\{<t\}\\\|y\_\{t\};

21if*yty\_\{t\}is an end\-of\-sequence token*then

22break;

23return*yy*;

## Appendix GDiscussion

The experiments consistently supporta global\-first interpretationof continual context parameterization\. The global update representation supplies a stable state of the complete context history, while activated memory evidence provides a targeted correction only when the query exposes predictive disagreement\. The sensitivity results reinforce this interpretation: performance peaks at an intermediate global update coefficient and a bounded local\-evidence weight, whereas excessive local weighting reduces both answer recovery and preservation of unaffected information\. Query dependence therefore determines where correction is needed without making the answer depend exclusively on a retrieved local fragment\.

The candidate\-rank analysis also clarifies why this combination is effective\. Much of the valid answer remains within the model’s high\-probability candidates even when it is not ranked first, suggesting that continual updates often create competition among retained signals rather than simply erasing the current answer\. Global update amplification and adaptive evidence decoding act on complementary parts of this problem: the former improves the representation of the current context state, and the latter selectively resolves residual query\-level competition\.

## Appendix HFuture Work

Our future work will proceed in two main directions\. On one hand, we will investigatemore faithful and capacity\-aware context parameterization methodsbeyond D2L, with particular attention to reducing information loss in long contexts and across multiple successive updates\. On the other hand, we will explore how to combinePLUME’s training\-free mechanisms withupdate\-aware training objectives, for example by using global parameter differences, activated memory evidence, and predictive disagreement as supervision for hypernetwork training and evidence routing\. We believe that combining efficient inference\-time adaptation with more reliable context encoding will be important for building scalable and continually updatable parametric memory\.

## Appendix IPrompts

This section provides the complete rubric prompts used for the LLM\-as\-a\-Judge evaluation described in Appendix[B\.4](https://arxiv.org/html/2609.14168#A2.SS4.SSS0.Px3)\.

##### D2 Semantic Equivalence\.

D2 evaluates whether the model output expresses the same answer as the current reference answer and preserves all information required to answer the query correctly\. The complete prompt is shown in Figure[17](https://arxiv.org/html/2609.14168#A9.F17)\.

Figure 17:Prompt used for D2 Semantic Equivalence evaluation\.The rubric determines whether a model output expresses the same answer as the current reference while preserving all required information\.
##### D3 Final\-Answer Agreement\.

D3 evaluates whether the final answer to which the model commits agrees with the current reference answer, including all qualifiers and relations required by the query\. The complete prompt is shown in Figure[18](https://arxiv.org/html/2609.14168#A9.F18)\.

Figure 18:Prompt used for D3 Final\-Answer Agreement evaluation\.The rubric determines whether the model’s committed final answer agrees with the current reference, including required qualifiers and relations\.