Self-Consolidating Language Models: Continual Knowledge Incorporation from Context

# Continual Knowledge Incorporation from Context
Source: [https://arxiv.org/html/2605.07076](https://arxiv.org/html/2605.07076)
## Self\-Consolidating Language Models: Continual Knowledge Incorporation from Context

Zekun Wang Anant Gupta11footnotemark:1Zihan Dong Christopher J\. MacLellan Georgia Institute of Technology \{zekun,agupta886,zdong312,cmaclell3\}@gatech\.edu

###### Abstract

Large language models \(LLMs\) increasingly receive information as streams of passages, conversations, and long\-context workflows\. While longer context windows expose more evidence, they do not ensure that useful information is preserved and reused\. We study continual context consolidation: writing current context into model weights while limiting interference with previously consolidated information\. We proposeSelf\-ConsolidatingLanguage Models \(SCoL\), a post\-training framework in which, given current context, an LLM learns to generate textual update instructions specifying which of its own Transformer layers should be updated\. Because committed updates change the model that later generates future selections, we train SCoL with meta\-reinforcement learning over an evolving model state\. We instantiate SCoL with supervised QA rewards on SQuAD knowledge incorporation and intrinsic likelihood\-based rewards for LongBench v2 long\-context consolidation\. Across both settings, SCoL improves acquisition and retention over prompting, summarization, batch test\-time training, and sequential finetuning baselines\. Analysis of learned selection patterns shows that SCoL encourages the LLM to generate sparse update locations that align with layers of high Fisher information, suggesting that the model learns to route plasticity toward loss\-sensitive regions while limiting interference\. Moreover, SCoL transfers from shorter meta\-training streams to longer LongBench v2 streams at evaluation, suggesting that our framework supports scalable streaming consolidation\.

## 1Introduction

Large language models increasingly operate over streaming context, including multi\-turn conversations, tool\-using agents, document workflows, and long\-horizon interactive tasks\. In these settings, the model must not only answer from the current prompt, but also preserve earlier information for later use\. Longer context windows help, but they do not guarantee reliable memory: models can underuse distant evidence and degrade as context length and reasoning complexity increaseLiuet al\.\([2024](https://arxiv.org/html/2605.07076#bib.bib18)\); Hsiehet al\.\([2024](https://arxiv.org/html/2605.07076#bib.bib69)\)\. Prior work manages long\-context information through retrieval or external memoryLewiset al\.\([2020](https://arxiv.org/html/2605.07076#bib.bib19)\); Borgeaudet al\.\([2022](https://arxiv.org/html/2605.07076#bib.bib21)\); Wanget al\.\([2023a](https://arxiv.org/html/2605.07076#bib.bib22)\), attention\-routing or memory mechanismsMohtashami and Jaggi \([2023](https://arxiv.org/html/2605.07076#bib.bib23)\), compressionChevalieret al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib25)\); Jianget al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib26)\), and agentic memory storesPackeret al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib28)\); Zhonget al\.\([2024](https://arxiv.org/html/2605.07076#bib.bib27)\)\. These methods are effective, but they often depend on retrieval quality, auxiliary modules, lossy compression, or repeated inference\-time processing of persistent context\. We study a complementary direction: using the model weights as an internal memory substrate\. Motivated by theories of memory consolidation, where experience is integrated into long\-term internal representations rather than kept only as transient contextMcClellandet al\.\([1995](https://arxiv.org/html/2605.07076#bib.bib7)\); Kumaranet al\.\([2016](https://arxiv.org/html/2605.07076#bib.bib8)\); Dudaiet al\.\([2015](https://arxiv.org/html/2605.07076#bib.bib9)\), we ask whether useful context can be written into parameters so it remains available after the original context is gone\.

This reframes long context inference as a continual learning problem\. The model must consolidate a stream of contexts into weights while preserving previously consolidated knowledge\. However, sequential neural network updates are prone to catastrophic interferenceMcCloskey and Cohen \([1989](https://arxiv.org/html/2605.07076#bib.bib55)\)\. Classical methods reduce interference by estimating parameter importance, for example with Fisher based regularizationKirkpatricket al\.\([2017](https://arxiv.org/html/2605.07076#bib.bib2)\), but reliable Fisher or Hessian style estimation is costly and implementation sensitive at LLM scalevan de Ven \([2025](https://arxiv.org/html/2605.07076#bib.bib35)\)\. We therefore ask whether the model itself can determine where adaptation should occur\.

Inspired by SEALZweigeret al\.\([2025](https://arxiv.org/html/2605.07076#bib.bib17)\), we propose*Self\-Consolidating Language Models*\(SCoL\), a post\-training framework in which a LLM learns to generate textual update instructions specifying with its own parameters to adapt for a given context\. We then attach LoRA adapters to the selected modules and perform a LoRA updateHuet al\.\([2022](https://arxiv.org/html/2605.07076#bib.bib53)\)\. Unlike SEAL, which focuses on generating self edits and update directives, our focus is where consolidation should occur under a drifting policy—\-each committed update changes the model that will choose future updates, so the policy itself is part of the evolving state\. We trained SCoL with meta\-reinforcement learning whose reward favors acquisition of the current context while penalizing forgetting of earlier contexts\.

We instantiate SCoL under two reward settings: a supervised QA reward for SQuAD\-style knowledge incorporationRajpurkaret al\.\([2016](https://arxiv.org/html/2605.07076#bib.bib10)\), and an intrinsic likelihood\-based reward for unlabeled long\-context streams evaluated on LongBench v2Baiet al\.\([2025](https://arxiv.org/html/2605.07076#bib.bib11)\)\. Across both settings, SCoL improves acquisition and retention over in\-context inference, summarization, batch test\-time training, and sequential fine\-tuning\. Its learned selections are sparse and align with high\-Fisher information layers, suggesting that SCoL routes plasticity toward loss\-sensitive regions while limiting interference\. On LongBench v2, selections meta\-trained on shorter streams also transfer to longer evaluation streams without full\-context access\.

Our contributions are:

- •We recast long\-context inference as continual context consolidation, where useful information from a stream of contexts is written into model weights while preserving previously consolidated knowledge\.
- •We proposed SCoL, a post\-training framework that trains an LLM with meta\-reinforcement learning to generate textual description of where in its own weights to update, and instantiate SCoL with supervised QA rewards and intrinsic likelihood\-based rewards, covering both labeled knowledge incorporation and unlabeled long\-context consolidation\.
- •We evaluate against in\-context inference, summarization, batch test\-time training, and sequential fine\-tuning baselines on SQuAD and LongBench v2, and show that teaching the LLM to generate update\-location selections improves acquisition and retention, produces sparse selections that align with Fisher information, and transfers from shorter training streams to longer LongBench v2 context streams\.

## 2Related work

##### Continual learning for large language models

Continual learning studies how models acquire new tasks, domains, or knowledge without overwriting prior behavior\. In LLMs, this appears in factual and lifelong model editingMenget al\.\([2022](https://arxiv.org/html/2605.07076#bib.bib36)\); Mitchellet al\.\([2022](https://arxiv.org/html/2605.07076#bib.bib58)\); Wanget al\.\([2024a](https://arxiv.org/html/2605.07076#bib.bib38)\); Chenet al\.\([2024](https://arxiv.org/html/2605.07076#bib.bib39)\), continual domain adaptationGururanganet al\.\([2020](https://arxiv.org/html/2605.07076#bib.bib40)\), and continual instruction or alignment tuningShiet al\.\([2025](https://arxiv.org/html/2605.07076#bib.bib70)\); Chenet al\.\([2026](https://arxiv.org/html/2605.07076#bib.bib31)\)\. Existing methods typically control forgetting by estimating parameter importance or preserving prior behavior through regularizationKirkpatricket al\.\([2017](https://arxiv.org/html/2605.07076#bib.bib2)\); Li and Hoiem \([2018](https://arxiv.org/html/2605.07076#bib.bib32)\), or by restricting updates to low\-rank, orthogonal, or modular parameter subspacesHuet al\.\([2022](https://arxiv.org/html/2605.07076#bib.bib53)\); Wanget al\.\([2023b](https://arxiv.org/html/2605.07076#bib.bib33)\); Wang and Li \([2024](https://arxiv.org/html/2605.07076#bib.bib34)\)\. These approaches are effective but often depend on fixed update rules, replay or distillation signals, hand\-designed parameter partitions, or explicit importance estimates that can be costly at LLM scalevan de Ven \([2025](https://arxiv.org/html/2605.07076#bib.bib35)\)\. Our work instead learns a context\-conditioned update policy: rather than computing weight importance directly, we post\-trained the LLM to decide where new contextual knowledge should be inserted so adaptation improves incorporation while minimizing forgetting\.

##### Inference with long context

Long\-context inference asks whether an LLM can reliably use information that appears far back in the prompt\. Although context windows continue to grow, models can still underuse distant evidence and degrade as contexts become longer or more complexLiuet al\.\([2024](https://arxiv.org/html/2605.07076#bib.bib18)\); Hsiehet al\.\([2024](https://arxiv.org/html/2605.07076#bib.bib69)\)\. Existing approaches improve access to long\-range information through retrieval or external memoryLewiset al\.\([2020](https://arxiv.org/html/2605.07076#bib.bib19)\); Guuet al\.\([2020](https://arxiv.org/html/2605.07076#bib.bib20)\); Borgeaudet al\.\([2022](https://arxiv.org/html/2605.07076#bib.bib21)\); Wanget al\.\([2023a](https://arxiv.org/html/2605.07076#bib.bib22)\), special attention or memory mechanismsMohtashami and Jaggi \([2023](https://arxiv.org/html/2605.07076#bib.bib23)\); Wanget al\.\([2023a](https://arxiv.org/html/2605.07076#bib.bib22)\), context compressionRaeet al\.\([2019](https://arxiv.org/html/2605.07076#bib.bib24)\); Chevalieret al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib25)\); Jianget al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib26)\), or agentic memory systemsZhonget al\.\([2024](https://arxiv.org/html/2605.07076#bib.bib27)\); Packeret al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib28)\); Parket al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib29)\); Shinnet al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib30)\)\. These methods are effective but often keep information outside the model weights or compress it before reuse, making performance depend on retrieval quality, memory management, or lossy summarization\. We study a complementary direction: consolidating context\-derived knowledge into localized parameter updates\. This reframes long\-context inference as continual knowledge incorporation, where the model must retain useful information from a stream of contexts without overwriting previously consolidated knowledge\.

##### Self\-adaptation with reinforcement learning

Reinforcement learning is often used to optimize LLMs as policies over text for alignment, reasoning, or task\-level rewardsOuyanget al\.\([2022](https://arxiv.org/html/2605.07076#bib.bib12)\); DeepSeek\-AI \([2025](https://arxiv.org/html/2605.07076#bib.bib13)\)\. In self\-adapting LMs, the target instead shifts toward learning*how the model should change*in response to new information, connecting to meta\-learning and expert\-iteration views of controlling an inner improvement processAnthonyet al\.\([2017](https://arxiv.org/html/2605.07076#bib.bib48)\)\. Recent work learns such adaptation strategies: Transformer\-Squared learns task\-conditioned self\-adaptation with RLSunet al\.\([2025](https://arxiv.org/html/2605.07076#bib.bib75)\), and SEAL trains an LLM to generate self\-edits and update directivesZweigeret al\.\([2025](https://arxiv.org/html/2605.07076#bib.bib17)\)\. CaMeLS is especially close to our setting because it meta\-learns online adaptation over context, but it does so by training a separate small autoregressive model to assign token\-level loss weights during fine\-tuningHuet al\.\([2023](https://arxiv.org/html/2605.07076#bib.bib14)\)\. By contrast, we proposed a post\-training paradigm in which the LLM itself learns to propose*where in its own weights*to update, without explicitly computing parameter importance\. Since each committed update changes the model used for future contexts, we frame the problem as a meta\-RL problem over a drifting model state, with the goal of continual knowledge consolidation under forgetting constraints\.

## 3Self\-Consolidating Language Models

We study a continual consolidation setting in which a single LLM ingests a stream of contexts and must internalize each into its weights, without replay and without retrieval at inference time\.[Section˜3\.1](https://arxiv.org/html/2605.07076#S3.SS1)formalizes continual context consolidation as a reinforcement learning problem whose reward decomposes into an acquisition term and a forgetting term\.[Section˜3\.2](https://arxiv.org/html/2605.07076#S3.SS2)presents SCoL, a meta\-reinforcement learning procedure that trains the LLM to generate textual update\-location selections under a drifting model state\.[Section˜3\.3](https://arxiv.org/html/2605.07076#S3.SS3)instantiates the reward under supervised and sparse\-supervision regimes\. The full procedure is given in[Algorithm˜1](https://arxiv.org/html/2605.07076#alg1)\.

### 3\.1Problem setup: continual context consolidation

Letπθ0\\pi\_\{\\theta\_\{0\}\}denote a base language model \(our policy\) with parametersθ0∈ℝd\\theta\_\{0\}\\in\\mathbb\{R\}^\{d\}, and let𝒞=\(c1,c2,…,cT\)\\mathcal\{C\}=\(c\_\{1\},c\_\{2\},\\ldots,c\_\{T\}\)be a stream of contexts, where eachct∈𝒳∗c\_\{t\}\\in\\mathcal\{X\}^\{\*\}is a token sequence\. We use*context*generically: it can be a text passage, a multi\-turn interaction history, or a long\-context window\. At steptt, the current modelπθt−1\\pi\_\{\\theta\_\{t\-1\}\}receivesctc\_\{t\}and generates atextualaction

at∼πθt−1\(⋅∣ct\),a\_\{t\}\\sim\\pi\_\{\\theta\_\{t\-1\}\}\(\\cdot\\mid c\_\{t\}\),\(1\)which is parsed into a structural update units, and in our main instantiation the units are Transformer layers\. We then update the current model by:

Δ​θt=Adapt​\(θt−1,ct,at\),θt=θt−1⊕Δ​θt\.\\Delta\\theta\_\{t\}=\\mathrm\{Adapt\}\(\\theta\_\{t\-1\},c\_\{t\},a\_\{t\}\),\\qquad\\theta\_\{t\}=\\theta\_\{t\-1\}\\,\{\\oplus\}\\,\\Delta\\theta\_\{t\}\.\(2\)The prompting template and parsing procedure are described in[Appendix˜A](https://arxiv.org/html/2605.07076#A1)\. The weightsθt\\theta\_\{t\}carry all accumulated history\. No retrieval index or auxiliary memory is maintained for generation\. During training, we keep a past\-context set𝒟<t\\mathcal\{D\}\_\{<t\}containing only the material needed toevaluateforgetting from steps1,…,t−11,\\ldots,t\-1\. At inference time, the model relies onθt\\theta\_\{t\}alone\. After each consolidation step, the environment returns

rt​\(θt;ct,𝒟<t\)=u​\(θt;ct\)−λ​f​\(θt;𝒟<t\),r\_\{t\}\(\\theta\_\{t\};c\_\{t\},\\mathcal\{D\}\_\{<t\}\)=u\(\\theta\_\{t\};c\_\{t\}\)\-\\lambda f\(\\theta\_\{t\};\\mathcal\{D\}\_\{<t\}\),\(3\)whereuumeasures acquisition of the current context,ffmeasures drift on previously consolidated material, andλ≥0\\lambda\\geq 0trades off acquisition and retention\. We keepuuandffabstract here and instantiate them in[Section˜3\.3](https://arxiv.org/html/2605.07076#S3.SS3)\.

The learning objective is to learn apre\-adaptationpolicyπθ0\\pi\_\{\\theta\_\{0\}\}whose own sequential updates produce high cumulative reward:

maxθ0⁡𝒥​\(θ0\)=𝔼at∼πθt−1\(⋅∣ct\)θt=θt−1⊕Adapt​\(θt−1,ct,mt\)​\[∑t=1Trt​\(θt;ct,𝒟<t\)\]\.\\max\_\{\\theta\_\{0\}\}\\;\\mathcal\{J\}\(\\theta\_\{0\}\)=\\mathbb\{E\}\_\{\\begin\{subarray\}\{c\}a\_\{t\}\\sim\\pi\_\{\\theta\_\{t\-1\}\}\(\\cdot\\mid c\_\{t\}\)\\\\ \\theta\_\{t\}=\\theta\_\{t\-1\}\\,\{\\oplus\}\\,\\mathrm\{Adapt\}\(\\theta\_\{t\-1\},c\_\{t\},m\_\{t\}\)\\end\{subarray\}\}\\left\[\\sum\_\{t=1\}^\{T\}r\_\{t\}\(\\theta\_\{t\};c\_\{t\},\\mathcal\{D\}\_\{<t\}\)\\right\]\.\(4\)The key point is that the policy parameters drift during the stream: the action at stepttis sampled fromπθt−1\\pi\_\{\\theta\_\{t\-1\}\}, whose parameters already contain all committed updates from earlier contexts\. Thus, the goal is not to learn a fixed context\-to\-update mapping, but to meta\-learn an initialization whose rolled\-out updates continue to produce good future update selections\.

Several nearby paradigms share aspects of this problem but differ in a central architectural commitment\. RAG\(Lewiset al\.,[2020](https://arxiv.org/html/2605.07076#bib.bib19)\)and memory\-augmented architectures\(Packeret al\.,[2023](https://arxiv.org/html/2605.07076#bib.bib28); Chevalieret al\.,[2023](https://arxiv.org/html/2605.07076#bib.bib25)\)handle new information by expanding an external store while leaving model parameters fixed\. Knowledge\-editing methods\(Menget al\.,[2022](https://arxiv.org/html/2605.07076#bib.bib36),[2023](https://arxiv.org/html/2605.07076#bib.bib57); Mitchellet al\.,[2022](https://arxiv.org/html/2605.07076#bib.bib58); Wanget al\.,[2024b](https://arxiv.org/html/2605.07076#bib.bib59)\)target isolated factual updates rather than streaming accumulation and general context learning\. Batch fine\-tuning\(Howard and Ruder,[2018](https://arxiv.org/html/2605.07076#bib.bib67); Raffelet al\.,[2020](https://arxiv.org/html/2605.07076#bib.bib68)\)requires simultaneous access to the full𝒞\\mathcal\{C\}\. SCoL instead commits each incoming context into the model weights online, while learning where such updates should occur\.

### 3\.2Learning to learn without forgetting with meta\-reinforcement learning

![Refer to caption](https://arxiv.org/html/2605.07076v1/x1.png)Figure 1:Training pipeline of the Self\-Consolidating Language Model \(SCoL\)\.Inner loop \(black\):at each contextCtxt\\mathrm\{Ctx\}\_\{t\}, the current model stateθt−1\(r\)\\theta^\{\(r\)\}\_\{t\-1\}samplesKKtextual actions, adapts a candidate model, and scores it with the reward in[Equation˜3](https://arxiv.org/html/2605.07076#S3.E3)\. The highest\-reward candidate is committed, advancing the running model state\.Outer loop \(red\):after the stream is exhausted, the induced preferential dataset trains the round\-start policyθ0\(r\)\\theta\_\{0\}^\{\(r\)\}with IPO, producingθ0\(r\+1\)\\theta\_\{0\}^\{\(r\+1\)\}for the next round\.SCoL parameterizes the adaptation decision as text generation\. For each context, the LLM emits a textual list of update locations\. We then attach LoRA modules\(Huet al\.,[2022](https://arxiv.org/html/2605.07076#bib.bib53)\)to the selected structural units and perform a CLM update onctc\_\{t\}:

Δ​θ⋆​\(at\)=arg⁡minΔ​θat⁡𝔼x∼ct​\[−log⁡pθt−1\+Δ​θat​\(x\)\]\.\\Delta\\theta^\{\\star\}\(a\_\{t\}\)=\\arg\\min\_\{\\Delta\\theta\_\{a\_\{t\}\}\}\\mathbb\{E\}\_\{x\\sim c\_\{t\}\}\\bigl\[\-\\log p\_\{\\theta\_\{t\-1\}\+\\Delta\\theta\_\{a\_\{t\}\}\}\(x\)\\bigr\]\.\(5\)
Algorithm 1Self\-Consolidating LM Training:
Learning to Learn Without Forgetting

1:

θ0\(1\)\\theta\_\{0\}^\{\(1\)\}, stream

𝒞=\(c1,…,cT\)\\mathcal\{C\}=\(c\_\{1\},\\ldots,c\_\{T\}\),

KK,

λ\\lambda, rounds

RR
2:trained policy

πθ0\(R\+1\)\\pi\_\{\\theta\_\{0\}^\{\(R\+1\)\}\}
3:for

r=1,…,Rr=1,\\ldots,Rdo

4:

θ0\(r\)←θ0\(r\)\\theta^\{\(r\)\}\_\{0\}\\leftarrow\\theta^\{\(r\)\}\_\{0\};

𝒫\(r\)←∅\\mathcal\{P\}^\{\(r\)\}\\leftarrow\\varnothing
5:for

t=1,…,Tt=1,\\ldots,Tdo

6:

at\(1\),…,at\(K\)∼πθt−1\(r\)\(⋅∣ct\)a\_\{t\}^\{\(1\)\},\\ldots,a\_\{t\}^\{\(K\)\}\\sim\\pi\_\{\\theta^\{\(r\)\}\_\{t\-1\}\}\(\\cdot\\mid c\_\{t\}\)
7:for

k=1,…,Kk=1,\\ldots,Kdo

8:

Δt\(k\)←Adapt​\(θt−1\(r\),ct,at\(k\)\)\\Delta\_\{t\}^\{\(k\)\}\\leftarrow\\textsc\{Adapt\}\(\\theta^\{\(r\)\}\_\{t\-1\},c\_\{t\},a\_\{t\}^\{\(k\)\}\)
9:

θt\(r,k\)←θt−1\(r\)⊕Δt\(k\)\\theta\_\{t\}^\{\(r,k\)\}\\leftarrow\\theta^\{\(r\)\}\_\{t\-1\}\\,\{\\oplus\}\\,\\Delta\_\{t\}^\{\(k\)\}
10:

rt\(k\)←u​\(θt\(r,k\);ct\)−λ​f​\(θ~t\(r,k\);𝒟<t\)r\_\{t\}^\{\(k\)\}\\leftarrow u\(\\theta\_\{t\}^\{\(r,k\)\};c\_\{t\}\)\-\\lambda f\(\\tilde\{\\theta\}\_\{t\}^\{\(r,k\)\};\\mathcal\{D\}\_\{<t\}\)
11:endfor

12:append

\{\(aw,al,ct\):rw\>rl\}\\\{\(a^\{w\},a^\{l\},c\_\{t\}\):r\_\{w\}\>r\_\{l\}\\\}to

𝒫\\mathcal\{P\}
13:

k⋆←arg⁡maxk⁡rt\(k\)k^\{\\star\}\\leftarrow\\arg\\max\_\{k\}r\_\{t\}^\{\(k\)\}
14:

θt\(r\)←θ~t\(r,k⋆\)\\theta^\{\(r\)\}\_\{t\}\\leftarrow\\tilde\{\\theta\}\_\{t\}^\{\(r,k^\{\\star\}\)\}
15:endfor

16:

θ0\(r\+1\)←arg⁡minθ⁡ℒIPO​\(θ;𝒫\(r\)\)\\theta\_\{0\}^\{\(r\+1\)\}\\leftarrow\\arg\\min\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{IPO\}\}\(\\theta;\\mathcal\{P\}^\{\(r\)\}\)
17:endfor

18:return

πθ0\(R\+1\)\\pi\_\{\\theta\_\{0\}^\{\(R\+1\)\}\}

This yields the candidate post consolidation model

θt\(a\)=θt−1⊕Δ​θ⋆​\(a\)\.\\theta\_\{t\}^\{\(a\)\}=\\theta\_\{t\-1\}\\,\{\\oplus\}\\,\\Delta\\theta^\{\\star\}\(a\)\.\(6\)Across candidates, the inner\-loop hyperparameters are fixed; candidates differ only in which structural units receive adapters\. Thus, the learned decision is where to adapt, not how strongly to adapt\. At each context, we sampleKKcandidate textual actions from the current, already\-drifted policy:

at\(1\),…,at\(K\)∼πθt−1\(⋅∣ct\)\.a\_\{t\}^\{\(1\)\},\\ldots,a\_\{t\}^\{\(K\)\}\\sim\\pi\_\{\\theta\_\{t\-1\}\}\(\\cdot\\mid c\_\{t\}\)\.\(7\)Each action is parsed, adapted, and scored:

rt\(k\)=rt​\(θt\(at\(k\)\);ct,𝒟<t\)\.r\_\{t\}^\{\(k\)\}=r\_\{t\}\\\!\\left\(\\theta\_\{t\}^\{\(a\_\{t\}^\{\(k\)\}\)\};c\_\{t\},\\mathcal\{D\}\_\{<t\}\\right\)\.\(8\)We then commit the highest\-reward candidate,

k⋆=arg⁡maxk⁡rt\(k\),θt←θt\(at\(k⋆\)\)\.k^\{\\star\}=\\arg\\max\_\{k\}r\_\{t\}^\{\(k\)\},\\qquad\\theta\_\{t\}\\leftarrow\\theta\_\{t\}^\{\(a\_\{t\}^\{\(k^\{\\star\}\)\}\)\}\.\(9\)This committed update changes the model that will generate the next action:πθt\\pi\_\{\\theta\_\{t\}\}is both the consolidated model and the policy used at the next context\. TheKKcandidate scores also induce preference data\. For each context, we construct

𝒫t=\{\(ct,aw,al\):rt​\(aw\)\>rt​\(al\)\},\\mathcal\{P\}\_\{t\}=\\left\\\{\(c\_\{t\},a^\{w\},a^\{l\}\):r\_\{t\}\(a^\{w\}\)\>r\_\{t\}\(a^\{l\}\)\\right\\\},\(10\)whereawa^\{w\}andala^\{l\}denote higher\- and lower\-reward textual actions for the same context\. Across the full inner stream, we collect𝒫\(r\)=⋃t=1T𝒫t\\mathcal\{P\}^\{\(r\)\}=\\bigcup\_\{t=1\}^\{T\}\\mathcal\{P\}\_\{t\}\.

After a full round ofTTsequential consolidations, we update the*pre\-adaptation*parametersθ0\(r\)\\theta\_\{0\}^\{\(r\)\}, not the final drifted parametersθT\(r\)\\theta\_\{T\}^\{\(r\)\}\. This is the meta\-learning step: the preference data are generated by rolling the current policy through a stream of self\-induced updates, but the outer update improves the round\-start model so that its future rollouts generate better update\-location selections\. We use Identity Preference Optimisation \(IPO\)\(Azaret al\.,[2024](https://arxiv.org/html/2605.07076#bib.bib50)\):

ℒIPO​\(θ\)=𝔼\(c,aw,al\)∼𝒫\(r\)​\[\(log⁡πθ​\(aw∣c\)πref​\(aw∣c\)−log⁡πθ​\(al∣c\)πref​\(al∣c\)−12​β\)2\],\\mathcal\{L\}\_\{\\mathrm\{IPO\}\}\(\\theta\)=\\mathbb\{E\}\_\{\(c,a^\{w\},a^\{l\}\)\\sim\\mathcal\{P\}^\{\(r\)\}\}\\left\[\\left\(\\log\\frac\{\\pi\_\{\\theta\}\(a^\{w\}\\mid c\)\}\{\\pi\_\{\\mathrm\{ref\}\}\(a^\{w\}\\mid c\)\}\-\\log\\frac\{\\pi\_\{\\theta\}\(a^\{l\}\\mid c\)\}\{\\pi\_\{\\mathrm\{ref\}\}\(a^\{l\}\\mid c\)\}\-\\frac\{1\}\{2\\beta\}\\right\)^\{2\}\\right\],\(11\)withπref=πθ0\(r\)\\pi\_\{\\mathrm\{ref\}\}=\\pi\_\{\\theta\_\{0\}^\{\(r\)\}\}fixed to the round\-start policy\. The minimizer becomes the next round\-start policy,

θ0\(r\+1\)=arg⁡minθ⁡ℒIPO​\(θ\)\.\\theta\_\{0\}^\{\(r\+1\)\}=\\arg\\min\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{IPO\}\}\(\\theta\)\.\(12\)The next round begins fromθ0\(r\+1\)\\theta\_\{0\}^\{\(r\+1\)\}and produces a fresh drifting trajectoryθ1\(r\+1\),…,θT\(r\+1\)\\theta\_\{1\}^\{\(r\+1\)\},\\ldots,\\theta\_\{T\}^\{\(r\+1\)\}\. We use IPO rather than supervised fine\-tuning on the top\-1 action because the action space consists of low\-entropy structured strings\. In preliminary experiments, SFT\-style reinforcement such as ReST\(Gulcehreet al\.,[2023](https://arxiv.org/html/2605.07076#bib.bib49)\)quickly concentrated probability mass on a few repeated layer lists\. IPO instead stops updating once the chosen–rejected log\-ratio reaches the finite margin1/\(2​β\)1/\(2\\beta\), which helps preserve diversity among plausible update\-location selections\. Additional comparisons between outer\-loop update algorithms are provided in[Section˜A\.11](https://arxiv.org/html/2605.07076#A1.SS11)\.

### 3\.3Reward instantiations and reward sparsity

The abstract reward of[Equation˜3](https://arxiv.org/html/2605.07076#S3.E3)admits two instantiations, distinguished by whether a context arrives with downstream supervision\.

Whenctc\_\{t\}is accompanied by a set of held\-out queries𝒬t=\{\(q,a\)\}\\mathcal\{Q\}\_\{t\}=\\\{\(q,a\)\\\}tied to the context, both reward terms are measured in the same units of downstream performance\. The acquisition term is the accuracy on𝒬t\\mathcal\{Q\}\_\{t\}of the candidate post\-consolidation model\. The forgetting term is the accumulated degradation on past query sets, measured against each past context’s own first\-consolidation baseline\. Concretely, for each past steps<ts<twe cache the accuracybs=Acc𝒬s​\(θs\+1\)b\_\{s\}=\\mathrm\{Acc\}\_\{\\mathcal\{Q\}\_\{s\}\}\(\\theta\_\{s\+1\}\)attained immediately aftercsc\_\{s\}was first consolidated, and we penalise any drop from this baseline under the candidate stateθt\(a\)\\theta\_\{t\}^\{\(a\)\}\. That is,𝒟<t=\{\(cs,𝒬s,bs\)\}s<t\\mathcal\{D\}\_\{<t\}=\\\{\(c\_\{s\},\\mathcal\{Q\}\_\{s\},b\_\{s\}\)\\\}\_\{s<t\}in this regime, and

r​\(a;ct,θt\)=Acc𝒬t​\(θt\(a\)\)−λt−1​∑s=1t−1\[bs−Acc𝒬s​\(θt\(a\)\)\]\.r\\bigl\(a;c\_\{t\},\\theta\_\{t\}\\bigr\)\\;=\\;\\mathrm\{Acc\}\_\{\\mathcal\{Q\}\_\{t\}\}\\\!\\bigl\(\\theta\_\{t\}^\{\(a\)\}\\bigr\)\\;\-\\;\\frac\{\\lambda\}\{t\-1\}\\sum\_\{s=1\}^\{t\-1\}\\\!\\Bigl\[b\_\{s\}\\;\-\\;\\mathrm\{Acc\}\_\{\\mathcal\{Q\}\_\{s\}\}\\\!\\bigl\(\\theta\_\{t\}^\{\(a\)\}\\bigr\)\\Bigr\]\.\(13\)
Many settings of interest do not provide such supervision\. Contexts stream in without labels; downstream performance is unavailable, noisy, or delayed\. In this regime𝒟<t=\{cs\}s<t\\mathcal\{D\}\_\{<t\}=\\\{c\_\{s\}\\\}\_\{s<t\}, and we instantiate both reward terms intrinsically\. A well\-consolidated modelθ′\\theta^\{\\prime\}should raise the likelihood ofctc\_\{t\}relative to the pre\-adaptation modelθt\\theta\_\{t\},

uintrinsic​\(θ′;ct\)=log⁡pθ′​\(ct\)−log⁡pθt​\(ct\),u\_\{\\mathrm\{intrinsic\}\}\\\!\\left\(\\theta^\{\\prime\};c\_\{t\}\\right\)\\;=\\;\\log p\_\{\\theta^\{\\prime\}\}\(c\_\{t\}\)\\;\-\\;\\log p\_\{\\theta\_\{t\}\}\(c\_\{t\}\),\(14\)and maintain the likelihood ofx∈𝒟<tx\\in\\mathcal\{D\}\_\{<t\}as measured by the output\-distribution drift on previously\-consolidated material\. Because𝒟<t\\mathcal\{D\}\_\{<t\}is an empirical collection of past contexts rather than samples drawn from the model, absolute log\-likelihood differences can disproportionately weight passages with higher baseline likelihood or lower entropy, leading to uneven retention where some contexts dominate the forgetting signal\. To ensure uniform retention across previously consolidated material, we instead measure forgetting in relative terms by normalizing the change in log\-likelihood by its pre\-adaptation value\. This yields a notion of fractional degradation that treats each passage comparably, regardless of scale, and better aligns with the objective of preserving all contexts without bias\. Concretely, we define

f~intrinsic​\(θ′;𝒟<t\)=𝔼x∼𝒟<t​\[log⁡pθt​\(x\)−log⁡pθ′​\(x\)\|log⁡pθt​\(x\)\|\]=𝔼x∼𝒟<t​\[uintrinsic​\(θ′;x\)log⁡pθt​\(x\)\],\\widetilde\{f\}\_\{\\mathrm\{intrinsic\}\}\\\!\\left\(\\theta^\{\\prime\};\\mathcal\{D\}\_\{<t\}\\right\)\\;=\\;\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{<t\}\}\\left\[\\frac\{\\log p\_\{\\theta\_\{t\}\}\(x\)\-\\log p\_\{\\theta^\{\\prime\}\}\(x\)\}\{\\bigl\|\\log p\_\{\\theta\_\{t\}\}\(x\)\\bigr\|\}\\right\]=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{<t\}\}\\left\[\\frac\{u\_\{\\mathrm\{intrinsic\}\}\\\!\\left\(\\theta^\{\\prime\};x\\right\)\}\{\\log p\_\{\\theta\_\{t\}\}\(x\)\}\\right\],\(15\)While this normalization ensures that each passage contributes comparably, it removes the original scale of log\-likelihood differences\. To restore a meaningful magnitude to the forgetting signal, we rescale by the average pre\-adaptation log\-likelihood over𝒟<t\\mathcal\{D\}\_\{<t\}, yielding

fintrinsic​\(θ′;𝒟<t\)=f~intrinsic​\(θ′;𝒟<t\)⋅𝔼x∼𝒟<t​\[\|log⁡pθt​\(x\)\|\],f\_\{\\mathrm\{intrinsic\}\}\\\!\\left\(\\theta^\{\\prime\};\\mathcal\{D\}\_\{<t\}\\right\)\\;=\\;\\widetilde\{f\}\_\{\\mathrm\{intrinsic\}\}\\\!\\left\(\\theta^\{\\prime\};\\mathcal\{D\}\_\{<t\}\\right\)\\cdot\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{<t\}\}\\left\[\\bigl\|\\log p\_\{\\theta\_\{t\}\}\(x\)\\bigr\|\\right\],\(16\)In both regimes the forgetting term is a drop relative to a per\-context baseline evaluated on the same test material, differing only in whether that material is labeled queries or the raw context\. Substituting[Equations˜14](https://arxiv.org/html/2605.07076#S3.E14)and[16](https://arxiv.org/html/2605.07076#S3.E16)into[Equation˜3](https://arxiv.org/html/2605.07076#S3.E3)yields

rsparse​\(a;ct,θt\)=uintrinsic​\(θt\(a\);ct\)−λ​fintrinsic​\(θt\(a\);𝒟<t\),r\_\{\\mathrm\{sparse\}\}\\\!\\bigl\(a;c\_\{t\},\\theta\_\{t\}\\bigr\)\\;=\\;u\_\{\\mathrm\{intrinsic\}\}\\\!\\bigl\(\\theta\_\{t\}^\{\(a\)\};c\_\{t\}\\bigr\)\\;\-\\;\\lambda\\,f\_\{\\mathrm\{intrinsic\}\}\\\!\\bigl\(\\theta\_\{t\}^\{\(a\)\};\\mathcal\{D\}\_\{<t\}\\bigr\),\(17\)applicable to rolling\-window long\-context consolidation\(Chenet al\.,[2023](https://arxiv.org/html/2605.07076#bib.bib66); Chevalieret al\.,[2023](https://arxiv.org/html/2605.07076#bib.bib25)\), memory\-compaction in agentic interaction histories\(Packeret al\.,[2023](https://arxiv.org/html/2605.07076#bib.bib28); Muet al\.,[2023](https://arxiv.org/html/2605.07076#bib.bib64)\), and any setting in which a context arrives unlabeled\.

We then evaluate our self\-consolidating LMs framework under the above two reward instantiations: context that comes with immediate downstream supervisions \([Section˜4](https://arxiv.org/html/2605.07076#S4)\), and context that has no downstream supervisions \([Section˜5](https://arxiv.org/html/2605.07076#S5)\)\.

## 4Continual knowledge injection

### 4\.1Experiment setup

##### Task and dataset\.

We evaluate continual knowledge incorporation, in which a language model is updated in sequence on a stream of SQuAD\(Rajpurkaret al\.,[2016](https://arxiv.org/html/2605.07076#bib.bib10)\)passages and held responsible for answering a short question\-answer set attached to each passage\. We prepare a disjoint training split used for meta\-training the policy and a validation split reserved for sequential\-update evaluation\.

![Refer to caption](https://arxiv.org/html/2605.07076v1/x2.png)Figure 2:Illustrations of experiment setup under two reward settings\.Top: each context \(SQuAD passages\) comes with an environment reward \(downstream QA\)\.Bottom: QA is evaluated at the end of the context stream\.Additional dataset details are provided in[Section˜A\.7](https://arxiv.org/html/2605.07076#A1.SS7)\.

##### Policy, inner adaptation, and meta update\.

We choose the policy as aQwen2\.5\-7B\-Instructmodel: at each context, the model is prompted to emit a list of up toB=10B=10transformer\-layer indices\. Additional results for finer granularities \(i\.e, projection layers\) are discussed in[Section˜A\.12](https://arxiv.org/html/2605.07076#A1.SS12)\. Our inner adaptation follows SEAL setting: for every candidate action, we attach a LoRA adapter to the selected layers and run the CLM update of[Equation˜5](https://arxiv.org/html/2605.07076#S3.E5)on a set of implications pre\-generated once per passage by a frozen SEAL\-pretrained model, for 10 epochs at learning rate2×10−42\\times 10^\{\-4\}and batch size11\. During meta\-training we streamT=50T=50passages per round; at each passage we sampleK=10K=10candidates actions from the current policy, score each candidate under of[Equation˜13](https://arxiv.org/html/2605.07076#S3.E13)using a frozenQwen2\.5\-7B\-Instructas a judge, commit the argmax to the running weights per[Equation˜9](https://arxiv.org/html/2605.07076#S3.E9), and accumulate pairwise comparisons of reward scores into a round\-level buffer\. At the end of each round the buffer is used to reinforce thepre\-adaptationbase policy with IPO \([Equation˜11](https://arxiv.org/html/2605.07076#S3.E11)\), producing the base policy that seeds the next round; we reportR=2R=2such rounds\. During evaluation, LLM only emits the selection once \(i\.e\.K=1K=1\) and commits the update\. Prompts and additional training detailes are in[Sections˜A\.1](https://arxiv.org/html/2605.07076#A1.SS1),[A\.2](https://arxiv.org/html/2605.07076#A1.SS2),[A\.3](https://arxiv.org/html/2605.07076#A1.SS3),[A\.4](https://arxiv.org/html/2605.07076#A1.SS4)and[A\.5](https://arxiv.org/html/2605.07076#A1.SS5)\.

##### Baselines and metrics\.

We compare against three baselines and three SCoL variants\. All methods use the same pre\-generated implications and inner LoRA update procedure, so the comparison isolates the effect of*where*adaptation is applied\.QA Promptingprompts the base model with the question only\.Batch TTTtrains a LoRA adapter jointly on all evaluation passages in an offline pass, serving as a batch\-access upper bound\.SEALcontinual\{\}\_\{\\text\{continual\}\}applies the SEAL knowledge\-incorporation procedure to the continual stream and serves as the continual LoRA fine\-tuning baseline\.SCoLλ=0\\textbf\{SCoL\}\_\{\\lambda=0\}removes the forgetting term from our reward, isolating its contribution\.SCoLuses the full reward withλ=1\\lambda=1\.

At evaluation we apply each method to a fixed validation stream ofN=100N=100passages and record the sequential\-update accuracy matrixM∈\[0,1\]N×NM\\in\[0,1\]^\{N\\times N\}whose entryMi,jM\_\{i,j\}is the accuracy on thejj\-th passage’s queries after the model has been edited through theii\-th passage \(forj≤ij\\leq i\)\. FromMMwe report two metrics for each round:immediate acquisition, the mean of the diagonal, measuring how well each update works at the moment it is committed andretention, the mean of the final\-row entries on past passages, measuring accuracy on everything learned earlier once the full stream has been processed\. Metric formulas and baseline details are given in[Sections˜A\.6](https://arxiv.org/html/2605.07076#A1.SS6)and[A\.8](https://arxiv.org/html/2605.07076#A1.SS8)\.

### 4\.2Results and discussions

Table[1](https://arxiv.org/html/2605.07076#S4.T1)shows that both SCoL and SCoLλ=0exceed all three baselines in immediate accuracy, indicating that the learned policy improves acquisition beyond prompting, batch TTT, and sequential updating\. Batch TTT performs worse than prompting only, despite seeing all 100 passages and implications jointly, with an immediate accuracy drop from 28\.17% to 26\.52%\. We interpret this as a declarative training, query evaluation mismatch\. The adapter is trained on passage text and implication strings under language modeling, but is later queried through QA\. This resembles the reversal curse\(Berglundet al\.,[2024](https://arxiv.org/html/2605.07076#bib.bib43)\), where facts learned in one direction are not reliably retrieved in another\. SEAL collapses under sequential updating, consistent with compounding drift in parameter efficient continual tuning\(Renet al\.,[2024](https://arxiv.org/html/2605.07076#bib.bib44); Wanget al\.,[2023b](https://arxiv.org/html/2605.07076#bib.bib33)\)\. Retention clarifies the role of the forgetting term\. Prompting only and Batch TTT are no update reference points whose retention equals to their immediate score\.

Table 1:Comparison of immediate accuracy and retention across methods on continual knowledge incorporation over 100 SQuAD passages\.SEAL retains only 1\.30% accuracy after sequential incorporation\. In contrast, SCoLλ=0achieves stronger immediate acquisition, but its retention remains limited at 14\.64%\. Using forgetting rewardffrecovers retention to 20\.46% while also improving immediate accuracy\. This suggests that explicitly rewarding retention can substantially reduce catastrophic forgetting in a self\-improving language model frameworkZweigeret al\.\([2025](https://arxiv.org/html/2605.07076#bib.bib17)\)\. Figure[3](https://arxiv.org/html/2605.07076#S4.F3)further shows that retention decays monotonically withoutff, whereas SCoL keeps retention nearly stationary and continues improving acquisition\.

![Refer to caption](https://arxiv.org/html/2605.07076v1/x3.png)

![Refer to caption](https://arxiv.org/html/2605.07076v1/x4.png)![Refer to caption](https://arxiv.org/html/2605.07076v1/x5.png)

Figure 3:Left:immediate accuracy \(top\) and retention \(bottom\) across IPO updates\.Top Right:top layer selection frequencies after the final IPO update, with the full reward variant in red and the acquisition only variant in blue\.Bottom Right:examples of per\-layer weight importance measured by Fisher information \(brighter is higher\) with SCoL’s selected layers marked by white stars\.The layer selection patterns in Figure[3](https://arxiv.org/html/2605.07076#S4.F3)indicate that the learned policy is not selecting arbitrary update sites\. Both SCoL and SCoLλ=0concentrate their selections in the last layer \(L​27L27\)\. This agrees with prior evidence that late blocks in autoregressive Transformers act as storage and retrieval sites for factual associations\(Gevaet al\.,[2021](https://arxiv.org/html/2605.07076#bib.bib42); Menget al\.,[2022](https://arxiv.org/html/2605.07076#bib.bib36)\)\. We further test a fixed\-selection ablation that always updates the last 10 layers\. It achieves28\.78%28\.78\\%immediate accuracy and20\.10%20\.10\\%retention, close to but below SCoL’s20\.46%20\.46\\%\. This supports the view that late layers preserve factual associations, and that SCoL learns to exploit this structure, while its stronger immediate accuracy suggests that context\-dependent layer selection is more robust than a fixed late\-layer strategy\. At the same time, both variants avoid several early and middle layers, withL​8L8andL​9L9rarely selected and changing little across IPO rounds \(see[Figure˜7](https://arxiv.org/html/2605.07076#A1.F7)\)\. We refer to Appendix[A\.9](https://arxiv.org/html/2605.07076#A1.SS9)for additional results and standard errors\.

##### Fisher alignment\.

We then ask what the LLM is learning to select, and how these generated layer selections contribute to retention\. Since Fisher information measures where the current passage loss is most sensitive in the running model, it provides a direct diagnostic for whether SCoL routes adaptation toward steep layerwise directions while keeping updates sparse\. We measure layerwise Fisher information on the running model’s base projection weights from each passage’s implication lines, with details in Appendix[A\.14](https://arxiv.org/html/2605.07076#A1.SS14)\. Given the layer selection setS​\(c\)S\(c\)generated for contextcc, we measure alignment asrecall​@​k​\(c\)=\|S​\(c\)∩Topk​\(c\)​\(F​\(c\)\)\|/k​\(c\)\\mathrm\{recall@\}k\(c\)=\|S\(c\)\\cap\\mathrm\{Top\}\_\{k\(c\)\}\(F\(c\)\)\|/k\(c\), wherek​\(c\)=\|S​\(c\)\|k\(c\)=\|S\(c\)\|andF​\(c\)F\(c\)is the layerwise Fisher vector\. A random selection of 10 layers \(out of 28 total layers\) has expected recall10/28=35\.7%10/28=35\.7\\%\.

Table 2:Layerwise Fisher alignment on 100 SQuAD passages\. Values reportrecall@k\(c\)against the Fisher top\-k​\(c\)k\(c\)layers for learned and random selections\.Table[2](https://arxiv.org/html/2605.07076#S4.T2)shows that SCoL reaches45\.1%±1\.6%45\.1\\%\_\{\\pm 1\.6\\%\}, while SCoLλ=0reaches41\.0%±1\.3%41\.0\\%\_\{\\pm 1\.3\\%\}, both clearly above the random selection baseline\. This provides evidence that the learned selections are not merely produced by the prompt format and do not degenerate to random sparse choices, and shows that IPO reinforces the LLM to generate update locations that follow high sensitivity layers for the current passage\. Figure[3](https://arxiv.org/html/2605.07076#S4.F3)shows SCoL selections overlaid on Fisher information heatmaps for sampled SQuAD passages, where the generated layer selections concentrate on high \(bright\) Fisher layers\. As a result, SCoL updates layers that are most responsive to the current passage, but does so under a sparse selection pattern that leaves most layers untouched\. Such selective plasticity matches the principle behind continual learning methods that reduce catastrophic interference by restricting adaptation to sparse, task specific update pathwaysZenkeet al\.\([2017](https://arxiv.org/html/2605.07076#bib.bib3)\); Mallya and Lazebnik \([2018](https://arxiv.org/html/2605.07076#bib.bib4)\); Serràet al\.\([2018](https://arxiv.org/html/2605.07076#bib.bib5)\)\.

## 5Long\-context consolidation

### 5\.1Experiment setup

##### Task, dataset, and evaluation\.

In many real world settings, models often receive information as a continuous stream \(e\.g\., daily updates to knowledge base\) without intermediate supervision, and must answer queries based on accumulated context\. To simulate this setting, we use LongBench\-v2\(Baiet al\.,[2025](https://arxiv.org/html/2605.07076#bib.bib11)\), where each example consists of a long context paired with a question requiring reasoning over the full passage\. We treat each passage as a sequential stream of fixed\-length \(2048\-token\) segments\. The objective is to consolidate information from the stream into model parameters, such that the model can answer the final query without access to the original context\. To evaluate both consolidation and generalization, we consider two regimes: \(1\)short, with context lengths of 16k–32k tokens, and \(2\)long, with context lengths of 32k–64k tokens\. The model is trained only on the short regime and evaluated on both\. We evaluate on4040short passages and2020long passages\. Due to the sparsity of the reward signal, evaluation is performed by sampling 10 candidate answers after consolidation and measuring mean accuracy\. Additional details are provided in[Section˜B\.2](https://arxiv.org/html/2605.07076#A2.SS2)\.

##### Training procedure\.

We useQwen2\.5\-7B\-Instructas our policy and reuse hyperparameters from[Section˜4](https://arxiv.org/html/2605.07076#S4)\. Meta\-training consists of 2 rounds, each processing 5 passages \(or streams\)\. For each passage, at each step, we sampleK=10K=10candidate actions, perform inner\-loop adaptation and use the intrinsic reward defined in[Equation˜17](https://arxiv.org/html/2605.07076#S3.E17)to select the highest\-scoring candidate\. The selected adapter is merged into the running model \([Equation˜9](https://arxiv.org/html/2605.07076#S3.E9)\) before proceeding to the next segment\.

##### Baselines\.

We compare methods across three settings:\(1\) In\-context\.Base: the base model answers questions without additional context\.Full Context: the base model is provided the entire passage\.Summarization: the base model is given a concise summarization of the passage byKimi\-K2\.6\(Moonshot AI,[2026](https://arxiv.org/html/2605.07076#bib.bib16)\)\.\(2\) Test\-time training\.Batch TTT: a single LoRA adapter is trained on the full passage before querying\.Sequential FT: Sequentially LoRA Fine\-Tuning the model over passage segments without any module selection\.\(3\) Self\-consolidation\.SCoL: our framework\.

### 5\.2Results and discussions

Table 3:Long\-context consolidation comparison across two context\-length regimes \(short: 16k–32k and long: 32k–64k tokens\)\.[Table˜3](https://arxiv.org/html/2605.07076#S5.T3)summarizes performance across both context regimes\. SCoL achieves the strongest performance in the short\-context setting at 42\.3%, outperforming all baselines\. In the long\-context regime, SCoL remains competitive at 37\.0%, significantly surpassing Sequential FT and exceeding in\-context methods\. These results demonstrate that explicit consolidation into model weights effectively integrates information across context lengths without requiring full access to the sequence\. Batch TTT serves as a performance upper bound, achieving 41\.5% in the long\-context regime due to its global optimization over the entire sequence\. In contrast, Sequential FT exhibits a performance collapse as context length increases \(36\.5%→23\.0%36\.5\\%\\rightarrow 23\.0\\%\), underscoring the challenge of catastrophic forgetting under naïve online updates\. SCoL consistently outperforms Sequential FT, confirming that structured consolidation is critical for stable continual adaptation\. In the in\-context setting, providing full context or a Kimi\-K2\.6 summary improves over the base model, but these gains diminish as context length increases\. This degradation, consistent with prior findings that context length alone can hurt performance\(Duet al\.,[2025](https://arxiv.org/html/2605.07076#bib.bib15)\), emphasizes the limitations of relying solely on context or lossy summarization and motivates the transition to persistent parameter\-based consolidation\.

Overall, weight\-consolidation approaches \(SCoL and Batch TTT\) systematically outperform in\-context methods, highlighting the importance of updating model parameters rather than relying purely on transient context\. Notably, although SCoL is meta\-trained on short\-context examples, it generalizes effectively to the long\-context regime, handling sequences up to twice the training length while remaining competitive with methods that operate on full context\. This suggests that our consolidation mechanism captures length\-agnostic structure and can extrapolate beyond its training distribution\. We hypothesize that further gains for SCoL could be achieved by scaling meta\-training to longer sequences or dynamically adapting the strength of the forgetting termλ\\lambdaas a function of context length\. We refer to[section˜B\.3](https://arxiv.org/html/2605.07076#A2.SS3)for additional results, discussions, and standard errors\.

## 6Conclusion, limitations, and future work

We introduced SCoL, a post\-training framework that treats long\-context inference as continual context consolidation\. Instead of keeping new information only in the prompt or an external memory, SCoL post\-trains the LLM to generate textual descriptions of where the current context should be written into its own weights\. We formulate consolidation as meta\-reinforcement learning problem, where each committed update changes the model state that will generate future update selections\. Across SQuAD knowledge incorporation and LongBench v2 long\-context consolidation, SCoL improves acquisition and retention over in\-context, summarization, batch, and sequential test\-time training baselines\. Analysis of learned selection patterns shows that SCoL encourages sparse update locations that aligns with layers assigned high Fisher information, suggesting that the model learns to route plasticity toward loss\-sensitive regions while limiting interference\. Results on LongBench v2 further suggest that selection behavior learned on shorter streams can transfer to longer streaming\.

##### Limitations\.

SCoL introduces computational overhead because rewards must be evaluated through inner adaptation\. At each context, we sample candidate layer selections, adapt a copy of the current model for each candidate, evaluate acquisition and retention, and then commit the best update\. Since each committed update changes the model state used for later contexts, this process is proceed sequentially along the stream, limiting parallelism across contexts\. Additionally, SCoL could exhibit mode collapse over longer IPO updates, concentrating on repeated layer selections and reducing context\-dependent diversity \(see Appendix[A\.13](https://arxiv.org/html/2605.07076#A1.SS13)\)\. Future work should study policy update algorithms with improved diversity regularization, such as KL control or batch\-level entropy penalties\.

## References

- T\. Anthony, Z\. Tian, and D\. Barber \(2017\)Thinking fast and slow with deep learning and tree search\.InAdvances in Neural Information Processing Systems 30 \(NIPS\),External Links:1705\.08439,[Link](https://arxiv.org/abs/1705.08439)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px3.p1.1)\.
- M\. G\. Azar, M\. Rowland, B\. Piot, D\. Guo, D\. Calandriello, M\. Valko, and R\. Munos \(2024\)A general theoretical paradigm to understand learning from human preferences\.InProceedings of the 27th International Conference on Artificial Intelligence and Statistics \(AISTATS\),External Links:2310\.12036,[Document](https://dx.doi.org/10.48550/arXiv.2310.12036),[Link](https://arxiv.org/abs/2310.12036)Cited by:[§A\.11](https://arxiv.org/html/2605.07076#A1.SS11.p1.4),[Table 5](https://arxiv.org/html/2605.07076#A1.T5.6.8.2.2),[§3\.2](https://arxiv.org/html/2605.07076#S3.SS2.p3.3)\.
- J\. Bai, S\. Bai, Y\. Chu, Z\. Cui, K\. Dang, X\. Deng, Y\. Fan, W\. Ge, Y\. Han, F\. Huang, B\. Hui, L\. Ji, M\. Li, J\. Lin, R\. Lin, D\. Liu, G\. Liu, C\. Lu, K\. Lu, J\. Ma, R\. Men, X\. Ren, X\. Ren, C\. Tan, S\. Tan, J\. Tu, P\. Wang, S\. Wang, W\. Wang, S\. Wu, B\. Xu, J\. Xu, A\. Yang, H\. Yang, J\. Yang, S\. Yang, Y\. Yao, B\. Yu, H\. Yuan, Z\. Yuan, J\. Zhang, X\. Zhang, Y\. Zhang, Z\. Zhang, C\. Zhou, J\. Zhou, X\. Zhou, and T\. Zhu \(2023\)Qwen technical report\.arXiv preprint arXiv:2309\.16609\.Cited by:[Table 13](https://arxiv.org/html/2605.07076#A4.T13.2.6.4.1.1.1)\.
- Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. Li \(2025\)LongBench v2: towards deeper understanding and reasoning on realistic long\-context multitasks\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vienna, Austria,pp\. 3639–3664\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.183),[Link](https://aclanthology.org/2025.acl-long.183/)Cited by:[Table 13](https://arxiv.org/html/2605.07076#A4.T13.2.4.2.1.1.1),[§1](https://arxiv.org/html/2605.07076#S1.p4.1),[§5\.1](https://arxiv.org/html/2605.07076#S5.SS1.SSS0.Px1.p1.2)\.
- L\. Berglund, M\. Tong, M\. Kaufmann, M\. Balesni, A\. C\. Stickland, T\. Korbak, and O\. Evans \(2024\)The reversal curse: LLMs trained on “A is B” fail to learn “B is A”\.InInternational Conference on Learning Representations,Cited by:[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.p1.1)\.
- S\. Borgeaud, A\. Mensch, J\. Hoffmann, T\. Cai, E\. Rutherford, K\. Millican, G\. B\. van den Driessche, J\. Lespiau, B\. Damoc, A\. Clark, D\. de Las Casas, A\. Guy, J\. Menick, R\. Ring, T\. Hennigan, S\. Huang, L\. Maggiore, C\. Jones, A\. Cassirer, A\. Brock, M\. Paganini, G\. Irving, O\. Vinyals, S\. Osindero, K\. Simonyan, J\. Rae, E\. Elsen, and L\. Sifre \(2022\)Improving language models by retrieving from trillions of tokens\.InProceedings of the 39th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.162,pp\. 2206–2240\.External Links:[Link](https://proceedings.mlr.press/v162/borgeaud22a.html)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Chen, Z\. Sun, H\. Ye, K\. Li, and X\. Lin \(2026\)Continual learning in large language models: methods, challenges, and opportunities\.arXiv preprint arXiv:2603\.12658\.External Links:[Link](https://arxiv.org/abs/2603.12658)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- Q\. Chen, T\. Zhang, X\. He, D\. Li, C\. Wang, L\. Huang, and H\. Xue’ \(2024\)Lifelong knowledge editing for LLMs with retrieval\-augmented continuous prompt learning\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 13565–13580\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.751),[Link](https://aclanthology.org/2024.emnlp-main.751/)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Chen, S\. Wong, L\. Chen, and Y\. Tian \(2023\)Extending context window of large language models via positional interpolation\.arXiv preprint arXiv:2306\.15595\.External Links:2306\.15595,[Link](https://arxiv.org/abs/2306.15595)Cited by:[§3\.3](https://arxiv.org/html/2605.07076#S3.SS3.p3.9)\.
- A\. Chevalier, A\. Wettig, A\. Ajith, and D\. Chen \(2023\)Adapting language models to compress contexts\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 3829–3846\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.232),[Link](https://aclanthology.org/2023.emnlp-main.232/)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1),[§3\.3](https://arxiv.org/html/2605.07076#S3.SS3.p3.9)\.
- DeepSeek\-AI \(2025\)DeepSeek\-R1 incentivizes reasoning in LLMs through reinforcement learning\.Nature645,pp\. 633–638\.External Links:[Document](https://dx.doi.org/10.1038/s41586-025-09422-z),[Link](https://www.nature.com/articles/s41586-025-09422-z)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. Du, M\. Tian, S\. Ronanki, S\. Rongali, S\. Bodapati, A\. Galstyan, A\. Wells, R\. Schwartz, E\. A\. Huerta, and H\. Peng \(2025\)Context length alone hurts llm performance despite perfect retrieval\.arXiv preprint arXiv:2510\.05381\.Cited by:[§5\.2](https://arxiv.org/html/2605.07076#S5.SS2.p1.1)\.
- Y\. Dudai, A\. Karni, and J\. Born \(2015\)The consolidation and transformation of memory\.Neuron88\(1\),pp\. 20–32\.External Links:[Document](https://dx.doi.org/10.1016/j.neuron.2015.09.004)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1)\.
- M\. Geva, R\. Schuster, J\. Berant, and O\. Levy \(2021\)Transformer feed\-forward layers are key\-value memories\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,Cited by:[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.p3.7)\.
- C\. Gulcehre, T\. L\. Paine, S\. Srinivasan, K\. Konyushkova, L\. Weerts, A\. Sharma, A\. Siddhant, A\. Ahern, M\. Wang, C\. Gu, W\. Macherey, A\. Doucet, O\. Firat, and N\. de Freitas \(2023\)Reinforced self\-training \(ReST\) for language modeling\.arXiv preprint arXiv:2308\.08998\.External Links:2308\.08998,[Document](https://dx.doi.org/10.48550/arXiv.2308.08998),[Link](https://arxiv.org/abs/2308.08998)Cited by:[§A\.11](https://arxiv.org/html/2605.07076#A1.SS11.p1.4),[§3\.2](https://arxiv.org/html/2605.07076#S3.SS2.p3.7)\.
- S\. Gururangan, A\. Marasović, S\. Swayamdipta, K\. Lo, I\. Beltagy, D\. Downey, and N\. A\. Smith \(2020\)Don’t stop pretraining: adapt language models to domains and tasks\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Online,pp\. 8342–8360\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.740),[Link](https://aclanthology.org/2020.acl-main.740/)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Guu, K\. Lee, Z\. Tung, P\. Pasupat, and M\. Chang \(2020\)Retrieval augmented language model pre\-training\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 3929–3938\.External Links:[Link](https://proceedings.mlr.press/v119/guu20a.html)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Howard and S\. Ruder \(2018\)Universal language model fine\-tuning for text classification\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 328–339\.External Links:1801\.06146,[Link](https://arxiv.org/abs/1801.06146)Cited by:[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1)\.
- C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, and B\. Ginsburg \(2024\)RULER: what’s the real context size of your long\-context language models?\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=kIoBbc76Sy)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:2106\.09685,[Link](https://arxiv.org/abs/2106.09685)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p3.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2605.07076#S3.SS2.p1.1)\.
- N\. Hu, E\. Mitchell, C\. Manning, and C\. Finn \(2023\)Meta\-learning online adaptation of language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 4418–4432\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.268),[Link](https://aclanthology.org/2023.emnlp-main.268/)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Jiang, Q\. Wu, C\. Lin, Y\. Yang, and L\. Qiu \(2023\)LLMLingua: compressing prompts for accelerated inference of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 13358–13376\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.825),[Link](https://aclanthology.org/2023.emnlp-main.825/)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska, D\. Hassabis, C\. Clopath, D\. Kumaran, and R\. Hadsell \(2017\)Overcoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p2.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- D\. Kumaran, D\. Hassabis, and J\. L\. McClelland \(2016\)What learning systems do intelligent agents need? complementary learning systems theory updated\.Trends in Cognitive Sciences20\(7\),pp\. 512–534\.External Links:[Document](https://dx.doi.org/10.1016/j.tics.2016.05.004)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33\.External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1)\.
- Z\. Li and D\. Hoiem \(2018\)Learning without forgetting\.IEEE transactions on pattern analysis and machine intelligence40\(12\),pp\. 2935–2947\.Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Li, C\. Chen, T\. Xu, Z\. Qin, J\. Xiao, Z\. Luo, and R\. Sun \(2024\)Preserving diversity in supervised fine\-tuning of large language models\.arXiv preprint arXiv:2408\.16673\.Cited by:[§A\.11](https://arxiv.org/html/2605.07076#A1.SS11.p3.2),[§A\.12](https://arxiv.org/html/2605.07076#A1.SS12.p4.1),[§A\.13](https://arxiv.org/html/2605.07076#A1.SS13.p3.1)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang \(2024\)Lost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638),[Link](https://aclanthology.org/2024.tacl-1.9/)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[Table 4](https://arxiv.org/html/2605.07076#A1.T4.3.6.3.2),[Table 5](https://arxiv.org/html/2605.07076#A1.T5.6.9.3.2)\.
- A\. Mallya and S\. Lazebnik \(2018\)PackNet: adding multiple tasks to a single network by iterative pruning\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,pp\. 7765–7773\.Cited by:[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.SSS0.Px1.p2.3)\.
- J\. L\. McClelland, B\. L\. McNaughton, and R\. C\. O’Reilly \(1995\)Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory\.Psychological Review102\(3\),pp\. 419–457\.External Links:[Document](https://dx.doi.org/10.1037/0033-295X.102.3.419)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1)\.
- M\. McCloskey and N\. J\. Cohen \(1989\)Catastrophic interference in connectionist networks: the sequential learning problem\.InPsychology of Learning and Motivation,Vol\.24,pp\. 109–165\.External Links:[Document](https://dx.doi.org/10.1016/S0079-7421%2808%2960536-8)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p2.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 17359–17372\.External Links:[Link](https://arxiv.org/abs/2202.05262)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1),[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.p3.7)\.
- K\. Meng, A\. S\. Sharma, A\. Andonian, Y\. Belinkov, and D\. Bau \(2023\)Mass\-editing memory in a transformer\.InInternational Conference on Learning Representations \(ICLR\),External Links:2210\.07229,[Link](https://arxiv.org/abs/2210.07229)Cited by:[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1)\.
- E\. Mitchell, C\. Lin, A\. Bosselut, C\. Finn, and C\. D\. Manning \(2022\)Fast model editing at scale\.InInternational Conference on Learning Representations \(ICLR\),External Links:2110\.11309,[Link](https://arxiv.org/abs/2110.11309)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1)\.
- A\. Mohtashami and M\. Jaggi \(2023\)Random\-access infinite context length for transformers\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://openreview.net/forum?id=7eHn64wOVy)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- Moonshot AI \(2026\)Kimi k2\.6 tech blog: advancing open\-source coding\.Note:Accessed: 2026\-05\-05External Links:[Link](https://www.kimi.com/blog/kimi-k2-6)Cited by:[§5\.1](https://arxiv.org/html/2605.07076#S5.SS1.SSS0.Px3.p1.1)\.
- J\. Mu, X\. L\. Li, and N\. Goodman \(2023\)Learning to compress prompts with gist tokens\.InAdvances in Neural Information Processing Systems 36 \(NeurIPS\),External Links:2304\.08467,[Link](https://arxiv.org/abs/2304.08467)Cited by:[§3\.3](https://arxiv.org/html/2605.07076#S3.SS3.p3.9)\.
- L\. O’Mahony, L\. Grinsztajn, H\. Schoelkopf, and S\. Biderman \(2024\)Attributing mode collapse in the fine\-tuning of large language models\.InICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models,Cited by:[§A\.11](https://arxiv.org/html/2605.07076#A1.SS11.p3.2),[§A\.13](https://arxiv.org/html/2605.07076#A1.SS13.p3.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Gray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. Lowe \(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Link](https://openreview.net/forum?id=TG8KACxEON)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez \(2023\)MemGPT: towards LLMs as operating systems\.arXiv preprint arXiv:2310\.08560\.External Links:[Link](https://arxiv.org/abs/2310.08560)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1),[§3\.3](https://arxiv.org/html/2605.07076#S3.SS3.p3.9)\.
- J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein \(2023\)Generative agents: interactive simulacra of human behavior\.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology,New York, NY, USA\.External Links:[Document](https://dx.doi.org/10.1145/3586183.3606763),[Link](https://doi.org/10.1145/3586183.3606763)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- J\. W\. Rae, A\. Potapenko, S\. M\. Jayakumar, and T\. P\. Lillicrap \(2019\)Compressive transformers for long\-range sequence modelling\.arXiv preprint arXiv:1911\.05507\.Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems 36 \(NeurIPS\),External Links:2305\.18290,[Link](https://arxiv.org/abs/2305.18290)Cited by:[§A\.11](https://arxiv.org/html/2605.07076#A1.SS11.p1.4)\.
- C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. Liu \(2020\)Exploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of Machine Learning Research21\(140\),pp\. 1–67\.External Links:[Link](https://jmlr.org/papers/v21/20-074.html)Cited by:[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1)\.
- P\. Rajpurkar, J\. Zhang, K\. Lopyrev, and P\. Liang \(2016\)SQuAD: 100,000\+ questions for machine comprehension of text\.InProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing,Austin, Texas,pp\. 2383–2392\.External Links:[Document](https://dx.doi.org/10.18653/v1/D16-1264),[Link](https://aclanthology.org/D16-1264/)Cited by:[Table 13](https://arxiv.org/html/2605.07076#A4.T13.2.3.1.1.1.1),[§1](https://arxiv.org/html/2605.07076#S1.p4.1),[§4\.1](https://arxiv.org/html/2605.07076#S4.SS1.SSS0.Px1.p1.1)\.
- W\. Ren, X\. Li, L\. Wang, T\. Zhao, and W\. Qin \(2024\)Analyzing and reducing catastrophic forgetting in parameter efficient tuning\.External Links:2402\.18865,[Link](https://arxiv.org/abs/2402.18865)Cited by:[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.p1.1)\.
- J\. Serrà, D\. Surís, M\. Miron, and A\. Karatzoglou \(2018\)Overcoming catastrophic forgetting with hard attention to the task\.InProceedings of the 35th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.80,pp\. 4548–4557\.Cited by:[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.SSS0.Px1.p2.3)\.
- H\. Shi, Z\. Xu, H\. Wang, W\. Qin, W\. Wang, Y\. Wang, Z\. Wang, S\. Ebrahimi, and H\. Wang \(2025\)Continual learning of large language models: a comprehensive survey\.ACM Computing Surveys58\(5\),pp\. 1–42\.Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- Q\. Sun, E\. Cetin, and Y\. Tang \(2025\)Transformer\-squared: self\-adaptive llms\.arXiv preprint arXiv:2501\.06252\.External Links:[Link](https://arxiv.org/abs/2501.06252)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px3.p1.1)\.
- G\. M\. van de Ven \(2025\)On the computation of the fisher information in continual learning\.arXiv preprint arXiv:2502\.11756\.Note:To appear in the ICLR 2025 Blogpost TrackExternal Links:[Link](https://arxiv.org/abs/2502.11756)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p2.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- P\. Wang, Z\. Li, N\. Zhang, Z\. Xu, Y\. Yao, Y\. Jiang, P\. Xie, F\. Huang, and H\. Chen \(2024a\)Wise: rethinking the knowledge memory for lifelong model editing of large language models\.Advances in Neural Information Processing Systems37,pp\. 53764–53797\.Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Wang and P\. Li \(2024\)LEMoE: advanced mixture of experts adaptor for lifelong model editing of large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 2551–2575\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.149),[Link](https://aclanthology.org/2024.emnlp-main.149/)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Wang, Y\. Zhu, H\. Liu, Z\. Zheng, C\. Chen, and J\. Li \(2024b\)Knowledge editing for large language models: a survey\.arXiv preprint arXiv:2310\.16218\.External Links:2310\.16218,[Link](https://arxiv.org/abs/2310.16218)Cited by:[§3\.1](https://arxiv.org/html/2605.07076#S3.SS1.p3.1)\.
- W\. Wang, L\. Dong, H\. Cheng, X\. Liu, X\. Yan, J\. Gao, and F\. Wei \(2023a\)Augmenting language models with long\-term memory\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://openreview.net/forum?id=BryMFPQ4L6)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- X\. Wang, T\. Chen, Q\. Ge, H\. Xia, R\. Bao, R\. Zheng, Q\. Zhang, T\. Gui, and X\. Huang \(2023b\)Orthogonal subspace learning for language model continual learning\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Singapore,pp\. 10658–10671\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.715),[Link](https://aclanthology.org/2023.findings-emnlp.715/)Cited by:[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.p1.1)\.
- J\. Wei, Y\. Tay, R\. Bommasani, C\. Raffel, B\. Zoph, S\. Borgeaud, D\. Yogatama, M\. Bosma, D\. Zhou, D\. Metzler,et al\.\(2022\)Emergent abilities of large language models\.arXiv preprint arXiv:2206\.07682\.Cited by:[§A\.12](https://arxiv.org/html/2605.07076#A1.SS12.p4.1)\.
- F\. Zenke, B\. Poole, and S\. Ganguli \(2017\)Continual learning through synaptic intelligence\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 3987–3995\.Cited by:[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.SSS0.Px1.p2.3)\.
- W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang \(2024\)MemoryBank: enhancing large language models with long\-term memory\.Proceedings of the AAAI Conference on Artificial Intelligence38\(17\),pp\. 19724–19731\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29946),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/29946)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p1.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px2.p1.1)\.
- A\. Zweiger, J\. Pari, H\. Guo, Y\. Kim, and P\. Agrawal \(2025\)Self\-adapting language models\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=JsNUE84Hxi)Cited by:[§1](https://arxiv.org/html/2605.07076#S1.p3.1),[§2](https://arxiv.org/html/2605.07076#S2.SS0.SSS0.Px3.p1.1),[§4\.2](https://arxiv.org/html/2605.07076#S4.SS2.p2.3)\.

## Appendix AContinual knowledge injection: implementation details

This appendix collects the reproducibility details for the experiments of[Section˜4](https://arxiv.org/html/2605.07076#S4)\.

### A\.1Selection prompt

The selection prompt is a Qwen chat\-template instance shown in[Figure˜4](https://arxiv.org/html/2605.07076#A1.F4)\. The prompt is parameterised by the passage title, the passage context, the implication string, the per\-round budgetBB, and the maximum layer index\. We instantiateB=10B=10and the maximum layer index at 27 for Qwen2\.5\-7B\.

```
<|im_start|>system
You are an assistant that selects which Transformer layers to update
so the model best memorizes the given passage and its implications.
<|im_end|>
<|im_start|>user
{title}
{context}

Implications:
{implications}

Select up to {budget} layer indices most critical for encoding this
knowledge. Each layer is an integer in [0, {max_layer}]. Be selective.
Output ONLY comma-separated integers, no descriptions.<|im_end|>
<|im_start|>assistant
```

Figure 4:Selection prompt template\.The model’s response is parsed deterministically: all integers in the output are extracted by regex, filtered to the range\[0,L−1\]\[0,L\-1\], deduplicated preserving first\-seen order, and truncated to the budgetBB\. Each retained layer index is expanded to its seven projection matrices for adapter attachment\. Malformed outputs that yield an empty parse produce an empty action, which in turn produces a near\-zero reward under the inner adaptation of[Equation˜5](https://arxiv.org/html/2605.07076#S3.E5); such rollouts are retained in the preference buffer but are, by construction, almost always the rejected element of any pair that includes them\.

### A\.2Implication generation

Implications are generated by the SEAL\-pretrained Qwen checkpoint after its second self\-training round, held frozen throughout all our experiments\. The generation prompt is shown in[Figure˜5](https://arxiv.org/html/2605.07076#A1.F5)\.

```
<|im_start|>system
You are an assistant tasked with analyzing the provided passage and
producing a list of implications derived directly or indirectly from
the content.
<|im_end|>
<|im_start|>user
{title}
{context}<|im_end|>
<|im_start|>assistant
```

Figure 5:Implication\-generation prompt template, used unmodified from SEAL\.Sampling uses temperature0\.70\.7, top\-pp0\.950\.95, a maximum of512512new tokens, and a single candidate per passage\. The resulting implication string is cached per split and reused verbatim across all runs \(ours and baselines\), so implication\-generation stochasticity does not contribute to variance between methods\.

For inner adaptation, each cached implication string is split on\-\-\-and newlines, capped at 30 sequences, and each sequence is templated as`\{title\}\\n\{sequence\}`\. The raw passage context is appended as an additional sequence, so the final training set for one rollout contains at most 31 sequences\. This templating is inherited from SEAL\.

### A\.3Inner LoRA configuration

[Table˜4](https://arxiv.org/html/2605.07076#A1.T4)lists the inner adaptation hyperparameters\.

Table 4:Inner LoRA configuration for one candidate rollout\.The commit policy merges the winning candidate’s adapter into the base weights at the end of each context and discards the others, so the running base modelθt\\theta\_\{t\}at stepttis a single fully\-consolidated Qwen2\.5\-7B checkpoint rather than a stack of adapters\.

### A\.4Outer IPO configuration

[Table˜5](https://arxiv.org/html/2605.07076#A1.T5)lists the outer preference\-optimisation configuration\.

Table 5:Outer\-loop IPO configuration\.The reference policyπref\\pi\_\{\\mathrm\{ref\}\}for the round\-rrIPO update is the round\-start base modelπθ\(r\)\\pi\_\{\\theta^\{\(r\)\}\}before any rollout\-induced drift, matching the formulation in[Section˜3\.2](https://arxiv.org/html/2605.07076#S3.SS2)\. After the IPO step, the updated base model becomesπθ\(r\+1\)\\pi\_\{\\theta^\{\(r\+1\)\}\}and is used both as the new rollout substrate for roundr\+1r\+1and as the new reference policy for that round’s outer update\.

### A\.5Reward and judge

The QA judge is a Qwen2\.5\-7B\-Instruct model served as a separate vLLM instance alongside the editable model\. Each judgement receives a single \(question, gold answer, student answer\) triple through the prompt in[Figure˜6](https://arxiv.org/html/2605.07076#A1.F6)and returns a binary decision\.

```
You are a grading assistant. Your job is to determine whether a
student’s answer correctly answers the question based solely on the
provided gold answer. Do not use any outside knowledge. The student
answer can include additional information, but it must at least
fully convey the gold answer and must not contradict it. Ignore
style, phrasing, or extra details that do not affect correctness.
Respond ONLY with ’yes’ or ’no’.
Question: {question}
Gold answer: {gold}
Student answer: {pred}
Is the student answer correct based solely on the gold answer?
Respond ’yes’ or ’no’.
```

Figure 6:Judge prompt\.The judge decodes greedily with a maximum of 8 new tokens\. A regex of the form\\b\(yes\|\|no\)\\bis run over the output and the last match is taken as the decision\. Outputs whose last match is not “yes” \(including outputs that produce no match at all\) are scored zero\. Accuracy on a single𝒬t\\mathcal\{Q\}\_\{t\}is the mean of the binary decisions over the roughly five questions in that passage\.

### A\.6Metric formulas

All main\-text metrics are summary statistics of the sequential\-edit accuracy matrixM∈\[0,1\]N×NM\\in\[0,1\]^\{N\\times N\}, withN=100N=100the length of the evaluation stream\. WritingMi,jM\_\{i,j\}for the accuracy on𝒬j\\mathcal\{Q\}\_\{j\}after editing throughcic\_\{i\}\(j≤ij\\leq i\), we report

Immediate acquisition=1N​∑iMi,i,\\displaystyle=\\;\\tfrac\{1\}\{N\}\\textstyle\\sum\_\{i\}M\_\{i,i\},Retention=1N−1​∑j<N−1MN−1,j,\\displaystyle=\\;\\tfrac\{1\}\{N\-1\}\\textstyle\\sum\_\{j<N\-1\}M\_\{N\-1,j\},

### A\.7Stream and round configuration

[Table˜6](https://arxiv.org/html/2605.07076#A1.T6)lists the continual\-stream configuration\.

Table 6:Stream configuration for[Section˜4](https://arxiv.org/html/2605.07076#S4)\.The training cache contains 250 shuffled passages;[Section˜4](https://arxiv.org/html/2605.07076#S4)uses the first two disjoint slices of 50 passages each\. The remaining 150 passages are reserved for extended\-rounds diagnostics \(LABEL:sec:app\-granularity\) and are not used by any main\-text result\.

### A\.8Baseline implementation details

##### QA Prompting\.

Prompting the base model with question only\.

##### Batch LoRA FT\.

LoRA attached to all 28 layers, trained jointly on the concatenated implications of all 100 evaluation passages for 10 epochs at learning rate2×10−42\\times 10^\{\-4\}\.

##### SEAL\.

The SEAL knowledge\-incorporation method applied to our continual stream, implemented as a sequential LoRA update per passage with the inner LoRA configuration of[Table˜4](https://arxiv.org/html/2605.07076#A1.T4)\.

##### SCoL, both variants\.

Identical configurations; the only difference is the value of the reward weightλ\\lambda\.

### A\.9Additional results

See standard errors in[Table˜7](https://arxiv.org/html/2605.07076#A1.T7)

Table 7:Standard errors for round\-2 SQuAD sequential editing results overK=3K=3inner\-training seeds\.![Refer to caption](https://arxiv.org/html/2605.07076v1/x6.png)Figure 7:Per\-layer change in selection count between the initial and final IPO updates\. Layers are sorted by SCoL’s largest gain \(left\) to largest drop \(right\)\. SCoL \(top, full reward\) concentrates onto a handful of layers and abandons most others, while SCoLλ=0\(bottom, acquisition only\) shifts selection mass differently\.
### A\.10Compute resources

All experiments ran on NVIDIA RTX PRO 6000 Blackwell GPUs \(96 GB each\)\. A single training round \(K=10K\{=\}10rollouts,T=50T\{=\}50passages\) took approximately4\.34\.3hours per configuration\. A single sequential matrix evaluation on theN=100N\{=\}100validation stream took5050to110110minutes on one GPU\.

### A\.11Outer Loop Algorithm Sweep: IPO, DPO, and ReST

We sweep three outer loop policy update algorithms while holding the inner loop, reward, stream, and rollout budget fixed\. Each selector is trained on one round ofT=50T\{=\}50passages withK=10K\{=\}10rollouts per passage, layer granularity,λ=1\\lambda\{=\}1, and evaluated on the sameN=100N\{=\}100validation stream\. We compare ReST\[[15](https://arxiv.org/html/2605.07076#bib.bib49)\], DPO\[[44](https://arxiv.org/html/2605.07076#bib.bib51)\], and IPO from Eq\. \(8\)\[[2](https://arxiv.org/html/2605.07076#bib.bib50)\]\. For each algorithm, we sweep learning rate, gradient accumulation, and the algorithm specific parameters\. All other hyperparameters match Table 2\.

Following the mode concentration concern in Section 3\.2, we report selector uniqueness as a diagnostic\. For each evaluated selector, we count the number of*distinct*selection completions emitted across the 100 validation passages, denoted uniq/100\. We also report top1%, the share of the most common completion\. uniq/100 is necessary but not sufficient for a useful selector, since high uniqueness can also reflect noisy selections\. We therefore read it jointly with immediate accuracy\.

Table 8:Outer loop algorithm sweep\. Retention uses the JSON summary key fromseq\_eval\.json, not the paper retention metric in Section A\.6\.IPO is the only update rule that jointly preserves uniq/100 above roughly 65 and reaches the strongest immediate accuracy, with 36\.60% to 38\.00% immediate accuracy and 65\.80% to 69\.30% retention atβ=0\.5\\beta\{=\}0\.5\. ReST shows the failure mode discussed in Section 3\.2\. Its aggressive setting reaches 41\.00% immediate accuracy, but retention drops to 27\.30%, while the most collapsed ReST setting has only 14 distinct completions and top1% of 61%\. This trend is consistent with mode concentration under cross entropy style updates\[[39](https://arxiv.org/html/2605.07076#bib.bib47),[27](https://arxiv.org/html/2605.07076#bib.bib78)\]\. DPO retains high uniq/100, often between 77 and 83, but underperforms IPO on the joint of immediate accuracy and retention\. This sweep motivates IPO withβ=0\.5\\beta\{=\}0\.5as the headline outer loop algorithm, and we use this configuration for all main text results\.

### A\.12Granularity Comparison: Layer, Module, and Per Projection

The action space in Eq\. \(4\) ranges overLLstructural units of the base model\. We instantiate three choices in the same codebase, with budgets chosen to keep the per rollout adapter parameter count comparable\. The layer setting usesL=28L\{=\}28and budgetB=10B\{=\}10, where each selected layer attaches LoRA adapters to seven projection modules\. The module setting usesL=56L\{=\}56and budgetB=20B\{=\}20, where each layer is split into attention and MLP units\. The per projection setting usesL=196L\{=\}196with budgetsB=70B\{=\}70andB=100B\{=\}100, where each q, k, v, o, gate, up, and down projection is a separate slot\. All settings share the inner LoRA configuration in Table 1, the IPO configuration in Table 2, the same stream,K=10K\{=\}10, and both reward variants\.

Table 9:Round 2 granularity comparison\. SCoL uses the full reward\. SCoLλ=0removes the forgetting term from the reward\.Layer and module granularity yield comparable Round 2 immediate accuracy, around 35\.00% across both reward variants, but the layer configuration is more stable across rounds and is therefore used in the main text\. Per projection granularity underperforms by roughly five to ten absolute points in immediate accuracy, despite its larger nominal action space\. Since uniq/100 remains high, often above 90, the failure is better explained as selector*noise*rather than selector collapse\.

This pattern suggests that action space expressivity is bottlenecked by structured generation\. As granularity becomes finer, the policy must emit longer and more constrained outputs, moving from comma separated layer indices to module pairs and then projection pairs\. With budgets of 70 or 100, the policy must produce a long list of valid pairs whose indices and projection names obey the parser grammar\. A 7B base policy struggles with this output length and schema fidelity, so the effective entropy of useful rollouts shrinks even as the nominal action space grows\.

Two future directions follow\. First, the gap between layer and per projection granularity may narrow with stronger base policies, since long structured generation is plausibly scaling sensitive\[[58](https://arxiv.org/html/2605.07076#bib.bib79)\]\. We state this only as a hypothesis\. Second, the outer loop update can be made more diversity preserving at extreme structured lengths, either through entropy regularization\[[27](https://arxiv.org/html/2605.07076#bib.bib78)\]or through an objective that scores list*compositions*rather than independent positions\. We leave both directions to future work\.

### A\.13Extended Round Dynamics and Mode Collapse

We also continue the outer loop beyond the two rounds reported in the main text\. This experiment is diagnostic rather than a headline result\. We run two additional rounds, R3 and R4, for layer and module granularity under both reward variants\. The inner loop, IPO objective, stream, and budget are unchanged\. After each round, we record uniq/100 and immediate accuracy\.

Table 10:Extended round diagnostics\. Each cell reports immediate accuracy and uniq/100\.uniq/100 declines with additional rounds, and the full reward variant collapses faster than the acquisition only variant\. On layer granularity, SCoL drops from 82 distinct completions at R2 to 5 at R4, while SCoLλ=0retains 33 at R4\. The module setting shows the same direction, with the full reward variant also collapsing in immediate accuracy to 6\.70% at R3\.

Round 2 should therefore be read as an early stopping point for selector quality, not as the asymptotic behavior of the fixed objective\. With the same IPO loss and buffer construction, later rounds concentrate probability mass on a small number of high reward actions observed during training and reduce exploration\. This is the mode concentration mechanism flagged in Section 3\.2 and observed in cross entropy and preference style fine tuning of LLMs\[[39](https://arxiv.org/html/2605.07076#bib.bib47),[27](https://arxiv.org/html/2605.07076#bib.bib78)\]\. For this reason, the main text reports R0 to R2 and treats further round scaling as a regularization problem for future work\.

### A\.14Layerwise Fisher computation

For each passagecc, letℐ​\(c\)\\mathcal\{I\}\(c\)denote its generated implication lines\. Fisher scores are computed on the running consolidated base model before attaching a new LoRA adapter for the passage\. For each implication linex∈ℐ​\(c\)x\\in\\mathcal\{I\}\(c\), we compute the autoregressive sum loss

ℒ​\(x;θ\)=−∑t=1\|x\|log⁡pθ​\(xt∣x<t\),\\mathcal\{L\}\(x;\\theta\)=\-\\sum\_\{t=1\}^\{\|x\|\}\\log p\_\{\\theta\}\(x\_\{t\}\\mid x\_\{<t\}\),whereθ\\thetadenotes the current consolidated base parameters\. We estimate diagonal Fisher importance by squared gradients:

Fj​\(c\)=1\|ℐ​\(c\)\|​∑x∈ℐ​\(c\)\(∂ℒ​\(x;θ\)∂θj\)2\.F\_\{j\}\(c\)=\\frac\{1\}\{\|\\mathcal\{I\}\(c\)\|\}\\sum\_\{x\\in\\mathcal\{I\}\(c\)\}\\left\(\\frac\{\\partial\\mathcal\{L\}\(x;\\theta\)\}\{\\partial\\theta\_\{j\}\}\\right\)^\{2\}\.Let𝒫ℓ\\mathcal\{P\}\_\{\\ell\}be the parameters of the seven base projection matrices in layerℓ\\ellthat are eligible for LoRA attachment\. The layerwise Fisher score is

Fℓ​\(c\)=∑j∈𝒫ℓFj​\(c\),ℓ∈\{0,…,27\}\.F\_\{\\ell\}\(c\)=\\sum\_\{j\\in\\mathcal\{P\}\_\{\\ell\}\}F\_\{j\}\(c\),\\qquad\\ell\\in\\\{0,\\ldots,27\\\}\.This yields a 28 dimensional Fisher vectorF​\(c\)F\(c\)for each passage\. These scores are used only for analysis and are never provided to the LLM when it generates layer selections\.

## Appendix BLong\-context Consolidation

### B\.1Hyperparameters

#### B\.1\.1Inner LoRA configuration

We retain the same LoRA configuration from[Section˜A\.9](https://arxiv.org/html/2605.07076#A1.SS9)and[Table˜4](https://arxiv.org/html/2605.07076#A1.T4)\. Additionally, all our chunks are of a fixed20482048token length calculated using theQwen2\.5\-7Btokenizer and we create a training set by further chunking these into128128token size subchunks\.

#### B\.1\.2Outer IPO configuration

We retain mostly the same IPO configuration from[Section˜A\.9](https://arxiv.org/html/2605.07076#A1.SS9)and[Table˜5](https://arxiv.org/html/2605.07076#A1.T5)\. We only change the preference margin threshold from0\.050\.05to0\.30\.3\. That is, given candidatesc1,c2c\_\{1\},c\_\{2\}with rewardsr1,r2r\_\{1\},r\_\{2\}respectively andr1\>r2r\_\{1\}\>r\_\{2\}, we put\{c1,c2\}\\\{c\_\{1\},c\_\{2\}\\\}in the IPO buffer iffr1−r2\>0\.2r\_\{1\}\-r\_\{2\}\>0\.2\.

### B\.2Implementation Details

##### SCoL\.

We retain the majority of the setup from[Appendix˜A](https://arxiv.org/html/2605.07076#A1), with the modifications presented in[Section˜B\.1](https://arxiv.org/html/2605.07076#A2.SS1)

##### In\-context settings\.

We use the baseQwen2\.5\-7Bmodel with temperature=1=1implemented using vLLM\.

ForSummarization, we promptKimi\-K2\.6using the template shown in[Figure˜8](https://arxiv.org/html/2605.07076#A2.F8)\. We setmaxTokensto 10,000 to discourage degenerate behavior where the model copies large portions of the original passage instead of compressing it\.

```
Please provide a dense and concise summary of the following passage in under
{maxTokens} tokens keeping all the important information:\n\n{context}\n\nSummary:
```

Figure 8:Summarization prompt used for Kimi\-K2\.6\.
##### Test\-time training\.

For both Batch TTT and Sequential FT, we attach LoRA adapters to all 28 transformer layers\. Training is performed either jointly on the full passage \(Batch TTT\) or incrementally over sequential chunks \(Sequential FT\)\. Each passage is evaluated using freshly initialized LoRA adapters trained under the same inner\-loop test\-time training configuration as given in[Section˜B\.1](https://arxiv.org/html/2605.07076#A2.SS1)\.

#### B\.2\.1Dataset Processing

Our goal is to evaluate whether long contexts can be continually consolidated into model parameters\. We therefore use LongBench\-v2, which contains naturally long documents \(e\.g\., books and multi\-section texts\) paired with reasoning\-based question\-answer tasks\.

Each document is treated as an independent context and processed as a stream of 2048\-token segments, simulating a setting where information arrives sequentially\. For training, we use 10 passages with lengths between 16k and 32k tokens\. We sample 40 different passages for evaluation and an additional 20 passages of lengths between 32k and 64k tokens\.

Tokenization and segmentation are performed using theQwen2\.5\-7Btokenizer\. All dataset splits are created with a fixed random seed of 42 for reproducibility\.

#### B\.2\.2Compute Resources

All experiments for this section are conducted on NVIDIA H200 GPUs with 8 CPUs\. Meta\-training \(2 rounds over 5 passages each\) takes approximately 8 hours\. Evaluation over the full benchmark \(40 short and 20 long passages\) requires approximately 6 hours\.

### B\.3Additional Results

##### Passage Log\-likelihoods\.

We further compare the final Log\-Likelihoods between batch TTT, sequential FT, and SCoL in[Table˜11](https://arxiv.org/html/2605.07076#A2.T11)\. Consistent with the main results, Batch TTT achieves the highest likelihood in both regimes \(−1\.26\-1\.26\), indicating that full\-sequence optimization yields the best fit to the data distribution\. In contrast, Sequential FT performs substantially worse \(−2\.32\-2\.32short,−2\.51\-2\.51long\), reinforcing that naïve online updates lead to degraded representations and poor predictive calibration\.

Table 11:Long\-context consolidation comparison of final log\-likelihoods across short and long regimes\.SCoL consistently improves over Sequential FT in both regimes \(−1\.44\-1\.44vs\.−2\.32\-2\.32in short,−1\.82\-1\.82vs\.−2\.51\-2\.51in long\), indicating that structured consolidation mitigates the degradation in likelihood associated with continual updates\. Notably, the gap between SCoL and Batch TTT is smaller in the short\-context regime \(0\.18\) than in the long\-context regime \(0\.56\), mirroring the trends observed in accuracy\.

We hypothesize that this widening gap reflects the same stability–plasticity tradeoff observed earlier\. In the short\-context setting, SCoL’s forgetting\-aware objective enables updates that remain close to the optimal batch solution, yielding likelihoods comparable to Batch TTT\. However, as context length increases, the constraint imposed by the forgetting term may limit the model’s ability to fully adapt to the extended sequence, leading to lower likelihood relative to Batch TTT\.

Overall, these results support the conclusion that SCoL provides a strong approximation to batch retraining while maintaining stability, substantially improving over naïve sequential updates, though some loss in optimality remains when full\-context adaptation is required\.

##### Layer selection frequencies\.

We also show the top layer selection frequencies for our Long Context Consolidation Setting in[Figure˜9](https://arxiv.org/html/2605.07076#A2.F9)\. Similar to the results in[Section˜4](https://arxiv.org/html/2605.07076#S4), we find that L27 is the selected most often but here we find that L26 and L25 are selected relatively less\. Additionally, the middle layers are selected least often matching the SQuAD setting\.

![Refer to caption](https://arxiv.org/html/2605.07076v1/x7.png)Figure 9:Layer selections for SCoL for the long context setting for short and long regimes\.
##### Full results table with standard errors\.

Table 12:Long\-context consolidation comparison across two context\-length regimes \(short: 16k–32k and long: 32k–64k tokens\)\.We provide the full results table for the long context experiments, along with the standard errors in table[Table˜12](https://arxiv.org/html/2605.07076#A2.T12)\.

## Appendix CBroader Impacts

The proposed SCoL framework enables long\-context internalization through persistent weight updates, offering significant improvements in computational efficiency and long\-term agent memory\. However, this shift from transient context to persistent weights introduces privacy risks, as consolidated sensitive information may be harder to redact\. Furthermore, continual self\-adaptation may inadvertently reinforce biases or internalize misinformation present in the input stream\.

## Appendix DLicenses

We provide details regarding the licenses of external assets used in this work, presented in Table[13](https://arxiv.org/html/2605.07076#A4.T13)\.

Table 13:Licenses for assets used in our experiments\.\*The majority of the Qwen series \(e\.g\., Qwen1\.5, Qwen2, Qwen2\.5\) are licensed under Apache 2\.0, with a few exceptions for specific legacy sizes governed by the Tongyi Qianwen License Agreement\.