When Does Continual Learning Require Learning

# When Does Continual Learning Require Learning
Source: [https://arxiv.org/html/2607.07847](https://arxiv.org/html/2607.07847)
Anne Harrington1,\*Nayan Saxena2Michael Murphy1Anastasia Borovykh3 Zeyu Yun1Sridhar Kamath2Ara Eindra Kyi2 Trevor Darrell1Jitendra Malik1Yutong Bai1,\*

1UC Berkeley2Independent3Capital Fund Management \{anneko, yutongbai\}@berkeley\.edu

###### Abstract

As large language models \(LLMs\) become increasingly capable, the next question is how can we enable models to continually learn? Today, the field largely frames this as a problem of context management and mitigating forgetting\. We argue this framing is incomplete: continual learning is fundamentally about increasing model competence as the world changes\. We disentangle this change along two axes — space, where the model encounters new domains, and time, where the underlying data drifts under a fixed task\. This framing lets us study continual learning under realistic conditions: new domains arrive over time, facts drift past their training cutoff, and agentic interactions accumulate state across episodes\. To evaluate methods under this setting, we recast widely used LLM benchmarks as sequential problems and introduce a single mechanism\-agnostic protocol that compares prompt\-based methods \(GEPA, ACE\), supervised learning \(SFT, SDFT\), reinforcement learning \(GRPO, SDPO\), and context compression \(Cartridges, In\-place TTT\)\. Prompt\-based methods fit each new stage quickly but degrade on future tasks\. Distillation\-based methods accumulate knowledge stably but struggle to update outdated facts\. Context compression improves efficiency without substantially improving the ability to learn new tasks\. Online reinforcement learning adapts most effectively to knowledge updates but remains sensitive to noisy reward signals\. Overall, our results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors, determining when adaptation must be learned inside model weights and when it can be achieved through external scaffolding\. We hope that understanding where each method succeeds and fails will guide the design of stronger continual learning systems\.†

## 1Introduction

11footnotetext:Equal Contribution\.†Code:[https://github\.com/anneharrington/studying\-cl](https://github.com/anneharrington/studying-cl)![Refer to caption](https://arxiv.org/html/2607.07847v1/figure_1_new.png)

Figure 1:A unified continual learning framework spanning axes of change and update mechanisms\.We identify four axes along which tasks evolve: domain shift, fact update, temporal drift, and agentic state\. All methods are cast within a common framework where an experience stream triggers updates to one of three internal components: context, parameters, or memory/state\. Existing methods are unified accordingly: prompting methods \(GEPA, ACE\) modify context; fine\-tuning methods \(SFT, SDFT\) and reinforcement learning methods \(GRPO, SDPO\) update parameters; and memory/compression methods \(Cartridges, In\-place TTT\) maintain external state\.Traditionally, continual learning has been defined as mitigating catastrophic forgetting\[[40](https://arxiv.org/html/2607.07847#bib.bib40)\]:*how do we learn taskBBwithout forgetting taskAA?*One approach to this problem is to augment or compose model parameters as proposed in superposition\[[6](https://arxiv.org/html/2607.07847#bib.bib6)\], adversarial networks\[[9](https://arxiv.org/html/2607.07847#bib.bib9)\]and side networks\[[57](https://arxiv.org/html/2607.07847#bib.bib57)\]\. However, this setup is not realistic in the current paradigm of large language models \(LLMs\), which often rely on large scale pre\-, mid\-, and post\-training to acquire general capability and after deployment need to be adapted to a*broad*range of potential use cases\. At the same time, standard LLM evaluations focus on measuring capabilities on static tasks such as general knowledge\[[51](https://arxiv.org/html/2607.07847#bib.bib51)\], domain specific abilities\[[17](https://arxiv.org/html/2607.07847#bib.bib17),[23](https://arxiv.org/html/2607.07847#bib.bib23),[45](https://arxiv.org/html/2607.07847#bib.bib45),[16](https://arxiv.org/html/2607.07847#bib.bib16)\], and challenging frontier problems\[[14](https://arxiv.org/html/2607.07847#bib.bib14),[38](https://arxiv.org/html/2607.07847#bib.bib38)\]; these however fail to measure the ability to become more competent under conditions that evolve through time – which is critical for practical deployment\.

Without a unified framework, the community approached continual learning as a memory problem solved with retrieval\-augmented generation\[[27](https://arxiv.org/html/2607.07847#bib.bib27)\], in\-context learning\[[3](https://arxiv.org/html/2607.07847#bib.bib3)\], smart context management\[[29](https://arxiv.org/html/2607.07847#bib.bib29),[56](https://arxiv.org/html/2607.07847#bib.bib56)\]and harnesses\[[59](https://arxiv.org/html/2607.07847#bib.bib59),[54](https://arxiv.org/html/2607.07847#bib.bib54),[26](https://arxiv.org/html/2607.07847#bib.bib26)\]; as a steering problem solved with prompt optimization\[[43](https://arxiv.org/html/2607.07847#bib.bib43),[47](https://arxiv.org/html/2607.07847#bib.bib47),[2](https://arxiv.org/html/2607.07847#bib.bib2)\]; as a reasoning problem solved with reinforcement learning and distillation\[[35](https://arxiv.org/html/2607.07847#bib.bib35),[60](https://arxiv.org/html/2607.07847#bib.bib60),[19](https://arxiv.org/html/2607.07847#bib.bib19),[48](https://arxiv.org/html/2607.07847#bib.bib48)\]; and as a compression problem solved with architectural changes\[[10](https://arxiv.org/html/2607.07847#bib.bib10),[30](https://arxiv.org/html/2607.07847#bib.bib30),[49](https://arxiv.org/html/2607.07847#bib.bib49),[11](https://arxiv.org/html/2607.07847#bib.bib11)\]\. Each of these takes a different bet on what continual learning actually requires, but the field still lacks unified insights across these methods to direct progress\.

In our work, we provide a unified framework to define and measure continual learning in modern LLMs\. We define continual learning as the problem of increasing competence as the world changes\. We consider change along two axes:*space*and*time*, as depicted in Figure[1](https://arxiv.org/html/2607.07847#S1.F1)\. Space corresponds to the classical notion of domain or task shift, where the model is asked to acquire a new skill on a different distribution without losing the previous one\. Time corresponds to a different and, more common form of change in real deployed systems: the task stays the same, but the underlying world drifts\. Time itself can be further split into two sub\-regimes that the literature usually conflates\. Slow trends \(e\.g\. yearly financial filings\) ask the model to stay aligned; the best strategy here is to keep extracting whatever structure survives across years and ignore year\-specific noise\. Discrete fact changes \(e\.g\. Wikipedia revisions\) ask the opposite – the right move is to actually rewrite the affected belief while leaving the rest of the model’s knowledge untouched\. A third sub\-regime is agentic accumulation: the task family itself stays fixed \(e\.g\., actions inside a web app\), but the environment’s state drifts as a direct consequence of the model’s own prior actions, so that what one step leaves behind becomes part of the task the model faces at the next step\. This second sub\-regime is the LLM analogue of the frame problem\[[39](https://arxiv.org/html/2607.07847#bib.bib39)\]: when the world updates, a learner must decide which of its beliefs are now outdated and which can remain fixed\.

To measure continual learning under our definition, we cast widely used LLM evaluations as problems that unfold sequentially under a single mechanism\-agnostic protocol, so prompt, weight, and architectural updates can be compared on equal ground\. On a common backbone \(Qwen3\-8B\)\[[53](https://arxiv.org/html/2607.07847#bib.bib53)\], we evaluate eight methods across four families: prompt optimization \(GEPA\[[2](https://arxiv.org/html/2607.07847#bib.bib2)\], ACE\[[59](https://arxiv.org/html/2607.07847#bib.bib59)\]\), offline supervised updates \(SFT, SDFT\[[48](https://arxiv.org/html/2607.07847#bib.bib48)\]\), online reinforcement learning \(GRPO\[[46](https://arxiv.org/html/2607.07847#bib.bib46)\], SDPO\[[19](https://arxiv.org/html/2607.07847#bib.bib19)\]\), and context\-compression \(Cartridges\[[10](https://arxiv.org/html/2607.07847#bib.bib10)\], In\-place TTT\[[11](https://arxiv.org/html/2607.07847#bib.bib11)\]\)\.

Interestingly, we observe a consistent set of tradeoffs across methods: \(i\) Prompting\-based approaches such as GEPA and ACE fit quickly and heavily to past trajectories, achieving strong backward accuracy, but encounter sharp degradation on future tasks\. \(ii\) Distillation\-based methods such as SDFT and SDPO accumulate knowledge more stably over time, yet struggle to rapidly incorporate new information or update outdated knowledge\. \(iii\) Context compression approaches, including Cartridges and In\-place TTT, improve efficiency and memory management but do not substantially improve the ability to learn new tasks\. \(iv\) In contrast, online reinforcement learning methods such as GRPO adapt more effectively to knowledge updating, though they remain highly sensitive to noisy or unstable reward signals\. \(v\) On the agentic axis, where the stage ordering is generated by the agent’s own actions rather than imposed by us, both a prompt\-based playbook \(ACE\) and weight\-based fine\-tuning \(SFT\) compound experience and beat their zero\-shot baselines at every chain length, though absolute success still decays as the chain grows\.

All together, we make the following contributions:

- •We formalize continual learning in LLMs as the problem of increasing competence over time\. Through our protocol, we provide a unified perspective that allows us to consider both parametric and non\-parametric update methods for continual learning\. We adapt these methods to the continually learning setting and align their evaluation metrics\.
- •We introduce a broad suite of sequential LLM tasks to evaluate continual learning\. These including domain adaptation, agentic tasks, financial analysis, and temporally\-dependent knowledge updates\.
- •We find that prompt\-based methods alone are insufficient across most regimes, calling for actual learning\. Different learning methods each have their own strengths depending on how the task and data shift over time\.

## 2Related Works

Continual Learning\.Traditionally, the field has considered continual learning as preventing the problem of catastrophic forgetting\[[40](https://arxiv.org/html/2607.07847#bib.bib40),[13](https://arxiv.org/html/2607.07847#bib.bib13)\]\. In this context, the problem is that sequential parameter updates can overwrite knowledge needed for previous tasks; this led to a focus on designing methods that reduce task interference\[[22](https://arxiv.org/html/2607.07847#bib.bib22),[15](https://arxiv.org/html/2607.07847#bib.bib15),[9](https://arxiv.org/html/2607.07847#bib.bib9),[57](https://arxiv.org/html/2607.07847#bib.bib57),[6](https://arxiv.org/html/2607.07847#bib.bib6)\]\. From another angle, continual learning has been positioned as maintaining plasticity\[[8](https://arxiv.org/html/2607.07847#bib.bib8)\], where models must not only remember old tasks, but also have the ability to learn new ones\[[36](https://arxiv.org/html/2607.07847#bib.bib36),[42](https://arxiv.org/html/2607.07847#bib.bib42)\]\.

Non\-parametric updates\.At deployment, we can adapt LLMs without weight changes by changing the inference\-time environment around a fixed model\. One family of methods changes the information available to the model, for example through retrieval\-augmented generation \(RAG\)\[[27](https://arxiv.org/html/2607.07847#bib.bib27)\], external memory, reusable long\-context representations\[[10](https://arxiv.org/html/2607.07847#bib.bib10)\], context compression\[[29](https://arxiv.org/html/2607.07847#bib.bib29)\], or recursive access to large external contexts\[[56](https://arxiv.org/html/2607.07847#bib.bib56)\]\. A closely related family changes the behavioral scaffold through which that information is used, including demonstrations\[[3](https://arxiv.org/html/2607.07847#bib.bib3)\], prompt instructions and agent contexts\[[2](https://arxiv.org/html/2607.07847#bib.bib2),[59](https://arxiv.org/html/2607.07847#bib.bib59),[55](https://arxiv.org/html/2607.07847#bib.bib55)\]and multi\-stage LM programs\[[44](https://arxiv.org/html/2607.07847#bib.bib44),[25](https://arxiv.org/html/2607.07847#bib.bib25),[28](https://arxiv.org/html/2607.07847#bib.bib28)\]\. More broadly, agentic systems can adapt by modifying their surrounding harnesses\[[54](https://arxiv.org/html/2607.07847#bib.bib54),[26](https://arxiv.org/html/2607.07847#bib.bib26)\], learn component\-level reward specifications\[[52](https://arxiv.org/html/2607.07847#bib.bib52)\]or improve through external programs and agent code modifications\[[4](https://arxiv.org/html/2607.07847#bib.bib4),[43](https://arxiv.org/html/2607.07847#bib.bib43),[58](https://arxiv.org/html/2607.07847#bib.bib58),[47](https://arxiv.org/html/2607.07847#bib.bib47)\]\.

Gradient\-based \(parametric\) updates\.Another adaption approach is to update weights or activations\. At one extreme, full fine\-tuning modifies all parameters of the model; while effective, this regime is often computationally expensive and unrealistic for frequent or online adaptation\. Activation editing\[[20](https://arxiv.org/html/2607.07847#bib.bib20)\]uses task vectors applied to all parameters\. More commonly, work studies*parameter\-efficient*methods, such as LoRA\[[18](https://arxiv.org/html/2607.07847#bib.bib18)\]and adapter\-based tuning\[[30](https://arxiv.org/html/2607.07847#bib.bib30),[33](https://arxiv.org/html/2607.07847#bib.bib33)\], which restrict updates to small subsets of parameters\. An important but underexplored regime involves severely constrained updates—very few gradient steps and tightly limited compute—closer to the conditions under which continual adaptation would need to occur in deployed systems\. Continual learning methods differ not only in where they store updates, but also in what learning signal drives the update\. OPSD\[[60](https://arxiv.org/html/2607.07847#bib.bib60),[35](https://arxiv.org/html/2607.07847#bib.bib35)\], SDFT\[[48](https://arxiv.org/html/2607.07847#bib.bib48)\]and SDPO\[[19](https://arxiv.org/html/2607.07847#bib.bib19)\]learn from self\-distillation, GRPO\[[46](https://arxiv.org/html/2607.07847#bib.bib46)\]and h1\[[41](https://arxiv.org/html/2607.07847#bib.bib41)\]optimize end\-of\-task rewards, LoRD\[[32](https://arxiv.org/html/2607.07847#bib.bib32)\]uses teacher–student distillation, sparse memory finetuning\[[30](https://arxiv.org/html/2607.07847#bib.bib30)\]localizes the next\-token objective to sparsely accessed memory slots, test\-time training\[[11](https://arxiv.org/html/2607.07847#bib.bib11)\]performs input\-conditioned weight updates during inference\.

## 3Method

### 3\.1A Protocol for Continual Learning

Setup\.Typically, continual\-learning evaluations are tightly coupled to a particular method: weight\-update methods use task accuracy with a fixed fine\-tuning budget; prompt\-optimization often uses few\-shot generalization with a fixed token budget; agentic frameworks rely on end\-to\-end pass rate\. As a result, prompt, weight, and architectural updates are rarely compared fairly against one another\. To gather broader insights about what methods are necessary for continual learning, we define a simple protocol that is compatible with any update operator\. To do so, we consider all forms of updates \(prompting, full weight, sparse weight/activation\) as potential strategies, and group methods into families based on which aspect of the model they change during optimization \(Tab\.[1](https://arxiv.org/html/2607.07847#S3.T1)\)\. Formally:

A learner sees stages𝒯1,…,𝒯K\\mathcal\{T\}\_\{1\},\\dots,\\mathcal\{T\}\_\{K\}in fixed order\. Each𝒯k\\mathcal\{T\}\_\{k\}specifies a training distribution𝒟ktr\\mathcal\{D\}\_\{k\}^\{\\text\{tr\}\}and an evaluation set𝒟keval\\mathcal\{D\}\_\{k\}^\{\\text\{eval\}\}\. Between stages the learner applies an update operator

θk=𝒰k​\(θk−1,𝒟ktr\),θ0=θbase\.\\theta\_\{k\}\\;=\\;\\mathcal\{U\}\_\{k\}\\\!\\left\(\\theta\_\{k\-1\},\\,\\mathcal\{D\}\_\{k\}^\{\\text\{tr\}\}\\right\),\\qquad\\theta\_\{0\}=\\theta\_\{\\text\{base\}\}\.𝒰k\\mathcal\{U\}\_\{k\}is unrestricted: it may modify model weights, edit a system prompt, write to or read from external memory, attach an adapter, or perform no update\. A method is fully specified by its family of𝒰k\\mathcal\{U\}\_\{k\}and a per\-stage compute budgetCC\(held constant across methods within a benchmark\)\.

Forgetting matrix\.After each stage we evaluate on every stage’s eval set, producing

Ri,j=acc​\(θi,𝒟jeval\),i∈\{0,…,K\},j∈\{1,…,K\},R\_\{i,j\}\\;=\\;\\mathrm\{acc\}\\\!\\left\(\\theta\_\{i\},\\,\\mathcal\{D\}\_\{j\}^\{\\text\{eval\}\}\\right\),\\qquad i\\in\\\{0,\\dots,K\\\},\\;j\\in\\\{1,\\dots,K\\\},with row0the base model before any update\. We report two transfer quantities:

BWT=1K−1​∑k=1K−1\(RK,k−Rk,k\),\\displaystyle=\\tfrac\{1\}\{K\-1\}\\textstyle\\sum\_\{k=1\}^\{K\-1\}\\big\(R\_\{K,k\}\-R\_\{k,k\}\\big\),FWT=1K−1​∑k=2K\(Rk−1,k−R0,k\)\.\\displaystyle=\\tfrac\{1\}\{K\-1\}\\textstyle\\sum\_\{k=2\}^\{K\}\\big\(R\_\{k\-1,k\}\-R\_\{0,k\}\\big\)\.BWT\[[33](https://arxiv.org/html/2607.07847#bib.bib33)\]measures whether later stages improve or damage earlier ones \(negativeBWTis catastrophic forgetting\)\.FWTmeasures whether prior stages prepare the model for stages it has not yet trained on, and is the sharper signal: it tests acquisition of structure that transfers, rather than fitting the current stage at the cost of the rest\.

### 3\.2Update Methods

We evaluate eight methods across four families: prompt optimization, supervised weight updates, reinforcement learning, and per\-stage architectural updates\. All use Qwen3\-8B \(non\-thinking\)\[[53](https://arxiv.org/html/2607.07847#bib.bib53)\]asθ0\\theta\_\{0\}and run under the protocol of §[3](https://arxiv.org/html/2607.07847#S3)\. Table[1](https://arxiv.org/html/2607.07847#S3.T1)states what each method carries across the stage boundary; the per\-family paragraphs that follow specify each𝒰k\\mathcal\{U\}\_\{k\}and its sequential adaptation\.

Table 1:The eight methods, organized by what crosses the stage boundary\. “Sequential adaptation” is the modification that lifts a single\-stage method to the staged protocol of §[3](https://arxiv.org/html/2607.07847#S3)\. Prompting refers to methods that update in natural language\. Offline corresponds to supervised learning whereas online refers to reinforcement learning\. Lastly compression notes methods that compress context in activation or weight space\.Prompt optimization \(GEPA, ACE\)Both methods leave model weights unchanged\. GEPA\[[2](https://arxiv.org/html/2607.07847#bib.bib2)\]optimizes the system prompt through a reflect\-and\-mutate loop scored on a validation minibatch; the best\-scoring mutations propagate forward\. ACE\[[59](https://arxiv.org/html/2607.07847#bib.bib59)\]maintains a markdown playbook edited incrementally through explicit add/keep/drop operations, which the authors show resists brevity\-bias and context collapse\. In our protocol,𝒰k\\mathcal\{U\}\_\{k\}initializes optimization for stagekkfrom the optimized state at the end of stagek−1k\{\-\}1\(the evolved prompt for GEPA, the playbook for ACE\), prepended to the default instruction for𝒯k\\mathcal\{T\}\_\{k\}\. No replay of prior data is performed\. Apparent forward transfer is therefore attributable to the carried prompt or playbook, not to revisiting training examples\.

Supervised / Offline Learning \(SFT, SDFT\)SFTfine\-tunes all parameters on𝒟ktr\\mathcal\{D\}\_\{k\}^\{\\text\{tr\}\}, resuming fromθk−1\\theta\_\{k\-1\}\.SDFT\[[48](https://arxiv.org/html/2607.07847#bib.bib48)\]is self\-distillation under a forward\-KL loss against soft targets from a teacher; in the original work the teacher is the model itself on demonstration\-conditioned inputs\. Our sequential adaptation: at stagekk, the teacher isθk−1\\theta\_\{k\-1\}, so the student learns the new stage under a regularizer pulling it toward its own previous\-stage distribution\. This is the smallest modification that lifts SDFT to a continual\-learning operator and is the only modification we make\. SFT and SDFT share a per\-stage update budget withinCCand differ only in the loss applied at each step\.

Reinforcement / Online Learning \(GRPO, SDPO\)GRPO\[[46](https://arxiv.org/html/2607.07847#bib.bib46)\]samples rollouts in groups, computes within\-group advantages from a verifiable reward, and updates the policy under a KL constraint\. For benchmarks without a verifiable reward \(10\-K, TempWiki\) we use a binary correctness signal from the ground\-truth label\.SDPO\[[19](https://arxiv.org/html/2607.07847#bib.bib19)\]is sequential DPO\. Our adaptation: at stagekk,θk−1\\theta\_\{k\-1\}generates both the chosen and rejected continuation pools, so the preference signal is implicitly relative to the model’s accumulated prior rather than to a fresh reference\. Both methods consumeCCas policy\-update steps\.

Architectural / Context Compression \(Cartridges, In\-place TTT\)These methods change the model’s update interface rather than the loss applied to its parameters\.Cartridges\[[10](https://arxiv.org/html/2607.07847#bib.bib10)\]freezes the backbone and learns a small per\-stage component, distilled from a teacher trace and stitched in at inference\. The backbone isθ0\\theta\_\{0\}for every stage; each stage’s update is structurally local\.In\-place TTT\[[11](https://arxiv.org/html/2607.07847#bib.bib11)\]performs an input\-conditioned weight update at inference using a self\-supervised auxiliary loss, reset between inputs\. We include In\-place TTT as a locality reference: its update is per\-input rather than per\-stage, which lets us isolate the locality axis from the accumulation axis in the experiment\.

## 4Experiments

For each task evaluation, we apply our general continual learning protocol and vary only how the world changes between optimization stages\. We work with the two axes of change:space, where the model moves across substantively different domains, andtime, where the model stays in one domain but the underlying data drifts across stages\. Within these axes we instantiate four settings:*independent domains*\(space\), where each stage is a different task;*selective fact change*\(time\), where some labels change between stages while others remain stable;*continuous temporal drift*\(time\), where the task is fixed but the data slides smoothly across stages; and and*agentic accumulation*\(time\), where the task family is fixed but the environment’s state drifts as a direct consequence of the agent’s own prior actions rather than an external clock or editor\. More details on task design choices can be found in Appendix[A\.2](https://arxiv.org/html/2607.07847#A1.SS2)\.

### 4\.1Domain Transfer

When a model is exposed to a new domain, it will need to acquire new skills or capabilities without catastrophically forgetting\. This is our evaluation setting and the classic test of continual learning and covers the axis of space\.

Setup\.Three unrelated supervised tasks are presented in a fixed order — ToolUse\[[50](https://arxiv.org/html/2607.07847#bib.bib50)\], FinQA\[[5](https://arxiv.org/html/2607.07847#bib.bib5)\], SciKE\-Bio\[[12](https://arxiv.org/html/2607.07847#bib.bib12)\]with each phase training a Qwen\-3\-8B on a 500\-example subsample of one task’s training set and validating against all three tasks at every step\. The subset of ToolUse and SciKE\-Bio are taken from\[[48](https://arxiv.org/html/2607.07847#bib.bib48)\]\. The task chain produces a forgetting matrix that answers "after seeing tasks1​…​i1\.\.\.i, how well does the model still do taskjj?" The three tasks deliberately share no surface\-level structure, so a method that nominally improves task acquisition can still fail by drifting the model’s output format or by overwriting earlier formats once it learns a later one\. For continual training, the best performing checkpoint on the current task is carried over to the next one\. Further details about the experimental setup including relevant hyperparameters are included in Appendix[A\.2\.1](https://arxiv.org/html/2607.07847#A1.SS2.SSS1)\.9

![Refer to caption](https://arxiv.org/html/2607.07847v1/x1.png)Figure 2:Per\-method accuracy across the three\-stage domain chain \(ToolUse→\\toFinQA→\\toSciKE\-Bio\) for Qwen3\-8B \(non\-thinking\)\. Each panel shows one method; the three colored curves are held\-out accuracy on each of the three eval sets across all training steps, with the corresponding panels marking stage boundaries\. Diagonal acquisition, the curve corresponding to the currently\-trained task, and the other two curves showing off\-diagonal forgetting, are visible side by side\.Prompt and harness optimization surfaces a speed\-vs\-stability trade\-off\.We observe that GEPA fits very quickly to some training stages such as FinQA, but has the most severe catastrophic forgetting when moving on to the next stage\. In Fig\.[2](https://arxiv.org/html/2607.07847#S4.F2)we observe that despite an almost 40% increase in performance during the FinQA training stage, after the training on biology accuracy drops almost entire back down the baseline accuracy before any training\. While ACE appears to be slight more stable than GEPA, it similarly improves and then immediately degrades at this same phase transition\.

Distillation\-based methods manage to accumulate knowledge\.We observe that SDFT and SDPO both finish the sequential training with FinQA and Bio above their zero\-shot baselines while keeping ToolUse near 55–60%; SDFT is the only method in our table whose three final\-stage scores are all at or above their respective zero\-shot levels\.

Context compression does not help learn new tasks\.Cartridges and In\-place TTT are less sensitive to than prompt methods to domain shift, but at the same time they do not show steady improvement across all tasks over training phases like SDFT\. Performance is largely flat or even simply degrades per task at each progressive training phase\. These methods never perform a full weight update, and thus appear to struggle to accumulate new capabilities under the domain shift task\.

### 4\.2Catastrophic Memorizing: Update Truly New Knowledge

In the previous experiment, we tested methods on their ability to stay aligned and accumulate knowledge across time\. Here, we ask the opposite: when presented with discrete fact changes, can a model rewrite the affected belief while leaving the rest of its knowledge untouched? Avoiding catastrophic forgetting is necessary, but so is knowing when and how to update\. The failure to do so is what we call ’catastrophic memorizing’\.

Data\.We construct the chain from four monthly Wikipedia mirrors covering the post\-pretraining\-cutoff window: 2025\-11\-20, 2025\-12\-01, 2026\-01\-01, and 2026\-02\-01\. For each adjacent pair of snapshots we extract a*drift set*of \(subject, relation, object\) triples whose object value changed between the snapshots, treating these as new world\-knowledge facts the model is being asked to acquire\. We also extract a*stable set*of \(subject, relation\) pairs whose object did not change across the entire window, and hold this set out across all phases as a forgetting probe: a method that improves drift accuracy by overwriting the model’s general world knowledge will pay for it on the stable probe\.

Setup\.To test for small volume adversarial updates evolving over time on Qwen\-3\-8B, Wikipedia is mirrored at four monthly snapshots, where every method updates from the same Wikipedia diff snapshot, in its method\-appropriate format\. Weight\-update and RL methods \(SFT, SDFT, SDPO, GRPO\) consume the 500 drift triples directly as Q/A pairs\. Cartridges and In\-place TTT consume the corresponding Wikipedia article bodies pulled from per\-snapshot – approximately 528 articles per slice\. Further details about the experimental setup including relevant hyperparameters are included in Appendix[A\.2\.2](https://arxiv.org/html/2607.07847#A1.SS2.SSS2)\. A filtered, continuously\-scored variant of this benchmark \(TempWiki\-Easy\), which isolates the update behavior from noisy diff categories and binary scoring, is analyzed in Appendix[A\.2\.3](https://arxiv.org/html/2607.07847#A1.SS2.SSS3)\.

![Refer to caption](https://arxiv.org/html/2607.07847v1/x2.png)Figure 3:F1 score across the three TempWiki slices for each method \(Qwen3\-8B, no thinking\)\. A method that genuinely learns the new facts should improve drift F1 while preserving stable F1; methods that conflate the two appear as diverging curves\. Cartridges and In\-place TTT preserve stable F1 most cleanly; SDFT and SDPO degrade stable F1 sharply; GEPA and ACE achieve the largest drift gains but pay heavily on stable\. GRPO is the only weight\-update method whose stable F1 trajectory does not move in the opposite direction of its drift trajectoryStability\-anchored methods fail to learn how to change\.Interestingly, SDFT and SDPO, which perform well on accumulating knowledge in noisy signal, struggle to learn changes in this setting\. SDFT’s stable\-fact F1 falls from≈\\approx0\.300\.30ats1s\_\{1\}to≈\\approx0\.150\.15ats3s\_\{3\}: by trying to interpolate the new slice against its own previous beliefs, the model ends up corrupting facts that were never supposed to move\. SDPO shows the same effect at smaller magnitude\. In contrast, GRPO shows strong performance in learning how to change while preserving performance averaging above baseline across the chain\+1\.6\+1\.6points\. GRPO witnesses the sharpest jump ats2s\_\{2\}of\+6\.2\+6\.2points which is the largest measured gain of any weight update method\.

Prompt evolution shows strong optimization ability on adapting to new knowledge at the cost of forgetting stable knowledge\.GEPA reaches the highest drift F1 in the comparison \(0\.18 ons1s\_\{1\}, 0\.24 ons3s\_\{3\}\) by writing the new fact directly into the playbook in a single optimization step\. The optimizer is rewarded on the validation set, which contains drift facts; it is not rewarded on stable facts, which it never sees\. So GEPA edits the playbook freely, and the same edits overwrite stable entries that depend on the same playbook tokens\. Stable F1 falls from 0\.32 ats1s\_\{1\}to 0\.26 ats3s\_\{3\}\. ACE shows the same profile although at a smaller magnitude\.

Context compression barely influences the base model\.For cartridges and In\-place TTT drift F1 stays within a few points of baseline across all three slices, with no separation in either direction\. Cartridges averages−0\.8\-0\.8on drift, with one\+3\.2\+3\.2inflection ons2s\_\{2\}and despite performing better than SDFT and SDPO, but shows no improvement compared to baseline, even harming the performance, belonging to the same category at times\.

### 4\.3Noisy Temporal Drift

From the previous experiments, we observe over longer time horizons, performance steadily degrades\. In our following temporal drift experiments, we dig deeper into the problem by setting up a task where the model can only improve if it manages to accumulate weak and noisy signals through time\.

Setup\.We construct a challenging temporal data task by taking a realistic case study in finance: predicting whether a stock moves up or down in the 30 days after its 10\-K filing\. This setting forces a model to operate inside a two\-sided time drift\. Some patterns in how firms describe risk, growth, and capital plans should be retained, while other signals are regime\-specific \(e\.g\. before and after Covid\) and stop working as markets evolve\.Doing well now does not guarantee doing well in the future\.A model that loses past knowledge cannot tell whether a new filing is unusual relative to history, and a model that holds too tightly to past years may overfit on spurious or transient regimes\.

We use full\-length SEC EDGAR filings \(typically 10–13K tokens\), with each filing labeledupordownbased on its 30\-day forward return\[[34](https://arxiv.org/html/2607.07847#bib.bib34),[37](https://arxiv.org/html/2607.07847#bib.bib37)\]\. Additional details on data can be found in Appendix[A\.2\.4](https://arxiv.org/html/2607.07847#A1.SS2.SSS4)\. We train sequentially on filing years 2015 through 2020 in chronological order\. After each year, we evaluate on every year in the range to measure both backward transfer \(did the model actually learn during training\) and forward transfer \(did it learn something general it can apply to future years\)\. The signal in any individual filing is known to be weak, but in aggregate a small, tradeable edge can be accumulated\[[37](https://arxiv.org/html/2607.07847#bib.bib37)\]\(see also Figure[8](https://arxiv.org/html/2607.07847#A1.F8)\)\. At baseline before training, Qwen3\-8B sits around 50% and accuracy is noisy across years\. Further details about the experimental setup including relevant hyperparameters are included in Appendix[A\.2\.4](https://arxiv.org/html/2607.07847#A1.SS2.SSS4)

![Refer to caption](https://arxiv.org/html/2607.07847v1/x3.png)Figure 4:Backward and forward accuracy on the 10\-K sentiment task, evaluated after sequential training on each year from 2015 to 2020 \(Qwen3\-8B, no thinking\)\. Blue: average accuracy on past years \(≤\\leqcurrent\)\. Red: average accuracy on future years \(\>\>current\)\. Stars: zero\-shot baseline per year\. Distillation\-based methods \(SDFT, SDPO, Cartridges\) sustain forward accuracy, while prompt\-based methods \(GEPA, ACE\) and GRPO degrade on future years\. SFT stays near chance throughout\.RL struggles when the reward signal is weak and noisy\.GRPO shows a significant accuracy drop both on forward\- and backward\-looking sentiment; its success depends heavily on the reward ranking inside each group\. If the reward is noisy, the algorithm may reinforce the wrong answer\.

Stability\-anchored methods show the strongest forward transfer with minimal past degradation\.SDFT’s past\-year accuracy stays flat around0\.500\.50while its future\-year accuracy climbs to≈\\approx0\.620\.62by 2020: distilling against its own previous state avoids both drift and over\-fitting to the current year\. Cartridges is the most stable method we tested, with past and future hovering around0\.550\.55–0\.580\.58throughout the chain\. In\-place TTT is the locality reference: its accuracy stays in a tight 34\.5–51\.5% band across the chain \(mean 46\.9%\) and never separates from baseline\. The reset\-per\-input update geometry is incompatible with absorbing a slow\-moving trend, but it pays no price for trying – the method neither acquires nor degrades\.

Prompting methods fail to extract useful knowledge for forward transfer\.On this task, similar to RL, GEPA also struggles with learning meaningful information from noisy signals\. ACE fits better on past\-year data, but also completely fails to predict future years’ trend, which suggests that it does not accumulate meaningful knowledge\.

Qualitatively, GEPA discovers*generic*financial\-analysis heuristics \(Figure[5](https://arxiv.org/html/2607.07847#S4.F5)\)\. These edits fail to capture characteristics of an ideal adaptive analyst e\.g\. learn to discard irrelevant sections, normalize sentiment relative to sector or company baselines, or distinguish between newly informative elements and those that are already priced in by the market\. While it is possible to construct a strategy with a positive Profit and Loss \(PnL\) with a rule as simple as focusing on relevant sections in the 10\-K \(see Figure[8](https://arxiv.org/html/2607.07847#A1.F8)\), consistent with the observed drop in forward accuracy after optimization, the optimized prompt reduces even this forward PnL\. We conclude that in this noisy, time\-changing regime, GEPA overfits to transient textual heuristics while failing to preserve exposure to economically meaningful signals that remain predictive out\-of\-sample\.

![Refer to caption](https://arxiv.org/html/2607.07847v1/gepa_prompt_diff.png)Figure 5:Prompt evolution on the 10\-K sentiment task: \(L\) how an idealised analyst may evolve its instructions over time, both noting persistent trends and updating outdated information; \(R\) the evolution of GEPA shows the addition of generic, static\-in\-time financial heuristics\.
### 4\.4Agentic Task

The previous two experiments test the time axis under drift that comes from outside the model\. Here we test a third sub\-regime of time introduced in §[3](https://arxiv.org/html/2607.07847#S3): the task family stays fixed \(actions inside a web app\), but the environment’s state drifts only because the agent itself acted on it, so state left behind by one step becomes part of the task the model faces at the next\.

Data\.We build the agentic suite on top of WebArena\-Infinity\[[61](https://arxiv.org/html/2607.07847#bib.bib61)\], seeding chain generation from each app’s description \(which enumerates the available features\) and its underlying state files \(which fix the entities those features can act on\)\. From these we generate sequential chains in which stepi\+1i\{\+\}1depends on a concrete piece of state that stepiileaves behind in the environment: in Gmail, create a label, rename it, then apply the renamed label to a specific email\. Each step is paired with a programmatic verifier that reads the app’s state directly, so chain success is checked against the environment rather than the agent’s narration\. Because the sequence arises from the agent’s own actions rather than an ordering we impose, we treat it as the most realistic instantiation of continual learning over time\.

Setup\.Beyond zero\-shot evaluation, we ask whether experience*compounds*across a chain: whether learning from earlier trajectories improves later chain success\. We add learning two ways on top of abrowser\-useagent driving a headless Chromium against the live app\. First, an ACE playbook\[[59](https://arxiv.org/html/2607.07847#bib.bib59)\]on Qwen\-32B, which accumulates reusable procedural guidance distilled from prior rollouts\. Second, supervised fine\-tuning of Qwen\-8B on successful traces from a stronger teacher agent, distilling the larger agent’s behavior into the smaller model \(per\-level teachers are given in Appendix[A\.2\.5](https://arxiv.org/html/2607.07847#A1.SS2.SSS5)\)\. Within a chain we keep a single agent instance alive across all steps: the browser session is not reset, and each follow\-up instruction is appended to the same conversation rather than starting a fresh context, so the model sees its own prior trajectory when acting on stepi\+1i\{\+\}1\. Each step is given a fixed action budget and is scored independently by its verifier\. Further details about the experimental setup including relevant hyperparameters are included in Appendix[A\.2\.5](https://arxiv.org/html/2607.07847#A1.SS2.SSS5)\.

Results\.Both update mechanisms raise end\-to\-end chain success over their no\-learning baseline at every horizon length \(Figure[6](https://arxiv.org/html/2607.07847#S4.F6)\)\. The ACE playbook lifts Qwen\-32B over its zero\-shot baseline at every length, most on the three\-step \(L3\) chains \(32\.4%32\.4\\%to46\.5%46\.5\\%\)\. Without training, Qwen\-8B collapses to zero past L1; supervised fine\-tuning on the teacher’s traces recovers substantial success \(40\.0%40\.0\\%at L3,20\.0%20\.0\\%at L5, and6\.7%6\.7\\%even at L10\), and overtakes both Qwen\-32B lines at L1 \(60\.0%60\.0\\%\)\. Absolute success still decays with chain length for every configuration, as accumulated trajectory state grows and later steps must be grounded in increasingly distant environmental cues, converging near the floor by L10\. This is the closest setting to deployed use, since the chain comes from the agent’s own actions rather than an ordering we picked, and future work could go deeper on this realistic agentic setting\.

![Refer to caption](https://arxiv.org/html/2607.07847v1/x4.png)Figure 6:End\-to\-end chain success rate across chain lengths L1–L10 on the Gmail agentic suite\. Color marks the model \(blue Qwen\-32B, red Qwen\-8B\); solid lines are the update mechanisms \(an ACE playbook on Qwen\-32B, and SFT on Qwen\-8B distilled from teacher traces\) and dashed lines are the corresponding zero\-shot baselines\. Both mechanisms improve over their baseline at every length, most on the three\-step \(L3\) chains\.

## 5Discussion

### 5\.1Conclusion

We introduced a unified framework for evaluating continual learning in large language models, defining it as the problem of increasing competence as the world changes along two axes: space \(domain shift\) and time \(temporal drift\)\. By recasting widely used LLM benchmarks as sequential problems and evaluating eight methods under a common mechanism\-agnostic protocol, we surface a consistent set of tradeoffs that span across method families\. No single method handles all task regimes well: prompt\-based methods fit quickly but overwrite prior capabilities; distillation\-based methods accumulate knowledge stably but resist new updates; context compression improves efficiency without improving task acquisition; and online reinforcement learning adapts most effectively to factual change but degrades under noisy reward signals\. Overall, our results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors\. We hope that understanding where each method succeeds and fails will help guide the design of more capable continual learning systems\.

### 5\.2Limitations and Challenges

To draw unified conclusions, we use a single model across all evaluations, Qwen3\-8B; the relative behavior of the tested methods may change for larger models or models in reasoning mode\. In addition, our benchmark suite captures only a subset of realistic environmental change: domain shifts, agentic sequences, noisy temporal drift, and discrete factual updates\. While this is a step beyond fixed benchmarks and already reveals that different regimes require different update behaviors, deployed LLMs will face many more open\-ended, changing, and uncertain environments and diverse use cases\. The results should therefore be read as evidence about which update behaviors are required in different regimes, not as a definitive ranking of continual\-learning algorithms\.

## Acknowledgement

We thank Matei Zaharia and Alyosha Efros for helpful discussions and feedback\. This work is partially supported by the Laude Institute, Modal, and the ONR MURI N00014\-21\-1\-2801\. AH was supported by the National Defense Science and Engineering Graduate \(NDSEG\) Fellowship Program\. This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550\-25\-C\-B010 in the amount of $31,607\.75\.

## References

- Agarwal et al\. \[2025\]Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al\.gpt\-oss\-120b & gpt\-oss\-20b model card\.*arXiv preprint arXiv:2508\.10925*, 2025\.
- Agrawal et al\. \[2025\]Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl\-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, et al\.Gepa: Reflective prompt evolution can outperform reinforcement learning\.*arXiv preprint arXiv:2507\.19457*, 2025\.
- Brown et al\. \[2020\]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al\.Language models are few\-shot learners\.*Advances in neural information processing systems*, 33:1877–1901, 2020\.
- Cemri et al\. \[2026\]Mert Cemri, Shubham Agrawal, Akshat Gupta, Shu Liu, Audrey Cheng, Qiuyang Mang, Ashwin Naren, Lutfi Eren Erdogan, Koushik Sen, Matei Zaharia, et al\.Adaevolve: Adaptive llm driven zeroth\-order optimization\.*arXiv preprint arXiv:2602\.20133*, 2026\.
- Chen et al\. \[2021\]Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting\-Hao Huang, Bryan R Routledge, et al\.Finqa: A dataset of numerical reasoning over financial data\.In*Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 3697–3711, 2021\.
- Cheung et al\. \[2019\]Brian Cheung, Alexander Terekhov, Yubei Chen, Pulkit Agrawal, and Bruno Olshausen\.Superposition of many models into one\.*Advances in neural information processing systems*, 32, 2019\.
- Comanici et al\. \[2025\]Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al\.Gemini 2\.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.*arXiv preprint arXiv:2507\.06261*, 2025\.
- Dohare et al\. \[2024\]Shibhansh Dohare, J Fernando Hernandez\-Garcia, Qingfeng Lan, Parash Rahman, A Rupam Mahmood, and Richard S Sutton\.Loss of plasticity in deep continual learning\.*Nature*, 632\(8026\):768–774, 2024\.
- Ebrahimi et al\. \[2020\]Sayna Ebrahimi, Franziska Meier, Roberto Calandra, Trevor Darrell, and Marcus Rohrbach\.Adversarial continual learning\.In*European conference on computer vision*, pages 386–402\. Springer, 2020\.
- Eyuboglu et al\. \[2025\]Sabri Eyuboglu, Ryan Ehrlich, Simran Arora, Neel Guha, Dylan Zinsley, Emily Liu, Will Tennien, Atri Rudra, James Zou, Azalia Mirhoseini, et al\.Cartridges: Lightweight and general\-purpose long context representations via self\-study\.*arXiv preprint arXiv:2506\.06266*, 2025\.
- Feng et al\. \[2026\]Guhao Feng, Shengjie Luo, Kai Hua, Ge Zhang, Di He, Wenhao Huang, and Tianle Cai\.In\-place test\-time training\.*arXiv preprint arXiv:2604\.06169*, 2026\.
- Feng et al\. \[2024\]Kehua Feng, Xinyi Shen, Weijie Wang, Xiang Zhuang, Yuqi Tang, Qiang Zhang, and Keyan Ding\.Sciknoweval: Evaluating multi\-level scientific knowledge of large language models\.*arXiv preprint arXiv:2406\.09098*, 2024\.
- French \[1999\]Robert M French\.Catastrophic forgetting in connectionist networks\.*Trends in cognitive sciences*, 3\(4\):128–135, 1999\.
- Glazer et al\. \[2024\]Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean\-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, et al\.Frontiermath: A benchmark for evaluating advanced mathematical reasoning in ai\.*arXiv preprint arXiv:2411\.04872*, 2024\.
- Golkar et al\. \[2019\]Siavash Golkar, Michael Kagan, and Kyunghyun Cho\.Continual learning via neural pruning\.*arXiv preprint arXiv:1903\.04476*, 2019\.
- Grady et al\. \[2026\]Thomas Grady, Kip Parker, Iliyan Zarov, Henry Course, Chengxi Taylor, and Ross Taylor\.Kellybench: A benchmark for long\-horizon sequential decision making\.*arXiv preprint arXiv:2604\.27865*, 2026\.
- Hendrycks et al\. \[2021\]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\.Measuring mathematical problem solving with the math dataset\.*arXiv preprint arXiv:2103\.03874*, 2021\.
- Hu et al\. \[2022\]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al\.Lora: Low\-rank adaptation of large language models\.*Iclr*, 1\(2\):3, 2022\.
- Hübotter et al\. \[2026\]Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, et al\.Reinforcement learning via self\-distillation\.*arXiv preprint arXiv:2601\.20802*, 2026\.
- Ilharco et al\. \[2022\]Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi\.Editing models with task arithmetic\.*arXiv preprint arXiv:2212\.04089*, 2022\.
- Jang et al\. \[2022\]Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, and Minjoon Seo\.Temporalwiki: A lifelong benchmark for training and evaluating ever\-evolving language models\.In*Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, pages 6237–6250, 2022\.
- Javed and White \[2019\]Khurram Javed and Martha White\.Meta\-learning representations for continual learning\.*Advances in neural information processing systems*, 32, 2019\.
- Jimenez et al\. \[2023\]Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan\.Swe\-bench: Can language models resolve real\-world github issues?In*The twelfth international conference on learning representations*, 2023\.
- \[24\]Aman Khan\.Financial reports sec\.URL[https://huggingface\.co/datasets/JanosAudran/financial\-reports\-sec](https://huggingface.co/datasets/JanosAudran/financial-reports-sec)\.
- Khattab et al\. \[2023\]Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Saiful Haq, Ashutosh Sharma, Thomas T Joshi, Hanna Moazam, Heather Miller, et al\.Dspy: compiling declarative language model calls into state\-of\-the\-art pipelines\.In*The Twelfth International Conference on Learning Representations*, 2023\.
- Lee et al\. \[2026\]Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn\.Meta\-harness: End\-to\-end optimization of model harnesses\.*arXiv preprint arXiv:2603\.28052*, 2026\.
- Lewis et al\. \[2020\]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.*Advances in neural information processing systems*, 33:9459–9474, 2020\.
- Li et al\. \[2026\]Hanchen Li, Runyuan He, Qizheng Zhang, Changxiu Ji, Qiuyang Mang, Xiaokun Chen, Lakshya A Agrawal, Wei\-Liang Liao, Eric Yang, Alvin Cheung, et al\.Combee: Scaling prompt learning for self\-improving language model agents\.*arXiv preprint arXiv:2604\.04247*, 2026\.
- Li et al\. \[2023\]Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin\.Compressing context to enhance inference efficiency of large language models\.In*Proceedings of the 2023 conference on empirical methods in natural language processing*, pages 6342–6353, 2023\.
- Lin et al\. \[2025\]Jessy Lin, Luke Zettlemoyer, Gargi Ghosh, Wen\-Tau Yih, Aram Markosyan, Vincent\-Pierre Berges, and Barlas Oğuz\.Continual learning via sparse memory finetuning\.*arXiv preprint arXiv:2510\.15103*, 2025\.
- Liu et al\. \[2025a\]Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al\.Deepseek\-v3\. 2: Pushing the frontier of open large language models\.*arXiv preprint arXiv:2512\.02556*, 2025a\.
- Liu et al\. \[2025b\]Ruiqi Liu, Boyu Diao, Libo Huang, Zijia An, Hangda Liu, Zhulin An, and Yongjun Xu\.Low\-redundancy distillation for continual learning\.*Pattern Recognition*, 167:111712, 2025b\.
- Lopez\-Paz and Ranzato \[2017\]David Lopez\-Paz and Marc’Aurelio Ranzato\.Gradient episodic memory for continual learning\.*Advances in neural information processing systems*, 30, 2017\.
- Loukas et al\. \[2021\]Lefteris Loukas, Manos Fergadiotis, Ion Androutsopoulos, and Prodromos Malakasiotis\.Edgar\-corpus, September 2021\.URL[https://doi\.org/10\.5281/zenodo\.5528490](https://doi.org/10.5281/zenodo.5528490)\.
- Lu and Lab \[2025\]Kevin Lu and Thinking Machines Lab\.On\-policy distillation\.*Thinking Machines Lab: Connectionism*, 2025\.doi:10\.64434/tml\.20251026\.https://thinkingmachines\.ai/blog/on\-policy\-distillation\.
- Lyle et al\. \[2022\]Clare Lyle, Mark Rowland, and Will Dabney\.Understanding and preventing capacity loss in reinforcement learning\.*arXiv preprint arXiv:2204\.09560*, 2022\.
- Magner et al\. \[2025\]Nicolás Magner, Pablo A Henríquez, and Aliro Sanhueza\.Decoding risk sentiment in 10\-k filings: Predictability for us stock indices\.*Finance Research Letters*, 81:107472, 2025\.
- Mang et al\. \[2025\]Qiuyang Mang, Wenhao Chai, Zhifei Li, Huanzhi Mao, Shang Zhou, Alexander Du, Hanchen Li, Shu Liu, Edwin Chen, Yichuan Wang, et al\.Frontiercs: Evolving challenges for evolving intelligence\.*arXiv preprint arXiv:2512\.15699*, 2025\.
- McCarthy and Hayes \[1981\]John McCarthy and Patrick J Hayes\.Some philosophical problems from the standpoint of artificial intelligence\.In*Readings in artificial intelligence*, pages 431–450\. Elsevier, 1981\.
- McCloskey and Cohen \[1989\]Michael McCloskey and Neal J Cohen\.Catastrophic interference in connectionist networks: The sequential learning problem\.In*Psychology of learning and motivation*, volume 24, pages 109–165\. Elsevier, 1989\.
- Motwani et al\. \[2025\]Sumeet Ramesh Motwani, Alesia Ivanova, Ziyang Cai, Philip Torr, Riashat Islam, Shital Shah, Christian Schroeder de Witt, and Charles London\.h1: Bootstrapping llms to reason over longer horizons via reinforcement learning\.*arXiv preprint arXiv:2510\.07312*, 2025\.
- Nikishin et al\. \[2022\]Evgenii Nikishin, Max Schwarzer, Pierluca D’Oro, Pierre\-Luc Bacon, and Aaron Courville\.The primacy bias in deep reinforcement learning\.In*International conference on machine learning*, pages 16828–16847\. PMLR, 2022\.
- Novikov et al\. \[2025\]Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po\-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al\.Alphaevolve: A coding agent for scientific and algorithmic discovery\.*arXiv preprint arXiv:2506\.13131*, 2025\.
- Opsahl\-Ong et al\. \[2024\]Krista Opsahl\-Ong, Michael J Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab\.Optimizing instructions and demonstrations for multi\-stage language model programs\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 9340–9366, 2024\.
- Ouyang et al\. \[2025\]Anne Ouyang, Simon Guo, Simran Arora, Alex L Zhang, William Hu, Christopher Ré, and Azalia Mirhoseini\.Kernelbench: Can llms write efficient gpu kernels?*arXiv preprint arXiv:2502\.10517*, 2025\.
- Shao et al\. \[2024\]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- Sharma \[2025\]Asankhaya Sharma\.Openevolve: an open\-source evolutionary coding agent, 2025\.URL[https://github\.com/algorithmicsuperintelligence/openevolve](https://github.com/algorithmicsuperintelligence/openevolve)\.
- Shenfeld et al\. \[2026\]Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal\.Self\-distillation enables continual learning\.*arXiv preprint arXiv:2601\.19897*, 2026\.
- Tandon et al\. \[2025\]Arnuv Tandon, Karan Dalal, Xinhao Li, Daniel Koceja, Marcel Rød, Sam Buchanan, Xiaolong Wang, Jure Leskovec, Sanmi Koyejo, Tatsunori Hashimoto, et al\.End\-to\-end test\-time training for long context\.*arXiv preprint arXiv:2512\.23675*, 2025\.
- Tang et al\. \[2023\]Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun\.Toolalpaca: Generalized tool learning for language models with 3000 simulated cases\.*arXiv preprint arXiv:2306\.05301*, 2023\.
- Wang et al\. \[2024\]Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al\.Mmlu\-pro: A more robust and challenging multi\-task language understanding benchmark\.*Advances in Neural Information Processing Systems*, 37:95266–95290, 2024\.
- Wu et al\. \[2025\]Shirley Wu, Parth Sarthi, Shiyu Zhao, Aaron Lee, Herumb Shandilya, Adrian Mladenic Grobelnik, Nurendra Choudhary, Eddie Huang, Karthik Subbian, Linjun Zhang, et al\.Optimas: Optimizing compound ai systems with globally aligned local rewards\.*arXiv preprint arXiv:2507\.03041*, 2025\.
- Yang et al\. \[2025\]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*, 2025\.
- Ye et al\. \[2026\]Haoran Ye, Xuning He, Vincent Arak, Haonan Dong, and Guojie Song\.Meta context engineering via agentic skill evolution\.*arXiv preprint arXiv:2601\.21557*, 2026\.
- Yuksekgonul et al\. \[2024\]Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou\.Textgrad: Automatic" differentiation" via text\.*arXiv preprint arXiv:2406\.07496*, 2024\.
- Zhang et al\. \[2025a\]Alex L Zhang, Tim Kraska, and Omar Khattab\.Recursive language models\.*arXiv preprint arXiv:2512\.24601*, 2025a\.
- Zhang et al\. \[2020\]Jeffrey O Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, and Jitendra Malik\.Side\-tuning: a baseline for network adaptation via additive side networks\.In*European conference on computer vision*, pages 698–714\. Springer, 2020\.
- Zhang et al\. \[2025b\]Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune\.Darwin godel machine: Open\-ended evolution of self\-improving agents\.*arXiv preprint arXiv:2505\.22954*, 2025b\.
- Zhang et al\. \[2025c\]Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, et al\.Agentic context engineering: Evolving contexts for self\-improving language models\.*arXiv preprint arXiv:2510\.04618*, 2025c\.
- Zhao et al\. \[2026\]Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover\.Self\-distilled reasoner: On\-policy self\-distillation for large language models\.*arXiv preprint arXiv:2601\.18734*, 2026\.
- Zhou \[2026\]Shuyan Zhou\.Webarena\-infinity: Generating browser environments with verifiable tasks at scale\.*shuyanzhou\.com*, March 2026\.URL[https://webarena\.dev/webarena\-infinity/](https://webarena.dev/webarena-infinity/)\.

## Appendix AAdditional Experimental Details

### A\.1Compute and Hyperparameters

Within each benchmark, every weight\-update method \(SFT, SDFT, SDPO, GRPO\) uses the same global batch \(32\), one epoch over 500 examples per phase \(≈\\approx16 optimizer steps\), and full\-weight updates\. The outer learning rate is set per\-method\-per\-benchmark to keep cumulative parameter displacement under the empirical collapse threshold for that label regime\[[8](https://arxiv.org/html/2607.07847#bib.bib8)\]\. Each training phase took no longer than 1 hour on a node of 8 H100s\.

Table 2:Outer learning rates for verl\-trained methods across benchmarks\. SFT trains on bare\-answer gold \(∼\\sim5–30 tokens\), so per\-token gradient signal is roughly10×10\\timesthe SDFT/SDPO signal, which trains on∼\\sim200\+ token rollout golds; lr=1×10−51\\\!\\times\\\!10^\{\-5\}over\-displaces the SFT chain in roughly one phase and the model collapses\. The single\-token\-gold regimes \(TempWiki everywhere; finance for GRPO specifically\) collapse to majority\-class at1×10−51\\\!\\times\\\!10^\{\-5\}, so we drop those to3×10−63\\\!\\times\\\!10^\{\-6\}\.Cartridges and In\-place TTT use their own optimization recipes; hyperparameters are reported per benchmark in the sections below\.

For prompting\-based methods, we used cloud API models via OpenRouter\. The total cost of all experiments \(including development and hyperparamter tuning\) was $1000 USD\.

### A\.2Task Settings

#### A\.2\.1Domain shift

The learner specialises, in sequence, across three substantively different domains: tool use \(ToolUse\[[50](https://arxiv.org/html/2607.07847#bib.bib50)\]\), finance \(FinQA\[[5](https://arxiv.org/html/2607.07847#bib.bib5)\]\), and biology \(SciKE\-Bio\[[12](https://arxiv.org/html/2607.07847#bib.bib12)\]\)\. Each stage provides 500 training instances and a 50\-row held\-out evaluation set per task; stages are presented in the canonical order\[ToolUse, FinQA, Bio\]so that per\-method results are directly comparable, with permutation\-averaged variants reported in the appendix to control for order effects\.

Tasks and data\.Each phase trains on a 500\-example subsample of one task’s training set and validates against all three tasks at every step\. The FinQA test set is unusually large \(1,125 prompts\); we deterministically subsample it to 128 prompts using the SDPO baseline’s seed \(np\.random\.RandomState\(1042\)\.permutation\(1125\)\[:128\], sorted\), so every method evaluates on identical FinQA prompts\.

Table 3:Domain shift benchmark per\-phase data\. The three tasks deliberately share no surface\-level structure, so a method that nominally improves task acquisition can still fail by drifting the model’s output format or by overwriting earlier formats once it learns a later one\.Per\-method hyperparameters\.SFT, SDFT, SDPO, and GRPO share the optimization budget in Table[2](https://arxiv.org/html/2607.07847#A1.T2)\. SDFT, SDPO, and GRPO additionally use 8 on\-policy rollouts per training prompt \(temperature 0\.6, top\-pp0\.95\)\. SDPO optimizes a DPO\-style pairwise objective over scored rollouts \(chosen = high\-score, rejected = low\-score\) against a frozen reference model\. GRPO uses PPO\-style clipped objective with per\-promptzz\-scored advantages over the 8\-rollout group\.

Cartridges configuration\.Each phase synthesizes a self\-study corpus from the cumulative train pool of phases 1\.\.ii: Qwen3\-8B \(running locally as a synthesis server\) is prompted with the four standard seed prompts \(use\_case,question,summarization,structuring\) to emit 8,192 conversation\-style Q/A pairs grounded in the corpus\. Those 8,192 conversations are then context\-distilled into a fresh trainable KV cache\. Base weights are frozen\. At evaluation the cartridge KV is loaded as a fixed prefix and the validation prompt is generated against it\.

Table 4:Cartridges configuration on the domain shift benchmark\.##### In\-place TTT configuration\.

Continual pretraining via VeOmni plaintext\-iterable JSONL with one record per training prompt \(cumulative pool of phases 1\.\.iiin order B\)\. Fast\-weight updates are activated on a sparse layer set \(paper\-recommended for continual training\)\. The chain hands each phase’s HF\-format checkpoint forward as the next phase’smodel\_path\.

Table 5:In\-place TTT configuration on the domain shift benchmark\. Thekltarget is for from\-scratch context lengthening and is not used here\.Scoring\.Each task has its own scorer\. A prediction in the wrong format scores zero — the benchmark measures task knowledge*and*format\-fidelity together\.

- •Tool\-use:regex\-extract everyAction: <name\>andAction Input: \{\.\.\.\}line; parse eachAction\_Inputbody as JSON; accept iff the multiset of predicted action names equals the gold and the mergedAction\_Inputdict equals the gold dict key\-by\-key\.
- •FinQA:extract the contents of the last\\boxed\{\.\.\.\}; accept iff\|pred−gold\|≤max⁡\(0\.005⋅\|gold\|,0\.01\)\|\\text\{pred\}\-\\text\{gold\}\|\\leq\\max\(0\.005\\cdot\|\\text\{gold\}\|,0\.01\)\(FinQA paper convention; exact\-match would penalize harmless rounding\)\.
- •SciKE\-Bio:look for a strict<answer\>X</answer\>tag; accept iff the single\-letterXequals the gold letter\.

#### A\.2\.2TempWiki Benchmark Details

Task and data\.Wikipedia is mirrored at four monthly snapshots — 2025\-11\-20, 2025\-12\-01, 2026\-01\-01, 2026\-02\-01 — and for each adjacent pair we extract the drift set of \(subject, relation, object\) triples whose object value changed across snapshots\. We also extract a stable set of \(subject, relation\) pairs whose answer did not change across the chain, and we hold this set out across all phases as a forgetting probe\.

Using these snapshots, we construct a chain with three phases corresponding to three diff\-pair slices, with 500 training drift triples, 50 validation drift triples, and 50 stable validation triples\. Existing temporal\-knowledge benchmarks such as the original TemporalWiki\[[21](https://arxiv.org/html/2607.07847#bib.bib21)\]stop at facts that have changed and do not audit unchanged knowledge, which makes the catastrophic\-memorization failure mode invisible\. Pairing the drift set with a held\-out stable probe is what lets the matrix expose the trade\-off methods make between acquiring the new fact and preserving the surrounding knowledge\.

Table 6:TempWiki per\-phase data\. The validation prompt is intentionally terse: a system message instructing the model to emit only the object string, a user message of the form<subject\> <relation\>, and a gold object string\. Two illustrative examples: \(drift\) user “São Paulo twinned administrative body”→\\togold “Buenos Aires”; \(stable\) user “Jibra’il Dallal place of death”→\\togold “Aleppo”\.Per\-method hyperparameters\.SFT/SDFT/SDPO/GRPO use the recipe from the 500 drift triples Q/A\-formatted directly; stable triples are held out and never appear in training\. Cartridges uses the same configuration as on the domain benchmark, with two adjustments: the source corpus is the cumulative Wikipedia article bodies rather than the cumulative training pool, and the slice\-1 article pool deliberately includes the stable\-val subjects so the synthesizer has reason to write Q/A about them; only drift articles are added on later slices, which keeps the stable probe legitimately held out from the gradient\-equivalent updates\. In\-place TTT uses the same fast\-weight recipe as on the 10\-K benchmark; per\-article records are formatted as=== Wikipedia article title=<sitelink\> as\_of=<snapshot\> ===\\n<body\>\\nfollowed by an optionalKnown facts:block holding the canonicalsubject relation \-\> goldlines\.

Table 7:Cartridges configuration on the TempWiki benchmark\.Table 8:In\-place TTT configuration on the TempWiki benchmark\.Scoring \(word\-level F1, paraphrase\-permissive\)\.The scorer computes word\-level F1\. The prediction is stripped of<think\>traces and chat suffixes; the model’s answer is taken to be the contents of an explicitAnswer: \.\.\.line if present, otherwise the first non\-empty line\. Normalization \(applied identically to prediction and gold\): lower\-case, strip the articlesa,an,the, strip all ASCII punctuation, collapse whitespace, tokenize by whitespace into a token bag\.

We score predictions with a token\-level F1 against the gold answer\. Both strings are normalized \(lowercased, articles and punctuation stripped, whitespace collapsed\) and split into tokens\. LetSSbe the number of tokens shared between the prediction and the gold \(counted with repetition\)\. Then

P=SNpred,R=SNgold,F1=2​P​RP\+R\.P=\\frac\{S\}\{N\_\{\\text\{pred\}\}\},\\qquad R=\\frac\{S\}\{N\_\{\\text\{gold\}\}\},\\qquad F\_\{1\}=\\frac\{2PR\}\{P\+R\}\.A prediction counts as a hit ifF1≥0\.5F\_\{1\}\\geq 0\.5, and zero otherwise\. We reportscore\_mean@8: the average hit rate across 8 rollouts per prompt and across all prompts in a slice\.

#### A\.2\.3TempWiki\-Easy: a filtered, continuously\-scored variant

The TempWiki drift set in the main text \(Fig\.[3](https://arxiv.org/html/2607.07847#S4.F3)\) pools*every*relation type whose object value changed between two snapshots\. Two properties of that construction make the catastrophic\-memorizing signal noisier than it should be\. First, many Wikipedia diffs are not genuine world\-knowledge updates: reformatting, disambiguation edits, list re\-orderings, and annotation\-style relations change the object string without there being a well\-posed “new fact” to learn\. Second, the object\-F1 scorer used there is effectively binary at the string level, so a near\-correct object \(e\.g\. a partial or alternately formatted name\) is scored identically to a wholly wrong one\. Both effects inflate the apparent divergence between drift and stable curves\. TempWiki\-Easy is a cleaned re\-cut of the same benchmark designed to isolate the update behavior from these artifacts\.

Construction\.We keep the four monthly snapshots and the identical three\-slice chain as TempWiki \(Nov→\\toDec 2025, Dec 2025→\\toJan 2026, Jan→\\toFeb 2026\), and apply three changes to the diff extraction:

- •Category allow\-list\.Only \(subject, relation\) pairs drawn from an allow\-list of relation types for which “change” is well\-defined \(single\-valued, factual, verifiable object\) are eligible\. Relations prone to formatting or annotation churn are excluded outright, which removes the pathological categories responsible for spurious drift/stable divergence in the original cut\.
- •Aggressive judged filtering\.Each surviving candidate diff is passed through an LLM judge that keeps the triple only if the edit reflects a semantically meaningful factual update between the two snapshots, discarding cosmetic or ambiguous edits\.
- •QA regeneration \+ continuous scoring\.We regenerate question/answer pairs over the filtered facts and, rather than thresholding the word\-level F1 into a hit atF1≥0\.5F\_\{1\}\\\!\\geq\\\!0\.5\(thescore\_mean@8hit rate used in Appendix[A\.2\.2](https://arxiv.org/html/2607.07847#A1.SS2.SSS2)\), report the continuous word\-levelF1F\_\{1\}directly\. Partially correct objects receive partial credit, so small real movements in knowledge are visible rather than being quantized to0/10/1\.

![Refer to caption](https://arxiv.org/html/2607.07847v1/x5.png)Figure 7:TempWiki\-Easy: object F1 across the three drift slices \(Qwen3\-8B, no thinking, seed 42\)\.The most visible difference from TempWiki Fig\.[3](https://arxiv.org/html/2607.07847#S4.F3)is that the stable\-fact trajectory \(purple dashed\) no longer trades off against drift\. In the original cut, SDFT’s stable F1 fell from to and SDPO’s collapsed to by the final slice; in TempWiki\-Easy both remain flat and essentially coincident with the zero\-shot stable baseline across all three slices\. With the confound removed, most methods sit on top of the zero\-shot drift baseline: SFT and SDFT neither learn the changed facts nor harm the stable ones\. The two exceptions are the reinforcement/distillation updaters\. SDPO climbs to 0\.22 on the final slice, above its 0\.17 zero\-shot baseline, while its stable F1 drifts only mildly GRPO shows the largest net effect: after the same mid\-chain dip it reaches∼\\sim0\.22 drift on Jan→\\toFeb while holding stable F1 flat\.

#### A\.2\.4Temporal Drift

10\-K Sentiment Benchmark Details

The core task is fixed \(predict the directional sentiment of a 10\-K filing\)\[[34](https://arxiv.org/html/2607.07847#bib.bib34),[37](https://arxiv.org/html/2607.07847#bib.bib37)\]but slides across calendar time: each stage corresponds to one fiscal year of 10\-K filings, presented in chronological order from 2015 to 2020\. After training on yearyywe evaluate on every other year, splitting the score into*past\-year*accuracy \(≤y\\leq y\) and*future\-year*accuracy \(\>y\>y\)\. The target is intentionally weak\-signal: real financial text is informative only in aggregate, and the question is precisely whether a learner can stay extract rules to align with a slow\-moving, noisy trend rather than rely on spurious local\-in\-time patterns\.

A second temporal benchmark constructed from Wikipedia revisions captured between November 2025 and February 2026\[[21](https://arxiv.org/html/2607.07847#bib.bib21)\]\. Stages correspond to three chronological slices \(s1,s2,s3s\_\{1\},s\_\{2\},s\_\{3\}\) of facts whose object value changes over the window\. We evaluate object\-level F1 on each slice after every stage, together with F1 on a control set of*stable*\(unchanged\) facts\. This exposes a failure mode invisible in standard CL benchmarks: methods that successfully update drifted facts may simultaneously degrade unchanged ones —the mirror image of forgetting\.

Task and data\.

Specifically, we construct a six\-year chain of 10\-K filings drawn from SEC EDGAR\[lefteris\_loukas\_2021\_5589195,[24](https://arxiv.org/html/2607.07847#bib.bib24)\], covering fiscal years 2015 through 2020 in chronological order\. Each phase contains∼\\sim500 training filings and 50 held\-out validation filings drawn from that year, with each filing represented in full \(typically 10–13K tokens\) wrapped between custom\[START OF FILING\]/\[END OF FILING\]markers\. Cumulative training tokens grow from∼\\sim6M after phase 1 to∼\\sim37M after phase 6\. The label is a single token,upordown, derived from the issuer’s 30\-day forward simple stock return joined to the filing by Central Index Key \(CIK\) and filing date\. Prior work on financial NLP either treats each year as an independent task or pools all years; we hold the task fixed and let the underlying data\-generating process drift, exposing methods to the actual condition under which deployed financial models age out of distribution\.

In this task,the model predicts the directional sign of an issuer’s 30\-day forward simple stock return from the textual content of its annual 10\-K filing\. The chain has six sequential phases for filing years 2015 through 2020 in chronological order, producing a 6×\\times6 matrix\. Each filing is full\-length SEC EDGAR text wrapped between custom\[START OF FILING\]/\[END OF FILING\]markers \(typically 10–13K tokens\); the gold label is a single token,upordown, derived from a precomputed returns table joined by issuer CIK and filing date\.

Per\-method hyperparameters\.SFT/SDFT/SDPO/GRPO use the optimization recipe from Table[A\.1](https://arxiv.org/html/2607.07847#A1.SS1)with two finance\-specific data settings:data\.max\_prompt\_length= 16,384 to fit the 10\-K body, anddata\.max\_response\_length= 256 since the response is a single token plus chat tail\.

Cartridges configuration\.Per phase, concatenate the cumulative pool of all filings seen so far \(years 1\.\.ii, shuffled at filing granularity\); synthesize 8,192 self\-study Q/A pairs grounded in that pool; distill into a fresh 2,048\-token KV cache\. The cartridge is loaded as a prefix at evaluation, with the full filing body still included in the user message\.

In\-place TTT configuration\.Continual pretraining on the same cumulative corpus, written as VeOmni plaintext\-iterable JSONL with one record per filing \(full body\)\. Fast\-weight updates are activated on a sparse layer set \(paper\-recommended for continual training\)\. The chain hands each phase’s converted HF checkpoint forward as the initial weights for the next phase\.

Table 9:In\-place TTT configuration on the 10\-K benchmark\. Thekltarget is for from\-scratch context lengthening and is not used here\.Scoring\.The prediction string is stripped of any<think\>trace and chat suffixes, then matched against the case\-insensitive whole\-word patterns`\\bup\\b`and`\\bdown\\b`\. The first match wins; ties break in favor ofup\. A prediction is correct iff the matched token equals the gold\.

Further Details on the 10K sentiment task with GEPA

PnL computation\.We construct a daily long\-short strategy from model predictions\. For each filing observed on daytt, the model predicts a directional positionpi∈−1,\+1p\_\{i\}\\in\{\-1,\+1\}, corresponding to DOWN or UP sentiment respectively\. Letrir\_\{i\}denote the realized 30\-day return following the filing\. Daily portfolio returns are computed as the average position\-weighted return across all filings observed on that dayRt=1Nt​∑i=1Ntpi​riR\_\{t\}=\\frac\{1\}\{N\_\{t\}\}\\sum\_\{i=1\}^\{N\_\{t\}\}p\_\{i\}r\_\{i\}, whereNtN\_\{t\}is the number of filings on daytt\. The cumulative PnL is obtained by cumulatively summing the daily returnsRtR\_\{t\}over time\. The optimized PnL curve is causal\. After optimizing the prompt on yearXX, predictions generated using the optimized prompt are evaluated only on yearX\+1X\+1, reflecting a sequential deployment setting\. Figure[8](https://arxiv.org/html/2607.07847#A1.F8)shows that it is possible to construct a strategy with a positive PnL pre\-Covid through a simple rule: attend only to a subset of sections\. Similar to the observed accuracy drop in Figure[4](https://arxiv.org/html/2607.07847#S4.F4), GEPA’s optimisation fails to generalise in a causal forward\-looking manner, dropping PnL performance\.

![Refer to caption](https://arxiv.org/html/2607.07847v1/sentiment10k_hf_yearly_cum_pnl_pred_pos_pooled_training_impact_mean_sections_section_2-section_5-section_7-section_7A-plus3.png)Figure 8:A PnL constructed from retaining only Items 2, 5, 7, 7A, 8, 9, and 9A from the 10\-K filings\. These sections capture the firm’s physical operating footprint, shareholder and capital\-allocation information, management’s discussion of performance and risks, market\-risk exposures, core financial statements, auditor changes or disagreements, and internal\-control quality\.Section\-wise sentiment with possibility to abstainWe additionally explored a setup in which the model was allowed to abstain from a prediction for a given filing sections during GEPA optimization, as not all sections are considered equally informative\. Figure[9](https://arxiv.org/html/2607.07847#A1.F9)reports the relative change in section utilization frequency under this setup\. Some sections became increasingly downweighted, while others were used more heavily by the optimized prompts\. Notably, some down or upweighted sections indeed led to accuracy increases, but not consistently\. Equally, Section 1A \(“Risk Factors”\), which prior work has shown to contain predictive risk sentiment information\[[37](https://arxiv.org/html/2607.07847#bib.bib37)\], remained largely unchanged\. This may indicate that the optimizer failed to infer its predictive usefulness\.

![Refer to caption](https://arxiv.org/html/2607.07847v1/sentiment10k_hf_yearly_none_by_section_delta_baseline_vs_optimized.png)Figure 9:The change in abstain\-rate per section\. On the aggregate, predicting more often \(abstain\-rate decrease\) or abstaining more often \(abstain\-rate increase\) does not consistently lead to an increase in accuracy, showing that GEPA fails to extract section\-level usefulness\.
#### A\.2\.5Agentic\.

##### Update\-mechanism hyperparameters\.

The two update mechanisms plotted in Figure[6](https://arxiv.org/html/2607.07847#S4.F6)are configured as follows\.

*ACE \(Qwen3\-32B\)\.*An online batch pipeline runs each batch of55chains live in the browser, then reflects and curates\. The reflector runs once per chain, tagging each existing bullet helpful, harmful, or neutral; the curator runs once per batch over the combined batch reflections with ADD\-only operations under a50,00050\{,\}000\-token budget\. The reflector/curator LLM is Bedrockqwen\.qwen3\-32b\-v1:0via the Converse API, withtemperature=0\.3=0\.3andmaxTokens=4096=4096\. The playbook has five fixed sections \(strategies\_and\_insights,common\_mistakes\_to\_avoid,problem\_solving\_heuristics,ui\_navigation\_tips,others\) and bullets in the format\[slug\-NNNNN\] helpful=H harmful=X :: content\. We run11epoch over55batches \(≈\\approx25–30 chains, no held\-out test\) and55seeds \(seed0–seed4\), each shuffling the chain order\. At evaluation the playbook is prepended to the browser\-use system prompt, the agent \(qwen3\-32b\) decodes at factory\-default settings, and each cell covers55seeds×\\times3030chains\. Per\-horizon action budgets and timeouts are L1:1010steps /300300s, L3:2020/900900s, L5:2525/12001200s, L10:4040/18001800s\.

*SFT \(Qwen3\-8B\)\.*We fine\-tune Qwen3\-8B with LoRA \(rank3232,α=64\\alpha=64\) on theq,k,v,o,gate,up,downprojections, learning rate5×10−55\\times 10^\{\-5\}, gradient accumulation44, maximum sequence length16,38416\{,\}384, for11epoch, since training beyond one epoch overfit and decreased performance on some task\-suite tasks\. Trajectories are distilled from a teacher agent \(Qwen3\-32B for L1–L5, Qwen3\-235B for L10\); per\-horizon trajectory and training\-example counts are given in Table[10](https://arxiv.org/html/2607.07847#A1.T10)\.

Table 10:Distillation data for the SFT mechanism, by horizon length\. Traj\.: kept \(successful\) trajectories; Train/Val: trajectory split; Steps: training\-step examples\.
##### Suite construction\.

We build the agentic suite on top of WebArena\-Infinity\[[61](https://arxiv.org/html/2607.07847#bib.bib61)\]\. Chain generation is seeded from two sources per app: the app’s description, which enumerates the available features, and its underlying state files, which fix the concrete entities those features can act on\. Conditioning on both lets us generate chains whose steps reference entities that actually exist in the environment\. From these we generate sequential chains of length11,33,55, and1010, in which stepi\+1i\{\+\}1depends on a concrete piece of state that stepiileaves behind in the environment\. In Gmail, for instance, the agent creates a label, renames it, and then applies the renamed label to a specific email, so that each later step is only well\-defined given the state produced by the earlier ones\. The full suite contains510510chains across four apps \(Gmail, Gmail Contacts, Handshake, PayPal\), spanning the four horizon lengths\.

##### Verification\.

Each step is paired with a programmatic verifier that reads the app’s state directly rather than parsing the agent’s narration, so success is checked against the environment\. A step is marked correct only when its verifier confirms the intended state change, and a chain of lengthLLsucceeds only if allLLof its steps pass\. Grounding verification in state ensures that an agent receives no credit for claiming to have completed a step it did not actually perform\.

##### Agent harness\.

All runs use abrowser\-useagent driving a headless Chromium against the live app\. Within a chain we keep a single agent instance alive across all steps: the browser session is not reset between steps, and each follow\-up instruction is appended to the same conversation rather than starting a fresh context, so that when acting on stepi\+1i\{\+\}1the model sees its own prior trajectory—observations, actions, and errors—from the preceding steps\. Each step is given a fixed action budget and is scored independently by its verifier\.

##### Zero\-shot models\.

In addition to the learning methods reported in the main body, we evaluate Deepseek\-v3\.2\[[31](https://arxiv.org/html/2607.07847#bib.bib31)\], GPT\-OSS\-120B\[[1](https://arxiv.org/html/2607.07847#bib.bib1)\], and Gemini\-2\.5\-flash\-lite\[[7](https://arxiv.org/html/2607.07847#bib.bib7)\]zero\-shot under the identical harness\. All three degrade sharply as chain length grows \(Figure[6](https://arxiv.org/html/2607.07847#S4.F6)\)\. The decay tracks the point at which the accumulated trajectory approaches the model’s context window: once earlier trajectory state has to be summarized to fit, the model loses access to the precise environmental cues it needs to ground later actions\. Gemini’s larger context window does not rescue this failure mode—it degrades faster than Deepseek despite more available context—suggesting that the bottleneck is the agent’s ability to actually*use*long trajectory information rather than raw context capacity\.

The same “does experience stick across tasks” question, but where the chain arises from the agent’s own actions rather than from a fixed ordering we picked\. A learner operates in sequential chains of web tasks built on WebArena Infinity\[[61](https://arxiv.org/html/2607.07847#bib.bib61)\], where the environment produced by taskiiis the starting state of taski\+1i\{\+\}1: an action taken early in the chain \(an archived email, an applied label, a navigation state\) persists and affects every subsequent task’s success\. We measure end\-to\-end pass rate as a function of chain lengthL∈\{1,3,5,10\}L\\in\\\{1,3,5,10\\\}, across three apps \(Gmail, Gmail Accounts, PayPal Wallet\) and three backbones \(Deepseek\-v3\.2\[[31](https://arxiv.org/html/2607.07847#bib.bib31)\], Gemini\-2\.5\-Flash\-Lite\[[7](https://arxiv.org/html/2607.07847#bib.bib7)\], GPT\-OSS\-120B\[[1](https://arxiv.org/html/2607.07847#bib.bib1)\]\)\. This is the closest of our benchmarks to deployed agentic use; we treat it as the natural\-regime counterpart to Domain shift’s controlled regime\.

Table 11:Chain pass rate on thegmailapp, by model and horizon length\. Denominator is 30 chains at every length\.Table 12:Chain pass rate on thegmail\-contactsapp, by model and horizon length\. Denominators are 55/32/20/12 chains at L1/L3/L5/L10\.Table 13:Chain pass rate on thepaypalapp, by model and horizon length\. Denominators are 50/24/14/12 chains at L1/L3/L5/L10\.Table 14:Chain pass rate on thehandshakeapp, by model and horizon length\. Denominators are 70/41/30/30 chains at L1/L3/L5/L10\.

## Appendix BBroader Impact

This work develops evaluation infrastructure for continual learning in large language models, with the goal of helping the research community build systems that remain competent as the world changes\. Better continual learning methods could have positive societal impact by reducing the need to retrain large models from scratch as new information becomes available, lowering the associated computational cost and energy consumption\. At the same time, systems that can efficiently update their knowledge after deployment also carry risks: a model that rapidly incorporates new information could more easily be steered toward harmful or false beliefs through targeted data injection\. Our work does not propose a new training method, but to the extent that it accelerates progress in continual learning, these dual\-use considerations apply\. We do not anticipate direct harms from the framework or evaluation protocols introduced here, as they are designed for measuring and understanding model behavior rather than enabling any specific application\.