面向智能体持续学习的循环式框架
摘要
提出 ReHarness 框架，通过在严格在线流式协议下结合前向读写与后向修订（利用轨迹反馈修订检索到的反思），实现 LLM 智能体的持续学习，在三个基准和三种模型骨干上均优于基线方法。
arXiv:2610.12668v1 Announce Type: new
Abstract: Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.
查看缓存全文
缓存时间: 2026/10/12 06:42
# Recurrent Harness for Agentic Continual Learning Source: [https://arxiv.org/html/2610.12668](https://arxiv.org/html/2610.12668) Yuxin XiaoAffiliation:Massachusetts Institute of TechnologyWork done during an internship at Meta MRS Research\.Jianyu WangAffiliation:Meta MRS ResearchYangtian ZhangAffiliation:Yale UniversityYang SongAffiliation:Meta MRS ResearchQifan WangAffiliation:Meta MRS ResearchBailin WangAffiliation:Meta MRS Research ###### Abstract Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning\. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization\. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol\. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream\. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state\. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw\-context baselines, memory\- and reflection\-based methods, and prompt optimization approaches\. Further analysis shows that ReHarness produces source\-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning\. ††date:October 8, 2026††correspondence:Yuxin Xiao at[yuxin102@mit\.edu](mailto:[email protected])and Bailin Wang at[bailin@meta\.com](mailto:[email protected])## 1Introduction Language model agents\([Guo et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib52);[Wang et al\., 2024a](https://arxiv.org/html/2610.12668#bib.bib54)\)increasingly move beyond isolated tasks to operate over long interaction streams\([Ai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib2);[Asawa et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib3);[He et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib4);[Zhu et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib55)\)\. In these settings, agents must adapt to distribution shifts while retaining useful experience from earlier instances in those streams\. Recent agent harnesses\([Guo et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib27);[Ning et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib26)\)enable such continual learning\([Wang et al\., 2024b](https://arxiv.org/html/2610.12668#bib.bib14);[Shi et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib15);[Zheng et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib16)\)by maintaining persistent runtime scaffolding around a fixed large language model \(LLM\), often through memory\([Chhikara et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib28);[Zhang et al\., 2025b](https://arxiv.org/html/2610.12668#bib.bib53);[Dorovatas et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib1)\), reflection\([Lingam et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib8);[Yao et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib41);[Zhang et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib5)\), and prompts\([Agrawal et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib24)\)\. This allows agents to improve online without updating model parameters\. However, existing work typically evolves the harness state only through forward information flow\. After each instance, these systems extract useful information\([Chhikara et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib28);[Yang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib40);[Zhang et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib36)\)or reflect on the resulting trajectory\([Lingam et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib8);[Yao et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib41);[Zhang et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib5)\), integrate the experience into a stateful harness, and later retrieve from it\([Wu et al\., 2026b](https://arxiv.org/html/2610.12668#bib.bib31);[Liu et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib32);[Zhang et al\., 2026d](https://arxiv.org/html/2610.12668#bib.bib33)\)to guide future decisions\. Nevertheless, once written, memories and reflections are rarely updated based on their effectiveness on later instances\. While some methods\([Chhikara et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib28);[Suzgun et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib68);[Zhang et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib5)\)curate the harness state through operations such as add and delete, these operations primarily maintain a coherent memory or reflection bank rather than incorporate feedback from downstream use\. Other methods\([Cai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib12);[Kang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib67);[Lee et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib34);[Zhang et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib36)\)instead optimize the harness state using additional training or validation sets, which are unavailable under a strictly online streaming protocol\. Under continual distribution shifts, outdated memories or overconfident reflections can therefore persist in the harness state and repeatedly misguide future decisions\. To address this gap, we propose ReHarness to close the recurrent loop for agentic continual learning\. ReHarness complements the forward information flow in existing work with a backward revision mechanism\. The revision asks what information from past experience could have been captured differently to better inform the current instance\. Specifically, after each instance, we use a revision agent to revisit the reflections retrieved for that instance\. We use the resulting trajectory as feedback for revision, while requiring any new content to remain grounded in the reflections’ source trajectories\. In this way, ReHarness propagates feedback backward through the textual harness state and carries the revised state forward to future instances\. This enables online harness adaptation under a strictly streaming protocol without instance replay or gradient\-based parameter updates\. Together, forward read and write, coupled with backward revision, form a recurrent harness that can promptly correct outdated memories or overconfident reflections under distribution shifts\. We demonstrate that ReHarness consistently improves continual learning across three benchmarks\([Ai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib2);[Asawa et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib3);[He et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib4)\)and three backbone LLMs\([Anthropic, 2026](https://arxiv.org/html/2610.12668#bib.bib72);[Google DeepMind, 2026](https://arxiv.org/html/2610.12668#bib.bib71);[OpenAI, 2026a](https://arxiv.org/html/2610.12668#bib.bib73)\)\. In particular, ReHarness outperforms strong raw\-context baselines, structured memory and reflection methods\([Chhikara et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib28);[Lingam et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib8);[Ouyang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib6);[Zhang et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib5)\), and prompt optimization approaches\([Agrawal et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib24)\)\. These improvements are statistically significant against all baselines, and the gains remain broadly consistent across the 16 evaluated tasks, including challenging settings where most baselines degrade relative to their stateless backbones\. Further analysis confirms that ReHarness produces source\-grounded, meaningful revisions and improves both stability and plasticity during continual learning\. Qualitative examples and harness state dynamics illustrate how ReHarness corrects early overgeneralizations and revises outdated experience after distribution shifts\. Finally, ablation studies show that ReHarness does not require a large revision set to outperform the strongest full\-history baseline, while using roughly one\-third as many tokens\. We summarize our contributions as follows: - •We propose ReHarness, which combines forward information flow with backward revision to form a recurrent harness for agentic continual learning in an online streaming setting\. - •We show that ReHarness outperforms strong baselines across three continual learning benchmarks and three LLM backbones, with statistically significant gains consistent across challenging tasks\. - •We demonstrate that ReHarness produces source\-grounded and meaningful revisions, adapts overgeneralized or outdated reflections under distribution shifts, and improves stability and plasticity\. ## 2Related Work #### Continual Learning\. Continual learning\([Wang et al\., 2024b](https://arxiv.org/html/2610.12668#bib.bib14);[Shi et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib15);[Zheng et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib16)\)aims to adapt models to evolving data streams under distribution shifts while preventing catastrophic forgetting\. For LLMs, recent work has extended this setting to test\-time adaptation\([Akyürek et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib20);[Hu et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib17);[Zuo et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib19);[Feng et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib18)\)and self\-evolving systems\([Tao et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib22);[Gao et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib21)\)that improve from interaction feedback\. In agentic settings, learning signals are often accumulated through memory\([Zhang et al\., 2025b](https://arxiv.org/html/2610.12668#bib.bib53);[Dorovatas et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib1);[Luo et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib56)\), prompts\([Agrawal et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib24)\), or persistent runtime states\([Park et al\., 2023](https://arxiv.org/html/2610.12668#bib.bib57);[Karten et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib23)\), enabling online adaptation while keeping the underlying LLM parameters fixed\. Recent benchmarks\([Shridhar et al\., 2021](https://arxiv.org/html/2610.12668#bib.bib59);[Zhou et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib58);[Ai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib2);[Asawa et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib3);[Harrington et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib25);[He et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib4)\)evaluate such systems over long interaction streams, measuring both plasticity to distribution shifts and stability of acquired capabilities\. In this work, we study agentic continual learning under a strictly streaming protocol and improve online adaptation in memory space by revising accumulated experience based on its downstream effectiveness\. #### Agent Harness\. Agent harnesses\([Guo et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib27);[Ning et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib26)\)provide runtime scaffolding around a fixed LLM, turning text generation into goal\-directed, multi\-step behavior\. A major line of work on agent harnesses for continual learning focuses on memory, including how raw experiences are stored\([Packer et al\., 2023](https://arxiv.org/html/2610.12668#bib.bib60);[Chhikara et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib28);[Wang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib29);[Rajesh et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib30)\), retrieved\([Liu et al\., 2026b](https://arxiv.org/html/2610.12668#bib.bib61);[Guo et al\., 2026b](https://arxiv.org/html/2610.12668#bib.bib62);[Wu et al\., 2026b](https://arxiv.org/html/2610.12668#bib.bib31);[Liu et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib32);[Zhang et al\., 2026d](https://arxiv.org/html/2610.12668#bib.bib33)\), and optimized\([Kang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib67);[Karten et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib23);[Lee et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib34);[Li et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib13);[Liu et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib35);[Ma et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib63);[Pan et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib66);[Xu et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib64);[Zhang et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib36);[Zhang et al\., 2026b](https://arxiv.org/html/2610.12668#bib.bib65)\)to support future instances\. Some methods optimize memory offline\([Cai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib12);[Ye et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib37)\), while others perform online in\-context learning\([Zheng et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib38);[Sarukkai et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib39)\)by accumulating past demonstrations\. More recent approaches instead abstract experience into high\-level reflections\([Lingam et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib8);[Suzgun et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib68);[Ouyang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib6);[Yao et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib41)\)and manage them through curation\([Li and Huang, 2026](https://arxiv.org/html/2610.12668#bib.bib43);[Wu et al\., 2026a](https://arxiv.org/html/2610.12668#bib.bib69);[Yan et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib42);[Yang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib40);[Zhang et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib5)\)\. However, existing approaches typically either rely on additional validation sets or trajectory replay for harness evolution, or support only forward information flow without adapting previously learned experience based on its downstream effectiveness\. Our work closes this recurrent loop in an online setting by complementing forward accumulation with backward revision\. Figure 1:From stateful agent harnesses to ReHarness\. \(a\) Traditional stateful harness propagates information through forward write and read, allowing an overgeneralized reflection to persist and mislead future instances\. \(b\) ReHarness adds backward revision to refine retrieved reflections using downstream feedback, preventing the same error from propagating to future instances\. ## 3Framework We first motivate recurrent agent harnesses by identifying limitations of forward\-only harness evolution in §[3\.1](https://arxiv.org/html/2610.12668#S3.SS1), then introduce textual backward feedback through revision to address the challenges of agentic continual learning in §[3\.2](https://arxiv.org/html/2610.12668#S3.SS2), and finally propose the ReHarness framework in §[3\.3](https://arxiv.org/html/2610.12668#S3.SS3)\. ### 3\.1From Stateful to Recurrent Agent Harnesses An agent harness for continual learning\([Ai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib2);[Asawa et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib3);[Dorovatas et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib1);[He et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib4)\)provides stateful runtime scaffolding around a decision agentπD\\pi\_\{D\}over a stream of instances\. After observing the previous instance trajectoryτt−1\\tau\_\{t\-1\}, a memory agentπM\\pi\_\{M\}produces a reflectionRt−1R\_\{t\-1\}and writes it into the current harness statehth\_\{t\}\. When the next instance arrives, the harness retrieves relevant reflections fromhth\_\{t\}to assistπD\\pi\_\{D\}, which responds to queryxtx\_\{t\}withyty\_\{t\}and receives rewardrtr\_\{t\}from the environment\. We denote the resulting trajectory of the current instance asτt=\(xt,yt,rt\)\\tau\_\{t\}=\(x\_\{t\},y\_\{t\},r\_\{t\}\)\. This process repeats throughout the stream\{τt\}t=1T\\\{\\tau\_\{t\}\\\}\_\{t=1\}^\{T\}\. Existing work\([Zhao et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib7);[Lingam et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib8);[Ouyang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib6);[Zhang et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib5)\)thus evolves the harness state through alternating forward write \(τt−1→Rt−1→ht\\tau\_\{t\-1\}\\rightarrow R\_\{t\-1\}\\rightarrow h\_\{t\}\) and forward read \(ht→Rt−1→τth\_\{t\}\\rightarrow R\_\{t\-1\}\\rightarrow\\tau\_\{t\}\)\. Figure[1](https://arxiv.org/html/2610.12668#S2.F1)\(a\) illustrates this forward information flow in stateful harnesses with a simple sorting example\. After correctly sorting\{3,7,4\}\\\{3,7,4\\\}in descending order atτt−1\\tau\_\{t\-1\},πM\\pi\_\{M\}writes the reflection “put larger numbers first” intohth\_\{t\}\. Atτt\\tau\_\{t\}, however, the query asks for ascending order\. RetrievingRt−1R\_\{t\-1\}therefore misleadsπD\\pi\_\{D\}, which produces an incorrect answer\. AlthoughπM\\pi\_\{M\}then writes a new reflectionRtR\_\{t\}recommending “put smaller numbers first,” the misleadingRt−1R\_\{t\-1\}remains inht\+1h\_\{t\+1\}and is retrieved again forτt\+1\\tau\_\{t\+1\}because the queries contain overlapping numbers\. Consequently, the same overgeneralized reflection misguidesπD\\pi\_\{D\}again\. This example shows that a stateful harness with only the forward path is incomplete for continual learning\. This forward\-only reflection process does not account for the effectiveness of reflections when reused in future instances under distribution shifts\. Misleading reflections can persist in the harness state and propagate errors through later retrievals\. We instead propose a*recurrent*agent harness that combines forward information flow with backward feedback from future instances\. ### 3\.2Textual Backward Feedback through Revision Agentic continual learning under a strictly streaming setting poses unique challenges for backward feedback\. First, each instance is observed only once, so we cannot return to past instances to collect additional counterfactual outcomes for offline optimization\([Cai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib12);[Li et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib13);[Pan et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib66)\)\. Second, we do not have additional validation sets to select among candidate harness updates\([Kang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib67);[Lee et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib34);[Zhang et al\., 2026b](https://arxiv.org/html/2610.12668#bib.bib65)\)\. Third, frontier API\-based agents\([Anthropic, 2026](https://arxiv.org/html/2610.12668#bib.bib72);[Google DeepMind, 2026](https://arxiv.org/html/2610.12668#bib.bib71);[OpenAI, 2026a](https://arxiv.org/html/2610.12668#bib.bib73)\)do not expose model parameters, precluding gradient\-based updates\. On the other hand, prior work\([Pryzant et al\., 2023](https://arxiv.org/html/2610.12668#bib.bib10);[Shinn et al\., 2023](https://arxiv.org/html/2610.12668#bib.bib11);[Yuksekgonul et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib9)\)shows that natural language feedback can provide an effective learning signal without parameter updates\. We therefore realize backward feedback in the textual harness state, allowing the harness to adapt online as the interaction stream evolves\. Specifically, we introduce a revision agentπR\\pi\_\{R\}that revisits reflections used for the current instance based on their observed effectiveness\. Using the same sorting example in Figure[1](https://arxiv.org/html/2610.12668#S2.F1)\(b\), after observing the current instance trajectoryτt\\tau\_\{t\},πR\\pi\_\{R\}revisesRt−1R\_\{t\-1\}in toRt−1′R^\{\\prime\}\_\{t\-1\}using its source trajectoryτt−1\\tau\_\{t\-1\}and the downstream evidence fromτt\\tau\_\{t\}\. The revision aims to produce a reflection that, if used in place ofRt−1R\_\{t\-1\}, could have prevented mistakes inτt\\tau\_\{t\}\. This typically involves incorporating previously missed evidence fromτt−1\\tau\_\{t\-1\}or correcting an exaggerated hypothesis inRt−1R\_\{t\-1\}\. Here,πR\\pi\_\{R\}adds a limiting condition toRt−1′R^\{\\prime\}\_\{t\-1\}by specifying “larger first only if descending\.” ReplacingRt−1R\_\{t\-1\}withRt−1′R^\{\\prime\}\_\{t\-1\}yields a revised harness stateht′h^\{\\prime\}\_\{t\}, establishing the backward feedback pathτt→Rt−1′→ht′\\tau\_\{t\}\\rightarrow R^\{\\prime\}\_\{t\-1\}\\rightarrow h^\{\\prime\}\_\{t\}\. Meanwhile, we useRt−1′R^\{\\prime\}\_\{t\-1\}as a demonstration for the forward write, which generates a new reflectionRtR\_\{t\}fromτt\\tau\_\{t\}with a similar limiting condition: “smaller first only if ascending\.” Combining the backward revision with this forward reflection yields the next harness stateht\+1=ht′⊕Rth\_\{t\+1\}=h^\{\\prime\}\_\{t\}\\oplus R\_\{t\}, completing the recurrent update\. Consequently, retrievingRt−1′R^\{\\prime\}\_\{t\-1\}fromht\+1h\_\{t\+1\}helpsπD\\pi\_\{D\}answerτt\+1\\tau\_\{t\+1\}correctly\. ### 3\.3ReHarness: Closing the Recurrent Loop We now instantiate this recurrent harness framework as ReHarness in Figure[1](https://arxiv.org/html/2610.12668#S2.F1)\(b\)\. #### Step 1: Forward Read\. Letht=\{Ri\}i<th\_\{t\}=\\\{R\_\{i\}\\\}\_\{i<t\}denote the current harness state\. Given queryxtx\_\{t\}, we provide the decision agentπD\\pi\_\{D\}with the top\-kkrelevant reflections retrieved fromhth\_\{t\}: ℐt=\{i1,…,ik\}=Retrieval\(xt,ht,k\),yt=πD\(xt,\{Ri\}i∈ℐt\)\.\\mathcal\{I\}\_\{t\}=\\\{i\_\{1\},\\dots,i\_\{k\}\\\}=\\operatorname\{Retrieval\}\(x\_\{t\},h\_\{t\},k\),\\qquad y\_\{t\}=\\pi\_\{D\}\\\!\\left\(x\_\{t\},\\\{R\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{t\}\}\\right\)\.\(1\)The environment then provides rewardrtr\_\{t\}, forming the instance trajectoryτt=\(xt,yt,rt\)\\tau\_\{t\}=\(x\_\{t\},y\_\{t\},r\_\{t\}\)\. #### Step 2: Backward Revision\. After observingτt\\tau\_\{t\}, the revision agentπR\\pi\_\{R\}independently revisits each retrieved reflection using its source trajectoryτi\\tau\_\{i\}and its effectiveness on the current instanceτt\\tau\_\{t\}: Ri′=πR\(Ri,τi,τt\),∀i∈ℐt\.R^\{\\prime\}\_\{i\}=\\pi\_\{R\}\\\!\\left\(R\_\{i\},\\tau\_\{i\};\\tau\_\{t\}\\right\),\\qquad\\forall i\\in\\mathcal\{I\}\_\{t\}\.\(2\)Importantly,τt\\tau\_\{t\}serves only as feedback for assessing and revisingRiR\_\{i\}\. Any new content inRi′R^\{\\prime\}\_\{i\}must remain grounded in its source trajectoryτi\\tau\_\{i\}to prevent information leakage\. The revision may therefore incorporate evidence missed inτi\\tau\_\{i\}, correct unsupported hypotheses inRiR\_\{i\}, or refine the conditions under which the reflection should apply\. Replacing the retrieved reflections with their revised versions while retaining all unretrieved reflections yields the intermediate stateht′h\_\{t\}^\{\\prime\}\. #### Step 3: Forward Write\. Finally, the memory agentπM\\pi\_\{M\}reflects onτt\\tau\_\{t\}to produce a new reflection: Rt=πM\(τt,\{Ri′\}i∈ℐt\),ht\+1=ht′⊕Rt\.R\_\{t\}=\\pi\_\{M\}\\\!\\left\(\\tau\_\{t\};\\\{R^\{\\prime\}\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{t\}\}\\right\),\\qquad h\_\{t\+1\}=h\_\{t\}^\{\\prime\}\\oplus R\_\{t\}\.\(3\)We promptπM\\pi\_\{M\}to explore operational evidence and retain only well\-supported interpretations or strategies\. The revised reflections\{Ri′\}i∈ℐt\\\{R^\{\\prime\}\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{t\}\}serve as demonstrations forπM\\pi\_\{M\}, while the prompt encouragesRtR\_\{t\}to capture complementary perspectives not covered by those demonstrations\. Table 1:Mean normalized gain across task streams for each benchmark–backbone combination\. Highest gains are in bold\. ReHarness outperforms all baselines, suggesting that backward revision of accumulated experience effectively adapts agent harnesses to continual distribution shifts\. ## 4Experiment ### 4\.1Experiment Setup #### Baselines\. We compare ReHarness against the following strong baselines\. - •Mem0\([Chhikara et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib28)\)maintains atomic memories as harness state, retrieving top\-kkentries for the decision agent and incrementally extracting memories after each instance\. - •RBank\([Ouyang et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib6)\)maintains an append\-only bank of reusable reflections as harness state and retrieves top\-kkreflections for each instance\. - •DoT\([Lingam et al\., 2025](https://arxiv.org/html/2610.12668#bib.bib8)\)generates diverse reflections from the top\-kkretrieved prior experiences and uses them as the runtime harness for the current instance\. - •ACE\([Zhang et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib5)\)maintains a global reflection playbook as its harness state and curates it through add, update, delete, and merge operations for coherence\. - •GEPA\-Onlineadapts GEPA\([Agrawal et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib24)\)to our strictly online streaming setting, preserving reflective optimization of the decision prompt as persistent harness state, but replaces the original validation\-based population search with sequential updates from each observed trajectory\. - •Full Rawuses all prior instance trajectories in the stream as harness context, whileTop\-K Rawretrieves only thekkmost relevant trajectories for in\-context learning\. We evaluate all methods with three frontier LLM backbones: Claude Opus 4\.8\([Anthropic, 2026](https://arxiv.org/html/2610.12668#bib.bib72)\), GPT\-5\.5\([OpenAI, 2026a](https://arxiv.org/html/2610.12668#bib.bib73)\), and Gemini 3\.7 Flash\([Google DeepMind, 2026](https://arxiv.org/html/2610.12668#bib.bib71)\), using the same backbone for all agents within each method\. We use each model’s default generation configuration with high thinking effort and exclude hidden reasoning traces from all harness operations\. Retrieval uses the top five entries ranked by Qwen3\-Embedding\-8B\([Zhang et al\., 2025a](https://arxiv.org/html/2610.12668#bib.bib51)\)and applies the same query\-to\-query comparison across all methods\. We provide the prompts used by ReHarness in §[A](https://arxiv.org/html/2610.12668#A1)\. #### Datasets\. We evaluate ReHarness against baselines on three continual learning benchmarks under a strictly online streaming protocol, with no training or validation set and no repeated trials\. - •CL\-Bench\([Asawa et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib3)\)contains six tasks spanning cohort studies, codebase adaptation\([Jimenez et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib44)\), database exploration\([Hou et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib45)\), sales prediction, blind spectrum monitoring, and exploitable poker\. Each task contains 12 to 120 multi\-round instances with explicit distribution shifts and task\-specific evaluation metrics\. - •MemoryBench\([Ai et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib2)\)contains long\-input, long\-output generation tasks across multiple domains\. We use its five English tasks from HelloBench\([Que et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib46)\)and WritingBench\([Wu et al\., 2026c](https://arxiv.org/html/2610.12668#bib.bib47)\), covering academic, creative, and political writing, with 17 to 47 single\-round instances per task, evaluated by GPT\-5\.6\-Sol\([OpenAI, 2026b](https://arxiv.org/html/2610.12668#bib.bib74)\)\. - •MemoryArena\([He et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib4)\)contains five domains spanning travel planning\([Xie et al\., 2024](https://arxiv.org/html/2610.12668#bib.bib50)\), web shopping\([Yao et al\., 2022](https://arxiv.org/html/2610.12668#bib.bib48)\), progressive web search\([Chen et al\., 2026](https://arxiv.org/html/2610.12668#bib.bib49)\), and formal reasoning in mathematics and physics, with 20 to 270 tasks per domain\. Each task comprises 2 to 16 interdependent instances, each involving multiple interaction rounds\. Across these benchmarks, we standardize each task as a continual learning stream and each constituent interaction unit as an instance \(i\.e\.,τt\\tau\_\{t\}\)\. ReHarness and all baselines maintain harness state across instances within a stream and reset it between streams\. To compare performance across tasks with different native metrics, we follow[Asawa et al\. \(2026\)](https://arxiv.org/html/2610.12668#bib.bib3)and report normalized gain,\(rmethod−rstateless\)/\(rmax−rstateless\)\(r\_\{\\text\{method\}\}\-r\_\{\\text\{stateless\}\}\)/\(r\_\{\\max\}\-r\_\{\\text\{stateless\}\}\), averaged over each stream\. Here,rmaxr\_\{\\max\}denotes the maximum achievable reward under each task’s native metric\. This metric quantifies the improvement over the stateless backbone agent from learning prior experience within the stream\. We repeat each experiment over three independently sampled stream rollouts\. Within each rollout, all methods share the same benchmark initialization and random seed, then evolve independently\. Figure 2:Task\-level normalized gain for each method, averaged across the three backbone LLMs\. Performance varies across tasks, while ReHarness remains effective on nearly all of them\. ### 4\.2ReHarness Consistently Improves Agentic Continual Learning #### Overall Performance\. Table[1](https://arxiv.org/html/2610.12668#S3.T1)reports the mean normalized gain across task streams for each benchmark–backbone combination\. ReHarness achieves the highest overall normalized gain of0\.2400\.240and ranks first or tied in eight of the nine combinations\. Consistent with[Asawa et al\. \(2026\)](https://arxiv.org/html/2610.12668#bib.bib3), raw\-context methods are strong baselines: Top\-K Raw and Full Raw outperform most structured memory and reflection methods on average, particularly on CL\-Bench and MemoryArena\. Reflection\-based methods, such as RBank, DoT, and ACE, are less consistent, with some settings yielding negative gains\. Similarly, GEPA\-Online degrades substantially on MemoryBench with Gemini, suggesting that prompt optimization alone is less robust for continual learning\. To assess statistical significance, we compare ReHarness against each baseline using two\-sided paired Wilcoxon signed\-rank tests with Bonferroni correction, treating each seeded task stream as one observation\. ReHarness significantly outperforms all baselines \(padj<0\.05p\_\{\\mathrm\{adj\}\}<0\.05\), including the strong raw\-context baselines Top\-K Raw \(padj=0\.039p\_\{\\mathrm\{adj\}\}=0\.039\) and Full Raw \(padj=0\.043p\_\{\\mathrm\{adj\}\}=0\.043\)\. Overall, these results suggest that revising reflections with ReHarness is more effective than retaining raw history or relying on forward\-only accumulation\. #### Task\-Level Results\. We next present the task\-level breakdown for each benchmark in Figure[2](https://arxiv.org/html/2610.12668#S4.F2), averaged across the three backbone LLMs\. The gains vary substantially across tasks, with many methods showing marginal or even negative improvements over their stateless backbones\. On CL\-Bench, ReHarness leads or remains close to the best on most tasks, but shows only marginal gains on exploitable poker\. On MemoryBench, while most methods perform well on HelloBench \(HB\-Academic and HB\-Creative\), only ReHarness achieves consistent gains across WritingBench \(WB\-Academic, WB\-Creative, and WB\-Politics\)\. On MemoryArena, Mem0 is notably strong on travel planning, and GEPA\-Online performs well on progressive web search, while ReHarness remains consistently competitive across all five domains\. A closer examination suggests that, unlike the more deterministic codebase and database settings, exploitable poker is more stochastic: hand\-level rewards can be weakly coupled with strategy quality due to hidden cards and chance\. Raw\-context baselines are less exposed to this credit\-assignment issue because they preserve the original hand trajectories and defer their interpretation to the decision agent at each new hand\. By contrast, ReHarness is less effective when downstream rewards provide a noisy signal for backward revision, and benefits most when feedback is diagnostic for revising reusable reflections\. The full raw results are in §[A](https://arxiv.org/html/2610.12668#A1)\. #### Token Efficiency\. We further examine the token consumption and API cost of ReHarness\. Despite requiring one reflection call for forward write andkkrevision calls for backward revision, in addition to the decision calls shared by all methods, ReHarness consumes fewer total tokens than the two strong raw\-context baselines\. This is because compact reflections substantially reduce the repeated decision\-time context cost of raw trajectories, especially for long, multi\-round instances\. Across one rollout over all CL\-Bench tasks with Claude, ReHarness consumes∼37\.19\{\\sim\}37\.19M input and∼4\.38\{\\sim\}4\.38M output tokens, totaling∼41\.57\{\\sim\}41\.57M\. By comparison, Top\-K Raw consumes∼52\.20\{\\sim\}52\.20M input and∼0\.46\{\\sim\}0\.46M output tokens \(∼52\.66\{\\sim\}52\.66M total\), while Full Raw consumes∼141\.68\{\\sim\}141\.68M input and∼0\.44\{\\sim\}0\.44M output tokens \(∼142\.12\{\\sim\}142\.12M total\), or1\.3×1\.3\\timesand3\.4×3\.4\\timesas many total tokens as ReHarness, respectively\. Thus, although ReHarness generates more output tokens, its input\-token savings more than offset this overhead in total consumption\. With prompt caching and Claude API pricing, the corresponding costs are approximately$85\\$85for Top\-K Raw,$254\\$254for ReHarness, and$314\\$314for Full Raw\. ReHarness therefore remains substantially cheaper than Full Raw while achieving a higher normalized gain \(0\.2170\.217vs\.0\.1910\.191\), whereas Top\-K Raw is cheaper but achieves a lower gain \(0\.1650\.165\)\. ## 5Analysis ### 5\.1Is ReHarness Revision Source\-Grounded? Table 2:Average grounded, leakage, and unsupported claim rates of revised reflections on CL\-Bench for each backbone, together with the average change per revision\. The grounded rate remains above95%95\\%for all three LLMs and decreases only marginally after revision, suggesting that ReHarness uses the downstream trajectory primarily as feedback rather than as a source of leaked information\.We first examine whether backward revision in Equation[2](https://arxiv.org/html/2610.12668#S3.E2)remains grounded in its source trajectoryτi\\tau\_\{i\}or imports information fromτt\\tau\_\{t\}\. Following[Min et al\. \(2023\)](https://arxiv.org/html/2610.12668#bib.bib70), we decompose each revised reflectionRi′R^\{\\prime\}\_\{i\}into factual claims using GPT\-5\.6\-Sol\([OpenAI, 2026b](https://arxiv.org/html/2610.12668#bib.bib74)\)\. We classify each claim as: \(1\) grounded, if entailed byτi\\tau\_\{i\}; \(2\) leaked, if not entailed byτi\\tau\_\{i\}but entailed byτt\\tau\_\{t\}; or \(3\) unsupported, if entailed by neither\. Table[2](https://arxiv.org/html/2610.12668#S5.T2)reports the average rates across CL\-Bench tasks for each backbone and their average change per revision\. Across all backbones, over95%95\\%of claims remain source\-grounded, with leakage at0\.80\.8–2\.1%2\.1\\%and unsupported claims at0\.50\.5–2\.6%2\.6\\%\. Moreover, revision changes these rates only marginally, with grounding decreasing by at most0\.3%0\.3\\%per revision\. These results indicate that the backbone LLMs largely follow the grounding constraint as intended, and ReHarness usesτt\\tau\_\{t\}primarily as feedback rather than as a source of leaked information\. ### 5\.2Is ReHarness Capable of Revision? We next validate whether ReHarness revises the harness state as intended\. The goal of revision is to use downstream feedback to improve previously learned experience for future instances\. We therefore consider a counterfactual replay that modifies the online streaming protocol\. After revising the harness state intoht′h^\{\\prime\}\_\{t\}based on the completed trajectoryτt\\tau\_\{t\}, we reuseht′h^\{\\prime\}\_\{t\}to assist the decision agent on the same instance\. This differs from ReHarness, which appliesht′h^\{\\prime\}\_\{t\}only to the next streamed instanceτt\+1\\tau\_\{t\+1\}\. Importantly,ht′h^\{\\prime\}\_\{t\}remains grounded in past trajectories and usesτt\\tau\_\{t\}only as feedback on their effectiveness\. The replay thus asks: given the current outcome, what information already available in past trajectories should have been captured differently to better inform that instance? Table[3](https://arxiv.org/html/2610.12668#S5.T3)reports task\-level normalized gains on CL\-Bench with Claude for ReHarness, counterfactual replay, and a no\-revision baseline that removes the revision agentπR\\pi\_\{R\}and retains only forward information flow\. Counterfactual replay substantially improves over ReHarness across all tasks\. This suggests that revision can recover or correct information from past trajectories using current feedback, producing a more effective harness state had those revisions been available beforehand\. Hence, counterfactual replay confirms that the backward mechanism produces meaningful revisions\. However, it violates the online streaming protocol\. ReHarness instead applies the revised stateht′h^\{\\prime\}\_\{t\}only to the next instanceτt\+1\\tau\_\{t\+1\}, using feedback fromτt\\tau\_\{t\}to improve future instances\. Compared with the no\-revision baseline, ReHarness achieves higher gains across all tasks, especially in database exploration, sales prediction, and blind spectrum monitoring\. This indicates that backward revision remains beneficial when applied only to subsequent instances in the stream\. Table 3:Normalized gain under counterfactual revision on CL\-Bench with Claude\. Counterfactual replay reapplies the revised harness to the same instance and substantially improves over ReHarness and the no\-revision baseline\. This shows that the backward mechanism enables meaningful revision\. ### 5\.3What Does ReHarness Revise during Continual Learning? Now that we have established the source grounding and capability of ReHarness revision, we use qualitative examples from CL\-Bench with Claude in Figure[3](https://arxiv.org/html/2610.12668#S5.F3)to examine what it revises\. Figure[3](https://arxiv.org/html/2610.12668#S5.F3)\(a\) shows an early\-stage example from blind spectrum monitoring, where the agent infers persistent transmitters from sequential spectrum scans\. After observingτ2\\tau\_\{2\}, the memory agentπM\\pi\_\{M\}overgeneralizes the evidence and recommends “report\[ing\] plausible additional channels to fill large silent gaps\.” Because the task rewards overlap with the ground\-truth occupied spectrum, both missing real transmitters and hallucinating nonexistent ones hurt performance\. When this reflection is retrieved fromh3h\_\{3\}forτ3\\tau\_\{3\}, the decision agentπD\\pi\_\{D\}follows the speculative guidance, adds unobserved transmitters, and receives a lower reward\. Using this outcome as backward feedback, the revision agentπR\\pi\_\{R\}marks the hypothesis as “speculative” and changes the guidance to “prioritize retaining confirmed peaks over adding unconfirmed ones,” producingh3′h^\{\\prime\}\_\{3\}\. When the revised reflection is retrieved fromh4h\_\{4\},πD\\pi\_\{D\}avoids speculative fillers and substantially improves the reward\. This example shows how ReHarness corrects an early overgeneralization before it misguides future decisions\. The second example in Figure[3](https://arxiv.org/html/2610.12668#S5.F3)\(b\) shows a late\-stage revision from database exploration in CL\-Bench, where the agent queries an unknown SQLite database\. Duringτ2\\tau\_\{2\},πD\\pi\_\{D\}fails to inspect null values in a price column “prc,” yetπM\\pi\_\{M\}eventually consolidates the reflection that “the graded value came from prc” after repeated use in the early stream\. Afterτ20\\tau\_\{20\}, CL\-Bench introduces a distribution shift by changing the schema and adding “prc\_v2\.” When the old\-schema reflection is retrieved fromh23h\_\{23\}forτ23\\tau\_\{23\},πD\\pi\_\{D\}checks only “prc,” while the correct answer lies in “prc\_v2\.” Using this failure as backward feedback,πR\\pi\_\{R\}changes the rule to “not assume which column is graded” and “enumerate all price\-like columns,” producingh23′h^\{\\prime\}\_\{23\}\. When this revised reflection is retrieved forτ25\\tau\_\{25\},πD\\pi\_\{D\}checks both “prc” and “prc\_v2” and obtains the correct answer in two queries, substantially improving the reward\. This example shows how ReHarness revises outdated strategies under distribution shifts through backward feedback\. §[A](https://arxiv.org/html/2610.12668#A1)provides the full trajectories, reflections, and revisions\. Figure 3:Qualitative examples of ReHarness revision during continual learning\. \(a\) In blind spectrum monitoring, ReHarness corrects an early overgeneralized reflection with evidence\-grounded guidance\. \(b\) In database exploration, ReHarness adapts an outdated reflection to a schema shift\.Table 4:Stability–plasticity decomposition on CL\-Bench\. ReHarness achieves the highest average gains in both, showing robust transfer across distribution shifts and effective adaptation afterward\. ### 5\.4How Does ReHarness Revise during Continual Learning? We then investigate how ReHarness revises accumulated experience during continual learning\. Following[Asawa et al\. \(2026\)](https://arxiv.org/html/2610.12668#bib.bib3), we decompose normalized gain on CL\-Bench into stability and plasticity\. Stability measures gain on the first instance after a distribution shift, before post\-shift feedback is available\. Plasticity measures gain on subsequent instances before the next shift, where methods can adapt to the current distribution\. Because the two categories contain different numbers of instances, their absolute gains are not directly comparable\. As shown in Table[4](https://arxiv.org/html/2610.12668#S5.T4), Mem0, RBank, DoT, ACE, and GEPA\-Online exhibit negative stability gains on average, indicating that their forward\-only harness states can become detrimental under distribution shifts\. In contrast, ReHarness achieves the highest stability gain on average and the highest plasticity gain for every backbone LLM\. These results suggest that backward revision enables more robust transfer across shifts while supporting effective adaptation once post\-shift feedback becomes available, as illustrated in Figure[3](https://arxiv.org/html/2610.12668#S5.F3)\. We further visualize harness state dynamics on CL\-Bench codebase adaptation with Claude\. Each transition fromhth\_\{t\}toht\+1h\_\{t\+1\}adds a new reflectionRtR\_\{t\}through forward write and revises retrieved reflections\{Ri′\}i∈ℐt\\\{R^\{\\prime\}\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{t\}\}through backward revision\. In Figure[4](https://arxiv.org/html/2610.12668#S5.F4)\(a\), diagonal heatmap entries mark new reflections, while later non\-zero entries in the same row indicate revisions\. We measure revision magnitude by character\-level edit distance between the old and revised reflections from the same source trajectory\. Before the distribution shift atτ9\\tau\_\{9\}, revisions to older reflections diminish as the harness state stabilizes and the gain rises\. After the shift, older reflections require larger revisions to adapt to the new distribution\. For example, the transition fromh10h\_\{10\}toh11h\_\{11\}substantially revises reflections fromτ0\\tau\_\{0\}andτ1\\tau\_\{1\}\. These revisions correct outdated experience while the gain remains relatively stable after the shift\. As the stream progresses, revision increasingly concentrates on recent reflections relevant to the shifted distribution\. Together, these dynamics show how ReHarness balances stability and plasticity by preserving useful experience while revising outdated reflections\. Figure 4:\(a\) Harness state dynamics by ReHarness on CL\-Bench codebase adaptation with Claude\. Revisions diminish as reflections stabilize before the distribution shift, then rise afterward to update outdated reflections\. \(b\) Top\-kkscaling on CL\-Bench with Claude\. ReHarness improves sharply up tok=5k=5and stabilizes for moderatekk, while larger values yield diminishing returns and higher cost\. ### 5\.5What Makes ReHarness Effective? Finally, we conduct ablations to identify what makes ReHarness effective\. In Figure[4](https://arxiv.org/html/2610.12668#S5.F4)\(b\), we vary the number of retrieved and subsequently revised reflections,kk, on a log scale for ReHarness and Top\-K Raw on CL\-Bench with Claude\. Top\-K Raw approaches Full Raw askkreaches2020, suggesting diminishing returns from additional in\-context examples\. For ReHarness, normalized gain rises sharply fromk=1k=1to55, remains relatively stable fork∈\{5,10,20\}k\\in\\\{5,10,20\\\}, and declines for largerkk\. This suggests that overly largekkcan introduce unnecessary revision and potential reflection drift while increasing computational cost\. ReHarness therefore does not require a large revision set\. We fixk=5k=5throughout the main experiments and report this sweep as a post\-hoc sensitivity analysis\. Furthermore, we ablate two components of ReHarness\. The effect of removing backward revision is already established in Table[3](https://arxiv.org/html/2610.12668#S5.T3)\. Removing the revised reflections\{Ri′\}i∈ℐt\\\{R^\{\\prime\}\_\{i\}\\\}\_\{i\\in\\mathcal\{I\}\_\{t\}\}used as demonstrations forπM\\pi\_\{M\}in Equation[3](https://arxiv.org/html/2610.12668#S3.E3)reduces the normalized gain on CL\-Bench with Claude from0\.2170\.217to0\.1910\.191, indicating that revised experience helps guide subsequent reflection generation\. ## 6Discussion In this paper, we propose ReHarness for agentic continual learning beyond forward\-only harness evolution\. ReHarness complements forward read and write with backward revision, using downstream feedback to revise past reflections while grounding new content in their source trajectories\. Across three continual learning benchmarks and three LLM backbones, ReHarness consistently outperforms strong baselines, with statistically significant gains across challenging tasks\. Further analyses show that ReHarness produces source\-grounded and meaningful revisions, corrects overgeneralized or outdated reflections under distribution shifts, and improves stability and plasticity\. Together, our results highlight recurrent harness adaptation as a promising direction for continually refining accumulated experience under a strictly streaming setting\. Despite these gains, ReHarness underperforms on exploitable poker in CL\-Bench, where downstream rewards provide noisy signals for backward revision\. Moreover, while ReHarness is more token\-efficient than the strongest full\-history raw\-context baseline, it still stores raw trajectories for future revision, incurring additional storage overhead\. We aim to address these limitations in future work\. ## References - Agrawalet al\.\(2026\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. KhattabGEPA: reflective prompt evolution can outperform reinforcement learning\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1),[5th item](https://arxiv.org/html/2610.12668#S4.I1.i5.p1.1)\. - Aiet al\.\(2026\)Q\. Ai, Y\. Tang, C\. Wang, J\. Long, W\. Su, and Y\. LIUMemoryBench: a benchmark for memory and continual learning in LLM systems\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1),[2nd item](https://arxiv.org/html/2610.12668#S4.I2.i2.p1.1)\. - Akyüreket al\.\(2025\)E\. Akyürek, M\. Damani, A\. Zweiger, L\. Qiu, H\. Guo, J\. Pari, Y\. Kim, and J\. AndreasThe surprising effectiveness of test\-time training for few\-shot learning\.InICML,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Anthropic \(2026\)AnthropicClaude opus 4\.8 system card\.Note:[https://www\.anthropic\.com/claude\-opus\-4\-8\-system\-card](https://www.anthropic.com/claude-opus-4-8-system-card)Accessed: 2026\-08\-15Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.12668#S4.SS1.SSS0.Px1.p1.2)\. - Asawaet al\.\(2026\)P\. Asawa, C\. M\. Glaze, G\. Orlanski, R\. Ramakrishnan, B\. Xu, A\. Biswal, V\. S\. Chen, F\. Sala, M\. Zaharia, and J\. E\. GonzalezContinual learning bench: evaluating frontier ai systems in real\-world stateful environments\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1),[1st item](https://arxiv.org/html/2610.12668#S4.I2.i1.p1.1),[§4\.1](https://arxiv.org/html/2610.12668#S4.SS1.SSS0.Px2.p1.2),[§4\.2](https://arxiv.org/html/2610.12668#S4.SS2.SSS0.Px1.p1.1),[§5\.4](https://arxiv.org/html/2610.12668#S5.SS4.p1.1)\. - Caiet al\.\(2026\)Y\. Cai, X\. Guo, X\. Huang, J\. Du, can huang, W\. Huang, W\. Ma, Y\. Hu, A\. Zeng, J\. Tang, and X\. SunFrom player to master: enhancing test\-time learning of LLM agents via reinforcement learning over memory\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Chenet al\.\(2026\)Z\. Chen, X\. Ma, S\. Zhuang, P\. Nie, K\. Zou, S\. Sharifymoghaddam, A\. Liu, J\. Green, K\. Patel, R\. Meng,et al\.BrowseComp\-plus: a fair and disentangled evaluation benchmark for deep search agents\.InACL,Cited by:[3rd item](https://arxiv.org/html/2610.12668#S4.I2.i3.p1.1)\. - Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: building production\-ready ai agents with scalable long\-term memory\.InECAI,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[1st item](https://arxiv.org/html/2610.12668#S4.I1.i1.p1.1)\. - Dorovataset al\.\(2026\)V\. Dorovatas, M\. Schwerin, A\. D\. Bagdanov, L\. Caccia, A\. Carta, L\. Charlin, B\. Hammer, T\. L\. Hayes, T\. Hess, C\. Kanan, D\. Kudithipudi, X\. Liu, V\. Lomonaco, J\. Mendez\-Mendez, D\. Patil, A\. Prabhu, E\. Ricci, T\. Tuytelaars, G\. M\. van de Ven, L\. Wang, J\. van de Weijer, J\. Choi, M\. Mundt, and R\. AljundiPosition: modular memory is the key to continual learning agents\.InICML Position Paper Track,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1)\. - Fenget al\.\(2026\)G\. Feng, S\. Luo, K\. Hua, G\. Zhang, W\. Huang, D\. He, and T\. CaiIn\-place test\-time training\.InICLR,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Gaoet al\.\(2026\)H\. Gao, J\. Geng, W\. Hua, M\. Hu, X\. Juan, H\. Liu, S\. Liu, J\. Qiu, X\. Qi, Q\. Ren, Y\. Wu, H\. WANG, H\. Xiao, Y\. Zhou, S\. Zhang, J\. Zhang, J\. Xiang, Y\. Fang, Q\. Zhao, D\. Liu, C\. Qian, Z\. Wang, M\. Hu, H\. Wang, Q\. Wu, H\. Ji, and M\. WangA survey of self\-evolving agents: what, when, how, and where to evolve on the path to artificial super intelligence\.Transactions on Machine Learning Research\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Google DeepMind \(2026\)Google DeepMindGemini 3\.7 flash: model card\.Note:[https://deepmind\.google/models/model\-cards/gemini\-3\-7\-flash/](https://deepmind.google/models/model-cards/gemini-3-7-flash/)Accessed: 2026\-08\-15Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.12668#S4.SS1.SSS0.Px1.p1.2)\. - Guoet al\.\(2026a\)J\. Guo, Z\. Hao, C\. Wang, C\. Fan, T\. Luo, H\. Li, Y\. Gao, H\. Mei, J\. Peng, R\. Xu,et al\.From question answering to task completion: a survey on agent system and harness design\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Guoet al\.\(2026b\)S\. Guo, Y\. Du, H\. Chen, Y\. Chang, and J\. WangCASCADE: case\-based continual adaptation for large language models during deployment\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Guoet al\.\(2024\)T\. Guo, X\. Chen, Y\. Wang, R\. Chang, S\. Pei, N\. V\. Chawla, O\. Wiest, and X\. ZhangLarge language model based multi\-agents: a survey of progress and challenges\.InIJCAI,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1)\. - Harringtonet al\.\(2026\)A\. Harrington, N\. Saxena, M\. Murphy, A\. Borovykh, Z\. Yun, S\. Kamath, A\. E\. Kyi, T\. Darrell, J\. Malik, and Y\. BaiWhen does continual learning require learning\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Heet al\.\(2026\)Z\. He, Y\. Wang, C\. Zhi, Y\. Hu, T\. Chen, L\. Yin, Z\. Chen, T\. A\. Wu, S\. Ouyang, Z\. Wang, J\. Pei, J\. McAuley, Y\. Choi, and A\. PentlandBenchmarking agent memory in interdependent multi\-session agentic tasks\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1),[3rd item](https://arxiv.org/html/2610.12668#S4.I2.i3.p1.1)\. - Houet al\.\(2026\)Y\. Hou, J\. Li, X\. Fu, Z\. He, A\. Yan, X\. Chen, and J\. McAuleyBridging language and items for retrieval and recommendation: benchmarking llms as semantic encoders\.InACL,Cited by:[1st item](https://arxiv.org/html/2610.12668#S4.I2.i1.p1.1)\. - Huet al\.\(2025\)J\. Hu, Z\. Zhang, G\. Chen, X\. Wen, C\. Shuai, W\. Luo, B\. Xiao, Y\. Li, and M\. TanTest\-time learning for large language models\.InICML,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. NarasimhanSWE\-bench: can language models resolve real\-world github issues?\.InICLR,Cited by:[1st item](https://arxiv.org/html/2610.12668#S4.I2.i1.p1.1)\. - Kanget al\.\(2026\)B\. Kang, J\. Gu, J\. Lv, W\. Li, L\. Wang, and Y\. GaoHarness continual learning: continual adaptation beyond model parameters\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Kartenet al\.\(2026\)S\. Karten, J\. Zhang, T\. Upaa Jr, R\. Feng, W\. Li, C\. Shi, C\. Jin, and K\. VodrahalliContinual harness: online adaptation for self\-improving foundation agents\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Leeet al\.\(2026\)Y\. Lee, R\. S\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. FinnMeta\-harness: end\-to\-end optimization of model harnesses\.InCOLM,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Li and Huang \(2026\)J\. Li and H\. HuangTextual stochastic gradient descent: discrete optimization of external memory for reasoning language agents\.InICML,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Liet al\.\(2026\)Y\. Li, Z\. Lin, A\. Deng, X\. Zhang, Y\. He, S\. Ji, T\. Cao, and B\. HooiJust\-in\-time reinforcement learning: continual learning in LLM agents without gradient updates\.InICML,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Lingamet al\.\(2025\)V\. Lingam, B\. O\. Tehrani, S\. Sanghavi, G\. Gupta, S\. Ghosh, L\. Liu, J\. Huan, and A\. DeorasEnhancing language model agents using diversity of thoughts\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1),[3rd item](https://arxiv.org/html/2610.12668#S4.I1.i3.p1.1)\. - Liuet al\.\(2026a\)J\. Liu, Y\. Su, P\. Xia, S\. Han, Z\. Zheng, C\. Xie, M\. Ding, and H\. YaoSimpleMem: efficient lifelong memory for LLM agents\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Liuet al\.\(2026b\)J\. Liu, X\. Ye, P\. Xia, Z\. Zheng, C\. Xie, M\. Ding, and H\. YaoEvolvemem: self\-evolving memory architecture via autoresearch for llm agents\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Liuet al\.\(2026c\)Z\. Liu, Z\. Shi, Y\. Sang, B\. He, M\. Lin, T\. Wei, D\. Wang, B\. Dumoulin, W\. Jin, and H\. LuAdaptive auto\-harness: sustained self\-improvement for agentic system deployment on open\-ended task streams\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Luoet al\.\(2026\)J\. Luo, Y\. Tian, C\. Cao, Z\. Luo, H\. Lin, K\. Li, C\. Kong, R\. Yang, and J\. MaFrom storage to experience: a survey on the evolution of llm agent memory mechanisms\.InACL Findings,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Maet al\.\(2026\)Y\. Ma, S\. Tu, Q\. Zhang, D\. Li, L\. Li, and D\. ZhaoMemChain: learning interpretable memory traces for memory\-augmented llm agents\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Minet al\.\(2023\)S\. Min, K\. Krishna, X\. Lyu, M\. Lewis, W\. Yih, P\. Koh, M\. Iyyer, L\. Zettlemoyer, and H\. HajishirziFActScore: fine\-grained atomic evaluation of factual precision in long form text generation\.InEMNLP,Cited by:[§5\.1](https://arxiv.org/html/2610.12668#S5.SS1.p1.1)\. - Ninget al\.\(2026\)X\. Ning, K\. Tieu, D\. Fu, T\. Wei, Z\. Li, Y\. Bei, J\. Zou, M\. Ai, Z\. Liu, T\. Li,et al\.Code as agent harness\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - OpenAI \(2026a\)OpenAIGPT\-5\.5 system card\.Note:[https://openai\.com/index/gpt\-5\-5\-system\-card/](https://openai.com/index/gpt-5-5-system-card/)Accessed: 2026\-08\-15Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.12668#S4.SS1.SSS0.Px1.p1.2)\. - OpenAI \(2026b\)OpenAIGPT\-5\.6 system card\.Note:[https://deploymentsafety\.openai\.com/gpt\-5\-6](https://deploymentsafety.openai.com/gpt-5-6)Accessed: 2026\-08\-15Cited by:[2nd item](https://arxiv.org/html/2610.12668#S4.I2.i2.p1.1),[§5\.1](https://arxiv.org/html/2610.12668#S5.SS1.p1.1)\. - Ouyanget al\.\(2026\)S\. Ouyang, J\. Yan, I\. Hsu, Y\. Chen, K\. Jiang, Z\. Wang, R\. Han, L\. Le, S\. Daruki, X\. Tang, V\. Tirumalashetty, G\. Lee, M\. Rofouei, H\. Lin, J\. Han, C\. Lee, and T\. PfisterReasoningBank: scaling agent self\-evolving with reasoning memory\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1),[2nd item](https://arxiv.org/html/2610.12668#S4.I1.i2.p1.1)\. - Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemgpt: towards llms as operating systems\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Panet al\.\(2026\)W\. Pan, S\. Liu, C\. Lin, J\. Zeng, X\. Tang, X\. Zhou, Y\. Lu, and X\. JiaRetrospective harness optimization: improving llm agents via self\-preference over trajectory rollouts\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Parket al\.\(2023\)J\. S\. Park, J\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. BernsteinGenerative agents: interactive simulacra of human behavior\.InUIST,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Pryzantet al\.\(2023\)R\. Pryzant, D\. Iter, J\. Li, Y\. Lee, C\. Zhu, and M\. ZengAutomatic prompt optimization with “gradient descent” and beam search\.InEMNLP,Cited by:[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Queet al\.\(2024\)H\. Que, F\. Duan, L\. He, Y\. Mou, W\. Zhou, J\. Liu, W\. Rong, Z\. M\. Wang, J\. Yang, G\. Zhang,et al\.Hellobench: evaluating long text generation capabilities of large language models\.arXiv preprint\.Cited by:[2nd item](https://arxiv.org/html/2610.12668#S4.I2.i2.p1.1)\. - Rajeshet al\.\(2026\)S\. Rajesh, P\. S\. Holur, M\. Y\. Turali, C\. Duan, and V\. RoychowdhuryPanini: continual learning in token space via structured memory\.InICML,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Sarukkaiet al\.\(2025\)V\. Sarukkai, Z\. Xie, and K\. FatahalianSelf\-generated in\-context examples improve LLM agents for sequential decision\-making tasks\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Shiet al\.\(2025\)H\. Shi, Z\. Xu, H\. Wang, W\. Qin, W\. Wang, Y\. Wang, Z\. Wang, S\. Ebrahimi, and H\. WangContinual learning of large language models: a comprehensive survey\.ACM Computing Surveys\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. R\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InNeurIPS,Cited by:[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Shridharet al\.\(2021\)M\. Shridhar, X\. Yuan, M\. Cote, Y\. Bisk, A\. Trischler, and M\. Hausknecht\{ALFW\}orld: aligning text and embodied environments for interactive learning\.InICLR,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Suzgunet al\.\(2026\)M\. Suzgun, M\. Yuksekgonul, F\. Bianchi, D\. Jurafsky, and J\. ZouDynamic cheatsheet: test\-time learning with adaptive memory\.InEACL,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Taoet al\.\(2024\)Z\. Tao, T\. Lin, X\. Chen, H\. Li, Y\. Wu, Y\. Li, Z\. Jin, F\. Huang, D\. Tao, and J\. ZhouA survey on self\-evolution of large language models\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Wanget al\.\(2024a\)L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin,et al\.A survey on large language model based autonomous agents\.Frontiers of computer science\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1)\. - Wanget al\.\(2024b\)L\. Wang, X\. Zhang, H\. Su, and J\. ZhuA comprehensive survey of continual learning: theory, method and application\.IEEE TPAMI\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Wanget al\.\(2026\)Z\. Wang, H\. Jin, Z\. Wang, W\. Wang, and L\. ZhangLightWM: training\-free hierarchical working memory for small language model agents\.InICML,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Wuet al\.\(2026a\)B\. Wu, Z\. Ding, J\. Huang, and Y\. ZhaoForget to improve: on\-device llm\-agent continual learning via budget\-curated memory\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Wuet al\.\(2026b\)Y\. Wu, L\. Zhang, Y\. Zhou, M\. Wang, B\. Peng, S\. Li, X\. Fan, and Z\. ZhaoRemember when it matters: proactive memory agent for long\-horizon agents\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Wuet al\.\(2026c\)Y\. Wu, J\. Mei, M\. Yan, C\. Li, S\. Lai, Y\. Ren, W\. Zijia, J\. Zhang, M\. Wu, Q\. Jin, and F\. HuangWritingBench: a comprehensive benchmark for generative writing\.InNeurIPS Datasets and Benchmarks Track,Cited by:[2nd item](https://arxiv.org/html/2610.12668#S4.I2.i2.p1.1)\. - Xieet al\.\(2024\)J\. Xie, K\. Zhang, J\. Chen, T\. Zhu, R\. Lou, Y\. Tian, Y\. Xiao, and Y\. SuTravelPlanner: a benchmark for real\-world planning with language agents\.InICML,Cited by:[3rd item](https://arxiv.org/html/2610.12668#S4.I2.i3.p1.1)\. - Xuet al\.\(2026\)W\. Xu, A\. Sordoni, C\. Singh, Z\. Gero, M\. Galley, X\. Yuan, and J\. GaoTest\-time learning with an evolving library\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Yanet al\.\(2026\)S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan,et al\.Memory\-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.InACL,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Yanget al\.\(2026\)K\. Yang, Z\. Chen, X\. He, J\. Jiang, M\. Galley, C\. Wang, J\. Gao, J\. Han, and C\. ZhaiPlugMem: a task\-agnostic plugin memory module for LLM agents\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Yaoet al\.\(2022\)S\. Yao, H\. Chen, J\. Yang, and K\. NarasimhanWebshop: towards scalable real\-world web interaction with grounded language agents\.InNeurIPS Datasets and Benchmarks Track,Cited by:[3rd item](https://arxiv.org/html/2610.12668#S4.I2.i3.p1.1)\. - Yaoet al\.\(2026\)T\. Yao, Y\. Chen, Y\. Zheng, P\. Li, Z\. Shen, and K\. ZhangParamMem: augmenting language agents with parametric reflective memory\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Yeet al\.\(2026\)Y\. Ye, H\. Jiang, F\. Jiang, T\. Lan, Y\. Du, B\. Fu, X\. Shi, Q\. Jia, L\. Wang, and W\. LuoUMEM: unified memory extraction and management framework for generalizable memory\.InICML,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Yuksekgonulet al\.\(2024\)M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, Z\. Huang, C\. Guestrin, and J\. ZouTextgrad: automatic” differentiation” via text\.arXiv preprint\.Cited by:[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Zhanget al\.\(2026a\)G\. Zhang, H\. Ren, C\. Zhan, J\. Wang, H\. Zhu, W\. Zhou, and S\. YANMemEvolve: meta\-evolution of agent memory systems\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Zhanget al\.\(2026b\)H\. Zhang, S\. Zhang, K\. Li, C\. Zhang, Y\. Chen, Y\. Zhang, L\. Bai, and S\. HuSelf\-harness: harnesses that improve themselves\.arXiv preprint\.Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.12668#S3.SS2.p1.1)\. - Zhanget al\.\(2026c\)Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li, U\. Thakker, J\. Zou, and K\. OlukotunAgentic context engineering: evolving contexts for self\-improving language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§1](https://arxiv.org/html/2610.12668#S1.p4.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1),[4th item](https://arxiv.org/html/2610.12668#S4.I1.i4.p1.1)\. - Zhanget al\.\(2025a\)Y\. Zhang, M\. Li, D\. Long, X\. Zhang, H\. Lin, B\. Yang, P\. Xie, A\. Yang, D\. Liu, J\. Lin,et al\.Qwen3 embedding: advancing text embedding and reranking through foundation models\.arXiv preprint\.Cited by:[§4\.1](https://arxiv.org/html/2610.12668#S4.SS1.SSS0.Px1.p1.2)\. - Zhanget al\.\(2026d\)Y\. Zhang, Y\. Li, A\. Payani, and L\. WangAdaMEM: test\-time adaptive memory for language agents\.InICML,Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p2.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Zhanget al\.\(2025b\)Z\. Zhang, Q\. Dai, X\. Bo, C\. Ma, R\. Li, X\. Chen, J\. Zhu, Z\. Dong, and J\. WenA survey on the memory mechanism of large language model\-based agents\.ACM Transactions on Information Systems\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpel: llm agents are experiential learners\.InAAAI,Cited by:[§3\.1](https://arxiv.org/html/2610.12668#S3.SS1.p1.1)\. - Zhenget al\.\(2025\)J\. Zheng, S\. Qiu, C\. Shi, and Q\. MaTowards lifelong learning of large language models: a survey\.ACM Computing Surveys\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1),[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Zhenget al\.\(2024\)L\. Zheng, R\. Wang, X\. Wang, and B\. AnSynapse: trajectory\-as\-exemplar prompting with memory for computer control\.InICLR,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px2.p1.1)\. - Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried,et al\.Webarena: a realistic web environment for building autonomous agents\.InICLR,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. - Zhuet al\.\(2026\)D\. Zhu, X\. Zhou, S\. Qin, X\. Zhu, H\. Ding, S\. Zhong, Z\. Wen, Z\. Xie, C\. Gou, L\. Ren,et al\.EdgeBench: unveiling scaling laws of learning from real\-world environments\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2610.12668#S1.p1.1)\. - Zuoet al\.\(2025\)Y\. Zuo, K\. Zhang, L\. Sheng, S\. Qu, G\. Cui, X\. Zhu, H\. Li, Y\. Zhang, X\. Long, E\. Hua, B\. Qi, Y\. Sun, Z\. Ma, L\. Yuan, N\. Ding, and B\. ZhouTTRL: test\-time reinforcement learning\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2610.12668#S2.SS0.SSS0.Px1.p1.1)\. ## Appendix AAppendix We report task\-level normalized gains for each LLM on CL\-Bench, MemoryBench, and MemoryArena in Tables[5](https://arxiv.org/html/2610.12668#A1.T5),[6](https://arxiv.org/html/2610.12668#A1.T6), and[7](https://arxiv.org/html/2610.12668#A1.T7), respectively\. We provide the prompts used by the decision agentπD\\pi\_\{D\}, memory agentπM\\pi\_\{M\}, and revision agentπR\\pi\_\{R\}in ReHarness in Figure[5](https://arxiv.org/html/2610.12668#A1.F5)\. We also provide the full qualitative revision examples discussed in §[5\.3](https://arxiv.org/html/2610.12668#S5.SS3)for blind spectrum monitoring and database exploration in Figures[6](https://arxiv.org/html/2610.12668#A1.F6)and[7](https://arxiv.org/html/2610.12668#A1.F7), respectively\. Table 5:Normalized gain for each LLM on each CL\-Bench task, with the highest value in bold\.Table 6:Normalized gain for each LLM on each MemoryBench task, with the highest value in bold\.Table 7:Normalized gain for each LLM on each MemoryArena task, with the highest value in bold\.Figure 5:Prompts used by the decision \(top\), memory \(middle\), and revision \(bottom\) agents in ReHarness\.Figure 6:Full qualitative revision example for blind spectrum monitoring from Figure[3](https://arxiv.org/html/2610.12668#S5.F3)\(a\)\.Figure 7:Full qualitative revision example for database exploration from Figure[3](https://arxiv.org/html/2610.12668#S5.F3)\(b\)\.
相似文章
面向智能体持续学习的循环Harness框架
论文提出ReHarness，一种面向智能体持续学习的循环harness框架，在严格在线流式协议下通过backward revision补充前向信息流，从而在分布偏移下改进语言模型智能体的记忆与反思更新。
ClawLoop：通过原生 harness 训练循环实现智能体自我改进
ClawLoop 提出一个无需外部教师模型的自改进训练框架，通过任务合成、训练与诊断反馈三个阶段形成自我维持循环，持续提升基于 harness 环境的 LLM 智能体能力，并在多个基准上验证了效果。
Self-Harness: 自我改进的Harness
Self-Harness 提出了一种新范式，其中基于LLM的智能体通过挖掘模型特定的弱点、提出框架修改，并通过回归测试验证这些修改，从而迭代地改进自身的运行框架，在Terminal-Bench-2.0上跨多个基础模型取得了显著的性能提升。
自我进化代理脚手架框架
提出一个自我进化代理框架（Harness RSI），在保持模型权重固定的情况下，通过反馈驱动的修订与进化优化，持续改进代理的软件脚手架（提示、记忆、工作流和可执行组件），并借助持续 PAC 学习为泛化与保留能力提供理论保证。
@TheTuringPost：本周必读研究 Harness-1：基于状态外化束带的搜索智能体的强化学习 R…
本文探讨了大型语言模型中持续学习的复兴，强调了离线整合（即“睡眠”）的必要性，以防止灾难性遗忘，并使模型在部署后能够保持更新和专业化。