Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents

# Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents
Source: [https://arxiv.org/html/2609.18304](https://arxiv.org/html/2609.18304)
Liuyi YaoAffiliation:School of Cyber Science and Engineering, Wuhan University Alibaba GroupEmail:[wanges17@whu\.edu\.cn](mailto:[email protected])Affiliation:Corresponding authorsYaliang LiAffiliation:School of Cyber Science and Engineering, Wuhan University Alibaba GroupEmail:[wu@whu\.edu\.cn](mailto:[email protected])Enshu WangEmail:[yly287738@alibaba\-inc\.com](mailto:[email protected])Libing WuAffiliation:Corresponding authorsEmail:[yaliang\.li@alibaba\-inc\.com](mailto:[email protected])

###### Abstract

Large language model \(LLM\) agents increasingly tackle long\-horizon tasks through multi\-step environment interaction, yet a single erroneous action can alter subsequent states and observations, causing errors to compound over time\. Existing methods either correct the context without repairing altered environment states or restore earlier states while discarding useful experience, making it difficult to both eliminate failure conditions and avoid repeating past mistakes\. We argue that reliable recovery should instead be treated as a rollback\-boundary control problem that jointly determines*when*to intervene,*where*to resume, and*what*information should survive recovery\. Based on this view, we propose Rollback\-Induced Reflection \(RIR\), a unified recovery framework that restores execution to a selected prior state while carrying forward reusable knowledge distilled from the abandoned trajectory to guide subsequent decisions\. We further characterize recovery through a unified operator over rollback depth and retained memory, providing a general view of state restoration and knowledge retention\. Experiments on three long\-horizon benchmarks demonstrate that RIR consistently improves task performance across multiple LLM backbones, with structured reflection memory preserving useful experience and selective rollback enabling efficient recovery\.

## 1Introduction

Large language model \(LLM\) agents can solve complex long\-horizon tasks through reasoning, tool use, and environment interaction\([Wei et al\., 2022](https://arxiv.org/html/2609.18304#bib.bib4);[Yao et al\., 2022](https://arxiv.org/html/2609.18304#bib.bib5);[Park et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib2)\)\. Yet such tasks are highly sensitive to erroneous actions: a single misstep can alter subsequent states and observations, causing later decisions to rely on corrupted context\. Errors therefore compound over time and can drive the agent progressively away from a valid solution trajectory\([Hao et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib18)\)\.

Existing remedies intervene at two levels\.Information\-levelmethods append corrective feedback to guide future decisions\([Madaan et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib6);[Shinn et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib7);[Zhao et al\., 2024](https://arxiv.org/html/2609.18304#bib.bib10);[Kim et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib13)\), but cannot undo environmental changes already caused by an erroneous action, and contaminated observations remaining in context may contradict the corrective advice itself\.State\-levelmethods instead restore execution to an earlier checkpoint and discard the erroneous suffix\([Zhou et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib14);[Li et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib15);[Zhang et al\., 2026c](https://arxiv.org/html/2609.18304#bib.bib17);[Hao et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib18)\), but state restoration alone does not specify what information from that suffix should survive\. Retain too little and the agent may repeat the same failure; overgeneralize a local failure and viable alternatives may be incorrectly ruled out\.

A reliable rollback mechanism must therefore answer three coupled questions\. First,when to roll back: is the current branch still productive exploration, or has it begun to propagate errors? Intervening too early curtails valid exploration; too late lets the consequences of an error compound\. Second,where to roll back: which checkpoint eliminates the conditions that produced the failure while preserving the most valid progress? Going too far back sacrifices completed work; not far enough leaves the underlying obstacle in place\. Third,what to roll back: which claims in the discarded suffix are invalidated along with the execution state, and which should survive as knowledge for the next attempt?

Together, these questions define a rollback boundary:whendecides whether to draw it,whereplaces it along the trajectory, andwhatcontrols which information crosses it\. At this boundary, the system restores the environment and branch\-local agent context to the selected checkpoint and removes the subsequent suffix from the active trajectory\. The suffix, however, need not be discarded wholesale\. State claims invalidated by restoration are removed, while reusable observations and lessons are distilled into reflective knowledge for the next attempt\. Rollback thus becomes both a state transition and an opportunity to reconstruct the decision context:rollback the world, keep the reflection\.

Building on this view, we proposeRollback\-Induced Reflection \(RIR\), a recovery\-control framework for long\-horizon LLM agents\. Forwhen,Hybrid Adaptive Reviewcombines agent\-initiated and adaptive scheduled reviews to assess whether the current branch should continue or recover\. Forwhere,Coarse\-to\-Fine Restore Localizationfirst narrows the search to a causally relevant interval and then selects a checkpoint that balances failure removal against progress preservation\. Forwhat,Rollback\-Consistent Reflection Memoryseparates branch\-local state restored with the checkpoint from reusable knowledge that persists across rollback\. The memory stores the stable task objective, reusable environment knowledge, past\-attempt milestones, and conditioned failure analysis, while deliberately excluding the agent’s current state to avoid reintroducing stale claims after restoration\.

Formally, we characterize RIR through a unified recovery operator parameterized by a rollback depthkkand the memoryℳ\+\\mathcal\{M\}^\{\+\}retained across the recovery boundary\. We prove that common correction and rollback mechanisms are restricted cases of the unified operator, and that their induced recovery\-policy classes are therefore contained within the RIR recovery space\. This policy\-class inclusion leads directly to an optimal\-value monotonicity result, under which the best task\-completion probability attainable by RIR is no lower than that of any such restricted mechanism\. Empirically, RIR consistently outperforms representative baselines across three long\-horizon benchmarks and two LLM backbones, improving average success rate by up to 6\.57 percentage points while maintaining selective recovery under constrained interaction budgets\. Our contributions are as follows:

- •We recast trajectory contamination in long\-horizon agent tasks as a problem ofrollback boundaryplacement and identify three coupled questions that recovery must resolve: when to intervene, where to resume, and what information should survive the rollback\.
- •We proposeRollback\-Induced Reflection \(RIR\), which combines adaptive review, coarse\-to\-fine restore localization, and rollback\-consistent reflection to recover execution state without discarding reusable knowledge from failed branches\.
- •We formalize RIR through a unified recovery operator and establish the generality of its recovery space together with an optimal\-value monotonicity result over the induced recovery\-policy classes\. Experiments on long\-horizon benchmarks further validate its effectiveness under constrained interaction budgets\.

## 2Related Work

LLM agents increasingly use context\-management mechanisms such as external memory, experience retrieval, and selective context construction to support long\-horizon interaction\([Yu et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib21);[Chhikara et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib22);[Jia et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib23);[Lu et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib24)\)\. However, these methods generally do not address what happens once an erroneous action has already altered the environment: whether and how execution itself should be recovered\([Zhang et al\., 2026c](https://arxiv.org/html/2609.18304#bib.bib17);[Hu et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib25);[Wu et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib16)\)\. We therefore focus on rollback\-oriented recovery and group prior work into two broad forms\.

Information\-level correction\.Self\-Refine\([Madaan et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib6)\), AgenTracer\([Zhang et al\., 2026b](https://arxiv.org/html/2609.18304#bib.bib26)\), Reflexion\([Shinn et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib7)\), and ReflAct\([Kim et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib13)\)improve subsequent decisions through feedback or reflection, while ExpeL\([Zhao et al\., 2024](https://arxiv.org/html/2609.18304#bib.bib10)\), AutoGuide\([Fu et al\., 2024](https://arxiv.org/html/2609.18304#bib.bib11)\), and G\-Memory\([Zhang et al\., 2026a](https://arxiv.org/html/2609.18304#bib.bib12)\)reuse experience across tasks\. Although effective feedback can improve future behavior\([Huang et al\., 2024](https://arxiv.org/html/2609.18304#bib.bib8);[Kamoi et al\., 2024](https://arxiv.org/html/2609.18304#bib.bib9)\), information\-level correction cannot undo environmental consequences already caused by erroneous actions, and retained branch\-local state claims may become stale after restoration\.

State\-level rollback\.GA\-Rollback\([Li et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib15)\)and WebRollback\([Zhang et al\., 2026c](https://arxiv.org/html/2609.18304#bib.bib17)\)explicitly restore earlier states in interactive trajectories, while SRC\([Hao et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib18)\)and DART\([Yang et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib19)\)study rollback for training\-data construction and structured recoverability, respectively\. The closest concurrent work, AgentRewind\([Zhuang et al\., 2026](https://arxiv.org/html/2609.18304#bib.bib20)\), checkpoints aligned agent and environment states and preserves textual memory across rewinds, but relies primarily on agent\-initiated recovery and does not explicitly model the validity of retained information after restoration\. These methods demonstrate the value of explicitly restoring execution state, but typically focus on failure detection or restore\-point selection rather than the full recovery boundary\.

In contrast, RIR treats the recovery boundary itself as an explicit control problem: it unifies state restoration and context reconstruction by jointly deciding*when*to recover,*where*to resume, and*what*information remains valid across the rollback boundary\. Unlike methods that treat reflection or rollback in isolation, RIR restores execution while preserving reusable knowledge and excluding state claims invalidated by restoration\.

## 3Problem Formulation

We consider a partially observable long\-horizon interactive taskℰ=\(𝒮,𝒜,𝒪,P,Ω,Rg\)\\mathcal\{E\}=\(\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{O\},P,\\Omega,R\_\{g\}\), where𝒮\\mathcal\{S\},𝒜\\mathcal\{A\}, and𝒪\\mathcal\{O\}denote the state, action, and observation spaces\. The environment evolves according toP⁡\(st\+1\|st,at\)P\(s\_\{t\+1\}\|s\_\{t\},a\_\{t\}\)and emits observations throughΩ⁡\(ot\+1\|st\+1\)\\Omega\(o\_\{t\+1\}\|s\_\{t\+1\}\)\. Given task objectivegg, the outcome functionRg​\(sT,yT\)R\_\{g\}\(s\_\{T\},y\_\{T\}\)evaluates the terminal execution between the final statesTs\_\{T\}and the ground truth stateyTy\_\{T\}, it is binary for verifiable tasks and may be real\-valued when graded rewards are available\.

At steptt, the base agent follows an LLM policyat∼πθ\(⋅\|g,ht,ℳ\),a\_\{t\}\\sim\\pi\_\{\\theta\}\(\\cdot\|g,h\_\{t\},\\mathcal\{M\}\),whereℳ\\mathcal\{M\}is the persistent Reflection Memory andht=\(o0,a0,…,at−1,ot\)h\_\{t\}=\(o\_\{0\},a\_\{0\},\\ldots,a\_\{t\-1\},o\_\{t\}\)is the interaction history of the current branch\. The key distinction is thathth\_\{t\}is checkpointed with execution, whereasℳ\\mathcal\{M\}is maintained outside checkpoints and may carry knowledge across recovery attempts\. The action space consists of final responses, ordinary task tools, and a recovery\-control tool:𝒜=𝒜resp∪𝒜tool,𝒜tool=𝒜task∪\{rollback\}\.\\mathcal\{A\}=\\mathcal\{A\}\_\{\\mathrm\{resp\}\}\\cup\\mathcal\{A\}\_\{\\mathrm\{tool\}\},\\mathcal\{A\}\_\{\\mathrm\{tool\}\}=\\mathcal\{A\}\_\{\\mathrm\{task\}\}\\cup\\\{\\texttt\{rollback\}\\\}\.Actions in𝒜task\\mathcal\{A\}\_\{\\mathrm\{task\}\}interact with the environment, while𝒜resp\\mathcal\{A\}\_\{\\mathrm\{resp\}\}terminates the episode with a final response\. Therollbacktool instead submits a recovery request and does not directly modify the environment\.

Rollback\.Before each executable action, the system stores a checkpointCi=\(si,hi\),C\_\{i\}=\(s\_\{i\},h\_\{i\}\),containing the environment state and branch\-local context at stepii\. Letℐt=\{i≤t∣Ci​is restorable at step​t\}\\mathcal\{I\}\_\{t\}=\\\{\\,i\\leq t\\mid C\_\{i\}\\text\{ is restorable at step \}t\\,\\\}denote the set of admissible restore points\. For a selected restore pointr∈ℐtr\\in\\mathcal\{I\}\_\{t\}, we define the trajectory discarded by recovery as the*abandoned suffix*:τr:t=\(ar,or\+1,…,at−1,ot\)\.\\tau\_\{r:t\}=\(a\_\{r\},o\_\{r\+1\},\\ldots,a\_\{t\-1\},o\_\{t\}\)\.

A rollback first extracts reusable knowledge fromτr:t\\tau\_\{r:t\}to update the Reflection Memory, and then restores the environment and branch\-local context to\(sr,hr\)\(s\_\{r\},h\_\{r\}\)\. Execution subsequently resumes from the restored checkpoint under the updated memory\. We definek=t−rk=t\-ras the*rollback depth*\. We allow the limiting casek=0k=0, corresponding to context correction without state restoration\.

Objective\.RIR leaves the parametersθ\\thetaof the base agent unchanged and instead introduces a test\-time recovery control policyΠ\\Pi\. Given an agent\-call budgetBagentB\_\{\\mathrm\{agent\}\}and a rollback budgetBrbB\_\{\\mathrm\{rb\}\}, RIR seeks to maximize task success under bounded interaction:

maxΠ⁡𝔼πθ,Π,ℰ​\[Rg​\(sT,yT\)\]s\.t\.Nagent≤Bagent,Nrb≤Brb\.\\max\_\{\\Pi\}\\;\\mathbb\{E\}\_\{\\pi\_\{\\theta\},\\Pi,\\mathcal\{E\}\}\\\!\\left\[R\_\{g\}\(s\_\{T\},y\_\{T\}\)\\right\]\\quad\\text\{s\.t\.\}\\quad N\_\{\\mathrm\{agent\}\}\\leq B\_\{\\mathrm\{agent\}\},\\;\\;N\_\{\\mathrm\{rb\}\}\\leq B\_\{\\mathrm\{rb\}\}\.\(1\)

## 4Rollback\-Induced Reflection Framework

### 4\.1Overview

As shown in Figure[1](https://arxiv.org/html/2609.18304#S4.F1), RIR organizes recovery around three coupled decisions:Whendecides whether the current branch should continue,Whereselects the checkpoint to resume from, andWhatdetermines which information from the abandoned suffix should survive the rollback\. Algorithm[1](https://arxiv.org/html/2609.18304#alg1)summarizes the complete execution loop\.

During execution, RIR stores checkpoints along the current branch\. A review is triggered either by an agent recovery request or by the adaptive review schedule\. If the branch is judged promising, the system continues and adjusts the next review interval\. Otherwise, RIR first localizes a restore checkpoint coarse\-to\-fine, then distills reflective knowledge from the suffix about to be abandoned\. Finally, the system restores the environment and the agent context while retaining the updated Reflection Memory to guide the new attempt\. The procedure can be summarized as:

ℳ\+=ϕ\(ℳ,τr:t\),\(st,ht,ℳ\)→rollback\(sr,hr,ℳ\+\),r=t−k,\\mathcal\{M\}^\{\+\}=\\phi\(\\mathcal\{M\},\\tau\_\{r:t\}\),\\quad\(s\_\{t\},h\_\{t\},\\mathcal\{M\}\)\\;\\xrightarrow\{\\;\\text\{rollback\}\\;\}\\;\(s\_\{r\},h\_\{r\},\\mathcal\{M\}^\{\+\}\),\\quad r=t\-k,\(2\)whereϕ\\phidenotes the information update instantiated in Section[4\.4](https://arxiv.org/html/2609.18304#S4.SS4)\. Together with rollback depthkk, the pair\(k,ℳ\+\)\(k,\\mathcal\{M\}^\{\+\}\)defines the unified recovery operator analyzed in Section[4\.5](https://arxiv.org/html/2609.18304#S4.SS5)\. Equation[2](https://arxiv.org/html/2609.18304#S4.E2)makes RIR’s central asymmetry explicit: the execution world and branch\-local context return to the past, while reflective knowledge acquired from interactions that have already occurred is carried forward\.

![Refer to caption](https://arxiv.org/html/2609.18304v1/RIR_FRAMEWORK_FINAL.png)Figure 1:Overview of the Rollback\-Induced Reflection \(RIR\) framework\.
### 4\.2When: Hybrid Adaptive Review

Intervening too early truncates legitimate exploration, whereas intervening too late allows errors to compound\. To strike a balance, RIR invokes a single reviewer through two complementary triggers:agent\-initiated review, where the agent callsrollbackas a soft recovery request, andadaptive scheduled review, where the controller inspects the trajectory after a dynamic review horizon\. The former captures failures recognized by the agent itself, while the latter detects stagnation or errors that the agent may overlook\. Their combination reduces reliance on either imperfect signal alone: agent requests enable timely intervention, while scheduled reviews provide an external safeguard against unnoticed failure\. The rollback tool schema is provided in Appendix\.

Lettrevt\_\{\\mathrm\{rev\}\}denote the most recent review step andLtL\_\{t\}the current review horizon\. A review is triggered when:

ReviewEventt=𝕀\[at=rollback\]∨𝕀\[t−trev≥Lt\]\.\\operatorname\{ReviewEvent\}\_\{t\}=\\mathbb\{I\}\\\!\\left\[a\_\{t\}=\\texttt\{rollback\}\\right\]\\;\\lor\\;\\mathbb\{I\}\\\!\\left\[t\-t\_\{\\mathrm\{rev\}\}\\geq L\_\{t\}\\right\]\.\(3\)Importantly, callingrollbackonly requests review and does not directly restore the environment\. This separation prevents the agent’s local uncertainty from being converted immediately into an irreversible recovery decision\. When a review is triggered, the LLM\-based reviewer evaluates the task objectivegg, current trajectoryhth\_\{t\}, Reflection MemoryMM, and auxiliary execution signalsξt\\xi\_\{t\}:

\(δt,ρt,ft,Δ​Et\)=Dψ​\(g,ht,ℳ,ξt\),\(\\delta\_\{t\},\\rho\_\{t\},f\_\{t\},\\Delta E\_\{t\}\)=D\_\{\\psi\}\(g,h\_\{t\},\\mathcal\{M\},\\xi\_\{t\}\),\(4\)whereδt∈\{Continue,Rollback\}\\delta\_\{t\}\\in\\\{\\textsc\{Continue\},\\textsc\{Rollback\}\\\}is the recovery verdict,ρt∈\[0,1\]\\rho\_\{t\}\\in\[0,1\]controls the urgency of the next review,ftf\_\{t\}is the failure diagnosis used when recovery is selected, andΔ​Et\\Delta E\_\{t\}contains newly established environment knowledge\. Here,ξt\\xi\_\{t\}summarizes lightweight execution signals such as repeated actions, no\-effect steps, and state cycles\. If the reviewer elects to continue, it usesρt\\rho\_\{t\}to set the next review horizon:

Lt\+1=Lmin\+\(1−ρt\)​\(Lmax−Lmin\),L\_\{t\+1\}=L\_\{\\min\}\+\(1\-\\rho\_\{t\}\)\\,\(L\_\{\\max\}\-L\_\{\\min\}\),\(5\)so higher urgency leads to earlier re\-evaluation\. This adaptive horizon concentrates review effort on uncertain trajectories while allowing stable branches longer uninterrupted exploration\. More details and the instruction prompt of the reviewer are provided in Appendix\.

### 4\.3Where: Coarse\-to\-Fine Restore Localization

Once recovery is approved, RIR must select a restore point from the stored checkpoints\. Identifying the precise erroneous step in a single pass over the full trajectory is unreliable: failures in long\-horizon tasks often arise from several interdependent decisions, while adjacent checkpoints may differ only marginally\. We therefore adopt a two\-stage coarse\-to\-fine localization procedure\.

In thecoarse causal range localizationstage, an LLM\-based selector uses the conditioned failure diagnosisftf\_\{t\}together with the executed trajectory to identify a historical interval likely to contain the decisive error or an unmet precondition:

\[l,u\]=Scoarse​\(g,ht,ℳ,ft\),l,u∈ℐt,l≤u\.\[l,u\]=S\_\{\\mathrm\{coarse\}\}\(g,h\_\{t\},\\mathcal\{M\},f\_\{t\}\),\\quad l,u\\in\\mathcal\{I\}\_\{t\},\\;\\;l\\leq u\.\(6\)Rather than prematurely committing to a single culprit action, this stage eliminates large portions of the trajectory that are unlikely to have contributed to the current failure and restricts the subsequent search to a compact causal range\.

In thefine checkpoint selectionstage, the selector compares compact state summariesσ⁡\(Ci\)\\sigma\(C\_\{i\}\)within the localized interval and chooses a restore point that removes the current obstruction while preserving as much valid progress as possible:

r=Sfine\(g,ft,ℳ,\{\(i,σ\(Ci\)\)∣i∈ℐt,l≤i≤u\}\)\.r=S\_\{\\mathrm\{fine\}\}\\\!\\left\(g,f\_\{t\},\\mathcal\{M\},\\\{\(i,\\sigma\(C\_\{i\}\)\)\\mid i\\in\\mathcal\{I\}\_\{t\},\\;l\\leq i\\leq u\\\}\\right\)\.\(7\)Hereσ⁡\(Ci\)\\sigma\(C\_\{i\}\)denotes a compact description derived from the checkpointed state, such as completed sub\-goals, current location, and key resources; it is computed from the checkpoint and is not maintained as an additional memory variable\. LetEscape⁡\(σ⁡\(Ci\),ft\)\\operatorname\{Escape\}\(\\sigma\(C\_\{i\}\),f\_\{t\}\)indicate whether restoring checkpointCiC\_\{i\}removes the failure condition characterized byftf\_\{t\}\. RIR follows the selection principle:

r⋆=max⁡\{i∈\[l,u\]∩ℐt\|Escape⁡\(σ⁡\(Ci\),ft\)=1\}\.r^\{\\star\}=\\max\\Big\\\{i\\in\[l,u\]\\cap\\mathcal\{I\}\_\{t\}\\;\\Big\|\\;\\operatorname\{Escape\}\(\\sigma\(C\_\{i\}\),f\_\{t\}\)=1\\Big\\\}\.\(8\)In other words, RIR first filters out checkpoints whose restored states still retain the obstruction identified inftf\_\{t\}, and then selects the*latest*checkpoint among the remaining candidates\. The first criterion avoids immediately recreating the same failure condition after restoration, while the second preserves the longest valid prefix of the executed trajectory\.

In practice,SfineS\_\{\\mathrm\{fine\}\}approximates Equation[8](https://arxiv.org/html/2609.18304#S4.E8)by jointly comparing the candidate state summaries against the conditioned failure diagnosis, and returns a restore point together with its justification and confidence score\. The coarse stage identifies*which region of the trajectory is causally relevant to the failure*, whereas the fine stage determines*which restorable state within that region provides the best point of re\-entry*\. This coarse\-to\-fine decomposition avoids reducing restore localization to single\-step error attribution and instead balances two competing objectives: removing the conditions that caused the failure and preserving as much correct progress as possible\. The detailed selector prompts are provided in Appendix\.

### 4\.4What: Rollback\-Consistent Reflection Memory

State restoration determines which trajectory suffix is removed, but not what information from that suffix should survive\. Discarding it entirely loses reusable experience, whereas retaining it verbatim may preserve branch\-local state claims that rollback has already invalidated\. RIR therefore maintains a structured Reflection Memoryℳ\\mathcal\{M\}outside the checkpoint:

ℳ=\(G,E,H,F\)\.\\mathcal\{M\}=\(G,E,H,F\)\.\(9\)Task Objective \(GG\)\.Stores a rollback\-invariant representation of the final objective and provides stable task\-level guidance to the reviewer and the selector\.

Environment Model \(EE\)\.Stores reusable knowledge about the environment across observed branches\. We organize it into three semantic channelsE=Eobs∪Eelim∪Eaff,E=E^\{\\mathrm\{obs\}\}\\cup E^\{\\mathrm\{elim\}\}\\cup E^\{\\mathrm\{aff\}\},corresponding respectively to observed facts, evidence\-supported eliminations, and action or tool affordances\. Entries are admitted only through constrained structured channels, preventing the narrative of an abandoned branch from re\-entering the prompt disguised as current state\. For observations whose truth may depend on an action subsequently undone by rollback, RIR preserves their provenance but marks them for re\-verification rather than treating them as facts about the restored world\.

Attempt History \(HH\)\.Records milestones and routes achieved in previous attempts as historical facts rather than claims about the current state\. A later attempt can therefore reuse a discovered route without assuming that resources, locations, or progress obtained before rollback remain valid after restoration\. The currently executing branch remains represented byhth\_\{t\}; its verified milestones are incorporated intoHHonly when that attempt terminates or is abandoned\.

Failure Analysis \(FF\)\.Summarizes why the most recently abandoned branch failed under the conditions that were in force at the time\. Failure patterns that become supported across attempts are instead promoted to the Environment Model\. Thus,FFexplains*why the latest branch failed under its specific conditions*, whereasEEcaptures*what is reusable about how the environment behaves*\.

Crucially, none of these fields represents the agent’s current branch\-local state; that information is supplied by the active trajectoryhth\_\{t\}and the restored checkpoint\(sr,hr\)\(s\_\{r\},h\_\{r\}\)\. This separation prevents stale state claims from crossing the rollback boundary\.

At each review, the reviewer extracts newly established environment knowledgeΔ​Et\\Delta E\_\{t\}\. If rollback is executed, the reflection patcher additionally examines the abandoned suffixτr:t\\tau\_\{r:t\}to extract historical milestonesΔ​Ht\\Delta H\_\{t\}and construct a new failure analysisFnewF\_\{\\mathrm\{new\}\}\. The resulting update is:

E\+=E∪Δ​Et,H\+=H∪Δ​Ht,F\+=Fnew,ℳ\+=\(G,E\+,H\+,F\+\)\.E^\{\+\}=E\\cup\\Delta E\_\{t\},\\quad H^\{\+\}=H\\cup\\Delta H\_\{t\},\\quad F^\{\+\}=F\_\{\\mathrm\{new\}\},\\quad\\mathcal\{M\}^\{\+\}=\(G,E^\{\+\},H^\{\+\},F^\{\+\}\)\.\(10\)Thus,GGremains fixed,EEandHHaccumulate reusable knowledge and history, whileFFis replaced after each failed attempt\. We denote this field\-specific update byϕ\\phi, which is applied in Equation[2](https://arxiv.org/html/2609.18304#S4.E2)to produce the updated memoryℳ\+\\mathcal\{M\}^\{\+\}\.

The updated memoryℳ\+\\mathcal\{M\}^\{\+\}replaces the previous Reflection Memory block in the system prompt rather than being appended to it\. Becauseℳ\\mathcal\{M\}is maintained outside checkpoints, restoringCrC\_\{r\}reverts the environment and branch\-local context while preserving the updated memory for the next attempt\. This realizes the central asymmetry of RIR:rollback the world, keep the reflection\.

Algorithm 1Rollback\-Induced Reflection \(RIR\)1:Task objective

gg, base agent

πθ\\pi\_\{\\theta\}, rollback budget

BrbB\_\{\\mathrm\{rb\}\}
2:Initialize Reflection Memory

ℳ\\mathcal\{M\}, review horizon

L←L0L\\leftarrow L\_\{0\}, and

Nrb←0N\_\{\\mathrm\{rb\}\}\\leftarrow 0
3:whiletask not terminateddo

4:Store checkpoint

Ct=\(st,ht\)C\_\{t\}=\(s\_\{t\},h\_\{t\}\)
5:Sample

at∼πθ\(⋅∣g,ht,ℳ\)a\_\{t\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid g,h\_\{t\},\\mathcal\{M\}\)
6:if

ata\_\{t\}is a final responsethen

7:return

ata\_\{t\}
8:if

at=rollbacka\_\{t\}=\\texttt\{rollback\}orreview horizon is reachedthen⊳\\trianglerightWhen⋅\\cdotreview the current branch

9:Reviewer evaluation

\(δt,ρt,ft,Δ​Et\)←𝒟ψ​\(g,ht,ℳ,ξt\)\(\\delta\_\{t\},\\rho\_\{t\},f\_\{t\},\\Delta E\_\{t\}\)\\leftarrow\\mathcal\{D\}\_\{\\psi\}\(g,h\_\{t\},\\mathcal\{M\},\\xi\_\{t\}\)
10:Merge

Δ​Et\\Delta E\_\{t\}into the Environment Model

11:if

δt=Rollback\\delta\_\{t\}=\\textsc\{Rollback\}and

Nrb<BrbN\_\{\\mathrm\{rb\}\}<B\_\{\\mathrm\{rb\}\}then

12:

\[l,u\]←𝒮coarse​\(g,ht,ℳ,ft\)\[l,u\]\\leftarrow\\mathcal\{S\}\_\{\\mathrm\{coarse\}\}\(g,h\_\{t\},\\mathcal\{M\},f\_\{t\}\)⊳\\trianglerightWhere⋅\\cdotlocalize a causal restore range

13:Select restore point

r←𝒮fine​\(g,ft,ℳ,\{\(i,σ⁡\(Ci\)\)\|l≤i≤u\}\)r\\leftarrow\\mathcal\{S\}\_\{\\mathrm\{fine\}\}\(g,f\_\{t\},\\mathcal\{M\},\\\{\(i,\\sigma\(C\_\{i\}\)\)\\;\|\\;l\\leq i\\leq u\\\}\)
14:Update

ℳ\+←ϕ\(ℳ,τr:t\)\\mathcal\{M\}^\{\+\}\\leftarrow\\phi\(\\mathcal\{M\},\\tau\_\{r:t\}\)⊳\\trianglerightWhat⋅\\cdotretain reusable knowledge

15:Restore the environment and branch\-local context to

CrC\_\{r\}
16:Reset the review horizon and increment

NrbN\_\{\\mathrm\{rb\}\}
17:continue

18:else

19:Adapt the next review horizon using

ρt\\rho\_\{t\}
20:if

at≠rollbacka\_\{t\}\\neq\\texttt\{rollback\}then

21:Execute

ata\_\{t\}and update the trajectory

### 4\.5Theoretical Analysis

RIR recovery is governed by two quantities: a rollback depthkk, and the updated memoryℳ\+\\mathcal\{M\}^\{\+\}carried across the recovery boundary\.Whendecides whether recovery is invoked,Whereselectskk, andWhatselectsℳ\+\\mathcal\{M\}^\{\+\}\.

Suppose recovery is triggered at steptt, with restore pointr=t−k∈ℐtr=t\-k\\in\\mathcal\{I\}\_\{t\}\. Let𝕄\(ℳ,τr:t\)\\mathbb\{M\}\(\\mathcal\{M\},\\tau\_\{r:t\}\)denote the set of memory configurations reachable by updatingℳ\\mathcal\{M\}with the abandoned suffixτr:t\\tau\_\{r:t\}, ranging fromℳ\\mathcal\{M\}itself to the full updateϕ\(ℳ,τr:t\)\\phi\(\\mathcal\{M\},\\tau\_\{r:t\}\)defined in Equation[10](https://arxiv.org/html/2609.18304#S4.E10)\. The RIR recovery operator is:

ℛk,ℳ\+:\(st,ht,ℳ\)↦\(sr,hr,ℳ\+\),k∈𝒦t,ℳ\+∈𝕄\(ℳ,τr:t\),\\mathcal\{R\}\_\{k,\\mathcal\{M\}^\{\+\}\}:\(s\_\{t\},h\_\{t\},\\mathcal\{M\}\)\\mapsto\(s\_\{r\},h\_\{r\},\\mathcal\{M\}^\{\+\}\),\\quad k\\in\\mathcal\{K\}\_\{t\},\\;\\mathcal\{M\}^\{\+\}\\in\\mathbb\{M\}\(\\mathcal\{M\},\\tau\_\{r:t\}\),\(11\)recovering Equation[2](https://arxiv.org/html/2609.18304#S4.E2)oncerrandℳ\+\\mathcal\{M\}^\{\+\}are instantiated by the selector andϕ\\phi\. At every recovery event, RIR may choose\(k,ℳ\+\)\(k,\\mathcal\{M\}^\{\+\}\)freely from the full space𝒦t×𝕄\(ℳ,τr:t\)\\mathcal\{K\}\_\{t\}\\times\\mathbb\{M\}\(\\mathcal\{M\},\\tau\_\{r:t\}\)\.

Proposition 1 \(Recovery\-Space Generality\)\.Consider a self\-correction or rollback mechanismjjwhose retained memory is also drawn from𝕄\(ℳ,τr:t\)\\mathbb\{M\}\(\\mathcal\{M\},\\tau\_\{r:t\}\)\. This putsjjand RIR on the same space for comparison\. Mechanismjjdiffers from RIR in one respect: at every recovery event, it restricts its choice of\(k,ℳ\+\)\(k,\\mathcal\{M\}^\{\+\}\)to a fixed proper subsetSj⊊𝒦t×𝕄\(ℳ,τr:t\)S\_\{j\}\\subsetneq\\mathcal\{K\}\_\{t\}\\times\\mathbb\{M\}\(\\mathcal\{M\},\\tau\_\{r:t\}\)\. Context\-only correction, intermediate rollback, and restart\-based recovery are common instances\. Each fixeskk, fixesℳ\+\\mathcal\{M\}^\{\+\}, or both\. BecauseSjS\_\{j\}is a proper subset of the space RIR can access, any policy realizable by mechanismjjis also realizable by RIR\.

LetΠRIR\\Pi\_\{\\mathrm\{RIR\}\}denote the policy class induced by free access to𝒦t×𝕄\(ℳ,τr:t\)\\mathcal\{K\}\_\{t\}\\times\\mathbb\{M\}\(\\mathcal\{M\},\\tau\_\{r:t\}\)at every recovery event, andΠj\\Pi\_\{j\}the policy class induced by mechanismjjconfined toSjS\_\{j\}, under matched base agent and interaction budget\. Proposition 1 givesΠj⊆ΠRIR\\Pi\_\{j\}\\subseteq\\Pi\_\{\\mathrm\{RIR\}\}\. For verifiable tasks, letRg∈\{0,1\}R\_\{g\}\\in\\\{0,1\\\}indicate task success,J⁡\(π\)=Prπ⁡\(Rg=1\)J\(\\pi\)=\\Pr\_\{\\pi\}\(R\_\{g\}=1\)the task\-completion probability of recovery control policyπ\\pi, andV⋆​\(Π\)=supπ∈ΠJ⁡\(π\)V^\{\\star\}\(\\Pi\)=\\sup\_\{\\pi\\in\\Pi\}J\(\\pi\)the best value attainable withinΠ\\Pi\.

Corollary 1 \(Optimal\-Value Monotonicity\)\.For every restricted mechanismjj,

V⋆​\(ΠRIR\)≥V⋆​\(Πj\)\.V^\{\\star\}\(\\Pi\_\{\\mathrm\{RIR\}\}\)\\geq V^\{\\star\}\(\\Pi\_\{j\}\)\.\(12\)SinceΠj⊆ΠRIR\\Pi\_\{j\}\\subseteq\\Pi\_\{\\mathrm\{RIR\}\}, a restricted mechanism can not attain a higher optimum than RIR’s recovery space allows, under matched agent and budget, and the same holds for expected return in place of success probability\. Full statements and proofs are provided in Appendix\.

## 5Experiments

### 5\.1Experimental Setup

Datasets & metrics\.We evaluate RIR on three long\-horizon agent benchmarks: ALFWorld\([Shridhar et al\., 2020](https://arxiv.org/html/2609.18304#bib.bib27)\), ScienceWorld\([Wang et al\., 2022](https://arxiv.org/html/2609.18304#bib.bib28)\), and GAIA\([Mialon et al\., 2024](https://arxiv.org/html/2609.18304#bib.bib29)\), covering embodied interaction, scientific experimentation, and open\-domain tool use, respectively\. We report success rate \(SR\) on ALFWorld and ScienceWorld, and use an LLM\-as\-a\-Judge evaluator on GAIA to assess semantic equivalence between predicted and reference answers\. On ScienceWorld, we additionally report normalized dense reward \(DR\) to measure partial task progress\. Further dataset and metric details are provided in Appendix\.

Baselines\.We compare against representative training\-free test\-time methods under the same LLM backbones and interaction budgets: ReAct\([Yao et al\., 2022](https://arxiv.org/html/2609.18304#bib.bib5)\)without explicit recovery; Self\-Refine\([Madaan et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib6)\)and Reflexion\([Shinn et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib7)\)for information\-level correction; LATS\([Zhou et al\., 2023](https://arxiv.org/html/2609.18304#bib.bib14)\)for search\-based recovery; and GA\-Rollback\([Li et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib15)\)for explicit state rollback\. We use Qwen3\-14B\([Yang et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib1)\)and DeepSeek\-V3\([Liu et al\., 2024](https://arxiv.org/html/2609.18304#bib.bib3)\)as backbone models and follow official implementations and recommended configurations whenever available\. Further baseline details are provided in Appendix\.

Implementation details\.We implement the base agent with AgentScope\([Gao et al\., 2025](https://arxiv.org/html/2609.18304#bib.bib30)\)\. Within each backbone setting, the acting agent and all RIR components use the same LLM, so recovery gains cannot be attributed to a stronger auxiliary model\. All methods share identical task interfaces and interaction budgets\. For GAIA, we additionally restore the workspace state during rollback, including files created or modified by the agent, to keep the external environment consistent with the recovered trajectory\. Separately, final\-answer evaluation on GAIA uses an independent Qwen3\.8\-Max judge that does not participate in trajectory generation or recovery\. Additional implementation details are provided in Appendix\.

### 5\.2Main Results

Overall performance\.As shown in Table[1](https://arxiv.org/html/2609.18304#S5.T1), RIR achieves the highest SR across all three benchmarks and both backbones\. With Qwen3\-14B, RIR reaches 47\.00% average SR, outperforming the strongest baseline by 3\.09 percentage points; with DeepSeek\-V3, the margin increases to 6\.57 points, yielding 69\.43% average SR\. RIR also obtains the highest ScienceWorld DR under both backbones\. Notably, the advantage persists as the backbone becomes stronger, suggesting that explicit recovery remains useful rather than being subsumed by improved base\-model capability\. This empirical pattern is consistent with our recovery\-space analysis: the broader recovery space provides additional useful choices in practice, although the theoretical result itself is a capacity statement rather than a performance guarantee\.

Table 1:Main results across three benchmarks\. SR denotes success rate and DR denotes normalized dense reward; Avg\. SR is the unweighted mean across the three benchmarks\.Boldandunderlineddenote the best and second\-best results within each backbone, respectively\.Figure 2:Efficiency trade\-off on ScienceWorld across both backbones\.Figure 3:Sensitivity to the adaptive review range\(Lmin,Lmax\)\(L\_\{\\min\},L\_\{\\max\}\)on ScienceWorld with DeepSeek\-V3\. Color denotes SR and orange contours denote average rollback frequency\.Table 2:Average executed rollbacks per task on ScienceWorld\. Methods without state\-level rollback are omitted\.Efficiency\.Figure[2](https://arxiv.org/html/2609.18304#S5.T2)compares test\-time overhead on ScienceWorld across both backbones\. RIR remains in the low\-overhead regime while requiring substantially fewer auxiliary LLM calls than the more expensive recovery baselines, and this pattern is consistent across Qwen3\-14B and DeepSeek\-V3\. Recovery is also selective rather than frequent: Table[2](https://arxiv.org/html/2609.18304#S5.T2)shows fewer than one executed rollback per task for RIR under both backbones, compared with approximately 2\.9 for GA\-Rollback\. Thus, the performance gains are associated with targeted correction rather than repeated rollback and re\-exploration\. Complete efficiency results are provided in Appendix\.

To complement the aggregate results, we further inspect representative successful and unsuccessful recovery trajectories in Appendix\. The cases illustrate both when rollback provides state\-level benefits that reflection alone cannot recover and when incorrect diagnosis can cause recovery to revisit an ineffective branch\.

### 5\.3Ablation Studies

When to review\.We compare agent\-only, scheduled\-only, and hybrid review under fixed or adaptive scheduling\. As shown in Table[3](https://arxiv.org/html/2609.18304#S5.T3), the full hybrid\-adaptive RIR achieves the highest task performance \(76\.75% SR, 0\.813 DR\)\. Importantly, Hybrid \(fixed\) performs worse despite triggering more reviews and rollbacks and using more auxiliary LLM calls\. Adaptive scheduling therefore appears to improve*when*the two review signals intervene, rather than simply increasing intervention frequency\. This supports the design of review timing as an adaptive control decision rather than a fixed periodic mechanism\.

Figure 4:Where\-to\-recover ablation on ScienceWorld with DeepSeek\-V3\.*No rollback*retains reflection but disables state restoration;*Restart*restores to the initial checkpoint;*Direct selection*chooses a checkpoint in one stage; and*RIR*uses coarse\-to\-fine restore localization\.Where to recover\.We next vary restore\-point localization while keeping the review policy and Reflection Memory unchanged\. We compare*No rollback*\(reflection only\),*Restart*,*Direct selection*, and the full coarse\-to\-fine RIR selector\. Figure[4](https://arxiv.org/html/2609.18304#S5.F4)shows that No rollback produces the shortest trajectories but substantially lower task performance, indicating that information\-level correction alone cannot replace state recovery\. Among rollback\-based variants, RIR achieves the highest SR \(76\.75%\) and DR \(0\.813\) while requiring the fewest steps \(42\.4\) and rollbacks \(0\.92\)\.

Table 3:Ablations on ScienceWorld with DeepSeek\-V3 as the backbone\.Avg\. Steps,Rev\.,RB, andAux\.report the per\-task averages of interaction steps, reviewer invocations, executed rollbacks, and auxiliary LLM calls, respectively\.Rollback vs\. Reflection\.We further isolate the two core recovery components by comparing*Reflection only*,*Rollback only*, and the full RIR\. Table[3](https://arxiv.org/html/2609.18304#S5.T3)shows that neither component alone matches the complete method: RIR improves SR by 6\.27 points over Reflection only and by 7\.38 points over Rollback only\. Reflection only is cheaper but cannot repair execution\-state errors, whereas Rollback only incurs greater interaction and inference cost without preserving reusable knowledge across attempts\. In terms of our recovery operator, these variants separately restrict either state restoration or the memory update, while RIR couples both dimensions\. The resulting gap therefore empirically supports their complementarity\.

Dynamic range setting\.We finally vary the adaptive review range\(Lmin,Lmax\)\(L\_\{\\min\},L\_\{\\max\}\)\. Figure[2](https://arxiv.org/html/2609.18304#S5.T2)shows that increasingLmaxL\_\{\\max\}generally improves performance, whereas an overly largeLminL\_\{\\min\}is detrimental\. The best setting,\(2,8\)\(2,8\), reaches 77\.49% SR with only 0\.77 rollbacks per task; the default\(4,8\)\(4,8\)remains close at 76\.75%, indicating that performance is not overly sensitive to a single optimum\. Overall, a wider range gives the reviewer more flexibility to intervene quickly on risky branches while allowing stable branches to proceed with fewer interruptions\.

## 6Conclusion

In this work, we propose Rollback\-Induced Reflection \(RIR\), a unified recovery framework for long\-horizon LLM agents\. RIR couples adaptive review, restore localization, and persistent reflection to address a key limitation of existing recovery methods: correcting execution state without losing useful experience from failed trajectories\. We further characterize recovery through a unified operator over rollback depth and updated reflection memory, showing that several common correction and rollback mechanisms arise as restricted cases of the RIR recovery space and establishing the corresponding optimal\-value monotonicity result\. Empirically, RIR improves task completion without relying on frequent intervention, while the ablation results show that review timing, restore localization, and reflection play complementary roles in successful recovery\. More broadly, our results suggest that reliable long\-horizon agents require recovery mechanisms that do more than undo mistakes: they must transform failed interaction into a better basis for subsequent decision\-making\.

## References

- Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: building production\-ready ai agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p1.1)\.
- Fuet al\.\(2024\)Y\. Fu, D\. Kim, J\. Kim, S\. Sohn, L\. Logeswaran, K\. Bae, and H\. LeeAutoguide: automated generation and selection of context\-aware guidelines for large language model agents\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 119919–119948\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p2.1)\.
- Gaoet al\.\(2025\)D\. Gao, Z\. Li, Y\. Xie, W\. Kuang, L\. Yao, B\. Qian, Z\. Ma, Y\. Cui, H\. Luo, S\. Li,et al\.AgentScope 1\.0: a developer\-centric framework for building agentic applications\.arXiv preprint arXiv:2508\.16279\.Cited by:[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p3.1)\.
- Haoet al\.\(2026\)L\. Hao, H\. Lin, H\. Li, Z\. Yang, H\. Hao, D\. Huang, H\. Yang, H\. Ge, Y\. Wu, Z\. H\. Yin,et al\.Speculative rollback correction for quality\-diverse web agent imitation\.arXiv preprint arXiv:2606\.12485\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p1.1),[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§2](https://arxiv.org/html/2609.18304#S2.p3.1)\.
- Huet al\.\(2025\)M\. Hu, T\. Fang, J\. Zhang, J\. Ma, Z\. Zhang, J\. Zhou, H\. Zhang, H\. Mi, D\. Yu, and I\. KingWebCoT: enhancing web agent reasoning by reconstructing chain\-of\-thought in reflection, branching, and rollback\.\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 5155–5173\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p1.1)\.
- Huanget al\.\(2024\)J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. Yu, X\. Song, and D\. ZhouLarge language models cannot self\-correct reasoning yet\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 32808–32824\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p2.1)\.
- Jiaet al\.\(2026\)R\. Jia, M\. Wu, Y\. Ding, J\. Lu, and Y\. ZhangAgent\-enhanced heterogeneous graph rag for academic question answering\.InProceedings of the ACM Web Conference,pp\. 8765–8768\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p1.1)\.
- Kamoiet al\.\(2024\)R\. Kamoi, Y\. Zhang, N\. Zhang, J\. Han, and R\. ZhangWhen can llms actually correct their own mistakes? a critical survey of self\-correction of llms\.Transactions of the Association for Computational Linguistics12,pp\. 1417–1440\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p2.1)\.
- Kimet al\.\(2025\)J\. Kim, S\. Rhee, M\. Kim, D\. Kim, S\. Lee, Y\. Sung, and K\. JungReflact: world\-grounded decision making in llm agents via goal\-state reflection\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 33421–33453\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§2](https://arxiv.org/html/2609.18304#S2.p2.1)\.
- Liet al\.\(2025\)X\. Li, K\. Chen, Y\. Long, X\. Bai, Y\. Xu, and M\. ZhangGenerator\-assistant stepwise rollback framework for large language model agent\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 17694–17711\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§2](https://arxiv.org/html/2609.18304#S2.p3.1),[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p2.1)\.
- Liuet al\.\(2024\)A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p2.1)\.
- Luet al\.\(2026\)M\. Lu, W\. Sun, W\. Du, Z\. Ling, X\. Yao, K\. Liu, and J\. ChenBeyond the context window: scaling agentic rl via end\-to\-end optimized context compression\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics,pp\. 21074–21125\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p1.1)\.
- Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 46534–46594\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§2](https://arxiv.org/html/2609.18304#S2.p2.1),[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p2.1)\.
- Mialonet al\.\(2024\)G\. Mialon, C\. Fourrier, T\. Wolf, Y\. LeCun, and T\. ScialomGaia: a benchmark for general ai assistants\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 9025–9049\.Cited by:[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p1.1)\.
- Parket al\.\(2023\)J\. S\. Park, J\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. BernsteinGenerative agents: interactive simulacra of human behavior\.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology,pp\. 1–22\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 8634–8652\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§2](https://arxiv.org/html/2609.18304#S2.p2.1),[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p2.1)\.
- Shridharet al\.\(2020\)M\. Shridhar, X\. Yuan, M\. Côté, Y\. Bisk, A\. Trischler, and M\. HausknechtAlfworld: aligning text and embodied environments for interactive learning\.arXiv preprint arXiv:2010\.03768\.Cited by:[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p1.1)\.
- Wanget al\.\(2022\)R\. Wang, P\. Jansen, M\. Côté, and P\. AmmanabroluScienceworld: is your agent smarter than a 5th grader?\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 11279–11298\.Cited by:[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p1.1)\.
- Wuet al\.\(2025\)Q\. Wu, P\. Gao, W\. Liu, and J\. LuanBacktrackagent: enhancing gui agent with error detection and backtracking mechanism\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 4250–4272\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p2.1)\.
- Yanget al\.\(2026\)K\. Yang, P\. Li, Z\. Wu, K\. Xu, H\. Huang, and X\. HuangDART: semantic recoverability for structured tool agents\.arXiv preprint arXiv:2605\.23311\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p3.1)\.
- Yaoet al\.\(2022\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReact: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p2.1)\.
- Yuet al\.\(2026\)Y\. Yu, L\. Yao, Y\. Xie, Q\. Tan, J\. Feng, Y\. Li, and L\. WuAgentic memory: learning unified long\-term and short\-term memory management for large language model agents\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics,pp\. 21457–21483\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p1.1)\.
- Zhanget al\.\(2026a\)G\. Zhang, M\. Fu, K\. Wang, F\. Wan, M\. Yu, and S\. YanG\-memory: tracing hierarchical memory for multi\-agent systems\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 12988–13018\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p2.1)\.
- Zhanget al\.\(2026b\)G\. Zhang, J\. Wang, J\. Chen, W\. Zhou, K\. Wang, and S\. YanAgentracer: who is inducing failure in the llm agentic systems?\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 11377–11399\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p2.1)\.
- Zhanget al\.\(2026c\)Z\. Zhang, T\. Fang, K\. Ma, W\. Yu, H\. Zhang, H\. Mi, and D\. YuWebRollback: enhancing web agents with explicit rollback mechanisms\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics,pp\. 187–197\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§2](https://arxiv.org/html/2609.18304#S2.p1.1),[§2](https://arxiv.org/html/2609.18304#S2.p3.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpel: llm agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§2](https://arxiv.org/html/2609.18304#S2.p2.1)\.
- Zhouet al\.\(2023\)A\. Zhou, K\. Yan, M\. Shlapentokh\-Rothman, H\. Wang, and Y\. WangLanguage agent tree search unifies reasoning acting and planning in language models\.arXiv preprint arXiv:2310\.04406\.Cited by:[§1](https://arxiv.org/html/2609.18304#S1.p2.1),[§5\.1](https://arxiv.org/html/2609.18304#S5.SS1.p2.1)\.
- Zhuanget al\.\(2026\)Y\. Zhuang, K\. Chen, Y\. Duan, S\. Zheng, J\. Li, and X\. ZhangAgentRewind: recoverable execution for long\-horizon LLM agents\.arXiv preprint arXiv:2608\.14380\.Cited by:[§2](https://arxiv.org/html/2609.18304#S2.p3.1)\.