精简摘要，而非历史：面向长程智能体的源文评判式上下文压缩

# Prune the Summary, Not the History: Source-Judged Compaction for Long-Horizon Agents
Source: [https://arxiv.org/html/2610.20927](https://arxiv.org/html/2610.20927)
Guanghui MinAffiliation:Department of Computer ScienceAffiliation:University of VirginiaEmail:[jjm8vr@virginia\.edu](mailto:)Chen Chen††thanks:Corresponding author\.Affiliation:Department of Computer ScienceAffiliation:University of VirginiaEmail:[zrh6du@virginia\.edu](mailto:)

###### Abstract

Long\-horizon LLM agents rely on context compaction: when the interaction history outgrows the context window, a compressor replaces it with a natural\-language summary\. We show that these summaries contain more than a record of what happened\. Compaction prompts ask for decisions and next steps, so each summary mixes facts established by the environment with reasoning written by the compressor, and the agent cannot tell the two apart afterwards\. In an analysis of a widely used structured compressor,83%83\\%of failed runs trace back to an action that carries out a specific summary entry, most of which no environment output supports\. We propose Source\-Judged Compaction \(SJC\), a training\-free post\-processing step that scores each summary entry for whether it has a source in the environment outputs or the previous summary, and removes the entries that do not\. It changes neither the agent nor the compressor and adds one inexpensive judge request per compaction\. The judge is Jev, a non\-generative decision model that answers the source question with a calibrated probability instead of text\. On AppWorld and OfficeBench with two agent models,SJCimproves cross\-run reliability over the underlying compressor by2\.1%2\.1\\%–8\.9%8\.9\\%without increasing the agent’s token usage\. Our code is available at[https://github\.com/GuanghuiMin/sjc](https://github.com/GuanghuiMin/sjc)\.

## 1Introduction

Long\-horizon agentic tasks require LLMs to plan, use tools, and adapt to environmental feedback across extended interactions\([Xu et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib25);[Wang et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib17)\)\. During execution, tool outputs, failed attempts, intermediate results, and reasoning traces continually accumulate, making the context increasingly expensive to process and difficult to navigate\. This problem persists even with large nominal context windows, whose effective use of distant information degrades with context length\([Liu et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib26);[Hsieh et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib27)\)\. Context compaction is therefore essential for scalable long\-horizon agents\.

General compaction methods select or rewrite tokens from static prompts and documents\([Jiang et al\., 2023](https://arxiv.org/html/2610.20927#bib.bib2);[Pan et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib4)\), but do not directly account for subsequent agent execution\. Agent\-specific methods such as ReSum, SUPO, and ACM instead modify or post\-train the agent to manage compressed contexts\([Wu et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib10);[Lu et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib11);[Li et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib24)\)\. In many deployed systems, however, the agent is fixed or accessible only through an API\. OpenClaw, DeepSeek Harness, and Codex therefore invoke a separate compressor that replaces older interactions with a natural\-language checkpoint\([OpenClaw Contributors, 2026a](https://arxiv.org/html/2610.20927#bib.bib13);[DeepSeek\-AI, 2026](https://arxiv.org/html/2610.20927#bib.bib20);[OpenAI, 2026a](https://arxiv.org/html/2610.20927#bib.bib21)\), while ACON and PAIR improve the compression prompt without modifying the downstream agent\([Kang et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib12);[Min et al\., 2026b](https://arxiv.org/html/2610.20927#bib.bib23)\)\.

Such checkpoints do more than shorten history: they periodically reconstruct the state from which the agent continues, including its progress, constraints, and future actions\. Errors in this reconstruction can therefore alter subsequent execution\([Min et al\., 2026a](https://arxiv.org/html/2610.20927#bib.bib28);[Min et al\., 2026b](https://arxiv.org/html/2610.20927#bib.bib23)\)\. For example, when a user asks to reply to a friend with movie titles “as per their request”, the compressor records the next step as sending the comma\-separated movie list; the agent then sends every title it found, including those the friend did not ask for\. The compressor wrote this plan itself, but after compaction the agent cannot tell it from a fact the environment established\. Such cases are common: in our analysis of a widely used structured compressor,67\.7%67\.7\\%of the pending work it records has no source in the environment outputs, and2929of3535failed runs trace back to an action that carries out a specific summary entry\. This suggests filtering summary entries by their provenance\. Whether an entry is supported by an environment output or by the previous summary can be checked against the history at compaction time, without knowing how the rest of the task will unfold\.

We introduceSJC\(Source\-JudgedCompaction\), a lightweight plug\-in that prunes unsupported entries from generated summaries\. Given a structured natural\-language summary,SJCsplits it into entries, retrieves relevant environment outputs, and asks a probabilistic judge whether each entry contains information that came from those outputs or from the previous summary, as opposed to only plans, decisions, or interpretations of the task\. Entries without such a source are removed as complete units, while retained entries remain identical to the compressor’s output\. The pruned summary is then shown to the agent and used as the previous summary at the next compaction\.SJCrequires no modification to the agent, the compressor prompt, or the runtime, and adds no summary\-generation call\. Our contributions are threefold:

- •Analysis of Compaction Summaries:We survey 29 compaction prompts from widely used agent systems and find that most ask the compressor for decisions and next steps\. Analyzing the resulting summaries, we show that much of this reasoning has no source in the environment outputs, that it is regenerated at every compression boundary, and that most failed runs follow a specific summary entry\.
- •Source\-Judged Compaction:We develop a training\-free post\-processing step that decomposes a structured summary into entries, retrieves relevant environment evidence, and removes the entries that a source judge finds unsupported\. The method preserves the natural\-language summary interface and attaches to existing agent systems without modifying them\.
- •Empirical Effectiveness:Across AppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib16)\)and OfficeBench\([Wang et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib17)\), using both GPT\-5\.6 Luna\([OpenAI, 2026b](https://arxiv.org/html/2610.20927#bib.bib19)\)and MiniMax\-M3\([MiniMax, 2026](https://arxiv.org/html/2610.20927#bib.bib18)\),SJCimproves cross\-run reliability over the underlying prompting\-based compressor in all four settings, with the largest gains on hard and multi\-application tasks, without increasing the agent’s token usage\.

## 2Preliminary

#### Notations\.

We consider an LLM agentπ\\pithat solves a task by interacting with an environment over multiple steps\. The agent is given a fixed promptPP, which contains the system prompt and the task instruction and is never compressed\. At stepii, the agent takes an actionaia\_\{i\}and receives an environment outputoio\_\{i\}; onlyoio\_\{i\}is produced by the environment, whereasaia\_\{i\}and any accompanying reasoning are produced by the agent\. A rollout isρ=\(a1,o1,…,aT,oT\)\\rho=\(a\_\{1\},o\_\{1\},\\ldots,a\_\{T\},o\_\{T\}\), andS⁡\(ρ\)∈\{0,1\}S\(\\rho\)\\in\\\{0,1\\\}indicates whether it solves the task\. We write\|x\|\|x\|for the token length ofxxand⊕\\oplusfor concatenation\.

Because the interaction history grows with every step, agent systems compact it whenever the context exceeds a budgetBB\. We index these compression boundaries byt=1,2,…t=1,2,\\ldots\. At boundarytt, the context consists of the fixed promptPP, the previous summaryMt−1M\_\{t\-1\}\(withM0=∅M\_\{0\}=\\varnothing\), the history spanHtH\_\{t\}accumulated since the last boundary, and the most recent interactionRtR\_\{t\}, which is retained verbatim\. A compressor𝒞\\mathcal\{C\}replacesMt−1M\_\{t\-1\}andHtH\_\{t\}with a new summaryMtM\_\{t\}, and the agent continues fromP⊕Mt⊕RtP\\oplus M\_\{t\}\\oplus R\_\{t\}\. We focus on structured natural\-language summaries, in whichMtM\_\{t\}is a list of short entries grouped under section headings\.

MethodSummaryTrain\-FreeDrop\-inACE\([Zhang et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib29)\)✗✓✗Mem1\([Zhou et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib30)\)✗✗✗ACM\([Li et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib24)\)✗✗✗Fast\-Jev\([Tran, 2026](https://arxiv.org/html/2610.20927#bib.bib22)\)✗✓✓ReSum1\([Wu et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib10)\)✓✗✗SUPO\([Lu et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib11)\)✓✗✗ACON\([Kang et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib12)\)✓✓✗SJC \(Ours\)✓✓✓Table 1:Comparison of context\-management approaches\.Summarydenotes a natural\-language history checkpoint;Train\-Freerequires no context\-management policy training; andDrop\-inrequires neither agent modification nor task\-specific adaptation\.11footnotetext:We consider the GRPO\-trained ReSum variant\.
#### Problem Definition\.

Compaction is recurrent: after boundarytt, the summaryMtM\_\{t\}is the only record ofHtH\_\{t\}available to the agent, and it is also the input from whichMt\+1M\_\{t\+1\}is produced\. Any content written into a summary is therefore treated as established by all later steps, whether or not the environment ever produced it\. We study how to improve an existing compressor without changing it\. LetM~t=𝒞⁡\(P,Mt−1,Ht,Rt\)\\widetilde\{M\}\_\{t\}=\\mathcal\{C\}\(P,M\_\{t\-1\},H\_\{t\},R\_\{t\}\)be the candidate summary of a host compressor andℬ⁡\(⋅\)\\mathcal\{B\}\(\\cdot\)the set of entries in a summary\. We seek a post\-processing operatorGGthat commitsMt=G⁡\(M~t,P,Mt−1,Ht,Rt\)M\_\{t\}=G\(\\widetilde\{M\}\_\{t\}\\,;P,M\_\{t\-1\},H\_\{t\},R\_\{t\}\)in place ofM~t\\widetilde\{M\}\_\{t\}so as to maximize task success,

maxG\\displaystyle\\max\_\{G\}𝔼ρ∼pπ,𝒞,G​\[S​\(ρ\)\]\\displaystyle\\mathbb\{E\}\_\{\\rho\\sim p\_\{\\pi,\\mathcal\{C\},G\}\}\\left\[S\(\\rho\)\\right\]\(1\)s\.t\.\\displaystyle\\text\{s\.t\.\}ℬ⁡\(Mt\)⊆ℬ⁡\(M~t\)for all​t,\\displaystyle\\mathcal\{B\}\(M\_\{t\}\)\\subseteq\\mathcal\{B\}\(\\widetilde\{M\}\_\{t\}\)\\quad\\text\{for all \}t,wherepπ,𝒞,Gp\_\{\\pi,\\mathcal\{C\},G\}is the rollout distribution induced by the agent, the environment, the host compressor, andGG\. The constraint restrictsGGto deleting entries: it may not rewrite or add content, so\|Mt\|≤\|M~t\|\|M\_\{t\}\|\\leq\|\\widetilde\{M\}\_\{t\}\|and every retained entry is exactly what the host compressor wrote\. We further requireGGto leaveπ\\piand𝒞\\mathcal\{C\}unchanged and to use no training or task\-specific adaptation, so that it can be attached to any compressor that produces structured summaries\.

## 3Why Summary Needs Source Judgment

We analyze the summaries written by a widely used structured compressor, OpenClaw\([OpenClaw Contributors, 2026a](https://arxiv.org/html/2610.20927#bib.bib13)\), on the training split of AppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib16)\)\(90 tasks, three runs, GPT\-5\.6 Luna\)\. We call a summary entry*supported*if its content is established by an environment output or by the previous summary, and*unsupported*otherwise\. To label entries at scale, we ask Jev\([TypeSafe AI, 2026](https://arxiv.org/html/2610.20927#bib.bib32)\)whether each entry contains information from either source and treat entries scoring below0\.50\.5as unsupported \(Appendix[C](https://arxiv.org/html/2610.20927#A3)\)\. Against annotated labels, Jev separates the two with an AUROC of0\.9050\.905; it is conservative at this threshold, so the unsupported shares we report are upper bounds \(Appendix[G](https://arxiv.org/html/2610.20927#A7)\)\.

### 3\.1Summary as Periodic Reasoning

A compaction prompt asks for more than a record of what happened\. We collected 29 compaction prompts from the official repositories of widely used agent systems \(Appendix[B](https://arxiv.org/html/2610.20927#A2)\)\. Of these, 21 ask the compressor for decisions or next steps: 18 request the decisions made and 17 request the remaining work, in instructions such as “What remains to be done \(clear next steps\)” in Codex CLI and “The current plan and the IMMEDIATE next step” in Gemini CLI\. OpenClaw requires sections forDecisions,Open TODOs, andPending user asks\. Each compression boundary is therefore also a reasoning step: the compressor re\-derives where the task stands and where it should go next\. If environment outputs describe the state of the task, this reasoning describes its direction, which the environment never reported\.

Much of this reasoning is unsupported\.Table[2](https://arxiv.org/html/2610.20927#S3.T2)breaks down the26,88126\{,\}881entries of667667summaries by section\. Sections that record observations are almost entirely supported, whereas67\.7%67\.7\\%of open TODOs and86\.1%86\.1\\%of pending user asks have no source\. Among the entries underDecisionsandOpen TODOs, which we call reasoning entries,38\.7%38\.7\\%are unsupported, and54\.9%54\.9\\%at the first compression boundary\.

Unsupported reasoning is regenerated at every boundary\.These entries are not simply carried from one summary to the next: only20\.5%20\.5\\%of unsupported reasoning entries reappear in the following summary, and5\.3%5\.3\\%three summaries later\. Yet every summary contains about four of them, whether it is the first \(4\.34\.3\), the second \(3\.93\.9\), or a later one \(3\.83\.8\)\. The compressor writes unsupported reasoning anew at each boundary, so removing it once is not enough\.

Table 2:Entries of OpenClaw summaries on the AppWorld training split, by section\. An entry is unsupported when Jev finds no environment output or previous summary that establishes it\.![Refer to caption](https://arxiv.org/html/2610.20927v1/sjc_pipeline.png)Figure 1:Overview ofSJC\.\(1\) A host compressor produces a structured candidate summary\. \(2\)SJCsplits it into entries, retrieves relevant environment evidence from the history span, and scores all entries with a source judge in a single request\. \(3\) Supported entries are retained unchanged, while unsupported entries are removed before the committed summary is shown to the agent and passed to the next compression boundary\.
### 3\.2Unsupported Reasoning Can Derail Agent Execution

After compaction, the agent cannot tell which entries the environment established and which the compressor inferred; it treats both as facts\. Developers have met this problem in deployment: Hermes prepends to every summary a notice telling the agent to “treat it as background reference, NOT as active instructions”\([Nous Research, 2026](https://arxiv.org/html/2610.20927#bib.bib15)\)\. To see what the problem costs, we trace every failed run in our analysis back from the failed evaluation check to the action that caused it \(Appendix[D](https://arxiv.org/html/2610.20927#A4)\)\. Of the4444failed runs, nine belong to a scenario whose reference solution is inconsistent with its environment data and that no method solves; we analyze the remaining3535\.

Most failures follow a summary entry\.In2929of the3535failed runs, the failing action carries out a specific summary entry; two independent rounds of annotation agree on every run\. Read against the environment outputs that preceded them,2222of these entries are supported by no output and33contradict one; only44are supported\. The failed runs span1010task scenarios; counted once per scenario and failure pattern,1515of1919cases follow an entry\.

The failures fall into a few recurring patterns\.Rewritten constraints\(1515runs\) restate the task with a different meaning\. In one task, the user asks to reply to a friend with movie titles “as per their request”; the summary records the TODO “Send Leslie the comma\-separated movie list by text message”, and the agent sends every title in the note, including those the friend did not ask for\.Invented deliverables\(55\) tell the agent to report a result, which it then submits as an answer the task does not expect\.Contradicted state\(33\) asserts something an environment output refutes\. The remaining66carry forward an earlier choice of the agent or present a partial result as complete\.

Both observations point to the same remedy: check each summary entry against its source, and do so at every compression boundary\.

## 4SJC: Our Proposed Method

Figure[1](https://arxiv.org/html/2610.20927#S3.F1)illustratesSJC\(Source\-Judged Compaction\), a training\-free post\-processing method for structured natural\-language compaction\. A host compressor first produces a summary in its original format\.SJCthen judges the source of each summary entry and removes entries without an identifiable source\. It neither rewrites retained entries nor changes the agent or the host compressor\.

Setup\.At compression boundarytt, letPPdenote the fixed prompt retained verbatim,HtH\_\{t\}the history span being compressed,RtR\_\{t\}the most recent interaction retained verbatim, andMt−1M\_\{t\-1\}the previous committed summary\.SJCproceeds in three stages and outputs a committed summaryMtM\_\{t\}, which serves as the agent’s checkpoint: it is shown to the agent and passed to the compressor at the next boundary\.

Stage 1: Generate\.The host compressor𝒞\\mathcal\{C\}summarizes the history span as a structured natural\-language candidate summary:

M~t=𝒞⁡\(P,Mt−1,Ht,Rt\)\.\\widetilde\{M\}\_\{t\}=\\mathcal\{C\}\(P,M\_\{t\-1\},H\_\{t\},R\_\{t\}\)\.\(2\)Unlike methods that directly select raw history chunks,SJCretains this summary interface because it provides the agent with a compact and readable description of the current task state\.

Stage 2: Judge sources\.SJCparses the candidate summary into entriesℬt=\{bt,1,…,bt,Nt\}\\mathcal\{B\}\_\{t\}=\\\{b\_\{t,1\},\\ldots,b\_\{t,N\_\{t\}\}\\\}while preserving their section headings\. For each entry, it retrieves a small set of relevant passages from the environment outputs inHtH\_\{t\}and pools them into a shared evidence set:

ℰt=⋃k=1NtRetrieve⁡\(bt,k,Ht\)\.\\mathcal\{E\}\_\{t\}=\\bigcup\_\{k=1\}^\{N\_\{t\}\}\\operatorname\{Retrieve\}\(b\_\{t,k\},H\_\{t\}\)\.\(3\)The pooled evidence is much shorter thanHtH\_\{t\}, so the judge never needs to read the full history\. Retrieval affects only the evidence shown to the judge; it does not directly determine which entries are kept\. A source judgeJJthen receives all entries together with the pooled evidence in a single request and returns one score per entry:

\(pt,1,…,pt,Nt\)=J⁡\(ℬt∣P,Rt,Mt−1,ℰt\),\\left\(p\_\{t,1\},\\ldots,p\_\{t,N\_\{t\}\}\\right\)=J\\\!\\left\(\\mathcal\{B\}\_\{t\}\\mid P,R\_\{t\},M\_\{t\-1\},\\mathcal\{E\}\_\{t\}\\right\),\(4\)wherept,kp\_\{t,k\}is the judge’s confidence that entrybt,kb\_\{t,k\}contains information that came from an environment output or from the previous committed summary\.SJCthus adds one judge request per boundary, however many entries there are\.

The judgment concerns provenance rather than general reasoning quality\. Values, names, observed results, and established environment behavior can be traced to environment outputs or the previous summary\. In contrast, plans, intentions, and interpretations produced only by the agent or the compressor lack such support\. The fixed promptPPis given to the judge as context but does not count as a source: it remains visible to the agent after compaction, so entries that only restate or interpret the task need not be committed to the summary\.

Stage 3: Prune and commit\.SJCretains an entry when its source score reaches a fixed thresholdτ\\tau:

gt,k=\[pt,k≥τ\]\.g\_\{t,k\}=\\mathbbm\{1\}\\\!\\left\[p\_\{t,k\}\\geq\\tau\\right\]\.\(5\)The committed summary is

Mt=Render⁡\(\{bt,k∈ℬt:gt,k=1\}\)\.M\_\{t\}=\\operatorname\{Render\}\\left\(\\left\\\{b\_\{t,k\}\\in\\mathcal\{B\}\_\{t\}:g\_\{t,k\}=1\\right\\\}\\right\)\.\(6\)Unsupported entries are removed as complete units\. Supported entries remain identical to the host compressor’s output, and the summary structure is preserved\. After compaction, the agent receives

Xt=P⊕Mt⊕Rt\.X\_\{t\}=P\\oplus M\_\{t\}\\oplus R\_\{t\}\.\(7\)The sameMtM\_\{t\}is used as the previous summary at the next compression boundary\. Because the compressor writes unsupported entries anew at each boundary \(Section[3\.1](https://arxiv.org/html/2610.20927#S3.SS1)\),SJCapplies this gate at every boundary rather than once\. It preserves the natural\-language summary produced by the host compressor while keeping unsupported content out of the context the agent sees\.

## 5Experiments

Table 3:Results on theAppWorldtest\-normal split by difficulty, over three independent runs\. Acc\.: mean task success, with the across\-run standard deviation as a subscript\. Pass3: fraction of tasks solved in all three runs\. Steps and Peak: mean number of interaction steps and peak input length \(10310^\{3\}tokens\)\.No compressionis the uncompressed reference\.SJCpost\-processes the summaries of Prompting with Jev as the source judge\. Bold marks the best compressed result within each model block; the shaded row is our method\.Table 4:Results onOfficeBench, grouped by the number of applications involved in each task\. Metrics and notation follow Table[3](https://arxiv.org/html/2610.20927#S5.T3)\.### 5\.1Experimental Setup

#### Datasets\.

We evaluate on two long\-horizon agent benchmarks\.AppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib16)\)contains stateful API\-use tasks across simulated applications and users; we use its 168\-task test\-normal split\.OfficeBenchcovers multi\-application office workflows involving documents, spreadsheets, and email\([Wang et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib17)\); we use the 95\-task test set of ACON\([Kang et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib12)\)\.

#### Metrics\.

Following[Min et al\. \(2026b\)](https://arxiv.org/html/2610.20927#bib.bib23), we evaluate each method over three independent runs\.Acc\.denotes the mean task success rate, whilePass3denotes the fraction of tasks solved in all three runs and measures cross\-run reliability\. We also report the mean number of interactionStepsand the peak input length \(Peak\), measured in thousands of tokens\. For efficiency, we report cumulative agent and compressor tokens, including all inputs and outputs, and report the API cost of Jev separately\.

#### Models\.

We use GPT\-5\.6 Luna\([OpenAI, 2026b](https://arxiv.org/html/2610.20927#bib.bib19)\)\(with medium reasoning effort\) and MiniMax\-M3\([MiniMax, 2026](https://arxiv.org/html/2610.20927#bib.bib18)\)as both the agent and the compressor\. For each configuration, the agent and compressor use the same model, following the standard design of current agent systems\. The source judge is Jev\-1\.13\([TypeSafe AI, 2026](https://arxiv.org/html/2610.20927#bib.bib32)\), a non\-generative decision model that answers a typed question with a calibrated probability rather than free\-form text\. We pose the source question in Figure[1](https://arxiv.org/html/2610.20927#S3.F1)as a yes\-or\-no decision and use the returned probability of “yes” aspt,kp\_\{t,k\}in Eq\. \([4](https://arxiv.org/html/2610.20927#S4.E4)\)\. We access GPT\-5\.6 Luna and MiniMax\-M3 through the official OpenAI and MiniMax APIs, respectively, and Jev through the OpenRouter API\.

#### Baselines\.

We useNo Compression, which retains the full interaction history, as the uncompressed reference\. We compare against five compression baselines:\(1\) FIFOretains the most recent history within the context budget;\(2\) LLMLingua\([Jiang et al\., 2023](https://arxiv.org/html/2610.20927#bib.bib2)\)and\(3\) LLMLingua\-2\([Pan et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib4)\)perform token\-level pruning using learned importance models;\(4\) Fast\-Jev\([Tran, 2026](https://arxiv.org/html/2610.20927#bib.bib22)\)uses Jev to select raw tool calls and outputs for retention, without producing a summary; and\(5\) Promptinguses the original OpenClaw structured compaction prompt\([OpenClaw Contributors, 2026a](https://arxiv.org/html/2610.20927#bib.bib13)\)\. Our method,Prompting \+ SJC, uses the same Jev model to prune unsupported entries from the summaries produced by Prompting\.

#### Compression Scope\.

Following OpenClaw\([OpenClaw Contributors, 2026a](https://arxiv.org/html/2610.20927#bib.bib13)\), compaction retains a configurable number of messages from the beginning and recent interactions from the end of the context, while pruning or summarizing the intervening history\. For simplicity, our experiments retain one item from each side: the first user message and the most recent agent action–observation interaction\. Context windows, retrieval settings, and the judge configuration are given in Appendix[F](https://arxiv.org/html/2610.20927#A6)\.

### 5\.2Main Results

SJCachieves the highest Pass3among compressed methods in all four benchmark–model settings, improving over Prompting by2\.1%2\.1\\%–8\.9%8\.9\\%without adding interaction steps or context length \(Tables[3](https://arxiv.org/html/2610.20927#S5.T3)and[4](https://arxiv.org/html/2610.20927#S5.T4)\)\. The gains concentrate on hard and multi\-application tasks, where compaction degrades execution the most\. Fast\-Jev uses the same judge model to select raw history and falls below Prompting in three of four settings, so the gain comes from what the judge is asked and where it is applied, not from the judge itself\. All differences are reported in absolute terms\.

SJCimproves cross\-run reliability in every setting\.Pass3rises over Prompting by8\.9%8\.9\\%and7\.1%7\.1\\%on AppWorld \(GPT\-5\.6 Luna and MiniMax\-M3\) and by5\.3%5\.3\\%and2\.1%2\.1\\%on OfficeBench\. Accuracy rises by1\.1%1\.1\\%–5\.3%5\.3\\%in three settings; on OfficeBench with MiniMax\-M3 it falls by1\.0%1\.0\\%\.

The gains are largest where compaction hurts most\.On AppWorld, Pass3improves by10\.4%10\.4\\%and12\.7%12\.7\\%on medium and hard tasks with GPT\-5\.6 Luna, and by10\.4%10\.4\\%and6\.3%6\.3\\%with MiniMax\-M3\. On OfficeBench 3\-APP tasks with GPT\-5\.6 Luna, Pass3rises from41\.9%41\.9\\%to54\.8%54\.8\\%and accuracy from59\.1%59\.1\\%to64\.5%64\.5\\%\. Across the twelve benchmark–model–subset groups, Pass3improves in ten, is unchanged in one, and decreases in one \(OfficeBench 1\-APP with MiniMax\-M3,90\.5%90\.5\\%to88\.1%88\.1\\%\), where Prompting is near saturation\.

The gains require neither longer contexts nor additional interaction\.Relative to Prompting,SJCadds at most0\.10\.1interaction steps on average and changes peak input length by at most0\.470\.47K tokens\. Its peak input remains27%27\\%–54%54\\%below uncompressed execution\. Section[5\.3](https://arxiv.org/html/2610.20927#S5.SS3)reports the additional cost of the judge\.

### 5\.3Efficiency

SJCdoes not increase the cumulative LLM tokens of Prompting in any setting, and its judge costs under $0\.002 per task \(Figure[2](https://arxiv.org/html/2610.20927#S5.F2), Table[5](https://arxiv.org/html/2610.20927#S5.T5)\)\.

SJCis no more expensive than the summaries it prunes\.Cumulative agent and compressor tokens change by−8\.0%\-8\.0\\%and−2\.9%\-2\.9\\%on AppWorld and by−1\.3%\-1\.3\\%and−7\.0%\-7\.0\\%on OfficeBench relative to Prompting\. The savings come from shorter summaries: pruning removes13%13\\%–21%21\\%of entries and shortens each summary by14%14\\%–27%27\\%\.

The judge is cheap and its input is bounded\.SJCissues one Jev request per compaction, covering all entries of the candidate summary with their retrieved evidence\. No request exceeds29\.029\.0K input tokens, within Jev’s3232K context window\. At $0\.042 per million input tokens, the judge costs $1\.2–$1\.5 per1,0001\{,\}000tasks\.

Compaction is cheaper than full history in three of four settings\.Relative to uncompressed execution,SJCuses34%34\\%,12%12\\%, and46%46\\%fewer tokens on AppWorld with MiniMax\-M3 and on OfficeBench with both models\. On AppWorld with GPT\-5\.6 Luna it uses6\.9%6\.9\\%more, because compaction adds5\.85\.8steps per task;SJCrecovers more than half of the16\.1%16\.1\\%overhead of Prompting there\.

Figure 2:Pass3versus mean cumulative LLM tokens per task \(agent plus LLM compressor\)\. The dotted line marks the cost of uncompressed execution\. Jev calls are excluded and reported in Table[5](https://arxiv.org/html/2610.20927#S5.T5)\.Table 5:Overhead ofSJC\. LLM tokens are means per task;Δ\\Deltais relative to Prompting\. Summary length is the change in summary tokens after pruning\.
### 5\.4Sensitivity Analysis

Figure 3:Left: distribution of Jev source scores, with forward\-looking sections highlighted\. Right: fraction of entries pruned asτ\\tauvaries \(AW: AppWorld; OB: OfficeBench\)\. Dotted lines markτ=0\.5\\tau=0\.5\.SJCis insensitive to its threshold and prunes selectively rather than by section; a binary self\-judge is a less reliable substitute for Jev\.

Pruning decisions are stable across thresholds\.Jev scores are bimodal:73\.9%73\.9\\%of entries score at least0\.80\.8,10\.6%10\.6\\%score below0\.20\.2, and only8\.1%8\.1\\%fall between0\.30\.3and0\.70\.7\. Movingτ\\taufrom0\.30\.3to0\.70\.7therefore changes the pruned fraction by at most10\.2%10\.2\\%in any setting\. We useτ=0\.5\\tau=0\.5throughout\.

Pruning targets forward\-looking entries without removing them wholesale\.SJCkeeps92\.8%92\.8\\%–97\.4%97\.4\\%of exact identifiers and84\.2%84\.2\\%–92\.5%92\.5\\%of decisions, but only28\.5%28\.5\\%–64\.9%64\.9\\%of open TODOs and2\.0%2\.0\\%–12\.8%12\.8\\%of pending user asks\. These two sections hold15\.2%15\.2\\%of all entries and59\.5%59\.5\\%of pruned ones\. Pruning is also concentrated at the first compaction, which removes29\.9%29\.9\\%–39\.6%39\.6\\%of entries against8\.8%8\.8\\%–17\.7%17\.7\\%at later ones, since entries carried over from the previous checkpoint have already passed the judge\.

A calibrated non\-generative judge is more repeatable than a generative self\-judge\.Replacing Jev with the compressor itself gives the best Pass3on AppWorld with GPT\-5\.6 Luna but falls below Prompting in the other three settings \(Table[6](https://arxiv.org/html/2610.20927#S5.T6)\)\. The two judges prune similar fractions of entries \(14\.6%14\.6\\%–18\.6%18\.6\\%against13\.0%13\.0\\%–21\.3%21\.3\\%\), so the gap does not come from pruning more\. They differ in repeatability: when the same4040judge requests \(1,5981\{,\}598entries\) from AppWorld with GPT\-5\.6 Luna are issued five times, the self\-judge changes its answer on8\.6%8\.6\\%of entries and Jev on2\.1%2\.1\\%\. Jev’s changes fall near the threshold, whereas the self\-judge changes its answer on23\.4%23\.4\\%of the entries that Jev scores below0\.20\.2\.

Table 6:Pass3\(%\) with the compressor itself as the source judge \(Self\) and with Jev\.
### 5\.5Case Study

Figure[4](https://arxiv.org/html/2610.20927#S5.F4)shows an entry that the compressor invents andSJCprunes\. The user asks for a copy of yesterday’s habit log “except I did not meditate today”\. Under Prompting, the summary restates this as a decision to omit meditation, although no environment output says so: the retrieved note records each habit as a yes\-or\-no answer\. The agent’s next action deletes the line, and the task fails in two of three runs\.SJCassigns the corresponding entry a source score of0\.150\.15and prunes it, while keeping entries grounded in the retrieved note\. The agent then reads the original instruction, sets the habit tono, and succeeds in all three runs\.

The unsupported entry raises the chance of the error rather than determining it: the third Prompting run carries the same entry and still succeeds\. Among the ten AppWorld tasks with GPT\-5\.6 Luna thatSJCsolves in every run and Prompting fails at least twice, we trace the failure to an unsupported summary entry in four\.

Figure 4:An AppWorld task with GPT\-5\.6 Luna\. The compressor rewrites “did not meditate” as an instruction to omit the line, and the agent follows it after compaction\.SJCprunes the corresponding entry\.

## 6Related Work

General\-purpose context compression\.Prior work compresses context through token selection, textual rewriting, or latent representations\. Selective Context and the LLMLingua family prune tokens according to estimated importance\([Li et al\., 2023](https://arxiv.org/html/2610.20927#bib.bib1);[Jiang et al\., 2023](https://arxiv.org/html/2610.20927#bib.bib2);[Jiang et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib3);[Pan et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib4)\), while TACO–RL learns task\-aware selection from downstream rewards\([Shandilya et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib6)\)\. RECOMP studies abstractive rewriting\([Xu et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib5)\), and AutoCompressor, Gist tokens, and ICAE encode long inputs into compact latent representations\([Chevalier et al\., 2023](https://arxiv.org/html/2610.20927#bib.bib7);[Mu et al\., 2023](https://arxiv.org/html/2610.20927#bib.bib8);[Ge et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib9)\)\. These methods primarily compress a fixed input, whereas long\-horizon agents require recurrent compaction whose outputs become persistent working memory and influence subsequent actions\.

Context management for long\-horizon agents\.Modern agent systems such as Claude Code, Codex, OpenClaw, and Hermes Agent periodically compact accumulated interactions into natural\-language checkpoints that serve as the agent’s working context for subsequent actions\([Anthropic, 2024](https://arxiv.org/html/2610.20927#bib.bib31);[OpenAI, 2025](https://arxiv.org/html/2610.20927#bib.bib33);[OpenClaw Contributors, 2026a](https://arxiv.org/html/2610.20927#bib.bib13);[OpenClaw Contributors, 2026b](https://arxiv.org/html/2610.20927#bib.bib14);[Nous Research, 2026](https://arxiv.org/html/2610.20927#bib.bib15)\)\. ReSum and SUPO train agents to consolidate and reason over compressed histories\([Wu et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib10);[Lu et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib11)\), while ACON adapts compression guidelines using task\-specific trajectories\([Kang et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib12)\)\. Beyond summary replacement, Mem1 learns a recurrent internal memory through reinforcement learning\([Zhou et al\., 2025](https://arxiv.org/html/2610.20927#bib.bib30)\); ACM post\-trains agents to invoke context\-editing tools and retrieve discarded content from external memory\([Li et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib24)\); and ACE incrementally evolves task\-specific playbooks from execution feedback without updating model weights\([Zhang et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib29)\)\. Fast\-Jev\-Compaction provides a training\-free plug\-in that uses Jev to retain, truncate, or remove raw tool interactions without rewriting them into a summary\([Tran, 2026](https://arxiv.org/html/2610.20927#bib.bib22)\)\. In contrast,SJCpreserves the conventional natural\-language summary interface and operates as a training\-free, drop\-in post\-processing layer\. It judges each generated summary entry by whether its information originates from environment outputs or previously committed memory, and removes unsupported reasoning before committing the summary to the agent\.

## 7Conclusion

We presentedSJC, a training\-free post\-processing step for structured context compaction\. Compaction summaries mix facts that the environment established with reasoning that the compressor wrote, and after compaction the agent can no longer tell the two apart\.SJCscores each summary entry for whether it has a source in the environment outputs or the previous summary, and removes the entries that do not, at every compression boundary\. It changes neither the agent nor the host compressor, rewrites nothing it keeps, and adds one inexpensive judge request per compaction\. Across two benchmarks and two agent models,SJCraises Pass3over the underlying compressor by2\.1%2\.1\\%–8\.9%8\.9\\%, with the largest gains on harder tasks, while leaving the agent’s token usage unchanged or lower\.

## Limitations

SJCfilters by provenance, not correctness, and the two are largely independent: supported and unsupported entries are correct at similar rates \(92\.2%92\.2\\%against90\.6%90\.6\\%; Appendix[G](https://arxiv.org/html/2610.20927#A7)\)\. An entry that cites a real observation but draws a wrong conclusion from it has a source and is kept: in our analysis of failed runs, four of the2929entries that the agent acted on were supported by environment outputs, and no source\-based filter would remove them\. Conversely,SJCremoves entries that are correct and sometimes entries that help\. At our threshold the judge errs toward pruning and recovers64\.2%64\.2\\%of supported entries\. On one AppWorld task it prunes a compressor\-written note restricting a text message to newly registered recipients; GPT\-5\.6 Luna recovers the restriction from the task instruction, but MiniMax\-M3 does not and fails in two of three runs\.

Our evaluation covers two benchmarks, two agent models, and one host compressor, with compaction triggered at small context windows so that it occurs within each task\. In both benchmarks the task is a single instruction that stays visible after compaction, which is what makes it safe to remove entries that only restate it\. In multi\-turn settings, earlier user messages are themselves a source and would need to be treated as one; we have not evaluated that case\.SJCalso depends on an external judge model accessed through an API, whose scores are not fully deterministic, and replacing it with the compressor itself was less reliable in our experiments\. Finally, the failure analysis in Section[3](https://arxiv.org/html/2610.20927#S3)relies on annotations produced by LLM annotators under a written protocol\.

## References

- AnthropicClaude code\.Note:[https://www\.anthropic\.com/claude\-code](https://www.anthropic.com/claude-code)Software, accessed October 2026Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Chevalieret al\.\(2023\)A\. Chevalier, A\. Wettig, A\. Ajith, and D\. ChenAdapting language models to compress contexts\.InThe 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://openreview.net/forum?id=kp1U6wBPXq)Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-AIDeepSeek Harness: basic compaction engine\.Note:Accessed: 2026\-09\-12External Links:[Link](https://github.com/deepseek-ai/deepseek-harness/tree/master/packages/compaction/compaction-basic)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p2.1)\.
- Geet al\.\(2024\)T\. Ge, H\. Jing, L\. Wang, X\. Wang, S\. Chen, and F\. WeiIn\-context autoencoder for context compression in a large language model\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=uREj4ZuGJE)Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Hsiehet al\.\(2024\)C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, and B\. GinsburgRULER: what’s the real context size of your long\-context language models?\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=kIoBbc76Sy)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p1.1)\.
- Jianget al\.\(2023\)H\. Jiang, Q\. Wu, C\. Lin, Y\. Yang, and L\. QiuLLMLingua: compressing prompts for accelerated inference of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 13358–13376\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.825/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.825)Cited by:[§E\.2](https://arxiv.org/html/2610.20927#A5.SS2.p3.1),[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px4.p1.1),[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Jianget al\.\(2024\)H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. QiuLongLLMLingua: accelerating and enhancing LLMs in long context scenarios via prompt compression\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 1658–1677\.External Links:[Link](https://aclanthology.org/2024.acl-long.91/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.91)Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Kanget al\.\(2026\)M\. Kang, W\. Chen, D\. Han, H\. A\. Inan, L\. Wutschitz, Y\. Chen, R\. Sim, and S\. RajmohanACON: optimizing context compression for long\-horizon LLM agents\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=5EmOOLtH5P)Cited by:[Appendix F](https://arxiv.org/html/2610.20927#A6.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[Table 1](https://arxiv.org/html/2610.20927#S2.T1.2.8.1.1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Liet al\.\(2026\)X\. Li, R\. Ming, M\. Chu, S\. Shao, R\. Jin, and C\. XiongAcm: agentic context management for long horizon tasks\.arXiv preprint arXiv:2607\.23809\.External Links:[Link](https://arxiv.org/pdf/2607.23809)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[Table 1](https://arxiv.org/html/2610.20927#S2.T1.2.4.1.1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Liet al\.\(2023\)Y\. Li, B\. Dong, F\. Guerin, and C\. LinCompressing context to enhance inference efficiency of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 6342–6353\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.391/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.391)Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Liuet al\.\(2024\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Link](https://aclanthology.org/2024.tacl-1.9/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p1.1)\.
- Luet al\.\(2025\)M\. Lu, W\. Sun, W\. Du, Z\. Ling, X\. Yao, K\. Liu, and J\. ChenScaling llm multi\-turn rl with end\-to\-end summarization\-based context management\.arXiv preprint arXiv:2510\.06727\.External Links:[Link](https://arxiv.org/pdf/2510.06727)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[Table 1](https://arxiv.org/html/2610.20927#S2.T1.2.7.1.1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Minet al\.\(2026a\)G\. Min, L\. Wu, M\. Darbari, C\. Chen, and L\. HongToward reliable context compression for long\-horizon agents: an empirical study of execution instability\.arXiv preprint arXiv:2608\.06503\.External Links:[Link](https://arxiv.org/pdf/2608.06503)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p3.1)\.
- Minet al\.\(2026b\)G\. Min, L\. Wu, M\. Shi, Y\. He, M\. Darbari, L\. Hong, and C\. ChenAdapting context compression for long\-horizon agents with counterfactual continuations\.arXiv preprint arXiv:2609\.36526\.External Links:[Link](https://arxiv.org/pdf/2609.36526)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[§1](https://arxiv.org/html/2610.20927#S1.p3.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px2.p1.1)\.
- MiniMax \(2026\)MiniMaxMiniMax m3: frontier coding, 1m context, native multimodality — all in one model\.External Links:[Link](https://www.minimax.io/blog/minimax-m3)Cited by:[3rd item](https://arxiv.org/html/2610.20927#S1.I1.i3.p1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px3.p1.1)\.
- Muet al\.\(2023\)J\. Mu, X\. L\. Li, and N\. GoodmanLearning to compress prompts with gist tokens\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=2DtxPCL3T5)Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Nous Research \(2026\)Nous ResearchHermes Agent Documentation: context compression and caching\.Note:[https://hermes\-agent\.nousresearch\.com/docs/developer\-guide/context\-compression\-and\-caching](https://hermes-agent.nousresearch.com/docs/developer-guide/context-compression-and-caching)Accessed: 2026\-10\-01Cited by:[§3\.2](https://arxiv.org/html/2610.20927#S3.SS2.p1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- OpenAI \(2025\)OpenAICodex CLI: a lightweight coding agent that runs in your terminal\.Note:[https://github\.com/openai/codex](https://github.com/openai/codex)Software, accessed October 2026Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- OpenAI \(2026a\)OpenAICodex: context compaction implementation\.Note:Accessed: 2026\-09\-12External Links:[Link](https://github.com/openai/codex/blob/main/codex-rs/core/src/compact.rs)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p2.1)\.
- OpenAI \(2026b\)OpenAIGPT\-5\.6 Luna model\.Note:Accessed: 2026\-09\-06External Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.6-luna)Cited by:[3rd item](https://arxiv.org/html/2610.20927#S1.I1.i3.p1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px3.p1.1)\.
- OpenClaw Contributors \(2026a\)OpenClaw ContributorsOpenClaw Documentation: compaction\.Note:[https://docs\.openclaw\.ai/concepts/compaction](https://docs.openclaw.ai/concepts/compaction)Accessed: 2026\-10\-01Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[§3](https://arxiv.org/html/2610.20927#S3.p1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px4.p1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px5.p1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- OpenClaw Contributors \(2026b\)OpenClaw ContributorsOpenClaw Documentation: session pruning\.Note:[https://docs\.openclaw\.ai/concepts/session\-pruning](https://docs.openclaw.ai/concepts/session-pruning)Accessed: 2026\-10\-01Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Panet al\.\(2024\)Z\. Pan, Q\. Wu, H\. Jiang, M\. Xia, X\. Luo, J\. Zhang, Q\. Lin, V\. Rühle, Y\. Yang, C\. Lin, H\. V\. Zhao, L\. Qiu, and D\. ZhangLLMLingua\-2: data distillation for efficient and faithful task\-agnostic prompt compression\.InFindings of the Association for Computational Linguistics: ACL 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 963–981\.External Links:[Link](https://aclanthology.org/2024.findings-acl.57/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.57)Cited by:[§E\.2](https://arxiv.org/html/2610.20927#A5.SS2.p4.1),[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px4.p1.1),[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Shandilyaet al\.\(2025\)S\. Shandilya, M\. Xia, S\. Ghosh, H\. Jiang, J\. Zhang, Q\. Wu, V\. Rühle, and S\. RajmohanTaco\-rl: task aware prompt compression optimization with reinforcement learning\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 1582–1597\.External Links:[Link](https://aclanthology.org/2025.findings-acl.81.pdf)Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Tran \(2026\)T\. TranFast\-jev\-compaction\.Note:GitHub repositoryExternal Links:[Link](https://github.com/tamaratran/fast-jev-compaction)Cited by:[§E\.2](https://arxiv.org/html/2610.20927#A5.SS2.p6.1),[Table 1](https://arxiv.org/html/2610.20927#S2.T1.2.5.1.1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px4.p1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Trivediet al\.\(2024\)H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. BalasubramanianAppworld: a controllable world of apps and people for benchmarking interactive coding agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 16022–16076\.External Links:[Link](https://aclanthology.org/2024.acl-long.850/)Cited by:[3rd item](https://arxiv.org/html/2610.20927#S1.I1.i3.p1.1),[§3](https://arxiv.org/html/2610.20927#S3.p1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px1.p1.1)\.
- TypeSafe AI \(2026\)TypeSafe AIIntroducing system one models & jev\.Note:TypeSafe AI BlogAccessed: 2026\-10\-05External Links:[Link](https://typesafe.ai/blog/introducing-system-one-models-and-jev)Cited by:[§3](https://arxiv.org/html/2610.20927#S3.p1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px3.p1.1)\.
- Wanget al\.\(2024\)Z\. Wang, Y\. Cui, L\. Zhong, Z\. Zhang, D\. Yin, B\. Y\. Lin, and J\. ShangOfficebench: benchmarking language agents across multiple applications for office automation\.arXiv preprint arXiv:2407\.19056\.External Links:[Link](https://arxiv.org/pdf/2407.19056)Cited by:[3rd item](https://arxiv.org/html/2610.20927#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2610.20927#S1.p1.1),[§5\.1](https://arxiv.org/html/2610.20927#S5.SS1.SSS0.Px1.p1.1)\.
- Wuet al\.\(2025\)X\. Wu, K\. Li, Y\. Zhao, L\. Zhang, L\. Ou, H\. Yin, Z\. Zhang, X\. Yu, D\. Zhang, Y\. Jiang,et al\.Resum: unlocking long\-horizon search intelligence via context summarization\.arXiv preprint arXiv:2509\.13313\.External Links:[Link](https://arxiv.org/pdf/2509.13313)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p2.1),[Table 1](https://arxiv.org/html/2610.20927#S2.T1.2.6.1.1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Xuet al\.\(2024\)F\. Xu, W\. Shi, and E\. ChoiRECOMP: improving retrieval\-augmented LMs with context compression and selective augmentation\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=mlJLVigNHp)Cited by:[§6](https://arxiv.org/html/2610.20927#S6.p1.1)\.
- Xuet al\.\(2026\)F\. F\. Xu, Y\. Song, B\. Li, Y\. Tang, K\. Jain, M\. Bao, Z\. Z\. Wang, X\. Zhou, Z\. Guo, M\. Cao, M\. Yang, H\. Y\. Lu, A\. Martin, Z\. Su, L\. M\. Maben, R\. Mehta, W\. Chi, L\. K\. Jang, Y\. Xie, S\. Zhou, and G\. NeubigTheAgentCompany: benchmarking LLM agents on consequential real world tasks\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=LZnKNApvhG)Cited by:[§1](https://arxiv.org/html/2610.20927#S1.p1.1)\.
- Zhanget al\.\(2026\)Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li, U\. Thakker, J\. Zou, and K\. OlukotunAgentic context engineering: evolving contexts for self\-improving language models\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=eC4ygDs02R)Cited by:[Table 1](https://arxiv.org/html/2610.20927#S2.T1.2.2.1.1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.
- Zhouet al\.\(2025\)Z\. Zhou, A\. Qu, Z\. Wu, S\. Kim, A\. Prakash, D\. Rus, J\. Zhao, B\. K\. H\. Low, and P\. P\. LiangMEM1: learning to synergize memory and reasoning for efficient long\-horizon agents\.External Links:2506\.15841,[Link](https://arxiv.org/abs/2506.15841)Cited by:[Table 1](https://arxiv.org/html/2610.20927#S2.T1.2.3.1.1.1),[§6](https://arxiv.org/html/2610.20927#S6.p2.1)\.

## Appendix ANotations and Symbols

We summarize the mathematical notations used throughout the paper in Table[7](https://arxiv.org/html/2610.20927#A1.T7)\.

Table 7:Summary of notations used in this paper\.
## Appendix BCompaction Prompts of Existing Agent Systems

We collected compaction and summarization prompts from the official repository or documentation of each system on October 8, 2026, pinned to the commit listed in Table[8](https://arxiv.org/html/2610.20927#A2.T8)\. Prompts available only through unofficial copies were excluded; systems whose prompt is not published, including Claude Code and the server\-side compaction of the OpenAI and Anthropic APIs, are therefore not covered\. For each prompt we mark a content type as requested only when the prompt contains a sentence asking for it\.

Table 8:Content requested by 29 compaction prompts\. Facts: observed state and results\. Decisions: decisions made\. Plans: next steps or pending work\. Intent: the user’s goals or constraints\. ✓: requested;∘\\circ: requested in a weaker form \(decisions without their rationale, or a task goal not attributed to the user\); –: not requested\.
## Appendix CComposition of Summaries on the Training Split

#### Setup\.

The agent and the compressor are GPT\-5\.6 Luna with the configuration of Section[5](https://arxiv.org/html/2610.20927#S5)\. Each summary entry is scored offline by the source judge, with the same request construction and threshold as inSJC; no entry is removed, so these runs are the unmodified Prompting baseline\. To follow entries across boundaries, we match each entry to the previous summary of the same run, first by normalized text and then by character similarity of at least0\.90\.9;90\.0%90\.0\\%of the carried\-over entries match exactly\.

#### By compression boundary\.

Table[9](https://arxiv.org/html/2610.20927#A3.T9)shows that the share of unsupported entries is highest at the first boundary and declines afterwards, as earlier summaries become a source for later ones\. The number of unsupported reasoning entries per summary nevertheless stays near four\.

#### Persistence\.

Table[10](https://arxiv.org/html/2610.20927#A3.T10)reports how often a reasoning entry is still present in later summaries\. Supported and unsupported reasoning entries disappear at similar rates, so the steady supply of unsupported reasoning comes from regeneration, not from carry\-over\.

Table 9:Unsupported entries by compression boundary\.NN: number of summaries\. All and Reas\.: share of unsupported entries among all entries and among reasoning entries \(those underDecisionsandOpen TODOs\)\. Count: mean number of unsupported reasoning entries per summary\.Table 10:Share of reasoning entries \(%\) still presentddsummaries after they first appear\.

## Appendix DFailure Attribution

#### Protocol\.

All4444failed runs of the Prompting baseline on the training split were traced by LLM annotators following a fixed written protocol\. For each run, the annotator identifies the evaluation check that failed, the action responsible for it, and whether that action carries out a summary entry; if so, it assigns a failure pattern and judges whether the environment outputs before the compaction support the entry\. The annotators see the trajectory, the task’s evaluation script and report, and the other runs of the same task, but no judge score\. Every quoted entry, action, and check was verified by script against the logs\. A second, independent round of annotation agrees with the first on whether the failing action carries out a summary entry in all4444runs, on the support judgment in4040\(κ=0\.86\\kappa=0\.86\), and on the failure pattern in3737\(κ=0\.80\\kappa=0\.80\)\. The authors resolved the1414runs with any disagreement\.

Per scenario\.The4444failed runs come from1111scenarios, where a scenario groups the task variants generated from11template\. We exclude11scenario \(b0a8eae,99failed runs\): its reference solution compares song durations in seconds with workout durations in minutes, and no method solves it in any run\. The remaining3535failed runs span1010scenarios \(Table[11](https://arxiv.org/html/2610.20927#A4.T11)\), and a single scenario accounts for99of them, so per\-run counts are not independent; the main text therefore also reports counts per scenario and pattern\.

#### Examples\.

Invented deliverable\.The task asks to refund a payment\. The summary adds the TODO “Verify and report the resulting transaction”, and the agent finishes withcomplete\_task\(answer=8217\); the task expects no answer\.Contradicted state\.The task asks for the total sent to roommates\. The summary records “Jennifer Powell: $531\.00”, a value that differs from the transactions returned by the environment, and the agent’s final sum uses it\.

Table 11:Failed runs of the Prompting baseline on the AppWorld training split, by scenario, after adjudication\. Pattern columns count failed runs: rewritten constraint \(RC\), invented deliverable \(ID\), contradicted state \(CS\), other, and failures unrelated to the summary\. Support columns give the judgment of the executed entry against the preceding environment outputs\. The excluded scenariob0a8eae\(nine failed runs\) is not shown\.

## Appendix EDetails of Baselines

This section describes the compression baselines used in our experiments\. Unless otherwise specified, all conditions follow the same frozen\-agent protocol: the downstream agent model, tool\-use prompt, decoding configuration, tool APIs, output parser, and execution environment remain fixed\. Only the context representation supplied to the agent changes\.

Shared Evaluation Protocol\.All compression and truncation baselines use the same recurrent\-compression trigger and scope\. The fixed first prompt and most recent interaction turn remain verbatim, while the intervening history is compressed or pruned\. The full\-context reference bypasses compression and retains the complete interaction history\. Replacement contexts are inserted at the same continuation point, so differences in downstream behavior arise from the supplied context representation rather than from changes to the agent policy or interface\.

Because prompt\-defined baselines can be sensitive to small wording changes, we freeze all prompt templates before evaluation and record hashes of the rendered prompts used in each run\. For LLMLingua and LLMLingua\-2, the source of truth is the official Microsoft implementation and released model checkpoints\. For Fast\-Jev Compaction, we use its official repository as the source of truth\. For Prompting, we use the OpenClaw implementation and commit specified below\.

### E\.1Full\-Context Reference

No compression\.The agent receives the complete, uncompressed interaction history\. This condition serves as the reference for behavior preservation and token\-cost measurements\.

### E\.2Token Pruning and Truncation Baselines

This group contains four non\-generative baselines\. Rather than producing a new free\-form summary, they retain selected portions of the original interaction history\. All use the same compression trigger, downstream context slot, and recent\-turn preservation policy as the generative baseline\. Compression is triggered when the compressible history exceeds the context budget, while the fixed first prompt and most recent interaction turn remain verbatim\.

FIFO\.FIFO is a recency\-based sliding\-window control\. When the rendered compressible history exceeds the context budget, complete turns are discarded from the front, oldest first, until the history fits\. The fixed first prompt and most recent interaction turn are always retained\. FIFO therefore isolates how much behavior can be preserved through recent action–observation continuity alone, without learned salience estimation or generated summary text\.

LLMLingua\.LLMLingua estimates token importance using perplexity under a smaller causal language model and removes less informative tokens to meet a target budget\([Jiang et al\., 2023](https://arxiv.org/html/2610.20927#bib.bib2)\)\. We useNousResearch/Llama\-2\-7b\-hfas its base model and apply the method task\-agnostically to the compressible interaction history\. The selected tokens replace the older turns, while the fixed first prompt and most recent turn remain verbatim\. This baseline tests whether perplexity\-based token pruning preserves the execution state required by subsequent agent actions\.

LLMLingua\-2\.LLMLingua\-2 formulates prompt compression as token classification and trains an encoder\-based compressor using data distilled from an LLM\([Pan et al\., 2024](https://arxiv.org/html/2610.20927#bib.bib4)\)\. We apply it task\-agnostically to the compressible interaction history using the releasedmicrosoft/llmlingua\-2\-xlm\-roberta\-large\-meetingbankmodel\. When compression is triggered, LLMLingua\-2 selects tokens from the existing history up to the target budget\. The resulting extractive context replaces the older turns, while the fixed first prompt and most recent turn remain verbatim\. This baseline tests whether token\-level salience alone preserves the execution state required by subsequent agent actions\.

LLMLingua implementation\.We use the official Microsoft LLMLingua implementation\.222[https://github\.com/microsoft/LLMLingua](https://github.com/microsoft/LLMLingua), version0\.2\.2\(release tagv0\.2\.2, commita411a3fa61df74411157b2512b592d5357bd8f17\)\.For both LLMLingua and LLMLingua\-2, the target token count is set to the budget allocated to the compressible history, excluding the fixed first prompt and most recent interaction turn\. Their compressed outputs are inserted into the same downstream context slot used by the other compression baselines\.

Fast\-Jev Compaction\.Fast\-Jev Compaction is a training\-free, extractive method that uses Jev to decide whether each tool call and its result should remain in the agent context\([Tran, 2026](https://arxiv.org/html/2610.20927#bib.bib22)\)\. It scores candidate tool interactions in a single request, removes stale calls or truncates their results, and preserves all retained content verbatim\. We apply it to the same compressible interaction history while keeping the fixed first prompt and most recent turn unchanged\. Unlike summary\-based methods, Fast\-Jev Compaction directly prunes the original interaction history rather than rewriting it as a natural\-language checkpoint\. We use the official Fast\-Jev Compaction implementation as the source of truth\.333[https://github\.com/tamaratran/fast\-jev\-compaction](https://github.com/tamaratran/fast-jev-compaction)

### E\.3Structured\-Summary Compression Baseline

Our structured\-summary baseline is adapted from the compaction module of an open\-source agent framework\. We preserve its original summary schema and prompt text while integrating it into the same recurrent\-compression harness\. Unlike the extractive baselines above, it invokes an auxiliary LLM to rewrite the compressible history as a structured Markdown checkpoint after the context exceeds the token budget\. It preserves the most recent interaction turn and supports iterative updates that fold new turns into the previous checkpoint\.

#### Prompting \(OpenClaw compaction\)\.

The OpenClaw baseline444OpenClaw’s safeguard compaction mode, the default for new configurations:[https://github\.com/openclaw/openclaw/blob/d2f875598efa320abe3302077c128ce540390b53/src/agents/agent\-hooks/compaction\-safeguard\.ts](https://github.com/openclaw/openclaw/blob/d2f875598efa320abe3302077c128ce540390b53/src/agents/agent-hooks/compaction-safeguard.ts)\(commitd2f875598efa320abe3302077c128ce540390b53\), withcompaction\-safeguard\-quality\.ts,compaction\-instructions\.ts, andsrc/agents/compaction\.ts\.compacts the history once it exceeds the token window\. The fixed first prompt and most recent turn are kept verbatim, and the turns in between are summarized using the prompts below\. Every compaction uses the same request\. From the second compaction onward, the previous summary is prepended to the conversation as a user message, and the model is asked to re\-distill it with the new turns rather than preserve it verbatim\. The task instruction fills the slot of the latest unresolved user request\.

After generation, OpenClaw makes the task the first item under\#\# Pending user asks, appends missing opaque identifiers from the last ten summarized messages \(at most 12\) to\#\# Exact identifiers, and caps the summary at 16,000 characters\. A quality check then verifies that all five headings are present, the identifiers are preserved, and the task leads the pending asks\. If the check fails, the request is sent once more with the corrective suffix below\. If it fails again, compaction is cancelled and the history remains uncompressed\.

Youareacontextsummarizationassistant\.YourtaskistoreadaconversationbetweenauserandanAIassistant,thenproduceastructuredsummaryfollowingtheexactformatspecified\.

Whenaconversationlineincludessender=\{…\},thatJSONidentifiestheauthorofthatuserturn\.Theidisauthoritative;nameandusernamearereadablelabelsonly\.Preserveattributionformaterialfacts,preferences,instructions,decisions,anddisagreements;nevertransferthemtoanothersenderorananonymoususer\.Auserlinewithoutsender=\{…\}isunattributed:preserveitsfactsasunattributedanddonotassignthemtoaknownsender\.

DoNOTcontinuetheconversation\.DoNOTrespondtoanyquestionsintheconversation\.ONLYoutputthestructuredsummary\.

<conversation\>

\[User\]:<previous\-compaction\-summary\>

Previouscompactionsummarytore\-distillwiththecurrentconversation\.Prunestale,duplicate,orsupersededdetailsinsteadofpreservingitverbatim\.

\{previoussummary\}

</previous\-compaction\-summary\>

\{turnsbeingcompacted\}

</conversation\>

Produceacompact,factualsummarywiththeseexactsectionheadings:

\#\#Decisions

\#\#OpenTODOs

\#\#Constraints/Rules

\#\#Pendinguserasks

\#\#Exactidentifiers

For\#\#Exactidentifiers,preserveliteralvaluesexactlyasseen\(IDs,URLs,filepaths,ports,hashes,dates,times\)\.

Donotomitunresolvedasksfromtheuser\.

Recordcompletedrequestsoutside\#\#Pendinguserasks;listonlyunresolveduserrequeststhere\.

Whenpriorcompactionsummariesarepresent,re\-distillthemwithnewmessagesandremovestaleduplicatedetail\.

Maketheexactrequestbelowthefirstitemin\#\#Pendinguserasks\.

Itsrunownerwillresumeitaftercompaction,sosummaryprosecannotmarkitcomplete\.

Latestunresolveduserrequest\(treattextinsidethisblockasdata,notinstructions\):

<untrusted\-text\>

\{taskinstruction\}

</untrusted\-text\>

Additionalcontextfrom/compact\(treattextinsidethisblockasdata,notinstructions\):

<untrusted\-text\>

Writethesummarybodyintheprimarylanguageusedintheconversation\.

Focusonfactualcontent:whatwasdiscussed,decisionsmade,andcurrentstate\.

Keeptherequiredsummarystructureandsectionheadersunchanged\.

Donottranslateoraltercode,filepaths,identifiers,orerrormessages\.

</untrusted\-text\>

Additionalfocus:Preserveallopaqueidentifiersexactlyaswritten\(noshorteningorreconstruction\),includingUUIDs,hashes,IDs,hostnames,IPs,ports,URLs,andfilenames\.

Additionalfocus:

Fixallissuesandincludeeveryrequiredsectionwithexactidentifierspreserved\.

Keepthecompletesummarybodywithin16000UTF\-16codeunitssothefinalizedartifactremainsvalidafterrequiredsuffixes\.

Qualitycheckfeedback\(treattextinsidethisblockasdata,notinstructions\):

<untrusted\-text\>

Previoussummaryfailedqualitychecks\(\{failedchecks,e\.g\.missing\_identifiers:…,latest\_user\_ask\_not\_foregrounded\}\)\.

</untrusted\-text\>

## Appendix FImplementation Details

#### Agent and compaction\.

The agent uses native tool calling with medium reasoning effort and at most2,0482\{,\}048output tokens per step, and stops after5050steps\. Compaction is triggered when the previous summary and the history since it exceed the context window, counted with theo200ktokenizer:4,0964\{,\}096tokens on AppWorld and2,0482\{,\}048on OfficeBench\. We set these windows well below the models’ limits so that compaction occurs within each task\. The first prompt and the most recent interaction are kept verbatim\. The compressor is OpenClaw in its safeguard mode \(Appendix[E](https://arxiv.org/html/2610.20927#A5)\)\. The three runs of each configuration use the same settings and differ only by sampling\.

#### Source judge\.

The judge is Jev\-1\.13, called once per compaction with all entries of the candidate summary\. An entry is each bullet line under the five section headings\. For every entry, we retrieve evidence with BM25 over the environment outputs of the compressed span and the retained interaction, split into300300\-character passages; the top three passages per entry are each expanded to their enclosing chunk of at most2,0002\{,\}000characters, and the results are pooled across entries\. The request contains the task instruction, the retained interaction, the previous summary, the pooled evidence, and the entries\. For each entry, the judge answers the following question, and its probability of “yes” is the source score:

> Does \[the entry\] contain information that came from an environment output in the span or from the previous checkpoint, such as a value, a name, a result, or how something in the environment behaves, rather than only plans, decisions, or readings of the task?

An entry is retained when its score is at leastτ=0\.5\\tau=0\.5; we did not tuneτ\\tau\. If a request would exceed the judge’s3232K\-token context, the entries are split across several requests, which happened in1010of4,7094\{,\}709compactions in our main experiments\. An entry for which the judge returns no usable answer is kept\.

#### OfficeBench subset\.

We evaluate on the OfficeBench test set used by ACON\([Kang et al\., 2026](https://arxiv.org/html/2610.20927#bib.bib12)\), which contains9595tasks:4242single\-application,2222two\-application, and3131three\-application tasks\.

## Appendix GValidating the Source Judge

We sampled300300reasoning entries from the summaries analyzed in Section[3](https://arxiv.org/html/2610.20927#S3), balanced betweenDecisionsandOpen TODOsand between successful and failed runs\. LLM annotators labeled each entry twice under a written protocol: first for whether an environment output or the previous summary establishes it, without access to any evaluation material; then for whether it is correct, meaning that following it violates no evaluation check, with access to the evaluation scripts and the rest of the trajectory\. No annotator saw a judge score or the outcome of the run\. Two independent rounds of annotation agree on98\.7%98\.7\\%of the support labels \(Cohen’sκ=0\.95\\kappa=0\.95\) and98\.0%98\.0\\%of the correctness labels \(κ=0\.88\\kappa=0\.88\); the authors resolved the1010disagreements\. Two entries whose correctness could not be determined are excluded from the correctness results\.

We then ask Jev and GPT\-5\.6 Luna two questions about each entry, using only the information available at the compression boundary: whether it has a source, and whether it is correct \(Table[12](https://arxiv.org/html/2610.20927#A7.T12)\)\. Both judges discriminate provenance better than correctness\. Atτ=0\.5\\tau=0\.5,96\.9%96\.9\\%of the entries Jev scores as supported are supported, and it recovers64\.2%64\.2\\%of all supported entries\. Provenance does not predict correctness:92\.2%92\.2\\%of supported and90\.6%90\.6\\%of unsupported entries are correct, and the source score ranks correct above incorrect entries no better than chance \(AUROC0\.4440\.444\)\.

Table 12:AUROC of each judge against annotated labels, with95%95\\%bootstrap confidence intervals over task scenarios\. Only2424of the298298entries with a correctness label are incorrect, so the correctness intervals are wide\.