终端智能体能信任自己的验证吗？诊断与改进自验证

# Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification
Source: [https://arxiv.org/html/2609.38812](https://arxiv.org/html/2609.38812)
††footnotetext:∗This work was done during an internship at Ant Group\.††footnotetext:†Corresponding Authors\.Yingfeng Luo1,∗Shaowei Wei2Daixin Wang2Dingyang Lin1Kaiyan Chang1Weiqiao Shan1Tong Zheng3Zhiqiang Zhang2Jingbo Zhu1Tong Xiao1,†Affiliation:1Northeastern UniversityAffiliation:2Inclusion AI, Ant GroupAffiliation:3University of Maryland, College Park

###### Abstract

Terminal agents rely on self\-verification to assess and correct their solutions as they solve tasks through interaction with command\-line environments\. Yet how trustworthy such self\-verification is remains poorly understood\. To investigate this question systematically, we introduce a diagnostic framework that identifies the first complete solution in each trajectory, determines whether it is objectively correct, and uses this ground truth to quantify the agent’s subsequent verification and recovery behavior\. Applying it to ten terminal agents on TerminalBench2\.1, we find that verification is nearly universal after a complete candidate is formed, yet only 61\.43% of incorrect candidates are detected and only 49\.36% of detected errors are successfully repaired\. These results show that the main weakness in self\-verification lies not in initiating verification, but in detecting and repairing errors\. Motivated by these findings, we proposeStudent\-ConditionedVerificationDistillation \(SCVD\), which lets the student first produce a candidate solution and distills a stronger teacher’s subsequent verification and recovery from the same interaction context\. Across three Qwen3\.5 backbones, SCVD improvesPass@1on TerminalBench2\.1 by 9\.74–16\.85 percentage points over the corresponding base models and by 4\.49–8\.61 points over the standard full\-trajectory distillation, while avoiding the pronounced out\-of\-distribution degradation of full\-trajectory distillation on SWE\-bench Verified\.

## 1Introduction

Terminal agents use language models to solve tasks through multi\-step interactions with command\-line environments, where they issue shell commands and observe the outcomes of their actions\. To complete a task autonomously, a terminal agent needs to determine whether its current solution satisfies the task or requires further revision, often by running tests, inspecting artifacts, or querying system state and interpreting the resulting feedback\. We refer to this process as*self\-verification*, with Figure[1](https://arxiv.org/html/2609.38812#S1.F1)\(a\) providing an illustrative example\. Self\-verification therefore provides a key feedback mechanism for detecting errors and guiding task completion\. When it breaks down, an agent may miss an existing defect and stop with an incorrect solution, or detect the defect but fail to repair it\. This raises a fundamental question for terminal agents:*can they trust their own verification?*

Prior work has shown that language models can use execution feedback to inspect and revise their own solutions\([Chen et al\., 2024](https://arxiv.org/html/2609.38812#bib.bib6),[Gou et al\., 2024](https://arxiv.org/html/2609.38812#bib.bib5)\)\. More recent studies have examined the utility and reliability of agent\-generated tests\([Chen et al\., 2026b](https://arxiv.org/html/2609.38812#bib.bib9),[Sun et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib10),[Tan et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib28)\), developed independent mechanisms for verifying generated patches\([Li et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib27)\), and analyzed how failures emerge and evolve throughout coding\-agent trajectories\([Zhao et al\., 2026b](https://arxiv.org/html/2609.38812#bib.bib14)\)\. However, these lines of work do not jointly establish, at the point of verification, what the agent’s own check concludes and whether the candidate being checked is objectively correct\. Without both pieces of information, a passing verification result may reflect either a correct solution or an undetected error, making it difficult to assess whether self\-verification correctly judges a candidate\. Assessing recovery further requires tracking whether errors exposed by the agent’s own checks are ultimately repaired\.

To address this gap, we introduce a diagnostic framework that combines trajectory annotation with candidate\-state replay\. For each trajectory, we identify the earliest point at which the agent has produced a complete candidate solution, characterize its subsequent verification outcome, and replay the trajectory up to the candidate boundary in a fresh environment to evaluate the candidate with the official task evaluator\. This allows us to compare the outcome of the agent’s own verification with the objective correctness of the candidate being checked, and to assess separately whether the agent initiates verification, detects existing errors, and successfully repairs detected errors\.

Applying this framework to ten terminal agents on TerminalBench2\.1\([Merrill et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib25)\), our analysis reveals a clear gap between attempting verification and actually benefiting from it\.Agents almost always attempt verification: they do so after forming a candidate in 99\.53% of eligible trajectories\. Yet this high verification rate does not translate into reliable error detection\.Although error signals are usually reliable, many errors still go undetected, with 92\.65% of error signals corresponding to objectively incorrect candidates but only 61\.43% of incorrect candidates being detected\. A no\-error signal is therefore only weak evidence of correctness, as only 51\.52% of candidates receiving such a signal are actually correct\. Even when an error is detected,agents often fail to recover successfully, with only 49\.36% of detected errors repaired\. This limitation is particularly notable given that repair success is strongly correlated with final task accuracy across agents \(Pearsonr=0\.98r=0\.98; Figure[2](https://arxiv.org/html/2609.38812#S2.F2)\(c\)\)\.*Terminal agents therefore know to verify, but their verification is not yet trustworthy, and they often fail to recover even when their checks expose an error\.*

Figure 1:Overview of solution\-level self\-verification and SCVD\.\(a\)After forming a candidate solution, the agent performs task\-relevant checks, repairs detected errors, and rechecks the result before termination\.\(b\)SCVD uses a teacher to continue from the student\-generated candidate state to verify and revise the solution\. After filtering successful trajectories and removing the temporary verification scaffold, the SFT loss is applied only to the teacher continuation\.These findings highlight substantial room to improve the reliability of verification and the effectiveness of recovery\. However, standard Full\-Trajectory Distillation \(FTD\) provides demonstrations of these behaviors on teacher\-generated candidates, which can differ in distribution from the student\-generated candidates encountered at inference time\. To address this mismatch, we proposeStudent\-ConditionedVerificationDistillation \(SCVD\)\. In SCVD, the student first produces a candidate solution, after which a stronger teacher continues from the same interaction context and environment state to demonstrate verification and recovery, with supervision applied only to the teacher continuation during fine\-tuning\. Figure[1](https://arxiv.org/html/2609.38812#S1.F1)\(b\) illustrates this process\. Across three Qwen3\.5 backbones, SCVD improvesPass@1on TerminalBench2\.1 by 9\.74–16\.85 percentage points over the corresponding base models and by 4\.49–8\.61 points over task\- and size\-matched FTD, while avoiding the pronounced performance degradation observed with FTD on the out\-of\-distribution SWE\-bench Verified benchmark\([Jimenez et al\., 2024](https://arxiv.org/html/2609.38812#bib.bib26)\)\.

In summary, we make three contributions\. We introduce a diagnostic framework for systematically analyzing self\-verification in terminal agents\. Applying it to ten agents, we find that agents almost always attempt verification, yet many errors still go undetected and many detected errors remain unrepaired\. Motivated by these findings, we propose SCVD, which distills a stronger teacher’s verification and recovery behavior conditioned on student\-generated candidates and consistently outperforms both the corresponding base models and FTD across three Qwen3\.5 backbones\.

## 2Diagnosing Self\-Verification in Terminal Agents

To systematically characterize self\-verification in terminal agents, we examine their behavior after they produce an initial candidate solution\. The first question is whether the agent attempts to verify the candidate \(RQ1: Verification Initiation\)\. If so, how trustworthy are the resulting verification outcomes \(RQ2: Verification Reliability\)? And when verification reveals an error, can the agent successfully correct it \(RQ3: Error Recovery\)? We next formalize this process and introduce a diagnostic framework for assessing these three capabilities\.

### 2\.1Formalizing Solution\-Level Self\-Verification

#### Task and candidate solution\.

We represent a terminal task as𝒯=\(x,ℰ,R⋆\)\\mathcal\{T\}=\(x,\\mathcal\{E\},R^\{\\star\}\), wherexxis the task instruction,ℰ\\mathcal\{E\}is the interactive terminal environment, andR⋆​\(x,s\)∈0,1R^\{\\star\}\(x,s\)\\in\{0,1\}is the official evaluator that determines whether environment statesssatisfies the task\. An agent interaction produces a trajectoryτ=\(o0,a0,o1,…,aT−1,oT\),\\tau=\(o\_\{0\},a\_\{0\},o\_\{1\},\\ldots,a\_\{T\-1\},o\_\{T\}\),whereata\_\{t\}denotes an agent action andoto\_\{t\}the observation\.

To analyze self\-verification at the solution level, we first determine whether the trajectory contains a complete candidate solution\. LetIc∈0,1I\_\{c\}\\in\{0,1\}indicate whether such a candidate exists\. WhenIc=1I\_\{c\}=1, we definetct\_\{c\}as the earliest interaction boundary at which the environment contains a complete solution that could, in principle, be evaluated for task completion\. The corresponding candidate state and its objective correctness are

yc:=R⋆​\(x,stc\),yc∈\{0,1\}\.y\_\{c\}:=R^\{\\star\}\(x,s\_\{t\_\{c\}\}\),\\qquad y\_\{c\}\\in\\\{0,1\\\}\.\(1\)
The candidate boundary partitions the trajectory into the prefix that produces the candidate,τpre=\(o0,a0,…,atc−1,otc\)\\tau\_\{\\mathrm\{pre\}\}=\(o\_\{0\},a\_\{0\},\\ldots,a\_\{t\_\{c\}\-1\},o\_\{t\_\{c\}\}\), and the subsequent interaction,τpost=\(atc,otc\+1,…,aT−1,oT\)\\tau\_\{\\mathrm\{post\}\}=\(a\_\{t\_\{c\}\},o\_\{t\_\{c\}\+1\},\\ldots,a\_\{T\-1\},o\_\{T\}\)\. We usetct\_\{c\}as the reference point for analyzing the agent’s subsequent verification and recovery behavior on the complete candidate withinτpost\\tau\_\{\\mathrm\{post\}\}\.

#### Verification outcome\.

After a complete candidate is formed, the agent may perform task\-relevant checks to assess whether the candidate satisfies the task, for example by running self\-generated tests, inspecting artifacts, or querying system state111Appendix[A](https://arxiv.org/html/2609.38812#A1)provides an annotated example from TerminalBench2\.1\.\. LetIv∈0,1I\_\{v\}\\in\{0,1\}indicate whether such verification occurs inτpost\\tau\_\{\\mathrm\{post\}\}\. WhenIv=1I\_\{v\}=1, we summarize the resulting verification outcome as

z∈𝒵=\{error,no\-error,unresolved\}\.z\\in\\mathcal\{Z\}=\\\{\\text\{error\},\\text\{no\-error\},\\text\{unresolved\}\\\}\.
Here,z=errorz=\\text\{error\}indicates that verification exposes a defect in the candidate,z=no\-errorz=\\text\{no\-error\}indicates that completed verification exposes no defect, andz=unresolvedz=\\text\{unresolved\}indicates that verification does not yield a usable result, for example because the check fails, times out, or the trajectory is truncated\.

Finally, we denote the final task outcome by

yf:=R⋆​\(x,sT\),yf∈\{0,1\}\.y\_\{f\}:=R^\{\\star\}\(x,s\_\{T\}\),\\qquad y\_\{f\}\\in\\\{0,1\\\}\.\(2\)
For trajectories withIc=Iv=1I\_\{c\}=I\_\{v\}=1, the tuple\(yc,z,yf\)\(y\_\{c\},z,y\_\{f\}\)captures three stages of the verification\-and\-recovery process: whether the candidate is objectively correct, what outcome the agent’s own verification produces, and whether the final solution is correct\. Comparingycy\_\{c\}withzzcharacterizes the reliability of verification, whileyfy\_\{f\}reveals whether the agent ultimately recovers from an error exposed during verification\. This formulation forms the basis of our diagnostic framework\.

Table 1:Verification outcomes and diagnostic metrics for solution\-level self\-verification\. Here,ycy\_\{c\}andyfy\_\{f\}denote the correctness of the candidate and final solutions, respectively;zzdenotes the verification outcome; andIcI\_\{c\}andIvI\_\{v\}indicate the presence of a complete candidate and whether verification is attempted\.\(a\) Verification outcome matrix

\(b\) Self\-verification diagnostic metricsRQ1: Verification InitiationRQ2: Verification ReliabilityRQ3: Error Recovery

### 2\.2Diagnostic Framework

Our diagnostic framework consists of three components: trajectory annotation, candidate\-state replay, and diagnostic metrics computed from their outputs\. Trajectory annotation identifies the candidate boundary and characterizes the agent’s subsequent verification behavior, while candidate\-state replay establishes the objective correctness of the candidate being verified\. Based on these quantities and the final task outcome, we define diagnostic metrics for verification initiation, verification reliability, and error recovery, corresponding to RQ1–RQ3\.

#### Trajectory annotation\.

We use LLM judges to annotate each trajectory with the quantities our framework requires\. Given the task instruction and recorded interaction, the judges determine whether a complete candidate is formed \(IcI\_\{c\}\), locate its earliest boundarytct\_\{c\}, determine whether subsequent verification occurs \(IvI\_\{v\}\), and classify the resulting verification outcomezz\. All judges follow the same annotation protocol, with prompt design, judge models, aggregation procedure, and calibration details provided in[SectionB\.1](https://arxiv.org/html/2609.38812#A2.SS1)in the appendix\.

#### Candidate\-state replay\.

For each trajectory withIc=Iv=1I\_\{c\}=I\_\{v\}=1, we initialize a fresh task environment and replay the recorded terminal actions up to the annotated candidate boundarytct\_\{c\}to reconstruct the candidate state\. We then invoke the official evaluatorR⋆R^\{\\star\}on the reconstructed state to obtain its objective correctnessycy\_\{c\}\. Trajectories whose candidate states cannot be faithfully reconstructed are excluded from replay\-based analysis; coverage statistics are reported in[Table4](https://arxiv.org/html/2609.38812#A2.T4)\.

#### Diagnostic metrics\.

UsingIvI\_\{v\},zz,ycy\_\{c\}, andyfy\_\{f\}, we define diagnostic metrics corresponding to RQ1–RQ3\. Table[1](https://arxiv.org/html/2609.38812#S2.T1)\(a\) summarizes the relationship between candidate correctness and verification outcome, while Table[1](https://arxiv.org/html/2609.38812#S2.T1)\(b\) provides the formal definitions of the metrics for verification initiation, verification reliability, and error recovery\. We also report initial candidate accuracy,ICA=Pr⁡\(yc=1\)\\mathrm\{ICA\}=\\Pr\(y\_\{c\}=1\), and final task accuracy to characterize initial candidate quality and overall task success\.

Figure 2:Self\-verification diagnostics across ten terminal agents on TerminalBench2\.1\. Metric definitions are given in Table[1](https://arxiv.org/html/2609.38812#S2.T1)\. \(a\) Diagnostic results across models; gray bars show the range across models, vertical ticks mark the mean for each metric, and the rightmost numbers report corresponding mean values\. \(b\) Improvement from initial candidates to final solutions\. \(c\) Relationship between repair success rate and final task accuracy across models\.

### 2\.3Key Findings

Unless otherwise noted, we first average results over the three runs for each model and then report macro\-averages across models\. Figure[2](https://arxiv.org/html/2609.38812#S2.F2)summarizes the aggregate results, with complete per\-model metrics and pipeline coverage statistics reported in[Table5](https://arxiv.org/html/2609.38812#A2.T5)in the appendix\.

#### Finding 1: Agents almost always initiate verification\.

Under the Terminus\-2 scaffold, agents almost always attempt verification after forming a complete candidate, with VTR averaging99\.53%99\.53\\%and ranging from98\.05%98\.05\\%to100\.0%100\.0\\%across models\. These attempts also almost always yield a usable outcome, with a mean VOR of99\.36%99\.36\\%\. Verification is therefore a near\-universal behavior after a complete candidate is formed, and these attempts usually yield a usable outcome\.222We further discuss the potential influence of the agent scaffold in[SectionB\.3](https://arxiv.org/html/2609.38812#A2.SS3)\.

#### Finding 2: Error signals are usually reliable, but many errors remain undetected\.

When verification exposes an error, that signal is usually correct, with ESP averaging92\.65%92\.65\\%\(85\.4785\.47–98\.69%98\.69\\%\)\. However, EDR averages only61\.43%61\.43\\%\(48\.9248\.92–71\.52%71\.52\\%\), meaning that nearly four in ten incorrect candidates escape detection\. Correct candidates, in turn, usually receive a no\-error signal, with CPR averaging90\.25%90\.25\\%\(86\.7586\.75–93\.11%93\.11\\%\)\. However, many incorrect candidates also receive a no\-error signal, limiting its value as evidence of correctness, with VPR averaging only51\.52%51\.52\\%\.

#### Finding 3: Even detected errors are repaired only about half of the time\.

When verification correctly exposes an erroneous candidate, agents successfully repair it in only49\.36%49\.36\\%of cases on average, with RSR ranging from23\.65%23\.65\\%to75\.68%75\.68\\%\. Detecting an error therefore does not necessarily translate into a correct final solution\. Yet post\-candidate interaction contributes substantially to task success\. As shown in Figure[2](https://arxiv.org/html/2609.38812#S2.F2)\(b\), accuracy increases by14\.614\.6–36\.536\.5percentage points from initial candidates to final solutions across the ten models, with an average gain of25\.425\.4points\. Figure[2](https://arxiv.org/html/2609.38812#S2.F2)\(c\) further shows that RSR is strongly correlated with final task accuracy across models \(Pearsonr=0\.98r=0\.98\), highlighting the importance of successful recovery for overall task performance\.

Diagnostic takeaway\.*Terminal agents almost always verify their work, but many incorrect candidates survive verification and many detected errors remain unrepaired\.*The main weaknesses therefore lie not in initiating verification, but in detecting existing errors and successfully recovering from them\.

## 3Learning to Verify from Student Trajectories

These findings point to verification and recovery after candidate formation as key behaviors to improve\. A natural approach is to distill high\-quality verification and recovery behavior from a stronger teacher\. However, in standard Full\-Trajectory Distillation \(FTD\), the teacher generates both the candidate solution and the subsequent interaction\. As a result, verification and recovery are demonstrated on teacher\-generated candidates, whereas at inference time the student must handle candidates produced by its own policy, which may differ substantially in quality and error patterns\([Lyu et al\., 2025](https://arxiv.org/html/2609.38812#bib.bib20),[Zhao et al\., 2026a](https://arxiv.org/html/2609.38812#bib.bib21)\)\. This creates a mismatch between the states seen during distillation and those on which the student must verify and recover at inference time\. To address this mismatch, we propose a simple approach,Student\-ConditionedVerificationDistillation \(SCVD\), in which the student first produces a candidate solution and a stronger teacher then continues from the same interaction context to verify and repair it\. The overall process is illustrated in Figure[1](https://arxiv.org/html/2609.38812#S1.F1)\(b\)\.

### 3\.1Student\-Conditioned Trajectory Collection

For each task𝒯=\(x,ℰ,R⋆\)\\mathcal\{T\}=\(x,\\mathcal\{E\},R^\{\\star\}\), the student policyπstu\\pi\_\{\\mathrm\{stu\}\}interacts with the environment until it forms a complete candidate at boundarytct\_\{c\}, producing the prefixτprestu\\tau\_\{\\mathrm\{pre\}\}^\{\\mathrm\{stu\}\}\. At this point, we hand control to a stronger teacher policyπtea\\pi\_\{\\mathrm\{tea\}\}, which continues from the same interaction history and environment state\. The candidate is therefore produced entirely by the student, while verification and any subsequent recovery are demonstrated by the teacher\.

At handoff, the scaffold provides the teacher with a collection\-only instructiongverg\_\{\\mathrm\{ver\}\}that directs it to verify the existing candidate using task\-relevant checks, repair any defect it finds, and recheck the result; the full prompt is provided in\. The teacher then produces a continuationτposttea\\tau\_\{\\mathrm\{post\}\}^\{\\mathrm\{tea\}\}, yielding the combined trajectoryτstu→tea=τprestu\|τposttea\\tau^\{\\mathrm\{stu\}\\rightarrow\\mathrm\{tea\}\}=\\tau\_\{\\mathrm\{pre\}\}^\{\\mathrm\{stu\}\}\\\|\\tau\_\{\\mathrm\{post\}\}^\{\\mathrm\{tea\}\}\. SCVD does not require the student candidate to be incorrect\. A correct candidate can provide supervision for how to verify and terminate, while an incorrect candidate can additionally provide a demonstration of error recovery\.

### 3\.2Continuation\-Only Distillation

After each rollout, we evaluate the final task state with the official evaluatorR⋆R^\{\\star\}and retain only trajectories satisfyingR⋆​\(x,sT\)=1R^\{\\star\}\(x,s\_\{T\}\)=1\. This filtering ensures that the distilled continuation ultimately leads to successful task completion\. We then remove the collection\-only instructiongverg\_\{\\mathrm\{ver\}\}so that the resulting context matches what is available at inference time\.

The resulting trajectories form the training set𝒟SC\\mathcal\{D\}\_\{\\mathrm\{SC\}\}\. For each trajectory, the student\-generated prefix is retained as context but masked from the loss, while supervision is applied only to the teacher continuation\. We initializeπθ\\pi\_\{\\theta\}fromπstu\\pi\_\{\\mathrm\{stu\}\}and optimize

ℒ⁡\(θ\)=−𝔼τ∼𝒟SC​\[∑t=tcT−1log⁡πθ​\(attea∣x,τ<tstu→tea\)\]\.\\mathcal\{L\}\(\\theta\)=\-\\mathbb\{E\}\_\{\\tau\\sim\\mathcal\{D\}\_\{\\mathrm\{SC\}\}\}\\left\[\\sum\_\{t=t\_\{c\}\}^\{T\-1\}\\log\\pi\_\{\\theta\}\\\!\\left\(a\_\{t\}^\{\\mathrm\{tea\}\}\\mid x,\\tau\_\{<t\}^\{\\mathrm\{stu\}\\rightarrow\\mathrm\{tea\}\}\\right\)\\right\]\.\(3\)

## 4Experiments

### 4\.1Experimental Setup

#### Models and training data\.

We use Qwen3\.5\-9B, Qwen3\.5\-27B, and Qwen3\.5\-35B\-A3B\([Team, 2026](https://arxiv.org/html/2609.38812#bib.bib1)\)as student backbones and GLM\-5\.2\([Zeng et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib2)\)as the teacher for trajectory collection\. Training tasks are drawn from a pool of 11,806 publicly available terminal tasks from TerminalTraj\-5K\([Wu et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib22)\), Terminal\-Bench\-Env\([Zhu et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib24)\), and SETA\([Shen et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib23)\)\. For each backbone, we retain the intersection of tasks for which both FTD and SCVD produce a verifier\-passing trajectory\. This yields task\- and size\-matched training sets, isolating the effect of trajectory construction from differences in task composition or data volume\. The resulting sets contain 7,419, 7,559, and 7,573 examples for Qwen3\.5\-9B, Qwen3\.5\-27B, and Qwen3\.5\-35B\-A3B, respectively\.

#### Training and evaluation\.

We perform full\-parameter supervised fine\-tuning for three epochs\. All fine\-tuned variants use identical optimization hyperparameters, which are provided in[Table8](https://arxiv.org/html/2609.38812#A3.T8)in the appendix\. We evaluate in\-domain performance on TerminalBench2\.1 and out\-of\-distribution generalization on SWE\-bench Verified\. All models are evaluated in thinking mode, using the Terminus\-2 scaffold for TerminalBench2\.1 and OpenHands for SWE\-bench Verified\. All trajectory\-collection and evaluation rollouts use a temperature of 1\.0, top\-kkof 20, top\-ppof 0\.95, and a maximum generation budget of 32k tokens\. For each setting, we conduct three independent evaluation runs and report meanPass@1with standard deviation; on TerminalBench2\.1, we additionally reportPass@3\.

### 4\.2Main Results

Table 2:Main results on TerminalBench2\.1 and the out\-of\-distribution SWE\-bench Verified benchmark\.Pass@1is the mean±\\pmstandard deviation over three runs, andΔ\\Deltadenotes the absolutePass@1difference from Base\.#### SCVD consistently improves task success\.

As shown in Table[2](https://arxiv.org/html/2609.38812#S4.T2)\(see Table[6](https://arxiv.org/html/2609.38812#A3.T6)for per\-run results\), SCVD improvesPass@1over the corresponding base models by 9\.74, 16\.85, and 11\.61 percentage points on Qwen3\.5\-9B, Qwen3\.5\-27B, and Qwen3\.5\-35B\-A3B, respectively, with an average gain of 12\.73 points\. The improvement is consistent across all three backbones and is also reflected inPass@3, which increases by 14\.61–15\.73 points over Base\.

#### Conditioning on student states is more effective than distilling complete teacher trajectories\.

Under task\- and size\-matched training data, SCVD outperforms FTD by 8\.61, 8\.24, and 4\.49Pass@1points across the three backbones, for an average gain of 7\.12 points\. The difference is most pronounced on Qwen3\.5\-9B, where FTD improves over Base by only 1\.12 points, compared with a 9\.74\-point gain from SCVD\. On Qwen3\.5\-27B, SCVD and FTD achieve the samePass@3of 70\.79%, while SCVD yields an 8\.24\-point higherPass@1, indicating more consistent success across independent runs\.

#### SCVD better preserves out\-of\-distribution performance\.

On SWE\-bench Verified, FTD substantially reducesPass@1relative to the corresponding base models, with drops of 25\.87, 6\.40, and 12\.27 percentage points across the three backbones\. In contrast, SCVD largely preserves the base performance of the 9B model and improves the 27B and 35B\-A3B models by 2\.20 and 4\.27 points, respectively\. Thus, the gains from SCVD on TerminalBench2\.1 do not come with the pronounced OOD degradation observed under FTD on SWE\-bench Verified\.

### 4\.3Where Do SCVD’s Gains Come From?

SCVD substantially improves final task success, but final accuracy alone does not reveal where these gains arise\. To connect post\-candidate behavior to end\-to\-end performance, we decompose final accuracy as follows:

Pr⁡\(yf=1\)⏟Final Accuracy=Pr⁡\(yc=1\)⏟Initial Accuracy\+Pr⁡\(yc=0,yf=1\)⏟W2R: Wrong→Right−Pr⁡\(yc=1,yf=0\)⏟R2W: Right→Wrong\.\\underbrace\{\\Pr\(y\_\{f\}=1\)\}\_\{\\text\{Final Accuracy\}\}=\\underbrace\{\\Pr\(y\_\{c\}=1\)\}\_\{\\text\{Initial Accuracy\}\}\+\\underbrace\{\\Pr\(y\_\{c\}=0,y\_\{f\}=1\)\}\_\{\\text\{W2R: Wrong $\\to$ Right\}\}\-\\underbrace\{\\Pr\(y\_\{c\}=1,y\_\{f\}=0\)\}\_\{\\text\{R2W: Right $\\to$ Wrong\}\}\.\(4\)We defineW2R−R2W\\mathrm\{W2R\}\-\\mathrm\{R2W\}as the*post\-candidate net gain*, which measures the improvement achieved after candidate formation\. We apply the diagnostic pipeline from Section[2\.2](https://arxiv.org/html/2609.38812#S2.SS2)to all three runs of Base, FTD, and SCVD for each backbone\.

Figure 3:Decomposition of final accuracy\. Blue denotes initially correct candidates that remain correct, green denotes successful transitions from an incorrect candidate to a correct final solution, and red denotes harmful transitions from a correct candidate to an incorrect final solution\.As shown in Figure[3](https://arxiv.org/html/2609.38812#S4.F3), SCVD improves both initial candidate quality and post\-candidate outcomes across all three backbones\. Relative to Base, W2R increases by 5\.62, 9\.74, and 8\.61 percentage points, while R2W remains below 2% for SCVD on every backbone\. Consequently, the average post\-candidate net gain increases from 19\.73% under Base to 23\.97% under FTD and 26\.97% under SCVD\. Averaged across the three backbones, SCVD improves initial candidate accuracy by 5\.50 points over Base, compared with a 7\.24\-point increase in post\-candidate net gain\. Thus, SCVD’s overall gains reflect improvements both before and after candidate formation, with a somewhat larger improvement in the post\-candidate stage\.

### 4\.4Does the Prefix Generator Matter?

To isolate the effect of the candidate\-state distribution, we train Qwen3\.5\-9B on four task\-matched datasets that differ only in the prefix generator: Qwen3\.5\-9B itself, Qwen3\.5\-27B, Qwen3\.5\-35B\-A3B, or the GLM\-5\.2 teacher\. In all settings, GLM\-5\.2 generates the continuation and the training loss is applied only to this continuation\. Table[3](https://arxiv.org/html/2609.38812#S4.T3)reports meanPass@1over three runs\.

Table 3:Effect of prefix source on Qwen3\.5\-9B\.The highest meanPass@1is obtained when the prefix is generated by the target student itself, reaching 33\.33%, compared with 31\.46% and 31\.83% for prefixes from the other two Qwen3\.5 models and 29\.21% for teacher\-generated prefixes\. Thus, matching the prefix source to the target student improves meanPass@1by 4\.12 points over the teacher\-prefix setting\. Overall, these results support conditioning teacher supervision on states induced by the target student\.

### 4\.5How Does SCVD Affect Inference Efficiency?

To assess whether SCVD’s accuracy gains come at a higher inference cost, we comparePass@1against agent turns, terminal tool calls, generated tokens, and total token cost\. To avoid confounding efficiency with differences in the tasks solved by each method, we compute cost statistics on the subset of tasks solved at least once by all three methods within each backbone, yielding 27, 43, and 39 shared tasks for 9B, 27B, and 35B\-A3B, respectively\.Pass@1is still evaluated over the full set of 89 tasks\. Detailed statistics are reported in Appendix[C](https://arxiv.org/html/2609.38812#A3)\.

As shown in Figure[4](https://arxiv.org/html/2609.38812#S4.F4), SCVD improvesPass@1while reducing agent turns by 12\.0–35\.6%, terminal tool calls by 1\.5–17\.1%, and total token cost by 0\.3–22\.4% relative to Base across all three backbones\. Generated tokens increase by2\.62\.6–4\.1×4\.1\\times, indicating that SCVD trades longer within\-turn generation for fewer interaction rounds\. Fewer interaction rounds also reduce repeated processing of the accumulated context, helping keep total token cost below or close to Base despite the increase in generated tokens\. Compared with FTD, SCVD achieves higher accuracy at comparable or moderately higher inference cost\. Overall, SCVD improves task success without relying on longer interaction trajectories, instead shifting inference toward fewer but more substantial turns\.

Figure 4:Performance versus inference cost on TerminalBench2\.1\. Costs are averaged over common\-success tasks within each backbone\.

## 5Related Work

#### Self\-verification in interactive agents\.

Prior work has explored how language models can inspect and improve their outputs through model\-generated feedback, external feedback, and explicit training\. Self\-Refine and Reflexion iteratively revise responses using model\-generated critiques or verbal reflections\([Madaan et al\., 2023](https://arxiv.org/html/2609.38812#bib.bib3),[Shinn et al\., 2023](https://arxiv.org/html/2609.38812#bib.bib4)\), while CRITIC and Self\-Debugging ground revision in external tools or execution feedback\([Gou et al\., 2024](https://arxiv.org/html/2609.38812#bib.bib5),[Chen et al\., 2024](https://arxiv.org/html/2609.38812#bib.bib6)\)\. Training\-based methods such as SCoRe and S2R directly optimize multi\-turn verification and revision behavior\([Kumar et al\., 2025](https://arxiv.org/html/2609.38812#bib.bib7),[Ma et al\., 2025](https://arxiv.org/html/2609.38812#bib.bib8)\), while recent work further shows that explicitly learning to self\-verify can improve both verification and reasoning performance\([Chen et al\., 2026a](https://arxiv.org/html/2609.38812#bib.bib29),[Zheng et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib30)\)\. In coding\-agent settings, executable environments further provide tests, compilers, program outputs, and system observations as potential verification signals\. However, agent\-generated tests often provide weak evidence, and current models can struggle to construct tests that distinguish correct implementations from realistic faults\([Chen et al\., 2026b](https://arxiv.org/html/2609.38812#bib.bib9),[Sun et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib10)\)\. Other work strengthens verification with separate verifier or reviewer agents\([Ma et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib11),[Wang et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib12)\), or analyzes trajectory structure and failure evolution over the broader interaction process\([Shu et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib13),[Zhao et al\., 2026b](https://arxiv.org/html/2609.38812#bib.bib14)\)\. We instead focus on the terminal agent’s own solution\-level verification after a complete candidate has been formed, replaying the candidate state and applying the official evaluator to relate the agent’s verification outcome to objective candidate correctness and subsequent recovery\.

#### Distilling interactive agent trajectories\.

Fine\-tuning on expert interaction traces is widely used to transfer planning and tool\-use behavior to smaller agents\([Chen et al\., 2023](https://arxiv.org/html/2609.38812#bib.bib15),[Zeng et al\., 2023](https://arxiv.org/html/2609.38812#bib.bib16)\), with recent methods further structuring supervision over reasoning and action spans\([Liu et al\., 2025](https://arxiv.org/html/2609.38812#bib.bib17)\)\. Full teacher\-trajectory imitation, however, primarily exposes the student to teacher\-induced states, reproducing the distribution\-shift problem of sequential imitation learning\([Ross et al\., 2011](https://arxiv.org/html/2609.38812#bib.bib18)\)\. For terminal agents, interaction structure can also matter more than teacher strength alone\([Yang et al\., 2026](https://arxiv.org/html/2609.38812#bib.bib19)\)\. Related student\-centered and on\-policy distillation methods condition expert supervision on learner\-generated rollouts\([Lyu et al\., 2025](https://arxiv.org/html/2609.38812#bib.bib20),[Zhao et al\., 2026a](https://arxiv.org/html/2609.38812#bib.bib21)\)\. We share the general principle that expert supervision should be conditioned on learner\-induced states, while targeting a distinct aspect of agent behavior: solution\-level self\-verification and recovery\. Our teacher takes over only after the student has formed a complete candidate in the terminal environment, demonstrates solution\-level verification and recovery from that same state, and contributes loss only on the continuation\.

## 6Conclusion

In this work, we studied solution\-level self\-verification in terminal agents by jointly characterizing objective candidate correctness, agents’ own verification outcomes, and final task outcomes\. Our diagnostic analysis shows that agents almost always initiate verification, yet many incorrect candidates remain undetected and only about half of detected errors are successfully repaired\. To address these weaknesses, we introduced Student\-Conditioned Verification Distillation \(SCVD\), which distills a stronger teacher’s verification and recovery behavior on student\-generated candidate solutions\. Across three Qwen3\.5 backbones, SCVD improves task success over both the corresponding base models and Full\-Trajectory Distillation \(FTD\), while avoiding the pronounced out\-of\-distribution degradation observed with FTD\. Taken together, our findings highlight error detection and recovery as central challenges for reliable self\-verification, and show that conditioning teacher supervision on student\-generated candidates can effectively improve these behaviors\.

## AI Use Statement

Generative AI tools were used as part of the research methodology, including LLM\-based trajectory annotation and the generation of teacher verification\-and\-recovery trajectories for training, as described in the paper\. We also used generative AI tools to assist with language editing and improving the clarity and organization of the manuscript\. All AI\-assisted outputs, analyses, and manuscript content were reviewed and verified by the authors\. The authors take full responsibility for the final content of this work\.

## Discussion and Limitations

Our results suggest that post\-candidate interaction is not merely a final consistency check, but an important stage at which agents can still substantially improve their solutions at test time\. The substantial gap between initial candidate accuracy and final task accuracy shows that this stage can materially change task outcomes\. However, realizing this potential requires more than simply performing additional checks: the agent must learn which aspects of a candidate warrant verification, what evidence is genuinely diagnostic, and how that evidence should guide subsequent revision\. How to teach agents to construct and perform checks that provide sufficiently strong evidence of candidate correctness remains an important open question\.

Verification and recovery are also inherently conditioned on the candidate being examined\. What should be checked, which evidence is informative, and what repair is appropriate all depend on the concrete state of the current solution\. This makes the candidate on which verification is demonstrated an important part of the supervision itself\. SCVD reflects this dependency by having the teacher demonstrate verification and recovery on candidates produced by the student rather than on independently generated teacher candidates\. Interestingly, although the training loss is applied only to the teacher continuation, SCVD also improves the accuracy of the student’s initial candidates\. This suggests that learning better verification and recovery behavior may influence earlier problem\-solving decisions rather than affecting only the post\-candidate continuation\. Understanding how such supervision shapes behavior before candidate formation remains an interesting direction for future work\.

Our analysis focuses on solution\-level verification after a complete candidate has been formed\. Agents may also verify partial plans, intermediate tool outputs, and individual decisions throughout a trajectory; understanding these finer\-grained forms of verification and their failure modes remains an important direction for future work\.

## References

- B\. Chen, C\. Shu, E\. Shareghi, N\. Collier, K\. Narasimhan, and S\. YaoFireAct: toward language agent fine\-tuning\.CoRRabs/2310\.05915\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.05915),[Document](https://dx.doi.org/10.48550/ARXIV.2310.05915),2310\.05915Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2024\)X\. Chen, M\. Lin, N\. Schärli, and D\. ZhouTeaching large language models to self\-debug\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=KuPixIqPiq)Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p2.1),[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2026a\)Y\. Chen, Y\. Wang, Y\. Zhang, Z\. Ye, Z\. Cai, Y\. Shi, Q\. Gu, H\. Su, X\. Cai, X\. Wang,et al\.Learning to self\-verify makes language models better reasoners\.arXiv preprint arXiv:2602\.07594\.Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2026b\)Z\. Chen, Z\. Sun, Y\. Shi, C\. Peng, X\. Gu, D\. Lo, and L\. JiangRethinking the value of agent\-generated tests for llm\-based software engineering agents\.CoRRabs/2602\.07900\.External Links:[Link](https://doi.org/10.48550/arXiv.2602.07900),[Document](https://dx.doi.org/10.48550/ARXIV.2602.07900),2602\.07900Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p2.1),[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Gouet al\.\(2024\)Z\. Gou, Z\. Shao, Y\. Gong, Y\. Shen, Y\. Yang, N\. Duan, and W\. ChenCRITIC: large language models can self\-correct with tool\-interactive critiquing\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=Sx038qxjek)Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p2.1),[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. NarasimhanSwe\-bench: can language models resolve real\-world github issues?\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 54107–54157\.Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p5.1)\.
- Kumaret al\.\(2025\)A\. Kumar, V\. Zhuang, R\. Agarwal, Y\. Su, J\. D\. Co\-Reyes, A\. Singh, K\. Baumli, S\. Iqbal, C\. Bishop, R\. Roelofs, L\. M\. Zhang, K\. McKinney, D\. Shrivastava, C\. Paduraru, G\. Tucker, D\. Precup, F\. M\. P\. Behbahani, and A\. FaustTraining language models to self\-correct via reinforcement learning\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=CjwERcAU7w)Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2026\)C\. Li, Y\. Xu, Z\. Wang, S\. H\. Tan,et al\.Independent patch verification for coding agents with a bidirectional reconstruct\-and\-verify framework\.arXiv preprint arXiv:2608\.08950\.Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p2.1)\.
- Liuet al\.\(2025\)J\. Liu, Z\. Kong, P\. Dong, C\. Yang, T\. Li, H\. Tang, G\. Yuan, W\. Niu, W\. Zhang, P\. Zhao, X\. Lin, D\. Huang, and Y\. WangStructured agent distillation for large language model\.CoRRabs/2505\.13820\.External Links:[Link](https://doi.org/10.48550/arXiv.2505.13820),[Document](https://dx.doi.org/10.48550/ARXIV.2505.13820),2505\.13820Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px2.p1.1)\.
- Lyuet al\.\(2025\)Y\. Lyu, C\. Wang, J\. Huang, and T\. XuStudent\-centered distillation narrows the agentic gap between small and large llms\.CoRRabs/2509\.14257\.External Links:[Link](https://doi.org/10.48550/arXiv.2509.14257),[Document](https://dx.doi.org/10.48550/ARXIV.2509.14257),2509\.14257Cited by:[§3](https://arxiv.org/html/2609.38812#S3.p1.1),[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px2.p1.1)\.
- Maet al\.\(2025\)R\. Ma, P\. Wang, C\. Liu, X\. Liu, J\. Chen, B\. Zhang, X\. Zhou, N\. Du, and J\. LiS2\{\}^\{\\mbox\{2\}\}r: teaching llms to self\-verify and self\-correct via reinforcement learning\.CoRRabs/2502\.12853\.External Links:[Link](https://doi.org/10.48550/arXiv.2502.12853),[Document](https://dx.doi.org/10.48550/ARXIV.2502.12853),2502\.12853Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Maet al\.\(2026\)Z\. Ma, J\. Zhang, X\. Zhang, J\. Yang, Z\. Zhang, J\. Zhang, Y\. Jing, L\. Zhang, H\. Zheng, W\. Zhao, J\. Lin, and B\. HuiScaling agentic verifier for competitive coding\.CoRRabs/2602\.04254\.External Links:[Link](https://doi.org/10.48550/arXiv.2602.04254),[Document](https://dx.doi.org/10.48550/ARXIV.2602.04254),2602\.04254Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. ClarkSelf\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/91edff07232fb1b55a505a9e9f6c0ff3-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Merrillet al\.\(2026\)M\. A\. Merrill, A\. G\. Shaw, N\. Carlini, B\. Li, H\. Raj, I\. Bercovich, L\. Shi, J\. Y\. Shin, T\. Walshe, E\. K\. Buchanan, J\. Shen, G\. Ye, H\. Lin, J\. Poulos, M\. Wang, M\. Nezhurina, J\. Jitsev, D\. Lu, O\. M\. Mastromichalakis, Z\. Xu, Z\. Chen, Y\. Liu, R\. Zhang, L\. L\. Chen, A\. Kashyap, J\. Uslu, J\. Li, J\. Wu, M\. Yan, S\. Bian, V\. Sharma, K\. Sun, S\. Dillmann, A\. Anand, A\. Lanpouthakoun, B\. Koopah, C\. Hu, E\. Guha, G\. H\. S\. Dreiman, J\. Zhu, K\. Krauth, L\. Zhong, N\. Muennighoff, R\. Amanfu, S\. Tan, S\. Pimpalgaonkar, T\. Aggarwal, X\. Lin, X\. Lan, X\. Zhao, Y\. Liang, Y\. Wang, Z\. Wang, C\. Zhou, D\. Heineman, H\. Liu, H\. Trivedi, J\. Yang, J\. Lin, M\. Shetty, M\. Yang, N\. Omi, N\. Raoof, S\. Li, T\. Y\. Zhuo, W\. Lin, Y\. Dai, Y\. Wang, W\. Chai, S\. Zhou, D\. Wahdany, Z\. She, J\. Hu, Z\. Dong, Y\. Zhu, S\. Cui, A\. Saiyed, A\. Kolbeinsson, J\. Hu, C\. M\. Rytting, R\. Marten, Y\. Wang, A\. Dimakis, A\. Konwinski, and L\. SchmidtTerminal\-bench: benchmarking agents on hard, realistic tasks in command line interfaces\.External Links:2601\.11868,[Link](https://arxiv.org/abs/2601.11868)Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p4.1)\.
- Rosset al\.\(2011\)S\. Ross, G\. J\. Gordon, and D\. BagnellA reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2011, Fort Lauderdale, USA, April 11\-13, 2011,G\. J\. Gordon, D\. B\. Dunson, and M\. Dudík \(Eds\.\),JMLR Proceedings, Vol\.15,pp\. 627–635\.External Links:[Link](http://proceedings.mlr.press/v15/ross11a/ross11a.pdf)Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px2.p1.1)\.
- Shenet al\.\(2026\)Q\. Shen, Z\. Huang, V\. Kamanuru, A\. Aliev, J\. Rainton, A\. Awelkair, Z\. Zeng, J\. Li, S\. Dong, Y\. Yuan,et al\.SETA: scaling environments for terminal agents\.arXiv preprint arXiv:2607\.10891\.Cited by:[§4\.1](https://arxiv.org/html/2609.38812#S4.SS1.SSS0.Px1.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Shuet al\.\(2026\)R\. Shu, C\. Y\. Chong, X\. Zhou, Y\. Peng, Z\. Wu, X\. Han, Z\. Zhuang, G\. Yuan, and Y\. WangWhat resolve rate hides: trajectory structure diagnostics for coding agents\.CoRRabs/2607\.06184\.External Links:[Link](https://doi.org/10.48550/arXiv.2607.06184),[Document](https://dx.doi.org/10.48550/ARXIV.2607.06184),2607\.06184Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Sunet al\.\(2026\)Y\. Sun, Y\. Zhao, Y\. Wang, Y\. Du, Z\. Ma, J\. Wang, M\. Zhang, K\. Zhang, and Z\. HuangSWE\-mutation: can llms generate reliable test suites in software engineering?\.InFindings of the Association for Computational Linguistics, ACL 2026, San Diego, California, United States, July 2\-7, 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),pp\. 39651–39674\.External Links:[Link](https://doi.org/10.18653/v1/2026.findings-acl.1976),[Document](https://dx.doi.org/10.18653/V1/2026.FINDINGS-ACL.1976)Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p2.1),[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Tanet al\.\(2026\)Y\. Tan, Z\. Yang, F\. Yang, Y\. Yao, Y\. Kang, L\. Wang, P\. Zhao, X\. Zhang, X\. Ma, Q\. Lin,et al\.Beyond fail\-to\-pass: iterative hardening of co\-generated bug reproduction tests and fixes\.arXiv preprint arXiv:2607\.19843\.Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p2.1)\.
- Team \(2026\)Q\. TeamQwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[§4\.1](https://arxiv.org/html/2609.38812#S4.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2026\)R\. Wang, J\. Chen, S\. Wang, C\. Tao, S\. Yang, Y\. Jiang, K\. Yap, L\. Shang, X\. Li, and H\. BaiSWE\-review: closing the loop on issue resolution with agentic code review\.CoRRabs/2607\.06065\.External Links:[Link](https://doi.org/10.48550/arXiv.2607.06065),[Document](https://dx.doi.org/10.48550/ARXIV.2607.06065),2607\.06065Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Wuet al\.\(2026\)S\. Wu, Y\. Li, Y\. Song, W\. Zhang, Y\. Wang, R\. Batista\-Navarro, X\. Yang, M\. Tang, B\. Dai, J\. Yang,et al\.Large\-scale terminal agentic trajectory generation from dockerized environments\.arXiv preprint arXiv:2602\.01244\.Cited by:[§4\.1](https://arxiv.org/html/2609.38812#S4.SS1.SSS0.Px1.p1.1)\.
- Yanget al\.\(2026\)S\. Yang, C\. Tao, J\. Chen, T\. Yu, R\. Wang, Y\. Jiang, Y\. Du, W\. Xu, J\. Xiong, T\. Wu, L\. Shang, X\. Li, N\. Wong, and H\. BaiWhat makes interaction trajectories effective for training terminal agents?\.CoRRabs/2606\.03461\.External Links:[Link](https://doi.org/10.48550/arXiv.2606.03461),[Document](https://dx.doi.org/10.48550/ARXIV.2606.03461),2606\.03461Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px2.p1.1)\.
- Zenget al\.\(2023\)A\. Zeng, M\. Liu, R\. Lu, B\. Wang, X\. Liu, Y\. Dong, and J\. TangAgentTuning: enabling generalized agent abilities for llms\.CoRRabs/2310\.12823\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.12823),[Document](https://dx.doi.org/10.48550/ARXIV.2310.12823),2310\.12823Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px2.p1.1)\.
- Zenget al\.\(2026\)A\. Zeng, X\. Lv, Z\. Hou, Z\. Du, Q\. Zheng, B\. Chen, D\. Yin, C\. Ge, C\. Huang, C\. Xie,et al\.Glm\-5: from vibe coding to agentic engineering\.arXiv preprint arXiv:2602\.15763\.Cited by:[§4\.1](https://arxiv.org/html/2609.38812#S4.SS1.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2026a\)S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. GroverSelf\-distilled reasoner: on\-policy self\-distillation for large language models\.CoRRabs/2601\.18734\.External Links:[Link](https://doi.org/10.48550/arXiv.2601.18734),[Document](https://dx.doi.org/10.48550/ARXIV.2601.18734),2601\.18734Cited by:[§3](https://arxiv.org/html/2609.38812#S3.p1.1),[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2026b\)X\. Zhao, H\. Li, S\. Li, T\. Zhao, E\. T\. Barr, F\. Sarro, and H\. YeFailure as a process: an anatomy of CLI coding agent trajectories\.CoRRabs/2607\.09510\.External Links:[Link](https://doi.org/10.48550/arXiv.2607.09510),[Document](https://dx.doi.org/10.48550/ARXIV.2607.09510),2607\.09510Cited by:[§1](https://arxiv.org/html/2609.38812#S1.p2.1),[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Zhenget al\.\(2026\)T\. Zheng, X\. Wu, Z\. Zhang, Z\. He, C\. Zhang, B\. Coleman, R\. Wei, D\. Bai, H\. Liu, R\. Liu, X\. Wang, Y\. Zhuan, W\. Kang, R\. Xiang, H\. Huang, X\. Cheng, and Y\. GuoDream\-rsi: recursive self\-improvement through evolving worlds\.arXiv preprint arXiv:2609\.14858\.Cited by:[§5](https://arxiv.org/html/2609.38812#S5.SS0.SSS0.Px1.p1.1)\.
- Zhuet al\.\(2026\)K\. Zhu, Y\. Nie, Y\. Li, Y\. Huang, J\. Wu, J\. Liu, X\. Sun, Z\. Yin, L\. Wang, Z\. Liu,et al\.Termigen: high\-fidelity environment and robust trajectory synthesis for terminal agents\.arXiv preprint arXiv:2602\.07274\.Cited by:[§4\.1](https://arxiv.org/html/2609.38812#S4.SS1.SSS0.Px1.p1.1)\.

## Appendix AExample of Solution\-Level Self\-Verification

We show an abridged but otherwise verbatim GLM\-5\.2 trajectory from the TerminalBench2\.1 task prove\-plus\-comm\. The excerpt starts when the agent constructs its first complete candidate and retains the model’s original content, tool calls, and terminal feedback\. Bracketed ellipses mark omitted unchanged lines or shell output\.

Gray, blue, and cyan sections reproduce the task and agent trajectory\. Orange and green sections are post\-hoc annotations added by us; they were not part of the trajectory or visible to the agent\.

A Terminal\-Agent Self\-Verification TraceTaskFix the incomplete proof of addition commutativity inplus\_comm\.v\. Add the missing Coq tactics, compile the proof withcoqc, and produceplus\_comm\.vo\.Agent content: constructing the initial candidateAnalysis:The fileplus\_comm\.vcontains an incomplete proof with twoadmitplaceholders: one for the base case and one for the inductive step\. We need replace those placeholders with valid Coq tactics and compile the file withcoqcto generateplus\_comm\.vo\.Plan:I will rewriteplus\_comm\.vwith a completed proof\. For the base case, I will use induction onmto provem = m \+ 0\. For the inductive step, I will rewrite by the inductive hypothesis and then useplus\_n\_Smto finish\. After editing, I will compile the file withcoqc\.[⬇](data:text/plain;base64,Y2F0ID4gcGx1c19jb21tLnYgPDwnRU9GJwpSZXF1aXJlIEltcG9ydCBBcml0aC4KClRoZW9yZW0gcGx1c19jb21tIDogZm9yYWxsIG4gbSA6IG5hdCwKICBuICsgbSA9IG0gKyBuLgpQcm9vZi4KICBpbnRyb3MgbiBtLgogIGluZHVjdGlvbiBuIGFzIFt8bicgSUhuJ10uCiAgLSBzaW1wbC4KICAgIGluZHVjdGlvbiBtIGFzIFt8bScgSUhtJ10uCiAgICArIHJlZmxleGl2aXR5LgogICAgKyBzaW1wbC4KICAgICAgcmV3cml0ZSBJSG0nLgogICAgICByZWZsZXhpdml0eS4KICAtIHNpbXBsLgogICAgcmV3cml0ZSBJSG4nLgogICAgcmV3cml0ZSBwbHVzX25fU20uCiAgICByZWZsZXhpdml0eS4KUWVkLgpFT0Y=)cat\>plus\_comm\.v<<’EOF’RequireImportArith\.Theoremplus\_comm:forallnm:nat,n\+m=m\+n\.Proof\.introsnm\.inductionnas\[\|n’IHn’\]\.\-simpl\.inductionmas\[\|m’IHm’\]\.\+reflexivity\.\+simpl\.rewriteIHm’\.reflexivity\.\-simpl\.rewriteIHn’\.rewriteplus\_n\_Sm\.reflexivity\.Qed\.EOFPost\-hoc annotation: Candidate boundaryThe proof now contains no placeholders and constitutes a complete, submit\-ready candidate\. We therefore placetct\_\{c\}immediately after this tool call\. Replaying the official evaluator at this point yieldsyc=0y\_\{c\}=0; this information is used only for post\-hoc diagnosis and is unavailable to the agent\.[⬇](data:text/plain;base64,Y29xYyBwbHVzX2NvbW0udgpscyAtbGEgcGx1c19jb21tLnYgcGx1c19jb21tLnZv)coqcplus\_comm\.vls\-laplus\_comm\.vplus\_comm\.vo[⬇](data:text/plain;base64,RmlsZSAiLi9wbHVzX2NvbW0udiIsIGxpbmUgMTMsIGNoYXJhY3RlcnMgNi0xNzoKRXJyb3I6IEluIGVudmlyb25tZW50Cm0nIDogbmF0CklIbScgOiBtJyA9IG0nICsgMApVbmFibGUgdG8gdW5pZnkgIlMgKG0nICsgMCArIDApIiB3aXRoICJTIChtJyArIDApIi4KCmxzOiBjYW5ub3QgYWNjZXNzICdwbHVzX2NvbW0udm8nOiBObyBzdWNoIGZpbGUgb3IgZGlyZWN0b3J5)File”\./plus\_comm\.v”,line13,characters6\-17:Error:Inenvironmentm’:natIHm’:m’=m’\+0Unabletounify”S\(m’\+0\+0\)”with”S\(m’\+0\)”\.ls:cannotaccess’plus\_comm\.vo’:NosuchfileordirectoryPost\-hoc annotation: Error exposedThe compiler is a task\-relevant check executed after the complete candidate has been formed\. Its feedback identifies a concrete proof failure, so the verification outcome isz=errorz=\\text\{error\}\.Agent content: interpreting the feedback and planning a repairAnalysis:The previous proof failed because the base case usedrewrite IHm’in the wrong direction, causing an invalid goal\. The inductive step was likely fine, but compilation stopped at the base case error\. We need fix the base case so thatS m’ = S \(m’ \+ 0\)reduces tom’ = m’ \+ 0, which is exactly the induction hypothesis\.Plan:Rewriteplus\_comm\.vwith a corrected base case usingf\_equalto reduce the successor equality, then compile withcoqcand list the generated\.vofile to confirm success\.[⬇](data:text/plain;base64,Y2F0ID4gcGx1c19jb21tLnYgPDwnRU9GJwpSZXF1aXJlIEltcG9ydCBBcml0aC4KClRoZW9yZW0gcGx1c19jb21tIDogZm9yYWxsIG4gbSA6IG5hdCwKICBuICsgbSA9IG0gKyBuLgpQcm9vZi4KICBpbnRyb3MgbiBtLgogIGluZHVjdGlvbiBuIGFzIFt8bicgSUhuJ10uCiAgLSBzaW1wbC4KICAgIGluZHVjdGlvbiBtIGFzIFt8bScgSUhtJ10uCiAgICArIHJlZmxleGl2aXR5LgogICAgKyBzaW1wbC4KICAgICAgZl9lcXVhbC4KICAgICAgZXhhY3QgSUhtJy4KICAtIHNpbXBsLgogICAgcmV3cml0ZSBJSG4nLgogICAgcmV3cml0ZSBwbHVzX25fU20uCiAgICByZWZsZXhpdml0eS4KUWVkLgpFT0YKCmNvcWMgcGx1c19jb21tLnY7IGVjaG8gImV4aXQ9XCQ/IjsgbHMgLWxhIHBsdXNfY29tbS52IHBsdXNfY29tbS52bw==)cat\>plus\_comm\.v<<’EOF’RequireImportArith\.Theoremplus\_comm:forallnm:nat,n\+m=m\+n\.Proof\.introsnm\.inductionnas\[\|n’IHn’\]\.\-simpl\.inductionmas\[\|m’IHm’\]\.\+reflexivity\.\+simpl\.f\_equal\.exactIHm’\.\-simpl\.rewriteIHn’\.rewriteplus\_n\_Sm\.reflexivity\.Qed\.EOFcoqcplus\_comm\.v;echo”exit=\\$?”;ls\-laplus\_comm\.vplus\_comm\.vo[⬇](data:text/plain;base64,ZXhpdD0wCi1ydy1yLS1yLS0gWy4uLl0gcGx1c19jb21tLnYKLXJ3LXItLXItLSBbLi4uXSBwbHVzX2NvbW0udm8=)exit=0\-rw\-r–r–\[…\]plus\_comm\.v\-rw\-r–r–\[…\]plus\_comm\.voAgent content: stopping decisionAnalysis:The proof inplus\_comm\.vhas been completed and successfully compiled withcoqc\. The output showsexit=0and the compiled artifactplus\_comm\.voexists\.Plan:No further commands are needed; the task is complete\.Post\-hoc diagnosis: Successful verification–repair trajectoryThe final official evaluation passes, givingyf=1y\_\{f\}=1\. The complete diagnostic state is therefore\(yc,z,yf\)=\(0,error,1\)\.\(y\_\{c\},z,y\_\{f\}\)=\(0,\\text\{error\},1\)\.The trace directly shows how a self\-generated, task\-relevant check converts an incorrect candidate into actionable feedback and supports a correct stopping decision after repair\.

## Appendix BDiagnostic Details and Full Results

### B\.1Annotation Protocol

#### Judge setup\.

For each model, we collect three independent runs on all 89 tasks in TerminalBench2\.1 using the Terminus\-2 agent scaffold, yielding 267 trajectories per model\. We annotate each trajectory independently with three LLM judges: DeepSeek\-V4\-Pro\-0813, GLM\-5\.2, and Kimi\-K3\. All judges use the same annotation prompt with thinking enabled and temperature set to zero\. They observe the task and recorded trajectory but not the official task reward, and are instructed to identify the agent’s behavior\. See the complete[annotation prompt](https://arxiv.org/html/2609.38812#prompt..annotation), with typographic formatting added for readability\.

To validate the reliability of this annotation procedure, we construct a held\-out calibration set of 20 trajectories whose annotations are manually reviewed and corrected\. Against these human\-validated labels, DeepSeek\-V4\-Pro\-0813, GLM\-5\.2, and Kimi\-K3 achieve joint accuracies of 86\.67%, 76\.67%, and 83\.33%, respectively, across candidate formation, verification triggering, and verification outcome\. These results provide empirical support for the reliability of our annotation pipeline at scale\.

#### Candidate\-boundary annotation\.

We locate the earliest complete candidate at the level of individual tool calls rather than whole agent steps\. This distinction is important because a single agent step may contain multiple tool calls, and the candidate may become complete after one call while a later call in the same step performs the first verification\. Accordingly, when a candidate is formed, the annotation records both the enclosingstep\_idand the zero\-based index of the exact tool call after which the candidate first becomes materially complete\. During replay, we replay the trajectory up to and including the annotated tool call, rather than replaying the entire enclosing agent step\.

#### Aggregation and agreement\.

We vote separately on the candidate\-boundary tuple\(Ic,candidate\_step,candidate\_tool\_call\_index,boundary\_type\)\(I\_\{c\},\\texttt\{candidate\\\_step\},\\\\ \\texttt\{candidate\\\_tool\\\_call\\\_index\},\\texttt\{boundary\\\_type\}\)and the verification\-semantics tuple\(Ic,Iv,z\)\(I\_\{c\},I\_\{v\},z\)\. An exact agreement between at least two judges determines each tuple\. If either tuple has no majority, we use the complete DeepSeek\-V4\-Pro\-0813 annotation, the strongest judge on semantic calibration, as the tie\-breaker\. Among the 2,669 available trajectories, all three annotations are available for 2,641\. Three\-way exact agreement is 74\.29% for the candidate\-boundary tuple and 85\.16% for the verification\-semantics tuple\. The grouped vote resolves 2,563 of 2,669 trajectories \(96\.03%\); the remaining 106 trajectories \(3\.97%\) use the tie\-breaker\.

### B\.2Candidate\-State Replay

For each trajectory withIc=Iv=1I\_\{c\}=I\_\{v\}=1, we initialize a fresh task environment and replay the recorded terminal actions through the annotated candidate boundary, including exactly the tool call identified bycandidate\_stepandcandidate\_tool\_call\_index\. We then invoke the official evaluator on the reconstructed candidate state to obtainycy\_\{c\}\. Failed replays and invalid evaluator outputs are excluded\. Overall, 2,498 of 2,549 submitted replays \(98\.00%\) produce valid labels; per\-model coverage is reported in Table[4](https://arxiv.org/html/2609.38812#A2.T4)\.

Table 4:Coverage of candidate formation and candidate\-state replay on TerminalBench2\.1\. Entries report count/denominator \(percentage\), pooled across three independent runs\. One Opus\-4\.8 trajectory was unavailable\.Table 5:Per\-model self\-verification diagnostics on TerminalBench2\.1\. All values are percentages averaged over three independent runs\. Initial Acc\. \(ICA\) and Final Acc\. denote accuracy at candidate formation and at the end of the trajectory, respectively; the remaining metrics are defined in Section[2\.2](https://arxiv.org/html/2609.38812#S2.SS2)\.
### B\.3Dependence on the Agent Scaffold

All diagnostic experiments use Terminus\-2333[https://github\.com/harbor\-framework/harbor/tree/main/src/harbor/agents/terminus\_2](https://github.com/harbor-framework/harbor/tree/main/src/harbor/agents/terminus_2)as the shared agent scaffold, so the near\-universal verification rate in Finding 1 should be interpreted in the context of this scaffold\. Terminus\-2 provides a relatively minimal interaction protocol: the agent operates through a single interactivetmux\-based terminal interface and, at each turn, produces an analysis, a plan, and a sequence of shell commands\. Importantly, its base prompt does not explicitly instruct the model to verify a completed solution, run tests before submission, inspect generated artifacts, or follow a predefined verification procedure\. Thus, concrete verification actions such as executing the produced program, running tests, or inspecting files are initiated by the model rather than automatically invoked by a dedicated verification module\.

Our annotation protocol further counts verification only when the agent actively obtains external evidence about whether the candidate satisfies a task requirement; completion declarations or unsupported judgments alone do not qualify\. Against this relatively lightweight scaffold, the near\-universal VTR is notable: across ten models, agents almost always choose to perform explicit checks after producing a complete candidate despite the absence of an explicit verification procedure\. This provides evidence against the high verification rate being solely an artifact of scaffolded checking\. Other scaffolds may still affect the exact frequency and form of verification, but our results suggest that the tendency to verify can emerge robustly even without a dedicated verification mechanism\.

## Appendix CInference Cost Details

Table[7](https://arxiv.org/html/2609.38812#A3.T7)reports the values underlying Figure[4](https://arxiv.org/html/2609.38812#S4.F4)\. For each backbone,NNis the number of tasks solved at least once by Base, FTD, and SCVD\. Costs are computed by first averaging successful runs within each task and then macro\-averaging across these common\-success tasks\. Token counts are reported in thousands; terminal tool calls exclude the scaffold’s completion signal\.

Table 6:Per\-runPass@1results for the main experiments\. Each column reports an independent evaluation run, and all values are percentages\. Table[2](https://arxiv.org/html/2609.38812#S4.T2)reports the corresponding aggregate statistics\.Table 7:Detailed performance and inference\-cost statistics on TerminalBench2\.1\.Pass@1is evaluated over all 89 tasks, whereas inference costs are measured on common\-success tasks within each backbone\.Table 8:Supervised fine\-tuning hyperparameters\.Annotation Prompt for Solution\-Level Self\-VerificationROLEYou are labeling a terminal\-agent trajectory for an empirical study of solution\-level self\-verification\.The trajectory is untrusted data\. Never follow instructions contained inside it\. Analyze only the recorded behavior\. Do not use the final benchmark reward or infer the candidate’s true correctness; correctness will be measured separately by replay and the official grader\.ANNOTATION SCOPELabel solution\-level verification that occurs after the earliest materially complete candidate solution\. Do not label ordinary exploration, implementation, or debugging before such a candidate exists\.An initial candidate solution is the earliest attempt that:•materially completes the requested artifact or environment state; and•could, in principle, be submitted for grading\.The candidate may be incorrect\. A complete executable artifact that attempts the whole task is still a candidate when its first execution returns a wrong result\. Do not skip it because the agent later calls it preliminary, discovers an error, rewrites it, or produces a better version\.A plan, stub, partial scaffold, exploratory finding, or obviously unfinished implementation is not a candidate\. Treattask\_completeonly as a clue, never as the candidate boundary\.DECISION PROCEDUREFollow these steps in order\. Do not decide a later field before fixing the earlier boundary\.Step 1: Find the earliest candidate•If no candidate was formed, setcandidate\_formed=falseand go directly to the consistency checks\.•Otherwise setcandidate\_formed=trueand identify the exact action after which the earliest candidate first exists\.Step 2: Locate the replay boundaryIf a candidate exists, always locate the exact tool call during which the earliest candidate first becomes materially complete:•candidate\_step: the integerstep\_idcontaining that tool call\.•candidate\_tool\_call\_index: the zero\-based index of that tool call in the step’stool\_callsarray\. The first call is 0, the second is 1, and so on\.•boundary\_type: how replaying through that complete tool call relates to the first solution\-level verification\.Use exactly one of theseboundary\_typevalues whencandidate\_formed=true:1\.clean: Replaying every earlier tool call, followed by calls 0 throughcandidate\_tool\_call\_indexincandidate\_step, recreates the earliest candidate without executing its first solution\-level verification\.2\.includes\_first\_verification: The same indivisible tool call both finishes the earliest candidate and performs its first solution\-level verification\. Replaying through the indexed call includes that verification, but the call does not modify or replace the candidate after obtaining the verification result\.Boundary rules:•If construction and the first check are separate tool calls in one agent step, select the construction call\. Sharing a step does not make the boundary invalid\.•Treat each recorded tool call as indivisible\. Onebash\_commandtool call may contain several shell commands in its keystrokes\. If the same tool call both finishes and verifies the candidate, record that tool call and use anincludes\_\*boundary type\. Never invent a boundary between shell commands inside one tool call\.•If no candidate exists, setcandidate\_step,candidate\_tool\_call\_index, andboundary\_typetonull\.•If a candidate exists, all three boundary fields must be non\-null\. Never usenullmerely because the exact boundary lies inside one tool call\.•Never invent a step or index\. The step must be an agent step in the trajectory and the index must identify an existing tool call in that step\.•Do not infercandidate\_stepfrom atool\_call\_id; tool\-call IDs need not encode their enclosing ATIF step\. Use only the explicitstep\_idof the object whosetool\_callsarray contains the selected call\.Step 3: Decide whether verification was triggeredSetverification\_triggered=trueonly if, after the candidate exists, the agent actively obtains external evidence about whether it satisfies at least one final task requirement\.Count as verification:•running tests or assertions;•executing the produced program on a concrete input;•compiling or linting the final artifact;•querying the configured service or endpoint;•checking the requested file, process, service, or environment state; and•deliberately usingcat,head,ls,grep,stat, or a similar command to inspect a produced artifact or expected property\.A narrow check still counts as verification of the property it checks\. Incidental shell output while creating a file is not a separate check\.Do not count:•commands used only to understand the task or untouched environment;•implementation and debugging before the candidate exists;•unsupported statements such as*looks correct*;•task\_completecalls or confirmation prompts by themselves\.Step 4: Isolate the first verification phaseStart at the first post\-candidate verification action\. Include consecutive checks of that same candidate\. Stop immediately before the first repair or modification caused by the verification evidence\.Classify only the actual command output or environment feedback in this phase\. Do not use later verification of a repaired candidate\.Step 5: Classifyverification\_outcomeUse exactly one of these labels:1\.exposes\_error: At least one check yields clear negative or unexpected evidence about the candidate, including a failed assertion, wrong output, candidate\-caused build failure, missing required artifact, or visible violation of a concrete task requirement\. Negative evidence does not require a nonzero exit code or explicit error message\. Ifcat,head,grep, a printed summary, or another successful inspection visibly contradicts a task requirement, useexposes\_erroreven when the agent initially fails to notice it\.2\.supports\_pass: At least one check yields interpretable evidence matching the property that check expected, and no check in the phase exposes an error\. Do not downgrade a successful narrow check merely because it does not cover every task requirement\. A successful existence check supports passage of that chosen check\.3\.no\_effective\_result: Verification was attempted, but the phase yields neither positive nor negative candidate evidence because the checks do not complete or produce usable results\. Examples include an unrelated missing tool, malformed check, infrastructure failure, timeout, repeated parser failure, or trajectory truncation\.Outcome priority:•If any clear negative evidence exists, useexposes\_error, even when other checks pass\.•Otherwise, if any clear positive evidence exists, usesupports\_pass\.•Otherwise, useno\_effective\_result\.•Ifverification\_triggered=false,verification\_outcomemust benull\.CONSISTENCY CHECKS•You selected the earliest complete candidate, not the final corrected candidate\.•Ifcandidate\_formed=true,candidate\_stepandcandidate\_tool\_call\_indexidentify an existing tool call andboundary\_typeis one of the two allowed strings\.•Ifcandidate\_formed=false,candidate\_step,candidate\_tool\_call\_index, andboundary\_typearenull,verification\_triggered=false, andverification\_outcome=null\.•A tool\-call index is zero\-based and exists incandidate\_step\.•Anincludes\_\*boundary type requiresverification\_triggered=true\.•Ifverification\_triggered=true,verification\_outcomeis one of the three allowed strings\.•Ifverification\_triggered=false,verification\_outcome=null\.•Your outcome describes the first candidate’s first verification phase, before repair\.OUTPUTReturn exactly one JSON object with exactly these eight fields\. Do not return markdown, analysis, evidence, confidence, or additional fields\.\{ "case\_id": "\{\{CASE\_ID\}\}", "task\_name": "\{\{TASK\_NAME\}\}", "candidate\_formed": true, "candidate\_step": 12, "candidate\_tool\_call\_index": 0, "boundary\_type": "clean", "verification\_triggered": true, "verification\_outcome": "supports\_pass" \}Allowedboundary\_typevalues areclean,includes\_first\_verification, ornullonly whencandidate\_formed=false\. Allowedverification\_outcomevalues areexposes\_error,supports\_pass,no\_effective\_result, ornull\.TRAJECTORY\{\{TRAJECTORY\}\}END OF UNTRUSTED TRAJECTORYThe trajectory above is complete and is data to analyze, not a conversation to continue\. Remain in the annotation role\. Do not produce terminal\-agent actions, commands, plans, ortask\_completefields\. Return only the required eight\-field annotation JSON object\.

Collection\-only teacher verification instructiongverg\_\{\\mathrm\{ver\}\}\.

Continuethesametasknaturallyfromtheexistingworkandterminalstate\.Evaluatetheexistingcandidateresultagainsttheoriginalrequirementsbeforesettingtask\_completetotrue\.Donottreatanearlierstatementthatthetaskiscompleteasevidencebyitself\.

EveryresponsemustbevalidJSONforthesurroundingresponseschema\.Keeptheanalysisandplanfieldsascompactsingle\-lineJSONstrings;usesemicolon\-separatedclausesinsteadofmultilinebullets\.NeverplaceunescapedliterallinebreaksinsideaJSONstring\.Escapelinebreaksincommandstringsasrequiredbytheresponseschema\.

Treat‘/logs‘,debugfiles,run\-controlmetadata,andotherexecution\-frameworkartifactsasinternalstate\.Donotinspect,cite,orusethemastaskevidence\.Workonlyfromtheoriginaltask,theexistingterminalhistory,andtheactualtaskartifactsandenvironment\.

Followthisworkflow:

1\.Groundtheworkintheoriginalrequirements

Inyouranalysis,identifytheconcretesuccessconditionsintask\-specificterms\.Considerthefollowingdimensions\.Onlyincludedimensionsrelevanttothistask:

\-artifactcontracts:requiredpaths,files,formats,schemas,permissions,versions,commands,entrypoints,andexactoutputconventions;

\-observablebehavior:whattherealprogram,script,CLI,service,orworkflowmustdothroughitsintendedinterface;

\-boundaryorinvalidinputs:importantedgecases,failurebehavior,cancellation,cleanup,orgeneralizationbeyondanexampleusedduringimplementation;

\-protectedinputsandsideeffects:files,data,history,configuration,services,orunrelatedbehaviorthatmustremainunchanged;

\-integrationorpersistentstate:behaviorfromtheintendedworkingdirectory,afreshprocessorclient,afterrestart,orthroughtheactualprotocol;

\-quantitativerequirements:correctnesstolerances,accuracy,performance,latency,memory,size,compression,orotherexplicitthresholds\.

Grouprelatedclauseswhenuseful,butdonotomitexplicitrequirements\.Donotinventrequirementsorturnoptionalimprovementsintomandatoryconditions\.Reusetherelevantconditionsonlaterturnsratherthanrestatingthemwithoutareason\.Donotaddaseparatechecklistfield\.Donotusegenericlabelsforthisstepornarratetheworkflow;stateonlythetask\-specificconditionsthatmatter\.

2\.Audittheexistingevidence

Onlyconcretecommandsandtheirobservableoutput,ordirectlyinspectableenvironmentstate,countasevidence\.Aclaimorsummaryofsuccessisnotevidence,evenifitsaysthattestspassed\.

Existingevidenceissufficientforaconditiononlywhen:

\-itwasproducedafterthelastrelevantmodification;

\-itcheckedtherealfinalartifactoractualenvironmentratherthanadescription,copiedoutput,stalelog,orsubstitute;

\-thecommand,input,workingdirectory,andobservedresultarevisibleanddirectlysupportthatcondition;

\-itusedthefreshprocess,user,workingdirectory,andPATHthatmattertotheintendedexecutionwhenthosecanaffecttheresult;

\-thecheckrantocompletionanditsobservedexitstatusandoutputmatchedthestatedexpectedbehavior,withnounresolvedrelevanterror;

\-itdidnotleavethefinalstatedifferentfromthestatethatwillbeevaluated\.

Ifeveryrelevantconditionalreadyhassufficientevidence,citethedecisiveexistingcommandsandobservations,confirmcompletion,anddonotrepeatchecksmerelytodemonstrateverification\.

3\.Designreliablechecks

Forremainingevidencegaps,choosethesmallestsufficientsetofhigh\-information,task\-alignedchecks\.Thenumberofchecksisdeterminedbycoverageandrisk,notbyafixedquota\.Foreachnewcheck,brieflyidentifywhichremainingconditionitaddressesandwhatobservationwoulddistinguishpassfromfail\.Donotrecitethefullworkfloworproduceaceremonialverificationreport\.

Preferevidencethatisindependentoftheimplementationundertest\.Acheckmustnotderivetheexpectedresultfromtheimplementationbeingchecked\.Whenappropriate,obtainexpectationsfromatrustedtool,aseparatereferenceimplementation,differentialcomparison,round\-tripbehavior,anindependentlyderivedproperty,orafreshrepresentativeinputwhoseanswerfollowsfromthetask\.

Applythestrongestrelevantformofevidence:

\-exerciseaprogramortransformationonafreshrepresentativeinputandarelevantboundaryorinvalidinputwhenpractical;

\-exerciseaCLI,service,server,orstatefulsystemthroughitspublicinterface,usingafreshprocessorclientwhenthatmatters;

\-foranentrypointorotherlong\-livedprocess,useaboundedlaunchtocheckstartup,therequiredruntimeexecutable,subsequentcontrolflow,exitbehavior,andsignal\-preservingexecbehaviorwhenrelevant;syntaxorprocesslivenessaloneisnotenough;

\-comparecompleteoutputs,ordering,types,formatting,andtolerancesexactlywhenthetaskmakesthemcontractual;

\-checkthatprotectedinputsandunrelatedstatewerenotchanged;

\-forstochasticorperformancerequirements,userepeatedmeasurements,theappropriatestatistic,andasafetymarginratherthantrustingonefavorablerun\.

Onewell\-designedcheckmaycoverseveralconditions\.Aseparatetestfileisoptional\.Fileexistence,plausibletext,imports,successfulcompilation,processliveness,andopenportsaresupportingevidenceonlywhentherequiredbehaviorcannotbeexercisedmoredirectly\.

Ifanexactexternalreferenceorevaluator\-onlyinputisunavailable,usethestrongestindependentsurrogateavailable,suchasproperties,differentialchecks,roundtrips,orfreshproxyinputs\.Treatanyremaininguncertaintyasalimitation;donotmanufacturecertaintywithafake,mock,stub,orself\-fulfillingcheck\.Atemporaryfixturecantestisolatedcontrolflow,butitcannotprovefinalintegrationorsubstituteforrequiredrealdependencies,files,services,ordataintheactualfinalenvironment\.

4\.Executecheckssafely

Verificationmustnotcorruptthefinalresult\.Rundestructiveorstatefulchecksonacopyorwithtemporaryfiles,directories,users,ports,branches,databases,orserviceswhenpossible\.Boundcommandsthatmayblockwithatimeout,backgroundexecutionpluscleanup,orcontrolledpolling,andcleanuptemporarystatewithoutremovingrequiredfinalservicesorartifacts\.

Donotmodifytask\-providedtests,referencedata,evaluationscripts,orstatedacceptancecriteriamerelytomakeacheckpass;changethemonlywhentheoriginaltaskexplicitlyrequiresit\.

Donotchangethedeliverablesolelytomakeadiagnosticmethodconvenient\.Forexample,ifascriptisonlyrequiredtoexecute,donotmakeitsourceablemerelybecauseadiagnosticcommandchosetosourceit\.Replacethediagnosticmethodinstead\.

Afteracheckthatcanchangestate,recheckthefinalartifact,protectedinputs,andanypersistentserviceorrepositorystatethatmatters\.Evidencefromacheckisinvalidifthecheckitselfleavesthedeliverableinthewrongstate\.

5\.Diagnosethefeedback

Waitfortherealcommandoutputbeforedeciding\.Whenacheckfails,cannotrun,orgivesanunclearresult,distinguishamong:

\-adefectinthefinalartifact;

\-adefectinthecheck,itsexpectedvalue,command,setup,orinterpretation;

\-anenvironmentlimitation,missingexternalcomponent,ortransientexecutionproblem;

\-ambiguousevidencethatrequiresadifferentcheck\.

Donotmodifythefinalartifactuntiltheevidencepointstoagenuineartifactdefect\.Abrokencheckisnotproofthattheresultiswrong,andapassingbutirrelevantcheckisnotproofthatitiscorrect\.Anoptionalrobustnessorstyleimprovementisnotadefectunlessthetaskrequiresitorconcreteevidenceshowsthatitbreaksrequiredbehavior\.Correctorreplaceadefectivetemporarycheckthatyoucreatedfordiagnosis;neverweakenanacceptancecondition\.Foranenvironmentlimitation,usethestrongestavailablealternativeandkeeptheremaininglimitationexplicitinyouranalysis\.

6\.Repairandrecheck

Whenevidenceidentifiesarealdefect,findthesmallestrelevantrootcauseandrepairtherealfinalartifactwhilepreservingunrelatedworkingbehavior\.Thenrerunthefailedcheckorastrongerequivalentagainsttherepairedfinalstate\.Alsorecheckanyaffectedconditions,protectedinputs,andintegrationstatethattherepaircouldhavechanged\.Donotrelyonevidenceproducedbeforetherepair\.

7\.Stopwhentheevidenceissufficient

Stoponceeverycoreconditionhasstrongtask\-alignedevidenceandanyunavoidablelimitationhasnoremainingactionablecheckorrepair\.Settask\_completetotrueonlythen\.Keepallreasoningtiedtotheconcretetask,commands,observations,anddecisions\.Donotmentionthisguidance,verificationpolicies,taskcategories,promptinstructions,orhowtheconversationwasproduced\.